From nobody Sat Feb 7 17:55:59 2026 Received: from mx0b-00069f02.pphosted.com (mx0b-00069f02.pphosted.com [205.220.177.32]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E203430E0D3; Mon, 3 Nov 2025 12:34:06 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=205.220.177.32 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1762173249; cv=fail; b=oFt81m2zLrZ8XhN+AgdesBcp3Sf6UXGzyNbgysCptOz4rOLS8DnCcl9R/SZBuSRaoPyHBuiHTl+gDBtBFBQ6C50VXpXOWee9cQozNHjNTwtFXKCEyAmwmzPD9HkGJAJcqLp+x0DSmgt8YcARlC21GGafXVHWZDby1rKNNLHcpW8= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1762173249; c=relaxed/simple; bh=6P80hvkhcRb9iNjVgx1Ux8MHyYX/xlGjFoXeLkgriNg=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: Content-Type:MIME-Version; b=UVmFUfMXBsbEnvuS1v2Zri521bnhzlnk7yEbgFrKqJy6d7eWSH+xlQHuA3wRC7OJ9Wz6WCwI5CkPG2cxlMayb807ckqDG/amku+gI68/CG0xKf/xA173gaAe2asy726uXAYt5mg1yJnrSpn5vDcFyPTSPB+88jrAIKBo1oQHe5E= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=oracle.com; spf=pass smtp.mailfrom=oracle.com; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b=K1lN50S9; dkim=pass (1024-bit key) header.d=oracle.onmicrosoft.com header.i=@oracle.onmicrosoft.com header.b=IXFm7i4z; arc=fail smtp.client-ip=205.220.177.32 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=oracle.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=oracle.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b="K1lN50S9"; dkim=pass (1024-bit key) header.d=oracle.onmicrosoft.com header.i=@oracle.onmicrosoft.com header.b="IXFm7i4z" Received: from pps.filterd (m0333520.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 5A3CEcsH017255; Mon, 3 Nov 2025 12:32:16 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=cc :content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s= corp-2025-04-25; bh=f03NiZg1DqFruMm084IJOTaL3bBz4ruFyVDEqvZTDPU=; b= K1lN50S9hslWNPaKlI3a+RNMJzRzzfSpu79tVFyIxFiK0xQrSalRArX16Dktz6rw SMu13pf9vTs3kttmn+WSS1VJkFx3QPv3e/K+D9DtQ8m+O2E81TS7lDHhkxUc3S3G 7q9qpWU110yj6dAaTwfqLFFyLwPTxyxzJ+aRGApYvp6pWvgPDY1acRREj2BNpdNr aLya/9UTOX1mylHmhB1saQ+hGAfKd3jApIWFn+6j2ryzvG7fj6zycjpaPTldFSe8 xu99NfmvaPJBArT4T8wRHVYrY3i6hKFWSFUTRa1rUXzephTs30rfpqjpNSHnw8cT FH3/7odAh0KDQq1KVwLbSg== Received: from iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta02.appoci.oracle.com [147.154.18.20]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 4a6v7kg0wk-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 03 Nov 2025 12:32:16 +0000 (GMT) Received: from pps.filterd (iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (8.18.1.2/8.18.1.2) with ESMTP id 5A3BKtua010909; Mon, 3 Nov 2025 12:32:15 GMT Received: from cy3pr05cu001.outbound.protection.outlook.com (mail-westcentralusazon11013025.outbound.protection.outlook.com [40.93.201.25]) by iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTPS id 4a58nbkss3-2 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 03 Nov 2025 12:32:15 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=SRN8yg6KMEUKKF+zl1tVqvH6MaLHe9hgVwT1G47e2vDelG6CzywYEju0ofGNQ9apJZR1yEQU/u1Jt6iYSMf0PNY2LdJ2UfutiIqQWhit/riLYkYP5ons4vX6ZPbRq/bf7udQCRg93KfB2T0ccFrjXIgG9BqtBWtpAlxdZ07E1j99y1l3kdiaPOeIkWlyt7c3bG8BBOdFRq7Y28M8IFpg+HwMpBmb3pU7kh7lmzV7qTrBfSw6A4Tkhi4L2T5MFubxyk1cSiX/1roOOmS9eg9RNmYN06bA4VARVOv/1aYvhIyCPuQqrssTccR7cQSqby77MfUBB/qhiL8Yx2zyplvWXg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=f03NiZg1DqFruMm084IJOTaL3bBz4ruFyVDEqvZTDPU=; b=AfrH5w3aQeaM4XzPxmYigl0pAJwIMgWgQPgGH+nSZ6Zk4G5VoI79lPbm84pvSCvKeC1SkomQkArX7a72KfeqcKqhkcohxTCU0hOeSPrKHQIdxDKk5jZlW8g+Gtkp2XT6SWZJUNq4eSwIVeLTzqpP3htNTzaREmLxGiUtqTdyjOyTrmS4VST/iQkE+JcRAk35p61cu26GDpaLIjykeiDbFxX2gTL8VyDY/5qb9S+RzxKxNVDiGcIJxf160mkSHUSf7drrRlCwwrIJXA86OMC8gXuURGqoZPMsPllm73Qo1VuuhMJK+fk2p8WMq8pClPlyHgwuopKo4DMcYteWQJx4nw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=f03NiZg1DqFruMm084IJOTaL3bBz4ruFyVDEqvZTDPU=; b=IXFm7i4zwqmRPzYHLARKXY6MhaSmCJhCsXMvGbyzwq9SjnlKWsf3uRCpyVkh5PpEa5dYXCMl6+uamVDaw4lbRq8dMkTNUqecgetwGY2tW0mzQ13WDDBtJOqk2aoNEx88wLqoxIbZRQ1qeHvTBGaUYYr0wIr1gszxlzMLMjy4p1c= Received: from DM4PR10MB8218.namprd10.prod.outlook.com (2603:10b6:8:1cc::16) by DS4PR10MB997575.namprd10.prod.outlook.com (2603:10b6:8:31e::5) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9275.16; Mon, 3 Nov 2025 12:32:12 +0000 Received: from DM4PR10MB8218.namprd10.prod.outlook.com ([fe80::2650:55cf:2816:5f2]) by DM4PR10MB8218.namprd10.prod.outlook.com ([fe80::2650:55cf:2816:5f2%2]) with mapi id 15.20.9253.018; Mon, 3 Nov 2025 12:32:12 +0000 From: Lorenzo Stoakes To: Andrew Morton Cc: Christian Borntraeger , Janosch Frank , Claudio Imbrenda , David Hildenbrand , Alexander Gordeev , Gerald Schaefer , Heiko Carstens , Vasily Gorbik , Sven Schnelle , Peter Xu , Alexander Viro , Christian Brauner , Jan Kara , Arnd Bergmann , Zi Yan , Baolin Wang , "Liam R . Howlett" , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Lance Yang , Muchun Song , Oscar Salvador , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Matthew Brost , Joshua Hahn , Rakie Kim , Byungchul Park , Gregory Price , Ying Huang , Alistair Popple , Axel Rasmussen , Yuanchu Xie , Wei Xu , Kemeng Shi , Kairui Song , Nhat Pham , Baoquan He , Chris Li , SeongJae Park , Matthew Wilcox , Jason Gunthorpe , Leon Romanovsky , Xu Xin , Chengming Zhou , Jann Horn , Miaohe Lin , Naoya Horiguchi , Pedro Falcato , Pasha Tatashin , Rik van Riel , Harry Yoo , Hugh Dickins , linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-s390@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, linux-arch@vger.kernel.org, damon@lists.linux.dev Subject: [PATCH 01/16] mm: correctly handle UFFD PTE markers Date: Mon, 3 Nov 2025 12:31:42 +0000 Message-ID: <6918f70e17e23341d2425328bb991e2e094f7a7e.1762171281.git.lorenzo.stoakes@oracle.com> X-Mailer: git-send-email 2.51.0 In-Reply-To: References: Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: LO4P123CA0594.GBRP123.PROD.OUTLOOK.COM (2603:10a6:600:295::23) To DM4PR10MB8218.namprd10.prod.outlook.com (2603:10b6:8:1cc::16) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DM4PR10MB8218:EE_|DS4PR10MB997575:EE_ X-MS-Office365-Filtering-Correlation-Id: 693c8eb7-6679-4361-6fa0-08de1ad50378 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|376014|7416014|366016|1800799024; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?agnsn2cISZMGSrC/8JBrH8aWq3A2QEaH3jDvf11/eo747CWenprAnm6uDIE1?= =?us-ascii?Q?Zq1nRshUrJMdUMWmJg2huR0W4WSGi+sjG6Tdi0LkDIq4/76OxiZwTPz8gULC?= =?us-ascii?Q?8aWv+IZqkKidE82vYdwk4MTmFFTknGCmIKmB+uO84/PUBd7CE2INbeVzr7iT?= =?us-ascii?Q?zwHTiWe4F6ns/JvUV9Knf3dZ2dEvD4zwyiF9x6/l8yptBolL3fOLZB9B4+DD?= =?us-ascii?Q?egckD5RjozD9DDMM7vOV+4XutqtMg83AvXTt3sa2mYkKvS7ov3WDz4qNYvSe?= =?us-ascii?Q?gx3WYJhd8FDAgJbiQTiYxjdxNwSpVno/pDXGHbpG0KR6sbK/gftQygiRvp/O?= =?us-ascii?Q?399dTLKFoFHlAAOsa7SSGVM+1YWmtbg3sLIkAb0/87ZyBHrvPTf6GTeEN0lm?= =?us-ascii?Q?oP8K7Wh+Qfihy7geWze1haApfV2GQtCpcSfN0bgt2Ch2UhzbNZqR9BUzsch4?= =?us-ascii?Q?TagW/BwScL3EsZhd5QMTyi/HfFUw5UOfueNRIgB+sR+CEJ255IquOg2Kz+AV?= =?us-ascii?Q?DG4V8zLeMp3t15MGBVE4rRYUdr3PbCpmwyOgUvBrfV68I3MGGJINZlxlvFtN?= =?us-ascii?Q?Dth1UbZ6VtFYYDKMYqkhsHzElaUnKKVK96hZlw08IJNZ7c2M0Dbv8eTy00vH?= =?us-ascii?Q?iaAD0GO2yWxuGAvXzgKTKolZnCqo/xU02LYGHapfHgpUhgRW09Qjucj7I0z7?= =?us-ascii?Q?Lf2AUQSDdmioNOeyGOtex93VYL4VU+6WO3Piv+hYFBIuG5x4wThTxNayfIya?= =?us-ascii?Q?ZUV7Tq/tRfjqVCW8NiY1+SmM9jBdonYYzZ+/YMWs3Vuj5RyJlYCqhwkUAWyW?= =?us-ascii?Q?xDoozAzvfhpBojAupnNH+Da+EcRC6tnGdMVST3hKtd4R/Yxqpkz7YV6SApkI?= =?us-ascii?Q?gPkVviVROKRaMh5HyrBv9KflRXi4RfxRkh4+ToxMyW8PNweD/Z3JyueweAv6?= =?us-ascii?Q?neSbQHhSgGVK65NIukBVp4WoDnmFwuSA8YwFVcoiLklBgScFIWRhqLESnSJ6?= =?us-ascii?Q?SIT8Mr1GTssQF3DM/PCxub7w1lZ0FPAmAWtMB2k9BY778ra6SsPLdJpX0y1B?= =?us-ascii?Q?eenvKphOCiQQEWY0cX2wIfOpHHMB+u9+argq+lmzHVhzBdn99mX1nznrj8Vq?= =?us-ascii?Q?k49JwuH48Wpt4iYUh2jpgVTMhvPTe02KAj4JfW0+qifJaCVt0HD5VyP1RwDM?= =?us-ascii?Q?/iTSnB/fgArJoFpjPHVLgy/bNXQ70mZAMyBtzt3YoBShnz7IHuVQPqh8yMH8?= =?us-ascii?Q?vVQs9Nx1JSMQ9m6nX5mjZ8COpPkRJTrZRSGAWY3c8XjTqJ+cAh80/4kH6Deo?= =?us-ascii?Q?PrIiq3kFBvXwqXUKAcMlw5C17lp6TijYWzkd60kxkzg5kWnocZJ1whH/BcPy?= =?us-ascii?Q?Bommdj91bsX8S7JiVgnaKeTVixejptdMHBVwOOuA7JW22rQqtFm6JvSzue3U?= =?us-ascii?Q?v1h/mgwTrhCn3aitFb6l1Cl3eWc3eRlC?= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DM4PR10MB8218.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(376014)(7416014)(366016)(1800799024);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?le4Dv8Rt36UlzZagHbrwNQwawh/zc9tVaTkFAz1IfEn9Y3P03MUfixipTKZt?= =?us-ascii?Q?EnQcIoDRY/Pq7hOmwwKzCN8BCNXQiU4rPY6uks8cFVYT5k0OoBW1mdsg9GDX?= =?us-ascii?Q?3ScTzZMnAnsQPOS9l0oJB4mUm1WGfsEJ9clSerOTYpm+iGRxKVh5/Yk+CBFD?= =?us-ascii?Q?g6G6jmx8zXVLoLAUbwkkTpJ27ujC7ws8u0pMs6ZfCPE57KLi+k2J5xLWgFQJ?= =?us-ascii?Q?9BAuUZgUsgK1fwvIPTG/UcXxs9YUILEzllsLtcBjC95C3H3qGHN2MzlwuYHz?= =?us-ascii?Q?XJqTWuDy0ERcLf0cif56EPj9Z0HQGC5aHKiaIoptSYc4bZ7qDao2zZUkh7X5?= =?us-ascii?Q?pl9BNfH7TdXcpBepYHNmRuvlHOMdhqgf5C7n0JMM0fuCR49wqOtqjpBKwKR+?= =?us-ascii?Q?5serVQURR/EUCYBHovanp9SServ5g4qI5Y5QK0Ho6pKT5881/Wn+hztFwgED?= =?us-ascii?Q?Dm7/xbbyd9KMzUob5LZQlgqwn97Hn1qwVBRDYg1Y06Q/NaHMV2s2xXJEzaUv?= =?us-ascii?Q?qLdAg00+sIeb2fd4EXkNoI6R8nA74V8mzqejcVikSbw1BSaK2LsCmHM3bD12?= =?us-ascii?Q?Tar6T9qpHe+DC/mzPrmYOQ5RelSaTGliofQOWAp/kbavoFDV8XvsOFYTNrUQ?= =?us-ascii?Q?ZJvo7OkoP1PSOq3pTjcvZEX/0OvCAcAzaO+umDriEngiyluuSqbygmo97SIg?= =?us-ascii?Q?lH7GIeb2WWgDt4+6O/wcXlC8xNhL+20980ZhLfwdkv8uUtHG2NUAk/YvcwQ9?= =?us-ascii?Q?ZEEXTTkjvq65EpEMSK8hg7F/yX1b/FqXw3yy2N6GEvvZ2iqypDG/NMU1M4fk?= =?us-ascii?Q?Rf3CL9kqeEX8NMj9MSGICOHQpK75/jvbdumI2Jg82TrH/Xhf8Che5hE6lk7i?= =?us-ascii?Q?TVb2EbleSVJ5mfMaJI11wy4FreSks8ci9MT0dwqV0AsKhLyjVNc5k3ZPzCBu?= =?us-ascii?Q?gJWIbY+pw28G4I9rw7wIdm7DvJbmbtvxdq+CaVIfaTX/vChe4VTSQzw3+Fnp?= =?us-ascii?Q?zKDkFTFg96KutdlvEUXl0H7vrosjASlJljreaeRVevSUmhLLH9xonQxkB6yH?= =?us-ascii?Q?j5UmMS8HJpU0HmQWARw+qwLU9fPbgjTQ1OyORQIlkJPuKTL0DSMnJzowzv0J?= =?us-ascii?Q?CIWHar15WnxT2cB/QNVzRanbgdjl3NjP9JQtvIZ/pTW/h5DtoafvC3SaNmKe?= =?us-ascii?Q?5TSaVq/bRrwMWNOe7b1dXx8sqtWXH76hJ0jGMY8BwPzsabHpE4SEj7bDO0ru?= =?us-ascii?Q?O6DPmuK+fuxoFHo+2HZifxz4p4WdtOQ/2y90h61sjDwhD0AFtxzjSPkOTjwq?= =?us-ascii?Q?WCVgfRiOT/DuhCWhvLhWNiKYCWk0OPPrkjF4eIup0FbJ+88nk5TDsGn3HQF3?= =?us-ascii?Q?5eHkJhpAwgXKbuGNSdPVuzhudIoYAiyQaC4+zKaleXgKoohZyB8jyAqeSGPM?= =?us-ascii?Q?/EVDu5Z+EJRKjgG8bJtELZC+7/csWQQYxwn7kQUr0iZbn/NlByjBTqW6DxC3?= =?us-ascii?Q?XnzxrTYyCUgTO8d4qjTusDdqKDsXRxTmzq8iILHWKNgRNFa3SSqj7lTSVJ5j?= =?us-ascii?Q?LARvJlu+BviZDzuM7BnYhyb4EiC1lCc4ikwD0ch+BPfnwUF1M7QS+htvy+Bi?= =?us-ascii?Q?7g=3D=3D?= X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: r11qO5Qi7oup/mUMPyrpsuJw5zo5x7vHFruHSRz7yqIAi3Iv9zYnhc+85fpqIQNk5FBYg2Tp5OJW38rp+jqxx0mDsLY8+s6VkNxJy3bj+oT64ixmCLLAoMdYCcPtnG5yoS33Kym8DuSK9X61lE1hsX6YIFwwh5Plua29YJxjJ4zZJsnyVsSopWvPju6BSqMxbqb992Ia8dscd+G7TTvR9rZrMQLnVc58DgOZGUZFKag3UEMEKVbemLGifN90r/lEfQpykPB8OQp2GcRL/MifgrToUfSy3t3PYYlPUQuwMY2EiUknJwW9EmswW/QVzuu6hi8rT4+0cXliDcCYEzu4WjgHBCTj3jjP21VsOm1TftYWNMtUAXLmPIHhqi+FRncO9cIUyLzUzxibH/FnmcqDdwcWYVWqaIImQe/Jae+cRaSw1ws01ZIvCAcAzl+GgT7t4aEH6heElPtBx/hjxWs8m4QNfUdZP4tV38EPCTvFOTi6GnjM4+NyhDQfIPY6452C+KH8vR5R1J4niFrF7duXI0YLeWCOUQ6PFvqjuRcscI0ktWkLrB7Ic6rf+pI8RQx8QrO5xDCRN5k6vMcnUqEPE9h40IJFoQFPWL/4y2JWgZc= X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: 693c8eb7-6679-4361-6fa0-08de1ad50378 X-MS-Exchange-CrossTenant-AuthSource: DM4PR10MB8218.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 03 Nov 2025 12:32:12.3421 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 7JRyxnrNacs+PD5TQi1b4Mh7Uut8rxvuVhacvRO1Ob1VBtrPonoT4SKhPR3la2n2NrkQDmU+wzCiNBHoVIXLkkI8euQg8UuQtKG8wMCKeUE= X-MS-Exchange-Transport-CrossTenantHeadersStamped: DS4PR10MB997575 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1121,Hydra:6.1.9,FMLib:17.12.100.49 definitions=2025-11-03_01,2025-11-03_01,2025-10-01_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 phishscore=0 mlxscore=0 bulkscore=0 malwarescore=0 adultscore=0 spamscore=0 mlxlogscore=999 suspectscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2510240000 definitions=main-2511030113 X-Proofpoint-ORIG-GUID: WDxPEJ_WXtWnWg5sE5Yt3pax4_yy1pUt X-Proofpoint-Spam-Details-Enc: AW1haW4tMjUxMTAzMDExMSBTYWx0ZWRfX0AqVvVsbrj3S IIBDLSiHFYJ5YOBW/UhS8Cl8fwgUptrqiGPakE/EiqzOrEQEPX4NaD8PsxXZgVo5Xm+RZlipq8d oHz4sjHkbN9zEybLVLoroW8TaCTCnLxcoNRMC65SyzBbCWU+e49Bs1G84/qjUIMWVAqKwx9ckTj smGhiOF5PMlOceQLF+LBJKszYOuVriSsQQH+u0cjdEqhIHNbvbjl5rREI+MSOKpc3QiTMhWrk5Q nnY1OHW2cDdmIpNvlaI7WV84BjJjnnxS89KmtYmTUNqn/EZv7UcfLs8HqiVLbwQERM7aUKUELBi ToFPAO/7h/cOySdN9V2ZCaTQaDGo6SSs6eQ8uBlXT2zz4qO6WQWJ2bdr7zosJMTZSOEXJuzKtxw cdeFi0k1e5bLUgeCDOnfpnOh8U5imFLUTI/iy2BkHty16jaktVs= X-Authority-Analysis: v=2.4 cv=Fbs6BZ+6 c=1 sm=1 tr=0 ts=6908a0d0 b=1 cx=c_pps a=e1sVV491RgrpLwSTMOnk8w==:117 a=e1sVV491RgrpLwSTMOnk8w==:17 a=6eWqkTHjU83fiwn7nKZWdM+Sl24=:19 a=z/mQ4Ysz8XfWz/Q5cLBRGdckG28=:19 a=lCpzRmAYbLLaTzLvsPZ7Mbvzbb8=:19 a=xqWC_Br6kY4A:10 a=6UeiqGixMTsA:10 a=GoEa3M9JfhUA:10 a=VkNPw1HP01LnGYTKEx00:22 a=yPCof4ZbAAAA:8 a=K7pD0KWnulOJboH6qJQA:9 cc=ntf awl=host:13657 X-Proofpoint-GUID: WDxPEJ_WXtWnWg5sE5Yt3pax4_yy1pUt Content-Type: text/plain; charset="utf-8" PTE markers were previously only concerned with UFFD-specific logic - that is, PTE entries with the UFFD WP marker set or those marked via UFFDIO_POISON. However since the introduction of guard markers in commit 7c53dfbdb024 ("mm: add PTE_MARKER_GUARD PTE marker"), this has no longer been the case. Issues have been avoided as guard regions are not permitted in conjunction with UFFD, but it still leaves very confusing logic in place, most notably the misleading and poorly named pte_none_mostly() and huge_pte_none_mostly(). This predicate returns true for PTE entries that ought to be treated as none, but only in certain circumstances, and on the assumption we are dealing with H/W poison markers or UFFD WP markers. This patch removes these functions and makes each invocation of these functions instead explicitly check what it needs to check. As part of this effort it introduces is_uffd_pte_marker() to explicitly determine if a marker in fact is used as part of UFFD or not. In the HMM logic we note that the only time we would need to check for a fault is in the case of a UFFD WP marker, otherwise we simply encounter a fault error (VM_FAULT_HWPOISON for H/W poisoned marker, VM_FAULT_SIGSEGV for a guard marker), so only check for the UFFD WP case. While we're here we also refactor code to make it easier to understand. Signed-off-by: Lorenzo Stoakes Reviewed-by: Vlastimil Babka --- fs/userfaultfd.c | 83 +++++++++++++++++++---------------- include/asm-generic/hugetlb.h | 8 ---- include/linux/swapops.h | 18 -------- include/linux/userfaultfd_k.h | 21 +++++++++ mm/hmm.c | 2 +- mm/hugetlb.c | 47 ++++++++++---------- mm/mincore.c | 17 +++++-- mm/userfaultfd.c | 27 +++++++----- 8 files changed, 123 insertions(+), 100 deletions(-) diff --git a/fs/userfaultfd.c b/fs/userfaultfd.c index 54c6cc7fe9c6..04c66b5001d5 100644 --- a/fs/userfaultfd.c +++ b/fs/userfaultfd.c @@ -233,40 +233,46 @@ static inline bool userfaultfd_huge_must_wait(struct = userfaultfd_ctx *ctx, { struct vm_area_struct *vma =3D vmf->vma; pte_t *ptep, pte; - bool ret =3D true; =20 assert_fault_locked(vmf); =20 ptep =3D hugetlb_walk(vma, vmf->address, vma_mmu_pagesize(vma)); if (!ptep) - goto out; + return true; =20 - ret =3D false; pte =3D huge_ptep_get(vma->vm_mm, vmf->address, ptep); =20 /* * Lockless access: we're in a wait_event so it's ok if it - * changes under us. PTE markers should be handled the same as none - * ptes here. + * changes under us. */ - if (huge_pte_none_mostly(pte)) - ret =3D true; + + /* If missing entry, wait for handler. */ + if (huge_pte_none(pte)) + return true; + /* UFFD PTE markers require handling. */ + if (is_uffd_pte_marker(pte)) + return true; + /* If VMA has UFFD WP faults enabled and WP fault, wait for handler. */ if (!huge_pte_write(pte) && (reason & VM_UFFD_WP)) - ret =3D true; -out: - return ret; + return true; + + /* Otherwise, if entry isn't present, let fault handler deal with it. */ + return false; } #else static inline bool userfaultfd_huge_must_wait(struct userfaultfd_ctx *ctx, struct vm_fault *vmf, unsigned long reason) { - return false; /* should never get here */ + /* Should never get here. */ + VM_WARN_ON_ONCE(1); + return false; } #endif /* CONFIG_HUGETLB_PAGE */ =20 /* - * Verify the pagetables are still not ok after having reigstered into + * Verify the pagetables are still not ok after having registered into * the fault_pending_wqh to avoid userland having to UFFDIO_WAKE any * userfault that has already been resolved, if userfaultfd_read_iter and * UFFDIO_COPY|ZEROPAGE are being run simultaneously on two different @@ -284,53 +290,55 @@ static inline bool userfaultfd_must_wait(struct userf= aultfd_ctx *ctx, pmd_t *pmd, _pmd; pte_t *pte; pte_t ptent; - bool ret =3D true; + bool ret; =20 assert_fault_locked(vmf); =20 pgd =3D pgd_offset(mm, address); if (!pgd_present(*pgd)) - goto out; + return true; p4d =3D p4d_offset(pgd, address); if (!p4d_present(*p4d)) - goto out; + return true; pud =3D pud_offset(p4d, address); if (!pud_present(*pud)) - goto out; + return true; pmd =3D pmd_offset(pud, address); again: _pmd =3D pmdp_get_lockless(pmd); if (pmd_none(_pmd)) - goto out; + return true; =20 - ret =3D false; if (!pmd_present(_pmd)) - goto out; + return false; =20 - if (pmd_trans_huge(_pmd)) { - if (!pmd_write(_pmd) && (reason & VM_UFFD_WP)) - ret =3D true; - goto out; - } + if (pmd_trans_huge(_pmd)) + return !pmd_write(_pmd) && (reason & VM_UFFD_WP); =20 pte =3D pte_offset_map(pmd, address); - if (!pte) { - ret =3D true; + if (!pte) goto again; - } + /* * Lockless access: we're in a wait_event so it's ok if it - * changes under us. PTE markers should be handled the same as none - * ptes here. + * changes under us. */ ptent =3D ptep_get(pte); - if (pte_none_mostly(ptent)) - ret =3D true; + + ret =3D true; + /* If missing entry, wait for handler. */ + if (pte_none(ptent)) + goto out; + /* UFFD PTE markers require handling. */ + if (is_uffd_pte_marker(ptent)) + goto out; + /* If VMA has UFFD WP faults enabled and WP fault, wait for handler. */ if (!pte_write(ptent) && (reason & VM_UFFD_WP)) - ret =3D true; - pte_unmap(pte); + goto out; =20 + ret =3D false; out: + pte_unmap(pte); return ret; } =20 @@ -490,12 +498,13 @@ vm_fault_t handle_userfault(struct vm_fault *vmf, uns= igned long reason) set_current_state(blocking_state); spin_unlock_irq(&ctx->fault_pending_wqh.lock); =20 - if (!is_vm_hugetlb_page(vma)) - must_wait =3D userfaultfd_must_wait(ctx, vmf, reason); - else + if (is_vm_hugetlb_page(vma)) { must_wait =3D userfaultfd_huge_must_wait(ctx, vmf, reason); - if (is_vm_hugetlb_page(vma)) hugetlb_vma_unlock_read(vma); + } else { + must_wait =3D userfaultfd_must_wait(ctx, vmf, reason); + } + release_fault_lock(vmf); =20 if (likely(must_wait && !READ_ONCE(ctx->released))) { diff --git a/include/asm-generic/hugetlb.h b/include/asm-generic/hugetlb.h index dcb8727f2b82..e1a2e1b7c8e7 100644 --- a/include/asm-generic/hugetlb.h +++ b/include/asm-generic/hugetlb.h @@ -97,14 +97,6 @@ static inline int huge_pte_none(pte_t pte) } #endif =20 -/* Please refer to comments above pte_none_mostly() for the usage */ -#ifndef __HAVE_ARCH_HUGE_PTE_NONE_MOSTLY -static inline int huge_pte_none_mostly(pte_t pte) -{ - return huge_pte_none(pte) || is_pte_marker(pte); -} -#endif - #ifndef __HAVE_ARCH_HUGE_PTEP_SET_WRPROTECT static inline void huge_ptep_set_wrprotect(struct mm_struct *mm, unsigned long addr, pte_t *ptep) diff --git a/include/linux/swapops.h b/include/linux/swapops.h index 2687928a8146..d1f665935cfc 100644 --- a/include/linux/swapops.h +++ b/include/linux/swapops.h @@ -469,24 +469,6 @@ static inline int is_guard_swp_entry(swp_entry_t entry) (pte_marker_get(entry) & PTE_MARKER_GUARD); } =20 -/* - * This is a special version to check pte_none() just to cover the case wh= en - * the pte is a pte marker. It existed because in many cases the pte mark= er - * should be seen as a none pte; it's just that we have stored some inform= ation - * onto the none pte so it becomes not-none any more. - * - * It should be used when the pte is file-backed, ram-based and backing - * userspace pages, like shmem. It is not needed upon pgtables that do not - * support pte markers at all. For example, it's not needed on anonymous - * memory, kernel-only memory (including when the system is during-boot), - * non-ram based generic file-system. It's fine to be used even there, bu= t the - * extra pte marker check will be pure overhead. - */ -static inline int pte_none_mostly(pte_t pte) -{ - return pte_none(pte) || is_pte_marker(pte); -} - static inline struct page *pfn_swap_entry_to_page(swp_entry_t entry) { struct page *p =3D pfn_to_page(swp_offset_pfn(entry)); diff --git a/include/linux/userfaultfd_k.h b/include/linux/userfaultfd_k.h index c0e716aec26a..da0b4fcc566f 100644 --- a/include/linux/userfaultfd_k.h +++ b/include/linux/userfaultfd_k.h @@ -479,4 +479,25 @@ static inline bool pte_swp_uffd_wp_any(pte_t pte) return false; } =20 + +static inline bool is_uffd_pte_marker(pte_t pte) +{ + swp_entry_t entry; + + if (pte_present(pte)) + return false; + + entry =3D pte_to_swp_entry(pte); + if (!is_pte_marker_entry(entry)) + return false; + + /* UFFD WP, poisoned swap entries are UFFD handled. */ + if (pte_marker_entry_uffd_wp(entry)) + return true; + if (is_poisoned_swp_entry(entry)) + return true; + + return false; +} + #endif /* _LINUX_USERFAULTFD_K_H */ diff --git a/mm/hmm.c b/mm/hmm.c index a56081d67ad6..43d4a91035ff 100644 --- a/mm/hmm.c +++ b/mm/hmm.c @@ -244,7 +244,7 @@ static int hmm_vma_handle_pte(struct mm_walk *walk, uns= igned long addr, uint64_t pfn_req_flags =3D *hmm_pfn; uint64_t new_pfn_flags =3D 0; =20 - if (pte_none_mostly(pte)) { + if (pte_none(pte) || pte_marker_uffd_wp(pte)) { required_fault =3D hmm_pte_need_fault(hmm_vma_walk, pfn_req_flags, 0); if (required_fault) diff --git a/mm/hugetlb.c b/mm/hugetlb.c index 1ea459723cce..01c784547d1e 100644 --- a/mm/hugetlb.c +++ b/mm/hugetlb.c @@ -6743,29 +6743,28 @@ vm_fault_t hugetlb_fault(struct mm_struct *mm, stru= ct vm_area_struct *vma, } =20 vmf.orig_pte =3D huge_ptep_get(mm, vmf.address, vmf.pte); - if (huge_pte_none_mostly(vmf.orig_pte)) { - if (is_pte_marker(vmf.orig_pte)) { - pte_marker marker =3D - pte_marker_get(pte_to_swp_entry(vmf.orig_pte)); - - if (marker & PTE_MARKER_POISONED) { - ret =3D VM_FAULT_HWPOISON_LARGE | - VM_FAULT_SET_HINDEX(hstate_index(h)); - goto out_mutex; - } else if (WARN_ON_ONCE(marker & PTE_MARKER_GUARD)) { - /* This isn't supported in hugetlb. */ - ret =3D VM_FAULT_SIGSEGV; - goto out_mutex; - } - } - + if (huge_pte_none(vmf.orig_pte)) /* - * Other PTE markers should be handled the same way as none PTE. - * * hugetlb_no_page will drop vma lock and hugetlb fault * mutex internally, which make us return immediately. */ return hugetlb_no_page(mapping, &vmf); + + if (is_pte_marker(vmf.orig_pte)) { + const pte_marker marker =3D + pte_marker_get(pte_to_swp_entry(vmf.orig_pte)); + + if (marker & PTE_MARKER_POISONED) { + ret =3D VM_FAULT_HWPOISON_LARGE | + VM_FAULT_SET_HINDEX(hstate_index(h)); + goto out_mutex; + } else if (WARN_ON_ONCE(marker & PTE_MARKER_GUARD)) { + /* This isn't supported in hugetlb. */ + ret =3D VM_FAULT_SIGSEGV; + goto out_mutex; + } + + return hugetlb_no_page(mapping, &vmf); } =20 ret =3D 0; @@ -6934,6 +6933,7 @@ int hugetlb_mfill_atomic_pte(pte_t *dst_pte, int ret =3D -ENOMEM; struct folio *folio; bool folio_in_pagecache =3D false; + pte_t dst_ptep; =20 if (uffd_flags_mode_is(flags, MFILL_ATOMIC_POISON)) { ptl =3D huge_pte_lock(h, dst_mm, dst_pte); @@ -7073,13 +7073,14 @@ int hugetlb_mfill_atomic_pte(pte_t *dst_pte, if (folio_test_hwpoison(folio)) goto out_release_unlock; =20 + ret =3D -EEXIST; + + dst_ptep =3D huge_ptep_get(dst_mm, dst_addr, dst_pte); /* - * We allow to overwrite a pte marker: consider when both MISSING|WP - * registered, we firstly wr-protect a none pte which has no page cache - * page backing it, then access the page. + * See comment about UFFD marker overwriting in + * mfill_atomic_install_pte(). */ - ret =3D -EEXIST; - if (!huge_pte_none_mostly(huge_ptep_get(dst_mm, dst_addr, dst_pte))) + if (!huge_pte_none(dst_ptep) && !is_uffd_pte_marker(dst_ptep)) goto out_release_unlock; =20 if (folio_in_pagecache) diff --git a/mm/mincore.c b/mm/mincore.c index 8ec4719370e1..151b2dbb783b 100644 --- a/mm/mincore.c +++ b/mm/mincore.c @@ -32,11 +32,22 @@ static int mincore_hugetlb(pte_t *pte, unsigned long hm= ask, unsigned long addr, spinlock_t *ptl; =20 ptl =3D huge_pte_lock(hstate_vma(walk->vma), walk->mm, pte); + /* * Hugepages under user process are always in RAM and never * swapped out, but theoretically it needs to be checked. */ - present =3D pte && !huge_pte_none_mostly(huge_ptep_get(walk->mm, addr, pt= e)); + if (!pte) { + present =3D 0; + } else { + const pte_t ptep =3D huge_ptep_get(walk->mm, addr, pte); + + if (huge_pte_none(ptep) || is_pte_marker(ptep)) + present =3D 0; + else + present =3D 1; + } + for (; addr !=3D end; vec++, addr +=3D PAGE_SIZE) *vec =3D present; walk->private =3D vec; @@ -175,8 +186,8 @@ static int mincore_pte_range(pmd_t *pmd, unsigned long = addr, unsigned long end, pte_t pte =3D ptep_get(ptep); =20 step =3D 1; - /* We need to do cache lookup too for pte markers */ - if (pte_none_mostly(pte)) + /* We need to do cache lookup too for UFFD pte markers */ + if (pte_none(pte) || is_uffd_pte_marker(pte)) __mincore_unmapped_range(addr, addr + PAGE_SIZE, vma, vec); else if (pte_present(pte)) { diff --git a/mm/userfaultfd.c b/mm/userfaultfd.c index 00122f42718c..cc4ce205bbec 100644 --- a/mm/userfaultfd.c +++ b/mm/userfaultfd.c @@ -178,6 +178,7 @@ int mfill_atomic_install_pte(pmd_t *dst_pmd, spinlock_t *ptl; struct folio *folio =3D page_folio(page); bool page_in_cache =3D folio_mapping(folio); + pte_t dst_ptep; =20 _dst_pte =3D mk_pte(page, dst_vma->vm_page_prot); _dst_pte =3D pte_mkdirty(_dst_pte); @@ -199,12 +200,15 @@ int mfill_atomic_install_pte(pmd_t *dst_pmd, } =20 ret =3D -EEXIST; + + dst_ptep =3D ptep_get(dst_pte); + /* - * We allow to overwrite a pte marker: consider when both MISSING|WP - * registered, we firstly wr-protect a none pte which has no page cache - * page backing it, then access the page. + * We are allowed to overwrite a UFFD pte marker: consider when both + * MISSING|WP registered, we firstly wr-protect a none pte which has no + * page cache page backing it, then access the page. */ - if (!pte_none_mostly(ptep_get(dst_pte))) + if (!pte_none(dst_ptep) && !is_uffd_pte_marker(dst_ptep)) goto out_unlock; =20 if (page_in_cache) { @@ -583,12 +587,15 @@ static __always_inline ssize_t mfill_atomic_hugetlb( goto out_unlock; } =20 - if (!uffd_flags_mode_is(flags, MFILL_ATOMIC_CONTINUE) && - !huge_pte_none_mostly(huge_ptep_get(dst_mm, dst_addr, dst_pte))) { - err =3D -EEXIST; - hugetlb_vma_unlock_read(dst_vma); - mutex_unlock(&hugetlb_fault_mutex_table[hash]); - goto out_unlock; + if (!uffd_flags_mode_is(flags, MFILL_ATOMIC_CONTINUE)) { + const pte_t ptep =3D huge_ptep_get(dst_mm, dst_addr, dst_pte); + + if (!huge_pte_none(ptep) && !is_uffd_pte_marker(ptep)) { + err =3D -EEXIST; + hugetlb_vma_unlock_read(dst_vma); + mutex_unlock(&hugetlb_fault_mutex_table[hash]); + goto out_unlock; + } } =20 err =3D hugetlb_mfill_atomic_pte(dst_pte, dst_vma, dst_addr, --=20 2.51.0 From nobody Sat Feb 7 17:55:59 2026 Received: from mx0b-00069f02.pphosted.com (mx0b-00069f02.pphosted.com [205.220.177.32]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 1774630CDBF; Mon, 3 Nov 2025 12:33:49 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=205.220.177.32 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1762173233; cv=fail; b=PYmVvjGCX+c9oKOvdh0SbE/RbuoqMMa1rQJmX0HPvqMeE6xfcNoGlwFZtQNpl5p5Bpxu2/4rCI3vKECXk/f2oNNsa3+x7WicE7WMwkC69ZvN3TOXIYmZeNoDzNoQPkXv6BWSU+XM0k+BE6Q+PHl3xeCEwdk3rokSlhjxWkEbW1w= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1762173233; c=relaxed/simple; bh=+1AITt9B1AH8OmblGMOUSIDpuarrQv8aDKLW/2lhCKo=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: Content-Type:MIME-Version; b=GI/MSgZZ4h2d5Djuk1LxYVpFE2jGdQJk3nnj+98HNYFmqdJh8Sa9Clwp7cnLdxDlGuZl9xi9Lxr9H8hlMavg28EmI3iAnDgmUC/MU7om97i/eMUWdLpXq8vbr2QQjFuxQu6nmHgZcLCRz0lJWKMHNsYrZ/YdbjC4nxfl+pFamB8= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=oracle.com; spf=pass smtp.mailfrom=oracle.com; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b=OqmSQevb; dkim=pass (1024-bit key) header.d=oracle.onmicrosoft.com header.i=@oracle.onmicrosoft.com header.b=zKeZohRM; arc=fail smtp.client-ip=205.220.177.32 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=oracle.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=oracle.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b="OqmSQevb"; dkim=pass (1024-bit key) header.d=oracle.onmicrosoft.com header.i=@oracle.onmicrosoft.com header.b="zKeZohRM" Received: from pps.filterd (m0246630.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 5A3CP2eG031542; Mon, 3 Nov 2025 12:32:19 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=cc :content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s= corp-2025-04-25; bh=egI0qRreT/yxUPZjzy2y6msSKFmp4kULwVBcR3j75/0=; b= OqmSQevb5hYABAs/AsxhJLSy4kr5i7yY3iu1xYnEvctE60GczN1xPC/O+qIZYrkz UdkBH6dG/W+wVfZ7+vae/7haiCY51NP9kbT1FYGwsOmwZPgDUvWCIx2xo0xQ73Ci rSgGaucuWDEct4Z1/1l+8TtGjwEQxO2lQhvCUZhlgUNFcVNuOLdMyT4BVBAAdqRt lf9C5b7gxkNonglbb3ccsv50MXH14x3tPfYD+peU53q4Qq7PLW5me4FvvsN2LmiC cCBkyhcB7LkIdWaUNvdezZBxcDFy5eczxANIjLodRSA4pfHEpqVU3r3Zhh+nO8zT Uqj7sDi80eH7T7QQ3Q12gA== Received: from iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta02.appoci.oracle.com [147.154.18.20]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 4a6vcb80f4-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 03 Nov 2025 12:32:19 +0000 (GMT) Received: from pps.filterd (iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (8.18.1.2/8.18.1.2) with ESMTP id 5A3BSiuB009599; Mon, 3 Nov 2025 12:32:18 GMT Received: from cy3pr05cu001.outbound.protection.outlook.com (mail-westcentralusazon11013032.outbound.protection.outlook.com [40.93.201.32]) by iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTPS id 4a58nbksty-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 03 Nov 2025 12:32:18 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=j6CxOjrgxBu2KsirXRZJIULdL0vXu4MuB6PRp9wxEEYPxb1ODVv3+59i8g/QBV9AS9Yf9MBL6WhBfUY2Y3amLeLM1P3GKANdewLOFULERLzf1E8UcNe/UBYpcnY5gqYWPziMbq8+qbHOiHVgSs0s+t3bxGa4Zj9VcYksF0DQmj8HHwkZod2fNuYMUhFeZZ2jAAalqCWWdNvkpe0DMyMwNnr/f3hV96xg6ySv7fmhrapxEHV/SCaJfn7a939FrUgV8DN2DxP2mLF8AxrozTd2pe1ETsLUTN/E98djCcaqASz3ccjRxlXahJIva3peUMqGrPbNOa3hEYePtEBgg/dHhQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=egI0qRreT/yxUPZjzy2y6msSKFmp4kULwVBcR3j75/0=; b=Ib7VdOQhWsrtfE8S0xDPKIjBvU7+4VRebBUj0HNL9XMpO5qw7AxXEa+o/EVFYf4bkkT33T9zev6ttXfMfEa2tz9FJ/Em3s6RpOzLa28kf2DXI5icYb/iI7xc2/AhvrJTQ3Q2qWCQLi0VZT3LS8sRpVUhTP4lzjiwk+mzl1n+SnALXEmx/Mg8QH5patryX4ji2IVF2Hei5YT/iRqLBvOmN0dlZABOElcu/isv8SO1R1NqfUjVKu2X8V78ynNhmMTpaOBFnqSxxo8AnC2iP9ZrujLcJ2E8zSckldkUIhIxhjO3q1njKo4Q/gOluESujeyKQhLWjVRwfG9m9Q0BjFUEwg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=egI0qRreT/yxUPZjzy2y6msSKFmp4kULwVBcR3j75/0=; b=zKeZohRM+ICEK3koBm3/QZz5QJmKtsJyekRZUq1sGxuc7a/unVOMg53NQKuuMKhouDtic3Z+lfQBRA4fAEoiTSQ1W2xhiTrosXgrnAbyIq8t0OhZz+SQ24kjK3qmFZ1vDkGuozKkBxnbJ+qB9qliBznghFCBTRKUo2pC/9zN3wI= Received: from DM4PR10MB8218.namprd10.prod.outlook.com (2603:10b6:8:1cc::16) by DS4PR10MB997575.namprd10.prod.outlook.com (2603:10b6:8:31e::5) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9275.16; Mon, 3 Nov 2025 12:32:14 +0000 Received: from DM4PR10MB8218.namprd10.prod.outlook.com ([fe80::2650:55cf:2816:5f2]) by DM4PR10MB8218.namprd10.prod.outlook.com ([fe80::2650:55cf:2816:5f2%2]) with mapi id 15.20.9253.018; Mon, 3 Nov 2025 12:32:14 +0000 From: Lorenzo Stoakes To: Andrew Morton Cc: Christian Borntraeger , Janosch Frank , Claudio Imbrenda , David Hildenbrand , Alexander Gordeev , Gerald Schaefer , Heiko Carstens , Vasily Gorbik , Sven Schnelle , Peter Xu , Alexander Viro , Christian Brauner , Jan Kara , Arnd Bergmann , Zi Yan , Baolin Wang , "Liam R . Howlett" , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Lance Yang , Muchun Song , Oscar Salvador , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Matthew Brost , Joshua Hahn , Rakie Kim , Byungchul Park , Gregory Price , Ying Huang , Alistair Popple , Axel Rasmussen , Yuanchu Xie , Wei Xu , Kemeng Shi , Kairui Song , Nhat Pham , Baoquan He , Chris Li , SeongJae Park , Matthew Wilcox , Jason Gunthorpe , Leon Romanovsky , Xu Xin , Chengming Zhou , Jann Horn , Miaohe Lin , Naoya Horiguchi , Pedro Falcato , Pasha Tatashin , Rik van Riel , Harry Yoo , Hugh Dickins , linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-s390@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, linux-arch@vger.kernel.org, damon@lists.linux.dev Subject: [PATCH 02/16] mm: introduce leaf entry type and use to simplify leaf entry logic Date: Mon, 3 Nov 2025 12:31:43 +0000 Message-ID: <2c75a316f1b91a502fad718de9b1bb151aafe717.1762171281.git.lorenzo.stoakes@oracle.com> X-Mailer: git-send-email 2.51.0 In-Reply-To: References: Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: LO2P265CA0145.GBRP265.PROD.OUTLOOK.COM (2603:10a6:600:9::13) To DM4PR10MB8218.namprd10.prod.outlook.com (2603:10b6:8:1cc::16) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DM4PR10MB8218:EE_|DS4PR10MB997575:EE_ X-MS-Office365-Filtering-Correlation-Id: f1081bde-c56f-483d-9d7f-08de1ad504ca X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|376014|7416014|366016|1800799024; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?Da68B1zaBnncsAX9UMZ7bws38gqT/P4MZWEQcoPzRBD50mjPnTpEHVbIOxKp?= =?us-ascii?Q?fhDrLXGwrvbWRZ2vHbFtL/A/K9kQDPYyM7NzliO/MaXiuJJWy8JJUvuQ+Ab6?= =?us-ascii?Q?wQ9PnpnUtr7itMg9n0HoQf+PVGXVlu8LVg/CYHmuKlIcEE0j34XuG21SzjTC?= =?us-ascii?Q?4tHRJWVgjv9EgrtK3PQM25KQSN9Sbm13WDYkKl/gxnWPx7QTBOJ2xfOM7qJu?= =?us-ascii?Q?0judCxHsKKlGpUeuU0d41+s9ltY6gaWeY6gv1Mpen63znlmzXZLwTNEJM6Pj?= =?us-ascii?Q?FR6qArhxurvj5cDFVFJ9vG5iiXm1HtPILGRXZmp4wpjwU+aQBeKRAzDRqlkX?= =?us-ascii?Q?QPlJUo+hM0agCc9L7tPIu5gQVHDJgerxmV8113cKk8MxcxJSyM0uY1vvSzwJ?= =?us-ascii?Q?iI3diQKQKUZYC7uKbUmYYJnfZ56+wedt/QVliEYcR7iEsU7ydptuPK1OCRPG?= =?us-ascii?Q?3+hzSwiPbQmLnaiWyHCgp2KIi9pWn8Ft3hfmxdQ36yKXNR5i5DyMTXF1DQhD?= =?us-ascii?Q?6EP+PSj+PaNTR6vsFlmyd6QsS6z7VUt3DdGe1nk/+YitKjwfFiDBOaZ8+4r4?= =?us-ascii?Q?s8S/oso5xnGtVwD5h6ai6LHrevT6fODbrClVLtT5FE0kSyhnN2kKzvVHDeIF?= =?us-ascii?Q?64p745TAXuENTAqvN3pOAyUM+IqQ1EoNi7C5UoeoHPdhxRT3xYkAPy7DS8Jr?= =?us-ascii?Q?LExhPBo5ZLu7JQzBJvrPRXyayLFBfkS9lhZNY8Z1veU4CMOS/ezDnux5qxB4?= =?us-ascii?Q?RgGvLvayBdNyT4EBoyEcAg1x/oNEPIzmHzfdilkbF1a8/in10u2dpwmeDQ01?= =?us-ascii?Q?lp5LCOCOBV4lKL7C2kbmMrpjNQhUlbQxW6/EH1ixWSoqiURmSgGeC5CwhZ/U?= =?us-ascii?Q?1VAy1LE4jM7qnkVCwbC/2X5q2P7kM/hDIvj7rQy7h4UwEeBxaquvlCwCLs9f?= =?us-ascii?Q?ckfNHH29JeN5LFu5Z6aMF/XyvWcC9zWZczzKIzCbHv6wyA3Cp65PTZ3CxQO5?= =?us-ascii?Q?hIVcpst3BOFMubJ/TwF3eINMuAfQhH9v+6UH1hePLlQPkLJVFEXxb2M9nO7X?= =?us-ascii?Q?kP5MYxEeCFdittZyxERvq5cVD9FeL4Zwe/LadFZshUTfyefmSgXCG1Nf3jtQ?= =?us-ascii?Q?LXzlHVLdD0fX0XVJoqQPxJ9RtCKdfR9zpaJFYZYZap1y2jF0YlyZZbCJByNy?= =?us-ascii?Q?zNwLemDVkxQp9G/GVy/9LeFeuSy/7YMpNoaMMa5WllNH9e+I0b2m1A3NSv6r?= =?us-ascii?Q?IR63jc5eIUZwciD3szwyaHmu0ztEg+M/yz1vA9u9n26p6r36/KLm85cL1gYN?= =?us-ascii?Q?8GOgrsIQ9Q0zBbazVe7pTCz0IKkNsEGN2CJ7P1qrvYp4wCEjD7f7SCLyAq6S?= =?us-ascii?Q?Bmsyl1lPkjgYMe6RbfUxEe+10khDh6PKDSnjahuNxuK4IEx0Z6+46VgH13a7?= =?us-ascii?Q?BlEVg5D0zvvh+LUgXeuzuBSB7+L88lZI?= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DM4PR10MB8218.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(376014)(7416014)(366016)(1800799024);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?nf9MPbvZTtg6QKTq7WwO1Jg9z+lRGPoydE/sOZEK7w+MrhVvprq//FcYb2eO?= =?us-ascii?Q?1UN+eFM3GKR71irPuiXZxh3UzOBAZHrSQ3gsPUDK4N605BjZnSsAI9WAjRUG?= =?us-ascii?Q?h35A9kVTnaVSFDw99S16+fqB/rLLB1gHOQVeZ3TC8M4JUISrRi1VfcMJ3uuD?= =?us-ascii?Q?6Z+Nh6XD8MC2ny23h+z6e1wyROZ8wxyb3+p1v9zlM20/CBr8arUFBNcR4MIX?= =?us-ascii?Q?Z5WIudfB41U+YzZim/RyinQajD1zQU7ehIX1ld/kKk+ch2yNmfK1uQT/A7S0?= =?us-ascii?Q?Cof7PNFjakJuIL2rKAT5R8JQoo5SaYReUVV3nRzT7WLqGGZFssDT7C3o5j7Q?= =?us-ascii?Q?KCpdk39EVExbbjL1oWcZ7DZSN/ydxs8EajG1go/Qf7gl8tOJyDwJBNoA5ZAR?= =?us-ascii?Q?BMxlwVmRGRc8OjLFPGLeKv5KQLdoX/hnbC0RUd6dnEX1PHVqu9YM+3303/3h?= =?us-ascii?Q?q+AkQrq0faKfUWgahrd+s3v9/OZ6PQidn7GnKRlaEkCaiFA0BFQ3opGU81iH?= =?us-ascii?Q?nT8/0rvimKxP2wzb+5OlxVNvf9OgXLYYGfkCiavn0RhBcya5Gkf1N4z9VInp?= =?us-ascii?Q?EsRP9AV+it/VGsZ10N5Emz3evM+bdnDAFnYTMjUXZqJncsA8uExxsh4hF7/O?= =?us-ascii?Q?A8LXJWoe51P42Dqpf7Tte9YJHwPEJbQk/PWmNbyk3PP6xqmzg3/Wc+QziUGE?= =?us-ascii?Q?/Z7KsBXR5zLgvJjqtSNgdl25hEnezU6flnZjqp6Hnj74lKvnL0MeblvlGdQ9?= =?us-ascii?Q?A+XeGfH25kPBBsRVC+Wxu38W4Gn6Y5mppytQooi6djfaDTRCUncddIxqBItP?= =?us-ascii?Q?yQUpZ5ND4fJIzI49Rob9Hwvy9/CvbUrdyD0ww35CubWdTYVIgj5ZNNhhXZ1T?= =?us-ascii?Q?eNfhRbpfRkhpAccFualNqhU+nCnw/y3NoLnTcVphAFEY/RO3vXqFTbzSK5tT?= =?us-ascii?Q?PUcnpm8g7iVi55qQJRAf/JIFamB3OlLh8L5Pd1Ngo3+1uXI/Dtb87M8yJVii?= =?us-ascii?Q?p8KUbfmtvyyAPv+h9HM/XVfnBmVSpYyV9XeH4+zTnMUyqlCcAhfnUDJXjF+m?= =?us-ascii?Q?GnaC9gTkdr+sZwYXyfvO1XjrPh+PPwE43PR1ZfzUVvliT3gz0EzKR4A00DUp?= =?us-ascii?Q?TQNhr3TJi0BnK708dHCpnJSVbi620eTF8LFw9GHxXGRbQtn9+3bnnhyDP5r+?= =?us-ascii?Q?mivAOsOkLmiiHhVcer85vTwoRTb0IWhP0g0E1RWeYkvTugkEl5TeiKNyDYNW?= =?us-ascii?Q?wVwlcHlBxW8GQRvX/Uq0tGc1k4O55ymlwrEEUHIVFcRK5Zo/jnGxcWhRbSXn?= =?us-ascii?Q?BP2fU7KeqNDB3KWVp2Tq9rXJXi1YN7ahxIUGB3AOtIEFiV9m6FEJlESWXAWd?= =?us-ascii?Q?Im2yIgfVVwhZ++I4yqLVtwZAf75G6Tx1YDQgGZDob6eGYncqoXm7HCoVIVPX?= =?us-ascii?Q?UD7gExcErh4q0dEgwN3a3H+kaeW9Mg7u5bKUd3f20CdfxPBnqJtvIzE47SE0?= =?us-ascii?Q?RKEj6NcFgwwVmjbHoYIZI+7VfbkSOO5CDb43Xr3pBlG1enFHQhhqtg6l4IrV?= =?us-ascii?Q?ixz2XNwQqt/ySljtGtUGvuri3VEiix3vyf9FThJzp8uawrK1oaKahb+nNvhB?= =?us-ascii?Q?TQ=3D=3D?= X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: 8K2sWm4qDSsYPytVPz0EXUQR6MYTi2YZJrRVejlDVQdvRO73XCQ9nTcQBUuBYqzGZqeWJdSrHrm0GB4RPxzWiI8QIKhksiEpYSXvvkHxBAU+4p7hRlNPr8PPv5EVITR3pDJot1IvGkYBWVfMthH2manJCHoxGp3mPWTeux/x15MkHvum42bKFnCD2juPhx+0az0QkaiX6e36hVJAHkMfn7JdIi2CmLfXm12aBn+gsjhIlGOUvRlCYfg2arNhlGPV0Bl9AKq6li4H5jJOTkk9VEZ2jvWTKrX83RPMLSGRk6hrXkwHzHHllJwvz3BfU6qvngvOyfFv5vYno8e5+NG1cXR1VGEhKnsmUJML9fOugtHrjOs+2CZywQtw5RmXWAv3C0RK8XNO1npFoAXc6rpIiBvtr8UPR50o/c4wlpzheFmavA29wPmyQKfNJ3yym8rQyPhR+GvT4uFe5+r4YIBa3HZPh7dwSX+P99WLvfl0huTd6GfOJSo/cH+uq2VDzw9PkcsfMH6VbznczfHyfON6GLuRpGU9MOmOtTMzW/7YMHq1nSj9e78GSnInCFlFML1i09B1LntnQ4kX2V1AIVvSq4AvApkm3h1Spnb4BhX1IfE= X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: f1081bde-c56f-483d-9d7f-08de1ad504ca X-MS-Exchange-CrossTenant-AuthSource: DM4PR10MB8218.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 03 Nov 2025 12:32:14.5739 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: Y63ycJdUZfccJ8wZes1xMrh1jM4M5lDy24Cxq6ilelFTxK1zdMMrLRq2YqNIqKdicbJnu/y9VnuPqwzcgpuMoLYHNaTqLxphbPGH0jwvqyw= X-MS-Exchange-Transport-CrossTenantHeadersStamped: DS4PR10MB997575 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1121,Hydra:6.1.9,FMLib:17.12.100.49 definitions=2025-11-03_01,2025-11-03_01,2025-10-01_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 phishscore=0 mlxscore=0 bulkscore=0 malwarescore=0 adultscore=0 spamscore=0 mlxlogscore=999 suspectscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2510240000 definitions=main-2511030113 X-Proofpoint-Spam-Details-Enc: AW1haW4tMjUxMTAzMDExMyBTYWx0ZWRfX3kjlXmLfqwqO i18TqG6fBFpWOLEN1MKPaAuUXH+gODd882rN3PrRw3xU6UirF23s368+5e47DPc8y5nShWKGbMf hOl72v+qBiXKWhZ2gCIEmbyzU21GnYeDpA/aB2ShzSL3h2BCbqw5yyPSv71/WadD1dug4j5+Qhf g0U70ACmIYZ7izkqPGHnXGybnD6R8bHsXBhpFAs4BOIBDFQj5eOwNtrw/QFtYQhKehbRCW7etaX zBmkgQzJe0XhuS+ecLNbE8TAKQerhikQMToSRyoh9MaFct06nfr4PJfEDCBI6cZVnEekjAV2lVs 9olGfJcnF14yjoVtnFabySmIHK/Su/U4O6DSw+c1rEOJoD7W6euSX90WpgeuTeXaXVDowwv399S P336QgcJcELbYynA3SaBFh9UWYSAUaJvCA8FRDmuP0lvuFJLNhY= X-Proofpoint-GUID: jeCFPboJg10bi9UMB3T-7mgcZkyYLDcX X-Proofpoint-ORIG-GUID: jeCFPboJg10bi9UMB3T-7mgcZkyYLDcX X-Authority-Analysis: v=2.4 cv=P7U3RyAu c=1 sm=1 tr=0 ts=6908a0d3 b=1 cx=c_pps a=e1sVV491RgrpLwSTMOnk8w==:117 a=e1sVV491RgrpLwSTMOnk8w==:17 a=6eWqkTHjU83fiwn7nKZWdM+Sl24=:19 a=z/mQ4Ysz8XfWz/Q5cLBRGdckG28=:19 a=lCpzRmAYbLLaTzLvsPZ7Mbvzbb8=:19 a=xqWC_Br6kY4A:10 a=6UeiqGixMTsA:10 a=GoEa3M9JfhUA:10 a=VkNPw1HP01LnGYTKEx00:22 a=yPCof4ZbAAAA:8 a=VwQbUJbxAAAA:8 a=QKXBBNbfDG0Dy1dDIDwA:9 cc=ntf awl=host:13657 Content-Type: text/plain; charset="utf-8" The kernel maintains leaf page table entries which contain either: - Nothing ('none' entries) - Present entries (that is stuff the hardware can navigate without fault) - Everything else that will cause a fault which the kernel handles In the 'everything else' group we include swap entries, but we also include a number of other things such as migration entries, device private entries and marker entries. Unfortunately this 'everything else' group expresses everything through a swp_entry_t type, and these entries are referred to swap entries even though they may well not contain a... swap entry. This is compounded by the rather mind-boggling concept of a non-swap swap entry (checked via non_swap_entry()) and the means by which we twist and turn to satisfy this. This patch lays the foundation for reducing this confusion. We refer to 'everything else' as a 'leaf entry'. And in fact we scoop up the 'none' entries into this concept also so we are left with: - Present entries - Leaf entries (which may be empty) This allows for radical simplification across the board - one can simply convert any leaf page table entry to a leaf entry via leafent_from_pte(). If the entry is present, we return an empty leaf entry, so it is assumed the caller is aware that they must differentiate between the two categories of page table entries, checking for the former via pte_present(). As a result, we can eliminate a number of places where we would otherwise need to use predicates to see if we can proceed with leaf page table entry conversion and instead just go ahead and do it unconditionally. We do so where we can, adjusting surrounding logic as necessary to integrate the new leaf_entry_t logic as far as seems reasonable at this stage. We typedef swp_entry_t to leaf_entry_t for the time being until the conversion can be complete, meaning everything remains compatible regardless of which type is used. We will eventually remove swp_entry_t when the conversion is complete. We introduce a new header file to keep things clear - leafops.h - this imports swapops.h so can direct replace swapops imports without issue, and we do so in all the files that require it. Additionally, add new leafops.h file to core mm maintainers entry. Signed-off-by: Lorenzo Stoakes --- MAINTAINERS | 1 + fs/proc/task_mmu.c | 26 +-- fs/userfaultfd.c | 6 +- include/linux/leafops.h | 394 ++++++++++++++++++++++++++++++++++ include/linux/mm_inline.h | 6 +- include/linux/swapops.h | 28 --- include/linux/userfaultfd_k.h | 51 +---- mm/hmm.c | 2 +- mm/hugetlb.c | 37 ++-- mm/madvise.c | 16 +- mm/memory.c | 41 ++-- mm/mincore.c | 6 +- mm/mprotect.c | 6 +- mm/mremap.c | 4 +- mm/page_vma_mapped.c | 11 +- mm/shmem.c | 7 +- mm/userfaultfd.c | 6 +- 17 files changed, 484 insertions(+), 164 deletions(-) create mode 100644 include/linux/leafops.h diff --git a/MAINTAINERS b/MAINTAINERS index 2628431dcdfe..314910a70bbf 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -16257,6 +16257,7 @@ T: git git://git.kernel.org/pub/scm/linux/kernel/gi= t/akpm/mm F: include/linux/gfp.h F: include/linux/gfp_types.h F: include/linux/highmem.h +F: include/linux/leafops.h F: include/linux/memory.h F: include/linux/mm.h F: include/linux/mm_*.h diff --git a/fs/proc/task_mmu.c b/fs/proc/task_mmu.c index fc35a0543f01..adac6c42749d 100644 --- a/fs/proc/task_mmu.c +++ b/fs/proc/task_mmu.c @@ -14,7 +14,7 @@ #include #include #include -#include +#include #include #include #include @@ -1230,11 +1230,11 @@ static int smaps_hugetlb_range(pte_t *pte, unsigned= long hmask, if (pte_present(ptent)) { folio =3D page_folio(pte_page(ptent)); present =3D true; - } else if (is_swap_pte(ptent)) { - swp_entry_t swpent =3D pte_to_swp_entry(ptent); + } else { + const leaf_entry_t entry =3D leafent_from_pte(ptent); =20 - if (is_pfn_swap_entry(swpent)) - folio =3D pfn_swap_entry_folio(swpent); + if (leafent_has_pfn(entry)) + folio =3D leafent_to_folio(entry); } =20 if (folio) { @@ -1955,9 +1955,9 @@ static pagemap_entry_t pte_to_pagemap_entry(struct pa= gemapread *pm, flags |=3D PM_SWAP; if (is_pfn_swap_entry(entry)) page =3D pfn_swap_entry_to_page(entry); - if (pte_marker_entry_uffd_wp(entry)) + if (leafent_is_uffd_wp_marker(entry)) flags |=3D PM_UFFD_WP; - if (is_guard_swp_entry(entry)) + if (leafent_is_guard_marker(entry)) flags |=3D PM_GUARD_REGION; } =20 @@ -2330,18 +2330,18 @@ static unsigned long pagemap_page_category(struct p= agemap_scan_private *p, if (pte_soft_dirty(pte)) categories |=3D PAGE_IS_SOFT_DIRTY; } else if (is_swap_pte(pte)) { - swp_entry_t swp; + leaf_entry_t entry; =20 categories |=3D PAGE_IS_SWAPPED; if (!pte_swp_uffd_wp_any(pte)) categories |=3D PAGE_IS_WRITTEN; =20 - swp =3D pte_to_swp_entry(pte); - if (is_guard_swp_entry(swp)) + entry =3D leafent_from_pte(pte); + if (leafent_is_guard_marker(entry)) categories |=3D PAGE_IS_GUARD; else if ((p->masks_of_interest & PAGE_IS_FILE) && - is_pfn_swap_entry(swp) && - !folio_test_anon(pfn_swap_entry_folio(swp))) + leafent_has_pfn(entry) && + !folio_test_anon(leafent_to_folio(entry))) categories |=3D PAGE_IS_FILE; =20 if (pte_swp_soft_dirty(pte)) @@ -2466,7 +2466,7 @@ static void make_uffd_wp_huge_pte(struct vm_area_stru= ct *vma, { unsigned long psize; =20 - if (is_hugetlb_entry_hwpoisoned(ptent) || is_pte_marker(ptent)) + if (is_hugetlb_entry_hwpoisoned(ptent) || pte_is_marker(ptent)) return; =20 psize =3D huge_page_size(hstate_vma(vma)); diff --git a/fs/userfaultfd.c b/fs/userfaultfd.c index 04c66b5001d5..e33e7df36927 100644 --- a/fs/userfaultfd.c +++ b/fs/userfaultfd.c @@ -29,7 +29,7 @@ #include #include #include -#include +#include #include #include =20 @@ -251,7 +251,7 @@ static inline bool userfaultfd_huge_must_wait(struct us= erfaultfd_ctx *ctx, if (huge_pte_none(pte)) return true; /* UFFD PTE markers require handling. */ - if (is_uffd_pte_marker(pte)) + if (pte_is_uffd_marker(pte)) return true; /* If VMA has UFFD WP faults enabled and WP fault, wait for handler. */ if (!huge_pte_write(pte) && (reason & VM_UFFD_WP)) @@ -330,7 +330,7 @@ static inline bool userfaultfd_must_wait(struct userfau= ltfd_ctx *ctx, if (pte_none(ptent)) goto out; /* UFFD PTE markers require handling. */ - if (is_uffd_pte_marker(ptent)) + if (pte_is_uffd_marker(ptent)) goto out; /* If VMA has UFFD WP faults enabled and WP fault, wait for handler. */ if (!pte_write(ptent) && (reason & VM_UFFD_WP)) diff --git a/include/linux/leafops.h b/include/linux/leafops.h new file mode 100644 index 000000000000..a1a25ca152ff --- /dev/null +++ b/include/linux/leafops.h @@ -0,0 +1,394 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#ifndef _LINUX_LEAFOPS_H +#define _LINUX_LEAFOPS_H + +#include +#include +#include + +/** + * leaf_entry_t - Describes a page table 'leaf entry'. + * + * Leaf entries are an abstract representation of all page table entries w= hich + * are non-present. Therefore these describe: + * + * - None or 'empty' entries. + * + * - All other entries which cause page faults and therefore encode + * software-controlled metadata. + * + * NOTE: While we transition from the confusing swp_entry_t type used for = this + * purpose, we simply alias this type. This will be removed once the + * transition is complete. + */ +typedef swp_entry_t leaf_entry_t; + +#ifdef CONFIG_MMU + +/* Temporary until swp_entry_t eliminated. */ +#define LEAF_TYPE_SHIFT SWP_TYPE_SHIFT + +enum leaf_entry_type { + /* Fundamental types. */ + LEAFENT_NONE, + LEAFENT_SWAP, + /* Migration types. */ + LEAFENT_MIGRATION_READ, + LEAFENT_MIGRATION_READ_EXCLUSIVE, + LEAFENT_MIGRATION_WRITE, + /* Device types. */ + LEAFENT_DEVICE_PRIVATE_READ, + LEAFENT_DEVICE_PRIVATE_WRITE, + LEAFENT_DEVICE_EXCLUSIVE, + /* H/W posion types. */ + LEAFENT_HWPOISON, + /* Marker types. */ + LEAFENT_MARKER, +}; + +/** + * leafent_mk_none() - Create an empty ('none') leaf entry. + * Returns: empty leaf entry. + */ +static inline leaf_entry_t leafent_mk_none(void) +{ + return ((leaf_entry_t) { 0 }); +} + +/** + * leafent_from_pte() - Obtain a leaf entry from a PTE entry. + * @pte: PTE entry. + * + * If @pte is present (therefore not a leaf entry) the function returns an= empty + * leaf entry. Otherwise, it returns a leaf entry. + * + * Returns: Leaf entry. + */ +static inline leaf_entry_t leafent_from_pte(pte_t pte) +{ + if (pte_present(pte)) + return leafent_mk_none(); + + /* Temporary until swp_entry_t eliminated. */ + return pte_to_swp_entry(pte); +} + +/** + * leafent_is_none() - Is the leaf entry empty? + * @entry: Leaf entry. + * + * Empty entries are typically the result of a 'none' page table leaf entry + * being converted to a leaf entry. + * + * Returns: true if the entry is empty, false otherwise. + */ +static inline bool leafent_is_none(leaf_entry_t entry) +{ + return entry.val =3D=3D 0; +} + +/** + * leafent_type() - Identify the type of leaf entry. + * @enntry: Leaf entry. + * + * Returns: the leaf entry type associated with @entry. + */ +static inline enum leaf_entry_type leafent_type(leaf_entry_t entry) +{ + unsigned int type_num; + + if (leafent_is_none(entry)) + return LEAFENT_NONE; + + type_num =3D entry.val >> LEAF_TYPE_SHIFT; + + if (type_num < MAX_SWAPFILES) + return LEAFENT_SWAP; + + switch (type_num) { +#ifdef CONFIG_MIGRATION + case SWP_MIGRATION_READ: + return LEAFENT_MIGRATION_READ; + case SWP_MIGRATION_READ_EXCLUSIVE: + return LEAFENT_MIGRATION_READ_EXCLUSIVE; + case SWP_MIGRATION_WRITE: + return LEAFENT_MIGRATION_WRITE; +#endif +#ifdef CONFIG_DEVICE_PRIVATE + case SWP_DEVICE_WRITE: + return LEAFENT_DEVICE_PRIVATE_WRITE; + case SWP_DEVICE_READ: + return LEAFENT_DEVICE_PRIVATE_READ; + case SWP_DEVICE_EXCLUSIVE: + return LEAFENT_DEVICE_EXCLUSIVE; +#endif +#ifdef CONFIG_MEMORY_FAILURE + case SWP_HWPOISON: + return LEAFENT_HWPOISON; +#endif + case SWP_PTE_MARKER: + return LEAFENT_MARKER; + } + + /* Unknown entry type. */ + VM_WARN_ON_ONCE(1); + return LEAFENT_NONE; +} + +/** + * leafent_is_swap() - Is this leaf entry a swap entry? + * @entry: Leaf entry. + * + * Returns: true if the leaf entry is a swap entry, otherwise false. + */ +static inline bool leafent_is_swap(leaf_entry_t entry) +{ + return leafent_type(entry) =3D=3D LEAFENT_SWAP; +} + +/** + * leafent_is_swap() - Is this leaf entry a migration entry? + * @entry: Leaf entry. + * + * Returns: true if the leaf entry is a migration entry, otherwise false. + */ +static inline bool leafent_is_migration(leaf_entry_t entry) +{ + switch (leafent_type(entry)) { + case LEAFENT_MIGRATION_READ: + case LEAFENT_MIGRATION_READ_EXCLUSIVE: + case LEAFENT_MIGRATION_WRITE: + return true; + default: + return false; + } +} + +/** + * leafent_is_device_private() - Is this leaf entry a device private entry? + * @entry: Leaf entry. + * + * Returns: true if the leaf entry is a device private entry, otherwise fa= lse. + */ +static inline bool leafent_is_device_private(leaf_entry_t entry) +{ + switch (leafent_type(entry)) { + case LEAFENT_DEVICE_PRIVATE_WRITE: + case LEAFENT_DEVICE_PRIVATE_READ: + return true; + default: + return false; + } +} + +static inline bool leafent_is_device_exclusive(leaf_entry_t entry) +{ + return leafent_type(entry) =3D=3D LEAFENT_DEVICE_EXCLUSIVE; +} + +/** + * leafent_is_hwpoison() - Is this leaf entry a hardware poison entry? + * @entry: Leaf entry. + * + * Returns: true if the leaf entry is a hardware poison entry, otherwise f= alse. + */ +static inline bool leafent_is_hwpoison(leaf_entry_t entry) +{ + return leafent_type(entry) =3D=3D LEAFENT_HWPOISON; +} + +/** + * leafent_is_marker() - Is this leaf entry a marker? + * @entry: Leaf entry. + * + * Returns: true if the leaf entry is a marker entry, otherwise false. + */ +static inline bool leafent_is_marker(leaf_entry_t entry) +{ + return leafent_type(entry) =3D=3D LEAFENT_MARKER; +} + +/** + * leafent_to_marker() - Obtain marker associated with leaf entry. + * @entry: Leaf entry, leafent_is_marker(@entry) must return true. + * + * Returns: Marker associated with the leaf entry. + */ +static inline pte_marker leafent_to_marker(leaf_entry_t entry) +{ + VM_WARN_ON_ONCE(!leafent_is_marker(entry)); + + return swp_offset(entry) & PTE_MARKER_MASK; +} + +/** + * leafent_has_pfn() - Does this leaf entry encode a valid PFN number? + * @entry: Leaf entry. + * + * A pfn swap entry is a special type of swap entry that always has a pfn = stored + * in the swap offset. They can either be used to represent unaddressable = device + * memory, to restrict access to a page undergoing migration or to represe= nt a + * pfn which has been hwpoisoned and unmapped. + * + * Returns: true if the leaf entry encodes a PFN, otherwise false. + */ +static inline bool leafent_has_pfn(leaf_entry_t entry) +{ + /* Make sure the swp offset can always store the needed fields. */ + BUILD_BUG_ON(SWP_TYPE_SHIFT < SWP_PFN_BITS); + + if (leafent_is_migration(entry)) + return true; + if (leafent_is_device_private(entry)) + return true; + if (leafent_is_device_exclusive(entry)) + return true; + if (leafent_is_hwpoison(entry)) + return true; + + return false; +} + +/** + * leafent_to_pfn() - Obtain PFN encoded within leaf entry. + * @entry: Leaf entry, leafent_has_pfn(@entry) must return true. + * + * Returns: The PFN associated with the leaf entry. + */ +static inline unsigned long leafent_to_pfn(leaf_entry_t entry) +{ + VM_WARN_ON_ONCE(!leafent_has_pfn(entry)); + + /* Temporary until swp_entry_t eliminated. */ + return swp_offset_pfn(entry); +} + +/** + * leafent_to_page() - Obtains struct page for PFN encoded within leaf ent= ry. + * @entry: Leaf entry, leafent_has_pfn(@entry) must return true. + * + * Returns: Pointer to the struct page associated with the leaf entry's PF= N. + */ +static inline struct page *leafent_to_page(leaf_entry_t entry) +{ + VM_WARN_ON_ONCE(!leafent_has_pfn(entry)); + + /* Temporary until swp_entry_t eliminated. */ + return pfn_swap_entry_to_page(entry); +} + +/** + * leafent_to_folio() - Obtains struct folio for PFN encoded within leaf e= ntry. + * @entry: Leaf entry, leafent_has_pfn(@entry) must return true. + * + * Returns: Pointer to the struct folio associated with the leaf entry's P= FN. + * Returns: + */ +static inline struct folio *leafent_to_folio(leaf_entry_t entry) +{ + VM_WARN_ON_ONCE(!leafent_has_pfn(entry)); + + /* Temporary until swp_entry_t eliminated. */ + return pfn_swap_entry_folio(entry); +} + +/** + * leafent_is_poison_marker() - Is this leaf entry a poison marker? + * @entry: Leaf entry. + * + * The poison marker is set via UFFDIO_POISON. Userfaultfd-specific. + * + * Returns: true if the leaf entry is a poison marker, otherwise false. + */ +static inline bool leafent_is_poison_marker(leaf_entry_t entry) +{ + if (!leafent_is_marker(entry)) + return false; + + return leafent_to_marker(entry) & PTE_MARKER_POISONED; +} + +/** + * leafent_is_guard_marker() - Is this leaf entry a guard region marker? + * @entry: Leaf entry. + * + * Returns: true if the leaf entry is a guard marker, otherwise false. + */ +static inline bool leafent_is_guard_marker(leaf_entry_t entry) +{ + if (!leafent_is_marker(entry)) + return false; + + return leafent_to_marker(entry) & PTE_MARKER_GUARD; +} + +/** + * leafent_is_uffd_wp_marker() - Is this leaf entry a userfautlfd write pr= otect + * marker? + * @entry: Leaf entry. + * + * Userfaultfd-specific. + * + * Returns: true if the leaf entry is a UFFD WP marker, otherwise false. + */ +static inline bool leafent_is_uffd_wp_marker(leaf_entry_t entry) +{ + if (!leafent_is_marker(entry)) + return false; + + return leafent_to_marker(entry) & PTE_MARKER_UFFD_WP; +} + +/** + * pte_is_marker() - Does the PTE entry encode a marker leaf entry? + * @pte: PTE entry. + * + * Returns: true if this PTE is a marker leaf entry, otherwise false. + */ +static inline bool pte_is_marker(pte_t pte) +{ + return leafent_is_marker(leafent_from_pte(pte)); +} + +/** + * pte_is_uffd_wp_marker() - Does this PTE entry encode a userfaultfd write + * protect marker leaf entry? + * @pte: PTE entry. + * + * Returns: true if this PTE is a UFFD WP marker leaf entry, otherwise fal= se. + */ +static inline bool pte_is_uffd_wp_marker(pte_t pte) +{ + const leaf_entry_t entry =3D leafent_from_pte(pte); + + return leafent_is_uffd_wp_marker(entry); +} + +/** + * pte_is_uffd_marker() - Does this PTE entry encode a userfault-specific = marker + * leaf entry? + * @entry: Leaf entry. + * + * It's useful to be able to determine which leaf entries encode UFFD-spec= ific + * markers so we can handle these correctly. + * + * Returns: true if this PTE entry is a UFFD-specific marker, otherwise fa= lse. + */ +static inline bool pte_is_uffd_marker(pte_t pte) +{ + const leaf_entry_t entry =3D leafent_from_pte(pte); + + if (!leafent_is_marker(entry)) + return false; + + /* UFFD WP, poisoned swap entries are UFFD-handled. */ + if (leafent_is_uffd_wp_marker(entry)) + return true; + if (leafent_is_poison_marker(entry)) + return true; + + return false; +} + +#endif /* CONFIG_MMU */ +#endif /* _LINUX_SWAPOPS_H */ diff --git a/include/linux/mm_inline.h b/include/linux/mm_inline.h index f6a2b2d20016..795b255abf65 100644 --- a/include/linux/mm_inline.h +++ b/include/linux/mm_inline.h @@ -8,7 +8,7 @@ #include #include #include -#include +#include =20 /** * folio_is_file_lru - Should the folio be on a file LRU or anon LRU? @@ -541,9 +541,9 @@ static inline bool mm_tlb_flush_nested(const struct mm_= struct *mm) * The caller should insert a new pte created with make_pte_marker(). */ static inline pte_marker copy_pte_marker( - swp_entry_t entry, struct vm_area_struct *dst_vma) + leaf_entry_t entry, struct vm_area_struct *dst_vma) { - pte_marker srcm =3D pte_marker_get(entry); + const pte_marker srcm =3D leafent_to_marker(entry); /* Always copy error entries. */ pte_marker dstm =3D srcm & (PTE_MARKER_POISONED | PTE_MARKER_GUARD); =20 diff --git a/include/linux/swapops.h b/include/linux/swapops.h index d1f665935cfc..0a4b3f51ecf5 100644 --- a/include/linux/swapops.h +++ b/include/linux/swapops.h @@ -426,21 +426,6 @@ static inline swp_entry_t make_pte_marker_entry(pte_ma= rker marker) return swp_entry(SWP_PTE_MARKER, marker); } =20 -static inline bool is_pte_marker_entry(swp_entry_t entry) -{ - return swp_type(entry) =3D=3D SWP_PTE_MARKER; -} - -static inline pte_marker pte_marker_get(swp_entry_t entry) -{ - return swp_offset(entry) & PTE_MARKER_MASK; -} - -static inline bool is_pte_marker(pte_t pte) -{ - return is_swap_pte(pte) && is_pte_marker_entry(pte_to_swp_entry(pte)); -} - static inline pte_t make_pte_marker(pte_marker marker) { return swp_entry_to_pte(make_pte_marker_entry(marker)); @@ -451,24 +436,11 @@ static inline swp_entry_t make_poisoned_swp_entry(voi= d) return make_pte_marker_entry(PTE_MARKER_POISONED); } =20 -static inline int is_poisoned_swp_entry(swp_entry_t entry) -{ - return is_pte_marker_entry(entry) && - (pte_marker_get(entry) & PTE_MARKER_POISONED); - -} - static inline swp_entry_t make_guard_swp_entry(void) { return make_pte_marker_entry(PTE_MARKER_GUARD); } =20 -static inline int is_guard_swp_entry(swp_entry_t entry) -{ - return is_pte_marker_entry(entry) && - (pte_marker_get(entry) & PTE_MARKER_GUARD); -} - static inline struct page *pfn_swap_entry_to_page(swp_entry_t entry) { struct page *p =3D pfn_to_page(swp_offset_pfn(entry)); diff --git a/include/linux/userfaultfd_k.h b/include/linux/userfaultfd_k.h index da0b4fcc566f..983c860a00f1 100644 --- a/include/linux/userfaultfd_k.h +++ b/include/linux/userfaultfd_k.h @@ -16,7 +16,7 @@ #include #include #include -#include +#include #include #include =20 @@ -434,32 +434,6 @@ static inline bool userfaultfd_wp_use_markers(struct v= m_area_struct *vma) return userfaultfd_wp_unpopulated(vma); } =20 -static inline bool pte_marker_entry_uffd_wp(swp_entry_t entry) -{ -#ifdef CONFIG_PTE_MARKER_UFFD_WP - return is_pte_marker_entry(entry) && - (pte_marker_get(entry) & PTE_MARKER_UFFD_WP); -#else - return false; -#endif -} - -static inline bool pte_marker_uffd_wp(pte_t pte) -{ -#ifdef CONFIG_PTE_MARKER_UFFD_WP - swp_entry_t entry; - - if (!is_swap_pte(pte)) - return false; - - entry =3D pte_to_swp_entry(pte); - - return pte_marker_entry_uffd_wp(entry); -#else - return false; -#endif -} - /* * Returns true if this is a swap pte and was uffd-wp wr-protected in eith= er * forms (pte marker or a normal swap pte), false otherwise. @@ -473,31 +447,10 @@ static inline bool pte_swp_uffd_wp_any(pte_t pte) if (pte_swp_uffd_wp(pte)) return true; =20 - if (pte_marker_uffd_wp(pte)) + if (pte_is_uffd_wp_marker(pte)) return true; #endif return false; } =20 - -static inline bool is_uffd_pte_marker(pte_t pte) -{ - swp_entry_t entry; - - if (pte_present(pte)) - return false; - - entry =3D pte_to_swp_entry(pte); - if (!is_pte_marker_entry(entry)) - return false; - - /* UFFD WP, poisoned swap entries are UFFD handled. */ - if (pte_marker_entry_uffd_wp(entry)) - return true; - if (is_poisoned_swp_entry(entry)) - return true; - - return false; -} - #endif /* _LINUX_USERFAULTFD_K_H */ diff --git a/mm/hmm.c b/mm/hmm.c index 43d4a91035ff..b11b4ebba945 100644 --- a/mm/hmm.c +++ b/mm/hmm.c @@ -244,7 +244,7 @@ static int hmm_vma_handle_pte(struct mm_walk *walk, uns= igned long addr, uint64_t pfn_req_flags =3D *hmm_pfn; uint64_t new_pfn_flags =3D 0; =20 - if (pte_none(pte) || pte_marker_uffd_wp(pte)) { + if (pte_none(pte) || pte_is_uffd_wp_marker(pte)) { required_fault =3D hmm_pte_need_fault(hmm_vma_walk, pfn_req_flags, 0); if (required_fault) diff --git a/mm/hugetlb.c b/mm/hugetlb.c index 01c784547d1e..cc7db8fd86db 100644 --- a/mm/hugetlb.c +++ b/mm/hugetlb.c @@ -28,7 +28,7 @@ #include #include #include -#include +#include #include #include #include @@ -5662,17 +5662,17 @@ int copy_hugetlb_page_range(struct mm_struct *dst, = struct mm_struct *src, entry =3D huge_pte_clear_uffd_wp(entry); set_huge_pte_at(dst, addr, dst_pte, entry, sz); } else if (unlikely(is_hugetlb_entry_migration(entry))) { - swp_entry_t swp_entry =3D pte_to_swp_entry(entry); + leaf_entry_t leafent =3D leafent_from_pte(entry); bool uffd_wp =3D pte_swp_uffd_wp(entry); =20 - if (!is_readable_migration_entry(swp_entry) && cow) { + if (!is_readable_migration_entry(leafent) && cow) { /* * COW mappings require pages in both * parent and child to be set to read. */ - swp_entry =3D make_readable_migration_entry( - swp_offset(swp_entry)); - entry =3D swp_entry_to_pte(swp_entry); + leafent =3D make_readable_migration_entry( + swp_offset(leafent)); + entry =3D swp_entry_to_pte(leafent); if (userfaultfd_wp(src_vma) && uffd_wp) entry =3D pte_swp_mkuffd_wp(entry); set_huge_pte_at(src, addr, src_pte, entry, sz); @@ -5680,9 +5680,9 @@ int copy_hugetlb_page_range(struct mm_struct *dst, st= ruct mm_struct *src, if (!userfaultfd_wp(dst_vma)) entry =3D huge_pte_clear_uffd_wp(entry); set_huge_pte_at(dst, addr, dst_pte, entry, sz); - } else if (unlikely(is_pte_marker(entry))) { - pte_marker marker =3D copy_pte_marker( - pte_to_swp_entry(entry), dst_vma); + } else if (unlikely(pte_is_marker(entry))) { + const leaf_entry_t leafent =3D leafent_from_pte(entry); + const pte_marker marker =3D copy_pte_marker(leafent, dst_vma); =20 if (marker) set_huge_pte_at(dst, addr, dst_pte, @@ -5798,7 +5798,7 @@ static void move_huge_pte(struct vm_area_struct *vma,= unsigned long old_addr, =20 pte =3D huge_ptep_get_and_clear(mm, old_addr, src_pte, sz); =20 - if (need_clear_uffd_wp && pte_marker_uffd_wp(pte)) + if (need_clear_uffd_wp && pte_is_uffd_wp_marker(pte)) huge_pte_clear(mm, new_addr, dst_pte, sz); else { if (need_clear_uffd_wp) { @@ -6617,7 +6617,7 @@ static vm_fault_t hugetlb_no_page(struct address_spac= e *mapping, * If this pte was previously wr-protected, keep it wr-protected even * if populated. */ - if (unlikely(pte_marker_uffd_wp(vmf->orig_pte))) + if (unlikely(pte_is_uffd_wp_marker(vmf->orig_pte))) new_pte =3D huge_pte_mkuffd_wp(new_pte); set_huge_pte_at(mm, vmf->address, vmf->pte, new_pte, huge_page_size(h)); =20 @@ -6750,9 +6750,9 @@ vm_fault_t hugetlb_fault(struct mm_struct *mm, struct= vm_area_struct *vma, */ return hugetlb_no_page(mapping, &vmf); =20 - if (is_pte_marker(vmf.orig_pte)) { + if (pte_is_marker(vmf.orig_pte)) { const pte_marker marker =3D - pte_marker_get(pte_to_swp_entry(vmf.orig_pte)); + leafent_to_marker(leafent_from_pte(vmf.orig_pte)); =20 if (marker & PTE_MARKER_POISONED) { ret =3D VM_FAULT_HWPOISON_LARGE | @@ -7080,7 +7080,7 @@ int hugetlb_mfill_atomic_pte(pte_t *dst_pte, * See comment about UFFD marker overwriting in * mfill_atomic_install_pte(). */ - if (!huge_pte_none(dst_ptep) && !is_uffd_pte_marker(dst_ptep)) + if (!huge_pte_none(dst_ptep) && !pte_is_uffd_marker(dst_ptep)) goto out_release_unlock; =20 if (folio_in_pagecache) @@ -7201,8 +7201,9 @@ long hugetlb_change_protection(struct vm_area_struct = *vma, if (unlikely(is_hugetlb_entry_hwpoisoned(pte))) { /* Nothing to do. */ } else if (unlikely(is_hugetlb_entry_migration(pte))) { - swp_entry_t entry =3D pte_to_swp_entry(pte); - struct folio *folio =3D pfn_swap_entry_folio(entry); + leaf_entry_t entry =3D leafent_from_pte(pte); + + struct folio *folio =3D leafent_to_folio(entry); pte_t newpte =3D pte; =20 if (is_writable_migration_entry(entry)) { @@ -7222,14 +7223,14 @@ long hugetlb_change_protection(struct vm_area_struc= t *vma, newpte =3D pte_swp_clear_uffd_wp(newpte); if (!pte_same(pte, newpte)) set_huge_pte_at(mm, address, ptep, newpte, psize); - } else if (unlikely(is_pte_marker(pte))) { + } else if (unlikely(pte_is_marker(pte))) { /* * Do nothing on a poison marker; page is * corrupted, permissions do not apply. Here * pte_marker_uffd_wp()=3D=3Dtrue implies !poison * because they're mutual exclusive. */ - if (pte_marker_uffd_wp(pte) && uffd_wp_resolve) + if (pte_is_uffd_wp_marker(pte) && uffd_wp_resolve) /* Safe to modify directly (non-present->none). */ huge_pte_clear(mm, address, ptep, psize); } else if (!huge_pte_none(pte)) { diff --git a/mm/madvise.c b/mm/madvise.c index fb1c86e630b6..27e078098926 100644 --- a/mm/madvise.c +++ b/mm/madvise.c @@ -29,7 +29,7 @@ #include #include #include -#include +#include #include #include =20 @@ -690,17 +690,16 @@ static int madvise_free_pte_range(pmd_t *pmd, unsigne= d long addr, * (page allocation + zeroing). */ if (!pte_present(ptent)) { - swp_entry_t entry; + leaf_entry_t entry =3D leafent_from_pte(ptent); =20 - entry =3D pte_to_swp_entry(ptent); - if (!non_swap_entry(entry)) { + if (leafent_is_swap(entry)) { max_nr =3D (end - addr) / PAGE_SIZE; nr =3D swap_pte_batch(pte, max_nr, ptent); nr_swap -=3D nr; free_swap_and_cache_nr(entry, nr); clear_not_present_full_ptes(mm, addr, pte, nr, tlb->fullmm); - } else if (is_hwpoison_entry(entry) || - is_poisoned_swp_entry(entry)) { + } else if (leafent_is_hwpoison(entry) || + leafent_is_poison_marker(entry)) { pte_clear_not_present_full(mm, addr, pte, tlb->fullmm); } continue; @@ -1071,8 +1070,9 @@ static bool is_valid_guard_vma(struct vm_area_struct = *vma, bool allow_locked) =20 static bool is_guard_pte_marker(pte_t ptent) { - return is_swap_pte(ptent) && - is_guard_swp_entry(pte_to_swp_entry(ptent)); + const leaf_entry_t entry =3D leafent_from_pte(ptent); + + return leafent_is_guard_marker(entry); } =20 static int guard_install_pud_entry(pud_t *pud, unsigned long addr, diff --git a/mm/memory.c b/mm/memory.c index 4c3a7e09a159..299ce5dcba76 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -60,7 +60,7 @@ #include #include #include -#include +#include #include #include #include @@ -109,7 +109,7 @@ static __always_inline bool vmf_orig_pte_uffd_wp(struct= vm_fault *vmf) if (!(vmf->flags & FAULT_FLAG_ORIG_PTE_VALID)) return false; =20 - return pte_marker_uffd_wp(vmf->orig_pte); + return pte_is_uffd_wp_marker(vmf->orig_pte); } =20 /* @@ -927,10 +927,10 @@ copy_nonpresent_pte(struct mm_struct *dst_mm, struct = mm_struct *src_mm, { vm_flags_t vm_flags =3D dst_vma->vm_flags; pte_t orig_pte =3D ptep_get(src_pte); + leaf_entry_t entry =3D leafent_from_pte(orig_pte); pte_t pte =3D orig_pte; struct folio *folio; struct page *page; - swp_entry_t entry =3D pte_to_swp_entry(orig_pte); =20 if (likely(!non_swap_entry(entry))) { if (swap_duplicate(entry) < 0) @@ -1016,7 +1016,7 @@ copy_nonpresent_pte(struct mm_struct *dst_mm, struct = mm_struct *src_mm, if (try_restore_exclusive_pte(src_vma, addr, src_pte, orig_pte)) return -EBUSY; return -ENOENT; - } else if (is_pte_marker_entry(entry)) { + } else if (leafent_is_marker(entry)) { pte_marker marker =3D copy_pte_marker(entry, dst_vma); =20 if (marker) @@ -1717,14 +1717,14 @@ static inline int zap_nonpresent_ptes(struct mmu_ga= ther *tlb, unsigned int max_nr, unsigned long addr, struct zap_details *details, int *rss, bool *any_skipped) { - swp_entry_t entry; + leaf_entry_t entry; int nr =3D 1; =20 *any_skipped =3D true; - entry =3D pte_to_swp_entry(ptent); - if (is_device_private_entry(entry) || - is_device_exclusive_entry(entry)) { - struct page *page =3D pfn_swap_entry_to_page(entry); + entry =3D leafent_from_pte(ptent); + if (leafent_is_device_private(entry) || + leafent_is_device_exclusive(entry)) { + struct page *page =3D leafent_to_page(entry); struct folio *folio =3D page_folio(page); =20 if (unlikely(!should_zap_folio(details, folio))) @@ -1739,7 +1739,7 @@ static inline int zap_nonpresent_ptes(struct mmu_gath= er *tlb, rss[mm_counter(folio)]--; folio_remove_rmap_pte(folio, page, vma); folio_put(folio); - } else if (!non_swap_entry(entry)) { + } else if (leafent_is_swap(entry)) { /* Genuine swap entries, hence a private anon pages */ if (!should_zap_cows(details)) return 1; @@ -1747,20 +1747,20 @@ static inline int zap_nonpresent_ptes(struct mmu_ga= ther *tlb, nr =3D swap_pte_batch(pte, max_nr, ptent); rss[MM_SWAPENTS] -=3D nr; free_swap_and_cache_nr(entry, nr); - } else if (is_migration_entry(entry)) { - struct folio *folio =3D pfn_swap_entry_folio(entry); + } else if (leafent_is_migration(entry)) { + struct folio *folio =3D leafent_to_folio(entry); =20 if (!should_zap_folio(details, folio)) return 1; rss[mm_counter(folio)]--; - } else if (pte_marker_entry_uffd_wp(entry)) { + } else if (leafent_is_uffd_wp_marker(entry)) { /* * For anon: always drop the marker; for file: only * drop the marker if explicitly requested. */ if (!vma_is_anonymous(vma) && !zap_drop_markers(details)) return 1; - } else if (is_guard_swp_entry(entry)) { + } else if (leafent_is_guard_marker(entry)) { /* * Ordinary zapping should not remove guard PTE * markers. Only do so if we should remove PTE markers @@ -1768,7 +1768,8 @@ static inline int zap_nonpresent_ptes(struct mmu_gath= er *tlb, */ if (!zap_drop_markers(details)) return 1; - } else if (is_hwpoison_entry(entry) || is_poisoned_swp_entry(entry)) { + } else if (leafent_is_hwpoison(entry) || + leafent_is_poison_marker(entry)) { if (!should_zap_cows(details)) return 1; } else { @@ -4390,7 +4391,7 @@ static vm_fault_t pte_marker_clear(struct vm_fault *v= mf) * * This should also cover the case where e.g. the pte changed * quickly from a PTE_MARKER_UFFD_WP into PTE_MARKER_POISONED. - * So is_pte_marker() check is not enough to safely drop the pte. + * So pte_is_marker() check is not enough to safely drop the pte. */ if (pte_same(vmf->orig_pte, ptep_get(vmf->pte))) pte_clear(vmf->vma->vm_mm, vmf->address, vmf->pte); @@ -4424,8 +4425,8 @@ static vm_fault_t pte_marker_handle_uffd_wp(struct vm= _fault *vmf) =20 static vm_fault_t handle_pte_marker(struct vm_fault *vmf) { - swp_entry_t entry =3D pte_to_swp_entry(vmf->orig_pte); - unsigned long marker =3D pte_marker_get(entry); + const leaf_entry_t entry =3D leafent_from_pte(vmf->orig_pte); + const pte_marker marker =3D leafent_to_marker(entry); =20 /* * PTE markers should never be empty. If anything weird happened, @@ -4442,7 +4443,7 @@ static vm_fault_t handle_pte_marker(struct vm_fault *= vmf) if (marker & PTE_MARKER_GUARD) return VM_FAULT_SIGSEGV; =20 - if (pte_marker_entry_uffd_wp(entry)) + if (leafent_is_uffd_wp_marker(entry)) return pte_marker_handle_uffd_wp(vmf); =20 /* This is an unknown pte marker */ @@ -4690,7 +4691,7 @@ vm_fault_t do_swap_page(struct vm_fault *vmf) } } else if (is_hwpoison_entry(entry)) { ret =3D VM_FAULT_HWPOISON; - } else if (is_pte_marker_entry(entry)) { + } else if (leafent_is_marker(entry)) { ret =3D handle_pte_marker(vmf); } else { print_bad_pte(vma, vmf->address, vmf->orig_pte, NULL); diff --git a/mm/mincore.c b/mm/mincore.c index 151b2dbb783b..e77c5bc88fc7 100644 --- a/mm/mincore.c +++ b/mm/mincore.c @@ -14,7 +14,7 @@ #include #include #include -#include +#include #include #include #include @@ -42,7 +42,7 @@ static int mincore_hugetlb(pte_t *pte, unsigned long hmas= k, unsigned long addr, } else { const pte_t ptep =3D huge_ptep_get(walk->mm, addr, pte); =20 - if (huge_pte_none(ptep) || is_pte_marker(ptep)) + if (huge_pte_none(ptep) || pte_is_marker(ptep)) present =3D 0; else present =3D 1; @@ -187,7 +187,7 @@ static int mincore_pte_range(pmd_t *pmd, unsigned long = addr, unsigned long end, =20 step =3D 1; /* We need to do cache lookup too for UFFD pte markers */ - if (pte_none(pte) || is_uffd_pte_marker(pte)) + if (pte_none(pte) || pte_is_uffd_marker(pte)) __mincore_unmapped_range(addr, addr + PAGE_SIZE, vma, vec); else if (pte_present(pte)) { diff --git a/mm/mprotect.c b/mm/mprotect.c index ab4e06cd9a69..d425be97db51 100644 --- a/mm/mprotect.c +++ b/mm/mprotect.c @@ -326,14 +326,14 @@ static long change_pte_range(struct mmu_gather *tlb, newpte =3D swp_entry_to_pte(entry); if (pte_swp_uffd_wp(oldpte)) newpte =3D pte_swp_mkuffd_wp(newpte); - } else if (is_pte_marker_entry(entry)) { + } else if (leafent_is_marker(entry)) { /* * Ignore error swap entries unconditionally, * because any access should sigbus/sigsegv * anyway. */ - if (is_poisoned_swp_entry(entry) || - is_guard_swp_entry(entry)) + if (leafent_is_poison_marker(entry) || + leafent_is_guard_marker(entry)) continue; /* * If this is uffd-wp pte marker and we'd like diff --git a/mm/mremap.c b/mm/mremap.c index 8ad06cf50783..7c21b2ad13f6 100644 --- a/mm/mremap.c +++ b/mm/mremap.c @@ -17,7 +17,7 @@ #include #include #include -#include +#include #include #include #include @@ -288,7 +288,7 @@ static int move_ptes(struct pagetable_move_control *pmc, pte =3D move_pte(pte, old_addr, new_addr); pte =3D move_soft_dirty_pte(pte); =20 - if (need_clear_uffd_wp && pte_marker_uffd_wp(pte)) + if (need_clear_uffd_wp && pte_is_uffd_wp_marker(pte)) pte_clear(mm, new_addr, new_ptep); else { if (need_clear_uffd_wp) { diff --git a/mm/page_vma_mapped.c b/mm/page_vma_mapped.c index 137ce27ff68c..e0560cc1ce18 100644 --- a/mm/page_vma_mapped.c +++ b/mm/page_vma_mapped.c @@ -3,7 +3,7 @@ #include #include #include -#include +#include =20 #include "internal.h" =20 @@ -107,15 +107,12 @@ static bool check_pte(struct page_vma_mapped_walk *pv= mw, unsigned long pte_nr) pte_t ptent =3D ptep_get(pvmw->pte); =20 if (pvmw->flags & PVMW_MIGRATION) { - swp_entry_t entry; - if (!is_swap_pte(ptent)) - return false; - entry =3D pte_to_swp_entry(ptent); + const leaf_entry_t entry =3D leafent_from_pte(ptent); =20 - if (!is_migration_entry(entry)) + if (!leafent_is_migration(entry)) return false; =20 - pfn =3D swp_offset_pfn(entry); + pfn =3D leafent_to_pfn(entry); } else if (is_swap_pte(ptent)) { swp_entry_t entry; =20 diff --git a/mm/shmem.c b/mm/shmem.c index 6580f3cd24bb..b761c105f84f 100644 --- a/mm/shmem.c +++ b/mm/shmem.c @@ -66,7 +66,7 @@ static struct vfsmount *shm_mnt __ro_after_init; #include #include #include -#include +#include #include #include #include @@ -2286,7 +2286,8 @@ static int shmem_swapin_folio(struct inode *inode, pg= off_t index, struct address_space *mapping =3D inode->i_mapping; struct mm_struct *fault_mm =3D vma ? vma->vm_mm : NULL; struct shmem_inode_info *info =3D SHMEM_I(inode); - swp_entry_t swap, index_entry; + swp_entry_t swap; + leaf_entry_t index_entry; struct swap_info_struct *si; struct folio *folio =3D NULL; bool skip_swapcache =3D false; @@ -2298,7 +2299,7 @@ static int shmem_swapin_folio(struct inode *inode, pg= off_t index, swap =3D index_entry; *foliop =3D NULL; =20 - if (is_poisoned_swp_entry(index_entry)) + if (leafent_is_poison_marker(index_entry)) return -EIO; =20 si =3D get_swap_device(index_entry); diff --git a/mm/userfaultfd.c b/mm/userfaultfd.c index cc4ce205bbec..055ec1050776 100644 --- a/mm/userfaultfd.c +++ b/mm/userfaultfd.c @@ -10,7 +10,7 @@ #include #include #include -#include +#include #include #include #include @@ -208,7 +208,7 @@ int mfill_atomic_install_pte(pmd_t *dst_pmd, * MISSING|WP registered, we firstly wr-protect a none pte which has no * page cache page backing it, then access the page. */ - if (!pte_none(dst_ptep) && !is_uffd_pte_marker(dst_ptep)) + if (!pte_none(dst_ptep) && !pte_is_uffd_marker(dst_ptep)) goto out_unlock; =20 if (page_in_cache) { @@ -590,7 +590,7 @@ static __always_inline ssize_t mfill_atomic_hugetlb( if (!uffd_flags_mode_is(flags, MFILL_ATOMIC_CONTINUE)) { const pte_t ptep =3D huge_ptep_get(dst_mm, dst_addr, dst_pte); =20 - if (!huge_pte_none(ptep) && !is_uffd_pte_marker(ptep)) { + if (!huge_pte_none(ptep) && !pte_is_uffd_marker(ptep)) { err =3D -EEXIST; hugetlb_vma_unlock_read(dst_vma); mutex_unlock(&hugetlb_fault_mutex_table[hash]); --=20 2.51.0 From nobody Sat Feb 7 17:55:59 2026 Received: from mx0b-00069f02.pphosted.com (mx0b-00069f02.pphosted.com [205.220.177.32]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9096F30DD10; Mon, 3 Nov 2025 12:34:10 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=205.220.177.32 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1762173254; cv=fail; b=oAW5mhg0T8g5GOztkUa0pqnz8oTX/nEOKf5PqTi9vP0yks2FLe6sEnVctaNVQqGFRoNU1FxapgDn2Otm2a2BNYUBILG1y3zqXJ58oTQsgOqfHXJDCy0wUiYa2SWOLhEWwbExMlEk2IcOh574kb9mRENESbeUoXIFiOG3fPoKQi0= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1762173254; c=relaxed/simple; bh=sboUBQ9sciNw2B2GbAhmGC7i+jR0TEQkIu1UlnvxzyI=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: Content-Type:MIME-Version; b=nlpmE/EVKVQkNM9l8tcy2D2ae8MeZDmLr9rt/Wl2yn3MMB1wJOxDmJNbgCg6qVj1FXjocKVuMKJACqKORrzWeL9vSTkT4m7VxHZn7z68toKeKFPTGUtFNrAsFwSfXkxOL9OtWTc9gvvRM9Y74T7G+Z67+aGoXGA5Z9ZfvHqARqQ= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=oracle.com; spf=pass smtp.mailfrom=oracle.com; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b=o+Rea1EQ; dkim=pass (1024-bit key) header.d=oracle.onmicrosoft.com header.i=@oracle.onmicrosoft.com header.b=A60B6A3R; arc=fail smtp.client-ip=205.220.177.32 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=oracle.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=oracle.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b="o+Rea1EQ"; dkim=pass (1024-bit key) header.d=oracle.onmicrosoft.com header.i=@oracle.onmicrosoft.com header.b="A60B6A3R" Received: from pps.filterd (m0246632.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 5A3CP2F0011016; Mon, 3 Nov 2025 12:32:21 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=cc :content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s= corp-2025-04-25; bh=E0whp4gfpwladQozAl95xCPKz4HIiPD0+Gt7POR0uIU=; b= o+Rea1EQvhnuJDtBPSf6KZcAgqRvOYvPcmCJI9gRMx7h5qLVAdxSK+UzCT3fIbvC us2dRCpjboIy2XSIQzXY7oaKqH8ADGeL3OsQAbOAF5K98Cq6Q0ewXTel4AAerF7F SsFlmFiQuuinpT7amCaUtj9txTXfuXv6002Q4hPtci8z9Y0WpnAZiShcNG8svwdb w843qfwmQFJ58Mfu8zvVvudsVnRnNyKA46DyMImRuLGcEIPoFs3aq7998Md5JBhO cRod/AZJKJTZ7QYPxZBrPKXqS65bxqHbCSMvGSrtF+eaKCFVlk/wLygzZOqvYC1N duIMWC5D3JoTwtJEsZfKXg== Received: from iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta02.appoci.oracle.com [147.154.18.20]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 4a6vca00ae-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 03 Nov 2025 12:32:20 +0000 (GMT) Received: from pps.filterd (iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (8.18.1.2/8.18.1.2) with ESMTP id 5A3BSiuD009599; Mon, 3 Nov 2025 12:32:20 GMT Received: from cy3pr05cu001.outbound.protection.outlook.com (mail-westcentralusazon11013032.outbound.protection.outlook.com [40.93.201.32]) by iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTPS id 4a58nbksty-2 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 03 Nov 2025 12:32:19 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=eNGttSffBpV5cQv0ufLGkPa91RWzMhxqoaTcMhrDTt7JlsEQrsAI52pv6q4KgQBp4wbWJYqoe4lV/j5jS/RrvEb6KPokO+DO51Il7qRYEedAvKc4KjYQY8vJMtJWedLcn7dEn0wuqv+956FdBgh1v0z6uLHX8UNafSWa6ArQNjuOsZuoRWuUzxYDp3EI7n9+4ki65K1dt4nBRDgjy7mauAEGvtBjUSiQuVGYVA3uU55GbybMysLDy2ILoDErN41WxkJWtQu3p/B395Hnv0+xnjErux9AyBELjSMcfnwX+dUZwkyYjaC3OsjmtNvmMNiqp0KMeTc9A0zm04YfAILHsg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=E0whp4gfpwladQozAl95xCPKz4HIiPD0+Gt7POR0uIU=; b=CXhIRmEV/DPGjarMBJcC8kkHxD/LwEMDeCfHHA022X1xACDOA4I40h+iL4tALMHBT6yliBRu6JXMSUzmkx7BMGfBPUJonCbGmKIypODfZ9Y83zPgq7t8R9mB2l97x3THyjFI2al8IrtuTLDGpHTPAW13aGiwMCU2hZnYVIAMjRFgtAiLFjJb1hs7JKBlqe9ZiPc5WERBOkVmoA80hQUeQxE+/PIQ3LWUHaTCU443nYjArwPut9+hrP7aWnEOn6X8dGhnYL5YV52mCJ3B62IbjRa93pj1Lq866nKp36DNQDTtBmcT8Gvrmw6jKZNFkAbJtWEdoPP6Cn8brJIIjDthNg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=E0whp4gfpwladQozAl95xCPKz4HIiPD0+Gt7POR0uIU=; b=A60B6A3RLBfxZVj9ScwIhZO6nyR7ODXUhbd3kYhO/HHlFdC18EZSGQ9Qe+R+vRy1ogpREYPHj0Is5eISV3bwnVqK91x4ARUtvbZHu8IcXXS83HRz7fp0YXUFKRVKbnb8Zi6v/l7dfIa+1dO9Ci2bLOTnwksCnnhDoVXpswjNK54= Received: from DM4PR10MB8218.namprd10.prod.outlook.com (2603:10b6:8:1cc::16) by DS4PR10MB997575.namprd10.prod.outlook.com (2603:10b6:8:31e::5) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9275.16; Mon, 3 Nov 2025 12:32:17 +0000 Received: from DM4PR10MB8218.namprd10.prod.outlook.com ([fe80::2650:55cf:2816:5f2]) by DM4PR10MB8218.namprd10.prod.outlook.com ([fe80::2650:55cf:2816:5f2%2]) with mapi id 15.20.9253.018; Mon, 3 Nov 2025 12:32:17 +0000 From: Lorenzo Stoakes To: Andrew Morton Cc: Christian Borntraeger , Janosch Frank , Claudio Imbrenda , David Hildenbrand , Alexander Gordeev , Gerald Schaefer , Heiko Carstens , Vasily Gorbik , Sven Schnelle , Peter Xu , Alexander Viro , Christian Brauner , Jan Kara , Arnd Bergmann , Zi Yan , Baolin Wang , "Liam R . Howlett" , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Lance Yang , Muchun Song , Oscar Salvador , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Matthew Brost , Joshua Hahn , Rakie Kim , Byungchul Park , Gregory Price , Ying Huang , Alistair Popple , Axel Rasmussen , Yuanchu Xie , Wei Xu , Kemeng Shi , Kairui Song , Nhat Pham , Baoquan He , Chris Li , SeongJae Park , Matthew Wilcox , Jason Gunthorpe , Leon Romanovsky , Xu Xin , Chengming Zhou , Jann Horn , Miaohe Lin , Naoya Horiguchi , Pedro Falcato , Pasha Tatashin , Rik van Riel , Harry Yoo , Hugh Dickins , linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-s390@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, linux-arch@vger.kernel.org, damon@lists.linux.dev Subject: [PATCH 03/16] mm: avoid unnecessary uses of is_swap_pte() Date: Mon, 3 Nov 2025 12:31:44 +0000 Message-ID: <8d93c62760e501cb7badd8cceb597a5af49d700c.1762171281.git.lorenzo.stoakes@oracle.com> X-Mailer: git-send-email 2.51.0 In-Reply-To: References: Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: LO4P123CA0051.GBRP123.PROD.OUTLOOK.COM (2603:10a6:600:152::20) To DM4PR10MB8218.namprd10.prod.outlook.com (2603:10b6:8:1cc::16) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DM4PR10MB8218:EE_|DS4PR10MB997575:EE_ X-MS-Office365-Filtering-Correlation-Id: 5932bb4b-9275-41ac-49a7-08de1ad50670 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|376014|7416014|366016|1800799024; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?5lYROkUco8yFxRtFssXGN5wOcdKYJRDq719oI9lcCcWe7dexUkIzeIH16ajz?= =?us-ascii?Q?ARlWVI+e0Uryty23f/KhyacW8wnOcF6AFPfikydW7sp7jDWptjdv35GLCxWz?= =?us-ascii?Q?erhF94RMtDQUHejoESZ7Ld95lqPyUfEKwyZ1dxMj3575sww+kg4DYz7HwDft?= =?us-ascii?Q?+g+p8jWJjrXTUnsZGxeE70vDsSRAQa3HOnByiFskPCkIymm6nGTXBly+2wHh?= =?us-ascii?Q?+02+viDVAaW43WEDKpnFfoyAQO+pONPgtBwdshvKW1SJvGQtKrZMX4fJLvrw?= =?us-ascii?Q?4WzOjd+ENSeqYbQX1iUUOAbOmshldSJB3aGGS9ewYs+gtvrLtBz6BOAB1l4h?= =?us-ascii?Q?Bq8CTJ5fcXDumH7p5S13Wwp9HxxvNV2wt6Qmz7GO8BNV7OW+I9BmcH5n9J5Z?= =?us-ascii?Q?neyBInJ4WUsiJWyq2nHIgwNgOyIVZ7gmyRFsxCmTsS6xSdQOjxR8cPbiItOL?= =?us-ascii?Q?WImEXN0YJRYJNIyGgzypZygykVg24Gl8e1iob9kg4H1xGhRaQ2vuviS9ICmf?= =?us-ascii?Q?pkpfh52hdgLZ6WXxbFy69L99XUvfAcEwmkGOuZbbgCBvyCPiTIWT+6vInp0D?= =?us-ascii?Q?58uVvFVJndyWTFbHcirE7ciZsrRB2JALrqWILA6eWW7Dm2pkCEQ0GxJVU/+I?= =?us-ascii?Q?yUdGqm+soiP0T722diRo/KwsAa5qA/Ai0UoGOuSwGL0HW/TfwUfbav8MNXn4?= =?us-ascii?Q?KTg9pK78140ShMCvkXRiRw+pSgX+XjfFgnNM8MyhEPBXHbiSqX8YZG4Cb0qB?= =?us-ascii?Q?wG7AUTmz6oqaCnFL36QsJgOSYxhCtQQclRtRlurcwfz48jNk9gkl9BfasFgm?= =?us-ascii?Q?GwapuN8FxnfBdnJhjIxduW6Gh47b029ItjEMuStxEwIY3tjRrEfgHzx0TQ7f?= =?us-ascii?Q?S0S5Wa6yVcq8PYVWPpjVm+gVktITif3OBPkHjh3copua/9pvloKN1HLL5QqR?= =?us-ascii?Q?bGvv7D0CNbpHF6lgf5BPDOzhvdHOlPRUf09Cw5TbTywVaoi562gSUQs7X7ZG?= =?us-ascii?Q?Kn2iAA4CQ6DzSTwq+zgQcZnEYaU/f38jwSMidktj6q+f4341cLUm1rsyr+SJ?= =?us-ascii?Q?RDFHFDXWIx2Z7qhvg56wqyOg3sO1hdlhA+SkBJmeYCcCWiiK4ScWvnRi0mzr?= =?us-ascii?Q?BpRoTaOIJPWqYS6mBzbBZkKeVy2TsYqozVEUfKmxJDus9mPo2CsLscn2vWX2?= =?us-ascii?Q?T+Jss4QtlQCE9fq4uuE1TdfuyTZ9G3echIBfxbHcL/8XYCT11iKU/Li5oC6w?= =?us-ascii?Q?bsMfdJKCMjwB4r3Vtq6HT5fpX1FFpsITGxlv5IixF+nW+r9y/C4E1MoPV0GO?= =?us-ascii?Q?5PlPRf36WZs98HRn0QgaC/ZI53ugcyNHgCfdLFV4WF+aVlTlZw9XUV7P4Rnp?= =?us-ascii?Q?30ectoSifdcED1doVhVl4i5eQL21RxBVhdw4h+STWmdo7XU+JxmLU2nsK6Nl?= =?us-ascii?Q?8x99PZ1KCHRkWf8g1d5tbdx3ChLS1FrF?= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DM4PR10MB8218.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(376014)(7416014)(366016)(1800799024);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?kNPsTvD8LyyIw3YQDsJAudbwaQoF3W2aoiypJGikT3fd82ehj+du3Ruk5iL8?= =?us-ascii?Q?cEFOewi6ha0lxjRUzXDelgAueZhd5HvtRHhRpHim7+n0YeKGET3MSM/IAEa6?= =?us-ascii?Q?ZFcVIYYPyGPZZco6EUt9rQMtiNHmP68CSwyMFhYlgswFT9yHWyRAnDf2/T2i?= =?us-ascii?Q?qEGJk+h+Os8HIp4QHNo4aQp29p7EkIUiZ5Uy8IyyOLOR21AY+Xkk2Fvur0k4?= =?us-ascii?Q?YaN34UvpfYas2fKsZdXEF00d3CvX65mAVO0hlEbd5N+MCBMbKJ0xuTJ5AJPQ?= =?us-ascii?Q?Mw34sy/9yJu0/PCD/AzainvzlHYUQMrMnAf6NoEv//JFjfaC/lT6YAc0GVfX?= =?us-ascii?Q?OMbPNwYzvhAOcFP5xGwoPFLaZIjnaDqclMxzi071VJCpJjxQTeXQWvbJwqHu?= =?us-ascii?Q?X2lsDffXC2nrP2FDcx87E5rSVC9gyRXMe5K7PFKenZYGelCO+CI9jfafGhGq?= =?us-ascii?Q?JrQXH9E1xTs3iyF1wfAqhKxk1y6XtQZN6aMG03OzfSr7R4lNiDkx9IjOS/Nn?= =?us-ascii?Q?T9IZrC1qWRHAtpJvhcQgpCHaRLKUlyUhyHgZ9arG9jrUHuPyiia1K4hM1CYF?= =?us-ascii?Q?ahh69tdqhzN2twffieIuK/dpW/mU5pnLeJs/H0mY+UU82xfy2q7pEPlxjy7s?= =?us-ascii?Q?1o09OzlPDZ3iU6Wjs3VAQtbtK2ocM5gH7dSpIwa7Jes3Q6gKiAlymIb7OzeE?= =?us-ascii?Q?6UiTj9hs8ctq42VUqL3NJGKpopLYDQDJX68AVArmEo04G67RfqZWYrtWMeI+?= =?us-ascii?Q?o/4HE6vZJRaZqDZ76DiimEjJikoRKUn+tcaIQG9t/5JzoCvCIPXzYsKfig3a?= =?us-ascii?Q?HI3pBlCQbFi1gs2RVIPTrP1OKnST0xpUhbTaGJjztAd2GL3Vu0A6CwcRPCl0?= =?us-ascii?Q?buBBkN2v5oVmFXUxSHnLavqj0DIkK0bYYsn8m2C6NmPLhL5KhAag2G8EbXJ2?= =?us-ascii?Q?tvMFuPReigENUXE54YHVJm3Nu5/CEap77S/Cw1F4bfqqYs/puXfqaD/q3rYB?= =?us-ascii?Q?ftFGI1rudZi03SWEVfStKkBYUFpoI06lc7+Zr3hwfOtG7CFaT/vFdNfFGmYy?= =?us-ascii?Q?HxsFpT8GcE8YItLIINwIJmzPIJew3BI0tADHzlhf8Zhn91LHE7Pumhv6DMH0?= =?us-ascii?Q?tWXieVIXlFHi3oQ96YcLUyjovk9Vo7PwuvaA7Ax9vRt3zC+NpIF3P410QeCg?= =?us-ascii?Q?wNdZrVaxClINsdAwLyE6iAROLmjR9KKl4TatRyHH3Yb7qFxD/iOFeS2WIyfe?= =?us-ascii?Q?iAqfP4h0JrFZDr1BoXhujpPJX1uVWPGyhNG6sPJvkGvUO5AgaIgSLkKpkGSm?= =?us-ascii?Q?kk25wlzDKiAfur7a/Cc+8R8SahGXjn/4ocBNNQpE92Yi3sWrlPxCjS8/ys12?= =?us-ascii?Q?I+TJBM15WN2p6zbSDaSxRCmmjKN0H+I/ncGW37zFhJJ14yCH5UrmhnR+yxrn?= =?us-ascii?Q?jrN1KzDRci8+3mvQuOc2cWpWyHEeFT96IpUg1dS04GjyOB+Um/hJ8VH4ruB1?= =?us-ascii?Q?C7qFt/mTA3TrhCLQ1ZA8lO9Q3OzgpHJRAo0GFZMP3wvJF1aQkrYF72dr0d9V?= =?us-ascii?Q?wuCpsao3MPFp3tK3Eo3mO5m5DzsURybtGyVfeO+ZiWg+fadnpAUh9iSgW7w1?= =?us-ascii?Q?0g=3D=3D?= X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: /V2hB3dEkr9Is/B1+o/da7G3pf+ZpguWcNhU1BxqviotP5tiW4qIOOl/pStIBiCy4g/QJaJnFr5/noqpHxSHf/vnv8EdjHPg/CPnXXnF65BUOrfb8u4IbeKTDYA1c4ShlhSfdRvSUnPDn4sNsz4HlXVbrqnmSBhbjKnIwWfVRMhnvhg34skaH1pmEy5dg7NDB6fd4iEWlfXpL3wrGWiSmJawBHNJ5HT9/TeQer5lqDjilnUDpukDNYbJCJb7CBkluzhLHXVPl4nyBohkzYa/NT2c/Un8u71zHLBlunaDZN1MFi4RhOkYRbs6e0GlF5nGhyy7H9Eio05JuTBgfrsG86G/gWkQXgpZ9N3SfMoa9kMKvHpN/A+oMkV+YdcQ50cP+2IYwXveua+rr0S8YRXEvScjy13MkI312WeofUsDCZWI+5njK5t9n49hAn1jn8AxWFmEMaFsB/QPcUIq2hvVg5sotZpoy2sPTC6Ii3aKGJTSwEB9jbmWD9chCmBl9mUzkbQwSKQNnbalmzlovFyAAzx5TBgN9Fd498KV2H4zcfxt0qLurrPJLsRaq8gxKFTqPc6jJ0BDDLsTFM43xy0eD9428LX5809w5WhfRvkAb1E= X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: 5932bb4b-9275-41ac-49a7-08de1ad50670 X-MS-Exchange-CrossTenant-AuthSource: DM4PR10MB8218.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 03 Nov 2025 12:32:17.3305 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: usINuvg4WC69rK3qLBEixbhYAdPwqbW/WRH7CLgdqg9F29vbD7W2FIaDvkwGHOSm77KoxFUQ9BLAMuAdajJZw3rJMeqm/X+Xsije0Bj0VqI= X-MS-Exchange-Transport-CrossTenantHeadersStamped: DS4PR10MB997575 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1121,Hydra:6.1.9,FMLib:17.12.100.49 definitions=2025-11-03_01,2025-11-03_01,2025-10-01_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 phishscore=0 mlxscore=0 bulkscore=0 malwarescore=0 adultscore=0 spamscore=0 mlxlogscore=999 suspectscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2510240000 definitions=main-2511030113 X-Proofpoint-GUID: 2Rw1G7RpxDI7BJ45vy9BJ3zFUqIDdHQF X-Authority-Analysis: v=2.4 cv=PPsCOPqC c=1 sm=1 tr=0 ts=6908a0d4 b=1 cx=c_pps a=e1sVV491RgrpLwSTMOnk8w==:117 a=e1sVV491RgrpLwSTMOnk8w==:17 a=6eWqkTHjU83fiwn7nKZWdM+Sl24=:19 a=z/mQ4Ysz8XfWz/Q5cLBRGdckG28=:19 a=lCpzRmAYbLLaTzLvsPZ7Mbvzbb8=:19 a=xqWC_Br6kY4A:10 a=6UeiqGixMTsA:10 a=GoEa3M9JfhUA:10 a=VkNPw1HP01LnGYTKEx00:22 a=yPCof4ZbAAAA:8 a=360PBzQsUBXz8B3xgqMA:9 cc=ntf awl=host:13657 X-Proofpoint-ORIG-GUID: 2Rw1G7RpxDI7BJ45vy9BJ3zFUqIDdHQF X-Proofpoint-Spam-Details-Enc: AW1haW4tMjUxMTAzMDExMyBTYWx0ZWRfXzZcIZVRhmXM7 rPLtBaAn+6g2Kpjmhb1IiqXps42WH8epS5HOY1j0bST79uewqrDY27eulNg1hHcsA8WKEmlKcnT l9MHTzyGD7J0e1dIk2I8tH/9DOfDHPsLK0wTgEqCJMZLIqaeToa3HyiQPBMy2SF0qZLzQ7G6NJ2 JXH6GPb8FO+QzK+DI6R5aF1FeXxlrcquzXclsP2GwWNGz6mPMbz++zcubOd5dg6ohxDGbp9oznm I67a3PAQFV6sS746niMqr3RRGb3QXXhFw6lvoYuRyATTXp01arfV0B7tSwrUl79S6BzGIqA6ito KhqBxa4D8MMB4Fvb89vlUMgssbnGmubiMLtne+YPUz3L1vccRxsASs3Vgv2tDwHB09AfMhhHajt h/6ar0Am97b5bsXoiGgEjdYZygaRT85+B7NlEY5mmP9nntM9ATc= Content-Type: text/plain; charset="utf-8" There's an established convention in the kernel that we treat PTEs as containing swap entries (and the unfortunately named non-swap swap entries) should they be neither empty (i.e. pte_none() evaluating true) nor present (i.e. pte_present() evaluating true). However, there is some inconsistency in how this is applied, as we also have the is_swap_pte() helper which explicitly performs this check: /* check whether a pte points to a swap entry */ static inline int is_swap_pte(pte_t pte) { return !pte_none(pte) && !pte_present(pte); } As this represents a predicate, and it's logical to assume that in order to establish that a PTE entry can correctly be manipulated as a swap/non-swap entry, this predicate seems as if it must first be checked. But we instead, we far more often utilise the established convention of checking pte_none() / pte_present() before operating on entries as if they were swap/non-swap. This patch works towards correcting this inconsistency by removing all uses of is_swap_pte() where we are already in a position where we perform pte_none()/pte_present() checks anyway or otherwise it is clearly logical to do so. We also take advantage of the fact that pte_swp_uffd_wp() is only set on swap entries. Additionally, update comments referencing to is_swap_pte() and non_swap_entry(). No functional change intended. Signed-off-by: Lorenzo Stoakes --- fs/proc/task_mmu.c | 49 ++++++++++++++++++++++++----------- include/linux/userfaultfd_k.h | 3 +-- mm/hugetlb.c | 6 ++--- mm/internal.h | 6 ++--- mm/khugepaged.c | 29 +++++++++++---------- mm/migrate.c | 2 +- mm/mprotect.c | 43 ++++++++++++++---------------- mm/mremap.c | 7 +++-- mm/page_table_check.c | 13 ++++++---- mm/page_vma_mapped.c | 31 +++++++++++----------- 10 files changed, 104 insertions(+), 85 deletions(-) diff --git a/fs/proc/task_mmu.c b/fs/proc/task_mmu.c index adac6c42749d..9914febdb60b 100644 --- a/fs/proc/task_mmu.c +++ b/fs/proc/task_mmu.c @@ -1017,7 +1017,9 @@ static void smaps_pte_entry(pte_t *pte, unsigned long= addr, young =3D pte_young(ptent); dirty =3D pte_dirty(ptent); present =3D true; - } else if (is_swap_pte(ptent)) { + } else if (pte_none(ptent)) { + smaps_pte_hole_lookup(addr, walk); + } else { swp_entry_t swpent =3D pte_to_swp_entry(ptent); =20 if (!non_swap_entry(swpent)) { @@ -1038,9 +1040,6 @@ static void smaps_pte_entry(pte_t *pte, unsigned long= addr, present =3D true; page =3D pfn_swap_entry_to_page(swpent); } - } else { - smaps_pte_hole_lookup(addr, walk); - return; } =20 if (!page) @@ -1611,6 +1610,9 @@ static inline void clear_soft_dirty(struct vm_area_st= ruct *vma, */ pte_t ptent =3D ptep_get(pte); =20 + if (pte_none(ptent)) + return; + if (pte_present(ptent)) { pte_t old_pte; =20 @@ -1620,7 +1622,7 @@ static inline void clear_soft_dirty(struct vm_area_st= ruct *vma, ptent =3D pte_wrprotect(old_pte); ptent =3D pte_clear_soft_dirty(ptent); ptep_modify_prot_commit(vma, addr, pte, old_pte, ptent); - } else if (is_swap_pte(ptent)) { + } else { ptent =3D pte_swp_clear_soft_dirty(ptent); set_pte_at(vma->vm_mm, addr, pte, ptent); } @@ -1923,6 +1925,9 @@ static pagemap_entry_t pte_to_pagemap_entry(struct pa= gemapread *pm, struct page *page =3D NULL; struct folio *folio; =20 + if (pte_none(pte)) + goto out; + if (pte_present(pte)) { if (pm->show_pfn) frame =3D pte_pfn(pte); @@ -1932,8 +1937,9 @@ static pagemap_entry_t pte_to_pagemap_entry(struct pa= gemapread *pm, flags |=3D PM_SOFT_DIRTY; if (pte_uffd_wp(pte)) flags |=3D PM_UFFD_WP; - } else if (is_swap_pte(pte)) { + } else { swp_entry_t entry; + if (pte_swp_soft_dirty(pte)) flags |=3D PM_SOFT_DIRTY; if (pte_swp_uffd_wp(pte)) @@ -1941,6 +1947,7 @@ static pagemap_entry_t pte_to_pagemap_entry(struct pa= gemapread *pm, entry =3D pte_to_swp_entry(pte); if (pm->show_pfn) { pgoff_t offset; + /* * For PFN swap offsets, keeping the offset field * to be PFN only to be compatible with old smaps. @@ -1969,6 +1976,8 @@ static pagemap_entry_t pte_to_pagemap_entry(struct pa= gemapread *pm, __folio_page_mapped_exclusively(folio, page)) flags |=3D PM_MMAP_EXCLUSIVE; } + +out: if (vma->vm_flags & VM_SOFTDIRTY) flags |=3D PM_SOFT_DIRTY; =20 @@ -2310,12 +2319,16 @@ static unsigned long pagemap_page_category(struct p= agemap_scan_private *p, struct vm_area_struct *vma, unsigned long addr, pte_t pte) { - unsigned long categories =3D 0; + unsigned long categories; + + if (pte_none(pte)) + return 0; =20 if (pte_present(pte)) { struct page *page; =20 - categories |=3D PAGE_IS_PRESENT; + categories =3D PAGE_IS_PRESENT; + if (!pte_uffd_wp(pte)) categories |=3D PAGE_IS_WRITTEN; =20 @@ -2329,10 +2342,11 @@ static unsigned long pagemap_page_category(struct p= agemap_scan_private *p, categories |=3D PAGE_IS_PFNZERO; if (pte_soft_dirty(pte)) categories |=3D PAGE_IS_SOFT_DIRTY; - } else if (is_swap_pte(pte)) { + } else { leaf_entry_t entry; =20 - categories |=3D PAGE_IS_SWAPPED; + categories =3D PAGE_IS_SWAPPED; + if (!pte_swp_uffd_wp_any(pte)) categories |=3D PAGE_IS_WRITTEN; =20 @@ -2360,12 +2374,12 @@ static void make_uffd_wp_pte(struct vm_area_struct = *vma, old_pte =3D ptep_modify_prot_start(vma, addr, pte); ptent =3D pte_mkuffd_wp(old_pte); ptep_modify_prot_commit(vma, addr, pte, old_pte, ptent); - } else if (is_swap_pte(ptent)) { - ptent =3D pte_swp_mkuffd_wp(ptent); - set_pte_at(vma->vm_mm, addr, pte, ptent); - } else { + } else if (pte_none(ptent)) { set_pte_at(vma->vm_mm, addr, pte, make_pte_marker(PTE_MARKER_UFFD_WP)); + } else { + ptent =3D pte_swp_mkuffd_wp(ptent); + set_pte_at(vma->vm_mm, addr, pte, ptent); } } =20 @@ -2434,6 +2448,9 @@ static unsigned long pagemap_hugetlb_category(pte_t p= te) { unsigned long categories =3D PAGE_IS_HUGE; =20 + if (pte_none(pte)) + return categories; + /* * According to pagemap_hugetlb_range(), file-backed HugeTLB * page cannot be swapped. So PAGE_IS_FILE is not checked for @@ -2441,6 +2458,7 @@ static unsigned long pagemap_hugetlb_category(pte_t p= te) */ if (pte_present(pte)) { categories |=3D PAGE_IS_PRESENT; + if (!huge_pte_uffd_wp(pte)) categories |=3D PAGE_IS_WRITTEN; if (!PageAnon(pte_page(pte))) @@ -2449,8 +2467,9 @@ static unsigned long pagemap_hugetlb_category(pte_t p= te) categories |=3D PAGE_IS_PFNZERO; if (pte_soft_dirty(pte)) categories |=3D PAGE_IS_SOFT_DIRTY; - } else if (is_swap_pte(pte)) { + } else { categories |=3D PAGE_IS_SWAPPED; + if (!pte_swp_uffd_wp_any(pte)) categories |=3D PAGE_IS_WRITTEN; if (pte_swp_soft_dirty(pte)) diff --git a/include/linux/userfaultfd_k.h b/include/linux/userfaultfd_k.h index 983c860a00f1..96b089dff4ef 100644 --- a/include/linux/userfaultfd_k.h +++ b/include/linux/userfaultfd_k.h @@ -441,9 +441,8 @@ static inline bool userfaultfd_wp_use_markers(struct vm= _area_struct *vma) static inline bool pte_swp_uffd_wp_any(pte_t pte) { #ifdef CONFIG_PTE_MARKER_UFFD_WP - if (!is_swap_pte(pte)) + if (pte_present(pte)) return false; - if (pte_swp_uffd_wp(pte)) return true; =20 diff --git a/mm/hugetlb.c b/mm/hugetlb.c index cc7db8fd86db..2e797abdee04 100644 --- a/mm/hugetlb.c +++ b/mm/hugetlb.c @@ -5798,13 +5798,13 @@ static void move_huge_pte(struct vm_area_struct *vm= a, unsigned long old_addr, =20 pte =3D huge_ptep_get_and_clear(mm, old_addr, src_pte, sz); =20 - if (need_clear_uffd_wp && pte_is_uffd_wp_marker(pte)) + if (need_clear_uffd_wp && pte_is_uffd_wp_marker(pte)) { huge_pte_clear(mm, new_addr, dst_pte, sz); - else { + } else { if (need_clear_uffd_wp) { if (pte_present(pte)) pte =3D huge_pte_clear_uffd_wp(pte); - else if (is_swap_pte(pte)) + else pte =3D pte_swp_clear_uffd_wp(pte); } set_huge_pte_at(mm, new_addr, dst_pte, pte, sz); diff --git a/mm/internal.h b/mm/internal.h index 116a1ba85e66..9465129367a4 100644 --- a/mm/internal.h +++ b/mm/internal.h @@ -325,8 +325,7 @@ unsigned int folio_pte_batch(struct folio *folio, pte_t= *ptep, pte_t pte, /** * pte_move_swp_offset - Move the swap entry offset field of a swap pte * forward or backward by delta - * @pte: The initial pte state; is_swap_pte(pte) must be true and - * non_swap_entry() must be false. + * @pte: The initial pte state; must be a swap entry * @delta: The direction and the offset we are moving; forward if delta * is positive; backward if delta is negative * @@ -352,8 +351,7 @@ static inline pte_t pte_move_swp_offset(pte_t pte, long= delta) =20 /** * pte_next_swp_offset - Increment the swap entry offset field of a swap p= te. - * @pte: The initial pte state; is_swap_pte(pte) must be true and - * non_swap_entry() must be false. + * @pte: The initial pte state; must be a swap entry. * * Increments the swap offset, while maintaining all other fields, includi= ng * swap type, and any swp pte bits. The resulting pte is returned. diff --git a/mm/khugepaged.c b/mm/khugepaged.c index f6ed1072ed6e..a97ff7bcb232 100644 --- a/mm/khugepaged.c +++ b/mm/khugepaged.c @@ -1019,7 +1019,8 @@ static int __collapse_huge_page_swapin(struct mm_stru= ct *mm, } =20 vmf.orig_pte =3D ptep_get_lockless(pte); - if (!is_swap_pte(vmf.orig_pte)) + if (pte_none(vmf.orig_pte) || + pte_present(vmf.orig_pte)) continue; =20 vmf.pte =3D pte; @@ -1276,7 +1277,19 @@ static int hpage_collapse_scan_pmd(struct mm_struct = *mm, for (addr =3D start_addr, _pte =3D pte; _pte < pte + HPAGE_PMD_NR; _pte++, addr +=3D PAGE_SIZE) { pte_t pteval =3D ptep_get(_pte); - if (is_swap_pte(pteval)) { + if (pte_none_or_zero(pteval)) { + ++none_or_zero; + if (!userfaultfd_armed(vma) && + (!cc->is_khugepaged || + none_or_zero <=3D khugepaged_max_ptes_none)) { + continue; + } else { + result =3D SCAN_EXCEED_NONE_PTE; + count_vm_event(THP_SCAN_EXCEED_NONE_PTE); + goto out_unmap; + } + } + if (!pte_present(pteval)) { ++unmapped; if (!cc->is_khugepaged || unmapped <=3D khugepaged_max_ptes_swap) { @@ -1296,18 +1309,6 @@ static int hpage_collapse_scan_pmd(struct mm_struct = *mm, goto out_unmap; } } - if (pte_none_or_zero(pteval)) { - ++none_or_zero; - if (!userfaultfd_armed(vma) && - (!cc->is_khugepaged || - none_or_zero <=3D khugepaged_max_ptes_none)) { - continue; - } else { - result =3D SCAN_EXCEED_NONE_PTE; - count_vm_event(THP_SCAN_EXCEED_NONE_PTE); - goto out_unmap; - } - } if (pte_uffd_wp(pteval)) { /* * Don't collapse the page if any of the small diff --git a/mm/migrate.c b/mm/migrate.c index ceee354ef215..862b2e261cf9 100644 --- a/mm/migrate.c +++ b/mm/migrate.c @@ -492,7 +492,7 @@ void migration_entry_wait(struct mm_struct *mm, pmd_t *= pmd, pte =3D ptep_get(ptep); pte_unmap(ptep); =20 - if (!is_swap_pte(pte)) + if (pte_none(pte) || pte_present(pte)) goto out; =20 entry =3D pte_to_swp_entry(pte); diff --git a/mm/mprotect.c b/mm/mprotect.c index d425be97db51..ac2cd613f76e 100644 --- a/mm/mprotect.c +++ b/mm/mprotect.c @@ -297,7 +297,26 @@ static long change_pte_range(struct mmu_gather *tlb, prot_commit_flush_ptes(vma, addr, pte, oldpte, ptent, nr_ptes, /* idx =3D */ 0, /* set_write =3D */ false, tlb); pages +=3D nr_ptes; - } else if (is_swap_pte(oldpte)) { + } else if (pte_none(oldpte)) { + /* + * Nobody plays with any none ptes besides + * userfaultfd when applying the protections. + */ + if (likely(!uffd_wp)) + continue; + + if (userfaultfd_wp_use_markers(vma)) { + /* + * For file-backed mem, we need to be able to + * wr-protect a none pte, because even if the + * pte is none, the page/swap cache could + * exist. Doing that by install a marker. + */ + set_pte_at(vma->vm_mm, addr, pte, + make_pte_marker(PTE_MARKER_UFFD_WP)); + pages++; + } + } else { swp_entry_t entry =3D pte_to_swp_entry(oldpte); pte_t newpte; =20 @@ -358,28 +377,6 @@ static long change_pte_range(struct mmu_gather *tlb, set_pte_at(vma->vm_mm, addr, pte, newpte); pages++; } - } else { - /* It must be an none page, or what else?.. */ - WARN_ON_ONCE(!pte_none(oldpte)); - - /* - * Nobody plays with any none ptes besides - * userfaultfd when applying the protections. - */ - if (likely(!uffd_wp)) - continue; - - if (userfaultfd_wp_use_markers(vma)) { - /* - * For file-backed mem, we need to be able to - * wr-protect a none pte, because even if the - * pte is none, the page/swap cache could - * exist. Doing that by install a marker. - */ - set_pte_at(vma->vm_mm, addr, pte, - make_pte_marker(PTE_MARKER_UFFD_WP)); - pages++; - } } } while (pte +=3D nr_ptes, addr +=3D nr_ptes * PAGE_SIZE, addr !=3D end); arch_leave_lazy_mmu_mode(); diff --git a/mm/mremap.c b/mm/mremap.c index 7c21b2ad13f6..62b6827abacf 100644 --- a/mm/mremap.c +++ b/mm/mremap.c @@ -158,6 +158,9 @@ static void drop_rmap_locks(struct vm_area_struct *vma) =20 static pte_t move_soft_dirty_pte(pte_t pte) { + if (pte_none(pte)) + return pte; + /* * Set soft dirty bit so we can notice * in userspace the ptes were moved. @@ -165,7 +168,7 @@ static pte_t move_soft_dirty_pte(pte_t pte) #ifdef CONFIG_MEM_SOFT_DIRTY if (pte_present(pte)) pte =3D pte_mksoft_dirty(pte); - else if (is_swap_pte(pte)) + else pte =3D pte_swp_mksoft_dirty(pte); #endif return pte; @@ -294,7 +297,7 @@ static int move_ptes(struct pagetable_move_control *pmc, if (need_clear_uffd_wp) { if (pte_present(pte)) pte =3D pte_clear_uffd_wp(pte); - else if (is_swap_pte(pte)) + else pte =3D pte_swp_clear_uffd_wp(pte); } set_ptes(mm, new_addr, new_ptep, pte, nr_ptes); diff --git a/mm/page_table_check.c b/mm/page_table_check.c index 4eeca782b888..43f75d2f7c36 100644 --- a/mm/page_table_check.c +++ b/mm/page_table_check.c @@ -185,12 +185,15 @@ static inline bool swap_cached_writable(swp_entry_t e= ntry) is_writable_migration_entry(entry); } =20 -static inline void page_table_check_pte_flags(pte_t pte) +static void page_table_check_pte_flags(pte_t pte) { - if (pte_present(pte) && pte_uffd_wp(pte)) - WARN_ON_ONCE(pte_write(pte)); - else if (is_swap_pte(pte) && pte_swp_uffd_wp(pte)) - WARN_ON_ONCE(swap_cached_writable(pte_to_swp_entry(pte))); + if (pte_present(pte)) { + WARN_ON_ONCE(pte_uffd_wp(pte) && pte_write(pte)); + } else if (pte_swp_uffd_wp(pte)) { + const swp_entry_t entry =3D pte_to_swp_entry(pte); + + WARN_ON_ONCE(swap_cached_writable(entry)); + } } =20 void __page_table_check_ptes_set(struct mm_struct *mm, pte_t *ptep, pte_t = pte, diff --git a/mm/page_vma_mapped.c b/mm/page_vma_mapped.c index e0560cc1ce18..4597a281356d 100644 --- a/mm/page_vma_mapped.c +++ b/mm/page_vma_mapped.c @@ -16,6 +16,7 @@ static inline bool not_found(struct page_vma_mapped_walk = *pvmw) static bool map_pte(struct page_vma_mapped_walk *pvmw, pmd_t *pmdvalp, spinlock_t **ptlp) { + bool is_migration; pte_t ptent; =20 if (pvmw->flags & PVMW_SYNC) { @@ -26,6 +27,7 @@ static bool map_pte(struct page_vma_mapped_walk *pvmw, pm= d_t *pmdvalp, return !!pvmw->pte; } =20 + is_migration =3D pvmw->flags & PVMW_MIGRATION; again: /* * It is important to return the ptl corresponding to pte, @@ -41,11 +43,14 @@ static bool map_pte(struct page_vma_mapped_walk *pvmw, = pmd_t *pmdvalp, =20 ptent =3D ptep_get(pvmw->pte); =20 - if (pvmw->flags & PVMW_MIGRATION) { - if (!is_swap_pte(ptent)) + if (pte_none(ptent)) { + return false; + } else if (pte_present(ptent)) { + if (is_migration) return false; - } else if (is_swap_pte(ptent)) { + } else if (!is_migration) { swp_entry_t entry; + /* * Handle un-addressable ZONE_DEVICE memory. * @@ -66,8 +71,6 @@ static bool map_pte(struct page_vma_mapped_walk *pvmw, pm= d_t *pmdvalp, if (!is_device_private_entry(entry) && !is_device_exclusive_entry(entry)) return false; - } else if (!pte_present(ptent)) { - return false; } spin_lock(*ptlp); if (unlikely(!pmd_same(*pmdvalp, pmdp_get_lockless(pvmw->pmd)))) { @@ -113,21 +116,17 @@ static bool check_pte(struct page_vma_mapped_walk *pv= mw, unsigned long pte_nr) return false; =20 pfn =3D leafent_to_pfn(entry); - } else if (is_swap_pte(ptent)) { - swp_entry_t entry; + } else if (pte_present(ptent)) { + pfn =3D pte_pfn(ptent); + } else { + const leaf_entry_t entry =3D leafent_from_pte(ptent); =20 /* Handle un-addressable ZONE_DEVICE memory */ - entry =3D pte_to_swp_entry(ptent); - if (!is_device_private_entry(entry) && - !is_device_exclusive_entry(entry)) - return false; - - pfn =3D swp_offset_pfn(entry); - } else { - if (!pte_present(ptent)) + if (!leafent_is_device_private(entry) && + !leafent_is_device_exclusive(entry)) return false; =20 - pfn =3D pte_pfn(ptent); + pfn =3D leafent_to_pfn(entry); } =20 if ((pfn + pte_nr - 1) < pvmw->pfn) --=20 2.51.0 From nobody Sat Feb 7 17:55:59 2026 Received: from mx0b-00069f02.pphosted.com (mx0b-00069f02.pphosted.com [205.220.177.32]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8D7E530F807; Mon, 3 Nov 2025 12:34:11 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=205.220.177.32 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1762173253; cv=fail; b=mYgDPhoHx/iUAMmim0A7W5vbTKNriRd2WtnzIKEk6MaAmkc4OpKBpLKCgyDiyjGuXyM09r83UK3oFTqPgXnT11/ObVa3MFI+yfE0QijD75St5nU3x5j/+4QqUu6hv/v8Q0YJhvfO+x+BZXO/Alc9ete/iaV/ZCccwCdbBmP6oR0= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1762173253; c=relaxed/simple; bh=Qo6daWNbBRlUhxp8OkqMSYXUjCxLzJ8bLqEynR0eIgE=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: Content-Type:MIME-Version; b=aUa05jwavaYIB8etj71v3M0bnrCT8dKVXf2nQA1OvBbrA42xwJmLBh4I2pOWA2sxDt5Pc1qaNblkBoLmD7LDMSLhWuqbGKJYs5I4NSpLOq8BCM7Wtn97oEfoCZFVBiPyMvWd9cnTwY0zayDaE/j0MirjQAUW3FEs7DcchbcM3Do= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=oracle.com; spf=pass smtp.mailfrom=oracle.com; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b=JTNV4tID; dkim=pass (1024-bit key) header.d=oracle.onmicrosoft.com header.i=@oracle.onmicrosoft.com header.b=IveFWjeu; arc=fail smtp.client-ip=205.220.177.32 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=oracle.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=oracle.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b="JTNV4tID"; dkim=pass (1024-bit key) header.d=oracle.onmicrosoft.com header.i=@oracle.onmicrosoft.com header.b="IveFWjeu" Received: from pps.filterd (m0333520.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 5A3CFDNc018204; Mon, 3 Nov 2025 12:32:25 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=cc :content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s= corp-2025-04-25; bh=Xb2WuoV1n9d7enOgQK2mVmgEDn5DmGN/bpAYSAaohe0=; b= JTNV4tIDc2J8RwGFKFb1abGOV8q4oUOfDiTjTpBCUFldKC/pzRSGFCW1CVslpkD1 GELjye8GKLzg4zf9IESYx+rBNAz2n9cux/I8b8DbUAjnbgMgpbBfkoyq2Ht2rMey MvIrGGrxw4wTNEu5lSgNHA0+cvWnh664BYRl22G1OLwZNgZ0+kJxMaYjpw81W0IH FW+/e9dEyy2IZEudyOLI9HsBVuWTDvGgebwzAzG6Zmy+7lzT7vSQXVK9c9U9djg7 ZNBNX4jn6UrYRzaG6U09fcuAH2CSsQBiBYSivHPIke0ScjpJEjawDOwe8qUyoBLm KeKZnEJT2SKRwyYdG1eXVQ== Received: from iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta03.appoci.oracle.com [130.35.103.27]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 4a6v7kg0wy-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 03 Nov 2025 12:32:24 +0000 (GMT) Received: from pps.filterd (iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com (8.18.1.2/8.18.1.2) with ESMTP id 5A3BVq3h039882; Mon, 3 Nov 2025 12:32:24 GMT Received: from co1pr03cu002.outbound.protection.outlook.com (mail-westus2azon11010026.outbound.protection.outlook.com [52.101.46.26]) by iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTPS id 4a58nhux3q-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 03 Nov 2025 12:32:23 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=S+b5Ewn71jlPbkv87ULG1XeE2wkItt9qP7MxJoxEHx6sYw0ebWItILuLSFOMyunP2Rm6Q8GTVWQ8DzGfmiWxnQlA7gxBSTf12QxEQDLMaJgrFNQ8b9pWnddPle2DAf8eRAkCCygguIf65YkRpicbV2UYUSIez6uK+uzQGEvCfTh5MYNVKn4lN0u5s6yhYpkH7fT0CQsQB/GCp3tGBWCKMB2kHKpcZUMluPA2US0FbMjF+G9B3pAV2IEB7yVnU9dWMIxQuIqM2yb8FGhYQNc1x567eG1Y6eNoRaHejmv+P5LDCrPjTM8G2EcYvBeSzb8O+NdUA5py4lSKKHeSi0oiIw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=Xb2WuoV1n9d7enOgQK2mVmgEDn5DmGN/bpAYSAaohe0=; b=NtvF1HTFSgj1tWIjiE7rclSE1IlQVtVyOSaO9wTj9eFFXsyhfYvG6cUthH1WpEd8RmftYtAmhDCMW+WMKW5wX1x6KMe5VmesbeJxkd5PsU7bdO3d5KoiEPALzRHvRqNd6XhF5wACW9oq12/hqAJTXtMgPnNtBh4kfdmBLpRyBuXeH7H6xWv53pLcApc5PbAbLCTcZIULBvS6r7Oa+byGYnJR6h3CuNnlvDrCqC6Q1XwvVasp4SLmYGlbCYLITFYLH8PutxA+85xFDTBzx+SIytFVJayXzSrULg+TrVHCicrk7jB8JABXlcx8mi5XLLzHrFvdsFhykfC4VJLHDmW+Ww== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Xb2WuoV1n9d7enOgQK2mVmgEDn5DmGN/bpAYSAaohe0=; b=IveFWjeuTm8neKaJYd+xj2N2Yyb7poONgFco+vsI1lYxX4M9a6Ttj03Un72KariaV6R84uDptff/857AUVPDA8+Flk8KB4lsycliomteRR/OUVktcpaxrm/hrtBWatsEY8lZHhmeVkua/79CtyLHEN+pxuptcpBikUD4ixrThEM= Received: from DM4PR10MB8218.namprd10.prod.outlook.com (2603:10b6:8:1cc::16) by DS4PR10MB997575.namprd10.prod.outlook.com (2603:10b6:8:31e::5) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9275.16; Mon, 3 Nov 2025 12:32:20 +0000 Received: from DM4PR10MB8218.namprd10.prod.outlook.com ([fe80::2650:55cf:2816:5f2]) by DM4PR10MB8218.namprd10.prod.outlook.com ([fe80::2650:55cf:2816:5f2%2]) with mapi id 15.20.9253.018; Mon, 3 Nov 2025 12:32:20 +0000 From: Lorenzo Stoakes To: Andrew Morton Cc: Christian Borntraeger , Janosch Frank , Claudio Imbrenda , David Hildenbrand , Alexander Gordeev , Gerald Schaefer , Heiko Carstens , Vasily Gorbik , Sven Schnelle , Peter Xu , Alexander Viro , Christian Brauner , Jan Kara , Arnd Bergmann , Zi Yan , Baolin Wang , "Liam R . Howlett" , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Lance Yang , Muchun Song , Oscar Salvador , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Matthew Brost , Joshua Hahn , Rakie Kim , Byungchul Park , Gregory Price , Ying Huang , Alistair Popple , Axel Rasmussen , Yuanchu Xie , Wei Xu , Kemeng Shi , Kairui Song , Nhat Pham , Baoquan He , Chris Li , SeongJae Park , Matthew Wilcox , Jason Gunthorpe , Leon Romanovsky , Xu Xin , Chengming Zhou , Jann Horn , Miaohe Lin , Naoya Horiguchi , Pedro Falcato , Pasha Tatashin , Rik van Riel , Harry Yoo , Hugh Dickins , linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-s390@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, linux-arch@vger.kernel.org, damon@lists.linux.dev Subject: [PATCH 04/16] mm: eliminate uses of is_swap_pte() when leafent_from_pte() suffices Date: Mon, 3 Nov 2025 12:31:45 +0000 Message-ID: X-Mailer: git-send-email 2.51.0 In-Reply-To: References: Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: LO4P265CA0128.GBRP265.PROD.OUTLOOK.COM (2603:10a6:600:2c6::16) To DM4PR10MB8218.namprd10.prod.outlook.com (2603:10b6:8:1cc::16) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DM4PR10MB8218:EE_|DS4PR10MB997575:EE_ X-MS-Office365-Filtering-Correlation-Id: ae58ccca-34de-472f-ab20-08de1ad50828 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|376014|7416014|366016|1800799024; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?fjzAqxr0RBw9klXJ9QecXtMab3h1EeQ/wfhllpR8m9al9dfea7pGG2pIZ5BV?= =?us-ascii?Q?80VFXBSemx7ViiKbFOTx2jLwSg4N+VXuY56DS4DNRPOiGqraqiTm+yBnrz56?= =?us-ascii?Q?Wr4o/81rhU1J+bvJk28nY/M54XmDtEFhV8oZr9H9VDMlWr0eVIyJhEejljLC?= =?us-ascii?Q?t9q3knL+/6YyHe4uOY1s52a5ZXoUdFlF1QgNjb7XCwaKJjbeklGqQ4TeSGE6?= =?us-ascii?Q?MZi63yxet8N482qTCHypRks+uo5C77vfKWm4pB/OdALOnp6LPQtX0QbYK8Jv?= =?us-ascii?Q?0+SXXkY9TyHfUk1lSz7rD/+1nuXaThAVyX4ESDJCmyZuv5Jb5vWP209qQbk2?= =?us-ascii?Q?oLjOXUxHbLezL44BlEgWECHu3E/dbkUrm2aq4zHd4iWKHNwCZhbKxiuVFk9f?= =?us-ascii?Q?5HXVe4i9Hjp3AhvrGldAEQdH46uJNAurUvPX3yRffgsOIF4rQHw3O7QCY9Xq?= =?us-ascii?Q?81PeJ0naLAv7roQdY0M1+gs1VXRKSPpC5jHP3PbadFJGYuG7+kGmVC3zSQuu?= =?us-ascii?Q?0DCWnuKKi05Kbhx2mscZDQHMi3Pbt6qecNW5lCvJfBwayK2rb9SE9JnsjQFc?= =?us-ascii?Q?CieEAMjMGKKe4SnnLHbMTS+as+ePXvXz9iF5NQXFBwanKaAweXUGRLuj8bUB?= =?us-ascii?Q?trogM3jkohSPmMj2y2uap8EUXngVC+6xSpdCuqtiCZQOLcs4B6Peel3kHVTK?= =?us-ascii?Q?Ob9IJVj68fXaypGjRTiaQ6swanRt5hqHYl5CM3M51bktaNYJIskyDuAMbbBX?= =?us-ascii?Q?R4xX99WKvsxcQ428gcoDEOjDSvFE2LBh5BqvL5pn7ErhpAE57fT6Rpyjlihj?= =?us-ascii?Q?gb5zJv2Eytw+NKv1fq05lda8DyCtb34sj6Zg3emEXTkvRnfjUEQj7iMh5W8+?= =?us-ascii?Q?cuENO6+LkDtpb5OnjhvNSRAUZ6BWKQArybApWSU+NjUB4vHKRIMPG+keCYFN?= =?us-ascii?Q?k6MpzDkbK1f55lV2KpaQqIoiEDEWMZErZBQWMKT6Hc5VfODxm6KRKLfi2UWn?= =?us-ascii?Q?V41hQpbcAtxAlDTJkOkrWEOpnsDZwuEpxUHer8SKR9GON+5LpDJXnnsp1K06?= =?us-ascii?Q?LQcM1/kkxWnNvsdecygHDZRE3SgkPaRqk2qro4GUIjiOzfN2ruYrzhE+JAbN?= =?us-ascii?Q?H6X2an6yld/2pLmT3J1tboigqyd6sQiQ12uvGg3uf15sFXiChxof+A0lPDUz?= =?us-ascii?Q?fK0+RXFLiFMoYY2Y09McdKidF6JxHgx/LciZNKfpvORfVL896CbyghuIpP4I?= =?us-ascii?Q?lT/ouh4uzO+OIrNMu/v2vJECYb6xZ4jP5diQdcOKcGLpooPsszLMI8W4kHgW?= =?us-ascii?Q?/T7FQ6+XkigAtSZI6TLuQgDMiM2IIeO5EQAp0reUGttPtecMlmFiKWdRem75?= =?us-ascii?Q?/ho+rPFR2gcGJS72MMoAJUISATwjLlY1QjTplgOC9BktA5sQhHdDM6f/jv6t?= =?us-ascii?Q?hbjph+MNZkDhH6gT7FlELnb6LXo3PUJW?= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DM4PR10MB8218.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(376014)(7416014)(366016)(1800799024);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?r8r27DwHGGA4/1hi0WECzdFBjAZNfjZ2LRQPauQ2WvouWIR9CA0qd5f88zZd?= =?us-ascii?Q?lEXHq9sdaFqIzQCnbGQX32Zsd7BVdCZQngUOivZo2XcMtfjfwHBbpSiouutc?= =?us-ascii?Q?u4GbKVopIh1wkeIsEypqIbOQF8vY6ES0c9QX2Bnjb7xZ8MVHEMQTKbrzSqCz?= =?us-ascii?Q?MrM25148//zZQuLmqFS2zF9OMysP8VPJpSlKL9aLUJGPGsnfz/nKzOJbOyG/?= =?us-ascii?Q?R6kVv8bDVcAdwLc/Lnu+8EttE144NnKWLUUCV7RvHk5AJAZ9ePeBGUMYJmTA?= =?us-ascii?Q?Aw8Bhj+hz1AuqA/Y+Lbf4AUXyxaQyA9iUM7EBbOfnBDp2cDXzxXBCVe8iSjP?= =?us-ascii?Q?1FBmYAJVSfXPLBQWEzXR8EbvtYBIk7h8ef+PNlnHjlJ8AjcfUb8ETz+mY3cr?= =?us-ascii?Q?iXL8tyXvKcaNvJd4tXTMdovW0OGKzlqzgIqCPPyAF74Ip+SxyYI6t5Il0DHf?= =?us-ascii?Q?Dz5JQk5OcDmNIRCb00vCEqth+PKPDDk8UQp+A0Q2/xVIPYcUrmp4wTICBpRN?= =?us-ascii?Q?xzRgPqPuG6meVZ58T5eqMZNHmnyp+ELJKaRsUEqzs4bM1M77YymT0M72HKAO?= =?us-ascii?Q?15KZnt7/G3KwpHAYbG/oruKBibzrvmLQr2J8tj1ILL70RZYYUr6ZPnfNW44T?= =?us-ascii?Q?6Zq00CCAvZng/tEqzkP8WIKCh0ONR11IvZeIB0d62sQ21XCd67Z2XqsH656j?= =?us-ascii?Q?YLzHlvhNXLL6ImNatEP5OZqBXvuz63qE6Mu6o6Gqi7cTe1FJ1nz1S0YLeXsm?= =?us-ascii?Q?S3umV4dW5vQC2OhC/rfp2xvY+DA1myCpZ/HEAUSjVQrnsdtdTkOa39d/dH2E?= =?us-ascii?Q?Hps39jZ96Wl8b1DVBW+pHy9LSoftwLb6i39ZRq1okvtPdYCLUfR7iHoYxbLO?= =?us-ascii?Q?qMOs75DvJIRb1Icf/HBB8uzDpTJqFbm4pogDsHzUOqbEr+gLFo1VjaL1jUON?= =?us-ascii?Q?M+eWPymThlwvUDMk0MT+wBnVbocwU2KHJeJIwIvFHrmZ06y5fpciq8ZtNNHl?= =?us-ascii?Q?OugjUMfO2y/mNXPM3+pqkLmEl2/pRxeU8DM3B6vTFg5HRYjw4eR7Yt7DUhVR?= =?us-ascii?Q?Ge50tbRhRflEoh9R5/mSV9r5PrRIMvseBMAHzDRG/rpwHze6bBrVjJxw9K7G?= =?us-ascii?Q?IPhjaESzOJLZ2qZz4hMI8ag+Pig0AdPwqJvTkVQ/vTJiIpT50hviQZ0GqP9N?= =?us-ascii?Q?scBtKO9jHWhBX7lJ7CcscHXuzYj5SnOSUoqY8DEG5l5pdRghUK/8aLoWwqhH?= =?us-ascii?Q?yFJJyQI2p3zp5k0pTaVQaX5uGPpVO4RhzsFrHJGGTKp3HNSRCty1FW51Z5v8?= =?us-ascii?Q?3AWL9lOjD38lfXW6PqkYwLg95CitpjTyZ7rsOsQcwA4fabOp1depqpbiMahP?= =?us-ascii?Q?dTREg/qm8X+OTqGHXtA/SPAzX3HfYZnBZer8V5UEqoX9rM8u7XKCBpO09B2P?= =?us-ascii?Q?jF3tY0PALXYN1hPXIOMQCl7Gl7ccgZXf+H1dmmLBhPZmJ9E02wYKvkbkvzAj?= =?us-ascii?Q?WGiv5YjKW7uCfv/HZ8ebGVwQNFYxMTO9efW1gjC3nq4NFmIWlBjHGlMZMxz7?= =?us-ascii?Q?rTknQ/mG46m8wVhNcBLAvIxZ3tPyuyC3/Cr27Zx9HMswmUR+ItTjJbUSIi56?= =?us-ascii?Q?nA=3D=3D?= X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: K4zTIdqELgezAlPNu2TDaX23x4LR+HLvZbjZrubCVmvqfjoOou8oWIY3raOXKdvKOuCoVhGO9iIZ5lHNFuxp+FK6u/3xZ9JWXvFHFXJCK/3ZxUTDpLKnIRmWaPPeRfACde+cG77g03sgh5r4Qh8loaiChmzhoA65kNvhDAS1ttZt3LO+FvwLj6c89IdLLEbVNhnQkdyeJkQRiaMDRcw+0vDMqIg+0VVmSYLTOWWsaNlvWtzvsZ/i+WZ+qj6OFFtNJU8+oSz6lxp9FYZ6Ppl6tXBkRmoQNOgmJQ4JKoJTsgRLW6XnnsauYmBsP0J1VSDE+Kp6U5jpWzkbgQfHoko1+TCkw9rmEivSE9CH1PUC+Nq2aFG0slyDz00vjkyBCzG1is/nS+/hzOxbjMKJbZaqRplvaxlLwPbxHybT5NqIOv11OGjwIzd4WnJV4z3qdNEqAWVnKcSIYLxGRvuXayXvCc/VrZQgazpGqCc/aVw8VOH/MElD9yerBVZr4cPZ0GtKH4yQnk7mtopLZ0mJNLdP/P3o+N15Slcip2xekthSEMhS9U7fAOTPMOXuXk5zRBfMnjoNsq+6mJ0Zx8/jHI/oPoNDinJWKbUArI5AZ3JXFK0= X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: ae58ccca-34de-472f-ab20-08de1ad50828 X-MS-Exchange-CrossTenant-AuthSource: DM4PR10MB8218.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 03 Nov 2025 12:32:20.1192 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: bKtzyZtGO0WL7dLWqFX41mjvIVh4tQF5Z6QAbRPGO1zxrfLqzf7nxC0Eo4gTpxw0KthzpPCy9bwYdzY+g8AXySc+BMPgxUri2Tu1sPN4YkI= X-MS-Exchange-Transport-CrossTenantHeadersStamped: DS4PR10MB997575 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1121,Hydra:6.1.9,FMLib:17.12.100.49 definitions=2025-11-03_01,2025-11-03_01,2025-10-01_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxlogscore=999 bulkscore=0 spamscore=0 suspectscore=0 mlxscore=0 adultscore=0 phishscore=0 malwarescore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2510240000 definitions=main-2511030113 X-Proofpoint-ORIG-GUID: fPeVpAZtYuW2SOzRmll5O3zK679bVVgq X-Proofpoint-Spam-Details-Enc: AW1haW4tMjUxMTAzMDExMSBTYWx0ZWRfX0g31nEblHkLa 1wkz33oWO3WDjQ2OyN9AyEakmmcEs/1UUYAXC0zJ1AFRhy3eLzvHoD3RM+IGd/IVet8GVU/i8Zw PrJP0gdP0veKq9H4ucBn8+DgbugHKnNr/qYPWRKzXmlWj3fQCwUm5U06Pz5CvRBCUJu2lT36iYs Ch6ry88lHDo0cl/Rsc1NciayAxTNafMnS9S+bVolqrbt52NOYqMOTEyIJvcrVNQV9gQ5njzvvYg Wx0Ynl745NoJFE3xDQbEiWHLsh+qodVH3bjGSm4fHVubpD6feIaMeR0GkNnkJf30gQu6lke0Zpv iN0PJOF39UdHwGkci71oLNSxI+wR5iIB4m0SlsZxE0clWtJ3nn/fJnpugtK5WFWhzQumsKOyhlp VogYakoQ7mrLDCNyIsSlcNZZj7OthgZ+3PvAskdbFLGbMNlhP1M= X-Authority-Analysis: v=2.4 cv=Fbs6BZ+6 c=1 sm=1 tr=0 ts=6908a0d8 b=1 cx=c_pps a=qoll8+KPOyaMroiJ2sR5sw==:117 a=qoll8+KPOyaMroiJ2sR5sw==:17 a=6eWqkTHjU83fiwn7nKZWdM+Sl24=:19 a=z/mQ4Ysz8XfWz/Q5cLBRGdckG28=:19 a=lCpzRmAYbLLaTzLvsPZ7Mbvzbb8=:19 a=xqWC_Br6kY4A:10 a=6UeiqGixMTsA:10 a=GoEa3M9JfhUA:10 a=VkNPw1HP01LnGYTKEx00:22 a=yPCof4ZbAAAA:8 a=RPBNNRGYkw5XAb79-8kA:9 cc=ntf awl=host:12124 X-Proofpoint-GUID: fPeVpAZtYuW2SOzRmll5O3zK679bVVgq Content-Type: text/plain; charset="utf-8" In cases where we can simply utilise the fact that leafent_from_pte() treats present entries as if they were none entries and thus eliminate spurious uses of is_swap_pte(), do so. No functional change intended. Signed-off-by: Lorenzo Stoakes --- mm/internal.h | 7 +++---- mm/madvise.c | 8 +++----- mm/swap_state.c | 12 ++++++------ mm/swapfile.c | 9 ++++----- 4 files changed, 16 insertions(+), 20 deletions(-) diff --git a/mm/internal.h b/mm/internal.h index 9465129367a4..e450a34c37dd 100644 --- a/mm/internal.h +++ b/mm/internal.h @@ -15,7 +15,7 @@ #include #include #include -#include +#include #include #include =20 @@ -380,13 +380,12 @@ static inline int swap_pte_batch(pte_t *start_ptep, i= nt max_nr, pte_t pte) { pte_t expected_pte =3D pte_next_swp_offset(pte); const pte_t *end_ptep =3D start_ptep + max_nr; - swp_entry_t entry =3D pte_to_swp_entry(pte); + const leaf_entry_t entry =3D leafent_from_pte(pte); pte_t *ptep =3D start_ptep + 1; unsigned short cgroup_id; =20 VM_WARN_ON(max_nr < 1); - VM_WARN_ON(!is_swap_pte(pte)); - VM_WARN_ON(non_swap_entry(entry)); + VM_WARN_ON(!leafent_is_swap(entry)); =20 cgroup_id =3D lookup_swap_cgroup_id(entry); while (ptep < end_ptep) { diff --git a/mm/madvise.c b/mm/madvise.c index 27e078098926..398721e9a1e5 100644 --- a/mm/madvise.c +++ b/mm/madvise.c @@ -195,7 +195,7 @@ static int swapin_walk_pmd_entry(pmd_t *pmd, unsigned l= ong start, =20 for (addr =3D start; addr < end; addr +=3D PAGE_SIZE) { pte_t pte; - swp_entry_t entry; + leaf_entry_t entry; struct folio *folio; =20 if (!ptep++) { @@ -205,10 +205,8 @@ static int swapin_walk_pmd_entry(pmd_t *pmd, unsigned = long start, } =20 pte =3D ptep_get(ptep); - if (!is_swap_pte(pte)) - continue; - entry =3D pte_to_swp_entry(pte); - if (unlikely(non_swap_entry(entry))) + entry =3D leafent_from_pte(pte); + if (unlikely(!leafent_is_swap(entry))) continue; =20 pte_unmap_unlock(ptep, ptl); diff --git a/mm/swap_state.c b/mm/swap_state.c index d20d238109f9..991256c6254e 100644 --- a/mm/swap_state.c +++ b/mm/swap_state.c @@ -12,7 +12,7 @@ #include #include #include -#include +#include #include #include #include @@ -732,7 +732,6 @@ static struct folio *swap_vma_readahead(swp_entry_t tar= g_entry, gfp_t gfp_mask, pte_t *pte =3D NULL, pentry; int win; unsigned long start, end, addr; - swp_entry_t entry; pgoff_t ilx; bool page_allocated; =20 @@ -744,16 +743,17 @@ static struct folio *swap_vma_readahead(swp_entry_t t= arg_entry, gfp_t gfp_mask, =20 blk_start_plug(&plug); for (addr =3D start; addr < end; ilx++, addr +=3D PAGE_SIZE) { + leaf_entry_t entry; + if (!pte++) { pte =3D pte_offset_map(vmf->pmd, addr); if (!pte) break; } pentry =3D ptep_get_lockless(pte); - if (!is_swap_pte(pentry)) - continue; - entry =3D pte_to_swp_entry(pentry); - if (unlikely(non_swap_entry(entry))) + entry =3D leafent_from_pte(pentry); + + if (!leafent_is_swap(entry)) continue; pte_unmap(pte); pte =3D NULL; diff --git a/mm/swapfile.c b/mm/swapfile.c index 543f303f101d..82a8b5d7e8d0 100644 --- a/mm/swapfile.c +++ b/mm/swapfile.c @@ -44,7 +44,7 @@ #include =20 #include -#include +#include #include #include "swap_table.h" #include "internal.h" @@ -2256,7 +2256,7 @@ static int unuse_pte_range(struct vm_area_struct *vma= , pmd_t *pmd, struct folio *folio; unsigned long offset; unsigned char swp_count; - swp_entry_t entry; + leaf_entry_t entry; int ret; pte_t ptent; =20 @@ -2267,11 +2267,10 @@ static int unuse_pte_range(struct vm_area_struct *v= ma, pmd_t *pmd, } =20 ptent =3D ptep_get_lockless(pte); + entry =3D leafent_from_pte(ptent); =20 - if (!is_swap_pte(ptent)) + if (!leafent_is_swap(entry)) continue; - - entry =3D pte_to_swp_entry(ptent); if (swp_type(entry) !=3D type) continue; =20 --=20 2.51.0 From nobody Sat Feb 7 17:55:59 2026 Received: from mx0b-00069f02.pphosted.com (mx0b-00069f02.pphosted.com [205.220.177.32]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8D59328EA56; Mon, 3 Nov 2025 12:38:21 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=205.220.177.32 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1762173503; cv=fail; b=RrMxsVOHQOWORSxGtm6J6k9ynlLhkRrscEsbZ55CbVklIMVDJynDYSIJVorVWqMfM2ERh6jkwWF3t2gI3UWYz+mJuVFfK2ODOPyrMJ9sSKmR0aW0fU0le7G/kcSmMmxD4826FO3gOZBzp76UUn/3Yvui6AKOj6mjn7ohebwe+xc= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1762173503; c=relaxed/simple; bh=5gSGw5qMSaaZJEYbbFS50M+kyjoi5kiCXfiINcmbvv0=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: Content-Type:MIME-Version; b=rQQyBabyBiULgdakLlVRvwKQxWqcrUxiMWR438AbR5I2oQbGaN4AS72v+NaKR6IXVFF5oty+Ioak4dbxPTlPPjXjBPZZ29CkAPLVBEGFRDed2uXd3ZezIYhTF0rrFRKwvN2MRm10WCsakrqXTtbJT30eGCGshbKT4olkCCeFU0g= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=oracle.com; spf=pass smtp.mailfrom=oracle.com; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b=AcpcuahU; dkim=pass (1024-bit key) header.d=oracle.onmicrosoft.com header.i=@oracle.onmicrosoft.com header.b=tjS7+hcz; arc=fail smtp.client-ip=205.220.177.32 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=oracle.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=oracle.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b="AcpcuahU"; dkim=pass (1024-bit key) header.d=oracle.onmicrosoft.com header.i=@oracle.onmicrosoft.com header.b="tjS7+hcz" Received: from pps.filterd (m0333520.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 5A3CEfR7017288; Mon, 3 Nov 2025 12:32:26 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=cc :content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s= corp-2025-04-25; bh=iLrIYijzaZOaS2Eo+nw5Ch9zPYhpC9Fl/f+qqBMU9IM=; b= AcpcuahUznelIcURzuAjJZnzUOb9yle49mGCNtq6eQIDkgm2K/uYh0vY2XrvZxDx CWvH2BU0Kl4rAstcPymIGsBk8sRkb+i5C87mq5ty99+ivNesVOJozO2l1hNi4a0y zUP0Lk94+k841tPr6VwkhAiuvIZCcmDrGmpG2lvFo4gmiqZjHQ6omHolrCEMd3w1 Li03NrWHybA5T1OGzKk60076d1JbxvyJogzbhAVW/91vZQ03dQaN8LkE8s1oJqq9 FHS7Xt5cU858fafRbGnJ41AP77ekKSdqsiOpnGKjJ+2tFhyUXHuOefIVNOCKzN7K Dwxs+suuDebs/i+ldcgsHQ== Received: from iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta03.appoci.oracle.com [130.35.103.27]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 4a6v7kg0x2-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 03 Nov 2025 12:32:26 +0000 (GMT) Received: from pps.filterd (iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com (8.18.1.2/8.18.1.2) with ESMTP id 5A3BVq3j039882; Mon, 3 Nov 2025 12:32:25 GMT Received: from co1pr03cu002.outbound.protection.outlook.com (mail-westus2azon11010026.outbound.protection.outlook.com [52.101.46.26]) by iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTPS id 4a58nhux3q-2 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 03 Nov 2025 12:32:25 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=FGbC4SbKb/y5dQ5ymRug58LgpnAkR3/52FnY8jbX/vZYKQnspvNH7NBuMATq78bSfSHbBV/8CyFKwDpE+b4bl+YIZ/S0uDH5/CKhpFariwCgapB+8FG1KQJsHQjS3VUt3CpcHk0aNmV+1/b4nigOLGL6vVKCRMLDIrosnkz8THIR2f2tqxJKmQ4C+0Y6sdUbbHxVwzR1NEC7s4GcGVBXd36jLslvlDf4G9IzDj65uMfRkk9k4S8yaxr5Ymrswi7dz6VXK/Guw9/8m1iQOLWgQsokbXDyJEC1Lghj7b4muiFZVqztYVjoETrfQX4FB9NiNcWAGcu4AuUEqewDHLrnuA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=iLrIYijzaZOaS2Eo+nw5Ch9zPYhpC9Fl/f+qqBMU9IM=; b=J0Of8FLoz0eEFUJDCoysoqYnwIbwzd07HaaM0rZtKvbZhXOAnCDbz7cCFrz6pyS8PRf4YWRjwSHXHNfwzRXfLnKFx5DKwMobXh8rbCr/kV98eZ6VAAWq8K+Tn0aimSqaLLATMc4vWfeKMh8El2rdgMOd97C311DY0HIyF8YFQrZOklHCvonkLJqAs1P1432PBAv/2oKx17G8x83utlaGLH6qq6X+4pr1ZVUivu81G9L/nLNd/wlYxplppoiJ2+AdJRrbYGZ3B+i7qGG88fK1Lydo5qBRJJ8v+dCTtJ5Z5VjYikU1AlrtFymuFJyHKn6YFG1sqYQtY8TZU3S48Mzkqw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=iLrIYijzaZOaS2Eo+nw5Ch9zPYhpC9Fl/f+qqBMU9IM=; b=tjS7+hczhpqUFdHJzu/510+/UwCk8QOnQ6c3C8SyWLY9FFIrM1tceYwB5EvjxEEt0Fg2fhdoTpCC4r9IP4dqvzzA57ItDN7NFEpm+CiAJ0dWLWDV3QFWLuRqLkYRhNPYAuU7atChPZfmpN5KolSeN19KrJPZeqMmMc61+XAh/iY= Received: from DM4PR10MB8218.namprd10.prod.outlook.com (2603:10b6:8:1cc::16) by DS4PR10MB997575.namprd10.prod.outlook.com (2603:10b6:8:31e::5) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9275.16; Mon, 3 Nov 2025 12:32:22 +0000 Received: from DM4PR10MB8218.namprd10.prod.outlook.com ([fe80::2650:55cf:2816:5f2]) by DM4PR10MB8218.namprd10.prod.outlook.com ([fe80::2650:55cf:2816:5f2%2]) with mapi id 15.20.9253.018; Mon, 3 Nov 2025 12:32:22 +0000 From: Lorenzo Stoakes To: Andrew Morton Cc: Christian Borntraeger , Janosch Frank , Claudio Imbrenda , David Hildenbrand , Alexander Gordeev , Gerald Schaefer , Heiko Carstens , Vasily Gorbik , Sven Schnelle , Peter Xu , Alexander Viro , Christian Brauner , Jan Kara , Arnd Bergmann , Zi Yan , Baolin Wang , "Liam R . Howlett" , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Lance Yang , Muchun Song , Oscar Salvador , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Matthew Brost , Joshua Hahn , Rakie Kim , Byungchul Park , Gregory Price , Ying Huang , Alistair Popple , Axel Rasmussen , Yuanchu Xie , Wei Xu , Kemeng Shi , Kairui Song , Nhat Pham , Baoquan He , Chris Li , SeongJae Park , Matthew Wilcox , Jason Gunthorpe , Leon Romanovsky , Xu Xin , Chengming Zhou , Jann Horn , Miaohe Lin , Naoya Horiguchi , Pedro Falcato , Pasha Tatashin , Rik van Riel , Harry Yoo , Hugh Dickins , linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-s390@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, linux-arch@vger.kernel.org, damon@lists.linux.dev Subject: [PATCH 05/16] mm: use leaf entries in debug pgtable + remove is_swap_pte() Date: Mon, 3 Nov 2025 12:31:46 +0000 Message-ID: <55e0040009cbcd3bb4d4752d9e435ce9779751ba.1762171281.git.lorenzo.stoakes@oracle.com> X-Mailer: git-send-email 2.51.0 In-Reply-To: References: Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: LO4P123CA0037.GBRP123.PROD.OUTLOOK.COM (2603:10a6:600:152::6) To DM4PR10MB8218.namprd10.prod.outlook.com (2603:10b6:8:1cc::16) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DM4PR10MB8218:EE_|DS4PR10MB997575:EE_ X-MS-Office365-Filtering-Correlation-Id: 126df1f6-9132-4c52-9cd1-08de1ad50951 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|376014|7416014|366016|1800799024; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?cSvrMfFDzAFCiL82fRHaWnlKjBMaa+xDZzTeE7cvXM8kDY2GJSiwdcLsUU6r?= =?us-ascii?Q?aaVriJKflaGLDHUdBLROjDx4jqeKYytixND0RhAXBVUNctuj93WoRLv6p/5N?= =?us-ascii?Q?atEps06MY+wqjn1Dk4loc4qDyaPAoLfpl4H7pQC0soAGxKgQPKCYoE1MGV5r?= =?us-ascii?Q?7nPLTzQMSGdDqYXxhoAfsotAlfOpOBWNW3uNxZOJqTR9cUSgcOKEEnwj7EYZ?= =?us-ascii?Q?JYMhDg7qZu5sjLnku7L9/LjgYcEyMCGhCgcVni+dpnE7G/sV6BHu/3dRxs1q?= =?us-ascii?Q?t/eOhKvi5GcIyOQQXZvP3z5hpowIgabj8RnYbukCeOF7dWvXuWdCwObvDan2?= =?us-ascii?Q?qV7qvbsBLCQmjhkapkRBega4cgl12VTGGWH2+bloMm0mW1p6QGR/JixW7yxL?= =?us-ascii?Q?TEfItV+l7pXm3MXCmlxM7Z3UVAD8DhMowchYT99bOFcmoeDRVr+Zpuw/6W27?= =?us-ascii?Q?xCZ1fJu6IL93fncHM6GjE+vLzBD0l4rQ6qipAmipMGdUTKYeW2w0PB7GxO/4?= =?us-ascii?Q?XmYXM95DOqeDonkNuASsjleM3bcGsx26mzOZHl6gt9Jrvqa3fKVK+ytQ7uF1?= =?us-ascii?Q?9CrftmE08KFn3SBs3+HpeKKiXk+RTG8ImrVuLy/oy9MMiXvpXJ+hMUrIGK3K?= =?us-ascii?Q?SItpeckZxm7V+mi5zwOULY73sZ7FEJwh4e7+Dcw1DOYTj3itKJuwzUj1Y1r5?= =?us-ascii?Q?RbtGx1EzxInOaRtcguE8WMYLAr02ypyfjOmucPKGmIp7o+/gE9Hd1+LDE5h/?= =?us-ascii?Q?jQ0l4G1NuVxWwYHx9IvnHIYZ+0/ckqFncbGuyueg8jFlloh84w9LWfyB1jYm?= =?us-ascii?Q?hTYEOXOliDS0J30mgchUNo4xBwuw1mSgd04080Nv4DS8hJjbxiXILjJPS0DN?= =?us-ascii?Q?N3y4f7haglTSy88JCwrU3GqzW3TzXLgVgCYHFw0Y4wYa9uMN21o9kO984jSy?= =?us-ascii?Q?MD5RNLMdKHI+UNJMmNabTwKvR/Bvlzv/nEM2nS5cXDV+vo54e1sQN9SN+v9G?= =?us-ascii?Q?qGl6DCweMJP6NMRAZ3d84SYQwlMiYB7KLVbu6f89esrtptaxpRVgRBEhRenN?= =?us-ascii?Q?SDglb+1GDh3Qqz3f0vltBj92NrBSlsEeSDtbDkhwJkmK1HoEgNJWpDbe/S2g?= =?us-ascii?Q?aNUGtyvjsm2HIy9rHdybhb9ZvqocWPQdvLTa1SGSuyJ+ZU4NaRRUwHg0Cp/e?= =?us-ascii?Q?3/bfwUkwwgn+pkdVkQ4lDJvO+BE6MZObzqwcK5chzuxaGe7Flj1TlyWLl9gB?= =?us-ascii?Q?lJ8vT9q6Q4yaUf+CAyn9SOEO9xXBYWL6MFKLduxKNpYyzv11Ojf2ItxJB1c1?= =?us-ascii?Q?qlERAOIZ/NQpBC/noNZ2lr5gV0qKhE2x11ZwZ4CHiVG2n4oohATi5tuvxFkx?= =?us-ascii?Q?61NmcaVymKG29FB9qUI/XjYZvHqpoQSgdnhktzKIkwPozA3l+c7pqpe0Xhyf?= =?us-ascii?Q?E/iiqBrpWS+5mHXxM7QoG+YZonQJIIQn?= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DM4PR10MB8218.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(376014)(7416014)(366016)(1800799024);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?a5Qr4Qg/hXBHNg0BQGd6E+94gqNg04RL3EsbeXm44zed+Uo+1c0BZn6uRPN7?= =?us-ascii?Q?21UJeHB4VQ91UyjITIazXpTLt7XrJ+fGJXXUcuFwKtl5fWoJDvc90FL6cAhd?= =?us-ascii?Q?GAAlAWo/8YgdKhlbGB/BsBplYfU23+Hu3uaU7SdZjNnNJFMUFKdayWZYgXRh?= =?us-ascii?Q?03YgMN4xDd7g0wiO4vf6fTFAvVskcAM8uDHJUChiMP9Kj8kwwvu08XQPoUS8?= =?us-ascii?Q?w4C7c5oTQ0aCn+5V7ys8lcyoJn19U50TaQZxkmd5Qr3cE7PgpPXX1ry5p4ZJ?= =?us-ascii?Q?BzzvrqjR8YhLKJ/9kOl4m/gxpkwHXUC4z3TSKmyD8lC7preLaxR19bHgeWj+?= =?us-ascii?Q?aUwd1bIi7ltoBY5G36Gmhj39rvfmEE2xAXlG+C6d3UEpGmJxa4eX1yUeYR2w?= =?us-ascii?Q?8EWRjXPPf1AJ21lq1Px0Ki+e5jwMV2GWQv82ddLTcawHcfC+cu/r9j8VJPP2?= =?us-ascii?Q?A4zz+zDnsG3QdG/WtjNHrqspoP9JNdQpHggROorkC0ntn8PqnCJhuQpm1pHo?= =?us-ascii?Q?wbTMQmO9b4xlx7OWjjfpsGpxZsyOskTBCXO1ifnPybm4/6yh6XiJBdi3Yywh?= =?us-ascii?Q?Xw8n2juw490lh5i/upo8tBA0BIpiq1td00xkFxiacEndLZbA24tEbDeafgTQ?= =?us-ascii?Q?dJc5qQgoX3FxlvK6ztoisS9kHjN3rtJPD3+hGzuq1T5J1mqr7HY2PPja9ffJ?= =?us-ascii?Q?kNcdmZsnQ8PsBGzVD7iuiz3Bhd5LJw047RkRKKJWVLY8+l6cCr/mMyNkXuC1?= =?us-ascii?Q?5EBATobp9F2FOiYJ3yvbyUg4S+B7P7kWsTeQ/Et8jCmQ73S/fmYRSshNGPpc?= =?us-ascii?Q?AV6lsh4F0etlVUtIb/1QW5JVwfSY1I5PJxTB3zuFToCtPH4CGmQ2gxbUa3At?= =?us-ascii?Q?tY2f4fkkVhXuE3kYloAwvOu8a6dHFgCiEMI5C1wcVlfHibaXs+OXTzruVgqo?= =?us-ascii?Q?Y5zOLdlAVwCLHjcH6qbeoneBfauPZct1a+lyiAnYI8U2ZBe+g/8zww1jzFzQ?= =?us-ascii?Q?AYMz8tpW0O/eOxktZYJC8CEL+7XIqHLrR7wmXEsH2kfJI+G5eaKOpGlRzezv?= =?us-ascii?Q?TShnKx7os8OLs516/4KHWab0AiCPHyZx4WMUGRyU+OP5t/L9V5g/8l00k1Pl?= =?us-ascii?Q?EPjAkIxueSJKL0pPXK1a4CPYImyeIsUNNLnEpbn3GlPS8X6DKHd/o6E/5E82?= =?us-ascii?Q?rYGa0MVd/23x7oUO75+BYYmh5OaQHJ5tAxJXNlAuNzy9L96nryq1Hpz0fplH?= =?us-ascii?Q?+eoTdsgiEL5QvSM9T/+d9fzWVcjEwkQ8zKa4yG/P94iqz0nXA4VdLNXuAuG1?= =?us-ascii?Q?GJoJoWn/Gkf6tg13Pz6XePx3v7XV5fFGXYhpAKwE/PJZkKGs8DnKknL0putb?= =?us-ascii?Q?4UfgAhS3qONuVi32N6WS1IvnK97ZmAq6kWbQ5XgdCIfIpKsKTzPisiFOCNQs?= =?us-ascii?Q?mF9hcpjj2FuDUa7zt+VhxHRxtULhOcwV/zrNmVYmRyPo0kYdwkiCkPiHZ4l8?= =?us-ascii?Q?8mo71yyvFD91Darxi4sIgnteSEpbE7Dg8Tv1Tasb/pmJ/AZYt6eq7uO+IUuj?= =?us-ascii?Q?Xwrkck3QSgJ5wOUgFPvjMiYenBtMn8dnm9AerLGG97KUHggWjesPINSCiybr?= =?us-ascii?Q?aw=3D=3D?= X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: dcEyKGz1cOzINzk2b8WOcHHoFU+Sdt6UeL0ywZgfRdkK6DJPclqleNL2PKlf2HxuQ3DyzwWwzUKBSjwmrYwNQbK/u+DKz5h2WMs7KeP0A+YktcCwjdzxdzb68oNdjQDEabj0HSMXTYDiCf4IDyGMzSvrPMAbWSqeV10+1TSCBEI/EmBTI2bQ2N2TbQE0WhvOVcVMhYj1lBU3Qa/Z1PHlGGXdNyndBOikPFANgn/iZfyrTfe61wXPxgCCFOA4/Yao7NyKw7Dfoyg2vYjo/Wmb6L2N3lUjHeg0tiIotJFdwEQdC5ErF3gtbfACjDxYMom7mkugU+1D84Jrqk4byCMnI7oe1YL4BnpIY59TGO7w4Uf9bD+AbcQDf73kxWgDPQDXINlodeiQxFpBich8WjsXxytNRvd5KXVCdegLGQRytxftZiqCwUehREP1N5r3mIWIdTMe22V7o/TC9+77woouK7ZQDtW+wjR1Rpy84sS+MILbyiyMdw3xtKyRjCcV/LAgJyjQU52fwvLaiKAWzSIKHMtL3SDbHZMLS6VJxFcqFlV2ChxuqYtBEwHtbJ1/6bBOuBI3MQ/7h5NqeM08pdSq9an4NvUidy7LZcrjQKHq2Yk= X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: 126df1f6-9132-4c52-9cd1-08de1ad50951 X-MS-Exchange-CrossTenant-AuthSource: DM4PR10MB8218.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 03 Nov 2025 12:32:22.1681 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: jKKbNzViYlx+hhVSGOW34kCvT0x6JEC8HRxECv+0vXglrKufCW9OVGuWctKo99FUIlv6Xh1vL+mzKu2vWs+j7rF3YWVnORGlyQwtPiMrxnw= X-MS-Exchange-Transport-CrossTenantHeadersStamped: DS4PR10MB997575 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1121,Hydra:6.1.9,FMLib:17.12.100.49 definitions=2025-11-03_01,2025-11-03_01,2025-10-01_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxlogscore=999 bulkscore=0 spamscore=0 suspectscore=0 mlxscore=0 adultscore=0 phishscore=0 malwarescore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2510240000 definitions=main-2511030113 X-Proofpoint-ORIG-GUID: hp4P3v8rnCaxwDKfCS7yowhymJMlEgck X-Proofpoint-Spam-Details-Enc: AW1haW4tMjUxMTAzMDExMSBTYWx0ZWRfX5YiGaT6lCXZL 9F8WHx/u0rXHzsT4v9xCUXQGSdsDD1nuANEhbeCnJTuuoVmo/gPgXrO5b+llGVIt/TJHZ2yo4S5 PeNNRCoe2/tcPoJKo05pG9vg/0+6I3xkLJMQTRtsSe5MCAGdtrqQHP3Cdi/4fKlui6gY/mtSkFu ZkHNWFZQK1al78L76hW8ML1TXG4PppgdEVkDbmMT0VIi4KST9/Dq+fiBh18cnjtSh+TWGUWlgWW GYFM/7HWxoKiy50ntUOlE1Ho4mwaqWCVypVqIyeOsN+80cjugTfyTWLefqwXrRDlCO13vBGF0X8 UJWxBCXbGEyX0rOkrOk69ghihrM9nh7gKoeN1dUB8lIWEF4pupUB9b1OdwjwJlPqrQxCPpqHBbP yeIek0lzLykyizWwKOHrrM+wpN3aidDugcG6fRtuI5FEh4Ene88= X-Authority-Analysis: v=2.4 cv=Fbs6BZ+6 c=1 sm=1 tr=0 ts=6908a0da b=1 cx=c_pps a=qoll8+KPOyaMroiJ2sR5sw==:117 a=qoll8+KPOyaMroiJ2sR5sw==:17 a=6eWqkTHjU83fiwn7nKZWdM+Sl24=:19 a=z/mQ4Ysz8XfWz/Q5cLBRGdckG28=:19 a=lCpzRmAYbLLaTzLvsPZ7Mbvzbb8=:19 a=xqWC_Br6kY4A:10 a=6UeiqGixMTsA:10 a=GoEa3M9JfhUA:10 a=VkNPw1HP01LnGYTKEx00:22 a=yPCof4ZbAAAA:8 a=JfhMdzgkenBfAv4L-EYA:9 cc=ntf awl=host:12124 X-Proofpoint-GUID: hp4P3v8rnCaxwDKfCS7yowhymJMlEgck Content-Type: text/plain; charset="utf-8" Remove invocations of is_swap_pte() in mm/debug_vm_pgtable.c and use leafent_from_pte() and leafent_is_swap() as necessary to replace this usage. We update the test code to use a 'true' swap entry throughout so we are guaranteed this is not a non-swap entry, so all asserts continue to operate correctly. With this change in place, we no longer use is_swap_pte() anywhere, so remove it. Signed-off-by: Lorenzo Stoakes --- include/linux/swapops.h | 6 ------ mm/debug_vm_pgtable.c | 39 ++++++++++++++++++++++++--------------- 2 files changed, 24 insertions(+), 21 deletions(-) diff --git a/include/linux/swapops.h b/include/linux/swapops.h index 0a4b3f51ecf5..a66ac4f2105c 100644 --- a/include/linux/swapops.h +++ b/include/linux/swapops.h @@ -120,12 +120,6 @@ static inline unsigned long swp_offset_pfn(swp_entry_t= entry) return swp_offset(entry) & SWP_PFN_MASK; } =20 -/* check whether a pte points to a swap entry */ -static inline int is_swap_pte(pte_t pte) -{ - return !pte_none(pte) && !pte_present(pte); -} - /* * Convert the arch-dependent pte representation of a swp_entry_t into an * arch-independent swp_entry_t. diff --git a/mm/debug_vm_pgtable.c b/mm/debug_vm_pgtable.c index 055e0e025b42..f0f5fbc13784 100644 --- a/mm/debug_vm_pgtable.c +++ b/mm/debug_vm_pgtable.c @@ -25,7 +25,7 @@ #include #include #include -#include +#include #include #include #include @@ -714,14 +714,16 @@ static void __init pte_soft_dirty_tests(struct pgtabl= e_debug_args *args) static void __init pte_swap_soft_dirty_tests(struct pgtable_debug_args *ar= gs) { pte_t pte; + leaf_entry_t entry; =20 if (!IS_ENABLED(CONFIG_MEM_SOFT_DIRTY)) return; =20 pr_debug("Validating PTE swap soft dirty\n"); pte =3D swp_entry_to_pte(args->swp_entry); - WARN_ON(!is_swap_pte(pte)); + entry =3D leafent_from_pte(pte); =20 + WARN_ON(!leafent_is_swap(entry)); WARN_ON(!pte_swp_soft_dirty(pte_swp_mksoft_dirty(pte))); WARN_ON(pte_swp_soft_dirty(pte_swp_clear_soft_dirty(pte))); } @@ -768,40 +770,47 @@ static void __init pmd_swap_soft_dirty_tests(struct p= gtable_debug_args *args) { =20 static void __init pte_swap_exclusive_tests(struct pgtable_debug_args *arg= s) { - swp_entry_t entry, entry2; + swp_entry_t entry; + leaf_entry_t leafent; pte_t pte; =20 pr_debug("Validating PTE swap exclusive\n"); entry =3D args->swp_entry; =20 pte =3D swp_entry_to_pte(entry); + leafent =3D leafent_from_pte(pte); + WARN_ON(pte_swp_exclusive(pte)); - WARN_ON(!is_swap_pte(pte)); - entry2 =3D pte_to_swp_entry(pte); - WARN_ON(memcmp(&entry, &entry2, sizeof(entry))); + WARN_ON(!leafent_is_swap(leafent)); + WARN_ON(memcmp(&entry, &leafent, sizeof(entry))); =20 pte =3D pte_swp_mkexclusive(pte); + leafent =3D leafent_from_pte(pte); + WARN_ON(!pte_swp_exclusive(pte)); - WARN_ON(!is_swap_pte(pte)); + WARN_ON(!leafent_is_swap(leafent)); WARN_ON(pte_swp_soft_dirty(pte)); - entry2 =3D pte_to_swp_entry(pte); - WARN_ON(memcmp(&entry, &entry2, sizeof(entry))); + WARN_ON(memcmp(&entry, &leafent, sizeof(entry))); =20 pte =3D pte_swp_clear_exclusive(pte); + leafent =3D leafent_from_pte(pte); + WARN_ON(pte_swp_exclusive(pte)); - WARN_ON(!is_swap_pte(pte)); - entry2 =3D pte_to_swp_entry(pte); - WARN_ON(memcmp(&entry, &entry2, sizeof(entry))); + WARN_ON(!leafent_is_swap(leafent)); + WARN_ON(memcmp(&entry, &leafent, sizeof(entry))); } =20 static void __init pte_swap_tests(struct pgtable_debug_args *args) { swp_entry_t arch_entry; + leaf_entry_t entry; pte_t pte1, pte2; =20 pr_debug("Validating PTE swap\n"); pte1 =3D swp_entry_to_pte(args->swp_entry); - WARN_ON(!is_swap_pte(pte1)); + entry =3D leafent_from_pte(pte1); + + WARN_ON(!leafent_is_swap(entry)); =20 arch_entry =3D __pte_to_swp_entry(pte1); pte2 =3D __swp_entry_to_pte(arch_entry); @@ -1218,8 +1227,8 @@ static int __init init_args(struct pgtable_debug_args= *args) =20 /* See generic_max_swapfile_size(): probe the maximum offset */ max_swap_offset =3D swp_offset(pte_to_swp_entry(swp_entry_to_pte(swp_entr= y(0, ~0UL)))); - /* Create a swp entry with all possible bits set */ - args->swp_entry =3D swp_entry((1 << MAX_SWAPFILES_SHIFT) - 1, max_swap_of= fset); + /* Create a swp entry with all possible bits set while still being swap. = */ + args->swp_entry =3D swp_entry(MAX_SWAPFILES - 1, max_swap_offset); =20 /* * Allocate (huge) pages because some of the tests need to access --=20 2.51.0 From nobody Sat Feb 7 17:55:59 2026 Received: from mx0b-00069f02.pphosted.com (mx0b-00069f02.pphosted.com [205.220.177.32]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 600F73126AD; Mon, 3 Nov 2025 12:34:30 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=205.220.177.32 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1762173272; cv=fail; b=YW1Fy2CaEVZGmKRceu80c6jrCxcaki0z1xvAO6xgOoQrHdd6GV1/hPKrBtccTslXo/YHSfU0pio6QZ+G0OoKhLuUoK+xlub2HhEgncV/NET+e+CaKLmeqqVZNwIRbJ91B+MSRuUVWBobFWklP9Sp0lDMHfqwMAwGYj0zLdw7LoQ= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1762173272; c=relaxed/simple; bh=bTGeoWEFRPBEOYbicw5Zn3NfHUd0VyZg0MJgxzSnhaI=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: Content-Type:MIME-Version; b=oi7388ucDM7WUZ6OJ0wZgfQy7IHHv9PKNqiJvYtpzaoypfZbM4eu4euOl2XOP7moqhOmN24cqCo6oUb1xq/LgWmyRXe1EWGPjIp0Qiehhw+QfcIOJUbhqM3jqx/Q9p0oZA0dgXbxLnLDsphmyrgeniCoq4b8U3qJqxyoW6YE/3k= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=oracle.com; spf=pass smtp.mailfrom=oracle.com; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b=XaiyYj+3; dkim=pass (1024-bit key) header.d=oracle.onmicrosoft.com header.i=@oracle.onmicrosoft.com header.b=xzSzCtTM; arc=fail smtp.client-ip=205.220.177.32 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=oracle.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=oracle.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b="XaiyYj+3"; dkim=pass (1024-bit key) header.d=oracle.onmicrosoft.com header.i=@oracle.onmicrosoft.com header.b="xzSzCtTM" Received: from pps.filterd (m0246630.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 5A3CP2eJ031542; Mon, 3 Nov 2025 12:32:30 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=cc :content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s= corp-2025-04-25; bh=oS6cqbGj7LwBiJddYfaMT+LmpgHZcgkQu6AukRJJYtg=; b= XaiyYj+3uzYomOSVjF+V8MOD7KVd/hAlSLAM3sgBBGiDV3VPwlmY8hxM7E90rfMQ WitX6uSD9/YpKKDUFUm7J4jPF1d1cjMMWMtYxUm4QjvNZyGmyp+QiGaFuvzhzyk3 m2+sC00DzlLIWGeey3GfbCYDrKSk8xe8RSHLvvp7mB+08d+tA9+6zuMqEyxQaDvc Va0DKalWQtbeLDN9r/xSjYmNsrVSu914IP4W5bcTgvdx9N+FoiL1Y/vjPJaU4hO0 fOx2hgONhEuwdkTx8V8fxPgS9HfzdXKoSFhbI/poEYx/HWz6kOp5S4Jhv+5+Zr7g /WlQ9B6oc6paVLxwceZoPQ== Received: from iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta03.appoci.oracle.com [130.35.103.27]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 4a6vcb80fn-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 03 Nov 2025 12:32:30 +0000 (GMT) Received: from pps.filterd (iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com (8.18.1.2/8.18.1.2) with ESMTP id 5A3AfW3j039684; Mon, 3 Nov 2025 12:32:29 GMT Received: from sj2pr03cu001.outbound.protection.outlook.com (mail-westusazon11012046.outbound.protection.outlook.com [52.101.43.46]) by iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTPS id 4a58nhux5g-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 03 Nov 2025 12:32:29 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=cInlqrQZY7OWAYy/hjOqK43XpSLGdHC6fcaUKD5MblEWGG0QHIxp8B2pAjd6c35IlbM3l2ErbeEyisEVtvUjqd+H1Umw415+BvyMjksAFofZeiyH6j/Wn8iYMbwBV+wAe2M4WzdL3Q08qVnYWhm9avbFPRyy+Jn0VQuwfgYSzQNWv0A87AV+EiRr3TK4gnl4Neap8cg8q1jHCZnrfq2f8dNrozdgkOeTVJW/g2KgVkpOPUbeBRV2gVT9TD+tDCEwdPNjwDXbU3WVqo+AkKDpBW2+uLOwgwtOgmrVUdSoW2iPrrabUhlCFRe+Bq+hUrbPslDZ36tJXlAVt97k3fMy2g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=oS6cqbGj7LwBiJddYfaMT+LmpgHZcgkQu6AukRJJYtg=; b=DU3PnknfxqVazjbnjGaO85Ud+Tj7i52VU4xXdccHKOS0crQvSjEwxYKfcbtE6AQ6qEQ1KdzcgLU+xSKKmE/jshZD3swdsJtfdMzHqQwbXhrjMmXv7N/bQQ0e7npRiw6x17DdEx58JYW2FK3dBTT+91DdjXeGdIJN8RnsjgV8KGreyEHsYQ6ebq5k6nIRnpk0PXYBCF7lHCtmNF1XLG/KAJ82emZ7aFw40ncxqo+ANk/La08pGsnDtBWRByBqa2rlKxRa5vkqUK8kJkWL5r/XpqseDtPHMq1EaRsFQzWUmEfjN8aSLyxr0yDYSbwZ7yOPPFYPx2EgLaHw7fVFzNPOHQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=oS6cqbGj7LwBiJddYfaMT+LmpgHZcgkQu6AukRJJYtg=; b=xzSzCtTMqVoT6N1CFaOIA9E5/TmOy5oqez6+roZd9/j4MJCeXNFws+3EYdnFlrNHSXVDmiY7sKvzJ7CWPcGJMyHSZ63jijXAQ6qV6ClnUa8ipgU7G4RBAPkZN7ccWHJjrZ5+O7Yo/J1wAWSk/dy/TVwggJh/Iz6yXdaPdBKt65o= Received: from DM4PR10MB8218.namprd10.prod.outlook.com (2603:10b6:8:1cc::16) by DS4PR10MB997575.namprd10.prod.outlook.com (2603:10b6:8:31e::5) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9275.16; Mon, 3 Nov 2025 12:32:25 +0000 Received: from DM4PR10MB8218.namprd10.prod.outlook.com ([fe80::2650:55cf:2816:5f2]) by DM4PR10MB8218.namprd10.prod.outlook.com ([fe80::2650:55cf:2816:5f2%2]) with mapi id 15.20.9253.018; Mon, 3 Nov 2025 12:32:25 +0000 From: Lorenzo Stoakes To: Andrew Morton Cc: Christian Borntraeger , Janosch Frank , Claudio Imbrenda , David Hildenbrand , Alexander Gordeev , Gerald Schaefer , Heiko Carstens , Vasily Gorbik , Sven Schnelle , Peter Xu , Alexander Viro , Christian Brauner , Jan Kara , Arnd Bergmann , Zi Yan , Baolin Wang , "Liam R . Howlett" , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Lance Yang , Muchun Song , Oscar Salvador , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Matthew Brost , Joshua Hahn , Rakie Kim , Byungchul Park , Gregory Price , Ying Huang , Alistair Popple , Axel Rasmussen , Yuanchu Xie , Wei Xu , Kemeng Shi , Kairui Song , Nhat Pham , Baoquan He , Chris Li , SeongJae Park , Matthew Wilcox , Jason Gunthorpe , Leon Romanovsky , Xu Xin , Chengming Zhou , Jann Horn , Miaohe Lin , Naoya Horiguchi , Pedro Falcato , Pasha Tatashin , Rik van Riel , Harry Yoo , Hugh Dickins , linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-s390@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, linux-arch@vger.kernel.org, damon@lists.linux.dev Subject: [PATCH 06/16] fs/proc/task_mmu: refactor pagemap_pmd_range() Date: Mon, 3 Nov 2025 12:31:47 +0000 Message-ID: <15916960ea5479fd2adcf35044fde08bc50d464f.1762171281.git.lorenzo.stoakes@oracle.com> X-Mailer: git-send-email 2.51.0 In-Reply-To: References: Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: LO4P123CA0698.GBRP123.PROD.OUTLOOK.COM (2603:10a6:600:37b::11) To DM4PR10MB8218.namprd10.prod.outlook.com (2603:10b6:8:1cc::16) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DM4PR10MB8218:EE_|DS4PR10MB997575:EE_ X-MS-Office365-Filtering-Correlation-Id: 428713ec-c928-4715-77f5-08de1ad50b75 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|376014|7416014|366016|1800799024; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?8cuWqjzFi0ddvZdgS000T9YUp0bxn34tqcn2yQadbYqEziVF2NJpSqxj0PRu?= =?us-ascii?Q?agwNK6qC9qjudGIWs+XxL40Iea4spGWRmOHeatd1f83a8P28TncS6+WB+QJc?= =?us-ascii?Q?xIpYICEB7fIxdJjDpHAsbtzFOby/2592hLFTODt9LfSunxw8uN48rTdjv/lJ?= =?us-ascii?Q?k+pgDoNc1WrnQC0nNC1bChmu7RGj2sICHbniYJGsDfIbAu9XFyRy2C4ZSvJQ?= =?us-ascii?Q?i5AC2Nf1ygGd48ZFs6N8IFFYCbINPoggxyu7cED8P8+/uuSdXBZWQ6GFbI2x?= =?us-ascii?Q?pzL/UB+K4a5kNrQP87uGMKTbRX9VttWXF/R1PLRg2dM1I9pWuaqG6dPE7Lla?= =?us-ascii?Q?zzX9RretjGIHMlWfVfpwG9xZS1eBH6HEOSKZwV3xbBym9t1N3tVIgi3kmacv?= =?us-ascii?Q?jLmVeWON2D76Nk+NVVSEkcSQkjdaVUZErxvTWL6cnd0t3chQq+TeZAwa69b7?= =?us-ascii?Q?iXcSR7nwlETVGot3/Q9ZI/QWGiZc1wLrZDrDB0AzTWvbqkyR5yH0LocrfhUI?= =?us-ascii?Q?QTiTAZjisnAjS/YDOiozzYXS1NkuoAgI9NrFnDChZOI4SCxmSL+xt4el0lT1?= =?us-ascii?Q?71p8mfEPzHf0f5UAv7sfFD/q66MNgEnoaO46rGxd0XpsSeG2cC+TNyA39RDI?= =?us-ascii?Q?VHqQQmP7MfXx8e4xl3Wc+8qbEIKq0QEwBxLwtGGJdqIPAgrkFgfrEacPR81b?= =?us-ascii?Q?4vSakzsP6Z4BIKKDi3KdmVEMf9yyI3735pxYjxI8jKHGcCE8+ZzIYEqbnAH0?= =?us-ascii?Q?VHDQDkxuRZiuXIATl1W6cmoBTeg7bkALyGKBUGNZgnsNREFC+OxvFAI9jskA?= =?us-ascii?Q?kFztrjdhfqjNm0mt/BE006APC3fcndzK0FxMalgtEiO0VagUAXO5xcErah5N?= =?us-ascii?Q?wQxtD2c+EC/lX02p3WKfTBryMrdAbc9J+5B8DUVguPKtQ/MwwdQpbo1rsx4F?= =?us-ascii?Q?XrW/d8ckU1KXqwPOI67sg3vflLyMZJeTANxqzwbppmZGs7sI9Fa77Jth58nW?= =?us-ascii?Q?IHvkaaTRk4z4wSnIw7f7sP55/vHJgNrEArhdLKZZ9jM2bqFf81c/uJn0amT7?= =?us-ascii?Q?ZbyeE9zeHg3vEtFk/FIW33Cg/jWyTdlkOb/0QukcLpIyldaOUQJTwhKsWujb?= =?us-ascii?Q?34gfCKRiw5SMblGWB/SJnVKymATxllYaS1IQZAAsxMYtaU394X/8Abvky8D2?= =?us-ascii?Q?GJNHHkNkLKYtK8oQ+whWAGfdSbiWBwgBebY3abV/0dTadnxoaiF/aKfOkk7l?= =?us-ascii?Q?4ZUPNu8pz5vyHPXDTWL7XBlyR8EqtjIXdAKsf+DLCqfPSpT9fPQNnWwM5zs9?= =?us-ascii?Q?aeGjM2vbbUtwWAMph+ezWI/x610fv3bZ2MhPrKgYelVCz+ga3qqgN9YCvKyF?= =?us-ascii?Q?7nkmsElzid1gKG+rSmpF3qBCODRvNYk0tOukzjX/cSfIl6Oeh4D3tn/ISpTN?= =?us-ascii?Q?oVCuoBIWDuNHINbkrhNUekkGwL6LBBYc?= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DM4PR10MB8218.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(376014)(7416014)(366016)(1800799024);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?0S7yms/LaH3Q/iFnk8d7YRv9+HqyiEIS5mRioA4RrcVvOJaNFJZ2caPLSt/f?= =?us-ascii?Q?TsxcyMQly/DyWM/P3lVKE+pU6kUHhWsvemnzxfBLNGWD9KFb083eKYBwrBhp?= =?us-ascii?Q?pwrzUovTo3jwy+jcc3dN9EUBNUj/dAmN9up+isaFBcfrUhYRMgZYsduewTog?= =?us-ascii?Q?bT4JDINlD6oZwZzyXDe5JWIdllqL2Ai5UWjf8jw44TotljXBCUXtBprXkl9i?= =?us-ascii?Q?YJUzpno5qTqjWZP89dvblGJEbT4UEyO8EfKBK9QCTZakaGQnB3COp/ivVSWe?= =?us-ascii?Q?9Pz8b545skJQ3reYdVIYtoD3rp7pyTw3KzajnalDdvrpIrN51eS6JCiUjmsC?= =?us-ascii?Q?XpUb7KMGXpOqhTJLKJFgY5Mia+/K181+ci77ONPPB2HCRc3TzzZozGHLQ3hM?= =?us-ascii?Q?EVgrWBA2B2jeUjznCCBC7DxE/FMXwh3N/bSp5aGc/BkyOQ0yLD5pM0lU/4Sq?= =?us-ascii?Q?YGNaeeYaqTww/+uM26+PWDGiej1rpx5figed25V8CuMfzziKoKhiUwxPJSub?= =?us-ascii?Q?tEeUdIUP7l/bvlVBKJ3OQNaM2wsqFkcNqxzmtLIqTVTHHvwI9bXeVTqj68Q/?= =?us-ascii?Q?h27eZDXrteK2qIpG9xbnq4MARR13JYbYgsB3Cue7mojOR7WnaUyk8mHCB17e?= =?us-ascii?Q?VVrZTqciE7CoaYJ8nMuQK7cTHC0QfK7AoaOxDSQdjAJ3fbZM742Bh8rJfjo9?= =?us-ascii?Q?Q+XPvdqZYRg61gsCbbPCFH/mY3NxVeC96QTt0CAagcitoYYIUfhoiJMlED3t?= =?us-ascii?Q?vP54LKek8pUPPEZTgswHgM16EhYzNWq2icTKB8CmMPA6mJz7gDjW/DgCWPRc?= =?us-ascii?Q?2EoJAJW5mLvC6t84NURn3KSY7VFTwyblZdu23GmeHuYuV0ykTV9FrUdzaksj?= =?us-ascii?Q?LnME1F9AR3z4r6LvCcLu4Ezgho6xrqB1niJHgePDv43nAbaRNpH0p2a2wNZ3?= =?us-ascii?Q?8sRCugQ6/4GbqaqFD0PUoWPGTJUQUX8Fg0nG6mZmYFXCPS8hoKDvW+F5TXS0?= =?us-ascii?Q?EgzPM3QUFS07w2/vxa2FE0OcT2ihlqSo2gtUEWC8AaF4CmCEm45a+/7/UaZa?= =?us-ascii?Q?3oTKWBCVypCPEDftX1epZ1BTKfJfxPexs/6cygUXGQQ3vsZ+WcHpahreqbwg?= =?us-ascii?Q?oRupO2lGy2FrFZGbmLM1MOhdsjs/iYzmoR5cptAnKw+u8XqCbUVZ67+dPAm3?= =?us-ascii?Q?+g2zM0ed78OiCnHio3I+4B6SOX6rHkgpjyf7PPpsZgAqoLgYugGQZ0l0mOYP?= =?us-ascii?Q?w0ERhPfCCzXb8YtioADpkVTX6HlfbRdf2IVXSlg1zNrmtje0OzhIe80w1CtW?= =?us-ascii?Q?MT0C8a7NKQpkGMwgaZ54+fcns/QqDPQWU7SvHQhZ1oHkKsihTjt5EjjHwcSn?= =?us-ascii?Q?JP4JUoqEF3y3nEN4MvtI2jXfZ0IlIn+PGYwxVqQDbMkuNQWEx3rDGrgdULYn?= =?us-ascii?Q?ZXNeULTQDCdBsV1IQhB+V2MvR8lVnDhtyQ9WXHpW59O89sgiT7LCxaUzNLaJ?= =?us-ascii?Q?ZGGvbsumTKDaMjHsEl378LO4XBk5ThEXKyDIom/1LplZYu8fcCwmjI7kjXtx?= =?us-ascii?Q?0MNLh104FkSUCo8bMTU54OaT+/gwxVfi/1/AoYrouMIGsvAGmamqOTCkYHP7?= =?us-ascii?Q?NA=3D=3D?= X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: u7ceobia6ss9yznVwGqYiQdfl4xdHuV1g6PvRn1aW9tgTBoLzTdeL1IzlfFSz1TBN8y9SnIxKMhsF354T0o5TX8SE885/4AV+dqpOWaIHhcv+U72isBvVrnDuf6OgAtefQ1VacLFAo9yrLyJ318oNEpUdZOn9gAqaSnR1NtMM57Jeedfl9ZYE35tc1xS9KawfN/uZ2c/F1rD1pM3NvAi5NFb7HokKXP8MTJu25cZKkkmVa6wLJh1zyDwZ+/vUJuu8SLyAWX2O1v/7Ti7471HY48gCIm6FcV+cgonCZwDCTW5dtDSngLDLW/hpW7eXfWKaG4BExk/oqSyGf6bUxNqpigNSSV6C/hwt4J52qnVofJGqneZXSVNDCuDt0u6mEPGcJrj61sP6g975iFJcSiWL3hnEP8YisF/0N6mxKWxqexJPZdtFEvjS+c2KODLEW4O/BQTvPFPZSgrcgwbCJEu0SEEwQqhV19cqa9yHau774MoAtj233it5ytjnvB8iHvzFA27kkNlnjTrVrDXfGLt6tEUOPUZ7HsQ7mFwikwBTXMOMUoTt6gjRd5FJdPrFXGQxsx6Vdh2e4xZtGpTMNXm3UXt7a72YAvHnliMoX7V3Bo= X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: 428713ec-c928-4715-77f5-08de1ad50b75 X-MS-Exchange-CrossTenant-AuthSource: DM4PR10MB8218.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 03 Nov 2025 12:32:25.7752 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: Hxphoep5zRVAm2QzGaaT9zaIOYKcFKRJdpKbUo//1Qbk/W7r92vY6ctSxHoTY/yOSk92friIfihgJXPer37INGWMeK3P40vWkceAmitsZnM= X-MS-Exchange-Transport-CrossTenantHeadersStamped: DS4PR10MB997575 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1121,Hydra:6.1.9,FMLib:17.12.100.49 definitions=2025-11-03_01,2025-11-03_01,2025-10-01_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxlogscore=999 bulkscore=0 spamscore=0 suspectscore=0 mlxscore=0 adultscore=0 phishscore=0 malwarescore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2510240000 definitions=main-2511030113 X-Proofpoint-Spam-Details-Enc: AW1haW4tMjUxMTAzMDExMyBTYWx0ZWRfX8T7eM2Go+2fZ zhDikK+Ep5RC+xD+ZlmUr5zumecGSJrSaW7P93JXdVCeEG5me9XjyZnItb/eMKbrkFMxNX5Ok93 daZfI2qxPHA/63DzSsOL7ytY0TAy9cTpV9z03L4vRWEtjdaGzujaImMNYS3ezfdhtArSkMEEq2q 2UFdVgkls2pVqnEgp9iL7TTaN8eLhvhwGem2LLPTH8osCk8k8m0inVpGzbg0E1RK4i1w7v8mSxB D7MXvbowkVZoBWmFJJSp/sJpxO1xpuCFWnzVemav3DwW6z8aFBS6WQe0E8/PTdMvM5dcW0HJPma K+p4iotJ5s4n67jQjuN1oiYvv6CKv0m4yv4IB7RJ3g7JlCVOGqxQ8/E9oQiFlrvTj7pHDFStIIQ rKIuZxQaQfizSPG6llJSphCzknKqB8wl6HUKZ6/ULHeI0r5sTRs= X-Proofpoint-GUID: G3nVNhYdb54WnVbgv2mpQ0pfZ9iIFoRU X-Proofpoint-ORIG-GUID: G3nVNhYdb54WnVbgv2mpQ0pfZ9iIFoRU X-Authority-Analysis: v=2.4 cv=P7U3RyAu c=1 sm=1 tr=0 ts=6908a0de b=1 cx=c_pps a=qoll8+KPOyaMroiJ2sR5sw==:117 a=qoll8+KPOyaMroiJ2sR5sw==:17 a=6eWqkTHjU83fiwn7nKZWdM+Sl24=:19 a=z/mQ4Ysz8XfWz/Q5cLBRGdckG28=:19 a=lCpzRmAYbLLaTzLvsPZ7Mbvzbb8=:19 a=xqWC_Br6kY4A:10 a=6UeiqGixMTsA:10 a=GoEa3M9JfhUA:10 a=VkNPw1HP01LnGYTKEx00:22 a=yPCof4ZbAAAA:8 a=r4PJAH3pRi85-HfG15IA:9 cc=ntf awl=host:12124 Content-Type: text/plain; charset="utf-8" Separate out THP logic so we can drop an indentation level and reduce the amount of noise in this function. We add pagemap_pmd_range_thp() for this purpose. While we're here, convert the VM_BUG_ON() to a VM_WARN_ON_ONCE() at the same time. No functional change intended. Signed-off-by: Lorenzo Stoakes --- fs/proc/task_mmu.c | 146 ++++++++++++++++++++++++--------------------- 1 file changed, 77 insertions(+), 69 deletions(-) diff --git a/fs/proc/task_mmu.c b/fs/proc/task_mmu.c index 9914febdb60b..bd0042851ba7 100644 --- a/fs/proc/task_mmu.c +++ b/fs/proc/task_mmu.c @@ -1984,90 +1984,98 @@ static pagemap_entry_t pte_to_pagemap_entry(struct = pagemapread *pm, return make_pme(frame, flags); } =20 -static int pagemap_pmd_range(pmd_t *pmdp, unsigned long addr, unsigned lon= g end, - struct mm_walk *walk) +#ifdef CONFIG_TRANSPARENT_HUGEPAGE +static int pagemap_pmd_range_thp(pmd_t *pmdp, unsigned long addr, + unsigned long end, struct vm_area_struct *vma, + struct pagemapread *pm) { - struct vm_area_struct *vma =3D walk->vma; - struct pagemapread *pm =3D walk->private; - spinlock_t *ptl; - pte_t *pte, *orig_pte; + unsigned int idx =3D (addr & ~PMD_MASK) >> PAGE_SHIFT; + u64 flags =3D 0, frame =3D 0; + pmd_t pmd =3D *pmdp; + struct page *page =3D NULL; + struct folio *folio =3D NULL; int err =3D 0; -#ifdef CONFIG_TRANSPARENT_HUGEPAGE =20 - ptl =3D pmd_trans_huge_lock(pmdp, vma); - if (ptl) { - unsigned int idx =3D (addr & ~PMD_MASK) >> PAGE_SHIFT; - u64 flags =3D 0, frame =3D 0; - pmd_t pmd =3D *pmdp; - struct page *page =3D NULL; - struct folio *folio =3D NULL; + if (vma->vm_flags & VM_SOFTDIRTY) + flags |=3D PM_SOFT_DIRTY; =20 - if (vma->vm_flags & VM_SOFTDIRTY) - flags |=3D PM_SOFT_DIRTY; + if (pmd_present(pmd)) { + page =3D pmd_page(pmd); =20 - if (pmd_present(pmd)) { - page =3D pmd_page(pmd); + flags |=3D PM_PRESENT; + if (pmd_soft_dirty(pmd)) + flags |=3D PM_SOFT_DIRTY; + if (pmd_uffd_wp(pmd)) + flags |=3D PM_UFFD_WP; + if (pm->show_pfn) + frame =3D pmd_pfn(pmd) + idx; + } else if (thp_migration_supported() && is_swap_pmd(pmd)) { + swp_entry_t entry =3D pmd_to_swp_entry(pmd); + unsigned long offset; =20 - flags |=3D PM_PRESENT; - if (pmd_soft_dirty(pmd)) - flags |=3D PM_SOFT_DIRTY; - if (pmd_uffd_wp(pmd)) - flags |=3D PM_UFFD_WP; - if (pm->show_pfn) - frame =3D pmd_pfn(pmd) + idx; - } -#ifdef CONFIG_ARCH_ENABLE_THP_MIGRATION - else if (is_swap_pmd(pmd)) { - swp_entry_t entry =3D pmd_to_swp_entry(pmd); - unsigned long offset; - - if (pm->show_pfn) { - if (is_pfn_swap_entry(entry)) - offset =3D swp_offset_pfn(entry) + idx; - else - offset =3D swp_offset(entry) + idx; - frame =3D swp_type(entry) | - (offset << MAX_SWAPFILES_SHIFT); - } - flags |=3D PM_SWAP; - if (pmd_swp_soft_dirty(pmd)) - flags |=3D PM_SOFT_DIRTY; - if (pmd_swp_uffd_wp(pmd)) - flags |=3D PM_UFFD_WP; - VM_BUG_ON(!is_pmd_migration_entry(pmd)); - page =3D pfn_swap_entry_to_page(entry); + if (pm->show_pfn) { + if (is_pfn_swap_entry(entry)) + offset =3D swp_offset_pfn(entry) + idx; + else + offset =3D swp_offset(entry) + idx; + frame =3D swp_type(entry) | + (offset << MAX_SWAPFILES_SHIFT); } -#endif + flags |=3D PM_SWAP; + if (pmd_swp_soft_dirty(pmd)) + flags |=3D PM_SOFT_DIRTY; + if (pmd_swp_uffd_wp(pmd)) + flags |=3D PM_UFFD_WP; + VM_WARN_ON_ONCE(!is_pmd_migration_entry(pmd)); + page =3D pfn_swap_entry_to_page(entry); + } =20 - if (page) { - folio =3D page_folio(page); - if (!folio_test_anon(folio)) - flags |=3D PM_FILE; - } + if (page) { + folio =3D page_folio(page); + if (!folio_test_anon(folio)) + flags |=3D PM_FILE; + } =20 - for (; addr !=3D end; addr +=3D PAGE_SIZE, idx++) { - u64 cur_flags =3D flags; - pagemap_entry_t pme; + for (; addr !=3D end; addr +=3D PAGE_SIZE, idx++) { + u64 cur_flags =3D flags; + pagemap_entry_t pme; =20 - if (folio && (flags & PM_PRESENT) && - __folio_page_mapped_exclusively(folio, page)) - cur_flags |=3D PM_MMAP_EXCLUSIVE; + if (folio && (flags & PM_PRESENT) && + __folio_page_mapped_exclusively(folio, page)) + cur_flags |=3D PM_MMAP_EXCLUSIVE; =20 - pme =3D make_pme(frame, cur_flags); - err =3D add_to_pagemap(&pme, pm); - if (err) - break; - if (pm->show_pfn) { - if (flags & PM_PRESENT) - frame++; - else if (flags & PM_SWAP) - frame +=3D (1 << MAX_SWAPFILES_SHIFT); - } + pme =3D make_pme(frame, cur_flags); + err =3D add_to_pagemap(&pme, pm); + if (err) + break; + if (pm->show_pfn) { + if (flags & PM_PRESENT) + frame++; + else if (flags & PM_SWAP) + frame +=3D (1 << MAX_SWAPFILES_SHIFT); } + } + return err; +} +#endif /* CONFIG_TRANSPARENT_HUGEPAGE */ + +static int pagemap_pmd_range(pmd_t *pmdp, unsigned long addr, unsigned lon= g end, + struct mm_walk *walk) +{ + struct vm_area_struct *vma =3D walk->vma; + struct pagemapread *pm =3D walk->private; + spinlock_t *ptl; + pte_t *pte, *orig_pte; + int err =3D 0; + +#ifdef CONFIG_TRANSPARENT_HUGEPAGE + ptl =3D pmd_trans_huge_lock(pmdp, vma); + if (ptl) { + err =3D pagemap_pmd_range_thp(pmdp, addr, end, vma, pm); spin_unlock(ptl); return err; } -#endif /* CONFIG_TRANSPARENT_HUGEPAGE */ +#endif =20 /* * We can assume that @vma always points to a valid one and @end never --=20 2.51.0 From nobody Sat Feb 7 17:55:59 2026 Received: from mx0b-00069f02.pphosted.com (mx0b-00069f02.pphosted.com [205.220.177.32]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id AAF8730F957; Mon, 3 Nov 2025 12:34:13 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=205.220.177.32 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1762173256; cv=fail; b=CoZy26Jvxap/vHIPNOC/xfsvvTWlr+ZwdKUPD8ssvtKEPmLoTo5iThCsugvYiVlFHjlo4W6ZTjR+U+nECrBXpWJrYsRTIwtyKwG1yOtnsdvgu2X5TfABSi7T8wWTKdQ5og0cvj4ck9armXiwWIz7UpKFKVZqSNvOgLiBW0N2/E8= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1762173256; c=relaxed/simple; bh=JsENxz2x8ib+itdjKINv3ZO8YbOGvYnldUFO4Xb0BtM=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: Content-Type:MIME-Version; b=TfuHdNrW8FSw6GUuEMRhv2x482lzarRBRYbFEIf9UW3ipCnxOzTOvEVyWTak/FI7pdO5cqftEZ3BSR+OK7UMPqHxY5MJ/hzhgIR2khYW/2t0f7L/AU6lMq5/y35M41qxODVhaY4p9Pc0Ma1bRjsG0PAIXDesBy7zbtphzI4QXZ4= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=oracle.com; spf=pass smtp.mailfrom=oracle.com; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b=lFg2yvNU; dkim=pass (1024-bit key) header.d=oracle.onmicrosoft.com header.i=@oracle.onmicrosoft.com header.b=ipWc10HK; arc=fail smtp.client-ip=205.220.177.32 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=oracle.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=oracle.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b="lFg2yvNU"; dkim=pass (1024-bit key) header.d=oracle.onmicrosoft.com header.i=@oracle.onmicrosoft.com header.b="ipWc10HK" Received: from pps.filterd (m0246632.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 5A3COjmw009969; Mon, 3 Nov 2025 12:32:31 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=cc :content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s= corp-2025-04-25; bh=pAKpiFuYj0Y2wnZNTmgZYg5+xZ73FBBzAh9o7DPvX8g=; b= lFg2yvNUX+c0o+alNp/lhVnP7zNev/3s8XE5bCctv4eCFJ9JUSWDKJWObQQ+qTrb /cmdh40h4KxKmawqO2WsYmlR1k201oOij9G6wMGCFgKXMZnLerkzsPRpYy7ILyjT Gd2DupNU6Y72BJRxjPoRdSZ3lFneJMs9GDnJoTLBcYD3+ynDkd/v13h0LabD3q2N R3Q9CE0u6yJhCuE3uWM1AbgtX1oCpaif6e3pUI4FRgIGIKvLlqQAkyTTeHwkJyeP O3FBYC4VSQMkHsqlAdZqDYHltdgdVM9tpLmDu0/wxXqonCNMj3Jg5LOHYyUqVdy5 Q7nAwQabIwde5A293OR/jg== Received: from iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta03.appoci.oracle.com [130.35.103.27]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 4a6vca00b0-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 03 Nov 2025 12:32:31 +0000 (GMT) Received: from pps.filterd (iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com (8.18.1.2/8.18.1.2) with ESMTP id 5A3AfW3l039684; Mon, 3 Nov 2025 12:32:31 GMT Received: from sj2pr03cu001.outbound.protection.outlook.com (mail-westusazon11012046.outbound.protection.outlook.com [52.101.43.46]) by iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTPS id 4a58nhux5g-2 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 03 Nov 2025 12:32:30 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=jatM09FTrt4CUPq6WWg7HrrUh2FWpa5g7vQc19om+FRQwhHeGqK8YmIfi0L30ruDHxGC0pBz5R9Fs38aaeArT94gvhtEX3939Z0dC9W6bmq5kSPG5ttzCtj43quu07031snMmQfHi+yavpaRsS9ZRHL0BRtk/27JBbfkB20+k1I0iy+1xhEqciJ3ovzakbWsK3jmh4xaNzasbFkb8+QeL4MMn8Fib8iqWP2TVai+j12E3x1tNGu0jTpmRaAwL0q9pLCYjAy0G6/qKQlnEMq8O7CKcxJKRsuT2Mutf02xhbHFS5RTIjunNln2o23isNl0KAKzgIkNqHHUxadkdpDX1w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=pAKpiFuYj0Y2wnZNTmgZYg5+xZ73FBBzAh9o7DPvX8g=; b=MO6dndXIKx+7SSpTv4sIizezn1UErszdiZUk36gvb+MVo2y0AqvpaNKbj59V2j0qzQzf2gf63+q3l11+L9evEj6yr1C8Qt9yRhxGkg1JIyZ8NXiMkId7K6WPnNYRi80rATKk/nbDzYqWzO+mgSn6iwJ9pIuNLzEQJejw5ZfVFZqWjckIMlWGzn+ft/+w4CxI5o/QV3FMsoo0C0AT3VlG+PsK4mA2w7MMOPKdIN9Zi+gegWKUqwHaHSY4EhPK/VNWwikVTMq0aRgGQiWUhjLwNrCr80mGOEf+wwbg4PDlRETvSK/tVvT2rufd0b+D9fBS+UY+rMspySom7CLYv9Iycw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=pAKpiFuYj0Y2wnZNTmgZYg5+xZ73FBBzAh9o7DPvX8g=; b=ipWc10HK2DwmaZ/8d1ZXxMKiMM4m1EuXll+byG37m3vI8QMzRX7/9tDNkaM6Eabha5poaRy/19q00wRWrQOK6cHZP0fsTfIxJz4yjYv1KCiZq6C+CO4vcgcUUdsL2+3joGVWRIscPOfJVDom0+QpdIKhl/oPAyC/ZJQ5B7fDX/U= Received: from DM4PR10MB8218.namprd10.prod.outlook.com (2603:10b6:8:1cc::16) by DS4PR10MB997575.namprd10.prod.outlook.com (2603:10b6:8:31e::5) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9275.16; Mon, 3 Nov 2025 12:32:28 +0000 Received: from DM4PR10MB8218.namprd10.prod.outlook.com ([fe80::2650:55cf:2816:5f2]) by DM4PR10MB8218.namprd10.prod.outlook.com ([fe80::2650:55cf:2816:5f2%2]) with mapi id 15.20.9253.018; Mon, 3 Nov 2025 12:32:28 +0000 From: Lorenzo Stoakes To: Andrew Morton Cc: Christian Borntraeger , Janosch Frank , Claudio Imbrenda , David Hildenbrand , Alexander Gordeev , Gerald Schaefer , Heiko Carstens , Vasily Gorbik , Sven Schnelle , Peter Xu , Alexander Viro , Christian Brauner , Jan Kara , Arnd Bergmann , Zi Yan , Baolin Wang , "Liam R . Howlett" , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Lance Yang , Muchun Song , Oscar Salvador , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Matthew Brost , Joshua Hahn , Rakie Kim , Byungchul Park , Gregory Price , Ying Huang , Alistair Popple , Axel Rasmussen , Yuanchu Xie , Wei Xu , Kemeng Shi , Kairui Song , Nhat Pham , Baoquan He , Chris Li , SeongJae Park , Matthew Wilcox , Jason Gunthorpe , Leon Romanovsky , Xu Xin , Chengming Zhou , Jann Horn , Miaohe Lin , Naoya Horiguchi , Pedro Falcato , Pasha Tatashin , Rik van Riel , Harry Yoo , Hugh Dickins , linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-s390@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, linux-arch@vger.kernel.org, damon@lists.linux.dev Subject: [PATCH 07/16] mm: avoid unnecessary use of is_swap_pmd() Date: Mon, 3 Nov 2025 12:31:48 +0000 Message-ID: <0f4219d2d682f1447b2ce0f87f5f5d935d418348.1762171281.git.lorenzo.stoakes@oracle.com> X-Mailer: git-send-email 2.51.0 In-Reply-To: References: Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: LO4P302CA0034.GBRP302.PROD.OUTLOOK.COM (2603:10a6:600:317::17) To DM4PR10MB8218.namprd10.prod.outlook.com (2603:10b6:8:1cc::16) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DM4PR10MB8218:EE_|DS4PR10MB997575:EE_ X-MS-Office365-Filtering-Correlation-Id: 96d851e1-085f-4e59-887c-08de1ad50d1e X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|376014|7416014|366016|1800799024; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?MyQ6sEFdDXIf/1OdKicn/idRNtijrMfJsUqwLKnw9kGUldJQu7cHOfKfTDBr?= =?us-ascii?Q?OAc80Kksc5KhV5QZ1nGWFq5Hpb+DTaT36bD4WQDfg++pcwgPKTAxV4XPgc75?= =?us-ascii?Q?XFlaDLGXhx2QwHcswCtkmlQ6MtVt0lNjPnEBiWEj5c/Yfl0Dqzk2hGfrgAiV?= =?us-ascii?Q?6dZnz3uAB/cV39K7usj7JWqfP0wUrFBVOi1tmzljlCvU+S4LlG3NCAdUUG1/?= =?us-ascii?Q?bgbDMbknCEoQSOeeG3aa1ZpNhng8hgZKCwZJU9npKXr41Gu6hdqjYNfnMChM?= =?us-ascii?Q?87hqt0UdeyJWXu5C8tQLVNA0CvaZ6zm5+yvRvnyfZ9WSz4oJTQfr0Y0y3RuP?= =?us-ascii?Q?l0SKgi0SBvHzOgpg5dHFtX5+Tey3DytG+KqhXvvppKzb90Taalmto6gl5n0z?= =?us-ascii?Q?3JJGhGFZpYSFFhGxWrDo+f5nXiReDqSfyAZta6y8l13M0dbj6XXA4/+6xkd7?= =?us-ascii?Q?awe1p33gfLoiGKFzs/vNasNJMCVcBGC2LCOehJqV9a0vj/LYOdZoC5cHz8sq?= =?us-ascii?Q?5btAc6GaOC0sIEalCHPIhNRSjQhKHIviXSxb+JXhiZN8TNtbtx6iodqARj7T?= =?us-ascii?Q?Msn/1hwKnfbOg54EOXxqgF+C2jWoKtBDkdkomViX3xlK/wkqelQCzQkG8FAt?= =?us-ascii?Q?AqmIQ2KP0I3b9DXEUV8j+nUX/E1is7Ix9unYL57gEbYOPaJzAgIB/1VNGtHn?= =?us-ascii?Q?KVKXmvIwdf80XPyd0u/U6fQJiaUlh5KlEYV9bgXtj9eYO8WhufBPuJhF+6bj?= =?us-ascii?Q?Uo1DEOi+wD41eWcDD6Vdk2LUJH/M8oJZJw3Sxo6x+lc3N9d526OQ3rDrogLI?= =?us-ascii?Q?dEk0TfKIAcIrTjp1vhmdec0YFtv0DNz2ddg/oozIp5WP22DK653q2qyHAmQ9?= =?us-ascii?Q?yVKH4f6vHM93y4B2Oz8j0G3u8G5MRiuaL8mzwPS7WfbhkQIeFLDsVcCziwO3?= =?us-ascii?Q?I0BmJizhIs2wtMRlKRkGWJaT270cYFq0dADY518wsih4YP1mZuaPFvJCzqha?= =?us-ascii?Q?5ixIzmsSQXvJWLJxb8Rqr96KR8uogFrErc++A22R7A3hZLQ1+Np7yl2WS3XN?= =?us-ascii?Q?FMQo8ZQO3W7hZHj3FpUHZFZXx6AFF3aNAu8U/uhgmiNDfo5GDJ2iWmqoU9Zg?= =?us-ascii?Q?Sl7897/6H3fEdH85J60TbtlwM6CYEbwVYd1BISVgrzog0ESSwk2zzw8//f6k?= =?us-ascii?Q?sW0o56y8mPn+iMC7P2w2+msLpUonzJxzWjsqtD7VH5VHhhIKo0wFG6itv6ee?= =?us-ascii?Q?txpAW/uVc6EO65k6uRFXXf78wRg3AU9EoaIUsrBXJ8U1v5XfH1b4IRD/hoQZ?= =?us-ascii?Q?ofbvkbGnxzmncFK7eKcVI6f4uDdcrPBen6vsvgkFP/hX0sNYc43VM6sbbN54?= =?us-ascii?Q?bB8BVWerPnbdByuO5AxXMPOioTs06hJLl+hlbCqINJf5Yyi2vAV+i9I/P1wK?= =?us-ascii?Q?TAomx9b4zcj3iVgaqeSHAqvxuGyYO1cZ?= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DM4PR10MB8218.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(376014)(7416014)(366016)(1800799024);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?iCp2ByX+br775fVCIsAFnA+gK+9+QNMQhzyU3VujdvFUfayx6PvtAISTI11r?= =?us-ascii?Q?mZ/m9PxLpnpXxzUdidPyW2u2Y0Gv/aOjySRNXzkZDPYCsoxNx7EWJtCzkIyW?= =?us-ascii?Q?hABWxg+wCteDl6+JsKMNJf8CzYf7FI20SHMyLuLjhmRwvAiV+94eZa19h4+x?= =?us-ascii?Q?552jH1oy3VsiPiYJQzk2MlKRPTWgfMc1Vb+yMyKq6JBWUcxHC7Ow3Ru8aGBh?= =?us-ascii?Q?E/o9+CrXNKlCEu5SmBkrUsf4whsj2xgt3P6zjANicpu9EXb/+RZAe2gTM0Xb?= =?us-ascii?Q?1AZSMCRuNXMrrnAtS70Tye/Gzs9v8h1ac4SO+SN8DXScI38ScdZor0NCwAwq?= =?us-ascii?Q?ElGELzo9b0vjrNv+fPRRAeZOH8GDWTrYcMWQ7vWjVplGt/JJklGpx+WUiaAU?= =?us-ascii?Q?4tFwK4nIqXwrZijX8LBWzIFMZH5nRCtLxZRXIKinzCbJacfd/wy2y2mAdqFI?= =?us-ascii?Q?zfpWX+PEK7KbPYB8fNMY3zIDaXB1/8FIm4QSHwm9hNtlxusV/g0BodID+EY2?= =?us-ascii?Q?cae5Zp8tosQg1vgKKCAk27/ApiPrSZKTvgrZ16YNercD38K7BeiAgz+3PXJ6?= =?us-ascii?Q?iUxmhrFHo7GZhWn6N2o32WhkbtkCfrIM2wk5K/renCA0X/z5ZYq3BVz2mtvg?= =?us-ascii?Q?EJ/Q6/pfQckJztYtesrGougkDgaErUbt9aLjuJxqncvY+RuxhCCM40U25fKA?= =?us-ascii?Q?buFFGYQrdjmMzrI2fVDfoywA1f5rGqvWQ1Ix/xN3IGMUzSGpY6XfQ/FXr2Q+?= =?us-ascii?Q?SGvLLUcsj+cTtdJDqmdXauGAyzuiTpQudUh33nWhgY9IoKaKrG5H7NHesy4r?= =?us-ascii?Q?sv3ming5LczgkrxFgKNnrU5oSN2I3//QA3CKfK7IXODQRy4YvM0EyKJAri45?= =?us-ascii?Q?rsKP1BeZPZ50bmFqReDTb6akNOlwnMRHjGCDo9f5hZD9OTCtZhofOnMSipIW?= =?us-ascii?Q?wHNcXIY8gy2iU8RfWjC1q+7qKJOBYcddo4aXGuSuH9yxK+7eTxE2ydOVpliQ?= =?us-ascii?Q?oNaJXCl+z2Y8KYOOvxkG6t+eZ6PE/eWcMF+LMumgB16W91RQ9fcw94kqn821?= =?us-ascii?Q?rrfHr2dHzMQvUjmYjOp56yHqAJ6wrb4lzBGIl+MznBEp2boXUMM1Ue9pk7Ts?= =?us-ascii?Q?bsQOzKCp4yXG5d4d2AdDHsE9PCcag8PU7vSAibY96OwjtkcEtyqdqb+FanUH?= =?us-ascii?Q?wWoABlSRa6mD6o3XUTcLtPCG08lPl53X/BiUk0KsVhjRCl5446JsYcNvGyWL?= =?us-ascii?Q?zU5ijpvYN1PQYWwPbtNGtkwXgLuaWkeNNvarO4V8zAXa94flbyMsr2t5Qd2L?= =?us-ascii?Q?+wGiOtltZf9FMH41VLUJte38S6STxI/BxQiVJYJ2NcLPzSP168xAC+1QUVHQ?= =?us-ascii?Q?lEmciDyeIY1alQcgYlqxPM0ewxpF+aECCQfgzmM/vw/211TZdXFKdFzsA/2G?= =?us-ascii?Q?62aCrhE1AiJ/HKcz//i7hHOMBvUK7Rpg1R1Qgd1Md97PCf+sLWKShxHwdN5C?= =?us-ascii?Q?x0sXa/tTygUNr6XKtGSgtB56cdF1oOHoVs89jyZ6DWL1XZSEe+o8mDOmqDae?= =?us-ascii?Q?AgB3UyiPzq06ziEY0NGSQUEIEtkGu9II+kIMaciilwO9fdGAt+zkAP2Tuuka?= =?us-ascii?Q?5Q=3D=3D?= X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: EhZ6+jT+AmXFJT4RLWYQQ/pv3WlqREkLPUQeIB+STUpOBTYmaPZjzWpONCXNVUip3SBd0q+8JOWsm2/dsdcNzeQVCqoyFEEFph+Xc81Hy+kMrYi1G9ztKEy4uNA1RJVs4TY+C7vKAPicipuzZxRUglrvapC3hsiRUJTK0+I2/cpsHk5xV5JiSufWJszXbH5S6yBE+yV229LCuHI3LTRQpjeUdY1F1KGdyBd/q25Gz/gMXwUqnzyMJhD48O3UrRMN3BPZNOt49wLVAdH5dHp9+IyCwCaJ0z6fH4oJaRWXVElm9TD/GryLISaljM6nuxwEOQp7el95rIN28xq4PmJI1cwf5w9L7FDczbG0vFwtwOco6Q5Ne+7fFGqNAxH5x/G6oGluf+tnSscDtZv7TRH8hDSD+WdDm0aSaWv/SXLJCcD6jcIRjJGPvJcxy24uEZJCysrNbw7w3/hYG+daFR+0zGdWwH9NSbUAqHRFrim6NoXj2ob10f5J2bEJb2Jkj1lM9tpwzR6G4NXkYlRcPU/KcBsv7polc0AHkS+z9DmELadv114g49Up1DmZ3Vp97rP1y9mhRRGMQaGWdiG1MwowKb0Rjs6+ntW2sDO00qKarEk= X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: 96d851e1-085f-4e59-887c-08de1ad50d1e X-MS-Exchange-CrossTenant-AuthSource: DM4PR10MB8218.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 03 Nov 2025 12:32:28.5339 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: GpeQ0CtLOcKgYkZzklRiP1k4zaArkzG9vTkZJTXzFookm4BEEkY/iXyVMgHIkiTco+luz8b4nfCVQb/i/8dAv2EzSniRVCv/8wYtJSXlp0c= X-MS-Exchange-Transport-CrossTenantHeadersStamped: DS4PR10MB997575 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1121,Hydra:6.1.9,FMLib:17.12.100.49 definitions=2025-11-03_01,2025-11-03_01,2025-10-01_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxlogscore=999 bulkscore=0 spamscore=0 suspectscore=0 mlxscore=0 adultscore=0 phishscore=0 malwarescore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2510240000 definitions=main-2511030113 X-Proofpoint-GUID: qDs-bmIrgyBhtGt8crFHpPeNuxn1JQqm X-Authority-Analysis: v=2.4 cv=PPsCOPqC c=1 sm=1 tr=0 ts=6908a0df b=1 cx=c_pps a=qoll8+KPOyaMroiJ2sR5sw==:117 a=qoll8+KPOyaMroiJ2sR5sw==:17 a=6eWqkTHjU83fiwn7nKZWdM+Sl24=:19 a=z/mQ4Ysz8XfWz/Q5cLBRGdckG28=:19 a=lCpzRmAYbLLaTzLvsPZ7Mbvzbb8=:19 a=xqWC_Br6kY4A:10 a=6UeiqGixMTsA:10 a=GoEa3M9JfhUA:10 a=VkNPw1HP01LnGYTKEx00:22 a=yPCof4ZbAAAA:8 a=96_WKPqrBKIKR3zHGkMA:9 cc=ntf awl=host:12124 X-Proofpoint-ORIG-GUID: qDs-bmIrgyBhtGt8crFHpPeNuxn1JQqm X-Proofpoint-Spam-Details-Enc: AW1haW4tMjUxMTAzMDExMyBTYWx0ZWRfX6LX1PmFZUFmP jKfsIMn/RLh0eXoHzLvCEds9JCdaeZJxHDneN9O1Ilh4/gZWceZdLTJUO33kb64H8fpwN1G3Zaf uBMk3NQhpFh+AIwsufzji9Hf93YMtFejrZP9zHE57poUn96dsOMK9qNIQmVa0xzd2v8hhcli8fi YIcIGRb2ZkLpTp5mFklyyLG2npJsb3lIVSX6xLfts5M52Mrg59fuMBVQtONnGaIRoBMLWBY8R3O Cv4pVmb1bzVwwnrtqKT8AOFVesj5OI0nchr903GFrRflcQMg7TL+vhAjA9bn7MKL9O1veaSJJo+ O4XBnV7kqlulx5LmrSpaqR8TUfi8hS+3rmv/ApdTwU9p2dsknHstIXZtqPebHvggWheSkgwDvik c7+d0iBtnfeEG0RwY3xeKedI5oOWgLyzY0By/4qCs9PADqu2qqE= Content-Type: text/plain; charset="utf-8" PMD 'non-swap' swap entries are currently used for PMD-level migration entries and device private entries. To add to the confusion in this terminology we use is_swap_pmd() in an inconsistent way similar to how is_swap_pte() was being used - sometimes adopting the convention that pmd_none(), !pmd_present() implies PMD 'swap' entry, sometimes not. This patch handles the low-hanging fruit of cases where we can simply substitute other predicates for is_swap_pmd(). No functional change intended. Signed-off-by: Lorenzo Stoakes --- fs/proc/task_mmu.c | 15 ++++++++++--- include/linux/swapops.h | 16 +++++++++++-- mm/huge_memory.c | 4 +++- mm/memory.c | 50 +++++++++++++++++++++++------------------ mm/page_table_check.c | 12 ++++++---- 5 files changed, 65 insertions(+), 32 deletions(-) diff --git a/fs/proc/task_mmu.c b/fs/proc/task_mmu.c index bd0042851ba7..0f02bda5d544 100644 --- a/fs/proc/task_mmu.c +++ b/fs/proc/task_mmu.c @@ -1059,10 +1059,12 @@ static void smaps_pmd_entry(pmd_t *pmd, unsigned lo= ng addr, bool present =3D false; struct folio *folio; =20 + if (pmd_none(*pmd)) + return; if (pmd_present(*pmd)) { page =3D vm_normal_page_pmd(vma, addr, *pmd); present =3D true; - } else if (unlikely(thp_migration_supported() && is_swap_pmd(*pmd))) { + } else if (unlikely(thp_migration_supported())) { swp_entry_t entry =3D pmd_to_swp_entry(*pmd); =20 if (is_pfn_swap_entry(entry)) @@ -1999,6 +2001,9 @@ static int pagemap_pmd_range_thp(pmd_t *pmdp, unsigne= d long addr, if (vma->vm_flags & VM_SOFTDIRTY) flags |=3D PM_SOFT_DIRTY; =20 + if (pmd_none(pmd)) + goto populate_pagemap; + if (pmd_present(pmd)) { page =3D pmd_page(pmd); =20 @@ -2009,7 +2014,7 @@ static int pagemap_pmd_range_thp(pmd_t *pmdp, unsigne= d long addr, flags |=3D PM_UFFD_WP; if (pm->show_pfn) frame =3D pmd_pfn(pmd) + idx; - } else if (thp_migration_supported() && is_swap_pmd(pmd)) { + } else if (thp_migration_supported()) { swp_entry_t entry =3D pmd_to_swp_entry(pmd); unsigned long offset; =20 @@ -2036,6 +2041,7 @@ static int pagemap_pmd_range_thp(pmd_t *pmdp, unsigne= d long addr, flags |=3D PM_FILE; } =20 +populate_pagemap: for (; addr !=3D end; addr +=3D PAGE_SIZE, idx++) { u64 cur_flags =3D flags; pagemap_entry_t pme; @@ -2398,6 +2404,9 @@ static unsigned long pagemap_thp_category(struct page= map_scan_private *p, { unsigned long categories =3D PAGE_IS_HUGE; =20 + if (pmd_none(pmd)) + return categories; + if (pmd_present(pmd)) { struct page *page; =20 @@ -2415,7 +2424,7 @@ static unsigned long pagemap_thp_category(struct page= map_scan_private *p, categories |=3D PAGE_IS_PFNZERO; if (pmd_soft_dirty(pmd)) categories |=3D PAGE_IS_SOFT_DIRTY; - } else if (is_swap_pmd(pmd)) { + } else { swp_entry_t swp; =20 categories |=3D PAGE_IS_SWAPPED; diff --git a/include/linux/swapops.h b/include/linux/swapops.h index a66ac4f2105c..3e8dd6ea94dd 100644 --- a/include/linux/swapops.h +++ b/include/linux/swapops.h @@ -509,7 +509,13 @@ static inline pmd_t swp_entry_to_pmd(swp_entry_t entry) =20 static inline int is_pmd_migration_entry(pmd_t pmd) { - return is_swap_pmd(pmd) && is_migration_entry(pmd_to_swp_entry(pmd)); + swp_entry_t entry; + + if (pmd_present(pmd)) + return 0; + + entry =3D pmd_to_swp_entry(pmd); + return is_migration_entry(entry); } #else /* CONFIG_ARCH_ENABLE_THP_MIGRATION */ static inline int set_pmd_migration_entry(struct page_vma_mapped_walk *pvm= w, @@ -557,7 +563,13 @@ static inline int is_pmd_migration_entry(pmd_t pmd) */ static inline int is_pmd_device_private_entry(pmd_t pmd) { - return is_swap_pmd(pmd) && is_device_private_entry(pmd_to_swp_entry(pmd)); + swp_entry_t entry; + + if (pmd_present(pmd)) + return 0; + + entry =3D pmd_to_swp_entry(pmd); + return is_device_private_entry(entry); } =20 #else /* CONFIG_ZONE_DEVICE && CONFIG_ARCH_ENABLE_THP_MIGRATION */ diff --git a/mm/huge_memory.c b/mm/huge_memory.c index f6c353a8d7bd..2e5196a68f14 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -2429,9 +2429,11 @@ static pmd_t move_soft_dirty_pmd(pmd_t pmd) =20 static pmd_t clear_uffd_wp_pmd(pmd_t pmd) { + if (pmd_none(pmd)) + return pmd; if (pmd_present(pmd)) pmd =3D pmd_clear_uffd_wp(pmd); - else if (is_swap_pmd(pmd)) + else pmd =3D pmd_swp_clear_uffd_wp(pmd); =20 return pmd; diff --git a/mm/memory.c b/mm/memory.c index 299ce5dcba76..a0ae4e23d487 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -1376,6 +1376,7 @@ copy_pmd_range(struct vm_area_struct *dst_vma, struct= vm_area_struct *src_vma, next =3D pmd_addr_end(addr, end); if (is_swap_pmd(*src_pmd) || pmd_trans_huge(*src_pmd)) { int err; + VM_BUG_ON_VMA(next-addr !=3D HPAGE_PMD_SIZE, src_vma); err =3D copy_huge_pmd(dst_mm, src_mm, dst_pmd, src_pmd, addr, dst_vma, src_vma); @@ -6350,35 +6351,40 @@ static vm_fault_t __handle_mm_fault(struct vm_area_= struct *vma, if (pmd_none(*vmf.pmd) && thp_vma_allowable_order(vma, vm_flags, TVA_PAGEFAULT, PMD_ORDER)) { ret =3D create_huge_pmd(&vmf); - if (!(ret & VM_FAULT_FALLBACK)) + if (ret & VM_FAULT_FALLBACK) + goto fallback; + else return ret; - } else { - vmf.orig_pmd =3D pmdp_get_lockless(vmf.pmd); + } =20 - if (unlikely(is_swap_pmd(vmf.orig_pmd))) { - if (is_pmd_device_private_entry(vmf.orig_pmd)) - return do_huge_pmd_device_private(&vmf); + vmf.orig_pmd =3D pmdp_get_lockless(vmf.pmd); + if (pmd_none(vmf.orig_pmd)) + goto fallback; =20 - if (is_pmd_migration_entry(vmf.orig_pmd)) - pmd_migration_entry_wait(mm, vmf.pmd); - return 0; - } - if (pmd_trans_huge(vmf.orig_pmd)) { - if (pmd_protnone(vmf.orig_pmd) && vma_is_accessible(vma)) - return do_huge_pmd_numa_page(&vmf); + if (unlikely(!pmd_present(vmf.orig_pmd))) { + if (is_pmd_device_private_entry(vmf.orig_pmd)) + return do_huge_pmd_device_private(&vmf); =20 - if ((flags & (FAULT_FLAG_WRITE|FAULT_FLAG_UNSHARE)) && - !pmd_write(vmf.orig_pmd)) { - ret =3D wp_huge_pmd(&vmf); - if (!(ret & VM_FAULT_FALLBACK)) - return ret; - } else { - huge_pmd_set_accessed(&vmf); - return 0; - } + if (is_pmd_migration_entry(vmf.orig_pmd)) + pmd_migration_entry_wait(mm, vmf.pmd); + return 0; + } + if (pmd_trans_huge(vmf.orig_pmd)) { + if (pmd_protnone(vmf.orig_pmd) && vma_is_accessible(vma)) + return do_huge_pmd_numa_page(&vmf); + + if ((flags & (FAULT_FLAG_WRITE|FAULT_FLAG_UNSHARE)) && + !pmd_write(vmf.orig_pmd)) { + ret =3D wp_huge_pmd(&vmf); + if (!(ret & VM_FAULT_FALLBACK)) + return ret; + } else { + huge_pmd_set_accessed(&vmf); + return 0; } } =20 +fallback: return handle_pte_fault(&vmf); } =20 diff --git a/mm/page_table_check.c b/mm/page_table_check.c index 43f75d2f7c36..f5f25e120f69 100644 --- a/mm/page_table_check.c +++ b/mm/page_table_check.c @@ -215,10 +215,14 @@ EXPORT_SYMBOL(__page_table_check_ptes_set); =20 static inline void page_table_check_pmd_flags(pmd_t pmd) { - if (pmd_present(pmd) && pmd_uffd_wp(pmd)) - WARN_ON_ONCE(pmd_write(pmd)); - else if (is_swap_pmd(pmd) && pmd_swp_uffd_wp(pmd)) - WARN_ON_ONCE(swap_cached_writable(pmd_to_swp_entry(pmd))); + if (pmd_present(pmd)) { + if (pmd_uffd_wp(pmd)) + WARN_ON_ONCE(pmd_write(pmd)); + } else if (pmd_swp_uffd_wp(pmd)) { + swp_entry_t entry =3D pmd_to_swp_entry(pmd); + + WARN_ON_ONCE(swap_cached_writable(entry)); + } } =20 void __page_table_check_pmds_set(struct mm_struct *mm, pmd_t *pmdp, pmd_t = pmd, --=20 2.51.0 From nobody Sat Feb 7 17:55:59 2026 Received: from mx0a-00069f02.pphosted.com (mx0a-00069f02.pphosted.com [205.220.165.32]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9276C30CDB5; Mon, 3 Nov 2025 12:34:53 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=205.220.165.32 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1762173295; cv=fail; b=Vn+4PCtpoZHNiBXIQNsUAQlxkcx47EoIrhKcNQEyePEjid9lITjsIqGXAekO/hVqi+6Wuwty4M5Ravgt7ufKtUWLNH3i2132JdYqd9amZNtw/vztapzk1ccxUYqjuYhWqNNJeymaTxYpohgj8bQAVmebqcIox/Aq8I2uzOC9dBA= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1762173295; c=relaxed/simple; bh=AZ3s0B+uhePQZXTS6i9X9ezX6E6ciUS6QFxCmlxX9io=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: Content-Type:MIME-Version; b=R2feE0q9t6BS3TyOlp9UV4flhRQcHTg44n/JsIxfsOpAGXpG8wcjsydY4XoYITACH5lTm92LH88NFkvGI5wdE0jOhB24yxTrfv10Rdlnf/InnfPAxdTqoa+vQnO9l7HXLW9mntg1OlkE2O8/ktmg9SnlO0/mkW/fR7196tPO2gM= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=oracle.com; spf=pass smtp.mailfrom=oracle.com; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b=S0Iwumke; dkim=pass (1024-bit key) header.d=oracle.onmicrosoft.com header.i=@oracle.onmicrosoft.com header.b=jisatVVU; arc=fail smtp.client-ip=205.220.165.32 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=oracle.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=oracle.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b="S0Iwumke"; dkim=pass (1024-bit key) header.d=oracle.onmicrosoft.com header.i=@oracle.onmicrosoft.com header.b="jisatVVU" Received: from pps.filterd (m0246629.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 5A3CTppr011573; Mon, 3 Nov 2025 12:32:36 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=cc :content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s= corp-2025-04-25; bh=vzCGIt3sJ5ONOndZRuqYi7OVSaXyL8N0gC2xsfawbow=; b= S0IwumkeSXNSIV1j+VErMC8UGnfcVwReF4OSLQgNQ52mRRs/b4x4J0cW9Nk6xYd4 YcasJmsKSX24Q2xI6IEmBg+CgS9D8KRvwS/z4An0d3RdYZ/wwegHTdAsUZdo6HZR gXqfhgFxQMpvAtZszr38uCLhhrBWzJrhtaH2NK4ugMTLw/quOc27TYNR2iN/QKiA KKsp8nNx3C4cPUgThoqkF9QxofXrdbe6nxj8GdeZbSf6yMXdL6XxFZPhUUXUhPBZ bPYTdaRbJosme0RgcfHvaItoqnBPmZTUgxoixwpgY6qKOOkEsSdjoWVTsjYT8//I YOyzOGU1vcl8NUgCjiJsCQ== Received: from iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta01.appoci.oracle.com [130.35.100.223]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 4a6vep006x-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 03 Nov 2025 12:32:36 +0000 (GMT) Received: from pps.filterd (iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com (8.18.1.2/8.18.1.2) with ESMTP id 5A3BDrp6008001; Mon, 3 Nov 2025 12:32:35 GMT Received: from sj2pr03cu001.outbound.protection.outlook.com (mail-westusazon11012031.outbound.protection.outlook.com [52.101.43.31]) by iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTPS id 4a58nbkq5b-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 03 Nov 2025 12:32:34 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=chnkFAd+6C10g8fTekQ2Iw57rRqYXZ+ufOUuH0uEqxQFu21zr2hv/NmRy7LbgxGpdrfmMRe88+oID7HjANqUvcfAgKN5VajaZqwnma7XaLaZsIKQEukzjwcBfky6O8CQW3WK3wVYVp9TsrOCxzwSXItV21qwFa/EJQsbgmFZOoit00snKIVSmk5UyZEgIcC5ilva4zfyLhwy0SsyDE91glSLWIti7Lurr76va7LcHpj1uEVQF/HJFxWuY6lC2Fo7kfebEDal4cg7/m0pmv1TimnnuP3Qbcbb03O6RwwtFCfryICp1zHRZ1xyejq9Vs0kMzhOnROF4FT2BR9NsWMwNw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=vzCGIt3sJ5ONOndZRuqYi7OVSaXyL8N0gC2xsfawbow=; b=JI0rvb51roUbwudL4axm2FmQm5xbHXq7ULEQB//wNsvhrqasnmWUP1oC0QEPgXWP4LzDfDtFFeosz+cQzbDeU/hh+ZE/DcWgXUGL7icc+c5etgP7NvTU0VZzU4zMV9yrp5MpLfhvsksIWg9k2D5DuA+GbLayoZ4VmcZMRWdzlK4Oduy5xClP/axTKwzk+iTRzexkoo/UV9VAQ0QTDod6uWGz4DocQxeZ8K7z0znPP+MtwqSVyF/VVDqcH0pjJkC/GjoaN7ng8DLkB0nJ7j31iid9AwRfM3aDNOw7A76s+P2Lkm+d0VaU43XLSYKFq1QtE4JDsv+B0LJ0gTAd/BB0Qg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=vzCGIt3sJ5ONOndZRuqYi7OVSaXyL8N0gC2xsfawbow=; b=jisatVVU3UZmwipCM00m4daQHRSYUkBKSzD6+eS66smYDJ5h7lRqF1wVGqnhWVjFeZQPo5rk85t9aTgQ4EUpPF8PBtjeKapLC4aRrRZvhXqYBhLJ8nyOtT1G3Hp2YDA2w/3bfzMgrXn80KzCq7ea0FqKkI/W7CZ1uBx4bCXe+VY= Received: from DM4PR10MB8218.namprd10.prod.outlook.com (2603:10b6:8:1cc::16) by DS4PR10MB997575.namprd10.prod.outlook.com (2603:10b6:8:31e::5) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9275.16; Mon, 3 Nov 2025 12:32:31 +0000 Received: from DM4PR10MB8218.namprd10.prod.outlook.com ([fe80::2650:55cf:2816:5f2]) by DM4PR10MB8218.namprd10.prod.outlook.com ([fe80::2650:55cf:2816:5f2%2]) with mapi id 15.20.9253.018; Mon, 3 Nov 2025 12:32:31 +0000 From: Lorenzo Stoakes To: Andrew Morton Cc: Christian Borntraeger , Janosch Frank , Claudio Imbrenda , David Hildenbrand , Alexander Gordeev , Gerald Schaefer , Heiko Carstens , Vasily Gorbik , Sven Schnelle , Peter Xu , Alexander Viro , Christian Brauner , Jan Kara , Arnd Bergmann , Zi Yan , Baolin Wang , "Liam R . Howlett" , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Lance Yang , Muchun Song , Oscar Salvador , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Matthew Brost , Joshua Hahn , Rakie Kim , Byungchul Park , Gregory Price , Ying Huang , Alistair Popple , Axel Rasmussen , Yuanchu Xie , Wei Xu , Kemeng Shi , Kairui Song , Nhat Pham , Baoquan He , Chris Li , SeongJae Park , Matthew Wilcox , Jason Gunthorpe , Leon Romanovsky , Xu Xin , Chengming Zhou , Jann Horn , Miaohe Lin , Naoya Horiguchi , Pedro Falcato , Pasha Tatashin , Rik van Riel , Harry Yoo , Hugh Dickins , linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-s390@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, linux-arch@vger.kernel.org, damon@lists.linux.dev Subject: [PATCH 08/16] mm/huge_memory: refactor copy_huge_pmd() non-present logic Date: Mon, 3 Nov 2025 12:31:49 +0000 Message-ID: X-Mailer: git-send-email 2.51.0 In-Reply-To: References: Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: LO2P265CA0248.GBRP265.PROD.OUTLOOK.COM (2603:10a6:600:8a::20) To DM4PR10MB8218.namprd10.prod.outlook.com (2603:10b6:8:1cc::16) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DM4PR10MB8218:EE_|DS4PR10MB997575:EE_ X-MS-Office365-Filtering-Correlation-Id: 97d10fe1-76b1-4aef-fbde-08de1ad50eee X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|376014|7416014|366016|1800799024; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?goTqqAEet28J/wcc/+aWlwlbOYlwLmxvEkA9Cgfp9lqMMKpLcyTp7WXjOvyV?= =?us-ascii?Q?8NItD7dgjNHvKR46lf/xeD9AfmgF+pUyBXgC6kTKJXyWdbr4TXAVgU+9vZc6?= =?us-ascii?Q?Qrh/v9aBpwB4+0xKsy8UqkX8d+aQ0iGhn79aR24TVVCk5NGD7U1eRbThyxxf?= =?us-ascii?Q?Md9dDfV5jHu6139K+xOSKHXqIPFU6bY9hklS3AaS6mli2T0F2uG2SJysbw8F?= =?us-ascii?Q?DkynwDq6r9Ib07XGp5kGSlVjr/OIpnnF/6PdnbOcm+6kdUHmuwKWCViuUu+i?= =?us-ascii?Q?+qgIxGnJVGbUhtLJQoAM6h6cnV0X14zjOQDXvxu0QWd8DXpksjI/r05AvsRY?= =?us-ascii?Q?bFNtGraa+YACu29YBq8gH4l/Bii03W+MvE3u5OWhAUbSq6+U0Ev7WsTcZCXR?= =?us-ascii?Q?z0hs2UE0J+zogGWHxkPzPDCBcYA09O+DyOk9xmCOYsV/QA31aIZKuHsnLrXe?= =?us-ascii?Q?uOB0OprRyU5aZ6JbGh1mWeOE9Zc+0pZ0YmwE5z4LKBiG/gmvzSQ1p/sNwbxX?= =?us-ascii?Q?lm6oMfAJ2M7DIVyyl6Wr9RVr9hhLabnXzP6n/6EqCymuThcyjqDAAgbyk0sZ?= =?us-ascii?Q?Qh/RGpjGaoa4WQSZFhItBPmK8UKs65b9zsUOfBFW7W2zhX2wfe8ZI40OTqDV?= =?us-ascii?Q?t78V3aH+/LHmoQex/BPoPh3t1i47/PR7za8ajR8UfATEfve6CCZgqGwwHSrL?= =?us-ascii?Q?w5tJbSAl4h3DU0pmA+CfC/VmipG0V5FITF+eVmyYXYuZYKS/rSfeoWGMoCCa?= =?us-ascii?Q?+Ich+nohSng4umnIA1lPYTh4Yfyct6Q3fYnErBfQROLs9Zf44Ck50hwQ/syb?= =?us-ascii?Q?sFVofekilZ6/s6lzVWL/M/AAmoJC8wSrvTibtEryhAcuix9YWZM6bjtkNGTv?= =?us-ascii?Q?hV2r+FRV74E81WwNpDEt1zKZZAIvz7e25rT+kW6cSukb0sNXIyFp21hFWmMI?= =?us-ascii?Q?OVkQNNNDWXSL5h8yfotqrLTPv7bbKZeCZUByIdBmCbLomPGy9mFXPRYxGyk3?= =?us-ascii?Q?v4UGRfLQjSYtUHPRrTAi6ZiJ6KlIEF5H1mbw2JdrMhlA4EcMkaEmLusm4Zfs?= =?us-ascii?Q?w7F41y2tI28TXchrAL+/xshZB5+ZslikOGMfGG7g8ZlXfBnb1LlExBPL66hw?= =?us-ascii?Q?A78PI0a/OfspSeHDkRHxZV2AESNdZGq5UHt5V7w2B6zDibOyiUvRHMNM7UxP?= =?us-ascii?Q?K080XajvSc+RMa+gJit4JXL6e+S0hMlqitBn2M3ZWA4Sh2t9bUNYSSD0+EKl?= =?us-ascii?Q?ugljUoOjDnlpR41jJjpCFC7G55qP8t+XHcYR9YH+/HhPxhccY6hNubczrlqw?= =?us-ascii?Q?T8hVFy5BBNc53usPRYHbpCa7wvBBLOa5l/tMAh70SSbfuzOpblURILDKo8Rr?= =?us-ascii?Q?Nw6m19d2omdKENebkzXJdWTpCkemOXiLFnB1BPf3AN7IToYeymOeR5ZT9LaU?= =?us-ascii?Q?R6tb4rA1ND7bYNxPQG9WvbRjWahga5j1?= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DM4PR10MB8218.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(376014)(7416014)(366016)(1800799024);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?1Mhuidb3Wy2VFdfYQqxOxxeCC0sjMr8gmiBFS8ysPg42DvwhDEUVekMvQGzH?= =?us-ascii?Q?PbG9wiZtBSaaPmYPDM2lPhU6/jVBp+QI46tT4gwYZF6pGX8uwIDOdL0xmtxz?= =?us-ascii?Q?yoomh0Ay66RQY6dz/eA7e5DHVIjdVRIoWlYHnWPqq30050MWbTxrM91nK6/Z?= =?us-ascii?Q?4RI3wPS8cq6srgFa7LslGhMl9bRPHhi+sM2uDrg4N+7xLQR6I1eM1ed7V+f1?= =?us-ascii?Q?zvwCkO/UzokcoQA7H+YbB2N4jFtrViiU43SG3TR1u/8+QoRHOyZP8cQ2XEaD?= =?us-ascii?Q?7NOJXkcXqFsuJzhMc0Sl3t9hMCedY+RtlyMe/uE1Uq2fWG4QmjVHVWHOvI73?= =?us-ascii?Q?2r77FgTliVkuaXPw/+DEj93jyp1Od7zYXgKnKxmJIlSTSDUYqrPMCpocMsup?= =?us-ascii?Q?M/L1tXL47IK0Q7D0PG6/1ga4pgoeV92Jd0YaJsQ/yDPUZPfrC3O8y+rg36q2?= =?us-ascii?Q?L+ZHofiCz1RUys3MTqL+YerThLc9EKXSm9KFJcfHBKu/vB4u+ccCFZJqlgr9?= =?us-ascii?Q?uXNuixkJECGL/vnvJrK0fog3C8+TMKCJbGxc/YM9Y/eC/zYLf8OB3QttpFme?= =?us-ascii?Q?TwQZjx9fst0tXqzmRZjWYb66OlBSEW2M1RYKeMSCDIAAgY/kcKqZoUTKtgZL?= =?us-ascii?Q?MJFalO1G28j1qKlPCgNIq9QJXAkYe9N+wy+3HycF4mOJqZbh+FHU1pr9iXjA?= =?us-ascii?Q?Y+gzV6RPxqHoY8TDoMRqSLuK2XiuvzCTqJEipc0pdhe5HbH5DGo/raIInqB7?= =?us-ascii?Q?MiJ+IlQcfqUY5U7rSUmuSf8Ixq2av0mOmjKotV2modN7bNd/gA6/kRxAfhS4?= =?us-ascii?Q?NJCFzDaWXu+PVseaV8dBVH7Ii9lYnwKRkgbsdETCZ48TEXtfIP+X3rmansQU?= =?us-ascii?Q?VWRNWZvl0f9wQjSqcDk5KZPhXvhGuLaWIxH3ojYLOJyJrkaTHgBxh84xwlcZ?= =?us-ascii?Q?SkKKWIyN8sVpyyBjE1pzFfjbHjJqjU5aOBCPIWjVHSA5KlIrsBLcX4VLP1wW?= =?us-ascii?Q?5AiJ/OxNK1LvPgouDJgnqJP9BpHDSWpaWqE00iUFUeN9gguh301A6TbJVOZS?= =?us-ascii?Q?QOQG5ovdg6Ykm3iC6qyhN7mwBRSEWMWZkQXilmw6k2k8TZ/VtPYspXE8Ofja?= =?us-ascii?Q?E3hSPWB/c0E+ZwVftkrhrHbBFZe+oZRbTCRQ2PUNsbo9v2euN9IyTIiJX206?= =?us-ascii?Q?ib44ySrA9HIfmbQjAv+H1Mbs5syZDZd/KhViFq5H9jvDXTAHPg0+MIipb+iG?= =?us-ascii?Q?FYneRzBU3gx46dLBc4PMOS6i/71IlWM+mEyt6JYzqTMUc+MFP+bDec+vm22U?= =?us-ascii?Q?/KYDT9UeJpNVDP7cCu6eoyCL3NU8zOa5h0C8GVUwTqhusAO1rnSR2PiNhlyq?= =?us-ascii?Q?grMfTxg54+QmKE0Ap20N+DWVg0IZPtF+4rAlDbt6icta0vVM5WX2BcjBuzK+?= =?us-ascii?Q?aMux0MF0ckjtZOT/5bgZv9lEjWLJSDJR7zL31OOyvQQmeRGZTp5Nc8j6qKNZ?= =?us-ascii?Q?8CiABNHY8lqPKtaUwyj19SusJcmhPhagxFzAibWDWWdPILeUgW/Q2u3hfjtV?= =?us-ascii?Q?LAO7YdJlPB1Q55Xs9zk6Tsbi7OS0XIGoVhDrYuF1uViSRpJDYXFMm4Gx4LBK?= =?us-ascii?Q?tQ=3D=3D?= X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: IckzrXV//19uGhfjeY695J7LPJJdAa26P3hjOzR0GEHXqiBcaTl9/vz1hX3m/LhKLR+9BB71Hts9Y1gfw0u+/dnKzzCT/qeMM1N3ETgWu6DWqNZ2KobcjEzvHEoi8HoAK6/+dh/t/lBDGkqt3JBurLeroZR92JN++WDJ+HjwX0FB3PD5X/5iSWHLQVSIy3zDXodZRAfyDW0P1c+cUsczjKtOYxysZjCGs46N28amM0SavD4o8JQHNYI+HrOmmKPiREkOa/KH+cRy4qsxoIbPnBzZv4+DozYb2sYrwukWNXnQ6sKDYM8inP11MBKdYQA3x5kqNKnuMbFnHOtpJoTr2RRoWEFq5ePEd3SpbmlyO4ydg5EepuWT/88iH6I2CqmVD3+Ps1V7E8czbmee4X+IpwU7P29hUv71Nv6Qav10wmAsVvAtUrey4+Dcss0Y4ifTAw5GhF4baW57PlGEX1gvYgdaol5vH7CUu/T8ICGU3WualEQIZldEIronqQeJSQGbTpOmr+z292YF+ozcVtcLravKEL+XzSM7oOcc9ggcPsqusRa5ilTU9EoGZSB41hpe/1glCwst2OIOqYV2U2aHn3oAEGEcjvndebKzqbXp8+c= X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: 97d10fe1-76b1-4aef-fbde-08de1ad50eee X-MS-Exchange-CrossTenant-AuthSource: DM4PR10MB8218.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 03 Nov 2025 12:32:31.4762 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: z4Eaiiw9o2tGM5ODSUaUULinKQ83cMxLSjl5oWgR3/IEUUMjy/3tSKDZrDptLZCt5iVYjR5/SVR/9yjCXR58EV41pG99nEyN2ef7+SiUcDI= X-MS-Exchange-Transport-CrossTenantHeadersStamped: DS4PR10MB997575 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1121,Hydra:6.1.9,FMLib:17.12.100.49 definitions=2025-11-03_01,2025-11-03_01,2025-10-01_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 phishscore=0 malwarescore=0 mlxlogscore=999 mlxscore=0 adultscore=0 spamscore=0 bulkscore=0 suspectscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2510240000 definitions=main-2511030113 X-Proofpoint-GUID: j_BMPxljbS4l8BBsb4L5dv0UdDfqOv1s X-Proofpoint-ORIG-GUID: j_BMPxljbS4l8BBsb4L5dv0UdDfqOv1s X-Authority-Analysis: v=2.4 cv=B9m0EetM c=1 sm=1 tr=0 ts=6908a0e4 b=1 cx=c_pps a=zPCbziy225d3KhSqZt3L1A==:117 a=zPCbziy225d3KhSqZt3L1A==:17 a=6eWqkTHjU83fiwn7nKZWdM+Sl24=:19 a=z/mQ4Ysz8XfWz/Q5cLBRGdckG28=:19 a=lCpzRmAYbLLaTzLvsPZ7Mbvzbb8=:19 a=xqWC_Br6kY4A:10 a=6UeiqGixMTsA:10 a=GoEa3M9JfhUA:10 a=VkNPw1HP01LnGYTKEx00:22 a=yPCof4ZbAAAA:8 a=9UmCEYkwnbpDqGD1g-YA:9 cc=ntf awl=host:12123 X-Proofpoint-Spam-Details-Enc: AW1haW4tMjUxMTAzMDExNCBTYWx0ZWRfX5sIhcI5yuHwt pyGRrFfRhatjT6dxPYKI+TCgTzklj3LazTsbyKPEkYy4XD01Rvs3uBRKCRoqgJW+qBOv5UoEnYR ysAV4sFqlhvPQ5b+4C0btp58DP1FSNoll6D5BcQJYKAccOQ65ydgcARCMm5A9nne1lNo/2NQ3Zg qi145m2l6SHpNHnHdRwYe6mW1GjwG7XYgm4I+P5RZNJxqNtv7bXrc6nKmEiZ0UN7OacojG2lKZE h3h4aU5W2bWa8OkqamQPq54Q25Gxxc8ozLnhB6tNATkf2ctT5Q04H51u9B1paa30WQkJk+pkBPH smV11p+uBMTfynDngXFfXbnzrmEh8Q39ijylv52xf4jBZYC1wXsA49BXtkjH1LguPcbPTrB7qAJ g2I7wolVarFOGePXJ7q/MpE8MGO5eI2igv01RpkCFalW/YiEx0I= Content-Type: text/plain; charset="utf-8" Right now we are inconsistent in our use of thp_migration_supported(): static inline bool thp_migration_supported(void) { return IS_ENABLED(CONFIG_ARCH_ENABLE_THP_MIGRATION); } And simply having arbitrary and ugly #ifdef CONFIG_ARCH_ENABLE_THP_MIGRATION blocks in code. This is exhibited in copy_huge_pmd(), which inserts a large #ifdef CONFIG_ARCH_ENABLE_THP_MIGRATION block and an if-branch which is difficult to follow It's difficult to follow the logic of such a large function and the non-present PMD logic is clearly separate as it sits in a giant if-branch. Therefore this patch both separates out the logic and utilises thp_migration_supported(). No functional change intended. Signed-off-by: Lorenzo Stoakes --- mm/huge_memory.c | 109 +++++++++++++++++++++++++---------------------- 1 file changed, 59 insertions(+), 50 deletions(-) diff --git a/mm/huge_memory.c b/mm/huge_memory.c index 2e5196a68f14..31116d69e289 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -1774,6 +1774,62 @@ void touch_pmd(struct vm_area_struct *vma, unsigned = long addr, update_mmu_cache_pmd(vma, addr, pmd); } =20 +static void copy_huge_non_present_pmd( + struct mm_struct *dst_mm, struct mm_struct *src_mm, + pmd_t *dst_pmd, pmd_t *src_pmd, unsigned long addr, + struct vm_area_struct *dst_vma, struct vm_area_struct *src_vma, + pmd_t pmd, pgtable_t pgtable) +{ + swp_entry_t entry =3D pmd_to_swp_entry(pmd); + struct folio *src_folio; + + VM_WARN_ON(!is_pmd_non_present_folio_entry(pmd)); + + if (is_writable_migration_entry(entry) || + is_readable_exclusive_migration_entry(entry)) { + entry =3D make_readable_migration_entry(swp_offset(entry)); + pmd =3D swp_entry_to_pmd(entry); + if (pmd_swp_soft_dirty(*src_pmd)) + pmd =3D pmd_swp_mksoft_dirty(pmd); + if (pmd_swp_uffd_wp(*src_pmd)) + pmd =3D pmd_swp_mkuffd_wp(pmd); + set_pmd_at(src_mm, addr, src_pmd, pmd); + } else if (is_device_private_entry(entry)) { + /* + * For device private entries, since there are no + * read exclusive entries, writable =3D !readable + */ + if (is_writable_device_private_entry(entry)) { + entry =3D make_readable_device_private_entry(swp_offset(entry)); + pmd =3D swp_entry_to_pmd(entry); + + if (pmd_swp_soft_dirty(*src_pmd)) + pmd =3D pmd_swp_mksoft_dirty(pmd); + if (pmd_swp_uffd_wp(*src_pmd)) + pmd =3D pmd_swp_mkuffd_wp(pmd); + set_pmd_at(src_mm, addr, src_pmd, pmd); + } + + src_folio =3D pfn_swap_entry_folio(entry); + VM_WARN_ON(!folio_test_large(src_folio)); + + folio_get(src_folio); + /* + * folio_try_dup_anon_rmap_pmd does not fail for + * device private entries. + */ + folio_try_dup_anon_rmap_pmd(src_folio, &src_folio->page, + dst_vma, src_vma); + } + + add_mm_counter(dst_mm, MM_ANONPAGES, HPAGE_PMD_NR); + mm_inc_nr_ptes(dst_mm); + pgtable_trans_huge_deposit(dst_mm, dst_pmd, pgtable); + if (!userfaultfd_wp(dst_vma)) + pmd =3D pmd_swp_clear_uffd_wp(pmd); + set_pmd_at(dst_mm, addr, dst_pmd, pmd); +} + int copy_huge_pmd(struct mm_struct *dst_mm, struct mm_struct *src_mm, pmd_t *dst_pmd, pmd_t *src_pmd, unsigned long addr, struct vm_area_struct *dst_vma, struct vm_area_struct *src_vma) @@ -1819,59 +1875,12 @@ int copy_huge_pmd(struct mm_struct *dst_mm, struct = mm_struct *src_mm, ret =3D -EAGAIN; pmd =3D *src_pmd; =20 -#ifdef CONFIG_ARCH_ENABLE_THP_MIGRATION - if (unlikely(is_swap_pmd(pmd))) { - swp_entry_t entry =3D pmd_to_swp_entry(pmd); - - VM_WARN_ON(!is_pmd_non_present_folio_entry(pmd)); - - if (is_writable_migration_entry(entry) || - is_readable_exclusive_migration_entry(entry)) { - entry =3D make_readable_migration_entry(swp_offset(entry)); - pmd =3D swp_entry_to_pmd(entry); - if (pmd_swp_soft_dirty(*src_pmd)) - pmd =3D pmd_swp_mksoft_dirty(pmd); - if (pmd_swp_uffd_wp(*src_pmd)) - pmd =3D pmd_swp_mkuffd_wp(pmd); - set_pmd_at(src_mm, addr, src_pmd, pmd); - } else if (is_device_private_entry(entry)) { - /* - * For device private entries, since there are no - * read exclusive entries, writable =3D !readable - */ - if (is_writable_device_private_entry(entry)) { - entry =3D make_readable_device_private_entry(swp_offset(entry)); - pmd =3D swp_entry_to_pmd(entry); - - if (pmd_swp_soft_dirty(*src_pmd)) - pmd =3D pmd_swp_mksoft_dirty(pmd); - if (pmd_swp_uffd_wp(*src_pmd)) - pmd =3D pmd_swp_mkuffd_wp(pmd); - set_pmd_at(src_mm, addr, src_pmd, pmd); - } - - src_folio =3D pfn_swap_entry_folio(entry); - VM_WARN_ON(!folio_test_large(src_folio)); - - folio_get(src_folio); - /* - * folio_try_dup_anon_rmap_pmd does not fail for - * device private entries. - */ - folio_try_dup_anon_rmap_pmd(src_folio, &src_folio->page, - dst_vma, src_vma); - } - - add_mm_counter(dst_mm, MM_ANONPAGES, HPAGE_PMD_NR); - mm_inc_nr_ptes(dst_mm); - pgtable_trans_huge_deposit(dst_mm, dst_pmd, pgtable); - if (!userfaultfd_wp(dst_vma)) - pmd =3D pmd_swp_clear_uffd_wp(pmd); - set_pmd_at(dst_mm, addr, dst_pmd, pmd); + if (unlikely(thp_migration_supported() && is_swap_pmd(pmd))) { + copy_huge_non_present_pmd(dst_mm, src_mm, dst_pmd, src_pmd, addr, + dst_vma, src_vma, pmd, pgtable); ret =3D 0; goto out_unlock; } -#endif =20 if (unlikely(!pmd_trans_huge(pmd))) { pte_free(dst_mm, pgtable); --=20 2.51.0 From nobody Sat Feb 7 17:55:59 2026 Received: from mx0a-00069f02.pphosted.com (mx0a-00069f02.pphosted.com [205.220.165.32]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id DEE1D30AD1A; Mon, 3 Nov 2025 12:33:46 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=205.220.165.32 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1762173228; cv=fail; b=sj22wlYEbxiNFbs/pLiur07yOheMjtkuQUKk6r3n9m5T0fR4kVirkkjX5k+zAhTUVuXJsgzJaSa9LDZXJClO11pDZPr3ArCZTv2IemhH3RAfzuvOR42T9tUpSac8/axU2+qiY1ByJzLMVACgfh8gOCmzLQkE4XrJSt3iIEPiWtc= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1762173228; c=relaxed/simple; bh=wEswhEPB/UrhDyPK9C9sI1IHFlT5+kgMCKmd18ehtZM=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: Content-Type:MIME-Version; b=VUwbg0BemrzNGiTYWHbXpD2oHEQOAal9eRxygYWJwOeJ2uM8kbmU4zDTEwBxtzl1jKnBuyBviuV+o9GqUO1uq1x6LINdiK98L2QSIMYHOZZR14Kv4a+/RYjtxtUeWSA5w6PsjHUUHlzrYSpkfiobyOoWBiH3vt8Vdp2OlACrsaI= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=oracle.com; spf=pass smtp.mailfrom=oracle.com; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b=Jul6ITvl; dkim=pass (1024-bit key) header.d=oracle.onmicrosoft.com header.i=@oracle.onmicrosoft.com header.b=tyv0uts+; arc=fail smtp.client-ip=205.220.165.32 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=oracle.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=oracle.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b="Jul6ITvl"; dkim=pass (1024-bit key) header.d=oracle.onmicrosoft.com header.i=@oracle.onmicrosoft.com header.b="tyv0uts+" Received: from pps.filterd (m0333521.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 5A3CTdoD019181; Mon, 3 Nov 2025 12:32:39 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=cc :content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s= corp-2025-04-25; bh=ES+ODnUpx6m2BxJg2i8m0/ywBdmg7etQq70a9R7KC9k=; b= Jul6ITvl1FqKpcPYqWVpvgiEcaVw5IlPT5OXCJtbLNttt/GfrFq8YjzV63eND8IS Lfx3AyzTTxQUmuOiBcpTGND5t9ilxYvj3UL86voG6EIiNhaAE9O8HhF0Eu57ruTG QLC1qOLN6rmjmPujJq2nUp+F4Xnrt0N6WRKLtWO3R7PAPc8QxZo6sSe3fQKqLIMK jx/+NYHNS5dGpH5JNBIbK1ijw6snClVqEJWuzDQcZTn7haeyHK3jpau6vfjWyVdZ puDDpXlKPbaDOjFSqzzcjtO9XM2+4qSevvIoCLLmgp1gVwklzMjzWM3oIq0Agz5w 4aHygWM91cLQ9bNjCa1jRQ== Received: from iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta03.appoci.oracle.com [130.35.103.27]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 4a6ven808r-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 03 Nov 2025 12:32:39 +0000 (GMT) Received: from pps.filterd (iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com (8.18.1.2/8.18.1.2) with ESMTP id 5A3BQYUx039758; Mon, 3 Nov 2025 12:32:37 GMT Received: from sj2pr03cu001.outbound.protection.outlook.com (mail-westusazon11012044.outbound.protection.outlook.com [52.101.43.44]) by iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTPS id 4a58nhux7r-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 03 Nov 2025 12:32:37 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=Q9054xQ4NV+44y4YYch76//RUOCadTeG3h9GURGb5zYREiFETUpJ1UT4uHPu0UKUiX1r1DQ71CP31LPGlnu0sQ0UkeYMFQ5rDvt3Dm3iybTqFkwXYURuJdIFjo5R+rb0oFcFpg1kbddEaumpS0KwUr4NHHxmnRg1H6yTQ0K3i4PkUaX6ZySMi5THYmsZfsPXU7CnAJguZEFLC7ZPghYCTLR6LCijQC/lh33njH68IbkccDwwgO2kwhWHBfdDK6jXedZ1iYnjpHCFVwXYoCfP1Xw7ISduxQZWlEdKcwWn9dfSgCSNB4T8bpL6mPHfq2Gdi7h2nQrERAXKFU14O+XKnw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=ES+ODnUpx6m2BxJg2i8m0/ywBdmg7etQq70a9R7KC9k=; b=OBDzDyVd/iuL0cAXd9bMNbcDYZXxLwbIYimi4n37HMAOOi5HTjPkW6T/doY3CRDGMhDrSKq1zyV487Jq+vOUBvooU4m7a8MjbsQrrCqIuAnNt9hsWA+LB/nL001neECbMyri3y8FojMKoq2CPhfuRI8sZ3R9SbWu6WrH8/3fP0VzBGRRKGUmuAqzdPUmUtL5G7x4IRln1BmmSH51vYQs1gw2z3Kpj0LW1sk1KE2fT2G9QsQMSrvkzFPcl9gP41I1rwlBh3xY1MzSHhXVcMT9Cbfwip47ykzN/n+WRBNiCcbOyMCcP63PbM7Ijfd1BrB5xAicWd54YMz1gTsy6NJwww== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=ES+ODnUpx6m2BxJg2i8m0/ywBdmg7etQq70a9R7KC9k=; b=tyv0uts+ZHiVhtsFuKJvAIAxiDlhoqpubu6r2jKQ0/Rn2MR1G5nKbMcBnKV3sA1YgmPZn3OBPnZbOyI3xa3ACgbqKhLux3Mz8Kb5mN31JZaUfBtfj8oBJRnHuT/asTLcPDNj+D8JZKJ+ZOcV2Sx7RRMNBXNPdjuhM0s12LqbmjI= Received: from DM4PR10MB8218.namprd10.prod.outlook.com (2603:10b6:8:1cc::16) by DS4PR10MB997575.namprd10.prod.outlook.com (2603:10b6:8:31e::5) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9275.16; Mon, 3 Nov 2025 12:32:34 +0000 Received: from DM4PR10MB8218.namprd10.prod.outlook.com ([fe80::2650:55cf:2816:5f2]) by DM4PR10MB8218.namprd10.prod.outlook.com ([fe80::2650:55cf:2816:5f2%2]) with mapi id 15.20.9253.018; Mon, 3 Nov 2025 12:32:34 +0000 From: Lorenzo Stoakes To: Andrew Morton Cc: Christian Borntraeger , Janosch Frank , Claudio Imbrenda , David Hildenbrand , Alexander Gordeev , Gerald Schaefer , Heiko Carstens , Vasily Gorbik , Sven Schnelle , Peter Xu , Alexander Viro , Christian Brauner , Jan Kara , Arnd Bergmann , Zi Yan , Baolin Wang , "Liam R . Howlett" , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Lance Yang , Muchun Song , Oscar Salvador , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Matthew Brost , Joshua Hahn , Rakie Kim , Byungchul Park , Gregory Price , Ying Huang , Alistair Popple , Axel Rasmussen , Yuanchu Xie , Wei Xu , Kemeng Shi , Kairui Song , Nhat Pham , Baoquan He , Chris Li , SeongJae Park , Matthew Wilcox , Jason Gunthorpe , Leon Romanovsky , Xu Xin , Chengming Zhou , Jann Horn , Miaohe Lin , Naoya Horiguchi , Pedro Falcato , Pasha Tatashin , Rik van Riel , Harry Yoo , Hugh Dickins , linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-s390@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, linux-arch@vger.kernel.org, damon@lists.linux.dev Subject: [PATCH 09/16] mm/huge_memory: refactor change_huge_pmd() non-present logic Date: Mon, 3 Nov 2025 12:31:50 +0000 Message-ID: <9e8089923649205975e9ae39c9f628b7ce8b06d9.1762171281.git.lorenzo.stoakes@oracle.com> X-Mailer: git-send-email 2.51.0 In-Reply-To: References: Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: LO4P302CA0037.GBRP302.PROD.OUTLOOK.COM (2603:10a6:600:317::9) To DM4PR10MB8218.namprd10.prod.outlook.com (2603:10b6:8:1cc::16) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DM4PR10MB8218:EE_|DS4PR10MB997575:EE_ X-MS-Office365-Filtering-Correlation-Id: c3d6d0c5-d4ac-4ad2-e24f-08de1ad510a6 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|376014|7416014|366016|1800799024; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?ugqtwimc0/B1xWcF0tkz/KMrrAiYTydOF+XSzPXi8fnBM9CVK2pRhBx7BKba?= =?us-ascii?Q?m/USBhJEG8k4o/uRgTSdReIS0vdUwAM7jTNLf1KL10oUWoNrNTFvb40EDN9F?= =?us-ascii?Q?dCxJ/g238+3J7Rp6CePV53PXtliddKoLeeGUazjEToMj9T2j2JQZo4R0Q7dV?= =?us-ascii?Q?4zki3hU7BnV2VCewqf11jNQ4LzJhp3AHbfB5MTs9akzgwKMUH5e9vFrPdXzf?= =?us-ascii?Q?aOc09IxZ8Ii5w4OyEiCprf8l7QJoFWhpmofo2UjWarRb5yjs/xfcZXh2ROxy?= =?us-ascii?Q?PX0aTtO0PqyAZvAnvxTgRNG2YJIIndGjfKiuuUX2z9tuvsg+QtFMlwuBu+bb?= =?us-ascii?Q?CswE8Dfvvtv9tNvg6YzkHY012ha6SCWx3sNQnJkIbSS62ZByrB05tZkf8A3t?= =?us-ascii?Q?9unEl/3iGWkN2jwPkDqSpbvtc7vlTwn0r+OC5968Kjd5AaCZGDnkdthk9l9a?= =?us-ascii?Q?HJL+aQYfZTYJE2oym8PJv+2y/GeGR8q0shn8K6cy0CnPrsIvax3GRjAddekR?= =?us-ascii?Q?eQcwFUqb8gzG/rDq+Itbg7QeRQI0gSGWPot9+PziAsFxWY5tamc1PG1HSnYI?= =?us-ascii?Q?dqQ36RvuwE7y+MMu9ZD26PmyyHTAmp18pVte+7kS3X+TEbpbkVC4kI3DKRU0?= =?us-ascii?Q?zcn4wKG4YfyrIIxskIwPToiU0fzpomQGIlsUFBwhHUls3GqCFWiT+q1Bl3BM?= =?us-ascii?Q?ClWXk5Fw3JZwR4V+UQO9n5/TccTnf6+gxbVjseAr9Zu1cktHUOGwYZTbuy0x?= =?us-ascii?Q?KwZC3+R+PPHFUaaQyJ02HSkg7nQEEKJU8GaB/qO0Ys+jPIUAyEotF8oKnppQ?= =?us-ascii?Q?1pgfn1+ZOlVX6CWAQKjCC16JF5qtQrZyLmMDU4MXcSdAky7v2nxNUrkw3yuE?= =?us-ascii?Q?k3VgIGl3cyGC40f8InbJBBz/e1YXXXbufnoiahsukVgKYrMa757vdFe5DGQr?= =?us-ascii?Q?ME9fMt2eT9qEmDTKcUuc0r69+OfvSN33qc6L6gPWEFEYl1JkYneJTqUEq6f4?= =?us-ascii?Q?LfGGTXyilqnz2c9vTjeyhzHxwrQhBcjklsczOgl3alhSL/8UriEg4cKufdxV?= =?us-ascii?Q?+BjBPNZo2JAIMYAPhqGUplDGfyfedO2GmaUWh9WZEY6ROnmlwMd/Lub9BxcS?= =?us-ascii?Q?Bluhq1Iatk3WLTdSs4fxDNeyXiM9uhTebZyfDcif7Ge5DOdinRUw/PcNL3Ew?= =?us-ascii?Q?qsuA9FXETzgczFUVYvdqPuQ+iuMxXIckOD0rh9cZA0kAy0+O4K/O74g49DOk?= =?us-ascii?Q?rNZT7tFEmhjh/Ne3QMaOTl7lspcw6W5aXsmvVJBG3+y5JLT72/n7FcV7M6ZR?= =?us-ascii?Q?q6GV7dliVn1Vt98daeik8PlwhPqqwUYpRUoeVoFp52OVhfb6LSPzXlWhAqFl?= =?us-ascii?Q?gvP3YclK788W0TVvjf3N2nnsXt9EotDc8WyMKCHQmy2oU2R+EecC5ZgIEAsE?= =?us-ascii?Q?Sz/aJoTX63rPE/Fk1Yl5wsuIy1HrvpBl?= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DM4PR10MB8218.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(376014)(7416014)(366016)(1800799024);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?juETJf4OrGyvNf8rWyCoU5r6GBrdOEWQMMBfqikFE7nfwa/u6Olli+99QhyA?= =?us-ascii?Q?urZZRicSMVAAOsi2tHKX3/OvLdWCRNdNaWmd9DWWhq6RrYdLvMLrZIze9epw?= =?us-ascii?Q?YHnG2Gs4B9nCxQNL0CxPjN5vsMkI/nB+l2LQY/+pYrNj7zKyfACm3+KRR3CR?= =?us-ascii?Q?aASEcGErwH0aTvRhakzvhCjr9SZFFmYsN67Ax+1CvK2HMuOUsc3ShmjfLnR7?= =?us-ascii?Q?tK+bSesr20uumfZxUTwMAsPi8NvehlrwHtgjwTrYlelN36STNx7JmfxhZVr2?= =?us-ascii?Q?pf7LG68lQXohDOLftq9t3bEqMZiMKgLl9JJyURespO0v/h4emSD0og5VPXkQ?= =?us-ascii?Q?AETiXbRXZOMJMW4qS0/3bWxEWLwMcKu/eXEwB/4DKjpvASrat52CKCTiX2lb?= =?us-ascii?Q?iKxfPTMAvGnAnamkoBgR9rB/tLSmHtZRGXn4UYR9fhsE08UwQDPKQf95VOra?= =?us-ascii?Q?PCbyLTlyW1+VB62oDLfOF3Z9NcL+apzP+mgUF6hLjnSc1WGqYt+x5apJnp5a?= =?us-ascii?Q?muPqziWFsLM5ifC+OoyhU8/56PcZxkM4iyhSP94WDKp5c/zOKQULydr+C8RO?= =?us-ascii?Q?4Ikr+FFHHqb54rXeHMGUzmpdhegidh7V1xKFd0+awSzURmDOR8kgi5jeo7zB?= =?us-ascii?Q?aMGcyWwTIKi4enyjJ0o6y5jqXFy00gN1MfK31VLRcTPxwSdhESbelr7CewMx?= =?us-ascii?Q?NJGKv2gY1u5A1fkvGZGzlcaAj0kU57rd9u8hkdvI+f7RhQOVZOgX3/YOGWEz?= =?us-ascii?Q?y6zehtjUxp4cx/0MZCZnQS8wv0b7faAyqGHolvQ52L8h3T2RFsEOprCGxhxV?= =?us-ascii?Q?vk9Vh6s4mjhyjzOOrT5zfNuzaK0NlwsG8RvjGZ9qHj7Fi3pvCJB1NCjJr+cd?= =?us-ascii?Q?BNWTFoT2mZxfwpGZRQEnkgQ/J2UywfbjYrlM2W9oyUAuoFlzvt2aw9krqCJs?= =?us-ascii?Q?SFQesw3RcsvCPqYF3yuhJhfpaGl++vCbv1te7QJqdoL4lUORu8FiI9dVBhZQ?= =?us-ascii?Q?ufa/eLIQSEzoXYjugr7vjYllv8h8s+6F77Xy7MjiWSiqzwKk0NvNgnd1IPdG?= =?us-ascii?Q?znKx1jAyeCE8X2wJwtryLjIUSxuKege7iQ3Si9WeUJLSdchAdDR+PZyY6aki?= =?us-ascii?Q?3nX5duzZKE7upeRa9wSAaA1cj0+39SRxrvOyMuVsgy0IxLzUbeh6SAi7xgwH?= =?us-ascii?Q?CecrlIJgdrA/LqZHX2gNA5nQNlnuTDb69MuUBjgibQSVugC6/olfRdTy8YOn?= =?us-ascii?Q?9ewF5VJfG9Nb4AYEJqb+2exobKEGq6SMrkjAOX+cTctNc3QfAtd4WQx7r1i/?= =?us-ascii?Q?mPcXrbkgUapDRWJRip7s920ee8qkAoRr+49iFJ3/vbo4PKlGYvnDKmZ8JRFk?= =?us-ascii?Q?cAiuzMc2WDafDH47oaDM9Q1wbFULm9teD06pp7MQ3lUYXd4DE8JpVt6kvM37?= =?us-ascii?Q?R0MsiMB9i7kCz4eRUG2mNvIH12K8NzQ2GdwA+122ThY7RkOE/wgDxy0b0q96?= =?us-ascii?Q?2iUUjDGJVmjHfyv3cm1t4b/D2Gs1ZoYaLmMjXTJuRdTiRHZu0GxKQUylx3EO?= =?us-ascii?Q?jQ6JV1FYTHUmywMVcUZL2MnX3rwJkhSdgVoM5fbIgyuhLfXTXrvVlMvGP5lz?= =?us-ascii?Q?Ig=3D=3D?= X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: mia1wwCrr6OdHK5gWlBYugzfDxctpIUElx0A4JaEDx3cKgLZ+3XTggrY49LBGcrsPejFZzViDGFHd+NJSw9EDU/FjOshXQ9ufqOkWjSFoethxShLTiJQFZyTSj1xIXkg+TlVNxj60SpeNJNOsrdXr/LzELAff78iK/Och+pueHiud4vu4b3ixReDvdOrsx/kSBz77KrMtxvikaziVlLplKQ94Q1C+GSvmLzitwLxjU/Dv0DMTFZkDHAUUqy6ejElIZAjGkxzdle/MPewTOua0AN6AXDwwKb8vRhHsG+1dT3m4tnCCcVHBYl27wMBa/jWQ5XSVt+xIFZUhItu1+qDjB+mhBhjbhlylVglJSTcpTMYqfDYUuhHy/63oYV5+ZTjaml2/MObzyMLhztbwVTQHGFMW9ZOBrWUuaj1zSU5iWL2SLYxoXOAThhCrA6nH/Eb1sFvpvsRx8MZg9YJ+EclvUscN74ImsI2s3cfYfrswtreX//1reMkrGVC80tmKRJpUhyFw1l4zswuzeRGhjmnDswi/kyQkgW2OHUsmCO6FBDnG8Xlj5u6EBvLlyF2NfLqU8QNm3vVRbnoNr2UbRiXVchAHzMTYgoi2QM1ztXUGyQ= X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: c3d6d0c5-d4ac-4ad2-e24f-08de1ad510a6 X-MS-Exchange-CrossTenant-AuthSource: DM4PR10MB8218.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 03 Nov 2025 12:32:34.4607 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: G9Qy1rl7kuaVjw/ycy3GBVmU525cSdV4x6gTyQtXTqk7v+/eQkNdysSQPNMT/7hYr4VjMfQ7Y29pel3GVrj1+efDMH30tsOR7avTW9HjxNw= X-MS-Exchange-Transport-CrossTenantHeadersStamped: DS4PR10MB997575 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1121,Hydra:6.1.9,FMLib:17.12.100.49 definitions=2025-11-03_01,2025-11-03_01,2025-10-01_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxlogscore=999 bulkscore=0 spamscore=0 suspectscore=0 mlxscore=0 adultscore=0 phishscore=0 malwarescore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2510240000 definitions=main-2511030113 X-Proofpoint-GUID: H-PPbuuIxN6ZPQNu7dlbJ1txbwa1nzqP X-Proofpoint-ORIG-GUID: H-PPbuuIxN6ZPQNu7dlbJ1txbwa1nzqP X-Proofpoint-Spam-Details-Enc: AW1haW4tMjUxMTAzMDExNCBTYWx0ZWRfXzkL5AWZnTkP3 ZXqrUeVDaCB7j6wYHQEst7Oghd9k5UZz7Jzxac4nG2zwBEMe8UOyXNj/5afINjDs9VFBoNw0FRR ak8VQvAfQPTZYvKaeBzRIHLHeXg34IQ/AAGc3x9/5B8DhhmdFHEgbUXRJ+DUc5/gDaGcLPtYFuN bb3xriCDGCytSzZ6kbC2hHrxvJJn43SfeCDCmyQqGZfTF9v9r+BMhIE4+sMTHUkk/jM+ekcLD1n w2WxG4prREn/TaxUvFc9HGL+b8wIJNIbmycy5DmZxxMTXnTU7ulk8HWvHfrJgu7R+5YqtUs+1yQ Gwp5HK9j7a1Cth+iHsbvb7KPciAKA1Ieg3cXextHutmH1VnDfDLJ6IgqnAIWEFZ3o/Q3U68tLzg /ut5EslzrolhH9DmsjikNb1yQPzpp4wAIBKyruTG0JTkGh9+uLM= X-Authority-Analysis: v=2.4 cv=MvBfKmae c=1 sm=1 tr=0 ts=6908a0e7 b=1 cx=c_pps a=qoll8+KPOyaMroiJ2sR5sw==:117 a=qoll8+KPOyaMroiJ2sR5sw==:17 a=6eWqkTHjU83fiwn7nKZWdM+Sl24=:19 a=z/mQ4Ysz8XfWz/Q5cLBRGdckG28=:19 a=lCpzRmAYbLLaTzLvsPZ7Mbvzbb8=:19 a=xqWC_Br6kY4A:10 a=6UeiqGixMTsA:10 a=GoEa3M9JfhUA:10 a=VkNPw1HP01LnGYTKEx00:22 a=yPCof4ZbAAAA:8 a=gAL6uhOEssAPNySJDCYA:9 cc=ntf awl=host:12124 Content-Type: text/plain; charset="utf-8" Similar to copy_huge_pmd(), there is a large mass of open-coded logic for the CONFIG_ARCH_ENABLE_THP_MIGRATION non-present entry case that does not use thp_migration_supported() consistently. Resolve this by separating out this logic and introduce change_non_present_huge_pmd(). No functional change intended. Signed-off-by: Lorenzo Stoakes --- mm/huge_memory.c | 72 ++++++++++++++++++++++++++---------------------- 1 file changed, 39 insertions(+), 33 deletions(-) diff --git a/mm/huge_memory.c b/mm/huge_memory.c index 31116d69e289..40a8a2c1e080 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -2499,6 +2499,42 @@ bool move_huge_pmd(struct vm_area_struct *vma, unsig= ned long old_addr, return false; } =20 +static void change_non_present_huge_pmd(struct mm_struct *mm, + unsigned long addr, pmd_t *pmd, bool uffd_wp, + bool uffd_wp_resolve) +{ + swp_entry_t entry =3D pmd_to_swp_entry(*pmd); + struct folio *folio =3D pfn_swap_entry_folio(entry); + pmd_t newpmd; + + VM_WARN_ON(!is_pmd_non_present_folio_entry(*pmd)); + if (is_writable_migration_entry(entry)) { + /* + * A protection check is difficult so + * just be safe and disable write + */ + if (folio_test_anon(folio)) + entry =3D make_readable_exclusive_migration_entry(swp_offset(entry)); + else + entry =3D make_readable_migration_entry(swp_offset(entry)); + newpmd =3D swp_entry_to_pmd(entry); + if (pmd_swp_soft_dirty(*pmd)) + newpmd =3D pmd_swp_mksoft_dirty(newpmd); + } else if (is_writable_device_private_entry(entry)) { + entry =3D make_readable_device_private_entry(swp_offset(entry)); + newpmd =3D swp_entry_to_pmd(entry); + } else { + newpmd =3D *pmd; + } + + if (uffd_wp) + newpmd =3D pmd_swp_mkuffd_wp(newpmd); + else if (uffd_wp_resolve) + newpmd =3D pmd_swp_clear_uffd_wp(newpmd); + if (!pmd_same(*pmd, newpmd)) + set_pmd_at(mm, addr, pmd, newpmd); +} + /* * Returns * - 0 if PMD could not be locked @@ -2527,41 +2563,11 @@ int change_huge_pmd(struct mmu_gather *tlb, struct = vm_area_struct *vma, if (!ptl) return 0; =20 -#ifdef CONFIG_ARCH_ENABLE_THP_MIGRATION - if (is_swap_pmd(*pmd)) { - swp_entry_t entry =3D pmd_to_swp_entry(*pmd); - struct folio *folio =3D pfn_swap_entry_folio(entry); - pmd_t newpmd; - - VM_WARN_ON(!is_pmd_non_present_folio_entry(*pmd)); - if (is_writable_migration_entry(entry)) { - /* - * A protection check is difficult so - * just be safe and disable write - */ - if (folio_test_anon(folio)) - entry =3D make_readable_exclusive_migration_entry(swp_offset(entry)); - else - entry =3D make_readable_migration_entry(swp_offset(entry)); - newpmd =3D swp_entry_to_pmd(entry); - if (pmd_swp_soft_dirty(*pmd)) - newpmd =3D pmd_swp_mksoft_dirty(newpmd); - } else if (is_writable_device_private_entry(entry)) { - entry =3D make_readable_device_private_entry(swp_offset(entry)); - newpmd =3D swp_entry_to_pmd(entry); - } else { - newpmd =3D *pmd; - } - - if (uffd_wp) - newpmd =3D pmd_swp_mkuffd_wp(newpmd); - else if (uffd_wp_resolve) - newpmd =3D pmd_swp_clear_uffd_wp(newpmd); - if (!pmd_same(*pmd, newpmd)) - set_pmd_at(mm, addr, pmd, newpmd); + if (thp_migration_supported() && is_swap_pmd(*pmd)) { + change_non_present_huge_pmd(mm, addr, pmd, uffd_wp, + uffd_wp_resolve); goto unlock; } -#endif =20 if (prot_numa) { =20 --=20 2.51.0 From nobody Sat Feb 7 17:55:59 2026 Received: from mx0a-00069f02.pphosted.com (mx0a-00069f02.pphosted.com [205.220.165.32]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 87E5030F957; Mon, 3 Nov 2025 12:34:25 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=205.220.165.32 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1762173268; cv=fail; b=KWXz/tHfKyXnKKyuVZOhNNR5/Z1ayPxlUtuj8IGuLlBY8zXn4HC8MsQ1usDlsJWoLFGawAOtdNRPLIllWWYXYabyx9p5Cp+jaXuwHeQj/0gU0YGCelaX7zNGAoAH5UZBPKYSHPYUJ3Ji1nwoXMj9bOxzm/ye33pKtBZgyIcGp/8= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1762173268; c=relaxed/simple; bh=0bhCH6BKAJKHCHuIJEOTigcn6iNhUER+/FHq6yvAFNA=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: Content-Type:MIME-Version; b=Sr/pHPMS8kCM7HZ/iaN2Rkri6RPbpubdMkINYScmdwFVSYYN1kLFB9FUVde6o/SSxchiXznL72YlKnXThFn8sv+BcgX83ec45uf2eOP+SImQBzs6p9NfPJ0NzSjoBioePhQqrmxAixddFNIYtXcBxfxQ8FQeV++wGsrFggGrvXo= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=oracle.com; spf=pass smtp.mailfrom=oracle.com; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b=ljUyvMHw; dkim=pass (1024-bit key) header.d=oracle.onmicrosoft.com header.i=@oracle.onmicrosoft.com header.b=E3BHRhCS; arc=fail smtp.client-ip=205.220.165.32 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=oracle.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=oracle.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b="ljUyvMHw"; dkim=pass (1024-bit key) header.d=oracle.onmicrosoft.com header.i=@oracle.onmicrosoft.com header.b="E3BHRhCS" Received: from pps.filterd (m0333521.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 5A3CTePi019190; Mon, 3 Nov 2025 12:32:43 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=cc :content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s= corp-2025-04-25; bh=IkriKesgB/vxrru3u50o8MIIikPKy++jEbdYcLdyS+g=; b= ljUyvMHwEm6hVcJsVLoM4JzqOLAqH74fynJZPy3G7d0Gq3tbVqFz0zC0V992V00R dU3i26wLXwcaNPetvomXjIhqQ4uO5+qvBWEMSTRqHWz/X3cF8t30FBXpT/Ar5Oz7 iZjesPvXhOXBm+KEncTQKvqwFtNKOdNdjgqp0euRIqxSWRQOJiGwVj1TvRd4jB51 mbmBmT+U5bFjLCNnlSNRL6fvJ/9To7LHDm9sWGKV5h8cD6yQ278ClH5GnPBXSORT KxKKXK83O+gipfQ0B9sljfXHGrzLtzAp8yzLCfVm1sO+p7m5i73OjCeIVVyDXsY0 96vC31Hp9s27zIgzDgcuLw== Received: from iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta01.appoci.oracle.com [130.35.100.223]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 4a6ven8091-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 03 Nov 2025 12:32:42 +0000 (GMT) Received: from pps.filterd (iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com (8.18.1.2/8.18.1.2) with ESMTP id 5A3BWos8007980; Mon, 3 Nov 2025 12:32:41 GMT Received: from sj2pr03cu001.outbound.protection.outlook.com (mail-westusazon11012056.outbound.protection.outlook.com [52.101.43.56]) by iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTPS id 4a58nbkq71-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 03 Nov 2025 12:32:41 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=BxVRd+l1SDMPWK8E9sG7hcvISMzCbF0bO1E7mxMWz3e7jSNNDIuNBOgUN3jbR24Fl8zAqcIi+WVNUC/lIPbXYbH87wtJqcWiWfUn7u9dIVSt/pJOQrZ4CujxVFfLiSeJQioYN9RYOWGDPaedHO572spAyNwNPMp+9kjD/4c9iDozxP/hTpp9585Dm0ck51PJJqwXtyFlSODG1hzpsTxsBmC4FoJ//lOyRvi10CrR8kwlN5m8n+pNzxZwSpFa1ryCbjVbCLjKINWZ2ZoyzK8EZi6EatXqYjGv/e5tKUyowU0OxzAoeGVpiTDTXQTNNWx2nT+rcr49HCHoMePZDPs6dg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=IkriKesgB/vxrru3u50o8MIIikPKy++jEbdYcLdyS+g=; b=WaYber9LmjNNXqdijTLeFnSvmMx3GUwV6R634ii0gKhI0VBbKoGkGhUCQ4i/NIXvguUiIUerZW16PUQv72zdaDfsGkYAtttkRNlvYKU/4kO1zO5AavWx5SwDgMjf9H3EFwi0jrWrBCJ73nCGpJ+jf2FCJVY9Euseac3UYvg4Tm4Z4httysu25Ldhw3OUZN3lO5NAsEFynGHtW9DxWwYX6DEjiVWvfFeeAroVUFNyZbgVCpiO6Xv3wKw+iBAaClsi5qCXjJKl1qY4XWnb+0tZDt8/LNhtNPfXjnWfllHcYHdloY2/Mwgn4VzeN/0lISQgsWfaB7+cbjSpI7N21zGuxw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=IkriKesgB/vxrru3u50o8MIIikPKy++jEbdYcLdyS+g=; b=E3BHRhCSppzi6Ao81RPa/0LKeBXw0osrASTpgndzE9wLmSh276RkMB/DrWmaAJy1gvl8uOIHAHohPTZx3HGcFaVm9XNqDNuQvoJmuDX/IN6DXRy8IWObUxKraedt8wGF4k5c7a7mtyUVGFj3yqpXWKI9U7gNnLCEBH413cdP3ZU= Received: from DM4PR10MB8218.namprd10.prod.outlook.com (2603:10b6:8:1cc::16) by DS4PR10MB997575.namprd10.prod.outlook.com (2603:10b6:8:31e::5) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9275.16; Mon, 3 Nov 2025 12:32:37 +0000 Received: from DM4PR10MB8218.namprd10.prod.outlook.com ([fe80::2650:55cf:2816:5f2]) by DM4PR10MB8218.namprd10.prod.outlook.com ([fe80::2650:55cf:2816:5f2%2]) with mapi id 15.20.9253.018; Mon, 3 Nov 2025 12:32:37 +0000 From: Lorenzo Stoakes To: Andrew Morton Cc: Christian Borntraeger , Janosch Frank , Claudio Imbrenda , David Hildenbrand , Alexander Gordeev , Gerald Schaefer , Heiko Carstens , Vasily Gorbik , Sven Schnelle , Peter Xu , Alexander Viro , Christian Brauner , Jan Kara , Arnd Bergmann , Zi Yan , Baolin Wang , "Liam R . Howlett" , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Lance Yang , Muchun Song , Oscar Salvador , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Matthew Brost , Joshua Hahn , Rakie Kim , Byungchul Park , Gregory Price , Ying Huang , Alistair Popple , Axel Rasmussen , Yuanchu Xie , Wei Xu , Kemeng Shi , Kairui Song , Nhat Pham , Baoquan He , Chris Li , SeongJae Park , Matthew Wilcox , Jason Gunthorpe , Leon Romanovsky , Xu Xin , Chengming Zhou , Jann Horn , Miaohe Lin , Naoya Horiguchi , Pedro Falcato , Pasha Tatashin , Rik van Riel , Harry Yoo , Hugh Dickins , linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-s390@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, linux-arch@vger.kernel.org, damon@lists.linux.dev Subject: [PATCH 10/16] mm: replace pmd_to_swp_entry() with leafent_from_pmd() Date: Mon, 3 Nov 2025 12:31:51 +0000 Message-ID: <38c26e75ed00263e9ecbebb0c045dd6d8183ec67.1762171281.git.lorenzo.stoakes@oracle.com> X-Mailer: git-send-email 2.51.0 In-Reply-To: References: Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: LO4P265CA0161.GBRP265.PROD.OUTLOOK.COM (2603:10a6:600:2c7::8) To DM4PR10MB8218.namprd10.prod.outlook.com (2603:10b6:8:1cc::16) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DM4PR10MB8218:EE_|DS4PR10MB997575:EE_ X-MS-Office365-Filtering-Correlation-Id: d53f5c61-41a5-4473-f97e-08de1ad5126a X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|376014|7416014|366016|1800799024; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?UGd5ZO5jcEVH0RUsubA99AJs5BekJRXy4sRZUoq7UiBNtJI/6cgTC1KbwHws?= =?us-ascii?Q?fNsUKO4J0sDMtlSzY0wpjEc+aUgvRa8VFcGcoWcqxztM8kN3dsRe3Kg5WDLg?= =?us-ascii?Q?PrltEN9VLDjlauoNS/pRC8+8v83e1PQJxOcgurVG2V4/W3zQfC3SoDqhmgI0?= =?us-ascii?Q?HFXxAkxkvHEPrUmil9zyGb+FLsKiO64n5gAIB2759LIQNl/FCqZRTJitjkyS?= =?us-ascii?Q?6AfEYUFSW4GveHXHkrlfDXdgh/y6nUWOuQMhFLEuBmyeQpAALBuvqbVlQ5dC?= =?us-ascii?Q?Lpjlp2vdVtVdmxslsX7NYwzx4r8dbstL6GtF8HYU6GCuQOll0VogoeVEh0o9?= =?us-ascii?Q?B7EW/HmLuT4JVivG9HJoz9MDi+FcAckZrqm94m1R0MjEe4MWBm9SiGPKPozA?= =?us-ascii?Q?Anld5LWnLEiYPhV8N04/hcj5AEZM5l0zrLGvrTGBAeMfnJjSp64dPtHDvPAx?= =?us-ascii?Q?CCqbj5vtXGVHVhbPhAAyn+k7Tn67g7GQ2ckiFhQ/Lkvj3e26bK1QaNTT5gSV?= =?us-ascii?Q?9uVOdXeOK8B8nz+umRkfU+z7uww3Q6CpZ/Qio7ldyUWd43plncT98HpO32SN?= =?us-ascii?Q?5/j1gC86/rxnXbRFnegzwqOk5od4Dh1f7Q5GQiQTaJNIJP+JCmUxrD8bpmiE?= =?us-ascii?Q?wrK5q3gFYcbUft6XlMgX/yDBxSmYxryO4ssLKp+mlFSTS+DIuXzi8n8XIN86?= =?us-ascii?Q?Gk+0zv8w22D+f/0LIMlhStN38dgz32tBADofI/S+CDFsveoDxzmobAt5iDFZ?= =?us-ascii?Q?JxHQeOBmKURugjBqZQp7rsq4V3KRj6vNK1/D7VrI1WlSYGeG+IDXj0ZFQ3+A?= =?us-ascii?Q?NeRyBL+KctOrhbFEhihTqWpAzyAExJ2m4mOqeBYg6daeRj+QNOBh1lMKWW8S?= =?us-ascii?Q?zebX3FDFEi00ciWEsrKyso8wLACwuK8xmut3XGStDTKmfeimu7YxNMb6qObY?= =?us-ascii?Q?rLJe52CpNQEKgeTCL1r2Nd5baYVpcuiX9p9XxnlNUolFAJyxhX7IMwIfZiub?= =?us-ascii?Q?etAmw8WIiSuo9NTnwvAhiC/GR0Lb2gIR51O8ZZECPv+eVHn5W70YsUCtteG6?= =?us-ascii?Q?/6gdpl/xlCKwjBxPnlU2JC4K1D2HSCEfrezhRRVz/Dc1/RdtvurLMispMt0d?= =?us-ascii?Q?5BcdIDU8YnX/TR6J2EvfF2fcCRI4GFa4GqNAkxfzvcfA4wDtFs871phWIFIu?= =?us-ascii?Q?SAOn7JUf9+47352zYfJsuy3dB9k73WHIY7PdGWWOanfVEQGw0OJTuYEbfgqR?= =?us-ascii?Q?1zIp9cXlbuMAFOwIzfeZHHETe7LS1UmsTpMbjqbqVGqKc+aQBFcVdL8hmfZr?= =?us-ascii?Q?D/qG+qtRzg19TXC4fjCCLKh82bInVOIAYWcQtmwhvA55TtOv9B8cyJ318PKr?= =?us-ascii?Q?GpDrrwOzwB81ZnoQskOjk5XMUzdTqIGGTLmtRHqmX9AiCvWO1h6ih8nMf4BG?= =?us-ascii?Q?TUR2BXK710dPVMl/7oeXojFRLJN3YgNW?= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DM4PR10MB8218.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(376014)(7416014)(366016)(1800799024);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?9uaiCnG6HuD96xxV6j9vwfa2rLsNjdIm4T9nbQYGGN8pc2yOgDTGUBnwXZ8A?= =?us-ascii?Q?giCo2jzmEVjMhf61kwg9UV1st5HNqimfkCPm+ngMpzJv9jbYcoKTjXDJfGmh?= =?us-ascii?Q?cLBcOpYSI0TNh97lAXlcer+dF3wkZLKcwANfp/EkXPvwYFnOpqr231pj28Cp?= =?us-ascii?Q?kkgEy/Wr2O40Om5ov2C54ZW6V5JZav4G8+GeJycLtPxPI3hIQz9o/5Mbuakh?= =?us-ascii?Q?w2caRMy9uty0tVQ7dn6M2ZquO2zUMN9MWaGQHwhTuIoGchujI4Xqbu8UpGpl?= =?us-ascii?Q?c55BW9AQdRaLZAoqwShWtf2emnJM2UEuTr8tyOF2n0SEO9AsPRXFPUBTpQSf?= =?us-ascii?Q?axK75JBh56sEhK+Ujc64n86/17B5SajXk38Y8L/HQq18aofgVeuhx2fXJ0Ky?= =?us-ascii?Q?a25jCbrhWI/z6iLFezJsGAG5U/Q/du3b5A09sWlVbwi0opl5+qPsim3nCTgo?= =?us-ascii?Q?1yDSIhugnHY6mV6FbXX/FmF1ilg0o8fXU16ktWQvYVSpZ3LMJxhahKBpmLPe?= =?us-ascii?Q?1jDczfCr6MZvkimIr7DmsjbVrBCG1R9lecwH5MBKxI7sQ0CiihbEKTCZe8T1?= =?us-ascii?Q?1vgU+D45i2I5firoRnRL0bYwrQvZvVGnF/xPZSDMtDyncdUR0G2u8lQ8URqI?= =?us-ascii?Q?GtissYy9EPO3KxTEsGoysgk26Nbbm6pDngmnYnt/K3wMWW/Yz0qBu6akh2FM?= =?us-ascii?Q?x9aVYDD0lTEannZaIbHLNqd31oUovKSag1jDiCNtpfBaCTT5CJTcl1AdZ1Nl?= =?us-ascii?Q?O5ujAheAeLHtIsLfps/20qMKpXaIIgYtGps7Xzh6+ilwhsb1hLwyOHeTjM8D?= =?us-ascii?Q?FDYD+DcgYD0rvTc5Pl1LloeD9kuGFVRPClgAZA19H/mQZVYVwtGFQnAhfPx2?= =?us-ascii?Q?GtKncSYMkjFxSrb4pqIw/D96Vi74xQ4TC+iZJhkhZGE+mlnP7E72xOcNOAiI?= =?us-ascii?Q?ZEVyvrzAWC5ff/nEITxvmVQg+ib0/GHzD33eulc5UmEw/OM6NBQOzVXFa9iq?= =?us-ascii?Q?TOZSHWsTwy0WN1lYS64BC1GGxWjSs4zDXZbbC9Xd4W60uchUyGkBLAuiKR2w?= =?us-ascii?Q?RfEcSB9/rlFg/7thIhQUt13geBYzQcC+zlVIljTqcWEU63Qkr/9cfeM70/6d?= =?us-ascii?Q?VB6De2c/50UcC8sw2aTKK+9DY/dL00EcQvaIHocdgbrZOEjrhdtXXtZ5P87Z?= =?us-ascii?Q?BeAOZFslR82SWqy5WEibfWzbc2lWi/QFdGiEQKnJmluCQxJsbldMT7AahFCL?= =?us-ascii?Q?HKAhW/ZfboHb0bWxt7rBk539Q5zwwGat/AKsWZvLG8BE8ndjLHe4VYZbtAjE?= =?us-ascii?Q?reGMtEsMg3rfU8PQuRXBOsTN2BhEMlrxyfDh61JFM2j1ymXn9GQIzUgBvjRd?= =?us-ascii?Q?KfGF9een65tBzuqwbjQPYB9ri/3sseJrkVlSRv/3lrBFRE4ds4p+BqNR1Sh7?= =?us-ascii?Q?zZP3McQ2paeAoJ7TYc9E4iuttYjgOX92aSvMzgTZF12QfrL/O17nv1MVCrlc?= =?us-ascii?Q?XReIFbBRsjYw3MundSLEjgHbaJG2esHBqICsM6MA7Zpr1Cmooo6aqzl72GJn?= =?us-ascii?Q?1LOqEaLYR1pin9jb68adcZ2sekSTaB8PjNmmN1RY5k+1Y+kuCT9XE2Je5za8?= =?us-ascii?Q?cw=3D=3D?= X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: tccvKpUClZ4LroST6HnWEsCBMU/C7wfXQtApRXYLQyhKKjiWL+4BYBrOxqD62Th0yFWaR14FLaoWCsXaOoXox9yM007CWBLwr/Vu+1gEfgxNug/c5Ho5/FRIaH3r8ISOJRw0Nn8nFm1sTxSyDO9G597iT/g1ZrlHqsxgRZXR58mrU//nOoyTjq+ZhiIaVdvvMbrAJSNB3e9oDVGDgD9y/FrbY5P37dJfKqRm8SV8UdlBNruwZYay0KlxoUNp0PCrJKwJUYMgRcja0rxm/DgKKoBmo3T+LZu63T7cCBEpXAAoogeyLNw/7Dp6eYY0LNdq0gjw9WxoTFoQ0BJHypsPqgh0HVLpNjSoEx9TE8QZl8iEBGKZL2Il02wYr3AKvKOlZweOu4Bht2SJMni79Yd9FrEqigQIzTmKhq52YSGuXCbcS3xtHKi79KWTioIAB/Ao5aaUJZTdQWoxudBgxM2m2DHVs1op5y0FFxk1spJCrGn0uf0PNl0Af9eAds7B0JmNY3Ny+fmJotE7/boOBIR1Bqw8jLHQ5VwSx3k5+puDcqpwA2ZRro2rdRefjZwHYn0JB0KCieuHCTNJ2u0OfkQqz/zKbhQioxjer2lapf2Bc68= X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: d53f5c61-41a5-4473-f97e-08de1ad5126a X-MS-Exchange-CrossTenant-AuthSource: DM4PR10MB8218.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 03 Nov 2025 12:32:37.4297 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: hOGYuVIFmUMD9veVjjBSRDF5Z5EVDF/cB2JJJcj4DjJDtq7Halbe4Z8texqC8SvWI+sB/i4B6/7QQQtTxRiCBcg2MICfH8p1XmeVcOfHJQ4= X-MS-Exchange-Transport-CrossTenantHeadersStamped: DS4PR10MB997575 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1121,Hydra:6.1.9,FMLib:17.12.100.49 definitions=2025-11-03_01,2025-11-03_01,2025-10-01_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 phishscore=0 malwarescore=0 mlxlogscore=999 mlxscore=0 adultscore=0 spamscore=0 bulkscore=0 suspectscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2510240000 definitions=main-2511030113 X-Proofpoint-GUID: a6bGNVlpbZQ_wwmP3ZoV83MQuvU3xvf5 X-Proofpoint-ORIG-GUID: a6bGNVlpbZQ_wwmP3ZoV83MQuvU3xvf5 X-Proofpoint-Spam-Details-Enc: AW1haW4tMjUxMTAzMDExNCBTYWx0ZWRfXxGRNX/EUwMOW 8oN47L6kSXuuZKusT3Y7HcOybqwAfL+1e+nSpbdFoEvVALE5vqzv9eyN3DDjeKTUO32Fi6QOZez gRUnhbsXyDZZooz3ZDhEtcPT9fW7T9WmIQcCnKkazyzTpryEnl/HSS03RYiKq67irurW0rNck/4 0+d0RJdnhg/MDk8L10MS2lKG9DuiLms3cIbnlupeS1MN3LIAGz/KqFAOeNIh/VvDB3MA/gogRgj TcLNshudSlHS2rhOLBCGmyYY6Yto1oZeu5qKlxVfgwr7P7CoQsm8uEBVnX1unVQXHPWk2uXrSYo OEzCVLofO+NMeqD58D93ncZ85KzEJ5F7+2YcT29J5s6brlvU+9zIQH2kkZXwF6ZLDw22nS/mGWy YbrRt9OzSMZWWxlAYd0cYFEU9nECzRSvOa+ZFibBF/zfbmmMfm4= X-Authority-Analysis: v=2.4 cv=MvBfKmae c=1 sm=1 tr=0 ts=6908a0eb b=1 cx=c_pps a=zPCbziy225d3KhSqZt3L1A==:117 a=zPCbziy225d3KhSqZt3L1A==:17 a=6eWqkTHjU83fiwn7nKZWdM+Sl24=:19 a=z/mQ4Ysz8XfWz/Q5cLBRGdckG28=:19 a=lCpzRmAYbLLaTzLvsPZ7Mbvzbb8=:19 a=xqWC_Br6kY4A:10 a=6UeiqGixMTsA:10 a=GoEa3M9JfhUA:10 a=VkNPw1HP01LnGYTKEx00:22 a=yPCof4ZbAAAA:8 a=1B2o6Llj8-7n3TOjnuQA:9 cc=ntf awl=host:12123 Content-Type: text/plain; charset="utf-8" Introduce leafent_from_pmd() to do the equivalent operation for PMDs that leafent_from_pte() fulfils, and cascade changes through code base accordingly, introducing helpers as necessary. We are then able to eliminate pmd_to_swp_entry(), is_pmd_migration_entry(), is_pmd_device_private_entry() and is_pmd_non_present_folio_entry(). This further establishes the use of leaf operations throughout the code base and further establishes the foundations for eliminating is_swap_pmd(). No functional change intended. Signed-off-by: Lorenzo Stoakes --- fs/proc/task_mmu.c | 27 +++--- include/linux/leafops.h | 210 ++++++++++++++++++++++++++++++++++++++++ include/linux/migrate.h | 3 +- include/linux/swapops.h | 100 ------------------- mm/damon/ops-common.c | 6 +- mm/filemap.c | 6 +- mm/hmm.c | 16 +-- mm/huge_memory.c | 98 +++++++++---------- mm/khugepaged.c | 4 +- mm/madvise.c | 2 +- mm/memory.c | 4 +- mm/mempolicy.c | 4 +- mm/migrate.c | 20 ++-- mm/migrate_device.c | 14 +-- mm/page_table_check.c | 16 +-- mm/page_vma_mapped.c | 15 +-- mm/pagewalk.c | 8 +- mm/rmap.c | 4 +- 18 files changed, 334 insertions(+), 223 deletions(-) diff --git a/fs/proc/task_mmu.c b/fs/proc/task_mmu.c index 0f02bda5d544..0ccdc21e60e0 100644 --- a/fs/proc/task_mmu.c +++ b/fs/proc/task_mmu.c @@ -1065,10 +1065,10 @@ static void smaps_pmd_entry(pmd_t *pmd, unsigned lo= ng addr, page =3D vm_normal_page_pmd(vma, addr, *pmd); present =3D true; } else if (unlikely(thp_migration_supported())) { - swp_entry_t entry =3D pmd_to_swp_entry(*pmd); + const leaf_entry_t entry =3D leafent_from_pmd(*pmd); =20 - if (is_pfn_swap_entry(entry)) - page =3D pfn_swap_entry_to_page(entry); + if (leafent_has_pfn(entry)) + page =3D leafent_to_page(entry); } if (IS_ERR_OR_NULL(page)) return; @@ -1654,7 +1654,7 @@ static inline void clear_soft_dirty_pmd(struct vm_are= a_struct *vma, pmd =3D pmd_clear_soft_dirty(pmd); =20 set_pmd_at(vma->vm_mm, addr, pmdp, pmd); - } else if (is_migration_entry(pmd_to_swp_entry(pmd))) { + } else if (pmd_is_migration_entry(pmd)) { pmd =3D pmd_swp_clear_soft_dirty(pmd); set_pmd_at(vma->vm_mm, addr, pmdp, pmd); } @@ -2015,12 +2015,12 @@ static int pagemap_pmd_range_thp(pmd_t *pmdp, unsig= ned long addr, if (pm->show_pfn) frame =3D pmd_pfn(pmd) + idx; } else if (thp_migration_supported()) { - swp_entry_t entry =3D pmd_to_swp_entry(pmd); + const leaf_entry_t entry =3D leafent_from_pmd(pmd); unsigned long offset; =20 if (pm->show_pfn) { - if (is_pfn_swap_entry(entry)) - offset =3D swp_offset_pfn(entry) + idx; + if (leafent_has_pfn(entry)) + offset =3D leafent_to_pfn(entry) + idx; else offset =3D swp_offset(entry) + idx; frame =3D swp_type(entry) | @@ -2031,7 +2031,7 @@ static int pagemap_pmd_range_thp(pmd_t *pmdp, unsigne= d long addr, flags |=3D PM_SOFT_DIRTY; if (pmd_swp_uffd_wp(pmd)) flags |=3D PM_UFFD_WP; - VM_WARN_ON_ONCE(!is_pmd_migration_entry(pmd)); + VM_WARN_ON_ONCE(!pmd_is_migration_entry(pmd)); page =3D pfn_swap_entry_to_page(entry); } =20 @@ -2425,8 +2425,6 @@ static unsigned long pagemap_thp_category(struct page= map_scan_private *p, if (pmd_soft_dirty(pmd)) categories |=3D PAGE_IS_SOFT_DIRTY; } else { - swp_entry_t swp; - categories |=3D PAGE_IS_SWAPPED; if (!pmd_swp_uffd_wp(pmd)) categories |=3D PAGE_IS_WRITTEN; @@ -2434,9 +2432,10 @@ static unsigned long pagemap_thp_category(struct pag= emap_scan_private *p, categories |=3D PAGE_IS_SOFT_DIRTY; =20 if (p->masks_of_interest & PAGE_IS_FILE) { - swp =3D pmd_to_swp_entry(pmd); - if (is_pfn_swap_entry(swp) && - !folio_test_anon(pfn_swap_entry_folio(swp))) + const leaf_entry_t entry =3D leafent_from_pmd(pmd); + + if (leafent_has_pfn(entry) && + !folio_test_anon(leafent_to_folio(entry))) categories |=3D PAGE_IS_FILE; } } @@ -2453,7 +2452,7 @@ static void make_uffd_wp_pmd(struct vm_area_struct *v= ma, old =3D pmdp_invalidate_ad(vma, addr, pmdp); pmd =3D pmd_mkuffd_wp(old); set_pmd_at(vma->vm_mm, addr, pmdp, pmd); - } else if (is_migration_entry(pmd_to_swp_entry(pmd))) { + } else if (pmd_is_migration_entry(pmd)) { pmd =3D pmd_swp_mkuffd_wp(pmd); set_pmd_at(vma->vm_mm, addr, pmdp, pmd); } diff --git a/include/linux/leafops.h b/include/linux/leafops.h index a1a25ca152ff..2d3bc4c866bd 100644 --- a/include/linux/leafops.h +++ b/include/linux/leafops.h @@ -73,6 +73,47 @@ static inline leaf_entry_t leafent_from_pte(pte_t pte) return pte_to_swp_entry(pte); } =20 +/** + * leafent_to_pte() - Obtain a PTE entry from a leaf entry. + * @entry: Leaf entry. + * + * This generates an architecture-specific PTE entry that can be utilised = to + * encode the metadata the leaf entry encodes. + * + * Returns: Architecture-specific PTE entry encoding leaf entry. + */ +static inline pte_t leafent_to_pte(leaf_entry_t entry) +{ + /* Temporary until swp_entry_t eliminated. */ + return swp_entry_to_pte(entry); +} + +/** + * leafent_from_pmd() - Obtain a leaf entry from a PMD entry. + * @pmd: PMD entry. + * + * If @pmd is present (therefore not a leaf entry) the function returns an= empty + * leaf entry. Otherwise, it returns a leaf entry. + * + * Returns: Leaf entry. + */ +static inline leaf_entry_t leafent_from_pmd(pmd_t pmd) +{ + leaf_entry_t arch_entry; + + if (pmd_present(pmd)) + return leafent_mk_none(); + + if (pmd_swp_soft_dirty(pmd)) + pmd =3D pmd_swp_clear_soft_dirty(pmd); + if (pmd_swp_uffd_wp(pmd)) + pmd =3D pmd_swp_clear_uffd_wp(pmd); + arch_entry =3D __pmd_to_swp_entry(pmd); + + /* Temporary until swp_entry_t eliminated. */ + return swp_entry(__swp_type(arch_entry), __swp_offset(arch_entry)); +} + /** * leafent_is_none() - Is the leaf entry empty? * @entry: Leaf entry. @@ -146,6 +187,43 @@ static inline bool leafent_is_swap(leaf_entry_t entry) return leafent_type(entry) =3D=3D LEAFENT_SWAP; } =20 +/** + * leafent_is_migration_write() - Is this leaf entry a writable migration = entry? + * @entry: Leaf entry. + * + * Returns: true if the leaf entry is a writable migration entry, otherwise + * false. + */ +static inline bool leafent_is_migration_write(leaf_entry_t entry) +{ + return leafent_type(entry) =3D=3D LEAFENT_MIGRATION_WRITE; +} + +/** + * leafent_is_migration_read() - Is this leaf entry a readable migration e= ntry? + * @entry: Leaf entry. + * + * Returns: true if the leaf entry is a readable migration entry, otherwise + * false. + */ +static inline bool leafent_is_migration_read(leaf_entry_t entry) +{ + return leafent_type(entry) =3D=3D LEAFENT_MIGRATION_READ; +} + +/** + * leafent_is_migration_read_exclusive() - Is this leaf entry an exclusive + * readable migration entry? + * @entry: Leaf entry. + * + * Returns: true if the leaf entry is an exclusive readable migration entr= y, + * otherwise false. + */ +static inline bool leafent_is_migration_read_exclusive(leaf_entry_t entry) +{ + return leafent_type(entry) =3D=3D LEAFENT_MIGRATION_READ_EXCLUSIVE; +} + /** * leafent_is_swap() - Is this leaf entry a migration entry? * @entry: Leaf entry. @@ -164,6 +242,19 @@ static inline bool leafent_is_migration(leaf_entry_t e= ntry) } } =20 +/** + * leafent_is_device_private_write() - Is this leaf entry a device private + * writable entry? + * @entry: Leaf entry. + * + * Returns: true if the leaf entry is a device private writable entry, oth= erwise + * false. + */ +static inline bool leafent_is_device_private_write(leaf_entry_t entry) +{ + return leafent_type(entry) =3D=3D LEAFENT_DEVICE_PRIVATE_WRITE; +} + /** * leafent_is_device_private() - Is this leaf entry a device private entry? * @entry: Leaf entry. @@ -181,6 +272,12 @@ static inline bool leafent_is_device_private(leaf_entr= y_t entry) } } =20 +/** + * leafent_is_device_exclusive() - Is this leaf entry a device-exclusive e= ntry? + * @entry: Leaf entry. + * + * Returns: true if the leaf entry is a device-exclusive entry, otherwise = false. + */ static inline bool leafent_is_device_exclusive(leaf_entry_t entry) { return leafent_type(entry) =3D=3D LEAFENT_DEVICE_EXCLUSIVE; @@ -339,6 +436,61 @@ static inline bool leafent_is_uffd_wp_marker(leaf_entr= y_t entry) return leafent_to_marker(entry) & PTE_MARKER_UFFD_WP; } =20 +#ifdef CONFIG_MIGRATION + +/** + * leafent_is_migration_young() - Does this migration entry contain an acc= essed + * bit? + * @entry: Leaf entry. + * + * If the architecture can support storing A/D bits in migration entries, = this + * determines whether the accessed (or 'young') bit was set on the migrate= d page + * table entry. + * + * Returns: true if the entry contains an accessed bit, otherwise false. + */ +static inline bool leafent_is_migration_young(leaf_entry_t entry) +{ + VM_WARN_ON_ONCE(!leafent_is_migration(entry)); + + if (migration_entry_supports_ad()) + return swp_offset(entry) & SWP_MIG_YOUNG; + /* Keep the old behavior of aging page after migration */ + return false; +} + +/** + * leafent_is_migration_dirty() - Does this migration entry contain a dirt= y bit? + * @entry: Leaf entry. + * + * If the architecture can support storing A/D bits in migration entries, = this + * determines whether the dirty bit was set on the migrated page table ent= ry. + * + * Returns: true if the entry contains a dirty bit, otherwise false. + */ +static inline bool leafent_is_migration_dirty(leaf_entry_t entry) +{ + VM_WARN_ON_ONCE(!leafent_is_migration(entry)); + + if (migration_entry_supports_ad()) + return swp_offset(entry) & SWP_MIG_DIRTY; + /* Keep the old behavior of clean page after migration */ + return false; +} + +#else /* CONFIG_MIGRATION */ + +static inline bool leafent_is_migration_young(leaf_entry_t entry) +{ + return false; +} + +static inline bool leafent_is_migration_dirty(leaf_entry_t entry) +{ + return false; +} +#endif /* CONFIG_MIGRATION */ + /** * pte_is_marker() - Does the PTE entry encode a marker leaf entry? * @pte: PTE entry. @@ -390,5 +542,63 @@ static inline bool pte_is_uffd_marker(pte_t pte) return false; } =20 +#if defined(CONFIG_ZONE_DEVICE) && defined(CONFIG_ARCH_ENABLE_THP_MIGRATIO= N) + +/** + * pmd_is_device_private_entry() - Check if PMD contains a device private = swap + * entry. + * @pmd: The PMD to check. + * + * Returns true if the PMD contains a swap entry that represents a device = private + * page mapping. This is used for zone device private pages that have been + * swapped out but still need special handling during various memory manag= ement + * operations. + * + * Return: true if PMD contains device private entry, false otherwise + */ +static inline bool pmd_is_device_private_entry(pmd_t pmd) +{ + return leafent_is_device_private(leafent_from_pmd(pmd)); +} + +#else /* CONFIG_ZONE_DEVICE && CONFIG_ARCH_ENABLE_THP_MIGRATION */ + +static inline bool pmd_is_device_private_entry(pmd_t pmd) +{ + return false; +} + +#endif /* CONFIG_ZONE_DEVICE && CONFIG_ARCH_ENABLE_THP_MIGRATION */ + +/** + * pmd_is_migration_entry() - Does this PMD entry encode a migration entry? + * @pmd: PMD entry. + * + * Returns: true if the PMD encodes a migration entry, otherwise false. + */ +static inline bool pmd_is_migration_entry(pmd_t pmd) +{ + return leafent_is_migration(leafent_from_pmd(pmd)); +} + +/** + * pmd_is_valid_leafent() - Is this PMD entry a valid leaf entry? + * @pmd: PMD entry. + * + * PMD leaf entries are valid only if they are device private or migration + * entries. This function asserts that a PMD leaf entry is valid in this + * respect. + * + * Returns: true if the PMD entry is a valid leaf entry, otherwise false. + */ +static inline bool pmd_is_valid_leafent(pmd_t pmd) +{ + const leaf_entry_t entry =3D leafent_from_pmd(pmd); + + /* Only device private, migration entries valid for PMD. */ + return leafent_is_device_private(entry) || + leafent_is_migration(entry); +} + #endif /* CONFIG_MMU */ #endif /* _LINUX_SWAPOPS_H */ diff --git a/include/linux/migrate.h b/include/linux/migrate.h index 41b4cc05a450..010b70c4dc3e 100644 --- a/include/linux/migrate.h +++ b/include/linux/migrate.h @@ -6,6 +6,7 @@ #include #include #include +#include =20 typedef struct folio *new_folio_t(struct folio *folio, unsigned long priva= te); typedef void free_folio_t(struct folio *folio, unsigned long private); @@ -65,7 +66,7 @@ bool isolate_folio_to_list(struct folio *folio, struct li= st_head *list); =20 int migrate_huge_page_move_mapping(struct address_space *mapping, struct folio *dst, struct folio *src); -void migration_entry_wait_on_locked(swp_entry_t entry, spinlock_t *ptl) +void migration_entry_wait_on_locked(leaf_entry_t entry, spinlock_t *ptl) __releases(ptl); void folio_migrate_flags(struct folio *newfolio, struct folio *folio); int folio_migrate_mapping(struct address_space *mapping, diff --git a/include/linux/swapops.h b/include/linux/swapops.h index 3e8dd6ea94dd..f1277647262d 100644 --- a/include/linux/swapops.h +++ b/include/linux/swapops.h @@ -283,14 +283,6 @@ static inline swp_entry_t make_migration_entry_young(s= wp_entry_t entry) return entry; } =20 -static inline bool is_migration_entry_young(swp_entry_t entry) -{ - if (migration_entry_supports_ad()) - return swp_offset(entry) & SWP_MIG_YOUNG; - /* Keep the old behavior of aging page after migration */ - return false; -} - static inline swp_entry_t make_migration_entry_dirty(swp_entry_t entry) { if (migration_entry_supports_ad()) @@ -299,14 +291,6 @@ static inline swp_entry_t make_migration_entry_dirty(s= wp_entry_t entry) return entry; } =20 -static inline bool is_migration_entry_dirty(swp_entry_t entry) -{ - if (migration_entry_supports_ad()) - return swp_offset(entry) & SWP_MIG_DIRTY; - /* Keep the old behavior of clean page after migration */ - return false; -} - extern void migration_entry_wait(struct mm_struct *mm, pmd_t *pmd, unsigned long address); extern void migration_entry_wait_huge(struct vm_area_struct *vma, unsigned= long addr, pte_t *pte); @@ -349,20 +333,11 @@ static inline swp_entry_t make_migration_entry_young(= swp_entry_t entry) return entry; } =20 -static inline bool is_migration_entry_young(swp_entry_t entry) -{ - return false; -} - static inline swp_entry_t make_migration_entry_dirty(swp_entry_t entry) { return entry; } =20 -static inline bool is_migration_entry_dirty(swp_entry_t entry) -{ - return false; -} #endif /* CONFIG_MIGRATION */ =20 #ifdef CONFIG_MEMORY_FAILURE @@ -487,18 +462,6 @@ extern void remove_migration_pmd(struct page_vma_mappe= d_walk *pvmw, =20 extern void pmd_migration_entry_wait(struct mm_struct *mm, pmd_t *pmd); =20 -static inline swp_entry_t pmd_to_swp_entry(pmd_t pmd) -{ - swp_entry_t arch_entry; - - if (pmd_swp_soft_dirty(pmd)) - pmd =3D pmd_swp_clear_soft_dirty(pmd); - if (pmd_swp_uffd_wp(pmd)) - pmd =3D pmd_swp_clear_uffd_wp(pmd); - arch_entry =3D __pmd_to_swp_entry(pmd); - return swp_entry(__swp_type(arch_entry), __swp_offset(arch_entry)); -} - static inline pmd_t swp_entry_to_pmd(swp_entry_t entry) { swp_entry_t arch_entry; @@ -507,23 +470,7 @@ static inline pmd_t swp_entry_to_pmd(swp_entry_t entry) return __swp_entry_to_pmd(arch_entry); } =20 -static inline int is_pmd_migration_entry(pmd_t pmd) -{ - swp_entry_t entry; - - if (pmd_present(pmd)) - return 0; - - entry =3D pmd_to_swp_entry(pmd); - return is_migration_entry(entry); -} #else /* CONFIG_ARCH_ENABLE_THP_MIGRATION */ -static inline int set_pmd_migration_entry(struct page_vma_mapped_walk *pvm= w, - struct page *page) -{ - BUILD_BUG(); -} - static inline void remove_migration_pmd(struct page_vma_mapped_walk *pvmw, struct page *new) { @@ -532,64 +479,17 @@ static inline void remove_migration_pmd(struct page_v= ma_mapped_walk *pvmw, =20 static inline void pmd_migration_entry_wait(struct mm_struct *m, pmd_t *p)= { } =20 -static inline swp_entry_t pmd_to_swp_entry(pmd_t pmd) -{ - return swp_entry(0, 0); -} - static inline pmd_t swp_entry_to_pmd(swp_entry_t entry) { return __pmd(0); } =20 -static inline int is_pmd_migration_entry(pmd_t pmd) -{ - return 0; -} #endif /* CONFIG_ARCH_ENABLE_THP_MIGRATION */ =20 -#if defined(CONFIG_ZONE_DEVICE) && defined(CONFIG_ARCH_ENABLE_THP_MIGRATIO= N) - -/** - * is_pmd_device_private_entry() - Check if PMD contains a device private = swap entry - * @pmd: The PMD to check - * - * Returns true if the PMD contains a swap entry that represents a device = private - * page mapping. This is used for zone device private pages that have been - * swapped out but still need special handling during various memory manag= ement - * operations. - * - * Return: 1 if PMD contains device private entry, 0 otherwise - */ -static inline int is_pmd_device_private_entry(pmd_t pmd) -{ - swp_entry_t entry; - - if (pmd_present(pmd)) - return 0; - - entry =3D pmd_to_swp_entry(pmd); - return is_device_private_entry(entry); -} - -#else /* CONFIG_ZONE_DEVICE && CONFIG_ARCH_ENABLE_THP_MIGRATION */ - -static inline int is_pmd_device_private_entry(pmd_t pmd) -{ - return 0; -} - -#endif /* CONFIG_ZONE_DEVICE && CONFIG_ARCH_ENABLE_THP_MIGRATION */ - static inline int non_swap_entry(swp_entry_t entry) { return swp_type(entry) >=3D MAX_SWAPFILES; } =20 -static inline int is_pmd_non_present_folio_entry(pmd_t pmd) -{ - return is_pmd_migration_entry(pmd) || is_pmd_device_private_entry(pmd); -} - #endif /* CONFIG_MMU */ #endif /* _LINUX_SWAPOPS_H */ diff --git a/mm/damon/ops-common.c b/mm/damon/ops-common.c index 971df8a16ba4..371fcc4d1b65 100644 --- a/mm/damon/ops-common.c +++ b/mm/damon/ops-common.c @@ -11,7 +11,7 @@ #include #include #include -#include +#include =20 #include "../internal.h" #include "ops-common.h" @@ -51,7 +51,7 @@ void damon_ptep_mkold(pte_t *pte, struct vm_area_struct *= vma, unsigned long addr if (likely(pte_present(pteval))) pfn =3D pte_pfn(pteval); else - pfn =3D swp_offset_pfn(pte_to_swp_entry(pteval)); + pfn =3D leafent_to_pfn(leafent_from_pte(pteval)); =20 folio =3D damon_get_folio(pfn); if (!folio) @@ -83,7 +83,7 @@ void damon_pmdp_mkold(pmd_t *pmd, struct vm_area_struct *= vma, unsigned long addr if (likely(pmd_present(pmdval))) pfn =3D pmd_pfn(pmdval); else - pfn =3D swp_offset_pfn(pmd_to_swp_entry(pmdval)); + pfn =3D leafent_to_pfn(leafent_from_pmd(pmdval)); =20 folio =3D damon_get_folio(pfn); if (!folio) diff --git a/mm/filemap.c b/mm/filemap.c index ff75bd89b68c..eb1f994291d8 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -21,7 +21,7 @@ #include #include #include -#include +#include #include #include #include @@ -1402,7 +1402,7 @@ static inline int folio_wait_bit_common(struct folio = *folio, int bit_nr, * This follows the same logic as folio_wait_bit_common() so see the comme= nts * there. */ -void migration_entry_wait_on_locked(swp_entry_t entry, spinlock_t *ptl) +void migration_entry_wait_on_locked(leaf_entry_t entry, spinlock_t *ptl) __releases(ptl) { struct wait_page_queue wait_page; @@ -1411,7 +1411,7 @@ void migration_entry_wait_on_locked(swp_entry_t entry= , spinlock_t *ptl) unsigned long pflags; bool in_thrashing; wait_queue_head_t *q; - struct folio *folio =3D pfn_swap_entry_folio(entry); + struct folio *folio =3D leafent_to_folio(entry); =20 q =3D folio_waitqueue(folio); if (!folio_test_uptodate(folio) && folio_test_workingset(folio)) { diff --git a/mm/hmm.c b/mm/hmm.c index b11b4ebba945..cbcabc48974f 100644 --- a/mm/hmm.c +++ b/mm/hmm.c @@ -18,7 +18,7 @@ #include #include #include -#include +#include #include #include #include @@ -334,19 +334,19 @@ static int hmm_vma_handle_absent_pmd(struct mm_walk *= walk, unsigned long start, struct hmm_vma_walk *hmm_vma_walk =3D walk->private; struct hmm_range *range =3D hmm_vma_walk->range; unsigned long npages =3D (end - start) >> PAGE_SHIFT; + const leaf_entry_t entry =3D leafent_from_pmd(pmd); unsigned long addr =3D start; - swp_entry_t entry =3D pmd_to_swp_entry(pmd); unsigned int required_fault; =20 - if (is_device_private_entry(entry) && - pfn_swap_entry_folio(entry)->pgmap->owner =3D=3D + if (leafent_is_device_private(entry) && + leafent_to_folio(entry)->pgmap->owner =3D=3D range->dev_private_owner) { unsigned long cpu_flags =3D HMM_PFN_VALID | hmm_pfn_flags_order(PMD_SHIFT - PAGE_SHIFT); - unsigned long pfn =3D swp_offset_pfn(entry); + unsigned long pfn =3D leafent_to_pfn(entry); unsigned long i; =20 - if (is_writable_device_private_entry(entry)) + if (leafent_is_device_private_write(entry)) cpu_flags |=3D HMM_PFN_WRITE; =20 /* @@ -365,7 +365,7 @@ static int hmm_vma_handle_absent_pmd(struct mm_walk *wa= lk, unsigned long start, required_fault =3D hmm_range_need_fault(hmm_vma_walk, hmm_pfns, npages, 0); if (required_fault) { - if (is_device_private_entry(entry)) + if (leafent_is_device_private(entry)) return hmm_vma_fault(addr, end, required_fault, walk); else return -EFAULT; @@ -407,7 +407,7 @@ static int hmm_vma_walk_pmd(pmd_t *pmdp, if (pmd_none(pmd)) return hmm_vma_walk_hole(start, end, -1, walk); =20 - if (thp_migration_supported() && is_pmd_migration_entry(pmd)) { + if (thp_migration_supported() && pmd_is_migration_entry(pmd)) { if (hmm_range_need_fault(hmm_vma_walk, hmm_pfns, npages, 0)) { hmm_vma_walk->last =3D addr; pmd_migration_entry_wait(walk->mm, pmdp); diff --git a/mm/huge_memory.c b/mm/huge_memory.c index 40a8a2c1e080..4f8d4cd106e8 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -1374,7 +1374,7 @@ vm_fault_t do_huge_pmd_device_private(struct vm_fault= *vmf) struct vm_area_struct *vma =3D vmf->vma; vm_fault_t ret =3D 0; spinlock_t *ptl; - swp_entry_t swp_entry; + leaf_entry_t entry; struct page *page; struct folio *folio; =20 @@ -1389,8 +1389,8 @@ vm_fault_t do_huge_pmd_device_private(struct vm_fault= *vmf) return 0; } =20 - swp_entry =3D pmd_to_swp_entry(vmf->orig_pmd); - page =3D pfn_swap_entry_to_page(swp_entry); + entry =3D leafent_from_pmd(vmf->orig_pmd); + page =3D leafent_to_page(entry); folio =3D page_folio(page); vmf->page =3D page; vmf->pte =3D NULL; @@ -1780,13 +1780,13 @@ static void copy_huge_non_present_pmd( struct vm_area_struct *dst_vma, struct vm_area_struct *src_vma, pmd_t pmd, pgtable_t pgtable) { - swp_entry_t entry =3D pmd_to_swp_entry(pmd); + leaf_entry_t entry =3D leafent_from_pmd(pmd); struct folio *src_folio; =20 - VM_WARN_ON(!is_pmd_non_present_folio_entry(pmd)); + VM_WARN_ON_ONCE(!pmd_is_valid_leafent(pmd)); =20 - if (is_writable_migration_entry(entry) || - is_readable_exclusive_migration_entry(entry)) { + if (leafent_is_migration_write(entry) || + leafent_is_migration_read_exclusive(entry)) { entry =3D make_readable_migration_entry(swp_offset(entry)); pmd =3D swp_entry_to_pmd(entry); if (pmd_swp_soft_dirty(*src_pmd)) @@ -1794,12 +1794,12 @@ static void copy_huge_non_present_pmd( if (pmd_swp_uffd_wp(*src_pmd)) pmd =3D pmd_swp_mkuffd_wp(pmd); set_pmd_at(src_mm, addr, src_pmd, pmd); - } else if (is_device_private_entry(entry)) { + } else if (leafent_is_device_private(entry)) { /* * For device private entries, since there are no * read exclusive entries, writable =3D !readable */ - if (is_writable_device_private_entry(entry)) { + if (leafent_is_device_private_write(entry)) { entry =3D make_readable_device_private_entry(swp_offset(entry)); pmd =3D swp_entry_to_pmd(entry); =20 @@ -1810,7 +1810,7 @@ static void copy_huge_non_present_pmd( set_pmd_at(src_mm, addr, src_pmd, pmd); } =20 - src_folio =3D pfn_swap_entry_folio(entry); + src_folio =3D leafent_to_folio(entry); VM_WARN_ON(!folio_test_large(src_folio)); =20 folio_get(src_folio); @@ -2270,7 +2270,7 @@ bool madvise_free_huge_pmd(struct mmu_gather *tlb, st= ruct vm_area_struct *vma, =20 if (unlikely(!pmd_present(orig_pmd))) { VM_BUG_ON(thp_migration_supported() && - !is_pmd_migration_entry(orig_pmd)); + !pmd_is_migration_entry(orig_pmd)); goto out; } =20 @@ -2368,11 +2368,10 @@ int zap_huge_pmd(struct mmu_gather *tlb, struct vm_= area_struct *vma, folio_remove_rmap_pmd(folio, page, vma); WARN_ON_ONCE(folio_mapcount(folio) < 0); VM_BUG_ON_PAGE(!PageHead(page), page); - } else if (is_pmd_non_present_folio_entry(orig_pmd)) { - swp_entry_t entry; + } else if (pmd_is_valid_leafent(orig_pmd)) { + const leaf_entry_t entry =3D leafent_from_pmd(orig_pmd); =20 - entry =3D pmd_to_swp_entry(orig_pmd); - folio =3D pfn_swap_entry_folio(entry); + folio =3D leafent_to_folio(entry); flush_needed =3D 0; =20 if (!thp_migration_supported()) @@ -2428,7 +2427,7 @@ static inline int pmd_move_must_withdraw(spinlock_t *= new_pmd_ptl, static pmd_t move_soft_dirty_pmd(pmd_t pmd) { #ifdef CONFIG_MEM_SOFT_DIRTY - if (unlikely(is_pmd_migration_entry(pmd))) + if (unlikely(pmd_is_migration_entry(pmd))) pmd =3D pmd_swp_mksoft_dirty(pmd); else if (pmd_present(pmd)) pmd =3D pmd_mksoft_dirty(pmd); @@ -2503,12 +2502,12 @@ static void change_non_present_huge_pmd(struct mm_s= truct *mm, unsigned long addr, pmd_t *pmd, bool uffd_wp, bool uffd_wp_resolve) { - swp_entry_t entry =3D pmd_to_swp_entry(*pmd); - struct folio *folio =3D pfn_swap_entry_folio(entry); + leaf_entry_t entry =3D leafent_from_pmd(*pmd); + const struct folio *folio =3D leafent_to_folio(entry); pmd_t newpmd; =20 - VM_WARN_ON(!is_pmd_non_present_folio_entry(*pmd)); - if (is_writable_migration_entry(entry)) { + VM_WARN_ON(!pmd_is_valid_leafent(*pmd)); + if (leafent_is_migration_write(entry)) { /* * A protection check is difficult so * just be safe and disable write @@ -2520,7 +2519,7 @@ static void change_non_present_huge_pmd(struct mm_str= uct *mm, newpmd =3D swp_entry_to_pmd(entry); if (pmd_swp_soft_dirty(*pmd)) newpmd =3D pmd_swp_mksoft_dirty(newpmd); - } else if (is_writable_device_private_entry(entry)) { + } else if (leafent_is_device_private_write(entry)) { entry =3D make_readable_device_private_entry(swp_offset(entry)); newpmd =3D swp_entry_to_pmd(entry); } else { @@ -2718,7 +2717,7 @@ int move_pages_huge_pmd(struct mm_struct *mm, pmd_t *= dst_pmd, pmd_t *src_pmd, pm =20 if (!pmd_trans_huge(src_pmdval)) { spin_unlock(src_ptl); - if (is_pmd_migration_entry(src_pmdval)) { + if (pmd_is_migration_entry(src_pmdval)) { pmd_migration_entry_wait(mm, &src_pmdval); return -EAGAIN; } @@ -2983,13 +2982,12 @@ static void __split_huge_pmd_locked(struct vm_area_= struct *vma, pmd_t *pmd, unsigned long addr; pte_t *pte; int i; - swp_entry_t entry; =20 VM_BUG_ON(haddr & ~HPAGE_PMD_MASK); VM_BUG_ON_VMA(vma->vm_start > haddr, vma); VM_BUG_ON_VMA(vma->vm_end < haddr + HPAGE_PMD_SIZE, vma); =20 - VM_WARN_ON(!is_pmd_non_present_folio_entry(*pmd) && !pmd_trans_huge(*pmd)= ); + VM_WARN_ON_ONCE(!pmd_is_valid_leafent(*pmd) && !pmd_trans_huge(*pmd)); =20 count_vm_event(THP_SPLIT_PMD); =20 @@ -3003,11 +3001,10 @@ static void __split_huge_pmd_locked(struct vm_area_= struct *vma, pmd_t *pmd, zap_deposited_table(mm, pmd); if (!vma_is_dax(vma) && vma_is_special_huge(vma)) return; - if (unlikely(is_pmd_migration_entry(old_pmd))) { - swp_entry_t entry; + if (unlikely(pmd_is_migration_entry(old_pmd))) { + const leaf_entry_t old_entry =3D leafent_from_pmd(old_pmd); =20 - entry =3D pmd_to_swp_entry(old_pmd); - folio =3D pfn_swap_entry_folio(entry); + folio =3D leafent_to_folio(old_entry); } else if (is_huge_zero_pmd(old_pmd)) { return; } else { @@ -3037,31 +3034,34 @@ static void __split_huge_pmd_locked(struct vm_area_= struct *vma, pmd_t *pmd, return __split_huge_zero_page_pmd(vma, haddr, pmd); } =20 + if (pmd_is_migration_entry(*pmd)) { + leaf_entry_t entry; =20 - if (is_pmd_migration_entry(*pmd)) { old_pmd =3D *pmd; - entry =3D pmd_to_swp_entry(old_pmd); - page =3D pfn_swap_entry_to_page(entry); + entry =3D leafent_from_pmd(old_pmd); + page =3D leafent_to_page(entry); folio =3D page_folio(page); =20 soft_dirty =3D pmd_swp_soft_dirty(old_pmd); uffd_wp =3D pmd_swp_uffd_wp(old_pmd); =20 - write =3D is_writable_migration_entry(entry); + write =3D leafent_is_migration_write(entry); if (PageAnon(page)) - anon_exclusive =3D is_readable_exclusive_migration_entry(entry); - young =3D is_migration_entry_young(entry); - dirty =3D is_migration_entry_dirty(entry); - } else if (is_pmd_device_private_entry(*pmd)) { + anon_exclusive =3D leafent_is_migration_read_exclusive(entry); + young =3D leafent_is_migration_young(entry); + dirty =3D leafent_is_migration_dirty(entry); + } else if (pmd_is_device_private_entry(*pmd)) { + leaf_entry_t entry; + old_pmd =3D *pmd; - entry =3D pmd_to_swp_entry(old_pmd); - page =3D pfn_swap_entry_to_page(entry); + entry =3D leafent_from_pmd(old_pmd); + page =3D leafent_to_page(entry); folio =3D page_folio(page); =20 soft_dirty =3D pmd_swp_soft_dirty(old_pmd); uffd_wp =3D pmd_swp_uffd_wp(old_pmd); =20 - write =3D is_writable_device_private_entry(entry); + write =3D leafent_is_device_private_write(entry); anon_exclusive =3D PageAnonExclusive(page); =20 /* @@ -3165,7 +3165,7 @@ static void __split_huge_pmd_locked(struct vm_area_st= ruct *vma, pmd_t *pmd, * Note that NUMA hinting access restrictions are not transferred to * avoid any possibility of altering permissions across VMAs. */ - if (freeze || is_pmd_migration_entry(old_pmd)) { + if (freeze || pmd_is_migration_entry(old_pmd)) { pte_t entry; swp_entry_t swp_entry; =20 @@ -3191,7 +3191,7 @@ static void __split_huge_pmd_locked(struct vm_area_st= ruct *vma, pmd_t *pmd, VM_WARN_ON(!pte_none(ptep_get(pte + i))); set_pte_at(mm, addr, pte + i, entry); } - } else if (is_pmd_device_private_entry(old_pmd)) { + } else if (pmd_is_device_private_entry(old_pmd)) { pte_t entry; swp_entry_t swp_entry; =20 @@ -3241,7 +3241,7 @@ static void __split_huge_pmd_locked(struct vm_area_st= ruct *vma, pmd_t *pmd, } pte_unmap(pte); =20 - if (!is_pmd_migration_entry(*pmd)) + if (!pmd_is_migration_entry(*pmd)) folio_remove_rmap_pmd(folio, page, vma); if (freeze) put_page(page); @@ -3254,7 +3254,7 @@ void split_huge_pmd_locked(struct vm_area_struct *vma= , unsigned long address, pmd_t *pmd, bool freeze) { VM_WARN_ON_ONCE(!IS_ALIGNED(address, HPAGE_PMD_SIZE)); - if (pmd_trans_huge(*pmd) || is_pmd_non_present_folio_entry(*pmd)) + if (pmd_trans_huge(*pmd) || pmd_is_valid_leafent(*pmd)) __split_huge_pmd_locked(vma, pmd, address, freeze); } =20 @@ -4855,12 +4855,12 @@ void remove_migration_pmd(struct page_vma_mapped_wa= lk *pvmw, struct page *new) unsigned long address =3D pvmw->address; unsigned long haddr =3D address & HPAGE_PMD_MASK; pmd_t pmde; - swp_entry_t entry; + leaf_entry_t entry; =20 if (!(pvmw->pmd && !pvmw->pte)) return; =20 - entry =3D pmd_to_swp_entry(*pvmw->pmd); + entry =3D leafent_from_pmd(*pvmw->pmd); folio_get(folio); pmde =3D folio_mk_pmd(folio, READ_ONCE(vma->vm_page_prot)); =20 @@ -4876,20 +4876,20 @@ void remove_migration_pmd(struct page_vma_mapped_wa= lk *pvmw, struct page *new) =20 if (pmd_swp_soft_dirty(*pvmw->pmd)) pmde =3D pmd_mksoft_dirty(pmde); - if (is_writable_migration_entry(entry)) + if (leafent_is_migration_write(entry)) pmde =3D pmd_mkwrite(pmde, vma); if (pmd_swp_uffd_wp(*pvmw->pmd)) pmde =3D pmd_mkuffd_wp(pmde); - if (!is_migration_entry_young(entry)) + if (!leafent_is_migration_young(entry)) pmde =3D pmd_mkold(pmde); /* NOTE: this may contain setting soft-dirty on some archs */ - if (folio_test_dirty(folio) && is_migration_entry_dirty(entry)) + if (folio_test_dirty(folio) && leafent_is_migration_dirty(entry)) pmde =3D pmd_mkdirty(pmde); =20 if (folio_test_anon(folio)) { rmap_t rmap_flags =3D RMAP_NONE; =20 - if (!is_readable_migration_entry(entry)) + if (!leafent_is_migration_read(entry)) rmap_flags |=3D RMAP_EXCLUSIVE; =20 folio_add_anon_rmap_pmd(folio, new, vma, haddr, rmap_flags); diff --git a/mm/khugepaged.c b/mm/khugepaged.c index a97ff7bcb232..1a08673b0d8b 100644 --- a/mm/khugepaged.c +++ b/mm/khugepaged.c @@ -17,7 +17,7 @@ #include #include #include -#include +#include #include #include #include @@ -941,7 +941,7 @@ static inline int check_pmd_state(pmd_t *pmd) * collapse it. Migration success or failure will eventually end * up with a present PMD mapping a folio again. */ - if (is_pmd_migration_entry(pmde)) + if (pmd_is_migration_entry(pmde)) return SCAN_PMD_MAPPED; if (!pmd_present(pmde)) return SCAN_PMD_NULL; diff --git a/mm/madvise.c b/mm/madvise.c index 398721e9a1e5..900f0f29e77b 100644 --- a/mm/madvise.c +++ b/mm/madvise.c @@ -390,7 +390,7 @@ static int madvise_cold_or_pageout_pte_range(pmd_t *pmd, =20 if (unlikely(!pmd_present(orig_pmd))) { VM_BUG_ON(thp_migration_supported() && - !is_pmd_migration_entry(orig_pmd)); + !pmd_is_migration_entry(orig_pmd)); goto huge_unlock; } =20 diff --git a/mm/memory.c b/mm/memory.c index a0ae4e23d487..ce2e3ce23f3b 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -6362,10 +6362,10 @@ static vm_fault_t __handle_mm_fault(struct vm_area_= struct *vma, goto fallback; =20 if (unlikely(!pmd_present(vmf.orig_pmd))) { - if (is_pmd_device_private_entry(vmf.orig_pmd)) + if (pmd_is_device_private_entry(vmf.orig_pmd)) return do_huge_pmd_device_private(&vmf); =20 - if (is_pmd_migration_entry(vmf.orig_pmd)) + if (pmd_is_migration_entry(vmf.orig_pmd)) pmd_migration_entry_wait(mm, vmf.pmd); return 0; } diff --git a/mm/mempolicy.c b/mm/mempolicy.c index 7ae3f5e2dee6..01c3b98f87a6 100644 --- a/mm/mempolicy.c +++ b/mm/mempolicy.c @@ -110,7 +110,7 @@ #include #include #include -#include +#include #include =20 #include @@ -647,7 +647,7 @@ static void queue_folios_pmd(pmd_t *pmd, struct mm_walk= *walk) struct folio *folio; struct queue_pages *qp =3D walk->private; =20 - if (unlikely(is_pmd_migration_entry(*pmd))) { + if (unlikely(pmd_is_migration_entry(*pmd))) { qp->nr_failed++; return; } diff --git a/mm/migrate.c b/mm/migrate.c index 862b2e261cf9..bb0429a5e5a4 100644 --- a/mm/migrate.c +++ b/mm/migrate.c @@ -16,7 +16,7 @@ #include #include #include -#include +#include #include #include #include @@ -353,7 +353,7 @@ static bool remove_migration_pte(struct folio *folio, rmap_t rmap_flags =3D RMAP_NONE; pte_t old_pte; pte_t pte; - swp_entry_t entry; + leaf_entry_t entry; struct page *new; unsigned long idx =3D 0; =20 @@ -379,22 +379,22 @@ static bool remove_migration_pte(struct folio *folio, folio_get(folio); pte =3D mk_pte(new, READ_ONCE(vma->vm_page_prot)); =20 - entry =3D pte_to_swp_entry(old_pte); - if (!is_migration_entry_young(entry)) + entry =3D leafent_from_pte(old_pte); + if (!leafent_is_migration_young(entry)) pte =3D pte_mkold(pte); - if (folio_test_dirty(folio) && is_migration_entry_dirty(entry)) + if (folio_test_dirty(folio) && leafent_is_migration_dirty(entry)) pte =3D pte_mkdirty(pte); if (pte_swp_soft_dirty(old_pte)) pte =3D pte_mksoft_dirty(pte); else pte =3D pte_clear_soft_dirty(pte); =20 - if (is_writable_migration_entry(entry)) + if (leafent_is_migration_write(entry)) pte =3D pte_mkwrite(pte, vma); else if (pte_swp_uffd_wp(old_pte)) pte =3D pte_mkuffd_wp(pte); =20 - if (folio_test_anon(folio) && !is_readable_migration_entry(entry)) + if (folio_test_anon(folio) && !leafent_is_migration_read(entry)) rmap_flags |=3D RMAP_EXCLUSIVE; =20 if (unlikely(is_device_private_page(new))) { @@ -404,7 +404,7 @@ static bool remove_migration_pte(struct folio *folio, else entry =3D make_readable_device_private_entry( page_to_pfn(new)); - pte =3D swp_entry_to_pte(entry); + pte =3D leafent_to_pte(entry); if (pte_swp_soft_dirty(old_pte)) pte =3D pte_swp_mksoft_dirty(pte); if (pte_swp_uffd_wp(old_pte)) @@ -543,9 +543,9 @@ void pmd_migration_entry_wait(struct mm_struct *mm, pmd= _t *pmd) spinlock_t *ptl; =20 ptl =3D pmd_lock(mm, pmd); - if (!is_pmd_migration_entry(*pmd)) + if (!pmd_is_migration_entry(*pmd)) goto unlock; - migration_entry_wait_on_locked(pmd_to_swp_entry(*pmd), ptl); + migration_entry_wait_on_locked(leafent_from_pmd(*pmd), ptl); return; unlock: spin_unlock(ptl); diff --git a/mm/migrate_device.c b/mm/migrate_device.c index c869b272e85a..5cb5ac2f0290 100644 --- a/mm/migrate_device.c +++ b/mm/migrate_device.c @@ -13,7 +13,7 @@ #include #include #include -#include +#include #include #include #include "internal.h" @@ -145,7 +145,6 @@ static int migrate_vma_collect_huge_pmd(pmd_t *pmdp, un= signed long start, struct folio *folio; struct migrate_vma *migrate =3D walk->private; spinlock_t *ptl; - swp_entry_t entry; int ret; unsigned long write =3D 0; =20 @@ -169,23 +168,24 @@ static int migrate_vma_collect_huge_pmd(pmd_t *pmdp, = unsigned long start, if (pmd_write(*pmdp)) write =3D MIGRATE_PFN_WRITE; } else if (!pmd_present(*pmdp)) { - entry =3D pmd_to_swp_entry(*pmdp); - folio =3D pfn_swap_entry_folio(entry); + const leaf_entry_t entry =3D leafent_from_pmd(*pmdp); =20 - if (!is_device_private_entry(entry) || + folio =3D leafent_to_folio(entry); + + if (!leafent_is_device_private(entry) || !(migrate->flags & MIGRATE_VMA_SELECT_DEVICE_PRIVATE) || (folio->pgmap->owner !=3D migrate->pgmap_owner)) { spin_unlock(ptl); return migrate_vma_collect_skip(start, end, walk); } =20 - if (is_migration_entry(entry)) { + if (leafent_is_migration(entry)) { migration_entry_wait_on_locked(entry, ptl); spin_unlock(ptl); return -EAGAIN; } =20 - if (is_writable_device_private_entry(entry)) + if (leafent_is_device_private_write(entry)) write =3D MIGRATE_PFN_WRITE; } else { spin_unlock(ptl); diff --git a/mm/page_table_check.c b/mm/page_table_check.c index f5f25e120f69..7a26ed53d875 100644 --- a/mm/page_table_check.c +++ b/mm/page_table_check.c @@ -8,7 +8,7 @@ #include #include #include -#include +#include =20 #undef pr_fmt #define pr_fmt(fmt) "page_table_check: " fmt @@ -179,10 +179,10 @@ void __page_table_check_pud_clear(struct mm_struct *m= m, pud_t pud) EXPORT_SYMBOL(__page_table_check_pud_clear); =20 /* Whether the swap entry cached writable information */ -static inline bool swap_cached_writable(swp_entry_t entry) +static inline bool leafent_cached_writable(leaf_entry_t entry) { - return is_writable_device_private_entry(entry) || - is_writable_migration_entry(entry); + return leafent_is_device_private(entry) || + leafent_is_migration_write(entry); } =20 static void page_table_check_pte_flags(pte_t pte) @@ -190,9 +190,9 @@ static void page_table_check_pte_flags(pte_t pte) if (pte_present(pte)) { WARN_ON_ONCE(pte_uffd_wp(pte) && pte_write(pte)); } else if (pte_swp_uffd_wp(pte)) { - const swp_entry_t entry =3D pte_to_swp_entry(pte); + const leaf_entry_t entry =3D leafent_from_pte(pte); =20 - WARN_ON_ONCE(swap_cached_writable(entry)); + WARN_ON_ONCE(leafent_cached_writable(entry)); } } =20 @@ -219,9 +219,9 @@ static inline void page_table_check_pmd_flags(pmd_t pmd) if (pmd_uffd_wp(pmd)) WARN_ON_ONCE(pmd_write(pmd)); } else if (pmd_swp_uffd_wp(pmd)) { - swp_entry_t entry =3D pmd_to_swp_entry(pmd); + const leaf_entry_t entry =3D leafent_from_pmd(pmd); =20 - WARN_ON_ONCE(swap_cached_writable(entry)); + WARN_ON_ONCE(leafent_cached_writable(entry)); } } =20 diff --git a/mm/page_vma_mapped.c b/mm/page_vma_mapped.c index 4597a281356d..b69b817ad180 100644 --- a/mm/page_vma_mapped.c +++ b/mm/page_vma_mapped.c @@ -242,18 +242,19 @@ bool page_vma_mapped_walk(struct page_vma_mapped_walk= *pvmw) */ pmde =3D pmdp_get_lockless(pvmw->pmd); =20 - if (pmd_trans_huge(pmde) || is_pmd_migration_entry(pmde)) { + if (pmd_trans_huge(pmde) || pmd_is_migration_entry(pmde)) { pvmw->ptl =3D pmd_lock(mm, pvmw->pmd); pmde =3D *pvmw->pmd; if (!pmd_present(pmde)) { - swp_entry_t entry; + leaf_entry_t entry; =20 if (!thp_migration_supported() || !(pvmw->flags & PVMW_MIGRATION)) return not_found(pvmw); - entry =3D pmd_to_swp_entry(pmde); - if (!is_migration_entry(entry) || - !check_pmd(swp_offset_pfn(entry), pvmw)) + entry =3D leafent_from_pmd(pmde); + + if (!leafent_is_migration(entry) || + !check_pmd(leafent_to_pfn(entry), pvmw)) return not_found(pvmw); return true; } @@ -273,9 +274,9 @@ bool page_vma_mapped_walk(struct page_vma_mapped_walk *= pvmw) * cannot return prematurely, while zap_huge_pmd() has * cleared *pmd but not decremented compound_mapcount(). */ - swp_entry_t entry =3D pmd_to_swp_entry(pmde); + const leaf_entry_t entry =3D leafent_from_pmd(pmde); =20 - if (is_device_private_entry(entry)) { + if (leafent_is_device_private(entry)) { pvmw->ptl =3D pmd_lock(mm, pvmw->pmd); return true; } diff --git a/mm/pagewalk.c b/mm/pagewalk.c index 9f91cf85a5be..fc2576235fde 100644 --- a/mm/pagewalk.c +++ b/mm/pagewalk.c @@ -5,7 +5,7 @@ #include #include #include -#include +#include =20 #include =20 @@ -966,10 +966,10 @@ struct folio *folio_walk_start(struct folio_walk *fw, goto found; } } else if ((flags & FW_MIGRATION) && - is_pmd_migration_entry(pmd)) { - swp_entry_t entry =3D pmd_to_swp_entry(pmd); + pmd_is_migration_entry(pmd)) { + const leaf_entry_t entry =3D leafent_from_pmd(pmd); =20 - page =3D pfn_swap_entry_to_page(entry); + page =3D leafent_to_page(entry); expose_page =3D false; goto found; } diff --git a/mm/rmap.c b/mm/rmap.c index 1954c538a991..99203bf7d346 100644 --- a/mm/rmap.c +++ b/mm/rmap.c @@ -57,7 +57,7 @@ #include #include #include -#include +#include #include #include #include @@ -2341,7 +2341,7 @@ static bool try_to_migrate_one(struct folio *folio, s= truct vm_area_struct *vma, if (likely(pmd_present(pmdval))) pfn =3D pmd_pfn(pmdval); else - pfn =3D swp_offset_pfn(pmd_to_swp_entry(pmdval)); + pfn =3D leafent_to_pfn(leafent_from_pmd(pmdval)); =20 subpage =3D folio_page(folio, pfn - folio_pfn(folio)); =20 --=20 2.51.0 From nobody Sat Feb 7 17:55:59 2026 Received: from mx0a-00069f02.pphosted.com (mx0a-00069f02.pphosted.com [205.220.165.32]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 95B073128AD; Mon, 3 Nov 2025 12:34:42 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=205.220.165.32 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1762173284; cv=fail; b=Q2BVzM/7HDK2XKzVYnKJNpqtJJU2cOS6DXvbtK6pm7Yh2LmdrIMfZp/CDaLDhFRkQFsW080eLAV8fLhQNFTj3Cx7udo+pzRXf8iUTRp0po2RhU4z3H1bO0DgwvSmzMZqkxIN9Gfm7trN7xSc41+k1DnMNIh1XU391NTTXpMT5n8= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1762173284; c=relaxed/simple; bh=GKKb0JMd8seT3zTDNqm885vtyRs+40jvFNZM7ohS0s4=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: Content-Type:MIME-Version; b=UrRfw+5CwJWJhmtxYyYl+G/bScXCP0oFUEOJp1kZenc6AL5dz6+04htkCshzTSZ3VnhDwi5TNP4pqsG34I7meNMqrosfFSVCIYAPWAdh8scYPDCZ74AVyX/XDwuMqy169ug3gUernihwvP4MkQCDTgRjYFXN4Rmbh5359fVRql0= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=oracle.com; spf=pass smtp.mailfrom=oracle.com; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b=Cwvm+ezk; dkim=pass (1024-bit key) header.d=oracle.onmicrosoft.com header.i=@oracle.onmicrosoft.com header.b=W6foDy6W; arc=fail smtp.client-ip=205.220.165.32 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=oracle.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=oracle.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b="Cwvm+ezk"; dkim=pass (1024-bit key) header.d=oracle.onmicrosoft.com header.i=@oracle.onmicrosoft.com header.b="W6foDy6W" Received: from pps.filterd (m0246629.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 5A3CU15N012004; Mon, 3 Nov 2025 12:32:47 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=cc :content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s= corp-2025-04-25; bh=1f1Webwe6P9DpsQg9iPMHyn4nFICr08ZhIuXFw4tvLM=; b= Cwvm+ezkdI65DsWjRzzzFLev5K2ip6uemH1Me5DMoFkqprCPcEYS1fIJLOwCR5dc ZNoll7VjC4CivILqfBE/mUsTKzXNHPqIHu610dlu/Np+ispJ3X2mkzgXyx2XTmzU XxXFoOVF2iykQYADQdsVA4sLHnTQ1sgNz54emHTj7FZX6x7lMdLXTCM/t87Vegpr eSmqRiUU05eaMZf4x695kpFI/OJOvWSnZ7ZkKqU90fQlIDNyhGOiYwfGx6BNw/0n aKdsms4xqAYHJSNp5r//8nr4g9iuWpezENPX6fuBEYZZHP+BMZnyfmsUhEOgZGn2 z5Vog2W0f3kM2+tGso6q3w== Received: from phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta02.appoci.oracle.com [147.154.114.232]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 4a6vep007r-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 03 Nov 2025 12:32:46 +0000 (GMT) Received: from pps.filterd (phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (8.18.1.2/8.18.1.2) with ESMTP id 5A3BY5jM020996; Mon, 3 Nov 2025 12:32:45 GMT Received: from sj2pr03cu001.outbound.protection.outlook.com (mail-westusazon11012064.outbound.protection.outlook.com [52.101.43.64]) by phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTPS id 4a58n83esg-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 03 Nov 2025 12:32:45 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=CUAvwquRMGwGw4GlRoh2PGDY62oYO6RgSxBBRjLEUMlmPioysI7WwW5LO4mBHKjFA8f643eumr4voNioNMW57ips1dZ3FU2BlFvXdIfraqN1tyYR8SgwraWpeb23hqteTaN1PVpkh2YEarin1eY1Dsc3i6QYrNMgZURzsW82yA76Ha9ZaqULa2KEclPCydqmAMXQv37EoYSIAk6TnbFPRWdRiPtrN4N2PQRagkVM59MNJRya8uk8NFFDSYWojfElwnpvOYLWaK1gO2eCIAqFi1Da4+8myOc+ZRgzzO/O6GgpWnKGTRGHumheW20Ko6Ju6Q6WP1iJ72I+8aLGDbknMw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=1f1Webwe6P9DpsQg9iPMHyn4nFICr08ZhIuXFw4tvLM=; b=cgskbfnm0zEZnYp1zZ6ZQAERdfWn5kvoqZj06LIebvT3/aR3U2wFKF+UI4a5u8DKEbiaJBLVPbidnR/B5YX9+fcOZhAnZ5oDPjGFIOGxhlKHb1TYvMG4G7jKciDx/wfLYcUN5X8O+QkMiiVTXFn5LGSCgBZTVIJnYgdBTcPo/HrOMjPXJ+LPGcnBIJ/2G+SDataY3q799FPPGcuV5Zb6IMT5k1ggr0LBBwBuIZDuX1KkfGg3sPbXsgms1ac0kyv6YnGZF4cO6YyYuPPfaJz3BKGFvwf5d6EIMLgRJ6J90Uo7ggZq13rcmIofA3kvcIZHiYvj0QdgS6p24BPb9tcZzA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=1f1Webwe6P9DpsQg9iPMHyn4nFICr08ZhIuXFw4tvLM=; b=W6foDy6WqBhAGoioOllr+mwxXL68I9qhyoWNu0U99pmieam1OeeM1iifg90s37IXOjO8gnLbtWscPVu5khXLKnps0FXJU546jXACS0ggzDEpN4QCNzbX92I0Y5H9WRTm97BWY4H5gMg8hgOKS+SRZr/AyOh8Z3/54aLHVofoLMg= Received: from DM4PR10MB8218.namprd10.prod.outlook.com (2603:10b6:8:1cc::16) by DS4PR10MB997575.namprd10.prod.outlook.com (2603:10b6:8:31e::5) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9275.16; Mon, 3 Nov 2025 12:32:41 +0000 Received: from DM4PR10MB8218.namprd10.prod.outlook.com ([fe80::2650:55cf:2816:5f2]) by DM4PR10MB8218.namprd10.prod.outlook.com ([fe80::2650:55cf:2816:5f2%2]) with mapi id 15.20.9253.018; Mon, 3 Nov 2025 12:32:41 +0000 From: Lorenzo Stoakes To: Andrew Morton Cc: Christian Borntraeger , Janosch Frank , Claudio Imbrenda , David Hildenbrand , Alexander Gordeev , Gerald Schaefer , Heiko Carstens , Vasily Gorbik , Sven Schnelle , Peter Xu , Alexander Viro , Christian Brauner , Jan Kara , Arnd Bergmann , Zi Yan , Baolin Wang , "Liam R . Howlett" , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Lance Yang , Muchun Song , Oscar Salvador , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Matthew Brost , Joshua Hahn , Rakie Kim , Byungchul Park , Gregory Price , Ying Huang , Alistair Popple , Axel Rasmussen , Yuanchu Xie , Wei Xu , Kemeng Shi , Kairui Song , Nhat Pham , Baoquan He , Chris Li , SeongJae Park , Matthew Wilcox , Jason Gunthorpe , Leon Romanovsky , Xu Xin , Chengming Zhou , Jann Horn , Miaohe Lin , Naoya Horiguchi , Pedro Falcato , Pasha Tatashin , Rik van Riel , Harry Yoo , Hugh Dickins , linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-s390@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, linux-arch@vger.kernel.org, damon@lists.linux.dev Subject: [PATCH 11/16] mm: introduce pmd_is_huge() and use where appropriate Date: Mon, 3 Nov 2025 12:31:52 +0000 Message-ID: X-Mailer: git-send-email 2.51.0 In-Reply-To: References: Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: LO4P123CA0114.GBRP123.PROD.OUTLOOK.COM (2603:10a6:600:192::11) To DM4PR10MB8218.namprd10.prod.outlook.com (2603:10b6:8:1cc::16) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DM4PR10MB8218:EE_|DS4PR10MB997575:EE_ X-MS-Office365-Filtering-Correlation-Id: 3bf3d899-7d0b-4fe8-3ee5-08de1ad5148e X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|376014|7416014|366016|1800799024; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?HH/kFa0LhEN11Smg3hLGdGzoTKs/knSMR+CzvxDFha23CMdJBRtdFFSOozMP?= =?us-ascii?Q?8Pb48mBO0++kbZoOmXnxF9+GYdcyw1uIPDcnF4myfJTBuZ/XyEKRpBkRDpMK?= =?us-ascii?Q?yZsLR5whnGHCA3Ce3zBasuvWq+d4erejUoOv28xs4Y+ooFbVyO/gJb++SGZt?= =?us-ascii?Q?uDfcYepsrVpLhbvdoWQZS8+veNbP8YFMjH6y5GKDKsONFECl0VrNK/lJ0dEO?= =?us-ascii?Q?mxtor4jTKRuhLnIPShyZm4FMl/uwDvULlewgjM7145oIHwohrM7VlCq/H+GK?= =?us-ascii?Q?+sCA3jLDocghxQD3+Z+F5ofBsRirC+Jw6eRoZuvf/1n7umiP9JwYalohNNXf?= =?us-ascii?Q?xmWrIfTt5V3niTcFjwiD26mRXyARyvbZb98RAqoFrer7+1vBl4ERFxIFPyXo?= =?us-ascii?Q?aacsNWQxOdqWPmWGFSrWd4sSnFeWfJgQnINVbw7Eg7t8ntAE8eFGXrD8Ng9j?= =?us-ascii?Q?7WxrV6hnHwCO9vamdo0q3GCscRhjpF12KdrSvKrqJdvVxCuwg0NcNmvj39me?= =?us-ascii?Q?vd11Bvul6/Q5dzrD8mY/AxLyTzQX/BvV9lRnC4zTTYY2CTszr+OoYHcbP00D?= =?us-ascii?Q?Up6kboJ13eE8WHDeDT7XIpUTI2EReHQqXuYaVbaopXm/XE4pNmF6PTWLICKD?= =?us-ascii?Q?qCIg6iPaA88YqEfJikXj8S8xI9TUFxbo4vv83KHxQjIRmDSjHCR/e8NnD6mW?= =?us-ascii?Q?Yz7gRJoT4ot6eg2T+0iIDcp2qrQEtIfKikIOGRbv4OPIcPUGlvQxRHoKAzqW?= =?us-ascii?Q?N446q2uuAJRwtnpPSxnqKJUgX5jTJ8GBcLa9xWsEYDCmmjYb1/yFXKTuMMHu?= =?us-ascii?Q?A3tHcXYY6Fbhfm7LKHJS340tmd4PZIKJHMCNuZ0g4PH3QGImAL53BF6rHLAT?= =?us-ascii?Q?ccfqUH/QqEbWT494QEKIUPYUgJ0nBXl7zFCWFUZf0rGWemAX9LpUGB0zJHSM?= =?us-ascii?Q?ajfdntemDuf6QJ2TSTo/LLenpMeTzkx1r/BOibCaPu2ZRN7Fp7wELqlKwdTn?= =?us-ascii?Q?utRgH4+PUBYlSWuzYJDLyVhP48i4t1k+baFq1houhNJbvn9TqfarIbYfQWTy?= =?us-ascii?Q?KSruuHT+BhCL0mzS8HuyJvfoxXCxPQdq3JRa80ZeTgD1LN7lT6DZxq5BcZUz?= =?us-ascii?Q?/5JOCQOtHVdKEKXUYsGjmUKiGbi4PO3BRjQrKS0wvLuL2QqaxD3b8wu8iiAb?= =?us-ascii?Q?fjV1SusRLwRTkLEG7tqv0FSclcvtO6JqRdTWesu6XAtFLPwPTQffuWlYZLyO?= =?us-ascii?Q?Ou/wvY6GdJJv+S4nwt9vsPitijS3SAgXeQXmgVQ2YGMfPxZB366E+aM0ci1C?= =?us-ascii?Q?acmrGkigJ0Oh/LlPAWAR9qGiDne0UaY1IIBJKK4S7RdJJcyfyozgyBhd1L1d?= =?us-ascii?Q?HRmjzljg+BN4Bmdy+DFXmEkoPiZkEniFnUfwJIbhT+R2n2WxipGwTkSvvRc6?= =?us-ascii?Q?ABSm1WRHNzcjs4lqFgCYbNktVmIZyEou?= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DM4PR10MB8218.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(376014)(7416014)(366016)(1800799024);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?B2MvbjbbG++9zBbqVmSPlGxqprhRsJCiOyAHlEcSETFBCx0TEgMK0Y0srCoq?= =?us-ascii?Q?/aPfMaiAvTyunSuHBn8A2t5tZq27Oic4F654tZgKxUlWBofWdv1JMzHK4PuI?= =?us-ascii?Q?hHB47/8ytg2aHZk/w5cSiVnL6NRbehZtS+bSYIYqV/kTnS8PBZZvRyAS8dLD?= =?us-ascii?Q?8faN7u02aLnBwd6XHR679lKewZnPWf5gfc3dCI15WBxlanbMr4+Q4kNIvkWq?= =?us-ascii?Q?UVwqEYYzWDZTeGEny4smJMG6wXXmr0Z0bwtvz1OhG/SYo68m1ieysi6t3Cm6?= =?us-ascii?Q?F3nNH7rVOyylvYCrurUWynWfxPxwXkPr1cQ5FBxQWLWW0B49wIhjhAWWObw6?= =?us-ascii?Q?B72y6RCTKianVtm/OasMO3YveNhcQPB7jdxYfdAzN286Vn7nx20aJP+tbvXs?= =?us-ascii?Q?rtgSSiaHK/GtvWy55JsQ+qvWIdtQ3IcB7CsJ1bKbXd3N2JHhlWB3UByS+d2r?= =?us-ascii?Q?8AekPIevPus0011adnNrF/SI2Tcw2tQklHiZsh70+5t1KofUgwLb1kox4QUq?= =?us-ascii?Q?mT4MN32PJs5yKjnjiNvRxr3oIjYvjoqFJZi0kAr4ni3y3a37tN0EYCDn5due?= =?us-ascii?Q?3+ngUTo6LcUS/JLRagSbQwZ6zIdVP8t2sE7AWSZbv//joBm1rgXWUbnTS7IE?= =?us-ascii?Q?kVUrUGaNsedamycmivcNaMtyewNbCBPPlq9YaILIVJ0T5tsfluvzdsvFxWpY?= =?us-ascii?Q?Gxh6qwt19fsKaqDCay6hnyZN6fI+TrlU52AaTtVLFztHQiKywG02weG3LMeE?= =?us-ascii?Q?o01uFb6hlVQsQLkY+bT4KhgyGfPfEI9evkuVD6UC2dNPAe81lRM/UQCeXnVc?= =?us-ascii?Q?64D9oyTfZghBpYeO+4J8b52SsXrmPJrqKbF23T/FiUqAqBi0QurLW8zjHxaP?= =?us-ascii?Q?uNe/np5ZcPkCdNBSx+6fVm12Dyvhh7e9mP5Hu6udpd6VrjGMAaxinE0EHVaf?= =?us-ascii?Q?YdHXKTCzn8Y290MAygJVJ6m0TeF8unTDg1xVTYBa8evhqxBVZwWFPXB4T93y?= =?us-ascii?Q?FZAfEfYmCGcvcoUzS4pnm8EalszW2s/8WVKjT7Iwpl8liZ6g0r07vv/vFpie?= =?us-ascii?Q?6sWxUVX+gIdFu3B4clG7Q1KpgQrg3Oio+EKw/3Pgcnron7LHeTF9blB4w6TE?= =?us-ascii?Q?SjxvyyO+PpOUpkwO+IbGJ482J8FDx+mjirx2jldAEcv6ITMDih2hovJmNRAU?= =?us-ascii?Q?FErHUzDDYMAhmWeDHyywu+PhQIp7CGpK4qtIPiKHMxUUG+TRTh65x0LMliUP?= =?us-ascii?Q?2YkOCRRWH8H0ftd+uZEfqBhk1DmwM0ZHKqNHJt8hPrDyJxlekbf7ublH7Rf0?= =?us-ascii?Q?scGdDR9doCrf4pKP0pdKpVw1DbpjvutzC0AsnYKZd67Y5ZRAofpSY5pC/0ZP?= =?us-ascii?Q?EDqopr1mSyc77sFJlxXpNFieLIHaMBUUlW7uV7Cne/sDYg901BIVMBPGFSGm?= =?us-ascii?Q?xXS9GqIRF3KylWJDEXIEnP9yQ5NUfsi+zw84fmJX5HiNizc7yYzmDE+XR4XV?= =?us-ascii?Q?BZ2XYK7j+ErP+qgiyufUWNSpRJWxnG1OE4EDr88NEorCxYb8oe8y9KnDJ4Vj?= =?us-ascii?Q?M0saoKnu3qhRxrXVKi1u2xCUt5lnAgIt3SkM85K+t7awIPRkQxiU4NBh8FPi?= =?us-ascii?Q?vw=3D=3D?= X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: N7SMsZ26nojRD1Pl8kCt/WXFw1bWbEK/sWSdykNa/L91wQj2QyuBnm5/Kx2x/jSwQ9K+VNLS1eoxhnfFfEE/MfkMROnh2dnc7VIZr4KI4nE8kBuDhRlcpC6v4OrrtTE2gMoM/cg/r8h6YhLhhg4Xo/4sWxhthMe4TSrlQQkY4r4zee/gP6cvmaE2nm7QjxDJUpvyGMYe8ELiY8GJqU9j0U2muC79LqtjcEyx8qZkfguXh4/afyc+/unor/n4JEgc4XFc7zXSLeHZwTPiIPUU1rmu2Ohw6zWotRTcsM7flhHLBKgSQiU3IExXeHskczs1jlStRp9mck12QZA7uRUsEXuktE+09RtWjVPVqS6o4Kf5y5FGktRMcEj8y+r8Lto9s+rmtJoFFH0QGYlWNFTM9cSE2U1zaHdWtmITgJ042TwH6rrS1WNxvKN7diLRkZUlu3PL/LBKfuuO1Wn7H4pUli7VgI++H0sQMoiMZcGdDMs6+w4yIfwDipMityWkh6TQsJxrjSo1AP9f+5mSikbV9r4nY6vs6wlg9bB/fEepygLU3wC1ytwh8wIKRnKGZSk6TN+Wj3dagDWVSSGDe1spsgxjWnO+G6qIrgEDgE8Th9s= X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: 3bf3d899-7d0b-4fe8-3ee5-08de1ad5148e X-MS-Exchange-CrossTenant-AuthSource: DM4PR10MB8218.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 03 Nov 2025 12:32:40.9054 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: dCeD2YCN+3KlhK2PgxKBqzRdI7JVDVei82w4mnUoSqi0MSdZw5XCtUaWyPyz0nrJeMNBy6fgWlkIYcNnmZOtvmrqctYZpUF+Xp/I+KHxdkU= X-MS-Exchange-Transport-CrossTenantHeadersStamped: DS4PR10MB997575 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1121,Hydra:6.1.9,FMLib:17.12.100.49 definitions=2025-11-03_01,2025-11-03_01,2025-10-01_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 phishscore=0 bulkscore=0 malwarescore=0 spamscore=0 suspectscore=0 mlxscore=0 adultscore=0 mlxlogscore=999 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2510240000 definitions=main-2511030113 X-Proofpoint-GUID: FEQr1aBUwY6_qf20eaVot9DsRWwDGzhK X-Proofpoint-ORIG-GUID: FEQr1aBUwY6_qf20eaVot9DsRWwDGzhK X-Authority-Analysis: v=2.4 cv=B9m0EetM c=1 sm=1 tr=0 ts=6908a0ee cx=c_pps a=OOZaFjgC48PWsiFpTAqLcw==:117 a=OOZaFjgC48PWsiFpTAqLcw==:17 a=6eWqkTHjU83fiwn7nKZWdM+Sl24=:19 a=z/mQ4Ysz8XfWz/Q5cLBRGdckG28=:19 a=lCpzRmAYbLLaTzLvsPZ7Mbvzbb8=:19 a=xqWC_Br6kY4A:10 a=6UeiqGixMTsA:10 a=GoEa3M9JfhUA:10 a=VkNPw1HP01LnGYTKEx00:22 a=yPCof4ZbAAAA:8 a=Z_ABK6pF82JkGIUUY4QA:9 a=UhEZJTgQB8St2RibIkdl:22 a=Z5ABNNGmrOfJ6cZ5bIyy:22 a=QOGEsqRv6VhmHaoFNykA:22 X-Proofpoint-Spam-Details-Enc: AW1haW4tMjUxMTAzMDExNCBTYWx0ZWRfX/PZWDxOUJy28 1GP8qOPr9TeAE6JZIJSMPbqBF9JV2zOgMmoX8oGLykavfvuy1QVHke2iEN1fFrsVKHTtIZH9tBq QixqpEY6voewAXgb7xXEjEINnQ16hbXtmNgLGtwO0/Bkg7jW3gcM0fUySSaMdByVw65ZguCr6dO M3yGSlgvHj7nFKml2boYhjdIk53m6+jZ0r+D1RlfZmn8n5+8mq3AyaqYbeD3z1fuhh1fIoh592m U1YHUzd/D6x6AqiLjtcJ0VvNU2NcwdYawC97KI9nm/ZCGnAAIe5OMgWYZQEeBmk3Mz+mNZXdADy bvBExZY/8oFiZ1WcQe8SF3FFsqxl1Tg01YssQETt5684/WIya79Gjd1jtikbwYtHGUnoGOlXq0R 8fip6SShG5AQqDDdhPV/Sv50tSTKOg== Content-Type: text/plain; charset="utf-8" The leaf entry PMD case is confusing as only migration entries and device private entries are valid at PMD level, not true swap entries. We repeatedly perform checks of the form is_swap_pmd() || pmd_trans_huge() which is itself confusing - it implies that leaf entries at PMD level exist and are different from huge entries. Address this confusion by introduced pmd_is_huge() which checks for either case. Sadly due to header dependency issues (huge_mm.h is included very early on in headers and cannot really rely on much else) we cannot pmd_is_valid_leafent() here. However since these are the only valid, handled cases the function is still achieving what it intends to do. We then replace all instances of is_swap_pmd() || pmd_trans_huge() with pmd_is_huge() invocations and adjust logic accordingly to accommodate this. No functional change intended. Signed-off-by: Lorenzo Stoakes --- include/linux/huge_mm.h | 39 +++++++++++++++++++++++++++++++++++---- include/linux/swapops.h | 6 ++++++ mm/huge_memory.c | 3 ++- mm/memory.c | 4 ++-- mm/mprotect.c | 2 +- mm/mremap.c | 2 +- 6 files changed, 47 insertions(+), 9 deletions(-) diff --git a/include/linux/huge_mm.h b/include/linux/huge_mm.h index cbb2243f8e56..a09b6d39f450 100644 --- a/include/linux/huge_mm.h +++ b/include/linux/huge_mm.h @@ -419,10 +419,36 @@ void reparent_deferred_split_queue(struct mem_cgroup = *memcg); void __split_huge_pmd(struct vm_area_struct *vma, pmd_t *pmd, unsigned long address, bool freeze); =20 +/** + * pmd_is_huge() - Is this PMD either a huge PMD entry or a leafentry? + * @pmd: The PMD to check. + * + * A huge PMD entry is a non-empty entry which is present and marked huge = or a + * huge laef entry. This check be performed without the appropriate locks + * held, in which case the condition should be rechecked after they are + * acquired. + * + * Returns: true if this PMD is huge, false otherwise. + */ +static inline bool pmd_is_huge(pmd_t pmd) +{ + if (pmd_present(pmd)) { + return pmd_trans_huge(pmd); + } else if (!pmd_none(pmd)) { + /* + * Non-present PMDs must be valid huge non-present entries. We + * cannot assert that here due to header dependency issues. + */ + return true; + } + + return false; +} + #define split_huge_pmd(__vma, __pmd, __address) \ do { \ pmd_t *____pmd =3D (__pmd); \ - if (is_swap_pmd(*____pmd) || pmd_trans_huge(*____pmd)) \ + if (pmd_is_huge(*____pmd)) \ __split_huge_pmd(__vma, __pmd, __address, \ false); \ } while (0) @@ -469,10 +495,10 @@ static inline int is_swap_pmd(pmd_t pmd) static inline spinlock_t *pmd_trans_huge_lock(pmd_t *pmd, struct vm_area_struct *vma) { - if (is_swap_pmd(*pmd) || pmd_trans_huge(*pmd)) + if (pmd_is_huge(*pmd)) return __pmd_trans_huge_lock(pmd, vma); - else - return NULL; + + return NULL; } static inline spinlock_t *pud_trans_huge_lock(pud_t *pud, struct vm_area_struct *vma) @@ -744,6 +770,11 @@ static inline struct folio *get_persistent_huge_zero_f= olio(void) { return NULL; } + +static inline bool pmd_is_huge(pmd_t pmd) +{ + return false; +} #endif /* CONFIG_TRANSPARENT_HUGEPAGE */ =20 static inline int split_folio_to_list_to_order(struct folio *folio, diff --git a/include/linux/swapops.h b/include/linux/swapops.h index f1277647262d..41cfc6d59054 100644 --- a/include/linux/swapops.h +++ b/include/linux/swapops.h @@ -471,6 +471,12 @@ static inline pmd_t swp_entry_to_pmd(swp_entry_t entry) } =20 #else /* CONFIG_ARCH_ENABLE_THP_MIGRATION */ +static inline int set_pmd_migration_entry(struct page_vma_mapped_walk *pvm= w, + struct page *page) +{ + BUILD_BUG(); +} + static inline void remove_migration_pmd(struct page_vma_mapped_walk *pvmw, struct page *new) { diff --git a/mm/huge_memory.c b/mm/huge_memory.c index 4f8d4cd106e8..fa4cad7d512f 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -2810,8 +2810,9 @@ int move_pages_huge_pmd(struct mm_struct *mm, pmd_t *= dst_pmd, pmd_t *src_pmd, pm spinlock_t *__pmd_trans_huge_lock(pmd_t *pmd, struct vm_area_struct *vma) { spinlock_t *ptl; + ptl =3D pmd_lock(vma->vm_mm, pmd); - if (likely(is_swap_pmd(*pmd) || pmd_trans_huge(*pmd))) + if (likely(pmd_is_huge(*pmd))) return ptl; spin_unlock(ptl); return NULL; diff --git a/mm/memory.c b/mm/memory.c index ce2e3ce23f3b..1412fc84172d 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -1374,7 +1374,7 @@ copy_pmd_range(struct vm_area_struct *dst_vma, struct= vm_area_struct *src_vma, src_pmd =3D pmd_offset(src_pud, addr); do { next =3D pmd_addr_end(addr, end); - if (is_swap_pmd(*src_pmd) || pmd_trans_huge(*src_pmd)) { + if (pmd_is_huge(*src_pmd)) { int err; =20 VM_BUG_ON_VMA(next-addr !=3D HPAGE_PMD_SIZE, src_vma); @@ -1923,7 +1923,7 @@ static inline unsigned long zap_pmd_range(struct mmu_= gather *tlb, pmd =3D pmd_offset(pud, addr); do { next =3D pmd_addr_end(addr, end); - if (is_swap_pmd(*pmd) || pmd_trans_huge(*pmd)) { + if (pmd_is_huge(*pmd)) { if (next - addr !=3D HPAGE_PMD_SIZE) __split_huge_pmd(vma, pmd, addr, false); else if (zap_huge_pmd(tlb, vma, pmd, addr)) { diff --git a/mm/mprotect.c b/mm/mprotect.c index ac2cd613f76e..2134e28257d0 100644 --- a/mm/mprotect.c +++ b/mm/mprotect.c @@ -474,7 +474,7 @@ static inline long change_pmd_range(struct mmu_gather *= tlb, goto next; =20 _pmd =3D pmdp_get_lockless(pmd); - if (is_swap_pmd(_pmd) || pmd_trans_huge(_pmd)) { + if (pmd_is_huge(_pmd)) { if ((next - addr !=3D HPAGE_PMD_SIZE) || pgtable_split_needed(vma, cp_flags)) { __split_huge_pmd(vma, pmd, addr, false); diff --git a/mm/mremap.c b/mm/mremap.c index 62b6827abacf..fdb0485ede74 100644 --- a/mm/mremap.c +++ b/mm/mremap.c @@ -850,7 +850,7 @@ unsigned long move_page_tables(struct pagetable_move_co= ntrol *pmc) if (!new_pmd) break; again: - if (is_swap_pmd(*old_pmd) || pmd_trans_huge(*old_pmd)) { + if (pmd_is_huge(*old_pmd)) { if (extent =3D=3D HPAGE_PMD_SIZE && move_pgt_entry(pmc, HPAGE_PMD, old_pmd, new_pmd)) continue; --=20 2.51.0 From nobody Sat Feb 7 17:56:00 2026 Received: from mx0a-00069f02.pphosted.com (mx0a-00069f02.pphosted.com [205.220.165.32]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B0B5E31281F; Mon, 3 Nov 2025 12:34:39 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=205.220.165.32 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1762173281; cv=fail; b=DjYsLdbu5m8hkTppbT1BXeOtilmQG4m8r2CWEXOhLL2Ymxdi1jgz6qQLmQNSf/T6XCtMQoYbBabRbyrmHJAcdkjhJGt/bgQ7bTnYOoYotvWMFMcHU4HBGoxMtqv3PVRnDfaaoix/6CbqahzS1BW7zNh2zrsRkAU0B2ChqXNK/50= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1762173281; c=relaxed/simple; bh=+rf3YXeuzjjthqg07CHrFcauklWS1z/O6DICvbKmiVo=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: Content-Type:MIME-Version; b=uL0fNswryUlyPRJPQ2sMo0RqKdvDFcYU4EV+e2E54IJI4WesiB1OfsoqKJ2LIgBmRwxAbUrdxoaRaxcBW25XhduIMAi6X4nlcafN5GX0/PHEMjDgHDrG6hfXBVBbUULdnsiZDXlpBE6L6uDytdSEJ7UmIWDMoqTSOYgR7n82f5w= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=oracle.com; spf=pass smtp.mailfrom=oracle.com; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b=bIpBkFD4; dkim=pass (1024-bit key) header.d=oracle.onmicrosoft.com header.i=@oracle.onmicrosoft.com header.b=Hk2Y8DAl; arc=fail smtp.client-ip=205.220.165.32 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=oracle.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=oracle.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b="bIpBkFD4"; dkim=pass (1024-bit key) header.d=oracle.onmicrosoft.com header.i=@oracle.onmicrosoft.com header.b="Hk2Y8DAl" Received: from pps.filterd (m0333521.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 5A3CTgRx019251; Mon, 3 Nov 2025 12:32:48 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=cc :content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s= corp-2025-04-25; bh=NvH+jJP+Gz965mvpFs/APDOgcaEOo3H79MwRURF5cS8=; b= bIpBkFD430qhiSDUyyfyHogVBDYabgYMaPN1U/ep3H9YpLOUZSOHswM5YMTTo5NQ d1ipIWC4HMVed1ojN/xZcVhgSj5vIIy3u+6s8I2sCI7WvnwNjiq8YrOMAhbiIziv HuaYMmYC1sJUaAaZNSSZA8MivtTTOgkrG3iDz+gpibub+MO1KboS3OaWR7T091gb yZU2fEaqRzF0kcvRwituvK1O8VTo8gXWe5eA71fTodMVAcP7S0eTquGqJnqdMIxO tHfPLWxEEgE+N+J8jVFJRGkK2B+wum2KRAZGC2VOgHkwrtGNZehYjNjYLLVWgzRR sUJngnrRJoaFAzrU7+1Apg== Received: from phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta02.appoci.oracle.com [147.154.114.232]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 4a6ven809b-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 03 Nov 2025 12:32:48 +0000 (GMT) Received: from pps.filterd (phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (8.18.1.2/8.18.1.2) with ESMTP id 5A3BY5jO020996; Mon, 3 Nov 2025 12:32:47 GMT Received: from sj2pr03cu001.outbound.protection.outlook.com (mail-westusazon11012064.outbound.protection.outlook.com [52.101.43.64]) by phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTPS id 4a58n83esg-2 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 03 Nov 2025 12:32:47 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=p005eSyhGeVUx5/uffNbRBnY8eH7zctKF/H/x5vw2Avt6brPneO8n1IJY5CDa5lK5vJM8iS8pQouk/O6ZJ1dexgY5yfybxmm7RyOIPfU0ShUpZYMZsHHLKF/FXzGUmCJwq4QwufDRvU5W2R6MsvcCvP7Tf6tddmnMkveOg9hzq3/hzdMR2MWMhYRUno7x/sHh6pnQpi1u4gUS9xXT/ILxTNSCfVQo3mfExa9Du2sEdPb9FfcGXiYz1T8ZsC8/pEqXGFBcHYF5MV0wtVe53eHBiEqNQFzwAyiUM5+XUmUbtSGHgJwrob2lpbguVcHhdIJ/m/nldFcrykuPOpi2P0w0w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=NvH+jJP+Gz965mvpFs/APDOgcaEOo3H79MwRURF5cS8=; b=khcs4y6HqyvEvoWf86IL0C+wUo7y682Tyn4rALYDRiu1w+hNur9cNXmAY2u3U6OkTDdJ6GCS0j7xVf7jZT6S0s/qfShw/Nig+C0nDubfxq0ULXRcEb9yrvQURNrl7Y+hs40oR0ZqfVhLkksRErzAXFw1es9ooNsgmnbYI/P/tyyzcG3fACHe9R+4kR+vkNRJV/cj1QO1b8A1MLGWqR3la1nnuU86EJCrPbrVm102LZfdFtaecMoUJnQTHsLcQg8/ne3G1h3XZhvNpXS//2W2vs0n+S82FqRCI5ZPQFFrUnaRWk++HS9o8TUKATQ1ZJG3OjTRLvzffgoar8EjLNIoeQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=NvH+jJP+Gz965mvpFs/APDOgcaEOo3H79MwRURF5cS8=; b=Hk2Y8DAlnXqCwr4W3eRho3pEENaqV59ELRSr8qlGIMMkVgL79hOIUaW+YsXjspPbaBT7AaVq4v4OIuC7BnRB1xVD/8te0eBi7oxM4blzWMi9gW3hpymcqviHW30RTjQxwxRQ5j5+zpIuP19lpSlaM8EiooTBXBKmfS5jZoa6ClA= Received: from DM4PR10MB8218.namprd10.prod.outlook.com (2603:10b6:8:1cc::16) by DS4PR10MB997575.namprd10.prod.outlook.com (2603:10b6:8:31e::5) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9275.16; Mon, 3 Nov 2025 12:32:44 +0000 Received: from DM4PR10MB8218.namprd10.prod.outlook.com ([fe80::2650:55cf:2816:5f2]) by DM4PR10MB8218.namprd10.prod.outlook.com ([fe80::2650:55cf:2816:5f2%2]) with mapi id 15.20.9253.018; Mon, 3 Nov 2025 12:32:44 +0000 From: Lorenzo Stoakes To: Andrew Morton Cc: Christian Borntraeger , Janosch Frank , Claudio Imbrenda , David Hildenbrand , Alexander Gordeev , Gerald Schaefer , Heiko Carstens , Vasily Gorbik , Sven Schnelle , Peter Xu , Alexander Viro , Christian Brauner , Jan Kara , Arnd Bergmann , Zi Yan , Baolin Wang , "Liam R . Howlett" , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Lance Yang , Muchun Song , Oscar Salvador , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Matthew Brost , Joshua Hahn , Rakie Kim , Byungchul Park , Gregory Price , Ying Huang , Alistair Popple , Axel Rasmussen , Yuanchu Xie , Wei Xu , Kemeng Shi , Kairui Song , Nhat Pham , Baoquan He , Chris Li , SeongJae Park , Matthew Wilcox , Jason Gunthorpe , Leon Romanovsky , Xu Xin , Chengming Zhou , Jann Horn , Miaohe Lin , Naoya Horiguchi , Pedro Falcato , Pasha Tatashin , Rik van Riel , Harry Yoo , Hugh Dickins , linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-s390@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, linux-arch@vger.kernel.org, damon@lists.linux.dev Subject: [PATCH 12/16] mm: remove remaining is_swap_pmd() users and is_swap_pmd() Date: Mon, 3 Nov 2025 12:31:53 +0000 Message-ID: <9685b7d28388da080af21919b8adeff0c4211e79.1762171281.git.lorenzo.stoakes@oracle.com> X-Mailer: git-send-email 2.51.0 In-Reply-To: References: Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: LO4P123CA0372.GBRP123.PROD.OUTLOOK.COM (2603:10a6:600:18e::17) To DM4PR10MB8218.namprd10.prod.outlook.com (2603:10b6:8:1cc::16) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DM4PR10MB8218:EE_|DS4PR10MB997575:EE_ X-MS-Office365-Filtering-Correlation-Id: c5aa7754-22ef-4648-a9ec-08de1ad51660 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|376014|7416014|366016|1800799024; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?IW1GR1ywC2wEJfvfVPthF5S/nmt51Zax/ThrIWfaTBXgeitE58Oer6PN2n//?= =?us-ascii?Q?6y44ukBGYjVYX7v5mE5GmMb+5DlQ0RaDBc0Z2B4tmGGcCbjLjg+9SiegV4Tl?= =?us-ascii?Q?mK8YmA6HS4Xvgz6/91pGFIj9s8Smi5GXGg02YIc8wyaQWrflsTV5GXBohTU+?= =?us-ascii?Q?4DDb/nSaYc0rREFDb3EpQ4HbKEcRdgQf0z8LdfyUke/TDtQHp3atu5KBPO9Q?= =?us-ascii?Q?eMWkRAbUs51E/EK/OSAxPKX2Y+8bPn35EbvxelJKqI7jQkqv9Jc1aA4nQ5sP?= =?us-ascii?Q?SeZzecUS21B+LNOWiPZfBrI5/bcNh3hoFISK6bDl9ul6eiO6v18Opsl5cyVZ?= =?us-ascii?Q?QH5xILSQyvCKMsOk7coOAXpjl0tVi7Xl/nz3w9Wht9BVS8ycLSYPrLMQhZfv?= =?us-ascii?Q?iwYx4mxiZiffKXN69zKYIl3wQqEwBct1l0iS5aYtx0lgWHZfVasd3KlI06OY?= =?us-ascii?Q?8QHpMqbj4bwLMETapOLdee0GTVNgBkn1VUlaWItiy4THlU6MyhYgDFXBSrV2?= =?us-ascii?Q?sr4q03t7/2Rl9Mei7VYNBqemlXO4DIkq6oSH7YghzmRYbLJUWCvZjuYlqMaC?= =?us-ascii?Q?iB+IWwj3Ewwp74N+GzagFD+WdSqgx9kBXbTlheuF+4Xw4a214TrozBdDbfJc?= =?us-ascii?Q?gDDQu/uAz1F9v9+cV1z63PUrUICwzAoRtYDjN6ggtW4s59scu2Jjk4CiKfgz?= =?us-ascii?Q?EGjv12HOKP8D2GrCSXVUJZMRr9FWjP/Vj+rue+QhnOJEN9F1nUlzYAwXzqka?= =?us-ascii?Q?vwzQDjuXqz5L+XgX+zs1qpCA35wcOfZqTHiI/3YitcHvvrcDwM0kQJWNnTJO?= =?us-ascii?Q?BlugEt/d2tTySImyJ6hGca2aYnGzsojGDWjL4zpwgHoZNgY5v49PhVnrmRjl?= =?us-ascii?Q?LfgTrRZzjaHsom+RZZtZCHAjCOdG1Yf4U4PrqzNq99hnzwjMdf2xsQr5blov?= =?us-ascii?Q?dSXBpn+9ispRO4WKa8yehr0jGsNc3fRx5xBRoP2uhqOo+KvSjmlSRJmOrlaF?= =?us-ascii?Q?BjMestFaUchxows2gYVcxefpNDZmy/GiR9p+WQlRFJIF/zPTQaK3dNG22VlA?= =?us-ascii?Q?VIOyOZ2HcquWviONCyNl5khrtYUGvnORPGTG6Qt0yMezyYmW+0UImR2zpng+?= =?us-ascii?Q?Ah7vYcC5nenxjWT3w7hHMeXpGq0MRZDa7nC3PDp6/5m3mvdxnyToHjoZahsq?= =?us-ascii?Q?dlYmzpPxS21ottANw0FcKFgfBDodxpZ0zr3qKhgVEfXHhBkA81UHN3xtdSEm?= =?us-ascii?Q?I/v6PonF4/US9FXfeEqPmvYnrq2YDGIrWdpZ2u7dRbtMCaTYqy8Zkfu1sGRZ?= =?us-ascii?Q?0W1aaYFz1My3SYKpTuh0yF/Nj+moarIAhMSjEWr9IRby+YLP3xFhLtOc43Zc?= =?us-ascii?Q?TQISl344ZaDGRxlNaoiezxERMAHyHrOm6TMEaSsHi5ludMJ2V5ujWMB7FrM+?= =?us-ascii?Q?xDdqNki/GsTbIy6WhGdfOvFJCRBEwbx/?= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DM4PR10MB8218.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(376014)(7416014)(366016)(1800799024);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?g6XvABi+8mwVtWF6s+dJ/pxbTNARo7m0Za4uWgccKvJjGLLi/T219ey80G3l?= =?us-ascii?Q?vCNuMg7FpioC2n3hjduDF8A4n51LE8lBu4wdrsoK8ykyokqw0v/R7tjBGP3R?= =?us-ascii?Q?yngpyHzGZs6bL1fLVMBmsR4j5E5r8H/Q4LLm5L75YUCH7PsBBKhc6gIIhoOj?= =?us-ascii?Q?VKigPIFIgH3bkBd28QgALThqOFN4f78eUjshlvvdtO5MqyNQXzhfKPlcuHU8?= =?us-ascii?Q?wg82VfRWcJWu2i0Wqd5m8ALkvu6wd+4SWQeDWNk65SB3Meuy5KThk+nFyBKa?= =?us-ascii?Q?fJr4z1xSaK7LmtPn0Y6TWKseiHeIw60omLfTkPvkkeT6e6WTqxamrmI/eeZ4?= =?us-ascii?Q?Uar8uMq0ELaFaQi06uXFHnjvCNha7m06qfIXQ88e55UNhfIC7avmRqvr7e4d?= =?us-ascii?Q?5rEyUzN5G6DVzdpiLFJa7UE9wrSqWPjBdXTJhhUNe6X+i1wczmYnGnoZ8HiV?= =?us-ascii?Q?TUwOxC3btXFn9GbrAlljl9nX8MnqfBJ3oEYXWIcr6xaAipgR+GdSora0yMLY?= =?us-ascii?Q?ig8fyUzNP3M3X7xAXh9TqOkkROtOg0z4Y0ka4H/gxJZg82rDWLSQKZowfjrM?= =?us-ascii?Q?2xYgRYF9VxKdGHHeSM89iftVmQDLbCxlG0/YKeqvzLCgWKY5l4+yKcTSjw6b?= =?us-ascii?Q?ImFn/3hDTcEgx94bzwKBv4xR9CCA0SBNX8c4PukFQJLiQwHXiBpzFsvMib2q?= =?us-ascii?Q?svi0LmgpbMPo7ADu/xlzj7brsNSFoNaMZ8J3myxGf0QDBwlBZF95AxFqmDRg?= =?us-ascii?Q?/rqb5Xz5oLuTJJqY/J+4ej6Kam3/dQh9OSK4750fS4VxKWEvoOyiQptQNaSM?= =?us-ascii?Q?1Zzo+qrZPFL19fI2Wf1ZewhZnjy8tRMvRSqjP7rkS00YIDUXtFMah/Kce5o4?= =?us-ascii?Q?OttjTXa/WdfdDr2bAeJk/qxhYVqNEbvReskRAETHubjtHLNZ7i1iJvyp+7Hl?= =?us-ascii?Q?gPY9fUs6Y940caXZbNnAnZZEHDu+pD05I8KDLdYZSFsVeA0fa5VNhTdBnyGK?= =?us-ascii?Q?ZTOj1NWuOyycRgv77HiG6xyrEjKn9aB0oEVLgeDyqv0qC3Fp9UkTthKg3O0g?= =?us-ascii?Q?9T8ogFOOLXHJS6xjAyf1u8maKw0INYc573iV62ajIiVU9ztZq7mXtwtCTKyU?= =?us-ascii?Q?erby6a/TbvWE0F8dvKt8norPL6YJYg3YrBOjWhiYxat/DFEhmKz2Z9ayZuKp?= =?us-ascii?Q?a5AtdOQRhdDBICCmZK2TytzR+adDoLGzfQpWf1BCbEZrXuJz1q5hFN1J+FUn?= =?us-ascii?Q?68ySSQ9Ml2IDPTikwHVLzMX8xFNfKlsIqAo15RBleDzKUmWGjpvEUx4L2f/h?= =?us-ascii?Q?qSxuSIYKZUzX+0IsR/MwlMLhNG+RNsh7OjUF5LHupVOfr7jeRNDadk/wKpYS?= =?us-ascii?Q?Sij2qWKAPbK+Haq/29oejKM4VYbF5DSTqT+S/XWGzjrFzPAC75LikweprKnu?= =?us-ascii?Q?gQhIjZ6rJRTl7NEYF+KAqkQE7WI2dMDjzMCuRVK/5RX3djgN8FuDL39B5Dvi?= =?us-ascii?Q?ygr+T1K9ZoHwaDA0vPNtt4Abo5/BHvm8VU+Bc/GS/MzJGANCddWJ2KowsBCT?= =?us-ascii?Q?sHGtE9XlL6zs6EmUI6xpkHfVQtsedb5981TlI59ji5dYOSdfYq3jqSeXE8q+?= =?us-ascii?Q?ZA=3D=3D?= X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: I3icXLNLtKcvw2OrZNos1hO9ZziFmW/jslARwPz8shjXZTqY7RJvpRMljVVUs34ygsL8TWhH2B2IR30BBJxmo3ztR934akLFgGgeZZwn4i9wlGSadiFzvJKsrkCo46EDuNbUjNGa1kBG7FKiX4s9V7/2gFdIhUy71Sq7u/P4+fxTtIPo87ghiP86NUsxGsFQk/7t+gmui2rAIgKXqYYdG2ghW+LMaGMs/MZIMZf6vxplZ/pjd+etoJeOqqiKrKojU+NyuWT5g9CDMw8wGnjIOUyQFBYXF6zHLXow0XfmQPmWyegtU5oQu8mDwv3Q10l2vgOMoUdoo9MPZNrJL2+Z3+YmaS4gqEHWpUDbJHkI7Z7SxSGbR5TlqZSfnISSWgAxRURTqQR2P94ISovMZHmGztAvCqeg2R4LGm3bTkfeTuv89bowUNGkdWg8OIKBQghK5d8XS5+HjWhUa863Opoifo6agXVKp924xnCyAinZwhJNioN+75qQcXQhDC4hNE19BeLOYSsfpZqs1j5FznpZrRhnWJYExy0kaRgtUx0OVef/QmETb1aA1shiCF3eyEMhnE4TKA1XQ8nFwGMTGH8zrcHOxVZv7hJIEYTyMEpe7Z0= X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: c5aa7754-22ef-4648-a9ec-08de1ad51660 X-MS-Exchange-CrossTenant-AuthSource: DM4PR10MB8218.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 03 Nov 2025 12:32:43.9848 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: x3/VWuHOIXZTQqScDPQOCSXKlr5zbC9zx/8eAJtF8MuHpYFkKEHaMwWmgTCDOIu6+oNVFWAPimWmN5d78Pttj99S7nCZQhfhckf9wxzyuoE= X-MS-Exchange-Transport-CrossTenantHeadersStamped: DS4PR10MB997575 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1121,Hydra:6.1.9,FMLib:17.12.100.49 definitions=2025-11-03_01,2025-11-03_01,2025-10-01_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 phishscore=0 bulkscore=0 malwarescore=0 spamscore=0 suspectscore=0 mlxscore=0 adultscore=0 mlxlogscore=999 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2510240000 definitions=main-2511030113 X-Proofpoint-GUID: ns6AufHmCaktHUPsBZr5ZLOiK4PJZU_h X-Proofpoint-ORIG-GUID: ns6AufHmCaktHUPsBZr5ZLOiK4PJZU_h X-Proofpoint-Spam-Details-Enc: AW1haW4tMjUxMTAzMDExNCBTYWx0ZWRfX71ifN4ftTSR7 N6bgOOQNwCyFiBc4t9FqiTI8r71L3kT6Axfz3ASMM6wrBZx0C3gW8VcysBMfdNMJoHw5Eay1brZ s078o7gMj2uS2MlNbYqCzAE8FjOg3aEcl/Z57+NRc69HbDyEiB+c0vWEyqtm7pT4dI3FUzWsWwT uENePEzTNtiRjMpT//ocUxl7O/Qsg8jp225+o8FShJEcoC/H8AGhP+IUX/ouGOSxKK/PQ+f3z/L 1OxkIMXhgRE/bI6V65xpHWvzeiQAamMZPjdmwqmCmN+KKMvi1+oKv+YaUWhClQ8r0Hn+cf/MfkE 91uQPiLm05Um93Uy+l7GgwG0g1peL8fziqrfE7uLnTraXbFGPMpWNEJ1Ui8vlZChLwAAFQXkyhw o1gqNAsp+ohtXN0P/2kQH/MqsU7+1g== X-Authority-Analysis: v=2.4 cv=MvBfKmae c=1 sm=1 tr=0 ts=6908a0f0 cx=c_pps a=OOZaFjgC48PWsiFpTAqLcw==:117 a=OOZaFjgC48PWsiFpTAqLcw==:17 a=6eWqkTHjU83fiwn7nKZWdM+Sl24=:19 a=z/mQ4Ysz8XfWz/Q5cLBRGdckG28=:19 a=lCpzRmAYbLLaTzLvsPZ7Mbvzbb8=:19 a=xqWC_Br6kY4A:10 a=6UeiqGixMTsA:10 a=GoEa3M9JfhUA:10 a=VkNPw1HP01LnGYTKEx00:22 a=yPCof4ZbAAAA:8 a=EdvhSjy_mcwGFXzFEA0A:9 a=UhEZJTgQB8St2RibIkdl:22 a=Z5ABNNGmrOfJ6cZ5bIyy:22 a=QOGEsqRv6VhmHaoFNykA:22 Content-Type: text/plain; charset="utf-8" Update copy_huge_pmd() and change_huge_pmd() to use pmd_is_valid_leafent() - as this checks for the only valid non-present huge PMD states. Also update mm/debug_vm_pgtable.c to explicitly test for a valid leaf PMD entry (which it was not before, which was incorrect), and have it test against pmd_is_huge() and pmd_is_valid_leafent() rather than is_swap_pmd(). With these changes done there are no further users of is_swap_pmd(), so remove it. Signed-off-by: Lorenzo Stoakes --- include/linux/huge_mm.h | 9 --------- mm/debug_vm_pgtable.c | 25 +++++++++++++++---------- mm/huge_memory.c | 5 +++-- 3 files changed, 18 insertions(+), 21 deletions(-) diff --git a/include/linux/huge_mm.h b/include/linux/huge_mm.h index a09b6d39f450..f381339842fa 100644 --- a/include/linux/huge_mm.h +++ b/include/linux/huge_mm.h @@ -486,11 +486,6 @@ void vma_adjust_trans_huge(struct vm_area_struct *vma,= unsigned long start, spinlock_t *__pmd_trans_huge_lock(pmd_t *pmd, struct vm_area_struct *vma); spinlock_t *__pud_trans_huge_lock(pud_t *pud, struct vm_area_struct *vma); =20 -static inline int is_swap_pmd(pmd_t pmd) -{ - return !pmd_none(pmd) && !pmd_present(pmd); -} - /* mmap_lock must be held on entry */ static inline spinlock_t *pmd_trans_huge_lock(pmd_t *pmd, struct vm_area_struct *vma) @@ -693,10 +688,6 @@ static inline void vma_adjust_trans_huge(struct vm_are= a_struct *vma, struct vm_area_struct *next) { } -static inline int is_swap_pmd(pmd_t pmd) -{ - return 0; -} static inline spinlock_t *pmd_trans_huge_lock(pmd_t *pmd, struct vm_area_struct *vma) { diff --git a/mm/debug_vm_pgtable.c b/mm/debug_vm_pgtable.c index f0f5fbc13784..8f247fcf1865 100644 --- a/mm/debug_vm_pgtable.c +++ b/mm/debug_vm_pgtable.c @@ -74,6 +74,7 @@ struct pgtable_debug_args { unsigned long fixed_pte_pfn; =20 swp_entry_t swp_entry; + swp_entry_t leaf_entry; }; =20 static void __init pte_basic_tests(struct pgtable_debug_args *args, int id= x) @@ -745,7 +746,7 @@ static void __init pmd_soft_dirty_tests(struct pgtable_= debug_args *args) WARN_ON(pmd_soft_dirty(pmd_clear_soft_dirty(pmd))); } =20 -static void __init pmd_swap_soft_dirty_tests(struct pgtable_debug_args *ar= gs) +static void __init pmd_leaf_soft_dirty_tests(struct pgtable_debug_args *ar= gs) { pmd_t pmd; =20 @@ -757,15 +758,16 @@ static void __init pmd_swap_soft_dirty_tests(struct p= gtable_debug_args *args) return; =20 pr_debug("Validating PMD swap soft dirty\n"); - pmd =3D swp_entry_to_pmd(args->swp_entry); - WARN_ON(!is_swap_pmd(pmd)); + pmd =3D swp_entry_to_pmd(args->leaf_entry); + WARN_ON(!pmd_is_huge(pmd)); + WARN_ON(!pmd_is_valid_leafent(pmd)); =20 WARN_ON(!pmd_swp_soft_dirty(pmd_swp_mksoft_dirty(pmd))); WARN_ON(pmd_swp_soft_dirty(pmd_swp_clear_soft_dirty(pmd))); } #else /* !CONFIG_TRANSPARENT_HUGEPAGE */ static void __init pmd_soft_dirty_tests(struct pgtable_debug_args *args) {= } -static void __init pmd_swap_soft_dirty_tests(struct pgtable_debug_args *ar= gs) { } +static void __init pmd_leaf_soft_dirty_tests(struct pgtable_debug_args *ar= gs) { } #endif /* CONFIG_TRANSPARENT_HUGEPAGE */ =20 static void __init pte_swap_exclusive_tests(struct pgtable_debug_args *arg= s) @@ -818,7 +820,7 @@ static void __init pte_swap_tests(struct pgtable_debug_= args *args) } =20 #ifdef CONFIG_ARCH_ENABLE_THP_MIGRATION -static void __init pmd_swap_tests(struct pgtable_debug_args *args) +static void __init pmd_leafent_tests(struct pgtable_debug_args *args) { swp_entry_t arch_entry; pmd_t pmd1, pmd2; @@ -827,15 +829,16 @@ static void __init pmd_swap_tests(struct pgtable_debu= g_args *args) return; =20 pr_debug("Validating PMD swap\n"); - pmd1 =3D swp_entry_to_pmd(args->swp_entry); - WARN_ON(!is_swap_pmd(pmd1)); + pmd1 =3D swp_entry_to_pmd(args->leaf_entry); + WARN_ON(!pmd_is_huge(pmd1)); + WARN_ON(!pmd_is_valid_leafent(pmd1)); =20 arch_entry =3D __pmd_to_swp_entry(pmd1); pmd2 =3D __swp_entry_to_pmd(arch_entry); WARN_ON(memcmp(&pmd1, &pmd2, sizeof(pmd1))); } #else /* !CONFIG_ARCH_ENABLE_THP_MIGRATION */ -static void __init pmd_swap_tests(struct pgtable_debug_args *args) { } +static void __init pmd_leafent_tests(struct pgtable_debug_args *args) { } #endif /* CONFIG_ARCH_ENABLE_THP_MIGRATION */ =20 static void __init swap_migration_tests(struct pgtable_debug_args *args) @@ -1229,6 +1232,8 @@ static int __init init_args(struct pgtable_debug_args= *args) max_swap_offset =3D swp_offset(pte_to_swp_entry(swp_entry_to_pte(swp_entr= y(0, ~0UL)))); /* Create a swp entry with all possible bits set while still being swap. = */ args->swp_entry =3D swp_entry(MAX_SWAPFILES - 1, max_swap_offset); + /* Create a non-present migration entry. */ + args->leaf_entry =3D make_writable_migration_entry(~0UL); =20 /* * Allocate (huge) pages because some of the tests need to access @@ -1318,12 +1323,12 @@ static int __init debug_vm_pgtable(void) pte_soft_dirty_tests(&args); pmd_soft_dirty_tests(&args); pte_swap_soft_dirty_tests(&args); - pmd_swap_soft_dirty_tests(&args); + pmd_leaf_soft_dirty_tests(&args); =20 pte_swap_exclusive_tests(&args); =20 pte_swap_tests(&args); - pmd_swap_tests(&args); + pmd_leafent_tests(&args); =20 swap_migration_tests(&args); =20 diff --git a/mm/huge_memory.c b/mm/huge_memory.c index fa4cad7d512f..cdb410e8e13b 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -1875,7 +1875,8 @@ int copy_huge_pmd(struct mm_struct *dst_mm, struct mm= _struct *src_mm, ret =3D -EAGAIN; pmd =3D *src_pmd; =20 - if (unlikely(thp_migration_supported() && is_swap_pmd(pmd))) { + if (unlikely(thp_migration_supported() && + pmd_is_valid_leafent(pmd))) { copy_huge_non_present_pmd(dst_mm, src_mm, dst_pmd, src_pmd, addr, dst_vma, src_vma, pmd, pgtable); ret =3D 0; @@ -2562,7 +2563,7 @@ int change_huge_pmd(struct mmu_gather *tlb, struct vm= _area_struct *vma, if (!ptl) return 0; =20 - if (thp_migration_supported() && is_swap_pmd(*pmd)) { + if (thp_migration_supported() && pmd_is_valid_leafent(*pmd)) { change_non_present_huge_pmd(mm, addr, pmd, uffd_wp, uffd_wp_resolve); goto unlock; --=20 2.51.0 From nobody Sat Feb 7 17:56:00 2026 Received: from mx0b-00069f02.pphosted.com (mx0b-00069f02.pphosted.com [205.220.177.32]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id DEB1C283CBF; Mon, 3 Nov 2025 12:38:56 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=205.220.177.32 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1762173538; cv=fail; b=IJkFYbYvCNXNO45eFDAcdPkbOBRIHNJlaxLEoAqzUubeU8vBAs707Q5jFc68GmfxjMtM/3zefHp6PIvR14OAHLDMjqBuIks9VDSGWUpkXyTn1/6/AjlH1kXfHzDdGZoyIUMNc7GGQNr8UmP8+Zht9BbAXBdm8rESAiCA3m5Y8TI= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1762173538; c=relaxed/simple; bh=QV5bnX+5mjZsiw57H9uq+znm8pFF1RGxxblM1BlZYSM=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: Content-Type:MIME-Version; b=uiSYxZvuMarhWdFWtsIRbu5IxCs0lV5ASEvahz9xCIw8vZRq0hwz3cJjpjEqmsVptg3wSjI87b3hec90QrHSjcYkAEH1XZE40eDbrqj0MSX03sd0PIgER5ta0ytklaAN/lg9MdAirrxZavWx7J/RWBjTrDEvO/QcpreT/nizqHw= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=oracle.com; spf=pass smtp.mailfrom=oracle.com; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b=cXnNz26J; dkim=pass (1024-bit key) header.d=oracle.onmicrosoft.com header.i=@oracle.onmicrosoft.com header.b=REWwY/kP; arc=fail smtp.client-ip=205.220.177.32 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=oracle.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=oracle.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b="cXnNz26J"; dkim=pass (1024-bit key) header.d=oracle.onmicrosoft.com header.i=@oracle.onmicrosoft.com header.b="REWwY/kP" Received: from pps.filterd (m0246630.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 5A3CP2eK031542; Mon, 3 Nov 2025 12:32:51 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=cc :content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s= corp-2025-04-25; bh=8VyPY5KuGo+/6XHmI/uGvnOahcidViWOsG67x7t6fyg=; b= cXnNz26JHf+IXn81QUNCP4nNS8I5YI+FXE6lW0R1FdluVyrZECykKQGlti/n0X45 zokKL3PwOlG6GFLuSEzZDUrvdHZLrKBA9tjqdPvIjU0aAvv0mufQ2MTE0bbAH2AT kIejMyTzD8HLMIDv12Xe2ad+wOxUKVgldq4MsvJ/HnC7OLsc23RG7iMbjg+aH1UL QoQUZYS8ivqKN8VyizqSgpoeJGjZ+rGVekBvO8WCRSaS5e/JR9Mjsha3+88dSdsl R3ezk8vM+tIxab/uLIgFtS8SzQ42Ph/58M6oH779JBZgJFDPvuKZqnYDwXyhxVU6 x1FSfULsT8vnNZpgfHyk+g== Received: from iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta01.appoci.oracle.com [130.35.100.223]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 4a6vcb80g2-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 03 Nov 2025 12:32:51 +0000 (GMT) Received: from pps.filterd (iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com (8.18.1.2/8.18.1.2) with ESMTP id 5A3BIR77007356; Mon, 3 Nov 2025 12:32:50 GMT Received: from sj2pr03cu001.outbound.protection.outlook.com (mail-westusazon11012014.outbound.protection.outlook.com [52.101.43.14]) by iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTPS id 4a58nbkq9b-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 03 Nov 2025 12:32:50 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=ujDj5HgC2srMi6TPzakZURIKkehbFWEljGEIDWIsVIspa9K5N2toTAlLB0ltXESYzngnExnYbt2+45afblnIsE5MC+fdsgVKfbGu8l7rxKnNFu7KLQ84uKyIXMXuRimgDvrMAO0GqAvDbTslmg3vfAWTLB/k9OH4iMa0olSMRQGHscwmmf5o1+6VBzRISr+74gWcABbf6y/zpB9alhx+gPVfB5VH8c6v5VuimJoCkGJM2unFXqr3ypaL2L0W809/w/yLWG2dChgobk/l3IecTZiH7puk8TeS5BDEYYuA9qGTJJY8MKeTvcB48bZgrV/nl0cqj2Qq4WMco/u1TV3fAw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=8VyPY5KuGo+/6XHmI/uGvnOahcidViWOsG67x7t6fyg=; b=oI1Tjb0YFOr2UEqhnlTeBauumjyWfA+wD6BH4biNm2gFyIVBkNDjavYOmzvmdEmTWo5jiCmVAmUlzKlkJhw6aTUbF8LZtQ50Dio51BnOtbLzSBuQO8z0CwhgA2jpCZFApqJTxUUw4+sv2JtEw/kfg3GryW3guee2iUuEGPauPgFoLJE7IDjztxeFUxJOgKbTlDJViLx73Gnob14xhNJR7AEtAaj64jfWMCxrn8IWES/zH+9Yw1XxRM4LJdPrtNLIihZrlb7dn31mDEqgTkNL5dvXDI0haeSAiAYXpK/NRP7CcIK/EsuKqi7yHISvJH7xhvxCzS2GYdaHaBF2YcUdMA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=8VyPY5KuGo+/6XHmI/uGvnOahcidViWOsG67x7t6fyg=; b=REWwY/kP+SPy+/fkB0z5wT7s/e8YuUG741fopviVgZGBKRBKSh8UUdUKXxrpMwjra1puzEBS0/U2fjrE9DfXpxe1ICfDAXmcX3EXlG6g0pqMyddBm+tt7AlcCaVHF47VH2QlFkdrdiQkFuIET20LpVHMqahLReIhUDglF+ZfRDg= Received: from DM4PR10MB8218.namprd10.prod.outlook.com (2603:10b6:8:1cc::16) by DS4PR10MB997575.namprd10.prod.outlook.com (2603:10b6:8:31e::5) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9275.16; Mon, 3 Nov 2025 12:32:47 +0000 Received: from DM4PR10MB8218.namprd10.prod.outlook.com ([fe80::2650:55cf:2816:5f2]) by DM4PR10MB8218.namprd10.prod.outlook.com ([fe80::2650:55cf:2816:5f2%2]) with mapi id 15.20.9253.018; Mon, 3 Nov 2025 12:32:47 +0000 From: Lorenzo Stoakes To: Andrew Morton Cc: Christian Borntraeger , Janosch Frank , Claudio Imbrenda , David Hildenbrand , Alexander Gordeev , Gerald Schaefer , Heiko Carstens , Vasily Gorbik , Sven Schnelle , Peter Xu , Alexander Viro , Christian Brauner , Jan Kara , Arnd Bergmann , Zi Yan , Baolin Wang , "Liam R . Howlett" , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Lance Yang , Muchun Song , Oscar Salvador , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Matthew Brost , Joshua Hahn , Rakie Kim , Byungchul Park , Gregory Price , Ying Huang , Alistair Popple , Axel Rasmussen , Yuanchu Xie , Wei Xu , Kemeng Shi , Kairui Song , Nhat Pham , Baoquan He , Chris Li , SeongJae Park , Matthew Wilcox , Jason Gunthorpe , Leon Romanovsky , Xu Xin , Chengming Zhou , Jann Horn , Miaohe Lin , Naoya Horiguchi , Pedro Falcato , Pasha Tatashin , Rik van Riel , Harry Yoo , Hugh Dickins , linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-s390@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, linux-arch@vger.kernel.org, damon@lists.linux.dev Subject: [PATCH 13/16] mm: remove non_swap_entry() and use leaf entry helpers instead Date: Mon, 3 Nov 2025 12:31:54 +0000 Message-ID: X-Mailer: git-send-email 2.51.0 In-Reply-To: References: Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: LO4P123CA0110.GBRP123.PROD.OUTLOOK.COM (2603:10a6:600:192::7) To DM4PR10MB8218.namprd10.prod.outlook.com (2603:10b6:8:1cc::16) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DM4PR10MB8218:EE_|DS4PR10MB997575:EE_ X-MS-Office365-Filtering-Correlation-Id: 8b1c06d2-439e-41a9-015d-08de1ad51820 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|376014|7416014|366016|1800799024; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?/w8ebB/us7l/zb6rl6NKOqxxb9k/ZGYOewqYUYcplatw0Zwan2Db+NEhLwUS?= =?us-ascii?Q?Ofz85phc0yV/fcybxqiOG39v68hrptBhBiHkq8jw1j7WjzNd3kEbiRo2D8El?= =?us-ascii?Q?41V5I88NNdlbECMJre2NUDmPBjwguxpdSHNQYqVY6+BRYsl5Xm8OREs9hPBm?= =?us-ascii?Q?olfReBszFdS8QVmqdlOpaLDviWM1T7iBHEQfA15L4lplw7rAGgtFgn/vqf6c?= =?us-ascii?Q?yIKxNZpR+nOdwe7QUg87pORC7Jaqakl6eM7fWCEIFfVsTeTmtEeWYbZheUaH?= =?us-ascii?Q?Gf5mHJcOSiqpZ/n3KdX261EVbU7kgRV6w/mZGwY5tDNPT8AEniFHNs3Zbohw?= =?us-ascii?Q?hjsM5w4+Ax7s118d2dTna9cohj6q84sXDVFODOGpfa0hwSvyChZ3Dq1FJ5AB?= =?us-ascii?Q?2uiSum7i7m7542S719h9k937seFF4o4AGEuj5nslum0DHgBH1sRBixbWlD6N?= =?us-ascii?Q?T8MIUvWbJFnljD54pCmuUkh7kfijvogqDSL7przwA8wiuIPlgLQjbU0ghVIn?= =?us-ascii?Q?Uz7hvITUIKd27tdltZw9nF6QQxmloM3CTCgswqaY65Gpb7qlOhjkwYR8Lw04?= =?us-ascii?Q?3ZGaCCrdvMywYgLKNZLpQQohcqWLDsmaa5jgQ/Zhqup29h+3oTIpJ0V97UiW?= =?us-ascii?Q?FiDdijNfwQGtnqzW107c4YoEfpTUY+KNe0PmMk+R9jAXdO6UpgyYDc4JhT7S?= =?us-ascii?Q?nZ9ezIJoGV1qNLJrYaHoXjbxXjDBE8m7o2uozLVmewZc03uyPh3BImaT5UEg?= =?us-ascii?Q?mVZJxQhieJy61rzOwxXpenXTjNJymVzNY1Hs1MQ0nQ2bWukUMcHWR/Pb7IAF?= =?us-ascii?Q?OV2RlTqvHhGnkfklQiTGI6FSBWfcVvg98NHaJTcUuQJSfAS4unXzFIXiKBbZ?= =?us-ascii?Q?k59NrQAjNgykG5YhhPA3gdfcn6+lbGqXHoK1X2Rg8YjZPeEzvMNFQemn1+nQ?= =?us-ascii?Q?5kUiCBX/hO8RAa9XS8VQ1UXqanZtDIy9nVMMMzruKwzoQaduJqnQF9QPjW1p?= =?us-ascii?Q?mVlgG6rbi3v2qhn0BPh1Tbu/G/aXZMzzBt30rgtydUujeRjU+EtEnscxzRfx?= =?us-ascii?Q?c8ExMKQPUV1wY1zmjMa6u2KvJIc5edodE86BbBnsgsRUf8f5GY5IQ75/eGxS?= =?us-ascii?Q?i3xTHdIAGpXdHkHrjjTtAM6GkM07n0sBtwrNSXrDebDh5r50ZnOGeVCQsxtq?= =?us-ascii?Q?Cu4IrqYVMp6eYn1mgaEenTwcoIqeiAqutV2slFTmE9iAuAB+eDWm5s1F+4GO?= =?us-ascii?Q?3MLLSZZepWSXNzvUGMa8qQE6PmHUEkXfjoe+J6UXycIvOBzBGRomMMp/UPz2?= =?us-ascii?Q?IhlWnN9BECcCLVdZ7rL8ki+H/dKXDf/STvg/h+6qhnjBQfApnbduMDrPo6qq?= =?us-ascii?Q?htwOm/QJBHLZE6i6WhoCVpQF5mr/cRy2BgoR9fWPcGI/2V8dFTB/aM60wecv?= =?us-ascii?Q?zT/N7XCQ3VxwiD1F+2PMQAou0nO/3grb?= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DM4PR10MB8218.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(376014)(7416014)(366016)(1800799024);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?2I2+n2sMOXdcVTdSNzhopvTm8p2buqVGk0bOjJdU0OSLOpCaXAa/dnBpGLEH?= =?us-ascii?Q?5F83u2RWN3wJ5F/ZKZm48G3+wV5R62y+jekhYijv5KcPmy2tmpWXwM/KhY5P?= =?us-ascii?Q?KQm5/pz7DV4bjxnlYnS1X8f26KaPswU1w5R5t/uXnEskIOMlci51Qo7jBb9s?= =?us-ascii?Q?PulQgEf1m9+o/2WGU5DH3YhbbVUvWiW91yrr/NVaGbPKSKjzQvpi9vpWvWjA?= =?us-ascii?Q?ymoyF78EpSfqnCLA7yILZz7nsrQkmFDBOXeRDyJjq+ILEbT/+oXx7im2xp50?= =?us-ascii?Q?1/+uLxmmwqmOxUeIwntoUvXtAgSN6NbhYElIaekIAjV9AKGfRPzlpTIVxn2W?= =?us-ascii?Q?0z9W05xt1gh0p/GmVvCdzUJz28/sSQP0lhumcZWf3Gw42RtbGeG1yafUYRMC?= =?us-ascii?Q?5sj2EUJ+h4DocRlQBHuLxmkrK7iX/RxFH6BfS/rP9Mj2EoaQJNBj06hDi2nj?= =?us-ascii?Q?p8+jUnrxzdJlfqJe5nIPXwCyKSXfRdOP7q//KTlcoFS26cYAXN+mF/HQ+38E?= =?us-ascii?Q?jGmgFDRtNKpB9aTJOAWoP0QMN7SbvWw5USAMmqfW6BuPKeJjmfg64u/+dVrH?= =?us-ascii?Q?sDMShNw6x5U+6hAwt0Os4wkkDKKEvEgycJpUk8Lz3LQV80JCuWKRL1iNuYJm?= =?us-ascii?Q?otPAYzCVHb1J/9K+a+HR+tl4k0Jg7ObN4TVpLoMo3HK+HQM5mSGCaRvrpCZQ?= =?us-ascii?Q?fmYTZKUfdXtEzUq2VumaKV1kTYHF/IrCpP3/UUg09bLg4mB0sPiPF/OqhLoR?= =?us-ascii?Q?GXU0T3Fr4D+0TZSxbFX1ukNFZki6Yzot/WZ83GI5XCU9HuAR8Z/9qYkIPLdv?= =?us-ascii?Q?ecTPYxrLWICUN8CSJNiCQWAXpt3SqiGXTpMuUHhcudsG2xEGAuWCmq1p9Crq?= =?us-ascii?Q?jGZigsd249dxyWM9KlaxbpdIh90AP4bsxA9+fYF+HMvGGB8MdJb/mqyjr7Sm?= =?us-ascii?Q?Iun0XOS0J8lw6ShRrOMiaGj6YdsXTTFbox6oaqFRKeZsyyEyM7PofChQ+1JI?= =?us-ascii?Q?jwHCaQceLReQPnQ+Q3OBXt5INoAtV5NA2yxhhg8l10txloXrJiwsQ+OORn/c?= =?us-ascii?Q?Eg/aY8NQsF3NtUdrJiwjmPBDi5UnAogzS7pqztGhj38MnIX03VLvwluF+ovM?= =?us-ascii?Q?r5O6gYQNxgaZKRPwpPx0L5XuY931e+4/kuRHxS6nDnd4iCU9oXaf2q7QN/aR?= =?us-ascii?Q?zrMqghdeJjsd8eFqCZK9k7GmhpGbvdS5iMbEg/UWhVCqqN/TVPjXu7imzVbL?= =?us-ascii?Q?Ay5rOE2J3HKeXEmQHfkckjGcByIpEf+m/B0c/UCI8I8Lcb4IEVcxhVbkmvwX?= =?us-ascii?Q?fSIR5n9iHy5FIMCHEBzFihVbQRWHVBYOCp22asxuHN1kbD2C+HQ/Fw8bxmWc?= =?us-ascii?Q?x4gLRzbRZSamC7gQeWIhFQVZTeAgM9Ftv16kdSI7GhdOs+RdU5uee32gJqDi?= =?us-ascii?Q?uN+/G/+k1hUvmUQGnwIUnadYXcdPe0k/yHdNJod5lv6/lnLXvJDDiJZkJy+S?= =?us-ascii?Q?LEMCV+2nWNKWbuCN2nFEv/k/JR4MTsMypiWJ7/BRcLu2nOn/S32GyeSNaQuH?= =?us-ascii?Q?ymEBn0VhGOs+kpWkj1tiZRssPbc5JQAVyY7vZVC23wy8h9f8m6roShW6BcrL?= =?us-ascii?Q?Vg=3D=3D?= X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: 2am32CdOPV5mdkVXOB0iB3rJpldHUAC+dq8wnQz3oy6LWSJ3lHfbp8v4wKVNs31XX4j1j/0VK6H2u0QIx0X38T3tebwGBBjw29+8tRfEpy3zJiFucWCrH21tHT1vkpnOG7SLjn0RD9kI+FVgx5iYE65g4/UkaazIxkX8FHk29gocFzUEhauwscaXJx5DDNrw/iP287AWIaKM1eraT6g42r+6W0zUQH2GTJUlmtqy/3RRiCbwcKH9aV7ED85fafGsORaZ0I5aSaZZ6qNZPSh8g+lGCgi1jibJZPq+1Ff+xzM1fN1QfUN0uQA9jbr/h7jb9/h9ba7cPvAJwfIB0XmLw6u1+Vt19f/RSCaN86I4tiERjCQz0+Iun9GjpON79Hj9i0FnP1mLL0Dq133OKXK7qMnChwOZFcNaIxmiHd8YP2TvIRPtuxzRUYcPSQ5iUJ532Uf1yz/Vq8IkLMip3KglQmQmOVi5/tK+1SYBamGdLajZguwvmdcavbzYYQhxHUxHwye24NgWyjALIh0zh3yL4zREuEyCo+yTbpATnlc872nWEkWbBv9NmHt4vuhZryDu668UknZdYCPSbwNZwTZBmznxW68uX3BB/kkkKQgqlak= X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: 8b1c06d2-439e-41a9-015d-08de1ad51820 X-MS-Exchange-CrossTenant-AuthSource: DM4PR10MB8218.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 03 Nov 2025 12:32:46.9902 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: x6t/+8BMd9EIudiQHYNpA+j9x7NgB7hvXAjZ83SGmUoArTfuL0iqDqEiZVdG/Nqihwph8V/CLJLzGNBz4MIEIMxRMtmZPf3ld25QfSNQ9ug= X-MS-Exchange-Transport-CrossTenantHeadersStamped: DS4PR10MB997575 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1121,Hydra:6.1.9,FMLib:17.12.100.49 definitions=2025-11-03_01,2025-11-03_01,2025-10-01_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 phishscore=0 malwarescore=0 mlxlogscore=999 mlxscore=0 adultscore=0 spamscore=0 bulkscore=0 suspectscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2510240000 definitions=main-2511030113 X-Proofpoint-Spam-Details-Enc: AW1haW4tMjUxMTAzMDExMyBTYWx0ZWRfX+6Ox4x/0+bdu PHyuHz4d4xOl0MsiUMhu5xpfVPTRPgzEMmICTNNw01+cT7bRBLfsjhepN0FgbyIaLanC14yo0ZD 8qDYUfNTftsDJLjmdk86vo5wen0R6G1uLtZBXXU61EPhjysmYKT3f4ZpicAT1LRyMfx2eIxwKyh fp/fvWC1kHHfbSYNLn5F7pCMWqPo5JW3vD0BUncKlN5SPcLib7J2CgelJKNuCGDofngI1u5kPJ0 jPLdtnXuWlXarLPDSjUX9eENae+os2IcDQOEOakUx5rEpMqD49/i996BN9P+09yDcaHHbUoWvlc sEzXdNiCCBM0sWixkzwC7pDNObuAqXESYj3muVZYeaSKV55GPLIiVMaZWlTAlv9TTePzBYjralr 2x4x5YT6P6y4ws3yJdH7oI4u4SXYagJsK9Fico0fj69YG8SVm+k= X-Proofpoint-GUID: GM8-RzTcd5bdfsZsIDBuDXt1jfxeO-J0 X-Proofpoint-ORIG-GUID: GM8-RzTcd5bdfsZsIDBuDXt1jfxeO-J0 X-Authority-Analysis: v=2.4 cv=P7U3RyAu c=1 sm=1 tr=0 ts=6908a0f3 b=1 cx=c_pps a=zPCbziy225d3KhSqZt3L1A==:117 a=zPCbziy225d3KhSqZt3L1A==:17 a=6eWqkTHjU83fiwn7nKZWdM+Sl24=:19 a=z/mQ4Ysz8XfWz/Q5cLBRGdckG28=:19 a=lCpzRmAYbLLaTzLvsPZ7Mbvzbb8=:19 a=xqWC_Br6kY4A:10 a=6UeiqGixMTsA:10 a=GoEa3M9JfhUA:10 a=VkNPw1HP01LnGYTKEx00:22 a=yPCof4ZbAAAA:8 a=jXjnEuVU91_iq225JjgA:9 cc=ntf awl=host:12123 Content-Type: text/plain; charset="utf-8" There is simply no need for the hugely confusing concept of 'non-swap' swap entries now we have the concept of leaf entries and relevant leafent_xxx() helpers. Adjust all callers to use these instead and remove non_swap_entry() altogether. No functional change intended. Signed-off-by: Lorenzo Stoakes --- arch/s390/mm/gmap_helpers.c | 18 +++++++++--------- arch/s390/mm/pgtable.c | 12 ++++++------ fs/proc/task_mmu.c | 12 ++++++------ include/linux/swapops.h | 5 ----- mm/filemap.c | 2 +- mm/hmm.c | 16 ++++++++-------- mm/madvise.c | 2 +- mm/memory.c | 36 ++++++++++++++++++------------------ mm/mincore.c | 2 +- mm/userfaultfd.c | 24 ++++++++++++------------ 10 files changed, 62 insertions(+), 67 deletions(-) diff --git a/arch/s390/mm/gmap_helpers.c b/arch/s390/mm/gmap_helpers.c index d4c3c36855e2..0cf856f38ade 100644 --- a/arch/s390/mm/gmap_helpers.c +++ b/arch/s390/mm/gmap_helpers.c @@ -11,27 +11,27 @@ #include #include #include -#include +#include #include #include #include #include =20 /** - * ptep_zap_swap_entry() - discard a swap entry. + * ptep_zap_leaf_entry() - discard a leaf entry. * @mm: the mm - * @entry: the swap entry that needs to be zapped + * @entry: the leaf entry that needs to be zapped * - * Discards the given swap entry. If the swap entry was an actual swap + * Discards the given leaf entry. If the leaf entry was an actual swap * entry (and not a migration entry, for example), the actual swapped * page is also discarded from swap. */ -static void ptep_zap_swap_entry(struct mm_struct *mm, swp_entry_t entry) +static void ptep_zap_leaf_entry(struct mm_struct *mm, leaf_entry_t entry) { - if (!non_swap_entry(entry)) + if (leafent_is_swap(entry)) dec_mm_counter(mm, MM_SWAPENTS); - else if (is_migration_entry(entry)) - dec_mm_counter(mm, mm_counter(pfn_swap_entry_folio(entry))); + else if (leafent_is_migration(entry)) + dec_mm_counter(mm, mm_counter(leafent_to_folio(entry))); free_swap_and_cache(entry); } =20 @@ -66,7 +66,7 @@ void gmap_helper_zap_one_page(struct mm_struct *mm, unsig= ned long vmaddr) preempt_disable(); pgste =3D pgste_get_lock(ptep); =20 - ptep_zap_swap_entry(mm, pte_to_swp_entry(*ptep)); + ptep_zap_leaf_entry(mm, leafent_from_pte(*ptep)); pte_clear(mm, vmaddr, ptep); =20 pgste_set_unlock(ptep, pgste); diff --git a/arch/s390/mm/pgtable.c b/arch/s390/mm/pgtable.c index 0fde20bbc50b..7805c5a3755e 100644 --- a/arch/s390/mm/pgtable.c +++ b/arch/s390/mm/pgtable.c @@ -16,7 +16,7 @@ #include #include #include -#include +#include #include #include #include @@ -683,12 +683,12 @@ void ptep_unshadow_pte(struct mm_struct *mm, unsigned= long saddr, pte_t *ptep) pgste_set_unlock(ptep, pgste); } =20 -static void ptep_zap_swap_entry(struct mm_struct *mm, swp_entry_t entry) +static void ptep_zap_leaf_entry(struct mm_struct *mm, leaf_entry_t entry) { - if (!non_swap_entry(entry)) + if (leafent_is_swap(entry)) dec_mm_counter(mm, MM_SWAPENTS); - else if (is_migration_entry(entry)) { - struct folio *folio =3D pfn_swap_entry_folio(entry); + else if (leafent_is_migration(entry)) { + const struct folio *folio =3D leafent_to_folio(entry); =20 dec_mm_counter(mm, mm_counter(folio)); } @@ -710,7 +710,7 @@ void ptep_zap_unused(struct mm_struct *mm, unsigned lon= g addr, if (!reset && pte_swap(pte) && ((pgstev & _PGSTE_GPS_USAGE_MASK) =3D=3D _PGSTE_GPS_USAGE_UNUSED || (pgstev & _PGSTE_GPS_ZERO))) { - ptep_zap_swap_entry(mm, pte_to_swp_entry(pte)); + ptep_zap_leaf_entry(mm, leafent_from_pte(pte)); pte_clear(mm, addr, ptep); } if (reset) diff --git a/fs/proc/task_mmu.c b/fs/proc/task_mmu.c index 0ccdc21e60e0..1df735d6b938 100644 --- a/fs/proc/task_mmu.c +++ b/fs/proc/task_mmu.c @@ -1020,13 +1020,13 @@ static void smaps_pte_entry(pte_t *pte, unsigned lo= ng addr, } else if (pte_none(ptent)) { smaps_pte_hole_lookup(addr, walk); } else { - swp_entry_t swpent =3D pte_to_swp_entry(ptent); + const leaf_entry_t entry =3D leafent_from_pte(ptent); =20 - if (!non_swap_entry(swpent)) { + if (leafent_is_swap(entry)) { int mapcount; =20 mss->swap +=3D PAGE_SIZE; - mapcount =3D swp_swapcount(swpent); + mapcount =3D swp_swapcount(entry); if (mapcount >=3D 2) { u64 pss_delta =3D (u64)PAGE_SIZE << PSS_SHIFT; =20 @@ -1035,10 +1035,10 @@ static void smaps_pte_entry(pte_t *pte, unsigned lo= ng addr, } else { mss->swap_pss +=3D (u64)PAGE_SIZE << PSS_SHIFT; } - } else if (is_pfn_swap_entry(swpent)) { - if (is_device_private_entry(swpent)) + } else if (leafent_has_pfn(entry)) { + if (leafent_is_device_private(entry)) present =3D true; - page =3D pfn_swap_entry_to_page(swpent); + page =3D leafent_to_page(entry); } } =20 diff --git a/include/linux/swapops.h b/include/linux/swapops.h index 41cfc6d59054..c8e6f927da48 100644 --- a/include/linux/swapops.h +++ b/include/linux/swapops.h @@ -492,10 +492,5 @@ static inline pmd_t swp_entry_to_pmd(swp_entry_t entry) =20 #endif /* CONFIG_ARCH_ENABLE_THP_MIGRATION */ =20 -static inline int non_swap_entry(swp_entry_t entry) -{ - return swp_type(entry) >=3D MAX_SWAPFILES; -} - #endif /* CONFIG_MMU */ #endif /* _LINUX_SWAPOPS_H */ diff --git a/mm/filemap.c b/mm/filemap.c index eb1f994291d8..980ea9f20993 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -4566,7 +4566,7 @@ static void filemap_cachestat(struct address_space *m= apping, swp_entry_t swp =3D radix_to_swp_entry(folio); =20 /* swapin error results in poisoned entry */ - if (non_swap_entry(swp)) + if (!leafent_is_swap(swp)) goto resched; =20 /* diff --git a/mm/hmm.c b/mm/hmm.c index cbcabc48974f..831ef855a55a 100644 --- a/mm/hmm.c +++ b/mm/hmm.c @@ -253,17 +253,17 @@ static int hmm_vma_handle_pte(struct mm_walk *walk, u= nsigned long addr, } =20 if (!pte_present(pte)) { - swp_entry_t entry =3D pte_to_swp_entry(pte); + const leaf_entry_t entry =3D leafent_from_pte(pte); =20 /* * Don't fault in device private pages owned by the caller, * just report the PFN. */ - if (is_device_private_entry(entry) && - page_pgmap(pfn_swap_entry_to_page(entry))->owner =3D=3D + if (leafent_is_device_private(entry) && + page_pgmap(leafent_to_page(entry))->owner =3D=3D range->dev_private_owner) { cpu_flags =3D HMM_PFN_VALID; - if (is_writable_device_private_entry(entry)) + if (leafent_is_device_private_write(entry)) cpu_flags |=3D HMM_PFN_WRITE; new_pfn_flags =3D swp_offset_pfn(entry) | cpu_flags; goto out; @@ -274,16 +274,16 @@ static int hmm_vma_handle_pte(struct mm_walk *walk, u= nsigned long addr, if (!required_fault) goto out; =20 - if (!non_swap_entry(entry)) + if (leafent_is_swap(entry)) goto fault; =20 - if (is_device_private_entry(entry)) + if (leafent_is_device_private(entry)) goto fault; =20 - if (is_device_exclusive_entry(entry)) + if (leafent_is_device_exclusive(entry)) goto fault; =20 - if (is_migration_entry(entry)) { + if (leafent_is_migration(entry)) { pte_unmap(ptep); hmm_vma_walk->last =3D addr; migration_entry_wait(walk->mm, pmdp, addr); diff --git a/mm/madvise.c b/mm/madvise.c index 900f0f29e77b..67bdfcb315b3 100644 --- a/mm/madvise.c +++ b/mm/madvise.c @@ -249,7 +249,7 @@ static void shmem_swapin_range(struct vm_area_struct *v= ma, continue; entry =3D radix_to_swp_entry(folio); /* There might be swapin error entries in shmem mapping. */ - if (non_swap_entry(entry)) + if (!leafent_is_swap(entry)) continue; =20 addr =3D vma->vm_start + diff --git a/mm/memory.c b/mm/memory.c index 1412fc84172d..3d118618bdeb 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -932,7 +932,7 @@ copy_nonpresent_pte(struct mm_struct *dst_mm, struct mm= _struct *src_mm, struct folio *folio; struct page *page; =20 - if (likely(!non_swap_entry(entry))) { + if (likely(leafent_is_swap(entry))) { if (swap_duplicate(entry) < 0) return -EIO; =20 @@ -950,12 +950,12 @@ copy_nonpresent_pte(struct mm_struct *dst_mm, struct = mm_struct *src_mm, set_pte_at(src_mm, addr, src_pte, pte); } rss[MM_SWAPENTS]++; - } else if (is_migration_entry(entry)) { - folio =3D pfn_swap_entry_folio(entry); + } else if (leafent_is_migration(entry)) { + folio =3D leafent_to_folio(entry); =20 rss[mm_counter(folio)]++; =20 - if (!is_readable_migration_entry(entry) && + if (!leafent_is_migration_read(entry) && is_cow_mapping(vm_flags)) { /* * COW mappings require pages in both parent and child @@ -964,15 +964,15 @@ copy_nonpresent_pte(struct mm_struct *dst_mm, struct = mm_struct *src_mm, */ entry =3D make_readable_migration_entry( swp_offset(entry)); - pte =3D swp_entry_to_pte(entry); + pte =3D leafent_to_pte(entry); if (pte_swp_soft_dirty(orig_pte)) pte =3D pte_swp_mksoft_dirty(pte); if (pte_swp_uffd_wp(orig_pte)) pte =3D pte_swp_mkuffd_wp(pte); set_pte_at(src_mm, addr, src_pte, pte); } - } else if (is_device_private_entry(entry)) { - page =3D pfn_swap_entry_to_page(entry); + } else if (leafent_is_device_private(entry)) { + page =3D leafent_to_page(entry); folio =3D page_folio(page); =20 /* @@ -996,7 +996,7 @@ copy_nonpresent_pte(struct mm_struct *dst_mm, struct mm= _struct *src_mm, * when a device driver is involved (you cannot easily * save and restore device driver state). */ - if (is_writable_device_private_entry(entry) && + if (leafent_is_device_private_write(entry) && is_cow_mapping(vm_flags)) { entry =3D make_readable_device_private_entry( swp_offset(entry)); @@ -1005,7 +1005,7 @@ copy_nonpresent_pte(struct mm_struct *dst_mm, struct = mm_struct *src_mm, pte =3D pte_swp_mkuffd_wp(pte); set_pte_at(src_mm, addr, src_pte, pte); } - } else if (is_device_exclusive_entry(entry)) { + } else if (leafent_is_device_exclusive(entry)) { /* * Make device exclusive entries present by restoring the * original entry then copying as for a present pte. Device @@ -4635,7 +4635,7 @@ vm_fault_t do_swap_page(struct vm_fault *vmf) rmap_t rmap_flags =3D RMAP_NONE; bool need_clear_cache =3D false; bool exclusive =3D false; - swp_entry_t entry; + leaf_entry_t entry; pte_t pte; vm_fault_t ret =3D 0; void *shadow =3D NULL; @@ -4647,15 +4647,15 @@ vm_fault_t do_swap_page(struct vm_fault *vmf) if (!pte_unmap_same(vmf)) goto out; =20 - entry =3D pte_to_swp_entry(vmf->orig_pte); - if (unlikely(non_swap_entry(entry))) { - if (is_migration_entry(entry)) { + entry =3D leafent_from_pte(vmf->orig_pte); + if (unlikely(!leafent_is_swap(entry))) { + if (leafent_is_migration(entry)) { migration_entry_wait(vma->vm_mm, vmf->pmd, vmf->address); - } else if (is_device_exclusive_entry(entry)) { - vmf->page =3D pfn_swap_entry_to_page(entry); + } else if (leafent_is_device_exclusive(entry)) { + vmf->page =3D leafent_to_page(entry); ret =3D remove_device_exclusive_entry(vmf); - } else if (is_device_private_entry(entry)) { + } else if (leafent_is_device_private(entry)) { if (vmf->flags & FAULT_FLAG_VMA_LOCK) { /* * migrate_to_ram is not yet ready to operate @@ -4666,7 +4666,7 @@ vm_fault_t do_swap_page(struct vm_fault *vmf) goto out; } =20 - vmf->page =3D pfn_swap_entry_to_page(entry); + vmf->page =3D leafent_to_page(entry); vmf->pte =3D pte_offset_map_lock(vma->vm_mm, vmf->pmd, vmf->address, &vmf->ptl); if (unlikely(!vmf->pte || @@ -4690,7 +4690,7 @@ vm_fault_t do_swap_page(struct vm_fault *vmf) } else { pte_unmap_unlock(vmf->pte, vmf->ptl); } - } else if (is_hwpoison_entry(entry)) { + } else if (leafent_is_hwpoison(entry)) { ret =3D VM_FAULT_HWPOISON; } else if (leafent_is_marker(entry)) { ret =3D handle_pte_marker(vmf); diff --git a/mm/mincore.c b/mm/mincore.c index e77c5bc88fc7..a1f48df5564e 100644 --- a/mm/mincore.c +++ b/mm/mincore.c @@ -74,7 +74,7 @@ static unsigned char mincore_swap(swp_entry_t entry, bool= shmem) * absent. Page table may contain migration or hwpoison * entries which are always uptodate. */ - if (non_swap_entry(entry)) + if (!leafent_is_swap(entry)) return !shmem; =20 /* diff --git a/mm/userfaultfd.c b/mm/userfaultfd.c index 055ec1050776..d11fa8eeaef2 100644 --- a/mm/userfaultfd.c +++ b/mm/userfaultfd.c @@ -1256,7 +1256,6 @@ static long move_pages_ptes(struct mm_struct *mm, pmd= _t *dst_pmd, pmd_t *src_pmd unsigned long dst_addr, unsigned long src_addr, unsigned long len, __u64 mode) { - swp_entry_t entry; struct swap_info_struct *si =3D NULL; pte_t orig_src_pte, orig_dst_pte; pte_t src_folio_pte; @@ -1430,19 +1429,20 @@ static long move_pages_ptes(struct mm_struct *mm, p= md_t *dst_pmd, pmd_t *src_pmd orig_dst_pte, orig_src_pte, dst_pmd, dst_pmdval, dst_ptl, src_ptl, &src_folio, len); - } else { + } else { /* !pte_present() */ struct folio *folio =3D NULL; + const leaf_entry_t entry =3D leafent_from_pte(orig_src_pte); =20 - entry =3D pte_to_swp_entry(orig_src_pte); - if (non_swap_entry(entry)) { - if (is_migration_entry(entry)) { - pte_unmap(src_pte); - pte_unmap(dst_pte); - src_pte =3D dst_pte =3D NULL; - migration_entry_wait(mm, src_pmd, src_addr); - ret =3D -EAGAIN; - } else - ret =3D -EFAULT; + if (leafent_is_migration(entry)) { + pte_unmap(src_pte); + pte_unmap(dst_pte); + src_pte =3D dst_pte =3D NULL; + migration_entry_wait(mm, src_pmd, src_addr); + + ret =3D -EAGAIN; + goto out; + } else if (!leafent_is_swap(entry)) { + ret =3D -EFAULT; goto out; } =20 --=20 2.51.0 From nobody Sat Feb 7 17:56:00 2026 Received: from mx0a-00069f02.pphosted.com (mx0a-00069f02.pphosted.com [205.220.165.32]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2714E313529; Mon, 3 Nov 2025 12:35:11 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=205.220.165.32 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1762173314; cv=fail; b=ADWVlBbGR6kP2Myti467WFrRZ4TxpU1pcHeFpEC0/IxZr7wTW4asI6fJ8TCEFi5Ht4uKnX+pPi6omWfMJQlfzh+oh1iVkC6lLT670MAhy4fvxVcvJakBjebgMPJRyuGwVVxl1C+M81roI2IBr5gmyTAR/IwbPnIu2Uh+js55bAA= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1762173314; c=relaxed/simple; bh=Qt6ov0F68f9RJQJuPoqHMwbTsBmZ6ZAwdCIuCVy+P0s=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: Content-Type:MIME-Version; b=B3i57JxpwHPLkzirFR+ya0Rzv0Jjg98KYigL+G8RCZUN533nI0smsPK5Vlip/b0CP9kxii6lWdBKDK7WcomU3tlimKbaJkCbv/6mEopYVVX0+KTJI1g/5Vwr7IcZStQ+gNEkmxh+VpEmKbYnFhKDcjigmKp5ng5j4giNkJVc4ec= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=oracle.com; spf=pass smtp.mailfrom=oracle.com; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b=iDd2suBZ; dkim=pass (1024-bit key) header.d=oracle.onmicrosoft.com header.i=@oracle.onmicrosoft.com header.b=bAWX2+Ng; arc=fail smtp.client-ip=205.220.165.32 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=oracle.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=oracle.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b="iDd2suBZ"; dkim=pass (1024-bit key) header.d=oracle.onmicrosoft.com header.i=@oracle.onmicrosoft.com header.b="bAWX2+Ng" Received: from pps.filterd (m0246629.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 5A3CTm5m011561; Mon, 3 Nov 2025 12:33:21 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=cc :content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s= corp-2025-04-25; bh=n1elD2KxJdX09ELZo0YYk/oZImFOlJN2Ko3RGZS4bMA=; b= iDd2suBZTrfKwb6+hgZ2JvlttyUFsAy5iha08/tVQ82Qly90voFVRdFyLSJclbN4 M/cEgMjzPOas8jWi0ZgYEM82rMbIlSVpGYc8T87P2BtYTpSgub+nMAn+0vrE1oT4 Zc7FmIIDtCr45GJNT7MS8WbIgUALTFWeO1m7TjC8Slv/DkH1xy3Xvhbp11UPHYeY 1MmCug/hTiTRxzADu8rpollUei3cCz23noq+NlvlanmDSgy+2XCzAEhP9bgCfduy HRd+Cn5G+NG+uCcHxQLkGZVZLEnJb9FSVUtIf2zzAR9yvnGbIB2e3Q9S49eLZwvk JAe0Cojxfsv81cXorDR3uw== Received: from phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta02.appoci.oracle.com [147.154.114.232]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 4a6vep009g-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 03 Nov 2025 12:33:21 +0000 (GMT) Received: from pps.filterd (phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (8.18.1.2/8.18.1.2) with ESMTP id 5A3Bx7kP021025; Mon, 3 Nov 2025 12:33:20 GMT Received: from sj2pr03cu001.outbound.protection.outlook.com (mail-westusazon11012047.outbound.protection.outlook.com [52.101.43.47]) by phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTPS id 4a58n83ev0-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 03 Nov 2025 12:33:20 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=yKpMCBXui5an7c8xyNPRbSJuOw5WMpSY6ZJ/Ba64Qh4CysiYSTtWqIaRtFYZi/uMDoC/R+0uIDQmMrV08uu+8yH875wnXw7Miv2f2RgsSIkFXm55kMmmSkUT5w3OAniaQz0KKwHzpzIp9FU/IqFXnn1aD7oB8x1G3MJmqyseDqEjCLBNbKSVnTPJaJsmFdEDN4J6obUWSAX7QSNYpUyVqs4WXLrCt7sPK/e1rz+O0C48/jmEnb6TS+xT3m3lW8rmuxDuSGTJ+meo1immd9THrFvpya9/9owOK0J28PUvZsybfJwy7Vo76Hw8SVXYTMzO5oggp7O+L8GyFI/XEaDSAQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=n1elD2KxJdX09ELZo0YYk/oZImFOlJN2Ko3RGZS4bMA=; b=k8gNxTa5wAYoOjBJwapaUsJfJliDyBU+Gb00+Bv+Tz1557GG7/0xonybsfSpSkEOEZjUeP251veNCxN4xPiCaa7K7RTUvmgoZvvnLAhBfqDW0cMqdDIhwnmg+NmdL2tWKd/NJHTu8WOHjgaoRaw28D2Cp6UKNLsDuTQhMHm+adI7kOrjThKkduO579EIdoFnja76ZRZKo3QfdHn4mbFvLl+I/SmOmntEuXqPOegutNiDY9ktYqE0V5oiksD+JmDQASDlpHBS13RhCbcFAmlnylLkt5VJ6krRF0ofFryhYUzPJfMN9v4O15c7cNU/l3ECvaJ/xgQHMfUXyEl0JqIpvg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=n1elD2KxJdX09ELZo0YYk/oZImFOlJN2Ko3RGZS4bMA=; b=bAWX2+NgB1v1MARfgHS8p+MYk6WwXuoDeyh1r9pXgV9AgIgn6GYSH7Dkjsjk6WrHQgXTVxW/47luyJTePathMO+dfPbqm12erg3h/OIUWQ2UvoC6IfHAgza1S5W9mFRvZY6qtFF3iWJnOaiKIZNE1KtZfuKx/OZl4O7MpwGSdmw= Received: from DM4PR10MB8218.namprd10.prod.outlook.com (2603:10b6:8:1cc::16) by DS4PR10MB997575.namprd10.prod.outlook.com (2603:10b6:8:31e::5) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9275.16; Mon, 3 Nov 2025 12:32:50 +0000 Received: from DM4PR10MB8218.namprd10.prod.outlook.com ([fe80::2650:55cf:2816:5f2]) by DM4PR10MB8218.namprd10.prod.outlook.com ([fe80::2650:55cf:2816:5f2%2]) with mapi id 15.20.9253.018; Mon, 3 Nov 2025 12:32:50 +0000 From: Lorenzo Stoakes To: Andrew Morton Cc: Christian Borntraeger , Janosch Frank , Claudio Imbrenda , David Hildenbrand , Alexander Gordeev , Gerald Schaefer , Heiko Carstens , Vasily Gorbik , Sven Schnelle , Peter Xu , Alexander Viro , Christian Brauner , Jan Kara , Arnd Bergmann , Zi Yan , Baolin Wang , "Liam R . Howlett" , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Lance Yang , Muchun Song , Oscar Salvador , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Matthew Brost , Joshua Hahn , Rakie Kim , Byungchul Park , Gregory Price , Ying Huang , Alistair Popple , Axel Rasmussen , Yuanchu Xie , Wei Xu , Kemeng Shi , Kairui Song , Nhat Pham , Baoquan He , Chris Li , SeongJae Park , Matthew Wilcox , Jason Gunthorpe , Leon Romanovsky , Xu Xin , Chengming Zhou , Jann Horn , Miaohe Lin , Naoya Horiguchi , Pedro Falcato , Pasha Tatashin , Rik van Riel , Harry Yoo , Hugh Dickins , linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-s390@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, linux-arch@vger.kernel.org, damon@lists.linux.dev Subject: [PATCH 14/16] mm: remove is_hugetlb_entry_[migration, hwpoisoned]() Date: Mon, 3 Nov 2025 12:31:55 +0000 Message-ID: X-Mailer: git-send-email 2.51.0 In-Reply-To: References: Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: LO4P265CA0080.GBRP265.PROD.OUTLOOK.COM (2603:10a6:600:2bd::10) To DM4PR10MB8218.namprd10.prod.outlook.com (2603:10b6:8:1cc::16) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DM4PR10MB8218:EE_|DS4PR10MB997575:EE_ X-MS-Office365-Filtering-Correlation-Id: 22141c06-f34f-44db-a4aa-08de1ad51a4b X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|376014|7416014|366016|1800799024; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?QoSA++OxI+fjGmPxu1J+QnCOOmhfIK5WWllSPCnU4zJ9xLGqoKsO/w968hnv?= =?us-ascii?Q?vPHN7UdudLGyKGPGDFSbG0WfRF5pmkyZPocAdnhx8Zj6wRQHk6Pb8/eNjJ4y?= =?us-ascii?Q?LKmtdFrbcgrlLwAUCY+uIiFJwUlqH4510B9Lg6p49W/MaBpKdECeMvg5msop?= =?us-ascii?Q?GxoptDTl+/w/vHVeZju2edU5rQniiOJdRJZHCy/J32DZE43rwq8ZFHECaJYY?= =?us-ascii?Q?9W5gOUznqAxbAFdcCvFH0yIe0ZR1VkUPDdChojvpq0fSQKQyTOll9+cwb4LI?= =?us-ascii?Q?QfJAdYAsGkje2Odp7F0z+nVVgwqypA4tgeCrp2GN9Fh2k8NyqjFCDWlUBajz?= =?us-ascii?Q?xX1hiq6gU5rkKzImJCie3YHpOdLtiVU3sCwn9b+fB8cT8OI+Pa1LSDlH4Z85?= =?us-ascii?Q?SLbIyjQRyXjvxbUZLzXFjFkUl3WINabBy79FM1LA4bI7HwMUBiFU97KjtMny?= =?us-ascii?Q?O8G5fWbFcR4RrNtJwy5ERcIvZHLWjPem4y/jTyyJJ9bBB7Cqr1MWIko8IDQm?= =?us-ascii?Q?Xp9jHmzwop9zNL7pZcsB8h0WowZArOYIMEq7Zr1kA6PAwa49d35yfMmORdIM?= =?us-ascii?Q?SPffoP7+n/lu+I0wPoarswOwtEqebKvkjBU21HgSc1qGirgrppQlsOTdftzI?= =?us-ascii?Q?uGx7yLWEXczfOVwoFqjLRc6hZmKnZKLlEujbPUEAnc5f7BdQ5p+ea+QtZCMS?= =?us-ascii?Q?FGLiJl3pUTgfb2B3ct1mm5KXT8PZL3ul9vuwEW1nxlKQCFbsQGxrowG+rObN?= =?us-ascii?Q?96ymOPcS+RvR247ZZtjU79rsp+0Pz0d//dFpOcoQ+5Q/G+GESqOhtFOXYWzQ?= =?us-ascii?Q?0YN+Wqalk7nMBMZVQ6ons4cdSnRDuLiKZqsTQ0HCIQwIpUKP3SwXiYAqAMza?= =?us-ascii?Q?s9qgUu1lamqMEMhwWhWrmb580s6nomJ3+gZcJ1tzoyDiFaXFhM44qt27rqR8?= =?us-ascii?Q?xqwIYeXcSYjIf3jy0Tbs4/pDyeWbsFu/UNEInWhbhGGd79hcW5ZKETgxXhM3?= =?us-ascii?Q?akg/HmvxTmugG/yJSyc4vzXm5NjjQb1IfV9S2Li5prmoSRpQfvP/6D1jXI7P?= =?us-ascii?Q?KYedsmpXCvucbhuTjrrfMrNXpvXXP598FwWgjR1dLO5StkhFeC5U4/gH69PQ?= =?us-ascii?Q?EcenkIRKKEDN64iuVyToAQs5jiCJyDl9kpfV0WuMeUEdRGSJ/62w7G0de4Vv?= =?us-ascii?Q?r/+NCs0Ut6gNa2wo1py/MB0Vre+VcCopDxnabtn9wZl8zEOT31fvmYahgDfJ?= =?us-ascii?Q?ZoGOXIqhqV7BypIU/qvpWcclXnS8aUljRaF+8dO35y690Ua+tGGfbBnv2eFe?= =?us-ascii?Q?a/K3zVSpU7OM/xqCkoQd+yy7ywMg6l5dg/5fIkA918q5T7o3h3b3I5kZW4Y2?= =?us-ascii?Q?cmUpDXHOb54zq++7QU0zsV5jbG9nJyx13Py4liOfJnQR/m/pHp6YychSnAco?= =?us-ascii?Q?0TKn4QG3yLZx9M3Uobodz7pKckyWx/sl?= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DM4PR10MB8218.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(376014)(7416014)(366016)(1800799024);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?Rv34q8Mq9CdsFw7/x84cKbGv8tkvBAGLzpLohkUcglpQMHqJjmLSiKJecLyD?= =?us-ascii?Q?C1NJdawUnW4pd9i79xjBOz4o9sLlvuaFE/16fxJfQMWJ5yNi2Zgfio3luiHq?= =?us-ascii?Q?SFjEl/9Vq0Fc9V4NcCwuV5NQ37vfEBoUxGLzkWLpvYj7OQyw+qyjnTPXgppz?= =?us-ascii?Q?/CHnXYMoDPkjvosFntz9GNDyW6lXbVPmxvmul0giRNS9kuDbUAvcwCUHfjQT?= =?us-ascii?Q?pZLZy1NqJwImxR3WacNs5TI9d2TWoGbZwQHjyTZ7TIAa/IBc2UUGUwNKkFza?= =?us-ascii?Q?qbG2hC70Lovtx1W4m3uM4wsYCLE5ownBPiVNo2mXbCejHMqNRAXfInqrmV/C?= =?us-ascii?Q?CwMQQuiZON5/Hrry0C+rifAdaShIB1vZRrJdz/0RQdO3+kqVK/LBL022l6RW?= =?us-ascii?Q?U7lCP1aFIRNP6nIeLL3LbKr9L/hw76WUbY61r8AmrYcbqDvfjXpJtTMnNqEz?= =?us-ascii?Q?NlZYp1cBPSdrBYDzoSQN5qBCjygV4ro63dAzRskk3mnbTSqKY8KcLH1J3A9/?= =?us-ascii?Q?+NvBA4pTL9GDZKdZTU5V6BMR92FvfL1mND5/9RFbMHoEhRpg+Uq0xl4zqkP3?= =?us-ascii?Q?JOYQTVXe8iMC8oRVw/06fpDqAqY/ZTRISBoDE7gCVPzXOpD6qH5H1HhUsf6X?= =?us-ascii?Q?NIhpE8cBr1u2HFvsiN31UBdc0uAvOuN2IC6zhbBaBrMG4oojw+6yZ+m9pXFl?= =?us-ascii?Q?y1wtU1kQ2a1ouuYsYCk7tv8cUMf2wooljX1NC8FgNXHGKXcGH8PmKVNFR/QJ?= =?us-ascii?Q?sUs1gRPRNrh4qVSSFrpjaQKCpjASU1OwBQupkLMcDYajyXtV0hUNWQofr0J2?= =?us-ascii?Q?XjkZprImO/u8j6gOZQtBbjRQ6jDTAC9hGildfMLxAMsIqh7ofD4+WtwaDTv9?= =?us-ascii?Q?qtrtBE497X4JIvlknBrxC7toY4MAon7RpSqglGeGrBxCMrkM6sgkvl9xVgxJ?= =?us-ascii?Q?ArLH+Pi5pZYdfBb3xEBVP8HjD79nHLXC8UdBzh/5XdCtknViuPy5iIFoOzIP?= =?us-ascii?Q?2zT7El2OU3vjEPGZ/7LdbGjrCJMB/RXbeCaVuK2fRBGCVEKbWyrvH32bCg+Z?= =?us-ascii?Q?5DhNb1XfMh/uxbomz1aiXbLVG//ioB5Sn6cixFxiDEqyq9PACkaEt997vOtu?= =?us-ascii?Q?q8kaTNIe2dZuAZy8TEz8zbw7+sblLnNCq92KKnS6/R39KErfQAB0f2d501p/?= =?us-ascii?Q?sPhAhc+iCPgwwu2MvUxE0DpHADje9W3NF5qQH0Ow4cMpolVNQIjKc6A/c9V1?= =?us-ascii?Q?WdIvYVyOgC8PB4yDEG+YtRnGnFU1QAZnJ7KUB6DH/1kXNB0KYwnTg8Cz3+bF?= =?us-ascii?Q?m6LiPXHQW8OdVLk/lVGkAkhY6P/GnSV8nZKgxSXJ08pFi+IeCS28d6w5wKVS?= =?us-ascii?Q?HlWHRwX2hEJMKj7nFe8qeNsemxQ2aezZayTgZEwxdkc/tuTPQ7WKPuz8DSTl?= =?us-ascii?Q?QpsW4YI4G0HQzav7X5+RbYzhWeWSK36A37X+hLldK/1C9+3h1OMxPb5zcyNP?= =?us-ascii?Q?vlqwsUHJxN5BtkxlCQjIA6OII/O9oLuce/efNV7xHSAty+LUHvMArsQN+pjj?= =?us-ascii?Q?On8b5HSMYmBop2ruvFKtR38gioZ8QbbgEI5nDvKwIXA0CEyvpNvDyF3AE815?= =?us-ascii?Q?Ew=3D=3D?= X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: sDpSDqGNN409ISE1jG6okA8zwSMwZFK10WyElYW8FWyM+WSMNqcSPkE/5GOu6YxpoGj2LWVba1ZdJXVZng1paNxQaVsbTT6JSN3LVv8TvcEjZ0V5/CO7606i9Ww+93KxnwSp/OzbJVnHVdLQOtnWj8l1jKyyluWFZlCSfHxe4ysEHqxhPeiCjcr1IarW+Dl/fxbaqULZEdtyrvd9H9et2CP9dCZjL/uZbjwS2SM2AXVbc8jhyl/SlTeNE4pDd3uqRwdR77qpeTIG22UBSia2sMKUixRi7HEC2bC4cAlvXkuIskj3awpOkA9aVDwQNfET5slnpSixqdQb9Q5V6WJYTCPtuqH0aDvcW2mT6CNQgpT8LL6HJwM5WudefN8fVwSHqD1NyodEQ9wltYqoOYUAN9742MPQWWM8c6M6AXYEAnAExmhTqXU7CcQXBDKuwr4kiUz2PR+PyPnaxQDgFJAosLxcc64MEz9lFE8jl1Fhs2Nfw2k0oyGs27Yi+ICScMGgnPXPAYLO/yRZiOnHZ6YghbQNdYT6fsVCVczuMX5UA3mBt+6LzOD8tYdSAvzxjlE6McZO/6Vc00jjk4ARspycUEX1+ly52cZ+VdMXwSsdrhg= X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: 22141c06-f34f-44db-a4aa-08de1ad51a4b X-MS-Exchange-CrossTenant-AuthSource: DM4PR10MB8218.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 03 Nov 2025 12:32:50.6332 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: UBwjQvShYdZWNAf1Vj9M+QcEY1lwnetWH1DU1iljJwTEZvzVWH4WVfcYyOoau6r72Ap0JCvDN1eW0QLYTeIVK8Yt+v8BBhW1FkuWDh02a0w= X-MS-Exchange-Transport-CrossTenantHeadersStamped: DS4PR10MB997575 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1121,Hydra:6.1.9,FMLib:17.12.100.49 definitions=2025-11-03_01,2025-11-03_01,2025-10-01_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 phishscore=0 bulkscore=0 malwarescore=0 spamscore=0 suspectscore=0 mlxscore=0 adultscore=0 mlxlogscore=999 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2510240000 definitions=main-2511030114 X-Proofpoint-GUID: 7r_xqn5qy8IBPfOFjJGMJRVTSGn4uU_6 X-Proofpoint-ORIG-GUID: 7r_xqn5qy8IBPfOFjJGMJRVTSGn4uU_6 X-Authority-Analysis: v=2.4 cv=B9m0EetM c=1 sm=1 tr=0 ts=6908a111 cx=c_pps a=OOZaFjgC48PWsiFpTAqLcw==:117 a=OOZaFjgC48PWsiFpTAqLcw==:17 a=6eWqkTHjU83fiwn7nKZWdM+Sl24=:19 a=z/mQ4Ysz8XfWz/Q5cLBRGdckG28=:19 a=lCpzRmAYbLLaTzLvsPZ7Mbvzbb8=:19 a=xqWC_Br6kY4A:10 a=6UeiqGixMTsA:10 a=GoEa3M9JfhUA:10 a=VkNPw1HP01LnGYTKEx00:22 a=yPCof4ZbAAAA:8 a=gaN7zjf0O2erDMybN6AA:9 a=UhEZJTgQB8St2RibIkdl:22 a=Z5ABNNGmrOfJ6cZ5bIyy:22 a=QOGEsqRv6VhmHaoFNykA:22 X-Proofpoint-Spam-Details-Enc: AW1haW4tMjUxMTAzMDExNCBTYWx0ZWRfX/PDaucsctCOb gAGIlnqRM/U02y4ZrjVl9jIb09EORSGuIKQItPve2VErczqHitMRJF9XRQrPKDsQAb2eXc8lfKr bgfBFzlcBOYKxEPUdxniCxR+Owvj93ZPca/bgSDIFpsgVwpw3Lo0V/KUQdW5lFfsC/zS15Y9Jec vLhc4TeHAJ2PMY0xWyBH1JZwMIqLeEabCwx7taCqxCvZdCOg6n+YxF6vushwn/bcYoU4/36H9IK VsqViSbkVT/VxvOVWU/YNxeufVRB03SLinYO8wuNthdtVsWGiw3LokfyGdkWyWA2ymyVzHtiP2X 0k4C4gfEnEx6M2oKcmdWwBjl9/dsv5Ucpaxek9JEQ/NQbv7iCLUmS/iRSS/VS5tVGu4PSH7HtMm MaoXz1/vrv5BjQNqRmcdPvPKYhxvOQ== Content-Type: text/plain; charset="utf-8" We do not need to have explicit helper functions for these, it adds a level of confusion and indirection when we can simply use leaf entry logic here instead and spell out the special huge_pte_none() case we must consider. No functional change intended. Signed-off-by: Lorenzo Stoakes --- fs/proc/task_mmu.c | 19 +++++---- include/linux/hugetlb.h | 2 - mm/hugetlb.c | 91 +++++++++++++++++------------------------ mm/mempolicy.c | 17 +++++--- mm/migrate.c | 15 +++++-- 5 files changed, 69 insertions(+), 75 deletions(-) diff --git a/fs/proc/task_mmu.c b/fs/proc/task_mmu.c index 1df735d6b938..82532c069039 100644 --- a/fs/proc/task_mmu.c +++ b/fs/proc/task_mmu.c @@ -2499,22 +2499,23 @@ static void make_uffd_wp_huge_pte(struct vm_area_st= ruct *vma, unsigned long addr, pte_t *ptep, pte_t ptent) { - unsigned long psize; + const unsigned long psize =3D huge_page_size(hstate_vma(vma)); + leaf_entry_t entry; =20 - if (is_hugetlb_entry_hwpoisoned(ptent) || pte_is_marker(ptent)) - return; + if (huge_pte_none(ptent)) + set_huge_pte_at(vma->vm_mm, addr, ptep, + make_pte_marker(PTE_MARKER_UFFD_WP), psize); =20 - psize =3D huge_page_size(hstate_vma(vma)); + entry =3D leafent_from_pte(ptent); + if (leafent_is_hwpoison(entry) || leafent_is_marker(entry)) + return; =20 - if (is_hugetlb_entry_migration(ptent)) + if (leafent_is_migration(entry)) set_huge_pte_at(vma->vm_mm, addr, ptep, pte_swp_mkuffd_wp(ptent), psize); - else if (!huge_pte_none(ptent)) + else huge_ptep_modify_prot_commit(vma, addr, ptep, ptent, huge_pte_mkuffd_wp(ptent)); - else - set_huge_pte_at(vma->vm_mm, addr, ptep, - make_pte_marker(PTE_MARKER_UFFD_WP), psize); } #endif /* CONFIG_HUGETLB_PAGE */ =20 diff --git a/include/linux/hugetlb.h b/include/linux/hugetlb.h index 2387513d6ae5..457d48ac7bcd 100644 --- a/include/linux/hugetlb.h +++ b/include/linux/hugetlb.h @@ -274,8 +274,6 @@ void hugetlb_vma_lock_release(struct kref *kref); long hugetlb_change_protection(struct vm_area_struct *vma, unsigned long address, unsigned long end, pgprot_t newprot, unsigned long cp_flags); -bool is_hugetlb_entry_migration(pte_t pte); -bool is_hugetlb_entry_hwpoisoned(pte_t pte); void hugetlb_unshare_all_pmds(struct vm_area_struct *vma); void fixup_hugetlb_reservations(struct vm_area_struct *vma); void hugetlb_split(struct vm_area_struct *vma, unsigned long addr); diff --git a/mm/hugetlb.c b/mm/hugetlb.c index 2e797abdee04..6c483ecd496f 100644 --- a/mm/hugetlb.c +++ b/mm/hugetlb.c @@ -5552,32 +5552,6 @@ static void set_huge_ptep_maybe_writable(struct vm_a= rea_struct *vma, set_huge_ptep_writable(vma, address, ptep); } =20 -bool is_hugetlb_entry_migration(pte_t pte) -{ - swp_entry_t swp; - - if (huge_pte_none(pte) || pte_present(pte)) - return false; - swp =3D pte_to_swp_entry(pte); - if (is_migration_entry(swp)) - return true; - else - return false; -} - -bool is_hugetlb_entry_hwpoisoned(pte_t pte) -{ - swp_entry_t swp; - - if (huge_pte_none(pte) || pte_present(pte)) - return false; - swp =3D pte_to_swp_entry(pte); - if (is_hwpoison_entry(swp)) - return true; - else - return false; -} - static void hugetlb_install_folio(struct vm_area_struct *vma, pte_t *ptep, unsigned lo= ng addr, struct folio *new_folio, pte_t old, unsigned long sz) @@ -5606,6 +5580,7 @@ int copy_hugetlb_page_range(struct mm_struct *dst, st= ruct mm_struct *src, unsigned long npages =3D pages_per_huge_page(h); struct mmu_notifier_range range; unsigned long last_addr_mask; + leaf_entry_t leafent; int ret =3D 0; =20 if (cow) { @@ -5653,16 +5628,16 @@ int copy_hugetlb_page_range(struct mm_struct *dst, = struct mm_struct *src, entry =3D huge_ptep_get(src_vma->vm_mm, addr, src_pte); again: if (huge_pte_none(entry)) { - /* - * Skip if src entry none. - */ - ; - } else if (unlikely(is_hugetlb_entry_hwpoisoned(entry))) { + /* Skip if src entry none. */ + goto next; + } + + leafent =3D leafent_from_pte(entry); + if (unlikely(leafent_is_hwpoison(leafent))) { if (!userfaultfd_wp(dst_vma)) entry =3D huge_pte_clear_uffd_wp(entry); set_huge_pte_at(dst, addr, dst_pte, entry, sz); - } else if (unlikely(is_hugetlb_entry_migration(entry))) { - leaf_entry_t leafent =3D leafent_from_pte(entry); + } else if (unlikely(leafent_is_migration(leafent))) { bool uffd_wp =3D pte_swp_uffd_wp(entry); =20 if (!is_readable_migration_entry(leafent) && cow) { @@ -5681,7 +5656,6 @@ int copy_hugetlb_page_range(struct mm_struct *dst, st= ruct mm_struct *src, entry =3D huge_pte_clear_uffd_wp(entry); set_huge_pte_at(dst, addr, dst_pte, entry, sz); } else if (unlikely(pte_is_marker(entry))) { - const leaf_entry_t leafent =3D leafent_from_pte(entry); const pte_marker marker =3D copy_pte_marker(leafent, dst_vma); =20 if (marker) @@ -5739,9 +5713,7 @@ int copy_hugetlb_page_range(struct mm_struct *dst, st= ruct mm_struct *src, } hugetlb_install_folio(dst_vma, dst_pte, addr, new_folio, src_pte_old, sz); - spin_unlock(src_ptl); - spin_unlock(dst_ptl); - continue; + goto next; } =20 if (cow) { @@ -5762,6 +5734,8 @@ int copy_hugetlb_page_range(struct mm_struct *dst, st= ruct mm_struct *src, set_huge_pte_at(dst, addr, dst_pte, entry, sz); hugetlb_count_add(npages, dst); } + +next: spin_unlock(src_ptl); spin_unlock(dst_ptl); } @@ -6770,8 +6744,10 @@ vm_fault_t hugetlb_fault(struct mm_struct *mm, struc= t vm_area_struct *vma, ret =3D 0; =20 /* Not present, either a migration or a hwpoisoned entry */ - if (!pte_present(vmf.orig_pte)) { - if (is_hugetlb_entry_migration(vmf.orig_pte)) { + if (!pte_present(vmf.orig_pte) && !huge_pte_none(vmf.orig_pte)) { + const leaf_entry_t leafent =3D leafent_from_pte(vmf.orig_pte); + + if (leafent_is_migration(leafent)) { /* * Release the hugetlb fault lock now, but retain * the vma lock, because it is needed to guard the @@ -6782,9 +6758,12 @@ vm_fault_t hugetlb_fault(struct mm_struct *mm, struc= t vm_area_struct *vma, mutex_unlock(&hugetlb_fault_mutex_table[hash]); migration_entry_wait_huge(vma, vmf.address, vmf.pte); return 0; - } else if (is_hugetlb_entry_hwpoisoned(vmf.orig_pte)) + } + if (leafent_is_hwpoison(leafent)) { ret =3D VM_FAULT_HWPOISON_LARGE | VM_FAULT_SET_HINDEX(hstate_index(h)); + } + goto out_mutex; } =20 @@ -7166,7 +7145,9 @@ long hugetlb_change_protection(struct vm_area_struct = *vma, i_mmap_lock_write(vma->vm_file->f_mapping); last_addr_mask =3D hugetlb_mask_last_page(h); for (; address < end; address +=3D psize) { + leaf_entry_t entry; spinlock_t *ptl; + ptep =3D hugetlb_walk(vma, address, psize); if (!ptep) { if (!uffd_wp) { @@ -7198,15 +7179,23 @@ long hugetlb_change_protection(struct vm_area_struc= t *vma, continue; } pte =3D huge_ptep_get(mm, address, ptep); - if (unlikely(is_hugetlb_entry_hwpoisoned(pte))) { - /* Nothing to do. */ - } else if (unlikely(is_hugetlb_entry_migration(pte))) { - leaf_entry_t entry =3D leafent_from_pte(pte); + if (huge_pte_none(pte)) { + if (unlikely(uffd_wp)) + /* Safe to modify directly (none->non-present). */ + set_huge_pte_at(mm, address, ptep, + make_pte_marker(PTE_MARKER_UFFD_WP), + psize); + goto next; + } =20 + entry =3D leafent_from_pte(pte); + if (unlikely(leafent_is_hwpoison(entry))) { + /* Nothing to do. */ + } else if (unlikely(leafent_is_migration(entry))) { struct folio *folio =3D leafent_to_folio(entry); pte_t newpte =3D pte; =20 - if (is_writable_migration_entry(entry)) { + if (leafent_is_migration_write(entry)) { if (folio_test_anon(folio)) entry =3D make_readable_exclusive_migration_entry( swp_offset(entry)); @@ -7233,7 +7222,7 @@ long hugetlb_change_protection(struct vm_area_struct = *vma, if (pte_is_uffd_wp_marker(pte) && uffd_wp_resolve) /* Safe to modify directly (non-present->none). */ huge_pte_clear(mm, address, ptep, psize); - } else if (!huge_pte_none(pte)) { + } else { pte_t old_pte; unsigned int shift =3D huge_page_shift(hstate_vma(vma)); =20 @@ -7246,16 +7235,10 @@ long hugetlb_change_protection(struct vm_area_struc= t *vma, pte =3D huge_pte_clear_uffd_wp(pte); huge_ptep_modify_prot_commit(vma, address, ptep, old_pte, pte); pages++; - } else { - /* None pte */ - if (unlikely(uffd_wp)) - /* Safe to modify directly (none->non-present). */ - set_huge_pte_at(mm, address, ptep, - make_pte_marker(PTE_MARKER_UFFD_WP), - psize); } - spin_unlock(ptl); =20 +next: + spin_unlock(ptl); cond_resched(); } /* diff --git a/mm/mempolicy.c b/mm/mempolicy.c index 01c3b98f87a6..f5b05754e6d5 100644 --- a/mm/mempolicy.c +++ b/mm/mempolicy.c @@ -768,16 +768,21 @@ static int queue_folios_hugetlb(pte_t *pte, unsigned = long hmask, unsigned long flags =3D qp->flags; struct folio *folio; spinlock_t *ptl; - pte_t entry; + pte_t ptep; =20 ptl =3D huge_pte_lock(hstate_vma(walk->vma), walk->mm, pte); - entry =3D huge_ptep_get(walk->mm, addr, pte); - if (!pte_present(entry)) { - if (unlikely(is_hugetlb_entry_migration(entry))) - qp->nr_failed++; + ptep =3D huge_ptep_get(walk->mm, addr, pte); + if (!pte_present(ptep)) { + if (!huge_pte_none(ptep)) { + const leaf_entry_t entry =3D leafent_from_pte(ptep); + + if (unlikely(leafent_is_migration(entry))) + qp->nr_failed++; + } + goto unlock; } - folio =3D pfn_folio(pte_pfn(entry)); + folio =3D pfn_folio(pte_pfn(ptep)); if (!queue_folio_required(folio, qp)) goto unlock; if (!(flags & (MPOL_MF_MOVE | MPOL_MF_MOVE_ALL)) || diff --git a/mm/migrate.c b/mm/migrate.c index bb0429a5e5a4..8f2c3c7d87ba 100644 --- a/mm/migrate.c +++ b/mm/migrate.c @@ -515,16 +515,18 @@ void migration_entry_wait(struct mm_struct *mm, pmd_t= *pmd, void migration_entry_wait_huge(struct vm_area_struct *vma, unsigned long a= ddr, pte_t *ptep) { spinlock_t *ptl =3D huge_pte_lockptr(hstate_vma(vma), vma->vm_mm, ptep); + leaf_entry_t entry; pte_t pte; =20 hugetlb_vma_assert_locked(vma); spin_lock(ptl); pte =3D huge_ptep_get(vma->vm_mm, addr, ptep); =20 - if (unlikely(!is_hugetlb_entry_migration(pte))) { - spin_unlock(ptl); - hugetlb_vma_unlock_read(vma); - } else { + if (huge_pte_none(pte)) + goto fail; + + entry =3D leafent_from_pte(pte); + if (leafent_is_migration(entry)) { /* * If migration entry existed, safe to release vma lock * here because the pgtable page won't be freed without the @@ -533,7 +535,12 @@ void migration_entry_wait_huge(struct vm_area_struct *= vma, unsigned long addr, p */ hugetlb_vma_unlock_read(vma); migration_entry_wait_on_locked(pte_to_swp_entry(pte), ptl); + return; } + +fail: + spin_unlock(ptl); + hugetlb_vma_unlock_read(vma); } #endif =20 --=20 2.51.0 From nobody Sat Feb 7 17:56:00 2026 Received: from mx0a-00069f02.pphosted.com (mx0a-00069f02.pphosted.com [205.220.165.32]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id CB51B3148B8; Mon, 3 Nov 2025 12:35:13 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=205.220.165.32 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1762173317; cv=fail; b=ScrmJq/TiomeABmAQ2SnCkUnmLJypt8jZe8lQtntUhVMFYz+wEK0x7Yo5zQKmj4sm+mpoMSkPQa3AxdkAADZoibMHJ18vVSuQ1BC0IQyWxiRli7Qw6iaCeT8B7ItArH140zjpuLjP5x/5ZlsKKj+XufVmxmHJTSvs/pC0vbsziw= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1762173317; c=relaxed/simple; bh=QBJKhRDFlr++Mm6mTlGCqUZyP/HJ1Ou8HEOsCdHgiRQ=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: Content-Type:MIME-Version; b=cfPT+JYXu8kU6U+Qu/kzCXeq298gCkp/18SmLSCBvVlgNUsT5Sa5fyw4Q0RvwcS/sLcBPYOZCsYM4ljFUqp5rOUNR5w2QxFVMI56x6J825gP4xCG/XJse5gboDHT5xjTRF2OmgJD9FDSwfNOxSrtgXYECMzgLu8JRX4lths7LT0= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=oracle.com; spf=pass smtp.mailfrom=oracle.com; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b=S0tHTa2p; dkim=pass (1024-bit key) header.d=oracle.onmicrosoft.com header.i=@oracle.onmicrosoft.com header.b=DMn+Sr8M; arc=fail smtp.client-ip=205.220.165.32 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=oracle.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=oracle.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b="S0tHTa2p"; dkim=pass (1024-bit key) header.d=oracle.onmicrosoft.com header.i=@oracle.onmicrosoft.com header.b="DMn+Sr8M" Received: from pps.filterd (m0246629.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 5A3CTpq4011573; Mon, 3 Nov 2025 12:33:24 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=cc :content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s= corp-2025-04-25; bh=V7Qyup/0b2hllUs/jtYn5BodrJWAtBT5wbbkGhOZbDc=; b= S0tHTa2p3Aw6EkakLFCLrcW96VyGWNlNwrp2X7Zv307EDR+0OV9LoPdCo67XsNPg FGmtPI5ZOaZHBwJn71ZIB3ze/BOkP3pB4t8k0xowe1SEM5Xpd/bz4JAvlPvR7vA+ cM8wfrQAKEzx0NeJ9eiHN9bgjzJfGzG2tdg+S7Um7pBB6Zt4RGH3XLEFMYwyhdI7 jlt0w4LxrdrX52YUUi02r9nNGG/wuy5uP10CzN/JKatkgPglw6031yLhewRORQUf OOwB3UBE1dPjWJvQIHJHt4BI+ofAimdypn8BkL06FJLX0KPKFzSsIFblJT22NPTo VpOio3fPTmMNQ6vpMHbcUQ== Received: from phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta02.appoci.oracle.com [147.154.114.232]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 4a6vep009r-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 03 Nov 2025 12:33:23 +0000 (GMT) Received: from pps.filterd (phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (8.18.1.2/8.18.1.2) with ESMTP id 5A3Bx7kR021025; Mon, 3 Nov 2025 12:33:23 GMT Received: from sj2pr03cu001.outbound.protection.outlook.com (mail-westusazon11012047.outbound.protection.outlook.com [52.101.43.47]) by phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTPS id 4a58n83ev0-2 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 03 Nov 2025 12:33:22 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=cJ/mwvMYK48MQTmUDELJbek7dOIE/u+pDHb7ZcnUOqs90S+yuvznigQhKCDu8YrkiMZhAR+frk5nrXm9tdWZdNuSFZUe9PEHw1t/jr963Y9mpuDZnabGxH1d03pGywS3erg03ccx7BaxMPsVpwKC5QFhUCFMfj8SpnnIFRRZSXrLD8MJDyguE47o83qzoc3kOd49J8cHc3UYZXcTEu5DlhUP2PPglFoQGNErOa78C0bnQIFSjqQzdI/5y2JIFu4SF9M226GwniVna4HAdWReJNB89n3VF+nKxXifYUjNkWJiaOE9eruwz4bQteBiTOkCbLEfdJdLB8fhPKrOmBqfHA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=V7Qyup/0b2hllUs/jtYn5BodrJWAtBT5wbbkGhOZbDc=; b=ZovJQkZCEQyc0W//ug0vu2SmzJZMm3epfLOs/VaaVVNpAKpQdA/hwpN7TmpT7mfs1HSukXbRQgWgdkWcdw6SusZkj6HTSMbY6+43svHi52Dh44et+zI+ibn+UTJ9qOTYxM/WoubFh0zth0JxMznGk9TrHFMsopT40SWDpl4lYhYrPFhZHaxOhCLN7NI1a1ibKbWDFppuATRGHKl5nntHKOCPJmxJg2x82tCY+nCWdNPE+WYB184C35F40pJ/m9f4hzFjd/ZskpN4JF6MxJajZeJoFWnhOLky/xE2Sdxlj8Dj+wIknI3AAk53BgT54mAgVItAMfSovBn2XAAIRXX4kg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=V7Qyup/0b2hllUs/jtYn5BodrJWAtBT5wbbkGhOZbDc=; b=DMn+Sr8MJJa2lAb18UraDh5rnyP+tEZ47dWCu9E7N6u1QRSndl7BPLEnqoOlBow/8E1nMuGvH7qrSXoKtKbpyvVMLxQdsVCjOXAzxWzP2xGSbcL9yI4ojmWHfylxdBbISLHch9J3o7jBZ/8WPWgjtdZqDZQ8VK55iw+kuFET3lY= Received: from DM4PR10MB8218.namprd10.prod.outlook.com (2603:10b6:8:1cc::16) by DS4PR10MB997575.namprd10.prod.outlook.com (2603:10b6:8:31e::5) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9275.16; Mon, 3 Nov 2025 12:32:53 +0000 Received: from DM4PR10MB8218.namprd10.prod.outlook.com ([fe80::2650:55cf:2816:5f2]) by DM4PR10MB8218.namprd10.prod.outlook.com ([fe80::2650:55cf:2816:5f2%2]) with mapi id 15.20.9253.018; Mon, 3 Nov 2025 12:32:53 +0000 From: Lorenzo Stoakes To: Andrew Morton Cc: Christian Borntraeger , Janosch Frank , Claudio Imbrenda , David Hildenbrand , Alexander Gordeev , Gerald Schaefer , Heiko Carstens , Vasily Gorbik , Sven Schnelle , Peter Xu , Alexander Viro , Christian Brauner , Jan Kara , Arnd Bergmann , Zi Yan , Baolin Wang , "Liam R . Howlett" , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Lance Yang , Muchun Song , Oscar Salvador , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Matthew Brost , Joshua Hahn , Rakie Kim , Byungchul Park , Gregory Price , Ying Huang , Alistair Popple , Axel Rasmussen , Yuanchu Xie , Wei Xu , Kemeng Shi , Kairui Song , Nhat Pham , Baoquan He , Chris Li , SeongJae Park , Matthew Wilcox , Jason Gunthorpe , Leon Romanovsky , Xu Xin , Chengming Zhou , Jann Horn , Miaohe Lin , Naoya Horiguchi , Pedro Falcato , Pasha Tatashin , Rik van Riel , Harry Yoo , Hugh Dickins , linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-s390@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, linux-arch@vger.kernel.org, damon@lists.linux.dev Subject: [PATCH 15/16] mm: eliminate further swapops predicates Date: Mon, 3 Nov 2025 12:31:56 +0000 Message-ID: <4710023e2a19d01efb023fff3e074d015f0bde10.1762171281.git.lorenzo.stoakes@oracle.com> X-Mailer: git-send-email 2.51.0 In-Reply-To: References: Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: LO4P123CA0295.GBRP123.PROD.OUTLOOK.COM (2603:10a6:600:196::12) To DM4PR10MB8218.namprd10.prod.outlook.com (2603:10b6:8:1cc::16) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DM4PR10MB8218:EE_|DS4PR10MB997575:EE_ X-MS-Office365-Filtering-Correlation-Id: b0f7229e-2f83-4090-f30d-08de1ad51bcf X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|376014|7416014|366016|1800799024; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?2/DYhK30Nd9Qj+PV/ogFx7jRJxfxTnQK/qMJlSfIOFSX693ID5ePgBjcIuYI?= =?us-ascii?Q?njG9BCRuwK7wJm5rWxAYPPvdofR7E9gi8d28pmw+IpwwfsN5Fq2cuu2RqDTP?= =?us-ascii?Q?ZDF+ILygf4KrZVitTLgXrzBVfWTJqKTgcnLpwjAkbxgkkhpoCMy9QwAJ30W2?= =?us-ascii?Q?EeEqhatsX7DlQ5AK9RPzgryM8JLwIjIeY6gyzB13HaOEd05B0mb8wEIlOvCd?= =?us-ascii?Q?1x1d8s2S+0sPGKvRB6KfIcbkwKNj8UCRCP1/+2jXUHy5qy0r9jbOf5JkamhU?= =?us-ascii?Q?WKx06JncKrTYOa0XZq4r+sfexprCc9HW1HU5YV8Aoi3lxRV51NLDukT+rGUB?= =?us-ascii?Q?0lj2CCwOasJ0CyFysq6vjr8twCXi8tT85Ap+omEhtAb3s0+2BOs6aynp5/rq?= =?us-ascii?Q?n2y6sXOnA077FThJErbvKmCFLIzY65rUIQmrQBb/Om3ai31syajtGTbhUIBH?= =?us-ascii?Q?JaNcf3g2A+0t7v+6RYuSkMFM3Ljr1hEgS32TG0RsJqsVPuvjmpFCEvjlK4B0?= =?us-ascii?Q?FB+sH5Z/LAzQ9IkV+aoMu+iu1vmgvzlFfyBgMObwCgJ6JZnb3ZBoT9xD3vFy?= =?us-ascii?Q?HTNN3MlN6t2Pt2uyyx3p8K7s2ooqEgrsGhW3ZCwxyZUUNJ/b0W/QT3K2gtmV?= =?us-ascii?Q?yV/t4h1OWb6eToxif+lNVogD4/7lzbvjl6kxhWnJ0BEVoqRj0YUQ1mrnDFOc?= =?us-ascii?Q?4DJT9xPDRWRVR0t9Zv/aQD9n2ACv5SSZdCkI0TEyn4kiDCqiVwFe3kaxBHx/?= =?us-ascii?Q?e32ZiIzQHpSM2D0f9lYSEa0x0YNHjlH0aDV11JHHV8IdAqyVaedDv2xViVF7?= =?us-ascii?Q?ZUz3dFEW+ckjn70cRR93MaeADuPNo7c9fK3Kc0FKkomQxbuMklVnZgOD+D6l?= =?us-ascii?Q?z6dq9wT29fbO/hQNN6rGYx3NWejORqz8QtK46GHqzl0FEveRYuxQrOoC7ibv?= =?us-ascii?Q?Sz0wTh2781cJntx1IvL/s8RYxwTkgTxb5ini0ypm+OyEyHBSgHbfKVeJcMyr?= =?us-ascii?Q?rTIY9z+yiisPay2cf/+AS2Gl41GrlSoaRxLoHIjqetduiIZjNMrz6Cf/S8Q6?= =?us-ascii?Q?8+WuVLupYqHEUUGJ1f674jUlUb4HfMaeNVkcO4ZtDdUajNT4IeqzcipZtvdX?= =?us-ascii?Q?NvejU2oGoHNSJEuK8XCV/Y0PMTfi3s7H8iEgj9pfLmoaOCZkyU1pF8nUchtu?= =?us-ascii?Q?O/LtKDId8Ba669aWLcvsk5aw1xbUtA5aCKODD8xf1T44TriUV83S52D5LSPf?= =?us-ascii?Q?gZ5dl1zn9ShS8EeQ69y5TDPYubjvsz8Esw+j63eqr4jigPo8qGSksdYppcqe?= =?us-ascii?Q?VB4FlXScAc0CnBvvA8akrO7Lh6FmWXslHJNnbAMVI6bamTfm7r+wjDbGjpul?= =?us-ascii?Q?6bH9b5UT3LjlLbs7qbEzV7FsEYhedxg7wUKXCn9t1ZT/DovQNcKdLIZzG9qZ?= =?us-ascii?Q?0CEFtCKElHx7AojlOU4ZRGda20ZL7xbf?= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DM4PR10MB8218.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(376014)(7416014)(366016)(1800799024);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?wbNSght04S4cEZJe/aAX4J9Qe3ezwh1Pm4hXIq2RDz84JNsjkhqogWbc9NRM?= =?us-ascii?Q?qVROjnY0Tg5LbpMxIRdaxVd/lqviGxnAcVoDTB8oeGmttcumdOPy8ghhSqum?= =?us-ascii?Q?FnbgC8twlvSQ8o8uHHvbQCEM+nRoQf56KEhA1JbXbbYzUHGXM917/vqDgZm2?= =?us-ascii?Q?gh1+hp28lpezIEQueKFKlmw/450WIrdsYHlmQHt/jpI/rvlZvfsVA2P8IptD?= =?us-ascii?Q?ZUf83//RdrFoVHxgXSGBDNbhndPmNhmrWJml1ZevzZonKHM/+zbdQlp8+afT?= =?us-ascii?Q?n7OcX4sF6FfVF5VZEkjrEsHgwqSw8TupYWtx57UT2aIZfeaOwhMkP4pUHGxB?= =?us-ascii?Q?BQWZCMXJmIsQd5luiCM6oslSRnfdqsQ26llusfnwXPEcekCbuPKq2NLmoAfE?= =?us-ascii?Q?oN0B6TTc7kzfrOfJb5hjpm2AYNEaI2pAXlI85+uwyvWVXKXoeqv08CBfa8Bh?= =?us-ascii?Q?VttwbgTkEApaNvr56VRyXGIs+4lopP+NcgVHwD8Ky6EHcnabMgai8yOa0wgP?= =?us-ascii?Q?RLKBc1sp+8JWZuDqTXatYxAKKITuq92MuogfLJIn8GeAlGZxzLoRhSeGjnSR?= =?us-ascii?Q?pkAZLl2iRzzrKeIfHcvE+J2f5sQDbnvxR1gEx1E6K6sAFqVLeCqebGoWGWYa?= =?us-ascii?Q?JmgrecJe6m7JMsqkVWPFyw15s0hS7mXkFmMpxOmdkd7SgHcPuUcIP0jIgKDf?= =?us-ascii?Q?NVkA1R8uEcebX8/FTTsQ23qczR7P7TDXriCZmzS+8Mtd0WBPnYYt517DFIIs?= =?us-ascii?Q?4shKi3fit5P4gRzXWfCxPG+l50pH4//jSONXrJF8B8Um2TRP0vcoaN7QBc4A?= =?us-ascii?Q?hoPpzeqeHaRHBEw0yaozuqwsbWUc5qD0kncI/h9TeCCGT5A7PW8eeFbvLXXQ?= =?us-ascii?Q?ouS0y0LG6fkM2KSUvM3b3eKxBzcnk3+1xCWS8mVNdh9Zp4LkQ7stJ0tnOiew?= =?us-ascii?Q?sITIaCrsHqclpBY+67zcBOffOjkc2ChkxPFsbLJ2aYfvWlpHq0fP0nGO4OgT?= =?us-ascii?Q?63AaTNB1gdAF0BX8NfPTw81BkOmNmEAnLwBe4PNs6L5kMTOe+Xwuc5iCPuhR?= =?us-ascii?Q?GGm89sSNDMgHf1Hvimdkl94qRBkJYWnJpLdLAtNEHwVLbr+qPLlNxDpQ5Y4p?= =?us-ascii?Q?dy3KW1DOto0VjOKOJ3IEnn7mSrmNltT+ukOhofwDkwh3R2L1A/CLspOoIZMt?= =?us-ascii?Q?db5Cu47MnPMDBkF6tAU+NLjbi8F2AXzmeV4QngcgY588bwOcH1hqi+5tD8fq?= =?us-ascii?Q?uflc0czYuco1irwBhnPFyxMyLGT1Gsk1TT1rkDdFe6i/4mUwifLEieiQKlmo?= =?us-ascii?Q?2awRRx8P8WzpPv2rWZQ8QCHvdHKmVgzDbuuCds6tAVe1k9h4CUROz0e8RHEt?= =?us-ascii?Q?aIM7ZIOT0hKRrF87P8wcFrJH5GKjPtuOlYhu9Lf/gFkGYV9ZgdvVhenF2ywJ?= =?us-ascii?Q?mktfRLpMkhE2VYdFctjVQ1ApzkbpiIUU2ZJPOPQWcJC+Bj7XztCTtmEqmGQ5?= =?us-ascii?Q?5rpA0T7WH1WhUnXaFQkP0DUONmPOAOc+NXNaGFo1xJaw+gAZ+8u54RjX4WQV?= =?us-ascii?Q?yoedROB5RsN5X0s91l4tJe1tGnIRIIMAWhzTwwlPDJtKl3llEeQhhXf952k2?= =?us-ascii?Q?OA=3D=3D?= X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: 0NUKFnmTGyrjsZxUz+eTi58Osc+aik08mkVs9SihQNUQZ2xWotd4C7HGMKhJUDU3VMJc5JRiL6kgnstotv1cuy/EOF/7/pE1XQFM0anG1k8B7bABt39h+RnOaGv73U+my0KcwvEXVS8pxHwfaGSWjqMLufNi/JnNTzwRlqCSKW3cZz2s0gfr1GGCs7usQy2YgTw2LS/GsiY+13QQOaHk5Z/NxDvRivfx3G4gdEqvel/JLGMzYLN4O5dpCD0uhpg6lVMzGV8HOJoTtipT+OoN+OXvv8EQ5F5QMOAaDIOlHMNocHhckRqzLeAUm8JXAItAw5N1aERHOWcWe5gi2Lmq2Wk2ZppGi8SDfIoYrFiae2yZ6eZp3KIJIHnV0kFMvoX287sOlbNmKvoNWMG4+WhqPmBpSbb61kcy2lQvE+AIm5e8oVKwlEpywf2sj4NJvdb3DHVXMzawcjQhT3iKIB6K8oIaoerYe2rwkPWIMNemADVUOibj9e1M/vFnM3Fajxd3EkIAkstRmjwp9IiIgysrs4Ed4tcUn//hdqVl0UoK4bWerxj0kf62q38/WYQefegRmKVebRUvgU/VzeGYiqn355xRKrC2nk753+AkxT3MBho= X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: b0f7229e-2f83-4090-f30d-08de1ad51bcf X-MS-Exchange-CrossTenant-AuthSource: DM4PR10MB8218.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 03 Nov 2025 12:32:53.1867 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: bwdE4o6ay+zQk/UO8VOR+Wv5WHd3XZANLMslKQ7CQttkPZ8PEcizIkq9H95hkHB2Nyni1+ahTJMm+CvZ6UNiMwpXyTRyb+r7fMiXqd9qdV8= X-MS-Exchange-Transport-CrossTenantHeadersStamped: DS4PR10MB997575 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1121,Hydra:6.1.9,FMLib:17.12.100.49 definitions=2025-11-03_01,2025-11-03_01,2025-10-01_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 phishscore=0 bulkscore=0 malwarescore=0 spamscore=0 suspectscore=0 mlxscore=0 adultscore=0 mlxlogscore=999 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2510240000 definitions=main-2511030114 X-Proofpoint-GUID: YxRiJ2glE3Gu_YqWiiyBz9mmLyRvVJum X-Proofpoint-ORIG-GUID: YxRiJ2glE3Gu_YqWiiyBz9mmLyRvVJum X-Authority-Analysis: v=2.4 cv=B9m0EetM c=1 sm=1 tr=0 ts=6908a113 cx=c_pps a=OOZaFjgC48PWsiFpTAqLcw==:117 a=OOZaFjgC48PWsiFpTAqLcw==:17 a=6eWqkTHjU83fiwn7nKZWdM+Sl24=:19 a=z/mQ4Ysz8XfWz/Q5cLBRGdckG28=:19 a=lCpzRmAYbLLaTzLvsPZ7Mbvzbb8=:19 a=xqWC_Br6kY4A:10 a=6UeiqGixMTsA:10 a=GoEa3M9JfhUA:10 a=VkNPw1HP01LnGYTKEx00:22 a=yPCof4ZbAAAA:8 a=12TsxHtCQ5kFwp23Kd0A:9 a=UhEZJTgQB8St2RibIkdl:22 a=Z5ABNNGmrOfJ6cZ5bIyy:22 a=QOGEsqRv6VhmHaoFNykA:22 X-Proofpoint-Spam-Details-Enc: AW1haW4tMjUxMTAzMDExNCBTYWx0ZWRfX2laHw6luLnUV V43ukoDlFc8d8mFUplHU8cCZ4gyNvGq3ks7IB4wEJu7PtNuibn8akwRQqVsjN8qQ6H9bKpmzRe+ /TMs6XuTIyq/x+3WtT3TrcCTvf0mqoHzerwVn3yJnFN6EbJJPmucf/MXMXOHl6N+K2Pxn8NID9/ FuZx2exdqu2r/ASLe9QZRetdsfQYsNccyDoatkGddxV+MO8udpJfyr+WLMhgXfzjnfx0iEljmx5 eGKCbpUk5qHGgsmfYA2yX0rVzmbdeXgNxeI4PyMxs8wFA34EwIlyUCtChQiPYdEbZJTD7le1eMK DHTvQ2YIDH5j/53QCpD6HBfWGdnIGAR2/vwfy0owJoEE6BnIra+uJPILlqoOszkLnESXfxJr1Cp /VftoF3j9Be190JIJJbfanAFJGEOdg== Content-Type: text/plain; charset="utf-8" Having converted so much of the code base to leaf entries, we can mop up some remaining cases. We replace is_pfn_swap_entry(), pfn_swap_entry_to_page(), is_writable_device_private_entry(), is_device_exclusive_entry(), is_migration_entry(), is_writable_migration_entry(), is_readable_migration_entry(), swp_offset_pfn() and pfn_swap_entry_folio() with leafent equivalents. No functional change intended. Signed-off-by: Lorenzo Stoakes --- fs/proc/task_mmu.c | 14 ++--- include/linux/leafops.h | 25 +++++++-- include/linux/swapops.h | 121 +--------------------------------------- mm/debug_vm_pgtable.c | 20 +++---- mm/hmm.c | 2 +- mm/hugetlb.c | 2 +- mm/ksm.c | 6 +- mm/memory-failure.c | 6 +- mm/memory.c | 3 +- mm/mempolicy.c | 4 +- mm/migrate.c | 6 +- mm/migrate_device.c | 10 ++-- mm/mprotect.c | 8 +-- mm/page_vma_mapped.c | 8 +-- mm/pagewalk.c | 7 +-- mm/rmap.c | 9 ++- 16 files changed, 75 insertions(+), 176 deletions(-) diff --git a/fs/proc/task_mmu.c b/fs/proc/task_mmu.c index 82532c069039..8a9894aefbca 100644 --- a/fs/proc/task_mmu.c +++ b/fs/proc/task_mmu.c @@ -1940,13 +1940,13 @@ static pagemap_entry_t pte_to_pagemap_entry(struct = pagemapread *pm, if (pte_uffd_wp(pte)) flags |=3D PM_UFFD_WP; } else { - swp_entry_t entry; + leaf_entry_t entry; =20 if (pte_swp_soft_dirty(pte)) flags |=3D PM_SOFT_DIRTY; if (pte_swp_uffd_wp(pte)) flags |=3D PM_UFFD_WP; - entry =3D pte_to_swp_entry(pte); + entry =3D leafent_from_pte(pte); if (pm->show_pfn) { pgoff_t offset; =20 @@ -1954,16 +1954,16 @@ static pagemap_entry_t pte_to_pagemap_entry(struct = pagemapread *pm, * For PFN swap offsets, keeping the offset field * to be PFN only to be compatible with old smaps. */ - if (is_pfn_swap_entry(entry)) - offset =3D swp_offset_pfn(entry); + if (leafent_has_pfn(entry)) + offset =3D leafent_to_pfn(entry); else offset =3D swp_offset(entry); frame =3D swp_type(entry) | (offset << MAX_SWAPFILES_SHIFT); } flags |=3D PM_SWAP; - if (is_pfn_swap_entry(entry)) - page =3D pfn_swap_entry_to_page(entry); + if (leafent_has_pfn(entry)) + page =3D leafent_to_page(entry); if (leafent_is_uffd_wp_marker(entry)) flags |=3D PM_UFFD_WP; if (leafent_is_guard_marker(entry)) @@ -2032,7 +2032,7 @@ static int pagemap_pmd_range_thp(pmd_t *pmdp, unsigne= d long addr, if (pmd_swp_uffd_wp(pmd)) flags |=3D PM_UFFD_WP; VM_WARN_ON_ONCE(!pmd_is_migration_entry(pmd)); - page =3D pfn_swap_entry_to_page(entry); + page =3D leafent_to_page(entry); } =20 if (page) { diff --git a/include/linux/leafops.h b/include/linux/leafops.h index 2d3bc4c866bd..b74d406ba648 100644 --- a/include/linux/leafops.h +++ b/include/linux/leafops.h @@ -357,7 +357,7 @@ static inline unsigned long leafent_to_pfn(leaf_entry_t= entry) VM_WARN_ON_ONCE(!leafent_has_pfn(entry)); =20 /* Temporary until swp_entry_t eliminated. */ - return swp_offset_pfn(entry); + return swp_offset(entry) & SWP_PFN_MASK; } =20 /** @@ -368,10 +368,16 @@ static inline unsigned long leafent_to_pfn(leaf_entry= _t entry) */ static inline struct page *leafent_to_page(leaf_entry_t entry) { + struct page *page =3D pfn_to_page(leafent_to_pfn(entry)); + VM_WARN_ON_ONCE(!leafent_has_pfn(entry)); + /* + * Any use of migration entries may only occur while the + * corresponding page is locked + */ + VM_WARN_ON_ONCE(leafent_is_migration(entry) && !PageLocked(page)); =20 - /* Temporary until swp_entry_t eliminated. */ - return pfn_swap_entry_to_page(entry); + return page; } =20 /** @@ -383,10 +389,17 @@ static inline struct page *leafent_to_page(leaf_entry= _t entry) */ static inline struct folio *leafent_to_folio(leaf_entry_t entry) { - VM_WARN_ON_ONCE(!leafent_has_pfn(entry)); + struct folio *folio =3D pfn_folio(leafent_to_pfn(entry)); =20 - /* Temporary until swp_entry_t eliminated. */ - return pfn_swap_entry_folio(entry); + VM_WARN_ON_ONCE(!leafent_has_pfn(entry)); + /* + * Any use of migration entries may only occur while the + * corresponding folio is locked. + */ + VM_WARN_ON_ONCE(leafent_is_migration(entry) && + !folio_test_locked(folio)); + + return folio; } =20 /** diff --git a/include/linux/swapops.h b/include/linux/swapops.h index c8e6f927da48..3d02b288c15e 100644 --- a/include/linux/swapops.h +++ b/include/linux/swapops.h @@ -28,7 +28,7 @@ #define SWP_OFFSET_MASK ((1UL << SWP_TYPE_SHIFT) - 1) =20 /* - * Definitions only for PFN swap entries (see is_pfn_swap_entry()). To + * Definitions only for PFN swap entries (see leafeant_has_pfn()). To * store PFN, we only need SWP_PFN_BITS bits. Each of the pfn swap entries * can use the extra bits to store other information besides PFN. */ @@ -66,8 +66,6 @@ #define SWP_MIG_YOUNG BIT(SWP_MIG_YOUNG_BIT) #define SWP_MIG_DIRTY BIT(SWP_MIG_DIRTY_BIT) =20 -static inline bool is_pfn_swap_entry(swp_entry_t entry); - /* Clear all flags but only keep swp_entry_t related information */ static inline pte_t pte_swp_clear_flags(pte_t pte) { @@ -109,17 +107,6 @@ static inline pgoff_t swp_offset(swp_entry_t entry) return entry.val & SWP_OFFSET_MASK; } =20 -/* - * This should only be called upon a pfn swap entry to get the PFN stored - * in the swap entry. Please refers to is_pfn_swap_entry() for definition - * of pfn swap entry. - */ -static inline unsigned long swp_offset_pfn(swp_entry_t entry) -{ - VM_BUG_ON(!is_pfn_swap_entry(entry)); - return swp_offset(entry) & SWP_PFN_MASK; -} - /* * Convert the arch-dependent pte representation of a swp_entry_t into an * arch-independent swp_entry_t. @@ -169,27 +156,11 @@ static inline swp_entry_t make_writable_device_privat= e_entry(pgoff_t offset) return swp_entry(SWP_DEVICE_WRITE, offset); } =20 -static inline bool is_device_private_entry(swp_entry_t entry) -{ - int type =3D swp_type(entry); - return type =3D=3D SWP_DEVICE_READ || type =3D=3D SWP_DEVICE_WRITE; -} - -static inline bool is_writable_device_private_entry(swp_entry_t entry) -{ - return unlikely(swp_type(entry) =3D=3D SWP_DEVICE_WRITE); -} - static inline swp_entry_t make_device_exclusive_entry(pgoff_t offset) { return swp_entry(SWP_DEVICE_EXCLUSIVE, offset); } =20 -static inline bool is_device_exclusive_entry(swp_entry_t entry) -{ - return swp_type(entry) =3D=3D SWP_DEVICE_EXCLUSIVE; -} - #else /* CONFIG_DEVICE_PRIVATE */ static inline swp_entry_t make_readable_device_private_entry(pgoff_t offse= t) { @@ -201,50 +172,14 @@ static inline swp_entry_t make_writable_device_privat= e_entry(pgoff_t offset) return swp_entry(0, 0); } =20 -static inline bool is_device_private_entry(swp_entry_t entry) -{ - return false; -} - -static inline bool is_writable_device_private_entry(swp_entry_t entry) -{ - return false; -} - static inline swp_entry_t make_device_exclusive_entry(pgoff_t offset) { return swp_entry(0, 0); } =20 -static inline bool is_device_exclusive_entry(swp_entry_t entry) -{ - return false; -} - #endif /* CONFIG_DEVICE_PRIVATE */ =20 #ifdef CONFIG_MIGRATION -static inline int is_migration_entry(swp_entry_t entry) -{ - return unlikely(swp_type(entry) =3D=3D SWP_MIGRATION_READ || - swp_type(entry) =3D=3D SWP_MIGRATION_READ_EXCLUSIVE || - swp_type(entry) =3D=3D SWP_MIGRATION_WRITE); -} - -static inline int is_writable_migration_entry(swp_entry_t entry) -{ - return unlikely(swp_type(entry) =3D=3D SWP_MIGRATION_WRITE); -} - -static inline int is_readable_migration_entry(swp_entry_t entry) -{ - return unlikely(swp_type(entry) =3D=3D SWP_MIGRATION_READ); -} - -static inline int is_readable_exclusive_migration_entry(swp_entry_t entry) -{ - return unlikely(swp_type(entry) =3D=3D SWP_MIGRATION_READ_EXCLUSIVE); -} =20 static inline swp_entry_t make_readable_migration_entry(pgoff_t offset) { @@ -310,23 +245,10 @@ static inline swp_entry_t make_writable_migration_ent= ry(pgoff_t offset) return swp_entry(0, 0); } =20 -static inline int is_migration_entry(swp_entry_t swp) -{ - return 0; -} - static inline void migration_entry_wait(struct mm_struct *mm, pmd_t *pmd, unsigned long address) { } static inline void migration_entry_wait_huge(struct vm_area_struct *vma, unsigned long addr, pte_t *pte) { } -static inline int is_writable_migration_entry(swp_entry_t entry) -{ - return 0; -} -static inline int is_readable_migration_entry(swp_entry_t entry) -{ - return 0; -} =20 static inline swp_entry_t make_migration_entry_young(swp_entry_t entry) { @@ -410,47 +332,6 @@ static inline swp_entry_t make_guard_swp_entry(void) return make_pte_marker_entry(PTE_MARKER_GUARD); } =20 -static inline struct page *pfn_swap_entry_to_page(swp_entry_t entry) -{ - struct page *p =3D pfn_to_page(swp_offset_pfn(entry)); - - /* - * Any use of migration entries may only occur while the - * corresponding page is locked - */ - BUG_ON(is_migration_entry(entry) && !PageLocked(p)); - - return p; -} - -static inline struct folio *pfn_swap_entry_folio(swp_entry_t entry) -{ - struct folio *folio =3D pfn_folio(swp_offset_pfn(entry)); - - /* - * Any use of migration entries may only occur while the - * corresponding folio is locked - */ - BUG_ON(is_migration_entry(entry) && !folio_test_locked(folio)); - - return folio; -} - -/* - * A pfn swap entry is a special type of swap entry that always has a pfn = stored - * in the swap offset. They can either be used to represent unaddressable = device - * memory, to restrict access to a page undergoing migration or to represe= nt a - * pfn which has been hwpoisoned and unmapped. - */ -static inline bool is_pfn_swap_entry(swp_entry_t entry) -{ - /* Make sure the swp offset can always store the needed fields */ - BUILD_BUG_ON(SWP_TYPE_SHIFT < SWP_PFN_BITS); - - return is_migration_entry(entry) || is_device_private_entry(entry) || - is_device_exclusive_entry(entry) || is_hwpoison_entry(entry); -} - struct page_vma_mapped_walk; =20 #ifdef CONFIG_ARCH_ENABLE_THP_MIGRATION diff --git a/mm/debug_vm_pgtable.c b/mm/debug_vm_pgtable.c index 8f247fcf1865..181fa2b25625 100644 --- a/mm/debug_vm_pgtable.c +++ b/mm/debug_vm_pgtable.c @@ -844,7 +844,7 @@ static void __init pmd_leafent_tests(struct pgtable_deb= ug_args *args) { } static void __init swap_migration_tests(struct pgtable_debug_args *args) { struct page *page; - swp_entry_t swp; + leaf_entry_t entry; =20 if (!IS_ENABLED(CONFIG_MIGRATION)) return; @@ -867,17 +867,17 @@ static void __init swap_migration_tests(struct pgtabl= e_debug_args *args) * be locked, otherwise it stumbles upon a BUG_ON(). */ __SetPageLocked(page); - swp =3D make_writable_migration_entry(page_to_pfn(page)); - WARN_ON(!is_migration_entry(swp)); - WARN_ON(!is_writable_migration_entry(swp)); + entry =3D make_writable_migration_entry(page_to_pfn(page)); + WARN_ON(!leafent_is_migration(entry)); + WARN_ON(!leafent_is_migration_write(entry)); =20 - swp =3D make_readable_migration_entry(swp_offset(swp)); - WARN_ON(!is_migration_entry(swp)); - WARN_ON(is_writable_migration_entry(swp)); + entry =3D make_readable_migration_entry(swp_offset(entry)); + WARN_ON(!leafent_is_migration(entry)); + WARN_ON(leafent_is_migration_write(entry)); =20 - swp =3D make_readable_migration_entry(page_to_pfn(page)); - WARN_ON(!is_migration_entry(swp)); - WARN_ON(is_writable_migration_entry(swp)); + entry =3D make_readable_migration_entry(page_to_pfn(page)); + WARN_ON(!leafent_is_migration(entry)); + WARN_ON(leafent_is_migration_write(entry)); __ClearPageLocked(page); } =20 diff --git a/mm/hmm.c b/mm/hmm.c index 831ef855a55a..618503c8fd1c 100644 --- a/mm/hmm.c +++ b/mm/hmm.c @@ -265,7 +265,7 @@ static int hmm_vma_handle_pte(struct mm_walk *walk, uns= igned long addr, cpu_flags =3D HMM_PFN_VALID; if (leafent_is_device_private_write(entry)) cpu_flags |=3D HMM_PFN_WRITE; - new_pfn_flags =3D swp_offset_pfn(entry) | cpu_flags; + new_pfn_flags =3D leafent_to_pfn(entry) | cpu_flags; goto out; } =20 diff --git a/mm/hugetlb.c b/mm/hugetlb.c index 6c483ecd496f..acb0c38c99a8 100644 --- a/mm/hugetlb.c +++ b/mm/hugetlb.c @@ -5640,7 +5640,7 @@ int copy_hugetlb_page_range(struct mm_struct *dst, st= ruct mm_struct *src, } else if (unlikely(leafent_is_migration(leafent))) { bool uffd_wp =3D pte_swp_uffd_wp(entry); =20 - if (!is_readable_migration_entry(leafent) && cow) { + if (!leafent_is_migration_read(leafent) && cow) { /* * COW mappings require pages in both * parent and child to be set to read. diff --git a/mm/ksm.c b/mm/ksm.c index 7cd19a6ce45f..f9dbe93fcffc 100644 --- a/mm/ksm.c +++ b/mm/ksm.c @@ -637,14 +637,14 @@ static int break_ksm_pmd_entry(pmd_t *pmdp, unsigned = long addr, unsigned long en if (pte_present(pte)) { folio =3D vm_normal_folio(walk->vma, addr, pte); } else if (!pte_none(pte)) { - swp_entry_t entry =3D pte_to_swp_entry(pte); + const leaf_entry_t entry =3D leafent_from_pte(pte); =20 /* * As KSM pages remain KSM pages until freed, no need to wait * here for migration to end. */ - if (is_migration_entry(entry)) - folio =3D pfn_swap_entry_folio(entry); + if (leafent_is_migration(entry)) + folio =3D leafent_to_folio(entry); } /* return 1 if the page is an normal ksm page or KSM-placed zero page */ found =3D (folio && folio_test_ksm(folio)) || is_ksm_zero_pte(pte); diff --git a/mm/memory-failure.c b/mm/memory-failure.c index acc35c881547..42cd4079c660 100644 --- a/mm/memory-failure.c +++ b/mm/memory-failure.c @@ -691,10 +691,10 @@ static int check_hwpoisoned_entry(pte_t pte, unsigned= long addr, short shift, if (pte_present(pte)) { pfn =3D pte_pfn(pte); } else { - swp_entry_t swp =3D pte_to_swp_entry(pte); + const leaf_entry_t entry =3D leafent_from_pte(pte); =20 - if (is_hwpoison_entry(swp)) - pfn =3D swp_offset_pfn(swp); + if (leafent_is_hwpoison(entry)) + pfn =3D leafent_to_pfn(entry); } =20 if (!pfn || pfn !=3D poisoned_pfn) diff --git a/mm/memory.c b/mm/memory.c index 3d118618bdeb..f7b837c3c4dd 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -902,7 +902,8 @@ static void restore_exclusive_pte(struct vm_area_struct= *vma, static int try_restore_exclusive_pte(struct vm_area_struct *vma, unsigned long addr, pte_t *ptep, pte_t orig_pte) { - struct page *page =3D pfn_swap_entry_to_page(pte_to_swp_entry(orig_pte)); + const leaf_entry_t entry =3D leafent_from_pte(orig_pte); + struct page *page =3D leafent_to_page(entry); struct folio *folio =3D page_folio(page); =20 if (folio_trylock(folio)) { diff --git a/mm/mempolicy.c b/mm/mempolicy.c index f5b05754e6d5..48c85642fbe2 100644 --- a/mm/mempolicy.c +++ b/mm/mempolicy.c @@ -705,7 +705,9 @@ static int queue_folios_pte_range(pmd_t *pmd, unsigned = long addr, if (pte_none(ptent)) continue; if (!pte_present(ptent)) { - if (is_migration_entry(pte_to_swp_entry(ptent))) + const leaf_entry_t entry =3D leafent_from_pte(ptent); + + if (leafent_is_migration(entry)) qp->nr_failed++; continue; } diff --git a/mm/migrate.c b/mm/migrate.c index 8f2c3c7d87ba..22e52e90cb21 100644 --- a/mm/migrate.c +++ b/mm/migrate.c @@ -483,7 +483,7 @@ void migration_entry_wait(struct mm_struct *mm, pmd_t *= pmd, spinlock_t *ptl; pte_t *ptep; pte_t pte; - swp_entry_t entry; + leaf_entry_t entry; =20 ptep =3D pte_offset_map_lock(mm, pmd, address, &ptl); if (!ptep) @@ -495,8 +495,8 @@ void migration_entry_wait(struct mm_struct *mm, pmd_t *= pmd, if (pte_none(pte) || pte_present(pte)) goto out; =20 - entry =3D pte_to_swp_entry(pte); - if (!is_migration_entry(entry)) + entry =3D leafent_from_pte(pte); + if (!leafent_is_migration(entry)) goto out; =20 migration_entry_wait_on_locked(entry, ptl); diff --git a/mm/migrate_device.c b/mm/migrate_device.c index 5cb5ac2f0290..490560245ab6 100644 --- a/mm/migrate_device.c +++ b/mm/migrate_device.c @@ -282,7 +282,7 @@ static int migrate_vma_collect_pmd(pmd_t *pmdp, unsigned long mpfn =3D 0, pfn; struct folio *folio; struct page *page; - swp_entry_t entry; + leaf_entry_t entry; pte_t pte; =20 pte =3D ptep_get(ptep); @@ -301,11 +301,11 @@ static int migrate_vma_collect_pmd(pmd_t *pmdp, * page table entry. Other special swap entries are not * migratable, and we ignore regular swapped page. */ - entry =3D pte_to_swp_entry(pte); - if (!is_device_private_entry(entry)) + entry =3D leafent_from_pte(pte); + if (!leafent_is_device_private(entry)) goto next; =20 - page =3D pfn_swap_entry_to_page(entry); + page =3D leafent_to_page(entry); pgmap =3D page_pgmap(page); if (!(migrate->flags & MIGRATE_VMA_SELECT_DEVICE_PRIVATE) || @@ -331,7 +331,7 @@ static int migrate_vma_collect_pmd(pmd_t *pmdp, =20 mpfn =3D migrate_pfn(page_to_pfn(page)) | MIGRATE_PFN_MIGRATE; - if (is_writable_device_private_entry(entry)) + if (leafent_is_device_private_write(entry)) mpfn |=3D MIGRATE_PFN_WRITE; } else { pfn =3D pte_pfn(pte); diff --git a/mm/mprotect.c b/mm/mprotect.c index 2134e28257d0..3358a3774db1 100644 --- a/mm/mprotect.c +++ b/mm/mprotect.c @@ -317,11 +317,11 @@ static long change_pte_range(struct mmu_gather *tlb, pages++; } } else { - swp_entry_t entry =3D pte_to_swp_entry(oldpte); + leaf_entry_t entry =3D leafent_from_pte(oldpte); pte_t newpte; =20 - if (is_writable_migration_entry(entry)) { - struct folio *folio =3D pfn_swap_entry_folio(entry); + if (leafent_is_migration_write(entry)) { + const struct folio *folio =3D leafent_to_folio(entry); =20 /* * A protection check is difficult so @@ -335,7 +335,7 @@ static long change_pte_range(struct mmu_gather *tlb, newpte =3D swp_entry_to_pte(entry); if (pte_swp_soft_dirty(oldpte)) newpte =3D pte_swp_mksoft_dirty(newpte); - } else if (is_writable_device_private_entry(entry)) { + } else if (leafent_is_device_private_write(entry)) { /* * We do not preserve soft-dirtiness. See * copy_nonpresent_pte() for explanation. diff --git a/mm/page_vma_mapped.c b/mm/page_vma_mapped.c index b69b817ad180..52755d58ddc5 100644 --- a/mm/page_vma_mapped.c +++ b/mm/page_vma_mapped.c @@ -49,7 +49,7 @@ static bool map_pte(struct page_vma_mapped_walk *pvmw, pm= d_t *pmdvalp, if (is_migration) return false; } else if (!is_migration) { - swp_entry_t entry; + leaf_entry_t entry; =20 /* * Handle un-addressable ZONE_DEVICE memory. @@ -67,9 +67,9 @@ static bool map_pte(struct page_vma_mapped_walk *pvmw, pm= d_t *pmdvalp, * For more details on device private memory see HMM * (include/linux/hmm.h or mm/hmm.c). */ - entry =3D pte_to_swp_entry(ptent); - if (!is_device_private_entry(entry) && - !is_device_exclusive_entry(entry)) + entry =3D leafent_from_pte(ptent); + if (!leafent_is_device_private(entry) && + !leafent_is_device_exclusive(entry)) return false; } spin_lock(*ptlp); diff --git a/mm/pagewalk.c b/mm/pagewalk.c index fc2576235fde..6cace2c8814a 100644 --- a/mm/pagewalk.c +++ b/mm/pagewalk.c @@ -1000,11 +1000,10 @@ struct folio *folio_walk_start(struct folio_walk *f= w, goto found; } } else if (!pte_none(pte)) { - swp_entry_t entry =3D pte_to_swp_entry(pte); + const leaf_entry_t entry =3D leafent_from_pte(pte); =20 - if ((flags & FW_MIGRATION) && - is_migration_entry(entry)) { - page =3D pfn_swap_entry_to_page(entry); + if ((flags & FW_MIGRATION) && leafent_is_migration(entry)) { + page =3D leafent_to_page(entry); expose_page =3D false; goto found; } diff --git a/mm/rmap.c b/mm/rmap.c index 99203bf7d346..061d988b6ddf 100644 --- a/mm/rmap.c +++ b/mm/rmap.c @@ -1969,7 +1969,7 @@ static bool try_to_unmap_one(struct folio *folio, str= uct vm_area_struct *vma, if (likely(pte_present(pteval))) { pfn =3D pte_pfn(pteval); } else { - pfn =3D swp_offset_pfn(pte_to_swp_entry(pteval)); + pfn =3D leafent_to_pfn(pte_to_swp_entry(pteval)); VM_WARN_ON_FOLIO(folio_test_hugetlb(folio), folio); } =20 @@ -2368,7 +2368,7 @@ static bool try_to_migrate_one(struct folio *folio, s= truct vm_area_struct *vma, if (likely(pte_present(pteval))) { pfn =3D pte_pfn(pteval); } else { - pfn =3D swp_offset_pfn(pte_to_swp_entry(pteval)); + pfn =3D leafent_to_pfn(pte_to_swp_entry(pteval)); VM_WARN_ON_FOLIO(folio_test_hugetlb(folio), folio); } =20 @@ -2453,8 +2453,11 @@ static bool try_to_migrate_one(struct folio *folio, = struct vm_area_struct *vma, folio_mark_dirty(folio); writable =3D pte_write(pteval); } else { + const leaf_entry_t entry =3D leafent_from_pte(pteval); + pte_clear(mm, address, pvmw.pte); - writable =3D is_writable_device_private_entry(pte_to_swp_entry(pteval)); + + writable =3D leafent_is_device_private_write(entry); } =20 VM_WARN_ON_FOLIO(writable && folio_test_anon(folio) && --=20 2.51.0 From nobody Sat Feb 7 17:56:00 2026 Received: from mx0a-00069f02.pphosted.com (mx0a-00069f02.pphosted.com [205.220.165.32]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0FA54283CBF; Mon, 3 Nov 2025 12:39:21 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=205.220.165.32 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1762173564; cv=fail; b=UsIsZi9QXghtn08dcfLlMMl1oZLfSpc3O/aMjCcn1zIKRLCwvaoQedT7kQGS5hGLqgd/1IccpHct4/HDlqSgQXHGWcHmMaIINYprCsNIFKElfKEh/21RlgKKcPoGnD2CbM6XPSbgWVfVVbNOKRK7RQLggsneCfEfHeJLOMXY+6w= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1762173564; c=relaxed/simple; bh=YRY/pYAYRZhm3vMIIezbxAmRnvw4oXHtJd2+fjTtIEs=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: Content-Type:MIME-Version; b=PVm/VUJAOK+aEU04Bzxxn46GzhdtGX+KxuWSQ1RPfxK3Thdd5PoKwUJG7ysZEwMh52SLfXnIhk2PAWhyYn8AKEQbdeHy/44TCPqeEUvbGLMDAcBWrUUOGQEtIYW3kGYQPPg/x1ZDBBfU3PF1BDVrw3KXhW4lWKd3ltePP/x+FiE= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=oracle.com; spf=pass smtp.mailfrom=oracle.com; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b=SiiylVVZ; dkim=pass (1024-bit key) header.d=oracle.onmicrosoft.com header.i=@oracle.onmicrosoft.com header.b=KEfqV2y4; arc=fail smtp.client-ip=205.220.165.32 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=oracle.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=oracle.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b="SiiylVVZ"; dkim=pass (1024-bit key) header.d=oracle.onmicrosoft.com header.i=@oracle.onmicrosoft.com header.b="KEfqV2y4" Received: from pps.filterd (m0333521.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 5A3CTgS4019251; Mon, 3 Nov 2025 12:33:25 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=cc :content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s= corp-2025-04-25; bh=eQP2gKV9s5EsyeOy3wWyaAvJjDWb1F7KV1e+Mn3O398=; b= SiiylVVZp4+3xML5eyhzxDWDWp658OrE6JQ63VUWydGj3jGt0aDsXZlda4uWYEhc u4VWfYBKtrP5ABIYAzYuz+mUzmIwkSjvo63OSx5SLmraTGbk7UOLyOIHuwW2wp0G nj5Uo6jStZ1bL+8Jl97B1LHd0PBE+/WGz5TavDUt1ChfR/TKgGPlU8NYZI6iXHgT TfeDvCz+GzQSyS5cEW+1bN9yVucyqRkuFulXbTkex9KsEzgWj6O9A2O03GynIbdz OuHkDxduYi6fHhNoLG1jP4xNZZSKAnVpDRfxLB/hUTYwipooWyY4mmRCSv6Fu8Xm 1fw8EgGDuX4ha9t8JPhVAw== Received: from phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta02.appoci.oracle.com [147.154.114.232]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 4a6ven80bg-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 03 Nov 2025 12:33:25 +0000 (GMT) Received: from pps.filterd (phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (8.18.1.2/8.18.1.2) with ESMTP id 5A3Bx7kT021025; Mon, 3 Nov 2025 12:33:24 GMT Received: from sj2pr03cu001.outbound.protection.outlook.com (mail-westusazon11012047.outbound.protection.outlook.com [52.101.43.47]) by phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTPS id 4a58n83ev0-3 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 03 Nov 2025 12:33:24 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=jBPJaFqQs9K5y3AUThrc08Qvm9a+hZ3YEwlz5K7G/3A2qxa8cAyQhYS1Dy/IU3nsKVJvEb96Esnc7ruC0DC4nEl68wrPf1RmvCleYSa614QpZCLnmlknkwHeBsWk/EsRAb5IpX9RY4iQ0IVnOclgVsIIlj1g+JNUBtOqWaWROo+yOM/kEGlwmA4ssc/BSV2wD34VGhwmmgXTDcbJI7Ogk95OwP2QmbL9RHZInzqlYZBCr2stHttFmNTv095+NVUuRFRQrRU9FqAw16EXkVjx/GU0nwfWl8qbSTBi3o5g4A7jRgwc+yCNCde9ODo3jTWd1dKzhAA7vWOCdFvbF6S56Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=eQP2gKV9s5EsyeOy3wWyaAvJjDWb1F7KV1e+Mn3O398=; b=PFsRSTCR7Z5NLMmCIDs3xNE90kgZBHODt1e4JCz+lZy07QG5mFZFAMr1TkNiMU+RNvr4YWZwPgNkKXKwjgDWyhCQs3U+uTR4iGullRh10BpWH8NE+263bGhBaqg88sLUWbH0PcpwmtKUDFt5A+6awuDj5b8VZ9kNZXj+CDpMA+ttmy5ieRMOeGsxTHZbQZ2Z5Q6CgTdYAzBiUPPeVdYZ0HOo3fbrm50R7M+5xndaOwvWJjpbnIZYW15JEjxtyWcyZc07C9XlOj4M/+H3r4IezjT0qWgDcxHQoRgxEr3fNaNHbRDnVpjTigppLPekYv5hjvQBmiCbQMfVb86ei1NymQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=eQP2gKV9s5EsyeOy3wWyaAvJjDWb1F7KV1e+Mn3O398=; b=KEfqV2y4RdrVBcvJ+E87/TF0m8e5njx8Ce1iRbvTYTRljJ47+ojMOKUGUhL33/n4iROj4QF9OZjIOoOWr42WObuLsaKqzk/wJLwBImZrrnEBAeK/8PcTRClhDirWP37lYIPgnO6StG2C+jhijQLCD0oHvjH8IBDNbbFinaSEsQ4= Received: from DM4PR10MB8218.namprd10.prod.outlook.com (2603:10b6:8:1cc::16) by DS4PR10MB997575.namprd10.prod.outlook.com (2603:10b6:8:31e::5) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9275.16; Mon, 3 Nov 2025 12:32:55 +0000 Received: from DM4PR10MB8218.namprd10.prod.outlook.com ([fe80::2650:55cf:2816:5f2]) by DM4PR10MB8218.namprd10.prod.outlook.com ([fe80::2650:55cf:2816:5f2%2]) with mapi id 15.20.9253.018; Mon, 3 Nov 2025 12:32:55 +0000 From: Lorenzo Stoakes To: Andrew Morton Cc: Christian Borntraeger , Janosch Frank , Claudio Imbrenda , David Hildenbrand , Alexander Gordeev , Gerald Schaefer , Heiko Carstens , Vasily Gorbik , Sven Schnelle , Peter Xu , Alexander Viro , Christian Brauner , Jan Kara , Arnd Bergmann , Zi Yan , Baolin Wang , "Liam R . Howlett" , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Lance Yang , Muchun Song , Oscar Salvador , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Matthew Brost , Joshua Hahn , Rakie Kim , Byungchul Park , Gregory Price , Ying Huang , Alistair Popple , Axel Rasmussen , Yuanchu Xie , Wei Xu , Kemeng Shi , Kairui Song , Nhat Pham , Baoquan He , Chris Li , SeongJae Park , Matthew Wilcox , Jason Gunthorpe , Leon Romanovsky , Xu Xin , Chengming Zhou , Jann Horn , Miaohe Lin , Naoya Horiguchi , Pedro Falcato , Pasha Tatashin , Rik van Riel , Harry Yoo , Hugh Dickins , linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-s390@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, linux-arch@vger.kernel.org, damon@lists.linux.dev Subject: [PATCH 16/16] mm: replace remaining pte_to_swp_entry() with leafent_from_pte() Date: Mon, 3 Nov 2025 12:31:57 +0000 Message-ID: <1518bb0d0e40ace2012e64590288046aef03781e.1762171281.git.lorenzo.stoakes@oracle.com> X-Mailer: git-send-email 2.51.0 In-Reply-To: References: Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: LO2P265CA0344.GBRP265.PROD.OUTLOOK.COM (2603:10a6:600:d::20) To DM4PR10MB8218.namprd10.prod.outlook.com (2603:10b6:8:1cc::16) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DM4PR10MB8218:EE_|DS4PR10MB997575:EE_ X-MS-Office365-Filtering-Correlation-Id: 398e10a1-4af1-458f-5b1a-08de1ad51d31 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|376014|7416014|366016|1800799024; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?jNWPGwyhe9gMlURguxQ6hVvjyD0vJxU48MGOaZAk+IcsaISzzS+5nuxWm8ni?= =?us-ascii?Q?WjEKgk24DNelBZj4zLf3lxyp6jHZAzgrkz2pyeiQj6JhQAHRWFb5CFqASknG?= =?us-ascii?Q?OABu6Fw/y8YY3yzp84eZYg2MO/ScEfn2qBIC6lC/y8IQ/pi7EpEhpH4OUkAA?= =?us-ascii?Q?pTIZelyLdhJzoc7DWoHZCxzrjqJGrKvV4Y5UWR5qBl41+bhG/SV8E6Cneg4d?= =?us-ascii?Q?EynI3Hn37mdZVLquVXwzRpoh0+1ZyfYZNNbSW8RBZb2M9C5R/N7jpUAXXCiH?= =?us-ascii?Q?3od2W+BBVh7hi9aWbxvnAfQa8LW8bZ9hB17B4co+Rk+f0q6D2tJVS4dcj37u?= =?us-ascii?Q?N5GH7bKwuH0CsW+A7P1xxJZiLaeEY+pj+m/yArLOq44KCxUcO5JapMJtdAws?= =?us-ascii?Q?nFORKUZMNNRY83HxhuKvz/PaflfrEE7rt9HkzpAZTMLqTM4MA1GAzLjCuTGv?= =?us-ascii?Q?K4A1MMhugn8xIozT5sFbxn1RkZqGXtHvbNZLOrVnqUnTaf9G8UelMjK0IgTm?= =?us-ascii?Q?x1W+gii26pw65nusaQI/xfJWU8lxPINb7TyznTAfR9lWmreKVP0IQDr8jvY4?= =?us-ascii?Q?rk4CsiCM1nGyk4Jrd8ALsUJEpFtu0W2cwxLsecGCQgUZw0kVTQeK2Uk2qnJN?= =?us-ascii?Q?knHJ7Rw8zIjLAkzxG2ADZ5U5EKlzCQUj/XHDNV3UfCPVhpQW6EkP5jAVrNlv?= =?us-ascii?Q?VZXLMn73ZqeRMwz7210Ls4yN4kvROyod9rw9IMQac8g4EdXXvEe+AsA27/3X?= =?us-ascii?Q?sefSSU+XKZEefdpNIReYLqu7rgNX9/XipdQRQ6iYqfjN02WCIla1hKCNK1dD?= =?us-ascii?Q?jPLj7GmP5zBemAkowMTlmaXBQpknzcQoE0B3fsrsyLfNe6Ncod1QPJRNU1Kh?= =?us-ascii?Q?qbk6DQFalXCA5K+D7fnIdERcOF+SquOtn1CIVobCHu3nWiQww3az3GHKoTBP?= =?us-ascii?Q?oBO3hdl0ZnDutT+FJsz3V35QA0bn8cpkUJg4mwVEn4VMasK6fsjD3+O9MHeO?= =?us-ascii?Q?Uk8G40nOoGuYGnNrXgfsDt24brs52YJBviCEFbZAOHijCo0sd+4h78nMxA1Z?= =?us-ascii?Q?y4M0UHT4999nXE48xcbkqcsh3CP0c7x/53I7777i2C3WRjYcBoggllvMZwye?= =?us-ascii?Q?0UmjgpQOQBQa69tSgVOmYehmRJ7OkpPjxbCM7Dl2GTOMmBUqohJujCOIjnk6?= =?us-ascii?Q?zGOucBD0m6d888zSJjgxrYD5WPv9Yq6f1a7m/PWtWtvuHvq4m0ML77+vZEsE?= =?us-ascii?Q?49BoXIJttxoz6fMy/gN5sLiNkkFA+U+CtMpb0RBYK+XjO7kSZcMBeXmydghi?= =?us-ascii?Q?z/qXdMRmrwZqxFSY816K8yf7c8cTtUk1Nm+/e5AUb/EGp2C9172nnXCI84Gx?= =?us-ascii?Q?auIluFN7fpdZCGR1+/a5VhyztIboGRIeS350uOi2i4LAvMQxp2ll9fBdpWOg?= =?us-ascii?Q?lW8uVOKpZyhMBOwN/CRdYkfgOm1jH6m/?= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DM4PR10MB8218.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(376014)(7416014)(366016)(1800799024);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?eaA7JsCZu2G0sWIkTeOeQTAJ3QdpK97M1Vvn2rMe1o12w7OkCbA0fgWuAxWF?= =?us-ascii?Q?Uz9eKVgsTENvK0Fg52hJ1/e0x2AwjXEkv2N3TWw3qM+Q5ZUKN8eM28tfrzS0?= =?us-ascii?Q?2UfisSTILdMah4U5X8lNvwp3rCQbA3R6vTvqrtuONfFgXSm0tHF23wkRhUJV?= =?us-ascii?Q?m1MAHhVn5S2n+1fDcnGC57Or2Sx7s4Zuf6elfPf5PHV7jnm6+l0Z16q8lEOJ?= =?us-ascii?Q?MoBbcan3UosbNRn/Z11/wfyjrHn2nTeJxfbwtGHp3Yq19EEpzf7udsfvPIUQ?= =?us-ascii?Q?diVg7x1nYm9A0kW0CumNltuMeffsch1NLUxwO0TybNM/tLcBTwOq6/KAeQtW?= =?us-ascii?Q?QCHdH5Z8pHLsvi+tZ27RQmrkGUSwFin6ilfWhZ4PqtUXWlk3o0ANotmKMXUL?= =?us-ascii?Q?7VvHfL3S7bX/VD2noJSS/as4v//TomiEt1UijFmKm5WTuJ2nqZj62kIFdcvr?= =?us-ascii?Q?ZryIOqTg35pR56Z/hUoAJSyloT2UW3vmA+qJkDYRBHTr6Q/S9U711MBvY+lx?= =?us-ascii?Q?KXuwai5FzCDILSBcTGV4cZJHQ5aZ2tk/hm4UZhM/4oJNRgMrHgziRgLVXUD/?= =?us-ascii?Q?QxoA/tZdWVKHPq6cYclzciKi1rnrguHtArSiJXpEACmhhE7iIOYfldo6JZ/6?= =?us-ascii?Q?+zLiX6GGPr3PGiFK4DS2gHnCSUK3gDz3fOjdgGAQH+qR5ria0auXYbP4xljc?= =?us-ascii?Q?Tc7LxkTgir6Klhp8TcUZDnRlsIP2Bi3RN/ho+0hOxjY64vQzKjRrUJO0cPOu?= =?us-ascii?Q?hFyB6R/+3sS4Koph7NNIbU18UIhsXuxMbbaLn4bs4kmfcmkIBwFLz0g4zu0x?= =?us-ascii?Q?j8WuJ9iZn/cOMLISnpKy7Y+25QNHOsBpgxQIiNyQAgJab7nYVjrSiTv4UD7Q?= =?us-ascii?Q?wn4AtHjN4DLjXkO9No19xsupN6FyMoqCyDKuM4FZs5deeuX0D9vGSqkLWPqT?= =?us-ascii?Q?rqx+f7dzcAXx0sJjqC83CEgPWUj8CD2nuxiTGj+W/QhCuRrwZobhUlrWKlav?= =?us-ascii?Q?u52eUiOi6Wc/ZQSXXozK59oCIqU5PYdXJlSiEgRMLZU9YmItYkgOGwpPnsad?= =?us-ascii?Q?gq+LIgHDopjgkEtlD7CA5QdMH7891DPpzyhJd3Nk9qrmkhWCnyuVs2ICX1rh?= =?us-ascii?Q?t0NKbcxbA3sbTAOG3NdDFrgkf0/jrrzTKD87nTkZbUb/FcmaNzVD5Q2eSWBe?= =?us-ascii?Q?Lx6FE7+SIriiTSDFFNDSh2D8qg+B4RkzuKTQpcEmLAku+ykxNCD7Bjpty2ny?= =?us-ascii?Q?nJL7BNVt6LAxSo9je7fWeAwBcoEzXi2V9bHgzwB3XYBKiBuPud9Py52E8r+9?= =?us-ascii?Q?Ajhq3eMmYcoaMnHKfN1s5tgaQ4ZLKTcTmCdsJManul6pZSLSsPv6cAabXcgQ?= =?us-ascii?Q?IYCeNA5CgQapXMZU5B2hqwjbtY0OSkEfu74VILzUuDXnIANP4gQVbESkoUQ1?= =?us-ascii?Q?lXvigEhaIW3M9sXx4sNAeUDs1ZdtN07hHLTPo2ggQ0f9VfuciVg50Teq6/7d?= =?us-ascii?Q?MkNxV+pLbHlJi7/SnxwNHGra1sUUrxYtI03LkQBkIa032rvGT2T1DHsYmjay?= =?us-ascii?Q?mq9+VRjz5tECDO5WGEIhHRIzp66SVIb/EBD3SYvcjZhpQcjyYXtpwzzRtacC?= =?us-ascii?Q?GQ=3D=3D?= X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: yWY76NyAX1hL5cMZoQoXtiva84MoAanLtzs305g3ASM7f9086uUqsbFO5aglUzjoHVTRMAOHjLZPDD8eGZF+6Lt2E/EMx5Q9TulRY5MLi45PkxRwrqnoX8psJysW8aYb4oe2rRF2OZlS3PITzmz6u3wrH0ewLju6FrHVe49XIeJQPoospKn65Ll44itfQbwq6B7tYQ7j/SzTA8o+RX7Df0bfLW3N0o1UV/jpI1p8zTYzxLh3wVAOvg1V1YILLGV9E7Z0I1Xwj8ZVLMrdooZFx8SgqqWh/Ros7iGjds2/EX5OI7yYR3qY7W2ielXNuTKpxEKr7TY3kqtHl+EC6Y4qi2msjOxGs2kfVjqP/0NgL0l0Uru0WCmNQezohApv4cMZooKWhdD8HnxlStSINIYvVxhSA8r989wqej4KHPIAQXo3HdsA8bMJZEiX5NCL/B34CC/cq/rfoYn9kLgsQ8Ocze5Y/0+JCLJ6U8lPEiSvn1uq114epJ4ioy3CnLzBYAX4zWiAzqaHY9TU0mQwBTkwHAuBMbBxxLZo+gA5SGs67r77nFfbyaAYBxY1PStHC2hyBvl9gAJwWWaWUkSlNYxEjGzbGmrzDko3I+flgi1zS2E= X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: 398e10a1-4af1-458f-5b1a-08de1ad51d31 X-MS-Exchange-CrossTenant-AuthSource: DM4PR10MB8218.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 03 Nov 2025 12:32:55.4940 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: wx9pgxsv4iKfUP+Gi1K9AqU7+bNuSulHVYRan66pDo8ZldHMolw9vUQ3VCW+1TMWPdZz4TQo/ZYFrvQDHSrx4rMFpwoLpCqvZC7iuhjcIDc= X-MS-Exchange-Transport-CrossTenantHeadersStamped: DS4PR10MB997575 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1121,Hydra:6.1.9,FMLib:17.12.100.49 definitions=2025-11-03_01,2025-11-03_01,2025-10-01_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 phishscore=0 bulkscore=0 malwarescore=0 spamscore=0 suspectscore=0 mlxscore=0 adultscore=0 mlxlogscore=999 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2510240000 definitions=main-2511030114 X-Proofpoint-GUID: 21ec7nF9pXtmHH-jFlT8pC53_-e4P0AY X-Proofpoint-ORIG-GUID: 21ec7nF9pXtmHH-jFlT8pC53_-e4P0AY X-Proofpoint-Spam-Details-Enc: AW1haW4tMjUxMTAzMDExNCBTYWx0ZWRfX08y3nCdPXBlg MT8honoUQYqB8yyTqMmaW1GavDyb9453eDdIy1PGuDYQnz9xTnCWP596S0a888gE5QJ0+vkFTqc +lgJalQzpYFImRb9S5uF4lMFPEtB1eQo8qhsLZaadd39Sz/uaruj2LhS8ngEQgYCEoBNgkdYxEG gmQ8tHK8OPcr5TKebfZVZlgt5L8Y5hJvdUT83fl6P/adWQgsZXObXAOdjI95eC8Mxbte5lyBMkf nvHr3U3SBxfbl0PvyxRpuiuy8p/paup+RCCqMYMzxv6jDH21REfIrt10Hlol3ZmslsndE63wajX iOB/8pbn/us2+s3MJWdtmqlp2xli5JxSZ7crCLaZE8K/bnSWmfhopv79omiUM5CdWyDsP1EVhZJ 87ZcR6lx0xUjuOq9tUcH6UzZusY+Cg== X-Authority-Analysis: v=2.4 cv=MvBfKmae c=1 sm=1 tr=0 ts=6908a115 cx=c_pps a=OOZaFjgC48PWsiFpTAqLcw==:117 a=OOZaFjgC48PWsiFpTAqLcw==:17 a=6eWqkTHjU83fiwn7nKZWdM+Sl24=:19 a=z/mQ4Ysz8XfWz/Q5cLBRGdckG28=:19 a=lCpzRmAYbLLaTzLvsPZ7Mbvzbb8=:19 a=xqWC_Br6kY4A:10 a=6UeiqGixMTsA:10 a=GoEa3M9JfhUA:10 a=VkNPw1HP01LnGYTKEx00:22 a=yPCof4ZbAAAA:8 a=o1UEzff51pvCjpypyTAA:9 a=UhEZJTgQB8St2RibIkdl:22 a=Z5ABNNGmrOfJ6cZ5bIyy:22 a=QOGEsqRv6VhmHaoFNykA:22 Content-Type: text/plain; charset="utf-8" There are straggler invocations of pte_to_swp_entry() lying around, replace all of these with the leaf entry equivalent - leafent_from_pte(). With those removed, eliminate pte_to_swp_entry() altogether. No functional change intended. Signed-off-by: Lorenzo Stoakes --- include/linux/leafops.h | 7 ++++++- include/linux/swapops.h | 13 ------------- mm/debug_vm_pgtable.c | 2 +- mm/internal.h | 7 +++++-- mm/memory-failure.c | 2 +- mm/memory.c | 16 ++++++++-------- mm/migrate.c | 2 +- mm/mincore.c | 4 +++- mm/rmap.c | 8 ++++++-- mm/swapfile.c | 5 +++-- 10 files changed, 34 insertions(+), 32 deletions(-) diff --git a/include/linux/leafops.h b/include/linux/leafops.h index b74d406ba648..ba970d4e2e17 100644 --- a/include/linux/leafops.h +++ b/include/linux/leafops.h @@ -66,11 +66,16 @@ static inline leaf_entry_t leafent_mk_none(void) */ static inline leaf_entry_t leafent_from_pte(pte_t pte) { + leaf_entry_t arch_entry; + if (pte_present(pte)) return leafent_mk_none(); =20 + pte =3D pte_swp_clear_flags(pte); + arch_entry =3D __pte_to_swp_entry(pte); + /* Temporary until swp_entry_t eliminated. */ - return pte_to_swp_entry(pte); + return swp_entry(__swp_type(arch_entry), __swp_offset(arch_entry)); } =20 /** diff --git a/include/linux/swapops.h b/include/linux/swapops.h index 3d02b288c15e..8cfc966eae48 100644 --- a/include/linux/swapops.h +++ b/include/linux/swapops.h @@ -107,19 +107,6 @@ static inline pgoff_t swp_offset(swp_entry_t entry) return entry.val & SWP_OFFSET_MASK; } =20 -/* - * Convert the arch-dependent pte representation of a swp_entry_t into an - * arch-independent swp_entry_t. - */ -static inline swp_entry_t pte_to_swp_entry(pte_t pte) -{ - swp_entry_t arch_entry; - - pte =3D pte_swp_clear_flags(pte); - arch_entry =3D __pte_to_swp_entry(pte); - return swp_entry(__swp_type(arch_entry), __swp_offset(arch_entry)); -} - /* * Convert the arch-independent representation of a swp_entry_t into the * arch-dependent pte representation. diff --git a/mm/debug_vm_pgtable.c b/mm/debug_vm_pgtable.c index 181fa2b25625..4526be294ecf 100644 --- a/mm/debug_vm_pgtable.c +++ b/mm/debug_vm_pgtable.c @@ -1229,7 +1229,7 @@ static int __init init_args(struct pgtable_debug_args= *args) init_fixed_pfns(args); =20 /* See generic_max_swapfile_size(): probe the maximum offset */ - max_swap_offset =3D swp_offset(pte_to_swp_entry(swp_entry_to_pte(swp_entr= y(0, ~0UL)))); + max_swap_offset =3D swp_offset(leafent_from_pte(leafent_to_pte(swp_entry(= 0, ~0UL)))); /* Create a swp entry with all possible bits set while still being swap. = */ args->swp_entry =3D swp_entry(MAX_SWAPFILES - 1, max_swap_offset); /* Create a non-present migration entry. */ diff --git a/mm/internal.h b/mm/internal.h index e450a34c37dd..0af87f6c2889 100644 --- a/mm/internal.h +++ b/mm/internal.h @@ -334,7 +334,7 @@ unsigned int folio_pte_batch(struct folio *folio, pte_t= *ptep, pte_t pte, */ static inline pte_t pte_move_swp_offset(pte_t pte, long delta) { - swp_entry_t entry =3D pte_to_swp_entry(pte); + const leaf_entry_t entry =3D leafent_from_pte(pte); pte_t new =3D __swp_entry_to_pte(__swp_entry(swp_type(entry), (swp_offset(entry) + delta))); =20 @@ -389,11 +389,14 @@ static inline int swap_pte_batch(pte_t *start_ptep, i= nt max_nr, pte_t pte) =20 cgroup_id =3D lookup_swap_cgroup_id(entry); while (ptep < end_ptep) { + leaf_entry_t entry; + pte =3D ptep_get(ptep); =20 if (!pte_same(pte, expected_pte)) break; - if (lookup_swap_cgroup_id(pte_to_swp_entry(pte)) !=3D cgroup_id) + entry =3D leafent_from_pte(pte); + if (lookup_swap_cgroup_id(entry) !=3D cgroup_id) break; expected_pte =3D pte_next_swp_offset(expected_pte); ptep++; diff --git a/mm/memory-failure.c b/mm/memory-failure.c index 42cd4079c660..0e64d070d27d 100644 --- a/mm/memory-failure.c +++ b/mm/memory-failure.c @@ -50,7 +50,7 @@ #include #include #include -#include +#include #include #include #include diff --git a/mm/memory.c b/mm/memory.c index f7b837c3c4dd..1c66ee83a7ab 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -1218,7 +1218,7 @@ copy_pte_range(struct vm_area_struct *dst_vma, struct= vm_area_struct *src_vma, spinlock_t *src_ptl, *dst_ptl; int progress, max_nr, ret =3D 0; int rss[NR_MM_COUNTERS]; - swp_entry_t entry =3D (swp_entry_t){0}; + leaf_entry_t entry =3D leafent_mk_none(); struct folio *prealloc =3D NULL; int nr; =20 @@ -1282,7 +1282,7 @@ copy_pte_range(struct vm_area_struct *dst_vma, struct= vm_area_struct *src_vma, dst_vma, src_vma, addr, rss); if (ret =3D=3D -EIO) { - entry =3D pte_to_swp_entry(ptep_get(src_pte)); + entry =3D leafent_from_pte(ptep_get(src_pte)); break; } else if (ret =3D=3D -EBUSY) { break; @@ -4456,13 +4456,13 @@ static struct folio *__alloc_swap_folio(struct vm_f= ault *vmf) { struct vm_area_struct *vma =3D vmf->vma; struct folio *folio; - swp_entry_t entry; + leaf_entry_t entry; =20 folio =3D vma_alloc_folio(GFP_HIGHUSER_MOVABLE, 0, vma, vmf->address); if (!folio) return NULL; =20 - entry =3D pte_to_swp_entry(vmf->orig_pte); + entry =3D leafent_from_pte(vmf->orig_pte); if (mem_cgroup_swapin_charge_folio(folio, vma->vm_mm, GFP_KERNEL, entry)) { folio_put(folio); @@ -4480,7 +4480,7 @@ static struct folio *__alloc_swap_folio(struct vm_fau= lt *vmf) static bool can_swapin_thp(struct vm_fault *vmf, pte_t *ptep, int nr_pages) { unsigned long addr; - swp_entry_t entry; + leaf_entry_t entry; int idx; pte_t pte; =20 @@ -4490,7 +4490,7 @@ static bool can_swapin_thp(struct vm_fault *vmf, pte_= t *ptep, int nr_pages) =20 if (!pte_same(pte, pte_move_swp_offset(vmf->orig_pte, -idx))) return false; - entry =3D pte_to_swp_entry(pte); + entry =3D leafent_from_pte(pte); if (swap_pte_batch(ptep, nr_pages, pte) !=3D nr_pages) return false; =20 @@ -4536,7 +4536,7 @@ static struct folio *alloc_swap_folio(struct vm_fault= *vmf) unsigned long orders; struct folio *folio; unsigned long addr; - swp_entry_t entry; + leaf_entry_t entry; spinlock_t *ptl; pte_t *pte; gfp_t gfp; @@ -4557,7 +4557,7 @@ static struct folio *alloc_swap_folio(struct vm_fault= *vmf) if (!zswap_never_enabled()) goto fallback; =20 - entry =3D pte_to_swp_entry(vmf->orig_pte); + entry =3D leafent_from_pte(vmf->orig_pte); /* * Get a list of all the (large) orders below PMD_ORDER that are enabled * and suitable for swapping THP. diff --git a/mm/migrate.c b/mm/migrate.c index 22e52e90cb21..567dfae4d9f8 100644 --- a/mm/migrate.c +++ b/mm/migrate.c @@ -534,7 +534,7 @@ void migration_entry_wait_huge(struct vm_area_struct *v= ma, unsigned long addr, p * lock release in migration_entry_wait_on_locked(). */ hugetlb_vma_unlock_read(vma); - migration_entry_wait_on_locked(pte_to_swp_entry(pte), ptl); + migration_entry_wait_on_locked(entry, ptl); return; } =20 diff --git a/mm/mincore.c b/mm/mincore.c index a1f48df5564e..a6194bbc0c25 100644 --- a/mm/mincore.c +++ b/mm/mincore.c @@ -202,7 +202,9 @@ static int mincore_pte_range(pmd_t *pmd, unsigned long = addr, unsigned long end, for (i =3D 0; i < step; i++) vec[i] =3D 1; } else { /* pte is a swap entry */ - *vec =3D mincore_swap(pte_to_swp_entry(pte), false); + const leaf_entry_t entry =3D leafent_from_pte(pte); + + *vec =3D mincore_swap(entry, false); } vec +=3D step; } diff --git a/mm/rmap.c b/mm/rmap.c index 061d988b6ddf..60c3cd70b6ea 100644 --- a/mm/rmap.c +++ b/mm/rmap.c @@ -1969,7 +1969,9 @@ static bool try_to_unmap_one(struct folio *folio, str= uct vm_area_struct *vma, if (likely(pte_present(pteval))) { pfn =3D pte_pfn(pteval); } else { - pfn =3D leafent_to_pfn(pte_to_swp_entry(pteval)); + const leaf_entry_t entry =3D leafent_from_pte(pteval); + + pfn =3D leafent_to_pfn(entry); VM_WARN_ON_FOLIO(folio_test_hugetlb(folio), folio); } =20 @@ -2368,7 +2370,9 @@ static bool try_to_migrate_one(struct folio *folio, s= truct vm_area_struct *vma, if (likely(pte_present(pteval))) { pfn =3D pte_pfn(pteval); } else { - pfn =3D leafent_to_pfn(pte_to_swp_entry(pteval)); + const leaf_entry_t entry =3D leafent_from_pte(pteval); + + pfn =3D leafent_to_pfn(entry); VM_WARN_ON_FOLIO(folio_test_hugetlb(folio), folio); } =20 diff --git a/mm/swapfile.c b/mm/swapfile.c index 82a8b5d7e8d0..86721fea1aa3 100644 --- a/mm/swapfile.c +++ b/mm/swapfile.c @@ -3201,8 +3201,9 @@ static int claim_swapfile(struct swap_info_struct *si= , struct inode *inode) */ unsigned long generic_max_swapfile_size(void) { - return swp_offset(pte_to_swp_entry( - swp_entry_to_pte(swp_entry(0, ~0UL)))) + 1; + const leaf_entry_t entry =3D swp_entry(0, ~0UL); + + return swp_offset(leafent_from_pte(leafent_to_pte(entry))) + 1; } =20 /* Can be overridden by an architecture for additional checks. */ --=20 2.51.0