From nobody Fri Dec 19 21:47:07 2025 Received: from mx0b-00069f02.pphosted.com (mx0b-00069f02.pphosted.com [205.220.177.32]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6D1F023C4F1; Sat, 8 Nov 2025 17:15:07 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=205.220.177.32 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1762622110; cv=fail; b=O5Ahq580UTQMUcMykiRZEKUQwxZ2BERgYIiq33ue3RLdD+TEyh01P8N/0EGnsQM/MSflDq9Epz1FmA892BeqpqekMbU2TnMltoi90jsHkgfCte2AoHw5bju7sxR145k/teaAdgdz6j7dn7bM72pzHItEUt1d/wc6fvmnEgCMRZ8= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1762622110; c=relaxed/simple; bh=HmcOTIlY6PYpGzxGc0yNCqnLEdtm7ax/JeDbe/Igp18=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: Content-Type:MIME-Version; b=l1upN7o5IsEUju8D56vmGeMKj4eZwixTTpqdP1FGbPpPV/Po+cHVlQq25rX1ZfoJcY9enkGSx/qt/nS68Xcnm07aZ8zaOx7zdCwe6mxjXSXcs+R3aIYwqrtLMGKFIhJ66GlLCOJ/s80a1c1TLm38wNX1U8wvS2dNVpnoWHcoQUQ= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=oracle.com; spf=pass smtp.mailfrom=oracle.com; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b=n6wnEALL; dkim=pass (1024-bit key) header.d=oracle.onmicrosoft.com header.i=@oracle.onmicrosoft.com header.b=q+LQvOt3; arc=fail smtp.client-ip=205.220.177.32 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=oracle.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=oracle.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b="n6wnEALL"; dkim=pass (1024-bit key) header.d=oracle.onmicrosoft.com header.i=@oracle.onmicrosoft.com header.b="q+LQvOt3" Received: from pps.filterd (m0333520.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 5A8GjTwk017440; Sat, 8 Nov 2025 17:08:58 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=cc :content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s= corp-2025-04-25; bh=IS+WCD4gx0ikSahGAzl3Mo97b95pzUAhU1bCbk4hhuM=; b= n6wnEALLLa6C78iesBxKpzXaw/hQ2gUeC9PZAMUhmRTQap01wU21WY2nLLvGvkxX R3mclPS3Yc1mh4eXU1ybCwmFQ47OtQWjSA6YSX6LrS/t+kLY1XUM/03iuciijoJ0 IdM7F1JuKrliVn/wqODe8xw/lC4/RfPQE1TTsOMI8oEtn8/I7Z/3a6yEdUhO71Y1 5DQFizytpf9v0bzPu6urMQ9O4/qrWag5v79v8zuJI6a6sOUPl/fKjqKBpy8qyhxD h1huQy7n1g+fAmFrmier8hbejGNzrRQQhsMw7cH41hSAc8jKpjy3JJPmZG0FNAwU 6kGGJWEL+VrGvUDSidDdXw== Received: from phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta03.appoci.oracle.com [138.1.37.129]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 4aa8se01ub-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Sat, 08 Nov 2025 17:08:57 +0000 (GMT) Received: from pps.filterd (phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (8.18.1.2/8.18.1.2) with ESMTP id 5A8B7Vn8040113; Sat, 8 Nov 2025 17:08:56 GMT Received: from mw6pr02cu001.outbound.protection.outlook.com (mail-westus2azon11012007.outbound.protection.outlook.com [52.101.48.7]) by phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTPS id 4a9va6pstt-2 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Sat, 08 Nov 2025 17:08:56 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=mOQyd6YVvMjo1wpYpoivODSda9bf9jaM7pcqvdAPIBg0EHufIuiOIOpi1TtQoXXpwMAf53DrRpp9yF5FQlrp9pb1kiCfOXXdlBZuSLBqHiv2qdWZ21tDLZuhrH60gv+fgvj6WVV3LfOAqBQPhWEuZD9FhF8UyDHcbEGuY2bPfGEUQ54hWhvz5KtyyVMBgw5hGlGjGUtGlR+xLyvnIc4IrbPsIOkinexe4AOQe5Rsq6MXCc7sQ3rKxiOHU0rdaoW2jWof1HzaDfBi4jH3VeRfln13zWcttM48kgPwPEDYzKk7zAujKC0mCq9XI6sJSliSlPkLbrReSQUV1j+K0tu6Dw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=IS+WCD4gx0ikSahGAzl3Mo97b95pzUAhU1bCbk4hhuM=; b=NF8ifJ2UIHAAhs1XIpOWFpnKC0JOiRkDjuc+Yn9+X24wC71S9jFkXrucAPlPOv7PVr3787URUYnkX7vUBvUciOsE39RURkEBivsO6XqQKj2sm2eAoqyp6ENTFeC2b1FrBbrh1ZMPXWTo8nnsjs9Cu0dTWTCobvpN1btXfKM42UDHYALOdejbwjt9c26d7i8M9b6lIxLtAH0/456vCsOzJ/FOPIvh+ukYtrdBvFl5jdnkJKeA9imOAdqVbCrH6EUipfPh1C4GNDD3I/ABxhkyXejqVBj8dn7jX2QVm0j53PSJJmHCATU3Hd9muqRqxjtpPXleUR+/VSk2sPAGFhdZeA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=IS+WCD4gx0ikSahGAzl3Mo97b95pzUAhU1bCbk4hhuM=; b=q+LQvOt3c5TD5SZw41jMBrh7iqT1G3D4/i3nqomTS7ZZZGNev/zw8tdCETsRjQ4N3OC4HsY+SgNqmNmGpW0uDVhz4/WUa83nH0ePGRZyUKwkDwNOReXj3zMYcWzkmW1R1UGRNF2BDlEEl5bqQh/EJ2YiESmG3XIfEgFBSjc0ZCU= Received: from DM4PR10MB8218.namprd10.prod.outlook.com (2603:10b6:8:1cc::16) by SJ5PPF04D2D7FA7.namprd10.prod.outlook.com (2603:10b6:a0f:fc02::785) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9298.15; Sat, 8 Nov 2025 17:08:51 +0000 Received: from DM4PR10MB8218.namprd10.prod.outlook.com ([fe80::2650:55cf:2816:5f2]) by DM4PR10MB8218.namprd10.prod.outlook.com ([fe80::2650:55cf:2816:5f2%7]) with mapi id 15.20.9298.010; Sat, 8 Nov 2025 17:08:51 +0000 From: Lorenzo Stoakes To: Andrew Morton Cc: Christian Borntraeger , Janosch Frank , Claudio Imbrenda , David Hildenbrand , Alexander Gordeev , Gerald Schaefer , Heiko Carstens , Vasily Gorbik , Sven Schnelle , Peter Xu , Alexander Viro , Christian Brauner , Jan Kara , Arnd Bergmann , Zi Yan , Baolin Wang , "Liam R . Howlett" , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Lance Yang , Muchun Song , Oscar Salvador , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Matthew Brost , Joshua Hahn , Rakie Kim , Byungchul Park , Gregory Price , Ying Huang , Alistair Popple , Axel Rasmussen , Yuanchu Xie , Wei Xu , Kemeng Shi , Kairui Song , Nhat Pham , Baoquan He , Chris Li , SeongJae Park , Matthew Wilcox , Jason Gunthorpe , Leon Romanovsky , Xu Xin , Chengming Zhou , Jann Horn , Miaohe Lin , Naoya Horiguchi , Pedro Falcato , Pasha Tatashin , Rik van Riel , Harry Yoo , Hugh Dickins , linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-s390@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, linux-arch@vger.kernel.org, damon@lists.linux.dev Subject: [PATCH v2 01/16] mm: correctly handle UFFD PTE markers Date: Sat, 8 Nov 2025 17:08:15 +0000 Message-ID: <0b50fd4b1d3241d0965e6b969fb49bcc14704d9b.1762621568.git.lorenzo.stoakes@oracle.com> X-Mailer: git-send-email 2.51.0 In-Reply-To: References: Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: LNXP265CA0015.GBRP265.PROD.OUTLOOK.COM (2603:10a6:600:5e::27) To DM4PR10MB8218.namprd10.prod.outlook.com (2603:10b6:8:1cc::16) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DM4PR10MB8218:EE_|SJ5PPF04D2D7FA7:EE_ X-MS-Office365-Filtering-Correlation-Id: 48973cbb-7c70-4736-cfce-08de1ee97d3c X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|1800799024|7416014|376014|366016; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?lf+ePcQWWISnPxJyUB/BgKrc/6AlgMwjUQ4DKA3bQZKF2hV2ZtsL+DAzzw6Q?= =?us-ascii?Q?VAMb98BERk6YkO8JoPyudh8L/b9rqKDvPZwSloiEITCethul08Kl3RBOWUN/?= =?us-ascii?Q?FfgygWbneo8OYpOMufiU7FEo+B6ZDWmXqGaEEUq9RtgBOuUTYw1egyW0bQwM?= =?us-ascii?Q?i6Z67Qga9/Sg10bGlI+PgYHzZ+Qp6EHOAAecJCnWD/TEzd8UikC1QGLRvfg2?= =?us-ascii?Q?w47w+jkcn97La2nqaW+ydwrktKzkMrzVwuCOgRa/02YIr3U8a9nHkiaypGGR?= =?us-ascii?Q?J5fM8JI4top+VmiOLHrIIM3XJv2GAqAaBrORHith8D95w6NftIMYFkUt+LgT?= =?us-ascii?Q?7KXWYeHLz33c8loiQJcoI+18K6OSktA8upC1zdePAvUy73L0TEEl1qMR7EcY?= =?us-ascii?Q?IVuBxTan9FGAm2libKn7fKDnanhQePHYB569VYQWK6kLqeL3RywKuUbFRxvr?= =?us-ascii?Q?zwQDA/D90zaLGRlsu3y3u06p0V92Luo36hyUMpFKb/GFXprtOy9pHaci9TX5?= =?us-ascii?Q?L1l9Gnqm7IPN0e7UkbZkxViRubUhIxh/qaL5eM/MjaHwbJMBKwW8lUQNMfTH?= =?us-ascii?Q?HLu+bY/hkkAbPPribPBillSHc4Ua4/GRTm/JWqR/maOiZ891RjM0dcSUZ+mA?= =?us-ascii?Q?E7SX3gEdUxUoAeUkckAI3W6/g4OyumnIzSDWW4mfYKn/llLM5wPd22RFaK+4?= =?us-ascii?Q?JHIoemPnQwsViLJkgq/8OblGI6Hcze28HgS3HaqN+5GPn57zGInmD7J3uMNM?= =?us-ascii?Q?ACLsuEyaE6CdpWSI3VHFgX8ORzGr10PMOYKdVaLvM8inA/YUt8/BllBzbEcu?= =?us-ascii?Q?017Z3Kk9BoaKceVkCzEPx5xrDZsNBcniXNCyI3lNcOVwSmkQcBPOxWZcYXU6?= =?us-ascii?Q?c/mdMIof7wc7F7w8LemORRHNIZ6HWcYFdvRoVOT4Xp/FKzjKQmx1u+bD4UiP?= =?us-ascii?Q?BswPJZE9y4j+kQj14qN21a03ONmDJZP6FTyArMStTykvPS3C7vjq1RJtrr6c?= =?us-ascii?Q?Re+qmYmNo73bP4Nwf99eN9KM2gQ5Qgt7NkaMaH5p4nIYxd0zCIhZxEIS+9RO?= =?us-ascii?Q?inJpiSz0z4+kWuGtotCZwYiuG+nmDXv3DlirduLDgAFA1hy65JH71EiWpXQp?= =?us-ascii?Q?TkMvyWbvPVQRr21N8pEulmJCarDN1tN3pNzzkGDQKLKENFzlOCmh2Q2RdGcM?= =?us-ascii?Q?3VX0Acg+MKS4HUhfOm46BgNYJJ0FLDAZRmyr38VUz2Y5+X/59742fs78bqVF?= =?us-ascii?Q?ng6UPjg3V4+E7rkfojY7r7CT2cZbRclS9BS2KjV461ujtxHZ0651fYWSquHt?= =?us-ascii?Q?oNOoqLnhLJEHbDSoWt3GT7SlY1CNkePeZd3U/dSsLRwaOOnd8eOlsKmh2Juv?= =?us-ascii?Q?CbHgaL2no2uRsccSR126gJBamBANi2ikKUyYVqlPyE1QNgbbcNs5OZTHZ/4g?= =?us-ascii?Q?R/owuygmV/FNXC88D2rgS7pkja+5RqU7?= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DM4PR10MB8218.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(1800799024)(7416014)(376014)(366016);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?8ugzTvX7vzE9PRab0ZWhgrrOTcluW4k89zTrTxPKIDcRCqlz9twOETRzTR6a?= =?us-ascii?Q?alu9sKIGV3wZihEw9fuKnbau2y3hFPrk4sILRhRMZho1GRYHPVA+Nf2LEKrM?= =?us-ascii?Q?b7AFGYikvUPL7kEkISda3A9XocCFMr9YWG563XEG/tZOkSpHwRrYXzs+5AFR?= =?us-ascii?Q?wSIkopvY86ED8Qosr4IHgbu8yfKq/osc/0RRE2DOO8X2kQzKSTJh6/xvRywl?= =?us-ascii?Q?yvQgiL2QAiyIEdARbGIbrgnbb/O4Js4HdiZqvQ5ncB8lsL26DLCXQRBv6bQD?= =?us-ascii?Q?Spu9u5IHMW2bUxBpxKBcq8qmZnveT94591cdBFt7E6c/U/iTtLwq4u2SDoZj?= =?us-ascii?Q?7cnlquBE/sKlotzAC4PccKstJtYqkoZnfSCMeTQIzzOmYauBEm7GsWx586Ga?= =?us-ascii?Q?40WnkZB8d0EIH+UtIXvYzrVNGQN8wXpiEc7w0tmbUF2bVGJDs7gOsAhFutLL?= =?us-ascii?Q?qSShJb1HAx9P3MJFe2cEK1FvHNDlAq+sjttS6Sy9kB3NNWJu+uR19rsO5zqU?= =?us-ascii?Q?84/pBXN1rs4WAzba6lETQSkAlZaYz/wg13Iu1/9kWE8y+l0jiaUAPZfObLg3?= =?us-ascii?Q?wv8b9L5b8/eg3TgsLBlnevOBzm20dwaa8v2Od8mmnLsTTEyjhEUxvzkzz5eV?= =?us-ascii?Q?m4mqwxPRablRz+lKh6q8kVDr8f9Op03YhwEuFgyrWSL95ZiPoRjln5x+A1gy?= =?us-ascii?Q?25/hfUoVnlSq46roqPMi7llAGEdfEi1NUcrq/umkKkOsVfoknJh5/lncOO3v?= =?us-ascii?Q?Yqn/sidAoEIkr8hpjXcwx+WkGr/yPh6jKeDtjEqpOwLpWRvGf7uLDbKF/YRr?= =?us-ascii?Q?gudLrwetBHg3qdgu9SyalL03S5PH0jglsoVJa3G3PJz1N74Oz9UhRZeKdrFx?= =?us-ascii?Q?hQsr9WfQI6ihdbQfqL573gZfRDQaBVRk6kNCktW9lLM8EKneiydjxe+pWS5x?= =?us-ascii?Q?M62XKHHpx/xDMMN4It031UsR6cMJigXG9p5AzsI+lNTg9uUKUC+gshXF2gXW?= =?us-ascii?Q?bdsePu+ReC111jZyFGoOb3vugg8RJ8XXoIFsHHOKqySQ28U4BshNfnkchM2g?= =?us-ascii?Q?2uFnTxOn+XbKSdByWQH4W01NPpujlEWRHHbrUqOP7silxSY0OYX/XnmUKVta?= =?us-ascii?Q?vSzJcZ6ud6fzvyiE9UuMmqsI2mhkIPUU/3g2IMOjy1SpqmBXGLxeVZfAiW9I?= =?us-ascii?Q?ZMtC1G2lXpCzKUMKQ8BU0pix7XkSntdFLf5bce+NnVXKq79+nRH5OcwRwAAC?= =?us-ascii?Q?mmztfm9Gtb/k5UEReXJg8NTCt5q9J1EOAeLxMblPdS88UKmBtQPV4Cef2AV2?= =?us-ascii?Q?wmJJV/63SOsrPIhiaiIsoSptmrZcpNncWpA85KUJF0m668hqmsyRARMVe1+i?= =?us-ascii?Q?SM2Veir5V5ve8ThJ3s4YfoKIGBXmC4C29JKc9Yc7E8shKgJ6LO0KlK2F0rGZ?= =?us-ascii?Q?JGVIQfzO3qXe8/W9kdZpnSmHX89UPzjqmLCz4vmgFxEhi2D+IHdlOO9IQw0a?= =?us-ascii?Q?v15rvLU2JeCxbQKgVhgjXuKIsVb6JfpWenzAV19qMI2lK5iAuhUtX4YFURkd?= =?us-ascii?Q?EJ4FUKmQyOzVY9bJKFu+2j43zbnEWcD6X4MIi3NZUcnfGEUQI2SLFHhuNEc4?= =?us-ascii?Q?6g=3D=3D?= X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: Pa+vvP9LRliD2zGI1VRvNwTYnnVfsWWA1Zu8pBZhPvF1m4NFwqjM9+/61n6aC/7D9/NoU+MUxWjgnH6k7F8EGf1IFd+Xgvez7UUCyugXS+u7KbB6NUxe+QmH0r/U/TBnRRV7gTBinsdtsXttOK/szsS9hEv3O6TEBoYT+PQ9SSDpUCByhT87dXnkIbQNLLKFuVWB55qXzQgCCamfi5CVnUUuXYNtVGBRPGbOjhpEsZaJOdg2lu86E0UmolSMquTIWxCR9iEDjDcazPeHL61gltJUOnJMgL2ygOaUBHRjlVSz47qt/BnAZ8RrLAPXnpDwEReb9ho0Yg1KyJtnPZXUFpe2PHmb0IgM1Ew0pUmhqNYqcOO241GSyh7jaCdk68Gj52qu2eoJ7hLJIeixdTpBXgFF0bZ5RFboGVaSSh3deuGGg5CfMhHOqhNYyh1fZv7XqqPxvyP/UHDJjxRTIJmeN1Ar4JXlaWMO/cCsrMvZ0Xeo7Z60ATHURZKIbVPZ36B6UQM1ZKxx33plr1q37bc8oY0mQuiutM/WbXtbrdJp4ImwzUNii19Xuu974uq9OwWRsVZKeoq0qmT04D/wuSqF+y0zCFqzKZINBA+ptvEy/WQ= X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: 48973cbb-7c70-4736-cfce-08de1ee97d3c X-MS-Exchange-CrossTenant-AuthSource: DM4PR10MB8218.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 Nov 2025 17:08:51.1352 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: mKzPMqksAhrYBcO5LTPcbnhnwATJJa8ncPGNAEcMorvx/Qlks3X+ZKouQ0wvgUF6RBSuEtaW8nnbbXW5s6o7UBnZGeKftSbRtwoRBs0ah1U= X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ5PPF04D2D7FA7 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1121,Hydra:6.1.9,FMLib:17.12.100.49 definitions=2025-11-08_04,2025-11-06_01,2025-10-01_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 adultscore=0 mlxscore=0 phishscore=0 mlxlogscore=999 spamscore=0 suspectscore=0 bulkscore=0 malwarescore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2510240000 definitions=main-2511080138 X-Proofpoint-GUID: 0hRw9a0T6CAuI5o18NW9Yz8cy-_Z-3l5 X-Proofpoint-Spam-Details-Enc: AW1haW4tMjUxMTA4MDEyNiBTYWx0ZWRfX0BXb6ryUjDDu Qx1E6tkd0oLHf8buBY2SKlsjRBN6eYfKgv2ihGlMkyw9qYcQP++Pnk7iibX6giNkuSoM6ht7/2d tnJ0vjeKKebFeDPaDERucl9byBW2u28lCNoIjflKuigLPWJpH2vPin/5R8meXHwr8HJtJxyup/N 4/WFj/TdqIZzYQ6q7FDHtPpwk8KSZazI0/R1Ty3Gyk74++9kMOx1WxO4kkVSXDp8dK0MYE/fDOv vf9z5/PIqQlDRehGqtG2g6CBGBXMbU6xjijaapvOCmJ1A63rE3DgOJ1wSv/FazsUP7iWvoJTNWP w//62uv69MMl3pG5A8iYg7JODOySrXasEsvNNnPCTi+peF3DoArkrIojxkoLdxtBhyMXAg939p1 FjKeC5jRrYcDXUlzk84afQN96YIfTg== X-Authority-Analysis: v=2.4 cv=XuL3+FF9 c=1 sm=1 tr=0 ts=690f7929 b=1 cx=c_pps a=WeWmnZmh0fydH62SvGsd2A==:117 a=WeWmnZmh0fydH62SvGsd2A==:17 a=6eWqkTHjU83fiwn7nKZWdM+Sl24=:19 a=z/mQ4Ysz8XfWz/Q5cLBRGdckG28=:19 a=lCpzRmAYbLLaTzLvsPZ7Mbvzbb8=:19 a=xqWC_Br6kY4A:10 a=6UeiqGixMTsA:10 a=GoEa3M9JfhUA:10 a=VkNPw1HP01LnGYTKEx00:22 a=yPCof4ZbAAAA:8 a=K7pD0KWnulOJboH6qJQA:9 X-Proofpoint-ORIG-GUID: 0hRw9a0T6CAuI5o18NW9Yz8cy-_Z-3l5 Content-Type: text/plain; charset="utf-8" PTE markers were previously only concerned with UFFD-specific logic - that is, PTE entries with the UFFD WP marker set or those marked via UFFDIO_POISON. However since the introduction of guard markers in commit 7c53dfbdb024 ("mm: add PTE_MARKER_GUARD PTE marker"), this has no longer been the case. Issues have been avoided as guard regions are not permitted in conjunction with UFFD, but it still leaves very confusing logic in place, most notably the misleading and poorly named pte_none_mostly() and huge_pte_none_mostly(). This predicate returns true for PTE entries that ought to be treated as none, but only in certain circumstances, and on the assumption we are dealing with H/W poison markers or UFFD WP markers. This patch removes these functions and makes each invocation of these functions instead explicitly check what it needs to check. As part of this effort it introduces is_uffd_pte_marker() to explicitly determine if a marker in fact is used as part of UFFD or not. In the HMM logic we note that the only time we would need to check for a fault is in the case of a UFFD WP marker, otherwise we simply encounter a fault error (VM_FAULT_HWPOISON for H/W poisoned marker, VM_FAULT_SIGSEGV for a guard marker), so only check for the UFFD WP case. While we're here we also refactor code to make it easier to understand. Reviewed-by: Vlastimil Babka Signed-off-by: Lorenzo Stoakes --- fs/userfaultfd.c | 83 +++++++++++++++++++---------------- include/asm-generic/hugetlb.h | 8 ---- include/linux/swapops.h | 18 -------- include/linux/userfaultfd_k.h | 21 +++++++++ mm/hmm.c | 2 +- mm/hugetlb.c | 47 ++++++++++---------- mm/mincore.c | 17 +++++-- mm/userfaultfd.c | 27 +++++++----- 8 files changed, 123 insertions(+), 100 deletions(-) diff --git a/fs/userfaultfd.c b/fs/userfaultfd.c index 54c6cc7fe9c6..04c66b5001d5 100644 --- a/fs/userfaultfd.c +++ b/fs/userfaultfd.c @@ -233,40 +233,46 @@ static inline bool userfaultfd_huge_must_wait(struct = userfaultfd_ctx *ctx, { struct vm_area_struct *vma =3D vmf->vma; pte_t *ptep, pte; - bool ret =3D true; =20 assert_fault_locked(vmf); =20 ptep =3D hugetlb_walk(vma, vmf->address, vma_mmu_pagesize(vma)); if (!ptep) - goto out; + return true; =20 - ret =3D false; pte =3D huge_ptep_get(vma->vm_mm, vmf->address, ptep); =20 /* * Lockless access: we're in a wait_event so it's ok if it - * changes under us. PTE markers should be handled the same as none - * ptes here. + * changes under us. */ - if (huge_pte_none_mostly(pte)) - ret =3D true; + + /* If missing entry, wait for handler. */ + if (huge_pte_none(pte)) + return true; + /* UFFD PTE markers require handling. */ + if (is_uffd_pte_marker(pte)) + return true; + /* If VMA has UFFD WP faults enabled and WP fault, wait for handler. */ if (!huge_pte_write(pte) && (reason & VM_UFFD_WP)) - ret =3D true; -out: - return ret; + return true; + + /* Otherwise, if entry isn't present, let fault handler deal with it. */ + return false; } #else static inline bool userfaultfd_huge_must_wait(struct userfaultfd_ctx *ctx, struct vm_fault *vmf, unsigned long reason) { - return false; /* should never get here */ + /* Should never get here. */ + VM_WARN_ON_ONCE(1); + return false; } #endif /* CONFIG_HUGETLB_PAGE */ =20 /* - * Verify the pagetables are still not ok after having reigstered into + * Verify the pagetables are still not ok after having registered into * the fault_pending_wqh to avoid userland having to UFFDIO_WAKE any * userfault that has already been resolved, if userfaultfd_read_iter and * UFFDIO_COPY|ZEROPAGE are being run simultaneously on two different @@ -284,53 +290,55 @@ static inline bool userfaultfd_must_wait(struct userf= aultfd_ctx *ctx, pmd_t *pmd, _pmd; pte_t *pte; pte_t ptent; - bool ret =3D true; + bool ret; =20 assert_fault_locked(vmf); =20 pgd =3D pgd_offset(mm, address); if (!pgd_present(*pgd)) - goto out; + return true; p4d =3D p4d_offset(pgd, address); if (!p4d_present(*p4d)) - goto out; + return true; pud =3D pud_offset(p4d, address); if (!pud_present(*pud)) - goto out; + return true; pmd =3D pmd_offset(pud, address); again: _pmd =3D pmdp_get_lockless(pmd); if (pmd_none(_pmd)) - goto out; + return true; =20 - ret =3D false; if (!pmd_present(_pmd)) - goto out; + return false; =20 - if (pmd_trans_huge(_pmd)) { - if (!pmd_write(_pmd) && (reason & VM_UFFD_WP)) - ret =3D true; - goto out; - } + if (pmd_trans_huge(_pmd)) + return !pmd_write(_pmd) && (reason & VM_UFFD_WP); =20 pte =3D pte_offset_map(pmd, address); - if (!pte) { - ret =3D true; + if (!pte) goto again; - } + /* * Lockless access: we're in a wait_event so it's ok if it - * changes under us. PTE markers should be handled the same as none - * ptes here. + * changes under us. */ ptent =3D ptep_get(pte); - if (pte_none_mostly(ptent)) - ret =3D true; + + ret =3D true; + /* If missing entry, wait for handler. */ + if (pte_none(ptent)) + goto out; + /* UFFD PTE markers require handling. */ + if (is_uffd_pte_marker(ptent)) + goto out; + /* If VMA has UFFD WP faults enabled and WP fault, wait for handler. */ if (!pte_write(ptent) && (reason & VM_UFFD_WP)) - ret =3D true; - pte_unmap(pte); + goto out; =20 + ret =3D false; out: + pte_unmap(pte); return ret; } =20 @@ -490,12 +498,13 @@ vm_fault_t handle_userfault(struct vm_fault *vmf, uns= igned long reason) set_current_state(blocking_state); spin_unlock_irq(&ctx->fault_pending_wqh.lock); =20 - if (!is_vm_hugetlb_page(vma)) - must_wait =3D userfaultfd_must_wait(ctx, vmf, reason); - else + if (is_vm_hugetlb_page(vma)) { must_wait =3D userfaultfd_huge_must_wait(ctx, vmf, reason); - if (is_vm_hugetlb_page(vma)) hugetlb_vma_unlock_read(vma); + } else { + must_wait =3D userfaultfd_must_wait(ctx, vmf, reason); + } + release_fault_lock(vmf); =20 if (likely(must_wait && !READ_ONCE(ctx->released))) { diff --git a/include/asm-generic/hugetlb.h b/include/asm-generic/hugetlb.h index dcb8727f2b82..e1a2e1b7c8e7 100644 --- a/include/asm-generic/hugetlb.h +++ b/include/asm-generic/hugetlb.h @@ -97,14 +97,6 @@ static inline int huge_pte_none(pte_t pte) } #endif =20 -/* Please refer to comments above pte_none_mostly() for the usage */ -#ifndef __HAVE_ARCH_HUGE_PTE_NONE_MOSTLY -static inline int huge_pte_none_mostly(pte_t pte) -{ - return huge_pte_none(pte) || is_pte_marker(pte); -} -#endif - #ifndef __HAVE_ARCH_HUGE_PTEP_SET_WRPROTECT static inline void huge_ptep_set_wrprotect(struct mm_struct *mm, unsigned long addr, pte_t *ptep) diff --git a/include/linux/swapops.h b/include/linux/swapops.h index 2687928a8146..d1f665935cfc 100644 --- a/include/linux/swapops.h +++ b/include/linux/swapops.h @@ -469,24 +469,6 @@ static inline int is_guard_swp_entry(swp_entry_t entry) (pte_marker_get(entry) & PTE_MARKER_GUARD); } =20 -/* - * This is a special version to check pte_none() just to cover the case wh= en - * the pte is a pte marker. It existed because in many cases the pte mark= er - * should be seen as a none pte; it's just that we have stored some inform= ation - * onto the none pte so it becomes not-none any more. - * - * It should be used when the pte is file-backed, ram-based and backing - * userspace pages, like shmem. It is not needed upon pgtables that do not - * support pte markers at all. For example, it's not needed on anonymous - * memory, kernel-only memory (including when the system is during-boot), - * non-ram based generic file-system. It's fine to be used even there, bu= t the - * extra pte marker check will be pure overhead. - */ -static inline int pte_none_mostly(pte_t pte) -{ - return pte_none(pte) || is_pte_marker(pte); -} - static inline struct page *pfn_swap_entry_to_page(swp_entry_t entry) { struct page *p =3D pfn_to_page(swp_offset_pfn(entry)); diff --git a/include/linux/userfaultfd_k.h b/include/linux/userfaultfd_k.h index c0e716aec26a..da0b4fcc566f 100644 --- a/include/linux/userfaultfd_k.h +++ b/include/linux/userfaultfd_k.h @@ -479,4 +479,25 @@ static inline bool pte_swp_uffd_wp_any(pte_t pte) return false; } =20 + +static inline bool is_uffd_pte_marker(pte_t pte) +{ + swp_entry_t entry; + + if (pte_present(pte)) + return false; + + entry =3D pte_to_swp_entry(pte); + if (!is_pte_marker_entry(entry)) + return false; + + /* UFFD WP, poisoned swap entries are UFFD handled. */ + if (pte_marker_entry_uffd_wp(entry)) + return true; + if (is_poisoned_swp_entry(entry)) + return true; + + return false; +} + #endif /* _LINUX_USERFAULTFD_K_H */ diff --git a/mm/hmm.c b/mm/hmm.c index a56081d67ad6..43d4a91035ff 100644 --- a/mm/hmm.c +++ b/mm/hmm.c @@ -244,7 +244,7 @@ static int hmm_vma_handle_pte(struct mm_walk *walk, uns= igned long addr, uint64_t pfn_req_flags =3D *hmm_pfn; uint64_t new_pfn_flags =3D 0; =20 - if (pte_none_mostly(pte)) { + if (pte_none(pte) || pte_marker_uffd_wp(pte)) { required_fault =3D hmm_pte_need_fault(hmm_vma_walk, pfn_req_flags, 0); if (required_fault) diff --git a/mm/hugetlb.c b/mm/hugetlb.c index 1ea459723cce..01c784547d1e 100644 --- a/mm/hugetlb.c +++ b/mm/hugetlb.c @@ -6743,29 +6743,28 @@ vm_fault_t hugetlb_fault(struct mm_struct *mm, stru= ct vm_area_struct *vma, } =20 vmf.orig_pte =3D huge_ptep_get(mm, vmf.address, vmf.pte); - if (huge_pte_none_mostly(vmf.orig_pte)) { - if (is_pte_marker(vmf.orig_pte)) { - pte_marker marker =3D - pte_marker_get(pte_to_swp_entry(vmf.orig_pte)); - - if (marker & PTE_MARKER_POISONED) { - ret =3D VM_FAULT_HWPOISON_LARGE | - VM_FAULT_SET_HINDEX(hstate_index(h)); - goto out_mutex; - } else if (WARN_ON_ONCE(marker & PTE_MARKER_GUARD)) { - /* This isn't supported in hugetlb. */ - ret =3D VM_FAULT_SIGSEGV; - goto out_mutex; - } - } - + if (huge_pte_none(vmf.orig_pte)) /* - * Other PTE markers should be handled the same way as none PTE. - * * hugetlb_no_page will drop vma lock and hugetlb fault * mutex internally, which make us return immediately. */ return hugetlb_no_page(mapping, &vmf); + + if (is_pte_marker(vmf.orig_pte)) { + const pte_marker marker =3D + pte_marker_get(pte_to_swp_entry(vmf.orig_pte)); + + if (marker & PTE_MARKER_POISONED) { + ret =3D VM_FAULT_HWPOISON_LARGE | + VM_FAULT_SET_HINDEX(hstate_index(h)); + goto out_mutex; + } else if (WARN_ON_ONCE(marker & PTE_MARKER_GUARD)) { + /* This isn't supported in hugetlb. */ + ret =3D VM_FAULT_SIGSEGV; + goto out_mutex; + } + + return hugetlb_no_page(mapping, &vmf); } =20 ret =3D 0; @@ -6934,6 +6933,7 @@ int hugetlb_mfill_atomic_pte(pte_t *dst_pte, int ret =3D -ENOMEM; struct folio *folio; bool folio_in_pagecache =3D false; + pte_t dst_ptep; =20 if (uffd_flags_mode_is(flags, MFILL_ATOMIC_POISON)) { ptl =3D huge_pte_lock(h, dst_mm, dst_pte); @@ -7073,13 +7073,14 @@ int hugetlb_mfill_atomic_pte(pte_t *dst_pte, if (folio_test_hwpoison(folio)) goto out_release_unlock; =20 + ret =3D -EEXIST; + + dst_ptep =3D huge_ptep_get(dst_mm, dst_addr, dst_pte); /* - * We allow to overwrite a pte marker: consider when both MISSING|WP - * registered, we firstly wr-protect a none pte which has no page cache - * page backing it, then access the page. + * See comment about UFFD marker overwriting in + * mfill_atomic_install_pte(). */ - ret =3D -EEXIST; - if (!huge_pte_none_mostly(huge_ptep_get(dst_mm, dst_addr, dst_pte))) + if (!huge_pte_none(dst_ptep) && !is_uffd_pte_marker(dst_ptep)) goto out_release_unlock; =20 if (folio_in_pagecache) diff --git a/mm/mincore.c b/mm/mincore.c index 8ec4719370e1..151b2dbb783b 100644 --- a/mm/mincore.c +++ b/mm/mincore.c @@ -32,11 +32,22 @@ static int mincore_hugetlb(pte_t *pte, unsigned long hm= ask, unsigned long addr, spinlock_t *ptl; =20 ptl =3D huge_pte_lock(hstate_vma(walk->vma), walk->mm, pte); + /* * Hugepages under user process are always in RAM and never * swapped out, but theoretically it needs to be checked. */ - present =3D pte && !huge_pte_none_mostly(huge_ptep_get(walk->mm, addr, pt= e)); + if (!pte) { + present =3D 0; + } else { + const pte_t ptep =3D huge_ptep_get(walk->mm, addr, pte); + + if (huge_pte_none(ptep) || is_pte_marker(ptep)) + present =3D 0; + else + present =3D 1; + } + for (; addr !=3D end; vec++, addr +=3D PAGE_SIZE) *vec =3D present; walk->private =3D vec; @@ -175,8 +186,8 @@ static int mincore_pte_range(pmd_t *pmd, unsigned long = addr, unsigned long end, pte_t pte =3D ptep_get(ptep); =20 step =3D 1; - /* We need to do cache lookup too for pte markers */ - if (pte_none_mostly(pte)) + /* We need to do cache lookup too for UFFD pte markers */ + if (pte_none(pte) || is_uffd_pte_marker(pte)) __mincore_unmapped_range(addr, addr + PAGE_SIZE, vma, vec); else if (pte_present(pte)) { diff --git a/mm/userfaultfd.c b/mm/userfaultfd.c index 00122f42718c..cc4ce205bbec 100644 --- a/mm/userfaultfd.c +++ b/mm/userfaultfd.c @@ -178,6 +178,7 @@ int mfill_atomic_install_pte(pmd_t *dst_pmd, spinlock_t *ptl; struct folio *folio =3D page_folio(page); bool page_in_cache =3D folio_mapping(folio); + pte_t dst_ptep; =20 _dst_pte =3D mk_pte(page, dst_vma->vm_page_prot); _dst_pte =3D pte_mkdirty(_dst_pte); @@ -199,12 +200,15 @@ int mfill_atomic_install_pte(pmd_t *dst_pmd, } =20 ret =3D -EEXIST; + + dst_ptep =3D ptep_get(dst_pte); + /* - * We allow to overwrite a pte marker: consider when both MISSING|WP - * registered, we firstly wr-protect a none pte which has no page cache - * page backing it, then access the page. + * We are allowed to overwrite a UFFD pte marker: consider when both + * MISSING|WP registered, we firstly wr-protect a none pte which has no + * page cache page backing it, then access the page. */ - if (!pte_none_mostly(ptep_get(dst_pte))) + if (!pte_none(dst_ptep) && !is_uffd_pte_marker(dst_ptep)) goto out_unlock; =20 if (page_in_cache) { @@ -583,12 +587,15 @@ static __always_inline ssize_t mfill_atomic_hugetlb( goto out_unlock; } =20 - if (!uffd_flags_mode_is(flags, MFILL_ATOMIC_CONTINUE) && - !huge_pte_none_mostly(huge_ptep_get(dst_mm, dst_addr, dst_pte))) { - err =3D -EEXIST; - hugetlb_vma_unlock_read(dst_vma); - mutex_unlock(&hugetlb_fault_mutex_table[hash]); - goto out_unlock; + if (!uffd_flags_mode_is(flags, MFILL_ATOMIC_CONTINUE)) { + const pte_t ptep =3D huge_ptep_get(dst_mm, dst_addr, dst_pte); + + if (!huge_pte_none(ptep) && !is_uffd_pte_marker(ptep)) { + err =3D -EEXIST; + hugetlb_vma_unlock_read(dst_vma); + mutex_unlock(&hugetlb_fault_mutex_table[hash]); + goto out_unlock; + } } =20 err =3D hugetlb_mfill_atomic_pte(dst_pte, dst_vma, dst_addr, --=20 2.51.0 From nobody Fri Dec 19 21:47:07 2025 Received: from mx0b-00069f02.pphosted.com (mx0b-00069f02.pphosted.com [205.220.177.32]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id AF85A2512FC; Sat, 8 Nov 2025 17:11:09 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=205.220.177.32 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1762621872; cv=fail; b=Fs4dbf1NiVYyEGY1VJsukI+2aOzeHSfp2aBDUFDmGUbok8OBPtt+81SnedvPPmaC31+sBvc5Q6gjuOVrcbD4Wj2fhLUS1NEUfVpgy1KyNlgTVSOv/E9UMorgqK/TZGLYOONlj8hE8AiLSMGPwkaCvO3kPfdwY38vP/W1d7lZQaA= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1762621872; c=relaxed/simple; bh=k0ElxpmS9HwZwo7xmLghbhMxnGpvnsPjGlxz7Jr4dHQ=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: Content-Type:MIME-Version; b=N48QhouHvXTCX6umi6tDsliG4r8oN0x3Bx78O+72u/wPmW9Pcssd6WS90dYihfrfOP3jA5SuWyYMB0IqjtURy74YV2VjFnRvRcwLqnUzYedDOPm5NtiO2DmrxoRmlrwFxvhqpIoJdvIGyVfkaDZ1ygahx5YtLJRw9l91hhlBGKw= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=oracle.com; spf=pass smtp.mailfrom=oracle.com; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b=PnNJohfD; dkim=pass (1024-bit key) header.d=oracle.onmicrosoft.com header.i=@oracle.onmicrosoft.com header.b=mt3BOkP9; arc=fail smtp.client-ip=205.220.177.32 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=oracle.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=oracle.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b="PnNJohfD"; dkim=pass (1024-bit key) header.d=oracle.onmicrosoft.com header.i=@oracle.onmicrosoft.com header.b="mt3BOkP9" Received: from pps.filterd (m0246630.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 5A8GfxDM009610; Sat, 8 Nov 2025 17:09:03 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=cc :content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s= corp-2025-04-25; bh=Au3FCKtwkRYNsFHZTSml++AMw+FgHqiZR3qsM68TKXE=; b= PnNJohfDQ+ScUvmLRIbxVDfZPXgZzq7u0n8AOS5ELn0wlXd1Cj/LbpqFQ3Xaz33o 2o6+szqtiIuSp+OCROx67QRzIBt2Ba8uOotSRzGdkzS0rAwiAAJY1XBYdCgUnRmI FWkuN4N6m1l7OjtSK6MeF5EISqKSGL1qTWWdny+qL2YWhG+fVGUCdnZUCzpN7wTy f+DBPLAFuT9dZqXoL0xGpkbeX7sDnaWukv1NOygZfZ+AUEj7hxAwS/Pc1rc/9icX OrXs3G3GnT3yIneYzskU8vadzcRTQoGO/GCLPa+NncF4AsvPiA/sZa83f8I4FlAV j4UeGgl9Kqxzwb9WKKfDOA== Received: from phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta01.appoci.oracle.com [138.1.114.2]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 4aa9k8g0hx-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Sat, 08 Nov 2025 17:09:02 +0000 (GMT) Received: from pps.filterd (phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (8.18.1.2/8.18.1.2) with ESMTP id 5A8GoW0U007552; Sat, 8 Nov 2025 17:09:01 GMT Received: from mw6pr02cu001.outbound.protection.outlook.com (mail-westus2azon11012068.outbound.protection.outlook.com [52.101.48.68]) by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTPS id 4a9va76mus-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Sat, 08 Nov 2025 17:09:01 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=XPJ1lHALHLSHWdHlIRjElVcl+7bXb38lV1DKzBg8DGOU69cpakSgNic/mjdhxPS09/PCPGm8l44MnU8wGdc918gRog2mrpxDH7wdrONwHEMPFL+IK/iG0YGFe6aaBXEofArTBwip3ZW0TVQGAp+YtKcJmKk76HkATrx9Z88oMJ2F9raYwcoDmciTiOrDv4De2R2ReUmLAKGOSba+C3vIkLhPwXC0H3m/E/jaW1OjrIM/sIuEOVtSMTlFKvuTlZjFLRFufIayeHv5Tt2adTWKd9RqbvpN99AWUH6Et/V4ZciwBNNyXIDTXHCY+m1tFQDODvc4bxGBk8X0/NUmsglNZA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=Au3FCKtwkRYNsFHZTSml++AMw+FgHqiZR3qsM68TKXE=; b=NnzMYe49Pd4Qf3S/aw/T4Nay4AjdK5qdhCXGoaq3ydjMhvAKmLcc1Kr+fWpCf1+g4a+4GESp5FwYBOLKGt8jIbS5DmWeryNg2sXYDxKUwlGR1AoM4XP/LJg35SdYeZIfQtiQbk93Yay/lD1awHuV1dyq5HcrofYOlTuL4JWG2fPf36h71DbY/h0hCZyN/jN3llKrtHSh0TjI9Yetb4pYiObS3yxzhynDEtzfQWyZy1xu8+R1GawCWs+mYtVUf9y6SO3fY5j3ZZ6Rg3VfIjB+7xAFla/ka+A2vhQNkwWyUCZ70kkR3z+eBDuQRRFXzzy68LUj++juN1eWqLTStWuJUQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Au3FCKtwkRYNsFHZTSml++AMw+FgHqiZR3qsM68TKXE=; b=mt3BOkP9Jmv/LJaGn6cqWXIIx69fKGTteQJpzodgJNynbViLkMq04+vaLxc9c6BpCEpH4NDnwboRDqqPrkPKq4xmwfcmJ3PjDHse5FXMKqO8PlMb68xFgYmgU9QUtTgMO2VTw+dz09PvUAcSWB9HD80A3o/D1h0UU7rGV223SNs= Received: from DM4PR10MB8218.namprd10.prod.outlook.com (2603:10b6:8:1cc::16) by SJ5PPF04D2D7FA7.namprd10.prod.outlook.com (2603:10b6:a0f:fc02::785) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9298.15; Sat, 8 Nov 2025 17:08:54 +0000 Received: from DM4PR10MB8218.namprd10.prod.outlook.com ([fe80::2650:55cf:2816:5f2]) by DM4PR10MB8218.namprd10.prod.outlook.com ([fe80::2650:55cf:2816:5f2%7]) with mapi id 15.20.9298.010; Sat, 8 Nov 2025 17:08:54 +0000 From: Lorenzo Stoakes To: Andrew Morton Cc: Christian Borntraeger , Janosch Frank , Claudio Imbrenda , David Hildenbrand , Alexander Gordeev , Gerald Schaefer , Heiko Carstens , Vasily Gorbik , Sven Schnelle , Peter Xu , Alexander Viro , Christian Brauner , Jan Kara , Arnd Bergmann , Zi Yan , Baolin Wang , "Liam R . Howlett" , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Lance Yang , Muchun Song , Oscar Salvador , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Matthew Brost , Joshua Hahn , Rakie Kim , Byungchul Park , Gregory Price , Ying Huang , Alistair Popple , Axel Rasmussen , Yuanchu Xie , Wei Xu , Kemeng Shi , Kairui Song , Nhat Pham , Baoquan He , Chris Li , SeongJae Park , Matthew Wilcox , Jason Gunthorpe , Leon Romanovsky , Xu Xin , Chengming Zhou , Jann Horn , Miaohe Lin , Naoya Horiguchi , Pedro Falcato , Pasha Tatashin , Rik van Riel , Harry Yoo , Hugh Dickins , linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-s390@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, linux-arch@vger.kernel.org, damon@lists.linux.dev Subject: [PATCH v2 02/16] mm: introduce leaf entry type and use to simplify leaf entry logic Date: Sat, 8 Nov 2025 17:08:16 +0000 Message-ID: X-Mailer: git-send-email 2.51.0 In-Reply-To: References: Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: LO2P123CA0097.GBRP123.PROD.OUTLOOK.COM (2603:10a6:600:139::12) To DM4PR10MB8218.namprd10.prod.outlook.com (2603:10b6:8:1cc::16) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DM4PR10MB8218:EE_|SJ5PPF04D2D7FA7:EE_ X-MS-Office365-Filtering-Correlation-Id: 607fb913-4c0e-4082-524b-08de1ee97f30 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|1800799024|7416014|376014|366016; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?qzxHBqiYkWUIN5l1nSIEUHJ6ZOvuE0+y8SuaBOsWpS+mpNwroCAQkjQ6aPGe?= =?us-ascii?Q?lAIFfUe938Quui0YlckaKcYY6+AKptLy4ec94wKszZYC14I+4Bt0FGXuqE6U?= =?us-ascii?Q?FzupAsa52ygVOuCQy6J71YF8CUs8PGjRz5ugTLup72yqT7jQJ47+g7jYdjIl?= =?us-ascii?Q?uNBkpLSqTyAIl2eYfEAYfsoqXcKY4utT5kgh11incJarBfqrEbk1HXGLZsFO?= =?us-ascii?Q?24AJi3hl8Hum8eVK0MurKmnG9x9Azp26MbCrPqliBdrt/DCdUVz1OCCdyfuK?= =?us-ascii?Q?XFG4p6mDuR1juhJ0P4PqrjhWpMTagvWSaB5yegtgp06602afczWx87XkiKQH?= =?us-ascii?Q?nq/fy7brNeUlEwmlR6iWxb/8apakqfBit/xnSbaRaskUlnfHVT3bytUPQeQ2?= =?us-ascii?Q?vHHiLq06fOoErvnyw4OYIA5FvUJ/JRS83xx6eL+b1bP0LfEDdil2rp2weAOa?= =?us-ascii?Q?As6Sk1OM6GCaBo0sG0kKUAnG9iwW0Qv1INJnnytDWdi2Vcv8G4VJME7T6hE4?= =?us-ascii?Q?Dg343m4ZXDevUZzkKHxTlv7BHu2lLXtPmwiPO1Fk1/eJabqed9exiTgFQpWq?= =?us-ascii?Q?cEHNUssoY6WYmsL8qxtgD70aXuk4QOb67pzVJZaLjLJJ/5DcGDqbfDVooD4e?= =?us-ascii?Q?s7e8lp8QKYlEu+JcemrdO46HDPYJTa/5e9CVQvuf610DzejdP3H+1JsRqslC?= =?us-ascii?Q?hAoow76epzpiuWJhTnPuZbOQl5uams9fqe78/2OzvkR1v8BUvIQ4NkWwxAVh?= =?us-ascii?Q?AphmYGhlZfEuB0HHAYJx9Iu9897aR2oXYEQPTzHBcShjB4K5pvDREPuxbT9H?= =?us-ascii?Q?bkYsQlvH+HX9flr0bhI417Og9B9nMhTASbn/Pa1wybjzqseUFy50qrfOx9nV?= =?us-ascii?Q?sAbnvjZAAKUP+tfqIbFz4EYNv/01RdtgTKq1AlSLRAQGv6K4reH2dvzaNKO7?= =?us-ascii?Q?WvtMfmhGwCqKIv2R+GBAdC1bqm01Lgap6otDg64GdCNiTAdAdV7wrm13xJUf?= =?us-ascii?Q?ydRETAA3FBAKTGXwKWn9Jzh+4TMgwD1dFELxZxQZMkUTNILSQaj0hH7rXMbW?= =?us-ascii?Q?Q2IYvtQ8Sn/pdiqcCEv60YBaSzNXGVO2jOb8LnUB8mgbN5wEV2EPLVEhDrp+?= =?us-ascii?Q?ZEZQDXMpvUZPcXs0Ckn4kCDcOe+nYUkeGP/TcKrmCndVy+fvDx7wveAeTqX3?= =?us-ascii?Q?pOJFSKldVUwRol0wubkNs6X8SF9gckb6d6LN314bSSn69EzMUoGabD499gi/?= =?us-ascii?Q?UOh42DTuC1e6KiOlELQkh89wjxv49iSx+wbw1FqUgKXXuNF0IgkQzZdMRL2F?= =?us-ascii?Q?dZtGIxOENPcs3EaLPUTz4fdBlwEkHFrc/LY/F8/VcRfi+lM+9sWTF3DLeS88?= =?us-ascii?Q?P0L+z9ggQXx76Iphrbuu6EUmfGEn27kHAxpAgaqVf07rieNNGw4GZDmyPIJ7?= =?us-ascii?Q?sIhYqU3UVEwlmLGzdyoE92H2DRAqUAt+?= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DM4PR10MB8218.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(1800799024)(7416014)(376014)(366016);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?0YjQ2WYApwsA4BSMnw/YveQfgksVI/78ExsknaalbXGY3UzSIDqRMjNhYcfr?= =?us-ascii?Q?cd2BPTvtsqwnwIsbMyjlRtqH9I2bvQ7n7njtP0CSsE4t38HdigS+fxMjzp/i?= =?us-ascii?Q?zWguv/S0EaG0YAwYmlX9BjexqmzF85gXukIccQKG1Bv4HpUlUBbJZkxhmfcy?= =?us-ascii?Q?g5x0IjROP7wPVAk1v4vOa+jSklJ0sl5U5CP4pYCzaDEw8azZcSMCA5rmbqCg?= =?us-ascii?Q?qFSP4fMz+Wb3aG0iz6lWbgXrLjeEHUrCBFvOlQwMSaQLFePL4qyPhpEeAnHw?= =?us-ascii?Q?FvXlWRXGeh/zFCN5UNHL0ExaqUu5vHmPOw9YSu0PK9Xvi7o8FqB1bdC+Jvwx?= =?us-ascii?Q?K5SbfEKxAAQtOrD35qrZ9c2yeeu3rv7Q02xN/MpfOUK9wRL2lazKL6mbekaB?= =?us-ascii?Q?n5BFArjqMt6lrOtgI+1wpp//c70T+LHOxXwGw9gSyvi3qiYbUyrpMh9ThbJJ?= =?us-ascii?Q?ejgFTzjaZ0PqXpjeIHaxiM7FgelIlGn8kz1GE2HVVLqaQ6CFaqljEXlaNs39?= =?us-ascii?Q?AGuQFHPhER7MDPhE4298rrKykoXBqxxIBx3dYRFHqelgE7MY700LxUjR5yaS?= =?us-ascii?Q?fyL0m50wkY/8yzQlwx94mEfs/QTZVflhK+GbypJBzAFrrsei1n86+z0wtrfv?= =?us-ascii?Q?S8F9zES75A64sDWua3eMyD7VjoBUCS86YIB6vgwR8CcTTiaQBqY2qFq8EW/g?= =?us-ascii?Q?Sa30yqedawnZ5qutWOGH5BxVn94WBkwfJmHBVf5tYVFPTpoASxZbzMHqLf+d?= =?us-ascii?Q?NjjnoWMImcmKBLpER61NJ9P59Ulxx2S+Wp/VUTjuqwNdgdo1XAc7TvQXNZ1C?= =?us-ascii?Q?OaRnHamD7o7l6PMVP6j8ZPoBVMgDcKsx0n4d8rwj4s+YW2jzfcTZvKnGPpzV?= =?us-ascii?Q?xjzKQdb9EThBi/3B2B2E1Z+O8FZgz0LFUV2PvTfpXRXxWjk+eJ6wqs35vYNz?= =?us-ascii?Q?ijTAEy2iVLppQuIeUmBbip7Jhr2CdJkzFwUZucecW+P//HsqUfKW01DSDGJg?= =?us-ascii?Q?A8ZVFg1k+TTdNC0/ozB4TA1UUZi3XDmbj8dfx8nnxG451XcoHpAFerhnabAx?= =?us-ascii?Q?8j61dzAoEyZN2TrpjN6I3M+KCLyWZU/cFeOChKOoX8oEtZ1/AgLkC3MOvxSb?= =?us-ascii?Q?lIVpbuSxo+KOLm8KXiWPEsBxW/hw3muw4H4JV8RbNEjoZllXUW3uXGyJK1U+?= =?us-ascii?Q?1Yl0Fo8YJfRTY/SWzV3xNuqIYq8VAjAql3VPN5PWbhchfzkrp1TwrwRbX0tP?= =?us-ascii?Q?lOmInExyUlG3YQrWRvZBmnRj3yrMlsr4NxeNLIGkW78bhl8EfVAjrL2KBNbY?= =?us-ascii?Q?Qmvqvkk/ChYbGqx177DiqAGkFCNiv3iUO9C3gKqcd9qjsfjc8RKR/ll3GSo7?= =?us-ascii?Q?uqPXEL2NpJ40v7A2TK8YBNtQu3css5JDqgYR9Tjtu95gWQWkOZnVAzumyI8q?= =?us-ascii?Q?c0x/P9RK8Yz4Dh1+QNYva2W2oGRZEe6tJMMSdAxnVk09uqMKn+S8voa7SUT9?= =?us-ascii?Q?dl+xYcGEuEwtDyDvdrV3SFCxz6VAqiqVFAc0W/XaM+Ylp/ovJGIfcefMo1QW?= =?us-ascii?Q?l4WAOr+Ws5DH5PO8t7XS13OmW1wrKNNRRDwkzDN6ODurqUZ1U18VJMqutdnB?= =?us-ascii?Q?Ig=3D=3D?= X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: 3zkyPO+OmlYwYErAiWSdTW+tp6irm2avXhijB/ZRSegqDqEpTC8efZp3SSVVxRg83CoyCdFLhChS87hmwxUfSczqy+6eyqi8C67DkrfDGsU6n7QgnnstmHZSNWqy/uA5srk74ArDPveyT1WVzykyD5OK5PrkoQlK6kP8+40BNKosQ34YW+VUAPlqGfsZMA8+TFKaWVOIvke/a6enKDDq+ruZw0v32slXNRSuSXWWnbvoVvoHUrMqPTezMnCBnkh9Ad69Yr9GIFAz7uhaxTS5Oy3/5a4i5KySALSfUyZfdB6cxBkUhbZTFQt+f93/hSxt1LeiLH7M6Fx6/BFOltpbX9wTdu5K8YEyv4ldZsBKVv9znfMzxJB9jQbeDr8l/BX2KPc+jmJL8gjMgRFjI0JXQ0/UcPO5Gcd4l1Hd2+V4l32T393aO2xYp1I7HdD9v3ecdu+sdIdMNGD0B8LICnIeUzKerkekHjY87PKk46htqbZrSDeMWM0CVr85enmUn7mflGYCs2UO+NPH3dPAms01bBSRlY2bITXsThiQhEymk4BG/H0R7ekf9BzLGz3PQzaYk8aCE13hYznIbc40+9rThohi4ULBw3B/HmJsiv+Ugyc= X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: 607fb913-4c0e-4082-524b-08de1ee97f30 X-MS-Exchange-CrossTenant-AuthSource: DM4PR10MB8218.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 Nov 2025 17:08:54.5055 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: jjsApbNh0KuafM8nQcJPigcVBv0j7g6TxR13GMoW+F3s/cya/X8ZFrrW4m/gkb9QPtQFybCHSezVWDfANZYd/IanUAzkKKL/9+R9EJxus9M= X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ5PPF04D2D7FA7 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1121,Hydra:6.1.9,FMLib:17.12.100.49 definitions=2025-11-08_04,2025-11-06_01,2025-10-01_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 adultscore=0 malwarescore=0 spamscore=0 mlxscore=0 bulkscore=0 suspectscore=0 mlxlogscore=999 phishscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2510240000 definitions=main-2511080138 X-Proofpoint-Spam-Details-Enc: AW1haW4tMjUxMTA4MDEzNCBTYWx0ZWRfX9YLIw8NNwIf7 wIvmGOlOg2Si5zRrLkTrNXLKerykD+1NpLj+IEEJ15QQsHlWFHRg1oFkdBRls3micGvQ4sTnrEB FR/05RKpWHC3+gz+9ncB7PxLuncxO3RJcyw9YxBh/4tYaaqoXpzF0c2rw5X1yIunTIbV7Pufbaf yFBO2Rf9LqlC0KMrBjzkUE6VDQobNuPsEhvtCA+bTe6C2CDtma5RDSYo9xdMoaHKjgLgf/17ETX c7w2GlK5+gQYsvEkSB2g3JlqOp1hyfLxiCE06cK6LseUp7pzsDC33LR2pKOuIXI11XhIIaWiPqu PkVS6D9bgUY/Z6EsiS6RopogX6AmLj8Vqv6AmsGgdRI/oIzHGYeg1yXwCQJXQuOHTIgRQxWfOWj 6H/mwYpRgiZNUEMtKcQQvW2xgaforw== X-Proofpoint-ORIG-GUID: akdu_74Cy9eicrb-Xx8wSKIsjuxTex2S X-Authority-Analysis: v=2.4 cv=U4ufzOru c=1 sm=1 tr=0 ts=690f792f cx=c_pps a=XiAAW1AwiKB2Y8Wsi+sD2Q==:117 a=XiAAW1AwiKB2Y8Wsi+sD2Q==:17 a=6eWqkTHjU83fiwn7nKZWdM+Sl24=:19 a=z/mQ4Ysz8XfWz/Q5cLBRGdckG28=:19 a=lCpzRmAYbLLaTzLvsPZ7Mbvzbb8=:19 a=xqWC_Br6kY4A:10 a=6UeiqGixMTsA:10 a=GoEa3M9JfhUA:10 a=VkNPw1HP01LnGYTKEx00:22 a=yPCof4ZbAAAA:8 a=VwQbUJbxAAAA:8 a=Xv-3kFOuIgxVCAwDjsYA:9 a=UhEZJTgQB8St2RibIkdl:22 a=Z5ABNNGmrOfJ6cZ5bIyy:22 a=QOGEsqRv6VhmHaoFNykA:22 X-Proofpoint-GUID: akdu_74Cy9eicrb-Xx8wSKIsjuxTex2S Content-Type: text/plain; charset="utf-8" The kernel maintains leaf page table entries which contain either: - Nothing ('none' entries) - Present entries (that is stuff the hardware can navigate without fault) - Everything else that will cause a fault which the kernel handles In the 'everything else' group we include swap entries, but we also include a number of other things such as migration entries, device private entries and marker entries. Unfortunately this 'everything else' group expresses everything through a swp_entry_t type, and these entries are referred to swap entries even though they may well not contain a... swap entry. This is compounded by the rather mind-boggling concept of a non-swap swap entry (checked via non_swap_entry()) and the means by which we twist and turn to satisfy this. This patch lays the foundation for reducing this confusion. We refer to 'everything else' as a 'software-define leaf entry' or 'softleaf'. for short And in fact we scoop up the 'none' entries into this concept also so we are left with: - Present entries. - Softleaf entries (which may be empty). This allows for radical simplification across the board - one can simply convert any leaf page table entry to a leaf entry via softleaf_from_pte(). If the entry is present, we return an empty leaf entry, so it is assumed the caller is aware that they must differentiate between the two categories of page table entries, checking for the former via pte_present(). As a result, we can eliminate a number of places where we would otherwise need to use predicates to see if we can proceed with leaf page table entry conversion and instead just go ahead and do it unconditionally. We do so where we can, adjusting surrounding logic as necessary to integrate the new softleaf_t logic as far as seems reasonable at this stage. We typedef swp_entry_t to softleaf_t for the time being until the conversion can be complete, meaning everything remains compatible regardless of which type is used. We will eventually remove swp_entry_t when the conversion is complete. We introduce a new header file to keep things clear - leafops.h - this imports swapops.h so can direct replace swapops imports without issue, and we do so in all the files that require it. Additionally, add new leafops.h file to core mm maintainers entry. Signed-off-by: Lorenzo Stoakes --- MAINTAINERS | 1 + fs/proc/task_mmu.c | 26 +-- fs/userfaultfd.c | 6 +- include/linux/leafops.h | 382 ++++++++++++++++++++++++++++++++++ include/linux/mm_inline.h | 6 +- include/linux/mm_types.h | 25 +++ include/linux/swapops.h | 28 --- include/linux/userfaultfd_k.h | 51 +---- mm/hmm.c | 2 +- mm/hugetlb.c | 37 ++-- mm/madvise.c | 16 +- mm/memory.c | 41 ++-- mm/mincore.c | 6 +- mm/mprotect.c | 6 +- mm/mremap.c | 4 +- mm/page_vma_mapped.c | 11 +- mm/shmem.c | 7 +- mm/userfaultfd.c | 6 +- 18 files changed, 497 insertions(+), 164 deletions(-) create mode 100644 include/linux/leafops.h diff --git a/MAINTAINERS b/MAINTAINERS index 2628431dcdfe..314910a70bbf 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -16257,6 +16257,7 @@ T: git git://git.kernel.org/pub/scm/linux/kernel/gi= t/akpm/mm F: include/linux/gfp.h F: include/linux/gfp_types.h F: include/linux/highmem.h +F: include/linux/leafops.h F: include/linux/memory.h F: include/linux/mm.h F: include/linux/mm_*.h diff --git a/fs/proc/task_mmu.c b/fs/proc/task_mmu.c index fc35a0543f01..24d26b49d870 100644 --- a/fs/proc/task_mmu.c +++ b/fs/proc/task_mmu.c @@ -14,7 +14,7 @@ #include #include #include -#include +#include #include #include #include @@ -1230,11 +1230,11 @@ static int smaps_hugetlb_range(pte_t *pte, unsigned= long hmask, if (pte_present(ptent)) { folio =3D page_folio(pte_page(ptent)); present =3D true; - } else if (is_swap_pte(ptent)) { - swp_entry_t swpent =3D pte_to_swp_entry(ptent); + } else { + const softleaf_t entry =3D softleaf_from_pte(ptent); =20 - if (is_pfn_swap_entry(swpent)) - folio =3D pfn_swap_entry_folio(swpent); + if (softleaf_has_pfn(entry)) + folio =3D softleaf_to_folio(entry); } =20 if (folio) { @@ -1955,9 +1955,9 @@ static pagemap_entry_t pte_to_pagemap_entry(struct pa= gemapread *pm, flags |=3D PM_SWAP; if (is_pfn_swap_entry(entry)) page =3D pfn_swap_entry_to_page(entry); - if (pte_marker_entry_uffd_wp(entry)) + if (softleaf_is_uffd_wp_marker(entry)) flags |=3D PM_UFFD_WP; - if (is_guard_swp_entry(entry)) + if (softleaf_is_guard_marker(entry)) flags |=3D PM_GUARD_REGION; } =20 @@ -2330,18 +2330,18 @@ static unsigned long pagemap_page_category(struct p= agemap_scan_private *p, if (pte_soft_dirty(pte)) categories |=3D PAGE_IS_SOFT_DIRTY; } else if (is_swap_pte(pte)) { - swp_entry_t swp; + softleaf_t entry; =20 categories |=3D PAGE_IS_SWAPPED; if (!pte_swp_uffd_wp_any(pte)) categories |=3D PAGE_IS_WRITTEN; =20 - swp =3D pte_to_swp_entry(pte); - if (is_guard_swp_entry(swp)) + entry =3D softleaf_from_pte(pte); + if (softleaf_is_guard_marker(entry)) categories |=3D PAGE_IS_GUARD; else if ((p->masks_of_interest & PAGE_IS_FILE) && - is_pfn_swap_entry(swp) && - !folio_test_anon(pfn_swap_entry_folio(swp))) + softleaf_has_pfn(entry) && + !folio_test_anon(softleaf_to_folio(entry))) categories |=3D PAGE_IS_FILE; =20 if (pte_swp_soft_dirty(pte)) @@ -2466,7 +2466,7 @@ static void make_uffd_wp_huge_pte(struct vm_area_stru= ct *vma, { unsigned long psize; =20 - if (is_hugetlb_entry_hwpoisoned(ptent) || is_pte_marker(ptent)) + if (is_hugetlb_entry_hwpoisoned(ptent) || pte_is_marker(ptent)) return; =20 psize =3D huge_page_size(hstate_vma(vma)); diff --git a/fs/userfaultfd.c b/fs/userfaultfd.c index 04c66b5001d5..e33e7df36927 100644 --- a/fs/userfaultfd.c +++ b/fs/userfaultfd.c @@ -29,7 +29,7 @@ #include #include #include -#include +#include #include #include =20 @@ -251,7 +251,7 @@ static inline bool userfaultfd_huge_must_wait(struct us= erfaultfd_ctx *ctx, if (huge_pte_none(pte)) return true; /* UFFD PTE markers require handling. */ - if (is_uffd_pte_marker(pte)) + if (pte_is_uffd_marker(pte)) return true; /* If VMA has UFFD WP faults enabled and WP fault, wait for handler. */ if (!huge_pte_write(pte) && (reason & VM_UFFD_WP)) @@ -330,7 +330,7 @@ static inline bool userfaultfd_must_wait(struct userfau= ltfd_ctx *ctx, if (pte_none(ptent)) goto out; /* UFFD PTE markers require handling. */ - if (is_uffd_pte_marker(ptent)) + if (pte_is_uffd_marker(ptent)) goto out; /* If VMA has UFFD WP faults enabled and WP fault, wait for handler. */ if (!pte_write(ptent) && (reason & VM_UFFD_WP)) diff --git a/include/linux/leafops.h b/include/linux/leafops.h new file mode 100644 index 000000000000..1376589d94b0 --- /dev/null +++ b/include/linux/leafops.h @@ -0,0 +1,382 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* + * Describes operations that can be performed on software-defined page tab= le + * leaf entries. These are abstracted from the hardware page table entries + * themselves by the softleaf_t type, see mm_types.h. + */ +#ifndef _LINUX_LEAFOPS_H +#define _LINUX_LEAFOPS_H + +#include +#include +#include + +#ifdef CONFIG_MMU + +/* Temporary until swp_entry_t eliminated. */ +#define LEAF_TYPE_SHIFT SWP_TYPE_SHIFT + +enum softleaf_type { + /* Fundamental types. */ + SOFTLEAF_NONE, + SOFTLEAF_SWAP, + /* Migration types. */ + SOFTLEAF_MIGRATION_READ, + SOFTLEAF_MIGRATION_READ_EXCLUSIVE, + SOFTLEAF_MIGRATION_WRITE, + /* Device types. */ + SOFTLEAF_DEVICE_PRIVATE_READ, + SOFTLEAF_DEVICE_PRIVATE_WRITE, + SOFTLEAF_DEVICE_EXCLUSIVE, + /* H/W posion types. */ + SOFTLEAF_HWPOISON, + /* Marker types. */ + SOFTLEAF_MARKER, +}; + +/** + * softleaf_mk_none() - Create an empty ('none') leaf entry. + * Returns: empty leaf entry. + */ +static inline softleaf_t softleaf_mk_none(void) +{ + return ((softleaf_t) { 0 }); +} + +/** + * softleaf_from_pte() - Obtain a leaf entry from a PTE entry. + * @pte: PTE entry. + * + * If @pte is present (therefore not a leaf entry) the function returns an= empty + * leaf entry. Otherwise, it returns a leaf entry. + * + * Returns: Leaf entry. + */ +static inline softleaf_t softleaf_from_pte(pte_t pte) +{ + if (pte_present(pte)) + return softleaf_mk_none(); + + /* Temporary until swp_entry_t eliminated. */ + return pte_to_swp_entry(pte); +} + +/** + * softleaf_is_none() - Is the leaf entry empty? + * @entry: Leaf entry. + * + * Empty entries are typically the result of a 'none' page table leaf entry + * being converted to a leaf entry. + * + * Returns: true if the entry is empty, false otherwise. + */ +static inline bool softleaf_is_none(softleaf_t entry) +{ + return entry.val =3D=3D 0; +} + +/** + * softleaf_type() - Identify the type of leaf entry. + * @enntry: Leaf entry. + * + * Returns: the leaf entry type associated with @entry. + */ +static inline enum softleaf_type softleaf_type(softleaf_t entry) +{ + unsigned int type_num; + + if (softleaf_is_none(entry)) + return SOFTLEAF_NONE; + + type_num =3D entry.val >> LEAF_TYPE_SHIFT; + + if (type_num < MAX_SWAPFILES) + return SOFTLEAF_SWAP; + + switch (type_num) { +#ifdef CONFIG_MIGRATION + case SWP_MIGRATION_READ: + return SOFTLEAF_MIGRATION_READ; + case SWP_MIGRATION_READ_EXCLUSIVE: + return SOFTLEAF_MIGRATION_READ_EXCLUSIVE; + case SWP_MIGRATION_WRITE: + return SOFTLEAF_MIGRATION_WRITE; +#endif +#ifdef CONFIG_DEVICE_PRIVATE + case SWP_DEVICE_WRITE: + return SOFTLEAF_DEVICE_PRIVATE_WRITE; + case SWP_DEVICE_READ: + return SOFTLEAF_DEVICE_PRIVATE_READ; + case SWP_DEVICE_EXCLUSIVE: + return SOFTLEAF_DEVICE_EXCLUSIVE; +#endif +#ifdef CONFIG_MEMORY_FAILURE + case SWP_HWPOISON: + return SOFTLEAF_HWPOISON; +#endif + case SWP_PTE_MARKER: + return SOFTLEAF_MARKER; + } + + /* Unknown entry type. */ + VM_WARN_ON_ONCE(1); + return SOFTLEAF_NONE; +} + +/** + * softleaf_is_swap() - Is this leaf entry a swap entry? + * @entry: Leaf entry. + * + * Returns: true if the leaf entry is a swap entry, otherwise false. + */ +static inline bool softleaf_is_swap(softleaf_t entry) +{ + return softleaf_type(entry) =3D=3D SOFTLEAF_SWAP; +} + +/** + * softleaf_is_swap() - Is this leaf entry a migration entry? + * @entry: Leaf entry. + * + * Returns: true if the leaf entry is a migration entry, otherwise false. + */ +static inline bool softleaf_is_migration(softleaf_t entry) +{ + switch (softleaf_type(entry)) { + case SOFTLEAF_MIGRATION_READ: + case SOFTLEAF_MIGRATION_READ_EXCLUSIVE: + case SOFTLEAF_MIGRATION_WRITE: + return true; + default: + return false; + } +} + +/** + * softleaf_is_device_private() - Is this leaf entry a device private entr= y? + * @entry: Leaf entry. + * + * Returns: true if the leaf entry is a device private entry, otherwise fa= lse. + */ +static inline bool softleaf_is_device_private(softleaf_t entry) +{ + switch (softleaf_type(entry)) { + case SOFTLEAF_DEVICE_PRIVATE_WRITE: + case SOFTLEAF_DEVICE_PRIVATE_READ: + return true; + default: + return false; + } +} + +static inline bool softleaf_is_device_exclusive(softleaf_t entry) +{ + return softleaf_type(entry) =3D=3D SOFTLEAF_DEVICE_EXCLUSIVE; +} + +/** + * softleaf_is_hwpoison() - Is this leaf entry a hardware poison entry? + * @entry: Leaf entry. + * + * Returns: true if the leaf entry is a hardware poison entry, otherwise f= alse. + */ +static inline bool softleaf_is_hwpoison(softleaf_t entry) +{ + return softleaf_type(entry) =3D=3D SOFTLEAF_HWPOISON; +} + +/** + * softleaf_is_marker() - Is this leaf entry a marker? + * @entry: Leaf entry. + * + * Returns: true if the leaf entry is a marker entry, otherwise false. + */ +static inline bool softleaf_is_marker(softleaf_t entry) +{ + return softleaf_type(entry) =3D=3D SOFTLEAF_MARKER; +} + +/** + * softleaf_to_marker() - Obtain marker associated with leaf entry. + * @entry: Leaf entry, softleaf_is_marker(@entry) must return true. + * + * Returns: Marker associated with the leaf entry. + */ +static inline pte_marker softleaf_to_marker(softleaf_t entry) +{ + VM_WARN_ON_ONCE(!softleaf_is_marker(entry)); + + return swp_offset(entry) & PTE_MARKER_MASK; +} + +/** + * softleaf_has_pfn() - Does this leaf entry encode a valid PFN number? + * @entry: Leaf entry. + * + * A pfn swap entry is a special type of swap entry that always has a pfn = stored + * in the swap offset. They can either be used to represent unaddressable = device + * memory, to restrict access to a page undergoing migration or to represe= nt a + * pfn which has been hwpoisoned and unmapped. + * + * Returns: true if the leaf entry encodes a PFN, otherwise false. + */ +static inline bool softleaf_has_pfn(softleaf_t entry) +{ + /* Make sure the swp offset can always store the needed fields. */ + BUILD_BUG_ON(SWP_TYPE_SHIFT < SWP_PFN_BITS); + + if (softleaf_is_migration(entry)) + return true; + if (softleaf_is_device_private(entry)) + return true; + if (softleaf_is_device_exclusive(entry)) + return true; + if (softleaf_is_hwpoison(entry)) + return true; + + return false; +} + +/** + * softleaf_to_pfn() - Obtain PFN encoded within leaf entry. + * @entry: Leaf entry, softleaf_has_pfn(@entry) must return true. + * + * Returns: The PFN associated with the leaf entry. + */ +static inline unsigned long softleaf_to_pfn(softleaf_t entry) +{ + VM_WARN_ON_ONCE(!softleaf_has_pfn(entry)); + + /* Temporary until swp_entry_t eliminated. */ + return swp_offset_pfn(entry); +} + +/** + * softleaf_to_page() - Obtains struct page for PFN encoded within leaf en= try. + * @entry: Leaf entry, softleaf_has_pfn(@entry) must return true. + * + * Returns: Pointer to the struct page associated with the leaf entry's PF= N. + */ +static inline struct page *softleaf_to_page(softleaf_t entry) +{ + VM_WARN_ON_ONCE(!softleaf_has_pfn(entry)); + + /* Temporary until swp_entry_t eliminated. */ + return pfn_swap_entry_to_page(entry); +} + +/** + * softleaf_to_folio() - Obtains struct folio for PFN encoded within leaf = entry. + * @entry: Leaf entry, softleaf_has_pfn(@entry) must return true. + * + * Returns: Pointer to the struct folio associated with the leaf entry's P= FN. + * Returns: + */ +static inline struct folio *softleaf_to_folio(softleaf_t entry) +{ + VM_WARN_ON_ONCE(!softleaf_has_pfn(entry)); + + /* Temporary until swp_entry_t eliminated. */ + return pfn_swap_entry_folio(entry); +} + +/** + * softleaf_is_poison_marker() - Is this leaf entry a poison marker? + * @entry: Leaf entry. + * + * The poison marker is set via UFFDIO_POISON. Userfaultfd-specific. + * + * Returns: true if the leaf entry is a poison marker, otherwise false. + */ +static inline bool softleaf_is_poison_marker(softleaf_t entry) +{ + if (!softleaf_is_marker(entry)) + return false; + + return softleaf_to_marker(entry) & PTE_MARKER_POISONED; +} + +/** + * softleaf_is_guard_marker() - Is this leaf entry a guard region marker? + * @entry: Leaf entry. + * + * Returns: true if the leaf entry is a guard marker, otherwise false. + */ +static inline bool softleaf_is_guard_marker(softleaf_t entry) +{ + if (!softleaf_is_marker(entry)) + return false; + + return softleaf_to_marker(entry) & PTE_MARKER_GUARD; +} + +/** + * softleaf_is_uffd_wp_marker() - Is this leaf entry a userfautlfd write p= rotect + * marker? + * @entry: Leaf entry. + * + * Userfaultfd-specific. + * + * Returns: true if the leaf entry is a UFFD WP marker, otherwise false. + */ +static inline bool softleaf_is_uffd_wp_marker(softleaf_t entry) +{ + if (!softleaf_is_marker(entry)) + return false; + + return softleaf_to_marker(entry) & PTE_MARKER_UFFD_WP; +} + +/** + * pte_is_marker() - Does the PTE entry encode a marker leaf entry? + * @pte: PTE entry. + * + * Returns: true if this PTE is a marker leaf entry, otherwise false. + */ +static inline bool pte_is_marker(pte_t pte) +{ + return softleaf_is_marker(softleaf_from_pte(pte)); +} + +/** + * pte_is_uffd_wp_marker() - Does this PTE entry encode a userfaultfd write + * protect marker leaf entry? + * @pte: PTE entry. + * + * Returns: true if this PTE is a UFFD WP marker leaf entry, otherwise fal= se. + */ +static inline bool pte_is_uffd_wp_marker(pte_t pte) +{ + const softleaf_t entry =3D softleaf_from_pte(pte); + + return softleaf_is_uffd_wp_marker(entry); +} + +/** + * pte_is_uffd_marker() - Does this PTE entry encode a userfault-specific = marker + * leaf entry? + * @entry: Leaf entry. + * + * It's useful to be able to determine which leaf entries encode UFFD-spec= ific + * markers so we can handle these correctly. + * + * Returns: true if this PTE entry is a UFFD-specific marker, otherwise fa= lse. + */ +static inline bool pte_is_uffd_marker(pte_t pte) +{ + const softleaf_t entry =3D softleaf_from_pte(pte); + + if (!softleaf_is_marker(entry)) + return false; + + /* UFFD WP, poisoned swap entries are UFFD-handled. */ + if (softleaf_is_uffd_wp_marker(entry)) + return true; + if (softleaf_is_poison_marker(entry)) + return true; + + return false; +} + +#endif /* CONFIG_MMU */ +#endif /* _LINUX_SWAPOPS_H */ diff --git a/include/linux/mm_inline.h b/include/linux/mm_inline.h index f6a2b2d20016..ca7a18351797 100644 --- a/include/linux/mm_inline.h +++ b/include/linux/mm_inline.h @@ -8,7 +8,7 @@ #include #include #include -#include +#include =20 /** * folio_is_file_lru - Should the folio be on a file LRU or anon LRU? @@ -541,9 +541,9 @@ static inline bool mm_tlb_flush_nested(const struct mm_= struct *mm) * The caller should insert a new pte created with make_pte_marker(). */ static inline pte_marker copy_pte_marker( - swp_entry_t entry, struct vm_area_struct *dst_vma) + softleaf_t entry, struct vm_area_struct *dst_vma) { - pte_marker srcm =3D pte_marker_get(entry); + const pte_marker srcm =3D softleaf_to_marker(entry); /* Always copy error entries. */ pte_marker dstm =3D srcm & (PTE_MARKER_POISONED | PTE_MARKER_GUARD); =20 diff --git a/include/linux/mm_types.h b/include/linux/mm_types.h index 5021047485a9..4f66a3206a63 100644 --- a/include/linux/mm_types.h +++ b/include/linux/mm_types.h @@ -285,6 +285,31 @@ typedef struct { unsigned long val; } swp_entry_t; =20 +/** + * typedef softleaf_t - Describes a page table software leaf entry, abstra= cted + * from its architecture-specific encoding. + * + * Page table leaf entries are those which do not reference any descendent= page + * tables but rather either reference a data page, are an empty (or 'none' + * entry), or contain a non-present entry. + * + * If referencing another page table or a data page then the page table en= try is + * pertinent to hardware - that is it tells the hardware how to decode the= page + * table entry. + * + * Otherwise it is a software-defined leaf page table entry, which this ty= pe + * describes. See leafops.h and specifically @softleaf_type for a list of = all + * possible kinds of software leaf entry. + * + * A softleaf_t entry is abstracted from the hardware page table entry, so= is + * not architecture-specific. + * + * NOTE: While we transition from the confusing swp_entry_t type used for = this + * purpose, we simply alias this type. This will be removed once the + * transition is complete. + */ +typedef swp_entry_t softleaf_t; + #if defined(CONFIG_MEMCG) || defined(CONFIG_SLAB_OBJ_EXT) /* We have some extra room after the refcount in tail pages. */ #define NR_PAGES_IN_LARGE_FOLIO diff --git a/include/linux/swapops.h b/include/linux/swapops.h index d1f665935cfc..0a4b3f51ecf5 100644 --- a/include/linux/swapops.h +++ b/include/linux/swapops.h @@ -426,21 +426,6 @@ static inline swp_entry_t make_pte_marker_entry(pte_ma= rker marker) return swp_entry(SWP_PTE_MARKER, marker); } =20 -static inline bool is_pte_marker_entry(swp_entry_t entry) -{ - return swp_type(entry) =3D=3D SWP_PTE_MARKER; -} - -static inline pte_marker pte_marker_get(swp_entry_t entry) -{ - return swp_offset(entry) & PTE_MARKER_MASK; -} - -static inline bool is_pte_marker(pte_t pte) -{ - return is_swap_pte(pte) && is_pte_marker_entry(pte_to_swp_entry(pte)); -} - static inline pte_t make_pte_marker(pte_marker marker) { return swp_entry_to_pte(make_pte_marker_entry(marker)); @@ -451,24 +436,11 @@ static inline swp_entry_t make_poisoned_swp_entry(voi= d) return make_pte_marker_entry(PTE_MARKER_POISONED); } =20 -static inline int is_poisoned_swp_entry(swp_entry_t entry) -{ - return is_pte_marker_entry(entry) && - (pte_marker_get(entry) & PTE_MARKER_POISONED); - -} - static inline swp_entry_t make_guard_swp_entry(void) { return make_pte_marker_entry(PTE_MARKER_GUARD); } =20 -static inline int is_guard_swp_entry(swp_entry_t entry) -{ - return is_pte_marker_entry(entry) && - (pte_marker_get(entry) & PTE_MARKER_GUARD); -} - static inline struct page *pfn_swap_entry_to_page(swp_entry_t entry) { struct page *p =3D pfn_to_page(swp_offset_pfn(entry)); diff --git a/include/linux/userfaultfd_k.h b/include/linux/userfaultfd_k.h index da0b4fcc566f..983c860a00f1 100644 --- a/include/linux/userfaultfd_k.h +++ b/include/linux/userfaultfd_k.h @@ -16,7 +16,7 @@ #include #include #include -#include +#include #include #include =20 @@ -434,32 +434,6 @@ static inline bool userfaultfd_wp_use_markers(struct v= m_area_struct *vma) return userfaultfd_wp_unpopulated(vma); } =20 -static inline bool pte_marker_entry_uffd_wp(swp_entry_t entry) -{ -#ifdef CONFIG_PTE_MARKER_UFFD_WP - return is_pte_marker_entry(entry) && - (pte_marker_get(entry) & PTE_MARKER_UFFD_WP); -#else - return false; -#endif -} - -static inline bool pte_marker_uffd_wp(pte_t pte) -{ -#ifdef CONFIG_PTE_MARKER_UFFD_WP - swp_entry_t entry; - - if (!is_swap_pte(pte)) - return false; - - entry =3D pte_to_swp_entry(pte); - - return pte_marker_entry_uffd_wp(entry); -#else - return false; -#endif -} - /* * Returns true if this is a swap pte and was uffd-wp wr-protected in eith= er * forms (pte marker or a normal swap pte), false otherwise. @@ -473,31 +447,10 @@ static inline bool pte_swp_uffd_wp_any(pte_t pte) if (pte_swp_uffd_wp(pte)) return true; =20 - if (pte_marker_uffd_wp(pte)) + if (pte_is_uffd_wp_marker(pte)) return true; #endif return false; } =20 - -static inline bool is_uffd_pte_marker(pte_t pte) -{ - swp_entry_t entry; - - if (pte_present(pte)) - return false; - - entry =3D pte_to_swp_entry(pte); - if (!is_pte_marker_entry(entry)) - return false; - - /* UFFD WP, poisoned swap entries are UFFD handled. */ - if (pte_marker_entry_uffd_wp(entry)) - return true; - if (is_poisoned_swp_entry(entry)) - return true; - - return false; -} - #endif /* _LINUX_USERFAULTFD_K_H */ diff --git a/mm/hmm.c b/mm/hmm.c index 43d4a91035ff..b11b4ebba945 100644 --- a/mm/hmm.c +++ b/mm/hmm.c @@ -244,7 +244,7 @@ static int hmm_vma_handle_pte(struct mm_walk *walk, uns= igned long addr, uint64_t pfn_req_flags =3D *hmm_pfn; uint64_t new_pfn_flags =3D 0; =20 - if (pte_none(pte) || pte_marker_uffd_wp(pte)) { + if (pte_none(pte) || pte_is_uffd_wp_marker(pte)) { required_fault =3D hmm_pte_need_fault(hmm_vma_walk, pfn_req_flags, 0); if (required_fault) diff --git a/mm/hugetlb.c b/mm/hugetlb.c index 01c784547d1e..a05edefec1ca 100644 --- a/mm/hugetlb.c +++ b/mm/hugetlb.c @@ -28,7 +28,7 @@ #include #include #include -#include +#include #include #include #include @@ -5662,17 +5662,17 @@ int copy_hugetlb_page_range(struct mm_struct *dst, = struct mm_struct *src, entry =3D huge_pte_clear_uffd_wp(entry); set_huge_pte_at(dst, addr, dst_pte, entry, sz); } else if (unlikely(is_hugetlb_entry_migration(entry))) { - swp_entry_t swp_entry =3D pte_to_swp_entry(entry); + softleaf_t softleaf =3D softleaf_from_pte(entry); bool uffd_wp =3D pte_swp_uffd_wp(entry); =20 - if (!is_readable_migration_entry(swp_entry) && cow) { + if (!is_readable_migration_entry(softleaf) && cow) { /* * COW mappings require pages in both * parent and child to be set to read. */ - swp_entry =3D make_readable_migration_entry( - swp_offset(swp_entry)); - entry =3D swp_entry_to_pte(swp_entry); + softleaf =3D make_readable_migration_entry( + swp_offset(softleaf)); + entry =3D swp_entry_to_pte(softleaf); if (userfaultfd_wp(src_vma) && uffd_wp) entry =3D pte_swp_mkuffd_wp(entry); set_huge_pte_at(src, addr, src_pte, entry, sz); @@ -5680,9 +5680,9 @@ int copy_hugetlb_page_range(struct mm_struct *dst, st= ruct mm_struct *src, if (!userfaultfd_wp(dst_vma)) entry =3D huge_pte_clear_uffd_wp(entry); set_huge_pte_at(dst, addr, dst_pte, entry, sz); - } else if (unlikely(is_pte_marker(entry))) { - pte_marker marker =3D copy_pte_marker( - pte_to_swp_entry(entry), dst_vma); + } else if (unlikely(pte_is_marker(entry))) { + const softleaf_t softleaf =3D softleaf_from_pte(entry); + const pte_marker marker =3D copy_pte_marker(softleaf, dst_vma); =20 if (marker) set_huge_pte_at(dst, addr, dst_pte, @@ -5798,7 +5798,7 @@ static void move_huge_pte(struct vm_area_struct *vma,= unsigned long old_addr, =20 pte =3D huge_ptep_get_and_clear(mm, old_addr, src_pte, sz); =20 - if (need_clear_uffd_wp && pte_marker_uffd_wp(pte)) + if (need_clear_uffd_wp && pte_is_uffd_wp_marker(pte)) huge_pte_clear(mm, new_addr, dst_pte, sz); else { if (need_clear_uffd_wp) { @@ -6617,7 +6617,7 @@ static vm_fault_t hugetlb_no_page(struct address_spac= e *mapping, * If this pte was previously wr-protected, keep it wr-protected even * if populated. */ - if (unlikely(pte_marker_uffd_wp(vmf->orig_pte))) + if (unlikely(pte_is_uffd_wp_marker(vmf->orig_pte))) new_pte =3D huge_pte_mkuffd_wp(new_pte); set_huge_pte_at(mm, vmf->address, vmf->pte, new_pte, huge_page_size(h)); =20 @@ -6750,9 +6750,9 @@ vm_fault_t hugetlb_fault(struct mm_struct *mm, struct= vm_area_struct *vma, */ return hugetlb_no_page(mapping, &vmf); =20 - if (is_pte_marker(vmf.orig_pte)) { + if (pte_is_marker(vmf.orig_pte)) { const pte_marker marker =3D - pte_marker_get(pte_to_swp_entry(vmf.orig_pte)); + softleaf_to_marker(softleaf_from_pte(vmf.orig_pte)); =20 if (marker & PTE_MARKER_POISONED) { ret =3D VM_FAULT_HWPOISON_LARGE | @@ -7080,7 +7080,7 @@ int hugetlb_mfill_atomic_pte(pte_t *dst_pte, * See comment about UFFD marker overwriting in * mfill_atomic_install_pte(). */ - if (!huge_pte_none(dst_ptep) && !is_uffd_pte_marker(dst_ptep)) + if (!huge_pte_none(dst_ptep) && !pte_is_uffd_marker(dst_ptep)) goto out_release_unlock; =20 if (folio_in_pagecache) @@ -7201,8 +7201,9 @@ long hugetlb_change_protection(struct vm_area_struct = *vma, if (unlikely(is_hugetlb_entry_hwpoisoned(pte))) { /* Nothing to do. */ } else if (unlikely(is_hugetlb_entry_migration(pte))) { - swp_entry_t entry =3D pte_to_swp_entry(pte); - struct folio *folio =3D pfn_swap_entry_folio(entry); + softleaf_t entry =3D softleaf_from_pte(pte); + + struct folio *folio =3D softleaf_to_folio(entry); pte_t newpte =3D pte; =20 if (is_writable_migration_entry(entry)) { @@ -7222,14 +7223,14 @@ long hugetlb_change_protection(struct vm_area_struc= t *vma, newpte =3D pte_swp_clear_uffd_wp(newpte); if (!pte_same(pte, newpte)) set_huge_pte_at(mm, address, ptep, newpte, psize); - } else if (unlikely(is_pte_marker(pte))) { + } else if (unlikely(pte_is_marker(pte))) { /* * Do nothing on a poison marker; page is * corrupted, permissions do not apply. Here * pte_marker_uffd_wp()=3D=3Dtrue implies !poison * because they're mutual exclusive. */ - if (pte_marker_uffd_wp(pte) && uffd_wp_resolve) + if (pte_is_uffd_wp_marker(pte) && uffd_wp_resolve) /* Safe to modify directly (non-present->none). */ huge_pte_clear(mm, address, ptep, psize); } else if (!huge_pte_none(pte)) { diff --git a/mm/madvise.c b/mm/madvise.c index fb1c86e630b6..2d5ad3cb37bb 100644 --- a/mm/madvise.c +++ b/mm/madvise.c @@ -29,7 +29,7 @@ #include #include #include -#include +#include #include #include =20 @@ -690,17 +690,16 @@ static int madvise_free_pte_range(pmd_t *pmd, unsigne= d long addr, * (page allocation + zeroing). */ if (!pte_present(ptent)) { - swp_entry_t entry; + softleaf_t entry =3D softleaf_from_pte(ptent); =20 - entry =3D pte_to_swp_entry(ptent); - if (!non_swap_entry(entry)) { + if (softleaf_is_swap(entry)) { max_nr =3D (end - addr) / PAGE_SIZE; nr =3D swap_pte_batch(pte, max_nr, ptent); nr_swap -=3D nr; free_swap_and_cache_nr(entry, nr); clear_not_present_full_ptes(mm, addr, pte, nr, tlb->fullmm); - } else if (is_hwpoison_entry(entry) || - is_poisoned_swp_entry(entry)) { + } else if (softleaf_is_hwpoison(entry) || + softleaf_is_poison_marker(entry)) { pte_clear_not_present_full(mm, addr, pte, tlb->fullmm); } continue; @@ -1071,8 +1070,9 @@ static bool is_valid_guard_vma(struct vm_area_struct = *vma, bool allow_locked) =20 static bool is_guard_pte_marker(pte_t ptent) { - return is_swap_pte(ptent) && - is_guard_swp_entry(pte_to_swp_entry(ptent)); + const softleaf_t entry =3D softleaf_from_pte(ptent); + + return softleaf_is_guard_marker(entry); } =20 static int guard_install_pud_entry(pud_t *pud, unsigned long addr, diff --git a/mm/memory.c b/mm/memory.c index 4c3a7e09a159..7493ed084b99 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -60,7 +60,7 @@ #include #include #include -#include +#include #include #include #include @@ -109,7 +109,7 @@ static __always_inline bool vmf_orig_pte_uffd_wp(struct= vm_fault *vmf) if (!(vmf->flags & FAULT_FLAG_ORIG_PTE_VALID)) return false; =20 - return pte_marker_uffd_wp(vmf->orig_pte); + return pte_is_uffd_wp_marker(vmf->orig_pte); } =20 /* @@ -927,10 +927,10 @@ copy_nonpresent_pte(struct mm_struct *dst_mm, struct = mm_struct *src_mm, { vm_flags_t vm_flags =3D dst_vma->vm_flags; pte_t orig_pte =3D ptep_get(src_pte); + softleaf_t entry =3D softleaf_from_pte(orig_pte); pte_t pte =3D orig_pte; struct folio *folio; struct page *page; - swp_entry_t entry =3D pte_to_swp_entry(orig_pte); =20 if (likely(!non_swap_entry(entry))) { if (swap_duplicate(entry) < 0) @@ -1016,7 +1016,7 @@ copy_nonpresent_pte(struct mm_struct *dst_mm, struct = mm_struct *src_mm, if (try_restore_exclusive_pte(src_vma, addr, src_pte, orig_pte)) return -EBUSY; return -ENOENT; - } else if (is_pte_marker_entry(entry)) { + } else if (softleaf_is_marker(entry)) { pte_marker marker =3D copy_pte_marker(entry, dst_vma); =20 if (marker) @@ -1717,14 +1717,14 @@ static inline int zap_nonpresent_ptes(struct mmu_ga= ther *tlb, unsigned int max_nr, unsigned long addr, struct zap_details *details, int *rss, bool *any_skipped) { - swp_entry_t entry; + softleaf_t entry; int nr =3D 1; =20 *any_skipped =3D true; - entry =3D pte_to_swp_entry(ptent); - if (is_device_private_entry(entry) || - is_device_exclusive_entry(entry)) { - struct page *page =3D pfn_swap_entry_to_page(entry); + entry =3D softleaf_from_pte(ptent); + if (softleaf_is_device_private(entry) || + softleaf_is_device_exclusive(entry)) { + struct page *page =3D softleaf_to_page(entry); struct folio *folio =3D page_folio(page); =20 if (unlikely(!should_zap_folio(details, folio))) @@ -1739,7 +1739,7 @@ static inline int zap_nonpresent_ptes(struct mmu_gath= er *tlb, rss[mm_counter(folio)]--; folio_remove_rmap_pte(folio, page, vma); folio_put(folio); - } else if (!non_swap_entry(entry)) { + } else if (softleaf_is_swap(entry)) { /* Genuine swap entries, hence a private anon pages */ if (!should_zap_cows(details)) return 1; @@ -1747,20 +1747,20 @@ static inline int zap_nonpresent_ptes(struct mmu_ga= ther *tlb, nr =3D swap_pte_batch(pte, max_nr, ptent); rss[MM_SWAPENTS] -=3D nr; free_swap_and_cache_nr(entry, nr); - } else if (is_migration_entry(entry)) { - struct folio *folio =3D pfn_swap_entry_folio(entry); + } else if (softleaf_is_migration(entry)) { + struct folio *folio =3D softleaf_to_folio(entry); =20 if (!should_zap_folio(details, folio)) return 1; rss[mm_counter(folio)]--; - } else if (pte_marker_entry_uffd_wp(entry)) { + } else if (softleaf_is_uffd_wp_marker(entry)) { /* * For anon: always drop the marker; for file: only * drop the marker if explicitly requested. */ if (!vma_is_anonymous(vma) && !zap_drop_markers(details)) return 1; - } else if (is_guard_swp_entry(entry)) { + } else if (softleaf_is_guard_marker(entry)) { /* * Ordinary zapping should not remove guard PTE * markers. Only do so if we should remove PTE markers @@ -1768,7 +1768,8 @@ static inline int zap_nonpresent_ptes(struct mmu_gath= er *tlb, */ if (!zap_drop_markers(details)) return 1; - } else if (is_hwpoison_entry(entry) || is_poisoned_swp_entry(entry)) { + } else if (softleaf_is_hwpoison(entry) || + softleaf_is_poison_marker(entry)) { if (!should_zap_cows(details)) return 1; } else { @@ -4390,7 +4391,7 @@ static vm_fault_t pte_marker_clear(struct vm_fault *v= mf) * * This should also cover the case where e.g. the pte changed * quickly from a PTE_MARKER_UFFD_WP into PTE_MARKER_POISONED. - * So is_pte_marker() check is not enough to safely drop the pte. + * So pte_is_marker() check is not enough to safely drop the pte. */ if (pte_same(vmf->orig_pte, ptep_get(vmf->pte))) pte_clear(vmf->vma->vm_mm, vmf->address, vmf->pte); @@ -4424,8 +4425,8 @@ static vm_fault_t pte_marker_handle_uffd_wp(struct vm= _fault *vmf) =20 static vm_fault_t handle_pte_marker(struct vm_fault *vmf) { - swp_entry_t entry =3D pte_to_swp_entry(vmf->orig_pte); - unsigned long marker =3D pte_marker_get(entry); + const softleaf_t entry =3D softleaf_from_pte(vmf->orig_pte); + const pte_marker marker =3D softleaf_to_marker(entry); =20 /* * PTE markers should never be empty. If anything weird happened, @@ -4442,7 +4443,7 @@ static vm_fault_t handle_pte_marker(struct vm_fault *= vmf) if (marker & PTE_MARKER_GUARD) return VM_FAULT_SIGSEGV; =20 - if (pte_marker_entry_uffd_wp(entry)) + if (softleaf_is_uffd_wp_marker(entry)) return pte_marker_handle_uffd_wp(vmf); =20 /* This is an unknown pte marker */ @@ -4690,7 +4691,7 @@ vm_fault_t do_swap_page(struct vm_fault *vmf) } } else if (is_hwpoison_entry(entry)) { ret =3D VM_FAULT_HWPOISON; - } else if (is_pte_marker_entry(entry)) { + } else if (softleaf_is_marker(entry)) { ret =3D handle_pte_marker(vmf); } else { print_bad_pte(vma, vmf->address, vmf->orig_pte, NULL); diff --git a/mm/mincore.c b/mm/mincore.c index 151b2dbb783b..e77c5bc88fc7 100644 --- a/mm/mincore.c +++ b/mm/mincore.c @@ -14,7 +14,7 @@ #include #include #include -#include +#include #include #include #include @@ -42,7 +42,7 @@ static int mincore_hugetlb(pte_t *pte, unsigned long hmas= k, unsigned long addr, } else { const pte_t ptep =3D huge_ptep_get(walk->mm, addr, pte); =20 - if (huge_pte_none(ptep) || is_pte_marker(ptep)) + if (huge_pte_none(ptep) || pte_is_marker(ptep)) present =3D 0; else present =3D 1; @@ -187,7 +187,7 @@ static int mincore_pte_range(pmd_t *pmd, unsigned long = addr, unsigned long end, =20 step =3D 1; /* We need to do cache lookup too for UFFD pte markers */ - if (pte_none(pte) || is_uffd_pte_marker(pte)) + if (pte_none(pte) || pte_is_uffd_marker(pte)) __mincore_unmapped_range(addr, addr + PAGE_SIZE, vma, vec); else if (pte_present(pte)) { diff --git a/mm/mprotect.c b/mm/mprotect.c index ab4e06cd9a69..0bae241eb7aa 100644 --- a/mm/mprotect.c +++ b/mm/mprotect.c @@ -326,14 +326,14 @@ static long change_pte_range(struct mmu_gather *tlb, newpte =3D swp_entry_to_pte(entry); if (pte_swp_uffd_wp(oldpte)) newpte =3D pte_swp_mkuffd_wp(newpte); - } else if (is_pte_marker_entry(entry)) { + } else if (softleaf_is_marker(entry)) { /* * Ignore error swap entries unconditionally, * because any access should sigbus/sigsegv * anyway. */ - if (is_poisoned_swp_entry(entry) || - is_guard_swp_entry(entry)) + if (softleaf_is_poison_marker(entry) || + softleaf_is_guard_marker(entry)) continue; /* * If this is uffd-wp pte marker and we'd like diff --git a/mm/mremap.c b/mm/mremap.c index 8ad06cf50783..7c21b2ad13f6 100644 --- a/mm/mremap.c +++ b/mm/mremap.c @@ -17,7 +17,7 @@ #include #include #include -#include +#include #include #include #include @@ -288,7 +288,7 @@ static int move_ptes(struct pagetable_move_control *pmc, pte =3D move_pte(pte, old_addr, new_addr); pte =3D move_soft_dirty_pte(pte); =20 - if (need_clear_uffd_wp && pte_marker_uffd_wp(pte)) + if (need_clear_uffd_wp && pte_is_uffd_wp_marker(pte)) pte_clear(mm, new_addr, new_ptep); else { if (need_clear_uffd_wp) { diff --git a/mm/page_vma_mapped.c b/mm/page_vma_mapped.c index 137ce27ff68c..be20468fb5a9 100644 --- a/mm/page_vma_mapped.c +++ b/mm/page_vma_mapped.c @@ -3,7 +3,7 @@ #include #include #include -#include +#include =20 #include "internal.h" =20 @@ -107,15 +107,12 @@ static bool check_pte(struct page_vma_mapped_walk *pv= mw, unsigned long pte_nr) pte_t ptent =3D ptep_get(pvmw->pte); =20 if (pvmw->flags & PVMW_MIGRATION) { - swp_entry_t entry; - if (!is_swap_pte(ptent)) - return false; - entry =3D pte_to_swp_entry(ptent); + const softleaf_t entry =3D softleaf_from_pte(ptent); =20 - if (!is_migration_entry(entry)) + if (!softleaf_is_migration(entry)) return false; =20 - pfn =3D swp_offset_pfn(entry); + pfn =3D softleaf_to_pfn(entry); } else if (is_swap_pte(ptent)) { swp_entry_t entry; =20 diff --git a/mm/shmem.c b/mm/shmem.c index 6580f3cd24bb..395ca58ac4a5 100644 --- a/mm/shmem.c +++ b/mm/shmem.c @@ -66,7 +66,7 @@ static struct vfsmount *shm_mnt __ro_after_init; #include #include #include -#include +#include #include #include #include @@ -2286,7 +2286,8 @@ static int shmem_swapin_folio(struct inode *inode, pg= off_t index, struct address_space *mapping =3D inode->i_mapping; struct mm_struct *fault_mm =3D vma ? vma->vm_mm : NULL; struct shmem_inode_info *info =3D SHMEM_I(inode); - swp_entry_t swap, index_entry; + swp_entry_t swap; + softleaf_t index_entry; struct swap_info_struct *si; struct folio *folio =3D NULL; bool skip_swapcache =3D false; @@ -2298,7 +2299,7 @@ static int shmem_swapin_folio(struct inode *inode, pg= off_t index, swap =3D index_entry; *foliop =3D NULL; =20 - if (is_poisoned_swp_entry(index_entry)) + if (softleaf_is_poison_marker(index_entry)) return -EIO; =20 si =3D get_swap_device(index_entry); diff --git a/mm/userfaultfd.c b/mm/userfaultfd.c index cc4ce205bbec..055ec1050776 100644 --- a/mm/userfaultfd.c +++ b/mm/userfaultfd.c @@ -10,7 +10,7 @@ #include #include #include -#include +#include #include #include #include @@ -208,7 +208,7 @@ int mfill_atomic_install_pte(pmd_t *dst_pmd, * MISSING|WP registered, we firstly wr-protect a none pte which has no * page cache page backing it, then access the page. */ - if (!pte_none(dst_ptep) && !is_uffd_pte_marker(dst_ptep)) + if (!pte_none(dst_ptep) && !pte_is_uffd_marker(dst_ptep)) goto out_unlock; =20 if (page_in_cache) { @@ -590,7 +590,7 @@ static __always_inline ssize_t mfill_atomic_hugetlb( if (!uffd_flags_mode_is(flags, MFILL_ATOMIC_CONTINUE)) { const pte_t ptep =3D huge_ptep_get(dst_mm, dst_addr, dst_pte); =20 - if (!huge_pte_none(ptep) && !is_uffd_pte_marker(ptep)) { + if (!huge_pte_none(ptep) && !pte_is_uffd_marker(ptep)) { err =3D -EEXIST; hugetlb_vma_unlock_read(dst_vma); mutex_unlock(&hugetlb_fault_mutex_table[hash]); --=20 2.51.0 From nobody Fri Dec 19 21:47:07 2025 Received: from mx0a-00069f02.pphosted.com (mx0a-00069f02.pphosted.com [205.220.165.32]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C9C1F23C4ED; Sat, 8 Nov 2025 17:10:17 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=205.220.165.32 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1762621819; cv=fail; b=n82r+dioAjLOUHVEPZTEsdmXXT/SQjpwfRtd8za+67kxUkK6QitYYVhaiYRt8waqG6htSS3z1wF9MEuXN3BKQKwKp3IpWIsYZZVcXlhs39XkA/OqC9NdUDhZG5eM4Raq8H3J7fepcs28t63UBYN501wvHoKrADKi905sAFdYE/Y= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1762621819; c=relaxed/simple; bh=T68qemNrU9aXNg9BwcHhuUskC8JepUns48gZSOBTgIY=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: Content-Type:MIME-Version; b=cusVAKI2IOO1B6qtiE3GcmQZz3dLMBKfN9kmaG6QlgY8Jhaz/1RuuqB4K0FryyQGzzhL3ZOlykwbsJS66y8lCVEG4Vw70qiDl2JrrRcyGo4CIisI4JJVYDGboPbmZYJGvTGNHB+MEc9ycDG0s/9cq9PyzmUFDBVQTBcpqPaFJpA= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=oracle.com; spf=pass smtp.mailfrom=oracle.com; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b=X7E3oaiu; dkim=pass (1024-bit key) header.d=oracle.onmicrosoft.com header.i=@oracle.onmicrosoft.com header.b=kGsxY1Ca; arc=fail smtp.client-ip=205.220.165.32 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=oracle.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=oracle.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b="X7E3oaiu"; dkim=pass (1024-bit key) header.d=oracle.onmicrosoft.com header.i=@oracle.onmicrosoft.com header.b="kGsxY1Ca" Received: from pps.filterd (m0246627.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 5A8FGYMI024142; Sat, 8 Nov 2025 17:09:04 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=cc :content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s= corp-2025-04-25; bh=nYYuTMwRkRPS0ue5pIa/JHKJQGWQ3ZtYL9FYx6anL54=; b= X7E3oaiuheyZudFCWwtGtg7d3hqJo4YlBvVRmM7eMPUaJc5tSuNTAecOg4rigtV5 +HbvBjffBIufMWRpijliobmmOJH19fs87SWkwBXmAf4wK1P6qzSYmzKh6MYh2kCV T2HBFq7B7MYiUEy67TldBO8LFF6d1xRYVagyYOyUGvqZTOODO3QJ0ShGDzepiidD wVfF7IVLpm/WlHokSmYAH+cMsUFVpOh7SV9XKJRrwe9AddGr+JlxIhGjdt6EC4Ou 9WZqSmIW8AtYztAsnfIId+x389dLunmxJyiV7TcPNiilxXxna0k1DRarZ1f7Nt+i 1WiQXrMCFd6fEv+R6jHdcA== Received: from phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta01.appoci.oracle.com [138.1.114.2]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 4aa6vh84sm-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Sat, 08 Nov 2025 17:09:04 +0000 (GMT) Received: from pps.filterd (phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (8.18.1.2/8.18.1.2) with ESMTP id 5A8GoW0W007552; Sat, 8 Nov 2025 17:09:03 GMT Received: from mw6pr02cu001.outbound.protection.outlook.com (mail-westus2azon11012068.outbound.protection.outlook.com [52.101.48.68]) by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTPS id 4a9va76mus-2 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Sat, 08 Nov 2025 17:09:03 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=Hi0wTXq1GMgJhgjK3qTBY4x4ZkTth2nRsCuKaAmIWsdtoka10mSXz4QjFgLec1Hl2AwKFmQ53JCnZXmLwx1JivmmYuVkLVFatln2X1y9xUk6vhq+JF6vHs/rhm4HlbRpJUyzgZRqNipvtv8GpvIu0zxsTPcTBWeJadNHfUhYCDwAYHyK6QRzlJ5ltptbgq1GXRBB8TImph+ZswXSXNI3RNVimphc+3Qwqb2bPSsJ7cyS/70iJ2vBJ//aVWWiWNfyxTWEIk7FQrWCJKHl2Jg9zAFqPUzT+pWLcRmIE7oZWwZvXhlYsTGz6S9H8Rz0DR9lcP5Eo/k31YA5qWBweZzzfg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=nYYuTMwRkRPS0ue5pIa/JHKJQGWQ3ZtYL9FYx6anL54=; b=NFYosp1D6GNI3ibmPZuB1/V4zOjsBbMbSmRj4mItUbnGizESmGBYAU3BHqGJ+6JZf+lUx0HHVO4CchmZRZlVDWBQuPFXKbtnJwHaD5v+dBMul+SFjH11tITr5qPwicSzUTLFxY61wUHKwbSmes+N/gssi+BlvhmdEzOwCP8NH4sq5F+39nqbzI0SMmaDDRgcddpkN0GzKu/PjyGsxhx7Kn2uW9qPcv+pU2nIux3OW3Qh5s/KlCMdN5Aal4ruILyQ97k1JlEO+6gINjy56hzBRDII51DQ8VdReQX2aaKy59Um6Ohfj5IUzoDuQn0FW9jZqeGY283ZyEzNejt3aH5miQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=nYYuTMwRkRPS0ue5pIa/JHKJQGWQ3ZtYL9FYx6anL54=; b=kGsxY1Car2I5JyZ94vuFvAFk8GV8+iBPMPcqZFNQhOnLaL0UQ4hcE2iQ6ghdbX9Gb3hJVKZxnd019m63aScO4K8CSuB/5szoyqZ/SsDjfDM3YhMNRxJQQwNmXHOcXKhACuknsGYJFL2Pu48vA+X1t083IqBzZPyy5RCFNT/T3v8= Received: from DM4PR10MB8218.namprd10.prod.outlook.com (2603:10b6:8:1cc::16) by SJ5PPF04D2D7FA7.namprd10.prod.outlook.com (2603:10b6:a0f:fc02::785) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9298.15; Sat, 8 Nov 2025 17:08:58 +0000 Received: from DM4PR10MB8218.namprd10.prod.outlook.com ([fe80::2650:55cf:2816:5f2]) by DM4PR10MB8218.namprd10.prod.outlook.com ([fe80::2650:55cf:2816:5f2%7]) with mapi id 15.20.9298.010; Sat, 8 Nov 2025 17:08:58 +0000 From: Lorenzo Stoakes To: Andrew Morton Cc: Christian Borntraeger , Janosch Frank , Claudio Imbrenda , David Hildenbrand , Alexander Gordeev , Gerald Schaefer , Heiko Carstens , Vasily Gorbik , Sven Schnelle , Peter Xu , Alexander Viro , Christian Brauner , Jan Kara , Arnd Bergmann , Zi Yan , Baolin Wang , "Liam R . Howlett" , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Lance Yang , Muchun Song , Oscar Salvador , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Matthew Brost , Joshua Hahn , Rakie Kim , Byungchul Park , Gregory Price , Ying Huang , Alistair Popple , Axel Rasmussen , Yuanchu Xie , Wei Xu , Kemeng Shi , Kairui Song , Nhat Pham , Baoquan He , Chris Li , SeongJae Park , Matthew Wilcox , Jason Gunthorpe , Leon Romanovsky , Xu Xin , Chengming Zhou , Jann Horn , Miaohe Lin , Naoya Horiguchi , Pedro Falcato , Pasha Tatashin , Rik van Riel , Harry Yoo , Hugh Dickins , linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-s390@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, linux-arch@vger.kernel.org, damon@lists.linux.dev Subject: [PATCH v2 03/16] mm: avoid unnecessary uses of is_swap_pte() Date: Sat, 8 Nov 2025 17:08:17 +0000 Message-ID: X-Mailer: git-send-email 2.51.0 In-Reply-To: References: Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: LNXP265CA0002.GBRP265.PROD.OUTLOOK.COM (2603:10a6:600:5e::14) To DM4PR10MB8218.namprd10.prod.outlook.com (2603:10b6:8:1cc::16) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DM4PR10MB8218:EE_|SJ5PPF04D2D7FA7:EE_ X-MS-Office365-Filtering-Correlation-Id: dbd14f20-d966-4f2e-9f4a-08de1ee98142 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|1800799024|7416014|376014|366016; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?CrzQNzkMROMG94lZ3CUBmB41DlplZGpfCSDQutx0yAkwvPywNLxVemLdOkQ+?= =?us-ascii?Q?vqxUMqU/3cbXnz+bzzMUveY9gAqrVhyr9YAoT6QEqDGe7XFT/1ofduLZeO9N?= =?us-ascii?Q?b3dezT1tL302mrDZMBlYZPYIIP3DGUCFk5h6HCA5Rq6flhX6cW0mVizLUTRu?= =?us-ascii?Q?wPwjf1qWeS2xKs9Cwlx1wKsJeSlJjKy1N/f7BUuPoluXWGN+19kBUF9n4vNp?= =?us-ascii?Q?5b28e+hEJSLsRB1LqreHneQb+4PHnalUXJEEh0qVsiaLlNAkFSYUclMTSYVM?= =?us-ascii?Q?ubTP+pqagUY8MEvJzRvIQ2voO+XXDoMA47UG8BvZDYEYzkOjXp/3pa9jW/jC?= =?us-ascii?Q?sOg8FAKSBwp8RXXh7ANxHHM81uNZOwPIYz8aiD6kSC5wTSyzv88TfoRofCoO?= =?us-ascii?Q?XmI/+tY1WHaatJMonBGPjQMbEf7hw0gqW52pEf0usjSnmnL4qT8gTcUn1QSH?= =?us-ascii?Q?zwOHYqxCWyAwnt5MnJgGaiMr4LD0m9oWmzpZqqystYY1W5qNtlQ7BX6SbF0R?= =?us-ascii?Q?fbfCjE2M1tCHKKQUIXX+n2Bj+EDabUey3jBtanXRL0ZDz4/+VrwkfOlOvFDn?= =?us-ascii?Q?2VGODfAK7MBwtNDN/Xjkr069AV7zrAw06NAfCQXflwMjsQfXvnzNTfPitbub?= =?us-ascii?Q?vLQ745SBxE946GFyhErEsQbxIjd6RDaPkcmmUr93bVnRo4rpqXbrRl+L+SEX?= =?us-ascii?Q?Pr7woAjQHzcPmIgaruKIbBuWNqkUpugLcLI2jWspYVfg20LoCHX4brflN2us?= =?us-ascii?Q?ie2UTgxrQCDMc5GwiL96fpYErcPHREQyI8ng0kq7Uj2XZIy+hpjib0YYLnq/?= =?us-ascii?Q?gmKchMnygHozivbiyss+P7/ldKwtevQXg9SFW0MOPQEDD7FLxfpW+47hA9Ux?= =?us-ascii?Q?UlOPPvEVU8+pQDQtCoXVvvkDCAbioBrnZCdTvKbsxCpg3/KBP59p90BPrRr1?= =?us-ascii?Q?0z13V7CSnrQmPc1cOV6fIqP5UEJUFhZyPJ7wS5W5NEzyAfF7Qav6XGgoY7nH?= =?us-ascii?Q?ziKbgrn/ngmN3Koa7KrFg+X3e3yt7wfgRRuxQpRHcC6Vzf/mlXZ5d+/OvMZ6?= =?us-ascii?Q?cYV7XJnDa/55ZRgeBjFHWHNOwtqjiC14rUf94iURO3C3zwBID8/boRrSvFjO?= =?us-ascii?Q?hFIziB6SdhPl/O+/MCvqjZxGMGMnKkLF4SKHlu2nLLXoU5phrm/HT9Ij/vQj?= =?us-ascii?Q?fF5DsNIvaWoIoCVNbNVLxJmI0BcluZjTBOe+juZHIq43GvM6qr1jz8R5waI0?= =?us-ascii?Q?42xoxB8uYf2q5QBmHpvstzYrrc/ht27HEs5TfMADh2tZq0HPCJiGXLqNmz0e?= =?us-ascii?Q?VKC1upGcZeIpCdpt5x1dzdNGvsGkfCrDmi5kPJfqmUe56BDbe91zeFhKD+k5?= =?us-ascii?Q?vGfVoMtGOlHGruT/n6LtKUgcH2T1PWRoXtEMm1qeMPCAPs1FzTXutr2tysl2?= =?us-ascii?Q?9q7KTdd/Wf2LIO8bkozBSglxieuoMbOb?= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DM4PR10MB8218.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(1800799024)(7416014)(376014)(366016);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?p9LLFZdVioY1na8MfOG3MaxrDuJ8S0/h67iwRos8cO2Nannefj3PtmFN60z6?= =?us-ascii?Q?Ns8V/YdZNVuqL4/dVvBqkNav/bljaeqqwDC3IZogjNe/1lRQ+ySWUfbpef/l?= =?us-ascii?Q?wr+S9uOodsYDpELiudP4O+cYVZ+04O6OzhSLZDD+hxbqZAdWmt82AMV8yM0J?= =?us-ascii?Q?TC2OoG91WCXjkq8+xkbeUupgk7modgdLJxGNNE080KQSAekCT/l02tBQZ6Bs?= =?us-ascii?Q?46BWzuzVNS1bxDu4roHyVk3Z1Ru15lZfp+Y35regynavVhwS0ewzb0VoQXiF?= =?us-ascii?Q?WP5PaEn0PXqR+z0a0/2h/4E8aSS2yadyhPqOPGZbiP0niWOBJ/0Cao3XJHtk?= =?us-ascii?Q?LJ7fAZJjESyVAr7QhKifxS4h8llvj662OgPi123etk8bk/WHTvybXowMCdhG?= =?us-ascii?Q?33Qu73bAq1PhqzZIXBlj7TXA98ztjrCx1BJVb48wKUnPtPSRO8TJ6N5lnlSd?= =?us-ascii?Q?SlBw6ImF2eGE3d6ssd6uLwST09fX2uR+pdKNOk5884YH55Lj+5HymBhAu4K7?= =?us-ascii?Q?/GPAZuSDZ67eSFauU02g4ipUDqodIdFFohCvJN+7fPaIPQnMrjlb8rJDzBH/?= =?us-ascii?Q?P7KVJNxpCTVMFjQ8Lotbo9rFJI/Cyn4hxMJcE1vtHziBdFOjOvGGWdXYSoiY?= =?us-ascii?Q?Tyey6SGRaWBGlgPZrd09HI0rqFwLRwcn84xCuRGZArh9eCBAa0IFaA/TaFmD?= =?us-ascii?Q?7xOr4R4RkjfTmgwXJxiJ6msIOoX3XiiP6g1rTiHY5TmRYZoTFfiUn36BbIGt?= =?us-ascii?Q?bnOEZRKlBQ0ERZKiZnGgQo0jITQ9rJoXrRJs7tKWg8eZULqTM9LBoEtDL09N?= =?us-ascii?Q?3MW6TgP1a0YB/3SFXHC+F8hMDMnAKAqXt6zNwGUCmI7thfpuDP7Av3OW6Xaf?= =?us-ascii?Q?tzFodnCuQ7sms2PIzFGoxCQxRcEyiVjiMp7OXw7Q/D3+pdcEgHT0siqQjHuR?= =?us-ascii?Q?tT90g3z9HorPEBauBhn1M1ubi3IgIWfnkrRbiVAvHSEsqLnyZ7mE4CzikG+U?= =?us-ascii?Q?SGvMhA6LKfdWWWF0YyTu01L7nqzc5VEJAoLCPMOlcmMkrhd4KV93V9ACCbBT?= =?us-ascii?Q?XvfhHX3ZPQP7EEwrccseER+FasfazHl7RRGQG3i0y9tfrdUOxwgwe4rsN2VP?= =?us-ascii?Q?aOnSWBTmHfU6zXn70dSBQFGbZ2FvaPkU/FbbLZ6EJE1pMK3tWKEn32lmqzQw?= =?us-ascii?Q?1rhPnotftmvgAmwcyvnyj//demm4kYQqZxrB8pfRiAW8K+z3DBE2CyzTU7ff?= =?us-ascii?Q?Y+Sf0Ob4xolcW1NpYEtH9i/mc62pJe7uEOHDbCLJiaVs72O/yEOcQt9SiOII?= =?us-ascii?Q?VylEfLZwEGTe04fXIN78nrIE27eAz05N9syAAzs8Ndc+2iALe5EdEKOXotBq?= =?us-ascii?Q?D/Zp1F5D0ascpxPiOXwJYafIP+nkgqJHw3G8Y+OUpDhxuc84hYSEzCekb1R1?= =?us-ascii?Q?qo6rBwxHzNxMzdszC/B/qJU2rRCPJ70TUoG4L5aL70pnlfmG+hnxXnLzy36j?= =?us-ascii?Q?CyRcumz58VhPmzVukz97z3jvnOrq9J51qTsDdq1k3QvXD8noRvpZ22WGV/8H?= =?us-ascii?Q?sGHBMU1VNVLt9l5T0tC4OaQiAXJWfT5QkBhTH0ULmklQEIJfMaIkesRVSmHO?= =?us-ascii?Q?GA=3D=3D?= X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: PVXhzf+058YjoPmi7/Xr5euO1pqvhzsAjnfQlCgPpZQ39hagpwYtJtPuHFrqRkve1knEBuRE9xgIICR53ekl5mysFu0BNxgJoTOt/DvOawIO/2QyB2w85tFB+b30/0oambj/8jgWjmAuw3lhxpSyiJpjzykYz9cTL8yrB2IeOEc1eI+gSTuoFN2Fmal6Pys0Jpx6Cu/MCWpOXjl0qMKFcSYZHkDFYFo4x5i7vskk0Svu1pwb3jNcVk3LqeN7tDwKaKHcs4+mWd2fi9dUqsEAVC00ky6cdp9ZsednZjCqRf/wkHMcb6bsXkE8e+TsmJOPqseXwsRK2UznSPVIl2sHxkh4psWQWj0bP1iAZfX/JMI+PBsqORvzaJnkw2RK4yJnvCktzqSruFJrc3B2RjTWmhKg9iAk1mTU9nUqvBaijP0NYV67FHPugZyU2sPYEerh/6c65VS7c/VKQeqcBxfIaBoelYe7JRT+RUnbWmwMYGwzPXlJc1HSyFeRN+KRovPvsy1bY51CpDVtSctBU1ydjg8BZgy3bke+4RCMjGBq9hjUPbDxP+F8bUWbpADv28SX5Y+uB6fBBPBuvtwWm+fpsL0RRYdjN9YxVArZwFgrSRI= X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: dbd14f20-d966-4f2e-9f4a-08de1ee98142 X-MS-Exchange-CrossTenant-AuthSource: DM4PR10MB8218.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 Nov 2025 17:08:57.9676 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: Z1gTtSvkccWqqg/PdDJv3oBDzKrTeqt9WvXyXUsSzo9ARk0w+6jGwYYncyvyxVMfS89fMdP7JH2RXKS5r+mqaswVNvXxIjYihp6kALojbN8= X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ5PPF04D2D7FA7 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1121,Hydra:6.1.9,FMLib:17.12.100.49 definitions=2025-11-08_04,2025-11-06_01,2025-10-01_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 adultscore=0 malwarescore=0 spamscore=0 mlxscore=0 bulkscore=0 suspectscore=0 mlxlogscore=999 phishscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2510240000 definitions=main-2511080138 X-Proofpoint-Spam-Details-Enc: AW1haW4tMjUxMTA4MDEwOSBTYWx0ZWRfX20PbHo7tL2P1 Xuvq/GdHWOC0NRc0MUHDestaT2LcYdG3cSYezvZPFVDybarRLDDbS12rrGS745JnOVGXB0dn2fF CKRj3EIiPqaUUD6Xr+lZBIxx+15FmGb3LTboTnk5jYxIHUJj4BXRHB4e7S/JcY0AXRTAUYavUz+ 1jZ7n5VuQeYu8k4tCdtC9Y5Yw5XEoI7zbJdVErg24cYUmzwd72124x7+kN1HOPYcCyknvkf6P2S J4etsM1x/z8CDjaiM4LgX1i4Jirl4Nx1ODidgsC8d002Kg+SvLWEg1iXHwdFqevEhZJ3wfTZONZ nsAUpnvog9qiTYPg/myhUMvgd+MefzPY5XpWzDyO8r24wSE5dTK+Zr1SSF1KrBJjbYDmWi8JFHm Nlh99v7tsycdKnln2b1VEMwgT6EVtg== X-Authority-Analysis: v=2.4 cv=IKEPywvG c=1 sm=1 tr=0 ts=690f7930 cx=c_pps a=XiAAW1AwiKB2Y8Wsi+sD2Q==:117 a=XiAAW1AwiKB2Y8Wsi+sD2Q==:17 a=6eWqkTHjU83fiwn7nKZWdM+Sl24=:19 a=z/mQ4Ysz8XfWz/Q5cLBRGdckG28=:19 a=lCpzRmAYbLLaTzLvsPZ7Mbvzbb8=:19 a=xqWC_Br6kY4A:10 a=6UeiqGixMTsA:10 a=GoEa3M9JfhUA:10 a=VkNPw1HP01LnGYTKEx00:22 a=yPCof4ZbAAAA:8 a=360PBzQsUBXz8B3xgqMA:9 a=UhEZJTgQB8St2RibIkdl:22 a=Z5ABNNGmrOfJ6cZ5bIyy:22 a=QOGEsqRv6VhmHaoFNykA:22 X-Proofpoint-ORIG-GUID: MYXIVuPmrmzpOiMOi7TnugqitufDnkdo X-Proofpoint-GUID: MYXIVuPmrmzpOiMOi7TnugqitufDnkdo Content-Type: text/plain; charset="utf-8" There's an established convention in the kernel that we treat PTEs as containing swap entries (and the unfortunately named non-swap swap entries) should they be neither empty (i.e. pte_none() evaluating true) nor present (i.e. pte_present() evaluating true). However, there is some inconsistency in how this is applied, as we also have the is_swap_pte() helper which explicitly performs this check: /* check whether a pte points to a swap entry */ static inline int is_swap_pte(pte_t pte) { return !pte_none(pte) && !pte_present(pte); } As this represents a predicate, and it's logical to assume that in order to establish that a PTE entry can correctly be manipulated as a swap/non-swap entry, this predicate seems as if it must first be checked. But we instead, we far more often utilise the established convention of checking pte_none() / pte_present() before operating on entries as if they were swap/non-swap. This patch works towards correcting this inconsistency by removing all uses of is_swap_pte() where we are already in a position where we perform pte_none()/pte_present() checks anyway or otherwise it is clearly logical to do so. We also take advantage of the fact that pte_swp_uffd_wp() is only set on swap entries. Additionally, update comments referencing to is_swap_pte() and non_swap_entry(). No functional change intended. Signed-off-by: Lorenzo Stoakes --- fs/proc/task_mmu.c | 49 ++++++++++++++++++++++++----------- include/linux/userfaultfd_k.h | 3 +-- mm/hugetlb.c | 6 ++--- mm/internal.h | 6 ++--- mm/khugepaged.c | 29 +++++++++++---------- mm/migrate.c | 2 +- mm/mprotect.c | 43 ++++++++++++++---------------- mm/mremap.c | 7 +++-- mm/page_table_check.c | 13 ++++++---- mm/page_vma_mapped.c | 31 +++++++++++----------- 10 files changed, 104 insertions(+), 85 deletions(-) diff --git a/fs/proc/task_mmu.c b/fs/proc/task_mmu.c index 24d26b49d870..ddbf177ecc45 100644 --- a/fs/proc/task_mmu.c +++ b/fs/proc/task_mmu.c @@ -1017,7 +1017,9 @@ static void smaps_pte_entry(pte_t *pte, unsigned long= addr, young =3D pte_young(ptent); dirty =3D pte_dirty(ptent); present =3D true; - } else if (is_swap_pte(ptent)) { + } else if (pte_none(ptent)) { + smaps_pte_hole_lookup(addr, walk); + } else { swp_entry_t swpent =3D pte_to_swp_entry(ptent); =20 if (!non_swap_entry(swpent)) { @@ -1038,9 +1040,6 @@ static void smaps_pte_entry(pte_t *pte, unsigned long= addr, present =3D true; page =3D pfn_swap_entry_to_page(swpent); } - } else { - smaps_pte_hole_lookup(addr, walk); - return; } =20 if (!page) @@ -1611,6 +1610,9 @@ static inline void clear_soft_dirty(struct vm_area_st= ruct *vma, */ pte_t ptent =3D ptep_get(pte); =20 + if (pte_none(ptent)) + return; + if (pte_present(ptent)) { pte_t old_pte; =20 @@ -1620,7 +1622,7 @@ static inline void clear_soft_dirty(struct vm_area_st= ruct *vma, ptent =3D pte_wrprotect(old_pte); ptent =3D pte_clear_soft_dirty(ptent); ptep_modify_prot_commit(vma, addr, pte, old_pte, ptent); - } else if (is_swap_pte(ptent)) { + } else { ptent =3D pte_swp_clear_soft_dirty(ptent); set_pte_at(vma->vm_mm, addr, pte, ptent); } @@ -1923,6 +1925,9 @@ static pagemap_entry_t pte_to_pagemap_entry(struct pa= gemapread *pm, struct page *page =3D NULL; struct folio *folio; =20 + if (pte_none(pte)) + goto out; + if (pte_present(pte)) { if (pm->show_pfn) frame =3D pte_pfn(pte); @@ -1932,8 +1937,9 @@ static pagemap_entry_t pte_to_pagemap_entry(struct pa= gemapread *pm, flags |=3D PM_SOFT_DIRTY; if (pte_uffd_wp(pte)) flags |=3D PM_UFFD_WP; - } else if (is_swap_pte(pte)) { + } else { swp_entry_t entry; + if (pte_swp_soft_dirty(pte)) flags |=3D PM_SOFT_DIRTY; if (pte_swp_uffd_wp(pte)) @@ -1941,6 +1947,7 @@ static pagemap_entry_t pte_to_pagemap_entry(struct pa= gemapread *pm, entry =3D pte_to_swp_entry(pte); if (pm->show_pfn) { pgoff_t offset; + /* * For PFN swap offsets, keeping the offset field * to be PFN only to be compatible with old smaps. @@ -1969,6 +1976,8 @@ static pagemap_entry_t pte_to_pagemap_entry(struct pa= gemapread *pm, __folio_page_mapped_exclusively(folio, page)) flags |=3D PM_MMAP_EXCLUSIVE; } + +out: if (vma->vm_flags & VM_SOFTDIRTY) flags |=3D PM_SOFT_DIRTY; =20 @@ -2310,12 +2319,16 @@ static unsigned long pagemap_page_category(struct p= agemap_scan_private *p, struct vm_area_struct *vma, unsigned long addr, pte_t pte) { - unsigned long categories =3D 0; + unsigned long categories; + + if (pte_none(pte)) + return 0; =20 if (pte_present(pte)) { struct page *page; =20 - categories |=3D PAGE_IS_PRESENT; + categories =3D PAGE_IS_PRESENT; + if (!pte_uffd_wp(pte)) categories |=3D PAGE_IS_WRITTEN; =20 @@ -2329,10 +2342,11 @@ static unsigned long pagemap_page_category(struct p= agemap_scan_private *p, categories |=3D PAGE_IS_PFNZERO; if (pte_soft_dirty(pte)) categories |=3D PAGE_IS_SOFT_DIRTY; - } else if (is_swap_pte(pte)) { + } else { softleaf_t entry; =20 - categories |=3D PAGE_IS_SWAPPED; + categories =3D PAGE_IS_SWAPPED; + if (!pte_swp_uffd_wp_any(pte)) categories |=3D PAGE_IS_WRITTEN; =20 @@ -2360,12 +2374,12 @@ static void make_uffd_wp_pte(struct vm_area_struct = *vma, old_pte =3D ptep_modify_prot_start(vma, addr, pte); ptent =3D pte_mkuffd_wp(old_pte); ptep_modify_prot_commit(vma, addr, pte, old_pte, ptent); - } else if (is_swap_pte(ptent)) { - ptent =3D pte_swp_mkuffd_wp(ptent); - set_pte_at(vma->vm_mm, addr, pte, ptent); - } else { + } else if (pte_none(ptent)) { set_pte_at(vma->vm_mm, addr, pte, make_pte_marker(PTE_MARKER_UFFD_WP)); + } else { + ptent =3D pte_swp_mkuffd_wp(ptent); + set_pte_at(vma->vm_mm, addr, pte, ptent); } } =20 @@ -2434,6 +2448,9 @@ static unsigned long pagemap_hugetlb_category(pte_t p= te) { unsigned long categories =3D PAGE_IS_HUGE; =20 + if (pte_none(pte)) + return categories; + /* * According to pagemap_hugetlb_range(), file-backed HugeTLB * page cannot be swapped. So PAGE_IS_FILE is not checked for @@ -2441,6 +2458,7 @@ static unsigned long pagemap_hugetlb_category(pte_t p= te) */ if (pte_present(pte)) { categories |=3D PAGE_IS_PRESENT; + if (!huge_pte_uffd_wp(pte)) categories |=3D PAGE_IS_WRITTEN; if (!PageAnon(pte_page(pte))) @@ -2449,8 +2467,9 @@ static unsigned long pagemap_hugetlb_category(pte_t p= te) categories |=3D PAGE_IS_PFNZERO; if (pte_soft_dirty(pte)) categories |=3D PAGE_IS_SOFT_DIRTY; - } else if (is_swap_pte(pte)) { + } else { categories |=3D PAGE_IS_SWAPPED; + if (!pte_swp_uffd_wp_any(pte)) categories |=3D PAGE_IS_WRITTEN; if (pte_swp_soft_dirty(pte)) diff --git a/include/linux/userfaultfd_k.h b/include/linux/userfaultfd_k.h index 983c860a00f1..96b089dff4ef 100644 --- a/include/linux/userfaultfd_k.h +++ b/include/linux/userfaultfd_k.h @@ -441,9 +441,8 @@ static inline bool userfaultfd_wp_use_markers(struct vm= _area_struct *vma) static inline bool pte_swp_uffd_wp_any(pte_t pte) { #ifdef CONFIG_PTE_MARKER_UFFD_WP - if (!is_swap_pte(pte)) + if (pte_present(pte)) return false; - if (pte_swp_uffd_wp(pte)) return true; =20 diff --git a/mm/hugetlb.c b/mm/hugetlb.c index a05edefec1ca..a74cde267c2a 100644 --- a/mm/hugetlb.c +++ b/mm/hugetlb.c @@ -5798,13 +5798,13 @@ static void move_huge_pte(struct vm_area_struct *vm= a, unsigned long old_addr, =20 pte =3D huge_ptep_get_and_clear(mm, old_addr, src_pte, sz); =20 - if (need_clear_uffd_wp && pte_is_uffd_wp_marker(pte)) + if (need_clear_uffd_wp && pte_is_uffd_wp_marker(pte)) { huge_pte_clear(mm, new_addr, dst_pte, sz); - else { + } else { if (need_clear_uffd_wp) { if (pte_present(pte)) pte =3D huge_pte_clear_uffd_wp(pte); - else if (is_swap_pte(pte)) + else pte =3D pte_swp_clear_uffd_wp(pte); } set_huge_pte_at(mm, new_addr, dst_pte, pte, sz); diff --git a/mm/internal.h b/mm/internal.h index 116a1ba85e66..9465129367a4 100644 --- a/mm/internal.h +++ b/mm/internal.h @@ -325,8 +325,7 @@ unsigned int folio_pte_batch(struct folio *folio, pte_t= *ptep, pte_t pte, /** * pte_move_swp_offset - Move the swap entry offset field of a swap pte * forward or backward by delta - * @pte: The initial pte state; is_swap_pte(pte) must be true and - * non_swap_entry() must be false. + * @pte: The initial pte state; must be a swap entry * @delta: The direction and the offset we are moving; forward if delta * is positive; backward if delta is negative * @@ -352,8 +351,7 @@ static inline pte_t pte_move_swp_offset(pte_t pte, long= delta) =20 /** * pte_next_swp_offset - Increment the swap entry offset field of a swap p= te. - * @pte: The initial pte state; is_swap_pte(pte) must be true and - * non_swap_entry() must be false. + * @pte: The initial pte state; must be a swap entry. * * Increments the swap offset, while maintaining all other fields, includi= ng * swap type, and any swp pte bits. The resulting pte is returned. diff --git a/mm/khugepaged.c b/mm/khugepaged.c index f6ed1072ed6e..a97ff7bcb232 100644 --- a/mm/khugepaged.c +++ b/mm/khugepaged.c @@ -1019,7 +1019,8 @@ static int __collapse_huge_page_swapin(struct mm_stru= ct *mm, } =20 vmf.orig_pte =3D ptep_get_lockless(pte); - if (!is_swap_pte(vmf.orig_pte)) + if (pte_none(vmf.orig_pte) || + pte_present(vmf.orig_pte)) continue; =20 vmf.pte =3D pte; @@ -1276,7 +1277,19 @@ static int hpage_collapse_scan_pmd(struct mm_struct = *mm, for (addr =3D start_addr, _pte =3D pte; _pte < pte + HPAGE_PMD_NR; _pte++, addr +=3D PAGE_SIZE) { pte_t pteval =3D ptep_get(_pte); - if (is_swap_pte(pteval)) { + if (pte_none_or_zero(pteval)) { + ++none_or_zero; + if (!userfaultfd_armed(vma) && + (!cc->is_khugepaged || + none_or_zero <=3D khugepaged_max_ptes_none)) { + continue; + } else { + result =3D SCAN_EXCEED_NONE_PTE; + count_vm_event(THP_SCAN_EXCEED_NONE_PTE); + goto out_unmap; + } + } + if (!pte_present(pteval)) { ++unmapped; if (!cc->is_khugepaged || unmapped <=3D khugepaged_max_ptes_swap) { @@ -1296,18 +1309,6 @@ static int hpage_collapse_scan_pmd(struct mm_struct = *mm, goto out_unmap; } } - if (pte_none_or_zero(pteval)) { - ++none_or_zero; - if (!userfaultfd_armed(vma) && - (!cc->is_khugepaged || - none_or_zero <=3D khugepaged_max_ptes_none)) { - continue; - } else { - result =3D SCAN_EXCEED_NONE_PTE; - count_vm_event(THP_SCAN_EXCEED_NONE_PTE); - goto out_unmap; - } - } if (pte_uffd_wp(pteval)) { /* * Don't collapse the page if any of the small diff --git a/mm/migrate.c b/mm/migrate.c index ceee354ef215..862b2e261cf9 100644 --- a/mm/migrate.c +++ b/mm/migrate.c @@ -492,7 +492,7 @@ void migration_entry_wait(struct mm_struct *mm, pmd_t *= pmd, pte =3D ptep_get(ptep); pte_unmap(ptep); =20 - if (!is_swap_pte(pte)) + if (pte_none(pte) || pte_present(pte)) goto out; =20 entry =3D pte_to_swp_entry(pte); diff --git a/mm/mprotect.c b/mm/mprotect.c index 0bae241eb7aa..a3e360a8cdec 100644 --- a/mm/mprotect.c +++ b/mm/mprotect.c @@ -297,7 +297,26 @@ static long change_pte_range(struct mmu_gather *tlb, prot_commit_flush_ptes(vma, addr, pte, oldpte, ptent, nr_ptes, /* idx =3D */ 0, /* set_write =3D */ false, tlb); pages +=3D nr_ptes; - } else if (is_swap_pte(oldpte)) { + } else if (pte_none(oldpte)) { + /* + * Nobody plays with any none ptes besides + * userfaultfd when applying the protections. + */ + if (likely(!uffd_wp)) + continue; + + if (userfaultfd_wp_use_markers(vma)) { + /* + * For file-backed mem, we need to be able to + * wr-protect a none pte, because even if the + * pte is none, the page/swap cache could + * exist. Doing that by install a marker. + */ + set_pte_at(vma->vm_mm, addr, pte, + make_pte_marker(PTE_MARKER_UFFD_WP)); + pages++; + } + } else { swp_entry_t entry =3D pte_to_swp_entry(oldpte); pte_t newpte; =20 @@ -358,28 +377,6 @@ static long change_pte_range(struct mmu_gather *tlb, set_pte_at(vma->vm_mm, addr, pte, newpte); pages++; } - } else { - /* It must be an none page, or what else?.. */ - WARN_ON_ONCE(!pte_none(oldpte)); - - /* - * Nobody plays with any none ptes besides - * userfaultfd when applying the protections. - */ - if (likely(!uffd_wp)) - continue; - - if (userfaultfd_wp_use_markers(vma)) { - /* - * For file-backed mem, we need to be able to - * wr-protect a none pte, because even if the - * pte is none, the page/swap cache could - * exist. Doing that by install a marker. - */ - set_pte_at(vma->vm_mm, addr, pte, - make_pte_marker(PTE_MARKER_UFFD_WP)); - pages++; - } } } while (pte +=3D nr_ptes, addr +=3D nr_ptes * PAGE_SIZE, addr !=3D end); arch_leave_lazy_mmu_mode(); diff --git a/mm/mremap.c b/mm/mremap.c index 7c21b2ad13f6..62b6827abacf 100644 --- a/mm/mremap.c +++ b/mm/mremap.c @@ -158,6 +158,9 @@ static void drop_rmap_locks(struct vm_area_struct *vma) =20 static pte_t move_soft_dirty_pte(pte_t pte) { + if (pte_none(pte)) + return pte; + /* * Set soft dirty bit so we can notice * in userspace the ptes were moved. @@ -165,7 +168,7 @@ static pte_t move_soft_dirty_pte(pte_t pte) #ifdef CONFIG_MEM_SOFT_DIRTY if (pte_present(pte)) pte =3D pte_mksoft_dirty(pte); - else if (is_swap_pte(pte)) + else pte =3D pte_swp_mksoft_dirty(pte); #endif return pte; @@ -294,7 +297,7 @@ static int move_ptes(struct pagetable_move_control *pmc, if (need_clear_uffd_wp) { if (pte_present(pte)) pte =3D pte_clear_uffd_wp(pte); - else if (is_swap_pte(pte)) + else pte =3D pte_swp_clear_uffd_wp(pte); } set_ptes(mm, new_addr, new_ptep, pte, nr_ptes); diff --git a/mm/page_table_check.c b/mm/page_table_check.c index 4eeca782b888..43f75d2f7c36 100644 --- a/mm/page_table_check.c +++ b/mm/page_table_check.c @@ -185,12 +185,15 @@ static inline bool swap_cached_writable(swp_entry_t e= ntry) is_writable_migration_entry(entry); } =20 -static inline void page_table_check_pte_flags(pte_t pte) +static void page_table_check_pte_flags(pte_t pte) { - if (pte_present(pte) && pte_uffd_wp(pte)) - WARN_ON_ONCE(pte_write(pte)); - else if (is_swap_pte(pte) && pte_swp_uffd_wp(pte)) - WARN_ON_ONCE(swap_cached_writable(pte_to_swp_entry(pte))); + if (pte_present(pte)) { + WARN_ON_ONCE(pte_uffd_wp(pte) && pte_write(pte)); + } else if (pte_swp_uffd_wp(pte)) { + const swp_entry_t entry =3D pte_to_swp_entry(pte); + + WARN_ON_ONCE(swap_cached_writable(entry)); + } } =20 void __page_table_check_ptes_set(struct mm_struct *mm, pte_t *ptep, pte_t = pte, diff --git a/mm/page_vma_mapped.c b/mm/page_vma_mapped.c index be20468fb5a9..a4e23818f37f 100644 --- a/mm/page_vma_mapped.c +++ b/mm/page_vma_mapped.c @@ -16,6 +16,7 @@ static inline bool not_found(struct page_vma_mapped_walk = *pvmw) static bool map_pte(struct page_vma_mapped_walk *pvmw, pmd_t *pmdvalp, spinlock_t **ptlp) { + bool is_migration; pte_t ptent; =20 if (pvmw->flags & PVMW_SYNC) { @@ -26,6 +27,7 @@ static bool map_pte(struct page_vma_mapped_walk *pvmw, pm= d_t *pmdvalp, return !!pvmw->pte; } =20 + is_migration =3D pvmw->flags & PVMW_MIGRATION; again: /* * It is important to return the ptl corresponding to pte, @@ -41,11 +43,14 @@ static bool map_pte(struct page_vma_mapped_walk *pvmw, = pmd_t *pmdvalp, =20 ptent =3D ptep_get(pvmw->pte); =20 - if (pvmw->flags & PVMW_MIGRATION) { - if (!is_swap_pte(ptent)) + if (pte_none(ptent)) { + return false; + } else if (pte_present(ptent)) { + if (is_migration) return false; - } else if (is_swap_pte(ptent)) { + } else if (!is_migration) { swp_entry_t entry; + /* * Handle un-addressable ZONE_DEVICE memory. * @@ -66,8 +71,6 @@ static bool map_pte(struct page_vma_mapped_walk *pvmw, pm= d_t *pmdvalp, if (!is_device_private_entry(entry) && !is_device_exclusive_entry(entry)) return false; - } else if (!pte_present(ptent)) { - return false; } spin_lock(*ptlp); if (unlikely(!pmd_same(*pmdvalp, pmdp_get_lockless(pvmw->pmd)))) { @@ -113,21 +116,17 @@ static bool check_pte(struct page_vma_mapped_walk *pv= mw, unsigned long pte_nr) return false; =20 pfn =3D softleaf_to_pfn(entry); - } else if (is_swap_pte(ptent)) { - swp_entry_t entry; + } else if (pte_present(ptent)) { + pfn =3D pte_pfn(ptent); + } else { + const softleaf_t entry =3D softleaf_from_pte(ptent); =20 /* Handle un-addressable ZONE_DEVICE memory */ - entry =3D pte_to_swp_entry(ptent); - if (!is_device_private_entry(entry) && - !is_device_exclusive_entry(entry)) - return false; - - pfn =3D swp_offset_pfn(entry); - } else { - if (!pte_present(ptent)) + if (!softleaf_is_device_private(entry) && + !softleaf_is_device_exclusive(entry)) return false; =20 - pfn =3D pte_pfn(ptent); + pfn =3D softleaf_to_pfn(entry); } =20 if ((pfn + pte_nr - 1) < pvmw->pfn) --=20 2.51.0 From nobody Fri Dec 19 21:47:08 2025 Received: from mx0b-00069f02.pphosted.com (mx0b-00069f02.pphosted.com [205.220.177.32]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9ABBD145329; Sat, 8 Nov 2025 17:10:08 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=205.220.177.32 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1762621810; cv=fail; b=KP0XM5OhSteAMATXDXm5egTQo9Kimi2bnyesGmWuTkutEl7swsBNhJZ9M/FTF55At1meLWktXzAxhhSU1XAoQaquOWBoSSwrN/+/F+QdMyHM5uMscvA4CJHNvEKqi4vTbNgk6D8NgOHDtL3lFORH4Yam0OG2fMDdnqQmu0I/18o= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1762621810; c=relaxed/simple; bh=7XmOOpaf5TmZLupavCzYibr5KX6ZsU6IglQlUo1EsoE=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: Content-Type:MIME-Version; b=FIT9Rae2gyXKCXw22l86uwUXY3RF4a2MpDXVgJ9Bu1o61d6En+Jew4XPWkrpf7415tp2EpTE0oPKZRRf5oLQbEftnGwFPavv23Efl2jSgCwpn0jlur6SXzFmKY0TydHumxIGIOxvkSaxGhRBVGnRzZ992y9+eTMvl0GlJVrae+o= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=oracle.com; spf=pass smtp.mailfrom=oracle.com; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b=Gkl1S+rM; dkim=pass (1024-bit key) header.d=oracle.onmicrosoft.com header.i=@oracle.onmicrosoft.com header.b=o2Uwfnma; arc=fail smtp.client-ip=205.220.177.32 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=oracle.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=oracle.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b="Gkl1S+rM"; dkim=pass (1024-bit key) header.d=oracle.onmicrosoft.com header.i=@oracle.onmicrosoft.com header.b="o2Uwfnma" Received: from pps.filterd (m0246630.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 5A8GeCYk007260; Sat, 8 Nov 2025 17:09:07 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=cc :content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s= corp-2025-04-25; bh=VW1rVEDry31QDVbfRseXtsv01MYxQ7aH996ZkSCtb9U=; b= Gkl1S+rMjLcxur1UUr+JX312njS0Lx5AcAJZJUbwL05byE7P21NhhVvJYOoLm/ye lI9GaRSMmZuzTiYya4IIwCM/T+qpkxfEx3QL1Tb7CjBT9ChbA9ENBElEF2LmbxXo RJtTa557NUKB0GREhnyIjx0h/i1QPPmGgjm7uhzXeXrBDseTTfP+htbCJ8oh3pW0 OAYTCK+bu7kmL/AMPakxDJqwKyLkJuzA4JLgNsW1UEeS/pyyfcKezSms/JJx3Uou RcEIpLfhWQX2azggM9LzSnXMDytv7DA6qosWNot6wN1p43pcvivC4D0scCiyBXYt UX0xp6y501k7ilyzCA36eg== Received: from iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta03.appoci.oracle.com [130.35.103.27]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 4aa9k8g0j0-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Sat, 08 Nov 2025 17:09:06 +0000 (GMT) Received: from pps.filterd (iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com (8.18.1.2/8.18.1.2) with ESMTP id 5A8Bgdsr010112; Sat, 8 Nov 2025 17:09:06 GMT Received: from mw6pr02cu001.outbound.protection.outlook.com (mail-westus2azon11012049.outbound.protection.outlook.com [52.101.48.49]) by iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTPS id 4a9vagemb7-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Sat, 08 Nov 2025 17:09:05 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=P/0Uzln7fGqreNchmpnQz+WW/yAS3QFieKuOqhoVjg9tIkJUYJefOIFuQgalrYRfcuc24k3BtlfO0p5z3GhKxjfgneLVRtpoIraou8qwqtaScSlXjvcQOPA27kwH0kWhr6qEuKdnmACMp1y+JiTGsgHnrCVHVGsyqBRRe4PhoWTpglawOwM0YPBgYXSu+8ZOjlcdZ8kjk+82LcRGnh7i5Rn+DA6WiVQYdmG3I8CBqS0d8uDF14sxOoUBNK1pTq2JLOZSndXV2iK/+eD7gHXJlAkTMoxQ1ICRY2kgNetUL/bxc7ngIltQqoa576Hux+69fULNwuW455h6LOJY4Y+WeA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=VW1rVEDry31QDVbfRseXtsv01MYxQ7aH996ZkSCtb9U=; b=xSVcOOMyOQtnqdx1xsserMKXX1wAEZ9lDk0CR8lDp4CQ6hxOzBpfblEcUulMSo8YABGb5LSZyEkhY26URTg3lGjGjZLE9PmtiUauR4KNv/VvKOm4850h2OKIXa6wIegMV2se0ISxvQT3EDZeZyoGAkDkV9VxA8xav+btgGEHhtECJYq4vEBSt1aJvkGzlfsGy02Jbqh50if01AhQisjaWHzHk6Z4wCgoBrW7lecaLWI5TG+mIwI3ekNADBOPsRaNy+WNQgf/UybszpaIIo0/vFIHWjVHAet/2kt6FKexYKeMfuOFe4LrE5KHaKCNXkZVWz9DyCQQX4A4C03AS/E+vg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=VW1rVEDry31QDVbfRseXtsv01MYxQ7aH996ZkSCtb9U=; b=o2Uwfnmaq+L86PjBr9DNH6SkHKYsAyYu6jqD2/UsQzpReAJYcEMzF8/MKWo1++9WT0gn7yFaw36F/+lbz1sxk/3e43vHKG5br7iyMSfJ8vlKIAy/g7Kw9nvPWgRtQtIPS+dKBr83q6T45RCivMGg05pZ+Y5LF5dZt6PHhX5wUmU= Received: from DM4PR10MB8218.namprd10.prod.outlook.com (2603:10b6:8:1cc::16) by SJ5PPF04D2D7FA7.namprd10.prod.outlook.com (2603:10b6:a0f:fc02::785) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9298.15; Sat, 8 Nov 2025 17:09:01 +0000 Received: from DM4PR10MB8218.namprd10.prod.outlook.com ([fe80::2650:55cf:2816:5f2]) by DM4PR10MB8218.namprd10.prod.outlook.com ([fe80::2650:55cf:2816:5f2%7]) with mapi id 15.20.9298.010; Sat, 8 Nov 2025 17:09:01 +0000 From: Lorenzo Stoakes To: Andrew Morton Cc: Christian Borntraeger , Janosch Frank , Claudio Imbrenda , David Hildenbrand , Alexander Gordeev , Gerald Schaefer , Heiko Carstens , Vasily Gorbik , Sven Schnelle , Peter Xu , Alexander Viro , Christian Brauner , Jan Kara , Arnd Bergmann , Zi Yan , Baolin Wang , "Liam R . Howlett" , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Lance Yang , Muchun Song , Oscar Salvador , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Matthew Brost , Joshua Hahn , Rakie Kim , Byungchul Park , Gregory Price , Ying Huang , Alistair Popple , Axel Rasmussen , Yuanchu Xie , Wei Xu , Kemeng Shi , Kairui Song , Nhat Pham , Baoquan He , Chris Li , SeongJae Park , Matthew Wilcox , Jason Gunthorpe , Leon Romanovsky , Xu Xin , Chengming Zhou , Jann Horn , Miaohe Lin , Naoya Horiguchi , Pedro Falcato , Pasha Tatashin , Rik van Riel , Harry Yoo , Hugh Dickins , linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-s390@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, linux-arch@vger.kernel.org, damon@lists.linux.dev Subject: [PATCH v2 04/16] mm: eliminate is_swap_pte() when softleaf_from_pte() suffices Date: Sat, 8 Nov 2025 17:08:18 +0000 Message-ID: <75c2e8fa38de383757a49bcc3f5c081be1e27a40.1762621568.git.lorenzo.stoakes@oracle.com> X-Mailer: git-send-email 2.51.0 In-Reply-To: References: Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: LO2P265CA0022.GBRP265.PROD.OUTLOOK.COM (2603:10a6:600:62::34) To DM4PR10MB8218.namprd10.prod.outlook.com (2603:10b6:8:1cc::16) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DM4PR10MB8218:EE_|SJ5PPF04D2D7FA7:EE_ X-MS-Office365-Filtering-Correlation-Id: 600d7dfe-ebd8-4089-294f-08de1ee9833c X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|1800799024|7416014|376014|366016; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?cfUDLf9y5Cjew+dPRxmaAXl0plPnuv7xBcnyVa/QvI/vX5N87YwFsfayOYgD?= =?us-ascii?Q?GDD6pSRgVOqa+mEJFEsTupBB20OBpP6nG5HhIfIZRzeFNnaq/gaPuakKB5Xd?= =?us-ascii?Q?LOfII7sjb/3VsdqQPg8/LJWKZRCzZv9mlwVZ8/kZCgb65jLVUNRyRmSgBnYV?= =?us-ascii?Q?FYSB1njK/vNgsh0jgAGpy8eeyQ6GGD7crtx80PiOPAkwMcVSzCob2GiXjLa+?= =?us-ascii?Q?pfbYrDzlJ3qZUrK0oKgw4c78NsEckH0xmzcMwlbDUhbvSdZ9NUGrhM1O7d9A?= =?us-ascii?Q?V8xLXzorCrvqkAVAt/cAqc1FOjO3Ya2snIu0GdYNi48N1OE9XXBSr3wRXYuU?= =?us-ascii?Q?+2QLsEPA5HZAqgdA11VO8QVeLTkzlk5OTUde17Jm4xNRJhnqJ/l5VQ9p8LhE?= =?us-ascii?Q?5HrUMyzxVKh294qlfqoQcSfc4mDx4/qafoFwxGhWBo6dn0K65l57Q0wNn6Rq?= =?us-ascii?Q?J+ZriuMThE2i6FtHpAQ4n0FhRO0rNK6zn7Au0TXnAkazpRll7xJwatHZGvgD?= =?us-ascii?Q?dY/7ua3IWlcST5RnDeKxPFGVY5fZRrR+0sHrm0Ua3WFDvkkn58aPsizbIyfl?= =?us-ascii?Q?l+YpGjlaIpwyli/0yg68AYjQnZN3sx44D3aFpf1M8vh6xNv6bFowy5Q6dxwt?= =?us-ascii?Q?qFWnvgODfMqAVmOJDb8RvooEjzSmS+jKOC1i421J+qFGNDZO+jKXl4WRX8lK?= =?us-ascii?Q?clV/SBcIKe+Oevf3w+jlfPG+5C5QV/qYetEsURMkSgz24D+DwyqSzeKSR9BL?= =?us-ascii?Q?lUyckxov4VUzPtQ8vntYE0EnVtvOurprtC5uhsZ+izyFEk1O0OUbXFlNsC0y?= =?us-ascii?Q?IbZpSkAAu+SvjMVSOcEGb2SzwDA6u2CZoDBAxBcs3nGPoSYXnS5OgRavDb8Y?= =?us-ascii?Q?XL7IZmq7lfJoqPT+/KucaWNqQSspH9yt5C+4EIapD0XfbKsyytu5v8Tr0+QJ?= =?us-ascii?Q?o4E8KA67cOWPr316CrolAb5ySZHpaQZ0HK9UxIZxOED+5dagkqpHQbPAz6FU?= =?us-ascii?Q?NfsGLi9VdPp1vsahk1l30/0RPwMvxytpSD3OLacGM8BK78FLF5v+2lMetKVM?= =?us-ascii?Q?2XSP42xcsbx9hkgqLOO5mOEmyrlo43jrVf8k9Zgd5Q1kyGImlnOAVDd8Aoco?= =?us-ascii?Q?q9KA4zlD0MVd2C9v4UTpiQmsxjoV3qfiHqUwN+IM9f6c7wGR3mcyyj0IOooY?= =?us-ascii?Q?4ZqA+4+mgm0GvabDqFO9a1tH5LUFGON903iUYfTGmuLacS7lt1UedYzV5VGG?= =?us-ascii?Q?O6lPbLd2wMwHOoQrlurOSXHWgEaIn7TexFB2ft5cXrePbtcqvegj2ltfDD4k?= =?us-ascii?Q?5fOUoapBfjW2B7C6DlcMn/nDQqDNBt5J24ApGujxgtC6O0DuCLC6VRJaJPuu?= =?us-ascii?Q?da6hCvBRs7hoHhuBnML6Ci7Q9ag9Bo1pXfoG0o6HyNcY3rHZEPONqsPfD6Sw?= =?us-ascii?Q?Rh0IvuERqGQi4fF8EHIgfjcAsgiWTdH0?= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DM4PR10MB8218.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(1800799024)(7416014)(376014)(366016);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?9+ue1x4m+E3eW2aNeuDKprxHLoxqbfJQWm1WrP4/OOGxBuDhB0Nos5AORzlB?= =?us-ascii?Q?Aj+UrEMLaIbVVJfmRGjRyXiOe5gtk+om2kARM9uFL1ld1Y3JF2CpR7uWUlyT?= =?us-ascii?Q?O/TdBvVXI4EKmOKfWAKMA7g6rAUNpBXRyEr/hJ3bsWmaQRgQIANY4mKSH7ob?= =?us-ascii?Q?RGjsuggujbMAlOZp4PytbboypGm5OO81bxZTaYMg6w732oORxtjscedFI5Ld?= =?us-ascii?Q?fPwISeJoLNJDNDIttkkIUNKDgUSOhZKwjGpwKFdzp3iCxUNSdMN8+wjGN5RJ?= =?us-ascii?Q?UvTx+mLMOHRmBOHFKW82N/bxpIPE9IgkdT+c26EIC7KNVTZKoO3msQUr80MA?= =?us-ascii?Q?+alELZbqAU+srs+wRyolcoTCgo0Vhr7V5WmL+WuA6t3EBQQrhYmb8/g67q3N?= =?us-ascii?Q?Mtun/z5qW48NzfNWlZDzetQuTNyJnpKyeqCs4/o4JMhbpCw3/62IVnQULMja?= =?us-ascii?Q?hJ0gjbcCzycLDcl+T4fD10tCL1S5jy7L8ZbKTxo/5PdLwf9hj9YMrpKprs/O?= =?us-ascii?Q?plQkkhm+9V5vL9xE7R0hK8wt2ehII3v/EIu+q72H++dQgpSg55wJi8UAk04L?= =?us-ascii?Q?I126VWOalaRnv6Sp+TKvzmqGUUiQWrMfmROrLWm2yI6S6eUcKEEtqFgww9/b?= =?us-ascii?Q?x8jMmnnxMFihBQZeb/zD0LozwFGJumgL+0nekgBIKghkJ/Pe4Z1JQkdEO7G3?= =?us-ascii?Q?rO1kb91kVklAqtfzblvDUvu1uIN991gxukJGtmZa7JEdyQ7j0roBVqwLJ0b6?= =?us-ascii?Q?vl5/s8fcu4/RAU13upbwBE3q9t7dvBmYH04Nz/ICNxsdYHi0UI4P37pfWpcc?= =?us-ascii?Q?FchVhz+cTMvMurhl5c/lMa2GLHWF1IVOuERX6fR0dts6xsjFMzHSNroL8QGG?= =?us-ascii?Q?tBXkkmYOFyvZ6S7ctJqfLoDmLhN91/VgbZN8JBYDy2w13TKBb9bZI+lcYmf9?= =?us-ascii?Q?gDDEsDJob+R8gdpv6iytI9ImXFMu55xLlT4Vn3fWf51szUcY3hceykyqcg2H?= =?us-ascii?Q?aIb/848U0fAxydatyyvvPxHkaFqg5o+4IgIrGqO6beDV1DYM0MbambffEYnC?= =?us-ascii?Q?b5AQCCbV0aP2G/SDLHYPBCr7SIv2l8HppTfdXz/x0c2QBVqr+YKw52K4pqr3?= =?us-ascii?Q?PeGHnBI9xsUoX1iqqzXPo1vbV9mw8vlndNH6JrLdXHNcQ+K/NtMX3sgO3sm4?= =?us-ascii?Q?7hYPdpYAvY7q0CaLhkRa5MLDPAbSRAtfrTgjTqvaLStM49nAYIW/misolwep?= =?us-ascii?Q?2DWMHj2v+9X6Y4LVrN6oDYMe4LaI6EQXptEhjaqE2T7oGWJmhFGIY6mGc8kG?= =?us-ascii?Q?QJogXrfD4iRVZzWcXJdBxUPsZh0z94teEVxZffm+3+KwHif3ZEt6qS3gKe34?= =?us-ascii?Q?bgGmC06LNYQPxWew8DImS5S42ge9JMibn4IQgYjVM8Hoodttj15xPBNHOCf7?= =?us-ascii?Q?6yjXNbns1zZCuUcwqGLchMRGcMVNaRjjIPzFoLfZLUKDrupF+QVqQwxCaeFE?= =?us-ascii?Q?md1tZTsPAeXhlRR4ZNTvqiW6QkJ83BRWRbeGysWxjxXID0aE9R86tzyY88uH?= =?us-ascii?Q?Q73gUSf/EyHqkqhHX/W+0TFflf6FVbooZvvo2+eURQjz++eeZsznw90HkgaZ?= =?us-ascii?Q?PA=3D=3D?= X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: MZC3Td3qWWcQayQSbWyyJur4x92igO4/1sJtS1de8iSx5jc8pwHId16tGfoPKSYZTSGXkQkSKqPcdGxQcZZLXAF7JzHDVOssV4YLVD4QTi5xrkiInGsFb5Joxa40qjHfSn98D8FW0s+H9A0LYJYzUcCU6o79WcpUjz9K6V948yOZNrmoTBsn0GLArzGS7kAsL03PEs94ua0HdrTsjPqg4pMXxfb2EtRVsvc8NQ4iP+Rc5jpW1SpdOeL87k/N5eEPWmRWt7GdqJYxmhmB6R9XyR6keeDMg7FL166P7DpsFgvJkcGV5yANcy0fP2H3G1IGGMvl0uT1CXPKP1+6ZiHJQ3pXIT33Mn7eCqOwl5cgQyuVk+Fo55rxww/argIzBF62X0y3kWZRaUS517zfQRJnU7JcaQlh/VNPMwfvBsG88NUkZRDgKayaoEyE9ZopYCj5l4NOm7hJ3ZJsr6y/txDrLHK7lpMf6VRsDioBbYksdsZuonwfT5NyV5YilUnJTO8COHXXZOrfcl8w9Tvh+Hqe+LxocxCWoPEkboR3r8RdAeNFSNCTpZKW3H4/oxNQd5SoeugpsGiSIO0pqp3QRnU/x/J3hgTMsJl7vCLdpTdCjFE= X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: 600d7dfe-ebd8-4089-294f-08de1ee9833c X-MS-Exchange-CrossTenant-AuthSource: DM4PR10MB8218.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 Nov 2025 17:09:01.1922 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: old6vNgrcIviA/XaisvC19TSMK17hts8pdMh4FRy4H15BBZ0jcQPwuLwW8S3LG0VzRzmQv3KqTr2g3x6Wts7u2g6n29SJvP5BB/zXFc+AzQ= X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ5PPF04D2D7FA7 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1121,Hydra:6.1.9,FMLib:17.12.100.49 definitions=2025-11-08_04,2025-11-06_01,2025-10-01_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 adultscore=0 suspectscore=0 mlxscore=0 phishscore=0 mlxlogscore=999 spamscore=0 bulkscore=0 malwarescore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2510240000 definitions=main-2511080138 X-Proofpoint-Spam-Details-Enc: AW1haW4tMjUxMTA4MDEzNCBTYWx0ZWRfXzVt2BptLdtCV EzzAexKn+67A4T84mUKODYl60jsFsEv+xxFX+ryASVmd4Ksxra7gpKbsdeDVyxMqy4R4ENXciyv r126040+sOSQ0Z5FKzF33pXmEqolWzZbAeqX5ihuV/8FQMoavYtB1sBUtgYZsKI7yxXJ5ohmW7z /VP0T5wWVQCnt3gGFls72YgDL6TTe39ZHWL+1eLCd8Lcse4EnVLQBytRjXX8+y3WuJ67U2mmPhW CXTfd3nHnY5YZI9B/lSMbggRCgu6X8pCCSasvxBnZxNdziEiwn003+qVbYE8rQCiPYXqP/hzM3L zf0y0VGixBTlghVCorL8BanAzN5LLMmMyjoUGI2OBxey5Ch7zx6HvJi+yBZ8/CUp0ZvSGC4TwsP VDqkqr2WvelzoFTYs7uMNqW+NoG5I3H3gBLi1pbgqyjh0hQ81r8= X-Proofpoint-ORIG-GUID: 8qlM6wBoRWxHUyL4DVzC9P-Ms4t2bohv X-Authority-Analysis: v=2.4 cv=U4ufzOru c=1 sm=1 tr=0 ts=690f7932 b=1 cx=c_pps a=qoll8+KPOyaMroiJ2sR5sw==:117 a=qoll8+KPOyaMroiJ2sR5sw==:17 a=6eWqkTHjU83fiwn7nKZWdM+Sl24=:19 a=z/mQ4Ysz8XfWz/Q5cLBRGdckG28=:19 a=lCpzRmAYbLLaTzLvsPZ7Mbvzbb8=:19 a=xqWC_Br6kY4A:10 a=6UeiqGixMTsA:10 a=GoEa3M9JfhUA:10 a=VkNPw1HP01LnGYTKEx00:22 a=yPCof4ZbAAAA:8 a=x3qbZFVQRPWiRpXx7eIA:9 cc=ntf awl=host:12097 X-Proofpoint-GUID: 8qlM6wBoRWxHUyL4DVzC9P-Ms4t2bohv Content-Type: text/plain; charset="utf-8" In cases where we can simply utilise the fact that softleaf_from_pte() treats present entries as if they were none entries and thus eliminate spurious uses of is_swap_pte(), do so. No functional change intended. Signed-off-by: Lorenzo Stoakes --- mm/internal.h | 7 +++---- mm/madvise.c | 8 +++----- mm/swap_state.c | 12 ++++++------ mm/swapfile.c | 9 ++++----- 4 files changed, 16 insertions(+), 20 deletions(-) diff --git a/mm/internal.h b/mm/internal.h index 9465129367a4..f0c7461bb02c 100644 --- a/mm/internal.h +++ b/mm/internal.h @@ -15,7 +15,7 @@ #include #include #include -#include +#include #include #include =20 @@ -380,13 +380,12 @@ static inline int swap_pte_batch(pte_t *start_ptep, i= nt max_nr, pte_t pte) { pte_t expected_pte =3D pte_next_swp_offset(pte); const pte_t *end_ptep =3D start_ptep + max_nr; - swp_entry_t entry =3D pte_to_swp_entry(pte); + const softleaf_t entry =3D softleaf_from_pte(pte); pte_t *ptep =3D start_ptep + 1; unsigned short cgroup_id; =20 VM_WARN_ON(max_nr < 1); - VM_WARN_ON(!is_swap_pte(pte)); - VM_WARN_ON(non_swap_entry(entry)); + VM_WARN_ON(!softleaf_is_swap(entry)); =20 cgroup_id =3D lookup_swap_cgroup_id(entry); while (ptep < end_ptep) { diff --git a/mm/madvise.c b/mm/madvise.c index 2d5ad3cb37bb..58d82495b6c6 100644 --- a/mm/madvise.c +++ b/mm/madvise.c @@ -195,7 +195,7 @@ static int swapin_walk_pmd_entry(pmd_t *pmd, unsigned l= ong start, =20 for (addr =3D start; addr < end; addr +=3D PAGE_SIZE) { pte_t pte; - swp_entry_t entry; + softleaf_t entry; struct folio *folio; =20 if (!ptep++) { @@ -205,10 +205,8 @@ static int swapin_walk_pmd_entry(pmd_t *pmd, unsigned = long start, } =20 pte =3D ptep_get(ptep); - if (!is_swap_pte(pte)) - continue; - entry =3D pte_to_swp_entry(pte); - if (unlikely(non_swap_entry(entry))) + entry =3D softleaf_from_pte(pte); + if (unlikely(!softleaf_is_swap(entry))) continue; =20 pte_unmap_unlock(ptep, ptl); diff --git a/mm/swap_state.c b/mm/swap_state.c index d20d238109f9..8881a79f200c 100644 --- a/mm/swap_state.c +++ b/mm/swap_state.c @@ -12,7 +12,7 @@ #include #include #include -#include +#include #include #include #include @@ -732,7 +732,6 @@ static struct folio *swap_vma_readahead(swp_entry_t tar= g_entry, gfp_t gfp_mask, pte_t *pte =3D NULL, pentry; int win; unsigned long start, end, addr; - swp_entry_t entry; pgoff_t ilx; bool page_allocated; =20 @@ -744,16 +743,17 @@ static struct folio *swap_vma_readahead(swp_entry_t t= arg_entry, gfp_t gfp_mask, =20 blk_start_plug(&plug); for (addr =3D start; addr < end; ilx++, addr +=3D PAGE_SIZE) { + softleaf_t entry; + if (!pte++) { pte =3D pte_offset_map(vmf->pmd, addr); if (!pte) break; } pentry =3D ptep_get_lockless(pte); - if (!is_swap_pte(pentry)) - continue; - entry =3D pte_to_swp_entry(pentry); - if (unlikely(non_swap_entry(entry))) + entry =3D softleaf_from_pte(pentry); + + if (!softleaf_is_swap(entry)) continue; pte_unmap(pte); pte =3D NULL; diff --git a/mm/swapfile.c b/mm/swapfile.c index 543f303f101d..684f78cd7dd1 100644 --- a/mm/swapfile.c +++ b/mm/swapfile.c @@ -44,7 +44,7 @@ #include =20 #include -#include +#include #include #include "swap_table.h" #include "internal.h" @@ -2256,7 +2256,7 @@ static int unuse_pte_range(struct vm_area_struct *vma= , pmd_t *pmd, struct folio *folio; unsigned long offset; unsigned char swp_count; - swp_entry_t entry; + softleaf_t entry; int ret; pte_t ptent; =20 @@ -2267,11 +2267,10 @@ static int unuse_pte_range(struct vm_area_struct *v= ma, pmd_t *pmd, } =20 ptent =3D ptep_get_lockless(pte); + entry =3D softleaf_from_pte(ptent); =20 - if (!is_swap_pte(ptent)) + if (!softleaf_is_swap(entry)) continue; - - entry =3D pte_to_swp_entry(ptent); if (swp_type(entry) !=3D type) continue; =20 --=20 2.51.0 From nobody Fri Dec 19 21:47:08 2025 Received: from mx0b-00069f02.pphosted.com (mx0b-00069f02.pphosted.com [205.220.177.32]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6E0A924166E; Sat, 8 Nov 2025 17:11:06 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=205.220.177.32 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1762621868; cv=fail; b=qxxNTRZQKMWQn6A45MdEzwHk4YFPDUci631V9QQ0f+dsBdstle1KY1G8UcpKCDv1oCUHzzoG5dW2sNr17afe+2CZdef20iqYsIQWDVqS8+NZtDaldF/4Q5Pkl+z5AQeCPsR0D8zCpJZmrlzWKGLHBl8wL4rLJEuswuI00L/kYUU= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1762621868; c=relaxed/simple; bh=Jc7k15MXVqD+LodAv3sGe3ypLfG9f5hKPjGmdnE6+KA=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: Content-Type:MIME-Version; b=AnlM20vLHYoz5glt4kpCRt1L1zFuXXeVdJxbnLoHzbqsV7zrS7D+1wtMd0ccmPJwW4tWzdy/9c6331hEBSkhFi6CT0vJgAB3YSnmpBS0qqTrDDu+pkEylp5E2g6wjwrLzc2FwEWxb9o6cCVjedACIJfSB9WzC42wLqi8/nwNfH4= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=oracle.com; spf=pass smtp.mailfrom=oracle.com; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b=Aj3h4dct; dkim=pass (1024-bit key) header.d=oracle.onmicrosoft.com header.i=@oracle.onmicrosoft.com header.b=Fr3WKaQz; arc=fail smtp.client-ip=205.220.177.32 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=oracle.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=oracle.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b="Aj3h4dct"; dkim=pass (1024-bit key) header.d=oracle.onmicrosoft.com header.i=@oracle.onmicrosoft.com header.b="Fr3WKaQz" Received: from pps.filterd (m0333520.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 5A8GjT4w017275; Sat, 8 Nov 2025 17:09:09 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=cc :content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s= corp-2025-04-25; bh=8WOrh/xELmUWEKs0VQab4I3hnqf3OQgLiTzmuIkkR3k=; b= Aj3h4dctMZiDfW8t9I6vMNzn2/iMWc3JboxVnv1CtxM87XgWF2GjO8e6Np0jv7lT UIoCgP8iynGStItKbgRgOvQ5ugTIjlGOurK+foxAEmhfJdiQ5L1uMrQHcbsQVCQb GL6SRGzop7FO0KaOsd90AK0JbxlFPjSGyJSWFqjxXLOWvRoDpgWt3/u1qV/N7YF8 fm41wWvQX+7A7IdlwJPpVhCB3BZMzopcUpMIS5/rgh7bhDIGwvzamwtg0h7kIR/l 9pfomoLx/TI0gElLf0R1AgdgP4ThJAB6sesgVnf6L2M/i0LsoPEzwUW7EjpY7VvI dUSWXOoF6dv+k2amsbQVyw== Received: from iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta03.appoci.oracle.com [130.35.103.27]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 4aa8se01uk-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Sat, 08 Nov 2025 17:09:09 +0000 (GMT) Received: from pps.filterd (iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com (8.18.1.2/8.18.1.2) with ESMTP id 5A8GJAIj009996; Sat, 8 Nov 2025 17:09:08 GMT Received: from byapr05cu005.outbound.protection.outlook.com (mail-westusazon11010033.outbound.protection.outlook.com [52.101.85.33]) by iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTPS id 4a9vagembr-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Sat, 08 Nov 2025 17:09:08 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=YvFjvc4lTs+sxVxr3kFu1cCEz0Wm37jbRSzc8a7nqj5wkrhllk+fWqdJFmAFanln5WeSNvYm43ZfvnVoxbg5t13Y4uIAogz1+3F7qqi+KvAOSo/fDZp9Tspb5uP94vKV+dVZoZL1W0BL9ewiQ3tLRM4Inc13pRb1CJnbvUGep9aPxY2IegSzdv8p+qSfxLAv/DGNTTdqqjulROAb4o00806plj6e6X0riZFMApXsyFc10RFuRVbMAJchRRZND4bz+P/Cmpfj+1D+pcXOroLxFV1rm14DhFwWvHfj8n0A2NGK/SBOMuHPFaIkpNlTjbE9xvmLX29dYkWacj7UWMNciw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=8WOrh/xELmUWEKs0VQab4I3hnqf3OQgLiTzmuIkkR3k=; b=e8BP4i3pyKWdPb61uTccDhizT0+e6QfD9v8wm+GHVkASZKfgv3NKPmlNJaHWm3kSaLxBTygBUrspNZjkHUif7w/RZeNZHAC0TKpY6CLQCp1t3Pwjd76AWLx3wHO365QsOMRTN1MI+1LCQqwRZGRKmioGUGtAClsW267WGWXf+P31yy508BKGDR3IduhRF/2oGEI12NcIjHrL1rK43volvZZuKar6QkiqYxely/MJRfRWFwe/KkWk8mOteD+f98jwULk7eApifoWUn+iyCCt68d/J7fdFdvDWNGvCrarLy2+MMudjJjUvgsnYr/KfumWVGmEkKddN6ftQnsoOAuyF4Q== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=8WOrh/xELmUWEKs0VQab4I3hnqf3OQgLiTzmuIkkR3k=; b=Fr3WKaQzg9nb3pOIi89AsFWlhdZ6lhyOvzdktzzkN8USDIO2GQByGJHZiJldP9oiTReRKsVZVMRu6eBOXiWnsAannCO2rxOewNSmq/XP1o5ADGD8Qqhh1vlHTC71qobnBxmKdRhdlVo/rc90CEGWsd98Cctj94myCdxtT0Ol9zU= Received: from DM4PR10MB8218.namprd10.prod.outlook.com (2603:10b6:8:1cc::16) by SJ5PPF04D2D7FA7.namprd10.prod.outlook.com (2603:10b6:a0f:fc02::785) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9298.15; Sat, 8 Nov 2025 17:09:04 +0000 Received: from DM4PR10MB8218.namprd10.prod.outlook.com ([fe80::2650:55cf:2816:5f2]) by DM4PR10MB8218.namprd10.prod.outlook.com ([fe80::2650:55cf:2816:5f2%7]) with mapi id 15.20.9298.010; Sat, 8 Nov 2025 17:09:03 +0000 From: Lorenzo Stoakes To: Andrew Morton Cc: Christian Borntraeger , Janosch Frank , Claudio Imbrenda , David Hildenbrand , Alexander Gordeev , Gerald Schaefer , Heiko Carstens , Vasily Gorbik , Sven Schnelle , Peter Xu , Alexander Viro , Christian Brauner , Jan Kara , Arnd Bergmann , Zi Yan , Baolin Wang , "Liam R . Howlett" , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Lance Yang , Muchun Song , Oscar Salvador , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Matthew Brost , Joshua Hahn , Rakie Kim , Byungchul Park , Gregory Price , Ying Huang , Alistair Popple , Axel Rasmussen , Yuanchu Xie , Wei Xu , Kemeng Shi , Kairui Song , Nhat Pham , Baoquan He , Chris Li , SeongJae Park , Matthew Wilcox , Jason Gunthorpe , Leon Romanovsky , Xu Xin , Chengming Zhou , Jann Horn , Miaohe Lin , Naoya Horiguchi , Pedro Falcato , Pasha Tatashin , Rik van Riel , Harry Yoo , Hugh Dickins , linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-s390@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, linux-arch@vger.kernel.org, damon@lists.linux.dev Subject: [PATCH v2 05/16] mm: use leaf entries in debug pgtable + remove is_swap_pte() Date: Sat, 8 Nov 2025 17:08:19 +0000 Message-ID: X-Mailer: git-send-email 2.51.0 In-Reply-To: References: Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: LO6P265CA0026.GBRP265.PROD.OUTLOOK.COM (2603:10a6:600:2ff::16) To DM4PR10MB8218.namprd10.prod.outlook.com (2603:10b6:8:1cc::16) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DM4PR10MB8218:EE_|SJ5PPF04D2D7FA7:EE_ X-MS-Office365-Filtering-Correlation-Id: 0ca9aacd-8205-456b-4507-08de1ee984db X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|1800799024|7416014|376014|366016; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?cc91QwHrufHWJo1l13EFwbJO1ip+M6a4jEw8ilngtQihk/y9P8tQYkabFifk?= =?us-ascii?Q?1cnZQjDc/gQwz3eha8wnsQ45cIQ8eC4+yy4zl9SM8uLyNsxdSuNPIb7up067?= =?us-ascii?Q?uPQSXQoYOOfWRqBKeeIx+QgiC8tczi//eh02Eool6fZE3Wo3OH1pGfq4HcSb?= =?us-ascii?Q?kLJcqpXtRHYbzu4Z+ijewhKb0GpJYg1RSL7u7q8FSDBb0lc2SVftF4gIslts?= =?us-ascii?Q?tqEnCiohjR1EOFMSbWjiBDU7RYhqdC4bWDc2QpcwDEYb7mf6t7muNC6TYMSi?= =?us-ascii?Q?uspAB4Ykz8gZNI6ZxA3qT2EFyfdGWFux5y/oQc7+eSifBYSk+E6qT1PQ6fls?= =?us-ascii?Q?dwrCC9UW7HMGBtEpt7W70EYGmIleCj76sOpRe6KfjIQI/pfaElytzPs98Rl6?= =?us-ascii?Q?DF4XAFmGSRy8v9Wte0W+ZEpy9LUi44vwCvWmToC/GS/2cPe+bd0iNQcWXuJw?= =?us-ascii?Q?X7W6sHlpqnSeB7qSgoz4blAiNowjqRCD9htV9sqPSsrk9gujLGs1OExNf4OR?= =?us-ascii?Q?gJpr5pF+pu4ZfZ60m/u3cH9LhRVEBwGnw/K2KPUywPOcDgVU5hqJOrhLXo5g?= =?us-ascii?Q?t5QsejxeolFotUairjY3AWitn5R7N+GTjcDpprjMndf8uOqYdinKyCbahjL2?= =?us-ascii?Q?WeetPpBhl6U/jVVIn6c1U9uLF/BeJ12gooqXi9dEOcoRjCBGgJ8G+xv2Dvo+?= =?us-ascii?Q?W2+Bkqwn6ov7QSAoQFpUXobg6yJjLqsyS9pOYDqpWRL2UUaqKIYZKoN1b22h?= =?us-ascii?Q?LwqzXpDhN10E+Z2w0keJ4lnSdX1n00Ev1FOnBIsKCcwuN5u3PFQAXrzn2KFY?= =?us-ascii?Q?XkkprKjLLdh8121yuRidcHhFMO+Dd0x/LOPyXmzfmehJX3lQdyHjywO3z3P0?= =?us-ascii?Q?crwP4ls01pN0foI1xvhUZzPLHbLzMgxmetDR2ZJwi+DvL7aOb2jef34wKRYi?= =?us-ascii?Q?FuZ0P34VWEqcNHLMr7+UqbSD6bOvJA3FkFxBhSs6inNgr+k21BkF+oGCeHVh?= =?us-ascii?Q?yUqo+UsM2pOT7kWO/TIBEFI7MyuNml57z7arQOMtF2aQzhYbXf2dVPjuEF6R?= =?us-ascii?Q?FjLiPcSZzGIt2QOxjRLQrL0Ssu6aA6jZR2/D9qYEdm0U6DWSBdtGv8YQNuJZ?= =?us-ascii?Q?2eJIuic+f4/tj8wkW4K/VKjEr0TeTVYYFppL0WGqEVSWiGaSVjyGi1XxNiWI?= =?us-ascii?Q?hj5ht58IgUl/jR3B1TH2ji0V2QDTtromGesWxYRkBm4qi9GVW8PrpSaBt8x5?= =?us-ascii?Q?u3gHt862fTvfZcWX1qni3lJEaVjMw+YYrjnA5k5/jLK4YcpWdoa071GZEbDC?= =?us-ascii?Q?K94/9joCuXJmvS3b4Y/B/WA8twxYN7wlJvLOyleHIzVQrvngaIX4BXzhBS6S?= =?us-ascii?Q?ozJeKGQTH2gT8gCA7xv2y+ODLEhmZuT8cdrOGbPFNED7XBRo48PRp0fHHWZy?= =?us-ascii?Q?lhlrdmgpwTCambKvzUj/c+WaW2up/prG?= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DM4PR10MB8218.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(1800799024)(7416014)(376014)(366016);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?z3nEX78a01Q9ZW80uv4f134q84dGlMfrHMi2pQqPOGFvHJrTJr5HkhiGKSpj?= =?us-ascii?Q?DAOYdoai2glsxeqHEd0eeP7IJdSxjXLyCt9YvvobKRorISPDYSRWGRSmElCW?= =?us-ascii?Q?jq/bAvBUhf2AMM+JF9RQQBk+0iPlFQg6l9M6Xsivce1S/SeracHKrRf1FMRq?= =?us-ascii?Q?Kcqpjbmfq0CDbfADsHWDu5NZeIQyFt7+lL8WXXjFbf8zcqCp1RdLy3ER7MVa?= =?us-ascii?Q?1C3jwD9caxpxlgCScuKwgLes/KEffDMWGu88Xp1L3jZTpe/7sChVfPNRqkS7?= =?us-ascii?Q?ab+gpbkjVQAAa5R64TUwqKUffbybaAX71UvN3GNNNRB1pK969bl+54yOhPZ6?= =?us-ascii?Q?DGdrdu5t9dV7pbmou6eVBm0D4VD/FCNYnMiMiiLrxakIQGl9HB/cVA7/MAdW?= =?us-ascii?Q?X7E7UVsOUNIH025rJiu5tJhfDNqqNLh5MAkw897oAPBy5mdpNN4pUlZslsUU?= =?us-ascii?Q?oodvb8qhZsQ6FhN1Qc0q1YCbJAv2A48zLbmN7IClxObvCDWvogEsVnk+zxXs?= =?us-ascii?Q?JxcmoebkW8faiV4i+hibnv3mFT9nNWDUQ66Gkfj+O1h467TBGSC58HpuyX12?= =?us-ascii?Q?/UuACBppQWUHWHISx6q8IVe07kywkgfKdCUSX3i/cYuuYKF9FAKaUTZtXrAr?= =?us-ascii?Q?AWgnsjUA2Lw6UZKyOsX/R8C2EFgNgWRVYPmZzSAprBan2w9nkmTmntvTiu4X?= =?us-ascii?Q?LrYFTG6+de1NWXoYivY24fD3bxmGmwGDBhcnC5ufTDxb5jl7lPK/L8kiX9fM?= =?us-ascii?Q?kO9HxSE2RxT7n2onPalZO08F5FGU5bxGw7p4h6Zd6a7NfRFPujUNOiAg9avI?= =?us-ascii?Q?7WSmy3V6g9S4y9nrKrK+fpgI+83HUpWISWDqpAzKPaaYwRc0lJbPjVIE1yfD?= =?us-ascii?Q?e6d+aKNg+lwOsbB/z5YpkDRQs9hT4IwFUWl2cIx1hacAFSXl0sIi7OA7d8vF?= =?us-ascii?Q?U5AwEcEp/xDmmaphwJGaqYznrD6EGCxH9JteHNztgt/1UVlYEFCXJ/WlPvAi?= =?us-ascii?Q?b7Pz1JbnD+PDP3hhgEu/TjQ8+/fIiK+STgKGWR4Uh2vCY7FcsRQTMsi+NKkW?= =?us-ascii?Q?gr6UxlnguTpCIxAvkERBmdex0hr5jtdGbOGSlX+dGWosNVukysf1Pe/J0yep?= =?us-ascii?Q?IV4wR94qIJMDfomlII0p3n1T+Mv1YjJimCS21wj40In0RlaQVmaiy/zPIxai?= =?us-ascii?Q?mX6YHG591pMPYJd9ejGhnZSuK5GMsuh177xiLjD+Afi9V1eeM2rju6y6VUu/?= =?us-ascii?Q?wnPU6EHeKmE5qSKousVj+KGbmKGbWWBAHfYc3fxoiMJYhZ4LFgLH2oD5OVkh?= =?us-ascii?Q?QMDLGnZdQixuvYFhPXCLhD812aKMlrYCLNWOBKWuHXsKlQksy0XVJ+ZrKwYP?= =?us-ascii?Q?MqwgC5TIiypn4RwDfJXCms9XDeRHR2cwcHzITkDrojpKa/1DVJPNulkgp1rl?= =?us-ascii?Q?IzvPd/+89Jd0LGg0MSRBD0f43Ag6sHE4i611pM+oHcrO/aNVXBQ8kNp006CM?= =?us-ascii?Q?OjwY/UdC5arswIS5O19AKtal+X7cDZdNCgb++M6TOp2qMzRAJmqi82/4LmM7?= =?us-ascii?Q?6hOEbV9FrPcIWetttJhhbiB/s6rKcc8TPYmbqYqyH1euY5X7YCWA9azy2IsA?= =?us-ascii?Q?EQ=3D=3D?= X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: qA3+vqIhykcsVZYqNnm3GV5LO+/6aJme/p+72ChkDDgfHxmoCw2zjlpVeyaTRdDUud6SZrXfrP9846CPVj8FltVTAY1so97TqXoNyiHgNuhJbFApga2KMhwHGOeSfnOwiuAkPTvre3pxtWyjJj+Jtlgc2uBznsQ4uuKbwj4xvQn0T2X//nTJflUbcOMGdfk2vOoSKHi3P1lipvu8BPRtX0lL/dgitNsOkrUHVeRyBIwRPcooAGlYIJ3Uqaj0YDfxq3xpKqq09jlwhzmqsaIEXOzZO36VSWWW7hGqEc4lHViw3RPWj2/8Ab6J2ewuKe0Iyo/sfrdvQIVc1jSPacA/E9WIWXiHa9Jy7OHnQpfvw5IzgxBNF8ELcrTntX8cIJ1w+Fxn8mHYO8nV9IGKZZBZrxToTUOfGgqKrh/VRbfs4ZSIouljivoUL0haLk8allSoF98rz/qXAR03mwdQUMr/aZ0zog80nYIsUUZFe155YY36cHAwiPvKXeUPvsNGaP4qgyndiz8YAdgTkBZanKKX5BBepg4VvUcs3JGAQUV0Itaa2uQs9QlL6rfujV/X2vYwVx/vpmGSsAUREn2pkoPAlVVNR5HHZRlzFvqTDj5f8M0= X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: 0ca9aacd-8205-456b-4507-08de1ee984db X-MS-Exchange-CrossTenant-AuthSource: DM4PR10MB8218.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 Nov 2025 17:09:03.9038 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 1JnKtIR8avMaNfqfalDIy3+JRl6KdTE8ADmE4zWsDennJnnIP2o1I+t2K9n7pUxG0SYXfiC/SsDO+pJLe5yyB5eFgccN9QmSAM4f8oiys5c= X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ5PPF04D2D7FA7 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1121,Hydra:6.1.9,FMLib:17.12.100.49 definitions=2025-11-08_04,2025-11-06_01,2025-10-01_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 adultscore=0 suspectscore=0 mlxscore=0 phishscore=0 mlxlogscore=999 spamscore=0 bulkscore=0 malwarescore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2510240000 definitions=main-2511080138 X-Proofpoint-GUID: Ppu5FyRV1xIBMRz-rFST-2xpuFs7Efdi X-Proofpoint-Spam-Details-Enc: AW1haW4tMjUxMTA4MDEyNiBTYWx0ZWRfX5IvDZlE09dY9 r/1xpOGBD+JCQf2U3UJov2DrGAVQrvVcp/SDAOnNKMyAs0OeWZPoVM2xtevtNAt5hBKUz+wSbw7 KXaLcNO6f6yQkAQ9jvQMESG1BT4DC0hwmaiAghfNnt+2i34mROXVd5PP6MQAi/Nz72aVOo97b4K UOeJaR8vlPz5rFZCYSxhSMBGR6AmIJfzxehDjf+Sk7AqYkKBtGM2dEcwZzhpv3Up0EpX4GfSfHz NmU1j3pUuuM2IYT1qWJ6ptILpDFtOjlrml82Ol7Uw6yJkU13JZv27fp2HJwStp+jtPODzWfl1/6 AwoCW0vb/leHlfEADLNEgUKKUgI6B42q7SbpnteuO8i1robhdT16FJJkC0ii2bIMUBYU/lIkq0g YrR2XXsMT8+nH08UoX7kIkjel7IjEmePlEpWiAHVe/ryHbjj6zk= X-Authority-Analysis: v=2.4 cv=XuL3+FF9 c=1 sm=1 tr=0 ts=690f7935 b=1 cx=c_pps a=qoll8+KPOyaMroiJ2sR5sw==:117 a=qoll8+KPOyaMroiJ2sR5sw==:17 a=6eWqkTHjU83fiwn7nKZWdM+Sl24=:19 a=z/mQ4Ysz8XfWz/Q5cLBRGdckG28=:19 a=lCpzRmAYbLLaTzLvsPZ7Mbvzbb8=:19 a=xqWC_Br6kY4A:10 a=6UeiqGixMTsA:10 a=GoEa3M9JfhUA:10 a=VkNPw1HP01LnGYTKEx00:22 a=yPCof4ZbAAAA:8 a=JfhMdzgkenBfAv4L-EYA:9 cc=ntf awl=host:12097 X-Proofpoint-ORIG-GUID: Ppu5FyRV1xIBMRz-rFST-2xpuFs7Efdi Content-Type: text/plain; charset="utf-8" Remove invocations of is_swap_pte() in mm/debug_vm_pgtable.c and use softleaf_from_pte() and softleaf_is_swap() as necessary to replace this usage. We update the test code to use a 'true' swap entry throughout so we are guaranteed this is not a non-swap entry, so all asserts continue to operate correctly. With this change in place, we no longer use is_swap_pte() anywhere, so remove it. Signed-off-by: Lorenzo Stoakes --- include/linux/swapops.h | 6 ------ mm/debug_vm_pgtable.c | 39 ++++++++++++++++++++++++--------------- 2 files changed, 24 insertions(+), 21 deletions(-) diff --git a/include/linux/swapops.h b/include/linux/swapops.h index 0a4b3f51ecf5..a66ac4f2105c 100644 --- a/include/linux/swapops.h +++ b/include/linux/swapops.h @@ -120,12 +120,6 @@ static inline unsigned long swp_offset_pfn(swp_entry_t= entry) return swp_offset(entry) & SWP_PFN_MASK; } =20 -/* check whether a pte points to a swap entry */ -static inline int is_swap_pte(pte_t pte) -{ - return !pte_none(pte) && !pte_present(pte); -} - /* * Convert the arch-dependent pte representation of a swp_entry_t into an * arch-independent swp_entry_t. diff --git a/mm/debug_vm_pgtable.c b/mm/debug_vm_pgtable.c index 055e0e025b42..fff311830959 100644 --- a/mm/debug_vm_pgtable.c +++ b/mm/debug_vm_pgtable.c @@ -25,7 +25,7 @@ #include #include #include -#include +#include #include #include #include @@ -714,14 +714,16 @@ static void __init pte_soft_dirty_tests(struct pgtabl= e_debug_args *args) static void __init pte_swap_soft_dirty_tests(struct pgtable_debug_args *ar= gs) { pte_t pte; + softleaf_t entry; =20 if (!IS_ENABLED(CONFIG_MEM_SOFT_DIRTY)) return; =20 pr_debug("Validating PTE swap soft dirty\n"); pte =3D swp_entry_to_pte(args->swp_entry); - WARN_ON(!is_swap_pte(pte)); + entry =3D softleaf_from_pte(pte); =20 + WARN_ON(!softleaf_is_swap(entry)); WARN_ON(!pte_swp_soft_dirty(pte_swp_mksoft_dirty(pte))); WARN_ON(pte_swp_soft_dirty(pte_swp_clear_soft_dirty(pte))); } @@ -768,40 +770,47 @@ static void __init pmd_swap_soft_dirty_tests(struct p= gtable_debug_args *args) { =20 static void __init pte_swap_exclusive_tests(struct pgtable_debug_args *arg= s) { - swp_entry_t entry, entry2; + swp_entry_t entry; + softleaf_t softleaf; pte_t pte; =20 pr_debug("Validating PTE swap exclusive\n"); entry =3D args->swp_entry; =20 pte =3D swp_entry_to_pte(entry); + softleaf =3D softleaf_from_pte(pte); + WARN_ON(pte_swp_exclusive(pte)); - WARN_ON(!is_swap_pte(pte)); - entry2 =3D pte_to_swp_entry(pte); - WARN_ON(memcmp(&entry, &entry2, sizeof(entry))); + WARN_ON(!softleaf_is_swap(softleaf)); + WARN_ON(memcmp(&entry, &softleaf, sizeof(entry))); =20 pte =3D pte_swp_mkexclusive(pte); + softleaf =3D softleaf_from_pte(pte); + WARN_ON(!pte_swp_exclusive(pte)); - WARN_ON(!is_swap_pte(pte)); + WARN_ON(!softleaf_is_swap(softleaf)); WARN_ON(pte_swp_soft_dirty(pte)); - entry2 =3D pte_to_swp_entry(pte); - WARN_ON(memcmp(&entry, &entry2, sizeof(entry))); + WARN_ON(memcmp(&entry, &softleaf, sizeof(entry))); =20 pte =3D pte_swp_clear_exclusive(pte); + softleaf =3D softleaf_from_pte(pte); + WARN_ON(pte_swp_exclusive(pte)); - WARN_ON(!is_swap_pte(pte)); - entry2 =3D pte_to_swp_entry(pte); - WARN_ON(memcmp(&entry, &entry2, sizeof(entry))); + WARN_ON(!softleaf_is_swap(softleaf)); + WARN_ON(memcmp(&entry, &softleaf, sizeof(entry))); } =20 static void __init pte_swap_tests(struct pgtable_debug_args *args) { swp_entry_t arch_entry; + softleaf_t entry; pte_t pte1, pte2; =20 pr_debug("Validating PTE swap\n"); pte1 =3D swp_entry_to_pte(args->swp_entry); - WARN_ON(!is_swap_pte(pte1)); + entry =3D softleaf_from_pte(pte1); + + WARN_ON(!softleaf_is_swap(entry)); =20 arch_entry =3D __pte_to_swp_entry(pte1); pte2 =3D __swp_entry_to_pte(arch_entry); @@ -1218,8 +1227,8 @@ static int __init init_args(struct pgtable_debug_args= *args) =20 /* See generic_max_swapfile_size(): probe the maximum offset */ max_swap_offset =3D swp_offset(pte_to_swp_entry(swp_entry_to_pte(swp_entr= y(0, ~0UL)))); - /* Create a swp entry with all possible bits set */ - args->swp_entry =3D swp_entry((1 << MAX_SWAPFILES_SHIFT) - 1, max_swap_of= fset); + /* Create a swp entry with all possible bits set while still being swap. = */ + args->swp_entry =3D swp_entry(MAX_SWAPFILES - 1, max_swap_offset); =20 /* * Allocate (huge) pages because some of the tests need to access --=20 2.51.0 From nobody Fri Dec 19 21:47:08 2025 Received: from mx0b-00069f02.pphosted.com (mx0b-00069f02.pphosted.com [205.220.177.32]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A9CC02561D1; Sat, 8 Nov 2025 17:11:12 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=205.220.177.32 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1762621874; cv=fail; b=fZ/qVC+LN+VoHTt7SkgANbEseudsUFU1D/Zrrif5AgkHtJ4wWTaCWunYmiTEMvmHghyWcOZ0nbj6S/A4SeSUobFz5dGwfPWHONy1wSkRwnULDVwK80DouV87dtlkBHov3+p4Chjh+PUvBL596NNibXX2+a3KduaMwv0uuHTxRJw= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1762621874; c=relaxed/simple; bh=oxEdzxGrJWd6pCTi5NwurPb3nyfqg6tRupFAuDaPM60=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: Content-Type:MIME-Version; b=MMVF3LiZnmbhyKb5BkTdMS/KfM9RmkFYFPxRhZSgmutzYp4/Eph+okx49YKnNLr21zaJXiQxAW0mY675QlSYRcsudtFqSxjEc5/86KG69TM8r9Q48dGFJ77iu/gm/BlavQGz9GBn/i0pxqUXuSBRJTbdGY+9TOkT8p6cVm89z9A= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=oracle.com; spf=pass smtp.mailfrom=oracle.com; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b=rjlvVkWG; dkim=pass (1024-bit key) header.d=oracle.onmicrosoft.com header.i=@oracle.onmicrosoft.com header.b=F1ebrr97; arc=fail smtp.client-ip=205.220.177.32 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=oracle.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=oracle.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b="rjlvVkWG"; dkim=pass (1024-bit key) header.d=oracle.onmicrosoft.com header.i=@oracle.onmicrosoft.com header.b="F1ebrr97" Received: from pps.filterd (m0246630.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 5A8GfF19008827; Sat, 8 Nov 2025 17:09:14 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=cc :content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s= corp-2025-04-25; bh=GuY2tVtboAEPardNoS8eL6m9Qb5SaAi7gKx5ePxzR3o=; b= rjlvVkWGFVs/8zpQktrFd8DH7FLbGObHIz9Jl6sOUhkNQ91NxBzxLTmppJdzYpFG jPbphP82UlYwBQelGjlKbDYQQZ9ICMKNssUZdug1j+C4k69rirzwcKilmOJ4wDv+ 49VGm5hvD82DkfrCYxrFg7a4oKfq5kistumI5QIGnEL0ktcOcyBHSComi9Wc5345 gOzEqoCUcKdDuW0EPsCsLLWdfHUxjjO6x3Y1KUxtBp9WBFPjSKHTzWuAieHmD86a QLUwaMzDz6E+1Qu61Xun+6uga8JsGKItu1gfd6bTeDfxY1iM3JjB7YsyK/kpz8C2 jcHBRAXaWGsJlyc+qg5kwQ== Received: from iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta02.appoci.oracle.com [147.154.18.20]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 4aa9k8g0j2-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Sat, 08 Nov 2025 17:09:13 +0000 (GMT) Received: from pps.filterd (iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (8.18.1.2/8.18.1.2) with ESMTP id 5A8H0Wj9000888; Sat, 8 Nov 2025 17:09:12 GMT Received: from cy3pr05cu001.outbound.protection.outlook.com (mail-westcentralusazon11013029.outbound.protection.outlook.com [40.93.201.29]) by iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTPS id 4a9vaaps5h-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Sat, 08 Nov 2025 17:09:12 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=cc4En3PYeMf4flhUPOpoGVabbk1fn1ticyJNTjuYn6nDDdgVvH+LH+mQTyvraidK+7uFTR2Tsv1uPJom6NVQVbZbO01YkJM418S904j8G3AnBpOn0nqvNSkO5hd7QwkpeAAZtCka1faeot6Wa9QiLOfHygZ0B41J9ZbeNtKgnLX54pu/AndDIQC5ihrYVYIZNAQASroBHjqnBjGYabJH8ORaCU2LmGq9b7aCFmhY67OhI707FZj5m375ZzzSFd2UuO71BSDNlRpGpCqz8b9uhk/tFREDDlVODvZe6VzKKIdG+Dj3Xy9W0xTHNymADEiXenQCdqWoVyFJ5bcfRMC5dg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=GuY2tVtboAEPardNoS8eL6m9Qb5SaAi7gKx5ePxzR3o=; b=mqTsv+/DZNgq0wpS6uLvcth44U/ayA9Hg5GSHlbF8pL8mqzkHNmJIfwL+LFxg4ruKDMvrbjhBLfR/g/WL073j68L79B1/wj/cUaEmmO/mIwRcPJbgj0+6cumdTU0XxcLzmgUYnziblKkiF9P1Pvx0GcR1qfVgfxR6pcSA2tOmsAoiVbMDXOo/7u7KJQlEMBwO/U3XGMPwocJWBirG3qBbOS/smHupBJnLGe7o3a6/8cs+C4imWMMEoqFt1P+w4sMD7WA85VqvKYD4NahZWkN4cY4pZcK8VNpnOxgDVF5Kl2sU1zgR/MoD07cdhH33+KvfBRZ4iLoHcnlMJjG6qXQhA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=GuY2tVtboAEPardNoS8eL6m9Qb5SaAi7gKx5ePxzR3o=; b=F1ebrr97bDf/5WUxPXLsFsA8H6Zn7nfjirVIDgdtot2G+z7aghPEQ2XU+TEf+Up2CrVB1pjN9hWNmunWpiinxbdZ2OBJthysXni0JUS0eKJbzDfmbo8sDzbQhuJmOHFE4AHwTJ0SgBXPZRwy7rjAPnWPTBz0phU8DJdjcOq27wU= Received: from DM4PR10MB8218.namprd10.prod.outlook.com (2603:10b6:8:1cc::16) by SJ5PPF04D2D7FA7.namprd10.prod.outlook.com (2603:10b6:a0f:fc02::785) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9298.15; Sat, 8 Nov 2025 17:09:06 +0000 Received: from DM4PR10MB8218.namprd10.prod.outlook.com ([fe80::2650:55cf:2816:5f2]) by DM4PR10MB8218.namprd10.prod.outlook.com ([fe80::2650:55cf:2816:5f2%7]) with mapi id 15.20.9298.010; Sat, 8 Nov 2025 17:09:06 +0000 From: Lorenzo Stoakes To: Andrew Morton Cc: Christian Borntraeger , Janosch Frank , Claudio Imbrenda , David Hildenbrand , Alexander Gordeev , Gerald Schaefer , Heiko Carstens , Vasily Gorbik , Sven Schnelle , Peter Xu , Alexander Viro , Christian Brauner , Jan Kara , Arnd Bergmann , Zi Yan , Baolin Wang , "Liam R . Howlett" , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Lance Yang , Muchun Song , Oscar Salvador , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Matthew Brost , Joshua Hahn , Rakie Kim , Byungchul Park , Gregory Price , Ying Huang , Alistair Popple , Axel Rasmussen , Yuanchu Xie , Wei Xu , Kemeng Shi , Kairui Song , Nhat Pham , Baoquan He , Chris Li , SeongJae Park , Matthew Wilcox , Jason Gunthorpe , Leon Romanovsky , Xu Xin , Chengming Zhou , Jann Horn , Miaohe Lin , Naoya Horiguchi , Pedro Falcato , Pasha Tatashin , Rik van Riel , Harry Yoo , Hugh Dickins , linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-s390@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, linux-arch@vger.kernel.org, damon@lists.linux.dev Subject: [PATCH v2 06/16] fs/proc/task_mmu: refactor pagemap_pmd_range() Date: Sat, 8 Nov 2025 17:08:20 +0000 Message-ID: <377aceac6e129305f4903ec74178cdfeeb22f2d8.1762621568.git.lorenzo.stoakes@oracle.com> X-Mailer: git-send-email 2.51.0 In-Reply-To: References: Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: LO2P265CA0014.GBRP265.PROD.OUTLOOK.COM (2603:10a6:600:62::26) To DM4PR10MB8218.namprd10.prod.outlook.com (2603:10b6:8:1cc::16) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DM4PR10MB8218:EE_|SJ5PPF04D2D7FA7:EE_ X-MS-Office365-Filtering-Correlation-Id: 37dea8b9-cdac-4621-d0d3-08de1ee98656 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|1800799024|7416014|376014|366016; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?5fmjPZJ9Z8JX0aBvrNp8v5oVjiQEUuwE30oVfQw7SsIYT+BgT6xOXBFafU2K?= =?us-ascii?Q?HAm64V3g2rPtrIoBLS7VKrEKs/kuQUgCpGYVg9N1pbbfIOZC3SGdkym4IjTU?= =?us-ascii?Q?VKwC3FSlqTh77XDAhC0i1kb3kZR0+4Reh5fYd9LsgYemHs8rRXm7tcyESPlz?= =?us-ascii?Q?HWgoortXvekOZH2l2a1lyWvqqq9uhFQHaDKBXAhllVuSZ59xTqvz9VCnyxtM?= =?us-ascii?Q?lBBLitN922qzjtYVkEuVxh8AWLWrfj0wLqaNW6V7h/8u1C114v2firOdx3uV?= =?us-ascii?Q?20va0o4hXGEiH7Amf6xnHytbruwDAf49Tr4Q6zWOPDOdAny332vWfPY3Zpv1?= =?us-ascii?Q?NM7rbv+DnIdO5HGJ2VsdRyR/zp5N1WxHPZBn+ZGPta1i3A1RLsFn73ny4/CJ?= =?us-ascii?Q?48eWfw1Ov86XG7j8wxVNS6dFrIbKm1G3BF7Xz2kr4pGlD5JGYCltrI/3EsIa?= =?us-ascii?Q?nZavX4kTUsVkNttvwEhtsohwYZ7UXN9ZQqcCvsls4NikRUiUlF/a0R71LvJR?= =?us-ascii?Q?TLGd0GRwpafio7XRGamMen7bhomfUrkVv/8gqhGnVGZboXNrzG79YgdknNLG?= =?us-ascii?Q?dbD4B9DVOm6PVCrqtQ0gSEkEZvn02dOvn2IoO/oJakzRO98o8bQFrXSybYtf?= =?us-ascii?Q?nEiLVIyPX+zOGWF361FU63UgnHMuqR+u5zgTbJTeHlYFFX2Z4slWYJUEdBwY?= =?us-ascii?Q?Czl59y/XICM7LNAFqt8ZzwpV/QK7AAoHbNzGraKUnnSPxXxWhdyR64t6ZGh4?= =?us-ascii?Q?ozBPMoWy15s2vqNhIyu7VVp1Uw7al7A3xPgJbLMjUzS3wBBNybdpuBCxpQb3?= =?us-ascii?Q?zpnP5V7LAD76yep7M95TlJMvBX/A582XCDQ9cn3yJSluxiu8gEWw1GIri5Ui?= =?us-ascii?Q?mNauP+jCRws/aKFcDRPiIg6GdHG9ATFVip1Ry+EVQ8QMs2woDk2fZFiEBuI7?= =?us-ascii?Q?/U0RcvcTX785YynczOOzppl0dJ4R14yGfqM8o/nDslbpVvpyw82j0tzBl5bz?= =?us-ascii?Q?kbkjvgc4UWd0EAB3G+hFeYbX4oDKIQLHKhAIiKLTH+fea4aW9e2GIgBJIoDg?= =?us-ascii?Q?+c9uD83msGJRNSEiH9R3CwbLa2Wm49NSoeCPXrXuROjQshUsnxemZCp6gMzL?= =?us-ascii?Q?37ks/RkUpUZ5qNHDmIj3EQVs/Iw012WebW0ZdE3Y2iJgc7t/lkN8aUewX9WH?= =?us-ascii?Q?ZE+xlk1SZJzm+uZt+lUFBKLHQBIQ+J6F6wDCC4rGvzdm8eBiM1K55mBH6+Tw?= =?us-ascii?Q?yvqor4WvheoIwt2NbGNXj7yU1ii02Eez8Z9Mx2Q9GWtbrrht5Za3ZQ1A9B1/?= =?us-ascii?Q?rEMiXJ/sSj15ZgJnhh077Vbp/ZemHw7znaBqdo1S3vJ678xPmPr6z5jt2mTT?= =?us-ascii?Q?hnXmFyg4so5pM76K2PtbACCAW1kgKP9MF5sFv101SRrzU5lKuKecBsDp4xF7?= =?us-ascii?Q?qlCbvtPyGv9dZ1f2wFaKkypYTvIbv3ZZ?= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DM4PR10MB8218.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(1800799024)(7416014)(376014)(366016);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?CM2K7Y/+weI7jnZlTK5vRvD55bIJ7WVk4N0Hq//3eOGL/pEEA4f581MWoqQF?= =?us-ascii?Q?C+p8TNCJt+PKxLoFg5Y1vRabjL2K3E7TCqiOGnWiHzMzZqcw+Aj0n40Wo7UN?= =?us-ascii?Q?vZEOXJsHRGYAw0yc+DSxUF1gBz19MCgOqePTenWfRp0B4RcEJ6nJaNx/7uMp?= =?us-ascii?Q?FY+rQ2FpGerJ6kNZmZMlqt9NzjXuSuRIuWFsDJhS3U/mWfelp2RdtB6J/u4y?= =?us-ascii?Q?qcrSZPd7kCMvql6rIEFyVyTH/l8KPDyL/HuRUmiR6sNtUgi2XOWPs8HTVLQO?= =?us-ascii?Q?Uhf7wrMIis0k18VF5iKyI1tnsPMPVUNvx+Tjzq88ejQerLNocYXFaZ85TAhq?= =?us-ascii?Q?RzsTZE4zVY5gIS23jHFjoPv0CvDIoRSIh4oV0Hhm4uL8oBBF6eDu30Fcm9fl?= =?us-ascii?Q?lQH2Vhqj8KczjCk8D6Wr3dPL1MyoOKoGdcghJVV7QJvFDcIRbBpN5wonWo8a?= =?us-ascii?Q?voo/ODzYRV0ICyj73NAzQC7TLu94tp6pLtafMrxYYKn1SXhpt5x6OLMHz17D?= =?us-ascii?Q?fBG1+Qh7e3lglyxn909EvY9SM7Gm8ZmMzl2Do1fPdGW7M+ukGDVr64XlWpCW?= =?us-ascii?Q?vDG+nXRplyGeXoXNAdQC8/onA82ZvaUQRgbb1fvDArGUqS72YgrdsOEfm9Rx?= =?us-ascii?Q?gFIX3ieNlN5UHHzoYa03F9pDbt1DV6JUrBurOBSS+DdLtDXg9d9kz1Gvf/Sm?= =?us-ascii?Q?+FyassHhlaopHjZdOlZwI+vx3xqQKmVUEG8xeQnp6C0QB8Pp0b0T1YZmtb7t?= =?us-ascii?Q?g2tGmCiNJ3ScVYO52xXw/LZzkaNM32IIAA6aghyZVwqXV0gLCehnjnwSPbtx?= =?us-ascii?Q?Y8eK4AKKLhcSb8gd7Pn1IK6kaOVCRtZdlrruF0OSyegJPLyZvaTwSN0HLx7/?= =?us-ascii?Q?zkRfna7JRt3WWK6PvIbJ5B7TuP1PlDuHyNORQmxc0k0ZWFWUoCCL+OQUzNpB?= =?us-ascii?Q?/58ASj+Wh3ImJwuLNPS0TB4uOb8HY1Z5y6nJMAGLiNRdzMEfKVB8XLkmampZ?= =?us-ascii?Q?ZG7mDbVyLViH/I0t5qCQrD8M3BPBxvrPrLRxOe91+y5QEk4aE0nOowVzg7cz?= =?us-ascii?Q?Qa9UB+UFwPLYKHGvR0Jbr3hCLhLTkp7TWjGFT9fZ7oTD+vUCg1tsQ0lWuteV?= =?us-ascii?Q?Hmf8wHQH9q6g6VlpjivnNcSaFRnierHEU5N77tQVpNt1ApaeC6lHp+E4yqg5?= =?us-ascii?Q?Ak51X6M/iHdy3/ydO2+kELE5+Rk285IiWqvTOKrWKF/FXdG1+UAWKSakyCf3?= =?us-ascii?Q?kg14iHsQ2i8c372hvVJL69EbXRaWIWS9cnaG7YuvY8D6DiqTc1/FoRFzg68H?= =?us-ascii?Q?ukqCT0KWcS++CeAyPMaPWGfkRnnPtVViGcFT3DCuLZy9vpSGmksTzg1GtMPP?= =?us-ascii?Q?xfJMwEgZDKP70mBEdXTpfkVXZS3U/KwpPqSjAFU0elVbBC6rYarLn6KVnuI+?= =?us-ascii?Q?sYar8D9hCYqscQfZr+gVP8FhuMc5t92cbspAOi/4DsNhLhMMgnEjSahjpJv4?= =?us-ascii?Q?u+0jwlzIV6lMIRTtidq1HFnHseZV6YiYES8FhwvGR6RitjmHJtQhRV3f3J7N?= =?us-ascii?Q?5+9+j2prpkrW4kSt1Yz5QyGyzwBXwKa6Tw+yvHz8BO6o91HJNcGoP2666/ZB?= =?us-ascii?Q?Ww=3D=3D?= X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: dsxRcSv2sgYs0cCD3ln5GxUuA2iCmYnGYQIoE5wTcArzULv9KN1OPh8ZFfV6sl+9yi6z5/V0WazhtTa66ZFw+tuVjDicgoPWWPGhJNIPCjjyaLR541z7R6MPsNgu7BlAh0+BMZRTXwvS24BO56eHotj2DlrvLjsYWEj3lmA4SdSE7wV8RvSducbLKlkQ5tnwPggBq+2LgGkZ//R9TwL59NsnG8cBXj70skoT0lq63zCefBbGYQRac9cO8cOCgrS9AULOybrJzpw1YQk8hyNmfvgR7AcHjrxd6opyAOvDemhtZ/ZS1lGPBwHgs1jO0iaPzSdDev02vKbek+Pke51MDV7hsRxaWm/AYh5Zp0DwIaPJvF/ZwRrYG3aATDXywrKpf1pJJITao/XO7Zcw5wJvk9MAYPi5vr5onj4zHWxP5PUt/aTCzWPdE7lg7pooQqNGg7TxGna/ypcqiojRSy6D0DGB5T/KvoyWS/oYEpUBRRb5s9lKUxgSA+2JPDwAzx/tniM7jx+sfGbZKpVppWDu+SLe/l22IcISUXV75rt5gc7/1sOho/FJXSbcyi7z3B+OR0GY7AB3x6J7XsS2aHBO1Qk3Xn+xKoPPkBduLNr6sqs= X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: 37dea8b9-cdac-4621-d0d3-08de1ee98656 X-MS-Exchange-CrossTenant-AuthSource: DM4PR10MB8218.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 Nov 2025 17:09:06.3948 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 1aSP9+MEbBcW2JhIMKBCxry1OlJ1JF0TJdG339W5aQ7Wc1zxUsbe4pezfQTa2y8O+xrCiTS+Kq9T3EQrWUmgUrrkgm5dn3SeRchbceZ32tE= X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ5PPF04D2D7FA7 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1121,Hydra:6.1.9,FMLib:17.12.100.49 definitions=2025-11-08_04,2025-11-06_01,2025-10-01_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 phishscore=0 adultscore=0 malwarescore=0 spamscore=0 suspectscore=0 bulkscore=0 mlxscore=0 mlxlogscore=999 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2510240000 definitions=main-2511080138 X-Proofpoint-Spam-Details-Enc: AW1haW4tMjUxMTA4MDEzNCBTYWx0ZWRfXzWAu4vtIcmo4 DNDGSVFug5aPDCUvYj5BIhUW8mUnRS8SxmXYvzh+rksLD1zq6gKA8lS4yOKNNgnBSwyk3hGdTbD weHRrXMW0242VgZYH9xMfjOiCEG6ZIfvl349RzUihGpii/a5q3+/iU5GYJvbPQfAeqWosw/6u5h NQ6Q6lMiPXzPVcB+MnJsK1hp3dSEttO1jT2zY555OzxItjOFRe/qfQn4KvYs3SK1xGQO62F2MyS mxT6qZC3cvsFOKq900xZ9156gv1Yg7r0GQdJojFVAK6Zqf+tdO0RBUhtaC0QU8U7a7NKH7o+0+2 RMMg6MSvHYZsOvlBTVI5b6F3oYSJ34gAc4LqxiMUXv67OiKNQaDxLH3vYaD7h8kyrKXt4Ex8Tgl Mq0QhfJuv+e4Vviha26wspk7ztqZgOsi51ie8n4NGNjSTnpkn9Y= X-Proofpoint-ORIG-GUID: kcEz9JlNB0PjmYUBq7hINNmC-rD41CHZ X-Authority-Analysis: v=2.4 cv=U4ufzOru c=1 sm=1 tr=0 ts=690f7939 b=1 cx=c_pps a=e1sVV491RgrpLwSTMOnk8w==:117 a=e1sVV491RgrpLwSTMOnk8w==:17 a=6eWqkTHjU83fiwn7nKZWdM+Sl24=:19 a=z/mQ4Ysz8XfWz/Q5cLBRGdckG28=:19 a=lCpzRmAYbLLaTzLvsPZ7Mbvzbb8=:19 a=xqWC_Br6kY4A:10 a=6UeiqGixMTsA:10 a=GoEa3M9JfhUA:10 a=VkNPw1HP01LnGYTKEx00:22 a=yPCof4ZbAAAA:8 a=r4PJAH3pRi85-HfG15IA:9 cc=ntf awl=host:13629 X-Proofpoint-GUID: kcEz9JlNB0PjmYUBq7hINNmC-rD41CHZ Content-Type: text/plain; charset="utf-8" Separate out THP logic so we can drop an indentation level and reduce the amount of noise in this function. We add pagemap_pmd_range_thp() for this purpose. While we're here, convert the VM_BUG_ON() to a VM_WARN_ON_ONCE() at the same time. No functional change intended. Signed-off-by: Lorenzo Stoakes --- fs/proc/task_mmu.c | 146 ++++++++++++++++++++++++--------------------- 1 file changed, 77 insertions(+), 69 deletions(-) diff --git a/fs/proc/task_mmu.c b/fs/proc/task_mmu.c index ddbf177ecc45..5ca18bd3b2d0 100644 --- a/fs/proc/task_mmu.c +++ b/fs/proc/task_mmu.c @@ -1984,90 +1984,98 @@ static pagemap_entry_t pte_to_pagemap_entry(struct = pagemapread *pm, return make_pme(frame, flags); } =20 -static int pagemap_pmd_range(pmd_t *pmdp, unsigned long addr, unsigned lon= g end, - struct mm_walk *walk) +#ifdef CONFIG_TRANSPARENT_HUGEPAGE +static int pagemap_pmd_range_thp(pmd_t *pmdp, unsigned long addr, + unsigned long end, struct vm_area_struct *vma, + struct pagemapread *pm) { - struct vm_area_struct *vma =3D walk->vma; - struct pagemapread *pm =3D walk->private; - spinlock_t *ptl; - pte_t *pte, *orig_pte; + unsigned int idx =3D (addr & ~PMD_MASK) >> PAGE_SHIFT; + u64 flags =3D 0, frame =3D 0; + pmd_t pmd =3D *pmdp; + struct page *page =3D NULL; + struct folio *folio =3D NULL; int err =3D 0; -#ifdef CONFIG_TRANSPARENT_HUGEPAGE =20 - ptl =3D pmd_trans_huge_lock(pmdp, vma); - if (ptl) { - unsigned int idx =3D (addr & ~PMD_MASK) >> PAGE_SHIFT; - u64 flags =3D 0, frame =3D 0; - pmd_t pmd =3D *pmdp; - struct page *page =3D NULL; - struct folio *folio =3D NULL; + if (vma->vm_flags & VM_SOFTDIRTY) + flags |=3D PM_SOFT_DIRTY; =20 - if (vma->vm_flags & VM_SOFTDIRTY) - flags |=3D PM_SOFT_DIRTY; + if (pmd_present(pmd)) { + page =3D pmd_page(pmd); =20 - if (pmd_present(pmd)) { - page =3D pmd_page(pmd); + flags |=3D PM_PRESENT; + if (pmd_soft_dirty(pmd)) + flags |=3D PM_SOFT_DIRTY; + if (pmd_uffd_wp(pmd)) + flags |=3D PM_UFFD_WP; + if (pm->show_pfn) + frame =3D pmd_pfn(pmd) + idx; + } else if (thp_migration_supported() && is_swap_pmd(pmd)) { + swp_entry_t entry =3D pmd_to_swp_entry(pmd); + unsigned long offset; =20 - flags |=3D PM_PRESENT; - if (pmd_soft_dirty(pmd)) - flags |=3D PM_SOFT_DIRTY; - if (pmd_uffd_wp(pmd)) - flags |=3D PM_UFFD_WP; - if (pm->show_pfn) - frame =3D pmd_pfn(pmd) + idx; - } -#ifdef CONFIG_ARCH_ENABLE_THP_MIGRATION - else if (is_swap_pmd(pmd)) { - swp_entry_t entry =3D pmd_to_swp_entry(pmd); - unsigned long offset; - - if (pm->show_pfn) { - if (is_pfn_swap_entry(entry)) - offset =3D swp_offset_pfn(entry) + idx; - else - offset =3D swp_offset(entry) + idx; - frame =3D swp_type(entry) | - (offset << MAX_SWAPFILES_SHIFT); - } - flags |=3D PM_SWAP; - if (pmd_swp_soft_dirty(pmd)) - flags |=3D PM_SOFT_DIRTY; - if (pmd_swp_uffd_wp(pmd)) - flags |=3D PM_UFFD_WP; - VM_BUG_ON(!is_pmd_migration_entry(pmd)); - page =3D pfn_swap_entry_to_page(entry); + if (pm->show_pfn) { + if (is_pfn_swap_entry(entry)) + offset =3D swp_offset_pfn(entry) + idx; + else + offset =3D swp_offset(entry) + idx; + frame =3D swp_type(entry) | + (offset << MAX_SWAPFILES_SHIFT); } -#endif + flags |=3D PM_SWAP; + if (pmd_swp_soft_dirty(pmd)) + flags |=3D PM_SOFT_DIRTY; + if (pmd_swp_uffd_wp(pmd)) + flags |=3D PM_UFFD_WP; + VM_WARN_ON_ONCE(!is_pmd_migration_entry(pmd)); + page =3D pfn_swap_entry_to_page(entry); + } =20 - if (page) { - folio =3D page_folio(page); - if (!folio_test_anon(folio)) - flags |=3D PM_FILE; - } + if (page) { + folio =3D page_folio(page); + if (!folio_test_anon(folio)) + flags |=3D PM_FILE; + } =20 - for (; addr !=3D end; addr +=3D PAGE_SIZE, idx++) { - u64 cur_flags =3D flags; - pagemap_entry_t pme; + for (; addr !=3D end; addr +=3D PAGE_SIZE, idx++) { + u64 cur_flags =3D flags; + pagemap_entry_t pme; =20 - if (folio && (flags & PM_PRESENT) && - __folio_page_mapped_exclusively(folio, page)) - cur_flags |=3D PM_MMAP_EXCLUSIVE; + if (folio && (flags & PM_PRESENT) && + __folio_page_mapped_exclusively(folio, page)) + cur_flags |=3D PM_MMAP_EXCLUSIVE; =20 - pme =3D make_pme(frame, cur_flags); - err =3D add_to_pagemap(&pme, pm); - if (err) - break; - if (pm->show_pfn) { - if (flags & PM_PRESENT) - frame++; - else if (flags & PM_SWAP) - frame +=3D (1 << MAX_SWAPFILES_SHIFT); - } + pme =3D make_pme(frame, cur_flags); + err =3D add_to_pagemap(&pme, pm); + if (err) + break; + if (pm->show_pfn) { + if (flags & PM_PRESENT) + frame++; + else if (flags & PM_SWAP) + frame +=3D (1 << MAX_SWAPFILES_SHIFT); } + } + return err; +} +#endif /* CONFIG_TRANSPARENT_HUGEPAGE */ + +static int pagemap_pmd_range(pmd_t *pmdp, unsigned long addr, unsigned lon= g end, + struct mm_walk *walk) +{ + struct vm_area_struct *vma =3D walk->vma; + struct pagemapread *pm =3D walk->private; + spinlock_t *ptl; + pte_t *pte, *orig_pte; + int err =3D 0; + +#ifdef CONFIG_TRANSPARENT_HUGEPAGE + ptl =3D pmd_trans_huge_lock(pmdp, vma); + if (ptl) { + err =3D pagemap_pmd_range_thp(pmdp, addr, end, vma, pm); spin_unlock(ptl); return err; } -#endif /* CONFIG_TRANSPARENT_HUGEPAGE */ +#endif =20 /* * We can assume that @vma always points to a valid one and @end never --=20 2.51.0 From nobody Fri Dec 19 21:47:08 2025 Received: from mx0b-00069f02.pphosted.com (mx0b-00069f02.pphosted.com [205.220.177.32]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 13F2825BEE1; Sat, 8 Nov 2025 17:11:14 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=205.220.177.32 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1762621877; cv=fail; b=bxdMKsl2ISFbo4IThDqPvoboA/etUYcNm96RcBz+5NZu2h03VaZxKPw2Rabkou7gPXT7wr6lvXQAdmhH7BtJUCfFUNSXonLetOH10hKnrz9fD60k7ZzTm37GeDzIKQHjL+8kGOpH+0HSxkF0kgKfCSiF6N0IJ/OTS+tt0/ACxsA= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1762621877; c=relaxed/simple; bh=gJ8zRazqzQ9hDxhtbg0qAlApDfcn449VuQ4f4oPaLcc=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: Content-Type:MIME-Version; b=DPwoBV6xDjfVJw8KCSD4UUhHF+EN4jkCJN9+BgMP7nX6D5Ty7HFsdLA3zsjyXr3hQNl6hrxZTodKfqCNQt9ip73gZkLByAbJQ5l0JHm+x5HOT+A1UM5tlVLH2HNDtNirohl/6jg7kCaRbjtX7MfubJ7yEEg1SZS07w5BOexBwh4= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=oracle.com; spf=pass smtp.mailfrom=oracle.com; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b=GipQuQPs; dkim=pass (1024-bit key) header.d=oracle.onmicrosoft.com header.i=@oracle.onmicrosoft.com header.b=C5AzFoAb; arc=fail smtp.client-ip=205.220.177.32 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=oracle.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=oracle.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b="GipQuQPs"; dkim=pass (1024-bit key) header.d=oracle.onmicrosoft.com header.i=@oracle.onmicrosoft.com header.b="C5AzFoAb" Received: from pps.filterd (m0333520.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 5A8GjUMU017463; Sat, 8 Nov 2025 17:09:18 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=cc :content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s= corp-2025-04-25; bh=0JzEYXLZhk+XPgEC4WpeH2fagEy4AWaKQdESXVSnsms=; b= GipQuQPs0/Pz8a02J5vqHXu1LI7kogsOXPavQheN+GUz9Bc6Z/toVnArxEt84ZNu T6BNv5rV2ANqdplZr1Nm8jNurCSF1ht/zWL1ORSU2ma+xmNeCDx3NHpj/u0S4oOV X9wRoFh6AwjP4Bt0Md7WOVoSaZ5NJOoVY9gRlpkVChs8+MBlemgLtDnomaBKTG6e JV5kdjNZbQA6oEuwlV6sNxN9DNNUWgFB8My1rjJ2AvxrJMZ4DtcVnb3Oi0JziFIs VaMLgv7m16g5CbFaM1/ZG2KP01vYnPiPuNWkjzQxM2YwXrT1evYcjmQ+k1QOfc2T VNKI78GEf9w9tLQY0twXSQ== Received: from iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta02.appoci.oracle.com [147.154.18.20]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 4aa8se01ur-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Sat, 08 Nov 2025 17:09:18 +0000 (GMT) Received: from pps.filterd (iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (8.18.1.2/8.18.1.2) with ESMTP id 5A8GXJIh000746; Sat, 8 Nov 2025 17:09:16 GMT Received: from sn4pr2101cu001.outbound.protection.outlook.com (mail-southcentralusazon11012012.outbound.protection.outlook.com [40.93.195.12]) by iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTPS id 4a9vaaps7e-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Sat, 08 Nov 2025 17:09:16 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=j2VYmo1J/LW4Ad3kl/buFGWJrRh41qEz3f3Zn/s05ibABS2I73Xzy8dGkZ/j3W1ujuZTQ+9MgqFJEn7OClc6iXKd08SCc7KFOB+FzyqDLeEUSUKYwVSl8pzNwiPIInzPrPehfKKAzI3vuFnVw9T24ZH4q8al4uaeWk0BZpzmEa8DzHeGwZmo0R1dSD0ABYH6UObd4rUK8uLfWpUBB/0/+endqke4+YX7h7NSReDsok7amqf9psNqUotXkRlLCiyqn7ZrNad3oGo/acwhKl5LVj3lLEY3SgnqbQwgn0heTlYVHp1CNa2qd5rXNIYkt+Wd7B1iFTV/6vDaIR68XERJCg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=0JzEYXLZhk+XPgEC4WpeH2fagEy4AWaKQdESXVSnsms=; b=fq/lh9m8MAb/PNQWM7kZ+g2nJWYrkPEQAyoyEOokhKTcKWYx8pbC1fFa9VHR9Io2ToVSi+JcjKrZqCy7ATcwNXXwAWSdOZagE7XStlCKAkoTU1AiZRa503SzmG7ykrA8jd5LOxCwrC4N6COJDEaFx4G7vT5qKeyXgOLgqukHUEWhhoTz7WXuJxGLpjh/r2zmsptt1gyQgb0jJc3ffSexedPtSSnDqqaounhsAoIXk9H0WgBgSnc0CW5P03IYRn0CeSeBZgCPVXXBRZsvH/LnQuusLQ1GEoGP1INoRxbfTJT0/rCj23M7v821tWtptFu1Jcgz8eGBv2K6rge2KEAq3w== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=0JzEYXLZhk+XPgEC4WpeH2fagEy4AWaKQdESXVSnsms=; b=C5AzFoAbrIJ8i6ka/Kl4kBWWhWJHgaBrGQz/r2kkeuZmucTjBiW6PCYXzLV0G8aLGnhhAUhWRvxJepKN/QpXJL2IPy4JKSWrVxQjvFiT5kIXO358RrBP8RB64XSI/ToS2W4M9KJgUhsrLX8DpkTVOGcSGGqETwjvU4i3nQQZOvE= Received: from DM4PR10MB8218.namprd10.prod.outlook.com (2603:10b6:8:1cc::16) by SJ5PPF04D2D7FA7.namprd10.prod.outlook.com (2603:10b6:a0f:fc02::785) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9298.15; Sat, 8 Nov 2025 17:09:09 +0000 Received: from DM4PR10MB8218.namprd10.prod.outlook.com ([fe80::2650:55cf:2816:5f2]) by DM4PR10MB8218.namprd10.prod.outlook.com ([fe80::2650:55cf:2816:5f2%7]) with mapi id 15.20.9298.010; Sat, 8 Nov 2025 17:09:09 +0000 From: Lorenzo Stoakes To: Andrew Morton Cc: Christian Borntraeger , Janosch Frank , Claudio Imbrenda , David Hildenbrand , Alexander Gordeev , Gerald Schaefer , Heiko Carstens , Vasily Gorbik , Sven Schnelle , Peter Xu , Alexander Viro , Christian Brauner , Jan Kara , Arnd Bergmann , Zi Yan , Baolin Wang , "Liam R . Howlett" , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Lance Yang , Muchun Song , Oscar Salvador , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Matthew Brost , Joshua Hahn , Rakie Kim , Byungchul Park , Gregory Price , Ying Huang , Alistair Popple , Axel Rasmussen , Yuanchu Xie , Wei Xu , Kemeng Shi , Kairui Song , Nhat Pham , Baoquan He , Chris Li , SeongJae Park , Matthew Wilcox , Jason Gunthorpe , Leon Romanovsky , Xu Xin , Chengming Zhou , Jann Horn , Miaohe Lin , Naoya Horiguchi , Pedro Falcato , Pasha Tatashin , Rik van Riel , Harry Yoo , Hugh Dickins , linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-s390@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, linux-arch@vger.kernel.org, damon@lists.linux.dev Subject: [PATCH v2 07/16] mm: avoid unnecessary use of is_swap_pmd() Date: Sat, 8 Nov 2025 17:08:21 +0000 Message-ID: <113e97a7319cd39c9e59b3baef40dc57ba9c7610.1762621568.git.lorenzo.stoakes@oracle.com> X-Mailer: git-send-email 2.51.0 In-Reply-To: References: Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: LO4P123CA0650.GBRP123.PROD.OUTLOOK.COM (2603:10a6:600:296::20) To DM4PR10MB8218.namprd10.prod.outlook.com (2603:10b6:8:1cc::16) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DM4PR10MB8218:EE_|SJ5PPF04D2D7FA7:EE_ X-MS-Office365-Filtering-Correlation-Id: 5d72be40-c8f3-416d-54f0-08de1ee987e8 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|1800799024|7416014|376014|366016; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?sMc8sWBm37x3rKxooeRXfC+rHXq+TwuoVROWR4bOcCQedAXffVK51l5B0pRb?= =?us-ascii?Q?3uODy0fImeyDh81K/wg9neI0qzvk6x9G2TxtRQhp1sbopsTwie5pscqoVAem?= =?us-ascii?Q?0IWlWKE7a5oowYaYPFazKNSA/agwk74dxJtue4mujaY4xIRSH+nDz1gfOW8D?= =?us-ascii?Q?LLRaDmrLVM2Vx0K8FA4ytqNjMzzSjLfGfmYwW/uPUaQJhXS41WodAo9P7w5k?= =?us-ascii?Q?kTsSmSO2ed9+X7wcOWl7gebYVLNY7sWnHIg+Hvu7huUlrG0rtb13X9QDK3HU?= =?us-ascii?Q?pdYmSrl1o3wTyu0FbGS5wjappjgF71I6uQlFtYw88ZuZgp5+344TbGxTQXba?= =?us-ascii?Q?338yECsHmI0P3AIsJYgLZZ3bMgqBJw3y3hu97FSKm+NBRTzeDTYw0BKvugeQ?= =?us-ascii?Q?A4JQoU75KoztzKwFw4UKJx4iWBzbp3v8l/kViV2Lgzr2y+vcuADvXRdLdd5s?= =?us-ascii?Q?w1d6JCUvj1RN9pMtGdazypFPdErdmKhf2hCNbeVq7fizqbyiO/By09EDdfNn?= =?us-ascii?Q?3rmwrmv6BcVux74HToy+AhMfClLW6JGTABor7Z+RKmQHyKT2vyxiigrJnXG1?= =?us-ascii?Q?fMyyBMesHeJVMvJwcwIpD6QpOyEE3KMv5XSM/IVy1chQe9jDYqIiOA3VWidx?= =?us-ascii?Q?P9GNYpmwbHQNfuJrTHBmCyPMcGMI8m+V8PFr2/sbVkusbx/yjTkWloLHGvZb?= =?us-ascii?Q?LyNNZRYwzeIuGsBMxJ8TKme4zp3bllCLKMQ0zyJ+Nks7KfdO/EAfx5T59ubi?= =?us-ascii?Q?vN1cmC10DSIvX1xpWGiCMQpXZVyo/wexW7ku6SgHvpiV0sberTjezFXwiYqh?= =?us-ascii?Q?2PuSldE5zZgnyWSt0HurSHPq+diJtiWY+MR7E3KCPm5AuPhGxYj4HMSo2NBS?= =?us-ascii?Q?miFmzEOdceJohnbSRQLmBz7ZA1O3myvP2/fqVLD3YMmTgzc+jRgTqYiG/er7?= =?us-ascii?Q?QpgHs/pWgmfHma6h1ppC097RWM7lEgS+By556DHfWvEo5Z4bSRvjAnTFvGWk?= =?us-ascii?Q?EN5cJpqnnQJq7A7NSpjS1RUCWfjZdPPfdLBNcYLmAbC9ke2u3OdHgC43lQsC?= =?us-ascii?Q?wsWOikMcsF8mmu+nvYNPifiMxyvOPB89Yo95OxKiGbbX7lTWLvL2rHTggwH1?= =?us-ascii?Q?7ti8bUrV1soyZ7ZOfmvKyyJUev+f2M6Ci7e+sBPgvnyw5kb+KTGnzVDj3gOo?= =?us-ascii?Q?0pIe+X7INVBOkIsNOqzzzyvyMt8A14HVI9TEM2gFPHOYU8eaWeqK1W1BW/m3?= =?us-ascii?Q?mn89w7EGb3AlTzerAUDsOY1jAQEMRomP3TdtOoNG+VJG0iry4BHdNOrbt0nw?= =?us-ascii?Q?D0n6ma2u7SsVAFLlpDQHq9j+Gp91gJUqcLKMegpuhnGZI6Jm19m8wCT/SpZW?= =?us-ascii?Q?khZHZww9dz1+YAzm8zr+aepnhOswrpsHkF1xRu2DOY2hd9cDhTU9C4eu8aIA?= =?us-ascii?Q?JKjt9JnI5n9dj/sECgk/3QqVjdqOdpxz?= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DM4PR10MB8218.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(1800799024)(7416014)(376014)(366016);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?KNAmA074Uj0Nx9gV/BK25o2boXC4Dd2ALJYAcwHM4w6/gl12B7u3NwKLY8AO?= =?us-ascii?Q?f1nFF8NwOhGG1X8AvaNfg+RLRZk2bK1IwtGXPyOsg3vquZN+Got98Q3wZrE4?= =?us-ascii?Q?PMKUdXpqQAcll9xUDxa2XGmER1Qe71fxC6UUxRVMxfOFJq3DBnnrG/BDC6cW?= =?us-ascii?Q?3WYGpbRLPbT1RAH0UzWsNpsT1QjkyE1fcMhc+aPTPGvktay0Es6mUWYPMe5A?= =?us-ascii?Q?F+gICiH/ZNCh/s4bfPect/DI8qw59W+kHXj7kRhzRKUd2XEXsMkEQgb+jgVh?= =?us-ascii?Q?vkrEaKqpvvfVU6bjszqI/sTercMfkwTP8CLcc9UQ6MLpeHlIzm0PyFZc8W2P?= =?us-ascii?Q?c6i7yq8IdgDMD89N6DEcXHNA5Xp5Vo9nbDgcwfCSKxg7IXlFVO6PgxUGnNv0?= =?us-ascii?Q?/0RyT2HkrMC+YoNl2+8QU3jkc0PEiNeR3hLozIICY0kuvxZ+skelee/cIhvB?= =?us-ascii?Q?VO/3Dxv2gCTSJRN1tqplBT+WNnACdy8m5025VI6+MJQ3uiC9bCQfcdAzPGQ6?= =?us-ascii?Q?X5JUKxVoinUEr4+YiYZA1UiJEJYzMpwCRWPu7T2gvoi5i1eDWqAQ5IJdhnP1?= =?us-ascii?Q?E6Z4Wxm7OrEmb5bxSr191xN1+p3vp1431q0gxGeyrI3tvTHjgkPgyyflhGUt?= =?us-ascii?Q?4W6H1d16o2jYB51OfPchfQKqFvoEAYUWz+DipOpUcHzlydY8E7i6+varM2oI?= =?us-ascii?Q?8NOlfjbwa5RIdJgSf+hxleDTZSp3rZ0bnoQOFx0tgQdyK2tMAB9VQZYsmUvv?= =?us-ascii?Q?9gO9OY3NTuHiLD7wJRblwJs6bW2m5rleslv53nhHyT6IKl7lnXEYlrFU/N4n?= =?us-ascii?Q?KJheCLyk838tYEuLdBFRObdaxQxOaVshRR6yHGN/hybw7C/dkYZK15HEfIy+?= =?us-ascii?Q?OfBHDeraR6c6MQxF8FBh18DMxV2IinWfbPagTZvOvAhq6OWAE6kAPkwl0wzT?= =?us-ascii?Q?FvOOocrf6PnBpo13U8AM+83HMHkHXihiIA0OhlU44TaFLj/oeAviJY+VnfT+?= =?us-ascii?Q?jeRtt2Awwq+am6FTpIuN93LBBC9wkrUMmuEV9p8Js0iwN1MONGUqfSQRQ3E3?= =?us-ascii?Q?+9gp+UiCk0WWMd2PmYfxFFyZnjZ5LZh4QN0ufpO1G+AgLjBlzsZ91MYsuv6/?= =?us-ascii?Q?B8wocShC72+grKPiBrU9ewKmZHQFd6aZzB7jus5yww4KtSfJXLm+hWXMMImw?= =?us-ascii?Q?PxWvEY4PeHVStSebNQppAP4GxCsOPWwVDZKUZQXvXmdMseo+EfY1Ad9jGbG8?= =?us-ascii?Q?KOgylXHQnr7Tt2PNQaQpZQ0PJBKdyECjyqBQ7eUNCB8PGbyVPLqOOkUk0Thm?= =?us-ascii?Q?RX02IQoAdlEu3WW9CJ1o458JBDYby7AwBi1vEaki/IyjrSb8Ht+Xf21z5gWD?= =?us-ascii?Q?mDlTB3wd9EHjSSpom+zrmx2mC8azoRfRw80gz9BGfYEKIQzJPC9ZIiB7n5dk?= =?us-ascii?Q?xxhddkMI2HtD4utARnD/rRm46zVPpDKN3NfQoqdnooOWlU9hpH5UOIPyUToe?= =?us-ascii?Q?3IuwQHc13SQTMXtfdVNMzJ880NF/pyNIcstdCf7DQ1EyV9GzSgkAHxKJ8M88?= =?us-ascii?Q?0pFm8dnkMEeknoNK1GnD85pVFslCKtAjeWomFbXweNX7uCWz4mZ87ZfTgHV1?= =?us-ascii?Q?eg=3D=3D?= X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: 59TjAGfuUfriGb8SupONIb5DvtZw7SDRUN5QKf8fBXqrly5agv+Aat/FowGfuRmVr9Tp8hbmVYpXY2I/TK5eXUG4Y4af9K1OnIP8fBH6i+KXn3LL1FgBWaptlNRvwyXoUrTX47YefKCQGxVkhWZibEjOttcwEkJ6BV8pTQg7I82sd7CdJ7JMMfyZ2i6frEO0zfMqcJXggH5CP8lNyew/zKUp06b4+Nj5VcKfvjaVz7EA0Zeqj1sTlAkK/GkJ0fKsFZyavmIr5zEDe6mrtY+a0l7wLcfmrghJHP1Yx3BG/88VyrdzAX4bYcnFCcM1z6VB8XB7UNrePULUQolNvX9qZmG8fC5nu5P7Q2H3OXF9AVwPCPynihNgpTWcUS+tPCj01T34agOXyvOvmvftfdtXSGTisgd5hOZDzSICVvZhPCNcHkmr/fPNEvxY/XOXuo2PnTMwfvhl6/+jHjmqYztyU5psoo+1NBBsqgRInEzdX/ttyThjYPiikL42LCVSIfsYQ3hO6jDJyxWYzh+zo0iE5K9WKJyVy4d80UIFOa7BdP8FgdMj6mLCiSawWWqBu3mlmu0ZLKoF3DEbMYule91h5x0qF1Z2nyZB2H7Sho17jFo= X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: 5d72be40-c8f3-416d-54f0-08de1ee987e8 X-MS-Exchange-CrossTenant-AuthSource: DM4PR10MB8218.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 Nov 2025 17:09:09.1400 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 1k3PnHtEKj6iaddCy0TbQNz2gC/2+7D2iiwuBBXtND4Zji9JAgpfIhbLe385weWMlAtLTq01MIKqyYysVZUg3Nl+XG4lDxbUL4AWx635f0A= X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ5PPF04D2D7FA7 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1121,Hydra:6.1.9,FMLib:17.12.100.49 definitions=2025-11-08_04,2025-11-06_01,2025-10-01_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 phishscore=0 adultscore=0 malwarescore=0 spamscore=0 suspectscore=0 bulkscore=0 mlxscore=0 mlxlogscore=999 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2510240000 definitions=main-2511080138 X-Proofpoint-GUID: YKM0l74Kqd0y9o4r67rP-U8SzaM6CAcW X-Proofpoint-Spam-Details-Enc: AW1haW4tMjUxMTA4MDEyNiBTYWx0ZWRfX0lsnixYacQh2 izdtSDSjwgiAG5MKrQ04y8mXQLhUQOaXU5vwh+2UaTpKhMRU7m7bDUvird82qlUNmRxAT4d7GOF JzNVFoOxQuiWpfMwSCDbJ0VLU+TzCEZA/HoXl6/Ex63MbloqH+055iyIw2iiP1GW1jy7rUzV74B byNFANIFnDmsR1MBnjoglrLAN2BZsgfTYjWUXchoxRW0E9DdHUcaV0AB6FXbrBcIZu1tsjjADV7 Ik0yDw2XDVlxWY1mYfdpUnf544ahaiSXR5z0+YQjNGwLc/fSw80MEblHTYBjtOn5QLhbLyOlpYC CB3oP7iFB8CmMgFzdAeEgSzxy0b2B5vdR7M3/c/nzqLwXXRDowiRrmobJlW1hnCndHBet5kH4Ey yo3P5kE+OHoAfxHtUfCX+SihyJdmM/INo3dou0o2hzc+1T0p9rc= X-Authority-Analysis: v=2.4 cv=XuL3+FF9 c=1 sm=1 tr=0 ts=690f793e b=1 cx=c_pps a=e1sVV491RgrpLwSTMOnk8w==:117 a=e1sVV491RgrpLwSTMOnk8w==:17 a=6eWqkTHjU83fiwn7nKZWdM+Sl24=:19 a=z/mQ4Ysz8XfWz/Q5cLBRGdckG28=:19 a=lCpzRmAYbLLaTzLvsPZ7Mbvzbb8=:19 a=xqWC_Br6kY4A:10 a=6UeiqGixMTsA:10 a=GoEa3M9JfhUA:10 a=VkNPw1HP01LnGYTKEx00:22 a=yPCof4ZbAAAA:8 a=96_WKPqrBKIKR3zHGkMA:9 cc=ntf awl=host:13629 X-Proofpoint-ORIG-GUID: YKM0l74Kqd0y9o4r67rP-U8SzaM6CAcW Content-Type: text/plain; charset="utf-8" PMD 'non-swap' swap entries are currently used for PMD-level migration entries and device private entries. To add to the confusion in this terminology we use is_swap_pmd() in an inconsistent way similar to how is_swap_pte() was being used - sometimes adopting the convention that pmd_none(), !pmd_present() implies PMD 'swap' entry, sometimes not. This patch handles the low-hanging fruit of cases where we can simply substitute other predicates for is_swap_pmd(). No functional change intended. Signed-off-by: Lorenzo Stoakes --- fs/proc/task_mmu.c | 15 ++++++++++--- include/linux/swapops.h | 16 +++++++++++-- mm/huge_memory.c | 4 +++- mm/memory.c | 50 +++++++++++++++++++++++------------------ mm/page_table_check.c | 12 ++++++---- 5 files changed, 65 insertions(+), 32 deletions(-) diff --git a/fs/proc/task_mmu.c b/fs/proc/task_mmu.c index 5ca18bd3b2d0..b68eabb26f29 100644 --- a/fs/proc/task_mmu.c +++ b/fs/proc/task_mmu.c @@ -1059,10 +1059,12 @@ static void smaps_pmd_entry(pmd_t *pmd, unsigned lo= ng addr, bool present =3D false; struct folio *folio; =20 + if (pmd_none(*pmd)) + return; if (pmd_present(*pmd)) { page =3D vm_normal_page_pmd(vma, addr, *pmd); present =3D true; - } else if (unlikely(thp_migration_supported() && is_swap_pmd(*pmd))) { + } else if (unlikely(thp_migration_supported())) { swp_entry_t entry =3D pmd_to_swp_entry(*pmd); =20 if (is_pfn_swap_entry(entry)) @@ -1999,6 +2001,9 @@ static int pagemap_pmd_range_thp(pmd_t *pmdp, unsigne= d long addr, if (vma->vm_flags & VM_SOFTDIRTY) flags |=3D PM_SOFT_DIRTY; =20 + if (pmd_none(pmd)) + goto populate_pagemap; + if (pmd_present(pmd)) { page =3D pmd_page(pmd); =20 @@ -2009,7 +2014,7 @@ static int pagemap_pmd_range_thp(pmd_t *pmdp, unsigne= d long addr, flags |=3D PM_UFFD_WP; if (pm->show_pfn) frame =3D pmd_pfn(pmd) + idx; - } else if (thp_migration_supported() && is_swap_pmd(pmd)) { + } else if (thp_migration_supported()) { swp_entry_t entry =3D pmd_to_swp_entry(pmd); unsigned long offset; =20 @@ -2036,6 +2041,7 @@ static int pagemap_pmd_range_thp(pmd_t *pmdp, unsigne= d long addr, flags |=3D PM_FILE; } =20 +populate_pagemap: for (; addr !=3D end; addr +=3D PAGE_SIZE, idx++) { u64 cur_flags =3D flags; pagemap_entry_t pme; @@ -2398,6 +2404,9 @@ static unsigned long pagemap_thp_category(struct page= map_scan_private *p, { unsigned long categories =3D PAGE_IS_HUGE; =20 + if (pmd_none(pmd)) + return categories; + if (pmd_present(pmd)) { struct page *page; =20 @@ -2415,7 +2424,7 @@ static unsigned long pagemap_thp_category(struct page= map_scan_private *p, categories |=3D PAGE_IS_PFNZERO; if (pmd_soft_dirty(pmd)) categories |=3D PAGE_IS_SOFT_DIRTY; - } else if (is_swap_pmd(pmd)) { + } else { swp_entry_t swp; =20 categories |=3D PAGE_IS_SWAPPED; diff --git a/include/linux/swapops.h b/include/linux/swapops.h index a66ac4f2105c..3e8dd6ea94dd 100644 --- a/include/linux/swapops.h +++ b/include/linux/swapops.h @@ -509,7 +509,13 @@ static inline pmd_t swp_entry_to_pmd(swp_entry_t entry) =20 static inline int is_pmd_migration_entry(pmd_t pmd) { - return is_swap_pmd(pmd) && is_migration_entry(pmd_to_swp_entry(pmd)); + swp_entry_t entry; + + if (pmd_present(pmd)) + return 0; + + entry =3D pmd_to_swp_entry(pmd); + return is_migration_entry(entry); } #else /* CONFIG_ARCH_ENABLE_THP_MIGRATION */ static inline int set_pmd_migration_entry(struct page_vma_mapped_walk *pvm= w, @@ -557,7 +563,13 @@ static inline int is_pmd_migration_entry(pmd_t pmd) */ static inline int is_pmd_device_private_entry(pmd_t pmd) { - return is_swap_pmd(pmd) && is_device_private_entry(pmd_to_swp_entry(pmd)); + swp_entry_t entry; + + if (pmd_present(pmd)) + return 0; + + entry =3D pmd_to_swp_entry(pmd); + return is_device_private_entry(entry); } =20 #else /* CONFIG_ZONE_DEVICE && CONFIG_ARCH_ENABLE_THP_MIGRATION */ diff --git a/mm/huge_memory.c b/mm/huge_memory.c index f6c353a8d7bd..2e5196a68f14 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -2429,9 +2429,11 @@ static pmd_t move_soft_dirty_pmd(pmd_t pmd) =20 static pmd_t clear_uffd_wp_pmd(pmd_t pmd) { + if (pmd_none(pmd)) + return pmd; if (pmd_present(pmd)) pmd =3D pmd_clear_uffd_wp(pmd); - else if (is_swap_pmd(pmd)) + else pmd =3D pmd_swp_clear_uffd_wp(pmd); =20 return pmd; diff --git a/mm/memory.c b/mm/memory.c index 7493ed084b99..fea079e5fb90 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -1376,6 +1376,7 @@ copy_pmd_range(struct vm_area_struct *dst_vma, struct= vm_area_struct *src_vma, next =3D pmd_addr_end(addr, end); if (is_swap_pmd(*src_pmd) || pmd_trans_huge(*src_pmd)) { int err; + VM_BUG_ON_VMA(next-addr !=3D HPAGE_PMD_SIZE, src_vma); err =3D copy_huge_pmd(dst_mm, src_mm, dst_pmd, src_pmd, addr, dst_vma, src_vma); @@ -6350,35 +6351,40 @@ static vm_fault_t __handle_mm_fault(struct vm_area_= struct *vma, if (pmd_none(*vmf.pmd) && thp_vma_allowable_order(vma, vm_flags, TVA_PAGEFAULT, PMD_ORDER)) { ret =3D create_huge_pmd(&vmf); - if (!(ret & VM_FAULT_FALLBACK)) + if (ret & VM_FAULT_FALLBACK) + goto fallback; + else return ret; - } else { - vmf.orig_pmd =3D pmdp_get_lockless(vmf.pmd); + } =20 - if (unlikely(is_swap_pmd(vmf.orig_pmd))) { - if (is_pmd_device_private_entry(vmf.orig_pmd)) - return do_huge_pmd_device_private(&vmf); + vmf.orig_pmd =3D pmdp_get_lockless(vmf.pmd); + if (pmd_none(vmf.orig_pmd)) + goto fallback; =20 - if (is_pmd_migration_entry(vmf.orig_pmd)) - pmd_migration_entry_wait(mm, vmf.pmd); - return 0; - } - if (pmd_trans_huge(vmf.orig_pmd)) { - if (pmd_protnone(vmf.orig_pmd) && vma_is_accessible(vma)) - return do_huge_pmd_numa_page(&vmf); + if (unlikely(!pmd_present(vmf.orig_pmd))) { + if (is_pmd_device_private_entry(vmf.orig_pmd)) + return do_huge_pmd_device_private(&vmf); =20 - if ((flags & (FAULT_FLAG_WRITE|FAULT_FLAG_UNSHARE)) && - !pmd_write(vmf.orig_pmd)) { - ret =3D wp_huge_pmd(&vmf); - if (!(ret & VM_FAULT_FALLBACK)) - return ret; - } else { - huge_pmd_set_accessed(&vmf); - return 0; - } + if (is_pmd_migration_entry(vmf.orig_pmd)) + pmd_migration_entry_wait(mm, vmf.pmd); + return 0; + } + if (pmd_trans_huge(vmf.orig_pmd)) { + if (pmd_protnone(vmf.orig_pmd) && vma_is_accessible(vma)) + return do_huge_pmd_numa_page(&vmf); + + if ((flags & (FAULT_FLAG_WRITE|FAULT_FLAG_UNSHARE)) && + !pmd_write(vmf.orig_pmd)) { + ret =3D wp_huge_pmd(&vmf); + if (!(ret & VM_FAULT_FALLBACK)) + return ret; + } else { + huge_pmd_set_accessed(&vmf); + return 0; } } =20 +fallback: return handle_pte_fault(&vmf); } =20 diff --git a/mm/page_table_check.c b/mm/page_table_check.c index 43f75d2f7c36..f5f25e120f69 100644 --- a/mm/page_table_check.c +++ b/mm/page_table_check.c @@ -215,10 +215,14 @@ EXPORT_SYMBOL(__page_table_check_ptes_set); =20 static inline void page_table_check_pmd_flags(pmd_t pmd) { - if (pmd_present(pmd) && pmd_uffd_wp(pmd)) - WARN_ON_ONCE(pmd_write(pmd)); - else if (is_swap_pmd(pmd) && pmd_swp_uffd_wp(pmd)) - WARN_ON_ONCE(swap_cached_writable(pmd_to_swp_entry(pmd))); + if (pmd_present(pmd)) { + if (pmd_uffd_wp(pmd)) + WARN_ON_ONCE(pmd_write(pmd)); + } else if (pmd_swp_uffd_wp(pmd)) { + swp_entry_t entry =3D pmd_to_swp_entry(pmd); + + WARN_ON_ONCE(swap_cached_writable(entry)); + } } =20 void __page_table_check_pmds_set(struct mm_struct *mm, pmd_t *pmdp, pmd_t = pmd, --=20 2.51.0 From nobody Fri Dec 19 21:47:08 2025 Received: from mx0a-00069f02.pphosted.com (mx0a-00069f02.pphosted.com [205.220.165.32]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id AEF4A26CE20; Sat, 8 Nov 2025 17:11:19 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=205.220.165.32 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1762621881; cv=fail; b=hX4UljMVlxdfK2KLlvmCr/5GtzjDFeRbGamZ683ZAgtG7HAmiEYObdjL+wBnriN6OtM7qM21DlgJOmoTGSQRl7RkAkwg1vBwsG2BgWgpRtyX9zh/1ZPcuB+tk4FCTNsSrAKwSlsY8r9494sS/Qu9v6/X6oS9Q/cZHY7AsqPGXBQ= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1762621881; c=relaxed/simple; bh=AZ3s0B+uhePQZXTS6i9X9ezX6E6ciUS6QFxCmlxX9io=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: Content-Type:MIME-Version; b=QYbuqewxexGv22LTa59oqmjlBhu2ukEEIBLoTHD/NSGCIfVQ/VM4cZMFEpj54V93C8hlBNfmmHPcNFffqZYFCvNuPzSHij90k/abeR2xSDsvRvtlvYZRgnoYC3qDMS36gn7ZH1TZAXJkQfHWOp57lzfFQHx9T50krUrYuGi/HfA= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=oracle.com; spf=pass smtp.mailfrom=oracle.com; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b=C1ATOrJo; dkim=pass (1024-bit key) header.d=oracle.onmicrosoft.com header.i=@oracle.onmicrosoft.com header.b=NO6Jp8C3; arc=fail smtp.client-ip=205.220.165.32 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=oracle.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=oracle.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b="C1ATOrJo"; dkim=pass (1024-bit key) header.d=oracle.onmicrosoft.com header.i=@oracle.onmicrosoft.com header.b="NO6Jp8C3" Received: from pps.filterd (m0333521.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 5A8Fg8vt024800; Sat, 8 Nov 2025 17:09:19 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=cc :content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s= corp-2025-04-25; bh=vzCGIt3sJ5ONOndZRuqYi7OVSaXyL8N0gC2xsfawbow=; b= C1ATOrJoQxkDXt4BLeVx0roHuBdz4EUmeUgSHrxZmekAmpD12zFui+3tcPk65PJA l20ljK1RMx+0NVsVXGy7QHn21U/2zFc6pXxoFDlcLCQ4LPM3JQhdPSiyYc93a9y6 MRh16ImaCQU1807adWmI2xJmz6USOcZm6E3+ECOGnEKDkxwO4WCABJm5iQS0Czfb VZQMhR5RRI3OfoDOdFyGEqhlBWAhvf27o4j8Ng2f7UTyxyeGdDrfP54W6TebeJDH 7SXHCRt3/J+WajWQXeeO55/ijRDh/241lRQNSI7FZRogGaV3C4G1YmqTfhknDEHI zqzmiSgD8oTj4XGS1sIVMg== Received: from iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta02.appoci.oracle.com [147.154.18.20]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 4a9vs8gnmj-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Sat, 08 Nov 2025 17:09:19 +0000 (GMT) Received: from pps.filterd (iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (8.18.1.2/8.18.1.2) with ESMTP id 5A8GXJIj000746; Sat, 8 Nov 2025 17:09:17 GMT Received: from sn4pr2101cu001.outbound.protection.outlook.com (mail-southcentralusazon11012012.outbound.protection.outlook.com [40.93.195.12]) by iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTPS id 4a9vaaps7e-2 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Sat, 08 Nov 2025 17:09:17 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=FE9KUIKWtTY6UjhavFiUJ93mTn+HtD+GVixm3Ym9HpsOEb7BivfLQU/LEF2y62cR0GfGCZ5/xr6L3ga+4OKqEazZJcATr5wv2QmsqnY/URalhv4wkW1OmpMpubFPtoSsEpLWZFg/HgmFguevmAB7ZiPQLrYvFfXdLviH/dG0tHH56CQG6r34eVCqfRcrnodlicTeLXSf6/IvSLwc0kAockNAII1SBKr6hRy40kgZn3HWXIGmdtjcurdB+VwVbea4BuA9OEvlnjYos4/kQwKtfSIuhQ8Htu2tkWi4ZM3Hx9Rofwq8zu2mBKc3lliM8OaALed52mYkKuw2yySsTnFPkg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=vzCGIt3sJ5ONOndZRuqYi7OVSaXyL8N0gC2xsfawbow=; b=kRX/oblssQXALZb8Dfvj5VGSzMnaJsWcLxdKPhMQWErnvU7PEQ4lFwmbJSAUsgfTTnnAHCqSqvShg0XZKRIslNFpf+DINMeU6y0RaUbOMHoiSFR3DIhlXQKVOdCoR5ZkypFMn5neOHNFTI0V3JCT38EKhdQjvYHo4Ec4FQ+YPr/rDIoGu3Hungla5i3PHy/oDPPrdFNpgii8bbdbIB7KAuFPFS8dPLOIXzntsIamtMRjjLq4a4kaj61cfLQtgEZux3+Ga/iZ08BnoGrvWswQNLGUYRtUYLCLVO7RNXoAvROLaVCfA8fHVotxFf8RQXu1/eI+UbBsPn88mjf2YP1gtw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=vzCGIt3sJ5ONOndZRuqYi7OVSaXyL8N0gC2xsfawbow=; b=NO6Jp8C3eqRoWeZELx1VBlmNUZv4mAnPsR5zQuaVFopxY7R2Hu5bOdSVv5y6avjK13yWPL9cSL4BH0heLE8kWS/QZM+QvL59pSFJlzte1BoZYlDziXnRMmEXxPxqp0GlRj7WC9J5YzZFWP5DUpQ+TfPqnN/r/T3V7uSzOEpeqYE= Received: from DM4PR10MB8218.namprd10.prod.outlook.com (2603:10b6:8:1cc::16) by SJ5PPF04D2D7FA7.namprd10.prod.outlook.com (2603:10b6:a0f:fc02::785) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9298.15; Sat, 8 Nov 2025 17:09:12 +0000 Received: from DM4PR10MB8218.namprd10.prod.outlook.com ([fe80::2650:55cf:2816:5f2]) by DM4PR10MB8218.namprd10.prod.outlook.com ([fe80::2650:55cf:2816:5f2%7]) with mapi id 15.20.9298.010; Sat, 8 Nov 2025 17:09:12 +0000 From: Lorenzo Stoakes To: Andrew Morton Cc: Christian Borntraeger , Janosch Frank , Claudio Imbrenda , David Hildenbrand , Alexander Gordeev , Gerald Schaefer , Heiko Carstens , Vasily Gorbik , Sven Schnelle , Peter Xu , Alexander Viro , Christian Brauner , Jan Kara , Arnd Bergmann , Zi Yan , Baolin Wang , "Liam R . Howlett" , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Lance Yang , Muchun Song , Oscar Salvador , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Matthew Brost , Joshua Hahn , Rakie Kim , Byungchul Park , Gregory Price , Ying Huang , Alistair Popple , Axel Rasmussen , Yuanchu Xie , Wei Xu , Kemeng Shi , Kairui Song , Nhat Pham , Baoquan He , Chris Li , SeongJae Park , Matthew Wilcox , Jason Gunthorpe , Leon Romanovsky , Xu Xin , Chengming Zhou , Jann Horn , Miaohe Lin , Naoya Horiguchi , Pedro Falcato , Pasha Tatashin , Rik van Riel , Harry Yoo , Hugh Dickins , linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-s390@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, linux-arch@vger.kernel.org, damon@lists.linux.dev Subject: [PATCH v2 08/16] mm/huge_memory: refactor copy_huge_pmd() non-present logic Date: Sat, 8 Nov 2025 17:08:22 +0000 Message-ID: <29f5631ec08be166eeb857d01602d925c4bd410f.1762621568.git.lorenzo.stoakes@oracle.com> X-Mailer: git-send-email 2.51.0 In-Reply-To: References: Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: LO4P123CA0027.GBRP123.PROD.OUTLOOK.COM (2603:10a6:600:151::14) To DM4PR10MB8218.namprd10.prod.outlook.com (2603:10b6:8:1cc::16) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DM4PR10MB8218:EE_|SJ5PPF04D2D7FA7:EE_ X-MS-Office365-Filtering-Correlation-Id: 7868c322-5723-4939-46ef-08de1ee989d8 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|1800799024|7416014|376014|366016; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?11/Wuc2ahGk/v4UJmqMEkoqtUgFgMEr+Wd83PovPZwyWyhsQ5884YDQUtLOa?= =?us-ascii?Q?5Mn3MkUAGnAkED8UXWP/t7NT4qvmH7jyY/h61u85q7bqAL3yI2kxsGHEt2ew?= =?us-ascii?Q?NC3SZv+Zlo8HBMnCssEgkdURc5kzJAY4VcyuIHzJvt4bJW1zyy6I2mSqAFrp?= =?us-ascii?Q?osoITLaQSHEEYft2f0fSuNtcfnmyn6Jj9L+u+r8YPtCUltKk9CTsVnGhm0sy?= =?us-ascii?Q?Ds+7V0Lhojpa7DGutNEuP8RpGn8FgLMKiYU6Nqu+Jj3vBQ4RJEEKWh8mRMZF?= =?us-ascii?Q?pBlzUDTxV9BLwWnclxw7YN2H0dszmI0b0KcvWQ3f3Uh/6IVphCAzu3YDXV8y?= =?us-ascii?Q?xXIV9k8Gx2NM/5G8s5SBHBXY+9xRQsIUxCKbu5+FRKeEDqEvL+LKwoB9z12/?= =?us-ascii?Q?L64k9IkwMSskupjIqgZHn1teoqXUhJ25xfXJ6YRY+9l4g2NcqAw8EsDY+wNj?= =?us-ascii?Q?ArCBmz9n8UjsA41JTfI7B59OMH7Nxx9JJ8rDkgpCp1p+DW7n57M956bk2JGo?= =?us-ascii?Q?gY/MO4jwcPJv8aUyOG7lKXFyqwpKQHZ1jz6cSvLtHJpnLPCUKQ/ItHLB/wkR?= =?us-ascii?Q?0ElC16kc2Z2Zopi3rTvwVyrOccUdZiitVV23G3fRpcHUH0W255mgjGxR5lei?= =?us-ascii?Q?WP2Rw7GFDRAXt+Z9Op2h0BCS10wRb3G027lOMUZ4e9UVxJqREdDZZoc4EENc?= =?us-ascii?Q?rjcgeWsk7WqNh+NR3iv6w0mLMO3wfTFYjm+Sa6wR1WyjUNW4L3q7CV41KulM?= =?us-ascii?Q?g+HggpoDKv9VbHDhtQQac2fs9oyTAVFLo1oUJpqDUW14NzAn5LCVVw0p91Lu?= =?us-ascii?Q?BmPQRGtlyDPKOzwDS6nVY0Hn6Fw45X4I1SD4BbiGXTIClybuyAB+9mhAfle8?= =?us-ascii?Q?2CdeubUKMIczbWX1XnHI2rvayoQU8wKcXI1Ajx5bt8zhSgKZFme4HnE/FjKX?= =?us-ascii?Q?KtTXh0xJv8sAmYAjizofapBvwU4ptaKXD5TDWQyo+KFTnEej2fSiyVZ7DjgP?= =?us-ascii?Q?u55Uj9N8k+XmybAzpoaMZ4CTSYYQL3zjy/7CksThgq6rGjugjwFwLzbBh3OW?= =?us-ascii?Q?CXCE+f6ONhajnIkgXdrU0jQ7DxVcYt6fZgUJSczRevNvh4poyxbdKptOgpTI?= =?us-ascii?Q?A6rER9d/k2AODB/qFusH0nMy2YAgBM7Mi5vI5OaMgU5TRTOwiqM2qHLLv4g7?= =?us-ascii?Q?9gRha7MGr7O04UN1TtZtVn3TnaWhbAmpCwyoyCMVV4Kv71Leo9fvSvjl3lGA?= =?us-ascii?Q?uxDwoXCKIFaGtQKhMCuCmNPcpwBiMFQo9gozd9pJ77Vw6bmlDBgwvFSCQe1T?= =?us-ascii?Q?Jg4TN27a3djANQmPQpqA+PtJPpaVrS7qT+ihMfEUwk2v70Sxk1z29Ybzg6kr?= =?us-ascii?Q?0PikMCtZbjgsJGab8dJgmcfaJITmzrvlHpsHQyNSTmW+pq66erIwuHQeSryW?= =?us-ascii?Q?v4vcU63PxJ7HIztvQgQY/kCC0M+p/Pu2?= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DM4PR10MB8218.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(1800799024)(7416014)(376014)(366016);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?soxCWSCM6lEsC1eV5QlNbvCRq5uM/zHjBguidm+jxW9FFJo7eKIiawIeCVFw?= =?us-ascii?Q?1U2khSpUnQsLR9bRE6NGTWW/xYAUV+Fo7uZ3UvKj7N4JSOeti0SFgcW+jC3V?= =?us-ascii?Q?+bEkqvfNrPozxud5htrN+X15ZUBMAQbWlX74I2EFLr2e9IABwUW5txzN1ixx?= =?us-ascii?Q?DMJD0dUJD/IVGm48mZ1L8Elza+BhfYtjGQuvo5kmq8C1H/iXgfCIviGU2ZBE?= =?us-ascii?Q?uL2p2vZB3clj3o7jYmw1IEj+Yx1205Iwg3drtpr2a0+MiPeXgNdh6GjHdEKB?= =?us-ascii?Q?JCn/lKWNDRtfNw/OCXTsUruO673PP9vMQLoxkDoZGihHfdTllxaRTqC+hinX?= =?us-ascii?Q?zEx6svCDgnUbAD3ogcX1phmcUo5MzIBIvPYtAgnKrmz4HIRqClcPrOrQM1jq?= =?us-ascii?Q?+Iet4huwosdmn7arDGjWAyBzmkOdUHVHSrLSfHMI6n2ZX/86tf2oH9JSWsux?= =?us-ascii?Q?HZt3poJxzIFAjfNzW2ZzMJLKATXim0sVrqjl4101NqodCsUqu0z+4QlG1dsB?= =?us-ascii?Q?RLcjwQiVzI/UgmZ2MHs1j0IKQse3tW4glWQCQKLxz4VjIXWID46bSFE6zvPE?= =?us-ascii?Q?3sTWK8JfARp25Oi9UWrVqyLW4RUcNbt+3iesmnqqE7XAncLKsgNQfMI1TcX4?= =?us-ascii?Q?kvceAPu9PkpISa80n5qTAnQCpZdcGdZUXZHgz+eXZRwOBCxs+WKSkN5FcqIK?= =?us-ascii?Q?eHg3Mk5I4x4l0P95ALK19Se7xKCI1/fa6FDdasNMqXwmUZ7e1l5oMJIHPUMi?= =?us-ascii?Q?OXR+iziJ3Yp4iN9tXd53XCYsppe4Asth12WqwFZPCfVNCAWEDn5HRfy4KKwt?= =?us-ascii?Q?Q0mnugcl7DtcSUcDsWNshFYHobElHMYmXbKAaG76+WkxRJudJVsUOetJeNdY?= =?us-ascii?Q?wnJgyBa/ZxZNz6CEU9yfkulGPtZ21RQ4cuQAjG6688B7mWtq4gP/kNv6wL6K?= =?us-ascii?Q?00btU5z8Qsx0e9lYLAm/VoqjkTkmGTgseJ8MtvDBCcw1DcX8N3KT3qptf3kP?= =?us-ascii?Q?kffl062IwN+WCGW9AbtAzlMM3ynqqD2YmphKgzU9fS2Fxxf16cO7uVQh4QGZ?= =?us-ascii?Q?e8TfNuqW4fuvmgTWUsDtBTUQBq9HkIxL+WgAZywo5xr5MqJV8AavRFQUOcx4?= =?us-ascii?Q?RGr6Lv6mLFes/Poj3srlV0T6zc4XiMxnf8XzhkaJWK8D6roqogDhoOaWg+mw?= =?us-ascii?Q?VgOTYTHyXm6ViEIStElXrE2zPQZGNeJ6frVYX7bpTKjkf/Ak/jAxJlEte+Jd?= =?us-ascii?Q?b30touqr5bZmzBD/mRAxUO6x9qv4pDDUYxbXz52JS1RnZuUC5kaOs3jV5dgG?= =?us-ascii?Q?N4CFoQApoCjWpaD4q0nJU3PzKcnv0SfTxFb7W4iI1cz6vvuczpQ8HsvfdYly?= =?us-ascii?Q?XxDBaulK/2S1ypGyj4zERNWa/IrWCFHC8h2LGHe1otGUFpj9v6rSY63y9nMd?= =?us-ascii?Q?nNnExZEmle3cFyoxjCou/7lNPsGddeD9eqvNULvEwSVgNf9wE8GGpfyZx5jI?= =?us-ascii?Q?wq25S9poszWv923edgFhmRFbIs50pcN1tgCdEiOYj3vHBzVmkyIaZkJb7kUy?= =?us-ascii?Q?520X+K94t5IePGvd7FbflYaIij1SwfwhhZDAimf6plidXG27tQmLD9oKS/42?= =?us-ascii?Q?FA=3D=3D?= X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: htaBqSJvUF6DyjbFkUiZ/7LtZKRIukc9UwISmaAGIVjvEy82rXNrjvs90hLZcanA5PGTs+euct3zpAUB98hOKy1RUv8PtLLmbapGV0D1dc9Nrmtnd0Esfcmzz73drUXjstkljcSuOtFKwP80OUwRXB1Ufm3zZRyubS+0SnZq1q0/jSpSRWVEzH6JJbibdj9h52sUhXk1eXnHDZcf7BHQjWgo0OU6ZnjSdAjOmhHR8E78Ke6RmiT6gQb0W8vs0cgR7uN8keE1yMr7V/N2hKKrwFIxmmnCogtsN9YlLwBWjPE1aTA5ovSgWIgBd2HTFIg3/bC7Qu8MKKkpH73DMVCnCp69JJby6mQtLoKsBm0KFBJ5FpMJHgymdgIG8rcBs7Dfvf8Tz6Ak+QoFRd5KlfN1ytEtTz6dCKm8tKdZ2TILqBJJx+teH/TedgyTPXTbjG70WHKPh2sRrs27Mg+LhKsMt/rg4km7tGGcfk3JCnKWwn/ElhUQoyOvUafdYeh8OhHZtZXSeIiHzXTPBSKxUEjjx9d6c0s/Y4nJDahCB/oDJte0Igh9E0WHG9cyBUb8antuOm5hTPFKrndZT30qyEugS3Ug27/ja+SCIKrWYYCMxok= X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: 7868c322-5723-4939-46ef-08de1ee989d8 X-MS-Exchange-CrossTenant-AuthSource: DM4PR10MB8218.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 Nov 2025 17:09:12.4426 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: yVwpzqfyIlX9eLVaHyjPb+Kuy7gaBsKARMdi83WyFY1IF3E9DOXv0iAX4GXgMthvqr06HxeArLWPKKn+SG2Cw//9nyxVADdtyOLImPabG8c= X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ5PPF04D2D7FA7 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1121,Hydra:6.1.9,FMLib:17.12.100.49 definitions=2025-11-08_04,2025-11-06_01,2025-10-01_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 phishscore=0 adultscore=0 malwarescore=0 spamscore=0 suspectscore=0 bulkscore=0 mlxscore=0 mlxlogscore=999 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2510240000 definitions=main-2511080138 X-Proofpoint-Spam-Details-Enc: AW1haW4tMjUxMTA4MDAxNiBTYWx0ZWRfX1QI9N8iRj+vS 7d3T4SnkdBY16FV8gCvedEWJaF8ml1cfbaDbbpO5JiFm0mHTDIPX3f5ekhxpibkZpBT9RFsTvBj KLUSUN+sP3AqbTNimfu8klJwzTwH+Y3BVglNzj6jAAnPhjFO6T2Rkn9ijGu7zZXRVByC1mBQg0w N2qXxcHEb4GsU/bJqfnw380aJ5HGcPpEdOFlTxcGybCabY0c6Ij6A8h3uF7q7LcftEUNQ5TUB5e V5HwTwjGW6aQhyETB8667TXboWb9IXRmzROZjG2cZxk+PNqtWMNdNNvAk7pSv8aWH5LmXUM5UVW s/SnG786aonufSBRTWbI+XLJclQ9EsofLwEKAcnZysQ9ZmNnWxxQFcf1saIxnvwhsvFAs5+dy9S jlO30FImW5kh0N0uF4AOkk0NiojZS1L6JTjUwidHnLq42/xPj2g= X-Proofpoint-ORIG-GUID: vHyhdle_Sbz1MW1C4wyF3Vxa7pZlaVUq X-Proofpoint-GUID: vHyhdle_Sbz1MW1C4wyF3Vxa7pZlaVUq X-Authority-Analysis: v=2.4 cv=eYgwvrEH c=1 sm=1 tr=0 ts=690f793f b=1 cx=c_pps a=e1sVV491RgrpLwSTMOnk8w==:117 a=e1sVV491RgrpLwSTMOnk8w==:17 a=6eWqkTHjU83fiwn7nKZWdM+Sl24=:19 a=z/mQ4Ysz8XfWz/Q5cLBRGdckG28=:19 a=lCpzRmAYbLLaTzLvsPZ7Mbvzbb8=:19 a=xqWC_Br6kY4A:10 a=6UeiqGixMTsA:10 a=GoEa3M9JfhUA:10 a=VkNPw1HP01LnGYTKEx00:22 a=yPCof4ZbAAAA:8 a=9UmCEYkwnbpDqGD1g-YA:9 cc=ntf awl=host:13629 Content-Type: text/plain; charset="utf-8" Right now we are inconsistent in our use of thp_migration_supported(): static inline bool thp_migration_supported(void) { return IS_ENABLED(CONFIG_ARCH_ENABLE_THP_MIGRATION); } And simply having arbitrary and ugly #ifdef CONFIG_ARCH_ENABLE_THP_MIGRATION blocks in code. This is exhibited in copy_huge_pmd(), which inserts a large #ifdef CONFIG_ARCH_ENABLE_THP_MIGRATION block and an if-branch which is difficult to follow It's difficult to follow the logic of such a large function and the non-present PMD logic is clearly separate as it sits in a giant if-branch. Therefore this patch both separates out the logic and utilises thp_migration_supported(). No functional change intended. Signed-off-by: Lorenzo Stoakes --- mm/huge_memory.c | 109 +++++++++++++++++++++++++---------------------- 1 file changed, 59 insertions(+), 50 deletions(-) diff --git a/mm/huge_memory.c b/mm/huge_memory.c index 2e5196a68f14..31116d69e289 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -1774,6 +1774,62 @@ void touch_pmd(struct vm_area_struct *vma, unsigned = long addr, update_mmu_cache_pmd(vma, addr, pmd); } =20 +static void copy_huge_non_present_pmd( + struct mm_struct *dst_mm, struct mm_struct *src_mm, + pmd_t *dst_pmd, pmd_t *src_pmd, unsigned long addr, + struct vm_area_struct *dst_vma, struct vm_area_struct *src_vma, + pmd_t pmd, pgtable_t pgtable) +{ + swp_entry_t entry =3D pmd_to_swp_entry(pmd); + struct folio *src_folio; + + VM_WARN_ON(!is_pmd_non_present_folio_entry(pmd)); + + if (is_writable_migration_entry(entry) || + is_readable_exclusive_migration_entry(entry)) { + entry =3D make_readable_migration_entry(swp_offset(entry)); + pmd =3D swp_entry_to_pmd(entry); + if (pmd_swp_soft_dirty(*src_pmd)) + pmd =3D pmd_swp_mksoft_dirty(pmd); + if (pmd_swp_uffd_wp(*src_pmd)) + pmd =3D pmd_swp_mkuffd_wp(pmd); + set_pmd_at(src_mm, addr, src_pmd, pmd); + } else if (is_device_private_entry(entry)) { + /* + * For device private entries, since there are no + * read exclusive entries, writable =3D !readable + */ + if (is_writable_device_private_entry(entry)) { + entry =3D make_readable_device_private_entry(swp_offset(entry)); + pmd =3D swp_entry_to_pmd(entry); + + if (pmd_swp_soft_dirty(*src_pmd)) + pmd =3D pmd_swp_mksoft_dirty(pmd); + if (pmd_swp_uffd_wp(*src_pmd)) + pmd =3D pmd_swp_mkuffd_wp(pmd); + set_pmd_at(src_mm, addr, src_pmd, pmd); + } + + src_folio =3D pfn_swap_entry_folio(entry); + VM_WARN_ON(!folio_test_large(src_folio)); + + folio_get(src_folio); + /* + * folio_try_dup_anon_rmap_pmd does not fail for + * device private entries. + */ + folio_try_dup_anon_rmap_pmd(src_folio, &src_folio->page, + dst_vma, src_vma); + } + + add_mm_counter(dst_mm, MM_ANONPAGES, HPAGE_PMD_NR); + mm_inc_nr_ptes(dst_mm); + pgtable_trans_huge_deposit(dst_mm, dst_pmd, pgtable); + if (!userfaultfd_wp(dst_vma)) + pmd =3D pmd_swp_clear_uffd_wp(pmd); + set_pmd_at(dst_mm, addr, dst_pmd, pmd); +} + int copy_huge_pmd(struct mm_struct *dst_mm, struct mm_struct *src_mm, pmd_t *dst_pmd, pmd_t *src_pmd, unsigned long addr, struct vm_area_struct *dst_vma, struct vm_area_struct *src_vma) @@ -1819,59 +1875,12 @@ int copy_huge_pmd(struct mm_struct *dst_mm, struct = mm_struct *src_mm, ret =3D -EAGAIN; pmd =3D *src_pmd; =20 -#ifdef CONFIG_ARCH_ENABLE_THP_MIGRATION - if (unlikely(is_swap_pmd(pmd))) { - swp_entry_t entry =3D pmd_to_swp_entry(pmd); - - VM_WARN_ON(!is_pmd_non_present_folio_entry(pmd)); - - if (is_writable_migration_entry(entry) || - is_readable_exclusive_migration_entry(entry)) { - entry =3D make_readable_migration_entry(swp_offset(entry)); - pmd =3D swp_entry_to_pmd(entry); - if (pmd_swp_soft_dirty(*src_pmd)) - pmd =3D pmd_swp_mksoft_dirty(pmd); - if (pmd_swp_uffd_wp(*src_pmd)) - pmd =3D pmd_swp_mkuffd_wp(pmd); - set_pmd_at(src_mm, addr, src_pmd, pmd); - } else if (is_device_private_entry(entry)) { - /* - * For device private entries, since there are no - * read exclusive entries, writable =3D !readable - */ - if (is_writable_device_private_entry(entry)) { - entry =3D make_readable_device_private_entry(swp_offset(entry)); - pmd =3D swp_entry_to_pmd(entry); - - if (pmd_swp_soft_dirty(*src_pmd)) - pmd =3D pmd_swp_mksoft_dirty(pmd); - if (pmd_swp_uffd_wp(*src_pmd)) - pmd =3D pmd_swp_mkuffd_wp(pmd); - set_pmd_at(src_mm, addr, src_pmd, pmd); - } - - src_folio =3D pfn_swap_entry_folio(entry); - VM_WARN_ON(!folio_test_large(src_folio)); - - folio_get(src_folio); - /* - * folio_try_dup_anon_rmap_pmd does not fail for - * device private entries. - */ - folio_try_dup_anon_rmap_pmd(src_folio, &src_folio->page, - dst_vma, src_vma); - } - - add_mm_counter(dst_mm, MM_ANONPAGES, HPAGE_PMD_NR); - mm_inc_nr_ptes(dst_mm); - pgtable_trans_huge_deposit(dst_mm, dst_pmd, pgtable); - if (!userfaultfd_wp(dst_vma)) - pmd =3D pmd_swp_clear_uffd_wp(pmd); - set_pmd_at(dst_mm, addr, dst_pmd, pmd); + if (unlikely(thp_migration_supported() && is_swap_pmd(pmd))) { + copy_huge_non_present_pmd(dst_mm, src_mm, dst_pmd, src_pmd, addr, + dst_vma, src_vma, pmd, pgtable); ret =3D 0; goto out_unlock; } -#endif =20 if (unlikely(!pmd_trans_huge(pmd))) { pte_free(dst_mm, pgtable); --=20 2.51.0 From nobody Fri Dec 19 21:47:08 2025 Received: from mx0b-00069f02.pphosted.com (mx0b-00069f02.pphosted.com [205.220.177.32]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3CEC523D28F; Sat, 8 Nov 2025 17:10:42 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=205.220.177.32 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1762621844; cv=fail; b=P5fFswXZpW4TJz1xReAUTQPuyDaMzgHL0owZMe27elM3fVivLyMS2oA0WcngTAhmEmdsO6DZ6qrIRew1iEb0eBeFAe+1I46uNFx8azcLFt2KGXe9+ObFqQ+vt5MJNUEk+QwgpyJKmub97NSten7FKwlFaIN1UyY6kVqxox9KvCM= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1762621844; c=relaxed/simple; bh=wEswhEPB/UrhDyPK9C9sI1IHFlT5+kgMCKmd18ehtZM=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: Content-Type:MIME-Version; b=ZTSG1PypHSeHWOULNzW/IiVbCLP4FQQ68zzNYnXw5KFG8eiU/x8mPwtDsgP31nvAn4QBWt9e87IZ4+LWTb643lrJ9Lkkdm5tDtTGdW60LFhSUcTJHaQW3O8KMuZ5zsBCu6iYUsomFDgdUJ2aoaUMe0L7XBZG5zH9CUN1ksXzz0s= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=oracle.com; spf=pass smtp.mailfrom=oracle.com; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b=OnM5mLJB; dkim=pass (1024-bit key) header.d=oracle.onmicrosoft.com header.i=@oracle.onmicrosoft.com header.b=LkX6gcNf; arc=fail smtp.client-ip=205.220.177.32 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=oracle.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=oracle.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b="OnM5mLJB"; dkim=pass (1024-bit key) header.d=oracle.onmicrosoft.com header.i=@oracle.onmicrosoft.com header.b="LkX6gcNf" Received: from pps.filterd (m0246630.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 5A8GfF1A008827; Sat, 8 Nov 2025 17:09:21 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=cc :content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s= corp-2025-04-25; bh=ES+ODnUpx6m2BxJg2i8m0/ywBdmg7etQq70a9R7KC9k=; b= OnM5mLJBkADXw8RlxMX7twRcHtpTl+zG1wc6SN4kyUJWzjQKT3q0L6YgNPQ8CYTC 5PvvWgv6sy/ifF9YcX8j1UP4m6X5eC/0UNkfvxx7G/d/XOaVuCtYJ4GzfiqI8G+I nHnaqFoK4WrZ1sD2qpClJsjDGvHmkpCJohQVY8TkzvgM3E4pTcNP0AzaGzno+Pwx 2a00L6McD8VIiRh26JuCMv2VTzyKbYq1Rop7jus8mrW6/wB2SULi8R/0GsZ19fMM ilujjboZZ8PzlfDnceB7NnXMcQ5EMVr1CvTfzV7Yj7iG8O1V+QFdKsddg3UAoFjP iOGYO52Oriy/wzARImJ0iw== Received: from phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta02.appoci.oracle.com [147.154.114.232]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 4aa9k8g0j7-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Sat, 08 Nov 2025 17:09:21 +0000 (GMT) Received: from pps.filterd (phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (8.18.1.2/8.18.1.2) with ESMTP id 5A8FUqDV020358; Sat, 8 Nov 2025 17:09:20 GMT Received: from sn4pr0501cu005.outbound.protection.outlook.com (mail-southcentralusazon11011059.outbound.protection.outlook.com [40.93.194.59]) by phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTPS id 4a9va6pmxb-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Sat, 08 Nov 2025 17:09:20 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=mtHwMzWUlOiGCYDT3yPWdtJuUkAxEPzDA+LZYrzhqTh6hRW0Ok0cK+9jh+xqOjYRhH0UgaS6SLlLMfvZewt9iL46HbH9SUDagTskQORqDzGbFUem+NAFvzu2OLXwqE5S2mrjg76D7c2ssrvAXmv1M7AvAVThr8NzLGe4nyv6K+NlRRpy4BYkTS70oftylOChRahDtjY0mxwgoeYxvZ0ideAtNR2yBco31BjGiNMB9Hqj13BzBB4FBbxs4y/saa7nWBeQQyvtXxNhTeaq4nJlzS/xWskM6L8PiE4PIx0D3awvT1gRUi1qUsSx+QKHfAeeIWifHnMnEYmACTsWIdZ6Rw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=ES+ODnUpx6m2BxJg2i8m0/ywBdmg7etQq70a9R7KC9k=; b=eKBCtqwLIEf57PM2pJ8JYRG/KBPy05YAcht8DBk8xJG49gyWPp/Zmne/gfuDCbHjwQ6982Jz3NMxe2AAjvFS1PqjC3RUa87Poj1nfp3rdKt3PwVj2Nzr6qeNg+yA4pFJ1rbJcruleh+1bsm/ZJG81ppYaNs02EUckOKkUD4O27yHbwqPd0l2mZXxQav39cXUORA7PzhIwwM8phDGotbrfpX+6ioIGZA9VT26CqsFxYJKmTLftPB4a6L+MsOsM858ueO9Larzov+P1pGj3bHlYViYMA/xSg0+8HTADV8SvpAhWTuf4OTJeLlMS1xTNYVZj/h/pa13FeT+b7UV7ng2+A== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=ES+ODnUpx6m2BxJg2i8m0/ywBdmg7etQq70a9R7KC9k=; b=LkX6gcNfUA2YOykfjoq8ukOy3qOHQixZ5gDOAL/ObS6+p3F8X/L5durtS6UlpZk51aHjERlv7cMkzrbH76/LRjVY1cbDTibwOvH7Ucjc87zHT0ykUw7V/CYpxj7i+2Oki9GpiZRQ+b4fqoSTaKAPMwvOaec6gHPUTQQ7uo0f6UU= Received: from DM4PR10MB8218.namprd10.prod.outlook.com (2603:10b6:8:1cc::16) by SJ5PPF04D2D7FA7.namprd10.prod.outlook.com (2603:10b6:a0f:fc02::785) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9298.15; Sat, 8 Nov 2025 17:09:15 +0000 Received: from DM4PR10MB8218.namprd10.prod.outlook.com ([fe80::2650:55cf:2816:5f2]) by DM4PR10MB8218.namprd10.prod.outlook.com ([fe80::2650:55cf:2816:5f2%7]) with mapi id 15.20.9298.010; Sat, 8 Nov 2025 17:09:15 +0000 From: Lorenzo Stoakes To: Andrew Morton Cc: Christian Borntraeger , Janosch Frank , Claudio Imbrenda , David Hildenbrand , Alexander Gordeev , Gerald Schaefer , Heiko Carstens , Vasily Gorbik , Sven Schnelle , Peter Xu , Alexander Viro , Christian Brauner , Jan Kara , Arnd Bergmann , Zi Yan , Baolin Wang , "Liam R . Howlett" , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Lance Yang , Muchun Song , Oscar Salvador , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Matthew Brost , Joshua Hahn , Rakie Kim , Byungchul Park , Gregory Price , Ying Huang , Alistair Popple , Axel Rasmussen , Yuanchu Xie , Wei Xu , Kemeng Shi , Kairui Song , Nhat Pham , Baoquan He , Chris Li , SeongJae Park , Matthew Wilcox , Jason Gunthorpe , Leon Romanovsky , Xu Xin , Chengming Zhou , Jann Horn , Miaohe Lin , Naoya Horiguchi , Pedro Falcato , Pasha Tatashin , Rik van Riel , Harry Yoo , Hugh Dickins , linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-s390@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, linux-arch@vger.kernel.org, damon@lists.linux.dev Subject: [PATCH v2 09/16] mm/huge_memory: refactor change_huge_pmd() non-present logic Date: Sat, 8 Nov 2025 17:08:23 +0000 Message-ID: X-Mailer: git-send-email 2.51.0 In-Reply-To: References: Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: LO4P123CA0652.GBRP123.PROD.OUTLOOK.COM (2603:10a6:600:296::13) To DM4PR10MB8218.namprd10.prod.outlook.com (2603:10b6:8:1cc::16) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DM4PR10MB8218:EE_|SJ5PPF04D2D7FA7:EE_ X-MS-Office365-Filtering-Correlation-Id: 242f6bd5-0ad0-4e5d-2dcc-08de1ee98bbf X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|1800799024|7416014|376014|366016; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?NQ2vAalz9bBOjio0XpIoArxJ7+RHb9fU04aVwX2Cx3IexUQFVz6weBtXuw98?= =?us-ascii?Q?d4sn08LHAsZqFzG2n3jUYANt8RyTMLzv3o3F3S3OaLRpjworcMK3ZxO26Wrt?= =?us-ascii?Q?IcqFb+5qztTqV8allNbhbN/xujsPqJaQWwot758u37DZiHS56bzHwA7OfYIq?= =?us-ascii?Q?jdi6fTNKJ1/f0sUaPo5IDrFZ1Dp6sVVNoOuWWGVq0G8Ol2uaRZcpwJGVZUGt?= =?us-ascii?Q?blHrLcRGJ1J/H1eZ9RlRTksP69CrEWM5sq0ZGLL0A1aAvqTQgGvT3bhPn1G1?= =?us-ascii?Q?UD1zf2kwaHfWtBRIKfGdyb2bvT1Dw78eoFpHvFKpe5MTvUupHJqCqTeNhwoE?= =?us-ascii?Q?9evaljdEOxrPIjfZJNHwfZrv8I/6yv7CkcpVZxIVy/EaAaHkYwpGZuUrIks5?= =?us-ascii?Q?RPQi32xs8Ets+Khckm2393c6f+uXypUItNFjXulw0p6F/XZ0sQZrvznVqcrN?= =?us-ascii?Q?o7rNh9HYJZaEQKwVHjw3F/Eox0L8D60jjaqwhXvAK3lBc3v/W3v3SzrgJ79T?= =?us-ascii?Q?vvOn5uoiJ8UhwIcJL7QODj+/mi5YrK1l4r8fizU2/ZyKcF6EvVfNGCyOpEie?= =?us-ascii?Q?ggo6TPNBrrv0R+1Ne3BWzQav0StMYqVaQ2T76E2bzNGdcr21p7jxiQuCCq06?= =?us-ascii?Q?FFtSJN6bIGWKA7cbWKu3ggcmQh54Oup8DCzegN2gcNtaCVk7PsDKAe3/yECe?= =?us-ascii?Q?DAAM/Hsn1qXcHkf9CJlbPFRzfqT+bzCr5T3nMzZ072F1VEwJM4IhN8OHY0hg?= =?us-ascii?Q?YqFKvDa6iWGvDZTleRYV/FXpw/vT/W2MFuq7hu0K5zrMsvoJEa6nXwKZV2ZP?= =?us-ascii?Q?+dUsVRXGi8jOwxeklEAuzOHnrsNKkV8/zwDf2PYJlzesrtwVQS/qYRN1Og6d?= =?us-ascii?Q?TeIu4vK3bOuuPbOMFtP40yd7PNGndIpRfoa1cG8B/0Rq9Saq9KomyG9AyX9a?= =?us-ascii?Q?bIcDV7uAPr6me1vqvsVpqdZMiehWAstek3bG3eaiWYiK/dvlXG0jkL1Ag/X0?= =?us-ascii?Q?Zun2P4CIQ1JfVlusI7x9dlF6KqJRsdbkueT6hXt5Fr8rce432gka7He9vhg6?= =?us-ascii?Q?hgkDgafCYL0xLEXfRWAdgnxCiCyc15zJ03l56NixTXogIShcXv8boIBE1xSP?= =?us-ascii?Q?kAeKGAWMOW4TPn+v0GMvo8FBNRPEqvxdTx2DVgNCv9lHKtCePdP8doplOGDq?= =?us-ascii?Q?ckVrE0DoCiJWYyTqUkJQq28wiFIeZgPfsAkvDyMJgMojLkaoPTr8nm30pkNh?= =?us-ascii?Q?LDE9B27iKzJXnzY9Gx7VjZkZgr1m8fKG0StARZC8/05k3ABHZ80W6hulkQUg?= =?us-ascii?Q?H02hNRlILkw1aKFBNm0FrdoOoGAfQVDr+JCREeiQObhann0LYwmYkgpKxwZw?= =?us-ascii?Q?UdX143Rll+sBZOosb1HdhzS1ppsSP5WDO/V67+HLS0WhG4Q6pO+k9mtzNPGQ?= =?us-ascii?Q?OANtDH2kXawF8cLcoaT7sQyxCzuszlKp?= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DM4PR10MB8218.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(1800799024)(7416014)(376014)(366016);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?XJjrJfnEg27XE0U+MYvckQNKlrtQtQXmIrQCijxlMjvPtSueBlJCfu7ahAbu?= =?us-ascii?Q?yKY+Il+Lc+nkza26X0aoF8A3DIBaCavpUxmSEiVthm6oaEtLPl72WoOOJAcY?= =?us-ascii?Q?Og+RHnZR8ph3d30X5BbVhO968UJxN0c2q5SCqf3ew0okHIq/pCOv5lSHUj6k?= =?us-ascii?Q?hj0KduALUJBDUh6AhK/7i6Mj26ku/VrrvB1ES+UzVicvbTAG/ocGCoIpUbVW?= =?us-ascii?Q?h0D96DnZbwSB7S7XI3C46VC/nrGdhmlLvEOVnnm69oqRe8eRrMRMg9xfiAFc?= =?us-ascii?Q?BZj2yBfZYyE1NyCvmTrK6+b5ITilz/0lZU7pskIkECBOVgZrfdG3A7acs0/T?= =?us-ascii?Q?Png5euzBnrrEGxGePcy4BQzX3/FMgPfyNDBKdLBiOkAgmV0r1VJrOxehSHXg?= =?us-ascii?Q?cKckD/ii0sfWvgPQpoSMppFc/FqEfN+TuhND3YQJWTncv+FqqtpDO8vihv6B?= =?us-ascii?Q?1Ijqx0+FiGbGAZwg5StZNQe8m5hLUP7apiGveEb7mFYkjBWnueJwIF00NF/8?= =?us-ascii?Q?mOKH7o2mSC57yZiYLxkxZQxnIJItzy9X7dBqvUedbUdo/qsT4jc9zOvhwtAt?= =?us-ascii?Q?aWZHbNFegdnayeKz/6xaozXQXoWD5lbnxzrobthsUfYjcJQwqNaedXrTRtxJ?= =?us-ascii?Q?jVKjK56ndqEJmIq3yN8cscmA8bkEfQY3kkV5SvDoWNNCDI19Uaa5lQiFleMe?= =?us-ascii?Q?FeHHkBllUPVMSZKtdNpnl09SQdowX/11n7Y3bL8YZJB408UPS7VjWcdBcL+s?= =?us-ascii?Q?BMQbLGeMSWjrkKWE/Mv9F0rPgzTMhEITduFpvEsuGuKW5CzOZj4GC1g7D7DU?= =?us-ascii?Q?vghM3xyVBSZDus2j//BqHYltPC2fdNoD2GaiMcPwyXWzyP/RGmdglC0ypTXe?= =?us-ascii?Q?rBV3t+3mViWVi3Uf1/CF+DKWNn1JKIHM8HJIke4+CkfTYNizoetyu/+XeRD4?= =?us-ascii?Q?rthqbezUwHV9yYauOVneo+U1k2gFeHMZ3bzodan5woP+So8qI0RQYrdULgsz?= =?us-ascii?Q?Wbnm4rZNx/fBbVYob4sc6ekwF4XgUuSfRZbATEwVjW2wE0bNXl+qbinrgDMh?= =?us-ascii?Q?UJ+9ek/yM1YdLs86O6w0Cg7H+yYnPhH0hvXAAWZ2FmAhQMgNVaY3kb0BtPDC?= =?us-ascii?Q?OFFsBSlsU11rqhLCbvWckAo7mJKmVz+SJB5s8AoFSMJzWeofKFmPh7wLBrR2?= =?us-ascii?Q?PPaRZ2r6kKGB0bq/LEzmYsz6iVAUdosxPAktLPi4QeiSnBWE9j6jeSY2GVqv?= =?us-ascii?Q?PV3jnz87/b0X7obFQNVWqeXQeFa5DYZlorZLuvxaXA2ZB2NhcFqGt3njiqZi?= =?us-ascii?Q?k25Y8RF/Og4eak0b9UpxTBNNvrTKgaq/G97CY8eQrPl2oLI/0nqrOd0tOZEi?= =?us-ascii?Q?3EB0B/sZJbzZnVfhviYNxFKus/BbmohGUIaeKU8blPP2egPDQD38ooW2f2Hn?= =?us-ascii?Q?90C4bEi2ovB2WWF8OPlCp7ZN89xC548jBI6qvfky38CeHOjr4Jy9FiUT7fpP?= =?us-ascii?Q?0shSwI+UFvQSp5KPMTCcMvFNwyeymy9B6isG+IX+zMHfJ1loR97zYDQpHfDQ?= =?us-ascii?Q?sqTnmRorJqn742rgv9gpw9U0ahE/OA38yATBnW9htfD7bOMv7Sb93NRhaIDO?= =?us-ascii?Q?RQ=3D=3D?= X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: t66THdOwnKf3Ska6WeV0Zxl05lIKfcOX3StwSi8/nar6MGB7YPRxLDAPCnXDrsy5eUAjFeii/OEw3FOABIUodV0Nq7kK3q8g3S5DTWpIbe1aD+amUYDtDs013F0OMARN4Fp3vJBkMRZkAn0y0vqybfPqCbqPKzZFc9Lx+oXh0wKZKkI97zfULuVetCRTiVacVtcQwB5w94OAB/g/KYgJbseofkuuING+IJBXgVMPqHUPExw18suaEv/EQJlAJxUztED8mNPutkLdaUmjYU2R7ykCrvKuRmqq6ATYQAJUU++iCY+boIxwx90bHeRETpf5Tr30c90NRmrBb+v0x2ZTUltXpJvxpRqIuGSxU/n0nHZHLVLNP8yNSjycEBbNAUb/FK20hGh0K8jqysElJqS7hFDairhItNvrGuvAMV81Cq9dzkgMtuLVNr7ieSzBjCp5k32S5qyWTmch7eIGpO5Q0Yik+NCkZcywQ5ylpK/N3YpvxPm7ZoEsLQD8s5vp7QgkXBubDw3nC35vkEp1kN5Sf1Pg+xhqd8nGXgRDqT3jvkAvp+6H9Eju9uW7mbDGd7IysM6CabpZ+82OqY+DeKj7nYBkYiHIV6ivE4BXuyHtXMA= X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: 242f6bd5-0ad0-4e5d-2dcc-08de1ee98bbf X-MS-Exchange-CrossTenant-AuthSource: DM4PR10MB8218.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 Nov 2025 17:09:15.6441 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: rBl9znN11xDezq5/sK3cIjYlef7vctQCNwgY4AVV9HLC7g1UG8ZoBn+7rQz/eJQ9nD4iy1D/2gMnwlkeiymICOwRJD/W2mhBrDheJRNZFjo= X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ5PPF04D2D7FA7 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1121,Hydra:6.1.9,FMLib:17.12.100.49 definitions=2025-11-08_04,2025-11-06_01,2025-10-01_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxlogscore=999 malwarescore=0 adultscore=0 bulkscore=0 suspectscore=0 spamscore=0 mlxscore=0 phishscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2510240000 definitions=main-2511080138 X-Proofpoint-Spam-Details-Enc: AW1haW4tMjUxMTA4MDEzNCBTYWx0ZWRfXyt4f+F2U5dc4 setjwsSoO6I9+ZdCCKEJ+6mFVmT5VdLIIZi4IeSBoBdgkze7OLW3I7Qm38JKiRiRO+K/NxRc1p6 P/qFT9AWulzhmaDFVspNpT5d4Vqdp6aW573TU9yXqAOdRC5lvoTzoyp7Dh+2o5H5TRxpZbAWS1g OsQMGhxZKGFcI+x0aai4/K849vgADQZi6s8MjxITsMuSTA3/S6zQghOBjZPv2xWdt9I0ph2L4Ll klh7K3KAPwyJVoGTT0LVXFf5ra13XToERhcZSInRZUQ97/Cx0VeqIgA2/nvgfn5iCrOCmP+xbc1 mimygRZEz+KGrGlwppu75iLjEYw60wmTYnVVTCayZowKvkR44rCI6dYuH/SUpP4ZMfdaxs5+yR1 p986iBTtOC3O1FL1qoMXV/hrXrtKhw== X-Proofpoint-ORIG-GUID: byv2T6MxfRUczHHbX0zqXEgFXZiXvsW1 X-Authority-Analysis: v=2.4 cv=U4ufzOru c=1 sm=1 tr=0 ts=690f7941 cx=c_pps a=OOZaFjgC48PWsiFpTAqLcw==:117 a=OOZaFjgC48PWsiFpTAqLcw==:17 a=6eWqkTHjU83fiwn7nKZWdM+Sl24=:19 a=z/mQ4Ysz8XfWz/Q5cLBRGdckG28=:19 a=lCpzRmAYbLLaTzLvsPZ7Mbvzbb8=:19 a=xqWC_Br6kY4A:10 a=6UeiqGixMTsA:10 a=GoEa3M9JfhUA:10 a=VkNPw1HP01LnGYTKEx00:22 a=yPCof4ZbAAAA:8 a=gAL6uhOEssAPNySJDCYA:9 a=UhEZJTgQB8St2RibIkdl:22 a=Z5ABNNGmrOfJ6cZ5bIyy:22 a=QOGEsqRv6VhmHaoFNykA:22 X-Proofpoint-GUID: byv2T6MxfRUczHHbX0zqXEgFXZiXvsW1 Content-Type: text/plain; charset="utf-8" Similar to copy_huge_pmd(), there is a large mass of open-coded logic for the CONFIG_ARCH_ENABLE_THP_MIGRATION non-present entry case that does not use thp_migration_supported() consistently. Resolve this by separating out this logic and introduce change_non_present_huge_pmd(). No functional change intended. Signed-off-by: Lorenzo Stoakes --- mm/huge_memory.c | 72 ++++++++++++++++++++++++++---------------------- 1 file changed, 39 insertions(+), 33 deletions(-) diff --git a/mm/huge_memory.c b/mm/huge_memory.c index 31116d69e289..40a8a2c1e080 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -2499,6 +2499,42 @@ bool move_huge_pmd(struct vm_area_struct *vma, unsig= ned long old_addr, return false; } =20 +static void change_non_present_huge_pmd(struct mm_struct *mm, + unsigned long addr, pmd_t *pmd, bool uffd_wp, + bool uffd_wp_resolve) +{ + swp_entry_t entry =3D pmd_to_swp_entry(*pmd); + struct folio *folio =3D pfn_swap_entry_folio(entry); + pmd_t newpmd; + + VM_WARN_ON(!is_pmd_non_present_folio_entry(*pmd)); + if (is_writable_migration_entry(entry)) { + /* + * A protection check is difficult so + * just be safe and disable write + */ + if (folio_test_anon(folio)) + entry =3D make_readable_exclusive_migration_entry(swp_offset(entry)); + else + entry =3D make_readable_migration_entry(swp_offset(entry)); + newpmd =3D swp_entry_to_pmd(entry); + if (pmd_swp_soft_dirty(*pmd)) + newpmd =3D pmd_swp_mksoft_dirty(newpmd); + } else if (is_writable_device_private_entry(entry)) { + entry =3D make_readable_device_private_entry(swp_offset(entry)); + newpmd =3D swp_entry_to_pmd(entry); + } else { + newpmd =3D *pmd; + } + + if (uffd_wp) + newpmd =3D pmd_swp_mkuffd_wp(newpmd); + else if (uffd_wp_resolve) + newpmd =3D pmd_swp_clear_uffd_wp(newpmd); + if (!pmd_same(*pmd, newpmd)) + set_pmd_at(mm, addr, pmd, newpmd); +} + /* * Returns * - 0 if PMD could not be locked @@ -2527,41 +2563,11 @@ int change_huge_pmd(struct mmu_gather *tlb, struct = vm_area_struct *vma, if (!ptl) return 0; =20 -#ifdef CONFIG_ARCH_ENABLE_THP_MIGRATION - if (is_swap_pmd(*pmd)) { - swp_entry_t entry =3D pmd_to_swp_entry(*pmd); - struct folio *folio =3D pfn_swap_entry_folio(entry); - pmd_t newpmd; - - VM_WARN_ON(!is_pmd_non_present_folio_entry(*pmd)); - if (is_writable_migration_entry(entry)) { - /* - * A protection check is difficult so - * just be safe and disable write - */ - if (folio_test_anon(folio)) - entry =3D make_readable_exclusive_migration_entry(swp_offset(entry)); - else - entry =3D make_readable_migration_entry(swp_offset(entry)); - newpmd =3D swp_entry_to_pmd(entry); - if (pmd_swp_soft_dirty(*pmd)) - newpmd =3D pmd_swp_mksoft_dirty(newpmd); - } else if (is_writable_device_private_entry(entry)) { - entry =3D make_readable_device_private_entry(swp_offset(entry)); - newpmd =3D swp_entry_to_pmd(entry); - } else { - newpmd =3D *pmd; - } - - if (uffd_wp) - newpmd =3D pmd_swp_mkuffd_wp(newpmd); - else if (uffd_wp_resolve) - newpmd =3D pmd_swp_clear_uffd_wp(newpmd); - if (!pmd_same(*pmd, newpmd)) - set_pmd_at(mm, addr, pmd, newpmd); + if (thp_migration_supported() && is_swap_pmd(*pmd)) { + change_non_present_huge_pmd(mm, addr, pmd, uffd_wp, + uffd_wp_resolve); goto unlock; } -#endif =20 if (prot_numa) { =20 --=20 2.51.0 From nobody Fri Dec 19 21:47:08 2025 Received: from mx0a-00069f02.pphosted.com (mx0a-00069f02.pphosted.com [205.220.165.32]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id AFAFE242D78; Sat, 8 Nov 2025 17:10:48 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=205.220.165.32 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1762621851; cv=fail; b=ba+o4ifUiruHqNpZlT1IzJmd+bwlfvrnvh71SuoBeEkudkxXFqaJyAQN3jzSxcClA/RH88IbyKWytd2bTYZACcwX67Q9YDDnfUGgTE0QoIebY8DsOJrf8Go9daJROMskQHm0WLy/VTU5lZRmZ5VSRJFglHKmaGSfyVAw9BWR//0= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1762621851; c=relaxed/simple; bh=zmWrD5tfCxpjD0F6zVZO3h8VZ7lZUzeAIoSKwQbTv+I=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: Content-Type:MIME-Version; b=JvPBjrA04bj7gvprb0296zWJPaVx3KSv0peZ3NlLwH6ZIy7lachzADRB+lgIdRDopXdWfMJl7JhAro0y1HUhtQ5KHtUOOpvPnt3g/OssorLTHFRsiRImbyc1t60Et9KOecgEhGOSw4HZL6OQJzCO9Z8BGvrBuQ3MSWYGv9JKpDI= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=oracle.com; spf=pass smtp.mailfrom=oracle.com; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b=KzA61X95; dkim=pass (1024-bit key) header.d=oracle.onmicrosoft.com header.i=@oracle.onmicrosoft.com header.b=mwYu1H9a; arc=fail smtp.client-ip=205.220.165.32 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=oracle.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=oracle.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b="KzA61X95"; dkim=pass (1024-bit key) header.d=oracle.onmicrosoft.com header.i=@oracle.onmicrosoft.com header.b="mwYu1H9a" Received: from pps.filterd (m0333521.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 5A8Ft8Hf009413; Sat, 8 Nov 2025 17:09:24 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=cc :content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s= corp-2025-04-25; bh=Lwvkms7GITNmCI+DeGWkYguhXaAQLuiGqXmPvm3p6YU=; b= KzA61X95mBhLV4YvM0/osNcXWS8m2Acg+uV5833zlyJTnQUk0WQ2Wbwl99Gn5ag0 9QV0kcBFYGFLEO/hnl2JPuxqtBplZHnLI8qyhzLr9r6+Io0X8GOSpw8/L6RODHqa 0vrbBw1n9HrgL/TkOL+9p88vT/eJQPqTIzBiU+6w575XVG15Cj9AD94lpzwOOhWE lZmbl6+27H09f3jAm7zGpuauiKi31nzDgn00aqLnaiclpbqrPIo28ta50ne4ujb4 Z2AXQ0Otmpr2H/Pfwtj5IvOJudrm8M/OufDNSs10ev15I7YdFzE2Ib9i6t4eq0ri RKqyJu7uZcb8dXRebxLosA== Received: from phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta01.appoci.oracle.com [138.1.114.2]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 4a9vs8gnmk-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Sat, 08 Nov 2025 17:09:24 +0000 (GMT) Received: from pps.filterd (phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (8.18.1.2/8.18.1.2) with ESMTP id 5A8C03c7007618; Sat, 8 Nov 2025 17:09:23 GMT Received: from sn4pr0501cu005.outbound.protection.outlook.com (mail-southcentralusazon11011047.outbound.protection.outlook.com [40.93.194.47]) by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTPS id 4a9va76n25-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Sat, 08 Nov 2025 17:09:23 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=YMtkn8PXsHw9For0ZQ1NkJf9j0GHK7wwZ57AzqWMazEiLIHhXsdxUByI2N87NF8SHSu0UjlMoCl1o0pzACZNJO9OG1CpZMhu76PU97CLnt6IMfUlNlfNZPDw1U/v2yQEahE0xn4aDUzFfdIbnjBjqeHtH771gtZSjGlqBNxKX2rj7aEloZzj/IzChe5QeGpdX2uoPRruyctDzoYWL6E56sDlArNxvohrZgCk58yMICU6OIn6bab+I8U5KX854RSXFaVyS0j9TwQKb+yKMmVSJUT/qvY6oTg5WUpkiKAg0SN2l+w0WhC71fMYW+9imkJE2Vpo182O0wz9Sco3zp/Yiw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=Lwvkms7GITNmCI+DeGWkYguhXaAQLuiGqXmPvm3p6YU=; b=pxfvWuYhEDrnWgWh2nRsiPZt/I0R44d3Yoqf+MUVcwEZdw7gjFHXnxz81ysix2WiTZsvDF0Mku8NY2SaZ+061SycByHKLKP0f6vDiI32E1MxqcpyKYqbC6BMnzZCtgzpgTy7jHh4+t22/MbMhxspnI2m2GAea+uWBq6ae7mxpp4CXXmsCz/M3OJyLepI6oa+Ugao65/H1ey2ke0ng93zCpgoy4WRZ5bpcXKqNEi4THbfuduvajRYgnTjbDVNpBFWcIUvTF1nsIjiwQK1vNXy+OhwISSsk0GBjM590yOpEM7zd1CZUPh3MVS0Gren79BINE80+KXAqkNDxAGmwvYBjQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Lwvkms7GITNmCI+DeGWkYguhXaAQLuiGqXmPvm3p6YU=; b=mwYu1H9aSk9gcfNEqy1bbrIu429BLsU8ONQBu3zeIOgjRrayDyttxN7+N33+HPoTmdkFiuHZrKaWPodQY9BAmxVw9X8Lb7xZPqdrwUMHrDwF+QyHPSwmBZYFckYizJiJdVsSFlJgOTK7l5fNmd8XB6ymsJV/8VjKAG30JhUytk8= Received: from DM4PR10MB8218.namprd10.prod.outlook.com (2603:10b6:8:1cc::16) by SJ5PPF04D2D7FA7.namprd10.prod.outlook.com (2603:10b6:a0f:fc02::785) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9298.15; Sat, 8 Nov 2025 17:09:18 +0000 Received: from DM4PR10MB8218.namprd10.prod.outlook.com ([fe80::2650:55cf:2816:5f2]) by DM4PR10MB8218.namprd10.prod.outlook.com ([fe80::2650:55cf:2816:5f2%7]) with mapi id 15.20.9298.010; Sat, 8 Nov 2025 17:09:18 +0000 From: Lorenzo Stoakes To: Andrew Morton Cc: Christian Borntraeger , Janosch Frank , Claudio Imbrenda , David Hildenbrand , Alexander Gordeev , Gerald Schaefer , Heiko Carstens , Vasily Gorbik , Sven Schnelle , Peter Xu , Alexander Viro , Christian Brauner , Jan Kara , Arnd Bergmann , Zi Yan , Baolin Wang , "Liam R . Howlett" , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Lance Yang , Muchun Song , Oscar Salvador , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Matthew Brost , Joshua Hahn , Rakie Kim , Byungchul Park , Gregory Price , Ying Huang , Alistair Popple , Axel Rasmussen , Yuanchu Xie , Wei Xu , Kemeng Shi , Kairui Song , Nhat Pham , Baoquan He , Chris Li , SeongJae Park , Matthew Wilcox , Jason Gunthorpe , Leon Romanovsky , Xu Xin , Chengming Zhou , Jann Horn , Miaohe Lin , Naoya Horiguchi , Pedro Falcato , Pasha Tatashin , Rik van Riel , Harry Yoo , Hugh Dickins , linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-s390@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, linux-arch@vger.kernel.org, damon@lists.linux.dev Subject: [PATCH v2 10/16] mm: replace pmd_to_swp_entry() with softleaf_from_pmd() Date: Sat, 8 Nov 2025 17:08:24 +0000 Message-ID: <093e438c240272d081548222900a5c3b205e9a5d.1762621568.git.lorenzo.stoakes@oracle.com> X-Mailer: git-send-email 2.51.0 In-Reply-To: References: Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: LO2P265CA0200.GBRP265.PROD.OUTLOOK.COM (2603:10a6:600:9e::20) To DM4PR10MB8218.namprd10.prod.outlook.com (2603:10b6:8:1cc::16) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DM4PR10MB8218:EE_|SJ5PPF04D2D7FA7:EE_ X-MS-Office365-Filtering-Correlation-Id: 7b007791-9e55-4fda-f5e8-08de1ee98d6f X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|1800799024|7416014|376014|366016; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?f8siK3gdiR+Oo62iYx2X2fS3HmkT7454cC2H3slMCBLnWfuPzTd+bNy3A3RX?= =?us-ascii?Q?NfVEJlzimq/V1FRHPoeA27jNAtMTGb/kY7Ygv8TPSyg5pGAyPGmQYnLQjH6L?= =?us-ascii?Q?4y52uBnkoSOb1RqZusz3ystCujbw7IL4pCw88pmkhypVjxBRCy8dpKH6PDLr?= =?us-ascii?Q?RIljLxatrI5sGu3cby5i9iryGki2/eiq1p+6YTlRoG1Sq6+FPs5h1ZbenCB/?= =?us-ascii?Q?Ggn0kS8NwpBTf5BqZzLURa2coN0XBEdcQti5E6YfJiU3cD4htqFGWEtYbCcu?= =?us-ascii?Q?1nM8DpBwHpL4CUQ8W2nAE+P9K+bghVoEuVyNnKQf8JGsy+bGxPrvk4icsx56?= =?us-ascii?Q?zlrdyBchep7F6dfaEkZrgMpvAgsOeBSx3iSo3zYMcONkKXA/p8Ei8P6fk0S7?= =?us-ascii?Q?28VJcZPtiSGB5TjNK1g45V7jo2AiJFBbf9RVn/9LR2aCdh2MyQg3ANgBTjGM?= =?us-ascii?Q?xMQssPLyOlqsoRdRtbTSUg2QELWl6jo3mHXu/3tpHJ5WbjCDASSlf+URcmb7?= =?us-ascii?Q?ZiCibDZjnoDRtkvF6ww2Rnj14sp1/3hoYDhh8Hxecs2NICtPKfd2NBRFx6gG?= =?us-ascii?Q?3ewzWSLxuXhgmkcJrsYrQWSz79X6e+v3YlvOf7Llz689+llz7yJeJ77SGuUr?= =?us-ascii?Q?qqD0P5+eLfblFSqAWSsjzXCRJ33VxWjvz3SchZZkHlS+W9epkdTb0idI91Bd?= =?us-ascii?Q?crtxRQiFcYUmpg1/M0UP3lFw2L2awUkPcnuTFNIkPQ9zB8lCplTK7kUnrSvE?= =?us-ascii?Q?PjHyZ8Aq7gE5Da9pRQPFXBEjMb8siM27lm6hnfNwfqBM6xSDcYjrgxsY1cHT?= =?us-ascii?Q?xZaWSvaqc9UU001GBwHBjvv5vSYmYLwLUDVACVh+SwWBJc/wxEVY4oDXK5Ji?= =?us-ascii?Q?IYV4cMqFF4vpbYVAEPO9qje/muKPAEKc+3YA1CeIpblUpokIMuFDJ4hhrcmt?= =?us-ascii?Q?PXELPRci8fzHdD6q7jq3eXmEXwBhfzWLkPCJ/lomzqD2jrQD7t4osHzQS0YC?= =?us-ascii?Q?bHHkwbj0p+L2p6Dpp6bNYKS3nfL+LtuPFV8ZZWBYoUjUdpTeLQsswLjurn6v?= =?us-ascii?Q?0z2Nr0BIbOatn9IqoVN4FBqt6wMjPRUP3X6RHbZfIiWdLGzxwivGvpgGYHHb?= =?us-ascii?Q?lwa1JmQvQ6BlWUPsG9BTOPoxidaTo1enbWieAuGW3MORVe29lE/tUH930t2f?= =?us-ascii?Q?gasuFPT18LHbJeK9TAPGCGI3rAak3/eQwxp6DS8pbUv5JpAOXhaY/WzyQKC5?= =?us-ascii?Q?SsVgDs7JLMRntd6k+juuGjOxU7UhqVpXZW8AGO/qWEQ929+3JzxkYwM1ZcMD?= =?us-ascii?Q?oxBdlw8LMSZhfjQ488MSq/1HovoA0XWW5I3jS4rZIhW6bj1pSFvnIlyW863y?= =?us-ascii?Q?LnFSdDMJBhlmQsYGLy9NLhmJqjzWWL0BN8L/87bmOIIds86y8WXFuejrcnxP?= =?us-ascii?Q?xbjXCBB2PIMYF+e5RVJH3+zSu46mHrLt?= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DM4PR10MB8218.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(1800799024)(7416014)(376014)(366016);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?DE3C4phIydjamWASC/HPaSJJ87WTgZ4SVQzyXewNOWreGktJumN/29tmtCNn?= =?us-ascii?Q?STj6OBS9xcvndXTF5miCznpDTRH5AT8Mo0u1zW8K7Okv8FotXhzJOrLRF9Bq?= =?us-ascii?Q?R/5YL1FT9e/sKwV6JKEeg0mqq3SF0C5kBTgd0pZG82qSo6tiCgk5UR4JXOS7?= =?us-ascii?Q?QeMNdz4CZ8yjTnJhWb8CUBacTXeFzEzNscH+a7kCzZArDcEjcv/KfLIEF3Pf?= =?us-ascii?Q?RuO4XHM4WGE5EHQDuxki4O+1oepcPx/hknXZc8kbSTubDUiDWNTwNt62pOWc?= =?us-ascii?Q?mN6ThviUZGYeXq5fJhQ5Wv03PkZ1zDVz4R2KOKV/YHTV0HpCYkg4ndstMSn/?= =?us-ascii?Q?K2yPoxcPE4CVBjmjVhgH5K18gfnpMsR0jCV1CIiLiNxdUYcdPiOKBQDjLQB6?= =?us-ascii?Q?EABh5xAX2OCrZgKb8j18G1mcNLQginZJzJY22XjuopB847nSEalnBNbz17m6?= =?us-ascii?Q?6D18E8w77BqCZ0DBEho1RnOy7A1Tff+e4ie1QT1t1nolxzPGfTAjtZmH62Yv?= =?us-ascii?Q?td/Q9p69xx+juUtF5yCZoVPBYZrF0E/JDn7S+irfNrN0RzMnxDTC3461ju2g?= =?us-ascii?Q?Cz/U/X6OC9p/ajYMQZ9t/PWNH+yH1IhU5L7aglF1S8a8Usi//uBsyPDH+H7u?= =?us-ascii?Q?GqNL+pHyKr91RQJed9lRIgoe/FWo3T2XJ3nk2nRRTlfEb2TXEsynJBAEuYfv?= =?us-ascii?Q?SwGaH2RVJSVOxK1XJ/5fcRHSiglqY8gCxLmV7LvsPJ2g0PJ362zSoxC8kury?= =?us-ascii?Q?7pxxQwpp6DbqfqNzYMQ/typICvyCKo1xP3HTU9Y0Shcdlb1zV0Zk31gnrX+P?= =?us-ascii?Q?nTCi/YX4YitOPxNqDKauSLReuL5zCqbGRt9XiVQx43G6OtW0A+mpKsme8Tox?= =?us-ascii?Q?4FoSmhdzAPHUjfCqoRAQZM5rpy2DUIGB+ybOs8ltJ+sg7E0amOPuVfck/RKc?= =?us-ascii?Q?8p23CXkg233/7jlR4kE8/7DcF/O8JI6Ks/AwkjWiQr5AlJXeumSU41PsEcLq?= =?us-ascii?Q?oTPsobv5gMm42LCIpbobVj6G/ASJmRb1lBeAOUqVI8YYWg7stkGI4a++iXhP?= =?us-ascii?Q?1yv6X+5SNXkx4hAgvrCn6b5Xj72Y/FOymY54fkhw+7WiDfUuWeErKqAlp46x?= =?us-ascii?Q?o71Q/SIcs1m6Nk0npU+Ic5bW2QaPUJviGFA23eYwpXUxkLTDQpjtkLeX/X3O?= =?us-ascii?Q?u1GTSlLVwZZbi3GoFcR+WH5RRgtqIDSrYXdggWQ6XNg+MKqU9cOnXzrAN5KU?= =?us-ascii?Q?NDI/YAkw0lwo8n8lLe9EksCo4PLzy+tlUkHHrkKZr7nZcTdUEIbIl7ETSfRt?= =?us-ascii?Q?JV3BsMN/qqMsWgw2UXXcUT/qZrktxo7SSv3LoWGtNKY26GnZpn8JFS06lYBK?= =?us-ascii?Q?MYoj5Joa4gSQip9qh182GMXvQRKq8CBbhrmmJ7u6/Y2998hXUjWr3CTBykzS?= =?us-ascii?Q?U07TRGN9KP1zxQFVLpHvRNAcZXV7GFwdrnDEF1BE8Bk4eKyT0vHbuK610uoD?= =?us-ascii?Q?zuJkS/TtMsUv/kgsikRuuBpuWnbJCqFHKc5UB0mZfSS+7c05LCvlTuRrg+Dc?= =?us-ascii?Q?IIk+FP5Fy8hAn1+NNQMrVxi9/P7J+yR7TYxm/68NQ1av1GOp9643vdTLoFZ7?= =?us-ascii?Q?1w=3D=3D?= X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: 16g33gYAMmvfrTV36gSCr/IrbmOhOKBpH1kAhhGVPD04TRxwOymroHzBygaeZ8KEXXq/JpRABiuwlVztDI6an47u3WQE6LOIBRI5dM+KcZ0J+srHpnrgYfmp5Sxyl99xejQetq3Jdo98yV6LIXV0I4PxfLD0zqR211cmKVhtMkaWh6l+vxO75VB+c7w+V5uErfFOfaWh1o5e85l0BSi55iU2vqiQM3UXbnvcEbS4OD5vH79vrDyEkcyv2PoqKCP2ubt2vC2un0u3dpEv1nG9Z+HWkIf2iCjY1xjvLrHfEDmPWgccmqb6xFWehJ7irjspV0KxUemo3zLM0AV9meySaY4CX1hX0AgdyxFB+iMlS7wBfHifNpGfai+Bx0oqJXElaGe9gM/pKSKSJBc8RB7ME1Tsyo41PHYiuFjauBgnA1xP4p4zRXci9Bi9mT0s96/i0CVxlrKZCAT9a+7cH6exI9fHJVVfb6RNPYalA87n00qoID+buuDKO6+5cwb1ELRzuJM0Ko512HOC1trRAQfIXczPwIVd8O0j8UEG6ar4HocVWomuBPJJRzNPxqwhU1xeZU7E4NLRP12G2gQYfF2uQ5rPTt1FCrzQFGneZ3+WITc= X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: 7b007791-9e55-4fda-f5e8-08de1ee98d6f X-MS-Exchange-CrossTenant-AuthSource: DM4PR10MB8218.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 Nov 2025 17:09:18.4178 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: cXA98lfSoNst71BQcO6nf8Q/dAYZPzglEvLNNiqD4YxE2RAYkJ1CwznvU/pkCxaIB6bkkIjGqPW/7n18Lpynm0tOs6jgiC0V8/XVqvSCxPc= X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ5PPF04D2D7FA7 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1121,Hydra:6.1.9,FMLib:17.12.100.49 definitions=2025-11-08_04,2025-11-06_01,2025-10-01_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 adultscore=0 malwarescore=0 spamscore=0 mlxscore=0 bulkscore=0 suspectscore=0 mlxlogscore=999 phishscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2510240000 definitions=main-2511080138 X-Proofpoint-Spam-Details-Enc: AW1haW4tMjUxMTA4MDAxNiBTYWx0ZWRfXylQr04b6364K qGwnEeXv6kNj0EnnJwMLyxdwTkh4uu7Ggo+pGON0b5W+btykXDJiJKjgi19ZLf6IjEuDoCnRDmr rSJygRhdByshg9Us2AdGlYNy+SCGjueTTDcdoU3E2272pLmUiLdscJALFGCkOhguJ8dpaJagLO+ AhKCCBD9BsLtZd4MaG535Clg28W3nwLXv2Zzqfnrc0EVhvWqkhkycCa5ihOCtCLOqPY2rGL2MFE avSQCldJ5CIsm7dy0/xCzgsxlkKZO/u+N3znDysR7AWYlBNW8aFoaljJILeqaf1nqpQ1KwfBG5h WaidaOrKxOPSwvJZIwDkAM95j87NljJQRFwJm797896Oa+srgk+FBiCRAy1UMYkWJVrVUm/W8D7 nWwH/O//atrh71lm3yljVIhjfKTxMw== X-Proofpoint-ORIG-GUID: t3BeLVaUEPjV95Z-AxSxuCIaWfjPhPX0 X-Proofpoint-GUID: t3BeLVaUEPjV95Z-AxSxuCIaWfjPhPX0 X-Authority-Analysis: v=2.4 cv=eYgwvrEH c=1 sm=1 tr=0 ts=690f7944 cx=c_pps a=XiAAW1AwiKB2Y8Wsi+sD2Q==:117 a=XiAAW1AwiKB2Y8Wsi+sD2Q==:17 a=6eWqkTHjU83fiwn7nKZWdM+Sl24=:19 a=z/mQ4Ysz8XfWz/Q5cLBRGdckG28=:19 a=lCpzRmAYbLLaTzLvsPZ7Mbvzbb8=:19 a=xqWC_Br6kY4A:10 a=6UeiqGixMTsA:10 a=GoEa3M9JfhUA:10 a=VkNPw1HP01LnGYTKEx00:22 a=yPCof4ZbAAAA:8 a=1B2o6Llj8-7n3TOjnuQA:9 a=UhEZJTgQB8St2RibIkdl:22 a=Z5ABNNGmrOfJ6cZ5bIyy:22 a=QOGEsqRv6VhmHaoFNykA:22 Content-Type: text/plain; charset="utf-8" Introduce softleaf_from_pmd() to do the equivalent operation for PMDs that softleaf_from_pte() fulfils, and cascade changes through code base accordingly, introducing helpers as necessary. We are then able to eliminate pmd_to_swp_entry(), is_pmd_migration_entry(), is_pmd_device_private_entry() and is_pmd_non_present_folio_entry(). This further establishes the use of leaf operations throughout the code base and further establishes the foundations for eliminating is_swap_pmd(). No functional change intended. Signed-off-by: Lorenzo Stoakes Reviewed-by: SeongJae Park --- fs/proc/task_mmu.c | 27 +++-- include/linux/leafops.h | 220 ++++++++++++++++++++++++++++++++++++++++ include/linux/migrate.h | 2 +- include/linux/swapops.h | 100 ------------------ mm/damon/ops-common.c | 6 +- mm/filemap.c | 6 +- mm/hmm.c | 16 +-- mm/huge_memory.c | 98 +++++++++--------- mm/khugepaged.c | 4 +- mm/madvise.c | 2 +- mm/memory.c | 4 +- mm/mempolicy.c | 4 +- mm/migrate.c | 20 ++-- mm/migrate_device.c | 14 +-- mm/page_table_check.c | 16 +-- mm/page_vma_mapped.c | 15 +-- mm/pagewalk.c | 8 +- mm/rmap.c | 4 +- 18 files changed, 343 insertions(+), 223 deletions(-) diff --git a/fs/proc/task_mmu.c b/fs/proc/task_mmu.c index b68eabb26f29..d982fdfcf057 100644 --- a/fs/proc/task_mmu.c +++ b/fs/proc/task_mmu.c @@ -1065,10 +1065,10 @@ static void smaps_pmd_entry(pmd_t *pmd, unsigned lo= ng addr, page =3D vm_normal_page_pmd(vma, addr, *pmd); present =3D true; } else if (unlikely(thp_migration_supported())) { - swp_entry_t entry =3D pmd_to_swp_entry(*pmd); + const softleaf_t entry =3D softleaf_from_pmd(*pmd); =20 - if (is_pfn_swap_entry(entry)) - page =3D pfn_swap_entry_to_page(entry); + if (softleaf_has_pfn(entry)) + page =3D softleaf_to_page(entry); } if (IS_ERR_OR_NULL(page)) return; @@ -1654,7 +1654,7 @@ static inline void clear_soft_dirty_pmd(struct vm_are= a_struct *vma, pmd =3D pmd_clear_soft_dirty(pmd); =20 set_pmd_at(vma->vm_mm, addr, pmdp, pmd); - } else if (is_migration_entry(pmd_to_swp_entry(pmd))) { + } else if (pmd_is_migration_entry(pmd)) { pmd =3D pmd_swp_clear_soft_dirty(pmd); set_pmd_at(vma->vm_mm, addr, pmdp, pmd); } @@ -2015,12 +2015,12 @@ static int pagemap_pmd_range_thp(pmd_t *pmdp, unsig= ned long addr, if (pm->show_pfn) frame =3D pmd_pfn(pmd) + idx; } else if (thp_migration_supported()) { - swp_entry_t entry =3D pmd_to_swp_entry(pmd); + const softleaf_t entry =3D softleaf_from_pmd(pmd); unsigned long offset; =20 if (pm->show_pfn) { - if (is_pfn_swap_entry(entry)) - offset =3D swp_offset_pfn(entry) + idx; + if (softleaf_has_pfn(entry)) + offset =3D softleaf_to_pfn(entry) + idx; else offset =3D swp_offset(entry) + idx; frame =3D swp_type(entry) | @@ -2031,7 +2031,7 @@ static int pagemap_pmd_range_thp(pmd_t *pmdp, unsigne= d long addr, flags |=3D PM_SOFT_DIRTY; if (pmd_swp_uffd_wp(pmd)) flags |=3D PM_UFFD_WP; - VM_WARN_ON_ONCE(!is_pmd_migration_entry(pmd)); + VM_WARN_ON_ONCE(!pmd_is_migration_entry(pmd)); page =3D pfn_swap_entry_to_page(entry); } =20 @@ -2425,8 +2425,6 @@ static unsigned long pagemap_thp_category(struct page= map_scan_private *p, if (pmd_soft_dirty(pmd)) categories |=3D PAGE_IS_SOFT_DIRTY; } else { - swp_entry_t swp; - categories |=3D PAGE_IS_SWAPPED; if (!pmd_swp_uffd_wp(pmd)) categories |=3D PAGE_IS_WRITTEN; @@ -2434,9 +2432,10 @@ static unsigned long pagemap_thp_category(struct pag= emap_scan_private *p, categories |=3D PAGE_IS_SOFT_DIRTY; =20 if (p->masks_of_interest & PAGE_IS_FILE) { - swp =3D pmd_to_swp_entry(pmd); - if (is_pfn_swap_entry(swp) && - !folio_test_anon(pfn_swap_entry_folio(swp))) + const softleaf_t entry =3D softleaf_from_pmd(pmd); + + if (softleaf_has_pfn(entry) && + !folio_test_anon(softleaf_to_folio(entry))) categories |=3D PAGE_IS_FILE; } } @@ -2453,7 +2452,7 @@ static void make_uffd_wp_pmd(struct vm_area_struct *v= ma, old =3D pmdp_invalidate_ad(vma, addr, pmdp); pmd =3D pmd_mkuffd_wp(old); set_pmd_at(vma->vm_mm, addr, pmdp, pmd); - } else if (is_migration_entry(pmd_to_swp_entry(pmd))) { + } else if (pmd_is_migration_entry(pmd)) { pmd =3D pmd_swp_mkuffd_wp(pmd); set_pmd_at(vma->vm_mm, addr, pmdp, pmd); } diff --git a/include/linux/leafops.h b/include/linux/leafops.h index 1376589d94b0..9be9a4e8ada4 100644 --- a/include/linux/leafops.h +++ b/include/linux/leafops.h @@ -61,6 +61,57 @@ static inline softleaf_t softleaf_from_pte(pte_t pte) return pte_to_swp_entry(pte); } =20 +/** + * softleaf_to_pte() - Obtain a PTE entry from a leaf entry. + * @entry: Leaf entry. + * + * This generates an architecture-specific PTE entry that can be utilised = to + * encode the metadata the leaf entry encodes. + * + * Returns: Architecture-specific PTE entry encoding leaf entry. + */ +static inline pte_t softleaf_to_pte(softleaf_t entry) +{ + /* Temporary until swp_entry_t eliminated. */ + return swp_entry_to_pte(entry); +} + +#ifdef CONFIG_ARCH_ENABLE_THP_MIGRATION +/** + * softleaf_from_pmd() - Obtain a leaf entry from a PMD entry. + * @pmd: PMD entry. + * + * If @pmd is present (therefore not a leaf entry) the function returns an= empty + * leaf entry. Otherwise, it returns a leaf entry. + * + * Returns: Leaf entry. + */ +static inline softleaf_t softleaf_from_pmd(pmd_t pmd) +{ + softleaf_t arch_entry; + + if (pmd_present(pmd)) + return softleaf_mk_none(); + + if (pmd_swp_soft_dirty(pmd)) + pmd =3D pmd_swp_clear_soft_dirty(pmd); + if (pmd_swp_uffd_wp(pmd)) + pmd =3D pmd_swp_clear_uffd_wp(pmd); + arch_entry =3D __pmd_to_swp_entry(pmd); + + /* Temporary until swp_entry_t eliminated. */ + return swp_entry(__swp_type(arch_entry), __swp_offset(arch_entry)); +} + +#else + +static inline softleaf_t softleaf_from_pmd(pmd_t pmd) +{ + return softleaf_mk_none(); +} + +#endif + /** * softleaf_is_none() - Is the leaf entry empty? * @entry: Leaf entry. @@ -134,6 +185,43 @@ static inline bool softleaf_is_swap(softleaf_t entry) return softleaf_type(entry) =3D=3D SOFTLEAF_SWAP; } =20 +/** + * softleaf_is_migration_write() - Is this leaf entry a writable migration= entry? + * @entry: Leaf entry. + * + * Returns: true if the leaf entry is a writable migration entry, otherwise + * false. + */ +static inline bool softleaf_is_migration_write(softleaf_t entry) +{ + return softleaf_type(entry) =3D=3D SOFTLEAF_MIGRATION_WRITE; +} + +/** + * softleaf_is_migration_read() - Is this leaf entry a readable migration = entry? + * @entry: Leaf entry. + * + * Returns: true if the leaf entry is a readable migration entry, otherwise + * false. + */ +static inline bool softleaf_is_migration_read(softleaf_t entry) +{ + return softleaf_type(entry) =3D=3D SOFTLEAF_MIGRATION_READ; +} + +/** + * softleaf_is_migration_read_exclusive() - Is this leaf entry an exclusive + * readable migration entry? + * @entry: Leaf entry. + * + * Returns: true if the leaf entry is an exclusive readable migration entr= y, + * otherwise false. + */ +static inline bool softleaf_is_migration_read_exclusive(softleaf_t entry) +{ + return softleaf_type(entry) =3D=3D SOFTLEAF_MIGRATION_READ_EXCLUSIVE; +} + /** * softleaf_is_swap() - Is this leaf entry a migration entry? * @entry: Leaf entry. @@ -152,6 +240,19 @@ static inline bool softleaf_is_migration(softleaf_t en= try) } } =20 +/** + * softleaf_is_device_private_write() - Is this leaf entry a device private + * writable entry? + * @entry: Leaf entry. + * + * Returns: true if the leaf entry is a device private writable entry, oth= erwise + * false. + */ +static inline bool softleaf_is_device_private_write(softleaf_t entry) +{ + return softleaf_type(entry) =3D=3D SOFTLEAF_DEVICE_PRIVATE_WRITE; +} + /** * softleaf_is_device_private() - Is this leaf entry a device private entr= y? * @entry: Leaf entry. @@ -169,6 +270,12 @@ static inline bool softleaf_is_device_private(softleaf= _t entry) } } =20 +/** + * softleaf_is_device_exclusive() - Is this leaf entry a device-exclusive = entry? + * @entry: Leaf entry. + * + * Returns: true if the leaf entry is a device-exclusive entry, otherwise = false. + */ static inline bool softleaf_is_device_exclusive(softleaf_t entry) { return softleaf_type(entry) =3D=3D SOFTLEAF_DEVICE_EXCLUSIVE; @@ -327,6 +434,61 @@ static inline bool softleaf_is_uffd_wp_marker(softleaf= _t entry) return softleaf_to_marker(entry) & PTE_MARKER_UFFD_WP; } =20 +#ifdef CONFIG_MIGRATION + +/** + * softleaf_is_migration_young() - Does this migration entry contain an ac= cessed + * bit? + * @entry: Leaf entry. + * + * If the architecture can support storing A/D bits in migration entries, = this + * determines whether the accessed (or 'young') bit was set on the migrate= d page + * table entry. + * + * Returns: true if the entry contains an accessed bit, otherwise false. + */ +static inline bool softleaf_is_migration_young(softleaf_t entry) +{ + VM_WARN_ON_ONCE(!softleaf_is_migration(entry)); + + if (migration_entry_supports_ad()) + return swp_offset(entry) & SWP_MIG_YOUNG; + /* Keep the old behavior of aging page after migration */ + return false; +} + +/** + * softleaf_is_migration_dirty() - Does this migration entry contain a dir= ty bit? + * @entry: Leaf entry. + * + * If the architecture can support storing A/D bits in migration entries, = this + * determines whether the dirty bit was set on the migrated page table ent= ry. + * + * Returns: true if the entry contains a dirty bit, otherwise false. + */ +static inline bool softleaf_is_migration_dirty(softleaf_t entry) +{ + VM_WARN_ON_ONCE(!softleaf_is_migration(entry)); + + if (migration_entry_supports_ad()) + return swp_offset(entry) & SWP_MIG_DIRTY; + /* Keep the old behavior of clean page after migration */ + return false; +} + +#else /* CONFIG_MIGRATION */ + +static inline bool softleaf_is_migration_young(softleaf_t entry) +{ + return false; +} + +static inline bool softleaf_is_migration_dirty(softleaf_t entry) +{ + return false; +} +#endif /* CONFIG_MIGRATION */ + /** * pte_is_marker() - Does the PTE entry encode a marker leaf entry? * @pte: PTE entry. @@ -378,5 +540,63 @@ static inline bool pte_is_uffd_marker(pte_t pte) return false; } =20 +#if defined(CONFIG_ZONE_DEVICE) && defined(CONFIG_ARCH_ENABLE_THP_MIGRATIO= N) + +/** + * pmd_is_device_private_entry() - Check if PMD contains a device private = swap + * entry. + * @pmd: The PMD to check. + * + * Returns true if the PMD contains a swap entry that represents a device = private + * page mapping. This is used for zone device private pages that have been + * swapped out but still need special handling during various memory manag= ement + * operations. + * + * Return: true if PMD contains device private entry, false otherwise + */ +static inline bool pmd_is_device_private_entry(pmd_t pmd) +{ + return softleaf_is_device_private(softleaf_from_pmd(pmd)); +} + +#else /* CONFIG_ZONE_DEVICE && CONFIG_ARCH_ENABLE_THP_MIGRATION */ + +static inline bool pmd_is_device_private_entry(pmd_t pmd) +{ + return false; +} + +#endif /* CONFIG_ZONE_DEVICE && CONFIG_ARCH_ENABLE_THP_MIGRATION */ + +/** + * pmd_is_migration_entry() - Does this PMD entry encode a migration entry? + * @pmd: PMD entry. + * + * Returns: true if the PMD encodes a migration entry, otherwise false. + */ +static inline bool pmd_is_migration_entry(pmd_t pmd) +{ + return softleaf_is_migration(softleaf_from_pmd(pmd)); +} + +/** + * pmd_is_valid_softleaf() - Is this PMD entry a valid leaf entry? + * @pmd: PMD entry. + * + * PMD leaf entries are valid only if they are device private or migration + * entries. This function asserts that a PMD leaf entry is valid in this + * respect. + * + * Returns: true if the PMD entry is a valid leaf entry, otherwise false. + */ +static inline bool pmd_is_valid_softleaf(pmd_t pmd) +{ + const softleaf_t entry =3D softleaf_from_pmd(pmd); + + /* Only device private, migration entries valid for PMD. */ + return softleaf_is_device_private(entry) || + softleaf_is_migration(entry); +} + #endif /* CONFIG_MMU */ #endif /* _LINUX_SWAPOPS_H */ diff --git a/include/linux/migrate.h b/include/linux/migrate.h index 41b4cc05a450..26ca00c325d9 100644 --- a/include/linux/migrate.h +++ b/include/linux/migrate.h @@ -65,7 +65,7 @@ bool isolate_folio_to_list(struct folio *folio, struct li= st_head *list); =20 int migrate_huge_page_move_mapping(struct address_space *mapping, struct folio *dst, struct folio *src); -void migration_entry_wait_on_locked(swp_entry_t entry, spinlock_t *ptl) +void migration_entry_wait_on_locked(softleaf_t entry, spinlock_t *ptl) __releases(ptl); void folio_migrate_flags(struct folio *newfolio, struct folio *folio); int folio_migrate_mapping(struct address_space *mapping, diff --git a/include/linux/swapops.h b/include/linux/swapops.h index 3e8dd6ea94dd..f1277647262d 100644 --- a/include/linux/swapops.h +++ b/include/linux/swapops.h @@ -283,14 +283,6 @@ static inline swp_entry_t make_migration_entry_young(s= wp_entry_t entry) return entry; } =20 -static inline bool is_migration_entry_young(swp_entry_t entry) -{ - if (migration_entry_supports_ad()) - return swp_offset(entry) & SWP_MIG_YOUNG; - /* Keep the old behavior of aging page after migration */ - return false; -} - static inline swp_entry_t make_migration_entry_dirty(swp_entry_t entry) { if (migration_entry_supports_ad()) @@ -299,14 +291,6 @@ static inline swp_entry_t make_migration_entry_dirty(s= wp_entry_t entry) return entry; } =20 -static inline bool is_migration_entry_dirty(swp_entry_t entry) -{ - if (migration_entry_supports_ad()) - return swp_offset(entry) & SWP_MIG_DIRTY; - /* Keep the old behavior of clean page after migration */ - return false; -} - extern void migration_entry_wait(struct mm_struct *mm, pmd_t *pmd, unsigned long address); extern void migration_entry_wait_huge(struct vm_area_struct *vma, unsigned= long addr, pte_t *pte); @@ -349,20 +333,11 @@ static inline swp_entry_t make_migration_entry_young(= swp_entry_t entry) return entry; } =20 -static inline bool is_migration_entry_young(swp_entry_t entry) -{ - return false; -} - static inline swp_entry_t make_migration_entry_dirty(swp_entry_t entry) { return entry; } =20 -static inline bool is_migration_entry_dirty(swp_entry_t entry) -{ - return false; -} #endif /* CONFIG_MIGRATION */ =20 #ifdef CONFIG_MEMORY_FAILURE @@ -487,18 +462,6 @@ extern void remove_migration_pmd(struct page_vma_mappe= d_walk *pvmw, =20 extern void pmd_migration_entry_wait(struct mm_struct *mm, pmd_t *pmd); =20 -static inline swp_entry_t pmd_to_swp_entry(pmd_t pmd) -{ - swp_entry_t arch_entry; - - if (pmd_swp_soft_dirty(pmd)) - pmd =3D pmd_swp_clear_soft_dirty(pmd); - if (pmd_swp_uffd_wp(pmd)) - pmd =3D pmd_swp_clear_uffd_wp(pmd); - arch_entry =3D __pmd_to_swp_entry(pmd); - return swp_entry(__swp_type(arch_entry), __swp_offset(arch_entry)); -} - static inline pmd_t swp_entry_to_pmd(swp_entry_t entry) { swp_entry_t arch_entry; @@ -507,23 +470,7 @@ static inline pmd_t swp_entry_to_pmd(swp_entry_t entry) return __swp_entry_to_pmd(arch_entry); } =20 -static inline int is_pmd_migration_entry(pmd_t pmd) -{ - swp_entry_t entry; - - if (pmd_present(pmd)) - return 0; - - entry =3D pmd_to_swp_entry(pmd); - return is_migration_entry(entry); -} #else /* CONFIG_ARCH_ENABLE_THP_MIGRATION */ -static inline int set_pmd_migration_entry(struct page_vma_mapped_walk *pvm= w, - struct page *page) -{ - BUILD_BUG(); -} - static inline void remove_migration_pmd(struct page_vma_mapped_walk *pvmw, struct page *new) { @@ -532,64 +479,17 @@ static inline void remove_migration_pmd(struct page_v= ma_mapped_walk *pvmw, =20 static inline void pmd_migration_entry_wait(struct mm_struct *m, pmd_t *p)= { } =20 -static inline swp_entry_t pmd_to_swp_entry(pmd_t pmd) -{ - return swp_entry(0, 0); -} - static inline pmd_t swp_entry_to_pmd(swp_entry_t entry) { return __pmd(0); } =20 -static inline int is_pmd_migration_entry(pmd_t pmd) -{ - return 0; -} #endif /* CONFIG_ARCH_ENABLE_THP_MIGRATION */ =20 -#if defined(CONFIG_ZONE_DEVICE) && defined(CONFIG_ARCH_ENABLE_THP_MIGRATIO= N) - -/** - * is_pmd_device_private_entry() - Check if PMD contains a device private = swap entry - * @pmd: The PMD to check - * - * Returns true if the PMD contains a swap entry that represents a device = private - * page mapping. This is used for zone device private pages that have been - * swapped out but still need special handling during various memory manag= ement - * operations. - * - * Return: 1 if PMD contains device private entry, 0 otherwise - */ -static inline int is_pmd_device_private_entry(pmd_t pmd) -{ - swp_entry_t entry; - - if (pmd_present(pmd)) - return 0; - - entry =3D pmd_to_swp_entry(pmd); - return is_device_private_entry(entry); -} - -#else /* CONFIG_ZONE_DEVICE && CONFIG_ARCH_ENABLE_THP_MIGRATION */ - -static inline int is_pmd_device_private_entry(pmd_t pmd) -{ - return 0; -} - -#endif /* CONFIG_ZONE_DEVICE && CONFIG_ARCH_ENABLE_THP_MIGRATION */ - static inline int non_swap_entry(swp_entry_t entry) { return swp_type(entry) >=3D MAX_SWAPFILES; } =20 -static inline int is_pmd_non_present_folio_entry(pmd_t pmd) -{ - return is_pmd_migration_entry(pmd) || is_pmd_device_private_entry(pmd); -} - #endif /* CONFIG_MMU */ #endif /* _LINUX_SWAPOPS_H */ diff --git a/mm/damon/ops-common.c b/mm/damon/ops-common.c index 971df8a16ba4..a218d9922234 100644 --- a/mm/damon/ops-common.c +++ b/mm/damon/ops-common.c @@ -11,7 +11,7 @@ #include #include #include -#include +#include =20 #include "../internal.h" #include "ops-common.h" @@ -51,7 +51,7 @@ void damon_ptep_mkold(pte_t *pte, struct vm_area_struct *= vma, unsigned long addr if (likely(pte_present(pteval))) pfn =3D pte_pfn(pteval); else - pfn =3D swp_offset_pfn(pte_to_swp_entry(pteval)); + pfn =3D softleaf_to_pfn(softleaf_from_pte(pteval)); =20 folio =3D damon_get_folio(pfn); if (!folio) @@ -83,7 +83,7 @@ void damon_pmdp_mkold(pmd_t *pmd, struct vm_area_struct *= vma, unsigned long addr if (likely(pmd_present(pmdval))) pfn =3D pmd_pfn(pmdval); else - pfn =3D swp_offset_pfn(pmd_to_swp_entry(pmdval)); + pfn =3D softleaf_to_pfn(softleaf_from_pmd(pmdval)); =20 folio =3D damon_get_folio(pfn); if (!folio) diff --git a/mm/filemap.c b/mm/filemap.c index ff75bd89b68c..950d93885e38 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -21,7 +21,7 @@ #include #include #include -#include +#include #include #include #include @@ -1402,7 +1402,7 @@ static inline int folio_wait_bit_common(struct folio = *folio, int bit_nr, * This follows the same logic as folio_wait_bit_common() so see the comme= nts * there. */ -void migration_entry_wait_on_locked(swp_entry_t entry, spinlock_t *ptl) +void migration_entry_wait_on_locked(softleaf_t entry, spinlock_t *ptl) __releases(ptl) { struct wait_page_queue wait_page; @@ -1411,7 +1411,7 @@ void migration_entry_wait_on_locked(swp_entry_t entry= , spinlock_t *ptl) unsigned long pflags; bool in_thrashing; wait_queue_head_t *q; - struct folio *folio =3D pfn_swap_entry_folio(entry); + struct folio *folio =3D softleaf_to_folio(entry); =20 q =3D folio_waitqueue(folio); if (!folio_test_uptodate(folio) && folio_test_workingset(folio)) { diff --git a/mm/hmm.c b/mm/hmm.c index b11b4ebba945..bc3fa699a4c6 100644 --- a/mm/hmm.c +++ b/mm/hmm.c @@ -18,7 +18,7 @@ #include #include #include -#include +#include #include #include #include @@ -334,19 +334,19 @@ static int hmm_vma_handle_absent_pmd(struct mm_walk *= walk, unsigned long start, struct hmm_vma_walk *hmm_vma_walk =3D walk->private; struct hmm_range *range =3D hmm_vma_walk->range; unsigned long npages =3D (end - start) >> PAGE_SHIFT; + const softleaf_t entry =3D softleaf_from_pmd(pmd); unsigned long addr =3D start; - swp_entry_t entry =3D pmd_to_swp_entry(pmd); unsigned int required_fault; =20 - if (is_device_private_entry(entry) && - pfn_swap_entry_folio(entry)->pgmap->owner =3D=3D + if (softleaf_is_device_private(entry) && + softleaf_to_folio(entry)->pgmap->owner =3D=3D range->dev_private_owner) { unsigned long cpu_flags =3D HMM_PFN_VALID | hmm_pfn_flags_order(PMD_SHIFT - PAGE_SHIFT); - unsigned long pfn =3D swp_offset_pfn(entry); + unsigned long pfn =3D softleaf_to_pfn(entry); unsigned long i; =20 - if (is_writable_device_private_entry(entry)) + if (softleaf_is_device_private_write(entry)) cpu_flags |=3D HMM_PFN_WRITE; =20 /* @@ -365,7 +365,7 @@ static int hmm_vma_handle_absent_pmd(struct mm_walk *wa= lk, unsigned long start, required_fault =3D hmm_range_need_fault(hmm_vma_walk, hmm_pfns, npages, 0); if (required_fault) { - if (is_device_private_entry(entry)) + if (softleaf_is_device_private(entry)) return hmm_vma_fault(addr, end, required_fault, walk); else return -EFAULT; @@ -407,7 +407,7 @@ static int hmm_vma_walk_pmd(pmd_t *pmdp, if (pmd_none(pmd)) return hmm_vma_walk_hole(start, end, -1, walk); =20 - if (thp_migration_supported() && is_pmd_migration_entry(pmd)) { + if (thp_migration_supported() && pmd_is_migration_entry(pmd)) { if (hmm_range_need_fault(hmm_vma_walk, hmm_pfns, npages, 0)) { hmm_vma_walk->last =3D addr; pmd_migration_entry_wait(walk->mm, pmdp); diff --git a/mm/huge_memory.c b/mm/huge_memory.c index 40a8a2c1e080..5876595b00d5 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -1374,7 +1374,7 @@ vm_fault_t do_huge_pmd_device_private(struct vm_fault= *vmf) struct vm_area_struct *vma =3D vmf->vma; vm_fault_t ret =3D 0; spinlock_t *ptl; - swp_entry_t swp_entry; + softleaf_t entry; struct page *page; struct folio *folio; =20 @@ -1389,8 +1389,8 @@ vm_fault_t do_huge_pmd_device_private(struct vm_fault= *vmf) return 0; } =20 - swp_entry =3D pmd_to_swp_entry(vmf->orig_pmd); - page =3D pfn_swap_entry_to_page(swp_entry); + entry =3D softleaf_from_pmd(vmf->orig_pmd); + page =3D softleaf_to_page(entry); folio =3D page_folio(page); vmf->page =3D page; vmf->pte =3D NULL; @@ -1780,13 +1780,13 @@ static void copy_huge_non_present_pmd( struct vm_area_struct *dst_vma, struct vm_area_struct *src_vma, pmd_t pmd, pgtable_t pgtable) { - swp_entry_t entry =3D pmd_to_swp_entry(pmd); + softleaf_t entry =3D softleaf_from_pmd(pmd); struct folio *src_folio; =20 - VM_WARN_ON(!is_pmd_non_present_folio_entry(pmd)); + VM_WARN_ON_ONCE(!pmd_is_valid_softleaf(pmd)); =20 - if (is_writable_migration_entry(entry) || - is_readable_exclusive_migration_entry(entry)) { + if (softleaf_is_migration_write(entry) || + softleaf_is_migration_read_exclusive(entry)) { entry =3D make_readable_migration_entry(swp_offset(entry)); pmd =3D swp_entry_to_pmd(entry); if (pmd_swp_soft_dirty(*src_pmd)) @@ -1794,12 +1794,12 @@ static void copy_huge_non_present_pmd( if (pmd_swp_uffd_wp(*src_pmd)) pmd =3D pmd_swp_mkuffd_wp(pmd); set_pmd_at(src_mm, addr, src_pmd, pmd); - } else if (is_device_private_entry(entry)) { + } else if (softleaf_is_device_private(entry)) { /* * For device private entries, since there are no * read exclusive entries, writable =3D !readable */ - if (is_writable_device_private_entry(entry)) { + if (softleaf_is_device_private_write(entry)) { entry =3D make_readable_device_private_entry(swp_offset(entry)); pmd =3D swp_entry_to_pmd(entry); =20 @@ -1810,7 +1810,7 @@ static void copy_huge_non_present_pmd( set_pmd_at(src_mm, addr, src_pmd, pmd); } =20 - src_folio =3D pfn_swap_entry_folio(entry); + src_folio =3D softleaf_to_folio(entry); VM_WARN_ON(!folio_test_large(src_folio)); =20 folio_get(src_folio); @@ -2270,7 +2270,7 @@ bool madvise_free_huge_pmd(struct mmu_gather *tlb, st= ruct vm_area_struct *vma, =20 if (unlikely(!pmd_present(orig_pmd))) { VM_BUG_ON(thp_migration_supported() && - !is_pmd_migration_entry(orig_pmd)); + !pmd_is_migration_entry(orig_pmd)); goto out; } =20 @@ -2368,11 +2368,10 @@ int zap_huge_pmd(struct mmu_gather *tlb, struct vm_= area_struct *vma, folio_remove_rmap_pmd(folio, page, vma); WARN_ON_ONCE(folio_mapcount(folio) < 0); VM_BUG_ON_PAGE(!PageHead(page), page); - } else if (is_pmd_non_present_folio_entry(orig_pmd)) { - swp_entry_t entry; + } else if (pmd_is_valid_softleaf(orig_pmd)) { + const softleaf_t entry =3D softleaf_from_pmd(orig_pmd); =20 - entry =3D pmd_to_swp_entry(orig_pmd); - folio =3D pfn_swap_entry_folio(entry); + folio =3D softleaf_to_folio(entry); flush_needed =3D 0; =20 if (!thp_migration_supported()) @@ -2428,7 +2427,7 @@ static inline int pmd_move_must_withdraw(spinlock_t *= new_pmd_ptl, static pmd_t move_soft_dirty_pmd(pmd_t pmd) { #ifdef CONFIG_MEM_SOFT_DIRTY - if (unlikely(is_pmd_migration_entry(pmd))) + if (unlikely(pmd_is_migration_entry(pmd))) pmd =3D pmd_swp_mksoft_dirty(pmd); else if (pmd_present(pmd)) pmd =3D pmd_mksoft_dirty(pmd); @@ -2503,12 +2502,12 @@ static void change_non_present_huge_pmd(struct mm_s= truct *mm, unsigned long addr, pmd_t *pmd, bool uffd_wp, bool uffd_wp_resolve) { - swp_entry_t entry =3D pmd_to_swp_entry(*pmd); - struct folio *folio =3D pfn_swap_entry_folio(entry); + softleaf_t entry =3D softleaf_from_pmd(*pmd); + const struct folio *folio =3D softleaf_to_folio(entry); pmd_t newpmd; =20 - VM_WARN_ON(!is_pmd_non_present_folio_entry(*pmd)); - if (is_writable_migration_entry(entry)) { + VM_WARN_ON(!pmd_is_valid_softleaf(*pmd)); + if (softleaf_is_migration_write(entry)) { /* * A protection check is difficult so * just be safe and disable write @@ -2520,7 +2519,7 @@ static void change_non_present_huge_pmd(struct mm_str= uct *mm, newpmd =3D swp_entry_to_pmd(entry); if (pmd_swp_soft_dirty(*pmd)) newpmd =3D pmd_swp_mksoft_dirty(newpmd); - } else if (is_writable_device_private_entry(entry)) { + } else if (softleaf_is_device_private_write(entry)) { entry =3D make_readable_device_private_entry(swp_offset(entry)); newpmd =3D swp_entry_to_pmd(entry); } else { @@ -2718,7 +2717,7 @@ int move_pages_huge_pmd(struct mm_struct *mm, pmd_t *= dst_pmd, pmd_t *src_pmd, pm =20 if (!pmd_trans_huge(src_pmdval)) { spin_unlock(src_ptl); - if (is_pmd_migration_entry(src_pmdval)) { + if (pmd_is_migration_entry(src_pmdval)) { pmd_migration_entry_wait(mm, &src_pmdval); return -EAGAIN; } @@ -2983,13 +2982,12 @@ static void __split_huge_pmd_locked(struct vm_area_= struct *vma, pmd_t *pmd, unsigned long addr; pte_t *pte; int i; - swp_entry_t entry; =20 VM_BUG_ON(haddr & ~HPAGE_PMD_MASK); VM_BUG_ON_VMA(vma->vm_start > haddr, vma); VM_BUG_ON_VMA(vma->vm_end < haddr + HPAGE_PMD_SIZE, vma); =20 - VM_WARN_ON(!is_pmd_non_present_folio_entry(*pmd) && !pmd_trans_huge(*pmd)= ); + VM_WARN_ON_ONCE(!pmd_is_valid_softleaf(*pmd) && !pmd_trans_huge(*pmd)); =20 count_vm_event(THP_SPLIT_PMD); =20 @@ -3003,11 +3001,10 @@ static void __split_huge_pmd_locked(struct vm_area_= struct *vma, pmd_t *pmd, zap_deposited_table(mm, pmd); if (!vma_is_dax(vma) && vma_is_special_huge(vma)) return; - if (unlikely(is_pmd_migration_entry(old_pmd))) { - swp_entry_t entry; + if (unlikely(pmd_is_migration_entry(old_pmd))) { + const softleaf_t old_entry =3D softleaf_from_pmd(old_pmd); =20 - entry =3D pmd_to_swp_entry(old_pmd); - folio =3D pfn_swap_entry_folio(entry); + folio =3D softleaf_to_folio(old_entry); } else if (is_huge_zero_pmd(old_pmd)) { return; } else { @@ -3037,31 +3034,34 @@ static void __split_huge_pmd_locked(struct vm_area_= struct *vma, pmd_t *pmd, return __split_huge_zero_page_pmd(vma, haddr, pmd); } =20 + if (pmd_is_migration_entry(*pmd)) { + softleaf_t entry; =20 - if (is_pmd_migration_entry(*pmd)) { old_pmd =3D *pmd; - entry =3D pmd_to_swp_entry(old_pmd); - page =3D pfn_swap_entry_to_page(entry); + entry =3D softleaf_from_pmd(old_pmd); + page =3D softleaf_to_page(entry); folio =3D page_folio(page); =20 soft_dirty =3D pmd_swp_soft_dirty(old_pmd); uffd_wp =3D pmd_swp_uffd_wp(old_pmd); =20 - write =3D is_writable_migration_entry(entry); + write =3D softleaf_is_migration_write(entry); if (PageAnon(page)) - anon_exclusive =3D is_readable_exclusive_migration_entry(entry); - young =3D is_migration_entry_young(entry); - dirty =3D is_migration_entry_dirty(entry); - } else if (is_pmd_device_private_entry(*pmd)) { + anon_exclusive =3D softleaf_is_migration_read_exclusive(entry); + young =3D softleaf_is_migration_young(entry); + dirty =3D softleaf_is_migration_dirty(entry); + } else if (pmd_is_device_private_entry(*pmd)) { + softleaf_t entry; + old_pmd =3D *pmd; - entry =3D pmd_to_swp_entry(old_pmd); - page =3D pfn_swap_entry_to_page(entry); + entry =3D softleaf_from_pmd(old_pmd); + page =3D softleaf_to_page(entry); folio =3D page_folio(page); =20 soft_dirty =3D pmd_swp_soft_dirty(old_pmd); uffd_wp =3D pmd_swp_uffd_wp(old_pmd); =20 - write =3D is_writable_device_private_entry(entry); + write =3D softleaf_is_device_private_write(entry); anon_exclusive =3D PageAnonExclusive(page); =20 /* @@ -3165,7 +3165,7 @@ static void __split_huge_pmd_locked(struct vm_area_st= ruct *vma, pmd_t *pmd, * Note that NUMA hinting access restrictions are not transferred to * avoid any possibility of altering permissions across VMAs. */ - if (freeze || is_pmd_migration_entry(old_pmd)) { + if (freeze || pmd_is_migration_entry(old_pmd)) { pte_t entry; swp_entry_t swp_entry; =20 @@ -3191,7 +3191,7 @@ static void __split_huge_pmd_locked(struct vm_area_st= ruct *vma, pmd_t *pmd, VM_WARN_ON(!pte_none(ptep_get(pte + i))); set_pte_at(mm, addr, pte + i, entry); } - } else if (is_pmd_device_private_entry(old_pmd)) { + } else if (pmd_is_device_private_entry(old_pmd)) { pte_t entry; swp_entry_t swp_entry; =20 @@ -3241,7 +3241,7 @@ static void __split_huge_pmd_locked(struct vm_area_st= ruct *vma, pmd_t *pmd, } pte_unmap(pte); =20 - if (!is_pmd_migration_entry(*pmd)) + if (!pmd_is_migration_entry(*pmd)) folio_remove_rmap_pmd(folio, page, vma); if (freeze) put_page(page); @@ -3254,7 +3254,7 @@ void split_huge_pmd_locked(struct vm_area_struct *vma= , unsigned long address, pmd_t *pmd, bool freeze) { VM_WARN_ON_ONCE(!IS_ALIGNED(address, HPAGE_PMD_SIZE)); - if (pmd_trans_huge(*pmd) || is_pmd_non_present_folio_entry(*pmd)) + if (pmd_trans_huge(*pmd) || pmd_is_valid_softleaf(*pmd)) __split_huge_pmd_locked(vma, pmd, address, freeze); } =20 @@ -4855,12 +4855,12 @@ void remove_migration_pmd(struct page_vma_mapped_wa= lk *pvmw, struct page *new) unsigned long address =3D pvmw->address; unsigned long haddr =3D address & HPAGE_PMD_MASK; pmd_t pmde; - swp_entry_t entry; + softleaf_t entry; =20 if (!(pvmw->pmd && !pvmw->pte)) return; =20 - entry =3D pmd_to_swp_entry(*pvmw->pmd); + entry =3D softleaf_from_pmd(*pvmw->pmd); folio_get(folio); pmde =3D folio_mk_pmd(folio, READ_ONCE(vma->vm_page_prot)); =20 @@ -4876,20 +4876,20 @@ void remove_migration_pmd(struct page_vma_mapped_wa= lk *pvmw, struct page *new) =20 if (pmd_swp_soft_dirty(*pvmw->pmd)) pmde =3D pmd_mksoft_dirty(pmde); - if (is_writable_migration_entry(entry)) + if (softleaf_is_migration_write(entry)) pmde =3D pmd_mkwrite(pmde, vma); if (pmd_swp_uffd_wp(*pvmw->pmd)) pmde =3D pmd_mkuffd_wp(pmde); - if (!is_migration_entry_young(entry)) + if (!softleaf_is_migration_young(entry)) pmde =3D pmd_mkold(pmde); /* NOTE: this may contain setting soft-dirty on some archs */ - if (folio_test_dirty(folio) && is_migration_entry_dirty(entry)) + if (folio_test_dirty(folio) && softleaf_is_migration_dirty(entry)) pmde =3D pmd_mkdirty(pmde); =20 if (folio_test_anon(folio)) { rmap_t rmap_flags =3D RMAP_NONE; =20 - if (!is_readable_migration_entry(entry)) + if (!softleaf_is_migration_read(entry)) rmap_flags |=3D RMAP_EXCLUSIVE; =20 folio_add_anon_rmap_pmd(folio, new, vma, haddr, rmap_flags); diff --git a/mm/khugepaged.c b/mm/khugepaged.c index a97ff7bcb232..1a08673b0d8b 100644 --- a/mm/khugepaged.c +++ b/mm/khugepaged.c @@ -17,7 +17,7 @@ #include #include #include -#include +#include #include #include #include @@ -941,7 +941,7 @@ static inline int check_pmd_state(pmd_t *pmd) * collapse it. Migration success or failure will eventually end * up with a present PMD mapping a folio again. */ - if (is_pmd_migration_entry(pmde)) + if (pmd_is_migration_entry(pmde)) return SCAN_PMD_MAPPED; if (!pmd_present(pmde)) return SCAN_PMD_NULL; diff --git a/mm/madvise.c b/mm/madvise.c index 58d82495b6c6..ffae3b566dc1 100644 --- a/mm/madvise.c +++ b/mm/madvise.c @@ -390,7 +390,7 @@ static int madvise_cold_or_pageout_pte_range(pmd_t *pmd, =20 if (unlikely(!pmd_present(orig_pmd))) { VM_BUG_ON(thp_migration_supported() && - !is_pmd_migration_entry(orig_pmd)); + !pmd_is_migration_entry(orig_pmd)); goto huge_unlock; } =20 diff --git a/mm/memory.c b/mm/memory.c index fea079e5fb90..bf2bbd0dbc97 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -6362,10 +6362,10 @@ static vm_fault_t __handle_mm_fault(struct vm_area_= struct *vma, goto fallback; =20 if (unlikely(!pmd_present(vmf.orig_pmd))) { - if (is_pmd_device_private_entry(vmf.orig_pmd)) + if (pmd_is_device_private_entry(vmf.orig_pmd)) return do_huge_pmd_device_private(&vmf); =20 - if (is_pmd_migration_entry(vmf.orig_pmd)) + if (pmd_is_migration_entry(vmf.orig_pmd)) pmd_migration_entry_wait(mm, vmf.pmd); return 0; } diff --git a/mm/mempolicy.c b/mm/mempolicy.c index 7ae3f5e2dee6..01c3b98f87a6 100644 --- a/mm/mempolicy.c +++ b/mm/mempolicy.c @@ -110,7 +110,7 @@ #include #include #include -#include +#include #include =20 #include @@ -647,7 +647,7 @@ static void queue_folios_pmd(pmd_t *pmd, struct mm_walk= *walk) struct folio *folio; struct queue_pages *qp =3D walk->private; =20 - if (unlikely(is_pmd_migration_entry(*pmd))) { + if (unlikely(pmd_is_migration_entry(*pmd))) { qp->nr_failed++; return; } diff --git a/mm/migrate.c b/mm/migrate.c index 862b2e261cf9..3b6bd374157d 100644 --- a/mm/migrate.c +++ b/mm/migrate.c @@ -16,7 +16,7 @@ #include #include #include -#include +#include #include #include #include @@ -353,7 +353,7 @@ static bool remove_migration_pte(struct folio *folio, rmap_t rmap_flags =3D RMAP_NONE; pte_t old_pte; pte_t pte; - swp_entry_t entry; + softleaf_t entry; struct page *new; unsigned long idx =3D 0; =20 @@ -379,22 +379,22 @@ static bool remove_migration_pte(struct folio *folio, folio_get(folio); pte =3D mk_pte(new, READ_ONCE(vma->vm_page_prot)); =20 - entry =3D pte_to_swp_entry(old_pte); - if (!is_migration_entry_young(entry)) + entry =3D softleaf_from_pte(old_pte); + if (!softleaf_is_migration_young(entry)) pte =3D pte_mkold(pte); - if (folio_test_dirty(folio) && is_migration_entry_dirty(entry)) + if (folio_test_dirty(folio) && softleaf_is_migration_dirty(entry)) pte =3D pte_mkdirty(pte); if (pte_swp_soft_dirty(old_pte)) pte =3D pte_mksoft_dirty(pte); else pte =3D pte_clear_soft_dirty(pte); =20 - if (is_writable_migration_entry(entry)) + if (softleaf_is_migration_write(entry)) pte =3D pte_mkwrite(pte, vma); else if (pte_swp_uffd_wp(old_pte)) pte =3D pte_mkuffd_wp(pte); =20 - if (folio_test_anon(folio) && !is_readable_migration_entry(entry)) + if (folio_test_anon(folio) && !softleaf_is_migration_read(entry)) rmap_flags |=3D RMAP_EXCLUSIVE; =20 if (unlikely(is_device_private_page(new))) { @@ -404,7 +404,7 @@ static bool remove_migration_pte(struct folio *folio, else entry =3D make_readable_device_private_entry( page_to_pfn(new)); - pte =3D swp_entry_to_pte(entry); + pte =3D softleaf_to_pte(entry); if (pte_swp_soft_dirty(old_pte)) pte =3D pte_swp_mksoft_dirty(pte); if (pte_swp_uffd_wp(old_pte)) @@ -543,9 +543,9 @@ void pmd_migration_entry_wait(struct mm_struct *mm, pmd= _t *pmd) spinlock_t *ptl; =20 ptl =3D pmd_lock(mm, pmd); - if (!is_pmd_migration_entry(*pmd)) + if (!pmd_is_migration_entry(*pmd)) goto unlock; - migration_entry_wait_on_locked(pmd_to_swp_entry(*pmd), ptl); + migration_entry_wait_on_locked(softleaf_from_pmd(*pmd), ptl); return; unlock: spin_unlock(ptl); diff --git a/mm/migrate_device.c b/mm/migrate_device.c index c869b272e85a..880f26a316f8 100644 --- a/mm/migrate_device.c +++ b/mm/migrate_device.c @@ -13,7 +13,7 @@ #include #include #include -#include +#include #include #include #include "internal.h" @@ -145,7 +145,6 @@ static int migrate_vma_collect_huge_pmd(pmd_t *pmdp, un= signed long start, struct folio *folio; struct migrate_vma *migrate =3D walk->private; spinlock_t *ptl; - swp_entry_t entry; int ret; unsigned long write =3D 0; =20 @@ -169,23 +168,24 @@ static int migrate_vma_collect_huge_pmd(pmd_t *pmdp, = unsigned long start, if (pmd_write(*pmdp)) write =3D MIGRATE_PFN_WRITE; } else if (!pmd_present(*pmdp)) { - entry =3D pmd_to_swp_entry(*pmdp); - folio =3D pfn_swap_entry_folio(entry); + const softleaf_t entry =3D softleaf_from_pmd(*pmdp); =20 - if (!is_device_private_entry(entry) || + folio =3D softleaf_to_folio(entry); + + if (!softleaf_is_device_private(entry) || !(migrate->flags & MIGRATE_VMA_SELECT_DEVICE_PRIVATE) || (folio->pgmap->owner !=3D migrate->pgmap_owner)) { spin_unlock(ptl); return migrate_vma_collect_skip(start, end, walk); } =20 - if (is_migration_entry(entry)) { + if (softleaf_is_migration(entry)) { migration_entry_wait_on_locked(entry, ptl); spin_unlock(ptl); return -EAGAIN; } =20 - if (is_writable_device_private_entry(entry)) + if (softleaf_is_device_private_write(entry)) write =3D MIGRATE_PFN_WRITE; } else { spin_unlock(ptl); diff --git a/mm/page_table_check.c b/mm/page_table_check.c index f5f25e120f69..9af1ecff5221 100644 --- a/mm/page_table_check.c +++ b/mm/page_table_check.c @@ -8,7 +8,7 @@ #include #include #include -#include +#include =20 #undef pr_fmt #define pr_fmt(fmt) "page_table_check: " fmt @@ -179,10 +179,10 @@ void __page_table_check_pud_clear(struct mm_struct *m= m, pud_t pud) EXPORT_SYMBOL(__page_table_check_pud_clear); =20 /* Whether the swap entry cached writable information */ -static inline bool swap_cached_writable(swp_entry_t entry) +static inline bool softleaf_cached_writable(softleaf_t entry) { - return is_writable_device_private_entry(entry) || - is_writable_migration_entry(entry); + return softleaf_is_device_private(entry) || + softleaf_is_migration_write(entry); } =20 static void page_table_check_pte_flags(pte_t pte) @@ -190,9 +190,9 @@ static void page_table_check_pte_flags(pte_t pte) if (pte_present(pte)) { WARN_ON_ONCE(pte_uffd_wp(pte) && pte_write(pte)); } else if (pte_swp_uffd_wp(pte)) { - const swp_entry_t entry =3D pte_to_swp_entry(pte); + const softleaf_t entry =3D softleaf_from_pte(pte); =20 - WARN_ON_ONCE(swap_cached_writable(entry)); + WARN_ON_ONCE(softleaf_cached_writable(entry)); } } =20 @@ -219,9 +219,9 @@ static inline void page_table_check_pmd_flags(pmd_t pmd) if (pmd_uffd_wp(pmd)) WARN_ON_ONCE(pmd_write(pmd)); } else if (pmd_swp_uffd_wp(pmd)) { - swp_entry_t entry =3D pmd_to_swp_entry(pmd); + const softleaf_t entry =3D softleaf_from_pmd(pmd); =20 - WARN_ON_ONCE(swap_cached_writable(entry)); + WARN_ON_ONCE(softleaf_cached_writable(entry)); } } =20 diff --git a/mm/page_vma_mapped.c b/mm/page_vma_mapped.c index a4e23818f37f..8137d2366722 100644 --- a/mm/page_vma_mapped.c +++ b/mm/page_vma_mapped.c @@ -242,18 +242,19 @@ bool page_vma_mapped_walk(struct page_vma_mapped_walk= *pvmw) */ pmde =3D pmdp_get_lockless(pvmw->pmd); =20 - if (pmd_trans_huge(pmde) || is_pmd_migration_entry(pmde)) { + if (pmd_trans_huge(pmde) || pmd_is_migration_entry(pmde)) { pvmw->ptl =3D pmd_lock(mm, pvmw->pmd); pmde =3D *pvmw->pmd; if (!pmd_present(pmde)) { - swp_entry_t entry; + softleaf_t entry; =20 if (!thp_migration_supported() || !(pvmw->flags & PVMW_MIGRATION)) return not_found(pvmw); - entry =3D pmd_to_swp_entry(pmde); - if (!is_migration_entry(entry) || - !check_pmd(swp_offset_pfn(entry), pvmw)) + entry =3D softleaf_from_pmd(pmde); + + if (!softleaf_is_migration(entry) || + !check_pmd(softleaf_to_pfn(entry), pvmw)) return not_found(pvmw); return true; } @@ -273,9 +274,9 @@ bool page_vma_mapped_walk(struct page_vma_mapped_walk *= pvmw) * cannot return prematurely, while zap_huge_pmd() has * cleared *pmd but not decremented compound_mapcount(). */ - swp_entry_t entry =3D pmd_to_swp_entry(pmde); + const softleaf_t entry =3D softleaf_from_pmd(pmde); =20 - if (is_device_private_entry(entry)) { + if (softleaf_is_device_private(entry)) { pvmw->ptl =3D pmd_lock(mm, pvmw->pmd); return true; } diff --git a/mm/pagewalk.c b/mm/pagewalk.c index 9f91cf85a5be..3067feb970d1 100644 --- a/mm/pagewalk.c +++ b/mm/pagewalk.c @@ -5,7 +5,7 @@ #include #include #include -#include +#include =20 #include =20 @@ -966,10 +966,10 @@ struct folio *folio_walk_start(struct folio_walk *fw, goto found; } } else if ((flags & FW_MIGRATION) && - is_pmd_migration_entry(pmd)) { - swp_entry_t entry =3D pmd_to_swp_entry(pmd); + pmd_is_migration_entry(pmd)) { + const softleaf_t entry =3D softleaf_from_pmd(pmd); =20 - page =3D pfn_swap_entry_to_page(entry); + page =3D softleaf_to_page(entry); expose_page =3D false; goto found; } diff --git a/mm/rmap.c b/mm/rmap.c index 1954c538a991..775710115a41 100644 --- a/mm/rmap.c +++ b/mm/rmap.c @@ -57,7 +57,7 @@ #include #include #include -#include +#include #include #include #include @@ -2341,7 +2341,7 @@ static bool try_to_migrate_one(struct folio *folio, s= truct vm_area_struct *vma, if (likely(pmd_present(pmdval))) pfn =3D pmd_pfn(pmdval); else - pfn =3D swp_offset_pfn(pmd_to_swp_entry(pmdval)); + pfn =3D softleaf_to_pfn(softleaf_from_pmd(pmdval)); =20 subpage =3D folio_page(folio, pfn - folio_pfn(folio)); =20 --=20 2.51.0 From nobody Fri Dec 19 21:47:08 2025 Received: from mx0b-00069f02.pphosted.com (mx0b-00069f02.pphosted.com [205.220.177.32]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 73F4E145329; Sat, 8 Nov 2025 17:11:14 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=205.220.177.32 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1762621876; cv=fail; b=g8S3ztnWg+J8ELxY9iXle/z346TIuw1tlKdZcdOEso9qmYL57UKRCUMwV7t1Jkx0uLPZjaGGunAhpE2eLird+fDfNsmSb6HQU4bPCF4jVQ9Um/HG8vf9Sd5Eu0xM3RQaMClwB9/i5VthGvZNjox7tCikhX/U0GH2aPVz40wEHq8= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1762621876; c=relaxed/simple; bh=POgjLibdleyIT97aLcMp4rGu38r3SsMhgSVUA/LXdP4=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: Content-Type:MIME-Version; b=ZG3Za343Yukw9sQqthHoojQhT4T4TFLojuuC5+syzbmQov2/5V9Sewqb3gVCnh+9hFu2d2pC0TmgSuOzSgZZQ93ar4lMr8ki9+xEoRrDr9kGaxm50xSrtji8xx+EVm7FS9G6hZ/9wBXKbMULL/ERT35mriLK1veQlG2UN6eitII= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=oracle.com; spf=pass smtp.mailfrom=oracle.com; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b=pfhozLsY; dkim=pass (1024-bit key) header.d=oracle.onmicrosoft.com header.i=@oracle.onmicrosoft.com header.b=V/R4nPKW; arc=fail smtp.client-ip=205.220.177.32 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=oracle.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=oracle.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b="pfhozLsY"; dkim=pass (1024-bit key) header.d=oracle.onmicrosoft.com header.i=@oracle.onmicrosoft.com header.b="V/R4nPKW" Received: from pps.filterd (m0333520.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 5A8GjT51017275; Sat, 8 Nov 2025 17:09:26 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=cc :content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s= corp-2025-04-25; bh=hnulhczD3XQkqRXiu7lNvxRJz88HMl5Uk5p8sHNK2Ms=; b= pfhozLsYLuo5vUIzEyhlrv/2tuKzgVI+sWy7TWDNIwMcj8FcFGwdBzC9FfdCMlAg Ghcnewisojj4R9RtgxT+0/VjMiIchW4pSoild2sqftMWCnDALKJOSzoSjgI7UkIN 1umpU3hvjUbOm3lCOAfs7bXx8yYMDT76CALjKFiZVmqmvUyxm7XOO5kri2wLgkQI ZXN2KMGHuypkQnYk4VMbdNKyfoUllEGYPgKmPhYlUDLj5d3uflpO67kZVQD0nvYB coMydkb72+6pa98+bIEXVqDKUTvFUJSTtJDvX0+NnHw8FXY/sXrKGH8TIrA6GMVg 0knC3gFh0n+64GXB3Jb59g== Received: from phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta01.appoci.oracle.com [138.1.114.2]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 4aa8se01uw-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Sat, 08 Nov 2025 17:09:25 +0000 (GMT) Received: from pps.filterd (phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (8.18.1.2/8.18.1.2) with ESMTP id 5A8C03c9007618; Sat, 8 Nov 2025 17:09:24 GMT Received: from sn4pr0501cu005.outbound.protection.outlook.com (mail-southcentralusazon11011047.outbound.protection.outlook.com [40.93.194.47]) by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTPS id 4a9va76n25-2 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Sat, 08 Nov 2025 17:09:24 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=IYdhFT9QsanE2SrngLg//h1qLd/XQJjKb7rZfpmvvXHTolXMsrnQtXLmBsNmUgw2bTkmnijq08hPGTDJZePe+oGTv4EHjwpPJoocyTCPXi/XbmJHaQBfkhUwYKfnl2wmtxf46K6NlpLFfxPFqWpgbhf+xctzX9PC0U3F6u7+P9uR/MmC5pQBCQvTosY2NNrij5w4g/ZGVtLsSuw2jlDIybSB6S/FuJWCwDwzyGFyWXBdgfgTRP0XFkQWE2yz0bLUaTooieFCFJwk9Lsi9vApS5oQ+FQNQNbkBQlRZg/mM095nwxaGEnSLhMsuaF9PoHiFpCpT5Txn+55oCPDl6fMig== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=hnulhczD3XQkqRXiu7lNvxRJz88HMl5Uk5p8sHNK2Ms=; b=SNCl66Y9m5TGc6Uck/oosQ4UA79V84eLUKHsoS4apkI7/jyRt4IcAwlapvb4e/cEIojYrCWQzPOiXcHk8rpK6jc8/HPqukeIQ2jKmYUHeuabG9va/s/fTbCvp+e5pH3z4TbNFj8W29cU0FL+CyEAJ6s3ZxwO3lT9asyvqyGcxBKrcxaMVPwR2Vb4K5yrBGXnlqyJo9TL8oDG55EAd0+pLMbJkuTCyZuccCvakTCmjYScl/3B3G6fZRUg/2D1DZ8OSwzN3xDwTNy+HBRBtL3CAPBEVgvkhCfQm8GrAGuD4CdUmBI4rzDAMCdy4a02XHcuGFoULgnDKrTTQFIxw5HlkQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=hnulhczD3XQkqRXiu7lNvxRJz88HMl5Uk5p8sHNK2Ms=; b=V/R4nPKW9D0P92YB7zoJJ4mzcTvjU+kx0aH/iKmLv6f4rukkwi/YzHCoVv1ov5hkAFhT8/fS7y+VKfNrrcGyNrr6Q1UaJNkRf+Tv734Tbzlx/yNYGhKjCEOJXA6eMaYUuYFVXg1V7bYObL6HiRyZs6Z+G463O/+2vdEq0kB8PIs= Received: from DM4PR10MB8218.namprd10.prod.outlook.com (2603:10b6:8:1cc::16) by SJ5PPF04D2D7FA7.namprd10.prod.outlook.com (2603:10b6:a0f:fc02::785) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9298.15; Sat, 8 Nov 2025 17:09:21 +0000 Received: from DM4PR10MB8218.namprd10.prod.outlook.com ([fe80::2650:55cf:2816:5f2]) by DM4PR10MB8218.namprd10.prod.outlook.com ([fe80::2650:55cf:2816:5f2%7]) with mapi id 15.20.9298.010; Sat, 8 Nov 2025 17:09:21 +0000 From: Lorenzo Stoakes To: Andrew Morton Cc: Christian Borntraeger , Janosch Frank , Claudio Imbrenda , David Hildenbrand , Alexander Gordeev , Gerald Schaefer , Heiko Carstens , Vasily Gorbik , Sven Schnelle , Peter Xu , Alexander Viro , Christian Brauner , Jan Kara , Arnd Bergmann , Zi Yan , Baolin Wang , "Liam R . Howlett" , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Lance Yang , Muchun Song , Oscar Salvador , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Matthew Brost , Joshua Hahn , Rakie Kim , Byungchul Park , Gregory Price , Ying Huang , Alistair Popple , Axel Rasmussen , Yuanchu Xie , Wei Xu , Kemeng Shi , Kairui Song , Nhat Pham , Baoquan He , Chris Li , SeongJae Park , Matthew Wilcox , Jason Gunthorpe , Leon Romanovsky , Xu Xin , Chengming Zhou , Jann Horn , Miaohe Lin , Naoya Horiguchi , Pedro Falcato , Pasha Tatashin , Rik van Riel , Harry Yoo , Hugh Dickins , linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-s390@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, linux-arch@vger.kernel.org, damon@lists.linux.dev Subject: [PATCH v2 11/16] mm: introduce pmd_is_huge() and use where appropriate Date: Sat, 8 Nov 2025 17:08:25 +0000 Message-ID: <5a8bfdf0f2018c3fa3b10171df7cab469dd3f259.1762621568.git.lorenzo.stoakes@oracle.com> X-Mailer: git-send-email 2.51.0 In-Reply-To: References: Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: LO2P265CA0506.GBRP265.PROD.OUTLOOK.COM (2603:10a6:600:13b::13) To DM4PR10MB8218.namprd10.prod.outlook.com (2603:10b6:8:1cc::16) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DM4PR10MB8218:EE_|SJ5PPF04D2D7FA7:EE_ X-MS-Office365-Filtering-Correlation-Id: 7e03f98a-4654-42fe-dfc9-08de1ee98eff X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|1800799024|7416014|376014|366016; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?rbf8wLMMMBW+My2lJwQVN70GNmFH1RP8V8OdLI4xet0fwIIU/ulg4nalQ3QL?= =?us-ascii?Q?/QdMnLvTl7/YaVdMmgm3P4Nl4hZn0RuVkNjx2t68KpYFG9sw54kJiGaiMLl1?= =?us-ascii?Q?dWVTrzy15yCg0sHH7JdHiEd1XDHji7K1fAPwOWyHVuKuDEpOOCY7+ERn5RA/?= =?us-ascii?Q?oeIMtSHXO5LeGRlKh+kEF+FB7N7vue/xFB6N69Oww8LsFluXhtVTwuu0ecwH?= =?us-ascii?Q?s/3o0Vo5oGJ95NkL81UhJFX0z9kcyizgkA+bAd/sSAp1NMJhLX7NUZH9+n8l?= =?us-ascii?Q?/uO/Y4Lws/gVpE79m3+2Hgv+/hZQ5R+ldTTyDzVKmkBi5uJM8AYUqt+BIFyP?= =?us-ascii?Q?EafSX6tsE2wnDk9F+ClNJAZmV4g36hvUoTcGrPFsAo+1WTbHxfflqJtL6O3Q?= =?us-ascii?Q?6JaT8FhBKLDIGTCSCV+MUwxRXPyvhlEUnwhI9LL7AQbDm+vK01HzvjtWPlzn?= =?us-ascii?Q?yHotZmVYmNDxZHoaDSpM5lG3OY5XuKOkKeSEeRbZfBHoeJcOQEL/gS+09B9d?= =?us-ascii?Q?NEbEP8waJCucqYLznKaOoV1daQOboGXg5XvgqOm222Z2xnr2qCmOYZI8t9d6?= =?us-ascii?Q?fgxWTa845k/sqIeGd3k8xfVyzjzD5T6gfutxlkTxn6CoPKL6S7McDPJwg+H7?= =?us-ascii?Q?ZS75bdc3SIqQWooEfoeEEkEARltWamqlTWhgSuvgnt1he9OvjR4Yyhay9rHV?= =?us-ascii?Q?l9jwnimwgAlmBDZxdsdK52rWPE0kS5vZZpwQFDjWEny1170Vc88ghr0UeA+D?= =?us-ascii?Q?L2vgfLFzchx3h2ZMtdYhNBcP4AqHeaDLfjxkE9HOc/fYBLHDLaZc72iyKZvg?= =?us-ascii?Q?MiCFxMv1E20xMb2Dm26gAK5CNiJLPtmdPCDDbuwXEh+VzvJQ1xKLXMp/wlaf?= =?us-ascii?Q?nmpnHSeHX22k7NPy/3XSZHSDycXINorqn+e1zJOwCuA0ti1xiC+1vcbCoLXm?= =?us-ascii?Q?MaREU82B0nRqvyTqbAKscr+g5amK2d8tU/6/ogjoaPdEOZYZpuZh6XFx4Pyg?= =?us-ascii?Q?S+R17h2exVs+caCvlBE9TVai+Eg0xbz4yS2c8iFv0sbkvSLWz1AHec5IKIK4?= =?us-ascii?Q?gEYkKvGwKgdYoBqnIsRGCXLw0Vnycuzc/CzVQPfyf04sqZgyCI0Y1BRUtIGe?= =?us-ascii?Q?S+1lXyPaldEwERjyi6mmmSererTtDxLAqX+kgFpNef+lGDwCTcg2r9Zr+V5w?= =?us-ascii?Q?Ddn+ucQD7qvkDdeR43z3pg/EHVPD/hsGtQZZ/ldYggbcZO6IxHbk+OCgJR+J?= =?us-ascii?Q?ophycb/FgqkHqr4mSvdRamO7avjs40erJjLxUhfl4p7bHT/54BiJZnjg1h3D?= =?us-ascii?Q?FjonTDaIl8SdjyASrvz9eTNZGm50JR8sDqSXUhCsuWJDMMjT/79vHnYeKxMX?= =?us-ascii?Q?MJjmGb3we2z9wbcn8dQGfbQVAh8cRUF8S4guY4CLnf277IuRKpQ7kCubWvOB?= =?us-ascii?Q?Xsbc5A3Ob0BMOVotcME8fVp9DXpArqIG?= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DM4PR10MB8218.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(1800799024)(7416014)(376014)(366016);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?ZU9sPq5jRt0GpqdJ48hcI0EgB1GqMQ8uhf/pY7DDn/zC6DnYHjRE63onyLZH?= =?us-ascii?Q?iYrcHZiiNsJ8zLwn5OQ8KJCQI0SVs+JHU2VwIBFyyzcgMZo+ZWjcrj/ZwvmG?= =?us-ascii?Q?D1Zve50VAuivRZfKf8mME/rzbf12MWEnjSoLDrTnFJou6/SBshqH3mBHStci?= =?us-ascii?Q?9ztF4uyQFvAl8NJLWWMQljTdQZcjtyugQskCwY/jUHJ02CvJi2GLOUyJuibq?= =?us-ascii?Q?O1nD2bDq/qZ32yiKg+WGA/fhbynJppLAvjOkhfdPg28eEymdL7NhwJxwovnv?= =?us-ascii?Q?tPC4tJKRTCTpDziCVhLdl52w4VvpII7yUptgTLg6U9uK4xc/jDVwDczm5Nub?= =?us-ascii?Q?uCsRjR5ApN1NwaKibnRpp0yKAmPdkS8Zn5VKNPE2dcPd9z2z7i7/kGo2hLSV?= =?us-ascii?Q?GiWI6auDf4r1Bwab+cgxv+edXSBg2PGKE5atULPJXroUVs1zJ+1qdGFEqjY1?= =?us-ascii?Q?8x8Em3Z7GPS0zVOWLJU5m1XXTr+5GN50Il8TIAwkbv2D0RQTgGYBg2x8ThAt?= =?us-ascii?Q?/piMStP2wlLBZ/jLHhDF/2UaZVZdoyzLCgKHzSzkR2MOsWH/kDo0A5kXyqmr?= =?us-ascii?Q?pKIFQMj8gFnaBAukF5W6BRW+NqdiX/YcXIt3UsIvu2eAO/mKTMzM5M43fpHy?= =?us-ascii?Q?X5ZxhPB3IR0uox2T+AR6ZSoVErU8YFfee0tTQgR0p4QBYk4k4XGQgZClumUo?= =?us-ascii?Q?uogvsIz3Dejlhd19P+QGnIqs2eo2V3Q5PUtjLLsLjSKyOPQD009NnhnEgrGt?= =?us-ascii?Q?r+VP8DoStyXb0H2RGkSchVkPsayuogZ0+yhPEuFD1bhNeS6ecLUDxiUN7420?= =?us-ascii?Q?5YFAIPUItx51uqK+DwptaQ8AyRH1WjpT7/9QW7Q+IiXk6gfLNE4HQdWbcYyF?= =?us-ascii?Q?kLLWWxTm//4Pgaaiih4etT2lLFy1JCDPxJn16hFEXgJ2GO2oxMfvjQiQVTCJ?= =?us-ascii?Q?YhjKE7hBlUpy6kpq/G8QQIBnEuB/2L54K4QLB6OcMG5zTsOW8g4AHeu19BNB?= =?us-ascii?Q?AqeOwztEI4q+8EQZNYpABzGPQ4nCaFAYlP4ktfuHF+sDNnmgEfi18tjHFY8q?= =?us-ascii?Q?qsPB0zc4muyHO0zA2ytBVuWjIRPBmQet4sF0xoJ9VujOTdEa2Sfh3modIPH6?= =?us-ascii?Q?v3SAgHNV7R88yXvxBBRQGDHdjtDq3ayptfVDfgchEjwKv8Ut6EoENjNuiOTN?= =?us-ascii?Q?pGmosR8+eC7vFjCabNdjaHwuMmVHuZT8VqEdU3fyhXBzFfDJLxWGFAVjFryE?= =?us-ascii?Q?YzIrbhHXjaHX/zQWt/QJN2tttlnJ361DFz1XTmOurXSz809O4M1Cyi3GYruK?= =?us-ascii?Q?wEKIo8U/GWHlKOR49HV8PMJdg9oS1NSGTUPYaCSfKQ6QfWhQ0IdRpV+fGt67?= =?us-ascii?Q?AxymEn1B195pV/RSndyK+iIx040ZGPqUzqreQl03j1LArw213kL6WbwV1y5u?= =?us-ascii?Q?c9Cbf0tQ5VgbTQ66U9YVFR/7YsMCuh3yUuaosuj0aGaYQupaYSIlGvVTGquu?= =?us-ascii?Q?AcVd+8/wtaZGREsfUTevcDv+pfs1Qjp8tTDJVqqeI0ORSTFmWd4NCpBGCDZs?= =?us-ascii?Q?S06/sNjwEa/LK/CQCGDtlfyhi4oLM3W9Q/8hmoAt5gJNd4hOqZCAnyglYup2?= =?us-ascii?Q?pg=3D=3D?= X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: 6wcwAg+sKr1zxBqOtVcFFLd6U8fFaBDkOrtaveWb/BJ6jpTn9P6FC82PQWRr0yfck7tdkTqWTnd1d0Y4g0CxaAhsSbiT1A+m55yA0dGqGXZ5FKK7LTNoQTgCOiLsoXkrEyi96WH2bElJY5krByoYcxheP+ruSb5S8wl7YZQuAvXRzKtens5HoZamrYPIGUeuH+bAa2B67cPZUls50r37wX9wlxqvSFoiKJ3kq+qTNVm3jjEhkAehY/guLbvz27H7xK5696cPxKzNHijHUcItxqpcZptQnhbJz2fg5ZE22DESFOHWIJJyxqcVYFyC66YvtUA0T+K53b6/sFmfhDDg13enRmViVi8m46mvO6V/+ClLXEgsemnDal4LBvyG4KLyE8y8gjjK1FrK7W1oAfVxCZGgPuFj5TDxFd4kA6N8NANy8V2aVXmfn1y56+UgYzdAPuSQiqM7/iPruoLvb2XcP3Pg5i6/lvGFCuFSKNvImRPIawzZLqmPwApK3YqMBshvMLvILB+Nxzn1eSH3X5dWRiW0iNbkTBj395UptsNtdl9FRZITFSxYfrYANoEtukuUSF/IoUA7woswhT5JdLamANAeA7biMYLUmOc5ETX+4gY= X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: 7e03f98a-4654-42fe-dfc9-08de1ee98eff X-MS-Exchange-CrossTenant-AuthSource: DM4PR10MB8218.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 Nov 2025 17:09:20.9921 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 6V4QfnCfNAeNz94PsHDVTnjLrYGjkbFdYNnoEx2q1pSeSSTdA8jbJ0TmNUY1q7KRsd3kHifH/MGjp8Lx3DMNKGmjlLjF7epDHCm5iMOyJYM= X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ5PPF04D2D7FA7 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1121,Hydra:6.1.9,FMLib:17.12.100.49 definitions=2025-11-08_04,2025-11-06_01,2025-10-01_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 adultscore=0 malwarescore=0 spamscore=0 mlxscore=0 bulkscore=0 suspectscore=0 mlxlogscore=999 phishscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2510240000 definitions=main-2511080138 X-Proofpoint-GUID: DrJQDBfU-FVNg-c1Usy-vIdkmo5-2bAl X-Proofpoint-Spam-Details-Enc: AW1haW4tMjUxMTA4MDEyNiBTYWx0ZWRfX0ywOzvtA43nV YCT7PDqxPWFBIGoSJ/1igXzH78/w8G12xHJv0BDNERyntRhVxTVTF1hJOwQG3U0iEir4yL2Eubh eZvVekzStiz66dAXkHOvhK0LD+Tn8dgPBSB9KhdLabjJ95vL9iZsCMTD/8TngP9KaZiOYTARQTV IDZoLATOibbjYZX85VwlT7+kpeBIVTXkxjARFyMMcaVmsV4XGJiFoOtgYXVrQifhuHofVi4IsJU s782xcV2lOFrQUqeu1kS9+N/OQbcijtxWtNmON2BqF0zcPuFJOQDit2phSklTjCi1EsNEnkvtf0 fjXM1A+440hlsFKfjUQrBQwiJmh8+sfhGsg76WFjdYhzaQMS1EvEtRGw13AE6iAdxVjgxUfeEaV FoIM0kSp4RH9yXiBy8/f+Rw6ZW9U7Q== X-Authority-Analysis: v=2.4 cv=XuL3+FF9 c=1 sm=1 tr=0 ts=690f7945 cx=c_pps a=XiAAW1AwiKB2Y8Wsi+sD2Q==:117 a=XiAAW1AwiKB2Y8Wsi+sD2Q==:17 a=6eWqkTHjU83fiwn7nKZWdM+Sl24=:19 a=z/mQ4Ysz8XfWz/Q5cLBRGdckG28=:19 a=lCpzRmAYbLLaTzLvsPZ7Mbvzbb8=:19 a=xqWC_Br6kY4A:10 a=6UeiqGixMTsA:10 a=GoEa3M9JfhUA:10 a=VkNPw1HP01LnGYTKEx00:22 a=yPCof4ZbAAAA:8 a=Z_ABK6pF82JkGIUUY4QA:9 a=UhEZJTgQB8St2RibIkdl:22 a=Z5ABNNGmrOfJ6cZ5bIyy:22 a=QOGEsqRv6VhmHaoFNykA:22 X-Proofpoint-ORIG-GUID: DrJQDBfU-FVNg-c1Usy-vIdkmo5-2bAl Content-Type: text/plain; charset="utf-8" The leaf entry PMD case is confusing as only migration entries and device private entries are valid at PMD level, not true swap entries. We repeatedly perform checks of the form is_swap_pmd() || pmd_trans_huge() which is itself confusing - it implies that leaf entries at PMD level exist and are different from huge entries. Address this confusion by introduced pmd_is_huge() which checks for either case. Sadly due to header dependency issues (huge_mm.h is included very early on in headers and cannot really rely on much else) we cannot use pmd_is_valid_softleaf() here. However since these are the only valid, handled cases the function is still achieving what it intends to do. We then replace all instances of is_swap_pmd() || pmd_trans_huge() with pmd_is_huge() invocations and adjust logic accordingly to accommodate this. No functional change intended. Signed-off-by: Lorenzo Stoakes --- include/linux/huge_mm.h | 39 +++++++++++++++++++++++++++++++++++---- include/linux/swapops.h | 6 ++++++ mm/huge_memory.c | 3 ++- mm/memory.c | 4 ++-- mm/mprotect.c | 2 +- mm/mremap.c | 2 +- 6 files changed, 47 insertions(+), 9 deletions(-) diff --git a/include/linux/huge_mm.h b/include/linux/huge_mm.h index cbb2243f8e56..403e13009631 100644 --- a/include/linux/huge_mm.h +++ b/include/linux/huge_mm.h @@ -419,10 +419,36 @@ void reparent_deferred_split_queue(struct mem_cgroup = *memcg); void __split_huge_pmd(struct vm_area_struct *vma, pmd_t *pmd, unsigned long address, bool freeze); =20 +/** + * pmd_is_huge() - Is this PMD either a huge PMD entry or a software leaf = entry? + * @pmd: The PMD to check. + * + * A huge PMD entry is a non-empty entry which is present and marked huge = or a + * software leaf entry. This check be performed without the appropriate lo= cks + * held, in which case the condition should be rechecked after they are + * acquired. + * + * Returns: true if this PMD is huge, false otherwise. + */ +static inline bool pmd_is_huge(pmd_t pmd) +{ + if (pmd_present(pmd)) { + return pmd_trans_huge(pmd); + } else if (!pmd_none(pmd)) { + /* + * Non-present PMDs must be valid huge non-present entries. We + * cannot assert that here due to header dependency issues. + */ + return true; + } + + return false; +} + #define split_huge_pmd(__vma, __pmd, __address) \ do { \ pmd_t *____pmd =3D (__pmd); \ - if (is_swap_pmd(*____pmd) || pmd_trans_huge(*____pmd)) \ + if (pmd_is_huge(*____pmd)) \ __split_huge_pmd(__vma, __pmd, __address, \ false); \ } while (0) @@ -469,10 +495,10 @@ static inline int is_swap_pmd(pmd_t pmd) static inline spinlock_t *pmd_trans_huge_lock(pmd_t *pmd, struct vm_area_struct *vma) { - if (is_swap_pmd(*pmd) || pmd_trans_huge(*pmd)) + if (pmd_is_huge(*pmd)) return __pmd_trans_huge_lock(pmd, vma); - else - return NULL; + + return NULL; } static inline spinlock_t *pud_trans_huge_lock(pud_t *pud, struct vm_area_struct *vma) @@ -744,6 +770,11 @@ static inline struct folio *get_persistent_huge_zero_f= olio(void) { return NULL; } + +static inline bool pmd_is_huge(pmd_t pmd) +{ + return false; +} #endif /* CONFIG_TRANSPARENT_HUGEPAGE */ =20 static inline int split_folio_to_list_to_order(struct folio *folio, diff --git a/include/linux/swapops.h b/include/linux/swapops.h index f1277647262d..41cfc6d59054 100644 --- a/include/linux/swapops.h +++ b/include/linux/swapops.h @@ -471,6 +471,12 @@ static inline pmd_t swp_entry_to_pmd(swp_entry_t entry) } =20 #else /* CONFIG_ARCH_ENABLE_THP_MIGRATION */ +static inline int set_pmd_migration_entry(struct page_vma_mapped_walk *pvm= w, + struct page *page) +{ + BUILD_BUG(); +} + static inline void remove_migration_pmd(struct page_vma_mapped_walk *pvmw, struct page *new) { diff --git a/mm/huge_memory.c b/mm/huge_memory.c index 5876595b00d5..2f0bdc987596 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -2810,8 +2810,9 @@ int move_pages_huge_pmd(struct mm_struct *mm, pmd_t *= dst_pmd, pmd_t *src_pmd, pm spinlock_t *__pmd_trans_huge_lock(pmd_t *pmd, struct vm_area_struct *vma) { spinlock_t *ptl; + ptl =3D pmd_lock(vma->vm_mm, pmd); - if (likely(is_swap_pmd(*pmd) || pmd_trans_huge(*pmd))) + if (likely(pmd_is_huge(*pmd))) return ptl; spin_unlock(ptl); return NULL; diff --git a/mm/memory.c b/mm/memory.c index bf2bbd0dbc97..087f31a291b4 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -1374,7 +1374,7 @@ copy_pmd_range(struct vm_area_struct *dst_vma, struct= vm_area_struct *src_vma, src_pmd =3D pmd_offset(src_pud, addr); do { next =3D pmd_addr_end(addr, end); - if (is_swap_pmd(*src_pmd) || pmd_trans_huge(*src_pmd)) { + if (pmd_is_huge(*src_pmd)) { int err; =20 VM_BUG_ON_VMA(next-addr !=3D HPAGE_PMD_SIZE, src_vma); @@ -1923,7 +1923,7 @@ static inline unsigned long zap_pmd_range(struct mmu_= gather *tlb, pmd =3D pmd_offset(pud, addr); do { next =3D pmd_addr_end(addr, end); - if (is_swap_pmd(*pmd) || pmd_trans_huge(*pmd)) { + if (pmd_is_huge(*pmd)) { if (next - addr !=3D HPAGE_PMD_SIZE) __split_huge_pmd(vma, pmd, addr, false); else if (zap_huge_pmd(tlb, vma, pmd, addr)) { diff --git a/mm/mprotect.c b/mm/mprotect.c index a3e360a8cdec..ab014ce17f9c 100644 --- a/mm/mprotect.c +++ b/mm/mprotect.c @@ -474,7 +474,7 @@ static inline long change_pmd_range(struct mmu_gather *= tlb, goto next; =20 _pmd =3D pmdp_get_lockless(pmd); - if (is_swap_pmd(_pmd) || pmd_trans_huge(_pmd)) { + if (pmd_is_huge(_pmd)) { if ((next - addr !=3D HPAGE_PMD_SIZE) || pgtable_split_needed(vma, cp_flags)) { __split_huge_pmd(vma, pmd, addr, false); diff --git a/mm/mremap.c b/mm/mremap.c index 62b6827abacf..fdb0485ede74 100644 --- a/mm/mremap.c +++ b/mm/mremap.c @@ -850,7 +850,7 @@ unsigned long move_page_tables(struct pagetable_move_co= ntrol *pmc) if (!new_pmd) break; again: - if (is_swap_pmd(*old_pmd) || pmd_trans_huge(*old_pmd)) { + if (pmd_is_huge(*old_pmd)) { if (extent =3D=3D HPAGE_PMD_SIZE && move_pgt_entry(pmc, HPAGE_PMD, old_pmd, new_pmd)) continue; --=20 2.51.0 From nobody Fri Dec 19 21:47:08 2025 Received: from mx0a-00069f02.pphosted.com (mx0a-00069f02.pphosted.com [205.220.165.32]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 326B52765C3; Sat, 8 Nov 2025 17:11:25 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=205.220.165.32 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1762621887; cv=fail; b=YYyf1C5j48qSK7GODc9R4FU9IPXTgvMc17yB3lk+WDpcyKY014+tvBUZZRFb59YUUzWITEOR6czRksEwKjB586pGEpzPPljSAuslxCn+dfUXh9oko9982jXfUooUwvlzQNDgc9MDGx+dMq5OIhc0xVOONFHR+x//S6eh2Jhgjgo= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1762621887; c=relaxed/simple; bh=dJwfRNiI3WXPCdC9hAF5POoUcmBmxlnAw611TSol11U=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: Content-Type:MIME-Version; b=cr+wx+JXeBMF4fk/t3vQ5RaVsqHQGcOi9y/gmUUz0oWg0x8mNEZO04P39bPGMr8FiYDCGCetH08WmhJfTR+75PBR47rI1cPCdv/k/zp4NTMkx51hoZ0OvlpyDsbgeOJD01auxUrdEwzTRSQAtn/fqTBbITxXZvKUpe7WIwVuEzI= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=oracle.com; spf=pass smtp.mailfrom=oracle.com; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b=M8kJnPZ/; dkim=pass (1024-bit key) header.d=oracle.onmicrosoft.com header.i=@oracle.onmicrosoft.com header.b=BrXHiydO; arc=fail smtp.client-ip=205.220.165.32 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=oracle.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=oracle.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b="M8kJnPZ/"; dkim=pass (1024-bit key) header.d=oracle.onmicrosoft.com header.i=@oracle.onmicrosoft.com header.b="BrXHiydO" Received: from pps.filterd (m0246629.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 5A8Gj7fC004131; Sat, 8 Nov 2025 17:09:34 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=cc :content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s= corp-2025-04-25; bh=3biM3jPk/T84wrFDEkrtg6nQEHWQ6QNNYccvVkfwbPk=; b= M8kJnPZ/BWkbS3fjW78BxI1er1G2KYM2SGUFKQHy2aCCmjjxrVPF6SnV6lL/47yb a+j1lVD80x5VLz1P17Y6MyoBYvvLNM+hCmbmDrNBb6Iy+6El99zP/Y7Ix+EfSu05 L2gpA2ch4OPnKgyIZt1HDNqGY7gCWu2j82U8rbrNjM/wn6SRbt3hSO6AIX/vJhln mK7tNjzfIIDChndX2DByIdNQABd0jExIHdgX4q6dIb8PE6OB8aIHvuEbiKvvY2CK WxJuNejOLsa7VGamXZkcMXVLwVfQaGM7wIm18Z3Qpju8e1EdlpZ3Dnc6y21qIypy 7uxzHsCkQgS7Ri4WZLfgcA== Received: from phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta01.appoci.oracle.com [138.1.114.2]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 4aa0jy8etf-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Sat, 08 Nov 2025 17:09:34 +0000 (GMT) Received: from pps.filterd (phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (8.18.1.2/8.18.1.2) with ESMTP id 5A8BijdH007604; Sat, 8 Nov 2025 17:09:33 GMT Received: from bl2pr02cu003.outbound.protection.outlook.com (mail-eastusazon11011002.outbound.protection.outlook.com [52.101.52.2]) by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTPS id 4a9va76n3y-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Sat, 08 Nov 2025 17:09:33 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=dOTfIvSvy7VEfVoDezXP+3qrV245+KgJ1iqNvUlbzA0VNOeCr22MBY10g3twKg2ImViFBBc0wThF2QwaIj8cLU7s2xtjldsN3y+P3UOng7QTZf2kVhe9MI9wihTco8JwavRkWJklUbk9F/s0pGjC8QL0liYHU3TRPh/VXVcjVDLJvGMgFELm0C3n3N2lpB6uE4Glo0l+115psK7OxtT2fbPFEX/4v4AzdV3yleAsDKVCcRV1cThQG2EnMDUvJTEHIY1p6P9RHUByFQBLBuieluxJ0YKdV7Bv121e2vnXYT8f47Np88cLpe+02aN8t8nEc4f/ZmTJy4rIi2IPdt0dpw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=3biM3jPk/T84wrFDEkrtg6nQEHWQ6QNNYccvVkfwbPk=; b=NTF0fWHlRWkz/p3weoYYEy2ppQlrem+6OMDWeQlLWOhw3Tzi6LdzLGFhRKTW7D5i7jvVAymiJ7yIct94r/6AcSb2LfwvI/4/Jm9zmHoupqmQkDusfIo8isIBChqSVuq1x5pYJVpYl4aXa+VvM6sFzj7kvCg2kpSQRSi1/MQo1pGA+4cEUiFC7NecSiL4uo49sikuiQ5d/eN5kV0CI71V4526/KyvImdVL5RATQjY5G7dHCd0TxIEpZwZXRUshB+AqaHPOa5JJf9Ylkm1g2QKRYanAGM+vXOp/IDyJ3/2rdES1hfy/P99BwK0ppcD/DyyY5lRIqkqGZhYpQcMwT2Nmg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=3biM3jPk/T84wrFDEkrtg6nQEHWQ6QNNYccvVkfwbPk=; b=BrXHiydOmNnv1Muz8aK2rSznTDnZxsE8wduq9tFfYZre8ajmwAid+xDa0zNlI5W4kr2UrZODuwa2WtaFrzzaLcyBGnEVrcqS42XzJ5laUNqP4ezLomEW133VEXvjfGG2YVm2Iwhxp5ObhLSwKT0VhQXlAf/6sLTp8wrxqyNYWtw= Received: from DM4PR10MB8218.namprd10.prod.outlook.com (2603:10b6:8:1cc::16) by SJ5PPF04D2D7FA7.namprd10.prod.outlook.com (2603:10b6:a0f:fc02::785) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9298.15; Sat, 8 Nov 2025 17:09:24 +0000 Received: from DM4PR10MB8218.namprd10.prod.outlook.com ([fe80::2650:55cf:2816:5f2]) by DM4PR10MB8218.namprd10.prod.outlook.com ([fe80::2650:55cf:2816:5f2%7]) with mapi id 15.20.9298.010; Sat, 8 Nov 2025 17:09:24 +0000 From: Lorenzo Stoakes To: Andrew Morton Cc: Christian Borntraeger , Janosch Frank , Claudio Imbrenda , David Hildenbrand , Alexander Gordeev , Gerald Schaefer , Heiko Carstens , Vasily Gorbik , Sven Schnelle , Peter Xu , Alexander Viro , Christian Brauner , Jan Kara , Arnd Bergmann , Zi Yan , Baolin Wang , "Liam R . Howlett" , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Lance Yang , Muchun Song , Oscar Salvador , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Matthew Brost , Joshua Hahn , Rakie Kim , Byungchul Park , Gregory Price , Ying Huang , Alistair Popple , Axel Rasmussen , Yuanchu Xie , Wei Xu , Kemeng Shi , Kairui Song , Nhat Pham , Baoquan He , Chris Li , SeongJae Park , Matthew Wilcox , Jason Gunthorpe , Leon Romanovsky , Xu Xin , Chengming Zhou , Jann Horn , Miaohe Lin , Naoya Horiguchi , Pedro Falcato , Pasha Tatashin , Rik van Riel , Harry Yoo , Hugh Dickins , linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-s390@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, linux-arch@vger.kernel.org, damon@lists.linux.dev Subject: [PATCH v2 12/16] mm: remove remaining is_swap_pmd() users and is_swap_pmd() Date: Sat, 8 Nov 2025 17:08:26 +0000 Message-ID: <37c1c28bb6e159e9ad9c52955580971b2462dbfa.1762621568.git.lorenzo.stoakes@oracle.com> X-Mailer: git-send-email 2.51.0 In-Reply-To: References: Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: LO4P265CA0157.GBRP265.PROD.OUTLOOK.COM (2603:10a6:600:2c7::14) To DM4PR10MB8218.namprd10.prod.outlook.com (2603:10b6:8:1cc::16) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DM4PR10MB8218:EE_|SJ5PPF04D2D7FA7:EE_ X-MS-Office365-Filtering-Correlation-Id: f3129fff-693d-4185-a3a5-08de1ee990f3 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|1800799024|7416014|376014|366016; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?H74mDRNhz2yI+4XbHgHwH3wKRQ+Qk1I0t6JnkWJWvrxXm63SMUD9k/wz5KWl?= =?us-ascii?Q?7pOKtv5OKW2BiBrZq+AogSiAYdlj3n8Z6w5Iw0IXREVTktyUrHweDvQGeaN1?= =?us-ascii?Q?93ZzrTK4x5WhyplFyOIYlEXm5m98GuJ+M2VZE9Tmb2GYwBiyZ6hExZSHMbEf?= =?us-ascii?Q?J3a3qXRpddrICdrgwUrLJ6P5SxOZu2Py96DdQZ2y6aV49+vyyDCVIHJE6Yw+?= =?us-ascii?Q?vXAGElJHLktazZ5WrzetQ+ei2+LFlF+nk1a9h5lipYwYUyVgtZCz0Bx2LieT?= =?us-ascii?Q?Iva6MkiKpBg+ITqw1HphlQ5ts052oQU3i34Cfa6YJALENRRQaozzpF85R3cv?= =?us-ascii?Q?iNuVL9O9fMWFwQqk+UhoJirniY2hiNx1MHm+4qmB+xr+8MnZy87C25+pZcQi?= =?us-ascii?Q?XSBGm8hJyrzHCvfSFfiYiTOnQF3IR7k6lr1osnG/qE9Hzu3Pb1X7BdMMT7If?= =?us-ascii?Q?UDMI61eCf6o/xjFgeMUgBsIfzAEubQchbFihPR40kPnCvjHTySfTRSrBg51H?= =?us-ascii?Q?9pN0+WwYOXXRzP+wCyPaYeMsxzBFzAm11aLw6qYjDZAfqZAYT1I3caMQ2kQp?= =?us-ascii?Q?QcJCbNp3SfpGJjJqLyhY8oE4m5vs3N1EOYC/zxUs1zZt03brfVqn8Tqddjsp?= =?us-ascii?Q?doxGsWD6RnK/UXmdLrkc50NVmEmoqam6tsEju5RZ92fJBIlryPxSp6wCE/mg?= =?us-ascii?Q?/YwWUFl1qgiaZx1Mr/MPwOh0RkbT2naSBqouTlGTHp/9ti7ShbZF1Ai7IWGD?= =?us-ascii?Q?jT+1LMxr4LP2euvpdobru5p71fWka4BXXlKf+Qq4AOKJi/bZqEoBNhzKFeyW?= =?us-ascii?Q?/EfJAuj4kQ2tVmVHsTxwa1XndYfFEsqBTuNf9MjP+JlZv2ihDp/fH/kJQip1?= =?us-ascii?Q?9noqZgjbtfnX/yHTamConLMP0HNM4optoVKFlw5bvx0CEWApcL8o1/42lmYP?= =?us-ascii?Q?VJjBKsMY1kC2TSR9nR1PNgCTygkGwtNIqdaEqTAbzYnU+QuYvhsXDyT+YP9Q?= =?us-ascii?Q?nA3hbZzRZ865mbdlIKnRDm2h9koIgMubImHKyCC+PCwyN2o2Ih4W5KoI+i15?= =?us-ascii?Q?p1jt382WR7EX4mZXHWS3d3IN/wnGr06WLkJuR0fuBpwmrInKcU7qJI/Vqltl?= =?us-ascii?Q?mOyAW20XKjPgSvKHxbXx1EwwhFsqbnnCHYHNI54jLSjBmNBqn8zT00Qn1wGN?= =?us-ascii?Q?J65iJ958BUFgxTbV7P8kEVWVHkCb7jByfFzJgmSOdhYuNNofQuVUrLJPbLv0?= =?us-ascii?Q?UbhHP0z6BygkJKZsLrkMTwuQQ2w4vwlHxcue8De6SS1SDFv2qMrUsxsrSnSR?= =?us-ascii?Q?kIMtbNZlJxqrYEu7UF1SUbO842cv//85F593Bt76YpEYhgjX17I7zK9q800x?= =?us-ascii?Q?hdzT/pmaCfx4KzWmFhgeJkDO/O9m4j8w3PJkxPImgAzDUWRUQsRXQhyfgEkw?= =?us-ascii?Q?chkr/j4fIcUMEsA/wnlxe6hx3sfdvp9S?= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DM4PR10MB8218.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(1800799024)(7416014)(376014)(366016);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?odXf4rtQ70W0QzCFdSFbmObmBCcm6gM45K5WIdJz8fIg6N5kq4SIsZWY9uwn?= =?us-ascii?Q?8cw8SzA6JvB2D3DvxnT9qLT8n3NADffUXN/CVgZ62pNrwkjdbYT/rP5+RBZE?= =?us-ascii?Q?VfFd8H4zD8j7cfyMDcq+0naXyqMWDNE9yGvN60mgZf6i4+6iN77UCyuwwPh6?= =?us-ascii?Q?vLb/6yKpTnLVqMh/3uSSlEjFN06vJPjsmkIihAvAvee1y/Cr+NjzDqtwGUCE?= =?us-ascii?Q?uaVtlgSI7cSj1jEqo6NMSzCXlTAweFjFPpb5P36ihetod76WDk9L6vcHMBKe?= =?us-ascii?Q?B/OTIie6GXw4DSIxUM/l+HlU+IbMoPVLn2d2kM9Rlgu1YHbvz2zScPP3yQlq?= =?us-ascii?Q?NfQs8iFSNXp5qbkJZUnxzCT7pRp2L4cI/Fi+y9bHzwjpkuqZeM1LF3BBAwnq?= =?us-ascii?Q?XOJxQO1K3y8zIo87OSzLyG4/t6Srxyqqlg/72C1xpZlFmYg18/DjB9pQhIyJ?= =?us-ascii?Q?96Zjg8YXQQ1Gk0IIiZIKI0ygoD3dikHpVcLPw5bbLQZQatjdeBVisxgMUFQx?= =?us-ascii?Q?g7/ZerCXKpdR5jm9J2hhkmjyyZ6jFUR64Sq7RAwpZ1wf8TukS0m76S/c2h0b?= =?us-ascii?Q?JJtxxK+03Yz+bKjRf3WDu2E7IyeOlU02DRVxoFC6smFrJAK31sZEp3tVCk+u?= =?us-ascii?Q?iNhNadY0jeZIpFeWnmkEVLz4PJHsKItcYAHK9LdJOeFZkcpDN0afHJtQfkNX?= =?us-ascii?Q?EpP1ILgjNtArmdNEYHi+iKUzz/XBdz8qeKCHUaNh3DllMHjDBl45pOLmX5/q?= =?us-ascii?Q?1M2lymx71Kdef12uzmMjlE/Yf4JDPjwEaWM007GimwpK1hdhimr7f1ar5ZmI?= =?us-ascii?Q?T43fG5uDLE3WDAF/3grN7nPeqYOQsVssc0n6/CEhyiLBhQ3NIsvqLNCsc+n4?= =?us-ascii?Q?PcI1/szQkXsHybWE7b726RFFF+PaVvtZzMpV7RCpjgbBickG5K3XpsgmZo0y?= =?us-ascii?Q?wKsvHzM2hgvvdPQmN34V0uhJ5JPMoMYfgu82A9NFjbMsZO41tZcDuJTLutD7?= =?us-ascii?Q?sBamMEJKhs52owH1axXerWDcWW1hUK+wlQMBaDlhbQJJY6Bqav9fY2QZ5oRG?= =?us-ascii?Q?DjQOaAgkl0BoF8Vg9CVg9OURa05dczbhXkfR8DxgoAPkHkFB6UOwhVY822M9?= =?us-ascii?Q?AIrOYz+8Fq8tFFmDewS4RK2n1L8hVLasJQx5CxSBw78uCS8Yf+eDUNFbBykT?= =?us-ascii?Q?oEUo3TAsV8nE/QOwGT9lozPYDkjxPlAoZjF8l9UCgo9wFokT2KQRQd5qtIlS?= =?us-ascii?Q?6KyuC7/7ahHyVJCQXIbqv8w8zYhx0Inw5f1nRjwa8jGpyhUP4eXdzCB2ED8y?= =?us-ascii?Q?XLZa9OY/eNxE+/KSU3qNCn32WCMZ7TBSkRXQXTjALFCbZfzqL/LkkYe0NmOb?= =?us-ascii?Q?fE0TCNiHvHbZk7+prw+87K/Xlaq2MTrjUOSdbYpEV83jmO3adCJxfjoVL4Jv?= =?us-ascii?Q?19CrqZdNyKB+OEUbHew7Z2IGMD2mWxo2vnhrUmicN8S8QUiWY2JADsQ3IbI2?= =?us-ascii?Q?u9zdGVJkFb2M5z/Z/LTpKvNtptOxE64GwgcCKkTyYDX3N5z2KmnsM7W3jSdE?= =?us-ascii?Q?HmA0awWrHwnSMItTQrt1OQT4DhBUI+QBG3OUfMtpfR8wJT1qklApeY0EAapL?= =?us-ascii?Q?4A=3D=3D?= X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: I4dcnvOX6jKLSwYUDBvbBPYgofnVL2RBh+afbU9m+zxxMIVuty1psTdvVYmeNUSWJaKdZlU/sfpYLgoHBwo5jklK0KiaxxBYWIUHxuPS/z4KzktTzLszA+lkyCTdgHRXwz6oMg1Vh5X/yZLnWFi/Tga99awbpfJDjcSjZV8xadvC8dI5bpVPWrgWuY61/Xe88bD1u3qYx3YAOE56jpoTfiOdggxIHjfoI8QJO+zLSvjQEVHlnIR5f8xhLRrzIbkyCVNI5qoz4Rfm8A9ekPOOBgIk805I3ghvmtM3orEj9tUYdylmWYkDBLCDw2iempXHaIlidk3kLqhUoLYquM2EGN2IQwZ2zIWNSyCecjZDwtw+6NnL9/Vek5/wNG6dhqcoFgLf8T5NhzuK8604GMQ0j6O1qOiQVNRvO0q4AuLFwLH2t5E7PQKUcma0NzQ67O/qreHtCuEKCdhy6A++qkS4gH269EPsHol6bY3JST25zb0OTM00Q9fFTtVge+z+iS2UGElqmXfcdDHXqFm7AIsMYW8b9SGYQ1zDXZvTCX0lFoMwT5WhAAunUaNXDDprf/a6Lw0f8it65cdmN0EU7Mels5hK37pDsPKhLGu6GAecGn4= X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: f3129fff-693d-4185-a3a5-08de1ee990f3 X-MS-Exchange-CrossTenant-AuthSource: DM4PR10MB8218.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 Nov 2025 17:09:24.2204 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: qKE/Kd+tv4cyUohNPFieaJ3Nyib8EH7eW7C+F7KI0Xx3y/y/KuNFd2MbF9TWXv60pqJuSeff52wmUFG3OSI1mP9nwj0sDxA3ys/0gyOwB8k= X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ5PPF04D2D7FA7 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1121,Hydra:6.1.9,FMLib:17.12.100.49 definitions=2025-11-08_04,2025-11-06_01,2025-10-01_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 adultscore=0 malwarescore=0 spamscore=0 mlxscore=0 bulkscore=0 suspectscore=0 mlxlogscore=999 phishscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2510240000 definitions=main-2511080138 X-Proofpoint-Spam-Details-Enc: AW1haW4tMjUxMTA4MDA1MCBTYWx0ZWRfX/QdaXZo69Qd/ hJ8TX+P4lkkZmCjNrwbfQjK0jCXf6xxdFlJJTjBC+23YWi1fp9UW3OLNx2DXRIEO+CPcqIKzwwp EhxTkLmFtB/j4DkNA61Mv8mE/Ftp1k3m1su3zAUhO1r4W9aZj8TpxZlP5AKXYIxDx0jOxo8pleE rT2e361270wiRrifmFl4by4IeOIpADq3Udh9sxeNX3X1lL3hP3OccTwtI7pc+kQFOF15GNlCxSm 3sj8DqVSTcd0P1FVepqqb3F8/nad+RFy+ia5T6kfCUhggdUvq6/8/68km1b8ty7FsxbD+M9nUU5 7i8yfrkDrjEeEcHn4oG7CVyg2OFgoW9DaKVlz664uXwyE8C/27kK3nugrsVAez+3wRVkRTc5cNZ 7n2JTP8QyzMWBwtjD1wYOFRXtYnJZA== X-Proofpoint-ORIG-GUID: rNwLXU3ZElVPKduMOgyiyv--k3gIkyfo X-Authority-Analysis: v=2.4 cv=IpkTsb/g c=1 sm=1 tr=0 ts=690f794e cx=c_pps a=XiAAW1AwiKB2Y8Wsi+sD2Q==:117 a=XiAAW1AwiKB2Y8Wsi+sD2Q==:17 a=6eWqkTHjU83fiwn7nKZWdM+Sl24=:19 a=z/mQ4Ysz8XfWz/Q5cLBRGdckG28=:19 a=lCpzRmAYbLLaTzLvsPZ7Mbvzbb8=:19 a=xqWC_Br6kY4A:10 a=6UeiqGixMTsA:10 a=GoEa3M9JfhUA:10 a=VkNPw1HP01LnGYTKEx00:22 a=yPCof4ZbAAAA:8 a=xbP9_UvFYPnkkC6NLJ8A:9 a=UhEZJTgQB8St2RibIkdl:22 a=Z5ABNNGmrOfJ6cZ5bIyy:22 a=QOGEsqRv6VhmHaoFNykA:22 X-Proofpoint-GUID: rNwLXU3ZElVPKduMOgyiyv--k3gIkyfo Content-Type: text/plain; charset="utf-8" Update copy_huge_pmd() and change_huge_pmd() to use pmd_is_valid_softleaf() - as this checks for the only valid non-present huge PMD states. Also update mm/debug_vm_pgtable.c to explicitly test for a valid leaf PMD entry (which it was not before, which was incorrect), and have it test against pmd_is_huge() and pmd_is_valid_softleaf() rather than is_swap_pmd(). With these changes done there are no further users of is_swap_pmd(), so remove it. Signed-off-by: Lorenzo Stoakes --- include/linux/huge_mm.h | 9 --------- mm/debug_vm_pgtable.c | 25 +++++++++++++++---------- mm/huge_memory.c | 5 +++-- 3 files changed, 18 insertions(+), 21 deletions(-) diff --git a/include/linux/huge_mm.h b/include/linux/huge_mm.h index 403e13009631..79f16b5aa5f0 100644 --- a/include/linux/huge_mm.h +++ b/include/linux/huge_mm.h @@ -486,11 +486,6 @@ void vma_adjust_trans_huge(struct vm_area_struct *vma,= unsigned long start, spinlock_t *__pmd_trans_huge_lock(pmd_t *pmd, struct vm_area_struct *vma); spinlock_t *__pud_trans_huge_lock(pud_t *pud, struct vm_area_struct *vma); =20 -static inline int is_swap_pmd(pmd_t pmd) -{ - return !pmd_none(pmd) && !pmd_present(pmd); -} - /* mmap_lock must be held on entry */ static inline spinlock_t *pmd_trans_huge_lock(pmd_t *pmd, struct vm_area_struct *vma) @@ -693,10 +688,6 @@ static inline void vma_adjust_trans_huge(struct vm_are= a_struct *vma, struct vm_area_struct *next) { } -static inline int is_swap_pmd(pmd_t pmd) -{ - return 0; -} static inline spinlock_t *pmd_trans_huge_lock(pmd_t *pmd, struct vm_area_struct *vma) { diff --git a/mm/debug_vm_pgtable.c b/mm/debug_vm_pgtable.c index fff311830959..608d1011ce03 100644 --- a/mm/debug_vm_pgtable.c +++ b/mm/debug_vm_pgtable.c @@ -74,6 +74,7 @@ struct pgtable_debug_args { unsigned long fixed_pte_pfn; =20 swp_entry_t swp_entry; + swp_entry_t leaf_entry; }; =20 static void __init pte_basic_tests(struct pgtable_debug_args *args, int id= x) @@ -745,7 +746,7 @@ static void __init pmd_soft_dirty_tests(struct pgtable_= debug_args *args) WARN_ON(pmd_soft_dirty(pmd_clear_soft_dirty(pmd))); } =20 -static void __init pmd_swap_soft_dirty_tests(struct pgtable_debug_args *ar= gs) +static void __init pmd_leaf_soft_dirty_tests(struct pgtable_debug_args *ar= gs) { pmd_t pmd; =20 @@ -757,15 +758,16 @@ static void __init pmd_swap_soft_dirty_tests(struct p= gtable_debug_args *args) return; =20 pr_debug("Validating PMD swap soft dirty\n"); - pmd =3D swp_entry_to_pmd(args->swp_entry); - WARN_ON(!is_swap_pmd(pmd)); + pmd =3D swp_entry_to_pmd(args->leaf_entry); + WARN_ON(!pmd_is_huge(pmd)); + WARN_ON(!pmd_is_valid_softleaf(pmd)); =20 WARN_ON(!pmd_swp_soft_dirty(pmd_swp_mksoft_dirty(pmd))); WARN_ON(pmd_swp_soft_dirty(pmd_swp_clear_soft_dirty(pmd))); } #else /* !CONFIG_TRANSPARENT_HUGEPAGE */ static void __init pmd_soft_dirty_tests(struct pgtable_debug_args *args) {= } -static void __init pmd_swap_soft_dirty_tests(struct pgtable_debug_args *ar= gs) { } +static void __init pmd_leaf_soft_dirty_tests(struct pgtable_debug_args *ar= gs) { } #endif /* CONFIG_TRANSPARENT_HUGEPAGE */ =20 static void __init pte_swap_exclusive_tests(struct pgtable_debug_args *arg= s) @@ -818,7 +820,7 @@ static void __init pte_swap_tests(struct pgtable_debug_= args *args) } =20 #ifdef CONFIG_ARCH_ENABLE_THP_MIGRATION -static void __init pmd_swap_tests(struct pgtable_debug_args *args) +static void __init pmd_softleaf_tests(struct pgtable_debug_args *args) { swp_entry_t arch_entry; pmd_t pmd1, pmd2; @@ -827,15 +829,16 @@ static void __init pmd_swap_tests(struct pgtable_debu= g_args *args) return; =20 pr_debug("Validating PMD swap\n"); - pmd1 =3D swp_entry_to_pmd(args->swp_entry); - WARN_ON(!is_swap_pmd(pmd1)); + pmd1 =3D swp_entry_to_pmd(args->leaf_entry); + WARN_ON(!pmd_is_huge(pmd1)); + WARN_ON(!pmd_is_valid_softleaf(pmd1)); =20 arch_entry =3D __pmd_to_swp_entry(pmd1); pmd2 =3D __swp_entry_to_pmd(arch_entry); WARN_ON(memcmp(&pmd1, &pmd2, sizeof(pmd1))); } #else /* !CONFIG_ARCH_ENABLE_THP_MIGRATION */ -static void __init pmd_swap_tests(struct pgtable_debug_args *args) { } +static void __init pmd_softleaf_tests(struct pgtable_debug_args *args) { } #endif /* CONFIG_ARCH_ENABLE_THP_MIGRATION */ =20 static void __init swap_migration_tests(struct pgtable_debug_args *args) @@ -1229,6 +1232,8 @@ static int __init init_args(struct pgtable_debug_args= *args) max_swap_offset =3D swp_offset(pte_to_swp_entry(swp_entry_to_pte(swp_entr= y(0, ~0UL)))); /* Create a swp entry with all possible bits set while still being swap. = */ args->swp_entry =3D swp_entry(MAX_SWAPFILES - 1, max_swap_offset); + /* Create a non-present migration entry. */ + args->leaf_entry =3D make_writable_migration_entry(~0UL); =20 /* * Allocate (huge) pages because some of the tests need to access @@ -1318,12 +1323,12 @@ static int __init debug_vm_pgtable(void) pte_soft_dirty_tests(&args); pmd_soft_dirty_tests(&args); pte_swap_soft_dirty_tests(&args); - pmd_swap_soft_dirty_tests(&args); + pmd_leaf_soft_dirty_tests(&args); =20 pte_swap_exclusive_tests(&args); =20 pte_swap_tests(&args); - pmd_swap_tests(&args); + pmd_softleaf_tests(&args); =20 swap_migration_tests(&args); =20 diff --git a/mm/huge_memory.c b/mm/huge_memory.c index 2f0bdc987596..d1a5c5f01d94 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -1875,7 +1875,8 @@ int copy_huge_pmd(struct mm_struct *dst_mm, struct mm= _struct *src_mm, ret =3D -EAGAIN; pmd =3D *src_pmd; =20 - if (unlikely(thp_migration_supported() && is_swap_pmd(pmd))) { + if (unlikely(thp_migration_supported() && + pmd_is_valid_softleaf(pmd))) { copy_huge_non_present_pmd(dst_mm, src_mm, dst_pmd, src_pmd, addr, dst_vma, src_vma, pmd, pgtable); ret =3D 0; @@ -2562,7 +2563,7 @@ int change_huge_pmd(struct mmu_gather *tlb, struct vm= _area_struct *vma, if (!ptl) return 0; =20 - if (thp_migration_supported() && is_swap_pmd(*pmd)) { + if (thp_migration_supported() && pmd_is_valid_softleaf(*pmd)) { change_non_present_huge_pmd(mm, addr, pmd, uffd_wp, uffd_wp_resolve); goto unlock; --=20 2.51.0 From nobody Fri Dec 19 21:47:08 2025 Received: from mx0a-00069f02.pphosted.com (mx0a-00069f02.pphosted.com [205.220.165.32]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4437123F27B; Sat, 8 Nov 2025 17:11:21 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=205.220.165.32 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1762621885; cv=fail; b=Y9coabP7bDaz1rLgMHKUE3mjTDTWuwY+gNv2iTcNXwFEWnI9u8p81tnCak5xPEI08i//rljGadactphpMfrn/gwkmblg5FuuvOyxzJmKEcIR9D5B0Imj+PIMK5qeNipDEdZ87OUIup2n8DOZU0nGeG6H1Cx1KqAEex9qdT0vw3g= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1762621885; c=relaxed/simple; bh=Tvs+4my9yc+vCQg1GhCyFmSQ7xuioNkL6B9uK3iRXfw=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: Content-Type:MIME-Version; b=e7vUJUJadra/Vv1hGUVSuLs2JXU1mkPtD0QcKSbsA7KI+lH8zswx0E6n7aicH7thxJRYG4XPJq3YBp7PDWJb7PfGrytiyH5FKB5RO2W1HNiOakVNsY4nmDZuu7BEp1yh+L4+JglZQR8nTC3cIkaqIzIgpAsmqeSis6R+D5XbVLA= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=oracle.com; spf=pass smtp.mailfrom=oracle.com; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b=FRNp5ks7; dkim=pass (1024-bit key) header.d=oracle.onmicrosoft.com header.i=@oracle.onmicrosoft.com header.b=eUUVSxCp; arc=fail smtp.client-ip=205.220.165.32 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=oracle.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=oracle.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b="FRNp5ks7"; dkim=pass (1024-bit key) header.d=oracle.onmicrosoft.com header.i=@oracle.onmicrosoft.com header.b="eUUVSxCp" Received: from pps.filterd (m0333521.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 5A8Fg8vu024800; Sat, 8 Nov 2025 17:09:36 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=cc :content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s= corp-2025-04-25; bh=Eb9WAlRir1/Mp42rNrETH8HLWzdOqSGxrC9P0/htsIM=; b= FRNp5ks78TXI8UoAG98FDfRrk1vrfcEDZgMcl6iYuuGlG5Sg3p8rlS5QkwQPU4pM A3YUq/+J05Y7Im7vAi8hAE7AbOtjbz74L2gNcI4aFT0PXZClFP92jfyLZwo+O3aQ O6MRb90RIZQJ72o7L54iUtvaqJ6UHnkNV0glU3EKN7EMw+saWhjx1y+pIp9HAQjE SXsaBc+ZXpEsLBUeE89bRqYm35+Uh4BIJtN+VETxPt0q2sLvQgSOWmttGq0YjvYX AJ2p41NhAubID2GrRbSTkjY+WyFh+YqB6mPY2Wkm5GhP/TnOetyj2S1M50wB/Unp lhv7s2MRLns4RAveFN7K9w== Received: from phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta01.appoci.oracle.com [138.1.114.2]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 4a9vs8gnmq-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Sat, 08 Nov 2025 17:09:35 +0000 (GMT) Received: from pps.filterd (phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (8.18.1.2/8.18.1.2) with ESMTP id 5A8BijdJ007604; Sat, 8 Nov 2025 17:09:35 GMT Received: from bl2pr02cu003.outbound.protection.outlook.com (mail-eastusazon11011002.outbound.protection.outlook.com [52.101.52.2]) by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTPS id 4a9va76n3y-2 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Sat, 08 Nov 2025 17:09:35 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=kL3m40sP016YAC2xrFNaph1/Denk3mVyz9mDYvBuX4dZniu24+gaAGKrQJdFK1amdHvoBPccQOxsvIk+TNtVXfAtaAgHQMkVANwv3ZOHfQK2/TITaopVv3+Flr49Xxl2S12PoIsKZ1Uv/BcaaVxDyuyVmLdf2yTk+CpG+cyVkWCyDMGtBlR3eoeXEmfkixpkWszw44msG9GYk1asPc3QlTHz2DlY9rcAv1PAi5hlOH/yKPNi8qeyJyOJlgTprLq48MSWXNGyJLRyBh+9PNUxN3ODcu4iXNparc3n7pHmaudA0XACvNL5dzBaV6fPSfl9rQ+9b7/NGOYBMTFfXhUynQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=Eb9WAlRir1/Mp42rNrETH8HLWzdOqSGxrC9P0/htsIM=; b=ro5R0BHgNU/DaDimNBQpE/vdUeN2iVZxBEOcOwmXzitOpdh9AfwN8PC/KTsG1WIIUDwWKz3SH0cQUFdnPNGPOenRQXQnQVrdGG/lcjr6Gw2OS22cMLKQYeJtQMLayADhpeIQZsyIYU4p+OQNB2PCRwU53/IyPbs1MNM93es0Yx8Gxv9rY643aYn/nKqUaGcmOYmSYeWX2TXDbg4zjwCvA05jHyyhkIBOAsxLwrlHDJfFK5I4IHm9seXSoH2CgZjIGGGlU3I6Y5LPojZjFa9R0QtlaHkZfHeACH/z/s/Ty1sGiXnulya/zNMkp8YqaCkHpbeBj1RJ0T35ZLKv1IZ/Lg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Eb9WAlRir1/Mp42rNrETH8HLWzdOqSGxrC9P0/htsIM=; b=eUUVSxCp5NnCYCTDuA668+L8O0snGPGxFNkw2VM3r/Y+uzRItK1caZxkTKVNQ/ruuquncurstWCozTtw7Y7WvBrSC6rtbI7Y+zu69lzIUNHpRmHmVU0oGJMRplY0x5tkF/rTeDO618/a36F0TbatDsiZjAVmgR2yvRNJ12K4MT4= Received: from DM4PR10MB8218.namprd10.prod.outlook.com (2603:10b6:8:1cc::16) by SJ5PPF04D2D7FA7.namprd10.prod.outlook.com (2603:10b6:a0f:fc02::785) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9298.15; Sat, 8 Nov 2025 17:09:28 +0000 Received: from DM4PR10MB8218.namprd10.prod.outlook.com ([fe80::2650:55cf:2816:5f2]) by DM4PR10MB8218.namprd10.prod.outlook.com ([fe80::2650:55cf:2816:5f2%7]) with mapi id 15.20.9298.010; Sat, 8 Nov 2025 17:09:27 +0000 From: Lorenzo Stoakes To: Andrew Morton Cc: Christian Borntraeger , Janosch Frank , Claudio Imbrenda , David Hildenbrand , Alexander Gordeev , Gerald Schaefer , Heiko Carstens , Vasily Gorbik , Sven Schnelle , Peter Xu , Alexander Viro , Christian Brauner , Jan Kara , Arnd Bergmann , Zi Yan , Baolin Wang , "Liam R . Howlett" , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Lance Yang , Muchun Song , Oscar Salvador , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Matthew Brost , Joshua Hahn , Rakie Kim , Byungchul Park , Gregory Price , Ying Huang , Alistair Popple , Axel Rasmussen , Yuanchu Xie , Wei Xu , Kemeng Shi , Kairui Song , Nhat Pham , Baoquan He , Chris Li , SeongJae Park , Matthew Wilcox , Jason Gunthorpe , Leon Romanovsky , Xu Xin , Chengming Zhou , Jann Horn , Miaohe Lin , Naoya Horiguchi , Pedro Falcato , Pasha Tatashin , Rik van Riel , Harry Yoo , Hugh Dickins , linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-s390@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, linux-arch@vger.kernel.org, damon@lists.linux.dev Subject: [PATCH v2 13/16] mm: remove non_swap_entry() and use softleaf helpers instead Date: Sat, 8 Nov 2025 17:08:27 +0000 Message-ID: <82298e05b0c33bd95fd07b71845286d4df880a1f.1762621568.git.lorenzo.stoakes@oracle.com> X-Mailer: git-send-email 2.51.0 In-Reply-To: References: Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: LO2P265CA0501.GBRP265.PROD.OUTLOOK.COM (2603:10a6:600:13b::8) To DM4PR10MB8218.namprd10.prod.outlook.com (2603:10b6:8:1cc::16) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DM4PR10MB8218:EE_|SJ5PPF04D2D7FA7:EE_ X-MS-Office365-Filtering-Correlation-Id: c630c2aa-a906-4838-a4fa-08de1ee99298 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|1800799024|7416014|376014|366016; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?NWeIBDPaMG2FatJdvEyMjrZCbD2QCUgBZUbPz08LpucKeEqnZs9h10mpaaft?= =?us-ascii?Q?1+59gKd/Ea9Un/NobxMWgxxe5wjGyrXKWoPPChgo7AWcRPG6HUPosRE7wLan?= =?us-ascii?Q?k4FY7r9LEcGVayRnWDO+vt+UsDrypo4WFpQ6GilBGDhvXquNmgSL7nrnoEde?= =?us-ascii?Q?1oENaMvUMCGyIfMATtuz9BHZP+YrhcmGcfBH4oPZtVY7wsulf0xQ1YjNMnHV?= =?us-ascii?Q?Oy6/rgCUB9sl0fkVfmxH5WZH61VaJHW5odWMa+nDZiIr6RRVJvcD9tS/j9nx?= =?us-ascii?Q?0TxAsqI4/C6uAl5cPCxlje72cCKJn4D0dJhv0eK/74P9BDWiJsGwJIfhJbdj?= =?us-ascii?Q?s9gpeC0hanO9blLBe3YzoiMQ6oBpSIB2nQ9Nfwo9u8hE9hB1U9aG+WcmVUhX?= =?us-ascii?Q?lPFLFjb7avbON7KFR4RCZ1PI+sDC1Fgd5am2URhjtjFXxysnWHCzLwq+Szol?= =?us-ascii?Q?ycDMg6DndXxG9jceM9TkgWcrajNFqX9ErNPUQAxCmfmQceR8aOFVU0JZhup9?= =?us-ascii?Q?yB9sUDSXijwHkQQnWp/nzV/QSoSIjlzUYCcZSzgT2OFFltZV+AokHStvX5le?= =?us-ascii?Q?Gx5JKneIKJmFJUv044OU37K3LGLuHQA4/6YFpjxwMdQ/5tDeWb6AddPF0x7h?= =?us-ascii?Q?Myyff7GxLQydyrXzYGlfPlY9wM+NNnrS9hCxwo68zmtm7Cm4bWBEIcSLVn2v?= =?us-ascii?Q?bE7L+nSzaXmtcdfXNARk3MrXcu+Bu537ybikwGr0nIwsa9cbay9ypEd/SGUE?= =?us-ascii?Q?eEt1scRPRfPoSyWi1L2vIYiosPsBK//gh/tZElp4heHXdAZyCXCBX5tOKCRK?= =?us-ascii?Q?UVFNsPCkbaV9VvYnw6sPXgHYiPlOIiNS7nXhXjuOVQcfIV5LnvPU8icdSlad?= =?us-ascii?Q?Bvqb8UBFduH+yFkpe9ZPIQrUt/4d8pgN0DeOSiZHFv486SXoJLI794AUW9mO?= =?us-ascii?Q?L0aCkNoOVfdye4L/3KbdA8LtZzAYvEan6EBxgSmwPsOf2UIVPlNA6qClSMOH?= =?us-ascii?Q?OK/rJiadmdjcGgjrvyxpsRJ+whgVOGAfD6+QWbIdtvS3qk2CuWs4DW7yzS31?= =?us-ascii?Q?BmW7yoaNDryW6cPmcfcObAfGSJ+VGZ5KQ4Afbl+m2l3rOkiNaADPlf9//oI4?= =?us-ascii?Q?nGExWmsVi4vY04bjDhi6HeK3tQdyvXuUyfilmDYZnQ78cWb2uTxKPFe8O8ds?= =?us-ascii?Q?+loQPAV6H9rVVA51/g1yh/pJ7BUVjv/NT4+waRLHdjdwCQgKbEl2OBWj1n9E?= =?us-ascii?Q?nxFSGfrGbzem/YlMDkv83G17y6hYJDbTwPgs1tOynlV4eTgV/c7vwTRVXr8w?= =?us-ascii?Q?mkBpYtkBVi0E3DkC9Vrf13d27v3PQYrla2lGOIAcMomvgRt/iaLvGPfW03DG?= =?us-ascii?Q?Gknf1jQ6pPUArVl0PwjFyp+8DHfXnkqo+ieII+QP88IOpyHaTf6I48F5l2oX?= =?us-ascii?Q?pLB4DK/7smfCp6AixYMs+ipn6giCZKDg?= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DM4PR10MB8218.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(1800799024)(7416014)(376014)(366016);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?diWlbuHzpwb0FIpclMZUgyyO6hA55vdzBf13AJ/WtKTz0czgBcW/tV52/qIT?= =?us-ascii?Q?4D4VcrdOVRZzkDOkqErySvFbljtadBtugm80DriWBihU74IPeAVl26HkkdwC?= =?us-ascii?Q?fCb0QffeIcqmctQGfD5eJZ74abOpkKpKg6JPZmdtp7k5bqHK3P1GYDcVs5r4?= =?us-ascii?Q?vcTdjlbv+AXo9NAEb1c/63g/c5DdAez6immXbVlkE2QgMnhcb8WdFzs+n4qC?= =?us-ascii?Q?GD6sI/xFzJ6bqEAclIbsY6cybYgjsEBPc36MeoI4yBYSr9TqPW5fNfNjPVEo?= =?us-ascii?Q?1f8Q2EVsaUjPyNyPu0m0IRD9ar7mIQosbnOENBqavVKQCmvfTP7AF5ovc2yz?= =?us-ascii?Q?XMFmQ70CHZOi8olIH3pFz6nT+lYAXS3WxdQpEhG88tXn04FJDJh8/QxuCps4?= =?us-ascii?Q?DnJv+sbegs5IZzsRsm152bUoQFxnuT/TPT/+mcXbki5ymwcV1EY18JeoUnCb?= =?us-ascii?Q?crgRhVb1EHB0KVJuYmQ03XXnHJnjH0bzqgqD0UvS9tS/RocOPW+YFhVYlvSo?= =?us-ascii?Q?t4P3xV935SUa5qNRJm9xcHZeJzDpqQLPnLPxToStuF+N9eoVXex1CXy2369s?= =?us-ascii?Q?D3+janGhI6wqHKCT+WeS4XaaSmRY6c94eMhNqsaia5CKTTW2N9We6k/l6xi4?= =?us-ascii?Q?rPmYUcfw5i4ezhrW7p9lujezj4zfwWGgByb2gsoYpJOVrd75327zpxmat4jQ?= =?us-ascii?Q?sHWzLuc4T+g8uu+nMuayMupMJKVM6wHz4X2mQfpzrFt/+6BMW7kG/UNHeH2H?= =?us-ascii?Q?U+wMU4/ADnrio5VO70Vaap1pBzh90+28VmjV4VFr2Ll8mbsWlo6czC93RTwr?= =?us-ascii?Q?8NAtsYbag9oUfpauoO1CgLDs4d911dv+VBS+QMt7W9fw7uFeIhXuasCdXE2e?= =?us-ascii?Q?S5UmRJ6UwnllzlYNZveWQMdtJBS19prxgEZ29E8GyXMg7WNSBNWnYVbZw8R3?= =?us-ascii?Q?mQhTXXFpxQzgUxMqpbe0fUGwxb+aE6exbi6SY/RTug4lMt4lwomuVYG93O7t?= =?us-ascii?Q?XMpyqNs47ttRhV9/3QRrJz4TmXR0Bb57K6g2ACR+f3qPESLeiGQPDL9uPM49?= =?us-ascii?Q?30CJxPZSFrpz1aqijn82kwKo9GYJpv+b4dp6eBRMEhLlkeM79CJLh3U/Pjr7?= =?us-ascii?Q?DyVoSsb4duq+jIlj+x+BGTRn9R3sReN1nIkzt/UBzbLdWkTeF2iHkrA/rwVN?= =?us-ascii?Q?GqbgGCJtO//v19DHOxQ0ONZj/Jw9R/CMNhU2FN21OQ3flr/JGXkRIXp9tcUv?= =?us-ascii?Q?aMRi3GhXrWiDwqFl4A8ZfLQ1URHIGj9GI5IZIBOb2rjIKiVwf/EjjmIRHJsM?= =?us-ascii?Q?QI1x0dn36ZYgjENWesYMSHlcgOXCN9p1KwDCBbjSQc6eV59lmDgdzIVxzpMf?= =?us-ascii?Q?60IXQIo3YoROdfgiiFjoMYeSZtFZJx8GXxsf6Y3TNBA1QaoIxsa5zU+cBiTz?= =?us-ascii?Q?Sb9YZJ2c/UL/S549ApKJONHia7GO48l8zjElwJkaYrzgFGV45S2/NPcTnP1d?= =?us-ascii?Q?ScpFlVTAQ5iM2y+5ikkx3TDT0vgWo+i8dRiyAe69m26+mVsj/U6tAcXZqzXj?= =?us-ascii?Q?IdboZwIsgMR1QL50zkelyfkD0bP9incE42bmykexdZTB8/axfzHW7OpXNySn?= =?us-ascii?Q?dQ=3D=3D?= X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: +Sc/0ReeyNb6JA8j5lLr5CFZbAyMqFfuYb/9pddIneAOe610vlXh/ngGNaigv8lJYwUDExU+Wrja6fp6c+iTaes9mpipQQZw5HBHkVTNT3c/da6D0YTqP/UmX3c5ECCt33GjMyHIgmWLGNA/86w4HPeiInZLR+FpOydSTgLlVz2J+cF5fsj1VuIHt07pEsqEIqm2tNlOXVfzGvxcVT3WEQOOtiWzbhxl2RChYCEn35qrDFdkiCluDiCeafjCN7IjjUSj047e3W2WmiXBA30yuJV17KGFPuSJFvoAOVjSTLrOZf5azzDiz+Jr99yuAB2Qi7qlOYRCA6clQZd/XLECf/3CEx+OY95V99lSpOBu6zs9NelFvOiz5DealdZ+hrRXYtvFTAxbOUF51qMltQMgQ0hN92phtWwnWEhlv3e/lt7WOoG1F1WOU+QDefdtSE8r7L4sdUKCNVmTNquQaqPzB9jgM8be2hR8mq3W+3v8LVG9OLFxnNQmXjjYoL3TOhZMQEgDkgt8PySfAdsBhOF5ST/ADOnd+0ugC/ZrIi6XzHXRQVQw+Rqdu6rsKRnKXRdIizIHTCwb3TGlQhyQNKNJNbfoyHHpuuZcGuyHSzAKDsM= X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: c630c2aa-a906-4838-a4fa-08de1ee99298 X-MS-Exchange-CrossTenant-AuthSource: DM4PR10MB8218.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 Nov 2025 17:09:26.9605 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: N92FvLOmKRaO5Cg7Dpnb2CTan2CpG5yVLt3Idxg8lW9YRhlJt2tS/jgbVI8U655fSlQsdF+bgGblDcfc+skKm+76hfGuvbIx7DCgQPLM/XU= X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ5PPF04D2D7FA7 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1121,Hydra:6.1.9,FMLib:17.12.100.49 definitions=2025-11-08_04,2025-11-06_01,2025-10-01_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 adultscore=0 malwarescore=0 spamscore=0 mlxscore=0 bulkscore=0 suspectscore=0 mlxlogscore=999 phishscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2510240000 definitions=main-2511080138 X-Proofpoint-Spam-Details-Enc: AW1haW4tMjUxMTA4MDAxNiBTYWx0ZWRfX0S08px7rgSwc tqCoYrU5uKufWz5W3wIEFEstTUSq5YI/DlA+Hx6Pp0lfX+y1KcwBMnpdU++kWKiub5wdSVgFmbN sYKoTkM52/TEcuMDbG/dLF/d/LtWqUWCD6MnWprsJeaMucXKJpHSpk8HAXLdbLYiMZ4kkNjByo4 7sB10EtPTGLF/K2W/QIgvDt4X0liegnZXnmCLjAhBkne59/FiydQ1Hr9xmvjyKQJN0HtHZ6I2al xm25bnViLE6e3OZEsY5hAl0BEiGC1k4xkchibtTgTjpUwgcDZYBIczJFmPboFj0XC+fAU4WHuLi G94koZL9e4S/yX7jr1F790KnxPF0z9cDvldYpjrlz13dOjxrTXEqT4x+GHjZvTV20l1KgFIxBLU jrZPCk2bKgArK70Z5HQ9kdh6shP4kg== X-Proofpoint-ORIG-GUID: SRi1feQYmhEO6v4iVTzHlmJ56Npb_Iy8 X-Proofpoint-GUID: SRi1feQYmhEO6v4iVTzHlmJ56Npb_Iy8 X-Authority-Analysis: v=2.4 cv=eYgwvrEH c=1 sm=1 tr=0 ts=690f794f cx=c_pps a=XiAAW1AwiKB2Y8Wsi+sD2Q==:117 a=XiAAW1AwiKB2Y8Wsi+sD2Q==:17 a=6eWqkTHjU83fiwn7nKZWdM+Sl24=:19 a=z/mQ4Ysz8XfWz/Q5cLBRGdckG28=:19 a=lCpzRmAYbLLaTzLvsPZ7Mbvzbb8=:19 a=xqWC_Br6kY4A:10 a=6UeiqGixMTsA:10 a=GoEa3M9JfhUA:10 a=VkNPw1HP01LnGYTKEx00:22 a=yPCof4ZbAAAA:8 a=jXjnEuVU91_iq225JjgA:9 a=UhEZJTgQB8St2RibIkdl:22 a=Z5ABNNGmrOfJ6cZ5bIyy:22 a=QOGEsqRv6VhmHaoFNykA:22 Content-Type: text/plain; charset="utf-8" There is simply no need for the hugely confusing concept of 'non-swap' swap entries now we have the concept of softleaf entries and relevant softleaf_xxx() helpers. Adjust all callers to use these instead and remove non_swap_entry() altogether. No functional change intended. Signed-off-by: Lorenzo Stoakes --- arch/s390/mm/gmap_helpers.c | 20 ++++++++++---------- arch/s390/mm/pgtable.c | 12 ++++++------ fs/proc/task_mmu.c | 12 ++++++------ include/linux/swapops.h | 5 ----- mm/filemap.c | 2 +- mm/hmm.c | 16 ++++++++-------- mm/madvise.c | 2 +- mm/memory.c | 36 ++++++++++++++++++------------------ mm/mincore.c | 2 +- mm/userfaultfd.c | 24 ++++++++++++------------ 10 files changed, 63 insertions(+), 68 deletions(-) diff --git a/arch/s390/mm/gmap_helpers.c b/arch/s390/mm/gmap_helpers.c index d4c3c36855e2..549f14ad08af 100644 --- a/arch/s390/mm/gmap_helpers.c +++ b/arch/s390/mm/gmap_helpers.c @@ -11,27 +11,27 @@ #include #include #include -#include +#include #include #include #include #include =20 /** - * ptep_zap_swap_entry() - discard a swap entry. + * ptep_zap_softleaf_entry() - discard a software leaf entry. * @mm: the mm - * @entry: the swap entry that needs to be zapped + * @entry: the software leaf entry that needs to be zapped * - * Discards the given swap entry. If the swap entry was an actual swap - * entry (and not a migration entry, for example), the actual swapped + * Discards the given software leaf entry. If the leaf entry was an actual + * swap entry (and not a migration entry, for example), the actual swapped * page is also discarded from swap. */ -static void ptep_zap_swap_entry(struct mm_struct *mm, swp_entry_t entry) +static void ptep_zap_softleaf_entry(struct mm_struct *mm, softleaf_t entry) { - if (!non_swap_entry(entry)) + if (softleaf_is_swap(entry)) dec_mm_counter(mm, MM_SWAPENTS); - else if (is_migration_entry(entry)) - dec_mm_counter(mm, mm_counter(pfn_swap_entry_folio(entry))); + else if (softleaf_is_migration(entry)) + dec_mm_counter(mm, mm_counter(softleaf_to_folio(entry))); free_swap_and_cache(entry); } =20 @@ -66,7 +66,7 @@ void gmap_helper_zap_one_page(struct mm_struct *mm, unsig= ned long vmaddr) preempt_disable(); pgste =3D pgste_get_lock(ptep); =20 - ptep_zap_swap_entry(mm, pte_to_swp_entry(*ptep)); + ptep_zap_softleaf_entry(mm, softleaf_from_pte(*ptep)); pte_clear(mm, vmaddr, ptep); =20 pgste_set_unlock(ptep, pgste); diff --git a/arch/s390/mm/pgtable.c b/arch/s390/mm/pgtable.c index 0fde20bbc50b..d670bfb47d9b 100644 --- a/arch/s390/mm/pgtable.c +++ b/arch/s390/mm/pgtable.c @@ -16,7 +16,7 @@ #include #include #include -#include +#include #include #include #include @@ -683,12 +683,12 @@ void ptep_unshadow_pte(struct mm_struct *mm, unsigned= long saddr, pte_t *ptep) pgste_set_unlock(ptep, pgste); } =20 -static void ptep_zap_swap_entry(struct mm_struct *mm, swp_entry_t entry) +static void ptep_zap_softleaf_entry(struct mm_struct *mm, softleaf_t entry) { - if (!non_swap_entry(entry)) + if (softleaf_is_swap(entry)) dec_mm_counter(mm, MM_SWAPENTS); - else if (is_migration_entry(entry)) { - struct folio *folio =3D pfn_swap_entry_folio(entry); + else if (softleaf_is_migration(entry)) { + struct folio *folio =3D softleaf_to_folio(entry); =20 dec_mm_counter(mm, mm_counter(folio)); } @@ -710,7 +710,7 @@ void ptep_zap_unused(struct mm_struct *mm, unsigned lon= g addr, if (!reset && pte_swap(pte) && ((pgstev & _PGSTE_GPS_USAGE_MASK) =3D=3D _PGSTE_GPS_USAGE_UNUSED || (pgstev & _PGSTE_GPS_ZERO))) { - ptep_zap_swap_entry(mm, pte_to_swp_entry(pte)); + ptep_zap_softleaf_entry(mm, softleaf_from_pte(pte)); pte_clear(mm, addr, ptep); } if (reset) diff --git a/fs/proc/task_mmu.c b/fs/proc/task_mmu.c index d982fdfcf057..6cb9e1691e18 100644 --- a/fs/proc/task_mmu.c +++ b/fs/proc/task_mmu.c @@ -1020,13 +1020,13 @@ static void smaps_pte_entry(pte_t *pte, unsigned lo= ng addr, } else if (pte_none(ptent)) { smaps_pte_hole_lookup(addr, walk); } else { - swp_entry_t swpent =3D pte_to_swp_entry(ptent); + const softleaf_t entry =3D softleaf_from_pte(ptent); =20 - if (!non_swap_entry(swpent)) { + if (softleaf_is_swap(entry)) { int mapcount; =20 mss->swap +=3D PAGE_SIZE; - mapcount =3D swp_swapcount(swpent); + mapcount =3D swp_swapcount(entry); if (mapcount >=3D 2) { u64 pss_delta =3D (u64)PAGE_SIZE << PSS_SHIFT; =20 @@ -1035,10 +1035,10 @@ static void smaps_pte_entry(pte_t *pte, unsigned lo= ng addr, } else { mss->swap_pss +=3D (u64)PAGE_SIZE << PSS_SHIFT; } - } else if (is_pfn_swap_entry(swpent)) { - if (is_device_private_entry(swpent)) + } else if (softleaf_has_pfn(entry)) { + if (softleaf_is_device_private(entry)) present =3D true; - page =3D pfn_swap_entry_to_page(swpent); + page =3D softleaf_to_page(entry); } } =20 diff --git a/include/linux/swapops.h b/include/linux/swapops.h index 41cfc6d59054..c8e6f927da48 100644 --- a/include/linux/swapops.h +++ b/include/linux/swapops.h @@ -492,10 +492,5 @@ static inline pmd_t swp_entry_to_pmd(swp_entry_t entry) =20 #endif /* CONFIG_ARCH_ENABLE_THP_MIGRATION */ =20 -static inline int non_swap_entry(swp_entry_t entry) -{ - return swp_type(entry) >=3D MAX_SWAPFILES; -} - #endif /* CONFIG_MMU */ #endif /* _LINUX_SWAPOPS_H */ diff --git a/mm/filemap.c b/mm/filemap.c index 950d93885e38..ab8ff5b2fc3b 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -4566,7 +4566,7 @@ static void filemap_cachestat(struct address_space *m= apping, swp_entry_t swp =3D radix_to_swp_entry(folio); =20 /* swapin error results in poisoned entry */ - if (non_swap_entry(swp)) + if (!softleaf_is_swap(swp)) goto resched; =20 /* diff --git a/mm/hmm.c b/mm/hmm.c index bc3fa699a4c6..d5c4e60fbfad 100644 --- a/mm/hmm.c +++ b/mm/hmm.c @@ -253,17 +253,17 @@ static int hmm_vma_handle_pte(struct mm_walk *walk, u= nsigned long addr, } =20 if (!pte_present(pte)) { - swp_entry_t entry =3D pte_to_swp_entry(pte); + const softleaf_t entry =3D softleaf_from_pte(pte); =20 /* * Don't fault in device private pages owned by the caller, * just report the PFN. */ - if (is_device_private_entry(entry) && - page_pgmap(pfn_swap_entry_to_page(entry))->owner =3D=3D + if (softleaf_is_device_private(entry) && + page_pgmap(softleaf_to_page(entry))->owner =3D=3D range->dev_private_owner) { cpu_flags =3D HMM_PFN_VALID; - if (is_writable_device_private_entry(entry)) + if (softleaf_is_device_private_write(entry)) cpu_flags |=3D HMM_PFN_WRITE; new_pfn_flags =3D swp_offset_pfn(entry) | cpu_flags; goto out; @@ -274,16 +274,16 @@ static int hmm_vma_handle_pte(struct mm_walk *walk, u= nsigned long addr, if (!required_fault) goto out; =20 - if (!non_swap_entry(entry)) + if (softleaf_is_swap(entry)) goto fault; =20 - if (is_device_private_entry(entry)) + if (softleaf_is_device_private(entry)) goto fault; =20 - if (is_device_exclusive_entry(entry)) + if (softleaf_is_device_exclusive(entry)) goto fault; =20 - if (is_migration_entry(entry)) { + if (softleaf_is_migration(entry)) { pte_unmap(ptep); hmm_vma_walk->last =3D addr; migration_entry_wait(walk->mm, pmdp, addr); diff --git a/mm/madvise.c b/mm/madvise.c index ffae3b566dc1..234178685793 100644 --- a/mm/madvise.c +++ b/mm/madvise.c @@ -249,7 +249,7 @@ static void shmem_swapin_range(struct vm_area_struct *v= ma, continue; entry =3D radix_to_swp_entry(folio); /* There might be swapin error entries in shmem mapping. */ - if (non_swap_entry(entry)) + if (!softleaf_is_swap(entry)) continue; =20 addr =3D vma->vm_start + diff --git a/mm/memory.c b/mm/memory.c index 087f31a291b4..ad336cbf1d88 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -932,7 +932,7 @@ copy_nonpresent_pte(struct mm_struct *dst_mm, struct mm= _struct *src_mm, struct folio *folio; struct page *page; =20 - if (likely(!non_swap_entry(entry))) { + if (likely(softleaf_is_swap(entry))) { if (swap_duplicate(entry) < 0) return -EIO; =20 @@ -950,12 +950,12 @@ copy_nonpresent_pte(struct mm_struct *dst_mm, struct = mm_struct *src_mm, set_pte_at(src_mm, addr, src_pte, pte); } rss[MM_SWAPENTS]++; - } else if (is_migration_entry(entry)) { - folio =3D pfn_swap_entry_folio(entry); + } else if (softleaf_is_migration(entry)) { + folio =3D softleaf_to_folio(entry); =20 rss[mm_counter(folio)]++; =20 - if (!is_readable_migration_entry(entry) && + if (!softleaf_is_migration_read(entry) && is_cow_mapping(vm_flags)) { /* * COW mappings require pages in both parent and child @@ -964,15 +964,15 @@ copy_nonpresent_pte(struct mm_struct *dst_mm, struct = mm_struct *src_mm, */ entry =3D make_readable_migration_entry( swp_offset(entry)); - pte =3D swp_entry_to_pte(entry); + pte =3D softleaf_to_pte(entry); if (pte_swp_soft_dirty(orig_pte)) pte =3D pte_swp_mksoft_dirty(pte); if (pte_swp_uffd_wp(orig_pte)) pte =3D pte_swp_mkuffd_wp(pte); set_pte_at(src_mm, addr, src_pte, pte); } - } else if (is_device_private_entry(entry)) { - page =3D pfn_swap_entry_to_page(entry); + } else if (softleaf_is_device_private(entry)) { + page =3D softleaf_to_page(entry); folio =3D page_folio(page); =20 /* @@ -996,7 +996,7 @@ copy_nonpresent_pte(struct mm_struct *dst_mm, struct mm= _struct *src_mm, * when a device driver is involved (you cannot easily * save and restore device driver state). */ - if (is_writable_device_private_entry(entry) && + if (softleaf_is_device_private_write(entry) && is_cow_mapping(vm_flags)) { entry =3D make_readable_device_private_entry( swp_offset(entry)); @@ -1005,7 +1005,7 @@ copy_nonpresent_pte(struct mm_struct *dst_mm, struct = mm_struct *src_mm, pte =3D pte_swp_mkuffd_wp(pte); set_pte_at(src_mm, addr, src_pte, pte); } - } else if (is_device_exclusive_entry(entry)) { + } else if (softleaf_is_device_exclusive(entry)) { /* * Make device exclusive entries present by restoring the * original entry then copying as for a present pte. Device @@ -4635,7 +4635,7 @@ vm_fault_t do_swap_page(struct vm_fault *vmf) rmap_t rmap_flags =3D RMAP_NONE; bool need_clear_cache =3D false; bool exclusive =3D false; - swp_entry_t entry; + softleaf_t entry; pte_t pte; vm_fault_t ret =3D 0; void *shadow =3D NULL; @@ -4647,15 +4647,15 @@ vm_fault_t do_swap_page(struct vm_fault *vmf) if (!pte_unmap_same(vmf)) goto out; =20 - entry =3D pte_to_swp_entry(vmf->orig_pte); - if (unlikely(non_swap_entry(entry))) { - if (is_migration_entry(entry)) { + entry =3D softleaf_from_pte(vmf->orig_pte); + if (unlikely(!softleaf_is_swap(entry))) { + if (softleaf_is_migration(entry)) { migration_entry_wait(vma->vm_mm, vmf->pmd, vmf->address); - } else if (is_device_exclusive_entry(entry)) { - vmf->page =3D pfn_swap_entry_to_page(entry); + } else if (softleaf_is_device_exclusive(entry)) { + vmf->page =3D softleaf_to_page(entry); ret =3D remove_device_exclusive_entry(vmf); - } else if (is_device_private_entry(entry)) { + } else if (softleaf_is_device_private(entry)) { if (vmf->flags & FAULT_FLAG_VMA_LOCK) { /* * migrate_to_ram is not yet ready to operate @@ -4666,7 +4666,7 @@ vm_fault_t do_swap_page(struct vm_fault *vmf) goto out; } =20 - vmf->page =3D pfn_swap_entry_to_page(entry); + vmf->page =3D softleaf_to_page(entry); vmf->pte =3D pte_offset_map_lock(vma->vm_mm, vmf->pmd, vmf->address, &vmf->ptl); if (unlikely(!vmf->pte || @@ -4690,7 +4690,7 @@ vm_fault_t do_swap_page(struct vm_fault *vmf) } else { pte_unmap_unlock(vmf->pte, vmf->ptl); } - } else if (is_hwpoison_entry(entry)) { + } else if (softleaf_is_hwpoison(entry)) { ret =3D VM_FAULT_HWPOISON; } else if (softleaf_is_marker(entry)) { ret =3D handle_pte_marker(vmf); diff --git a/mm/mincore.c b/mm/mincore.c index e77c5bc88fc7..e1d50f198c42 100644 --- a/mm/mincore.c +++ b/mm/mincore.c @@ -74,7 +74,7 @@ static unsigned char mincore_swap(swp_entry_t entry, bool= shmem) * absent. Page table may contain migration or hwpoison * entries which are always uptodate. */ - if (non_swap_entry(entry)) + if (!softleaf_is_swap(entry)) return !shmem; =20 /* diff --git a/mm/userfaultfd.c b/mm/userfaultfd.c index 055ec1050776..bd1f74a7a5ac 100644 --- a/mm/userfaultfd.c +++ b/mm/userfaultfd.c @@ -1256,7 +1256,6 @@ static long move_pages_ptes(struct mm_struct *mm, pmd= _t *dst_pmd, pmd_t *src_pmd unsigned long dst_addr, unsigned long src_addr, unsigned long len, __u64 mode) { - swp_entry_t entry; struct swap_info_struct *si =3D NULL; pte_t orig_src_pte, orig_dst_pte; pte_t src_folio_pte; @@ -1430,19 +1429,20 @@ static long move_pages_ptes(struct mm_struct *mm, p= md_t *dst_pmd, pmd_t *src_pmd orig_dst_pte, orig_src_pte, dst_pmd, dst_pmdval, dst_ptl, src_ptl, &src_folio, len); - } else { + } else { /* !pte_present() */ struct folio *folio =3D NULL; + const softleaf_t entry =3D softleaf_from_pte(orig_src_pte); =20 - entry =3D pte_to_swp_entry(orig_src_pte); - if (non_swap_entry(entry)) { - if (is_migration_entry(entry)) { - pte_unmap(src_pte); - pte_unmap(dst_pte); - src_pte =3D dst_pte =3D NULL; - migration_entry_wait(mm, src_pmd, src_addr); - ret =3D -EAGAIN; - } else - ret =3D -EFAULT; + if (softleaf_is_migration(entry)) { + pte_unmap(src_pte); + pte_unmap(dst_pte); + src_pte =3D dst_pte =3D NULL; + migration_entry_wait(mm, src_pmd, src_addr); + + ret =3D -EAGAIN; + goto out; + } else if (!softleaf_is_swap(entry)) { + ret =3D -EFAULT; goto out; } =20 --=20 2.51.0 From nobody Fri Dec 19 21:47:08 2025 Received: from mx0b-00069f02.pphosted.com (mx0b-00069f02.pphosted.com [205.220.177.32]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8004F274FF5; Sat, 8 Nov 2025 17:11:23 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=205.220.177.32 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1762621885; cv=fail; b=ZZ5Ueuv2/EYRwfWCTWK9kYX6K6IOKmEp5uNcN55Kg/OnN1eoXLN25zZ3IaIYelZKUG3SrYkCw6Z9ITUl99Z+lpJiJcvAnCII3pHnyL9KAVmQZ4KaQCX+PVohQu+PLYwoG0sGoyHROfASweRNlx43ZI1xr8BLs0+q5BjIVN8Sb1I= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1762621885; c=relaxed/simple; bh=Ywwk8BgU1vvgEYMYYm12WFq+m5B/cdbVU9dI9FbCrwo=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: Content-Type:MIME-Version; b=vGqYkJo47TYfTJDC+t5qK91K4urXie558t9fjwfP1Tv9wQwOXvNp+X95TRV6y8sSyYhBZDKspYA5wTmDmjx5a56QqkSy0SuFfytvCKxi096zcaF7PFLb3TZKZntt/1K/nl17H5tJraxS4hIuys4+mELoUN2ke1s5EwLG3qyN0bA= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=oracle.com; spf=pass smtp.mailfrom=oracle.com; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b=PEkUjeLm; dkim=pass (1024-bit key) header.d=oracle.onmicrosoft.com header.i=@oracle.onmicrosoft.com header.b=UuVMCwVG; arc=fail smtp.client-ip=205.220.177.32 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=oracle.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=oracle.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b="PEkUjeLm"; dkim=pass (1024-bit key) header.d=oracle.onmicrosoft.com header.i=@oracle.onmicrosoft.com header.b="UuVMCwVG" Received: from pps.filterd (m0246630.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 5A8GfxDP009610; Sat, 8 Nov 2025 17:09:38 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=cc :content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s= corp-2025-04-25; bh=8Mj4dZ2xkCZUyYXA/0EAZ/Vac+RE+RRWNsVC2E5wqGE=; b= PEkUjeLmPb9+kY8XGRtNR8PWx0nwQYN5vTnGUrrlSb3JOAiaqgdzM59IswnsNYNZ r+fnY67bawA00rdLFIEeULUp/subDLOm3OEm/wbn/KXAzGrMRtwOl1MpfIv0NGaT 604jNwXvq8ghJoRaZRemxHNBiu+BGqKJTDIAAdH/94jHhqOs2zTFqCY05HYFlnot Zt/dztHuPclA+oHX3fnFbmeCKizktx+qQChT0zvMECFT2D6aDKxqksk01oAWUH/1 5Sjc+X1a7GQyLwhdz1SDJYrsZigAtr2Mgm8t8JiPFvisUrWB6vqXOg7xnm+7F/lr bg+/fWtvRbxAegNN1fdA2g== Received: from phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta01.appoci.oracle.com [138.1.114.2]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 4aa9k8g0jc-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Sat, 08 Nov 2025 17:09:37 +0000 (GMT) Received: from pps.filterd (phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (8.18.1.2/8.18.1.2) with ESMTP id 5A8BijdL007604; Sat, 8 Nov 2025 17:09:36 GMT Received: from bl2pr02cu003.outbound.protection.outlook.com (mail-eastusazon11011002.outbound.protection.outlook.com [52.101.52.2]) by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTPS id 4a9va76n3y-3 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Sat, 08 Nov 2025 17:09:36 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=rgc+RWYMEnRKEVjDlIkfO4/C+KnuUPtAACyUTjocdq5oM6WaF2HBU3waMMpj3lm4yLjL3dDOtueIqDr2tyNZe2vSyWGqaXM7oT2wL2eA93X2jN6o4uGV+I/lOEBw2yhUql1qjQpL1aL0aa8FmIDZZgv+7AHcQBp0bktjBtggsM8ConMtHLj43/G2kt928JIiOXxVthTFgfJ0eH19H8H/eKBAqeXxub7EsCQKckudJFR6ItXmn7DT7rN2j1u9geGvl+oqa/LVyRctv+hufSynk0vGobAdfGAHsKTm+QTxSwiaH9+oda+s7mP7cT+pDlVXRFU2J+fNMmzK9jwc5aOuZA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=8Mj4dZ2xkCZUyYXA/0EAZ/Vac+RE+RRWNsVC2E5wqGE=; b=kX/O980FsRV4DXg80F9OSAF1K34r5X9k4K+4+RPmAtGwa6pMVk2xKjPntOcjurlnTLBKYe2IgvBLzNNJJwq/nQyIlKQWZG0xJO4EI1nQadPGjtUHW7eQ+QjTtn0tdavmJx60NQkb3r5QFcwsCgym2IfUfbZBCUKPw8WtPSa886O1FFfyYnEoyDbg/eT/vgDOVXaX7chP0xse9SGj0L484l2hO/YP5MEtNTS0qZphJjuoC/OAe0MDKzIgn4vYXcWVt6v+M5sNYaPJt05XeADbsM43lp72kdRUXC/CrIde6w59cfPdWkw9xSayNuU0q4cl7OshHwLTKL+iYZsW+kc57w== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=8Mj4dZ2xkCZUyYXA/0EAZ/Vac+RE+RRWNsVC2E5wqGE=; b=UuVMCwVGA4L/p7SG6AIkurGKzO3sAZ9oyenyDoF6xhkYcUN8RMINxYtHxFVMTb52Gto7d7SV1s3cF/85YI4VSRU6FVH6eYfFYCGQUP0inUDJlg67sKgUM/s8rR9CccH1VD1rSb5AlVMzD5xJvJqszT96flvPp+N2dPEQA7avHAI= Received: from DM4PR10MB8218.namprd10.prod.outlook.com (2603:10b6:8:1cc::16) by SJ5PPF04D2D7FA7.namprd10.prod.outlook.com (2603:10b6:a0f:fc02::785) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9298.15; Sat, 8 Nov 2025 17:09:30 +0000 Received: from DM4PR10MB8218.namprd10.prod.outlook.com ([fe80::2650:55cf:2816:5f2]) by DM4PR10MB8218.namprd10.prod.outlook.com ([fe80::2650:55cf:2816:5f2%7]) with mapi id 15.20.9298.010; Sat, 8 Nov 2025 17:09:30 +0000 From: Lorenzo Stoakes To: Andrew Morton Cc: Christian Borntraeger , Janosch Frank , Claudio Imbrenda , David Hildenbrand , Alexander Gordeev , Gerald Schaefer , Heiko Carstens , Vasily Gorbik , Sven Schnelle , Peter Xu , Alexander Viro , Christian Brauner , Jan Kara , Arnd Bergmann , Zi Yan , Baolin Wang , "Liam R . Howlett" , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Lance Yang , Muchun Song , Oscar Salvador , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Matthew Brost , Joshua Hahn , Rakie Kim , Byungchul Park , Gregory Price , Ying Huang , Alistair Popple , Axel Rasmussen , Yuanchu Xie , Wei Xu , Kemeng Shi , Kairui Song , Nhat Pham , Baoquan He , Chris Li , SeongJae Park , Matthew Wilcox , Jason Gunthorpe , Leon Romanovsky , Xu Xin , Chengming Zhou , Jann Horn , Miaohe Lin , Naoya Horiguchi , Pedro Falcato , Pasha Tatashin , Rik van Riel , Harry Yoo , Hugh Dickins , linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-s390@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, linux-arch@vger.kernel.org, damon@lists.linux.dev Subject: [PATCH v2 14/16] mm: remove is_hugetlb_entry_[migration, hwpoisoned]() Date: Sat, 8 Nov 2025 17:08:28 +0000 Message-ID: <3ee4f29bf3b5acccbc7933f0ed8e8cc1b9ca0fa7.1762621568.git.lorenzo.stoakes@oracle.com> X-Mailer: git-send-email 2.51.0 In-Reply-To: References: Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: LO4P123CA0190.GBRP123.PROD.OUTLOOK.COM (2603:10a6:600:1a4::15) To DM4PR10MB8218.namprd10.prod.outlook.com (2603:10b6:8:1cc::16) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DM4PR10MB8218:EE_|SJ5PPF04D2D7FA7:EE_ X-MS-Office365-Filtering-Correlation-Id: a8c0cff1-3120-4ff1-8d15-08de1ee99486 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|1800799024|7416014|376014|366016; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?MjzoLYR5Tu7x61mj5+tnYY6c8s30haLDVFc+l8FsywFAge0LgoWuq8S9UO0V?= =?us-ascii?Q?JRHYiG21iwWCQSA3jEkUh0vOKeuLLYaOIv0VHIgEBbJxh9podzLiTH7OB90d?= =?us-ascii?Q?EVpQp7STEAcyfd3k2aAVOH8jtwuyZwSqW98hBsSUcxjAP2HGRaxRYoPXxLQc?= =?us-ascii?Q?PB3gMEI5Y98iu/B/8eAuPfHA2PfiYsQ09zTknmBhZDorGa/n6byrwNG+43SL?= =?us-ascii?Q?TyD2cRcbBnOHtWOaGWODAbKiJTcC1hl8cuW9KRwwXhfkGUibiHx7byMSdhMU?= =?us-ascii?Q?yLdX+HREXMtcPiUORRfNWY6jHnf6CYasz4B6Ydy1y/BOV+PuLTONcMnzJNK1?= =?us-ascii?Q?AdMLTnuGI9gl7VJ0I+Xor1HcXNVcaOrcPW9lwQW/g3xqggmBgUUF/nx57nJ2?= =?us-ascii?Q?pEx4RJM7IpWEvZkV3h0mZkP2pfNOWj5+syg+e4+3Lxt1QNdnFgbeB7qFWG9N?= =?us-ascii?Q?kkdqEU/A4L1k72npwP0/N8vow8eLzNgTs+99LHuozOedO3t8r6BJinQT2nDL?= =?us-ascii?Q?wCi272BdiZ6hGP+FD1/m7KOrTkk0oLiuTDqdISAku7rkSu+CrH9Exd7TI8PS?= =?us-ascii?Q?yJHj1ioral7d77IamGZnYNZaOGDga6XEFIkbJ1Ea9ylR7KVPR8m2dm+ZbpOo?= =?us-ascii?Q?o4Wy8Xbv4MB0JFwAJuPBsdhPdWfiRrq/+AaK1C1qqHo7QJ6EAH9M2ltZM/Je?= =?us-ascii?Q?uG0vmbPcCMCA/qJjziefp6ZuYkj80kctTdrxf39814aibbczOrbJuUB0OoSg?= =?us-ascii?Q?RfX6yh6Tsabg0ytEiB1+VcJ0xY9v8NZ1Liz5WUNNCo0pym81CVqkvxc3utL2?= =?us-ascii?Q?OyWp7X6SJVH6WqhXHg18YhjbRJp5pt5yWTV2daGzjshKydtc8I+cVnvh5mAW?= =?us-ascii?Q?HWtPfZIFcCnE6cAoH2Nq33zS23qVKgv2dKGZPiOZl7T2UzZlRdOZodsyrlG7?= =?us-ascii?Q?fUW91BgLcQ+pQpwSM8cGwa/kWCVCz0IgTkmSpBfIQaxrSNvRmBEpUsFm33rd?= =?us-ascii?Q?nu5V9UKp//SDkDUrEStiWHlprqnOcShHWlITRFKsn+njQAYi6jLv8YkO07QY?= =?us-ascii?Q?kLYfhb4/UJoIwxx0uJ2Fw6ZoJJ8lJS5VtkryO7qjZEozVXOgEJA920JMsdoG?= =?us-ascii?Q?OAdQ758b4lVSFXQP4HC2N/U/TR1k6YfxOnibgIhgTrTwZ6nlYCWZCdv8JaeG?= =?us-ascii?Q?6zS8bOSy/jirTzbSIsU1POhg9DAiloSzXyaNbAGjBvWV1hXDlubsy8qefxOn?= =?us-ascii?Q?pOLv16ZDWgdKIg7gROC562W5SMCZG7LW/eVHdWZEVb5fKs+Pe9jqWwkyB9Lv?= =?us-ascii?Q?8KYGxfmiWv57b6RrLIrJP+88EK4cZ1e6JIne+nkKL/lspPiEskseDH7iuJGd?= =?us-ascii?Q?5xb6Surm5OuYs9Axuj+Oz1yT09SON6e5YsptrzqPAyi9t6sPKpPlNed+zpgi?= =?us-ascii?Q?53DsZz0QXl0P10N18GBedJi6oAVykw5T?= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DM4PR10MB8218.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(1800799024)(7416014)(376014)(366016);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?L3xUl8PebdlsBhHtVqmWQT09HD8/0hRhf1DIBPDbEL4/OdmKesnIA380xiGH?= =?us-ascii?Q?YNd+htligFWsjabkAdUYWZMHpH9ohOFq0XzOBXnZUk7UjoIzsjTca0baTKhI?= =?us-ascii?Q?VIa6U/DRZ2bYkY0TUuvNEg4RUZVqGnf6XZ7Ae4LmcLRn2ngNEoW0KTljxkaN?= =?us-ascii?Q?xT37jkToW1+5x5gF7eOBxM+GijS7OqJjSmNm8PbIAmWWqYP6K0ggwqCyU6Ws?= =?us-ascii?Q?CUGSMsG5jvkKKUFcpfTE6EOv3oC3EfRRuXtFKRkj4ji5VU3iGJclWviryt3W?= =?us-ascii?Q?Y1XURxBWEwWVB7pJNBNLItm6n4wCy/TPwVHC+ib2ZOPWJatqHJDxJQWuFBqV?= =?us-ascii?Q?NHtFgGHhe40+8Gg1fNfLFi5WxVNaSqpE/cGWGz7pWlXxBMKjwq6VFTrKXra6?= =?us-ascii?Q?4nwTpYilif5VzNumIfCNMqlq9VvyGuBNtaFYP+twc8jwj9tg1ZzUkOMqFxNA?= =?us-ascii?Q?Z4Z5sRLaOO8V3DGb/NBt2pES/KYg4sjuXyQj44yQ9/yPmUopFqpG0/y49Erd?= =?us-ascii?Q?BChyrX0+IWAHGHvo83HaYLD1Hx00PN+gYMiokpkjlDeSg3JJmBNP0Gji0iG/?= =?us-ascii?Q?OiqeBiGQ+nh3lWJE35NGALaNA4qhD6FWI+Tir3qWF4KU273B88cmgaKTM76j?= =?us-ascii?Q?KHGxqidqRxCT2pZMlXQpxs63WWxLIjQ93gvMvfLWgleUSE9MXbov1fLq5UsW?= =?us-ascii?Q?ZEwa7RKyUXt4zF1ZHDTGJOMpdHeEgC7a/Ng6uP79MPMf53lptfbTaxRdiZ7W?= =?us-ascii?Q?3NEQ0m4p322kxTJHtJhCr1HXC3joJe9BtMsQ76SQXhiq6Vce1bn30AcaZDFQ?= =?us-ascii?Q?VgjYqla4WoPKcusaGSUl2yHODbT7EA1t75TkhcIBDOabUQuXCrELjiLcIGoN?= =?us-ascii?Q?0FrdhzudLuuDX993as3VhflnmMlFZBS5zftUlBXMvTTWxEhZvwCSzzndr1oO?= =?us-ascii?Q?mcKyYHP78VyUcQwzLrrE69zmdOUOquo268NKbc8av74czVCfYFeCjEoHunCP?= =?us-ascii?Q?YqhlYT69kcIZEgxOg1D5cbc4rPobEvJpoCI9DyqddTmqV4B+Vwy38I6NSogv?= =?us-ascii?Q?RuAsvDNemBNdCuOvaPHiC3HtfiJGmTAEmX9Do3knliDcC2htRfl4xTTm7nCu?= =?us-ascii?Q?NFzNovkbcJS3Q2NxpOCaU/hQfDCbXjFuUILbVKLYMd0v8AB2GaoR8XR4+2nx?= =?us-ascii?Q?93We1wFSuwHDEduaEQuazdNwoWbC2jkl6m06byXbKABQYO5AIGhVJwFhOWQI?= =?us-ascii?Q?rsYb4j3GWDHAWQJLbzkbzlCBWkjf25vvEIW5fsfRr6pcGOhZRnB2I224UqTs?= =?us-ascii?Q?EHRTyC04+zswefJpgQD06iepT8GCjTTqZx4DLeeqXu0tXtng9W0U2MQApIkK?= =?us-ascii?Q?DqRzNOLPAuwI9Qqy/7KsB69lBXjWbCyBjb7wRspUyNCJhVWstm5FHgGdba8s?= =?us-ascii?Q?/bDxm+NOWmcNW4KWBfveGKVcCesOpBKNa2TA625SOGH8sb+uXXxOS2Rt9uVa?= =?us-ascii?Q?4XimLYzdFaqKAbv3/diJM7/fVGc5phDeoYJEe3OcwGlP6y/h9qZ7fUs9IZ9I?= =?us-ascii?Q?HMDovvuZllemAyuliLsXCEew5Sq3fZaDf2lir33nwHCJe4hupO8LvoX66wf1?= =?us-ascii?Q?gQ=3D=3D?= X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: B+iyiUSP+tIbWyue4OaZTX/8a8K8coQJi9t4r3bbs/cCRX3cw0P1fvVeHLqOs9+nPFIPbfpEiDRmxuf9p/tFYjQsB8nAGc0zlegikUkiLxZlrRlD90JsH79le+chnkF3iEAl88ziWw2vi6qU57xgeqIXDXGzL+EYB5XnqHosqrEEYIH4LvC7m9o56T1cpqzTnec5yQLZrOwVfvT59HtdsZ4NPreE7CevCLnUWv//NykI3z2HoFM4cqkUhmiKiUdQkeduwxeLaC3kDFLBtWIYgWdAYb2As3bVNjBuCgqxIgK2C+xdWlcBJ9retkaC/Vkkdruj930sigjyzgJhlHTbndiYPfcnOAhemhbs/VySKIPNWy6bIEDfIaOvE+ALcXT5xMgOWUXcJljavKRHxZUB0QcEhZJK37gjsIiGpFGeoLrsLCl5ob7w1D9p/vMLQDogNaf8eEihqnER2b2b19RlI69+NwNtI5MOYQ9+P1rWQZdTn+c3sXAijAGIR/Pz+gnLTvD9OzVjqR83UG4DJguh61p64rQt2s6LNTkRj5rCvktPIBT06WvppscxfA6ipzzQHy/VMbcbwComyhVunlbC66OAh8ler9D/FP6RVg8zbv8= X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: a8c0cff1-3120-4ff1-8d15-08de1ee99486 X-MS-Exchange-CrossTenant-AuthSource: DM4PR10MB8218.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 Nov 2025 17:09:30.3061 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: QcP+huCDNpWAVbdPvhXNAjsIpG3CSgFyyywQRkdwYDJPfiK9XM/3na4l9rmuRmwz3vU8N1vsRUTxAETdht2LygvpqYQYiCEv8fRr1HgdQlM= X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ5PPF04D2D7FA7 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1121,Hydra:6.1.9,FMLib:17.12.100.49 definitions=2025-11-08_04,2025-11-06_01,2025-10-01_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 adultscore=0 malwarescore=0 spamscore=0 mlxscore=0 bulkscore=0 suspectscore=0 mlxlogscore=999 phishscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2510240000 definitions=main-2511080138 X-Proofpoint-Spam-Details-Enc: AW1haW4tMjUxMTA4MDEzNCBTYWx0ZWRfX4kQt5gqNkoRs /XxHG3AXFt2w6S+LEYh11WU0eW1zrsa1nr2F1YcZwq44DPkDE7NR/NDh3LkbkADkP43Xg7a/beR uJk5jXD8/mmq1oLAMoAMfM6PMOkZBFzkMV8E6kVj7Sa2zQxrnqFAixK5d5IMIKDEs7j4CFVv+wb y10tmwzD5RLQdA2cvt/fTn3DV2UBZJBA9XQOmyHEs2TgtUUfLss2tb+50+DX/QfiPul3U40VsLr 3JW6x5RzMfK2zCqS4S4E1XA2H/CE+dj7xYfECsPBxTK9a+jGtC2Kf436iTTrKcrCfwF0ETTgFjH n54nWDBr10lRUZVSQoLyC+2336gk4mlUbqCeSGH2D9adREG+0D5Cs58KAH+w3hgbBnE6T++ggpQ zegP4uhjoLjORfL9JfrJRICeSruZpA== X-Proofpoint-ORIG-GUID: 64lNaFGdsw4FcKQtvyJueusz5VZ3ZkB7 X-Authority-Analysis: v=2.4 cv=U4ufzOru c=1 sm=1 tr=0 ts=690f7951 cx=c_pps a=XiAAW1AwiKB2Y8Wsi+sD2Q==:117 a=XiAAW1AwiKB2Y8Wsi+sD2Q==:17 a=6eWqkTHjU83fiwn7nKZWdM+Sl24=:19 a=z/mQ4Ysz8XfWz/Q5cLBRGdckG28=:19 a=lCpzRmAYbLLaTzLvsPZ7Mbvzbb8=:19 a=xqWC_Br6kY4A:10 a=6UeiqGixMTsA:10 a=GoEa3M9JfhUA:10 a=VkNPw1HP01LnGYTKEx00:22 a=yPCof4ZbAAAA:8 a=gaN7zjf0O2erDMybN6AA:9 a=UhEZJTgQB8St2RibIkdl:22 a=Z5ABNNGmrOfJ6cZ5bIyy:22 a=QOGEsqRv6VhmHaoFNykA:22 X-Proofpoint-GUID: 64lNaFGdsw4FcKQtvyJueusz5VZ3ZkB7 Content-Type: text/plain; charset="utf-8" We do not need to have explicit helper functions for these, it adds a level of confusion and indirection when we can simply use software leaf entry logic here instead and spell out the special huge_pte_none() case we must consider. No functional change intended. Signed-off-by: Lorenzo Stoakes --- fs/proc/task_mmu.c | 19 +++++---- include/linux/hugetlb.h | 2 - mm/hugetlb.c | 91 +++++++++++++++++------------------------ mm/mempolicy.c | 17 +++++--- mm/migrate.c | 15 +++++-- 5 files changed, 69 insertions(+), 75 deletions(-) diff --git a/fs/proc/task_mmu.c b/fs/proc/task_mmu.c index 6cb9e1691e18..3cdefa7546db 100644 --- a/fs/proc/task_mmu.c +++ b/fs/proc/task_mmu.c @@ -2499,22 +2499,23 @@ static void make_uffd_wp_huge_pte(struct vm_area_st= ruct *vma, unsigned long addr, pte_t *ptep, pte_t ptent) { - unsigned long psize; + const unsigned long psize =3D huge_page_size(hstate_vma(vma)); + softleaf_t entry; =20 - if (is_hugetlb_entry_hwpoisoned(ptent) || pte_is_marker(ptent)) - return; + if (huge_pte_none(ptent)) + set_huge_pte_at(vma->vm_mm, addr, ptep, + make_pte_marker(PTE_MARKER_UFFD_WP), psize); =20 - psize =3D huge_page_size(hstate_vma(vma)); + entry =3D softleaf_from_pte(ptent); + if (softleaf_is_hwpoison(entry) || softleaf_is_marker(entry)) + return; =20 - if (is_hugetlb_entry_migration(ptent)) + if (softleaf_is_migration(entry)) set_huge_pte_at(vma->vm_mm, addr, ptep, pte_swp_mkuffd_wp(ptent), psize); - else if (!huge_pte_none(ptent)) + else huge_ptep_modify_prot_commit(vma, addr, ptep, ptent, huge_pte_mkuffd_wp(ptent)); - else - set_huge_pte_at(vma->vm_mm, addr, ptep, - make_pte_marker(PTE_MARKER_UFFD_WP), psize); } #endif /* CONFIG_HUGETLB_PAGE */ =20 diff --git a/include/linux/hugetlb.h b/include/linux/hugetlb.h index 2387513d6ae5..457d48ac7bcd 100644 --- a/include/linux/hugetlb.h +++ b/include/linux/hugetlb.h @@ -274,8 +274,6 @@ void hugetlb_vma_lock_release(struct kref *kref); long hugetlb_change_protection(struct vm_area_struct *vma, unsigned long address, unsigned long end, pgprot_t newprot, unsigned long cp_flags); -bool is_hugetlb_entry_migration(pte_t pte); -bool is_hugetlb_entry_hwpoisoned(pte_t pte); void hugetlb_unshare_all_pmds(struct vm_area_struct *vma); void fixup_hugetlb_reservations(struct vm_area_struct *vma); void hugetlb_split(struct vm_area_struct *vma, unsigned long addr); diff --git a/mm/hugetlb.c b/mm/hugetlb.c index a74cde267c2a..b702b161ab35 100644 --- a/mm/hugetlb.c +++ b/mm/hugetlb.c @@ -5552,32 +5552,6 @@ static void set_huge_ptep_maybe_writable(struct vm_a= rea_struct *vma, set_huge_ptep_writable(vma, address, ptep); } =20 -bool is_hugetlb_entry_migration(pte_t pte) -{ - swp_entry_t swp; - - if (huge_pte_none(pte) || pte_present(pte)) - return false; - swp =3D pte_to_swp_entry(pte); - if (is_migration_entry(swp)) - return true; - else - return false; -} - -bool is_hugetlb_entry_hwpoisoned(pte_t pte) -{ - swp_entry_t swp; - - if (huge_pte_none(pte) || pte_present(pte)) - return false; - swp =3D pte_to_swp_entry(pte); - if (is_hwpoison_entry(swp)) - return true; - else - return false; -} - static void hugetlb_install_folio(struct vm_area_struct *vma, pte_t *ptep, unsigned lo= ng addr, struct folio *new_folio, pte_t old, unsigned long sz) @@ -5606,6 +5580,7 @@ int copy_hugetlb_page_range(struct mm_struct *dst, st= ruct mm_struct *src, unsigned long npages =3D pages_per_huge_page(h); struct mmu_notifier_range range; unsigned long last_addr_mask; + softleaf_t softleaf; int ret =3D 0; =20 if (cow) { @@ -5653,16 +5628,16 @@ int copy_hugetlb_page_range(struct mm_struct *dst, = struct mm_struct *src, entry =3D huge_ptep_get(src_vma->vm_mm, addr, src_pte); again: if (huge_pte_none(entry)) { - /* - * Skip if src entry none. - */ - ; - } else if (unlikely(is_hugetlb_entry_hwpoisoned(entry))) { + /* Skip if src entry none. */ + goto next; + } + + softleaf =3D softleaf_from_pte(entry); + if (unlikely(softleaf_is_hwpoison(softleaf))) { if (!userfaultfd_wp(dst_vma)) entry =3D huge_pte_clear_uffd_wp(entry); set_huge_pte_at(dst, addr, dst_pte, entry, sz); - } else if (unlikely(is_hugetlb_entry_migration(entry))) { - softleaf_t softleaf =3D softleaf_from_pte(entry); + } else if (unlikely(softleaf_is_migration(softleaf))) { bool uffd_wp =3D pte_swp_uffd_wp(entry); =20 if (!is_readable_migration_entry(softleaf) && cow) { @@ -5681,7 +5656,6 @@ int copy_hugetlb_page_range(struct mm_struct *dst, st= ruct mm_struct *src, entry =3D huge_pte_clear_uffd_wp(entry); set_huge_pte_at(dst, addr, dst_pte, entry, sz); } else if (unlikely(pte_is_marker(entry))) { - const softleaf_t softleaf =3D softleaf_from_pte(entry); const pte_marker marker =3D copy_pte_marker(softleaf, dst_vma); =20 if (marker) @@ -5739,9 +5713,7 @@ int copy_hugetlb_page_range(struct mm_struct *dst, st= ruct mm_struct *src, } hugetlb_install_folio(dst_vma, dst_pte, addr, new_folio, src_pte_old, sz); - spin_unlock(src_ptl); - spin_unlock(dst_ptl); - continue; + goto next; } =20 if (cow) { @@ -5762,6 +5734,8 @@ int copy_hugetlb_page_range(struct mm_struct *dst, st= ruct mm_struct *src, set_huge_pte_at(dst, addr, dst_pte, entry, sz); hugetlb_count_add(npages, dst); } + +next: spin_unlock(src_ptl); spin_unlock(dst_ptl); } @@ -6770,8 +6744,10 @@ vm_fault_t hugetlb_fault(struct mm_struct *mm, struc= t vm_area_struct *vma, ret =3D 0; =20 /* Not present, either a migration or a hwpoisoned entry */ - if (!pte_present(vmf.orig_pte)) { - if (is_hugetlb_entry_migration(vmf.orig_pte)) { + if (!pte_present(vmf.orig_pte) && !huge_pte_none(vmf.orig_pte)) { + const softleaf_t softleaf =3D softleaf_from_pte(vmf.orig_pte); + + if (softleaf_is_migration(softleaf)) { /* * Release the hugetlb fault lock now, but retain * the vma lock, because it is needed to guard the @@ -6782,9 +6758,12 @@ vm_fault_t hugetlb_fault(struct mm_struct *mm, struc= t vm_area_struct *vma, mutex_unlock(&hugetlb_fault_mutex_table[hash]); migration_entry_wait_huge(vma, vmf.address, vmf.pte); return 0; - } else if (is_hugetlb_entry_hwpoisoned(vmf.orig_pte)) + } + if (softleaf_is_hwpoison(softleaf)) { ret =3D VM_FAULT_HWPOISON_LARGE | VM_FAULT_SET_HINDEX(hstate_index(h)); + } + goto out_mutex; } =20 @@ -7166,7 +7145,9 @@ long hugetlb_change_protection(struct vm_area_struct = *vma, i_mmap_lock_write(vma->vm_file->f_mapping); last_addr_mask =3D hugetlb_mask_last_page(h); for (; address < end; address +=3D psize) { + softleaf_t entry; spinlock_t *ptl; + ptep =3D hugetlb_walk(vma, address, psize); if (!ptep) { if (!uffd_wp) { @@ -7198,15 +7179,23 @@ long hugetlb_change_protection(struct vm_area_struc= t *vma, continue; } pte =3D huge_ptep_get(mm, address, ptep); - if (unlikely(is_hugetlb_entry_hwpoisoned(pte))) { - /* Nothing to do. */ - } else if (unlikely(is_hugetlb_entry_migration(pte))) { - softleaf_t entry =3D softleaf_from_pte(pte); + if (huge_pte_none(pte)) { + if (unlikely(uffd_wp)) + /* Safe to modify directly (none->non-present). */ + set_huge_pte_at(mm, address, ptep, + make_pte_marker(PTE_MARKER_UFFD_WP), + psize); + goto next; + } =20 + entry =3D softleaf_from_pte(pte); + if (unlikely(softleaf_is_hwpoison(entry))) { + /* Nothing to do. */ + } else if (unlikely(softleaf_is_migration(entry))) { struct folio *folio =3D softleaf_to_folio(entry); pte_t newpte =3D pte; =20 - if (is_writable_migration_entry(entry)) { + if (softleaf_is_migration_write(entry)) { if (folio_test_anon(folio)) entry =3D make_readable_exclusive_migration_entry( swp_offset(entry)); @@ -7233,7 +7222,7 @@ long hugetlb_change_protection(struct vm_area_struct = *vma, if (pte_is_uffd_wp_marker(pte) && uffd_wp_resolve) /* Safe to modify directly (non-present->none). */ huge_pte_clear(mm, address, ptep, psize); - } else if (!huge_pte_none(pte)) { + } else { pte_t old_pte; unsigned int shift =3D huge_page_shift(hstate_vma(vma)); =20 @@ -7246,16 +7235,10 @@ long hugetlb_change_protection(struct vm_area_struc= t *vma, pte =3D huge_pte_clear_uffd_wp(pte); huge_ptep_modify_prot_commit(vma, address, ptep, old_pte, pte); pages++; - } else { - /* None pte */ - if (unlikely(uffd_wp)) - /* Safe to modify directly (none->non-present). */ - set_huge_pte_at(mm, address, ptep, - make_pte_marker(PTE_MARKER_UFFD_WP), - psize); } - spin_unlock(ptl); =20 +next: + spin_unlock(ptl); cond_resched(); } /* diff --git a/mm/mempolicy.c b/mm/mempolicy.c index 01c3b98f87a6..dee95d5ecfd4 100644 --- a/mm/mempolicy.c +++ b/mm/mempolicy.c @@ -768,16 +768,21 @@ static int queue_folios_hugetlb(pte_t *pte, unsigned = long hmask, unsigned long flags =3D qp->flags; struct folio *folio; spinlock_t *ptl; - pte_t entry; + pte_t ptep; =20 ptl =3D huge_pte_lock(hstate_vma(walk->vma), walk->mm, pte); - entry =3D huge_ptep_get(walk->mm, addr, pte); - if (!pte_present(entry)) { - if (unlikely(is_hugetlb_entry_migration(entry))) - qp->nr_failed++; + ptep =3D huge_ptep_get(walk->mm, addr, pte); + if (!pte_present(ptep)) { + if (!huge_pte_none(ptep)) { + const softleaf_t entry =3D softleaf_from_pte(ptep); + + if (unlikely(softleaf_is_migration(entry))) + qp->nr_failed++; + } + goto unlock; } - folio =3D pfn_folio(pte_pfn(entry)); + folio =3D pfn_folio(pte_pfn(ptep)); if (!queue_folio_required(folio, qp)) goto unlock; if (!(flags & (MPOL_MF_MOVE | MPOL_MF_MOVE_ALL)) || diff --git a/mm/migrate.c b/mm/migrate.c index 3b6bd374157d..48f98a6c1ad2 100644 --- a/mm/migrate.c +++ b/mm/migrate.c @@ -515,16 +515,18 @@ void migration_entry_wait(struct mm_struct *mm, pmd_t= *pmd, void migration_entry_wait_huge(struct vm_area_struct *vma, unsigned long a= ddr, pte_t *ptep) { spinlock_t *ptl =3D huge_pte_lockptr(hstate_vma(vma), vma->vm_mm, ptep); + softleaf_t entry; pte_t pte; =20 hugetlb_vma_assert_locked(vma); spin_lock(ptl); pte =3D huge_ptep_get(vma->vm_mm, addr, ptep); =20 - if (unlikely(!is_hugetlb_entry_migration(pte))) { - spin_unlock(ptl); - hugetlb_vma_unlock_read(vma); - } else { + if (huge_pte_none(pte)) + goto fail; + + entry =3D softleaf_from_pte(pte); + if (softleaf_is_migration(entry)) { /* * If migration entry existed, safe to release vma lock * here because the pgtable page won't be freed without the @@ -533,7 +535,12 @@ void migration_entry_wait_huge(struct vm_area_struct *= vma, unsigned long addr, p */ hugetlb_vma_unlock_read(vma); migration_entry_wait_on_locked(pte_to_swp_entry(pte), ptl); + return; } + +fail: + spin_unlock(ptl); + hugetlb_vma_unlock_read(vma); } #endif =20 --=20 2.51.0 From nobody Fri Dec 19 21:47:08 2025 Received: from mx0a-00069f02.pphosted.com (mx0a-00069f02.pphosted.com [205.220.165.32]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 91B80275B1A; Sat, 8 Nov 2025 17:11:24 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=205.220.165.32 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1762621887; cv=fail; b=Hq/45iyvZt8QmSR8uEj8wB73LC82Q3UygvwK1LDQH13EcL76ytCp8v0LQyOw9tYTZQdgDFw4Ji+N03OKkBC7fkcUmkWm44T62huAsQK7+Y6tvqmJEb5r2O2wIbsx/DGseQK8pqX5Pm9lRPZOaVS97XP1S65UKuXpxSZJvMubOPU= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1762621887; c=relaxed/simple; bh=udHzDGOHZEOC4G9rQvtfjp+gPD4docKcXGTCD83HiVg=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: Content-Type:MIME-Version; b=Fk8JU3AQsW4pbkjX6DG9adHcLt8a90Qbf3hPCavgJXw2Hof2lK92Dr1uazYKedOWUxwRyDXHw3+tNOAppzXcIykMjSHuB4tWnEkZwPhXRtBddpu+IhO7x/slTuVB9jB6N5qLZEKj4MaL9H0RsXrz0QUWAR91HBhUorBEv+/n5ZY= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=oracle.com; spf=pass smtp.mailfrom=oracle.com; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b=GzjJd8PZ; dkim=pass (1024-bit key) header.d=oracle.onmicrosoft.com header.i=@oracle.onmicrosoft.com header.b=Bp5nz1dk; arc=fail smtp.client-ip=205.220.165.32 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=oracle.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=oracle.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b="GzjJd8PZ"; dkim=pass (1024-bit key) header.d=oracle.onmicrosoft.com header.i=@oracle.onmicrosoft.com header.b="Bp5nz1dk" Received: from pps.filterd (m0246629.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 5A8GYAUZ020534; Sat, 8 Nov 2025 17:09:39 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=cc :content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s= corp-2025-04-25; bh=QatDEgNLAi109GS5YaZpw6b//HS/5gq8cfLzJsx21Gk=; b= GzjJd8PZRlvubnwARtxQerq7cLZSyKk21VuMpbnMtjSAC6uD6RfCjgfl/TqQrMAk Q16UhWe2Me5kprr/qK6FGQm9P9LSQgTiax9C2JHsq/BWwyRbYA6lyfbPcTjlp15P fmRC2TWCGSz+CpBRZN/pD8Z2m7CZIUVMNNWhcMl/GTLTHxZFpaJX2hkdUOO8Ib3P HgGgNwYcjfeDNdn/sUskI95mqHPBd1vC99UBjTcRmDJfo7IZJhBQokIiF4bai473 paRB+ffQJVgw6ZeA3x6c7oXE9MmvjQA//MTy8um+JgiSyi/MTuOGY+8wLBDvg24i /IjNeGlvFQPXcmMeS55ugw== Received: from phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta01.appoci.oracle.com [138.1.114.2]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 4aa0jy8etj-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Sat, 08 Nov 2025 17:09:39 +0000 (GMT) Received: from pps.filterd (phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (8.18.1.2/8.18.1.2) with ESMTP id 5A8BijdN007604; Sat, 8 Nov 2025 17:09:38 GMT Received: from bl2pr02cu003.outbound.protection.outlook.com (mail-eastusazon11011002.outbound.protection.outlook.com [52.101.52.2]) by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTPS id 4a9va76n3y-4 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Sat, 08 Nov 2025 17:09:38 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=M1KFk3dJkBpRXBahWzlNAi/LhgPcCa07u0UoJ9RANPm4UcBSEQ7ZZwwdWTx9qavGXtHoDBPjb3c2vKj1ApVoiNM/nIFwEB88wAwL6cYaKMCTJhUbRDiVqheO0yrUjpxmoiQkopMLK8QAyK44vQbf0ByFMoL3INjU8urLXFuhir8Yjs/mkXMw4Bs9hDieZvHsaeMtcwIK5hqh/0FiCUFoxZX/OkY65JIeuH3K0EqwwSJ/+FS8C3zYI9z40unK0MUSyjsuc/oFYFA1E34sLFE55v1/Y4FoDrlqN83Hqn5urDgtwM6tB5A2ZrcTq7yoFKbDaPkRM/T9QD1Mo++UpG+0gw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=QatDEgNLAi109GS5YaZpw6b//HS/5gq8cfLzJsx21Gk=; b=eZe7NiZxHSBbOXIcDkPzRd/fzvEyF8jFuVKj9mflngSbWwk11IzpHyYj6qlxCYpQemulNkUom6GL7LKkzTzTmJGNPezth1by5WZKx2MaV9XS4Uka60nT7tXXPj0EZX0NE0++ffWCXi6yvBd4G0qg0SIH8ZQrF4YJHNCf722qyWe7z+b/lKi5w1NPmKwdVSO1enn9DazV7vfR6MAkqJSePtTfqGnQr2dltsqh4XpVcxxL6YHRBiGlnjyXPKWXGrLa4ZbufiTq9M72ExD0u65C/+SYBJLMyFsDEudyB69PfM7kuNAVo4N9f4DTItdBqaCx9ssJ3BoxxS0//pscawBhAg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=QatDEgNLAi109GS5YaZpw6b//HS/5gq8cfLzJsx21Gk=; b=Bp5nz1dkdqk+/yk8ZWEMug1B9N0AJubImh/wyHmYyKZU8zD0TrYkx4070mYziOwT/UBGiJ3j+WXPyGRCAaRkBIgHNRdIC4bDwHRdBEph8LW7Ps9cRiUb3g9Opd0brjwfXKqhzwIN70H15/G81sXPQy3hklu2zRr87cYWP0zRn3A= Received: from DM4PR10MB8218.namprd10.prod.outlook.com (2603:10b6:8:1cc::16) by SJ5PPF04D2D7FA7.namprd10.prod.outlook.com (2603:10b6:a0f:fc02::785) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9298.15; Sat, 8 Nov 2025 17:09:33 +0000 Received: from DM4PR10MB8218.namprd10.prod.outlook.com ([fe80::2650:55cf:2816:5f2]) by DM4PR10MB8218.namprd10.prod.outlook.com ([fe80::2650:55cf:2816:5f2%7]) with mapi id 15.20.9298.010; Sat, 8 Nov 2025 17:09:33 +0000 From: Lorenzo Stoakes To: Andrew Morton Cc: Christian Borntraeger , Janosch Frank , Claudio Imbrenda , David Hildenbrand , Alexander Gordeev , Gerald Schaefer , Heiko Carstens , Vasily Gorbik , Sven Schnelle , Peter Xu , Alexander Viro , Christian Brauner , Jan Kara , Arnd Bergmann , Zi Yan , Baolin Wang , "Liam R . Howlett" , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Lance Yang , Muchun Song , Oscar Salvador , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Matthew Brost , Joshua Hahn , Rakie Kim , Byungchul Park , Gregory Price , Ying Huang , Alistair Popple , Axel Rasmussen , Yuanchu Xie , Wei Xu , Kemeng Shi , Kairui Song , Nhat Pham , Baoquan He , Chris Li , SeongJae Park , Matthew Wilcox , Jason Gunthorpe , Leon Romanovsky , Xu Xin , Chengming Zhou , Jann Horn , Miaohe Lin , Naoya Horiguchi , Pedro Falcato , Pasha Tatashin , Rik van Riel , Harry Yoo , Hugh Dickins , linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-s390@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, linux-arch@vger.kernel.org, damon@lists.linux.dev Subject: [PATCH v2 15/16] mm: eliminate further swapops predicates Date: Sat, 8 Nov 2025 17:08:29 +0000 Message-ID: <6bcfda48cf63eefee73671798a43e4b9aed23cc0.1762621568.git.lorenzo.stoakes@oracle.com> X-Mailer: git-send-email 2.51.0 In-Reply-To: References: Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: LO4P265CA0085.GBRP265.PROD.OUTLOOK.COM (2603:10a6:600:2bd::14) To DM4PR10MB8218.namprd10.prod.outlook.com (2603:10b6:8:1cc::16) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DM4PR10MB8218:EE_|SJ5PPF04D2D7FA7:EE_ X-MS-Office365-Filtering-Correlation-Id: ecd7fc15-db32-43b8-b60d-08de1ee996a4 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|1800799024|7416014|376014|366016; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?XOM1YFhi/I9unxaBV8C3TeOX8gq/9FM+pbiXR18Zfj/qK/Ao8eR/B6vRCbcu?= =?us-ascii?Q?psp3Hw2f6v8ZxGcrL4NCzjhNJXvGgDZ+BwPSXGnbVpZkv9dNWhuF5UJptL2S?= =?us-ascii?Q?cKLJKaTFdxUrYOHPj4LFMe8p1qFTSgQSsQhJ4hRwy/ayXGsAqK1+hUV9lFOk?= =?us-ascii?Q?N6ToNMqk6sHPZeY5yVxr8WYFM7ABjppl28J+2CC4AwysoWZybPW8TfKbnKg+?= =?us-ascii?Q?kKqkCVTmOtAJPZf2yVO/qEYqfDx7MLtTZQG/bBt/2CnImfBoPQlOBdXJZ8ZX?= =?us-ascii?Q?r3vOaWwy2CW3m6V/DGzg4an09COKmVI6uChSUYJ0AZb0Np5tld397/htHaQa?= =?us-ascii?Q?XNhHFaURgJbBG6pqnm/ft70/Tj25Uw3f7XJPCBY0hGFsvvlUqWS2qrUilqi8?= =?us-ascii?Q?8iJSAgEXg6KmRU5ADp4rPpT/ou4CX3XKx1MWocNt8nWT+89XnoW5DnYvhmZS?= =?us-ascii?Q?p1iOQEcWhqR+sh3USoYKeDKxzxSv3hM8SAMpBtTJND2WAeArJGORGaSrGdqG?= =?us-ascii?Q?URNzX2+8XRs6pHDVQky1d459c6TMPN6NE/pWo2HkEaqqGPTEo/zN2oIfe80k?= =?us-ascii?Q?ZWTp1QAS5kmO4lnc6iV7hfJ52gAtz1+bmTiWCA5w/bqMjOGnAeEt2OUWyN4B?= =?us-ascii?Q?vP/tVTcPbZgf6uyTJ3lFRKPBYTVv9hZNbHcYRg+R+Cy2h6g4NX9A06PZCxly?= =?us-ascii?Q?ExMttLu8ov7Kp7Gr6RydjQ3BDnJP6gnsZd+rLHURvmDccIMLbPpQAWbp9Dtp?= =?us-ascii?Q?8h6imzdJmxfoa2NI71w/2979xq63nckeDuzaj19QfNFVWEgxn0dCxv2XHHBa?= =?us-ascii?Q?3Fv3UGzrUwYYnYcw0BnutQ86Y9WfRz23nWS81h9LghZ7yADiQtabsOL5Got+?= =?us-ascii?Q?/kt8k44JfOOPLr6t9ZLL5HpLlFKZERVhJ1LjOCha2F2I76t0rfTux9lTH39h?= =?us-ascii?Q?kV/tPc+6abGnKDzereFfg7do2BP9lXyjH//7rTXyoEzuIFTlBMKndmCWj4D6?= =?us-ascii?Q?J6hjsOyWwx7NfsKu05NrV3/r2Y52npwdgArbmCVcBn13zLt2le8e5PcGZkSO?= =?us-ascii?Q?70Okv6yCy/DaOCtuxOZXn2E+6ZiCTLbuCgACTEfPGH28ZeZSKuH5mMRDIg1s?= =?us-ascii?Q?ovCNl2R9QsupKURtxJImc4r4RnrGDvkOs+38LaKJhwJQScaprtRmCWkLeqLX?= =?us-ascii?Q?Z2tX2QA56YPr84nNITMFyUANSgDPOi9rsP9ifOpIerp6li70QI7Zcf7C7oJF?= =?us-ascii?Q?GtdpqWnuArENpdFGnc+XxWC1J0GDSfwdA+NGNS+aTLy2pdV0ShsXYgk5yBb+?= =?us-ascii?Q?czkjDl7uam9p1Zyq73Vt/TMm8EjD5DdKauaCjzbzBdtexSupl9KSBjsvAg9X?= =?us-ascii?Q?moU7ZOdYABKgwcVLIv0S11MhFS2ELA2OEV7vCRbNQcU7d4GNCgSFNbXn7g99?= =?us-ascii?Q?/BLMANCRV6fBJG2CVHg/SegmbFsutguH?= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DM4PR10MB8218.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(1800799024)(7416014)(376014)(366016);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?ZODy6E8eICFymg3wGs4PaoktVs/ChTjYJR01mppYXCN4txEAPbgClBlymc9T?= =?us-ascii?Q?DLdez1aY+UGDS2ruBiltwl6YxOXfcB95KHadTtjv16HLRbTErXunx6sYp0Eb?= =?us-ascii?Q?J0CwqnEAfQFKras7lHeR7DbQz7y4RZg5KJlWTCrUwAvqQnNgCNQ2GxqOpK3H?= =?us-ascii?Q?rU1dc0oqZA66B9JB+Zo5ArlPmMEDSaayLEQVXdf0c/Vx9sLwoIzLR2dr9wYs?= =?us-ascii?Q?K+sNC6hripny9zAvk5ZcBPwmN/FGHKpKX3NhSvfYNEH4fr5n1hQozKz3GMtL?= =?us-ascii?Q?N3TH5e5/0uLimPymVK8ySOT/ncSzKN3ZKUzjnmis7RBDQTs/yHOFtplcDfB1?= =?us-ascii?Q?cwiXrr8j8t5MycQYYrsrFNe50uQz5/9IVbPBCB/X+5gzU8mJrC8e529KODaB?= =?us-ascii?Q?hmWQSsp69o+/7TGpresjjJEZ61JpovnnWmQu7vFC9WVPCdFADZ1RB6TEs09a?= =?us-ascii?Q?UqSU0dq559FH7hg7Bgq52zm2y/SSm/u5xh8Dsu5Ixdi0PEYSRcRz5X8iv+hR?= =?us-ascii?Q?Hdvk41MLlENzdhmTAIqH1LtgWpP8SCo1spawbZOezGxb/Y9BawICy6+NzBKi?= =?us-ascii?Q?zyNU8SKhoXt6tAJE1755+k3C2/V4H9iI94sjwDFC+ayQXgy540h4bM/X2VYk?= =?us-ascii?Q?ly0CGXi/3KIus6cJ/RmF1jjPnqebYw2EysJaunF7pbsU2lwR44GWLy40w3qP?= =?us-ascii?Q?uq/pNwY2Bbf2hDjnvK0qC/1+AZdt+ZafkgmTuMmJngbKqj1DBQTvwW4P/BIj?= =?us-ascii?Q?NkTlVl6WubfHCYeB9NfimyH+guDoe/EmNSCdmf6qebt//RFTQ/z3H9Va/wXg?= =?us-ascii?Q?zsC5khG/DiSTrEMDUzcSs41uXaEuyahD5u7QcU5VeTPUD7+ZGwuTnDwhzV+v?= =?us-ascii?Q?BFKxpsQVLb0xcBdfBkY/rUi2+Byo6Hfw0XBv15dbumEjL1zj/mN8alBBHkH2?= =?us-ascii?Q?u9DmClAFQ2OtUR2GVtUutYOMvrZsBTx6EDJqQ19/QzJQdJCEaZSQBqNhTvET?= =?us-ascii?Q?BZBlPwve9oiK3NTTRfT95D1b9cT28QxkiErBEQrLrwOr8M3tT5fdjkIeMnve?= =?us-ascii?Q?bFg1fniHAKTMywN+T1xxJCVtvZJnq/eYkTQHvk+azh7ZXgy7F3dXXWIQGEhF?= =?us-ascii?Q?ZUIaISE4p3+MrgSy+XzpSJDjKfaw6fmYnsf8H0lMJeaAagY0e6yPDy/b6u6y?= =?us-ascii?Q?YRv87Qkk+13A6BGM4N+MWGtukd81WHAEF8K8XfXo5mzdCmQbnIDfwoO3ha9P?= =?us-ascii?Q?z6YxrzpXYxVUZRPWEw1AwwjvJVCneafL1xnjSDcxoNnHHr/VVzgyp8+8HVuM?= =?us-ascii?Q?F/B41uuzVkdGgSdaFo8Gna251Tz2iuGhYLolJiPVw8/L8UKFi3Ha7bNkoqBE?= =?us-ascii?Q?iWvXRdcjbWHdICsC0H5OgNcejhJXKaTbvSSzHwC8eerYOgkwnJk4/6PQ3OvS?= =?us-ascii?Q?xejiIYNnqWshokp2LfZvEyIa7RwzbFgj8ghl6Vp7NTlLKzIG35c+Jnl9XUQe?= =?us-ascii?Q?Ak7qCif2x/+qeIPIdSUAU6Qg6wkTqwbllBFaaV8eKsSV0tJ6W3bFK5wOU1PR?= =?us-ascii?Q?uL12ulnT6fBZWR7s3+qQivsF/xh17HKv/2JiLIjHKU41RiYk/Eeawjc2pMBS?= =?us-ascii?Q?LA=3D=3D?= X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: 0Wv+tQGg+r5kYevN7NhhgrXHIvN+oCcPB2ov9WNf5L7ZRx8x4tX+ygYbUw8Pej0FfAgDmoYAnzWuHK/d6NVNx3rSGo6HG7I2fmciktku2/hBh1DLM8gsJwf5zQSvhku5fF1q3wTRNnB6R6PsSh+6QtoZmYKyR+lvAYfSNiJ63LOrYQhgNfu9e0M78EFDzknfKA7sWLTNFWkrZE/WzmmKug0rfo96dx7bpC9VnkLXLGT/HPojCheVRY6mOeWroeU9CAIZkvdQE62GyKm6dtiEgJaIJFG3yoQGEE6t156BXurKhm3S23ubSA/G9IukKfQcBORs8/3JlNysvrR7PHkUtMkJZmbHUmQ8oceCgVbApCVM6g7ISVBW6gx3dwQWTB/zrq78tC9iBssDjDgvLMPL4YyrJ3xkC8THeFgdNwQ2yRMZgI00d1L3oLdvaCwuwjBQZxJ6dkqOAwOACm/8x426DzRP92flR+CeBWLCMIb4QVyDbkAhjSScNahp/a9MBU+WJA6gkn33la5RmbAxseP593uS0bs/dChLRaIIKe8eHJMDMs8beB1qSHaPqHmnWzCCqZiZl+WIGCSl/UM7jSfJynXiNaUU9QEAfK1brUdr2BM= X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: ecd7fc15-db32-43b8-b60d-08de1ee996a4 X-MS-Exchange-CrossTenant-AuthSource: DM4PR10MB8218.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 Nov 2025 17:09:33.8603 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: zmyf3kj+NDHUGRv1TtSOoo6SdkYvX/oWm/C4ayp/614s2h9XQN1ru3WSPOXXkNcYiFaz0Yx11QuXLHR4tZeEnqKvEqar54cgDtHsVbCcilw= X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ5PPF04D2D7FA7 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1121,Hydra:6.1.9,FMLib:17.12.100.49 definitions=2025-11-08_04,2025-11-06_01,2025-10-01_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 adultscore=0 malwarescore=0 spamscore=0 mlxscore=0 bulkscore=0 suspectscore=0 mlxlogscore=999 phishscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2510240000 definitions=main-2511080138 X-Proofpoint-Spam-Details-Enc: AW1haW4tMjUxMTA4MDA1MCBTYWx0ZWRfX6ougYj9ECSWl lBros9yDBF2waFmCaUKGpHD4xepoLncgOzfvXeP50rpfntzaKbrv3T+he6p+an3psrESswws0tj ZB2IcyHhUL++1mjeyi+0ulmqRrQZ8II508J8z53dLxTJ8q9Be0F4wjQuWRsKJsPIChCwDXH7mGM EsCx4LS/vlfWeropV5yrE+BV+avadY341v7AbBh8iCaVGWq/5Wx0r+iu8QGebuzdoPneb0kZoAr lP4A3ucGM6llvjN4jTI2wqosy6OpeuTWt4rwhFu4zi4Ckv5qjI/M9LtRkI72p7jaUlRqrJiZ19B U5ipfnfEBXMC5hoXJCEw3omicd4zBlhLzjGnAFzp1ude3owsjYtKrCIl9KETUhDMuQP2gB25qlK 6zTIsLr7bvnR6VoJh3AaW3C4C2ftDw== X-Proofpoint-ORIG-GUID: 4IelBXJEfTWxYuPPu7HJD_H_pvGBWRAK X-Authority-Analysis: v=2.4 cv=IpkTsb/g c=1 sm=1 tr=0 ts=690f7953 cx=c_pps a=XiAAW1AwiKB2Y8Wsi+sD2Q==:117 a=XiAAW1AwiKB2Y8Wsi+sD2Q==:17 a=6eWqkTHjU83fiwn7nKZWdM+Sl24=:19 a=z/mQ4Ysz8XfWz/Q5cLBRGdckG28=:19 a=lCpzRmAYbLLaTzLvsPZ7Mbvzbb8=:19 a=xqWC_Br6kY4A:10 a=6UeiqGixMTsA:10 a=GoEa3M9JfhUA:10 a=VkNPw1HP01LnGYTKEx00:22 a=yPCof4ZbAAAA:8 a=12TsxHtCQ5kFwp23Kd0A:9 a=UhEZJTgQB8St2RibIkdl:22 a=Z5ABNNGmrOfJ6cZ5bIyy:22 a=QOGEsqRv6VhmHaoFNykA:22 X-Proofpoint-GUID: 4IelBXJEfTWxYuPPu7HJD_H_pvGBWRAK Content-Type: text/plain; charset="utf-8" Having converted so much of the code base to software leaf entries, we can mop up some remaining cases. We replace is_pfn_swap_entry(), pfn_swap_entry_to_page(), is_writable_device_private_entry(), is_device_exclusive_entry(), is_migration_entry(), is_writable_migration_entry(), is_readable_migration_entry(), swp_offset_pfn() and pfn_swap_entry_folio() with softleaf equivalents. No functional change intended. Signed-off-by: Lorenzo Stoakes --- fs/proc/task_mmu.c | 14 ++--- include/linux/leafops.h | 25 +++++++-- include/linux/swapops.h | 121 +--------------------------------------- mm/debug_vm_pgtable.c | 20 +++---- mm/hmm.c | 2 +- mm/hugetlb.c | 2 +- mm/ksm.c | 6 +- mm/memory-failure.c | 6 +- mm/memory.c | 3 +- mm/mempolicy.c | 4 +- mm/migrate.c | 6 +- mm/migrate_device.c | 10 ++-- mm/mprotect.c | 8 +-- mm/page_vma_mapped.c | 8 +-- mm/pagewalk.c | 7 +-- mm/rmap.c | 9 ++- 16 files changed, 75 insertions(+), 176 deletions(-) diff --git a/fs/proc/task_mmu.c b/fs/proc/task_mmu.c index 3cdefa7546db..4deded872c46 100644 --- a/fs/proc/task_mmu.c +++ b/fs/proc/task_mmu.c @@ -1940,13 +1940,13 @@ static pagemap_entry_t pte_to_pagemap_entry(struct = pagemapread *pm, if (pte_uffd_wp(pte)) flags |=3D PM_UFFD_WP; } else { - swp_entry_t entry; + softleaf_t entry; =20 if (pte_swp_soft_dirty(pte)) flags |=3D PM_SOFT_DIRTY; if (pte_swp_uffd_wp(pte)) flags |=3D PM_UFFD_WP; - entry =3D pte_to_swp_entry(pte); + entry =3D softleaf_from_pte(pte); if (pm->show_pfn) { pgoff_t offset; =20 @@ -1954,16 +1954,16 @@ static pagemap_entry_t pte_to_pagemap_entry(struct = pagemapread *pm, * For PFN swap offsets, keeping the offset field * to be PFN only to be compatible with old smaps. */ - if (is_pfn_swap_entry(entry)) - offset =3D swp_offset_pfn(entry); + if (softleaf_has_pfn(entry)) + offset =3D softleaf_to_pfn(entry); else offset =3D swp_offset(entry); frame =3D swp_type(entry) | (offset << MAX_SWAPFILES_SHIFT); } flags |=3D PM_SWAP; - if (is_pfn_swap_entry(entry)) - page =3D pfn_swap_entry_to_page(entry); + if (softleaf_has_pfn(entry)) + page =3D softleaf_to_page(entry); if (softleaf_is_uffd_wp_marker(entry)) flags |=3D PM_UFFD_WP; if (softleaf_is_guard_marker(entry)) @@ -2032,7 +2032,7 @@ static int pagemap_pmd_range_thp(pmd_t *pmdp, unsigne= d long addr, if (pmd_swp_uffd_wp(pmd)) flags |=3D PM_UFFD_WP; VM_WARN_ON_ONCE(!pmd_is_migration_entry(pmd)); - page =3D pfn_swap_entry_to_page(entry); + page =3D softleaf_to_page(entry); } =20 if (page) { diff --git a/include/linux/leafops.h b/include/linux/leafops.h index 9be9a4e8ada4..d593093ba70c 100644 --- a/include/linux/leafops.h +++ b/include/linux/leafops.h @@ -355,7 +355,7 @@ static inline unsigned long softleaf_to_pfn(softleaf_t = entry) VM_WARN_ON_ONCE(!softleaf_has_pfn(entry)); =20 /* Temporary until swp_entry_t eliminated. */ - return swp_offset_pfn(entry); + return swp_offset(entry) & SWP_PFN_MASK; } =20 /** @@ -366,10 +366,16 @@ static inline unsigned long softleaf_to_pfn(softleaf_= t entry) */ static inline struct page *softleaf_to_page(softleaf_t entry) { + struct page *page =3D pfn_to_page(softleaf_to_pfn(entry)); + VM_WARN_ON_ONCE(!softleaf_has_pfn(entry)); + /* + * Any use of migration entries may only occur while the + * corresponding page is locked + */ + VM_WARN_ON_ONCE(softleaf_is_migration(entry) && !PageLocked(page)); =20 - /* Temporary until swp_entry_t eliminated. */ - return pfn_swap_entry_to_page(entry); + return page; } =20 /** @@ -381,10 +387,17 @@ static inline struct page *softleaf_to_page(softleaf_= t entry) */ static inline struct folio *softleaf_to_folio(softleaf_t entry) { - VM_WARN_ON_ONCE(!softleaf_has_pfn(entry)); + struct folio *folio =3D pfn_folio(softleaf_to_pfn(entry)); =20 - /* Temporary until swp_entry_t eliminated. */ - return pfn_swap_entry_folio(entry); + VM_WARN_ON_ONCE(!softleaf_has_pfn(entry)); + /* + * Any use of migration entries may only occur while the + * corresponding folio is locked. + */ + VM_WARN_ON_ONCE(softleaf_is_migration(entry) && + !folio_test_locked(folio)); + + return folio; } =20 /** diff --git a/include/linux/swapops.h b/include/linux/swapops.h index c8e6f927da48..3d02b288c15e 100644 --- a/include/linux/swapops.h +++ b/include/linux/swapops.h @@ -28,7 +28,7 @@ #define SWP_OFFSET_MASK ((1UL << SWP_TYPE_SHIFT) - 1) =20 /* - * Definitions only for PFN swap entries (see is_pfn_swap_entry()). To + * Definitions only for PFN swap entries (see leafeant_has_pfn()). To * store PFN, we only need SWP_PFN_BITS bits. Each of the pfn swap entries * can use the extra bits to store other information besides PFN. */ @@ -66,8 +66,6 @@ #define SWP_MIG_YOUNG BIT(SWP_MIG_YOUNG_BIT) #define SWP_MIG_DIRTY BIT(SWP_MIG_DIRTY_BIT) =20 -static inline bool is_pfn_swap_entry(swp_entry_t entry); - /* Clear all flags but only keep swp_entry_t related information */ static inline pte_t pte_swp_clear_flags(pte_t pte) { @@ -109,17 +107,6 @@ static inline pgoff_t swp_offset(swp_entry_t entry) return entry.val & SWP_OFFSET_MASK; } =20 -/* - * This should only be called upon a pfn swap entry to get the PFN stored - * in the swap entry. Please refers to is_pfn_swap_entry() for definition - * of pfn swap entry. - */ -static inline unsigned long swp_offset_pfn(swp_entry_t entry) -{ - VM_BUG_ON(!is_pfn_swap_entry(entry)); - return swp_offset(entry) & SWP_PFN_MASK; -} - /* * Convert the arch-dependent pte representation of a swp_entry_t into an * arch-independent swp_entry_t. @@ -169,27 +156,11 @@ static inline swp_entry_t make_writable_device_privat= e_entry(pgoff_t offset) return swp_entry(SWP_DEVICE_WRITE, offset); } =20 -static inline bool is_device_private_entry(swp_entry_t entry) -{ - int type =3D swp_type(entry); - return type =3D=3D SWP_DEVICE_READ || type =3D=3D SWP_DEVICE_WRITE; -} - -static inline bool is_writable_device_private_entry(swp_entry_t entry) -{ - return unlikely(swp_type(entry) =3D=3D SWP_DEVICE_WRITE); -} - static inline swp_entry_t make_device_exclusive_entry(pgoff_t offset) { return swp_entry(SWP_DEVICE_EXCLUSIVE, offset); } =20 -static inline bool is_device_exclusive_entry(swp_entry_t entry) -{ - return swp_type(entry) =3D=3D SWP_DEVICE_EXCLUSIVE; -} - #else /* CONFIG_DEVICE_PRIVATE */ static inline swp_entry_t make_readable_device_private_entry(pgoff_t offse= t) { @@ -201,50 +172,14 @@ static inline swp_entry_t make_writable_device_privat= e_entry(pgoff_t offset) return swp_entry(0, 0); } =20 -static inline bool is_device_private_entry(swp_entry_t entry) -{ - return false; -} - -static inline bool is_writable_device_private_entry(swp_entry_t entry) -{ - return false; -} - static inline swp_entry_t make_device_exclusive_entry(pgoff_t offset) { return swp_entry(0, 0); } =20 -static inline bool is_device_exclusive_entry(swp_entry_t entry) -{ - return false; -} - #endif /* CONFIG_DEVICE_PRIVATE */ =20 #ifdef CONFIG_MIGRATION -static inline int is_migration_entry(swp_entry_t entry) -{ - return unlikely(swp_type(entry) =3D=3D SWP_MIGRATION_READ || - swp_type(entry) =3D=3D SWP_MIGRATION_READ_EXCLUSIVE || - swp_type(entry) =3D=3D SWP_MIGRATION_WRITE); -} - -static inline int is_writable_migration_entry(swp_entry_t entry) -{ - return unlikely(swp_type(entry) =3D=3D SWP_MIGRATION_WRITE); -} - -static inline int is_readable_migration_entry(swp_entry_t entry) -{ - return unlikely(swp_type(entry) =3D=3D SWP_MIGRATION_READ); -} - -static inline int is_readable_exclusive_migration_entry(swp_entry_t entry) -{ - return unlikely(swp_type(entry) =3D=3D SWP_MIGRATION_READ_EXCLUSIVE); -} =20 static inline swp_entry_t make_readable_migration_entry(pgoff_t offset) { @@ -310,23 +245,10 @@ static inline swp_entry_t make_writable_migration_ent= ry(pgoff_t offset) return swp_entry(0, 0); } =20 -static inline int is_migration_entry(swp_entry_t swp) -{ - return 0; -} - static inline void migration_entry_wait(struct mm_struct *mm, pmd_t *pmd, unsigned long address) { } static inline void migration_entry_wait_huge(struct vm_area_struct *vma, unsigned long addr, pte_t *pte) { } -static inline int is_writable_migration_entry(swp_entry_t entry) -{ - return 0; -} -static inline int is_readable_migration_entry(swp_entry_t entry) -{ - return 0; -} =20 static inline swp_entry_t make_migration_entry_young(swp_entry_t entry) { @@ -410,47 +332,6 @@ static inline swp_entry_t make_guard_swp_entry(void) return make_pte_marker_entry(PTE_MARKER_GUARD); } =20 -static inline struct page *pfn_swap_entry_to_page(swp_entry_t entry) -{ - struct page *p =3D pfn_to_page(swp_offset_pfn(entry)); - - /* - * Any use of migration entries may only occur while the - * corresponding page is locked - */ - BUG_ON(is_migration_entry(entry) && !PageLocked(p)); - - return p; -} - -static inline struct folio *pfn_swap_entry_folio(swp_entry_t entry) -{ - struct folio *folio =3D pfn_folio(swp_offset_pfn(entry)); - - /* - * Any use of migration entries may only occur while the - * corresponding folio is locked - */ - BUG_ON(is_migration_entry(entry) && !folio_test_locked(folio)); - - return folio; -} - -/* - * A pfn swap entry is a special type of swap entry that always has a pfn = stored - * in the swap offset. They can either be used to represent unaddressable = device - * memory, to restrict access to a page undergoing migration or to represe= nt a - * pfn which has been hwpoisoned and unmapped. - */ -static inline bool is_pfn_swap_entry(swp_entry_t entry) -{ - /* Make sure the swp offset can always store the needed fields */ - BUILD_BUG_ON(SWP_TYPE_SHIFT < SWP_PFN_BITS); - - return is_migration_entry(entry) || is_device_private_entry(entry) || - is_device_exclusive_entry(entry) || is_hwpoison_entry(entry); -} - struct page_vma_mapped_walk; =20 #ifdef CONFIG_ARCH_ENABLE_THP_MIGRATION diff --git a/mm/debug_vm_pgtable.c b/mm/debug_vm_pgtable.c index 608d1011ce03..64db85a80558 100644 --- a/mm/debug_vm_pgtable.c +++ b/mm/debug_vm_pgtable.c @@ -844,7 +844,7 @@ static void __init pmd_softleaf_tests(struct pgtable_de= bug_args *args) { } static void __init swap_migration_tests(struct pgtable_debug_args *args) { struct page *page; - swp_entry_t swp; + softleaf_t entry; =20 if (!IS_ENABLED(CONFIG_MIGRATION)) return; @@ -867,17 +867,17 @@ static void __init swap_migration_tests(struct pgtabl= e_debug_args *args) * be locked, otherwise it stumbles upon a BUG_ON(). */ __SetPageLocked(page); - swp =3D make_writable_migration_entry(page_to_pfn(page)); - WARN_ON(!is_migration_entry(swp)); - WARN_ON(!is_writable_migration_entry(swp)); + entry =3D make_writable_migration_entry(page_to_pfn(page)); + WARN_ON(!softleaf_is_migration(entry)); + WARN_ON(!softleaf_is_migration_write(entry)); =20 - swp =3D make_readable_migration_entry(swp_offset(swp)); - WARN_ON(!is_migration_entry(swp)); - WARN_ON(is_writable_migration_entry(swp)); + entry =3D make_readable_migration_entry(swp_offset(entry)); + WARN_ON(!softleaf_is_migration(entry)); + WARN_ON(softleaf_is_migration_write(entry)); =20 - swp =3D make_readable_migration_entry(page_to_pfn(page)); - WARN_ON(!is_migration_entry(swp)); - WARN_ON(is_writable_migration_entry(swp)); + entry =3D make_readable_migration_entry(page_to_pfn(page)); + WARN_ON(!softleaf_is_migration(entry)); + WARN_ON(softleaf_is_migration_write(entry)); __ClearPageLocked(page); } =20 diff --git a/mm/hmm.c b/mm/hmm.c index d5c4e60fbfad..f91c38d4507a 100644 --- a/mm/hmm.c +++ b/mm/hmm.c @@ -265,7 +265,7 @@ static int hmm_vma_handle_pte(struct mm_walk *walk, uns= igned long addr, cpu_flags =3D HMM_PFN_VALID; if (softleaf_is_device_private_write(entry)) cpu_flags |=3D HMM_PFN_WRITE; - new_pfn_flags =3D swp_offset_pfn(entry) | cpu_flags; + new_pfn_flags =3D softleaf_to_pfn(entry) | cpu_flags; goto out; } =20 diff --git a/mm/hugetlb.c b/mm/hugetlb.c index b702b161ab35..f7f18a3ea495 100644 --- a/mm/hugetlb.c +++ b/mm/hugetlb.c @@ -5640,7 +5640,7 @@ int copy_hugetlb_page_range(struct mm_struct *dst, st= ruct mm_struct *src, } else if (unlikely(softleaf_is_migration(softleaf))) { bool uffd_wp =3D pte_swp_uffd_wp(entry); =20 - if (!is_readable_migration_entry(softleaf) && cow) { + if (!softleaf_is_migration_read(softleaf) && cow) { /* * COW mappings require pages in both * parent and child to be set to read. diff --git a/mm/ksm.c b/mm/ksm.c index 7cd19a6ce45f..b911df37f04e 100644 --- a/mm/ksm.c +++ b/mm/ksm.c @@ -637,14 +637,14 @@ static int break_ksm_pmd_entry(pmd_t *pmdp, unsigned = long addr, unsigned long en if (pte_present(pte)) { folio =3D vm_normal_folio(walk->vma, addr, pte); } else if (!pte_none(pte)) { - swp_entry_t entry =3D pte_to_swp_entry(pte); + const softleaf_t entry =3D softleaf_from_pte(pte); =20 /* * As KSM pages remain KSM pages until freed, no need to wait * here for migration to end. */ - if (is_migration_entry(entry)) - folio =3D pfn_swap_entry_folio(entry); + if (softleaf_is_migration(entry)) + folio =3D softleaf_to_folio(entry); } /* return 1 if the page is an normal ksm page or KSM-placed zero page */ found =3D (folio && folio_test_ksm(folio)) || is_ksm_zero_pte(pte); diff --git a/mm/memory-failure.c b/mm/memory-failure.c index acc35c881547..6e79da3de221 100644 --- a/mm/memory-failure.c +++ b/mm/memory-failure.c @@ -691,10 +691,10 @@ static int check_hwpoisoned_entry(pte_t pte, unsigned= long addr, short shift, if (pte_present(pte)) { pfn =3D pte_pfn(pte); } else { - swp_entry_t swp =3D pte_to_swp_entry(pte); + const softleaf_t entry =3D softleaf_from_pte(pte); =20 - if (is_hwpoison_entry(swp)) - pfn =3D swp_offset_pfn(swp); + if (softleaf_is_hwpoison(entry)) + pfn =3D softleaf_to_pfn(entry); } =20 if (!pfn || pfn !=3D poisoned_pfn) diff --git a/mm/memory.c b/mm/memory.c index ad336cbf1d88..accd275cd651 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -902,7 +902,8 @@ static void restore_exclusive_pte(struct vm_area_struct= *vma, static int try_restore_exclusive_pte(struct vm_area_struct *vma, unsigned long addr, pte_t *ptep, pte_t orig_pte) { - struct page *page =3D pfn_swap_entry_to_page(pte_to_swp_entry(orig_pte)); + const softleaf_t entry =3D softleaf_from_pte(orig_pte); + struct page *page =3D softleaf_to_page(entry); struct folio *folio =3D page_folio(page); =20 if (folio_trylock(folio)) { diff --git a/mm/mempolicy.c b/mm/mempolicy.c index dee95d5ecfd4..acb9bf89f619 100644 --- a/mm/mempolicy.c +++ b/mm/mempolicy.c @@ -705,7 +705,9 @@ static int queue_folios_pte_range(pmd_t *pmd, unsigned = long addr, if (pte_none(ptent)) continue; if (!pte_present(ptent)) { - if (is_migration_entry(pte_to_swp_entry(ptent))) + const softleaf_t entry =3D softleaf_from_pte(ptent); + + if (softleaf_is_migration(entry)) qp->nr_failed++; continue; } diff --git a/mm/migrate.c b/mm/migrate.c index 48f98a6c1ad2..182a5b7b2ead 100644 --- a/mm/migrate.c +++ b/mm/migrate.c @@ -483,7 +483,7 @@ void migration_entry_wait(struct mm_struct *mm, pmd_t *= pmd, spinlock_t *ptl; pte_t *ptep; pte_t pte; - swp_entry_t entry; + softleaf_t entry; =20 ptep =3D pte_offset_map_lock(mm, pmd, address, &ptl); if (!ptep) @@ -495,8 +495,8 @@ void migration_entry_wait(struct mm_struct *mm, pmd_t *= pmd, if (pte_none(pte) || pte_present(pte)) goto out; =20 - entry =3D pte_to_swp_entry(pte); - if (!is_migration_entry(entry)) + entry =3D softleaf_from_pte(pte); + if (!softleaf_is_migration(entry)) goto out; =20 migration_entry_wait_on_locked(entry, ptl); diff --git a/mm/migrate_device.c b/mm/migrate_device.c index 880f26a316f8..c50abbd32f21 100644 --- a/mm/migrate_device.c +++ b/mm/migrate_device.c @@ -282,7 +282,7 @@ static int migrate_vma_collect_pmd(pmd_t *pmdp, unsigned long mpfn =3D 0, pfn; struct folio *folio; struct page *page; - swp_entry_t entry; + softleaf_t entry; pte_t pte; =20 pte =3D ptep_get(ptep); @@ -301,11 +301,11 @@ static int migrate_vma_collect_pmd(pmd_t *pmdp, * page table entry. Other special swap entries are not * migratable, and we ignore regular swapped page. */ - entry =3D pte_to_swp_entry(pte); - if (!is_device_private_entry(entry)) + entry =3D softleaf_from_pte(pte); + if (!softleaf_is_device_private(entry)) goto next; =20 - page =3D pfn_swap_entry_to_page(entry); + page =3D softleaf_to_page(entry); pgmap =3D page_pgmap(page); if (!(migrate->flags & MIGRATE_VMA_SELECT_DEVICE_PRIVATE) || @@ -331,7 +331,7 @@ static int migrate_vma_collect_pmd(pmd_t *pmdp, =20 mpfn =3D migrate_pfn(page_to_pfn(page)) | MIGRATE_PFN_MIGRATE; - if (is_writable_device_private_entry(entry)) + if (softleaf_is_device_private_write(entry)) mpfn |=3D MIGRATE_PFN_WRITE; } else { pfn =3D pte_pfn(pte); diff --git a/mm/mprotect.c b/mm/mprotect.c index ab014ce17f9c..476a29cc89bf 100644 --- a/mm/mprotect.c +++ b/mm/mprotect.c @@ -317,11 +317,11 @@ static long change_pte_range(struct mmu_gather *tlb, pages++; } } else { - swp_entry_t entry =3D pte_to_swp_entry(oldpte); + softleaf_t entry =3D softleaf_from_pte(oldpte); pte_t newpte; =20 - if (is_writable_migration_entry(entry)) { - struct folio *folio =3D pfn_swap_entry_folio(entry); + if (softleaf_is_migration_write(entry)) { + const struct folio *folio =3D softleaf_to_folio(entry); =20 /* * A protection check is difficult so @@ -335,7 +335,7 @@ static long change_pte_range(struct mmu_gather *tlb, newpte =3D swp_entry_to_pte(entry); if (pte_swp_soft_dirty(oldpte)) newpte =3D pte_swp_mksoft_dirty(newpte); - } else if (is_writable_device_private_entry(entry)) { + } else if (softleaf_is_device_private_write(entry)) { /* * We do not preserve soft-dirtiness. See * copy_nonpresent_pte() for explanation. diff --git a/mm/page_vma_mapped.c b/mm/page_vma_mapped.c index 8137d2366722..b38a1d00c971 100644 --- a/mm/page_vma_mapped.c +++ b/mm/page_vma_mapped.c @@ -49,7 +49,7 @@ static bool map_pte(struct page_vma_mapped_walk *pvmw, pm= d_t *pmdvalp, if (is_migration) return false; } else if (!is_migration) { - swp_entry_t entry; + softleaf_t entry; =20 /* * Handle un-addressable ZONE_DEVICE memory. @@ -67,9 +67,9 @@ static bool map_pte(struct page_vma_mapped_walk *pvmw, pm= d_t *pmdvalp, * For more details on device private memory see HMM * (include/linux/hmm.h or mm/hmm.c). */ - entry =3D pte_to_swp_entry(ptent); - if (!is_device_private_entry(entry) && - !is_device_exclusive_entry(entry)) + entry =3D softleaf_from_pte(ptent); + if (!softleaf_is_device_private(entry) && + !softleaf_is_device_exclusive(entry)) return false; } spin_lock(*ptlp); diff --git a/mm/pagewalk.c b/mm/pagewalk.c index 3067feb970d1..d6e29da60d09 100644 --- a/mm/pagewalk.c +++ b/mm/pagewalk.c @@ -1000,11 +1000,10 @@ struct folio *folio_walk_start(struct folio_walk *f= w, goto found; } } else if (!pte_none(pte)) { - swp_entry_t entry =3D pte_to_swp_entry(pte); + const softleaf_t entry =3D softleaf_from_pte(pte); =20 - if ((flags & FW_MIGRATION) && - is_migration_entry(entry)) { - page =3D pfn_swap_entry_to_page(entry); + if ((flags & FW_MIGRATION) && softleaf_is_migration(entry)) { + page =3D softleaf_to_page(entry); expose_page =3D false; goto found; } diff --git a/mm/rmap.c b/mm/rmap.c index 775710115a41..345466ad396b 100644 --- a/mm/rmap.c +++ b/mm/rmap.c @@ -1969,7 +1969,7 @@ static bool try_to_unmap_one(struct folio *folio, str= uct vm_area_struct *vma, if (likely(pte_present(pteval))) { pfn =3D pte_pfn(pteval); } else { - pfn =3D swp_offset_pfn(pte_to_swp_entry(pteval)); + pfn =3D softleaf_to_pfn(pte_to_swp_entry(pteval)); VM_WARN_ON_FOLIO(folio_test_hugetlb(folio), folio); } =20 @@ -2368,7 +2368,7 @@ static bool try_to_migrate_one(struct folio *folio, s= truct vm_area_struct *vma, if (likely(pte_present(pteval))) { pfn =3D pte_pfn(pteval); } else { - pfn =3D swp_offset_pfn(pte_to_swp_entry(pteval)); + pfn =3D softleaf_to_pfn(pte_to_swp_entry(pteval)); VM_WARN_ON_FOLIO(folio_test_hugetlb(folio), folio); } =20 @@ -2453,8 +2453,11 @@ static bool try_to_migrate_one(struct folio *folio, = struct vm_area_struct *vma, folio_mark_dirty(folio); writable =3D pte_write(pteval); } else { + const softleaf_t entry =3D softleaf_from_pte(pteval); + pte_clear(mm, address, pvmw.pte); - writable =3D is_writable_device_private_entry(pte_to_swp_entry(pteval)); + + writable =3D softleaf_is_device_private_write(entry); } =20 VM_WARN_ON_FOLIO(writable && folio_test_anon(folio) && --=20 2.51.0 From nobody Fri Dec 19 21:47:08 2025 Received: from mx0a-00069f02.pphosted.com (mx0a-00069f02.pphosted.com [205.220.165.32]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C747E1448E0; Sat, 8 Nov 2025 17:15:39 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=205.220.165.32 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1762622141; cv=fail; b=Q3RTlb1eaq8d8Mpr7zd1eKHtUfslEpmS5sv2RBthVrDwn8WnAy3BbMe7QtMu8QUyLoGAm42uxmzK03PoOhyu40L6GR+Stpt/8KL0flH8VgaGRZh3Wd1R9JWLXZf/xsFl8GoXEtRXWvibZpo3ayCVx29kNyol87Cd/7XL8osgz7E= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1762622141; c=relaxed/simple; bh=SGCSom7NQN4a5UuTz6cnQyKevdTqh7TJ1tPjPhx7f2g=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: Content-Type:MIME-Version; b=fqZJyY6z8Sv9mEG/Jm1/Tbv5tByCF8GzwjaJmvQhnaHbS3VXDGUbnRtVIoYp29RZ00fbxuUI/+Vo4lFuCDUgr91joFoetr7fvH2eyo4d+7AmDoQqwDse5EC+3Uru/rAFXS7D7TLk/fHgytZ+ZTYtbnxdQs9Yk8NxX6+MJilb1Yc= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=oracle.com; spf=pass smtp.mailfrom=oracle.com; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b=ltaDTdCN; dkim=pass (1024-bit key) header.d=oracle.onmicrosoft.com header.i=@oracle.onmicrosoft.com header.b=Sn+nWGYp; arc=fail smtp.client-ip=205.220.165.32 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=oracle.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=oracle.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b="ltaDTdCN"; dkim=pass (1024-bit key) header.d=oracle.onmicrosoft.com header.i=@oracle.onmicrosoft.com header.b="Sn+nWGYp" Received: from pps.filterd (m0246627.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 5A8FpXSu008953; Sat, 8 Nov 2025 17:09:41 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=cc :content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s= corp-2025-04-25; bh=FrZyx+G0nFRqqG3Ukf0WKDJ2lnH9WlkIF2fwTUy9qzA=; b= ltaDTdCNB0SOBwpDtGmVIwAyDSPFxLYSI7I/0+PHMGh+MxT2ysbUZfgYqHnv/MYJ n8u0R1xHrG5PZxUW/5pyYY9OzVESZ6HhP0j3vKFU2kmr/u7TfkcOqtahvD7hrZ4b hgp+70rW8YngtLBC3Me3veI5WyZTQpsxGkbn/OxvQKkQ6Gv2rq/DNUSRDB3Ja8HN QIcVax0Pf5zBL7i+o5MY9mospjwOvUKqxurvKUvs2PeA2I4Q3kCW8vERFtJNgBno V6J7ycVaXhIfbmHYggqBPMb9ZWSeJkIqPigwPFmuKmysXChilHKc1g75lABnzOuM hUybIgNB4QdOAvgvSMzxBg== Received: from phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta01.appoci.oracle.com [138.1.114.2]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 4aa6vh84sx-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Sat, 08 Nov 2025 17:09:40 +0000 (GMT) Received: from pps.filterd (phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (8.18.1.2/8.18.1.2) with ESMTP id 5A8BijdP007604; Sat, 8 Nov 2025 17:09:40 GMT Received: from bl2pr02cu003.outbound.protection.outlook.com (mail-eastusazon11011002.outbound.protection.outlook.com [52.101.52.2]) by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTPS id 4a9va76n3y-5 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Sat, 08 Nov 2025 17:09:39 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=W7Icn/7ZMDwUYC0tzdkywBrUfmnsjao1Cmc80PWs7bH8XqKu/0C5SMgPE/G1tfnw/oo0F6rrwF2IaTKDgJ6bsacjGsyW3wTzEXpJXDfKc3qWAN60/gcBc7JvC3/6CEFqIUJYK7fLZ2R0qjnqMQHhKyRTzK2rSU+zHiHxwsDTtRX9YvjeqqLMlaOExQHqo4RcoEIvHlnCxMyDBmV7nsKv0zhEaYQkyaJCBpFrgBfqbwTVUk0gImQkFJWkSZUfPXbVV4j4mLFetU7F10sPbRAg/TDO3120kqFhqz20/C0WMn07sUUetOt1xxMRKFa6lW/RgjnN/gtgHd2b9yR0WbGI7g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=FrZyx+G0nFRqqG3Ukf0WKDJ2lnH9WlkIF2fwTUy9qzA=; b=XNgeIMAJ6EDwDj43whEruOIlOKYDZLGwdYcx5sfsWVNpACJOOMuI3xugNaDbR6WJjAotgx1gpZnlE0alJtOH+tP1MCmtgoNVTJAYMkD19NqQuBLEx/6ArQ2ifvfGvwXcUc0n3KBUkcMbNEG/3y8GjhXcCKOV8fr6MP/LqvV2XyNGWDJZFL3tIQSTMyoEYurGNgravYRYz4yEZ1pkOnGkcVh3yP8WT7nm4II8LwyY68g7VDith4MQaukUJrC5SIP5YNLAjRzOIRl60F+hptnb+vkDAl+RrX5FaYh0jphU5UG2G3sGXFoqmeIHWtXtJPzCI9SsCNydYaLdfkopE2vS9g== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=FrZyx+G0nFRqqG3Ukf0WKDJ2lnH9WlkIF2fwTUy9qzA=; b=Sn+nWGYp7FEl8gbAdpFsSsmUixi7VLF6QhyUzpbGwXzt1MgKSRopw9p869jcUO2ZPLpkzaomE9a+OCUSbJYDMihSzplh5lLcl/NjpU7GHW7KsUxwWIIta9z780WK3Uv3iEZvAoEWC7IrLggSonS2bEPwuLu1D8vzGNgTwhJmf4s= Received: from DM4PR10MB8218.namprd10.prod.outlook.com (2603:10b6:8:1cc::16) by SJ5PPF04D2D7FA7.namprd10.prod.outlook.com (2603:10b6:a0f:fc02::785) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9298.15; Sat, 8 Nov 2025 17:09:36 +0000 Received: from DM4PR10MB8218.namprd10.prod.outlook.com ([fe80::2650:55cf:2816:5f2]) by DM4PR10MB8218.namprd10.prod.outlook.com ([fe80::2650:55cf:2816:5f2%7]) with mapi id 15.20.9298.010; Sat, 8 Nov 2025 17:09:36 +0000 From: Lorenzo Stoakes To: Andrew Morton Cc: Christian Borntraeger , Janosch Frank , Claudio Imbrenda , David Hildenbrand , Alexander Gordeev , Gerald Schaefer , Heiko Carstens , Vasily Gorbik , Sven Schnelle , Peter Xu , Alexander Viro , Christian Brauner , Jan Kara , Arnd Bergmann , Zi Yan , Baolin Wang , "Liam R . Howlett" , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Lance Yang , Muchun Song , Oscar Salvador , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Matthew Brost , Joshua Hahn , Rakie Kim , Byungchul Park , Gregory Price , Ying Huang , Alistair Popple , Axel Rasmussen , Yuanchu Xie , Wei Xu , Kemeng Shi , Kairui Song , Nhat Pham , Baoquan He , Chris Li , SeongJae Park , Matthew Wilcox , Jason Gunthorpe , Leon Romanovsky , Xu Xin , Chengming Zhou , Jann Horn , Miaohe Lin , Naoya Horiguchi , Pedro Falcato , Pasha Tatashin , Rik van Riel , Harry Yoo , Hugh Dickins , linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-s390@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, linux-arch@vger.kernel.org, damon@lists.linux.dev Subject: [PATCH v2 16/16] mm: replace remaining pte_to_swp_entry() with softleaf_from_pte() Date: Sat, 8 Nov 2025 17:08:30 +0000 Message-ID: X-Mailer: git-send-email 2.51.0 In-Reply-To: References: Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: LO6P123CA0022.GBRP123.PROD.OUTLOOK.COM (2603:10a6:600:313::15) To DM4PR10MB8218.namprd10.prod.outlook.com (2603:10b6:8:1cc::16) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DM4PR10MB8218:EE_|SJ5PPF04D2D7FA7:EE_ X-MS-Office365-Filtering-Correlation-Id: 3390fe3f-cd74-4368-a37c-08de1ee9983e X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|1800799024|7416014|376014|366016; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?bXMYWIH8W09RyWA5UlMUCEzqPEVW7AOcAvEttErrJzqoWmkMbp23cMnItpc2?= =?us-ascii?Q?tElmyFSt/6zVpUqBpBEW3XQTiyHXZg9tHN4RLJGn9yp+xrRcvmwo8qHg4neN?= =?us-ascii?Q?aILbA3EoATTTCYovE3lsdTc3GKQhFYjAXDggq1I5Vc8SBjiUgPPWyLMFPcPa?= =?us-ascii?Q?8huS6WEzf/y7lH1ubQUCOMCkMlMCNc0lksC2DKJ5H0+duJIW70Y7kWM7BJAn?= =?us-ascii?Q?0eSNvm06LkdmZknilE5Zs7JWAb5QSTQHIVcR1t34eH7kAduA33tfCS0VDRvT?= =?us-ascii?Q?1WFD3TvmJeAyOI0aA7rtsEPReDtdhVGGHQFE0fFaYomGaeCH70/muFNFyxt+?= =?us-ascii?Q?7Jc2CgNk1EiKFRjGRULCx40qfMUSSsVLKa6BIdqEHenTRBXHdHMxHCQgx00v?= =?us-ascii?Q?ePndOVJvB/Qc6kqi+An69QQRnTh6k2Zaui+i4y78CoJhMxtwb/uPTATPtWBi?= =?us-ascii?Q?wR4Cw9d8lSaccaXg+BqmHRIo1up14G8j/9bSgDEE7/C1xhuWYm0l1fhyKMVA?= =?us-ascii?Q?l9dg0SUhtQYVFvZKs1idD7ARZdRAOD/xcUiiFRzD7HJ42Cu/0NB0hZMrGLpM?= =?us-ascii?Q?jgw3Se3dg9ju+a75q2wXd4WF96KRFmpUzeCnZGdUOq5sGbfvg6o8iyep7hyg?= =?us-ascii?Q?q3+w4fC73caqA4HYCwWTLyhnNPY8YuAvFMRQXnOYVzQztzwpfvdA6+dwkvNW?= =?us-ascii?Q?+GchHirfR1y6dDQ7LFdXv6uT2Kiybw2gVsW6UBcNWkrRIvoqX7E+caUqP0Ol?= =?us-ascii?Q?zboZM0RwtGXlB77/x6wZdpfQ6AXzTVyhpqBACxGxYMKZrlzsaMOmmy8UFcbK?= =?us-ascii?Q?PVedDBNgh6e9pJAf4Kb2qFU4XFoanFXU6qggE9ZCCC9Uj4CXjGifltW/Lz8W?= =?us-ascii?Q?UzzMEd1FktCWgmBTDSuE+F5nz1cnkj3UO79KERLgVEXHelukXXz5LOswbVJv?= =?us-ascii?Q?QyJHECpa82IuNg8/8VkI7mfkgF47i+aDrBSY/58njAmJo86Co5EGtwGdg8Iu?= =?us-ascii?Q?bXTloI2AXawKZP/VzHhfC7y/8lMT9TaHf9UI9jtS6TrkdO2SipEFsSueVx//?= =?us-ascii?Q?rMaIIFQQb0RrDdEbauCMUdTl5oguiAaJPti0QInU0Pv6EaRfxkbwi0VCpZ3Y?= =?us-ascii?Q?y0jmSgaaYu3Q4xHhb59gKAArbOnu00z5JKZnzn350PD405SUzoghsTMIidJI?= =?us-ascii?Q?BsXYCiq25AVWFlcJTemYpFrcgdrKfYlz5+vPW18nBTyGQK5u4QZYgiGgVELv?= =?us-ascii?Q?pFbDJLC42a5ZDaeMNyHvWTtI416F87c2tRmvGTFoHkGtsiSYxJClUycZnR0a?= =?us-ascii?Q?4JckgaIqhTeO3Z6fGkPRHR1Ah6KecbtE/5Ddlp11J7Nqp9GTgvA+G8hEqau2?= =?us-ascii?Q?IVFNny5JApAtl4CsoU15/03WM8kwgs9Xx/l85wCxAoEiYqL2kUj3u+9gAOHe?= =?us-ascii?Q?uRpxEw0ZPqvmw0Yi245SFfwe7qjrWVmt?= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DM4PR10MB8218.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(1800799024)(7416014)(376014)(366016);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?0cB5fGeAudFamLkH+3asl93S+AWPK6FGMtdR04gh5wzZF8SCTT4M6waFZ9AS?= =?us-ascii?Q?ZQ5F5mg1q8qqUlGjgzOlzRvX2q9OqRNsYhaJejetLMLloLqAHwhNOBzJBq7H?= =?us-ascii?Q?iAMGo/dBPck1Xb0RNriaklQnS3UDstS2TZmy9mJANlPre1whicpcMm+AgSV8?= =?us-ascii?Q?yU4e+16EwcIfFwBFNbWk019VIDmvplT86ArlhCoCoJkkGciJsM14jYiq6NgL?= =?us-ascii?Q?AwLazdz+vhFwbLt7bborlzyArV//bNfwZenBzxFQBLHc3S69t8QWlCPrIM9Y?= =?us-ascii?Q?x9lH2THLzArhcx+ZNFjxJ05wm5/01tCG0DABaboezxD2Etjhr94yBhoEqAX3?= =?us-ascii?Q?2H/sv8I784xqTvsbgLMR2o0m9TTIT+4PUqfKtO9q/QtIZ4cQyxrkIfroG2zI?= =?us-ascii?Q?4TOKOiWKZN/6mOKhXXiGawqyPD38ast1Ct0b4gR1CJgjpXTudhyrM4VRz4/u?= =?us-ascii?Q?8aDc28D22/PW3cEBGHskocRxSaBAaijlspUKKanCyHkT8J6/3H5EdCGYPh7e?= =?us-ascii?Q?ThIB2z1Vf5lfRXq8+EPjeh0nDTcwpy/WKcs4SR+LcWQFA05EHtgmBft8awM1?= =?us-ascii?Q?47Pm9Nd0lzoEi5UNAuVI/IsRRh9tlYze7GzhGaOfH+zzy8o5QBE95jVuxI0f?= =?us-ascii?Q?xWa7BveTArXp+SYk8hKl6sqIU4VulBrK4rNoOp/JWXAnx26YKIia0D8gAi5e?= =?us-ascii?Q?+FXpGZEnb7Wi8aynbS3pXjxEgMykU5vdqaTXtMUDDJHOv3gAe+CzeyQdplOx?= =?us-ascii?Q?K1xjmqCaG6vUEdgaFCj3+T/ykUOzfa/W9H9qRgMDSAxQZwecZ37WpELRNHXu?= =?us-ascii?Q?kIbYFo3S+AfhAY/7uFSse+FlHCeoeEBB2G2QNdy8He8T9TR7UhAhQvVyDh82?= =?us-ascii?Q?FtmovccWAWqB6SkNQAzO5F4BiqaNXD/lCo1oQA3UGJk7jSbndR3++4CYKwbn?= =?us-ascii?Q?l7OXWGqUWlyHBtjPaiTw4u4ZRzw9RYflbV5oEj6xunwTDeeT9Gh2DCfSmDc2?= =?us-ascii?Q?AoRt/9SGYCWcHwqqnDTT5kQF6yZhy4VRaFUPh1UopPn0hjPYwHfI730oKBBl?= =?us-ascii?Q?61oYI+Y0YteghfiqmRtGDdUhCHSoEMmNJyD4JqihYlevaKJv5W91R6NJPoLZ?= =?us-ascii?Q?XcKnVfh2WWWr6ZkbT/5Ew73J79hrKfA25ImQOEGdxrlfMKvYwBNp6GqDZrSy?= =?us-ascii?Q?L8j+IEfTZDvkJIP9KKr90GM01fTDyQ817ehkrN6QeXtubCbYF+52u1lY+oUa?= =?us-ascii?Q?BvFDMxMSGp910w20cjOw2gcpckQqPRw8DT/jZcGqHZK7EliPxjdqAcxw72Id?= =?us-ascii?Q?U1y75mR7pUm3J7fe8oUmqPvM+KVCq+V9tuoDvu7WpyUsTdAU45nmPKRm8DfU?= =?us-ascii?Q?BO3qyQ9QTxCiF1wTT7luC2noeF8WA3M2dX+MSsDCgZKeZICEdLFp4j1COJ89?= =?us-ascii?Q?67+7mq50CntC3uBTNF5mg8/1r8vlPvdxMyVUZcJIHPRntx/qcTidGfMgcMwC?= =?us-ascii?Q?jNpwx9CuhB6zBImucQenWbXtyyJRPTe24ijrQMjL9y1lYfTZoh060A2hViIW?= =?us-ascii?Q?iKvcVSTg/vpky7n9Nc/U77V8shl/FKcUW1GhKipjxYNeoULxEk7f/y64QKxj?= =?us-ascii?Q?eA=3D=3D?= X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: gARDCpDLL0W3OEX5MTcIs82YZw+BooQ3qtbgpF0oQHALY+LyiAqS4aO4lyiPB/z5X4PD0Xd6g2e191+gBhkWl3AMPaiOfzE0Y6OUdamQRO0MQ+7eDWMMym4QzkfSAw0F5tZWpRrVPDyS+zU2IpBB/7sQC8yg1LKmUOffk9BzGPIZcpkqCUDLMRJGqDKq8FAa9upYzAvOdrwyd6QFDCph8vGRWkCwfcXJQE6j1fWMy5WV9cD6+Pehbz9GmdPx0gCmwmCNiwaJvJ0IYTEbJkDD4fkMhnVAWtmsvYlo7MGAPam4lPoAiE+OAPr4TP2cgnVM1xjFYoDqh908AWqitGgDWZ1RqmiAGpjJm7lgl6H/51hC0Vz+Du4ZbNtN18d5t7voEwdAQncWDgDRiSsdV0QhZzwpnWESYaM1AtMhyMUHdDAvFCzhehnZx6WLWpUdCT4Zg3EJkX0cyBV7MVE89E0WZogHssr+/8yDPtqmI4c+R3B/clq8b3YgWikkNWHcbkswbiQq50KmBZP8iQXTWtY03NOvAwK9sAzW7aHneu8GvMPUFT1sYxIl5LXCuNRSqzZAfNX8abqlI1Nf/iT3pO6L/ysTWZo6fEiE8VfQurPgE3s= X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: 3390fe3f-cd74-4368-a37c-08de1ee9983e X-MS-Exchange-CrossTenant-AuthSource: DM4PR10MB8218.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 Nov 2025 17:09:36.4345 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: uPZ7oowgd6wCzvPHYX9m1zBW7tjeaipr4m+HQOQvnB6Sx8LYsaCE23KzFNsFcmM+BCTFng3U4gPINc435NkysH7j1H9Hym+zHS+VaS1d3oc= X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ5PPF04D2D7FA7 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1121,Hydra:6.1.9,FMLib:17.12.100.49 definitions=2025-11-08_04,2025-11-06_01,2025-10-01_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 adultscore=0 malwarescore=0 spamscore=0 mlxscore=0 bulkscore=0 suspectscore=0 mlxlogscore=999 phishscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2510240000 definitions=main-2511080138 X-Proofpoint-Spam-Details-Enc: AW1haW4tMjUxMTA4MDEwOSBTYWx0ZWRfXz+p3vkxt8l3v /eSGjeNr9D7wiMnuR2Pm2unoed0sy9JVcgS5LbdBKzVVqpk6LQsLVQvtHGkY6G0574DxtE1Vtej Wq6RvIjuRzwKHNJIm6e+yhUlwsCorO/7qtFx4J4riOU/tQWM3VVN5Kx2EZaIA7YxGcqkCDETmyQ auQVu06mUB7vcx5h4EnVD1zwAshLE1cWHYSCUQHB2SoFiEuj1xO4YYOSB2i16iGg3Z1HcFbjrOr cI5o1c4w/ELPdzio52L8KwSFZ4nMAgdp1mBnzJ0hmRVNJmPg3lR6quZ2QUgZWkmr4B9HF4eHfIU AKrVpnChaxtguHtdHBSxcJZypEtceqlzfpij6rTxM7EYJlYWjQFLNdcH2WDCj9Uv/sry1pF5lBk IOBYuki7qeXjnzd3aeh2up7COX/k2Q== X-Authority-Analysis: v=2.4 cv=IKEPywvG c=1 sm=1 tr=0 ts=690f7954 cx=c_pps a=XiAAW1AwiKB2Y8Wsi+sD2Q==:117 a=XiAAW1AwiKB2Y8Wsi+sD2Q==:17 a=6eWqkTHjU83fiwn7nKZWdM+Sl24=:19 a=z/mQ4Ysz8XfWz/Q5cLBRGdckG28=:19 a=lCpzRmAYbLLaTzLvsPZ7Mbvzbb8=:19 a=xqWC_Br6kY4A:10 a=6UeiqGixMTsA:10 a=GoEa3M9JfhUA:10 a=VkNPw1HP01LnGYTKEx00:22 a=yPCof4ZbAAAA:8 a=ZBIxWEUt1cXBTM7wFXgA:9 a=UhEZJTgQB8St2RibIkdl:22 a=Z5ABNNGmrOfJ6cZ5bIyy:22 a=QOGEsqRv6VhmHaoFNykA:22 X-Proofpoint-ORIG-GUID: dKqGnB-33fAtF3jkKw6ilcXfKxOljOCj X-Proofpoint-GUID: dKqGnB-33fAtF3jkKw6ilcXfKxOljOCj Content-Type: text/plain; charset="utf-8" There are straggler invocations of pte_to_swp_entry() lying around, replace all of these with the software leaf entry equivalent - softleaf_from_pte(). With those removed, eliminate pte_to_swp_entry() altogether. No functional change intended. Signed-off-by: Lorenzo Stoakes --- include/linux/leafops.h | 7 ++++++- include/linux/swapops.h | 13 ------------- mm/debug_vm_pgtable.c | 2 +- mm/internal.h | 7 +++++-- mm/memory-failure.c | 2 +- mm/memory.c | 16 ++++++++-------- mm/migrate.c | 2 +- mm/mincore.c | 4 +++- mm/rmap.c | 8 ++++++-- mm/swapfile.c | 5 +++-- 10 files changed, 34 insertions(+), 32 deletions(-) diff --git a/include/linux/leafops.h b/include/linux/leafops.h index d593093ba70c..a464a7e08c76 100644 --- a/include/linux/leafops.h +++ b/include/linux/leafops.h @@ -54,11 +54,16 @@ static inline softleaf_t softleaf_mk_none(void) */ static inline softleaf_t softleaf_from_pte(pte_t pte) { + softleaf_t arch_entry; + if (pte_present(pte)) return softleaf_mk_none(); =20 + pte =3D pte_swp_clear_flags(pte); + arch_entry =3D __pte_to_swp_entry(pte); + /* Temporary until swp_entry_t eliminated. */ - return pte_to_swp_entry(pte); + return swp_entry(__swp_type(arch_entry), __swp_offset(arch_entry)); } =20 /** diff --git a/include/linux/swapops.h b/include/linux/swapops.h index 3d02b288c15e..8cfc966eae48 100644 --- a/include/linux/swapops.h +++ b/include/linux/swapops.h @@ -107,19 +107,6 @@ static inline pgoff_t swp_offset(swp_entry_t entry) return entry.val & SWP_OFFSET_MASK; } =20 -/* - * Convert the arch-dependent pte representation of a swp_entry_t into an - * arch-independent swp_entry_t. - */ -static inline swp_entry_t pte_to_swp_entry(pte_t pte) -{ - swp_entry_t arch_entry; - - pte =3D pte_swp_clear_flags(pte); - arch_entry =3D __pte_to_swp_entry(pte); - return swp_entry(__swp_type(arch_entry), __swp_offset(arch_entry)); -} - /* * Convert the arch-independent representation of a swp_entry_t into the * arch-dependent pte representation. diff --git a/mm/debug_vm_pgtable.c b/mm/debug_vm_pgtable.c index 64db85a80558..1eae87dbef73 100644 --- a/mm/debug_vm_pgtable.c +++ b/mm/debug_vm_pgtable.c @@ -1229,7 +1229,7 @@ static int __init init_args(struct pgtable_debug_args= *args) init_fixed_pfns(args); =20 /* See generic_max_swapfile_size(): probe the maximum offset */ - max_swap_offset =3D swp_offset(pte_to_swp_entry(swp_entry_to_pte(swp_entr= y(0, ~0UL)))); + max_swap_offset =3D swp_offset(softleaf_from_pte(softleaf_to_pte(swp_entr= y(0, ~0UL)))); /* Create a swp entry with all possible bits set while still being swap. = */ args->swp_entry =3D swp_entry(MAX_SWAPFILES - 1, max_swap_offset); /* Create a non-present migration entry. */ diff --git a/mm/internal.h b/mm/internal.h index f0c7461bb02c..985605ba3364 100644 --- a/mm/internal.h +++ b/mm/internal.h @@ -334,7 +334,7 @@ unsigned int folio_pte_batch(struct folio *folio, pte_t= *ptep, pte_t pte, */ static inline pte_t pte_move_swp_offset(pte_t pte, long delta) { - swp_entry_t entry =3D pte_to_swp_entry(pte); + const softleaf_t entry =3D softleaf_from_pte(pte); pte_t new =3D __swp_entry_to_pte(__swp_entry(swp_type(entry), (swp_offset(entry) + delta))); =20 @@ -389,11 +389,14 @@ static inline int swap_pte_batch(pte_t *start_ptep, i= nt max_nr, pte_t pte) =20 cgroup_id =3D lookup_swap_cgroup_id(entry); while (ptep < end_ptep) { + softleaf_t entry; + pte =3D ptep_get(ptep); =20 if (!pte_same(pte, expected_pte)) break; - if (lookup_swap_cgroup_id(pte_to_swp_entry(pte)) !=3D cgroup_id) + entry =3D softleaf_from_pte(pte); + if (lookup_swap_cgroup_id(entry) !=3D cgroup_id) break; expected_pte =3D pte_next_swp_offset(expected_pte); ptep++; diff --git a/mm/memory-failure.c b/mm/memory-failure.c index 6e79da3de221..ca2204c4647e 100644 --- a/mm/memory-failure.c +++ b/mm/memory-failure.c @@ -50,7 +50,7 @@ #include #include #include -#include +#include #include #include #include diff --git a/mm/memory.c b/mm/memory.c index accd275cd651..f9a2c608aff9 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -1218,7 +1218,7 @@ copy_pte_range(struct vm_area_struct *dst_vma, struct= vm_area_struct *src_vma, spinlock_t *src_ptl, *dst_ptl; int progress, max_nr, ret =3D 0; int rss[NR_MM_COUNTERS]; - swp_entry_t entry =3D (swp_entry_t){0}; + softleaf_t entry =3D softleaf_mk_none(); struct folio *prealloc =3D NULL; int nr; =20 @@ -1282,7 +1282,7 @@ copy_pte_range(struct vm_area_struct *dst_vma, struct= vm_area_struct *src_vma, dst_vma, src_vma, addr, rss); if (ret =3D=3D -EIO) { - entry =3D pte_to_swp_entry(ptep_get(src_pte)); + entry =3D softleaf_from_pte(ptep_get(src_pte)); break; } else if (ret =3D=3D -EBUSY) { break; @@ -4456,13 +4456,13 @@ static struct folio *__alloc_swap_folio(struct vm_f= ault *vmf) { struct vm_area_struct *vma =3D vmf->vma; struct folio *folio; - swp_entry_t entry; + softleaf_t entry; =20 folio =3D vma_alloc_folio(GFP_HIGHUSER_MOVABLE, 0, vma, vmf->address); if (!folio) return NULL; =20 - entry =3D pte_to_swp_entry(vmf->orig_pte); + entry =3D softleaf_from_pte(vmf->orig_pte); if (mem_cgroup_swapin_charge_folio(folio, vma->vm_mm, GFP_KERNEL, entry)) { folio_put(folio); @@ -4480,7 +4480,7 @@ static struct folio *__alloc_swap_folio(struct vm_fau= lt *vmf) static bool can_swapin_thp(struct vm_fault *vmf, pte_t *ptep, int nr_pages) { unsigned long addr; - swp_entry_t entry; + softleaf_t entry; int idx; pte_t pte; =20 @@ -4490,7 +4490,7 @@ static bool can_swapin_thp(struct vm_fault *vmf, pte_= t *ptep, int nr_pages) =20 if (!pte_same(pte, pte_move_swp_offset(vmf->orig_pte, -idx))) return false; - entry =3D pte_to_swp_entry(pte); + entry =3D softleaf_from_pte(pte); if (swap_pte_batch(ptep, nr_pages, pte) !=3D nr_pages) return false; =20 @@ -4536,7 +4536,7 @@ static struct folio *alloc_swap_folio(struct vm_fault= *vmf) unsigned long orders; struct folio *folio; unsigned long addr; - swp_entry_t entry; + softleaf_t entry; spinlock_t *ptl; pte_t *pte; gfp_t gfp; @@ -4557,7 +4557,7 @@ static struct folio *alloc_swap_folio(struct vm_fault= *vmf) if (!zswap_never_enabled()) goto fallback; =20 - entry =3D pte_to_swp_entry(vmf->orig_pte); + entry =3D softleaf_from_pte(vmf->orig_pte); /* * Get a list of all the (large) orders below PMD_ORDER that are enabled * and suitable for swapping THP. diff --git a/mm/migrate.c b/mm/migrate.c index 182a5b7b2ead..c01bc0ddf819 100644 --- a/mm/migrate.c +++ b/mm/migrate.c @@ -534,7 +534,7 @@ void migration_entry_wait_huge(struct vm_area_struct *v= ma, unsigned long addr, p * lock release in migration_entry_wait_on_locked(). */ hugetlb_vma_unlock_read(vma); - migration_entry_wait_on_locked(pte_to_swp_entry(pte), ptl); + migration_entry_wait_on_locked(entry, ptl); return; } =20 diff --git a/mm/mincore.c b/mm/mincore.c index e1d50f198c42..62c9603a5414 100644 --- a/mm/mincore.c +++ b/mm/mincore.c @@ -202,7 +202,9 @@ static int mincore_pte_range(pmd_t *pmd, unsigned long = addr, unsigned long end, for (i =3D 0; i < step; i++) vec[i] =3D 1; } else { /* pte is a swap entry */ - *vec =3D mincore_swap(pte_to_swp_entry(pte), false); + const softleaf_t entry =3D softleaf_from_pte(pte); + + *vec =3D mincore_swap(entry, false); } vec +=3D step; } diff --git a/mm/rmap.c b/mm/rmap.c index 345466ad396b..d871f2eb821c 100644 --- a/mm/rmap.c +++ b/mm/rmap.c @@ -1969,7 +1969,9 @@ static bool try_to_unmap_one(struct folio *folio, str= uct vm_area_struct *vma, if (likely(pte_present(pteval))) { pfn =3D pte_pfn(pteval); } else { - pfn =3D softleaf_to_pfn(pte_to_swp_entry(pteval)); + const softleaf_t entry =3D softleaf_from_pte(pteval); + + pfn =3D softleaf_to_pfn(entry); VM_WARN_ON_FOLIO(folio_test_hugetlb(folio), folio); } =20 @@ -2368,7 +2370,9 @@ static bool try_to_migrate_one(struct folio *folio, s= truct vm_area_struct *vma, if (likely(pte_present(pteval))) { pfn =3D pte_pfn(pteval); } else { - pfn =3D softleaf_to_pfn(pte_to_swp_entry(pteval)); + const softleaf_t entry =3D softleaf_from_pte(pteval); + + pfn =3D softleaf_to_pfn(entry); VM_WARN_ON_FOLIO(folio_test_hugetlb(folio), folio); } =20 diff --git a/mm/swapfile.c b/mm/swapfile.c index 684f78cd7dd1..1204fb0726d5 100644 --- a/mm/swapfile.c +++ b/mm/swapfile.c @@ -3201,8 +3201,9 @@ static int claim_swapfile(struct swap_info_struct *si= , struct inode *inode) */ unsigned long generic_max_swapfile_size(void) { - return swp_offset(pte_to_swp_entry( - swp_entry_to_pte(swp_entry(0, ~0UL)))) + 1; + const softleaf_t entry =3D swp_entry(0, ~0UL); + + return swp_offset(softleaf_from_pte(softleaf_to_pte(entry))) + 1; } =20 /* Can be overridden by an architecture for additional checks. */ --=20 2.51.0