From nobody Mon Nov 25 22:30:55 2024 Received: from mx0a-00069f02.pphosted.com (mx0a-00069f02.pphosted.com [205.220.165.32]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E9C1D1E5735 for ; Wed, 23 Oct 2024 20:39:09 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=205.220.165.32 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1729715951; cv=fail; b=Nm1xxOzn1LTsb7KlFr7wO5GLjGW8qlX70O7IMNPxJtmhmd5+iGjfYAGlZaMvOKlV2htN64YFSR4+Z1ldCGX6p/8HDf0Yo5/nT53/Nq5wi+78AdTcphafYFcfRQ7xCJ2QzfKzC/VmWc3MLNwmv7kSXU+bnO8FpOglkhjFD1W/Bw0= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1729715951; c=relaxed/simple; bh=U/VqoPvSp3QgbdtOKlkpWFmnYRwaaHQU0SRBD47aUoI=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: Content-Type:MIME-Version; b=MdWqtC79wojDnOKjOnZxgezNfuErUxoH32gozt0OVGgZNooZ+PkHU6gmfIStJ0i6bSoMrZQRQJqsNCN8CPfRapw9NeAgglpquXaZgzWM7Y/yB4r8m2Bdg4cmcJtpkjw0vVMeZh058jvymZ7pQco9kfHI3P0SaxDRmJLLmc2rHJI= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=oracle.com; spf=pass smtp.mailfrom=oracle.com; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b=mDLV1RSF; dkim=pass (1024-bit key) header.d=oracle.onmicrosoft.com header.i=@oracle.onmicrosoft.com header.b=ncxS35y3; arc=fail smtp.client-ip=205.220.165.32 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=oracle.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=oracle.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b="mDLV1RSF"; dkim=pass (1024-bit key) header.d=oracle.onmicrosoft.com header.i=@oracle.onmicrosoft.com header.b="ncxS35y3" Received: from pps.filterd (m0246627.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 49NKcaVB026722; Wed, 23 Oct 2024 20:38:56 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=cc :content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s= corp-2023-11-20; bh=GoV/SXOqaoQJF6Y962A0upUaI4C5tg8OVEAFdKWNTU8=; b= mDLV1RSFcwAqq0lHdBl8LBxAJeqYvj+hjmbIEIjxiO5cOb211FyG7zHA6X09W4pJ Mt545YKd0HNS0zAGdbMbXjdT4J3Fu1esPtrJ5Qj6x0k8elW8SOhbr14cQ9/GxZge XNh8wg1zxG6SQ4b+A2CSFkykoPIPt2F2HsEBAwXhJ4Sb0NXP26os4lME1p9EOXJ+ rb/4gUuewS7J4b4JTOy47NFyYpknKiKM8lFKQ31WgrfvjrXDSQr7hzVHKmPknlYH mLmazHk4b5rNSERD7d0191yyb+amxVztPBHJzpuQmTEQmsGJG9AJbDoE3PI/3gSX 802bH8QnpdP6PEgHUBonXw== Received: from iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta01.appoci.oracle.com [130.35.100.223]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 42c55v14fv-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 23 Oct 2024 20:38:56 +0000 (GMT) Received: from pps.filterd (iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com (8.18.1.2/8.18.1.2) with ESMTP id 49NJ8ndL027496; Wed, 23 Oct 2024 20:38:55 GMT Received: from nam10-mw2-obe.outbound.protection.outlook.com (mail-mw2nam10lp2042.outbound.protection.outlook.com [104.47.55.42]) by iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTPS id 42emh3bjwp-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 23 Oct 2024 20:38:55 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=L6YR/W7+tVMaZTrxwZKJRP09virM6kTQGbGO32O5li79bMi44RQtiQm9Hy1Vlbo3sC84nThclgr+dqnm/kwCwhID/5pehiV03+AhCzrQgWadtI+KuL5RoSvaW3cH7RBiyYqzKX45Y8s2Rys0R+pOLjikb23PAtS5P6XMKr2gvnu5vsCsL9TRsOsGGVppR2S6HMszTcPVT3rDao+r/1anu8H5ceNXmk6HePnzFw8cZr6GMGUTkx+UOoTJ+be1ecf69XxO+oP887w/32EdWDlvjhntrga7ou5C6fMu/8xBT8srnyKLpFACfyMAvSqvagfsbSRkqs79f5J4BeV/pXvsag== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=GoV/SXOqaoQJF6Y962A0upUaI4C5tg8OVEAFdKWNTU8=; b=S56PHo9kk3XqtuW1igiW/eFZxNGCihVFzpiTJjiN1ti0gz19Ieuju2SGaLMWbGqRYcPnygJ74zMhlGxMkOOttiLSZmwFfZLjDlwSI8Qbc+q6yxBvYXJq4iskM2akhHNjqGG6DyOgpY9+qOYNfAonCCjxtDKLPGdPblXmkZ8OFUVETxkhUqts8pGAyMwfum2PdFX0sBCv8tuxZ8FMPNHA5+3HJeNZtgkI4Rdjfw8pN7AwCQPXZa6gWrL5ITecFNv7Kf7kH033mGOe4U07j4k4OkpSJojDVPjI52c5UESeAor6Ev0PGKW4/yPDIpcIda6r+fawzHsLzvcPrPpdEy5hSA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=GoV/SXOqaoQJF6Y962A0upUaI4C5tg8OVEAFdKWNTU8=; b=ncxS35y32JBpihG9WaLuv0ymC4l/nkwVtj32Sy81dL1WWA+44B+ijyfNSbWYGeQ8vxiTztK6O5DtWveO2BY9XJ0ig16QwUkQcxASAmLk+T2vbeVpmTC0eUVw/9L4Dv1AfZa/samrcY7nzeCSomr9rTbGniulTgChqHnNwllqLyI= Received: from BYAPR10MB3366.namprd10.prod.outlook.com (2603:10b6:a03:14f::25) by SJ0PR10MB5833.namprd10.prod.outlook.com (2603:10b6:a03:3ed::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8093.17; Wed, 23 Oct 2024 20:38:52 +0000 Received: from BYAPR10MB3366.namprd10.prod.outlook.com ([fe80::baf2:dff1:d471:1c9]) by BYAPR10MB3366.namprd10.prod.outlook.com ([fe80::baf2:dff1:d471:1c9%6]) with mapi id 15.20.8069.024; Wed, 23 Oct 2024 20:38:52 +0000 From: Lorenzo Stoakes To: Andrew Morton Cc: "Liam R . Howlett" , Vlastimil Babka , Jann Horn , linux-kernel@vger.kernel.org, linux-mm@kvack.org, Linus Torvalds , Peter Xu Subject: [PATCH hotfix 6.12 v2 1/8] mm: avoid unsafe VMA hook invocation when error arises on mmap hook Date: Wed, 23 Oct 2024 21:38:26 +0100 Message-ID: <69f3c04df1ece2b7d402a29451ec19290ff429a4.1729715266.git.lorenzo.stoakes@oracle.com> X-Mailer: git-send-email 2.47.0 In-Reply-To: References: Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: LO4P302CA0012.GBRP302.PROD.OUTLOOK.COM (2603:10a6:600:2c2::19) To BYAPR10MB3366.namprd10.prod.outlook.com (2603:10b6:a03:14f::25) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BYAPR10MB3366:EE_|SJ0PR10MB5833:EE_ X-MS-Office365-Filtering-Correlation-Id: 4b27283d-c3df-483f-f08e-08dcf3a2b479 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|366016|1800799024|376014|10070799003; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?5FMzz/agnPPgTvbwQPjJjvoTCLL02RKS9WTSlDJDr++1bRblfAge84g4xg3r?= =?us-ascii?Q?WuSoDBs+FHgo9lfJ52CM1COyc+V87TT2OrBns/WPDE75Ylh4KOZ+fmq44Dqt?= =?us-ascii?Q?yoacgSxJxd23G1khMTnUGl7cyU7jVBQzqBUVZn6n87ykncRYm1O7MmCjxH2p?= =?us-ascii?Q?bsLxPrSoscAUWpe+HilBQy9Y2DjNHZ4U/hPnaw1VVBfcsdltPZ9KoBqbZHtv?= =?us-ascii?Q?1XzJA/v5SipeUeSSlMwZVD6GnK1fobd34xsbYy7T8rFzaeBWSzDi77PdSiII?= =?us-ascii?Q?mo/hXj6o+cNr4a61WzfgEMNGh7b2K+w5zb3tmqfCwzv7JdZckSlG6LGhdvzt?= =?us-ascii?Q?zu0hs78OAu0BjUgKQrAyO6fOMvcXp7+RtGRtE3Wf6BpXJlu04OR0jrNs/6Lt?= =?us-ascii?Q?Q1ynLI2Au+aeVFGrhQ39h4aP6UYLRVD/+pyOacifjDEcSBVgmDs8v0lYUJDT?= =?us-ascii?Q?ibuu6a9X2itlgzS3yLPSHkS+k3oTb2FBJXlUzjUULuYltTRDMTBmHXPpINlw?= =?us-ascii?Q?BkIxGaH8Crtke1NhKddL5WBq01M4FYafAB+1ktayJcPsJpnsIyD9Ms3C0Zjf?= =?us-ascii?Q?ihn9ch+ooG3gxxu6T1OoCtl/lI1lMrhDn9sl3AepFIa5IwAGArdnLOuQpzW9?= =?us-ascii?Q?osp7ZZROeJ8vBEAPBFf7VBCDHn+b/8gIuq1FsZYND2dquhEKtEBYWFslk4Vj?= =?us-ascii?Q?gbjLOt0OFoI7Zry2vVdqvEvo2xkq4wpW9A/C9wv+wEEU+B8g9hp9p4Me7j99?= =?us-ascii?Q?ac35GAcico9VswOiTwS4eDtgs4AYDqSwkTsjWd/bVCGrbp4Mo9EbQ3t3KDDb?= =?us-ascii?Q?ivbz04/3XQ3o7OOKlqzmQzHJAFCUVwRgKAzjJ1htDyLg4WDn3v2Jcm6QE8e2?= =?us-ascii?Q?+/FZ5+SjfhcAJT9mw+BqEr9w3MNAQM/7/mcorkMaXnb6pXCogwJ4qtQj/p3u?= =?us-ascii?Q?1TWMdpSJQ2czE0jRKoV5JzWlVxK+vn2qnu52l5ZPLn3uuawoYKTSDQoeUqGt?= =?us-ascii?Q?NQXk18qKiG+MgxrsXf4QoIsRJw2Vvfjzc1mct4+LtYD0jj5DZ9lnpSbsHH5b?= =?us-ascii?Q?5KP8UxT3wkg+8U0SPE/hYsFjFzKU+8HeXz5efNanqKSb7ZXXyrhxE6fH7k9u?= =?us-ascii?Q?eYS4/QCsHQfyY2s+Pr7Trix1qX5casA2lyX3LTIyHpmnbc55l8iYzNWe7MGj?= =?us-ascii?Q?FfP4Djs8w7tztHhigHkH0bcORD4PY+bITZljWT0elkTZZFU/E5xR2lrBaHdh?= =?us-ascii?Q?xX2ee9EGdqyrRwSX2e8aaegY0uQCYW89Rm8JlFfTIFFL9XFBn/qghXg+UYK0?= =?us-ascii?Q?YTQVYv6f6+T8ceK4nZQWz8LG?= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BYAPR10MB3366.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(366016)(1800799024)(376014)(10070799003);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?ZMQ1Gvd6zCBXXxtzXig2Frz3zU9AKhAbnEhHOfc2H/iZ95Uuze+EX2IX8lz2?= =?us-ascii?Q?Nb1OL+uS19rMEA/Uq6+PW9dXTCs+NE88HBUSjrw902xMBYtaB9bwGnmDdL1V?= =?us-ascii?Q?6JxFAPMV7b+nlwtm0peRdxj1Ir9OxGSaOJRgf/9Brie2F1JLswFc9IWgtnzY?= =?us-ascii?Q?JfDDFTD1Jz47+uAbSgit5OSPA4C6mi6qGVq1pj7wRkdpSShlEc0QytggG0E1?= =?us-ascii?Q?IcFITZa96kSvibGN0UbmbjF7WPw+s4zsV464L4vQnco17NzMcYOla8dTQrjJ?= =?us-ascii?Q?Mp7nLt4SWbOJpfJEugpgI5rKaDxOr9NsSD6uxJ869/LZXDkQLgoV6AOABKN7?= =?us-ascii?Q?2iiEydxzBPeXCG5lVF5asww96Iu+kRUuf6kR0g1O/U3O6gzUI0jTC+v3koaS?= =?us-ascii?Q?mL/OIzZlOpWi/aikQ202alVr1z/Ik2eREgJ6ChFQipHQLGpxekisA3YmlpNO?= =?us-ascii?Q?Gwi1QRGfLsiM0SBXk6LSE6P0/RuvKsmAqi03r8Mq4yA35nHNRPhUMCEKe/MQ?= =?us-ascii?Q?coVtiWIMzZWRwFFv0cKDLOVk056hOqhFr3gWvkdA0ytZfZi9SVBqHXGoWE3e?= =?us-ascii?Q?hp8m4HzIOCKmR5ITQ5xXXV3fmmDkLYdQi8ZtQ0QqTbpEt0nn9qlsR4mn08fK?= =?us-ascii?Q?Az6PcJYmAUkGsNI96N55AU5JVUC4IPdLX0ggLZpHlm37hv7cGP0fec4dT4FF?= =?us-ascii?Q?NQCwvkKy0+kDrzMt6VjtUHKoCIDUUZCm8AqtRKH4EUeUQiOtlSQIUx8EyYoj?= =?us-ascii?Q?zFjq2ktgKJNB4ZN+NTKq2PZbd3KM/kFaQMct4Uubhp6sIiaghr6wyctAqfM3?= =?us-ascii?Q?9gSx8yzMKsuRD6KlkenDUYfPNNwAogB/GvrjRAQ9mBUUepD3x5vq8j9ZolYe?= =?us-ascii?Q?rFJab+oh09L8gcrI94hyE0llK6mc5/4AOxB9jNRLRO8dD3i4voO6V6jMeaOi?= =?us-ascii?Q?4OVefD7FmmOj+ykKaMxaWp05Q2uS3i1zK0s+qsDIonlB1WYgzuQ4ULQczA2D?= =?us-ascii?Q?opx7w5+HA5nJe242SGzKNohsFu5JpiS8Q07y3AxYgb6y5vfWTv465xcfJDW1?= =?us-ascii?Q?cfqmyYIWj3NSQl9+GsD1XdS+Ya9PhPr/EFHP/STH0gRqlO4PT2DfRrDVEwPh?= =?us-ascii?Q?D/gcV+d7CXmxF2xjGScVr+L+Fdy2jobvPWF90vVdHSWTCrqyxrqm+4W9mVm9?= =?us-ascii?Q?tlgC5Mdgwuiqpi0VzRDvqe8NNMjwL/t52R3FdhrcaUjrxIqatpp/wXhi2eV1?= =?us-ascii?Q?8EHsd2M/CBb4QahQ3uHHpuuCQN9hmq5fMibjHdYn50I+SbZeF625+rWmNTOo?= =?us-ascii?Q?HVC7+NEXJJHoRIWn5+7qET7L5aqC3FmU6TqH4jK95aWp2sQ4C0+m+34UTorU?= =?us-ascii?Q?e4q7agSK0L4GLsrPDE/zCDN/4+xAbyfQTkEx2fplsW42aTdjd3QRJFqvFeFG?= =?us-ascii?Q?mVNY41rzQ1Hli3kvAwchk2v6X6MFNN2Bp8kE7CThF4ikisqR70I1IHa3oGIv?= =?us-ascii?Q?uLGuklQo/8+lmL4DdDzk03uPmWlvCFZj4582nOaQ9uxeOg/9v8tK5MCsiH+I?= =?us-ascii?Q?sn95/Vep/giMOQ+Rg3lXYdt5MT+wCmDN+VmccrMmb6kaSiWY8B+xiGwa4PK9?= =?us-ascii?Q?2rLIL0orvVj2pm1G9wzAvHefOIk5Zfxt42TO22oOQbVO6tQ6yIpW40UJvNvv?= =?us-ascii?Q?FRU0rg=3D=3D?= X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: Aou/9qqGc0ReEn9gOXVADm31mcSjK85tauEwTvkT6x1Ywv/Z7ZJDn2espdSmNpbwhSJccawy6Krvc1ChCOm9/nnUj5+zrKpPakhEEq+5zpcy5lczvudacdzi80GPab1i+oaXrdaf8z6vYDRgMLwvPGfr8S/XTLSiJ6L60iXc203q9uRC8KSda6/92dSHYD1+/dD+Dj2rExUfxY0CG8HDZhDiR72bWpdsyY1AD343G73ssKqYtJKMuzaiLgttvt1INNS1o5KMX8R55ZAk5xTCYlz1yMkBhfUpkUEKMCEVGCZ1jXE4X1RtcA/mNX1zIzb9M1o4w80pJHr4OLIkEi74q5t4q64SwlPKo5WiTAFwW8ZXxaMPKYCfFfw7aj3Z3AwSICHkG3Jap5u+F3wqyogzA07JA3J6bDLjY6tw24ngd/vb13Of+okoqA44wsolyMMIBQskLMxYFNvjkJWZG2v4DxazNX81UVw1sgDWIa2C/497GhORFamrsYTN2QSqflahJ4chbltjrpmJANmkKnYxwZ2vktbAI1FC42I46yRVYnTuUM8RV7Gqh+sL+HrtEpP5r7V8t1wvNKLhJjw1q0X9x+JSTOSPVYaKq7q6ZTYeW68= X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: 4b27283d-c3df-483f-f08e-08dcf3a2b479 X-MS-Exchange-CrossTenant-AuthSource: BYAPR10MB3366.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 23 Oct 2024 20:38:52.0661 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: uIA4w2DI3F9qNt+lZNffz4TVxRCEz6XXtN+4FM2FuiLKuMOCANnJNIjZkypxE5ydWJNPlVmlzcxOrxhc218pXulGWx6P86cmPVK1aFB/8DU= X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ0PR10MB5833 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1051,Hydra:6.0.680,FMLib:17.12.62.30 definitions=2024-10-23_16,2024-10-23_01,2024-09-30_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=0 phishscore=0 malwarescore=0 adultscore=0 mlxscore=0 mlxlogscore=999 spamscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2409260000 definitions=main-2410230132 X-Proofpoint-GUID: 9UzNX2otY6yG9b7VdoE1gg2T7uVYINpX X-Proofpoint-ORIG-GUID: 9UzNX2otY6yG9b7VdoE1gg2T7uVYINpX Content-Type: text/plain; charset="utf-8" After an attempted mmap() fails, we are no longer in a situation where we can safely interact with VMA hooks. This is currently not enforced, meaning that we need complicated handling to ensure we do not incorrectly call these hooks. We can avoid the whole issue by treating the VMA as suspect the moment that the file->f_ops->mmap() function reports an error by replacing whatever VMA operations were installed with a dummy empty set of VMA operations. We do so through a new helper function internal to mm - mmap_file() - which is both more logically named than the existing call_mmap() function and correctly isolates handling of the vm_op reassignment to mm. All the existing invocations of call_mmap() outside of mm are ultimately nested within the call_mmap() from mm, which we now replace. It is therefore safe to leave call_mmap() in place as a convenience function (and to avoid churn). The invokers are: ovl_file_operations -> mmap -> ovl_mmap() -> backing_file_mmap() coda_file_operations -> mmap -> coda_file_mmap() shm_file_operations -> shm_mmap() shm_file_operations_huge -> shm_mmap() dma_buf_fops -> dma_buf_mmap_internal -> i915_dmabuf_ops -> i915_gem_dmabuf_mmap() None of these callers interact with vm_ops or mappings in a problematic way on error, quickly exiting out. Reported-by: Jann Horn Fixes: deb0f6562884 ("mm/mmap: undo ->mmap() when arch_validate_flags() fai= ls") Cc: stable Reviewed-by: Liam R. Howlett Reviewed-by: Vlastimil Babka Reviewed-by: Jann Horn Signed-off-by: Lorenzo Stoakes --- mm/internal.h | 27 +++++++++++++++++++++++++++ mm/mmap.c | 6 +++--- mm/nommu.c | 4 ++-- 3 files changed, 32 insertions(+), 5 deletions(-) diff --git a/mm/internal.h b/mm/internal.h index 508f7802dd2b..af032e76dfd4 100644 --- a/mm/internal.h +++ b/mm/internal.h @@ -108,6 +108,33 @@ static inline void *folio_raw_mapping(const struct fol= io *folio) return (void *)(mapping & ~PAGE_MAPPING_FLAGS); } +/* + * This is a file-backed mapping, and is about to be memory mapped - invok= e its + * mmap hook and safely handle error conditions. On error, VMA hooks will = be + * mutated. + * + * @file: File which backs the mapping. + * @vma: VMA which we are mapping. + * + * Returns: 0 if success, error otherwise. + */ +static inline int mmap_file(struct file *file, struct vm_area_struct *vma) +{ + int err =3D call_mmap(file, vma); + + if (likely(!err)) + return 0; + + /* + * OK, we tried to call the file hook for mmap(), but an error + * arose. The mapping is in an inconsistent state and we most not invoke + * any further hooks on it. + */ + vma->vm_ops =3D &vma_dummy_vm_ops; + + return err; +} + #ifdef CONFIG_MMU /* Flags for folio_pte_batch(). */ diff --git a/mm/mmap.c b/mm/mmap.c index 1ba0878bbc30..10f4ccaf491b 100644 --- a/mm/mmap.c +++ b/mm/mmap.c @@ -1425,7 +1425,7 @@ unsigned long mmap_region(struct file *file, unsigned= long addr, /* * clear PTEs while the vma is still in the tree so that rmap * cannot race with the freeing later in the truncate scenario. - * This is also needed for call_mmap(), which is why vm_ops + * This is also needed for mmap_file(), which is why vm_ops * close function is called. */ vms_clean_up_area(&vms, &mas_detach); @@ -1450,7 +1450,7 @@ unsigned long mmap_region(struct file *file, unsigned= long addr, if (file) { vma->vm_file =3D get_file(file); - error =3D call_mmap(file, vma); + error =3D mmap_file(file, vma); if (error) goto unmap_and_free_vma; @@ -1473,7 +1473,7 @@ unsigned long mmap_region(struct file *file, unsigned= long addr, vma_iter_config(&vmi, addr, end); /* - * If vm_flags changed after call_mmap(), we should try merge + * If vm_flags changed after mmap_file(), we should try merge * vma again as we may succeed this time. */ if (unlikely(vm_flags !=3D vma->vm_flags && vmg.prev)) { diff --git a/mm/nommu.c b/mm/nommu.c index 385b0c15add8..f9ccc02458ec 100644 --- a/mm/nommu.c +++ b/mm/nommu.c @@ -885,7 +885,7 @@ static int do_mmap_shared_file(struct vm_area_struct *v= ma) { int ret; - ret =3D call_mmap(vma->vm_file, vma); + ret =3D mmap_file(vma->vm_file, vma); if (ret =3D=3D 0) { vma->vm_region->vm_top =3D vma->vm_region->vm_end; return 0; @@ -918,7 +918,7 @@ static int do_mmap_private(struct vm_area_struct *vma, * happy. */ if (capabilities & NOMMU_MAP_DIRECT) { - ret =3D call_mmap(vma->vm_file, vma); + ret =3D mmap_file(vma->vm_file, vma); /* shouldn't return success if we're not sharing */ if (WARN_ON_ONCE(!is_nommu_shared_mapping(vma->vm_flags))) ret =3D -ENOSYS; -- 2.47.0 From nobody Mon Nov 25 22:30:55 2024 Received: from mx0b-00069f02.pphosted.com (mx0b-00069f02.pphosted.com [205.220.177.32]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 253581E7C28 for ; Wed, 23 Oct 2024 20:39:11 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=205.220.177.32 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1729715954; cv=fail; b=VpDlepvPLaxoSzpjCzO4uSbcN1kqjDI4pL5pk6L2o8D5eOeAt2AQMijvS5Sj0GPdAwN5xwTeVqeQ9K6GgSnxo0yyc0iisfOqHZ7wxoOC4iZ6ZF/6JZA3d9cVRLK4n/loHnw0F6H6FiJHBYE990hTVF5/YW2zuQVLab23exbnXlU= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1729715954; c=relaxed/simple; bh=0AtljQsUjyNvv7nmAyLP3+47OrwaqkiAmdVw9my8rbo=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: Content-Type:MIME-Version; b=GtnU4MLxZ40QwKKITRosoMh9PwBQdUEMuEqdnOTSZ9UUwD2dZXBG9aWcQMhTlLrams/HeZGh7wttal1UkdBBSKgaewXo10JBoBaIdbTMeAWIZrsnyWDHwQ5v9GoUHgw5Lo+EfZgHD8SqD8TRnKUAb6lAZMXj1omOeJAME6ZBcR0= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=oracle.com; spf=pass smtp.mailfrom=oracle.com; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b=Ab7rGpCW; dkim=pass (1024-bit key) header.d=oracle.onmicrosoft.com header.i=@oracle.onmicrosoft.com header.b=geJxMFBR; arc=fail smtp.client-ip=205.220.177.32 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=oracle.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=oracle.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b="Ab7rGpCW"; dkim=pass (1024-bit key) header.d=oracle.onmicrosoft.com header.i=@oracle.onmicrosoft.com header.b="geJxMFBR" Received: from pps.filterd (m0246631.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 49NKcZKS025393; Wed, 23 Oct 2024 20:39:00 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=cc :content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s= corp-2023-11-20; bh=isYXjEUHVJpnaUR/SKlmRO3SkZ58UHRLRDJ+bIB+HNg=; b= Ab7rGpCWlni6wz5UCHnIpmQGIT2Lc5L7WvQBXZhlk22Bk2Gb0ib5YhY557RbXpTU muZuPIJr64Bwnq+eN9bFHci6b9ZM3/FWVGj5UbgljPoH69ePxhATc7ZLP6+OxkvY j2XlUJ1y8JWy2qfxi4UnwRpfIibGkRsDuEj9wNIqz0dgvwzQf8SlbnbrP3FtHRWc aMPtTKnZ0LUBaDecWzn+ENwO+QkLFetXQTsp90eC+p09pKxRF7amLNS1Z1BwQi/O jD45y8n6kq5yxH8UYJMkP/RVS4M0f3rBoYR5Kozg/F+cGzRm7+yQnXcLASSfW6g4 2y2tkXK+PaenWgsvebublA== Received: from iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta03.appoci.oracle.com [130.35.103.27]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 42c57qh5ct-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 23 Oct 2024 20:39:00 +0000 (GMT) Received: from pps.filterd (iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com (8.18.1.2/8.18.1.2) with ESMTP id 49NJUVje039523; Wed, 23 Oct 2024 20:39:00 GMT Received: from nam04-mw2-obe.outbound.protection.outlook.com (mail-mw2nam04lp2170.outbound.protection.outlook.com [104.47.73.170]) by iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTPS id 42emhbj44w-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 23 Oct 2024 20:38:59 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=q2bBUG09nj28x37rNkTfmaziMfGAxuLy0e+ffwaG6sv7WIfxqhH1HkpsfsqHhksRf3iej3mjAzi35d6aanOr2ojSY+kqU2IivzwyaU7pNE3pV1UyKbNutLF9qMCBGisG1cSuZGKg44HlkGy02VSUr9BAD1zS61+DA9l/Dob8hZgrkgTjClvpgwHiYV1K8vWgGQx0HVVszrve8giSvO7Ae6Dc0oJkpfaoXLaa1YoWAL3GXmxUJLYbiEOJQYkXGIxMdN67n0cb5cKzd+WyezjD8ca01ibZRffRjh9rHAFPPdVkbiiQ1tKavgXzrJHdIOLnLI+zJpoUQ2A4yVmqlVMWgA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=isYXjEUHVJpnaUR/SKlmRO3SkZ58UHRLRDJ+bIB+HNg=; b=l+/IrsnWzOSAvwLGlhBR32DHcceENHoFnfo1jKIXlZ4c+kwDRe+4WoDtDElOz+3EXwTXIMaNSWTT9tRbOR9wn4Q3yAk1kvyUNb3h1ufvJVz85mrdegkV6uwMq0oEhlMoQMlNrTLpo08imDma6sgub3mApOj2frpTIRwCrJ3O1ByapN4URWt0FoaZAMbQQ7wsi37KQd8U+/uJuI2ByRRzHeMeNP7WLDhDGkHTmPzFRyLm9imFlYc19HQDJWPsPz5B1Jg+tghraKQT+LwHDUegmJe0+13lDoh37t8/4BDwghimuO/ElRiqZaJEZvwLhHa683yJDSiB+GU/fOspuEiOaw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=isYXjEUHVJpnaUR/SKlmRO3SkZ58UHRLRDJ+bIB+HNg=; b=geJxMFBRfc2bgwob1tvBUEdI29x02RnKDAJj4nQOntFb5m7StU+yWM7TygScCfl72nwXGv6ftGYKouwij/cWpdl8CdrtNJ43Aj9SNtyygUwC/xPcylMnY123pYJwUEdqF/+vT6HZFvg0mjHE6U3f2TltfZfpC1Qo2EuYgEfJImU= Received: from BYAPR10MB3366.namprd10.prod.outlook.com (2603:10b6:a03:14f::25) by SJ0PR10MB5833.namprd10.prod.outlook.com (2603:10b6:a03:3ed::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8093.17; Wed, 23 Oct 2024 20:38:57 +0000 Received: from BYAPR10MB3366.namprd10.prod.outlook.com ([fe80::baf2:dff1:d471:1c9]) by BYAPR10MB3366.namprd10.prod.outlook.com ([fe80::baf2:dff1:d471:1c9%6]) with mapi id 15.20.8069.024; Wed, 23 Oct 2024 20:38:57 +0000 From: Lorenzo Stoakes To: Andrew Morton Cc: "Liam R . Howlett" , Vlastimil Babka , Jann Horn , linux-kernel@vger.kernel.org, linux-mm@kvack.org, Linus Torvalds , Peter Xu Subject: [PATCH hotfix 6.12 v2 2/8] mm: unconditionally close VMAs on error Date: Wed, 23 Oct 2024 21:38:27 +0100 Message-ID: <72a81a6fb997508db644313a5fdf4d239f620da6.1729715266.git.lorenzo.stoakes@oracle.com> X-Mailer: git-send-email 2.47.0 In-Reply-To: References: Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: LO4P123CA0028.GBRP123.PROD.OUTLOOK.COM (2603:10a6:600:151::15) To BYAPR10MB3366.namprd10.prod.outlook.com (2603:10b6:a03:14f::25) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BYAPR10MB3366:EE_|SJ0PR10MB5833:EE_ X-MS-Office365-Filtering-Correlation-Id: 35cb7490-a50d-4333-26c7-08dcf3a2b767 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|366016|1800799024|376014|10070799003; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?YAEl1lPPiYImrNxjt+zyAmv1NiHnF0PHzrzNgteahTq6cb54YpKIdepOsTG1?= =?us-ascii?Q?ioI38ltRS3mULSbid+9iHoIAheTMF6G7eLojlWct0NsI3xx3AF+YDoG72xT6?= =?us-ascii?Q?WiJU+SPLZ63yRLCJYFSV/J44BmAiVqdxnRr986bEs4sY0xFk04Axj0pFIFtF?= =?us-ascii?Q?SmDAMX19kwb4pRdf4fyB4tOF8U/CG7ORem4acEMNySD5yN4M2OvTScRb7NdK?= =?us-ascii?Q?4kxnROBvrKlV01A5vnJk14Py3l72WvXlEk2JYHCE3wrNTPM9B3gTahWyNTEi?= =?us-ascii?Q?+S1n3SY28fDGoso4ReTf6qccZXq63dn6lP6W4gv/+sY+beMFqxucLmfs0AFn?= =?us-ascii?Q?RNow6z2VXfpXtAe4NAbk5g7Y2R4lJqASP8FWbmofezH7E/cFAq2YubMTZP7/?= =?us-ascii?Q?iKrB9FGZYUEbK2cfS2QHnvixKZjCN/lDwtFhoI8ZXpa38QQNyT1Z5s54sZS5?= =?us-ascii?Q?JPYbjyB8v3UuwvIUbbF+eZxUqe4eIZYlk+8JGgdzaAnkY+ZVspnjBVgEv7SR?= =?us-ascii?Q?APrngLczPukcIbmQrx9ynOhPyTYiUf5FGQKrtW2C6KU+Es1EQgZKisaEGCpS?= =?us-ascii?Q?Hc5abFtJO/APf/VI5qJJDOqLh2vJxGAjf5Gr5oW0tGqGSu6lBJQNuGn89wN8?= =?us-ascii?Q?mogAaKcTGDOiHTR5VmEO03txBnbRhvvsXWHRQKtsMXtWmqb365IRK8tDynqO?= =?us-ascii?Q?UzCU2Pj3+7UrPB0rW2wD2vJAoEUFywBU1a8Uo7kWmMpwZeAILCJ2gWJ2600s?= =?us-ascii?Q?cQdSGRC51fFQ+O0z5bcnP3GcD8JWsDyS4NWd3rJkspd1PNoFOErFITGA7e7T?= =?us-ascii?Q?zPg5AMIdgu8skAq2IfY1VlDy+ZQ+STXZ9YMdtTlBimzW2Ll6WX9oJVmteCCB?= =?us-ascii?Q?Na1QzVs+baCqEWB2xw9BxWt1vRIyrM0qKeNzmIKsZRDSl7D4EdAQZDvPIiQO?= =?us-ascii?Q?FJhixCkh9iwuwaYg7u18Z9NS3OYi7qMxBuknBap+XqoLShCWvZjBlsX8hNP2?= =?us-ascii?Q?c0AAJF372VoJ1FJyxdhPQYUhmN24rHA1pqFPt47wboEITMw3zcO9GEkt4+Ng?= =?us-ascii?Q?GFeWxVF79ORCUgGx1oQgmusOukhGC1Jj4sm+SuLePlwNaIJF/WTHlQqLWXGK?= =?us-ascii?Q?1IJM4RGfGTfN99U6vw8M2tYIZV032tkq1GpjjXHhU1CkZSzDaKUBPLvu8LCh?= =?us-ascii?Q?/6zzKWQTlwZORSDRM95EH6v26a+5sQUZjPfgcLoi8qu40nfoJi0b9DkZDEaY?= =?us-ascii?Q?+trIdOof7IAL+szQob2Hfr0iZNkm7fgaq65cNhPtSb6pE1tFfvmyaR1JJhm9?= =?us-ascii?Q?YUW9mIiLM+STKi/Zwoulc0Z7?= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BYAPR10MB3366.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(366016)(1800799024)(376014)(10070799003);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?Sm/bshCd9g8cr+jL+Ur/LwBauj31IbvNij81H1GV4X1pSM5O9yOtm3KMO20B?= =?us-ascii?Q?jJzeCaVnreUbc+l9Hq+zs4IpeLXwk0b6XwGrP3vruyAHwpUj+7bkPKo9WTKy?= =?us-ascii?Q?njZPlryi8rF4339t1itEsEN9B3q77iw9kwn84Kf0GJdWOJ2lQjlXdIgGWrtQ?= =?us-ascii?Q?kAa2B6T9eYr2j1mjmCggjqdUQQUrn88EtbgGqTfQLyuAyrqeqwxNTEA06N16?= =?us-ascii?Q?B4gXtNFdj5eWVAaOT2X3C+71/G0PudsyUgL04ux9x9tU46EAxSWgoYn1WIIT?= =?us-ascii?Q?qODY04Vth40fM3ijQ41TGFXU1bAf7J1cljviVe6E5oQHUX4K9BLFtFd2hVAK?= =?us-ascii?Q?yNaCMK6fz75rFXCfnBmMxO4G829OttYtiCTer7+R8ljf4+BcWSeA0R0bzEZS?= =?us-ascii?Q?qDz7afpp6JcO10yAeJNoxISERPumL4DGQLLFQ4Bdc4Jxa/D0WWdZ2dLw7Z6t?= =?us-ascii?Q?pRB4H06AQIEF2Bt/pzh2yhLg/3L+d6iMmZwUCmS3uvWOlqupGb8rlmBMUlP5?= =?us-ascii?Q?qV5+NKkGB0SdtPhGVOYPWiR9/ZCgryzWjfgXz1OmmJ7TYeiwxVbdp1gB336f?= =?us-ascii?Q?mM9LCNapISLSvE9PTl25GhcuZqJZ8/z4+ocWmUMJm2xUhXI/k7WHdqA9lCqo?= =?us-ascii?Q?eRNRVQo41//mBrVucJDSlFv0ZIuapkcd1BCxN19FE5QGz7Zhhrjg9DYWjDdd?= =?us-ascii?Q?NmvzzKdyplc1JW+nbEYzpsUwAaQn1jjQA6wy4j5cE6vEp1mujVJ70+NeGY5E?= =?us-ascii?Q?m2JZrk9zHaKVSExctVxn+dxncHrsKmFN9d57pl3eIdehF0jajPgm44BkA+IT?= =?us-ascii?Q?NSVnV5TfNUmVET/U/Pxs6rKArz2HwkapQo5VqGhYiBhTag/zosPVikI6k2AB?= =?us-ascii?Q?FX8ipm6MW2ukzdj8p6aQOt4pc8eTAG1fAPPJI60UXAv5TOSoCr6L+zFT7bUe?= =?us-ascii?Q?saLIfXoNipBs8UseAZToVFtvh2QPGgjZCUxh1APeVLKgxB/dCcE1jBwamKjb?= =?us-ascii?Q?QummqqacWeIGPMO3s8Pm6ylonVTATncOLYCu+ZbkpF1h4P/vT6rII/RKt89H?= =?us-ascii?Q?klQfAAxpSmyS92duU1mqXq5z+P6sxvSlPa80saLUidHPstPNjhFd0LgSAKgB?= =?us-ascii?Q?WBvxLkCzGt4ZEJMvwazM0C6T0icqmdmMosArj7/ptnA/2bL4/wrqBw2WMQmW?= =?us-ascii?Q?hqXBs2LkMAS7iBL0WoBeFZhkK4LjAu1qgCo26PywpEWQ6bJrf//2Cs4jUEUZ?= =?us-ascii?Q?ARdAg4hRwRpwOZ7lfhPriGLqxYQ3EphvR1tSsqFiousPU+reFOf7k2gmPWf9?= =?us-ascii?Q?PSJNBzdNslxyDRogMzhEQkkfYPi/CEfBCgnNhibjUpY941n2zkGX3Cj3WCot?= =?us-ascii?Q?++jYq81TR9Ex1eeFJAigqjhcjXjsCY7qofZII51XTvtnRAfi9QSHHdNa9nF0?= =?us-ascii?Q?TLaBL4CmRo2fL1AUqZPImKBxahKs15rq1jZWfxIhgl2Jk3ilpGAygCAUEsVh?= =?us-ascii?Q?w79SfGatVU4jnEcBX9OaaT4UdOnhVtJlp+jRDDEeh0wH0CTqu+b9D7JqKsSD?= =?us-ascii?Q?H2Z0kqx1AVIuE+UekNek2Np1PVrZjdPtJN7Ozck9XVmoyL/drSs5sfpQfJi5?= =?us-ascii?Q?X4LiemRJEFRu5VO7rXF2U2VqTTDIcwWb8cWbOId7LqAPZRM8jDEWbYxh4nik?= =?us-ascii?Q?x7MyNg=3D=3D?= X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: sJvXq9AZbcscLgU/dpaqPmk1P9sXog6t4+B+bsosd2zPDz0/fHnMwQcZUDlHcbCxpsa4zhqMNUNbcZP3/rDIcgaYjQjwQjKZV2vheEz6YNuezWJ9+FPGcu9Ij2r3FAyIsCxiIWIyBk7Jb6a3SX5NNeEGzLu4rBBKbaIZqfbdIZrtpP0ndbW/sXzLsvXp9jbjwMUn/tvd1/+X8PtfDVLyW7FGCO1V2F/4mk0rx78841NKFdYlAOmmLoKApINDD43QHPIrldcD9mqbvajb8ZMphCtOuveb4dp2VRca3Ag+z+WVrjFsirCxaeGatPc1ktgX2X/kymObxMTN1fjNOam6JdYgjSuhJU5s0e8cdHipBzP3GSma/yaQXTWTHiankjztmhYRx9zyS+YEFg7UnNtc0ClbYgsVEHd6tFvK/PzTIMKR6cY3iJpiJWuQON+rJElhjs1WjL7frlz33MjTDMANBPKyISdZflg11GYKFIGYN85Vkp7rWB5wOi+pWj4r9RD4btNiEW4u9UONX6TudQWpY9zHUcuH1oeGol1y9xXixusWgf9ZshSMbSU8LHYmRsXF73SK0pXgBJwQCGFFT7PTnfcdep7dj4oqvp7w2bHkyjA= X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: 35cb7490-a50d-4333-26c7-08dcf3a2b767 X-MS-Exchange-CrossTenant-AuthSource: BYAPR10MB3366.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 23 Oct 2024 20:38:56.9640 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: k6RfI3Q4fRgYXsv8RSxSb5/u/rKfOdXBd6jEGTI/n8lLoPG1k85wTWEXzwaO5y+TRdvL4iOlrQRXcB4AV+qPWEPaf+UHneqDzrGY7h43fnY= X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ0PR10MB5833 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1051,Hydra:6.0.680,FMLib:17.12.62.30 definitions=2024-10-23_16,2024-10-23_01,2024-09-30_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 spamscore=0 mlxscore=0 phishscore=0 suspectscore=0 bulkscore=0 adultscore=0 malwarescore=0 mlxlogscore=999 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2409260000 definitions=main-2410230132 X-Proofpoint-ORIG-GUID: 9god8jj7Mp6doCKpcF5T5EvsObyK1tZs X-Proofpoint-GUID: 9god8jj7Mp6doCKpcF5T5EvsObyK1tZs Content-Type: text/plain; charset="utf-8" Incorrect invocation of VMA callbacks when the VMA is no longer in a consistent state is bug prone and risky to perform. With regards to the important vm_ops->close() callback We have gone to great lengths to try to track whether or not we ought to close VMAs. Rather than doing so and risking making a mistake somewhere, instead unconditionally close and reset vma->vm_ops to an empty dummy operations set with a NULL .close operator. We introduce a new function to do so - vma_close() - and simplify existing vms logic which tracked whether we needed to close or not. This simplifies the logic, avoids incorrect double-calling of the .close() callback and allows us to update error paths to simply call vma_close() unconditionally - making VMA closure idempotent. Reported-by: Jann Horn Fixes: deb0f6562884 ("mm/mmap: undo ->mmap() when arch_validate_flags() fai= ls") Cc: stable Reviewed-by: Vlastimil Babka Reviewed-by: Liam R. Howlett Reviewed-by: Jann Horn Signed-off-by: Lorenzo Stoakes --- mm/internal.h | 18 ++++++++++++++++++ mm/mmap.c | 5 ++--- mm/nommu.c | 3 +-- mm/vma.c | 14 +++++--------- mm/vma.h | 4 +--- 5 files changed, 27 insertions(+), 17 deletions(-) diff --git a/mm/internal.h b/mm/internal.h index af032e76dfd4..0c4bf09bf788 100644 --- a/mm/internal.h +++ b/mm/internal.h @@ -135,6 +135,24 @@ static inline int mmap_file(struct file *file, struct = vm_area_struct *vma) return err; } +/* + * If the VMA has a close hook then close it, and since closing it might l= eave + * it in an inconsistent state which makes the use of any hooks suspect, c= lear + * them down by installing dummy empty hooks. + */ +static inline void vma_close(struct vm_area_struct *vma) +{ + if (vma->vm_ops && vma->vm_ops->close) { + vma->vm_ops->close(vma); + + /* + * The mapping is in an inconsistent state, and no further hooks + * may be invoked upon it. + */ + vma->vm_ops =3D &vma_dummy_vm_ops; + } +} + #ifdef CONFIG_MMU /* Flags for folio_pte_batch(). */ diff --git a/mm/mmap.c b/mm/mmap.c index 10f4ccaf491b..d55c58e99a54 100644 --- a/mm/mmap.c +++ b/mm/mmap.c @@ -1576,8 +1576,7 @@ unsigned long mmap_region(struct file *file, unsigned= long addr, return addr; close_and_free_vma: - if (file && !vms.closed_vm_ops && vma->vm_ops && vma->vm_ops->close) - vma->vm_ops->close(vma); + vma_close(vma); if (file || vma->vm_file) { unmap_and_free_vma: @@ -1937,7 +1936,7 @@ void exit_mmap(struct mm_struct *mm) do { if (vma->vm_flags & VM_ACCOUNT) nr_accounted +=3D vma_pages(vma); - remove_vma(vma, /* unreachable =3D */ true, /* closed =3D */ false); + remove_vma(vma, /* unreachable =3D */ true); count++; cond_resched(); vma =3D vma_next(&vmi); diff --git a/mm/nommu.c b/mm/nommu.c index f9ccc02458ec..635d028d647b 100644 --- a/mm/nommu.c +++ b/mm/nommu.c @@ -589,8 +589,7 @@ static int delete_vma_from_mm(struct vm_area_struct *vm= a) */ static void delete_vma(struct mm_struct *mm, struct vm_area_struct *vma) { - if (vma->vm_ops && vma->vm_ops->close) - vma->vm_ops->close(vma); + vma_close(vma); if (vma->vm_file) fput(vma->vm_file); put_nommu_region(vma->vm_region); diff --git a/mm/vma.c b/mm/vma.c index 3c5a80876725..bb7cfa2dc282 100644 --- a/mm/vma.c +++ b/mm/vma.c @@ -323,11 +323,10 @@ static bool can_vma_merge_right(struct vma_merge_stru= ct *vmg, /* * Close a vm structure and free it. */ -void remove_vma(struct vm_area_struct *vma, bool unreachable, bool closed) +void remove_vma(struct vm_area_struct *vma, bool unreachable) { might_sleep(); - if (!closed && vma->vm_ops && vma->vm_ops->close) - vma->vm_ops->close(vma); + vma_close(vma); if (vma->vm_file) fput(vma->vm_file); mpol_put(vma_policy(vma)); @@ -1115,9 +1114,7 @@ void vms_clean_up_area(struct vma_munmap_struct *vms, vms_clear_ptes(vms, mas_detach, true); mas_set(mas_detach, 0); mas_for_each(mas_detach, vma, ULONG_MAX) - if (vma->vm_ops && vma->vm_ops->close) - vma->vm_ops->close(vma); - vms->closed_vm_ops =3D true; + vma_close(vma); } /* @@ -1160,7 +1157,7 @@ void vms_complete_munmap_vmas(struct vma_munmap_struc= t *vms, /* Remove and clean up vmas */ mas_set(mas_detach, 0); mas_for_each(mas_detach, vma, ULONG_MAX) - remove_vma(vma, /* =3D */ false, vms->closed_vm_ops); + remove_vma(vma, /* unreachable =3D */ false); vm_unacct_memory(vms->nr_accounted); validate_mm(mm); @@ -1684,8 +1681,7 @@ struct vm_area_struct *copy_vma(struct vm_area_struct= **vmap, return new_vma; out_vma_link: - if (new_vma->vm_ops && new_vma->vm_ops->close) - new_vma->vm_ops->close(new_vma); + vma_close(new_vma); if (new_vma->vm_file) fput(new_vma->vm_file); diff --git a/mm/vma.h b/mm/vma.h index 55457cb68200..75558b5e9c8c 100644 --- a/mm/vma.h +++ b/mm/vma.h @@ -42,7 +42,6 @@ struct vma_munmap_struct { int vma_count; /* Number of vmas that will be removed */ bool unlock; /* Unlock after the munmap */ bool clear_ptes; /* If there are outstanding PTE to be cle= ared */ - bool closed_vm_ops; /* call_mmap() was encountered, so vmas may be close= d */ /* 1 byte hole */ unsigned long nr_pages; /* Number of pages being removed */ unsigned long locked_vm; /* Number of locked pages */ @@ -198,7 +197,6 @@ static inline void init_vma_munmap(struct vma_munmap_st= ruct *vms, vms->unmap_start =3D FIRST_USER_ADDRESS; vms->unmap_end =3D USER_PGTABLES_CEILING; vms->clear_ptes =3D false; - vms->closed_vm_ops =3D false; } #endif @@ -269,7 +267,7 @@ int do_vmi_munmap(struct vma_iterator *vmi, struct mm_s= truct *mm, unsigned long start, size_t len, struct list_head *uf, bool unlock); -void remove_vma(struct vm_area_struct *vma, bool unreachable, bool closed); +void remove_vma(struct vm_area_struct *vma, bool unreachable); void unmap_region(struct ma_state *mas, struct vm_area_struct *vma, struct vm_area_struct *prev, struct vm_area_struct *next); -- 2.47.0 From nobody Mon Nov 25 22:30:55 2024 Received: from mx0a-00069f02.pphosted.com (mx0a-00069f02.pphosted.com [205.220.165.32]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4DD701F471B for ; Wed, 23 Oct 2024 20:39:17 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=205.220.165.32 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1729715958; cv=fail; b=VEb8XWtFZtJnZ8+tVuIf1BaVUAe/mS2RRw+2SW5dEeDHJOurc9cVDBK13QVQfcs6IVAWgnC6RAc4EJE9T8TIZbxZaafTtCuuE7SVJv6Iw4iXViev+doQzaB3EPMZNgxgJYB/V9aaLMBzvtD07N+m6YXKikr6L7sQ57VqPSPm1pE= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1729715958; c=relaxed/simple; bh=ZDlslQsibN1rGDSDxSonHj9X4+3bg7PC2SBRCECqv04=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: Content-Type:MIME-Version; b=RYPBt415ht6yrf0xMPSGWmoc4yJGst/RNRt7bxJ1dg1E9rKCLgkp8w3bNgycv945N/9FrrToYpjvaR+iUDFGRYozVLJjVwVpXZW7xYhsqMOZkboOSBK5Wfk3o+GgIZP86tpJvE75cSoZctLQ8Rkcq2xW21XGRhAJMRS+uiUP1iU= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=oracle.com; spf=pass smtp.mailfrom=oracle.com; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b=g/Noy20H; dkim=pass (1024-bit key) header.d=oracle.onmicrosoft.com header.i=@oracle.onmicrosoft.com header.b=P7GtL/xO; arc=fail smtp.client-ip=205.220.165.32 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=oracle.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=oracle.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b="g/Noy20H"; dkim=pass (1024-bit key) header.d=oracle.onmicrosoft.com header.i=@oracle.onmicrosoft.com header.b="P7GtL/xO" Received: from pps.filterd (m0333521.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 49NKd40K032473; Wed, 23 Oct 2024 20:39:04 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=cc :content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s= corp-2023-11-20; bh=0rDE1LjjLhK0evyvP5sZKz477C4Wubbi+LdXiNbsMzY=; b= g/Noy20HV5Dd8HzkCO2gi2P98GR09+Nti773nFRj0swO5hRJEEkl0SICGDuR1yjN x/usyNE+3+l+7NEcGAyW3WTxUYkbTxRyxxKngGPVWWB3l5hgvon9PjQRID0MjbUo xSHT1qcTGY8BfiIwYe7s3AfeF4XfJji1qydmljWleGKqNrpyn9bMte0I1JY5Idyj Dr1Oo0TikhDPax6Wvf8V03xJ1Nf+pBxczlKyn/51oE3gdMlJYUew/Y0J/pdKvc+H 2H2WCXUyTJf3r43N43a8h9C7lGxdgK/osAd0diCn/I3AfAktW2LxUBuHT++5TDiY wnbw26zHdUfUfYMxoPCxYg== Received: from phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta02.appoci.oracle.com [147.154.114.232]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 42c53us4cs-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 23 Oct 2024 20:39:04 +0000 (GMT) Received: from pps.filterd (phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (8.18.1.2/8.18.1.2) with ESMTP id 49NJk0V1025440; Wed, 23 Oct 2024 20:39:03 GMT Received: from nam04-bn8-obe.outbound.protection.outlook.com (mail-bn8nam04lp2045.outbound.protection.outlook.com [104.47.74.45]) by phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTPS id 42emha1aka-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 23 Oct 2024 20:39:03 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=ukZhib82aRZmQFL9/FpnX8fp81MEqcz9CbalGwf7886y12xhvVJvcpSHdUed1wD9ciTeCtRIZqNWb1vhxpVEDzZqCZBuwEYNU1/+5gvXXwJWSGeztD+nrDqkC6kjhWmp2JpuqDpciKDv/ZZjNY3z7l4UThhJF42xreW4qik19ewin18LRVqle6xY95CtJc3Zo6vCmzWSGlPPWMCM5qrfiM2GrTdfqyEuATs94sceLFvZOGeqHfPfRFkHX6pQtGGoiVfANsVzQ2y2AgWtlMhkaTFPpolKOobK7GuhNouZBtMrhEtbO8+ALQDB1b4xeMDBSmwxJzR1//p3KpSfJokBlQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=0rDE1LjjLhK0evyvP5sZKz477C4Wubbi+LdXiNbsMzY=; b=o1Ae8zZ0NQ/8Z49B3jS29y0ofG6GPNoy8MH6dFi+pIW8PwByde3ZdEq9qLymOJnMcu9t8aXEA3oLtJHXQY3G29pXNKsufZEnl6QmJgOOu81c/S2BI6BMyb9MQnSagsCqShCYJKH5Ifv9CdyRHQooDY+9MlbjTm23Qzwmi9fRZIMMv1h0e7SyA+nIka+E3Tt2YgwvMHgvSmDQ/1/c76jTQV/Jgq22rU6u6CXGleTAu4wfPrbsUi76l72/T2DoHnvdedR1nK/d2vy/gNuyZ/yPVPY8apxtzO6KGY/C0MvIGisHm4MsCFlYPvqRAqrxAQtjXwV5+hbv+Cs8VbmKJmJdqg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=0rDE1LjjLhK0evyvP5sZKz477C4Wubbi+LdXiNbsMzY=; b=P7GtL/xOOyVCLqxowaVBS/nPBC2Fm5xAwRfacJaMVA60zd3mnd2DPFRomMh0QyN1sz7HwaMegoOuWjpcHaZPp+UR7fChpbHuSWyQ15MZodZUjA8Xxn9nMXlMk1Bvj8999hQRGgS/OP1+zyjaHao8JyOeoUXAWK77LSPolCws73s= Received: from BYAPR10MB3366.namprd10.prod.outlook.com (2603:10b6:a03:14f::25) by SJ0PR10MB5833.namprd10.prod.outlook.com (2603:10b6:a03:3ed::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8093.17; Wed, 23 Oct 2024 20:39:00 +0000 Received: from BYAPR10MB3366.namprd10.prod.outlook.com ([fe80::baf2:dff1:d471:1c9]) by BYAPR10MB3366.namprd10.prod.outlook.com ([fe80::baf2:dff1:d471:1c9%6]) with mapi id 15.20.8069.024; Wed, 23 Oct 2024 20:39:00 +0000 From: Lorenzo Stoakes To: Andrew Morton Cc: "Liam R . Howlett" , Vlastimil Babka , Jann Horn , linux-kernel@vger.kernel.org, linux-mm@kvack.org, Linus Torvalds , Peter Xu Subject: [PATCH hotfix 6.12 v2 3/8] mm: refactor map_deny_write_exec() Date: Wed, 23 Oct 2024 21:38:28 +0100 Message-ID: X-Mailer: git-send-email 2.47.0 In-Reply-To: References: Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: LO0P265CA0006.GBRP265.PROD.OUTLOOK.COM (2603:10a6:600:355::17) To BYAPR10MB3366.namprd10.prod.outlook.com (2603:10b6:a03:14f::25) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BYAPR10MB3366:EE_|SJ0PR10MB5833:EE_ X-MS-Office365-Filtering-Correlation-Id: b00151d5-ffde-4ec8-a869-08dcf3a2b989 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|366016|1800799024|376014|10070799003; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?3KII73MuJfN9qT1zp5XFQbPmvQVTagMFSElgtq9NteCNqzfw1sA/KE4MpILE?= =?us-ascii?Q?WNo3DMdYQL6t1xfu4W4GJCWTp6pDWuaLG7OBOw/yIiQJSuzrT0LV+QjB1k78?= =?us-ascii?Q?s80bjeJ9awBfmgmg6XdAd/5xiw0ah/IwB2wUgj7Kjz0P//MmyCO9JtQJ5Xj7?= =?us-ascii?Q?N9tJjrwK5CmEIDfPOF5ZEOvbTvvP1wwNa3IMhXDy4XxRRyudlSey0idNxx+t?= =?us-ascii?Q?5e9l0A4juf4ewwzezC3PNYGpLStP5jGvWoglXgBd+Nld/Gz+8ZYv0Kli1yXm?= =?us-ascii?Q?PdJi5aCPv6nQNOgH7+fcLqpuBaYb6kThdVAmlBMabJxZTTXEiYz9gKWW8eMn?= =?us-ascii?Q?pDY7Okn2VZK6V7791JdAz8PJnTaPUzHvQOks5rRmKg/iy/5Nie/8cw4kO0ws?= =?us-ascii?Q?nSyp+tiiUVpiPMygQ6VwyyCFt3tDLjKq5vK6ALY9wx9/v691ZxsS/X5O5kPX?= =?us-ascii?Q?V/reaPNs4drsZqnNfbL4adPlMGc2zE01Hjjo6vM1pEQYorMY6Glb27uON3sd?= =?us-ascii?Q?SvIGm41Q0/K9358qp+4nXmllyharE6JF/SmTXsZGaWsP0DLky7FJyLjvu5vw?= =?us-ascii?Q?9XWkuJjAlBgRrPgRCzbFB3ykzskl1lotGwFf4BF7P/kHSnyU1piiLa45GGYn?= =?us-ascii?Q?1lHHNDa2qjF/NEugmjZ3Yt7igWSUazqn7oD3IFze6IsOY+u5gjE22f4UQeMn?= =?us-ascii?Q?RDQ7s1OaGId8nTaJYAz2nbspv4FZFy8U9+6Ku+utI+LWp5KSIwCPk3U4nyQ+?= =?us-ascii?Q?tXkT6d3hG4pLL7boixPGzE7yRj+4aAwdCjPpoaWpfwt1f/Jznv6v8KDp47N0?= =?us-ascii?Q?8lDZyBgOynEutU0iufm0NKhUByriF40OGpCAtIQVbOkpsseINT7DYnKB/wed?= =?us-ascii?Q?ASLsbLhqhypholQ8BxpaB7cHFWymubXDFWizLFZBJblHsj9XaNS+0PWZLY3B?= =?us-ascii?Q?kLoV/Hppbay5U5LoSunAyPYX3tOu+MPoQuHetktVDsbTi6u93c7z6CU01STi?= =?us-ascii?Q?190n/9+8uTYeAe/WM6Z9fBA5Gg9+84zsmApLkXqgguZ2YYtrdXK0nhHj4x4P?= =?us-ascii?Q?gRX8ZkyRyy5Oh9h+Twcy0OQKutkP6QnhBnaxHFzgtMH0OVRAcoD/1JsH1SpB?= =?us-ascii?Q?1MMayQLtnxB+j/UT/32UKAiZxmvEzUMUFpOjYWZ+QkdJWA0xmTu1VofNqoWC?= =?us-ascii?Q?StiSva4jk5reu9ZiWBJQV48hVg1lzxnDRPVg0EXq5Pc+wiUxx5RTIUC0q/Fi?= =?us-ascii?Q?oSGELkW3qYOZ828pWSwTuBB6etQa8mxcLNsUA3v1OYwbq0PgjTxwFuyNo5ro?= =?us-ascii?Q?oZ2SEExvLyzawxlGRllHhJKZ?= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BYAPR10MB3366.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(366016)(1800799024)(376014)(10070799003);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?Zdr+XzLVmwsicy+us10ugEYhiO8in6ZhBsuapSYtVC4/6QEKzFCbBG9maceT?= =?us-ascii?Q?4qfs+oHXHAgiISlqUJS6KRpbCA40tAbAqqJ4dHiDPzV0/BaAKn+mLRmi6XC4?= =?us-ascii?Q?y7YIqPdX15AvI1bBGc+hqBio5ZgBV7+naYrvFzsWt7uoWUvRLOA9VpV7xhxa?= =?us-ascii?Q?q7e1qjpGcC0U2bdHzvfR7o/QpJtypUDhFTBU27ZpWHTUkSK+BvNZtUmsvQ01?= =?us-ascii?Q?gF3rFyh7WhHHLDWCk2Vlp8HTsHFYopJGYdcv8+Cm47b/0dn2Evy0oVQS8dtv?= =?us-ascii?Q?TE3QGTaid+OP2OQdduQ+HK5Ys9p8wmahXQ2H4zEa24gmrUqY8HtI57RcQBdW?= =?us-ascii?Q?WZr1kXgfNb6M0snEfINVoL/SuXATpbfWV6/FdY0oLda+ggkoMBWhBLs1/XBm?= =?us-ascii?Q?eNt8TgoJYneSVETOsi9svLsnQU4gBKCBKMQ5Qq1ZG9EdrNMzGvpA9aA0/72d?= =?us-ascii?Q?3jP59AfSw2w+SdAXjRMuc5qmtjvBIjhzH7j7HI8bP9Lqed8vG52k00AWxoQk?= =?us-ascii?Q?dNN8oJCbReKXVBiHB2MqBecvyr4/Y83GGd6ao3OA8VYovjjieyoEG7mSWQ+6?= =?us-ascii?Q?qnhgTg6BWcB0enSHE8kQClb9h8Y4BTiiWqI2izO9g9ZlT3YRJMItE6cKSqa0?= =?us-ascii?Q?J6du8c/bP3KoKJasAWWWMFlxnXWWFarYQ8CNBRBf1Q/3pEQ4GdTYEjWHJG6P?= =?us-ascii?Q?xVE7qR1k6WSsluNSvdgg5wmWhz1WUWjnyc+/z91VAYLYCAVbnYJmIlulPp+3?= =?us-ascii?Q?dSG7soJzLQoSQbkkgLwpXL2P5GMkzD8YiZf2Ql3jqVy3vopFaQ+mfK59ACwZ?= =?us-ascii?Q?kn4yCGJVmihKmwky65+Tig34DIJjsYKJ2+Sia+tkV2gyO3Qoeiv54+yhE16n?= =?us-ascii?Q?jOCWk6qA6fuJGdH02r+iAayzivKJD3kCdrBclj2WRM3JrUUwxLItGLtHXlKi?= =?us-ascii?Q?mihRKSs5nTYdi4uNbxd9LRr9rND8tyq3OQ1nT6UyIhym3idSWw9vRQEpb+ul?= =?us-ascii?Q?HlfG+O8lc3y86hkgfOeGCPd6cwUsKNWDTMlGXC+UraAf2PrFOHBL6eRWzsIQ?= =?us-ascii?Q?EBcOL16iwQbJIMUl9oeh/2zbY1EOcW39bIyg6Xml+uetvGdqqGq/IW+vCZ6W?= =?us-ascii?Q?oJPynV1IF047JEUhTLCWkh/nsMele1binyVfzZrc7jCztcOAihDrQpODZ8+D?= =?us-ascii?Q?+apDxqFyTK6OH3GOe4rmHE8cmSZ3+rd8U6zYTyTHNoQPxqkIIhCfCkTkDf0O?= =?us-ascii?Q?/X1aSJsYiQzOjWicc3T+MW4J475nitlAsXvgiIHq2dkI9ufCPU76XRYkZYff?= =?us-ascii?Q?MQNS5KR0bSLaa8UP3ImgZY6nQ57TBmw4K6fhkg8LqD2ocvKcfhMJpGyFuiSU?= =?us-ascii?Q?+n5kGSRmeQS4PrIJuYL0RBHiJ/Bibtl+k+Y87XRdXkCljjdeDdBCH7xLkt/h?= =?us-ascii?Q?CAbe+ygKFvkq7giDDQuxbk9cqAruAoc+vCDN2bm0r/X0PX4DqeFy+tg82z93?= =?us-ascii?Q?8DI2GdnSZ9CeaCB86DnTLeXXMpenTJo7+IMvlCkZW8dshjJVgncKsI3UeyiM?= =?us-ascii?Q?m0FuJi4IU4+uER0go75bfqkOSCi20AssNR0nTcApnWbPY+/ZmE14P5Xw9OWJ?= =?us-ascii?Q?EYvMaQ2vZU0mDEQCNMSgDPUBZiNCrRGsOnA3sT6M1n57vPJcaFtfk5RTDv0S?= =?us-ascii?Q?pff4Uw=3D=3D?= X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: Ur3x4okU2WYLZJyUQ2s5URPO5LYOc5/m15FdUdQJkqpUSb4zMYH2ZkgiuZbQrghSy6qw+cDUAzv4OBu9NIX1kY6tCLl8YR+EwQZgRA3002rDxymZzlyYkKJGy0i5n76bap4fC2Lw3XSyDpbY8toXt89p9LjoaC4hg6UeeRt2yQy1AjuGTEAhBj+lBMicbdyXU3D3sFnuUhOIdFNsyxOSf1rp0MsgtcO7mgJwKAvS1z4pwRbh2r/S3ZAcxv+T9MMj+AwWedns0obFi37jUgblElq+Sb18cvriz35k24UFHPUBD6fqj3rDgzEyAcYBm9TQkGdYh+AFdxQZ9jqZ5n4tIG6RLL50hHk3lHhuKBphYsNKUz99SGTpnU69/8OhArb17upMFSipedV6Tn5umwfe8hg2/6OQLM19yFU4BYVm6pRHbCiKAnAGaVfi3o4ihLOA/h/A4rujEvpdqoa+dqAmPyQmNPW5/P5T+5cZ1ra2TTnSbYLRj08bRgayq75aVfjrvRQ+/D2nTm5fmhLWJbx4V5UbQUEkDXcYZM5IWPq2bnlH3SRY0QUXPXP8fSoapX06PVrI2BTNi10NjxuXDvTin/R8OHm0Rc/1NfMSZb6RGyg= X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: b00151d5-ffde-4ec8-a869-08dcf3a2b989 X-MS-Exchange-CrossTenant-AuthSource: BYAPR10MB3366.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 23 Oct 2024 20:39:00.3846 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: xF6uWvu602bfRFdFB2BKx9MRQ/tLrwtBFrcx+TWAECQqV7oYMd3ivopAVSPiBSRYFiA3dmfQjp+BSnowM5Q2MbPF7/Ol3kBD3ii2InxYBRQ= X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ0PR10MB5833 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1051,Hydra:6.0.680,FMLib:17.12.62.30 definitions=2024-10-23_16,2024-10-23_01,2024-09-30_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 malwarescore=0 spamscore=0 adultscore=0 suspectscore=0 mlxscore=0 bulkscore=0 mlxlogscore=927 phishscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2409260000 definitions=main-2410230132 X-Proofpoint-GUID: zAz0mEDJK1N6tTkXJSG4x5pi-wPazSyK X-Proofpoint-ORIG-GUID: zAz0mEDJK1N6tTkXJSG4x5pi-wPazSyK Content-Type: text/plain; charset="utf-8" Refactor the map_deny_write_exec() to not unnecessarily require a VMA parameter but rather to accept VMA flags parameters, which allows us to use this function early in mmap_region() in a subsequent commit. While we're here, we refactor the function to be more readable and add some additional documentation. Reported-by: Jann Horn Fixes: deb0f6562884 ("mm/mmap: undo ->mmap() when arch_validate_flags() fai= ls") Cc: stable Reviewed-by: Liam R. Howlett Reviewed-by: Vlastimil Babka Reviewed-by: Jann Horn Signed-off-by: Lorenzo Stoakes --- include/linux/mman.h | 21 ++++++++++++++++++--- mm/mmap.c | 2 +- mm/mprotect.c | 2 +- mm/vma.h | 2 +- 4 files changed, 21 insertions(+), 6 deletions(-) diff --git a/include/linux/mman.h b/include/linux/mman.h index bcb201ab7a41..8ddca62d6460 100644 --- a/include/linux/mman.h +++ b/include/linux/mman.h @@ -188,16 +188,31 @@ static inline bool arch_memory_deny_write_exec_suppor= ted(void) * * d) mmap(PROT_READ | PROT_EXEC) * mmap(PROT_READ | PROT_EXEC | PROT_BTI) + * + * This is only applicable if the user has set the Memory-Deny-Write-Execu= te + * (MDWE) protection mask for the current process. + * + * @old specifies the VMA flags the VMA originally possessed, and @new the= ones + * we propose to set. + * + * Return: false if proposed change is OK, true if not ok and should be de= nied. */ -static inline bool map_deny_write_exec(struct vm_area_struct *vma, unsign= ed long vm_flags) +static inline bool map_deny_write_exec(unsigned long old, unsigned long ne= w) { + /* If MDWE is disabled, we have nothing to deny. */ if (!test_bit(MMF_HAS_MDWE, ¤t->mm->flags)) return false; - if ((vm_flags & VM_EXEC) && (vm_flags & VM_WRITE)) + /* If the new VMA is not executable, we have nothing to deny. */ + if (!(new & VM_EXEC)) + return false; + + /* Under MDWE we do not accept newly writably executable VMAs... */ + if (new & VM_WRITE) return true; - if (!(vma->vm_flags & VM_EXEC) && (vm_flags & VM_EXEC)) + /* ...nor previously non-executable VMAs becoming executable. */ + if (!(old & VM_EXEC)) return true; return false; diff --git a/mm/mmap.c b/mm/mmap.c index d55c58e99a54..66edf0ebba94 100644 --- a/mm/mmap.c +++ b/mm/mmap.c @@ -1508,7 +1508,7 @@ unsigned long mmap_region(struct file *file, unsigned= long addr, vma_set_anonymous(vma); } - if (map_deny_write_exec(vma, vma->vm_flags)) { + if (map_deny_write_exec(vma->vm_flags, vma->vm_flags)) { error =3D -EACCES; goto close_and_free_vma; } diff --git a/mm/mprotect.c b/mm/mprotect.c index 0c5d6d06107d..6f450af3252e 100644 --- a/mm/mprotect.c +++ b/mm/mprotect.c @@ -810,7 +810,7 @@ static int do_mprotect_pkey(unsigned long start, size_t= len, break; } - if (map_deny_write_exec(vma, newflags)) { + if (map_deny_write_exec(vma->vm_flags, newflags)) { error =3D -EACCES; break; } diff --git a/mm/vma.h b/mm/vma.h index 75558b5e9c8c..d58068c0ff2e 100644 --- a/mm/vma.h +++ b/mm/vma.h @@ -42,7 +42,7 @@ struct vma_munmap_struct { int vma_count; /* Number of vmas that will be removed */ bool unlock; /* Unlock after the munmap */ bool clear_ptes; /* If there are outstanding PTE to be cle= ared */ - /* 1 byte hole */ + /* 2 byte hole */ unsigned long nr_pages; /* Number of pages being removed */ unsigned long locked_vm; /* Number of locked pages */ unsigned long nr_accounted; /* Number of VM_ACCOUNT pages */ -- 2.47.0 From nobody Mon Nov 25 22:30:55 2024 Received: from mx0b-00069f02.pphosted.com (mx0b-00069f02.pphosted.com [205.220.177.32]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C98E61E5735 for ; Wed, 23 Oct 2024 20:39:17 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=205.220.177.32 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1729715959; cv=fail; b=RyVjPR7LwPyBnuPWj/VZ0VLJhHkH8sS8l8wHw658IBZjpFW7WBvhFh5942lWh/ey90DbahF5+c8QxXSdYU2K5EFD2FcLJ+ou1sHba3QdsIk33kt/38N4yj9NL0hoUZR/KNjHQ0j53Mpkz1zxdXBEzWxql8GTWyX3dZmu2yRbEGY= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1729715959; c=relaxed/simple; bh=JOJbhOxD4VQ8W1+wXrAHjuEzE8gcLZBteHOJ6l88gNs=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: Content-Type:MIME-Version; b=XFP64WyBehxDN1EXzR1lYScEeqil2Vh0u6X1WCMuXj8emKl9nKSgbrCczUC8YeZYzkJWVxAgaadpovreDDkKtXJ9P8cu4DOq+wENe/XSWBX4J90sWzRLBQf5yaNEXKFICpPnHBnC1W/wJgnZSIYWwv/3zzoHkrNAClIEP7+5CUg= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=oracle.com; spf=pass smtp.mailfrom=oracle.com; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b=jAeG5v8X; dkim=pass (1024-bit key) header.d=oracle.onmicrosoft.com header.i=@oracle.onmicrosoft.com header.b=eePXmu1B; arc=fail smtp.client-ip=205.220.177.32 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=oracle.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=oracle.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b="jAeG5v8X"; dkim=pass (1024-bit key) header.d=oracle.onmicrosoft.com header.i=@oracle.onmicrosoft.com header.b="eePXmu1B" Received: from pps.filterd (m0246630.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 49NKcXg3010429; Wed, 23 Oct 2024 20:39:09 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=cc :content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s= corp-2023-11-20; bh=TGZw0tx4qXQwwlFjxRJyFwIfVly//uJpCzOdUNUOgFU=; b= jAeG5v8XZLqcJ45zSJSyjzuBEPD/zPWzaPdMFHokfimcdg2Cli0eksS7I1c/xQv5 zz6NAKc3+tIEWaZyd28lH8sszfJHwAxbMNw9bj7VO5LfZZ1iPzNyuCgbci4TG57x NpWENmG2G7ibjaRTbFErF101o6Yh2P01ZPy29oEkzJjv8gUkY1k/4hBnokE1cTzh +htXd0fKNjF1acn8JGkEEaY/tKLa+yeuGWQ36+PNbVGYnfX38qg18bAzUmVH4ex/ VM4+VRlj81evpJZMIIy6fIFPRoLYxT5BsG5dZciqUK95X6me9rWqrv0BWi71K7hA 3t77Z3lMn5hD/HT5tIHVbw== Received: from phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta01.appoci.oracle.com [138.1.114.2]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 42cqv3g8w4-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 23 Oct 2024 20:39:08 +0000 (GMT) Received: from pps.filterd (phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (8.18.1.2/8.18.1.2) with ESMTP id 49NKIAZm031093; Wed, 23 Oct 2024 20:39:07 GMT Received: from nam10-bn7-obe.outbound.protection.outlook.com (mail-bn7nam10lp2049.outbound.protection.outlook.com [104.47.70.49]) by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTPS id 42emh219dg-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 23 Oct 2024 20:39:07 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=M09eoLqT9dIRokBOCZ8VH8KjBuyNg++ZuzTbXkkCixxzcoEKNhCKFo1kUBF856WaFNdmP+CnKkBMQKKkWhKKKNwQOyGspdqCryE0C7X5V+3Jf77Nk3VYdhQq9pLaymyAG785MIkVL8FYddPq5JgY5qKQBkoMcgx7mr2xE4no7pGBNTTiN6X8aEzMbVfbTqyCBbNQvNcry/sUJciPrl5XLpUI7W1yuMMgLBSvSfQ4zRwPaWwjvbr9DIay1kCOmXLoyw4rSXaccWAyPNDY5sDtJOrnL/IqG10w6TPR8X6vxAA7eDDUycAs6unjo12cjBV6QdjX3LZfI32GrvPfKejKxg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=TGZw0tx4qXQwwlFjxRJyFwIfVly//uJpCzOdUNUOgFU=; b=xtdBNCZHiE4K1za8MuGHwyxtVUqQkkByV0iCZGS+4VasDJK/YkIwI/IGx8Qq7RVIaYJ1U1izGVyicYhKtxv866L0+Hgd+GYlM72h921CHstCEnzU4C6j1x/sXoZL5D2LlIcAApBd/B17FYApT4fpiB9Uza2LcJethXt42c8Vmc/Fs0ZWeBTZewSpQ8fu0x/xElpWTRd2abQ6b80PGNs5NShHArZd75I3UM2E1CUTHp2NZJnTvteFwEoYrHs6WMRGGO6kSN130Ja+sWF0Mt8xfVS2e0rpvBdcyfzlJXquD3IiIiprEoSwBhVqgGNKWPnRXSJw9hUZy5PW4Q31YgNCxw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=TGZw0tx4qXQwwlFjxRJyFwIfVly//uJpCzOdUNUOgFU=; b=eePXmu1BUWNfaqKR/QeDV67sr+b8MgN7zjRwU+TOvEDcTXF4b6EUZhqcwF+XSW3Yz3xdP7eMk4C98ZMvyUeorCQvTjdOWXkjW64EvTaDsjGs4lnmzZDc/epOg5vRrkEv3M4UF83IRCGMYKzjY4OwhLnxkkg1mjCaAiYIjROuWFw= Received: from BYAPR10MB3366.namprd10.prod.outlook.com (2603:10b6:a03:14f::25) by SJ0PR10MB5833.namprd10.prod.outlook.com (2603:10b6:a03:3ed::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8093.17; Wed, 23 Oct 2024 20:39:04 +0000 Received: from BYAPR10MB3366.namprd10.prod.outlook.com ([fe80::baf2:dff1:d471:1c9]) by BYAPR10MB3366.namprd10.prod.outlook.com ([fe80::baf2:dff1:d471:1c9%6]) with mapi id 15.20.8069.024; Wed, 23 Oct 2024 20:39:04 +0000 From: Lorenzo Stoakes To: Andrew Morton Cc: "Liam R . Howlett" , Vlastimil Babka , Jann Horn , linux-kernel@vger.kernel.org, linux-mm@kvack.org, Linus Torvalds , Peter Xu Subject: [PATCH hotfix 6.12 v2 4/8] mm: resolve faulty mmap_region() error path behaviour Date: Wed, 23 Oct 2024 21:38:29 +0100 Message-ID: <6e8deda970b982e1e8ffd876e3cef342c292fbb5.1729715266.git.lorenzo.stoakes@oracle.com> X-Mailer: git-send-email 2.47.0 In-Reply-To: References: Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: LO2P265CA0264.GBRP265.PROD.OUTLOOK.COM (2603:10a6:600:8a::36) To BYAPR10MB3366.namprd10.prod.outlook.com (2603:10b6:a03:14f::25) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BYAPR10MB3366:EE_|SJ0PR10MB5833:EE_ X-MS-Office365-Filtering-Correlation-Id: 6c0c3067-c6e4-4580-1dca-08dcf3a2bbcd X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|366016|1800799024|376014|10070799003; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?ezwjPU10Hy8v9RBw6meAkObqWSYxnJqvMbDdLpNfouFhlGfV1wYr0Fx/ONJT?= =?us-ascii?Q?L5GvGVfEwmwB0rmI5yA1vNBQz/Fsde1o0W/2IgOhU270478pxlMqV0924tiw?= =?us-ascii?Q?aqygyTf2BGVb6Dvuk2cPrprYiuB2YuLB1MhVkfNjoQAQlToWZfVgKXSlXwwD?= =?us-ascii?Q?Lor2kpmqCWHfLVSyzvUbDuSAxsxR4K3Q9MjLWxgW6T43e9P7PBBbeywEn994?= =?us-ascii?Q?fQ9yKrL6HtnYIBg4Qchua3bR+OgaNA8akh7qPvVUar6s1yCEaGGRQDE+bwhz?= =?us-ascii?Q?tmw3EyDbCuXdJJwnpmAOeBSaWOcxTIMontab2S3omGKv2X0wc5wtpwDQUG3N?= =?us-ascii?Q?03vxfGb3FecQR5FD4mkfNxA+p5q2rWLHhf9Szluu57VJf2WE2UpGkbjyjKh7?= =?us-ascii?Q?wKtZQ0I4rajEqNnB1a/i5mcl/f46qLeXdn0ucwziPhuZawqiNYMNLMwY6Wn2?= =?us-ascii?Q?JU5nGtTQorzoEexPuTrs4Wsm/hEC9IjpJppvjrKUtCCSpQWsYYEtIEfAttdR?= =?us-ascii?Q?NJYetfEfWZBeB49T5vsHvNP/bub5u5bKJ/9ourlBKg7xI6y7+SPcA+Krt66p?= =?us-ascii?Q?CnrYcPwnh94ZFoEe7KZkUH9rYjUs+9xs/sT/0l/OoO8X8UHN49AwlNuZT7KD?= =?us-ascii?Q?RLz+XKfM6yo1BcsGUjDCCb7TQDK8b0mftINlFUR9Lo/CfKzXIYS9SYhGwow7?= =?us-ascii?Q?XBm9ajVV6PiEuZdIbww9ZdFr9kfg67N3vkhqyFDdriXY9KVwqbMfso/hYWzm?= =?us-ascii?Q?gvtkv4i04LKlKI4M5iGAZZvk0oU/CQS1F5LNQckCe13yZo6VpiAPgJPdIuEd?= =?us-ascii?Q?lK2pQShTevEjcppMExok2yT7W0taVR9UE8uLrZcFKYiVO7N6C8o2SrWtWbSR?= =?us-ascii?Q?l9ESndSXVChCqxaQFpbNTi2Rvr9J5SG8wJ/2ZKtJx45S1EQ26vVzqr31SJ9K?= =?us-ascii?Q?z/XVkm1z6cYS+CRDk/B2k9b2HrCpDM6y2qVhvJ70N00y29zyvr9C2lNwICAE?= =?us-ascii?Q?665HuepvGZY5DYHhTp/zHl1mtvhLOlR1L+nzz3V8a1JeOk58Qi2+ddL/Hgdp?= =?us-ascii?Q?nplk8z2qYKkwDLa6bCm6Zt4z6As3YVNOkWU4rTlpZh5g25Ua1NXyq1zFx+kU?= =?us-ascii?Q?IZ+KD5MVtpOATQq2jsXxoZ/dLyB3DE6r2i6UBc9rmk3C4ekS2HEFtLZ+l/ie?= =?us-ascii?Q?p7FwDwJRveOwBDCVgP148Ap2g5hrXHQ4iGpVnsbegYC5d/5o0iohhFz66DQg?= =?us-ascii?Q?z3SVsuM2J+snOy2Uc8e5UPPJ82an8YHm36GRtXCa4085ZqZJlhnTaS2CfbJm?= =?us-ascii?Q?B8MkjKcHC5afEuj/yM3gkTbW?= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BYAPR10MB3366.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(366016)(1800799024)(376014)(10070799003);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?hmCLnWVgMs20n9QGLaLm+loQlmuWwQt8YP8ZbFYsGzXA5weWX9GNI72jSNVc?= =?us-ascii?Q?dxVNfgvFz1ejAHdMYulAVtHmSUTc+rWEWyk1lW2gy8Xcwefk3f/ulK5WHot6?= =?us-ascii?Q?ImM3qE06J7azJZ+Xf/ovgHeMwndnzhfnnv9sFN4oxa7+ltSLq8+poRZC6xww?= =?us-ascii?Q?xUgT3xcbCQ7ZqKYY+5rFxI/xn/+gCoHiCvlcAViP/F2C5xTviRL+QftxczEH?= =?us-ascii?Q?62m+fCsAFJGrkFuuw0y3YffNpJKVKWQvG87sK5i0k4eDaZh3syWLlRCj16s6?= =?us-ascii?Q?emVG96wxh+8Aq3Ua3S/eiy6IEJiqFZttT5KKJSWH4kLCk7DLwA8uGZotUhws?= =?us-ascii?Q?6ugLWGifq8JPacb+qAoI992gSqjAtC0LSRQFznJNcds39U5zBTeBm7T1Qme2?= =?us-ascii?Q?yfbQAxhovuGA6ia36ZBLHUd6uXVKzwBOjdIlgGEP9umtCA5m6nE3nrfZVwHh?= =?us-ascii?Q?Hp6atPvNygg1s4QCj6LrXp/YBwYcnxdoedFDYpyInJ0lNXZCbBqDn7LkKGwf?= =?us-ascii?Q?ZQAeL0c8BhwcPE52mgnSsess0LH1FOyAmd7hiZtHq/DQiZDCnam0qKy8Dytj?= =?us-ascii?Q?zkrGPEx4gVphpcatsGNetaIS2Z5UrBOeiQZuca89ImK+3lOdhD/a7dpk0Kfw?= =?us-ascii?Q?20XiyH08NjditDfGAvAYAnIrQ6i+3rJoKgMZeDoy+JJMUMuTwYwjEr6QQ+zD?= =?us-ascii?Q?R9k/KmYI935r1HSl2vFiLTivQCwgyfx4s9J1ZRaykVpmTZYLOABPJgkWiObc?= =?us-ascii?Q?rb0LDGDhLYXduT24dNPKWeQb9j2lgojMENNFC/DpzuJfQ1xzgsmuPi6RoSvB?= =?us-ascii?Q?YEgH6+t6JV9we8mqt++cFTlQAEBwC/q+DeeOHXqR2Mg32HfRxF3hHrLOGFm3?= =?us-ascii?Q?/aOW0xtqRVU/8o3ZEehb6eNCdC9uX41Bjx6QnJbmK1FzymzA6eKQf+5CzCy5?= =?us-ascii?Q?ZfiUjFSpOy+6+PMVEcS8A3ROJlm57U4lRmHAwgJf23AQcKp/eEbAT7mpAX35?= =?us-ascii?Q?NTPj2gXePW5vWhD51RSVug7fwIfkXCuNErGagNYLGN4gQaDUIpcUJj/sNnr2?= =?us-ascii?Q?fIYi2sSYy0ZHrG/mK9sTGeikv8GivP6Ko2v/ji0yfVUfdWp510Ubq37ST8JX?= =?us-ascii?Q?vkSl8X33w6gYHebDwBTSMMAyI0eGEsgkGStSjao8SY+WXlE6enr39uphImW/?= =?us-ascii?Q?R7ZAa033AQK9y2PDFfxqVHx8zbwmQZKgKROP5mQ06kM/qPd0iB8ntW5br2k5?= =?us-ascii?Q?UAjuzAmW2AH4ECxgAIChI5BFJQaubyZHH6KMXzst9P5SpnCULUKLKUojOp6A?= =?us-ascii?Q?WSseDMg1q84dXGz1VplcTp18UXigd8lAlBEDoX1BOfktCKF8pv4131KLMda5?= =?us-ascii?Q?ZcAIAiVeo543syuGzhCy619qiwAg8PeUENIKRZnXAyfnEyxrfn2HGsT/TCCf?= =?us-ascii?Q?J+z0ve+qGH4KnPVtnkSOge1KHdgzX24SVzAaVvz8nTWT5ilf+KWxMRbAFdzZ?= =?us-ascii?Q?Y7Cec4DLeQDNK6DpNhYqeVCdhSqGKj4AROKLeT2wkOnt/+r4dhRWzshCpC3T?= =?us-ascii?Q?+MQw8REFGZqRNPsNQT9xiHJRNJrcoF1gp+2sPjcfV9rQCi/xwIgz1w+7QL7G?= =?us-ascii?Q?1vtOhy6ioaPOgbA28JnrmaYJRmw5a6WC4+VA0uqj7kgKweKoBMEwU+Xh1ErI?= =?us-ascii?Q?59oiAQ=3D=3D?= X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: tEbNw632Zv6KGAYJHvI9GjavZn3Cb5f6XLBHw1s5zcEx/MW96BLH8n6JoLk+QFrmLhr8x5ugvQ4pzMU1GUyZlbrU1AsreSguZfQOFEnaqO3TCXgKw2CC+IVZqZdVT8nvjeBJ26g8+5b9Mu5ncqXpgQt1PewNRWGhujYhDCo8KMxzAcgHRmyvef4G5XftqCbNP0VwLUtYvV3analAENl4eaM41pXQyiX8ekuvoXKNg/H/f7pm4AH3puejseZnGIzMLaYjvaAXakg5uZ/YMAK2YybS9jG54TWQXnW64AmKY9YOlQZ62h16ggJggI1BVoA+MrwgvPi7/N4QrZoDi7AoEL8c63LQdvZIEK+gOCf+nGiPajse3lKVxhWKX6pWARwc67QpawGVJTrSPhlyBEB9ejiU5Rei+1PDUwMfs/LNbDVtz46Dv4NoRAkwuztebSNxn9He0IMZSBqAawcVFottF7nB5sHjYSa3XYDFQlrC+vALqQ9+2FxGRGnT25UYSOBGTsSlW1bnczczpeWx0YzCwbjFzlZl7S67xeaJJvbuRmVySsyO7KoI+GKK3RmNJEO+hJLBoszthfJXCF4EyXv3Am9jGaX+PU4H0WPyle05f5k= X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: 6c0c3067-c6e4-4580-1dca-08dcf3a2bbcd X-MS-Exchange-CrossTenant-AuthSource: BYAPR10MB3366.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 23 Oct 2024 20:39:04.1804 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: PB5FehlCXSNSyjAf6hA96DYkmPRwniofeSwsGqvT4t3mOjD/l2idYRyhSggNvZAfdaOHFMsYi+eKOzDnl8ZCBqr8X96QchsDfxUd7iBF25Q= X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ0PR10MB5833 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1051,Hydra:6.0.680,FMLib:17.12.62.30 definitions=2024-10-23_16,2024-10-23_01,2024-09-30_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 adultscore=0 mlxlogscore=999 phishscore=0 suspectscore=0 mlxscore=0 spamscore=0 bulkscore=0 malwarescore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2409260000 definitions=main-2410230132 X-Proofpoint-ORIG-GUID: cJ4ZA9dO-MjJLmDSTGZ47qRkGazPDiZv X-Proofpoint-GUID: cJ4ZA9dO-MjJLmDSTGZ47qRkGazPDiZv Content-Type: text/plain; charset="utf-8" The mmap_region() function is somewhat terrifying, with spaghetti-like control flow and numerous means by which issues can arise and incomplete state, memory leaks and other unpleasantness can occur. A large amount of the complexity arises from trying to handle errors late in the process of mapping a VMA, which forms the basis of recently observed issues with resource leaks and observable inconsistent state. Taking advantage of previous patches in this series we move a number of checks earlier in the code, simplifying things by moving the core of the logic into a static internal function __mmap_region(). Doing this allows us to perform a number of checks up front before we do any real work, and allows us to unwind the writable unmap check unconditionally as required and to perform a CONFIG_DEBUG_VM_MAPLE_TREE validation unconditionally also. We move a number of things here: 1. We preallocate memory for the iterator before we call the file-backed memory hook, allowing us to exit early and avoid having to perform complicated and error-prone close/free logic. We carefully free iterator state on both success and error paths. 2. The enclosing mmap_region() function handles the mapping_map_writable() logic early. Previously the logic had the mapping_map_writable() at the point of mapping a newly allocated file-backed VMA, and a matching mapping_unmap_writable() on success and error paths. We now do this unconditionally if this is a file-backed, shared writable mapping. If a driver changes the flags to eliminate VM_MAYWRITE, however doing so does not invalidate the seal check we just performed, and we in any case always decrement the counter in the wrapper. We perform a debug assert to ensure a driver does not attempt to do the opposite. 3. We also move arch_validate_flags() up into the mmap_region() function. This is only relevant on arm64 and sparc64, and the check is only meaningful for SPARC with ADI enabled. We explicitly add a warning for this arch if a driver invalidates this check, though the code ought eventually to be fixed to eliminate the need for this. With all of these measures in place, we no longer need to explicitly close the VMA on error paths, as we place all checks which might fail prior to a call to any driver mmap hook. This eliminates an entire class of errors, makes the code easier to reason about and more robust. Reported-by: Jann Horn Fixes: deb0f6562884 ("mm/mmap: undo ->mmap() when arch_validate_flags() fai= ls") Cc: stable Reviewed-by: Liam R. Howlett Reviewed-by: Vlastimil Babka Signed-off-by: Lorenzo Stoakes Acked-by: Vlastimil Babka Tested-by: Mark Brown --- mm/mmap.c | 119 +++++++++++++++++++++++++++++------------------------- 1 file changed, 65 insertions(+), 54 deletions(-) diff --git a/mm/mmap.c b/mm/mmap.c index 66edf0ebba94..e686d57ed9f7 100644 --- a/mm/mmap.c +++ b/mm/mmap.c @@ -1361,20 +1361,18 @@ int do_munmap(struct mm_struct *mm, unsigned long s= tart, size_t len, return do_vmi_munmap(&vmi, mm, start, len, uf, false); } -unsigned long mmap_region(struct file *file, unsigned long addr, +static unsigned long __mmap_region(struct file *file, unsigned long addr, unsigned long len, vm_flags_t vm_flags, unsigned long pgoff, struct list_head *uf) { struct mm_struct *mm =3D current->mm; struct vm_area_struct *vma =3D NULL; pgoff_t pglen =3D PHYS_PFN(len); - struct vm_area_struct *merge; unsigned long charged =3D 0; struct vma_munmap_struct vms; struct ma_state mas_detach; struct maple_tree mt_detach; unsigned long end =3D addr + len; - bool writable_file_mapping =3D false; int error; VMA_ITERATOR(vmi, mm, addr); VMG_STATE(vmg, mm, &vmi, addr, end, vm_flags, pgoff); @@ -1448,28 +1446,26 @@ unsigned long mmap_region(struct file *file, unsign= ed long addr, vm_flags_init(vma, vm_flags); vma->vm_page_prot =3D vm_get_page_prot(vm_flags); + if (vma_iter_prealloc(&vmi, vma)) { + error =3D -ENOMEM; + goto free_vma; + } + if (file) { vma->vm_file =3D get_file(file); error =3D mmap_file(file, vma); if (error) - goto unmap_and_free_vma; - - if (vma_is_shared_maywrite(vma)) { - error =3D mapping_map_writable(file->f_mapping); - if (error) - goto close_and_free_vma; - - writable_file_mapping =3D true; - } + goto unmap_and_free_file_vma; + /* Drivers cannot alter the address of the VMA. */ + WARN_ON_ONCE(addr !=3D vma->vm_start); /* - * Expansion is handled above, merging is handled below. - * Drivers should not alter the address of the VMA. + * Drivers should not permit writability when previously it was + * disallowed. */ - if (WARN_ON((addr !=3D vma->vm_start))) { - error =3D -EINVAL; - goto close_and_free_vma; - } + VM_WARN_ON_ONCE(vm_flags !=3D vma->vm_flags && + !(vm_flags & VM_MAYWRITE) && + (vma->vm_flags & VM_MAYWRITE)); vma_iter_config(&vmi, addr, end); /* @@ -1477,6 +1473,8 @@ unsigned long mmap_region(struct file *file, unsigned= long addr, * vma again as we may succeed this time. */ if (unlikely(vm_flags !=3D vma->vm_flags && vmg.prev)) { + struct vm_area_struct *merge; + vmg.flags =3D vma->vm_flags; /* If this fails, state is reset ready for a reattempt. */ merge =3D vma_merge_new_range(&vmg); @@ -1494,7 +1492,7 @@ unsigned long mmap_region(struct file *file, unsigned= long addr, vma =3D merge; /* Update vm_flags to pick up the change. */ vm_flags =3D vma->vm_flags; - goto unmap_writable; + goto file_expanded; } vma_iter_config(&vmi, addr, end); } @@ -1503,26 +1501,15 @@ unsigned long mmap_region(struct file *file, unsign= ed long addr, } else if (vm_flags & VM_SHARED) { error =3D shmem_zero_setup(vma); if (error) - goto free_vma; + goto free_iter_vma; } else { vma_set_anonymous(vma); } - if (map_deny_write_exec(vma->vm_flags, vma->vm_flags)) { - error =3D -EACCES; - goto close_and_free_vma; - } - - /* Allow architectures to sanity-check the vm_flags */ - if (!arch_validate_flags(vma->vm_flags)) { - error =3D -EINVAL; - goto close_and_free_vma; - } - - if (vma_iter_prealloc(&vmi, vma)) { - error =3D -ENOMEM; - goto close_and_free_vma; - } +#ifdef CONFIG_SPARC64 + /* TODO: Fix SPARC ADI! */ + WARN_ON_ONCE(!arch_validate_flags(vm_flags)); +#endif /* Lock the VMA since it is modified after insertion into VMA tree */ vma_start_write(vma); @@ -1536,10 +1523,7 @@ unsigned long mmap_region(struct file *file, unsigne= d long addr, */ khugepaged_enter_vma(vma, vma->vm_flags); - /* Once vma denies write, undo our temporary denial count */ -unmap_writable: - if (writable_file_mapping) - mapping_unmap_writable(file->f_mapping); +file_expanded: file =3D vma->vm_file; ksm_add_vma(vma); expanded: @@ -1572,23 +1556,17 @@ unsigned long mmap_region(struct file *file, unsign= ed long addr, vma_set_page_prot(vma); - validate_mm(mm); return addr; -close_and_free_vma: - vma_close(vma); - - if (file || vma->vm_file) { -unmap_and_free_vma: - fput(vma->vm_file); - vma->vm_file =3D NULL; +unmap_and_free_file_vma: + fput(vma->vm_file); + vma->vm_file =3D NULL; - vma_iter_set(&vmi, vma->vm_end); - /* Undo any partial mapping done by a device driver. */ - unmap_region(&vmi.mas, vma, vmg.prev, vmg.next); - } - if (writable_file_mapping) - mapping_unmap_writable(file->f_mapping); + vma_iter_set(&vmi, vma->vm_end); + /* Undo any partial mapping done by a device driver. */ + unmap_region(&vmi.mas, vma, vmg.prev, vmg.next); +free_iter_vma: + vma_iter_free(&vmi); free_vma: vm_area_free(vma); unacct_error: @@ -1598,10 +1576,43 @@ unsigned long mmap_region(struct file *file, unsign= ed long addr, abort_munmap: vms_abort_munmap_vmas(&vms, &mas_detach); gather_failed: - validate_mm(mm); return error; } +unsigned long mmap_region(struct file *file, unsigned long addr, + unsigned long len, vm_flags_t vm_flags, unsigned long pgoff, + struct list_head *uf) +{ + unsigned long ret; + bool writable_file_mapping =3D false; + + /* Check to see if MDWE is applicable. */ + if (map_deny_write_exec(vm_flags, vm_flags)) + return -EACCES; + + /* Allow architectures to sanity-check the vm_flags. */ + if (!arch_validate_flags(vm_flags)) + return -EINVAL; + + /* Map writable and ensure this isn't a sealed memfd. */ + if (file && is_shared_maywrite(vm_flags)) { + int error =3D mapping_map_writable(file->f_mapping); + + if (error) + return error; + writable_file_mapping =3D true; + } + + ret =3D __mmap_region(file, addr, len, vm_flags, pgoff, uf); + + /* Clear our write mapping regardless of error. */ + if (writable_file_mapping) + mapping_unmap_writable(file->f_mapping); + + validate_mm(current->mm); + return ret; +} + static int __vm_munmap(unsigned long start, size_t len, bool unlock) { int ret; -- 2.47.0 From nobody Mon Nov 25 22:30:55 2024 Received: from mx0b-00069f02.pphosted.com (mx0b-00069f02.pphosted.com [205.220.177.32]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 096DD1F9432 for ; Wed, 23 Oct 2024 20:39:20 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=205.220.177.32 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1729715962; cv=fail; b=qOP99vK7qTvwGvcrrJc0HBLM7Ufzw1j++sJspdwXtqX66vdWostscu5uWyC0lnNSgcrgfI+nTvGB7d4NWP/E7zPj8yCVdAPbZ8//lYA6v/6QqcXA4f5J88z95Sjz/NZqttTYH2sUh9jKa+x7W7x2yp8hyiljnQnRDT5ZLRdTcnA= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1729715962; c=relaxed/simple; bh=AxS50HEFoaGKIBrw8l/jhmM+v0+UIkXrTa1Olihoogc=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: Content-Type:MIME-Version; b=fiKjTDdkfWB2Z85D3mjEMx2qc2ldZVfnhjH9sydTAUdOqbTh9OC9U0Dn6wub4/GYvaE/Zxkqk5PboGQPWL8wjg2aAXB3C97jpK6dRuMHAG94JGjk5rhEH/B37oDSXFncJ2GkmWBoSAn+OUWddN8cRA2edOUwLICgVCJyOMyHQBw= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=oracle.com; spf=pass smtp.mailfrom=oracle.com; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b=hAFSTcir; dkim=pass (1024-bit key) header.d=oracle.onmicrosoft.com header.i=@oracle.onmicrosoft.com header.b=0DZ0wkn6; arc=fail smtp.client-ip=205.220.177.32 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=oracle.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=oracle.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b="hAFSTcir"; dkim=pass (1024-bit key) header.d=oracle.onmicrosoft.com header.i=@oracle.onmicrosoft.com header.b="0DZ0wkn6" Received: from pps.filterd (m0246630.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 49NKcTV1010295; Wed, 23 Oct 2024 20:39:10 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=cc :content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s= corp-2023-11-20; bh=WP57i7v+tW2ioFRTEIKKj1mu6dNHF0V1UcxvMSTIrXU=; b= hAFSTcir4XzhAtoaNGmWtT4iaKFWY5VrW4VhiHZ84g/5KhGDqOQUxBg1EK4Y/UeD 551e/XoDioC6vlHfoQtGhVE41c1srMLwIdowU2Szhe0/f5YX3aNKScdQ3SHXJruN /ey33TbtHz2fEdg52vNUd2b3BuW5O/3H+ErH+kUd/+APP38ccEh9zIZIE0kps2tI aXKYkLA13dDGaqE0GSu0YLSMLSgsmP4OV0cTyFpYxY2aBz/kJ9K/GI5y1u2MEmT2 SJ1YE9LF0pJixfsYDuBZcm8XPH9mBUQXi90P+szGDuYti6hXNe9OZ0BD+CNK2rXY erSDFhHKS1QIHmrf7uFlhA== Received: from iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta01.appoci.oracle.com [130.35.100.223]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 42cqv3g8w5-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 23 Oct 2024 20:39:10 +0000 (GMT) Received: from pps.filterd (iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com (8.18.1.2/8.18.1.2) with ESMTP id 49NJEEmp027438; Wed, 23 Oct 2024 20:39:09 GMT Received: from nam10-dm6-obe.outbound.protection.outlook.com (mail-dm6nam10lp2045.outbound.protection.outlook.com [104.47.58.45]) by iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTPS id 42emh3bk7q-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 23 Oct 2024 20:39:09 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=naBVmz/G/DX+Vz902PvbV1U+beSEKgfmGYxZs3mESRePJsVIBAV1nopw3Ncwc+NNav6I0H5CkZcu8FnUw3EiftMKkv4Wm1RTHtXDeSFFYAl/Eu4e4Sp/SqrPIQW6jQUEs7/XrGVBSMkPtOOWII5rlonS791Rp5TIGoU7Iv0X30czkCFZcv6bhT9TK7ZMLqJ1RVD/365Q1Mv5rgET3COR6EFA+arXEkbrOcvA2t294kQ8rk6ueEilgxjdYGsaJfBxjNTG6h96ZL+hFZcNPA7C3T5B8+U/xZmRfrFXk2yA1n5YrGBfKPcO1Wk16NnxHsj2a/BbCIjA+Azgj0hcMIa/Nw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=WP57i7v+tW2ioFRTEIKKj1mu6dNHF0V1UcxvMSTIrXU=; b=cZnHEw/ZjEdN/5c8o98g3+K0LIRzZf4Z1T8UnBHePNEs9jxWgNwU4qIvYYerjswG00D6HJgPBt/TM4ibdArJTTgCTxc9iNPPP1csOYuOtQ+ICm4IBE0cho4b490nZQ5R3tMImecQiYdr26UbMWJMZ0QKYs4FI2YAGYpkKk4AdMJoGwolTngZBojJQRnTnP49P9U9e2zF18JHMyi93UVcyVH0mnYFX0TC62X0doL/G4W40D7QxlAKuz+OglOlJhJw4Cblcw8Qdj/a1oJvAax9xwMqNDIWMhWrxahtPRfqn/u22O+j72wDWf8NR7tQmJofvSRRn+w/3+MNcM0a4/5WxQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=WP57i7v+tW2ioFRTEIKKj1mu6dNHF0V1UcxvMSTIrXU=; b=0DZ0wkn6jxXqp0MDI9dIyUXvpnC8ssyevmYVAkJJOU5tW5x7fXYFW54X9RmiL6TCYmo0Ej9BaYJtbjpQUSFHLBU9T1UJbCr3oyVHn5r0wQfGlQ3FNa1UK2Op/jPOhHDVHJQ9Go9zoLkQC/e68Uw3IbgBS98SnS1RSpnZCE/THJk= Received: from BYAPR10MB3366.namprd10.prod.outlook.com (2603:10b6:a03:14f::25) by SJ0PR10MB5833.namprd10.prod.outlook.com (2603:10b6:a03:3ed::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8093.17; Wed, 23 Oct 2024 20:39:07 +0000 Received: from BYAPR10MB3366.namprd10.prod.outlook.com ([fe80::baf2:dff1:d471:1c9]) by BYAPR10MB3366.namprd10.prod.outlook.com ([fe80::baf2:dff1:d471:1c9%6]) with mapi id 15.20.8069.024; Wed, 23 Oct 2024 20:39:07 +0000 From: Lorenzo Stoakes To: Andrew Morton Cc: "Liam R . Howlett" , Vlastimil Babka , Jann Horn , linux-kernel@vger.kernel.org, linux-mm@kvack.org, Linus Torvalds , Peter Xu Subject: [PATCH v2 5/8] tools: testing: add additional vma_internal.h stubs Date: Wed, 23 Oct 2024 21:38:30 +0100 Message-ID: <4eddd714a8d6dfb052032dfda5aa28d9f4bd134a.1729715266.git.lorenzo.stoakes@oracle.com> X-Mailer: git-send-email 2.47.0 In-Reply-To: References: Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: LO0P265CA0011.GBRP265.PROD.OUTLOOK.COM (2603:10a6:600:355::19) To BYAPR10MB3366.namprd10.prod.outlook.com (2603:10b6:a03:14f::25) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BYAPR10MB3366:EE_|SJ0PR10MB5833:EE_ X-MS-Office365-Filtering-Correlation-Id: e7539a0c-afad-49dd-a908-08dcf3a2bd80 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|366016|1800799024|376014|10070799003; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?j5OZBnSH9TEJb/SI+ZMzI4ApzF/7cYxZWsbkqOh5Ey3MxSUiRZCf7HT5vab/?= =?us-ascii?Q?VrdhpcG7eH3X5xTLZQxK06D2AIyqm60V/jLgmAdoiZHdQoFnrB/gtMWZR0tQ?= =?us-ascii?Q?6SF88DJNNqzY5ZEQ8LJLTpLLVF0LkFdJLcy+Mw6cjCwsQqnEP9L3MsxilVuV?= =?us-ascii?Q?uDHrUQSmOEDyC7d2kE8SVt11cH8sA8EG/VLTMCuXECiAQov1sRyEY5zcsK50?= =?us-ascii?Q?Jb+n69CPybDgXQspCDzu6uaj7W+T9ryV/T0PaB9thZe/TaQ3bItA12hrpcK4?= =?us-ascii?Q?xYYGoDZwTqnS25LRsU+xQ/yVr7INAdC94RW1ACtzZESxDSq4EP9mUwVzM+kH?= =?us-ascii?Q?pYrILrNn+6G6yX273g2z2LRKyElQDgVqV32yojQB9W+aWRgAv6cniiaM98up?= =?us-ascii?Q?Lo+bYtAjnl38ip4rXCYznwnRVLUJCgkOntComVXH2gLSTZSBE8VEVp7v7UUa?= =?us-ascii?Q?xVYCum0ho6zQS7z3RQgpJvGtzmHf7bkdLeh2FxyONFlZ3wICkKhoIZVeKjrH?= =?us-ascii?Q?WEHQIXaApwQ7TROezZxfMN78xJBIJ/47cSutVtgkGB9gIX4inIL1l+YMWvql?= =?us-ascii?Q?EIW0ns+4rzD1ba61lDgrub2/kUpiDnmo8yEMxx0DWFT4WLrDIyFj8td5GHp7?= =?us-ascii?Q?PMtJb1pteByVPzb5AxogVkc+OkDse1uMkfuD3I3MovbJVOaSUImc0PkcP6kM?= =?us-ascii?Q?QkXx/bgtsS6fyeol4bVaLUW3BZgmeR9CGl9vg2GsdLqZyFtKAGWZU6w55EA5?= =?us-ascii?Q?YdsMqU3CJagzIxSZCFWImaRrBSXwN1WgZplwVa7y3gxuVHwJRa7dFPrZviES?= =?us-ascii?Q?GKZjWDSQUbMh5DU4w01bjg6UMIXBxMNsSLVTiBQw6C1A4sPeOdKKJUNB6qDQ?= =?us-ascii?Q?onS44Od/aMvXbCfQaOLVORmAfPwPmYqJ1pCViwAq+E7CYyeo+dAmR+Iz5uj3?= =?us-ascii?Q?v3THbys0TCD6/A2X0snhgyfIq98asgBixMZliVyZBa1tpgKd2LW9JIP1JgmT?= =?us-ascii?Q?fpevb4/NFhbxtJEZ9MCsBrqpweIoOG5hk9AWKEuWFxb9YcoVGAsiMtNRU7bu?= =?us-ascii?Q?AwsITgYZzqLDetVsmk/x3HjUKlSt4a0dHS93aCrRnNPSwqgVNo6yZJqFSoEW?= =?us-ascii?Q?29/ARHIbEGdi0YJizkOZDGxspLLAAJOEXnvpRO+g8b4E2/Aw13NXTZ5vLkWj?= =?us-ascii?Q?5VJZtwEKr2Ax9AiT1T9Dx4OPRsT5rncsr8OhqagPl8zPUWrFTPRfquAAZGns?= =?us-ascii?Q?6u8Qd83P+zNg4txVPEvmzBTpyS8YhaHrfrKLDS6KpGofync/OiDJDCnhybgp?= =?us-ascii?Q?0Q4=3D?= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BYAPR10MB3366.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(366016)(1800799024)(376014)(10070799003);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?m9BI3ITDfDXmGd2Fk8MP08qk8v7pYbir3BFFH/BUMakbTwBv229isnJ3RgdI?= =?us-ascii?Q?GEcqZIM0Nf9BIocM1mlDt6gyGpcukkwGspuefGQD9reDeqCaz0Hk1s02NVYI?= =?us-ascii?Q?RfjEfmhQF7NE8Q+BemvSsjZZffeLfK6U7PlwiGeCaU0vCSfsDhArA81N2qlU?= =?us-ascii?Q?hHTEeOstbceild51mShInKrPD/FMr0RY5W9VGvol58ErpeZABzLfXg7R4F2S?= =?us-ascii?Q?OVV3nH2Lmqk4kufnUKjYj+IwAQi+RJcgoP6hrQ36inF4Bfxqgilu8QZkeH6p?= =?us-ascii?Q?bqA7rKUL1fgNHPiWTnpGIAI4gMrNk6KWShjywTDowC/+LQW+FHgke0h/t3E3?= =?us-ascii?Q?r9Eur26T+zFeZ3xnlLzB3l9JKrF/27Au+mGx5rrvolqxhS4mHiwou1mEl+kh?= =?us-ascii?Q?HlbCjy8XIiYh8IkKBHrw1dMbqJ1wKfz9t3I5o7s+sfj0MEro2KBzi8LUTwGO?= =?us-ascii?Q?Q3qQnqP3GBqYjzn/jjzJ6mjQAh5C63K1J7v6imG6PPWWdaP7CPnuaeB3EmjJ?= =?us-ascii?Q?PJpCln+bBLBtk/I7VS5ojFHTRIUaJmjuJoX+P8nJa9kZSSVMOiviRpkHFHzW?= =?us-ascii?Q?waoGi3i49LPXbBnmZxYbjE0mTjgoYplF1P9TOu0wd3EuMG7+BcrlRidxoqYJ?= =?us-ascii?Q?BKttjrctc6Vt6mBN4DpAHmHxjCTKkV1EEAjelFucrtjfFnIdUOjnSmluhPEi?= =?us-ascii?Q?k+rqqxfU8owhM68pFbTPkEr1pbC67oAdMQhTHuxrKe4R0q6Z09rgJnfv9kbC?= =?us-ascii?Q?Z2dYZLdDIBgO+b1RiujK/5dCI0sH1iCc5dFn0k6U7uti7QUeHa5AitpbRwWN?= =?us-ascii?Q?VrXuPrCfQj5TEvcyQQM53KYlXabZr48o0ydmcPRewTBP+2ybbPvqCcoioGSV?= =?us-ascii?Q?9kMflfXwQUb6mxehZLHkBA5waR6LPRfq5ArCYJoWqttFY+8gZ+RBxLU2F1Bh?= =?us-ascii?Q?mq490EVU1UjWoarNC+6Uocyk80shP1XMUwHbimRGsAIMfZuNLe2cyWMbCQAv?= =?us-ascii?Q?HW9v0WFzfHQumabkMM+tA3ykUBFXH7jY1AV+qz7mG6KynctstxHMdfQIlSgX?= =?us-ascii?Q?I+TsfGCaK06X+zRQkazkbviKR0ZA3hI+PB4WUIFe6pUtM28aJF6mpMxt56wb?= =?us-ascii?Q?6s6AP3hSEDwfIbvwjMeHia8V6PLlaK0sZ55tPw1pO1tGHd7zHsBNE+hjDWKY?= =?us-ascii?Q?yd06x5aMg6EbUjXy58fpv8ltUO+2ZFFapqy1KEOL18yqSZbrCwPOvUsF1uTk?= =?us-ascii?Q?lIpcI4w/RpU0Kgkx2vfdaSn6QicokBTr6QvIm2RtCcrEFEGJGg7mGheKursc?= =?us-ascii?Q?OwImcDLAto7GSk2ns+Zr5A9ucuI3YjsoS0PuIetdpc+j+sSm32n4KqTc1mj6?= =?us-ascii?Q?u/yogNTg+YUaE0q1tpdGao2o+QM0IZZYEi0Mvb+ZRF6IZo48XCZGMI9twPcl?= =?us-ascii?Q?IflXc7YxFfb14zgpo7R8Bj1ouNbnYgnX7lltedHxrG8yQ2hn+ncp8fXrxoFG?= =?us-ascii?Q?QNfqvTZqMk5oRPoPgIZUvTbdpUGrGRMAlW5vp7nRbWkRGTnPxlcE+u3wTALn?= =?us-ascii?Q?sNYtdycWuD70adtlDh0lB7kysA22glWXgb0ThdDQHNPmLOByTr6OTghrwe6j?= =?us-ascii?Q?mAdvVRXUpSKHkVTfHkFzMsbV+jCEq7kQp6JMGEZM1x2ZXG9jY4P/fCLIWbkO?= =?us-ascii?Q?rfWntA=3D=3D?= X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: 9H9kWvGOKyr1Ok9lcTYt19AvDMwNGXtE3Swbq8c2NTt3cEyNiEJ/rzo7hWRfYeYJN6E0cMIhEWknVs4Tvkwr5IjAgKu5Ges0ZnNLtAIBx4Ca0iwb+8y6RpegtclOrv22ll5OttAFP0FwArruSC9z8v1SdfhQfrEaQ5RQLDsMT05cbGuPGp8Y61zwqaUvOzURIw1lJFaxaAWqOw7YfqKqsHOjgHnLw0F8MJJCGekvG/BRV3GH35FYDbNR2Uj+cuH7yL46wZURS0bvEwc7YBp0weZ4u892k95ULVI3kKdwaCdgRwWrdO2kextRO0QyqmB523D/yvY5f5uLTiOYaT4J7E4QpR685d2R2SAR97uTDJfjPlhOdxL93L+q6AwK6qRDgbjunHM2/24B00TdZTCZBrmn5y+eGGwCV3yxzUR01DM0CyFRiZsKR/HTvxa4TweyMqJbqOHW6sWoPpgc0xS+pvyzFF/aTarzYreCIdQmzXHKeTLmL/ibg1rMFbxNb9vJ3YFbuXqalMXN5iTKiPkUBcLUS6upgvmbWxEptwbd7l3poybhLh+TxV6ACdcM9E+MKL8uzrQKJzwzPUlOZ1/llkJDvmgg/hl7WHKxKBHKISY= X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: e7539a0c-afad-49dd-a908-08dcf3a2bd80 X-MS-Exchange-CrossTenant-AuthSource: BYAPR10MB3366.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 23 Oct 2024 20:39:07.1473 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: ImhGGEXLaUoogCJwlHqfecr6wyRW0PbNhX+89ZQN69en0LH9bAqZTiDC6kOTFuGxgF5zhMM97NeipCraKRRkurD8rY+Los0VEoj3KWEUoHk= X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ0PR10MB5833 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1051,Hydra:6.0.680,FMLib:17.12.62.30 definitions=2024-10-23_16,2024-10-23_01,2024-09-30_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=0 phishscore=0 malwarescore=0 adultscore=0 mlxscore=0 mlxlogscore=999 spamscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2409260000 definitions=main-2410230132 X-Proofpoint-ORIG-GUID: MrrK6r_56xooXujw_gIF9lXeizzJIn5J X-Proofpoint-GUID: MrrK6r_56xooXujw_gIF9lXeizzJIn5J Content-Type: text/plain; charset="utf-8" Add some additional vma_internal.h stubs in preparation for __mmap_region() being moved to mm/vma.c. Without these the move would result in the tests no longer compiling. Signed-off-by: Lorenzo Stoakes --- tools/testing/vma/vma_internal.h | 115 ++++++++++++++++++++++++++++++- 1 file changed, 114 insertions(+), 1 deletion(-) diff --git a/tools/testing/vma/vma_internal.h b/tools/testing/vma/vma_inter= nal.h index c5b9da034511..77eba27365a2 100644 --- a/tools/testing/vma/vma_internal.h +++ b/tools/testing/vma/vma_internal.h @@ -44,7 +44,9 @@ #define VM_LOCKED 0x00002000 #define VM_IO 0x00004000 #define VM_DONTEXPAND 0x00040000 +#define VM_LOCKONFAULT 0x00080000 #define VM_ACCOUNT 0x00100000 +#define VM_NORESERVE 0x00200000 #define VM_MIXEDMAP 0x10000000 #define VM_STACK VM_GROWSDOWN #define VM_SHADOW_STACK VM_NONE @@ -53,6 +55,14 @@ #define VM_ACCESS_FLAGS (VM_READ | VM_WRITE | VM_EXEC) #define VM_SPECIAL (VM_IO | VM_DONTEXPAND | VM_PFNMAP | VM_MIXEDMAP) +/* This mask represents all the VMA flag bits used by mlock */ +#define VM_LOCKED_MASK (VM_LOCKED | VM_LOCKONFAULT) + +#ifdef CONFIG_64BIT +/* VM is sealed, in vm_flags */ +#define VM_SEALED _BITUL(63) +#endif + #define FIRST_USER_ADDRESS 0UL #define USER_PGTABLES_CEILING 0UL @@ -698,8 +708,9 @@ static inline void tlb_finish_mmu(struct mmu_gather *) { } -static inline void get_file(struct file *) +static inline struct file *get_file(struct file *f) { + return f; } static inline int vma_dup_policy(struct vm_area_struct *, struct vm_area_s= truct *) @@ -920,4 +931,106 @@ static inline bool signal_pending(void *) return false; } +static inline bool is_file_hugepages(struct file *) +{ + return false; +} + +static inline int security_vm_enough_memory_mm(struct mm_struct *, long) +{ + return true; +} + +static inline bool may_expand_vm(struct mm_struct *, vm_flags_t, unsigned = long) +{ + return true; +} + +static inline void vm_flags_init(struct vm_area_struct *vma, + vm_flags_t flags) +{ + vma->__vm_flags =3D flags; +} + +static inline void vm_flags_set(struct vm_area_struct *vma, + vm_flags_t flags) +{ + vma_start_write(vma); + vma->__vm_flags |=3D flags; +} + +static inline void vm_flags_clear(struct vm_area_struct *vma, + vm_flags_t flags) +{ + vma_start_write(vma); + vma->__vm_flags &=3D ~flags; +} + +static inline int call_mmap(struct file *, struct vm_area_struct *) +{ + return 0; +} + +static inline int shmem_zero_setup(struct vm_area_struct *) +{ + return 0; +} + +static inline void vma_set_anonymous(struct vm_area_struct *vma) +{ + vma->vm_ops =3D NULL; +} + +static inline void ksm_add_vma(struct vm_area_struct *) +{ +} + +static inline void perf_event_mmap(struct vm_area_struct *) +{ +} + +static inline bool vma_is_dax(struct vm_area_struct *) +{ + return false; +} + +static inline struct vm_area_struct *get_gate_vma(struct mm_struct *) +{ + return NULL; +} + +bool vma_wants_writenotify(struct vm_area_struct *vma, pgprot_t vm_page_pr= ot); + +/* Update vma->vm_page_prot to reflect vma->vm_flags. */ +static inline void vma_set_page_prot(struct vm_area_struct *vma) +{ + unsigned long vm_flags =3D vma->vm_flags; + pgprot_t vm_page_prot; + + /* testing: we inline vm_pgprot_modify() to avoid clash with vma.h. */ + vm_page_prot =3D pgprot_modify(vma->vm_page_prot, vm_get_page_prot(vm_fla= gs)); + + if (vma_wants_writenotify(vma, vm_page_prot)) { + vm_flags &=3D ~VM_SHARED; + /* testing: we inline vm_pgprot_modify() to avoid clash with vma.h. */ + vm_page_prot =3D pgprot_modify(vm_page_prot, vm_get_page_prot(vm_flags)); + } + /* remove_protection_ptes reads vma->vm_page_prot without mmap_lock */ + WRITE_ONCE(vma->vm_page_prot, vm_page_prot); +} + +static inline bool arch_validate_flags(unsigned long) +{ + return true; +} + +static inline void vma_close(struct vm_area_struct *) +{ +} + +static inline int mmap_file(struct file *, struct vm_area_struct *) +{ + return 0; +} + #endif /* __MM_VMA_INTERNAL_H */ -- 2.47.0 From nobody Mon Nov 25 22:30:55 2024 Received: from mx0b-00069f02.pphosted.com (mx0b-00069f02.pphosted.com [205.220.177.32]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 47F371FBC8C for ; Wed, 23 Oct 2024 20:39:24 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=205.220.177.32 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1729715966; cv=fail; b=QCkMiur2Ld+WLqpmkUWiuSt1BzJ+8FoUVuoNq7HZ36wfyeLA5TlJ91Bdslo/+8BXdxloDnL/lVqwDwYAV/WE7RdjgdLBP9zI3MojdG1vW6dOAPTMaYF0eZ/7yTLF/95aNg+aGRUGumOZIpgD57cnlg/vwjRHvnd9g/Y7ujLghgg= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1729715966; c=relaxed/simple; bh=9HHSHDknJDIkcUn4ReqKeG8cLbPBsaDVzMHMLx5oIyk=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: Content-Type:MIME-Version; b=q2LrM/F4QPKQi1mANCi8D06eZ4AoDhv9UfMRmn0LdG/wnUVUoHOO+EhjI+vLi0EU+jAyqSEq4dtv8F1g5n092lLwiW7tm8r5wEwUZBFZFKHVjAuJw5NcAjfoeavT+qlQ28177BC5htKMW5cWk5LR1DUcdJlovbOpFDWkAzGsDow= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=oracle.com; spf=pass smtp.mailfrom=oracle.com; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b=EW2lFWf4; dkim=pass (1024-bit key) header.d=oracle.onmicrosoft.com header.i=@oracle.onmicrosoft.com header.b=uUan/e/v; arc=fail smtp.client-ip=205.220.177.32 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=oracle.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=oracle.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b="EW2lFWf4"; dkim=pass (1024-bit key) header.d=oracle.onmicrosoft.com header.i=@oracle.onmicrosoft.com header.b="uUan/e/v" Received: from pps.filterd (m0246632.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 49NKcahE005402; Wed, 23 Oct 2024 20:39:14 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=cc :content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s= corp-2023-11-20; bh=UJegJK5P09zD1reHH/LUQnoUje3wTD/Wt9CcED7Sn9Q=; b= EW2lFWf4vrwwN+wbHNdicCoop7reb+TshGdFiP+NOdmgweQ4WaE2R3J5YxAZkPC2 YrdAJFgZ5BcCAmgt872/idZvNSLzuTlTiW943vHXAK8xNSJCJuF4BfFwcr4fUut8 cwf3ugpfLzR/OCjCFpoeHidUTavT50J0WHtlPVEFTC9mAYrQECEeCZfHsDD/7iFQ wkuma2gMtaKKKKmNd28B7oWIGQugFWreEUK6ttfteERkqm92kji0svUr3njp4zie J3YtjDM1n0QkMBPWHgXbnwYUmMWcovwn6LmKzd6lrXVT6WSbkQsUOMQfOtEeRxOt ti/3uxgHEyOBnujGjnxLcw== Received: from iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta03.appoci.oracle.com [130.35.103.27]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 42ckkr0e4b-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 23 Oct 2024 20:39:13 +0000 (GMT) Received: from pps.filterd (iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com (8.18.1.2/8.18.1.2) with ESMTP id 49NJMpf1039579; Wed, 23 Oct 2024 20:39:13 GMT Received: from nam04-bn8-obe.outbound.protection.outlook.com (mail-bn8nam04lp2045.outbound.protection.outlook.com [104.47.74.45]) by iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTPS id 42emhbj4dt-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 23 Oct 2024 20:39:13 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=LVvBTp5U49Slfa4B4lOXOjKxhoXckoPfmcjdJj/9pXZ7xp8P/3v5W5OuM1HpFQslonNGbxT+cRYpTv6CCMXU29Phl6WBEUtPFgmHrYLmpwB/rFJbenXHNSl3G8A1djbdWiJu3iO5bO6it3exjizhDOmJ7dVpjBj47co3ioW1dMoL1WK3B86dxDrU7tVq0yC6dEmHtJrtMI9TloV4LTCLTcgPfFowqZTc8S5+9pDfJIsG9uq7dshvrzcdkrh+RhvgjGrEl2Wpq+OqFcTSJcFnE0c2PEL+HNUIwkE55zr7UjO3RNVs4+Z3RM+3xRe1HI2MDZ72z83eQ1KZPBMvsJfFLA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=UJegJK5P09zD1reHH/LUQnoUje3wTD/Wt9CcED7Sn9Q=; b=Aol7WYtd/+FXTyUuuo5f84wA1CtItML9tRjIKCDLpG2dHib3PaK5rzqTajTBg2Yy1+h8GD9Ki1+W4Wh/c9LQON+52eeTaGiqV825O59UMnAXPU6GAKt+w+TaLZX+24/z9QbD1Fo6oD3qSRa5vG+3N0t1ir3qO7iBp7s4OB8Yf/1q5wFbDR/J++L0KRRswNkSWQ7F5/QXciQ9q990dsLJJDaXBJWbRfVRhhYypVbt8aqUiH7Kz7Qfhua0PrQRrIjN9akZ5uVCRR6lRNiFlt1EcdfNFqmZdqInednSbr/My4N9O0q9C4vAQI35OIQByTi/z590imRO6bDZRCx6oZCHIQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=UJegJK5P09zD1reHH/LUQnoUje3wTD/Wt9CcED7Sn9Q=; b=uUan/e/vq83MDUXwp+vQnpf6O8SxMr/DyF/vCwrdIIkQ6gGWJhXQuM1LD+zk9LBQd6tuYj7AREL3DfZ6loAgfZFYPPxsYxkReiuwGVCMVgUqVsxPDN3HFlVGJB+2CSy8NTwrc11j/d03Dth1pzTEZiW3n5XlqaCDiycehO5InnY= Received: from BYAPR10MB3366.namprd10.prod.outlook.com (2603:10b6:a03:14f::25) by SJ0PR10MB5833.namprd10.prod.outlook.com (2603:10b6:a03:3ed::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8093.17; Wed, 23 Oct 2024 20:39:10 +0000 Received: from BYAPR10MB3366.namprd10.prod.outlook.com ([fe80::baf2:dff1:d471:1c9]) by BYAPR10MB3366.namprd10.prod.outlook.com ([fe80::baf2:dff1:d471:1c9%6]) with mapi id 15.20.8069.024; Wed, 23 Oct 2024 20:39:10 +0000 From: Lorenzo Stoakes To: Andrew Morton Cc: "Liam R . Howlett" , Vlastimil Babka , Jann Horn , linux-kernel@vger.kernel.org, linux-mm@kvack.org, Linus Torvalds , Peter Xu Subject: [PATCH v2 6/8] mm: isolate mmap internal logic to mm/vma.c Date: Wed, 23 Oct 2024 21:38:31 +0100 Message-ID: <67d9ba63e203452431c7ee7802d3988bd1258ab6.1729715266.git.lorenzo.stoakes@oracle.com> X-Mailer: git-send-email 2.47.0 In-Reply-To: References: Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: LO4P265CA0178.GBRP265.PROD.OUTLOOK.COM (2603:10a6:600:311::9) To BYAPR10MB3366.namprd10.prod.outlook.com (2603:10b6:a03:14f::25) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BYAPR10MB3366:EE_|SJ0PR10MB5833:EE_ X-MS-Office365-Filtering-Correlation-Id: 8809d1e3-4e40-4a9a-409b-08dcf3a2bfa3 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|366016|1800799024|376014|10070799003; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?0JAkxuIxstkFo81uIE4+QA9VSAX9uPTibYEip5676E1B6vJTXMwstWEHaauK?= =?us-ascii?Q?BQvFaEiMTYFhzDc98w8qq4wWzMTKW43DwfFTR03jHXZWSoZOzCg3OqbiTtCd?= =?us-ascii?Q?Whbjmd4szs3Hcc7ZxqFu3OTlPMV9+g09oLugt5m4ewWBSCuexiDYknec1JFb?= =?us-ascii?Q?GTAV5bbCAUdaYwl38v8xY7YvWP6RCXEn4PEOEpbsgaaaoZggDOOIr9hqLxOz?= =?us-ascii?Q?8BaI3QQmfExKAuK8AQE58KCANpei2SVZv5iBbmWlhvImhqzFw6KIjn5efczE?= =?us-ascii?Q?jxbM5kBmSaoBKCIc6f0oowDrNT8lDx2+PX42s3F2k00xTS457+szpFNnxMks?= =?us-ascii?Q?Sa1HFDil7jCVe3AmcR0MN7V6QXr81RYoNcej+JAgZydk/f9ajfOzCrYYbd2p?= =?us-ascii?Q?u2/C8SR6kk4PmNuBtsJVrHboc2Mu1w7VeSZwiPssmGk7cXrgnH0gl1aU9iV/?= =?us-ascii?Q?6yzyzjtZRydd/tBYuHHv+LHf4JCMqGGf6bmVnEUies0VuC99ESk1vqAgLajF?= =?us-ascii?Q?lpuJ+vtb6mePq+9Xm2kcfR/BGw/EhcF/DjEQmqJl+tzuABfpgL22EnA9PTVZ?= =?us-ascii?Q?jJ6lpD60/MGsKyOU45oLB7gaDJ1+VyUCKo8oYsDmmoT5duyHqiOlXlEMt0I1?= =?us-ascii?Q?e6OcsaeTq6q8iEg8VZNZnZ/o4Sir+JcjF87rqwsGPBnGvNwOBSGN9PcqlOlh?= =?us-ascii?Q?sFh0Ynm5XiGreD8qN66+SLGpWnf7DbQxjCi2hYiuayHaNDFzN4rOQkGcNCsT?= =?us-ascii?Q?6ZyMim5KAZiqfa9bioEd982UGqnEjiQTYDcbYorrt4s27ldQf3Z6mHOTBH0A?= =?us-ascii?Q?wPLZZKEvePM1sLSBmCsYkbwKmReF6Nf+Z7I5N0SE82Y9qJDVg9lRSAbU7Sez?= =?us-ascii?Q?NGYMhneu8AhQz+CFISYoh/efg0sUGKDY5w/TL7O8y2RL9zfUJtUBJfqGkPTJ?= =?us-ascii?Q?Gp36MQwyqP10AlSBGgmOB3S/vbf4cWmj6hY0zjVLekQ98iwPOn9iKzmDnD8W?= =?us-ascii?Q?4D2KxFXGmOSZFZv5vmGw+G1+xHGQa1BoymKWLLSry/3Lt6mmHAfPGX22uhFn?= =?us-ascii?Q?ncZZwKrCqbY6aJl9YhzUpVsDBW/jb78sMZeyjscJahpt5zkSClysQZqbNhga?= =?us-ascii?Q?p8mutU5fxT1u4ncYDgy/Ix1hVhgMGzuZPEaMmpuqvr/1d/S3ADJNQwpMtxk+?= =?us-ascii?Q?E7Nq8ExXcyURzjJD3zsM0csBkbXFZNiwnwj8YiTbL8KEQMh5Rw/r5PUWHFJI?= =?us-ascii?Q?RujxBHh6RDcWVsXYyiNN0MluPRMkw1f9pFwOCKFiaJMBBRko5YJ9vw+BUmOe?= =?us-ascii?Q?26dfRszJhvygkV6xpnsEVwLM?= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BYAPR10MB3366.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(366016)(1800799024)(376014)(10070799003);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?ajRKHCrAXrIqzquSqHEty1gvdwmcowUrkvqBD27duIRY5OZAry4Ic93RZ1Yd?= =?us-ascii?Q?Bcqh0uLj9rhGaJ/oqoyNBzRTpupU2dU2yOENo5hmpvwiNeSu/Bjp0qQHLIej?= =?us-ascii?Q?ZdSGYj7vVqBeoli7n3jFSUwg/TGo+j5Y5UIqDpTjhQ3QMRr55OkbCFgC6UWe?= =?us-ascii?Q?pgHsMWBTgUhrnRBHEjF0XmaDcGVoAPYCctz8Vqd8hDJhJX3tV4IRcJXB7rrc?= =?us-ascii?Q?//feMcidVr0lKRYIZFVOjC9VCoNP7q77sbit223hZeZFf/R+xvjFhb76qdCX?= =?us-ascii?Q?GLl45TuTsL8QQH59MBuoh5Sg51h5mlHzO8TnV51ir0TH4EiZ1tzK6aDawMCw?= =?us-ascii?Q?91NwitWSMwdGP1tQ5oVaS8YNABgEY0GgNBxirQ3VXXGLzrAvtK2Pt+G4QuID?= =?us-ascii?Q?DWKy+DpCb5/N3ojT2aSZ9b89p7h2uBGzffAb51qxxqfbEZD4lxvJGmaPzQwC?= =?us-ascii?Q?pjmh4HmAoPTYLDINSHb5voMBd4AGUt5JUAP4r6etMGub9yE4usSFz6wwMfk5?= =?us-ascii?Q?G2i3XnqMYnL5O3tDpr4oEKKMgs5irLiCjiBaGmV1/BXG+JXQ9Ffhbs0yBAfC?= =?us-ascii?Q?Z3AUkUd4HGJmvHVzcZEFSztzWjQLYeEDY9Oa3TVLEqpNWYNOPffix2bDnzrB?= =?us-ascii?Q?WtzfjeuoVDl2caPEOTsfayEiYTathu9+Uxy5Zl7LFR/vKJfJJtJhuX2XPMXt?= =?us-ascii?Q?5VBPsiaV+fzfGa33qcrci7bt8GIgHML08xew7W/y39+V2kIXgMEENtY9XtJz?= =?us-ascii?Q?2JntLEJh4DvfNBCE1iiuswF2WUtC8onLhGEm9eywma0kS37b81kCFqTSKuct?= =?us-ascii?Q?aIIVU3fg/ty0MF8+K3IguDGnB0VfYq7/UqNHSrrnnPTfcWGgrWvdKpw4APf2?= =?us-ascii?Q?2g/jBpkSRDVwYXi+ep0HxxdXI9zRYSjRe+FuBZolMIHUTbLLPWz403a8I9vq?= =?us-ascii?Q?+akFWB1dkNadrZ6tx0eoxpV/GYdoeDB3+jVmh1Vk7e5qkt1PFXjzSyHi9Ey/?= =?us-ascii?Q?l+zgT5MlaU54ZqiTz1RjsrGNXA/5V5LVgw4xRaSMPSzhm4iD0BbzddbwiV0A?= =?us-ascii?Q?gQ1cqp59+3YmcStujOW56b2oyKIsVyGT/nPQoABNlqyrRHwDLLY64ceLSWV2?= =?us-ascii?Q?Q7RAdPgnCSNUfp03qrWuObmKb/L2tkVNqfofVSqBLJyhQ2ZUa7tVyu0aGanP?= =?us-ascii?Q?bamqgYcFXEmeBFdM8ydDjC8TdmEMFLipOHKMVa7NKt/GiLxj8WW90K58udVA?= =?us-ascii?Q?UMMhG2m9ETp/r/zocObik2WP0IH1DQz7dfN+Gsbytcmfa/ofQQrW/EhQ9vel?= =?us-ascii?Q?cHBeq6CmNgVzGPf6GZ7dugoft4rJ9BvfzpcX/ihg6RUT0CdIlzrMokIgpLgR?= =?us-ascii?Q?TrM4aUPeQl7jhT10ZhDDQmJd4e1npd5rESSO46vv1RiBGr1VMmU99wP6dMx2?= =?us-ascii?Q?J1RsYBUkDR2X1OuZ/epzy/H/3igV0ruLBwjP4TNOx5YoJXZadijF0xwqkZKi?= =?us-ascii?Q?Lefi0nTwgY/MoapxHanFv4P1VLdLjY0xdPPA5wY5Rw1tGCUA7FSK5d6zeYA+?= =?us-ascii?Q?oBbrTlt3/HyVbiGks6QTVP3vn3srCVjaV4YtRZX3mwb85ehyiFEN+pu5M83A?= =?us-ascii?Q?ovqV9yUDr6VBT5qPNKUmL2AuBv43Xi1YnRamqJNNqiZ+/zlKJCtvmfKDkkan?= =?us-ascii?Q?5WKYQg=3D=3D?= X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: JqPHJ9r8rqQ0KV12cUKRpKfcWLpwkkOdRrVHisRdYByUwJr90wAeftzxahtXhn9s0K0dEJHSbHf46W9jee9j9IHcK1nvjV6lKJzd/D04qH06EvM1DKzXhs5CahPVF+uajf6vvbHRgep+DFHXM5ECw1cVwySRNz+6DrnQNBb9wWJzuVziPDAhN46on4PRJ7Fi2RGQLkvqgsZNpBybHktZuOTbn8FcfnKYytXCdVgK9iq0SPtU2kO42Bqg1vgGwWhz4I5n1AMtyD9Wjq2cqmJdKDUumzLLYRHl7gReEQsriHoi7VI4BxRo2GMs4SguVT0cejA8UzRCGfBk1ZJydIqHGD6KV63gSCgz8t0TDDE3E+7bqdssvBOISeyQGQkFAwkxj6kOABmyJpx2edB8mDVRR/Morbob4S+9gfycvGyToMq0ByvB4bi57IuX4wBZoPw2SAgiWHgdTb6yyOL3IIn1NzH0zVQFhKd60EoLxiq7R8O0mBL7osIz3uhXJQNcdB2wHfFp/D6Fkjz8uXdoAkTxP/fbX8kDO1hiY2lz34sZgwmFoy5EznmyVuaNLlppc6BBr9WAVtBrpDOFOHL1vB3p8XuvCc/p/i5uJrcXhTcWb/c= X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: 8809d1e3-4e40-4a9a-409b-08dcf3a2bfa3 X-MS-Exchange-CrossTenant-AuthSource: BYAPR10MB3366.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 23 Oct 2024 20:39:10.7486 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: /Icfi16DXD6lROd2jR6UWGONfDQNg5CXv+9cF2DSbu1wtGGk3NmxwXPRRhB7rwH1BKIlkO4ER6hgoo/NqFiWtkTjzFjdBVQbxk5oxx3KSCI= X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ0PR10MB5833 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1051,Hydra:6.0.680,FMLib:17.12.62.30 definitions=2024-10-23_16,2024-10-23_01,2024-09-30_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 spamscore=0 mlxscore=0 phishscore=0 suspectscore=0 bulkscore=0 adultscore=0 malwarescore=0 mlxlogscore=999 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2409260000 definitions=main-2410230132 X-Proofpoint-GUID: NdrEEQCvf8Dfy0wg6hR1z00SdQhSvRPk X-Proofpoint-ORIG-GUID: NdrEEQCvf8Dfy0wg6hR1z00SdQhSvRPk Content-Type: text/plain; charset="utf-8" In previous commits we effected improvements to the mmap() logic in mmap_region() and its newly introduced internal implementation function __mmap_region(). However as these changes are intended to be backported, we kept the delta as small as is possible and made as few changes as possible to the newly introduced mm/vma.* files. Take the opportunity to move this logic to mm/vma.c which not only isolates it, but also makes it available for later userland testing which can help us catch such logic errors far earlier. Signed-off-by: Lorenzo Stoakes Reviewed-by: Vlastimil Babka --- mm/mmap.c | 234 --------------------------------- mm/vma.c | 323 +++++++++++++++++++++++++++++++++++++++++++++- mm/vma.h | 97 +------------- mm/vma_internal.h | 5 + 4 files changed, 329 insertions(+), 330 deletions(-) diff --git a/mm/mmap.c b/mm/mmap.c index e686d57ed9f7..0affd1a0687f 100644 --- a/mm/mmap.c +++ b/mm/mmap.c @@ -577,22 +577,6 @@ SYSCALL_DEFINE1(old_mmap, struct mmap_arg_struct __use= r *, arg) } #endif /* __ARCH_WANT_SYS_OLD_MMAP */ =20 -/* - * We account for memory if it's a private writeable mapping, - * not hugepages and VM_NORESERVE wasn't set. - */ -static inline bool accountable_mapping(struct file *file, vm_flags_t vm_fl= ags) -{ - /* - * hugetlb has its own accounting separate from the core VM - * VM_HUGETLB may not be set yet so we cannot check for that flag. - */ - if (file && is_file_hugepages(file)) - return false; - - return (vm_flags & (VM_NORESERVE | VM_SHARED | VM_WRITE)) =3D=3D VM_WRITE; -} - /** * unmapped_area() - Find an area between the low_limit and the high_limit= with * the correct alignment and offset, all from @info. Note: current->mm is = used @@ -1361,224 +1345,6 @@ int do_munmap(struct mm_struct *mm, unsigned long s= tart, size_t len, return do_vmi_munmap(&vmi, mm, start, len, uf, false); } =20 -static unsigned long __mmap_region(struct file *file, unsigned long addr, - unsigned long len, vm_flags_t vm_flags, unsigned long pgoff, - struct list_head *uf) -{ - struct mm_struct *mm =3D current->mm; - struct vm_area_struct *vma =3D NULL; - pgoff_t pglen =3D PHYS_PFN(len); - unsigned long charged =3D 0; - struct vma_munmap_struct vms; - struct ma_state mas_detach; - struct maple_tree mt_detach; - unsigned long end =3D addr + len; - int error; - VMA_ITERATOR(vmi, mm, addr); - VMG_STATE(vmg, mm, &vmi, addr, end, vm_flags, pgoff); - - vmg.file =3D file; - /* Find the first overlapping VMA */ - vma =3D vma_find(&vmi, end); - init_vma_munmap(&vms, &vmi, vma, addr, end, uf, /* unlock =3D */ false); - if (vma) { - mt_init_flags(&mt_detach, vmi.mas.tree->ma_flags & MT_FLAGS_LOCK_MASK); - mt_on_stack(mt_detach); - mas_init(&mas_detach, &mt_detach, /* addr =3D */ 0); - /* Prepare to unmap any existing mapping in the area */ - error =3D vms_gather_munmap_vmas(&vms, &mas_detach); - if (error) - goto gather_failed; - - vmg.next =3D vms.next; - vmg.prev =3D vms.prev; - vma =3D NULL; - } else { - vmg.next =3D vma_iter_next_rewind(&vmi, &vmg.prev); - } - - /* Check against address space limit. */ - if (!may_expand_vm(mm, vm_flags, pglen - vms.nr_pages)) { - error =3D -ENOMEM; - goto abort_munmap; - } - - /* - * Private writable mapping: check memory availability - */ - if (accountable_mapping(file, vm_flags)) { - charged =3D pglen; - charged -=3D vms.nr_accounted; - if (charged) { - error =3D security_vm_enough_memory_mm(mm, charged); - if (error) - goto abort_munmap; - } - - vms.nr_accounted =3D 0; - vm_flags |=3D VM_ACCOUNT; - vmg.flags =3D vm_flags; - } - - /* - * clear PTEs while the vma is still in the tree so that rmap - * cannot race with the freeing later in the truncate scenario. - * This is also needed for mmap_file(), which is why vm_ops - * close function is called. - */ - vms_clean_up_area(&vms, &mas_detach); - vma =3D vma_merge_new_range(&vmg); - if (vma) - goto expanded; - /* - * Determine the object being mapped and call the appropriate - * specific mapper. the address has already been validated, but - * not unmapped, but the maps are removed from the list. - */ - vma =3D vm_area_alloc(mm); - if (!vma) { - error =3D -ENOMEM; - goto unacct_error; - } - - vma_iter_config(&vmi, addr, end); - vma_set_range(vma, addr, end, pgoff); - vm_flags_init(vma, vm_flags); - vma->vm_page_prot =3D vm_get_page_prot(vm_flags); - - if (vma_iter_prealloc(&vmi, vma)) { - error =3D -ENOMEM; - goto free_vma; - } - - if (file) { - vma->vm_file =3D get_file(file); - error =3D mmap_file(file, vma); - if (error) - goto unmap_and_free_file_vma; - - /* Drivers cannot alter the address of the VMA. */ - WARN_ON_ONCE(addr !=3D vma->vm_start); - /* - * Drivers should not permit writability when previously it was - * disallowed. - */ - VM_WARN_ON_ONCE(vm_flags !=3D vma->vm_flags && - !(vm_flags & VM_MAYWRITE) && - (vma->vm_flags & VM_MAYWRITE)); - - vma_iter_config(&vmi, addr, end); - /* - * If vm_flags changed after mmap_file(), we should try merge - * vma again as we may succeed this time. - */ - if (unlikely(vm_flags !=3D vma->vm_flags && vmg.prev)) { - struct vm_area_struct *merge; - - vmg.flags =3D vma->vm_flags; - /* If this fails, state is reset ready for a reattempt. */ - merge =3D vma_merge_new_range(&vmg); - - if (merge) { - /* - * ->mmap() can change vma->vm_file and fput - * the original file. So fput the vma->vm_file - * here or we would add an extra fput for file - * and cause general protection fault - * ultimately. - */ - fput(vma->vm_file); - vm_area_free(vma); - vma =3D merge; - /* Update vm_flags to pick up the change. */ - vm_flags =3D vma->vm_flags; - goto file_expanded; - } - vma_iter_config(&vmi, addr, end); - } - - vm_flags =3D vma->vm_flags; - } else if (vm_flags & VM_SHARED) { - error =3D shmem_zero_setup(vma); - if (error) - goto free_iter_vma; - } else { - vma_set_anonymous(vma); - } - -#ifdef CONFIG_SPARC64 - /* TODO: Fix SPARC ADI! */ - WARN_ON_ONCE(!arch_validate_flags(vm_flags)); -#endif - - /* Lock the VMA since it is modified after insertion into VMA tree */ - vma_start_write(vma); - vma_iter_store(&vmi, vma); - mm->map_count++; - vma_link_file(vma); - - /* - * vma_merge_new_range() calls khugepaged_enter_vma() too, the below - * call covers the non-merge case. - */ - khugepaged_enter_vma(vma, vma->vm_flags); - -file_expanded: - file =3D vma->vm_file; - ksm_add_vma(vma); -expanded: - perf_event_mmap(vma); - - /* Unmap any existing mapping in the area */ - vms_complete_munmap_vmas(&vms, &mas_detach); - - vm_stat_account(mm, vm_flags, pglen); - if (vm_flags & VM_LOCKED) { - if ((vm_flags & VM_SPECIAL) || vma_is_dax(vma) || - is_vm_hugetlb_page(vma) || - vma =3D=3D get_gate_vma(current->mm)) - vm_flags_clear(vma, VM_LOCKED_MASK); - else - mm->locked_vm +=3D pglen; - } - - if (file) - uprobe_mmap(vma); - - /* - * New (or expanded) vma always get soft dirty status. - * Otherwise user-space soft-dirty page tracker won't - * be able to distinguish situation when vma area unmapped, - * then new mapped in-place (which must be aimed as - * a completely new data area). - */ - vm_flags_set(vma, VM_SOFTDIRTY); - - vma_set_page_prot(vma); - - return addr; - -unmap_and_free_file_vma: - fput(vma->vm_file); - vma->vm_file =3D NULL; - - vma_iter_set(&vmi, vma->vm_end); - /* Undo any partial mapping done by a device driver. */ - unmap_region(&vmi.mas, vma, vmg.prev, vmg.next); -free_iter_vma: - vma_iter_free(&vmi); -free_vma: - vm_area_free(vma); -unacct_error: - if (charged) - vm_unacct_memory(charged); - -abort_munmap: - vms_abort_munmap_vmas(&vms, &mas_detach); -gather_failed: - return error; -} - unsigned long mmap_region(struct file *file, unsigned long addr, unsigned long len, vm_flags_t vm_flags, unsigned long pgoff, struct list_head *uf) diff --git a/mm/vma.c b/mm/vma.c index bb7cfa2dc282..0a2965be582d 100644 --- a/mm/vma.c +++ b/mm/vma.c @@ -1103,7 +1103,7 @@ static inline void vms_clear_ptes(struct vma_munmap_s= truct *vms, vms->clear_ptes =3D false; } =20 -void vms_clean_up_area(struct vma_munmap_struct *vms, +static void vms_clean_up_area(struct vma_munmap_struct *vms, struct ma_state *mas_detach) { struct vm_area_struct *vma; @@ -1126,7 +1126,7 @@ void vms_clean_up_area(struct vma_munmap_struct *vms, * used for the munmap() and may downgrade the lock - if requested. Every= thing * needed to be done once the vma maple tree is updated. */ -void vms_complete_munmap_vmas(struct vma_munmap_struct *vms, +static void vms_complete_munmap_vmas(struct vma_munmap_struct *vms, struct ma_state *mas_detach) { struct vm_area_struct *vma; @@ -1167,6 +1167,23 @@ void vms_complete_munmap_vmas(struct vma_munmap_stru= ct *vms, __mt_destroy(mas_detach->tree); } =20 +/* + * reattach_vmas() - Undo any munmap work and free resources + * @mas_detach: The maple state with the detached maple tree + * + * Reattach any detached vmas and free up the maple tree used to track the= vmas. + */ +static void reattach_vmas(struct ma_state *mas_detach) +{ + struct vm_area_struct *vma; + + mas_set(mas_detach, 0); + mas_for_each(mas_detach, vma, ULONG_MAX) + vma_mark_detached(vma, false); + + __mt_destroy(mas_detach->tree); +} + /* * vms_gather_munmap_vmas() - Put all VMAs within a range into a maple tree * for removal at a later date. Handles splitting first and last if neces= sary @@ -1177,7 +1194,7 @@ void vms_complete_munmap_vmas(struct vma_munmap_struc= t *vms, * * Return: 0 on success, error otherwise */ -int vms_gather_munmap_vmas(struct vma_munmap_struct *vms, +static int vms_gather_munmap_vmas(struct vma_munmap_struct *vms, struct ma_state *mas_detach) { struct vm_area_struct *next =3D NULL; @@ -1315,6 +1332,39 @@ int vms_gather_munmap_vmas(struct vma_munmap_struct = *vms, return error; } =20 +/* + * init_vma_munmap() - Initializer wrapper for vma_munmap_struct + * @vms: The vma munmap struct + * @vmi: The vma iterator + * @vma: The first vm_area_struct to munmap + * @start: The aligned start address to munmap + * @end: The aligned end address to munmap + * @uf: The userfaultfd list_head + * @unlock: Unlock after the operation. Only unlocked on success + */ +static void init_vma_munmap(struct vma_munmap_struct *vms, + struct vma_iterator *vmi, struct vm_area_struct *vma, + unsigned long start, unsigned long end, struct list_head *uf, + bool unlock) +{ + vms->vmi =3D vmi; + vms->vma =3D vma; + if (vma) { + vms->start =3D start; + vms->end =3D end; + } else { + vms->start =3D vms->end =3D 0; + } + vms->unlock =3D unlock; + vms->uf =3D uf; + vms->vma_count =3D 0; + vms->nr_pages =3D vms->locked_vm =3D vms->nr_accounted =3D 0; + vms->exec_vm =3D vms->stack_vm =3D vms->data_vm =3D 0; + vms->unmap_start =3D FIRST_USER_ADDRESS; + vms->unmap_end =3D USER_PGTABLES_CEILING; + vms->clear_ptes =3D false; +} + /* * do_vmi_align_munmap() - munmap the aligned region from @start to @end. * @vmi: The vma iterator @@ -2069,3 +2119,270 @@ void mm_drop_all_locks(struct mm_struct *mm) =20 mutex_unlock(&mm_all_locks_mutex); } + +/* + * We account for memory if it's a private writeable mapping, + * not hugepages and VM_NORESERVE wasn't set. + */ +static bool accountable_mapping(struct file *file, vm_flags_t vm_flags) +{ + /* + * hugetlb has its own accounting separate from the core VM + * VM_HUGETLB may not be set yet so we cannot check for that flag. + */ + if (file && is_file_hugepages(file)) + return false; + + return (vm_flags & (VM_NORESERVE | VM_SHARED | VM_WRITE)) =3D=3D VM_WRITE; +} + +/* + * vms_abort_munmap_vmas() - Undo as much as possible from an aborted munm= ap() + * operation. + * @vms: The vma unmap structure + * @mas_detach: The maple state with the detached maple tree + * + * Reattach any detached vmas, free up the maple tree used to track the vm= as. + * If that's not possible because the ptes are cleared (and vm_ops->closed= () may + * have been called), then a NULL is written over the vmas and the vmas are + * removed (munmap() completed). + */ +static void vms_abort_munmap_vmas(struct vma_munmap_struct *vms, + struct ma_state *mas_detach) +{ + struct ma_state *mas =3D &vms->vmi->mas; + + if (!vms->nr_pages) + return; + + if (vms->clear_ptes) + return reattach_vmas(mas_detach); + + /* + * Aborting cannot just call the vm_ops open() because they are often + * not symmetrical and state data has been lost. Resort to the old + * failure method of leaving a gap where the MAP_FIXED mapping failed. + */ + mas_set_range(mas, vms->start, vms->end - 1); + mas_store_gfp(mas, NULL, GFP_KERNEL|__GFP_NOFAIL); + /* Clean up the insertion of the unfortunate gap */ + vms_complete_munmap_vmas(vms, mas_detach); +} + +unsigned long __mmap_region(struct file *file, unsigned long addr, + unsigned long len, vm_flags_t vm_flags, unsigned long pgoff, + struct list_head *uf) +{ + struct mm_struct *mm =3D current->mm; + struct vm_area_struct *vma =3D NULL; + pgoff_t pglen =3D PHYS_PFN(len); + unsigned long charged =3D 0; + struct vma_munmap_struct vms; + struct ma_state mas_detach; + struct maple_tree mt_detach; + unsigned long end =3D addr + len; + int error; + VMA_ITERATOR(vmi, mm, addr); + VMG_STATE(vmg, mm, &vmi, addr, end, vm_flags, pgoff); + + vmg.file =3D file; + /* Find the first overlapping VMA */ + vma =3D vma_find(&vmi, end); + init_vma_munmap(&vms, &vmi, vma, addr, end, uf, /* unlock =3D */ false); + if (vma) { + mt_init_flags(&mt_detach, vmi.mas.tree->ma_flags & MT_FLAGS_LOCK_MASK); + mt_on_stack(mt_detach); + mas_init(&mas_detach, &mt_detach, /* addr =3D */ 0); + /* Prepare to unmap any existing mapping in the area */ + error =3D vms_gather_munmap_vmas(&vms, &mas_detach); + if (error) + goto gather_failed; + + vmg.next =3D vms.next; + vmg.prev =3D vms.prev; + vma =3D NULL; + } else { + vmg.next =3D vma_iter_next_rewind(&vmi, &vmg.prev); + } + + /* Check against address space limit. */ + if (!may_expand_vm(mm, vm_flags, pglen - vms.nr_pages)) { + error =3D -ENOMEM; + goto abort_munmap; + } + + /* + * Private writable mapping: check memory availability + */ + if (accountable_mapping(file, vm_flags)) { + charged =3D pglen; + charged -=3D vms.nr_accounted; + if (charged) { + error =3D security_vm_enough_memory_mm(mm, charged); + if (error) + goto abort_munmap; + } + + vms.nr_accounted =3D 0; + vm_flags |=3D VM_ACCOUNT; + vmg.flags =3D vm_flags; + } + + /* + * clear PTEs while the vma is still in the tree so that rmap + * cannot race with the freeing later in the truncate scenario. + * This is also needed for mmap_file(), which is why vm_ops + * close function is called. + */ + vms_clean_up_area(&vms, &mas_detach); + vma =3D vma_merge_new_range(&vmg); + if (vma) + goto expanded; + /* + * Determine the object being mapped and call the appropriate + * specific mapper. the address has already been validated, but + * not unmapped, but the maps are removed from the list. + */ + vma =3D vm_area_alloc(mm); + if (!vma) { + error =3D -ENOMEM; + goto unacct_error; + } + + vma_iter_config(&vmi, addr, end); + vma_set_range(vma, addr, end, pgoff); + vm_flags_init(vma, vm_flags); + vma->vm_page_prot =3D vm_get_page_prot(vm_flags); + + if (vma_iter_prealloc(&vmi, vma)) { + error =3D -ENOMEM; + goto free_vma; + } + + if (file) { + vma->vm_file =3D get_file(file); + error =3D mmap_file(file, vma); + if (error) + goto unmap_and_free_file_vma; + + /* Drivers cannot alter the address of the VMA. */ + WARN_ON_ONCE(addr !=3D vma->vm_start); + /* + * Drivers should not permit writability when previously it was + * disallowed. + */ + VM_WARN_ON_ONCE(vm_flags !=3D vma->vm_flags && + !(vm_flags & VM_MAYWRITE) && + (vma->vm_flags & VM_MAYWRITE)); + + vma_iter_config(&vmi, addr, end); + /* + * If vm_flags changed after mmap_file(), we should try merge + * vma again as we may succeed this time. + */ + if (unlikely(vm_flags !=3D vma->vm_flags && vmg.prev)) { + struct vm_area_struct *merge; + + vmg.flags =3D vma->vm_flags; + /* If this fails, state is reset ready for a reattempt. */ + merge =3D vma_merge_new_range(&vmg); + + if (merge) { + /* + * ->mmap() can change vma->vm_file and fput + * the original file. So fput the vma->vm_file + * here or we would add an extra fput for file + * and cause general protection fault + * ultimately. + */ + fput(vma->vm_file); + vm_area_free(vma); + vma =3D merge; + /* Update vm_flags to pick up the change. */ + vm_flags =3D vma->vm_flags; + goto file_expanded; + } + vma_iter_config(&vmi, addr, end); + } + + vm_flags =3D vma->vm_flags; + } else if (vm_flags & VM_SHARED) { + error =3D shmem_zero_setup(vma); + if (error) + goto free_iter_vma; + } else { + vma_set_anonymous(vma); + } + +#ifdef CONFIG_SPARC64 + /* TODO: Fix SPARC ADI! */ + WARN_ON_ONCE(!arch_validate_flags(vm_flags)); +#endif + + /* Lock the VMA since it is modified after insertion into VMA tree */ + vma_start_write(vma); + vma_iter_store(&vmi, vma); + mm->map_count++; + vma_link_file(vma); + + /* + * vma_merge_new_range() calls khugepaged_enter_vma() too, the below + * call covers the non-merge case. + */ + khugepaged_enter_vma(vma, vma->vm_flags); + +file_expanded: + file =3D vma->vm_file; + ksm_add_vma(vma); +expanded: + perf_event_mmap(vma); + + /* Unmap any existing mapping in the area */ + vms_complete_munmap_vmas(&vms, &mas_detach); + + vm_stat_account(mm, vm_flags, pglen); + if (vm_flags & VM_LOCKED) { + if ((vm_flags & VM_SPECIAL) || vma_is_dax(vma) || + is_vm_hugetlb_page(vma) || + vma =3D=3D get_gate_vma(current->mm)) + vm_flags_clear(vma, VM_LOCKED_MASK); + else + mm->locked_vm +=3D pglen; + } + + if (file) + uprobe_mmap(vma); + + /* + * New (or expanded) vma always get soft dirty status. + * Otherwise user-space soft-dirty page tracker won't + * be able to distinguish situation when vma area unmapped, + * then new mapped in-place (which must be aimed as + * a completely new data area). + */ + vm_flags_set(vma, VM_SOFTDIRTY); + + vma_set_page_prot(vma); + + return addr; + +unmap_and_free_file_vma: + fput(vma->vm_file); + vma->vm_file =3D NULL; + + vma_iter_set(&vmi, vma->vm_end); + /* Undo any partial mapping done by a device driver. */ + unmap_region(&vmi.mas, vma, vmg.prev, vmg.next); +free_iter_vma: + vma_iter_free(&vmi); +free_vma: + vm_area_free(vma); +unacct_error: + if (charged) + vm_unacct_memory(charged); + +abort_munmap: + vms_abort_munmap_vmas(&vms, &mas_detach); +gather_failed: + return error; +} diff --git a/mm/vma.h b/mm/vma.h index d58068c0ff2e..388d34748674 100644 --- a/mm/vma.h +++ b/mm/vma.h @@ -165,99 +165,6 @@ static inline int vma_iter_store_gfp(struct vma_iterat= or *vmi, return 0; } =20 -#ifdef CONFIG_MMU -/* - * init_vma_munmap() - Initializer wrapper for vma_munmap_struct - * @vms: The vma munmap struct - * @vmi: The vma iterator - * @vma: The first vm_area_struct to munmap - * @start: The aligned start address to munmap - * @end: The aligned end address to munmap - * @uf: The userfaultfd list_head - * @unlock: Unlock after the operation. Only unlocked on success - */ -static inline void init_vma_munmap(struct vma_munmap_struct *vms, - struct vma_iterator *vmi, struct vm_area_struct *vma, - unsigned long start, unsigned long end, struct list_head *uf, - bool unlock) -{ - vms->vmi =3D vmi; - vms->vma =3D vma; - if (vma) { - vms->start =3D start; - vms->end =3D end; - } else { - vms->start =3D vms->end =3D 0; - } - vms->unlock =3D unlock; - vms->uf =3D uf; - vms->vma_count =3D 0; - vms->nr_pages =3D vms->locked_vm =3D vms->nr_accounted =3D 0; - vms->exec_vm =3D vms->stack_vm =3D vms->data_vm =3D 0; - vms->unmap_start =3D FIRST_USER_ADDRESS; - vms->unmap_end =3D USER_PGTABLES_CEILING; - vms->clear_ptes =3D false; -} -#endif - -int vms_gather_munmap_vmas(struct vma_munmap_struct *vms, - struct ma_state *mas_detach); - -void vms_complete_munmap_vmas(struct vma_munmap_struct *vms, - struct ma_state *mas_detach); - -void vms_clean_up_area(struct vma_munmap_struct *vms, - struct ma_state *mas_detach); - -/* - * reattach_vmas() - Undo any munmap work and free resources - * @mas_detach: The maple state with the detached maple tree - * - * Reattach any detached vmas and free up the maple tree used to track the= vmas. - */ -static inline void reattach_vmas(struct ma_state *mas_detach) -{ - struct vm_area_struct *vma; - - mas_set(mas_detach, 0); - mas_for_each(mas_detach, vma, ULONG_MAX) - vma_mark_detached(vma, false); - - __mt_destroy(mas_detach->tree); -} - -/* - * vms_abort_munmap_vmas() - Undo as much as possible from an aborted munm= ap() - * operation. - * @vms: The vma unmap structure - * @mas_detach: The maple state with the detached maple tree - * - * Reattach any detached vmas, free up the maple tree used to track the vm= as. - * If that's not possible because the ptes are cleared (and vm_ops->closed= () may - * have been called), then a NULL is written over the vmas and the vmas are - * removed (munmap() completed). - */ -static inline void vms_abort_munmap_vmas(struct vma_munmap_struct *vms, - struct ma_state *mas_detach) -{ - struct ma_state *mas =3D &vms->vmi->mas; - if (!vms->nr_pages) - return; - - if (vms->clear_ptes) - return reattach_vmas(mas_detach); - - /* - * Aborting cannot just call the vm_ops open() because they are often - * not symmetrical and state data has been lost. Resort to the old - * failure method of leaving a gap where the MAP_FIXED mapping failed. - */ - mas_set_range(mas, vms->start, vms->end - 1); - mas_store_gfp(mas, NULL, GFP_KERNEL|__GFP_NOFAIL); - /* Clean up the insertion of the unfortunate gap */ - vms_complete_munmap_vmas(vms, mas_detach); -} - int do_vmi_align_munmap(struct vma_iterator *vmi, struct vm_area_struct *vma, struct mm_struct *mm, unsigned long start, @@ -336,6 +243,10 @@ bool vma_wants_writenotify(struct vm_area_struct *vma,= pgprot_t vm_page_prot); int mm_take_all_locks(struct mm_struct *mm); void mm_drop_all_locks(struct mm_struct *mm); =20 +unsigned long __mmap_region(struct file *file, unsigned long addr, + unsigned long len, vm_flags_t vm_flags, unsigned long pgoff, + struct list_head *uf); + static inline bool vma_wants_manual_pte_write_upgrade(struct vm_area_struc= t *vma) { /* diff --git a/mm/vma_internal.h b/mm/vma_internal.h index b930ab12a587..fc5f172a36bd 100644 --- a/mm/vma_internal.h +++ b/mm/vma_internal.h @@ -17,8 +17,10 @@ #include #include #include +#include #include #include +#include #include #include #include @@ -32,11 +34,14 @@ #include #include #include +#include #include #include #include #include #include +#include +#include #include #include #include --=20 2.47.0 From nobody Mon Nov 25 22:30:55 2024 Received: from mx0b-00069f02.pphosted.com (mx0b-00069f02.pphosted.com [205.220.177.32]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7A00D1FDFBD for ; Wed, 23 Oct 2024 20:39:27 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=205.220.177.32 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1729715969; cv=fail; b=uFImR0Pd+W2tqyFrOin3o1GgRGGdL6l1j3G/opE8eLuqok/tW7GF1TX/9leXhA5FTWNznwd0G4LK4oDuG3BIL7P63K1ksSBT2EAV3ZeWxwIS2h64ypInZYEmsLFremss/QhoYcBIpxW8Mkf0ejfs1YPb3VvYsD7oW6GDcfzrbP0= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1729715969; c=relaxed/simple; bh=5jKXsmd8IKys0YPguzxwU0FnHF0EyFftdn3PFcECWgc=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: Content-Type:MIME-Version; b=K+tDSDDLnBIZ5OA03BMu4W/Gpr7DeaFm8UAdztPPT0UJiYZMGKClK0nRNJ+qM7JaziN2Grem8lP/1D8c8TTFepUEkhyiOqRKUQm6v+7Gx7wIuZWWttfC7y1Prg46owMTnf0Msw8jbPnJesvDCNi9Z/90K3oTYeshLPT55zTUyQg= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=oracle.com; spf=pass smtp.mailfrom=oracle.com; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b=P7wUYVpc; dkim=pass (1024-bit key) header.d=oracle.onmicrosoft.com header.i=@oracle.onmicrosoft.com header.b=aSeU5KxM; arc=fail smtp.client-ip=205.220.177.32 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=oracle.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=oracle.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b="P7wUYVpc"; dkim=pass (1024-bit key) header.d=oracle.onmicrosoft.com header.i=@oracle.onmicrosoft.com header.b="aSeU5KxM" Received: from pps.filterd (m0246631.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 49NKcSvA025102; Wed, 23 Oct 2024 20:39:18 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=cc :content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s= corp-2023-11-20; bh=vHCZVUqc1OUCr81cPgBjdxfmyaJ7aPEXNpHoog10F9s=; b= P7wUYVpc+GNTuQiiEwHeiZn8qnZCiF9c5Rdl6vecgUhjY3TVMG6iaNfJSZ2F38VH BOUKXAB0to8774cVS8k9joaKAIfwpjj6SddEOfHggRe4+Ic8+Q4Ut3Z4/LgVNVuM Vh69eDdFE+tb1H+NNn+luE1i+hajunk6rIYWwCWxq/qLpIvP1y8bql5g3LGGOdGu 018vIggzJeRBejDN2YsIqUkD3T3pLZ/D5FlwoAmwqlbX78ZJQKXRDw9G67dXfZNp 1vD5UTdNYZXVOjsS56WvBmFghf1MikvDLw3mxBXFkwxCeVNm5JdF5lKKxggY8EhU WURzoezomfsyIMVWkCUotg== Received: from phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta02.appoci.oracle.com [147.154.114.232]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 42c57qh5dh-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 23 Oct 2024 20:39:18 +0000 (GMT) Received: from pps.filterd (phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (8.18.1.2/8.18.1.2) with ESMTP id 49NJKtmU025386; Wed, 23 Oct 2024 20:39:17 GMT Received: from nam10-dm6-obe.outbound.protection.outlook.com (mail-dm6nam10lp2047.outbound.protection.outlook.com [104.47.58.47]) by phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTPS id 42emha1at9-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 23 Oct 2024 20:39:17 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=s/7paXTJIYpvUJ5CQ15I+7Th0MbvZOBUmJsnqS7NwHsTZ5O04KlnQpiwg9Le/jLKSkf+1L2PzLkEWUlWumahE1PDVT9/m+lUiQO5o2jYEKh9zSAOsmcIAn8SYcVVyweD49FZFZ9Bn56o/kZhAlO6iO+Qaxr719j9TzsN1eIc3QNXMuL09qTyCAoNJ3/L8uW+RLRXd+1x4Tv4I3Ury41Me+tAZK50DXhJ//XTRmUFaMC11LCUM8bMHdeMmtD0UFfyiTyLUNx5Zc9Saudt9VTS5m1/wbC1wJ8UoQWIw6xkJHskmVTLL54bn3cT72WmtMnOV2AFnHiGX5pUmmjru8ZU2w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=vHCZVUqc1OUCr81cPgBjdxfmyaJ7aPEXNpHoog10F9s=; b=l9jukNyjFBl1AmFBMijPCTXSIuY5q6ik3ugsGeTKOjGwFlvWpHhsFNbqH+FhtS/W+7fGTUmIvus2Jg7Wa2hTpf/hghBII7dDxw1TanUWjGd0vYO0bLZY5qcNKeA8ig3S/TUbWFDmfulzxp4R7edFb8/Fdtj//H4IQr6SY4/tkP8MkdpZGPH28Fx3Mtd5O7G9Xz8siJrjgbhImb5sNU5A6rTpt5n2G/bGLvbXidnlAY+STUkSQ4k/OGFPW9b+Sk7/Rq3kcxpFxvkIFpJrdUDw2ESuvP8eMBUvYKPAcAgaH7p3Qxc7dZIuBESQEOG8dPtbBEPouu0VIfnS3WONMwcASg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=vHCZVUqc1OUCr81cPgBjdxfmyaJ7aPEXNpHoog10F9s=; b=aSeU5KxM0qsk1TDAtN7elBVri98OSH1SbhHkLz9hq9U3baGvQ9SkQDBKqMMYDXpwMSUS7Mggm5rplnWPBuRQcVYDJexaJRPAkO8EZ/qHzZknycNwl1XrDL5AOTNUsl0vjLCuIxfL6lXweEVhUQiKrmAN14RZetcCYBslRmdUXIQ= Received: from BYAPR10MB3366.namprd10.prod.outlook.com (2603:10b6:a03:14f::25) by SJ0PR10MB5833.namprd10.prod.outlook.com (2603:10b6:a03:3ed::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8093.17; Wed, 23 Oct 2024 20:39:14 +0000 Received: from BYAPR10MB3366.namprd10.prod.outlook.com ([fe80::baf2:dff1:d471:1c9]) by BYAPR10MB3366.namprd10.prod.outlook.com ([fe80::baf2:dff1:d471:1c9%6]) with mapi id 15.20.8069.024; Wed, 23 Oct 2024 20:39:14 +0000 From: Lorenzo Stoakes To: Andrew Morton Cc: "Liam R . Howlett" , Vlastimil Babka , Jann Horn , linux-kernel@vger.kernel.org, linux-mm@kvack.org, Linus Torvalds , Peter Xu Subject: [PATCH v2 7/8] mm: refactor __mmap_region() Date: Wed, 23 Oct 2024 21:38:32 +0100 Message-ID: <0b1da31b49d47ccb930d36f509d50d04c0422b73.1729715266.git.lorenzo.stoakes@oracle.com> X-Mailer: git-send-email 2.47.0 In-Reply-To: References: Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: LO4P123CA0017.GBRP123.PROD.OUTLOOK.COM (2603:10a6:600:150::22) To BYAPR10MB3366.namprd10.prod.outlook.com (2603:10b6:a03:14f::25) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BYAPR10MB3366:EE_|SJ0PR10MB5833:EE_ X-MS-Office365-Filtering-Correlation-Id: 32a31267-82d1-455a-9d55-08dcf3a2c1e7 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|366016|1800799024|376014|10070799003; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?uARFujKRRbAN7Duma0lB6AwYvt3jkfbmkkAQthcqAyALiSE9ubD+ELRYoxwh?= =?us-ascii?Q?jKWerQyQtn6zcbYVTjFKac+AE50lRUrkF/URsb8jYyEc28xqj4/x1nUzRB5W?= =?us-ascii?Q?V1YpR21ID2WPp3C0oKrWeDWfAatOADIDtPJfpXcJ3k3jJGm4yhHedzgjAhSg?= =?us-ascii?Q?vERzHxDiIWbf4N23gj/kqo+/U/A/FnjQUE21TmXo/JvcbZRQbcHQ7HcYve2D?= =?us-ascii?Q?l63SwU47caX5IBIP4zGL+Wzgv47GWth08/1nqL1HarRwpMUCnIld0eUCNHoE?= =?us-ascii?Q?A2cMPCeW+gi+x3vqR2akiI+RRlebFqXkbAQEhsw3ocSpzRX5zH8u2SH2kRqL?= =?us-ascii?Q?AZ4CD7oSxzcxGHGspAkpzl+5Hoa5Ovef0Gr5XjwV+LJrO4qq06cpxW0DGUVu?= =?us-ascii?Q?SNzQJTOdTo7/Pv1CuCsZ8A1s4cE4qPqrdu9foegncfD1AbfD+jO+misMnh6e?= =?us-ascii?Q?Uw/18wnH9js2iWbSLxBRGbs3Nx4d1WviNFKZbc2kIt1lTRJMs/uByDE+hSaR?= =?us-ascii?Q?sxT0YNDUwKjondn2XoBjEv5ROE3iGBo4kKGfwkrnOF1108IGFW7znoyzSmhP?= =?us-ascii?Q?pUGuL0yT8jcXUSuXxTAQ8jtwVTzy/QmqarPZ6iTWDBWIvgcQx+TxGL4f/Jc0?= =?us-ascii?Q?0EiwrHgMGtAPNSlYI3lw300eYoRfgK7pKZP1K23Jaz1nexhsNG1FAtREL7Ze?= =?us-ascii?Q?UING5WYXxnzvPJkAw5cQ8qlOBONFvjdFjcSvIR7HylC8dwZebTpHaLb4LMrv?= =?us-ascii?Q?6ZE+CJZhmQCODqYEqKJpU9QqJvM2Yywt+RW0lRS7FWVR6gAM+FsFhrp3a7mc?= =?us-ascii?Q?0wUksSb7AmFQZp9HdFf99drHZmoxwnKl03Gr/Roev8fSZ2UKHVLkIRAoel05?= =?us-ascii?Q?vmPx5D5XrYQGx464daMLVSkIIdpsIQj8oz0HfaytZD011dm8NtSvPf/bMW6A?= =?us-ascii?Q?wpnDI1aSXUFMx5Ai4gfSi+etsXDSzMH6XxyC3tiJm3005mDCopC/KRnrOKmg?= =?us-ascii?Q?t3mP5jvlj2ca86WvbxmZEuVipfoRiM8UyHrWIIaaz/6yHvFzMvoNOKGXqyyG?= =?us-ascii?Q?3EG4r0SAKJcjbAQ7i13kwKNUzc7RAN5UJcW4fKpO7RoTD2y2ub38X0U3blHQ?= =?us-ascii?Q?k9LFEbxX2mKiy9FKMsKvrgTmYO7/jqM821BoNwgb+XHZ2eziVvgFRF/4zH6G?= =?us-ascii?Q?9To35PzEA3IfPCdesEwwdbGIU/VFBGr1jJVyefrOrzfhBZ9M8d85USb5r4dT?= =?us-ascii?Q?TC80sOFezwN86g3k3Xz5OsZp7rW5k212tKKfeQKGPnDh3Asfq0kRjpE2RYbI?= =?us-ascii?Q?wvvgGYIYIBo1GRfXGNQ6QKod?= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BYAPR10MB3366.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(366016)(1800799024)(376014)(10070799003);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?Ru2HcCFNLNwVzbo3C81gm8DttFacSl48iTH7TkBQD2VvvETvDbrdOtm8jbK2?= =?us-ascii?Q?G3aStzj6sDoSzUXm+cZ37iax7za0hiMYTq7kmfJ5h8yCnwFhdvLUXaCffLBf?= =?us-ascii?Q?TK7fgDZ29etazyiEThkJeRLAGuyz6Ziizjp3QPkmrzTn3g1hk+KBD8OhUcZ8?= =?us-ascii?Q?5S5vzg1HhZnt/4sfzdDiHguXgrrdZSrP2KgudBD1xn/uXrbzUqPXDAo45NJk?= =?us-ascii?Q?vnhjNAyBb3+0U1ZQ5y6pHswwm9UUPISoUX73y5osno0o00chBReEQyFwaFYl?= =?us-ascii?Q?GuGteV/SLOktWEqxVWJo00ONK6zmOyjRSKrS0fw8nhzmSDA/e7MyE5Q/nFAG?= =?us-ascii?Q?Q8kjkHND4XTG2QQTlpNCV/rRpJu3//vsFXi8lrP7+YFHvCdRFNzHzyqo98yA?= =?us-ascii?Q?ge6VBFrw4nvmZDChyvnj9ASD5uC5vcKyHLMlIzug0WgBF3PoAgahsT5FZUbB?= =?us-ascii?Q?fcSkD2kNq5pywdp971OqfHuCCE/fACC7ZpWuhlab0RxNkw7X0L8rKmpJ2MqW?= =?us-ascii?Q?mML/wSCp9rYhLcO9W3/GQ0eqOgxv4h02Mr3ZjuMcuGF7i/f4o4DMfx7EGQpg?= =?us-ascii?Q?/9t5PKQxADTVLuMx7BSSgQjrkdvNPCbeK2h/vzaJQKhOTKdBJdSLPtgoWemo?= =?us-ascii?Q?7ytC2Gv/MrjJbK1JqVYTfzoB1FHnHZM8vE0m7Ap+nhVm5ZqWrzIWEU+aF1EE?= =?us-ascii?Q?o1CqqEEmdon7y5iWcqFV+epjWGVUxK4D+7FJh8vwLFo8J076I06XsMRzFXU7?= =?us-ascii?Q?c2u/uO3ZrDyQXFcW04EIqNtTPhTVHRsDb8Y4QgFJadQ0Y2YnZRpJR5vOcvLI?= =?us-ascii?Q?TD7tbXLRqjqYlHt+3XFPFMu8OUzl474zv6ON2KVWLf7yAoQB5TT3Pm70HKHL?= =?us-ascii?Q?MRiqKA+Fcjw/37dJBDAY1PhToZY/KRWZas65V7yApvMVfJN/0O9fxKgMHND3?= =?us-ascii?Q?ar/iZWecI2V6Q1nnCuJ5UWp5DDc6maW0MUtD/sGUFlK35q3wRL8GlPSDroYa?= =?us-ascii?Q?ZCEZUTldPmwKciHt0klop/3TUgLz0XEberXEeeD47anIhfjvgpEGJSR2Wybz?= =?us-ascii?Q?ostqzhJxblY6XxjEiYBb32vlyTLL9/t319lz6oThdQPBsSOaGWV9k8XY6Iyf?= =?us-ascii?Q?ziyqD8Z4/4se0IhQnc1kwSs6TJXK8mjpHQrIndceFMctcVcd7cF2Q8xToyMc?= =?us-ascii?Q?SFMdSmNNiwLHTF5nhG3B3t6s7wwHdxnWZ6DMQoFIcIWlaIVk0IAKfzq+AfcZ?= =?us-ascii?Q?y6M4W1C0vJdQ7t0Ye6212mPlMnJA0uf7EVTP4yfRJja57z0SPttCURQWbWkr?= =?us-ascii?Q?QQP2nkkhhWmdKCUeb+Ku++PF1pERePNad0OldCaiwQWkcfXL4WZsegBymk7r?= =?us-ascii?Q?wsVqSG3E/2sZzP0ssbMVBKhQz/XKLN4bXJU5SH/q5QWUbyykjamGjn6KaVC7?= =?us-ascii?Q?bCJ0ducIkogkYwurZ4SnTkAZ6Qz6Y57vZnTWgi+43A/au1vcWgO6qKtb7nJK?= =?us-ascii?Q?xhkwr6zZGRgbtPZM7jXfvq3l9Fa7IAmoeCoimhDMH/aqC7f4WjKLFcv1MfSb?= =?us-ascii?Q?b71odnJscGctvoVtrBlrQFo9eaxWgOzBaoNjIBMn9TmJN8t2mfy+zKpPvn+u?= =?us-ascii?Q?hKkIlyerPbdNkolWCjjobOyvqXD20ui7fFv+PBg2UiBn1c4KegrdPKdZh5d4?= =?us-ascii?Q?bQCuFg=3D=3D?= X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: iwBiPogV/6hg07Mqq4QM0M1jkIzKR+e1hPWxsnIAMx35O4Yd/ozGYQCOqkcoiU6+YgJ9aZzcah3EpYIpkSJ+GR4vYrfoZHOk/kqgagzapozNNuDeDf1nnLDWYOZ1eGmecsVpsG63Vrla9wrNWEWajji4xOZuWICvCqq0q04u2dN4uweHaI6PBCNNCK9lW31OQKjWH2oF02LMIwSbDxAfzgZawqbB5ccLc+Rkfs2rUjJi7bit8UFi434cl0EhCXV3u1BaPeb9/kJ87qJLVpkXgUqQtSEHck4pLlmLIcRTWYx4HzLdKHCuuPmLMClMMgeVi/tDeg26Zwe/SWsy9KH/sC9HR1IGIqSGo73+NJYKDAAzd+Slw31hfEYd9nxtezUy/LCINvEIG6S5LsLav44SuN0S7ZllD4D1paMVUlNz4pj83/lV5+DDQQswfDG5DtuLnpgJsbbRzbVlr8+xbMthnu7oYbGPHW7jBU8JB9BqceLUB5STc2KsGiXyAqm+bUOH9+HJJrxyhTCURvXkIDYWPFR1ApqRiuZi3D00n3SxwmgnyzAnv1uns9uJ1FjykNrBiH0osesFQfmkqAgeURCysgVn55jPjwfEfWHDALFxomg= X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: 32a31267-82d1-455a-9d55-08dcf3a2c1e7 X-MS-Exchange-CrossTenant-AuthSource: BYAPR10MB3366.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 23 Oct 2024 20:39:14.4229 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: iMDX2IZYFu4d5iSGMtEyJYMFVSGDRUqKKt6b5qlYTdC87p6eBqOagXmrqDXQknb4h9W273BhoJEAV3zk6n+FYYX3mQjxVmzbYQS++7rwRdA= X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ0PR10MB5833 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1051,Hydra:6.0.680,FMLib:17.12.62.30 definitions=2024-10-23_16,2024-10-23_01,2024-09-30_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 malwarescore=0 spamscore=0 adultscore=0 suspectscore=0 mlxscore=0 bulkscore=0 mlxlogscore=999 phishscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2409260000 definitions=main-2410230132 X-Proofpoint-ORIG-GUID: CBlrgsLu13UUVspl6ZZk8r7P3AeOqAD4 X-Proofpoint-GUID: CBlrgsLu13UUVspl6ZZk8r7P3AeOqAD4 Content-Type: text/plain; charset="utf-8" We have seen bugs and resource leaks arise from the complexity of the __mmap_region() function. This, and the generally deeply fragile error handling logic and complexity which makes understanding the function difficult make it highly desirable to refactor it into something readable. Achieve this by separating the function into smaller logical parts which are easier to understand and follow, and which importantly very significantly simplify the error handling. Note that we now call vms_abort_munmap_vmas() in more error paths than we used to, however in cases where no abort need occur, vms->nr_pages will be equal to zero and we simply exit this function without doing more than we would have done previously. Importantly, the invocation of the driver mmap hook via mmap_file() now has very simple and obvious handling (this was previously the most problematic part of the mmap() operation). Signed-off-by: Lorenzo Stoakes Reviewed-by: Vlastimil Babka --- mm/vma.c | 401 ++++++++++++++++++++++++++++++++++++------------------- 1 file changed, 261 insertions(+), 140 deletions(-) diff --git a/mm/vma.c b/mm/vma.c index 0a2965be582d..065f5e1f65be 100644 --- a/mm/vma.c +++ b/mm/vma.c @@ -7,6 +7,40 @@ #include "vma_internal.h" #include "vma.h" =20 +struct mmap_state { + struct mm_struct *mm; + struct vma_iterator *vmi; + + unsigned long addr; + unsigned long end; + pgoff_t pgoff; + unsigned long pglen; + unsigned long flags; + struct file *file; + + unsigned long charged; + + struct vm_area_struct *prev; + struct vm_area_struct *next; + + /* Unmapping state. */ + struct vma_munmap_struct vms; + struct ma_state mas_detach; + struct maple_tree mt_detach; +}; + +#define MMAP_STATE(name, mm_, vmi_, addr_, len_, pgoff_, flags_, file_) \ + struct mmap_state name =3D { \ + .mm =3D mm_, \ + .vmi =3D vmi_, \ + .addr =3D addr_, \ + .end =3D addr + len, \ + .pgoff =3D pgoff_, \ + .pglen =3D PHYS_PFN(len_), \ + .flags =3D flags_, \ + .file =3D file_, \ + } + static inline bool is_mergeable_vma(struct vma_merge_struct *vmg, bool mer= ge_next) { struct vm_area_struct *vma =3D merge_next ? vmg->next : vmg->prev; @@ -2169,188 +2203,259 @@ static void vms_abort_munmap_vmas(struct vma_munm= ap_struct *vms, vms_complete_munmap_vmas(vms, mas_detach); } =20 -unsigned long __mmap_region(struct file *file, unsigned long addr, - unsigned long len, vm_flags_t vm_flags, unsigned long pgoff, - struct list_head *uf) +/* + * __mmap_prepare() - Prepare to gather any overlapping VMAs that need to = be + * unmapped once the map operation is completed, check limits, account map= ping + * and clean up any pre-existing VMAs. + * + * @map: Mapping state. + * @vmg: VMA merge state. + * @uf: Userfaultfd context list. + * + * Returns: 0 on success, error code otherwise. + */ +static int __mmap_prepare(struct mmap_state *map, struct vma_merge_struct = *vmg, + struct list_head *uf) { - struct mm_struct *mm =3D current->mm; - struct vm_area_struct *vma =3D NULL; - pgoff_t pglen =3D PHYS_PFN(len); - unsigned long charged =3D 0; - struct vma_munmap_struct vms; - struct ma_state mas_detach; - struct maple_tree mt_detach; - unsigned long end =3D addr + len; int error; - VMA_ITERATOR(vmi, mm, addr); - VMG_STATE(vmg, mm, &vmi, addr, end, vm_flags, pgoff); - - vmg.file =3D file; - /* Find the first overlapping VMA */ - vma =3D vma_find(&vmi, end); - init_vma_munmap(&vms, &vmi, vma, addr, end, uf, /* unlock =3D */ false); - if (vma) { - mt_init_flags(&mt_detach, vmi.mas.tree->ma_flags & MT_FLAGS_LOCK_MASK); - mt_on_stack(mt_detach); - mas_init(&mas_detach, &mt_detach, /* addr =3D */ 0); + struct vma_iterator *vmi =3D map->vmi; + struct vma_munmap_struct *vms =3D &map->vms; + + /* Find the first overlapping VMA and initialise unmap state. */ + vms->vma =3D vma_find(vmi, map->end); + init_vma_munmap(vms, vmi, vms->vma, map->addr, map->end, uf, + /* unlock =3D */ false); + + /* OK, we have overlapping VMAs - prepare to unmap them. */ + if (vms->vma) { + mt_init_flags(&map->mt_detach, + vmi->mas.tree->ma_flags & MT_FLAGS_LOCK_MASK); + mt_on_stack(map->mt_detach); + mas_init(&map->mas_detach, &map->mt_detach, /* addr =3D */ 0); /* Prepare to unmap any existing mapping in the area */ - error =3D vms_gather_munmap_vmas(&vms, &mas_detach); - if (error) - goto gather_failed; + error =3D vms_gather_munmap_vmas(vms, &map->mas_detach); + if (error) { + /* On error VMAs will already have been reattached. */ + vms->nr_pages =3D 0; + return error; + } =20 - vmg.next =3D vms.next; - vmg.prev =3D vms.prev; - vma =3D NULL; + map->next =3D vms->next; + map->prev =3D vms->prev; } else { - vmg.next =3D vma_iter_next_rewind(&vmi, &vmg.prev); + map->next =3D vma_iter_next_rewind(vmi, &map->prev); } =20 + /* Set up vmg for merge attempt. */ + vmg->file =3D map->file; + vmg->prev =3D map->prev; + vmg->next =3D map->next; + /* Check against address space limit. */ - if (!may_expand_vm(mm, vm_flags, pglen - vms.nr_pages)) { - error =3D -ENOMEM; - goto abort_munmap; - } + if (!may_expand_vm(map->mm, map->flags, map->pglen - vms->nr_pages)) + return -ENOMEM; =20 - /* - * Private writable mapping: check memory availability - */ - if (accountable_mapping(file, vm_flags)) { - charged =3D pglen; - charged -=3D vms.nr_accounted; - if (charged) { - error =3D security_vm_enough_memory_mm(mm, charged); + /* Private writable mapping: check memory availability. */ + if (accountable_mapping(map->file, map->flags)) { + map->charged =3D map->pglen; + map->charged -=3D vms->nr_accounted; + if (map->charged) { + error =3D security_vm_enough_memory_mm(map->mm, map->charged); if (error) - goto abort_munmap; + return error; } =20 - vms.nr_accounted =3D 0; - vm_flags |=3D VM_ACCOUNT; - vmg.flags =3D vm_flags; + vms->nr_accounted =3D 0; + map->flags |=3D VM_ACCOUNT; } =20 /* - * clear PTEs while the vma is still in the tree so that rmap + * Clear PTEs while the vma is still in the tree so that rmap * cannot race with the freeing later in the truncate scenario. * This is also needed for mmap_file(), which is why vm_ops * close function is called. */ - vms_clean_up_area(&vms, &mas_detach); - vma =3D vma_merge_new_range(&vmg); - if (vma) - goto expanded; + vms_clean_up_area(vms, &map->mas_detach); + + return 0; +} + +static int __mmap_new_file_vma(struct mmap_state *map, + struct vma_merge_struct *vmg, + struct vm_area_struct **vmap, bool *mergedp) +{ + struct vma_iterator *vmi =3D map->vmi; + struct vm_area_struct *vma =3D *vmap; + int error; + + vma->vm_file =3D get_file(map->file); + error =3D mmap_file(vma->vm_file, vma); + if (error) { + fput(vma->vm_file); + vma->vm_file =3D NULL; + + vma_iter_set(vmi, vma->vm_end); + /* Undo any partial mapping done by a device driver. */ + unmap_region(&vmi->mas, vma, map->prev, map->next); + + return error; + } + + /* Drivers cannot alter the address of the VMA. */ + WARN_ON_ONCE(map->addr !=3D vma->vm_start); + /* + * Drivers should not permit writability when previously it was + * disallowed. + */ + VM_WARN_ON_ONCE(map->flags !=3D vma->vm_flags && + !(map->flags & VM_MAYWRITE) && + (vma->vm_flags & VM_MAYWRITE)); + + vma_iter_config(vmi, map->addr, map->end); + /* + * If flags changed after mmap_file(), we should try merge + * vma again as we may succeed this time. + */ + if (unlikely(map->flags !=3D vma->vm_flags && map->prev)) { + struct vm_area_struct *merge; + + vmg->flags =3D vma->vm_flags; + /* If this fails, state is reset ready for a reattempt. */ + merge =3D vma_merge_new_range(vmg); + + if (merge) { + /* + * ->mmap() can change vma->vm_file and fput + * the original file. So fput the vma->vm_file + * here or we would add an extra fput for file + * and cause general protection fault + * ultimately. + */ + fput(vma->vm_file); + vm_area_free(vma); + vma =3D merge; + *mergedp =3D true; + } else { + vma_iter_config(vmi, map->addr, map->end); + } + } + + map->flags =3D vma->vm_flags; + *vmap =3D vma; + return 0; +} + +/* + * __mmap_new_vma() - Allocate a new VMA for the region, as merging was not + * possible. + * + * An exception to this is if the mapping is file-backed, and the underlyi= ng + * driver changes the VMA flags, permitting a subsequent merge of the VMA,= in + * which case the returned VMA is one that was merged on a second attempt. + * + * @map: Mapping state. + * @vmg: VMA merge state. + * @vmap: Output pointer for the new VMA. + * + * Returns: Zero on success, or an error. + */ +static int __mmap_new_vma(struct mmap_state *map, struct vma_merge_struct = *vmg, + struct vm_area_struct **vmap) +{ + struct vma_iterator *vmi =3D map->vmi; + int error =3D 0; + bool merged =3D false; + struct vm_area_struct *vma; + /* * Determine the object being mapped and call the appropriate * specific mapper. the address has already been validated, but * not unmapped, but the maps are removed from the list. */ - vma =3D vm_area_alloc(mm); - if (!vma) { - error =3D -ENOMEM; - goto unacct_error; - } + vma =3D vm_area_alloc(map->mm); + if (!vma) + return -ENOMEM; =20 - vma_iter_config(&vmi, addr, end); - vma_set_range(vma, addr, end, pgoff); - vm_flags_init(vma, vm_flags); - vma->vm_page_prot =3D vm_get_page_prot(vm_flags); + vma_iter_config(vmi, map->addr, map->end); + vma_set_range(vma, map->addr, map->end, map->pgoff); + vm_flags_init(vma, map->flags); + vma->vm_page_prot =3D vm_get_page_prot(map->flags); =20 - if (vma_iter_prealloc(&vmi, vma)) { + if (vma_iter_prealloc(vmi, vma)) { error =3D -ENOMEM; goto free_vma; } =20 - if (file) { - vma->vm_file =3D get_file(file); - error =3D mmap_file(file, vma); - if (error) - goto unmap_and_free_file_vma; - - /* Drivers cannot alter the address of the VMA. */ - WARN_ON_ONCE(addr !=3D vma->vm_start); - /* - * Drivers should not permit writability when previously it was - * disallowed. - */ - VM_WARN_ON_ONCE(vm_flags !=3D vma->vm_flags && - !(vm_flags & VM_MAYWRITE) && - (vma->vm_flags & VM_MAYWRITE)); - - vma_iter_config(&vmi, addr, end); - /* - * If vm_flags changed after mmap_file(), we should try merge - * vma again as we may succeed this time. - */ - if (unlikely(vm_flags !=3D vma->vm_flags && vmg.prev)) { - struct vm_area_struct *merge; - - vmg.flags =3D vma->vm_flags; - /* If this fails, state is reset ready for a reattempt. */ - merge =3D vma_merge_new_range(&vmg); - - if (merge) { - /* - * ->mmap() can change vma->vm_file and fput - * the original file. So fput the vma->vm_file - * here or we would add an extra fput for file - * and cause general protection fault - * ultimately. - */ - fput(vma->vm_file); - vm_area_free(vma); - vma =3D merge; - /* Update vm_flags to pick up the change. */ - vm_flags =3D vma->vm_flags; - goto file_expanded; - } - vma_iter_config(&vmi, addr, end); - } - - vm_flags =3D vma->vm_flags; - } else if (vm_flags & VM_SHARED) { + if (map->file) + error =3D __mmap_new_file_vma(map, vmg, &vma, &merged); + else if (map->flags & VM_SHARED) error =3D shmem_zero_setup(vma); - if (error) - goto free_iter_vma; - } else { + else vma_set_anonymous(vma); - } + + if (error) + goto free_iter_vma; + + if (merged) + goto file_expanded; =20 #ifdef CONFIG_SPARC64 /* TODO: Fix SPARC ADI! */ - WARN_ON_ONCE(!arch_validate_flags(vm_flags)); + WARN_ON_ONCE(!arch_validate_flags(map->flags)); #endif =20 /* Lock the VMA since it is modified after insertion into VMA tree */ vma_start_write(vma); - vma_iter_store(&vmi, vma); - mm->map_count++; + vma_iter_store(vmi, vma); + map->mm->map_count++; vma_link_file(vma); =20 /* * vma_merge_new_range() calls khugepaged_enter_vma() too, the below * call covers the non-merge case. */ - khugepaged_enter_vma(vma, vma->vm_flags); + khugepaged_enter_vma(vma, map->flags); =20 file_expanded: - file =3D vma->vm_file; ksm_add_vma(vma); -expanded: + *vmap =3D vma; + return 0; + +free_iter_vma: + vma_iter_free(vmi); +free_vma: + vm_area_free(vma); + return error; +} + +/* + * __mmap_complete() - Unmap any VMAs we overlap, account memory mapping + * statistics, handle locking and finalise the VMA. + * + * @map: Mapping state. + * @vma: Merged or newly allocated VMA for the mmap()'d region. + */ +static void __mmap_complete(struct mmap_state *map, struct vm_area_struct = *vma) +{ + struct mm_struct *mm =3D map->mm; + unsigned long vm_flags =3D vma->vm_flags; + perf_event_mmap(vma); =20 - /* Unmap any existing mapping in the area */ - vms_complete_munmap_vmas(&vms, &mas_detach); + /* Unmap any existing mapping in the area. */ + vms_complete_munmap_vmas(&map->vms, &map->mas_detach); =20 - vm_stat_account(mm, vm_flags, pglen); + vm_stat_account(mm, vma->vm_flags, map->pglen); if (vm_flags & VM_LOCKED) { if ((vm_flags & VM_SPECIAL) || vma_is_dax(vma) || is_vm_hugetlb_page(vma) || - vma =3D=3D get_gate_vma(current->mm)) + vma =3D=3D get_gate_vma(mm)) vm_flags_clear(vma, VM_LOCKED_MASK); else - mm->locked_vm +=3D pglen; + mm->locked_vm +=3D map->pglen; } =20 - if (file) + if (vma->vm_file) uprobe_mmap(vma); =20 /* @@ -2363,26 +2468,42 @@ unsigned long __mmap_region(struct file *file, unsi= gned long addr, vm_flags_set(vma, VM_SOFTDIRTY); =20 vma_set_page_prot(vma); +} =20 - return addr; +unsigned long __mmap_region(struct file *file, unsigned long addr, + unsigned long len, vm_flags_t vm_flags, unsigned long pgoff, + struct list_head *uf) +{ + struct mm_struct *mm =3D current->mm; + struct vm_area_struct *vma; + int error; + VMA_ITERATOR(vmi, mm, addr); + VMG_STATE(vmg, mm, &vmi, addr, addr + len, vm_flags, pgoff); + MMAP_STATE(map, mm, &vmi, addr, len, pgoff, vm_flags, file); =20 -unmap_and_free_file_vma: - fput(vma->vm_file); - vma->vm_file =3D NULL; + error =3D __mmap_prepare(&map, &vmg, uf); + if (error) + goto abort_munmap; =20 - vma_iter_set(&vmi, vma->vm_end); - /* Undo any partial mapping done by a device driver. */ - unmap_region(&vmi.mas, vma, vmg.prev, vmg.next); -free_iter_vma: - vma_iter_free(&vmi); -free_vma: - vm_area_free(vma); -unacct_error: - if (charged) - vm_unacct_memory(charged); + /* Attempt to merge with adjacent VMAs... */ + vmg.flags =3D map.flags; + vma =3D vma_merge_new_range(&vmg); + if (!vma) { + /* ...but if we can't, allocate a new VMA. */ + error =3D __mmap_new_vma(&map, &vmg, &vma); + if (error) + goto unacct_error; + } + + __mmap_complete(&map, vma); =20 + return addr; + + /* Accounting was done by __mmap_prepare(). */ +unacct_error: + if (map.charged) + vm_unacct_memory(map.charged); abort_munmap: - vms_abort_munmap_vmas(&vms, &mas_detach); -gather_failed: + vms_abort_munmap_vmas(&map.vms, &map.mas_detach); return error; } --=20 2.47.0 From nobody Mon Nov 25 22:30:55 2024 Received: from mx0b-00069f02.pphosted.com (mx0b-00069f02.pphosted.com [205.220.177.32]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 645AF1FE101 for ; Wed, 23 Oct 2024 20:39:31 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=205.220.177.32 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1729715973; cv=fail; b=KHwEZqOsKIlANSMULlu24nskPSCHQMjsyos+n2IeGwG4kcEbjNe1BfqP2jMOm5PH4z08rtx1qPw3OH6jUuc6CZnoam4pAW+kbSSmCl6PeyYW3wyoyNL9saxPrvMxmEgIQWuDDsgicrq4b+HbiIcn+gieBOSlU0n8wVEP5EoBu9Q= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1729715973; c=relaxed/simple; bh=c+eCOGJZ+Zh3ywALXgZ/UOe4ZFcyQZlfqRps5mPLMN4=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: Content-Type:MIME-Version; b=ro4GW1Eynody15SzHofPa1cNfso8H8ji5g7qxqtPFOhqMueiBTnRT/ACR721A0VzpSwusav1w3XI/AyOAs6IZGjU5CRtHX5rIVoiUGdQdoIHo8sTpDsu3HwY74D/e2awJwh42Wgfm2h3jqtdn3qeEGmAOgP4aw9USqcZ+uuTFF8= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=oracle.com; spf=pass smtp.mailfrom=oracle.com; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b=KNSMlhMS; dkim=pass (1024-bit key) header.d=oracle.onmicrosoft.com header.i=@oracle.onmicrosoft.com header.b=GFqBd5jp; arc=fail smtp.client-ip=205.220.177.32 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=oracle.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=oracle.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b="KNSMlhMS"; dkim=pass (1024-bit key) header.d=oracle.onmicrosoft.com header.i=@oracle.onmicrosoft.com header.b="GFqBd5jp" Received: from pps.filterd (m0246630.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 49NKcaOe010483; Wed, 23 Oct 2024 20:39:22 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=cc :content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s= corp-2023-11-20; bh=GwbeFR97uKuCzPgrgBwD5NPAUJTeqMBP0i3dribbKhQ=; b= KNSMlhMSEWAYRqXQBGhaC/RZY5VJ9JQ5PdotPIm4QDqnrk0Wr6cE13dPhNGH7GYl QhfBh953PDn1TJjkheoNQNhBqRIaOJGUERv+UOZ7JXwzeVawAI4ODpVaVgBlxueU +Kwp3D1k45FbAYdYMkL3uyfq0Li7TYVJRgura9o2BpjMIXd0BitvQQK68+dTTJ+S 0Q07fphzlBQoGyxw1i+GjWh9LAyTwKNzPZKWhk+iTRaGEQNdyjSvdSw/Evvmr/6B r0Xj3slVU3gGSfCPNqY5PcxQp2nJ+FzZznqjq5EUICRbeZg7lHEiiYAwTTZ3PImD XzK//tegIRHxzYDJSOR0HA== Received: from iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta01.appoci.oracle.com [130.35.100.223]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 42cqv3g8wg-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 23 Oct 2024 20:39:22 +0000 (GMT) Received: from pps.filterd (iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com (8.18.1.2/8.18.1.2) with ESMTP id 49NJJHjq027526; Wed, 23 Oct 2024 20:39:21 GMT Received: from nam12-mw2-obe.outbound.protection.outlook.com (mail-mw2nam12lp2046.outbound.protection.outlook.com [104.47.66.46]) by iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTPS id 42emh3bkfu-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 23 Oct 2024 20:39:21 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=whSU4dnrtKXW4Rz5+pNN4L6xbV7w81YPF49v9gWPI4TqCMCXeYQpnH+ZT48Fu4TJI7HfG594cd/XQnaMxRwhdq9r974MfXuSTCHJbGJCVcjgNs0aibb1+FOKLVIoGUa54Pp9QJp3bJ//OxL/MfvhAziO+gCKDbK6vfVyGUlf2RmonmGHWq99WB1Z+jCPGrB8vV+gOKxnK6CygnDQH4tUB7HwxQ0A1TMKoxQljfBRptI+xtnf+wwRZYi3uVjVvG/Evg8xyOdrW9OUy1sYtXERFREEUWNvzJF0u3MgDE9KzILm+GFugFYyHDzp7SOlm+I0eQ6facjQeyM2TR4idz9XRQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=GwbeFR97uKuCzPgrgBwD5NPAUJTeqMBP0i3dribbKhQ=; b=qUk8ZtarKbE7li9ykmqO8A6C0WWZZb1OodbTRlpYapon0MS+np31RXqnLm6oMVX+aKwTt8g8NmOMs+edMESbqEoqco+dtmlxFa6hSTpDkd7lKFhbWgScqcLjmWeFjxhHlAyCpfZM4WWL80xKame91af1JKcyUvBreTISe3h/dcgZwLh6DBEpWFb04NjgYFNmD9u8goyiA36HoIaWFpYeouMvK4VwLzxZsHvHuDk0xDEWWxPP15E2BL41N5GQSY+eVlXK4ff2caYI8NbOgk97dJTwqZKBqOnoYHbkgTv57r3juNQ9IrQjg+H3EMzBit0bY4RvVmHK+TFey1yiDZE98w== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=GwbeFR97uKuCzPgrgBwD5NPAUJTeqMBP0i3dribbKhQ=; b=GFqBd5jpj9muymT+SWrnW+fj8ApLyGzO2JEvYT0O2zxJP1n2rQHk7aw1jiJWY8/F6oGqdzn7/Jo2XOsDqu0S+qTyMsBtkN5+RWhTHFZIUDiK7Id0m1rXtlVO9aEkkDjT/H+g4s4u+4yqrugLh9jdL7yFXx9aWmbbKsmJ+dvrwZU= Received: from BYAPR10MB3366.namprd10.prod.outlook.com (2603:10b6:a03:14f::25) by PH8PR10MB6338.namprd10.prod.outlook.com (2603:10b6:510:1cd::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8093.20; Wed, 23 Oct 2024 20:39:18 +0000 Received: from BYAPR10MB3366.namprd10.prod.outlook.com ([fe80::baf2:dff1:d471:1c9]) by BYAPR10MB3366.namprd10.prod.outlook.com ([fe80::baf2:dff1:d471:1c9%6]) with mapi id 15.20.8069.024; Wed, 23 Oct 2024 20:39:18 +0000 From: Lorenzo Stoakes To: Andrew Morton Cc: "Liam R . Howlett" , Vlastimil Babka , Jann Horn , linux-kernel@vger.kernel.org, linux-mm@kvack.org, Linus Torvalds , Peter Xu Subject: [PATCH v2 8/8] mm: defer second attempt at merge on mmap() Date: Wed, 23 Oct 2024 21:38:33 +0100 Message-ID: X-Mailer: git-send-email 2.47.0 In-Reply-To: References: Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: LO2P265CA0269.GBRP265.PROD.OUTLOOK.COM (2603:10a6:600:a1::17) To BYAPR10MB3366.namprd10.prod.outlook.com (2603:10b6:a03:14f::25) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BYAPR10MB3366:EE_|PH8PR10MB6338:EE_ X-MS-Office365-Filtering-Correlation-Id: c715a4b6-e8fd-4d7a-d582-08dcf3a2c438 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|376014|1800799024|10070799003|366016; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?uYFLPUMO3sryJ9098QEuykuHJR3SNKI7C8dbadX4jtZRmPQSEHBRNkViU2uH?= =?us-ascii?Q?+xfI0jh92l29jdtP/8sPeSWva0AdMKhlu1cshrwqBY6vDtUy3FSsjVWbidP0?= =?us-ascii?Q?fIx+syhjPfVRPwyUHZZK0L8usmIw9PKv3TlHV4ULPGh7UqmhSOB2eENiZ6Hh?= =?us-ascii?Q?qYV3ttwZnuNr5i9B84OkjVuxSCcZQm1u5jjSqvNAjIK6ZO0g6uxfVAfFXdqQ?= =?us-ascii?Q?RbGXQJohvYMuqwBKDHsy6QGJoT2Dgy27A6Zfp7HQOlpktsPVzZKUdMn0qtEE?= =?us-ascii?Q?OWYIGAlQXEyN+oJ+jD8kS0ThiQw16MiJ2IFl4TDDZFIUbCdzu0IHCrVLR6dk?= =?us-ascii?Q?2/3YxdmbaH6xJ9sjabgfwIctEy1mn7lfOXnpk/m85RRz9lwrK8KuP66zxZiP?= =?us-ascii?Q?UgcmgipQ7YseOjzlD0YQLtsSqDZUJJtIzPabDymnRL/++ti5Wn60rvgOBIwu?= =?us-ascii?Q?WXZm5NpsPPfy6Q+bWIl/wEE2neBN8ieoAJg35JhHQKADhvxUm7A3jPX0elKK?= =?us-ascii?Q?lX5OVoprrfBFWZrH9eQvBB+92b5O0YMYu0AfNnFpy+cLD21ZIgJScIkwI5HH?= =?us-ascii?Q?sG7y5oKU2N9+DjB5qW+1ZffnvBq4JfFyTPo4NJuzUHjgKESYbsfGDsP/E4mr?= =?us-ascii?Q?8hU02NrFbkR6O0+qFjYiQ1UNY5WsC5R/XK8ihwNHJccVZZsOGBNmHT/xMKxz?= =?us-ascii?Q?mKLCf3CG+twrj75CZ8IS8/gnXBhh8e7RdMyDQHuSTAT1HWQUCvbo8FPOeEaq?= =?us-ascii?Q?A2fomFdsmzqKzROupV2mO4khW4zTScIc39aMknKACl9k69eUJDxFLES2/WVk?= =?us-ascii?Q?7F2sa56ImQHuVwUpT9YWBro6x2cMQRcmFKDr5RrcuuPNzTlKUnIasv0ZEssQ?= =?us-ascii?Q?xkQe3gKMmmJvDZr3ZdOufiuD850tJjyuL/4/k3k6YLsKZOr899rb4frXzkBO?= =?us-ascii?Q?KkLGjtuC1MOzF7J7cee7SNv5ZQ3/0G/YRNASyQe4Bcic6aQvk1aNVkJ7B2px?= =?us-ascii?Q?L/u9uJKtAa8SmvBP0CqxJ6/JbqOtfjZdkj7543bjea580O5cM0xdesohi/YY?= =?us-ascii?Q?U5YvdtlH0hf/f2jnl/crBDxl6h8iQxwmMHtrCZBiEO81RYrqo7PotEOgn/Zl?= =?us-ascii?Q?+F5KxZJaA/aQ9Sev5Qp1YnwCnYiYZpMrtPguoqKyrTbwqtmoGH3s4VQ0Cqn9?= =?us-ascii?Q?5Lc80HID+4Ox7ZSarr3+VkqAFUHEo4Q2RzzddfxudODjWCYyV2cmsgdlBfsM?= =?us-ascii?Q?7L5AfulO3RAQD69xXJXd1ouyUFeZ3LBhTXT+9cy8q1yDc2uIQ+fxYA8OM62+?= =?us-ascii?Q?5BKPt3kpHPwlfaLWPgWxfIPo?= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BYAPR10MB3366.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(376014)(1800799024)(10070799003)(366016);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?tnRX8wrGTW03TSbkpwMn7wz+U5RDViPCK8PzHZxQDS0Gz9fjgI8DqnV+zxMF?= =?us-ascii?Q?1Pfetm7PveOPDxYE7yWSI/8xo6Hpqrm8xCcurnFrw7jU4L1p0WpTCr7GtpDM?= =?us-ascii?Q?UJ/0Dowjksd822kdYRKUNUcE6J5UEgg3MidyTE+7Prq5V7KwR2Ho9XBBCnFh?= =?us-ascii?Q?Nef+3YFgLbJjfpRaU+vDJyysLLeUslZ2kc5h+nRRBO7RTwrTCOh85cSezy9h?= =?us-ascii?Q?G2Nh67C/as7iW1Q4D80qUh5JWe8pM/iuzv5s845+rum7kiUWs/yOs/3w4UQ1?= =?us-ascii?Q?bXdGJUjWKn/2pCXXbb2rAgFodVpX2xvrN/1aJrgxeHWKcFtpAWHo/0aDJXhv?= =?us-ascii?Q?sDE1y+VB4Z+cEHIbX41qPNLcDiSK2eRylWyc22/uQ/adSGILOYeSFfuXRytS?= =?us-ascii?Q?JlwXgnDKq3lUw739eF3F55PUD9Nu793fSncZ1Ez7DZLw4h7r1fQUbmJeZPJ4?= =?us-ascii?Q?l1BYwOts8Gz7qVYV48820CoAMYjSGT5XTus4IHX8/btqe+WY07SNuwLpNI2s?= =?us-ascii?Q?Pe11Q1cX6hPDxlzz32iLU/ugMCZUEsd1kgsy0vbxXP+YEZhshuXRX5ZO7JA2?= =?us-ascii?Q?c3f6dm4Rm/VLE7SirAqGl/6LeNiC89LwNRqM2UkzfRZJCmOs4nhYYO03Hz1w?= =?us-ascii?Q?3Xjsib7VSZ7lKDl1cJTJiCMjQQa5nWQ4c8k34z/v9YFlZCkRhekqmJMUuVRd?= =?us-ascii?Q?FCwtQDA32UJoTCvbLA3J7YjIW4csB4nzj+8Kf0JWy3JwJ4UVPWgUw8hjvnQ6?= =?us-ascii?Q?bjTg8k6J/Rjx76lvSL/DODinYb6OUlYZDS5egxl9W/vxfsKerIEEWfjDhUqV?= =?us-ascii?Q?5Cb4g1ko7tKROm9dF6exqjxH37fRphfGKR/2uvwttQw2FVlWjgWkAgnZq+g6?= =?us-ascii?Q?s1oTBAidwgEPkaIuvTPkilK4ku8SuwHNrlF+g4+RII4qykUFjNVg5qcML5Jz?= =?us-ascii?Q?VzQwllY7NsSdH2pXqoGdcswisWf4zNzMeSD/OulXCSJTI2eFiciQhOfvQ9U8?= =?us-ascii?Q?SSBdmZdyCId6d9KsCo30bQS4U40NyeDXFEI4svC3v9OTSN3Qbyv1UeWHITeQ?= =?us-ascii?Q?M6Ja9rY5zY4BSipjGN48W0oVqEJHCyk+Hv70dVuinC22pzgWSgUhDyPUIUtG?= =?us-ascii?Q?3VNwIqt4Km/+sQ8RQL1boD/hC3VjQ5gUPjpp3K+nIyXamPwWeihEf4R/Z2c9?= =?us-ascii?Q?/T491rJBiHkjUJZs8rqTX46cmwkDXUnugbo9QUKCg9Be4zZTUaJXb1AFRMfF?= =?us-ascii?Q?6uvQN9Y0YohRrotOYjPG/zu9QJmmocUp22IbxVxtufcCZk3q2rVPFGTLEJN4?= =?us-ascii?Q?vF0emR2oNun3NszkiXPmQtDv7JGSt730tPExbgNJMcBebOpiZYVj7GAKOCdH?= =?us-ascii?Q?yGB7W/0J/gmkhWdnkOkmFc+wcWaT8qnUD50buaK0flNND3r167B4mWWZTamc?= =?us-ascii?Q?HiIHGwJbkVYrYbLezMxdphZNap45oqzBxcxPMGt1pfB8oNZULIZKElLIDFx3?= =?us-ascii?Q?guCVAhz9ptc3mNv79oFmE+KwrDXn3pL9cstKMq1q6EEPgHXJrMKIEffDnD//?= =?us-ascii?Q?Gmu5vXC+K6wu3+2aRiqkOUfpbHHG/nKHZBTX73N0tOM6sD1Srb3V0iHxz2rb?= =?us-ascii?Q?W7oXXnMCxqh5HjVo4vQw4CwjT5jIP1sB4+8upYi5uboiFc3ZZN8YmpbdePRo?= =?us-ascii?Q?d3YQKQ=3D=3D?= X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: 0GjygpMZ85rQ/nCkoK+tPa+puJoRZsDoQGtv7mjiprcUfIOnz9hClfAaSlEQ2jSvg3Kp3D0PmgtzjTqZSaycCJMZhsZpdVpHlN9LpzwOmhQr0qJfg0m8I8aTLxAlyYd60qjRqoC17CZex/XZRITqGKHn7vnILoNF+B7WmefPUfGG6ERi3jOpjbpTteku/5vsabsdDdKmkc5aqqlIyI2uXpjIZM9UkiEJXCtLrwSOe+nVKuN88N9DPecK4sVnS5FOKiMKK4roEOaS9bxMjEhiMPgeTqhMCKfACAKf3osuwfzAS//c7Hy0jFbcxq7N806RzhHJfJ2ju0Yof1pKpHn6eZKqrKSkEkO4eubLNU4mwm2/zW9jmnuz07z6SzRggyPrSE9ruCdrwAPn5AJxgo7rSzYZ9v5VrfjQ6yd+D1prsEUIBQRSWNUMavkh1jgvGbhC+C20jBYlDfqxGkG7ZF3q5C9rzCWcRuBrzxXbRbQwWqrElwmMLbnlitJitLLfqF72I3YPhL5NYRIVw6aXAe5a6YmeJLoa5TmSP9h9zG18NDp/GMi3zqlOUS5KY8IbbyrLBxAF7SXv2IMJW7cCQcXa/1kFaev+Tx1PCKFUTFJBqBI= X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: c715a4b6-e8fd-4d7a-d582-08dcf3a2c438 X-MS-Exchange-CrossTenant-AuthSource: BYAPR10MB3366.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 23 Oct 2024 20:39:18.4376 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: BYrbbkG9MvNzsE2Cb/JXOvn1JSwB+cUlAVQgFNseqOxZCJdpLohFtbAsuvU1qLONmnNQwVRjE1q+4wbIkBlALtZeXFs63VbBcimZeAwQcYc= X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH8PR10MB6338 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1051,Hydra:6.0.680,FMLib:17.12.62.30 definitions=2024-10-23_16,2024-10-23_01,2024-09-30_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=0 phishscore=0 malwarescore=0 adultscore=0 mlxscore=0 mlxlogscore=999 spamscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2409260000 definitions=main-2410230132 X-Proofpoint-ORIG-GUID: YzzipgxMoHitkMQ4xa8d9FIDOn6-IItv X-Proofpoint-GUID: YzzipgxMoHitkMQ4xa8d9FIDOn6-IItv Content-Type: text/plain; charset="utf-8" Rather than trying to merge again when ostensibly allocating a new VMA, instead defer until the VMA is added and attempt to merge the existing range. This way we have no complicated unwinding logic midway through the process of mapping the VMA. In addition this removes limitations on the VMA not being able to be the first in the virtual memory address space which was previously implicitly required. It also performs this merge after the final flag adjustments are performed, something that was not done previously and thus might have prevented possibly valid merges in the past. In theory, for this very same reason, we should unconditionally attempt merge here, however this is likely to have a performance impact so it is better to avoid this given the unlikely outcome of a merge. The vmg state will already have been reset by the first attempt at a merge so we only need to reset the iterator, set the vma and flags and try again. Signed-off-by: Lorenzo Stoakes Reviewed-by: Vlastimil Babka --- mm/vma.c | 75 ++++++++++++++++++++------------------------------------ 1 file changed, 26 insertions(+), 49 deletions(-) diff --git a/mm/vma.c b/mm/vma.c index 065f5e1f65be..c493ecebf394 100644 --- a/mm/vma.c +++ b/mm/vma.c @@ -19,6 +19,7 @@ struct mmap_state { struct file *file; =20 unsigned long charged; + bool retry_merge; =20 struct vm_area_struct *prev; struct vm_area_struct *next; @@ -2280,9 +2281,9 @@ static int __mmap_prepare(struct mmap_state *map, str= uct vma_merge_struct *vmg, return 0; } =20 + static int __mmap_new_file_vma(struct mmap_state *map, - struct vma_merge_struct *vmg, - struct vm_area_struct **vmap, bool *mergedp) + struct vm_area_struct **vmap) { struct vma_iterator *vmi =3D map->vmi; struct vm_area_struct *vma =3D *vmap; @@ -2311,37 +2312,11 @@ static int __mmap_new_file_vma(struct mmap_state *m= ap, !(map->flags & VM_MAYWRITE) && (vma->vm_flags & VM_MAYWRITE)); =20 - vma_iter_config(vmi, map->addr, map->end); - /* - * If flags changed after mmap_file(), we should try merge - * vma again as we may succeed this time. - */ - if (unlikely(map->flags !=3D vma->vm_flags && map->prev)) { - struct vm_area_struct *merge; - - vmg->flags =3D vma->vm_flags; - /* If this fails, state is reset ready for a reattempt. */ - merge =3D vma_merge_new_range(vmg); - - if (merge) { - /* - * ->mmap() can change vma->vm_file and fput - * the original file. So fput the vma->vm_file - * here or we would add an extra fput for file - * and cause general protection fault - * ultimately. - */ - fput(vma->vm_file); - vm_area_free(vma); - vma =3D merge; - *mergedp =3D true; - } else { - vma_iter_config(vmi, map->addr, map->end); - } - } + /* If the flags change (and are mergeable), let's retry later. */ + map->retry_merge =3D vma->vm_flags !=3D map->flags && !(vma->vm_flags & V= M_SPECIAL); =20 + vma_iter_config(vmi, map->addr, map->end); map->flags =3D vma->vm_flags; - *vmap =3D vma; return 0; } =20 @@ -2349,22 +2324,15 @@ static int __mmap_new_file_vma(struct mmap_state *m= ap, * __mmap_new_vma() - Allocate a new VMA for the region, as merging was not * possible. * - * An exception to this is if the mapping is file-backed, and the underlyi= ng - * driver changes the VMA flags, permitting a subsequent merge of the VMA,= in - * which case the returned VMA is one that was merged on a second attempt. - * * @map: Mapping state. - * @vmg: VMA merge state. * @vmap: Output pointer for the new VMA. * * Returns: Zero on success, or an error. */ -static int __mmap_new_vma(struct mmap_state *map, struct vma_merge_struct = *vmg, - struct vm_area_struct **vmap) +static int __mmap_new_vma(struct mmap_state *map, struct vm_area_struct **= vmap) { struct vma_iterator *vmi =3D map->vmi; int error =3D 0; - bool merged =3D false; struct vm_area_struct *vma; =20 /* @@ -2387,7 +2355,7 @@ static int __mmap_new_vma(struct mmap_state *map, str= uct vma_merge_struct *vmg, } =20 if (map->file) - error =3D __mmap_new_file_vma(map, vmg, &vma, &merged); + error =3D __mmap_new_file_vma(map, &vma); else if (map->flags & VM_SHARED) error =3D shmem_zero_setup(vma); else @@ -2396,9 +2364,6 @@ static int __mmap_new_vma(struct mmap_state *map, str= uct vma_merge_struct *vmg, if (error) goto free_iter_vma; =20 - if (merged) - goto file_expanded; - #ifdef CONFIG_SPARC64 /* TODO: Fix SPARC ADI! */ WARN_ON_ONCE(!arch_validate_flags(map->flags)); @@ -2415,8 +2380,6 @@ static int __mmap_new_vma(struct mmap_state *map, str= uct vma_merge_struct *vmg, * call covers the non-merge case. */ khugepaged_enter_vma(vma, map->flags); - -file_expanded: ksm_add_vma(vma); *vmap =3D vma; return 0; @@ -2430,13 +2393,17 @@ static int __mmap_new_vma(struct mmap_state *map, s= truct vma_merge_struct *vmg, =20 /* * __mmap_complete() - Unmap any VMAs we overlap, account memory mapping - * statistics, handle locking and finalise the VMA. + * statistics, handle locking and finalise the VMA, + * attempt a final merge if required. * * @map: Mapping state. * @vma: Merged or newly allocated VMA for the mmap()'d region. + * @vmg: VMA merge state. */ -static void __mmap_complete(struct mmap_state *map, struct vm_area_struct = *vma) +static void __mmap_complete(struct mmap_state *map, struct vm_area_struct = *vma, + struct vma_merge_struct *vmg) { + struct mm_struct *mm =3D map->mm; unsigned long vm_flags =3D vma->vm_flags; =20 @@ -2468,6 +2435,16 @@ static void __mmap_complete(struct mmap_state *map, = struct vm_area_struct *vma) vm_flags_set(vma, VM_SOFTDIRTY); =20 vma_set_page_prot(vma); + + /* OK VMA flags changed in __mmap_new_vma(), try a merge again. */ + if (map->retry_merge) { + vma_iter_config(map->vmi, map->addr, map->end); + vmg->vma =3D vma; + vmg->flags =3D map->flags; + vmg->next =3D NULL; /* Must be set by merge logic. */ + + vma_merge_existing_range(vmg); + } } =20 unsigned long __mmap_region(struct file *file, unsigned long addr, @@ -2490,12 +2467,12 @@ unsigned long __mmap_region(struct file *file, unsi= gned long addr, vma =3D vma_merge_new_range(&vmg); if (!vma) { /* ...but if we can't, allocate a new VMA. */ - error =3D __mmap_new_vma(&map, &vmg, &vma); + error =3D __mmap_new_vma(&map, &vma); if (error) goto unacct_error; } =20 - __mmap_complete(&map, vma); + __mmap_complete(&map, vma, &vmg); =20 return addr; =20 --=20 2.47.0