From nobody Thu Dec 18 00:48:13 2025 Received: from NAM10-BN7-obe.outbound.protection.outlook.com (mail-bn7nam10on2052.outbound.protection.outlook.com [40.107.92.52]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 99E1C1DDD6 for ; Wed, 24 Jul 2024 13:01:26 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.92.52 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1721826088; cv=fail; b=kYsn8imNBzQjYIL15Ze4FAHJLSFQcEfBFDuA0NulrCK9ZZI48xrmV6mBmn21TUdc1isUS2J7OATxAO4zt8uuVffCybEUrSl9zpdWfEaPHQivq3RdTouwYA6BiCA9JfS3kpzLieYJ+CWg3g5m7OTNhi72AwfTYw5rdtq+Pc9J6EE= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1721826088; c=relaxed/simple; bh=4RL53TyXABv+z09SL25+El5InSf0tHke9Q0yZXBGtRU=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: Content-Type:MIME-Version; b=pMOE5OxtQetcrdAGxHZr9UKLVdwJB4mVzQVrpTqiXGcqNWLoMbUN4asvoyqMFg5+9BcFyztw4yaUxKeFMvmNncfkhPYO4DWwte8OUw5Do630QKLt6HUQUwSuEepH2kENYS0trp9nDkjE1njJG8mzn0FW6wmbQTdlkaagwBRYGGw= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=tchEVv18; arc=fail smtp.client-ip=40.107.92.52 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="tchEVv18" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=f9z4ZGbwwXS9S5F/jfmyudOgkHbciwouJoRA9UBNHyMGJHgOag1t3mBSVDbw1zAgsN3jpj+3ypfPbQRnbzIHq+NHW4j5xcgLFvxQUTAERetUbVzP9NPXr3kmDrILa6N5nsACCyL0BYrGbGV6mu08a6Lz69cKZQcdwtZtnM0sdxiQgFmfbNRYVa0D+2aEdJK4RAtzhW9yhewLtwlAJoGXsC4BzJKbAJx9oX7fd6myCQLaox1THurIGxdcygQvcK8NS5AGDSej06MNSRTWj20A9r6UoNCXgNHjA8qeKs3cL0/QAA6KaYA9nikPX3ivAQ8bOkXBALL6aEXXmGtnv9TF9A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=RF4UiEA4wU82ykwfKNn44Zy0CR5GrMNLSl7SIWdKIUg=; b=yolt4R2LgHQOAJsaubsZMxi6ppuEk8m1FucJ7dTBrSDYkWYpJ9JZeOevhNbausPa5vbvEhUXnnPmDVoEcu6J2WlpbUmcw5peOBz41+FQXOcN9nwNhc4ZKpETnMYzOmiUr1aREglRqaGRkyTrIZnL6M4EGnklN/JTxFO5aCqH0YqbaSrJaFyhZwMvrw0msNQC1ue0Ol++8AIztVhAtq4So6LYFY2ttrOFbaLJ/hBftXtpwZ3abmdh1s6NFBTYytE5FlJp8Lsn5NUY4LWv7HMs/l+QcZn/kQMaLiBkX93bv3dETZ5BtKgkMj4ijCCOUbW//a5QlXolLwPxnuVm8MwIuQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=RF4UiEA4wU82ykwfKNn44Zy0CR5GrMNLSl7SIWdKIUg=; b=tchEVv186QOETpCUKilziV2rEPCKlCE3Zlb9yx33o1d1GMK8TPpOwxUEfm+aySITGZy8Fcyo02kBC20XhNG5KqPuHBay5UAhUtj38j9jS509/lAX9/av6eshVNw9hhS+CWHOJ3Gf2KgMZsr3xAvqYiMIJYC8fI9LJzvKcJxn5wbi0562NYf+8tWIKvgElupGE3tCd3ygQfUMuIG8lHyDdiMnJ8CdViy14Lh1uqCwZNURXTRU4ss38eXRXl/TpwfGPik1HpksN84WP4qj/Kk3eZdUUVMMhxeSqQ9A9p7jPh8Ni6yAe5yTiLs526V1/7dqHu+jFg59FT313poKNc4QrA== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; Received: from DS7PR12MB5744.namprd12.prod.outlook.com (2603:10b6:8:73::18) by SJ0PR12MB6709.namprd12.prod.outlook.com (2603:10b6:a03:44a::20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7762.28; Wed, 24 Jul 2024 13:01:20 +0000 Received: from DS7PR12MB5744.namprd12.prod.outlook.com ([fe80::f018:13a9:e165:6b7e]) by DS7PR12MB5744.namprd12.prod.outlook.com ([fe80::f018:13a9:e165:6b7e%4]) with mapi id 15.20.7784.017; Wed, 24 Jul 2024 13:01:20 +0000 From: Zi Yan To: Andrew Morton , linux-mm@kvack.org Cc: David Hildenbrand , "Huang, Ying" , Baolin Wang , Kefeng Wang , Lorenzo Stoakes , linux-kernel@vger.kernel.org, Zi Yan Subject: [PATCH v3 1/3] memory tiering: read last_cpupid correctly in do_huge_pmd_numa_page() Date: Wed, 24 Jul 2024 09:01:13 -0400 Message-ID: <20240724130115.793641-2-ziy@nvidia.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240724130115.793641-1-ziy@nvidia.com> References: <20240724130115.793641-1-ziy@nvidia.com> Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: CH0PR04CA0026.namprd04.prod.outlook.com (2603:10b6:610:76::31) To DS7PR12MB5744.namprd12.prod.outlook.com (2603:10b6:8:73::18) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DS7PR12MB5744:EE_|SJ0PR12MB6709:EE_ X-MS-Office365-Filtering-Correlation-Id: 2f1ca61c-409a-49d5-ee69-08dcabe0b601 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|1800799024|366016|376014; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?4lVY2L5sPY1vBz9ZJcog+R5da7NaPth5CGMN0oPa96S1yTjWzAOk8x9MFe2P?= =?us-ascii?Q?t09Jn6rCxxdEvcP/omH9NeBPebdJJcASeSki8KwMELHn6OCcg8MroyB2miUN?= =?us-ascii?Q?pT90EqABIV2fHF5YTkNppsKIfJYtUWTp6Ir9h5XhRyYobSFNr3ZiqedSSih6?= =?us-ascii?Q?OB+yY6cG/YKDbZxVvdmNAiXNJO8yI8jdIbNUOy4DBiuCgbMJHDMyCizBWxmc?= =?us-ascii?Q?ezAR+5eRvuSPP+cRBTkNjBqlVgBacwIbE+7Jc+h5K5nTuGLmXwYu42PyHpzH?= =?us-ascii?Q?hmo66o/D6fhuMPkT70X2I/qQLi+zX6a/Zv1FyLaZcPzGXmCVac4mN2n5ioyD?= =?us-ascii?Q?q++0R8/GGgZtkkZXronYuEYtAlgwGDW8lFVv/L8+gNSDCOtZqoHnKseehiwH?= =?us-ascii?Q?cIqScRCzzYBCWNcv2/AIf02Q6Newiqb65G4aK1BirUxZQe8cA/4X3HMlrkGi?= =?us-ascii?Q?g1JC3ehhEU6Jxv+aZT0Cpjmhiuwm6PGZzONRFQnKgppGibLGi1jLOcteWozn?= =?us-ascii?Q?lMCv/ABfPoUedP1SNSDW7ZDxLoTxLMzq4IDQ/+Ixbcy27wVFxhtLz7FmgDyR?= =?us-ascii?Q?WydtdKHG8E1EmLUyMvhTnXyAQMkENPNuajVk9Q72j/uvLeRErXfxwrPzm9Hc?= =?us-ascii?Q?BGV+C8TJJNvg/tQee4DHo8Tvr4NBczYaauW+F5WWFBCr2XUw9dspGDckx8kr?= =?us-ascii?Q?au7CDoBUjWuWJJ8UzyCJN4/86qQEDLu2/kgxu+9Xel3blB5Uh3Kgz2+79BGb?= =?us-ascii?Q?LPwRNHTmZjYH4u4y3t+3IEWTNc5QfEJYcPfCeGmeKE1Mn/0iruZun39gmWse?= =?us-ascii?Q?Mtt6pRxfRRPZZbrExuPUBpbXzmfB17108FlnLV6+plIrIk10OHxyZYjY4S1b?= =?us-ascii?Q?eE9tu2hGhaROkglSU0OxvHhXPwkPW6w40L1bU4Ref69CHYCo9Yw6pi1U1fk2?= =?us-ascii?Q?ko371enplLTkpSlK+kVXo6gFmlxh47kf5WWXRJlnw+HSS+HvFxeio+GjKHG1?= =?us-ascii?Q?ZUypD9CK0Y2C8RkAo9qIJECdQkqvXAWziLPJiGnKHZeAHNvdFVkMSkL/pOJ2?= =?us-ascii?Q?KvHYHxXIWJd3Ut17wGkPJTO4h+db3lhcrdleMK8u/8S643M7Wg1X7oLfGLKt?= =?us-ascii?Q?hvc9UO1FxF7clvM+KQdbW7NTpVybBoh35iDDWRspTNVUe8QZYMMzel4WOCs+?= =?us-ascii?Q?Ku/WdZN6Nm+D6JHPM0A6CQUfJKeSho00feS4q37UHCSKl7GNrcea45wWSfM0?= =?us-ascii?Q?FTbtx/cCipdliHMeJUlesUqLecG1+HxwrXLd/BNZ5Nv2dim3xAxKH6UpQytD?= =?us-ascii?Q?HI18xXkdH5N2/QhZgod4LLHqCeVvnJSL2La5PdXyEsdJAA=3D=3D?= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DS7PR12MB5744.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(1800799024)(366016)(376014);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?k+BradzlQMkJcPuGzShRCMifRUJo8E8VQeYwim3XTATTZCxESEU0RslJvtih?= =?us-ascii?Q?nZcZmPrWv6jDT8OttmeF6Fstowlfe+ll917bPcfF+wXS83M2bzG6m8YZIoxW?= =?us-ascii?Q?dT75VA2dGvXSgcdGok9X5XRKDXPTfHVE/6tQf9LuHUplzcwW/9MVUD0iURx0?= =?us-ascii?Q?Hyhqf5ZavxBtshF/MkbttYMvvyAOZEClIb43fi+HShotCkyrGN8adInWUTDn?= =?us-ascii?Q?RXbwj+/GWHMwk2wuiJVS1FdlzHyVajarX7bKi6TC+PZ5UDB+BIofRWitZNMZ?= =?us-ascii?Q?bGUlIAhK95AiPp6JvSJwLNX6RM8jDa3mCeqUhytdvNJLqb7MZF+fY1BxVdiV?= =?us-ascii?Q?bccIc0uBfcEK3fPoVBlI8+QPIhPxK4eOZbr1P7Of3oVz/4p1ufNZWguQmnWW?= =?us-ascii?Q?CEQMV/xNWzSFOShPwRDZzGXexW4H16/1HwI2Oh9dNcKeJe64dYY3946ZWsIX?= =?us-ascii?Q?Dgz8i6hdvIqSJd/9+4zOK1wYvAt1GxSZJS/5ujZRS+fc3acVn0KUOP+2/ZoD?= =?us-ascii?Q?rVfyy+7C6sIsiqWclWTsTXaRgKBxinXKAht8S8dGJF4F/QdFq03diNe/n4AK?= =?us-ascii?Q?BwNXII5toDvinKS62KXH/Uiw7cKAlVlgisYWz3CbbSUZiPxN3j9Q2juDe8qk?= =?us-ascii?Q?i3RvAe9SZg0a/jbSVYpK1dRvYOIMpeqd0S2Tp0JPlzrukWGOfiOfOfk4MoJy?= =?us-ascii?Q?heFMGjMEl29MbRz7hBhiSjl+FRWRPr9eCmSAqErHLNjs7mzmN5wasLpVcdUo?= =?us-ascii?Q?ZMVr/RD52RLToa65sjiTJDRQLTS6zyCjgWZ8Zlx/8x94ign1IoIi6HtJG658?= =?us-ascii?Q?aq4OECtWzZb3NG0AE1VEPqFZPlCzEIorQ+A3eRvIIsZW6wHyDbKz3Pzc+KCC?= =?us-ascii?Q?lQjIzm8iIkcFMR9iG6qf2vfjCZ8RTPLPYfmW9AGAc+eDWWyGKQSIxhGc3316?= =?us-ascii?Q?xqNCNjdkc+TPtLItpRFjQKQWvsJHeZ3vi23Xheb8TWUgDPWFUF7vnySF/Kvq?= =?us-ascii?Q?zLg++SW0Mhzc80o8Il+Yn0jYnRTf7JT38cBpnwvvj2xJswp4HVNWvvpcXXuZ?= =?us-ascii?Q?nu9BWIplXAcjS3B4DYRYsbH5nKCvRnKU048idoueRJjngS0YGB7jgSjszOmm?= =?us-ascii?Q?6Qg14cfkD32LlAU1TLVSL5Dvi2AyAgTvoeBfTwzPJnPQeiSB5rgcF69sZg2G?= =?us-ascii?Q?mmlH4vDRL/w3Vb4aycSfNRKWLsIGy4PPcg9Q2pPuHSXcVxYNzgcUGbSo2Qn8?= =?us-ascii?Q?6mS/2AwxmmPGMoUdOcMn0Rm0lX95cNaJDPpG3VZd3x/WkwD5YCNo7U5VBYSY?= =?us-ascii?Q?V2wT2eKCyqIg14mvG3CVWx1OuuhmRJ2OUHO2AF1py41onQ89ZkcN4VdUbDqO?= =?us-ascii?Q?hC/K6ftwh288xxZ6LhXVX7Hnu2X7XKQ1ffwxAXLqu/f7MAGcSiv2G0NrlcTI?= =?us-ascii?Q?mjimbjQtt5iZUHdp65lIV1ItGFsgKSSS3jBnFEZR0BdXXEpQoa4RTAmSoKcH?= =?us-ascii?Q?Pl1rIM68a2dj1PkT/te9GLjHEDfTHXTVSbJ9xaH/Ct0wUmo+6iHA7PUthiog?= =?us-ascii?Q?vuJf3xFxMpdhFOdNK1k=3D?= X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: 2f1ca61c-409a-49d5-ee69-08dcabe0b601 X-MS-Exchange-CrossTenant-AuthSource: DS7PR12MB5744.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 24 Jul 2024 13:01:19.5805 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 9y6nMqlNhwrk0oEiD/g1cVpkYSBaPTJTKg1G9uvxozmeVdRr2wfQCH407TShuFGM X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ0PR12MB6709 Content-Type: text/plain; charset="utf-8" last_cpupid is only available when memory tiering is off or the folio is in toptier node. Complete the check to read last_cpupid when it is available. Before the fix, the default last_cpupid will be used even if memory tiering mode is turned off at runtime instead of the actual value. This can prevent task_numa_fault() from getting right numa fault stats, but should not cause any crash. User might see performance changes after the fix. Reported-by: David Hildenbrand Closes: https://lore.kernel.org/linux-mm/9af34a6b-ca56-4a64-8aa6-ade65f1092= 88@redhat.com/ Fixes: 33024536bafd ("memory tiering: hot page selection with hint page fau= lt latency") Signed-off-by: Zi Yan Reviewed-by: "Huang, Ying" Reviewed-by: Baolin Wang Acked-by: David Hildenbrand Reviewed-by: Kefeng Wang --- mm/huge_memory.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/mm/huge_memory.c b/mm/huge_memory.c index 88dffc0023b5..15234b2e252e 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -1707,7 +1707,8 @@ vm_fault_t do_huge_pmd_numa_page(struct vm_fault *vmf) * For memory tiering mode, cpupid of slow memory page is used * to record page access time. So use default value. */ - if (node_is_toptier(nid)) + if (!(sysctl_numa_balancing_mode & NUMA_BALANCING_MEMORY_TIERING) || + node_is_toptier(nid)) last_cpupid =3D folio_last_cpupid(folio); target_nid =3D numa_migrate_prep(folio, vmf, haddr, nid, &flags); if (target_nid =3D=3D NUMA_NO_NODE) --=20 2.43.0 From nobody Thu Dec 18 00:48:13 2025 Received: from NAM10-BN7-obe.outbound.protection.outlook.com (mail-bn7nam10on2052.outbound.protection.outlook.com [40.107.92.52]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id AEBD9158D7C for ; Wed, 24 Jul 2024 13:01:30 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.92.52 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1721826092; cv=fail; b=PwTQ10hZ3g/6/yak69s8FPz9eGNON99sok4alsNlTkIL4YdGQoqZLe+3s/tu0Ji1xh7RPsByob3hhWBdlSOpU+iZYa70Up+a4thhC/zrRWKIo1LhdYsuzO8WQMC88pgxAbF/9o3kdl/slBhHt5fwDZlCa6Paw61EI8cRdFtvA64= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1721826092; c=relaxed/simple; bh=hu8kH3FF9LZYABQo4ls3hsYgu2TMYUZ9hPuHSoWZMx0=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: Content-Type:MIME-Version; b=jICJqkqexUTFbj3w45TK8V4zsKvOlnM8rj9cTwXiCA5DDR8Pd7b4W49tls1uKIfOvqEAhb45MEVuGgHyyBDmyUWf4XH8xNRronOliF2xOg+b1Fc2xJe8B2Rhvlvmzk3GXkVb+y8LRJw/D2VV5B3X/w/8NlLxjR12VDTKtZh20Cc= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=qstCRrGE; arc=fail smtp.client-ip=40.107.92.52 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="qstCRrGE" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=iQUX2ghHrM5OKpN6mlllpqbSjxg8AGuU1bv+7m0T7GZGWgOQwpe5419hI5fWhexQkCqYSUMc9Z4zwePSm7Z91Lf/Wij8D+mDONilKyZDnjmBSEauhyYRD4bScgQ8BSjWvzQya4tHKNa/UO5XnnbdsmubbnJV3hP0ne2ABNlK19L1boakZCA2iBOz45RhaoTAvYF7NEmWzIRzMtT48m+AYmLTPwLKa+DdnCYA2GdkJpv2zbYH8vcUE6NZ4RQYMz/9Tqs4sLOgZ/+807ODB2NROvxKBiQxmOLaTGBA5HVwTZbZd6EMGGU1LUeW3kj1b+s3y7r2gZtezGR5sEZECdaVyA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=de3HtV97/3HfU1Wwqz2WEK7UWyyj5aobEIaoBUZtd2I=; b=kh9GUWVzbVpJHzbVLeQM6m7Q89u59cKJA1TKEkJDX9jEA1OPiYKK235KD0mjjfLeP1+prh1NjaLk5N1k4vdh6VmYBwBgTXz8YYxFKs477+AnNY5CQ6AjF00K2QZw1bRMGMC8lcUJJqhs5tuECF8QbiguZ6VQ1OgzE+3DblFLBTgQ31016Ky7cq/EdsGIMyaiCeaohYmvTn+F92Cyv1MiG0Aq0IZs2qCY1g9EIX39ReFYUzDZNci7vObDJzSvEucVelVeKIjw5dSpLRimo8Kylf+35ZGHkzJ6sjRumXzNXJ9Oxi0ogFzTsfYk1tM07Tftwf46Xs1a/BWO+HNbYyQK4w== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=de3HtV97/3HfU1Wwqz2WEK7UWyyj5aobEIaoBUZtd2I=; b=qstCRrGE4Nx6o0ooMqjKn0DTR7PgDBleyJ8YHQUw+b94ShTkW+s7TR2GEPiDgrz7g5nc/7SgRkgt+mQrTlzQsO8IVwRnG9gyO74cJBFXNIxBjKcNWWEuszzyJwM9e4fP491l7R9RQkiEFZAzR5Gw5mcJggdTre4SPzVBvRLGs0599uG0xumkiKlUbePnEoxG+T/8qPXEKn+uC/1hKan+7GgmYs9+WixZj0zzYJxtckZmWrFVNqA7NKtK95uchfg4qypMtGUJJ6HohkMjn1Cu6TZDj7ZzYq09vM1JF/YIjzHupshyoC+s5rw3BOjY0cyJCmXay2+j6bQh1lmDpTTgqQ== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; Received: from DS7PR12MB5744.namprd12.prod.outlook.com (2603:10b6:8:73::18) by SJ0PR12MB6709.namprd12.prod.outlook.com (2603:10b6:a03:44a::20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7762.28; Wed, 24 Jul 2024 13:01:21 +0000 Received: from DS7PR12MB5744.namprd12.prod.outlook.com ([fe80::f018:13a9:e165:6b7e]) by DS7PR12MB5744.namprd12.prod.outlook.com ([fe80::f018:13a9:e165:6b7e%4]) with mapi id 15.20.7784.017; Wed, 24 Jul 2024 13:01:21 +0000 From: Zi Yan To: Andrew Morton , linux-mm@kvack.org Cc: David Hildenbrand , "Huang, Ying" , Baolin Wang , Kefeng Wang , Lorenzo Stoakes , linux-kernel@vger.kernel.org, Zi Yan Subject: [PATCH v3 2/3] memory tiering: introduce folio_use_access_time() check Date: Wed, 24 Jul 2024 09:01:14 -0400 Message-ID: <20240724130115.793641-3-ziy@nvidia.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240724130115.793641-1-ziy@nvidia.com> References: <20240724130115.793641-1-ziy@nvidia.com> Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: CH0PR04CA0007.namprd04.prod.outlook.com (2603:10b6:610:76::12) To DS7PR12MB5744.namprd12.prod.outlook.com (2603:10b6:8:73::18) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DS7PR12MB5744:EE_|SJ0PR12MB6709:EE_ X-MS-Office365-Filtering-Correlation-Id: 82296d42-9d23-40f2-3fe4-08dcabe0b6d5 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|1800799024|366016|376014; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?apb+s7tSbTXjL5nvvoIW4QtKIEZTFkZ57bc7CpbmYIo63CRLAKtkBcffFr0T?= =?us-ascii?Q?3yhdMUPrtzbLXuoUx8fRXvX2OR4YKScJNujVU8N1PV4Y9cwENCQ0Br0Bx9Yf?= =?us-ascii?Q?kZRbzHNH31oqiVFcuq1TwBU3LWZ5YirmJ85/wYsXhP/8pctF/Z2V/zs1MEqR?= =?us-ascii?Q?W5E38SxmC4QJtkgMLLwaSvtWzcV+6cXeNUkdThS/zPJ8mvsAO6qHudGdhjDJ?= =?us-ascii?Q?bD8zKDfAwJyNE5N5swQTPZHkjZL9BidbJJ7JPa7c9GvIMApMAH2bmH14fdh2?= =?us-ascii?Q?Qry+KP1s+5/+i4mRXG24KBi4w/j6QNY43zR3jGYVqArxxT0H2iOQyaxPztgA?= =?us-ascii?Q?2+yd9ooqHJuHlnMtnvqHxFPN5AXJiX7T5bs9Njvqvqm5vm48MtN/rwFstCmV?= =?us-ascii?Q?R+hpqNOIDi5vuZrmXTHyIdAkb1vaB9E0uCCwMOujCsNbXCnQwaqKriOXAxt+?= =?us-ascii?Q?8mNPemmWwQtm7XeAnnH8xkj/BuOZO235OPRr5doGBucoT5agrNIfcV4M92Cu?= =?us-ascii?Q?mSmEtJaMD5DPfVygzRjlNNBqMnzeOKYUDEtQVyh+hRL2g+HcxEOYeRwUMur6?= =?us-ascii?Q?PP3lUH2F4ybd0OFF6p2tbztVzgw1wGnX0mDFM1RCn+1mt5DHGe77byvKmaoI?= =?us-ascii?Q?LapWKV0HPd19jgde8nDVQxNcJAkJ3BEPIsuGATD+Jza8wl68rTvDziFduTOM?= =?us-ascii?Q?YKap7uZHtu7mOUr+NUmwIcA+6fjpLYiplF6W5IoLuln6DbPOLuc9FCLPTruD?= =?us-ascii?Q?q81I5PPbYkM8TnhuoET/zxt+mrbpPWCxqffO/ygIAe7aNzsmOiN00JJAYGF/?= =?us-ascii?Q?iyRYl6KknEBWO0r7QFIktk5M0o5xCgbt5i8xyLfBIRVEG9j2C70qb76E5oYw?= =?us-ascii?Q?e2bmsqZjnwhjDKeITbR5Ma6TxBDa3mYP7/KuPN1LraoCeEznzIGsm+TQmll6?= =?us-ascii?Q?Cu9NJKNOJvIY9+oFpMEXM8Q6E3QqND9BiThOwHGlpBAjlrnZJlBTVJ/xMhrY?= =?us-ascii?Q?0/kix2RXDowBnZhF4+1Be4idDNnN7VqpNCiMr67Yph9LNhGqO7zAuktol5pK?= =?us-ascii?Q?RgQROYB72VBzTWvL7lGLpLAy5CQLpqSpZ20uEJNygEygR//qR1A08z0J6vAK?= =?us-ascii?Q?xHI/gfCygtNlqSGwjp6FGWzucwR2JwObxeqMTA6NWfso3jZOkxLkgO3Qi4Zs?= =?us-ascii?Q?GyvubY8RgP28YBfATRahY1pmDnwOpcEmEnMlaGpAlp1nXPML/Z51jH6p823T?= =?us-ascii?Q?Qm/JY72u62jbTyBK5Aedf0oG7lGnA4cOsBwP8OKvYuX6dYo/AmvuBOFLkO7N?= =?us-ascii?Q?8goAEslJFYDU0Hc8s8K0YpHgHNzLM4VqMmELpe0UKyRMAw=3D=3D?= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DS7PR12MB5744.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(1800799024)(366016)(376014);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?M8E3r/N+6BRIya+hBsfajwiI0IXubOWiFRwJjr59foP3qFoYdEWtyIP2RSFO?= =?us-ascii?Q?dE9eZH2CClhiVquNue0zG8oRtY/SSAxlzzgNGzjjK9KBy+LN2GqDNqAHC8hM?= =?us-ascii?Q?+YYabIv+t04a3rCVomkNmqTMGKECmoCr2NVwYciukKrgz7hM251zZkiiUBEj?= =?us-ascii?Q?aEGduxwgrz0wTVBMvWJul9zwBYVDyqYmphhDywye8odnR7tLluLyH4VAtabn?= =?us-ascii?Q?ora9r/U0/1eJp/jyIUzI5wwPJBbCO0f4eOKK38lfwl7T65meHdSjq1do19L/?= =?us-ascii?Q?WpfpQyEIm2hmj+1Yz/2DD2hoZMy5Yzs5jCWFmtUxORmlbpKHI3V6u7LNWfEq?= =?us-ascii?Q?yqH+ISluanX+njeFtwT0BlKkwAL7z4TBhnOE+k2kpVReVtGQk5Afw27MAmg3?= =?us-ascii?Q?nLsHeQBcOT0G2PoleJ15sJNi/zUs51tohPQS8rmxb+dX/Ot7HNH+3TGKCq+l?= =?us-ascii?Q?9Y3wMMu1exs/OdJzOLCj2KJMzRBaF32V6q42VfzLCWBecTldUHN6H4xWUKNT?= =?us-ascii?Q?6FV1C4UpkDDU7/vixkABzwG3wEIdMEtnY9IFsjRSK0mqrqgytqFU89zok/BA?= =?us-ascii?Q?v6f5YmxL5faLsxDciSYWY4FD3bQiA3AA6QzBBzpjY4BCZ/bK2t1R1l8+Xqfq?= =?us-ascii?Q?o2PXfVYHYIaYom9Luve9CHREsgBL+C/kc6pVsWnFHCTz0jgmPJowg8vBClo2?= =?us-ascii?Q?p7X56gwxwWmdIePxufXwZvFzBtNKreTkevqhhAk7gY6ukdTxghjCww8fBNfR?= =?us-ascii?Q?LLXSp5QV34Mbv5ZH0BVrPYl8eWCjIrEq9kWE59TdYHU9MpEYAGsS7Lw7bLcD?= =?us-ascii?Q?sx3YinXO2UuqQp5oyoQpvLdRyua84JVtnLETLiSmuY2ehdR0vmFx0AO3duUh?= =?us-ascii?Q?amrhyOzd6hyaVzKhSBAUNLaOW9/PH6nB6g19c1eWTL/y9CVoPlHF8fffrZkT?= =?us-ascii?Q?fveVL29HNIZr5Afa1SSsfSr4vhkGQ042iuDKuD3FNLhFqnj3VTNOLHHHT1g8?= =?us-ascii?Q?Jx9ANa+zgQk8W+2Vpdj1HbTas7l2XmXV6cO9qf9HOnAnmZe4RfeTwZ2aXNTe?= =?us-ascii?Q?Hi46rh2lZiO7jB023QXVZmHzRLum4RvKv5mzvzj/PGHJhCe+VE7KPovFlxX/?= =?us-ascii?Q?UrWBZbxmrjLHddiZViB9nJtzQarlbwXx5Q7NJJuJuWkN2qUHSMGKqCwoSY0y?= =?us-ascii?Q?Lgvd3GAoK5gZiR6DN7LgtM/TMIGv6N0IAj1QTCTyek9hlNEHKlyzTAaERhtJ?= =?us-ascii?Q?k13KKt+1PgRhO27wRKYv3LZfr7KX9/PKvAsoo3ad8enLFH0iI4XIkSXbL4ed?= =?us-ascii?Q?btZwx28B9oGg684AR9fVpUbeM9wG+a2dfpEapquv0dk3NV+ya1L4Ap+E676e?= =?us-ascii?Q?LuyswF6wg1rnpULw6nBx/326t48oLUrZX2EkDPni/nXX0+uOEJfgEo7JoDCM?= =?us-ascii?Q?ytlZmOvqaZ1xOYp50MqrzHnD6KrGyjBRo+ijm6/pne4pKeWt5/z1eghZC/xP?= =?us-ascii?Q?ZrLTu4ikXq7pP8DGADmpnV43A2/6Ev+b0d/iVwMrE5GdVcuM86VKkG4Hy1/+?= =?us-ascii?Q?M8flo/Olq/QH19jCcqQ=3D?= X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: 82296d42-9d23-40f2-3fe4-08dcabe0b6d5 X-MS-Exchange-CrossTenant-AuthSource: DS7PR12MB5744.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 24 Jul 2024 13:01:20.9717 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: TBpMoBRiLx/QDGcrqF6nwlt+WjghURwjXy7NFjFHORPQSiX2HCw5Oo5OYT8gv441 X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ0PR12MB6709 Content-Type: text/plain; charset="utf-8" If memory tiering mode is on and a folio is not in the top tier memory, folio's cpupid field is repurposed to store page access time. Instead of an open coded check, use a function to encapsulate the check. Signed-off-by: Zi Yan Reviewed-by: "Huang, Ying" Acked-by: David Hildenbrand Reviewed-by: Kefeng Wang --- include/linux/mm.h | 6 ++++++ kernel/sched/fair.c | 3 +-- mm/huge_memory.c | 6 ++---- mm/memory-tiers.c | 19 +++++++++++++++++++ mm/memory.c | 3 +-- mm/mprotect.c | 3 +-- 6 files changed, 30 insertions(+), 10 deletions(-) diff --git a/include/linux/mm.h b/include/linux/mm.h index 17753a463e01..2c6ccf088c7b 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -1717,6 +1717,8 @@ static inline void vma_set_access_pid_bit(struct vm_a= rea_struct *vma) __set_bit(pid_bit, &vma->numab_state->pids_active[1]); } } + +bool folio_use_access_time(struct folio *folio); #else /* !CONFIG_NUMA_BALANCING */ static inline int folio_xchg_last_cpupid(struct folio *folio, int cpupid) { @@ -1770,6 +1772,10 @@ static inline bool cpupid_match_pid(struct task_stru= ct *task, int cpupid) static inline void vma_set_access_pid_bit(struct vm_area_struct *vma) { } +static inline bool folio_use_access_time(struct folio *folio) +{ + return false; +} #endif /* CONFIG_NUMA_BALANCING */ =20 #if defined(CONFIG_KASAN_SW_TAGS) || defined(CONFIG_KASAN_HW_TAGS) diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index 9057584ec06d..416e29b56cc4 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -1840,8 +1840,7 @@ bool should_numa_migrate_memory(struct task_struct *p= , struct folio *folio, * The pages in slow memory node should be migrated according * to hot/cold instead of private/shared. */ - if (sysctl_numa_balancing_mode & NUMA_BALANCING_MEMORY_TIERING && - !node_is_toptier(src_nid)) { + if (folio_use_access_time(folio)) { struct pglist_data *pgdat; unsigned long rate_limit; unsigned int latency, th, def_th; diff --git a/mm/huge_memory.c b/mm/huge_memory.c index 15234b2e252e..5c0a6a4e3a6e 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -1707,8 +1707,7 @@ vm_fault_t do_huge_pmd_numa_page(struct vm_fault *vmf) * For memory tiering mode, cpupid of slow memory page is used * to record page access time. So use default value. */ - if (!(sysctl_numa_balancing_mode & NUMA_BALANCING_MEMORY_TIERING) || - node_is_toptier(nid)) + if (!folio_use_access_time(folio)) last_cpupid =3D folio_last_cpupid(folio); target_nid =3D numa_migrate_prep(folio, vmf, haddr, nid, &flags); if (target_nid =3D=3D NUMA_NO_NODE) @@ -2061,8 +2060,7 @@ int change_huge_pmd(struct mmu_gather *tlb, struct vm= _area_struct *vma, toptier) goto unlock; =20 - if (sysctl_numa_balancing_mode & NUMA_BALANCING_MEMORY_TIERING && - !toptier) + if (folio_use_access_time(folio)) folio_xchg_access_time(folio, jiffies_to_msecs(jiffies)); } diff --git a/mm/memory-tiers.c b/mm/memory-tiers.c index 4775b3a3dabe..2a642ea86cb2 100644 --- a/mm/memory-tiers.c +++ b/mm/memory-tiers.c @@ -6,6 +6,7 @@ #include #include #include +#include =20 #include "internal.h" =20 @@ -50,6 +51,24 @@ static const struct bus_type memory_tier_subsys =3D { .dev_name =3D "memory_tier", }; =20 +#ifdef CONFIG_NUMA_BALANCING +/** + * folio_use_access_time - check if a folio reuses cpupid for page access = time + * @folio: folio to check + * + * folio's _last_cpupid field is repurposed by memory tiering. In memory + * tiering mode, cpupid of slow memory folio (not toptier memory) is used = to + * record page access time. + * + * Return: the folio _last_cpupid is used to record page access time + */ +bool folio_use_access_time(struct folio *folio) +{ + return (sysctl_numa_balancing_mode & NUMA_BALANCING_MEMORY_TIERING) && + !node_is_toptier(folio_nid(folio)); +} +#endif + #ifdef CONFIG_MIGRATION static int top_tier_adistance; /* diff --git a/mm/memory.c b/mm/memory.c index 802d0d8a40f9..833d2cad6eb2 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -5337,8 +5337,7 @@ static vm_fault_t do_numa_page(struct vm_fault *vmf) * For memory tiering mode, cpupid of slow memory page is used * to record page access time. So use default value. */ - if ((sysctl_numa_balancing_mode & NUMA_BALANCING_MEMORY_TIERING) && - !node_is_toptier(nid)) + if (folio_use_access_time(folio)) last_cpupid =3D (-1 & LAST_CPUPID_MASK); else last_cpupid =3D folio_last_cpupid(folio); diff --git a/mm/mprotect.c b/mm/mprotect.c index 222ab434da54..37cf8d249405 100644 --- a/mm/mprotect.c +++ b/mm/mprotect.c @@ -161,8 +161,7 @@ static long change_pte_range(struct mmu_gather *tlb, if (!(sysctl_numa_balancing_mode & NUMA_BALANCING_NORMAL) && toptier) continue; - if (sysctl_numa_balancing_mode & NUMA_BALANCING_MEMORY_TIERING && - !toptier) + if (folio_use_access_time(folio)) folio_xchg_access_time(folio, jiffies_to_msecs(jiffies)); } --=20 2.43.0 From nobody Thu Dec 18 00:48:13 2025 Received: from NAM10-BN7-obe.outbound.protection.outlook.com (mail-bn7nam10on2052.outbound.protection.outlook.com [40.107.92.52]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 94C8E158D66 for ; Wed, 24 Jul 2024 13:01:32 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.92.52 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1721826094; cv=fail; b=OMk1ymtBWj7JOlZ/KyDh0VdSfXeGfmTW4b+3gtjY2gEjWqCVhXoa1e9GIr7tgTwTc6i7it+UigW2gKNgdKJ3vVL/2vag7gPHRq7319R49hwoxxsltUIujyzwXATZsTXq8EDKtluoYSModmhivlVZvUUMBmwvSYnsFdv4yL6PzaA= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1721826094; c=relaxed/simple; bh=THC5hz5JzXz4sahAczdf2zYLeTEAVprhp4QtLuCbubs=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: Content-Type:MIME-Version; b=MhvF+U/3iVOIycpkAsHAWnGF1sqSbx1KF0W6huji0OseuI07oS0Rn2HEaigLxndL+0GaR9e68lVhLfpaRUN24PORvtJDDkdYkGeVWAOySMINTWVb8id13CZJ43aAGCXiqO7mRurSUApmny6rwyr0SsMCQG43sp3ezu2qqaURYgs= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=s0vQVOpe; arc=fail smtp.client-ip=40.107.92.52 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="s0vQVOpe" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=oTXQPqJNxFxP6o17/mkav4CxxEShfcrWQuM9qYsOm6BRvtQKC26A64WhWXRQ05/Cisnhalb4r9h/8g6KXGbZVnyY3c4IzpFLVTebne0tThpBP5HafW6v2K1CIJOlOBY41ihYt5HXjMjOa4gHRxUt/wj85A5V55s2dgZ+jPMcG3YRa+UBMRJu6JrMW2D1ff/M5+teKIGYDXJ2M/zgNrrco0EZa5ypnfTH3RMCSuyxXhpcYgJFG0f3bdkIVqmst3afh6mV75jSAvwwswCgOSmAix6K5Jak0poEQdq2/jEQuqyXCls9GHPxE+3UEbMewM/CRCgNJBj6MYZzntsJb7kodQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=FetlK/722fVi6ZVNeIZFKwGEKahpMrmor89WHb8Wxko=; b=POdlAqhXttN4PRJRPmH5IxnikKN4/itPBOkPYZB+6f/rQp7j2od+bTXwxqDp8XAoggdbMgcAT4vvwu+yuh5i4LI8Eg8I30iOoHOR5YOsYtnK8FzxeSMbQywSVthT+Lud41f6RF/7HhQtxOqW7jUESzwc/eIkMwgAvO/5Mn+Qvua+W//e+YAlOxAR/+0g2Jk+OZsnDl2YDTrJv4bBDFqVkO5gfHyO43Tg+JccoY+RTUWBuctv5d7HtiM5AdeU5vUrgvwTIBRT7MhErzM2Q3fIzsWIfnMHjquHte/BtUA5b+3od9/v3ElJdZR1GLGDFQpfq0MI0JoY6MZHH15EP7z8vQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=FetlK/722fVi6ZVNeIZFKwGEKahpMrmor89WHb8Wxko=; b=s0vQVOpedj38HyNtjrjdCuIwlXmgFVUn+DRELWIoHNSIVmF4a7CLnAKq0ooPoPMZFs9ZdUe4UShITheGTsx4Q7ED4FbO/XpJGbNbmhxY3ZZ/z6YynrUAdUkbz77rvjY7kzEo9M0qdgkPkL9/dbVEhc5Diy3P4xz44VIJeYRPmpDTWciBZYF9vYj6s69WOqQxk0DlIc+Atwc29dQakDrXnaCUggslUi4rzjWpFJRKJHyVqWQi3NFcbHMaA+47+fLg5VffYq4eT3QFKEm/pTW1bKJ3MC5EjjV7cqs4mSHlE4oTkQ3LtQ8/y+VIChHnQ7sBDk6Qs18VjrYW4eMvuaByzg== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; Received: from DS7PR12MB5744.namprd12.prod.outlook.com (2603:10b6:8:73::18) by SJ0PR12MB6709.namprd12.prod.outlook.com (2603:10b6:a03:44a::20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7762.28; Wed, 24 Jul 2024 13:01:22 +0000 Received: from DS7PR12MB5744.namprd12.prod.outlook.com ([fe80::f018:13a9:e165:6b7e]) by DS7PR12MB5744.namprd12.prod.outlook.com ([fe80::f018:13a9:e165:6b7e%4]) with mapi id 15.20.7784.017; Wed, 24 Jul 2024 13:01:22 +0000 From: Zi Yan To: Andrew Morton , linux-mm@kvack.org Cc: David Hildenbrand , "Huang, Ying" , Baolin Wang , Kefeng Wang , Lorenzo Stoakes , linux-kernel@vger.kernel.org, Zi Yan Subject: [PATCH v3 3/3] memory tiering: count PGPROMOTE_SUCCESS when mem tiering is enabled. Date: Wed, 24 Jul 2024 09:01:15 -0400 Message-ID: <20240724130115.793641-4-ziy@nvidia.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240724130115.793641-1-ziy@nvidia.com> References: <20240724130115.793641-1-ziy@nvidia.com> Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: CH0PR04CA0010.namprd04.prod.outlook.com (2603:10b6:610:76::15) To DS7PR12MB5744.namprd12.prod.outlook.com (2603:10b6:8:73::18) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DS7PR12MB5744:EE_|SJ0PR12MB6709:EE_ X-MS-Office365-Filtering-Correlation-Id: 3da80656-9e10-43cc-8961-08dcabe0b79c X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|1800799024|366016|376014; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?MnTThk9tzg2CzTQwcX0JG8vwlbLhvLKjoRcJsBF9KM8hFvuwrWGLZ4SEiIHf?= =?us-ascii?Q?Qb1+2MSu1zm7rBG4lUd5d1omzx1w1P3f0RihwIXRR6si/gbh9iAB8iFImALD?= =?us-ascii?Q?q1HQnl6fM+JEhz7zXCJQOgllnH6AvFq6zdIL0SP1Bs/n1IzqmTneupGbD06W?= =?us-ascii?Q?oweb0fB+cBKlMHWsj7WBcoZNzaU3Huue3jWfrnVJK2Z+yHHhIVe+Cd9iE79X?= =?us-ascii?Q?Q8KJeKsLEWkTv5pujwBPdMnq8sxrnjsoXQVJsGvgwLnQvuODWxWYA1ieZFnC?= =?us-ascii?Q?YWnjK40g3//QqQo2GocCNZKIPrNp4+LESPPtmxKehvA3AE6GrMi453hAlils?= =?us-ascii?Q?RT1DyKrDbKhseIQ/7hQzN/6FUdkByMwXA7Z14ZzSANIIlF5IT7kG/vkmp9AZ?= =?us-ascii?Q?4oVSPJ6MeJ5nO/7JTKZBQ8qWYJQ4nq657fK4HImK+bgrWKYMvV6Ajeujumrj?= =?us-ascii?Q?tdeHZBut6mXT5300NpT0pnsFXczWvqpb5WJmK7dejsbijRPZ8dvHmZJ02cBv?= =?us-ascii?Q?EE57iJafUqFEZuEf/GgspEtv17Fv8WUvjMdzwXynpGVf71ysuT+OGpXw68eW?= =?us-ascii?Q?rZ4tP8MxRqIEDIKHgLoZ7nQT1JC4bIulz1J6NFQe/w8cWNIEeuVSZtEkXRzw?= =?us-ascii?Q?uHnOEA3wAsSpJxLkIn86FGoPIbUCnqyf0KwaH9p2EUXgagdhrqRdI8/Yq0to?= =?us-ascii?Q?9lNluTbLxV/l3jpBk8b8zW0ohZsEbErKJxZ/4VVM1oIsPIYs3etr1LVBEV/D?= =?us-ascii?Q?/W3pszYb8kybkvqZk1mVrgJneuYocivXzQLtccHqdQ1tt0o9CqA8JifmoD77?= =?us-ascii?Q?3qSO5SAjgDGAF/KpvSOueMl6ujS0x46Xpm1Gac5XDwZTwDRrvdEh1bIJx9EO?= =?us-ascii?Q?kyOqq5fqLO05DOM3nrCocJ+H+NzAwa21tEnYAuA/XuZGUOrtiqwuBdT9rgGk?= =?us-ascii?Q?hnRpGPbCmEnXXBmYnW3CAk/BFp0zrtrMSSkEnXantDFvX1MBQw4ZbkGuZBHb?= =?us-ascii?Q?cxCdOhBpG5mqwCi2/EZFwWeNrFhDMahd0E3JfrZDHgJq4QC/RD9atob151OL?= =?us-ascii?Q?wcefrAzIz2G/I8sORAMft4FKtIT3ODKtXSLG8/+hFuOPTzfIWPXTnz10mDgi?= =?us-ascii?Q?XEthc2TYC+9FrrPObO/egv8QDHNAYsaS4oIXAQyLsmJKhcHJ3UHN81QcIvo7?= =?us-ascii?Q?RbWEj3QuavjHI0Cz39pBM1rtXOS/REFh8j1vhPSdEQ8oEpxGMWE4RyGvizSd?= =?us-ascii?Q?GtJzny3DzcayRgCgL7TVrPqvfmYqV5FG7jwbvzQqBnVtAowXUYEcH5gGnsoq?= =?us-ascii?Q?EBdaDd7HckIscUhS8mCozr7mV265ncBUakRYT8iQIFf1DxvauCEnSZthXtXT?= =?us-ascii?Q?7hQY6qo=3D?= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DS7PR12MB5744.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(1800799024)(366016)(376014);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?zzQ1yI72LD6yMPbozV0+u377efR9JRHC/i/T5QGO6ERNk25V2cqhGmWPLuvv?= =?us-ascii?Q?YvbwrlODVPXdgypVFy8HfDGdtk/g8+35Ir3DZb0QRXz5K4PB/VPkBhjd6n3F?= =?us-ascii?Q?P23v+X/LdEN+gRh11sdYfIAJ5AxOhkEmPfGSQ36O2vLVhTFRzWyQMG8AOjBJ?= =?us-ascii?Q?fCGZ62CpAikNF+PAmXt7ykn0dyyWwv8EbuiXwSe1iOWf/DnjeUsBVHrZb8XF?= =?us-ascii?Q?ywYeWYfhzUN8tYVGlyT8hjiKhKNafyOFShXCArrTmFIrQWfUq2wQwvO17dME?= =?us-ascii?Q?WNKxt7GuXAAci2eMox2Dq6esra6eFH8/yiob28yZRRzNhokiGdi73YbH2Vx9?= =?us-ascii?Q?MDTv8QCAHdQEaI+4N8Pq3pu2lOmPNbvjv5i9j43K2L0C4H4Gwe1t6G+pU6pV?= =?us-ascii?Q?Iu/kYaNFUQoEJbOxWcEqpFc4Ag4bVmkmWcR4ji6sDkchAGKubcDfrBqZNp/E?= =?us-ascii?Q?lTGCiAaV9+n3nAMNEYPKrCgGprlTZ43Q7VUhMv4yeb5F3VXfD0NPZb/LgYjl?= =?us-ascii?Q?7U4wC1H3y/SxP/ET8QWdYKJ4gHdQTi3ZRB9Snbm6lKUebtMahYo4KE/aaXls?= =?us-ascii?Q?7OfRXPH0/G7tnrmhP9zW6IKPWew19GRl8Sqcv5sjkmqCy58CgT5VIg0foUN4?= =?us-ascii?Q?GLrBGQyvuNLqo19vXNK/mIp1/4VJBcvNCPjFbeMXLjCSQ2VsFhkhGbRk97iC?= =?us-ascii?Q?VMrscbeEJw3il/LSIr98PAUHDh+0/q+Eqv35OD0RlFVnaTIGDeF4VLG89QVa?= =?us-ascii?Q?uohqu2qHffLhXDDG714oUyF7WbmFh77uvZLOd2kg6JBLk8sBY1bbY60XPgIj?= =?us-ascii?Q?qXoSKmJXl6xmrPTBkA19m8pg3b6EUVvXT6pnH3I8Fwiz5F5gkr7/+7FYKSou?= =?us-ascii?Q?nLg66T0QUKQb91KcK91f+e/rwfZfxlPakVZEwEH9OhFtMuYjCbVtzMtMdTD7?= =?us-ascii?Q?Ph67ZQrMTmip+9wms7/+QSN7nCNOkuSiGGK/0hVnQq7+oklAbh1+w2grAGBG?= =?us-ascii?Q?lshaVfxdWFDFEiZo7847kWLOQnf+h+Q/wZapjg3Dg/5Gz4U6Na3IIEYNiIYN?= =?us-ascii?Q?YRFFlVDOc97Rb7GKxm/Wu8dbc3rOnl6na5ayLs669lyEWaM/0lAGsRRCfN7f?= =?us-ascii?Q?+wmef9Tn/qvMa87rB4+AW6BJ7njyDDnL6wU8Yh1p8gjr10CwvxpQMzK/YNUQ?= =?us-ascii?Q?HkggxTp9AZmRu8ibSqisUZswNih3SyziRWHphMuesNg3tSV69jI1slXp9A1s?= =?us-ascii?Q?yGD5KTVfFXrA6B3st2s/niGpSPWRNbCPhQJq7Th1XwbyPdRliVpq9v+1DgxT?= =?us-ascii?Q?tsVZnnV4j5QXNbZaTWRudcJQccYduLcc/3us9ZDhDrzCqimdUvEMw4Vak5RN?= =?us-ascii?Q?urBV0BtpZe5eLe2ruBUBzPOZwdIOS8du/tYweWHYWK95BnzOjO0YhE6zWAZ+?= =?us-ascii?Q?IySZLIj0L0xK2OMerRHi2LgrBnRTHFbchlrNicZJzIPrmRluPpbOT4Fzx6G6?= =?us-ascii?Q?ehT7/P1C555WNCMu1eObd8NjKovtnMk1QSpDvVIBNTU6yB8Hg2VMi+8UQwgJ?= =?us-ascii?Q?qQTf/krJP4Pe9t/ybcU=3D?= X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: 3da80656-9e10-43cc-8961-08dcabe0b79c X-MS-Exchange-CrossTenant-AuthSource: DS7PR12MB5744.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 24 Jul 2024 13:01:22.3983 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: F75X5XeByluzv5oykT1mLdYpXLwtwhry+BFd7ffSyE7sQMuCoBXyX0c8saSm6aoV X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ0PR12MB6709 Content-Type: text/plain; charset="utf-8" memory tiering can be enabled/disabled at runtime and sysctl_numa_balancing_mode & NUMA_BALANCING_MEMORY_TIERING is used to check it. In migrate_misplaced_folio(), the check is missing when PGPROMOTE_SUCCESS is incremented. Add the missing check. Reported-by: Kefeng Wang Closes: https://lore.kernel.org/linux-mm/f4ae2c9c-fe40-4807-bdb2-64cf2d716c= 1a@huawei.com/ Fixes: 33024536bafd ("memory tiering: hot page selection with hint page fau= lt latency") Signed-off-by: Zi Yan Acked-by: David Hildenbrand Reviewed-by: Kefeng Wang --- mm/migrate.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/mm/migrate.c b/mm/migrate.c index e7296c0fb5d5..826afb82ca56 100644 --- a/mm/migrate.c +++ b/mm/migrate.c @@ -2630,7 +2630,9 @@ int migrate_misplaced_folio(struct folio *folio, stru= ct vm_area_struct *vma, putback_movable_pages(&migratepages); if (nr_succeeded) { count_vm_numa_events(NUMA_PAGE_MIGRATE, nr_succeeded); - if (!node_is_toptier(folio_nid(folio)) && node_is_toptier(node)) + if ((sysctl_numa_balancing_mode & NUMA_BALANCING_MEMORY_TIERING) + && !node_is_toptier(folio_nid(folio)) + && node_is_toptier(node)) mod_node_page_state(pgdat, PGPROMOTE_SUCCESS, nr_succeeded); } --=20 2.43.0