From nobody Tue Apr 7 18:35:25 2026 Received: from CY7PR03CU001.outbound.protection.outlook.com (mail-westcentralusazon11010053.outbound.protection.outlook.com [40.93.198.53]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 975B437474B for ; Thu, 12 Mar 2026 04:45:17 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.93.198.53 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1773290718; cv=fail; b=OXbihFRzI2rGPFn3Mfra+LI9bq7331JEMxENRTABdXBaZiZw7vGzj8JsycuMBCH9mEAtsCq9p37LL/g1hdADDWbiiso9kA9oSXiV+4W72er8NtFA5dKZKYg2bANtDoDIwCsJ3fw4RLC7/GbyAUtJ8IMhunqD5ar4d20HsXXWxvA= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1773290718; c=relaxed/simple; bh=a849sjcJfH5lcNOaZnmaVokfDSd8BNY/+noBpg93j/0=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=GCOtNTJIlzKXNDOcugyqNdr9VNHW3DuTiIjbYoi7XYCSZuAzp4z0OUOiA0hUcNfZ66QyGNAOED2fD2CJ7LUNVY085JBZTfFeMzn9P0Dc2qIHTL1spb1tP44T60LVQCL8MNMqBIJxLyu02k3Cc6sb1LE//WJhvFPtk7Uc19kiEMs= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com; spf=fail smtp.mailfrom=amd.com; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b=b5j7y85v; arc=fail smtp.client-ip=40.93.198.53 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=amd.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b="b5j7y85v" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=YZCXyup8rqxN+SqLQmaHEO4+sbF6sszeby96cXAsL9uqb6WdF6BX05KlO+1HoNkuPU2B4LApDdyDfQ/PDn5X6gI+hXby0HmEHQmhHRJxiFiFSNEvTlUB+cLln/APy79vS69+zSeoNLvRl9URtAE1vQ9JFdv840kdiZVyvmKUaM1pawFRFQTVC7iT+T2BY8mPWw757cJsvzLXwhucw1OIKEL2mfTPFXxDJcUDvU+CiTOyse8tuHQMmoCTlNm70xT+vh8S6uthxTje8v9RKOChvgpE4WOa/7SozFKY0miqPSG4o6S5fvHoc4uyEYCOkUgoG2wR75a4cz83TSWBOBQ5kg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=IxNdfFMUX9sLJwTaUdqtNmtf/2iyMJqtySDJtle3f58=; b=guIXbXLP8/zGM4esUKQtFCLuyRjumBwhgmZR6NYgDbctef6eRYxwvA50lqIm7BT1nJ6gEPSF2z43MGC0PY5t112nE+RfN/RLZ4xwGFiI+LqaQx4OsVQRM4Du1TsQBaUN8rnaUZQY8QfTI2wjvhTqA/uvH3YmlKdGhWf52YDst0QNb04ZdpcIMtUb7X/a0QA0OlKT+rFkY478OB1X3llb9S89dF+NWaFPrZbKruMf3xqhy9jhwTZkvqUPypNfWYYUV40g6RzJ8L+y3Q3lGgTtUazCROzeSY6vx/wgF01W4qhVRaZdyCWh3MBxC5s4VuGTUeUmV70YFutN1R2SaRcrmQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=redhat.com smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=IxNdfFMUX9sLJwTaUdqtNmtf/2iyMJqtySDJtle3f58=; b=b5j7y85vez0kY1s6D3JmGX7VCya2Z8SJrXD/gaGSCtfHbci88VrCUw5oHvTxI3vnGq8PEKe5noQ6V7EsTSL5XUAZkBRaf4rAcB3iI4Gw6mkeCOyq3TJb54aoTdLscjjWXGSa1DeYD7E2nwWFz5Vb0fXoKtgl5ArAcBsFjpZPWjQ= Received: from BY3PR05CA0040.namprd05.prod.outlook.com (2603:10b6:a03:39b::15) by BL3PR12MB6401.namprd12.prod.outlook.com (2603:10b6:208:3b1::8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9700.13; Thu, 12 Mar 2026 04:45:13 +0000 Received: from MWH0EPF000C6187.namprd02.prod.outlook.com (2603:10b6:a03:39b::4) by BY3PR05CA0040.outlook.office365.com (2603:10b6:a03:39b::15) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.9678.25 via Frontend Transport; Thu, 12 Mar 2026 04:45:12 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=satlexmb07.amd.com; pr=C Received: from satlexmb07.amd.com (165.204.84.17) by MWH0EPF000C6187.mail.protection.outlook.com (10.167.249.119) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9678.18 via Frontend Transport; Thu, 12 Mar 2026 04:45:12 +0000 Received: from BLRKPRNAYAK.amd.com (10.180.168.240) by satlexmb07.amd.com (10.181.42.216) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.17; Wed, 11 Mar 2026 23:45:05 -0500 From: K Prateek Nayak To: Ingo Molnar , Peter Zijlstra , Juri Lelli , Vincent Guittot , Valentin Schneider , CC: Dietmar Eggemann , Steven Rostedt , Ben Segall , Mel Gorman , Chen Yu , Shrikanth Hegde , Li Chen , "Gautham R. Shenoy" , K Prateek Nayak Subject: [PATCH v4 1/9] sched/topology: Compute sd_weight considering cpuset partitions Date: Thu, 12 Mar 2026 04:44:26 +0000 Message-ID: <20260312044434.1974-2-kprateek.nayak@amd.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20260312044434.1974-1-kprateek.nayak@amd.com> References: <20260312044434.1974-1-kprateek.nayak@amd.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: satlexmb07.amd.com (10.181.42.216) To satlexmb07.amd.com (10.181.42.216) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: MWH0EPF000C6187:EE_|BL3PR12MB6401:EE_ X-MS-Office365-Filtering-Correlation-Id: e9c16e44-7c8e-44ae-c934-08de7ff225af X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|376014|36860700016|7416014|82310400026|1800799024|56012099003|18002099003|22082099003; X-Microsoft-Antispam-Message-Info: wf090kATV+nXWThCKLYseAiSoBdVVpplqKZkyobGzwBgpm+ow/xaGWIOPzj9Qnsd1tw10FqbvkgqXZmIY3Ijsslg5fNJIWi3p44BiUe/EuyuC/4tjMF4XV0T6m2PfWwOn8F69G/0ULQHsNIaqsbIi3rIqeiaadZoGLEKD38w0RhY951uiyMeCPrhtuukhvwf4dSvglaD1fZ/i5hDgGtyMiNYr+/nF8eFMC0KcspG0m9E4KFzvt6gaudosn5eIAV1/LBJcFoM+JIlsZeoFDOV2s6NP58GaQ7g4oRH/eE4gsQ7RJS80NInR6OROiIkqgeTQQ6xx8LtRf+jRH6oZqgjDDS5q3sBdd3Ury8LsIfG4LHLWDm9oGcJPrdtPPu4+zHfdaTULV8i56cMGjYFkB4fla7DFr5jxLsEVXVLfhk/5VHgOTEulrOlrx9WmeW0cX2PPW8s/9BFjMu3j0CY+BxInqM2Pwsrmpp9tNeyrJVQZmhnTsMOb6Y7+XAp+jPC7/4/+jwZPNWwJZ6hdcNzdS6WBPBIpUupW9zQflEPLqAmnxOY3BwPitkflNPIJP7fHEq2F65qv8r2C9x1qW+BUa8KYro3wsWxMdqeUgiBWGp5unMNxxXfPkoijNOPVxt4zhqrEb7pr4Irr0buVtuYbZ+1BEgWKk19yzQG/fdJ4cogCsog52j4P9ysTk0/O0U86rtHpTYu3hu+Sn45vmXnQupneOI7+fnVo2a3gGnVHeSH8YUQhvksOhE21zbSdHvqCZx8pGsw0LbEyIbW7xgVZDkjTA== X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:satlexmb07.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230040)(376014)(36860700016)(7416014)(82310400026)(1800799024)(56012099003)(18002099003)(22082099003);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: p6nDM/lv9TJNB9hNIwFEyhrrQ6IMGvV/YGb06RtoDLstyY0/CABw+E0BuGt+Mn1pawSaXz3S5Y+rCagFpifGrC3c2IZQF79/9RlniabBG+rpfphkq8tLamgHueofTVlmZSflN+WLq0x149CeUvAMsfcXR2wk5u40ijSfxOs4VK0Q8zuK1bz7OB8WAfV91TwHD3tK+ykwWmfenJd5qDtLTEIF8XSP6OQhAld7forHPFDMRw8393+DU7ydEOJIFuC2XIUZQTBPe925YkjzaXOt/UhCFdyd9AHPUsPoIBB11K3v74hmlf409mGeOuBexWlUy/anH7RGOl0GGMhMaaIA5MWREy9r2cGEGeIFzWI5k3HI2WDzQmwxKpRwM/Z2KQSIPxbWA07pfz3L9w62mytR6PJE5az7/V4FSDEcdgKZJvcYOnZxr9T8So1rxbH9NbRP X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 12 Mar 2026 04:45:12.3544 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: e9c16e44-7c8e-44ae-c934-08de7ff225af X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[satlexmb07.amd.com] X-MS-Exchange-CrossTenant-AuthSource: MWH0EPF000C6187.namprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: BL3PR12MB6401 Content-Type: text/plain; charset="utf-8" The "sd_weight" used for calculating the load balancing interval, and its limits, considers the span weight of the entire topology level without accounting for cpuset partitions. For example, consider a large system of 128CPUs divided into 8 * 16CPUs partition which is typical when deploying virtual machines: [ PKG Domain: 128CPUs ] [Partition0: 16CPUs][Partition1: 16CPUs] ... [Partition7: 16CPUs] Although each partition only contains 16CPUs, the load balancing interval is set to a minimum of 128 jiffies considering the span of the entire domain with 128CPUs which can lead to longer imbalances within the partition although balancing within is cheaper with 16CPUs. Compute the "sd_weight" after computing the "sd_span" considering the cpu_map covered by the partition, and set the load balancing interval, and its limits accordingly. For the above example, the balancing intervals for the partitions PKG domain changes as follows: before after balance_interval 128 16 min_interval 128 16 max_interval 256 32 Intervals are now proportional to the CPUs in the partitioned domain as was intended by the original formula. Fixes: cb83b629bae03 ("sched/numa: Rewrite the CONFIG_NUMA sched domain sup= port") Reviewed-by: Shrikanth Hegde Reviewed-by: Chen Yu Reviewed-by: Valentin Schneider Signed-off-by: K Prateek Nayak Reviewed-by: Dietmar Eggemann Tested-by: Dietmar Eggemann --- Changelog v3..v4: o Illustrated the changes in the load balancing intervals with an example. (Shrikanth) o Collected the tags from Chenyu, Shrikanth, and Valentin. (Thanks a ton!) --- kernel/sched/topology.c | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/kernel/sched/topology.c b/kernel/sched/topology.c index 061f8c85f555..34b20b0e1867 100644 --- a/kernel/sched/topology.c +++ b/kernel/sched/topology.c @@ -1645,8 +1645,6 @@ sd_init(struct sched_domain_topology_level *tl, struct cpumask *sd_span; u64 now =3D sched_clock(); =20 - sd_weight =3D cpumask_weight(tl->mask(tl, cpu)); - if (tl->sd_flags) sd_flags =3D (*tl->sd_flags)(); if (WARN_ONCE(sd_flags & ~TOPOLOGY_SD_FLAGS, @@ -1654,8 +1652,6 @@ sd_init(struct sched_domain_topology_level *tl, sd_flags &=3D TOPOLOGY_SD_FLAGS; =20 *sd =3D (struct sched_domain){ - .min_interval =3D sd_weight, - .max_interval =3D 2*sd_weight, .busy_factor =3D 16, .imbalance_pct =3D 117, =20 @@ -1675,7 +1671,6 @@ sd_init(struct sched_domain_topology_level *tl, , =20 .last_balance =3D jiffies, - .balance_interval =3D sd_weight, =20 /* 50% success rate */ .newidle_call =3D 512, @@ -1693,6 +1688,11 @@ sd_init(struct sched_domain_topology_level *tl, cpumask_and(sd_span, cpu_map, tl->mask(tl, cpu)); sd_id =3D cpumask_first(sd_span); =20 + sd_weight =3D cpumask_weight(sd_span); + sd->min_interval =3D sd_weight; + sd->max_interval =3D 2 * sd_weight; + sd->balance_interval =3D sd_weight; + sd->flags |=3D asym_cpu_capacity_classify(sd_span, cpu_map); =20 WARN_ONCE((sd->flags & (SD_SHARE_CPUCAPACITY | SD_ASYM_CPUCAPACITY)) =3D= =3D --=20 2.34.1 From nobody Tue Apr 7 18:35:25 2026 Received: from SJ2PR03CU001.outbound.protection.outlook.com (mail-westusazon11012000.outbound.protection.outlook.com [52.101.43.0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D0A9C282F12 for ; Thu, 12 Mar 2026 04:45:34 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=52.101.43.0 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1773290736; cv=fail; b=XMutXD9oLu0JleUV4Q6ZWsuLSONqKPzZNvlvD+0VS+ZkXByZFQ1Qwzcferu/P6zQUl7ggCEUxcSQp2HbIX6z8/sYE2uAxmSto0EUrbYmFoL5cF4qymOe3YHbnnBa7TIZMGzuMA7Fh5fqupGB8tzbDy2B3u+bgvJ9OlOCtB/8g1M= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1773290736; c=relaxed/simple; bh=xJ5r/u4BB4f+xzk4uQcyKW/cFENKg4+ZqPOWFwTGMXo=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=VLpLhE4wIkFomCdSili3duJ1UexAvbCq6oBtJ3adluhlx7QUQWyMDL9xA/txYYCSjGrh2v9IFrYcJNhMB6r3zwClL3WSID7zvEhrdWVuKtbBgP6iekOrdxetsZN9rM/RoI9aAgQ+ZcQTZWIpE1qMqGvxEhQ4EAfqo2KB+jQckYM= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com; spf=fail smtp.mailfrom=amd.com; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b=ZymWcLTh; arc=fail smtp.client-ip=52.101.43.0 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=amd.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b="ZymWcLTh" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=Obt3XPr8Vky50i0X+ab27Nin1avk82AFZeIExGHDONM0zYpibsVLysGFqi438Rq0f/je+9yZR0K57Wh7k8WatLvuvTXUsIi1rv0AHcMeSWpqQdb5g3m6RhITHge3vmCZB7BALIdrj1LyP1fH9A8Duo8CG++HFk5JGKTN4A7gEq3RHkpZZt0D9cNC6FgmEocONAShvFH4OtckNtG2erzW0MrRBOfFSP6axNbbxS2gBStpMLBefHDNdXGsWH3IE2g+4w3X0lHOHqhJg9hZPhtPSgrPlQWZaGIhnVBPr0FaY0MTocDjYdR0pMJNktoP9OwFusJnev0ED+YkYQvb/gdSsQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=sndsr/nWZsBT9PmROpdsCtyS8Ibr9qDOo4WaIKJCvXk=; b=wfD2aMPnaBxmnHomWVR2m4YGU+CSbACTVmjTvjNUxpjLOIyK2gsG3F/CWkjkBEshvUaCsWzuH/TKLQW+wMMABhm+c6COJGQm0QhsLHmVp7HWFYCXKtUcBvMMnwjj514u0ptpuGC2D/nwwOL3QgjYcKSBXZzuDNXysjvvQMTxGHRSiBx0xTnAKKEZmtWH7AesUXreevp+ypJkPuCh8PR62MAVE0Y1UAdnfIuY1rBwFTzXE8URr4M6OEXDKx9wUTc34cY+1ni4SMVGdR4Ji76RTWC23MNv7iyDpBsXdoj1Ubf3j6SI94DP/9sxCBaLB3Ilio0rgo+m5iU9BcsRD7Qxhw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=redhat.com smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=sndsr/nWZsBT9PmROpdsCtyS8Ibr9qDOo4WaIKJCvXk=; b=ZymWcLTh7uzTmEZ4O8EcDvVFawoXRry+eBywRFTq0f5q6JomlSx5Tjr0DkBvrU+WhEW0ko1P+NuJ5Lg0AFVZHkEhCByPWUHyScgqtv+h9knr5n95GH/hNpbgGG8D197I1IMwmkg+J2042GXecxmNDX2PNqhqiBP9YkxdMVSRCx8= Received: from BY3PR05CA0023.namprd05.prod.outlook.com (2603:10b6:a03:254::28) by IA1PR12MB8238.namprd12.prod.outlook.com (2603:10b6:208:3f9::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9723.4; Thu, 12 Mar 2026 04:45:28 +0000 Received: from MWH0EPF000C6188.namprd02.prod.outlook.com (2603:10b6:a03:254:cafe::1b) by BY3PR05CA0023.outlook.office365.com (2603:10b6:a03:254::28) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.9700.15 via Frontend Transport; Thu, 12 Mar 2026 04:45:28 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=satlexmb07.amd.com; pr=C Received: from satlexmb07.amd.com (165.204.84.17) by MWH0EPF000C6188.mail.protection.outlook.com (10.167.249.120) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9678.18 via Frontend Transport; Thu, 12 Mar 2026 04:45:27 +0000 Received: from BLRKPRNAYAK.amd.com (10.180.168.240) by satlexmb07.amd.com (10.181.42.216) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.17; Wed, 11 Mar 2026 23:45:22 -0500 From: K Prateek Nayak To: Ingo Molnar , Peter Zijlstra , Juri Lelli , Vincent Guittot , Valentin Schneider , CC: Dietmar Eggemann , Steven Rostedt , Ben Segall , Mel Gorman , Chen Yu , Shrikanth Hegde , Li Chen , "Gautham R. Shenoy" , K Prateek Nayak Subject: [PATCH v4 2/9] sched/topology: Extract "imb_numa_nr" calculation into a separate helper Date: Thu, 12 Mar 2026 04:44:27 +0000 Message-ID: <20260312044434.1974-3-kprateek.nayak@amd.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20260312044434.1974-1-kprateek.nayak@amd.com> References: <20260312044434.1974-1-kprateek.nayak@amd.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: satlexmb07.amd.com (10.181.42.216) To satlexmb07.amd.com (10.181.42.216) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: MWH0EPF000C6188:EE_|IA1PR12MB8238:EE_ X-MS-Office365-Filtering-Correlation-Id: 3d106c8a-68f3-42c6-fa02-08de7ff22ed5 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|82310400026|1800799024|36860700016|7416014|376014|13003099007|18002099003|22082099003|56012099003; X-Microsoft-Antispam-Message-Info: 42odSlB68YQ1sEQ4UA9kEJ5ffy4xrqjEzyzyGG1rPPRy42Ot4WdspAnYxLQJQNd/jBbwah/vjYWmlv46fsPGU+wwJE97/IKYtr1e+KbnpIyuP2sHyvFoCfPSYd51Z/oVxdgSYzqY6dWE6POPS2cfAqMZDfQgXypUAdQJgrXO/jv2YHmyULE8vXAlSjx1DZEo82s+4vDSuiwHxovpNTvr5VevWKNwHR7XDoC3CYrjdxozz36WtBnNmrqBPlaRDleN/DALBICrdA4HVwcWjcG5KVZpZyUy6NF13/rNXua3sVG9UG1dmU+Fx3DTpuCE3Em2flmSYsmqEAvaGM/QVZMrEN0ICiomn4aKm4B0so7OnLufEP3Ao254nFGQeTWyvtnXylerjwRAnhCs2s7FLvymxB0WI2m2w5UHlACY5GusCf7Hwyrxu1qxNSHR6PpoGaCdt8AAlvFrqrkeCjZGhFAp0Y2tIK5glaBQTYSAMFlVbYMErJ5s3M8f3v7wN4f+q6G9NEfJn/TOAZIjlMn72jmfij6pBTL2RjHaSNAGIPVaG1nTu+MUa/mpmME5FsnTPnofjwVGe5d/DEF/fU2+9ACIcL1312KTYUmhiTFv4PBM7mEdF/TDncN3Bbv5hUX8dvCBpxUuROmZTrNehn7OdqPaFe9ebtopenL8kJyOMJJ+jbbvZff9T+Z4L4iRxd6kpJ71JSM16HiFwgTuy+bQS76JotcrEIpYL9hBArbCnkGj9i8= X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:satlexmb07.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230040)(82310400026)(1800799024)(36860700016)(7416014)(376014)(13003099007)(18002099003)(22082099003)(56012099003);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: QMDEJERan+TqD6K4RI9gxu9orprI6XjhAWJg11BGNwY0G757XZTTp22OjHL8V2EWg8b3IpWrlP5sOxxZlFfXPWJLuH/Iawoe9pgBFLBqtF5Xwo91oAkJxxr12bJ4I89cKazxJs3vn5atSavS/O4No7ui/B8gB61gp515XOlosAIzY8iMLWyL+mNfgamcXXHBLzyFsgk4sVxAE2JPIePa6VvLfLk8kggJykNboTUtDTxNa3FxgF+TZEorfmAQX6RT7gfrILQY7oasKlVd5xJNBFBaiq5Y1CuomHessikFgFOKNRnZzRj22BTOsc9p5E7tKO0A49WJUdUBlMSkhtEo/+5UwPbkOVPJ/jZNCqYXYkPKv+dJz2+2n0g7FSDjDp4TSJvkE2y6OwPd7U6u0YI+2DtE+WLzZm7i8U6T+LDOor3aJSuqO4wTliBueQw1QqwG X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 12 Mar 2026 04:45:27.6961 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 3d106c8a-68f3-42c6-fa02-08de7ff22ed5 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[satlexmb07.amd.com] X-MS-Exchange-CrossTenant-AuthSource: MWH0EPF000C6188.namprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: IA1PR12MB8238 Content-Type: text/plain; charset="utf-8" Subsequent changes to assign "sd->shared" from "s_data" would necessitate finding the topmost SD_SHARE_LLC to assign shared object to. This is very similar to the "imb_numa_nr" computation loop except that "imb_numa_nr" cares about the first domain without the SD_SHARE_LLC flag (immediate parent of sd_llc) whereas the "sd->shared" assignment would require sd_llc itself. Extract the "imb_numa_nr" calculation into a helper adjust_numa_imbalance() and use the current loop in the build_sched_domains() to find the sd_llc. While at it, guard the call behind CONFIG_NUMA's status since "imb_numa_nr" only makes sense on NUMA enabled configs with SD_NUMA domains. No functional changes intended. Suggested-by: Valentin Schneider Signed-off-by: K Prateek Nayak Reviewed-by: Dietmar Eggemann Tested-by: Dietmar Eggemann --- Changelog v3..v4: o New patch based on the suggestion from Valentin and Chenyu in https://lore.kernel.org/lkml/xhsmh343e43fd.mognet@vschneid-thinkpadt14sge= n2i.remote.csb/ Notable deviation is moving the entire "imb_numa_nr" loop into the adjust_numa_imbalance() helper to keep all the bits in one place instead of passing "imb" and "imb_span" as references to the helper. o Guarded the call behind CONFIG_NUMA's status to save overhead when NUMA domains don't exist. --- kernel/sched/topology.c | 133 ++++++++++++++++++++++++---------------- 1 file changed, 80 insertions(+), 53 deletions(-) diff --git a/kernel/sched/topology.c b/kernel/sched/topology.c index 34b20b0e1867..7f25c784c038 100644 --- a/kernel/sched/topology.c +++ b/kernel/sched/topology.c @@ -2551,6 +2551,74 @@ static bool topology_span_sane(const struct cpumask = *cpu_map) return true; } =20 +/* + * Calculate an allowed NUMA imbalance such that LLCs do not get + * imbalanced. + */ +static void adjust_numa_imbalance(struct sched_domain *sd_llc) +{ + struct sched_domain *parent; + unsigned int imb_span =3D 1; + unsigned int imb =3D 0; + unsigned int nr_llcs; + + WARN_ON(!(sd_llc->flags & SD_SHARE_LLC)); + WARN_ON(!sd_llc->parent); + + /* + * For a single LLC per node, allow an + * imbalance up to 12.5% of the node. This is + * arbitrary cutoff based two factors -- SMT and + * memory channels. For SMT-2, the intent is to + * avoid premature sharing of HT resources but + * SMT-4 or SMT-8 *may* benefit from a different + * cutoff. For memory channels, this is a very + * rough estimate of how many channels may be + * active and is based on recent CPUs with + * many cores. + * + * For multiple LLCs, allow an imbalance + * until multiple tasks would share an LLC + * on one node while LLCs on another node + * remain idle. This assumes that there are + * enough logical CPUs per LLC to avoid SMT + * factors and that there is a correlation + * between LLCs and memory channels. + */ + nr_llcs =3D sd_llc->parent->span_weight / sd_llc->span_weight; + if (nr_llcs =3D=3D 1) + imb =3D sd_llc->parent->span_weight >> 3; + else + imb =3D nr_llcs; + + imb =3D max(1U, imb); + sd_llc->parent->imb_numa_nr =3D imb; + + /* + * Set span based on the first NUMA domain. + * + * NUMA systems always add a NODE domain before + * iterating the NUMA domains. Since this is before + * degeneration, start from sd_llc's parent's + * parent which is the lowest an SD_NUMA domain can + * be relative to sd_llc. + */ + parent =3D sd_llc->parent->parent; + while (parent && !(parent->flags & SD_NUMA)) + parent =3D parent->parent; + + imb_span =3D parent ? parent->span_weight : sd_llc->parent->span_weight; + + /* Update the upper remainder of the topology */ + parent =3D sd_llc->parent; + while (parent) { + int factor =3D max(1U, (parent->span_weight / imb_span)); + + parent->imb_numa_nr =3D imb * factor; + parent =3D parent->parent; + } +} + /* * Build sched domains for a given set of CPUs and attach the sched domains * to the individual CPUs @@ -2608,62 +2676,21 @@ build_sched_domains(const struct cpumask *cpu_map, = struct sched_domain_attr *att } } =20 - /* - * Calculate an allowed NUMA imbalance such that LLCs do not get - * imbalanced. - */ for_each_cpu(i, cpu_map) { - unsigned int imb =3D 0; - unsigned int imb_span =3D 1; + sd =3D *per_cpu_ptr(d.sd, i); + if (!sd) + continue; =20 - for (sd =3D *per_cpu_ptr(d.sd, i); sd; sd =3D sd->parent) { - struct sched_domain *child =3D sd->child; - - if (!(sd->flags & SD_SHARE_LLC) && child && - (child->flags & SD_SHARE_LLC)) { - struct sched_domain __rcu *top_p; - unsigned int nr_llcs; - - /* - * For a single LLC per node, allow an - * imbalance up to 12.5% of the node. This is - * arbitrary cutoff based two factors -- SMT and - * memory channels. For SMT-2, the intent is to - * avoid premature sharing of HT resources but - * SMT-4 or SMT-8 *may* benefit from a different - * cutoff. For memory channels, this is a very - * rough estimate of how many channels may be - * active and is based on recent CPUs with - * many cores. - * - * For multiple LLCs, allow an imbalance - * until multiple tasks would share an LLC - * on one node while LLCs on another node - * remain idle. This assumes that there are - * enough logical CPUs per LLC to avoid SMT - * factors and that there is a correlation - * between LLCs and memory channels. - */ - nr_llcs =3D sd->span_weight / child->span_weight; - if (nr_llcs =3D=3D 1) - imb =3D sd->span_weight >> 3; - else - imb =3D nr_llcs; - imb =3D max(1U, imb); - sd->imb_numa_nr =3D imb; - - /* Set span based on the first NUMA domain. */ - top_p =3D sd->parent; - while (top_p && !(top_p->flags & SD_NUMA)) { - top_p =3D top_p->parent; - } - imb_span =3D top_p ? top_p->span_weight : sd->span_weight; - } else { - int factor =3D max(1U, (sd->span_weight / imb_span)); + /* First, find the topmost SD_SHARE_LLC domain */ + while (sd->parent && (sd->parent->flags & SD_SHARE_LLC)) + sd =3D sd->parent; =20 - sd->imb_numa_nr =3D imb * factor; - } - } + /* + * In presence of higher domains, adjust the + * NUMA imbalance stats for the hierarchy. + */ + if (IS_ENABLED(CONFIG_NUMA) && (sd->flags & SD_SHARE_LLC) && sd->parent) + adjust_numa_imbalance(sd); } =20 /* Calculate CPU capacity for physical packages and nodes */ --=20 2.34.1 From nobody Tue Apr 7 18:35:25 2026 Received: from MW6PR02CU001.outbound.protection.outlook.com (mail-westus2azon11012063.outbound.protection.outlook.com [52.101.48.63]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4D2F6258EC2 for ; Thu, 12 Mar 2026 04:45:47 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=52.101.48.63 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1773290748; cv=fail; b=ecHYV3l8Xk7QATwt5ZexFah8RNTaL3h3o4U3EtJB+1rRiE95FLjEwpR3+5WdHbyB66OvOMX1bANRfEo0XAVH//FbAGRIJgtmiqHVD6agNHt6fsMfEYus40eXw9CxgypLQnK08nlb8qPz8pFx4irbX2pP+YDmoyJpnETNq6LSIRk= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1773290748; c=relaxed/simple; bh=aOmwBif/DGTbfASibfq5vRLTbnip82u0RYKm4BbNA34=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=h4Xg9Xf28aCcPgsQpkOnWaJcM+o+WC5xNAz1rB70Jpjh9EgE8cidIZY61TtPkLIAlyTrdzzPwe7XKqvz4kd3bz4olOM4uQL/AAXg0DOyHsylr0tlCUUWdwril5AW0HlEIENsKoSBH7Fhq3s5nRTUIy1GcNCQJUcLSvlrmBO9H18= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com; spf=fail smtp.mailfrom=amd.com; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b=ysMVuCqU; arc=fail smtp.client-ip=52.101.48.63 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=amd.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b="ysMVuCqU" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=WidsOc9TUvMlQ5Qjenz1lZlDR/U7HHMNMCB+mdhPXo78fu72mET/n0M2mOBxjqQbpCFxZSMIUYBvt3NOwO8c367LKIMgk77B1iBWMTglzWxUd4xgkBIaWc4hvOgoT/HK4joJHKW5wpG64mVjHudycPyFDgKzYnjQnEGcnkpAEDwwsJaQ5BB0CQONVZ/mv6vCNeIP4n8t2de/ExFdvF/3YJqyJl1obFx6t3ehwkJMSMdzkMUGmK3KO/RBrieTTuTAZu4LAoWwVyeYh/zeI+OvQ3e5pqCqFOHGplNPi0tAvDkxgHlYC5A+pqMMYWmf08Lb+yvn5srCtH3AWlUjbsequw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=1EIHRon7Ebi++wu9JMXzmPoiJVPvd3G3OWpRiQ6UUkw=; b=XY+se+5vsbmE1OvB2a5M7q/vn7r3WSJd99lBn9Su8p6z8Hvy57K590V2I+bClZkjJU9X0wauaLgsUfJB94yLk5wrDW8zWrkKjRQP2JV9yk2L7ogxpc2PboImumj8oVgvze3mXhCx3EbFvY4De88l5RyBzy6IhpXogPEz3zHd7hTdJBQxkvur/+r1wAj3z+z9qawJoM7lNc23IooaCSW5wwagtXnb16vviM8yQo3wj89lgPlAsgNqBPui3uHdD4dXSLVTM7zlN8TrXqf5UBjl77cMPib+Z8kJ4PTEudaiDTdG8m93m0RZV0EYVan2P27VoXUqZ8GPHTP8kQD2kCo4sQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=redhat.com smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=1EIHRon7Ebi++wu9JMXzmPoiJVPvd3G3OWpRiQ6UUkw=; b=ysMVuCqUdENA/TmcotsOzKiOzcShRt1w1Sy/9FvWnrw7eUtFepSLU0LAuAVty8bh//OKMawSiNskM4EEld3Sh8dtVB7fTCHq15/2gkDlh4RxYsqxdM2kTq4j+aFL7S6kRX7R75KysSQ+wHYatVCgxYq8hMjfc/J6bdmJ88UykpY= Received: from BY3PR05CA0015.namprd05.prod.outlook.com (2603:10b6:a03:254::20) by SAWPR12MB999142.namprd12.prod.outlook.com (2603:10b6:806:4e1::12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9723.6; Thu, 12 Mar 2026 04:45:44 +0000 Received: from MWH0EPF000C6188.namprd02.prod.outlook.com (2603:10b6:a03:254:cafe::5c) by BY3PR05CA0015.outlook.office365.com (2603:10b6:a03:254::20) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.9678.23 via Frontend Transport; Thu, 12 Mar 2026 04:45:44 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=satlexmb07.amd.com; pr=C Received: from satlexmb07.amd.com (165.204.84.17) by MWH0EPF000C6188.mail.protection.outlook.com (10.167.249.120) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9678.18 via Frontend Transport; Thu, 12 Mar 2026 04:45:44 +0000 Received: from BLRKPRNAYAK.amd.com (10.180.168.240) by satlexmb07.amd.com (10.181.42.216) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.17; Wed, 11 Mar 2026 23:45:39 -0500 From: K Prateek Nayak To: Ingo Molnar , Peter Zijlstra , Juri Lelli , Vincent Guittot , Valentin Schneider , CC: Dietmar Eggemann , Steven Rostedt , Ben Segall , Mel Gorman , Chen Yu , Shrikanth Hegde , Li Chen , "Gautham R. Shenoy" , K Prateek Nayak Subject: [PATCH v4 3/9] sched/topology: Allocate per-CPU sched_domain_shared in s_data Date: Thu, 12 Mar 2026 04:44:28 +0000 Message-ID: <20260312044434.1974-4-kprateek.nayak@amd.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20260312044434.1974-1-kprateek.nayak@amd.com> References: <20260312044434.1974-1-kprateek.nayak@amd.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: satlexmb07.amd.com (10.181.42.216) To satlexmb07.amd.com (10.181.42.216) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: MWH0EPF000C6188:EE_|SAWPR12MB999142:EE_ X-MS-Office365-Filtering-Correlation-Id: 93eb87c7-3bf3-40a3-87d5-08de7ff238bd X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|1800799024|36860700016|82310400026|7416014|376014|22082099003|56012099003|18002099003; X-Microsoft-Antispam-Message-Info: hjZUraF/D2Hbfz5zShuuTPsQNr3ooTXRu26YMCY9GVfPPJjs5V3xtgbVZ6jeIbK2iqQAd3fa1en5s2u/uyY+8LsGRws5RX0/bkJh+nJqUbKuIwjn38xFeksjG7kQc37gf7Iq30WYj7pnDV3a660RiQ2WKKf+6++DI8D5pc/cgSLWpQv/6pU7ZR/Rek+UbRFP65j4mYIeDc3PQNoWjPZsni9XKcTDOH1GAjCWbPJKiFkoHzozfc01E97HzC24K+wrzG4eG3QcIRumP68tqypbkeHljNy/GyeKzY5+WGeOWimDrvqgwu5CxVhJ1/+P8pkYg0B+JZ2fb6mwyfp6R5CGi3xnwzQbKDP31+0PsAgiEvbCEy+23SCXWEN/rYIRMPtAnZRoJs8vZAK+AeGZHZbH5joNItILugQkn2tS+lNj7MeIcg1H/gP5IMNRTw4fZV+cr9jh1pA4htIY4EFKt79dB3piI3Y8UVWmxxa7plZbyKCDZdTxB6gQ3fpDPXaUF5cnGbWtnykwKbq64WDfR5cOi1C6rj55HUF2hXCQrKJlU+Cjjmt0fdHB9zh0xwMSMqDSPSrnWupmk5zGHGeP+AGoN4haLsP2fMuEIcJRJBkVne2E9CyaFvtmF1OkSYC6CUt2Nl9Rfc8Iq8CjIUnkiXnx7wNswh4sa0EeC42zNCeSSvpVdu4FcTlBes4ObkzFl62iW5bYhec5GyjaEJwJzpe3XUGdwR2pWYC9i7VlAfPKV/Dsl4N0tSer+aAEZde/yldR5nf9HpXwPKT4xmpekW7GMA== X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:satlexmb07.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230040)(1800799024)(36860700016)(82310400026)(7416014)(376014)(22082099003)(56012099003)(18002099003);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: Cz2aDHOAZ+inwG1xIDG8y4gYsett6ZVDr873fucC68PrXPG1pmXJzPxXy+dOjzZo5/YPLjVa5y2GsrdOkOGIQB5klybGXZII2niTgzZ4VJE/DhsgOwmbEumlRDqLFQfQTzAz9SipgBEmkFWD9uJ2oXh1IZEtXMK9HYe4oIb5+9tBR2A5UQd45HkuG+9nupQe6Ax083XsGhthEst+nF4GhcIh0/Ht6Q5nFgbX9SsQOo0KIpKEJObGPjO/3hlzk3oaThy5oeD+4a78RHj0wcp82e//4HrfMd6EtfXWc57MS9BWDjyVHy+sh6QxWKmCVMDB+0+wzYSC0Hpfbz0eFoFtWE9LjsPvcU6gIdrm1Gzl/iTxh0On+mMpcwPCcBYVG94DhCJyuS93ECcZ7mrIRWMQvz77ErUgsLA32Hd0dIoOBMmgUms2y0bi5O52rxYgUEVZ X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 12 Mar 2026 04:45:44.3213 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 93eb87c7-3bf3-40a3-87d5-08de7ff238bd X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[satlexmb07.amd.com] X-MS-Exchange-CrossTenant-AuthSource: MWH0EPF000C6188.namprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: SAWPR12MB999142 Content-Type: text/plain; charset="utf-8" The "sched_domain_shared" object is allocated for every topology level in __sdt_alloc() and is freed post sched domain rebuild if they aren't assigned during sd_init(). "sd->shared" is only assigned for SD_SHARE_LLC domains and out of all the assigned objects, only "sd_llc_shared" is ever used by the scheduler. Since only "sd_llc_shared" is ever used, and since SD_SHARE_LLC domains never overlap, allocate only a single range of per-CPU "sched_domain_shared" object with s_data instead of doing it per topology level. The subsequent commit uses the degeneration path to correctly assign the "sd->shared" to the topmost SD_SHARE_LLC domain. No functional changes are expected at this point. Reviewed-by: Valentin Schneider Reviewed-by: Chen Yu Signed-off-by: K Prateek Nayak Reviewed-by: Dietmar Eggemann Tested-by: Dietmar Eggemann --- Changelog v3..v4: o Collected tags from Valentin, and Chenyu. (Thanks a ton!) --- kernel/sched/topology.c | 48 ++++++++++++++++++++++++++++++++++++++++- 1 file changed, 47 insertions(+), 1 deletion(-) diff --git a/kernel/sched/topology.c b/kernel/sched/topology.c index 7f25c784c038..f0541c6511fa 100644 --- a/kernel/sched/topology.c +++ b/kernel/sched/topology.c @@ -782,6 +782,7 @@ cpu_attach_domain(struct sched_domain *sd, struct root_= domain *rd, int cpu) } =20 struct s_data { + struct sched_domain_shared * __percpu *sds; struct sched_domain * __percpu *sd; struct root_domain *rd; }; @@ -789,6 +790,7 @@ struct s_data { enum s_alloc { sa_rootdomain, sa_sd, + sa_sd_shared, sa_sd_storage, sa_none, }; @@ -1535,6 +1537,9 @@ static void set_domain_attribute(struct sched_domain = *sd, static void __sdt_free(const struct cpumask *cpu_map); static int __sdt_alloc(const struct cpumask *cpu_map); =20 +static void __sds_free(struct s_data *d, const struct cpumask *cpu_map); +static int __sds_alloc(struct s_data *d, const struct cpumask *cpu_map); + static void __free_domain_allocs(struct s_data *d, enum s_alloc what, const struct cpumask *cpu_map) { @@ -1546,6 +1551,9 @@ static void __free_domain_allocs(struct s_data *d, en= um s_alloc what, case sa_sd: free_percpu(d->sd); fallthrough; + case sa_sd_shared: + __sds_free(d, cpu_map); + fallthrough; case sa_sd_storage: __sdt_free(cpu_map); fallthrough; @@ -1561,9 +1569,11 @@ __visit_domain_allocation_hell(struct s_data *d, con= st struct cpumask *cpu_map) =20 if (__sdt_alloc(cpu_map)) return sa_sd_storage; + if (__sds_alloc(d, cpu_map)) + return sa_sd_shared; d->sd =3D alloc_percpu(struct sched_domain *); if (!d->sd) - return sa_sd_storage; + return sa_sd_shared; d->rd =3D alloc_rootdomain(); if (!d->rd) return sa_sd; @@ -2466,6 +2476,42 @@ static void __sdt_free(const struct cpumask *cpu_map) } } =20 +static int __sds_alloc(struct s_data *d, const struct cpumask *cpu_map) +{ + int j; + + d->sds =3D alloc_percpu(struct sched_domain_shared *); + if (!d->sds) + return -ENOMEM; + + for_each_cpu(j, cpu_map) { + struct sched_domain_shared *sds; + + sds =3D kzalloc_node(sizeof(struct sched_domain_shared), + GFP_KERNEL, cpu_to_node(j)); + if (!sds) + return -ENOMEM; + + *per_cpu_ptr(d->sds, j) =3D sds; + } + + return 0; +} + +static void __sds_free(struct s_data *d, const struct cpumask *cpu_map) +{ + int j; + + if (!d->sds) + return; + + for_each_cpu(j, cpu_map) + kfree(*per_cpu_ptr(d->sds, j)); + + free_percpu(d->sds); + d->sds =3D NULL; +} + static struct sched_domain *build_sched_domain(struct sched_domain_topolog= y_level *tl, const struct cpumask *cpu_map, struct sched_domain_attr *attr, struct sched_domain *child, int cpu) --=20 2.34.1 From nobody Tue Apr 7 18:35:25 2026 Received: from BN8PR05CU002.outbound.protection.outlook.com (mail-eastus2azon11011033.outbound.protection.outlook.com [52.101.57.33]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4C0D62C0F6F for ; Thu, 12 Mar 2026 04:46:07 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=52.101.57.33 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1773290769; cv=fail; b=e5zn6TeEqce1jWPtOfkW2cYtxcMSZlwysqxNo6FkdL7ARJA3P2zLt4FX1HVq8Ut4aCVWTAOlC+WKhZQ+xaQ5n/W4qVCskZMpNnI2dOzymprr4tR4oOig4s5mTp/0VwvBrNdfGHs73AHRHDwTLsvwasxBmizeQTNfhxrF52nGpX0= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1773290769; c=relaxed/simple; bh=p3uBEeUqyZKjonLGMoKue49qOA/3G9jJwDK9psnr/SQ=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=k6dRGElZL1ugQR3P6ELslQvb+d9wGKekXypBxeTJE8ao8oXMTY3NODTWpF8Ko/32kS5AQaYaGeh5PI3FlvZ+ru1ZUUre8YalzxnRB9m7FrvISqIDmySe0TP/VMENkgOAOs8bx9WomHhWo46jxlGYlysVontDykICplFzJwF35G4= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com; spf=fail smtp.mailfrom=amd.com; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b=x24cqun9; arc=fail smtp.client-ip=52.101.57.33 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=amd.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b="x24cqun9" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=wFmUAI7LprAdi27TOqnaaf1376hjfZ5RcZmqBNPVKpqLYJFkZfbBsS+htylRct8pBzhnuTGCmmVsy6lyEK9Gduv9iRr1BarQsuz5XhHSbV3cn7c3cS4eP6eGVLfP+vVO4AGY81x1gzbZT83HWTvL/o1LSjEsPvplNcHi2V56Nxr8HuGt2gtD4Er9OzfngFpBxCMU9KuGAeSGp8i2x0swNT1Wnj2bPano9C6wplWaR5gHxBGoRk9wfbtqCzyVJ2dMIkFC5B5Kq/mI/tHoYiBSnKc3ofKTzObdILohGWoNeAcJTNIzUTUhd7+Y9DvO0cfyCMt+XdCoZeoy4nSSOWSmbw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=3h1acG/U+8bWFA699D5Axzf2oLX1SjhQTHmRxsQWaVM=; b=Kvri8maX60zTkV8gHQ+afu5gY32t0tYdwibgoZTnZj1uA4MGk5Ny+TuOZOt9zB2hcYPug5KA+DsuC1qpAbXL44bc4uj2gRygf5bLfQAOc+VubZLbfrzz3oh9narUP2rr3+DHSR8sM0m9AxnBv6Uidt3eJ3LkcBShEYEioEJA4uxB0bnEbjZRTTXBQqzRIIToqRZXgHdHGzREyPzdPwlWk2+E/cwuZDFQMib8Ah25/b5EH81UFEsUTbGW1Y4oKipR2tIXN1hXlXPopLI+iwgfu0wMZ7J+T5HR/zZX3QR//fgLZmhkDlTDZ9MAI25yt8mNWwK0Sp/nDoe4rue9VHtfaw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=redhat.com smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=3h1acG/U+8bWFA699D5Axzf2oLX1SjhQTHmRxsQWaVM=; b=x24cqun9H+dwG1ARB1ky8GUsNrK/U6W/usuCz+/eMufYnLtUaX6XnKSPHGQtwHIQ1y7Bo7WMJBKMNbudM404O9IRQ9W8bSA1RzdhR6bsS2KLnkpucQk4fwU0SOgi2yZcwQhGkcnZ49ukgrYSSOEL8++bn9Xt+o3o76vU2OVIO4Q= Received: from BY3PR05CA0058.namprd05.prod.outlook.com (2603:10b6:a03:39b::33) by DS0PR12MB7747.namprd12.prod.outlook.com (2603:10b6:8:138::20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9723.6; Thu, 12 Mar 2026 04:46:02 +0000 Received: from MWH0EPF000C6187.namprd02.prod.outlook.com (2603:10b6:a03:39b:cafe::c6) by BY3PR05CA0058.outlook.office365.com (2603:10b6:a03:39b::33) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.9678.24 via Frontend Transport; Thu, 12 Mar 2026 04:46:02 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=satlexmb07.amd.com; pr=C Received: from satlexmb07.amd.com (165.204.84.17) by MWH0EPF000C6187.mail.protection.outlook.com (10.167.249.119) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9678.18 via Frontend Transport; Thu, 12 Mar 2026 04:46:02 +0000 Received: from BLRKPRNAYAK.amd.com (10.180.168.240) by satlexmb07.amd.com (10.181.42.216) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.17; Wed, 11 Mar 2026 23:45:56 -0500 From: K Prateek Nayak To: Ingo Molnar , Peter Zijlstra , Juri Lelli , Vincent Guittot , Valentin Schneider , CC: Dietmar Eggemann , Steven Rostedt , Ben Segall , Mel Gorman , Chen Yu , Shrikanth Hegde , Li Chen , "Gautham R. Shenoy" , K Prateek Nayak Subject: [PATCH v4 4/9] sched/topology: Switch to assigning "sd->shared" from s_data Date: Thu, 12 Mar 2026 04:44:29 +0000 Message-ID: <20260312044434.1974-5-kprateek.nayak@amd.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20260312044434.1974-1-kprateek.nayak@amd.com> References: <20260312044434.1974-1-kprateek.nayak@amd.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: satlexmb07.amd.com (10.181.42.216) To satlexmb07.amd.com (10.181.42.216) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: MWH0EPF000C6187:EE_|DS0PR12MB7747:EE_ X-MS-Office365-Filtering-Correlation-Id: 06349fca-5f73-4d9b-d44a-08de7ff24386 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|1800799024|36860700016|82310400026|7416014|376014|22082099003|56012099003|18002099003; X-Microsoft-Antispam-Message-Info: Xw+ibPFyQIl6ccrjxTKtV8VTCmMhCGBCAIxSOy4/1/nvs+s7Ntv6bqJQW23GQ1CZDj5XZU8Cv0dcXHg6LoY/f82UCkoEMueZ+pAH6aScqnjWGdKnTOyI8VXZZoRzlNm5eaAbCILgi0tL/MJlwDDSyCflYepG41TmVeFybYeXAfqIoqwWtRqjShwOmU9bj15fy5n5TBPHh343h3xqKjT4FlX/BIVvAuI3B/FtSPn5qQYL1J+uXeC1KMAxP/ddJr9sLJedHMDjYFzxOUgWWhIVczJTvsrMbTAf8bIa3dxXDKzpaQ2LtaCgz6vAIiQ79AO9hg45Mo8RIHh1j2/hk4U6Qze+8zORJBIbGpW+KsSAhz7mx8aLO7o2KbEchYHZBBkQEAX81xN9SqbPyJi3ckE1IEr5j+POV/JkqhWHGNRSRcE6K1+IV3VJK3O7S87bCob4J9iTvXX26dbzdw4nX+yXkWbHaOjuXCtZG6uYLERbbX6gXXwInyASvaz9gTOnZzjzwGJJiH3azMbJQo9XdnJRMEEpW8DDJWMVwfoa4JOzsuDpets1E7ctQbYVCPAWdtKTEt3rkKzQg7NfzwkW5vOCZqe7zsy4dpGRP3RfCHkAMjBTVDvvoottvLHRnBUQNlyuYHoJpSKj6PWMC72YHQUN22smn0gcLR/xq3QHeYUkATI4cNZ3p4CwBUWTRfFvda5uHS5/aaLGTySbn/jmA1ryrLd7ANsZVVI6Pvo/FDnTRp+ZQLq3+kKUWixc6fRrJM96TsgwS7qQY4vL8G/uKscQ0w== X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:satlexmb07.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230040)(1800799024)(36860700016)(82310400026)(7416014)(376014)(22082099003)(56012099003)(18002099003);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: ZXZkBLRDQ7CTRur6lqmhRbTs6FIrfCv91QjPpSpjgtxwTfdcr+XtkhQ7LY3fki01L3xPm/H3QtS3RMpcHUE/cB+gofN1P7c6eMbX5nOfDxJPQz/v3dCuftqir+1DhlJFJQD1QLLgxMvi35mwbnERFVBDAG12F689mIEQbHi0URWc09w5EyI44AfEZsP7tetOyRCP2m2Nu2aNoT3V9EPBQWpbtz6Kf/p+ckWF0ePiFwwEEMn+QhTjQBPPHQxUxfLkQmY+RoOcZjy3uRrSz7QEDmhE5qzzuHklFQc5DH/jzxLx366oVEYJ6xkmMHRnTHY/AiSNwbN6tHqRq+AU1+Asehljy2u5WsXv6b1BAp4cxALXFJRgO/PAFRZFlO7qCzmV8wPpjbTDOnhH8bgKrkyO2S18QSU8nul1qnXVWthK7qFu8d6NoXDlx7gLXeExSfCX X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 12 Mar 2026 04:46:02.3571 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 06349fca-5f73-4d9b-d44a-08de7ff24386 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[satlexmb07.amd.com] X-MS-Exchange-CrossTenant-AuthSource: MWH0EPF000C6187.namprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DS0PR12MB7747 Content-Type: text/plain; charset="utf-8" Use the "sched_domain_shared" object allocated in s_data for "sd->shared" assignments. Assign "sd->shared" for the topmost SD_SHARE_LLC domain before degeneration and rely on the degeneration path to correctly pass down the shared object to "sd_llc". sd_degenerate_parent() ensures degenerating domains must have the same sched_domain_span() which ensures 1:1 passing down of the shared object. If the topmost SD_SHARE_LLC domain degenerates, the shared object is freed from destroy_sched_domain() when the last reference is dropped. claim_allocations() NULLs out the objects that have been assigned as "sd->shared" and the unassigned ones are freed from the __sds_free() path. To keep all the claim_allocations() bits in one place, claim_allocations() has been extended to accept "s_data" and iterate the domains internally to free both "sched_domain_shared" and the per-topology-level data for the particular CPU in one place. Post cpu_attach_domain(), all reclaims of "sd->shared" are handled via call_rcu() on the sched_domain object via destroy_sched_domains_rcu(). Signed-off-by: K Prateek Nayak Reviewed-by: Dietmar Eggemann Tested-by: Dietmar Eggemann --- Changelog v3..v4: o Moved claiming the per-CPU "d.sds" reference into claim_allocations() to keep everything in one place. (Shrikanth) o Slightly different diff as a result of moving the "imb_numa_nr" calculation into a separate helper in Patch 2. --- kernel/sched/topology.c | 73 +++++++++++++++++++++++++---------------- 1 file changed, 44 insertions(+), 29 deletions(-) diff --git a/kernel/sched/topology.c b/kernel/sched/topology.c index f0541c6511fa..ebd955faab40 100644 --- a/kernel/sched/topology.c +++ b/kernel/sched/topology.c @@ -685,6 +685,9 @@ static void update_top_cache_domain(int cpu) if (sd) { id =3D cpumask_first(sched_domain_span(sd)); size =3D cpumask_weight(sched_domain_span(sd)); + + /* If sd_llc exists, sd_llc_shared should exist too. */ + WARN_ON_ONCE(!sd->shared); sds =3D sd->shared; } =20 @@ -733,6 +736,13 @@ cpu_attach_domain(struct sched_domain *sd, struct root= _domain *rd, int cpu) if (sd_parent_degenerate(tmp, parent)) { tmp->parent =3D parent->parent; =20 + /* Pick reference to parent->shared. */ + if (parent->shared) { + WARN_ON_ONCE(tmp->shared); + tmp->shared =3D parent->shared; + parent->shared =3D NULL; + } + if (parent->parent) { parent->parent->child =3D tmp; parent->parent->groups->flags =3D tmp->flags; @@ -1586,21 +1596,28 @@ __visit_domain_allocation_hell(struct s_data *d, co= nst struct cpumask *cpu_map) * sched_group structure so that the subsequent __free_domain_allocs() * will not free the data we're using. */ -static void claim_allocations(int cpu, struct sched_domain *sd) +static void claim_allocations(int cpu, struct s_data *d) { - struct sd_data *sdd =3D sd->private; + struct sched_domain *sd; + + if (atomic_read(&(*per_cpu_ptr(d->sds, cpu))->ref)) + *per_cpu_ptr(d->sds, cpu) =3D NULL; =20 - WARN_ON_ONCE(*per_cpu_ptr(sdd->sd, cpu) !=3D sd); - *per_cpu_ptr(sdd->sd, cpu) =3D NULL; + for (sd =3D *per_cpu_ptr(d->sd, cpu); sd; sd =3D sd->parent) { + struct sd_data *sdd =3D sd->private; =20 - if (atomic_read(&(*per_cpu_ptr(sdd->sds, cpu))->ref)) - *per_cpu_ptr(sdd->sds, cpu) =3D NULL; + WARN_ON_ONCE(*per_cpu_ptr(sdd->sd, cpu) !=3D sd); + *per_cpu_ptr(sdd->sd, cpu) =3D NULL; =20 - if (atomic_read(&(*per_cpu_ptr(sdd->sg, cpu))->ref)) - *per_cpu_ptr(sdd->sg, cpu) =3D NULL; + if (atomic_read(&(*per_cpu_ptr(sdd->sds, cpu))->ref)) + *per_cpu_ptr(sdd->sds, cpu) =3D NULL; =20 - if (atomic_read(&(*per_cpu_ptr(sdd->sgc, cpu))->ref)) - *per_cpu_ptr(sdd->sgc, cpu) =3D NULL; + if (atomic_read(&(*per_cpu_ptr(sdd->sg, cpu))->ref)) + *per_cpu_ptr(sdd->sg, cpu) =3D NULL; + + if (atomic_read(&(*per_cpu_ptr(sdd->sgc, cpu))->ref)) + *per_cpu_ptr(sdd->sgc, cpu) =3D NULL; + } } =20 #ifdef CONFIG_NUMA @@ -1740,16 +1757,6 @@ sd_init(struct sched_domain_topology_level *tl, sd->cache_nice_tries =3D 1; } =20 - /* - * For all levels sharing cache; connect a sched_domain_shared - * instance. - */ - if (sd->flags & SD_SHARE_LLC) { - sd->shared =3D *per_cpu_ptr(sdd->sds, sd_id); - atomic_inc(&sd->shared->ref); - atomic_set(&sd->shared->nr_busy_cpus, sd_weight); - } - sd->private =3D sdd; =20 return sd; @@ -2731,12 +2738,20 @@ build_sched_domains(const struct cpumask *cpu_map, = struct sched_domain_attr *att while (sd->parent && (sd->parent->flags & SD_SHARE_LLC)) sd =3D sd->parent; =20 - /* - * In presence of higher domains, adjust the - * NUMA imbalance stats for the hierarchy. - */ - if (IS_ENABLED(CONFIG_NUMA) && (sd->flags & SD_SHARE_LLC) && sd->parent) - adjust_numa_imbalance(sd); + if (sd->flags & SD_SHARE_LLC) { + int sd_id =3D cpumask_first(sched_domain_span(sd)); + + sd->shared =3D *per_cpu_ptr(d.sds, sd_id); + atomic_set(&sd->shared->nr_busy_cpus, sd->span_weight); + atomic_inc(&sd->shared->ref); + + /* + * In presence of higher domains, adjust the + * NUMA imbalance stats for the hierarchy. + */ + if (IS_ENABLED(CONFIG_NUMA) && sd->parent) + adjust_numa_imbalance(sd); + } } =20 /* Calculate CPU capacity for physical packages and nodes */ @@ -2744,10 +2759,10 @@ build_sched_domains(const struct cpumask *cpu_map, = struct sched_domain_attr *att if (!cpumask_test_cpu(i, cpu_map)) continue; =20 - for (sd =3D *per_cpu_ptr(d.sd, i); sd; sd =3D sd->parent) { - claim_allocations(i, sd); + claim_allocations(i, &d); + + for (sd =3D *per_cpu_ptr(d.sd, i); sd; sd =3D sd->parent) init_sched_groups_capacity(i, sd); - } } =20 /* Attach the domains */ --=20 2.34.1 From nobody Tue Apr 7 18:35:25 2026 Received: from BYAPR05CU005.outbound.protection.outlook.com (mail-westusazon11010018.outbound.protection.outlook.com [52.101.85.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8871D302CD5 for ; Thu, 12 Mar 2026 04:46:23 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=52.101.85.18 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1773290784; cv=fail; b=tfmO8X+kWK3XIddGtFOfbcQ8LQ2IIz0+6NkbY65Bq7Zs7Apcgl21O+XxbLo8c22b6bZGDVQadDw7N3SPBNHhAsy/+sTcLuULKq/ukjLmGWgSdW7PDdRtUNAYskiaxMA2pkggrb523eHRixe7m5Zjrr2f5QCos82h3ASkkMmS7tg= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1773290784; c=relaxed/simple; bh=YbMChKSN1xitcCNMqxrmFSRzfDnpQmk88nYRUVi4po4=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=obF1VzfKafRHoJQBE9GB2A194KGSLoT7/jFygGTK/zVqth4m6p/3pn29dJ45UumhA/6ylRCV5BTSRCgrFrBFFl7rzez+3AAeE04a/Xn7PZaWThvptVuPxo3gvApx2u+/EJIL2kmAyiUvJSSA7ZFP2gpQ3DZn1syyWBrD/2tI/T8= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com; spf=fail smtp.mailfrom=amd.com; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b=lDt8kvWp; arc=fail smtp.client-ip=52.101.85.18 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=amd.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b="lDt8kvWp" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=m7kxF/iSQphNWzmO9dzNNyB5OsP9wcrkVNwhk0ZtSBWVtQzV45N/nZ4w06GS0AjyB7fTcWNhXeviOH5aLa4ws9FCOGKykqYO63LUYDyIGPaDWqbY8TSpxswui0rNlziuIhMjwrNGCdeswrsNiDVCfA0R6xyrmVfA6T1GGvg60K8JPlXqQdzGRB/Xon+F2f6ifL9UXtNLp6YUSYeQ3sZerHV6vwWyQ/1f2MqAHsZOX136g4ZW+XPZ5nAI1Hf3THhTJNJaoKajUFoA4eCeuq1RQhIVdcR9jZ9SfJuIeevCyHgczeTkaAHQu/Gk3UohSeVNrrciuyuyi72QWK4mnGGGjw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=aIKQ3PAY/DAOorp7ADSZNYihdJx7RSKOjuUmJDyyBG0=; b=X/0wTECU6WeNgGEbNJpIC4NBdtZW77AeFTNs9MIWLou3ETfWG+ENflUEheOg1jWkjBX5Jukxny8N66btuiW0HOyHZVIq9ayGNjrAsfUscwuBDx1E6QNGtUYG0mEqepb19R8DmqALho2YH5gSwOFj2syrwj655Bk1P+hIRVfUJd2sAJL4xQn+LDwrD3nxq7zdqyOM1bSYJ0SXK4Umgi9zcDWIjGCh5i/+8LOoqsDVr4sRHvy/rVskNDtfBIten6svdaR2TpWjfJIa/K6Q2dS+RSCMTUYqROxrDTp6Fzcbrxu8USMSz5vYIE6Rx0kdNgoyyAcFc+StFjGbCM/xko/Gkw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=redhat.com smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=aIKQ3PAY/DAOorp7ADSZNYihdJx7RSKOjuUmJDyyBG0=; b=lDt8kvWpfNUQxLpGQ+nfECYIVj+aV15aqTR1xzp0VheDc/Ubd2zsEYsrYi7tppU2ne7Z6h+52hW5gprpe3u+dtXSDEKbldSbCCTgu7XqIQhqFzTVG6ODD+CB7XLtKEIPohbX0kRE0VnDI3yTgEDu8RqbvNk5V6EOOuIAoCnCLDI= Received: from SJ0PR05CA0067.namprd05.prod.outlook.com (2603:10b6:a03:332::12) by SN7PR12MB6792.namprd12.prod.outlook.com (2603:10b6:806:267::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9723.6; Thu, 12 Mar 2026 04:46:20 +0000 Received: from MWH0EPF000C6185.namprd02.prod.outlook.com (2603:10b6:a03:332:cafe::e0) by SJ0PR05CA0067.outlook.office365.com (2603:10b6:a03:332::12) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.9678.25 via Frontend Transport; Thu, 12 Mar 2026 04:46:19 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=satlexmb07.amd.com; pr=C Received: from satlexmb07.amd.com (165.204.84.17) by MWH0EPF000C6185.mail.protection.outlook.com (10.167.249.117) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9678.18 via Frontend Transport; Thu, 12 Mar 2026 04:46:19 +0000 Received: from BLRKPRNAYAK.amd.com (10.180.168.240) by satlexmb07.amd.com (10.181.42.216) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.17; Wed, 11 Mar 2026 23:46:14 -0500 From: K Prateek Nayak To: Ingo Molnar , Peter Zijlstra , Juri Lelli , Vincent Guittot , Valentin Schneider , CC: Dietmar Eggemann , Steven Rostedt , Ben Segall , Mel Gorman , Chen Yu , Shrikanth Hegde , Li Chen , "Gautham R. Shenoy" , K Prateek Nayak Subject: [PATCH v4 5/9] sched/topology: Remove sched_domain_shared allocation with sd_data Date: Thu, 12 Mar 2026 04:44:30 +0000 Message-ID: <20260312044434.1974-6-kprateek.nayak@amd.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20260312044434.1974-1-kprateek.nayak@amd.com> References: <20260312044434.1974-1-kprateek.nayak@amd.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: satlexmb07.amd.com (10.181.42.216) To satlexmb07.amd.com (10.181.42.216) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: MWH0EPF000C6185:EE_|SN7PR12MB6792:EE_ X-MS-Office365-Filtering-Correlation-Id: 2f57330c-aca8-475a-5079-08de7ff24dae X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|82310400026|1800799024|7416014|376014|36860700016|22082099003|18002099003|56012099003; X-Microsoft-Antispam-Message-Info: LXQQCYz580930IDMP9hjPbDv8xrcOEm7hZTEN6QXatLTJChaxgjj47DdfXhWuEinMAmTqHlPWvF3dAAienJ4ZapiUbY7zmTOPrSQXBcczEeV7btPxn6PfT+DN4uzir8m85eHrhns0zP4Gq3kp3rovuGTaUElTe+nCK1I+7x5CS02AdRS3Q2K597FYfE7l327g3QqTghxH9U70Sjfcptwena9gc2Z/het2KHdJc8KKV37/u7z9RcMZ2lIP7lgYVP9pBmMOgoqmiRULd6kUMBVxDKYGoB6UKQOnBGAO0S1CkxkF1byE9Kx11z8livWdYNGrigIBlgSmRY7hIXvryXivZw5XoB965mh+LRDoz/viUOstukxW62BAmyTIMDigq4SdNAb5+RKJuwYtGaK9LLPkxvHWKQHs13P6aMsh3TZkjwQNCStDYMW1myXD6znBpJG9ZYaSOD3jD9DSsxAnPTQ2RoeYTBG1cEsOKMnJikVGqdTqd41BwLc4S5ZQ4Y1GVUPHLVpeEUlORamhvo1c8FYK1UKIASlnqN3zfuy70qJ1NU28ZPLq49QwdDJvkF4cNcsRwM+lEJDVOkM0PpUI2H7/DTygEfuHFRj0z2LIzqv35bZF228ErxT7CpuSN57Qd3oMjA4EcjzcgorTbp9psGmCydxNif1Mh5ClXAlo5Vz8+tqmmJRpw8rrG+ZQLptqhPQ5ZM16klSzwdVIgyO9X/KXv3e2SPex6SXvZSQP5Bt9NI/asqdm8y/m6/bRWttVCxnbXjlNgt59Ongl87m8ExL4A== X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:satlexmb07.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230040)(82310400026)(1800799024)(7416014)(376014)(36860700016)(22082099003)(18002099003)(56012099003);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: E08hHAGdaEILdXhrXjOqjpuYvne8jO5jQbQaVww2our30GBeBlIHCc/c0SDJ1HaUnhrJMmtu9BYVrDNKMP/0rlfaRsD4piIG0tLsxCtxPzjfKV/4eEbPO2//6MNsK+bwhdtqizLFoY0o/jiDM+oxFNPFNdfAKT9B8vFKXFAZqg2xXUazBhNvvhb9OR6F4rM+O+U786e3srpNVpvAzGn+ycWCXGNQiRH6zJgYI0rmD5fEyYzoXyBkSDQ+I79lfmkwyqHF70ngJiCMvgwshiJTWzh3DHRxrrYWb/0kAGE/+ZBGvbGWa5lRhjdOwWsdp2V01539v0nhaAOwRieO9sfzlguSK02WasVUT4pbWlNbKX47kxcq++vFrbf+Ga18Iv6chBIf4kuBeV0XeF/OV2VPyL2syUewxmqQ17/KKxO1eR8xsCl/0tIFz3OUgIFZDsoV X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 12 Mar 2026 04:46:19.3928 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 2f57330c-aca8-475a-5079-08de7ff24dae X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[satlexmb07.amd.com] X-MS-Exchange-CrossTenant-AuthSource: MWH0EPF000C6185.namprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: SN7PR12MB6792 Content-Type: text/plain; charset="utf-8" Now that "sd->shared" assignments are using the sched_domain_shared objects allocated with s_data, remove the sd_data based allocations. Reviewed-by: Valentin Schneider Signed-off-by: K Prateek Nayak Reviewed-by: Dietmar Eggemann Tested-by: Dietmar Eggemann --- Changelog v3..v4: o Collected tag from Valentin. (Thanks a ton!) --- include/linux/sched/topology.h | 1 - kernel/sched/topology.c | 19 ------------------- 2 files changed, 20 deletions(-) diff --git a/include/linux/sched/topology.h b/include/linux/sched/topology.h index a1e1032426dc..51c29581f15e 100644 --- a/include/linux/sched/topology.h +++ b/include/linux/sched/topology.h @@ -172,7 +172,6 @@ typedef int (*sched_domain_flags_f)(void); =20 struct sd_data { struct sched_domain *__percpu *sd; - struct sched_domain_shared *__percpu *sds; struct sched_group *__percpu *sg; struct sched_group_capacity *__percpu *sgc; }; diff --git a/kernel/sched/topology.c b/kernel/sched/topology.c index ebd955faab40..963007d83216 100644 --- a/kernel/sched/topology.c +++ b/kernel/sched/topology.c @@ -1609,9 +1609,6 @@ static void claim_allocations(int cpu, struct s_data = *d) WARN_ON_ONCE(*per_cpu_ptr(sdd->sd, cpu) !=3D sd); *per_cpu_ptr(sdd->sd, cpu) =3D NULL; =20 - if (atomic_read(&(*per_cpu_ptr(sdd->sds, cpu))->ref)) - *per_cpu_ptr(sdd->sds, cpu) =3D NULL; - if (atomic_read(&(*per_cpu_ptr(sdd->sg, cpu))->ref)) *per_cpu_ptr(sdd->sg, cpu) =3D NULL; =20 @@ -2392,10 +2389,6 @@ static int __sdt_alloc(const struct cpumask *cpu_map) if (!sdd->sd) return -ENOMEM; =20 - sdd->sds =3D alloc_percpu(struct sched_domain_shared *); - if (!sdd->sds) - return -ENOMEM; - sdd->sg =3D alloc_percpu(struct sched_group *); if (!sdd->sg) return -ENOMEM; @@ -2406,7 +2399,6 @@ static int __sdt_alloc(const struct cpumask *cpu_map) =20 for_each_cpu(j, cpu_map) { struct sched_domain *sd; - struct sched_domain_shared *sds; struct sched_group *sg; struct sched_group_capacity *sgc; =20 @@ -2417,13 +2409,6 @@ static int __sdt_alloc(const struct cpumask *cpu_map) =20 *per_cpu_ptr(sdd->sd, j) =3D sd; =20 - sds =3D kzalloc_node(sizeof(struct sched_domain_shared), - GFP_KERNEL, cpu_to_node(j)); - if (!sds) - return -ENOMEM; - - *per_cpu_ptr(sdd->sds, j) =3D sds; - sg =3D kzalloc_node(sizeof(struct sched_group) + cpumask_size(), GFP_KERNEL, cpu_to_node(j)); if (!sg) @@ -2465,8 +2450,6 @@ static void __sdt_free(const struct cpumask *cpu_map) kfree(*per_cpu_ptr(sdd->sd, j)); } =20 - if (sdd->sds) - kfree(*per_cpu_ptr(sdd->sds, j)); if (sdd->sg) kfree(*per_cpu_ptr(sdd->sg, j)); if (sdd->sgc) @@ -2474,8 +2457,6 @@ static void __sdt_free(const struct cpumask *cpu_map) } free_percpu(sdd->sd); sdd->sd =3D NULL; - free_percpu(sdd->sds); - sdd->sds =3D NULL; free_percpu(sdd->sg); sdd->sg =3D NULL; free_percpu(sdd->sgc); --=20 2.34.1 From nobody Tue Apr 7 18:35:25 2026 Received: from CY3PR05CU001.outbound.protection.outlook.com (mail-westcentralusazon11013002.outbound.protection.outlook.com [40.93.201.2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id BF2252C0F6F for ; Thu, 12 Mar 2026 04:46:41 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.93.201.2 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1773290802; cv=fail; b=gl5GsJ/4GsQYR/ZJqpbYGVL3mGjTfvJ3pNRmHO2SJDN/k1JrL6bBjrwyF0VsqoMJer/MyMGl1PL7wv0zBaDNPahri2L8Oa6N/4RqPqwxkEvZwSR88jzWW9ciCGMvx6G/rkRcozSPuuDxBDG++Il44xvpq9D2u7tpABGgtRJxIn8= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1773290802; c=relaxed/simple; bh=jy9SY8CLF9yCGEIdRbMDmcFSyqA1b5yG3zjIl4PFWZE=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=DpDop6nSMNXHZJx8O5DaxRk0lZf1nu6v7upUGOqbATYOrgPhJPMOJJUQ5xGegMxNXZrPBM2fEeJ1CbWquiWHEon4KDE/yizP7y/OFo5x7ppizJ8Iui3JmQtwXvfnK5LNuKaNO9m9fmO9tYYxOHQzu7pQj6AP3d2En5ClaaSKGw0= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com; spf=fail smtp.mailfrom=amd.com; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b=Q2qMBzHj; arc=fail smtp.client-ip=40.93.201.2 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=amd.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b="Q2qMBzHj" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=pgEiIC1J6OPY2ZwXWTAwZCexqVEJV42cnxhrHVAcUXmyr0fLlA+i4w6eFWAocV2m3w4ASGLnL30AN/G/eTLJo8GRrGvHjUubotZKAJDkk2XVCm75Zmp7yVI8FE0ZO4FTu3t1HlJazrj7A8Fx+JK8FawFH37gPJtUxuC17Gsa5oTd4e9wWzJymnLbMGorJk6X1+/yg0v1va8ukl11RwgeRiCRyTbN8yTKTowvVhqTgR06UdLEgxXtfx4VsfR0DozBtt1AUrRkLVAqxw8mKhujA6Rry0b+gsUQjoTHbU2R3YhyQmFFJmJiCY4t3JEaa2J2XlCZi54/PH/Q4asj5qITwQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=DWAk+Q5zHH83P4KSQDo1cHNjBLZqqYg19aFgBXxiz+s=; b=TqZvlrqsBuLmVmkx70F4fXbBHLfI2xr91VJrW8QwXwTaAMbcENi9ymr7sPNHy4AoJIXUON8P6ujm/Llq6h0/1V0ilvZblLTwiRv20ArbQhRx0dky404+poM1ZAT0jsXAu8n1aL37sbsVPmPr7o0WlmrmgaAVDdmS/oA20fR05DUkJfWZc1tYAFhXhiF6SDm9DKSVU+DIvfgWKaUz+y9ArMOKq0QZEyWhHSGshvvavSYMMaKQLxswERwVKa/KCVqCJtIuXHzdHFy4W9218td+19PetvVga6hVgdCAkou2vEJtJ/SH57LhM3SN4iKnGfv+F6oYUsWKMYhYUmMCMvsB+Q== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=redhat.com smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=DWAk+Q5zHH83P4KSQDo1cHNjBLZqqYg19aFgBXxiz+s=; b=Q2qMBzHjlzE2w9LXdhMWIZyPHpgZY3ssz3vkWyRosgcDpAgxaGTSbha+4/eRXeRso9ob6+PS7U/xe4LQF8gw/QQ5zR+gtG4/4FUFU4jnqLVV6NvigKsnyKBElIHQNHvNwyMPcRplN+j21wNJbrZnAxjxb0Ghi0BAt6A5AXoWRNI= Received: from SJ0PR05CA0081.namprd05.prod.outlook.com (2603:10b6:a03:332::26) by CY3PR12MB9655.namprd12.prod.outlook.com (2603:10b6:930:100::17) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9700.11; Thu, 12 Mar 2026 04:46:39 +0000 Received: from MWH0EPF000C6184.namprd02.prod.outlook.com (2603:10b6:a03:332:cafe::a6) by SJ0PR05CA0081.outlook.office365.com (2603:10b6:a03:332::26) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.9678.25 via Frontend Transport; Thu, 12 Mar 2026 04:46:38 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=satlexmb07.amd.com; pr=C Received: from satlexmb07.amd.com (165.204.84.17) by MWH0EPF000C6184.mail.protection.outlook.com (10.167.249.116) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9678.18 via Frontend Transport; Thu, 12 Mar 2026 04:46:38 +0000 Received: from BLRKPRNAYAK.amd.com (10.180.168.240) by satlexmb07.amd.com (10.181.42.216) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.17; Wed, 11 Mar 2026 23:46:31 -0500 From: K Prateek Nayak To: Ingo Molnar , Peter Zijlstra , Juri Lelli , Vincent Guittot , Valentin Schneider , CC: Dietmar Eggemann , Steven Rostedt , Ben Segall , Mel Gorman , Chen Yu , Shrikanth Hegde , Li Chen , "Gautham R. Shenoy" , K Prateek Nayak Subject: [PATCH v4 6/9] sched/core: Check for rcu_read_lock_any_held() in idle_get_state() Date: Thu, 12 Mar 2026 04:44:31 +0000 Message-ID: <20260312044434.1974-7-kprateek.nayak@amd.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20260312044434.1974-1-kprateek.nayak@amd.com> References: <20260312044434.1974-1-kprateek.nayak@amd.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: satlexmb07.amd.com (10.181.42.216) To satlexmb07.amd.com (10.181.42.216) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: MWH0EPF000C6184:EE_|CY3PR12MB9655:EE_ X-MS-Office365-Filtering-Correlation-Id: ea0e7d97-abb0-4c34-e2b4-08de7ff2592f X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|82310400026|36860700016|1800799024|376014|7416014|56012099003|22082099003|18002099003; X-Microsoft-Antispam-Message-Info: Wam9XPDJrv+Xq8DE5OYQuKxJqn+IEAt3+ypv5+tSn7iqjDq7SMrZpRzzyZ030QcrzfNs/VAOt/zenkrxbvBYxzhcnlm4e/F+Gym3H2KcZvvpmJMIRdDL1QdEpRiEVjzNY4gxrPrFtU9vgn2JZQrfblJh6JQIUTte3mnaprtISPDSSA/dfqcNn2RLL6q7igWcjSxsNH1oKTequP7327MC8OhF0aVUb0fY7r7WtjITI4nutnw4xGzR3l9qda+monm4EArS2tGrY9aRcrHQd6AlhKT6Vg+luBl6LO3MREXwFYKCXBGFh0qPxgMmEggQ1Kvvk9iZSFhp1/vmrlDbikaEwRk89riR13Mmfmf49c0AlvRnjCu5we2nG6At3IHOoaAsQZJI+wZOQPuc0ETEXVft3D2SB1HzaUdRFiP3l8JvgaUnQRKXxicy9TJv6Yx4kpJCEgVZ1fzt7P1kavw6WGLWtQHBaEMO0bfoIyXzdPuuZ/ytrBdEF/PKN5IOa9tGbFRoTfcJn9RoR6M+8bnc7cTDtN2sVnUfqAI73izCu41hSXql0yQvQgRq3EgP1Tnz1mRM+VnVE9IyUPgJ4xMblE4OY1p1WDJv70Wtz1kNmzh3iUzKIYClSl13wZW2p8fw6Pftwf800ee4cQg//stE3EkxWE8QdbetjFMLudLCYBTp9wuoB7d5DIYgarmm+qHJz2wcf4HgGpKC0r37rXOjijMkOwh95eH3qsDPczoUn38GHSXUjnDpodEEg+9ycOH/LGNa0GCYaWFFVAGpdd0b2/6SLA== X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:satlexmb07.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230040)(82310400026)(36860700016)(1800799024)(376014)(7416014)(56012099003)(22082099003)(18002099003);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: p2uXHZ1k4NJj/E91qCXgm7VqLsCltjKO95e5L9NCnQ0/GVH5fXi1+AhFjMVOhTQePn8o5cg0OemVIWP0kW+foJzXJ4R3emnhdzDSgFeoSFC5M/AaQPxwKIe+AqUvAly+GoZaeFkK6Ekuf+t0PyqMeo3F37a+xQ0LNviylE6Fa/gJh0wokR/PbXlqCKu34YkTFOw4Dy6oUIHmdqt75oWw66JCsNjlVwAlX2sh3BIjHW29t6qWGsIDT9qBn69KMFy9UDPVtupjUZ7CXzh2BTmf4SF8L+UX51deehKzbw1wWUu5tm7kPm/6TJfFrsaEuQ7Bu1uSJYFyVgnKgwYFaDcb7arv1BUINM9LhXS89My0jI2HGht5H7WGPTEvdGoQqw2cjIM4RP3ldnaPczxgJQdR7VbZsShj7ZPnvqp1AJJfkc32Huo5xlAYGHD14PpdKvkE X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 12 Mar 2026 04:46:38.7507 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: ea0e7d97-abb0-4c34-e2b4-08de7ff2592f X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[satlexmb07.amd.com] X-MS-Exchange-CrossTenant-AuthSource: MWH0EPF000C6184.namprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY3PR12MB9655 Content-Type: text/plain; charset="utf-8" Similar to commit 71fedc41c23b ("sched/fair: Switch to rcu_dereference_all()"), switch to checking for rcu_read_lock_any_held() in idle_get_state() to allow removing superfluous rcu_read_lock() regions in the fair task's wakeup path where the pi_lock is held and IRQs are disabled. Signed-off-by: K Prateek Nayak Reviewed-by: Dietmar Eggemann Tested-by: Dietmar Eggemann --- Changelog v3..v4: o No changes. --- kernel/sched/sched.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/kernel/sched/sched.h b/kernel/sched/sched.h index 953d89d71804..c4fc7726f82a 100644 --- a/kernel/sched/sched.h +++ b/kernel/sched/sched.h @@ -2853,7 +2853,7 @@ static inline void idle_set_state(struct rq *rq, =20 static inline struct cpuidle_state *idle_get_state(struct rq *rq) { - WARN_ON_ONCE(!rcu_read_lock_held()); + WARN_ON_ONCE(!rcu_read_lock_any_held()); =20 return rq->idle_state; } --=20 2.34.1 From nobody Tue Apr 7 18:35:25 2026 Received: from BL0PR03CU003.outbound.protection.outlook.com (mail-eastusazon11012023.outbound.protection.outlook.com [52.101.53.23]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0387333344A for ; Thu, 12 Mar 2026 04:46:57 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=52.101.53.23 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1773290819; cv=fail; b=uUBmR7MgvzJVDQjW5QFgRJO2VdVt52nhKD4SHb+qSe1HK/dnKUlxEc35Cqvupp9/niislk6qymjHWWY6WJe5T+6hbBnll2d/t44nZ0sQiY4W2k+Qw+gKw4lStNBCUHDbs5EBqEzrm/mJTbQL/tK1lHpwQBc1rSchnrIdVtfTBLs= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1773290819; c=relaxed/simple; bh=r8XAAf/ioU9miosWvJ8wWy+KrfT69TejIwFb4b5HtDk=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=ePpk1JPLM4J4xr3eNt6l4CInemX30n57Y1QfhP3CVbW0VhCPZFUE/lPnfdHqWIggJDpVXXr8zB7RoRm/MD0wBnweUB3nnxEZJGs4hw5bEk9V6NrlKlPrwxSpzNe78y+7L1vC1Vn0zKyDfX9Ghj1e13T9WEPKvNbZF8/GUfZ96iI= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com; spf=fail smtp.mailfrom=amd.com; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b=x+vq9W9N; arc=fail smtp.client-ip=52.101.53.23 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=amd.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b="x+vq9W9N" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=ipmtgEuLvNpwlrlqcC8UeyjnmUEfSerrGXUW0QooW9do+IncDJCbtig+5uFB2yoLFaSKfngTebF5VZNxFHcOimFlm3tBAryI877O25L5solmhur50ofCBrPmIIvGxV10XOTtw9FPDV5dS27SMGm7ZQAKvX9mqBzWOvqClpT91q8sez06amcwO5DdTxljAQD23qnc81EPHMlxMKEvM0xoDJn9S5xBXfBPLJeODeuE4GJVWcK2gTUvmnSxORdfgJroqyk7EyGeoqAMfswG13T/pDex2PIdPr9tXBSMbAqSbYeVCPYywO/e5r/bGsHTCzG1wgp2miw+BMfEeMKHJ0u0WQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=jJscSzlCikEMvRD3ay3lAUM1k1CKBMjQT9Bp5uipPDs=; b=yZgVqHQATXR2tOgbw/y+tCZMYsO/oUZujNgImWvbM12wqQ3z6lZrd3Oz524RGKnLBCRuFwCgqCjOyngFQGS47i/4QbChsNV+StoZ2bTqHF/pGpiVLXIbeip3eQc7gWcbQ9848LfChvCbMZAGh/B1FB6XzmMgYancg/PgZVZjlXq0mlZth+xoG6yefvoZJSGoZPg6WZmj4d+evtLAli1Wfs3BBJEO3Tla3K+TQfH5ruaZEgMYCDNu7feF8S76Q/u4JKp3p4bew+3kiAbdnhiLPIDulYoqJgygqgUwTOVmEiqxMSnLp2p90iEDIDExavW9qXV1sUX4Mx7IPf0m/VInmg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=redhat.com smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=jJscSzlCikEMvRD3ay3lAUM1k1CKBMjQT9Bp5uipPDs=; b=x+vq9W9Nn0LcmeqopdtpH2I+uRBjfbRoClHF7I8ojKuAthghoIBoICmdcidhyTzMJHuXVT8T1MKH9fWJe7tpEV1Dc/0XOcXhndz392gCtgYhCcZhHJf+dUBTgdCIMON0zjlZtYmDUJMmuE0j047OygQLNog+FVi7HvlO1aBVaeg= Received: from SJ0PR05CA0085.namprd05.prod.outlook.com (2603:10b6:a03:332::30) by MW5PR12MB5598.namprd12.prod.outlook.com (2603:10b6:303:193::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9700.11; Thu, 12 Mar 2026 04:46:54 +0000 Received: from MWH0EPF000C6185.namprd02.prod.outlook.com (2603:10b6:a03:332:cafe::b1) by SJ0PR05CA0085.outlook.office365.com (2603:10b6:a03:332::30) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.9678.25 via Frontend Transport; Thu, 12 Mar 2026 04:46:54 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=satlexmb07.amd.com; pr=C Received: from satlexmb07.amd.com (165.204.84.17) by MWH0EPF000C6185.mail.protection.outlook.com (10.167.249.117) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9678.18 via Frontend Transport; Thu, 12 Mar 2026 04:46:53 +0000 Received: from BLRKPRNAYAK.amd.com (10.180.168.240) by satlexmb07.amd.com (10.181.42.216) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.17; Wed, 11 Mar 2026 23:46:48 -0500 From: K Prateek Nayak To: Ingo Molnar , Peter Zijlstra , Juri Lelli , Vincent Guittot , Valentin Schneider , CC: Dietmar Eggemann , Steven Rostedt , Ben Segall , Mel Gorman , Chen Yu , Shrikanth Hegde , Li Chen , "Gautham R. Shenoy" , K Prateek Nayak Subject: [PATCH v4 7/9] sched/fair: Remove superfluous rcu_read_lock() in the wakeup path Date: Thu, 12 Mar 2026 04:44:32 +0000 Message-ID: <20260312044434.1974-8-kprateek.nayak@amd.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20260312044434.1974-1-kprateek.nayak@amd.com> References: <20260312044434.1974-1-kprateek.nayak@amd.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: satlexmb07.amd.com (10.181.42.216) To satlexmb07.amd.com (10.181.42.216) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: MWH0EPF000C6185:EE_|MW5PR12MB5598:EE_ X-MS-Office365-Filtering-Correlation-Id: 6a0a3ae7-d66f-46d9-0117-08de7ff2623a X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|1800799024|82310400026|36860700016|7416014|376014|22082099003|18002099003|56012099003; X-Microsoft-Antispam-Message-Info: xBCEIIIFSJAJvadqPJnH2EFtpn/kOomvGnZTyvxEBnr4Df0yR0zGTNNNukoa0qOvx/VmqoXgkvSPCwlX6gbmY5hEDqA5CT3MMaYAi2M28hDlgfyQ9MyroZfhA8SNzPrca6eEyBbMTB8tlnI5XijLxVPkNCadW/z7ysv0Z3j/B3NftjFdGFYCbonHv7kaWweOk2+evEOMhS8d1jEj4t/fciofozOhvUYC8nMJYAutPXC5Ux9DXC0czMo3mNBEn5QJsW7RJIPEffb1jQywAYGHPDVLxDkpMoeBolF7991wexoaox/GsLuTpGLcjR7GyurXkn00y7tWRCaj3LLArbUjS5yQfIdbnLQ+KJ2h8gh0YSQr90HrUluhOVTI6+eCEXuZnPjP8Y+hOmS5Zo++4uab2Fhmc4rUSQMw8QNsbREvhUYm1BV5tMQnaSpWToZmAKacYlkUixvGeiJRIvQecMp7rdi0hjcQb07SoMN7HsrArKkqOcii8lnnyv32E9cE1kHFZnXLgmhvolcEmP+19gOM9D8V4BnoMe2F/E+5W6NbheOHrw6YjPbVSkWu5U1E4Mpt1oUkPYWp99VoZWTs6n+3Yp3nhRbP9/rp2dj0HczrVqOs25DpGhjalfofSp3jUtKDT6vxUBrqQeGC3VHjBaro2TOVjkAxnbu22oYxsWfNVw6JBvfOAteZ1ybyRKLkewM7/qthl89SboCvaLowL2HyW8mNkQQQh0oWIBm6lWYv7xnaAEY8Lx9qzTX59kshNCYigHlFY9T7OI7MJKzpNXkhsw== X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:satlexmb07.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230040)(1800799024)(82310400026)(36860700016)(7416014)(376014)(22082099003)(18002099003)(56012099003);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: cpaaZkJhxYL8yoU7QHFEGux6d4tLAktf3rHpTrqA6p+8KEBCN8WK0VuyxFDendJ2SFoscB8cw32GrjvoVcvWDcLgRnn5lzazqaAiu293q8R6YgSAh4jSfaV15QUxen4QGjxLaX9Gn1FqxKVDqyCGGKz3mbW3CZ95JnHJkCO52GPKPi1TLMHCwRPUBVRtYHcNkPES0lAp0LRXfsVWBKnP5CxK14N65LmV9aseuJ/ncoXgcjUQt0Jx5NeaZnw8e85EcbbA58Cr8fuufVnxbqj9Ac6pJe/hBvyhXt1NBBKnnKcIgZlsvDhGxVdSt6WsSuBZOs8u1XIqBBNfmKrSIowv9ZzOPrAuDulwAh6TjrPsr18lFLGFpldVT5FOFz6AX71JUe7tdrXstF9fYqo/34qudfCB0N0HnotpMClJcJ6LSbMoMWOmkyrT7QLWuQjRpwSv X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 12 Mar 2026 04:46:53.8615 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 6a0a3ae7-d66f-46d9-0117-08de7ff2623a X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[satlexmb07.amd.com] X-MS-Exchange-CrossTenant-AuthSource: MWH0EPF000C6185.namprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: MW5PR12MB5598 Content-Type: text/plain; charset="utf-8" select_task_rq_fair() is always called with p->pi_lock held and IRQs disabled which makes it equivalent of an RCU read-side. Since commit 71fedc41c23b ("sched/fair: Switch to rcu_dereference_all()") switched to using rcu_dereference_all() in the wakeup path, drop the explicit rcu_read_{lock,unlock}() in the fair task's wakeup path. Future plans to reuse select_task_rq_fair() / find_energy_efficient_cpu() in the fair class' balance callback will do so with IRQs disabled and will comply with the requirements of rcu_dereference_all() which makes this safe keeping in mind future development plans too. Signed-off-by: K Prateek Nayak Reviewed-by: Dietmar Eggemann Reviewed-by: K Prateek Nayak Tested-by: Dietmar Eggemann --- Changelog v3..v4: o No changes. --- kernel/sched/fair.c | 33 ++++++++++++--------------------- 1 file changed, 12 insertions(+), 21 deletions(-) diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index d57c02e82f3a..28853c0abb83 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -8570,10 +8570,9 @@ static int find_energy_efficient_cpu(struct task_str= uct *p, int prev_cpu) struct perf_domain *pd; struct energy_env eenv; =20 - rcu_read_lock(); pd =3D rcu_dereference_all(rd->pd); if (!pd) - goto unlock; + return target; =20 /* * Energy-aware wake-up happens on the lowest sched_domain starting @@ -8583,13 +8582,13 @@ static int find_energy_efficient_cpu(struct task_st= ruct *p, int prev_cpu) while (sd && !cpumask_test_cpu(prev_cpu, sched_domain_span(sd))) sd =3D sd->parent; if (!sd) - goto unlock; + return target; =20 target =3D prev_cpu; =20 sync_entity_load_avg(&p->se); if (!task_util_est(p) && p_util_min =3D=3D 0) - goto unlock; + return target; =20 eenv_task_busy_time(&eenv, p, prev_cpu); =20 @@ -8684,7 +8683,7 @@ static int find_energy_efficient_cpu(struct task_stru= ct *p, int prev_cpu) prev_cpu); /* CPU utilization has changed */ if (prev_delta < base_energy) - goto unlock; + return target; prev_delta -=3D base_energy; prev_actual_cap =3D cpu_actual_cap; best_delta =3D min(best_delta, prev_delta); @@ -8708,7 +8707,7 @@ static int find_energy_efficient_cpu(struct task_stru= ct *p, int prev_cpu) max_spare_cap_cpu); /* CPU utilization has changed */ if (cur_delta < base_energy) - goto unlock; + return target; cur_delta -=3D base_energy; =20 /* @@ -8725,7 +8724,6 @@ static int find_energy_efficient_cpu(struct task_stru= ct *p, int prev_cpu) best_actual_cap =3D cpu_actual_cap; } } - rcu_read_unlock(); =20 if ((best_fits > prev_fits) || ((best_fits > 0) && (best_delta < prev_delta)) || @@ -8733,11 +8731,6 @@ static int find_energy_efficient_cpu(struct task_str= uct *p, int prev_cpu) target =3D best_energy_cpu; =20 return target; - -unlock: - rcu_read_unlock(); - - return target; } =20 /* @@ -8782,7 +8775,6 @@ select_task_rq_fair(struct task_struct *p, int prev_c= pu, int wake_flags) want_affine =3D !wake_wide(p) && cpumask_test_cpu(cpu, p->cpus_ptr); } =20 - rcu_read_lock(); for_each_domain(cpu, tmp) { /* * If both 'cpu' and 'prev_cpu' are part of this domain, @@ -8808,14 +8800,13 @@ select_task_rq_fair(struct task_struct *p, int prev= _cpu, int wake_flags) break; } =20 - if (unlikely(sd)) { - /* Slow path */ - new_cpu =3D sched_balance_find_dst_cpu(sd, p, cpu, prev_cpu, sd_flag); - } else if (wake_flags & WF_TTWU) { /* XXX always ? */ - /* Fast path */ - new_cpu =3D select_idle_sibling(p, prev_cpu, new_cpu); - } - rcu_read_unlock(); + /* Slow path */ + if (unlikely(sd)) + return sched_balance_find_dst_cpu(sd, p, cpu, prev_cpu, sd_flag); + + /* Fast path */ + if (wake_flags & WF_TTWU) + return select_idle_sibling(p, prev_cpu, new_cpu); =20 return new_cpu; } --=20 2.34.1 From nobody Tue Apr 7 18:35:25 2026 Received: from SN4PR0501CU005.outbound.protection.outlook.com (mail-southcentralusazon11011025.outbound.protection.outlook.com [40.93.194.25]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C0AF332ED55 for ; Thu, 12 Mar 2026 04:47:15 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.93.194.25 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1773290837; cv=fail; b=eK+XQFE+mPHoZo7b3iLxMJ18exc+m5R59irJBwzExr72m1RQGRn06v/aFAKFiMHxdn9MF3ZpUtFPliUEAhzpj/VQ5JClaEryaUV8s4j6xYE77gb6qjz/zrhgS4WrHKVSVYbJbSKQtHEDp2bX2JG1uKqXi8PahP/D9qWu04uwCbM= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1773290837; c=relaxed/simple; bh=lixwbeNHaNCWvZ0W7KuzH4GrCQ9p+R8Zt8K5NNp1GNU=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=Rh5i+by4wAcdcvRaInM5ZD9AqQ911B066sMXL1GuYt50jzW1LV3VV0S2y45PwOsjbb5luaOJ8jXqiHehHkWNIRdxKU/KmXOQGG51JDbfUO/dGhhO2kz8vcvEu6WyEZv74p6NHjzsX/WQ3pVR2UrY251Qn5Ma9GbMqNCacA4zxYk= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com; spf=fail smtp.mailfrom=amd.com; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b=hxWXBnnr; arc=fail smtp.client-ip=40.93.194.25 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=amd.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b="hxWXBnnr" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=ibRf9ci2xIVEtfEB8a0aiBrZ3sd/LAVM9YzAKeG8aHMF7qAtqFT6p3TXhDYMqRr7KweO8pPOmr0W9qeryEbtH1b0LIdbMc5zNM7WSjlsFT/uyaERuGGtvEL/vxy/F1o3NZ/WToYVTG/guy1T8hQeEtpmkqoNmlCbPe78OVD0In2393oZX199EZINMtoH58f9NSZjHSJciz6rVgR0a2aexTJMdZ6jlMQVa4HboYZw+IyrOwba8fVA912c6k/rbKPiHyxK4QnP9xmE9e3G71X7w/pzlcQr9RcKj/j8uLS7slDbZVxdqnAWCHS9atgHAUazo6oPnbOEoOof09Sba3c9Og== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=5IVhtr7G8cFOqnQ2B43XS1dn1C3w7ivnFK2LDxm65OU=; b=sxL95mLKPnmYe6kyS2LISGjKI0BGYiJ0cMa63BU6AUVJSlhsM3kaGyFn9KxNU2EvQ5L4UMIAOe5SvqtYFFj7i6yPoTkka1cZdb+P8oPBB/WJrkhwXlzEf3pFWPBD02Jbiiv+HB2n76OVNjuT0Ge2Va8Ou1VdQKnmEZEDik0ZhL6BT1S/14oZwiCJYSMMo3Ujex7aaYxrdsmIm7YGAo7096Hu9pTkOj2YdlR0ZFi/B/5anKiTlgJ0aeH+ZvTDgWzE9VTK7lfBZMp2QextFXad+EI5Zg8+R7a+hh02wOGf0KRgg6PECiYbCEXTG4vTJ0bjDqquyQx9OHmQ6FAnFiT9Cg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=redhat.com smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=5IVhtr7G8cFOqnQ2B43XS1dn1C3w7ivnFK2LDxm65OU=; b=hxWXBnnrrng6pHWlmXzeDPWQwCkXylbEaXDY2JYJYv3AdwFacs2lBH1cJDS1FPxEc6rfx3U87x6KwrZWFaTz4xqXHgLV8LMn2vPP0eXfdQDKCj9GV9nIyf8UDcIP/PGMJgl2Ff2D7fhnVsvQs32Uds53gOLpmikVUIIo9dS+low= Received: from MW4PR03CA0243.namprd03.prod.outlook.com (2603:10b6:303:b4::8) by DS0PR12MB6560.namprd12.prod.outlook.com (2603:10b6:8:d0::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9723.6; Thu, 12 Mar 2026 04:47:11 +0000 Received: from MWH0EPF000C6186.namprd02.prod.outlook.com (2603:10b6:303:b4:cafe::c3) by MW4PR03CA0243.outlook.office365.com (2603:10b6:303:b4::8) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.9678.27 via Frontend Transport; Thu, 12 Mar 2026 04:46:44 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=satlexmb07.amd.com; pr=C Received: from satlexmb07.amd.com (165.204.84.17) by MWH0EPF000C6186.mail.protection.outlook.com (10.167.249.118) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9678.18 via Frontend Transport; Thu, 12 Mar 2026 04:47:10 +0000 Received: from BLRKPRNAYAK.amd.com (10.180.168.240) by satlexmb07.amd.com (10.181.42.216) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.17; Wed, 11 Mar 2026 23:47:06 -0500 From: K Prateek Nayak To: Ingo Molnar , Peter Zijlstra , Juri Lelli , Vincent Guittot , Valentin Schneider , CC: Dietmar Eggemann , Steven Rostedt , Ben Segall , Mel Gorman , Chen Yu , Shrikanth Hegde , Li Chen , "Gautham R. Shenoy" , K Prateek Nayak Subject: [PATCH v4 8/9] sched/fair: Simplify the entry condition for update_idle_cpu_scan() Date: Thu, 12 Mar 2026 04:44:33 +0000 Message-ID: <20260312044434.1974-9-kprateek.nayak@amd.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20260312044434.1974-1-kprateek.nayak@amd.com> References: <20260312044434.1974-1-kprateek.nayak@amd.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: satlexmb07.amd.com (10.181.42.216) To satlexmb07.amd.com (10.181.42.216) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: MWH0EPF000C6186:EE_|DS0PR12MB6560:EE_ X-MS-Office365-Filtering-Correlation-Id: acc8c5e9-ca0f-4cdd-6e39-08de7ff26c46 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|1800799024|36860700016|82310400026|7416014|376014|22082099003|56012099003|18002099003; X-Microsoft-Antispam-Message-Info: 8IEeGofJ94+EmDcgGA+GoqgPDkCNLZX2zgbpiilzQL/gTgVmtFGJK28Ic6XEa3qWVQjHA9zDXGXBt1eoxcIoJyQuS3B7/fpdftdatM4U0Pq4nz/sDWE+v0QwtP7lwe0eNyBeE1hKvQpr6ECNmToKpzowt+3X5S5PEpsvPqNdUM6LxAxTlUAp/niVH6EEOXuAmPuJwXQSsSDqMQieDGBF80jDV2y/L/lrJMwfrCA94X5CWIYk0AgqJQBSrdGegvCdPKqMciS1BMAlOtx+qk0xJyDH1jrPjHKVV5aP2Ics+v3pxBY+PGS/M/I98AoETzGHMJQR9f2fXxvryfXuLSDSJQeM/nLt4hsZvkTVQWtXSvYA5cErutwXOxHZkeJSQXkPuAy8iBrahj71RAeo+DqTXRtC9cUqoQCjulUMAUp+3PwnXoalKummpOkZnboE6jHneEe+mPXJ+iWSqgXNmnF6KP1VqeERwRDCDPgxjjVIi5X5OlCUfYCv6JJtGh3wWeSyNofCj6ziy1AxuxGjg0quj9Rc/6RBB8Bigw7Z52lRBhmx+XoUbH9howem2MTie2k+f1mqiBQrpWk6HWuOfkmZIO2YxfGE5JOK8LoBIP31shz/fsU5R9QIEIHILc4WPg7jQ+red6ODNEHfOCCaXjy89lwb0ADZV6QOzPenUiH6aXU5eaNOm1mmK+mTyVioy2sjyMIv8CrInnPWJbKZr1XvXuNVgpIHPxqVzylJqtJWVTJImPwI6cQNxloT0r9BzMpv97jpa/LmTKHhF7PnOiWqWg== X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:satlexmb07.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230040)(1800799024)(36860700016)(82310400026)(7416014)(376014)(22082099003)(56012099003)(18002099003);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: TVVbFnJ5Pqs99lp6gqdtxdKD4oUJXwNG5sGtRA91Wxd+yPrdzt5eyiEiz/eX694afSEloFSNoQqJT8Xhr/53gGTcVgShkItijYjC1ZAKaduBkQGNKc2ox9PURub8D5wlgJj9OWpEf+HMJbs6vhsk3J3tqgZV3jUXWUe8/fGWNweTW1Bm+buEfdiwVQl1KHyYh5H0yEizT10QKieGW/yY51cqcUuxlP5qnvl790kY6k8f8nJtcKN4npAbHQxtErVW2gDitQyD71BTvq+lOUzmj/ZG4cnT1nmfF93amY8683mb5yj2eKjIQk/DkQDODgPTOpfsMTJe4Fepf9zkVdGwB5NnukdgpDzZI9+AbSyS8qk9tD+2MnFA0Sgj8G+Wv5LOWUBp5SYnaDFCatqPLf73c8IJxO1Ln2aCiWjqk/7BgXsXroXXnWrYh5foZHM4bgf7 X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 12 Mar 2026 04:47:10.7461 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: acc8c5e9-ca0f-4cdd-6e39-08de7ff26c46 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[satlexmb07.amd.com] X-MS-Exchange-CrossTenant-AuthSource: MWH0EPF000C6186.namprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DS0PR12MB6560 Content-Type: text/plain; charset="utf-8" Only the topmost SD_SHARE_LLC domain has the "sd->shared" assigned. Simply use "sd->shared" as an indicator for load balancing at the highest SD_SHARE_LLC domain in update_idle_cpu_scan() instead of relying on llc_size. Reviewed-by: Chen Yu Signed-off-by: K Prateek Nayak Reviewed-by: Dietmar Eggemann Tested-by: Dietmar Eggemann --- Changelog v3..v4: o Collected tags from Chenyu. (Thanks a ton!) --- kernel/sched/fair.c | 10 ++++------ 1 file changed, 4 insertions(+), 6 deletions(-) diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index 28853c0abb83..d7e4de909a63 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -11234,6 +11234,7 @@ static void update_idle_cpu_scan(struct lb_env *env, unsigned long sum_util) { struct sched_domain_shared *sd_share; + struct sched_domain *sd =3D env->sd; int llc_weight, pct; u64 x, y, tmp; /* @@ -11247,11 +11248,7 @@ static void update_idle_cpu_scan(struct lb_env *en= v, if (!sched_feat(SIS_UTIL) || env->idle =3D=3D CPU_NEWLY_IDLE) return; =20 - llc_weight =3D per_cpu(sd_llc_size, env->dst_cpu); - if (env->sd->span_weight !=3D llc_weight) - return; - - sd_share =3D rcu_dereference_all(per_cpu(sd_llc_shared, env->dst_cpu)); + sd_share =3D sd->shared; if (!sd_share) return; =20 @@ -11285,10 +11282,11 @@ static void update_idle_cpu_scan(struct lb_env *e= nv, */ /* equation [3] */ x =3D sum_util; + llc_weight =3D sd->span_weight; do_div(x, llc_weight); =20 /* equation [4] */ - pct =3D env->sd->imbalance_pct; + pct =3D sd->imbalance_pct; tmp =3D x * x * pct * pct; do_div(tmp, 10000 * SCHED_CAPACITY_SCALE); tmp =3D min_t(long, tmp, SCHED_CAPACITY_SCALE); --=20 2.34.1 From nobody Tue Apr 7 18:35:25 2026 Received: from CY3PR05CU001.outbound.protection.outlook.com (mail-westcentralusazon11013071.outbound.protection.outlook.com [40.93.201.71]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id CFF3B38F63F for ; Thu, 12 Mar 2026 04:47:32 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.93.201.71 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1773290854; cv=fail; b=cOIxXr/aPPm0zkikEldq9r3VTc+L5mB9E1y08XO88k8LvvVmHfpgzKG/zWkQ9pVW5z2p3jCQmvYJa7004HIHD2YNGoELz4Y/VVG1SuR6Rq6NMKvzLF/4fHAQ14+rop/nbFT4Q6jJOCnK5pPkeTabxsBuSkqT6TgqBlhJeiJTeyY= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1773290854; c=relaxed/simple; bh=y6uWHH8vvSwTzrdgl2cyNURrPpjenYBEhSgzGZk+llg=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=uQJGVABiiuX/cPdp7m36BjmmC8oUEoJgOnXU4JZbwWnZfiCJlmcwFdpJe5HQwKn/d/TM2CbtGKq1+repsynYK/n6uIt3Qkwe2mfHUXA2eFYGjUq+PIrFPiknsteLz2B7oRB6lYcygwmhq+bjm7SW8dbPv81TxGR/7fmp16jdsi4= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com; spf=fail smtp.mailfrom=amd.com; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b=efwihaTR; arc=fail smtp.client-ip=40.93.201.71 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=amd.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b="efwihaTR" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=NW+f7Ch0b8pSXZgpgQeLMr1PkV9Qwm6PaIEtoKrX0l4GhxtZfkQO40U1TBeslxeztEfnEkahy7n4TJ3qXLJ/eGgFvXgyFR8g8uA4YQn2ftafKjXtbo9lTmcDpisNrnPNGC8rDgv2/wkhMwkZJeX8QX71G94rOrbY+FdYbt22r8vCRZe1taSmiq1+IrY6NyPkn0jm0+75ZVo2wvbeHSZe4Leq8hr6eRjmI0J5ntPzK+kOl132raWls3ixTJ8wn0c6qXuAaoY54xgYx1rQ2IQbWq7UwiA5U6mDv8ce4SfQ2RrkQpgaJXFX7nG7S5Lyeito7mtK+109ImQv61VxZWBKtw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=zGvDOhVfxYIqi0ngjEzw5tcHC4b11ox0hSe60uKX/Eo=; b=D/ZUXULoyeJuJQ7Ef24vOdHYOyBHkSiy37qZTljf2RFlgf+OcUBeoswUGxmSqJ/0hA9PnnltGCLVctkeRk5u+RUJHyc0CeBxjHwnNAgPG418SP0f2q20P3wG+KUV0yYNjB0DrqIVv0s7b+qCbPxzapDDADx61A/YmpReIiu5pYG1pEdGp3XbbojxWwwrBYo2F+7lHloZF9nuwkn06s7g2cpwi7ekQ2Uj0mGHxVtMtqqS1KQAEEumniPr/XgKBqDsVJLt9sJvHUFJj+Qik/pgWiZ1x2noemFZ3liFRoMOkfv5BYnk18lBW6HQlmw7ex3aoxcLh9sssEzXTW1kXmLiUg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=redhat.com smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=zGvDOhVfxYIqi0ngjEzw5tcHC4b11ox0hSe60uKX/Eo=; b=efwihaTRPpChNlYUUmk8rKiEx/OhTQERFOfQYuAY9DDqsx8xntbFoBS81WdsmdKGc5XLZbYWOwBaYCukh3BHnFaMVuucqbdJT7BRlLtpxjSDdvaZ9fRJHtFlJ5P8ul0Wjeadw9WUK9E244js4Ds88JhWvQzHLRu3Vxj8besHHvM= Received: from SJ0PR05CA0069.namprd05.prod.outlook.com (2603:10b6:a03:332::14) by SA0PR12MB4350.namprd12.prod.outlook.com (2603:10b6:806:92::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9723.4; Thu, 12 Mar 2026 04:47:28 +0000 Received: from MWH0EPF000C6184.namprd02.prod.outlook.com (2603:10b6:a03:332:cafe::16) by SJ0PR05CA0069.outlook.office365.com (2603:10b6:a03:332::14) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.9678.22 via Frontend Transport; Thu, 12 Mar 2026 04:47:28 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=satlexmb07.amd.com; pr=C Received: from satlexmb07.amd.com (165.204.84.17) by MWH0EPF000C6184.mail.protection.outlook.com (10.167.249.116) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9678.18 via Frontend Transport; Thu, 12 Mar 2026 04:47:28 +0000 Received: from BLRKPRNAYAK.amd.com (10.180.168.240) by satlexmb07.amd.com (10.181.42.216) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.17; Wed, 11 Mar 2026 23:47:23 -0500 From: K Prateek Nayak To: Ingo Molnar , Peter Zijlstra , Juri Lelli , Vincent Guittot , Valentin Schneider , CC: Dietmar Eggemann , Steven Rostedt , Ben Segall , Mel Gorman , Chen Yu , Shrikanth Hegde , Li Chen , "Gautham R. Shenoy" , K Prateek Nayak Subject: [PATCH v4 9/9] sched/fair: Simplify SIS_UTIL handling in select_idle_cpu() Date: Thu, 12 Mar 2026 04:44:34 +0000 Message-ID: <20260312044434.1974-10-kprateek.nayak@amd.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20260312044434.1974-1-kprateek.nayak@amd.com> References: <20260312044434.1974-1-kprateek.nayak@amd.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: satlexmb07.amd.com (10.181.42.216) To satlexmb07.amd.com (10.181.42.216) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: MWH0EPF000C6184:EE_|SA0PR12MB4350:EE_ X-MS-Office365-Filtering-Correlation-Id: 3e111d46-194b-486b-1253-08de7ff276e4 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|7416014|376014|1800799024|36860700016|82310400026|56012099003|22082099003|18002099003; X-Microsoft-Antispam-Message-Info: /m4sU8MNq0VGHY80P7G5oZ8GkePeIOrXU5sUD40SQ5sV+tIXhwqukP61uq5t0Yk46E3zM1ejjQxcPadHrz7/34pCpZMs2LGnV55E0YWaR2e9zTOyKZwoefXy1EGk932cO4fzLi1hDRglXd0fSQFlWWNk14TM2QyqyYnbfkbSwrXJqXbBi2eE4eqQMochdcgO0vTRq7VLjVY/mp5aVNMaMhk32lM4w78M+ekxd+53LM3YFcJc+xqMu+vqdNQnRY6k7d7iN+W7jiw6T7wYBZYE5/a98NI0xafA085wzQgUu+JscM2vRBy7tnTvOVd8No7n784VHFkTpwLBKUPB1X/kZOvHOPuBBZju/ItwA9ikewkf58IvfGU77N6GcCYAMDYIWDag4E1Cl1IDbKnXi1ZbWHRTD8NqZCFcAtqACGFxZDi6iLfsN3X2/FBtfqX8bYGqc0YQONLyp+/8eUC1sbsj6ilkDn386mIMG0sri89R0HPnt9Bi4bOpbd+gVydA73nl9XLS+cqkl8tGgAZW4O81ZTTbR8Y3jH9hbMRYVMEcVkTrpMU63chOeex9pTIuphPt9wxZjLvf0P+8ABx4iJchekTeWJ1wbTdl/AB1viCU0T4j0vTBqcI375QzaZP/lQuOdRxFincntaW9QL0BbmcEtOWbBnSVjBDEd7mPIGonwCOI4T2yC1CiPKDXGKmW6vrbxa0bX4ySjY1WNApdxFcZfbvAdBFGo9xVtOUDP4TU6RQj/dLS6JOuwIGc4R7YbyTZezDoWAyCn1Ifx+oGgkdrcw== X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:satlexmb07.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230040)(7416014)(376014)(1800799024)(36860700016)(82310400026)(56012099003)(22082099003)(18002099003);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: WcrsHLlBNF/k2kN92Op4X4kzpjIhaHaPaU2fM/J3lAfUshK4sHePDF8/dWWVCnSIKffchQj9+mE8K0oPVU+kwsW5Hesmp7bxNwmBe97z7Uino0tzci9nmKJSCJpbrKrJHrJCh6fquTmUVJMj4XvAo938juNqlImv4AdY1LKd9S9tuoCFPp+7mVQ5//O0YwOTrGUOA/FuiqpkVFYajaLnU/oESh5SAUz9bD6hHZFyeseSBl89AHXkgcgezdixQ8vzbzXVogXSjxkbaNQl+ErCpHFVpJbEiJO9CifHzRpPBqApJAdVnKgcqNfd84CIw03BFOhVLd9IMzf/BKEjJyCIewGx/WJenSQxjCZYtLCXQeRecAYboY5MP64rPNO12UYOn6a+V9pdMZgxW2w05Ag+wC+8auF+Lbrln7PlLrtIqvGXbpFxtqTI8yvBcFM2OfXy X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 12 Mar 2026 04:47:28.5320 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 3e111d46-194b-486b-1253-08de7ff276e4 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[satlexmb07.amd.com] X-MS-Exchange-CrossTenant-AuthSource: MWH0EPF000C6184.namprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA0PR12MB4350 Content-Type: text/plain; charset="utf-8" Use the "sd_llc" passed to select_idle_cpu() to obtain the "sd_llc_shared" instead of dereferencing the per-CPU variable. Since "sd->shared" is always reclaimed at the same time as "sd" via call_rcu() and update_top_cache_domain() always ensures a valid "sd->shared" assignment when "sd_llc" is present, "sd_llc->shared" can always be dereferenced without needing an additional check. While at it move the cpumask_and() operation after the SIS_UTIL bailout check to avoid unnecessarily computing the cpumask. Reviewed-by: Chen Yu Reviewed-by: Shrikanth Hegde Signed-off-by: K Prateek Nayak Reviewed-by: Dietmar Eggemann Tested-by: Dietmar Eggemann --- Changelog v3..v4: o Collected tags from Chenyu, and Shrikanth. (Thanks a ton!) o Added a brief comment in select_idle_cpu() on why directly dereferencing "sd->shared" is safe as long as an RCU-protected reference to "sd" exists. (Shrikanth). --- kernel/sched/fair.c | 27 ++++++++++++++++----------- 1 file changed, 16 insertions(+), 11 deletions(-) diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index d7e4de909a63..8dbf63d460b8 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -7876,21 +7876,26 @@ static int select_idle_cpu(struct task_struct *p, s= truct sched_domain *sd, bool { struct cpumask *cpus =3D this_cpu_cpumask_var_ptr(select_rq_mask); int i, cpu, idle_cpu =3D -1, nr =3D INT_MAX; - struct sched_domain_shared *sd_share; - - cpumask_and(cpus, sched_domain_span(sd), p->cpus_ptr); =20 if (sched_feat(SIS_UTIL)) { - sd_share =3D rcu_dereference_all(per_cpu(sd_llc_shared, target)); - if (sd_share) { - /* because !--nr is the condition to stop scan */ - nr =3D READ_ONCE(sd_share->nr_idle_scan) + 1; - /* overloaded LLC is unlikely to have idle cpu/core */ - if (nr =3D=3D 1) - return -1; - } + /* + * Increment because !--nr is the condition to stop scan. + * + * Since "sd" is "sd_llc" for target CPU dereferenced in the + * caller, it is safe to directly dereference "sd->shared". + * Topology bits always ensure it assigned for "sd_llc" abd it + * cannot disappear as long as we have a RCU protected + * reference to one the associated "sd" here. + */ + nr =3D READ_ONCE(sd->shared->nr_idle_scan) + 1; + /* overloaded LLC is unlikely to have idle cpu/core */ + if (nr =3D=3D 1) + return -1; } =20 + if (!cpumask_and(cpus, sched_domain_span(sd), p->cpus_ptr)) + return -1; + if (static_branch_unlikely(&sched_cluster_active)) { struct sched_group *sg =3D sd->groups; =20 --=20 2.34.1