From nobody Tue Apr 7 20:07:41 2026 Received: from CY7PR03CU001.outbound.protection.outlook.com (mail-westcentralusazon11010053.outbound.protection.outlook.com [40.93.198.53]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 975B437474B for ; Thu, 12 Mar 2026 04:45:17 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.93.198.53 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1773290718; cv=fail; b=OXbihFRzI2rGPFn3Mfra+LI9bq7331JEMxENRTABdXBaZiZw7vGzj8JsycuMBCH9mEAtsCq9p37LL/g1hdADDWbiiso9kA9oSXiV+4W72er8NtFA5dKZKYg2bANtDoDIwCsJ3fw4RLC7/GbyAUtJ8IMhunqD5ar4d20HsXXWxvA= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1773290718; c=relaxed/simple; bh=a849sjcJfH5lcNOaZnmaVokfDSd8BNY/+noBpg93j/0=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=GCOtNTJIlzKXNDOcugyqNdr9VNHW3DuTiIjbYoi7XYCSZuAzp4z0OUOiA0hUcNfZ66QyGNAOED2fD2CJ7LUNVY085JBZTfFeMzn9P0Dc2qIHTL1spb1tP44T60LVQCL8MNMqBIJxLyu02k3Cc6sb1LE//WJhvFPtk7Uc19kiEMs= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com; spf=fail smtp.mailfrom=amd.com; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b=b5j7y85v; arc=fail smtp.client-ip=40.93.198.53 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=amd.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b="b5j7y85v" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=YZCXyup8rqxN+SqLQmaHEO4+sbF6sszeby96cXAsL9uqb6WdF6BX05KlO+1HoNkuPU2B4LApDdyDfQ/PDn5X6gI+hXby0HmEHQmhHRJxiFiFSNEvTlUB+cLln/APy79vS69+zSeoNLvRl9URtAE1vQ9JFdv840kdiZVyvmKUaM1pawFRFQTVC7iT+T2BY8mPWw757cJsvzLXwhucw1OIKEL2mfTPFXxDJcUDvU+CiTOyse8tuHQMmoCTlNm70xT+vh8S6uthxTje8v9RKOChvgpE4WOa/7SozFKY0miqPSG4o6S5fvHoc4uyEYCOkUgoG2wR75a4cz83TSWBOBQ5kg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=IxNdfFMUX9sLJwTaUdqtNmtf/2iyMJqtySDJtle3f58=; b=guIXbXLP8/zGM4esUKQtFCLuyRjumBwhgmZR6NYgDbctef6eRYxwvA50lqIm7BT1nJ6gEPSF2z43MGC0PY5t112nE+RfN/RLZ4xwGFiI+LqaQx4OsVQRM4Du1TsQBaUN8rnaUZQY8QfTI2wjvhTqA/uvH3YmlKdGhWf52YDst0QNb04ZdpcIMtUb7X/a0QA0OlKT+rFkY478OB1X3llb9S89dF+NWaFPrZbKruMf3xqhy9jhwTZkvqUPypNfWYYUV40g6RzJ8L+y3Q3lGgTtUazCROzeSY6vx/wgF01W4qhVRaZdyCWh3MBxC5s4VuGTUeUmV70YFutN1R2SaRcrmQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=redhat.com smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=IxNdfFMUX9sLJwTaUdqtNmtf/2iyMJqtySDJtle3f58=; b=b5j7y85vez0kY1s6D3JmGX7VCya2Z8SJrXD/gaGSCtfHbci88VrCUw5oHvTxI3vnGq8PEKe5noQ6V7EsTSL5XUAZkBRaf4rAcB3iI4Gw6mkeCOyq3TJb54aoTdLscjjWXGSa1DeYD7E2nwWFz5Vb0fXoKtgl5ArAcBsFjpZPWjQ= Received: from BY3PR05CA0040.namprd05.prod.outlook.com (2603:10b6:a03:39b::15) by BL3PR12MB6401.namprd12.prod.outlook.com (2603:10b6:208:3b1::8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9700.13; Thu, 12 Mar 2026 04:45:13 +0000 Received: from MWH0EPF000C6187.namprd02.prod.outlook.com (2603:10b6:a03:39b::4) by BY3PR05CA0040.outlook.office365.com (2603:10b6:a03:39b::15) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.9678.25 via Frontend Transport; Thu, 12 Mar 2026 04:45:12 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=satlexmb07.amd.com; pr=C Received: from satlexmb07.amd.com (165.204.84.17) by MWH0EPF000C6187.mail.protection.outlook.com (10.167.249.119) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9678.18 via Frontend Transport; Thu, 12 Mar 2026 04:45:12 +0000 Received: from BLRKPRNAYAK.amd.com (10.180.168.240) by satlexmb07.amd.com (10.181.42.216) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.17; Wed, 11 Mar 2026 23:45:05 -0500 From: K Prateek Nayak To: Ingo Molnar , Peter Zijlstra , Juri Lelli , Vincent Guittot , Valentin Schneider , CC: Dietmar Eggemann , Steven Rostedt , Ben Segall , Mel Gorman , Chen Yu , Shrikanth Hegde , Li Chen , "Gautham R. Shenoy" , K Prateek Nayak Subject: [PATCH v4 1/9] sched/topology: Compute sd_weight considering cpuset partitions Date: Thu, 12 Mar 2026 04:44:26 +0000 Message-ID: <20260312044434.1974-2-kprateek.nayak@amd.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20260312044434.1974-1-kprateek.nayak@amd.com> References: <20260312044434.1974-1-kprateek.nayak@amd.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: satlexmb07.amd.com (10.181.42.216) To satlexmb07.amd.com (10.181.42.216) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: MWH0EPF000C6187:EE_|BL3PR12MB6401:EE_ X-MS-Office365-Filtering-Correlation-Id: e9c16e44-7c8e-44ae-c934-08de7ff225af X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|376014|36860700016|7416014|82310400026|1800799024|56012099003|18002099003|22082099003; X-Microsoft-Antispam-Message-Info: wf090kATV+nXWThCKLYseAiSoBdVVpplqKZkyobGzwBgpm+ow/xaGWIOPzj9Qnsd1tw10FqbvkgqXZmIY3Ijsslg5fNJIWi3p44BiUe/EuyuC/4tjMF4XV0T6m2PfWwOn8F69G/0ULQHsNIaqsbIi3rIqeiaadZoGLEKD38w0RhY951uiyMeCPrhtuukhvwf4dSvglaD1fZ/i5hDgGtyMiNYr+/nF8eFMC0KcspG0m9E4KFzvt6gaudosn5eIAV1/LBJcFoM+JIlsZeoFDOV2s6NP58GaQ7g4oRH/eE4gsQ7RJS80NInR6OROiIkqgeTQQ6xx8LtRf+jRH6oZqgjDDS5q3sBdd3Ury8LsIfG4LHLWDm9oGcJPrdtPPu4+zHfdaTULV8i56cMGjYFkB4fla7DFr5jxLsEVXVLfhk/5VHgOTEulrOlrx9WmeW0cX2PPW8s/9BFjMu3j0CY+BxInqM2Pwsrmpp9tNeyrJVQZmhnTsMOb6Y7+XAp+jPC7/4/+jwZPNWwJZ6hdcNzdS6WBPBIpUupW9zQflEPLqAmnxOY3BwPitkflNPIJP7fHEq2F65qv8r2C9x1qW+BUa8KYro3wsWxMdqeUgiBWGp5unMNxxXfPkoijNOPVxt4zhqrEb7pr4Irr0buVtuYbZ+1BEgWKk19yzQG/fdJ4cogCsog52j4P9ysTk0/O0U86rtHpTYu3hu+Sn45vmXnQupneOI7+fnVo2a3gGnVHeSH8YUQhvksOhE21zbSdHvqCZx8pGsw0LbEyIbW7xgVZDkjTA== X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:satlexmb07.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230040)(376014)(36860700016)(7416014)(82310400026)(1800799024)(56012099003)(18002099003)(22082099003);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: p6nDM/lv9TJNB9hNIwFEyhrrQ6IMGvV/YGb06RtoDLstyY0/CABw+E0BuGt+Mn1pawSaXz3S5Y+rCagFpifGrC3c2IZQF79/9RlniabBG+rpfphkq8tLamgHueofTVlmZSflN+WLq0x149CeUvAMsfcXR2wk5u40ijSfxOs4VK0Q8zuK1bz7OB8WAfV91TwHD3tK+ykwWmfenJd5qDtLTEIF8XSP6OQhAld7forHPFDMRw8393+DU7ydEOJIFuC2XIUZQTBPe925YkjzaXOt/UhCFdyd9AHPUsPoIBB11K3v74hmlf409mGeOuBexWlUy/anH7RGOl0GGMhMaaIA5MWREy9r2cGEGeIFzWI5k3HI2WDzQmwxKpRwM/Z2KQSIPxbWA07pfz3L9w62mytR6PJE5az7/V4FSDEcdgKZJvcYOnZxr9T8So1rxbH9NbRP X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 12 Mar 2026 04:45:12.3544 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: e9c16e44-7c8e-44ae-c934-08de7ff225af X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[satlexmb07.amd.com] X-MS-Exchange-CrossTenant-AuthSource: MWH0EPF000C6187.namprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: BL3PR12MB6401 Content-Type: text/plain; charset="utf-8" The "sd_weight" used for calculating the load balancing interval, and its limits, considers the span weight of the entire topology level without accounting for cpuset partitions. For example, consider a large system of 128CPUs divided into 8 * 16CPUs partition which is typical when deploying virtual machines: [ PKG Domain: 128CPUs ] [Partition0: 16CPUs][Partition1: 16CPUs] ... [Partition7: 16CPUs] Although each partition only contains 16CPUs, the load balancing interval is set to a minimum of 128 jiffies considering the span of the entire domain with 128CPUs which can lead to longer imbalances within the partition although balancing within is cheaper with 16CPUs. Compute the "sd_weight" after computing the "sd_span" considering the cpu_map covered by the partition, and set the load balancing interval, and its limits accordingly. For the above example, the balancing intervals for the partitions PKG domain changes as follows: before after balance_interval 128 16 min_interval 128 16 max_interval 256 32 Intervals are now proportional to the CPUs in the partitioned domain as was intended by the original formula. Fixes: cb83b629bae03 ("sched/numa: Rewrite the CONFIG_NUMA sched domain sup= port") Reviewed-by: Shrikanth Hegde Reviewed-by: Chen Yu Reviewed-by: Valentin Schneider Signed-off-by: K Prateek Nayak --- Changelog v3..v4: o Illustrated the changes in the load balancing intervals with an example. (Shrikanth) o Collected the tags from Chenyu, Shrikanth, and Valentin. (Thanks a ton!) --- kernel/sched/topology.c | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/kernel/sched/topology.c b/kernel/sched/topology.c index 061f8c85f555..34b20b0e1867 100644 --- a/kernel/sched/topology.c +++ b/kernel/sched/topology.c @@ -1645,8 +1645,6 @@ sd_init(struct sched_domain_topology_level *tl, struct cpumask *sd_span; u64 now =3D sched_clock(); =20 - sd_weight =3D cpumask_weight(tl->mask(tl, cpu)); - if (tl->sd_flags) sd_flags =3D (*tl->sd_flags)(); if (WARN_ONCE(sd_flags & ~TOPOLOGY_SD_FLAGS, @@ -1654,8 +1652,6 @@ sd_init(struct sched_domain_topology_level *tl, sd_flags &=3D TOPOLOGY_SD_FLAGS; =20 *sd =3D (struct sched_domain){ - .min_interval =3D sd_weight, - .max_interval =3D 2*sd_weight, .busy_factor =3D 16, .imbalance_pct =3D 117, =20 @@ -1675,7 +1671,6 @@ sd_init(struct sched_domain_topology_level *tl, , =20 .last_balance =3D jiffies, - .balance_interval =3D sd_weight, =20 /* 50% success rate */ .newidle_call =3D 512, @@ -1693,6 +1688,11 @@ sd_init(struct sched_domain_topology_level *tl, cpumask_and(sd_span, cpu_map, tl->mask(tl, cpu)); sd_id =3D cpumask_first(sd_span); =20 + sd_weight =3D cpumask_weight(sd_span); + sd->min_interval =3D sd_weight; + sd->max_interval =3D 2 * sd_weight; + sd->balance_interval =3D sd_weight; + sd->flags |=3D asym_cpu_capacity_classify(sd_span, cpu_map); =20 WARN_ONCE((sd->flags & (SD_SHARE_CPUCAPACITY | SD_ASYM_CPUCAPACITY)) =3D= =3D --=20 2.34.1