From nobody Fri Dec 19 13:09:44 2025 Received: from CY7PR03CU001.outbound.protection.outlook.com (mail-westcentralusazon11010036.outbound.protection.outlook.com [40.93.198.36]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7700732AAAB for ; Mon, 8 Dec 2025 09:33:59 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.93.198.36 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1765186440; cv=fail; b=AkA1bIfz8iGy1eogytJ69PgXO+c7dZI5PKkcJWkrYKPLGHWRydWnps5b1gL+R9ScRfD3aSLtO1LoNbfjtfOMb2jjiFzuqREAmsUZNVSwtr4z+ZuQ68XMY/AKwJ1/A7pWWNcW80tDtyvHwf63tT0/JHrdE+NUGiZc7UsIMRn+3Sg= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1765186440; c=relaxed/simple; bh=i+wdzwnIsnxpQoHzSQfJ31eNweN07ifBT2oROY8GTyI=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=I/u54QXu1tqprTKm+oWacOpSnxF5gjQjxtEiAZPWEuWVn8FeMvfar04CYGA2r544tds5lusRE0dqx1SUgg/GM9P38BwJ7O/3nTxnD0QInHziRUDtm5ZsxA3p8zKshHpmpk5pSYMAogvBrk1u5bh05LB2p24ujZJqoWB7Q7Y9h8Y= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com; spf=fail smtp.mailfrom=amd.com; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b=OoJaTNuf; arc=fail smtp.client-ip=40.93.198.36 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=amd.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b="OoJaTNuf" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=hWhNvVgrxLWaIRnNRQ6WuH0hgVDmE4DVvStBqVYgZWdivP1jmew2va/otRoAMlnLw/0SQOTKmxVggMcuor0ovnPa4r3DKA6rsqXDsHy83hIYbuFWbkJpeGBf1TNybKaJNSOfxKQTy9zEHFerjh81BhaNV9if6osghf5hd0+1RphHELfi+fW/qL9pq3NmENYn/s4q14TRsGXSyzGhr53/DjXKGV8Lp7SyP+2l9pJ//U8XL7vDzb1/lGuVkMWvZxDitKyjhnsr37k5b+DfacWdsTfBLASGOaCjwqCilVXbFA8bt7+Ctad/GmjYuw/9eMylJcItLJbA0FSf2gRv1U/sgA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=b7/YO+GS/cKuGKKWm+lGbD6gSxLMiR0IrNUJtOcw8uA=; b=frkRB4fLynKNbGtdutYNjB9hKBPoNUvWU1xI3mzLUjOiHOLJJKDVYHSGsujyXXDgdseV8aV3g+1oGZuL2TdSxJpIDYA/15OnpnqzD8HWDmOK19PyzsAhIKsBVXOtEcerkgDJuOGMnGT13GlGfcdlmvg9gWmYTcE3cuxUdJZ9ZAPDAG8Nbqbc4AhE7UXLwujLCbvK/y14u5n7Oerv+P0+lgF7L+3SOIVQwoiuH9/9J4jz0D5QnzmA7TrlHLz4Ef/+x+cJN2CSkmnTSKGjgdSPKk3wV6HtmhqgG4Mw0kdPPXhUHHw6fRBe48L1fP2b12cHykJYn7o8vG6Ke8vbQwtFyQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=redhat.com smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=b7/YO+GS/cKuGKKWm+lGbD6gSxLMiR0IrNUJtOcw8uA=; b=OoJaTNufhARisL0B1VP/5nYa6uWOoKSYhWUoFp961UWz0FxjPbkdwqxDiHbOFn1cdn80hT9UatyUvFPyVkY7kiUNRgIu4BHaQr1XWMPzBRW1msZ++q14REpZ1uz603aZfrQ2Xz3CMGHrT1RswP0uVfgkmf7FIdpxSQf/2JhZCqI= Received: from CH0PR13CA0048.namprd13.prod.outlook.com (2603:10b6:610:b2::23) by BL1PR12MB5948.namprd12.prod.outlook.com (2603:10b6:208:39b::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9388.14; Mon, 8 Dec 2025 09:33:55 +0000 Received: from DS3PEPF0000C37B.namprd04.prod.outlook.com (2603:10b6:610:b2:cafe::e4) by CH0PR13CA0048.outlook.office365.com (2603:10b6:610:b2::23) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.9412.6 via Frontend Transport; Mon, 8 Dec 2025 09:33:54 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=satlexmb07.amd.com; pr=C Received: from satlexmb07.amd.com (165.204.84.17) by DS3PEPF0000C37B.mail.protection.outlook.com (10.167.23.5) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9412.4 via Frontend Transport; Mon, 8 Dec 2025 09:33:54 +0000 Received: from BLRKPRNAYAK.amd.com (10.180.168.240) by satlexmb07.amd.com (10.181.42.216) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.17; Mon, 8 Dec 2025 03:33:49 -0600 From: K Prateek Nayak To: Ingo Molnar , Peter Zijlstra , Juri Lelli , Vincent Guittot , Anna-Maria Behnsen , Frederic Weisbecker , Thomas Gleixner CC: , Dietmar Eggemann , Steven Rostedt , Ben Segall , Mel Gorman , Valentin Schneider , K Prateek Nayak , "Gautham R. Shenoy" , Swapnil Sapkal , Shrikanth Hegde , Chen Yu Subject: [RESEND RFC PATCH v2 21/29] sched/fair: Introduce sched_asym_prefer_idle() for ILB kick Date: Mon, 8 Dec 2025 09:27:07 +0000 Message-ID: <20251208092744.32737-21-kprateek.nayak@amd.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20251208083602.31898-1-kprateek.nayak@amd.com> References: <20251208083602.31898-1-kprateek.nayak@amd.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: satlexmb07.amd.com (10.181.42.216) To satlexmb07.amd.com (10.181.42.216) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DS3PEPF0000C37B:EE_|BL1PR12MB5948:EE_ X-MS-Office365-Filtering-Correlation-Id: 6b224a35-5c5c-476d-e3d8-08de363ce7d4 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|1800799024|82310400026|36860700013|376014|7416014; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?Vss68bgW6CtUOjduvKEDL1MQEDJqJmej+en/MgZ3FpCHMwg6QnXxtPTEEShp?= =?us-ascii?Q?ZFZrOYyQ+3+reqb4zhtoz9rLgs+qaKCzrRL64glRXBawEFTfWMaONLYlBltt?= =?us-ascii?Q?QbOFDzoUg66rg++BWfLPuL53xu2a7z9ERKDS92a1N3kV74CQ+WfR9zI5Jf+/?= =?us-ascii?Q?+UeIv7IDWX7gl5uKcxuhqlcMxRROnVKJZRsPjiz7Nz75pDFmGD6bZgLsavM+?= =?us-ascii?Q?4/nK5kEke+/J+lBMicYbYIXlQNA0V8zzKRTJR1Bar3muZ+5qGSBEWjIdZwM1?= =?us-ascii?Q?O5AdcaQuUpvMI9CMnQmOwKGkqVtbkP6JGh5Ul96nPUiGt9uw2XHe9Rq2TaN5?= =?us-ascii?Q?CpGp4PDoOtXJTUTL0oE6n9tOiuvwZcKnKVd3+huFejYFOQY5V+vifxORqyaq?= =?us-ascii?Q?NWlgmdFTLwipxwWmyoQW+c+ZJoUwvmqX8Zstne0/2dQMd7ktGmvZmdEEWfTK?= =?us-ascii?Q?zwwT1h1ZxIqwngftybb1Hm1uAYzufq/q+3SOg2utI871EzndwaChBlaE6nHJ?= =?us-ascii?Q?jRHcI7HPbc1c99h/ZDmIyVDpq70UMK28lBP14bGlJbAwLxFUVLjhdsjD16tG?= =?us-ascii?Q?GzHPzdTD58aNpevHMfwGeWihEQrihtsB4dd0z7/Opm6ULMDlxYj5Fog4SsL9?= =?us-ascii?Q?w3e7BUwUTnPrTno50sM82U2NOEaRpf/3NinYJQXjoX4hz3tX24j0VhKSD9y+?= =?us-ascii?Q?4zhiy7wFp/otm7NT1xH0qxaWuwE3kDaFhRsrhx8sA9NQ/UlKa8g5HoGQDCHN?= =?us-ascii?Q?tGkrctoBTPseZQJFlQMfAVDV11SHNXJhN4ap+y3Ar+/NbmHBnBjcpOAfiCqv?= =?us-ascii?Q?xnaaV0+sTGzdpiWnHs2SxHMeC9GL98SpIL81ANDQ72O1DPu4jHAtul5iH07M?= =?us-ascii?Q?m6AMwRz3mjU7SeoBta0yKEHdxPC9sqZY1+8/JXcauHSb3cdr3F88DoUm+lof?= =?us-ascii?Q?UMGZ34JD4oWFxA03dnAaDvPWixhg9z3pfXhXFivCTnN1i5ThXDLjEhGng3GV?= =?us-ascii?Q?ho/pSfKtXjcTpFdIMl5Fmzm7Io6ngYtnmlTWK9drVwEuotUDd4amAdHxKYre?= =?us-ascii?Q?U0A5Cchv7MFCfzwy1IZFicI4k/4XfFlKxi4mJwHXWxCosjP78XR2QOSxe8vO?= =?us-ascii?Q?N4RurYpzLUazLldfs9f+xUx0T9nllUeYpppIysKWHv24x1XrbOtNZt/LD1LW?= =?us-ascii?Q?lQ1Rk0BnvjXsR+enssMtSh4JM/J8BP7H7rvDnweYfRcq+YNOEHCTVD07DuZ5?= =?us-ascii?Q?/001RtpBNZp/dCNAys2YIKauKBHQeZpG4buXDHissWZ2Q+6xwFFd6xsj8bH+?= =?us-ascii?Q?JBFNmjOjP273G/VlD+Q6Gjv128AYOaW6SwfNcpXkAf5nUa4rj4JypO9Jt89t?= =?us-ascii?Q?Q9O6alvhrIAn/2X48eyst+7Ln5Aoi4SdaKcGhd6RW0Xj2EoybHfyOgHEWRuw?= =?us-ascii?Q?F1BYzqGFtH/vltd2/IzuqCpcaPxsPMS1gGdqHe1ZTM5K6EWxA8HdSSetemoY?= =?us-ascii?Q?aaLIWrMJp0Rr18G5aMsQvwHpqbCdONdOtITMkpiqfh4JuUeXWW+8OrDJPGca?= =?us-ascii?Q?g/n+tUnCldiJvR2JQaQ=3D?= X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:satlexmb07.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230040)(1800799024)(82310400026)(36860700013)(376014)(7416014);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 Dec 2025 09:33:54.7803 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 6b224a35-5c5c-476d-e3d8-08de363ce7d4 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[satlexmb07.amd.com] X-MS-Exchange-CrossTenant-AuthSource: DS3PEPF0000C37B.namprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: BL1PR12MB5948 Content-Type: text/plain; charset="utf-8" nohz_balancer_kick() currently depends on "nohz.idle_cpus_maks" to kick off an idle balance if it finds a more preferred CPU in nohz idle state. Move the detection logic into sched_asym_prefer_idle() and use the distributed "sd->shared" based nohz tracking to kick ILB if a more preferred CPU is found idle. Signed-off-by: K Prateek Nayak --- kernel/sched/fair.c | 70 ++++++++++++++++++++++++++++++++++----------- 1 file changed, 54 insertions(+), 16 deletions(-) diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index dbbb1f3251d2..ca119c57a2a9 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -12393,6 +12393,56 @@ static inline int find_new_ilb(void) return -1; } =20 +static inline int +__asym_prefer_idle(int cpu, struct sched_domain *sd_asym, struct cpumask *= idle_mask) +{ + int i; + + for_each_cpu_and(i, idle_mask, sched_domain_span(sd_asym)) { + if (sched_asym(sd_asym, i, cpu)) + return 1; + } + + return 0; +} + +static int sched_asym_prefer_idle(int cpu, struct sched_domain *sd_asym) +{ + struct sched_domain *sd =3D rcu_dereference(per_cpu(sd_nohz, cpu)); + struct sched_domain_shared *sds; + + if (!sd) + return 0; + + /* + * If sd_asym_packing is a subset of sd_llc, don't traverse + * the nohz_shared_list and directly use the mask from + * sd_nohz->shared to find it a preferred CPU is idling. + */ + if (sd_asym->flags & SD_SHARE_LLC) { + sds =3D sd->shared; + + if (sds && + atomic_read(&sds->nr_idle_cpus) && + __asym_prefer_idle(cpu, sd_asym, sds->nohz_idle_cpus_mask)) + return 1; + + return 0; + } + + /* Try the entire list id sd_asym_packing is larger than sd_nohz. */ + list_for_each_entry_rcu(sds, &nohz_shared_list, nohz_list_node) { + /* No idle CPUs in this domain */ + if (!atomic_read(&sds->nr_idle_cpus)) + continue; + + if (__asym_prefer_idle(cpu, sd_asym, sds->nohz_idle_cpus_mask)) + return 1; + } + + return 0; +} + /* * Kick a CPU to do the NOHZ balancing, if it is time for it, via a cross-= CPU * SMP function call (IPI). @@ -12447,7 +12497,7 @@ static void nohz_balancer_kick(struct rq *rq) unsigned long now =3D jiffies; struct sched_domain_shared *sds; struct sched_domain *sd; - int nr_busy, i, cpu =3D rq->cpu; + int nr_busy, cpu =3D rq->cpu; unsigned int flags =3D 0; =20 if (unlikely(rq->idle_balance)) @@ -12493,21 +12543,9 @@ static void nohz_balancer_kick(struct rq *rq) } =20 sd =3D rcu_dereference(per_cpu(sd_asym_packing, cpu)); - if (sd) { - /* - * When ASYM_PACKING; see if there's a more preferred CPU - * currently idle; in which case, kick the ILB to move tasks - * around. - * - * When balancing between cores, all the SMT siblings of the - * preferred CPU must be idle. - */ - for_each_cpu_and(i, sched_domain_span(sd), nohz.idle_cpus_mask) { - if (sched_asym(sd, i, cpu)) { - flags =3D NOHZ_STATS_KICK | NOHZ_BALANCE_KICK; - goto unlock; - } - } + if (sd && sched_asym_prefer_idle(cpu, sd)) { + flags =3D NOHZ_STATS_KICK | NOHZ_BALANCE_KICK; + goto unlock; } =20 sd =3D rcu_dereference(per_cpu(sd_asym_cpucapacity, cpu)); --=20 2.43.0