From nobody Thu Apr 2 21:51:46 2026 Received: from PH8PR06CU001.outbound.protection.outlook.com (mail-westus3azon11012068.outbound.protection.outlook.com [40.107.209.68]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 374854014AD for ; Thu, 26 Mar 2026 15:12:34 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.209.68 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1774537956; cv=fail; b=MKxlPygZKG9RK+m+JpD+OyiFZfSLhEo3qRcERHNBaxWlsSCh4FJVkiqMVhc3OaTP+Wz87+IjIQvcBQb9Bpwnixmr6Y8xzg+9uAA9jRbLVYR1CyuSD78G4TfhpT2IZSjpfkuGFiMM7OZtfmEbYCAg2OywCQu8fvSNiT2mFOmvl/s= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1774537956; c=relaxed/simple; bh=tUoQrXG3Gp2q96ZCppKbCI4b3DgP6oi9fptJ7xiyYZw=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: Content-Type:MIME-Version; b=K53LTSD32myn2ZwWtXKvndaR/XhD8MKNT0MF5r+l0d8yL1Mx6GvEJAVntUkww86GJtibp5vU+4jtMJpcgknyw1EoqSdrws8hKgoGTR3S0QdcLiXVK4igqghkL1khiLp0dztqQmpoVKZTSpHCWt6VGP5MrJYH4auuc5FL5v49qsY= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=I3bT9fX3; arc=fail smtp.client-ip=40.107.209.68 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="I3bT9fX3" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=NtyCXlmYatNTItpsGCZ5jLRV+ZO8IkIGZFLd6CWvIWWPmMY+SZBbL9GjTnnT/gAos9rYa0Pbp2yuPv8uLYSSovTJYMQiQ0Mqbs5t9CWIC9aGDqpAMxjqS+YHeS0w5cEWxK9DqJphhN7jlbMQN1XzBeMBlp9Jh0bR2D/R7dJ8gbS8SEuu55kw4hiTAje6bAVopWlK2r6qKZzTDOcL/VfyYD0ATENkKoGbO8bL0u8HWVSOwfpqgOcu6Y8/QBeKibBzgdd6pt8WMI20fExcPkhjs9c595xOX8GaaB7EwIAGHy1lUO6UKsUJj5YWUzxQtjMPDPWshlF6nN6T17jtKvVf1g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=4ryI5MizuV6tTxqWf2cul6jgmqvZujOu5sPGM7sIohs=; b=xHox8imchXi1EsFAcA65A6TAGLhKeJYzjhw5fw//TJ9XHBUKKHNyw4uG1unI1jleevqy5mbXrxy8MIzaR83NN2AAPTZ9os6V6F3/foEgTGgsGFsq2f+Xz6/DrEDdwiS0z//OjCR0GbZoLDjJ0Kuq5lypQ0yvQ6tzL755canwf7rOqbB16wB0cTQP91x/DpkLJi6lkxGkYmeRXCpdpLTOSjEzR07stQGQXpd/GLkp5Co6rNW4zUUiGp4txi52Vv9QyO76dR8TpbhSMuq7z8i+zGtXFdf8hNzZK2sKiNZYx0dMac+i2KqMilqn9/iKGPdTAYc8KRsmo+mw9W8gC7fjag== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=4ryI5MizuV6tTxqWf2cul6jgmqvZujOu5sPGM7sIohs=; b=I3bT9fX34NevUQb261WVYk8bN5aOCe/oowJAhhYUrg3I7e494MI1aNTjLg8VK18anePpj3NwKCkUpwwRjsxTG/E879ra5DSjdOPdBkBI/bjtZ3ZyMklA660y0AoEbJBOz75cHhNvNx7JDbJPQJHWDjM1+sNzujFpEhedzVPAfvIrLbJSVjNC0Tg4reTs+N9Ql14xUruBylHCcohCBDTYT9R8OyTGTciKLV6/gGX93OM7qFQ6xaj75lMOrmTkawXlJmO00hnav3j9ZsfTS9/riSWY1upyupmMO0iQfQ8GAf2DnfrYNrOoFQ8IB8gjqWH6/SIG7IMuHOndw/hoLjbRXg== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; Received: from LV8PR12MB9620.namprd12.prod.outlook.com (2603:10b6:408:2a1::19) by CY1PR12MB9627.namprd12.prod.outlook.com (2603:10b6:930:104::9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9745.20; Thu, 26 Mar 2026 15:12:30 +0000 Received: from LV8PR12MB9620.namprd12.prod.outlook.com ([fe80::299d:f5e0:3550:1528]) by LV8PR12MB9620.namprd12.prod.outlook.com ([fe80::299d:f5e0:3550:1528%5]) with mapi id 15.20.9745.019; Thu, 26 Mar 2026 15:12:30 +0000 From: Andrea Righi To: Ingo Molnar , Peter Zijlstra , Juri Lelli , Vincent Guittot Cc: Dietmar Eggemann , Steven Rostedt , Ben Segall , Mel Gorman , Valentin Schneider , Christian Loehle , Koba Ko , Felix Abecassis , Balbir Singh , linux-kernel@vger.kernel.org Subject: [PATCH 1/4] sched/fair: Prefer fully-idle SMT cores in asym-capacity idle selection Date: Thu, 26 Mar 2026 16:02:34 +0100 Message-ID: <20260326151211.1862600-2-arighi@nvidia.com> X-Mailer: git-send-email 2.53.0 In-Reply-To: <20260326151211.1862600-1-arighi@nvidia.com> References: <20260326151211.1862600-1-arighi@nvidia.com> Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: SJ0PR13CA0137.namprd13.prod.outlook.com (2603:10b6:a03:2c6::22) To LV8PR12MB9620.namprd12.prod.outlook.com (2603:10b6:408:2a1::19) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: LV8PR12MB9620:EE_|CY1PR12MB9627:EE_ X-MS-Office365-Filtering-Correlation-Id: ba16985d-8833-4a99-bf1c-08de8b4a1952 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|366016|1800799024|376014|7416014|56012099003|22082099003|18002099003; X-Microsoft-Antispam-Message-Info: 8Kly82nl/iqmZfgWA6wEFz7XmSr/DnIlzpj9C/K1PQNAt22ZOsOp3MWMfVB//tRN9veOEStmJdJTMLLoOAvQPSI7udO31TaE8fdA52Bw7faqc8kXp9QyqBi/UuARyqgs5HA0oErsQb9hmIUlGmfESPkJTpzcT5R7hadNmrpbC+AxFOhkuJqW0nM7RGRY7FKeD00zgLCWbKEYCpbvRvcik2qmUppQvgqrLExZENtHp+1L1pdratbC6Z+4BBOuyxGPBhxJRocDa1Jlf7AwknnyAzvzuVelm2tXpw/zPCmVdlromTIo7Dczf0kSUM9Y/zoFMUWw93clQ+1ICt3efUWY/fj943GwPSiZzyc71QZXnOSYdXBOEEKUNFYnQNh+G+wjSJrUh33UIeXTHX8ssNClHHdwnS3Fiu3cdwXY9uxLAd9bQqB8GcmcIGIMKQvIBbkqeP6Wz1YFdJNuvVhPw1gL/1Jk5dmwW+Eg/enxAMkvxODNX1sCOIxbSeFf4otPI70Vc05eMYun8kokPbnL9IisjSdgOJgc+6Ooy+K85/lBDOuwBH8Q1mV5/3TDTlnu/K7F/Teq7eijTRO9xaylh9Ssl95F07SEwBZSb7zzVk9AOU0uiNZuQQgVcA8/s9xpFzR1kQ2b9KXf4FZAmoZsbLuk8qAhVrGtxOf5oFqjkfyk1wt3MK/lbycLsNzIT89WiDx3S2FtShOrp6JKRK+mOCc7zUkYcEQdON8k22xlU2IKg3o= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:LV8PR12MB9620.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(366016)(1800799024)(376014)(7416014)(56012099003)(22082099003)(18002099003);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?38mkHVx1eN2n/D5yWDU7Pxv5qYexJ0SfjwBYWZd57XbqBlHggFE45TZJoygw?= =?us-ascii?Q?IVy6SKY9ueWJ2xnTlHZItTLrFwb3Dm9BZvFU+Ri0KWgEduylGzNbgH5iSnsH?= =?us-ascii?Q?45FDhiNH2e8ZNoquBefO35OoweAxkaCM/VSW8i++7VGdFjnnqHatkNmcCvH6?= =?us-ascii?Q?eMY0gucS56TtzD5J+Zgm9oJkwbhf1cD9Wl/tRAXXH10nM+jyVxzUDJwoluPO?= =?us-ascii?Q?nTIu0DK3qBdWgvTpYHc7bQvELKF2iGJFvHHh0TUchzphcRAqlIIZ2nuhB2wG?= =?us-ascii?Q?ACJUAEoNxMk3GjsYPL/u06u5LhExo30O4kSgWTrl3+OpIuHzq/2Ru0iigt8K?= =?us-ascii?Q?tuis1Gw0lWBZ821DGgcFvfs6GAvA399GmGN5OFolQgRjqsW7vRLjaLy3zg4q?= =?us-ascii?Q?+myzUNjTY/JRgsv7v90Di2S058NsJXr0lDIeFftY96H1ke5XVsns6p1j68Bb?= =?us-ascii?Q?eM7K0PxUOClMOjHBdftIXM2OAsyLvdLEjhRWan1PcD6+c7WQ9+UJgAoOAF+3?= =?us-ascii?Q?m4h2en0/tcXmQiKYjhWydgartUNYHWdUfzx/YrBnr3FvaFmMyWuq/RcBJDI2?= =?us-ascii?Q?VIAhGKqTNw8cPtb/o5/u5WB3Yd/2pveoAV5pd/Cc70tXShwifB3uGMnWe+Bn?= =?us-ascii?Q?rmUtPMbnyUXfIe2ZLuvM5TscnXq+G4g5RuVfRnD5lId8nqRmfobxVIXp7LIp?= =?us-ascii?Q?QuxR1saLrxvoGKQN9D8AdMor+2aVOwHjGK5cIVpMiQvbbK/D1t0c8WCp+lVR?= =?us-ascii?Q?azP9fR+t70beFNUqQqZDy1hVDLYhH9tHAkj4FeRVU3KfqVFCjrX0/1C7OlLd?= =?us-ascii?Q?+7j4/OxzNXwx0o4yqyJ/8+rw16u1xwPixUVFQ+7Al1GqYSdP1vZ24oSeDJFL?= =?us-ascii?Q?HyhMOMs4O4WukOWKdVCpQMitBb57yZjiDnS9kxsv6SlOn04Nj5Ra4vOET4Tc?= =?us-ascii?Q?yj0EO/q7EjEqHWIgovlwfOk57DvmMAj8HtW8ve3wZjVL+RHtWMmQdzgoHQOy?= =?us-ascii?Q?KocbcxdxxdnpQk84Y+vl2wA9yPmEYntbLN+3mwHAD1TRsqlyrvKub+uCthxX?= =?us-ascii?Q?DVd9vaJ8I6T2kPzIQWKkspRgTelbV+K8DAgsI2GhQ3rTlb89AAdLJX2ELMrf?= =?us-ascii?Q?TxaKmAYD+O5TdFf7wg5IxQgL1dwaJ8wdPcXgTHJA/52sVebqH64unkJjw6GS?= =?us-ascii?Q?28Ex+i87BCFTz/6vuFKzB5cEpRFX3wY8OynyOQBiRoHGdn0um0he0xRGHPFY?= =?us-ascii?Q?Zcitxt6BaaQ1HnkNuBV7YQMcORE2+E332WnyVM2QofUpQEj4Nn2Ow9pEvjaJ?= =?us-ascii?Q?90DgDAOmW0BUiSoMK1KvW70A5G48cdk6pGnSv7XdBmvZ570tDatE3iYzk0Co?= =?us-ascii?Q?7vBZ3NsOdKy1lvn90PNzuaaVZ/i3C8ckfXuXCcHltnLyqjOpteT5YsBlTyhi?= =?us-ascii?Q?YWtE/JQJfmGD2YWCRoJ9ejcYLqiLfw0Lr2834B9pSexiZuBKmKswLb9RdexV?= =?us-ascii?Q?8WPt5BWXjLtIKA4BfftU3/hkdjDET5nc8ED5fgSq0aKZi64bFM12i9uNskOp?= =?us-ascii?Q?mavKp7j1cDauF450g6dumyZWegvj4y+oTkQ76DfosGE7KQhmaB6B29DeZunU?= =?us-ascii?Q?SKqmVyC4f0riO1Hml3Sn8hgoBP6rsWLL5TJTfcZl4MSKzNP4rxkbmTJokEHi?= =?us-ascii?Q?bjDKOOz5iP0LIPoSQCNj7nTfaZfLKR+44hztHIpBjBt8YpQK3ZmNQS/WRXhw?= =?us-ascii?Q?L7oqg4C/yw=3D=3D?= X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: ba16985d-8833-4a99-bf1c-08de8b4a1952 X-MS-Exchange-CrossTenant-AuthSource: LV8PR12MB9620.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 26 Mar 2026 15:12:30.4340 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: dMe2cpqKaeQPan72Y3MtatU5VRv1geIH9tsXtZOqp/cwc5/ZDp2613kuk57XzowiQyhGQldAvH6GnWWZthLg9w== X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY1PR12MB9627 Content-Type: text/plain; charset="utf-8" On systems with asymmetric CPU capacity (e.g., ACPI/CPPC reporting different per-core frequencies), the wakeup path uses select_idle_capacity() and prioritizes idle CPUs with higher capacity for better task placement. However, when those CPUs belong to SMT cores, their effective capacity can be much lower than the nominal capacity when the sibling thread is busy: SMT siblings compete for shared resources, so a "high capacity" CPU that is idle but whose sibling is busy does not deliver its full capacity. This effective capacity reduction cannot be modeled by the static capacity value alone. Introduce SMT awareness in the asym-capacity idle selection policy: when SMT is active prefer fully-idle SMT cores over partially-idle ones. A two-phase selection first tries only CPUs on fully idle cores, then falls back to any idle CPU if none fit. Prioritizing fully-idle SMT cores yields better task placement because the effective capacity of partially-idle SMT cores is reduced; always preferring them when available leads to more accurate capacity usage on task wakeup. On an SMT system with asymmetric CPU capacities, SMT-aware idle selection has been shown to improve throughput by around 15-18% for CPU-bound workloads, running an amount of tasks equal to the amount of SMT cores. Cc: Vincent Guittot Cc: Dietmar Eggemann Cc: Christian Loehle Cc: Koba Ko Reported-by: Felix Abecassis Signed-off-by: Andrea Righi --- kernel/sched/fair.c | 86 +++++++++++++++++++++++++++++++++++++++------ 1 file changed, 75 insertions(+), 11 deletions(-) diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index d57c02e82f3a1..9a95628669851 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -7940,14 +7940,21 @@ static int select_idle_cpu(struct task_struct *p, s= truct sched_domain *sd, bool * Scan the asym_capacity domain for idle CPUs; pick the first idle one on= which * the task fits. If no CPU is big enough, but there are idle ones, try to * maximize capacity. + * + * When @prefer_idle_cores is true (asym + SMT and idle cores exist), pref= er + * CPUs on fully-idle cores over partially-idle ones in a single pass: tra= ck + * the best candidate among idle-core CPUs and the best among any idle CPU, + * then return the idle-core candidate if found, else the best any-idle. */ static int -select_idle_capacity(struct task_struct *p, struct sched_domain *sd, int t= arget) +select_idle_capacity(struct task_struct *p, struct sched_domain *sd, int t= arget, + bool prefer_idle_cores) { - unsigned long task_util, util_min, util_max, best_cap =3D 0; - int fits, best_fits =3D 0; - int cpu, best_cpu =3D -1; + unsigned long task_util, util_min, util_max, best_cap =3D 0, best_cap_cor= e =3D 0; + int fits, best_fits =3D 0, best_fits_core =3D 0; + int cpu, best_cpu =3D -1, best_cpu_core =3D -1; struct cpumask *cpus; + bool on_idle_core; =20 cpus =3D this_cpu_cpumask_var_ptr(select_rq_mask); cpumask_and(cpus, sched_domain_span(sd), p->cpus_ptr); @@ -7962,16 +7969,58 @@ select_idle_capacity(struct task_struct *p, struct = sched_domain *sd, int target) if (!choose_idle_cpu(cpu, p)) continue; =20 + on_idle_core =3D is_core_idle(cpu); + if (prefer_idle_cores && !on_idle_core) { + /* Track best among any idle CPU for fallback */ + fits =3D util_fits_cpu(task_util, util_min, util_max, cpu); + if (fits > 0) { + /* + * Full fit: strictly better than fits 0 / -1; + * among several, prefer higher capacity. + */ + if (best_cpu < 0 || best_fits <=3D 0 || + (best_fits > 0 && cpu_cap > best_cap)) { + best_cap =3D cpu_cap; + best_cpu =3D cpu; + best_fits =3D fits; + } + continue; + } + if (best_fits > 0) + continue; + if (fits < 0) + cpu_cap =3D get_actual_cpu_capacity(cpu); + if ((fits < best_fits) || + ((fits =3D=3D best_fits) && (cpu_cap > best_cap))) { + best_cap =3D cpu_cap; + best_cpu =3D cpu; + best_fits =3D fits; + } + continue; + } + fits =3D util_fits_cpu(task_util, util_min, util_max, cpu); =20 /* This CPU fits with all requirements */ - if (fits > 0) - return cpu; + if (fits > 0) { + if (prefer_idle_cores && on_idle_core) + return cpu; + if (!prefer_idle_cores) + return cpu; + /* + * Prefer idle cores: record and keep looking for + * idle-core fit. + */ + best_cap =3D cpu_cap; + best_cpu =3D cpu; + best_fits =3D fits; + continue; + } /* * Only the min performance hint (i.e. uclamp_min) doesn't fit. * Look for the CPU with best capacity. */ - else if (fits < 0) + if (fits < 0) cpu_cap =3D get_actual_cpu_capacity(cpu); =20 /* @@ -7984,8 +8033,17 @@ select_idle_capacity(struct task_struct *p, struct s= ched_domain *sd, int target) best_cpu =3D cpu; best_fits =3D fits; } + if (prefer_idle_cores && on_idle_core && + ((fits < best_fits_core) || + ((fits =3D=3D best_fits_core) && (cpu_cap > best_cap_core)))) { + best_cap_core =3D cpu_cap; + best_cpu_core =3D cpu; + best_fits_core =3D fits; + } } =20 + if (prefer_idle_cores && best_cpu_core >=3D 0) + return best_cpu_core; return best_cpu; } =20 @@ -7994,12 +8052,17 @@ static inline bool asym_fits_cpu(unsigned long util, unsigned long util_max, int cpu) { - if (sched_asym_cpucap_active()) + if (sched_asym_cpucap_active()) { /* * Return true only if the cpu fully fits the task requirements * which include the utilization and the performance hints. + * + * When SMT is active, also require that the core has no busy + * siblings. */ - return (util_fits_cpu(util, util_min, util_max, cpu) > 0); + return (!sched_smt_active() || is_core_idle(cpu)) && + (util_fits_cpu(util, util_min, util_max, cpu) > 0); + } =20 return true; } @@ -8097,8 +8160,9 @@ static int select_idle_sibling(struct task_struct *p,= int prev, int target) * capacity path. */ if (sd) { - i =3D select_idle_capacity(p, sd, target); - return ((unsigned)i < nr_cpumask_bits) ? i : target; + i =3D select_idle_capacity(p, sd, target, + sched_smt_active() && test_idle_cores(target)); + return ((unsigned int)i < nr_cpumask_bits) ? i : target; } } =20 --=20 2.53.0