From nobody Tue Jul 14 12:48:06 2026 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8FB3636607D for ; Mon, 22 Jun 2026 23:55:33 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.18 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1782172535; cv=none; b=DLcgT3gc28OlY1iBcBb6pTL+NJ3/0ohE8Oj27wxKw51uQLmTv2DdkXwQQsOBJA12nrzfYjavI2Sl/1xzA5BvsFZbvhvmEKVsEq9eoLTqPwZV2pN/pKM8SHXMtLTv0X3PMJzqVVvulensT0GMIQluzO2j5lmUzKzotuxo+6nNwd0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1782172535; c=relaxed/simple; bh=rIv5DUFF/rIw+1AEZVR7V7zJ0tyvCad2Xb+WNroBq84=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=tLC1fJMQ007R7BofSe6+1iPvH659QMfpShwrDqh9siQHxlRdg5x2LW5Z1sLnoqb8SX6BWNfK4fxhN9yFV9V4fauJRzWXf/Hj9o39F6mDAX7yJgdyHSBwIFSFcxsVH0yFpCxVeA4buw7Fj/CYllQ7aP63jvESgk5K9rsbgqOhZ1I= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com; spf=pass smtp.mailfrom=linux.intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=X4orRZc6; arc=none smtp.client-ip=192.198.163.18 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="X4orRZc6" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1782172534; x=1813708534; h=from:date:subject:mime-version:content-transfer-encoding: message-id:references:in-reply-to:to:cc; bh=rIv5DUFF/rIw+1AEZVR7V7zJ0tyvCad2Xb+WNroBq84=; b=X4orRZc6vyuJ9k/dvwRZIV1u9lT0UwkoUul/wkEFuj8VeWOGaOA3iYlR Z3JpO/SETCEVX971ANIEjxX9195WSqDtPvM4kUtGNlEpqgIDh+WqBxnjR gFgAXrY6P0nJfmbkQqNP45/HgoeDpEFPtM3kqJ4R0SGV+ScZPQR/T3JT+ g9oeer4DLjR6Oz98gnYdr3sJN2CMzv1N3WdN3pnwNWrtvwSiNzcORSgqV X2FjnlzIuIjrSKzVfzo3RPGh6cjNH0sQ/5gy/kd+j3aitsOcvykvSCwe9 KjMM8ztQyWik+GV7x9wvVxAO+Pps8MMvZZZYz4gLURqXCEXP6N3/HEFgU A==; X-CSE-ConnectionGUID: oeozvhlTSgS4pE8/4Bm/1w== X-CSE-MsgGUID: pGsXBzFfTFyGQvstCJzLPA== X-IronPort-AV: E=McAfee;i="6800,10657,11825"; a="82014479" X-IronPort-AV: E=Sophos;i="6.24,219,1774335600"; d="scan'208";a="82014479" Received: from fmviesa003.fm.intel.com ([10.60.135.143]) by fmvoesa112.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 22 Jun 2026 16:55:31 -0700 X-CSE-ConnectionGUID: tWX+ML2BTA6ahZcvSTbOGQ== X-CSE-MsgGUID: flGQMeo5QdyQBBN/8gJZlg== X-ExtLoop1: 1 Received: from unknown (HELO [172.25.112.21]) ([172.25.112.21]) by fmviesa003.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 22 Jun 2026 16:55:30 -0700 From: Ricardo Neri Date: Mon, 22 Jun 2026 17:05:51 -0700 Subject: [PATCH v5 1/6] sched/fair: Do not skip CPUs of similar capacity with busy SMT siblings Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260622-rneri-fix-cas-clusters-v5-1-19968f2d1497@linux.intel.com> References: <20260622-rneri-fix-cas-clusters-v5-0-19968f2d1497@linux.intel.com> In-Reply-To: <20260622-rneri-fix-cas-clusters-v5-0-19968f2d1497@linux.intel.com> To: Ingo Molnar , Peter Zijlstra , Juri Lelli , Vincent Guittot , Dietmar Eggemann , Steven Rostedt , Ben Segall , Mel Gorman , Valentin Schneider , Tim C Chen , Chen Yu , Christian Loehle , K Prateek Nayak , Barry Song Cc: "Rafael J. Wysocki" , Andrea Righi , Len Brown , ricardo.neri@intel.com, linux-kernel@vger.kernel.org, Ricardo Neri X-Mailer: b4 0.13.0 X-Developer-Signature: v=1; a=ed25519-sha256; t=1782173185; l=2203; i=ricardo.neri-calderon@linux.intel.com; s=20250602; h=from:subject:message-id; bh=rIv5DUFF/rIw+1AEZVR7V7zJ0tyvCad2Xb+WNroBq84=; b=QQaaoBEpGHJH2mnYN+7N4TxkHRt8nJjnG4hOBsFaPZGoTIhTkirgMbgGWQ/Ho9jG2rRCag3tz p0Yb0withTYBQ5m7sSeCC2u7/ro2PcvdVfQlPGSAa83O9/a6Od7oaQ4 X-Developer-Key: i=ricardo.neri-calderon@linux.intel.com; a=ed25519; pk=NfZw5SyQ2lxVfmNMaMR6KUj3+0OhcwDPyRzFDH9gY2w= When picking a busiest CPU with only one running task, the function sched_balance_find_src_rq() skips candidate CPUs if the destination CPU has less than ~5% extra capacity. This condition only holds if all the SMT siblings of a CPU are idle. SMT siblings share the computing resources of a physical core and this results in reduced capacity if more than one sibling is busy. Skipping a CPU as described would prevent the load balancer from pulling tasks from a scheduling group previously and correctly identified as group_smt_balance (i.e., one with more than one task running). Do not skip a candidate CPU of similar capacity if it has busy SMT siblings. Reviewed-by: K Prateek Nayak Signed-off-by: Ricardo Neri Reviewed-by: Vincent Guittot Tested-by: Andrea Righi --- Changes in v5: * Optimized logic to identify CPUs with busy SMT siblings only when needed. (Prateek, Chen Yu) * Added Reviewed-by tag from Prateek. Thanks! * Christian also provided his Reviewed-by tag, but the patch changed significantly since then. I did not think it was correct to keep it without him reviewing the updated patch first. Changes in v4: * Introduced this patch. Changes in v3: * N/A Changes in v2: * N/A --- kernel/sched/fair.c | 14 +++++++++++--- 1 file changed, 11 insertions(+), 3 deletions(-) diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index d78467ec6ee1..892abd7fcc18 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -12976,9 +12976,17 @@ static struct rq *sched_balance_find_src_rq(struct= lb_env *env, * average load. */ if (env->sd->flags & SD_ASYM_CPUCAPACITY && - !capacity_greater(capacity_of(env->dst_cpu), capacity) && - nr_running =3D=3D 1) - continue; + nr_running =3D=3D 1) { + bool smt_degraded_cap =3D sched_smt_active() && !is_core_idle(i); + + /* + * Busy SMT siblings reduce the capacity of CPU @i. Do + * not skip it in this case. + */ + if (!smt_degraded_cap && + !capacity_greater(capacity_of(env->dst_cpu), capacity)) + continue; + } =20 /* * Make sure we only pull tasks from a CPU of lower priority --=20 2.43.0 From nobody Tue Jul 14 12:48:06 2026 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id F39E5372063 for ; Mon, 22 Jun 2026 23:55:34 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.18 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1782172536; cv=none; b=WBDbwMmR/LkXttGWr8JWk9sY84ueYQVHuwg40NRpRdZ+/y0F2Q2x21P9vKDOBtI7JDZwVDOR0mTeB1rVsy7TbEmwSOqEpry/i/yj+TAqZdt0etZozg0UkIR6Q556Z9sqdQFf4jyh6i9m7xWfXRi/x+tqsTvaI4EXa8NPA6BmyJw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1782172536; c=relaxed/simple; bh=gokVUSh2aCHv64iZ9iLvkNxqTSkJDs8XN4M4koe4Cuw=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=RO/Lw1A4p/sq/S2wI29Va5qzF6acIdSwzTOmMGyh+4gX79hO3m9eXYwOQiTEJi/aJ2GWhji0jLyPFOInv7UzRhOqPskegua99VLMltaORzKVgmylf0aEajuP9oE3V7HhWI3nEu23gh8aYZSwmsefNr+jRousxKY+YXo9v+MioMQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com; spf=pass smtp.mailfrom=linux.intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=BxSf3tO8; arc=none smtp.client-ip=192.198.163.18 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="BxSf3tO8" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1782172535; x=1813708535; h=from:date:subject:mime-version:content-transfer-encoding: message-id:references:in-reply-to:to:cc; bh=gokVUSh2aCHv64iZ9iLvkNxqTSkJDs8XN4M4koe4Cuw=; b=BxSf3tO8m1FpBS+o0M37xH2aeeujSLGQzZZ8x12eTzMkDqFeL4Oz5sKv 524eItbWx07j1B5bWDO4Yc+PqMo/WywfhhikZkVbUr0OOtfwBpU+F9bDW m6h75480JUJgb9KvjiRwgEWx420kiomdnLraOjQ/EDJa/snOYy1Ut8aTJ 1mYnjIie6Nkblkfj8u39w+nmbTMh7q2qHc2eUYJGd2lh9uFJV39CILldR UQ6O4C56+mOg/7HXjmn74lSv54w+YJUSvmNcPK6Y1WL7DYpXSAok7iA4f LaS9TTl183ScmeYb0vC6msFOK+NUksgILzXMvxPqH34+xKOyhFuJSlAEY Q==; X-CSE-ConnectionGUID: XNdaxO1QThSrzHcf4MlvXQ== X-CSE-MsgGUID: OY2kEUbBR9SM9xaLWrPXcQ== X-IronPort-AV: E=McAfee;i="6800,10657,11825"; a="82014491" X-IronPort-AV: E=Sophos;i="6.24,219,1774335600"; d="scan'208";a="82014491" Received: from fmviesa003.fm.intel.com ([10.60.135.143]) by fmvoesa112.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 22 Jun 2026 16:55:31 -0700 X-CSE-ConnectionGUID: WEJNhn3NSrqOaH5o+Kp8dw== X-CSE-MsgGUID: +RsXrgmaSWys7rRh0JNcYQ== X-ExtLoop1: 1 Received: from unknown (HELO [172.25.112.21]) ([172.25.112.21]) by fmviesa003.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 22 Jun 2026 16:55:30 -0700 From: Ricardo Neri Date: Mon, 22 Jun 2026 17:05:52 -0700 Subject: [PATCH v5 2/6] sched/fair: Also gate overloaded status update for SD_ASYM_CPUCAPACITY Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260622-rneri-fix-cas-clusters-v5-2-19968f2d1497@linux.intel.com> References: <20260622-rneri-fix-cas-clusters-v5-0-19968f2d1497@linux.intel.com> In-Reply-To: <20260622-rneri-fix-cas-clusters-v5-0-19968f2d1497@linux.intel.com> To: Ingo Molnar , Peter Zijlstra , Juri Lelli , Vincent Guittot , Dietmar Eggemann , Steven Rostedt , Ben Segall , Mel Gorman , Valentin Schneider , Tim C Chen , Chen Yu , Christian Loehle , K Prateek Nayak , Barry Song Cc: "Rafael J. Wysocki" , Andrea Righi , Len Brown , ricardo.neri@intel.com, linux-kernel@vger.kernel.org, Ricardo Neri X-Mailer: b4 0.13.0 X-Developer-Signature: v=1; a=ed25519-sha256; t=1782173185; l=1527; i=ricardo.neri-calderon@linux.intel.com; s=20250602; h=from:subject:message-id; bh=gokVUSh2aCHv64iZ9iLvkNxqTSkJDs8XN4M4koe4Cuw=; b=n2xQUwalGB3oIJuJNye0BW2THI+900LFdDdfF05tQ6NDObl9xF0Bjf52VFjAWdmYZq9r41JVS bj38p3u/fq9DyEDD/FUTUSvnqFJU3chJig2DWBIOWcvu/hk/WiS2K33 X-Developer-Key: i=ricardo.neri-calderon@linux.intel.com; a=ed25519; pk=NfZw5SyQ2lxVfmNMaMR6KUj3+0OhcwDPyRzFDH9gY2w= The argument sg_overloaded of update_sg_lb_stats() is only consumed when balancing at the root domain. It only makes sense to update it in such a case. Commit 3229adbe7875 ("sched/fair: Do not compute overloaded status unnecessarily during lb") updated the logic accordingly but missed the case in which the root domain has the SD_ASYM_CPUCAPACITY flag. Fix this. Fixes: 3229adbe7875 ("sched/fair: Do not compute overloaded status unnecess= arily during lb") Tested-by: Christian Loehle Reported-by: Chen Yu Signed-off-by: Ricardo Neri Reviewed-by: Vincent Guittot Tested-by: Andrea Righi --- Changes in v5: * Added Tested-by tag from Christian. Thanks! Changes in v4: * Introduced this patch. Changes in v3: * N/A Changes in v2: * N/A --- kernel/sched/fair.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index 892abd7fcc18..31baa0000616 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -11863,7 +11863,9 @@ static inline void update_sg_lb_stats(struct lb_env= *env, /* Check for a misfit task on the cpu */ if (sgs->group_misfit_task_load < rq->misfit_task_load) { sgs->group_misfit_task_load =3D rq->misfit_task_load; - *sg_overloaded =3D 1; + + if (balancing_at_rd) + *sg_overloaded =3D 1; } } else if (env->idle && sched_reduced_capacity(rq, env->sd)) { /* Check for a task running on a CPU with reduced capacity */ --=20 2.43.0 From nobody Tue Jul 14 12:48:06 2026 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 40030372665 for ; Mon, 22 Jun 2026 23:55:35 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.18 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1782172536; cv=none; b=i3Bi0gahwJdSpZolaCCZu1fiRw03RZ29mrFk9gDZRDpMW4DSfuE7yJtjpBC91+DoOYk8OY+2Fj7ugx8Hxq1B+59R7Vz5oIa4ywPVq6T0aCHBo1A7scTwtOUM7Q7Ht9Xr3sfe/PMtvhGfgEtKKRvaw34oCcLh6HyYlT0hHqge6w8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1782172536; c=relaxed/simple; bh=xqTTARIjfuZL/TzYafjUsaZtElpWOzKIQUdIG0iqn5Q=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=sXLTAzd4CSxB+Fmyp6h0F9mImSxd7wJ8y194YUjvF2DQASOMgfl8IHyT7ZUUHxbWUMrmjWGbUJ2T6om4Y6kGLNzXpt4/22dy+Y48FKU6rpRqwXOWsxoojwMNkk5/X5TG1rHIJ/ygLdo08SseHCut1289jP3wTKjF+4Yerk4N/lo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com; spf=pass smtp.mailfrom=linux.intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=l95Uw6PA; arc=none smtp.client-ip=192.198.163.18 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="l95Uw6PA" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1782172535; x=1813708535; h=from:date:subject:mime-version:content-transfer-encoding: message-id:references:in-reply-to:to:cc; bh=xqTTARIjfuZL/TzYafjUsaZtElpWOzKIQUdIG0iqn5Q=; b=l95Uw6PAUIy8ALYdbHvev+FHDiFf2LiJfPrvfdz+bLkHvjUPy1CPiJdG gZ/vXflRFh23m75ZPlC3KFI+lokF3Qyz3q19fd53SGLLFGIJZEl8uB0fx n/gBad21pbDQAQcrDlmpNTzVjhdqPjNGg8mKUu+72+ocHXBpWh0OLK6ej f5hiVLLjPqmZXtTVFphvTrN0szyfohUUYZJ6MT2QzncGlcY1pyGZl/4GR IzC1k6+9IHheWI1b/K/TW4v299lsgebe4ZGM/grZ3c5hCr+qBMYKuwhPW czzp+EBptVvDWiAX769bkL4pDCr6gPK2EMgUjSuKI261cebJrk5N70xtF A==; X-CSE-ConnectionGUID: PAwXpLY+T4OnOlKsx1nmWg== X-CSE-MsgGUID: cPf9Zl5fSHWOe4QaMwQ3wA== X-IronPort-AV: E=McAfee;i="6800,10657,11825"; a="82014494" X-IronPort-AV: E=Sophos;i="6.24,219,1774335600"; d="scan'208";a="82014494" Received: from fmviesa003.fm.intel.com ([10.60.135.143]) by fmvoesa112.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 22 Jun 2026 16:55:31 -0700 X-CSE-ConnectionGUID: E/D5BFCYRDqKmEN7T6HYYg== X-CSE-MsgGUID: tKrZY11BThG2skw3+NpzAQ== X-ExtLoop1: 1 Received: from unknown (HELO [172.25.112.21]) ([172.25.112.21]) by fmviesa003.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 22 Jun 2026 16:55:30 -0700 From: Ricardo Neri Date: Mon, 22 Jun 2026 17:05:53 -0700 Subject: [PATCH v5 3/6] sched/fair: Check CPU capacity before comparing group types during load balance Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260622-rneri-fix-cas-clusters-v5-3-19968f2d1497@linux.intel.com> References: <20260622-rneri-fix-cas-clusters-v5-0-19968f2d1497@linux.intel.com> In-Reply-To: <20260622-rneri-fix-cas-clusters-v5-0-19968f2d1497@linux.intel.com> To: Ingo Molnar , Peter Zijlstra , Juri Lelli , Vincent Guittot , Dietmar Eggemann , Steven Rostedt , Ben Segall , Mel Gorman , Valentin Schneider , Tim C Chen , Chen Yu , Christian Loehle , K Prateek Nayak , Barry Song Cc: "Rafael J. Wysocki" , Andrea Righi , Len Brown , ricardo.neri@intel.com, linux-kernel@vger.kernel.org, Vincent Guittot , Ricardo Neri X-Mailer: b4 0.13.0 X-Developer-Signature: v=1; a=ed25519-sha256; t=1782173185; l=3169; i=ricardo.neri-calderon@linux.intel.com; s=20250602; h=from:subject:message-id; bh=xqTTARIjfuZL/TzYafjUsaZtElpWOzKIQUdIG0iqn5Q=; b=A3WWDsiTUPVAdOSZTXen9tZTWxtqQeCQ1gNbLf8Sys+ShuPxOmp1i4NS1JB0WLt71iRCypwjv +lH5z4cGXcoB0hy19Susff4fsRutbC39bXyZ0VnMXW6eUn236YUX3ji X-Developer-Key: i=ricardo.neri-calderon@linux.intel.com; a=ed25519; pk=NfZw5SyQ2lxVfmNMaMR6KUj3+0OhcwDPyRzFDH9gY2w= update_sd_pick_busiest() may incorrectly select a fully_busy group as the busiest group when its per-CPU capacity exceeds that of the destination CPU. This happens because the type of busiest group is initialized to group_has_spare and allows the fully_busy group to win the type comparison. update_sd_pick_busiest() should not choose a candidate scheduling group with at most one runnable task if its per-CPU capacity is greater than that of the destination CPU. Such a check already exists, but it is done too late: after the type comparison, preventing a subsequent fully_busy group of equal per-CPU capacity from being correctly selected. Move this check to occur before comparing group types. Fixes: 0b0695f2b34a ("sched/fair: Rework load_balance()") Reviewed-by: Christian Loehle Reviewed-by: Chen Yu Reviewed-by: Tim Chen Reviewed-by: Vincent Guittot Tested-by: Christian Loehle Signed-off-by: Ricardo Neri Tested-by: Andrea Righi --- Changes in v5: * Added Tested-by tag from Christian. Thanks! Changes in v4: * Dropped note on SMT not being affected since SMT + asym capacity is now supported. * Added Reviewed-by tags from Vincent, Tim, and Chen Yu. Thanks! Changes in v3: * Added a Fixes tag. (Christian) * Added Reviewed-by tag from Christian. Thanks! Changes in v2: * Added a note clarifying that SMT and SD_ASYM_CPUCAPACITY are mutually exclusive. (Tim) * Kept parentheses around bitwise operators for clarity. * Rewrote patch description for clarity. --- kernel/sched/fair.c | 22 +++++++++++----------- 1 file changed, 11 insertions(+), 11 deletions(-) diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index 31baa0000616..030675e249b5 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -11944,6 +11944,17 @@ static bool update_sd_pick_busiest(struct lb_env *= env, sds->local_stat.group_type !=3D group_has_spare)) return false; =20 + /* + * Candidate sg has no more than one task per CPU and has higher + * per-CPU capacity. Migrating tasks to less capable CPUs may harm + * throughput. Maximize throughput, power/energy consequences are not + * considered. + */ + if ((env->sd->flags & SD_ASYM_CPUCAPACITY) && + (sgs->group_type <=3D group_fully_busy) && + (capacity_greater(sg->sgc->min_capacity, capacity_of(env->dst_cpu)))) + return false; + if (sgs->group_type > busiest->group_type) return true; =20 @@ -12050,17 +12061,6 @@ static bool update_sd_pick_busiest(struct lb_env *= env, break; } =20 - /* - * Candidate sg has no more than one task per CPU and has higher - * per-CPU capacity. Migrating tasks to less capable CPUs may harm - * throughput. Maximize throughput, power/energy consequences are not - * considered. - */ - if ((env->sd->flags & SD_ASYM_CPUCAPACITY) && - (sgs->group_type <=3D group_fully_busy) && - (capacity_greater(sg->sgc->min_capacity, capacity_of(env->dst_cpu)))) - return false; - return true; } =20 --=20 2.43.0 From nobody Tue Jul 14 12:48:06 2026 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 99BB636A377 for ; Mon, 22 Jun 2026 23:55:35 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.18 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1782172546; cv=none; b=mxjYCwlEpVafsDN06jDR5CCRcbTLm6FR+YxQY/tTWsvV5x89f/vhxWHqxTGToXijd2Zh8X4vcBQ+OSWOhAFZor1a3iZjFF/0nKPN8aq1EjDRS40/+/3sR9pvZJYoKY8pDUXHyQv6xvCPP66lY3phUcQkNgYrTL/L25jd0Y/HMJA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1782172546; c=relaxed/simple; bh=J4+/+R6VTGfzsFmzqKqyh/tsCycDY1llfY6VbmDL8VI=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=j///IISm6pqnu2jXBRjQIQRkjBCp7LCmns/mi51BH2eA7HaVuYBch6IAOos8e4CkbrcwUfR0+A0VDucKh7V267KwYIrZbA9WheimSmai5IHkbkYv3P+wI5rQz5Dj6KbMpc9HZ1zFc2wT0flOfMmWJL6eTo48EWzF4WKIAwd9k1w= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com; spf=pass smtp.mailfrom=linux.intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=iNozAGBL; arc=none smtp.client-ip=192.198.163.18 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="iNozAGBL" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1782172536; x=1813708536; h=from:date:subject:mime-version:content-transfer-encoding: message-id:references:in-reply-to:to:cc; bh=J4+/+R6VTGfzsFmzqKqyh/tsCycDY1llfY6VbmDL8VI=; b=iNozAGBLfANfQJNX5585ExShUPB3YVmbAn+LE2aCH0C1JvupXKhGEWtU 9AQnk4JfdtywLUrUtfkkONUo2a/2NpIWEKSJh2RAdCTqV6Urf+a8UC3VM ypaKBq5eBEXw7sTmL9xKtdcfxZo5NKfcm4T2zXTJGbsQIK5p3mUid27Jn m6GhfYK85WXkKICrqI4VQDjt5a082IsLBr/3gJRZURT1GUV6LLZZExEbt WcznJicckF/aq9///kregCaajuslJI7m4SFHqyyS8HSeokwZ+ouWSmVdh drBZvTHC+aGmBXN5T6D2ZGbY6Q6JR8Qcr0gjXYS9odFJ+H6G1pU2CNcHq Q==; X-CSE-ConnectionGUID: 2Q07oUGvQCG/VjjyQRpb5g== X-CSE-MsgGUID: aAdQBugPRR2IyUy57fX1BQ== X-IronPort-AV: E=McAfee;i="6800,10657,11825"; a="82014513" X-IronPort-AV: E=Sophos;i="6.24,219,1774335600"; d="scan'208";a="82014513" Received: from fmviesa003.fm.intel.com ([10.60.135.143]) by fmvoesa112.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 22 Jun 2026 16:55:31 -0700 X-CSE-ConnectionGUID: ltpl03PpR/m09H7/9TzoNg== X-CSE-MsgGUID: pGbBQIHWTwq72PXhhuv06A== X-ExtLoop1: 1 Received: from unknown (HELO [172.25.112.21]) ([172.25.112.21]) by fmviesa003.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 22 Jun 2026 16:55:30 -0700 From: Ricardo Neri Date: Mon, 22 Jun 2026 17:05:54 -0700 Subject: [PATCH v5 4/6] sched/fair: Skip misfit load accounting when the destination CPU cannot help Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260622-rneri-fix-cas-clusters-v5-4-19968f2d1497@linux.intel.com> References: <20260622-rneri-fix-cas-clusters-v5-0-19968f2d1497@linux.intel.com> In-Reply-To: <20260622-rneri-fix-cas-clusters-v5-0-19968f2d1497@linux.intel.com> To: Ingo Molnar , Peter Zijlstra , Juri Lelli , Vincent Guittot , Dietmar Eggemann , Steven Rostedt , Ben Segall , Mel Gorman , Valentin Schneider , Tim C Chen , Chen Yu , Christian Loehle , K Prateek Nayak , Barry Song Cc: "Rafael J. Wysocki" , Andrea Righi , Len Brown , ricardo.neri@intel.com, linux-kernel@vger.kernel.org, Ricardo Neri X-Mailer: b4 0.13.0 X-Developer-Signature: v=1; a=ed25519-sha256; t=1782173185; l=3269; i=ricardo.neri-calderon@linux.intel.com; s=20250602; h=from:subject:message-id; bh=J4+/+R6VTGfzsFmzqKqyh/tsCycDY1llfY6VbmDL8VI=; b=2aC3YEOTCYnEEnLsCOUdWqLrUhy8uEpKALBw3NmLwxL0hqDBE6sQ0V+MejeoXfSDXAZuN15Tw SU9au2e8V5MC4OTyiEWiReKMh3DSl0aVybkPbAVnB6Av9W6uatjK+e9 X-Developer-Key: i=ricardo.neri-calderon@linux.intel.com; a=ed25519; pk=NfZw5SyQ2lxVfmNMaMR6KUj3+0OhcwDPyRzFDH9gY2w= In domains with asymmetric capacity, identifying misfit load in a scheduling group is not useful when the destination CPU cannot help (i.e., its capacity exceeds the group's maximum CPU capacity by less than ~5%). In such cases, it also prevents load balance among clusters of equal capacity when CONFIG_SCHED_CLUSTER is enabled. This happens because update_sd_pick_busiest() skips candidate groups of type misfit_task if the destination CPU has similar capacity. Skipping misfit load accounting in this situation allows the group to be classified as has_spare or fully_busy and lets load balancing proceed. Keep marking scheduling groups as overloaded when misfit tasks are present. The sg_overloaded flag propagates to the root domain and allows bigger CPUs in it to help via newly idle balance. Reviewed-by: Christian Loehle Reviewed-by: Chen Yu Reviewed-by: Vincent Guittot Tested-by: Christian Loehle Signed-off-by: Ricardo Neri Tested-by: Andrea Righi --- Changes in v5: * Added Tested-by tag from Christian. Thanks! Changes in v4: * Added Reviewed-by tags from Vincent and Chen Yu. Thanks! Changes in v3: * Added Reviewed-by tag from Christian. Thanks! Changes in v2: * Moved the check of the destination CPU capacity inside the code block used for SD_ASYM_CPUCAPACITY. v1 inadvertently broke the mutual exclusion of the sched_reduced_capacity() path. * Keep marking the root domain as overloaded to allow bigger CPUs to help. (sashiko) * Fixed patch description to clarify that the capacity_greater() looks for differences of 5% or more. (Christian) * Reworded the patch description for clarity. * I did not include the Reviewed-by tag from Christian since the patch changed functionally. --- kernel/sched/fair.c | 21 +++++++++++++++++---- 1 file changed, 17 insertions(+), 4 deletions(-) diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index 030675e249b5..e55eb019d2c9 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -11860,12 +11860,25 @@ static inline void update_sg_lb_stats(struct lb_e= nv *env, continue; =20 if (sd_flags & SD_ASYM_CPUCAPACITY) { - /* Check for a misfit task on the cpu */ - if (sgs->group_misfit_task_load < rq->misfit_task_load) { - sgs->group_misfit_task_load =3D rq->misfit_task_load; - + if (rq->misfit_task_load) { + /* + * Always mark the root domain overloaded so big + * CPUs can pick up misfit tasks via newly idle + * balance. + */ if (balancing_at_rd) *sg_overloaded =3D 1; + + /* + * Only account misfit load if @dst_cpu can + * help; otherwise, the group may be classified + * as misfit_task and update_sd_pick_busiest() + * will skip it. + */ + if (capacity_greater(capacity_of(env->dst_cpu), + group->sgc->max_capacity) && + (sgs->group_misfit_task_load < rq->misfit_task_load)) + sgs->group_misfit_task_load =3D rq->misfit_task_load; } } else if (env->idle && sched_reduced_capacity(rq, env->sd)) { /* Check for a task running on a CPU with reduced capacity */ --=20 2.43.0 From nobody Tue Jul 14 12:48:06 2026 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9969437267A for ; Mon, 22 Jun 2026 23:55:36 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.18 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1782172545; cv=none; b=qKylTeA+6Ta0Be1GfHlGr/H45KF3DdbmYmlZfqNyS5MtQdRPWla9pb7LDgY0CA7yjNM3PM7nt3czLk0MsXzOU/5XxxutII3v6chKserTEzfDDmkF7/MyFutJ2MFsGSKQVmEkVr2mqpNLvJcMBy/Tm7lFfxiiCuMrBIAWXahCZS4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1782172545; c=relaxed/simple; bh=s1L8PZtBTG0G0ZvPFpeJy8t6uKj4ioQP+QTgcaxGoiM=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=Bsevl7wmFKcO/0P812LhldbUZzv4JdmVUZAYQlzoK3ck3eC31jZC/azBYVB1jU2cD3VZ7HQZ4Q6CbSpVKyNdvMKtiq34XF+W8gjCJfP2S4Q6tjoLqq2brnlX2XPsCKZtHjVI/G3L5DI0rzmvxpb6s0p+o3hBhCbA/z5sY5A2gnE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com; spf=pass smtp.mailfrom=linux.intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=NhkKDUJO; arc=none smtp.client-ip=192.198.163.18 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="NhkKDUJO" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1782172537; x=1813708537; h=from:date:subject:mime-version:content-transfer-encoding: message-id:references:in-reply-to:to:cc; bh=s1L8PZtBTG0G0ZvPFpeJy8t6uKj4ioQP+QTgcaxGoiM=; b=NhkKDUJOclNy7/FvHL/gs7NHp7WfRF8k8sPeiucp78oZP9rJO4Bs4XF9 Pi5KsPBMnEN3uFfXVDcBQVgaUSvE64S1F2GyMbor2ZN4k6Ig1l3ivcKsL stmgeWrFEFUpqJzW3HU7YZ/+POyK//Q3bzQ/DgnkxxORcj6/EoR1aXKDp wd1bIHR9QLpPDLZ2qeShO28IHp4cey0rDqpoaEnkJBjpUDCKmziCc55l9 Fb0ShTtxa+DwutH/eC8pKqVIbbZ9l5TqGaILcUlD6NUDnYZ+J7uWf5GcI z+Q6GGXsn70dRAFnxn9s7MV1Szo+ZRdT+cQKtvqryy9rAC2ZnX7G/b5JC g==; X-CSE-ConnectionGUID: bnrYiv6RT3a+PnvXjo2otg== X-CSE-MsgGUID: Ij8if79fQ4e46dcSQtkPFQ== X-IronPort-AV: E=McAfee;i="6800,10657,11825"; a="82014518" X-IronPort-AV: E=Sophos;i="6.24,219,1774335600"; d="scan'208";a="82014518" Received: from fmviesa003.fm.intel.com ([10.60.135.143]) by fmvoesa112.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 22 Jun 2026 16:55:31 -0700 X-CSE-ConnectionGUID: myzI0vfIR5iuDnattPJwbA== X-CSE-MsgGUID: gTohNCbMSl6cqEqgsIUL0A== X-ExtLoop1: 1 Received: from unknown (HELO [172.25.112.21]) ([172.25.112.21]) by fmviesa003.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 22 Jun 2026 16:55:30 -0700 From: Ricardo Neri Date: Mon, 22 Jun 2026 17:05:55 -0700 Subject: [PATCH v5 5/6] sched/fair: Allow load balancing between CPUs of identical capacity Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260622-rneri-fix-cas-clusters-v5-5-19968f2d1497@linux.intel.com> References: <20260622-rneri-fix-cas-clusters-v5-0-19968f2d1497@linux.intel.com> In-Reply-To: <20260622-rneri-fix-cas-clusters-v5-0-19968f2d1497@linux.intel.com> To: Ingo Molnar , Peter Zijlstra , Juri Lelli , Vincent Guittot , Dietmar Eggemann , Steven Rostedt , Ben Segall , Mel Gorman , Valentin Schneider , Tim C Chen , Chen Yu , Christian Loehle , K Prateek Nayak , Barry Song Cc: "Rafael J. Wysocki" , Andrea Righi , Len Brown , ricardo.neri@intel.com, linux-kernel@vger.kernel.org, Ricardo Neri X-Mailer: b4 0.13.0 X-Developer-Signature: v=1; a=ed25519-sha256; t=1782173185; l=2681; i=ricardo.neri-calderon@linux.intel.com; s=20250602; h=from:subject:message-id; bh=s1L8PZtBTG0G0ZvPFpeJy8t6uKj4ioQP+QTgcaxGoiM=; b=wbRS4jz7m9MzVZI8ncsDxP/0MRb/yaGOs6ImYtpI66iDokNqp+z/2jtAlPsyWjWsHsJrC1hGT jPUCDRnhwuwD3h8wRlbmA+M2aGqYGyY3BgbGUC063rLeXyOhGuajCzW X-Developer-Key: i=ricardo.neri-calderon@linux.intel.com; a=ed25519; pk=NfZw5SyQ2lxVfmNMaMR6KUj3+0OhcwDPyRzFDH9gY2w= sched_balance_find_src_rq() avoids selecting a runqueue with a single running task as busiest if doing so results in migrating the task to a CPU with less than ~5% of extra capacity. It also unintentionally prevents migrations between CPUs of identical capacity. When CONFIG_SCHED_CLUSTER is enabled, load should be balanced across clusters of CPUs with the same capacity. Allowing migration between CPUs of identical capacity is necessary to meet this goal. Use arch_scale_cpu_capacity() to reflect architectural capacity, excluding runtime reductions due to side activity or thermal pressure. Guard this check with the sched_cluster_active static key so that systems without cluster topology are unaffected. Tested-by: Christian Loehle Signed-off-by: Ricardo Neri Tested-by: Andrea Righi --- Changes in v5: * Optimized logic to identify same-arch clusters only when needed. * Added Tested-by tag from Christian. Thanks! Changes in v4: * Implemented the check for cluster with a local variable for improved readability. Changes in v3: * Reverted the inverted capacity check; the inverted form incorrectly allows migrations to CPUs of slightly less capacity. * Guarded the check for architectural capacity with the sched_cluster_active static key. Changes in v2: * Used arch_scale_cpu_capacity() instead of capacity_of() to ignore runtime variability. * Inverted the check for runtime capacity. (Christian) * Reworded patch description for clarity. --- kernel/sched/fair.c | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index e55eb019d2c9..f4eb55cad54d 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -12992,13 +12992,20 @@ static struct rq *sched_balance_find_src_rq(struc= t lb_env *env, */ if (env->sd->flags & SD_ASYM_CPUCAPACITY && nr_running =3D=3D 1) { + bool same_arch_cluster =3D static_branch_unlikely(&sched_cluster_active= ) && + (arch_scale_cpu_capacity(env->dst_cpu) =3D=3D + arch_scale_cpu_capacity(i)); bool smt_degraded_cap =3D sched_smt_active() && !is_core_idle(i); =20 /* * Busy SMT siblings reduce the capacity of CPU @i. Do * not skip it in this case. + * + * CONFIG_SCHED_CLUSTER requires balancing load across clusters + * of identical capacity. Use architectural capacity to ignore + * runtime variability. */ - if (!smt_degraded_cap && + if (!smt_degraded_cap && !same_arch_cluster && !capacity_greater(capacity_of(env->dst_cpu), capacity)) continue; } --=20 2.43.0 From nobody Tue Jul 14 12:48:06 2026 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id F3152372ED0 for ; Mon, 22 Jun 2026 23:55:36 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.18 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1782172545; cv=none; b=spr0mnuXZQhU+gETK0TmrgfeKWt4neWldAgqqNaQKnpCg+8wdGspDJjSHEep92uo63sg+UvgyEGUf3XgEGVKRyPsk0zKGpxoK7qtJDzkGX4rkcgifEmgF6AzY0H5SyRVF/L4QxaBoMgeoHgqjGRsdNvtFa3Bm568AjwR07O5m8Y= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1782172545; c=relaxed/simple; bh=x2K39AFJmdRhNQGFrE7+3vKUL0yCoNjw5mmM5ALREMY=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=m3VPCY7Ytm2KMjh3NuL3nfVXkfpfFqzjlEpVtn/9v1pFbSbNMfqt2othBngYcY1cTttOSYShINqWtmSA73o49p22D3mq7tzWybj0dHGqMYqH+aDrjBy46CLlTucG2/crxcUOKYcU+v6acrqT1FYY4/Fe3uV3XQisSl66/zjj4ZI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com; spf=pass smtp.mailfrom=linux.intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=Ig9fwkgG; arc=none smtp.client-ip=192.198.163.18 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="Ig9fwkgG" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1782172537; x=1813708537; h=from:date:subject:mime-version:content-transfer-encoding: message-id:references:in-reply-to:to:cc; bh=x2K39AFJmdRhNQGFrE7+3vKUL0yCoNjw5mmM5ALREMY=; b=Ig9fwkgGHK8KNe4xf7sKtpIt6AKzHsEdCNGCyH6uiApQ9cUc1YkvYh0y YuMlPnLqsClA+b5eugvoLs+RrJT3zNAAJEoVCw+fea/cLtTaDxTGVIfnh tT5s9wmENNuQg591tFtdnzjOwZqTMxvq1IxxyyV7Kd5j7WmwYhHO9Whdz bM9+zm7G1JjwARkKYIVxxfxDxNo66zbwgdlya5DkH71OnWPa1s6asqacI qsGiRA5168U+qL9Y8omNj2wcKKcmqyrVlI8YV9IZrVBfJfMnTFbxxPKtw TmxQtnk9RmPbM49F+gYT2e3Uk7ZTEDWL5Je2PYKttzh8VpN6Zs5T4nW9R A==; X-CSE-ConnectionGUID: mfSQX5nZQGWcr5OB8peFSA== X-CSE-MsgGUID: Syuhni2eQra0RZA7nkw8aw== X-IronPort-AV: E=McAfee;i="6800,10657,11825"; a="82014534" X-IronPort-AV: E=Sophos;i="6.24,219,1774335600"; d="scan'208";a="82014534" Received: from fmviesa003.fm.intel.com ([10.60.135.143]) by fmvoesa112.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 22 Jun 2026 16:55:32 -0700 X-CSE-ConnectionGUID: Rxl4dsmoS9KEg7CIXU1IDw== X-CSE-MsgGUID: 75raZ35DQHG9lLuKX6HreA== X-ExtLoop1: 1 Received: from unknown (HELO [172.25.112.21]) ([172.25.112.21]) by fmviesa003.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 22 Jun 2026 16:55:30 -0700 From: Ricardo Neri Date: Mon, 22 Jun 2026 17:05:56 -0700 Subject: [PATCH v5 6/6] sched/topology: Do not clear SD_PREFER_SIBLING in domains with clusters Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260622-rneri-fix-cas-clusters-v5-6-19968f2d1497@linux.intel.com> References: <20260622-rneri-fix-cas-clusters-v5-0-19968f2d1497@linux.intel.com> In-Reply-To: <20260622-rneri-fix-cas-clusters-v5-0-19968f2d1497@linux.intel.com> To: Ingo Molnar , Peter Zijlstra , Juri Lelli , Vincent Guittot , Dietmar Eggemann , Steven Rostedt , Ben Segall , Mel Gorman , Valentin Schneider , Tim C Chen , Chen Yu , Christian Loehle , K Prateek Nayak , Barry Song Cc: "Rafael J. Wysocki" , Andrea Righi , Len Brown , ricardo.neri@intel.com, linux-kernel@vger.kernel.org, Ricardo Neri X-Mailer: b4 0.13.0 X-Developer-Signature: v=1; a=ed25519-sha256; t=1782173186; l=3739; i=ricardo.neri-calderon@linux.intel.com; s=20250602; h=from:subject:message-id; bh=x2K39AFJmdRhNQGFrE7+3vKUL0yCoNjw5mmM5ALREMY=; b=0nqSXB7aCnrNcT1Eviaxx7hcs1N55rP8BHHvLVZSff1Eb8PILqwflhzBfVpn6WJmmsTC3n+C2 iga/LXSDNCLAcBUePimakCeE7c0kZAzdqWkfBbzrZqGUGYcRPFGEC5o X-Developer-Key: i=ricardo.neri-calderon@linux.intel.com; a=ed25519; pk=NfZw5SyQ2lxVfmNMaMR6KUj3+0OhcwDPyRzFDH9gY2w= Some topologies have scheduling domains that contain CPUs of asymmetric capacity, grouped into two or more clusters of equal-capacity CPUs sharing an L2 cache. When CONFIG_SCHED_CLUSTER is enabled, load must be balanced across these clusters. Do not clear SD_PREFER_SIBLING in the child domains to indicate to the load balancer that it should spread load among cluster siblings. Checks for capacity in update_sd_pick_busiest(), sched_balance_find_src_group(), and sched_balance_find_src_rq() prevent migrations from high- to low-capacity CPUs if the busiest group is not overloaded. CPUs with spare capacity, big or small, have always helped overloaded groups. Once the overloading condition disappears, misfit load will still be used to move high-utilization tasks to bigger CPUs if they have spare capacity. Adding the SD_PREFER_SIBLING flag shifts load balancing in shared-LLC domains from equalizing the number of idle CPUs to equalizing the number of running tasks. This also enables migrations among clusters from newly- idle load balance, where the outgoing task is already dequeued but the CPU has not yet transitioned to idle. Reviewed-by: Tim Chen Tested-by: Christian Loehle Signed-off-by: Ricardo Neri Tested-by: Andrea Righi --- Changes in v5: * Improved inline comments for accuracy. * Added Tested-by tag from Christian. Thanks! Changes in v4: * Added Reviewed-by tag from Tim. Thanks! Changes in v3: * Updated documentation of SD_PREFER_SIBLING. * Expanded the patch description to explain the behavior when overloaded groups are involved. Changes in v2: * Reworded the patch description for clarity. * Kept parentheses around bitwise operators for clarity. --- include/linux/sched/sd_flags.h | 3 ++- kernel/sched/topology.c | 14 ++++++++++++-- 2 files changed, 14 insertions(+), 3 deletions(-) diff --git a/include/linux/sched/sd_flags.h b/include/linux/sched/sd_flags.h index 42839cfa2778..f9a46fb8cacf 100644 --- a/include/linux/sched/sd_flags.h +++ b/include/linux/sched/sd_flags.h @@ -147,7 +147,8 @@ SD_FLAG(SD_ASYM_PACKING, SDF_NEEDS_GROUPS) * Prefer to place tasks in a sibling domain * * Set up until domains start spanning NUMA nodes. Close to being a SHARED= _CHILD - * flag, but cleared below domains with SD_ASYM_CPUCAPACITY. + * flag, but cleared below domains with SD_ASYM_CPUCAPACITY unless those c= hild + * domains have clusters of CPUs sharing cache. * * NEEDS_GROUPS: Load balancing flag. */ diff --git a/kernel/sched/topology.c b/kernel/sched/topology.c index 622e2e01974c..261b407d0936 100644 --- a/kernel/sched/topology.c +++ b/kernel/sched/topology.c @@ -1995,8 +1995,18 @@ sd_init(struct sched_domain_topology_level *tl, /* * Convert topological properties into behaviour. */ - /* Don't attempt to spread across CPUs of different capacities. */ - if ((sd->flags & SD_ASYM_CPUCAPACITY) && sd->child) + /* + * Don't attempt to spread across CPUs of different capacities. + * + * If the child domain has clusters of CPUs sharing L2 cache, keep the + * flag to spread tasks across clusters of identical capacity. Checks in + * the load balancer prevent task migrations from high- to low-capacity + * CPUs unless the source group is overloaded. Migrations to a lower- + * capacity CPU can happen if a higher-capacity group is overloaded and + * a lower-capacity CPU has spare capacity. + */ + if ((sd->flags & SD_ASYM_CPUCAPACITY) && sd->child && + !(sd->child->flags & SD_CLUSTER)) sd->child->flags &=3D ~SD_PREFER_SIBLING; =20 if (sd->flags & SD_SHARE_CPUCAPACITY) { --=20 2.43.0