From nobody Thu Apr 16 00:54:29 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8EF72C433FE for ; Tue, 22 Nov 2022 20:28:29 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234833AbiKVU20 (ORCPT ); Tue, 22 Nov 2022 15:28:26 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37968 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232554AbiKVU2O (ORCPT ); Tue, 22 Nov 2022 15:28:14 -0500 Received: from mga12.intel.com (mga12.intel.com [192.55.52.136]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E11EE1AD83 for ; Tue, 22 Nov 2022 12:28:13 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1669148893; x=1700684893; h=from:to:cc:subject:date:message-id:in-reply-to: references; bh=0yHmKdRsV591tHGMnKRYVefaIOCtc6d0I/pJdG+LlvU=; b=VixAVveTTgBILU537x5lOBygOGbEtNITiTtO12/uMiymItMal26+/PfT EWaE9taAOryiZQ6bDPqaWr3nfsCf0VphmrraexHIxFL8sTO/zJld679p9 g+BtA8xRQgcjsCVwlYe5hV/mBO56OETLCkqBk3XEIvP81yfF7wt5Z7WNG 4B7c3G1gLvUn9oT0eIuM7Ai8jn4qPUVeJoOhz7pXI5HjeiHiRbq6lW7VZ OcLRgnPwvKooEJTPH1hpsI2JW6MGrIgGRQLxgRexjNnc954PMgPISKr+B abdROcQZJWxzIgBvFXP66hfbPPHvYVOS91j28vHgdmyQjzgT8TcHZL+LG Q==; X-IronPort-AV: E=McAfee;i="6500,9779,10539"; a="293616500" X-IronPort-AV: E=Sophos;i="5.96,185,1665471600"; d="scan'208";a="293616500" Received: from orsmga005.jf.intel.com ([10.7.209.41]) by fmsmga106.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 22 Nov 2022 12:28:12 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10539"; a="816228121" X-IronPort-AV: E=Sophos;i="5.96,185,1665471600"; d="scan'208";a="816228121" Received: from ranerica-svr.sc.intel.com ([172.25.110.23]) by orsmga005.jf.intel.com with ESMTP; 22 Nov 2022 12:28:12 -0800 From: Ricardo Neri To: "Peter Zijlstra (Intel)" , Juri Lelli , Vincent Guittot Cc: Ricardo Neri , "Ravi V. Shankar" , Ben Segall , Daniel Bristot de Oliveira , Dietmar Eggemann , Len Brown , Mel Gorman , "Rafael J. Wysocki" , Srinivas Pandruvada , Steven Rostedt , Tim Chen , Valentin Schneider , x86@kernel.org, linux-kernel@vger.kernel.org, Ricardo Neri , "Tim C . Chen" Subject: [PATCH v2 1/7] sched/fair: Generalize asym_packing logic for SMT local sched group Date: Tue, 22 Nov 2022 12:35:26 -0800 Message-Id: <20221122203532.15013-2-ricardo.neri-calderon@linux.intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20221122203532.15013-1-ricardo.neri-calderon@linux.intel.com> References: <20221122203532.15013-1-ricardo.neri-calderon@linux.intel.com> Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" When balancing load between two physical cores, an idle destination CPU can help another core only if all of its SMT siblings are also idle. Otherwise, there is not increase in throughput. It does not matter whether the other core is composed of SMT siblings. Simply check if there are any tasks running on the local group and the other core has exactly one busy CPU before proceeding. Let find_busiest_group() handle the case of more than one busy CPU. This is true for SMT2, SMT4, SMT8, etc. Cc: Ben Segall Cc: Daniel Bristot de Oliveira Cc: Dietmar Eggemann Cc: Len Brown Cc: Mel Gorman Cc: Rafael J. Wysocki Cc: Srinivas Pandruvada Cc: Steven Rostedt Cc: Tim C. Chen Cc: Valentin Schneider Cc: x86@kernel.org Cc: linux-kernel@vger.kernel.org Reviewed-by: Len Brown Signed-off-by: Ricardo Neri --- Changes since v1: * Reworded commit message and inline comments for clarity. * Stated that this changeset does not impact STM4 or SMT8. --- kernel/sched/fair.c | 29 +++++++++-------------------- 1 file changed, 9 insertions(+), 20 deletions(-) diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index e4a0b8bd941c..18c672ff39ef 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -8900,12 +8900,10 @@ static bool asym_smt_can_pull_tasks(int dst_cpu, st= ruct sd_lb_stats *sds, struct sched_group *sg) { #ifdef CONFIG_SCHED_SMT - bool local_is_smt, sg_is_smt; + bool local_is_smt; int sg_busy_cpus; =20 local_is_smt =3D sds->local->flags & SD_SHARE_CPUCAPACITY; - sg_is_smt =3D sg->flags & SD_SHARE_CPUCAPACITY; - sg_busy_cpus =3D sgs->group_weight - sgs->idle_cpus; =20 if (!local_is_smt) { @@ -8926,25 +8924,16 @@ static bool asym_smt_can_pull_tasks(int dst_cpu, st= ruct sd_lb_stats *sds, return sched_asym_prefer(dst_cpu, sg->asym_prefer_cpu); } =20 - /* @dst_cpu has SMT siblings. */ - - if (sg_is_smt) { - int local_busy_cpus =3D sds->local->group_weight - - sds->local_stat.idle_cpus; - int busy_cpus_delta =3D sg_busy_cpus - local_busy_cpus; - - if (busy_cpus_delta =3D=3D 1) - return sched_asym_prefer(dst_cpu, sg->asym_prefer_cpu); - - return false; - } - /* - * @sg does not have SMT siblings. Ensure that @sds::local does not end - * up with more than one busy SMT sibling and only pull tasks if there - * are not busy CPUs (i.e., no CPU has running tasks). + * @dst_cpu has SMT siblings. Do asym_packing load balancing only if + * all its siblings are idle (moving tasks between physical cores in + * which some SMT siblings are busy results in the same throughput). + * + * If the difference in the number of busy CPUs is two or more, let + * find_busiest_group() take care of it. We only care if @sg has + * exactly one busy CPU. This covers SMT and non-SMT sched groups. */ - if (!sds->local_stat.sum_nr_running) + if (sg_busy_cpus =3D=3D 1 && !sds->local_stat.sum_nr_running) return sched_asym_prefer(dst_cpu, sg->asym_prefer_cpu); =20 return false; --=20 2.25.1 From nobody Thu Apr 16 00:54:29 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 565F9C433FE for ; Tue, 22 Nov 2022 20:28:37 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234869AbiKVU2f (ORCPT ); Tue, 22 Nov 2022 15:28:35 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37986 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234490AbiKVU2O (ORCPT ); Tue, 22 Nov 2022 15:28:14 -0500 Received: from mga12.intel.com (mga12.intel.com [192.55.52.136]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1FBB6165A6 for ; Tue, 22 Nov 2022 12:28:14 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1669148894; x=1700684894; h=from:to:cc:subject:date:message-id:in-reply-to: references; bh=y/1MAtHpQYdgN3xn2hBOyK3a06TFyZ6nWBTd2NULu+4=; b=MxSPRBvh003O+2p68QIC1ZbnVoGu99y4zO9Yy24dMEH3kuTwg1Icre7+ uDKuvzxf3eCuDeyF9YK1i07kq+IddE+20oF8s7oJPrV7CCYBOzeKkWgQj FMqOqvYzzyOF/3GUulFs+iBLkZskZKp2GnSDeEslNnZ2TG6OSV/A18wku K6PFVnT+sBUZeZCwwjo1yUdvuIwZzxpNKSXXsy/iB/ufuEKPoecm48iDm 3F7/PFXNPKveqHQacpxV0IeK2bzGpzo9CeCcWW8rJ/k8VXzVij18OvEZ+ D831iiw0WsFamIkkW66TfCIuW0FOPLsrh9V5XmEG6WJHw/SRt+zleXnyn w==; X-IronPort-AV: E=McAfee;i="6500,9779,10539"; a="293616506" X-IronPort-AV: E=Sophos;i="5.96,185,1665471600"; d="scan'208";a="293616506" Received: from orsmga005.jf.intel.com ([10.7.209.41]) by fmsmga106.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 22 Nov 2022 12:28:13 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10539"; a="816228127" X-IronPort-AV: E=Sophos;i="5.96,185,1665471600"; d="scan'208";a="816228127" Received: from ranerica-svr.sc.intel.com ([172.25.110.23]) by orsmga005.jf.intel.com with ESMTP; 22 Nov 2022 12:28:12 -0800 From: Ricardo Neri To: "Peter Zijlstra (Intel)" , Juri Lelli , Vincent Guittot Cc: Ricardo Neri , "Ravi V. Shankar" , Ben Segall , Daniel Bristot de Oliveira , Dietmar Eggemann , Len Brown , Mel Gorman , "Rafael J. Wysocki" , Srinivas Pandruvada , Steven Rostedt , Tim Chen , Valentin Schneider , x86@kernel.org, linux-kernel@vger.kernel.org, Ricardo Neri , "Tim C . Chen" Subject: [PATCH v2 2/7] sched: Prepare sched_asym_prefer() to handle idle state of SMT siblings Date: Tue, 22 Nov 2022 12:35:27 -0800 Message-Id: <20221122203532.15013-3-ricardo.neri-calderon@linux.intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20221122203532.15013-1-ricardo.neri-calderon@linux.intel.com> References: <20221122203532.15013-1-ricardo.neri-calderon@linux.intel.com> Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" The throughput of an SMT sibling decreases if one or more of its siblings are also busy. Idle, lower-priority cores can help. Thus, it is necessary to consider the idle state of the SMT siblings of CPUs when selecting by priority. In some cases, sched_asym_prefer() does not care about the idle state (when building sched domains or looking at the priority of the preferred CPU in a sched group). Add a new parameter to check the state of the SMT siblings of a CPU when applicable. While here, remove a spurious newline. Cc: Ben Segall Cc: Daniel Bristot de Oliveira Cc: Dietmar Eggemann Cc: Len Brown Cc: Mel Gorman Cc: Rafael J. Wysocki Cc: Srinivas Pandruvada Cc: Steven Rostedt Cc: Tim C. Chen Cc: Valentin Schneider Cc: x86@kernel.org Cc: linux-kernel@vger.kernel.org Signed-off-by: Ricardo Neri --- Changes since v1: * Introduced this patch --- kernel/sched/fair.c | 17 ++++++++++------- kernel/sched/sched.h | 8 ++++++-- kernel/sched/topology.c | 6 +++++- 3 files changed, 21 insertions(+), 10 deletions(-) diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index 18c672ff39ef..d18947a9c03e 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -8921,7 +8921,7 @@ static bool asym_smt_can_pull_tasks(int dst_cpu, stru= ct sd_lb_stats *sds, * can help if it has higher priority and is idle (i.e., * it has no running tasks). */ - return sched_asym_prefer(dst_cpu, sg->asym_prefer_cpu); + return sched_asym_prefer(dst_cpu, sg->asym_prefer_cpu, false); } =20 /* @@ -8934,7 +8934,7 @@ static bool asym_smt_can_pull_tasks(int dst_cpu, stru= ct sd_lb_stats *sds, * exactly one busy CPU. This covers SMT and non-SMT sched groups. */ if (sg_busy_cpus =3D=3D 1 && !sds->local_stat.sum_nr_running) - return sched_asym_prefer(dst_cpu, sg->asym_prefer_cpu); + return sched_asym_prefer(dst_cpu, sg->asym_prefer_cpu, false); =20 return false; #else @@ -8952,7 +8952,8 @@ sched_asym(struct lb_env *env, struct sd_lb_stats *sd= s, struct sg_lb_stats *sgs (group->flags & SD_SHARE_CPUCAPACITY)) return asym_smt_can_pull_tasks(env->dst_cpu, sds, sgs, group); =20 - return sched_asym_prefer(env->dst_cpu, group->asym_prefer_cpu); + /* Neither env::dst_cpu nor group::asym_prefer_cpu have SMT siblings. */ + return sched_asym_prefer(env->dst_cpu, group->asym_prefer_cpu, false); } =20 static inline bool @@ -9118,7 +9119,9 @@ static bool update_sd_pick_busiest(struct lb_env *env, =20 case group_asym_packing: /* Prefer to move from lowest priority CPU's work */ - if (sched_asym_prefer(sg->asym_prefer_cpu, sds->busiest->asym_prefer_cpu= )) + if (sched_asym_prefer(sg->asym_prefer_cpu, + sds->busiest->asym_prefer_cpu, + false)) return false; break; =20 @@ -10060,7 +10063,7 @@ static struct rq *find_busiest_queue(struct lb_env = *env, =20 /* Make sure we only pull tasks from a CPU of lower priority */ if ((env->sd->flags & SD_ASYM_PACKING) && - sched_asym_prefer(i, env->dst_cpu) && + sched_asym_prefer(i, env->dst_cpu, true) && nr_running =3D=3D 1) continue; =20 @@ -10153,7 +10156,7 @@ asym_active_balance(struct lb_env *env) * highest priority CPUs. */ return env->idle !=3D CPU_NOT_IDLE && (env->sd->flags & SD_ASYM_PACKING) = && - sched_asym_prefer(env->dst_cpu, env->src_cpu); + sched_asym_prefer(env->dst_cpu, env->src_cpu, true); } =20 static inline bool @@ -10889,7 +10892,7 @@ static void nohz_balancer_kick(struct rq *rq) * around. */ for_each_cpu_and(i, sched_domain_span(sd), nohz.idle_cpus_mask) { - if (sched_asym_prefer(i, cpu)) { + if (sched_asym_prefer(i, cpu, true)) { flags =3D NOHZ_STATS_KICK | NOHZ_BALANCE_KICK; goto unlock; } diff --git a/kernel/sched/sched.h b/kernel/sched/sched.h index a4a20046e586..0fc7c0130755 100644 --- a/kernel/sched/sched.h +++ b/kernel/sched/sched.h @@ -795,8 +795,12 @@ static inline long se_weight(struct sched_entity *se) return scale_load_down(se->load.weight); } =20 - -static inline bool sched_asym_prefer(int a, int b) +/* + * Used to compare specific CPUs. Also when comparing the preferred CPU of= a + * sched group or building the sched domains; in such cases checking the s= tate + * of SMT siblings, if any, is not needed. + */ +static inline bool sched_asym_prefer(int a, int b, bool check_smt) { return arch_asym_cpu_priority(a) > arch_asym_cpu_priority(b); } diff --git a/kernel/sched/topology.c b/kernel/sched/topology.c index 8739c2a5a54e..8154ef590b9f 100644 --- a/kernel/sched/topology.c +++ b/kernel/sched/topology.c @@ -1282,7 +1282,11 @@ static void init_sched_groups_capacity(int cpu, stru= ct sched_domain *sd) for_each_cpu(cpu, sched_group_span(sg)) { if (max_cpu < 0) max_cpu =3D cpu; - else if (sched_asym_prefer(cpu, max_cpu)) + /* + * We want the CPU priorities unaffected by the idle + * state of its SMT siblings, if any. + */ + else if (sched_asym_prefer(cpu, max_cpu, false)) max_cpu =3D cpu; } sg->asym_prefer_cpu =3D max_cpu; --=20 2.25.1 From nobody Thu Apr 16 00:54:29 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 715E9C433FE for ; Tue, 22 Nov 2022 20:28:33 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234855AbiKVU2b (ORCPT ); Tue, 22 Nov 2022 15:28:31 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37752 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234457AbiKVU2P (ORCPT ); Tue, 22 Nov 2022 15:28:15 -0500 Received: from mga12.intel.com (mga12.intel.com [192.55.52.136]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A3B1F1ADB9 for ; Tue, 22 Nov 2022 12:28:14 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1669148894; x=1700684894; h=from:to:cc:subject:date:message-id:in-reply-to: references; bh=Tu8yb4rTFtiG5/ROXb+qs9Ym1yY05u5ArJRxVYfGmFo=; b=YptDn/a3/nVFOVtzeQfSQeV98c6tT1NwvCbizeZq55KsOlAwv4ge4FUe 8aIQAfVMfwBhZo3q+3Kh+36152bdNfkn2358g6SNsOcKl0wFhO0MzZGS5 qFrKDTXl5sg4/VMENHGAbzwUkLOT/gYLSP2rI2etx8Swn9cFru1tBsoMC zqD6sAD2A7EdQyhKrAdJCGqjzX0DyPQNG1Ak5v700hb6Y7V/RJb0deiGZ EZQiAKhcVC9NscnX/CG3ZuQudCv4pcAukhWC3ya5FxmaHKfIWcMVXWdNj IU8PDE03v+VaojhDTTG908ULMkBDBPBDmlVPjY+6mo8aF6kMTARamaUYn Q==; X-IronPort-AV: E=McAfee;i="6500,9779,10539"; a="293616507" X-IronPort-AV: E=Sophos;i="5.96,185,1665471600"; d="scan'208";a="293616507" Received: from orsmga005.jf.intel.com ([10.7.209.41]) by fmsmga106.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 22 Nov 2022 12:28:14 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10539"; a="816228132" X-IronPort-AV: E=Sophos;i="5.96,185,1665471600"; d="scan'208";a="816228132" Received: from ranerica-svr.sc.intel.com ([172.25.110.23]) by orsmga005.jf.intel.com with ESMTP; 22 Nov 2022 12:28:13 -0800 From: Ricardo Neri To: "Peter Zijlstra (Intel)" , Juri Lelli , Vincent Guittot Cc: Ricardo Neri , "Ravi V. Shankar" , Ben Segall , Daniel Bristot de Oliveira , Dietmar Eggemann , Len Brown , Mel Gorman , "Rafael J. Wysocki" , Srinivas Pandruvada , Steven Rostedt , Tim Chen , Valentin Schneider , x86@kernel.org, linux-kernel@vger.kernel.org, Ricardo Neri , "Tim C . Chen" Subject: [PATCH v2 3/7] sched: Teach arch_asym_cpu_priority() the idle state of SMT siblings Date: Tue, 22 Nov 2022 12:35:28 -0800 Message-Id: <20221122203532.15013-4-ricardo.neri-calderon@linux.intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20221122203532.15013-1-ricardo.neri-calderon@linux.intel.com> References: <20221122203532.15013-1-ricardo.neri-calderon@linux.intel.com> Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Some processors (e.g., Intel processors with ITMT) use asym_packing to balance load between physical cores with SMT. In such case, a core with all its SMT siblings idle is more desirable than another with one or more busy siblings. Other processors (e.g, Power7 with SMT8) use asym_packing to balance load among SMT siblings of different priority, regardless of their idle state. Add a new parameter, check_smt, that architectures can use as needed. Cc: Ben Segall Cc: Daniel Bristot de Oliveira Cc: Dietmar Eggemann Cc: Len Brown Cc: Mel Gorman Cc: Rafael J. Wysocki Cc: Srinivas Pandruvada Cc: Steven Rostedt Cc: Tim C. Chen Cc: Valentin Schneider Cc: x86@kernel.org Cc: linux-kernel@vger.kernel.org Suggested-by: Peter Zijlstra (Intel) Signed-off-by: Ricardo Neri --- Changes since v1: * Introduced this patch. --- arch/x86/kernel/itmt.c | 2 +- include/linux/sched/topology.h | 2 +- kernel/sched/fair.c | 5 ++++- kernel/sched/sched.h | 3 ++- 4 files changed, 8 insertions(+), 4 deletions(-) diff --git a/arch/x86/kernel/itmt.c b/arch/x86/kernel/itmt.c index 9ff480e94511..4cb5a5e4fa47 100644 --- a/arch/x86/kernel/itmt.c +++ b/arch/x86/kernel/itmt.c @@ -167,7 +167,7 @@ void sched_clear_itmt_support(void) mutex_unlock(&itmt_update_mutex); } =20 -int arch_asym_cpu_priority(int cpu) +int arch_asym_cpu_priority(int cpu, bool check_smt) { return per_cpu(sched_core_priority, cpu); } diff --git a/include/linux/sched/topology.h b/include/linux/sched/topology.h index 816df6cc444e..87b64b9776f6 100644 --- a/include/linux/sched/topology.h +++ b/include/linux/sched/topology.h @@ -63,7 +63,7 @@ static inline int cpu_numa_flags(void) } #endif =20 -extern int arch_asym_cpu_priority(int cpu); +extern int arch_asym_cpu_priority(int cpu, bool check_smt); =20 struct sched_domain_attr { int relax_domain_level; diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index d18947a9c03e..0e4251f83807 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -142,8 +142,11 @@ __setup("sched_thermal_decay_shift=3D", setup_sched_th= ermal_decay_shift); #ifdef CONFIG_SMP /* * For asym packing, by default the lower numbered CPU has higher priority. + * + * When doing ASYM_PACKING at the "MC" or higher domains, architectures may + * want to check the idle state of the SMT siblngs of @cpu. */ -int __weak arch_asym_cpu_priority(int cpu) +int __weak arch_asym_cpu_priority(int cpu, bool check_smt) { return -cpu; } diff --git a/kernel/sched/sched.h b/kernel/sched/sched.h index 0fc7c0130755..e5e52c2e82de 100644 --- a/kernel/sched/sched.h +++ b/kernel/sched/sched.h @@ -802,7 +802,8 @@ static inline long se_weight(struct sched_entity *se) */ static inline bool sched_asym_prefer(int a, int b, bool check_smt) { - return arch_asym_cpu_priority(a) > arch_asym_cpu_priority(b); + return arch_asym_cpu_priority(a, check_smt) > + arch_asym_cpu_priority(b, check_smt); } =20 struct perf_domain { --=20 2.25.1 From nobody Thu Apr 16 00:54:29 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 217A1C43217 for ; Tue, 22 Nov 2022 20:28:40 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234885AbiKVU2h (ORCPT ); Tue, 22 Nov 2022 15:28:37 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37756 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234402AbiKVU2P (ORCPT ); Tue, 22 Nov 2022 15:28:15 -0500 Received: from mga12.intel.com (mga12.intel.com [192.55.52.136]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id DC03C1B1D8 for ; Tue, 22 Nov 2022 12:28:14 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1669148894; x=1700684894; h=from:to:cc:subject:date:message-id:in-reply-to: references; bh=ZxiRvtsJwz9dFNPg5MHP4zjCzO9YSRx6bjDINyhvJ5o=; b=YO5abYmcCWkxOVmwE0RoQ8gdQ02Gl9vS8d2LzhQcaNrhIaclRTjZDDva w5ZimWTsceDPCaNUq56YuyLrJ8tzVeStPDv6caiGQTFbfd2pNFRc6Zy2D reQHuqADcqYYODzbEyaRuUVGxwJiw7DSV7WqyPBLunxBEVkDHmPrcLA1I UUY3hn5dNJFqTXvHa/SUhP3KGvaJaV4b1fi2wwDzyIm8bhFTnUresYOk5 j2whgaklJBIUugC1QQ6HyQ9yf+en30WsRKYO3LyW8ReX5RPzsZGvzdgp0 5SJWvlE6+5jr2zf4Ay+e+/CIo4vLXHms8hlce/MztAlwoGMTNeqowluFK g==; X-IronPort-AV: E=McAfee;i="6500,9779,10539"; a="293616509" X-IronPort-AV: E=Sophos;i="5.96,185,1665471600"; d="scan'208";a="293616509" Received: from orsmga005.jf.intel.com ([10.7.209.41]) by fmsmga106.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 22 Nov 2022 12:28:14 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10539"; a="816228136" X-IronPort-AV: E=Sophos;i="5.96,185,1665471600"; d="scan'208";a="816228136" Received: from ranerica-svr.sc.intel.com ([172.25.110.23]) by orsmga005.jf.intel.com with ESMTP; 22 Nov 2022 12:28:14 -0800 From: Ricardo Neri To: "Peter Zijlstra (Intel)" , Juri Lelli , Vincent Guittot Cc: Ricardo Neri , "Ravi V. Shankar" , Ben Segall , Daniel Bristot de Oliveira , Dietmar Eggemann , Len Brown , Mel Gorman , "Rafael J. Wysocki" , Srinivas Pandruvada , Steven Rostedt , Tim Chen , Valentin Schneider , x86@kernel.org, linux-kernel@vger.kernel.org, Ricardo Neri , "Tim C . Chen" Subject: [PATCH v2 4/7] sched/fair: Introduce sched_smt_siblings_idle() Date: Tue, 22 Nov 2022 12:35:29 -0800 Message-Id: <20221122203532.15013-5-ricardo.neri-calderon@linux.intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20221122203532.15013-1-ricardo.neri-calderon@linux.intel.com> References: <20221122203532.15013-1-ricardo.neri-calderon@linux.intel.com> Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Architectures that implement arch_asym_cpu_priority() may need to know the idle state of the SMT siblings of a CPU. The scheduler has this information and functionality. Expose it. Move the existing functionality outside of the NUMA code. Cc: Ben Segall Cc: Daniel Bristot de Oliveira Cc: Dietmar Eggemann Cc: Len Brown Cc: Mel Gorman Cc: Rafael J. Wysocki Cc: Srinivas Pandruvada Cc: Steven Rostedt Cc: Tim C. Chen Cc: Valentin Schneider Cc: x86@kernel.org Cc: linux-kernel@vger.kernel.org Signed-off-by: Ricardo Neri --- Changes since v1: * Introduced this patch. --- include/linux/sched.h | 2 ++ kernel/sched/fair.c | 39 ++++++++++++++++++++++----------------- 2 files changed, 24 insertions(+), 17 deletions(-) diff --git a/include/linux/sched.h b/include/linux/sched.h index ffb6eb55cd13..0d01c64ac737 100644 --- a/include/linux/sched.h +++ b/include/linux/sched.h @@ -2426,4 +2426,6 @@ static inline void sched_core_fork(struct task_struct= *p) { } =20 extern void sched_set_stop_task(int cpu, struct task_struct *stop); =20 +extern bool sched_smt_siblings_idle(int cpu); + #endif diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index 0e4251f83807..9517c48df50e 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -1052,6 +1052,28 @@ update_stats_curr_start(struct cfs_rq *cfs_rq, struc= t sched_entity *se) * Scheduling class queueing methods: */ =20 +static inline bool is_core_idle(int cpu) +{ +#ifdef CONFIG_SCHED_SMT + int sibling; + + for_each_cpu(sibling, cpu_smt_mask(cpu)) { + if (cpu =3D=3D sibling) + continue; + + if (!idle_cpu(sibling)) + return false; + } +#endif + + return true; +} + +bool sched_smt_siblings_idle(int cpu) +{ + return is_core_idle(cpu); +} + #ifdef CONFIG_NUMA #define NUMA_IMBALANCE_MIN 2 =20 @@ -1691,23 +1713,6 @@ struct numa_stats { int idle_cpu; }; =20 -static inline bool is_core_idle(int cpu) -{ -#ifdef CONFIG_SCHED_SMT - int sibling; - - for_each_cpu(sibling, cpu_smt_mask(cpu)) { - if (cpu =3D=3D sibling) - continue; - - if (!idle_cpu(sibling)) - return false; - } -#endif - - return true; -} - struct task_numa_env { struct task_struct *p; =20 --=20 2.25.1 From nobody Thu Apr 16 00:54:29 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2AC9FC4332F for ; Tue, 22 Nov 2022 20:28:44 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234895AbiKVU2n (ORCPT ); Tue, 22 Nov 2022 15:28:43 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37766 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234470AbiKVU2P (ORCPT ); Tue, 22 Nov 2022 15:28:15 -0500 Received: from mga12.intel.com (mga12.intel.com [192.55.52.136]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5576A17055 for ; Tue, 22 Nov 2022 12:28:15 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1669148895; x=1700684895; h=from:to:cc:subject:date:message-id:in-reply-to: references; bh=vXIaD/zsIWPHn5n3olE8jagfMylwb+X/0KOXnCS5E6Q=; b=ezwAMMGqxlaicR1khS+T/6eMe09a2oG4+ubhca8mbuxeAjebbBBQvccE yzAEvaI1egytNfUP8SgWJBb7VEL8ioI+Tz1KLvCUnlhHstuikCyyrzj7P ziiLnTyZ2qqndpeHd7fW90As0qVFZKL9ayyWp6s4rnQfvVsYS1GPU1ScP tMKKJds2SCMB58oLW3toHRRzpxH9YwcTbI2jGCtxXd+kKO5ZxvH/NwzdI NZgL9e+EEUdUUislcc9xH6c+EExe3QlcVv07D3Lz3K2YFBGJ0WonXyCXs 8Neit6x8iKNal3IW2uBcWWK2VMF6J5l0k5wIZVsj/BwshL8RHETA7PZA1 g==; X-IronPort-AV: E=McAfee;i="6500,9779,10539"; a="293616511" X-IronPort-AV: E=Sophos;i="5.96,185,1665471600"; d="scan'208";a="293616511" Received: from orsmga005.jf.intel.com ([10.7.209.41]) by fmsmga106.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 22 Nov 2022 12:28:15 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10539"; a="816228141" X-IronPort-AV: E=Sophos;i="5.96,185,1665471600"; d="scan'208";a="816228141" Received: from ranerica-svr.sc.intel.com ([172.25.110.23]) by orsmga005.jf.intel.com with ESMTP; 22 Nov 2022 12:28:14 -0800 From: Ricardo Neri To: "Peter Zijlstra (Intel)" , Juri Lelli , Vincent Guittot Cc: Ricardo Neri , "Ravi V. Shankar" , Ben Segall , Daniel Bristot de Oliveira , Dietmar Eggemann , Len Brown , Mel Gorman , "Rafael J. Wysocki" , Srinivas Pandruvada , Steven Rostedt , Tim Chen , Valentin Schneider , x86@kernel.org, linux-kernel@vger.kernel.org, Ricardo Neri , "Tim C . Chen" Subject: [PATCH v2 5/7] x86/sched: Remove SD_ASYM_PACKING from the "SMT" domain Date: Tue, 22 Nov 2022 12:35:30 -0800 Message-Id: <20221122203532.15013-6-ricardo.neri-calderon@linux.intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20221122203532.15013-1-ricardo.neri-calderon@linux.intel.com> References: <20221122203532.15013-1-ricardo.neri-calderon@linux.intel.com> Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" There is no difference between any of the SMT siblings of a physical core. asym_packing load balancing is not needed among siblings. When balancing load among physical cores, the scheduler now considers the state of the siblings when checking the priority of a CPU. Cc: Ben Segall Cc: Daniel Bristot de Oliveira Cc: Dietmar Eggemann Cc: Len Brown Cc: Mel Gorman Cc: Rafael J. Wysocki Cc: Srinivas Pandruvada Cc: Steven Rostedt Cc: Tim C. Chen Cc: Valentin Schneider Cc: x86@kernel.org Cc: linux-kernel@vger.kernel.org Signed-off-by: Ricardo Neri --- Changes since v1: * Introduced this patch. --- arch/x86/kernel/smpboot.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/arch/x86/kernel/smpboot.c b/arch/x86/kernel/smpboot.c index 3f3ea0287f69..c3de98224cb4 100644 --- a/arch/x86/kernel/smpboot.c +++ b/arch/x86/kernel/smpboot.c @@ -545,7 +545,7 @@ static int x86_core_flags(void) #ifdef CONFIG_SCHED_SMT static int x86_smt_flags(void) { - return cpu_smt_flags() | x86_sched_itmt_flags(); + return cpu_smt_flags(); } #endif #ifdef CONFIG_SCHED_CLUSTER --=20 2.25.1 From nobody Thu Apr 16 00:54:29 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1C566C4332F for ; Tue, 22 Nov 2022 20:28:47 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234904AbiKVU2p (ORCPT ); Tue, 22 Nov 2022 15:28:45 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37764 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234528AbiKVU2Q (ORCPT ); Tue, 22 Nov 2022 15:28:16 -0500 Received: from mga12.intel.com (mga12.intel.com [192.55.52.136]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id DDEBE14D07 for ; Tue, 22 Nov 2022 12:28:15 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1669148895; x=1700684895; h=from:to:cc:subject:date:message-id:in-reply-to: references; bh=bhiwDeK4ICYY9tIKImH8FX+AqBh+ctstEwbC4wBb3kI=; b=UsM+efA2znobHYPGHWvHjZfJetf7YQSNUD2EqNJq44+DhN4OJsRSU38t q7u0pB+hGpumF4AJMOtyMZoB5Nm0FhTY2GaVyNaod7DK1RI7bLcWk1n06 nFpRgTJTEIUi2jzsS3X8tL6t1MWWzCVx3Ol5ujOGZTINcWU/YAPaj79dy irEktVOVkX2qEyAbRV8G+f+dgVIsosVSQ7kNqg7h1/cyBduRPJKzXJLuY PD/PFfdIivrjP0tXRoP/WLfFskbBfTAIUmHsLeNv74GVQAB7IlMwt3DMg dyWLRouVx+RVBAEI8rqCLvTCZiOdduA8oGUr6A/bX3H/UkyUZRjLQLsra w==; X-IronPort-AV: E=McAfee;i="6500,9779,10539"; a="293616514" X-IronPort-AV: E=Sophos;i="5.96,185,1665471600"; d="scan'208";a="293616514" Received: from orsmga005.jf.intel.com ([10.7.209.41]) by fmsmga106.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 22 Nov 2022 12:28:15 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10539"; a="816228144" X-IronPort-AV: E=Sophos;i="5.96,185,1665471600"; d="scan'208";a="816228144" Received: from ranerica-svr.sc.intel.com ([172.25.110.23]) by orsmga005.jf.intel.com with ESMTP; 22 Nov 2022 12:28:15 -0800 From: Ricardo Neri To: "Peter Zijlstra (Intel)" , Juri Lelli , Vincent Guittot Cc: Ricardo Neri , "Ravi V. Shankar" , Ben Segall , Daniel Bristot de Oliveira , Dietmar Eggemann , Len Brown , Mel Gorman , "Rafael J. Wysocki" , Srinivas Pandruvada , Steven Rostedt , Tim Chen , Valentin Schneider , x86@kernel.org, linux-kernel@vger.kernel.org, Ricardo Neri , "Tim C . Chen" Subject: [PATCH v2 6/7] x86/sched/itmt: Give all SMT siblings of a core the same priority Date: Tue, 22 Nov 2022 12:35:31 -0800 Message-Id: <20221122203532.15013-7-ricardo.neri-calderon@linux.intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20221122203532.15013-1-ricardo.neri-calderon@linux.intel.com> References: <20221122203532.15013-1-ricardo.neri-calderon@linux.intel.com> Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Currently, each SMT sibling is given a priority that ensures that in partially busy systems load is evenly spread among cores. There is, however, no difference in performance among the SMT siblings of a core. Having different priorities for each SMT sibling triggers unnecessary load balancing towards the higher-priority sibling. Cc: Ben Segall Cc: Daniel Bristot de Oliveira Cc: Dietmar Eggemann Cc: Len Brown Cc: Mel Gorman Cc: Rafael J. Wysocki Cc: Srinivas Pandruvada Cc: Steven Rostedt Cc: Tim C. Chen Cc: Valentin Schneider Cc: x86@kernel.org Cc: linux-kernel@vger.kernel.org Reviewed-by: Len Brown Signed-off-by: Ricardo Neri --- Changes since v1: * Reworded commit message for clarity. --- arch/x86/kernel/itmt.c | 23 +++++------------------ 1 file changed, 5 insertions(+), 18 deletions(-) diff --git a/arch/x86/kernel/itmt.c b/arch/x86/kernel/itmt.c index 4cb5a5e4fa47..2be49ce4f94a 100644 --- a/arch/x86/kernel/itmt.c +++ b/arch/x86/kernel/itmt.c @@ -174,32 +174,19 @@ int arch_asym_cpu_priority(int cpu, bool check_smt) =20 /** * sched_set_itmt_core_prio() - Set CPU priority based on ITMT - * @prio: Priority of cpu core - * @core_cpu: The cpu number associated with the core + * @prio: Priority of @cpu + * @cpu: The CPU number * * The pstate driver will find out the max boost frequency * and call this function to set a priority proportional - * to the max boost frequency. CPU with higher boost + * to the max boost frequency. CPUs with higher boost * frequency will receive higher priority. * * No need to rebuild sched domain after updating * the CPU priorities. The sched domains have no * dependency on CPU priorities. */ -void sched_set_itmt_core_prio(int prio, int core_cpu) +void sched_set_itmt_core_prio(int prio, int cpu) { - int cpu, i =3D 1; - - for_each_cpu(cpu, topology_sibling_cpumask(core_cpu)) { - int smt_prio; - - /* - * Ensure that the siblings are moved to the end - * of the priority chain and only used when - * all other high priority cpus are out of capacity. - */ - smt_prio =3D prio * smp_num_siblings / (i * i); - per_cpu(sched_core_priority, cpu) =3D smt_prio; - i++; - } + per_cpu(sched_core_priority, cpu) =3D prio; } --=20 2.25.1 From nobody Thu Apr 16 00:54:29 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id A2D0BC433FE for ; Tue, 22 Nov 2022 20:28:51 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234909AbiKVU2u (ORCPT ); Tue, 22 Nov 2022 15:28:50 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37790 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234546AbiKVU2R (ORCPT ); Tue, 22 Nov 2022 15:28:17 -0500 Received: from mga12.intel.com (mga12.intel.com [192.55.52.136]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 8DAD515805 for ; Tue, 22 Nov 2022 12:28:16 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1669148896; x=1700684896; h=from:to:cc:subject:date:message-id:in-reply-to: references; bh=jiYnvM57QdQau/H42shFzT9M5SGTODsNBjdPxNUJhmI=; b=NKkxiZ5Fji9Mkd6TVOXWtDrKNQtE8xJFI7uZ6aSxJFzMgHhILdr0ssmV r5ep01bB53BDuAEVNBvek40Lsgc/1t53rkw4UyJay949WgIc9kRau3h2/ yAPe2fA3H8F1eNzGMTb3v6q0i/PtPLYeqhX3EKw0azbWMt9lakzI2JFP+ 0za1zHOj8Qb/eQwJJumNo3w7VaqG3koknHzmLvOhp7uvbnY+u/awOkiAW cblmNU1rCO7pJAhb//e/iwl17pqffbyr4gghdJ0mWabcLNr88942yt6kS zOr3zB5oeAV8TCcZwTAW7KnTL/CN3WSYIRoqj0teW/MWe3NfCI+YHJuoI w==; X-IronPort-AV: E=McAfee;i="6500,9779,10539"; a="293616515" X-IronPort-AV: E=Sophos;i="5.96,185,1665471600"; d="scan'208";a="293616515" Received: from orsmga005.jf.intel.com ([10.7.209.41]) by fmsmga106.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 22 Nov 2022 12:28:16 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10539"; a="816228147" X-IronPort-AV: E=Sophos;i="5.96,185,1665471600"; d="scan'208";a="816228147" Received: from ranerica-svr.sc.intel.com ([172.25.110.23]) by orsmga005.jf.intel.com with ESMTP; 22 Nov 2022 12:28:15 -0800 From: Ricardo Neri To: "Peter Zijlstra (Intel)" , Juri Lelli , Vincent Guittot Cc: Ricardo Neri , "Ravi V. Shankar" , Ben Segall , Daniel Bristot de Oliveira , Dietmar Eggemann , Len Brown , Mel Gorman , "Rafael J. Wysocki" , Srinivas Pandruvada , Steven Rostedt , Tim Chen , Valentin Schneider , x86@kernel.org, linux-kernel@vger.kernel.org, Ricardo Neri , "Tim C . Chen" Subject: [PATCH v2 7/7] x86/sched/itmt: Consider the idle state of SMT siblings Date: Tue, 22 Nov 2022 12:35:32 -0800 Message-Id: <20221122203532.15013-8-ricardo.neri-calderon@linux.intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20221122203532.15013-1-ricardo.neri-calderon@linux.intel.com> References: <20221122203532.15013-1-ricardo.neri-calderon@linux.intel.com> Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Cores with more than one busy SMT sibling need help from fully idle cores, even if they have lower priority. Return a priority of 0 if any of the SMT siblings of a CPU are busy. This lets lower-priority idle cores pull tasks from the CPU. Only do this if the scheduler cares about the idle state of the siblings. Cc: Ben Segall Cc: Daniel Bristot de Oliveira Cc: Dietmar Eggemann Cc: Len Brown Cc: Mel Gorman Cc: Rafael J. Wysocki Cc: Srinivas Pandruvada Cc: Steven Rostedt Cc: Tim C. Chen Cc: Valentin Schneider Cc: x86@kernel.org Cc: linux-kernel@vger.kernel.org Signed-off-by: Ricardo Neri --- Changes since v1: * Introduced this patch. --- arch/x86/kernel/itmt.c | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/arch/x86/kernel/itmt.c b/arch/x86/kernel/itmt.c index 2be49ce4f94a..cb22944969a1 100644 --- a/arch/x86/kernel/itmt.c +++ b/arch/x86/kernel/itmt.c @@ -169,7 +169,10 @@ void sched_clear_itmt_support(void) =20 int arch_asym_cpu_priority(int cpu, bool check_smt) { - return per_cpu(sched_core_priority, cpu); + if (!check_smt || sched_smt_siblings_idle(cpu)) + return per_cpu(sched_core_priority, cpu); + + return 0; } =20 /** --=20 2.25.1