From nobody Fri Jan 2 19:07:44 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 11A3FE95A8E for ; Mon, 9 Oct 2023 10:36:35 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1345904AbjJIKge (ORCPT ); Mon, 9 Oct 2023 06:36:34 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46986 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1345870AbjJIKga (ORCPT ); Mon, 9 Oct 2023 06:36:30 -0400 Received: from mail-wr1-x42f.google.com (mail-wr1-x42f.google.com [IPv6:2a00:1450:4864:20::42f]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 50E629D for ; Mon, 9 Oct 2023 03:36:28 -0700 (PDT) Received: by mail-wr1-x42f.google.com with SMTP id ffacd0b85a97d-3296a998234so2517865f8f.2 for ; Mon, 09 Oct 2023 03:36:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1696847787; x=1697452587; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=re7b+0McMMJs79AIsRmZVXLCw9N+5e5Qqk3aklm3ETc=; b=kmsDUtAsrjuA2Wdc3hsWu2yZ/c5ilHfeqWH+Uuy+u4uj5O8kL9oPPTgZY1WcBjFw00 2rUWYUsebVOmdH72oqY/vP81dvhWhiUfXh6lx45D1eZclu88SolVvC3xPlKMhiL4KxUL OsoXCXNSp2WTNhwCszD/iWRBsRZfzua0Xb6RpVUpMIjjE9HYaaqfL9WRUU7dMOyDe4sH vkT7+eHAg0DNZ11O6t3qkkbDkYcrT/s7n6EdvchnMJXVUeqWblDWx7NE2ATUrcMeoPGL /9dQCqKGrKIeiqxQDcC1RWTBjHngA05H9WfR8uqH5iM6z7/sFuRxb5Eg7pIC3oKUji5I SI5A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1696847787; x=1697452587; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=re7b+0McMMJs79AIsRmZVXLCw9N+5e5Qqk3aklm3ETc=; b=apk2+i6zWJP/SRTykMZ21amPIzTTL4m6mQ6AMXsGNsYDYTv+uj+/EatZ9X9GywGSxH vKrERWB7wK07M2ELe5RMiUM0o6cuiFMtIpyLryjcVgeMjH2OfezsM+Vi6kHSKnVMb5Sv p+QppYpAoxtenI6AYDWh0vOoGMHqMUERHB9hAuU8pcRFWKwFe0h+YNbJvT3q9bC8diEF oV9HRhUIAt2yoUw3lfwyabB9kwe/BQcIW/DTHX+4dqZ0RvbMuqeIkSlqU0HzVCMdcWQ5 H4dUJkbmykuNb+k0eMxE0zi4DKyfO3rvcixneNm5ckJ2bejYOIjhHxptuxX7Qf8cl37h yq/w== X-Gm-Message-State: AOJu0YzNyLJvjLhBasgm95F2glwPInD0eebDfrtxqJUqbLGKERztczTn WF6m634gNX7HhDWp+LXjivpTOI5+JMmxVFbt+ac= X-Google-Smtp-Source: AGHT+IEjQw3F2QpiyYMDHKQ0Ydoc0UsVzOGqUY2ht0reqHXmAAoEP6EJ50UQeRQWWBd0aG7tZrfNCQ== X-Received: by 2002:adf:f505:0:b0:327:ffef:ba51 with SMTP id q5-20020adff505000000b00327ffefba51mr13155370wro.49.1696847786785; Mon, 09 Oct 2023 03:36:26 -0700 (PDT) Received: from vingu-book.. ([2a01:e0a:f:6020:53f1:24bc:5e47:821d]) by smtp.gmail.com with ESMTPSA id f16-20020adfdb50000000b0031ff89af0e4sm9226722wrj.99.2023.10.09.03.36.25 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 09 Oct 2023 03:36:26 -0700 (PDT) From: Vincent Guittot To: linux@armlinux.org.uk, catalin.marinas@arm.com, will@kernel.org, paul.walmsley@sifive.com, palmer@dabbelt.com, aou@eecs.berkeley.edu, sudeep.holla@arm.com, gregkh@linuxfoundation.org, rafael@kernel.org, mingo@redhat.com, peterz@infradead.org, juri.lelli@redhat.com, dietmar.eggemann@arm.com, rostedt@goodmis.org, bsegall@google.com, mgorman@suse.de, bristot@redhat.com, vschneid@redhat.com, viresh.kumar@linaro.org, lukasz.luba@arm.com, ionela.voinescu@arm.com, pierre.gondois@arm.com, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org, linux-pm@vger.kernel.org Cc: conor.dooley@microchip.com, suagrfillet@gmail.com, ajones@ventanamicro.com, lftan@kernel.org, Vincent Guittot Subject: [PATCH v2 1/6] sched: consolidate and cleanup access to CPU's max compute capacity Date: Mon, 9 Oct 2023 12:36:16 +0200 Message-Id: <20231009103621.374412-2-vincent.guittot@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20231009103621.374412-1-vincent.guittot@linaro.org> References: <20231009103621.374412-1-vincent.guittot@linaro.org> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Remove struct rq cpu_capacity_orig field and use arch_scale_cpu_capacity() instead. Scheduler uses 3 methods to get access to the CPU's max compute capacity: - arch_scale_cpu_capacity(cpu) which is the default way to get CPU's capaci= ty. - cpu_capacity_orig field which is periodically updated with arch_scale_cpu_capacity(). - capacity_orig_of(cpu) which encapsulates rq->cpu_capacity_orig. There is no real need to save the value returned by arch_scale_cpu_capacity= () in struct rq. arch_scale_cpu_capacity() returns: - either a per_cpu variable. - or a const value for systems which have only one capacity. Remove cpu_capacity_orig and use arch_scale_cpu_capacity() everywhere. No functional changes. some tests of Arm64 small SMP device (hikey): no noticeable changes HMP device (RB5): hackbench shows minor improvement (1-2%) large smp (thx2): hackbench and tbench shows minor improvement (1%) Signed-off-by: Vincent Guittot Reviewed-by: Dietmar Eggemann Tested-by: Lukasz Luba --- Documentation/scheduler/sched-capacity.rst | 13 +++++++------ kernel/sched/core.c | 2 +- kernel/sched/cpudeadline.c | 2 +- kernel/sched/deadline.c | 4 ++-- kernel/sched/fair.c | 18 ++++++++---------- kernel/sched/rt.c | 2 +- kernel/sched/sched.h | 6 ------ kernel/sched/topology.c | 7 +++++-- 8 files changed, 25 insertions(+), 29 deletions(-) diff --git a/Documentation/scheduler/sched-capacity.rst b/Documentation/sch= eduler/sched-capacity.rst index e2c1cf743158..de414b33dd2a 100644 --- a/Documentation/scheduler/sched-capacity.rst +++ b/Documentation/scheduler/sched-capacity.rst @@ -39,14 +39,15 @@ per Hz, leading to:: ------------------- =20 Two different capacity values are used within the scheduler. A CPU's -``capacity_orig`` is its maximum attainable capacity, i.e. its maximum -attainable performance level. A CPU's ``capacity`` is its ``capacity_orig`= ` to -which some loss of available performance (e.g. time spent handling IRQs) is -subtracted. +``original capacity`` is its maximum attainable capacity, i.e. its maximum +attainable performance level. This original capacity is returned by +the function arch_scale_cpu_capacity(). A CPU's ``capacity`` is its ``orig= inal +capacity`` to which some loss of available performance (e.g. time spent +handling IRQs) is subtracted. =20 Note that a CPU's ``capacity`` is solely intended to be used by the CFS cl= ass, -while ``capacity_orig`` is class-agnostic. The rest of this document will = use -the term ``capacity`` interchangeably with ``capacity_orig`` for the sake = of +while ``original capacity`` is class-agnostic. The rest of this document w= ill use +the term ``capacity`` interchangeably with ``original capacity`` for the s= ake of brevity. =20 1.3 Platform examples diff --git a/kernel/sched/core.c b/kernel/sched/core.c index cf6d3fdd4eb5..a3f9cd52eec5 100644 --- a/kernel/sched/core.c +++ b/kernel/sched/core.c @@ -9929,7 +9929,7 @@ void __init sched_init(void) #ifdef CONFIG_SMP rq->sd =3D NULL; rq->rd =3D NULL; - rq->cpu_capacity =3D rq->cpu_capacity_orig =3D SCHED_CAPACITY_SCALE; + rq->cpu_capacity =3D SCHED_CAPACITY_SCALE; rq->balance_callback =3D &balance_push_callback; rq->active_balance =3D 0; rq->next_balance =3D jiffies; diff --git a/kernel/sched/cpudeadline.c b/kernel/sched/cpudeadline.c index 57c92d751bcd..95baa12a1029 100644 --- a/kernel/sched/cpudeadline.c +++ b/kernel/sched/cpudeadline.c @@ -131,7 +131,7 @@ int cpudl_find(struct cpudl *cp, struct task_struct *p, if (!dl_task_fits_capacity(p, cpu)) { cpumask_clear_cpu(cpu, later_mask); =20 - cap =3D capacity_orig_of(cpu); + cap =3D arch_scale_cpu_capacity(cpu); =20 if (cap > max_cap || (cpu =3D=3D task_cpu(p) && cap =3D=3D max_cap)) { diff --git a/kernel/sched/deadline.c b/kernel/sched/deadline.c index d98408a274e5..7039a8d5ae9b 100644 --- a/kernel/sched/deadline.c +++ b/kernel/sched/deadline.c @@ -132,7 +132,7 @@ static inline unsigned long __dl_bw_capacity(const stru= ct cpumask *mask) int i; =20 for_each_cpu_and(i, mask, cpu_active_mask) - cap +=3D capacity_orig_of(i); + cap +=3D arch_scale_cpu_capacity(i); =20 return cap; } @@ -144,7 +144,7 @@ static inline unsigned long __dl_bw_capacity(const stru= ct cpumask *mask) static inline unsigned long dl_bw_capacity(int i) { if (!sched_asym_cpucap_active() && - capacity_orig_of(i) =3D=3D SCHED_CAPACITY_SCALE) { + arch_scale_cpu_capacity(i) =3D=3D SCHED_CAPACITY_SCALE) { return dl_bw_cpus(i) << SCHED_CAPACITY_SHIFT; } else { RCU_LOCKDEP_WARN(!rcu_read_lock_sched_held(), diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index 04fbcbda97d5..7e2027c810e7 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -4713,7 +4713,7 @@ static inline void util_est_update(struct cfs_rq *cfs= _rq, * To avoid overestimation of actual task utilization, skip updates if * we cannot grant there is idle time in this CPU. */ - if (task_util(p) > capacity_orig_of(cpu_of(rq_of(cfs_rq)))) + if (task_util(p) > arch_scale_cpu_capacity(cpu_of(rq_of(cfs_rq)))) return; =20 /* @@ -4761,14 +4761,14 @@ static inline int util_fits_cpu(unsigned long util, return fits; =20 /* - * We must use capacity_orig_of() for comparing against uclamp_min and + * We must use arch_scale_cpu_capacity() for comparing against uclamp_min= and * uclamp_max. We only care about capacity pressure (by using * capacity_of()) for comparing against the real util. * * If a task is boosted to 1024 for example, we don't want a tiny * pressure to skew the check whether it fits a CPU or not. * - * Similarly if a task is capped to capacity_orig_of(little_cpu), it + * Similarly if a task is capped to arch_scale_cpu_capacity(little_cpu), = it * should fit a little cpu even if there's some pressure. * * Only exception is for thermal pressure since it has a direct impact @@ -4780,7 +4780,7 @@ static inline int util_fits_cpu(unsigned long util, * For uclamp_max, we can tolerate a drop in performance level as the * goal is to cap the task. So it's okay if it's getting less. */ - capacity_orig =3D capacity_orig_of(cpu); + capacity_orig =3D arch_scale_cpu_capacity(cpu); capacity_orig_thermal =3D capacity_orig - arch_scale_thermal_pressure(cpu= ); =20 /* @@ -7261,7 +7261,7 @@ select_idle_capacity(struct task_struct *p, struct sc= hed_domain *sd, int target) * Look for the CPU with best capacity. */ else if (fits < 0) - cpu_cap =3D capacity_orig_of(cpu) - thermal_load_avg(cpu_rq(cpu)); + cpu_cap =3D arch_scale_cpu_capacity(cpu) - thermal_load_avg(cpu_rq(cpu)= ); =20 /* * First, select CPU which fits better (-1 being better than 0). @@ -7503,7 +7503,7 @@ cpu_util(int cpu, struct task_struct *p, int dst_cpu,= int boost) util =3D max(util, util_est); } =20 - return min(util, capacity_orig_of(cpu)); + return min(util, arch_scale_cpu_capacity(cpu)); } =20 unsigned long cpu_util_cfs(int cpu) @@ -9294,8 +9294,6 @@ static void update_cpu_capacity(struct sched_domain *= sd, int cpu) unsigned long capacity =3D scale_rt_capacity(cpu); struct sched_group *sdg =3D sd->groups; =20 - cpu_rq(cpu)->cpu_capacity_orig =3D arch_scale_cpu_capacity(cpu); - if (!capacity) capacity =3D 1; =20 @@ -9371,7 +9369,7 @@ static inline int check_cpu_capacity(struct rq *rq, struct sched_domain *sd) { return ((rq->cpu_capacity * sd->imbalance_pct) < - (rq->cpu_capacity_orig * 100)); + (arch_scale_cpu_capacity(cpu_of(rq)) * 100)); } =20 /* @@ -9382,7 +9380,7 @@ check_cpu_capacity(struct rq *rq, struct sched_domain= *sd) static inline int check_misfit_status(struct rq *rq, struct sched_domain *= sd) { return rq->misfit_task_load && - (rq->cpu_capacity_orig < rq->rd->max_cpu_capacity || + (arch_scale_cpu_capacity(rq->cpu) < rq->rd->max_cpu_capacity || check_cpu_capacity(rq, sd)); } =20 diff --git a/kernel/sched/rt.c b/kernel/sched/rt.c index 88fc98601413..72f0a0767059 100644 --- a/kernel/sched/rt.c +++ b/kernel/sched/rt.c @@ -471,7 +471,7 @@ static inline bool rt_task_fits_capacity(struct task_st= ruct *p, int cpu) min_cap =3D uclamp_eff_value(p, UCLAMP_MIN); max_cap =3D uclamp_eff_value(p, UCLAMP_MAX); =20 - cpu_cap =3D capacity_orig_of(cpu); + cpu_cap =3D arch_scale_cpu_capacity(cpu); =20 return cpu_cap >=3D min(min_cap, max_cap); } diff --git a/kernel/sched/sched.h b/kernel/sched/sched.h index 649eb9ec0657..74195eb39eaa 100644 --- a/kernel/sched/sched.h +++ b/kernel/sched/sched.h @@ -1033,7 +1033,6 @@ struct rq { struct sched_domain __rcu *sd; =20 unsigned long cpu_capacity; - unsigned long cpu_capacity_orig; =20 struct balance_callback *balance_callback; =20 @@ -2967,11 +2966,6 @@ static inline void cpufreq_update_util(struct rq *rq= , unsigned int flags) {} #endif =20 #ifdef CONFIG_SMP -static inline unsigned long capacity_orig_of(int cpu) -{ - return cpu_rq(cpu)->cpu_capacity_orig; -} - /** * enum cpu_util_type - CPU utilization type * @FREQUENCY_UTIL: Utilization used to select frequency diff --git a/kernel/sched/topology.c b/kernel/sched/topology.c index a7b50bba7829..1cc595907363 100644 --- a/kernel/sched/topology.c +++ b/kernel/sched/topology.c @@ -2488,12 +2488,15 @@ build_sched_domains(const struct cpumask *cpu_map, = struct sched_domain_attr *att /* Attach the domains */ rcu_read_lock(); for_each_cpu(i, cpu_map) { + unsigned long capacity; + rq =3D cpu_rq(i); sd =3D *per_cpu_ptr(d.sd, i); =20 + capacity =3D arch_scale_cpu_capacity(i); /* Use READ_ONCE()/WRITE_ONCE() to avoid load/store tearing: */ - if (rq->cpu_capacity_orig > READ_ONCE(d.rd->max_cpu_capacity)) - WRITE_ONCE(d.rd->max_cpu_capacity, rq->cpu_capacity_orig); + if (capacity > READ_ONCE(d.rd->max_cpu_capacity)) + WRITE_ONCE(d.rd->max_cpu_capacity, capacity); =20 cpu_attach_domain(sd, d.rd, i); } --=20 2.34.1 From nobody Fri Jan 2 19:07:44 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 14978E95A8E for ; Mon, 9 Oct 2023 10:36:45 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1346019AbjJIKgn (ORCPT ); Mon, 9 Oct 2023 06:36:43 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47028 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1345888AbjJIKgc (ORCPT ); Mon, 9 Oct 2023 06:36:32 -0400 Received: from mail-wm1-x32c.google.com (mail-wm1-x32c.google.com [IPv6:2a00:1450:4864:20::32c]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D01D3BA for ; Mon, 9 Oct 2023 03:36:29 -0700 (PDT) Received: by mail-wm1-x32c.google.com with SMTP id 5b1f17b1804b1-4056ce55e7eso41020005e9.2 for ; Mon, 09 Oct 2023 03:36:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1696847788; x=1697452588; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=1IicB4ODERuC1IEqNgt/JtBz6z2h9Ys3ExqnEipNKQY=; b=AasJ6UtCQrpQt+8GtUfwZdiYa81u585pfCUctZVy8RZLsEdtULx8chhLCtjK5RLOWH /WrYoJ8QJ/R+T01N4u6AK0uWobVZGtDL+WdEN2K/rQTF0PUlo3+NMQGFwvFGlkWMSu5+ uTNvgN5ZXhoslm0foOsgUd+NcHcUeVKpIY/PEaFtd+AYCeVOqHZsgwKh7L2gh8qONXJm bz+8Su8VmsHNJ+MLh3Ff6Y1ia0CBw59dj6hFgb57L6ugeW+Gfy0J/sHq95/wUWRJa3e2 +Dvj1n0t2iT1JLGLpV/XCY1lDFhuqT8YD+rQ0A8YBnY5DtSTlApOXgD7qwjjgSk5qoyW yEOQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1696847788; x=1697452588; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=1IicB4ODERuC1IEqNgt/JtBz6z2h9Ys3ExqnEipNKQY=; b=Th9PEY5Ya1DlF3qL0J35JdlN4YW8fWHVeCRJBegNNKEZFh7wzcjTSmipcVIqK1B6JK U4slSTvSM11jmM1I8nn7Q6oYkwroxKRs+eSw1yN9AoK13BmcqErwCFaaqI3l6yNLt47J m2cgt9+9WAT35KinN+JmbC6F3+xU8O6bVuDB0QYPl6JZeg7HwDGbTKk3iTloXWbTawxz 5B6BCxaX7n0ljbn2c97Rw9md/oQpi25xcT5kFY/C6+xrmuRQ8A7RDyejOSRe5Blw55o+ 3DM8Ln7FMCZM39tQJ8DyVRgNKsUbcrTFhxCiqv3Kw8gLHMHzbx22DkHxg2pmMA7q44Fl DYoQ== X-Gm-Message-State: AOJu0YwTXPrB6arfPwQqMOvh5+ZZoUNfe3pyD29g2WlpwNGSf61PZBz6 nGmAxofmMl5fNjV7zoKL+XZEtw== X-Google-Smtp-Source: AGHT+IGYCDsjL+s+eXaJYDwmVGq+Pm1XzCDysQm4pemCHQ7SCDtFRnahF5GOdbFaRBcM0Tt7ZFT2Dw== X-Received: by 2002:a5d:4d02:0:b0:317:6fb5:bafc with SMTP id z2-20020a5d4d02000000b003176fb5bafcmr12403144wrt.65.1696847788095; Mon, 09 Oct 2023 03:36:28 -0700 (PDT) Received: from vingu-book.. ([2a01:e0a:f:6020:53f1:24bc:5e47:821d]) by smtp.gmail.com with ESMTPSA id f16-20020adfdb50000000b0031ff89af0e4sm9226722wrj.99.2023.10.09.03.36.26 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 09 Oct 2023 03:36:27 -0700 (PDT) From: Vincent Guittot To: linux@armlinux.org.uk, catalin.marinas@arm.com, will@kernel.org, paul.walmsley@sifive.com, palmer@dabbelt.com, aou@eecs.berkeley.edu, sudeep.holla@arm.com, gregkh@linuxfoundation.org, rafael@kernel.org, mingo@redhat.com, peterz@infradead.org, juri.lelli@redhat.com, dietmar.eggemann@arm.com, rostedt@goodmis.org, bsegall@google.com, mgorman@suse.de, bristot@redhat.com, vschneid@redhat.com, viresh.kumar@linaro.org, lukasz.luba@arm.com, ionela.voinescu@arm.com, pierre.gondois@arm.com, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org, linux-pm@vger.kernel.org Cc: conor.dooley@microchip.com, suagrfillet@gmail.com, ajones@ventanamicro.com, lftan@kernel.org, Vincent Guittot Subject: [PATCH v2 2/6] topology: add a new arch_scale_freq_reference Date: Mon, 9 Oct 2023 12:36:17 +0200 Message-Id: <20231009103621.374412-3-vincent.guittot@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20231009103621.374412-1-vincent.guittot@linaro.org> References: <20231009103621.374412-1-vincent.guittot@linaro.org> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Create a new method to get a unique and fixed max frequency. Currently cpuinfo.max_freq or the highest (or last) state of performance domain are used as the max frequency when computing the frequency for a level of utilization but: - cpuinfo_max_freq can change at runtime. boost is one example of such change. - cpuinfo.max_freq and last item of the PD can be different leading to different results between cpufreq and energy model. We need to save the reference frequency that has been used when computing the CPUs capacity and use this fixed and coherent value to convert between frequency and CPU's capacity. In fact, we already save the frequency that has been used when computing the capacity of each CPU. We extend the precision to save khZ instead of Mhz currently and we modify the type to be aligned with other variables used when converting frequency to capacity and the other way. Signed-off-by: Vincent Guittot Reviewed-by: Lukasz Luba Tested-by: Lukasz Luba --- arch/arm/include/asm/topology.h | 1 + arch/arm64/include/asm/topology.h | 1 + arch/riscv/include/asm/topology.h | 1 + drivers/base/arch_topology.c | 29 ++++++++++++++--------------- include/linux/arch_topology.h | 7 +++++++ 5 files changed, 24 insertions(+), 15 deletions(-) diff --git a/arch/arm/include/asm/topology.h b/arch/arm/include/asm/topolog= y.h index c7d2510e5a78..853c4f81ba4a 100644 --- a/arch/arm/include/asm/topology.h +++ b/arch/arm/include/asm/topology.h @@ -13,6 +13,7 @@ #define arch_set_freq_scale topology_set_freq_scale #define arch_scale_freq_capacity topology_get_freq_scale #define arch_scale_freq_invariant topology_scale_freq_invariant +#define arch_scale_freq_ref topology_get_freq_ref #endif =20 /* Replace task scheduler's default cpu-invariant accounting */ diff --git a/arch/arm64/include/asm/topology.h b/arch/arm64/include/asm/top= ology.h index 9fab663dd2de..a323b109b9c4 100644 --- a/arch/arm64/include/asm/topology.h +++ b/arch/arm64/include/asm/topology.h @@ -23,6 +23,7 @@ void update_freq_counters_refs(void); #define arch_set_freq_scale topology_set_freq_scale #define arch_scale_freq_capacity topology_get_freq_scale #define arch_scale_freq_invariant topology_scale_freq_invariant +#define arch_scale_freq_ref topology_get_freq_ref =20 #ifdef CONFIG_ACPI_CPPC_LIB #define arch_init_invariance_cppc topology_init_cpu_capacity_cppc diff --git a/arch/riscv/include/asm/topology.h b/arch/riscv/include/asm/top= ology.h index e316ab3b77f3..61183688bdd5 100644 --- a/arch/riscv/include/asm/topology.h +++ b/arch/riscv/include/asm/topology.h @@ -9,6 +9,7 @@ #define arch_set_freq_scale topology_set_freq_scale #define arch_scale_freq_capacity topology_get_freq_scale #define arch_scale_freq_invariant topology_scale_freq_invariant +#define arch_scale_freq_ref topology_get_freq_ref =20 /* Replace task scheduler's default cpu-invariant accounting */ #define arch_scale_cpu_capacity topology_get_cpu_scale diff --git a/drivers/base/arch_topology.c b/drivers/base/arch_topology.c index b741b5ba82bd..9a073c2d2086 100644 --- a/drivers/base/arch_topology.c +++ b/drivers/base/arch_topology.c @@ -19,6 +19,7 @@ #include #include #include +#include =20 #define CREATE_TRACE_POINTS #include @@ -26,7 +27,8 @@ static DEFINE_PER_CPU(struct scale_freq_data __rcu *, sft_data); static struct cpumask scale_freq_counters_mask; static bool scale_freq_invariant; -static DEFINE_PER_CPU(u32, freq_factor) =3D 1; +DEFINE_PER_CPU(unsigned long, capacity_ref_freq) =3D 1; +EXPORT_PER_CPU_SYMBOL_GPL(capacity_ref_freq); =20 static bool supports_scale_freq_counters(const struct cpumask *cpus) { @@ -170,9 +172,9 @@ DEFINE_PER_CPU(unsigned long, thermal_pressure); * operating on stale data when hot-plug is used for some CPUs. The * @capped_freq reflects the currently allowed max CPUs frequency due to * thermal capping. It might be also a boost frequency value, which is big= ger - * than the internal 'freq_factor' max frequency. In such case the pressure - * value should simply be removed, since this is an indication that there = is - * no thermal throttling. The @capped_freq must be provided in kHz. + * than the internal 'capacity_ref_freq' max frequency. In such case the + * pressure value should simply be removed, since this is an indication th= at + * there is no thermal throttling. The @capped_freq must be provided in kH= z. */ void topology_update_thermal_pressure(const struct cpumask *cpus, unsigned long capped_freq) @@ -183,10 +185,7 @@ void topology_update_thermal_pressure(const struct cpu= mask *cpus, =20 cpu =3D cpumask_first(cpus); max_capacity =3D arch_scale_cpu_capacity(cpu); - max_freq =3D per_cpu(freq_factor, cpu); - - /* Convert to MHz scale which is used in 'freq_factor' */ - capped_freq /=3D 1000; + max_freq =3D arch_scale_freq_ref(cpu); =20 /* * Handle properly the boost frequencies, which should simply clean @@ -279,13 +278,13 @@ void topology_normalize_cpu_scale(void) =20 capacity_scale =3D 1; for_each_possible_cpu(cpu) { - capacity =3D raw_capacity[cpu] * per_cpu(freq_factor, cpu); + capacity =3D raw_capacity[cpu] * per_cpu(capacity_ref_freq, cpu); capacity_scale =3D max(capacity, capacity_scale); } =20 pr_debug("cpu_capacity: capacity_scale=3D%llu\n", capacity_scale); for_each_possible_cpu(cpu) { - capacity =3D raw_capacity[cpu] * per_cpu(freq_factor, cpu); + capacity =3D raw_capacity[cpu] * per_cpu(capacity_ref_freq, cpu); capacity =3D div64_u64(capacity << SCHED_CAPACITY_SHIFT, capacity_scale); topology_set_cpu_scale(cpu, capacity); @@ -321,15 +320,15 @@ bool __init topology_parse_cpu_capacity(struct device= _node *cpu_node, int cpu) cpu_node, raw_capacity[cpu]); =20 /* - * Update freq_factor for calculating early boot cpu capacities. + * Update capacity_ref_freq for calculating early boot cpu capacities. * For non-clk CPU DVFS mechanism, there's no way to get the * frequency value now, assuming they are running at the same - * frequency (by keeping the initial freq_factor value). + * frequency (by keeping the initial capacity_ref_freq value). */ cpu_clk =3D of_clk_get(cpu_node, 0); if (!PTR_ERR_OR_ZERO(cpu_clk)) { - per_cpu(freq_factor, cpu) =3D - clk_get_rate(cpu_clk) / 1000; + per_cpu(capacity_ref_freq, cpu) =3D + clk_get_rate(cpu_clk) / HZ_PER_KHZ; clk_put(cpu_clk); } } else { @@ -411,7 +410,7 @@ init_cpu_capacity_callback(struct notifier_block *nb, cpumask_andnot(cpus_to_visit, cpus_to_visit, policy->related_cpus); =20 for_each_cpu(cpu, policy->related_cpus) - per_cpu(freq_factor, cpu) =3D policy->cpuinfo.max_freq / 1000; + per_cpu(capacity_ref_freq, cpu) =3D policy->cpuinfo.max_freq; =20 if (cpumask_empty(cpus_to_visit)) { topology_normalize_cpu_scale(); diff --git a/include/linux/arch_topology.h b/include/linux/arch_topology.h index a07b510e7dc5..38ca6c76af56 100644 --- a/include/linux/arch_topology.h +++ b/include/linux/arch_topology.h @@ -27,6 +27,13 @@ static inline unsigned long topology_get_cpu_scale(int c= pu) =20 void topology_set_cpu_scale(unsigned int cpu, unsigned long capacity); =20 +DECLARE_PER_CPU(unsigned long, capacity_ref_freq); + +static inline unsigned long topology_get_freq_ref(int cpu) +{ + return per_cpu(capacity_ref_freq, cpu); +} + DECLARE_PER_CPU(unsigned long, arch_freq_scale); =20 static inline unsigned long topology_get_freq_scale(int cpu) --=20 2.34.1 From nobody Fri Jan 2 19:07:44 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id D733CE95A97 for ; Mon, 9 Oct 2023 10:36:40 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1345976AbjJIKgj (ORCPT ); Mon, 9 Oct 2023 06:36:39 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47068 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1345908AbjJIKge (ORCPT ); Mon, 9 Oct 2023 06:36:34 -0400 Received: from mail-wm1-x32b.google.com (mail-wm1-x32b.google.com [IPv6:2a00:1450:4864:20::32b]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0FB549F for ; Mon, 9 Oct 2023 03:36:31 -0700 (PDT) Received: by mail-wm1-x32b.google.com with SMTP id 5b1f17b1804b1-40566f8a093so40290105e9.3 for ; Mon, 09 Oct 2023 03:36:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1696847789; x=1697452589; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=kNKKT3nz1YObI5oerkbE50Couf3/jlljKZ/fkPRsfb8=; b=KKeeHEgwkzQ7bGffr2RggX/J5Aa4SZbzSD22Y5kids44XDdQavj8NZcLWx99fYBZ44 wIcGSDTNGZfBUNPtXTPGSYvprHajbjEV5o3pkN9iozzacYoFjWK0YowgKFPjbyxhUr/N sj9jnBgC4u8sVATjbsjYf0H32w6xoMuBLH4CdlRu4BmmZkHjZvKyjwPmlp+SR6paFCiG 5vHmH2oc5AFL7ysO68tWlkmkvKm/FlEnSh7vog1EsBd11/nWf5R8L6d903K82pOls5oX OByluKj0k3RjPvFjgO9OGw3mKzpAlxhqx6BGxGRs+z2Y0e6+w0Ws8l9y2fI1DXs0KydH 5S3Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1696847789; x=1697452589; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=kNKKT3nz1YObI5oerkbE50Couf3/jlljKZ/fkPRsfb8=; b=lj8sDb7d71reqkL7KiXT4F0zf7LumGTZoCDr8XzpiedJ2wzPCrCjmaESUsGdByA00f J4HbqQHABjQJclGEFmtECddmbofAqNk1XOSQGqD1c27G+9t0AeMuKpQEsC1I6JRnCPlU +YXmoVf92S3Tk6wYI71GF8kpnOnFzYjIadtfFKasJ09QVXwmKi8YffV9c2VjdoCN/gWw QfwoBrZfbFR1wccOMLz6daWMFTIqdb1eL8MRDFZBUtgXg46zhNfuUz1eC5O0LzXeCbLQ tq1nMcQZnwQZvHq/45EkeqNGL17i0Z6PLBzEjQwVx6E8+f2Hoi9mrcMK8QBJcf0mRqOu 22wQ== X-Gm-Message-State: AOJu0YwOfLAMsJNjZUOvy9OlvOyrjSjzcUyRJh1Vki9iaQkkk1QUHh0f 5eIioeVMhslEpGuADYABkjo3zQ== X-Google-Smtp-Source: AGHT+IG26Qla65j//2waJk5p+C3Wp1IkFGp+wjCfVC7nMqMP8R+PELrHfGujdUavK8O0HgXO8xjo4Q== X-Received: by 2002:a5d:4586:0:b0:321:5e2f:37e1 with SMTP id p6-20020a5d4586000000b003215e2f37e1mr13248647wrq.19.1696847789459; Mon, 09 Oct 2023 03:36:29 -0700 (PDT) Received: from vingu-book.. ([2a01:e0a:f:6020:53f1:24bc:5e47:821d]) by smtp.gmail.com with ESMTPSA id f16-20020adfdb50000000b0031ff89af0e4sm9226722wrj.99.2023.10.09.03.36.28 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 09 Oct 2023 03:36:28 -0700 (PDT) From: Vincent Guittot To: linux@armlinux.org.uk, catalin.marinas@arm.com, will@kernel.org, paul.walmsley@sifive.com, palmer@dabbelt.com, aou@eecs.berkeley.edu, sudeep.holla@arm.com, gregkh@linuxfoundation.org, rafael@kernel.org, mingo@redhat.com, peterz@infradead.org, juri.lelli@redhat.com, dietmar.eggemann@arm.com, rostedt@goodmis.org, bsegall@google.com, mgorman@suse.de, bristot@redhat.com, vschneid@redhat.com, viresh.kumar@linaro.org, lukasz.luba@arm.com, ionela.voinescu@arm.com, pierre.gondois@arm.com, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org, linux-pm@vger.kernel.org Cc: conor.dooley@microchip.com, suagrfillet@gmail.com, ajones@ventanamicro.com, lftan@kernel.org, Vincent Guittot Subject: [PATCH v2 3/6] cpufreq: use the fixed and coherent frequency for scaling capacity Date: Mon, 9 Oct 2023 12:36:18 +0200 Message-Id: <20231009103621.374412-4-vincent.guittot@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20231009103621.374412-1-vincent.guittot@linaro.org> References: <20231009103621.374412-1-vincent.guittot@linaro.org> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" cpuinfo.max_freq can change at runtime because of boost as an example. This implies that the value could be different from the frequency that has been used to compute the capacity of a CPU. The new arch_scale_freq_ref() returns a fixed and coherent frequency that can be used to compute the capacity for a given frequency. Signed-off-by: Vincent Guittot Acked-by: Viresh Kumar Reviewed-by: Lukasz Luba Tested-by: Lukasz Luba --- drivers/cpufreq/cpufreq.c | 4 ++-- include/linux/cpufreq.h | 9 +++++++++ 2 files changed, 11 insertions(+), 2 deletions(-) diff --git a/drivers/cpufreq/cpufreq.c b/drivers/cpufreq/cpufreq.c index 60ed89000e82..8c4f9c2f9c44 100644 --- a/drivers/cpufreq/cpufreq.c +++ b/drivers/cpufreq/cpufreq.c @@ -454,7 +454,7 @@ void cpufreq_freq_transition_end(struct cpufreq_policy = *policy, =20 arch_set_freq_scale(policy->related_cpus, policy->cur, - policy->cpuinfo.max_freq); + arch_scale_freq_ref(policy->cpu)); =20 spin_lock(&policy->transition_lock); policy->transition_ongoing =3D false; @@ -2174,7 +2174,7 @@ unsigned int cpufreq_driver_fast_switch(struct cpufre= q_policy *policy, =20 policy->cur =3D freq; arch_set_freq_scale(policy->related_cpus, freq, - policy->cpuinfo.max_freq); + arch_scale_freq_ref(policy->cpu)); cpufreq_stats_record_transition(policy, freq); =20 if (trace_cpu_frequency_enabled()) { diff --git a/include/linux/cpufreq.h b/include/linux/cpufreq.h index 71d186d6933a..bbc483b4b6e5 100644 --- a/include/linux/cpufreq.h +++ b/include/linux/cpufreq.h @@ -1211,6 +1211,15 @@ void arch_set_freq_scale(const struct cpumask *cpus, { } #endif + +#ifndef arch_scale_freq_ref +static __always_inline +unsigned int arch_scale_freq_ref(int cpu) +{ + return 0; +} +#endif + /* the following are really really optional */ extern struct freq_attr cpufreq_freq_attr_scaling_available_freqs; extern struct freq_attr cpufreq_freq_attr_scaling_boost_freqs; --=20 2.34.1 From nobody Fri Jan 2 19:07:44 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id CFDF3E95A97 for ; Mon, 9 Oct 2023 10:36:47 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1346096AbjJIKgq (ORCPT ); Mon, 9 Oct 2023 06:36:46 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47078 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1345885AbjJIKge (ORCPT ); Mon, 9 Oct 2023 06:36:34 -0400 Received: from mail-wr1-x434.google.com (mail-wr1-x434.google.com [IPv6:2a00:1450:4864:20::434]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 57BABCA for ; Mon, 9 Oct 2023 03:36:32 -0700 (PDT) Received: by mail-wr1-x434.google.com with SMTP id ffacd0b85a97d-3248ac76acbso3945283f8f.1 for ; Mon, 09 Oct 2023 03:36:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1696847791; x=1697452591; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=RmtdGUvSawQ4q9KDGPlbeId1/iTLhEjjDMVIwSogQm4=; b=BoQxuiosFjVPZt8rIcS2jhV7/cOVSW3fBHkNztfizdSO555RdCP+O62fPV2dAzkqy3 6gRLD5DdtjvXAlT5j5SNvqku6Bk0Y8KRTu3YOoYmj1H0+99gVGSo8YR/kpP1k2uetAOH GJEU8vOxXy7MYI0OPlghIoPpyMigXVv9+MyrEuqzAu9C8e/35KffZ03iMcvtsmTIIqGY DXz/y9Tg1YzPNZYIHRtPJrVL8QwSH/AbYuU2EvsKfPjFG2IolWIa8VE+KMioqA67CHzR sFOohL1SEGzZrUVFI0I3QF0DP0GXG2+c2HM5TNk/HMXxtKfyObT/vhE2Rm+C8mp/c70w vPRA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1696847791; x=1697452591; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=RmtdGUvSawQ4q9KDGPlbeId1/iTLhEjjDMVIwSogQm4=; b=De9DIygEBMEVh3umj6P05h6wSJV4bCncKFvCDc8HmZ+Ct3jaNHYaPQE/gb3S+wUK2S f4sELo3zxE//WOHund/h3/yseG6oGoWmRp9Y/CgHeqbKhtFUlgbCxCY6bNST9DMn+VpF +Wl5e+zuEMCIzA1haa3bvw6cFoPAIG8CvAPclpBPQ1Hsxgwinp88hgBQE2Xauy3vJp0Z jgWs+3QVpvzU0l0kKZkoNS3e+lBEuYjeFMQ//92s8feBpMeqcT//pkSHodu7wx53vwg1 rJlk4j301KnsnMFKN/6JBuiMfqt7sRgFMLweVoHvCJm5e99s7HmAorxYOE6YUujRsHfS jhKg== X-Gm-Message-State: AOJu0YwDArDJRZUBaZ4QF6h1hhyqfoBXEDk1GW6X4kfXXBVffEg28FzW VKZInDVlh7fs8kkmpYpJ9DdvCw== X-Google-Smtp-Source: AGHT+IFsDHg/Rs8MQPYQNA1a7DXBKSokGpRKvXSJgz2PNurvi4j34eIaCQtgEUCXe0uK4YQzK2pDKA== X-Received: by 2002:a05:6000:91:b0:320:67:5580 with SMTP id m17-20020a056000009100b0032000675580mr12443168wrx.5.1696847790816; Mon, 09 Oct 2023 03:36:30 -0700 (PDT) Received: from vingu-book.. ([2a01:e0a:f:6020:53f1:24bc:5e47:821d]) by smtp.gmail.com with ESMTPSA id f16-20020adfdb50000000b0031ff89af0e4sm9226722wrj.99.2023.10.09.03.36.29 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 09 Oct 2023 03:36:30 -0700 (PDT) From: Vincent Guittot To: linux@armlinux.org.uk, catalin.marinas@arm.com, will@kernel.org, paul.walmsley@sifive.com, palmer@dabbelt.com, aou@eecs.berkeley.edu, sudeep.holla@arm.com, gregkh@linuxfoundation.org, rafael@kernel.org, mingo@redhat.com, peterz@infradead.org, juri.lelli@redhat.com, dietmar.eggemann@arm.com, rostedt@goodmis.org, bsegall@google.com, mgorman@suse.de, bristot@redhat.com, vschneid@redhat.com, viresh.kumar@linaro.org, lukasz.luba@arm.com, ionela.voinescu@arm.com, pierre.gondois@arm.com, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org, linux-pm@vger.kernel.org Cc: conor.dooley@microchip.com, suagrfillet@gmail.com, ajones@ventanamicro.com, lftan@kernel.org, Vincent Guittot Subject: [PATCH v2 4/6] cpufreq/schedutil: use a fixed reference frequency Date: Mon, 9 Oct 2023 12:36:19 +0200 Message-Id: <20231009103621.374412-5-vincent.guittot@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20231009103621.374412-1-vincent.guittot@linaro.org> References: <20231009103621.374412-1-vincent.guittot@linaro.org> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" cpuinfo.max_freq can change at runtime because of boost as an example. This implies that the value could be different than the one that has been used when computing the capacity of a CPU. The new arch_scale_freq_ref() returns a fixed and coherent reference frequency that can be used when computing a frequency based on utilization. Use this arch_scale_freq_ref() when available and fallback to policy otherwise. Signed-off-by: Vincent Guittot Reviewed-by: Lukasz Luba Tested-by: Lukasz Luba --- kernel/sched/cpufreq_schedutil.c | 26 ++++++++++++++++++++++++-- 1 file changed, 24 insertions(+), 2 deletions(-) diff --git a/kernel/sched/cpufreq_schedutil.c b/kernel/sched/cpufreq_schedu= til.c index 4492608b7d7f..1fa7e74add8f 100644 --- a/kernel/sched/cpufreq_schedutil.c +++ b/kernel/sched/cpufreq_schedutil.c @@ -114,6 +114,28 @@ static void sugov_deferred_update(struct sugov_policy = *sg_policy) } } =20 +/** + * cpufreq_get_capacity_ref_freq - get the reference frequency of a given = CPU that + * has been used to correlate frequency and compute capacity. + * @policy: the cpufreq policy of the CPU in question. + * @use_current: Fallback to current freq instead of policy->cpuinfo.max_f= req. + * + * Return: the reference CPU frequency to compute a capacity. + */ +static __always_inline +unsigned long get_capacity_ref_freq(struct cpufreq_policy *policy) +{ + unsigned int freq =3D arch_scale_freq_ref(policy->cpu); + + if (freq) + return freq; + + if (arch_scale_freq_invariant()) + return policy->cpuinfo.max_freq; + + return policy->cur; +} + /** * get_next_freq - Compute a new frequency for a given cpufreq policy. * @sg_policy: schedutil policy object to compute the new frequency for. @@ -140,10 +162,10 @@ static unsigned int get_next_freq(struct sugov_policy= *sg_policy, unsigned long util, unsigned long max) { struct cpufreq_policy *policy =3D sg_policy->policy; - unsigned int freq =3D arch_scale_freq_invariant() ? - policy->cpuinfo.max_freq : policy->cur; + unsigned int freq; =20 util =3D map_util_perf(util); + freq =3D get_capacity_ref_freq(policy); freq =3D map_util_freq(util, freq, max); =20 if (freq =3D=3D sg_policy->cached_raw_freq && !sg_policy->need_freq_updat= e) --=20 2.34.1 From nobody Fri Jan 2 19:07:44 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id BC094E95A8E for ; Mon, 9 Oct 2023 10:36:50 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1346118AbjJIKgt (ORCPT ); Mon, 9 Oct 2023 06:36:49 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47068 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1345916AbjJIKgf (ORCPT ); Mon, 9 Oct 2023 06:36:35 -0400 Received: from mail-wr1-x42f.google.com (mail-wr1-x42f.google.com [IPv6:2a00:1450:4864:20::42f]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id CF3B2DB for ; Mon, 9 Oct 2023 03:36:33 -0700 (PDT) Received: by mail-wr1-x42f.google.com with SMTP id ffacd0b85a97d-325e9cd483eso4181121f8f.2 for ; Mon, 09 Oct 2023 03:36:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1696847792; x=1697452592; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=FXD0QUw+ffX/G//mljsP8uVRxSPaNtugfzKgqO7nW9U=; b=hnyiZ8kx7BATKD47TcsSy9fXoBlQiTLbElApM1luHBCfrVTchy3UUaK2iIBYlMQgFU k/TqLI7PtsnQNeNUPMZdf9ESFq8DoMy/mhlSehvtVc/NUYLTt6gehjOPQjYX3+jXGhzY PnF9eMW5fHW7eTxgrzlJ2CQ3ppyzfISqPHzJUW1KyrMLooc04n+SQb0nyRp9r+Ysc3fO 1Tn6QhMDa578reGzRaW71apGOiFvc5X4+yUooK6TtrsqKHzb2okpmj5Jx+qrplm7Hepg JfixR2uoI842zbNcRRV+7RWaXNoI6UO+UubLfm7feyxF5d8XOPj3iydEjojuDMvNxfFd BF5w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1696847792; x=1697452592; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=FXD0QUw+ffX/G//mljsP8uVRxSPaNtugfzKgqO7nW9U=; b=eR3YtwBXLzdpcIeacPvRHp/QbA3JmCAEiSALQCULQQT0pFtOZ7oZoxiZPlwzz95BZA ELQpbFxjddPup1GGo13UhWh5TB6G6jUvrxka2MEqPKLatOc2Fr/FST3pygbI7ySXLuK/ EVVBdH41tHT4jiGO/rsYAqWbLenUHwTxonVozYPguiQzK38Vy+GtRMnOVMb7xfTuloUq DqdPSx077tjvsCoYbg1NtvwJiIFUfqX+ZD6N0bXf4NVhH5tqCKi0q1YyC3/h9dxBEM7G dHMV7fQ9heWdrW60kZjZ6U3Mqi/TCtj9zCcy43YMG/yVDMAZNQUFaV6bXwpPGce/anrw 2gpg== X-Gm-Message-State: AOJu0YwqWeYmJ3DERiC21KhEjVzqjBB3pzbrzNdmKk3qOFG6Alx5vqID 9MLhePvnSGukBwqLJ0w3Q1+mXA== X-Google-Smtp-Source: AGHT+IHr3aX+9E7V3PEwqGZuQkX9W7nIP+EyPQGbKJ+lRYHFakcvBtj79RtXlZzvZ8DwRc3af55fNw== X-Received: by 2002:adf:ed10:0:b0:320:933:4524 with SMTP id a16-20020adfed10000000b0032009334524mr13404865wro.2.1696847792183; Mon, 09 Oct 2023 03:36:32 -0700 (PDT) Received: from vingu-book.. ([2a01:e0a:f:6020:53f1:24bc:5e47:821d]) by smtp.gmail.com with ESMTPSA id f16-20020adfdb50000000b0031ff89af0e4sm9226722wrj.99.2023.10.09.03.36.30 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 09 Oct 2023 03:36:31 -0700 (PDT) From: Vincent Guittot To: linux@armlinux.org.uk, catalin.marinas@arm.com, will@kernel.org, paul.walmsley@sifive.com, palmer@dabbelt.com, aou@eecs.berkeley.edu, sudeep.holla@arm.com, gregkh@linuxfoundation.org, rafael@kernel.org, mingo@redhat.com, peterz@infradead.org, juri.lelli@redhat.com, dietmar.eggemann@arm.com, rostedt@goodmis.org, bsegall@google.com, mgorman@suse.de, bristot@redhat.com, vschneid@redhat.com, viresh.kumar@linaro.org, lukasz.luba@arm.com, ionela.voinescu@arm.com, pierre.gondois@arm.com, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org, linux-pm@vger.kernel.org Cc: conor.dooley@microchip.com, suagrfillet@gmail.com, ajones@ventanamicro.com, lftan@kernel.org, Vincent Guittot Subject: [PATCH v2 5/6] energy_model: use a fixed reference frequency Date: Mon, 9 Oct 2023 12:36:20 +0200 Message-Id: <20231009103621.374412-6-vincent.guittot@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20231009103621.374412-1-vincent.guittot@linaro.org> References: <20231009103621.374412-1-vincent.guittot@linaro.org> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" The last item of a performance domain is not always the performance point that has been used to compute CPU's capacity. This can lead to different target frequency compared with other part of the system like schedutil and would result in wrong energy estimation. A new arch_scale_freq_ref() is available to return a fixed and coherent frequency reference that can be used when computing the CPU's frequency for an level of utilization. Use this function to get this reference frequency. Energy model is never used without defining arch_scale_freq_ref() but can be compiled. Define a default arch_scale_freq_ref() returning 0 in such case. Signed-off-by: Vincent Guittot Reviewed-by: Lukasz Luba Tested-by: Lukasz Luba --- include/linux/energy_model.h | 14 +++++++++++--- 1 file changed, 11 insertions(+), 3 deletions(-) diff --git a/include/linux/energy_model.h b/include/linux/energy_model.h index b9caa01dfac4..1b0c8490d4bd 100644 --- a/include/linux/energy_model.h +++ b/include/linux/energy_model.h @@ -204,6 +204,14 @@ struct em_perf_state *em_pd_get_efficient_state(struct= em_perf_domain *pd, return ps; } =20 +#ifndef arch_scale_freq_ref +static __always_inline +unsigned int arch_scale_freq_ref(int cpu) +{ + return 0; +} +#endif + /** * em_cpu_energy() - Estimates the energy consumed by the CPUs of a * performance domain @@ -224,7 +232,7 @@ static inline unsigned long em_cpu_energy(struct em_per= f_domain *pd, unsigned long max_util, unsigned long sum_util, unsigned long allowed_cpu_cap) { - unsigned long freq, scale_cpu; + unsigned long freq, ref_freq, scale_cpu; struct em_perf_state *ps; int cpu; =20 @@ -241,11 +249,11 @@ static inline unsigned long em_cpu_energy(struct em_p= erf_domain *pd, */ cpu =3D cpumask_first(to_cpumask(pd->cpus)); scale_cpu =3D arch_scale_cpu_capacity(cpu); - ps =3D &pd->table[pd->nr_perf_states - 1]; + ref_freq =3D arch_scale_freq_ref(cpu); =20 max_util =3D map_util_perf(max_util); max_util =3D min(max_util, allowed_cpu_cap); - freq =3D map_util_freq(max_util, ps->frequency, scale_cpu); + freq =3D map_util_freq(max_util, ref_freq, scale_cpu); =20 /* * Find the lowest performance state of the Energy Model above the --=20 2.34.1 From nobody Fri Jan 2 19:07:44 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id AC31FE95A91 for ; Mon, 9 Oct 2023 10:36:54 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1346147AbjJIKgx (ORCPT ); Mon, 9 Oct 2023 06:36:53 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:34708 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1345937AbjJIKgh (ORCPT ); Mon, 9 Oct 2023 06:36:37 -0400 Received: from mail-wr1-x42f.google.com (mail-wr1-x42f.google.com [IPv6:2a00:1450:4864:20::42f]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 2AC8CAB for ; Mon, 9 Oct 2023 03:36:35 -0700 (PDT) Received: by mail-wr1-x42f.google.com with SMTP id ffacd0b85a97d-3296b49c546so2468783f8f.3 for ; Mon, 09 Oct 2023 03:36:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1696847793; x=1697452593; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=zLqmJToKVvGx3RPF93C/D7OTDNmDiwUSULcdREcteNg=; b=V4ndD/FvTN8lDLyDZ+iL1ssnIQXGKkAxLC7Gh+qcxuxRwg/FhAD/TVT73fKADskQrU 1cveajp1LH72QU65QuUopll3bPyK9bDBOJzUAmTmoa3nZluUL7vesFFIU+SQQR/KkuPr 8m08QwvP9FKNhgHIVeNVG2Tt0F7BqhbJHr0F7duxV63YusafmwMjcKyO6ZK3axtUsZqo w6XTcqnQ+3blMZxf4mH+kQlo60LS+gNWeoaOCXY2kmIQFg75gPCpMyc+NfeFtkhM8ijY l4L5NejI8U/JD8NdUPBtpAmSVxw4F46tlb5HhFn7m+0/Rs8dWJz1T9wSMBZ6qSoOEcyH csog== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1696847793; x=1697452593; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=zLqmJToKVvGx3RPF93C/D7OTDNmDiwUSULcdREcteNg=; b=ZGO7PZ2oGLs5ubx6SVJG+mq9gt9rYJ29u9RmN/K4mq624XpUT6MKl8oiy2lBtfCj6W EQdyRrpx3x4H/6TiI19oB4i0ZtxLbxpI+PtlcKrh6vzwctg0DeMMEquEjZvTvQdmyK2k 6YYo/0HAkF4U4gAoX/k2iI13Vvv9u2y8HS/4CRiD7Xg6zvEjHT1XoMNoC2A5yrWCr/Rz t1wehAtRGcfYb+Ghft4WwayuvDa5FcWw4S1IfGMQnP98z545QxtQw4QRHmMeZhie38KZ wXJItXurLVjCsnoJRI4vjk13TNNMmKfswezZJCyI5bmsQap81BsGF24UGp4CEPeQc6NU wQow== X-Gm-Message-State: AOJu0Yzwge8SNJRRMUOvH67RFqfl6RFotMlo0IeoY4EImMaP0kOFjgNF /a+pQUM/cSrEZ40pEc0amZsyNw== X-Google-Smtp-Source: AGHT+IE1c9bbdHMwBLMomM9sWbKgUcq5hfVG4KlvDcosRRJkN4Ti2TteiXopQtpHeenYl1QcJ2PxJw== X-Received: by 2002:a5d:614b:0:b0:31c:8c93:61e3 with SMTP id y11-20020a5d614b000000b0031c8c9361e3mr12270603wrt.60.1696847793608; Mon, 09 Oct 2023 03:36:33 -0700 (PDT) Received: from vingu-book.. ([2a01:e0a:f:6020:53f1:24bc:5e47:821d]) by smtp.gmail.com with ESMTPSA id f16-20020adfdb50000000b0031ff89af0e4sm9226722wrj.99.2023.10.09.03.36.32 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 09 Oct 2023 03:36:33 -0700 (PDT) From: Vincent Guittot To: linux@armlinux.org.uk, catalin.marinas@arm.com, will@kernel.org, paul.walmsley@sifive.com, palmer@dabbelt.com, aou@eecs.berkeley.edu, sudeep.holla@arm.com, gregkh@linuxfoundation.org, rafael@kernel.org, mingo@redhat.com, peterz@infradead.org, juri.lelli@redhat.com, dietmar.eggemann@arm.com, rostedt@goodmis.org, bsegall@google.com, mgorman@suse.de, bristot@redhat.com, vschneid@redhat.com, viresh.kumar@linaro.org, lukasz.luba@arm.com, ionela.voinescu@arm.com, pierre.gondois@arm.com, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org, linux-pm@vger.kernel.org Cc: conor.dooley@microchip.com, suagrfillet@gmail.com, ajones@ventanamicro.com, lftan@kernel.org, Vincent Guittot Subject: [PATCH v2 6/6] cpufreq/cppc: set the frequency used for capacity computation Date: Mon, 9 Oct 2023 12:36:21 +0200 Message-Id: <20231009103621.374412-7-vincent.guittot@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20231009103621.374412-1-vincent.guittot@linaro.org> References: <20231009103621.374412-1-vincent.guittot@linaro.org> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" cppc cpufreq driver can register an artificial energy model. In such case, it also have to register the frequency that is used to define the CPU capacity Signed-off-by: Vincent Guittot Tested-by: Lukasz Luba --- drivers/cpufreq/cppc_cpufreq.c | 18 ++++++++++++++++++ 1 file changed, 18 insertions(+) diff --git a/drivers/cpufreq/cppc_cpufreq.c b/drivers/cpufreq/cppc_cpufreq.c index fe08ca419b3d..24c6ba349f01 100644 --- a/drivers/cpufreq/cppc_cpufreq.c +++ b/drivers/cpufreq/cppc_cpufreq.c @@ -636,6 +636,21 @@ static int populate_efficiency_class(void) return 0; } =20 + +static void cppc_cpufreq_set_capacity_ref_freq(struct cpufreq_policy *poli= cy) +{ + struct cppc_perf_caps *perf_caps; + struct cppc_cpudata *cpu_data; + unsigned int ref_freq; + + cpu_data =3D policy->driver_data; + perf_caps =3D &cpu_data->perf_caps; + + ref_freq =3D cppc_cpufreq_perf_to_khz(cpu_data, perf_caps->highest_perf); + + per_cpu(capacity_ref_freq, policy->cpu) =3D ref_freq; +} + static void cppc_cpufreq_register_em(struct cpufreq_policy *policy) { struct cppc_cpudata *cpu_data; @@ -643,6 +658,9 @@ static void cppc_cpufreq_register_em(struct cpufreq_pol= icy *policy) EM_ADV_DATA_CB(cppc_get_cpu_power, cppc_get_cpu_cost); =20 cpu_data =3D policy->driver_data; + + cppc_cpufreq_set_capacity_ref_freq(policy); + em_dev_register_perf_domain(get_cpu_device(policy->cpu), get_perf_level_count(policy), &em_cb, cpu_data->shared_cpu_map, 0); --=20 2.34.1