From nobody Wed Dec 17 11:00:48 2025 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 9523757CAB for ; Mon, 3 Jun 2024 08:22:12 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=217.140.110.172 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1717402933; cv=none; b=F0pPWskCNB7ys+0oelMv0T99yNVkAMH5HoHBsfgbDyDxjuV2kiJoQ3lVlJIYiZ94I773ZNbvb2OXO5mBj03DLu1O68tGVWjvLIv27RKrXIc4odY48m2IkgFoT6MKHL4CdP2yOIQ7GOAokQvURWY/AMmsdKXJlquU/w01nFWfW1w= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1717402933; c=relaxed/simple; bh=2eSMXYfYYs6HArv4V6/WQUuvgtIzkW1Gs3FiPPKAfJY=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=OC2SwPAxlvxcw1LukBwy3lLq4aPdsuqhZxflRzXKrwWbkRCAsoQgYkDrfhnMfVOUN9TzUS/KMhBufNxj8CYadFO968ACW6lMi6jaXzvZ3TXI/JCtND7ibYT2Hi2sYgKcxj7V58HG7vRN2NjGT8RZNnBrqRlp6rW0hyFdgJJxwCo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com; spf=pass smtp.mailfrom=arm.com; arc=none smtp.client-ip=217.140.110.172 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 4D70C1063; Mon, 3 Jun 2024 01:22:36 -0700 (PDT) Received: from e125905.cambridge.arm.com (e125905.cambridge.arm.com [10.1.194.73]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 4FDEC3F792; Mon, 3 Jun 2024 01:22:10 -0700 (PDT) From: Beata Michalska To: linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, ionela.voinescu@arm.com, sudeep.holla@arm.com, will@kernel.org, catalin.marinas@arm.com, vincent.guittot@linaro.org, vanshikonda@os.amperecomputing.com Cc: sumitg@nvidia.com, yang@os.amperecomputing.com, lihuisong@huawei.com, viresh.kumar@linaro.org, rafael@kernel.org Subject: [PATCH v6 1/4] arch_topology: init capacity_freq_ref to 0 Date: Mon, 3 Jun 2024 09:21:51 +0100 Message-Id: <20240603082154.3830591-2-beata.michalska@arm.com> In-Reply-To: <20240603082154.3830591-1-beata.michalska@arm.com> References: <20240603082154.3830591-1-beata.michalska@arm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Ionela Voinescu It's useful to have capacity_freq_ref initialized to 0 for users of arch_scale_freq_ref() to detect when capacity_freq_ref was not yet set. The only scenario affected by this change in the init value is when a cpufreq driver is never loaded. As a result, the only setter of a cpu scale factor remains the call of topology_normalize_cpu_scale() from parse_dt_topology(). There we cannot use the value 0 of capacity_freq_ref so we have to compensate for its uninitialized state. Signed-off-by: Ionela Voinescu Signed-off-by: Beata Michalska Reviewed-by: Vincent Guittot Reviewed-by: Sudeep Holla --- drivers/base/arch_topology.c | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/drivers/base/arch_topology.c b/drivers/base/arch_topology.c index c66d070207a0..b8217ce9082c 100644 --- a/drivers/base/arch_topology.c +++ b/drivers/base/arch_topology.c @@ -27,7 +27,7 @@ static DEFINE_PER_CPU(struct scale_freq_data __rcu *, sft_data); static struct cpumask scale_freq_counters_mask; static bool scale_freq_invariant; -DEFINE_PER_CPU(unsigned long, capacity_freq_ref) =3D 1; +DEFINE_PER_CPU(unsigned long, capacity_freq_ref) =3D 0; EXPORT_PER_CPU_SYMBOL_GPL(capacity_freq_ref); =20 static bool supports_scale_freq_counters(const struct cpumask *cpus) @@ -292,13 +292,15 @@ void topology_normalize_cpu_scale(void) =20 capacity_scale =3D 1; for_each_possible_cpu(cpu) { - capacity =3D raw_capacity[cpu] * per_cpu(capacity_freq_ref, cpu); + capacity =3D raw_capacity[cpu] * + (per_cpu(capacity_freq_ref, cpu) ?: 1); capacity_scale =3D max(capacity, capacity_scale); } =20 pr_debug("cpu_capacity: capacity_scale=3D%llu\n", capacity_scale); for_each_possible_cpu(cpu) { - capacity =3D raw_capacity[cpu] * per_cpu(capacity_freq_ref, cpu); + capacity =3D raw_capacity[cpu] * + (per_cpu(capacity_freq_ref, cpu) ?: 1); capacity =3D div64_u64(capacity << SCHED_CAPACITY_SHIFT, capacity_scale); topology_set_cpu_scale(cpu, capacity); --=20 2.25.1 From nobody Wed Dec 17 11:00:48 2025 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 6A8C15BAC3 for ; Mon, 3 Jun 2024 08:22:14 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=217.140.110.172 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1717402936; cv=none; b=ZQcWmutcUCNYD5MDv17GR+e+ndU1bgIhOWOKpMqiacfT6n1rAAwzggKxD5EjmfJiqGdE/AlIUKxeHiA97TVm3aUVxSx1yZ/njHpZSc1m6jRcLNIET3Pl3uzrpD6vgAiWzlxLFDeMwlgAeXGLeAgtSLTX8kv7VDz6+PdwniORCw0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1717402936; c=relaxed/simple; bh=mcByuKxJJ46kPM0AYKATs2LizIwbORX/DJJd+bwMDLQ=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=DYtCXWASA+dxWVXuUNj7Gea8fZwn8VYtYY6GlnE4B+La5K4m1iYy8kYYPlQiH2Mr4/yHJyPpGKSIScyfEg/rVXI0Q13PLIoUXcQSm8yyOdDmfRHSbVGi2ziKo6XjX3MLCItkYt8pIRYRcwHLYxU2exzpjcTtg0jYr8aBwmzymP4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com; spf=pass smtp.mailfrom=arm.com; arc=none smtp.client-ip=217.140.110.172 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 3541A106F; Mon, 3 Jun 2024 01:22:38 -0700 (PDT) Received: from e125905.cambridge.arm.com (e125905.cambridge.arm.com [10.1.194.73]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 37BE93F792; Mon, 3 Jun 2024 01:22:12 -0700 (PDT) From: Beata Michalska To: linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, ionela.voinescu@arm.com, sudeep.holla@arm.com, will@kernel.org, catalin.marinas@arm.com, vincent.guittot@linaro.org, vanshikonda@os.amperecomputing.com Cc: sumitg@nvidia.com, yang@os.amperecomputing.com, lihuisong@huawei.com, viresh.kumar@linaro.org, rafael@kernel.org Subject: [PATCH v6 2/4] arm64: amu: Delay allocating cpumask for AMU FIE support Date: Mon, 3 Jun 2024 09:21:52 +0100 Message-Id: <20240603082154.3830591-3-beata.michalska@arm.com> In-Reply-To: <20240603082154.3830591-1-beata.michalska@arm.com> References: <20240603082154.3830591-1-beata.michalska@arm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" For the time being, the amu_fie_cpus cpumask is being exclusively used by the AMU-related internals of FIE support and is guaranteed to be valid on every access currently made. Still the mask is not being invalidated on one of the error handling code paths, which leaves a soft spot with theoretical risk of uaf for CPUMASK_OFFSTACK cases. To make things sound, delay allocating said cpumask (for CPUMASK_OFFSTACK) avoiding otherwise nasty sanitising case failing to register the cpufreq policy notifications. Signed-off-by: Beata Michalska --- arch/arm64/kernel/topology.c | 22 +++++++++------------- 1 file changed, 9 insertions(+), 13 deletions(-) diff --git a/arch/arm64/kernel/topology.c b/arch/arm64/kernel/topology.c index 1a2c72f3e7f8..e475ec2705e1 100644 --- a/arch/arm64/kernel/topology.c +++ b/arch/arm64/kernel/topology.c @@ -194,11 +194,16 @@ static void amu_fie_setup(const struct cpumask *cpus) int cpu; =20 /* We are already set since the last insmod of cpufreq driver */ - if (unlikely(cpumask_subset(cpus, amu_fie_cpus))) - return; + if (cpumask_available(amu_fie_cpus) && + unlikely(cpumask_subset(cpus, amu_fie_cpus))) + return; =20 for_each_cpu(cpu, cpus) { - if (!freq_counters_valid(cpu)) + if (!cpumask_available(amu_fie_cpus) && + !zalloc_cpumask_var(&amu_fie_cpus, GFP_KERNEL)) + return; + + if (!freq_counters_valid(cpu)) return; } =20 @@ -237,17 +242,8 @@ static struct notifier_block init_amu_fie_notifier =3D= { =20 static int __init init_amu_fie(void) { - int ret; - - if (!zalloc_cpumask_var(&amu_fie_cpus, GFP_KERNEL)) - return -ENOMEM; - - ret =3D cpufreq_register_notifier(&init_amu_fie_notifier, + return cpufreq_register_notifier(&init_amu_fie_notifier, CPUFREQ_POLICY_NOTIFIER); - if (ret) - free_cpumask_var(amu_fie_cpus); - - return ret; } core_initcall(init_amu_fie); =20 --=20 2.25.1 From nobody Wed Dec 17 11:00:48 2025 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 613516D1B5 for ; Mon, 3 Jun 2024 08:22:16 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=217.140.110.172 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1717402937; cv=none; b=h3LqKPCEujbXPtDBtkPD/g3IQPInCb8HjpLnH0Fcs4CHZHF4y3yS0a7Lmqh5OQezNWZV+puxr1wHebLR9Pm6B8uSKF4wunqlT05aMO2KmVydWTOdyoBxw3yzOWAih5jv901B/6aXgs2C8fnEfRw9MVymnZGvH4wbw83ZmKA/OYw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1717402937; c=relaxed/simple; bh=vNnXNDcnk0jg+rsIxxjUuCFZ3eXPKDcEJ9hSB4piQV8=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=i4Q0Mn+J9ZyvaomKCLr/Fq+mhVOiYni/ZgUG9OecUx/5D6d+MvF3VtUbXhjJJdErcCwtE8Lx3DNTJy4HAbR7grf6EodisLNXG63HwpHvikE71JIV8QFJxyyMCF8cKyRMg+xrLHAGQriepY0vfmsiLmA53nNtKU2cQcoPN/1riDo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com; spf=pass smtp.mailfrom=arm.com; arc=none smtp.client-ip=217.140.110.172 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 1D6AA11FB; Mon, 3 Jun 2024 01:22:40 -0700 (PDT) Received: from e125905.cambridge.arm.com (e125905.cambridge.arm.com [10.1.194.73]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 1FE873F792; Mon, 3 Jun 2024 01:22:14 -0700 (PDT) From: Beata Michalska To: linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, ionela.voinescu@arm.com, sudeep.holla@arm.com, will@kernel.org, catalin.marinas@arm.com, vincent.guittot@linaro.org, vanshikonda@os.amperecomputing.com Cc: sumitg@nvidia.com, yang@os.amperecomputing.com, lihuisong@huawei.com, viresh.kumar@linaro.org, rafael@kernel.org Subject: [PATCH v6 3/4] arm64: Provide an AMU-based version of arch_freq_get_on_cpu Date: Mon, 3 Jun 2024 09:21:53 +0100 Message-Id: <20240603082154.3830591-4-beata.michalska@arm.com> In-Reply-To: <20240603082154.3830591-1-beata.michalska@arm.com> References: <20240603082154.3830591-1-beata.michalska@arm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" With the Frequency Invariance Engine (FIE) being already wired up with sched tick and making use of relevant (core counter and constant counter) AMU counters, getting the current frequency for a given CPU, can be achieved by utilizing the frequency scale factor which reflects an average CPU frequency for the last tick period length. The solution is partially based on APERF/MPERF implementation of arch_freq_get_on_cpu. Suggested-by: Ionela Voinescu Signed-off-by: Beata Michalska --- arch/arm64/kernel/topology.c | 110 +++++++++++++++++++++++++++++++---- 1 file changed, 100 insertions(+), 10 deletions(-) diff --git a/arch/arm64/kernel/topology.c b/arch/arm64/kernel/topology.c index e475ec2705e1..2c002d2c3e0b 100644 --- a/arch/arm64/kernel/topology.c +++ b/arch/arm64/kernel/topology.c @@ -17,6 +17,7 @@ #include #include #include +#include =20 #include #include @@ -88,18 +89,28 @@ int __init parse_acpi_topology(void) * initialized. */ static DEFINE_PER_CPU_READ_MOSTLY(unsigned long, arch_max_freq_scale) =3D = 1UL << (2 * SCHED_CAPACITY_SHIFT); -static DEFINE_PER_CPU(u64, arch_const_cycles_prev); -static DEFINE_PER_CPU(u64, arch_core_cycles_prev); static cpumask_var_t amu_fie_cpus; =20 +struct amu_cntr_sample { + u64 arch_const_cycles_prev; + u64 arch_core_cycles_prev; + unsigned long last_update; +}; + +static DEFINE_PER_CPU_SHARED_ALIGNED(struct amu_cntr_sample, cpu_amu_sampl= es); + void update_freq_counters_refs(void) { - this_cpu_write(arch_core_cycles_prev, read_corecnt()); - this_cpu_write(arch_const_cycles_prev, read_constcnt()); + struct amu_cntr_sample *amu_sample =3D this_cpu_ptr(&cpu_amu_samples); + + amu_sample->arch_core_cycles_prev =3D read_corecnt(); + amu_sample->arch_const_cycles_prev =3D read_constcnt(); } =20 static inline bool freq_counters_valid(int cpu) { + struct amu_cntr_sample *amu_sample =3D per_cpu_ptr(&cpu_amu_samples, cpu); + if ((cpu >=3D nr_cpu_ids) || !cpumask_test_cpu(cpu, cpu_present_mask)) return false; =20 @@ -108,8 +119,8 @@ static inline bool freq_counters_valid(int cpu) return false; } =20 - if (unlikely(!per_cpu(arch_const_cycles_prev, cpu) || - !per_cpu(arch_core_cycles_prev, cpu))) { + if (unlikely(!amu_sample->arch_const_cycles_prev || + !amu_sample->arch_core_cycles_prev)) { pr_debug("CPU%d: cycle counters are not enabled.\n", cpu); return false; } @@ -152,17 +163,22 @@ void freq_inv_set_max_ratio(int cpu, u64 max_rate) =20 static void amu_scale_freq_tick(void) { + struct amu_cntr_sample *amu_sample =3D this_cpu_ptr(&cpu_amu_samples); u64 prev_core_cnt, prev_const_cnt; u64 core_cnt, const_cnt, scale; =20 - prev_const_cnt =3D this_cpu_read(arch_const_cycles_prev); - prev_core_cnt =3D this_cpu_read(arch_core_cycles_prev); + prev_const_cnt =3D amu_sample->arch_const_cycles_prev; + prev_core_cnt =3D amu_sample->arch_core_cycles_prev; =20 update_freq_counters_refs(); =20 - const_cnt =3D this_cpu_read(arch_const_cycles_prev); - core_cnt =3D this_cpu_read(arch_core_cycles_prev); + const_cnt =3D amu_sample->arch_const_cycles_prev; + core_cnt =3D amu_sample->arch_core_cycles_prev; =20 + /* + * This should not happen unless the AMUs have been reset and the + * counter values have not been restored - unlikely + */ if (unlikely(core_cnt <=3D prev_core_cnt || const_cnt <=3D prev_const_cnt)) return; @@ -182,6 +198,8 @@ static void amu_scale_freq_tick(void) =20 scale =3D min_t(unsigned long, scale, SCHED_CAPACITY_SCALE); this_cpu_write(arch_freq_scale, (unsigned long)scale); + + amu_sample->last_update =3D jiffies; } =20 static struct scale_freq_data amu_sfd =3D { @@ -189,6 +207,78 @@ static struct scale_freq_data amu_sfd =3D { .set_freq_scale =3D amu_scale_freq_tick, }; =20 +static __always_inline bool amu_fie_cpu_supported(unsigned int cpu) +{ + return cpumask_available(amu_fie_cpus) && + cpumask_test_cpu(cpu, amu_fie_cpus); +} + +#define AMU_SAMPLE_EXP_MS 20 + +unsigned int arch_freq_get_on_cpu(int cpu) +{ + struct amu_cntr_sample *amu_sample; + unsigned int start_cpu =3D cpu; + unsigned long last_update; + unsigned int freq =3D 0; + u64 scale; + + if (!amu_fie_cpu_supported(cpu) || !arch_scale_freq_ref(cpu)) + return 0; + +retry: + amu_sample =3D per_cpu_ptr(&cpu_amu_samples, cpu); + + last_update =3D amu_sample->last_update; + + /* + * For those CPUs that are in full dynticks mode, + * and those that have not seen tick for a while + * try an alternative source for the counters (and thus freq scale), + * if available, for given policy: + * this boils down to identifying an active cpu within the same freq + * domain, if any. + */ + if (!housekeeping_cpu(cpu, HK_TYPE_TICK) || + time_is_before_jiffies(last_update + msecs_to_jiffies(AMU_SAMPLE_EXP_= MS))) { + struct cpufreq_policy *policy =3D cpufreq_cpu_get(cpu); + int ref_cpu =3D cpu; + + if (!policy) + goto leave; + + if (!policy_is_shared(policy)) { + cpufreq_cpu_put(policy); + goto leave; + } + + do { + ref_cpu =3D cpumask_next_wrap(ref_cpu, policy->cpus, + start_cpu, false); + + } while (ref_cpu < nr_cpu_ids && idle_cpu(ref_cpu)); + + cpufreq_cpu_put(policy); + + if (ref_cpu >=3D nr_cpu_ids) + /* No alternative to pull info from */ + goto leave; + + cpu =3D ref_cpu; + goto retry; + } + /* + * Reversed computation to the one used to determine + * the arch_freq_scale value + * (see amu_scale_freq_tick for details) + */ + scale =3D arch_scale_freq_capacity(cpu); + freq =3D scale * arch_scale_freq_ref(cpu); + freq >>=3D SCHED_CAPACITY_SHIFT; +leave: + return freq; +} + static void amu_fie_setup(const struct cpumask *cpus) { int cpu; --=20 2.25.1 From nobody Wed Dec 17 11:00:48 2025 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 4C79F7829C for ; Mon, 3 Jun 2024 08:22:17 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=217.140.110.172 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1717402939; cv=none; b=J0l1c6aIfaQ8VozRVQxh1lRNO1t4rFLdDg6rcRRmnGjwf/MY2Zd+j3DhHXQQCasJtwHhYLDT8JCFypMyTS7dIy5hY7I7QcpoR7OVo75SKLMdR8GgaBiTLnMPpMk0CWzO2f5eSU5nQKVXLdMBwL3J33+um0IKI4XIsq8E0Y46c2M= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1717402939; c=relaxed/simple; bh=FrYBEuC2EWkcA/x4vedBv4/lo/U8SlIy+bWz2wH/mds=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=uGJ4WerKUW+aqqg2Z2vQx+cT1712z6bhrE5ub4i8Iw18PGnU9PjVQI4463vDbAz5IbF+4ecwKzN2HBK7/Ukee1mnnwDmQutRXi/Q8CRFx5tRxH7XE58XKoCH7uF9C6xZXkyABpCqRnHIvmpeJVCeKkYJzMCvzJW8thv+EwMCD3o= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com; spf=pass smtp.mailfrom=arm.com; arc=none smtp.client-ip=217.140.110.172 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 0514E1042; Mon, 3 Jun 2024 01:22:42 -0700 (PDT) Received: from e125905.cambridge.arm.com (e125905.cambridge.arm.com [10.1.194.73]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 07C043F792; Mon, 3 Jun 2024 01:22:15 -0700 (PDT) From: Beata Michalska To: linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, ionela.voinescu@arm.com, sudeep.holla@arm.com, will@kernel.org, catalin.marinas@arm.com, vincent.guittot@linaro.org, vanshikonda@os.amperecomputing.com Cc: sumitg@nvidia.com, yang@os.amperecomputing.com, lihuisong@huawei.com, viresh.kumar@linaro.org, rafael@kernel.org Subject: [PATCH v6 4/4] arm64: Update AMU-based freq scale factor on entering idle Date: Mon, 3 Jun 2024 09:21:54 +0100 Message-Id: <20240603082154.3830591-5-beata.michalska@arm.com> In-Reply-To: <20240603082154.3830591-1-beata.michalska@arm.com> References: <20240603082154.3830591-1-beata.michalska@arm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Now that the frequency scale factor has been activated for retrieving current frequency on a given CPU, trigger its update upon entering idle. This will, to an extent, allow querying last known frequency in a non-invasive way. It will also improve the frequency scale factor accuracy when a CPU entering idle did not receive a tick for a while. As a consequence, for idle cores, the reported frequency will be the last one observed before entering the idle state. Suggested-by: Vanshidhar Konda Signed-off-by: Beata Michalska --- arch/arm64/kernel/topology.c | 17 +++++++++++++++-- 1 file changed, 15 insertions(+), 2 deletions(-) diff --git a/arch/arm64/kernel/topology.c b/arch/arm64/kernel/topology.c index 2c002d2c3e0b..56c5b2e632b4 100644 --- a/arch/arm64/kernel/topology.c +++ b/arch/arm64/kernel/topology.c @@ -213,6 +213,19 @@ static __always_inline bool amu_fie_cpu_supported(unsi= gned int cpu) cpumask_test_cpu(cpu, amu_fie_cpus); } =20 +void arch_cpu_idle_enter(void) +{ + unsigned int cpu =3D smp_processor_id(); + + if (!amu_fie_cpu_supported(cpu)) + return; + + /* Kick in AMU update but only if one has not happened already */ + if (housekeeping_cpu(cpu, HK_TYPE_TICK) && + time_is_before_jiffies(per_cpu(cpu_amu_samples.last_update, cpu))) + amu_scale_freq_tick(); +} + #define AMU_SAMPLE_EXP_MS 20 =20 unsigned int arch_freq_get_on_cpu(int cpu) @@ -239,8 +252,8 @@ unsigned int arch_freq_get_on_cpu(int cpu) * this boils down to identifying an active cpu within the same freq * domain, if any. */ - if (!housekeeping_cpu(cpu, HK_TYPE_TICK) || - time_is_before_jiffies(last_update + msecs_to_jiffies(AMU_SAMPLE_EXP_= MS))) { + if (!housekeeping_cpu(cpu, HK_TYPE_TICK) || (!idle_cpu(cpu) && + time_is_before_jiffies(last_update + msecs_to_jiffies(AMU_SAMPLE_EXP_= MS)))) { struct cpufreq_policy *policy =3D cpufreq_cpu_get(cpu); int ref_cpu =3D cpu; =20 --=20 2.25.1