From nobody Fri Oct 2 11:41:07 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 896AA3C9EF6; Sun, 2 Aug 2026 16:53:41 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1785689622; cv=none; b=TX4L95Yhx7V7WaK0q+lVu+NwPJtK3vW6ZhCf51wLEhO0dfvniKU8vxotsGYXDh7DJEJ/9/PRO9hupT3QH1PS1zrZpapa0XBf3sU+V02uBtnkRFA40jn1Bf/J9E3ByzZWBp/1mCTbHd0qyPC/vH65MxXEgW+AwCVVg4Sh0RwMfqg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1785689622; c=relaxed/simple; bh=b/VNPkwPUumPkXZC0NVwXxj5iEweX9a5MwCOIgyy3CY=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=Aj8Jmm9H7kXB18J/x8XK+aGweEJxwLVMtvVqUo6qCOwKbZPFD6ojnR6XmQl2ucYGbpuTdPcbHw+tl3lWct5gEuJ5euWojVY4IjDmSG/QQnhbhLlpXbVDKJ8Oz+a0KkUzbzIqi3g/zGeu56shntutP3I5AxA439Mb0yFrp/7csd0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=ma03lMFj; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="ma03lMFj" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 512C21F00A3A; Sun, 2 Aug 2026 16:53:41 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1785689621; bh=DebX7evhEtlgthyyjGu18FUeYtqUAeNC8V77aBBdbns=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=ma03lMFjEIGpCCK44yCD73/RZdEU9x93nIEs2F3en0qOqsY0FmfYB3vaNfwB+Ltxl wL+bii+DHKGgsy2WfccmrkYxZeg4hW6q+hrl8W4DYjitINoxZSDus4nsmMPNJmCyYx naZzIQgzp+D4UwvnQGHeWHVhIMsSeZ5rfCjPATc+2SDLX0c0ulpHmhuHwsgSW2sNXz KzK64QI9AA3L1ciSS3yVze7La95ayIWUhVOQQtXgdJp8m7WLy3ynD7FJHyMKONfPmV BcmoPM44zc0wlYBry3D+sikoWFM9+EYRl0HGoSqXMJ73jfJZpYf9YT84PVWU9HgjUn lGMfvgdUzYCdA== Received: from sofa.misterjones.org ([185.219.108.64] helo=valley-girl.lan) by disco-boy.misterjones.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.98.2) (envelope-from ) id 1wqZRT-0000000BVIZ-10vj; Sun, 02 Aug 2026 16:53:39 +0000 From: Marc Zyngier To: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org Cc: Catalin Marinas , Will Deacon , Mark Rutland , Thomas Gleixner , Ben Horgan , Daniel Lezcano , stable@vger.kernel.org Subject: [PATCH v3 1/8] clocksource/drivers/arm_arch_timer: Add read_sched_clock_is_arch_counter() predicate Date: Sun, 2 Aug 2026 17:53:20 +0100 Message-ID: <20260802165327.385217-2-maz@kernel.org> X-Mailer: git-send-email 2.47.3 In-Reply-To: <20260802165327.385217-1-maz@kernel.org> References: <20260802165327.385217-1-maz@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-SA-Exim-Connect-IP: 185.219.108.64 X-SA-Exim-Rcpt-To: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, catalin.marinas@arm.com, will@kernel.org, mark.rutland@arm.com, tglx@kernel.org, ben.horgan@arm.com, daniel.lezcano@linaro.org, stable@vger.kernel.org X-SA-Exim-Mail-From: maz@kernel.org X-SA-Exim-Scanned: No (on disco-boy.misterjones.org); SAEximRunCond expanded to false Content-Type: text/plain; charset="utf-8" The PMUv3 driver really wants to know whether sched_clock() is backed by the architected timer. However, updates to the timer driver have broken the check, as there is more than a single accessor for the counters, depending on the level of brokenness of the HW. Add a predicate that checks for those accessors, and update the PMUv3 driver to DTRT. Fixes: 24ee7607b286b ("arm64/arch_timer: Provide noinstr sched_clock_read()= functions") Signed-off-by: Marc Zyngier Cc: stable@vger.kernel.org --- drivers/clocksource/arm_arch_timer.c | 10 ++++++++++ drivers/perf/arm_pmuv3.c | 2 +- include/clocksource/arm_arch_timer.h | 8 +++++++- 3 files changed, 18 insertions(+), 2 deletions(-) diff --git a/drivers/clocksource/arm_arch_timer.c b/drivers/clocksource/arm= _arch_timer.c index 4adf756423de9..c02e97f9b7af5 100644 --- a/drivers/clocksource/arm_arch_timer.c +++ b/drivers/clocksource/arm_arch_timer.c @@ -951,6 +951,16 @@ static void __init arch_counter_register(void) sched_clock_register(scr, width, arch_timer_rate); } =20 +bool read_sched_clock_is_arch_counter(const struct clock_read_data *crd) +{ + u64 (*rd)(void) =3D crd->read_sched_clock; + + return (rd =3D=3D raw_counter_get_cntvct_stable || + rd =3D=3D raw_counter_get_cntpct_stable || + rd =3D=3D arch_counter_get_cntvct || + rd =3D=3D arch_counter_get_cntpct); +} + static void arch_timer_stop(struct clock_event_device *clk) { pr_debug("disable IRQ%d cpu #%d\n", clk->irq, smp_processor_id()); diff --git a/drivers/perf/arm_pmuv3.c b/drivers/perf/arm_pmuv3.c index 8014ff766cff5..ed880145d41b3 100644 --- a/drivers/perf/arm_pmuv3.c +++ b/drivers/perf/arm_pmuv3.c @@ -1624,7 +1624,7 @@ void arch_perf_update_userpage(struct perf_event *eve= nt, do { rd =3D sched_clock_read_begin(&seq); =20 - if (rd->read_sched_clock !=3D arch_timer_read_counter) + if (!read_sched_clock_is_arch_counter(rd)) return; =20 userpg->time_mult =3D rd->mult; diff --git a/include/clocksource/arm_arch_timer.h b/include/clocksource/arm= _arch_timer.h index 2eda895f19f54..84cdbaffcfb5e 100644 --- a/include/clocksource/arm_arch_timer.h +++ b/include/clocksource/arm_arch_timer.h @@ -84,13 +84,15 @@ struct arch_timer_mem { struct arch_timer_mem_frame frame[ARCH_TIMER_MEM_MAX_FRAMES]; }; =20 +struct clock_read_data; + #ifdef CONFIG_ARM_ARCH_TIMER =20 extern u32 arch_timer_get_rate(void); extern u64 (*arch_timer_read_counter)(void); extern struct arch_timer_kvm_info *arch_timer_get_kvm_info(void); extern bool arch_timer_evtstrm_available(void); - +extern bool read_sched_clock_is_arch_counter(const struct clock_read_data = *); #else =20 static inline u32 arch_timer_get_rate(void) @@ -108,6 +110,10 @@ static inline bool arch_timer_evtstrm_available(void) return false; } =20 +static bool read_sched_clock_is_arch_counter(const struct clock_read_data = *crd) +{ + return false; +} #endif =20 #endif --=20 2.47.3 From nobody Fri Oct 2 11:41:07 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id AE67A3CA49A for ; Sun, 2 Aug 2026 16:53:41 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1785689623; cv=none; b=ii5fFx/biqWAX6oKwlpHt9rnr4UIq8ZHbhXJxjhUKrTaFodgQEFAdcy3a47S2uC2zWxP2O4NJGCfKLw9VZtNx1U0ZCuDVyGCy4qzZXUdEe4jgoRBEymZQXFBfmi0qxZROmpbcjm/RxbZj6d/feQ1s9Bo4UuJDLpz4G2FuPhr+JU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1785689623; c=relaxed/simple; bh=lTyvs9RSvYS7vFxRN0tFMD1+EpASGPretnpUtJjJ+bM=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=XtGGlTvMdN0SeB3mKLo+8lyt7huVN1mHxZEb6jYQyMMWR513Yo4hFmkNlOSDOWVZAq9NPQLpdEikRBhMnUgJcQIa75ixPTqh4dKbBxKiwaez3/r4x8NHpobKybe234Z43gFBq5UpJUGU52gjA4LS/rHV4OLO5WvRJIGiz196E+8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=ISHKPQef; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="ISHKPQef" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 933B71F000E9; Sun, 2 Aug 2026 16:53:41 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1785689621; bh=frSe+XQnis0Oph5HgKHmJyD+L45rNWS6tMBKUWzAmJQ=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=ISHKPQefFHyKqP5/QFFo9F67iZ0sZK66AprGjw418KGtjlaKVlM5wdei2qxX/uqVJ YRa7DOoj5rJn07bHI+ZPcxyE6/tvX19vwxJN2vqw7Jgz8TxuVziWsy1i0Cuae8Qs+W tfelHDB8YILzsNBqVcshQAAuWJgGTQGfI3M/mxoG727MwGkTNnE+5KfhelkMv7eWg3 PyidwIziSMPkCVx7LrAcqbfiqJktVAHU3/EmlGjU6o041S4GGEn6yQdQYApYSLA5SH y2oi8gOHVD+bvUT7ZOBMMpjgLMFPutWMhpRnhZpumhmDVPrjXdRm6yPvTAm2SD6SVw MbpoLdlhQkwbA== Received: from sofa.misterjones.org ([185.219.108.64] helo=valley-girl.lan) by disco-boy.misterjones.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.98.2) (envelope-from ) id 1wqZRT-0000000BVIZ-312t; Sun, 02 Aug 2026 16:53:39 +0000 From: Marc Zyngier To: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org Cc: Catalin Marinas , Will Deacon , Mark Rutland , Thomas Gleixner , Ben Horgan , Daniel Lezcano Subject: [PATCH v3 2/8] clocksource/drivers/arm_arch_timer: Add a static key indicating the need for a runtime workaround Date: Sun, 2 Aug 2026 17:53:21 +0100 Message-ID: <20260802165327.385217-3-maz@kernel.org> X-Mailer: git-send-email 2.47.3 In-Reply-To: <20260802165327.385217-1-maz@kernel.org> References: <20260802165327.385217-1-maz@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-SA-Exim-Connect-IP: 185.219.108.64 X-SA-Exim-Rcpt-To: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, catalin.marinas@arm.com, will@kernel.org, mark.rutland@arm.com, tglx@kernel.org, ben.horgan@arm.com, daniel.lezcano@linaro.org X-SA-Exim-Mail-From: maz@kernel.org X-SA-Exim-Scanned: No (on disco-boy.misterjones.org); SAEximRunCond expanded to false Content-Type: text/plain; charset="utf-8" In order to decide whether we can read the architected counter without disabling preemption to look up a workaround, introduce a static key that denotes whether a workaround is required at all. The behaviour of this new static key is a bit unusual: - it starts as 'true', indicating that workarounds are required - when all possible CPUs have booted at least once, and that it has been established that none of them require a workaround, the key flips to 'false' Of course, as long as not all the CPUs have booted once, you are stuck with slow accessors, but that's what you get for not sharing your toys. Things are made a bit complicated because static keys cannot be flipped from a CPUHP callback. Instead, schedule a deferred work from there. Yes, this is fun. Nothing is making use of this stuff yet, but watch this space. Signed-off-by: Marc Zyngier --- drivers/clocksource/arm_arch_timer.c | 35 ++++++++++++++++++++++++++++ 1 file changed, 35 insertions(+) diff --git a/drivers/clocksource/arm_arch_timer.c b/drivers/clocksource/arm= _arch_timer.c index c02e97f9b7af5..8ee317e33c864 100644 --- a/drivers/clocksource/arm_arch_timer.c +++ b/drivers/clocksource/arm_arch_timer.c @@ -90,6 +90,8 @@ static int arch_counter_get_width(void) /* * Architected system timer support. */ +static inline bool arch_counter_broken_accessors(void); + static noinstr u64 raw_counter_get_cntpct_stable(void) { return __arch_counter_get_cntpct_stable(); @@ -555,10 +557,42 @@ static bool arch_timer_counter_has_wa(void) { return atomic_read(&timer_unstable_counter_workaround_in_use); } + +static DEFINE_STATIC_KEY_TRUE(broken_cnt_accessors); + +static inline bool arch_counter_broken_accessors(void) +{ + return static_branch_unlikely(&broken_cnt_accessors); +} + +static void enable_direct_accessors(struct work_struct *wk) +{ + pr_info("Enabling direct accessors\n"); + static_branch_disable(&broken_cnt_accessors); +} + +static void arch_timer_set_direct_accessors(void) +{ + static DECLARE_WORK(enable_accessors_wk, enable_direct_accessors); + int cpu; + + if (!arch_counter_broken_accessors()) + return; + + /* Each CPU with non-zero IRQ has booted at least once */ + for_each_cpu(cpu, cpu_possible_mask) + if (!per_cpu_ptr(arch_timer_evt, cpu)->irq) + return; + + if (!arch_timer_counter_has_wa()) + schedule_work(&enable_accessors_wk); +} #else #define arch_timer_check_ool_workaround(t,a) do { } while(0) #define arch_timer_this_cpu_has_cntvct_wa() ({false;}) #define arch_timer_counter_has_wa() ({false;}) +static inline bool arch_counter_broken_accessors(void) { return false ; } +#define arch_timer_set_direct_accessors() do { } while(0) #endif /* CONFIG_ARM_ARCH_TIMER_OOL_WORKAROUND */ =20 static __always_inline irqreturn_t timer_handler(const int access, @@ -841,6 +875,7 @@ static int arch_timer_starting_cpu(unsigned int cpu) } =20 arch_counter_set_user_access(); + arch_timer_set_direct_accessors(); =20 return 0; } --=20 2.47.3 From nobody Fri Oct 2 11:41:07 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 37FE83CAA49 for ; Sun, 2 Aug 2026 16:53:42 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1785689624; cv=none; b=dW1GuAu7p/n+pa7lBWuogxXKClMwh+GM2BEzZGKnpWp2sljQ5nGGl5x05ZosUnj3S4i3GK1yXjuZLQqJqxsNHSqAtLf08U94HyRhLKINF/7JZnn2I0Jg8QFOKLytXxSAgm9OEwQdGqjtrITYGRQkmDbFpId1imJEfNKIf9Fp7uk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1785689624; c=relaxed/simple; bh=82dVbqROx6vZYLs8uhqb6IT0srAPqmKyngtBeXm4hE0=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=oniC74KUXy8p0JkdzaaboB4yKEKjse/lGkKGzCoNKj391cEiRd5/Tk3u9U6nZlSpV3qlL9asH4rvRZD4kamYBQcIAaYX7Fd60Ii0cTLoGSJKhekPIkWIwPUsBooRrfQ7kKzAnLfBMOTpZOVxqMJQDhNm9tNA8P2bxf3ElPIvoE8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=P/s9zczN; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="P/s9zczN" Received: by smtp.kernel.org (Postfix) with ESMTPSA id F2FF41F00A3E; Sun, 2 Aug 2026 16:53:41 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1785689622; bh=l1E7eRaD45mE8Ul8FpAjG4IxkSoELLrr3ZiMXm4Q0lg=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=P/s9zczNz132ZOtuHL3raiD+3ZJ7B879S6nJaCwPAU7opM9PJcnHK4if8lD5WO74d XWEsGOQm/OH3c8Bj61MrvEpm2zvyTw5+Qu0nvy1p+s0wOx1PH/QIb9AJasm1v6E3AW mgFAJJRot2my7SuocvMiy/GEg3E1CztBUKpcyMgmMwxJ+ePhDU9vJpY7u3k9sT4sxZ Plzna+uSCiac3RGHx09ItFAxMIb/enwWuBIr2C9nlKHotGyFj0otOthpMUw2ndJsy4 UqB803E91mupNooSXFNs7sxixHRU1fbb1EdsMYe2GP2ZYTQmDKkR9CXjlpm7m6EeHe n8KI1w4i88Udw== Received: from sofa.misterjones.org ([185.219.108.64] helo=valley-girl.lan) by disco-boy.misterjones.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.98.2) (envelope-from ) id 1wqZRU-0000000BVIZ-0F1R; Sun, 02 Aug 2026 16:53:40 +0000 From: Marc Zyngier To: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org Cc: Catalin Marinas , Will Deacon , Mark Rutland , Thomas Gleixner , Ben Horgan , Daniel Lezcano Subject: [PATCH v3 3/8] clocksource/drivers/arm_arch_timer: Convert counter accessors to a static key alternative Date: Sun, 2 Aug 2026 17:53:22 +0100 Message-ID: <20260802165327.385217-4-maz@kernel.org> X-Mailer: git-send-email 2.47.3 In-Reply-To: <20260802165327.385217-1-maz@kernel.org> References: <20260802165327.385217-1-maz@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-SA-Exim-Connect-IP: 185.219.108.64 X-SA-Exim-Rcpt-To: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, catalin.marinas@arm.com, will@kernel.org, mark.rutland@arm.com, tglx@kernel.org, ben.horgan@arm.com, daniel.lezcano@linaro.org X-SA-Exim-Mail-From: maz@kernel.org X-SA-Exim-Scanned: No (on disco-boy.misterjones.org); SAEximRunCond expanded to false Content-Type: text/plain; charset="utf-8" Now that we have a reliable static key to control whether our counter accessors need to be worked around, use it in these accessors and simplify the logic that picks which accessor to use. Signed-off-by: Marc Zyngier --- drivers/clocksource/arm_arch_timer.c | 44 ++++++++++++++-------------- 1 file changed, 22 insertions(+), 22 deletions(-) diff --git a/drivers/clocksource/arm_arch_timer.c b/drivers/clocksource/arm= _arch_timer.c index 8ee317e33c864..f645aa562895b 100644 --- a/drivers/clocksource/arm_arch_timer.c +++ b/drivers/clocksource/arm_arch_timer.c @@ -92,9 +92,12 @@ static int arch_counter_get_width(void) */ static inline bool arch_counter_broken_accessors(void); =20 -static noinstr u64 raw_counter_get_cntpct_stable(void) +static noinstr u64 raw_counter_get_cntpct(void) { - return __arch_counter_get_cntpct_stable(); + if (arch_counter_broken_accessors()) + return __arch_counter_get_cntpct_stable(); + + return __arch_counter_get_cntpct(); } =20 static notrace u64 arch_counter_get_cntpct_stable(void) @@ -108,12 +111,18 @@ static notrace u64 arch_counter_get_cntpct_stable(voi= d) =20 static noinstr u64 arch_counter_get_cntpct(void) { + if (arch_counter_broken_accessors()) + return arch_counter_get_cntpct_stable(); + return __arch_counter_get_cntpct(); } =20 -static noinstr u64 raw_counter_get_cntvct_stable(void) +static noinstr u64 raw_counter_get_cntvct(void) { - return __arch_counter_get_cntvct_stable(); + if (arch_counter_broken_accessors()) + return __arch_counter_get_cntvct_stable(); + + return __arch_counter_get_cntvct(); } =20 static notrace u64 arch_counter_get_cntvct_stable(void) @@ -127,6 +136,9 @@ static notrace u64 arch_counter_get_cntvct_stable(void) =20 static noinstr u64 arch_counter_get_cntvct(void) { + if (arch_counter_broken_accessors()) + return arch_counter_get_cntvct_stable(); + return __arch_counter_get_cntvct(); } =20 @@ -950,21 +962,11 @@ static void __init arch_counter_register(void) if ((IS_ENABLED(CONFIG_ARM64) && !is_hyp_mode_available()) || arch_timer_uses_ppi =3D=3D ARCH_TIMER_VIRT_PPI || arch_timer_uses_ppi =3D=3D ARCH_TIMER_HYP_VIRT_PPI) { - if (arch_timer_counter_has_wa()) { - rd =3D arch_counter_get_cntvct_stable; - scr =3D raw_counter_get_cntvct_stable; - } else { - rd =3D arch_counter_get_cntvct; - scr =3D arch_counter_get_cntvct; - } + rd =3D arch_counter_get_cntvct; + scr =3D raw_counter_get_cntvct; } else { - if (arch_timer_counter_has_wa()) { - rd =3D arch_counter_get_cntpct_stable; - scr =3D raw_counter_get_cntpct_stable; - } else { - rd =3D arch_counter_get_cntpct; - scr =3D arch_counter_get_cntpct; - } + rd =3D arch_counter_get_cntpct; + scr =3D raw_counter_get_cntpct; } =20 arch_timer_read_counter =3D rd; @@ -990,10 +992,8 @@ bool read_sched_clock_is_arch_counter(const struct clo= ck_read_data *crd) { u64 (*rd)(void) =3D crd->read_sched_clock; =20 - return (rd =3D=3D raw_counter_get_cntvct_stable || - rd =3D=3D raw_counter_get_cntpct_stable || - rd =3D=3D arch_counter_get_cntvct || - rd =3D=3D arch_counter_get_cntpct); + return (rd =3D=3D raw_counter_get_cntvct || + rd =3D=3D raw_counter_get_cntpct); } =20 static void arch_timer_stop(struct clock_event_device *clk) --=20 2.47.3 From nobody Fri Oct 2 11:41:07 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 74CC03CAE80 for ; Sun, 2 Aug 2026 16:53:42 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1785689624; cv=none; b=fi4uV4PGnPUIA/vvU6MrDlnZA/HThtNXkJ8fzYBa8LtLG1/XjYpqY8rQK2UsqAav7CwXR/COh5kXQ2a/UMYrKnAajuObNEC7TGidAqH8hdB4ayKQBx8EZntPHgGs3lFpINxijx1sHt2BNHW8jnnZYHowBnSjn3fzLFw9LTmePwI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1785689624; c=relaxed/simple; bh=Dx2YA1KqlxtSazStCMOfdDCRKPW5jgznMF4uivzvqJA=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=UfKJ4JY/H1w5XXi+RExt3BS+LqsJ/ikH0+J5EcF2ULP5Ezn2D9T8mon/pCVoXwIq51hp7tHJFjd4jGkiSatsoykQ7yGki1CeMERQYynyL3m/Wr19/S3xsqoOaSWOElhok00N7oPmDq+ohTnciGgzhrXdENSpJYoeu0x9ex/qcic= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=hZPwECgL; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="hZPwECgL" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 5A7F11F00ACF; Sun, 2 Aug 2026 16:53:42 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1785689622; bh=dEzw2i7HRKSDGO8dkn+XkGr8/pmxZoR0KreFTkdRu2g=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=hZPwECgLePdW81OBR1YR4NyrGRDDIKetg8utx56Gs5HhhCA9+DOL7gMeRG4i2U4mm ZNGmprY/crJiz0WWdjdjn78Mruk/vcbbGqyHJLo0JU8rf/jbWK07omNTXGI27b88mL KhZ/ASyNRc2DboxnoDIukeYNZMEOH/fxoxO0tNS4p+ZxXnARs4XOIuX/dxkOMh5ID+ 62b1KL9sh0Khy9z9Kh0PcG9v4WwvntCWcfGSVvZJdUcFP90+D1H/pjACczB4iGfohJ +l0FxFMYx5knd5ZIli4kKnqnqmC5D1TZzhoF2Bj/E60SgLd1BE9cwdnMQagP2sn8Xs FScbAHyg+UDoA== Received: from sofa.misterjones.org ([185.219.108.64] helo=valley-girl.lan) by disco-boy.misterjones.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.98.2) (envelope-from ) id 1wqZRU-0000000BVIZ-1kBi; Sun, 02 Aug 2026 16:53:40 +0000 From: Marc Zyngier To: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org Cc: Catalin Marinas , Will Deacon , Mark Rutland , Thomas Gleixner , Ben Horgan , Daniel Lezcano Subject: [PATCH v3 4/8] clocksource/drivers/arm_arch_timer: Drop the arch_counter_get_cnt{p,v}ct_stable() accessors Date: Sun, 2 Aug 2026 17:53:23 +0100 Message-ID: <20260802165327.385217-5-maz@kernel.org> X-Mailer: git-send-email 2.47.3 In-Reply-To: <20260802165327.385217-1-maz@kernel.org> References: <20260802165327.385217-1-maz@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-SA-Exim-Connect-IP: 185.219.108.64 X-SA-Exim-Rcpt-To: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, catalin.marinas@arm.com, will@kernel.org, mark.rutland@arm.com, tglx@kernel.org, ben.horgan@arm.com, daniel.lezcano@linaro.org X-SA-Exim-Mail-From: maz@kernel.org X-SA-Exim-Scanned: No (on disco-boy.misterjones.org); SAEximRunCond expanded to false Content-Type: text/plain; charset="utf-8" Further simplify the counter accessors by eliminating the *_stable() ones, which serve little purpose at this stage. Signed-off-by: Marc Zyngier --- drivers/clocksource/arm_arch_timer.c | 38 +++++++++------------------- 1 file changed, 12 insertions(+), 26 deletions(-) diff --git a/drivers/clocksource/arm_arch_timer.c b/drivers/clocksource/arm= _arch_timer.c index f645aa562895b..5085e87706f99 100644 --- a/drivers/clocksource/arm_arch_timer.c +++ b/drivers/clocksource/arm_arch_timer.c @@ -100,19 +100,12 @@ static noinstr u64 raw_counter_get_cntpct(void) return __arch_counter_get_cntpct(); } =20 -static notrace u64 arch_counter_get_cntpct_stable(void) +static notrace u64 arch_counter_get_cntpct(void) { - u64 val; - preempt_disable_notrace(); - val =3D __arch_counter_get_cntpct_stable(); - preempt_enable_notrace(); - return val; -} - -static noinstr u64 arch_counter_get_cntpct(void) -{ - if (arch_counter_broken_accessors()) - return arch_counter_get_cntpct_stable(); + if (arch_counter_broken_accessors()) { + guard(preempt_notrace)(); + return __arch_counter_get_cntpct_stable(); + } =20 return __arch_counter_get_cntpct(); } @@ -125,19 +118,12 @@ static noinstr u64 raw_counter_get_cntvct(void) return __arch_counter_get_cntvct(); } =20 -static notrace u64 arch_counter_get_cntvct_stable(void) +static notrace u64 arch_counter_get_cntvct(void) { - u64 val; - preempt_disable_notrace(); - val =3D __arch_counter_get_cntvct_stable(); - preempt_enable_notrace(); - return val; -} - -static noinstr u64 arch_counter_get_cntvct(void) -{ - if (arch_counter_broken_accessors()) - return arch_counter_get_cntvct_stable(); + if (arch_counter_broken_accessors()) { + guard(preempt_notrace)(); + return __arch_counter_get_cntvct_stable(); + } =20 return __arch_counter_get_cntvct(); } @@ -342,10 +328,10 @@ void erratum_set_next_event_generic(const int access,= unsigned long evt, ctrl &=3D ~ARCH_TIMER_CTRL_IT_MASK; =20 if (access =3D=3D ARCH_TIMER_PHYS_ACCESS) { - cval =3D evt + arch_counter_get_cntpct_stable(); + cval =3D evt + arch_counter_get_cntpct(); write_sysreg(cval, cntp_cval_el0); } else { - cval =3D evt + arch_counter_get_cntvct_stable(); + cval =3D evt + arch_counter_get_cntvct(); write_sysreg(cval, cntv_cval_el0); } =20 --=20 2.47.3 From nobody Fri Oct 2 11:41:07 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D3CC73CB2EA for ; Sun, 2 Aug 2026 16:53:42 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1785689624; cv=none; b=faB4ddmj+EguDWjv0/d8zKlbEOAIGmW0/MquSDVNpzIM3WY/LITbGL8OQJMuSRquR1DL0Xlu0/t2jfiPUw25LCwGvyclyDpsYTDYgZpEKfMyKs4os+piMmwT6FqPbMmKmTEQCUh7mbUkDLCNi/7JSM5vAkr1jCd32nllsr4I5oo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1785689624; c=relaxed/simple; bh=Ljr8Y8G4OWXk0i2OFlFuD9kNK/QO9C9ZOhFi5IaPU3M=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=RREmyADDNJ0ROlcXXGd7VrCZjxr3Sy7T5lv5tF0jGkBqMWlZbprPi19BaswbrnOvGDbtDD8COezG4/IYo9aAvuKYj7yJrA4fV98IeU8ZJe3axCrVeQyqDD0wAyXhJ7ibahwIMUYJ78eKtMzRKraAzJQPeOBM6Igt6/IC+MSZnyg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=cH70UKcj; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="cH70UKcj" Received: by smtp.kernel.org (Postfix) with ESMTPSA id AF14A1F00A3F; Sun, 2 Aug 2026 16:53:42 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1785689622; bh=ZAAyG8SRRqbLdivjdRdMUZVSQc5/o7iuh79T8Z5zcL0=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=cH70UKcj1v4XNHl3vPR/tf6CtySyuW6jNWyslMQYi1GJBQIchHJ7vBETdMqJgyZ/I Wb2gbUcmfVus5T4dxY/RPIhRYUMabZTyLqe+USI+B0jtuuLNmPy9X7qUuKX79W5D9S BOPRgGeTegqs/n6bK/VYETVWxHIj0v0J1djqLb06jVb7Vq3f+AUEaiLl0JJ7dMfCK/ ABwWpJNHYJHr+thOBAu697YxcOgnvo2VCegHdG7T4DtyjFal3xUDyfqCq431DRYipA KpddCi8wOiR55JfYAq1YaBWNqqPirGKxzk9zWZhRYJQAf2yxlz+lPON5+EZtZFO4V7 vfy+ALN9QQIoA== Received: from sofa.misterjones.org ([185.219.108.64] helo=valley-girl.lan) by disco-boy.misterjones.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.98.2) (envelope-from ) id 1wqZRU-0000000BVIZ-3eqS; Sun, 02 Aug 2026 16:53:41 +0000 From: Marc Zyngier To: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org Cc: Catalin Marinas , Will Deacon , Mark Rutland , Thomas Gleixner , Ben Horgan , Daniel Lezcano Subject: [PATCH v3 5/8] clocksource/drivers/arm_arch_timer: Turn arch_timer_read_counter into a function Date: Sun, 2 Aug 2026 17:53:24 +0100 Message-ID: <20260802165327.385217-6-maz@kernel.org> X-Mailer: git-send-email 2.47.3 In-Reply-To: <20260802165327.385217-1-maz@kernel.org> References: <20260802165327.385217-1-maz@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-SA-Exim-Connect-IP: 185.219.108.64 X-SA-Exim-Rcpt-To: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, catalin.marinas@arm.com, will@kernel.org, mark.rutland@arm.com, tglx@kernel.org, ben.horgan@arm.com, daniel.lezcano@linaro.org X-SA-Exim-Mail-From: maz@kernel.org X-SA-Exim-Scanned: No (on disco-boy.misterjones.org); SAEximRunCond expanded to false Content-Type: text/plain; charset="utf-8" Now that we control the workarounds applied to the counters with a static key, we can also do the same thing for the selection between virtual and physical counters, as this selection is done on the boot CPU, where we have the freedom to flip static keys. With this, there is no need for arch_timer_read_counter to be a function pointer, and we can directly promote it to an actual function. While we're at it, apply the same static key treatment to the raw_* accessors. Signed-off-by: Marc Zyngier --- drivers/clocksource/arm_arch_timer.c | 38 +++++++++++++++++----------- include/clocksource/arm_arch_timer.h | 2 +- 2 files changed, 24 insertions(+), 16 deletions(-) diff --git a/drivers/clocksource/arm_arch_timer.c b/drivers/clocksource/arm= _arch_timer.c index 5085e87706f99..747f51d9225c2 100644 --- a/drivers/clocksource/arm_arch_timer.c +++ b/drivers/clocksource/arm_arch_timer.c @@ -134,9 +134,21 @@ static notrace u64 arch_counter_get_cntvct(void) * to exist on arm64. arm doesn't use this before DT is probed so even * if we don't have the cp15 accessors we won't have a problem. */ -u64 (*arch_timer_read_counter)(void) __ro_after_init =3D arch_counter_get_= cntvct; +static DEFINE_STATIC_KEY_TRUE(arch_counter_is_virtual); + +u64 notrace arch_timer_read_counter(void) +{ + return static_branch_likely(&arch_counter_is_virtual) ? + arch_counter_get_cntvct() : arch_counter_get_cntpct(); +} EXPORT_SYMBOL_GPL(arch_timer_read_counter); =20 +static u64 noinstr raw_arch_timer_read_counter(void) +{ + return static_branch_likely(&arch_counter_is_virtual) ? + raw_counter_get_cntvct() : raw_counter_get_cntpct(); +} + static u64 arch_counter_read(struct clocksource *cs) { return arch_timer_read_counter(); @@ -940,22 +952,19 @@ struct arch_timer_kvm_info *arch_timer_get_kvm_info(v= oid) =20 static void __init arch_counter_register(void) { - u64 (*scr)(void); - u64 (*rd)(void); u64 start_count; int width; =20 - if ((IS_ENABLED(CONFIG_ARM64) && !is_hyp_mode_available()) || - arch_timer_uses_ppi =3D=3D ARCH_TIMER_VIRT_PPI || - arch_timer_uses_ppi =3D=3D ARCH_TIMER_HYP_VIRT_PPI) { - rd =3D arch_counter_get_cntvct; - scr =3D raw_counter_get_cntvct; - } else { - rd =3D arch_counter_get_cntpct; - scr =3D raw_counter_get_cntpct; + switch (arch_timer_uses_ppi) { + case ARCH_TIMER_PHYS_SECURE_PPI: + case ARCH_TIMER_PHYS_NONSECURE_PPI: + case ARCH_TIMER_HYP_PPI: + static_branch_disable(&arch_counter_is_virtual); + break; + default: + break; } =20 - arch_timer_read_counter =3D rd; clocksource_counter.vdso_clock_mode =3D vdso_default; =20 width =3D arch_counter_get_width(); @@ -971,15 +980,14 @@ static void __init arch_counter_register(void) timecounter_init(&arch_timer_kvm_info.timecounter, &cyclecounter, start_count); =20 - sched_clock_register(scr, width, arch_timer_rate); + sched_clock_register(raw_arch_timer_read_counter, width, arch_timer_rate); } =20 bool read_sched_clock_is_arch_counter(const struct clock_read_data *crd) { u64 (*rd)(void) =3D crd->read_sched_clock; =20 - return (rd =3D=3D raw_counter_get_cntvct || - rd =3D=3D raw_counter_get_cntpct); + return (rd =3D=3D raw_arch_timer_read_counter); } =20 static void arch_timer_stop(struct clock_event_device *clk) diff --git a/include/clocksource/arm_arch_timer.h b/include/clocksource/arm= _arch_timer.h index 84cdbaffcfb5e..8deb815ed0a08 100644 --- a/include/clocksource/arm_arch_timer.h +++ b/include/clocksource/arm_arch_timer.h @@ -89,7 +89,7 @@ struct clock_read_data; #ifdef CONFIG_ARM_ARCH_TIMER =20 extern u32 arch_timer_get_rate(void); -extern u64 (*arch_timer_read_counter)(void); +extern u64 arch_timer_read_counter(void); extern struct arch_timer_kvm_info *arch_timer_get_kvm_info(void); extern bool arch_timer_evtstrm_available(void); extern bool read_sched_clock_is_arch_counter(const struct clock_read_data = *); --=20 2.47.3 From nobody Fri Oct 2 11:41:07 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8B22B3CB2E7 for ; Sun, 2 Aug 2026 16:53:43 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1785689625; cv=none; b=kPyZX1irSPXu8/dDR5+MGwnnoO79YFLdZqTUl6UjsS4Elf+YIqCZJ7WQ9FBkzxNVQgGd4InDSx8aHJ0lL/rDWM3McgjqEeBYRXGeVs3p/jgmcZIMcnTFBFFn+0rOS4AhgiSNuln9S3v+QOxZ5Y0btn/8cDw7ONxszCUJohgEl0w= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1785689625; c=relaxed/simple; bh=d59v2ypClbCc8c5Hd0Ef1c4VRvvmQVeQAL6wALV2raY=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=oS0nQQpOTsPeJUFkVC0TxHbrxn9Nq3itYboBd9bsLCb/eQ7Im3J2bsf1xQCcTMgwCthR1FK+ouc6cGDraf6FYG6mD+10oPPO5HdvUr/wgsr2vRmN6urFxU5KoF24qA+H7U/zcAS+vgBc23hPWUyHwyUSjTor1A7lJty3Kylno4M= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=jd1MSEcN; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="jd1MSEcN" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 402DC1F00ADE; Sun, 2 Aug 2026 16:53:43 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1785689623; bh=KbS6tAMDWYU94iLSiqDXxb03HkeRPSt0gzD9443d824=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=jd1MSEcN/ND/t2vB7gQSkN/B2k2z8ChAMwJrvoVaABc/BZrrWZBMNRnYJ7SW1Gi7O DizMAR3jJqLHVyKn1rc16gE08VuEWeBMuTq3a2Jb+i89BC5rjvzJSAl9ugKUuxE3Uc WelHoACDYW8QFPLMuq4LF1sGpY3pbun0AtFtkJlUl8UIQ7jmEWatEdKMtc6zZcgfrn GgDIkrSemPXw2h993BMcKdRnEwbjxtsERjfL3dYRvpJGvZ24EBdwA4btAxtejsMuDc vUqRp4tQIBOwrS9MUhW1BSwib2bLPZEJ91o7ldyxEWFPOYLyMD6yPShwdmlXEB8ubF fTJ+Jm+gZZEjQ== Received: from sofa.misterjones.org ([185.219.108.64] helo=valley-girl.lan) by disco-boy.misterjones.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.98.2) (envelope-from ) id 1wqZRV-0000000BVIZ-16iI; Sun, 02 Aug 2026 16:53:41 +0000 From: Marc Zyngier To: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org Cc: Catalin Marinas , Will Deacon , Mark Rutland , Thomas Gleixner , Ben Horgan , Daniel Lezcano Subject: [PATCH v3 6/8] clocksource/drivers/arm_arch_timer: Add command-line control over the counter errata management Date: Sun, 2 Aug 2026 17:53:25 +0100 Message-ID: <20260802165327.385217-7-maz@kernel.org> X-Mailer: git-send-email 2.47.3 In-Reply-To: <20260802165327.385217-1-maz@kernel.org> References: <20260802165327.385217-1-maz@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-SA-Exim-Connect-IP: 185.219.108.64 X-SA-Exim-Rcpt-To: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, catalin.marinas@arm.com, will@kernel.org, mark.rutland@arm.com, tglx@kernel.org, ben.horgan@arm.com, daniel.lezcano@linaro.org X-SA-Exim-Mail-From: maz@kernel.org X-SA-Exim-Scanned: No (on disco-boy.misterjones.org); SAEximRunCond expanded to false Content-Type: text/plain; charset="utf-8" While we now have a safe way to use "fast" accessors once all CPUs have booted, this leaves people booting with maxcpus=3D on non-broken hardware stuck with the "slow" counter accessors. Give these people a way out by adding a new command-line parameter aptly named clocksource.arm_arch_timer.cnt_errata, which allows the user to promise that no erratum handling is required for the counters by setting this value to 0. Warnings will be emitted if the user has over-promised. Suggested-by; Will Deacon Signed-off-by: Marc Zyngier Suggested-by; Will Deacon --- Documentation/admin-guide/kernel-parameters.txt | 13 +++++++++++++ drivers/clocksource/arm_arch_timer.c | 17 ++++++++++++++++- 2 files changed, 29 insertions(+), 1 deletion(-) diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentatio= n/admin-guide/kernel-parameters.txt index b5493a7f8f228..f0e6534269c2b 100644 --- a/Documentation/admin-guide/kernel-parameters.txt +++ b/Documentation/admin-guide/kernel-parameters.txt @@ -830,6 +830,19 @@ Kernel parameters loops can be debugged more effectively on production systems. =20 + clocksource.arm_arch_timer.cnt_errata=3D + [ARM64,EARLY] + Format: + Enable/disable the counter errata management. + Enabling it switches over to fast accessors once it is + known that no CPU requires any workaround while reading + the counters. + Disabling it will bypass workarounds when reading the + counters, even if not all the CPUs have been probed. + Warnings will be produced if the need for a workaround + is detected. + Default is enabled. + clocksource.verify_n_cpus=3D [KNL] Limit the number of CPUs checked for clocksources marked with CLOCK_SOURCE_VERIFY_PERCPU that diff --git a/drivers/clocksource/arm_arch_timer.c b/drivers/clocksource/arm= _arch_timer.c index 747f51d9225c2..b3b31d4f4815f 100644 --- a/drivers/clocksource/arm_arch_timer.c +++ b/drivers/clocksource/arm_arch_timer.c @@ -500,8 +500,10 @@ void arch_timer_enable_workaround(const struct arch_ti= mer_erratum_workaround *wa per_cpu(timer_unstable_counter_workaround, i) =3D wa; } =20 - if (wa->read_cntvct_el0 || wa->read_cntpct_el0) + if (wa->read_cntvct_el0 || wa->read_cntpct_el0) { + WARN_ON_ONCE(!arch_counter_broken_accessors()); atomic_set(&timer_unstable_counter_workaround_in_use, 1); + } =20 /* * Don't use the vdso fastpath if errata require using the @@ -597,12 +599,22 @@ static void arch_timer_set_direct_accessors(void) if (!arch_timer_counter_has_wa()) schedule_work(&enable_accessors_wk); } + +static bool cnt_errata_config __initdata =3D true; + +static int __init early_cnt_errata(char *buf) +{ + return kstrtobool(buf, &cnt_errata_config); +} +early_param("clocksource.arm_arch_timer.cnt_errata", early_cnt_errata); #else #define arch_timer_check_ool_workaround(t,a) do { } while(0) #define arch_timer_this_cpu_has_cntvct_wa() ({false;}) #define arch_timer_counter_has_wa() ({false;}) static inline bool arch_counter_broken_accessors(void) { return false ; } #define arch_timer_set_direct_accessors() do { } while(0) +#define enable_direct_accessors(w) do { } while(0) +#define cnt_errata_config false #endif /* CONFIG_ARM_ARCH_TIMER_OOL_WORKAROUND */ =20 static __always_inline irqreturn_t timer_handler(const int access, @@ -955,6 +967,9 @@ static void __init arch_counter_register(void) u64 start_count; int width; =20 + if (!cnt_errata_config) + enable_direct_accessors(NULL); + switch (arch_timer_uses_ppi) { case ARCH_TIMER_PHYS_SECURE_PPI: case ARCH_TIMER_PHYS_NONSECURE_PPI: --=20 2.47.3 From nobody Fri Oct 2 11:41:07 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D07663CC32D for ; Sun, 2 Aug 2026 16:53:43 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1785689625; cv=none; b=FBiz5gXKTm83ngYL9uC4qThxZqaN3DLcGF5A4oC0F38AzXVTU/ATQREwUTc2D+o7dHRET3eJfOtcsSLzQ5GUFJ744Ig+BW0rjxXq//FkMoswY9W1kdxuu73XpQxB/YkIZv/8Q4oAjUXqoduexYyLIpP6vyISCVxs5q5GiEepYDA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1785689625; c=relaxed/simple; bh=H0L3w+o6UadGsjx3EfHoFmROp0sOs/Nw64D6PDQZPs4=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=LlVBIAJyKQS8QlrzMEzWzLGMq7XHly6smPscGv1MUSOWh8DLEWHET7ztA2XLqmVQnfoUTNePfDYiO+DIYaZU+lGG6KzKTms3n31nWGAm2p+U4qlFbpoxVW9epOIAZIkqg/6u61U2pVP/WKiotQiSoql7Dv/5dFALPorPHFD776U= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=HWqIds1J; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="HWqIds1J" Received: by smtp.kernel.org (Postfix) with ESMTPSA id B3B461F01559; Sun, 2 Aug 2026 16:53:43 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1785689623; bh=IPrlliaNT6d54S3Oz/pAE+1e1wLs+2Mvnn7Uhn7fUaM=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=HWqIds1JtqxoC+GXg+J61Hb/rl+q23KepnybmBwzwlbJW9acaz/wb0XEcS1Tmz7Ul Sw6zLP7QaMh53864joQB3Tk+CLRt+l+qa3d4mqwaNt9QK7OYaQLDH/Hlwa59wXSOh4 qQ8XAocO5+86eR3C7qHx7O8XsbOtZvgJ2AEfpWvFwF5HxCCo9Vfsn+gdmh+Vf3fADg bz3BPlgm5YbLrkJZ3H0M1f6OeN8yr7e8PnO0Y8ZE62T44TS3g2i7RAt2w3+r0mp8bo WTQJpfIGyvIaRbF7P8swq164KxT6zVmPq73/rdXigAmcK0FWRBovF+a4o1WlKEfeJv 22yp6tbStDCgQ== Received: from sofa.misterjones.org ([185.219.108.64] helo=valley-girl.lan) by disco-boy.misterjones.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.98.2) (envelope-from ) id 1wqZRV-0000000BVIZ-3G7g; Sun, 02 Aug 2026 16:53:42 +0000 From: Marc Zyngier To: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org Cc: Catalin Marinas , Will Deacon , Mark Rutland , Thomas Gleixner , Ben Horgan , Daniel Lezcano Subject: [PATCH v3 7/8] clocksource/drivers/arm_arch_timer: Expose a direct accessor for the virtual counter Date: Sun, 2 Aug 2026 17:53:26 +0100 Message-ID: <20260802165327.385217-8-maz@kernel.org> X-Mailer: git-send-email 2.47.3 In-Reply-To: <20260802165327.385217-1-maz@kernel.org> References: <20260802165327.385217-1-maz@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-SA-Exim-Connect-IP: 185.219.108.64 X-SA-Exim-Rcpt-To: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, catalin.marinas@arm.com, will@kernel.org, mark.rutland@arm.com, tglx@kernel.org, ben.horgan@arm.com, daniel.lezcano@linaro.org X-SA-Exim-Mail-From: maz@kernel.org X-SA-Exim-Scanned: No (on disco-boy.misterjones.org); SAEximRunCond expanded to false Content-Type: text/plain; charset="utf-8" We allow access to the architected counter via arch_timer_read_counter(). However, this accessor can either be the virtual or the physical view of the counter, depending on how the kernel has been booted. At the same time, we have some architectural features (such as WFIT, WFET) that rely on the virtual counter, and nothing else. If implementations were perfect, we'd rely on reading CNTVCT_EL0, and be done with it. However, we have a bunch of broken implementations in the wild, which rely on preemption being disabled and other costly workarounds. In order to provide decent performance on non-broken HW while still supporting the legacy horrors, expose arch_timer_read_vcounter() as a new helper that hides this complexity. Obviously, this is simply a global alias of arch_counter_get_cntvct(). Signed-off-by: Marc Zyngier --- drivers/clocksource/arm_arch_timer.c | 2 ++ include/clocksource/arm_arch_timer.h | 1 + 2 files changed, 3 insertions(+) diff --git a/drivers/clocksource/arm_arch_timer.c b/drivers/clocksource/arm= _arch_timer.c index b3b31d4f4815f..9c7501ce57073 100644 --- a/drivers/clocksource/arm_arch_timer.c +++ b/drivers/clocksource/arm_arch_timer.c @@ -128,6 +128,8 @@ static notrace u64 arch_counter_get_cntvct(void) return __arch_counter_get_cntvct(); } =20 +u64 arch_timer_read_vcounter(void) __attribute__((alias("arch_counter_get_= cntvct"))); + /* * Default to cp15 based access because arm64 uses this function for * sched_clock() before DT is probed and the cp15 method is guaranteed diff --git a/include/clocksource/arm_arch_timer.h b/include/clocksource/arm= _arch_timer.h index 8deb815ed0a08..32fd531ac009f 100644 --- a/include/clocksource/arm_arch_timer.h +++ b/include/clocksource/arm_arch_timer.h @@ -90,6 +90,7 @@ struct clock_read_data; =20 extern u32 arch_timer_get_rate(void); extern u64 arch_timer_read_counter(void); +extern u64 arch_timer_read_vcounter(void); extern struct arch_timer_kvm_info *arch_timer_get_kvm_info(void); extern bool arch_timer_evtstrm_available(void); extern bool read_sched_clock_is_arch_counter(const struct clock_read_data = *); --=20 2.47.3 From nobody Fri Oct 2 11:41:07 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B8C083CD8C9 for ; Sun, 2 Aug 2026 16:53:44 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1785689626; cv=none; b=g77HzxrO6D8yInyv+hKiPKUZS92yhDrzgpZPyATYE9+1XxCxUhm5BME1ZMYrQztVeEGZ+fWx5xL5b8xSoXoVb33xiQAuTzk8xF2+fRyKQipBX2Pu6JuTaf9nBS7ZVcnzR9zTwPlN27RXFsu/IQ/e70e+LJC3Uw3CiHLxlo7kO1s= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1785689626; c=relaxed/simple; bh=2EFYN/oBtCdJKnWuKFE8dOyMNgl8TmPd50By+s9SVIo=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=q/1fw9Ws9CP2WCHi2pOA6p8fZVsqJbp9JGP7r2zBxfVloXXauHfUOtV90gnkA7H1m1YxCRcZCZnHSlMQF48oHDc1TrTvr0zy1apZ7jhZVIbUn4yRIuVkbcI1QwGP4aZXBU6HMILo8kk7dT1P+tf3P4bjcsSbrCoW1L8ABjSjaiE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=Enxcngrd; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="Enxcngrd" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 977831F00ACF; Sun, 2 Aug 2026 16:53:44 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1785689624; bh=1ddJMjzmSVlj3ZFAumHWC0fEgCC+ifQjE0giD0DMzI8=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=EnxcngrdrxYgrQ68dHejrhk3OZR5ULOTHf7u56rU1kxV5VCoRKQ/RGs6wwfSR5aB6 lzDrdsH5yw1nYoqv5Mo7QVB0YSPBEBVS7dJSv2R0vAiKy+Tkco/0ZBEjTsdX5GzKvC QYBRK/a39+WiSCQ64jri1S9014b8v01NR4J6iSeBCBYWMg6gAU0XofFScnrCIHwuWO JKEU7c0w5LD5/kiz0a1xgiazTeJqiuYafE4P3RnCc0Pxn/k9fudyRXi/+8NNXAnPpG 3J7GDNaEQfuBQsWU4jVlj+P5iNoAJO1qD62pgXCwE0yzw4jfz7GnKPMpvw4pSnjFPS in7fUY5C4xkcA== Received: from sofa.misterjones.org ([185.219.108.64] helo=valley-girl.lan) by disco-boy.misterjones.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.98.2) (envelope-from ) id 1wqZRW-0000000BVIZ-1GnL; Sun, 02 Aug 2026 16:53:42 +0000 From: Marc Zyngier To: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org Cc: Catalin Marinas , Will Deacon , Mark Rutland , Thomas Gleixner , Ben Horgan , Daniel Lezcano Subject: [PATCH v3 8/8] arm64: Convert __delay_cycles() to arch_timer_read_vcounter() Date: Sun, 2 Aug 2026 17:53:27 +0100 Message-ID: <20260802165327.385217-9-maz@kernel.org> X-Mailer: git-send-email 2.47.3 In-Reply-To: <20260802165327.385217-1-maz@kernel.org> References: <20260802165327.385217-1-maz@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-SA-Exim-Connect-IP: 185.219.108.64 X-SA-Exim-Rcpt-To: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, catalin.marinas@arm.com, will@kernel.org, mark.rutland@arm.com, tglx@kernel.org, ben.horgan@arm.com, daniel.lezcano@linaro.org X-SA-Exim-Mail-From: maz@kernel.org X-SA-Exim-Scanned: No (on disco-boy.misterjones.org); SAEximRunCond expanded to false Content-Type: text/plain; charset="utf-8" Relax the need for disabling preemption in __delay_cycles() by using arch_timer_read_vcounter(), which will disable preemption only when this is actually required. Acked-by: Catalin Marinas Signed-off-by: Marc Zyngier --- arch/arm64/lib/delay.c | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/arch/arm64/lib/delay.c b/arch/arm64/lib/delay.c index e278e060e78a9..a667df920697d 100644 --- a/arch/arm64/lib/delay.c +++ b/arch/arm64/lib/delay.c @@ -32,10 +32,9 @@ static inline unsigned long xloops_to_cycles(unsigned lo= ng xloops) * Note that userspace cannot change the offset behind our back either, * as the vcpu mutex is held as long as KVM_RUN is in progress. */ -static cycles_t notrace __delay_cycles(void) +static cycles_t __delay_cycles(void) { - guard(preempt_notrace)(); - return __arch_counter_get_cntvct_stable(); + return arch_timer_read_vcounter(); } =20 void __delay(unsigned long cycles) --=20 2.47.3