From nobody Mon May 11 04:11:53 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 22C8EC433EF for ; Fri, 15 Apr 2022 09:19:47 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1351716AbiDOJWM (ORCPT ); Fri, 15 Apr 2022 05:22:12 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49466 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S236043AbiDOJWI (ORCPT ); Fri, 15 Apr 2022 05:22:08 -0400 Received: from galois.linutronix.de (Galois.linutronix.de [IPv6:2a0a:51c0:0:12e:550::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 54C881A3AD for ; Fri, 15 Apr 2022 02:19:37 -0700 (PDT) Message-ID: <20220415091920.956045162@linutronix.de> DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1650014375; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: references:references; bh=noWWiVLpEJoThugg8Q2sCmImgY4XdCfFhEBVel656pk=; b=R+xxBPeZnUu0Sr99Y+kIAv5+gNnFVafzMvgCt0b+0Zf9VZjHK906ZcpZ8wvegz25nRYY89 6TjPIaVpWwp9/e4zExnjsGyOawokOVF8KoheErZVkuYKWikNhEzhw7YMAqx0PBM+x4IGCP /kJvVtVMpt1AsPIUeoPDWdEepcA1zt/xUDwa7nVfDBNtDrG5mTPHkKGBdScb0xIyILflvS gQXgrmESjxu6qCX3h0ctEKcjDB4RBP/aliP6kDubNMz3cRc0G/lymIx3UWZgnS4B2P1/dn jN2VIe7bRlvDbIvBZmX3aSe20iqc8bgxtKICuHfY2LZboPH7imoxEccxysuXOA== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1650014375; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: references:references; bh=noWWiVLpEJoThugg8Q2sCmImgY4XdCfFhEBVel656pk=; b=4uaoWYVBWKwpB2oDj7eU0kUNuQuDmfn4t9+DRjlqOOA2tmKQayum2Aryrd93/1Ql2iuSLy aWOVbbjdp9JI95Aw== From: Thomas Gleixner To: LKML Cc: Peter Zijlstra , John Stultz , Stephen Boyd Subject: [patch 1/3] timekeeping: Annotate ktime_get_boot_fast_ns() with data_race() References: <20220415091223.187169221@linutronix.de> MIME-Version: 1.0 Date: Fri, 15 Apr 2022 11:19:35 +0200 (CEST) Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Accessing timekeeper::offset_boot in ktime_get_boot_fast_ns() is an intended data race as the reader side cannot synchronize with a writer and there is no space in struct tk_read_base of the NMI safe timekeeper. Mark it so. Signed-off-by: Thomas Gleixner --- kernel/time/timekeeping.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) --- a/kernel/time/timekeeping.c +++ b/kernel/time/timekeeping.c @@ -528,7 +528,7 @@ u64 notrace ktime_get_boot_fast_ns(void) { struct timekeeper *tk =3D &tk_core.timekeeper; =20 - return (ktime_get_mono_fast_ns() + ktime_to_ns(tk->offs_boot)); + return (ktime_get_mono_fast_ns() + ktime_to_ns(data_race(tk->offs_boot))); } EXPORT_SYMBOL_GPL(ktime_get_boot_fast_ns); From nobody Mon May 11 04:11:53 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3379BC433EF for ; Fri, 15 Apr 2022 09:19:51 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1351729AbiDOJWQ (ORCPT ); Fri, 15 Apr 2022 05:22:16 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49464 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1351706AbiDOJWI (ORCPT ); Fri, 15 Apr 2022 05:22:08 -0400 Received: from galois.linutronix.de (Galois.linutronix.de [IPv6:2a0a:51c0:0:12e:550::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 54CD31C906 for ; Fri, 15 Apr 2022 02:19:38 -0700 (PDT) Message-ID: <20220415091921.014937522@linutronix.de> DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1650014377; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: references:references; bh=A2Bd4Iuntr6LsfOvKP9T0K7+QZfm+hdiWWJ5DcSeC7o=; b=UR0CYmeHCzZKQeVIOlwqI3keptYofbrykXAiQmgYGEdHHd6pykRNCiXZ4+p+l3J9iwv741 E1ULf/j5frMjn0MZbiLy0Ri9aff1VtYQrBdmOjSkfq6CkwDJWRVW32o4VVqXGX/97BopDT pEuuHj0lhaG/liN21zaZ1w55yQ+tNJx1ERjYgBLK0evpMsu0d6Urzyxx0jJxKM84HBqcHe PMgkjMV6Z+rN/x0Y12pMLXfoJfOT+Jo0FYAm+YnopyuTc0KD6mMWXNiyslFX6d/WLZSnbo sMuZH3mjv6rXk9uFkYS2FcdTpdS4CT62cu56pIobjcLy9CPppcads3V0BBXUjA== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1650014377; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: references:references; bh=A2Bd4Iuntr6LsfOvKP9T0K7+QZfm+hdiWWJ5DcSeC7o=; b=qU4J3AR2oTOqy5ZlXpBxfqO9ifItbnuBd7TS7vduOzIuO9YkkmBRy2WQV6050DrKm2dOBE jbZGZvX9ILBuSaBA== From: Thomas Gleixner To: LKML Cc: Peter Zijlstra , John Stultz , Stephen Boyd Subject: [patch 2/3] timekeeping: Mark tk_clock_read() __always_inline References: <20220415091223.187169221@linutronix.de> MIME-Version: 1.0 Date: Fri, 15 Apr 2022 11:19:36 +0200 (CEST) Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Compilers can uninline this which makes the notrace annotation of the NMI safe accessors moot. Signed-off-by: Thomas Gleixner --- kernel/time/timekeeping.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) --- a/kernel/time/timekeeping.c +++ b/kernel/time/timekeeping.c @@ -186,7 +186,7 @@ static inline void tk_update_sleep_time( * a read of the fast-timekeeper tkrs (which is protected by its own locki= ng * and update logic). */ -static inline u64 tk_clock_read(const struct tk_read_base *tkr) +static __always_inline u64 tk_clock_read(const struct tk_read_base *tkr) { struct clocksource *clock =3D READ_ONCE(tkr->clock); From nobody Mon May 11 04:11:53 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id E3E1DC433F5 for ; Fri, 15 Apr 2022 09:19:49 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1351723AbiDOJWP (ORCPT ); Fri, 15 Apr 2022 05:22:15 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49468 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1351705AbiDOJWI (ORCPT ); Fri, 15 Apr 2022 05:22:08 -0400 Received: from galois.linutronix.de (Galois.linutronix.de [IPv6:2a0a:51c0:0:12e:550::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 12C521CFEF for ; Fri, 15 Apr 2022 02:19:40 -0700 (PDT) Message-ID: <20220415091921.072296632@linutronix.de> DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1650014378; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: references:references; bh=EAbcvD3w/Qe5XFk2KaNTjkMhsV3VfKHB6dq2gEtX3x0=; b=sWHaIiPVM3s8LDFAOnl+hV1E4oYspbf6r1Zum5/5haqyt+89+pvLXo6PP1uQiE8tSSQ3hl sV7a+/3ypOazj8dCXnYHv0hE1a6zQAQsHkiKQygNttOkLrACgTjg+Y/NSjBJmwtD7gvh4/ nwj+V7MxvGE3WkkgR7THeUjLb8ugZQ/Vin+wZIz8tzvaZl/DMIt7X/9Y55aCI0/21X6VXZ u33xVZD/eisayrAHj8q7Zwriq+OzxFAk5XF+3eftO9zgpw7dWs92z3D5bvQ4jyqhlqpXBh mAR8UZhQ6Wz98BuWCHjCZRH5dqGcYhUADM0lFNIZ35vT8OPKSIuzqO+Qq7VwFw== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1650014378; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: references:references; bh=EAbcvD3w/Qe5XFk2KaNTjkMhsV3VfKHB6dq2gEtX3x0=; b=t4ilOYRan2rlKdrvnCSZOppW4U6oz9/xdDY4Qxoe+Wonpcj8fpxrq0JGk5MuL/L01+Wv15 1iJ2ueBEkf0x6uBw== From: Thomas Gleixner To: LKML Cc: Peter Zijlstra , John Stultz , Stephen Boyd Subject: [patch 3/3] timekeeping: Consolidate fast timekeeper References: <20220415091223.187169221@linutronix.de> MIME-Version: 1.0 Date: Fri, 15 Apr 2022 11:19:38 +0200 (CEST) Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Provide a inline function which replaces the copy & pasta. Signed-off-by: Thomas Gleixner Acked-by: Peter Zijlstra (Intel) --- kernel/time/timekeeping.c | 20 ++++++++++---------- 1 file changed, 10 insertions(+), 10 deletions(-) --- a/kernel/time/timekeeping.c +++ b/kernel/time/timekeeping.c @@ -429,6 +429,14 @@ static void update_fast_timekeeper(const memcpy(base + 1, base, sizeof(*base)); } =20 +static __always_inline u64 fast_tk_get_delta_ns(struct tk_read_base *tkr) +{ + u64 delta, cycles =3D tk_clock_read(tkr); + + delta =3D clocksource_delta(cycles, tkr->cycle_last, tkr->mask); + return timekeeping_delta_to_ns(tkr, delta); +} + static __always_inline u64 __ktime_get_fast_ns(struct tk_fast *tkf) { struct tk_read_base *tkr; @@ -439,12 +447,7 @@ static __always_inline u64 __ktime_get_f seq =3D raw_read_seqcount_latch(&tkf->seq); tkr =3D tkf->base + (seq & 0x01); now =3D ktime_to_ns(tkr->base); - - now +=3D timekeeping_delta_to_ns(tkr, - clocksource_delta( - tk_clock_read(tkr), - tkr->cycle_last, - tkr->mask)); + now +=3D fast_tk_get_delta_ns(tkr); } while (read_seqcount_latch_retry(&tkf->seq, seq)); =20 return now; @@ -560,10 +563,7 @@ static __always_inline u64 __ktime_get_r tkr =3D tkf->base + (seq & 0x01); basem =3D ktime_to_ns(tkr->base); baser =3D ktime_to_ns(tkr->base_real); - - delta =3D timekeeping_delta_to_ns(tkr, - clocksource_delta(tk_clock_read(tkr), - tkr->cycle_last, tkr->mask)); + delta =3D fast_tk_get_delta_ns(tkr); } while (read_seqcount_latch_retry(&tkf->seq, seq)); =20 if (mono)