1. Patchew
  2. linux
  3. [PATCH 0/5] tick/nohz: cleanups and fixes v2
  4. View patch

[PATCH 3/5] tick/nohz: Don't shutdown the lowres tick from itself

Frederic Weisbecker posted 5 patches 2 years, 3 months ago
[PATCH 3/5] tick/nohz: Don't shutdown the lowres tick from itself
Posted by Frederic Weisbecker 2 years, 3 months ago 
In lowres dynticks mode, just like in highres dynticks mode, when there
is no tick to program in the future, the tick eventually gets
deactivated either:

* From the idle loop if in idle mode.
* From the IRQ exit if in full dynticks mode.

Therefore there is no need to deactivate it from the tick itself. This
just just brings more overhead in the idle tick path for no reason.

Fixes: 62c1256d5447 ("timers/nohz: Switch to ONESHOT_STOPPED in the low-res handler when the tick is stopped")
Signed-off-by: Frederic Weisbecker <frederic@kernel.org>
---
 kernel/time/tick-sched.c | 24 +++++++++++++-----------
 1 file changed, 13 insertions(+), 11 deletions(-)

diff --git a/kernel/time/tick-sched.c b/kernel/time/tick-sched.c
index 95a8d1d118a2..8e9a9dcf60d5 100644
--- a/kernel/time/tick-sched.c
+++ b/kernel/time/tick-sched.c
@@ -1403,18 +1403,16 @@ static void tick_nohz_lowres_handler(struct clock_event_device *dev)
 	tick_sched_do_timer(ts, now);
 	tick_sched_handle(ts, regs);
 
-	if (unlikely(ts->tick_stopped)) {
-		/*
-		 * The clockevent device is not reprogrammed, so change the
-		 * clock event device to ONESHOT_STOPPED to avoid spurious
-		 * interrupts on devices which might not be truly one shot.
-		 */
-		tick_program_event(KTIME_MAX, 1);
-		return;
+	/*
+	 * In dynticks mode, tick reprogram is deferred:
+	 * - to the idle task if in dynticks-idle
+	 * - to IRQ exit if in full-dynticks.
+	 */
+	if (likely(!ts->tick_stopped)) {
+		hrtimer_forward(&ts->sched_timer, now, TICK_NSEC);
+		tick_program_event(hrtimer_get_expires(&ts->sched_timer), 1);
 	}
 
-	hrtimer_forward(&ts->sched_timer, now, TICK_NSEC);
-	tick_program_event(hrtimer_get_expires(&ts->sched_timer), 1);
 }
 
 static inline void tick_nohz_activate(struct tick_sched *ts, int mode)
@@ -1519,7 +1517,11 @@ static enum hrtimer_restart tick_nohz_highres_handler(struct hrtimer *timer)
 	else
 		ts->next_tick = 0;
 
-	/* No need to reprogram if we are in idle or full dynticks mode */
+	/*
+	 * In dynticks mode, tick reprogram is deferred:
+	 * - to the idle task if in dynticks-idle
+	 * - to IRQ exit if in full-dynticks.
+	 */
 	if (unlikely(ts->tick_stopped))
 		return HRTIMER_NORESTART;
 
-- 
2.34.1
Re: [PATCH 3/5] tick/nohz: Don't shutdown the lowres tick from itself
Posted by Joel Fernandes 2 years, 3 months ago 
On Tue, Sep 12, 2023 at 6:44 AM Frederic Weisbecker <frederic@kernel.org> wrote:
>
> In lowres dynticks mode, just like in highres dynticks mode, when there
> is no tick to program in the future, the tick eventually gets
> deactivated either:
>
> * From the idle loop if in idle mode.
> * From the IRQ exit if in full dynticks mode.
>
> Therefore there is no need to deactivate it from the tick itself. This
> just just brings more overhead in the idle tick path for no reason.
>
> Fixes: 62c1256d5447 ("timers/nohz: Switch to ONESHOT_STOPPED in the low-res handler when the tick is stopped")
> Signed-off-by: Frederic Weisbecker <frederic@kernel.org>

If on some weird hardware, say  ts->next_tick = KTIME_MAX but a
spurious timer interrupt went off and tick_nohz_handler() did get
called (yeah weird hypothetical situation), then in
tick_nohz_stop_tick() we might early return from:

/* Skip reprogram of event if its not changed */
if (ts->tick_stopped && (expires == ts->next_tick))

without no "eventual" reprogramming.

Maybe we should also reprogram with KTIME_MAX in such a situation?
Then we can get rid of it from tick_nohz_handler() for the common case
as you are doing.

So for weird hardware, with this patch we are not doing an extra
tick_program_event(KTIME_MAX, 1); like Nick was doing. That makes me a
tad bit nervous.

Otherwise your patch looks correct to me (for hardware that tends not
to misbehave).

thanks,

 - Joel


> ---
>  kernel/time/tick-sched.c | 24 +++++++++++++-----------
>  1 file changed, 13 insertions(+), 11 deletions(-)
>
> diff --git a/kernel/time/tick-sched.c b/kernel/time/tick-sched.c
> index 95a8d1d118a2..8e9a9dcf60d5 100644
> --- a/kernel/time/tick-sched.c
> +++ b/kernel/time/tick-sched.c
> @@ -1403,18 +1403,16 @@ static void tick_nohz_lowres_handler(struct clock_event_device *dev)
>         tick_sched_do_timer(ts, now);
>         tick_sched_handle(ts, regs);
>
> -       if (unlikely(ts->tick_stopped)) {
> -               /*
> -                * The clockevent device is not reprogrammed, so change the
> -                * clock event device to ONESHOT_STOPPED to avoid spurious
> -                * interrupts on devices which might not be truly one shot.
> -                */
> -               tick_program_event(KTIME_MAX, 1);
> -               return;
> +       /*
> +        * In dynticks mode, tick reprogram is deferred:
> +        * - to the idle task if in dynticks-idle
> +        * - to IRQ exit if in full-dynticks.
> +        */
> +       if (likely(!ts->tick_stopped)) {
> +               hrtimer_forward(&ts->sched_timer, now, TICK_NSEC);
> +               tick_program_event(hrtimer_get_expires(&ts->sched_timer), 1);
>         }
>
> -       hrtimer_forward(&ts->sched_timer, now, TICK_NSEC);
> -       tick_program_event(hrtimer_get_expires(&ts->sched_timer), 1);
>  }
>
>  static inline void tick_nohz_activate(struct tick_sched *ts, int mode)
> @@ -1519,7 +1517,11 @@ static enum hrtimer_restart tick_nohz_highres_handler(struct hrtimer *timer)
>         else
>                 ts->next_tick = 0;
>
> -       /* No need to reprogram if we are in idle or full dynticks mode */
> +       /*
> +        * In dynticks mode, tick reprogram is deferred:
> +        * - to the idle task if in dynticks-idle
> +        * - to IRQ exit if in full-dynticks.
> +        */
>         if (unlikely(ts->tick_stopped))
>                 return HRTIMER_NORESTART;
>
> --
> 2.34.1
>
Re: [PATCH 3/5] tick/nohz: Don't shutdown the lowres tick from itself
Posted by Frederic Weisbecker 2 years, 3 months ago 
On Wed, Sep 13, 2023 at 09:17:21PM -0400, Joel Fernandes wrote:
> On Tue, Sep 12, 2023 at 6:44 AM Frederic Weisbecker <frederic@kernel.org> wrote:
> >
> > In lowres dynticks mode, just like in highres dynticks mode, when there
> > is no tick to program in the future, the tick eventually gets
> > deactivated either:
> >
> > * From the idle loop if in idle mode.
> > * From the IRQ exit if in full dynticks mode.
> >
> > Therefore there is no need to deactivate it from the tick itself. This
> > just just brings more overhead in the idle tick path for no reason.
> >
> > Fixes: 62c1256d5447 ("timers/nohz: Switch to ONESHOT_STOPPED in the low-res handler when the tick is stopped")
> > Signed-off-by: Frederic Weisbecker <frederic@kernel.org>
> 
> If on some weird hardware, say  ts->next_tick = KTIME_MAX but a
> spurious timer interrupt went off and tick_nohz_handler() did get
> called (yeah weird hypothetical situation), then in
> tick_nohz_stop_tick() we might early return from:
> 
> /* Skip reprogram of event if its not changed */
> if (ts->tick_stopped && (expires == ts->next_tick))
> 
> without no "eventual" reprogramming.
> 
> Maybe we should also reprogram with KTIME_MAX in such a situation?
> Then we can get rid of it from tick_nohz_handler() for the common case
> as you are doing.
> 
> So for weird hardware, with this patch we are not doing an extra
> tick_program_event(KTIME_MAX, 1); like Nick was doing. That makes me a
> tad bit nervous.

So when a tick happens, ts->next_tick is reset to 0 (in tick_sched_handle()).
This way if a timer interrupt fires too early, and that includes also timer
interrupts when next_tick is KTIME_MAX, the timer is always reprogrammed upon
the next idle loop iteration. So this shouldn't happen.

Thanks.
Re: [PATCH 3/5] tick/nohz: Don't shutdown the lowres tick from itself
Posted by Joel Fernandes 2 years, 3 months ago 
On Thu, Sep 14, 2023 at 5:29 AM Frederic Weisbecker <frederic@kernel.org> wrote:
>
> On Wed, Sep 13, 2023 at 09:17:21PM -0400, Joel Fernandes wrote:
> > On Tue, Sep 12, 2023 at 6:44 AM Frederic Weisbecker <frederic@kernel.org> wrote:
> > >
> > > In lowres dynticks mode, just like in highres dynticks mode, when there
> > > is no tick to program in the future, the tick eventually gets
> > > deactivated either:
> > >
> > > * From the idle loop if in idle mode.
> > > * From the IRQ exit if in full dynticks mode.
> > >
> > > Therefore there is no need to deactivate it from the tick itself. This
> > > just just brings more overhead in the idle tick path for no reason.
> > >
> > > Fixes: 62c1256d5447 ("timers/nohz: Switch to ONESHOT_STOPPED in the low-res handler when the tick is stopped")
> > > Signed-off-by: Frederic Weisbecker <frederic@kernel.org>
> >
> > If on some weird hardware, say  ts->next_tick = KTIME_MAX but a
> > spurious timer interrupt went off and tick_nohz_handler() did get
> > called (yeah weird hypothetical situation), then in
> > tick_nohz_stop_tick() we might early return from:
> >
> > /* Skip reprogram of event if its not changed */
> > if (ts->tick_stopped && (expires == ts->next_tick))
> >
> > without no "eventual" reprogramming.
> >
> > Maybe we should also reprogram with KTIME_MAX in such a situation?
> > Then we can get rid of it from tick_nohz_handler() for the common case
> > as you are doing.
> >
> > So for weird hardware, with this patch we are not doing an extra
> > tick_program_event(KTIME_MAX, 1); like Nick was doing. That makes me a
> > tad bit nervous.
>
> So when a tick happens, ts->next_tick is reset to 0 (in tick_sched_handle()).
> This way if a timer interrupt fires too early, and that includes also timer
> interrupts when next_tick is KTIME_MAX, the timer is always reprogrammed upon
> the next idle loop iteration. So this shouldn't happen.

Ah you are right, I missed that. So then I am good with it:
Reviewed-by: Joel Fernandes (Google) <joel@joelfernandes.org>

thanks,

 - Joel
[tip: timers/core] tick/nohz: Don't shutdown the lowres tick from itself
Posted by tip-bot2 for Frederic Weisbecker 2 years, 2 months ago 
The following commit has been merged into the timers/core branch of tip:

Commit-ID:     4f7f4409af289715f44685f250e380ce2cbffc7e
Gitweb:        https://git.kernel.org/tip/4f7f4409af289715f44685f250e380ce2cbffc7e
Author:        Frederic Weisbecker <frederic@kernel.org>
AuthorDate:    Tue, 12 Sep 2023 12:44:04 +02:00
Committer:     Thomas Gleixner <tglx@linutronix.de>
CommitterDate: Wed, 27 Sep 2023 16:58:10 +02:00

tick/nohz: Don't shutdown the lowres tick from itself

In lowres dynticks mode, just like in highres dynticks mode, when there
is no tick to program in the future, the tick eventually gets
deactivated either:

  * From the idle loop if in idle mode.
  * From the IRQ exit if in full dynticks mode.

Therefore there is no need to deactivate it from the tick itself. This
just just brings more overhead in the idle tick path for no reason.

Signed-off-by: Frederic Weisbecker <frederic@kernel.org>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Reviewed-by: Joel Fernandes (Google) <joel@joelfernandes.org>
Link: https://lore.kernel.org/r/20230912104406.312185-4-frederic@kernel.org

---
 kernel/time/tick-sched.c | 24 +++++++++++++-----------
 1 file changed, 13 insertions(+), 11 deletions(-)

diff --git a/kernel/time/tick-sched.c b/kernel/time/tick-sched.c
index 95a8d1d..8e9a9dc 100644
--- a/kernel/time/tick-sched.c
+++ b/kernel/time/tick-sched.c
@@ -1403,18 +1403,16 @@ static void tick_nohz_lowres_handler(struct clock_event_device *dev)
 	tick_sched_do_timer(ts, now);
 	tick_sched_handle(ts, regs);
 
-	if (unlikely(ts->tick_stopped)) {
-		/*
-		 * The clockevent device is not reprogrammed, so change the
-		 * clock event device to ONESHOT_STOPPED to avoid spurious
-		 * interrupts on devices which might not be truly one shot.
-		 */
-		tick_program_event(KTIME_MAX, 1);
-		return;
+	/*
+	 * In dynticks mode, tick reprogram is deferred:
+	 * - to the idle task if in dynticks-idle
+	 * - to IRQ exit if in full-dynticks.
+	 */
+	if (likely(!ts->tick_stopped)) {
+		hrtimer_forward(&ts->sched_timer, now, TICK_NSEC);
+		tick_program_event(hrtimer_get_expires(&ts->sched_timer), 1);
 	}
 
-	hrtimer_forward(&ts->sched_timer, now, TICK_NSEC);
-	tick_program_event(hrtimer_get_expires(&ts->sched_timer), 1);
 }
 
 static inline void tick_nohz_activate(struct tick_sched *ts, int mode)
@@ -1519,7 +1517,11 @@ static enum hrtimer_restart tick_nohz_highres_handler(struct hrtimer *timer)
 	else
 		ts->next_tick = 0;
 
-	/* No need to reprogram if we are in idle or full dynticks mode */
+	/*
+	 * In dynticks mode, tick reprogram is deferred:
+	 * - to the idle task if in dynticks-idle
+	 * - to IRQ exit if in full-dynticks.
+	 */
 	if (unlikely(ts->tick_stopped))
 		return HRTIMER_NORESTART;

Patchew 2.1-devel-b67e4c2

© 2016 - 2025 Red Hat, Inc.