From nobody Sat Feb 7 18:56:14 2026 Received: from mail-pj1-f74.google.com (mail-pj1-f74.google.com [209.85.216.74]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5A60514EC4E for ; Tue, 28 Jan 2025 06:33:16 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.74 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1738045998; cv=none; b=cMBzq4dbKZoYCGtyAa9tEW1ZgS9k4SGTQQASiTvoimQczTYNY72ksEY+KS6jOBsnGpSSSz/q6vs6EDbgukFhohlwjyA3LZ5TKVgFzOTSe9gC1vEkJboqmltCl6EkAQ69K+7p70+k1EvZKoJYvZxqOG1h6v4DrkSs4L72rBosUzY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1738045998; c=relaxed/simple; bh=Ywx2lDMIKbEPFsCOyAZdE+f0b/tHPcwWm2RNe/nz3EM=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=PPYXCZuAe3CMwGyTfy37KN5JFJFxNoG+bCBRLrEUYXI4ogSpjjMhqDCCZtGgSEi6MbMNwbN0TwxdA6iv8ZcbvuCV2upyvz2INspm1XZ051BVbCmrcf3A0ddkzADLBXS/EhCEKCUTDRDdk/IFSDO+4zakN4VLHywhZ1K9DYd602s= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--jstultz.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=gCjra4Ht; arc=none smtp.client-ip=209.85.216.74 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--jstultz.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="gCjra4Ht" Received: by mail-pj1-f74.google.com with SMTP id 98e67ed59e1d1-2efa0eb9dacso10634765a91.1 for ; Mon, 27 Jan 2025 22:33:16 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1738045995; x=1738650795; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=bmHDIoXwtLIeK+W+R0oS1uWFT5fHAwfyn6cZe4DJK/Y=; b=gCjra4Ht+Q6a4HhftzX9cn3XDUcRsNESYCKu+G7JS2bWC8chTE2nEcqkfCOdiBqWAy TpOj+XZJ2OiDcDI6vNz7bjyO0b5ogLojFPXj3A/PqHkKMhtDJ1yqmX7RvSl4IswSOKWf KeM5roJCs4VD9sQuUKb94DXAttCdMHzZ6DRUqeun7+H5iz+hEPXrfH6D5Qp3mwSv+8uy 4xaPKmntb44SExW2NDGPikEjf34ILGdzCf+2BBRvs4V/8PpDeYMgQQ1J5QJqAMayoEPE JC5b/gQa3dwg/c6Cf4OHwN34E3jov2cEByO5pdnsPgmr8rjbXcn/7nfNHh6p5tKw3lCo cyAQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1738045995; x=1738650795; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=bmHDIoXwtLIeK+W+R0oS1uWFT5fHAwfyn6cZe4DJK/Y=; b=jShPkXV+km+EMYYoEychCdtLAtFl8tB8DlvXDUZmD3pCWff1Y4MIxKO0+pHBu8KGu8 zYQvYtZZQbnTORmQxuaj1zXR+jsNDX0FN2nmPT9jHCC25CsAAzMIkGiNTq8EOTaxpxA8 ob43odncr5jYh+k04nuneqczJ495ihhMtYA15r2ALFZkbz0idtNdA4dKLryvHvGq5izl TFuf41VJHBgNErgVvifNxnCdfLaaGHwsp0jHKuvRMKw63o6VmxYlPc8wk2l/UT831bHk fg7RfnkMDp1+nPtgxbAmVu7G+LY+KDifmdCKWZ4S00n5OWcpkTlP82TZ+wx2fPRX8ACM Tnlw== X-Gm-Message-State: AOJu0Yw3N0kpyrkQHbRJ211sryIHB3NvOsMvOEEkS1OF2CueZtQYa5N5 eqi6x+VnPbEgfkVCd9wbXzK1jDwbKO0G8W8JRM5tTFBKPVCXGYszQbhQHjSdjvdNXJf7lGTCwg8 +5EJyzTAIzsuZFjangDnHfQ7aGj44R4UYgU8JWEwfYVbimfGE5dyRZf6rQx+VgcxN2Ry6UwYqll Wpj7324B7WVun+qp2kkEVsgKqLpPgub5meU3ApqtcyHMiq X-Google-Smtp-Source: AGHT+IEXqr6TT88yGAcYv0DY03C+DEGJCyZmakQgb/hJ/Lt1Ia4Y+KWmECiSC//VtC4cXlJlFpx1zerC/YlN X-Received: from pjwx5.prod.google.com ([2002:a17:90a:c2c5:b0:2f5:4762:e778]) (user=jstultz job=prod-delivery.src-stubby-dispatcher) by 2002:a17:90b:280e:b0:2ee:7c65:ae8e with SMTP id 98e67ed59e1d1-2f782c700c6mr63129172a91.11.1738045995314; Mon, 27 Jan 2025 22:33:15 -0800 (PST) Date: Mon, 27 Jan 2025 22:32:53 -0800 In-Reply-To: <20250128063301.3879317-1-jstultz@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250128063301.3879317-1-jstultz@google.com> X-Mailer: git-send-email 2.48.1.262.g85cc9f2d1e-goog Message-ID: <20250128063301.3879317-2-jstultz@google.com> Subject: [RFC][PATCH 1/3] time/tick: Pipe tick count down through cputime accounting From: John Stultz To: LKML Cc: John Stultz , Anna-Maria Behnsen , Frederic Weisbecker , Ingo Molnar , Thomas Gleixner , Peter Zijlstra , Juri Lelli , Vincent Guittot , Dietmar Eggemann , Steven Rostedt , Ben Segall , Mel Gorman , Valentin Schneider , Stephen Boyd , Yury Norov , Bitao Hu , Andrew Morton , kernel-team@android.com Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" In working up the dynHZ patch, I found that the skipping of ticks would result in large latencies for itimers. As I dug into it, I realized there is still some logic that assumes we don't miss ticks, resulting in late expiration of cputime timers. So this patch pipes the actual number of ticks passed down through cputime accounting. Cc: Anna-Maria Behnsen Cc: Frederic Weisbecker Cc: Ingo Molnar Cc: Thomas Gleixner Cc: Peter Zijlstra Cc: Juri Lelli Cc: Vincent Guittot Cc: Dietmar Eggemann Cc: Steven Rostedt Cc: Ben Segall Cc: Mel Gorman Cc: Valentin Schneider Cc: Stephen Boyd Cc: Yury Norov Cc: Bitao Hu Cc: Andrew Morton Cc: kernel-team@android.com Signed-off-by: John Stultz --- include/linux/kernel_stat.h | 4 ++-- kernel/sched/cputime.c | 6 +++--- kernel/time/tick-common.c | 2 +- kernel/time/tick-legacy.c | 2 +- kernel/time/tick-sched.c | 29 ++++++++++++++++------------- kernel/time/timekeeping.h | 2 +- kernel/time/timer.c | 4 ++-- 7 files changed, 26 insertions(+), 23 deletions(-) diff --git a/include/linux/kernel_stat.h b/include/linux/kernel_stat.h index b97ce2df376f9..4b5169cd8db04 100644 --- a/include/linux/kernel_stat.h +++ b/include/linux/kernel_stat.h @@ -127,12 +127,12 @@ extern void account_idle_time(u64); extern u64 get_idle_time(struct kernel_cpustat *kcs, int cpu); =20 #ifdef CONFIG_VIRT_CPU_ACCOUNTING_NATIVE -static inline void account_process_tick(struct task_struct *tsk, int user) +static inline void account_process_tick(struct task_struct *tsk, unsigned = long ticks, int user) { vtime_flush(tsk); } #else -extern void account_process_tick(struct task_struct *, int user); +extern void account_process_tick(struct task_struct *, unsigned long ticks= , int user); #endif =20 extern void account_idle_ticks(unsigned long ticks); diff --git a/kernel/sched/cputime.c b/kernel/sched/cputime.c index 0bed0fa1acd98..9948da0d80842 100644 --- a/kernel/sched/cputime.c +++ b/kernel/sched/cputime.c @@ -471,7 +471,7 @@ void thread_group_cputime_adjusted(struct task_struct *= p, u64 *ut, u64 *st) * @p: the process that the CPU time gets accounted to * @user_tick: indicates if the tick is a user or a system tick */ -void account_process_tick(struct task_struct *p, int user_tick) +void account_process_tick(struct task_struct *p, unsigned long ticks, int = user_tick) { u64 cputime, steal; =20 @@ -479,11 +479,11 @@ void account_process_tick(struct task_struct *p, int = user_tick) return; =20 if (sched_clock_irqtime) { - irqtime_account_process_tick(p, user_tick, 1); + irqtime_account_process_tick(p, user_tick, ticks); return; } =20 - cputime =3D TICK_NSEC; + cputime =3D ticks * TICK_NSEC; steal =3D steal_account_process_time(ULONG_MAX); =20 if (steal >=3D cputime) diff --git a/kernel/time/tick-common.c b/kernel/time/tick-common.c index a47bcf71defcf..ae5c5befdc58b 100644 --- a/kernel/time/tick-common.c +++ b/kernel/time/tick-common.c @@ -98,7 +98,7 @@ static void tick_periodic(int cpu) update_wall_time(); } =20 - update_process_times(user_mode(get_irq_regs())); + update_process_times(1, user_mode(get_irq_regs())); profile_tick(CPU_PROFILING); } =20 diff --git a/kernel/time/tick-legacy.c b/kernel/time/tick-legacy.c index af225b32f5b37..dbc156e69802b 100644 --- a/kernel/time/tick-legacy.c +++ b/kernel/time/tick-legacy.c @@ -32,6 +32,6 @@ void legacy_timer_tick(unsigned long ticks) raw_spin_unlock(&jiffies_lock); update_wall_time(); } - update_process_times(user_mode(get_irq_regs())); + update_process_times(ticks, user_mode(get_irq_regs())); profile_tick(CPU_PROFILING); } diff --git a/kernel/time/tick-sched.c b/kernel/time/tick-sched.c index fa058510af9c1..983790923aee9 100644 --- a/kernel/time/tick-sched.c +++ b/kernel/time/tick-sched.c @@ -54,7 +54,7 @@ static ktime_t last_jiffies_update; /* * Must be called with interrupts disabled ! */ -static void tick_do_update_jiffies64(ktime_t now) +static unsigned long tick_do_update_jiffies64(ktime_t now) { unsigned long ticks =3D 1; ktime_t delta, nextp; @@ -70,7 +70,7 @@ static void tick_do_update_jiffies64(ktime_t now) */ if (IS_ENABLED(CONFIG_64BIT)) { if (ktime_before(now, smp_load_acquire(&tick_next_period))) - return; + return 0; } else { unsigned int seq; =20 @@ -84,7 +84,7 @@ static void tick_do_update_jiffies64(ktime_t now) } while (read_seqcount_retry(&jiffies_seq, seq)); =20 if (ktime_before(now, nextp)) - return; + return 0; } =20 /* Quick check failed, i.e. update is required. */ @@ -95,7 +95,7 @@ static void tick_do_update_jiffies64(ktime_t now) */ if (ktime_before(now, tick_next_period)) { raw_spin_unlock(&jiffies_lock); - return; + return 0; } =20 write_seqcount_begin(&jiffies_seq); @@ -147,6 +147,7 @@ static void tick_do_update_jiffies64(ktime_t now) =20 raw_spin_unlock(&jiffies_lock); update_wall_time(); + return ticks; } =20 /* @@ -203,10 +204,10 @@ static inline void tick_sched_flag_clear(struct tick_= sched *ts, =20 #define MAX_STALLED_JIFFIES 5 =20 -static void tick_sched_do_timer(struct tick_sched *ts, ktime_t now) +static unsigned long tick_sched_do_timer(struct tick_sched *ts, ktime_t no= w) { int tick_cpu, cpu =3D smp_processor_id(); - + unsigned long ticks =3D 0; /* * Check if the do_timer duty was dropped. We don't care about * concurrency: This happens only when the CPU in charge went @@ -229,7 +230,7 @@ static void tick_sched_do_timer(struct tick_sched *ts, = ktime_t now) =20 /* Check if jiffies need an update */ if (tick_cpu =3D=3D cpu) - tick_do_update_jiffies64(now); + ticks =3D tick_do_update_jiffies64(now); =20 /* * If the jiffies update stalled for too long (timekeeper in stop_machine= () @@ -240,7 +241,7 @@ static void tick_sched_do_timer(struct tick_sched *ts, = ktime_t now) ts->last_tick_jiffies =3D READ_ONCE(jiffies); } else { if (++ts->stalled_jiffies =3D=3D MAX_STALLED_JIFFIES) { - tick_do_update_jiffies64(now); + ticks +=3D tick_do_update_jiffies64(now); ts->stalled_jiffies =3D 0; ts->last_tick_jiffies =3D READ_ONCE(jiffies); } @@ -248,9 +249,10 @@ static void tick_sched_do_timer(struct tick_sched *ts,= ktime_t now) =20 if (tick_sched_flag_test(ts, TS_FLAG_INIDLE)) ts->got_idle_tick =3D 1; + return ticks; } =20 -static void tick_sched_handle(struct tick_sched *ts, struct pt_regs *regs) +static void tick_sched_handle(struct tick_sched *ts, unsigned long ticks, = struct pt_regs *regs) { /* * When we are idle and the tick is stopped, we have to touch @@ -264,7 +266,7 @@ static void tick_sched_handle(struct tick_sched *ts, st= ruct pt_regs *regs) tick_sched_flag_test(ts, TS_FLAG_STOPPED)) { touch_softlockup_watchdog_sched(); if (is_idle_task(current)) - ts->idle_jiffies++; + ts->idle_jiffies +=3D ticks; /* * In case the current tick fired too early past its expected * expiration, make sure we don't bypass the next clock reprogramming @@ -273,7 +275,7 @@ static void tick_sched_handle(struct tick_sched *ts, st= ruct pt_regs *regs) ts->next_tick =3D 0; } =20 - update_process_times(user_mode(regs)); + update_process_times(ticks, user_mode(regs)); profile_tick(CPU_PROFILING); } =20 @@ -286,15 +288,16 @@ static enum hrtimer_restart tick_nohz_handler(struct = hrtimer *timer) struct tick_sched *ts =3D container_of(timer, struct tick_sched, sched_ti= mer); struct pt_regs *regs =3D get_irq_regs(); ktime_t now =3D ktime_get(); + unsigned long ticks; =20 - tick_sched_do_timer(ts, now); + ticks =3D tick_sched_do_timer(ts, now); =20 /* * Do not call when we are not in IRQ context and have * no valid 'regs' pointer */ if (regs) - tick_sched_handle(ts, regs); + tick_sched_handle(ts, ticks, regs); else ts->next_tick =3D 0; =20 diff --git a/kernel/time/timekeeping.h b/kernel/time/timekeeping.h index 543beba096c75..3f93af06ce5c3 100644 --- a/kernel/time/timekeeping.h +++ b/kernel/time/timekeeping.h @@ -22,7 +22,7 @@ static inline int sched_clock_suspend(void) { return 0; } static inline void sched_clock_resume(void) { } #endif =20 -extern void update_process_times(int user); +extern void update_process_times(unsigned long ticks, int user); extern void do_timer(unsigned long ticks); extern void update_wall_time(void); =20 diff --git a/kernel/time/timer.c b/kernel/time/timer.c index a5860bf6d16f9..335ed23d1cb2d 100644 --- a/kernel/time/timer.c +++ b/kernel/time/timer.c @@ -2509,12 +2509,12 @@ static void run_local_timers(void) * Called from the timer interrupt handler to charge one tick to the curre= nt * process. user_tick is 1 if the tick is user time, 0 for system. */ -void update_process_times(int user_tick) +void update_process_times(unsigned long ticks, int user_tick) { struct task_struct *p =3D current; =20 /* Note: this timer irq context must be accounted for as well. */ - account_process_tick(p, user_tick); + account_process_tick(p, ticks, user_tick); run_local_timers(); rcu_sched_clock_irq(user_tick); #ifdef CONFIG_IRQ_WORK --=20 2.48.1.262.g85cc9f2d1e-goog