From nobody Mon Jun 8 05:26:21 2026 Received: from galois.linutronix.de (Galois.linutronix.de [193.142.43.55]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7402F339861; Tue, 2 Jun 2026 19:30:10 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=193.142.43.55 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780428612; cv=none; b=dBnpdwKSjZ7z4l3qeZG42nw3+Nkl+gYt+qWy5MuUEJJ4GiEtR/Y1hHcUE9Clk6OiW3hB7nukIDSo47oZuNHzc3xGxTvvc5Y+Pf7lwut0mckaf3ofcsmp8dkXWqivoS4FY33YvTx7/8PMIz2DQjDztJS8reXEu15fDPyT4X5Nqvo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780428612; c=relaxed/simple; bh=GOFzIzeoPsvS/BUYaUjVV4F7Oa4dxgO9dgVz2oG1zbg=; h=Date:From:To:Subject:Cc:In-Reply-To:References:MIME-Version: Message-ID:Content-Type; b=ArUPwhm1bxUG2JvpZU+qyZ6mfnk4pST19AS3d7ayzVL0JbSh6HAlOiit6AFFsXhTCiYYFMXY8pj3plY+ej0TTPOyt4W68Joey5KCuj/z/S7xmb2fsSHttk4ObhiYlJUZZqRG7ZcNG/tlEhoNSrqZ83FbpePhFe7G14BWiCreCXk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linutronix.de; spf=pass smtp.mailfrom=linutronix.de; dkim=pass (2048-bit key) header.d=linutronix.de header.i=@linutronix.de header.b=meVmuIP0; dkim=permerror (0-bit key) header.d=linutronix.de header.i=@linutronix.de header.b=W5VNTWh0; arc=none smtp.client-ip=193.142.43.55 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linutronix.de Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linutronix.de Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=linutronix.de header.i=@linutronix.de header.b="meVmuIP0"; dkim=permerror (0-bit key) header.d=linutronix.de header.i=@linutronix.de header.b="W5VNTWh0" Date: Tue, 02 Jun 2026 19:30:07 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1780428608; h=from:from:sender:sender:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=hLvE3+crlIgwfZocs95mhpZ9nn1isKcG2PxC9Dph/4k=; b=meVmuIP0DYMHuqI3EuHcnMSeeVePCFM3ExAuuAT7cDxpQKbAUsbo+73UWAsyC76NdEpgb0 gTY4rnfBhXHiLS5QUyvOZJVu3Xhowq2NFJOxTTDYhitCXHw3BsQhgkjEDeWxgnLyERUPt7 TEi5BFcJA9e8xRAzUODxYQ5ATzE1bXfDc2cznFbUaXmVJqw74UaDl94l9LYyZBDe38bR7X jUDS8gc3O2vlMaciGb/JxJrxqkVGgUfboQdRFUA+wGMlzCwAb6nrVg+Y5d5zWSpRDkeEUx Mg/GTG6MVgmVN/Oq37adEd0HRy9lkkfm3wPNu5NnjY66RA4yixEyRuknXLDPXg== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1780428608; h=from:from:sender:sender:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=hLvE3+crlIgwfZocs95mhpZ9nn1isKcG2PxC9Dph/4k=; b=W5VNTWh0awIDwA+PGqIEUpj+DTem9w6rP0yIF44QX1Tapg/IFbNkCcxxVrFOtEQTjzyZjk I25qLgfua/dyR7Cw== From: "tip-bot2 for Frederic Weisbecker" Sender: tip-bot2@linutronix.de Reply-to: linux-kernel@vger.kernel.org To: linux-tip-commits@vger.kernel.org Subject: [tip: timers/nohz] tick/sched: Consolidate idle time fetching APIs Cc: Frederic Weisbecker , Thomas Gleixner , Shrikanth Hegde , x86@kernel.org, linux-kernel@vger.kernel.org In-Reply-To: <20260508131647.43868-13-frederic@kernel.org> References: <20260508131647.43868-13-frederic@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-ID: <178042860742.710.1580286849897142847.tip-bot2@tip-bot2> Robot-ID: Robot-Unsubscribe: Contact to get blacklisted from these emails Precedence: bulk Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable The following commit has been merged into the timers/nohz branch of tip: Commit-ID: 127b2eb44f36d5d7059f1af425b5800cb27440f9 Gitweb: https://git.kernel.org/tip/127b2eb44f36d5d7059f1af425b5800cb= 27440f9 Author: Frederic Weisbecker AuthorDate: Fri, 08 May 2026 15:16:44 +02:00 Committer: Thomas Gleixner CommitterDate: Tue, 02 Jun 2026 21:27:26 +02:00 tick/sched: Consolidate idle time fetching APIs Fetching the idle cputime is available through a variety of accessors all over the place depending on the different accounting flavours and needs: - idle vtime generic accounting can be accessed by kcpustat_field(), kcpustat_cpu_fetch(), get_idle/iowait_time() and get_cpu_idle/iowait_time_us() - dynticks-idle accounting can only be accessed by get_idle/iowait_time() or get_cpu_idle/iowait_time_us() - CONFIG_NO_HZ_COMMON=3Dn idle accounting can be accessed by kcpustat_fie= ld() kcpustat_cpu_fetch(), or get_idle/iowait_time() but not by get_cpu_idle/iowait_time_us() Moreover get_idle/iowait_time() relies on get_cpu_idle/iowait_time_us() with a non-sensical conversion to microseconds and back to nanoseconds on the way. Start consolidating the APIs with removing get_idle/iowait_time() and make kcpustat_field() and kcpustat_cpu_fetch() work for all cases. Signed-off-by: Frederic Weisbecker Signed-off-by: Thomas Gleixner Tested-by: Shrikanth Hegde Link: https://patch.msgid.link/20260508131647.43868-13-frederic@kernel.org --- fs/proc/stat.c | 40 ++---------------------- fs/proc/uptime.c | 8 +---- include/linux/kernel_stat.h | 34 +++++++++++++++++--- kernel/sched/cputime.c | 61 +++++++++++++++++++++++------------- 4 files changed, 76 insertions(+), 67 deletions(-) diff --git a/fs/proc/stat.c b/fs/proc/stat.c index 8b444e8..c00468a 100644 --- a/fs/proc/stat.c +++ b/fs/proc/stat.c @@ -22,38 +22,6 @@ #define arch_irq_stat() 0 #endif =20 -u64 get_idle_time(struct kernel_cpustat *kcs, int cpu) -{ - u64 idle, idle_usecs =3D -1ULL; - - if (cpu_online(cpu)) - idle_usecs =3D get_cpu_idle_time_us(cpu, NULL); - - if (idle_usecs =3D=3D -1ULL) - /* !NO_HZ or cpu offline so we can rely on cpustat.idle */ - idle =3D kcs->cpustat[CPUTIME_IDLE]; - else - idle =3D idle_usecs * NSEC_PER_USEC; - - return idle; -} - -static u64 get_iowait_time(struct kernel_cpustat *kcs, int cpu) -{ - u64 iowait, iowait_usecs =3D -1ULL; - - if (cpu_online(cpu)) - iowait_usecs =3D get_cpu_iowait_time_us(cpu, NULL); - - if (iowait_usecs =3D=3D -1ULL) - /* !NO_HZ or cpu offline so we can rely on cpustat.iowait */ - iowait =3D kcs->cpustat[CPUTIME_IOWAIT]; - else - iowait =3D iowait_usecs * NSEC_PER_USEC; - - return iowait; -} - static void show_irq_gap(struct seq_file *p, unsigned int gap) { static const char zeros[] =3D " 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0"; @@ -105,8 +73,8 @@ static int show_stat(struct seq_file *p, void *v) user +=3D cpustat[CPUTIME_USER]; nice +=3D cpustat[CPUTIME_NICE]; system +=3D cpustat[CPUTIME_SYSTEM]; - idle +=3D get_idle_time(&kcpustat, i); - iowait +=3D get_iowait_time(&kcpustat, i); + idle +=3D cpustat[CPUTIME_IDLE]; + iowait +=3D cpustat[CPUTIME_IOWAIT]; irq +=3D cpustat[CPUTIME_IRQ]; softirq +=3D cpustat[CPUTIME_SOFTIRQ]; steal +=3D cpustat[CPUTIME_STEAL]; @@ -146,8 +114,8 @@ static int show_stat(struct seq_file *p, void *v) user =3D cpustat[CPUTIME_USER]; nice =3D cpustat[CPUTIME_NICE]; system =3D cpustat[CPUTIME_SYSTEM]; - idle =3D get_idle_time(&kcpustat, i); - iowait =3D get_iowait_time(&kcpustat, i); + idle =3D cpustat[CPUTIME_IDLE]; + iowait =3D cpustat[CPUTIME_IOWAIT]; irq =3D cpustat[CPUTIME_IRQ]; softirq =3D cpustat[CPUTIME_SOFTIRQ]; steal =3D cpustat[CPUTIME_STEAL]; diff --git a/fs/proc/uptime.c b/fs/proc/uptime.c index b5343d2..433aa94 100644 --- a/fs/proc/uptime.c +++ b/fs/proc/uptime.c @@ -18,12 +18,8 @@ static int uptime_proc_show(struct seq_file *m, void *v) int i; =20 idle_nsec =3D 0; - for_each_possible_cpu(i) { - struct kernel_cpustat kcs; - - kcpustat_cpu_fetch(&kcs, i); - idle_nsec +=3D get_idle_time(&kcs, i); - } + for_each_possible_cpu(i) + idle_nsec +=3D kcpustat_field(CPUTIME_IDLE, i); =20 ktime_get_boottime_ts64(&uptime); timens_add_boottime(&uptime); diff --git a/include/linux/kernel_stat.h b/include/linux/kernel_stat.h index 9343353..3680519 100644 --- a/include/linux/kernel_stat.h +++ b/include/linux/kernel_stat.h @@ -110,32 +110,59 @@ extern void kcpustat_dyntick_start(u64 now); extern void kcpustat_dyntick_stop(u64 now); extern void kcpustat_irq_enter(u64 now); extern void kcpustat_irq_exit(u64 now); +extern u64 kcpustat_field_idle(int cpu); +extern u64 kcpustat_field_iowait(int cpu); =20 static inline bool kcpustat_idle_dyntick(void) { return __this_cpu_read(kernel_cpustat.idle_dyntick); } #else +static inline u64 kcpustat_field_idle(int cpu) +{ + return kcpustat_cpu(cpu).cpustat[CPUTIME_IDLE]; +} +static inline u64 kcpustat_field_iowait(int cpu) +{ + return kcpustat_cpu(cpu).cpustat[CPUTIME_IOWAIT]; +} + static inline bool kcpustat_idle_dyntick(void) { return false; } #endif /* CONFIG_NO_HZ_COMMON */ =20 +/* Fetch cputime values when vtime is disabled on a CPU */ +static inline u64 kcpustat_field_default(enum cpu_usage_stat usage, int cp= u) +{ + if (usage =3D=3D CPUTIME_IDLE) + return kcpustat_field_idle(cpu); + if (usage =3D=3D CPUTIME_IOWAIT) + return kcpustat_field_iowait(cpu); + return kcpustat_cpu(cpu).cpustat[usage]; +} + +static inline void kcpustat_cpu_fetch_default(struct kernel_cpustat *dst, = int cpu) +{ + *dst =3D kcpustat_cpu(cpu); + dst->cpustat[CPUTIME_IDLE] =3D kcpustat_field_idle(cpu); + dst->cpustat[CPUTIME_IOWAIT] =3D kcpustat_field_iowait(cpu); +} + #ifdef CONFIG_VIRT_CPU_ACCOUNTING_GEN extern u64 kcpustat_field(enum cpu_usage_stat usage, int cpu); extern void kcpustat_cpu_fetch(struct kernel_cpustat *dst, int cpu); #else static inline u64 kcpustat_field(enum cpu_usage_stat usage, int cpu) { - return kcpustat_cpu(cpu).cpustat[usage]; + return kcpustat_field_default(usage, cpu); } =20 static inline void kcpustat_cpu_fetch(struct kernel_cpustat *dst, int cpu) { - *dst =3D kcpustat_cpu(cpu); + kcpustat_cpu_fetch_default(dst, cpu); } - #endif /* !CONFIG_VIRT_CPU_ACCOUNTING_GEN */ =20 extern void account_user_time(struct task_struct *, u64); @@ -145,7 +172,6 @@ extern void account_system_index_time(struct task_struc= t *, u64, enum cpu_usage_stat); extern void account_steal_time(u64); extern void account_idle_time(u64); -extern u64 get_idle_time(struct kernel_cpustat *kcs, int cpu); =20 #ifdef CONFIG_VIRT_CPU_ACCOUNTING_NATIVE static inline void account_process_tick(struct task_struct *tsk, int user) diff --git a/kernel/sched/cputime.c b/kernel/sched/cputime.c index 4c00163..c91fd67 100644 --- a/kernel/sched/cputime.c +++ b/kernel/sched/cputime.c @@ -490,24 +490,14 @@ void kcpustat_irq_exit(u64 now) kcpustat_idle_start(kc, now); } =20 -static u64 get_cpu_sleep_time_us(int cpu, enum cpu_usage_stat idx, - bool compute_delta, u64 *last_update_time) +static u64 kcpustat_field_dyntick(int cpu, enum cpu_usage_stat idx, + bool compute_delta, u64 now) { struct kernel_cpustat *kc =3D &kcpustat_cpu(cpu); u64 *cpustat =3D kc->cpustat; unsigned int seq; - ktime_t now; u64 idle; =20 - now =3D ktime_get(); - if (last_update_time) - *last_update_time =3D ktime_to_us(now); - - if (vtime_generic_enabled_cpu(cpu)) { - idle =3D kcpustat_field(idx, cpu); - goto to_us; - } - do { seq =3D read_seqcount_begin(&kc->idle_sleeptime_seq); =20 @@ -516,12 +506,42 @@ static u64 get_cpu_sleep_time_us(int cpu, enum cpu_us= age_stat idx, idle +=3D (now - kc->idle_entrytime); } while (read_seqcount_retry(&kc->idle_sleeptime_seq, seq)); =20 -to_us: - do_div(idle, NSEC_PER_USEC); - return idle; } =20 +u64 kcpustat_field_idle(int cpu) +{ + return kcpustat_field_dyntick(cpu, CPUTIME_IDLE, + !nr_iowait_cpu(cpu), ktime_get()); +} +EXPORT_SYMBOL_GPL(kcpustat_field_idle); + +u64 kcpustat_field_iowait(int cpu) +{ + return kcpustat_field_dyntick(cpu, CPUTIME_IOWAIT, + nr_iowait_cpu(cpu), ktime_get()); +} +EXPORT_SYMBOL_GPL(kcpustat_field_iowait); + +static u64 get_cpu_sleep_time_us(int cpu, enum cpu_usage_stat idx, + bool compute_delta, u64 *last_update_time) +{ + ktime_t now =3D ktime_get(); + u64 res; + + if (vtime_generic_enabled_cpu(cpu)) + res =3D kcpustat_field(idx, cpu); + else + res =3D kcpustat_field_dyntick(cpu, idx, compute_delta, now); + + do_div(res, NSEC_PER_USEC); + + if (last_update_time) + *last_update_time =3D ktime_to_us(now); + + return res; +} + /** * get_cpu_idle_time_us - get the total idle time of a CPU * @cpu: CPU number to query @@ -569,7 +589,6 @@ u64 get_cpu_iowait_time_us(int cpu, u64 *last_update_ti= me) nr_iowait_cpu(cpu), last_update_time); } EXPORT_SYMBOL_GPL(get_cpu_iowait_time_us); - #endif /* CONFIG_NO_HZ_COMMON */ =20 /* @@ -1123,8 +1142,8 @@ u64 kcpustat_field(enum cpu_usage_stat usage, int cpu) struct rq *rq; int err; =20 - if (!vtime_accounting_enabled_cpu(cpu)) - return val; + if (!vtime_generic_enabled_cpu(cpu)) + return kcpustat_field_default(usage, cpu); =20 rq =3D cpu_rq(cpu); =20 @@ -1219,8 +1238,8 @@ void kcpustat_cpu_fetch(struct kernel_cpustat *dst, i= nt cpu) struct rq *rq; int err; =20 - if (!vtime_accounting_enabled_cpu(cpu)) { - *dst =3D *src; + if (!vtime_generic_enabled_cpu(cpu)) { + kcpustat_cpu_fetch_default(dst, cpu); return; } =20 @@ -1233,7 +1252,7 @@ void kcpustat_cpu_fetch(struct kernel_cpustat *dst, i= nt cpu) curr =3D rcu_dereference(rq->curr); if (WARN_ON_ONCE(!curr)) { rcu_read_unlock(); - *dst =3D *src; + kcpustat_cpu_fetch_default(dst, cpu); return; } =20