From nobody Thu Feb 12 23:03:15 2026 Received: from mail-pl1-f176.google.com (mail-pl1-f176.google.com [209.85.214.176]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id DB6E11119A; Fri, 3 Jan 2025 02:24:30 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.176 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1735871072; cv=none; b=GCdsgwdmcrgcCHQdUK9e+gfIjclkbUJlSNWJyTkmXhx9k0cHH94tvm5Xy0myVITAhmPpT0QYctuU3rxbsmr/njk14NwlIwfJdU2sKzizAfW/38Xg6s/222wUXyjVwop0ORgfIGMoaVwLlw1u6wFMBK9TLbDXp5BRDJ2AgXCDXJE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1735871072; c=relaxed/simple; bh=9Fx1bLgQOBc/4RgmNBn61ZmqQ0quU4HxJT5F6P7PL9o=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version:Content-Type; b=IczWWXiMH43PyPPjCP0AThkQ2HdIuyr/2cPSWhPHSkHLFvow0M2Mhhz69WjEIkM0TSoS85DqIo58euPhJ70CyDQxvJeuShwcHzEB9BePmI0qo9eDxTWHolDgZVUq3+5gONGTUFz6WHtfIq8WIx5tdyYVqxKs4tvA71FxoAXEU3I= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=J7P0GkUO; arc=none smtp.client-ip=209.85.214.176 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="J7P0GkUO" Received: by mail-pl1-f176.google.com with SMTP id d9443c01a7336-21619108a6bso151580535ad.3; Thu, 02 Jan 2025 18:24:30 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1735871070; x=1736475870; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=/0wgvUkZl83NVS3YwL+hYGXPkTDH9VTAKHlm2OealRw=; b=J7P0GkUOGeIRlL1wo5gN7S828AwCcboNjgtCA855fBJMfO23jmwfV4DAHgVkKrGfNu ffZ314qEcYeffkj0Hb8ePus2bfwvAVB97Irt0Y233is8UZvcpl36RNT0BY+5OpmxvPUm seKZufyamNrol6OheZyz6IMEUR1yKdELv9P/1Mv+sxlY2TruR3Nd5UXOgdT7gmkYGblp sGtgpnj8vezOKmxpkxcEh6lZy6XWPCwy5XCWg/plD6+EsOj9A8Doj9Bvv7Y+iBw5V5ms v5rtpYgk2qg61q5NflfqJBTzMADB0i3EhZbqU7Xk7O8wFi6ZwDXXsOyz6P57fwqVmlvt ndIA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1735871070; x=1736475870; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=/0wgvUkZl83NVS3YwL+hYGXPkTDH9VTAKHlm2OealRw=; b=sz1ffFBM2cF6HT1AgPQgKFVcF5s8eF1qMM7xWHy6wdHYfms+DNazTpNvU9XwDq3idI r6/WGsBYn46V04BxZcBDdQcWZ+DGd7lKZUGgNBJw6L4ksEMvLQWZ+VMInHvmo5K/tq/R AsY1/nkiyycTkL/P+E5piS1RZKDb3AAzIhHTbfkb/PZzFxLS59aMAyX/PBI3nUrEMuoX Z8UAzqPuxOvrbXWLt2jXVZ5VbffRyD2B8URhWtaHNakeArsePG8Mfm6lDN/Nx5bjv6yf 4Ng0aEcJU2n3P5Y5yHVHfF6rMe2V4D7E+Pq0RdmnzNFd4A8cMqsTksHPRlvzoveEoux5 GyPQ== X-Forwarded-Encrypted: i=1; AJvYcCVSRZstS8A8AWfIK+rg1YrOV4kVIYn8MW8Sh3AsYWplzSrykjDi5j+bLnRt/R//ao5Hc4yKptOc@vger.kernel.org, AJvYcCWAP6ehKHjFM9ESF9yW0tZm2N1v0QbxHD7uGzvZk3/ElKfsF3i2ZGkkVSNwcnKpW4V4uwqzWMp1kpu/4m6/@vger.kernel.org X-Gm-Message-State: AOJu0YwUmLLdwtiZP6nuD6BUYwDuvXeXaqc0eC72ZAxVMyhC1M+LQOMF JEBVvOXRnYtxf37wKKnlnEzY4RwTU+ZSLmGCRxYegSbLYeJr63lJ X-Gm-Gg: ASbGnctfi61zXpXae7GAZ/aVTWuVXJD31zP2ffi6Rcyie5F2N/aBk3qyJY3qSUJkACw 0hm+w1mmkmCWC4ika4zo26rQP2HR3zgWj7P0XMVYBUYRjo6HdTWAc64EeKe6vr2J6XPm6EtRNSd s7Sr1Jn1S5ViZUyckV+UGiNLPVni/HScwE29NHRk5M+tewgRTB6vPoiOO4EvXMfSvvhHr9GUwQj I+hIVvOEHiLDdk+LBnTZmRkqk4+saU4Mlb/xtIQGlJRKC/LKyHyLY5ctaQHUZKZhwsNpYeJcBmK uwwB3X8= X-Google-Smtp-Source: AGHT+IGaMRVrMVo90n2hd4S1mw3+w06zglgpXD9jY6ueOVcIc5Kac5fP7r5VdQKnCjYZiPHjKXPO4w== X-Received: by 2002:a17:903:1209:b0:215:b9a7:5282 with SMTP id d9443c01a7336-219e6eb417dmr636867265ad.26.1735871070061; Thu, 02 Jan 2025 18:24:30 -0800 (PST) Received: from localhost.localdomain ([180.159.118.224]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-219dca007f3sm234184145ad.228.2025.01.02.18.24.23 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Thu, 02 Jan 2025 18:24:29 -0800 (PST) From: Yafang Shao To: peterz@infradead.org, mingo@redhat.com Cc: mkoutny@suse.com, hannes@cmpxchg.org, juri.lelli@redhat.com, vincent.guittot@linaro.org, dietmar.eggemann@arm.com, rostedt@goodmis.org, bsegall@google.com, mgorman@suse.de, vschneid@redhat.com, surenb@google.com, linux-kernel@vger.kernel.org, cgroups@vger.kernel.org, lkp@intel.com, Yafang Shao Subject: [PATCH v8 1/4] sched: Define sched_clock_irqtime as static key Date: Fri, 3 Jan 2025 10:24:06 +0800 Message-Id: <20250103022409.2544-2-laoar.shao@gmail.com> X-Mailer: git-send-email 2.37.1 (Apple Git-137.1) In-Reply-To: <20250103022409.2544-1-laoar.shao@gmail.com> References: <20250103022409.2544-1-laoar.shao@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Since CPU time accounting is a performance-critical path, let's define sched_clock_irqtime as a static key to minimize potential overhead. Signed-off-by: Yafang Shao Reviewed-by: Michal Koutn=C3=BD Reviewed-by: Vincent Guittot --- kernel/sched/cputime.c | 16 +++++++--------- kernel/sched/sched.h | 13 +++++++++++++ 2 files changed, 20 insertions(+), 9 deletions(-) diff --git a/kernel/sched/cputime.c b/kernel/sched/cputime.c index 0bed0fa1acd9..5d9143dd0879 100644 --- a/kernel/sched/cputime.c +++ b/kernel/sched/cputime.c @@ -9,6 +9,8 @@ =20 #ifdef CONFIG_IRQ_TIME_ACCOUNTING =20 +DEFINE_STATIC_KEY_FALSE(sched_clock_irqtime); + /* * There are no locks covering percpu hardirq/softirq time. * They are only modified in vtime_account, on corresponding CPU @@ -22,16 +24,14 @@ */ DEFINE_PER_CPU(struct irqtime, cpu_irqtime); =20 -static int sched_clock_irqtime; - void enable_sched_clock_irqtime(void) { - sched_clock_irqtime =3D 1; + static_branch_enable(&sched_clock_irqtime); } =20 void disable_sched_clock_irqtime(void) { - sched_clock_irqtime =3D 0; + static_branch_disable(&sched_clock_irqtime); } =20 static void irqtime_account_delta(struct irqtime *irqtime, u64 delta, @@ -57,7 +57,7 @@ void irqtime_account_irq(struct task_struct *curr, unsign= ed int offset) s64 delta; int cpu; =20 - if (!sched_clock_irqtime) + if (!irqtime_enabled()) return; =20 cpu =3D smp_processor_id(); @@ -90,8 +90,6 @@ static u64 irqtime_tick_accounted(u64 maxtime) =20 #else /* CONFIG_IRQ_TIME_ACCOUNTING */ =20 -#define sched_clock_irqtime (0) - static u64 irqtime_tick_accounted(u64 dummy) { return 0; @@ -478,7 +476,7 @@ void account_process_tick(struct task_struct *p, int us= er_tick) if (vtime_accounting_enabled_this_cpu()) return; =20 - if (sched_clock_irqtime) { + if (irqtime_enabled()) { irqtime_account_process_tick(p, user_tick, 1); return; } @@ -507,7 +505,7 @@ void account_idle_ticks(unsigned long ticks) { u64 cputime, steal; =20 - if (sched_clock_irqtime) { + if (irqtime_enabled()) { irqtime_account_idle_ticks(ticks); return; } diff --git a/kernel/sched/sched.h b/kernel/sched/sched.h index aef716c41edb..7e8c73110884 100644 --- a/kernel/sched/sched.h +++ b/kernel/sched/sched.h @@ -3233,6 +3233,12 @@ struct irqtime { }; =20 DECLARE_PER_CPU(struct irqtime, cpu_irqtime); +DECLARE_STATIC_KEY_FALSE(sched_clock_irqtime); + +static inline int irqtime_enabled(void) +{ + return static_branch_likely(&sched_clock_irqtime); +} =20 /* * Returns the irqtime minus the softirq time computed by ksoftirqd. @@ -3253,6 +3259,13 @@ static inline u64 irq_time_read(int cpu) return total; } =20 +#else + +static inline int irqtime_enabled(void) +{ + return 0; +} + #endif /* CONFIG_IRQ_TIME_ACCOUNTING */ =20 #ifdef CONFIG_CPU_FREQ --=20 2.43.5 From nobody Thu Feb 12 23:03:15 2026 Received: from mail-pl1-f176.google.com (mail-pl1-f176.google.com [209.85.214.176]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 848AF1459F7; Fri, 3 Jan 2025 02:24:38 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.176 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1735871080; cv=none; b=cUqJ/hA6wJi71Ex6YT5+mW+Gb+xyC3W2Brrjv+e644oz/LBOsV6h2sSQbedHgFjuo/l9J/ueUUOvpASMxY6ANKWcwf9AgdAzPq/9wI9Y5ZGMXz3nRvvu4yO24PvlNHMz9xg+RTV/hQBJE2M0IzNjyBIfsZgsKETnEfBK22B01cQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1735871080; c=relaxed/simple; bh=p623LhB5/pgHGul4CMWJvgPu3Z/w63bVmRJAxgcQvok=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version:Content-Type; b=SbndelySfxk/L1zPHT36Bc3ZDJ268TfCmmpioQHWAsmf8QvRYB+6Uc7+PiYhmJdFF3RL/oinO3jX61pO5Pqsdfcg0gNT28WlJaxblTyQaDicKEPbfy09l/6pHK2tjDCDvSINTbkTdxdaA05JsWGwIiIoCgNRRhearouFMN6BLik= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=ShxseopL; arc=none smtp.client-ip=209.85.214.176 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="ShxseopL" Received: by mail-pl1-f176.google.com with SMTP id d9443c01a7336-2163bd70069so65173015ad.0; Thu, 02 Jan 2025 18:24:38 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1735871078; x=1736475878; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=6pfOoxJev37uPhhhtmnxcIpBCI1zYq5lZd98FYyO5K4=; b=ShxseopLyUnfa38lRIgcFmC2ejH0v4dm6Hn4XuvBMNJnCVWToiNex1SSjcAuB9D5U0 NSF66sGm/VFdMJGPqAoAUqZqp/3gcyxySD1K1bomnPFU67XTZYC31mvrcJHhwcRC6/Ua +cTIKuOg6Xwdj4m5QW0uKvlwkwNOqmRa92SLfYZAMLyPn7XB3oeVJfmZqVYRYcDKoJTf qrfLSkN1ywTsMWSavFooD9rDnbFfdMj3+NS8B9vl5JyLxuAqLj7U4VCEW32mSzprGH+e T1smc5iNfAOmpvjgTSPyj79a4AdytgFOVHtyJJzC86/CSAUxaW3OGhIADDgxoIkRSBFk cScg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1735871078; x=1736475878; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=6pfOoxJev37uPhhhtmnxcIpBCI1zYq5lZd98FYyO5K4=; b=RZTtLDPptk4TgZF5XLw/eYDlY5kgRLLt5T2anLuoctGLgoyanMmMsPgHUMJ7/+ncpv XhlwF9LimO0JtyvSkd5ChxfVHP/LzORbJ0lcT6sT3OgLipREIm8w22ZU1k3Cf51uMREb fuDqsF1zO/VzCTsIbh3QsrJYKgPdFu18FTKoE2e0oO4QNpzpmBF5wPHeGHtaAmZp8pXi qbKsUFSOvgb9OR6TIIHKS1g2Tn+w+uOnb26Shqxkvgx7NMoENitOD/9+nLKPI0v4sbVu 1sY8ejYCZ5CpnP0cB8AUe/C+4iwussObEsY/lZRTjB0QmxMfre8MEiII+KpK2+6z5sma BCqg== X-Forwarded-Encrypted: i=1; AJvYcCUnrzZcPsMZ74SJzfNa5KtdFzw4CAmRDfKq5n+0yMiNkKS8W/OSl4wYcJgWJkvzriZOXKlFc+g6Mp35eOVu@vger.kernel.org, AJvYcCWRUbjUG39OSE9pBe+0eFSrDM6pAr5gKmIZtWOEUsB0kJkqcu/vBAdnjaimI1GWokx3Gx7hxWHl@vger.kernel.org X-Gm-Message-State: AOJu0YwiLZK96hvyqtQFgRpPg3NxQdWCmk+nXAsJQKrEDlDdLRnNRpYn Yz9ces3Pl54brQW/FQHw+gMuSiYxu+AeFxBFl2lJbv8HbRLsCcBJ X-Gm-Gg: ASbGncsL3Nd36EokBPmq3AZqIKZd6CZQKeUPGVYGBFCZwnmNtYcro6xtUHgZgVonfP6 Q9deAedn/O73VudfY9LwRtdG3oNtxwPOHTOmwFDATyqXqxpyiukTXBnP5FUVty1D8sMyxnWuDyf KtHdatryji4RD4dOI49LQjudAP78ZIlVmMq/rxKnNNR5m81h3Y9NiEsEVc4w6kUHCF92kb6qszn iyo8N2sKYJAeEiv+3BD6aGq4a/W4yv6aKjvUy5KxT56OPV7iH84YJQYvOgfmbOc/qPJ0nSKHsdY QvA6tdk= X-Google-Smtp-Source: AGHT+IFjV4O/xJ4uaqSufRAEt3pchg5BDxp7Rwle86ok33kSe8uitOTPdv/zDJMB1OW96j7KNrpwYw== X-Received: by 2002:a17:903:22cc:b0:216:50c6:6b47 with SMTP id d9443c01a7336-219e6f266bdmr658362965ad.46.1735871077603; Thu, 02 Jan 2025 18:24:37 -0800 (PST) Received: from localhost.localdomain ([180.159.118.224]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-219dca007f3sm234184145ad.228.2025.01.02.18.24.30 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Thu, 02 Jan 2025 18:24:37 -0800 (PST) From: Yafang Shao To: peterz@infradead.org, mingo@redhat.com Cc: mkoutny@suse.com, hannes@cmpxchg.org, juri.lelli@redhat.com, vincent.guittot@linaro.org, dietmar.eggemann@arm.com, rostedt@goodmis.org, bsegall@google.com, mgorman@suse.de, vschneid@redhat.com, surenb@google.com, linux-kernel@vger.kernel.org, cgroups@vger.kernel.org, lkp@intel.com, Yafang Shao Subject: [PATCH v8 2/4] sched: Don't account irq time if sched_clock_irqtime is disabled Date: Fri, 3 Jan 2025 10:24:07 +0800 Message-Id: <20250103022409.2544-3-laoar.shao@gmail.com> X-Mailer: git-send-email 2.37.1 (Apple Git-137.1) In-Reply-To: <20250103022409.2544-1-laoar.shao@gmail.com> References: <20250103022409.2544-1-laoar.shao@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable sched_clock_irqtime may be disabled due to the clock source, in which case IRQ time should not be accounted. Let's add a conditional check to avoid unnecessary logic. Signed-off-by: Yafang Shao Reviewed-by: Michal Koutn=C3=BD Reviewed-by: Vincent Guittot --- kernel/sched/core.c | 44 +++++++++++++++++++++++--------------------- 1 file changed, 23 insertions(+), 21 deletions(-) diff --git a/kernel/sched/core.c b/kernel/sched/core.c index 84902936a620..22dfcd3e92ed 100644 --- a/kernel/sched/core.c +++ b/kernel/sched/core.c @@ -740,29 +740,31 @@ static void update_rq_clock_task(struct rq *rq, s64 d= elta) s64 __maybe_unused steal =3D 0, irq_delta =3D 0; =20 #ifdef CONFIG_IRQ_TIME_ACCOUNTING - irq_delta =3D irq_time_read(cpu_of(rq)) - rq->prev_irq_time; + if (irqtime_enabled()) { + irq_delta =3D irq_time_read(cpu_of(rq)) - rq->prev_irq_time; =20 - /* - * Since irq_time is only updated on {soft,}irq_exit, we might run into - * this case when a previous update_rq_clock() happened inside a - * {soft,}IRQ region. - * - * When this happens, we stop ->clock_task and only update the - * prev_irq_time stamp to account for the part that fit, so that a next - * update will consume the rest. This ensures ->clock_task is - * monotonic. - * - * It does however cause some slight miss-attribution of {soft,}IRQ - * time, a more accurate solution would be to update the irq_time using - * the current rq->clock timestamp, except that would require using - * atomic ops. - */ - if (irq_delta > delta) - irq_delta =3D delta; + /* + * Since irq_time is only updated on {soft,}irq_exit, we might run into + * this case when a previous update_rq_clock() happened inside a + * {soft,}IRQ region. + * + * When this happens, we stop ->clock_task and only update the + * prev_irq_time stamp to account for the part that fit, so that a next + * update will consume the rest. This ensures ->clock_task is + * monotonic. + * + * It does however cause some slight miss-attribution of {soft,}IRQ + * time, a more accurate solution would be to update the irq_time using + * the current rq->clock timestamp, except that would require using + * atomic ops. + */ + if (irq_delta > delta) + irq_delta =3D delta; =20 - rq->prev_irq_time +=3D irq_delta; - delta -=3D irq_delta; - delayacct_irq(rq->curr, irq_delta); + rq->prev_irq_time +=3D irq_delta; + delta -=3D irq_delta; + delayacct_irq(rq->curr, irq_delta); + } #endif #ifdef CONFIG_PARAVIRT_TIME_ACCOUNTING if (static_key_false((¶virt_steal_rq_enabled))) { --=20 2.43.5 From nobody Thu Feb 12 23:03:15 2026 Received: from mail-pl1-f174.google.com (mail-pl1-f174.google.com [209.85.214.174]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id CC1CD84D34; Fri, 3 Jan 2025 02:24:45 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.174 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1735871087; cv=none; b=mZHyDCmaCehWZ6KH1p3rB9Q7J1n9T1LYAcZFl3dnGOA6dBXc4WBQULVz9QMEkCQ50WDsXYDNgCdlkejVIZXDddXidwvI/ACeT58o05XGp+THNJWn0sDliDoOhQ+8qyKsGpPw1u5Yo72zuwt08alvPmoB/d7QrfHK+ma2QHxSXV0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1735871087; c=relaxed/simple; bh=9hkP9CNLB1Zz6HKgM9bvLOyCa2y3jW0g8uuQlxjgwmY=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version:Content-Type; b=DsiuCv/6uyXjBXsQRnbJfGy9wr/FMftPJWdKWdQIPUTTDThB8WDoDuQDHE2K9MTVKwjcGcaBTYInCUolRs6kjTzZUVEGcrdgRVU4rEH8E12PTH7ixHHhjUVQrBQc97meKLTRn8q3xfo3fvX9Z5nXhOfi9BYaePdmcM2G6VfKNZM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=XcOBfX3N; arc=none smtp.client-ip=209.85.214.174 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="XcOBfX3N" Received: by mail-pl1-f174.google.com with SMTP id d9443c01a7336-21669fd5c7cso165646475ad.3; Thu, 02 Jan 2025 18:24:45 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1735871085; x=1736475885; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=yWIE+8nwQHpR1+w81wvyLMSrJdm6DucI0YocL1dx8X8=; b=XcOBfX3NmUkrbLy9te8a8/NPH9x3KOS4LeZ0XbnhqGH15qpNG3WGIfhgOPUGeEZA6v MjYNHW4zOIsuJnDljWpLDuKjle7yrOYlLShZ66PSykr0CkblND2Pr3gC8JZHZkUEhMbi ejsxBUAKKcyal6kQLJJxwZRPzBHJX17t7U4JVCEI0BuHZq2ItPyRYjIf21URPK9b6nl3 uHlVcnmKSDhCpA6QfAbCBKX9YFJBBYOMkQ8nSd7R2Z4nlsRiZnup0K3fHpsyz2ponZgP o5E+YQOJ0XsYFbFhyUf9SDBr97m5qWFZKhUPWrpxGzyQI3yFAbILK/mmm5L28BR8b0lT VTdQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1735871085; x=1736475885; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=yWIE+8nwQHpR1+w81wvyLMSrJdm6DucI0YocL1dx8X8=; b=GVZqz1Z9DUV26cV5+hQVCHwRzE2lmtxT9B9F+gCT2HohQDQb4kbXxCIV1OkyGJAlzF k4m4ekN4WeZIr+O4vd2SY7q1PaRyXKcrTEVg14DnDjOgkJ+9Rg8u9mrUIlSTs6069ElM JSJPjFsDz+2fafd2D65QHVwq5cDN5lWQ/XAtz3Io+Y/DLoTaNufag4sfLssigfGnCBJQ DsvvYGci5UBoS/X+/1VjlKerXnYLuJfXc/bIctAUrXqNpLtmGNE/bsnNkSYMAhKRb2/D POw+yQd6vNxnIY5wyjUfQtKeQ7SNBergDa38ZCbmJ3CWOAzY5gQY2tV/ole5GDia+tp0 LLag== X-Forwarded-Encrypted: i=1; AJvYcCVID2DsVGi6vSVGawyYS7EYlQTrTWcDJESL7BwEUiUrsBYGi0oTrM8KFZ7hxoKwCs88hWAz3FDG@vger.kernel.org, AJvYcCVcEXlZ/b7ha3ZObQni/D7rVg2PTF55gnhRhs0ljf0UNBylMR3z/tFaNnQcDy2kmGpNF4FIhcHslf3ws+Zp@vger.kernel.org X-Gm-Message-State: AOJu0YxfJREJWmBlCMfl9uuianJThJxfBGM4imKf+pdkvAc1qHSVRigd 3LeQ3bqHANAB42RzH2UVKbnyvc4BgOQA8V6ktRYazYZ47jO/XJZZ X-Gm-Gg: ASbGncv0Q9TF//6yukUjfXxsHZIu1QEcd5BArlLf3WGeW0Y5nvgeXgZ9WqA0OViyj1Q qMwRCRjYlntPmwqjo67i+LJmSypkomi8xaHX+1K5BsWnN1wVlgooKbDIKpHvHQ4w3kuzI0Ap6DW BOX+w1eDdF9wVM9euPWaF9SiOmo0z7E1cvpO0zNqi79EoDzSSG/lY1IINnfE1sLJ724hvAlkJqw s0z/fDaul5twEagvbdFTjV7+2LDn0IW+1Sgsxc77KhlKEbwdgKTP9d6p/QXo5p/i63X6Acvh4lo C84qcFs= X-Google-Smtp-Source: AGHT+IGoep/6dfNw3Hd9Fb43f/Z7Syxbr1uL7dDLZJ1mEXIKN7hrQDCZeoslkNaWGJGikMtU1psqDA== X-Received: by 2002:a17:903:1209:b0:216:36ff:ba33 with SMTP id d9443c01a7336-219e6ebcfc7mr624597085ad.26.1735871085003; Thu, 02 Jan 2025 18:24:45 -0800 (PST) Received: from localhost.localdomain ([180.159.118.224]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-219dca007f3sm234184145ad.228.2025.01.02.18.24.38 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Thu, 02 Jan 2025 18:24:44 -0800 (PST) From: Yafang Shao To: peterz@infradead.org, mingo@redhat.com Cc: mkoutny@suse.com, hannes@cmpxchg.org, juri.lelli@redhat.com, vincent.guittot@linaro.org, dietmar.eggemann@arm.com, rostedt@goodmis.org, bsegall@google.com, mgorman@suse.de, vschneid@redhat.com, surenb@google.com, linux-kernel@vger.kernel.org, cgroups@vger.kernel.org, lkp@intel.com, Yafang Shao Subject: [PATCH v8 3/4] sched, psi: Don't account irq time if sched_clock_irqtime is disabled Date: Fri, 3 Jan 2025 10:24:08 +0800 Message-Id: <20250103022409.2544-4-laoar.shao@gmail.com> X-Mailer: git-send-email 2.37.1 (Apple Git-137.1) In-Reply-To: <20250103022409.2544-1-laoar.shao@gmail.com> References: <20250103022409.2544-1-laoar.shao@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable sched_clock_irqtime may be disabled due to the clock source. When disabled, irq_time_read() won't change over time, so there is nothing to account. We can save iterating the whole hierarchy on every tick and context switch. Signed-off-by: Yafang Shao Acked-by: Johannes Weiner Reviewed-by: Michal Koutn=C3=BD Signed-off-by: Yafang Shao --- kernel/sched/psi.c | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/kernel/sched/psi.c b/kernel/sched/psi.c index 84dad1511d1e..bb56805e3d47 100644 --- a/kernel/sched/psi.c +++ b/kernel/sched/psi.c @@ -998,7 +998,7 @@ void psi_account_irqtime(struct rq *rq, struct task_str= uct *curr, struct task_st s64 delta; u64 irq; =20 - if (static_branch_likely(&psi_disabled)) + if (static_branch_likely(&psi_disabled) || !irqtime_enabled()) return; =20 if (!curr->pid) @@ -1240,6 +1240,11 @@ int psi_show(struct seq_file *m, struct psi_group *g= roup, enum psi_res res) if (static_branch_likely(&psi_disabled)) return -EOPNOTSUPP; =20 +#ifdef CONFIG_IRQ_TIME_ACCOUNTING + if (!irqtime_enabled() && res =3D=3D PSI_IRQ) + return -EOPNOTSUPP; +#endif + /* Update averages before reporting them */ mutex_lock(&group->avgs_lock); now =3D sched_clock(); --=20 2.43.5 From nobody Thu Feb 12 23:03:15 2026 Received: from mail-pl1-f176.google.com (mail-pl1-f176.google.com [209.85.214.176]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 54FB5190688; Fri, 3 Jan 2025 02:24:53 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.176 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1735871095; cv=none; b=AXu9o6nLppaQoeB04eywXAdY2ZN2OU1KoqvqWjycmkZcyyVjFDwv+aAtFmDwY8RIU5Q5ti0N1haLlG+FquDHATeNMbaKqs8xucyy9ev14NBxjUTkm8KrNzyoDlGbXxnvdWupv8Mr1MLT5mI9cz6Kz9FjeUNNn7xY+4UBMyGmNOU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1735871095; c=relaxed/simple; bh=KEU7CWczGDLyD7YwHYxR3y7xe8tazXwkEwl9zxlXFls=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version:Content-Type; b=mZ3ykyYyWsk/6ZF3oKdWhME3s0QW5XGouMVFzDUlcb1PrX+3RS0GUcxlEKWphi/NkpifhZ2uyNbupn7MIORZx9XAcGBMMXzBma2Zjg1CFMLfwH5hQUqbnM8wRBSywacUebiQlH0jMh6vfQ33qD3Ej05d9NQNkPwth2vV4YuKnSw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=Iac178Ln; arc=none smtp.client-ip=209.85.214.176 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="Iac178Ln" Received: by mail-pl1-f176.google.com with SMTP id d9443c01a7336-2165448243fso196581655ad.1; Thu, 02 Jan 2025 18:24:53 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1735871092; x=1736475892; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=avljg6opUL/YIQ884XhrX8PNb9qrQzwcO+9J8KL8WNo=; b=Iac178LnMUTCMViSmANU3RdzgKvnrxY6UNN5kcxuxVbbLdUYq1Bxp3IIHbxDZ7fboI a0tD8ugGOLROwfFt8hNq0ak4RUFjbGCvQM3m6EMcMBp2FqfYKvNt19PCBNvEtSrLvLdt CC0LRW1VjWoCQ7SFju6ns0gmYTGnB3UH38MaVHLSVHApVGRT63QEpQkxcoZUt4PxxBau 4rLe/gbi3dep64uxHznjEBGTF4QXAZfAUWM53p78g3OLgTRtYjq/1VQ6RqNcnqUqXD/0 JwzhEGw6X4KETendbnK2KMpl/gJWCQ5WKbs4RuuuI08R+69FTncLWHtWT1YUcaEcjnkm dQUA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1735871092; x=1736475892; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=avljg6opUL/YIQ884XhrX8PNb9qrQzwcO+9J8KL8WNo=; b=DNPx0Kh44gpaILQhZtWl85JMfka1uE55Q2tpmO85cj/GnAF2/gDETBtt4ZhdmEahuR /t08BKomUPf1aPGVt88LxtEMPv9q1nuMcBu6ScoakRTEErQixyyqFnMn+tXUxipoco+9 YTHO28EmQUaYFEAhBAE/vVatxG1K6WBzzEwedkWYYxr9EO2TQ2/Eh8SrZ9qSqZkpGTRk p9822fJdHlC+FINdSY9RnRIb0Jak6WxkHwbxiwj2psMlkBxExD18beOV6yZ3HvsEv/tU R4uBwwrDaT0ek89G+FpjnkW/dLTXnd+qhL11RvZxeGC7zClwTe7ywvag8i0Sq/QfLM6k kFxQ== X-Forwarded-Encrypted: i=1; AJvYcCU6EABUEWxZS0qwY+bNpooMndieatjNituUKgG1p58trum9Y1Q66upHHOgdgB8zHKzK4KA8xJ9WOkXY3qas@vger.kernel.org, AJvYcCV6deXNbcpu/dKWE+9xYWR21LJlOw6x790Wo3+QPxt2piBwT/4QGL6sGH0YDmGRjQA1J7umSTv3@vger.kernel.org X-Gm-Message-State: AOJu0Yx2nSAERO5HHdT9F8oBzZaIJOT1SQiEow8vnzwhKHk3ME/Wh1cJ mIanUXHnEqjI63dP15pVaYbWQrmbuIncdHpe3gKqSwWnxd71asdn X-Gm-Gg: ASbGncsPX4svwgMLvgVzXkdhPxHt1Jo+FTNcfHgEpJxvf7W79j4LOGomeWmqFMGsBJb WY5oqpGCtHpbThTt+imQEqchv+8svH9CzIeG6VLnbujL2hGY73s5naZWzTb55+WdJSYjgkY37sx 3yWSBEzPBPdGcZLJuz9AzfM27Md23MhXp4WtZV7WZSVowzSQHWFC1Kuq1F9lKYHuduNXs8QHxbg SjP2et/1jocC1qNdAzrogL8D/eWANqwDmZuqwvYcPyH1eYI8iqgoIRZXbztcD1oVgSkN2xuMcWg XVO5wtM= X-Google-Smtp-Source: AGHT+IHBk2iQ2PW4PxELu2coNq5s3P9s/AA319Toiatpmai8xVat5IMDzQXHiLBCQ50YuN3XU5IlpQ== X-Received: by 2002:a17:903:32c6:b0:216:25a2:2ebe with SMTP id d9443c01a7336-219e6e9fb3amr701989925ad.19.1735871092453; Thu, 02 Jan 2025 18:24:52 -0800 (PST) Received: from localhost.localdomain ([180.159.118.224]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-219dca007f3sm234184145ad.228.2025.01.02.18.24.45 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Thu, 02 Jan 2025 18:24:52 -0800 (PST) From: Yafang Shao To: peterz@infradead.org, mingo@redhat.com Cc: mkoutny@suse.com, hannes@cmpxchg.org, juri.lelli@redhat.com, vincent.guittot@linaro.org, dietmar.eggemann@arm.com, rostedt@goodmis.org, bsegall@google.com, mgorman@suse.de, vschneid@redhat.com, surenb@google.com, linux-kernel@vger.kernel.org, cgroups@vger.kernel.org, lkp@intel.com, Yafang Shao Subject: [PATCH v8 4/4] sched: Fix cgroup irq time for CONFIG_IRQ_TIME_ACCOUNTING Date: Fri, 3 Jan 2025 10:24:09 +0800 Message-Id: <20250103022409.2544-5-laoar.shao@gmail.com> X-Mailer: git-send-email 2.37.1 (Apple Git-137.1) In-Reply-To: <20250103022409.2544-1-laoar.shao@gmail.com> References: <20250103022409.2544-1-laoar.shao@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable After enabling CONFIG_IRQ_TIME_ACCOUNTING to monitor IRQ pressure in our container environment, we observed several noticeable behavioral changes. One of our IRQ-heavy services, such as Redis, reported a significant reduction in CPU usage after upgrading to the new kernel with CONFIG_IRQ_TIME_ACCOUNTING enabled. However, despite adding more threads to handle an increased workload, the CPU usage could not be raised. In other words, even though the container=E2=80=99s CPU usage appeared low, it= was unable to process more workloads to utilize additional CPU resources, which caused issues. This behavior can be demonstrated using netperf: function start_server() { for j in `seq 1 3`; do netserver -p $[12345+j] > /dev/null & done } server_ip=3D$1 function start_client() { # That applies to cgroup2 as well. mkdir -p /sys/fs/cgroup/cpuacct/test echo $$ > /sys/fs/cgroup/cpuacct/test/cgroup.procs for j in `seq 1 3`; do port=3D$[12345+j] taskset -c 0 netperf -H ${server_ip} -l ${run_time:-30000} \ -t TCP_STREAM -p $port -- -D -m 1k -M 1K -s 8k -S 8k \ > /dev/null & done } start_server start_client We can verify the CPU usage of the test cgroup using cpuacct.stat. The output shows: system: 53 user: 2 The CPU usage of the cgroup is relatively low at around 55%, but this usage doesn't increase, even with more netperf tasks. The reason is that CPU0 is at 100% utilization, as confirmed by mpstat: 02:56:22 PM CPU %usr %nice %sys %iowait %irq %soft %steal = %guest %gnice %idle 02:56:23 PM 0 0.99 0.00 55.45 0.00 0.99 42.57 0.00 = 0.00 0.00 0.00 02:56:23 PM CPU %usr %nice %sys %iowait %irq %soft %steal = %guest %gnice %idle 02:56:24 PM 0 2.00 0.00 55.00 0.00 0.00 43.00 0.00 = 0.00 0.00 0.00 It is clear that the %soft is excluded in the cgroup of the interrupted task. This behavior is unexpected. We should include IRQ time in the cgroup to reflect the pressure the group is under. After a thorough analysis, I discovered that this change in behavior is due to commit 305e6835e055 ("sched: Do not account irq time to current task"), which altered whether IRQ time should be charged to the interrupted task. While I agree that a task should not be penalized by random interrupts, the task itself cannot progress while interrupted. Therefore, the interrupted time should be reported to the user. The system metric in cpuacct.stat is crucial in indicating whether a container is under heavy system pressure, including IRQ/softirq activity. Hence, IRQ/softirq time should be included in the cpuacct system usage, which also applies to cgroup2=E2=80=99s rstat. The reason it doesn't just add the cgroup_account_*() to irqtime_account_irq() is that it might result in performance hit to hold the rq_lock in the critical path. Taking inspiration from commit ddae0ca2a8fe ("sched: Move psi_account_irqtime() out of update_rq_clock_task() hotpath"), I've now adapted the approach to handle it in a non-critical path, reducing the performance impact. Signed-off-by: Yafang Shao Cc: Johannes Weiner Cc: Michal Koutn=C3=BD --- kernel/sched/core.c | 33 +++++++++++++++++++++++++++++++-- kernel/sched/psi.c | 13 +++---------- kernel/sched/sched.h | 2 +- kernel/sched/stats.h | 7 ++++--- 4 files changed, 39 insertions(+), 16 deletions(-) diff --git a/kernel/sched/core.c b/kernel/sched/core.c index 22dfcd3e92ed..7faacf320af9 100644 --- a/kernel/sched/core.c +++ b/kernel/sched/core.c @@ -5622,6 +5622,35 @@ __setup("resched_latency_warn_ms=3D", setup_resched_= latency_warn_ms); static inline u64 cpu_resched_latency(struct rq *rq) { return 0; } #endif /* CONFIG_SCHED_DEBUG */ =20 +#ifdef CONFIG_IRQ_TIME_ACCOUNTING +static void account_irqtime(struct rq *rq, struct task_struct *curr, + struct task_struct *prev) +{ + int cpu =3D smp_processor_id(); + s64 delta; + u64 irq; + + if (!irqtime_enabled()) + return; + + irq =3D irq_time_read(cpu); + delta =3D (s64)(irq - rq->irq_time); + if (delta < 0) + return; + + rq->irq_time =3D irq; + psi_account_irqtime(rq, curr, prev, delta); + cgroup_account_cputime(curr, delta); + /* We account both softirq and irq into CPUTIME_IRQ */ + cgroup_account_cputime_field(curr, CPUTIME_IRQ, delta); +} +#else +static inline void account_irqtime(struct rq *rq, struct task_struct *curr, + struct task_struct *prev) +{ +} +#endif + /* * This function gets called by the timer code, with HZ frequency. * We call it with interrupts disabled. @@ -5644,7 +5673,7 @@ void sched_tick(void) rq_lock(rq, &rf); donor =3D rq->donor; =20 - psi_account_irqtime(rq, donor, NULL); + account_irqtime(rq, donor, NULL); =20 update_rq_clock(rq); hw_pressure =3D arch_scale_hw_pressure(cpu_of(rq)); @@ -6751,7 +6780,7 @@ static void __sched notrace __schedule(int sched_mode) ++*switch_count; =20 migrate_disable_switch(rq, prev); - psi_account_irqtime(rq, prev, next); + account_irqtime(rq, prev, next); psi_sched_switch(prev, next, block); =20 trace_sched_switch(preempt, prev, next, prev_state); diff --git a/kernel/sched/psi.c b/kernel/sched/psi.c index bb56805e3d47..6fa3233fcba9 100644 --- a/kernel/sched/psi.c +++ b/kernel/sched/psi.c @@ -990,15 +990,14 @@ void psi_task_switch(struct task_struct *prev, struct= task_struct *next, } =20 #ifdef CONFIG_IRQ_TIME_ACCOUNTING -void psi_account_irqtime(struct rq *rq, struct task_struct *curr, struct t= ask_struct *prev) +void psi_account_irqtime(struct rq *rq, struct task_struct *curr, struct t= ask_struct *prev, + s64 delta) { int cpu =3D task_cpu(curr); struct psi_group *group; struct psi_group_cpu *groupc; - s64 delta; - u64 irq; =20 - if (static_branch_likely(&psi_disabled) || !irqtime_enabled()) + if (static_branch_likely(&psi_disabled)) return; =20 if (!curr->pid) @@ -1009,12 +1008,6 @@ void psi_account_irqtime(struct rq *rq, struct task_= struct *curr, struct task_st if (prev && task_psi_group(prev) =3D=3D group) return; =20 - irq =3D irq_time_read(cpu); - delta =3D (s64)(irq - rq->psi_irq_time); - if (delta < 0) - return; - rq->psi_irq_time =3D irq; - do { u64 now; =20 diff --git a/kernel/sched/sched.h b/kernel/sched/sched.h index 7e8c73110884..570cc19a0060 100644 --- a/kernel/sched/sched.h +++ b/kernel/sched/sched.h @@ -1227,7 +1227,7 @@ struct rq { =20 #ifdef CONFIG_IRQ_TIME_ACCOUNTING u64 prev_irq_time; - u64 psi_irq_time; + u64 irq_time; #endif #ifdef CONFIG_PARAVIRT u64 prev_steal_time; diff --git a/kernel/sched/stats.h b/kernel/sched/stats.h index 8ee0add5a48a..b9ed9fea5ab7 100644 --- a/kernel/sched/stats.h +++ b/kernel/sched/stats.h @@ -111,10 +111,11 @@ void psi_task_change(struct task_struct *task, int cl= ear, int set); void psi_task_switch(struct task_struct *prev, struct task_struct *next, bool sleep); #ifdef CONFIG_IRQ_TIME_ACCOUNTING -void psi_account_irqtime(struct rq *rq, struct task_struct *curr, struct t= ask_struct *prev); +void psi_account_irqtime(struct rq *rq, struct task_struct *curr, + struct task_struct *prev, s64 delta); #else static inline void psi_account_irqtime(struct rq *rq, struct task_struct *= curr, - struct task_struct *prev) {} + struct task_struct *prev, s64 delta) {} #endif /*CONFIG_IRQ_TIME_ACCOUNTING */ /* * PSI tracks state that persists across sleeps, such as iowaits and @@ -224,7 +225,7 @@ static inline void psi_sched_switch(struct task_struct = *prev, struct task_struct *next, bool sleep) {} static inline void psi_account_irqtime(struct rq *rq, struct task_struct *= curr, - struct task_struct *prev) {} + struct task_struct *prev, s64 delta) {} #endif /* CONFIG_PSI */ =20 #ifdef CONFIG_SCHED_INFO --=20 2.43.5