From nobody Wed Dec 17 17:59:32 2025 Received: from mail-pf1-f172.google.com (mail-pf1-f172.google.com [209.85.210.172]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 35A76D2FB; Sun, 15 Dec 2024 03:23:28 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.172 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734233010; cv=none; b=Cp4vZsbFfapxsAm8lEyCL7yqKm9FNKApj/NSaqKO9IYcajXvz2PGk4OyRkm6VM9tuOIUbe3FEIkIrXd1mm3XHnIFBCtd3M+TJZQ8f/TJn2WbADr6CMKGkM9mRbnZzStrVVsFR7bg5xThPUB8EEzFohriPFBvwk3V9pCUEmkuzHM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734233010; c=relaxed/simple; bh=9Fx1bLgQOBc/4RgmNBn61ZmqQ0quU4HxJT5F6P7PL9o=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version:Content-Type; b=n+/pm5mfsA41IQc3IQ8vdrf3QvUvxngHKjZl40b3Rayq6Cgce/gF6tPlsme2D8PwwLVUkcGeHyBgGS13EjCaEBoGIeTgg3Fh0qkkPbyHoelFKSGuTagbVmwVzz8ZRcRqMUpi3RHN38LCfpgVKG5mmhHlE6kPW9R1iWcthuxev3s= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=jrQ8yyGO; arc=none smtp.client-ip=209.85.210.172 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="jrQ8yyGO" Received: by mail-pf1-f172.google.com with SMTP id d2e1a72fcca58-725ef0397aeso2545654b3a.2; Sat, 14 Dec 2024 19:23:28 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1734233008; x=1734837808; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=/0wgvUkZl83NVS3YwL+hYGXPkTDH9VTAKHlm2OealRw=; b=jrQ8yyGO3K+ebBSKIxDzRe4buGrundPNhuF90W/xBE7QrZYzozVhKfhgqWwOxCzjtJ 43BSG/rccJPzOFzT0tIWUoDXtpMxxgqd05z9cIb26BzEvnXQnDS1EVxHHjvmtI0b5p2N lqZQufGplnkOYRbkB8xr4SCXkrw5bb6twW1UwAAJSKpXn8LIT1QjfQbFyJ3fncNeU1c7 /BaCwks6fjJpnB8wo9RG+qc398IhY6JjtNAp0U2yECj+lw2HnjRhRIJRQCo3bCtMfxwU cC9PDLwb7qeieqhftKHlM5FcFdpAvdDGu4FUEiYJfupUbmgLrU+UUUcz0L+HZu+le8xH vDAA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1734233008; x=1734837808; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=/0wgvUkZl83NVS3YwL+hYGXPkTDH9VTAKHlm2OealRw=; b=V3CqM/PzYjxa/N7M65U5Fw3P4P5bSQ8SkaYxzI2N2doAmm5wJgBA9S+fcy64Ouar7V /CzGmJ3u5j6W0RXPatbV8U9gUiRvCSDwTn4nZjlRqACKN1EO8a0mnvVlTP5vJID1oN9I JVt0bY0RRndLjp0MmPrulHAwBza0VTLcLXuwdz9FERpAAaiIAOtni80hEnafTyYuFCDB NdGEYl6HZMYnrRqi610Mua1DY2xnGH2Deji1HueIUrGb6M57R2FHhjhXn6qBtYpNpg7u ZjHLSnwSDjxbcwVW9xb7HwANpAek1DtOpRpfSV/19DZ7xwrWxHceHPYUSHJK+8HuD47G L5vQ== X-Forwarded-Encrypted: i=1; AJvYcCVU2ubDh0ANgUDKf3OYsYsUPKckzJdBjMu97nvG73DDeW083fKJAJKwPe2UcIr8BtUY98PY5pl0@vger.kernel.org, AJvYcCW4q23bGKOgIKjOD5g+PLakpFiWXz9ONne/2zuefuD5yupRYXSx7R0LSG5VMyPvFb7LtqBm2PycxvpePRf0@vger.kernel.org X-Gm-Message-State: AOJu0Yzi3JHF6F4T0JKaGSSJVpBcPWpBm7GKoaPPAEeRl2lLmG9JS029 V3o+I6acmaWI9O/8TX1+l2/OQGYMHbjHViT1jsUxFtSXLx0RCYFp X-Gm-Gg: ASbGncvf637xbJGn2hpc7Wn25Mv1js5b6/gelSyH+ZTb7FaddCRy+mfujMJyjr9SwpG mIcNEvuZ7H81sQWJLvpNPlgCfE1dQBNVo253eFiPW+nluJzZPZZX70ckmsWYjxNAah1jvC5J7TB E/ls8CPosACV44SflemyyI+D7/xlMxapDsvFg8X8EdEPXlAnIW71hyovr64tsieRdPBJbuUx5zi 607BcQlN9lbCIvJ6uRDjVTcbCuFaqy9D48aZ+xL4fxp6mvvDA2gOOMkzU2Mvh0IyiZsU2NRaY3L VE2fw3o= X-Google-Smtp-Source: AGHT+IEbPALpZd2u3ybIHeJ4WyQxRtL/5fmLBTSz7fHi/ONzKvUYJz74pdimEeyV84Yyb51h9wOplQ== X-Received: by 2002:a05:6a00:b56:b0:727:3c37:d5f9 with SMTP id d2e1a72fcca58-7290c26863dmr12959610b3a.26.1734233008413; Sat, 14 Dec 2024 19:23:28 -0800 (PST) Received: from localhost.localdomain ([180.159.118.224]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-72918b78f1esm2176586b3a.92.2024.12.14.19.23.24 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Sat, 14 Dec 2024 19:23:27 -0800 (PST) From: Yafang Shao To: mingo@redhat.com, peterz@infradead.org, mkoutny@suse.com, hannes@cmpxchg.org Cc: juri.lelli@redhat.com, vincent.guittot@linaro.org, dietmar.eggemann@arm.com, rostedt@goodmis.org, bsegall@google.com, mgorman@suse.de, vschneid@redhat.com, surenb@google.com, linux-kernel@vger.kernel.org, cgroups@vger.kernel.org, Yafang Shao Subject: [PATCH v7 1/4] sched: Define sched_clock_irqtime as static key Date: Sun, 15 Dec 2024 11:23:12 +0800 Message-Id: <20241215032315.43698-2-laoar.shao@gmail.com> X-Mailer: git-send-email 2.37.1 (Apple Git-137.1) In-Reply-To: <20241215032315.43698-1-laoar.shao@gmail.com> References: <20241215032315.43698-1-laoar.shao@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Since CPU time accounting is a performance-critical path, let's define sched_clock_irqtime as a static key to minimize potential overhead. Signed-off-by: Yafang Shao Reviewed-by: Michal Koutn=C3=BD --- kernel/sched/cputime.c | 16 +++++++--------- kernel/sched/sched.h | 13 +++++++++++++ 2 files changed, 20 insertions(+), 9 deletions(-) diff --git a/kernel/sched/cputime.c b/kernel/sched/cputime.c index 0bed0fa1acd9..5d9143dd0879 100644 --- a/kernel/sched/cputime.c +++ b/kernel/sched/cputime.c @@ -9,6 +9,8 @@ =20 #ifdef CONFIG_IRQ_TIME_ACCOUNTING =20 +DEFINE_STATIC_KEY_FALSE(sched_clock_irqtime); + /* * There are no locks covering percpu hardirq/softirq time. * They are only modified in vtime_account, on corresponding CPU @@ -22,16 +24,14 @@ */ DEFINE_PER_CPU(struct irqtime, cpu_irqtime); =20 -static int sched_clock_irqtime; - void enable_sched_clock_irqtime(void) { - sched_clock_irqtime =3D 1; + static_branch_enable(&sched_clock_irqtime); } =20 void disable_sched_clock_irqtime(void) { - sched_clock_irqtime =3D 0; + static_branch_disable(&sched_clock_irqtime); } =20 static void irqtime_account_delta(struct irqtime *irqtime, u64 delta, @@ -57,7 +57,7 @@ void irqtime_account_irq(struct task_struct *curr, unsign= ed int offset) s64 delta; int cpu; =20 - if (!sched_clock_irqtime) + if (!irqtime_enabled()) return; =20 cpu =3D smp_processor_id(); @@ -90,8 +90,6 @@ static u64 irqtime_tick_accounted(u64 maxtime) =20 #else /* CONFIG_IRQ_TIME_ACCOUNTING */ =20 -#define sched_clock_irqtime (0) - static u64 irqtime_tick_accounted(u64 dummy) { return 0; @@ -478,7 +476,7 @@ void account_process_tick(struct task_struct *p, int us= er_tick) if (vtime_accounting_enabled_this_cpu()) return; =20 - if (sched_clock_irqtime) { + if (irqtime_enabled()) { irqtime_account_process_tick(p, user_tick, 1); return; } @@ -507,7 +505,7 @@ void account_idle_ticks(unsigned long ticks) { u64 cputime, steal; =20 - if (sched_clock_irqtime) { + if (irqtime_enabled()) { irqtime_account_idle_ticks(ticks); return; } diff --git a/kernel/sched/sched.h b/kernel/sched/sched.h index aef716c41edb..7e8c73110884 100644 --- a/kernel/sched/sched.h +++ b/kernel/sched/sched.h @@ -3233,6 +3233,12 @@ struct irqtime { }; =20 DECLARE_PER_CPU(struct irqtime, cpu_irqtime); +DECLARE_STATIC_KEY_FALSE(sched_clock_irqtime); + +static inline int irqtime_enabled(void) +{ + return static_branch_likely(&sched_clock_irqtime); +} =20 /* * Returns the irqtime minus the softirq time computed by ksoftirqd. @@ -3253,6 +3259,13 @@ static inline u64 irq_time_read(int cpu) return total; } =20 +#else + +static inline int irqtime_enabled(void) +{ + return 0; +} + #endif /* CONFIG_IRQ_TIME_ACCOUNTING */ =20 #ifdef CONFIG_CPU_FREQ --=20 2.43.5 From nobody Wed Dec 17 17:59:32 2025 Received: from mail-pg1-f170.google.com (mail-pg1-f170.google.com [209.85.215.170]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 700D01B815; Sun, 15 Dec 2024 03:23:33 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.215.170 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734233014; cv=none; b=owv0rdgxklv+yMrC23hylXfBrpW2APoD1/uLKz5cjj3T8sETsWvM0UosLbmQAzqJooXaKFhPfkzICQee24pVu2NQiKe2AyDLUfYUgxCLtpEE/qTulD3JPgEdWFmfxnHDtXyGXHE3CyI9TwmGeO5+HLBaCtqU2+mAgiB2wejfg/Q= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734233014; c=relaxed/simple; bh=p623LhB5/pgHGul4CMWJvgPu3Z/w63bVmRJAxgcQvok=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version:Content-Type; b=C23V2uQjQgJ9S4IEG7cMn65dmINl7kiPArDbWfniGOvPo7ZQYFvUcpO4ROR/fKMH+A0g+dENvEWfvYgkgcC+bTnZfLi4QBtuQo4qrAkIgM1kj6gJ6+Lx/0zOa3yrm2wkuVKwQgW6zQwNWNKCuq+xVOXZvX3seOnA9ovol2YkD/8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=i3GEG3Nl; arc=none smtp.client-ip=209.85.215.170 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="i3GEG3Nl" Received: by mail-pg1-f170.google.com with SMTP id 41be03b00d2f7-7fd2ff40782so2699903a12.2; Sat, 14 Dec 2024 19:23:33 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1734233013; x=1734837813; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=6pfOoxJev37uPhhhtmnxcIpBCI1zYq5lZd98FYyO5K4=; b=i3GEG3NlvHo5Yknb0XT4FUex627vyJJf6XzRvJYFp/SMmiKwhHAVxmnq7w/DnxClH2 7M01u4nOgrglSzO5N/2b3A7Fe0wLgx+IQv/kpRu35YLDckbEg+aP3a+ETMvl7ZWSlDpv 8rg+DEXw9GuCftHi+Ajrvw8XDMGYETMaVvvbOBINptFaftXN510CeTSBY9BZ1wBvYbrD C522nD3eqBemuEdxmueCBzgeqyiSeu375swX74b5GyEQVNl+4SW5pUxD2ES1bxvfd55O 4EGs6lnoFaL0E4LFD1vWJE6mwnW008+Rar10GypmpV0qjweK6WTYtromNQWPV6shMV5k S/XA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1734233013; x=1734837813; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=6pfOoxJev37uPhhhtmnxcIpBCI1zYq5lZd98FYyO5K4=; b=ujK9A4skyQh42LvH5DZEqN67FmJ89+xtyYRuxePzSvh+LrcsJl7ei+vN6L5PLz6aDr sckliyoKqzmKF3yZACSxzDE3C5c1BO64N6YkSiLzc9v63TjFaVtqtJUVMVut/NImUoOz bZemmo0x6gEACiOIoaoEScD3J3LemvuOBceXAL5zV5zxV02tOiaBQjMdvMGpq3vej6d3 CS3XTD3t3e25qcmRjvr+YgU0Ki/uRCkYaYQRxzPRhUh423zXbUoU6YJhwi7dOLbG820C WNtS04UA6CbyCPb1NYKzRbXbNPBydHKikT5pjTk9sFhPL2CPa3xDml9fE4o3jG+gfekn URAA== X-Forwarded-Encrypted: i=1; AJvYcCUIkVp7oULs6J+NxwFHOyLC+4LRT/mOwBeC3tzmobEMPYSvBjwgJ4GdMhnQe9mOs9m5aVgLZ0SEX9oTC/pI@vger.kernel.org, AJvYcCXIPE7PajEE0/9WAMVLmLRx5/Pn7th9riPVw6QVYGm1rKb5vduZCEjZ/lL/rLKQgEpxUFyFWonq@vger.kernel.org X-Gm-Message-State: AOJu0YxvO+h0hHNX0vFJLmlBTT2SlN2fjUdRYBOkTqgvOhnlrZpuD3LT KaVh0OOUUZoHEl25jCk95OhYIorweBF0K29+vOW56wHB8fI5Jajb X-Gm-Gg: ASbGnctrbdwt0+Fxx1wtyXpefGcs+lBPTJzJhh6yKx4b1EBpO9HU4wXtlRS2mlO02gs EoWrKJGPsGjiPFPuOymx7oPEnQFqfqe/8Dqu6WWjCDg1I+7jJUS3ggne0ftXaeDiXUTvaNmEhya A69QkrlRhyEWX/oOKNMTv5pKznUdL6/MicixtL0LY4nYphzmcozxym2FFLLM7WdtA1C7tkZY3NA PWzIOPIhci11ma3vJy1KlFII30zfLJS9tZzuXSwYSjRhn7TSUOrvznvyrBUsr0ApS/pBnDpmqQt mX/t5BE= X-Google-Smtp-Source: AGHT+IFGvb2CHVzwR9hPAjtJk1WgngOZ/5HoaHh3EGO72pAtuelyrHlLg5uIQRaxsUMmJf9cl7cODA== X-Received: by 2002:a05:6a20:9143:b0:1e1:9f77:da92 with SMTP id adf61e73a8af0-1e1dfde6d4dmr10598466637.33.1734233012781; Sat, 14 Dec 2024 19:23:32 -0800 (PST) Received: from localhost.localdomain ([180.159.118.224]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-72918b78f1esm2176586b3a.92.2024.12.14.19.23.28 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Sat, 14 Dec 2024 19:23:32 -0800 (PST) From: Yafang Shao To: mingo@redhat.com, peterz@infradead.org, mkoutny@suse.com, hannes@cmpxchg.org Cc: juri.lelli@redhat.com, vincent.guittot@linaro.org, dietmar.eggemann@arm.com, rostedt@goodmis.org, bsegall@google.com, mgorman@suse.de, vschneid@redhat.com, surenb@google.com, linux-kernel@vger.kernel.org, cgroups@vger.kernel.org, Yafang Shao Subject: [PATCH v7 2/4] sched: Don't account irq time if sched_clock_irqtime is disabled Date: Sun, 15 Dec 2024 11:23:13 +0800 Message-Id: <20241215032315.43698-3-laoar.shao@gmail.com> X-Mailer: git-send-email 2.37.1 (Apple Git-137.1) In-Reply-To: <20241215032315.43698-1-laoar.shao@gmail.com> References: <20241215032315.43698-1-laoar.shao@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable sched_clock_irqtime may be disabled due to the clock source, in which case IRQ time should not be accounted. Let's add a conditional check to avoid unnecessary logic. Signed-off-by: Yafang Shao Reviewed-by: Michal Koutn=C3=BD --- kernel/sched/core.c | 44 +++++++++++++++++++++++--------------------- 1 file changed, 23 insertions(+), 21 deletions(-) diff --git a/kernel/sched/core.c b/kernel/sched/core.c index 84902936a620..22dfcd3e92ed 100644 --- a/kernel/sched/core.c +++ b/kernel/sched/core.c @@ -740,29 +740,31 @@ static void update_rq_clock_task(struct rq *rq, s64 d= elta) s64 __maybe_unused steal =3D 0, irq_delta =3D 0; =20 #ifdef CONFIG_IRQ_TIME_ACCOUNTING - irq_delta =3D irq_time_read(cpu_of(rq)) - rq->prev_irq_time; + if (irqtime_enabled()) { + irq_delta =3D irq_time_read(cpu_of(rq)) - rq->prev_irq_time; =20 - /* - * Since irq_time is only updated on {soft,}irq_exit, we might run into - * this case when a previous update_rq_clock() happened inside a - * {soft,}IRQ region. - * - * When this happens, we stop ->clock_task and only update the - * prev_irq_time stamp to account for the part that fit, so that a next - * update will consume the rest. This ensures ->clock_task is - * monotonic. - * - * It does however cause some slight miss-attribution of {soft,}IRQ - * time, a more accurate solution would be to update the irq_time using - * the current rq->clock timestamp, except that would require using - * atomic ops. - */ - if (irq_delta > delta) - irq_delta =3D delta; + /* + * Since irq_time is only updated on {soft,}irq_exit, we might run into + * this case when a previous update_rq_clock() happened inside a + * {soft,}IRQ region. + * + * When this happens, we stop ->clock_task and only update the + * prev_irq_time stamp to account for the part that fit, so that a next + * update will consume the rest. This ensures ->clock_task is + * monotonic. + * + * It does however cause some slight miss-attribution of {soft,}IRQ + * time, a more accurate solution would be to update the irq_time using + * the current rq->clock timestamp, except that would require using + * atomic ops. + */ + if (irq_delta > delta) + irq_delta =3D delta; =20 - rq->prev_irq_time +=3D irq_delta; - delta -=3D irq_delta; - delayacct_irq(rq->curr, irq_delta); + rq->prev_irq_time +=3D irq_delta; + delta -=3D irq_delta; + delayacct_irq(rq->curr, irq_delta); + } #endif #ifdef CONFIG_PARAVIRT_TIME_ACCOUNTING if (static_key_false((¶virt_steal_rq_enabled))) { --=20 2.43.5 From nobody Wed Dec 17 17:59:32 2025 Received: from mail-pf1-f169.google.com (mail-pf1-f169.google.com [209.85.210.169]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 238508F5B; Sun, 15 Dec 2024 03:23:37 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.169 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734233019; cv=none; b=iAIfqn4W8DEgOutfc8geujWlzWF5LwtOqVIxXxLjAmymgI51QcqA5R4Jd/oYC4boYZ+dLR/n7HvhFmRQOxjSbu5h7iXlh/Q8q831gaVvgL5jUAtWSBP0TmYjqc05Ihe/fsnfEpmCIvxuVkKqeqtXnQwdMoKDoAcBu79p09Dza/o= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734233019; c=relaxed/simple; bh=jjsXex1Xl/raNdbKQJH8s8+9pwFrdykkMbetCt2qLW0=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version:Content-Type; b=b8U3kmTfSZJUZ+j63sqSL1e8zkbO4kZisAmTl5oectGxmVgZ9q5ziRY5SQz7+UIJYYpoTbRTMIvCVEdgMsCAUlmDwqkrPjus4Pc/nZTAmB2YVDvVjuDjbs9IRHa3tO4kY0H0wOMv6E+k/AzbPF2nP44dATZGemvXsUZtUkwE9DU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=m4dNmkCb; arc=none smtp.client-ip=209.85.210.169 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="m4dNmkCb" Received: by mail-pf1-f169.google.com with SMTP id d2e1a72fcca58-725f2f79ed9so2276367b3a.2; Sat, 14 Dec 2024 19:23:37 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1734233017; x=1734837817; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=2tiXygfzzGb3qZx3mLMCF1CBYULfvbDzHXwBRr7eEbg=; b=m4dNmkCbQyhfBcIZYc4aMjD24rEKbEiRBSgPxqynRw4hsNZMRzuhaYesOdFSgstPbn Oh/OCH0REMWFI4h7QppRGFhCh4HTVwSnCMWsuHNeICDRy2R4Jdk6+fCy/+oKfbzEsrUn qeGZ6iMWoUGxzLY2qGhjdY4Ajul4mxM6JiV4jVuDOoxmAti6dbVLtmKn+2/C2jyiSiHU C8+CV9wsqC+5T7dU0fpY7loydsbBsdE4MAQbVfdmmDLh6KAwcQYxwSQJbYGT2k4lRpK+ TAwyY9T6/HkGsGEZxZReXPO0Fc/WQRNRvAYQKGDE6zvaFbRT2z9SJ4pO+IMAmfOStmet capg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1734233017; x=1734837817; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=2tiXygfzzGb3qZx3mLMCF1CBYULfvbDzHXwBRr7eEbg=; b=i2NwFSTdAAoRtjKxLhr/mQxNbYTKiIS7sOWHAKdaSka65YivJVDYv/m7TGIC9Ek86U UNn7ylZ7G9VC/JLkyjz7VryfuSNELxpVVXH8HN6m2v2SfYYmYh6O2LbyStY7LRWVJeBc cWOBrol2DES0VhBtOHbYaio4Oh4O1fRxpqtV2xfBkxYj4SaMtWZKQuCQusn3l+gpTEin MSBd0wePRSwWBj9q4hYTipsRCvCa6i8arb2Cg0o4+MFDZRWfWyCpJMNS5k3wLp9aE1ZW /dVaTuceRUnGxP/uJt3S221vh7Lv7jBBRsoidRVbtA9hCWq5PC2HiPHv5KEwJUqUatv/ 7i5w== X-Forwarded-Encrypted: i=1; AJvYcCXCJ2ozSUa6yYrLpwIDQRRlcHEEpdqu9PBQOM81TM5pswTSerhvRjV6lAWpZQ5UpFTDggz7z5Hu@vger.kernel.org, AJvYcCXcH5j1MPShUOHDQ+n4HL3RFDUtXvinaHHXWEXY30JaP0baI/IQ3wym35dG1UEnHK2pXtctaT9m3Gz8X0HC@vger.kernel.org X-Gm-Message-State: AOJu0Yz3EvIGw+hKOUw2Ud6S4oCOZf0Gw4AnGbe3JPbO5kaHaxt86OIg d9fTTxCE2/hIDDO0HHG/nh6G+mhcjbc3tcGhspKUcoZNhg9pX8jKNRYSaqcNwrs= X-Gm-Gg: ASbGncsnckv4foU6VPUsdlt92BWWR2ZUxaatgvIflGPsrdfssBeD6RnIcrPTKTBILyC EXCiVC6DRNmDR2Q4MEBg2G2QiAOLivdp6NxiW8IOB1dV7FIBhJRjUdCmhzMv+G8aFJQgLDUoFMd 4h5CgX2/rM5T9Kf8XtEP1rL3Fcq1SBV+VP37Nh5v92RXpKXMZuCoyqOHXqy5ZMIx7aGr0tR2qbX zdrKly9AIQHWNu1FrY1Jo5kd0JWjcbfVbGV0nMlmrWXHVCj1tk7Wapb3aEJw//7iC5BUCqHmY5z Y0cxLd0= X-Google-Smtp-Source: AGHT+IFM5aopze3V3Q2iMtHWoWSjzxTzpDMhGsveZtQ8dcWisJ4flmNz+ag2QXcc27PQ+eqIJu+Nsw== X-Received: by 2002:a05:6a00:3309:b0:727:3ac3:7f9c with SMTP id d2e1a72fcca58-7290c12ee39mr12104699b3a.10.1734233017383; Sat, 14 Dec 2024 19:23:37 -0800 (PST) Received: from localhost.localdomain ([180.159.118.224]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-72918b78f1esm2176586b3a.92.2024.12.14.19.23.33 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Sat, 14 Dec 2024 19:23:36 -0800 (PST) From: Yafang Shao To: mingo@redhat.com, peterz@infradead.org, mkoutny@suse.com, hannes@cmpxchg.org Cc: juri.lelli@redhat.com, vincent.guittot@linaro.org, dietmar.eggemann@arm.com, rostedt@goodmis.org, bsegall@google.com, mgorman@suse.de, vschneid@redhat.com, surenb@google.com, linux-kernel@vger.kernel.org, cgroups@vger.kernel.org, Yafang Shao Subject: [PATCH v7 3/4] sched, psi: Don't account irq time if sched_clock_irqtime is disabled Date: Sun, 15 Dec 2024 11:23:14 +0800 Message-Id: <20241215032315.43698-4-laoar.shao@gmail.com> X-Mailer: git-send-email 2.37.1 (Apple Git-137.1) In-Reply-To: <20241215032315.43698-1-laoar.shao@gmail.com> References: <20241215032315.43698-1-laoar.shao@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable sched_clock_irqtime may be disabled due to the clock source. When disabled, irq_time_read() won't change over time, so there is nothing to account. We can save iterating the whole hierarchy on every tick and context switch. Signed-off-by: Yafang Shao Acked-by: Johannes Weiner Reviewed-by: Michal Koutn=C3=BD --- kernel/sched/psi.c | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/kernel/sched/psi.c b/kernel/sched/psi.c index 84dad1511d1e..893d454c82ae 100644 --- a/kernel/sched/psi.c +++ b/kernel/sched/psi.c @@ -998,7 +998,7 @@ void psi_account_irqtime(struct rq *rq, struct task_str= uct *curr, struct task_st s64 delta; u64 irq; =20 - if (static_branch_likely(&psi_disabled)) + if (static_branch_likely(&psi_disabled) || !irqtime_enabled()) return; =20 if (!curr->pid) @@ -1240,6 +1240,9 @@ int psi_show(struct seq_file *m, struct psi_group *gr= oup, enum psi_res res) if (static_branch_likely(&psi_disabled)) return -EOPNOTSUPP; =20 + if (!irqtime_enabled() && res =3D=3D PSI_IRQ) + return -EOPNOTSUPP; + /* Update averages before reporting them */ mutex_lock(&group->avgs_lock); now =3D sched_clock(); --=20 2.43.5 From nobody Wed Dec 17 17:59:32 2025 Received: from mail-pf1-f181.google.com (mail-pf1-f181.google.com [209.85.210.181]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9FF17BE46; Sun, 15 Dec 2024 03:23:42 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.181 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734233024; cv=none; b=JA4Xz+9P0fuG5MqakgmtcKac/zLpZMJz7H9KCKMWhP/DqwDAihjlqbKVtWDNb72oNvz9tEB7+KQ9OsBdpsAUkcVXea1yP7OfOnfMBkKsyoBN09mu8ghgVCyeLsB0UvfB88Zq5iJIuGx0i274VaKn36hWTIHqm79HeJ+zB1MtVPo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734233024; c=relaxed/simple; bh=SNPX4EfxgJ85jzAxQBtGA4qctFecg38aQ1ua4Td2t5k=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version:Content-Type; b=E8XCjkly9Ke9DXWd/U2F6kT2rPNQsQbMvsFIjq242EWHNlvu8JIbam65gaC3jT06bdAkOr1hvhsEs9KlN7DPeW1h1Hhg+KvrPbvrmT3n6sM9UEdDGx8y8y4NbSsXPaXaXmIx7sRhZ5YEcnVL/eA7nAwQFWAd3CyXe2p/DA9laKQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=GKwJHLyE; arc=none smtp.client-ip=209.85.210.181 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="GKwJHLyE" Received: by mail-pf1-f181.google.com with SMTP id d2e1a72fcca58-725f2f79ed9so2276396b3a.2; Sat, 14 Dec 2024 19:23:42 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1734233022; x=1734837822; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=HlcFHISrD5XleVb0eDpHCT0UGjFhU+jp5XrXYYSVbls=; b=GKwJHLyEDxh3LfdFDgPoVe3bp9zzF7IdP1aKdeMUB0n+nv5ojNR5oAwRnVUBsRAbTI wlNh5JudxJAzSH00z2tPYnHZaLKYLA9McL5uo2+KdnBaYzDG6rY3yqkf/w4hJtoIy5GG 8LEWA3bSZLiMFmfsrYIDmuc6mM5HuxP540qtK3K+KZRcNqSo36UqP/CFSFP1P9EeL+xX AVra04HNYLZSmV6kkarr5zds5iNOtiFR06NaTh2WGn11rTecZgZyCq2QwJ7xhz0GrTFo 2GWLjB3EgIYFgw8e4FOvh5ij9WXhZyjBVTjbL/3h2oRWKUVcsezoSmSgjZWvaeDVYiX0 PaVA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1734233022; x=1734837822; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=HlcFHISrD5XleVb0eDpHCT0UGjFhU+jp5XrXYYSVbls=; b=LiztvW5xngbikxQgUYIwYshvaDjKbeFlQgJVbKw7Wii6QdNnEluyY2Z5kDDExgjbFH ANdoF8CtBSqrR/YeZlatC1c7mwWbnddnaJjl7Iblubigq+7yQF3gT1DOGmnerBMmPpdL dNC4t4h1+GLhcRPXB20MyKjHcrFveHsOqBWl2/HmfL8X1XcpwPmQsr7+IHBg37WOYKL8 T9z6NC9WHGlB3AcH0V9fj0UZT3E5tuuyCNw32C2RIuP0N0lE3pLLjH8UJ+rp/TDobs4l HmbBs9n84bUf2A5rNhXyDvzF7Tm0uhwuwD4CnoDCGnifQOKMiKnRw358z3ZOMYvV15Qo YJXQ== X-Forwarded-Encrypted: i=1; AJvYcCUs5N4F541ycZYQUZ8e2MZyNdv2CyWa6PV4ioBxAX/yFjaN7iH+iC6yT9aJW6BNjVt4mN1KBGdCaYn5+XBI@vger.kernel.org, AJvYcCWBaCutmHhXyrm3eBXlNYzi45piB9Ew75PRti7zAIqFySBbEXVDb052cEqReLfiI3C7KSJEyjoB@vger.kernel.org X-Gm-Message-State: AOJu0YyrrRESYKm2Dz2NxJTrqoizV9gn97U3AJGdQ1H1JMfQCUbFuASs 3RWSun2Q2SL20JQ5zsWcdDTmjk0uO8tPaQYn8bOnnhkjToqMSdTZ X-Gm-Gg: ASbGnctawdFjR63I979qJqHA+/YqnnSCGNdnJta37cKRfUTSBT97xM+OwF5N5MqgSm1 kVA1DNDgUwspuULqbpVv2r20ZFLU6pt1ZeYEnL9cJql6DZVL3xa+VrNBC6FyZELHkU5O4LYsho1 GBxgLN3npKUKu0HIbF7C2xzRa+mfoKXJ0Y+lYbJshSnVHMEmdswG+ktezX+p+yLHVJE7jkupZIu QUfJuDuZejS9u2bCJlTY/zYzFRjbqq8AyeBkuMBa1yD8j1hMlA3FeS6RxN19uj1rNMkdqSZIxty IV08G7Q= X-Google-Smtp-Source: AGHT+IE5HvOgkTTToCtWzeJMvFIMGWPY1+8ZEeVewiPMo0n/M8Wf2FR/N7vOTzFNq+S0YBQF3i70ZQ== X-Received: by 2002:a05:6a21:789d:b0:1db:ff9c:6f3a with SMTP id adf61e73a8af0-1e1dfe05229mr12942954637.42.1734233021853; Sat, 14 Dec 2024 19:23:41 -0800 (PST) Received: from localhost.localdomain ([180.159.118.224]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-72918b78f1esm2176586b3a.92.2024.12.14.19.23.37 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Sat, 14 Dec 2024 19:23:41 -0800 (PST) From: Yafang Shao To: mingo@redhat.com, peterz@infradead.org, mkoutny@suse.com, hannes@cmpxchg.org Cc: juri.lelli@redhat.com, vincent.guittot@linaro.org, dietmar.eggemann@arm.com, rostedt@goodmis.org, bsegall@google.com, mgorman@suse.de, vschneid@redhat.com, surenb@google.com, linux-kernel@vger.kernel.org, cgroups@vger.kernel.org, Yafang Shao Subject: [PATCH v7 4/4] sched: Fix cgroup irq time for CONFIG_IRQ_TIME_ACCOUNTING Date: Sun, 15 Dec 2024 11:23:15 +0800 Message-Id: <20241215032315.43698-5-laoar.shao@gmail.com> X-Mailer: git-send-email 2.37.1 (Apple Git-137.1) In-Reply-To: <20241215032315.43698-1-laoar.shao@gmail.com> References: <20241215032315.43698-1-laoar.shao@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable After enabling CONFIG_IRQ_TIME_ACCOUNTING to monitor IRQ pressure in our container environment, we observed several noticeable behavioral changes. One of our IRQ-heavy services, such as Redis, reported a significant reduction in CPU usage after upgrading to the new kernel with CONFIG_IRQ_TIME_ACCOUNTING enabled. However, despite adding more threads to handle an increased workload, the CPU usage could not be raised. In other words, even though the container=E2=80=99s CPU usage appeared low, it= was unable to process more workloads to utilize additional CPU resources, which caused issues. This behavior can be demonstrated using netperf: function start_server() { for j in `seq 1 3`; do netserver -p $[12345+j] > /dev/null & done } server_ip=3D$1 function start_client() { # That applies to cgroup2 as well. mkdir -p /sys/fs/cgroup/cpuacct/test echo $$ > /sys/fs/cgroup/cpuacct/test/cgroup.procs for j in `seq 1 3`; do port=3D$[12345+j] taskset -c 0 netperf -H ${server_ip} -l ${run_time:-30000} \ -t TCP_STREAM -p $port -- -D -m 1k -M 1K -s 8k -S 8k \ > /dev/null & done } start_server start_client We can verify the CPU usage of the test cgroup using cpuacct.stat. The output shows: system: 53 user: 2 The CPU usage of the cgroup is relatively low at around 55%, but this usage doesn't increase, even with more netperf tasks. The reason is that CPU0 is at 100% utilization, as confirmed by mpstat: 02:56:22 PM CPU %usr %nice %sys %iowait %irq %soft %steal = %guest %gnice %idle 02:56:23 PM 0 0.99 0.00 55.45 0.00 0.99 42.57 0.00 = 0.00 0.00 0.00 02:56:23 PM CPU %usr %nice %sys %iowait %irq %soft %steal = %guest %gnice %idle 02:56:24 PM 0 2.00 0.00 55.00 0.00 0.00 43.00 0.00 = 0.00 0.00 0.00 It is clear that the %soft is excluded in the cgroup of the interrupted task. This behavior is unexpected. We should include IRQ time in the cgroup to reflect the pressure the group is under. After a thorough analysis, I discovered that this change in behavior is due to commit 305e6835e055 ("sched: Do not account irq time to current task"), which altered whether IRQ time should be charged to the interrupted task. While I agree that a task should not be penalized by random interrupts, the task itself cannot progress while interrupted. Therefore, the interrupted time should be reported to the user. The system metric in cpuacct.stat is crucial in indicating whether a container is under heavy system pressure, including IRQ/softirq activity. Hence, IRQ/softirq time should be included in the cpuacct system usage, which also applies to cgroup2=E2=80=99s rstat. The reason it doesn't just add the cgroup_account_*() to irqtime_account_irq() is that it might result in performance hit to hold the rq_lock in the critical path. Taking inspiration from commit ddae0ca2a8fe ("sched: Move psi_account_irqtime() out of update_rq_clock_task() hotpath"), I've now adapted the approach to handle it in a non-critical path, reducing the performance impact. Signed-off-by: Yafang Shao Cc: Johannes Weiner Cc: Michal Koutn=C3=BD --- kernel/sched/core.c | 33 +++++++++++++++++++++++++++++++-- kernel/sched/psi.c | 13 +++---------- kernel/sched/sched.h | 2 +- kernel/sched/stats.h | 7 ++++--- 4 files changed, 39 insertions(+), 16 deletions(-) diff --git a/kernel/sched/core.c b/kernel/sched/core.c index 22dfcd3e92ed..7faacf320af9 100644 --- a/kernel/sched/core.c +++ b/kernel/sched/core.c @@ -5622,6 +5622,35 @@ __setup("resched_latency_warn_ms=3D", setup_resched_= latency_warn_ms); static inline u64 cpu_resched_latency(struct rq *rq) { return 0; } #endif /* CONFIG_SCHED_DEBUG */ =20 +#ifdef CONFIG_IRQ_TIME_ACCOUNTING +static void account_irqtime(struct rq *rq, struct task_struct *curr, + struct task_struct *prev) +{ + int cpu =3D smp_processor_id(); + s64 delta; + u64 irq; + + if (!irqtime_enabled()) + return; + + irq =3D irq_time_read(cpu); + delta =3D (s64)(irq - rq->irq_time); + if (delta < 0) + return; + + rq->irq_time =3D irq; + psi_account_irqtime(rq, curr, prev, delta); + cgroup_account_cputime(curr, delta); + /* We account both softirq and irq into CPUTIME_IRQ */ + cgroup_account_cputime_field(curr, CPUTIME_IRQ, delta); +} +#else +static inline void account_irqtime(struct rq *rq, struct task_struct *curr, + struct task_struct *prev) +{ +} +#endif + /* * This function gets called by the timer code, with HZ frequency. * We call it with interrupts disabled. @@ -5644,7 +5673,7 @@ void sched_tick(void) rq_lock(rq, &rf); donor =3D rq->donor; =20 - psi_account_irqtime(rq, donor, NULL); + account_irqtime(rq, donor, NULL); =20 update_rq_clock(rq); hw_pressure =3D arch_scale_hw_pressure(cpu_of(rq)); @@ -6751,7 +6780,7 @@ static void __sched notrace __schedule(int sched_mode) ++*switch_count; =20 migrate_disable_switch(rq, prev); - psi_account_irqtime(rq, prev, next); + account_irqtime(rq, prev, next); psi_sched_switch(prev, next, block); =20 trace_sched_switch(preempt, prev, next, prev_state); diff --git a/kernel/sched/psi.c b/kernel/sched/psi.c index 893d454c82ae..68657444efb9 100644 --- a/kernel/sched/psi.c +++ b/kernel/sched/psi.c @@ -990,15 +990,14 @@ void psi_task_switch(struct task_struct *prev, struct= task_struct *next, } =20 #ifdef CONFIG_IRQ_TIME_ACCOUNTING -void psi_account_irqtime(struct rq *rq, struct task_struct *curr, struct t= ask_struct *prev) +void psi_account_irqtime(struct rq *rq, struct task_struct *curr, struct t= ask_struct *prev, + s64 delta) { int cpu =3D task_cpu(curr); struct psi_group *group; struct psi_group_cpu *groupc; - s64 delta; - u64 irq; =20 - if (static_branch_likely(&psi_disabled) || !irqtime_enabled()) + if (static_branch_likely(&psi_disabled)) return; =20 if (!curr->pid) @@ -1009,12 +1008,6 @@ void psi_account_irqtime(struct rq *rq, struct task_= struct *curr, struct task_st if (prev && task_psi_group(prev) =3D=3D group) return; =20 - irq =3D irq_time_read(cpu); - delta =3D (s64)(irq - rq->psi_irq_time); - if (delta < 0) - return; - rq->psi_irq_time =3D irq; - do { u64 now; =20 diff --git a/kernel/sched/sched.h b/kernel/sched/sched.h index 7e8c73110884..570cc19a0060 100644 --- a/kernel/sched/sched.h +++ b/kernel/sched/sched.h @@ -1227,7 +1227,7 @@ struct rq { =20 #ifdef CONFIG_IRQ_TIME_ACCOUNTING u64 prev_irq_time; - u64 psi_irq_time; + u64 irq_time; #endif #ifdef CONFIG_PARAVIRT u64 prev_steal_time; diff --git a/kernel/sched/stats.h b/kernel/sched/stats.h index 8ee0add5a48a..b9ed9fea5ab7 100644 --- a/kernel/sched/stats.h +++ b/kernel/sched/stats.h @@ -111,10 +111,11 @@ void psi_task_change(struct task_struct *task, int cl= ear, int set); void psi_task_switch(struct task_struct *prev, struct task_struct *next, bool sleep); #ifdef CONFIG_IRQ_TIME_ACCOUNTING -void psi_account_irqtime(struct rq *rq, struct task_struct *curr, struct t= ask_struct *prev); +void psi_account_irqtime(struct rq *rq, struct task_struct *curr, + struct task_struct *prev, s64 delta); #else static inline void psi_account_irqtime(struct rq *rq, struct task_struct *= curr, - struct task_struct *prev) {} + struct task_struct *prev, s64 delta) {} #endif /*CONFIG_IRQ_TIME_ACCOUNTING */ /* * PSI tracks state that persists across sleeps, such as iowaits and @@ -224,7 +225,7 @@ static inline void psi_sched_switch(struct task_struct = *prev, struct task_struct *next, bool sleep) {} static inline void psi_account_irqtime(struct rq *rq, struct task_struct *= curr, - struct task_struct *prev) {} + struct task_struct *prev, s64 delta) {} #endif /* CONFIG_PSI */ =20 #ifdef CONFIG_SCHED_INFO --=20 2.43.5