From nobody Sun Dec 14 13:53:29 2025 Received: from mail-pf1-f176.google.com (mail-pf1-f176.google.com [209.85.210.176]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 18F6E1F193D; Wed, 11 Dec 2024 13:17:52 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.176 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1733923074; cv=none; b=DwJ3XXXXlOVh3Ps/YWaHbsW3v/w70tE79UcFHqUpT3Wins72BEeR7lmAW80MHjXBgclPzu0ndYcAdm/yT0gFajKDdZGmLBlG4yw/yVLGtmmckdwIEysSh64XukEEdiugLJDyaJX6nEBEsOA7trQTRQsD2dajQT4wL5ub0RiF5sY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1733923074; c=relaxed/simple; bh=YKut9eFLbv8wkhW3Aw73UqZamaMGzSnQhbMHmznk78U=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version:Content-Type; b=aQOj2MdZ+hZhu52MfhlnFW83U+NW7oy38oDmcHnOemPKffBGXaaFpXT3iQxn1Icvbv9heW7xnuXk5A4ibk2UCdwmjbQWERe8vS31I8yYM6OirGolXG2hv2S+A7XZN3IMqC5/dd4BeewQorhU1JRjD/dnuffliQSgHwjXEaQUm4I= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=lQJiAX3+; arc=none smtp.client-ip=209.85.210.176 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="lQJiAX3+" Received: by mail-pf1-f176.google.com with SMTP id d2e1a72fcca58-725ea1e19f0so2899718b3a.3; Wed, 11 Dec 2024 05:17:52 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1733923072; x=1734527872; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=99YVoxP4xRvn770uvDfh6WBhPwP6s830fS2ZBhYtLpw=; b=lQJiAX3+R33diuZA6JY6PhBI6aUrNpWIioms+AyDrWF7LA1ql50j5eECqZUtFft3D5 mEAvBEQ3NrIqyylmo2SNKjMPWFynJUgO6bAfsfOdKI/2+drdQzTS5hBapsoiUmAeDagQ WdEBBdEU7Jgyxu9HRNEZ+/cvB6ugqxGcfO2U60fPP+2ackKthqpiEMkdKBF/C82V0I+T 6+SV0rKtKjLMNwr+VzZz6BsjrBmTAuCQUXCqBEXHA0836VuxeskxTWhhDy1u2xL8euk5 66ssDiThVGA77Veru+QRuBGs0506nMIL2xjhxZFX5V+X7k+bub8pTaiUFnHnFl/aBej1 vk6Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1733923072; x=1734527872; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=99YVoxP4xRvn770uvDfh6WBhPwP6s830fS2ZBhYtLpw=; b=nJjrdfYFzTgX4uEoIH4J6QKm/gepu0tbXsRyc4BCrWPreQ4ExNGp9I8B3Kw/WYDQon nLq3Sj5IMD6oMIqcKdhxpECyGjRjRbDIAEhtymeCdEFXHvGZCNipel9aNclZNXxcIdTX hnWvc+hrZBxC987b9+mFXhb21iK7DLOmolayiNwF+xY7dpIcdrtN5Et5tAy42BXHVB1h KciQm4jhqh/LoyaqFwuR1jcb506AF5REW+TLXOp332+FxyeI1o9mYUXTz4dVznCH6Ik/ NNFvC6/ShNmJB9DTVXB8Nxi4ejxxxXGohhYl5lHQa0RSBKPaFc7UIuy/OOeGJoocLjsm y8Cg== X-Forwarded-Encrypted: i=1; AJvYcCUHJdoeDgW0vXcpFd+tyJeYuSwvvSDD7gFrgsKk2pFnXW5xUVDCjvhKWgvgEP9RqgOJZUg4YktX@vger.kernel.org, AJvYcCWBwl6yxrPT6n56AjPxuvjbnViUdxWy5RML+6A8Emovwaynefn0XsYZQgvPOXSkriMIL/4fpvrODzVkSf4k@vger.kernel.org X-Gm-Message-State: AOJu0YwzzS9KKpKqZyIbUXNOSIRGPoLgAxjvWuba9dkIdhyEdS4Mdqby ryHEhh95TUJ5S08nbVH4uelPDuJFvUHkCDn6jy1dNGVQrItqX6Oa87LibxOTg04= X-Gm-Gg: ASbGncvTfQTl9+m24AKqDlKvCdIQxBY1aEHde9dtsmlXKxTzRsROwoyQc7dLjZZZApl PIh6gR1vh9E7NxTc7owAy1SLXs192ZA7af4FCOVdZwZtJdTal8+w3ygVBm+/6m4Asx3k41Pwzmk LZhSCIkUjQ+LMmbKy/H8uWJ2EK8M6yOL3KryNGfw0EyaaWHTmcS50WBijmcKbikPEscjZGcLpLs 66BKMtq0u7zJHwiKf7LKNUD686yMGJcGAHPVAualkRYC0Ed7qnXGBL1XuzS6tZ5xskGrI29leY4 7frV X-Google-Smtp-Source: AGHT+IEpTLOhH60CQ9QFyGXAa5ujTRDB5UlMLhxPcUvvyDmyIDQ5Vlt1Hhgh0NjxPhEg4UpxYksp0g== X-Received: by 2002:a05:6a00:c90:b0:725:f4c6:6b71 with SMTP id d2e1a72fcca58-728ed48d927mr4810163b3a.20.1733923072269; Wed, 11 Dec 2024 05:17:52 -0800 (PST) Received: from localhost.localdomain ([180.159.118.224]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-725eeb536a1sm5272628b3a.115.2024.12.11.05.17.44 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Wed, 11 Dec 2024 05:17:51 -0800 (PST) From: Yafang Shao To: mingo@redhat.com, peterz@infradead.org, mkoutny@suse.com, hannes@cmpxchg.org Cc: juri.lelli@redhat.com, vincent.guittot@linaro.org, dietmar.eggemann@arm.com, rostedt@goodmis.org, bsegall@google.com, mgorman@suse.de, vschneid@redhat.com, surenb@google.com, linux-kernel@vger.kernel.org, cgroups@vger.kernel.org, Yafang Shao Subject: [PATCH v6 1/4] sched: Define sched_clock_irqtime as static key Date: Wed, 11 Dec 2024 21:17:26 +0800 Message-Id: <20241211131729.43996-2-laoar.shao@gmail.com> X-Mailer: git-send-email 2.37.1 (Apple Git-137.1) In-Reply-To: <20241211131729.43996-1-laoar.shao@gmail.com> References: <20241211131729.43996-1-laoar.shao@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Since CPU time accounting is a performance-critical path, let's define sched_clock_irqtime as a static key to minimize potential overhead. Signed-off-by: Yafang Shao Reviewed-by: Michal Koutn=C3=BD --- kernel/sched/cputime.c | 16 +++++++--------- kernel/sched/sched.h | 13 +++++++++++++ 2 files changed, 20 insertions(+), 9 deletions(-) diff --git a/kernel/sched/cputime.c b/kernel/sched/cputime.c index 0bed0fa1acd9..5d9143dd0879 100644 --- a/kernel/sched/cputime.c +++ b/kernel/sched/cputime.c @@ -9,6 +9,8 @@ =20 #ifdef CONFIG_IRQ_TIME_ACCOUNTING =20 +DEFINE_STATIC_KEY_FALSE(sched_clock_irqtime); + /* * There are no locks covering percpu hardirq/softirq time. * They are only modified in vtime_account, on corresponding CPU @@ -22,16 +24,14 @@ */ DEFINE_PER_CPU(struct irqtime, cpu_irqtime); =20 -static int sched_clock_irqtime; - void enable_sched_clock_irqtime(void) { - sched_clock_irqtime =3D 1; + static_branch_enable(&sched_clock_irqtime); } =20 void disable_sched_clock_irqtime(void) { - sched_clock_irqtime =3D 0; + static_branch_disable(&sched_clock_irqtime); } =20 static void irqtime_account_delta(struct irqtime *irqtime, u64 delta, @@ -57,7 +57,7 @@ void irqtime_account_irq(struct task_struct *curr, unsign= ed int offset) s64 delta; int cpu; =20 - if (!sched_clock_irqtime) + if (!irqtime_enabled()) return; =20 cpu =3D smp_processor_id(); @@ -90,8 +90,6 @@ static u64 irqtime_tick_accounted(u64 maxtime) =20 #else /* CONFIG_IRQ_TIME_ACCOUNTING */ =20 -#define sched_clock_irqtime (0) - static u64 irqtime_tick_accounted(u64 dummy) { return 0; @@ -478,7 +476,7 @@ void account_process_tick(struct task_struct *p, int us= er_tick) if (vtime_accounting_enabled_this_cpu()) return; =20 - if (sched_clock_irqtime) { + if (irqtime_enabled()) { irqtime_account_process_tick(p, user_tick, 1); return; } @@ -507,7 +505,7 @@ void account_idle_ticks(unsigned long ticks) { u64 cputime, steal; =20 - if (sched_clock_irqtime) { + if (irqtime_enabled()) { irqtime_account_idle_ticks(ticks); return; } diff --git a/kernel/sched/sched.h b/kernel/sched/sched.h index 6c54a57275cc..38aceb7289b7 100644 --- a/kernel/sched/sched.h +++ b/kernel/sched/sched.h @@ -3209,6 +3209,12 @@ struct irqtime { }; =20 DECLARE_PER_CPU(struct irqtime, cpu_irqtime); +DECLARE_STATIC_KEY_FALSE(sched_clock_irqtime); + +static inline int irqtime_enabled(void) +{ + return static_branch_likely(&sched_clock_irqtime); +} =20 /* * Returns the irqtime minus the softirq time computed by ksoftirqd. @@ -3229,6 +3235,13 @@ static inline u64 irq_time_read(int cpu) return total; } =20 +#else + +static inline int irqtime_enabled(void) +{ + return 0; +} + #endif /* CONFIG_IRQ_TIME_ACCOUNTING */ =20 #ifdef CONFIG_CPU_FREQ --=20 2.43.5 From nobody Sun Dec 14 13:53:29 2025 Received: from mail-pf1-f179.google.com (mail-pf1-f179.google.com [209.85.210.179]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 826B022B59D; Wed, 11 Dec 2024 13:18:01 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.179 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1733923083; cv=none; b=h41BBIWf6c71D4JC2DoYrebh7VTL2CnZxMMHF/gVw/TYt/W0fEDQ9vwYGrCRge/KtOjS95eoXJAey0Ct9kCiSv33ydENt5LWLUURvm2DfXmsxAVuO8dwB+AtxzEDqsMj63CioFSCu0fLcdBk0BexPX7snPz30Ozg8lbi5BQenug= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1733923083; c=relaxed/simple; bh=d50T/0zGgVUyO6cd0DNoB3JWuubb13PgiEr69YXBdWw=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version:Content-Type; b=A7FF6n1xe8TgkXwaqEVrqvvOgZxN/pLVnR1gv7LIZ7os2/E8x/yxjuWMDLimM4wluoq4yynNbVwd0r0hIJU5Bz8+FtbzCYO/lUDLNDvBevOvl153pQRis1oF+JI/GnQtyM4BSNzSmkN4SO8cNbsw1OFgyw+DRGzsGh669tp4wXE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=WzoOU8WS; arc=none smtp.client-ip=209.85.210.179 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="WzoOU8WS" Received: by mail-pf1-f179.google.com with SMTP id d2e1a72fcca58-72739105e02so2738671b3a.0; Wed, 11 Dec 2024 05:18:01 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1733923081; x=1734527881; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=rfhKjLsWJbVZ1NKZ4UFy6xoLQKlWruWzaoZaagpdc1c=; b=WzoOU8WSZWWWKh7FhEslogeNHFNl4FiUGFPJcQQeukYuCVqRQYMexkbvlUYV65H4P/ 6dPoxk7IvfqgZkdu4lfD50xR7lZxJE6JblYnSU7HUG8WIHhO/JaxJ0KFPQJ5z8nkpO30 EQUxPDuBJGqOP/p/D3S/NZKb2iJskf16USTmRmANs788bd/QRGrrxfvSGTtHhbvYs0+5 XuEQGAQ1G0hJBi+tkymsu4i4OkyAHkNP/I2crUoOgAxDYK7aT4Ijcq9nT5TlMWDPlqsY kHYYnK6KvEQRrDn599Yu0Av4leIw5wfudG/EQaH4QN8lZsKhdtCv6KIHNgxFE48UYLL1 6H7A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1733923081; x=1734527881; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=rfhKjLsWJbVZ1NKZ4UFy6xoLQKlWruWzaoZaagpdc1c=; b=D0Uh54HPTHwaLF0v3zRMOWCn5WznspTKOIHbaf5DeK52h4XwYqn/+J5nuJJvJzfPBf rzAGQ9/R6oVPieCcZPA4mFUtJmS/pOCDu2WMOjyK6H/JKLRFgZb5b3WfvegFEcVG8oqu Ku8K8gLy4oNZVfICDL1SPBjY8sqHOC7sVPPH2ecycnIWYG52ieflDH+Z5nI9erw/kUz6 +73e0xQKGdS5qXMcJc81CCcRKb6FHTNIvji6Oyf6n0jv0h65pv031giRfK7XMUhkx7Bh Z42Q6JfiVBvk0ChK6PIm0/BnLctC86Q/Rkqidr6HqgzgJE99S8l0nuk8HaENmyoyYt/F t+AQ== X-Forwarded-Encrypted: i=1; AJvYcCVHg0lXXlZtMB3Aop8HVRYI9aa8dreglGq/iYbc5UStI5c32sZYwdD/ZGYMSisOKDZvfOUtg8LS@vger.kernel.org, AJvYcCWgTFmNyh4uSH0NlZysI+a6qGWevUqE0vBVVFc/pfhURMFgHo6UaTos0LMdF/U3XBTzshDzrvZInOdUt03m@vger.kernel.org X-Gm-Message-State: AOJu0Yy04+IhpDj4lJwKxMW3PvX+6ps2X1t5zHUc0CnmKHQtNvWD82aX 7sKiURZ6xCCGfYIg2tkVkMD+IGK4JZNF9Sn5LEH7298ACUcqs5V0 X-Gm-Gg: ASbGncvRpkOkArqawK8GgcWhyFjfA/zbsxNGXpTUqsUbb0YqzeekbTg4Yrsfjk8oJI+ DPkKLHDaSAnxiUHTxgEq/FUcG4l3qO1Zu/45Cqu0GgcPrpQxVJ79SQRnIOfFD7+O52hNmaS9buL 5qinMuK7n2RZIKX4W1z5RQ47G5ZEcAvkwm7zzhxZAOYtGqJ/pZqeEEImliNa3FNCfhr379Z4MmX pBRuwLJlWBG0yhJC0Jdv0RtUS7hN3lr63Mq9yQeQVsXeSXj1U1T4J96xGLjbwAtA+EOwxS8cvzc RXbM X-Google-Smtp-Source: AGHT+IHsWU4lPadKWw3hYEQiOSN5Npg7sIvfqlTKCMi3MM6NHDZEvh4Hdt3Tnlp1BJbmxGKJgdtXMQ== X-Received: by 2002:a05:6a21:4995:b0:1e1:c1a7:67ef with SMTP id adf61e73a8af0-1e1c1a7694bmr4344711637.30.1733923080647; Wed, 11 Dec 2024 05:18:00 -0800 (PST) Received: from localhost.localdomain ([180.159.118.224]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-725eeb536a1sm5272628b3a.115.2024.12.11.05.17.52 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Wed, 11 Dec 2024 05:17:59 -0800 (PST) From: Yafang Shao To: mingo@redhat.com, peterz@infradead.org, mkoutny@suse.com, hannes@cmpxchg.org Cc: juri.lelli@redhat.com, vincent.guittot@linaro.org, dietmar.eggemann@arm.com, rostedt@goodmis.org, bsegall@google.com, mgorman@suse.de, vschneid@redhat.com, surenb@google.com, linux-kernel@vger.kernel.org, cgroups@vger.kernel.org, Yafang Shao Subject: [PATCH v6 2/4] sched: Don't account irq time if sched_clock_irqtime is disabled Date: Wed, 11 Dec 2024 21:17:27 +0800 Message-Id: <20241211131729.43996-3-laoar.shao@gmail.com> X-Mailer: git-send-email 2.37.1 (Apple Git-137.1) In-Reply-To: <20241211131729.43996-1-laoar.shao@gmail.com> References: <20241211131729.43996-1-laoar.shao@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable sched_clock_irqtime may be disabled due to the clock source, in which case IRQ time should not be accounted. Let's add a conditional check to avoid unnecessary logic. Signed-off-by: Yafang Shao Reviewed-by: Michal Koutn=C3=BD --- kernel/sched/core.c | 44 +++++++++++++++++++++++--------------------- 1 file changed, 23 insertions(+), 21 deletions(-) diff --git a/kernel/sched/core.c b/kernel/sched/core.c index 719e0ed1e976..a2a8d3d8da25 100644 --- a/kernel/sched/core.c +++ b/kernel/sched/core.c @@ -740,29 +740,31 @@ static void update_rq_clock_task(struct rq *rq, s64 d= elta) s64 __maybe_unused steal =3D 0, irq_delta =3D 0; =20 #ifdef CONFIG_IRQ_TIME_ACCOUNTING - irq_delta =3D irq_time_read(cpu_of(rq)) - rq->prev_irq_time; + if (irqtime_enabled()) { + irq_delta =3D irq_time_read(cpu_of(rq)) - rq->prev_irq_time; =20 - /* - * Since irq_time is only updated on {soft,}irq_exit, we might run into - * this case when a previous update_rq_clock() happened inside a - * {soft,}IRQ region. - * - * When this happens, we stop ->clock_task and only update the - * prev_irq_time stamp to account for the part that fit, so that a next - * update will consume the rest. This ensures ->clock_task is - * monotonic. - * - * It does however cause some slight miss-attribution of {soft,}IRQ - * time, a more accurate solution would be to update the irq_time using - * the current rq->clock timestamp, except that would require using - * atomic ops. - */ - if (irq_delta > delta) - irq_delta =3D delta; + /* + * Since irq_time is only updated on {soft,}irq_exit, we might run into + * this case when a previous update_rq_clock() happened inside a + * {soft,}IRQ region. + * + * When this happens, we stop ->clock_task and only update the + * prev_irq_time stamp to account for the part that fit, so that a next + * update will consume the rest. This ensures ->clock_task is + * monotonic. + * + * It does however cause some slight miss-attribution of {soft,}IRQ + * time, a more accurate solution would be to update the irq_time using + * the current rq->clock timestamp, except that would require using + * atomic ops. + */ + if (irq_delta > delta) + irq_delta =3D delta; =20 - rq->prev_irq_time +=3D irq_delta; - delta -=3D irq_delta; - delayacct_irq(rq->curr, irq_delta); + rq->prev_irq_time +=3D irq_delta; + delta -=3D irq_delta; + delayacct_irq(rq->curr, irq_delta); + } #endif #ifdef CONFIG_PARAVIRT_TIME_ACCOUNTING if (static_key_false((¶virt_steal_rq_enabled))) { --=20 2.43.5 From nobody Sun Dec 14 13:53:29 2025 Received: from mail-pf1-f171.google.com (mail-pf1-f171.google.com [209.85.210.171]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C3F0E23369B; Wed, 11 Dec 2024 13:18:08 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.171 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1733923090; cv=none; b=WMu9XRWh/+2pi5Tv5rxj+f3YPBjqPdtvhX1iOV4unaJ7KucQLn+cYs379pi6WoIWWddshYV1ri+QmdhAvCDPBVKSbyLMU0GdDoPf9FlUQ9tnWZ79eDEA/+uZntuZdPrLxRx6dnBGTJVGFbMz/RpRZHam71Pv/ytSC+UyxqZwB6s= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1733923090; c=relaxed/simple; bh=vKkrBqKm9COEooD0h7vQYFL3hcLGirif2vplKVkqqdg=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version:Content-Type; b=FpA+f/rZaboYB9PRjx49WXv7B7b4omIBNyNiguid3lWwA6novHrbrEKPFaT3u29niVRj4qrZzTl4TT8y2D6sKTSLLutZBVGm73J5nnExxl+1ZsD9KfHuy56X8MvBjVyqHt3W6LjiJ+7U8aCsBfWkkc63zAUzVjqhUjvPlBgQ938= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=HE6ZVgxk; arc=none smtp.client-ip=209.85.210.171 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="HE6ZVgxk" Received: by mail-pf1-f171.google.com with SMTP id d2e1a72fcca58-725ea1e19f0so2899987b3a.3; Wed, 11 Dec 2024 05:18:08 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1733923088; x=1734527888; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=/0KyEyNTdp/tWCBWO59LjYCklod3SgANT6I6z/q0u0k=; b=HE6ZVgxk9jFq3qANyltb/DrBkflZFd1NRo4WyvaOSBSG0mLXN4lUmazvkNEZnwYOMm lKemjK1A4l60zJJ5KxYviPw80QOgj/H8EoTktsURihMkhJB3LbuSVLi3p6YZFIYz64XK j9/ES5IqqFLJYwVfkb5dnSJ+HeszGt50KV3WLhkRzohrA7NDsyDz0zzflQZk4CfXD1sm 7ql4rk+OUNEUZdMxWpZH2maq34k/HIMoY1sT48DfsRFBHf8OSqV2eC+NptWtTtMv0Mp7 B/DJjJrRMBJ6ozwoexNZuSP9TFM1PEZ/pCorn5x2k/3blqHAr9OJ5OmyrFsZvz5UjjUc zgZg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1733923088; x=1734527888; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=/0KyEyNTdp/tWCBWO59LjYCklod3SgANT6I6z/q0u0k=; b=L4BNPhGM84DhYeHuSiMEHebXCW2ifwpCXylJhPaeTzdZ1NvJRBS/yO8I7RWZj2Jbut P0wzP5PcabakrXhG5Ko4H6VQ4A9jFYP3yC+yFIK1uIibVRMypZJOrNPQcpRe4lm95SSU QbwfsKi/vffodMMMuT3tUmW85v0XzeiyasYqUkHKDURTisyuWmeW9njGFkXjNUfLbblJ wcmszzIpbqQ+5gYld4+iqOXXnmhvsd8/jguhbyA95bUWCVoO8FpW8EBK0q4QTCswuJQw zsMhnxo/BAx0Tw7emHm6ghxtyJa9pPOM+mW4H5psH3DtH4swzcisf3aO5fI9ENKGGUbY ATjw== X-Forwarded-Encrypted: i=1; AJvYcCU/TGTyFLXYzCD8hz6xkILnO6VFWDm5dzUc9Nr1VFJRCiuqgjlZoFbWHHmUgMfZgJt/+z8d2WH7B3I77QeI@vger.kernel.org, AJvYcCXv7w40WQmdnn/mzCuMSpdrfyqbu4cw+MqC3bHroVJ5H58BOfs+I5eKDGniykB2rBa0IBX1k0uS@vger.kernel.org X-Gm-Message-State: AOJu0YzPgGmCEycwyLBnCZMWxL+GFHBsHay/WTOgIBJYCf8/ealxtqAQ ksCAWNKBM6ES3Fl4m2HVJFn90PTCQOboogUJG8Cd2xFe+7sUS5qw X-Gm-Gg: ASbGncsf9szkJy9YVIShikXasaBIqYUjWaLUVxCvCSuyfGRJY0SNBg7RHu5sO5/FHiX XShUN16QhQrQ3U3DCSvT5z3rmt57BxrPm8MHJiAF6OiGw83+rVjIE4OuK3VI4WHiuWXuVBRDjIv QOydriZ3/TtPw8JGfhQSvBkRIh3Zjz+4MtqQ0/fvPhDsV1joStfYbJ87EFAyh9JuQQ9YWuPaYfc q38MHl9OjdbyINVBte5GicfKuYLjAKuFxJDvLdmG6U65GAc1lHbOmyE6Gg9IhzZaVEEpzsLIB83 dwER X-Google-Smtp-Source: AGHT+IHH1w+18MF9BXwc6uIn6C8RpvtOv/r6dioYOS2ofRZ3nJQGv4zZXZsl4bf7K+zxuTc1TmRo/A== X-Received: by 2002:a05:6a20:43ac:b0:1e1:c07b:b087 with SMTP id adf61e73a8af0-1e1c109a134mr4471981637.0.1733923088020; Wed, 11 Dec 2024 05:18:08 -0800 (PST) Received: from localhost.localdomain ([180.159.118.224]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-725eeb536a1sm5272628b3a.115.2024.12.11.05.18.01 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Wed, 11 Dec 2024 05:18:07 -0800 (PST) From: Yafang Shao To: mingo@redhat.com, peterz@infradead.org, mkoutny@suse.com, hannes@cmpxchg.org Cc: juri.lelli@redhat.com, vincent.guittot@linaro.org, dietmar.eggemann@arm.com, rostedt@goodmis.org, bsegall@google.com, mgorman@suse.de, vschneid@redhat.com, surenb@google.com, linux-kernel@vger.kernel.org, cgroups@vger.kernel.org, Yafang Shao Subject: [PATCH v6 3/4] sched, psi: Don't account irq time if sched_clock_irqtime is disabled Date: Wed, 11 Dec 2024 21:17:28 +0800 Message-Id: <20241211131729.43996-4-laoar.shao@gmail.com> X-Mailer: git-send-email 2.37.1 (Apple Git-137.1) In-Reply-To: <20241211131729.43996-1-laoar.shao@gmail.com> References: <20241211131729.43996-1-laoar.shao@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable sched_clock_irqtime may be disabled due to the clock source. When disabled, irq_time_read() won't change over time, so there is nothing to account. We can save iterating the whole hierarchy on every tick and context switch. Signed-off-by: Yafang Shao Acked-by: Johannes Weiner Reviewed-by: Michal Koutn=C3=BD --- kernel/sched/psi.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/kernel/sched/psi.c b/kernel/sched/psi.c index 84dad1511d1e..6836c34d063e 100644 --- a/kernel/sched/psi.c +++ b/kernel/sched/psi.c @@ -998,7 +998,7 @@ void psi_account_irqtime(struct rq *rq, struct task_str= uct *curr, struct task_st s64 delta; u64 irq; =20 - if (static_branch_likely(&psi_disabled)) + if (static_branch_likely(&psi_disabled) || !irqtime_enabled()) return; =20 if (!curr->pid) @@ -1286,7 +1286,7 @@ struct psi_trigger *psi_trigger_create(struct psi_gro= up *group, char *buf, bool privileged; u32 window_us; =20 - if (static_branch_likely(&psi_disabled)) + if (static_branch_likely(&psi_disabled) || !irqtime_enabled()) return ERR_PTR(-EOPNOTSUPP); =20 /* --=20 2.43.5 From nobody Sun Dec 14 13:53:29 2025 Received: from mail-pf1-f180.google.com (mail-pf1-f180.google.com [209.85.210.180]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C0A0F2368F7; Wed, 11 Dec 2024 13:18:15 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.180 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1733923097; cv=none; b=sBtu0llm4NrS/ssvBFmecPrabUTiNsee6CapJi+lRkj3ogoa92L6JvuovEfAnmTRzT7SMl0BeYTJ3xVpby5kx9prfs5j7H1h533GRklCb5oRGoRaVNC8aap4tEUd+s0vKRXf8gbqUDZuoBBmDTEy9Geu33ikmAFPd9PKW1VEb4M= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1733923097; c=relaxed/simple; bh=0nPHnor7nHwzRB+2pWju/d6pFw1bPo1wapd6R1Uqq9Y=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version:Content-Type; b=PwhVmN3g7GN1gOqRnIj+t5cpxXoSK4uEgJzB6GhL7+zw6hA+cN8RkEIbXhfe1tgluCU1fjM4V9Fs6vqFQQoG7KP5KIbwgYP5fGUB44rjXPHsmyxcYJDEvXeQhayQM89Dx2tTn/mVw5tFdCSC2Pb2+dFeydqFvFURJbEzqFAbypY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=kYQuxN7s; arc=none smtp.client-ip=209.85.210.180 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="kYQuxN7s" Received: by mail-pf1-f180.google.com with SMTP id d2e1a72fcca58-728e3826211so1417999b3a.0; Wed, 11 Dec 2024 05:18:15 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1733923095; x=1734527895; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=QEsuW6xrVYcwy+l9jRBOUVbolWKY/E8cX1evWjlrM9A=; b=kYQuxN7sBpnZluOSpGkfARnvm7muyuq07NTJf1dWypVguGUQSW0nE8mGNhDO4lVwQn hkoZUL2JN3+TbzjzVpGOHqEfxTYRYXtZarKQHKqp4t0pXDUOoszwnxIbeaNvjcjXjgvV yhcEPu+2FSQ8RncFxqfL80btOjxCCeEpG5DtntKxHgbmjCPOmHn3frKCB6JC+Hcfsj4B SAI8ERJZ0VrSXbhfTEp7DH6bU1yyXhD0vaPBgw0lRg5HP2ZZ/hSbsUM+6IZnZDevMtnk eyQlURxO5unM0kVXzH4rAFjjf9EtIqTUZ+u/DwS9xLDYkPN12g7B2FVDSd1/sA0jC2SG lZxA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1733923095; x=1734527895; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=QEsuW6xrVYcwy+l9jRBOUVbolWKY/E8cX1evWjlrM9A=; b=F5YeZIn7bhdiX8CHbkrXxTa2hDkcRxwknCzwZR7HEg5PJFT32Au83i/JfNR8jakSgc 7qFeqw/MLv2V+K+9Ud2eDmsffnmBFxi/QctDzySGt9xEMEaUV/qS631CknQUuYYSy72n PcLgrLag/voxFIcQfBNzKfgaEz3D7tLRaJkZekJW2Xuxh5bDGDetIxuN1JPyLRfpGQO5 nEeKrzKDhR+Zt9+fJ8OoKq/zorHnwmCwB99N/FShsh7UEzfF0yXDzVSuG7nyIewz7mRA odX6MgWO/HdRVEXbbwabceliQPIXZfutIBSYjUmQBRbzmrn/tYJYR4aVJwtwUnhKvrAC jt7Q== X-Forwarded-Encrypted: i=1; AJvYcCUZ4YcBVS+vMe+tJ7uVrfGfTVRwJpQqDbPp4TpzfAgUoyy7Mlv/JStMPAuhYY2RPll+WLKf0FvO@vger.kernel.org, AJvYcCUd91mnyUtc7gPRp7saBoX0xJuKvaRz7xArknsyeqRExnyylvvURdkI/RR6LTEw3YtPRN/R87dae2e1kLYK@vger.kernel.org X-Gm-Message-State: AOJu0YyI8SwVidQycGXl0rIro2+qXmKXW+EBaZ7dYJQMK7eFeVMEgvhd wTXKshQdCCravX4kVU2L5Y97okqQXftv1qZhK4jxHD0vY05teqX9 X-Gm-Gg: ASbGncvKjh9dkhU1pz/BQ3djxQJnLyMTWutVePWLuFsRy0BPYi+SuQco5t2La6Mjx4q 8MQqncBbZ7WWCZ/RBTbtmRhLtJ65fQspdork/gEj+dszQ8nrpgjyflKBKUusebAbTZUwc7XNMmK fl4srLNbudZ8S+3Bc5gNIwU5IKODzavVag68mVZZSYEjIiVHyMfZXVrV6WNIVYGg4KqJxbWLl6G 9KBn2NSB94gnhi7vfZEtQ8IicLbKLvRPvkmZmIUPEqcC6/K7D/Eitz7PMVC1AZumQFQs2InDZYw 85Hh X-Google-Smtp-Source: AGHT+IEKnUknPfSY2YGuF8kUHgT1E2xf6sfPu7asatsIw6sT5Oi6r2qwtNLSQWDHXngXRLpt1v0uSg== X-Received: by 2002:a05:6a00:179b:b0:727:3935:dc83 with SMTP id d2e1a72fcca58-728ed3dca59mr4559921b3a.10.1733923095031; Wed, 11 Dec 2024 05:18:15 -0800 (PST) Received: from localhost.localdomain ([180.159.118.224]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-725eeb536a1sm5272628b3a.115.2024.12.11.05.18.08 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Wed, 11 Dec 2024 05:18:14 -0800 (PST) From: Yafang Shao To: mingo@redhat.com, peterz@infradead.org, mkoutny@suse.com, hannes@cmpxchg.org Cc: juri.lelli@redhat.com, vincent.guittot@linaro.org, dietmar.eggemann@arm.com, rostedt@goodmis.org, bsegall@google.com, mgorman@suse.de, vschneid@redhat.com, surenb@google.com, linux-kernel@vger.kernel.org, cgroups@vger.kernel.org, Yafang Shao Subject: [PATCH v6 4/4] sched: Fix cgroup irq time for CONFIG_IRQ_TIME_ACCOUNTING Date: Wed, 11 Dec 2024 21:17:29 +0800 Message-Id: <20241211131729.43996-5-laoar.shao@gmail.com> X-Mailer: git-send-email 2.37.1 (Apple Git-137.1) In-Reply-To: <20241211131729.43996-1-laoar.shao@gmail.com> References: <20241211131729.43996-1-laoar.shao@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable After enabling CONFIG_IRQ_TIME_ACCOUNTING to monitor IRQ pressure in our container environment, we observed several noticeable behavioral changes. One of our IRQ-heavy services, such as Redis, reported a significant reduction in CPU usage after upgrading to the new kernel with CONFIG_IRQ_TIME_ACCOUNTING enabled. However, despite adding more threads to handle an increased workload, the CPU usage could not be raised. In other words, even though the container=E2=80=99s CPU usage appeared low, it= was unable to process more workloads to utilize additional CPU resources, which caused issues. This behavior can be demonstrated using netperf: function start_server() { for j in `seq 1 3`; do netserver -p $[12345+j] > /dev/null & done } server_ip=3D$1 function start_client() { # That applies to cgroup2 as well. mkdir -p /sys/fs/cgroup/cpuacct/test echo $$ > /sys/fs/cgroup/cpuacct/test/cgroup.procs for j in `seq 1 3`; do port=3D$[12345+j] taskset -c 0 netperf -H ${server_ip} -l ${run_time:-30000} \ -t TCP_STREAM -p $port -- -D -m 1k -M 1K -s 8k -S 8k \ > /dev/null & done } start_server start_client We can verify the CPU usage of the test cgroup using cpuacct.stat. The output shows: system: 53 user: 2 The CPU usage of the cgroup is relatively low at around 55%, but this usage doesn't increase, even with more netperf tasks. The reason is that CPU0 is at 100% utilization, as confirmed by mpstat: 02:56:22 PM CPU %usr %nice %sys %iowait %irq %soft %steal = %guest %gnice %idle 02:56:23 PM 0 0.99 0.00 55.45 0.00 0.99 42.57 0.00 = 0.00 0.00 0.00 02:56:23 PM CPU %usr %nice %sys %iowait %irq %soft %steal = %guest %gnice %idle 02:56:24 PM 0 2.00 0.00 55.00 0.00 0.00 43.00 0.00 = 0.00 0.00 0.00 It is clear that the %soft is excluded in the cgroup of the interrupted task. This behavior is unexpected. We should include IRQ time in the cgroup to reflect the pressure the group is under. After a thorough analysis, I discovered that this change in behavior is due to commit 305e6835e055 ("sched: Do not account irq time to current task"), which altered whether IRQ time should be charged to the interrupted task. While I agree that a task should not be penalized by random interrupts, the task itself cannot progress while interrupted. Therefore, the interrupted time should be reported to the user. The system metric in cpuacct.stat is crucial in indicating whether a container is under heavy system pressure, including IRQ/softirq activity. Hence, IRQ/softirq time should be included in the cpuacct system usage, which also applies to cgroup2=E2=80=99s rstat. The reason it doesn't just add the cgroup_account_*() to irqtime_account_irq() is that it might result in performance hit to hold the rq_lock in the critical path. Taking inspiration from commit ddae0ca2a8fe ("sched: Move psi_account_irqtime() out of update_rq_clock_task() hotpath"), I've now adapted the approach to handle it in a non-critical path, reducing the performance impact. Signed-off-by: Yafang Shao Cc: Johannes Weiner Cc: Michal Koutn=C3=BD --- kernel/sched/core.c | 33 +++++++++++++++++++++++++++++++-- kernel/sched/psi.c | 13 +++---------- kernel/sched/sched.h | 2 +- kernel/sched/stats.h | 7 ++++--- 4 files changed, 39 insertions(+), 16 deletions(-) diff --git a/kernel/sched/core.c b/kernel/sched/core.c index a2a8d3d8da25..cf14ea8649df 100644 --- a/kernel/sched/core.c +++ b/kernel/sched/core.c @@ -5579,6 +5579,35 @@ __setup("resched_latency_warn_ms=3D", setup_resched_= latency_warn_ms); static inline u64 cpu_resched_latency(struct rq *rq) { return 0; } #endif /* CONFIG_SCHED_DEBUG */ =20 +#ifdef CONFIG_IRQ_TIME_ACCOUNTING +static void account_irqtime(struct rq *rq, struct task_struct *curr, + struct task_struct *prev) +{ + int cpu =3D smp_processor_id(); + s64 delta; + u64 irq; + + if (!irqtime_enabled()) + return; + + irq =3D irq_time_read(cpu); + delta =3D (s64)(irq - rq->irq_time); + if (delta < 0) + return; + + rq->irq_time =3D irq; + psi_account_irqtime(rq, curr, prev, delta); + cgroup_account_cputime(curr, delta); + /* We account both softirq and irq into CPUTIME_IRQ */ + cgroup_account_cputime_field(curr, CPUTIME_IRQ, delta); +} +#else +static inline void account_irqtime(struct rq *rq, struct task_struct *curr, + struct task_struct *prev) +{ +} +#endif + /* * This function gets called by the timer code, with HZ frequency. * We call it with interrupts disabled. @@ -5600,7 +5629,7 @@ void sched_tick(void) rq_lock(rq, &rf); =20 curr =3D rq->curr; - psi_account_irqtime(rq, curr, NULL); + account_irqtime(rq, curr, NULL); =20 update_rq_clock(rq); hw_pressure =3D arch_scale_hw_pressure(cpu_of(rq)); @@ -6683,7 +6712,7 @@ static void __sched notrace __schedule(int sched_mode) ++*switch_count; =20 migrate_disable_switch(rq, prev); - psi_account_irqtime(rq, prev, next); + account_irqtime(rq, prev, next); psi_sched_switch(prev, next, block); =20 trace_sched_switch(preempt, prev, next, prev_state); diff --git a/kernel/sched/psi.c b/kernel/sched/psi.c index 6836c34d063e..7341d33d9118 100644 --- a/kernel/sched/psi.c +++ b/kernel/sched/psi.c @@ -990,15 +990,14 @@ void psi_task_switch(struct task_struct *prev, struct= task_struct *next, } =20 #ifdef CONFIG_IRQ_TIME_ACCOUNTING -void psi_account_irqtime(struct rq *rq, struct task_struct *curr, struct t= ask_struct *prev) +void psi_account_irqtime(struct rq *rq, struct task_struct *curr, struct t= ask_struct *prev, + s64 delta) { int cpu =3D task_cpu(curr); struct psi_group *group; struct psi_group_cpu *groupc; - s64 delta; - u64 irq; =20 - if (static_branch_likely(&psi_disabled) || !irqtime_enabled()) + if (static_branch_likely(&psi_disabled)) return; =20 if (!curr->pid) @@ -1009,12 +1008,6 @@ void psi_account_irqtime(struct rq *rq, struct task_= struct *curr, struct task_st if (prev && task_psi_group(prev) =3D=3D group) return; =20 - irq =3D irq_time_read(cpu); - delta =3D (s64)(irq - rq->psi_irq_time); - if (delta < 0) - return; - rq->psi_irq_time =3D irq; - do { u64 now; =20 diff --git a/kernel/sched/sched.h b/kernel/sched/sched.h index 38aceb7289b7..09d920e8556c 100644 --- a/kernel/sched/sched.h +++ b/kernel/sched/sched.h @@ -1223,7 +1223,7 @@ struct rq { =20 #ifdef CONFIG_IRQ_TIME_ACCOUNTING u64 prev_irq_time; - u64 psi_irq_time; + u64 irq_time; #endif #ifdef CONFIG_PARAVIRT u64 prev_steal_time; diff --git a/kernel/sched/stats.h b/kernel/sched/stats.h index 767e098a3bd1..17eefe5876a5 100644 --- a/kernel/sched/stats.h +++ b/kernel/sched/stats.h @@ -111,10 +111,11 @@ void psi_task_change(struct task_struct *task, int cl= ear, int set); void psi_task_switch(struct task_struct *prev, struct task_struct *next, bool sleep); #ifdef CONFIG_IRQ_TIME_ACCOUNTING -void psi_account_irqtime(struct rq *rq, struct task_struct *curr, struct t= ask_struct *prev); +void psi_account_irqtime(struct rq *rq, struct task_struct *curr, + struct task_struct *prev, s64 delta); #else static inline void psi_account_irqtime(struct rq *rq, struct task_struct *= curr, - struct task_struct *prev) {} + struct task_struct *prev, s64 delta) {} #endif /*CONFIG_IRQ_TIME_ACCOUNTING */ /* * PSI tracks state that persists across sleeps, such as iowaits and @@ -215,7 +216,7 @@ static inline void psi_sched_switch(struct task_struct = *prev, struct task_struct *next, bool sleep) {} static inline void psi_account_irqtime(struct rq *rq, struct task_struct *= curr, - struct task_struct *prev) {} + struct task_struct *prev, s64 delta) {} #endif /* CONFIG_PSI */ =20 #ifdef CONFIG_SCHED_INFO --=20 2.43.5