From nobody Sun Feb 8 06:11:34 2026 Received: from mail-pj1-f54.google.com (mail-pj1-f54.google.com [209.85.216.54]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 40C2228684 for ; Mon, 23 Sep 2024 09:01:18 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.54 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1727082080; cv=none; b=ozfm0eGjkRMT0ZCkftReifpBnco3kfhw2EDxiV4QVWww70+u13EXAnM8WH/1WmWUbyNEG21aR16p3LyITgxI/m1XV0TaP7OV+1i4Ooo/gGiVUSfkaO0zZEnW/2h7Jnkeczx+c6GP5dmAj6kSEXkHpClmhtaMPYwXSW+ptCNprrc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1727082080; c=relaxed/simple; bh=aK3Ccy/qEp8lMx5Lda2LMCann47R9uZzRju5t0KsK2I=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version:Content-Type; b=BwzbF2xUW6sOJ2QS2VZ0W2Hn8c4/0MMrLZs4LChCa/vRsPWAZEHGhATmZKza5nB+OxVraTM1TQmbi2AcFBgncZecvD89MvswKco+URzKYlHSBv6XoxsG7SbAMktQq1Em+Lmu7nCSHdI/l3u/dxIcpKFpY0fCrRDmPV0+v53rAaI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=KF0KuK4t; arc=none smtp.client-ip=209.85.216.54 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="KF0KuK4t" Received: by mail-pj1-f54.google.com with SMTP id 98e67ed59e1d1-2db928dbd53so3261326a91.2 for ; Mon, 23 Sep 2024 02:01:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1727082077; x=1727686877; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=26Hym3zd0zIr6OsuYGjxkOtQ+t36plo/exdQlFtHHE8=; b=KF0KuK4tBm4x2jhLlKla3kYM6F984MM6mLCbMdh0NxoFN8bosDkKq6bnNHgXyFNGkm QGCx1CHUWrVF3SavgD4gf1qX6GTJdbvD8jdRhpn7UytUoSPeiWLDLwZh71oxtkPkJOdx is034xPpG/MEEpmv0x0sLJQIB8E2ofspJwI7Bl3y5soKwtNwWwwfCfZfvqEkFQuhX1y3 iOjMSIdoq20diINqt0MhhLosB0iXL6h2s55saSMnWLJTcQJ03Oas9n9lYWPCqKyB2XXn Zkb8GCBXra5WBC4YFqLaHkqOB3cxPKHq7/5QtccchIeAOddt+BCxl+Xg+fhzpldhRxyM uVpg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1727082077; x=1727686877; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=26Hym3zd0zIr6OsuYGjxkOtQ+t36plo/exdQlFtHHE8=; b=IRd8UnLXtP5wnlunM9NehEsqE5I9HpJRceNFno0if1RMT0sZ6/HBi6f4uptI4cmIC0 KpaQwidqJXaZepROM9bScFWsmVopsXYywdftBJXZNmWnCjUpmjfdjOWcYf2hnXswVxtk fEn53gAh9XgKI0ZCayEnm0yI0SoJuGOD8qoJF8m0GohMQ445AL7xq2VOm5t5eCRLLPLV jZTkw2+KnimzP303aIaY4WsQxO7Rr2S76vOeEFdCorJPesKrpBmNvwBLM9p3wKQVw+zo 9ZJoDsqyNO6vcJcNNMRwlsSZySuAnPuqe2CCHa11poGPgoPGDqAq8/24fJzjjYKt0zQ/ CXrg== X-Gm-Message-State: AOJu0Yy+1iJ4Rb2J8itIm+Wc0avpzsB8vCsUEldJ650mHhqfN5w4xKM5 j1U3535SDdOpZfIucz9QZJmkKYDutJaPRMbTWGXi5RiIvAqPoExL X-Google-Smtp-Source: AGHT+IHBFc/Q2nBf2o7S+Rqdldzgs4tyEE4NwcA+QpVdfluqiERN8RaTWcTwbk5+2iWDZGucRayFIQ== X-Received: by 2002:a17:90a:d908:b0:2d8:83ce:d4c0 with SMTP id 98e67ed59e1d1-2dd7f40ad24mr14320519a91.13.1727082077277; Mon, 23 Sep 2024 02:01:17 -0700 (PDT) Received: from localhost.localdomain ([39.144.43.152]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-2dd7f8ca487sm6768872a91.39.2024.09.23.02.01.10 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Mon, 23 Sep 2024 02:01:16 -0700 (PDT) From: Yafang Shao To: mingo@redhat.com, peterz@infradead.org, juri.lelli@redhat.com, vincent.guittot@linaro.org, dietmar.eggemann@arm.com, rostedt@goodmis.org, bsegall@google.com, mgorman@suse.de, vschneid@redhat.com Cc: linux-kernel@vger.kernel.org, Yafang Shao Subject: [PATCH 1/3] sched: Fix cgroup irq accounting for CONFIG_IRQ_TIME_ACCOUNTING Date: Mon, 23 Sep 2024 17:00:26 +0800 Message-Id: <20240923090028.16368-2-laoar.shao@gmail.com> X-Mailer: git-send-email 2.30.1 (Apple Git-130) In-Reply-To: <20240923090028.16368-1-laoar.shao@gmail.com> References: <20240923090028.16368-1-laoar.shao@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable After enabling CONFIG_IRQ_TIME_ACCOUNTING to monitor IRQ pressure in our container environment, we observed several noticeable behavioral changes. One of our IRQ-heavy services, such as Redis, reported a significant reduction in CPU usage after upgrading to the new kernel with CONFIG_IRQ_TIME_ACCOUNTING enabled. However, despite adding more threads to handle an increased workload, the CPU usage could not be raised. In other words, even though the container=E2=80=99s CPU usage appeared low, it= was unable to process more workloads to utilize additional CPU resources, which caused issues. This behavior can be demonstrated using netperf: function start_server() { for j in `seq 1 3`; do netserver -p $[12345+j] > /dev/null & done } server_ip=3D$1 function start_client() { # That applies to cgroup2 as well. mkdir -p /sys/fs/cgroup/cpuacct/test echo $$ > /sys/fs/cgroup/cpuacct/test/cgroup.procs for j in `seq 1 3`; do port=3D$[12345+j] taskset -c 0 netperf -H ${server_ip} -l ${run_time:-30000} \ -t TCP_STREAM -p $port -- -D -m 1k -M 1K -s 8k -S 8k \ > /dev/null & done } start_server start_client We can verify the CPU usage of the test cgroup using cpuacct.stat. The output shows: system: 53 user: 2 The CPU usage of the cgroup is relatively low at around 55%, but this usage doesn't increase, even with more netperf tasks. The reason is that CPU0 is at 100% utilization, as confirmed by mpstat: 02:56:22 PM CPU %usr %nice %sys %iowait %irq %soft %steal = %guest %gnice %idle 02:56:23 PM 0 0.99 0.00 55.45 0.00 0.99 42.57 0.00 = 0.00 0.00 0.00 02:56:23 PM CPU %usr %nice %sys %iowait %irq %soft %steal = %guest %gnice %idle 02:56:24 PM 0 2.00 0.00 55.00 0.00 0.00 43.00 0.00 = 0.00 0.00 0.00 This behavior is unexpected. We should account for IRQ time to the cgroup to reflect the pressure the group is under. After a thorough analysis, I discovered that this change in behavior is due to commit 305e6835e055 ("sched: Do not account irq time to current task"), which altered whether IRQ time should be charged to the interrupted task. While I agree that a task should not be penalized by random interrupts, the task itself cannot progress while interrupted. Therefore, the interrupted time should be reported to the user. The system metric in cpuacct.stat is crucial in indicating whether a container is under heavy system pressure, including IRQ/softirq activity. Hence, IRQ/softirq time should be accounted for in the cpuacct system usage, which also applies to cgroup2=E2=80=99s rstat. This patch reintroduces IRQ/softirq accounting to cgroups. Signed-off-by: Yafang Shao --- kernel/sched/cputime.c | 10 +++++++--- 1 file changed, 7 insertions(+), 3 deletions(-) diff --git a/kernel/sched/cputime.c b/kernel/sched/cputime.c index 0bed0fa1acd9..4586db5cf3a8 100644 --- a/kernel/sched/cputime.c +++ b/kernel/sched/cputime.c @@ -34,7 +34,8 @@ void disable_sched_clock_irqtime(void) sched_clock_irqtime =3D 0; } =20 -static void irqtime_account_delta(struct irqtime *irqtime, u64 delta, +static void irqtime_account_delta(struct task_struct *curr, + struct irqtime *irqtime, u64 delta, enum cpu_usage_stat idx) { u64 *cpustat =3D kcpustat_this_cpu->cpustat; @@ -44,6 +45,9 @@ static void irqtime_account_delta(struct irqtime *irqtime= , u64 delta, irqtime->total +=3D delta; irqtime->tick_delta +=3D delta; u64_stats_update_end(&irqtime->sync); + + cgroup_account_cputime(curr, delta); + cgroup_account_cputime_field(curr, idx, delta); } =20 /* @@ -72,9 +76,9 @@ void irqtime_account_irq(struct task_struct *curr, unsign= ed int offset) * that do not consume any time, but still wants to run. */ if (pc & HARDIRQ_MASK) - irqtime_account_delta(irqtime, delta, CPUTIME_IRQ); + irqtime_account_delta(curr, irqtime, delta, CPUTIME_IRQ); else if ((pc & SOFTIRQ_OFFSET) && curr !=3D this_cpu_ksoftirqd()) - irqtime_account_delta(irqtime, delta, CPUTIME_SOFTIRQ); + irqtime_account_delta(curr, irqtime, delta, CPUTIME_SOFTIRQ); } =20 static u64 irqtime_tick_accounted(u64 maxtime) --=20 2.43.5 From nobody Sun Feb 8 06:11:34 2026 Received: from mail-pg1-f174.google.com (mail-pg1-f174.google.com [209.85.215.174]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4BB9A1946C3 for ; Mon, 23 Sep 2024 09:01:23 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.215.174 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1727082084; cv=none; b=PPhdV62Fh7JLZpDZ5l5occNEG0GOgrBstpZ1I56PBuC51YPVYRvpDQFk35F/m1h6VWXCp0kfeXIeHr+FeVYIperIEiBNgpHuecbLAgqStlM7Cbf7Lbz/ZTNg8l6pil1jmoXGYQpgMGyJkPThWo5FWk6BEQpZF/1Mq6NEjX9jpZM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1727082084; c=relaxed/simple; bh=ebaEp4qgWBJ6q1S+WqYncxTKs1M2+e7QNkGq1Dioxp0=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=HbND2LX7wmk5SDCol4J8F8nQ/IcOkH7Xl5HcMuCyyax/q6O5v+fuc9b/zrcq6PQWJlJol17zqpVOPC9nz6Ac8YVpJLGO5Mcomi0AW/f/nMZXjCvt6gwqsYlWD2hxkXGSSvjx6x6ikqGd5Wtf94E+W+Pw7BtAIi6JuMEB2NVKNEE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=WHf/o0xG; arc=none smtp.client-ip=209.85.215.174 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="WHf/o0xG" Received: by mail-pg1-f174.google.com with SMTP id 41be03b00d2f7-7b0c9bbddb4so2743018a12.3 for ; Mon, 23 Sep 2024 02:01:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1727082082; x=1727686882; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=23Ofto8ZNkiE6V91gkyCDbtFPh5A2fyseeuvv0xtnBE=; b=WHf/o0xGO1j4BIdN51WudlLG7CfJC37mpiZWS1s+Solfxn8dvNfHBj1Y61gHW8jB7u 5+BvGepL7AGKlLFmzcOAxeNVO8xfQUJ/nCgatCxQS9yxcd/SrqUZXRgp2dBRRwW4X3A5 zdUjCTmApgUcXypf+cKpo/dNdzeoDP0Tlpa5heVEcKrskTlQp3mRZJZZ2uZgs9f2oV/v i4ScYgzf6BjF74KDoR4aNhsxu68F2riwdGKaTbJQz7UUw/DdZlLpXuOL6vOmFS1rqeK9 rxjK8IO5tvREKhBt0QXu0XInz9anz+qqcKPwgDO9ok7mVEuB2b+QeuLdC5TLmi7XFkmZ ABbw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1727082082; x=1727686882; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=23Ofto8ZNkiE6V91gkyCDbtFPh5A2fyseeuvv0xtnBE=; b=vX5D5RwsLfLb7mlbNxCQVcN3outBgR3laCY3jKkORtpNSz3b7V5wl8tmGVNun7lAed lAxqb2qPWnUgzWy7EGXR4Xfd9jeG3aXIUxfVezTe5nfjyCad9lWeNHj1WAJTGeaeIVeB BFKQ92gxL2g/olSaPEiUb3GLmmt/8ScU5La78o141dCPQyxJNmVzQV+yxjC7bzjw1cci CcBITQd/bl07ujV13mJwTQE+QKraMmWfHUHon+lTuhuLcLOsq/Cgh8DDA3MpCJCTCXmw ZTEe29BN5tbtTVGCPs+KR6TP9tdFO07cTlwg1hbXypnNQMe2gdMIOSwGNMGjwjMdXvtl WC5A== X-Gm-Message-State: AOJu0Ywkb1pGEKFYrcc68b9qJ/Dhxs234xlkzVFibHTHmCxrXOUPAuxu OlcxPKqzYiOSv50aUVJIUtuuEUDfaIVljG2tUG+yt8/Zq4M/aQBL X-Google-Smtp-Source: AGHT+IEfwqDx4aMnlQUqypf9YkQ/UQsMIguXiKc+NhiWwMRpgYtI2wvqkQGPEY5D7UlBq9LPG2NtNg== X-Received: by 2002:a17:90b:17c6:b0:2d8:e524:797b with SMTP id 98e67ed59e1d1-2dd7f416072mr13601148a91.18.1727082082589; Mon, 23 Sep 2024 02:01:22 -0700 (PDT) Received: from localhost.localdomain ([39.144.43.152]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-2dd7f8ca487sm6768872a91.39.2024.09.23.02.01.17 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Mon, 23 Sep 2024 02:01:22 -0700 (PDT) From: Yafang Shao To: mingo@redhat.com, peterz@infradead.org, juri.lelli@redhat.com, vincent.guittot@linaro.org, dietmar.eggemann@arm.com, rostedt@goodmis.org, bsegall@google.com, mgorman@suse.de, vschneid@redhat.com Cc: linux-kernel@vger.kernel.org, Yafang Shao Subject: [PATCH 2/3] sched: Define sched_clock_irqtime as static key Date: Mon, 23 Sep 2024 17:00:27 +0800 Message-Id: <20240923090028.16368-3-laoar.shao@gmail.com> X-Mailer: git-send-email 2.30.1 (Apple Git-130) In-Reply-To: <20240923090028.16368-1-laoar.shao@gmail.com> References: <20240923090028.16368-1-laoar.shao@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Since CPU time accounting is a performance-critical path, let's define sched_clock_irqtime as a static key to minimize potential overhead. Signed-off-by: Yafang Shao --- kernel/sched/cputime.c | 16 +++++++--------- kernel/sched/sched.h | 1 + 2 files changed, 8 insertions(+), 9 deletions(-) diff --git a/kernel/sched/cputime.c b/kernel/sched/cputime.c index 4586db5cf3a8..de9a36e0fe04 100644 --- a/kernel/sched/cputime.c +++ b/kernel/sched/cputime.c @@ -7,6 +7,8 @@ #include #endif =20 +DEFINE_STATIC_KEY_FALSE(sched_clock_irqtime); + #ifdef CONFIG_IRQ_TIME_ACCOUNTING =20 /* @@ -22,16 +24,14 @@ */ DEFINE_PER_CPU(struct irqtime, cpu_irqtime); =20 -static int sched_clock_irqtime; - void enable_sched_clock_irqtime(void) { - sched_clock_irqtime =3D 1; + static_branch_enable(&sched_clock_irqtime); } =20 void disable_sched_clock_irqtime(void) { - sched_clock_irqtime =3D 0; + static_branch_disable(&sched_clock_irqtime); } =20 static void irqtime_account_delta(struct task_struct *curr, @@ -61,7 +61,7 @@ void irqtime_account_irq(struct task_struct *curr, unsign= ed int offset) s64 delta; int cpu; =20 - if (!sched_clock_irqtime) + if (!static_branch_likely(&sched_clock_irqtime)) return; =20 cpu =3D smp_processor_id(); @@ -94,8 +94,6 @@ static u64 irqtime_tick_accounted(u64 maxtime) =20 #else /* CONFIG_IRQ_TIME_ACCOUNTING */ =20 -#define sched_clock_irqtime (0) - static u64 irqtime_tick_accounted(u64 dummy) { return 0; @@ -482,7 +480,7 @@ void account_process_tick(struct task_struct *p, int us= er_tick) if (vtime_accounting_enabled_this_cpu()) return; =20 - if (sched_clock_irqtime) { + if (static_branch_likely(&sched_clock_irqtime)) { irqtime_account_process_tick(p, user_tick, 1); return; } @@ -511,7 +509,7 @@ void account_idle_ticks(unsigned long ticks) { u64 cputime, steal; =20 - if (sched_clock_irqtime) { + if (static_branch_likely(&sched_clock_irqtime)) { irqtime_account_idle_ticks(ticks); return; } diff --git a/kernel/sched/sched.h b/kernel/sched/sched.h index 8063db62b027..db7d541eebff 100644 --- a/kernel/sched/sched.h +++ b/kernel/sched/sched.h @@ -3174,6 +3174,7 @@ struct irqtime { }; =20 DECLARE_PER_CPU(struct irqtime, cpu_irqtime); +DECLARE_STATIC_KEY_FALSE(sched_clock_irqtime); =20 /* * Returns the irqtime minus the softirq time computed by ksoftirqd. --=20 2.43.5 From nobody Sun Feb 8 06:11:34 2026 Received: from mail-pg1-f170.google.com (mail-pg1-f170.google.com [209.85.215.170]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 246D61946C9 for ; Mon, 23 Sep 2024 09:01:27 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.215.170 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1727082089; cv=none; b=nVrxQJlzMiCFbX3IGsI3CAckPyZsyPFOBnj1iIP2fjlBargdhf3KA9oULHLMhOxxcuMn37IcgUgnd9VtMMxgFYlsox58sR9LuhzuxhlGHv30coQn85eL7u9iGj6M0iBb/RIpRA2sToh7cRFSpx23qVPQFKTNg7CcChg6lDZcy0M= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1727082089; c=relaxed/simple; bh=Eb8d5lyXFhSllOGcwU1VmnjYWiuaJB9lPUZZG8MTkDE=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=R2hBlQ4ymAwN+TZSYJgdxifBNeRkbAwViOt0d4RINrGTl+21NDXipvHPwW9s6PabCfg8MYqJGHX2RO7HHyHi0CRSemhR6XojSbb3dlvw1vjYfzuBvWRAadrYl1I/OP5uypQM7MxKPCBIuUD04GZoyzHBA8sHMGn5ffQIo2LVAVI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=NRfN3xlX; arc=none smtp.client-ip=209.85.215.170 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="NRfN3xlX" Received: by mail-pg1-f170.google.com with SMTP id 41be03b00d2f7-7d4fbe62bf5so2299560a12.0 for ; Mon, 23 Sep 2024 02:01:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1727082087; x=1727686887; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=DCA0RTe4B5CkvnInMX18gUuR5vgyuSITkXk+PtSiiZc=; b=NRfN3xlXWgJ2p40+MEXhoD1RDL/5yl9LTiIEjfRWl+TEEmbmA6I+JyiupOKNuynF6G lJSmnaOsGFC+4PEYjuO/kCWL0PcZjIrzYKOxMTtZ9nfhBE6mvlmiA9nG30ImxrPpaLOB 0jj+u+nz8P0uClj0X1ULviuWfuZKyfsSDaq4J1j+r+o4Q62QrjBc5aETQH0aGocdwIOH b/iFiD7e2O2UAtNQQ5OTNnvM4wDIzOFZjrKu2p70ioCAWnTXkDBImvBpaUvJHCX2XYu4 7PdkH5eBQOcTAYIHYXCu00WVCIlTOmgZ4ELrUbExwFLTP4dnkV3isuTcMSR3nFeq6sLg zLSg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1727082087; x=1727686887; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=DCA0RTe4B5CkvnInMX18gUuR5vgyuSITkXk+PtSiiZc=; b=D9dFmPZQbSXw4sabawmBBaKH9U9pYlTGKqJDUJ0s5q4hPQRI4Wdt3F/Na5sEYwpsJu 7nr2zqNGsts71penE3m41YmvlxbNp2aaPNtoS+/qosaJAEMwNLyVUbxOhipoISYyr19h fjLaxIxO5C5+Pfdz/ZqQ6FdLxc6IIY1QpJgQmnHeblU4AIFe0MxHf7hLzrcsr6H4eJMT 1mVt3jLPXX/e1unjFl4WaCweabPETMFcwjgwyUW2PqLuksl+t0hhq2lCCDnzvQKGWBlO GOTuQqkBNgPvGa6QXZL57BGm938QFzCDD6PzD2mQCXS94wZt6UBs38QdJf1Z9nQOFIUq aJSQ== X-Gm-Message-State: AOJu0Yy5RXzlQq/EKwh2SDfAq6P/WCRSf2F/WOTpGbR6k6C7mCDsiKrc xL0A6eZQtDGPPTzZwCu8UOKG865Mr5JUK3oxqj0Za9ZWg1fVknBz X-Google-Smtp-Source: AGHT+IFQiaVsuB7T4CDhBig9cPff+916x+W54KDCddVoWqeOCfSucKRgf/rCV1csMNGhz6oNpsrebg== X-Received: by 2002:a17:90b:485:b0:2cf:c9ab:e747 with SMTP id 98e67ed59e1d1-2dd7f36cb4cmr14325238a91.1.1727082087430; Mon, 23 Sep 2024 02:01:27 -0700 (PDT) Received: from localhost.localdomain ([39.144.43.152]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-2dd7f8ca487sm6768872a91.39.2024.09.23.02.01.22 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Mon, 23 Sep 2024 02:01:26 -0700 (PDT) From: Yafang Shao To: mingo@redhat.com, peterz@infradead.org, juri.lelli@redhat.com, vincent.guittot@linaro.org, dietmar.eggemann@arm.com, rostedt@goodmis.org, bsegall@google.com, mgorman@suse.de, vschneid@redhat.com Cc: linux-kernel@vger.kernel.org, Yafang Shao Subject: [PATCH 3/3] sched: Don't account irq time if sched_clock_irqtime is disabled Date: Mon, 23 Sep 2024 17:00:28 +0800 Message-Id: <20240923090028.16368-4-laoar.shao@gmail.com> X-Mailer: git-send-email 2.30.1 (Apple Git-130) In-Reply-To: <20240923090028.16368-1-laoar.shao@gmail.com> References: <20240923090028.16368-1-laoar.shao@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" sched_clock_irqtime may be disabled due to the clock source, in which case IRQ time should not be accounted. Let's add a conditional check to avoid unnecessary logic. Signed-off-by: Yafang Shao --- kernel/sched/core.c | 44 +++++++++++++++++++++++--------------------- 1 file changed, 23 insertions(+), 21 deletions(-) diff --git a/kernel/sched/core.c b/kernel/sched/core.c index b6cc1cf499d6..8b633a14a60f 100644 --- a/kernel/sched/core.c +++ b/kernel/sched/core.c @@ -735,29 +735,31 @@ static void update_rq_clock_task(struct rq *rq, s64 d= elta) s64 __maybe_unused steal =3D 0, irq_delta =3D 0; =20 #ifdef CONFIG_IRQ_TIME_ACCOUNTING - irq_delta =3D irq_time_read(cpu_of(rq)) - rq->prev_irq_time; + if (static_branch_likely(&sched_clock_irqtime)) { + irq_delta =3D irq_time_read(cpu_of(rq)) - rq->prev_irq_time; =20 - /* - * Since irq_time is only updated on {soft,}irq_exit, we might run into - * this case when a previous update_rq_clock() happened inside a - * {soft,}IRQ region. - * - * When this happens, we stop ->clock_task and only update the - * prev_irq_time stamp to account for the part that fit, so that a next - * update will consume the rest. This ensures ->clock_task is - * monotonic. - * - * It does however cause some slight miss-attribution of {soft,}IRQ - * time, a more accurate solution would be to update the irq_time using - * the current rq->clock timestamp, except that would require using - * atomic ops. - */ - if (irq_delta > delta) - irq_delta =3D delta; + /* + * Since irq_time is only updated on {soft,}irq_exit, we might run into + * this case when a previous update_rq_clock() happened inside a + * {soft,}IRQ region. + * + * When this happens, we stop ->clock_task and only update the + * prev_irq_time stamp to account for the part that fit, so that a next + * update will consume the rest. This ensures ->clock_task is + * monotonic. + * + * It does however cause some slight miss-attribution of {soft,}IRQ + * time, a more accurate solution would be to update the irq_time using + * the current rq->clock timestamp, except that would require using + * atomic ops. + */ + if (irq_delta > delta) + irq_delta =3D delta; =20 - rq->prev_irq_time +=3D irq_delta; - delta -=3D irq_delta; - delayacct_irq(rq->curr, irq_delta); + rq->prev_irq_time +=3D irq_delta; + delta -=3D irq_delta; + delayacct_irq(rq->curr, irq_delta); + } #endif #ifdef CONFIG_PARAVIRT_TIME_ACCOUNTING if (static_key_false((¶virt_steal_rq_enabled))) { --=20 2.43.5