From nobody Tue Dec 16 04:35:41 2025 Received: from mail-pj1-f44.google.com (mail-pj1-f44.google.com [209.85.216.44]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C37714C85; Tue, 22 Apr 2025 13:47:46 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.44 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745329669; cv=none; b=WpxPQpOqMXQ/6dAx4ZHNgOqId2wTuyn5mqHIT4Dpxc+qN5KnhcnenQbOqWBpIEKHxdFrc2DDst0vm9qBZaNQJXAX/Zgmwurv6SnQ/bcjTdLOW64vj2FGV/DcMc2D1th6xDcrU3/raVWZEGdI9btUoq5++krPib3EaCp1+FmXacM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745329669; c=relaxed/simple; bh=rihYLSUzxeuZAJ3jukl1xScoPoA5Pkb6Z1BYdQxPJX4=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=mDuq4fNcNh4mSIBUkkK6jhMMV/D0rZ1Lv0JlfzdlmIJh2e85VHZQXL78OvwPoIx7ObySTlAww9duoVk2fdFSWSwi/WvMvLlj9b3r6rlL6nTcvGltSSzRJ0E507w8+PPMdi48ko2D0/SYXtlkIL47baA5QpEfPkFBDmowJ52nhO0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=QVoz7Avm; arc=none smtp.client-ip=209.85.216.44 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="QVoz7Avm" Received: by mail-pj1-f44.google.com with SMTP id 98e67ed59e1d1-2ff67f44fcaso1147850a91.3; Tue, 22 Apr 2025 06:47:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1745329666; x=1745934466; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=uSIs1Ai8/oR/BRxFfkqnndhf7X1ZjCjQIqFS36x91Ck=; b=QVoz7AvmyrrK3DAzZLjsGbDuuIoCYqfErTWncLRiYaFk0QzNt12GPhS0QFFwT/n66s /kGMTQM/1WXmPd64pZiFyDVLzm5qGf3bnylB/YQ1Pyf8DgE38XLge9j7Hd1wlqoO1+QB y9VINpf43ctNUyp+P9N63ASc1e3v/VZb/5ojHR3X59at4UK2P110o1VK7hxhaqwltEyk KumRB99BGp7qdEgcRE7cptqqEvlL71BE6ilBlzng1wSkeemznLGijxwnnQddcKPP/ZIu B0pZBTg9ycLJE60Lnf1Tjf66Z5HOQpXEUEmlZe1zTCcdb7zWN5RnnIRhLmUM6fTe2ZJE I7+A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1745329666; x=1745934466; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=uSIs1Ai8/oR/BRxFfkqnndhf7X1ZjCjQIqFS36x91Ck=; b=cnJeLUVtDvscEAn7URgwW09V55dSnuJ8+cHRX33kQJcnWNOkGQbOKzck96Q52BZxOi gNZtU6bvdtcMmGGYqLNeCF5rMPBngaMHpKSLT+bihDSSflCj4LAaUU2qFWRln4LdfItf iOnepBijyGlXR+C83j80zJTnvvHyiSE7t1Z2iKG/07sXeTBEzF600Tmh2VVWtJ5ZXwR0 ZpJvoylnUXzR1Sx7m3x6CK6kr2Sw0dcugwZr0opaPgHRxnH8nl68V7W0fhGj4C8SNkj6 mAZgJ5pLe8Nyzocnz2VDehQ6+AGoY4uv6Zk7ErFlSOYONj4m5+0RydI7n+kpYcF97A6N 01aA== X-Forwarded-Encrypted: i=1; AJvYcCUsPWpCyC5M/UUYhefh1YYiydRssL5vTZ6vDsnbmZ5Otnso63WqQg3B4a1I4sa8YEecT4c9o8Syr1C8R8Q=@vger.kernel.org X-Gm-Message-State: AOJu0YyR5e3Fk6Iz+tH1+6gq0n69pT7nzwfFw1qwOnFxFuk5gWfRfDEj mhI97/C3w6ETTOMW8opcR4KntdhvGGZHEAHaLWJhTZu4bsbMpTW2Ob1oZbu7kdo= X-Gm-Gg: ASbGncsu9nLd0G1qDqFE4EAQITJfZKFfkG4X87Yxt4FoqZ1EX92JNGDDxkxglTZD7D1 eE20SjVT6J5LuyKVn3de9a8PeOPKz+BD7LnoKEEpK9EUYOESQ/aEpzdjArRmAJSaVHRNfjQEwxf dg+OaMnFg0eNa109cB291sBXqtBNaab1eRMvFta4lwhe6tI9iJJ9Ec5qHvw+h7dlE4lEwlbjpka rXqrb20A0Dx+VktLQaiUDsmUsSkSl+V3MqnTp3nxLcY5wDsTpDWRLcWG+e4LS8MzocxJ03PP+2J TuVNIICKywVGO5OdIsFR2r+TbDQiDIT3NR7PihAOX+L5EmkbU6D4lruib0sIB0M= X-Google-Smtp-Source: AGHT+IEIXnLU/LiQqCvB+iD2Kr7qAeNMDx8ndPjJodX/aRk8gAMlbo/Ec7dFUU+hiw23QjCIpta0cQ== X-Received: by 2002:a17:90b:3e85:b0:301:1c11:aa7a with SMTP id 98e67ed59e1d1-3087bba8573mr8635622a91.3.1745329665853; Tue, 22 Apr 2025 06:47:45 -0700 (PDT) Received: from MGG23TF6W0.corp.ebay.com ([202.76.247.146]) by smtp.googlemail.com with ESMTPSA id 98e67ed59e1d1-3087e05b2bbsm8695214a91.42.2025.04.22.06.47.39 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Tue, 22 Apr 2025 06:47:45 -0700 (PDT) From: Jianlin Lv X-Google-Original-From: Jianlin Lv To: bpf@vger.kernel.org Cc: ast@kernel.org, daniel@iogearbox.net, andrii@kernel.org, martin.lau@linux.dev, eddyz87@gmail.com, song@kernel.org, yonghong.song@linux.dev, john.fastabend@gmail.com, kpsingh@kernel.org, sdf@fomichev.me, haoluo@google.com, jolsa@kernel.org, mingo@redhat.com, peterz@infradead.org, juri.lelli@redhat.com, vincent.guittot@linaro.org, dietmar.eggemann@arm.com, rostedt@goodmis.org, bsegall@google.com, mgorman@suse.de, vschneid@redhat.com, linux-kernel@vger.kernel.org, iecedge@gmail.com, jianlv@ebay.com Subject: [RFC PATCH bpf-next 1/2] Enhance BPF execution timing by excluding IRQ time Date: Tue, 22 Apr 2025 21:47:26 +0800 Message-Id: <73fdbbf9aafd3e24e12bb58f89c70959fb3a37f1.1745250534.git.iecedge@gmail.com> X-Mailer: git-send-email 2.39.3 (Apple Git-146) In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Jianlin Lv This commit excludes IRQ time from the total execution duration of BPF programs. When CONFIG_IRQ_TIME_ACCOUNTING is enabled, IRQ time is accounted for separately, offering a more accurate assessment of CPU usage for BPF programs. Signed-off-by: Jianlin Lv --- include/linux/filter.h | 24 ++++++++++++++++++++++-- 1 file changed, 22 insertions(+), 2 deletions(-) diff --git a/include/linux/filter.h b/include/linux/filter.h index f5cf4d35d83e..3e0f975176a6 100644 --- a/include/linux/filter.h +++ b/include/linux/filter.h @@ -703,12 +703,32 @@ static __always_inline u32 __bpf_prog_run(const struc= t bpf_prog *prog, cant_migrate(); if (static_branch_unlikely(&bpf_stats_enabled_key)) { struct bpf_prog_stats *stats; - u64 duration, start =3D sched_clock(); + u64 duration, start, start_time, end_time, irq_delta; unsigned long flags; + unsigned int cpu; =20 - ret =3D dfunc(ctx, prog->insnsi, prog->bpf_func); + #ifdef CONFIG_IRQ_TIME_ACCOUNTING + if (in_task()) { + cpu =3D get_cpu(); + put_cpu(); + start_time =3D irq_time_read(cpu); + } + #endif =20 + start =3D sched_clock(); + ret =3D dfunc(ctx, prog->insnsi, prog->bpf_func); duration =3D sched_clock() - start; + + #ifdef CONFIG_IRQ_TIME_ACCOUNTING + if (in_task()) { + end_time =3D irq_time_read(cpu); + if (end_time > start_time) { + irq_delta =3D end_time - start_time; + duration -=3D irq_delta; + } + } + #endif + stats =3D this_cpu_ptr(prog->stats); flags =3D u64_stats_update_begin_irqsave(&stats->syncp); u64_stats_inc(&stats->cnt); --=20 2.34.1 From nobody Tue Dec 16 04:35:41 2025 Received: from mail-pg1-f169.google.com (mail-pg1-f169.google.com [209.85.215.169]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D61544C85; Tue, 22 Apr 2025 13:47:53 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.215.169 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745329675; cv=none; b=eCwUD+HkIFFqL7SX94GDxAigNTlSDIioVqmIUYRI7sdOAt+VbINQ78hmdA6p6aLy6+6GXCaYsuR/fZAeLPgOM1KZAt9pnZHCLcu3hmRiyopJ3Gix0rm4vGdV/rHhdCmu7OwnVr8c5MKf/mTX8TjzCrwNdCfxgHA6nE3ciWFr8fQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745329675; c=relaxed/simple; bh=FoNPp8gvjLy3QZyMyd34WJbEIvriF/TJs1PaU5tXNLI=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=Vyg08Y5czSNXF4ku6npTXb5bQJZXiZ71xDW/sE3xiandARRDH5MsqKJVjFVsBgfO149BShxhXxB+MglaOWyY/suK5cDmj85g5Vw2ErNiS9XPk/UpBXRV4xX+V3IMn5BbubzIyvv9whnvtOkWhwWhC/Q2s++Rpd1e+GTM/oNekB0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=ajfS0Fan; arc=none smtp.client-ip=209.85.215.169 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="ajfS0Fan" Received: by mail-pg1-f169.google.com with SMTP id 41be03b00d2f7-af9b16eca8eso318205a12.1; Tue, 22 Apr 2025 06:47:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1745329673; x=1745934473; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=0+ByOwbXTp0uyTPHkOtuOVbDZK9B4SVuPq+zJXZT0mE=; b=ajfS0FanXhaoNTgxopjlCzPB+D1KyfWBYmtmBYMO7BYMpuPITN602caBjzzb0aO2cc ixG84DT+SK+2sexjOHz2GAzcdRGUAqwmXq+smx+Oc/nLhd84hHr+f57zKA8ywoQ6EGWm ThpyV7LCdaaXx6ByZnXkIsWIG9HeRNRUavYS9ZYvFCPj9NXX+PXqMDKq33ZXBieFDzcB 0Od/hlWT4WZIEgpw4c5kvWSS/S/r0HkOxK9U07bZBSbLnxbE4Ero75GOiQHbjCaMNJbO wxAySc6MAEKZwLkqlKmpbE/f9wHKMA5UMIJjPL47RtTEpiNbqonD+raLUoqFSFjrPigB mvFQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1745329673; x=1745934473; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=0+ByOwbXTp0uyTPHkOtuOVbDZK9B4SVuPq+zJXZT0mE=; b=pxS4+qpiYjzBXvK87jJploRibdA+ulLrkDhRoSaKi4ftkrQAO4AWqOWFK2junwlSxF quH0AOYlVGcfcDtckC5d/y9djRuRs3OBGFMXn2S55XIY8IIStWRbuO8fRzfQKOZq5d7x qO8EHh1ttbzF70Q81sWmVKTmuqo0uburgAyke+ljeqMBTpVfKPGBq+fQmtPymS9i74IZ ViTo0EAJhDgD8SPbMK7LddO2kRklJEYtEEKd/K5FJSVaW7mXU5LbxIXYyudpHVlj4nUX iP7s6RnLboZRiiTWKwBiOqX9y2sIPcz2GpSrIDqQeUgjJjfVAatHv98DTzuFS1vIolrc A8UQ== X-Forwarded-Encrypted: i=1; AJvYcCXYQFXWH4rBCZV2R2+43pDqCDm3asAK+5qCO1kU0BCm6kj2jm9Aa+xy9xKVSH2a2zNHP0bAVzdREn9SqBA=@vger.kernel.org X-Gm-Message-State: AOJu0YxrpsY8PQvFQQyn+YUU4kE6I7ZCH8d9pLtahZyw2TLeV1vzOS7v xhNyQyqZqaklIseQcQc5jHNAPiaNMLbVnruAaOs2+IUWJIhsYFYyFnFQImdG+g4= X-Gm-Gg: ASbGnctCID57KAeAd+CS81ZsTLXR050ZbloYVjAiVap4Hyw7QskbgePRfJXHQKF7hR9 oS8+qYd3o1f1g6dfzgSZ35oEunJNLgdCzCOv93yaLcs8X7syWc6299Z/cbm8mwr1i6OGuwURjdj tUUpepELLdSzu3P52/FtdoHLsiuacHJtcj/owAbrqsOQEFi5UXSlcHeJmjT3YF12DvarFds6UMe A7oZkbCXjDx7f0XNUYOw+XVuk30fL1Zbj5bx2FX/DiOfjmLXRmtT12d4ehSMZyqQjcxGNZPWPf8 I7kVQ39l8vymgdhagR9Se65aOSrbnOwkjITuuoFXOV9tDqhr8dtkk7Fs+QHTpIE= X-Google-Smtp-Source: AGHT+IFqu8ojOTnhEF4HKyFdc0yN4q+/NQI7lPzrfCLv+6d557zDM0nA6BiAAW+1s5Fbihn5L1GVhw== X-Received: by 2002:a17:90b:1c91:b0:2ee:f59a:94d3 with SMTP id 98e67ed59e1d1-3087ba5ccbamr8020134a91.0.1745329672828; Tue, 22 Apr 2025 06:47:52 -0700 (PDT) Received: from MGG23TF6W0.corp.ebay.com ([202.76.247.146]) by smtp.googlemail.com with ESMTPSA id 98e67ed59e1d1-3087e05b2bbsm8695214a91.42.2025.04.22.06.47.46 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Tue, 22 Apr 2025 06:47:52 -0700 (PDT) From: Jianlin Lv X-Google-Original-From: Jianlin Lv To: bpf@vger.kernel.org Cc: ast@kernel.org, daniel@iogearbox.net, andrii@kernel.org, martin.lau@linux.dev, eddyz87@gmail.com, song@kernel.org, yonghong.song@linux.dev, john.fastabend@gmail.com, kpsingh@kernel.org, sdf@fomichev.me, haoluo@google.com, jolsa@kernel.org, mingo@redhat.com, peterz@infradead.org, juri.lelli@redhat.com, vincent.guittot@linaro.org, dietmar.eggemann@arm.com, rostedt@goodmis.org, bsegall@google.com, mgorman@suse.de, vschneid@redhat.com, linux-kernel@vger.kernel.org, iecedge@gmail.com, jianlv@ebay.com Subject: [RFC PATCH bpf-next 2/2] Export irq_time_read for BPF module usage Date: Tue, 22 Apr 2025 21:47:27 +0800 Message-Id: <75aef5f2b9d9292ae919f2af9f82a8618f9b191e.1745250534.git.iecedge@gmail.com> X-Mailer: git-send-email 2.39.3 (Apple Git-146) In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Jianlin Lv Move irq_time_read function to kernel/sched/core.c and export for external use when CONFIG_IRQ_TIME_ACCOUNTING is enabled. Signed-off-by: Jianlin Lv --- include/linux/sched.h | 4 ++++ kernel/sched/core.c | 22 ++++++++++++++++++++++ kernel/sched/sched.h | 19 ------------------- 3 files changed, 26 insertions(+), 19 deletions(-) diff --git a/include/linux/sched.h b/include/linux/sched.h index f96ac1982893..3b83ac99b533 100644 --- a/include/linux/sched.h +++ b/include/linux/sched.h @@ -2281,4 +2281,8 @@ static __always_inline void alloc_tag_restore(struct = alloc_tag *tag, struct allo #define alloc_tag_restore(_tag, _old) do {} while (0) #endif =20 +#ifdef CONFIG_IRQ_TIME_ACCOUNTING +extern inline u64 irq_time_read(int cpu); +#endif + #endif diff --git a/kernel/sched/core.c b/kernel/sched/core.c index cfaca3040b2f..c840d1ffdaca 100644 --- a/kernel/sched/core.c +++ b/kernel/sched/core.c @@ -10747,3 +10747,25 @@ void sched_enq_and_set_task(struct sched_enq_and_s= et_ctx *ctx) set_next_task(rq, ctx->p); } #endif /* CONFIG_SCHED_CLASS_EXT */ + +#ifdef CONFIG_IRQ_TIME_ACCOUNTING +/* + * Returns the irqtime minus the softirq time computed by ksoftirqd. + * Otherwise ksoftirqd's sum_exec_runtime is subtracted its own runtime + * and never move forward. + */ +inline u64 irq_time_read(int cpu) +{ + struct irqtime *irqtime =3D &per_cpu(cpu_irqtime, cpu); + unsigned int seq; + u64 total; + + do { + seq =3D __u64_stats_fetch_begin(&irqtime->sync); + total =3D irqtime->total; + } while (__u64_stats_fetch_retry(&irqtime->sync, seq)); + + return total; +} +EXPORT_SYMBOL(irq_time_read); +#endif /* CONFIG_IRQ_TIME_ACCOUNTING */ diff --git a/kernel/sched/sched.h b/kernel/sched/sched.h index 47972f34ea70..d2fd3772114e 100644 --- a/kernel/sched/sched.h +++ b/kernel/sched/sched.h @@ -3209,25 +3209,6 @@ static inline int irqtime_enabled(void) return sched_clock_irqtime; } =20 -/* - * Returns the irqtime minus the softirq time computed by ksoftirqd. - * Otherwise ksoftirqd's sum_exec_runtime is subtracted its own runtime - * and never move forward. - */ -static inline u64 irq_time_read(int cpu) -{ - struct irqtime *irqtime =3D &per_cpu(cpu_irqtime, cpu); - unsigned int seq; - u64 total; - - do { - seq =3D __u64_stats_fetch_begin(&irqtime->sync); - total =3D irqtime->total; - } while (__u64_stats_fetch_retry(&irqtime->sync, seq)); - - return total; -} - #else =20 static inline int irqtime_enabled(void) --=20 2.34.1