From nobody Sun Feb 8 22:08:31 2026 Received: from fanzine2.igalia.com (fanzine.igalia.com [178.60.130.6]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D3B2D145B3F for ; Fri, 31 Jan 2025 07:09:58 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=178.60.130.6 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1738307401; cv=none; b=gW78/vrCb42m6gvhg0+2UFNcXGXxymHptlBkzVizJYybFP6mYRtjNbqdyARD+M/kY9kboKITIdiKqkbccsaQzhENLDUBzfjxknG/0UNReKvzh2mGF0Cju4qpIyoYgY/0IGa7SB4oIodikSCgN2WVy/RjHAP+pCmqH3oO+89SqZ0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1738307401; c=relaxed/simple; bh=o1Yo6Yo7fZFV6OCfNCbKfJM/kbpQzIj3FE0+3TQE2g4=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=qRWPUQRGA6M+pKQx2drfpQXt/VG1VAYGWCu7kDdpWkBoxhBLFw8Ko1mcHBc+xk9Uv+e/6EPi2Q0ii7vcA+FDJExmsi0TXNzIQ5SA5eY49foxU6R+KHR6Z+9EeSRDAsCtm+C8rxPRyBYAuYgmVWdYtt7kp8+/ZzkfpECXYmas648= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=igalia.com; spf=pass smtp.mailfrom=igalia.com; dkim=pass (2048-bit key) header.d=igalia.com header.i=@igalia.com header.b=A7HQXr1S; arc=none smtp.client-ip=178.60.130.6 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=igalia.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=igalia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=igalia.com header.i=@igalia.com header.b="A7HQXr1S" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=igalia.com; s=20170329; h=Content-Transfer-Encoding:MIME-Version:References:In-Reply-To: Message-ID:Date:Subject:Cc:To:From:Sender:Reply-To:Content-Type:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Id:List-Help:List-Unsubscribe:List-Subscribe: List-Post:List-Owner:List-Archive; bh=ew2+CCsZd4F3S7dVu3kPbF1zR6N1+Mxu2aPCZoPthO4=; b=A7HQXr1Sq8Ygjh1wHZamDiLF64 kuq7yDzZyTNe9Yyc8UoCUFyHofU+ZSFa9tHbuRpv7WIEYYQNtK2cxiCZdenZTV0cQgF5/SNykaS6t j8kNdCAdFuhjo8xi5oY1DleKIW3JS1+frMpC5iD5j1KOmFvUVR/1VBTh6NHDyZmln9DZPg7aGBF6s mjU21tGvFRkzjGi3KG1tjng5isC8w7i2lS7s0HdntxC/DkmO+3Y2AzYMm5GjbSEh3S28lo9Y7XJs9 jl+Gclk43VQJ1YCIiRm255RUpARLrR9uXmmsGfCnSYSFUOJRZsyfw05aIZd6Ha3cEf6/BitdSTPIV HwwaqnBg==; Received: from [58.29.143.236] (helo=localhost) by fanzine2.igalia.com with utf8esmtpsa (Cipher TLS1.3:ECDHE_SECP256R1__RSA_PSS_RSAE_SHA256__AES_256_GCM:256) (Exim) id 1tdl9s-001KY4-Ru; Fri, 31 Jan 2025 08:09:51 +0100 From: Changwoo Min To: tj@kernel.org, void@manifault.com, arighi@nvidia.com Cc: kernel-dev@igalia.com, linux-kernel@vger.kernel.org, Changwoo Min Subject: [PATCH v3 01/11] sched_ext: Implement event counter infrastructure Date: Fri, 31 Jan 2025 16:09:28 +0900 Message-ID: <20250131070938.95551-2-changwoo@igalia.com> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20250131070938.95551-1-changwoo@igalia.com> References: <20250131070938.95551-1-changwoo@igalia.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Collect the statistics of specific types of behavior in the sched_ext core, which are not easily visible but still interesting to an scx scheduler. An event type is defined in 'struct scx_event_stats.' When an event occurs, its counter is accumulated using 'scx_add_event()' and '__scx_add_event()' to per-CPU 'struct scx_event_stats' for efficiency. 'scx_bpf_events()' aggregates all the per-CPU counters and exposes a system-wide counters. For convenience and readability of the code, 'scx_agg_event()' and 'scx_dump_event()' are provided. The collected events can be observed after a BPF scheduler is unloaded beforea new BPF scheduler is loaded so the per-CPU 'struct scx_event_stats' are reset. Signed-off-by: Changwoo Min --- kernel/sched/ext.c | 103 +++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 103 insertions(+) diff --git a/kernel/sched/ext.c b/kernel/sched/ext.c index 5f6a425d4ffe..4e28e88e88d4 100644 --- a/kernel/sched/ext.c +++ b/kernel/sched/ext.c @@ -1440,6 +1440,64 @@ static struct task_struct *scx_task_iter_next_locked= (struct scx_task_iter *iter) return p; } =20 +/* + * Collection of event counters. Event types are placed in descending orde= r. + */ +struct scx_event_stats { +}; + +/* + * The event counter is organized by a per-CPU variable to minimize the + * accounting overhead without synchronization. A system-wide view on the + * event counter is constructed when requested by scx_bpf_get_event_stat(). + */ +static DEFINE_PER_CPU(struct scx_event_stats, event_stats_cpu); + +/** + * scx_add_event - Increase an event counter for 'name' by 'cnt' + * @name: an event name defined in struct scx_event_stats + * @cnt: the number of the event occured + * + * This can be used when preemption is not disabled. + */ +#define scx_add_event(name, cnt) do { \ + this_cpu_add(event_stats_cpu.name, cnt); \ +} while(0) + +/** + * __scx_add_event - Increase an event counter for 'name' by 'cnt' + * @name: an event name defined in struct scx_event_stats + * @cnt: the number of the event occured + * + * This should be used only when preemption is disabled. + */ +#define __scx_add_event(name, cnt) do { \ + __this_cpu_add(event_stats_cpu.name, cnt); \ +} while(0) + +/** + * scx_agg_event - Aggregate an event counter 'kind' from 'src_e' to 'dst_= e' + * @dst_e: destination event stats + * @src_e: source event stats + * @kind: a kind of event to be aggregated + */ +#define scx_agg_event(dst_e, src_e, kind) do { \ + (dst_e)->kind +=3D READ_ONCE((src_e)->kind); \ +} while(0) + +/** + * scx_dump_event - Dump an event 'kind' in 'events' to 's' + * @s: output seq_buf + * @events: event stats + * @kind: a kind of event to dump + */ +#define scx_dump_event(s, events, kind) do { \ + dump_line(&(s), "%30s: %16llu", #kind, (events)->kind); \ +} while (0) + + +static void scx_bpf_events(struct scx_event_stats *events, size_t events__= sz); + static enum scx_ops_enable_state scx_ops_enable_state(void) { return atomic_read(&scx_ops_enable_state_var); @@ -4785,6 +4843,7 @@ static void scx_dump_state(struct scx_exit_info *ei, = size_t dump_len) .at_jiffies =3D jiffies, }; struct seq_buf s; + struct scx_event_stats events; unsigned long flags; char *buf; int cpu; @@ -4893,6 +4952,12 @@ static void scx_dump_state(struct scx_exit_info *ei,= size_t dump_len) rq_unlock(rq, &rf); } =20 + dump_newline(&s); + dump_line(&s, "Event counters"); + dump_line(&s, "--------------"); + + scx_bpf_events(&events, sizeof(events)); + if (seq_buf_has_overflowed(&s) && dump_len >=3D sizeof(trunc_marker)) memcpy(ei->dump + dump_len - sizeof(trunc_marker), trunc_marker, sizeof(trunc_marker)); @@ -5000,6 +5065,15 @@ static int scx_ops_enable(struct sched_ext_ops *ops,= struct bpf_link *link) =20 mutex_lock(&scx_ops_enable_mutex); =20 + /* + * Clear event counters so a new scx scheduler gets + * fresh event counter values. + */ + for_each_possible_cpu(cpu) { + struct scx_event_stats *e =3D per_cpu_ptr(&event_stats_cpu, cpu); + memset(e, 0, sizeof(*e)); + } + if (!scx_ops_helper) { WRITE_ONCE(scx_ops_helper, scx_create_rt_helper("sched_ext_ops_helper")); @@ -7001,6 +7075,34 @@ __bpf_kfunc u64 scx_bpf_now(void) return clock; } =20 +/* + * scx_bpf_events - Get a system-wide event counter to + * @events: output buffer from a BPF program + * @events__sz: @events len, must end in '__sz'' for the verifier + */ +__bpf_kfunc void scx_bpf_events(struct scx_event_stats *events, + size_t events__sz) +{ + struct scx_event_stats e_sys, *e_cpu; + int cpu; + + /* Aggregate per-CPU event counters into the system-wide counters. */ + memset(&e_sys, 0, sizeof(e_sys)); + for_each_possible_cpu(cpu) { + e_cpu =3D per_cpu_ptr(&event_stats_cpu, cpu); + } + + /* + * We cannot entirely trust a BPF-provided size since a BPF program + * might be compiled against a different vmlinux.h, of which + * scx_event_stats would be larger (a newer vmlinux.h) or smaller + * (an older vmlinux.h). Hence, we use the smaller size to avoid + * memory corruption. + */ + events__sz =3D min(events__sz, sizeof(*events)); + memcpy(events, &e_sys, events__sz); +} + __bpf_kfunc_end_defs(); =20 BTF_KFUNCS_START(scx_kfunc_ids_any) @@ -7033,6 +7135,7 @@ BTF_ID_FLAGS(func, scx_bpf_cpu_rq) BTF_ID_FLAGS(func, scx_bpf_task_cgroup, KF_RCU | KF_ACQUIRE) #endif BTF_ID_FLAGS(func, scx_bpf_now) +BTF_ID_FLAGS(func, scx_bpf_events, KF_TRUSTED_ARGS) BTF_KFUNCS_END(scx_kfunc_ids_any) =20 static const struct btf_kfunc_id_set scx_kfunc_set_any =3D { --=20 2.48.1 From nobody Sun Feb 8 22:08:31 2026 Received: from fanzine2.igalia.com (fanzine.igalia.com [178.60.130.6]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E8D3613A3ED for ; Fri, 31 Jan 2025 07:10:01 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=178.60.130.6 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1738307404; cv=none; b=T1tymZThtEbtIL2DvJQIS0x7Aglo23XFJi2nvIgFO5DqGoXWMn9FTjUis2MCkKQvxSEVRUJqOQQuJSjh/IL5wMikAf+pcytAIOkCyZs2Bb//UaKF1bcQ6LwAJadXsy1FjazPvwHbVwg4pE2zwdAyhxtrl5gz+I5UbnyrWBM6w0o= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1738307404; c=relaxed/simple; bh=EVM2OyD7ITS8CjCs3HeW0si0C7xpIaQVqCPuwTKw9aE=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=M9FWIVf1CvaIy7tT1nMQjwqU9+ZdGUlywib9vb1eeynElscoAHNH0KX91JXf5mdG8oCW+wRsrZRsNjjuWY80SZLMu17Xko0DDDEO2Olc06KqtRvRxGQTjFMJncrqUL9uK7kS6k7BYP9lpdeMT/JrmiJ/BYoT1caaY02LJAS25UA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=igalia.com; spf=pass smtp.mailfrom=igalia.com; dkim=pass (2048-bit key) header.d=igalia.com header.i=@igalia.com header.b=Pa1dEgA8; arc=none smtp.client-ip=178.60.130.6 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=igalia.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=igalia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=igalia.com header.i=@igalia.com header.b="Pa1dEgA8" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=igalia.com; s=20170329; h=Content-Transfer-Encoding:MIME-Version:References:In-Reply-To: Message-ID:Date:Subject:Cc:To:From:Sender:Reply-To:Content-Type:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Id:List-Help:List-Unsubscribe:List-Subscribe: List-Post:List-Owner:List-Archive; bh=bGbZyxBQi2Nkt3T6KZ5le7N8O0Lnv0KnbNQBek0nrjI=; b=Pa1dEgA8aZuwFenNNcmqidmZFL ngWjhImKhX/rV0+U6jcz0YxWrJJlX10VzfUj9Ee79R6es2vqKS8VbQxf5ZNPYIvjWvstUPwOcZS84 cEIbvmx7TXTuim9vyf4mMF8GuUSMBNMn+HEaaF49Zf2jHDX1EFnhlqwhrSgkZFoNvw0zpey93bVLR vjdefWPbfnfTNs+NzfqGkVk3YNiuRR4/EzresgcNKsZ7NqBYCSELoFRF4W3s+WfeKEDJ8Fz21I5tw aG0bg1vNLTWIEucw6wykMUOh8aM/gkj7U1BQlnW5iuRs8RCOcNvjKwIlFI6eZubWZZf3knSskxNb8 sLfEbh6w==; Received: from [58.29.143.236] (helo=localhost) by fanzine2.igalia.com with utf8esmtpsa (Cipher TLS1.3:ECDHE_SECP256R1__RSA_PSS_RSAE_SHA256__AES_256_GCM:256) (Exim) id 1tdl9x-001KYP-Lz; Fri, 31 Jan 2025 08:09:56 +0100 From: Changwoo Min To: tj@kernel.org, void@manifault.com, arighi@nvidia.com Cc: kernel-dev@igalia.com, linux-kernel@vger.kernel.org, Changwoo Min Subject: [PATCH v3 02/11] sched_ext: Add an event, SCX_EV_SELECT_CPU_FALLBACK Date: Fri, 31 Jan 2025 16:09:29 +0900 Message-ID: <20250131070938.95551-3-changwoo@igalia.com> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20250131070938.95551-1-changwoo@igalia.com> References: <20250131070938.95551-1-changwoo@igalia.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Add a core event, SCX_EV_SELECT_CPU_FALLBACK, which represents how many tim= es ops.select_cpu() returns a CPU that the task can't use. __scx_add_event() is used since the caller holds an rq lock, so the preemption has already been disabled. Signed-off-by: Changwoo Min --- include/linux/sched/ext.h | 1 + kernel/sched/ext.c | 14 ++++++++++++++ 2 files changed, 15 insertions(+) diff --git a/include/linux/sched/ext.h b/include/linux/sched/ext.h index 1d70a9867fb1..f7545430a548 100644 --- a/include/linux/sched/ext.h +++ b/include/linux/sched/ext.h @@ -146,6 +146,7 @@ struct sched_ext_entity { u32 weight; s32 sticky_cpu; s32 holding_cpu; + s32 selected_cpu; u32 kf_mask; /* see scx_kf_mask above */ struct task_struct *kf_tasks[2]; /* see SCX_CALL_OP_TASK() */ atomic_long_t ops_state; diff --git a/kernel/sched/ext.c b/kernel/sched/ext.c index 4e28e88e88d4..4d3b32aca48d 100644 --- a/kernel/sched/ext.c +++ b/kernel/sched/ext.c @@ -1444,6 +1444,11 @@ static struct task_struct *scx_task_iter_next_locked= (struct scx_task_iter *iter) * Collection of event counters. Event types are placed in descending orde= r. */ struct scx_event_stats { + /* + * If ops.select_cpu() returns a CPU which can't be used by the task, + * the core scheduler code silently picks a fallback CPU. + */ + u64 SCX_EV_SELECT_CPU_FALLBACK; }; =20 /* @@ -2170,6 +2175,11 @@ static void enqueue_task_scx(struct rq *rq, struct t= ask_struct *p, int enq_flags do_enqueue_task(rq, p, enq_flags, sticky_cpu); out: rq->scx.flags &=3D ~SCX_RQ_IN_WAKEUP; + + if ((enq_flags & SCX_ENQ_CPU_SELECTED) && + (cpu_of(rq) !=3D p->scx.selected_cpu)) { + __scx_add_event(SCX_EV_SELECT_CPU_FALLBACK, 1); + } } =20 static void ops_dequeue(struct task_struct *p, u64 deq_flags) @@ -3240,6 +3250,7 @@ static int select_task_rq_scx(struct task_struct *p, = int prev_cpu, int wake_flag =20 cpu =3D SCX_CALL_OP_TASK_RET(SCX_KF_ENQUEUE | SCX_KF_SELECT_CPU, select_cpu, p, prev_cpu, wake_flags); + p->scx.selected_cpu =3D cpu; *ddsp_taskp =3D NULL; if (ops_cpu_valid(cpu, "from ops.select_cpu()")) return cpu; @@ -3250,6 +3261,7 @@ static int select_task_rq_scx(struct task_struct *p, = int prev_cpu, int wake_flag s32 cpu; =20 cpu =3D scx_select_cpu_dfl(p, prev_cpu, wake_flags, &found); + p->scx.selected_cpu =3D cpu; if (found) { p->scx.slice =3D SCX_SLICE_DFL; p->scx.ddsp_dsq_id =3D SCX_DSQ_LOCAL; @@ -4957,6 +4969,7 @@ static void scx_dump_state(struct scx_exit_info *ei, = size_t dump_len) dump_line(&s, "--------------"); =20 scx_bpf_events(&events, sizeof(events)); + scx_dump_event(s, &events, SCX_EV_SELECT_CPU_FALLBACK); =20 if (seq_buf_has_overflowed(&s) && dump_len >=3D sizeof(trunc_marker)) memcpy(ei->dump + dump_len - sizeof(trunc_marker), @@ -7090,6 +7103,7 @@ __bpf_kfunc void scx_bpf_events(struct scx_event_stat= s *events, memset(&e_sys, 0, sizeof(e_sys)); for_each_possible_cpu(cpu) { e_cpu =3D per_cpu_ptr(&event_stats_cpu, cpu); + scx_agg_event(&e_sys, e_cpu, SCX_EV_SELECT_CPU_FALLBACK); } =20 /* --=20 2.48.1 From nobody Sun Feb 8 22:08:31 2026 Received: from fanzine2.igalia.com (fanzine.igalia.com [178.60.130.6]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2520715B546 for ; Fri, 31 Jan 2025 07:10:05 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=178.60.130.6 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1738307407; cv=none; b=FS1jNmmV+OBYA/baxNUQ3+eKIGl6rgsV/9g+QJc3/VQ0DnkQB2ZBpWUQ58jrYgkFf+LG459O2TKvzkok9Rt9Ele5FboiqUbV650/4Phul0kJQPgFbGIGMVFFLgxzpnLVu6Nzy32LVM1IzW/b3DuKi147c/b3l9sFuhQzd0ZiT70= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1738307407; c=relaxed/simple; bh=c1Il5sS3fT3lVg9aTCV1w+wu5xAvQXF+dUafJxi9yvY=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=kTfWq50+3MhkFD3x48u6R9CqBKirAIYozE4BOpLz9uu8RV9uZXG6jjVRSMX6/+xaBgQrnhtLpjUDNomTDTNSPovoLxnrBKrct/VrHYaTolba6GnegRFEdoqlCAYwXLj1mzVIciJ4R4km3i4fK4SUhaUmDzfsx/99RQrwl2iARTo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=igalia.com; spf=pass smtp.mailfrom=igalia.com; dkim=pass (2048-bit key) header.d=igalia.com header.i=@igalia.com header.b=C6fTr8O+; arc=none smtp.client-ip=178.60.130.6 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=igalia.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=igalia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=igalia.com header.i=@igalia.com header.b="C6fTr8O+" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=igalia.com; s=20170329; h=Content-Transfer-Encoding:MIME-Version:References:In-Reply-To: Message-ID:Date:Subject:Cc:To:From:Sender:Reply-To:Content-Type:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Id:List-Help:List-Unsubscribe:List-Subscribe: List-Post:List-Owner:List-Archive; bh=Ynq2LvCTfrKNO0aYPqmVtw2j6T8KCcEwVFZ3rO6djNY=; b=C6fTr8O+MvS331Ah+3j3neyvxf g1qVkjo+8kH1rWZD7FjLdU4rLNP0ilGZipfTBnzQZjyZoBrb64WSOy8PofUF4qKcjuMJANERLHLe4 t66RxSMVoVvTSTrvH6EQGB5S4M5IGUK0kDCeXgjhzSsI3SY1nI/7LZHTYGfT7b+GwdCqLfMDTNPas P1y1pCij2bfAYHVImEDF38GCdjV1f90BE3McSsD4CoynVVRd87mswEROoHlcA5sOqH/WlOeo6r2LM 4b+dy+PIaQLrF1S8I3f0zbfOd5PHIQz6fyL1B4+ajX8IpuNMDm6929Ca2QlZ4c0UpzC6GfV3wbpUz ZX/2PZ5A==; Received: from [58.29.143.236] (helo=localhost) by fanzine2.igalia.com with utf8esmtpsa (Cipher TLS1.3:ECDHE_SECP256R1__RSA_PSS_RSAE_SHA256__AES_256_GCM:256) (Exim) id 1tdlA2-001KYq-BU; Fri, 31 Jan 2025 08:10:00 +0100 From: Changwoo Min To: tj@kernel.org, void@manifault.com, arighi@nvidia.com Cc: kernel-dev@igalia.com, linux-kernel@vger.kernel.org, Changwoo Min Subject: [PATCH v3 03/11] sched_ext: Add an event, SCX_EV_DISPATCH_LOCAL_DSQ_OFFLINE Date: Fri, 31 Jan 2025 16:09:30 +0900 Message-ID: <20250131070938.95551-4-changwoo@igalia.com> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20250131070938.95551-1-changwoo@igalia.com> References: <20250131070938.95551-1-changwoo@igalia.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Add a core event, SCX_EV_DISPATCH_LOCAL_DSQ_OFFLINE, which represents how many times a BPF scheduler tries to dispatch to an offlined local DSQ. __scx_add_event() is used since the caller holds an rq lock, so the preemption has already been disabled. Signed-off-by: Changwoo Min --- kernel/sched/ext.c | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/kernel/sched/ext.c b/kernel/sched/ext.c index 4d3b32aca48d..041b0af3551a 100644 --- a/kernel/sched/ext.c +++ b/kernel/sched/ext.c @@ -1449,6 +1449,12 @@ struct scx_event_stats { * the core scheduler code silently picks a fallback CPU. */ u64 SCX_EV_SELECT_CPU_FALLBACK; + + /* + * When dispatching to a local DSQ, the CPU may have gone offline in + * the meantime. In this case, the task is bounced to the global DSQ. + */ + u64 SCX_EV_DISPATCH_LOCAL_DSQ_OFFLINE; }; =20 /* @@ -2644,6 +2650,7 @@ static void dispatch_to_local_dsq(struct rq *rq, stru= ct scx_dispatch_q *dst_dsq, if (unlikely(!task_can_run_on_remote_rq(p, dst_rq, true))) { dispatch_enqueue(find_global_dsq(p), p, enq_flags | SCX_ENQ_CLEAR_OPSS); + __scx_add_event(SCX_EV_DISPATCH_LOCAL_DSQ_OFFLINE, 1); return; } =20 @@ -4970,6 +4977,7 @@ static void scx_dump_state(struct scx_exit_info *ei, = size_t dump_len) =20 scx_bpf_events(&events, sizeof(events)); scx_dump_event(s, &events, SCX_EV_SELECT_CPU_FALLBACK); + scx_dump_event(s, &events, SCX_EV_DISPATCH_LOCAL_DSQ_OFFLINE); =20 if (seq_buf_has_overflowed(&s) && dump_len >=3D sizeof(trunc_marker)) memcpy(ei->dump + dump_len - sizeof(trunc_marker), @@ -7104,6 +7112,7 @@ __bpf_kfunc void scx_bpf_events(struct scx_event_stat= s *events, for_each_possible_cpu(cpu) { e_cpu =3D per_cpu_ptr(&event_stats_cpu, cpu); scx_agg_event(&e_sys, e_cpu, SCX_EV_SELECT_CPU_FALLBACK); + scx_agg_event(&e_sys, e_cpu, SCX_EV_DISPATCH_LOCAL_DSQ_OFFLINE); } =20 /* --=20 2.48.1 From nobody Sun Feb 8 22:08:31 2026 Received: from fanzine2.igalia.com (fanzine.igalia.com [178.60.130.6]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6EC101714C6 for ; Fri, 31 Jan 2025 07:10:11 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=178.60.130.6 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1738307413; cv=none; b=mi2FHaK4TRal0W1Piqqgzb7FHk4Pdiohdtj75NWzHQ5Mb0hsY17XqBYIu2N7+0PSis4d5LxvA07YA2xOjUdM2qmsCIqE9Lk5GIo6yCmF+mM2BgdnafrcSzGX/JqZUXo4e4uik5Cn2EHE8/YmrMfphR3LSaeiRRJB0qlHys9Nj60= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1738307413; c=relaxed/simple; bh=8vHu4xE+SSmDHI0c8AnDSXYni7h7urbVNTurAdRsVuU=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=PebSrpsx5b337r3V74TZuiXhV8wmLzaELPhK8um7KJAojpclS/pL00wH+BRK5+Vmui0TIA0CHn8o30R2NWFV54MdRm13vhD73pK6qDa033reZLtrRF8nQoTw+dyvTRLw9D3Av+E3wOgtco5LJimrTU7bFt30H7jANzbpwiZfxEA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=igalia.com; spf=pass smtp.mailfrom=igalia.com; dkim=pass (2048-bit key) header.d=igalia.com header.i=@igalia.com header.b=ZhHMT+Wy; arc=none smtp.client-ip=178.60.130.6 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=igalia.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=igalia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=igalia.com header.i=@igalia.com header.b="ZhHMT+Wy" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=igalia.com; s=20170329; h=Content-Transfer-Encoding:MIME-Version:References:In-Reply-To: Message-ID:Date:Subject:Cc:To:From:Sender:Reply-To:Content-Type:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Id:List-Help:List-Unsubscribe:List-Subscribe: List-Post:List-Owner:List-Archive; bh=AOi9u7Zk84KsH8av/AW4gcZtLuwHZ/58/8SnsZwDWjg=; b=ZhHMT+Wym/x/QXRWjybr2beVeU +Ssqssn9zrkNZ5Rbli70GQtjXtl2MrfNzXoAylljMWkSiQpYapX9nOSidHbqdEx8tIiDMBEbqbAuR Vx1capXZYtw+6JK/LpWgu7RWgikI5j9TqGisLsCXte9dQ2jk7VCRuBQjlR2WelSjoyeWiGMWCFFap JqBIxkuofGQkKMfelAiqMeglx6b8ccg2Nxh/3xq0TWWRvYPUqTnxlKQgMTNXKhgPodRwejUsICgnJ eh1Ik5oZWcDfwXjfsHEVJCO2ni58GkLabbx+1/raFgrvN5ZO96BF/ibSsHcjy/7Zjfbwicxgi0P/k 8CkPLnHw==; Received: from [58.29.143.236] (helo=localhost) by fanzine2.igalia.com with utf8esmtpsa (Cipher TLS1.3:ECDHE_SECP256R1__RSA_PSS_RSAE_SHA256__AES_256_GCM:256) (Exim) id 1tdlA6-001KZB-Lz; Fri, 31 Jan 2025 08:10:05 +0100 From: Changwoo Min To: tj@kernel.org, void@manifault.com, arighi@nvidia.com Cc: kernel-dev@igalia.com, linux-kernel@vger.kernel.org, Changwoo Min Subject: [PATCH v3 04/11] sched_ext: Add an event, SCX_EV_DISPATCH_KEEP_LAST Date: Fri, 31 Jan 2025 16:09:31 +0900 Message-ID: <20250131070938.95551-5-changwoo@igalia.com> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20250131070938.95551-1-changwoo@igalia.com> References: <20250131070938.95551-1-changwoo@igalia.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Add a core event, SCX_EV_DISPATCH_KEEP_LAST, which represents how many times a task is continued to run without ops.enqueue() when SCX_OPS_ENQ_LAST is not set. __scx_add_event() is used since the caller holds an rq lock, so the preemption has already been disabled. Signed-off-by: Changwoo Min --- kernel/sched/ext.c | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/kernel/sched/ext.c b/kernel/sched/ext.c index 041b0af3551a..7147f730850b 100644 --- a/kernel/sched/ext.c +++ b/kernel/sched/ext.c @@ -1455,6 +1455,12 @@ struct scx_event_stats { * the meantime. In this case, the task is bounced to the global DSQ. */ u64 SCX_EV_DISPATCH_LOCAL_DSQ_OFFLINE; + + /* + * If SCX_OPS_ENQ_LAST is not set, the number of times that a task + * continued to run because there were no other tasks on the CPU. + */ + u64 SCX_EV_DISPATCH_KEEP_LAST; }; =20 /* @@ -2908,6 +2914,7 @@ static int balance_one(struct rq *rq, struct task_str= uct *prev) if (prev_on_rq && (!static_branch_unlikely(&scx_ops_enq_last) || scx_rq_bypassing(rq))) { rq->scx.flags |=3D SCX_RQ_BAL_KEEP; + __scx_add_event(SCX_EV_DISPATCH_KEEP_LAST, 1); goto has_tasks; } rq->scx.flags &=3D ~SCX_RQ_IN_BALANCE; @@ -4978,6 +4985,7 @@ static void scx_dump_state(struct scx_exit_info *ei, = size_t dump_len) scx_bpf_events(&events, sizeof(events)); scx_dump_event(s, &events, SCX_EV_SELECT_CPU_FALLBACK); scx_dump_event(s, &events, SCX_EV_DISPATCH_LOCAL_DSQ_OFFLINE); + scx_dump_event(s, &events, SCX_EV_DISPATCH_KEEP_LAST); =20 if (seq_buf_has_overflowed(&s) && dump_len >=3D sizeof(trunc_marker)) memcpy(ei->dump + dump_len - sizeof(trunc_marker), @@ -7113,6 +7121,7 @@ __bpf_kfunc void scx_bpf_events(struct scx_event_stat= s *events, e_cpu =3D per_cpu_ptr(&event_stats_cpu, cpu); scx_agg_event(&e_sys, e_cpu, SCX_EV_SELECT_CPU_FALLBACK); scx_agg_event(&e_sys, e_cpu, SCX_EV_DISPATCH_LOCAL_DSQ_OFFLINE); + scx_agg_event(&e_sys, e_cpu, SCX_EV_DISPATCH_KEEP_LAST); } =20 /* --=20 2.48.1 From nobody Sun Feb 8 22:08:31 2026 Received: from fanzine2.igalia.com (fanzine.igalia.com [178.60.130.6]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0A642187FE4 for ; Fri, 31 Jan 2025 07:10:15 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=178.60.130.6 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1738307416; cv=none; b=oXPncRLJZecC56lIVyXz9pss0c4/rAhPuLgEQ+rvxUgi8jG/eBuNwBDe4y9sy2LSrS15LuZt3/2Tcf47ZHR0Xvz36k7NuHmqkw7IKrZBstQsRTm+W9yawadsFK+gWT4PmtzxgJWIz4Cop9tYn1IRwQ5eAA3wY+BUEIU2NP1LdLs= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1738307416; c=relaxed/simple; bh=mDwzX9VrQutUfjwLuixleqNUJED35UXVAOa5fYcdFAw=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=qojp4GHVyI1y37KhjBCGO1ExpKFFKKwAFjDvbPE0hRU0Lf/MKOqybNoTYaObxK1hu7yVSXyjZS/Cf87Djnhb0RKaUhVMDYSLOD34OQsdGkfFxaVseUPhVHhsQpwB7QVtRIZjT5HNh8plXmciICyz4ePJtqxJdhprgXO+niAYoDI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=igalia.com; spf=pass smtp.mailfrom=igalia.com; dkim=pass (2048-bit key) header.d=igalia.com header.i=@igalia.com header.b=rba8Y1uU; arc=none smtp.client-ip=178.60.130.6 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=igalia.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=igalia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=igalia.com header.i=@igalia.com header.b="rba8Y1uU" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=igalia.com; s=20170329; h=Content-Transfer-Encoding:MIME-Version:References:In-Reply-To: Message-ID:Date:Subject:Cc:To:From:Sender:Reply-To:Content-Type:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Id:List-Help:List-Unsubscribe:List-Subscribe: List-Post:List-Owner:List-Archive; bh=bxwZ2LxOeAlK9DczuHndKbfrLafoKJEyndGUY9K2ifA=; b=rba8Y1uUxksQD78vj1/1msY5je DEx1gCPqEKSbpyTfKW1yVZoGqG2VDWr9a03o/DcrM75LCqVeODCvJnyEl2VHoo5qcIYNXM/VdOkOw BO/vzO3ybO5C8E7hXHe27D1XOV2tveRkXg5cfrylzah8J67VFuVTOq2LxNRjzteBhZTT6faK1nGqF 0srD+sw4p+GDgFqSHVCJXvNOiX0IE53E7pY2EEBSi+WSsWzyyGlV6rYAcXM0A0TKquhtOwkvD5che RN0n83n6LZIv9bSCcp4XiIv4/IX1Ixs3xSTO2Z5aAtrunl4f9s8nICMlPpf/gklKWt2pboOvZAjlp nVSgYSHg==; Received: from [58.29.143.236] (helo=localhost) by fanzine2.igalia.com with utf8esmtpsa (Cipher TLS1.3:ECDHE_SECP256R1__RSA_PSS_RSAE_SHA256__AES_256_GCM:256) (Exim) id 1tdlAA-001KZa-SW; Fri, 31 Jan 2025 08:10:09 +0100 From: Changwoo Min To: tj@kernel.org, void@manifault.com, arighi@nvidia.com Cc: kernel-dev@igalia.com, linux-kernel@vger.kernel.org, Changwoo Min Subject: [PATCH v3 05/11] sched_ext: Add an event, SCX_EV_ENQ_SKIP_EXITING Date: Fri, 31 Jan 2025 16:09:32 +0900 Message-ID: <20250131070938.95551-6-changwoo@igalia.com> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20250131070938.95551-1-changwoo@igalia.com> References: <20250131070938.95551-1-changwoo@igalia.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Add a core event, SCX_EV_ENQ_SKIP_EXITING, which represents how many times a task is enqueued to a local DSQ when exiting if SCX_OPS_ENQ_EXITING is not set. __scx_add_event() is used since the caller holds an rq lock, so the preemption has already been disabled. Signed-off-by: Changwoo Min --- kernel/sched/ext.c | 11 ++++++++++- 1 file changed, 10 insertions(+), 1 deletion(-) diff --git a/kernel/sched/ext.c b/kernel/sched/ext.c index 7147f730850b..37f86a84ac5d 100644 --- a/kernel/sched/ext.c +++ b/kernel/sched/ext.c @@ -1461,6 +1461,12 @@ struct scx_event_stats { * continued to run because there were no other tasks on the CPU. */ u64 SCX_EV_DISPATCH_KEEP_LAST; + + /* + * If SCX_OPS_ENQ_EXITING is not set, the number of times that a task + * is dispatched to a local DSQ when exiting. + */ + u64 SCX_EV_ENQ_SKIP_EXITING; }; =20 /* @@ -2068,8 +2074,10 @@ static void do_enqueue_task(struct rq *rq, struct ta= sk_struct *p, u64 enq_flags, =20 /* see %SCX_OPS_ENQ_EXITING */ if (!static_branch_unlikely(&scx_ops_enq_exiting) && - unlikely(p->flags & PF_EXITING)) + unlikely(p->flags & PF_EXITING)) { + __scx_add_event(SCX_EV_ENQ_SKIP_EXITING, 1); goto local; + } =20 if (!SCX_HAS_OP(enqueue)) goto global; @@ -4986,6 +4994,7 @@ static void scx_dump_state(struct scx_exit_info *ei, = size_t dump_len) scx_dump_event(s, &events, SCX_EV_SELECT_CPU_FALLBACK); scx_dump_event(s, &events, SCX_EV_DISPATCH_LOCAL_DSQ_OFFLINE); scx_dump_event(s, &events, SCX_EV_DISPATCH_KEEP_LAST); + scx_dump_event(s, &events, SCX_EV_ENQ_SKIP_EXITING); =20 if (seq_buf_has_overflowed(&s) && dump_len >=3D sizeof(trunc_marker)) memcpy(ei->dump + dump_len - sizeof(trunc_marker), --=20 2.48.1 From nobody Sun Feb 8 22:08:31 2026 Received: from fanzine2.igalia.com (fanzine.igalia.com [178.60.130.6]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5E686155A25 for ; Fri, 31 Jan 2025 07:10:21 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=178.60.130.6 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1738307422; cv=none; b=p68evd1cxxsmHDNttDB5n7MMPifVJEw9QGp/5c36SAy3OTDo4Ma0H7axnD223M1rYrSY6Y+KX0+mnvqYmGxMduEdpGk9ggsMmmNxBQ0RxUJNrShDlOPZVSCsw1avSEbJIF4X8SyfrV/H2CwLmH7shNSbM+PcuzXguVaf6cAXCao= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1738307422; c=relaxed/simple; bh=vVTmVove9mAvo8Cw+DJNPc3wotFTo2uOH6r2mhjUcXI=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=RAKstAvlZrLLlYnrM5zGVtiGA8njfp6lfwreQ3OFYEr+/CNDpumrcSCOkG6rxIUef0me5gkLk2+QjPieYhSzD7/spLTUWanwW+4KvezeiCIA7iClyvtGrD0jlEOppFzZe+F0XECqLQa8W+GDDNyP5Odm2ny08P7YcUSKTpUi/ZI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=igalia.com; spf=pass smtp.mailfrom=igalia.com; dkim=pass (2048-bit key) header.d=igalia.com header.i=@igalia.com header.b=F/hq1Uhh; arc=none smtp.client-ip=178.60.130.6 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=igalia.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=igalia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=igalia.com header.i=@igalia.com header.b="F/hq1Uhh" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=igalia.com; s=20170329; h=Content-Transfer-Encoding:MIME-Version:References:In-Reply-To: Message-ID:Date:Subject:Cc:To:From:Sender:Reply-To:Content-Type:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Id:List-Help:List-Unsubscribe:List-Subscribe: List-Post:List-Owner:List-Archive; bh=BnttP+/65C5cMgnCxDHDe3eQwYOHd71rGHs5NrkOf4M=; b=F/hq1UhhRLANz/knCxKfsV9Hyj oB4Rp76k2QbXaJ/l7SI3us/bdPEpK66sHmFHZYf+Ntud+eav/sTuZShEbh0We9uqkK9EUKwEe7/97 nTGim91pKiqwAzphEUWpYoKZZAU9WqFP+sh9ZFhIiJ0QlwwQujadruL7g+4UaI2qi+eXRDH/tWVnU dI4iVw/7K8lNWfAvOFieHTtiUKv1TTlp5gfzaP7b+LZ2/6Sw87nmyO37uSGKENZgDS3pJBemX5T4t avsKAsGHYtk7xD0h6Wa63y30M10h5M6kuwES/zq4JamxY/H416ou/1scPh102mVWG/KHiicxJy3Ey aSGRvF4Q==; Received: from [58.29.143.236] (helo=localhost) by fanzine2.igalia.com with utf8esmtpsa (Cipher TLS1.3:ECDHE_SECP256R1__RSA_PSS_RSAE_SHA256__AES_256_GCM:256) (Exim) id 1tdlAF-001KZz-PE; Fri, 31 Jan 2025 08:10:14 +0100 From: Changwoo Min To: tj@kernel.org, void@manifault.com, arighi@nvidia.com Cc: kernel-dev@igalia.com, linux-kernel@vger.kernel.org, Changwoo Min Subject: [PATCH v3 06/11] sched_ext: Add an event, SCX_EV_BYPASS_ACTIVATE Date: Fri, 31 Jan 2025 16:09:33 +0900 Message-ID: <20250131070938.95551-7-changwoo@igalia.com> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20250131070938.95551-1-changwoo@igalia.com> References: <20250131070938.95551-1-changwoo@igalia.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Add a core event, SCX_EV_BYPASS_ACTIVATE, which represents how many times the bypass mode has been triggered. Signed-off-by: Changwoo Min --- kernel/sched/ext.c | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/kernel/sched/ext.c b/kernel/sched/ext.c index 37f86a84ac5d..07d54b52e971 100644 --- a/kernel/sched/ext.c +++ b/kernel/sched/ext.c @@ -1467,6 +1467,11 @@ struct scx_event_stats { * is dispatched to a local DSQ when exiting. */ u64 SCX_EV_ENQ_SKIP_EXITING; + + /* + * The number of times the bypassing mode has been activated. + */ + u64 SCX_EV_BYPASS_ACTIVATE; }; =20 /* @@ -4400,6 +4405,7 @@ static void scx_ops_bypass(bool bypass) WARN_ON_ONCE(scx_ops_bypass_depth <=3D 0); if (scx_ops_bypass_depth !=3D 1) goto unlock; + scx_add_event(SCX_EV_BYPASS_ACTIVATE, 1); } else { scx_ops_bypass_depth--; WARN_ON_ONCE(scx_ops_bypass_depth < 0); @@ -4995,6 +5001,7 @@ static void scx_dump_state(struct scx_exit_info *ei, = size_t dump_len) scx_dump_event(s, &events, SCX_EV_DISPATCH_LOCAL_DSQ_OFFLINE); scx_dump_event(s, &events, SCX_EV_DISPATCH_KEEP_LAST); scx_dump_event(s, &events, SCX_EV_ENQ_SKIP_EXITING); + scx_dump_event(s, &events, SCX_EV_BYPASS_ACTIVATE); =20 if (seq_buf_has_overflowed(&s) && dump_len >=3D sizeof(trunc_marker)) memcpy(ei->dump + dump_len - sizeof(trunc_marker), @@ -7131,6 +7138,7 @@ __bpf_kfunc void scx_bpf_events(struct scx_event_stat= s *events, scx_agg_event(&e_sys, e_cpu, SCX_EV_SELECT_CPU_FALLBACK); scx_agg_event(&e_sys, e_cpu, SCX_EV_DISPATCH_LOCAL_DSQ_OFFLINE); scx_agg_event(&e_sys, e_cpu, SCX_EV_DISPATCH_KEEP_LAST); + scx_agg_event(&e_sys, e_cpu, SCX_EV_BYPASS_ACTIVATE); } =20 /* --=20 2.48.1 From nobody Sun Feb 8 22:08:31 2026 Received: from fanzine2.igalia.com (fanzine.igalia.com [178.60.130.6]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3BD5D1991CD for ; Fri, 31 Jan 2025 07:10:23 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=178.60.130.6 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1738307425; cv=none; b=uvrIYHsvvAuGhavHUcl6vGcL1H8ExWMMIGTDcsGK/qTceQhLuSz1nQ0+lcicGutRNChK2M3ezufQNOYV+2jKm6m6AoXl+PKMuV2h0IJvOPne+/RwEyDSBOmI/SI0WH+VTH24F6yqmUPhyC/8lhyYfGv0MrTf+qmqW3a2uOoEuUg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1738307425; c=relaxed/simple; bh=mXdQzqXcH4zKY5o96idyYEdbQkIzhskgmnZIKyFbBcY=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=HXLjCVi7iPJiPYeTLY8farKTngP18154pFVBKIR7Ece5gxtMtGwcu72XYhS7ht7VGCzaXtRS/MNRy8CzEfWVv4XuBJQhDa1vRCFEfFi00Aj7grmhG2jVycNYW2xYVTOHL2dLlUzPKDPQGf4gjo0eRYtBCaMZUYAAM9MkaLKxNgU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=igalia.com; spf=pass smtp.mailfrom=igalia.com; dkim=pass (2048-bit key) header.d=igalia.com header.i=@igalia.com header.b=sBB+tfEP; arc=none smtp.client-ip=178.60.130.6 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=igalia.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=igalia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=igalia.com header.i=@igalia.com header.b="sBB+tfEP" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=igalia.com; s=20170329; h=Content-Transfer-Encoding:MIME-Version:References:In-Reply-To: Message-ID:Date:Subject:Cc:To:From:Sender:Reply-To:Content-Type:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Id:List-Help:List-Unsubscribe:List-Subscribe: List-Post:List-Owner:List-Archive; bh=fWzhjrX4qthxXZF2x/QTL0MwkUO4yL8TN0VU/ObpBQc=; b=sBB+tfEPw2hzTaW01saDldMcNM 3tG1ng38mQlgWrXohbYY5pOMohM0WxQC21K6kWSw16R8lolvTQOqD/hyrRGc78dRrRoBNWkaSxtxw x5ygpBqXizOF2NccX72Kzv+N8EMiA951RpmiKY4YpgsBcKROJwzb69srNtYCOaLofgRUl0BOt1NUa nlUn9zt2Hi8x+r944MLwRt4y3sf1bZOMWZ4D45syfQvsdcKv6enA9+rOXqa910/z8CjLQy0biw2nf pDcd03MclhM7+hFZeS5gfwKhvcCJmW3WTL/3jnBkCWzTINHStf2A8gkwRB6vZF2YkIRuRzPEoczxG sWyMU/LA==; Received: from [58.29.143.236] (helo=localhost) by fanzine2.igalia.com with utf8esmtpsa (Cipher TLS1.3:ECDHE_SECP256R1__RSA_PSS_RSAE_SHA256__AES_256_GCM:256) (Exim) id 1tdlAK-001Ka9-9w; Fri, 31 Jan 2025 08:10:18 +0100 From: Changwoo Min To: tj@kernel.org, void@manifault.com, arighi@nvidia.com Cc: kernel-dev@igalia.com, linux-kernel@vger.kernel.org, Changwoo Min Subject: [PATCH v3 07/11] sched_ext: Add an event, SCX_EV_BYPASS_DISPATCH Date: Fri, 31 Jan 2025 16:09:34 +0900 Message-ID: <20250131070938.95551-8-changwoo@igalia.com> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20250131070938.95551-1-changwoo@igalia.com> References: <20250131070938.95551-1-changwoo@igalia.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Add a core event, SCX_EV_BYPASS_DISPATCH, which represents how many tasks have been dispatched in the bypass mode. __scx_add_event() is used since the caller holds an rq lock, so the preemption has already been disabled. Signed-off-by: Changwoo Min --- kernel/sched/ext.c | 27 +++++++++++++++++++++++---- 1 file changed, 23 insertions(+), 4 deletions(-) diff --git a/kernel/sched/ext.c b/kernel/sched/ext.c index 07d54b52e971..236cdb0071eb 100644 --- a/kernel/sched/ext.c +++ b/kernel/sched/ext.c @@ -1468,6 +1468,11 @@ struct scx_event_stats { */ u64 SCX_EV_ENQ_SKIP_EXITING; =20 + /* + * The number of tasks dispatched in the bypassing mode. + */ + u64 SCX_EV_BYPASS_DISPATCH; + /* * The number of times the bypassing mode has been activated. */ @@ -2869,11 +2874,17 @@ static int balance_one(struct rq *rq, struct task_s= truct *prev) } =20 /* if there already are tasks to run, nothing to do */ - if (rq->scx.local_dsq.nr) + if (rq->scx.local_dsq.nr) { + if (scx_rq_bypassing(rq)) + __scx_add_event(SCX_EV_BYPASS_DISPATCH, 1); goto has_tasks; + } =20 - if (consume_global_dsq(rq)) + if (consume_global_dsq(rq)) { + if (scx_rq_bypassing(rq)) + __scx_add_event(SCX_EV_BYPASS_DISPATCH, 1); goto has_tasks; + } =20 if (!SCX_HAS_OP(dispatch) || scx_rq_bypassing(rq) || !scx_rq_online(rq)) goto no_tasks; @@ -2899,10 +2910,16 @@ static int balance_one(struct rq *rq, struct task_s= truct *prev) rq->scx.flags |=3D SCX_RQ_BAL_KEEP; goto has_tasks; } - if (rq->scx.local_dsq.nr) + if (rq->scx.local_dsq.nr) { + if (scx_rq_bypassing(rq)) + __scx_add_event(SCX_EV_BYPASS_DISPATCH, 1); goto has_tasks; - if (consume_global_dsq(rq)) + } + if (consume_global_dsq(rq)) { + if (scx_rq_bypassing(rq)) + __scx_add_event(SCX_EV_BYPASS_DISPATCH, 1); goto has_tasks; + } =20 /* * ops.dispatch() can trap us in this loop by repeatedly @@ -5001,6 +5018,7 @@ static void scx_dump_state(struct scx_exit_info *ei, = size_t dump_len) scx_dump_event(s, &events, SCX_EV_DISPATCH_LOCAL_DSQ_OFFLINE); scx_dump_event(s, &events, SCX_EV_DISPATCH_KEEP_LAST); scx_dump_event(s, &events, SCX_EV_ENQ_SKIP_EXITING); + scx_dump_event(s, &events, SCX_EV_BYPASS_DISPATCH); scx_dump_event(s, &events, SCX_EV_BYPASS_ACTIVATE); =20 if (seq_buf_has_overflowed(&s) && dump_len >=3D sizeof(trunc_marker)) @@ -7138,6 +7156,7 @@ __bpf_kfunc void scx_bpf_events(struct scx_event_stat= s *events, scx_agg_event(&e_sys, e_cpu, SCX_EV_SELECT_CPU_FALLBACK); scx_agg_event(&e_sys, e_cpu, SCX_EV_DISPATCH_LOCAL_DSQ_OFFLINE); scx_agg_event(&e_sys, e_cpu, SCX_EV_DISPATCH_KEEP_LAST); + scx_agg_event(&e_sys, e_cpu, SCX_EV_BYPASS_DISPATCH); scx_agg_event(&e_sys, e_cpu, SCX_EV_BYPASS_ACTIVATE); } =20 --=20 2.48.1 From nobody Sun Feb 8 22:08:31 2026 Received: from fanzine2.igalia.com (fanzine.igalia.com [178.60.130.6]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B83711AF0C3 for ; Fri, 31 Jan 2025 07:10:28 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=178.60.130.6 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1738307430; cv=none; b=cWg9ktdIBIKyIyuE12P3LroeAM95zXqYuRh850PTtk+UXE9LkPMiKINKLYdlNOuq9F7rDVxNjumhD137nymdDsBrGI33Qh4xhau1kiViuEPDNyqQo+K1TtSDCPUSyiYR65TRqdsOxbI3nlqt3VDCndf7zy7MFmAUum0UWJ7kDl4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1738307430; c=relaxed/simple; bh=/ZvKVk8rZeGtmp6VLLD4DU6idDxy5o/EYItE8QUyDF8=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=DH5NWw4LoUhPY42FVm9oDe10wl2zUsDWa2p3AJHOw456hfZiCSrNRdJLm7sKQpD/Y5bTi823T8MQdab3MRZ33YglKNTpYdl8XAshCF4+wX1+c3MDR+CRKsOfPsAGvpFDZwIc6BLy3Eo/M71rVw9EI5m6naYR99pnzrTY3hcDOEs= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=igalia.com; spf=pass smtp.mailfrom=igalia.com; dkim=pass (2048-bit key) header.d=igalia.com header.i=@igalia.com header.b=jM9xGB+F; arc=none smtp.client-ip=178.60.130.6 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=igalia.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=igalia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=igalia.com header.i=@igalia.com header.b="jM9xGB+F" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=igalia.com; s=20170329; h=Content-Transfer-Encoding:MIME-Version:References:In-Reply-To: Message-ID:Date:Subject:Cc:To:From:Sender:Reply-To:Content-Type:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Id:List-Help:List-Unsubscribe:List-Subscribe: List-Post:List-Owner:List-Archive; bh=hLEJlMt6aWjNLe6pMLu/dCCqgGrbqAA/7mCNiug+4io=; b=jM9xGB+FmyBhZ+Yg7cTQ3DgPFS HkXaVIB0koZjKKhcTFsYlgsyXikCe1RX7FcF3WLDnttbwF3ai3sibI/2p45NCSr1Y6l+36rWjXNgY jZRQwk/qkCBXJxykR6RwFmFge3VQzOvd5olzCG9sSUp07TnmqcxQMq7s9z1opGtby6OfCMm89WWEi 0jqfIY2iAz6rjnZPE9qmotBKH9rX3+yzED8fiMVH17uYD/NpMK6jhKrhtVN9b7/yZjuHC0Ph9ydyo JZFdBelepxP9AnabJnnzjwiXax7TBGG7MNy4iHs0pLHRPM+RXKdjWaUvL3pN+qw5IhIHx3xhHYtDK SmJWbXHg==; Received: from [58.29.143.236] (helo=localhost) by fanzine2.igalia.com with utf8esmtpsa (Cipher TLS1.3:ECDHE_SECP256R1__RSA_PSS_RSAE_SHA256__AES_256_GCM:256) (Exim) id 1tdlAO-001KaL-Pr; Fri, 31 Jan 2025 08:10:23 +0100 From: Changwoo Min To: tj@kernel.org, void@manifault.com, arighi@nvidia.com Cc: kernel-dev@igalia.com, linux-kernel@vger.kernel.org, Changwoo Min Subject: [PATCH v3 08/11] sched_ext: Add an event, SCX_EV_BYPASS_DURATION Date: Fri, 31 Jan 2025 16:09:35 +0900 Message-ID: <20250131070938.95551-9-changwoo@igalia.com> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20250131070938.95551-1-changwoo@igalia.com> References: <20250131070938.95551-1-changwoo@igalia.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Add a core event, SCX_EV_BYPASS_DURATION, which represents the total duration of bypass modes in nanoseconds. Signed-off-by: Changwoo Min --- kernel/sched/ext.c | 16 ++++++++++++++++ 1 file changed, 16 insertions(+) diff --git a/kernel/sched/ext.c b/kernel/sched/ext.c index 236cdb0071eb..b4871409bd1c 100644 --- a/kernel/sched/ext.c +++ b/kernel/sched/ext.c @@ -1468,6 +1468,11 @@ struct scx_event_stats { */ u64 SCX_EV_ENQ_SKIP_EXITING; =20 + /* + * The total duration of bypass modes in nanoseconds. + */ + u64 SCX_EV_BYPASS_DURATION; + /* * The number of tasks dispatched in the bypassing mode. */ @@ -1529,6 +1534,12 @@ static DEFINE_PER_CPU(struct scx_event_stats, event_= stats_cpu); } while (0) =20 =20 +/* + * The last time the bypass mode started. + * This is used to measure SCX_EV_BYPASS_DURATION. + */ +static unsigned long scx_bypass_timestamp; + static void scx_bpf_events(struct scx_event_stats *events, size_t events__= sz); =20 static enum scx_ops_enable_state scx_ops_enable_state(void) @@ -4422,12 +4433,15 @@ static void scx_ops_bypass(bool bypass) WARN_ON_ONCE(scx_ops_bypass_depth <=3D 0); if (scx_ops_bypass_depth !=3D 1) goto unlock; + scx_bypass_timestamp =3D ktime_get_ns(); scx_add_event(SCX_EV_BYPASS_ACTIVATE, 1); } else { scx_ops_bypass_depth--; WARN_ON_ONCE(scx_ops_bypass_depth < 0); if (scx_ops_bypass_depth !=3D 0) goto unlock; + scx_add_event(SCX_EV_BYPASS_DURATION, + ktime_get_ns() - scx_bypass_timestamp); } =20 atomic_inc(&scx_ops_breather_depth); @@ -5018,6 +5032,7 @@ static void scx_dump_state(struct scx_exit_info *ei, = size_t dump_len) scx_dump_event(s, &events, SCX_EV_DISPATCH_LOCAL_DSQ_OFFLINE); scx_dump_event(s, &events, SCX_EV_DISPATCH_KEEP_LAST); scx_dump_event(s, &events, SCX_EV_ENQ_SKIP_EXITING); + scx_dump_event(s, &events, SCX_EV_BYPASS_DURATION); scx_dump_event(s, &events, SCX_EV_BYPASS_DISPATCH); scx_dump_event(s, &events, SCX_EV_BYPASS_ACTIVATE); =20 @@ -7156,6 +7171,7 @@ __bpf_kfunc void scx_bpf_events(struct scx_event_stat= s *events, scx_agg_event(&e_sys, e_cpu, SCX_EV_SELECT_CPU_FALLBACK); scx_agg_event(&e_sys, e_cpu, SCX_EV_DISPATCH_LOCAL_DSQ_OFFLINE); scx_agg_event(&e_sys, e_cpu, SCX_EV_DISPATCH_KEEP_LAST); + scx_agg_event(&e_sys, e_cpu, SCX_EV_BYPASS_DURATION); scx_agg_event(&e_sys, e_cpu, SCX_EV_BYPASS_DISPATCH); scx_agg_event(&e_sys, e_cpu, SCX_EV_BYPASS_ACTIVATE); } --=20 2.48.1 From nobody Sun Feb 8 22:08:31 2026 Received: from fanzine2.igalia.com (fanzine.igalia.com [178.60.130.6]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id EC01B1B0F36 for ; Fri, 31 Jan 2025 07:10:32 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=178.60.130.6 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1738307434; cv=none; b=AFBPQmEGWZReT/N6fSN3rIygkRxZDcGMuRjcp/gBHP1CEQKjZ89ldGCruJdNkuDZYBaLNOr1P5F8vT5QgS804dzAJATbBi2xH5zwKdmvyCpIlGbyCB/LJNhMODpjVWFXVHznUciWKBeoqORdAFJcOAE8gUJC2p/956/RTrtjhgc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1738307434; c=relaxed/simple; bh=aDGAYJA1+sGeb7oAD54GNtzvb6oRlMYsAZN8uyPoh7g=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=unS+SsbOgqrWWY9M0Q79oDmv5lC5cUm+rPz4Pgn7HQCrWtfwXyrB7duOeUAq2w306lZO+duVp3fXvTMm1i2msPPdQBTLgVZH3Dnqmkz16w9pPmAwr8AJ84gbptWSmxiKLgOGK4ybJHtse4AZP9c+dwF1E2RQF6TE1eT0Nbc7SGs= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=igalia.com; spf=pass smtp.mailfrom=igalia.com; dkim=pass (2048-bit key) header.d=igalia.com header.i=@igalia.com header.b=hqYFnY0J; arc=none smtp.client-ip=178.60.130.6 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=igalia.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=igalia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=igalia.com header.i=@igalia.com header.b="hqYFnY0J" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=igalia.com; s=20170329; h=Content-Transfer-Encoding:MIME-Version:References:In-Reply-To: Message-ID:Date:Subject:Cc:To:From:Sender:Reply-To:Content-Type:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Id:List-Help:List-Unsubscribe:List-Subscribe: List-Post:List-Owner:List-Archive; bh=6wmy9wiY+BIcaE6mz6azlkKLSB01DI1uDlstSmP/ppg=; b=hqYFnY0JCXUFl1YVGwrgemi+F8 xN+YYzwTz0bvJIPaVZcYHJDu9645lquekmbUeTMFPxdXKXETCTRxQC0yIMc8hp+lehmFaqti1GKsz X5ERA30wg+0AHebu7JNPPjo9O3XuebYXBXbY0bnBWXdLujdjWWMJ3tm2WP1kvvwIsc62LIJv7rTt6 VUVJcAidexOnJxGjuru58mHMPfsZpfBSj4T39W9EElD45i/cGJFFnxWxZvKrZpVM5eVjZezwboD0h EJ/UM/1RUOXNfAiEBYeEdNFgSLBW53oCM+Ene3GwSaxJN27JTAtgEztSZXnuGg1iWdgK7qbzJNxnN zfoXcivA==; Received: from [58.29.143.236] (helo=localhost) by fanzine2.igalia.com with utf8esmtpsa (Cipher TLS1.3:ECDHE_SECP256R1__RSA_PSS_RSAE_SHA256__AES_256_GCM:256) (Exim) id 1tdlAS-001KaU-Oy; Fri, 31 Jan 2025 08:10:27 +0100 From: Changwoo Min To: tj@kernel.org, void@manifault.com, arighi@nvidia.com Cc: kernel-dev@igalia.com, linux-kernel@vger.kernel.org, Changwoo Min Subject: [PATCH v3 09/11] sched_ext: Add scx_bpf_events() and scx_read_event() for BPF schedulers Date: Fri, 31 Jan 2025 16:09:36 +0900 Message-ID: <20250131070938.95551-10-changwoo@igalia.com> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20250131070938.95551-1-changwoo@igalia.com> References: <20250131070938.95551-1-changwoo@igalia.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" scx_bpf_events() is added to the header files so the BPF scheduler can use it. Also, scx_read_event() is added to read an event type in a compatible way. Signed-off-by: Changwoo Min --- tools/sched_ext/include/scx/common.bpf.h | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/tools/sched_ext/include/scx/common.bpf.h b/tools/sched_ext/inc= lude/scx/common.bpf.h index f254a39b86a5..705540003024 100644 --- a/tools/sched_ext/include/scx/common.bpf.h +++ b/tools/sched_ext/include/scx/common.bpf.h @@ -78,6 +78,10 @@ struct rq *scx_bpf_cpu_rq(s32 cpu) __ksym; struct cgroup *scx_bpf_task_cgroup(struct task_struct *p) __ksym __weak; u64 scx_bpf_now(void) __ksym __weak; =20 +void scx_bpf_events(struct scx_event_stats *events, size_t events__sz) __k= sym __weak; +#define scx_read_event(e, name) \ + (bpf_core_field_exists((e)->name) ? (e)->name : 0) + /* * Use the following as @it__iter when calling scx_bpf_dsq_move[_vtime]() = from * within bpf_for_each() loops. --=20 2.48.1 From nobody Sun Feb 8 22:08:31 2026 Received: from fanzine2.igalia.com (fanzine.igalia.com [178.60.130.6]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id EB8001B0408 for ; Fri, 31 Jan 2025 07:10:36 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=178.60.130.6 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1738307438; cv=none; b=ARvlrUsVsPkQcYjFenRMW8aZJRamQtRP62qLfcaTV0Bcw4mxvjoPcSIPJlrAVjC16zK07ZWxiqhju0Xc3AjnBO9mE4Xc8cP7l0aXlqm7pHaCgJsJhCFiGwsge6aZVBwraG1A33NObYRHoi2CH6KfFSiMwOr+aPLmGzzlT3R+Cr8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1738307438; c=relaxed/simple; bh=EFRN06YrP7HFKEdVqjzP/kCbiwK4F1EK5Ov4W3EfMrc=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=C+71JdXUr7cilUc0BNxCNsygtAdJE2brRAKEvWH8dMJXEbtO06j1+r8/nMvrzguZK2yT9jJ2pgo4KygHx/S3fxGBVBYJje6ri5g7k6esXeJOCLaIRPGKRKgGboTQvI3M+WJUzJU+52oOzAOiKD64+UvwvmAO+mCkMyPvvTuvsgA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=igalia.com; spf=pass smtp.mailfrom=igalia.com; dkim=pass (2048-bit key) header.d=igalia.com header.i=@igalia.com header.b=pfIltwQl; arc=none smtp.client-ip=178.60.130.6 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=igalia.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=igalia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=igalia.com header.i=@igalia.com header.b="pfIltwQl" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=igalia.com; s=20170329; h=Content-Transfer-Encoding:MIME-Version:References:In-Reply-To: Message-ID:Date:Subject:Cc:To:From:Sender:Reply-To:Content-Type:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Id:List-Help:List-Unsubscribe:List-Subscribe: List-Post:List-Owner:List-Archive; bh=BBLeT2PN/FG2eFSj93ESZQLZAda1GTvmFqlnUkA5aUU=; b=pfIltwQl+7OLaT/lqix49Y39oK 3p88BcpXfVY8zBUUmtAiS2cAnO8/I/azujspX7VoF5hPvNekZQYf1i8/xiJCGMMR9XJNM8MfiwZFx wmdSbtu4U7W24QlyeD6yqVxiGpvh/sJquhZi5UNZiCjkWnrtjkpFxK5tpR7tjCWjCqsqYZbZDQpgZ CdppNsKBr8ECv0i/GhqRWoSNLWXhUiv0UZxdUuuFRcrdHqgexgSuDDM5LAop3SD0ssH71vyCtNw67 pPQF5jfhvZerGmrarm4nVZIO4WKmqkjh83vuHfqVFusN6V6unhYt5QE9riqsAcT5KOw2JU+rEgFfT fplRuNCg==; Received: from [58.29.143.236] (helo=localhost) by fanzine2.igalia.com with utf8esmtpsa (Cipher TLS1.3:ECDHE_SECP256R1__RSA_PSS_RSAE_SHA256__AES_256_GCM:256) (Exim) id 1tdlAW-001Kad-SL; Fri, 31 Jan 2025 08:10:31 +0100 From: Changwoo Min To: tj@kernel.org, void@manifault.com, arighi@nvidia.com Cc: kernel-dev@igalia.com, linux-kernel@vger.kernel.org, Changwoo Min Subject: [PATCH v3 10/11] sched_ext: Print core event count in scx_central scheduler Date: Fri, 31 Jan 2025 16:09:37 +0900 Message-ID: <20250131070938.95551-11-changwoo@igalia.com> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20250131070938.95551-1-changwoo@igalia.com> References: <20250131070938.95551-1-changwoo@igalia.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Modify the scx_central scheduler to print the core event counter every second. Signed-off-by: Changwoo Min --- tools/sched_ext/scx_central.bpf.c | 21 +++++++++++++++++++++ 1 file changed, 21 insertions(+) diff --git a/tools/sched_ext/scx_central.bpf.c b/tools/sched_ext/scx_centra= l.bpf.c index 50bc1737c167..376c14d5dd0d 100644 --- a/tools/sched_ext/scx_central.bpf.c +++ b/tools/sched_ext/scx_central.bpf.c @@ -256,6 +256,7 @@ static int central_timerfn(void *map, int *key, struct = bpf_timer *timer) u64 now =3D scx_bpf_now(); u64 nr_to_kick =3D nr_queued; s32 i, curr_cpu; + struct scx_event_stats events; =20 curr_cpu =3D bpf_get_smp_processor_id(); if (timer_pinned && (curr_cpu !=3D central_cpu)) { @@ -291,6 +292,26 @@ static int central_timerfn(void *map, int *key, struct= bpf_timer *timer) =20 bpf_timer_start(timer, TIMER_INTERVAL_NS, BPF_F_TIMER_CPU_PIN); __sync_fetch_and_add(&nr_timers, 1); + + /* print event counters every second */ + if (nr_timers % 1000 =3D=3D 0) { + scx_bpf_events(&events, sizeof(events)); + + bpf_printk("%35s: %llu\n", "SCX_EV_SELECT_CPU_FALLBACK", + scx_read_event(&events, SCX_EV_SELECT_CPU_FALLBACK)); + bpf_printk("%35s: %llu\n", "SCX_EV_DISPATCH_LOCAL_DSQ_OFFLINE", + scx_read_event(&events, SCX_EV_DISPATCH_LOCAL_DSQ_OFFLINE)); + bpf_printk("%35s: %llu\n", "SCX_EV_DISPATCH_KEEP_LAST", + scx_read_event(&events, SCX_EV_DISPATCH_KEEP_LAST)); + bpf_printk("%35s: %llu\n", "SCX_EV_ENQ_SKIP_EXITING", + scx_read_event(&events, SCX_EV_ENQ_SKIP_EXITING)); + bpf_printk("%35s: %llu\n", "SCX_EV_BYPASS_DURATION", + scx_read_event(&events, SCX_EV_BYPASS_DURATION)); + bpf_printk("%35s: %llu\n", "SCX_EV_BYPASS_DISPATCH", + scx_read_event(&events, SCX_EV_BYPASS_DISPATCH)); + bpf_printk("%35s: %llu\n", "SCX_EV_BYPASS_ACTIVATE", + scx_read_event(&events, SCX_EV_BYPASS_ACTIVATE)); + } return 0; } =20 --=20 2.48.1 From nobody Sun Feb 8 22:08:31 2026 Received: from fanzine2.igalia.com (fanzine.igalia.com [178.60.130.6]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 430CA1B424F for ; Fri, 31 Jan 2025 07:10:40 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=178.60.130.6 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1738307441; cv=none; b=g3r/8JO+Ew4A78GrivsyarTLgn+gIPlUHt5eXip55Q3l1HO+fmhzsMknfC0VVxutnhywM0noJ1Nt33W7QFZSHFDVKDk+zQgEuxLj4EU/lL3/JMqwpKO3Vlyp9DBnMmgZpbdk9ZL1P4YQVQ8BOLJHJ5ItmPWIX6mWe9x4mqwHGDQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1738307441; c=relaxed/simple; bh=8otb0KKbmGHKVolYJnJWTGAB7EZCUUlADsUx9BPsyUU=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=uu6/9D5tNdn/1xkANEzavQHtWDQXEHdZnveGdqN9K81l2XOFYN6PcoYgzStJ0p9M2aIzfoX8apK3cXVUGDI+AOxcP7yN/4EPswXf5uXQ2av64yXVEYpl5tiJ2WA2pqxwZmufVczoIINoBHCt0KG6nj96P2q5uZErLD21+MTgJM0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=igalia.com; spf=pass smtp.mailfrom=igalia.com; dkim=pass (2048-bit key) header.d=igalia.com header.i=@igalia.com header.b=fubl1sb/; arc=none smtp.client-ip=178.60.130.6 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=igalia.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=igalia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=igalia.com header.i=@igalia.com header.b="fubl1sb/" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=igalia.com; s=20170329; h=Content-Transfer-Encoding:MIME-Version:References:In-Reply-To: Message-ID:Date:Subject:Cc:To:From:Sender:Reply-To:Content-Type:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Id:List-Help:List-Unsubscribe:List-Subscribe: List-Post:List-Owner:List-Archive; bh=PeObDfboIG5spdonwor3QD5AbHj0/Es/1OS62aZus6E=; b=fubl1sb/GRKvbkzQoYS9ASjG3r NifqygzojXTxO4kCGNEu6qHif2kFYTFUpIGebYjAPuLlWnDlqZhlqJPud62xQERplbdTSos3Tj9YB 7QYN3gZtXUDV6mTQmFExTiG1w2eUtjKd/CyH4lDo5IwWjbrx69S5c8vUwm/I7e2PDByg5A1Ni2D3e pKDSKsShmvnHicb1pvCKwDTbaEw7uf0wRzSidyNJCzBKdUk43bdYU46/P7RoT7RN27sHWch9etQ98 ZlJ1Hn/bqL8tgQ4twC17nXUj9d0lcBoWmBLR0pslk6sQCDhbooKwIT2QBzXowCqq/dzl1tzn9C9RA B/Eo5Xwg==; Received: from [58.29.143.236] (helo=localhost) by fanzine2.igalia.com with utf8esmtpsa (Cipher TLS1.3:ECDHE_SECP256R1__RSA_PSS_RSAE_SHA256__AES_256_GCM:256) (Exim) id 1tdlAa-001Kak-SE; Fri, 31 Jan 2025 08:10:35 +0100 From: Changwoo Min To: tj@kernel.org, void@manifault.com, arighi@nvidia.com Cc: kernel-dev@igalia.com, linux-kernel@vger.kernel.org, Changwoo Min Subject: [PATCH v3 11/11] sched_ext: Print core event count in scx_qmap scheduler Date: Fri, 31 Jan 2025 16:09:38 +0900 Message-ID: <20250131070938.95551-12-changwoo@igalia.com> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20250131070938.95551-1-changwoo@igalia.com> References: <20250131070938.95551-1-changwoo@igalia.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Modify the scx_qmap scheduler to print the core event counter every second. Signed-off-by: Changwoo Min --- tools/sched_ext/scx_qmap.bpf.c | 19 +++++++++++++++++++ 1 file changed, 19 insertions(+) diff --git a/tools/sched_ext/scx_qmap.bpf.c b/tools/sched_ext/scx_qmap.bpf.c index 3a20bb0c014a..5edb79742e37 100644 --- a/tools/sched_ext/scx_qmap.bpf.c +++ b/tools/sched_ext/scx_qmap.bpf.c @@ -763,6 +763,8 @@ static void dump_shared_dsq(void) =20 static int monitor_timerfn(void *map, int *key, struct bpf_timer *timer) { + struct scx_event_stats events; + bpf_rcu_read_lock(); dispatch_highpri(true); bpf_rcu_read_unlock(); @@ -772,6 +774,23 @@ static int monitor_timerfn(void *map, int *key, struct= bpf_timer *timer) if (print_shared_dsq) dump_shared_dsq(); =20 + scx_bpf_events(&events, sizeof(events)); + + bpf_printk("%35s: %llu\n", "SCX_EV_SELECT_CPU_FALLBACK", + scx_read_event(&events, SCX_EV_SELECT_CPU_FALLBACK)); + bpf_printk("%35s: %llu\n", "SCX_EV_DISPATCH_LOCAL_DSQ_OFFLINE", + scx_read_event(&events, SCX_EV_DISPATCH_LOCAL_DSQ_OFFLINE)); + bpf_printk("%35s: %llu\n", "SCX_EV_DISPATCH_KEEP_LAST", + scx_read_event(&events, SCX_EV_DISPATCH_KEEP_LAST)); + bpf_printk("%35s: %llu\n", "SCX_EV_ENQ_SKIP_EXITING", + scx_read_event(&events, SCX_EV_ENQ_SKIP_EXITING)); + bpf_printk("%35s: %llu\n", "SCX_EV_BYPASS_DURATION", + scx_read_event(&events, SCX_EV_BYPASS_DURATION)); + bpf_printk("%35s: %llu\n", "SCX_EV_BYPASS_DISPATCH", + scx_read_event(&events, SCX_EV_BYPASS_DISPATCH)); + bpf_printk("%35s: %llu\n", "SCX_EV_BYPASS_ACTIVATE", + scx_read_event(&events, SCX_EV_BYPASS_ACTIVATE)); + bpf_timer_start(timer, ONE_SEC_IN_NS, 0); return 0; } --=20 2.48.1