From nobody Mon Feb 9 03:34:12 2026 Received: from fanzine2.igalia.com (fanzine.igalia.com [178.60.130.6]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C565E7405A for ; Sun, 26 Jan 2025 10:16:38 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=178.60.130.6 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1737886600; cv=none; b=UfilpDOQGObt91X6G5diqLsmkhyCT7oz6AzwkRK/dPL6eWGLvNZaFWpE7/j6S6ZUOjdPQgwOGsVl+t+BFj8mtBcdTk44/v5T65UTAO1JXOpLS5Z9/wSltXcuzmYS6p/8/SW2ziAy7zcbMyqh5gtdYLDf1CcAvlPOPIu4pmvRVto= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1737886600; c=relaxed/simple; bh=7cO7v9VrBt8Q/1B9c8G2VaIb6Yut8ZPqx3F4+UAgkUs=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=eIFkU2jIobREo1ymd4ndWQyAXVVYfu46l0yoyLuicXtXEd9gtzUl4j0hIndvamfdwwbWOAe5fKykq145z/LOt94xGAb4dRjPPfRxcXnqLy/wIR8n/TfpRl+ne7ZCyDtbh4GrlDtzgri0pR9LhiuTWN38DQU5O69pBIM7QyS2bDQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=igalia.com; spf=pass smtp.mailfrom=igalia.com; dkim=pass (2048-bit key) header.d=igalia.com header.i=@igalia.com header.b=BGwYiR80; arc=none smtp.client-ip=178.60.130.6 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=igalia.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=igalia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=igalia.com header.i=@igalia.com header.b="BGwYiR80" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=igalia.com; s=20170329; h=Content-Transfer-Encoding:MIME-Version:References:In-Reply-To: Message-ID:Date:Subject:Cc:To:From:Sender:Reply-To:Content-Type:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Id:List-Help:List-Unsubscribe:List-Subscribe: List-Post:List-Owner:List-Archive; bh=NQWsEzzUQu0YzWLi7+L4fktgRz0hV9Wyj3yXYf30BP0=; b=BGwYiR80nPHUXi15E5tfF9kNwO RCq2MVbCEIAyTU6A3SriyVNIMxpGxTSAV+q/5EaG5zjoW58VXgEzl6l1PLOEOTaJkBYNvO0BtJofO C0C52J0hHU0eZhvy8u5vlKTeXJGI6weIiankATACUkubIlEqH70FM0wLYxJ7SVVGLGxXq7sO4gnKV jHyYvHNbp1w32W1an8bmaj6o+iwOO7uwWCTFd8WFbPCjwKx5/FyKEu58SzzXXr8LZkBRI1jh/conn /IPygp/s4OAJKZwSA/HXPO3mVWHBRYjRlQpptu5LKdlnT7/OwZp3x6fw2VwDRoC/oo3b73HvG/DUL G9Trnc1g==; Received: from [58.29.143.236] (helo=localhost) by fanzine2.igalia.com with utf8esmtpsa (Cipher TLS1.3:ECDHE_SECP256R1__RSA_PSS_RSAE_SHA256__AES_256_GCM:256) (Exim) id 1tbzgm-002liq-2f; Sun, 26 Jan 2025 11:16:24 +0100 From: Changwoo Min To: tj@kernel.org, void@manifault.com, arighi@nvidia.com Cc: kernel-dev@igalia.com, linux-kernel@vger.kernel.org, Changwoo Min Subject: [PATCH v2 01/11] sched_ext: Implement event counter infrastructure Date: Sun, 26 Jan 2025 19:16:04 +0900 Message-ID: <20250126101614.232388-2-changwoo@igalia.com> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20250126101614.232388-1-changwoo@igalia.com> References: <20250126101614.232388-1-changwoo@igalia.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Collect the statistics of specific types of behavior in the sched_ext core, which are not easily visible but still interesting to an scx scheduler. An event type is defined in 'struct scx_event_stats.' When an event occurs, its counter is accumulated using 'scx_add_event()' and '__scx_add_event()' to per-CPU 'struct scx_event_stats' for efficiency. 'scx_bpf_event_stats()' aggregates all the per-CPU counters and exposes a system-wide counters. For convenience and readability of the code, 'scx_agg_event()' and 'scx_dump_event()' are provided. The collected events can be observed after a BPF scheduler is unloaded beforea new BPF scheduler is loaded so the per-CPU 'struct scx_event_stats' are reset. Signed-off-by: Changwoo Min --- kernel/sched/ext.c | 103 +++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 103 insertions(+) diff --git a/kernel/sched/ext.c b/kernel/sched/ext.c index 7081c7be5f62..dbf659d593a3 100644 --- a/kernel/sched/ext.c +++ b/kernel/sched/ext.c @@ -1458,6 +1458,64 @@ static struct task_struct *scx_task_iter_next_locked= (struct scx_task_iter *iter) return p; } =20 +/* + * Collection of event counters. Event types are placed in descending orde= r. + */ +struct scx_event_stats { +}; + +/* + * The event counter is organized by a per-CPU variable to minimize the + * accounting overhead without synchronization. A system-wide view on the + * event counter is constructed when requested by scx_bpf_get_event_stat(). + */ +static DEFINE_PER_CPU(struct scx_event_stats, event_stats_cpu); + +/** + * scx_add_event - Increase an event counter for 'name' by 'cnt' + * @name: an event name defined in struct scx_event_stats + * @cnt: the number of the event occured + * + * This can be used when preemption is not disabled. + */ +#define scx_add_event(name, cnt) do { \ + this_cpu_add(event_stats_cpu.name, cnt); \ +} while(0) + +/** + * __scx_add_event - Increase an event counter for 'name' by 'cnt' + * @name: an event name defined in struct scx_event_stats + * @cnt: the number of the event occured + * + * This should be used only when preemption is disabled. + */ +#define __scx_add_event(name, cnt) do { \ + __this_cpu_add(event_stats_cpu.name, cnt); \ +} while(0) + +/** + * scx_agg_event - Aggregate an event counter 'kind' from 'src_e' to 'dst_= e' + * @dst_e: destination event stats + * @src_e: source event stats + * @kind: a kind of event to be aggregated + */ +#define scx_agg_event(dst_e, src_e, kind) do { \ + (dst_e)->kind +=3D READ_ONCE((src_e)->kind); \ +} while(0) + +/** + * scx_dump_event - Dump an event 'kind' in 'events' to 's' + * @s: output seq_buf + * @events: event stats + * @kind: a kind of event to dump + */ +#define scx_dump_event(s, events, kind) do { \ + dump_line(&(s), "%30s: %16llu", #kind, (events)->kind); \ +} while (0) + + +static void scx_bpf_event_stats(struct scx_event_stats *events, size_t eve= nts__sz); + static enum scx_ops_enable_state scx_ops_enable_state(void) { return atomic_read(&scx_ops_enable_state_var); @@ -5310,6 +5368,7 @@ static void scx_dump_state(struct scx_exit_info *ei, = size_t dump_len) .at_jiffies =3D jiffies, }; struct seq_buf s; + struct scx_event_stats events; unsigned long flags; char *buf; int cpu; @@ -5418,6 +5477,12 @@ static void scx_dump_state(struct scx_exit_info *ei,= size_t dump_len) rq_unlock(rq, &rf); } =20 + dump_newline(&s); + dump_line(&s, "Event counters"); + dump_line(&s, "--------------"); + + scx_bpf_event_stats(&events, sizeof(events)); + if (seq_buf_has_overflowed(&s) && dump_len >=3D sizeof(trunc_marker)) memcpy(ei->dump + dump_len - sizeof(trunc_marker), trunc_marker, sizeof(trunc_marker)); @@ -5525,6 +5590,15 @@ static int scx_ops_enable(struct sched_ext_ops *ops,= struct bpf_link *link) =20 mutex_lock(&scx_ops_enable_mutex); =20 + /* + * Clear event counters so a new scx scheduler gets + * fresh event counter values. + */ + for_each_possible_cpu(cpu) { + struct scx_event_stats *e =3D per_cpu_ptr(&event_stats_cpu, cpu); + memset(e, 0, sizeof(*e)); + } + if (!scx_ops_helper) { WRITE_ONCE(scx_ops_helper, scx_create_rt_helper("sched_ext_ops_helper")); @@ -7721,6 +7795,34 @@ __bpf_kfunc u64 scx_bpf_now(void) return clock; } =20 +/* + * scx_bpf_event_stats - Get a system-wide event counter to + * @events: output buffer from a BPF program + * @events__sz: @events len, must end in '__sz'' for the verifier + */ +__bpf_kfunc void scx_bpf_event_stats(struct scx_event_stats *events, + size_t events__sz) +{ + struct scx_event_stats e_sys, *e_cpu; + int cpu; + + /* Aggregate per-CPU event counters into the system-wide counters. */ + memset(&e_sys, 0, sizeof(e_sys)); + for_each_possible_cpu(cpu) { + e_cpu =3D per_cpu_ptr(&event_stats_cpu, cpu); + } + + /* + * We cannot entirely trust a BPF-provided size since a BPF program + * might be compiled against a different vmlinux.h, of which + * scx_event_stats would be larger (a newer vmlinux.h) or smaller + * (an older vmlinux.h). Hence, we use the smaller size to avoid + * memory corruption. + */ + events__sz =3D min(events__sz, sizeof(*events)); + memcpy(events, &e_sys, events__sz); +} + __bpf_kfunc_end_defs(); =20 BTF_KFUNCS_START(scx_kfunc_ids_any) @@ -7753,6 +7855,7 @@ BTF_ID_FLAGS(func, scx_bpf_cpu_rq) BTF_ID_FLAGS(func, scx_bpf_task_cgroup, KF_RCU | KF_ACQUIRE) #endif BTF_ID_FLAGS(func, scx_bpf_now) +BTF_ID_FLAGS(func, scx_bpf_event_stats, KF_TRUSTED_ARGS) BTF_KFUNCS_END(scx_kfunc_ids_any) =20 static const struct btf_kfunc_id_set scx_kfunc_set_any =3D { --=20 2.48.1 From nobody Mon Feb 9 03:34:12 2026 Received: from fanzine2.igalia.com (fanzine.igalia.com [178.60.130.6]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7B52E154C08 for ; Sun, 26 Jan 2025 10:16:40 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=178.60.130.6 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1737886602; cv=none; b=n1r9WZ7CExC81mpnncGaKBcsVJ1q1lGNJ+nTvaut3DADKxkf5pbTr7U6/rSAmTBJKrkVT2gmnb+jYacFCv+KBstyXkyTS/Ttw39Zs6IHMZB943JQvqIEON0X49bdQXjBeg4Cc4GMBNcSjvNRKrArNRFtXql829BI7i3k7kH9g6c= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1737886602; c=relaxed/simple; bh=uL/HlqVUlbYGnkpKOGPui3ph5x4rrtkI8EiNtGBqRTI=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=Ji9Fxa2/6gf8P5R/uXq9OKMvXynxmEUoSiRa8Kq5Jy+x0ybShlBHOa+NgzVVoHlvkZgcdwoaVVYN7/E48IoF/+Y1nTRNFxkatSKMBBz9o1OX4UyLzpfn5use8xl1/e9ybAmejLqlYu6JPOE3GBpvvIdaIXINx6EFpOJmP1T4cxw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=igalia.com; spf=pass smtp.mailfrom=igalia.com; dkim=pass (2048-bit key) header.d=igalia.com header.i=@igalia.com header.b=U5WM6Ivj; arc=none smtp.client-ip=178.60.130.6 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=igalia.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=igalia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=igalia.com header.i=@igalia.com header.b="U5WM6Ivj" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=igalia.com; s=20170329; h=Content-Transfer-Encoding:MIME-Version:References:In-Reply-To: Message-ID:Date:Subject:Cc:To:From:Sender:Reply-To:Content-Type:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Id:List-Help:List-Unsubscribe:List-Subscribe: List-Post:List-Owner:List-Archive; bh=8xwt3cnHOZGh9d64Z5XegGyfq5lcdIzONy9hfckbFns=; b=U5WM6IvjY2/HG2U1D4oCJ00na7 SYlFXd/W1LDlYu7hVo3nzASun0exEaWmlYIP4FjPZhHRD/N9dk/XrSRgXqadUlZ2yccoWngOfEuid 73+FgnQFuLCFrTjVLXHx9I88YNDiDq69ZRgaO7XM2lJF2YBeM+idHHwjo8mrSnkKBvq82ou4y1vHo v9DfSNBfV2mOniO2g6Rebmh4XLm4WXzomDUNzDhuZJJM3dyrjjiPmU/9XDBUGs3wtEPJon24VYxdk 1fKnHADV0dAmHgfhdm1qouMhTJ9Y5mLiiypfhSfEqZD7Z73gMM4rEN5TUQqHGrd5HWyX8BpkRioPG V+ZV3jgw==; Received: from [58.29.143.236] (helo=localhost) by fanzine2.igalia.com with utf8esmtpsa (Cipher TLS1.3:ECDHE_SECP256R1__RSA_PSS_RSAE_SHA256__AES_256_GCM:256) (Exim) id 1tbzgp-002liy-TK; Sun, 26 Jan 2025 11:16:28 +0100 From: Changwoo Min To: tj@kernel.org, void@manifault.com, arighi@nvidia.com Cc: kernel-dev@igalia.com, linux-kernel@vger.kernel.org, Changwoo Min Subject: [PATCH v2 02/11] sched: Move is_cpu_allowed() to the header Date: Sun, 26 Jan 2025 19:16:05 +0900 Message-ID: <20250126101614.232388-3-changwoo@igalia.com> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20250126101614.232388-1-changwoo@igalia.com> References: <20250126101614.232388-1-changwoo@igalia.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" is_cpu_allowed() can be used to specific scheduling policies (e.g., sched_ext), so move it to the header. Signed-off-by: Changwoo Min --- kernel/sched/core.c | 30 ------------------------------ kernel/sched/sched.h | 30 ++++++++++++++++++++++++++++++ 2 files changed, 30 insertions(+), 30 deletions(-) diff --git a/kernel/sched/core.c b/kernel/sched/core.c index 901170708e2a..eaaee182999c 100644 --- a/kernel/sched/core.c +++ b/kernel/sched/core.c @@ -2427,36 +2427,6 @@ static inline bool rq_has_pinned_tasks(struct rq *rq) return rq->nr_pinned; } =20 -/* - * Per-CPU kthreads are allowed to run on !active && online CPUs, see - * __set_cpus_allowed_ptr() and select_fallback_rq(). - */ -static inline bool is_cpu_allowed(struct task_struct *p, int cpu) -{ - /* When not in the task's cpumask, no point in looking further. */ - if (!task_allowed_on_cpu(p, cpu)) - return false; - - /* migrate_disabled() must be allowed to finish. */ - if (is_migration_disabled(p)) - return cpu_online(cpu); - - /* Non kernel threads are not allowed during either online or offline. */ - if (!(p->flags & PF_KTHREAD)) - return cpu_active(cpu); - - /* KTHREAD_IS_PER_CPU is always allowed. */ - if (kthread_is_per_cpu(p)) - return cpu_online(cpu); - - /* Regular kernel threads don't get to stay during offline. */ - if (cpu_dying(cpu)) - return false; - - /* But are allowed during online. */ - return cpu_online(cpu); -} - /* * This is how migration works: * diff --git a/kernel/sched/sched.h b/kernel/sched/sched.h index 38e0e323dda2..30c401d940f4 100644 --- a/kernel/sched/sched.h +++ b/kernel/sched/sched.h @@ -2675,6 +2675,36 @@ static inline cpumask_t *alloc_user_cpus_ptr(int nod= e) =20 #endif /* !CONFIG_SMP */ =20 +/* + * Per-CPU kthreads are allowed to run on !active && online CPUs, see + * __set_cpus_allowed_ptr() and select_fallback_rq(). + */ +static inline bool is_cpu_allowed(struct task_struct *p, int cpu) +{ + /* When not in the task's cpumask, no point in looking further. */ + if (!task_allowed_on_cpu(p, cpu)) + return false; + + /* migrate_disabled() must be allowed to finish. */ + if (is_migration_disabled(p)) + return cpu_online(cpu); + + /* Non kernel threads are not allowed during either online or offline. */ + if (!(p->flags & PF_KTHREAD)) + return cpu_active(cpu); + + /* KTHREAD_IS_PER_CPU is always allowed. */ + if (kthread_is_per_cpu(p)) + return cpu_online(cpu); + + /* Regular kernel threads don't get to stay during offline. */ + if (cpu_dying(cpu)) + return false; + + /* But are allowed during online. */ + return cpu_online(cpu); +} + #ifdef CONFIG_CPU_IDLE =20 static inline void idle_set_state(struct rq *rq, --=20 2.48.1 From nobody Mon Feb 9 03:34:12 2026 Received: from fanzine2.igalia.com (fanzine.igalia.com [178.60.130.6]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2B95A42056 for ; Sun, 26 Jan 2025 10:16:37 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=178.60.130.6 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1737886599; cv=none; b=IuVksndBFZFi13MhqS1/IMGFXyKwe2zDduTdYXoAXWr9/e589GvczYX6IpebFVZ/5WLycRbX7TFa7uD2vBhAqnOi4Eewjg6LwpG4QCScxl8d1sei1aGn9V9Eih+mJiOjD7DFcqDxo6NEaijpJLI2EeWOq9tQrga6T5m8s2kSQes= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1737886599; c=relaxed/simple; bh=hRytXuPdZ4eY/Vi7LG+Z66mcYe0k8R44dTBELteGLmQ=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=Nl11qniGp4UFK5zcTXUN0c6G8xKkV+R+Ik8FIF267OSqwIhWrbV9Z+gY6ilHej3OAgpTOgEklt7GIlK6khBCNfIONYsWuZHo9nZpm51C/iIC5KXVnGqwnf7VfCTDTgWSFqNTEVNpkZE40QBZGxV7e6nlpPYxzHOcjOlYEKOw9EU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=igalia.com; spf=pass smtp.mailfrom=igalia.com; dkim=pass (2048-bit key) header.d=igalia.com header.i=@igalia.com header.b=Qk367XkY; arc=none smtp.client-ip=178.60.130.6 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=igalia.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=igalia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=igalia.com header.i=@igalia.com header.b="Qk367XkY" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=igalia.com; s=20170329; h=Content-Transfer-Encoding:MIME-Version:References:In-Reply-To: Message-ID:Date:Subject:Cc:To:From:Sender:Reply-To:Content-Type:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Id:List-Help:List-Unsubscribe:List-Subscribe: List-Post:List-Owner:List-Archive; bh=sPqJ9e0DzmFdhShg1je6pjpQv6coY3BIeaACfDk8MII=; b=Qk367XkYQD0IIKIdWj4zITtX1B z7Xvn4chig6AN0UrdAyt0PT1AAZKa1JqNz5kxMepKsU0F+XmXMXBIVmoj0EbnJCen5NnRUjZKQSB3 e/Gc/jpKildpNK3neDp5VdS9/7x6VJTJzcNIkrkHbV6jDtyVZt6CLnS+JHQBRjGiihWwjbab5IDyQ kUH5dY/vcjdHjCGNJRbk6bnzSG5MReuQ7J05GzLxMdqg2vvHjTTz1+7r7HlUtZi5T6IYxmrJFz/Ys 5E75oSYYrktM+LpFLIH3D+1Dfh//2G+HGb8kkGRGA6nN0slS5jvJuYtRCfeFwug3wKTxiSA007Wjr wnQWhS1w==; Received: from [58.29.143.236] (helo=localhost) by fanzine2.igalia.com with utf8esmtpsa (Cipher TLS1.3:ECDHE_SECP256R1__RSA_PSS_RSAE_SHA256__AES_256_GCM:256) (Exim) id 1tbzgt-002lj8-QT; Sun, 26 Jan 2025 11:16:32 +0100 From: Changwoo Min To: tj@kernel.org, void@manifault.com, arighi@nvidia.com Cc: kernel-dev@igalia.com, linux-kernel@vger.kernel.org, Changwoo Min Subject: [PATCH v2 03/11] sched_ext: Add an event, SELECT_CPU_FALLBACK Date: Sun, 26 Jan 2025 19:16:06 +0900 Message-ID: <20250126101614.232388-4-changwoo@igalia.com> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20250126101614.232388-1-changwoo@igalia.com> References: <20250126101614.232388-1-changwoo@igalia.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Add a core event, SELECT_CPU_FALLBACK, which represents how many times ops.select_cpu() returns a CPU that the task can't use. __scx_add_event() is used since the caller holds a p->pi_lock, so the preemption has already been disabled. Signed-off-by: Changwoo Min --- kernel/sched/ext.c | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/kernel/sched/ext.c b/kernel/sched/ext.c index dbf659d593a3..727b1f8b623e 100644 --- a/kernel/sched/ext.c +++ b/kernel/sched/ext.c @@ -1462,6 +1462,11 @@ static struct task_struct *scx_task_iter_next_locked= (struct scx_task_iter *iter) * Collection of event counters. Event types are placed in descending orde= r. */ struct scx_event_stats { + /* + * If ops.select_cpu() returns a CPU which can't be used by the task, + * the core scheduler code silently picks a fallback CPU. + */ + u64 SELECT_CPU_FALLBACK; }; =20 /* @@ -3663,6 +3668,10 @@ static int select_task_rq_scx(struct task_struct *p,= int prev_cpu, int wake_flag cpu =3D SCX_CALL_OP_TASK_RET(SCX_KF_ENQUEUE | SCX_KF_SELECT_CPU, select_cpu, p, prev_cpu, wake_flags); *ddsp_taskp =3D NULL; + + if (unlikely(!is_cpu_allowed(p, cpu))) + __scx_add_event(SELECT_CPU_FALLBACK, 1); + if (ops_cpu_valid(cpu, "from ops.select_cpu()")) return cpu; else @@ -5482,6 +5491,7 @@ static void scx_dump_state(struct scx_exit_info *ei, = size_t dump_len) dump_line(&s, "--------------"); =20 scx_bpf_event_stats(&events, sizeof(events)); + scx_dump_event(s, &events, SELECT_CPU_FALLBACK); =20 if (seq_buf_has_overflowed(&s) && dump_len >=3D sizeof(trunc_marker)) memcpy(ei->dump + dump_len - sizeof(trunc_marker), @@ -7810,6 +7820,7 @@ __bpf_kfunc void scx_bpf_event_stats(struct scx_event= _stats *events, memset(&e_sys, 0, sizeof(e_sys)); for_each_possible_cpu(cpu) { e_cpu =3D per_cpu_ptr(&event_stats_cpu, cpu); + scx_agg_event(&e_sys, e_cpu, SELECT_CPU_FALLBACK); } =20 /* --=20 2.48.1 From nobody Mon Feb 9 03:34:12 2026 Received: from fanzine2.igalia.com (fanzine.igalia.com [178.60.130.6]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 69939157493 for ; Sun, 26 Jan 2025 10:16:42 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=178.60.130.6 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1737886603; cv=none; b=GweMYjdpu8M/Q1K7g5vi4LTanYM4MUlLACBV0f13QbYzEn+U+/RBI9liTy5HxPb7AOqZbWkifvsFQigW/v8+qOrv05W21zWV9rz2NyJu5CWUJ7EXfS4bNccc5fVore7rNT2YlYoWfCLeMxsaft8dvUlQdu7NB23N2dRw8fLWK2w= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1737886603; c=relaxed/simple; bh=z0Bk4o0ovuqfLIHT1hnnzGweAJnfYHLhacDV5JwgOLo=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=TBucCghIDmFhrJoyIFU6oweaQMaimpNBAHImENcPxjZiOPo0UIopcc8M2z8f79zQwBYZf7cQA3UqZOQNQm9aXQtZ95H77QZwP15vDIgfhWohLwIFM5NBPJ1bzsiodTFNf3q6u19JYCgOK672J+9ru1jWVRKe+kA1rc3fzJwsD/Q= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=igalia.com; spf=pass smtp.mailfrom=igalia.com; dkim=pass (2048-bit key) header.d=igalia.com header.i=@igalia.com header.b=S9TPPNqJ; arc=none smtp.client-ip=178.60.130.6 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=igalia.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=igalia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=igalia.com header.i=@igalia.com header.b="S9TPPNqJ" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=igalia.com; s=20170329; h=Content-Transfer-Encoding:MIME-Version:References:In-Reply-To: Message-ID:Date:Subject:Cc:To:From:Sender:Reply-To:Content-Type:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Id:List-Help:List-Unsubscribe:List-Subscribe: List-Post:List-Owner:List-Archive; bh=yr033ggJ5eKQWklOiqZgHxlCuDTOp6YLDA0oE6TY4PI=; b=S9TPPNqJ3au/zHCuBqQrgqf//K tyE7N7ZUm2bC4nVZ0wsA1nO5pMiQQkPJuHbv5683RVKOdijT1hNaaYsUcVzgX/ond9Y44yDQ5FAz+ 5La9wj5DkA6qnf+2NrU3CBjn3F6ttOWpVwna2EvfirB2+9zWaffYQV1MZi+qMJS/nwkxZc/jduIGT 2tcy98A5uPZ2XUEC6NuS5VSCZBfcaFU8ecGsM+9yvNHhhZw/sy9Iu73NGrzkuEWYW38lFtEovOKIs 6KNipLILE3IfreETssl4ui8yFwRMGZapH2ynlJbLgK8zQQlQCjaGEaf7ZH7P3ilOyzsJOYPDtOKo/ JeeP5FNA==; Received: from [58.29.143.236] (helo=localhost) by fanzine2.igalia.com with utf8esmtpsa (Cipher TLS1.3:ECDHE_SECP256R1__RSA_PSS_RSAE_SHA256__AES_256_GCM:256) (Exim) id 1tbzgx-002ljG-O4; Sun, 26 Jan 2025 11:16:36 +0100 From: Changwoo Min To: tj@kernel.org, void@manifault.com, arighi@nvidia.com Cc: kernel-dev@igalia.com, linux-kernel@vger.kernel.org, Changwoo Min Subject: [PATCH v2 04/11] sched_ext: Add an event, DISPATCH_LOCAL_DSQ_OFFLINE Date: Sun, 26 Jan 2025 19:16:07 +0900 Message-ID: <20250126101614.232388-5-changwoo@igalia.com> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20250126101614.232388-1-changwoo@igalia.com> References: <20250126101614.232388-1-changwoo@igalia.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Add a core event, DISPATCH_LOCAL_DSQ_OFFLINE, which represents how many times a BPF scheduler tries to dispatch to an offlined local DSQ. __scx_add_event() is used since the caller holds an rq lock, so the preemption has already been disabled. Signed-off-by: Changwoo Min --- kernel/sched/ext.c | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/kernel/sched/ext.c b/kernel/sched/ext.c index 727b1f8b623e..38cf7ea6c385 100644 --- a/kernel/sched/ext.c +++ b/kernel/sched/ext.c @@ -1467,6 +1467,12 @@ struct scx_event_stats { * the core scheduler code silently picks a fallback CPU. */ u64 SELECT_CPU_FALLBACK; + + /* + * When dispatching to a local DSQ, the CPU may have gone offline in + * the meantime. In this case, the task is bounced to the global DSQ. + */ + u64 DISPATCH_LOCAL_DSQ_OFFLINE; }; =20 /* @@ -2654,6 +2660,7 @@ static void dispatch_to_local_dsq(struct rq *rq, stru= ct scx_dispatch_q *dst_dsq, if (unlikely(!task_can_run_on_remote_rq(p, dst_rq, true))) { dispatch_enqueue(find_global_dsq(p), p, enq_flags | SCX_ENQ_CLEAR_OPSS); + __scx_add_event(DISPATCH_LOCAL_DSQ_OFFLINE, 1); return; } =20 @@ -5492,6 +5499,7 @@ static void scx_dump_state(struct scx_exit_info *ei, = size_t dump_len) =20 scx_bpf_event_stats(&events, sizeof(events)); scx_dump_event(s, &events, SELECT_CPU_FALLBACK); + scx_dump_event(s, &events, DISPATCH_LOCAL_DSQ_OFFLINE); =20 if (seq_buf_has_overflowed(&s) && dump_len >=3D sizeof(trunc_marker)) memcpy(ei->dump + dump_len - sizeof(trunc_marker), @@ -7821,6 +7829,7 @@ __bpf_kfunc void scx_bpf_event_stats(struct scx_event= _stats *events, for_each_possible_cpu(cpu) { e_cpu =3D per_cpu_ptr(&event_stats_cpu, cpu); scx_agg_event(&e_sys, e_cpu, SELECT_CPU_FALLBACK); + scx_agg_event(&e_sys, e_cpu, DISPATCH_LOCAL_DSQ_OFFLINE); } =20 /* --=20 2.48.1 From nobody Mon Feb 9 03:34:12 2026 Received: from fanzine2.igalia.com (fanzine.igalia.com [178.60.130.6]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id EA0AA16F0FE for ; Sun, 26 Jan 2025 10:16:44 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=178.60.130.6 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1737886607; cv=none; b=f3lGQ1h5Xb0eJ4Gkju4q45rw5pbyHOv0dzIbjL8hOX2pbmVDWu2u9xI7cv4fMn7eOGY5xhBquHG/5FvTL6w2rZd5OLslL6KPG/rf6PHrtEFzOP6mXZMU1pz4EH+lFR09XQOlFBF0ulCTlJ+sp8QgeASBexr0lTq2LkfoVR7UrNA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1737886607; c=relaxed/simple; bh=jvn6M1H82lAPFjaam6d8ioO5412AzdXJQMOjF4pehxE=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=NM0ef4lsXhc/QYcz4QoH6ivKbGWr+mSYIq8eS/uEin4UJF0D6E02cfEyZkGm48ZoreQAwy5BpU6Yd/oJ4TpMq1wWXwJ6vCLOBQNQ8IKAxWpABku8aMrax1OTlYechEoOYOFIaK3vNQVjUX1csJvX9j9I/Q/aCo0l2ToRpzmfBsA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=igalia.com; spf=pass smtp.mailfrom=igalia.com; dkim=pass (2048-bit key) header.d=igalia.com header.i=@igalia.com header.b=LcRCtiMh; arc=none smtp.client-ip=178.60.130.6 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=igalia.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=igalia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=igalia.com header.i=@igalia.com header.b="LcRCtiMh" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=igalia.com; s=20170329; h=Content-Transfer-Encoding:MIME-Version:References:In-Reply-To: Message-ID:Date:Subject:Cc:To:From:Sender:Reply-To:Content-Type:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Id:List-Help:List-Unsubscribe:List-Subscribe: List-Post:List-Owner:List-Archive; bh=R48sgLH8zDXen4TRES0X+oz+R/agplkssVKv0K4R94g=; b=LcRCtiMh7kPGex3+1npEHk/UZt v0pm3hO1pToCgQCEiKxa3YkEqQd8qrQfxphDBfttOl2XEzWHGr2I7NFrY5UvoidwYNvK3XRMunko5 +WFg0ypynqFkA9EzcSUwEuhuwVoMKT7Oja63xMMkFscr7GbrpvwL53tsBl3wVVmSQk2QCTHeAk6Yr X+lVD4CVU4kAi4peAZSFdJ+fgrkhUBOuPebCREziVQEUoplF529i7FJDPVW+67xW0QvoFPICQWur8 HvNgACqRkTCSOkyB2ukliAq+BPT6WutgXG8YFg104rQni+kAn7fLSb4CJxqvvJIrDqA+YzqWM/rCS JQ+xnlDQ==; Received: from [58.29.143.236] (helo=localhost) by fanzine2.igalia.com with utf8esmtpsa (Cipher TLS1.3:ECDHE_SECP256R1__RSA_PSS_RSAE_SHA256__AES_256_GCM:256) (Exim) id 1tbzh1-002ljP-Jl; Sun, 26 Jan 2025 11:16:39 +0100 From: Changwoo Min To: tj@kernel.org, void@manifault.com, arighi@nvidia.com Cc: kernel-dev@igalia.com, linux-kernel@vger.kernel.org, Changwoo Min Subject: [PATCH v2 05/11] sched_ext: Add an event, DISPATCH_KEEP_LAST Date: Sun, 26 Jan 2025 19:16:08 +0900 Message-ID: <20250126101614.232388-6-changwoo@igalia.com> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20250126101614.232388-1-changwoo@igalia.com> References: <20250126101614.232388-1-changwoo@igalia.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Add a core event, DISPATCH_KEEP_LAST, which represents how many times a task is continued to run without ops.enqueue() when SCX_OPS_ENQ_LAST is not set. __scx_add_event() is used since the caller holds an rq lock, so the preemption has already been disabled. Signed-off-by: Changwoo Min --- kernel/sched/ext.c | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/kernel/sched/ext.c b/kernel/sched/ext.c index 38cf7ea6c385..d3a1ef58202c 100644 --- a/kernel/sched/ext.c +++ b/kernel/sched/ext.c @@ -1473,6 +1473,12 @@ struct scx_event_stats { * the meantime. In this case, the task is bounced to the global DSQ. */ u64 DISPATCH_LOCAL_DSQ_OFFLINE; + + /* + * If SCX_OPS_ENQ_LAST is not set, the number of times that a task + * continued to run because there were no other tasks on the CPU. + */ + u64 DISPATCH_KEEP_LAST; }; =20 /* @@ -2915,6 +2921,7 @@ static int balance_one(struct rq *rq, struct task_str= uct *prev) if (prev_on_rq && (!static_branch_unlikely(&scx_ops_enq_last) || scx_rq_bypassing(rq))) { rq->scx.flags |=3D SCX_RQ_BAL_KEEP; + __scx_add_event(DISPATCH_KEEP_LAST, 1); goto has_tasks; } rq->scx.flags &=3D ~SCX_RQ_IN_BALANCE; @@ -5500,6 +5507,7 @@ static void scx_dump_state(struct scx_exit_info *ei, = size_t dump_len) scx_bpf_event_stats(&events, sizeof(events)); scx_dump_event(s, &events, SELECT_CPU_FALLBACK); scx_dump_event(s, &events, DISPATCH_LOCAL_DSQ_OFFLINE); + scx_dump_event(s, &events, DISPATCH_KEEP_LAST); =20 if (seq_buf_has_overflowed(&s) && dump_len >=3D sizeof(trunc_marker)) memcpy(ei->dump + dump_len - sizeof(trunc_marker), @@ -7830,6 +7838,7 @@ __bpf_kfunc void scx_bpf_event_stats(struct scx_event= _stats *events, e_cpu =3D per_cpu_ptr(&event_stats_cpu, cpu); scx_agg_event(&e_sys, e_cpu, SELECT_CPU_FALLBACK); scx_agg_event(&e_sys, e_cpu, DISPATCH_LOCAL_DSQ_OFFLINE); + scx_agg_event(&e_sys, e_cpu, DISPATCH_KEEP_LAST); } =20 /* --=20 2.48.1 From nobody Mon Feb 9 03:34:12 2026 Received: from fanzine2.igalia.com (fanzine.igalia.com [178.60.130.6]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B68D5186E40 for ; Sun, 26 Jan 2025 10:16:49 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=178.60.130.6 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1737886611; cv=none; b=hpZyyJorW8OMMmfj1Z09vscpqwwVaUa7Kv4TTbnsg4whV+Lt0TX++KMBS5Ueq2FT9bfEORmRnE3CvZi942e+Ti7KKN3TeunSgnwo/HTAoXPkxIK9VEbhDHhSvohXM0mn8iLvGh4xeJn32Tb1wZ/QbQ2gB3xlAYxNdL7qL4klnmY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1737886611; c=relaxed/simple; bh=zFuS9ZcQDsAjS5HqYYoPXtxPu+UeaBqU+gSayoA/bb8=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=eMYZnvGgvngkAz0WsVWZaELkV2NjI1x6mttdMAmD3/z6tNpuOAXIs53/6Hg+a9CojZozvKDNxi3yESR8PwWz/UxeId32/rxqL9kblz1azeH+nxbHz4EEOK2RyBiCrwkRlpf9e7HjHYUfHBlP9IxgEMDT8kXcs95cB6kLLF5eSAg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=igalia.com; spf=pass smtp.mailfrom=igalia.com; dkim=pass (2048-bit key) header.d=igalia.com header.i=@igalia.com header.b=JlXhciw2; arc=none smtp.client-ip=178.60.130.6 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=igalia.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=igalia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=igalia.com header.i=@igalia.com header.b="JlXhciw2" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=igalia.com; s=20170329; h=Content-Transfer-Encoding:MIME-Version:References:In-Reply-To: Message-ID:Date:Subject:Cc:To:From:Sender:Reply-To:Content-Type:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Id:List-Help:List-Unsubscribe:List-Subscribe: List-Post:List-Owner:List-Archive; bh=MIWvX2Di+r1WJIro6JO8UvyngHbAGwuTEV1An1K7toc=; b=JlXhciw2ZX9T5S3SY3Ea1nNqyi 8uqtfHMvFCwmbir/dCjv+t3UDdYM2255FhzGoSbMCJakgA1VIyiVDWh8j6gJCFg/lG6kYCCIGHYRr O1dGbVBxF2nnHazXF8xnGug2/RlvtIEs8NTBfEwsQXsFACOVRy9bGjesZAgddL932+9vgG3e3S9ux 2EN4hvUV6IhbdemJ+eBqZlcgzC2O4QU6jcaX7YDLc8s7SxioT0ggBY5XO+TAfgJDulmbVajhlP8Z+ o6qMNe2pGbo95nNnNVPE8mKzddGJvtE0j+9qWVkrxFfWTjQWzbkyiTlrWCFDeQGgW8sQzaLId2EhU jHV/79sQ==; Received: from [58.29.143.236] (helo=localhost) by fanzine2.igalia.com with utf8esmtpsa (Cipher TLS1.3:ECDHE_SECP256R1__RSA_PSS_RSAE_SHA256__AES_256_GCM:256) (Exim) id 1tbzh5-002ljX-GH; Sun, 26 Jan 2025 11:16:43 +0100 From: Changwoo Min To: tj@kernel.org, void@manifault.com, arighi@nvidia.com Cc: kernel-dev@igalia.com, linux-kernel@vger.kernel.org, Changwoo Min Subject: [PATCH v2 06/11] sched_ext: Add an event, ENQ_SKIP_EXITING Date: Sun, 26 Jan 2025 19:16:09 +0900 Message-ID: <20250126101614.232388-7-changwoo@igalia.com> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20250126101614.232388-1-changwoo@igalia.com> References: <20250126101614.232388-1-changwoo@igalia.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Add a core event, ENQ_SKIP_EXITING, which represents how many times a task is enqueued to a local DSQ when exiting if SCX_OPS_ENQ_EXITING is not set. __scx_add_event() is used since the caller holds an rq lock, so the preemption has already been disabled. Signed-off-by: Changwoo Min --- kernel/sched/ext.c | 11 ++++++++++- 1 file changed, 10 insertions(+), 1 deletion(-) diff --git a/kernel/sched/ext.c b/kernel/sched/ext.c index d3a1ef58202c..6dc86bc06596 100644 --- a/kernel/sched/ext.c +++ b/kernel/sched/ext.c @@ -1479,6 +1479,12 @@ struct scx_event_stats { * continued to run because there were no other tasks on the CPU. */ u64 DISPATCH_KEEP_LAST; + + /* + * If SCX_OPS_ENQ_EXITING is not set, the number of times that a task + * is dispatched to a local DSQ when exiting. + */ + u64 ENQ_SKIP_EXITING; }; =20 /* @@ -2086,8 +2092,10 @@ static void do_enqueue_task(struct rq *rq, struct ta= sk_struct *p, u64 enq_flags, =20 /* see %SCX_OPS_ENQ_EXITING */ if (!static_branch_unlikely(&scx_ops_enq_exiting) && - unlikely(p->flags & PF_EXITING)) + unlikely(p->flags & PF_EXITING)) { + __scx_add_event(ENQ_SKIP_EXITING, 1); goto local; + } =20 if (!SCX_HAS_OP(enqueue)) goto global; @@ -5508,6 +5516,7 @@ static void scx_dump_state(struct scx_exit_info *ei, = size_t dump_len) scx_dump_event(s, &events, SELECT_CPU_FALLBACK); scx_dump_event(s, &events, DISPATCH_LOCAL_DSQ_OFFLINE); scx_dump_event(s, &events, DISPATCH_KEEP_LAST); + scx_dump_event(s, &events, ENQ_SKIP_EXITING); =20 if (seq_buf_has_overflowed(&s) && dump_len >=3D sizeof(trunc_marker)) memcpy(ei->dump + dump_len - sizeof(trunc_marker), --=20 2.48.1 From nobody Mon Feb 9 03:34:12 2026 Received: from fanzine2.igalia.com (fanzine.igalia.com [178.60.130.6]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 49CF818A6DE for ; Sun, 26 Jan 2025 10:16:53 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=178.60.130.6 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1737886615; cv=none; b=UD80YqpGFCwD/rN22Amb3dhdA2TYct5hAw6KrbBAM9yMaqax/COUKQikoUGAsU7YYcKj9bTs0ytzmJqOII5FwFwEkdBP/npUVW6nNymMfIZYcU/H9ZStkhznRKEQte5WlPeYFKFjU5jEBmZmv1HvW2JgMzN3ruCrCs/dXmwSKA0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1737886615; c=relaxed/simple; bh=7bHON4e27xWbsJiRjtZTVY8cLmPu1U/T/7HEhj2W7kM=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=iX+u/ByFrgyiLs30xybeG+9NTm8C6OM8iT3spU8W3YAKah8jqVw63w5HZ2gVlawGfoAqZIYR6dOGnrCU8QcMN4sKpxNnI6XKbY20NFbDNEjEV2d+w+ACPbtVZrdFeFuygSNeIZX4CM5a1QqvCnPILvphCpxjsTtJTqlIv18zy40= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=igalia.com; spf=pass smtp.mailfrom=igalia.com; dkim=pass (2048-bit key) header.d=igalia.com header.i=@igalia.com header.b=MuQSqPwg; arc=none smtp.client-ip=178.60.130.6 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=igalia.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=igalia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=igalia.com header.i=@igalia.com header.b="MuQSqPwg" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=igalia.com; s=20170329; h=Content-Transfer-Encoding:MIME-Version:References:In-Reply-To: Message-ID:Date:Subject:Cc:To:From:Sender:Reply-To:Content-Type:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Id:List-Help:List-Unsubscribe:List-Subscribe: List-Post:List-Owner:List-Archive; bh=/vlVwR28KueFbCuYHtO3aKMp3rFjIy6xOaEP1Yibd+w=; b=MuQSqPwgx5pdMSvwJGgZOa6fpm HGQpbWMo56XnWvFDZc2yP/ImmmYxDwexJopIkQeUXzbZo+3+IhAZvMCbUnp/Qvtd1lYB+RH0OUfaY k8UIxQc2Xn5ZFqKVhmJBbc9cImm34G89WepGhxnr6u+BqFePIONEauMHyvfnehVjw9kt8TKM/fy3b qQ+54Rd1LZhWi2PdaSyn5OUoN8Kg2/DC4KmGNJK2VA/gmkkeF+GhzP3pm5LAFFgKftvjJg99tzWts ZsCqns751BBv2/ejNA4haCcGXVOftFBIpz7gux84JOfHYD7HE2G6WbMwW88vNbhw6m2h99hgB5kV+ rMXkU1gA==; Received: from [58.29.143.236] (helo=localhost) by fanzine2.igalia.com with utf8esmtpsa (Cipher TLS1.3:ECDHE_SECP256R1__RSA_PSS_RSAE_SHA256__AES_256_GCM:256) (Exim) id 1tbzh9-002ljf-CG; Sun, 26 Jan 2025 11:16:47 +0100 From: Changwoo Min To: tj@kernel.org, void@manifault.com, arighi@nvidia.com Cc: kernel-dev@igalia.com, linux-kernel@vger.kernel.org, Changwoo Min Subject: [PATCH v2 07/11] sched_ext: Add an event, BYPASS_ACTIVATE Date: Sun, 26 Jan 2025 19:16:10 +0900 Message-ID: <20250126101614.232388-8-changwoo@igalia.com> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20250126101614.232388-1-changwoo@igalia.com> References: <20250126101614.232388-1-changwoo@igalia.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Add a core event, BYPASS_ACTIVATE, which represents how many times the bypass mode has been triggered. Signed-off-by: Changwoo Min --- kernel/sched/ext.c | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/kernel/sched/ext.c b/kernel/sched/ext.c index 6dc86bc06596..bb363809e484 100644 --- a/kernel/sched/ext.c +++ b/kernel/sched/ext.c @@ -1485,6 +1485,11 @@ struct scx_event_stats { * is dispatched to a local DSQ when exiting. */ u64 ENQ_SKIP_EXITING; + + /* + * The number of times the bypassing mode has been activated. + */ + u64 BYPASS_ACTIVATE; }; =20 /* @@ -3707,6 +3712,7 @@ static int select_task_rq_scx(struct task_struct *p, = int prev_cpu, int wake_flag p->scx.slice =3D SCX_SLICE_DFL; p->scx.ddsp_dsq_id =3D SCX_DSQ_LOCAL; } + return cpu; } } @@ -4922,6 +4928,7 @@ static void scx_ops_bypass(bool bypass) WARN_ON_ONCE(scx_ops_bypass_depth <=3D 0); if (scx_ops_bypass_depth !=3D 1) goto unlock; + scx_add_event(BYPASS_ACTIVATE, 1); } else { scx_ops_bypass_depth--; WARN_ON_ONCE(scx_ops_bypass_depth < 0); @@ -5517,6 +5524,7 @@ static void scx_dump_state(struct scx_exit_info *ei, = size_t dump_len) scx_dump_event(s, &events, DISPATCH_LOCAL_DSQ_OFFLINE); scx_dump_event(s, &events, DISPATCH_KEEP_LAST); scx_dump_event(s, &events, ENQ_SKIP_EXITING); + scx_dump_event(s, &events, BYPASS_ACTIVATE); =20 if (seq_buf_has_overflowed(&s) && dump_len >=3D sizeof(trunc_marker)) memcpy(ei->dump + dump_len - sizeof(trunc_marker), @@ -7848,6 +7856,7 @@ __bpf_kfunc void scx_bpf_event_stats(struct scx_event= _stats *events, scx_agg_event(&e_sys, e_cpu, SELECT_CPU_FALLBACK); scx_agg_event(&e_sys, e_cpu, DISPATCH_LOCAL_DSQ_OFFLINE); scx_agg_event(&e_sys, e_cpu, DISPATCH_KEEP_LAST); + scx_agg_event(&e_sys, e_cpu, BYPASS_ACTIVATE); } =20 /* --=20 2.48.1 From nobody Mon Feb 9 03:34:12 2026 Received: from fanzine2.igalia.com (fanzine.igalia.com [178.60.130.6]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id AFDC718BC1D for ; Sun, 26 Jan 2025 10:16:57 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=178.60.130.6 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1737886619; cv=none; b=E38jQ99WwlZT7NFODfHC16IiWK5i/TH+vknGchSY+p/fPTZz1CaVIdvA4f+27+a5AwVMy9mAKolXFgqp8fRoGjqXrhfbyZMS8igRZabDyBHlxtaEc7WwdnOyjSZexufaBku415/+T0ChHhHpfvPueu6PI6XanBUCMGDj/nA8+84= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1737886619; c=relaxed/simple; bh=9TaXQqvuSL7TEUQg/WSTgMs5+1Yji7gBh1T3lSwtOSo=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=OB42sDKH7NYrH6VTXl49F3vXGU5aXUOu9Ylv88Rjh4AtVYdPTRWuvPzBPI9PAPK7kOQn3RZSUx5H11oB6svDdWNHLA3jUHKpdj9TALelpLNq0XiFjaEqT9lJuVv6cpWadUcYeYnB4ATY0ECmXFrOEFg0fUwyxn230O38N4hf0/k= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=igalia.com; spf=pass smtp.mailfrom=igalia.com; dkim=pass (2048-bit key) header.d=igalia.com header.i=@igalia.com header.b=AmNe5FJY; arc=none smtp.client-ip=178.60.130.6 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=igalia.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=igalia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=igalia.com header.i=@igalia.com header.b="AmNe5FJY" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=igalia.com; s=20170329; h=Content-Transfer-Encoding:MIME-Version:References:In-Reply-To: Message-ID:Date:Subject:Cc:To:From:Sender:Reply-To:Content-Type:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Id:List-Help:List-Unsubscribe:List-Subscribe: List-Post:List-Owner:List-Archive; bh=VhxrEoBVqhZs+PeQRjdWIdXVZhQu3ui3kpaiw/mKR7w=; b=AmNe5FJYDHJV+yCdUzE0j7lNGq ikmRlB4wXrledqovER9/Lk4Bb1hfPvPZD39n8lrkpHXs66xTqmo4i9owK7//NETo0rZI9t58Yb4W0 hBX8mWtx+neyIQKMc+9NkdDiZUFgiLn9pIWppGeWnavYaXGleKCSiCaRpgN+lA63NjuXWz3s74ycz Zr0OLGOLxFHBqYy18m6Z/QwQH3GRRqsk6XOW6KbqNONjR6i/2iVTo+MAY3zylOIXMWf6wrwbGRURi VzKZBT5cfxThLyCEts2NigesX03XtkDXvLP+qdZNyCV7Y8v3PdT7+/Hua1SYchS1AZz/Qb8dT3H+q 7GNVMXXg==; Received: from [58.29.143.236] (helo=localhost) by fanzine2.igalia.com with utf8esmtpsa (Cipher TLS1.3:ECDHE_SECP256R1__RSA_PSS_RSAE_SHA256__AES_256_GCM:256) (Exim) id 1tbzhD-002ljo-70; Sun, 26 Jan 2025 11:16:51 +0100 From: Changwoo Min To: tj@kernel.org, void@manifault.com, arighi@nvidia.com Cc: kernel-dev@igalia.com, linux-kernel@vger.kernel.org, Changwoo Min Subject: [PATCH v2 08/11] sched_ext: Add an event, BYPASS_DISPATCH Date: Sun, 26 Jan 2025 19:16:11 +0900 Message-ID: <20250126101614.232388-9-changwoo@igalia.com> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20250126101614.232388-1-changwoo@igalia.com> References: <20250126101614.232388-1-changwoo@igalia.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Add a core event, BYPASS_DISPATCH, which represents how many tasks have been dispatched in the bypass mode. __scx_add_event() is used since the caller holds an rq lock, so the preemption has already been disabled. Signed-off-by: Changwoo Min --- kernel/sched/ext.c | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/kernel/sched/ext.c b/kernel/sched/ext.c index bb363809e484..9a680774c1fc 100644 --- a/kernel/sched/ext.c +++ b/kernel/sched/ext.c @@ -1486,6 +1486,11 @@ struct scx_event_stats { */ u64 ENQ_SKIP_EXITING; =20 + /* + * The number of tasks dispatched in the bypassing mode. + */ + u64 BYPASS_DISPATCH; + /* * The number of times the bypassing mode has been activated. */ @@ -2941,6 +2946,8 @@ static int balance_one(struct rq *rq, struct task_str= uct *prev) return false; =20 has_tasks: + if (scx_rq_bypassing(rq)) + __scx_add_event(BYPASS_DISPATCH, 1); rq->scx.flags &=3D ~SCX_RQ_IN_BALANCE; return true; } @@ -5524,6 +5531,7 @@ static void scx_dump_state(struct scx_exit_info *ei, = size_t dump_len) scx_dump_event(s, &events, DISPATCH_LOCAL_DSQ_OFFLINE); scx_dump_event(s, &events, DISPATCH_KEEP_LAST); scx_dump_event(s, &events, ENQ_SKIP_EXITING); + scx_dump_event(s, &events, BYPASS_DISPATCH); scx_dump_event(s, &events, BYPASS_ACTIVATE); =20 if (seq_buf_has_overflowed(&s) && dump_len >=3D sizeof(trunc_marker)) @@ -7856,6 +7864,7 @@ __bpf_kfunc void scx_bpf_event_stats(struct scx_event= _stats *events, scx_agg_event(&e_sys, e_cpu, SELECT_CPU_FALLBACK); scx_agg_event(&e_sys, e_cpu, DISPATCH_LOCAL_DSQ_OFFLINE); scx_agg_event(&e_sys, e_cpu, DISPATCH_KEEP_LAST); + scx_agg_event(&e_sys, e_cpu, BYPASS_DISPATCH); scx_agg_event(&e_sys, e_cpu, BYPASS_ACTIVATE); } =20 --=20 2.48.1 From nobody Mon Feb 9 03:34:12 2026 Received: from fanzine2.igalia.com (fanzine.igalia.com [178.60.130.6]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 97F8518DF6B for ; Sun, 26 Jan 2025 10:17:00 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=178.60.130.6 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1737886622; cv=none; b=jmz8+NmxZtN5dLiWvhT9gh4+ndxaXOm9DfONJG9wIBfP+5/XN8A94S7Dlcnnc05xYfL+tSoOAk8DZ+l636Vt4QIwbdH0xGZnajjAyagArm6BNJrpBwN9Nl7o0yXU2bH4ZKZQSXOeCQILtEEde9i34IaJMmiUfKpS0dMdtwIQAVY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1737886622; c=relaxed/simple; bh=+W6vFJY9+FF0wu/Jbcv0gpBfOYDHI5nJ0cTjiu1aL/w=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=F3l77ZNAwoWizRszFWXjDtnjXPszp/S0EhqPhWjo4a8NwrvLUjUD22H8Edxdd6AMY/KuDicNZ1zXAwxZP9qI2ij5N3Wd0IzraKkAMNxXMuyb5gqCqsdo9yXbhsS1s3bBPLDM1BmxT4imy4HZ6NLg+veXlHOwiV8pjAd0WNMmMH8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=igalia.com; spf=pass smtp.mailfrom=igalia.com; dkim=pass (2048-bit key) header.d=igalia.com header.i=@igalia.com header.b=T8c5xIY0; arc=none smtp.client-ip=178.60.130.6 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=igalia.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=igalia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=igalia.com header.i=@igalia.com header.b="T8c5xIY0" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=igalia.com; s=20170329; h=Content-Transfer-Encoding:MIME-Version:References:In-Reply-To: Message-ID:Date:Subject:Cc:To:From:Sender:Reply-To:Content-Type:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Id:List-Help:List-Unsubscribe:List-Subscribe: List-Post:List-Owner:List-Archive; bh=D+St6nzHZZyPcvp1hwe9i/erQr5vrDh7K1HkwOLxKeI=; b=T8c5xIY0wic/WtyeNS+zd2iAYt WF7us7WDtrSRJHlqQSDEw74C4ZiXKcF8zJ4pVn5sUJGbp/gVYigZulJt77Y3oZ4SCZ9w+uN5HOjtu KRYwePfmBky9V/kJ5GdXK/iIas3Ht12ELou0bJBpl6DmA6fyWrBmmGQTS4AOG5zjnQ2q6OuFrlsco V9GKItdhNOpQ2uRR8OOZmrVq+WWxvu8cxSXtIDgD++Jj2tiLVuldJNJeu/0apbcIFyIYynmrDYhrp q9URBffertCFPIbyNLMX7ItqaLmOtVVdzx2wUbpWNyYS4Pxyw82jIfsmVUxK5yt/vOZK9klkDQ73Q chXzdtuA==; Received: from [58.29.143.236] (helo=localhost) by fanzine2.igalia.com with utf8esmtpsa (Cipher TLS1.3:ECDHE_SECP256R1__RSA_PSS_RSAE_SHA256__AES_256_GCM:256) (Exim) id 1tbzhH-002ljz-3U; Sun, 26 Jan 2025 11:16:55 +0100 From: Changwoo Min To: tj@kernel.org, void@manifault.com, arighi@nvidia.com Cc: kernel-dev@igalia.com, linux-kernel@vger.kernel.org, Changwoo Min Subject: [PATCH v2 09/11] sched_ext: Add an event, BYPASS_DURATION Date: Sun, 26 Jan 2025 19:16:12 +0900 Message-ID: <20250126101614.232388-10-changwoo@igalia.com> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20250126101614.232388-1-changwoo@igalia.com> References: <20250126101614.232388-1-changwoo@igalia.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Add a core event, BYPASS_DURATION, which represents the total duration of bypass modes in nanoseconds. Signed-off-by: Changwoo Min --- kernel/sched/ext.c | 16 ++++++++++++++++ 1 file changed, 16 insertions(+) diff --git a/kernel/sched/ext.c b/kernel/sched/ext.c index 9a680774c1fc..c71c89cfd1c8 100644 --- a/kernel/sched/ext.c +++ b/kernel/sched/ext.c @@ -1486,6 +1486,11 @@ struct scx_event_stats { */ u64 ENQ_SKIP_EXITING; =20 + /* + * The total duration of bypass modes in nanoseconds. + */ + u64 BYPASS_DURATION; + /* * The number of tasks dispatched in the bypassing mode. */ @@ -1547,6 +1552,12 @@ static DEFINE_PER_CPU(struct scx_event_stats, event_= stats_cpu); } while (0) =20 =20 +/* + * The last time the bypass mode started. + * This is used to measure BYPASS_DURATION. + */ +static unsigned long scx_bypass_timestamp; + static void scx_bpf_event_stats(struct scx_event_stats *events, size_t eve= nts__sz); =20 static enum scx_ops_enable_state scx_ops_enable_state(void) @@ -4935,12 +4946,15 @@ static void scx_ops_bypass(bool bypass) WARN_ON_ONCE(scx_ops_bypass_depth <=3D 0); if (scx_ops_bypass_depth !=3D 1) goto unlock; + scx_bypass_timestamp =3D ktime_get_ns(); scx_add_event(BYPASS_ACTIVATE, 1); } else { scx_ops_bypass_depth--; WARN_ON_ONCE(scx_ops_bypass_depth < 0); if (scx_ops_bypass_depth !=3D 0) goto unlock; + scx_add_event(BYPASS_DURATION, + ktime_get_ns() - scx_bypass_timestamp); } =20 atomic_inc(&scx_ops_breather_depth); @@ -5531,6 +5545,7 @@ static void scx_dump_state(struct scx_exit_info *ei, = size_t dump_len) scx_dump_event(s, &events, DISPATCH_LOCAL_DSQ_OFFLINE); scx_dump_event(s, &events, DISPATCH_KEEP_LAST); scx_dump_event(s, &events, ENQ_SKIP_EXITING); + scx_dump_event(s, &events, BYPASS_DURATION); scx_dump_event(s, &events, BYPASS_DISPATCH); scx_dump_event(s, &events, BYPASS_ACTIVATE); =20 @@ -7864,6 +7879,7 @@ __bpf_kfunc void scx_bpf_event_stats(struct scx_event= _stats *events, scx_agg_event(&e_sys, e_cpu, SELECT_CPU_FALLBACK); scx_agg_event(&e_sys, e_cpu, DISPATCH_LOCAL_DSQ_OFFLINE); scx_agg_event(&e_sys, e_cpu, DISPATCH_KEEP_LAST); + scx_agg_event(&e_sys, e_cpu, BYPASS_DURATION); scx_agg_event(&e_sys, e_cpu, BYPASS_DISPATCH); scx_agg_event(&e_sys, e_cpu, BYPASS_ACTIVATE); } --=20 2.48.1 From nobody Mon Feb 9 03:34:12 2026 Received: from fanzine2.igalia.com (fanzine.igalia.com [178.60.130.6]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6FC4C190468 for ; Sun, 26 Jan 2025 10:17:04 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=178.60.130.6 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1737886625; cv=none; b=i4B5Ayd8AbPi81pQ6f5bkf/012plt4imMAYw3KU0OxZenie1FYi1XWgadxRbJvEnDfgIqqAt+EyMZRK91OpN8tRTTqLetlYi7G3Nse6rdmW5P/UFGvJJeYKXShtymNdGU5p5nq4lMtfmUZeNJU1hGZnZGcd2e2RNtaCp+DR8Jjk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1737886625; c=relaxed/simple; bh=M3SN7MPLGUM2nK0Je0aoi6GQZHIaxEO8JIbO28ajVz8=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=q5mUJhs9H8XNIp7A4aWt/TuD/ZEp2OCAFFSuFC3xWia/0mFQWwFGM+UJk0wT4QjeLQ5roPqlftUTs5WcE86IV971WsasR5hooXBFKnkrylKEBVvXcFs3YFKImMxlb10sYrZhnb/THKrkFZ6vZTOmuRvXspizAp2nkaf6y5HfN8A= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=igalia.com; spf=pass smtp.mailfrom=igalia.com; dkim=pass (2048-bit key) header.d=igalia.com header.i=@igalia.com header.b=Bckny9E3; arc=none smtp.client-ip=178.60.130.6 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=igalia.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=igalia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=igalia.com header.i=@igalia.com header.b="Bckny9E3" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=igalia.com; s=20170329; h=Content-Transfer-Encoding:MIME-Version:References:In-Reply-To: Message-ID:Date:Subject:Cc:To:From:Sender:Reply-To:Content-Type:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Id:List-Help:List-Unsubscribe:List-Subscribe: List-Post:List-Owner:List-Archive; bh=Uhp8yg/et1zDGRAYLkxp3n7e2Rsaq16kuZHE776QPxI=; b=Bckny9E3Ex60GQFwo38aTID/em Tnanak9CUKg6rV1PPAJXcUSOQ5uj+OmT7re07sGsp71NK9CXsn8XJpDsUF4fX02nfwyI3XjVyDtCV M3You86w263+tYy4urYTVz5tlJASMHw3oeUH70cCVp+9hCFF0Bw282NzNJ/9oG/jZ1n5efJaCAP3Y ica5x1ub/o72zYUX1tVPrGvbZKaRfC+H+1FvBh7mbkxVCvwPqt8VdMS9Nms0oqnh/c2qzALgLVEGb H1IW/1CuOxneX7zDUI+Irl8m1kMkK1li02p351R0c2TjH0LuJ7sxwdinKq+Z4FuGYUL2H4cDwwSM4 P5AV9uZQ==; Received: from [58.29.143.236] (helo=localhost) by fanzine2.igalia.com with utf8esmtpsa (Cipher TLS1.3:ECDHE_SECP256R1__RSA_PSS_RSAE_SHA256__AES_256_GCM:256) (Exim) id 1tbzhL-002lkA-0F; Sun, 26 Jan 2025 11:16:59 +0100 From: Changwoo Min To: tj@kernel.org, void@manifault.com, arighi@nvidia.com Cc: kernel-dev@igalia.com, linux-kernel@vger.kernel.org, Changwoo Min Subject: [PATCH v2 10/11] sched_ext: Add scx_bpf_event_stats() and scx_read_event() for BPF schedulers Date: Sun, 26 Jan 2025 19:16:13 +0900 Message-ID: <20250126101614.232388-11-changwoo@igalia.com> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20250126101614.232388-1-changwoo@igalia.com> References: <20250126101614.232388-1-changwoo@igalia.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" scx_bpf_event_stats() is added to the header files so the BPF scheduler can use it. Also, scx_read_event() is added to read an event type in a compatible way. Signed-off-by: Changwoo Min --- tools/sched_ext/include/scx/common.bpf.h | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/tools/sched_ext/include/scx/common.bpf.h b/tools/sched_ext/inc= lude/scx/common.bpf.h index f3e15e9efa76..71db653c8eb2 100644 --- a/tools/sched_ext/include/scx/common.bpf.h +++ b/tools/sched_ext/include/scx/common.bpf.h @@ -78,6 +78,10 @@ struct rq *scx_bpf_cpu_rq(s32 cpu) __ksym; struct cgroup *scx_bpf_task_cgroup(struct task_struct *p) __ksym __weak; u64 scx_bpf_now(void) __ksym __weak; =20 +void scx_bpf_event_stats(struct scx_event_stats *events, size_t events__sz= ) __ksym __weak; +#define scx_read_event(e, name) \ + (bpf_core_field_exists((e)->name) ? (e)->name : 0) + /* * Use the following as @it__iter when calling scx_bpf_dsq_move[_vtime]() = from * within bpf_for_each() loops. --=20 2.48.1 From nobody Mon Feb 9 03:34:12 2026 Received: from fanzine2.igalia.com (fanzine.igalia.com [178.60.130.6]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2751A156F41 for ; Sun, 26 Jan 2025 10:17:08 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=178.60.130.6 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1737886629; cv=none; b=ODNG6FTQFuEhPB56LwWEBQUrdmbtrKtldOQl0NEcezv++OciYDgU+1ukFqqfxqiqqPzj9A3kbwI3vQqyAPfwq+J9Cw1Zgy8LiITaTAyo3l/JQcIeJI7ecswTc/O2e1+wstt3S/aK+8ew3e0qroWgHGX6CtVGwgHyHXTjysUYJ5Q= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1737886629; c=relaxed/simple; bh=WD0hkEeBoE/7Ad+cGMyhRgGJhmyAVe0I4st02IK/CYM=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=XRCkftPxbzYFWs1cpzaACcTUfsuMRaj1EuK99Vh6jLkx3QgaGfbWSbkXr8WucPWZVqWH0wTCQWbzRke212duCkw1JYbgOsk1XYtAEcZ1H2Vza1PcNTU2zqnW5VG9RSzLpzoz3wSAYvhTz77Ku9vD2ye5EWoeeweTmoKBwg/KOTk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=igalia.com; spf=pass smtp.mailfrom=igalia.com; dkim=pass (2048-bit key) header.d=igalia.com header.i=@igalia.com header.b=CVnPwfxr; arc=none smtp.client-ip=178.60.130.6 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=igalia.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=igalia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=igalia.com header.i=@igalia.com header.b="CVnPwfxr" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=igalia.com; s=20170329; h=Content-Transfer-Encoding:MIME-Version:References:In-Reply-To: Message-ID:Date:Subject:Cc:To:From:Sender:Reply-To:Content-Type:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Id:List-Help:List-Unsubscribe:List-Subscribe: List-Post:List-Owner:List-Archive; bh=IIupw5+B4K5ftxiiZqbt/DQaDjkqliFQIoGzKU5zFXc=; b=CVnPwfxr0g+kfizuHkat72XXot ZsA9JSkJyl1UMjKb6YzDvc/uyYPC8EguWozhwh0Tt+CapWDUGFp3cO3Oi/WgsbwpJGZDb69gZmlzn TDtCI3dbdMHhR//JuOPo7Y3Y6/AJTc4URsA5ckjL3lwMZwd1RgfivD8JNsQNl2zivJFxZ1YU/OF28 NKw0mU3YO0RzTpl85+lcg6kDEeGQgpXnZEbhlmq8H2CR8SUWlxf11tb6hGKsCUno2PjfCeRPyubAg Pabe+ZE4tjlHfyfffCqo4+piW23xyQvZeQbzDmOXzLrV6rxkezzt1OmxdCyT11guOSSxbCbGmCzaJ 5tmOiljA==; Received: from [58.29.143.236] (helo=localhost) by fanzine2.igalia.com with utf8esmtpsa (Cipher TLS1.3:ECDHE_SECP256R1__RSA_PSS_RSAE_SHA256__AES_256_GCM:256) (Exim) id 1tbzhO-002lkI-Uz; Sun, 26 Jan 2025 11:17:03 +0100 From: Changwoo Min To: tj@kernel.org, void@manifault.com, arighi@nvidia.com Cc: kernel-dev@igalia.com, linux-kernel@vger.kernel.org, Changwoo Min Subject: [PATCH v2 11/11] sched_ext: Print core event count in scx_central scheduler Date: Sun, 26 Jan 2025 19:16:14 +0900 Message-ID: <20250126101614.232388-12-changwoo@igalia.com> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20250126101614.232388-1-changwoo@igalia.com> References: <20250126101614.232388-1-changwoo@igalia.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Modify the scx_central scheduler to print the core event counter every second. Signed-off-by: Changwoo Min --- tools/sched_ext/scx_central.bpf.c | 21 +++++++++++++++++++++ 1 file changed, 21 insertions(+) diff --git a/tools/sched_ext/scx_central.bpf.c b/tools/sched_ext/scx_centra= l.bpf.c index 50bc1737c167..96b3c08132ea 100644 --- a/tools/sched_ext/scx_central.bpf.c +++ b/tools/sched_ext/scx_central.bpf.c @@ -256,6 +256,7 @@ static int central_timerfn(void *map, int *key, struct = bpf_timer *timer) u64 now =3D scx_bpf_now(); u64 nr_to_kick =3D nr_queued; s32 i, curr_cpu; + struct scx_event_stats events; =20 curr_cpu =3D bpf_get_smp_processor_id(); if (timer_pinned && (curr_cpu !=3D central_cpu)) { @@ -291,6 +292,26 @@ static int central_timerfn(void *map, int *key, struct= bpf_timer *timer) =20 bpf_timer_start(timer, TIMER_INTERVAL_NS, BPF_F_TIMER_CPU_PIN); __sync_fetch_and_add(&nr_timers, 1); + + /* print event counters every second */ + if (nr_timers % 1000 =3D=3D 0) { + scx_bpf_event_stats(&events, sizeof(events)); + + bpf_printk("%30s: %llu\n", "SELECT_CPU_FALLBACK", + scx_read_event(&events, SELECT_CPU_FALLBACK)); + bpf_printk("%30s: %llu\n", "DISPATCH_LOCAL_DSQ_OFFLINE", + scx_read_event(&events, DISPATCH_LOCAL_DSQ_OFFLINE)); + bpf_printk("%30s: %llu\n", "DISPATCH_KEEP_LAST", + scx_read_event(&events, DISPATCH_KEEP_LAST)); + bpf_printk("%30s: %llu\n", "ENQ_SKIP_EXITING", + scx_read_event(&events, ENQ_SKIP_EXITING)); + bpf_printk("%30s: %llu\n", "BYPASS_DURATION", + scx_read_event(&events, BYPASS_DURATION)); + bpf_printk("%30s: %llu\n", "BYPASS_DISPATCH", + scx_read_event(&events, BYPASS_DISPATCH)); + bpf_printk("%30s: %llu\n", "BYPASS_ACTIVATE", + scx_read_event(&events, BYPASS_ACTIVATE)); + } return 0; } =20 --=20 2.48.1