From nobody Sun Oct 5 14:44:53 2025 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 29C4C26CE3A for ; Fri, 1 Aug 2025 14:17:55 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=217.140.110.172 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1754057877; cv=none; b=QIb18JcYctzBMiM6DWDq8XZXdaGkTmmmrq5gC089VtdZsMaYMErvcWcjqsz6klZzniA0B7yAzpP5jP3tNxpEv0pN1SrDJpvhIF+hHRZAnArbalIShkofNPCjgkq2keMH+sMiz85t+WUKSGEVW0/mgsUxmOwhzgKsf7HAsSrV8fs= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1754057877; c=relaxed/simple; bh=2sYU4jHFbjyd1yjtFmZkOj2kjSLSWAglphbzQJ89pv4=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=bbkHBigLRJbXcBZqwpIDh2MYTLgJG8q23xAeJPBc2Ey0Ob8pXIEMO/eCxm3lxISCDQUYGb1Ytl5cOi6bIV48xG2i9g4s3BI04tZuf4QfxUbWqh+7LtyI5gl/VXLAJawMdzYMrae2Ow7fxH0WXvKagZgsNcpjQkkYfTIrEasCMwo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com; spf=pass smtp.mailfrom=arm.com; arc=none smtp.client-ip=217.140.110.172 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 5186716A3; Fri, 1 Aug 2025 07:17:47 -0700 (PDT) Received: from e127648.cambridge.arm.com (e127648.arm.com [10.1.27.49]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 627073F673; Fri, 1 Aug 2025 07:17:53 -0700 (PDT) From: Christian Loehle To: christian.loehle@arm.com, linux-kernel@vger.kernel.org, sched-ext@lists.linux.dev, tj@kernel.org, void@manifault.com, arighi@nvidia.com Cc: mingo@redhat.com, peterz@infradead.org Subject: [PATCH 1/3] sched_ext: Mark scx_bpf_cpu_rq as NULL returnable Date: Fri, 1 Aug 2025 15:17:39 +0100 Message-Id: <20250801141741.355059-2-christian.loehle@arm.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20250801141741.355059-1-christian.loehle@arm.com> References: <20250801141741.355059-1-christian.loehle@arm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" scx_bpf_cpu_rq() obviously returns NULL on invalid cpu. Mark it as such. While kf_cpu_valid() will trigger scx_ops_error() that leads to the BPF scheduler exiting, this isn't guaranteed to be immediate, allowing for a dereference of a NULL scx_bpf_cpu_rq() return value. Cc: stable@vger.kernel.org Fixes: 6203ef73fa5c ("sched/ext: Add BPF function to fetch rq") Signed-off-by: Christian Loehle Acked-by: Andrea Righi --- kernel/sched/ext.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/kernel/sched/ext.c b/kernel/sched/ext.c index 7dd5cbcb7a06..b734f55f3318 100644 --- a/kernel/sched/ext.c +++ b/kernel/sched/ext.c @@ -7599,7 +7599,7 @@ BTF_ID_FLAGS(func, scx_bpf_get_online_cpumask, KF_ACQ= UIRE) BTF_ID_FLAGS(func, scx_bpf_put_cpumask, KF_RELEASE) BTF_ID_FLAGS(func, scx_bpf_task_running, KF_RCU) BTF_ID_FLAGS(func, scx_bpf_task_cpu, KF_RCU) -BTF_ID_FLAGS(func, scx_bpf_cpu_rq) +BTF_ID_FLAGS(func, scx_bpf_cpu_rq, KF_RET_NULL) #ifdef CONFIG_CGROUP_SCHED BTF_ID_FLAGS(func, scx_bpf_task_cgroup, KF_RCU | KF_ACQUIRE) #endif --=20 2.34.1 From nobody Sun Oct 5 14:44:53 2025 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 6A3D7248873 for ; Fri, 1 Aug 2025 14:17:59 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=217.140.110.172 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1754057880; cv=none; b=OktYU4vSRW726gBdOvf3h5MU5TZ9QEBWm+tCEDZ6piapcVEXuSTKKYc96dWqhIdZ++9Ed2RXjlmUMGHh+bCm7iS+xnRozGxylzqj6TatgRDmeJ6Ownm0/w2SGKz/WR+1qBapWxyuumWRWz8QhnTA6Sr/SH0aQUE2zpvzTiV/dvg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1754057880; c=relaxed/simple; bh=iATzpBZIaYhRZrN9pHWg+tfbUaPW7A7zMYAgThJhUKM=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=ush9D3wWpC8FvIbOj9QQrjsPcF3lF67aOGUgC6qOLE5IrtzM8wNRPQcS9j+rKbqDaLusv4dqDBtmsheGf0QhyQs7aaIHmiroajsZOFsYiVgjDYuh0gvY6oxrnqabyt2ZlPhI6a1F0Jzs1J06tnNaOkteorah6u3q1sjHYeKB2yE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com; spf=pass smtp.mailfrom=arm.com; arc=none smtp.client-ip=217.140.110.172 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id C6B961515; Fri, 1 Aug 2025 07:17:50 -0700 (PDT) Received: from e127648.cambridge.arm.com (e127648.arm.com [10.1.27.49]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id D376C3F673; Fri, 1 Aug 2025 07:17:56 -0700 (PDT) From: Christian Loehle To: christian.loehle@arm.com, linux-kernel@vger.kernel.org, sched-ext@lists.linux.dev, tj@kernel.org, void@manifault.com, arighi@nvidia.com Cc: mingo@redhat.com, peterz@infradead.org Subject: [PATCH 2/3] sched_ext: Provide scx_bpf_remote_curr() Date: Fri, 1 Aug 2025 15:17:40 +0100 Message-Id: <20250801141741.355059-3-christian.loehle@arm.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20250801141741.355059-1-christian.loehle@arm.com> References: <20250801141741.355059-1-christian.loehle@arm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Provide scx_bpf_remote_curr() as a way for scx schedulers to check the curr task of a remote rq, without assuming its lock or acquiring any. Many scx schedulers make use of scx_bpf_cpu_rq() to check a remote curr (e.g. to see if it should be preempted). This is problematic because scx_bpf_cpu_rq() provides access to all fields of struct rq, most of which aren't safe to use without holding the associated rq lock. Signed-off-by: Christian Loehle --- kernel/sched/ext.c | 15 +++++++++++++++ tools/sched_ext/include/scx/common.bpf.h | 1 + 2 files changed, 16 insertions(+) diff --git a/kernel/sched/ext.c b/kernel/sched/ext.c index b734f55f3318..92e66bb0b5f2 100644 --- a/kernel/sched/ext.c +++ b/kernel/sched/ext.c @@ -7436,6 +7436,20 @@ __bpf_kfunc struct rq *scx_bpf_cpu_rq(s32 cpu) return cpu_rq(cpu); } =20 +/** + * scx_bpf_remote_curr - Fetch the curr of a rq without acquiring its rq l= ock + * @cpu: CPU of the rq + * + * Neither a rq lock nor a task reference is acquired. + */ +__bpf_kfunc struct task_struct *scx_bpf_remote_curr(s32 cpu) +{ + if (!kf_cpu_valid(cpu, NULL)) + return NULL; + + return cpu_rq(cpu)->curr; +} + /** * scx_bpf_task_cgroup - Return the sched cgroup of a task * @p: task of interest @@ -7600,6 +7614,7 @@ BTF_ID_FLAGS(func, scx_bpf_put_cpumask, KF_RELEASE) BTF_ID_FLAGS(func, scx_bpf_task_running, KF_RCU) BTF_ID_FLAGS(func, scx_bpf_task_cpu, KF_RCU) BTF_ID_FLAGS(func, scx_bpf_cpu_rq, KF_RET_NULL) +BTF_ID_FLAGS(func, scx_bpf_remote_curr, KF_RET_NULL) #ifdef CONFIG_CGROUP_SCHED BTF_ID_FLAGS(func, scx_bpf_task_cgroup, KF_RCU | KF_ACQUIRE) #endif diff --git a/tools/sched_ext/include/scx/common.bpf.h b/tools/sched_ext/inc= lude/scx/common.bpf.h index d4e21558e982..e5d4ef124532 100644 --- a/tools/sched_ext/include/scx/common.bpf.h +++ b/tools/sched_ext/include/scx/common.bpf.h @@ -91,6 +91,7 @@ s32 scx_bpf_pick_any_cpu(const cpumask_t *cpus_allowed, u= 64 flags) __ksym; bool scx_bpf_task_running(const struct task_struct *p) __ksym; s32 scx_bpf_task_cpu(const struct task_struct *p) __ksym; struct rq *scx_bpf_cpu_rq(s32 cpu) __ksym; +struct task_struct *scx_bpf_remote_curr(s32 cpu) __ksym; struct cgroup *scx_bpf_task_cgroup(struct task_struct *p) __ksym __weak; u64 scx_bpf_now(void) __ksym __weak; void scx_bpf_events(struct scx_event_stats *events, size_t events__sz) __k= sym __weak; --=20 2.34.1 From nobody Sun Oct 5 14:44:53 2025 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 5813F26E71C for ; Fri, 1 Aug 2025 14:18:02 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=217.140.110.172 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1754057883; cv=none; b=C26dvybJ3VoDSryTLUtI3MXLpXoGiW7YkT1V+eO9SzQD/lI0SLSL11QGavSVv3Jvm8Z+ah7THPwLAvsuIVt8XNs7gKBtsib4x78RZIeX9VOKgvoituBkYzQPWHK+ICiAwEXEomH6idP43sXVfAo9QxKqB/u9QXjHSdlQvl9H4yE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1754057883; c=relaxed/simple; bh=Ym6ERqnzHQoUR3s3jCQ9RI780xASY9USRgCsxRkOzoA=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=nPZK6V0ZIu2P27ve+C0H8SAARgsJgnhNz0SV4DNifFzRWLEOAKHU6zrCPVaY93DmiALrvsKvTrs+p2jMjWTvHPX/lD2dEn4PE0p/j506Yvyjnqa6tGho4F6Nr3UcKxf/69HbPQixLVWiBJd3dHeFTCSehc9vo0KR3rR2i+XizcA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com; spf=pass smtp.mailfrom=arm.com; arc=none smtp.client-ip=217.140.110.172 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id A66981515; Fri, 1 Aug 2025 07:17:53 -0700 (PDT) Received: from e127648.cambridge.arm.com (e127648.arm.com [10.1.27.49]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id D0AA43F673; Fri, 1 Aug 2025 07:17:59 -0700 (PDT) From: Christian Loehle To: christian.loehle@arm.com, linux-kernel@vger.kernel.org, sched-ext@lists.linux.dev, tj@kernel.org, void@manifault.com, arighi@nvidia.com Cc: mingo@redhat.com, peterz@infradead.org Subject: [PATCH 3/3] sched_ext: Guarantee rq lock on scx_bpf_cpu_rq() Date: Fri, 1 Aug 2025 15:17:41 +0100 Message-Id: <20250801141741.355059-4-christian.loehle@arm.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20250801141741.355059-1-christian.loehle@arm.com> References: <20250801141741.355059-1-christian.loehle@arm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Most fields in scx_bpf_cpu_rq() assume that its rq_lock is held. Furthermore they become meaningless without rq lock, too. Only return scx_bpf_cpu_rq() when we hold rq lock of that rq. All upstream scx schedulers can be converted into the new scx_bpf_remote_curr() instead. Signed-off-by: Christian Loehle --- kernel/sched/ext.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/kernel/sched/ext.c b/kernel/sched/ext.c index 92e66bb0b5f2..627df3088fd0 100644 --- a/kernel/sched/ext.c +++ b/kernel/sched/ext.c @@ -7425,7 +7425,7 @@ __bpf_kfunc s32 scx_bpf_task_cpu(const struct task_st= ruct *p) } =20 /** - * scx_bpf_cpu_rq - Fetch the rq of a CPU + * scx_bpf_cpu_rq - Fetch the rq of a CPU if its rq lock is currently held * @cpu: CPU of the rq */ __bpf_kfunc struct rq *scx_bpf_cpu_rq(s32 cpu) @@ -7433,7 +7433,7 @@ __bpf_kfunc struct rq *scx_bpf_cpu_rq(s32 cpu) if (!kf_cpu_valid(cpu, NULL)) return NULL; =20 - return cpu_rq(cpu); + return this_cpu_read(locked_rq) =3D=3D cpu_rq(cpu) ? cpu_rq(cpu) : NULL; } =20 /** --=20 2.34.1