From nobody Sun Oct 5 10:44:49 2025 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 4E709245016; Tue, 5 Aug 2025 11:10:57 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=217.140.110.172 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1754392258; cv=none; b=H7Tubhlgc04+2qBXet5UP6S83t6DBGeqK4RMRqlBlZB5b+p06+lug6v3uNpzsL66UJZ6etAcJX3TkILX6zLFHOgiu0ijo+AIyPFzLh7BsIjsScj2S9obh6sJDSu8hx/D+gL3GjYOsJq/HCd4XD4P9O79VVt8hG0kyqgqX05lfgk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1754392258; c=relaxed/simple; bh=hjOL3ON5fPPF6nNMGzaUMSvif7RbckVL3EFMwbn3B9g=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=RLQYqCjUlP3Oy1wNdK5AXCYHPGGfYWj1VCS8MsLFW3ZWFJZLpmxLyu5b9o9HmixYyTUtB+iKo8fToY7/GIZCCq+ORHGF6VJEP2NJgtf6zhLaRoDSUGkT7ABK+GWK/4lpqmo1Fk/0XF5SGycLs4A8D+A+rLoa1hlCxTNr4vkKVsQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com; spf=pass smtp.mailfrom=arm.com; arc=none smtp.client-ip=217.140.110.172 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id D99932BC2; Tue, 5 Aug 2025 04:10:42 -0700 (PDT) Received: from e127648.cambridge.arm.com (e127648.arm.com [10.1.27.68]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 6D8813F673; Tue, 5 Aug 2025 04:10:48 -0700 (PDT) From: Christian Loehle To: tj@kernel.org, arighi@nvidia.com, void@manifault.com Cc: linux-kernel@vger.kernel.org, sched-ext@lists.linux.dev, changwoo@igalia.com, hodgesd@meta.com, mingo@redhat.com, peterz@infradead.org, Christian Loehle , stable@vger.kernel.org Subject: [PATCH v3 1/3] sched_ext: Mark scx_bpf_cpu_rq as NULL returnable Date: Tue, 5 Aug 2025 12:10:34 +0100 Message-Id: <20250805111036.130121-2-christian.loehle@arm.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20250805111036.130121-1-christian.loehle@arm.com> References: <20250805111036.130121-1-christian.loehle@arm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" scx_bpf_cpu_rq() obviously returns NULL on invalid cpu. Mark it as such. While kf_cpu_valid() will trigger scx_ops_error() that leads to the BPF scheduler exiting, this isn't guaranteed to be immediate, allowing for a dereference of a NULL scx_bpf_cpu_rq() return value. Cc: stable@vger.kernel.org Fixes: 6203ef73fa5c ("sched/ext: Add BPF function to fetch rq") Signed-off-by: Christian Loehle Acked-by: Andrea Righi Reviewed-by: Andrea Righi --- kernel/sched/ext.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/kernel/sched/ext.c b/kernel/sched/ext.c index 7dedc9a16281..3ea3f0f18030 100644 --- a/kernel/sched/ext.c +++ b/kernel/sched/ext.c @@ -7589,7 +7589,7 @@ BTF_ID_FLAGS(func, scx_bpf_get_online_cpumask, KF_ACQ= UIRE) BTF_ID_FLAGS(func, scx_bpf_put_cpumask, KF_RELEASE) BTF_ID_FLAGS(func, scx_bpf_task_running, KF_RCU) BTF_ID_FLAGS(func, scx_bpf_task_cpu, KF_RCU) -BTF_ID_FLAGS(func, scx_bpf_cpu_rq) +BTF_ID_FLAGS(func, scx_bpf_cpu_rq, KF_RET_NULL) #ifdef CONFIG_CGROUP_SCHED BTF_ID_FLAGS(func, scx_bpf_task_cgroup, KF_RCU | KF_ACQUIRE) #endif --=20 2.34.1 From nobody Sun Oct 5 10:44:49 2025 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 9417B241673 for ; Tue, 5 Aug 2025 11:10:54 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=217.140.110.172 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1754392256; cv=none; b=LjyR/lnHxiVFbLzEKdv9zIK9jO1GFDc8up5acq3n25LC3wV2L4YqXsPNrHuu411kYSmjKanqG7MPGEo06ROmr/yiaWx7p3SAgdf4mVkyVrRjtKCba0yqsvvHtWJCWcpo+Td6U1xCwXDTtndn6/+fxe4ltMes91/WaM/7FaLG9bg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1754392256; c=relaxed/simple; bh=T2X+D7Iro9Z0/J5jP5X6XGioyv6EcvCHjXQcPHQxzXU=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=DQ5DCSQVd5g5V4nmDg/TNNdhEvVn0cjlfkQTkN3B0Bij8uuL8LCLvy+wXFS0/OamO3cy60OJ5vKmq34Z8AdiTsCK5PyCd4GQg2QQQNLKDHkx/6qdRyJ/YR1YBa0YEahFlxVoSWRSrX6bch1ZLIwu7w/z20mXLJNLrmagwUu/9o8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com; spf=pass smtp.mailfrom=arm.com; arc=none smtp.client-ip=217.140.110.172 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 1897D2BCB; Tue, 5 Aug 2025 04:10:46 -0700 (PDT) Received: from e127648.cambridge.arm.com (e127648.arm.com [10.1.27.68]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 084853F673; Tue, 5 Aug 2025 04:10:51 -0700 (PDT) From: Christian Loehle To: tj@kernel.org, arighi@nvidia.com, void@manifault.com Cc: linux-kernel@vger.kernel.org, sched-ext@lists.linux.dev, changwoo@igalia.com, hodgesd@meta.com, mingo@redhat.com, peterz@infradead.org, Christian Loehle Subject: [PATCH v3 2/3] sched_ext: Provide scx_bpf_task_acquire_remote_curr() Date: Tue, 5 Aug 2025 12:10:35 +0100 Message-Id: <20250805111036.130121-3-christian.loehle@arm.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20250805111036.130121-1-christian.loehle@arm.com> References: <20250805111036.130121-1-christian.loehle@arm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Provide scx_bpf_task_acquire_remote_curr() as a way for scx schedulers to check the curr task of a remote rq without assuming its lock is held. Many scx schedulers make use of scx_bpf_cpu_rq() to check a remote curr (e.g. to see if it should be preempted). This is problematic because scx_bpf_cpu_rq() provides access to all fields of struct rq, most of which aren't safe to use without holding the associated rq lock. Signed-off-by: Christian Loehle Reviewed-by: Andrea Righi --- kernel/sched/ext.c | 24 ++++++++++++++++++++++++ tools/sched_ext/include/scx/common.bpf.h | 1 + 2 files changed, 25 insertions(+) diff --git a/kernel/sched/ext.c b/kernel/sched/ext.c index 3ea3f0f18030..3e2fa0b1eb57 100644 --- a/kernel/sched/ext.c +++ b/kernel/sched/ext.c @@ -7426,6 +7426,29 @@ __bpf_kfunc struct rq *scx_bpf_cpu_rq(s32 cpu) return cpu_rq(cpu); } =20 +/** + * scx_bpf_task_acquire_remote_curr - Fetch the curr task of a rq without + * acquiring its rq lock + * @cpu: CPU of the rq + * + * Increments the refcount of the task_struct which needs to be released u= sing + * bpf_task_release(). + */ +__bpf_kfunc struct task_struct *scx_bpf_task_acquire_remote_curr(s32 cpu) +{ + struct task_struct *p; + + if (!kf_cpu_valid(cpu, NULL)) + return NULL; + + rcu_read_lock(); + p =3D cpu_rq(cpu)->curr; + if (p) + p =3D refcount_inc_not_zero(&p->rcu_users) ? p : NULL; + rcu_read_unlock(); + return p; +} + /** * scx_bpf_task_cgroup - Return the sched cgroup of a task * @p: task of interest @@ -7590,6 +7613,7 @@ BTF_ID_FLAGS(func, scx_bpf_put_cpumask, KF_RELEASE) BTF_ID_FLAGS(func, scx_bpf_task_running, KF_RCU) BTF_ID_FLAGS(func, scx_bpf_task_cpu, KF_RCU) BTF_ID_FLAGS(func, scx_bpf_cpu_rq, KF_RET_NULL) +BTF_ID_FLAGS(func, scx_bpf_task_acquire_remote_curr, KF_RET_NULL | KF_ACQU= IRE) #ifdef CONFIG_CGROUP_SCHED BTF_ID_FLAGS(func, scx_bpf_task_cgroup, KF_RCU | KF_ACQUIRE) #endif diff --git a/tools/sched_ext/include/scx/common.bpf.h b/tools/sched_ext/inc= lude/scx/common.bpf.h index d4e21558e982..bdd68f3100b7 100644 --- a/tools/sched_ext/include/scx/common.bpf.h +++ b/tools/sched_ext/include/scx/common.bpf.h @@ -91,6 +91,7 @@ s32 scx_bpf_pick_any_cpu(const cpumask_t *cpus_allowed, u= 64 flags) __ksym; bool scx_bpf_task_running(const struct task_struct *p) __ksym; s32 scx_bpf_task_cpu(const struct task_struct *p) __ksym; struct rq *scx_bpf_cpu_rq(s32 cpu) __ksym; +struct task_struct *scx_bpf_task_acquire_remote_curr(s32 cpu) __ksym; struct cgroup *scx_bpf_task_cgroup(struct task_struct *p) __ksym __weak; u64 scx_bpf_now(void) __ksym __weak; void scx_bpf_events(struct scx_event_stats *events, size_t events__sz) __k= sym __weak; --=20 2.34.1 From nobody Sun Oct 5 10:44:49 2025 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 3FB94246798 for ; Tue, 5 Aug 2025 11:10:58 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=217.140.110.172 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1754392259; cv=none; b=RMFgG/Kmsh7AVsToduHgUyPQLgYKstv/li9dGI8T1LDCxnCIPznIt4kFHPa2ZyXciHJUazRxYcl0d2yQndwGBf53uxjphe/+6SDyIy7gldcl+YNWK7jfz9XyqxijisYk01fmQOqmMaLqjvC1k43cnawubga3IVye9Cf4Yo7nXtk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1754392259; c=relaxed/simple; bh=PUDAo6grH9DktBE1Hai0CoaFv7psRNK9cyOz30Tu3Uo=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=BJQSfETnaoWgjBRbgQfUYUpVr834pFfpfkOOT8XJ/LsLJ6NFaUYDrzi1NB371OORn0hMDiIIT1CUZtZoBycPmUC0LI9kaqNRkBxAfgcMK58yN7tcKKC0MvScZfe+OB3vFTE4dWrUHDaXAxaxdKaoinWlPm6vGyjzGb2OSXeg+Bc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com; spf=pass smtp.mailfrom=arm.com; arc=none smtp.client-ip=217.140.110.172 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 7880B113E; Tue, 5 Aug 2025 04:10:49 -0700 (PDT) Received: from e127648.cambridge.arm.com (e127648.arm.com [10.1.27.68]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 650F63F673; Tue, 5 Aug 2025 04:10:55 -0700 (PDT) From: Christian Loehle To: tj@kernel.org, arighi@nvidia.com, void@manifault.com Cc: linux-kernel@vger.kernel.org, sched-ext@lists.linux.dev, changwoo@igalia.com, hodgesd@meta.com, mingo@redhat.com, peterz@infradead.org, Christian Loehle Subject: [PATCH v3 3/3] sched_ext: Guarantee rq lock on scx_bpf_cpu_rq() Date: Tue, 5 Aug 2025 12:10:36 +0100 Message-Id: <20250805111036.130121-4-christian.loehle@arm.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20250805111036.130121-1-christian.loehle@arm.com> References: <20250805111036.130121-1-christian.loehle@arm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Most fields in scx_bpf_cpu_rq() assume that its rq_lock is held. Furthermore they become meaningless without rq lock, too. Only return scx_bpf_cpu_rq() if we hold rq lock of that rq. All upstream scx schedulers can be converted into the new scx_bpf_task_acquire_remote_curr() instead. Signed-off-by: Christian Loehle Reviewed-by: Andrea Righi --- kernel/sched/ext.c | 12 +++++++++++- 1 file changed, 11 insertions(+), 1 deletion(-) diff --git a/kernel/sched/ext.c b/kernel/sched/ext.c index 3e2fa0b1eb57..a66cf654f33e 100644 --- a/kernel/sched/ext.c +++ b/kernel/sched/ext.c @@ -7420,10 +7420,20 @@ __bpf_kfunc s32 scx_bpf_task_cpu(const struct task_= struct *p) */ __bpf_kfunc struct rq *scx_bpf_cpu_rq(s32 cpu) { + struct rq *rq; + if (!kf_cpu_valid(cpu, NULL)) return NULL; =20 - return cpu_rq(cpu); + preempt_disable(); + rq =3D cpu_rq(cpu); + if (rq !=3D scx_locked_rq()) { + scx_kf_error("Accessing not locked rq %d", cpu); + rq =3D NULL; + } + preempt_enable(); + + return rq; } =20 /** --=20 2.34.1