From nobody Thu Apr 9 16:34:58 2026 Received: from galois.linutronix.de (Galois.linutronix.de [193.142.43.55]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 79C3E13E02A; Fri, 3 Apr 2026 12:30:32 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=193.142.43.55 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1775219433; cv=none; b=em5sem70OR5xW8h8hiKQf6yYRskZA71r3vfKnBvy4DuDGVfaCnBJGFDwEckauFzhdVoq4ZXFdy4pY7zVOAoTtfjcSKsoMZFqxs3UyeHGplikGklYwY4ItzWnDzQ0zD8r2eIA/8XTf3Ky3ihYz+57/I7ypVbd5pbin/sQVmFUy1o= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1775219433; c=relaxed/simple; bh=TozxdRpu/3QA3vETZeGisBtBk7HkNFYHOTQXRhjxH14=; h=Date:From:To:Subject:Cc:In-Reply-To:References:MIME-Version: Message-ID:Content-Type; b=TrQMIHwQk3v7Oj5C23P/eMUOndGZapBWl5W263vJ/XglOouhamniT/vxWJECVNkJ1msV0ElnUwArUukvjuhhSaG7ISE/McpL0dUcp1Skgf9GYTlWZ/RWTm+HBz79j8+uhMjFAm0SjbkyJRoJj8lJ+/cTZh/rr1CXn0tntG2oFyI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linutronix.de; spf=pass smtp.mailfrom=linutronix.de; dkim=pass (2048-bit key) header.d=linutronix.de header.i=@linutronix.de header.b=k5nXKAWM; dkim=permerror (0-bit key) header.d=linutronix.de header.i=@linutronix.de header.b=WO9TCSQk; arc=none smtp.client-ip=193.142.43.55 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linutronix.de Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linutronix.de Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=linutronix.de header.i=@linutronix.de header.b="k5nXKAWM"; dkim=permerror (0-bit key) header.d=linutronix.de header.i=@linutronix.de header.b="WO9TCSQk" Date: Fri, 03 Apr 2026 12:30:28 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1775219429; h=from:from:sender:sender:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Osf6zr/reJ8r9rXnenGA0ZVGWDghFeujT2/sUvpSn+Y=; b=k5nXKAWMvcTSXDKctjupErXGEqXh94wO1Rpnm+Mb0+Q7BXrijdhIly4doZ2Zaf8h5zNY3D 3/07k6z9sKM08HjdQJ8yb5iPK/L0z6Ds+7pHQ17KwmpP5o6mhQXzAllE0W51sfY6fNjcRv WSEEQ2VzUfBQBXJZ+tHqNPnH+103DgQP0iLYwvb9QrxubIkQHkFIQ5OyrK2QBvottxa7I6 xmL1qUaGAiT0n2UMGhyYrJG0Uv4kX1gwn+vrspI83VHvtX0HxHPPs3ZUN7ab1GrOStK0X0 Dquj84p4nekxFNr2GmY7oOTDGxaoNWLMEiWkUQi7RGiTh3J7jYQEfjOiF9GNag== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1775219429; h=from:from:sender:sender:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Osf6zr/reJ8r9rXnenGA0ZVGWDghFeujT2/sUvpSn+Y=; b=WO9TCSQkwuZ5MOf551G3g3I2m51lNKCs8w9FP/TC9bzsRLii9N/bhACpZpeXR7iHWXHzmQ THBG4fsyrOooAaDg== From: "tip-bot2 for John Stultz" Sender: tip-bot2@linutronix.de Reply-to: linux-kernel@vger.kernel.org To: linux-tip-commits@vger.kernel.org Subject: [tip: sched/core] sched: Make class_schedulers avoid pushing current, and get rid of proxy_tag_curr() Cc: K Prateek Nayak , Peter Zijlstra , John Stultz , x86@kernel.org, linux-kernel@vger.kernel.org In-Reply-To: <20260324191337.1841376-2-jstultz@google.com> References: <20260324191337.1841376-2-jstultz@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-ID: <177521942817.1647592.9508131167253851909.tip-bot2@tip-bot2> Robot-ID: Robot-Unsubscribe: Contact to get blacklisted from these emails Precedence: bulk Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable The following commit has been merged into the sched/core branch of tip: Commit-ID: e0ca8991b2de6c9dfe6fcd8a0364951b2bd56797 Gitweb: https://git.kernel.org/tip/e0ca8991b2de6c9dfe6fcd8a0364951b2= bd56797 Author: John Stultz AuthorDate: Tue, 24 Mar 2026 19:13:16=20 Committer: Peter Zijlstra CommitterDate: Fri, 03 Apr 2026 14:23:38 +02:00 sched: Make class_schedulers avoid pushing current, and get rid of proxy_ta= g_curr() With proxy-execution, the scheduler selects the donor, but for blocked donors, we end up running the lock owner. This caused some complexity, because the class schedulers make sure to remove the task they pick from their pushable task lists, which prevents the donor from being migrated, but there wasn't then anything to prevent rq->curr from being migrated if rq->curr !=3D rq->donor. This was sort of hacked around by calling proxy_tag_curr() on the rq->curr task if we were running something other then the donor. proxy_tag_curr() did a dequeue/enqueue pair on the rq->curr task, allowing the class schedulers to remove it from their pushable list. The dequeue/enqueue pair was wasteful, and additonally K Prateek highlighted that we didn't properly undo things when we stopped proxying, leaving the lock owner off the pushable list. After some alternative approaches were considered, Peter suggested just having the RT/DL classes just avoid migrating when task_on_cpu(). So rework pick_next_pushable_dl_task() and the rt pick_next_pushable_task() functions so that they skip over the first pushable task if it is on_cpu. Then just drop all of the proxy_tag_curr() logic. Fixes: be39617e38e0 ("sched: Fix proxy/current (push,pull)ability") Closes: https://lore.kernel.org/lkml/e735cae0-2cc9-4bae-b761-fcb082ed3e94@a= md.com/ Reported-by: K Prateek Nayak Suggested-by: Peter Zijlstra Signed-off-by: John Stultz Signed-off-by: Peter Zijlstra (Intel) Link: https://patch.msgid.link/20260324191337.1841376-2-jstultz@google.com --- kernel/sched/core.c | 24 ------------------------ kernel/sched/deadline.c | 18 ++++++++++++++++-- kernel/sched/rt.c | 15 ++++++++++++--- 3 files changed, 28 insertions(+), 29 deletions(-) diff --git a/kernel/sched/core.c b/kernel/sched/core.c index 7c7d4bf..2974168 100644 --- a/kernel/sched/core.c +++ b/kernel/sched/core.c @@ -6702,23 +6702,6 @@ find_proxy_task(struct rq *rq, struct task_struct *d= onor, struct rq_flags *rf) } #endif /* SCHED_PROXY_EXEC */ =20 -static inline void proxy_tag_curr(struct rq *rq, struct task_struct *owner) -{ - if (!sched_proxy_exec()) - return; - /* - * pick_next_task() calls set_next_task() on the chosen task - * at some point, which ensures it is not push/pullable. - * However, the chosen/donor task *and* the mutex owner form an - * atomic pair wrt push/pull. - * - * Make sure owner we run is not pushable. Unfortunately we can - * only deal with that by means of a dequeue/enqueue cycle. :-/ - */ - dequeue_task(rq, owner, DEQUEUE_NOCLOCK | DEQUEUE_SAVE); - enqueue_task(rq, owner, ENQUEUE_NOCLOCK | ENQUEUE_RESTORE); -} - /* * __schedule() is the main scheduler function. * @@ -6871,9 +6854,6 @@ keep_resched: */ RCU_INIT_POINTER(rq->curr, next); =20 - if (!task_current_donor(rq, next)) - proxy_tag_curr(rq, next); - /* * The membarrier system call requires each architecture * to have a full memory barrier after updating @@ -6907,10 +6887,6 @@ keep_resched: /* Also unlocks the rq: */ rq =3D context_switch(rq, prev, next, &rf); } else { - /* In case next was already curr but just got blocked_donor */ - if (!task_current_donor(rq, next)) - proxy_tag_curr(rq, next); - rq_unpin_lock(rq, &rf); __balance_callbacks(rq, NULL); raw_spin_rq_unlock_irq(rq); diff --git a/kernel/sched/deadline.c b/kernel/sched/deadline.c index 9e253a8..27359a1 100644 --- a/kernel/sched/deadline.c +++ b/kernel/sched/deadline.c @@ -2805,12 +2805,26 @@ static int find_later_rq(struct task_struct *task) =20 static struct task_struct *pick_next_pushable_dl_task(struct rq *rq) { - struct task_struct *p; + struct task_struct *i, *p =3D NULL; + struct rb_node *next_node; =20 if (!has_pushable_dl_tasks(rq)) return NULL; =20 - p =3D __node_2_pdl(rb_first_cached(&rq->dl.pushable_dl_tasks_root)); + next_node =3D rb_first_cached(&rq->dl.pushable_dl_tasks_root); + while (next_node) { + i =3D __node_2_pdl(next_node); + /* make sure task isn't on_cpu (possible with proxy-exec) */ + if (!task_on_cpu(rq, i)) { + p =3D i; + break; + } + + next_node =3D rb_next(next_node); + } + + if (!p) + return NULL; =20 WARN_ON_ONCE(rq->cpu !=3D task_cpu(p)); WARN_ON_ONCE(task_current(rq, p)); diff --git a/kernel/sched/rt.c b/kernel/sched/rt.c index 3d823f5..4e5f195 100644 --- a/kernel/sched/rt.c +++ b/kernel/sched/rt.c @@ -1858,13 +1858,22 @@ static int find_lowest_rq(struct task_struct *task) =20 static struct task_struct *pick_next_pushable_task(struct rq *rq) { - struct task_struct *p; + struct plist_head *head =3D &rq->rt.pushable_tasks; + struct task_struct *i, *p =3D NULL; =20 if (!has_pushable_tasks(rq)) return NULL; =20 - p =3D plist_first_entry(&rq->rt.pushable_tasks, - struct task_struct, pushable_tasks); + plist_for_each_entry(i, head, pushable_tasks) { + /* make sure task isn't on_cpu (possible with proxy-exec) */ + if (!task_on_cpu(rq, i)) { + p =3D i; + break; + } + } + + if (!p) + return NULL; =20 BUG_ON(rq->cpu !=3D task_cpu(p)); BUG_ON(task_current(rq, p));