From nobody Thu Apr 9 16:32:36 2026 Received: from mail-pj1-f73.google.com (mail-pj1-f73.google.com [209.85.216.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B3FA32FFF8F for ; Sat, 7 Mar 2026 07:40:00 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.73 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1772869202; cv=none; b=lfccZMSgiWHSd9yKcv5aasTYHVGyoF5gVi5/1sCZZY7tTQKduw5i7rieQyG4FiXhWOiKcLv3qguN3dkXIiUSNxXICyP8W6gs46sk8VtZ5UCXzwpPG/1aN8COt8J04y8GKeBj86Ocja3+xBXACLOH5UXT2r5a38v98JqUWR6H8Vg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1772869202; c=relaxed/simple; bh=OBRFG0tAKBRQu1JC6lZgugBHACMzl+VAVkSwAgpA+8Q=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=bqQbxInl4FtwAwgkcXfn84/D+vAvU+E5Jor+6RvdZ4vDGl+K8pDcpTG0Nb9GrdOOEaHWMhBP3TReBPHPoN7vD99kRnWQUbm62Lz8ingQ+PMk/0O9ZMm65UvV+mzsq6is2dWfxcJtnJOqsKs4H2RMFYg1rMmyl+a0r2soxzncBYw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--jstultz.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=hjcSaYYs; arc=none smtp.client-ip=209.85.216.73 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--jstultz.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="hjcSaYYs" Received: by mail-pj1-f73.google.com with SMTP id 98e67ed59e1d1-35983ee9f3bso6838252a91.1 for ; Fri, 06 Mar 2026 23:40:00 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1772869200; x=1773474000; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=MuNDyPW6ZGbw7bSvWxEYw0ebn6KU48xRHzPeURleqjo=; b=hjcSaYYsGUt/Dmkwjf7H88FBzjb6CzwPDuQZN4M5eS+MPF3GEamBP5P7c3ew8MxV2g Alrqi1FBUjVA9dyCeoB92RHPazbgacWbotBzPN1qieQGljf3NnppxuBMyLJBoFTXGHJh UdfMi4upmu31MlSHQ75I0KcK3a0pBT99FdE5yUuADvTgcFJUyAebEodJsLvhfKFB8ugp X5zhgecUdyDfP4/Ds9UM9JGZX3JaI9j6R0VRNssMOQ1hqz2NW2bYYHHs/PpuwA98lbPw 1t4kBWJ7xMxHdYuPaniLbpNX/bDdTTPGIK2v8MPlSvLqG7giQJjb3IuKCXoWqnsvi9aM JMDA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1772869200; x=1773474000; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=MuNDyPW6ZGbw7bSvWxEYw0ebn6KU48xRHzPeURleqjo=; b=gQbX3STHpBkxYSMg0KS6MbYJJDGwW/DyRNviSF4f3JvRHRjM1ChGth7WFr1+ECiRlk pFFeF0S+rDW7+sUoJsuOyUtgSO4/texBHBbJouDQiPnMzqS474mJQvAclfLGpPr2/HeC 1RxZyXKGl86sJkHEgfaQkxlnQstKFTSB0hH19hVTao9Lep5gmU4f0L9fpIHEjfmptjII ITeNFoDopDxaOpIpxeCikAE8BEFrCJ6VjMEFe8UGx1VyfBWUQzbQ8N60YzsuHVA5ggmR HzHDbM0IbZPsAfJGjj6je6U0/rxeGtuH0Nv4FRMibBksrEHmjjjzqbLNvHPdVn22/rw3 +h8Q== X-Gm-Message-State: AOJu0YzNUEmznt965/45VvWsav8EUMTgUx0ElQ9mpmD3f4p9uVkEUq4m f+5QcAtIT12wXgsBTWXhVkc2anjBsxmNAMXetiF9ENJ+OYdUFMTrhlL4eo5T9WG4wNWNsRH2iL/ LOaBHwH1dceqFMIoKcrenreE6gWHy8B3DT27G7I3XeBhjnvTCkhYaeG0v14GYQh2beuLGaysVoE XLwi6lIlbFdtKPxRIP3+xvEkYQi+S3zGBPJrUzzXcvkIadf7Pt X-Received: from pjpo11.prod.google.com ([2002:a17:90a:9f8b:b0:354:3742:4ba7]) (user=jstultz job=prod-delivery.src-stubby-dispatcher) by 2002:a17:90b:3dcc:b0:359:2204:a29 with SMTP id 98e67ed59e1d1-359be320b93mr4079789a91.27.1772869199553; Fri, 06 Mar 2026 23:39:59 -0800 (PST) Date: Sat, 7 Mar 2026 07:39:29 +0000 In-Reply-To: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: X-Mailer: git-send-email 2.53.0.473.g4a7958ca14-goog Message-ID: <20260307073953.3735702-1-jstultz@google.com> Subject: [RFC][PATCH] sched: Make class_schedulers avoid pushing current, and get rid of proxy_tag_curr() From: John Stultz To: LKML Cc: John Stultz , K Prateek Nayak , Peter Zijlstra , Ingo Molnar , Juri Lelli , Vincent Guittot , Dietmar Eggemann , Steven Rostedt , Ben Segall , Mel Gorman , Valentin Schneider , Suleiman Souhlal , Johannes Weiner , Joel Fernandes , Qais Yousef , zhidao su , Xuewen Yan , kuyo chang , hupu , kernel-team@android.com Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" With proxy-execution, the scheduler selects the donor, but for blocked donors, we end up running the lock owner. This caused some complexity, because the class schedulers make sure to remove the task they pick from their pushable task lists, which prevents the donor from being migrated, but there wasn't then anything to prevent rq->curr from being migrated if rq->curr !=3D rq->donor. This was sort of hacked around by calling proxy_tag_curr() on the rq->curr task if we were running something other then the donor. proxy_tag_curr() did a dequeue/enqueue pair on the rq->curr task, allowing the class schedulers to remove it from their pushable list. The dequeue/enqueue pair was wasteful, and additionally K Prateek highlighted that we didn't properly undo things when we stopped proxying, leaving the lock owner off the pushable list. After some alternative approaches were considered, Peter suggested just having the RT/DL classes just avoid migrating when task_on_cpu(). So rework pick_next_pushable_dl_task() and the rt pick_next_pushable_task() functions so that they skip over the first pushable task if it is on_cpu. Then just drop all of the proxy_tag_curr() logic. Fixes: be39617e38e0 ("sched: Fix proxy/current (push,pull)ability") Reported-by: K Prateek Nayak Closes: https://lore.kernel.org/lkml/e735cae0-2cc9-4bae-b761-fcb082ed3e94@a= md.com/ Suggested-by: Peter Zijlstra Signed-off-by: John Stultz --- Cc: Ingo Molnar Cc: Peter Zijlstra Cc: Juri Lelli Cc: Vincent Guittot Cc: Dietmar Eggemann CC: Steven Rostedt Cc: Ben Segall Cc: Mel Gorman Cc: Valentin Schneider Cc: Suleiman Souhlal Cc: K Prateek Nayak Cc: Johannes Weiner Cc: Joel Fernandes Cc: Qais Yousef Cc: zhidao su Cc: Xuewen Yan Cc: kuyo chang Cc: hupu Cc: kernel-team@android.com --- kernel/sched/core.c | 24 ------------------------ kernel/sched/deadline.c | 16 ++++++++++++++-- kernel/sched/rt.c | 15 ++++++++++++--- 3 files changed, 26 insertions(+), 29 deletions(-) diff --git a/kernel/sched/core.c b/kernel/sched/core.c index 6960c1bfc741a..88db2b2bf3d46 100644 --- a/kernel/sched/core.c +++ b/kernel/sched/core.c @@ -6725,23 +6725,6 @@ find_proxy_task(struct rq *rq, struct task_struct *d= onor, struct rq_flags *rf) } #endif /* SCHED_PROXY_EXEC */ =20 -static inline void proxy_tag_curr(struct rq *rq, struct task_struct *owner) -{ - if (!sched_proxy_exec()) - return; - /* - * pick_next_task() calls set_next_task() on the chosen task - * at some point, which ensures it is not push/pullable. - * However, the chosen/donor task *and* the mutex owner form an - * atomic pair wrt push/pull. - * - * Make sure owner we run is not pushable. Unfortunately we can - * only deal with that by means of a dequeue/enqueue cycle. :-/ - */ - dequeue_task(rq, owner, DEQUEUE_NOCLOCK | DEQUEUE_SAVE); - enqueue_task(rq, owner, ENQUEUE_NOCLOCK | ENQUEUE_RESTORE); -} - /* * __schedule() is the main scheduler function. * @@ -6891,9 +6874,6 @@ static void __sched notrace __schedule(int sched_mode) */ RCU_INIT_POINTER(rq->curr, next); =20 - if (!task_current_donor(rq, next)) - proxy_tag_curr(rq, next); - /* * The membarrier system call requires each architecture * to have a full memory barrier after updating @@ -6928,10 +6908,6 @@ static void __sched notrace __schedule(int sched_mod= e) /* Also unlocks the rq: */ rq =3D context_switch(rq, prev, next, &rf); } else { - /* In case next was already curr but just got blocked_donor */ - if (!task_current_donor(rq, next)) - proxy_tag_curr(rq, next); - rq_unpin_lock(rq, &rf); __balance_callbacks(rq); raw_spin_rq_unlock_irq(rq); diff --git a/kernel/sched/deadline.c b/kernel/sched/deadline.c index c4402542ef44f..2cf2c1ac83493 100644 --- a/kernel/sched/deadline.c +++ b/kernel/sched/deadline.c @@ -2556,12 +2556,24 @@ static int find_later_rq(struct task_struct *task) =20 static struct task_struct *pick_next_pushable_dl_task(struct rq *rq) { - struct task_struct *p; + struct task_struct *p =3D NULL; + struct rb_node *next_node; =20 if (!has_pushable_dl_tasks(rq)) return NULL; =20 - p =3D __node_2_pdl(rb_first_cached(&rq->dl.pushable_dl_tasks_root)); + next_node =3D rb_first_cached(&rq->dl.pushable_dl_tasks_root); + while (next_node) { + p =3D __node_2_pdl(next_node); + /* make sure task isn't on_cpu (possible with proxy-exec) */ + if (!task_on_cpu(rq, p)) + break; + + next_node =3D rb_next(next_node); + } + + if (!p) + return NULL; =20 WARN_ON_ONCE(rq->cpu !=3D task_cpu(p)); WARN_ON_ONCE(task_current(rq, p)); diff --git a/kernel/sched/rt.c b/kernel/sched/rt.c index fb07dcfc60a24..5dcbe776aadd2 100644 --- a/kernel/sched/rt.c +++ b/kernel/sched/rt.c @@ -1847,13 +1847,22 @@ static int find_lowest_rq(struct task_struct *task) =20 static struct task_struct *pick_next_pushable_task(struct rq *rq) { - struct task_struct *p; + struct plist_head *head =3D &rq->rt.pushable_tasks; + struct task_struct *i, *p =3D NULL; =20 if (!has_pushable_tasks(rq)) return NULL; =20 - p =3D plist_first_entry(&rq->rt.pushable_tasks, - struct task_struct, pushable_tasks); + plist_for_each_entry(i, head, pushable_tasks) { + /* make sure task isn't on_cpu (possible with proxy-exec) */ + if (!task_on_cpu(rq, i)) { + p =3D i; + break; + } + } + + if (!p) + return NULL; =20 BUG_ON(rq->cpu !=3D task_cpu(p)); BUG_ON(task_current(rq, p)); --=20 2.53.0.473.g4a7958ca14-goog