From nobody Tue Dec 2 01:04:13 2025 Received: from mail-pl1-f202.google.com (mail-pl1-f202.google.com [209.85.214.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 489E833B6E7 for ; Mon, 24 Nov 2025 22:31:22 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1764023484; cv=none; b=r1a1tR1C6Lccuj/nXd0zvRYmOPwRmbG9FSj94PlTzGrSX38pAbCuShd+VhtrZtqQh+Tb5ZxvitAuEWThsg0CRYS2qQ+jOGI/ee77FQTUrKteY2quIY1DAVmTgdBxGuRY4cdCDIDSU9EbOeeSdOkpFHmHQoOWg0L751Gyi9WcsYQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1764023484; c=relaxed/simple; bh=TW8zeEM/1CA64cg/blrMfVCMKQfLJJaz6GJxdx1KyBg=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=mYOryuAVCAVUEU9KuBk5CJlDyq6/CAL6+CiDp7xqtwitCP9NBlxrDHoufebjHbqKVVK1rG9uLWQ3zP3qq70WL6iDrCZbUWITdEa5ByzENkQ/R1OK5qohD4VbKLsdoUmLDSr25ROYBrhAPSTBbA7GbWxCzOUxB5tZDn1F/YOaMMo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--jstultz.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=0LswTFzn; arc=none smtp.client-ip=209.85.214.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--jstultz.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="0LswTFzn" Received: by mail-pl1-f202.google.com with SMTP id d9443c01a7336-299d221b749so113403445ad.3 for ; Mon, 24 Nov 2025 14:31:22 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1764023481; x=1764628281; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=+eGRLCqNRsTWy8Dxi9Oqg9w9aCOF0cGLTzg7uF3KP78=; b=0LswTFzncG5ZIY7Ql47nD1l8SZepN5Pw0zn4sFraOjIcUHXAJ+4PDNEkH2e3dUGBf6 nV2pONJpqmWS0hh2r6PqbBVWsp0tCKdJyLQ68ffQyvPGVNGaeI06yPYh1E1k6ib6R6ot lQEqMRjIjMqNUxBLFFGTTlEy+98hiuC+CcAWpmtoWo9IeDhEwXK7UItFycieQKGc9feK kUBR0IvxJLPZOvgXZLCFjdaQXHb9AAu8dXcCjqRsqtCawYU57q7nKmIL7FwSAoEvSd/n LxX3mOEe3Gh7iC4LolSxGgAQEQa2Bam7uFf2DruyZSYSU4A2SOiXu9Xap5CTCUTd0yaL 2xWg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1764023481; x=1764628281; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=+eGRLCqNRsTWy8Dxi9Oqg9w9aCOF0cGLTzg7uF3KP78=; b=T3z/SL/K6KSaFVt2j6JLgQCNFwRpJo9C+S6xcm2LLDwbKmrisUFpe4atRiKpNIFiCx 9KVVVncCoc329qfoK4TJbhh6r/LmlAnJwxY0Br4a9pQa8EnsC9+v7PS7IUHVSDiOkK+k c++OW+llHupnHTU/R3jUiXacm+qVrBmRy08LcrgxzeiEqancFF+7qqZ7i0qJpZCbNFjM ULJgilEJHNhwQurV7+pb/3uPAZeTUdaS0uC9W2Z93PArLBg1Vq+PJ6bxfM+RzBnfmZE0 Mee2Ji1MVX1vw5ZvVeUD9ILvE2eSbIy9QSd7DnhkInYhSXN07kYE51xgSrdZqRcsGa+6 9AXw== X-Gm-Message-State: AOJu0YzowRY5/cxi9Y9d7jyLbu/fDAVM1Ovt6lJ3CqAoj+P40j1s79UL RMeO1AExON8qTHBsfIpkeZC1RjlRQ/4sdVodG89EpqvcXXoyyZ52MN2Yv2RdxENaMhvdQFY5U07 /WXrE3cff9VuDEYLMmpUW32y/41HTq1SsKrsoDAtj+IReXlFRcaaXgr1QIaWHdQBhU8W+pPVp+8 Ffkqtk51KrBiWc3JyejjULMFbh5t+SDFNSkUz3r+GltdnT2Tkv X-Google-Smtp-Source: AGHT+IHlD/HkJ2w3yxQ+ro9nXcGuXIJ/DYo3BOV+viWV7MhG5dfNv7OPYD0mwA/FcTH+vH8sQZAYojMstN+5 X-Received: from plbjy3.prod.google.com ([2002:a17:903:42c3:b0:269:770b:9520]) (user=jstultz job=prod-delivery.src-stubby-dispatcher) by 2002:a17:902:ebc1:b0:27e:f018:d312 with SMTP id d9443c01a7336-29b6c3c71c1mr133536365ad.1.1764023481390; Mon, 24 Nov 2025 14:31:21 -0800 (PST) Date: Mon, 24 Nov 2025 22:30:57 +0000 In-Reply-To: <20251124223111.3616950-1-jstultz@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20251124223111.3616950-1-jstultz@google.com> X-Mailer: git-send-email 2.52.0.487.g5c8c507ade-goog Message-ID: <20251124223111.3616950-6-jstultz@google.com> Subject: [PATCH v24 05/11] sched: Add logic to zap balance callbacks if we pick again From: John Stultz To: LKML Cc: John Stultz , K Prateek Nayak , Joel Fernandes , Qais Yousef , Ingo Molnar , Peter Zijlstra , Juri Lelli , Vincent Guittot , Dietmar Eggemann , Valentin Schneider , Steven Rostedt , Ben Segall , Zimuzo Ezeozue , Mel Gorman , Will Deacon , Waiman Long , Boqun Feng , "Paul E. McKenney" , Metin Kaya , Xuewen Yan , Thomas Gleixner , Daniel Lezcano , Suleiman Souhlal , kuyo chang , hupu , kernel-team@android.com Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" With proxy-exec, a task is selected to run via pick_next_task(), and then if it is a mutex blocked task, we call find_proxy_task() to find a runnable owner. If the runnable owner is on another cpu, we will need to migrate the selected donor task away, after which we will pick_again can call pick_next_task() to choose something else. However, in the first call to pick_next_task(), we may have had a balance_callback setup by the class scheduler. After we pick again, its possible pick_next_task_fair() will be called which calls sched_balance_newidle() and sched_balance_rq(). This will throw a warning: [ 8.796467] rq->balance_callback && rq->balance_callback !=3D &balance_p= ush_callback [ 8.796467] WARNING: CPU: 32 PID: 458 at kernel/sched/sched.h:1750 sched= _balance_rq+0xe92/0x1250 ... [ 8.796467] Call Trace: [ 8.796467] [ 8.796467] ? __warn.cold+0xb2/0x14e [ 8.796467] ? sched_balance_rq+0xe92/0x1250 [ 8.796467] ? report_bug+0x107/0x1a0 [ 8.796467] ? handle_bug+0x54/0x90 [ 8.796467] ? exc_invalid_op+0x17/0x70 [ 8.796467] ? asm_exc_invalid_op+0x1a/0x20 [ 8.796467] ? sched_balance_rq+0xe92/0x1250 [ 8.796467] sched_balance_newidle+0x295/0x820 [ 8.796467] pick_next_task_fair+0x51/0x3f0 [ 8.796467] __schedule+0x23a/0x14b0 [ 8.796467] ? lock_release+0x16d/0x2e0 [ 8.796467] schedule+0x3d/0x150 [ 8.796467] worker_thread+0xb5/0x350 [ 8.796467] ? __pfx_worker_thread+0x10/0x10 [ 8.796467] kthread+0xee/0x120 [ 8.796467] ? __pfx_kthread+0x10/0x10 [ 8.796467] ret_from_fork+0x31/0x50 [ 8.796467] ? __pfx_kthread+0x10/0x10 [ 8.796467] ret_from_fork_asm+0x1a/0x30 [ 8.796467] This is because if a RT task was originally picked, it will setup the rq->balance_callback with push_rt_tasks() via set_next_task_rt(). Once the task is migrated away and we pick again, we haven't processed any balance callbacks, so rq->balance_callback is not in the same state as it was the first time pick_next_task was called. To handle this, add a zap_balance_callbacks() helper function which cleans up the balance callbacks without running them. This should be ok, as we are effectively undoing the state set in the first call to pick_next_task(), and when we pick again, the new callback can be configured for the donor task actually selected. Reviewed-by: K Prateek Nayak Signed-off-by: John Stultz --- v20: * Tweaked to avoid build issues with different configs v22: * Spelling fix suggested by K Prateek * Collapsed the stub implementation to one line as suggested by K Prateek * Zap callbacks when we resched idle, as suggested by K Prateek v24: * Don't conditionalize function on CONFIG_SCHED_PROXY_EXEC as the callers will be optimized out if that is unset, and the dead function will be removed, as suggsted by K Prateek Cc: Joel Fernandes Cc: Qais Yousef Cc: Ingo Molnar Cc: Peter Zijlstra Cc: Juri Lelli Cc: Vincent Guittot Cc: Dietmar Eggemann Cc: Valentin Schneider Cc: Steven Rostedt Cc: Ben Segall Cc: Zimuzo Ezeozue Cc: Mel Gorman Cc: Will Deacon Cc: Waiman Long Cc: Boqun Feng Cc: "Paul E. McKenney" Cc: Metin Kaya Cc: Xuewen Yan Cc: K Prateek Nayak Cc: Thomas Gleixner Cc: Daniel Lezcano Cc: Suleiman Souhlal Cc: kuyo chang Cc: hupu Cc: kernel-team@android.com --- kernel/sched/core.c | 37 +++++++++++++++++++++++++++++++++++-- 1 file changed, 35 insertions(+), 2 deletions(-) diff --git a/kernel/sched/core.c b/kernel/sched/core.c index cfe71c2764558..8a3f9f63916dd 100644 --- a/kernel/sched/core.c +++ b/kernel/sched/core.c @@ -4970,6 +4970,34 @@ static inline void finish_task(struct task_struct *p= rev) smp_store_release(&prev->on_cpu, 0); } =20 +/* + * Only called from __schedule context + * + * There are some cases where we are going to re-do the action + * that added the balance callbacks. We may not be in a state + * where we can run them, so just zap them so they can be + * properly re-added on the next time around. This is similar + * handling to running the callbacks, except we just don't call + * them. + */ +static void zap_balance_callbacks(struct rq *rq) +{ + struct balance_callback *next, *head; + bool found =3D false; + + lockdep_assert_rq_held(rq); + + head =3D rq->balance_callback; + while (head) { + if (head =3D=3D &balance_push_callback) + found =3D true; + next =3D head->next; + head->next =3D NULL; + head =3D next; + } + rq->balance_callback =3D found ? &balance_push_callback : NULL; +} + static void do_balance_callbacks(struct rq *rq, struct balance_callback *h= ead) { void (*func)(struct rq *rq); @@ -6901,10 +6929,15 @@ static void __sched notrace __schedule(int sched_mo= de) rq_set_donor(rq, next); if (unlikely(task_is_blocked(next))) { next =3D find_proxy_task(rq, next, &rf); - if (!next) + if (!next) { + /* zap the balance_callbacks before picking again */ + zap_balance_callbacks(rq); goto pick_again; - if (next =3D=3D rq->idle) + } + if (next =3D=3D rq->idle) { + zap_balance_callbacks(rq); goto keep_resched; + } } picked: clear_tsk_need_resched(prev); --=20 2.52.0.487.g5c8c507ade-goog