[PATCH 3/5] sched: Change rcu_dereference_check_sched_domain() to rcu-sched

Peter Zijlstra posted 5 patches 4 days, 5 hours ago
[PATCH 3/5] sched: Change rcu_dereference_check_sched_domain() to rcu-sched
Posted by Peter Zijlstra 4 days, 5 hours ago
By changing rcu_dereference_check_sched_domain() to use
rcu_dereference_sched_check() it also considers preempt_disable() to
be equivalent to rcu_read_lock().

Since rcu fully implies rcu_sched this has absolutely no change in
behaviour, but it does allow removing a bunch of otherwise redundant
rcu_read_lock() noise.

Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
---
 kernel/sched/fair.c  |    9 +--------
 kernel/sched/sched.h |    2 +-
 2 files changed, 2 insertions(+), 9 deletions(-)

--- a/kernel/sched/fair.c
+++ b/kernel/sched/fair.c
@@ -12853,21 +12853,16 @@ static int sched_balance_newidle(struct
 	 */
 	rq_unpin_lock(this_rq, rf);
 
-	rcu_read_lock();
 	sd = rcu_dereference_check_sched_domain(this_rq->sd);
-	if (!sd) {
-		rcu_read_unlock();
+	if (!sd)
 		goto out;
-	}
 
 	if (!get_rd_overloaded(this_rq->rd) ||
 	    this_rq->avg_idle < sd->max_newidle_lb_cost) {
 
 		update_next_balance(sd, &next_balance);
-		rcu_read_unlock();
 		goto out;
 	}
-	rcu_read_unlock();
 
 	/*
 	 * Include sched_balance_update_blocked_averages() in the cost
@@ -12880,7 +12875,6 @@ static int sched_balance_newidle(struct
 	rq_modified_clear(this_rq);
 	raw_spin_rq_unlock(this_rq);
 
-	rcu_read_lock();
 	for_each_domain(this_cpu, sd) {
 		u64 domain_cost;
 
@@ -12930,7 +12924,6 @@ static int sched_balance_newidle(struct
 		if (pulled_task || !continue_balancing)
 			break;
 	}
-	rcu_read_unlock();
 
 	raw_spin_rq_lock(this_rq);
 
--- a/kernel/sched/sched.h
+++ b/kernel/sched/sched.h
@@ -2009,7 +2009,7 @@ queue_balance_callback(struct rq *rq,
 }
 
 #define rcu_dereference_check_sched_domain(p) \
-	rcu_dereference_check((p), lockdep_is_held(&sched_domains_mutex))
+	rcu_dereference_sched_check((p), lockdep_is_held(&sched_domains_mutex))
 
 /*
  * The domain tree (rq->sd) is protected by RCU's quiescent state transition.
Re: [PATCH 3/5] sched: Change rcu_dereference_check_sched_domain() to rcu-sched
Posted by Peter Zijlstra 3 days, 10 hours ago
On Thu, Nov 27, 2025 at 04:39:46PM +0100, Peter Zijlstra wrote:
> By changing rcu_dereference_check_sched_domain() to use
> rcu_dereference_sched_check() it also considers preempt_disable() to
> be equivalent to rcu_read_lock().
> 
> Since rcu fully implies rcu_sched this has absolutely no change in
> behaviour, but it does allow removing a bunch of otherwise redundant
> rcu_read_lock() noise.

This goes sideways with NUMABALANCING=y, it needs a little more. I'll
have a poke.
Re: [PATCH 3/5] sched: Change rcu_dereference_check_sched_domain() to rcu-sched
Posted by Peter Zijlstra 3 days, 10 hours ago
On Fri, Nov 28, 2025 at 11:57:23AM +0100, Peter Zijlstra wrote:
> On Thu, Nov 27, 2025 at 04:39:46PM +0100, Peter Zijlstra wrote:
> > By changing rcu_dereference_check_sched_domain() to use
> > rcu_dereference_sched_check() it also considers preempt_disable() to
> > be equivalent to rcu_read_lock().
> > 
> > Since rcu fully implies rcu_sched this has absolutely no change in
> > behaviour, but it does allow removing a bunch of otherwise redundant
> > rcu_read_lock() noise.
> 
> This goes sideways with NUMABALANCING=y, it needs a little more. I'll
> have a poke.

Bah, so I overlooked that rcu_dereference_sched() checks
rcu_sched_lock_map while rcu_dereference() checks rcu_lock_map.

Paul, with RCU being unified, how much sense does it make that the rcu
validation stuff is still fully separated?

Case at hand, I'm trying to remove a bunch of
rcu_read_lock()/rcu_read_unlock() noise from deep inside the scheduler
where I know IRQs are disabled.

But the rcu checking thing is still living in the separated universe and
giving me pain.
Re: [PATCH 3/5] sched: Change rcu_dereference_check_sched_domain() to rcu-sched
Posted by Paul E. McKenney 3 days, 10 hours ago
On Fri, Nov 28, 2025 at 12:04:16PM +0100, Peter Zijlstra wrote:
> On Fri, Nov 28, 2025 at 11:57:23AM +0100, Peter Zijlstra wrote:
> > On Thu, Nov 27, 2025 at 04:39:46PM +0100, Peter Zijlstra wrote:
> > > By changing rcu_dereference_check_sched_domain() to use
> > > rcu_dereference_sched_check() it also considers preempt_disable() to
> > > be equivalent to rcu_read_lock().
> > > 
> > > Since rcu fully implies rcu_sched this has absolutely no change in
> > > behaviour, but it does allow removing a bunch of otherwise redundant
> > > rcu_read_lock() noise.
> > 
> > This goes sideways with NUMABALANCING=y, it needs a little more. I'll
> > have a poke.
> 
> Bah, so I overlooked that rcu_dereference_sched() checks
> rcu_sched_lock_map while rcu_dereference() checks rcu_lock_map.
> 
> Paul, with RCU being unified, how much sense does it make that the rcu
> validation stuff is still fully separated?
> 
> Case at hand, I'm trying to remove a bunch of
> rcu_read_lock()/rcu_read_unlock() noise from deep inside the scheduler
> where I know IRQs are disabled.
> 
> But the rcu checking thing is still living in the separated universe and
> giving me pain.

Would rcu_dereference_all_check() do what you need?  It is happy with an
online CPU that RCU is watching as long as either preemption is disabled
(which includes IRQs being disabled) or any/all of rcu_read_lock(),
rcu_read_lock_bh(), and rcu_read_lock_sched().

							Thanx, Paul
Re: [PATCH 3/5] sched: Change rcu_dereference_check_sched_domain() to rcu-sched
Posted by Peter Zijlstra 3 days, 9 hours ago
On Fri, Nov 28, 2025 at 03:21:41AM -0800, Paul E. McKenney wrote:

> Would rcu_dereference_all_check() do what you need?

Yes, clearly I should have read more of that file.

Let me go try that.