Eric reports that by the time we call netdev_lock_ops after
rtnl_unlock/rtnl_lock, the dev might point to an invalid device.
Add extra dev_hold/dev_put to make sure the device pointer
is still valid after rtnl_unlock.

Fixes: a0527ee2df3f ("net: hold netdev instance lock during qdisc ndo_setup_tc")
Reported-by: Eric Dumazet <edumazet@google.com>
Link: https://lore.kernel.org/netdev/20250305163732.2766420-1-sdf@fomichev.me/T/#me8dfd778ea4c4463acab55644e3f9836bc608771
Signed-off-by: Stanislav Fomichev <sdf@fomichev.me>
---
 net/sched/sch_api.c | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/net/sched/sch_api.c b/net/sched/sch_api.c
index aef39f6dc6a8..85ddb811780c 100644
--- a/net/sched/sch_api.c
+++ b/net/sched/sch_api.c
@@ -1280,11 +1280,13 @@ static struct Qdisc *qdisc_create(struct net_device *dev,
 			 * We replay the request because the device may
 			 * go away in the mean time.
 			 */
+			dev_hold(dev);
 			netdev_unlock_ops(dev);
 			rtnl_unlock();
 			request_module(NET_SCH_ALIAS_PREFIX "%s", name);
 			rtnl_lock();
 			netdev_lock_ops(dev);
+			dev_put(dev);
 			ops = qdisc_lookup_ops(kind);
 			if (ops != NULL) {
 				/* We will try again qdisc_lookup_ops,
-- 
2.48.1

On Thu, 20 Mar 2025 09:51:03 -0700 Stanislav Fomichev wrote:
>  			rtnl_lock();
>  			netdev_lock_ops(dev);
> +			dev_put(dev);
>  			ops = qdisc_lookup_ops(kind);

I'm not sure if this is a correct sequence. Do we guarantee that locks
will be taken before device is freed? I mean we do:

	dev = netdev_wait_allrefs_any()
	free_netdev(dev)
		mutex_destroy(dev->lock)

without explicitly taking rtnl_lock() or netdev_lock(), so the moment
that dev_put() is called the device may get freed from another thread
- while its locked here.

My mental model is that taking the instance lock on a dev for which we
only have a ref requires a dance implemented in __netdev_put_lock().

On 03/24, Jakub Kicinski wrote:
> On Thu, 20 Mar 2025 09:51:03 -0700 Stanislav Fomichev wrote:
> >  			rtnl_lock();
> >  			netdev_lock_ops(dev);
> > +			dev_put(dev);
> >  			ops = qdisc_lookup_ops(kind);
> 
> I'm not sure if this is a correct sequence. Do we guarantee that locks
> will be taken before device is freed? I mean we do:
> 
> 	dev = netdev_wait_allrefs_any()
> 	free_netdev(dev)
> 		mutex_destroy(dev->lock)
> 
> without explicitly taking rtnl_lock() or netdev_lock(), so the moment
> that dev_put() is called the device may get freed from another thread
> - while its locked here.
> 
> My mental model is that taking the instance lock on a dev for which we
> only have a ref requires a dance implemented in __netdev_put_lock().

Good point, didn't think about it. In this case, I think I need to
get back to exposing locked/unlocked signal back to the callers.
Even with __netdev_put_lock, there is a case where the netdev is
dead and can't be relocked. Will add some new 'bool *locked' argument
and reset it here; the caller will skip unlock when 'locked == false'.
LMK if you have better ideas, otherwise will post something tomorrow.

On Mon, 24 Mar 2025 15:22:17 -0700 Stanislav Fomichev wrote:
> > I'm not sure if this is a correct sequence. Do we guarantee that locks
> > will be taken before device is freed? I mean we do:
> > 
> > 	dev = netdev_wait_allrefs_any()
> > 	free_netdev(dev)
> > 		mutex_destroy(dev->lock)
> > 
> > without explicitly taking rtnl_lock() or netdev_lock(), so the moment
> > that dev_put() is called the device may get freed from another thread
> > - while its locked here.
> > 
> > My mental model is that taking the instance lock on a dev for which we
> > only have a ref requires a dance implemented in __netdev_put_lock().  
> 
> Good point, didn't think about it. In this case, I think I need to
> get back to exposing locked/unlocked signal back to the callers.
> Even with __netdev_put_lock, there is a case where the netdev is
> dead and can't be relocked. Will add some new 'bool *locked' argument
> and reset it here; the caller will skip unlock when 'locked == false'.
> LMK if you have better ideas, otherwise will post something tomorrow.

I wonder if we can bubble up this module loading business all the way
to tc_modify_qdisc(), before we look up the dev. At this point we
already checked that user has permissions, so we can load the module,
whether the request is valid or not? Instead of adding another bool
we can probably kill the "replay" silliness.