From nobody Sat Sep 26 11:47:16 2026 Received: from mail-pg1-f180.google.com (mail-pg1-f180.google.com [209.85.215.180]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 78BBC313E00 for ; Wed, 2 Sep 2026 04:13:10 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.215.180 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788322391; cv=none; b=qOpqpzZ4Qapan+6m+UVQqF8995JS/EET82ZIUDn91IEC5cZuXoSEsft06shQIdokLmWf5terMxibG/MVp4q0Phr3pDH5rh9jkUJc4QZEgAxPNLTrGJWZXioVJXDwTlXpPPIMXCOslp+6d1A+rS453LdVGXW1L5thYdvpdkcozaw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788322391; c=relaxed/simple; bh=wfnnTQieTn+d6PKBiadsq2iAcevEPfNp65Ncrww9voc=; h=From:To:Subject:Date:Message-ID:MIME-Version; b=YYpw5TeohSEdNBgWnZ/yASuztsdzm3dwDci9H7qHOYGP7iMHcNIcFutoTo6M2EmNTwJOz4OF9cMhHjv6xImJ+YLlBDVlNG0uxbGTdL2tefvHjqHH0m0V8fReADTg5oB3+XPuJoXBy+c5062hBOYABJyUZtPOeRRmx/VxEp9C/2c= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=Nxvf97eP; arc=none smtp.client-ip=209.85.215.180 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="Nxvf97eP" Received: by mail-pg1-f180.google.com with SMTP id 41be03b00d2f7-cc1bcdb3c4cso467215a12.2 for ; Tue, 01 Sep 2026 21:13:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1788322390; x=1788927190; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:message-id:date:subject:to :from:from:to:cc:subject:date:message-id:reply-to:content-type; bh=HJQlClbQvZ6uUjHbTDqtPdCsJtKFzLCVooXGjPJ5bEo=; b=Nxvf97ePm+6+J4JjTZ2uASoCcp4s2A5m7X8wX/FZRqEeqGpe2siyiRK2+v5JgATAnN wEfPAmkzPGzAQRL6lMfu2qH7WIQcOVTvuQ6rjWDgNQOv7cCyETeUqYfbKLr+p5yTdVhZ 6APzpeA3G7MzqsQL9qUKZLP5sj19QasDI6cHku+AoPNlbRN3+Mh5OmLELbw31MHC6C3L ggT72okFWc/S/x5Thpk1RgQucNlxuS4hrt1Ny2iI5BboWIrGxLzJXgYxir0QxmpzWy/t vd5cKrGhe0DO6YTCuzrRkZuZh+VovVBEFoBi4sXZDwaY3JMJYf6NE62QOfoN3AeiL1yV JRaQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1788322390; x=1788927190; h=content-transfer-encoding:mime-version:message-id:date:subject:to :from:x-gm-gg:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to:content-type; bh=HJQlClbQvZ6uUjHbTDqtPdCsJtKFzLCVooXGjPJ5bEo=; b=iQMRyikEBtiYFNir43cX2eBdfUE1U0oBAcvB0fbT5YxEM3JJZ1fHBEvt4ymVsJiWYN 4i8XQyRD+kD/dhYo9YpXr6lHYbwhJZmeGkKdXtgtREG4umdDVeojqHBOQnYSUla7KqBH bh9owh83W7Z6jKsXXrUnwLXq+jmbjF97qeN1xPh1mhPlwHI3YsNlozX/2Sap82rlw7Yn LpAWszFKFeKq7rZSUJyWnagrEd1bueICjsW+tYXTXQXL5wCBP1JDTnlDNxhl5ElN08XQ BRNC9vJVXjPNemmasUmSJyDKHNU43XmOVLfxeJkr442DMx8QGoxd+5tMTYA3hS9uwDpO hK8g== X-Forwarded-Encrypted: i=1; AKwUvBwvIcImwOKSY6rovU67FWvWg0kR2TcMR97bjTboxO08JEJ2mmS+wvP+u2N1/rZXpewiKXviDAzHOQxUFGk=@vger.kernel.org X-Gm-Message-State: AFuF++meazc/0iAy2G/MyXrpEtlJexIoY8vIHG3HEd1AZydT6ieqmCr6 zQNoLwnhi4igLWRn8ouYexaMtQo4Rb8Lf98atz6W5xiO9j0eEWoD/11c X-Gm-Gg: AYBFou256dS90uxqLmXKNv962Z5D4KMUwVNctmhA75XQcWtB3e/LO3ONYQUJFOjCDq1 bqsbPZ4kbQAnNslv72+LGGQLhmmjDVsR4NG01f4+8jllv21GIvPAuRoXOk63qtLWdTxG8MjXoY6 ddOaKCodgzWI1HH3xyKVlxCqYUtG6hZt7WqRQmt6FfcoOH9/hQbTFRWrBEEGyADAJxqqQmXzN1D 2LqttpBL4Na8MHWcoGD4ZVI4NqCvMZqU0c4YRwam6bQ8XG/FXBuhuvtGyeVpOm6KiSHLQQ5P6F1 EXM1ovDThNzOYpRxYoiGAZVgZgViwVeA0+hrcLpurVoR+rJuicNv/pcdjZjNxF1Pe+fXS14aAMk 6tylrO6zgBf9TjCj1S6InJ5d6OeqoKYAA8wbRflCPCwWI19vdcN57H7HtWhkTFSxvsC1a0KAbf/ 16QhTnippUgjBPEI3tXiSoGg1Qflg6CLSTFm1opCO/CLxMmaM4i0Q2FQOUTn2hA7/BPS5X6Mgv+ RX++L6uwN9SGIIdCTo= X-Received: by 2002:a17:90b:1d4c:b0:398:a145:5d3d with SMTP id 98e67ed59e1d1-39aedf0956bmr2868438a91.6.1788322389608; Tue, 01 Sep 2026 21:13:09 -0700 (PDT) Received: from localhost.localdomain ([103.120.31.178]) by smtp.gmail.com with ESMTPSA id 5a478bee46e88-32f079f1f4fsm2532214eec.11.2026.09.01.21.13.01 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Tue, 01 Sep 2026 21:13:08 -0700 (PDT) From: Khawar Ahemad To: bpf@vger.kernel.org, netdev@vger.kernel.org, linux-kernel@vger.kernel.org, magnus.karlsson@intel.com, maciej.fijalkowski@intel.com, sdf@fomichev.me, ast@kernel.org, daniel@iogearbox.net, hawk@kernel.org, john.fastabend@gmail.com, kuba@kernel.org, pabeni@redhat.com, edumazet@google.com, horms@kernel.org, syzbot+aa48b5fe7bfda62d1682@syzkaller.appspotmail.com Subject: [PATCH bpf-next v5] xsk: Fix circular locking dependency between &net->xdp.lock, &xs->mutex, and netdev_lock_ops Date: Wed, 2 Sep 2026 09:42:57 +0530 Message-ID: <20260902041257.58374-1-ahemadkhawar123@gmail.com> X-Mailer: git-send-email 2.54.0 Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" syzbot reported a circular locking dependency involving &net->xdp.lock, &port->pnodes_lock, netdev_lock_ops(), and &xs->mutex: -> #3 (&net->xdp.lock): xsk_notifier+0x3d/0x2c0 net/xdp/xsk.c:2106 ipvlan_device_event+0x310/0x4e0 drivers/net/ipvlan/ipvlan_main.c:834 unregister_netdevice_many_notify+0x808/0x18b0 net/core/dev.c:12518 -> #2 (&port->pnodes_lock): ipvlan_device_event+0x85/0x4e0 drivers/net/ipvlan/ipvlan_main.c:795 notifier_call_chain+0xb5/0x410 kernel/notifier.c:85 -> #1 (&dev_instance_lock_key / netdev_lock_ops): netdev_lock_ops include/net/netdev_lock.h:42 [inline] xsk_bind+0x331/0x11d0 net/xdp/xsk.c:1627 -> #0 (&xs->mutex): xsk_diag_fill net/xdp/xsk_diag.c:113 [inline] xsk_diag_dump+0x2e0/0x4e0 net/xdp/xsk_diag.c:166 The cycle exists through the following dependency chain: 1. xsk_diag_dump() acquired &xs->mutex while holding &net->xdp.lock (#0). 2. xsk_bind() acquired netdev_lock_ops() while holding &xs->mutex (#1). 3. Device unregistration in ipvlan_device_event() acquired &port->pnodes_lock (#2) and called xsk_notifier(), which acquired &net->xdp.lock (#3). Break the circular dependency by decoupling the locking in xsk_diag_dump() and xsk_notifier(): 1. In xsk_diag_dump(), avoid holding &net->xdp.lock while calling xsk_diag_fill(). Instead, locate the target socket under &net->xdp.lock, take a temporary socket reference via sock_hold(), release &net->xdp.lock, and call xsk_diag_fill() (which acquires &xs->mutex) with sock_put(). To preserve dump continuation across buffer exhaustion, distinguish -ENOENT (when an unbound socket is skipped) from -EMSGSIZE (when the skb is full and the cursor must be retained for the next dump callback). 2. In xsk_notifier(), split device unregistration into two phases: - First, unbind all matching sockets under &net->xdp.lock and &xs->mutex. - Then, release &net->xdp.lock and perform device queue teardown by sweeping the device queues via xsk_get_pool_from_qid() and calling xp_clear_dev(pool) outside all AF_XDP locks. Fixes: 975b11ae9077 ("xsk: add socket allocate, create and bind") Reported-by: syzbot+aa48b5fe7bfda62d1682@syzkaller.appspotmail.com Closes: https://syzkaller.appspot.com/bug?extid=3Daa48b5fe7bfda62d1682 Signed-off-by: Khawar Ahemad --- v4 -> v5: - Rebase cleanly on latest bpf-next master. - Link to v4: https://lore.kernel.org/bpf/20260826174744.3394-1-ahemadkhawa= r123@gmail.com/ v3 -> v4: - Rebase cleanly on latest bpf-next master to resolve merge conflict. - Update commit message to accurately describe the full 4-lock dependency chain (&net->xdp.lock, &port->pnodes_lock, netdev_lock_ops, &xs->mutex) from the syzbot report. - Link to v3: https://lore.kernel.org/bpf/20260826173019.2917-1-ahemadkhawa= r123@gmail.com/ v2 -> v3: - Fix direct AB-BA lock inversion in xsk_notifier() by performing device queue sweeps via xsk_get_pool_from_qid() outside &net->xdp.lock. - Eliminate &net->xdp.lock -> &xs->mutex in xsk_diag_dump() by taking a temporary socket reference under &net->xdp.lock and releasing the lock prior to xsk_diag_fill(). - Distinguish -ENOENT (skipped unbound socket) from -EMSGSIZE (buffer exhaustion) to preserve dump continuation without infinite loops. - Link to v2: https://lore.kernel.org/bpf/20260826162110.99879-1-ahemadkhaw= ar123@gmail.com/ v1 -> v2: - Avoid reordering locks in xsk_bind() to preserve errno precedence. - Link to v1: https://lore.kernel.org/bpf/20260825152152.86092-1-ahemadkhaw= ar123@gmail.com/ net/xdp/xsk.c | 16 +++++++++--- net/xdp/xsk_diag.c | 65 +++++++++++++++++++++++++++++++--------------- 2 files changed, 57 insertions(+), 24 deletions(-) diff --git a/net/xdp/xsk.c b/net/xdp/xsk.c index 7855ee09c4..e72344fccb 100644 --- a/net/xdp/xsk.c +++ b/net/xdp/xsk.c @@ -2099,7 +2099,9 @@ static int xsk_notifier(struct notifier_block *this, { struct net_device *dev =3D netdev_notifier_info_to_dev(ptr); struct net *net =3D dev_net(dev); + unsigned int max_queues; struct sock *sk; + u16 qid; =20 switch (msg) { case NETDEV_UNREGISTER: @@ -2114,13 +2116,21 @@ static int xsk_notifier(struct notifier_block *this, sk_error_report(sk); =20 xsk_unbind_dev(xs); - - /* Clear device references. */ - xp_clear_dev(xs->pool); } mutex_unlock(&xs->mutex); } mutex_unlock(&net->xdp.lock); + + /* Clear device references outside AF_XDP locks to avoid + * lock inversion with netdev_lock_ops(). + */ + max_queues =3D max(dev->real_num_rx_queues, dev->real_num_tx_queues); + for (qid =3D 0; qid < max_queues; qid++) { + struct xsk_buff_pool *pool =3D xsk_get_pool_from_qid(dev, qid); + + if (pool) + xp_clear_dev(pool); + } break; } return NOTIFY_DONE; diff --git a/net/xdp/xsk_diag.c b/net/xdp/xsk_diag.c index 0170363eb5..bad0b13064 100644 --- a/net/xdp/xsk_diag.c +++ b/net/xdp/xsk_diag.c @@ -97,6 +97,7 @@ static int xsk_diag_fill(struct sock *sk, struct sk_buff = *nlskb, struct xdp_sock *xs =3D xdp_sk(sk); struct xdp_diag_msg *msg; struct nlmsghdr *nlh; + int err =3D -EMSGSIZE; =20 nlh =3D nlmsg_put(nlskb, portid, seq, SOCK_DIAG_BY_FAMILY, sizeof(*msg), flags); @@ -111,8 +112,10 @@ static int xsk_diag_fill(struct sock *sk, struct sk_bu= ff *nlskb, sock_diag_save_cookie(sk, msg->xdiag_cookie); =20 mutex_lock(&xs->mutex); - if (READ_ONCE(xs->state) =3D=3D XSK_UNBOUND) + if (READ_ONCE(xs->state) =3D=3D XSK_UNBOUND) { + err =3D -ENOENT; goto out_nlmsg_trim; + } =20 if ((req->xdiag_show & XDP_SHOW_INFO) && xsk_diag_put_info(xs, nlskb)) goto out_nlmsg_trim; @@ -145,7 +148,7 @@ static int xsk_diag_fill(struct sock *sk, struct sk_buf= f *nlskb, out_nlmsg_trim: mutex_unlock(&xs->mutex); nlmsg_cancel(nlskb, nlh); - return -EMSGSIZE; + return err; } =20 static int xsk_diag_dump(struct sk_buff *nlskb, struct netlink_callback *c= b) @@ -153,28 +156,48 @@ static int xsk_diag_dump(struct sk_buff *nlskb, struc= t netlink_callback *cb) struct xdp_diag_req *req =3D nlmsg_data(cb->nlh); struct net *net =3D sock_net(nlskb->sk); int num =3D 0, s_num =3D cb->args[0]; - struct sock *sk; - - mutex_lock(&net->xdp.lock); - - sk_for_each(sk, &net->xdp.list) { - if (!net_eq(sock_net(sk), net)) - continue; - if (num++ < s_num) - continue; - - if (xsk_diag_fill(sk, nlskb, req, - sk_user_ns(NETLINK_CB(cb->skb).sk), - NETLINK_CB(cb->skb).portid, - cb->nlh->nlmsg_seq, NLM_F_MULTI, - sock_i_ino(sk)) < 0) { - num--; - break; + struct sock *sk, *target_sk; + int err; + + for (;;) { + target_sk =3D NULL; + num =3D 0; + + mutex_lock(&net->xdp.lock); + sk_for_each(sk, &net->xdp.list) { + if (!net_eq(sock_net(sk), net)) + continue; + if (num++ =3D=3D s_num) { + sock_hold(sk); + target_sk =3D sk; + break; + } } + mutex_unlock(&net->xdp.lock); + + if (!target_sk) + break; + + err =3D xsk_diag_fill(target_sk, nlskb, req, + sk_user_ns(NETLINK_CB(cb->skb).sk), + NETLINK_CB(cb->skb).portid, + cb->nlh->nlmsg_seq, NLM_F_MULTI, + sock_i_ino(target_sk)); + sock_put(target_sk); + + /* + * xsk_diag_fill() returns: + * 0: entry added successfully. + * -ENOENT: socket is unbound, skip it. + * -EMSGSIZE: skb is full, retry this socket on the next dump callback. + */ + if (err =3D=3D -EMSGSIZE) + break; + + s_num++; } =20 - mutex_unlock(&net->xdp.lock); - cb->args[0] =3D num; + cb->args[0] =3D s_num; return nlskb->len; } =20 --=20 2.54.0 (Apple Git-157)