[PATCH mptcp-net] mptcp: fix possible deadlock in subflow_error_report

Paolo Abeni posted 1 patch 1 year, 1 month ago
Patches applied successfully (tree, apply log)
git fetch https://github.com/multipath-tcp/mptcp_net-next tags/patchew/c2265383cb2190247f7dec66fb67182e558bf5a1.1676889543.git.pabeni@redhat.com
Maintainers: Matthieu Baerts <matthieu.baerts@tessares.net>, "David S. Miller" <davem@davemloft.net>, Eric Dumazet <edumazet@google.com>, Jakub Kicinski <kuba@kernel.org>, Paolo Abeni <pabeni@redhat.com>, Mat Martineau <mathew.j.martineau@linux.intel.com>
net/mptcp/subflow.c | 7 +++++++
1 file changed, 7 insertions(+)
[PATCH mptcp-net] mptcp: fix possible deadlock in subflow_error_report
Posted by Paolo Abeni 1 year, 1 month ago
Christoph reported a possible deadlock while the TCP stack
destroys an unaccepted subflow due to an incoming reset: the
MPTCP socket error path tries to acquire the msk-level socket
lock while TCP still owns the listener socket accept queue
spinlock, and the reverse dependency already exists in the
TCP stack.

Note that the above is actually a lockdep false positive, as
the chain involves two separate sockets. A different per-socket
lockdep key will address the issue, but such a change will be
quite invasive.

Instead, we can simply stop earlier the socket error handling
for orphaned or unaccepted subflows, breaking the critical
lockdep chain. Error handling in such a scenario is a no-op.

Reported-and-tested-by: Christoph Paasch <cpaasch@apple.com>
Fixes: 15cc10453398 ("mptcp: deliver ssk errors to msk")
Closes: https://github.com/multipath-tcp/mptcp_net-next/issues/355
Signed-off-by: Paolo Abeni <pabeni@redhat.com>
---
Note that the fixes tag above is different from the bisected, much
more recent, commit pointed by syzkaller, but I believe (code
inspection shows) that the DL scenario is present since it.
---
 net/mptcp/subflow.c | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/net/mptcp/subflow.c b/net/mptcp/subflow.c
index 8b46311b8d5e..a14c3a0c7c00 100644
--- a/net/mptcp/subflow.c
+++ b/net/mptcp/subflow.c
@@ -1437,6 +1437,13 @@ static void subflow_error_report(struct sock *ssk)
 {
 	struct sock *sk = mptcp_subflow_ctx(ssk)->conn;
 
+	/* bail early if this is a no-op, so that we avoid introducing a
+	 * problematic lockdep dependency between TCP accept queue lock
+	 * and msk socket spinlock
+	 */
+	if (!sk->sk_socket)
+		return;
+
 	mptcp_data_lock(sk);
 	if (!sock_owned_by_user(sk))
 		__mptcp_error_report(sk);
-- 
2.39.2
Re: [PATCH mptcp-net] mptcp: fix possible deadlock in subflow_error_report
Posted by Matthieu Baerts 1 year, 1 month ago
Hi Paolo,

On 20/02/2023 15:27, Paolo Abeni wrote:
> Christoph reported a possible deadlock while the TCP stack
> destroys an unaccepted subflow due to an incoming reset: the
> MPTCP socket error path tries to acquire the msk-level socket
> lock while TCP still owns the listener socket accept queue
> spinlock, and the reverse dependency already exists in the
> TCP stack.
> 
> Note that the above is actually a lockdep false positive, as
> the chain involves two separate sockets. A different per-socket
> lockdep key will address the issue, but such a change will be
> quite invasive.
> 
> Instead, we can simply stop earlier the socket error handling
> for orphaned or unaccepted subflows, breaking the critical
> lockdep chain. Error handling in such a scenario is a no-op.
> 
> Reported-and-tested-by: Christoph Paasch <cpaasch@apple.com>
> Fixes: 15cc10453398 ("mptcp: deliver ssk errors to msk")
> Closes: https://github.com/multipath-tcp/mptcp_net-next/issues/355
> Signed-off-by: Paolo Abeni <pabeni@redhat.com>
> ---
> Note that the fixes tag above is different from the bisected, much
> more recent, commit pointed by syzkaller, but I believe (code
> inspection shows) that the DL scenario is present since it.

Thank you for the patch and the note here!

Just applied in our tree (fixes for -net) with my RvB tag:

New patches for t/upstream-net:
- b06fdd2ba8a7: mptcp: fix possible deadlock in subflow_error_report
- Results: 1231bdd38564..275b1fab043a (export-net)
- Results: 1e2b17c72708..ce3a45c82f32 (export)

Tests are now in progress:

https://cirrus-ci.com/github/multipath-tcp/mptcp_net-next/export-net/20230220T160207
https://cirrus-ci.com/github/multipath-tcp/mptcp_net-next/export/20230220T160207

Cheers,
Matt
-- 
Tessares | Belgium | Hybrid Access Solutions
www.tessares.net