From nobody Thu Oct 15 23:34:42 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7C11338D40C; Tue, 29 Sep 2026 07:17:59 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790666280; cv=none; b=EdRvHYpelvHHVSAHeMYG95dYgpoTktu/vS8LagfpKdEIjnrC9wU2+w2vGjuG8xFhAZFJ96g0MtHq3YSk724LyxAJsVRgdEKXAcNMovZq1D/k8pBAZSTvkM8x2V3dKW4J5VabL+2GYWc4NEe7417jMu7PdwVe/WZdmi/3fn/6vNc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790666280; c=relaxed/simple; bh=snEPbV1Rs5Iwmw37FZg6klbVdvKpy5Azj6dF8C6m/K4=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=KVHCm7Ovlxen9rmpLWGsxiNF22ASFfEzfYMeObPuRLeUr69JhDqWo9yip46ieKy8K57TgTykqkbZBN3Hag/UCDtyGJF9ROmGiTnctnj0kiP5x8iNNrnpztHreJyzwEHymt7/LY0khmQIVj81nVgwIPobpNYD6AMxkNi6VCnGpzw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=UQr1/Ucb; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="UQr1/Ucb" Received: by smtp.kernel.org (Postfix) with ESMTPSA id DB0A71F00893; Tue, 29 Sep 2026 07:17:55 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1790666279; bh=YKI1nQjCRL2uE20uHP6f+tRLz/3aALJMUkPrXOsf6RA=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=UQr1/UcbrS2Udwr23E9XKOfZTKssIy6dd33xd2uXULdHlDV39fOaEV7oEEITgW0l5 V5HmfeOsvplx9ODkxUmOO4igyTryxb8pMckyIurSKiRT30Gpse4CWjfdJvznU1a2Ig KwGPpI+V/VXMw4iKFihnanfTbLTCzz0KtF1h1X9BQfOL+AC2gHF8hFs4vq1fyddljl Due4q/+KOceOLH/3jIJEf3PGP9xlb/lwWUxCeMdQ4SmMoqNnL/oaWe3d7AA/x0krXm sbgrRbGaxzY9obpAhGayjFdDQdu9StK8iFd7fQ2WuwvISH9w/CbXhesLWvTS3Q9ANc +nJ5IdNtB3GNg== From: Eric Dumazet To: "David S . Miller" , Jakub Kicinski , Paolo Abeni Cc: Simon Horman , Neal Cardwell , Kuniyuki Iwashima , edumazet@google.com, netdev@vger.kernel.org, Alexander Aring , David Teigland , gfs2@lists.linux.dev, John Fastabend , Jakub Sitnicki , Sabrina Dubroca , Jiayuan Chen , Matthieu Baerts , Mat Martineau , Geliang Tang , mptcp@lists.linux.dev, Wen Gu , Dust Li , "D. Wythe" , Chuck Lever , Jeff Layton , NeilBrown , Olga Kornievskaia , Dai Ngo , Tom Talpey , Trond Myklebust , Anna Schumaker , linux-nfs@vger.kernel.org, Allison Henderson , rds-devel@oss.oracle.com, Philipp Reisner , Lars Ellenberg , =?UTF-8?q?Christoph=20B=C3=B6hmwalder?= , Jens Axboe , drbd-dev@lists.linux.dev, Keith Busch , Christoph Hellwig , Sagi Grimberg , Chaitanya Kulkarni , linux-nvme@lists.infradead.org, Ilya Dryomov , Alex Markuze , Viacheslav Dubeyko , ceph-devel@vger.kernel.org, Eric Dumazet , stable@vger.kernel.org Subject: [PATCH v3 net-next 1/9] dlm: fix send buffer backpressure handling Date: Tue, 29 Sep 2026 07:17:35 +0000 Message-ID: <20260929071743.23624-2-edumazet@kernel.org> X-Mailer: git-send-email 2.56.0.rc1.315.gc6ed9934b7-goog In-Reply-To: <20260929071743.23624-1-edumazet@kernel.org> References: <20260929071743.23624-1-edumazet@kernel.org> Precedence: bulk X-Mailing-List: mptcp@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" lowcomms.c tests and clears SOCKWQ_ASYNC_NOSPACE in con->sock->flags, but this bit has not been stored there for ten years. Commit 9cd3e072b0be ("net: rename SOCK_ASYNC_NOSPACE and SOCK_ASYNC_WAITDATA") mechanically renamed the two dlm users, then commit ceb5d58b2170 ("net: fix sock_wake_async() rcu protection") moved the bit from socket->flags to the RCU protected socket_wq->flags, where it is reachable only through sk_set_bit() and sk_clear_bit(), and is only maintained for sockets having SOCK_FASYNC set. dlm uses kernel sockets, which never have SOCK_FASYNC set, and never sets the bit itself, so the test in send_to_sock() has been false ever since. The consequence is that when sock_sendmsg() returns -EAGAIN because the socket send buffer is full, dlm no longer sets CF_APP_LIMITED, does not increment sk_write_pending, and does not return DLM_IO_END to wait for lowcomms_write_space(). It returns DLM_IO_RESCHED instead, and process_send_sockets() immediately requeues the send work. A connection to a peer that is slow to drain thus keeps cycling through sock_sendmsg() and -EAGAIN, burning CPU, instead of sleeping until TCP reports that space is available again. Test SOCK_NOSPACE instead. This is the bit that lives in socket->flags, that TCP sets whenever sendmsg() returns -EAGAIN for lack of send buffer space (tcp_sendmsg_locked() and sk_stream_wait_memory()), and that lowcomms_write_space() already clears. This restores the semantics dlm had before the bit moved. Also remove the clear_bit() of SOCKWQ_ASYNC_NOSPACE from lowcomms_write_space(), for the same reason. SCTP connections are deliberately left as they are. SCTP does not set SOCK_NOSPACE, and it never calls sk->sk_write_space(): sctp_wfree() ends up in sctp_wake_up_waiters(), which calls sctp_write_space() directly. lowcomms_write_space() is thus never invoked for an SCTP connection, and the test added here stays false, so send_to_sock() keeps returning DLM_IO_RESCHED as it does today. This is the only safe behavior, as returning DLM_IO_END would wait for a callback that never comes. Fixes: ceb5d58b2170 ("net: fix sock_wake_async() rcu protection") Cc: stable@vger.kernel.org Cc: Alexander Aring Cc: David Teigland Acked-by: Alexander Aring Reviewed-by: Kuniyuki Iwashima Signed-off-by: Eric Dumazet --- fs/dlm/lowcomms.c | 6 ++---- 1 file changed, 2 insertions(+), 4 deletions(-) diff --git a/fs/dlm/lowcomms.c b/fs/dlm/lowcomms.c index 2aff1c7c17de49c41f9fd02a1fae00bcc9e6af5b..abe9ae4c643fae061d2e168c032= 19a4f4e1bc007 100644 --- a/fs/dlm/lowcomms.c +++ b/fs/dlm/lowcomms.c @@ -522,10 +522,8 @@ static void lowcomms_write_space(struct sock *sk) clear_bit(SOCK_NOSPACE, &con->sock->flags); =20 spin_lock_bh(&con->writequeue_lock); - if (test_and_clear_bit(CF_APP_LIMITED, &con->flags)) { + if (test_and_clear_bit(CF_APP_LIMITED, &con->flags)) con->sock->sk->sk_write_pending--; - clear_bit(SOCKWQ_ASYNC_NOSPACE, &con->sock->flags); - } =20 lowcomms_queue_swork(con); spin_unlock_bh(&con->writequeue_lock); @@ -1391,7 +1389,7 @@ static int send_to_sock(struct connection *con) if (ret =3D=3D -EAGAIN || ret =3D=3D 0) { lock_sock(con->sock->sk); spin_lock_bh(&con->writequeue_lock); - if (test_bit(SOCKWQ_ASYNC_NOSPACE, &con->sock->flags) && + if (test_bit(SOCK_NOSPACE, &con->sock->flags) && !test_and_set_bit(CF_APP_LIMITED, &con->flags)) { /* Notify TCP that we're limited by the * application window size. --=20 2.56.0.rc1.315.gc6ed9934b7-goog From nobody Thu Oct 15 23:34:42 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 558E738D40C; Tue, 29 Sep 2026 07:18:04 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790666285; cv=none; b=PTu/zGa6fWw89fmLbE32LXyR7pjwtiVEOFA0V4Vb4VOKjHrk48Ck0SB0UykuPq7lnkIaaHISIP0n6JwTMvJGukc71+DIaBEla6ksAByfKQPie1UYW3QBRggHEPMZXDFLL6aKtHnZq1mszmdvX27DAYAuaunPOOcIpFEmFQ4oGBc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790666285; c=relaxed/simple; bh=5UXroT4ozZJi/qpjBlxD2oY1hjR4vPrAoH7Z9cWT+UI=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=U4Wt++NYwDNW1rD8Lnqr2x90TjAAZ47342sRDEQbL3f7t5ZB8F1mnpufXqVxQkoHlaIaE+M/wsqyYuMWjaFHbnkE7Ilz29lpro5Edd6dnsjwj04h5ydlavqyxywj544kqA0EeES5zEO7OPDV173248hFc6aNxR/lXrBoYH/64Rs= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=WehzzmjZ; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="WehzzmjZ" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 785EF1F000FF; Tue, 29 Sep 2026 07:18:00 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1790666284; bh=SFFZIu+j5I6h3HlIFMk62fFaGpeXSzP26xjR5/D9TzM=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=WehzzmjZ45IohjVy5hfTAPbHpGU1ITF9GaZBqXw/Mcx51EE93kXd2wMk7imB6cFFR 070e4lgyFBRtpB559fWxVXIimlNS3fi8i6qLrn9fxluho0WsT69njsTDI49FSEkC0e Q9+1MW702fpEd6Su7eYM1DNsgtQgj69PNjuTtvKASr8a5wvbl+LIIjjoT3D7n4QKVq G6eIlihPxfEYmvRh3OFQnjdnKUVa6g32mF+qDINFdKosiq+dnPc+0T2k2RA7DElaZM MfvZm3Wv51uiy+SkwhgFPbMn/T1odzBm2UiBZL9/OEHNC3CCSm0mRmmXgUZxdW/KSo LqqjZcbAK26WQ== From: Eric Dumazet To: "David S . Miller" , Jakub Kicinski , Paolo Abeni Cc: Simon Horman , Neal Cardwell , Kuniyuki Iwashima , edumazet@google.com, netdev@vger.kernel.org, Alexander Aring , David Teigland , gfs2@lists.linux.dev, John Fastabend , Jakub Sitnicki , Sabrina Dubroca , Jiayuan Chen , Matthieu Baerts , Mat Martineau , Geliang Tang , mptcp@lists.linux.dev, Wen Gu , Dust Li , "D. Wythe" , Chuck Lever , Jeff Layton , NeilBrown , Olga Kornievskaia , Dai Ngo , Tom Talpey , Trond Myklebust , Anna Schumaker , linux-nfs@vger.kernel.org, Allison Henderson , rds-devel@oss.oracle.com, Philipp Reisner , Lars Ellenberg , =?UTF-8?q?Christoph=20B=C3=B6hmwalder?= , Jens Axboe , drbd-dev@lists.linux.dev, Keith Busch , Christoph Hellwig , Sagi Grimberg , Chaitanya Kulkarni , linux-nvme@lists.infradead.org, Ilya Dryomov , Alex Markuze , Viacheslav Dubeyko , ceph-devel@vger.kernel.org, Eric Dumazet Subject: [PATCH v3 net-next 2/9] net: add sk_set_nospace() and sk_clear_nospace() Date: Tue, 29 Sep 2026 07:17:36 +0000 Message-ID: <20260929071743.23624-3-edumazet@kernel.org> X-Mailer: git-send-email 2.56.0.rc1.315.gc6ed9934b7-goog In-Reply-To: <20260929071743.23624-1-edumazet@kernel.org> References: <20260929071743.23624-1-edumazet@kernel.org> Precedence: bulk X-Mailing-List: mptcp@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" SOCK_NOSPACE lives in sk->sk_socket->flags and is manipulated from about thirty places in the tree, all of them open coding the sk->sk_socket dereference, some with a NULL check, some without, and some reaching the struct socket by yet another path. Add sk_set_nospace() and sk_clear_nospace() helpers and convert the core networking setters and clearers to them; the following patches convert the remaining subsystem callers so that "git grep _bit(SOCK_NOSPACE" only reports the two helpers and the remaining test_bit() sites. Callers that had no NULL check are all called from user context with a socket attached, so folding the check into the helpers only makes them more robust. No functional change intended. This is a preparation patch: a following one gives TCP a cheaper private copy of this bit, and needs a single choke point to keep it in sync. Reviewed-by: Kuniyuki Iwashima Signed-off-by: Eric Dumazet --- include/net/sock.h | 2 ++ net/core/sock.c | 37 +++++++++++++++++++++++++++++++++++-- net/core/stream.c | 6 +++--- net/ipv4/tcp.c | 4 ++-- net/ipv4/tcp_bpf.c | 2 +- net/kcm/kcmsock.c | 4 ++-- net/mptcp/protocol.c | 4 ++-- net/smc/af_smc.c | 2 +- net/smc/smc_tx.c | 6 +++--- net/tls/tls_sw.c | 2 +- 10 files changed, 52 insertions(+), 17 deletions(-) diff --git a/include/net/sock.h b/include/net/sock.h index 60ea55dc18854a9759f5df618cc8c904d2323e95..f4dd2e105171386f1b68f808175= 457c51aff183a 100644 --- a/include/net/sock.h +++ b/include/net/sock.h @@ -1129,6 +1129,8 @@ static inline void sk_forward_alloc_add(struct sock *= sk, int val) } =20 void sk_stream_write_space(struct sock *sk); +void sk_set_nospace(struct sock *sk); +void sk_clear_nospace(struct sock *sk); =20 /* OOB backlog add */ static inline void __sk_add_backlog(struct sock *sk, struct sk_buff *skb) diff --git a/net/core/sock.c b/net/core/sock.c index 2948dffcc3e1b49a9a55e30f1380ec165a88859f..11a22aec7e414152aab115e8d11= e30067ab3775f 100644 --- a/net/core/sock.c +++ b/net/core/sock.c @@ -2957,6 +2957,39 @@ void sock_kzfree_s(struct sock *sk, void *mem, int s= ize) } EXPORT_SYMBOL(sock_kzfree_s); =20 +/** + * sk_set_nospace - tell the transport a writer is waiting for space + * @sk: socket + * + * Must be called before the final check of the available send space, + * so that the transport can not miss the request and forget to call + * sk->sk_write_space() once space is available again. + */ +void sk_set_nospace(struct sock *sk) +{ + struct socket *sock =3D sk->sk_socket; + + if (sock) + set_bit(SOCK_NOSPACE, &sock->flags); +} +EXPORT_SYMBOL(sk_set_nospace); + +/** + * sk_clear_nospace - tell the transport no writer is waiting for space + * @sk: socket + * + * Called from ->sk_write_space() handlers, once send space has been + * made available to writers. + */ +void sk_clear_nospace(struct sock *sk) +{ + struct socket *sock =3D sk->sk_socket; + + if (sock) + clear_bit(SOCK_NOSPACE, &sock->flags); +} +EXPORT_SYMBOL(sk_clear_nospace); + /* It is almost wait_for_tcp_memory minus release_sock/lock_sock. I think, these locks should be removed for datagram sockets. */ @@ -2970,7 +3003,7 @@ static long sock_wait_for_wmem(struct sock *sk, long = timeo) break; if (signal_pending(current)) break; - set_bit(SOCK_NOSPACE, &sk->sk_socket->flags); + sk_set_nospace(sk); prepare_to_wait(sk_sleep(sk), &wait, TASK_INTERRUPTIBLE); if (refcount_read(&sk->sk_wmem_alloc) < READ_ONCE(sk->sk_sndbuf)) break; @@ -3011,7 +3044,7 @@ struct sk_buff *sock_alloc_send_pskb(struct sock *sk,= unsigned long header_len, break; =20 sk_set_bit(SOCKWQ_ASYNC_NOSPACE, sk); - set_bit(SOCK_NOSPACE, &sk->sk_socket->flags); + sk_set_nospace(sk); err =3D -EAGAIN; if (!timeo) goto failure; diff --git a/net/core/stream.c b/net/core/stream.c index 2d748581862d0eda8523f6f371cd2402e1dd6e01..a853b60afdc35a4735c017eb16c= 748cf2fab1b99 100644 --- a/net/core/stream.c +++ b/net/core/stream.c @@ -37,7 +37,7 @@ void sk_stream_write_space(struct sock *sk) struct socket_wq *wq; =20 if (__sk_stream_is_writeable(sk, 1) && sock) { - clear_bit(SOCK_NOSPACE, &sock->flags); + sk_clear_nospace(sk); =20 rcu_read_lock(); wq =3D rcu_dereference(sk->sk_wq); @@ -143,7 +143,7 @@ int sk_stream_wait_memory(struct sock *sk, long *timeo_= p) if (sk_stream_memory_free(sk) && !vm_wait) break; =20 - set_bit(SOCK_NOSPACE, &sk->sk_socket->flags); + sk_set_nospace(sk); sk->sk_write_pending++; ret =3D sk_wait_event(sk, ¤t_timeo, READ_ONCE(sk->sk_err) || (READ_ONCE(sk->sk_shutdown) & SEND_SHUTDOWN) || @@ -177,7 +177,7 @@ int sk_stream_wait_memory(struct sock *sk, long *timeo_= p) * When TCP receives ACK packets that make room, tcp_check_space() * only calls tcp_new_space() if SOCK_NOSPACE is set. */ - set_bit(SOCK_NOSPACE, &sk->sk_socket->flags); + sk_set_nospace(sk); err =3D -EAGAIN; goto out; do_interrupted: diff --git a/net/ipv4/tcp.c b/net/ipv4/tcp.c index 3ac4856852794736c5d49f042ecd08e4246bdd6d..1cde000cfab4704e6756872f6dd= ec16851ccc55d 100644 --- a/net/ipv4/tcp.c +++ b/net/ipv4/tcp.c @@ -607,7 +607,7 @@ __poll_t tcp_poll(struct file *file, struct socket *soc= k, poll_table *wait) mask |=3D EPOLLOUT | EPOLLWRNORM; } else { /* send SIGIO later */ sk_set_bit(SOCKWQ_ASYNC_NOSPACE, sk); - set_bit(SOCK_NOSPACE, &sk->sk_socket->flags); + sk_set_nospace(sk); =20 /* Race breaker. If space is freed after * wspace test but before the flags are set, @@ -1398,7 +1398,7 @@ int tcp_sendmsg_locked(struct sock *sk, struct msghdr= *msg, size_t size) continue; =20 wait_for_space: - set_bit(SOCK_NOSPACE, &sk->sk_socket->flags); + sk_set_nospace(sk); tcp_remove_empty_skb(sk); if (copied) tcp_push(sk, flags & ~MSG_MORE, mss_now, diff --git a/net/ipv4/tcp_bpf.c b/net/ipv4/tcp_bpf.c index 2e234d155b5e616d496611d1f367c34ed90f3000..64d74d92af52d1ca2fa3d24910a= 254a5092466f2 100644 --- a/net/ipv4/tcp_bpf.c +++ b/net/ipv4/tcp_bpf.c @@ -602,7 +602,7 @@ static int tcp_bpf_sendmsg(struct sock *sk, struct msgh= dr *msg, size_t size) goto out_err; continue; wait_for_sndbuf: - set_bit(SOCK_NOSPACE, &sk->sk_socket->flags); + sk_set_nospace(sk); wait_for_memory: err =3D sk_stream_wait_memory(sk, &timeo); if (err) { diff --git a/net/kcm/kcmsock.c b/net/kcm/kcmsock.c index 71af69d442f211996e514d8d76a4c9d658f7b352..b29c0ed9caf5220ff741584e496= 32da45626beb7 100644 --- a/net/kcm/kcmsock.c +++ b/net/kcm/kcmsock.c @@ -735,7 +735,7 @@ static void kcm_tx_work(struct work_struct *w) /* Primarily for SOCK_SEQPACKET sockets */ if (likely(sk->sk_socket) && test_bit(SOCK_NOSPACE, &sk->sk_socket->flags)) { - clear_bit(SOCK_NOSPACE, &sk->sk_socket->flags); + sk_clear_nospace(sk); sk->sk_write_space(sk); } =20 @@ -779,7 +779,7 @@ static int kcm_sendmsg(struct socket *sock, struct msgh= dr *msg, size_t len) /* Call the sk_stream functions to manage the sndbuf mem. */ if (!sk_stream_memory_free(sk)) { kcm_push(kcm); - set_bit(SOCK_NOSPACE, &sk->sk_socket->flags); + sk_set_nospace(sk); err =3D sk_stream_wait_memory(sk, &timeo); if (err) goto out_error; diff --git a/net/mptcp/protocol.c b/net/mptcp/protocol.c index e89a69ab927c9139c68c9039327cb0e55c33356b..74b1a512072878a54b229bc80aa= 76bb3e417c862 100644 --- a/net/mptcp/protocol.c +++ b/net/mptcp/protocol.c @@ -2106,7 +2106,7 @@ static int mptcp_sendmsg(struct sock *sk, struct msgh= dr *msg, size_t len) continue; =20 wait_for_memory: - set_bit(SOCK_NOSPACE, &sk->sk_socket->flags); + sk_set_nospace(sk); __mptcp_push_pending(sk, msg->msg_flags); ret =3D sk_stream_wait_memory(sk, &timeo); if (ret) @@ -4472,7 +4472,7 @@ static __poll_t mptcp_check_writeable(struct mptcp_so= ck *msk) if (__mptcp_stream_is_writeable(sk, 1)) return EPOLLOUT | EPOLLWRNORM; =20 - set_bit(SOCK_NOSPACE, &sk->sk_socket->flags); + sk_set_nospace(sk); smp_mb__after_atomic(); /* NOSPACE is changed by mptcp_write_space() */ if (__mptcp_stream_is_writeable(sk, 1)) return EPOLLOUT | EPOLLWRNORM; diff --git a/net/smc/af_smc.c b/net/smc/af_smc.c index e9f93b3ab435bbcce0a493f24205bd91bdf15b6a..4f2afbbdef2d7e62431e14562dc= 26bdfcdaa9f79 100644 --- a/net/smc/af_smc.c +++ b/net/smc/af_smc.c @@ -2919,7 +2919,7 @@ __poll_t smc_poll(struct file *file, struct socket *s= ock, mask |=3D EPOLLOUT | EPOLLWRNORM; } else { sk_set_bit(SOCKWQ_ASYNC_NOSPACE, sk); - set_bit(SOCK_NOSPACE, &sk->sk_socket->flags); + sk_set_nospace(sk); =20 if (sk->sk_state !=3D SMC_INIT) { /* Race breaker the same way as tcp_poll(). */ diff --git a/net/smc/smc_tx.c b/net/smc/smc_tx.c index 3144b4b1fe29013cabc63d72671d256761a4c367..52e5395484fd3e0c4b1356226ec= dcf4fb190d487 100644 --- a/net/smc/smc_tx.c +++ b/net/smc/smc_tx.c @@ -48,7 +48,7 @@ static void smc_tx_write_space(struct sock *sk) if (atomic_read(&smc->conn.sndbuf_space) && sock) { if (test_bit(SOCK_NOSPACE, &sock->flags)) SMC_STAT_RMB_TX_FULL(smc, !smc->conn.lnk); - clear_bit(SOCK_NOSPACE, &sock->flags); + sk_clear_nospace(sk); rcu_read_lock(); wq =3D rcu_dereference(sk->sk_wq); if (skwq_has_sleeper(wq)) @@ -100,7 +100,7 @@ static int smc_tx_wait(struct smc_sock *smc, int flags) } if (!timeo) { /* ensure EPOLLOUT is subsequently generated */ - set_bit(SOCK_NOSPACE, &sk->sk_socket->flags); + sk_set_nospace(sk); rc =3D -EAGAIN; break; } @@ -111,7 +111,7 @@ static int smc_tx_wait(struct smc_sock *smc, int flags) sk_clear_bit(SOCKWQ_ASYNC_NOSPACE, sk); if (atomic_read(&conn->sndbuf_space) && !conn->urg_tx_pend) break; /* at least 1 byte of free & no urgent data */ - set_bit(SOCK_NOSPACE, &sk->sk_socket->flags); + sk_set_nospace(sk); sk_wait_event(sk, &timeo, READ_ONCE(sk->sk_err) || (READ_ONCE(sk->sk_shutdown) & SEND_SHUTDOWN) || diff --git a/net/tls/tls_sw.c b/net/tls/tls_sw.c index d1ad31986cf2cee88afbde43a2908791eeabb0fb..312e51270f293dc4df8efbec498= 473c4c3b48d15 100644 --- a/net/tls/tls_sw.c +++ b/net/tls/tls_sw.c @@ -966,7 +966,7 @@ static int tls_sw_sendmsg_locked(struct sock *sk, struc= t msghdr *msg, continue; =20 wait_for_sndbuf: - set_bit(SOCK_NOSPACE, &sk->sk_socket->flags); + sk_set_nospace(sk); wait_for_memory: ret =3D sk_stream_wait_memory(sk, &timeo); if (ret) { --=20 2.56.0.rc1.315.gc6ed9934b7-goog From nobody Thu Oct 15 23:34:42 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 346CE38DC62; Tue, 29 Sep 2026 07:18:08 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790666290; cv=none; b=bREWtqfqUCGBUIZwdLqLOWtKF6qNM30oft1sIYT0qJGzhpipHNedSf7yyijepUigP5j0nbqFublsIwozmPdyqXsZhc4x24xowgwez6RzZYF6/0Dc74Kvt9pwymclKmgJF5UH7IyA+Aoemd+8wA1dEkNs8+kRwtWe91jLMoIC+Vo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790666290; c=relaxed/simple; bh=ldm+w9eNgKayOOg9zqGBUrUGr6woRc0ajPzZb0o6l4g=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=azWTdu0o7Nvc03wD9GPoE1BBaaDnjfJHk+otWOmw6lZNglzgAFqWcdfUlENgmtSJt8CFHh7lOx33kiWnGS8AJtfAP7pVe6hG7YviCIQpY17n3cJGn/VrUJH6+xNF1hm/U40A1SzOSXoPCYuN/g061WJPIZzRPaNTpLZxAPN63AA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=LHPMDU1s; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="LHPMDU1s" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 656C11F00893; Tue, 29 Sep 2026 07:18:05 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1790666288; bh=IJFRpgGenZGFgmE716szjfJPy4R2IRxV7+FW89IFnek=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=LHPMDU1stnoIOzkVui5b8vIjoubzbqbSkB4Jj1qRkY6vs6k1Y8cf0xmvP9HdU+YYk swI3Mapse3ZMpePDOfv9JPB2K6iAg5gqIteBCHA4COYT3fRb609BKChwFGSCxSziUc Biv+bVQseOx+MUK9r1S5tpgS7zGeRiWE0sh9iij7+BLslOUPKSFPtF8IiBWh+NDmiB T6hCU9+gd6sD7bGJw4O+ZgDRrGfEzJAVQ21WozmNFHiLL1QcS91U27S7+RXqvICDh8 0EwVWsV8vAEy1EuFG+wTLl4Px1nPEY48mKz5HPwvfvg8LuX+TzyYQ+Z1TEW7ADYcVh 1Rae7OOHJcrOQ== From: Eric Dumazet To: "David S . Miller" , Jakub Kicinski , Paolo Abeni Cc: Simon Horman , Neal Cardwell , Kuniyuki Iwashima , edumazet@google.com, netdev@vger.kernel.org, Alexander Aring , David Teigland , gfs2@lists.linux.dev, John Fastabend , Jakub Sitnicki , Sabrina Dubroca , Jiayuan Chen , Matthieu Baerts , Mat Martineau , Geliang Tang , mptcp@lists.linux.dev, Wen Gu , Dust Li , "D. Wythe" , Chuck Lever , Jeff Layton , NeilBrown , Olga Kornievskaia , Dai Ngo , Tom Talpey , Trond Myklebust , Anna Schumaker , linux-nfs@vger.kernel.org, Allison Henderson , rds-devel@oss.oracle.com, Philipp Reisner , Lars Ellenberg , =?UTF-8?q?Christoph=20B=C3=B6hmwalder?= , Jens Axboe , drbd-dev@lists.linux.dev, Keith Busch , Christoph Hellwig , Sagi Grimberg , Chaitanya Kulkarni , linux-nvme@lists.infradead.org, Ilya Dryomov , Alex Markuze , Viacheslav Dubeyko , ceph-devel@vger.kernel.org, Eric Dumazet Subject: [PATCH v3 net-next 3/9] sunrpc: use sk_set_nospace() and sk_clear_nospace() Date: Tue, 29 Sep 2026 07:17:37 +0000 Message-ID: <20260929071743.23624-4-edumazet@kernel.org> X-Mailer: git-send-email 2.56.0.rc1.315.gc6ed9934b7-goog In-Reply-To: <20260929071743.23624-1-edumazet@kernel.org> References: <20260929071743.23624-1-edumazet@kernel.org> Precedence: bulk X-Mailing-List: mptcp@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Use the new helpers instead of open coding the SOCK_NOSPACE manipulation, so that TCP can later maintain a cheaper private copy of this bit. No functional change intended. Reviewed-by: Kuniyuki Iwashima Signed-off-by: Eric Dumazet Acked-by: Chuck Lever --- net/sunrpc/svcsock.c | 4 ++-- net/sunrpc/xprtsock.c | 4 ++-- 2 files changed, 4 insertions(+), 4 deletions(-) diff --git a/net/sunrpc/svcsock.c b/net/sunrpc/svcsock.c index 50e5e7f5b762de28c34d0f58cb0c6feb1e3ce79f..2d8cdee0798d540ba8eae589dea= ac5bd3ff889fa 100644 --- a/net/sunrpc/svcsock.c +++ b/net/sunrpc/svcsock.c @@ -792,11 +792,11 @@ static int svc_udp_has_wspace(struct svc_xprt *xprt) * Set the SOCK_NOSPACE flag before checking the available * sock space. */ - set_bit(SOCK_NOSPACE, &svsk->sk_sock->flags); + sk_set_nospace(svsk->sk_sk); required =3D atomic_read(&svsk->sk_xprt.xpt_reserved) + serv->sv_max_mesg; if (required*2 > sock_wspace(svsk->sk_sk)) return 0; - clear_bit(SOCK_NOSPACE, &svsk->sk_sock->flags); + sk_clear_nospace(svsk->sk_sk); return 1; } =20 diff --git a/net/sunrpc/xprtsock.c b/net/sunrpc/xprtsock.c index 7f60723fa64d84887e260e5e130e44cdbec6f850..b97c70c12f627630509510facfd= 3c3cf96ab6cb6 100644 --- a/net/sunrpc/xprtsock.c +++ b/net/sunrpc/xprtsock.c @@ -858,7 +858,7 @@ static int xs_nospace(struct rpc_rqst *req, struct sock= _xprt *transport) if (xprt_connected(xprt)) { /* wait for more buffer space */ set_bit(XPRT_SOCK_NOSPACE, &transport->sock_state); - set_bit(SOCK_NOSPACE, &sk->sk_socket->flags); + sk_set_nospace(sk); sk->sk_write_pending++; xprt_wait_for_buffer_space(xprt); } else @@ -1615,7 +1615,7 @@ static void xs_write_space(struct sock *sk) =20 if (!sk->sk_socket) return; - clear_bit(SOCK_NOSPACE, &sk->sk_socket->flags); + sk_clear_nospace(sk); =20 if (unlikely(!(xprt =3D xprt_from_sock(sk)))) return; --=20 2.56.0.rc1.315.gc6ed9934b7-goog From nobody Thu Oct 15 23:34:42 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id ACB4D391855; Tue, 29 Sep 2026 07:18:13 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790666294; cv=none; b=c2jNIBatUC6ix6+Qw1C1DWXZ5gewPSFi2SCGOLvVGobKGQEUTE4xuZI/fig1sZYPA3DuVg2B6+ta+dMpvmNpZQ9B6DCEwUptSdwHiIprMsov+Mu7WKLlBTvzxEdYCZe96V0cWm5i4uccNsmjtyM5yvjaGFLvF9NrTHc6f3vmxvk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790666294; c=relaxed/simple; bh=O6hSXIksPZx9kHh9bRdjJ62Y0nt3rUSNhzlYGqfFT8Y=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=JiuLd3vJM2ayVt8xdu01giAAXTgwWagQv8gH0ep3baBW8LEnIMSMdwwYscSAlf2lAg8rw/p4b1iMs1vBYYAXq6cflqW/K+as83O4xGrMfQ/8iTrjQ/Tp6e6XTVbNgVvl1Jx4KcBoEoFWEDmz49+UueRC0iSVCIQ25CtGlDa65/A= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=BbuCT+m8; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="BbuCT+m8" Received: by smtp.kernel.org (Postfix) with ESMTPSA id C31591F00899; Tue, 29 Sep 2026 07:18:09 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1790666293; bh=OPajyF9FpTj0kdlawINv7kUV6GbsYhAG/0HAkx7dZ6s=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=BbuCT+m8zPy06hZWnuow0+uWftZurI4ASu5Wb63Dvh7l0cVGZT3j4uvj9VIqpVzz8 dMLqI2peRMPS/ii2Z0wjkWcoKZDuZCJhsnRRIatPEHFC4OAC2cByCn8Qk22HbiBX8o XHkBrJrYZo/RlzzdRCZXV3LNVc/lGpoqSQH4fZj1ZXRbCEUJ5EsptZTAa2qHzhzs8J xtlomL4J5ccBDKN5WiHp+gxQjTIdpSIKMrnJQFpqdgtTNQr8E9H1HolnlIlE+uQwTX 5WkZgPd4ZmXn946h7vjskb/VzO9IfS7PrORPEZGFQrCqmMhUwhA66q3I66s7QaJSdu wVNJ2CU1FzAdw== From: Eric Dumazet To: "David S . Miller" , Jakub Kicinski , Paolo Abeni Cc: Simon Horman , Neal Cardwell , Kuniyuki Iwashima , edumazet@google.com, netdev@vger.kernel.org, Alexander Aring , David Teigland , gfs2@lists.linux.dev, John Fastabend , Jakub Sitnicki , Sabrina Dubroca , Jiayuan Chen , Matthieu Baerts , Mat Martineau , Geliang Tang , mptcp@lists.linux.dev, Wen Gu , Dust Li , "D. Wythe" , Chuck Lever , Jeff Layton , NeilBrown , Olga Kornievskaia , Dai Ngo , Tom Talpey , Trond Myklebust , Anna Schumaker , linux-nfs@vger.kernel.org, Allison Henderson , rds-devel@oss.oracle.com, Philipp Reisner , Lars Ellenberg , =?UTF-8?q?Christoph=20B=C3=B6hmwalder?= , Jens Axboe , drbd-dev@lists.linux.dev, Keith Busch , Christoph Hellwig , Sagi Grimberg , Chaitanya Kulkarni , linux-nvme@lists.infradead.org, Ilya Dryomov , Alex Markuze , Viacheslav Dubeyko , ceph-devel@vger.kernel.org, Eric Dumazet Subject: [PATCH v3 net-next 4/9] rds: use sk_set_nospace() Date: Tue, 29 Sep 2026 07:17:38 +0000 Message-ID: <20260929071743.23624-5-edumazet@kernel.org> X-Mailer: git-send-email 2.56.0.rc1.315.gc6ed9934b7-goog In-Reply-To: <20260929071743.23624-1-edumazet@kernel.org> References: <20260929071743.23624-1-edumazet@kernel.org> Precedence: bulk X-Mailing-List: mptcp@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Use the new helpers instead of open coding the SOCK_NOSPACE manipulation, so that TCP can later maintain a cheaper private copy of this bit. No functional change intended. Reviewed-by: Kuniyuki Iwashima Signed-off-by: Eric Dumazet Acked-by: Allison Henderson --- net/rds/tcp_send.c | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/net/rds/tcp_send.c b/net/rds/tcp_send.c index 7c52acc749cf4d4003c749a2646c2b4f87f207fe..824222c475119d91e333123c5af= c9291005a8696 100644 --- a/net/rds/tcp_send.c +++ b/net/rds/tcp_send.c @@ -100,7 +100,7 @@ int rds_tcp_xmit(struct rds_connection *conn, struct rd= s_message *rm, =20 if (hdr_off < sizeof(struct rds_header)) { /* see rds_tcp_write_space() */ - set_bit(SOCK_NOSPACE, &tc->t_sock->sk->sk_socket->flags); + sk_set_nospace(tc->t_sock->sk); =20 ret =3D rds_tcp_sendmsg(tc->t_sock, (void *)&rm->m_inc.i_hdr + hdr_off, @@ -221,6 +221,5 @@ void rds_tcp_write_space(struct sock *sk) */ write_space(sk); =20 - if (sk->sk_socket) - set_bit(SOCK_NOSPACE, &sk->sk_socket->flags); + sk_set_nospace(sk); } --=20 2.56.0.rc1.315.gc6ed9934b7-goog From nobody Thu Oct 15 23:34:42 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id BB56C397338; Tue, 29 Sep 2026 07:18:17 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790666298; cv=none; b=oaJ4YxfOylk6HEP59Afk0KpESF7mJYcpf0Q29uiR1I03redTuThqJXF7YSTdB0EJIJg6/tYGPL6v97fX4/yL13MOdVFj2nuEske3S7wIR6hubyAdQcV7v8bg79yPHK4vO3m+A4HZY6hiM5TtBcBaGcqxxZdXi52Wqp3SGzkIje0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790666298; c=relaxed/simple; bh=7Ef0u1n4rA80U3wvj+1MNSMfNuxnKiPp45+RAjtvn+Q=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=cQBRifxFmcskO5KJSFG7fv1GFo4RDGWqiD2ZuxUA2zzDyliMNjardVFgKPVlZcNbvDTMmZUsMISArWf1NQ8v+1BaW2oDiVdqtMGjyXRoKKJ2K4m1FthDxyQhKkFdLQU1zqmForwWg6fZo0R3hqZYDKLLmLBxkhK4ijsc2qWbets= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=VwivzD3a; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="VwivzD3a" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 318A41F000FF; Tue, 29 Sep 2026 07:18:14 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1790666297; bh=hThQis9jrIYHM2qp/AUyAye12+rzpdk99d0bmSnxuAo=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=VwivzD3a6Tfbg1hAmWgU/0lvAsOcO/D2hXgF4I0INZapunQeWFiA/vY9+Pj8um46k ooyPMj18vfhDBUXjxz3dhKxrC+XN2JW9+9CM6Kpr4craL9R0MJ26TlUwOoQk0DosJu gTHOTAGt2AbJdtEAc/Se1u3UgBdqRffpKtTmiwlVip3IaR4H8tSqGTpPXp7sS/0lT7 RuQ5LJQxjPDDmef9IVOvE7Ymzh2NTA1DTF+GbYrPFmTDsuyfVORv6qnKeLEbwRO0gn iTkQMEQ0zTsXm9q0ymW/E/4EZGn948lnU6kHmrmasVInSPxWwImDhDQUPdijALzpug 1/z5MjigjDm7g== From: Eric Dumazet To: "David S . Miller" , Jakub Kicinski , Paolo Abeni Cc: Simon Horman , Neal Cardwell , Kuniyuki Iwashima , edumazet@google.com, netdev@vger.kernel.org, Alexander Aring , David Teigland , gfs2@lists.linux.dev, John Fastabend , Jakub Sitnicki , Sabrina Dubroca , Jiayuan Chen , Matthieu Baerts , Mat Martineau , Geliang Tang , mptcp@lists.linux.dev, Wen Gu , Dust Li , "D. Wythe" , Chuck Lever , Jeff Layton , NeilBrown , Olga Kornievskaia , Dai Ngo , Tom Talpey , Trond Myklebust , Anna Schumaker , linux-nfs@vger.kernel.org, Allison Henderson , rds-devel@oss.oracle.com, Philipp Reisner , Lars Ellenberg , =?UTF-8?q?Christoph=20B=C3=B6hmwalder?= , Jens Axboe , drbd-dev@lists.linux.dev, Keith Busch , Christoph Hellwig , Sagi Grimberg , Chaitanya Kulkarni , linux-nvme@lists.infradead.org, Ilya Dryomov , Alex Markuze , Viacheslav Dubeyko , ceph-devel@vger.kernel.org, Eric Dumazet Subject: [PATCH v3 net-next 5/9] dlm: use sk_set_nospace() and sk_clear_nospace() Date: Tue, 29 Sep 2026 07:17:39 +0000 Message-ID: <20260929071743.23624-6-edumazet@kernel.org> X-Mailer: git-send-email 2.56.0.rc1.315.gc6ed9934b7-goog In-Reply-To: <20260929071743.23624-1-edumazet@kernel.org> References: <20260929071743.23624-1-edumazet@kernel.org> Precedence: bulk X-Mailing-List: mptcp@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Use the new helpers instead of open coding the SOCK_NOSPACE manipulation, so that TCP can later maintain a cheaper private copy of this bit. No functional change intended. Reviewed-by: Kuniyuki Iwashima Signed-off-by: Eric Dumazet --- fs/dlm/lowcomms.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/fs/dlm/lowcomms.c b/fs/dlm/lowcomms.c index abe9ae4c643fae061d2e168c03219a4f4e1bc007..6aa68a24acab574429664acc3f5= ecc25ddd1553d 100644 --- a/fs/dlm/lowcomms.c +++ b/fs/dlm/lowcomms.c @@ -519,7 +519,7 @@ static void lowcomms_write_space(struct sock *sk) { struct connection *con =3D sock2con(sk); =20 - clear_bit(SOCK_NOSPACE, &con->sock->flags); + sk_clear_nospace(sk); =20 spin_lock_bh(&con->writequeue_lock); if (test_and_clear_bit(CF_APP_LIMITED, &con->flags)) @@ -1394,7 +1394,7 @@ static int send_to_sock(struct connection *con) /* Notify TCP that we're limited by the * application window size. */ - set_bit(SOCK_NOSPACE, &con->sock->sk->sk_socket->flags); + sk_set_nospace(con->sock->sk); con->sock->sk->sk_write_pending++; =20 clear_bit(CF_SEND_PENDING, &con->flags); --=20 2.56.0.rc1.315.gc6ed9934b7-goog From nobody Thu Oct 15 23:34:42 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9FD5238E8CA; Tue, 29 Sep 2026 07:18:22 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790666303; cv=none; b=Bo7kMKj+m4t0CGbcyVOgaGSkK6w370NSCwt8koZd4UNVGzEBy9QVaXo6jVADN1zHgO0+Uf5PsY0tx+JVsB+1jx6moym8ulz44KF8aBvufNokyTs8VUDcFGqYXmmDnt2OdRmzsYeg3kIUIr/ZD1korh3LZRf7CtXGpjWITVQvU5A= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790666303; c=relaxed/simple; bh=rDxvOjm7grrJIvYuyxm1pKCpGBApcCr6KlaorucIEUA=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=JJjtE/zF7nt73Mpu/heW8Lj1bc+n0h4O0J5QzG5c7vUVbsYczXkZSWfZy29CycUqj5ejvkerUPm9mcKKQjPUIVv320mS2fy51Ra4Vd038pQD8otB9xOCxORP29p/agyjWpIE2jzi72a1saB6ecPddNsP6sDO8W5rbt8ckFb9I7g= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=Utmyw0ZM; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="Utmyw0ZM" Received: by smtp.kernel.org (Postfix) with ESMTPSA id C9C0B1F00893; Tue, 29 Sep 2026 07:18:18 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1790666302; bh=RnWUSUoPlsuP4rnCrMGHZfdBwvKHsEHp7ONS81OIlUg=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=Utmyw0ZM96CbsgrYi7Di59lazewSJzkqgeCES4nSZti+IkvQCMxSaEX88ZKJjxOaj zUWbZuXoe7uutd0a8VirVY4s6libYi8v+CFbziQeDQ6HwyA5Hr1P2B0tKrg6pDK9G9 6EV/14xbrkSI+W3pL6lhkD/dP2yzga6gZbzWU0pJOkwFlkK4BmSrbFCpeitPYiTNkg o3atTJQsP2tS6/yI0SMxpo56E1/28mcKwHn+27jtxmmb6hW3J09MQHWkzuimU/ghVA WMuho6JPxiSj2sOwc0JQkxgLh2KpNt85qfi8fzgt2G0Kddq2PFgtzuUEmSix9n/c1I +Riy/PYJ/j7ew== From: Eric Dumazet To: "David S . Miller" , Jakub Kicinski , Paolo Abeni Cc: Simon Horman , Neal Cardwell , Kuniyuki Iwashima , edumazet@google.com, netdev@vger.kernel.org, Alexander Aring , David Teigland , gfs2@lists.linux.dev, John Fastabend , Jakub Sitnicki , Sabrina Dubroca , Jiayuan Chen , Matthieu Baerts , Mat Martineau , Geliang Tang , mptcp@lists.linux.dev, Wen Gu , Dust Li , "D. Wythe" , Chuck Lever , Jeff Layton , NeilBrown , Olga Kornievskaia , Dai Ngo , Tom Talpey , Trond Myklebust , Anna Schumaker , linux-nfs@vger.kernel.org, Allison Henderson , rds-devel@oss.oracle.com, Philipp Reisner , Lars Ellenberg , =?UTF-8?q?Christoph=20B=C3=B6hmwalder?= , Jens Axboe , drbd-dev@lists.linux.dev, Keith Busch , Christoph Hellwig , Sagi Grimberg , Chaitanya Kulkarni , linux-nvme@lists.infradead.org, Ilya Dryomov , Alex Markuze , Viacheslav Dubeyko , ceph-devel@vger.kernel.org, Eric Dumazet Subject: [PATCH v3 net-next 6/9] drbd: use sk_set_nospace() Date: Tue, 29 Sep 2026 07:17:40 +0000 Message-ID: <20260929071743.23624-7-edumazet@kernel.org> X-Mailer: git-send-email 2.56.0.rc1.315.gc6ed9934b7-goog In-Reply-To: <20260929071743.23624-1-edumazet@kernel.org> References: <20260929071743.23624-1-edumazet@kernel.org> Precedence: bulk X-Mailing-List: mptcp@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Use the new helpers instead of open coding the SOCK_NOSPACE manipulation, so that TCP can later maintain a cheaper private copy of this bit. No functional change intended. Reviewed-by: Kuniyuki Iwashima Signed-off-by: Eric Dumazet Acked-by: Christoph B=C3=B6hmwalder --- drivers/block/drbd/drbd_worker.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/drivers/block/drbd/drbd_worker.c b/drivers/block/drbd/drbd_wor= ker.c index 0697f99fed18d7e70f00b7c559a131517ee8aa42..8e50131b5a5692a8dfb99eafe70= 6602153be94b7 100644 --- a/drivers/block/drbd/drbd_worker.c +++ b/drivers/block/drbd/drbd_worker.c @@ -635,8 +635,7 @@ static int make_resync_request(struct drbd_peer_device = *const peer_device, int c int sndbuf =3D sk->sk_sndbuf; if (queued > sndbuf / 2) { requeue =3D 1; - if (sk->sk_socket) - set_bit(SOCK_NOSPACE, &sk->sk_socket->flags); + sk_set_nospace(sk); } } else requeue =3D 1; --=20 2.56.0.rc1.315.gc6ed9934b7-goog From nobody Thu Oct 15 23:34:42 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B082F3451D6; Tue, 29 Sep 2026 07:18:26 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790666307; cv=none; b=RBLjp6Jahmr5xERtxbc+XruWCXWW/F8RA54pvLKZMTzsbN/Z55Yc5oUr/V1C9Qsl7l93poLN3SuvcI4wIL4TKYyCYs0hNukJfH0RUyMAglc6xUaiy7kOQH6D4/xU1kRcED5jPMIr/SAkV9S/P4714OUlJ6UxZp+KMikE6m6zoKw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790666307; c=relaxed/simple; bh=HjoQsE6HQdzGw+KMpLI8hOVBrhCx1wyGTA1hNtXTzZQ=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=j9AxIIwr9GhjCnCHcZJf6smZghi89dGSgpPIa1iII+78Bry5957xQ87emKdLd+Pf4u7pJiwoYexkYQaXGtPwIOc7J+y2dFg6DtOsjLE/2pl6q5eOnBqdy5+/u9u2RJTcrdXtUwO0yxxX8L0DxwI6IIWDjwvTEb1tESSBYn6xabs= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=H1cdlZ0K; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="H1cdlZ0K" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 159F81F00898; Tue, 29 Sep 2026 07:18:23 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1790666306; bh=SzyVW5II+GzZ2Kkc45gez6YC7wp+NQFrBHSESlX7gQI=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=H1cdlZ0KoyRcwCIuvoJcNR/Y4PP9Q7y1AW2bQfnybfPcty/KPw69k279QkahNdmMQ AcMS3CTyIBCTR8WlcE0it4bnS8sRPN7WyUBPDJ3iDhKqaJ1qo5XbU4CEdCso1ZqkM/ JQ9SFbTDWRw1COa8Fmi+dUuFRCy4FzTvhhAm4jU8Uv19cIncCZCeyyUIsVT3IP/W4O v3MBnLIH+LQMqy8Rf21QfbOQDSpZWcSKtOehWAemHNSySobnSvhmc/p6PvQHiD5pTz eZAs9ej6HDQJLaL8gj/P1EB+//rV0i8X0PDaF712sNC/Erv1bcnqWzzNh+EkC2vWhj ZHCvKJcaFL7nw== From: Eric Dumazet To: "David S . Miller" , Jakub Kicinski , Paolo Abeni Cc: Simon Horman , Neal Cardwell , Kuniyuki Iwashima , edumazet@google.com, netdev@vger.kernel.org, Alexander Aring , David Teigland , gfs2@lists.linux.dev, John Fastabend , Jakub Sitnicki , Sabrina Dubroca , Jiayuan Chen , Matthieu Baerts , Mat Martineau , Geliang Tang , mptcp@lists.linux.dev, Wen Gu , Dust Li , "D. Wythe" , Chuck Lever , Jeff Layton , NeilBrown , Olga Kornievskaia , Dai Ngo , Tom Talpey , Trond Myklebust , Anna Schumaker , linux-nfs@vger.kernel.org, Allison Henderson , rds-devel@oss.oracle.com, Philipp Reisner , Lars Ellenberg , =?UTF-8?q?Christoph=20B=C3=B6hmwalder?= , Jens Axboe , drbd-dev@lists.linux.dev, Keith Busch , Christoph Hellwig , Sagi Grimberg , Chaitanya Kulkarni , linux-nvme@lists.infradead.org, Ilya Dryomov , Alex Markuze , Viacheslav Dubeyko , ceph-devel@vger.kernel.org, Eric Dumazet Subject: [PATCH v3 net-next 7/9] nvme-tcp: use sk_clear_nospace() Date: Tue, 29 Sep 2026 07:17:41 +0000 Message-ID: <20260929071743.23624-8-edumazet@kernel.org> X-Mailer: git-send-email 2.56.0.rc1.315.gc6ed9934b7-goog In-Reply-To: <20260929071743.23624-1-edumazet@kernel.org> References: <20260929071743.23624-1-edumazet@kernel.org> Precedence: bulk X-Mailing-List: mptcp@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Use the new helpers instead of open coding the SOCK_NOSPACE manipulation, so that TCP can later maintain a cheaper private copy of this bit. No functional change intended. Reviewed-by: Kuniyuki Iwashima Signed-off-by: Eric Dumazet --- drivers/nvme/host/tcp.c | 2 +- drivers/nvme/target/tcp.c | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/drivers/nvme/host/tcp.c b/drivers/nvme/host/tcp.c index 921934028e0b22c2e4c9fd60d09ed8e50051262d..b853ca2d598fa4cb8ea9cb66513= f1e1070cd1fb4 100644 --- a/drivers/nvme/host/tcp.c +++ b/drivers/nvme/host/tcp.c @@ -1133,7 +1133,7 @@ static void nvme_tcp_write_space(struct sock *sk) read_lock_bh(&sk->sk_callback_lock); queue =3D sk->sk_user_data; if (likely(queue && sk_stream_is_writeable(sk))) { - clear_bit(SOCK_NOSPACE, &sk->sk_socket->flags); + sk_clear_nospace(sk); /* Ensure pending TLS partial records are retried */ if (nvme_tcp_queue_tls(queue)) queue->write_space(sk); diff --git a/drivers/nvme/target/tcp.c b/drivers/nvme/target/tcp.c index e598101752620448423925d2385b606e0b4346e8..c38ea3501ea5def168b6f180b56= 734b99d21bdea 100644 --- a/drivers/nvme/target/tcp.c +++ b/drivers/nvme/target/tcp.c @@ -1686,7 +1686,7 @@ static void nvmet_tcp_write_space(struct sock *sk) } =20 if (sk_stream_is_writeable(sk)) { - clear_bit(SOCK_NOSPACE, &sk->sk_socket->flags); + sk_clear_nospace(sk); queue_work_on(queue_cpu(queue), nvmet_tcp_wq, &queue->io_work); } out: --=20 2.56.0.rc1.315.gc6ed9934b7-goog From nobody Thu Oct 15 23:34:42 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id EB16438DC45; Tue, 29 Sep 2026 07:18:30 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790666312; cv=none; b=XE0v1vYEd5rZ/U6qlmWRNlclhoB4IEDlCp1qnDYMFSFDXvs95VYX16fcPHQANQzhv6Tl5ztmig9j6A4jdxBI6d5+86Rb1MUKhg+6EGZun8m1nUp+leK2YD2P1WoWpo3aubouYfyBVlPZNWUPIvz/QSLdfa1UvLArDXamHlEqcUk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790666312; c=relaxed/simple; bh=hLjVNr62yYkjIgaGFa9SsdIqSYR1I5cZ9kOXmpD4/dI=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=PfVhGcxGGyNp/4woSZpHev9+zvu4o6UwN4eg68cmPrkWPcVo3DhQ/E0dCev/pSQegrU1wgWTovHWtf0bRdDOj+6Qr0IcmHmnlI6WzAaLXAvYizAKS2g3chuDSYh3eWWjZa/oV/oZnEAZh1wA5taerltIswR3Q+QhXpEP1UcU70o= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=FmRxfTDP; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="FmRxfTDP" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 6D8C91F000FF; Tue, 29 Sep 2026 07:18:27 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1790666310; bh=984oS/2M5KIktVHs6exbd7VGWIOJaQY+nCN+ILc+rzw=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=FmRxfTDPEZousgH1mjwF32qw+pB3arJrOFJKVf9mYNrECcwDd0kY5zJEDCAeOfA+h 8QMxbbyA3qAZGDy07ZV0BxvdAAP3J68e49cPkJt6Qqx6otSeCjZ7L53yR+diox0onh 0f3FLaVkDB96yhiRoQ/51IQ8Jjw9vtUtF6xRj1BTZbdWu8Sg+6hWOhiXzqXplYNXdr A6ZOhgMIkCEDC5tqlKqrUVUUTFn3iZyu1kpzjEUylNVswdGDZ2IZTOmqFxGGutWa/U ADZwjBHN4QGRKgik1vrs/ngbSim7hOwW5sYh9KJo12K4Dl1epBaHdQ/sTKvrSDq5Xt XHscnmim4ChLA== From: Eric Dumazet To: "David S . Miller" , Jakub Kicinski , Paolo Abeni Cc: Simon Horman , Neal Cardwell , Kuniyuki Iwashima , edumazet@google.com, netdev@vger.kernel.org, Alexander Aring , David Teigland , gfs2@lists.linux.dev, John Fastabend , Jakub Sitnicki , Sabrina Dubroca , Jiayuan Chen , Matthieu Baerts , Mat Martineau , Geliang Tang , mptcp@lists.linux.dev, Wen Gu , Dust Li , "D. Wythe" , Chuck Lever , Jeff Layton , NeilBrown , Olga Kornievskaia , Dai Ngo , Tom Talpey , Trond Myklebust , Anna Schumaker , linux-nfs@vger.kernel.org, Allison Henderson , rds-devel@oss.oracle.com, Philipp Reisner , Lars Ellenberg , =?UTF-8?q?Christoph=20B=C3=B6hmwalder?= , Jens Axboe , drbd-dev@lists.linux.dev, Keith Busch , Christoph Hellwig , Sagi Grimberg , Chaitanya Kulkarni , linux-nvme@lists.infradead.org, Ilya Dryomov , Alex Markuze , Viacheslav Dubeyko , ceph-devel@vger.kernel.org, Eric Dumazet Subject: [PATCH v3 net-next 8/9] libceph: use sk_clear_nospace() Date: Tue, 29 Sep 2026 07:17:42 +0000 Message-ID: <20260929071743.23624-9-edumazet@kernel.org> X-Mailer: git-send-email 2.56.0.rc1.315.gc6ed9934b7-goog In-Reply-To: <20260929071743.23624-1-edumazet@kernel.org> References: <20260929071743.23624-1-edumazet@kernel.org> Precedence: bulk X-Mailing-List: mptcp@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Use the new helpers instead of open coding the SOCK_NOSPACE manipulation, so that TCP can later maintain a cheaper private copy of this bit. No functional change intended. Reviewed-by: Kuniyuki Iwashima Signed-off-by: Eric Dumazet --- net/ceph/messenger.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/net/ceph/messenger.c b/net/ceph/messenger.c index 212e7797f9e48edde4a4536bba6f1ff293628b5b..cc7db8c61db37a998b698566caa= 7c936fa83fe4a 100644 --- a/net/ceph/messenger.c +++ b/net/ceph/messenger.c @@ -375,7 +375,7 @@ static void ceph_sock_write_space(struct sock *sk) if (ceph_con_flag_test(con, CEPH_CON_F_WRITE_PENDING)) { if (sk_stream_is_writeable(sk)) { dout("%s %p queueing write work\n", __func__, con); - clear_bit(SOCK_NOSPACE, &sk->sk_socket->flags); + sk_clear_nospace(sk); queue_con(con); } } else { --=20 2.56.0.rc1.315.gc6ed9934b7-goog From nobody Thu Oct 15 23:34:42 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7E9F238D40C; Tue, 29 Sep 2026 07:18:35 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790666317; cv=none; b=ZoH3q04Ntm50H5UScgs4Lyo9stUaONFXHbSdIEdMWQEs1HmJwnxyE/3WFO8rSBgOQy5gcWst8W8RJIrMT3/ZGpC+Ejps6dDzSvM2cBznpcCqQ0FDTK+/CVB/EsKQO+5kfGRxp6SG3ss8O4eKLyjnnw2JtOFr7S0K9Qt/Odrk7kY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790666317; c=relaxed/simple; bh=jQmGCc2/iG+Fxzc7NtK/xVAyJnW/IOSF7+4undCVxuk=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=CScTLqNLx+oVrML1jEUKl/FKy0VHRy+7Je5mRxwulKOmMZHI4n8q2dGHOgHwfjXmXz2lY+vNzQhRFSDgX3kL+1lwbVQqmFz97W4uHAvZ2zpYtovR6jstOIhDfJGg+IEsKfkxFBw69Bslmbt1MrZegTekIIe9YUg4GViBTOXZHrs= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=QNKqNJNl; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="QNKqNJNl" Received: by smtp.kernel.org (Postfix) with ESMTPSA id B38011F00898; Tue, 29 Sep 2026 07:18:31 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1790666315; bh=k2Vfim/g6/2BnfyDLSTpDSiAzp7UsiZpcueZY9H3mdc=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=QNKqNJNl7YHdFdjAi0Dpebnz6k7OXyNfPXP8S9d5ATNnmI5ULQvWiXdm8A8c6rcWV tSmDrJUQAh4PTHVGE6Aohe0xr+6RiVGYHSbA5/vbhUx5tBrVGdI7NvgGetc4veYzy3 pOq/DK3bysdQRinO5VEk5mlRZ1SabFJHPLRpVs5Hi0M/mXdS6jcltyVPe9wUlFQQl5 IllyxSaa98ynlSIpjkmfVJ9aq83IFOjaLFGl0mxIcfNB9wBtl2+2lJOw/v/pOpdvHf bmPVLeDKyZUSOBGC13zrTRdCmHmgspdqNm7D/Fqd0MPkV69HOfquO7cCnQKbO+2SRU qRI8t5S2Xh40w== From: Eric Dumazet To: "David S . Miller" , Jakub Kicinski , Paolo Abeni Cc: Simon Horman , Neal Cardwell , Kuniyuki Iwashima , edumazet@google.com, netdev@vger.kernel.org, Alexander Aring , David Teigland , gfs2@lists.linux.dev, John Fastabend , Jakub Sitnicki , Sabrina Dubroca , Jiayuan Chen , Matthieu Baerts , Mat Martineau , Geliang Tang , mptcp@lists.linux.dev, Wen Gu , Dust Li , "D. Wythe" , Chuck Lever , Jeff Layton , NeilBrown , Olga Kornievskaia , Dai Ngo , Tom Talpey , Trond Myklebust , Anna Schumaker , linux-nfs@vger.kernel.org, Allison Henderson , rds-devel@oss.oracle.com, Philipp Reisner , Lars Ellenberg , =?UTF-8?q?Christoph=20B=C3=B6hmwalder?= , Jens Axboe , drbd-dev@lists.linux.dev, Keith Busch , Christoph Hellwig , Sagi Grimberg , Chaitanya Kulkarni , linux-nvme@lists.infradead.org, Ilya Dryomov , Alex Markuze , Viacheslav Dubeyko , ceph-devel@vger.kernel.org, Eric Dumazet Subject: [PATCH v3 net-next 9/9] tcp: add tp->tcp_nospace Date: Tue, 29 Sep 2026 07:17:43 +0000 Message-ID: <20260929071743.23624-10-edumazet@kernel.org> X-Mailer: git-send-email 2.56.0.rc1.315.gc6ed9934b7-goog In-Reply-To: <20260929071743.23624-1-edumazet@kernel.org> References: <20260929071743.23624-1-edumazet@kernel.org> Precedence: bulk X-Mailing-List: mptcp@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" tcp_check_space() runs for every incoming ACK and for every packet we send, and reads SOCK_NOSPACE from sk->sk_socket->flags. struct socket lives in its own cache line, which the TCP fast paths do not otherwise touch, so testing sk->sk_socket->flags pulls in an extra cache line that is cold when the working set is large. Add tp->tcp_nospace, a mirror of SOCK_NOSPACE placed right after tp->chrono_type in the tcp_sock_write_tx group, in a cache line both the transmit and the ACK paths already touch (bytes_sent, data_segs_out, delivered, bytes_acked, chrono_type). It fits in an existing 3-byte hole before chrono_start, so no other field moves and sizeof(struct tcp_sock) is unchanged. The two flags are now only changed from sk_set_nospace() and sk_clear_nospace(), which maintain this invariant: SOCK_NOSPACE set =3D> tp->tcp_nospace set sk_set_nospace() sets SOCK_NOSPACE before tp->tcp_nospace (using smp_mb__after_atomic() and smp_store_mb()), while sk_clear_nospace() clears tp->tcp_nospace before SOCK_NOSPACE (using smp_mb__before_atomic()). If a lockless sk_set_nospace() from tcp_poll() races with sk_clear_nospace() and SOCK_NOSPACE ends up set, set_bit(SOCK_NOSPACE) happened after clear_bit(SOCK_NOSPACE), so tp->tcp_nospace =3D 1 happens after tp->tcp_nospace =3D 0 as well. tcp_check_space() can thus test tp->tcp_nospace alone and leave the authoritative SOCK_NOSPACE test to __tcp_check_space(). The invariant is one directional on purpose: a stale tp->tcp_nospace only costs an extra call to __tcp_check_space(), which is what we do unconditionally today, while a stale SOCK_NOSPACE would cost a missed EPOLLOUT. Note the smp_mb() is kept. tcp_poll() sets the flag without the socket lock, and the store-buffer pattern it forms with tcp_check_space() needs a full barrier on both sides. MPTCP subflows share the struct socket of their parent, hence its SOCK_NOSPACE bit, which can not be mirrored in the subflow tcp_sock. Pin their tp->tcp_nospace in subflow_ulp_init(), so that they always reach __tcp_check_space() and keep the current behavior. Microbenchmark on an AMD EPYC 7B13, 64 threads, each thread calling tcp_check_space() in a loop over a private set of sockets. Numbers are cycles per call above a baseline loop that performs the work the callers already did (the cache lines tcp_write_xmit() and tcp_clean_rtx_queue() touched) but not tcp_check_space() itself, so they are the marginal cost of the function. Median of 11 runs. The set size controls whether struct socket is still cached: sockets/thread before after 1 +0.6 +0.7 256 (64 KB) +8.3 +3.9 4096 (1 MB) +22.4 +1.1 262144 (64 MB) +48.4 -0.8 Signed-off-by: Eric Dumazet --- .../networking/net_cachelines/tcp_sock.rst | 1 + include/linux/tcp.h | 3 ++ include/net/tcp.h | 30 ++++++++++++++++++- net/core/sock.c | 17 ++++++++--- net/ipv4/tcp.c | 1 + net/ipv4/tcp_input.c | 8 ++++- net/mptcp/subflow.c | 5 ++++ 7 files changed, 59 insertions(+), 6 deletions(-) diff --git a/Documentation/networking/net_cachelines/tcp_sock.rst b/Documen= tation/networking/net_cachelines/tcp_sock.rst index 0f6088c4ab8bb872e7fc86f02479592e84c0247a..103328dc409e4485a3f989c9534= 8c1bc74ab20c6 100644 --- a/Documentation/networking/net_cachelines/tcp_sock.rst +++ b/Documentation/networking/net_cachelines/tcp_sock.rst @@ -50,6 +50,7 @@ u8:1 tcp_usec_ts rea= d_mostly read_m u32 chrono_start read_write = tcp_chrono_start/stop(tcp_write_xmit,tcp_cwnd_validate,= tcp_send_syn_data) u32[3] chrono_stat read_write = tcp_chrono_start/stop(tcp_write_xmit,tcp_cwnd_validate,= tcp_send_syn_data) u8:2 chrono_type read_write = tcp_chrono_start/stop(tcp_write_xmit,tcp_cwnd_validate,= tcp_send_syn_data) +u8 tcp_nospace read_mostly = read_mostly tcp_check_space(tx);tcp_check_space(rx) u8:1 rate_app_limited = read_write tcp_rate_gen u8:1 fastopen_connect u8:1 fastopen_no_cookie diff --git a/include/linux/tcp.h b/include/linux/tcp.h index 6a8c77719322f9caee305d954a107892c76d4ef7..e4d1720f49f08f3c12f5ba66fe8= fe3379d71b1e0 100644 --- a/include/linux/tcp.h +++ b/include/linux/tcp.h @@ -269,6 +269,9 @@ struct tcp_sock { */ u32 snd_sml; /* Last byte of the most recently transmitted small packet */ u8 chrono_type; /* current chronograph type */ + u8 tcp_nospace; /* mirrors SOCK_NOSPACE, must be set whenever + * SOCK_NOSPACE is set. + */ u32 chrono_start; /* Start time in jiffies of a TCP chrono */ u32 chrono_stat[3]; /* Time in jiffies for chrono_stat stats */ u32 write_seq; /* Tail(+1) of data held in tcp send buffer */ diff --git a/include/net/tcp.h b/include/net/tcp.h index 96a1a23e47fac9b2f8717a4e129f468d40c6cfb5..08c04f7b32960e66ae5885dc937= 10569abda62e5 100644 --- a/include/net/tcp.h +++ b/include/net/tcp.h @@ -783,14 +783,42 @@ void tcp_done_with_error(struct sock *sk, int err); void tcp_reset(struct sock *sk, struct sk_buff *skb); void tcp_fin(struct sock *sk); void __tcp_check_space(struct sock *sk); + +/* Mirror of SOCK_NOSPACE in tcp_sock, maintained by sk_set_nospace() + * and sk_clear_nospace(). + * + * MPTCP subflows share the parent socket, and thus its SOCK_NOSPACE bit. + * Keep their mirror always set (see subflow_ulp_init()) so that they + * always reach __tcp_check_space() and behave as before. + */ +static inline void tcp_set_nospace(struct sock *sk) +{ + if (sk_is_tcp(sk)) { + /* pairs with smp_mb__before_atomic() in tcp_clear_nospace() */ + smp_mb__after_atomic(); + /* pairs with smp_mb() in tcp_check_space() */ + smp_store_mb(tcp_sk(sk)->tcp_nospace, 1); + } +} + +static inline void tcp_clear_nospace(struct sock *sk) +{ + if (sk_is_tcp(sk) && !sk_is_mptcp(sk)) { + WRITE_ONCE(tcp_sk(sk)->tcp_nospace, 0); + /* pairs with smp_mb__after_atomic() in tcp_set_nospace() */ + smp_mb__before_atomic(); + } +} + static inline void tcp_check_space(struct sock *sk) { /* pairs with tcp_poll() */ smp_mb(); =20 - if (sk->sk_socket && test_bit(SOCK_NOSPACE, &sk->sk_socket->flags)) + if (unlikely(READ_ONCE(tcp_sk(sk)->tcp_nospace))) __tcp_check_space(sk); } + void tcp_sack_compress_send_ack(struct sock *sk); =20 static inline void tcp_cleanup_skb(struct sk_buff *skb) diff --git a/net/core/sock.c b/net/core/sock.c index 11a22aec7e414152aab115e8d11e30067ab3775f..946f614f665b3608d1f310da908= d82de622ec5c0 100644 --- a/net/core/sock.c +++ b/net/core/sock.c @@ -2969,8 +2969,15 @@ void sk_set_nospace(struct sock *sk) { struct socket *sock =3D sk->sk_socket; =20 - if (sock) - set_bit(SOCK_NOSPACE, &sock->flags); + if (!sock) + return; + /* Set SOCK_NOSPACE before tp->tcp_nospace (paired with + * sk_clear_nospace() clearing tp->tcp_nospace before SOCK_NOSPACE) + * so a concurrent clear cannot leave SOCK_NOSPACE set with + * tp->tcp_nospace cleared. + */ + set_bit(SOCK_NOSPACE, &sock->flags); + tcp_set_nospace(sk); } EXPORT_SYMBOL(sk_set_nospace); =20 @@ -2985,8 +2992,10 @@ void sk_clear_nospace(struct sock *sk) { struct socket *sock =3D sk->sk_socket; =20 - if (sock) - clear_bit(SOCK_NOSPACE, &sock->flags); + if (!sock) + return; + tcp_clear_nospace(sk); + clear_bit(SOCK_NOSPACE, &sock->flags); } EXPORT_SYMBOL(sk_clear_nospace); =20 diff --git a/net/ipv4/tcp.c b/net/ipv4/tcp.c index 1cde000cfab4704e6756872f6ddec16851ccc55d..f44cbd3e76178a5ec5b40e420b3= 2836477c29266 100644 --- a/net/ipv4/tcp.c +++ b/net/ipv4/tcp.c @@ -5242,6 +5242,7 @@ static void __init tcp_struct_check(void) CACHELINE_ASSERT_GROUP_MEMBER(struct tcp_sock, tcp_sock_write_tx, first_t= x_mstamp); CACHELINE_ASSERT_GROUP_MEMBER(struct tcp_sock, tcp_sock_write_tx, deliver= ed_mstamp); CACHELINE_ASSERT_GROUP_MEMBER(struct tcp_sock, tcp_sock_write_tx, snd_sml= ); + CACHELINE_ASSERT_GROUP_MEMBER(struct tcp_sock, tcp_sock_write_tx, tcp_nos= pace); CACHELINE_ASSERT_GROUP_MEMBER(struct tcp_sock, tcp_sock_write_tx, chrono_= start); CACHELINE_ASSERT_GROUP_MEMBER(struct tcp_sock, tcp_sock_write_tx, chrono_= stat); CACHELINE_ASSERT_GROUP_MEMBER(struct tcp_sock, tcp_sock_write_tx, write_s= eq); diff --git a/net/ipv4/tcp_input.c b/net/ipv4/tcp_input.c index 892ff256e235272a8483a11949cc98b319dd7cc9..cdea30d07fe2c2a38c3d38da27a= d3274221d5958 100644 --- a/net/ipv4/tcp_input.c +++ b/net/ipv4/tcp_input.c @@ -6098,8 +6098,14 @@ static void tcp_new_space(struct sock *sk) */ void __tcp_check_space(struct sock *sk) { + struct socket *sock =3D sk->sk_socket; + + /* tp->tcp_nospace is only a hint, SOCK_NOSPACE is authoritative. */ + if (!sock || !test_bit(SOCK_NOSPACE, &sock->flags)) + return; + tcp_new_space(sk); - if (!test_bit(SOCK_NOSPACE, &sk->sk_socket->flags)) + if (!test_bit(SOCK_NOSPACE, &sock->flags)) tcp_chrono_stop(sk, TCP_CHRONO_SNDBUF_LIMITED); } =20 diff --git a/net/mptcp/subflow.c b/net/mptcp/subflow.c index f0a6725d2c3762def75c879e769e9da37f92215a..e297c88be503b42979886baaf43= acfb6b7e78055 100644 --- a/net/mptcp/subflow.c +++ b/net/mptcp/subflow.c @@ -2001,6 +2001,11 @@ static int subflow_ulp_init(struct sock *sk) pr_debug("subflow=3D%p, family=3D%d\n", ctx, sk->sk_family); =20 tp->is_mptcp =3D 1; + /* Subflows share the MPTCP socket, and thus its SOCK_NOSPACE bit, + * which tcp_check_space() can not mirror. Pin the mirror so that + * __tcp_check_space() always tests the shared bit. + */ + tp->tcp_nospace =3D 1; ctx->icsk_af_ops =3D icsk->icsk_af_ops; icsk->icsk_af_ops =3D subflow_default_af_ops(sk); ctx->tcp_state_change =3D sk->sk_state_change; --=20 2.56.0.rc1.315.gc6ed9934b7-goog