From nobody Sun Feb 8 16:12:17 2026 Delivered-To: wpasupplicant.patchew@gmail.com Received: by 2002:ab0:590e:0:0:0:0:0 with SMTP id n14csp995394uad; Fri, 29 Jul 2022 08:34:03 -0700 (PDT) X-Google-Smtp-Source: AA6agR5nzk2yBnM73S6QMyaMrut9pAHxlzm3E/Qg5cF2f3+sizgq16rv38LE8Si3RMpekm54AzUe X-Received: by 2002:a17:90b:4ac1:b0:1ef:c1ba:e717 with SMTP id mh1-20020a17090b4ac100b001efc1bae717mr4682358pjb.92.1659108842938; Fri, 29 Jul 2022 08:34:02 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1659108842; cv=none; d=google.com; s=arc-20160816; b=bJt5s46JbOTfA/GXlf+Ug1RIC28o4vQFTyYLtTJJ13AzhAGgAMtx3wkUg+qqaxLOBX 2Knv+i4gEV13MjwQeEUCr5CI0kep/jSD3BOs8QNAFzp/E8VoQzmNtPRzL//zC3QUBNJB 1sVRLWuyInWsf5bpZYPFohWHQPJ7+6sN4nXe0089Yzru5w29CAklCVmmcZE8VpJMDb3n ZLg6F0dNrtP0IljreSVEHoBoXMnwnPTbhTlNWwNtTofFZ5EUeFJXCPx31gHf3KfTgILm a69Vvp4dD/fn6xVOEt9ihp3Ou//OxeB1uudxE/5WAx9w0u1LMp7YzOFrOEpBs8S36U3I v4uA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:to:from:dkim-signature; bh=di1M1iI/C4rvUB+B/Yb9KszVmeyN1pzBSPraUBr5na0=; b=gjeJkXiXmZKnjuAshIYkl150GnLjEOYBXW7oA9fbgYKsyF1ZyEn5W7v8pgamGslKEm l7bOVig+xgxFY7jYMHLNvAtyYdj3ypO5KfewWoudzcqaLpP/lFsV3KGotGnGeHXB9Ikz vbc4fmXd8oIMVKBn6nQN//6VHotqmH0Smh7u7Ocqc9695NSoze+sG9Qn9N82pZHxRmXm MtoDzO4F0Oh5NjbxzRzcKhj5Bl/zLW26GxCPx/8FCHfiEvdcLuwMoVwv1xhTlKocDFRN 0djk9OJGDuYG5+NVwxTwc9ruJhMQj01Z138QPIp3hDrMMl4iIALhKhCLXWWePnSeHcbu i9nQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@redhat.com header.s=mimecast20190719 header.b=QOG15R1l; spf=pass (google.com: domain of mptcp+bounces-6069-wpasupplicant.patchew=gmail.com@lists.linux.dev designates 2604:1380:45e3:2400::1 as permitted sender) smtp.mailfrom="mptcp+bounces-6069-wpasupplicant.patchew=gmail.com@lists.linux.dev"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Return-Path: Received: from sv.mirrors.kernel.org (sv.mirrors.kernel.org. [2604:1380:45e3:2400::1]) by mx.google.com with ESMTPS id f3-20020a17090a4a8300b001f11257549fsi3865980pjh.84.2022.07.29.08.34.02 for (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Fri, 29 Jul 2022 08:34:02 -0700 (PDT) Received-SPF: pass (google.com: domain of mptcp+bounces-6069-wpasupplicant.patchew=gmail.com@lists.linux.dev designates 2604:1380:45e3:2400::1 as permitted sender) client-ip=2604:1380:45e3:2400::1; Authentication-Results: mx.google.com; dkim=pass header.i=@redhat.com header.s=mimecast20190719 header.b=QOG15R1l; spf=pass (google.com: domain of mptcp+bounces-6069-wpasupplicant.patchew=gmail.com@lists.linux.dev designates 2604:1380:45e3:2400::1 as permitted sender) smtp.mailfrom="mptcp+bounces-6069-wpasupplicant.patchew=gmail.com@lists.linux.dev"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sv.mirrors.kernel.org (Postfix) with ESMTPS id 703D5280E34 for ; Fri, 29 Jul 2022 15:34:02 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id C8AE04C67; Fri, 29 Jul 2022 15:34:01 +0000 (UTC) X-Original-To: mptcp@lists.linux.dev Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5949A4A29 for ; Fri, 29 Jul 2022 15:33:57 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1659108836; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=di1M1iI/C4rvUB+B/Yb9KszVmeyN1pzBSPraUBr5na0=; b=QOG15R1lcpJSXs6oVhrVb5dDCx6zJNqIw9KD/pKI8RhPGrVr5bU81j9A31mJ54foYeJzRi 3sUhmGLlz+PtEni3vj/7ZnEQOz8n4lA0TsTt/gebX8lHXELLrdbjun2Nv77Kp2uVjHfMYR oH/Zpo1p/HJNiyxHtTi1PXEwi9Q1iOI= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-460-2KXdJmfdNBqGFig2Sg9m3A-1; Fri, 29 Jul 2022 11:33:54 -0400 X-MC-Unique: 2KXdJmfdNBqGFig2Sg9m3A-1 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.rdu2.redhat.com [10.11.54.5]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id B1D738032F0 for ; Fri, 29 Jul 2022 15:33:54 +0000 (UTC) Received: from gerbillo.redhat.com (unknown [10.39.192.205]) by smtp.corp.redhat.com (Postfix) with ESMTP id 41C0790A11 for ; Fri, 29 Jul 2022 15:33:54 +0000 (UTC) From: Paolo Abeni To: mptcp@lists.linux.dev Subject: [PATCH mptcp-next 3/4] mptcp: move msk input path under full msk socket lock Date: Fri, 29 Jul 2022 17:33:03 +0200 Message-Id: <3fdd375269bc0bad45ccadfca268dcb3ac3389c1.1659107989.git.pabeni@redhat.com> In-Reply-To: References: Precedence: bulk X-Mailing-List: mptcp@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.11.54.5 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=pabeni@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8"; x-default="true" Signed-off-by: Paolo Abeni Reported-by: kernel test robot --- net/mptcp/protocol.c | 79 ++++++++++++++++++++------------------------ net/mptcp/protocol.h | 2 +- 2 files changed, 36 insertions(+), 45 deletions(-) diff --git a/net/mptcp/protocol.c b/net/mptcp/protocol.c index 17e2dbe43639..b9402a13a69d 100644 --- a/net/mptcp/protocol.c +++ b/net/mptcp/protocol.c @@ -795,7 +795,7 @@ static bool move_skbs_to_msk(struct mptcp_sock *msk, st= ruct sock *ssk) return moved > 0; } =20 -void mptcp_data_ready(struct sock *sk, struct sock *ssk) +void __mptcp_data_ready(struct sock *sk, struct sock *ssk) { struct mptcp_subflow_context *subflow =3D mptcp_subflow_ctx(ssk); =20 @@ -807,10 +807,17 @@ void mptcp_data_ready(struct sock *sk, struct sock *s= sk) return; =20 /* Wake-up the reader only for in-sequence data */ - mptcp_data_lock(sk); if (move_skbs_to_msk(mptcp_sk(sk), ssk)) sk->sk_data_ready(sk); +} =20 +void mptcp_data_ready(struct sock *sk, struct sock *ssk) +{ + mptcp_data_lock(sk); + if (!sock_owned_by_user(sk)) + __mptcp_data_ready(sk, ssk); + else + __set_bit(MPTCP_DEQUEUE, &mptcp_sk(sk)->cb_flags); mptcp_data_unlock(sk); } =20 @@ -1059,6 +1066,7 @@ static void mptcp_clean_una_wakeup(struct sock *sk) mptcp_data_unlock(sk); } =20 + static void mptcp_enter_memory_pressure(struct sock *sk) { struct mptcp_subflow_context *subflow; @@ -1768,16 +1776,22 @@ static int mptcp_sendmsg(struct sock *sk, struct ms= ghdr *msg, size_t len) return copied ? : ret; } =20 -static int __mptcp_recvmsg_mskq(struct mptcp_sock *msk, +static bool __mptcp_move_skbs(struct sock *sk); + +static int __mptcp_recvmsg_mskq(struct sock *sk, struct msghdr *msg, size_t len, int flags, struct scm_timestamping_internal *tss, int *cmsg_flags) { + struct mptcp_sock *msk =3D mptcp_sk(sk); struct sk_buff *skb, *tmp; int copied =3D 0; =20 - skb_queue_walk_safe(&msk->receive_queue, skb, tmp) { + if (skb_queue_empty(&sk->sk_receive_queue) && !__mptcp_move_skbs(sk)) + return 0; + + skb_queue_walk_safe(&sk->sk_receive_queue, skb, tmp) { u32 offset =3D MPTCP_SKB_CB(skb)->offset; u32 data_len =3D skb->len - offset; u32 count =3D min_t(size_t, len - copied, data_len); @@ -1811,7 +1825,7 @@ static int __mptcp_recvmsg_mskq(struct mptcp_sock *ms= k, /* we will bulk release the skb memory later */ skb->destructor =3D NULL; WRITE_ONCE(msk->rmem_released, msk->rmem_released + skb->truesize); - __skb_unlink(skb, &msk->receive_queue); + __skb_unlink(skb, &sk->sk_receive_queue); __kfree_skb(skb); } =20 @@ -1932,16 +1946,9 @@ static void __mptcp_update_rmem(struct sock *sk) WRITE_ONCE(msk->rmem_released, 0); } =20 -static void __mptcp_splice_receive_queue(struct sock *sk) +static bool __mptcp_move_skbs(struct sock *sk) { struct mptcp_sock *msk =3D mptcp_sk(sk); - - skb_queue_splice_tail_init(&sk->sk_receive_queue, &msk->receive_queue); -} - -static bool __mptcp_move_skbs(struct mptcp_sock *msk) -{ - struct sock *sk =3D (struct sock *)msk; unsigned int moved =3D 0; bool ret, done; =20 @@ -1949,37 +1956,29 @@ static bool __mptcp_move_skbs(struct mptcp_sock *ms= k) struct sock *ssk =3D mptcp_subflow_recv_lookup(msk); bool slowpath; =20 - /* we can have data pending in the subflows only if the msk - * receive buffer was full at subflow_data_ready() time, - * that is an unlikely slow path. - */ - if (likely(!ssk)) + if (unlikely(!ssk)) break; =20 slowpath =3D lock_sock_fast(ssk); - mptcp_data_lock(sk); __mptcp_update_rmem(sk); done =3D __mptcp_move_skbs_from_subflow(msk, ssk, &moved); - mptcp_data_unlock(sk); =20 if (unlikely(ssk->sk_err)) __mptcp_error_report(sk); unlock_sock_fast(ssk, slowpath); } while (!done); =20 - /* acquire the data lock only if some input data is pending */ ret =3D moved > 0; if (!RB_EMPTY_ROOT(&msk->out_of_order_queue) || - !skb_queue_empty_lockless(&sk->sk_receive_queue)) { - mptcp_data_lock(sk); + !skb_queue_empty(&sk->sk_receive_queue)) { __mptcp_update_rmem(sk); ret |=3D __mptcp_ofo_queue(msk); - __mptcp_splice_receive_queue(sk); - mptcp_data_unlock(sk); } - if (ret) + if (ret) { + mptcp_cleanup_rbuf(msk); mptcp_check_data_fin((struct sock *)msk); - return !skb_queue_empty(&msk->receive_queue); + } + return ret; } =20 static unsigned int mptcp_inq_hint(const struct sock *sk) @@ -1987,7 +1986,7 @@ static unsigned int mptcp_inq_hint(const struct sock = *sk) const struct mptcp_sock *msk =3D mptcp_sk(sk); const struct sk_buff *skb; =20 - skb =3D skb_peek(&msk->receive_queue); + skb =3D skb_peek(&sk->sk_receive_queue); if (skb) { u64 hint_val =3D msk->ack_seq - MPTCP_SKB_CB(skb)->map_seq; =20 @@ -2033,7 +2032,7 @@ static int mptcp_recvmsg(struct sock *sk, struct msgh= dr *msg, size_t len, while (copied < len) { int bytes_read; =20 - bytes_read =3D __mptcp_recvmsg_mskq(msk, msg, len - copied, flags, &tss,= &cmsg_flags); + bytes_read =3D __mptcp_recvmsg_mskq(sk, msg, len - copied, flags, &tss, = &cmsg_flags); if (unlikely(bytes_read < 0)) { if (!copied) copied =3D bytes_read; @@ -2045,9 +2044,6 @@ static int mptcp_recvmsg(struct sock *sk, struct msgh= dr *msg, size_t len, /* be sure to advertise window change */ mptcp_cleanup_rbuf(msk); =20 - if (skb_queue_empty(&msk->receive_queue) && __mptcp_move_skbs(msk)) - continue; - /* only the master socket status is relevant here. The exit * conditions mirror closely tcp_recvmsg() */ @@ -2074,7 +2070,7 @@ static int mptcp_recvmsg(struct sock *sk, struct msgh= dr *msg, size_t len, /* race breaker: the shutdown could be after the * previous receive queue check */ - if (__mptcp_move_skbs(msk)) + if (__mptcp_move_skbs(sk)) continue; break; } @@ -2111,9 +2107,8 @@ static int mptcp_recvmsg(struct sock *sk, struct msgh= dr *msg, size_t len, } } =20 - pr_debug("msk=3D%p rx queue empty=3D%d:%d copied=3D%d", - msk, skb_queue_empty_lockless(&sk->sk_receive_queue), - skb_queue_empty(&msk->receive_queue), copied); + pr_debug("msk=3D%p rx queue empty=3D%d copied=3D%d", + msk, skb_queue_empty(&sk->sk_receive_queue), copied); if (!(flags & MSG_PEEK)) mptcp_rcv_space_adjust(msk, copied); =20 @@ -2566,7 +2561,6 @@ static int __mptcp_init_sock(struct sock *sk) INIT_LIST_HEAD(&msk->join_list); INIT_LIST_HEAD(&msk->rtx_queue); INIT_WORK(&msk->work, mptcp_worker); - __skb_queue_head_init(&msk->receive_queue); msk->out_of_order_queue =3D RB_ROOT; msk->first_pending =3D NULL; msk->rmem_fwd_alloc =3D 0; @@ -3048,12 +3042,8 @@ void mptcp_destroy_common(struct mptcp_sock *msk, un= signed int flags) mptcp_for_each_subflow_safe(msk, subflow, tmp) __mptcp_close_ssk(sk, mptcp_subflow_tcp_sock(subflow), subflow, flags); =20 - /* move to sk_receive_queue, sk_stream_kill_queues will purge it */ - mptcp_data_lock(sk); - skb_queue_splice_tail_init(&msk->receive_queue, &sk->sk_receive_queue); __skb_queue_purge(&sk->sk_receive_queue); skb_rbtree_purge(&msk->out_of_order_queue); - mptcp_data_unlock(sk); =20 /* move all the rx fwd alloc into the sk_mem_reclaim_final in * inet_sock_destruct() will dispose it @@ -3135,6 +3125,8 @@ static void mptcp_release_cb(struct sock *sk) __mptcp_flush_join_list(sk); if (flags & BIT(MPTCP_PUSH_PENDING)) __mptcp_push_pending(sk, 0); + if ((flags & BIT(MPTCP_DEQUEUE)) && __mptcp_move_skbs(sk)) + sk->sk_data_ready(sk); if (flags & BIT(MPTCP_RETRANSMIT)) __mptcp_retrans(sk); =20 @@ -3383,7 +3375,7 @@ static int mptcp_ioctl(struct sock *sk, int cmd, unsi= gned long arg) return -EINVAL; =20 lock_sock(sk); - __mptcp_move_skbs(msk); + __mptcp_move_skbs(sk); answ =3D mptcp_inq_hint(sk); release_sock(sk); break; @@ -3619,8 +3611,7 @@ static __poll_t mptcp_check_readable(struct mptcp_soc= k *msk) /* Concurrent splices from sk_receive_queue into receive_queue will * always show at least one non-empty queue when checked in this order. */ - if (skb_queue_empty_lockless(&((struct sock *)msk)->sk_receive_queue) && - skb_queue_empty_lockless(&msk->receive_queue)) + if (skb_queue_empty_lockless(&((struct sock *)msk)->sk_receive_queue)) return 0; =20 return EPOLLIN | EPOLLRDNORM; diff --git a/net/mptcp/protocol.h b/net/mptcp/protocol.h index a54f42462a71..99c710e1ff5c 100644 --- a/net/mptcp/protocol.h +++ b/net/mptcp/protocol.h @@ -125,6 +125,7 @@ #define MPTCP_FLUSH_JOIN_LIST 5 #define MPTCP_CONNECTED 6 #define MPTCP_RESET_SCHEDULER 7 +#define MPTCP_DEQUEUE 8 =20 static inline bool before64(__u64 seq1, __u64 seq2) { @@ -288,7 +289,6 @@ struct mptcp_sock { struct work_struct work; struct sk_buff *ooo_last_skb; struct rb_root out_of_order_queue; - struct sk_buff_head receive_queue; struct list_head conn_list; struct list_head rtx_queue; struct mptcp_data_frag *first_pending; --=20 2.35.3