From nobody Sun Feb 8 19:37:51 2026 Delivered-To: wpasupplicant.patchew@gmail.com Received: by 2002:a05:6638:38c:0:0:0:0 with SMTP id y12csp1899782jap; Thu, 6 Jan 2022 16:21:06 -0800 (PST) X-Google-Smtp-Source: ABdhPJyGfBS7KtyZvsLPqQNdssLbeVXeJK5J0mEMb42wH+BigPx+Ol1Qhc3Vtm5kobkPZR+BvCjf X-Received: by 2002:a17:90a:8b18:: with SMTP id y24mr12705711pjn.26.1641514866776; Thu, 06 Jan 2022 16:21:06 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1641514866; cv=none; d=google.com; s=arc-20160816; b=yLhJmaka82dhE1IuYxfRMyVwThJ082T3oJxuEN45XTb4OB868sdrEJ3mFzAv/oIoPK Qg1spAv8M0u1oZpmhyBmpXaSyRZK/V8VAdCMUz0RES4btJb6WGtiWkZKlPJILshKWAVs YUNzqP4uf1HKdlybWtRejFTknt0K0Q/GZ4SmejhZnNqrIGysMrPrJ/neT7s+MOni25qZ kNCpsInlcL8xh5E0hIz+uawjqS/tJBBNjGh+X7FI3saMiV0pw9gmUgwmclbDnPvwsMw7 NlrZpTPvKnimPVVWM8KjRpwBKQ2knhbFAD/s4In6+DVbIVy02CyCCx/JzUzLaGY32qul 8z0w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from:dkim-signature; bh=ifbUmT/wyLlcX90OzHKnTQuJC1VLyyFdeFu6GfyJ53Y=; b=YQPoyEwyu7x+oACByKboceWBiv4n4WVuDdJ+tXLbzG322kCe8Em9TXsgZJLbcLM1mM 9TUzHQEp8I1ihw0RxPxFQIuzRd95XJ3pvO13zJQ/lMCQ8P9InfvNqUldC0xt7CyOxCYw YMze6PMu4AC53gGslK4g875ioD1xZnQgRTOO58/y5uQWUh52j/ggth8K/RERsYgzcy4r y0CP8W0qsUA2M36wWxhHyeqRKiMgvbu6fFLFBBTj+0eBDnfIQTWj8LcdVOsEdDwSOLFf zmTMuzerwBrgnSIz+bY+NiWM0fFqFdQJDZU/gEgFhghf5Q0ZhEW8+toSMFjOUqIbbr0S UA6w== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=gSTZYo1E; spf=pass (google.com: domain of mptcp+bounces-2958-wpasupplicant.patchew=gmail.com@lists.linux.dev designates 147.75.69.165 as permitted sender) smtp.mailfrom="mptcp+bounces-2958-wpasupplicant.patchew=gmail.com@lists.linux.dev"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Return-Path: Received: from sjc.edge.kernel.org (sjc.edge.kernel.org. [147.75.69.165]) by mx.google.com with ESMTPS id q7si3412082pgq.497.2022.01.06.16.21.06 for (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Thu, 06 Jan 2022 16:21:06 -0800 (PST) Received-SPF: pass (google.com: domain of mptcp+bounces-2958-wpasupplicant.patchew=gmail.com@lists.linux.dev designates 147.75.69.165 as permitted sender) client-ip=147.75.69.165; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=gSTZYo1E; spf=pass (google.com: domain of mptcp+bounces-2958-wpasupplicant.patchew=gmail.com@lists.linux.dev designates 147.75.69.165 as permitted sender) smtp.mailfrom="mptcp+bounces-2958-wpasupplicant.patchew=gmail.com@lists.linux.dev"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sjc.edge.kernel.org (Postfix) with ESMTPS id 3074C3E0F35 for ; Fri, 7 Jan 2022 00:21:06 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 53BBE2CAA; Fri, 7 Jan 2022 00:20:56 +0000 (UTC) X-Original-To: mptcp@lists.linux.dev Received: from mga05.intel.com (mga05.intel.com [192.55.52.43]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0E1DE2CB1 for ; Fri, 7 Jan 2022 00:20:55 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1641514855; x=1673050855; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=TfRLYQUUvmMscHCH67d3go38s+1HcbG004kqYoXQKCM=; b=gSTZYo1E8HZIKvfMxk6NI392PXovSKd6aChioUNFu0+fREJ86VsEpbL8 387kIMtId/HdC75TbUARpvkuxkx0XOxjG/pPmrOTS+kBigWMAGaj0RrYJ SQRHEhFGatcQBs12vVku4sdBYdiDSMXqyNZ9XGryWF30vCz6dedFlPuuN iTv49YfER98NjhJWe0lJ2+4YIuVVbgZy8UeY9/kQ8fB8MjcPa2f0vS/zU ndMav816FoIko/X/KZ6BcY5++F2j2beCT1oojSkRLFf7kwhYsA2uplVlE UNqLiwRd9jOoXvvlNMu38MpyAMOJNt5Rkm5SbbunEYbxZFEO8HBXsirbY A==; X-IronPort-AV: E=McAfee;i="6200,9189,10217"; a="329111063" X-IronPort-AV: E=Sophos;i="5.88,268,1635231600"; d="scan'208";a="329111063" Received: from fmsmga002.fm.intel.com ([10.253.24.26]) by fmsmga105.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 06 Jan 2022 16:20:37 -0800 X-IronPort-AV: E=Sophos;i="5.88,268,1635231600"; d="scan'208";a="618508521" Received: from mjmartin-desk2.amr.corp.intel.com (HELO mjmartin-desk2.intel.com) ([10.209.94.200]) by fmsmga002-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 06 Jan 2022 16:20:37 -0800 From: Mat Martineau To: netdev@vger.kernel.org Cc: Paolo Abeni , davem@davemloft.net, kuba@kernel.org, matthieu.baerts@tessares.net, mptcp@lists.linux.dev, Mat Martineau Subject: [PATCH net-next 13/13] mptcp: avoid atomic bit manipulation when possible Date: Thu, 6 Jan 2022 16:20:26 -0800 Message-Id: <20220107002026.375427-14-mathew.j.martineau@linux.intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220107002026.375427-1-mathew.j.martineau@linux.intel.com> References: <20220107002026.375427-1-mathew.j.martineau@linux.intel.com> Precedence: bulk X-Mailing-List: mptcp@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Paolo Abeni Currently the msk->flags bitmask carries both state for the mptcp_release_cb() - mostly touched under the mptcp data lock - and others state info touched even outside such lock scope. As a consequence, msk->flags is always manipulated with atomic operations. This change splits such bitmask in two separate fields, so that we use plain bit operations when touching the cb-related info. The MPTCP_PUSH_PENDING bit needs additional care, as it is the only CB related field currently accessed either under the mptcp data lock or the mptcp socket lock. Let's add another mask just for such bit's sake. Signed-off-by: Paolo Abeni Signed-off-by: Mat Martineau --- net/mptcp/protocol.c | 47 +++++++++++++++++++++++--------------------- net/mptcp/protocol.h | 18 ++++++++++------- net/mptcp/subflow.c | 4 ++-- 3 files changed, 38 insertions(+), 31 deletions(-) diff --git a/net/mptcp/protocol.c b/net/mptcp/protocol.c index c5f64fb0474d..62d418813503 100644 --- a/net/mptcp/protocol.c +++ b/net/mptcp/protocol.c @@ -763,7 +763,7 @@ static bool move_skbs_to_msk(struct mptcp_sock *msk, st= ruct sock *ssk) if (!sock_owned_by_user(sk)) __mptcp_error_report(sk); else - set_bit(MPTCP_ERROR_REPORT, &msk->flags); + __set_bit(MPTCP_ERROR_REPORT, &msk->cb_flags); } =20 /* If the moves have caught up with the DATA_FIN sequence number @@ -1517,9 +1517,8 @@ static void mptcp_update_post_push(struct mptcp_sock = *msk, =20 void mptcp_check_and_set_pending(struct sock *sk) { - if (mptcp_send_head(sk) && - !test_bit(MPTCP_PUSH_PENDING, &mptcp_sk(sk)->flags)) - set_bit(MPTCP_PUSH_PENDING, &mptcp_sk(sk)->flags); + if (mptcp_send_head(sk)) + mptcp_sk(sk)->push_pending |=3D BIT(MPTCP_PUSH_PENDING); } =20 void __mptcp_push_pending(struct sock *sk, unsigned int flags) @@ -2134,7 +2133,7 @@ static void mptcp_retransmit_timer(struct timer_list = *t) mptcp_schedule_work(sk); } else { /* delegate our work to tcp_release_cb() */ - set_bit(MPTCP_RETRANSMIT, &msk->flags); + __set_bit(MPTCP_RETRANSMIT, &msk->cb_flags); } bh_unlock_sock(sk); sock_put(sk); @@ -2840,7 +2839,9 @@ static int mptcp_disconnect(struct sock *sk, int flag= s) =20 mptcp_destroy_common(msk); msk->last_snd =3D NULL; - msk->flags =3D 0; + WRITE_ONCE(msk->flags, 0); + msk->cb_flags =3D 0; + msk->push_pending =3D 0; msk->recovery =3D false; msk->can_ack =3D false; msk->fully_established =3D false; @@ -3021,7 +3022,7 @@ void __mptcp_data_acked(struct sock *sk) if (!sock_owned_by_user(sk)) __mptcp_clean_una(sk); else - set_bit(MPTCP_CLEAN_UNA, &mptcp_sk(sk)->flags); + __set_bit(MPTCP_CLEAN_UNA, &mptcp_sk(sk)->cb_flags); =20 if (mptcp_pending_data_fin_ack(sk)) mptcp_schedule_work(sk); @@ -3040,22 +3041,23 @@ void __mptcp_check_push(struct sock *sk, struct soc= k *ssk) else if (xmit_ssk) mptcp_subflow_delegate(mptcp_subflow_ctx(xmit_ssk), MPTCP_DELEGATE_SEND= ); } else { - set_bit(MPTCP_PUSH_PENDING, &mptcp_sk(sk)->flags); + __set_bit(MPTCP_PUSH_PENDING, &mptcp_sk(sk)->cb_flags); } } =20 +#define MPTCP_FLAGS_PROCESS_CTX_NEED (BIT(MPTCP_PUSH_PENDING) | \ + BIT(MPTCP_RETRANSMIT) | \ + BIT(MPTCP_FLUSH_JOIN_LIST)) + /* processes deferred events and flush wmem */ static void mptcp_release_cb(struct sock *sk) + __must_hold(&sk->sk_lock.slock) { + struct mptcp_sock *msk =3D mptcp_sk(sk); + for (;;) { - unsigned long flags =3D 0; - - if (test_and_clear_bit(MPTCP_PUSH_PENDING, &mptcp_sk(sk)->flags)) - flags |=3D BIT(MPTCP_PUSH_PENDING); - if (test_and_clear_bit(MPTCP_RETRANSMIT, &mptcp_sk(sk)->flags)) - flags |=3D BIT(MPTCP_RETRANSMIT); - if (test_and_clear_bit(MPTCP_FLUSH_JOIN_LIST, &mptcp_sk(sk)->flags)) - flags |=3D BIT(MPTCP_FLUSH_JOIN_LIST); + unsigned long flags =3D (msk->cb_flags & MPTCP_FLAGS_PROCESS_CTX_NEED) | + msk->push_pending; if (!flags) break; =20 @@ -3066,7 +3068,8 @@ static void mptcp_release_cb(struct sock *sk) * datapath acquires the msk socket spinlock while helding * the subflow socket lock */ - + msk->push_pending =3D 0; + msk->cb_flags &=3D ~flags; spin_unlock_bh(&sk->sk_lock.slock); if (flags & BIT(MPTCP_FLUSH_JOIN_LIST)) __mptcp_flush_join_list(sk); @@ -3082,11 +3085,11 @@ static void mptcp_release_cb(struct sock *sk) /* be sure to set the current sk state before tacking actions * depending on sk_state */ - if (test_and_clear_bit(MPTCP_CONNECTED, &mptcp_sk(sk)->flags)) + if (__test_and_clear_bit(MPTCP_CONNECTED, &msk->cb_flags)) __mptcp_set_connected(sk); - if (test_and_clear_bit(MPTCP_CLEAN_UNA, &mptcp_sk(sk)->flags)) + if (__test_and_clear_bit(MPTCP_CLEAN_UNA, &msk->cb_flags)) __mptcp_clean_una_wakeup(sk); - if (test_and_clear_bit(MPTCP_ERROR_REPORT, &mptcp_sk(sk)->flags)) + if (__test_and_clear_bit(MPTCP_ERROR_REPORT, &msk->cb_flags)) __mptcp_error_report(sk); =20 __mptcp_update_rmem(sk); @@ -3128,7 +3131,7 @@ void mptcp_subflow_process_delegated(struct sock *ssk) if (!sock_owned_by_user(sk)) __mptcp_subflow_push_pending(sk, ssk); else - set_bit(MPTCP_PUSH_PENDING, &mptcp_sk(sk)->flags); + __set_bit(MPTCP_PUSH_PENDING, &mptcp_sk(sk)->cb_flags); mptcp_data_unlock(sk); mptcp_subflow_delegated_done(subflow, MPTCP_DELEGATE_SEND); } @@ -3247,7 +3250,7 @@ bool mptcp_finish_join(struct sock *ssk) } else { sock_hold(ssk); list_add_tail(&subflow->node, &msk->join_list); - set_bit(MPTCP_FLUSH_JOIN_LIST, &msk->flags); + __set_bit(MPTCP_FLUSH_JOIN_LIST, &msk->cb_flags); } mptcp_data_unlock(parent); =20 diff --git a/net/mptcp/protocol.h b/net/mptcp/protocol.h index 962f3b6b6a1d..a77f512d5ad7 100644 --- a/net/mptcp/protocol.h +++ b/net/mptcp/protocol.h @@ -110,18 +110,20 @@ /* MPTCP TCPRST flags */ #define MPTCP_RST_TRANSIENT BIT(0) =20 -/* MPTCP socket flags */ +/* MPTCP socket atomic flags */ #define MPTCP_NOSPACE 1 #define MPTCP_WORK_RTX 2 #define MPTCP_WORK_EOF 3 #define MPTCP_FALLBACK_DONE 4 #define MPTCP_WORK_CLOSE_SUBFLOW 5 -#define MPTCP_PUSH_PENDING 6 -#define MPTCP_CLEAN_UNA 7 -#define MPTCP_ERROR_REPORT 8 -#define MPTCP_RETRANSMIT 9 -#define MPTCP_FLUSH_JOIN_LIST 10 -#define MPTCP_CONNECTED 11 + +/* MPTCP socket release cb flags */ +#define MPTCP_PUSH_PENDING 1 +#define MPTCP_CLEAN_UNA 2 +#define MPTCP_ERROR_REPORT 3 +#define MPTCP_RETRANSMIT 4 +#define MPTCP_FLUSH_JOIN_LIST 5 +#define MPTCP_CONNECTED 6 =20 static inline bool before64(__u64 seq1, __u64 seq2) { @@ -250,6 +252,8 @@ struct mptcp_sock { u32 token; int rmem_released; unsigned long flags; + unsigned long cb_flags; + unsigned long push_pending; bool recovery; /* closing subflow write queue reinjected */ bool can_ack; bool fully_established; diff --git a/net/mptcp/subflow.c b/net/mptcp/subflow.c index a1cd39f97659..5bedc7e88977 100644 --- a/net/mptcp/subflow.c +++ b/net/mptcp/subflow.c @@ -388,7 +388,7 @@ static void mptcp_set_connected(struct sock *sk) if (!sock_owned_by_user(sk)) __mptcp_set_connected(sk); else - set_bit(MPTCP_CONNECTED, &mptcp_sk(sk)->flags); + __set_bit(MPTCP_CONNECTED, &mptcp_sk(sk)->cb_flags); mptcp_data_unlock(sk); } =20 @@ -1274,7 +1274,7 @@ static void subflow_error_report(struct sock *ssk) if (!sock_owned_by_user(sk)) __mptcp_error_report(sk); else - set_bit(MPTCP_ERROR_REPORT, &mptcp_sk(sk)->flags); + __set_bit(MPTCP_ERROR_REPORT, &mptcp_sk(sk)->cb_flags); mptcp_data_unlock(sk); } =20 --=20 2.34.1