[PATCH mptcp-next v6 06/13] mptcp: multi subflows push_pending

Geliang Tang posted 13 patches 3 years, 4 months ago
Maintainers: Mat Martineau <mathew.j.martineau@linux.intel.com>, Matthieu Baerts <matthieu.baerts@tessares.net>, "David S. Miller" <davem@davemloft.net>, Eric Dumazet <edumazet@google.com>, Jakub Kicinski <kuba@kernel.org>, Paolo Abeni <pabeni@redhat.com>, Andrii Nakryiko <andrii@kernel.org>, Mykola Lysenko <mykolal@fb.com>, Alexei Starovoitov <ast@kernel.org>, Daniel Borkmann <daniel@iogearbox.net>, Martin KaFai Lau <martin.lau@linux.dev>, Song Liu <song@kernel.org>, Yonghong Song <yhs@fb.com>, John Fastabend <john.fastabend@gmail.com>, KP Singh <kpsingh@kernel.org>, Stanislav Fomichev <sdf@google.com>, Hao Luo <haoluo@google.com>, Jiri Olsa <jolsa@kernel.org>, Shuah Khan <shuah@kernel.org>
There is a newer version of this series
[PATCH mptcp-next v6 06/13] mptcp: multi subflows push_pending
Posted by Geliang Tang 3 years, 4 months ago
This patch adds the multiple subflows support for __mptcp_push_pending().
Use mptcp_sched_get_send() wrapper instead of mptcp_subflow_get_send() in
it.

Check the subflow scheduled flags to test which subflow or subflows are
picked by the scheduler, use them to send data.

Signed-off-by: Geliang Tang <geliang.tang@suse.com>
---
 net/mptcp/protocol.c | 32 ++++++++++++++++++++------------
 1 file changed, 20 insertions(+), 12 deletions(-)

diff --git a/net/mptcp/protocol.c b/net/mptcp/protocol.c
index 817e539d1d12..86ac38d10bc4 100644
--- a/net/mptcp/protocol.c
+++ b/net/mptcp/protocol.c
@@ -1561,22 +1561,30 @@ static int __subflow_push_pending(struct sock *sk, struct sock *ssk,
 void __mptcp_push_pending(struct sock *sk, unsigned int flags)
 {
 	struct mptcp_sock *msk = mptcp_sk(sk);
-	struct mptcp_sendmsg_info info = {
-				.flags = flags,
-	};
-	struct sock *ssk;
 	int ret = 0;
 
 again:
-	while (mptcp_send_head(sk) && (ssk = mptcp_subflow_get_send(msk))) {
-		lock_sock(ssk);
-		ret = __subflow_push_pending(sk, ssk, &info);
-		release_sock(ssk);
+	while (mptcp_send_head(sk) && !mptcp_sched_get_send(msk)) {
+		struct mptcp_subflow_context *subflow;
+		struct mptcp_sendmsg_info info = {
+			.flags = flags,
+		};
 
-		if (ret <= 0) {
-			if (ret == -EAGAIN)
-				goto again;
-			goto out;
+		mptcp_for_each_subflow(msk, subflow) {
+			if (READ_ONCE(subflow->scheduled)) {
+				struct sock *ssk = mptcp_subflow_tcp_sock(subflow);
+
+				lock_sock(ssk);
+				ret = __subflow_push_pending(sk, ssk, &info);
+				release_sock(ssk);
+
+				if (ret <= 0) {
+					if (ret == -EAGAIN)
+						goto again;
+					goto out;
+				}
+				mptcp_subflow_set_scheduled(subflow, false);
+			}
 		}
 	}
 
-- 
2.35.3
Re: [PATCH mptcp-next v6 06/13] mptcp: multi subflows push_pending
Posted by Mat Martineau 3 years, 4 months ago
On Tue, 11 Oct 2022, Geliang Tang wrote:

> This patch adds the multiple subflows support for __mptcp_push_pending().
> Use mptcp_sched_get_send() wrapper instead of mptcp_subflow_get_send() in
> it.
>
> Check the subflow scheduled flags to test which subflow or subflows are
> picked by the scheduler, use them to send data.
>
> Signed-off-by: Geliang Tang <geliang.tang@suse.com>
> ---
> net/mptcp/protocol.c | 32 ++++++++++++++++++++------------
> 1 file changed, 20 insertions(+), 12 deletions(-)
>
> diff --git a/net/mptcp/protocol.c b/net/mptcp/protocol.c
> index 817e539d1d12..86ac38d10bc4 100644
> --- a/net/mptcp/protocol.c
> +++ b/net/mptcp/protocol.c
> @@ -1561,22 +1561,30 @@ static int __subflow_push_pending(struct sock *sk, struct sock *ssk,
> void __mptcp_push_pending(struct sock *sk, unsigned int flags)
> {
> 	struct mptcp_sock *msk = mptcp_sk(sk);
> -	struct mptcp_sendmsg_info info = {
> -				.flags = flags,
> -	};
> -	struct sock *ssk;
> 	int ret = 0;
>
> again:
> -	while (mptcp_send_head(sk) && (ssk = mptcp_subflow_get_send(msk))) {
> -		lock_sock(ssk);
> -		ret = __subflow_push_pending(sk, ssk, &info);
> -		release_sock(ssk);
> +	while (mptcp_send_head(sk) && !mptcp_sched_get_send(msk)) {
> +		struct mptcp_subflow_context *subflow;
> +		struct mptcp_sendmsg_info info = {
> +			.flags = flags,
> +		};
>
> -		if (ret <= 0) {
> -			if (ret == -EAGAIN)
> -				goto again;
> -			goto out;
> +		mptcp_for_each_subflow(msk, subflow) {
> +			if (READ_ONCE(subflow->scheduled)) {
> +				struct sock *ssk = mptcp_subflow_tcp_sock(subflow);
> +
> +				lock_sock(ssk);
> +				ret = __subflow_push_pending(sk, ssk, &info);
> +				release_sock(ssk);

Geliang -

It seems like this doesn't allow for redundant transmits, since 
__subflow_push_pending() updates msk->first_pending. Did you find that 
redundant sends were working correctly (maybe I've misunderstood)?

- Mat

> +
> +				if (ret <= 0) {
> +					if (ret == -EAGAIN)
> +						goto again;
> +					goto out;
> +				}
> +				mptcp_subflow_set_scheduled(subflow, false);
> +			}
> 		}
> 	}
>
> -- 
> 2.35.3
>
>
>

--
Mat Martineau
Intel
Re: [PATCH mptcp-next v6 06/13] mptcp: multi subflows push_pending
Posted by Geliang Tang 3 years, 3 months ago
On Tue, Oct 11, 2022 at 03:30:43PM -0700, Mat Martineau wrote:
> On Tue, 11 Oct 2022, Geliang Tang wrote:
> 
> > This patch adds the multiple subflows support for __mptcp_push_pending().
> > Use mptcp_sched_get_send() wrapper instead of mptcp_subflow_get_send() in
> > it.
> > 
> > Check the subflow scheduled flags to test which subflow or subflows are
> > picked by the scheduler, use them to send data.
> > 
> > Signed-off-by: Geliang Tang <geliang.tang@suse.com>
> > ---
> > net/mptcp/protocol.c | 32 ++++++++++++++++++++------------
> > 1 file changed, 20 insertions(+), 12 deletions(-)
> > 
> > diff --git a/net/mptcp/protocol.c b/net/mptcp/protocol.c
> > index 817e539d1d12..86ac38d10bc4 100644
> > --- a/net/mptcp/protocol.c
> > +++ b/net/mptcp/protocol.c
> > @@ -1561,22 +1561,30 @@ static int __subflow_push_pending(struct sock *sk, struct sock *ssk,
> > void __mptcp_push_pending(struct sock *sk, unsigned int flags)
> > {
> > 	struct mptcp_sock *msk = mptcp_sk(sk);
> > -	struct mptcp_sendmsg_info info = {
> > -				.flags = flags,
> > -	};
> > -	struct sock *ssk;
> > 	int ret = 0;
> > 
> > again:
> > -	while (mptcp_send_head(sk) && (ssk = mptcp_subflow_get_send(msk))) {
> > -		lock_sock(ssk);
> > -		ret = __subflow_push_pending(sk, ssk, &info);
> > -		release_sock(ssk);
> > +	while (mptcp_send_head(sk) && !mptcp_sched_get_send(msk)) {
> > +		struct mptcp_subflow_context *subflow;
> > +		struct mptcp_sendmsg_info info = {
> > +			.flags = flags,
> > +		};
> > 
> > -		if (ret <= 0) {
> > -			if (ret == -EAGAIN)
> > -				goto again;
> > -			goto out;
> > +		mptcp_for_each_subflow(msk, subflow) {
> > +			if (READ_ONCE(subflow->scheduled)) {
> > +				struct sock *ssk = mptcp_subflow_tcp_sock(subflow);
> > +
> > +				lock_sock(ssk);
> > +				ret = __subflow_push_pending(sk, ssk, &info);
> > +				release_sock(ssk);
> 
> Geliang -
> 
> It seems like this doesn't allow for redundant transmits, since
> __subflow_push_pending() updates msk->first_pending. Did you find that
> redundant sends were working correctly (maybe I've misunderstood)?

Hi Mat,

Redundant sends are not support in this series yet. It will be added in
this __subflow_push_pending() function in the next series with the
selftests patches.

There are still some issues that haven't been solved in the redundant
support patches. I'm still working on them.

Thanks,
-Geliang

> 
> - Mat
> 
> > +
> > +				if (ret <= 0) {
> > +					if (ret == -EAGAIN)
> > +						goto again;
> > +					goto out;
> > +				}
> > +				mptcp_subflow_set_scheduled(subflow, false);
> > +			}
> > 		}
> > 	}
> > 
> > -- 
> > 2.35.3
> > 
> > 
> > 
> 
> --
> Mat Martineau
> Intel
Re: [PATCH mptcp-next v6 06/13] mptcp: multi subflows push_pending
Posted by Geliang Tang 3 years, 3 months ago
Geliang Tang <geliang.tang@suse.com> 于2022年10月12日周三 18:37写道:
>
> On Tue, Oct 11, 2022 at 03:30:43PM -0700, Mat Martineau wrote:
> > On Tue, 11 Oct 2022, Geliang Tang wrote:
> >
> > > This patch adds the multiple subflows support for __mptcp_push_pending().
> > > Use mptcp_sched_get_send() wrapper instead of mptcp_subflow_get_send() in
> > > it.
> > >
> > > Check the subflow scheduled flags to test which subflow or subflows are
> > > picked by the scheduler, use them to send data.
> > >
> > > Signed-off-by: Geliang Tang <geliang.tang@suse.com>
> > > ---
> > > net/mptcp/protocol.c | 32 ++++++++++++++++++++------------
> > > 1 file changed, 20 insertions(+), 12 deletions(-)
> > >
> > > diff --git a/net/mptcp/protocol.c b/net/mptcp/protocol.c
> > > index 817e539d1d12..86ac38d10bc4 100644
> > > --- a/net/mptcp/protocol.c
> > > +++ b/net/mptcp/protocol.c
> > > @@ -1561,22 +1561,30 @@ static int __subflow_push_pending(struct sock *sk, struct sock *ssk,
> > > void __mptcp_push_pending(struct sock *sk, unsigned int flags)
> > > {
> > >     struct mptcp_sock *msk = mptcp_sk(sk);
> > > -   struct mptcp_sendmsg_info info = {
> > > -                           .flags = flags,
> > > -   };
> > > -   struct sock *ssk;
> > >     int ret = 0;
> > >
> > > again:
> > > -   while (mptcp_send_head(sk) && (ssk = mptcp_subflow_get_send(msk))) {
> > > -           lock_sock(ssk);
> > > -           ret = __subflow_push_pending(sk, ssk, &info);
> > > -           release_sock(ssk);
> > > +   while (mptcp_send_head(sk) && !mptcp_sched_get_send(msk)) {
> > > +           struct mptcp_subflow_context *subflow;
> > > +           struct mptcp_sendmsg_info info = {
> > > +                   .flags = flags,
> > > +           };
> > >
> > > -           if (ret <= 0) {
> > > -                   if (ret == -EAGAIN)
> > > -                           goto again;
> > > -                   goto out;
> > > +           mptcp_for_each_subflow(msk, subflow) {
> > > +                   if (READ_ONCE(subflow->scheduled)) {
> > > +                           struct sock *ssk = mptcp_subflow_tcp_sock(subflow);
> > > +
> > > +                           lock_sock(ssk);
> > > +                           ret = __subflow_push_pending(sk, ssk, &info);
> > > +                           release_sock(ssk);
> >
> > Geliang -
> >
> > It seems like this doesn't allow for redundant transmits, since
> > __subflow_push_pending() updates msk->first_pending. Did you find that
> > redundant sends were working correctly (maybe I've misunderstood)?
>
> Hi Mat,
>
> Redundant sends are not support in this series yet. It will be added in
> this __subflow_push_pending() function in the next series with the
> selftests patches.
>
> There are still some issues that haven't been solved in the redundant
> support patches. I'm still working on them.

Hi Mat,  I have good news. I finally solved these issues today and
sent a new version, "BPF redundant scheduler" v13 . Now redundant
sends are support in it, and all selftests (mptcp_connect.sh,
mptcp_join.sh and simult_flows.sh) passed.

>
> Thanks,
> -Geliang
>
> >
> > - Mat
> >
> > > +
> > > +                           if (ret <= 0) {
> > > +                                   if (ret == -EAGAIN)
> > > +                                           goto again;
> > > +                                   goto out;
> > > +                           }
> > > +                           mptcp_subflow_set_scheduled(subflow, false);
> > > +                   }
> > >             }
> > >     }
> > >
> > > --
> > > 2.35.3
> > >
> > >
> > >
> >
> > --
> > Mat Martineau
> > Intel
>