[RFC PATCH mptcp-next v15 0/5] mptcp: Fast Open Mechanism

Dmytro Shytyi posted 5 patches 1 month ago
Patches applied successfully (tree, apply log)
git fetch https://github.com/multipath-tcp/mptcp_net-next tags/patchew/20221106152411.655-1-dmytro@shytyi.net
Maintainers: Mat Martineau <mathew.j.martineau@linux.intel.com>, Matthieu Baerts <matthieu.baerts@tessares.net>, "David S. Miller" <davem@davemloft.net>, Eric Dumazet <edumazet@google.com>, Jakub Kicinski <kuba@kernel.org>, Paolo Abeni <pabeni@redhat.com>, Shuah Khan <shuah@kernel.org>
net/mptcp/Makefile                            |   2 +-
net/mptcp/fastopen.c                          |  66 ++++++++++
net/mptcp/options.c                           |   5 +
net/mptcp/protocol.c                          |  20 +--
net/mptcp/protocol.h                          |  18 +++
net/mptcp/sockopt.c                           |   5 +-
net/mptcp/subflow.c                           |  42 ++++++
.../selftests/net/mptcp/mptcp_connect.c       | 121 +++++++++++++-----
.../selftests/net/mptcp/mptcp_connect.sh      |  21 +++
9 files changed, 253 insertions(+), 47 deletions(-)
create mode 100644 net/mptcp/fastopen.c
[RFC PATCH mptcp-next v15 0/5] mptcp: Fast Open Mechanism
Posted by Dmytro Shytyi 1 month ago
These patches focus on the Initiator and partially on Listener side.
The next options in userspace are available:
a) sendto(..., ..., ..., MSG_FASTOPEN, ..., ...);
b) setsockopt(..., SOL_TCP, TCP_FASTOPEN, ..., ...);

These patches implement Appendix-B of RFC8684 (MPTFO).

We would like to credit Paulo Abeni, Mat Martineau, Matthieu Baerts and
Benjamin Hesmans for advices and ideas that improved these patches.

Signed-off-by: Dmytro Shytyi <dmytro@shytyi.net>
---
v14 -> v15:
 - Reduce code size to perform MPTFO selftests. Reuse run_tests_lo().
 - In v15 proposed change of selftests introduce a reproducible issue.

Observed behavior: 
# INFO: with MPTFO start
# ns2 MPTCP -> ns1 (10.0.1.1:10054      ) MPTCP (duration    81ms) [ OK ]
# ns2 MPTCP -> ns1 (10.0.1.1:10055      ) MPTCP (duration    85ms) [ FAIL ] file received by server does not match (in, out):
# -rw------- 1 root root 4561948 Nov  6 14:44 /tmp/tmp.9LKqAENkkp
# Trailing bytes are:
# MPTCP_TEST_FILE_END_MARKER
# -rw------- 1 root root 4560528 Nov  6 14:46 /tmp/tmp.sZoeneg8ix
# Trailing bytes are:
# MPTCP_TEST_FILE_END_MARKER

Analysis:
- It seems like at the second round when data supposed to be sent in SYN, output data misses first 1420 bytes, that should be a part of the [Syn] packet. 
- With local tests (without selftest) the server in userspace receives the data in the SYN during the second round.
- To confirm that data is sent in SYN, the tcpdump was launched during tests (context of tests with tcpdump: in func sock_connect_mptcp the var was changed: char wbuf[8192] to char wbuf[511] for observavility purposes.
Output of pcap is presented below.
root@9a011dab3a59:/mnt/mptcp_net-next/tools/testing/selftests/net/mptcp# tcpdump -r 635ee290-Fz3fsx-ns2-ns1-MPTCP-MPTCP-10.0.1.1-10042-listener.pcap | grep -w "[S]"
reading from file 635ee290-Fz3fsx-ns2-ns1-MPTCP-MPTCP-10.0.1.1-10042-listener.pcap, link-type LINUX_SLL2 (Linux cooked v2), snapshot length 65535
20:51:50.770789 sit0  In  IP 10.0.1.2.32944 > 10.0.1.1.10042: Flags [S], seq 980486723, win 64240, options [mss 1460,sackOK,TS val 2478157679 ecr 0,nop,wscale 7,tfo  cookiereq,nop,nop,mptcp capable v1], length 0
20:51:50.770807 sit0  Out IP 10.0.1.1.10042 > 10.0.1.2.32944: Flags [S.], seq 1788401934, ack 980486724, win 65160, options [mss 1460,nop,nop,sackOK,nop,wscale 7,tfo  cookie de367be85d45564d,nop,nop,mptcp capable v1 {0x2b02096a5416249e}], length 0
root@9a011dab3a59:/mnt/mptcp_net-next/tools/testing/selftests/net/mptcp# tcpdump -r 635ee290-Fz3fsx-ns2-ns1-MPTCP-MPTCP-10.0.1.1-10043-listener.pcap | grep -w "[S]"
reading from file 635ee290-Fz3fsx-ns2-ns1-MPTCP-MPTCP-10.0.1.1-10043-listener.pcap, link-type LINUX_SLL2 (Linux cooked v2), snapshot length 65535
20:51:53.766945 sit0  In  IP 10.0.1.2.50620 > 10.0.1.1.10043: Flags [S], seq 831892394:831892905, win 64240, options [mss 1460,sackOK,TS val 2478160675 ecr 0,nop,wscale 7,tfo  cookie de367be85d45564d,nop,nop,mptcp capable v1], length 511
20:51:53.766965 sit0  Out IP 10.0.1.1.10043 > 10.0.1.2.50620: Flags [S.], seq 4054084662, ack 831892906, win 65160, options [mss 1460,sackOK,TS val 1954362321 ecr 2478160675,nop,wscale 7,mptcp capable v1 {0x7bf8471b32013c7d}], length 0
root@9a011dab3a59:/mnt/mptcp_net-next/tools/testing/selftests/net/mptcp# tcpdump -r 635ee290-Fz3fsx-ns2-ns1-MPTCP-MPTCP-10.0.1.1-10044-listener.pcap | grep -w "[S]"
reading from file 635ee290-Fz3fsx-ns2-ns1-MPTCP-MPTCP-10.0.1.1-10044-listener.pcap, link-type LINUX_SLL2 (Linux cooked v2), snapshot length 65535
20:51:57.084750 sit0  In  IP 10.0.1.2.43624 > 10.0.1.1.10044: Flags [S], seq 835270974:835271485, win 64240, options [mss 1460,sackOK,TS val 2478163993 ecr 0,nop,wscale 7,tfo  cookie de367be85d45564d,nop,nop,mptcp capable v1], length 511
20:51:57.084772 sit0  Out IP 10.0.1.1.10044 > 10.0.1.2.43624: Flags [S.], seq 650287874, ack 835271486, win 65160, options [mss 1460,sackOK,TS val 1954365639 ecr 2478163993,nop,wscale 7,mptcp capable v1 {0x7b3cb5cf253aec1b}], length 0

---

Dmytro Shytyi (5):
  mptcp: introduce MSG_FASTOPEN flag
  mptcp: implement delayed seq generation for passive fastopen
  mptcp: add subflow_v(4,6)_send_synack()
  mptcp: add TCP_FASTOPEN sock option
  selftests: mptcp: mptfo Initiator/Listener

 net/mptcp/Makefile                            |   2 +-
 net/mptcp/fastopen.c                          |  66 ++++++++++
 net/mptcp/options.c                           |   5 +
 net/mptcp/protocol.c                          |  20 +--
 net/mptcp/protocol.h                          |  18 +++
 net/mptcp/sockopt.c                           |   5 +-
 net/mptcp/subflow.c                           |  42 ++++++
 .../selftests/net/mptcp/mptcp_connect.c       | 121 +++++++++++++-----
 .../selftests/net/mptcp/mptcp_connect.sh      |  21 +++
 9 files changed, 253 insertions(+), 47 deletions(-)
 create mode 100644 net/mptcp/fastopen.c

-- 
2.34.1
Re: [RFC PATCH mptcp-next v15 0/5] mptcp: Fast Open Mechanism
Posted by Matthieu Baerts 4 weeks ago
Hi Dmytro, Paolo,

On 06/11/2022 16:24, Dmytro Shytyi wrote:
> These patches focus on the Initiator and partially on Listener side.
> The next options in userspace are available:
> a) sendto(..., ..., ..., MSG_FASTOPEN, ..., ...);
> b) setsockopt(..., SOL_TCP, TCP_FASTOPEN, ..., ...);
> 
> These patches implement Appendix-B of RFC8684 (MPTFO).
> 
> We would like to credit Paulo Abeni, Mat Martineau, Matthieu Baerts and
> Benjamin Hesmans for advices and ideas that improved these patches.
> 
> Signed-off-by: Dmytro Shytyi <dmytro@shytyi.net>

(...)

> Dmytro Shytyi (5):
>   mptcp: introduce MSG_FASTOPEN flag
>   mptcp: implement delayed seq generation for passive fastopen
>   mptcp: add subflow_v(4,6)_send_synack()
>   mptcp: add TCP_FASTOPEN sock option
>   selftests: mptcp: mptfo Initiator/Listener

Thank you for the patches, reviews and Squash-to patches!

These patches and the ones from Paolo are now in our tree (feat. for
net-next-next) with Paolo's ACK on patch 1/5 and 4/5 and the Co-Dev ones
automatically added on the 3 others.

Please note that I had two minor conflicts when applying Dmytro's
patches before the BPF series already in our tree + some issues reported
by checkpatch I didn't fix (yet).

I also had more complex conflicts in Paolo's first squash-to patch: it
is modifying both patches 2/5 and 3/5, I had to split it. Please check
everything is still OK after that.

New patches for t/upstream:
- c87fe09b3765: mptcp: introduce MSG_FASTOPEN flag
- b08f0e118b1e: mptcp: implement delayed seq generation for passive fastopen
- bbd23a7de970: mptcp: add subflow_v(4,6)_send_synack()
- a2e147cb3f0a: mptcp: add TCP_FASTOPEN sock option
- 132e3b9f088e: selftests: mptcp: mptfo Initiator/Listener
- Results: 5f9e0bf4aacf..120554402db6 (export)

Paolo's patches:

- 512d25e34a03: mptcp: track accurately the incoming MPC suboption type
- Results: 120554402db6..a6011a4a534a (export)

- c0cda5b7f267: "squashed" (with conflicts) in "mptcp: implement delayed
seq generation for passive fastopen" (part 1)
- 7f98dbf4215e: "Signed-off-by" + "Co-developed-by"
- 6d231fd79244: conflict in t/mptcp-add-subflow_v-4-6-_send_synack
- b2bb9c9570a0: "squashed" in "mptcp: add subflow_v(4,6)_send_synack()"
(part 2)
- 7d3cac0330e4: "Signed-off-by" + "Co-developed-by"
- Results: a6011a4a534a..7141982e8319 (export)

- 27f8b8024c74: "squashed" in "selftests: mptcp: mptfo Initiator/Listener"
- ebe619034d7f: "Signed-off-by" + "Co-developed-by"
- Results: 7141982e8319..dfaa717d485a (export)


Tests are now in progress:

https://cirrus-ci.com/github/multipath-tcp/mptcp_net-next/export/20221110T185015

Cheers,
Matt
-- 
Tessares | Belgium | Hybrid Access Solutions
www.tessares.net