The SYN + MPTCP_CAPABLE packets could be explicitly dropped by firewall
somewhere in the network, e.g. with:

  iptables -t filter -A FORWARD -p tcp --tcp-option 30 -j DROP

The idea of this series is to fallback to TCP after 3 SYN+MPC drop
(patch 2). If the connection succeeds after the fallback, it very likely
means a blackhole has been detected. In this case (patch 3), MPTCP can
be disabled for a certain period of time, 1h by default. If after this
period, MPTCP is still blocked, the period is doubled.

This should help applications which want to use MPTCP by default on the
client side if available.

This series has been validated by a new packetdrill test:

  https://github.com/multipath-tcp/packetdrill/pull/156

Some questions:

- Should we let the user changes the number of retransmissions (2)
  before falling back to TCP? For TFO, the data are never retransmitted
  in a SYN. Maybe that's different here? A sysctl knob could always be
  added later on.
- Should we globally disable all MPTCP connections if any retransmission
  after the fallback is a success instead of only the first one? I guess
  we reduce risks of accidents by only looking at the following
  retransmission after the fallback.
- Is one hour a good time for the fallback?

For later:

- The restriction could be done per oif (sk_dst_get(ssk)->dev), but we
  would need to store it somehow, or per MPTCP entrypoint. Or let the PM
  calling mptcp_active_enable() when a new endpoint is added.
- Other cases could trigger mptcp_active_disable(): e.g. some fallbacks
  or corruptions in the middle of the connections.

Signed-off-by: Matthieu Baerts (NGI0) <matttbe@kernel.org>
---
Matthieu Baerts (NGI0) (3):
      mptcp: export mptcp_subflow_early_fallback()
      mptcp: fallback to TCP after SYN+MPC drops
      mptcp: disable active MPTCP in case of blackhole

 Documentation/networking/mptcp-sysctl.rst |  11 +++
 include/net/mptcp.h                       |   4 +
 net/ipv4/tcp_timer.c                      |   1 +
 net/mptcp/ctrl.c                          | 133 ++++++++++++++++++++++++++++++
 net/mptcp/mib.c                           |   3 +
 net/mptcp/mib.h                           |   3 +
 net/mptcp/protocol.c                      |  18 ++--
 net/mptcp/protocol.h                      |  16 +++-
 net/mptcp/subflow.c                       |   4 +
 9 files changed, 182 insertions(+), 11 deletions(-)
---
base-commit: 44106bc7908a7c9e461983032567a918e1d62b91
change-id: 20240822-mptcp-fallback-x-mpc-491bb35a8e66

Best regards,
-- 
Matthieu Baerts (NGI0) <matttbe@kernel.org>

Hello,

On 02/09/2024 11:09, Matthieu Baerts (NGI0) wrote:
> The SYN + MPTCP_CAPABLE packets could be explicitly dropped by firewall
> somewhere in the network, e.g. with:
> 
>   iptables -t filter -A FORWARD -p tcp --tcp-option 30 -j DROP
> 
> The idea of this series is to fallback to TCP after 3 SYN+MPC drop
> (patch 2). If the connection succeeds after the fallback, it very likely
> means a blackhole has been detected. In this case (patch 3), MPTCP can
> be disabled for a certain period of time, 1h by default. If after this
> period, MPTCP is still blocked, the period is doubled.
> 
> This should help applications which want to use MPTCP by default on the
> client side if available.
> 
> This series has been validated by a new packetdrill test:
> 
>   https://github.com/multipath-tcp/packetdrill/pull/156
> 
> Some questions:
> 
> - Should we let the user changes the number of retransmissions (2)
>   before falling back to TCP? For TFO, the data are never retransmitted
>   in a SYN. Maybe that's different here? A sysctl knob could always be
>   added later on.
> - Should we globally disable all MPTCP connections if any retransmission
>   after the fallback is a success instead of only the first one? I guess
>   we reduce risks of accidents by only looking at the following
>   retransmission after the fallback.
> - Is one hour a good time for the fallback?
> 
> For later:
> 
> - The restriction could be done per oif (sk_dst_get(ssk)->dev), but we
>   would need to store it somehow, or per MPTCP entrypoint. Or let the PM
>   calling mptcp_active_enable() when a new endpoint is added.
> - Other cases could trigger mptcp_active_disable(): e.g. some fallbacks
>   or corruptions in the middle of the connections.

I suggest addressing these questions later. Due to some external
circumstances, it is not possible to have more reviews for the moment. I
think there is no need to block that longer, fixes can always be
addressed later on.

New patches for t/upstream:
- a09026a9e8c4: mptcp: export mptcp_subflow_early_fallback()
- 4216aa51fb10: mptcp: fallback to TCP after SYN+MPC drops
- d0c7c7172eac: mptcp: disable active MPTCP in case of blackhole
- 0fd52c49c60b: conflict in
t/mptcp-annotate-data-races-around-subflow-fully_established
- Results: 29da3bae8ab9..fc8995744495 (export)

Tests are now in progress:

- export:
https://github.com/multipath-tcp/mptcp_net-next/commit/ca4d36d160ee56fa9bd50dc0006ad34d454d8f48/checks

Cheers,
Matt
-- 
Sponsored by the NGI0 Core fund.

Hi Matthieu,

Thank you for your modifications, that's great!

Our CI did some validations and here is its report:

- KVM Validation: normal: Success! ✅
- KVM Validation: debug: Success! ✅
- KVM Validation: btf (only bpftest_all): Success! ✅
- Task: https://github.com/multipath-tcp/mptcp_net-next/actions/runs/10670881116

Initiator: Patchew Applier
Commits: https://github.com/multipath-tcp/mptcp_net-next/commits/5f9d22a2f88b
Patchwork: https://patchwork.kernel.org/project/mptcp/list/?series=885758


If there are some issues, you can reproduce them using the same environment as
the one used by the CI thanks to a docker image, e.g.:

    $ cd [kernel source code]
    $ docker run -v "${PWD}:${PWD}:rw" -w "${PWD}" --privileged --rm -it \
        --pull always mptcp/mptcp-upstream-virtme-docker:latest \
        auto-normal

For more details:

    https://github.com/multipath-tcp/mptcp-upstream-virtme-docker


Please note that despite all the efforts that have been already done to have a
stable tests suite when executed on a public CI like here, it is possible some
reported issues are not due to your modifications. Still, do not hesitate to
help us improve that ;-)

Cheers,
MPTCP GH Action bot
Bot operated by Matthieu Baerts (NGI0 Core)