The core benches a subflow via the "stale" bit after
net.mptcp.stale_loss_cnt retransmission intervals without progress.
That bit belongs to the core and is tightly coupled to the RTO: it
gets cleared the moment a packet is acked on the subflow. A packet
scheduler has no way to bench a subflow for its own reasons, such as
too high latency or instability, and keep it benched independently of
ack traffic.

Reusing "stale" doesn't work, and "scheduled" doesn't either: the core
clears it right after every send/retrans pass, so it can't express a
standing "leave this subflow alone for now" decision, and nothing
outside the scheduler that set it can ever see it.

This adds a scheduler-owned "avoid" bool that stays set until the
scheduler clears it, is honoured by the default in-kernel subflow
picker on both the send and retransmit paths, and is visible through
MPTCP diag alongside the existing BKUP/FULLY_ESTABLISHED flags.

The avoid state is kept as a standalone bool instead of using the
adjacent flag bitfield. This avoids a race with 
mptcp_subflow_data_available(),
which clears map_valid in the same bitfield from the receive path while
the scheduler can update avoid from BPF context. The avoid accesses use
READ_ONCE()/WRITE_ONCE().

The default in-kernel subflow picker skips avoided subflows for both
send and retransmit operations. Custom schedulers can explicitly
honour the avoid state according to their own scheduling policy.

The avoid state is exposed to BPF struct_ops schedulers through
mptcp_subflow_set_avoid(), and is also exposed through MPTCP diag as
MPTCP_SUBFLOW_FLAG_AVOID. The kfunc is marked __bpf_kfunc so it remains
available for BPF BTF resolution.

The BPF selftest sets the avoid flag once on the first get_send() call
and relies on the state remaining set across subsequent scheduler
rounds. The test uses per-socket BPF storage for the scheduler state
and validates the avoided-subflow behaviour across rounds.

Probing an avoided subflow to decide when to clear it (mentioned in
issue 348) is left out on purpose: that's scheduler policy, not core
mechanism, and belongs to whoever writes the actual latency/stability
heuristics on top of this.

Patch 1 adds the flag, the kfunc, the default-scheduler enforcement,
and diag support.

Patch 2 adds the BPF selftest exercising persistence of the avoid state
across scheduler rounds.

The patches have been reviewed with the concurrency concern raised by
Sashiko addressed, and the resulting patch series passes checkpatch
with no errors. The remaining warning is the standard MAINTAINERS
warning for the newly added selftest file.

Link: https://github.com/multipath-tcp/mptcp_net-next/issues/349

Kalpan Jani (2):
mptcp: sched: add subflow avoid flag and enforce it in default sched
selftests: mptcp: bpf: exercise the subflow avoid flag across rounds

Hi Kalpan,

Thank you for your modifications, that's great!

Our CI did some validations and here is its report:

- KVM Validation: normal (except selftest_mptcp_join): Unstable: 1 failed test(s): packetdrill_fastclose ⚠️ 
- KVM Validation: normal (only selftest_mptcp_join): Success! ✅
- KVM Validation: debug (except selftest_mptcp_join): Success! ✅
- KVM Validation: debug (only selftest_mptcp_join): Success! ✅
- KVM Validation: btf-normal (only bpftest_all): Success! ✅
- KVM Validation: btf-debug (only bpftest_all): Success! ✅
- Perf: Success! ✅
- Task: https://github.com/multipath-tcp/mptcp_net-next/actions/runs/34828601595

Initiator: Patchew Applier
Commits: https://github.com/multipath-tcp/mptcp_net-next/commits/b7129410a677
Patchwork: https://patchwork.kernel.org/project/mptcp/list/?series=1164353


If there are some issues, you can reproduce them using the same environment as
the one used by the CI thanks to a docker image, e.g.:

    $ cd [kernel source code]
    $ docker run -v "${PWD}:${PWD}:rw" -w "${PWD}" --privileged --rm -it \
        --pull always mptcp/mptcp-upstream-virtme-docker:latest \
        auto-normal

For more details:

    https://github.com/multipath-tcp/mptcp-upstream-virtme-docker


Please note that despite all the efforts that have been already done to have a
stable tests suite when executed on a public CI like here, it is possible some
reported issues are not due to your modifications. Still, do not hesitate to
help us improve that ;-)

Cheers,
MPTCP GH Action bot
Bot operated by Matthieu Baerts (NGI0 Core)