[PATCH v6 mptcp-next 0/7] mptcp: address stall under memory pressure

Paolo Abeni posted 7 patches 1 week, 3 days ago
Patches applied successfully (tree, apply log)
git fetch https://github.com/multipath-tcp/mptcp_net-next tags/patchew/cover.1778835009.git.pabeni@redhat.com
There is a newer version of this series
net/mptcp/mib.c      |   2 +
net/mptcp/mib.h      |   2 +
net/mptcp/options.c  |  67 +++++++++++-
net/mptcp/protocol.c | 249 ++++++++++++++++++++++++++++++++-----------
net/mptcp/protocol.h |   4 +
net/mptcp/subflow.c  |   1 +
6 files changed, 259 insertions(+), 66 deletions(-)
[PATCH v6 mptcp-next 0/7] mptcp: address stall under memory pressure
Posted by Paolo Abeni 1 week, 3 days ago
This an attempt to fix the data transfer stall reported by Geliang and
Gang more carefully enforcing memory constraints at the MPTCP level.

This iteration presents a significant change WRT the previous one,
avoiding entirely the collapse attempt on memory pressure. Note that
this choice represent a trade off: collapsing allow much faster transfer
(to be more accurate: order of magnitude less slow) under some extreme
conditions, but makes transfer slower and much more CPU intensive for
less unlikely conditions.

As a consequence of the above the `mptcp_data.multi_chunk_sendfile`
test-case needs a 240 seconds timeout to complete successfully:

TEST_F_TIMEOUT(mptcp, multi_chunk_sendfile, 240)

The solution performing data collapsing would need similar long timeout
for the multiproc tests cases: mutliproc_even, mutliproc_readers,
mutliproc_writers, mutliproc_sendpage_even, mutliproc_sendpage_readers,
mutliproc_sendpage_writers.

Patch 1 is new in v6, and is actually a fix for an old issue (targeting
net), included here just for my convenience.

Patch 2 and 3 makes the admission check much more strict for incoming
packets exceeding the memory limits, with some exception for fallback
sockets.
Patch 4 makes implement OoO queue pruning for MPTCP and patch 5
addresses an edge scenario that could still lead to transfer stall
under memory pressure.
Finally patch 6 and 7 improve the MPTCP-level retransmission schema to
make recovery from memory pressure/after MPTCP-level drop significanly
faster.
---
Paolo Abeni (7):
  mptcp: fix missing wakeups in edge scenarios
  mptcp: explicitly drop over memory limits
  mptcp: enforce hard limit on backlog flushing
  mptcp: implemented OoO queue pruning
  mptcp: track prune recovery status
  mptcp: move the retrans loop to a separate helper
  mptcp: let the retrans scheduler do its job.

 net/mptcp/mib.c      |   2 +
 net/mptcp/mib.h      |   2 +
 net/mptcp/options.c  |  67 +++++++++++-
 net/mptcp/protocol.c | 249 ++++++++++++++++++++++++++++++++-----------
 net/mptcp/protocol.h |   4 +
 net/mptcp/subflow.c  |   1 +
 6 files changed, 259 insertions(+), 66 deletions(-)

-- 
2.54.0
Re: [PATCH v6 mptcp-next 0/7] mptcp: address stall under memory pressure
Posted by MPTCP CI 1 week, 3 days ago
Hi Paolo,

Thank you for your modifications, that's great!

Our CI did some validations and here is its report:

- KVM Validation: normal (except selftest_mptcp_join): Success! ✅
- KVM Validation: normal (only selftest_mptcp_join): Success! ✅
- KVM Validation: debug (except selftest_mptcp_join): Success! ✅
- KVM Validation: debug (only selftest_mptcp_join): Success! ✅
- KVM Validation: btf-normal (only bpftest_all): Success! ✅
- KVM Validation: btf-debug (only bpftest_all): Success! ✅
- Task: https://github.com/multipath-tcp/mptcp_net-next/actions/runs/25910612780

Initiator: Patchew Applier
Commits: https://github.com/multipath-tcp/mptcp_net-next/commits/0c52f631034a
Patchwork: https://patchwork.kernel.org/project/mptcp/list/?series=1095238


If there are some issues, you can reproduce them using the same environment as
the one used by the CI thanks to a docker image, e.g.:

    $ cd [kernel source code]
    $ docker run -v "${PWD}:${PWD}:rw" -w "${PWD}" --privileged --rm -it \
        --pull always mptcp/mptcp-upstream-virtme-docker:latest \
        auto-normal

For more details:

    https://github.com/multipath-tcp/mptcp-upstream-virtme-docker


Please note that despite all the efforts that have been already done to have a
stable tests suite when executed on a public CI like here, it is possible some
reported issues are not due to your modifications. Still, do not hesitate to
help us improve that ;-)

Cheers,
MPTCP GH Action bot
Bot operated by Matthieu Baerts (NGI0 Core)
Re: [PATCH v6 mptcp-next 0/7] mptcp: address stall under memory pressure
Posted by Paolo Abeni 1 week, 3 days ago
On 5/15/26 11:07 AM, Paolo Abeni wrote:
> This an attempt to fix the data transfer stall reported by Geliang and
> Gang more carefully enforcing memory constraints at the MPTCP level.
> 
> This iteration presents a significant change WRT the previous one,
> avoiding entirely the collapse attempt on memory pressure. Note that
> this choice represent a trade off: collapsing allow much faster transfer
> (to be more accurate: order of magnitude less slow) under some extreme
> conditions, but makes transfer slower and much more CPU intensive for
> less unlikely conditions.
> 
> As a consequence of the above the `mptcp_data.multi_chunk_sendfile`
> test-case needs a 240 seconds timeout to complete successfully:
> 
> TEST_F_TIMEOUT(mptcp, multi_chunk_sendfile, 240)
> 
> The solution performing data collapsing would need similar long timeout
> for the multiproc tests cases: mutliproc_even, mutliproc_readers,
> mutliproc_writers, mutliproc_sendpage_even, mutliproc_sendpage_readers,
> mutliproc_sendpage_writers.
> 
> Patch 1 is new in v6, and is actually a fix for an old issue (targeting
> net), included here just for my convenience.
> 
> Patch 2 and 3 makes the admission check much more strict for incoming
> packets exceeding the memory limits, with some exception for fallback
> sockets.
> Patch 4 makes implement OoO queue pruning for MPTCP and patch 5
> addresses an edge scenario that could still lead to transfer stall
> under memory pressure.
> Finally patch 6 and 7 improve the MPTCP-level retransmission schema to
> make recovery from memory pressure/after MPTCP-level drop significanly
> faster.

@Geliang, @Gang: could you please have a spin at this iteration? Note
that you must increase the timeout for the
mptcp_data.multi_chunk_sendfile test-case, as mentioned above.

Side note: with the "collapse" code this revision also omitted a related
few refactor patches, that I still plan to upstream later, since the
effect is a nice cleanup and reducing differences VS plain TCP.

/P
Re: [PATCH v6 mptcp-next 0/7] mptcp: address stall under memory pressure
Posted by gang.yan@linux.dev 1 week ago
May 15, 2026 at 5:29 PM, "Paolo Abeni" <pabeni@redhat.com mailto:pabeni@redhat.com?to=%22Paolo%20Abeni%22%20%3Cpabeni%40redhat.com%3E > wrote:


> 
> On 5/15/26 11:07 AM, Paolo Abeni wrote:
> 
> > 
> > This an attempt to fix the data transfer stall reported by Geliang and
> >  Gang more carefully enforcing memory constraints at the MPTCP level.
> >  
> >  This iteration presents a significant change WRT the previous one,
> >  avoiding entirely the collapse attempt on memory pressure. Note that
> >  this choice represent a trade off: collapsing allow much faster transfer
> >  (to be more accurate: order of magnitude less slow) under some extreme
> >  conditions, but makes transfer slower and much more CPU intensive for
> >  less unlikely conditions.
> >  
> >  As a consequence of the above the `mptcp_data.multi_chunk_sendfile`
> >  test-case needs a 240 seconds timeout to complete successfully:
> >  
> >  TEST_F_TIMEOUT(mptcp, multi_chunk_sendfile, 240)
> >  
> >  The solution performing data collapsing would need similar long timeout
> >  for the multiproc tests cases: mutliproc_even, mutliproc_readers,
> >  mutliproc_writers, mutliproc_sendpage_even, mutliproc_sendpage_readers,
> >  mutliproc_sendpage_writers.
> >  
> >  Patch 1 is new in v6, and is actually a fix for an old issue (targeting
> >  net), included here just for my convenience.
> >  
> >  Patch 2 and 3 makes the admission check much more strict for incoming
> >  packets exceeding the memory limits, with some exception for fallback
> >  sockets.
> >  Patch 4 makes implement OoO queue pruning for MPTCP and patch 5
> >  addresses an edge scenario that could still lead to transfer stall
> >  under memory pressure.
> >  Finally patch 6 and 7 improve the MPTCP-level retransmission schema to
> >  make recovery from memory pressure/after MPTCP-level drop significanly
> >  faster.
> > 
> @Geliang, @Gang: could you please have a spin at this iteration? Note
> that you must increase the timeout for the
> mptcp_data.multi_chunk_sendfile test-case, as mentioned above.
> 
Hi Paolo,

Thanks a lot for the new patch set. We have encountered some issues during
our testing. It will take us some time to analyze and locate the root cause.
We will post updates on the mailing list as soon as we make progress.

Best regards
Gang

> Side note: with the "collapse" code this revision also omitted a related
> few refactor patches, that I still plan to upstream later, since the
> effect is a nice cleanup and reducing differences VS plain TCP.
> 
> /P
>
Re: [PATCH v6 mptcp-next 0/7] mptcp: address stall under memory pressure
Posted by Paolo Abeni 1 week ago
On 5/18/26 10:13 AM, gang.yan@linux.dev wrote:
> May 15, 2026 at 5:29 PM, "Paolo Abeni" <pabeni@redhat.com mailto:pabeni@redhat.com?to=%22Paolo%20Abeni%22%20%3Cpabeni%40redhat.com%3E > wrote:
>> On 5/15/26 11:07 AM, Paolo Abeni wrote:
>>> This an attempt to fix the data transfer stall reported by Geliang and
>>>  Gang more carefully enforcing memory constraints at the MPTCP level.
>>>  
>>>  This iteration presents a significant change WRT the previous one,
>>>  avoiding entirely the collapse attempt on memory pressure. Note that
>>>  this choice represent a trade off: collapsing allow much faster transfer
>>>  (to be more accurate: order of magnitude less slow) under some extreme
>>>  conditions, but makes transfer slower and much more CPU intensive for
>>>  less unlikely conditions.
>>>  
>>>  As a consequence of the above the `mptcp_data.multi_chunk_sendfile`
>>>  test-case needs a 240 seconds timeout to complete successfully:
>>>  
>>>  TEST_F_TIMEOUT(mptcp, multi_chunk_sendfile, 240)
>>>  
>>>  The solution performing data collapsing would need similar long timeout
>>>  for the multiproc tests cases: mutliproc_even, mutliproc_readers,
>>>  mutliproc_writers, mutliproc_sendpage_even, mutliproc_sendpage_readers,
>>>  mutliproc_sendpage_writers.
>>>  
>>>  Patch 1 is new in v6, and is actually a fix for an old issue (targeting
>>>  net), included here just for my convenience.
>>>  
>>>  Patch 2 and 3 makes the admission check much more strict for incoming
>>>  packets exceeding the memory limits, with some exception for fallback
>>>  sockets.
>>>  Patch 4 makes implement OoO queue pruning for MPTCP and patch 5
>>>  addresses an edge scenario that could still lead to transfer stall
>>>  under memory pressure.
>>>  Finally patch 6 and 7 improve the MPTCP-level retransmission schema to
>>>  make recovery from memory pressure/after MPTCP-level drop significanly
>>>  faster.
>>>
>> @Geliang, @Gang: could you please have a spin at this iteration? Note
>> that you must increase the timeout for the
>> mptcp_data.multi_chunk_sendfile test-case, as mentioned above.
>>
> Hi Paolo,
> 
> Thanks a lot for the new patch set. We have encountered some issues during
> our testing. It will take us some time to analyze and locate the root cause.
> We will post updates on the mailing list as soon as we make progress.

I run a few hundred iterations of the mptcp_data test without observing
any issue (still running). Can you please share which issue are you
observing, the relevant tests case and the build type (debug/non debug)?

Thanks,

Paolo
Re: [PATCH v6 mptcp-next 0/7] mptcp: address stall under memory pressure
Posted by gang.yan@linux.dev 6 days, 8 hours ago
May 19, 2026 at 12:30 AM, "Paolo Abeni" <pabeni@redhat.com mailto:pabeni@redhat.com?to=%22Paolo%20Abeni%22%20%3Cpabeni%40redhat.com%3E > wrote:


> 
> On 5/18/26 10:13 AM, gang.yan@linux.dev wrote:
> 
> > 
> > May 15, 2026 at 5:29 PM, "Paolo Abeni" <pabeni@redhat.com mailto:pabeni@redhat.com?to=%22Paolo%20Abeni%22%20%3Cpabeni%40redhat.com%3E > wrote:
> > 
> > > 
> > > On 5/15/26 11:07 AM, Paolo Abeni wrote:
> > > 
> >  This an attempt to fix the data transfer stall reported by Geliang and
> >  Gang more carefully enforcing memory constraints at the MPTCP level.
> >  
> >  This iteration presents a significant change WRT the previous one,
> >  avoiding entirely the collapse attempt on memory pressure. Note that
> >  this choice represent a trade off: collapsing allow much faster transfer
> >  (to be more accurate: order of magnitude less slow) under some extreme
> >  conditions, but makes transfer slower and much more CPU intensive for
> >  less unlikely conditions.
> >  
> >  As a consequence of the above the `mptcp_data.multi_chunk_sendfile`
> >  test-case needs a 240 seconds timeout to complete successfully:
> >  
> >  TEST_F_TIMEOUT(mptcp, multi_chunk_sendfile, 240)
> >  
> >  The solution performing data collapsing would need similar long timeout
> >  for the multiproc tests cases: mutliproc_even, mutliproc_readers,
> >  mutliproc_writers, mutliproc_sendpage_even, mutliproc_sendpage_readers,
> >  mutliproc_sendpage_writers.
> >  
> >  Patch 1 is new in v6, and is actually a fix for an old issue (targeting
> >  net), included here just for my convenience.
> >  
> >  Patch 2 and 3 makes the admission check much more strict for incoming
> >  packets exceeding the memory limits, with some exception for fallback
> >  sockets.
> >  Patch 4 makes implement OoO queue pruning for MPTCP and patch 5
> >  addresses an edge scenario that could still lead to transfer stall
> >  under memory pressure.
> >  Finally patch 6 and 7 improve the MPTCP-level retransmission schema to
> >  make recovery from memory pressure/after MPTCP-level drop significanly
> >  faster.
> > 
> > > 
> > > @Geliang, @Gang: could you please have a spin at this iteration? Note
> > >  that you must increase the timeout for the
> > >  mptcp_data.multi_chunk_sendfile test-case, as mentioned above.
> > > 
> >  Hi Paolo,
> >  
> >  Thanks a lot for the new patch set. We have encountered some issues during
> >  our testing. It will take us some time to analyze and locate the root cause.
> >  We will post updates on the mailing list as soon as we make progress.
> > 
> I run a few hundred iterations of the mptcp_data test without observing
> any issue (still running). Can you please share which issue are you
> observing, the relevant tests case and the build type (debug/non debug)?
> 
> Thanks,
>
Hi Paolo,

Sorry for not providing the specific details earlier in my first message.
The issue we are seeing occurs specifically when KTLS is enabled.

Today I ran several sets of comparative tests (many hundreds iterations), and the
current results suggest that the integration framework between KTLS and MPTCP
may need some adjustments. We are investigating the root cause.

Thanks for your help.

Cherrs,
Gang

> Paolo
>
Re: [PATCH v6 mptcp-next 0/7] mptcp: address stall under memory pressure
Posted by Paolo Abeni 6 days, 1 hour ago
On 5/19/26 10:50 AM, gang.yan@linux.dev wrote:
> May 19, 2026 at 12:30 AM, "Paolo Abeni" <pabeni@redhat.com mailto:pabeni@redhat.com?to=%22Paolo%20Abeni%22%20%3Cpabeni%40redhat.com%3E > wrote:
>> On 5/18/26 10:13 AM, gang.yan@linux.dev wrote:
>>>  Thanks a lot for the new patch set. We have encountered some issues during
>>>  our testing. It will take us some time to analyze and locate the root cause.
>>>  We will post updates on the mailing list as soon as we make progress.
>>>
>> I run a few hundred iterations of the mptcp_data test without observing
>> any issue (still running). Can you please share which issue are you
>> observing, the relevant tests case and the build type (debug/non debug)?
>
> Sorry for not providing the specific details earlier in my first message.
> The issue we are seeing occurs specifically when KTLS is enabled.
> 
> Today I ran several sets of comparative tests (many hundreds iterations), and the
> current results suggest that the integration framework between KTLS and MPTCP
> may need some adjustments. We are investigating the root cause.

Reading the above, I understand that testing vs plain MPTCP gives good
results, am I correct? In any case I'll post a v7 to deal with some of
the feedback reported by sashiko on v6.

Thanks,

Paolo