From nobody Sat Sep 26 12:32:58 2026 Received: from sender6-of-o52.zoho.com (sender6-of-o52.zoho.com [165.173.180.52]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id AFE3F3D9027 for ; Fri, 11 Sep 2026 09:58:08 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=pass smtp.client-ip=165.173.180.52 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789120690; cv=pass; b=Og0f1DBVP6gXVPeTEyVxqxZcfShIfjLWhG73I9yaLoZxaaMI93dw3soQ1fLyLA5MAOhlkcdts4PaV93njvJAifSgj6nOUFLEhG1NYdJoCmEIYO0bKjJE9MTpaItSv3y8EjhFShCwh8NmVaHH4jF+Fj+OMKHFDIwk+b1qr9wsr/M= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789120690; c=relaxed/simple; bh=rwKkAmfKb2W1kmh4zsjOb0NlrArYQIeKEzK1//x0qzU=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=iunN8ES5nQCZgZ+34iudnQwZ5z+bpdTre4+DrQFn4BfpVhCyYVUCfaw7FuJm116puk0hVFqwzGeQMjiaQpWuYPLIzJ3o9yVIJsL8v0Fc2DENbERxuORNUbovh9aTcfdvOLmo/11niAw/MY1JlNpYL5lSmR44/78G6xMzcrXeCtQ= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=mpiricsoftware.com; spf=pass smtp.mailfrom=mpiricsoftware.com; dkim=fail (0-bit key) header.d=mpiricsoftware.com header.i=shardul.b@mpiricsoftware.com header.b=t4I0ct9r reason="key not found in DNS"; arc=pass smtp.client-ip=165.173.180.52 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=mpiricsoftware.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=mpiricsoftware.com Authentication-Results: smtp.subspace.kernel.org; dkim=fail reason="key not found in DNS" (0-bit key) header.d=mpiricsoftware.com header.i=shardul.b@mpiricsoftware.com header.b="t4I0ct9r" ARC-Seal: i=1; a=rsa-sha256; t=1789120678; cv=none; d=zohomail.com; s=zohoarc; b=AbxJ/OvrHYMGhAXrBlKANbwuTgxo3z8dqry0/bv9oBNslIN0HoYfVOvlaeM5fxjsNsIii3awxXj1dSkIOwx4thmzenj8lb8znkn5ffQ2MahIbspV87apdFM1u37CmJBE0PSHDPvGh7052bHq+5sBkWm7N7BcI9zGBtteemKQsNQ= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1789120678; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:MIME-Version:Message-ID:Subject:Subject:To:To:Message-Id:Reply-To; bh=9nbWN+oEUxSze9sT6JkzIRY9/EqEqpRjIr2rFD7yPWc=; b=W8abGGb0/7nc3SpkGO172lOPx9ugJSVHrQOLccgNNxLot77qLN5kjpuFuAi1gc87ai1dRN25SBmKvnfA/5W7HyOaCuQLUu6OH0HRt5y1GVOJveeTdusO3MwLvijmtFBEa0NQZd36BGWa4a+MUsoPUrZJUuvgxwzj8zgAyxlVrDk= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass header.i=mpiricsoftware.com; spf=pass smtp.mailfrom=shardul.b@mpiricsoftware.com; dmarc=pass header.from= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; t=1789120678; s=mpiric; d=mpiricsoftware.com; i=shardul.b@mpiricsoftware.com; h=From:From:Date:Date:Subject:Subject:MIME-Version:Content-Type:Content-Transfer-Encoding:Message-Id:Message-Id:In-Reply-To:To:To:Cc:Cc:Reply-To; bh=9nbWN+oEUxSze9sT6JkzIRY9/EqEqpRjIr2rFD7yPWc=; b=t4I0ct9rW7snu+Yp3PFpKhnOdid2OWgYtM3rE2TZhPNP3GKMvcaMIw5wVOKMXgwW ix17uF+ssVVZ6LI9zluuzx9yGEHnxublIp/lrbI6HMz+/phsPvrt4hSGdnnMau6okIr NMwy4ZbBmIUl/M+tSPTEQzV5r3+LzhLSmnBglkGU= Received: by smtp.zohomail.com with SMTPS id 1789120677164968.410148950479; Fri, 11 Sep 2026 02:57:57 -0700 (PDT) From: Shardul Bankar Date: Fri, 11 Sep 2026 15:27:44 +0530 Subject: [PATCH mptcp-next v4 1/4] mptcp: sched: avoid truncating the pacing rate in the scheduler Precedence: bulk X-Mailing-List: mptcp@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260911-mptcp_penalise_send_v2-v4-1-820f2be4a8fe@mpiricsoftware.com> References: <20260911-mptcp_penalise_send_v2-v4-0-820f2be4a8fe@mpiricsoftware.com> In-Reply-To: <20260911-mptcp_penalise_send_v2-v4-0-820f2be4a8fe@mpiricsoftware.com> To: mptcp@lists.linux.dev Cc: "Matthieu Baerts (NGI0)" , Janak Patel , Kalpan Jani , Shardul Bankar , Shardul Bankar X-Mailer: b4 0.15.2 X-ZohoMailClient: External mptcp_subflow_get_send() derives each subflow's linger time and a running pacing-rate average through a u32 "pace", while sk_pacing_rate and avg_pacing_rate are unsigned long. The u32 truncates pacing rates above 4 GB/s, skewing subflow selection on fast paths. The running average has a second problem that widening "pace" does not address: it forms two rate * bytes products before dividing, and only the avg_pacing_rate term carries a (u64) cast. On 32-bit systems the uncast sk_pacing_rate * burst wraps at ordinary pacing rates. On 64-bit that product is safe, but avg_pacing_rate * wmem still wraps when a high pacing rate meets a send queue near the 2GB ceiling sk_wmem_queued allows. Either way a wrapped term can report the fastest subflow as the slowest. Widen "pace" to unsigned long, divide the linger time with a 64-bit divisor, and compute each product of the average in a 128-bit intermediate with mul_u64_u64_div_u64(), which bounds the error at one unit per term. Dividing each term before multiplying would avoid the wrap more cheaply, but that quotient truncates to zero once the queue holds a second or more of data, which is exactly the bufferbloated subflow this estimate needs to describe. Fixes: d5f49190def6 ("mptcp: allow picking different xmit subflows") Fixes: 3ce0852c86b9 ("mptcp: enforce HoL-blocking estimation") Suggested-by: Matthieu Baerts (NGI0) Assisted-by: LLM Signed-off-by: Shardul Bankar --- net/mptcp/protocol.c | 14 +++++++++----- 1 file changed, 9 insertions(+), 5 deletions(-) diff --git a/net/mptcp/protocol.c b/net/mptcp/protocol.c index 0b24e0afedfb..1c3c1c902d1d 100644 --- a/net/mptcp/protocol.c +++ b/net/mptcp/protocol.c @@ -1628,7 +1628,8 @@ struct sock *mptcp_subflow_get_send(struct mptcp_sock= *msk) struct subflow_send_info send_info[SSK_MODE_MAX]; struct mptcp_subflow_context *subflow; struct sock *sk =3D (struct sock *)msk; - u32 pace, burst, wmem; + unsigned long pace; + u32 burst, wmem; int i, nr_active =3D 0; struct sock *ssk; u64 linger_time; @@ -1659,7 +1660,7 @@ struct sock *mptcp_subflow_get_send(struct mptcp_sock= *msk) continue; } =20 - linger_time =3D div_u64((u64)READ_ONCE(ssk->sk_wmem_queued) << 32, pace); + linger_time =3D div64_u64((u64)READ_ONCE(ssk->sk_wmem_queued) << 32, pac= e); if (linger_time < send_info[backup].linger_time) { send_info[backup].ssk =3D ssk; send_info[backup].linger_time =3D linger_time; @@ -1692,9 +1693,12 @@ struct sock *mptcp_subflow_get_send(struct mptcp_soc= k *msk) return ssk; =20 subflow =3D mptcp_subflow_ctx(ssk); - subflow->avg_pacing_rate =3D div_u64((u64)subflow->avg_pacing_rate * wmem= + - READ_ONCE(ssk->sk_pacing_rate) * burst, - burst + wmem); + /* 64-bit arithmetic overflows at high rate with a large queue */ + subflow->avg_pacing_rate =3D + mul_u64_u64_div_u64(subflow->avg_pacing_rate, wmem, + burst + wmem) + + mul_u64_u64_div_u64(READ_ONCE(ssk->sk_pacing_rate), burst, + burst + wmem); msk->snd_burst =3D burst; return ssk; } --=20 2.34.1 From nobody Sat Sep 26 12:32:58 2026 Received: from sender6-of-o52.zoho.com (sender6-of-o52.zoho.com [165.173.180.52]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 038113D9531 for ; Fri, 11 Sep 2026 09:58:15 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=pass smtp.client-ip=165.173.180.52 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789120697; cv=pass; b=rX2uo57g0r6RRRik00+taatL1Zj3PITt3Q+qViXL2dzmZXTGVBKT1dYYogzQ2EVuC/+Xe8ltPAp5Qqc0cENhxnxttP89b7c7qDRuTfBj06+XvKQmdbCmXj887bxkYEQItwYFpGGu9yt19GEJi2XATJ7BKVUxWkmWtS9Yn5Tf4AI= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789120697; c=relaxed/simple; bh=yPqctiTxMcMoB+5vfxTctzGVoEZTl64GYI7J4p/puxI=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=gcfC058SA5Qn6tspUEGgafvKBbsz194NTSc0lnTx+sGWHRxQ8VHnZdfYzCVIwfXlQQXQY1nUq4SRNFl24K4uKhM8vKDkG70ycfiI8BqmJTyIAXc3BeiK8ogyINseOsTKHfsmk2zIEXpuMvp0WFlY61IRlYqBoQRGHwGraiRBW30= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=mpiricsoftware.com; spf=pass smtp.mailfrom=mpiricsoftware.com; dkim=fail (0-bit key) header.d=mpiricsoftware.com header.i=shardul.b@mpiricsoftware.com header.b=qLK3UP4w reason="key not found in DNS"; arc=pass smtp.client-ip=165.173.180.52 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=mpiricsoftware.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=mpiricsoftware.com Authentication-Results: smtp.subspace.kernel.org; dkim=fail reason="key not found in DNS" (0-bit key) header.d=mpiricsoftware.com header.i=shardul.b@mpiricsoftware.com header.b="qLK3UP4w" ARC-Seal: i=1; a=rsa-sha256; t=1789120680; cv=none; d=zohomail.com; s=zohoarc; b=AEs+/yehj75mo/75ZBws4g0dPGx7RYE/h0bcUnmNPlSYbIcvOwPkY9u84V24vBxsGFRnN1wJPxkWoUq3IabazYqcRtPkKJiBakpnaPCTk4Ae65cUjzxs/dyIH9qhblhnQPikn07MrLfJCyqNqPr9eyjiYXrbsi322D/ouWhc1pM= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1789120680; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:MIME-Version:Message-ID:Subject:Subject:To:To:Message-Id:Reply-To; bh=tQewcVk8YvVG4bh8ZcBDavyRZVpbUnkoO5TTjMhd4TM=; b=UiyHk/fhkNQF6PHTHAC2KRTkLoSspAmXvRT0qo++65yv0KAdSu4s5lmzt5SnAwIQZsI0L/z47JS97M43c1kWEd6FyJClLiUm66ncZHQ7kYoIZ0Hiy40R0WZ8Xa2Hmhc/RQ5DktQfDLDkHXYM8ToCGI4OBO4BaRnDCa02qHrRhBw= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass header.i=mpiricsoftware.com; spf=pass smtp.mailfrom=shardul.b@mpiricsoftware.com; dmarc=pass header.from= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; t=1789120680; s=mpiric; d=mpiricsoftware.com; i=shardul.b@mpiricsoftware.com; h=From:From:Date:Date:Subject:Subject:MIME-Version:Content-Type:Content-Transfer-Encoding:Message-Id:Message-Id:In-Reply-To:To:To:Cc:Cc:Reply-To; bh=tQewcVk8YvVG4bh8ZcBDavyRZVpbUnkoO5TTjMhd4TM=; b=qLK3UP4wM9jENrgTwmP6LMw1WJBNXsEKEafTlPiIm6wVIcbu5OhvZDxvIoZdw7XW 9z1eFRH2re26SuZI/SM+QYVvgPyQ7XL7bQu+fkp3btufm4UAtbBWaESUB8nDALX1uoN lk7qsRGNiAg2FeP7/C5BouO785hdnSgVXKXNFkVM= Received: by smtp.zohomail.com with SMTPS id 1789120679804421.458447629494; Fri, 11 Sep 2026 02:57:59 -0700 (PDT) From: Shardul Bankar Date: Fri, 11 Sep 2026 15:27:45 +0530 Subject: [PATCH mptcp-next v4 2/4] mptcp: sched: penalise a slow subflow by halving its cwnd Precedence: bulk X-Mailing-List: mptcp@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260911-mptcp_penalise_send_v2-v4-2-820f2be4a8fe@mpiricsoftware.com> References: <20260911-mptcp_penalise_send_v2-v4-0-820f2be4a8fe@mpiricsoftware.com> In-Reply-To: <20260911-mptcp_penalise_send_v2-v4-0-820f2be4a8fe@mpiricsoftware.com> To: mptcp@lists.linux.dev Cc: "Matthieu Baerts (NGI0)" , Janak Patel , Kalpan Jani , Shardul Bankar , Shardul Bankar X-Mailer: b4 0.15.2 X-ZohoMailClient: External A poorly-performing but usable subflow (high latency, loss, bufferbloat) can soak up connection resources and cause head-of-line blocking of the aggregate stream. Give the default packet scheduler a way to send less than such a subflow's full congestion window. Once a subflow has been picked for transmission, flag it for penalisation when: - its smoothed pacing rate (avg_pacing_rate) is below half that of the fastest non-backup path, keying on rate, not RTT, so a slow-but-high-throughput path is left alone; - the fastest path is cwnd-limited (saturated), so shifting load off the slow path is worthwhile; - the subflow is in TCP_CA_Open, so its cwnd is not already being reduced by loss recovery; - its cwnd is still above MPTCP_PENALISE_MIN_CWND, so a subflow already at the floor is left to recover instead of being churned by a halving that can no longer reduce it; - it has not been penalised in the last RTT. Backup subflows do not take part in the fastest-path comparison. While an active subflow exists a backup carries no data, so both its rate estimate and its cwnd-limited state go stale; using one as the reference would penalise the subflow that is actually sending. The reduction halves tcp_snd_cwnd, floored at MPTCP_PENALISE_MIN_CWND, and halves ssthresh when cwnd is at or above it, following the out-of-tree mptcp.org implementation's ssthresh handling. The congestion control is left to raise the window again; this adds no MPTCP-side probe or timer. Split the conditions by the lock that covers them. The scheduler runs under the msk lock and only marks a candidate, using the rate comparison against the fastest subflow. The conditions belonging to the subflow itself (the cwnd floor, the CA_Open state and the once-per-RTT limiter) are evaluated where the penalty is applied, under that subflow's own socket lock, so the limiter's timestamp is written and read under one lock. The rate comparison reads the fastest subflow's state without holding its lock: that is a different socket from the one being pushed, and taking a second subflow's socket lock in the send path is not acceptable. The field is a bitfield, so READ_ONCE() does not apply; annotate it with data_race(). Halving the window without telling the congestion control is deliberately a simple first step: not every congestion control reacts to an external change the same way, and some may undo it quickly. Closes: https://github.com/multipath-tcp/mptcp_net-next/issues/345 Link: https://github.com/multipath-tcp/mptcp/blob/825b62d597b5/net/mptcp/mp= tcp_sched.c#L300-L330 Assisted-by: LLM Co-developed-by: Matthieu Baerts (NGI0) Signed-off-by: Shardul Bankar --- net/mptcp/protocol.c | 65 ++++++++++++++++++++++++++++++++++++++++++++++++= +--- net/mptcp/protocol.h | 2 ++ 2 files changed, 64 insertions(+), 3 deletions(-) diff --git a/net/mptcp/protocol.c b/net/mptcp/protocol.c index 1c3c1c902d1d..0417bfb3db7f 100644 --- a/net/mptcp/protocol.c +++ b/net/mptcp/protocol.c @@ -1619,6 +1619,45 @@ bool mptcp_subflow_active(struct mptcp_subflow_conte= xt *subflow) #define SSK_MODE_BACKUP 1 #define SSK_MODE_MAX 2 =20 +/* Penalise a subflow pacing below the fastest path's rate / this ratio */ +#define MPTCP_PENALISE_RATE_RATIO 2 +/* cwnd floor for the penalty: skip a subflow already there, and never red= uce + * below it (halving at the floor only churns congestion control). + */ +#define MPTCP_PENALISE_MIN_CWND 2 + +/* Rate-limit the penalty to at most once per subflow RTT, so the congesti= on + * control can grow the window back between reductions. + */ +static bool mptcp_penalise_throttle_ok(struct mptcp_subflow_context *subfl= ow) +{ + struct sock *ssk =3D mptcp_subflow_tcp_sock(subflow); + u32 rtt =3D usecs_to_jiffies(READ_ONCE(tcp_sk(ssk)->srtt_us) >> 3); + + return tcp_jiffies32 - subflow->last_penalise >=3D max_t(u32, rtt, 1); +} + +/* Halve cwnd, and ssthresh when cwnd is at or above it, under the subflow + * socket lock: the scheduler only marks a candidate, the conditions on th= is + * subflow are evaluated here, on current state. + */ +static void mptcp_penalise_cwnd(struct sock *ssk) +{ + struct mptcp_subflow_context *subflow =3D mptcp_subflow_ctx(ssk); + struct tcp_sock *tp =3D tcp_sk(ssk); + u32 cwnd =3D tcp_snd_cwnd(tp); + + subflow->penalise =3D false; + if (cwnd <=3D MPTCP_PENALISE_MIN_CWND || + inet_csk(ssk)->icsk_ca_state !=3D TCP_CA_Open || + !mptcp_penalise_throttle_ok(subflow)) + return; + subflow->last_penalise =3D tcp_jiffies32; + tcp_snd_cwnd_set(tp, max_t(u32, cwnd >> 1, MPTCP_PENALISE_MIN_CWND)); + if (cwnd >=3D tp->snd_ssthresh) + tp->snd_ssthresh =3D max_t(u32, tp->snd_ssthresh >> 1, 2); +} + /* implement the mptcp packet scheduler; * returns the subflow that will transmit the next DSS * additionally updates the rtx timeout @@ -1628,10 +1667,11 @@ struct sock *mptcp_subflow_get_send(struct mptcp_so= ck *msk) struct subflow_send_info send_info[SSK_MODE_MAX]; struct mptcp_subflow_context *subflow; struct sock *sk =3D (struct sock *)msk; - unsigned long pace; + unsigned long pace, max_pace =3D 0; u32 burst, wmem; + bool penal_cand, fast_limited; int i, nr_active =3D 0; - struct sock *ssk; + struct sock *ssk, *fastest =3D NULL; u64 linger_time; long tout =3D 0; =20 @@ -1660,6 +1700,12 @@ struct sock *mptcp_subflow_get_send(struct mptcp_soc= k *msk) continue; } =20 + /* fastest non-backup path; slower ones get throttled below */ + if (!backup && pace > max_pace) { + max_pace =3D pace; + fastest =3D ssk; + } + linger_time =3D div64_u64((u64)READ_ONCE(ssk->sk_wmem_queued) << 32, pac= e); if (linger_time < send_info[backup].linger_time) { send_info[backup].ssk =3D ssk; @@ -1687,12 +1733,22 @@ struct sock *mptcp_subflow_get_send(struct mptcp_so= ck *msk) if (!ssk || !sk_stream_memory_free(ssk)) return NULL; =20 + /* Flag a slow subflow for cwnd halving, applied in the push path */ + subflow =3D mptcp_subflow_ctx(ssk); + penal_cand =3D fastest && ssk !=3D fastest && + subflow->avg_pacing_rate < max_pace / MPTCP_PENALISE_RATE_RATIO; + /* We use data_race() because tcp_cwnd_validate() might change + * is_cwnd_limited under us + */ + fast_limited =3D penal_cand && data_race(tcp_is_cwnd_limited(fastest)); + burst =3D min(MPTCP_SEND_BURST_SIZE, mptcp_wnd_end(msk) - msk->snd_nxt); wmem =3D READ_ONCE(ssk->sk_wmem_queued); + /* the conditions on this subflow are re-checked on apply, under its lock= */ + subflow->penalise =3D burst && fast_limited; if (!burst) return ssk; =20 - subflow =3D mptcp_subflow_ctx(ssk); /* 64-bit arithmetic overflows at high rate with a large queue */ subflow->avg_pacing_rate =3D mul_u64_u64_div_u64(subflow->avg_pacing_rate, wmem, @@ -1752,6 +1808,9 @@ static int __subflow_push_pending(struct sock *sk, st= ruct sock *ssk, struct mptcp_data_frag *dfrag; int len, copied =3D 0, err =3D 0; =20 + if (mptcp_subflow_ctx(ssk)->penalise) + mptcp_penalise_cwnd(ssk); + while ((dfrag =3D mptcp_send_head(sk))) { info->sent =3D dfrag->already_sent; info->limit =3D dfrag->data_len; diff --git a/net/mptcp/protocol.h b/net/mptcp/protocol.h index 4453db7a33cf..1f2a709864d9 100644 --- a/net/mptcp/protocol.h +++ b/net/mptcp/protocol.h @@ -603,6 +603,7 @@ struct mptcp_subflow_context { __unused : 9; bool data_avail; bool scheduled; + bool penalise; /* scheduler flagged this subflow for cwnd halving */ bool pm_listener; /* a listener managed by the kernel PM? */ bool fully_established; /* path validated */ u32 lent_mem_frag; @@ -625,6 +626,7 @@ struct mptcp_subflow_context { */ =20 u32 subflow_id; + u32 last_penalise; /* tcp_jiffies32 of the last cwnd penalty */ =20 long delegated_status; unsigned long fail_tout; --=20 2.34.1 From nobody Sat Sep 26 12:32:58 2026 Received: from sender6-of-o52.zoho.com (sender6-of-o52.zoho.com [165.173.180.52]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5E9CF3AEF51 for ; Fri, 11 Sep 2026 09:58:24 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=pass smtp.client-ip=165.173.180.52 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789120705; cv=pass; b=s1R2MFH+uOsDdTXZUX62q5Cthpctoj8Wnd4trte8U58ssgasuvXWpdi9hM3+mHcAwUqyasa+iRXipamKqA5lMyniIcabKv20e3y4VYpnRNV2i85wIyEUx2fenxO7H0SRzDIleljTOK5Hfp831w/X0n71ey5HXgdkWiLDQ469iM0= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789120705; c=relaxed/simple; bh=0dfsUC5gSaxXM6gADQHRcF4eB/0FyPmNAqUm0+CJsiY=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=CrRRS9F4nHIXpf2h0Ff7cO47y/csz5EnejeGLSfATQcPl/Y+rQdVy8WyyBjEs0wmUbaQfKB/pbi0lwko0qf9qe/bTOGbkyIHmSjJbtpuyMCvwXVHIXtwUaCJWwg+NgFg3oD9DoEbdZKCTjlG9EkqB+444JPzbg/bbfT1MQBaK3M= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=mpiricsoftware.com; spf=pass smtp.mailfrom=mpiricsoftware.com; dkim=fail (0-bit key) header.d=mpiricsoftware.com header.i=shardul.b@mpiricsoftware.com header.b=fEfSVmuW reason="key not found in DNS"; arc=pass smtp.client-ip=165.173.180.52 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=mpiricsoftware.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=mpiricsoftware.com Authentication-Results: smtp.subspace.kernel.org; dkim=fail reason="key not found in DNS" (0-bit key) header.d=mpiricsoftware.com header.i=shardul.b@mpiricsoftware.com header.b="fEfSVmuW" ARC-Seal: i=1; a=rsa-sha256; t=1789120684; cv=none; d=zohomail.com; s=zohoarc; b=iO9ZrtVRY1dgDQrmkdssXoqviOgEFGbChnzmCZUwg1utnqF8s+NIlOCW1/sc+opnEXz/OvmLVu4O+eW00ld9yn+0xWoGs4a9WovgBVVttyjUErcw03roBL5x3XHSqGepv7GHrlXnqnSbX8UwsTehiRq+rYpsDB4ccHy4OmEhfhs= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1789120684; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:MIME-Version:Message-ID:Subject:Subject:To:To:Message-Id:Reply-To; bh=C5L7rW8GyfHzw+j6WS/3A3UlQfBV8N4PPBg+exF+n6w=; b=gIqagtyoZAiaAqtzviuFaM02RhGiUjpZSB7Wwwb4jCopROsijJiQDaUW/yfiYAXA59pUiC+UtEWoSnOpebl9M9tX4iD7FSrhkU3uHUU/fRB0V2ZDEm4f1HIK4jzS430mAJH718kNZm9gIsrimuKFHQVBmC2sdLszG9h6UI0mrTY= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass header.i=mpiricsoftware.com; spf=pass smtp.mailfrom=shardul.b@mpiricsoftware.com; dmarc=pass header.from= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; t=1789120684; s=mpiric; d=mpiricsoftware.com; i=shardul.b@mpiricsoftware.com; h=From:From:Date:Date:Subject:Subject:MIME-Version:Content-Type:Content-Transfer-Encoding:Message-Id:Message-Id:In-Reply-To:To:To:Cc:Cc:Reply-To; bh=C5L7rW8GyfHzw+j6WS/3A3UlQfBV8N4PPBg+exF+n6w=; b=fEfSVmuW9cfqAvfP+XIUkIxvx//cDbJlpO+roWh61VrWrDPIFAgGq2ciWiXcZvLx 934W/g0hCCcJ+xDRhOMwH0DLKa7r+/Ha1hH9yGtPsRfiYFJXElwr4GSzk8IhVwviGW2 6k1vPc4/PoXPMgiQQZ7mc2Hl6HDvvkd1bD4ipiis= Received: by smtp.zohomail.com with SMTPS id 1789120682434540.6945165920732; Fri, 11 Sep 2026 02:58:02 -0700 (PDT) From: Shardul Bankar Date: Fri, 11 Sep 2026 15:27:46 +0530 Subject: [PATCH mptcp-next v4 3/4] mptcp: sched: do not penalise when receive-window-limited Precedence: bulk X-Mailing-List: mptcp@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260911-mptcp_penalise_send_v2-v4-3-820f2be4a8fe@mpiricsoftware.com> References: <20260911-mptcp_penalise_send_v2-v4-0-820f2be4a8fe@mpiricsoftware.com> In-Reply-To: <20260911-mptcp_penalise_send_v2-v4-0-820f2be4a8fe@mpiricsoftware.com> To: mptcp@lists.linux.dev Cc: "Matthieu Baerts (NGI0)" , Janak Patel , Kalpan Jani , Shardul Bankar , Shardul Bankar X-Mailer: b4 0.15.2 X-ZohoMailClient: External The penalty in the previous patch shifts load off a slow subflow onto the fastest one, which only helps if the fastest path can absorb it. When the connection is receive-window-limited (the receiver's advertised window, not our congestion window, is the bottleneck), the fastest path is capped by that shared window too and cannot send more, so halving the slow path's cwnd just sheds its throughput. Gate on the application's queued data fitting within the send window: penalise only while write_seq <=3D wnd_end. If the application has queued past the window edge the receive window is the binding constraint, so skip the penalty. Neither write_seq (application demand) nor wnd_end (peer window) is derived from cwnd, so the test is not made circular by the window itself suppressing cwnd. Assisted-by: LLM Co-developed-by: Matthieu Baerts (NGI0) Signed-off-by: Shardul Bankar --- net/mptcp/protocol.c | 10 +++++++++- 1 file changed, 9 insertions(+), 1 deletion(-) diff --git a/net/mptcp/protocol.c b/net/mptcp/protocol.c index 0417bfb3db7f..24e1d6bd8577 100644 --- a/net/mptcp/protocol.c +++ b/net/mptcp/protocol.c @@ -1637,6 +1637,14 @@ static bool mptcp_penalise_throttle_ok(struct mptcp_= subflow_context *subflow) return tcp_jiffies32 - subflow->last_penalise >=3D max_t(u32, rtt, 1); } =20 +/* Like tcp_snd_wnd_test() but without an skb: true while queued data stil= l fits + * the send window, i.e. not receive-window-limited. + */ +static bool mptcp_snd_wnd_test(const struct mptcp_sock *msk) +{ + return !after64(msk->write_seq, mptcp_wnd_end(msk)); +} + /* Halve cwnd, and ssthresh when cwnd is at or above it, under the subflow * socket lock: the scheduler only marks a candidate, the conditions on th= is * subflow are evaluated here, on current state. @@ -1745,7 +1753,7 @@ struct sock *mptcp_subflow_get_send(struct mptcp_sock= *msk) burst =3D min(MPTCP_SEND_BURST_SIZE, mptcp_wnd_end(msk) - msk->snd_nxt); wmem =3D READ_ONCE(ssk->sk_wmem_queued); /* the conditions on this subflow are re-checked on apply, under its lock= */ - subflow->penalise =3D burst && fast_limited; + subflow->penalise =3D burst && fast_limited && mptcp_snd_wnd_test(msk); if (!burst) return ssk; =20 --=20 2.34.1 From nobody Sat Sep 26 12:32:58 2026 Received: from sender5-of-o52.zoho.com (sender5-of-o52.zoho.com [165.173.182.52]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 259623AEF51 for ; Fri, 11 Sep 2026 09:58:31 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=pass smtp.client-ip=165.173.182.52 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789120713; cv=pass; b=cncRHie5/oEUihUI3kobKAjnZXNy/SdhPLRgrMKiVseU1WEbwWZlEUNULedyPISpHChWYlTHpKGOpoOcmnMKKmGQzcsoeuYrycAEmJXmuYKBxPCbW8vOSckAljbXkSJdNsk2mAcWeoWBvp5zZ7M0e2QX91FIelOIFcyjk0nJJO8= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789120713; c=relaxed/simple; bh=QVhF43U47EOFGckHlU0zUQe5lHlnFxDkqHBpzI7ED8Y=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=rCSxcu9aONNGPrY06Y7TZRNLTuM/XwYxC3TrZlCK9DjTjBhjF2H+CR3Mr3fGB6s1LQnWRUSGiphZKmutE9H20OhcDYSJeEe8IJ07p5BTpGMmg1FRvE9ooPWbNyUDDp7prVBIltySr+xCMUiQ5PTj3msCOG/KxnHi+cyKt1Xmx5s= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=mpiricsoftware.com; spf=pass smtp.mailfrom=mpiricsoftware.com; dkim=fail (0-bit key) header.d=mpiricsoftware.com header.i=shardul.b@mpiricsoftware.com header.b=BTXNktvW reason="key not found in DNS"; arc=pass smtp.client-ip=165.173.182.52 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=mpiricsoftware.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=mpiricsoftware.com Authentication-Results: smtp.subspace.kernel.org; dkim=fail reason="key not found in DNS" (0-bit key) header.d=mpiricsoftware.com header.i=shardul.b@mpiricsoftware.com header.b="BTXNktvW" ARC-Seal: i=1; a=rsa-sha256; t=1789120686; cv=none; d=zohomail.com; s=zohoarc; b=b5SOXiZorBy/SvDhh/kKdM74JuQ5Q8+Wtt6n5d/VtLpQhx+zqvtCuvOlIH+XvphtHYGcoZw+wy7huFtFDk39f/iEirJxFP8zd+xpgmZWplXtL2GmnVULDw/eDIwua3cDr6jmlzTIpynDrUt2s7tqgD50RXAXa3Ir5ZiuuIihdBs= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1789120686; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:MIME-Version:Message-ID:Subject:Subject:To:To:Message-Id:Reply-To; bh=HOInaFlfUeMybGO4powUer8F0tv5KGhqMCQIEeTMSP8=; b=PUrtaESVvKci0MwU/+tkbs6+eBmjq3NPGKw+2zRfBjC+uxITxYFSLv8qUlOCj4zAcNhFdyZWHHCFeIZTp+agkh7wTpsqzze+wsmzOEQtVJyg/ReNLr7XPUOVfrcmTcKa/W7KiJCBN+BbXyeDy5d7+EPuBKZiagJwLN8W+LxWRnE= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass header.i=mpiricsoftware.com; spf=pass smtp.mailfrom=shardul.b@mpiricsoftware.com; dmarc=pass header.from= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; t=1789120686; s=mpiric; d=mpiricsoftware.com; i=shardul.b@mpiricsoftware.com; h=From:From:Date:Date:Subject:Subject:MIME-Version:Content-Type:Content-Transfer-Encoding:Message-Id:Message-Id:In-Reply-To:To:To:Cc:Cc:Reply-To; bh=HOInaFlfUeMybGO4powUer8F0tv5KGhqMCQIEeTMSP8=; b=BTXNktvWmW9riZ4DMM+FKD8uRse+FrxaqyMNeX4yCS6QVd9uz/pp5+S9dDYMc3Yk MZkQSU1TLGN0gFoKu2P9m+xUPiCAKpQFiAiTblK6pPKQlOzNcIVQpWj5V+oa05RDTPG TbO2b79Gju9Fx+BNtwYlMypGVKAir1/GWNjEu8qk= Received: by smtp.zohomail.com with SMTPS id 1789120685060799.4295343833461; Fri, 11 Sep 2026 02:58:05 -0700 (PDT) From: Shardul Bankar Date: Fri, 11 Sep 2026 15:27:47 +0530 Subject: [PATCH mptcp-next v4 4/4] mptcp: sched: add penalise counters and tracepoint Precedence: bulk X-Mailing-List: mptcp@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260911-mptcp_penalise_send_v2-v4-4-820f2be4a8fe@mpiricsoftware.com> References: <20260911-mptcp_penalise_send_v2-v4-0-820f2be4a8fe@mpiricsoftware.com> In-Reply-To: <20260911-mptcp_penalise_send_v2-v4-0-820f2be4a8fe@mpiricsoftware.com> To: mptcp@lists.linux.dev Cc: "Matthieu Baerts (NGI0)" , Janak Patel , Kalpan Jani , Shardul Bankar , Shardul Bankar X-Mailer: b4 0.15.2 X-ZohoMailClient: External Add observability for the cwnd-halving penalty introduced in the previous patches. Two MPTcpExt SNMP counters: - CwndPenalised: times a subflow cwnd was actually halved; - PenalCandidate: times the rate trigger picked a slow subflow. The two together show how often the rate trigger fired and how many of those candidates were actually halved. The difference is the combined effect of the remaining conditions, so it does not isolate any one of them. Add a mptcp_subflow_penalise tracepoint. The penalise verdict is decided after the subflow-selection loop, once the fastest path is known, so it is exposed through a dedicated tracepoint rather than the per-candidate mptcp_subflow_get_send emit. It reports the selected subflow's pace, the fastest path's pace, its cwnd and the send burst alongside the verdict, so a slow subflow and its window can be watched from a single line. The verdict is emitted once the send burst is known, so it accounts for the window check. The conditions local to the subflow are evaluated later, under that subflow's own lock, so a set verdict here means the scheduler picked the subflow as a candidate; CwndPenalised remains the count of penalties actually applied. Assisted-by: LLM Co-developed-by: Matthieu Baerts (NGI0) Signed-off-by: Shardul Bankar --- include/trace/events/mptcp.h | 33 +++++++++++++++++++++++++++++++++ net/mptcp/mib.c | 2 ++ net/mptcp/mib.h | 2 ++ net/mptcp/protocol.c | 4 ++++ 4 files changed, 41 insertions(+) diff --git a/include/trace/events/mptcp.h b/include/trace/events/mptcp.h index 22882bd03459..f16e04b81322 100644 --- a/include/trace/events/mptcp.h +++ b/include/trace/events/mptcp.h @@ -68,6 +68,39 @@ TRACE_EVENT(mptcp_subflow_get_send, __entry->backup, __entry->ratio) ); =20 +TRACE_EVENT(mptcp_subflow_penalise, + + TP_PROTO(struct mptcp_subflow_context *subflow, unsigned long max_pace, + u32 burst), + + TP_ARGS(subflow, max_pace, burst), + + TP_STRUCT__entry( + __field(u64, pace) + __field(u64, max_pace) + __field(u32, cwnd) + __field(u32, burst) + __field(bool, penalise) + ), + + TP_fast_assign( + struct sock *ssk =3D mptcp_subflow_tcp_sock(subflow); + + __entry->pace =3D subflow->avg_pacing_rate; + __entry->max_pace =3D max_pace; + if (ssk && sk_fullsock(ssk)) + __entry->cwnd =3D tcp_snd_cwnd(tcp_sk(ssk)); + else + __entry->cwnd =3D 0; + __entry->burst =3D burst; + __entry->penalise =3D subflow->penalise; + ), + + TP_printk("pace=3D%llu max_pace=3D%llu cwnd=3D%u burst=3D%u penalise=3D%d= ", + __entry->pace, __entry->max_pace, + __entry->cwnd, __entry->burst, __entry->penalise) +); + DECLARE_EVENT_CLASS(mptcp_dump_mpext, =20 TP_PROTO(struct mptcp_ext *mpext), diff --git a/net/mptcp/mib.c b/net/mptcp/mib.c index 608cb568897c..b39b5511345d 100644 --- a/net/mptcp/mib.c +++ b/net/mptcp/mib.c @@ -95,6 +95,8 @@ static const struct snmp_mib mptcp_snmp_list[] =3D { SNMP_MIB_ITEM("BacklogDrop", MPTCP_MIB_BACKLOGDROP), SNMP_MIB_ITEM("RcvPruned", MPTCP_MIB_RCVPRUNED), SNMP_MIB_ITEM("OFOPruned", MPTCP_MIB_OFOPRUNED), + SNMP_MIB_ITEM("CwndPenalised", MPTCP_MIB_CWNDPENALISED), + SNMP_MIB_ITEM("PenalCandidate", MPTCP_MIB_PENALCAND), }; =20 /* mptcp_mib_alloc - allocate percpu mib counters diff --git a/net/mptcp/mib.h b/net/mptcp/mib.h index 1ebdb55e9534..a7b5bf684af7 100644 --- a/net/mptcp/mib.h +++ b/net/mptcp/mib.h @@ -98,6 +98,8 @@ enum linux_mptcp_mib_field { MPTCP_MIB_BACKLOGDROP, /* Backlog over memory limit */ MPTCP_MIB_RCVPRUNED, /* Dropped due to memory constraints */ MPTCP_MIB_OFOPRUNED, /* MPTCP-level OoO queue pruned */ + MPTCP_MIB_CWNDPENALISED, /* subflow cwnd halved by the scheduler */ + MPTCP_MIB_PENALCAND, /* scheduler picked a slow (low-rate) subflow */ __MPTCP_MIB_MAX }; =20 diff --git a/net/mptcp/protocol.c b/net/mptcp/protocol.c index 24e1d6bd8577..682710ab262e 100644 --- a/net/mptcp/protocol.c +++ b/net/mptcp/protocol.c @@ -1662,6 +1662,7 @@ static void mptcp_penalise_cwnd(struct sock *ssk) return; subflow->last_penalise =3D tcp_jiffies32; tcp_snd_cwnd_set(tp, max_t(u32, cwnd >> 1, MPTCP_PENALISE_MIN_CWND)); + MPTCP_INC_STATS(sock_net(ssk), MPTCP_MIB_CWNDPENALISED); if (cwnd >=3D tp->snd_ssthresh) tp->snd_ssthresh =3D max_t(u32, tp->snd_ssthresh >> 1, 2); } @@ -1745,6 +1746,8 @@ struct sock *mptcp_subflow_get_send(struct mptcp_sock= *msk) subflow =3D mptcp_subflow_ctx(ssk); penal_cand =3D fastest && ssk !=3D fastest && subflow->avg_pacing_rate < max_pace / MPTCP_PENALISE_RATE_RATIO; + if (penal_cand) + MPTCP_INC_STATS(sock_net(ssk), MPTCP_MIB_PENALCAND); /* We use data_race() because tcp_cwnd_validate() might change * is_cwnd_limited under us */ @@ -1754,6 +1757,7 @@ struct sock *mptcp_subflow_get_send(struct mptcp_sock= *msk) wmem =3D READ_ONCE(ssk->sk_wmem_queued); /* the conditions on this subflow are re-checked on apply, under its lock= */ subflow->penalise =3D burst && fast_limited && mptcp_snd_wnd_test(msk); + trace_mptcp_subflow_penalise(subflow, max_pace, burst); if (!burst) return ssk; =20 --=20 2.34.1