From nobody Thu Sep 18 06:42:56 2025 Delivered-To: wpasupplicant.patchew@gmail.com Received: by 2002:a9f:3042:0:0:0:0:0 with SMTP id i2csp1273956uab; Mon, 6 Jun 2022 07:41:54 -0700 (PDT) X-Google-Smtp-Source: ABdhPJwXA+GRYuBCSgmBdqDU9F9xs++vMArnNlkea498xWe7cvmnfB5wx2f1l23k8YNgBalOpHCT X-Received: by 2002:a05:6830:4188:b0:60b:83f8:e8d0 with SMTP id r8-20020a056830418800b0060b83f8e8d0mr10297360otu.32.1654526514533; Mon, 06 Jun 2022 07:41:54 -0700 (PDT) Return-Path: Received: from sv.mirrors.kernel.org (sv.mirrors.kernel.org. [139.178.88.99]) by mx.google.com with ESMTPS id i13-20020a9d4a8d000000b0060b16fc089bsi15745179otf.20.2022.06.06.07.41.54 for (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Mon, 06 Jun 2022 07:41:54 -0700 (PDT) Received-SPF: pass (google.com: domain of mptcp+bounces-5572-wpasupplicant.patchew=gmail.com@lists.linux.dev designates 139.178.88.99 as permitted sender) client-ip=139.178.88.99; Authentication-Results: mx.google.com; dkim=pass header.i=@suse.com header.s=mimecast20200619 header.b=B+InulMO; arc=fail (body hash mismatch); spf=pass (google.com: domain of mptcp+bounces-5572-wpasupplicant.patchew=gmail.com@lists.linux.dev designates 139.178.88.99 as permitted sender) smtp.mailfrom="mptcp+bounces-5572-wpasupplicant.patchew=gmail.com@lists.linux.dev"; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=suse.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sv.mirrors.kernel.org (Postfix) with ESMTPS id C86FC280A6E for ; Mon, 6 Jun 2022 14:41:53 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id C926028E7; Mon, 6 Jun 2022 14:41:52 +0000 (UTC) X-Original-To: mptcp@lists.linux.dev Received: from de-smtp-delivery-102.mimecast.com (de-smtp-delivery-102.mimecast.com [194.104.111.102]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0882928E6 for ; Mon, 6 Jun 2022 14:41:50 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619; t=1654526509; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=65fdPogXFwX9Lqq8VkGpRFvSWZ4mZlUKxIbEOzaZd9w=; b=B+InulMOctCutcmImYmZlbre6iWPUFIwjf94us6z1t8QtY5t+iJ126M4ppi+pR+nCA/qVD dpMR3qZeVROcY2wPfl498RAyL9y74XrvopeBmBGsEHGxxctzk5LALnPIlbonhuz1+xYgdI yaHmblR8B/epPLLP3yr7a6TmqS9nOEo= Received: from EUR05-AM6-obe.outbound.protection.outlook.com (mail-am6eur05lp2109.outbound.protection.outlook.com [104.47.18.109]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id de-mta-34-kaCWs45aN4SbuhP7pf-_SA-1; Mon, 06 Jun 2022 16:41:45 +0200 X-MC-Unique: kaCWs45aN4SbuhP7pf-_SA-1 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=OJa2K48qJqQuMj1Nm71LtpO8JF6BBiWsfseTPwHzeoEiCI01SofCzlqCjQRFI99iuQ3Iz/w4Wj35ERtVYHll00qdNchiOyVMAo7iDINayaegWyOHqGvqjHFG5NigLXEL7pOqLOa+XpkY8DuX6TFI19tAQCnsWAUuo0yHg8qLymzRTPzg6MR8tnPj/VqKJEBovgu0H/K3j7MyKPHZExE4/K+L5m4/y0x0+GZPPH7i12Ypswa0kFGDM4x2Xy3oAliqBb4h/YlhR+IDpbFPB6Si6HRo1AkkxSUfysjKnlZ6RSAyyvkZOLGQWY6Hd8yxkLOT43M2MbVWXMm/FcHBUn/x6g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=oIfwPhKvoMmPsO9Ij9WERPrC4/WDGmgP5zpnZr6BrBo=; b=OCSvU6GND7AGIR7TmuwUow15JI3jm9PLoQA3fnNPLE/IU+zuwxSWEXHUbCkTmOaOIzS/b8icpO+aIPBZ96mi56z6XKcTr5RQLRItsShxXT9gRSqZwFyZUSfa8i4pX+58q2eWONF7UthRqocJlZL/j8hO9bf/eex7rhmsp7zFJhf7obbFxAe4UnVcZi8bc7vcIZq2faMhrEreLJCR60CpWojfHck3x/TPQKsY5U9wpfaSFIjCXvCVTiDAu/d6Fhg99m/z6/anZqdO4uNsqONdsHHK7SCI3VyR1AOkKeF/sj2U8u5yhaMB2zgAEDt+VHzqAkeqgEfJpY8QePFFZUHs+g== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com; dkim=pass header.d=suse.com; arc=none Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=suse.com; Received: from HE1PR0402MB3497.eurprd04.prod.outlook.com (2603:10a6:7:83::14) by VI1PR0402MB3536.eurprd04.prod.outlook.com (2603:10a6:803:2::12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5314.19; Mon, 6 Jun 2022 14:41:45 +0000 Received: from HE1PR0402MB3497.eurprd04.prod.outlook.com ([fe80::8002:50a5:a57a:d8fe]) by HE1PR0402MB3497.eurprd04.prod.outlook.com ([fe80::8002:50a5:a57a:d8fe%5]) with mapi id 15.20.5314.019; Mon, 6 Jun 2022 14:41:44 +0000 From: Geliang Tang To: mptcp@lists.linux.dev CC: Geliang Tang Subject: [PATCH mptcp-next v5 1/7] Squash to "mptcp: add get_subflow wrappers" Date: Mon, 6 Jun 2022 22:41:16 +0800 Message-ID: X-Mailer: git-send-email 2.34.1 In-Reply-To: References: Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: SG2PR02CA0063.apcprd02.prod.outlook.com (2603:1096:4:54::27) To HE1PR0402MB3497.eurprd04.prod.outlook.com (2603:10a6:7:83::14) Precedence: bulk X-Mailing-List: mptcp@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 45dd8662-09a8-4edf-223e-08da47caad7f X-MS-TrafficTypeDiagnostic: VI1PR0402MB3536:EE_ X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 7qK2DPEAvTHcR7XMwQRSdHs7rIqIWCy4a5sbHp6cfXGlg2dyBbi+8pXe+abo6m2uZ7z0lutso72dvWInjtKPKSBqMjs5VxRA240nt/X9O5QzR8zvsH+UfXpVZt5p1SWWcoiSzFQyag3XbSBym2wghbIfOMu+lbBTYozg1+4NnQQOexyoKJ7xl79lAjqTdg63pjB3bAXxJnRm53UGmWi73Ei115yIHNwQztDJHIK02ziAfdo2Tes58Xt9XXRMaBY/9o0DUrUK7TnZUolW9xLkU7vQVb5JL6DCGkbt/9pRX53SSXRee7XvXdBA9AwVFM/YZbyMDMP+NvCG5BNFKQQ2/ToqyzzHS1o/4hAP3QzEPfcWdjF4BX0LYIrE4VnQBfEPmLp7oWhFmpf4n5uBjLUvxLU99q9mFjjtH1LgrkVfGwHXoxqLy2aZ4QbxMJn8+YOnTiD3ED+VynGeMn1rXliXa068Ns9lhxfwbSpMhjlYBubMqZgMdo48Bt/YK8GS8LXCrQ5SLyl7hcKOh50N6ZOkXhys5ySWGvpP35OnGcp0o7efuyt82uaH9+TPZBD5xMRX1kmjHXKXwo4OHahytORI6sCfvit5VOdMBYOaQqrUqG5OEoYJ55AqMtdLkWxGh+SSOmNSGA5rP4Xd5h1KhFkyFq9gy5XzWDvud3sTThdaDdzBBxFYmkepNwQ4h/9fAj5x X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:HE1PR0402MB3497.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230001)(366004)(83380400001)(107886003)(36756003)(66946007)(186003)(2616005)(6486002)(6512007)(26005)(6666004)(6506007)(38100700002)(66556008)(66476007)(8936002)(44832011)(316002)(86362001)(4326008)(508600001)(8676002)(5660300002)(6916009)(2906002)(13296009);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?MsK2ntqCVybHRe6fwI2CUOUY0aM8p2DLQ3SLATh1zWqM0pANDfN5RvcSPg96?= =?us-ascii?Q?H6wxfBWYi45f7Lo+XPROxhu2yRpgqgtRwDSMQh4M5lwvnWtr8ZNab1+KhMXc?= =?us-ascii?Q?U4Q2P/r3Ru8msueIocspex8Ah4FJpSAMPJIcZ6iSVlN/rf4p7N0yAX1G8MEH?= =?us-ascii?Q?TJIzCTzQvVb8ieVD5+xLq5u4MaNdMLkB+ux3Tiy/uI1F2uSbFryY8hV0OCZR?= =?us-ascii?Q?I1xFv8ktcl72CyZ5J+I/RUFsz3Y1j1eYy+p7ZAgNzHWf4rob0u0s5hVivXy4?= =?us-ascii?Q?H4DSfFYrgSj88WnRMH9ssIc9ta2I+z7JKG0rFMQRIE8h9lqa4XlBw5lPejRj?= =?us-ascii?Q?HqWGrCkZwunrv0itMzZ4+la/MEXlIABtths/QqHl//ahcr5njQzal6eLTpnX?= =?us-ascii?Q?vIAYUuMnUFJVHWVvw61Ckyg+bnKilwwTLyh5kK1VvdxOXRwkYM+Zhib1HOCS?= =?us-ascii?Q?qdCxI255hglqTAVBe+9PybM5sHGX2fF50g0vUkG5a6T+U0lmZP4SNR+DeXv1?= =?us-ascii?Q?glyKNfOkXiGFr+XJhBJ3LvaZ9xmsJXrMlbUUk2TYBZdacFXH6g6j942Mj+9T?= =?us-ascii?Q?R9dn9OpbP6K7o9/74HkGU0ZtltPzHlj7oPZmIDAZa6MbHgQjP3W6XS3FEdcc?= =?us-ascii?Q?UT0k+MiznALE16xkYqNyjVQr2IoWtCHBBM+9WnmU0+8j2dOsnEbr+ehM+AMB?= =?us-ascii?Q?y2MNgzq6TMgVF1QoK/v/3X5Tm/zcS26pAfv+Ht/oRmu6x9mhjrTyulX+ckCA?= =?us-ascii?Q?B5CTd1iTsmnN3G1mEMt4US9Rd8p+WVQziulAiisezU4fm+RnyMXOenI8EeJo?= =?us-ascii?Q?d4ZF9tu720RZJZMAUfxv9AWUwzOSCqVqXCdqnj+7O6IUGUlvo0a+sdUAkB49?= =?us-ascii?Q?hcWarB/3g8tkvXv1BE0+jlY+T0GkyV1pkRL35z4u7UTsfXf3KuwtRX5FdAIv?= =?us-ascii?Q?6miuYDCZ/Cq/IEF4eowUTkitmbLqfDFW6oPMUYEopHCsuqjDA1LiGFCIMB0n?= =?us-ascii?Q?22Wwaf3sxK1g66sAOc2gBFXu7qGAnuR3g9+lXk9g7Ule14rIdG59FyG7or56?= =?us-ascii?Q?ijcp22/JD8fwRZAzRYBVXAArM9SNZgMfmLi4bcVQ4Q6hSChhzWHiDiBRsYxj?= =?us-ascii?Q?WofRirqe1eBVAKYdnKtPiD+rY+J75sDMqg61fov5S2YtCgnQpliVm30VrAaK?= =?us-ascii?Q?IT7WcXnKCYqG8sb6Lm8A+ymL2znn06L53cTcSZV3wjB0L72FErgvMqT2XvNb?= =?us-ascii?Q?IboWl941V/VNyBV8fx+3uaLqgWG/3Oe8vVv7qsXp63LQRbjEz0b84IK7/9O5?= =?us-ascii?Q?50QP2FGc6jap2oVEN2DyjbBLwrC3LqGePkDmelHIvpPR7OJ8qcxruDQcKhqe?= =?us-ascii?Q?sYXnMsPHE5OV8LoOv+Uk9wFrVvM8IcHTfyqoKixPNfDBZiRXeMZSWLBTRn+/?= =?us-ascii?Q?6D08SRdkLE7OY7F6opM+yrFH0Sa7OmKx6eQI3O4kpbc12NJn6wBc27T/OlBE?= =?us-ascii?Q?Na86jCCkYYdP5pE6d98se8R1umNEM7AFccnbG47p+a3GEcjbY/QrsAgeM52g?= =?us-ascii?Q?x4hKLpUDTvugMi51eOOhAkoWj2ml5XLeptKjqeR6NOX5hhdb/p1JNAJDuaNC?= =?us-ascii?Q?IPqS8FMii4Ix6dDAirie13rz6P/f0sLkLegqruvAAvVGgYNEvNiXxbflZS1Z?= =?us-ascii?Q?g8rX4Hmv/cbhxBxl5Wjz1j8/BFD9a+pefecTMC+SI+brUE+kSuGkEdYMNgDg?= =?us-ascii?Q?x/gHgRHs5lK6Kj83JAA7ayBPuGf/Y78=3D?= X-OriginatorOrg: suse.com X-MS-Exchange-CrossTenant-Network-Message-Id: 45dd8662-09a8-4edf-223e-08da47caad7f X-MS-Exchange-CrossTenant-AuthSource: HE1PR0402MB3497.eurprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 06 Jun 2022 14:41:44.8446 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: cZkG5Jeb4+BExGHSa23pvo23xYiAgXSt5rRDZa1qKJlbXP8mkKlCMmC3DQbFe1Rn4Ml78Bzpx4QW2AUOznpSOA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0402MB3536 Content-Type: text/plain; charset="utf-8" ''' mptcp: add sched_get_send wrapper This patch defines a wrapper named mptcp_sched_get_send(), invoke mptcp_subflow_get_send() or get_subflow() of msk->sched in it. Set the subflow pointers array in struct mptcp_sched_data before invoking get_subflow(), then it can be used in get_subflow() in the BPF contexts. Move sock_owned_by_me() and the fallback check code from mptcp_subflow_get_send() into this wrapper. ''' Signed-off-by: Geliang Tang --- net/mptcp/protocol.c | 15 +++++++---- net/mptcp/protocol.h | 4 +-- net/mptcp/sched.c | 60 ++++++++++++-------------------------------- 3 files changed, 27 insertions(+), 52 deletions(-) diff --git a/net/mptcp/protocol.c b/net/mptcp/protocol.c index d6aef4b13b8a..8d93df73a9e3 100644 --- a/net/mptcp/protocol.c +++ b/net/mptcp/protocol.c @@ -1567,7 +1567,7 @@ void __mptcp_push_pending(struct sock *sk, unsigned i= nt flags) int ret =3D 0; =20 prev_ssk =3D ssk; - ssk =3D mptcp_sched_get_send(msk); + ssk =3D mptcp_subflow_get_send(msk); =20 /* First check. If the ssk has changed since * the last round, release prev_ssk @@ -1634,7 +1634,7 @@ static void __mptcp_subflow_push_pending(struct sock = *sk, struct sock *ssk) * check for a different subflow usage only after * spooling the first chunk of data */ - xmit_ssk =3D first ? ssk : mptcp_sched_get_send(mptcp_sk(sk)); + xmit_ssk =3D first ? ssk : mptcp_subflow_get_send(mptcp_sk(sk)); if (!xmit_ssk) goto out; if (xmit_ssk !=3D ssk) { @@ -2195,12 +2195,17 @@ static void mptcp_timeout_timer(struct timer_list *= t) * * A backup subflow is returned only if that is the only kind available. */ -struct sock *mptcp_subflow_get_retrans(struct mptcp_sock *msk) +static struct sock *mptcp_subflow_get_retrans(struct mptcp_sock *msk) { struct sock *backup =3D NULL, *pick =3D NULL; struct mptcp_subflow_context *subflow; int min_stale_count =3D INT_MAX; =20 + sock_owned_by_me((const struct sock *)msk); + + if (__mptcp_check_fallback(msk)) + return NULL; + mptcp_for_each_subflow(msk, subflow) { struct sock *ssk =3D mptcp_subflow_tcp_sock(subflow); =20 @@ -2453,7 +2458,7 @@ static void __mptcp_retrans(struct sock *sk) mptcp_clean_una_wakeup(sk); =20 /* first check ssk: need to kick "stale" logic */ - ssk =3D mptcp_sched_get_retrans(msk); + ssk =3D mptcp_subflow_get_retrans(msk); dfrag =3D mptcp_rtx_head(sk); if (!dfrag) { if (mptcp_data_fin_enabled(msk)) { @@ -3107,7 +3112,7 @@ void __mptcp_check_push(struct sock *sk, struct sock = *ssk) return; =20 if (!sock_owned_by_user(sk)) { - struct sock *xmit_ssk =3D mptcp_sched_get_send(mptcp_sk(sk)); + struct sock *xmit_ssk =3D mptcp_subflow_get_send(mptcp_sk(sk)); =20 if (xmit_ssk =3D=3D ssk) __mptcp_subflow_push_pending(sk, ssk); diff --git a/net/mptcp/protocol.h b/net/mptcp/protocol.h index d406b5afbee4..0fb2970a7a2d 100644 --- a/net/mptcp/protocol.h +++ b/net/mptcp/protocol.h @@ -632,9 +632,7 @@ void mptcp_release_sched(struct mptcp_sock *msk); void mptcp_subflow_set_scheduled(struct mptcp_subflow_context *subflow, bool scheduled); struct sock *mptcp_subflow_get_send(struct mptcp_sock *msk); -struct sock *mptcp_subflow_get_retrans(struct mptcp_sock *msk); -struct sock *mptcp_sched_get_send(struct mptcp_sock *msk); -struct sock *mptcp_sched_get_retrans(struct mptcp_sock *msk); +struct sock *mptcp_sched_get_send(struct mptcp_sock *msk, int *err); =20 static inline bool __mptcp_subflow_active(struct mptcp_subflow_context *su= bflow) { diff --git a/net/mptcp/sched.c b/net/mptcp/sched.c index 8858e1fc8b74..a0585182a2b1 100644 --- a/net/mptcp/sched.c +++ b/net/mptcp/sched.c @@ -117,63 +117,35 @@ static int mptcp_sched_data_init(struct mptcp_sock *m= sk, bool reinject, return 0; } =20 -struct sock *mptcp_sched_get_send(struct mptcp_sock *msk) +struct sock *mptcp_sched_get_send(struct mptcp_sock *msk, int *err) { struct mptcp_sched_data data; struct sock *ssk =3D NULL; - int i; =20 + *err =3D -EINVAL; sock_owned_by_me((struct sock *)msk); =20 /* the following check is moved out of mptcp_subflow_get_send */ if (__mptcp_check_fallback(msk)) { - if (!msk->first) - return NULL; - return sk_stream_memory_free(msk->first) ? msk->first : NULL; - } - - if (!msk->sched) - return mptcp_subflow_get_send(msk); - - mptcp_sched_data_init(msk, false, &data); - msk->sched->get_subflow(msk, &data); - - for (i =3D 0; i < MPTCP_SUBFLOWS_MAX; i++) { - if (data.contexts[i] && READ_ONCE(data.contexts[i]->scheduled)) { - ssk =3D data.contexts[i]->tcp_sock; - msk->last_snd =3D ssk; - break; + if (msk->first && sk_stream_memory_free(msk->first)) { + mptcp_subflow_set_scheduled(mptcp_subflow_ctx(msk->first), true); + *err =3D 0; + return msk->first; } - } - - return ssk; -} - -struct sock *mptcp_sched_get_retrans(struct mptcp_sock *msk) -{ - struct mptcp_sched_data data; - struct sock *ssk =3D NULL; - int i; - - sock_owned_by_me((const struct sock *)msk); - - /* the following check is moved out of mptcp_subflow_get_retrans */ - if (__mptcp_check_fallback(msk)) return NULL; + } =20 - if (!msk->sched) - return mptcp_subflow_get_retrans(msk); + if (!msk->sched) { + ssk =3D mptcp_subflow_get_send(msk); + if (!ssk) + return NULL; + mptcp_subflow_set_scheduled(mptcp_subflow_ctx(ssk), true); + *err =3D 0; + return ssk; + } =20 - mptcp_sched_data_init(msk, true, &data); + mptcp_sched_data_init(msk, false, &data); msk->sched->get_subflow(msk, &data); =20 - for (i =3D 0; i < MPTCP_SUBFLOWS_MAX; i++) { - if (data.contexts[i] && READ_ONCE(data.contexts[i]->scheduled)) { - ssk =3D data.contexts[i]->tcp_sock; - msk->last_snd =3D ssk; - break; - } - } - return ssk; } --=20 2.34.1 From nobody Thu Sep 18 06:42:56 2025 Delivered-To: wpasupplicant.patchew@gmail.com Received: by 2002:a9f:3042:0:0:0:0:0 with SMTP id i2csp1273972uab; Mon, 6 Jun 2022 07:41:58 -0700 (PDT) X-Google-Smtp-Source: ABdhPJx6q7ZJbnjO1pCA8F7uSLyBSrEI2/jgAtI8txBGlMV6BwCcVqAze/z7mE5cbP6lmwRgsSbj X-Received: by 2002:a05:6870:15c3:b0:ed:9980:db99 with SMTP id k3-20020a05687015c300b000ed9980db99mr31805798oad.154.1654526517994; Mon, 06 Jun 2022 07:41:57 -0700 (PDT) Return-Path: Received: from da.mirrors.kernel.org (da.mirrors.kernel.org. [139.178.84.19]) by mx.google.com with ESMTPS id q12-20020a9d57cc000000b0060bfd5c8419si2756760oti.192.2022.06.06.07.41.57 for (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Mon, 06 Jun 2022 07:41:57 -0700 (PDT) Received-SPF: pass (google.com: domain of mptcp+bounces-5573-wpasupplicant.patchew=gmail.com@lists.linux.dev designates 139.178.84.19 as permitted sender) client-ip=139.178.84.19; Authentication-Results: mx.google.com; dkim=pass header.i=@suse.com header.s=mimecast20200619 header.b="hA5/f4Vp"; arc=fail (body hash mismatch); spf=pass (google.com: domain of mptcp+bounces-5573-wpasupplicant.patchew=gmail.com@lists.linux.dev designates 139.178.84.19 as permitted sender) smtp.mailfrom="mptcp+bounces-5573-wpasupplicant.patchew=gmail.com@lists.linux.dev"; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=suse.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by da.mirrors.kernel.org (Postfix) with ESMTPS id CFEFA2E09FB for ; Mon, 6 Jun 2022 14:41:57 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 87FFF28E7; Mon, 6 Jun 2022 14:41:56 +0000 (UTC) X-Original-To: mptcp@lists.linux.dev Received: from de-smtp-delivery-102.mimecast.com (de-smtp-delivery-102.mimecast.com [194.104.109.102]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B9D9028E6 for ; Mon, 6 Jun 2022 14:41:54 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619; t=1654526512; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=SAS/nxTFHmcQQYO6aqQXh2hejItJPR3JTvQUD96qles=; b=hA5/f4Vp3zpoL4+fmjRc1BGtlNfIAp2sfHvUU6CaUbakepVLhV5FP42QbV41+5q2XpwzQ4 9hrlYnhhzugPCGAVZVJ/dgw5I5+uo0xfz7JFsMpWPT2tzJK4P/GM4OUo9UcdCZ4SqMzGvE ZZk7Bk1C8oltrclwBPHgKzsLkEQJXtU= Received: from EUR04-VI1-obe.outbound.protection.outlook.com (mail-vi1eur04lp2053.outbound.protection.outlook.com [104.47.14.53]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id de-mta-41-30_tSvwTMI6Jqp9OzC1xUw-1; Mon, 06 Jun 2022 16:41:51 +0200 X-MC-Unique: 30_tSvwTMI6Jqp9OzC1xUw-1 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Z4gKEylBCptu0sbAMa52Qv4Mmf8cfV1pSICz79AK9ySK68KYgsAbjvMyzRo9psXcFEEY+rcYHjKcs3xHFq3NhW8BB+s6ykGTOTld+PPz7B3Euq3wqtyUARmP81+b//Q8p4pQmuNczxoM6ejojsRPSLh48inlUKi0s7kDMHb0diAKK+zndMnaYX8LUI9pHuG1CaqhX1lAhN5NQaQalzWnq8QRCKVMjCoLn8uMTQU4WEUXIWcW+4UCNVTwpvsJh+RDCDLhvIP2ImT9PF1XaAY7UsPn+Lw8zsFGZ/vvgWEthsQ0amBLRHQtA9gqW2Q968uhRjx8rl9y3Z0KD9wxbmEpHQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=R4IjezjvT81PTWumNUj6jkR/8XNh6dzZCSnosWWuLMU=; b=nlTvk99iEZ8ePe3utcu9rISo9fblXAKbnjjNGo1QmJP6snXQhcqNDOUrWEqhpyejSVRZg4Qx1g5KoeBgiOlltvA6r8wCknUlhEfrb30/yq07DQ0SG3r0GkzWMZ1B6Gi+aCccricN/7VVla9gi4SZ0zKnlyH9E5ukQF25a73+JFTuV80SFxbYeKpeKZ5Vt63KRiEJGhQnb16FBaQ0NJ4uJ1owu1gy7zdfHBPAEoTGG9cCE0eYeYOzD7iIXs4K0eMHZ1sQWkYNPG20kC2dsyS4MawolvbVL/KMBcdxkFUGXoa9ZWbwXKPp06nyp5YPRy9WuuOZPuZODlwzP4SZUDOwHw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com; dkim=pass header.d=suse.com; arc=none Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=suse.com; Received: from HE1PR0402MB3497.eurprd04.prod.outlook.com (2603:10a6:7:83::14) by VI1PR0402MB3536.eurprd04.prod.outlook.com (2603:10a6:803:2::12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5314.19; Mon, 6 Jun 2022 14:41:51 +0000 Received: from HE1PR0402MB3497.eurprd04.prod.outlook.com ([fe80::8002:50a5:a57a:d8fe]) by HE1PR0402MB3497.eurprd04.prod.outlook.com ([fe80::8002:50a5:a57a:d8fe%5]) with mapi id 15.20.5314.019; Mon, 6 Jun 2022 14:41:50 +0000 From: Geliang Tang To: mptcp@lists.linux.dev CC: Geliang Tang Subject: [PATCH mptcp-next v5 2/7] mptcp: redundant subflows push pending Date: Mon, 6 Jun 2022 22:41:17 +0800 Message-ID: X-Mailer: git-send-email 2.34.1 In-Reply-To: References: Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: SG2PR02CA0073.apcprd02.prod.outlook.com (2603:1096:4:90::13) To HE1PR0402MB3497.eurprd04.prod.outlook.com (2603:10a6:7:83::14) Precedence: bulk X-Mailing-List: mptcp@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 1b9c2144-3cb8-4824-8212-08da47cab11e X-MS-TrafficTypeDiagnostic: VI1PR0402MB3536:EE_ X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 2RboeZN1O3VyaisppjgYLcJttlOYVWjN5E+SieIeuag9Hnc8mOZQ51q4BfigaGrH32glWQZjTILyTOdFdNRJp0VR0tF9/Vpui5XKmPra+cVMt50gFsnA++DSGM7gQOLtVAI5qo5arcT9MKkH/1VDGg6Vk8gu/bXflbo50XJXM7DLSDJ/eFMrw/S4wsvYi8xqWlk7XMe0sw0xMELjTowT9RwcKrRIjMx/pB8zoOebyXIPxIp/BlSv5qAWvpKHYKProrD1v1WKJQlwNqGqEMAdlZ0rOkCb5LyZT0/pl6+i1W02um92pAllIEqthR868qqStPzFOo7S8PdpRCjXyK7qrcqQGjm94UZiPJb5Lbz+7GHD6pQPEkT0xXNFZ19j9ECIuZyE6SXJUWROu/YVZy35PJolEZBuyLxZZOuoW2LEXFVhgK6zdQiz3gIdSeV2TniuLxKJyAHB9lItGKR0swzJUqRcEwD84xesLB1weF/dqEEs/0NNjv/+LrOKlM3JZh2g1W7K21FKuwTVWq4C3HFKhN4fLVPjl1WT2CpFHqU+f4azvzZJCRImNSwkB2Gag8QeOIPsFnLO+70hqMrIyvsrDz1mXadgsIoutwrHJNopYWy5fNfZIIrn1zehpQq4pnlb943pNLI8oixE2z8WDWMoRcL+F1FNqbUhFGhReB+71qmPBkQGQ6qU2E0Hv9B17uZvei+ZdRarqUx/AZRn406uRw== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:HE1PR0402MB3497.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230001)(366004)(83380400001)(107886003)(36756003)(66946007)(186003)(2616005)(6486002)(6512007)(26005)(6666004)(6506007)(38100700002)(66556008)(66476007)(8936002)(44832011)(316002)(86362001)(4326008)(508600001)(8676002)(5660300002)(6916009)(2906002)(13296009);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?6m68HdV1lDpA+Z7dpUXjpJtmMJu5/+bh331UKtbbWe38Vo7P5Hl//FBUiQK0?= =?us-ascii?Q?DQBzrRWDwR8dV4X+LqXqpUoIhGLXKD/aTHv0HEr5T0CfrMIt5vYWpIZMId+P?= =?us-ascii?Q?ELWYlAPijRLH1g2UCqOSAuA6x4GwCinJ3ZbhUwJ89IR2EZ7EAcXNqXy6caJT?= =?us-ascii?Q?igtj3TmejIvhcKsFM4gG7ufHYfGyHGu8VlWFN0wxIYnnuPVzLq2KORj2MPl4?= =?us-ascii?Q?UGdchRloxqT/0FjgEmOJzTXiyp/YOL2rxM0Tq8DCpsgVjmrcXXUVI5EDp51B?= =?us-ascii?Q?SMaI9A65U1lkIMb2v5P8nweRIwXAc1CYKysbOIk9gckpABR1MNZpYSZF563i?= =?us-ascii?Q?PUYYomUwxSzkEnn3HpqSjp7SittC5wEEevbIVTTkVovjpAZVMJ8PXKbum7+i?= =?us-ascii?Q?KEz+k6FGdWEqXxRfgRBW8NXgQ2zpJhChPC6rrm7Q+hA5346sm0KVNTRaDd7L?= =?us-ascii?Q?b4WAOOWSEJpZor6yoEsTJTz5+Sh2+sU3dTo53DI7kz08rJc3nX/EnlRD3P95?= =?us-ascii?Q?C3zzWK5UcfOK3MHkGwGokSmrZoK/xA59vgzYBITrlcevHMHTiWaHd3bNfXOg?= =?us-ascii?Q?TYZQd3JWATRgbGHhTHb9zf2KbN2nLdgUhY6zEVduFIdTgLd/2k+bwVw3zTmf?= =?us-ascii?Q?/G0XEQp5S5m+7S20io0O2dyzh56uI45wxUQH8eVmg8H3c1rOQEKidOUaEk+k?= =?us-ascii?Q?AbjBOdxUk0V5l0BnuVI5H2PWqcRR4YVDg07nvj67K+kMCpQb6G2n2JSl7u/l?= =?us-ascii?Q?38Dn55Hb4EojRaga3qy/O+D+/w77mWBTTViygsR3lyqt8IpA5T7TAJPf8oZH?= =?us-ascii?Q?g5GeuvVHFg1mRNVk5tuNt2xuTdCWnqbzMWjdGRZAPA61jNF7yq4pFA2MPYAi?= =?us-ascii?Q?l4Z24GjKftdIysAinm3ts7xIdrj0Remv4DGdqAKimW21U3gT0pQo8jsWUyux?= =?us-ascii?Q?lsfV6IWfIlseKbEjCrpuML+mOYqX2eOksu7dyDXwLUXGQkx5wx5PUtEyjjQf?= =?us-ascii?Q?Lp6Zsxkf6r5BbH1uTL7knhvz6wE2f06n99hA3mTD1FF6AAt2ez4bbARAfJmw?= =?us-ascii?Q?D5OVBrOLFsW64kkeGcQxllqB5d+yEc1XLRjFksmix/bnayULU4WtwypiAPyb?= =?us-ascii?Q?Tz7lWAE+l2UtRz3PbEMW1FCzNo+PKaS0/mw2dl5gwcLor4qtZXmGtftRbHwh?= =?us-ascii?Q?WRGiQLLgUF5KWtj2J75lgo319C4OR6k7ZB4qi/XZ6zXXNKRd5b1/gTTK9pZV?= =?us-ascii?Q?y/0fg+WUoEkm74tT8oB7ule8Gw35cp8fQwNMF7tC5vy1BSW/HOdwKEa1ecw0?= =?us-ascii?Q?vak4ckFaLQTR1cggQvQ2n4xeRNLaPiexxXz2mBW5BKbdSOuBv2N1uEZEPH3x?= =?us-ascii?Q?qE9QZ9pxgQpcs7V0OWGMdiAJEJNRRp8rmQ4TTbM87YBNBuQ7Ekrqmwla/uZ3?= =?us-ascii?Q?iW9TiQxGPoKXKnk/mz2hBT+XD9j3L7/otNmqjlBYLwqFyo3cvvNrnoZCKIUC?= =?us-ascii?Q?6pESOna3FHpmJTpXkNm6zploZTXCsVT3kXP+nhqsfuVATS/qU8HXH5cz6TTk?= =?us-ascii?Q?BSAdIB9iqMQWudAL4WgDYW+XHK/BZXNcFRVe8fcymG9Fn4SL1w7ghEfxzWoy?= =?us-ascii?Q?9P9wPlo8UdrRqzHhcTB6fcbSOMsWZQ3N4anMrabO/hc8o5v1QokF2KnbFETw?= =?us-ascii?Q?P4bcml7t70GFJ8+/HRxyxL/mVELGUm5oizUF/ucwzVLgfizRBOGFq8NmMfB3?= =?us-ascii?Q?kHxI6fEOt8uPjRgRjoQYyno05SRhA00=3D?= X-OriginatorOrg: suse.com X-MS-Exchange-CrossTenant-Network-Message-Id: 1b9c2144-3cb8-4824-8212-08da47cab11e X-MS-Exchange-CrossTenant-AuthSource: HE1PR0402MB3497.eurprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 06 Jun 2022 14:41:50.8441 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 11SC4me9XqUKqpV8Xm1OTDqWc84pd9/AkgBWLpl5NTssjc+vTeM8bykQqXJ/9plifW8dh3UUZN74S6wts1vngg== X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0402MB3536 Content-Type: text/plain; charset="utf-8" This patch adds the redundant subflows support for __mptcp_push_pending(). Use mptcp_sched_get_send() wrapper instead of mptcp_subflow_get_send() in it. Check the subflow scheduled flags to test which subflow or subflows are picked by the scheduler, use them to send data. Signed-off-by: Geliang Tang --- net/mptcp/protocol.c | 63 ++++++++++++++++++++++---------------------- net/mptcp/subflow.c | 1 - 2 files changed, 32 insertions(+), 32 deletions(-) diff --git a/net/mptcp/protocol.c b/net/mptcp/protocol.c index 8d93df73a9e3..8bf48387f9d8 100644 --- a/net/mptcp/protocol.c +++ b/net/mptcp/protocol.c @@ -1551,58 +1551,59 @@ void mptcp_check_and_set_pending(struct sock *sk) =20 void __mptcp_push_pending(struct sock *sk, unsigned int flags) { - struct sock *prev_ssk =3D NULL, *ssk =3D NULL; struct mptcp_sock *msk =3D mptcp_sk(sk); + struct mptcp_subflow_context *subflow; struct mptcp_sendmsg_info info =3D { - .flags =3D flags, + .flags =3D flags, }; struct mptcp_data_frag *dfrag; - int len, copied =3D 0; + int len, copied =3D 0, err =3D 0; + struct sock *ssk =3D NULL; =20 while ((dfrag =3D mptcp_send_head(sk))) { info.sent =3D dfrag->already_sent; info.limit =3D dfrag->data_len; len =3D dfrag->data_len - dfrag->already_sent; while (len > 0) { - int ret =3D 0; + int ret =3D 0, max =3D 0; =20 - prev_ssk =3D ssk; - ssk =3D mptcp_subflow_get_send(msk); - - /* First check. If the ssk has changed since - * the last round, release prev_ssk - */ - if (ssk !=3D prev_ssk && prev_ssk) - mptcp_push_release(prev_ssk, &info); - if (!ssk) + mptcp_sched_get_send(msk, &err); + if (err) goto out; =20 - /* Need to lock the new subflow only if different - * from the previous one, otherwise we are still - * helding the relevant lock - */ - if (ssk !=3D prev_ssk) - lock_sock(ssk); + mptcp_for_each_subflow(msk, subflow) { + if (READ_ONCE(subflow->scheduled)) { + ssk =3D mptcp_subflow_tcp_sock(subflow); + if (!ssk) + goto out; =20 - ret =3D mptcp_sendmsg_frag(sk, ssk, dfrag, &info); - if (ret <=3D 0) { - mptcp_push_release(ssk, &info); - goto out; + lock_sock(ssk); + + ret =3D mptcp_sendmsg_frag(sk, ssk, dfrag, &info); + if (ret <=3D 0) { + mptcp_push_release(ssk, &info); + goto out; + } + + if (ret > max) + max =3D ret; + + mptcp_push_release(ssk, &info); + + msk->last_snd =3D ssk; + mptcp_subflow_set_scheduled(subflow, false); + } } =20 - info.sent +=3D ret; - copied +=3D ret; - len -=3D ret; + info.sent +=3D max; + copied +=3D max; + len -=3D max; =20 - mptcp_update_post_push(msk, dfrag, ret); + mptcp_update_post_push(msk, dfrag, max); } WRITE_ONCE(msk->first_pending, mptcp_send_next(sk)); } =20 - /* at this point we held the socket lock for the last subflow we used */ - if (ssk) - mptcp_push_release(ssk, &info); - out: /* ensure the rtx timer is running */ if (!mptcp_timer_pending(sk)) diff --git a/net/mptcp/subflow.c b/net/mptcp/subflow.c index 8841e8cd9ad8..e7864a413192 100644 --- a/net/mptcp/subflow.c +++ b/net/mptcp/subflow.c @@ -881,7 +881,6 @@ static bool validate_mapping(struct sock *ssk, struct s= k_buff *skb) subflow->map_data_len))) { /* Mapping does covers past subflow data, invalid */ dbg_bad_map(subflow, ssn); - return false; } return true; } --=20 2.34.1 From nobody Thu Sep 18 06:42:56 2025 Delivered-To: wpasupplicant.patchew@gmail.com Received: by 2002:a9f:3042:0:0:0:0:0 with SMTP id i2csp1274002uab; Mon, 6 Jun 2022 07:42:03 -0700 (PDT) X-Google-Smtp-Source: ABdhPJygmAWRDPgHYwt+DDuDRhrHReytkNAGpmLgfbggebyEyfTp6qH6mIGOS66krxlMGN+PoTyR X-Received: by 2002:a05:6808:168b:b0:2f7:338b:7a55 with SMTP id bb11-20020a056808168b00b002f7338b7a55mr30537309oib.133.1654526523576; Mon, 06 Jun 2022 07:42:03 -0700 (PDT) Return-Path: Received: from sv.mirrors.kernel.org (sv.mirrors.kernel.org. [139.178.88.99]) by mx.google.com with ESMTPS id g78-20020a9d12d4000000b0060bfeee3228si2216670otg.40.2022.06.06.07.42.03 for (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Mon, 06 Jun 2022 07:42:03 -0700 (PDT) Received-SPF: pass (google.com: domain of mptcp+bounces-5574-wpasupplicant.patchew=gmail.com@lists.linux.dev designates 139.178.88.99 as permitted sender) client-ip=139.178.88.99; Authentication-Results: mx.google.com; dkim=pass header.i=@suse.com header.s=mimecast20200619 header.b=StImm1xt; arc=fail (body hash mismatch); spf=pass (google.com: domain of mptcp+bounces-5574-wpasupplicant.patchew=gmail.com@lists.linux.dev designates 139.178.88.99 as permitted sender) smtp.mailfrom="mptcp+bounces-5574-wpasupplicant.patchew=gmail.com@lists.linux.dev"; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=suse.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sv.mirrors.kernel.org (Postfix) with ESMTPS id 21869280A8F for ; Mon, 6 Jun 2022 14:42:03 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 397D128E7; Mon, 6 Jun 2022 14:42:02 +0000 (UTC) X-Original-To: mptcp@lists.linux.dev Received: from de-smtp-delivery-102.mimecast.com (de-smtp-delivery-102.mimecast.com [194.104.109.102]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 90B2428E6 for ; Mon, 6 Jun 2022 14:42:00 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619; t=1654526518; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=pKuy+jGUIVghnYatYVovAm/x3/tZ8YbQItIPy4YeB6g=; b=StImm1xtBV9VM2tnGK0M7I1g9T5pU+r3bHIYlJ+hb4u02REn2+RuYjhKR3kXGZkkPX0tOM DfjOmhcagAwMrZxV4+33sb2LYzDzXHjkxTKCRqMaVyn3/iXAfO6tM6QGzjIchZalorvd3W G90r7QtuqZ2LwDRNywVSpEi6fO/gpcU= Received: from EUR04-VI1-obe.outbound.protection.outlook.com (mail-vi1eur04lp2052.outbound.protection.outlook.com [104.47.14.52]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id de-mta-9-0WzzQ9kmNzmT7wicpKZ1xQ-1; Mon, 06 Jun 2022 16:41:57 +0200 X-MC-Unique: 0WzzQ9kmNzmT7wicpKZ1xQ-1 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=SnP0C+8RiPwV/If68AGUI9OsE+vtJkI2FQMM3OiBpi09Zv6UCFfd3xnzfOhCQ5UqlSr4trzAv7d+vN04diXC6gFHCAmqz+lL+xN0z8V616q+lyYYrtnCbrT9r9fvYvx9aGKK5wIFrWd+9s8XE+XC2jFCurLMaywD3JfVyfzQ5qR/RJG3cSnjr4xI95MYvHjGXKaXhQq9cBmZbLt13bhejATvUYgUaoo/XJOpldwLOVjKWVH1zYLV/YzwCZYHVHIWJGikHtY12OjNML8FTLMZ4DsTZ3Q318YAE/6DDPZsLuTpTuSyBDUUOIV44gBMgaa8gjE3UhUf/hiY/oJ+vv7egQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=83WHn3ZZJYjV8PsMxZHlIzybq3Xsxk/FaQbLslWUMqM=; b=TmMWxXZadLHXtXzTlwsBQHD0gMKjITUWg5E3U/StplVczXhrdpaWQI79LZUK59SJ9lxrz3CS6+HVQ3ZX/SNNDJS5aujV3Ex2txKA5HH9bDkX6JgI2WYrsWc0ajDU58SDjsJYHisFFEpC2gMpSFcV+wDQrd3TIRPbp/T86dfXqEzWkehqyuDs3ZPI1NfGAUj2s22ytBL+UcJIlw/ogY9Mo7zP6k9TC/HyRhZmkGPPhOXjrqgSxnSxaYPgCDWO1HZmYbJU/6zT4s9I6T2Qd4jG/yLW3+Vnx6Agm+Xk2gxKsUaI3bxkQShgMeIBrWJyKCSHwmtU2kgQC2VRbASmmNJVlA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com; dkim=pass header.d=suse.com; arc=none Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=suse.com; Received: from HE1PR0402MB3497.eurprd04.prod.outlook.com (2603:10a6:7:83::14) by VI1PR0402MB3536.eurprd04.prod.outlook.com (2603:10a6:803:2::12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5314.19; Mon, 6 Jun 2022 14:41:57 +0000 Received: from HE1PR0402MB3497.eurprd04.prod.outlook.com ([fe80::8002:50a5:a57a:d8fe]) by HE1PR0402MB3497.eurprd04.prod.outlook.com ([fe80::8002:50a5:a57a:d8fe%5]) with mapi id 15.20.5314.019; Mon, 6 Jun 2022 14:41:56 +0000 From: Geliang Tang To: mptcp@lists.linux.dev CC: Geliang Tang Subject: [PATCH mptcp-next v5 3/7] mptcp: placeholder for subflow_push_pending Date: Mon, 6 Jun 2022 22:41:18 +0800 Message-ID: <9073029677e1ad5e96a2d08fe60fd9ee68511c57.1654526068.git.geliang.tang@suse.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: References: Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: SG2PR02CA0091.apcprd02.prod.outlook.com (2603:1096:4:90::31) To HE1PR0402MB3497.eurprd04.prod.outlook.com (2603:10a6:7:83::14) Precedence: bulk X-Mailing-List: mptcp@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: c010fdc2-b8ce-485b-31ea-08da47cab4ad X-MS-TrafficTypeDiagnostic: VI1PR0402MB3536:EE_ X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: TDSS4666KEMQisf1HPCenksz+KCDRCqVIRUG3cNa/gBr3bM221IMhKpD8ENTqbB39MgVDckxO5UT2vp05Khtj0p37CRt9kpHDu9kqo0mI0fqkdEl/Lscv275PV/oR37oyUF5H/RKyg1g9NhgYNUnXqk2T0wvfLgrGBwZVUCcMDb9sVyW9MbFNYsPhjkPq2E2fozH2OJvYr9ltKvaV0taWfQSDoK/+tPCXvPxsAz5XQudYEF9SN9YkCxGwsIdfrdQwpRGAPwz9F6Tti1ex/9VVjKlQSu18bkpXb/vUooNl9rpUlcjALvqdyvykBfmCqgb+5w51vsBYGrj6Wjzq79lZYrEsHBMBZ18E0F4REn6DeqXsPARHNjHFoaNzAvoob1TrcOyfd7cKeBRD3wSYUcClLjpFPuzmv+bevbJCr4Q7TPRq+sbZrHPnOUFxrz8mlGpAMi8BLOKmm6MOAtflK09xPs5H0VIv7rNbTU0/o5HQlIxNx+NJpnQ26WDFI2P5/wzu/KhNj+sr20gzd1bad63u0YDPJgNfezyV7DY6RIxTN+3PlHB3ntHlYBv2DEkUwUtG7bJM+2f1cBglrn6Bp8blMFpF1YeAWSHLgYPlk/hy5nWWoxoYRTMeVB2NwySPG1Rr+WfhyDVG5vBAx6T3h9JBoQ2RePS36JR2JSTBQPYL6pG2139CeoDFrYAq3Yx/LPZafiKm7qfpbKkWRkls4RD/Q== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:HE1PR0402MB3497.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230001)(366004)(83380400001)(107886003)(36756003)(66946007)(186003)(2616005)(6486002)(6512007)(26005)(6666004)(6506007)(38100700002)(66556008)(66476007)(8936002)(44832011)(316002)(86362001)(4326008)(508600001)(8676002)(5660300002)(6916009)(2906002)(13296009);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?MoM2Lu1cXcRaUSshMZPRQ2r6RNAOWFbBXVyomdVPLal8hhvPSosYc8s1xxqZ?= =?us-ascii?Q?6Kx7MOs7Xbjxx8twkqTn4U25Y/Z7a0MhBUWOxQOFOS62DQZ2v/iEWmqNN5RH?= =?us-ascii?Q?zCtA8Q02jYeQcmtwupJeWs6Cgl7yvVOHmUUylX3F3VF8a1fx8JOZ9DBtKk2W?= =?us-ascii?Q?xoy32Ooiv96IAD8R0RZwIfe8XexhSZS+VVhSsMFKToFLjE+KR9rJ1MnoNmLF?= =?us-ascii?Q?O5kbTq0YGcToK/7THd/yQBDj32cRx+NHOZROFd7Y+oaP9uq93EY/zbszpY6L?= =?us-ascii?Q?M96ZZ06jmkZLzb4/FvdwUOSG+DmU8F3SADPlvFmvbm4iiIKmBNwmzYCVxA2u?= =?us-ascii?Q?koFIui40gTmgB0Rhs5luPrlIuNSKX6nHjDeiasdmEQkWn0LWt9ySV9tCtr5y?= =?us-ascii?Q?/6NwfN0m2MRIqBFFbP9yVSPdw4sXb1+hwc7yAa8Zv9DdHp8zi+Y0I70oeiz/?= =?us-ascii?Q?DII0FWRP4RJomehU0RjWlfd5V9KklruTXu2Sjw2sXg5mOczFd6EeuF0w6Ai7?= =?us-ascii?Q?RBSe0NbwhAQL0SGWFy0diwTeTjtpfFa+MA1X+rxp5BEiqrmR4TTwPT886QqS?= =?us-ascii?Q?vTrt9qqfsbKlltgRgPFHY6KefgX2mJCJdJqkEVKBcW+ZVWWVaZRsUIUHgj4K?= =?us-ascii?Q?2Dt5NDdgSi9dfWciKhAgcBnNTktudLnFqhxuRkM5pFbKn92Y9IHDW10LZHin?= =?us-ascii?Q?A3zdjSdattNAlZZ2QM4q2S6bUJXytLIpG+fxlSfsrmMxv0nyc8URZrVxbjCR?= =?us-ascii?Q?Z/XBHklHigt8H8hd8YQvgP+9WsKGFsaem/ZvPm2R3fjwcYJbvmIyDeXU+xVc?= =?us-ascii?Q?gLwVpHko/45usXOGOwFhKL7d4Y5D4XjFT/RlApR+g/atKh5tm1c5eApVDP+o?= =?us-ascii?Q?zHSNkKEXf3zTNYReK6YxeJQB8XpOxs5c03gqHqC3nQ4pzpQj1FGmuwG7ZU35?= =?us-ascii?Q?1XjSd1zIR1PpCI9s+eVj27diANj5giUkwYOKhvsjI2kDMr8RRC9S3EiPjimh?= =?us-ascii?Q?dsMNvPbjCz8B+svhiaTe582jP5gY7zO3CdQfaY10QmGiXfTR4m2OQCoa9vLw?= =?us-ascii?Q?M8d8AYDTzL5pF76YQpcchhJSJgRbdjG/RJVfTlvfz5nERueLaUImi/X+DPxA?= =?us-ascii?Q?gADDzajdy7JcYYcPF3fi+gBANXRNWOryb52CU4aV7OqBVSAtv8X1u3qZTVuq?= =?us-ascii?Q?hW9WN1eg7nek2tPHJMg3RLU3AFzuw7iH4iF3q3HtYZcO3PIQRashwtIMMCE7?= =?us-ascii?Q?mOP+Wn4BG+69yAjY9yfmorfxTGX6N5bZ/eCFwYiffYCh/RTVojspUyo0C/jI?= =?us-ascii?Q?oWCe9FRxjrya2IWEFrv2rJXs4+jKdoVbMGpKgOFMpwC08RdfD+N2HTaSp9wF?= =?us-ascii?Q?+fAMc4svm3JZawDpgO8kQyxF142in1KMh1L2uR+PrL81bLzVH8oV6cdhYGlZ?= =?us-ascii?Q?7kcuwjSgCjuNbnC8KL/gcFbOK6psbVOx7rDAvZzpG1Vy2m8oYU1T72FgjOgm?= =?us-ascii?Q?tkUKAdulVHTzIH7ZTaHap4koV70aAcl39NR3C2X8l1qE9H8pVN4rH9qs1ekc?= =?us-ascii?Q?j4D75t/0Qfex88gULV6JIb0zCHjnXFRAPpmkQcHRHN04eeF3+Ju36im1CZyw?= =?us-ascii?Q?iSejn1LCTVUugp6OwoMGOXPCmLtSD05QwiP29+DUQ6INNnSbJCTQ07copFPY?= =?us-ascii?Q?rIqD4sAZ88RhU221yitFrJCWAs8Tv5BifxjoQhUdFQxiwCr6A2Xhs3572p1A?= =?us-ascii?Q?W+Nws5fopk/VR7afXzpnlng2JbYdeYM=3D?= X-OriginatorOrg: suse.com X-MS-Exchange-CrossTenant-Network-Message-Id: c010fdc2-b8ce-485b-31ea-08da47cab4ad X-MS-Exchange-CrossTenant-AuthSource: HE1PR0402MB3497.eurprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 06 Jun 2022 14:41:56.8280 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: KVUQOlq83Ye7xxIsIm6/1/CVeEsGY1aXnP+FB7NnIIBTVYfFXF9Wt2GX6kl/y5AWW7w1cH1JzNUZEyqmTsOLKg== X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0402MB3536 Content-Type: text/plain; charset="utf-8" Redundant subflows are not supported in __mptcp_subflow_push_pending() yet. This patch adds a placeholder in mptcp_sched_get_send() to pick the first subflow for the redundant subflows case. Signed-off-by: Geliang Tang --- net/mptcp/protocol.c | 8 +++++--- net/mptcp/sched.c | 15 +++++++++++++++ 2 files changed, 20 insertions(+), 3 deletions(-) diff --git a/net/mptcp/protocol.c b/net/mptcp/protocol.c index 8bf48387f9d8..bcaf91562e2b 100644 --- a/net/mptcp/protocol.c +++ b/net/mptcp/protocol.c @@ -1629,13 +1629,13 @@ static void __mptcp_subflow_push_pending(struct soc= k *sk, struct sock *ssk) info.limit =3D dfrag->data_len; len =3D dfrag->data_len - dfrag->already_sent; while (len > 0) { - int ret =3D 0; + int ret =3D 0, err =3D 0; =20 /* the caller already invoked the packet scheduler, * check for a different subflow usage only after * spooling the first chunk of data */ - xmit_ssk =3D first ? ssk : mptcp_subflow_get_send(mptcp_sk(sk)); + xmit_ssk =3D first ? ssk : mptcp_sched_get_send(mptcp_sk(sk), &err); if (!xmit_ssk) goto out; if (xmit_ssk !=3D ssk) { @@ -3109,11 +3109,13 @@ void __mptcp_data_acked(struct sock *sk) =20 void __mptcp_check_push(struct sock *sk, struct sock *ssk) { + int err =3D 0; + if (!mptcp_send_head(sk)) return; =20 if (!sock_owned_by_user(sk)) { - struct sock *xmit_ssk =3D mptcp_subflow_get_send(mptcp_sk(sk)); + struct sock *xmit_ssk =3D mptcp_sched_get_send(mptcp_sk(sk), &err); =20 if (xmit_ssk =3D=3D ssk) __mptcp_subflow_push_pending(sk, ssk); diff --git a/net/mptcp/sched.c b/net/mptcp/sched.c index a0585182a2b1..527ea07982c5 100644 --- a/net/mptcp/sched.c +++ b/net/mptcp/sched.c @@ -119,6 +119,7 @@ static int mptcp_sched_data_init(struct mptcp_sock *msk= , bool reinject, =20 struct sock *mptcp_sched_get_send(struct mptcp_sock *msk, int *err) { + struct mptcp_subflow_context *subflow; struct mptcp_sched_data data; struct sock *ssk =3D NULL; =20 @@ -147,5 +148,19 @@ struct sock *mptcp_sched_get_send(struct mptcp_sock *m= sk, int *err) mptcp_sched_data_init(msk, false, &data); msk->sched->get_subflow(msk, &data); =20 + mptcp_for_each_subflow(msk, subflow) { + if (READ_ONCE(subflow->scheduled)) { + /* + * TODO: Redundant subflows are not supported in + * __mptcp_subflow_push_pending() yet. Here's a + * placeholder to pick the first subflow for the + * redundant subflows case. + */ + ssk =3D subflow->tcp_sock; + *err =3D 0; + break; + } + } + return ssk; } --=20 2.34.1 From nobody Thu Sep 18 06:42:56 2025 Delivered-To: wpasupplicant.patchew@gmail.com Received: by 2002:a9f:3042:0:0:0:0:0 with SMTP id i2csp1274073uab; Mon, 6 Jun 2022 07:42:14 -0700 (PDT) X-Google-Smtp-Source: ABdhPJx8QqRvDkSqIAPemR5gEG3D7YWyove9hqFe9+yPPARYKijv6DHsni/tqkyqj0aqnh/yjKOX X-Received: by 2002:a05:6870:e990:b0:f2:c296:25a7 with SMTP id r16-20020a056870e99000b000f2c29625a7mr13295670oao.72.1654526534427; Mon, 06 Jun 2022 07:42:14 -0700 (PDT) Return-Path: Received: from da.mirrors.kernel.org (da.mirrors.kernel.org. [139.178.84.19]) by mx.google.com with ESMTPS id e1-20020a0568301f2100b0060bfd8c6062si3050749oth.227.2022.06.06.07.42.14 for (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Mon, 06 Jun 2022 07:42:14 -0700 (PDT) Received-SPF: pass (google.com: domain of mptcp+bounces-5575-wpasupplicant.patchew=gmail.com@lists.linux.dev designates 139.178.84.19 as permitted sender) client-ip=139.178.84.19; Authentication-Results: mx.google.com; dkim=pass header.i=@suse.com header.s=mimecast20200619 header.b=Y3QWE4R+; arc=fail (body hash mismatch); spf=pass (google.com: domain of mptcp+bounces-5575-wpasupplicant.patchew=gmail.com@lists.linux.dev designates 139.178.84.19 as permitted sender) smtp.mailfrom="mptcp+bounces-5575-wpasupplicant.patchew=gmail.com@lists.linux.dev"; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=suse.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by da.mirrors.kernel.org (Postfix) with ESMTPS id B37E92E0A13 for ; Mon, 6 Jun 2022 14:42:12 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 7CF1028E7; Mon, 6 Jun 2022 14:42:11 +0000 (UTC) X-Original-To: mptcp@lists.linux.dev Received: from de-smtp-delivery-102.mimecast.com (de-smtp-delivery-102.mimecast.com [194.104.111.102]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A598528E6 for ; Mon, 6 Jun 2022 14:42:09 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619; t=1654526527; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=WpjUNxakoDYsJrD+L6cn9Mgn3TQ8B/71ZLk405R4/pE=; b=Y3QWE4R+oqSxxJivDNYGaI1qhciglTTO/ehK0sa2SbqkVOZRZDpGcQ6av1CdjLxMLS6hln fraV+GuvShtsx5Dwlr+qNuJOzbrOkeFCIqhSIgP0pqjsYwDuZh+y5J7UeQCmBLXXyvtjVH WXczZVsKb/vdHEwtOsYtDOzeX9WZOjM= Received: from EUR01-VE1-obe.outbound.protection.outlook.com (mail-ve1eur01lp2056.outbound.protection.outlook.com [104.47.1.56]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id de-mta-34-KJy8ApwPOeeWWXxBo2j_qw-1; Mon, 06 Jun 2022 16:42:05 +0200 X-MC-Unique: KJy8ApwPOeeWWXxBo2j_qw-1 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=ZnWHo6q5Phxz+pSWo90ame023k/vo6Fx0gCg+DNzD0+Ibh6frmhH8FckLjTBqRqtVyR756ssxRfm4UihkD3SU4y+MDb13h+Gc6+Q6FvTtzQc1542J7w0AuGJQrIaWUr8nj9xIJAyhlhRHPZvRXC1LgAHNnmHLJy6xKBWrfLDWOPZKr/Rj5yLTDap9k6KNqQX7wApWO3buyM3jAIJ0Z2OS8sL+boySuY+5VeqOnBHatA2KjjqFQwCnA6gsIuBGdl1tkUpaSRSf2J2WRsPYK/IB0wn+6VOrxItl25WT/uJGbRvvqdAgOrbnCYlWkhsdMgGaYwlZ2TcecSr4r/+jIkTsQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=41A+AfoNiIQMProk4LPWgAFwqELIrOxhERIvdoH7a1k=; b=QcRmaSQxzmWzTDUXGZbiYI7wGMy/6WuDMVlVuAeb2ZvVYOADk0YzuqSwAow0qhaE3IovM18Ij8cIBIxC6N/HzdrEhlX5GWY2dzqP3327MaaED1Q+NIyFwjaXmheIbookV7BkMUL2sr0FJbKDJYM6iKgEwFhwCf8e5n3MxaXuZYC1/DfeuAGcdZtdXZaTNUoZkwaISKHcazVlvL278XYV4tEi/5EFj937tpq8LLO2W6H9nQ/SSGyOHlGoiG2IuRQT3x/KdwJYWXizBh18Lgup3eoCyJcHy23F2+5Zch2vNjIcDw6lDr3KbyVKILC0Fqw1J4q9grsbT5FoIWMwoMdGxg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com; dkim=pass header.d=suse.com; arc=none Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=suse.com; Received: from HE1PR0402MB3497.eurprd04.prod.outlook.com (2603:10a6:7:83::14) by AM0PR0402MB3364.eurprd04.prod.outlook.com (2603:10a6:208:17::28) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5314.19; Mon, 6 Jun 2022 14:42:03 +0000 Received: from HE1PR0402MB3497.eurprd04.prod.outlook.com ([fe80::8002:50a5:a57a:d8fe]) by HE1PR0402MB3497.eurprd04.prod.outlook.com ([fe80::8002:50a5:a57a:d8fe%5]) with mapi id 15.20.5314.019; Mon, 6 Jun 2022 14:42:03 +0000 From: Geliang Tang To: mptcp@lists.linux.dev CC: Geliang Tang Subject: [PATCH mptcp-next v5 4/7] mptcp: add sched_get_retrans wrapper Date: Mon, 6 Jun 2022 22:41:19 +0800 Message-ID: X-Mailer: git-send-email 2.34.1 In-Reply-To: References: Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: SG2PR02CA0081.apcprd02.prod.outlook.com (2603:1096:4:90::21) To HE1PR0402MB3497.eurprd04.prod.outlook.com (2603:10a6:7:83::14) Precedence: bulk X-Mailing-List: mptcp@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 204a3fd7-29be-4d0f-791b-08da47cab896 X-MS-TrafficTypeDiagnostic: AM0PR0402MB3364:EE_ X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: QdVtWyB3xQQ5g03Nf0yOfItpMLm6kgdccnUYeyPgKq2e072k5xkPuqcm5o2bJEm17e44a8X9lIUFkoBlj5nFBdF+0p0NETmwwc5HIgKPVDs6I3VHAbUJw9HOH++wtnIuLuj593YieZQi5ZRd/IsgcHK8BHbxuNN7pcAoempd+Z/T4da9JrMvCD5y1tsLqnbQ8/mKz/DOYUYv6kywS5Q5e3NFM+S8hNmJaqpAWvUztv1s2j6a69iqnUtEreTJfIIoyrOYhh6HuY3S7SrvhXT+u0O3iKHMQ/FS5P+DsH/fldS6iMDu1fRfeT2S01hLVGSvjdIzfLAdYZNVXAVmHHT2YYH1GqBTGMDpRZLsw+CzbWmJt2UMImnLsz2bZVuan59DHPXvMGSbENgcdEmjirSKBC1FgQKCcHM8eA7gv8CegwuLlhzqmXdOS12uZjpzRBgKRmdDcR6pfOeWpZ2/7101UJ/6f727XQidn021PwhGKNKJOXDl3RDl/r0FEQOr6UZJWxvpnKEOTIEh0lWgEA8xUOQYSgmpYSj7XJZ6u61BaWY0717KN4nxO69xkylN6iqpeo1EEZqAlewhG6L2o3ZckBX8VUsqt8L/AmmcBVbpQpZq6tF+BPfp4TlDmsBppAx6EnNSL0SKwVBMgvZxPtnlLzYmxrZuOINjl8dF9QZnzYiR8xjCwC1eIMoC4DwUao2O X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:HE1PR0402MB3497.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230001)(366004)(6916009)(36756003)(8936002)(66476007)(38100700002)(107886003)(26005)(316002)(66946007)(5660300002)(6512007)(4326008)(66556008)(8676002)(83380400001)(44832011)(6506007)(6666004)(2616005)(508600001)(6486002)(186003)(86362001)(2906002)(13296009);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?3EB71473mG9v1BNSvIY6/e28HorxVWOiEQI6Xa50W5/EAXHJ8UFXsIyZ4+YC?= =?us-ascii?Q?uagryJgmzcq8AxoWGoaGNHt62XIgb5w/3Ic8vqkC0oqXPUuaF6Hs8wz/xVfL?= =?us-ascii?Q?M/oWJC9NqIzyafkpodD8Pl9AQ8L4DMeRrH0Aul8DLPuwGo6C4iJG4I/dEG4r?= =?us-ascii?Q?YRUGCb8BJl3ZJcaroKSCqBhSZribZJHeDFQjNIeg6YWaynRxwikjwjU5QQ97?= =?us-ascii?Q?XhG8THekiv3FLmE1dUzmW5/xPO9WSAayVdLEaGJWeV/ivnbCLKjId+JwluPr?= =?us-ascii?Q?J9i1pkakEM92v8jPPjnlG73VfuOPEI0Z26iSLRuEFNr0DPm5dr9SXpwpKgMT?= =?us-ascii?Q?pVr9ty9qayI0DxV4pUFfwOzVcfT9n8Vvqw2xHoG3th7KZDTm5R/VYQHs9ZJA?= =?us-ascii?Q?kapiuGg2tG7AVHzq5W7bcwFgo/QtJ+FLelF0Xw50hYXkcW79jPFQVi2BuTDi?= =?us-ascii?Q?v6KQUg7O5h9THXH145Hqx4G8wHuyaetGYtGxQDzdVe8HsB4PPgQR7qc2TVCi?= =?us-ascii?Q?TuhzyUJvqoa32wQlWnPsh1XYiLs3OuUW70Wk60CrzT7B/QUgSE8EO/t9o/qK?= =?us-ascii?Q?RbykBG5FxOWwMSJ12qCKUPc5Phz2nFSA3qgkF4W+Cyft7LFeqG6tGs47BQ5U?= =?us-ascii?Q?tJ414YEnxuJ5z+QfCglF1ZoVbIQ8nHoZF9AD0abj/VsRvFrmfMKQA5/wG9Il?= =?us-ascii?Q?LKMxXXuDKm4G67Cv95dQFT2e4pXSXElha1zR423m+YjSh7rDrTT50uWnd3OZ?= =?us-ascii?Q?ZDCsyPIPb4u0ouEBhUu9tNyZvSuZOQRa3wgB8xOF9r6vCh2IsYyRpxF+tezX?= =?us-ascii?Q?su77JiDLXq0wRPFqNDpymsPv5MA3VlRf+2ls6kdFCJhUzHZyt88+5guBb+Kw?= =?us-ascii?Q?pYyv8AEEvzB5ZeSWSXO4h8Qt1J/r1ivc/9Cd9bfGmrY8utc/ITJkTlMxX/rN?= =?us-ascii?Q?+Wztf13ARz8chR89Qg8F5HccSKh9t8mfShi69ClKVcn3NiR2IKyRrdCaZqbO?= =?us-ascii?Q?GqHU6jki/qDrdKWgepaktSfmGMli/9/HuN4CaRgFC0XJZYt1hnkY2JmQZlah?= =?us-ascii?Q?+wMgjLJlu55UaMdWtqKbUk0KL+O3QP9a3kA4jHTbHMLsC/sOCq/uEOjqDaDl?= =?us-ascii?Q?zaaYD9Gs+gKesXlLQ1z/OU9xGvrBw8GciVgiQAU2OmO2bo7jMy1X1PSr8DVh?= =?us-ascii?Q?We0xIu8Bubts5jwl8QQAWNrqb5ZGCPG+Q+Vg0o/Vk9VEUP2oOV/zELp6i5FE?= =?us-ascii?Q?NC4bg/x+X3x6c/gAtqiXODZQwQQ+1i1Bhuyj2UR47Ew91QJSU6om+oTvtq0M?= =?us-ascii?Q?0oYxyANZzTfslqN4jJj1ZMgQrOOPfiQT/fTBcNPkXjrYchIq139TtVukOyIU?= =?us-ascii?Q?yNC++ka05cIKG6Co5z0orSXs28erYG+9yLmXZkPwQJm7RLb0KUr9JVN818MH?= =?us-ascii?Q?frSF/ydgZIiJrgLEDJyP23SN+wtpIHPX78vAtkmAlO5ZamGB8fwnUCuQOpEb?= =?us-ascii?Q?YrW9uFQ1ykcdWugViEvtYKTrOYDSatPSjssPPeASIO4cMXuVHoknaR6zX8Dl?= =?us-ascii?Q?bMd40wO9lABtz0dL3RfUUdIzMBmJfQ+exeME44sB0H3mA1gaMdLr4VTtMh+i?= =?us-ascii?Q?U33T07qDfO0UdklM0qVcJP6BlpunIjUOViwKHJRfqfmeU1OeaZ/UjfwqHwyW?= =?us-ascii?Q?dtO1QxNFAcGhso1jz+GoUfUATMgYZU2m5PqldJh/Xsh/Q2URJHf5LXqrPQeS?= =?us-ascii?Q?U9mZhXdx2779jiShO5uEsSxeO/HR0zY=3D?= X-OriginatorOrg: suse.com X-MS-Exchange-CrossTenant-Network-Message-Id: 204a3fd7-29be-4d0f-791b-08da47cab896 X-MS-Exchange-CrossTenant-AuthSource: HE1PR0402MB3497.eurprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 06 Jun 2022 14:42:03.4525 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: Ep4kYaUdK0XK3CHbTWdrgqpG5PGnIbJE4Dj9ioEiAn6lTkMXKn09JfxtER3Xpj4MhElYuyQlX4Dmam1yaphQBQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM0PR0402MB3364 Content-Type: text/plain; charset="utf-8" This patch defines a wrapper named mptcp_sched_get_retrans(), invoke mptcp_subflow_get_retrans() or get_subflow() of msk->sched in it. Move sock_owned_by_me() and the fallback check code from mptcp_subflow_get_retans() into this wrapper. Signed-off-by: Geliang Tang --- net/mptcp/protocol.c | 7 +------ net/mptcp/protocol.h | 2 ++ net/mptcp/sched.c | 25 +++++++++++++++++++++++++ 3 files changed, 28 insertions(+), 6 deletions(-) diff --git a/net/mptcp/protocol.c b/net/mptcp/protocol.c index bcaf91562e2b..b583454cfc91 100644 --- a/net/mptcp/protocol.c +++ b/net/mptcp/protocol.c @@ -2196,17 +2196,12 @@ static void mptcp_timeout_timer(struct timer_list *= t) * * A backup subflow is returned only if that is the only kind available. */ -static struct sock *mptcp_subflow_get_retrans(struct mptcp_sock *msk) +struct sock *mptcp_subflow_get_retrans(struct mptcp_sock *msk) { struct sock *backup =3D NULL, *pick =3D NULL; struct mptcp_subflow_context *subflow; int min_stale_count =3D INT_MAX; =20 - sock_owned_by_me((const struct sock *)msk); - - if (__mptcp_check_fallback(msk)) - return NULL; - mptcp_for_each_subflow(msk, subflow) { struct sock *ssk =3D mptcp_subflow_tcp_sock(subflow); =20 diff --git a/net/mptcp/protocol.h b/net/mptcp/protocol.h index 0fb2970a7a2d..3b6b9e43e261 100644 --- a/net/mptcp/protocol.h +++ b/net/mptcp/protocol.h @@ -632,7 +632,9 @@ void mptcp_release_sched(struct mptcp_sock *msk); void mptcp_subflow_set_scheduled(struct mptcp_subflow_context *subflow, bool scheduled); struct sock *mptcp_subflow_get_send(struct mptcp_sock *msk); +struct sock *mptcp_subflow_get_retrans(struct mptcp_sock *msk); struct sock *mptcp_sched_get_send(struct mptcp_sock *msk, int *err); +int mptcp_sched_get_retrans(struct mptcp_sock *msk); =20 static inline bool __mptcp_subflow_active(struct mptcp_subflow_context *su= bflow) { diff --git a/net/mptcp/sched.c b/net/mptcp/sched.c index 527ea07982c5..0ac80eb9a428 100644 --- a/net/mptcp/sched.c +++ b/net/mptcp/sched.c @@ -164,3 +164,28 @@ struct sock *mptcp_sched_get_send(struct mptcp_sock *m= sk, int *err) =20 return ssk; } + +int mptcp_sched_get_retrans(struct mptcp_sock *msk) +{ + struct mptcp_sched_data data; + struct sock *ssk =3D NULL; + + sock_owned_by_me((const struct sock *)msk); + + /* the following check is moved out of mptcp_subflow_get_retrans */ + if (__mptcp_check_fallback(msk)) + return -EINVAL; + + if (!msk->sched) { + ssk =3D mptcp_subflow_get_retrans(msk); + if (!ssk) + return -EINVAL; + mptcp_subflow_set_scheduled(mptcp_subflow_ctx(ssk), true); + return 0; + } + + mptcp_sched_data_init(msk, true, &data); + msk->sched->get_subflow(msk, &data); + + return 0; +} --=20 2.34.1 From nobody Thu Sep 18 06:42:56 2025 Delivered-To: wpasupplicant.patchew@gmail.com Received: by 2002:a9f:3042:0:0:0:0:0 with SMTP id i2csp1274089uab; Mon, 6 Jun 2022 07:42:17 -0700 (PDT) X-Google-Smtp-Source: ABdhPJw//vXVPGPqTPNh7SFaBjZNDc1GrxR+6F3X/dJSLp7knJYwiRq7y46jg8Px4KRNkg//r9c7 X-Received: by 2002:ac8:5c94:0:b0:304:c37f:422e with SMTP id r20-20020ac85c94000000b00304c37f422emr18123114qta.531.1654526537400; Mon, 06 Jun 2022 07:42:17 -0700 (PDT) Return-Path: Received: from sv.mirrors.kernel.org (sv.mirrors.kernel.org. [2604:1380:45e3:2400::1]) by mx.google.com with ESMTPS id ay31-20020a05620a179f00b006a3287488c4si8587211qkb.291.2022.06.06.07.42.17 for (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Mon, 06 Jun 2022 07:42:17 -0700 (PDT) Received-SPF: pass (google.com: domain of mptcp+bounces-5576-wpasupplicant.patchew=gmail.com@lists.linux.dev designates 2604:1380:45e3:2400::1 as permitted sender) client-ip=2604:1380:45e3:2400::1; Authentication-Results: mx.google.com; dkim=pass header.i=@suse.com header.s=mimecast20200619 header.b=fiG9yEAx; arc=fail (body hash mismatch); spf=pass (google.com: domain of mptcp+bounces-5576-wpasupplicant.patchew=gmail.com@lists.linux.dev designates 2604:1380:45e3:2400::1 as permitted sender) smtp.mailfrom="mptcp+bounces-5576-wpasupplicant.patchew=gmail.com@lists.linux.dev"; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=suse.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sv.mirrors.kernel.org (Postfix) with ESMTPS id D9A3E280A9A for ; Mon, 6 Jun 2022 14:42:16 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id C1BAB28E7; Mon, 6 Jun 2022 14:42:15 +0000 (UTC) X-Original-To: mptcp@lists.linux.dev Received: from de-smtp-delivery-102.mimecast.com (de-smtp-delivery-102.mimecast.com [194.104.111.102]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 04FFC28E6 for ; Mon, 6 Jun 2022 14:42:13 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619; t=1654526532; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=43Xc4zAAVm2/IrmRTfHq1KjtAt6cuVdaLuPi08o3GQk=; b=fiG9yEAxejr4ey0/MvfphDWXi/HsG2Hwj9EO4rz7qzhqYYMnNBCbXs9NwFmCdtdvqc+zyJ r8OEQq8pJSUsRhhhU9BHgIMZm/b4BdwPE12vLyem2X8hzkH2XZq3F2VsDoJPeDQ0sizzKB IlLHvhffvl4fU5cw7bz0TNDwxihpX9M= Received: from EUR01-VE1-obe.outbound.protection.outlook.com (mail-ve1eur01lp2056.outbound.protection.outlook.com [104.47.1.56]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id de-mta-46-DPMB7QG_OYqS3bhYTdVGcg-1; Mon, 06 Jun 2022 16:42:11 +0200 X-MC-Unique: DPMB7QG_OYqS3bhYTdVGcg-1 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=WitOxU4dw+2LT91JF3bROTkRwos1q4N0r9WUsI7QWDJUS7U+13s72p/Kq5J9oyqYEAmMqup/7sEq2zPzlCDawapzRC/kcEOQ7XH2CH2NI2Lg9fddYLXGr8hVJ0KtdK6SVzMdGfYVXnJR8D4DKwNBibZ2jG12ew/6/uDcgKub0/pAfThASDH/kw27NzdT4QhIjzXQRqaW+AClr+DeEikCI+gtdcIPFbihNIpaT5nK4dbRzxXFsVIp2XoL11qVCdpCJyPhhMuQcykLyFSzO/b+/3YceUKhLKAICnesq/oodFjc2LVRtW8bf4shGBkf6lE5FKefaU0tQ2WozaWqKgExZw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=5nPHL/BQKU/to20kzs5KBBhBIgjNu3B8oHLkt8jhXhc=; b=T2T8BADg4yniP8hPHiHo+Pl1O1Nz9HlspM6TBbhUafN7TNAbajXAMZ6uX3kdfiJdk5Mqa0MB6LfBfaakX5JoHztIFSCrONzIBotNAV7OQt1or3gEHmRzXkq7aS/CD6rDwamRoY1+BxytZY/Sdzsqyp7z10lTrOcZkXwA4dSA2RVXgWVGTAitT0WPKXGjXsXg/89FR5sYPX848UtkuieaYsI9jKcgxeMYrLcx6//gCD1fPmTJlfu0OIZEeUYV8MR8SVk1F5Ek4RSLn4r/xclktO/J9eXRi6j29eF+pv+tAggyQqkBvvJ3Rihk7ILG3K8q3Ve8hfrtGKBEEgclkdEfAw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com; dkim=pass header.d=suse.com; arc=none Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=suse.com; Received: from HE1PR0402MB3497.eurprd04.prod.outlook.com (2603:10a6:7:83::14) by AM0PR0402MB3364.eurprd04.prod.outlook.com (2603:10a6:208:17::28) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5314.19; Mon, 6 Jun 2022 14:42:10 +0000 Received: from HE1PR0402MB3497.eurprd04.prod.outlook.com ([fe80::8002:50a5:a57a:d8fe]) by HE1PR0402MB3497.eurprd04.prod.outlook.com ([fe80::8002:50a5:a57a:d8fe%5]) with mapi id 15.20.5314.019; Mon, 6 Jun 2022 14:42:10 +0000 From: Geliang Tang To: mptcp@lists.linux.dev CC: Geliang Tang Subject: [PATCH mptcp-next v5 5/7] mptcp: redundant subflows retrans support Date: Mon, 6 Jun 2022 22:41:20 +0800 Message-ID: <625ee43a83e329cab83eac38cf3c96bc7de330fa.1654526068.git.geliang.tang@suse.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: References: Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: SG2PR02CA0093.apcprd02.prod.outlook.com (2603:1096:4:90::33) To HE1PR0402MB3497.eurprd04.prod.outlook.com (2603:10a6:7:83::14) Precedence: bulk X-Mailing-List: mptcp@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: e37d4213-d04e-4392-8504-08da47cabc84 X-MS-TrafficTypeDiagnostic: AM0PR0402MB3364:EE_ X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: Yp36ZxE6EStIKz+24AeLdtMnjpluUwCE/QLrtRv4Lh4X3JBwe/h0XuiMvUogCaRacbf7hOsmwKiVPVO8Fl7a8ciyIxEhL3D1m9h1B4V8uAz1EIcl0z5gX+LGTdqxfavHGJshr3v4aJiBFt6/npRl2VCAoIiT88IKr1zo55UzYupuLMS3wxlnXiE2QOUvcgDzHKNZ3AJsTrNFW8ahItH6CistPhjWpFK+DIApL6I0mLck+nYGQ3BKyo3JLror3UVFmyy+UWjnLR6WbGwm8SjWxohrgMNRl1YsDhUHpzMU4q6THRxMvCJyI1fa2A+dxRFKfA7HelKrg9vNn6/GGlXffQZbxV3fWrEdAwayMGMWWZw+oclRqBGqLGa53e4vOZlxdbp9C5Wr0PYwbxtv8SMr4h9WcNDJpgpdmHQYd7xdPBcvVjms/PyqBs4LVw+XuaK17H/4SXAdPOE9KxtgQWF21qdRyGDlr6Ojxk+6dLbgH5diMApfSef5le6AEBM28IEmRtMQvzjvIKfdW82ps9awD9s4w1DOYgdNGmGM+6pu/Irjrwu+awNYuWac+SXyOHXEIcjWiQnqNJnq0mr0Ibo30DO4JL5nyyvY7wSW2pmdxhpqLWxiBFBeh72+3BsWFRSgdPzmaYZR+cu0Vf89gU0312qZu3h+P+YWbcD44GgC9MJsmLd5w5KU3lhGdDIXP5LFE5VTAiIfwBh+BU6udvzcwQ== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:HE1PR0402MB3497.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230001)(366004)(6916009)(36756003)(8936002)(66476007)(38100700002)(107886003)(26005)(316002)(66946007)(5660300002)(6512007)(4326008)(66556008)(8676002)(83380400001)(44832011)(6506007)(6666004)(2616005)(508600001)(6486002)(186003)(86362001)(2906002)(13296009);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?/lI0aiMa+djBccMDFJhAV66i943iQZGlD7z5+u1cB/ZLwF7Wv2PL3/rFfj4g?= =?us-ascii?Q?hkBFo5Dgz++bsVs2eIrZj/rPCmloCDidBMTQ18j7Arl9BRPwoMpPN1XuusUk?= =?us-ascii?Q?yLR57OxMRIKbXuJs4rU4yFp2YuC2QDQL+GwyNMNiQ3xSfspMfKbBXD+dXViu?= =?us-ascii?Q?oGeghFb9UMLpowODMgYWFGWAvD3hRAuP0+FkRt51wRV7sx0Ta0e7q/VwPrwC?= =?us-ascii?Q?QUcmi5PjG1lvWwgzJOWfcjQ7ufV9ZdbSFhylAICaMLZDk1FxpECCr8ZZQUQz?= =?us-ascii?Q?rkiWDMjXEW+22kaoleOpoNb9NOmwjURcD+gzFtdKzkIl3Uk4m8PiHqqXXflU?= =?us-ascii?Q?Sv7fxY7uukN/v1MUHToS0rmZZhbMLyIMfyRN2DZWW9lcHMSceqraZlwJdl05?= =?us-ascii?Q?Y2LX24ez+q8YmCutzlLjR6tv24fQ12fBegXblQGAI98obfzoAvTKejLLqUcw?= =?us-ascii?Q?61rNp706SC8tmOyGyGRxkNaN3I8vL08QAID0atMmPJw2MAiMjLRwcph74t+F?= =?us-ascii?Q?JLS8EPnMokPo1JpiLcGjPNCnaJR7Wc6KcwKCfjR4elG9GWIYmn2rYcUQMNh0?= =?us-ascii?Q?uN+XhDnMC3Bqf6RkOTxgMa8SpsyRkwsAsV3kfOPNzCmbFExqSHO1qxH9Qvdx?= =?us-ascii?Q?THNynS41x/PEPraOKOpZ7TQWMKcOjVuoUTwDNxGjPjHf1Gb6UHEOdUfSUsO+?= =?us-ascii?Q?1yTjdNxSiYJGUc0HXvflgzFFgI999N3XHJNyYhKK9dI2XdREiybhH/YcE42F?= =?us-ascii?Q?43KJ0jW4wCi/dE/2sLfRqic0t1iHVhDfp63nA/bkxZag8Q6FbdjYygDgWDfN?= =?us-ascii?Q?ahabUJK64/XRKjniSmg6Q93+FRfmf3s12CTAKQgddnbmBt/2IHlYahBQ27g0?= =?us-ascii?Q?5tvtFZ1Ra4owamThpcIe2DlBqAh/G+0V+w0pp9+LZGLvtG9crrl7afxV3kMN?= =?us-ascii?Q?BFxzsLWCtiuccTeatBBFEm51WWjMryLTL3SWTNuledP5TRespRvplxB5mZmL?= =?us-ascii?Q?fWLBgVKoD9kvXCj74qbcwO8wGQ6zyVdrAZDir/TiUCJ3QNNV9nwPXtKuYCiD?= =?us-ascii?Q?vbFumpbN6/+VbqK35gUAld7DqcOqLXoOpkDgEwgkn2iMsBeuDIV3fAjuAcPx?= =?us-ascii?Q?fQiU054v73DfcmrcOuuoqcDdhBHL70tSjGoxT7/MXN0Uw3f1WwVBCwbSRTib?= =?us-ascii?Q?v0zDzxteyDPZH1+qzouoXhURpkvtaMu2pAOfnzhJWmjt5ZkIJWbNrzZ5Ggh4?= =?us-ascii?Q?ixYqFZF82wEuOdmUfiyMRRAUyOmKW/Wq0PUS/42tYck/E5sLVN+X7iyR3awV?= =?us-ascii?Q?W/0gGivV9VxhG4z1ooeW4BIWheB6TeojZO2OB4WqpriBh0Ilog9KWUGUrhtO?= =?us-ascii?Q?dTiZx8sgUckQOSgg5JdI8FV0muFNWifnl2M5llDHD+3f5+JXUQ4chlZuZGuF?= =?us-ascii?Q?wmk3mimzueSlMj5CFNFYUINHer1yO2XeF3HpmY8SpHyR/Ov8Lwyqx0Vby6M1?= =?us-ascii?Q?yAMP5jrBuD//WisIR7jHT5z+CPYouNUyyBBldvDzSZ1rhTQ6kHnxq+RYJ+jH?= =?us-ascii?Q?JV0YnvY6h12NNaf0w05L+kuxkb6+gLPNebXTrRkAJKsxwhrFbfjOu/G6SICp?= =?us-ascii?Q?scNJFQWVAwMaWsacnUgouH+q8t9apnhnjX+debRHq95aEUDcA4GnIW5ZrRyd?= =?us-ascii?Q?sCX3ucbxxpN6is2CJmwmyy1iB/C0iIstZcOubxUW0VsdSrHr5XlKdeXvrrW0?= =?us-ascii?Q?3Xm29wHZWthBMOL3OuY3TYdiF6Z9kaA=3D?= X-OriginatorOrg: suse.com X-MS-Exchange-CrossTenant-Network-Message-Id: e37d4213-d04e-4392-8504-08da47cabc84 X-MS-Exchange-CrossTenant-AuthSource: HE1PR0402MB3497.eurprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 06 Jun 2022 14:42:10.1394 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: FTq56yIawXBQjzdeGdpFFLMSkpQ/aZTJCp5OPThQb2VKfoiZaZA910TT9s9JBhh3PXnAy4R7Wd2FOH1F6TOpIA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM0PR0402MB3364 Content-Type: text/plain; charset="utf-8" This patch adds the redundant subflows support for __mptcp_retrans(). In it, use sched_get_retrans() wrapper instead of mptcp_subflow_get_retrans(). Iterate each subflow of msk, check the scheduled flag to test if it is picked by the scheduler. If so, use it to send data. Signed-off-by: Geliang Tang --- net/mptcp/protocol.c | 45 ++++++++++++++++++++++++++++++-------------- 1 file changed, 31 insertions(+), 14 deletions(-) diff --git a/net/mptcp/protocol.c b/net/mptcp/protocol.c index b583454cfc91..f39318f5a21e 100644 --- a/net/mptcp/protocol.c +++ b/net/mptcp/protocol.c @@ -2445,16 +2445,17 @@ static void mptcp_check_fastclose(struct mptcp_sock= *msk) static void __mptcp_retrans(struct sock *sk) { struct mptcp_sock *msk =3D mptcp_sk(sk); + struct mptcp_subflow_context *subflow; struct mptcp_sendmsg_info info =3D {}; struct mptcp_data_frag *dfrag; size_t copied =3D 0; struct sock *ssk; - int ret; + int err; =20 mptcp_clean_una_wakeup(sk); =20 /* first check ssk: need to kick "stale" logic */ - ssk =3D mptcp_subflow_get_retrans(msk); + err =3D mptcp_sched_get_retrans(msk); dfrag =3D mptcp_rtx_head(sk); if (!dfrag) { if (mptcp_data_fin_enabled(msk)) { @@ -2473,32 +2474,48 @@ static void __mptcp_retrans(struct sock *sk) goto reset_timer; } =20 - if (!ssk) + if (err) goto reset_timer; =20 - lock_sock(ssk); - /* limit retransmission to the bytes already sent on some subflows */ info.sent =3D 0; info.limit =3D READ_ONCE(msk->csum_enabled) ? dfrag->data_len : dfrag->al= ready_sent; while (info.sent < info.limit) { - ret =3D mptcp_sendmsg_frag(sk, ssk, dfrag, &info); - if (ret <=3D 0) - break; + int ret =3D 0, max =3D 0; + + mptcp_for_each_subflow(msk, subflow) { + if (READ_ONCE(subflow->scheduled)) { + ssk =3D mptcp_subflow_tcp_sock(subflow); + if (!ssk) + goto reset_timer; + + lock_sock(ssk); =20 + ret =3D mptcp_sendmsg_frag(sk, ssk, dfrag, &info); + if (ret <=3D 0) + break; + + if (ret > max) + max =3D ret; + + tcp_push(ssk, 0, info.mss_now, tcp_sk(ssk)->nonagle, + info.size_goal); + + release_sock(ssk); + + msk->last_snd =3D ssk; + mptcp_subflow_set_scheduled(subflow, false); + } + } MPTCP_INC_STATS(sock_net(sk), MPTCP_MIB_RETRANSSEGS); - copied +=3D ret; - info.sent +=3D ret; + copied +=3D max; + info.sent +=3D max; } if (copied) { dfrag->already_sent =3D max(dfrag->already_sent, info.sent); - tcp_push(ssk, 0, info.mss_now, tcp_sk(ssk)->nonagle, - info.size_goal); WRITE_ONCE(msk->allow_infinite_fallback, false); } =20 - release_sock(ssk); - reset_timer: mptcp_check_and_set_pending(sk); =20 --=20 2.34.1 From nobody Thu Sep 18 06:42:56 2025 Delivered-To: wpasupplicant.patchew@gmail.com Received: by 2002:a9f:3042:0:0:0:0:0 with SMTP id i2csp1274118uab; Mon, 6 Jun 2022 07:42:22 -0700 (PDT) X-Google-Smtp-Source: ABdhPJwSqyl913aPKTP+5U1nCBdKv2poayO4IRijTaYb5AwJHXicm3cDO5lnNdEvfi6LzupwlyZH X-Received: by 2002:a05:6870:4210:b0:f3:20cd:7904 with SMTP id u16-20020a056870421000b000f320cd7904mr24565084oac.284.1654526542622; Mon, 06 Jun 2022 07:42:22 -0700 (PDT) Return-Path: Received: from da.mirrors.kernel.org (da.mirrors.kernel.org. [139.178.84.19]) by mx.google.com with ESMTPS id c12-20020a9d6c8c000000b0060ae98c4308si17461702otr.96.2022.06.06.07.42.22 for (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Mon, 06 Jun 2022 07:42:22 -0700 (PDT) Received-SPF: pass (google.com: domain of mptcp+bounces-5577-wpasupplicant.patchew=gmail.com@lists.linux.dev designates 139.178.84.19 as permitted sender) client-ip=139.178.84.19; Authentication-Results: mx.google.com; dkim=pass header.i=@suse.com header.s=mimecast20200619 header.b=bLP5+vWU; arc=fail (body hash mismatch); spf=pass (google.com: domain of mptcp+bounces-5577-wpasupplicant.patchew=gmail.com@lists.linux.dev designates 139.178.84.19 as permitted sender) smtp.mailfrom="mptcp+bounces-5577-wpasupplicant.patchew=gmail.com@lists.linux.dev"; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=suse.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by da.mirrors.kernel.org (Postfix) with ESMTPS id 70D012E0A1E for ; Mon, 6 Jun 2022 14:42:22 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 3828228E7; Mon, 6 Jun 2022 14:42:21 +0000 (UTC) X-Original-To: mptcp@lists.linux.dev Received: from de-smtp-delivery-102.mimecast.com (de-smtp-delivery-102.mimecast.com [194.104.109.102]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B8A2F28E6 for ; Mon, 6 Jun 2022 14:42:19 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619; t=1654526537; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=/SsBCKaGSnSib8yKzRg7exL65wED8f+Ur/LUxp2VQOk=; b=bLP5+vWU/q8o/fcPhMCra49+lLj88Wq2frbGlNJUdVaJ2OWSrYC917JzN6CIf0evNBEDM7 fQLHgNODpAaxAk1W4yIlv3Y6AKmIDwnU9lL6yZg+es3bXB+lsXDppfKsdRRtLoffhorde4 WQxX5dzUHL8ypjI05aF6ZEabDWdzR+0= Received: from EUR01-VE1-obe.outbound.protection.outlook.com (mail-ve1eur01lp2051.outbound.protection.outlook.com [104.47.1.51]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id de-mta-9-CGnik2giPqytLI-vFGKHSQ-1; Mon, 06 Jun 2022 16:42:17 +0200 X-MC-Unique: CGnik2giPqytLI-vFGKHSQ-1 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Ku/JNs521udC6emg3SydwIe1VLbIbAhLaofipNTlZKQ3aTPhNcK9bPR5yi6ivpPfWnooGcqY7bxxpHoJ4O+dppkHuuaKlNFOCeHYq9slatSTHqq8bGVhTckBxJwizG6O20AZWeXo/wVJW/sHUSxozQT+c239+QxN5/ZZ8lDXi9ucxOjVpgQcNPExm16NrnJcXDeYjQMPvQVnaWg5sQt6eitAlaz7E7sXEkPTn1QEh+owmyKTekGFOc0qFB5lFjdeObdidgZYnASQcD46v7ZeAfZsx4W85SRCqUlq1pLsHZNNzx/nGLVfmNJh9FlmMYnM19bBeHrLmcAJDfeibPSYsg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=uP3UBW9MaHYoUECvpYCXr/Jf/GWdIt6ZXQXJmbYnsTE=; b=iUz01iZDiSffIhIbHPFo3x8GPAHIFn8ke/pwWqaET2FtxWCpqV4OgsKhwEnX5HIbbQ9rOSFOCD3yX4LtoFVTb0b/QEVv0aWMAqF2TKo1gz/xhW1PIJMheXAmLUXhDsbHaxlgjDA+FaX295EBPtFOFe8k+MfoD2tzybvZrhr1/7cwoLawFZHgV85wQu1t6Se4Dy8JAkbufmTrs9L/3FvDPERaomETLkWZgNqg7AFYsBirxDW3iKnmcl+JdoYTAjaEqnJlUeR1V8zbL8asOVjD0RPpCUd/MvdHntsII1WE/nfi6OlQPYWdt0d0rDpsKbrq1IQXSqKECrMlGlNGcJ1Hgg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com; dkim=pass header.d=suse.com; arc=none Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=suse.com; Received: from HE1PR0402MB3497.eurprd04.prod.outlook.com (2603:10a6:7:83::14) by AM0PR0402MB3364.eurprd04.prod.outlook.com (2603:10a6:208:17::28) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5314.19; Mon, 6 Jun 2022 14:42:16 +0000 Received: from HE1PR0402MB3497.eurprd04.prod.outlook.com ([fe80::8002:50a5:a57a:d8fe]) by HE1PR0402MB3497.eurprd04.prod.outlook.com ([fe80::8002:50a5:a57a:d8fe%5]) with mapi id 15.20.5314.019; Mon, 6 Jun 2022 14:42:15 +0000 From: Geliang Tang To: mptcp@lists.linux.dev CC: Geliang Tang Subject: [PATCH mptcp-next v5 6/7] selftests/bpf: Add bpf_red scheduler Date: Mon, 6 Jun 2022 22:41:21 +0800 Message-ID: <1deb3cdb95a1e4e274fee02bb01298cf617abb67.1654526068.git.geliang.tang@suse.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: References: Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: SG2PR02CA0081.apcprd02.prod.outlook.com (2603:1096:4:90::21) To HE1PR0402MB3497.eurprd04.prod.outlook.com (2603:10a6:7:83::14) Precedence: bulk X-Mailing-List: mptcp@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 75616a9a-8159-4df9-5bab-08da47cac01a X-MS-TrafficTypeDiagnostic: AM0PR0402MB3364:EE_ X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: XOqDOOfdsJ+3aVdkT2FRptOTcyND3DYmGJtww4ee3uX8xaw9jPy8OYZL3v2nFCPDN1bkt+8Dk0T2cU8i0RKhWUo/SavwOE9NrLKnkM1YgZUEtQ0S7DqDN6L9N0dk6zVhY5qOs8iStRo/5dTTqs9t7tzpXnAGQHmggMVh1RE2bV1/rp/1gcU4CKMTALkxBHyUr7jK2DLJ0S3ds+gWkc57dlw7wiOt/9nJKcY+0tcEhYGW16+1OVJyxuSYCM8Eo95eG/5F4Q/JR4hT5oIY/pqZTxPt00B+wlDT20g4MTA4EwlQmLYpW006v8fLc84EA6fwlZ6FxAogK6NjN31OYJEUU3Cnp6JdtwMVtLbk1xGgyUeskk4D16mBVlr5mEvkQuiNqNHz223kmumYFLZjFHlzqCTKKBc6E1cwHvtIfx9WH2UDp+685BwKdcbuCmfiY/1u6H/DDfucuIbXhwKyvz54+cMfueGB1CrxDo0iNKA6FtftL/4yqPgKC13Oj/s3GBr+tHPMtCPhvl/SFAkFyOtNNQ8xu//XYi9cLEJ5PcPTJ17T51S3U8RZ2QXUF4ejz9UcDcigwZKoHG1HifU4YNkmYZW9myG9u97uEMdJOXRPRcXsocNcaYnlKJw7yTfmvz/4ULAko9i/Yt9nFG18xuQ3q/suCOE+Jeyg+4JfOuo7vBIQyyhgb+Cg7P208Z6lGUPAtX4AeWuxy6tXfI+VhYcI/Q== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:HE1PR0402MB3497.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230001)(366004)(6916009)(36756003)(8936002)(66476007)(38100700002)(107886003)(26005)(316002)(66946007)(5660300002)(6512007)(4326008)(66556008)(8676002)(44832011)(6506007)(6666004)(2616005)(508600001)(6486002)(186003)(86362001)(2906002)(13296009);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?Eonerbmo5/0w9mHRtS0cIyyAajbIEnWlvr6LrKlwSnCj39CUj8MQsJCaA3zP?= =?us-ascii?Q?pmw11hQP9qZt1aMhTIC7yvoywlNRODSYG1POyCA/xZLHF0VgXOen+G0yMXrt?= =?us-ascii?Q?rhbwCgFgHVO5zbUKDVGTfOZj2FbhhmM/BFQUM24T4vdKYHLFDwY4okIvqZ8g?= =?us-ascii?Q?ruhSdvopb7s9AVAhNTkHNbPRVKXhkmzHEweH0LOQmuBd1F3EMJWD/mEp8OY+?= =?us-ascii?Q?0xFroTKR0ow+Vjr7daX3Zc0WR3YMdcf24sSiqEoh2+EmPhifP6h1JnI+UpyU?= =?us-ascii?Q?CbJ+oQN0lvZMddysLwuS5buK1JLF5xgqzOTkXj30eqwtIU1ixUJlT485lk3o?= =?us-ascii?Q?V0XB0XlmYkwOlCGUuFWNy/+zD/RKo/pbt1qwbl/3dhy2h8zsxpl84jgVhY9K?= =?us-ascii?Q?HtGpTRb5v8RItmuPQNn6r3B4OkdO1zKJPTqQ1F/IR6TJVwuyt9KtMl3Yx5Lt?= =?us-ascii?Q?20t7MIUo2tEHBGRXbIO91qqLCIeIkS8Rvq1gArhdrnBY6dqGW22rc+Pn4rm/?= =?us-ascii?Q?+McPiQevZIiOXY3a4Yvlh15eTdXdWS9QgzeCxDw07KhKopFt3/E4aFuQkNKC?= =?us-ascii?Q?eceIT9gtjxSRb8nSPpMmpgFvhp42aSBXbMvidasMDpWNkdsMoqoFn+aZFdJo?= =?us-ascii?Q?nhb67HaSLDELKE2+kDr3ArXIk70mQpU/OxwL7DYTLGe28pcG505tC3W6trNJ?= =?us-ascii?Q?WrFg88BM2ppf6pn78KgKnfUmr2m6M7KnPcHB1qbJhyCROc0sufsFq9toyssV?= =?us-ascii?Q?XjCqNQo+84d9ksWTv5zxSKYMHLj1eVRV2q+lfuHC/E17Iuoip9QZtSuAxIOr?= =?us-ascii?Q?5JxiKESte8kiOWr630gOzn/i8UanykyuA71WJcYz+qTSq8DbouSPwdy9GDv3?= =?us-ascii?Q?kH0prfkikBpt08F43FKmK+Rbnpkng6GeEzicHWloT0zlos22Yyn44hperE9E?= =?us-ascii?Q?7BlfMbraDlVBkqRJkxEM9bc8REDgqmhGq9lxtNW1Yq/ME+Qkv7ja0LpvurxW?= =?us-ascii?Q?146gmiKYCggJaeV5bFNt6AUBcWOrU9iEdK313bsEDJk88HWoQ5896VF6nRNW?= =?us-ascii?Q?vf74/137z2lFtyNKpVYuK/tzEB/g2kGqD29UYmjySgyhNk6RNYIoRZ5K16MG?= =?us-ascii?Q?gaW0DNHad2ayyEp2eAw+Fl+vnWSvolItNoeoy3ONTug90ZVTYgDg1v8fXC3t?= =?us-ascii?Q?+Yo/5eu8K6oRHNNbJcOnqxpAIHNFiqZlx9ufS3v7IDdOD6Ze+9y7GAakK5oX?= =?us-ascii?Q?OKjzKkrECW3BgBJNPW32+ZTk6i7MmnOFwWYeV2F8o6BLEeCZgY1DvAlsPW0i?= =?us-ascii?Q?jC8q5B8gN6SLwEuKFUeTdqwM5PEipbQPKBaQI5h0ENvAJFU0iH3375fiu+A9?= =?us-ascii?Q?clbHx2QSlRSMnz1aPdio3wUAmQf5RrlJSFfIluklfEwmUPaTICebGyDL5h+d?= =?us-ascii?Q?38E3JutFYds4lPGBRgnTqy+F12EOfmL5FhpVzgsbGnZIaGlwj8E8MvSN6eQk?= =?us-ascii?Q?33PJUgmrhv44ptyjSp0QV1N5sy/KVvz+eyAPA0eKoLExs1DeD2Bu8BOpXOwT?= =?us-ascii?Q?h782W5qZDlbCsv+TKdQixn3wUf9MlU1P/ZhZx5ySPGrCTP/KfjVDUj03a291?= =?us-ascii?Q?J3nbRgfO+l+a7ILW1g//C49vWgxYBt6C/Ym2N13i/vUqutSA2NDH82hRQKNK?= =?us-ascii?Q?Q7Zlyv1GUEd26goERu8LT/bOp61+lL9qMaQOFuN/Aay8DC4ZFpnLmhtnweKw?= =?us-ascii?Q?JVchfkfme/imIim/YtAbd5MuJlKWVYE=3D?= X-OriginatorOrg: suse.com X-MS-Exchange-CrossTenant-Network-Message-Id: 75616a9a-8159-4df9-5bab-08da47cac01a X-MS-Exchange-CrossTenant-AuthSource: HE1PR0402MB3497.eurprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 06 Jun 2022 14:42:15.9046 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: t2aaa5kHsFGkDEUK42qffSZPB9ElZDU2fSnB+RoFziuYJXvInkpFP4PQ16/wFnvlMvR5qQIc2npV8J3ZmDrwcA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM0PR0402MB3364 Content-Type: text/plain; charset="utf-8" This patch implements the redundant BPF MPTCP scheduler, named bpf_red, which sends all packets redundantly on all available subflows. Signed-off-by: Geliang Tang --- .../selftests/bpf/progs/mptcp_bpf_red.c | 36 +++++++++++++++++++ 1 file changed, 36 insertions(+) create mode 100644 tools/testing/selftests/bpf/progs/mptcp_bpf_red.c diff --git a/tools/testing/selftests/bpf/progs/mptcp_bpf_red.c b/tools/test= ing/selftests/bpf/progs/mptcp_bpf_red.c new file mode 100644 index 000000000000..58f90473e495 --- /dev/null +++ b/tools/testing/selftests/bpf/progs/mptcp_bpf_red.c @@ -0,0 +1,36 @@ +// SPDX-License-Identifier: GPL-2.0 +/* Copyright (c) 2022, SUSE. */ + +#include +#include "bpf_tcp_helpers.h" + +char _license[] SEC("license") =3D "GPL"; + +SEC("struct_ops/mptcp_sched_red_init") +void BPF_PROG(mptcp_sched_red_init, const struct mptcp_sock *msk) +{ +} + +SEC("struct_ops/mptcp_sched_red_release") +void BPF_PROG(mptcp_sched_red_release, const struct mptcp_sock *msk) +{ +} + +void BPF_STRUCT_OPS(bpf_red_get_subflow, const struct mptcp_sock *msk, + struct mptcp_sched_data *data) +{ + for (int i =3D 0; i < MPTCP_SUBFLOWS_MAX; i++) { + if (!data->contexts[i]) + break; + + mptcp_subflow_set_scheduled(data->contexts[i], true); + } +} + +SEC(".struct_ops") +struct mptcp_sched_ops red =3D { + .init =3D (void *)mptcp_sched_red_init, + .release =3D (void *)mptcp_sched_red_release, + .get_subflow =3D (void *)bpf_red_get_subflow, + .name =3D "bpf_red", +}; --=20 2.34.1 From nobody Thu Sep 18 06:42:56 2025 Delivered-To: wpasupplicant.patchew@gmail.com Received: by 2002:a9f:3042:0:0:0:0:0 with SMTP id i2csp1274151uab; Mon, 6 Jun 2022 07:42:29 -0700 (PDT) X-Google-Smtp-Source: ABdhPJy+uC67LpiV2dbiXzuedEDowR2AVpTVp2zXT2FJuwneIJZkTnkVatBsrwkl7wlbFTazeLK9 X-Received: by 2002:a05:6870:5590:b0:e9:1eea:e381 with SMTP id n16-20020a056870559000b000e91eeae381mr13702901oao.221.1654526549124; Mon, 06 Jun 2022 07:42:29 -0700 (PDT) Return-Path: Received: from da.mirrors.kernel.org (da.mirrors.kernel.org. [139.178.84.19]) by mx.google.com with ESMTPS id ay30-20020a056808301e00b003261af3133esi23144807oib.96.2022.06.06.07.42.29 for (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Mon, 06 Jun 2022 07:42:29 -0700 (PDT) Received-SPF: pass (google.com: domain of mptcp+bounces-5578-wpasupplicant.patchew=gmail.com@lists.linux.dev designates 139.178.84.19 as permitted sender) client-ip=139.178.84.19; Authentication-Results: mx.google.com; dkim=pass header.i=@suse.com header.s=mimecast20200619 header.b=Wn7w4Efh; arc=fail (body hash mismatch); spf=pass (google.com: domain of mptcp+bounces-5578-wpasupplicant.patchew=gmail.com@lists.linux.dev designates 139.178.84.19 as permitted sender) smtp.mailfrom="mptcp+bounces-5578-wpasupplicant.patchew=gmail.com@lists.linux.dev"; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=suse.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by da.mirrors.kernel.org (Postfix) with ESMTPS id F35532E0A01 for ; Mon, 6 Jun 2022 14:42:28 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id A183C28E7; Mon, 6 Jun 2022 14:42:27 +0000 (UTC) X-Original-To: mptcp@lists.linux.dev Received: from de-smtp-delivery-102.mimecast.com (de-smtp-delivery-102.mimecast.com [194.104.111.102]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3421228E6 for ; Mon, 6 Jun 2022 14:42:26 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619; t=1654526544; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=dcxFYknSSqn2iCp/EC6eSvsoTZmmfAXbqcIbOIVxtBU=; b=Wn7w4EfhMvUojkNUP7XI5KeYusmQywC5fQzhh+TYt/ruD0GccTPW1V/878xBoj6lAIZg/k 1X4+bK3JAeSb00HPlJcXcdJSNEIUyPZFJxb1vA5M5HT9voTOIZyOMIoa88rcZ0yzTJtWle AUqTv0LYk2EpCDMbLsNjD3nLYmBIwbE= Received: from EUR01-VE1-obe.outbound.protection.outlook.com (mail-ve1eur01lp2052.outbound.protection.outlook.com [104.47.1.52]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id de-mta-32-TTzik33xMQyweqewqVhXMQ-1; Mon, 06 Jun 2022 16:42:23 +0200 X-MC-Unique: TTzik33xMQyweqewqVhXMQ-1 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=EmIqp4cgTDdISQ6LVo2Aoap2KjutQwzuKBBm7CLAjQOVchkQuZVCMH305xdrkdSVod6KzoX74GGH/7uzlS+8UaKBTi/Du/FH4DvO9fevDSzGMywUPj+Way8BwRjls7vZnH3JROchlwhu4sIHl0g8PqamDZXBv02x3u6lG67/5y5FHu2AN7FauFB5FoqhYWpr9POUnTXaQhMzWsHuWYZBSt8cOcJoqG+DZfxKzvp77oq8Vt711jiQaIX9/AE2Z5LiIN2ODInspohpZtj4MOqre+xJdPTYctMfL2cZfaGufR9E4hlvp/fUUlP97TcCV4IkeWQeHsTGZoPjRyh3L1k29w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=1MJQgoHc2RXthhjd/g/4xWmqUBrv08pWo3t2q2k3m3E=; b=l+ownWXfg3xEvQQAWRlasGpc/klSXGsGuvQ3vfJod1aTvNqDbVyL0FlW1JAoqbBOWUDzKWXl9ASTNU/48rJQRhI0MGKOSUTKpytVjERtRdaLBIWQybRxm5qD9F8l0TJZglQwG/i2mZz5FCA0Kxox+XysmlDyLR8aH0LVQT+6nsYGkKCnWO9rt/ruPd/TMt6bl1iKtN7SDrOviFdcoaQS5ciwBAArXEcepqT/Yc+VAwU3Zu0Y0ylIvkeW3S1s/QbLfz+AuaQhWZ392b0hufFyD75uhRLSrvXL1bYjbylvdvxmq3CxXMIlysVrsglU9vVwB1r9oDZ5RhUPVwjt5OCrNg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com; dkim=pass header.d=suse.com; arc=none Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=suse.com; Received: from HE1PR0402MB3497.eurprd04.prod.outlook.com (2603:10a6:7:83::14) by AM0PR0402MB3364.eurprd04.prod.outlook.com (2603:10a6:208:17::28) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5314.19; Mon, 6 Jun 2022 14:42:22 +0000 Received: from HE1PR0402MB3497.eurprd04.prod.outlook.com ([fe80::8002:50a5:a57a:d8fe]) by HE1PR0402MB3497.eurprd04.prod.outlook.com ([fe80::8002:50a5:a57a:d8fe%5]) with mapi id 15.20.5314.019; Mon, 6 Jun 2022 14:42:22 +0000 From: Geliang Tang To: mptcp@lists.linux.dev CC: Geliang Tang Subject: [PATCH mptcp-next v5 7/7] selftests/bpf: Add bpf_red test Date: Mon, 6 Jun 2022 22:41:22 +0800 Message-ID: <6a85c083498b77aa387de4e21c683b2230f0317f.1654526068.git.geliang.tang@suse.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: References: Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: SG2PR02CA0068.apcprd02.prod.outlook.com (2603:1096:4:54::32) To HE1PR0402MB3497.eurprd04.prod.outlook.com (2603:10a6:7:83::14) Precedence: bulk X-Mailing-List: mptcp@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 3210e297-553f-487a-3e8d-08da47cac420 X-MS-TrafficTypeDiagnostic: AM0PR0402MB3364:EE_ X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 6UzfcztExaMaoyMTFIVny/GS2ouaBh5URg381snrl4bbsc9K8Gb21iX9gOjI70kMBFA0/lxNKhuzqTSzn+YzTp/rU5N0n6IRq95Z6AwSBAnKHlRsco8aJr3PfgndLRxkzjr0M1hkMu5wPre6CQg+2bbjOne8GiUJ5l3m455E7kq901FrANzMQ78hvaSK1C5f5w0BMW2DmwWMKUNg2+Dl0L4oqtMwhv2XajmX3DutX2CVpFCZuVw6iw8ubFE23PbgW8qhIfeD1wWoGiX8TXEZZ+1CzCNWz0xpEDJPm0JrQp8c/Nr476wOvDI6cmQMdHGMCH8HVp/WCgYXrgbX5w58iDq+SwZ8T42drCfapJDuR1LULbVeFNG9HdjFpBwyultEzRTJJhp325ICxjOOJCPBLY4oDTMEV4dP7VjrFIi0ZSWmTe8jyOfeXyzIZTqJ6umVLzd31VZseOu8EoA5+c38MZy+sV5LJ83albLcgmfcFnBj5VtIQqd7PxhyQZfyEDLADVk1B2cfGh+AzJGdnc9iHfv4bhlSK77DdNzSceZ+pX3PSzRaruyiFVB4ZvVk9H15T/j+7KOuZ87d0q2XOIkNUbSD5sJoT+70cJv1FSmNjKBMFi+lBZ0Nbm7yIqrWE6WQ/TP92djmPgh8ePshsqezt/aoMiZyWlWADjknoodewbqAZmkjyng1c8pDZuwiQrlM X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:HE1PR0402MB3497.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230001)(366004)(6916009)(36756003)(8936002)(66476007)(38100700002)(107886003)(26005)(316002)(66946007)(5660300002)(6512007)(4326008)(66556008)(8676002)(83380400001)(44832011)(6506007)(2616005)(508600001)(6486002)(186003)(86362001)(2906002)(13296009);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?s+UWBmTOUCLy3U5vsbXfrH++jytPIYTRxtEnViMQikYOqZcFo8FfSvEBc/4b?= =?us-ascii?Q?B16cYTqMl9VANLhE5EGTxtK9ubh7US3nAv+0qwENQqD7HAK9qe02eUthjaXN?= =?us-ascii?Q?0nm1HALBYlA4Or5Z7tRx6FNj8GLEOzgB5LmB/kEN9+v6gZiGiEtfF5lm+VDE?= =?us-ascii?Q?IVVf4FEq0j5EmbJSex83Hvo6RbDX3FnRu32cohzcLiwXDXpo3k482jvwKpq8?= =?us-ascii?Q?r4Nm5dG+sDdavE30NMyI+Zn6e54V8tiIeZV2iPNsYkv63mLZBrtmv4SWuwJO?= =?us-ascii?Q?us7vVcyMbkNqTbeGMJfGZkchSb3GMEd4uc7hruv8b0l4QHtvaBVrLf9lCDsJ?= =?us-ascii?Q?eObUx3e/TpoSvkl+L3bbvTD+PR3a80lSmfCbHQ16vf8KRHrR/lfIjBoAOuxg?= =?us-ascii?Q?/z+gZ/gZ5QdqjqvRZxVwiSY/RJw/9OykXW1Trb38oj5DRvmF/xL/ThR3Shk+?= =?us-ascii?Q?iNBoRBasaVU/iH3+w+BtYcj/UPuJuYzJGf9xuLdDguuBEO9IWcjHubu/S1F1?= =?us-ascii?Q?koZyWOdnsvUCDT5qHB78b5c/HbR/PbI275I5VmS2vay9HFLNZUcAVVwdkS2/?= =?us-ascii?Q?r5DPc6q7Fy9IMpiOyCHV5QYeCYlC13lxCrH+42tGnAOuRq59waFjwrgs2yLU?= =?us-ascii?Q?ws7KBK5swkbzPxoyJjqI1oo2WoYY5HE/RpSG4SNK6BbhRbk8kQufCyVTFHZB?= =?us-ascii?Q?jPG+mrtDreO35Fhcs5QhkX+n8d0LrAcpXuut7gjft6x4fSBEUbvubVmEaeHu?= =?us-ascii?Q?QRH2GLh+pwKhH439tGFPVK/p/TLnTxwUV8VUcep7uLNyiLY7zUUl9C2w+2+a?= =?us-ascii?Q?KrErVf/i4ygTZf0/l73n00xFLQbpmm/eAL6W9TJ6o8feBV6WrW5OLxgTVlhk?= =?us-ascii?Q?Ly/gTaYeV1e3f/S3cpxu03uBtA3LV8gy9rSI4MFcOME4iTPWwxSqcZZZFX1f?= =?us-ascii?Q?7SdNXuUlVpEqbLBu44kGd/ueEffxfcILfAoj73LabQZllZkZ2KXg2yn3UFZ0?= =?us-ascii?Q?X+6ALZocLOaYqcxR7ZCSUwrYxcyLbWemPucBnvyepTs8tWqC6Zie4oij+fFZ?= =?us-ascii?Q?DVJHpVIpE0VGR8iizMuk8qnYfgGS5uGHve6srcDCLie+twfYDwqPXM5d1ZWk?= =?us-ascii?Q?2GKSfe17BcPGTxs/iij2csVPP19K6qry6NSvSOlG5pAuJxXqk86wf2ANEoRW?= =?us-ascii?Q?vLwPki024vjDjfe4jWhlSFpygAnWTQv4P+g21pjfZP1b0n4uUgNon9xRtyQB?= =?us-ascii?Q?6yujFeCNOiy1ohe6jjF2XwuV7f7fdQm82ArJOtC2nDF70+dFVLYsGbtpckXp?= =?us-ascii?Q?juZdq1DGYoA8F82kn5ERydTr3towG4m2NGbizmwcHk23I4bGpuy9fy/lix/2?= =?us-ascii?Q?S2ZmYKMtuENp8tewZLc+0YTu+rCnaPESjEB0KPTmIB7Ce3NhH+Loe8xhCO2G?= =?us-ascii?Q?bKIHoYY050d8zjYHv62cW+CGoKOiaSecT3awmu0n65R1AmfTHFC9S2iZbbLA?= =?us-ascii?Q?MY9g9vtXBEPeJ7T9n+JiM7QWnyS5rDMEfUadcNDdEInrV3bv3BlNPWR3ERLb?= =?us-ascii?Q?kdHe3d/sbiGdEAKLY3nJSVkHMcGIMHZF3lrwCbnkHuW1XKyf4+asEPpaTLR6?= =?us-ascii?Q?gsEZ4qPutSGb37cWvVjHzJYBF9Q8Qu+Xojz1WuNRxJH34Ep/qeJIUqi06XIr?= =?us-ascii?Q?wnnPk5S/7Hcx7g/q1vID5gBjM0SFA8au8Nb5tH2PiSt/Ed6zgZ0DAyCmCF9z?= =?us-ascii?Q?/jnbI88VyrOqz4fZ0bbbrKbX7AGBCkI=3D?= X-OriginatorOrg: suse.com X-MS-Exchange-CrossTenant-Network-Message-Id: 3210e297-553f-487a-3e8d-08da47cac420 X-MS-Exchange-CrossTenant-AuthSource: HE1PR0402MB3497.eurprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 06 Jun 2022 14:42:22.8103 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: Mpuw6tFwiuJsvRCBL1gLDYtIvkNzm1WoXedrr8h5JQ7wSUDKLV/z7dRfXut/z3s1ZFIs6BfPl3l9CXAXKWu1HQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM0PR0402MB3364 Content-Type: text/plain; charset="utf-8" This patch adds the redundant BPF MPTCP scheduler test: test_red(). Use sysctl to set net.mptcp.scheduler to use this sched. Add two veth net devices to simulate the multiple addresses case. Use 'ip mptcp endpoint' command to add the new endpoint ADDR_2 to PM netlink. Send data and check bytes_sent of 'ss' output after it to make sure the data has been redundantly sent on both net devices. Signed-off-by: Geliang Tang --- .../testing/selftests/bpf/prog_tests/mptcp.c | 34 +++++++++++++++++++ 1 file changed, 34 insertions(+) diff --git a/tools/testing/selftests/bpf/prog_tests/mptcp.c b/tools/testing= /selftests/bpf/prog_tests/mptcp.c index 8b2fe1004323..9f3242c7920a 100644 --- a/tools/testing/selftests/bpf/prog_tests/mptcp.c +++ b/tools/testing/selftests/bpf/prog_tests/mptcp.c @@ -9,6 +9,7 @@ #include "mptcp_bpf_first.skel.h" #include "mptcp_bpf_bkup.skel.h" #include "mptcp_bpf_rr.skel.h" +#include "mptcp_bpf_red.skel.h" =20 #ifndef TCP_CA_NAME_MAX #define TCP_CA_NAME_MAX 16 @@ -381,6 +382,37 @@ static void test_rr(void) mptcp_bpf_rr__destroy(rr_skel); } =20 +static void test_red(void) +{ + struct mptcp_bpf_red *red_skel; + int server_fd, client_fd; + struct bpf_link *link; + + red_skel =3D mptcp_bpf_red__open_and_load(); + if (!ASSERT_OK_PTR(red_skel, "bpf_red__open_and_load")) + return; + + link =3D bpf_map__attach_struct_ops(red_skel->maps.red); + if (!ASSERT_OK_PTR(link, "bpf_map__attach_struct_ops")) { + mptcp_bpf_red__destroy(red_skel); + return; + } + + sched_init("subflow", "bpf_red"); + server_fd =3D start_mptcp_server(AF_INET, ADDR_1, 0, 0); + client_fd =3D connect_to_fd(server_fd, 0); + + send_data(server_fd, client_fd); + ASSERT_OK(has_bytes_sent(ADDR_1), "has_bytes_sent addr 1"); + ASSERT_OK(has_bytes_sent(ADDR_2), "has_bytes_sent addr 2"); + + close(client_fd); + close(server_fd); + sched_cleanup(); + bpf_link__destroy(link); + mptcp_bpf_red__destroy(red_skel); +} + void test_mptcp(void) { if (test__start_subtest("base")) @@ -391,4 +423,6 @@ void test_mptcp(void) test_bkup(); if (test__start_subtest("rr")) test_rr(); + if (test__start_subtest("red")) + test_red(); } --=20 2.34.1