From nobody Tue Jun 23 04:04:26 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id CE488C433EF for ; Fri, 11 Mar 2022 03:02:29 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1345893AbiCKDD3 (ORCPT ); Thu, 10 Mar 2022 22:03:29 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37820 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234969AbiCKDDY (ORCPT ); Thu, 10 Mar 2022 22:03:24 -0500 Received: from smtp.tom.com (smtprz02.163.net [106.3.154.235]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id BE63FF65ED for ; Thu, 10 Mar 2022 19:02:22 -0800 (PST) Received: from localhost (localhost [127.0.0.1]) by vip-app02.163.net (Postfix) with ESMTP id 0A3F844017C for ; Fri, 11 Mar 2022 11:02:22 +0800 (CST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=tom.com; s=mail; t=1646967742; bh=ScMuHjAXfamD9Fmid4TX5rIBXmPUVWR9zJbXFToXGJQ=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=CtYK0p0wjAoQBZD17mfNjHOqKcrWYTtVv89I7EsySz4YZfAC18TLTaMlZn2kJwba4 ikH/BKGWD49vA/Hlo1jxt3aq7yU2OXO1CJHejrsPYvxIvqsROMEC9qaNoR6CgJRQAu Ldoe5lc3sPcCst5a7FQRwbjGDvtFj9wLpfhfHPCU= Received: from localhost (HELO smtp.tom.com) ([127.0.0.1]) by localhost (TOM SMTP Server) with SMTP ID -1056028449 for ; Fri, 11 Mar 2022 11:02:22 +0800 (CST) X-Virus-Scanned: Debian amavisd-new at mxtest.tom.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=tom.com; s=mail; t=1646967742; bh=ScMuHjAXfamD9Fmid4TX5rIBXmPUVWR9zJbXFToXGJQ=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=CtYK0p0wjAoQBZD17mfNjHOqKcrWYTtVv89I7EsySz4YZfAC18TLTaMlZn2kJwba4 ikH/BKGWD49vA/Hlo1jxt3aq7yU2OXO1CJHejrsPYvxIvqsROMEC9qaNoR6CgJRQAu Ldoe5lc3sPcCst5a7FQRwbjGDvtFj9wLpfhfHPCU= Received: from localhost.localdomain (unknown [101.93.196.13]) by antispamvip.163.net (Postfix) with ESMTPA id 4BA691541574; Fri, 11 Mar 2022 11:02:18 +0800 (CST) From: Mingbao Sun To: Keith Busch , Jens Axboe , Christoph Hellwig , Sagi Grimberg , Chaitanya Kulkarni , linux-nvme@lists.infradead.org, linux-kernel@vger.kernel.org, Eric Dumazet , "David S . Miller" , Hideaki YOSHIFUJI , David Ahern , Jakub Kicinski , netdev@vger.kernel.org Cc: sunmingbao@tom.com, tyler.sun@dell.com, ping.gan@dell.com, yanxiu.cai@dell.com, libin.zhang@dell.com, ao.sun@dell.com Subject: [PATCH 1/3] tcp: export symbol tcp_set_congestion_control Date: Fri, 11 Mar 2022 11:01:11 +0800 Message-Id: <20220311030113.73384-2-sunmingbao@tom.com> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20220311030113.73384-1-sunmingbao@tom.com> References: <20220311030113.73384-1-sunmingbao@tom.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org From: Mingbao Sun congestion-control could have a noticeable impaction on the performance of TCP-based communications. This is of course true to NVMe/TCP in the kernel. Different congestion-controls (e.g., cubic, dctcp) are suitable for different scenarios. Proper adoption of congestion control would benefit the performance. On the contrary, the performance could be destroyed. So to gain excellent performance against different network environments, NVMe/TCP tends to support specifying the congestion-control. This means NVMe/TCP (a kernel user) needs to set the congestion-control of its TCP sockets. Since the kernel API 'kernel_setsockopt' was removed, and since the function =E2=80=98tcp_set_congestion_control=E2=80=99 is just the real unde= rlying guy handling this job, so it makes sense to get it exported. Signed-off-by: Mingbao Sun Reviewed-by: Christoph Hellwig --- net/ipv4/tcp_cong.c | 1 + 1 file changed, 1 insertion(+) diff --git a/net/ipv4/tcp_cong.c b/net/ipv4/tcp_cong.c index db5831e6c136..5d77f3e7278e 100644 --- a/net/ipv4/tcp_cong.c +++ b/net/ipv4/tcp_cong.c @@ -383,6 +383,7 @@ int tcp_set_congestion_control(struct sock *sk, const c= har *name, bool load, rcu_read_unlock(); return err; } +EXPORT_SYMBOL_GPL(tcp_set_congestion_control); =20 /* Slow start is used when congestion window is no greater than the slow s= tart * threshold. We base on RFC2581 and also handle stretch ACKs properly. --=20 2.26.2 From nobody Tue Jun 23 04:04:26 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id DCDD3C433FE for ; Fri, 11 Mar 2022 03:02:35 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1345897AbiCKDDg (ORCPT ); Thu, 10 Mar 2022 22:03:36 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38068 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1345896AbiCKDDd (ORCPT ); Thu, 10 Mar 2022 22:03:33 -0500 Received: from smtp.tom.com (smtprz02.163.net [106.3.154.235]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id 101F1C12E0 for ; Thu, 10 Mar 2022 19:02:26 -0800 (PST) Received: from localhost (localhost [127.0.0.1]) by vip-app02.163.net (Postfix) with ESMTP id 42D90440164 for ; Fri, 11 Mar 2022 11:02:26 +0800 (CST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=tom.com; s=mail; t=1646967746; bh=nLgk51poUCRUSZi9wNWoz/pgzjUVdty8YV18a/uDvxc=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Z2gcxj/qpRx7YdsZCe8ljz2trLrBja9qgr1aR9GtR63QP1Cup/Nuqbi0GYTFrZtl8 kB+wb0J/j7aT+A5CxDxjEGOwdlQnBdaSN2BgCuvQHIf6oe12TkLtbQgXKii/P3B0w4 a9rtZFJ6tbYooE6SE7HKhDp0ywrVIL2/S59IHA2o= Received: from localhost (HELO smtp.tom.com) ([127.0.0.1]) by localhost (TOM SMTP Server) with SMTP ID -1193079104 for ; Fri, 11 Mar 2022 11:02:26 +0800 (CST) X-Virus-Scanned: Debian amavisd-new at mxtest.tom.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=tom.com; s=mail; t=1646967746; bh=nLgk51poUCRUSZi9wNWoz/pgzjUVdty8YV18a/uDvxc=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Z2gcxj/qpRx7YdsZCe8ljz2trLrBja9qgr1aR9GtR63QP1Cup/Nuqbi0GYTFrZtl8 kB+wb0J/j7aT+A5CxDxjEGOwdlQnBdaSN2BgCuvQHIf6oe12TkLtbQgXKii/P3B0w4 a9rtZFJ6tbYooE6SE7HKhDp0ywrVIL2/S59IHA2o= Received: from localhost.localdomain (unknown [101.93.196.13]) by antispamvip.163.net (Postfix) with ESMTPA id 45AA215414F8; Fri, 11 Mar 2022 11:02:22 +0800 (CST) From: Mingbao Sun To: Keith Busch , Jens Axboe , Christoph Hellwig , Sagi Grimberg , Chaitanya Kulkarni , linux-nvme@lists.infradead.org, linux-kernel@vger.kernel.org, Eric Dumazet , "David S . Miller" , Hideaki YOSHIFUJI , David Ahern , Jakub Kicinski , netdev@vger.kernel.org Cc: sunmingbao@tom.com, tyler.sun@dell.com, ping.gan@dell.com, yanxiu.cai@dell.com, libin.zhang@dell.com, ao.sun@dell.com Subject: [PATCH 2/3] nvme-tcp: support specifying the congestion-control Date: Fri, 11 Mar 2022 11:01:12 +0800 Message-Id: <20220311030113.73384-3-sunmingbao@tom.com> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20220311030113.73384-1-sunmingbao@tom.com> References: <20220311030113.73384-1-sunmingbao@tom.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" From: Mingbao Sun congestion-control could have a noticeable impaction on the performance of TCP-based communications. This is of course true to NVMe_over_TCP. Different congestion-controls (e.g., cubic, dctcp) are suitable for different scenarios. Proper adoption of congestion control would benefit the performance. On the contrary, the performance could be destroyed. Though we can specify the congestion-control of NVMe_over_TCP via writing '/proc/sys/net/ipv4/tcp_congestion_control', but this also changes the congestion-control of all the future TCP sockets that have not been explicitly assigned the congestion-control, thus bringing potential impaction on their performance. So it makes sense to make NVMe_over_TCP support specifying the congestion-control. And this commit addresses the host side. Implementation approach: a new option called 'tcp_congestion' was created in fabrics opt_tokens for 'nvme connect' command to passed in the congestion-control specified by the user. Then later in nvme_tcp_alloc_queue, the specified congestion-control would be applied to the relevant sockets of the host side. Signed-off-by: Mingbao Sun --- drivers/nvme/host/fabrics.c | 12 ++++++++++++ drivers/nvme/host/fabrics.h | 2 ++ drivers/nvme/host/tcp.c | 15 ++++++++++++++- 3 files changed, 28 insertions(+), 1 deletion(-) diff --git a/drivers/nvme/host/fabrics.c b/drivers/nvme/host/fabrics.c index ee79a6d639b4..79d5f0dbafd3 100644 --- a/drivers/nvme/host/fabrics.c +++ b/drivers/nvme/host/fabrics.c @@ -548,6 +548,7 @@ static const match_table_t opt_tokens =3D { { NVMF_OPT_TOS, "tos=3D%d" }, { NVMF_OPT_FAIL_FAST_TMO, "fast_io_fail_tmo=3D%d" }, { NVMF_OPT_DISCOVERY, "discovery" }, + { NVMF_OPT_TCP_CONGESTION, "tcp_congestion=3D%s" }, { NVMF_OPT_ERR, NULL } }; =20 @@ -829,6 +830,16 @@ static int nvmf_parse_options(struct nvmf_ctrl_options= *opts, case NVMF_OPT_DISCOVERY: opts->discovery_nqn =3D true; break; + case NVMF_OPT_TCP_CONGESTION: + p =3D match_strdup(args); + if (!p) { + ret =3D -ENOMEM; + goto out; + } + + kfree(opts->tcp_congestion); + opts->tcp_congestion =3D p; + break; default: pr_warn("unknown parameter or missing value '%s' in ctrl creation reque= st\n", p); @@ -947,6 +958,7 @@ void nvmf_free_options(struct nvmf_ctrl_options *opts) kfree(opts->subsysnqn); kfree(opts->host_traddr); kfree(opts->host_iface); + kfree(opts->tcp_congestion); kfree(opts); } EXPORT_SYMBOL_GPL(nvmf_free_options); diff --git a/drivers/nvme/host/fabrics.h b/drivers/nvme/host/fabrics.h index c3203ff1c654..25fdc169949d 100644 --- a/drivers/nvme/host/fabrics.h +++ b/drivers/nvme/host/fabrics.h @@ -68,6 +68,7 @@ enum { NVMF_OPT_FAIL_FAST_TMO =3D 1 << 20, NVMF_OPT_HOST_IFACE =3D 1 << 21, NVMF_OPT_DISCOVERY =3D 1 << 22, + NVMF_OPT_TCP_CONGESTION =3D 1 << 23, }; =20 /** @@ -117,6 +118,7 @@ struct nvmf_ctrl_options { unsigned int nr_io_queues; unsigned int reconnect_delay; bool discovery_nqn; + const char *tcp_congestion; bool duplicate_connect; unsigned int kato; struct nvmf_host *host; diff --git a/drivers/nvme/host/tcp.c b/drivers/nvme/host/tcp.c index 10fc45d95b86..f2a6df35374a 100644 --- a/drivers/nvme/host/tcp.c +++ b/drivers/nvme/host/tcp.c @@ -1487,6 +1487,18 @@ static int nvme_tcp_alloc_queue(struct nvme_ctrl *nc= trl, if (nctrl->opts->tos >=3D 0) ip_sock_set_tos(queue->sock->sk, nctrl->opts->tos); =20 + if (nctrl->opts->mask & NVMF_OPT_TCP_CONGESTION) { + ret =3D tcp_set_congestion_control(queue->sock->sk, + nctrl->opts->tcp_congestion, + true, true); + if (ret) { + dev_err(nctrl->device, + "failed to set TCP congestion to %s: %d\n", + nctrl->opts->tcp_congestion, ret); + goto err_sock; + } + } + /* Set 10 seconds timeout for icresp recvmsg */ queue->sock->sk->sk_rcvtimeo =3D 10 * HZ; =20 @@ -2650,7 +2662,8 @@ static struct nvmf_transport_ops nvme_tcp_transport = =3D { NVMF_OPT_HOST_TRADDR | NVMF_OPT_CTRL_LOSS_TMO | NVMF_OPT_HDR_DIGEST | NVMF_OPT_DATA_DIGEST | NVMF_OPT_NR_WRITE_QUEUES | NVMF_OPT_NR_POLL_QUEUES | - NVMF_OPT_TOS | NVMF_OPT_HOST_IFACE, + NVMF_OPT_TOS | NVMF_OPT_HOST_IFACE | + NVMF_OPT_TCP_CONGESTION, .create_ctrl =3D nvme_tcp_create_ctrl, }; =20 --=20 2.26.2 From nobody Tue Jun 23 04:04:26 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id B9B8CC433FE for ; Fri, 11 Mar 2022 03:02:38 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1345920AbiCKDDj (ORCPT ); Thu, 10 Mar 2022 22:03:39 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38180 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1345912AbiCKDDe (ORCPT ); Thu, 10 Mar 2022 22:03:34 -0500 Received: from smtp.tom.com (smtprz02.163.net [106.3.154.235]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id 169122610 for ; Thu, 10 Mar 2022 19:02:29 -0800 (PST) Received: from localhost (localhost [127.0.0.1]) by vip-app02.163.net (Postfix) with ESMTP id 3CACD44017A for ; Fri, 11 Mar 2022 11:02:29 +0800 (CST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=tom.com; s=mail; t=1646967749; bh=t7WJRPIus0dQzi8s/xkccP/qagUo8vwUmNLGEBagdNw=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=KVz/qPC4RA5hBu6pNftnA1v2BJHh7Kn7EU8MZotGbfa6NypPZ5ZLu8mOFbcR5Cbex xUqNiKJ8KJl9cF+VAw2s9XBL89ecv6/7YrW+v1N5RqHCWs/LW9xsQ/V25MiNtm9l11 Cj3lS6CcwbrA9wlX/6QAnhImAOOSHXHxa84lPhng= Received: from localhost (HELO smtp.tom.com) ([127.0.0.1]) by localhost (TOM SMTP Server) with SMTP ID 35945974 for ; Fri, 11 Mar 2022 11:02:29 +0800 (CST) X-Virus-Scanned: Debian amavisd-new at mxtest.tom.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=tom.com; s=mail; t=1646967749; bh=t7WJRPIus0dQzi8s/xkccP/qagUo8vwUmNLGEBagdNw=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=KVz/qPC4RA5hBu6pNftnA1v2BJHh7Kn7EU8MZotGbfa6NypPZ5ZLu8mOFbcR5Cbex xUqNiKJ8KJl9cF+VAw2s9XBL89ecv6/7YrW+v1N5RqHCWs/LW9xsQ/V25MiNtm9l11 Cj3lS6CcwbrA9wlX/6QAnhImAOOSHXHxa84lPhng= Received: from localhost.localdomain (unknown [101.93.196.13]) by antispamvip.163.net (Postfix) with ESMTPA id 8196E15415A8; Fri, 11 Mar 2022 11:02:25 +0800 (CST) From: Mingbao Sun To: Keith Busch , Jens Axboe , Christoph Hellwig , Sagi Grimberg , Chaitanya Kulkarni , linux-nvme@lists.infradead.org, linux-kernel@vger.kernel.org, Eric Dumazet , "David S . Miller" , Hideaki YOSHIFUJI , David Ahern , Jakub Kicinski , netdev@vger.kernel.org Cc: sunmingbao@tom.com, tyler.sun@dell.com, ping.gan@dell.com, yanxiu.cai@dell.com, libin.zhang@dell.com, ao.sun@dell.com Subject: [PATCH 3/3] nvmet-tcp: support specifying the congestion-control Date: Fri, 11 Mar 2022 11:01:13 +0800 Message-Id: <20220311030113.73384-4-sunmingbao@tom.com> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20220311030113.73384-1-sunmingbao@tom.com> References: <20220311030113.73384-1-sunmingbao@tom.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" From: Mingbao Sun congestion-control could have a noticeable impaction on the performance of TCP-based communications. This is of course true to NVMe_over_TCP. Different congestion-controls (e.g., cubic, dctcp) are suitable for different scenarios. Proper adoption of congestion control would benefit the performance. On the contrary, the performance could be destroyed. Though we can specify the congestion-control of NVMe_over_TCP via writing '/proc/sys/net/ipv4/tcp_congestion_control', but this also changes the congestion-control of all the future TCP sockets that have not been explicitly assigned the congestion-control, thus bringing potential impaction on their performance. So it makes sense to make NVMe_over_TCP support specifying the congestion-control. And this commit addresses the target side. Implementation approach: the following new file entry was created for user to specify the congestion-control of each nvmet port. '/sys/kernel/config/nvmet/ports/X/tcp_congestion' Then later in nvmet_tcp_add_port, the specified congestion-control would be applied to the listening socket of the nvmet port. Signed-off-by: Mingbao Sun --- drivers/nvme/target/configfs.c | 37 ++++++++++++++++++++++++++++++++++ drivers/nvme/target/nvmet.h | 1 + drivers/nvme/target/tcp.c | 11 ++++++++++ 3 files changed, 49 insertions(+) diff --git a/drivers/nvme/target/configfs.c b/drivers/nvme/target/configfs.c index 091a0ca16361..644e89bb0ee9 100644 --- a/drivers/nvme/target/configfs.c +++ b/drivers/nvme/target/configfs.c @@ -222,6 +222,41 @@ static ssize_t nvmet_addr_trsvcid_store(struct config_= item *item, =20 CONFIGFS_ATTR(nvmet_, addr_trsvcid); =20 +static ssize_t nvmet_tcp_congestion_show(struct config_item *item, + char *page) +{ + struct nvmet_port *port =3D to_nvmet_port(item); + + return snprintf(page, PAGE_SIZE, "%s\n", + port->tcp_congestion ? port->tcp_congestion : ""); +} + +static ssize_t nvmet_tcp_congestion_store(struct config_item *item, + const char *page, size_t count) +{ + struct nvmet_port *port =3D to_nvmet_port(item); + int len; + char *buf; + + len =3D strcspn(page, "\n"); + if (!len) + return -EINVAL; + + if (nvmet_is_port_enabled(port, __func__)) + return -EACCES; + + buf =3D kmemdup_nul(page, len, GFP_KERNEL); + if (!buf) + return -ENOMEM; + + kfree(port->tcp_congestion); + port->tcp_congestion =3D buf; + + return count; +} + +CONFIGFS_ATTR(nvmet_, tcp_congestion); + static ssize_t nvmet_param_inline_data_size_show(struct config_item *item, char *page) { @@ -1597,6 +1632,7 @@ static void nvmet_port_release(struct config_item *it= em) list_del(&port->global_entry); =20 kfree(port->ana_state); + kfree(port->tcp_congestion); kfree(port); } =20 @@ -1605,6 +1641,7 @@ static struct configfs_attribute *nvmet_port_attrs[] = =3D { &nvmet_attr_addr_treq, &nvmet_attr_addr_traddr, &nvmet_attr_addr_trsvcid, + &nvmet_attr_tcp_congestion, &nvmet_attr_addr_trtype, &nvmet_attr_param_inline_data_size, #ifdef CONFIG_BLK_DEV_INTEGRITY diff --git a/drivers/nvme/target/nvmet.h b/drivers/nvme/target/nvmet.h index 69637bf8f8e1..76a57c4c3456 100644 --- a/drivers/nvme/target/nvmet.h +++ b/drivers/nvme/target/nvmet.h @@ -145,6 +145,7 @@ struct nvmet_port { struct config_group ana_groups_group; struct nvmet_ana_group ana_default_group; enum nvme_ana_state *ana_state; + const char *tcp_congestion; void *priv; bool enabled; int inline_data_size; diff --git a/drivers/nvme/target/tcp.c b/drivers/nvme/target/tcp.c index 83ca577f72be..489c46e396b9 100644 --- a/drivers/nvme/target/tcp.c +++ b/drivers/nvme/target/tcp.c @@ -1741,6 +1741,17 @@ static int nvmet_tcp_add_port(struct nvmet_port *npo= rt) if (so_priority > 0) sock_set_priority(port->sock->sk, so_priority); =20 + if (nport->tcp_congestion) { + ret =3D tcp_set_congestion_control(port->sock->sk, + nport->tcp_congestion, + true, true); + if (ret) { + pr_err("failed to set port socket's congestion to %s: %d\n", + nport->tcp_congestion, ret); + goto err_sock; + } + } + ret =3D kernel_bind(port->sock, (struct sockaddr *)&port->addr, sizeof(port->addr)); if (ret) { --=20 2.26.2