From nobody Sat Oct 24 20:10:35 2026 Received: from DM5PR21CU001.outbound.protection.outlook.com (mail-centralusazon11011014.outbound.protection.outlook.com [52.101.62.14]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 159CD2BEC2A; Sun, 11 Oct 2026 13:33:31 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=52.101.62.14 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1791725614; cv=fail; b=eT3p79hlFvarPF/AmR3OfGd/QEAuzgUc62Kjsdn49EtS1iTAAANT43KYTdXJu/zvoeyVUEpl17kGJwSRqN2+2EcTkk1NCiDZvD9o2Y0kp0uID1kbbTYVh6J0CtorWcADuYC/rPtSdhZZIOy9Vs5bX4ZamdOs0UyYUX6TkFeArrA= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1791725614; c=relaxed/simple; bh=KJqb0kxIVDJGIdOtSo1lJCc3+8OmSMWzfk3x1lLTyRU=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=Z9JaQaicHw9dRvmVguSMzs9SmMWK/JYegEOlHvzIZlf8myIKwUz1Eu0Sisjhv5h0vMK85sNoDpb949NdbfGjc4mY31POy4l18BwQ0ilCIGUDwFrb3NG5BX7x8JEZj7qSoKr4YqvaeaKY19eJvtZrTF+CHb/2h/9GsYVRI9xpNX4= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=Lxh/hIXU; arc=fail smtp.client-ip=52.101.62.14 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="Lxh/hIXU" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=Kkq/dyVWXWYZBtURIRUebR7wB+kPk9KH6qQzET/shDx38iIfIZGcEQPej+npS1o3V29LTLEc0T0e8rda7G70741VP9D3xyGfZpyyzulf2ROawHC7PUIL2es87QUhzB7cQDfWvNq8yaLmr/CAGPF/SZqI53iNHTGoYdciMcO9Tk2Ytw/hdOZGRT8JqGlZaqqa0SgyMxG30suGgnGaqYo/Nj+Qlo/mkcxo6YsC743tvwKzaBoaqU+RyvyOAje5SWDAefSUvNvPyqTw4/yUux/z1Bo6hJBxTYk/2uSPePQEhQcDuaxWz95eaIkci7HImEZIrRcIoxMchk7g8UTLSnLigQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=plxIjL2f6Uwl2oo1f7j5zwXDHL5tCrCCR45m7CTBVsw=; b=T0E3QrquiH1GJKDb5Tl64P/F0qA7NRHRl2U5gW/GZzzwWc6z9XUjjl9xCIGoEUBZOhfLUjd9f4pkfrLFatH2iiHhlnDkX4Vf1V4wSnLOLEzizrsM11cGlFr6vwCGSmVjsN2/KB+hU776zgwy3fLCX6lO3FBhDDZ8ZXCZu9oClbIr0w90e917P/ZIAiUtmbpQ7Hb7+n2/GDlhJXCGrdt0aUTwSVV+xbfPjsiL7/witSCghAg2c1n861qoJjup017teSV5uD+8AwXKWFB7b2YugPUMzh5NQPiFKkCpWVy5kX8K2w4bT0HBq4ChndmbGHaCHTxA9cP41VIDv2zvlwLfIg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.160) smtp.rcpttodomain=lunn.ch smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=plxIjL2f6Uwl2oo1f7j5zwXDHL5tCrCCR45m7CTBVsw=; b=Lxh/hIXUKQTZRs8Ecyx2A64mHdonUlIdP5JeyBjyOKimrK56SO7fCSMti+MrlG2wmZCb3bAzNDXQJQi/gmLDPLru89A4pZpWtDI7+szfDqwz4KTONAyHCCq89MlLrOdD41O8IEB/PVgv5tLdoiyoPnl1dEkedSbic44kWpnPevSCbO/STz4TtdORbLJxKYuFjbDRhVKE/6GovWkaUvmD+jJFFfS1MgY4lUI52B02HMFujZVbo5Qh9Pg2KiP0MuiIPl1nmuye7F/BK/K83x5jbwc8PJkg4+atGLm5abAq9tG2/OnJEoDuHHlX1jJT6Xfe6MoQu9TZdIE6VCaBT+285g== Received: from SJ0PR05CA0142.namprd05.prod.outlook.com (2603:10b6:a03:33d::27) by DS5PPF5E0E7945E.namprd12.prod.outlook.com (2603:10b6:f:fc00::650) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.496.16; Sun, 11 Oct 2026 13:33:23 +0000 Received: from MWH0EPF000A672F.namprd04.prod.outlook.com (2603:10b6:a03:33d:cafe::67) by SJ0PR05CA0142.outlook.office365.com (2603:10b6:a03:33d::27) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.21.522.4 via Frontend Transport; Sun, 11 Oct 2026 13:33:23 +0000 X-MS-Exchange-Authentication-Results: mx.microsoft.com 1; spf=pass (sender IP is 216.228.117.160) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.160 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.160; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.160) by MWH0EPF000A672F.mail.protection.outlook.com (10.167.249.21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.522.6 via Frontend Transport; Sun, 11 Oct 2026 13:33:22 +0000 Received: from rnnvmail203.nvidia.com (10.129.68.9) by mail.nvidia.com (10.129.200.66) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.49; Sun, 11 Oct 2026 06:33:10 -0700 Received: from rnnvmail205.nvidia.com (10.129.68.10) by rnnvmail203.nvidia.com (10.129.68.9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.49; Sun, 11 Oct 2026 06:33:09 -0700 Received: from vdi.nvidia.com (10.127.8.10) by mail.nvidia.com (10.129.68.10) with Microsoft SMTP Server id 15.2.2562.49 via Frontend Transport; Sun, 11 Oct 2026 06:33:02 -0700 From: Tariq Toukan To: Andrew Lunn , "David S. Miller" , Eric Dumazet , Jakub Kicinski , , Paolo Abeni CC: Alexei Starovoitov , , "Carolina Jubran" , Cosmin Ratiu , "Daniel Borkmann" , Dragos Tatulea , "Gal Pressman" , Jesper Dangaard Brouer , "John Fastabend" , Kees Cook , "Leon Romanovsky" , open list , , Li RongQing , Mark Bloch , Richard Cochran , "Saeed Mahameed" , Shahar Shitrit , Stanislav Fomichev , Tariq Toukan , =?UTF-8?q?Toke=20H=C3=B8iland-J=C3=B8rgensen?= , =?UTF-8?q?Toke=20H=C3=B8iland-J=C3=B8rgensen?= , "Adrian Moreno" , Michal Schmidt Subject: [PATCH net-next 1/2] net/mlx5e: Batch NAPI RCU grace periods on channel close Date: Sun, 11 Oct 2026 16:32:11 +0300 Message-ID: <20261011133212.807979-2-tariqt@nvidia.com> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20261011133212.807979-1-tariqt@nvidia.com> References: <20261011133212.807979-1-tariqt@nvidia.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-NV-OnPremToCloud: ExternallySecured X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: MWH0EPF000A672F:EE_|DS5PPF5E0E7945E:EE_ X-MS-Office365-Filtering-Correlation-Id: ab4cb91b-af7c-47a3-5488-08df279c38c7 X-LD-Processed: 43083d15-7273-40c1-b7db-39efd9ccc17a,ExtAddr X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|82310400026|376014|7416014|23010399003|1800799024|36860700016|6133799003|18002099003|22082099003|3023799007|11063799006|10067099003|5023799004|56012099006|261009223027099003; X-Microsoft-Antispam-Message-Info: 6vJd0ZkKE73M9ms0mU2He64y68m7I4c2BFHvy7kP4cME24ai2C9rT4AKEO6xj4RaXbQ7srEIIyqAK428UyVszf8FYhY/+RjLYjXHKrzBC41HvPrAWu7BisRQtdDW7GQWBXw35Wre4SnLvym8UOg9iTPOoq1Ud9QjNT3zf6uftOalXjrK2+XQ7Dm4LJ4HnPnJALU2LtSUvIb8G9fJmly1c98ZxGQ078XNFqLwP0eccSouXYjejsl6DX4DlRkMIt9JEAxfo5D/NkEesAruvdA+wTStVuzPYstaHptBodvJFVvIa5eqfuC4NR+74TdWHtzLJpd7VNFLvUdCjsHkicoiqx5EUjL4azLMx3ntAFPspbFYUAlCRGd5oYCKGosW3335/ImIt6nREI582YzEmRbc9NGfbzTFtjNR49KYdxu4zx5Z7kea8Q+VL48LCamMmAK8wo+9ZUk8kKgmIDHKdXnnwLAszwrrQe8KMoAmc5LJuo597tiYK13kRUurFLh6t9bUpQlKrUywWpkxFPXoVrLEy71q4mjGkN2w6o1bZlOhBW9L/aX++WH3IXbu9TLHoL8zifiTlQfQoAr6E8fLSAlILUItW6XWcFmBf9wRcIAqdyn7tk8aWJ9YYrIFrqdtwo3qw8sd88GZEkYPdUguIqqLAXhXoRf7WcnCY88k5ULgxMhsRLb2C9WeMw6Sl5MMQ85tcWcHnXgZc3gL54OG/D16WA== X-Forefront-Antispam-Report: CIP:216.228.117.160;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:dc6edge1.nvidia.com;CAT:NONE;SFS:(13230040)(82310400026)(376014)(7416014)(23010399003)(1800799024)(36860700016)(6133799003)(18002099003)(22082099003)(3023799007)(11063799006)(10067099003)(5023799004)(56012099006)(261009223027099003);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: PVysFxw/Yz0rc45Z6P3Io7cpFLOH4cP8UtDToe7yUYuCvgL7X31KUl5cu9sYtsxjxJ9ZxvCG1/ulsL5VHDNoDzoG29quOHpjdr/q3LulkvK9wVHFhfSp1ndND4jqxwbxcTX/UOig+MWYvL+JEZ5N7RzXuQT6clD95iXqMRk+iomNZm++w/pR7TQmwJm25pLHikvkEBhN6FndG78xNJ0Kbum8F2Kr1OMrqrDFvOkJbTWRNCvciQu0/hSHGIGLM30CTPWdfTGx0g+OXh9CHcUB0cPfXXVI2LYK2KlO3PSi6J4r1DSJHRgzZWnO5wInvjP7h88EdcBevDQSeJzlupY/lKyxVPtYQPTnqpamRbZcrCTyEQSdSbozInGtNBkq0fAGFLepZZTgTfrIsrmiAfP0SCMlbtA4ZCDWuy4qRgKU4L0Gz2uvTUBrzu5plFeIDn/3 X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 11 Oct 2026 13:33:22.9502 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: ab4cb91b-af7c-47a3-5488-08df279c38c7 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.117.160];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: MWH0EPF000A672F.namprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DS5PPF5E0E7945E Content-Type: text/plain; charset="utf-8" From: Carolina Jubran Channel close currently invokes synchronize_net() repeatedly while tearing down each channel. RQ, ICOSQ, TX SQ and NAPI teardown each require a grace period, causing the number of waits to scale with the number of channels. TX SQ teardown adds a wait for each traffic class, while XDP and QoS can add further calls. Batch these waits across all channels. Split channel deactivate and close into pre_sync and post_sync phases. First clear the ENABLED state on all queues, then wait for a single grace period before continuing with the teardown. Similarly, unlink all NAPI instances with __netif_napi_del_locked(), issue one synchronize_net() after all of them have been unlinked, and only then free the channels. Keep the existing per object synchronize_net() calls for single queue paths. 248 channels, rcu_normal=3D1, ip link set eth2 down: Before: 10.183s After: 0.715s (~14x faster) Signed-off-by: Carolina Jubran Signed-off-by: Tariq Toukan Acked-by: Toke H=C3=B8iland-J=C3=B8rgensen --- drivers/net/ethernet/mellanox/mlx5/core/en.h | 5 + .../net/ethernet/mellanox/mlx5/core/en/ptp.c | 19 +- .../net/ethernet/mellanox/mlx5/core/en/ptp.h | 4 +- .../net/ethernet/mellanox/mlx5/core/en/qos.c | 69 +++++- .../net/ethernet/mellanox/mlx5/core/en/qos.h | 4 +- .../net/ethernet/mellanox/mlx5/core/en/xdp.h | 4 +- .../mellanox/mlx5/core/en/xsk/setup.c | 27 ++- .../mellanox/mlx5/core/en/xsk/setup.h | 3 + .../net/ethernet/mellanox/mlx5/core/en_main.c | 204 +++++++++++++++--- 9 files changed, 282 insertions(+), 57 deletions(-) diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en.h b/drivers/net/eth= ernet/mellanox/mlx5/core/en.h index a3945195f454..1a169fce075a 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en.h +++ b/drivers/net/ethernet/mellanox/mlx5/core/en.h @@ -1103,6 +1103,7 @@ int mlx5e_open_xdpsq(struct mlx5e_channel *c, struct = mlx5e_params *params, struct mlx5e_sq_param *param, struct xsk_buff_pool *xsk_pool, struct mlx5e_xdpsq *sq, bool is_redirect); void mlx5e_close_xdpsq(struct mlx5e_xdpsq *sq); +void mlx5e_close_xdpsq_post_sync(struct mlx5e_xdpsq *sq); =20 struct mlx5e_create_cq_param { struct net_device *netdev; @@ -1159,8 +1160,10 @@ int mlx5e_ptp_rx_manage_fs_ctx(struct mlx5e_priv *pr= iv, void *ctx); int mlx5e_flush_rq(struct mlx5e_rq *rq, int curr_state); void mlx5e_activate_rq(struct mlx5e_rq *rq); void mlx5e_deactivate_rq(struct mlx5e_rq *rq); +void mlx5e_deactivate_rq_pre_sync(struct mlx5e_rq *rq); void mlx5e_activate_icosq(struct mlx5e_icosq *icosq); void mlx5e_deactivate_icosq(struct mlx5e_icosq *icosq); +void mlx5e_deactivate_icosq_pre_sync(struct mlx5e_icosq *icosq); =20 int mlx5e_modify_sq(struct mlx5_core_dev *mdev, u32 sqn, struct mlx5e_modify_sq_param *p); @@ -1170,6 +1173,8 @@ int mlx5e_open_txqsq(struct mlx5e_channel *c, u32 tis= n, int txq_ix, struct mlx5e_sq_stats *sq_stats); void mlx5e_activate_txqsq(struct mlx5e_txqsq *sq); void mlx5e_deactivate_txqsq(struct mlx5e_txqsq *sq); +void mlx5e_deactivate_txqsq_pre_sync(struct mlx5e_txqsq *sq); +void mlx5e_deactivate_txqsq_post_sync(struct mlx5e_txqsq *sq); void mlx5e_free_txqsq(struct mlx5e_txqsq *sq); void mlx5e_tx_disable_queue(struct netdev_queue *txq); int mlx5e_alloc_txqsq_db(struct mlx5e_txqsq *sq, int numa); diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en/ptp.c b/drivers/net= /ethernet/mellanox/mlx5/core/en/ptp.c index 17084d76209e..3fa629228878 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en/ptp.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/en/ptp.c @@ -775,7 +775,7 @@ static int mlx5e_ptp_open_queues(struct mlx5e_ptp *c, return err; } =20 -static void mlx5e_ptp_close_queues(struct mlx5e_ptp *c) +void mlx5e_ptp_close_queues(struct mlx5e_ptp *c) { if (test_bit(MLX5E_PTP_STATE_RX, c->state)) { mlx5e_close_rq(&c->rq); @@ -932,7 +932,6 @@ void mlx5e_ptp_close(struct mlx5e_ptp *c) { mlx5e_ptp_close_queues(c); netif_napi_del_locked(&c->napi); - kvfree(c); } =20 @@ -954,18 +953,28 @@ void mlx5e_ptp_activate_channel(struct mlx5e_ptp *c) mlx5e_trigger_napi_sched(&c->napi); } =20 -void mlx5e_ptp_deactivate_channel(struct mlx5e_ptp *c) +void mlx5e_ptp_deactivate_channel_pre_sync(struct mlx5e_ptp *c) { int tc; =20 if (test_bit(MLX5E_PTP_STATE_RX, c->state)) { netif_queue_set_napi(c->netdev, c->rq.ix, NETDEV_QUEUE_TYPE_RX, NULL); - mlx5e_deactivate_rq(&c->rq); + mlx5e_deactivate_rq_pre_sync(&c->rq); + } + + if (test_bit(MLX5E_PTP_STATE_TX, c->state)) { + for (tc =3D 0; tc < c->num_tc; tc++) + mlx5e_deactivate_txqsq_pre_sync(&c->ptpsq[tc].txqsq); } +} + +void mlx5e_ptp_deactivate_channel_post_sync(struct mlx5e_ptp *c) +{ + int tc; =20 if (test_bit(MLX5E_PTP_STATE_TX, c->state)) { for (tc =3D 0; tc < c->num_tc; tc++) - mlx5e_deactivate_txqsq(&c->ptpsq[tc].txqsq); + mlx5e_deactivate_txqsq_post_sync(&c->ptpsq[tc].txqsq); } =20 napi_disable_locked(&c->napi); diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en/ptp.h b/drivers/net= /ethernet/mellanox/mlx5/core/en/ptp.h index 2a457a2ed707..b42ce33635a0 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en/ptp.h +++ b/drivers/net/ethernet/mellanox/mlx5/core/en/ptp.h @@ -129,8 +129,10 @@ static inline bool mlx5e_ptpsq_metadata_freelist_empty= (struct mlx5e_ptpsq *ptpsq int mlx5e_ptp_open(struct mlx5e_priv *priv, struct mlx5e_params *params, u8 lag_port, struct mlx5e_ptp **cp); void mlx5e_ptp_close(struct mlx5e_ptp *c); +void mlx5e_ptp_close_queues(struct mlx5e_ptp *c); void mlx5e_ptp_activate_channel(struct mlx5e_ptp *c); -void mlx5e_ptp_deactivate_channel(struct mlx5e_ptp *c); +void mlx5e_ptp_deactivate_channel_pre_sync(struct mlx5e_ptp *c); +void mlx5e_ptp_deactivate_channel_post_sync(struct mlx5e_ptp *c); int mlx5e_ptp_get_rqn(struct mlx5e_ptp *c, u32 *rqn); int mlx5e_ptp_alloc_rx_fs(struct mlx5e_flow_steering *fs, const struct mlx5e_profile *profile); diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en/qos.c b/drivers/net= /ethernet/mellanox/mlx5/core/en/qos.c index ac4979087e0d..3e59862a0e60 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en/qos.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/en/qos.c @@ -233,15 +233,20 @@ void mlx5e_close_qos_sq(struct mlx5e_priv *priv, u16 = qid) kfree(sq); } =20 -void mlx5e_qos_close_queues(struct mlx5e_channel *c) +static struct mlx5e_txqsq __rcu ** +mlx5e_qos_unpublish_queues(struct mlx5e_channel *c) +{ + return rcu_replace_pointer(c->qos_sqs, NULL, + lockdep_is_held(&c->priv->state_lock)); +} + +static void mlx5e_qos_free_queues(struct mlx5e_channel *c, + struct mlx5e_txqsq __rcu **qos_sqs) { - struct mlx5e_txqsq __rcu **qos_sqs; int i; =20 - qos_sqs =3D rcu_replace_pointer(c->qos_sqs, NULL, lockdep_is_held(&c->pri= v->state_lock)); if (!qos_sqs) return; - synchronize_rcu(); /* Sync with NAPI. */ =20 for (i =3D 0; i < c->qos_sqs_size; i++) { struct mlx5e_txqsq *sq; @@ -258,6 +263,27 @@ void mlx5e_qos_close_queues(struct mlx5e_channel *c) kvfree(qos_sqs); } =20 +void mlx5e_qos_close_queues_post_sync(struct mlx5e_channel *c) +{ + struct mlx5e_txqsq __rcu **qos_sqs; + + qos_sqs =3D mlx5e_qos_unpublish_queues(c); + /* Caller guarantees that NAPI is stopped or was never enabled. */ + mlx5e_qos_free_queues(c, qos_sqs); +} + +void mlx5e_qos_close_queues(struct mlx5e_channel *c) +{ + struct mlx5e_txqsq __rcu **qos_sqs; + + qos_sqs =3D mlx5e_qos_unpublish_queues(c); + if (!qos_sqs) + return; + + synchronize_rcu(); /* Sync with NAPI. */ + mlx5e_qos_free_queues(c, qos_sqs); +} + void mlx5e_qos_close_all_queues(struct mlx5e_channels *chs) { int i; @@ -326,11 +352,10 @@ void mlx5e_qos_activate_queues(struct mlx5e_priv *pri= v) mlx5e_htb_enumerate_leaves(priv->htb, mlx5e_activate_qos_sq, priv); } =20 -void mlx5e_qos_deactivate_queues(struct mlx5e_channel *c) +void mlx5e_qos_deactivate_queues_pre_sync(struct mlx5e_channel *c) { struct mlx5e_params *params =3D &c->priv->channels.params; struct mlx5e_txqsq __rcu **qos_sqs; - u16 txq_ix; int i; =20 qos_sqs =3D mlx5e_state_dereference(c->priv, c->qos_sqs); @@ -346,7 +371,30 @@ void mlx5e_qos_deactivate_queues(struct mlx5e_channel = *c) continue; =20 qos_dbg(c->mdev, "Deactivate QoS SQ qid %u\n", qid); - mlx5e_deactivate_txqsq(sq); + mlx5e_deactivate_txqsq_pre_sync(sq); + } +} + +void mlx5e_qos_deactivate_queues_post_sync(struct mlx5e_channel *c) +{ + struct mlx5e_params *params =3D &c->priv->channels.params; + struct mlx5e_txqsq __rcu **qos_sqs; + int i; + + qos_sqs =3D mlx5e_state_dereference(c->priv, c->qos_sqs); + if (!qos_sqs) + return; + + for (i =3D 0; i < c->qos_sqs_size; i++) { + u16 qid =3D params->num_channels * i + c->ix; + struct mlx5e_txqsq *sq; + u16 txq_ix; + + sq =3D mlx5e_state_dereference(c->priv, qos_sqs[i]); + if (!sq) + continue; + + mlx5e_deactivate_txqsq_post_sync(sq); =20 txq_ix =3D mlx5e_qid_from_qos(&c->priv->channels, qid); =20 @@ -361,7 +409,12 @@ void mlx5e_qos_deactivate_all_queues(struct mlx5e_chan= nels *chs) int i; =20 for (i =3D 0; i < chs->num; i++) - mlx5e_qos_deactivate_queues(chs->c[i]); + mlx5e_qos_deactivate_queues_pre_sync(chs->c[i]); + + synchronize_net(); + + for (i =3D 0; i < chs->num; i++) + mlx5e_qos_deactivate_queues_post_sync(chs->c[i]); } =20 void mlx5e_reactivate_qos_sq(struct mlx5e_priv *priv, u16 qid, struct netd= ev_queue *txq) diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en/qos.h b/drivers/net= /ethernet/mellanox/mlx5/core/en/qos.h index 4947afa23b73..8d171a1079e9 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en/qos.h +++ b/drivers/net/ethernet/mellanox/mlx5/core/en/qos.h @@ -28,9 +28,11 @@ void mlx5e_reset_qdisc(struct net_device *dev, u16 qid); =20 int mlx5e_qos_open_queues(struct mlx5e_priv *priv, struct mlx5e_channels *= chs); void mlx5e_qos_activate_queues(struct mlx5e_priv *priv); -void mlx5e_qos_deactivate_queues(struct mlx5e_channel *c); +void mlx5e_qos_deactivate_queues_pre_sync(struct mlx5e_channel *c); +void mlx5e_qos_deactivate_queues_post_sync(struct mlx5e_channel *c); void mlx5e_qos_deactivate_all_queues(struct mlx5e_channels *chs); void mlx5e_qos_close_queues(struct mlx5e_channel *c); +void mlx5e_qos_close_queues_post_sync(struct mlx5e_channel *c); void mlx5e_qos_close_all_queues(struct mlx5e_channels *chs); int mlx5e_qos_alloc_queues(struct mlx5e_priv *priv, struct mlx5e_channels = *chs); =20 diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en/xdp.h b/drivers/net= /ethernet/mellanox/mlx5/core/en/xdp.h index 5b8f4094d553..ea557a8310e5 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en/xdp.h +++ b/drivers/net/ethernet/mellanox/mlx5/core/en/xdp.h @@ -130,14 +130,12 @@ static inline void mlx5e_xdp_tx_enable(struct mlx5e_p= riv *priv) set_bit(MLX5E_STATE_XDP_ACTIVE, &priv->state); } =20 -static inline void mlx5e_xdp_tx_disable(struct mlx5e_priv *priv) +static inline void mlx5e_xdp_tx_disable_pre_sync(struct mlx5e_priv *priv) { if (priv->channels.params.xdp_prog) clear_bit(MLX5E_STATE_XDP_ACTIVE, &priv->state); =20 clear_bit(MLX5E_STATE_XDP_TX_ENABLED, &priv->state); - /* Let other device's napi(s) and XSK wakeups see our new state. */ - synchronize_net(); } =20 static inline bool mlx5e_xdp_tx_is_enabled(struct mlx5e_priv *priv) diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en/xsk/setup.c b/drive= rs/net/ethernet/mellanox/mlx5/core/en/xsk/setup.c index 11500fd213a5..deb84ef29ca1 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en/xsk/setup.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/en/xsk/setup.c @@ -160,23 +160,35 @@ int mlx5e_open_xsk(struct mlx5e_priv *priv, struct ml= x5e_params *params, err_close_rx_cq: mlx5e_close_cq(&c->xskrq.cq); =20 + c->xskrq.xsk_pool =3D NULL; + c->xsksq.xsk_pool =3D NULL; + return err; } =20 -void mlx5e_close_xsk(struct mlx5e_channel *c) +void mlx5e_close_xsk_pre_sync(struct mlx5e_channel *c) { clear_bit(MLX5E_CHANNEL_STATE_XSK, c->state); - synchronize_net(); /* Sync with NAPI. */ +} =20 +void mlx5e_close_xsk_post_sync(struct mlx5e_channel *c) +{ mlx5e_close_rq(&c->xskrq); mlx5e_close_cq(&c->xskrq.cq); - mlx5e_close_xdpsq(&c->xsksq); + mlx5e_close_xdpsq_post_sync(&c->xsksq); mlx5e_close_cq(&c->xsksq.cq); =20 memset(&c->xskrq, 0, sizeof(c->xskrq)); memset(&c->xsksq, 0, sizeof(c->xsksq)); } =20 +void mlx5e_close_xsk(struct mlx5e_channel *c) +{ + mlx5e_close_xsk_pre_sync(c); + synchronize_net(); /* Sync with XSK wakeup and NAPI. */ + mlx5e_close_xsk_post_sync(c); +} + void mlx5e_activate_xsk(struct mlx5e_channel *c) { /* ICOSQ recovery deactivates RQs. Suspend the recovery to avoid @@ -189,7 +201,7 @@ void mlx5e_activate_xsk(struct mlx5e_channel *c) /* TX queue is created active. */ } =20 -void mlx5e_deactivate_xsk(struct mlx5e_channel *c) +void mlx5e_deactivate_xsk_pre_sync(struct mlx5e_channel *c) { /* ICOSQ recovery may reactivate XSKRQ if clear_bit is called in the * middle of recovery. Suspend the recovery to avoid it. @@ -197,7 +209,12 @@ void mlx5e_deactivate_xsk(struct mlx5e_channel *c) mlx5e_reporter_icosq_suspend_recovery(c); clear_bit(MLX5E_RQ_STATE_ENABLED, &c->xskrq.state); mlx5e_reporter_icosq_resume_recovery(c); - synchronize_net(); /* Sync with NAPI to prevent mlx5e_post_rx_wqes. */ =20 /* TX queue is disabled on close. */ } + +void mlx5e_deactivate_xsk(struct mlx5e_channel *c) +{ + mlx5e_deactivate_xsk_pre_sync(c); + synchronize_net(); /* Sync with NAPI to prevent mlx5e_post_rx_wqes. */ +} diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en/xsk/setup.h b/drive= rs/net/ethernet/mellanox/mlx5/core/en/xsk/setup.h index 664ec78192c3..66a0b09452f7 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en/xsk/setup.h +++ b/drivers/net/ethernet/mellanox/mlx5/core/en/xsk/setup.h @@ -17,7 +17,10 @@ int mlx5e_open_xsk(struct mlx5e_priv *priv, struct mlx5e= _params *params, struct xsk_buff_pool *pool, struct mlx5e_channel *c); void mlx5e_close_xsk(struct mlx5e_channel *c); +void mlx5e_close_xsk_pre_sync(struct mlx5e_channel *c); +void mlx5e_close_xsk_post_sync(struct mlx5e_channel *c); void mlx5e_activate_xsk(struct mlx5e_channel *c); void mlx5e_deactivate_xsk(struct mlx5e_channel *c); +void mlx5e_deactivate_xsk_pre_sync(struct mlx5e_channel *c); =20 #endif /* __MLX5_EN_XSK_SETUP_H__ */ diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_main.c b/drivers/ne= t/ethernet/mellanox/mlx5/core/en_main.c index f90b5a34e097..fe4e5d0b2e9a 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en_main.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_main.c @@ -1458,9 +1458,14 @@ void mlx5e_activate_rq(struct mlx5e_rq *rq) set_bit(MLX5E_RQ_STATE_ENABLED, &rq->state); } =20 -void mlx5e_deactivate_rq(struct mlx5e_rq *rq) +void mlx5e_deactivate_rq_pre_sync(struct mlx5e_rq *rq) { clear_bit(MLX5E_RQ_STATE_ENABLED, &rq->state); +} + +void mlx5e_deactivate_rq(struct mlx5e_rq *rq) +{ + mlx5e_deactivate_rq_pre_sync(rq); synchronize_net(); /* Sync with NAPI to prevent mlx5e_post_rx_wqes. */ } =20 @@ -1954,13 +1959,15 @@ void mlx5e_tx_disable_queue(struct netdev_queue *tx= q) __netif_tx_unlock_bh(txq); } =20 -void mlx5e_deactivate_txqsq(struct mlx5e_txqsq *sq) +void mlx5e_deactivate_txqsq_pre_sync(struct mlx5e_txqsq *sq) { - struct mlx5_wq_cyc *wq =3D &sq->wq; - netif_queue_set_napi(sq->netdev, sq->txq_ix, NETDEV_QUEUE_TYPE_TX, NULL); clear_bit(MLX5E_SQ_STATE_ENABLED, &sq->state); - synchronize_net(); /* Sync with NAPI to prevent netif_tx_wake_queue. */ +} + +void mlx5e_deactivate_txqsq_post_sync(struct mlx5e_txqsq *sq) +{ + struct mlx5_wq_cyc *wq =3D &sq->wq; =20 mlx5e_tx_disable_queue(sq->txq); =20 @@ -1978,6 +1985,13 @@ void mlx5e_deactivate_txqsq(struct mlx5e_txqsq *sq) } } =20 +void mlx5e_deactivate_txqsq(struct mlx5e_txqsq *sq) +{ + mlx5e_deactivate_txqsq_pre_sync(sq); + synchronize_net(); /* Sync with NAPI to prevent netif_tx_wake_queue. */ + mlx5e_deactivate_txqsq_post_sync(sq); +} + void mlx5e_close_txqsq(struct mlx5e_txqsq *sq) { struct mlx5_core_dev *mdev =3D sq->mdev; @@ -2089,9 +2103,14 @@ void mlx5e_activate_icosq(struct mlx5e_icosq *icosq) set_bit(MLX5E_SQ_STATE_ENABLED, &icosq->state); } =20 -void mlx5e_deactivate_icosq(struct mlx5e_icosq *icosq) +void mlx5e_deactivate_icosq_pre_sync(struct mlx5e_icosq *icosq) { clear_bit(MLX5E_SQ_STATE_ENABLED, &icosq->state); +} + +void mlx5e_deactivate_icosq(struct mlx5e_icosq *icosq) +{ + mlx5e_deactivate_icosq_pre_sync(icosq); synchronize_net(); /* Sync with NAPI. */ } =20 @@ -2141,18 +2160,28 @@ int mlx5e_open_xdpsq(struct mlx5e_channel *c, struc= t mlx5e_params *params, return err; } =20 -void mlx5e_close_xdpsq(struct mlx5e_xdpsq *sq) +static void mlx5e_destroy_xdpsq(struct mlx5e_xdpsq *sq) { struct mlx5e_channel *c =3D sq->channel; =20 - clear_bit(MLX5E_SQ_STATE_ENABLED, &sq->state); - synchronize_net(); /* Sync with NAPI. */ - mlx5e_destroy_sq(c->mdev, sq->sqn); mlx5e_free_xdpsq_descs(sq); mlx5e_free_xdpsq(sq); } =20 +void mlx5e_close_xdpsq_post_sync(struct mlx5e_xdpsq *sq) +{ + clear_bit(MLX5E_SQ_STATE_ENABLED, &sq->state); + mlx5e_destroy_xdpsq(sq); +} + +void mlx5e_close_xdpsq(struct mlx5e_xdpsq *sq) +{ + clear_bit(MLX5E_SQ_STATE_ENABLED, &sq->state); + synchronize_net(); /* Sync with NAPI. */ + mlx5e_destroy_xdpsq(sq); +} + static struct mlx5e_xdpsq *mlx5e_open_xdpredirect_sq(struct mlx5e_channel = *c, struct mlx5e_params *params, struct mlx5e_channel_param *cparam, @@ -2184,6 +2213,13 @@ static struct mlx5e_xdpsq *mlx5e_open_xdpredirect_sq= (struct mlx5e_channel *c, return ERR_PTR(err); } =20 +static void mlx5e_close_xdpredirect_sq_post_sync(struct mlx5e_xdpsq *xdpsq) +{ + mlx5e_close_xdpsq_post_sync(xdpsq); + mlx5e_close_cq(&xdpsq->cq); + kvfree(xdpsq); +} + static void mlx5e_close_xdpredirect_sq(struct mlx5e_xdpsq *xdpsq) { mlx5e_close_xdpsq(xdpsq); @@ -2749,6 +2785,27 @@ static void mlx5e_close_queues(struct mlx5e_channel = *c) mlx5e_close_cq(&c->icosq.cq); } =20 +static void mlx5e_close_queues_post_sync(struct mlx5e_channel *c) +{ + if (c->xdp) + mlx5e_close_xdpsq_post_sync(&c->rq_xdpsq); + /* The same ICOSQ is used for UMRs for both RQ and XSKRQ. */ + cancel_work_sync(&c->icosq.recover_work); + mlx5e_close_rq(&c->rq); + mlx5e_close_sqs(c); + mlx5e_close_icosq(&c->icosq); + mutex_destroy(&c->icosq_recovery_lock); + if (c->async_icosq) + mlx5e_close_async_icosq(c->async_icosq); + if (c->xdp) + mlx5e_close_cq(&c->rq_xdpsq.cq); + mlx5e_close_cq(&c->rq.cq); + if (c->xdpsq) + mlx5e_close_xdpredirect_sq_post_sync(c->xdpsq); + mlx5e_close_tx_cqs(c); + mlx5e_close_cq(&c->icosq.cq); +} + static u8 mlx5e_enumerate_lag_port(struct mlx5_core_dev *mdev, int ix) { u16 port_aff_bias =3D mlx5_core_is_pf(mdev) ? 0 : MLX5_CAP_GEN(mdev, vhca= _id); @@ -2936,35 +2993,56 @@ static void mlx5e_activate_channel(struct mlx5e_cha= nnel *c) netif_queue_set_napi(c->netdev, c->ix, NETDEV_QUEUE_TYPE_RX, &c->napi); } =20 -static void mlx5e_deactivate_channel(struct mlx5e_channel *c) +static void mlx5e_deactivate_channel_pre_sync(struct mlx5e_channel *c) { int tc; =20 netif_queue_set_napi(c->netdev, c->ix, NETDEV_QUEUE_TYPE_RX, NULL); =20 if (test_bit(MLX5E_CHANNEL_STATE_XSK, c->state)) - mlx5e_deactivate_xsk(c); + mlx5e_deactivate_xsk_pre_sync(c); else - mlx5e_deactivate_rq(&c->rq); + mlx5e_deactivate_rq_pre_sync(&c->rq); =20 if (c->async_icosq) - mlx5e_deactivate_icosq(c->async_icosq); - mlx5e_deactivate_icosq(&c->icosq); + mlx5e_deactivate_icosq_pre_sync(c->async_icosq); + mlx5e_deactivate_icosq_pre_sync(&c->icosq); for (tc =3D 0; tc < c->num_tc; tc++) - mlx5e_deactivate_txqsq(&c->sq[tc]); - mlx5e_qos_deactivate_queues(c); + mlx5e_deactivate_txqsq_pre_sync(&c->sq[tc]); + mlx5e_qos_deactivate_queues_pre_sync(c); +} + +static void mlx5e_deactivate_channel_post_sync(struct mlx5e_channel *c) +{ + int tc; + + for (tc =3D 0; tc < c->num_tc; tc++) + mlx5e_deactivate_txqsq_post_sync(&c->sq[tc]); + mlx5e_qos_deactivate_queues_post_sync(c); =20 napi_disable_locked(&c->napi); } =20 -static void mlx5e_close_channel(struct mlx5e_channel *c) +static void mlx5e_close_channel_pre_sync(struct mlx5e_channel *c) { if (test_bit(MLX5E_CHANNEL_STATE_XSK, c->state)) - mlx5e_close_xsk(c); - mlx5e_close_queues(c); - mlx5e_qos_close_queues(c); - netif_napi_del_locked(&c->napi); + mlx5e_close_xsk_pre_sync(c); +} + +static void mlx5e_close_channel_post_sync(struct mlx5e_channel *c) +{ + if (c->xskrq.xsk_pool) + mlx5e_close_xsk_post_sync(c); + mlx5e_close_queues_post_sync(c); + mlx5e_qos_close_queues_post_sync(c); +} =20 +static void mlx5e_close_channel(struct mlx5e_channel *c) +{ + mlx5e_close_channel_pre_sync(c); + synchronize_net(); + mlx5e_close_channel_post_sync(c); + netif_napi_del_locked(&c->napi); kvfree(c); } =20 @@ -3060,33 +3138,72 @@ static int mlx5e_wait_channels_min_rx_wqes(struct m= lx5e_channels *chs) return err ? -ETIMEDOUT : 0; } =20 -static void mlx5e_deactivate_channels(struct mlx5e_channels *chs) +static void mlx5e_deactivate_channels_pre_sync(struct mlx5e_channels *chs) { int i; =20 if (chs->ptp) - mlx5e_ptp_deactivate_channel(chs->ptp); + mlx5e_ptp_deactivate_channel_pre_sync(chs->ptp); =20 for (i =3D 0; i < chs->num; i++) - mlx5e_deactivate_channel(chs->c[i]); + mlx5e_deactivate_channel_pre_sync(chs->c[i]); } =20 -void mlx5e_close_channels(struct mlx5e_channels *chs) +static void mlx5e_deactivate_channels_post_sync(struct mlx5e_channels *chs) +{ + int i; + + if (chs->ptp) + mlx5e_ptp_deactivate_channel_post_sync(chs->ptp); + + for (i =3D 0; i < chs->num; i++) + mlx5e_deactivate_channel_post_sync(chs->c[i]); +} + +static void mlx5e_close_channels_pre_sync(struct mlx5e_channels *chs) +{ + int i; + + for (i =3D 0; i < chs->num; i++) + mlx5e_close_channel_pre_sync(chs->c[i]); +} + +static void mlx5e_close_channels_post_sync(struct mlx5e_channels *chs) { int i; =20 - ASSERT_RTNL(); if (chs->ptp) { - mlx5e_ptp_close(chs->ptp); + mlx5e_ptp_close_queues(chs->ptp); + __netif_napi_del_locked(&chs->ptp->napi); + } + + for (i =3D 0; i < chs->num; i++) { + mlx5e_close_channel_post_sync(chs->c[i]); + __netif_napi_del_locked(&chs->c[i]->napi); + } + + synchronize_net(); + + if (chs->ptp) { + kvfree(chs->ptp); chs->ptp =3D NULL; } + for (i =3D 0; i < chs->num; i++) - mlx5e_close_channel(chs->c[i]); + kvfree(chs->c[i]); =20 kfree(chs->c); chs->num =3D 0; } =20 +void mlx5e_close_channels(struct mlx5e_channels *chs) +{ + ASSERT_RTNL(); + mlx5e_close_channels_pre_sync(chs); + synchronize_net(); + mlx5e_close_channels_post_sync(chs); +} + static int mlx5e_modify_tirs_packet_merge(struct mlx5e_priv *priv) { struct mlx5e_rx_res *res =3D priv->rx_res; @@ -3376,7 +3493,7 @@ static void mlx5e_cancel_tx_timeout_work(struct mlx5e= _priv *priv) cancel_work_sync(&priv->tx_timeout_work); } =20 -void mlx5e_deactivate_priv_channels(struct mlx5e_priv *priv) +static void mlx5e_deactivate_priv_channels_pre_sync(struct mlx5e_priv *pri= v) { if (priv->rx_res) mlx5e_rx_res_channels_deactivate(priv->rx_res); @@ -3394,8 +3511,21 @@ void mlx5e_deactivate_priv_channels(struct mlx5e_pri= v *priv) */ netif_tx_disable(priv->netdev); =20 - mlx5e_xdp_tx_disable(priv); - mlx5e_deactivate_channels(&priv->channels); + mlx5e_xdp_tx_disable_pre_sync(priv); + mlx5e_deactivate_channels_pre_sync(&priv->channels); +} + +static void mlx5e_deactivate_priv_channels_post_sync(struct mlx5e_priv *pr= iv) +{ + mlx5e_deactivate_channels_post_sync(&priv->channels); +} + +void mlx5e_deactivate_priv_channels(struct mlx5e_priv *priv) +{ + mlx5e_deactivate_priv_channels_pre_sync(priv); + /* One grace period for XDP, RQ, ICOSQ, TX, QoS, XSK and PTP. */ + synchronize_net(); + mlx5e_deactivate_priv_channels_post_sync(priv); } =20 static int mlx5e_switch_priv_params(struct mlx5e_priv *priv, @@ -3611,8 +3741,14 @@ int mlx5e_close_locked(struct net_device *netdev) clear_bit(MLX5E_STATE_OPENED, &priv->state); =20 netif_carrier_off(priv->netdev); - mlx5e_deactivate_priv_channels(priv); - mlx5e_close_channels(&priv->channels); + + mlx5e_deactivate_priv_channels_pre_sync(priv); + mlx5e_close_channels_pre_sync(&priv->channels); + + synchronize_net(); + + mlx5e_deactivate_priv_channels_post_sync(priv); + mlx5e_close_channels_post_sync(&priv->channels); =20 return 0; } --=20 2.44.0 From nobody Sat Oct 24 20:10:35 2026 Received: from BL0PR03CU003.outbound.protection.outlook.com (mail-eastusazon11012063.outbound.protection.outlook.com [52.101.53.63]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5ABB444781B; Sun, 11 Oct 2026 13:33:33 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=52.101.53.63 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1791725614; cv=fail; b=ZVO4un7x+I94uRkV7QJMH2TsepSgliFBLLfvd8TcG23nf+Zo2Ynt0T8MKMBlh8lRUPbiKwT2HMWz7Qu7D0DNmfmpKLj0FwbcunYT+S+ikXd8dFEW64UsHol0GJU4N85j21RzF1GrFRblFrgGss9lEeLcIhhPUDPf11Jpa/3wks8= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1791725614; c=relaxed/simple; bh=elq1Nnw2J63M2tK9xQud0Rdh03Yq9t/T5+4G65yLFIs=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=JUs37351cl3dCcVq781ZLUOKLPMyhTEO85nQMhZ4bE4W3oInkgEvvyNw8bcl4rwRds2iwUMWlxbMI0lyfW1YBPG3FrRfz7SXCtSLBlG7YGV5TqsqmpGhtPqyhKss7wPRWrSbAe53js9OWHMKtDC2TmlZFIvdnERpI5LWTc0K4NI= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=RBgjEx+p; arc=fail smtp.client-ip=52.101.53.63 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="RBgjEx+p" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=L8iF4UngmdG3iwkehSVE56GVv4DURPwyw/kigXGll9A11Q4eAFcmgwFgywSAnY+Dv2/VAxynjATG0B+2CpQFhrV7msImbOiEqN38QIS3r0I0RXHkIAiMLgik/CIx6pVdMATH1l7vD0q01VHJTUNMjvG1gHXo6airgBy3qpssduLrA1lhNSrxqX3ISCDTKc4ju3Xw5A49EPUZzQPN6D/eewdg8s1RCP/m9cdiFTpNJF8hfwY6IxXYPborZZ98tCP2KjHXiG4shtZtyuWHdvqT5qIvT+D+9YI3bHxfiJwardFVvjY9Cy7MTfcgcgEYcGFPUjbl/buE/qYdm+4Y4jYjyg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=emdCEx4AyVfEEJiS5fxAN31pCT7OD4EF6bnUNHctlCA=; b=Zwp2btLyUD5cqL7udQrYper+2RzVFr8p5o+/gKofFiLhqLWENKthemlOY8Xbv8UuJSG++IcTd8Jy6NUnJCmRW5DcGqXDbQNYlgJ96KLY1KW5oXVjahiPxLTWoVBV6oGw/DbZ0hSBkmDnBnGpiqiuTBgQJHVJyd+nPSh1ho0dEu3lL79ADtkqbF7QsAgddy71GNUHkG0yRyv1uGfpiekCTUyoT7WJRRwDSDWyESFgk23cyXBFzvhgkSw2VVF6K9K/yqJxeDYjY1pQi/d5zNGR7znpzZGTrevRwXi0oW0OpveyXQX+YxoH/h9VMYOKNwMO1k9l4S8ncXej6GOYp3H8uA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.161) smtp.rcpttodomain=lunn.ch smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=emdCEx4AyVfEEJiS5fxAN31pCT7OD4EF6bnUNHctlCA=; b=RBgjEx+pun2y/hkS0MuIEUyvaeNJXIIN145ztISSpDyoT3lVjk0/HCvKNcF8ncs0sjpKd0DXVYu+CTJQWVFVQBokYm93aFJtpJxjOmCHbaCFZPtCywEuMtXCBeiFSujjZgoExi8P4BR5rpzQQCz+wNgEcmootcBDEFH/kEnpnxTivf9fkh0VVU4VEK93UPBzFKn/bgrAxYz6hGDvwBb24s9EJLtFKJJt1PPduxcla4TWnFgdkQpFYQTBAYeqngX6DibDiOEfSY0SXecbXsxBjdCNy3O09SGvfu0f1TLgeRdpWikAG6NYfWfKO6mvApSRFVRfUqVXO0pQAZFCsGJCcQ== Received: from BN9PR03CA0066.namprd03.prod.outlook.com (2603:10b6:408:fc::11) by BN7PPFEE0F400A9.namprd12.prod.outlook.com (2603:10b6:40f:fc02::6e9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.496.20; Sun, 11 Oct 2026 13:33:27 +0000 Received: from BN1PEPF00004680.namprd03.prod.outlook.com (2603:10b6:408:fc:cafe::5c) by BN9PR03CA0066.outlook.office365.com (2603:10b6:408:fc::11) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.21.496.5 via Frontend Transport; Sun, 11 Oct 2026 13:33:27 +0000 X-MS-Exchange-Authentication-Results: mx.microsoft.com 1; spf=pass (sender IP is 216.228.117.161) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.161 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.161; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.161) by BN1PEPF00004680.mail.protection.outlook.com (10.167.243.85) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.522.6 via Frontend Transport; Sun, 11 Oct 2026 13:33:27 +0000 Received: from rnnvmail203.nvidia.com (10.129.68.9) by mail.nvidia.com (10.129.200.67) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.49; Sun, 11 Oct 2026 06:33:17 -0700 Received: from rnnvmail205.nvidia.com (10.129.68.10) by rnnvmail203.nvidia.com (10.129.68.9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.49; Sun, 11 Oct 2026 06:33:17 -0700 Received: from vdi.nvidia.com (10.127.8.10) by mail.nvidia.com (10.129.68.10) with Microsoft SMTP Server id 15.2.2562.49 via Frontend Transport; Sun, 11 Oct 2026 06:33:10 -0700 From: Tariq Toukan To: Andrew Lunn , "David S. Miller" , Eric Dumazet , Jakub Kicinski , , Paolo Abeni CC: Alexei Starovoitov , , "Carolina Jubran" , Cosmin Ratiu , "Daniel Borkmann" , Dragos Tatulea , "Gal Pressman" , Jesper Dangaard Brouer , "John Fastabend" , Kees Cook , "Leon Romanovsky" , open list , , Li RongQing , Mark Bloch , Richard Cochran , "Saeed Mahameed" , Shahar Shitrit , Stanislav Fomichev , Tariq Toukan , =?UTF-8?q?Toke=20H=C3=B8iland-J=C3=B8rgensen?= , =?UTF-8?q?Toke=20H=C3=B8iland-J=C3=B8rgensen?= , "Adrian Moreno" , Michal Schmidt Subject: [PATCH net-next 2/2] net/mlx5e: Batch ICOSQ trigger RCU synchronization Date: Sun, 11 Oct 2026 16:32:12 +0300 Message-ID: <20261011133212.807979-3-tariqt@nvidia.com> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20261011133212.807979-1-tariqt@nvidia.com> References: <20261011133212.807979-1-tariqt@nvidia.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-NV-OnPremToCloud: ExternallySecured X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BN1PEPF00004680:EE_|BN7PPFEE0F400A9:EE_ X-MS-Office365-Filtering-Correlation-Id: 8600a066-67cc-42d7-3aaa-08df279c3b48 X-LD-Processed: 43083d15-7273-40c1-b7db-39efd9ccc17a,ExtAddr X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|1800799024|36860700016|23010399003|376014|7416014|82310400026|10067099003|261009223027099003|11063799006|6133799003|18002099003|56012099006|22082099003; X-Microsoft-Antispam-Message-Info: SNUDGuX4uWIYr4965kUU3qwYngBt+Y4uYC6QXNsXkdWEl0hPOYH4IFNVBdAUq7jO05snAgVdPO/owlWuDxHNvmBdOTzxt/viCWb5bo4sN8tcmGznnio18+KypJGq5J+X/zWAaftFSELRibFWwrt4XEYHtD7gn4JH+XnyozxCUunW0viBbsPgdaFwNobrNAcqwkZtmTear+2NwAzsrJNA0RcOfesAdtiJiI4MdvA1fBxK/oHvgd6SRlwekdfimKO9JydZc8UHKFfaC7lo1uPzs2ZRTZ06ol5Pt/oGyg1Htt9V8Ox8Qh5URIPtLl8WOB65jYpXHC5Neq7YwyUgVpQ9rffVN5vVOsJXy175D6sXX2L184B2HVH5ZHtthCF+uEjn3cg0CNP3zMXvtNjiGeRDg5LOcHU4uBRIgZRjUcuwgMPY3gN8WJJryD67NHxSrVpSaHi87m0sBsDx3tZih29lh0csFuSWivQcTqX0hcnmab2UptQIfNJgh+7c72GJQOBEsa7Afyv6ExO7VwZbH7h9HLoFc1XRR7ttLpM+Rlh5Nw7ysi5aqhQBf6SyXBwjlyGXMHy0UFXY+CKfO3XPJIkZ5l7z5FN2BjtTtsmjs7C2AKZSvGekfwLk/E89Obu8Qta3ZuGSzY/GMcaqppVz3f8TpdEj0t/jA2ROB0GKBirqhufeDHzZFLig6N9eRQLk27ByxQuj2UxIaDZJNQd0PdQdOQ== X-Forefront-Antispam-Report: CIP:216.228.117.161;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:dc6edge2.nvidia.com;CAT:NONE;SFS:(13230040)(1800799024)(36860700016)(23010399003)(376014)(7416014)(82310400026)(10067099003)(261009223027099003)(11063799006)(6133799003)(18002099003)(56012099006)(22082099003);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: WvAl5n9D/b+QX7DgfkohCdXUF04e6qvLy/plUw4fAx4qCQ3qrXzuKCD4Qfan6rILSfO3Ccw1BSO3krGTmlyO0FoUJWNzYJFjIUCwW+GFFVxc7n/1lYGL3h/TwKTFjHsh1k/Z5xKuxHqgHR5nnJIyWy8DLpxmLDLb+ej2WCmJ/VSl8K256IFkWSoR08hKT1DKmq+NrMwoQAzAQ/kwB6sV9pDHBIGu9kgfAkTD4iOPdh4Nl9C2R2b/Hwtz5Z4/Agn8SVBYxE66uN+4RxoyQCjHX9YrBHm55ZVYm7Fh+qYoRv5p2d27mMgVq4O/KjunUqGggE6uLCEyh+VsfrwagzLuLxzDX+Pn+Tugx251Evz5nPkbUNOR8QtnepPMwFiQKTNCDoNJe7WELXNaIIMM1mLp4cXYKCgMckx8lOiixC51HCG0oIh5AUyO7tfOToBCnoLS X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 11 Oct 2026 13:33:27.1048 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 8600a066-67cc-42d7-3aaa-08df279c3b48 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.117.161];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: BN1PEPF00004680.namprd03.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN7PPFEE0F400A9 Content-Type: text/plain; charset="utf-8" From: Carolina Jubran During channel activation, mlx5e_activate_channels() triggers the ICOSQ once for every channel in order to kick NAPI. mlx5e_trigger_napi_icosq() sets MLX5E_SQ_STATE_LOCK_NEEDED and waits for an RCU grace period before serializing the trigger with ICOSQ users in NAPI context. Mark all channel ICOSQs as requiring locking first, wait for a single grace period, and then trigger all of them. Keep mlx5e_trigger_napi_icosq() with its existing synchronization semantics for standalone callers. 248 channels, rcu_normal=3D1, ip link set eth2 up: Before: 3.095s After: 1.794s (~1.7x faster)=20 Signed-off-by: Carolina Jubran Signed-off-by: Tariq Toukan Acked-by: Toke H=C3=B8iland-J=C3=B8rgensen --- .../net/ethernet/mellanox/mlx5/core/en_main.c | 28 +++++++++++++++---- 1 file changed, 22 insertions(+), 6 deletions(-) diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_main.c b/drivers/ne= t/ethernet/mellanox/mlx5/core/en_main.c index fe4e5d0b2e9a..58e4874b664f 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en_main.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_main.c @@ -2837,14 +2837,11 @@ static int mlx5e_channel_stats_alloc(struct mlx5e_p= riv *priv, int ix, int cpu) return 0; } =20 -void mlx5e_trigger_napi_icosq(struct mlx5e_channel *c) +static void mlx5e_trigger_napi_icosq_post_sync(struct mlx5e_channel *c) { struct mlx5e_icosq *sq =3D &c->icosq; bool locked; =20 - set_bit(MLX5E_SQ_STATE_LOCK_NEEDED, &sq->state); - synchronize_net(); - locked =3D mlx5e_icosq_sync_lock(sq); mlx5e_trigger_irq(sq); mlx5e_icosq_sync_unlock(sq, locked); @@ -2852,6 +2849,13 @@ void mlx5e_trigger_napi_icosq(struct mlx5e_channel *= c) clear_bit(MLX5E_SQ_STATE_LOCK_NEEDED, &sq->state); } =20 +void mlx5e_trigger_napi_icosq(struct mlx5e_channel *c) +{ + set_bit(MLX5E_SQ_STATE_LOCK_NEEDED, &c->icosq.state); + synchronize_net(); + mlx5e_trigger_napi_icosq_post_sync(c); +} + void mlx5e_trigger_napi_async_icosq(struct mlx5e_channel *c) { struct mlx5e_icosq *sq =3D c->async_icosq; @@ -3099,6 +3103,19 @@ int mlx5e_open_channels(struct mlx5e_priv *priv, return err; } =20 +static void mlx5e_trigger_napi_icosqs(struct mlx5e_channels *chs) +{ + int i; + + for (i =3D 0; i < chs->num; i++) + set_bit(MLX5E_SQ_STATE_LOCK_NEEDED, &chs->c[i]->icosq.state); + + synchronize_net(); + + for (i =3D 0; i < chs->num; i++) + mlx5e_trigger_napi_icosq_post_sync(chs->c[i]); +} + static void mlx5e_activate_channels(struct mlx5e_priv *priv, struct mlx5e_= channels *chs) { int i; @@ -3109,8 +3126,7 @@ static void mlx5e_activate_channels(struct mlx5e_priv= *priv, struct mlx5e_channe if (priv->htb) mlx5e_qos_activate_queues(priv); =20 - for (i =3D 0; i < chs->num; i++) - mlx5e_trigger_napi_icosq(chs->c[i]); + mlx5e_trigger_napi_icosqs(chs); =20 if (chs->ptp) mlx5e_ptp_activate_channel(chs->ptp); --=20 2.44.0