From nobody Thu Jun 18 08:01:02 2026 Received: from SA9PR02CU001.outbound.protection.outlook.com (mail-southcentralusazon11013024.outbound.protection.outlook.com [40.93.196.24]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3A4F42F745C; Fri, 1 May 2026 04:17:58 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.93.196.24 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777609081; cv=fail; b=qRPB4OaxkL7r6CfVbcmU8CMuMkYbosEWYQO3TWJoJAEcYQqVJ7wPedbjROE2yqhh+NyKbgDC6408qP962GAFI1qepZYeK4IIlKIvt3yAAksRcvchO/XqnAQCcMr7A71pjyaYcSxbcfI3c0edNTItgPU7cQZAEn2WqbkSXj93HSw= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777609081; c=relaxed/simple; bh=5MIInt+Y+u7mKnfRlPU99I8yEv7BH08wZEB2WflIYFw=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=cP2keY6HY7GYhz6f47Ctm3CDksoCObhKEeNF3UaVKDl5U5z+PzFo2s2vdDDyMK2VyfZRKbpvNv9RB/vNbzHvgRXhGlWVtqnOQoinXhNXmGDITe07IGqX8YLFDRj6X7EduOE8EO4a3Tx2H5dYbtvS0YrWP9b2erxZ4PeBOFzhCxI= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=RXRAcZ3n; arc=fail smtp.client-ip=40.93.196.24 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="RXRAcZ3n" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=Y8n59S7BeTcMxrmssCk5xasn+VUKZdbCjA6MJE9ViBtFDtqZQj/hx+fQJY31yir9vvbOXBBqhRhs5HhVKjw6G789NMDdIXGLpKQrX7iwexNIZecNt0IEsbZLSo2Aw5os9n2XRPeXaLS6PbS9uGhgMqTEwSkxtg48lUX6GRLIhTsKtk03XoXIQN9oHNaGPt2Ic6ZEQqAhugAT2KDQUEDqzRcq1yXKif1Yc+lW+xn9Ta6sXJrv6GvWjrRPtfU3w7VTO6u0QDhOyajoFyp2p1YFQ4G3bEQRPqFuharH1deGZPP8L1NGZYJm+NRL1ewgPZQjD6uZIL6t8AB3Dwc/BuCDrA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=pYugGDI18ti/LwbPPrYqcQpKwQjMG5B0ihj2okDAXoE=; b=NY7tBHHGSorGqSTt1vRE0xI4aJik3g4wDdB3Lyn63zDo8ZZzHZrGEZZvKDMHe8PpJgYM4aft2Ahw5LCG3qhKso745hW3KTL9LyZXNRC27l/+jRb6Xe9si9DarBCjt3yo8uFEoBeHbHMgcMqItKPeBc76p2i2ArHHVDqtZM7B3p3ugE0wjfiNxmDNZTyIZnTlwM2f+OF2CJ7XNbltNKJS3p9gU7KM0Wm5VB+M6u/Rlyrww4FtG7X4o1A4UnEkH39n9OnXEIOqq2U/5hLXedYnGithHQV/qm6GiZxE7ZiHgU4lQHcbQSqw31DnBCeskEEoCXKpQdAKcx0Znh84hb5muQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.160) smtp.rcpttodomain=google.com smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=pYugGDI18ti/LwbPPrYqcQpKwQjMG5B0ihj2okDAXoE=; b=RXRAcZ3nNWNMe549PhUBTQBx1yYdMAj6bIbJKQrW3XaB8DK4W5nzIN6SyzJs3xLaTXEdzurwW9v/ZKgIaWyFl5TtFu024oI9P4MrvTBbBKuuE5ViQV3WXOljvQ60CORl6oaegFF8WqqNMQKovfvXFX5DtYS5+RdBrz/XB88IYINTcG96LH1gNKS6egTN822jTwNbovdBYTPyi2wrCvAWNwcY1epc2uMwciuvSOZFwX+Pac3HLxoQLCxznR+DCh+m7PbVx1U/TelW5+wIPfuWKJOZy4nXvgbbJ3McP6H8GUgPTnctgwJIHdA8JgBh+JhqH18Yuf36UHPLRx8flUvN5A== Received: from DM6PR06CA0040.namprd06.prod.outlook.com (2603:10b6:5:54::17) by DS2PR12MB9615.namprd12.prod.outlook.com (2603:10b6:8:275::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9870.22; Fri, 1 May 2026 04:17:45 +0000 Received: from SA2PEPF000015C6.namprd03.prod.outlook.com (2603:10b6:5:54:cafe::c9) by DM6PR06CA0040.outlook.office365.com (2603:10b6:5:54::17) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.9870.23 via Frontend Transport; Fri, 1 May 2026 04:17:45 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.160) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.160 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.160; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.160) by SA2PEPF000015C6.mail.protection.outlook.com (10.167.241.196) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9870.22 via Frontend Transport; Fri, 1 May 2026 04:17:45 +0000 Received: from rnnvmail201.nvidia.com (10.129.68.8) by mail.nvidia.com (10.129.200.66) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.20; Thu, 30 Apr 2026 21:17:27 -0700 Received: from rnnvmail203.nvidia.com (10.129.68.9) by rnnvmail201.nvidia.com (10.129.68.8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.20; Thu, 30 Apr 2026 21:17:26 -0700 Received: from vdi.nvidia.com (10.127.8.10) by mail.nvidia.com (10.129.68.9) with Microsoft SMTP Server id 15.2.2562.20 via Frontend Transport; Thu, 30 Apr 2026 21:17:19 -0700 From: Tariq Toukan To: Eric Dumazet , Jakub Kicinski , Paolo Abeni , Andrew Lunn , "David S. Miller" CC: Leon Romanovsky , Jason Gunthorpe , "Saeed Mahameed" , Tariq Toukan , Mark Bloch , Shay Drory , Or Har-Toov , Edward Srouji , Maher Sanalla , Simon Horman , Gerd Bayer , Moshe Shemesh , Kees Cook , Patrisious Haddad , Parav Pandit , Carolina Jubran , Cosmin Ratiu , , , , Gal Pressman , Dragos Tatulea Subject: [PATCH net-next V2 1/7] net/mlx5: Lag: refactor representor reload handling Date: Fri, 1 May 2026 07:16:27 +0300 Message-ID: <20260501041633.231662-2-tariqt@nvidia.com> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20260501041633.231662-1-tariqt@nvidia.com> References: <20260501041633.231662-1-tariqt@nvidia.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-NV-OnPremToCloud: ExternallySecured X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: SA2PEPF000015C6:EE_|DS2PR12MB9615:EE_ X-MS-Office365-Filtering-Correlation-Id: 32cedbe3-5287-4e22-e503-08dea73898ad X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|7416014|376014|1800799024|82310400026|36860700016|22082099003|18002099003|56012099003; X-Microsoft-Antispam-Message-Info: MENthB2JhaKxcYoH+ZQ8Fyvx0y88gV7V54Fa4jZf3l0NlniIS0zQSaTSK1K79ZpaJvx4olSoWHctNmOMN9OqiloBilshwtB3IduuGnyyImJhDWbVDCmQlooBAWwaKtYjDGSsK3Qf4JJ5ELo9JWoAy+zcTrXIPQSeunYpBCF2Eu6JGNmswv3r09jSgBv8SEWaFr2bIJPxfIIGt5cTUUTMR2rWheMWfiKGrSZp9Bqy0KOWhBsuem7/i8InmySGxk25JUkQxTvYBEbzd25pIy2uuUNTvXKJB93TAF3V+JpEk7b7Wljczqf6PXw+00+kwhuUBCflF4cse7Oi128DheEYOeNHdlkz8l1doCTvtU7hC2u/tiuVfFV0XDe380y5x4VvJzhrJtVLKEUcCtBpejKAL8HPDAgnMeHgI6mfos/sGHFJ4NtPPf/SNPYtcvL+BkRYiCPwSP8UWYQCdvSTWcvqVZmQI5G4p/KsHDIQnvtOukBSqM92mqVEMfGdXIhPbWyDCafS01KujnVd46HHjEpdmQC9BnbZryZgghagfGrNKuprFnHidKh1iVv5ygzlBS0+bxMhf51fnTwyvcGvNf+LDMdDfqoEdhrt8S0JRSLGyThCEX+oSt7IkGRhz1LNOHHKjKAnQJakLeTZ2coDfbQ0DSpgYrWseJQr0z7YU0Ath4hieez9PdMGiC4F6Cr8QbpBd6qZYLnkLjKQHPOAko4c5wxbr1d2vJABiS7DaoK+voI= X-Forefront-Antispam-Report: CIP:216.228.117.160;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:dc6edge1.nvidia.com;CAT:NONE;SFS:(13230040)(7416014)(376014)(1800799024)(82310400026)(36860700016)(22082099003)(18002099003)(56012099003);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: auWDZA/4UV9b5cElqL/sQdc9yfrvHj8SQm8Mq54YyGwTFcfwtNAz80msu3qv/IroMvBNblrUUxzlnZWFXmbLOyogc2TxrCXdh3110zNmEyt7IYnwoMikz/uBT5ksbgbiX5VUXyUOUyP3aO/jF5ix252FU4rAQM+I5uAggzhZ35JtdEjsdQ50cnReAtayl6wLvNK9E3WC0DW69KPewghos+P3KjiW26xp9ogvmE3Q25/vY7hXKxY0yfQpC7bL6T9lV0bCI65e4xEsDDXFN0de6MOV6bwPO/EfN+0RLvmI8g0UIf98j9DUFGHIf7Q57ETWo7T6ktFlMOj17orvgT43HBev8VNCPqHnA2+DYKJv3hFkvA7zCUC1uHRPGLKT3tXzaF7EDQARbsm/NoTJa9IUXxmCW7AtZf7AU944i99HPjW7kAmQIKMgQCFgNy3jsIBs X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 01 May 2026 04:17:45.3277 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 32cedbe3-5287-4e22-e503-08dea73898ad X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.117.160];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: SA2PEPF000015C6.namprd03.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DS2PR12MB9615 From: Mark Bloch Representor reload during LAG/MPESW transitions has to be repeated in several flows, and each open=E2=80=91coded loop was easy to get out of sync when adding new flags or tweaking error handling. Move the sequencing into a single helper so that all call sites share the same ordering and checks Signed-off-by: Mark Bloch Reviewed-by: Shay Drori Signed-off-by: Tariq Toukan --- .../net/ethernet/mellanox/mlx5/core/lag/lag.c | 45 +++++++++++-------- .../net/ethernet/mellanox/mlx5/core/lag/lag.h | 2 + .../ethernet/mellanox/mlx5/core/lag/mpesw.c | 12 ++--- 3 files changed, 33 insertions(+), 26 deletions(-) diff --git a/drivers/net/ethernet/mellanox/mlx5/core/lag/lag.c b/drivers/ne= t/ethernet/mellanox/mlx5/core/lag/lag.c index 449e4bd86c06..a474f970e056 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/lag/lag.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/lag/lag.c @@ -1093,6 +1093,27 @@ void mlx5_lag_remove_devices(struct mlx5_lag *ldev) } } =20 +int mlx5_lag_reload_ib_reps(struct mlx5_lag *ldev, u32 flags, bool cont_on= _fail) +{ + struct lag_func *pf; + int ret; + int i; + + mlx5_ldev_for_each(i, 0, ldev) { + pf =3D mlx5_lag_pf(ldev, i); + if (!(pf->dev->priv.flags & flags)) { + struct mlx5_eswitch *esw; + + esw =3D pf->dev->priv.eswitch; + ret =3D mlx5_eswitch_reload_ib_reps(esw); + if (ret && !cont_on_fail) + return ret; + } + } + + return 0; +} + void mlx5_disable_lag(struct mlx5_lag *ldev) { bool shared_fdb =3D test_bit(MLX5_LAG_MODE_FLAG_SHARED_FDB, &ldev->mode_f= lags); @@ -1130,9 +1151,8 @@ void mlx5_disable_lag(struct mlx5_lag *ldev) mlx5_lag_add_devices(ldev); =20 if (shared_fdb) - mlx5_ldev_for_each(i, 0, ldev) - if (!(mlx5_lag_pf(ldev, i)->dev->priv.flags & MLX5_PRIV_FLAGS_DISABLE_A= LL_ADEV)) - mlx5_eswitch_reload_ib_reps(mlx5_lag_pf(ldev, i)->dev->priv.eswitch); + mlx5_lag_reload_ib_reps(ldev, MLX5_PRIV_FLAGS_DISABLE_ALL_ADEV, + true); } =20 bool mlx5_lag_shared_fdb_supported(struct mlx5_lag *ldev) @@ -1388,10 +1408,8 @@ static void mlx5_do_bond(struct mlx5_lag *ldev) if (err) { if (shared_fdb || roce_lag) mlx5_lag_add_devices(ldev); - if (shared_fdb) { - mlx5_ldev_for_each(i, 0, ldev) - mlx5_eswitch_reload_ib_reps(mlx5_lag_pf(ldev, i)->dev->priv.eswitch); - } + if (shared_fdb) + mlx5_lag_reload_ib_reps(ldev, 0, true); =20 return; } @@ -1409,24 +1427,15 @@ static void mlx5_do_bond(struct mlx5_lag *ldev) mlx5_nic_vport_enable_roce(dev); } } else if (shared_fdb) { - int i; - dev0->priv.flags &=3D ~MLX5_PRIV_FLAGS_DISABLE_IB_ADEV; mlx5_rescan_drivers_locked(dev0); - - mlx5_ldev_for_each(i, 0, ldev) { - err =3D mlx5_eswitch_reload_ib_reps(mlx5_lag_pf(ldev, i)->dev->priv.es= witch); - if (err) - break; - } - + err =3D mlx5_lag_reload_ib_reps(ldev, 0, false); if (err) { dev0->priv.flags |=3D MLX5_PRIV_FLAGS_DISABLE_IB_ADEV; mlx5_rescan_drivers_locked(dev0); mlx5_deactivate_lag(ldev); mlx5_lag_add_devices(ldev); - mlx5_ldev_for_each(i, 0, ldev) - mlx5_eswitch_reload_ib_reps(mlx5_lag_pf(ldev, i)->dev->priv.eswitch); + mlx5_lag_reload_ib_reps(ldev, 0, true); mlx5_core_err(dev0, "Failed to enable lag\n"); return; } diff --git a/drivers/net/ethernet/mellanox/mlx5/core/lag/lag.h b/drivers/ne= t/ethernet/mellanox/mlx5/core/lag/lag.h index 6c911374f409..daca8ebd5256 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/lag/lag.h +++ b/drivers/net/ethernet/mellanox/mlx5/core/lag/lag.h @@ -199,4 +199,6 @@ int mlx5_get_next_ldev_func(struct mlx5_lag *ldev, int = start_idx); int mlx5_lag_get_dev_index_by_seq(struct mlx5_lag *ldev, int seq); int mlx5_lag_num_devs(struct mlx5_lag *ldev); int mlx5_lag_num_netdevs(struct mlx5_lag *ldev); +int mlx5_lag_reload_ib_reps(struct mlx5_lag *ldev, u32 flags, + bool cont_on_fail); #endif /* __MLX5_LAG_H__ */ diff --git a/drivers/net/ethernet/mellanox/mlx5/core/lag/mpesw.c b/drivers/= net/ethernet/mellanox/mlx5/core/lag/mpesw.c index 5eea12a6887a..edcd06f3be7a 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/lag/mpesw.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/lag/mpesw.c @@ -70,7 +70,6 @@ static int mlx5_lag_enable_mpesw(struct mlx5_lag *ldev) int idx =3D mlx5_lag_get_dev_index_by_seq(ldev, MLX5_LAG_P1); struct mlx5_core_dev *dev0; int err; - int i; =20 if (ldev->mode =3D=3D MLX5_LAG_MODE_MPESW) return 0; @@ -103,11 +102,9 @@ static int mlx5_lag_enable_mpesw(struct mlx5_lag *ldev) =20 dev0->priv.flags &=3D ~MLX5_PRIV_FLAGS_DISABLE_IB_ADEV; mlx5_rescan_drivers_locked(dev0); - mlx5_ldev_for_each(i, 0, ldev) { - err =3D mlx5_eswitch_reload_ib_reps(mlx5_lag_pf(ldev, i)->dev->priv.eswi= tch); - if (err) - goto err_rescan_drivers; - } + err =3D mlx5_lag_reload_ib_reps(ldev, 0, false); + if (err) + goto err_rescan_drivers; =20 mlx5_lag_set_vports_agg_speed(ldev); =20 @@ -119,8 +116,7 @@ static int mlx5_lag_enable_mpesw(struct mlx5_lag *ldev) mlx5_deactivate_lag(ldev); err_add_devices: mlx5_lag_add_devices(ldev); - mlx5_ldev_for_each(i, 0, ldev) - mlx5_eswitch_reload_ib_reps(mlx5_lag_pf(ldev, i)->dev->priv.eswitch); + mlx5_lag_reload_ib_reps(ldev, 0, true); mlx5_mpesw_metadata_cleanup(ldev); return err; } --=20 2.44.0 From nobody Thu Jun 18 08:01:02 2026 Received: from SN4PR2101CU001.outbound.protection.outlook.com (mail-southcentralusazon11012048.outbound.protection.outlook.com [40.93.195.48]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 862352DEA89; Fri, 1 May 2026 04:17:56 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.93.195.48 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777609078; cv=fail; b=jRhT7cnMkD1gbetzdHD3KmkSiIlhS4eo5E7b8+KAZxIWFQrunQ7/PLUPCQCF1dWJpwywLFf1JRz+4vgO63YGN3HTXRqhixX31jDk/QF+TbFnXH7IYpKLr0WfOwpxmHK4oEMBNWyDytvRRoSGGDyV025N2F406Rp3qLWxXqCHeDg= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777609078; c=relaxed/simple; bh=UZ2Cr3jMR6Oh0M51DlIO2rm8B4yYRl2uCpZ3xHgoXI0=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=p/yQ+LpJkjjsGy+lcQSosOoGlijX7DFEeFdOMNY+9+8x4TJDdeqSCuToiBOFXZ0aeIKO5IRxADUKdpXJPIAhZ68XZ3ldNT/fM2FPvE0OAR2qWys+1eG2uQyMk+lvx9mqkqZO9rYLD6+uNqDkqxCvRcmGCx/85Br97W0oeNyejFo= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=rmiNGIPz; arc=fail smtp.client-ip=40.93.195.48 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="rmiNGIPz" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=vRSk8Pj2qvtNS0saIb0oey/8m6WmI7FmV2gdaxZm/bixYLXNlLbxzQpMTz0rX0VybVEaCOhi9gomCiwCffKlmS2SbuSjlnpeXy+XbyiksjA240vIBCo45F52ii7infhQhoNzYnT/p0b5Lzare1sEsNqKrH4TIGu5AJDSKXBvGmIXDG3Kx/pbkYkAt1gH9ris1/4zpaE/1A942BR1cfyElUZNZrvXBckiG7xF2RTV/JaN8ruCc9Pzb9U3/xEcwo2hlriYwEUB7V39oyKLaYAZtItzM7yh0uiYoQUZ7CD6Lyn8Ei/rh+KDsiOmlrYvtYqvi6m9H8mL9cs25ZL1dzdanw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=7viYO4/NrXv3vnHbPEhfLcHIKHelDhljDb8aLbW+ez4=; b=AYrz9HEawVpLD4uxhXovC2fAHVs9LvaFPJ64O32oh9mPn01HrxjE73h6fnnUJuyBGJkTu/31CxjD7h5qxI/lTw8UHpfnojvjeFCzkcycZl5Z93sp/Keqjxp5K5wqaR5knVptE8ksAs7I3Jlk7LwuRGrpXSGey9Vtx1QbxhnYaBoniP+/C2AO+2hK1CFNZU4sipC3QdQH8K5dUtUwNcpHWY2YHJReZg51ECww2wcAt01/RNsX+iFOWxcEY4pRudOgSZHZvu9KvVK/MF+XLysEmNBtEhQQY1sjdmcKE5C52ARZoSlYyisbC10nt/AH5676rlFZDUJM4gd+/0WtDmCE3w== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.161) smtp.rcpttodomain=google.com smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=7viYO4/NrXv3vnHbPEhfLcHIKHelDhljDb8aLbW+ez4=; b=rmiNGIPzJZoc215rwNU/jlcsP36ONBiNxIrbVI8yzwVLqyBtZi0SlOOV2DeYNs75/TD07cXR3Amvkh5jh1mwSWPt9CMeDgreJu+yfAf6YcRHnCw5Mt1NF9mLcstWn0H6EuSKbs1oKctNLYWmbIbq66c4XawsqJ4kVn0F8UJo7eHSC7Bwid7oKqZoqrO1WlZGZXHTiaNVOmzbG2PrzJliju7YI4vgV0P7hRddyxHk8Y4Xkrk4LEI7UjuJVzvEiRES+Xet2MnxkAHhiGPI7PtXnXLMr2skmqHIcVcoqF7QZmF0cvgFYcIogb67o6M8oP2VeZa3zMCo/EDJpKSZVu6WIA== Received: from PH7P220CA0088.NAMP220.PROD.OUTLOOK.COM (2603:10b6:510:32c::32) by BY5PR12MB4067.namprd12.prod.outlook.com (2603:10b6:a03:212::17) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9870.23; Fri, 1 May 2026 04:17:50 +0000 Received: from SA2PEPF00003F68.namprd04.prod.outlook.com (2603:10b6:510:32c:cafe::40) by PH7P220CA0088.outlook.office365.com (2603:10b6:510:32c::32) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.9870.21 via Frontend Transport; Fri, 1 May 2026 04:17:50 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.161) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.161 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.161; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.161) by SA2PEPF00003F68.mail.protection.outlook.com (10.167.248.43) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9870.22 via Frontend Transport; Fri, 1 May 2026 04:17:49 +0000 Received: from rnnvmail201.nvidia.com (10.129.68.8) by mail.nvidia.com (10.129.200.67) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.20; Thu, 30 Apr 2026 21:17:35 -0700 Received: from rnnvmail203.nvidia.com (10.129.68.9) by rnnvmail201.nvidia.com (10.129.68.8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.20; Thu, 30 Apr 2026 21:17:34 -0700 Received: from vdi.nvidia.com (10.127.8.10) by mail.nvidia.com (10.129.68.9) with Microsoft SMTP Server id 15.2.2562.20 via Frontend Transport; Thu, 30 Apr 2026 21:17:27 -0700 From: Tariq Toukan To: Eric Dumazet , Jakub Kicinski , Paolo Abeni , Andrew Lunn , "David S. Miller" CC: Leon Romanovsky , Jason Gunthorpe , "Saeed Mahameed" , Tariq Toukan , Mark Bloch , Shay Drory , Or Har-Toov , Edward Srouji , Maher Sanalla , Simon Horman , Gerd Bayer , Moshe Shemesh , Kees Cook , Patrisious Haddad , Parav Pandit , Carolina Jubran , Cosmin Ratiu , , , , Gal Pressman , Dragos Tatulea Subject: [PATCH net-next V2 2/7] net/mlx5: E-Switch, add representor lifecycle lock Date: Fri, 1 May 2026 07:16:28 +0300 Message-ID: <20260501041633.231662-3-tariqt@nvidia.com> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20260501041633.231662-1-tariqt@nvidia.com> References: <20260501041633.231662-1-tariqt@nvidia.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-NV-OnPremToCloud: ExternallySecured X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: SA2PEPF00003F68:EE_|BY5PR12MB4067:EE_ X-MS-Office365-Filtering-Correlation-Id: 8a4c6d2c-1093-4852-f6cf-08dea7389b5e X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|376014|7416014|82310400026|36860700016|1800799024|56012099003|18002099003|22082099003; X-Microsoft-Antispam-Message-Info: 52IJ9QYwCwSkcEMshaEeqxAQ7ro3J576PCaJK/o/AEpUmYQd3psxaesZIW4veReMDu6h2YCsZcTX0AaLGz+T9VA4RUdWltmQsvT0COfU6/TMRgLcrH12mGFyKHPS6FnG8od+YXUdYorunzjKoRzG/UI/6emr5pWRWHhm9xfbCTBSHet6yK+jsrw31eZEHTEaNdm/QV+J3C8rPf1J7GVhDcP3wSLqcsafBRgEf7QMJh9ktsPUZZ5vqb3MR3HUxCjQMxwgXWcem37b8zt40zZ3g2kmkbjNUijoA03/ZZm5TWv3MqHFGlWXnyd3WU9h/zdY4Qu5FX0bNdjpUwykqY4F5IDvFDbnrRZuC95M/BkYKM+qlghlkjUhYqRgm5xQ/KHm0uk+Rs97OkMOVVoCDvXO7W4LJ+U+dMZM9y55dk2OSvtt3Cqlv8X8++LdlKWE8Gfha6x1x4+1yzySINylT4Nm+CSvbU+1iIhnmByV4LlMmh8aUl5FPD5J22Pto33PXw7laDiBU78uyLvvwJ7v9TGFyK+Cx9O2/rWpX+g3MDWz5QaD5zOA3mkCXJW/zeskav14vAcohAYVNHGgJ08c2GCBzU62fjLosr4bhouuHGp62Y3iS9S6U4fZxTwpVHYoKmQDT77Ea40dw7Glm+gdFN4eO5xmBYjqhqIAH1mw1fXO28dwwD1SEUMnbTAIY+91d46kC35DL570q1393zTtwrv9fqHPl9FgPWnHn0KQxVJFWog= X-Forefront-Antispam-Report: CIP:216.228.117.161;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:dc6edge2.nvidia.com;CAT:NONE;SFS:(13230040)(376014)(7416014)(82310400026)(36860700016)(1800799024)(56012099003)(18002099003)(22082099003);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: bnsRa5023++WmmuehtbUTxXQIUw/w4tEM2aXb/pZ8ywgi/M7UKY6mlo6x4onVjAMi33glVKhV25AQ7v0I5BGI5QcuMkaHZq660PY4ffCKz0p0DKOWEjEdHhpnzQgKhHee28+crkLYvEN+Z945OHm2E2XrXkiuf7t9WJMqHQzC+eTeSVGQ59VS51s7c/N9U/U4nMqAsW4qxDp9eSI0hi+bpkhYa73DH3I7S1ByWdX3H4hlLMFykSjir/4pIMTg/z37L/tfpwqvrWImJhblDWW/OOAkYptgorIicgK931IYU7ctBkATPwP4rIdZRLU4rVpmC35oPHQ8Qpb1hOsEKpLRNDsQgaCLOTfX41ijIOt/W1xHLXTWNS9rObOYHCH3Jr7JaT8JW94VImxzx56pFJSFIF40vm6c4aMixYK/x51USo9sTREmpFFtaKQfMk5l+83 X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 01 May 2026 04:17:49.8217 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 8a4c6d2c-1093-4852-f6cf-08dea7389b5e X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.117.161];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: SA2PEPF00003F68.namprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: BY5PR12MB4067 Content-Type: text/plain; charset="utf-8" From: Mark Bloch Add a per-E-Switch mutex for serializing representor lifecycle work and provide small helpers for taking and dropping it. Initialize and destroy the mutex with the E-Switch offloads state. Add the lock and helper API first. Follow-up patches will take the lock in the individual representor lifecycle components. This keeps the functional changes split by component and leaves this patch without intended behavior change, making the series easier to review and bisectable. Signed-off-by: Mark Bloch Signed-off-by: Tariq Toukan --- drivers/net/ethernet/mellanox/mlx5/core/eswitch.h | 6 ++++++ .../ethernet/mellanox/mlx5/core/eswitch_offloads.c | 12 ++++++++++++ 2 files changed, 18 insertions(+) diff --git a/drivers/net/ethernet/mellanox/mlx5/core/eswitch.h b/drivers/ne= t/ethernet/mellanox/mlx5/core/eswitch.h index 2fd601bd102f..3858690e09b4 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/eswitch.h +++ b/drivers/net/ethernet/mellanox/mlx5/core/eswitch.h @@ -316,6 +316,7 @@ struct mlx5_esw_offload { DECLARE_HASHTABLE(termtbl_tbl, 8); struct mutex termtbl_mutex; /* protects termtbl hash */ struct xarray vhca_map; + struct mutex reps_lock; /* protects representor load/unload/register */ const struct mlx5_eswitch_rep_ops *rep_ops[NUM_REP_TYPES]; u8 inline_mode; atomic64_t num_flows; @@ -951,6 +952,8 @@ mlx5_esw_lag_demux_fg_create(struct mlx5_eswitch *esw, struct mlx5_flow_handle * mlx5_esw_lag_demux_rule_create(struct mlx5_eswitch *esw, u16 vport_num, struct mlx5_flow_table *lag_ft); +void mlx5_esw_reps_block(struct mlx5_eswitch *esw); +void mlx5_esw_reps_unblock(struct mlx5_eswitch *esw); #else /* CONFIG_MLX5_ESWITCH */ /* eswitch API stubs */ static inline int mlx5_eswitch_init(struct mlx5_core_dev *dev) { return 0= ; } @@ -1028,6 +1031,9 @@ mlx5_esw_host_functions_enabled(const struct mlx5_cor= e_dev *dev) return true; } =20 +static inline void mlx5_esw_reps_block(struct mlx5_eswitch *esw) {} +static inline void mlx5_esw_reps_unblock(struct mlx5_eswitch *esw) {} + static inline bool mlx5_esw_vport_vhca_id(struct mlx5_eswitch *esw, u16 vportn, u16 *vhca_id) { diff --git a/drivers/net/ethernet/mellanox/mlx5/core/eswitch_offloads.c b/d= rivers/net/ethernet/mellanox/mlx5/core/eswitch_offloads.c index 69ddf56e2fc9..6a5143b63dfd 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/eswitch_offloads.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/eswitch_offloads.c @@ -2413,6 +2413,16 @@ static int esw_create_restore_table(struct mlx5_eswi= tch *esw) return err; } =20 +void mlx5_esw_reps_block(struct mlx5_eswitch *esw) +{ + mutex_lock(&esw->offloads.reps_lock); +} + +void mlx5_esw_reps_unblock(struct mlx5_eswitch *esw) +{ + mutex_unlock(&esw->offloads.reps_lock); +} + static void esw_mode_change(struct mlx5_eswitch *esw, u16 mode) { mlx5_devcom_comp_lock(esw->dev->priv.hca_devcom_comp); @@ -2645,6 +2655,7 @@ static void esw_offloads_cleanup_reps(struct mlx5_esw= itch *esw) mlx5_esw_for_each_rep(esw, i, rep) mlx5_esw_offloads_rep_cleanup(esw, rep); xa_destroy(&esw->offloads.vport_reps); + mutex_destroy(&esw->offloads.reps_lock); } =20 static int esw_offloads_init_reps(struct mlx5_eswitch *esw) @@ -2654,6 +2665,7 @@ static int esw_offloads_init_reps(struct mlx5_eswitch= *esw) int err; =20 xa_init(&esw->offloads.vport_reps); + mutex_init(&esw->offloads.reps_lock); =20 mlx5_esw_for_each_vport(esw, i, vport) { err =3D mlx5_esw_offloads_rep_add(esw, vport); --=20 2.44.0 From nobody Thu Jun 18 08:01:02 2026 Received: from CY3PR05CU001.outbound.protection.outlook.com (mail-westcentralusazon11013051.outbound.protection.outlook.com [40.93.201.51]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E9E782F6184; Fri, 1 May 2026 04:18:05 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.93.201.51 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777609087; cv=fail; b=MsFmco2R+cC4/PJJUNuubwxMjzsPAIAupMONBBaJszN7oLqeuALh3wce1odPt/RRtUfNSg/vDd7U4drVFPcKTOqDw4BZe07xGkMBB8A+FDCPwU35ggbONmUAeyNThBnTImc02rleqqXov2aHpPBqgFMRbx0/J727SlQsZpY3cTk= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777609087; c=relaxed/simple; bh=kRXVOzEuNO1LoWonja6ngg99LOrhbZNA/YXU22pKq7Y=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=lGPCTzjtnV9NgNpVfvgphDGKr7IVdrKAr/6iBtB6noOVzv/X68Yi4+q1HcKnAZNmXsoQ9sZRxQve/Fg4CsmnUUqUaSIBWxMwwwlqs9purA0pnb86XL/tPsRuL1GMSeTnq3DwjS1iuSumumVm/0Nmo5/LxuZsQv4VEhCq2CWGVtg= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=OmPCRlNQ; arc=fail smtp.client-ip=40.93.201.51 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="OmPCRlNQ" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=W/Bd4Rv+z1KK68QhMVHr6eKq3+Leme26uIGSXReUX/R+B6tJYkNuzGVFUT5U8V3tDFJB3c6eGkSTlF07H2pbLkLr2bBE6uFJ8eLODPwpKtMtQwcZcJ4sp4vlvw/KPIhdmIfDLweRJOuNJhjNPDa7zJ1jKm7mjQWuPVmRKs0f2jEX8WuFPg72pwFakXzX2g/Y6eBNNdiLoRLEnRKPSUTsWbnsmSfu+Rpp6fRpousPZ97lruqplK974zpTLqeKOmvvgKhT94ZuoQs6hI7bPJyzTITzK9O/JZgtFqiztnizCh8ZekE/qj4t/FcWbIymBwHNcHOCQO2Do0RIomkjBvKj9w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=vgTZFWlC4b/UIair2f0ioc7IeaDP2a2LXWdZCJ5DuK4=; b=ZuQmB24nXPdH6EgHg7bSHFXzk1BKWhk0KDV02WHHvmOShl8PAEz1jZwFXZIgx0qD6PBnu09Nd/B0hH0y2oXMxaZNrTIlDy45Ngab0nEBCkDC0OZB1V8/344J6gdzx77lkEMz2P6B4W18iWcmDgnVh6nzKedxPwTpgw8KSH/NXYzl13dNbzy2z4cUxT7WNvSX27cHFYYv6j4uZsAsd7NkPQ3z9OFpGt+6j2GrLJ6EnV39nE6TvQXZGkZAuHm7MKEBw/45PX9tSrul3vXPEa2gWmHZr02lhjZKgl9qccZSItkJPlVYcwVspm0eg8d1k1VCk5By2lIfhPNbfKfVfYLz5w== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.161) smtp.rcpttodomain=google.com smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=vgTZFWlC4b/UIair2f0ioc7IeaDP2a2LXWdZCJ5DuK4=; b=OmPCRlNQhHnFQA86X4E4O2kLr/PYGgvjt5XxmSJPJ3UdSr97h/JJKIguOwvfcMrJRuZcZOFEJrfrlYx8LSSJq8lv4J57tAh7f0wo1RZs+hiE4NwtkPd+S4+20T6BWpGVXFNFCQ2dBkMXWnXzdkGxr5g+z7TnpOi7EiDR5ZH0JVA1+VqM0qSjKmFDjORGs3xtTklcTZt3GQMmMvF55yOS0dQSWtOANw04CKeZZaDJ+EydGhcCcQx7DXCfeRIDNOU0uHCEd+TW44hXTgkjGsPm4Wu5bPBgU4XVEDlIF/SwvkpYE97tRgzcU1eq9eZLOct0rDvCyDQwNb6QS8XBDGsgcg== Received: from SN1PR12CA0067.namprd12.prod.outlook.com (2603:10b6:802:20::38) by MN2PR12MB4176.namprd12.prod.outlook.com (2603:10b6:208:1d5::20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9870.22; Fri, 1 May 2026 04:17:59 +0000 Received: from SA2PEPF00003F66.namprd04.prod.outlook.com (2603:10b6:802:20:cafe::84) by SN1PR12CA0067.outlook.office365.com (2603:10b6:802:20::38) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.9870.21 via Frontend Transport; Fri, 1 May 2026 04:17:59 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.161) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.161 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.161; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.161) by SA2PEPF00003F66.mail.protection.outlook.com (10.167.248.41) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9891.9 via Frontend Transport; Fri, 1 May 2026 04:17:58 +0000 Received: from rnnvmail203.nvidia.com (10.129.68.9) by mail.nvidia.com (10.129.200.67) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.20; Thu, 30 Apr 2026 21:17:42 -0700 Received: from rnnvmail203.nvidia.com (10.129.68.9) by rnnvmail203.nvidia.com (10.129.68.9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.20; Thu, 30 Apr 2026 21:17:42 -0700 Received: from vdi.nvidia.com (10.127.8.10) by mail.nvidia.com (10.129.68.9) with Microsoft SMTP Server id 15.2.2562.20 via Frontend Transport; Thu, 30 Apr 2026 21:17:35 -0700 From: Tariq Toukan To: Eric Dumazet , Jakub Kicinski , Paolo Abeni , Andrew Lunn , "David S. Miller" CC: Leon Romanovsky , Jason Gunthorpe , "Saeed Mahameed" , Tariq Toukan , Mark Bloch , Shay Drory , Or Har-Toov , Edward Srouji , Maher Sanalla , Simon Horman , Gerd Bayer , Moshe Shemesh , Kees Cook , Patrisious Haddad , Parav Pandit , Carolina Jubran , Cosmin Ratiu , , , , Gal Pressman , Dragos Tatulea Subject: [PATCH net-next V2 3/7] net/mlx5: Lag, avoid LAG and representor lock cycles Date: Fri, 1 May 2026 07:16:29 +0300 Message-ID: <20260501041633.231662-4-tariqt@nvidia.com> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20260501041633.231662-1-tariqt@nvidia.com> References: <20260501041633.231662-1-tariqt@nvidia.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-NV-OnPremToCloud: ExternallySecured X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: SA2PEPF00003F66:EE_|MN2PR12MB4176:EE_ X-MS-Office365-Filtering-Correlation-Id: fc06be2e-6032-42b0-b197-08dea738a0cf X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|376014|7416014|82310400026|36860700016|1800799024|56012099003|18002099003|22082099003; X-Microsoft-Antispam-Message-Info: wfd4xCD9vuSmM/DZTv1XYAHoFDyelg4rrD9JdRjtUe250xLtAFk9BENIX8hLxmOzyFvMC77xSlFreqpgijeHKuqNqyz5VvQDPKw2vmohScmdYL/U81vv28CfGacbo+3PmGnWLYmmxsaEhUPBfhS595Ih+C2/yT6u27x1Tgvvt48HpA1g6k1uLSkLI1xS1hYtg+mRfHRxGiiT1TvUtyeJZdZq40/5UyrZXn2G9Pq4j4LnEyvdrqo1dqJE428zNW7+ZA2+jbB1S5GiNnf9f/QXCAl58TiD5yYxp+WXtRcC2lc/80SIDAs/EkmVZPa3TOC6NlQmISYxU7b/waMTVsbFslp9oAsbPn34CYAKgAHlTbVAF5t9xuQBpvEFRzhM87qBq+O1EwjWIlKDaga+aLfuDsfAxED4FZOT+Wb7b1MQ1uPC1niUoW1vqpvBEG/acp+d6esRMjGa1dbz+hkEyiSbFR06nocxHg5nvW5qmpChGEt1fEdDvMKPli+neZjj9+bM2lRrCwrrHgvCLmT1Xd2PeoksvH3I3OwNlb0TyqO8KEY2DL2oRzY7lw617YKCKx33riRxsoTJVgVd54qLGvnM+ExxT5evMpV49WU3QwcxrmDU4OzYgA46JWKKj634Y8TllTam6BbnLfvtS3ZzP3CkM8TVf0njiYTx5UR9vrlpQRv/bECGztPorQI3vMKxM8dTbWFCa1qqTcF5EZoaraDgSrgAudYuvLpRUiPAQ07rKBU= X-Forefront-Antispam-Report: CIP:216.228.117.161;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:dc6edge2.nvidia.com;CAT:NONE;SFS:(13230040)(376014)(7416014)(82310400026)(36860700016)(1800799024)(56012099003)(18002099003)(22082099003);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: jz85U8bh0Cfl4E15umQ4DIPAbVgPx9BRp54hsUcP+BLKtJhhQhY0BJz/5PGkYRLdcz/yV2HHggcWorCRtv/TKIFKymul9Szs2LXz44iEZOstyiS/x2xFVaoVBU1wPHzVF6vfaA7ykAXdnKh4AtBSEt6ARljFyxF6rNjQ3a1feaC/IojZFEAZ4pS9rSHp4zam08HmKaGGJpHZ6Uiz11bVjKLscgepTeExgRjRkh5S39CoN7CfFEPXNZNkfXlrOfxGCLPcH05r824C55dzaTG0pInyc8pCBqa/3JfXKKRWYsS0pn+0q5mwbCO/HGumU1G05Q5lEbIjtbcrYUMjZtTtivPztKTDAuldnA2ic4x/w6EaEXXdAV6anfJKw41IGgc4mU14mMo9JT0h9Gqn1msDW+MbhVpE4Vbq83VZ0q+r5RuROOljog8DjVHqcNAt5lDo X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 01 May 2026 04:17:58.9626 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: fc06be2e-6032-42b0-b197-08dea738a0cf X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.117.161];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: SA2PEPF00003F66.namprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR12MB4176 Content-Type: text/plain; charset="utf-8" From: Mark Bloch The LAG shared-FDB and multiport E-Switch transitions rescan auxiliary devices and reload IB representors while holding ldev->lock. Driver bind/unbind paths may register or unregister E-Switch representor ops, and representor load paths may enter LAG code, so holding ldev->lock across those calls creates lock-order cycles with the E-Switch representor lock. Keep the devcom component locked for the transition, but drop ldev->lock before rescanning auxiliary devices or reloading IB representors. Mark the LAG transition as in progress while the lock is dropped and assert the devcom lock where the helper relies on it. This preserves LAG serialization while avoiding ldev->lock nesting under E-Switch representor registration. Signed-off-by: Mark Bloch Signed-off-by: Tariq Toukan --- .../net/ethernet/mellanox/mlx5/core/lag/lag.c | 142 ++++++++++++++---- .../net/ethernet/mellanox/mlx5/core/lag/lag.h | 7 +- .../ethernet/mellanox/mlx5/core/lag/mpesw.c | 10 +- .../ethernet/mellanox/mlx5/core/lib/devcom.c | 8 + .../ethernet/mellanox/mlx5/core/lib/devcom.h | 1 + 5 files changed, 134 insertions(+), 34 deletions(-) diff --git a/drivers/net/ethernet/mellanox/mlx5/core/lag/lag.c b/drivers/ne= t/ethernet/mellanox/mlx5/core/lag/lag.c index a474f970e056..e77f9931c39c 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/lag/lag.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/lag/lag.c @@ -1063,37 +1063,99 @@ bool mlx5_lag_check_prereq(struct mlx5_lag *ldev) return true; } =20 -void mlx5_lag_add_devices(struct mlx5_lag *ldev) +static void mlx5_lag_assert_locked_transition(struct mlx5_lag *ldev) { + struct mlx5_devcom_comp_dev *devcom =3D NULL; struct lag_func *pf; int i; =20 - mlx5_ldev_for_each(i, 0, ldev) { - pf =3D mlx5_lag_pf(ldev, i); - if (pf->dev->priv.flags & MLX5_PRIV_FLAGS_DISABLE_ALL_ADEV) - continue; + lockdep_assert_held(&ldev->lock); =20 - pf->dev->priv.flags &=3D ~MLX5_PRIV_FLAGS_DISABLE_IB_ADEV; - mlx5_rescan_drivers_locked(pf->dev); + i =3D mlx5_get_next_ldev_func(ldev, 0); + if (i < MLX5_MAX_PORTS) { + pf =3D mlx5_lag_pf(ldev, i); + devcom =3D pf->dev->priv.hca_devcom_comp; } + mlx5_devcom_comp_assert_locked(devcom); } =20 -void mlx5_lag_remove_devices(struct mlx5_lag *ldev) +static void mlx5_lag_drop_lock_for_reps(struct mlx5_lag *ldev) +{ + mlx5_lag_assert_locked_transition(ldev); + + /* Keep PF membership stable while ldev->lock is dropped. Device add + * and remove paths observe mode_changes_in_progress and retry. + */ + ldev->mode_changes_in_progress++; + mutex_unlock(&ldev->lock); +} + +static void mlx5_lag_retake_lock_after_reps(struct mlx5_lag *ldev) { + mutex_lock(&ldev->lock); + ldev->mode_changes_in_progress--; +} + +void mlx5_lag_rescan_dev_locked(struct mlx5_lag *ldev, + struct mlx5_core_dev *dev, + bool enable) +{ + if (dev->priv.flags & MLX5_PRIV_FLAGS_DISABLE_ALL_ADEV) + return; + + if (enable) + dev->priv.flags &=3D ~MLX5_PRIV_FLAGS_DISABLE_IB_ADEV; + else + dev->priv.flags |=3D MLX5_PRIV_FLAGS_DISABLE_IB_ADEV; + + /* Auxiliary bus probe/remove can register or unregister representor + * callbacks and take reps_lock. Drop ldev->lock so the only ordering + * remains reps_lock -> ldev->lock from representor callbacks. + */ + mlx5_lag_drop_lock_for_reps(ldev); + mlx5_rescan_drivers_locked(dev); + mlx5_lag_retake_lock_after_reps(ldev); +} + +static void mlx5_lag_rescan_devices_locked(struct mlx5_lag *ldev, bool ena= ble) +{ + struct mlx5_core_dev *devs[MLX5_MAX_PORTS]; struct lag_func *pf; + int num_devs =3D 0; int i; =20 + mlx5_lag_assert_locked_transition(ldev); + mlx5_ldev_for_each(i, 0, ldev) { pf =3D mlx5_lag_pf(ldev, i); if (pf->dev->priv.flags & MLX5_PRIV_FLAGS_DISABLE_ALL_ADEV) continue; =20 - pf->dev->priv.flags |=3D MLX5_PRIV_FLAGS_DISABLE_IB_ADEV; - mlx5_rescan_drivers_locked(pf->dev); + if (enable) + pf->dev->priv.flags &=3D ~MLX5_PRIV_FLAGS_DISABLE_IB_ADEV; + else + pf->dev->priv.flags |=3D MLX5_PRIV_FLAGS_DISABLE_IB_ADEV; + devs[num_devs++] =3D pf->dev; } + + mlx5_lag_drop_lock_for_reps(ldev); + for (i =3D 0; i < num_devs; i++) + mlx5_rescan_drivers_locked(devs[i]); + mlx5_lag_retake_lock_after_reps(ldev); } =20 -int mlx5_lag_reload_ib_reps(struct mlx5_lag *ldev, u32 flags, bool cont_on= _fail) +void mlx5_lag_add_devices(struct mlx5_lag *ldev) +{ + mlx5_lag_rescan_devices_locked(ldev, true); +} + +void mlx5_lag_remove_devices(struct mlx5_lag *ldev) +{ + mlx5_lag_rescan_devices_locked(ldev, false); +} + +static int mlx5_lag_reload_ib_reps_unlocked(struct mlx5_lag *ldev, u32 fla= gs, + bool cont_on_fail) { struct lag_func *pf; int ret; @@ -1105,7 +1167,9 @@ int mlx5_lag_reload_ib_reps(struct mlx5_lag *ldev, u3= 2 flags, bool cont_on_fail) struct mlx5_eswitch *esw; =20 esw =3D pf->dev->priv.eswitch; + mlx5_esw_reps_block(esw); ret =3D mlx5_eswitch_reload_ib_reps(esw); + mlx5_esw_reps_unblock(esw); if (ret && !cont_on_fail) return ret; } @@ -1114,6 +1178,34 @@ int mlx5_lag_reload_ib_reps(struct mlx5_lag *ldev, u= 32 flags, bool cont_on_fail) return 0; } =20 +static int mlx5_lag_reload_ib_reps(struct mlx5_lag *ldev, u32 flags, + bool cont_on_fail) +{ + int ret; + + /* The HCA devcom component lock serializes LAG mode transitions while + * ldev->lock is dropped here. Dropping ldev->lock is required because + * the reload takes the per-E-Switch reps_lock, and representor + * load/unload callbacks can re-enter LAG netdev add/remove and take + * ldev->lock. Keep the ordering reps_lock -> ldev->lock. + */ + mlx5_lag_drop_lock_for_reps(ldev); + ret =3D mlx5_lag_reload_ib_reps_unlocked(ldev, flags, cont_on_fail); + mlx5_lag_retake_lock_after_reps(ldev); + + return ret; +} + +int mlx5_lag_reload_ib_reps_from_locked(struct mlx5_lag *ldev, u32 flags, + bool cont_on_fail) +{ + int ret; + + ret =3D mlx5_lag_reload_ib_reps(ldev, flags, cont_on_fail); + + return ret; +} + void mlx5_disable_lag(struct mlx5_lag *ldev) { bool shared_fdb =3D test_bit(MLX5_LAG_MODE_FLAG_SHARED_FDB, &ldev->mode_f= lags); @@ -1132,10 +1224,7 @@ void mlx5_disable_lag(struct mlx5_lag *ldev) if (shared_fdb) { mlx5_lag_remove_devices(ldev); } else if (roce_lag) { - if (!(dev0->priv.flags & MLX5_PRIV_FLAGS_DISABLE_ALL_ADEV)) { - dev0->priv.flags |=3D MLX5_PRIV_FLAGS_DISABLE_IB_ADEV; - mlx5_rescan_drivers_locked(dev0); - } + mlx5_lag_rescan_dev_locked(ldev, dev0, false); mlx5_ldev_for_each(i, 0, ldev) { if (i =3D=3D idx) continue; @@ -1151,8 +1240,9 @@ void mlx5_disable_lag(struct mlx5_lag *ldev) mlx5_lag_add_devices(ldev); =20 if (shared_fdb) - mlx5_lag_reload_ib_reps(ldev, MLX5_PRIV_FLAGS_DISABLE_ALL_ADEV, - true); + mlx5_lag_reload_ib_reps_from_locked(ldev, + MLX5_PRIV_FLAGS_DISABLE_ALL_ADEV, + true); } =20 bool mlx5_lag_shared_fdb_supported(struct mlx5_lag *ldev) @@ -1409,7 +1499,8 @@ static void mlx5_do_bond(struct mlx5_lag *ldev) if (shared_fdb || roce_lag) mlx5_lag_add_devices(ldev); if (shared_fdb) - mlx5_lag_reload_ib_reps(ldev, 0, true); + mlx5_lag_reload_ib_reps_from_locked(ldev, 0, + true); =20 return; } @@ -1417,8 +1508,7 @@ static void mlx5_do_bond(struct mlx5_lag *ldev) if (roce_lag) { struct mlx5_core_dev *dev; =20 - dev0->priv.flags &=3D ~MLX5_PRIV_FLAGS_DISABLE_IB_ADEV; - mlx5_rescan_drivers_locked(dev0); + mlx5_lag_rescan_dev_locked(ldev, dev0, true); mlx5_ldev_for_each(i, 0, ldev) { if (i =3D=3D idx) continue; @@ -1427,15 +1517,15 @@ static void mlx5_do_bond(struct mlx5_lag *ldev) mlx5_nic_vport_enable_roce(dev); } } else if (shared_fdb) { - dev0->priv.flags &=3D ~MLX5_PRIV_FLAGS_DISABLE_IB_ADEV; - mlx5_rescan_drivers_locked(dev0); - err =3D mlx5_lag_reload_ib_reps(ldev, 0, false); + mlx5_lag_rescan_dev_locked(ldev, dev0, true); + err =3D mlx5_lag_reload_ib_reps_from_locked(ldev, 0, + false); if (err) { - dev0->priv.flags |=3D MLX5_PRIV_FLAGS_DISABLE_IB_ADEV; - mlx5_rescan_drivers_locked(dev0); + mlx5_lag_rescan_dev_locked(ldev, dev0, false); mlx5_deactivate_lag(ldev); mlx5_lag_add_devices(ldev); - mlx5_lag_reload_ib_reps(ldev, 0, true); + mlx5_lag_reload_ib_reps_from_locked(ldev, 0, + true); mlx5_core_err(dev0, "Failed to enable lag\n"); return; } diff --git a/drivers/net/ethernet/mellanox/mlx5/core/lag/lag.h b/drivers/ne= t/ethernet/mellanox/mlx5/core/lag/lag.h index daca8ebd5256..6afe7707d076 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/lag/lag.h +++ b/drivers/net/ethernet/mellanox/mlx5/core/lag/lag.h @@ -164,6 +164,9 @@ void mlx5_disable_lag(struct mlx5_lag *ldev); void mlx5_lag_remove_devices(struct mlx5_lag *ldev); int mlx5_deactivate_lag(struct mlx5_lag *ldev); void mlx5_lag_add_devices(struct mlx5_lag *ldev); +void mlx5_lag_rescan_dev_locked(struct mlx5_lag *ldev, + struct mlx5_core_dev *dev, + bool enable); struct mlx5_devcom_comp_dev *mlx5_lag_get_devcom_comp(struct mlx5_lag *lde= v); =20 #ifdef CONFIG_MLX5_ESWITCH @@ -199,6 +202,6 @@ int mlx5_get_next_ldev_func(struct mlx5_lag *ldev, int = start_idx); int mlx5_lag_get_dev_index_by_seq(struct mlx5_lag *ldev, int seq); int mlx5_lag_num_devs(struct mlx5_lag *ldev); int mlx5_lag_num_netdevs(struct mlx5_lag *ldev); -int mlx5_lag_reload_ib_reps(struct mlx5_lag *ldev, u32 flags, - bool cont_on_fail); +int mlx5_lag_reload_ib_reps_from_locked(struct mlx5_lag *ldev, u32 flags, + bool cont_on_fail); #endif /* __MLX5_LAG_H__ */ diff --git a/drivers/net/ethernet/mellanox/mlx5/core/lag/mpesw.c b/drivers/= net/ethernet/mellanox/mlx5/core/lag/mpesw.c index edcd06f3be7a..8a349f8fd823 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/lag/mpesw.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/lag/mpesw.c @@ -100,9 +100,8 @@ static int mlx5_lag_enable_mpesw(struct mlx5_lag *ldev) goto err_add_devices; } =20 - dev0->priv.flags &=3D ~MLX5_PRIV_FLAGS_DISABLE_IB_ADEV; - mlx5_rescan_drivers_locked(dev0); - err =3D mlx5_lag_reload_ib_reps(ldev, 0, false); + mlx5_lag_rescan_dev_locked(ldev, dev0, true); + err =3D mlx5_lag_reload_ib_reps_from_locked(ldev, 0, false); if (err) goto err_rescan_drivers; =20 @@ -111,12 +110,11 @@ static int mlx5_lag_enable_mpesw(struct mlx5_lag *lde= v) return 0; =20 err_rescan_drivers: - dev0->priv.flags |=3D MLX5_PRIV_FLAGS_DISABLE_IB_ADEV; - mlx5_rescan_drivers_locked(dev0); + mlx5_lag_rescan_dev_locked(ldev, dev0, false); mlx5_deactivate_lag(ldev); err_add_devices: mlx5_lag_add_devices(ldev); - mlx5_lag_reload_ib_reps(ldev, 0, true); + mlx5_lag_reload_ib_reps_from_locked(ldev, 0, true); mlx5_mpesw_metadata_cleanup(ldev); return err; } diff --git a/drivers/net/ethernet/mellanox/mlx5/core/lib/devcom.c b/drivers= /net/ethernet/mellanox/mlx5/core/lib/devcom.c index 4b5ac2db55ce..d40c53193ea8 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/lib/devcom.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/lib/devcom.c @@ -3,6 +3,7 @@ =20 #include #include +#include #include "lib/devcom.h" #include "lib/mlx5.h" #include "mlx5_core.h" @@ -438,3 +439,10 @@ int mlx5_devcom_comp_trylock(struct mlx5_devcom_comp_d= ev *devcom) return 0; return down_write_trylock(&devcom->comp->sem); } + +void mlx5_devcom_comp_assert_locked(struct mlx5_devcom_comp_dev *devcom) +{ + if (!devcom) + return; + lockdep_assert_held_write(&devcom->comp->sem); +} diff --git a/drivers/net/ethernet/mellanox/mlx5/core/lib/devcom.h b/drivers= /net/ethernet/mellanox/mlx5/core/lib/devcom.h index 91e5ae529d5c..316052a85ca5 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/lib/devcom.h +++ b/drivers/net/ethernet/mellanox/mlx5/core/lib/devcom.h @@ -75,5 +75,6 @@ void *mlx5_devcom_get_next_peer_data_rcu(struct mlx5_devc= om_comp_dev *devcom, void mlx5_devcom_comp_lock(struct mlx5_devcom_comp_dev *devcom); void mlx5_devcom_comp_unlock(struct mlx5_devcom_comp_dev *devcom); int mlx5_devcom_comp_trylock(struct mlx5_devcom_comp_dev *devcom); +void mlx5_devcom_comp_assert_locked(struct mlx5_devcom_comp_dev *devcom); =20 #endif /* __LIB_MLX5_DEVCOM_H__ */ --=20 2.44.0 From nobody Thu Jun 18 08:01:02 2026 Received: from BL0PR03CU003.outbound.protection.outlook.com (mail-eastusazon11012060.outbound.protection.outlook.com [52.101.53.60]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 35343303A35; Fri, 1 May 2026 04:18:19 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=52.101.53.60 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777609101; cv=fail; b=Gr/JNsS05AHrFZjCEg5BF0VTVUOLiVmQFAacLSvG80TU5pXwbJb083hU9pQlpZ1s5q/xREGANFbA+f596oIRhms0gek7LsyjRJ+pjblxwHYLgphPlYpK06h/GscZRUlYj3L5m7ZQ/O6vFWnQBKD3LivqlzYcE/BJk+MC9JWSJaI= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777609101; c=relaxed/simple; bh=JCsxAMgLE7uq7AGozlg/RGrbvvWX8C/9o/nlVMg84x4=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=UvtUh56NIiCyC4IOgOXPt5DZlqu1WFq38OljFlFKJoLr30SheNT10XYu5MCw8OBGzN8MZky8+0i5EIZ6nBXF0v5+T+ejpx8vb+xGaAdfIfcTiNS740RzTUN8dhFHzUP+WEDhOo/+MIjDOKWv84cRBU9ac0wBDrBeFc8aAjcDraA= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=qf6zPVu8; arc=fail smtp.client-ip=52.101.53.60 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="qf6zPVu8" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=PWdm2ShWAXeKPkqF46EY9AY/3uErI986uV7OVx+rIkF/YgE+APitczkyo1hKTAvm73oLhWPLkGjiGwJVnXDIz4Wu1Fx0jBxg/cr4T1HgnqmY2Z3whgnKU3EZ/2iRadX7rEbzP1wc3sx/IpLT6qt3X7RsA5naU4xBqRCwLrdbpLWNK8WgwZQtsHpBzSOVZRvPRK8IZiQUaExKbRx4S0DGmsqkRPla2xv87d22egyY9OUf5Cn608ooeCFM1VmayJi9hXbV63OlUDW6k5YBz6ek30NWCx2UnknV0TQy78qsOgvW+BXtrN/PwQHW48wK29RBuHIn54A3OOgFG2fja1Q98w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=Cmv/C+x6J6gcyHasImi8Fwhl9P8fIBGyN18hIeVFGdQ=; b=NzvjHo8piuo3tTFLi0RrgL0JwINWYJ6iNiNbGkeEINqZlPm8Rauf/2JI354yeYFjbyAryLMxqQN7KWUrN7+GwXPTdrvtdPtFQ0LcinlS1CMAI0MdstDKzYIa2RkjOIKoJTwP5RcjpF54ZGpPnOzW1N0CCIgyriSreMAqT6FVcz1yPm1Eq2KPyjlHPbVwNnvAsOmYgA7MP69cCPpSd2JAy2AYzMhAyRlUty7s/4HOz5dHEkg1EHW8ssVpYh6ZfRYVSXy6ZnS3ymrzCwpk8DozNZuxbKTk3yMy4Jw4GbdgnCYWC8XYE8uSbszEk9Ytn2Ct3yRpVq79eX3M36SQlCQVZg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.161) smtp.rcpttodomain=google.com smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Cmv/C+x6J6gcyHasImi8Fwhl9P8fIBGyN18hIeVFGdQ=; b=qf6zPVu8sQ1BVIziqohORba8FUfejaFljx4dQm8ii4f5atbAjxuw6SPKDp+MuJMXfBGzLTvcllZCS/rvsC+MG+p1boa76UPO7vEql4zZQ60Q0I+vnMkjt8ivH9aziZ6e0GzyXDq7KObEvl6cV/QIyowHkbfsCDe375cm9Hu5sDzVev1pp8FiPXTgHg5dQT9HunOw1XupmiXlMTtI/zFHEAHeJn98QFkN3hTiTWkrYb5/DoXUrvl5iM0eOxnsrc8ijh9ine2TxD+y8Sa58sbz6Z/M8GEicba3L5xJxyFa+CJWRISstTetzcPe0f2a8bQasKFQ64+IVneeIdVj0i+rfg== Received: from CH5PR02CA0005.namprd02.prod.outlook.com (2603:10b6:610:1ed::7) by CY8PR12MB7515.namprd12.prod.outlook.com (2603:10b6:930:93::6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9870.22; Fri, 1 May 2026 04:18:08 +0000 Received: from CH2PEPF000000A0.namprd02.prod.outlook.com (2603:10b6:610:1ed:cafe::ad) by CH5PR02CA0005.outlook.office365.com (2603:10b6:610:1ed::7) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.9870.23 via Frontend Transport; Fri, 1 May 2026 04:18:08 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.161) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.161 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.161; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.161) by CH2PEPF000000A0.mail.protection.outlook.com (10.167.244.26) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9891.9 via Frontend Transport; Fri, 1 May 2026 04:18:08 +0000 Received: from rnnvmail203.nvidia.com (10.129.68.9) by mail.nvidia.com (10.129.200.67) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.20; Thu, 30 Apr 2026 21:17:50 -0700 Received: from rnnvmail203.nvidia.com (10.129.68.9) by rnnvmail203.nvidia.com (10.129.68.9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.20; Thu, 30 Apr 2026 21:17:49 -0700 Received: from vdi.nvidia.com (10.127.8.10) by mail.nvidia.com (10.129.68.9) with Microsoft SMTP Server id 15.2.2562.20 via Frontend Transport; Thu, 30 Apr 2026 21:17:42 -0700 From: Tariq Toukan To: Eric Dumazet , Jakub Kicinski , Paolo Abeni , Andrew Lunn , "David S. Miller" CC: Leon Romanovsky , Jason Gunthorpe , "Saeed Mahameed" , Tariq Toukan , Mark Bloch , Shay Drory , Or Har-Toov , Edward Srouji , Maher Sanalla , Simon Horman , Gerd Bayer , Moshe Shemesh , Kees Cook , Patrisious Haddad , Parav Pandit , Carolina Jubran , Cosmin Ratiu , , , , Gal Pressman , Dragos Tatulea Subject: [PATCH net-next V2 4/7] net/mlx5: E-Switch, serialize representor lifecycle Date: Fri, 1 May 2026 07:16:30 +0300 Message-ID: <20260501041633.231662-5-tariqt@nvidia.com> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20260501041633.231662-1-tariqt@nvidia.com> References: <20260501041633.231662-1-tariqt@nvidia.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-NV-OnPremToCloud: ExternallySecured X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CH2PEPF000000A0:EE_|CY8PR12MB7515:EE_ X-MS-Office365-Filtering-Correlation-Id: 1f4447e4-1b35-4469-b98c-08dea738a634 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|376014|1800799024|7416014|82310400026|36860700016|22082099003|18002099003|56012099003; X-Microsoft-Antispam-Message-Info: Prf3uE0d1VG1z6WQZTYmGGUMvaYgVTwRc/E3pblaRFr/7ovPbYD9iBTaPH+9INMIXbm8Y4riy17mX7pcWZnciV/8PdpF6vZu4fwh293JsTke8ZDKJwRW7dlQZ8LfcVccsSmxxQ6mruUFRdp4AtY3teFK5B55f3HPvkJHZKvEOLKgZqf4Pi4bOqXrMQIorcSq85LWhp7aDc6/ll/zxoxvWWevW+7Lk55PRa2VTNasGKKoXGKnhjvXqd57BkIq35dDrdUPmNRCyygdMWnG2ZHoiYnfj8v/k4tv73Tp9WCkjQNLBqhY76UYmO/C9QQbXZESpCWHm0j72PU/br3O7R/8jcrTF32quN8dL11CfgxGodoFtQs5u8sgH1jGhZehtzIACkB2l/5ZkP/ghpY6pR+YKXU7cCJ7fvR5+TCwpbFBpVn0IFRkpU/sxPpRvuRJZnSKfZpExP4bicKT9d/SU7Q6to1jrJRtQL65wtuf3TljYxj2jjtad73lIQoygaAinP+FitXiceRRaEVuz5iEi5rYEhHFMCZfEBMnSuXzvKosNsW9u/V0ulJvSXVYrg7JbLtcCU+rmKXPPvbzAP6szd15n88TOKNZwgZRlajmSZtMevjjspPYTvIubYdJERzwSNI4s4t/JnbTA29V5XKytWTQjnafKrZwa5UIG3OvPpyfD9yY4e0B7PCcYZOVAId6Bob0aknIOrwgdvn4avi92F7ns8jpwKNiOgpvRFO9H8qngpI= X-Forefront-Antispam-Report: CIP:216.228.117.161;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:dc6edge2.nvidia.com;CAT:NONE;SFS:(13230040)(376014)(1800799024)(7416014)(82310400026)(36860700016)(22082099003)(18002099003)(56012099003);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: xP4e601LJwqf0HGssNezcvccREdZAlL4+tVK+KNB2frWJ1wHhDcNI5/nx+McOn5VuQoWhA+yNzF2aY3Lo+slSijuLAGRG8PuR+PBI23aO8OlOOInRhGCzAQfunpVHl3plklsXvmVo/Hc20Cg98Myi4QGSkt/js7H6sCO/JZU2qoeFDNY8If8UFD5t6K0b8Rsr0oQDzQzkgY+iC9i6I8ZW7oyNyjof749bDCWl17K93dmDaLJB2jxlqeSLNbjcb/c2LmT2adzH4aE0T3i9/R4hrTRnxRe5wngA86RcTo23YlQ8+Yu+eXqDUkh5O66PbtEI1ui2XUkIo+CzFvpqvISiz3bXIIxcvsCFWX3Den1OTVxDgqfMJGZcqFcMCfoZLUY15r+NxWwO/ulFvE7HUkX0EJH9ZmgfxV74Nc2a7Ypa/mw/dEO2l5fWIj+NqDH4vVg X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 01 May 2026 04:18:08.0079 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 1f4447e4-1b35-4469-b98c-08dea738a634 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.117.161];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: CH2PEPF000000A0.namprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY8PR12MB7515 Content-Type: text/plain; charset="utf-8" From: Mark Bloch Representor callbacks can be registered and unregistered while the E-Switch is already in switchdev mode, and the same E-Switch may also be reconfigured by devlink, VF changes and SF changes. Serialize these paths with the per-E-Switch representor mutex instead of relying on ad-hoc bit state and wait queues. Take the representor lock around the mode transition, VF/SF representor changes and representor ops registration. Keep mode_lock and the representor lock unnested by using the operation flag while the mode lock is dropped. During mode changes, drop the representor lock around the auxiliary bus rescan because driver bind/unbind may register or unregister representor ops. Split representor ops registration into locked public wrappers and blocked internal helpers, clear the ops pointer on unregister, and add nested wrappers for the shared-FDB master IB path that registers peer representor ops while another E-Switch representor lock is already held. Signed-off-by: Mark Bloch Signed-off-by: Tariq Toukan --- drivers/infiniband/hw/mlx5/ib_rep.c | 6 +- .../net/ethernet/mellanox/mlx5/core/eswitch.c | 10 ++ .../mellanox/mlx5/core/eswitch_offloads.c | 102 ++++++++++++++++-- .../ethernet/mellanox/mlx5/core/sf/devlink.c | 5 + include/linux/mlx5/eswitch.h | 6 ++ 5 files changed, 119 insertions(+), 10 deletions(-) diff --git a/drivers/infiniband/hw/mlx5/ib_rep.c b/drivers/infiniband/hw/ml= x5/ib_rep.c index 1709b628702e..65d8767d1830 100644 --- a/drivers/infiniband/hw/mlx5/ib_rep.c +++ b/drivers/infiniband/hw/mlx5/ib_rep.c @@ -262,9 +262,10 @@ mlx5_ib_vport_rep_unload(struct mlx5_eswitch_rep *rep) struct mlx5_core_dev *peer_mdev; struct mlx5_eswitch *esw; =20 + /* Called while the master E-Switch reps_lock is held. */ mlx5_lag_for_each_peer_mdev(mdev, peer_mdev, i) { esw =3D peer_mdev->priv.eswitch; - mlx5_eswitch_unregister_vport_reps(esw, REP_IB); + mlx5_eswitch_unregister_vport_reps_nested(esw, REP_IB); } mlx5_ib_release_transport(mdev); } @@ -284,9 +285,10 @@ static void mlx5_ib_register_peer_vport_reps(struct ml= x5_core_dev *mdev) struct mlx5_eswitch *esw; int i; =20 + /* Called while the master E-Switch reps_lock is held. */ mlx5_lag_for_each_peer_mdev(mdev, peer_mdev, i) { esw =3D peer_mdev->priv.eswitch; - mlx5_eswitch_register_vport_reps(esw, &rep_ops, REP_IB); + mlx5_eswitch_register_vport_reps_nested(esw, &rep_ops, REP_IB); } } =20 diff --git a/drivers/net/ethernet/mellanox/mlx5/core/eswitch.c b/drivers/ne= t/ethernet/mellanox/mlx5/core/eswitch.c index 66a773a99876..f70737437954 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/eswitch.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/eswitch.c @@ -1712,6 +1712,7 @@ int mlx5_eswitch_enable(struct mlx5_eswitch *esw, int= num_vfs) mlx5_lag_disable_change(esw->dev); =20 mlx5_eswitch_invalidate_wq(esw); + mlx5_esw_reps_block(esw); =20 if (!mlx5_esw_is_fdb_created(esw)) { ret =3D mlx5_eswitch_enable_locked(esw, num_vfs); @@ -1735,6 +1736,8 @@ int mlx5_eswitch_enable(struct mlx5_eswitch *esw, int= num_vfs) } } =20 + mlx5_esw_reps_unblock(esw); + if (toggle_lag) mlx5_lag_enable_change(esw->dev); =20 @@ -1759,6 +1762,7 @@ void mlx5_eswitch_disable_sriov(struct mlx5_eswitch *= esw, bool clear_vf) esw->esw_funcs.num_vfs, esw->esw_funcs.num_ec_vfs, esw->enabled_vports); =20 mlx5_eswitch_invalidate_wq(esw); + mlx5_esw_reps_block(esw); =20 if (!mlx5_core_is_ecpf(esw->dev)) { mlx5_eswitch_unload_vf_vports(esw, esw->esw_funcs.num_vfs); @@ -1770,6 +1774,8 @@ void mlx5_eswitch_disable_sriov(struct mlx5_eswitch *= esw, bool clear_vf) mlx5_eswitch_clear_ec_vf_vports_info(esw); } =20 + mlx5_esw_reps_unblock(esw); + if (esw->mode =3D=3D MLX5_ESWITCH_OFFLOADS) { struct devlink *devlink =3D priv_to_devlink(esw->dev); =20 @@ -1825,7 +1831,11 @@ void mlx5_eswitch_disable(struct mlx5_eswitch *esw) =20 devl_assert_locked(priv_to_devlink(esw->dev)); mlx5_lag_disable_change(esw->dev); + + mlx5_esw_reps_block(esw); mlx5_eswitch_disable_locked(esw); + mlx5_esw_reps_unblock(esw); + esw->mode =3D MLX5_ESWITCH_LEGACY; mlx5_lag_enable_change(esw->dev); } diff --git a/drivers/net/ethernet/mellanox/mlx5/core/eswitch_offloads.c b/d= rivers/net/ethernet/mellanox/mlx5/core/eswitch_offloads.c index 6a5143b63dfd..d4ac07c995b9 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/eswitch_offloads.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/eswitch_offloads.c @@ -36,6 +36,7 @@ #include #include #include +#include #include "mlx5_core.h" #include "eswitch.h" #include "esw/indir_table.h" @@ -2413,11 +2414,21 @@ static int esw_create_restore_table(struct mlx5_esw= itch *esw) return err; } =20 +static void mlx5_esw_assert_reps_locked(struct mlx5_eswitch *esw) +{ + lockdep_assert_held(&esw->offloads.reps_lock); +} + void mlx5_esw_reps_block(struct mlx5_eswitch *esw) { mutex_lock(&esw->offloads.reps_lock); } =20 +static void mlx5_esw_reps_block_nested(struct mlx5_eswitch *esw) +{ + mutex_lock_nested(&esw->offloads.reps_lock, SINGLE_DEPTH_NESTING); +} + void mlx5_esw_reps_unblock(struct mlx5_eswitch *esw) { mutex_unlock(&esw->offloads.reps_lock); @@ -2425,21 +2436,22 @@ void mlx5_esw_reps_unblock(struct mlx5_eswitch *esw) =20 static void esw_mode_change(struct mlx5_eswitch *esw, u16 mode) { + mlx5_esw_reps_unblock(esw); mlx5_devcom_comp_lock(esw->dev->priv.hca_devcom_comp); if (esw->dev->priv.flags & MLX5_PRIV_FLAGS_DISABLE_IB_ADEV || mlx5_core_mp_enabled(esw->dev)) { esw->mode =3D mode; - mlx5_rescan_drivers_locked(esw->dev); - mlx5_devcom_comp_unlock(esw->dev->priv.hca_devcom_comp); - return; + goto out; } =20 esw->dev->priv.flags |=3D MLX5_PRIV_FLAGS_DISABLE_IB_ADEV; mlx5_rescan_drivers_locked(esw->dev); esw->mode =3D mode; esw->dev->priv.flags &=3D ~MLX5_PRIV_FLAGS_DISABLE_IB_ADEV; +out: mlx5_rescan_drivers_locked(esw->dev); mlx5_devcom_comp_unlock(esw->dev->priv.hca_devcom_comp); + mlx5_esw_reps_block(esw); } =20 static void mlx5_esw_fdb_drop_destroy(struct mlx5_eswitch *esw) @@ -2776,6 +2788,8 @@ void esw_offloads_cleanup(struct mlx5_eswitch *esw) static int __esw_offloads_load_rep(struct mlx5_eswitch *esw, struct mlx5_eswitch_rep *rep, u8 rep_type) { + mlx5_esw_assert_reps_locked(esw); + if (atomic_cmpxchg(&rep->rep_data[rep_type].state, REP_REGISTERED, REP_LOADED) =3D=3D REP_REGISTERED) return esw->offloads.rep_ops[rep_type]->load(esw->dev, rep); @@ -2786,6 +2800,8 @@ static int __esw_offloads_load_rep(struct mlx5_eswitc= h *esw, static void __esw_offloads_unload_rep(struct mlx5_eswitch *esw, struct mlx5_eswitch_rep *rep, u8 rep_type) { + mlx5_esw_assert_reps_locked(esw); + if (atomic_cmpxchg(&rep->rep_data[rep_type].state, REP_LOADED, REP_REGISTERED) =3D=3D REP_LOADED) { if (rep_type =3D=3D REP_ETH) @@ -3691,6 +3707,7 @@ static void esw_vfs_changed_event_handler(struct mlx5= _eswitch *esw) if (new_num_vfs =3D=3D esw->esw_funcs.num_vfs || host_pf_disabled) goto free; =20 + mlx5_esw_reps_block(esw); /* Number of VFs can only change from "0 to x" or "x to 0". */ if (esw->esw_funcs.num_vfs > 0) { mlx5_eswitch_unload_vf_vports(esw, esw->esw_funcs.num_vfs); @@ -3700,9 +3717,11 @@ static void esw_vfs_changed_event_handler(struct mlx= 5_eswitch *esw) err =3D mlx5_eswitch_load_vf_vports(esw, new_num_vfs, MLX5_VPORT_UC_ADDR_CHANGE); if (err) - goto free; + goto unblock; } esw->esw_funcs.num_vfs =3D new_num_vfs; +unblock: + mlx5_esw_reps_unblock(esw); free: kvfree(out); } @@ -4188,9 +4207,14 @@ int mlx5_devlink_eswitch_mode_set(struct devlink *de= vlink, u16 mode, goto unlock; } =20 + /* Keep mode_lock and reps_lock unnested. The operation flag excludes + * mode users while mode_lock is dropped before taking reps_lock. + */ esw->eswitch_operation_in_progress =3D true; up_write(&esw->mode_lock); =20 + mlx5_esw_reps_block(esw); + if (mlx5_mode =3D=3D MLX5_ESWITCH_OFFLOADS && !mlx5_devlink_netdev_netns_immutable_set(devlink, true)) { NL_SET_ERR_MSG_MOD(extack, @@ -4223,6 +4247,10 @@ int mlx5_devlink_eswitch_mode_set(struct devlink *de= vlink, u16 mode, skip: if (mlx5_mode =3D=3D MLX5_ESWITCH_OFFLOADS && err) mlx5_devlink_netdev_netns_immutable_set(devlink, false); + /* Reconfiguration is done; drop reps_lock before taking mode_lock again + * to clear the operation flag. + */ + mlx5_esw_reps_unblock(esw); down_write(&esw->mode_lock); esw->eswitch_operation_in_progress =3D false; unlock: @@ -4496,9 +4524,10 @@ mlx5_eswitch_vport_has_rep(const struct mlx5_eswitch= *esw, u16 vport_num) return true; } =20 -void mlx5_eswitch_register_vport_reps(struct mlx5_eswitch *esw, - const struct mlx5_eswitch_rep_ops *ops, - u8 rep_type) +static void +mlx5_eswitch_register_vport_reps_blocked(struct mlx5_eswitch *esw, + const struct mlx5_eswitch_rep_ops *ops, + u8 rep_type) { struct mlx5_eswitch_rep_data *rep_data; struct mlx5_eswitch_rep *rep; @@ -4513,9 +4542,40 @@ void mlx5_eswitch_register_vport_reps(struct mlx5_es= witch *esw, } } } + +static void +mlx5_eswitch_register_vport_reps_locked(struct mlx5_eswitch *esw, + const struct mlx5_eswitch_rep_ops *ops, + u8 rep_type, bool nested) +{ + if (nested) + mlx5_esw_reps_block_nested(esw); + else + mlx5_esw_reps_block(esw); + mlx5_eswitch_register_vport_reps_blocked(esw, ops, rep_type); + mlx5_esw_reps_unblock(esw); +} + +void mlx5_eswitch_register_vport_reps(struct mlx5_eswitch *esw, + const struct mlx5_eswitch_rep_ops *ops, + u8 rep_type) +{ + mlx5_eswitch_register_vport_reps_locked(esw, ops, rep_type, false); +} EXPORT_SYMBOL(mlx5_eswitch_register_vport_reps); =20 -void mlx5_eswitch_unregister_vport_reps(struct mlx5_eswitch *esw, u8 rep_t= ype) +void +mlx5_eswitch_register_vport_reps_nested(struct mlx5_eswitch *esw, + const struct mlx5_eswitch_rep_ops *ops, + u8 rep_type) +{ + mlx5_eswitch_register_vport_reps_locked(esw, ops, rep_type, true); +} +EXPORT_SYMBOL(mlx5_eswitch_register_vport_reps_nested); + +static void +mlx5_eswitch_unregister_vport_reps_blocked(struct mlx5_eswitch *esw, + u8 rep_type) { struct mlx5_eswitch_rep *rep; unsigned long i; @@ -4525,9 +4585,35 @@ void mlx5_eswitch_unregister_vport_reps(struct mlx5_= eswitch *esw, u8 rep_type) =20 mlx5_esw_for_each_rep(esw, i, rep) atomic_set(&rep->rep_data[rep_type].state, REP_UNREGISTERED); + + esw->offloads.rep_ops[rep_type] =3D NULL; +} + +static void +mlx5_eswitch_unregister_vport_reps_locked(struct mlx5_eswitch *esw, + u8 rep_type, bool nested) +{ + if (nested) + mlx5_esw_reps_block_nested(esw); + else + mlx5_esw_reps_block(esw); + mlx5_eswitch_unregister_vport_reps_blocked(esw, rep_type); + mlx5_esw_reps_unblock(esw); +} + +void mlx5_eswitch_unregister_vport_reps(struct mlx5_eswitch *esw, u8 rep_t= ype) +{ + mlx5_eswitch_unregister_vport_reps_locked(esw, rep_type, false); } EXPORT_SYMBOL(mlx5_eswitch_unregister_vport_reps); =20 +void mlx5_eswitch_unregister_vport_reps_nested(struct mlx5_eswitch *esw, + u8 rep_type) +{ + mlx5_eswitch_unregister_vport_reps_locked(esw, rep_type, true); +} +EXPORT_SYMBOL(mlx5_eswitch_unregister_vport_reps_nested); + void *mlx5_eswitch_get_uplink_priv(struct mlx5_eswitch *esw, u8 rep_type) { struct mlx5_eswitch_rep *rep; diff --git a/drivers/net/ethernet/mellanox/mlx5/core/sf/devlink.c b/drivers= /net/ethernet/mellanox/mlx5/core/sf/devlink.c index 8503e532f423..2fc69897e35b 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/sf/devlink.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/sf/devlink.c @@ -245,8 +245,10 @@ static int mlx5_sf_add(struct mlx5_core_dev *dev, stru= ct mlx5_sf_table *table, if (IS_ERR(sf)) return PTR_ERR(sf); =20 + mlx5_esw_reps_block(esw); err =3D mlx5_eswitch_load_sf_vport(esw, sf->hw_fn_id, MLX5_VPORT_UC_ADDR_= CHANGE, &sf->dl_port, new_attr->controller, new_attr->sfnum); + mlx5_esw_reps_unblock(esw); if (err) goto esw_err; *dl_port =3D &sf->dl_port.dl_port; @@ -367,7 +369,10 @@ int mlx5_devlink_sf_port_del(struct devlink *devlink, struct mlx5_sf_table *table =3D dev->priv.sf_table; struct mlx5_sf *sf =3D mlx5_sf_by_dl_port(dl_port); =20 + mlx5_esw_reps_block(dev->priv.eswitch); mlx5_sf_del(table, sf); + mlx5_esw_reps_unblock(dev->priv.eswitch); + return 0; } =20 diff --git a/include/linux/mlx5/eswitch.h b/include/linux/mlx5/eswitch.h index 3b29a3c6794d..a0dd162baa78 100644 --- a/include/linux/mlx5/eswitch.h +++ b/include/linux/mlx5/eswitch.h @@ -63,7 +63,13 @@ struct mlx5_eswitch_rep { void mlx5_eswitch_register_vport_reps(struct mlx5_eswitch *esw, const struct mlx5_eswitch_rep_ops *ops, u8 rep_type); +void +mlx5_eswitch_register_vport_reps_nested(struct mlx5_eswitch *esw, + const struct mlx5_eswitch_rep_ops *ops, + u8 rep_type); void mlx5_eswitch_unregister_vport_reps(struct mlx5_eswitch *esw, u8 rep_t= ype); +void mlx5_eswitch_unregister_vport_reps_nested(struct mlx5_eswitch *esw, + u8 rep_type); void *mlx5_eswitch_get_proto_dev(struct mlx5_eswitch *esw, u16 vport_num, u8 rep_type); --=20 2.44.0 From nobody Thu Jun 18 08:01:02 2026 Received: from SJ2PR03CU001.outbound.protection.outlook.com (mail-westusazon11012038.outbound.protection.outlook.com [52.101.43.38]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 699A929ACF7; Fri, 1 May 2026 04:18:17 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=52.101.43.38 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777609098; cv=fail; b=sZmKXocx+0DY1+/0rCsXpffwbvV9WNL5yoPbCx4HdEd41CxK5nNAkpMg+qGhTVYwzMP+1JKm1ajnKZRrU4otpo8QnxL+koc0G2rQ8+LmB5OypsPWJBhUMEK1IgeO6AOjH1zbss6Hi+hsKTuRZKMEsI170tIozWPg6Si8DYRPuIc= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777609098; c=relaxed/simple; bh=l66hHtKVpxBvGz9836qhT47MRfUtCDdVrrKx+xIPbYc=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=t81HbPC3QLIvpaxM2TqdmnZulXvX83UwK6dj5yC2xd/k+nQoEutJJbCWKOPnhbC2WbUqEXDMHoCojWGEgjNL2PcDDuV3pHHAeevw+HRSSsikaiYpycHhBiqvDj+Jruz9DHgePgTUuyDu/zrDDwro4Wfke8JMCO1kWE35QbnzAkU= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=hQRj7cqt; arc=fail smtp.client-ip=52.101.43.38 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="hQRj7cqt" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=gnVkuPdA7mrQRw4Lca3PjT3frZKpdKpb/YGgQdsLrHOI0gLL5Up+cEUxuDfo1VO5rtSgP04k8ODOD/g2mEy5HHlqK/32h6yEAqVL44EufCLnaK+ViNc3Jax59e/GTJ2KFGELebQHstEjC0L7GBqCZ8lVoMKhAqDF/yC5SoYWH9zRMsmHEma3ryjXJxqLhQSb7X2HszUu/vp4Bp4O87eB0tu8rz84Ew9ivbhXC6lXayF8esqOaj0kN4Nzvm3ZeMvdsecOosbV796X0HQubBkzrI3Y/HpsYqAjaE3gp+1/8xBVu+Ufj8i4OJ/WjS/zBr+JpNR+E/8fr6PQew35Gjz4ig== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=XwJhjQTMMlNGkts5Zb4PSJpGDpwAbJ6i06E8cH+ae5w=; b=D4a8TTruf8HDewj4NHbc5nUXDfZoSQce6XK2bK7AmReTJAsmNH5cUQ5MoYJCI+ueNPgUyQEVIIp8pDO24SFactrBwO/f2qtdUT6+5itBSl7KwhQtmYBP4gkZG9MueWbF37vlrCo96z4cDg4nbZwAyz8Lz4gdkOOR1+u7RjLIGJwTWn6pynui9gw1Xn/XBdkW3D2Cp++kikgxTZAK8CqvM18oXAvH1ajbod65ieMs+W5W80w8m+yXUfX6RlZxzjPRWRq5d4A4s0/9aWh7FWDhCCIKaXURXXp98iMMIwxfgOikHOztRybH6IB5tHUICI/xXt39Bduflbut/u7kSYH01Q== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.160) smtp.rcpttodomain=google.com smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=XwJhjQTMMlNGkts5Zb4PSJpGDpwAbJ6i06E8cH+ae5w=; b=hQRj7cqtIRbWbe3SkiYJd+lSgoLSQWH7lYCfu/ANnNQGSGsnRjefUiUg+UNexpFO8Z3RLoPSsUQ799STfZLKzmyzgMETXjzp/f0mt7HGUCiN7J/5RsfDrc7OuihuK8sESK+WQxIQBslcsNQbXOfvFQ+2k1khQoRBXoWAQUb66oWPtSXFj6bUcYt44M6PzeX7NMT/ZFryBnEpgY3+Z0ohXWaMl5HKu3OyOp/NfhFTgGpyZkVuoau11wh7nprP8gK+snzZ87VF/clXhf0gP9sBLzJOjKk6SEQVU8pdl0YlyTztO6kV1ox8nUXI1hbRKwMjGVVfBLLGqM0BHY7My2wOrg== Received: from SA0PR11CA0195.namprd11.prod.outlook.com (2603:10b6:806:1bc::20) by DS2PR12MB9565.namprd12.prod.outlook.com (2603:10b6:8:279::13) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9870.17; Fri, 1 May 2026 04:18:12 +0000 Received: from SA2PEPF000015CB.namprd03.prod.outlook.com (2603:10b6:806:1bc:cafe::23) by SA0PR11CA0195.outlook.office365.com (2603:10b6:806:1bc::20) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.9870.21 via Frontend Transport; Fri, 1 May 2026 04:18:11 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.160) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.160 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.160; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.160) by SA2PEPF000015CB.mail.protection.outlook.com (10.167.241.201) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9891.9 via Frontend Transport; Fri, 1 May 2026 04:18:11 +0000 Received: from rnnvmail205.nvidia.com (10.129.68.10) by mail.nvidia.com (10.129.200.66) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.20; Thu, 30 Apr 2026 21:17:58 -0700 Received: from rnnvmail203.nvidia.com (10.129.68.9) by rnnvmail205.nvidia.com (10.129.68.10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.20; Thu, 30 Apr 2026 21:17:57 -0700 Received: from vdi.nvidia.com (10.127.8.10) by mail.nvidia.com (10.129.68.9) with Microsoft SMTP Server id 15.2.2562.20 via Frontend Transport; Thu, 30 Apr 2026 21:17:50 -0700 From: Tariq Toukan To: Eric Dumazet , Jakub Kicinski , Paolo Abeni , Andrew Lunn , "David S. Miller" CC: Leon Romanovsky , Jason Gunthorpe , "Saeed Mahameed" , Tariq Toukan , Mark Bloch , Shay Drory , Or Har-Toov , Edward Srouji , Maher Sanalla , Simon Horman , Gerd Bayer , Moshe Shemesh , Kees Cook , Patrisious Haddad , Parav Pandit , Carolina Jubran , Cosmin Ratiu , , , , Gal Pressman , Dragos Tatulea Subject: [PATCH net-next V2 5/7] net/mlx5: E-Switch, unwind only newly loaded representor types Date: Fri, 1 May 2026 07:16:31 +0300 Message-ID: <20260501041633.231662-6-tariqt@nvidia.com> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20260501041633.231662-1-tariqt@nvidia.com> References: <20260501041633.231662-1-tariqt@nvidia.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-NV-OnPremToCloud: ExternallySecured X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: SA2PEPF000015CB:EE_|DS2PR12MB9565:EE_ X-MS-Office365-Filtering-Correlation-Id: fc394765-8a62-4292-1704-08dea738a875 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|36860700016|376014|7416014|82310400026|1800799024|22082099003|18002099003|56012099003; X-Microsoft-Antispam-Message-Info: usY6XojpEnC7Ai2RkBwxD0skeh/yx+TrEf/oqVsu8kQ361OAH0GeYg7Be5V6yY5N2LMQ1ko/3EFAk1N6NvryPoxIj2/B46xTZ1l9FW92WHmsB/NX5D9SWrnEfRkVrqre8l/FIbtKomYLn1QW9w03o/gm9IYjKF/eayU0BQ/GrdtgxDhNT6OffoTctWIRpp79YsMoYry20qyZOe1pONhRf48wktINuitrioXvOpxvSNyt1g5Gj1Bk0Yhve7lR4dzdk8+SSRhVdhPYKwfP4hvNdL+znVQF+wfUNiJAB8ZqoDRFKQwapaV54JpTMB3Oeoe7aRy/Zdp0zt/Hy/JjCeUyo2q0yAu1EyPCZ0ljVk1WJnybOwJjOrU5jEqaNALto7YsdZ5uL++ftGbWkid96tlMPd4E4TX1C5gkIPdFmQDEN9JtqZNzV87IDfwTweNahoyM6ueQSZPShyuhqtorOe2PHsi9cW+ky8kHBk9pufBabtSq1zwD+e1O8MXKRW9MKthV9yckY8xvk8SiUttVGf4HP2aY14NuVjJ3GoXbN0uQXg8T+FAgOjzs5PDP9lTG2SYYyuScQofBssUDpO+zZANMSTBKRqFUaZWbEjItjhSfFV7TLx683gScmMBZf5bgstu95wgH2v8+nqJcdkW88phgdxSiYF83Bo0JiDDVykU7+2sX3cLSrf0ZV0CixwUc/Ypy4iqUoPFOGnXdJAqoIV7koC0UHUOdulkRfgmnlMJ4vtc= X-Forefront-Antispam-Report: CIP:216.228.117.160;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:dc6edge1.nvidia.com;CAT:NONE;SFS:(13230040)(36860700016)(376014)(7416014)(82310400026)(1800799024)(22082099003)(18002099003)(56012099003);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: p9mMedaHHuP0kd5QXnzGls05B/nZg61lF2unD4ByQxTXcbKLc10yzFZ6MYdmn0i3H+H44FCmv5JPhNV2Jtyu6ZHxFBBxUlxSsNXe6igrdeCSOUmBvLi2gQG/BXlBRdtXpvmOmFEhSKawK2/7dbsPznLT9aRIs5mqoupy3UyqQwx3MocoHGX1U1mCl67HOfZkMnfj15Gy8DMs4GQxd0wuQLwHWD0tVLnVqzS9RawSA2lHxOoZMeMCxsXj/POxD7GYA/PFxuQaJWtc2RsFD6jIWrygiktWY5f66sldJE2p78VuRGiCfbP5HB3+yoU4TCx2BGwwz0lIxJEnRwYgp1nujUoKS1RYAmx0TCP3Fj7dnzqfl7yfa77sfN88Nx7AntyZnQQqs7OiGBB+mf8OZ1ybOn5yXQOhKvXj2u5uiMkKMDV0vWLFs1oNTKQjaneE/fJY X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 01 May 2026 04:18:11.7923 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: fc394765-8a62-4292-1704-08dea738a875 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.117.160];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: SA2PEPF000015CB.namprd03.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DS2PR12MB9565 Content-Type: text/plain; charset="utf-8" From: Mark Bloch __esw_offloads_load_rep() may return success without invoking the representor load callback when the representor type is already loaded. On a later load failure, mlx5_esw_offloads_rep_load() unconditionally unloaded all previously iterated representor types. This could unload representor types that were already loaded before this load attempt. Track which representor types were actually loaded by the current call and unwind only those on error. Also restore the representor state back to REP_REGISTERED when the load callback itself fails. Signed-off-by: Mark Bloch Signed-off-by: Tariq Toukan --- .../mellanox/mlx5/core/eswitch_offloads.c | 38 ++++++++++++++----- 1 file changed, 29 insertions(+), 9 deletions(-) diff --git a/drivers/net/ethernet/mellanox/mlx5/core/eswitch_offloads.c b/d= rivers/net/ethernet/mellanox/mlx5/core/eswitch_offloads.c index d4ac07c995b9..8f656253981b 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/eswitch_offloads.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/eswitch_offloads.c @@ -2786,13 +2786,28 @@ void esw_offloads_cleanup(struct mlx5_eswitch *esw) } =20 static int __esw_offloads_load_rep(struct mlx5_eswitch *esw, - struct mlx5_eswitch_rep *rep, u8 rep_type) + struct mlx5_eswitch_rep *rep, + u8 rep_type, bool *newly_loaded) { + int err; + mlx5_esw_assert_reps_locked(esw); =20 + if (newly_loaded) + *newly_loaded =3D false; + if (atomic_cmpxchg(&rep->rep_data[rep_type].state, - REP_REGISTERED, REP_LOADED) =3D=3D REP_REGISTERED) - return esw->offloads.rep_ops[rep_type]->load(esw->dev, rep); + REP_REGISTERED, REP_LOADED) !=3D REP_REGISTERED) + return 0; + + err =3D esw->offloads.rep_ops[rep_type]->load(esw->dev, rep); + if (err) { + atomic_set(&rep->rep_data[rep_type].state, REP_REGISTERED); + return err; + } + + if (newly_loaded) + *newly_loaded =3D true; =20 return 0; } @@ -2822,22 +2837,27 @@ static void __unload_reps_all_vport(struct mlx5_esw= itch *esw, u8 rep_type) static int mlx5_esw_offloads_rep_load(struct mlx5_eswitch *esw, u16 vport_= num) { struct mlx5_eswitch_rep *rep; + unsigned long loaded =3D 0; + bool newly_loaded; int rep_type; int err; =20 rep =3D mlx5_eswitch_get_rep(esw, vport_num); for (rep_type =3D 0; rep_type < NUM_REP_TYPES; rep_type++) { - err =3D __esw_offloads_load_rep(esw, rep, rep_type); + err =3D __esw_offloads_load_rep(esw, rep, rep_type, + &newly_loaded); if (err) goto err_reps; + if (newly_loaded) + loaded |=3D BIT(rep_type); } =20 return 0; =20 err_reps: - atomic_set(&rep->rep_data[rep_type].state, REP_REGISTERED); - for (--rep_type; rep_type >=3D 0; rep_type--) - __esw_offloads_unload_rep(esw, rep, rep_type); + while (--rep_type >=3D 0) + if (test_bit(rep_type, &loaded)) + __esw_offloads_unload_rep(esw, rep, rep_type); return err; } =20 @@ -3591,13 +3611,13 @@ int mlx5_eswitch_reload_ib_reps(struct mlx5_eswitch= *esw) if (atomic_read(&rep->rep_data[REP_ETH].state) !=3D REP_LOADED) return 0; =20 - ret =3D __esw_offloads_load_rep(esw, rep, REP_IB); + ret =3D __esw_offloads_load_rep(esw, rep, REP_IB, NULL); if (ret) return ret; =20 mlx5_esw_for_each_rep(esw, i, rep) { if (atomic_read(&rep->rep_data[REP_ETH].state) =3D=3D REP_LOADED) - __esw_offloads_load_rep(esw, rep, REP_IB); + __esw_offloads_load_rep(esw, rep, REP_IB, NULL); } =20 return 0; --=20 2.44.0 From nobody Thu Jun 18 08:01:02 2026 Received: from SA9PR02CU001.outbound.protection.outlook.com (mail-southcentralusazon11013054.outbound.protection.outlook.com [40.93.196.54]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 75E7B2FD1B5; Fri, 1 May 2026 04:18:28 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.93.196.54 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777609110; cv=fail; b=nxtWGcLxaR7lOIHAqvYPICE3BwyDKEpyt4Qqj943ZNEjE7N1/eTquHW9IjU/BZPJ0/BiwQkD1ExhTXh38uCfTS7sCdqYHSsJAmqlu2BhYGI1cmuwrlJzza/yhBF20cIwjs1BQgeE9vf7yYpfYC6uPXYgaWOJgU/Je4kz2M8kQpU= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777609110; c=relaxed/simple; bh=iBgOJhsoF/XdffJ+Ih/otQhgNbJuqaASCkQMyD4wAKE=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=SfpFX84BN6aL821LSu3LxD8qKDVqIkMeUH5n2scG6zKBvUBFkqRpXY1WoBIEHYHQW+PN9Vrqnau4gVJ8oxpM+HVLh7Fv8SiGE4QEJAQffV0Q0L+O+zTHaiJsXY4IHb+Mpnk301jvGpOSjb7VMw9jDapwxGsrquNW6TbBbpIzNHc= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=julTlREs; arc=fail smtp.client-ip=40.93.196.54 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="julTlREs" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=iXURPX01MHU2J9ayn2bS8bWg29/gsL6nukrOoH/25m8pZsv+LEMEMob4cm9BJVh+oVjgVSU+EGcyV7XJUT2dprj91m5Kn4qWUE2oZffvUgtfAKumeYY0IVmR8fzBPm1b2KG/AOlc2CJ1sPslD2f6yjdU5Ad5lB2DxvytLJtjKqEWdIJrbt2OTlmn/4Tv3p2TofRih8Wq3rUbOw2SHi7RgZRQsviajqBPPio0Ca/lv+laZOlikcXiiNoOf3JJ1BPLP3g39Z/N0QbYdnlnemhju093EyPRJ02a27KRjfSelibrbufPA2O7TspKjWXspdzjbJHSgIJiTDV3svrhUpWACQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=4/2dFiuZh4y4sV8OX0bU27/kv47tWg3tYnbtUUS2Zh4=; b=f5WIEHfNO7xDXLgHeDpiwiMaTn2M8otN7xoCUM2hZzRj9H7J85QJ1OWLpkPCisjDdMuHyBBfaLzDttk9jwtKqWyZcY3sXMqFazln6AUDIy/HZS12zfFubNEktGv7YZxcdNsho/z4yyv4SrQmGaNzzoTcEzSUqXQbOVepf/p7aLt0jTwrATObFV1IFnClw8ZAFAVQ2YAdR4yXqOrBzgyLg6Y1qHTp+W+0WQcH4A1UcbEqg9NwiKSvo3pejJv5VMVDs/IGHCU+WV0q6cmbww35tMcCtP+GHyoz2bLNW+qgfuXxcaQ7L8x6I8CSNNSr2LaetnjmMSJTGhax4j5+TRbBOg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.161) smtp.rcpttodomain=google.com smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=4/2dFiuZh4y4sV8OX0bU27/kv47tWg3tYnbtUUS2Zh4=; b=julTlREslz7jML+UQTwoYM/7QzcxtpvX9Y38o+CZTd5pTrsaYwdV2rsXqgSBgyUIRG4ERcV3llmJKTcIvApB/cvJiASymLCQdpwJvQHyzPiN7QQp9mnVrO6yBHwPOsJ8KT2sC8gSl9DCJ/nWfuGP96lha0iFmQGq2Bo8719cCYiVzHEavYJO61OzlNr0qvTx8ACJnXgqvbSMKrsnve9vvZNC902YVbrV7UQGAOhmYJy5WtySGUsWirFbNCNN95ngJBbyRw8zM+Mmhqfl6gsmptTQI2AFUAfnvzLBqZ0hGFW1GNXkJCSZ9hXbmmxrhemz98g4aFRQkfDQYXhlUkzxhQ== Received: from CH5PR02CA0013.namprd02.prod.outlook.com (2603:10b6:610:1ed::27) by DS0PR12MB9039.namprd12.prod.outlook.com (2603:10b6:8:de::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9870.21; Fri, 1 May 2026 04:18:22 +0000 Received: from CH2PEPF000000A0.namprd02.prod.outlook.com (2603:10b6:610:1ed:cafe::13) by CH5PR02CA0013.outlook.office365.com (2603:10b6:610:1ed::27) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.9870.22 via Frontend Transport; Fri, 1 May 2026 04:18:22 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.161) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.161 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.161; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.161) by CH2PEPF000000A0.mail.protection.outlook.com (10.167.244.26) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9891.9 via Frontend Transport; Fri, 1 May 2026 04:18:22 +0000 Received: from rnnvmail203.nvidia.com (10.129.68.9) by mail.nvidia.com (10.129.200.67) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.20; Thu, 30 Apr 2026 21:18:05 -0700 Received: from rnnvmail203.nvidia.com (10.129.68.9) by rnnvmail203.nvidia.com (10.129.68.9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.20; Thu, 30 Apr 2026 21:18:05 -0700 Received: from vdi.nvidia.com (10.127.8.10) by mail.nvidia.com (10.129.68.9) with Microsoft SMTP Server id 15.2.2562.20 via Frontend Transport; Thu, 30 Apr 2026 21:17:58 -0700 From: Tariq Toukan To: Eric Dumazet , Jakub Kicinski , Paolo Abeni , Andrew Lunn , "David S. Miller" CC: Leon Romanovsky , Jason Gunthorpe , "Saeed Mahameed" , Tariq Toukan , Mark Bloch , Shay Drory , Or Har-Toov , Edward Srouji , Maher Sanalla , Simon Horman , Gerd Bayer , Moshe Shemesh , Kees Cook , Patrisious Haddad , Parav Pandit , Carolina Jubran , Cosmin Ratiu , , , , Gal Pressman , Dragos Tatulea Subject: [PATCH net-next V2 6/7] net/mlx5: E-switch, load reps via work queue after registration Date: Fri, 1 May 2026 07:16:32 +0300 Message-ID: <20260501041633.231662-7-tariqt@nvidia.com> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20260501041633.231662-1-tariqt@nvidia.com> References: <20260501041633.231662-1-tariqt@nvidia.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-NV-OnPremToCloud: ExternallySecured X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CH2PEPF000000A0:EE_|DS0PR12MB9039:EE_ X-MS-Office365-Filtering-Correlation-Id: 0aa39089-19b6-4713-e490-08dea738aeb1 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|82310400026|376014|7416014|36860700016|1800799024|18002099003|56012099003|22082099003; X-Microsoft-Antispam-Message-Info: yTNCBpfpK48xnl3gMXzrJ9D8Adt/QKvwvdhi/BnAoqLZcYLdMXKDXATBQqN8/S2MZYx62yDiYQJGSsQIl0hITkm89FwaNUKchjU0Hf2BHC88jC3vOZGCQf4EydxTE4KE+8rgbWokw2WSBQwAq7qTkpcLjFB5teQ/zIl95uqH9dR3tAo0POo9Lo3VMNFZ0RfF+mILwwF/FYHLossaIZMXvEiN4bJXykzhKC6qU803uyJEIQdphfqJqlDbXLwRWeHjTXo9gvffdnVSJnV3pe3MXnXK7q/U3GdOalR4/pKcGxge/NECgd17ltJtuPpwAqE3d66VDmuUYGjf7RiGZ/9UBUMGDGAnYbZBvyxJFeRtPNN9xSxcLjgPMda17H7CgnXIzlH1k+msW0YdbCUb9HHkycB96EC7Y+8dOL70kJRnmzMm+XCKiKjVpvjeTb4JecFGlflJ/TWzIumpGvSikTm0fi9fS0AbxCribKa2GQDju4tiWuzbSMfafbds0PzKvR08jBEGKgPnlKTcLSYxoiLC6o0HK92GwbnNn/OLXyyOlIoQujOKOLfrYBW54CmRMZPXj6VS2x1JMtMwYpPn1tt0x8iqnoRpIBJ/gFe9B6n77mc0I4J+fWwS7yT6Zg16W87Lja4It7jwxS9ubUQgWmbB0O772c/vIrft1XAQDbeZJZKnAUUNOhXenXy7VTJwVg279T3kS6crp1gjMPzn9ZBgR2WVR1YhE47dXlPzx9AkcG0= X-Forefront-Antispam-Report: CIP:216.228.117.161;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:dc6edge2.nvidia.com;CAT:NONE;SFS:(13230040)(82310400026)(376014)(7416014)(36860700016)(1800799024)(18002099003)(56012099003)(22082099003);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: +iKBh3GekNDAC3W5QUHk+h7lX5sVSug7k8LAabxW7h3L470Tjl3X+h3L9Bj/n+5A0jyFXzdOK8lIES5+gBkGcb/n2sW2mZTBHKpMEesz3Zv8l8M9Uez5Fib7jqckaX/8kRFOeL5fRLiwIvJhWSYQIilHq1O/dUAwx4FFwI/6MlW0reEImL0vFRjniyXIODy6ZnTNokaxd52asTeqiYfS2VWpTPz8DbE3vL64XoqoTCx9vbFZB5/ACAs/L+WRrhmOLRlE6VkueRhB0+NobY2SAKZyL0zPfkRnEFut5ZI8zp96j7tPHAcfRTZDQzcCpcYD8d6Yyqqr4bZL2Ym7dzKd/7J6+JOEE4uKvcBZFCOCaDgokq1dbAd4hAY5pJCTTTc9hU+8NHKiHJSohoMrrMZ1jVRIqfvPIHCV8kNLecM9W5znuax0hSoTJqhG5d+sFLbC X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 01 May 2026 04:18:22.2456 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 0aa39089-19b6-4713-e490-08dea738aeb1 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.117.161];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: CH2PEPF000000A0.namprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DS0PR12MB9039 Content-Type: text/plain; charset="utf-8" From: Mark Bloch mlx5_eswitch_register_vport_reps() only installs representor callbacks and marks the rep type as registered. If the E-Switch is already in switchdev mode, the newly registered rep type must then be loaded for already enabled vports. That load path needs to run under the devlink lock, which is not held by the auxiliary driver registration context. Queue the reload to the E-Switch workqueue, whose handler acquires the devlink lock, and load the relevant representors from there. The unregister path is unchanged and still unloads representors synchronously while tearing down the registered callbacks. Signed-off-by: Mark Bloch Signed-off-by: Tariq Toukan --- .../mellanox/mlx5/core/eswitch_offloads.c | 34 +++++++++++++++++++ 1 file changed, 34 insertions(+) diff --git a/drivers/net/ethernet/mellanox/mlx5/core/eswitch_offloads.c b/d= rivers/net/ethernet/mellanox/mlx5/core/eswitch_offloads.c index 8f656253981b..f26d1652dd05 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/eswitch_offloads.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/eswitch_offloads.c @@ -4563,6 +4563,38 @@ mlx5_eswitch_register_vport_reps_blocked(struct mlx5= _eswitch *esw, } } =20 +static void mlx5_eswitch_reload_reps_blocked(struct mlx5_eswitch *esw) +{ + struct mlx5_vport *vport; + unsigned long i; + + if (esw->mode !=3D MLX5_ESWITCH_OFFLOADS) + return; + + if (mlx5_esw_offloads_rep_load(esw, MLX5_VPORT_UPLINK)) + return; + + mlx5_esw_for_each_vport(esw, i, vport) { + if (!vport) + continue; + if (!vport->enabled) + continue; + if (vport->vport =3D=3D MLX5_VPORT_UPLINK) + continue; + if (!mlx5_eswitch_vport_has_rep(esw, vport->vport)) + continue; + + mlx5_esw_offloads_rep_load(esw, vport->vport); + } +} + +static void mlx5_eswitch_reload_reps(struct mlx5_eswitch *esw) +{ + mlx5_esw_reps_block(esw); + mlx5_eswitch_reload_reps_blocked(esw); + mlx5_esw_reps_unblock(esw); +} + static void mlx5_eswitch_register_vport_reps_locked(struct mlx5_eswitch *esw, const struct mlx5_eswitch_rep_ops *ops, @@ -4574,6 +4606,8 @@ mlx5_eswitch_register_vport_reps_locked(struct mlx5_e= switch *esw, mlx5_esw_reps_block(esw); mlx5_eswitch_register_vport_reps_blocked(esw, ops, rep_type); mlx5_esw_reps_unblock(esw); + + mlx5_esw_add_work(esw, mlx5_eswitch_reload_reps); } =20 void mlx5_eswitch_register_vport_reps(struct mlx5_eswitch *esw, --=20 2.44.0 From nobody Thu Jun 18 08:01:02 2026 Received: from CH1PR05CU001.outbound.protection.outlook.com (mail-northcentralusazon11010047.outbound.protection.outlook.com [52.101.193.47]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 72A682F90C9; Fri, 1 May 2026 04:18:41 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=52.101.193.47 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777609123; cv=fail; b=EvlgRQ0pWK7vmk31iagmw5b1zakQPC5ymqL/JjPE4hpocTkNNFXKDIGbFpw4dE/WUjIHAmHtLSfSoqI5/Or4KyCr/wMUkSmzZChxCEgQHylHrBj7QqOwBl8NGsAvbdiq3TRCSrdlQ/HDGManvolIbgx0prAgTf4VxYf4cJLLI0Y= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777609123; c=relaxed/simple; bh=vrq34g7znpO2Qqpqt8u3pmA9GpdXKpul7ekKj2S1sAc=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=R/cBlL/V5EB/ry5Nr3ChPYE3vf7GuXbdeZMcry4t3AkFHoftndEfd+RV+dfp//DJbVnpE/wX43Hp43S11tDa1RaVRwdBtUxSirm+3nKxUm6ZPM28GH+TvygBNKtNAK9UvciXbz6rDq5WpKWYGJveRpNNmmq+jkFheZDSw/gRWeA= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=JYV186V9; arc=fail smtp.client-ip=52.101.193.47 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="JYV186V9" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=RJww/h0X4yRnwaEoc9bn3Y0av2pwsgl70/suJwsE2bHG3KYJp0G6PJySLPvB3gmu8tjjLlOZXv4enM2GO/nWi8efpabj4KWsg0RGhgg2eX0Hbil1JdPCVLlRS+X6A+5JTHYGLVXbUwKrYZz9YSQ+p/dyVe+/4MjauTleaxi4IRMXNl7mgfBLxLEanmfzfxcHQTBVfioI1Yuo3c0ySILHR1r71XK+A67H+Tp+2ToPLVCX8AHdyeknUTh0sUhL25/Dx4V3ttonToTqCcSgqcR3BVEWBghxgUE2uRgio39oz898HbTgZ4Fd7iS2WilkeNkSC87WNaixpwy8TVHDIPog6g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=7dfoD2jB6LcnrT0r4uXRYo/+TRabmPd4uF2DxjEJHQ4=; b=HC/O1jOy5eeMmqieZmjsMPliVJxCfbwsciQv1C9q1u7jKSnscVEhiM1KjeQNYTMNjCBI0PF03w4rZPLF+5IissrQaSfo349rrf4FIzHMWumbNsMGO1AAgpNYZfdk0KPQqd/m/zHyJE0e97Fq4YwwV+3V0RO6PQMPIEfY0GPh96egQBlbqOQDAYeXsPGyDPnAFXtBPQ+WwtEAQcamCX5vUrA9SwXRsP5xbZdezQfiE2UdZaeajeOjbFfE/VePwz7e1skQwP41q3FG1M9r5uc9k/Q6gT0PpJJi0//kA4UbwL2wKhpUKcO+tJGrPceO1RgNEw3QE/Bi6HO80mqjltX5gg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.160) smtp.rcpttodomain=google.com smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=7dfoD2jB6LcnrT0r4uXRYo/+TRabmPd4uF2DxjEJHQ4=; b=JYV186V9vvcbcgk6W/abNXQMetaZXTmOUeBgfBhrxD5HYXaBiAdtzmJGI6vokXJqLJyD+tkKSHnPp7CD3bKrs7BC5+chnYUw81D6KpE3oB3ufAIHcnlfdN5LPmB57L8Gpy6RLJBApRNnSpY9K8A7kVS9K3du66cHXYgWfekA6bkITcImYKUAKtA7uWGfLPlFknwlwxVCwhXtPvNNhLDqFZ4d5TDliQjEnp5utmYh8g5t/AKuyb+dD/Ejgtr8s1BTICwTNmyOJk2x7ZotX/71UGEG5Dg0iq4CgHEY/oWEXNSdm4AJyRr/t7r4VdpkYonxHPmIhFxwX8C+TqqV200ung== Received: from PH7PR13CA0011.namprd13.prod.outlook.com (2603:10b6:510:174::26) by CH2PR12MB4038.namprd12.prod.outlook.com (2603:10b6:610:7b::19) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9870.20; Fri, 1 May 2026 04:18:29 +0000 Received: from SA2PEPF000015CD.namprd03.prod.outlook.com (2603:10b6:510:174:cafe::87) by PH7PR13CA0011.outlook.office365.com (2603:10b6:510:174::26) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.9891.8 via Frontend Transport; Fri, 1 May 2026 04:18:29 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.160) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.160 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.160; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.160) by SA2PEPF000015CD.mail.protection.outlook.com (10.167.241.203) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9891.9 via Frontend Transport; Fri, 1 May 2026 04:18:28 +0000 Received: from rnnvmail203.nvidia.com (10.129.68.9) by mail.nvidia.com (10.129.200.66) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.20; Thu, 30 Apr 2026 21:18:13 -0700 Received: from rnnvmail203.nvidia.com (10.129.68.9) by rnnvmail203.nvidia.com (10.129.68.9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.20; Thu, 30 Apr 2026 21:18:12 -0700 Received: from vdi.nvidia.com (10.127.8.10) by mail.nvidia.com (10.129.68.9) with Microsoft SMTP Server id 15.2.2562.20 via Frontend Transport; Thu, 30 Apr 2026 21:18:05 -0700 From: Tariq Toukan To: Eric Dumazet , Jakub Kicinski , Paolo Abeni , Andrew Lunn , "David S. Miller" CC: Leon Romanovsky , Jason Gunthorpe , "Saeed Mahameed" , Tariq Toukan , Mark Bloch , Shay Drory , Or Har-Toov , Edward Srouji , Maher Sanalla , Simon Horman , Gerd Bayer , Moshe Shemesh , Kees Cook , Patrisious Haddad , Parav Pandit , Carolina Jubran , Cosmin Ratiu , , , , Gal Pressman , Dragos Tatulea Subject: [PATCH net-next V2 7/7] net/mlx5: Add profile to auto-enable switchdev mode at device init Date: Fri, 1 May 2026 07:16:33 +0300 Message-ID: <20260501041633.231662-8-tariqt@nvidia.com> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20260501041633.231662-1-tariqt@nvidia.com> References: <20260501041633.231662-1-tariqt@nvidia.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-NV-OnPremToCloud: ExternallySecured X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: SA2PEPF000015CD:EE_|CH2PR12MB4038:EE_ X-MS-Office365-Filtering-Correlation-Id: d110495b-e169-4bfc-2cd4-08dea738b29f X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|7416014|1800799024|376014|82310400026|36860700016|18002099003|56012099003|22082099003; X-Microsoft-Antispam-Message-Info: rQgVvqScC93/is3a+adR2D0Tas1y04X9Yq0HAFOVBHP8W0a7upqrSz9CLgq/yIgq/4rAy7fkGlYXMsPxnZVI0c5Nnl1GqKeAA08floOq5YtqOghc2cJb8mH7MarSuxPyN6orCpNSnTkG6h0heeGSoRiGaPO0FxNWX7M9AufrjuMLdvivX4ZslMq5Xmw4lLiamMHkl6W8e6ky2khF87VXVTDKGkt6LciloX5PqfBGUvbuBXW8/xiXyYE0C2SVQDFG4+X5JyiYnHmmb8gG2Eb06bR3dyENZqOb3AFIlxZ0cIg7Y17fUhf2ZtSRvd9c40Zj6cuJq/5Cz9rYeS2jV0qZ+jMuGw9lfCM9pAsGYJLWWo7kaKdStrL8HqYdk/hmHUhw77gQ2WycAyq1QoR1gxpXrS+DvQK6wdnE3lx5CblMyi3VOFuAmepDk9aaazSDBEiGpzEVeXvpqaC8QxF8BMzoFIbxDBQafkavi0H1Sm4HvWv4TFaVPiOAJHOEC6khuh83SD3nXSC/K7ra6A3EUOtPJcMLLl+U7G7k4wMBuCrVf7zSzYJtcHvVN7un81sfFKUue3Ev+lysNM9ltxZeBSdHEMB472drycNXLFm4zA2+tJCoCDmZXnbAv/s9RWIBBOroYx48XQV18+l/MBTvqABetgXFNV+qjLAhWBjJ9oxS0X88JVJhssu0LNIr8ZFBakR9b0iACmvIDCg99AO3iikJoVgRnVf6IGZQ4hp6iOqTlBI= X-Forefront-Antispam-Report: CIP:216.228.117.160;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:dc6edge1.nvidia.com;CAT:NONE;SFS:(13230040)(7416014)(1800799024)(376014)(82310400026)(36860700016)(18002099003)(56012099003)(22082099003);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: ok2uk8EC1RNUj/ugC7j4Svl0FTJC5kq6Ab2NUYJ4TyInJTN4yi2FH2+PbL8dW0JxKVIQm+vQXDzQZMzHBrGIrKaJPEBMkmDntE1RwDOAAHLiDa7FTcbDwP512R9L133WTRARWCx4lObMwoOMhhFPa65Rl5IQ7csN85AuMTmjSJbjw/nvPaEi+dQYdd34huyML9/ARCOljJ2urOppPBCjr6CQcG5YrNDzkSa5mFHpMmUMD73+ICArWjJ2WbhXT2SWninaLByIw9wg44zrDl9Jzpcu+JAwbUwBUK6fzPeOCsY1L6pDCCvryRqfhl3kKtQpiaZgh/b+G3WOgS8ctxQ4M5AMm3cgRNIpD48HKgvqx3XpRevVU+nz5Fa2tzy2/KoXKvNMmzu5ygkXsEWwAYRFzrS5YcE0ZwmcGxBBWIwC1Kh7I4OoXQovji2lJJb7lTiU X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 01 May 2026 04:18:28.8489 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: d110495b-e169-4bfc-2cd4-08dea738b29f X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.117.160];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: SA2PEPF000015CD.namprd03.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: CH2PR12MB4038 Content-Type: text/plain; charset="utf-8" From: Mark Bloch Deployments that always operate in switchdev mode currently require manual devlink configuration after driver probe, which complicates automated provisioning. Introduce MLX5_PROF_MASK_DEF_SWITCHDEV, a new profile mask bit, and profile index 8. When a device is initialized or reloaded with this profile, the driver automatically switches the e-switch to switchdev mode by calling mlx5_devlink_eswitch_mode_set() immediately after bringing the device online. A no-op stub of mlx5_devlink_eswitch_mode_set() is added for builds without CONFIG_MLX5_ESWITCH. Signed-off-by: Mark Bloch Reviewed-by: Moshe Shemesh Signed-off-by: Tariq Toukan --- .../net/ethernet/mellanox/mlx5/core/eswitch.h | 6 +++ .../net/ethernet/mellanox/mlx5/core/main.c | 43 ++++++++++++++++++- include/linux/mlx5/driver.h | 2 + 3 files changed, 50 insertions(+), 1 deletion(-) diff --git a/drivers/net/ethernet/mellanox/mlx5/core/eswitch.h b/drivers/ne= t/ethernet/mellanox/mlx5/core/eswitch.h index 3858690e09b4..cfb9595f9de8 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/eswitch.h +++ b/drivers/net/ethernet/mellanox/mlx5/core/eswitch.h @@ -1049,6 +1049,12 @@ mlx5_esw_lag_demux_rule_create(struct mlx5_eswitch *= esw, u16 vport_num, return ERR_PTR(-EOPNOTSUPP); } =20 +static inline int +mlx5_devlink_eswitch_mode_set(struct devlink *devlink, u16 mode, + struct netlink_ext_ack *extack) +{ + return -EOPNOTSUPP; +} #endif /* CONFIG_MLX5_ESWITCH */ =20 #endif /* __MLX5_ESWITCH_H__ */ diff --git a/drivers/net/ethernet/mellanox/mlx5/core/main.c b/drivers/net/e= thernet/mellanox/mlx5/core/main.c index 74827e8ca125..4cdda15ed7f5 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/main.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/main.c @@ -86,7 +86,7 @@ MODULE_PARM_DESC(debug_mask, "debug mask: 1 =3D dump cmd = data, 2 =3D dump cmd exec t =20 static unsigned int prof_sel =3D MLX5_DEFAULT_PROF; module_param_named(prof_sel, prof_sel, uint, 0444); -MODULE_PARM_DESC(prof_sel, "profile selector. Valid range 0 - 2"); +MODULE_PARM_DESC(prof_sel, "profile selector. Valid range 0 - 3 and 8"); =20 static u32 sw_owner_id[4]; #define MAX_SW_VHCA_ID (BIT(__mlx5_bit_sz(cmd_hca_cap_2, sw_vhca_id)) - 1) @@ -99,6 +99,8 @@ enum { =20 #define LOG_MAX_SUPPORTED_QPS 0xff =20 +#define MLX5_PROF_SEL_LAST_NIC 3 +#define MLX5_PROF_SEL_FIRST_ESW 8 static struct mlx5_profile profile[] =3D { [0] =3D { .mask =3D 0, @@ -120,6 +122,11 @@ static struct mlx5_profile profile[] =3D { .log_max_qp =3D LOG_MAX_SUPPORTED_QPS, .num_cmd_caches =3D 0, }, + [8] =3D { + .mask =3D MLX5_PROF_MASK_DEF_SWITCHDEV | MLX5_PROF_MASK_QP_SIZE, + .log_max_qp =3D LOG_MAX_SUPPORTED_QPS, + .num_cmd_caches =3D MLX5_NUM_COMMAND_CACHES, + }, }; =20 static int wait_fw_init(struct mlx5_core_dev *dev, u32 max_wait_mili, @@ -1385,6 +1392,22 @@ static void mlx5_unload(struct mlx5_core_dev *dev) mlx5_free_bfreg(dev, &dev->priv.bfreg); } =20 +static void mlx5_set_default_switchdev(struct mlx5_core_dev *dev) +{ + int err; + + /* Default switchdev is best-effort; keep the device usable on + * failure. + */ + err =3D mlx5_devlink_eswitch_mode_set(priv_to_devlink(dev), + DEVLINK_ESWITCH_MODE_SWITCHDEV, + NULL); + if (err && err !=3D -EOPNOTSUPP) + mlx5_core_warn(dev, + "Failed to set switchdev as default, continuing in current mode,= err(%d)\n", + err); +} + int mlx5_init_one_devl_locked(struct mlx5_core_dev *dev) { bool light_probe =3D mlx5_dev_is_lightweight(dev); @@ -1431,6 +1454,10 @@ int mlx5_init_one_devl_locked(struct mlx5_core_dev *= dev) mlx5_core_err(dev, "mlx5_hwmon_dev_register failed with error code %d\n"= , err); =20 mutex_unlock(&dev->intf_state_mutex); + + if (dev->profile.mask & MLX5_PROF_MASK_DEF_SWITCHDEV) + mlx5_set_default_switchdev(dev); + return 0; =20 err_register: @@ -1532,6 +1559,10 @@ int mlx5_load_one_devl_locked(struct mlx5_core_dev *= dev, bool recovery) goto err_attach; =20 mutex_unlock(&dev->intf_state_mutex); + + if (dev->profile.mask & MLX5_PROF_MASK_DEF_SWITCHDEV) + mlx5_set_default_switchdev(dev); + return 0; =20 err_attach: @@ -2314,6 +2345,16 @@ static void mlx5_core_verify_params(void) MLX5_DEFAULT_PROF); prof_sel =3D MLX5_DEFAULT_PROF; } + + if (prof_sel > MLX5_PROF_SEL_LAST_NIC && + prof_sel < MLX5_PROF_SEL_FIRST_ESW) { + pr_warn("mlx5_core: WARNING: Invalid module parameter prof_sel %d invali= d range %d - %d, changing back to default (%d)\n", + prof_sel, + MLX5_PROF_SEL_LAST_NIC + 1, + MLX5_PROF_SEL_FIRST_ESW - 1, + MLX5_DEFAULT_PROF); + prof_sel =3D MLX5_DEFAULT_PROF; + } } =20 static int __init mlx5_init(void) diff --git a/include/linux/mlx5/driver.h b/include/linux/mlx5/driver.h index 04b96c5abb57..65298c07df4d 100644 --- a/include/linux/mlx5/driver.h +++ b/include/linux/mlx5/driver.h @@ -705,6 +705,8 @@ struct mlx5_st; =20 enum { MLX5_PROF_MASK_QP_SIZE =3D (u64)1 << 0, + MLX5_PROF_MASK_MR_CACHE =3D (u64)1 << 1, + MLX5_PROF_MASK_DEF_SWITCHDEV =3D (u64)1 << 2, }; =20 struct mlx5_profile { --=20 2.44.0