From nobody Thu Apr 16 06:47:32 2026 Received: from mx0f-003ea501.gpphosted.com (mx0f-003ea501.gpphosted.com [66.159.228.33]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 00EA137702B; Mon, 2 Mar 2026 05:58:14 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=66.159.228.33 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1772431105; cv=fail; b=D5IS6430AtWB39CX1zm00xtXQXStUmUZvCW58gxnwGKgUBRtHwUN7dG0yRvkKu629ZGzgJltmmrtfWVEe7fvT5gi98SAVtnmYsANvjNh+iKSEeACG5rQMNfyyep5yEtngi2yoxTO0BZLBDQApkqlWb23VwisFoW1XpFJ2NHPHc0= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1772431105; c=relaxed/simple; bh=OmDhQo3kPKINyiXyPJlwDyM8HrunX9DQoBsqxwBtFbk=; h=From:To:CC:Subject:Date:Message-ID:Content-Type:MIME-Version; b=jq9yBAfznVDSz/jAOez7QAhYg4TUBPqpOXwOLeG60/fyMiCAznT62Ukkofbn1DCq5iqn6u/L16rCKaAGU6b8DK7kQZfH7LSxNOyREq74EML7Wa9YXSFEMLp74NOkZuCe+FYruwMeLK6jN5Zsj09hnfnLZUBviv9+6wi6u3szN2I= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=spacex.com; spf=pass smtp.mailfrom=spacex.com; dkim=pass (2048-bit key) header.d=spacex.com header.i=@spacex.com header.b=YD72bIX0; dkim=pass (2048-bit key) header.d=spacexgcchigh.onmicrosoft.us header.i=@spacexgcchigh.onmicrosoft.us header.b=CMaCJt6x; arc=fail smtp.client-ip=66.159.228.33 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=spacex.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=spacex.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=spacex.com header.i=@spacex.com header.b="YD72bIX0"; dkim=pass (2048-bit key) header.d=spacexgcchigh.onmicrosoft.us header.i=@spacexgcchigh.onmicrosoft.us header.b="CMaCJt6x" Received: from pps.filterd (m0418030.ppops.net [127.0.0.1]) by mx0f-003ea501.gpphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 6225pfdg008858; Mon, 2 Mar 2026 05:52:14 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=spacex.com; h=from : to : cc : subject : date : message-id : content-type : content-id : content-transfer-encoding : mime-version; s=dkim-cloud1; bh=OmDhQo3kPKINyiXyPJlwDyM8HrunX9DQoBsqxwBtFbk=; b=YD72bIX0SILNBymB/dUeBeTtG2W47isgMMIPLDg2+5HboOCH2cnISvpOaNqK2YrwDqGu f0CRtovGOuc7V26yhwAPQsY+v95IyISSc4v5OLeKBuj614gpqzg8wlFWEtyen3aU2DON +teh8ZU2tklKlmXhPwnXCmffyhWPtJ/LmDOggmk6QtX5PYSkbE4x8DCIXgbf67hiQZy1 GuE4y3yckvW/EvywLsHyVW1IQvqGxmXeD8p0fFVkEerBSvAi6Cs56hHIfCLsGN0ejR9l T5olhgN+h0+N5yhhDUHW291ibu++lxDeq7YJhN6sr3xNLar4jgTBSjAEAMmprSd1EuTy 9g== Received: from usg02-bn3-obe.outbound.protection.office365.us (mail-bn3usg02on0058.outbound.protection.office365.us [23.103.208.58]) by mx0f-003ea501.gpphosted.com (PPS) with ESMTPS id 4cn4s8800q-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 02 Mar 2026 05:52:14 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector5401; d=microsoft.com; cv=none; b=QIpIlb33X8NahBmR7u+avt4PiPDEKO9+yPumgkZxQBIGcy5nmxmCuGDJb9lhOoJBeoBGRRjCwsk8qrI638gv+9YzAsrzaxWycjzgYII6MynsGHmrANAjMJxQatIziVcPgsCkzCgN2cQ0ztRZlKQR0A17vxk5mBR2CpajaPVQqCnbfaPOdlDEO7s9zrz2GXGGNQTx1TKp7PqrROO3zmWCZ6b10ZUbLY5ifoCev9ldjgZh8OZ5Sgo0QNjywxHZNLB5khy1fo1yElZQcZO5P82ga0oYFXQtji08z1tOItUI5jJvlr+JGQT9ONR8CanALzgB3URJCFPq5lo56Rz7Chd59A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector5401; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=OmDhQo3kPKINyiXyPJlwDyM8HrunX9DQoBsqxwBtFbk=; b=DETpnHPs1m+FKNBLjZFQoI/iTw0cRf6YK9VMbbr0FmP1cS7SvCY9fMZktaph4q3t0hPwKS/rlqRgHzSbsy2lYn2beJbF0UsLz/63hB2emK7ZHWID70YzlkjifQA0Ssd5bskdK94BoVKcMp6tciuh0A8G6QDa2LCwzziKxgcLv/WYT91QALuaeEM+WHH4zXcLFtled//o6+LJqKrK4yUCd9DCuEk1Sx+VWjqg7cg98wNndRGJCDUp7Icw//bPD8wrkBOxVPZ/wysEGnoUQ6xrABzLz1lDe96LeubLWI4BXI1SihCmODA3VRAWg/IPfiFjIImRm/e5eJqMMoXJ+4Ecug== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=spacex.com; dmarc=pass action=none header.from=spacex.com; dkim=pass header.d=spacex.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=spacexgcchigh.onmicrosoft.us; s=selector1-spacexgcchigh-onmicrosoft-us; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=OmDhQo3kPKINyiXyPJlwDyM8HrunX9DQoBsqxwBtFbk=; b=CMaCJt6x36nSE1zQjhls3tQQJwDYdKYWbTtVBeLivf/AQ3AiFvEYwicN/a8oI91nKiaPDqy17aIn+pVh4JuBoiG9cZsSfHA0Mi0mPktIlAOqHKKxODaoiB8/uC0dE0e+gReBTAfAtNXQCkgat3tmwG2UcBNS3p7SUTCRKaxpuMz2ycynGRfdC6HRCPQzMMzFxxmcrWNuTXUFc+/RszifgMIVMaj5QswIkwnZvLqb9NulZLgv5mJAuy5OYEcY7yjcDzquB/bHS2AB2W3l9VWtXsxSVJ3arWSrjWTMAiIT9af/BBGrUPC9WQqAMhuWCH1cdaHN/gcWW+yK77Ia3Nmijw== Received: from PH1P110MB1186.NAMP110.PROD.OUTLOOK.COM (2001:489a:200:18d::6) by BN0P110MB1676.NAMP110.PROD.OUTLOOK.COM (2001:489a:200:186::9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9654.20; Mon, 2 Mar 2026 05:52:09 +0000 Received: from PH1P110MB1186.NAMP110.PROD.OUTLOOK.COM ([fe80::d215:34eb:6fa9:247d]) by PH1P110MB1186.NAMP110.PROD.OUTLOOK.COM ([fe80::d215:34eb:6fa9:247d%4]) with mapi id 15.20.9611.013; Mon, 2 Mar 2026 05:52:09 +0000 From: Finn Dayton To: "netdev@vger.kernel.org" CC: "ast@kernel.org" , "daniel@iogearbox.net" , "davem@davemloft.net" , "kuba@kernel.org" , "hawk@kernel.org" , "john.fastabend@gmail.com" , "sdf@fomichev.me" , "saeedm@nvidia.com" , "leon@kernel.org" , "tariqt@nvidia.com" , "mbloch@nvidia.com" , "andrew+netdev@lunn.ch" , "edumazet@google.com" , "pabeni@redhat.com" , "bpf@vger.kernel.org" , "linux-rdma@vger.kernel.org" , "linux-kernel@vger.kernel.org" , Finn Dayton Subject: [PATCH net-next v2] net/mlx5e: Precompute xdpsq assignments for mlx5e_xdp_xmit() Thread-Topic: [PATCH net-next v2] net/mlx5e: Precompute xdpsq assignments for mlx5e_xdp_xmit() Thread-Index: AQHcqgi1vSXu4N8FMU+ckU6VWHv+fg== Date: Mon, 2 Mar 2026 05:52:09 +0000 Message-ID: <60E0EC79-4E2B-4874-9CEB-6558706A910A@spacex.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ms-publictraffictype: Email x-ms-traffictypediagnostic: PH1P110MB1186:EE_|BN0P110MB1676:EE_ x-ms-office365-filtering-correlation-id: 813b954d-23e7-4851-a862-08de781fd7d5 x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0;ARA:13230040|1800799024|366016|7416014|38070700021; x-microsoft-antispam-message-info: FIHsAEaY1L7kg3/rGVXvfonQxUEXRUAPFDD36PwEU3XxBrc8M6b6Ed4BhVVVlGhJ0ZPQ28kLRKDK/7Pw4RqY3ROFilPjrINaeqqDHNv4LmF/5KK/1V0ftxanSEIpO7+959nI1ARB56I95tqwEJB4smGClhpNTXR2NsD5e9E9b6bZccHJhFICGNKjg2G3whsdXF0sSIth678vMVM5/s6NaW/kpX9H0BcaJ5Jlf2uOaJM/WwtykD20DHqQaEnHLhV5YL04UsLv67hYmyXu3MfAV4uyRfsrYpDGy2zdiEt7ze/nFTK0phj6ZvU5tK4iZH5JcSQ7Xy4O4l3eg9Dwyb2CKairflbsmZ7vnG6VCrF5/NiBkwc7MBYrvMz1Uf/fUwzR7Dbw02ttqob4ZXIFWPq9fORfZATcx1bUlBygncbbgHMfKJdqJZQiaRjMy+OjX/xVhEqZZiK1u3jUS597zIJAXWf/6vtdHlAq92DpS9ZZW48YUyC+kVM6sGFPKQIxfrnzay4dqyNcL1rHcXWlBrtnkxRbgLALulQxQRazLt/sKshadKGkEcey+3QcY+x+Skc4TY37iQsgZY+OuxgjAH/MkUv8//v5GvQ5KgVyxBzJ2buXV3AgOe7/64482DD0kca+2u0QEzDiyjYMwwl/hjiwvWEnMRmrrlNwhA17HtSLbp2Jx6mImrbLZBbkSyTqILsgxatUmfxFZS7vNKVWt0wBdAa0XORIeqZur8dyYtoD8SCnKB+DZBl110G2hjbM2MklDpQOxEDxerSMXlI1kQNeF7f8zb8Ohy/eSrvwKIQN0WM= x-forefront-antispam-report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:PH1P110MB1186.NAMP110.PROD.OUTLOOK.COM;PTR:;CAT:NONE;SFS:(13230040)(1800799024)(366016)(7416014)(38070700021);DIR:OUT;SFP:1101; x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?utf-8?B?SXJYWklLbmpMdnVCTmhlQlRTU3h1d2FxdDJiWkxycUhnRGl5elJyMWNBQjRh?= =?utf-8?B?UGQveWRpYVIvdkY0eXgrTzZjaThhYzhuZVh0TWo4YnhMS3F1VXpualAwaUZ6?= =?utf-8?B?T2wxU08yTEFXamVvL3g5dk55Vy9JRmx2QjJCS2cvb2RUcU9LbGsvZnpQVEtC?= =?utf-8?B?MThuWUVmanlaNVVkSkQvYTZWWDFHWWtqT1c5akZxZGtwdVZMSStBQTY3V3l2?= =?utf-8?B?MmZ6YU41UVFoVmZNZml2c0FoK3VtL0JVQ1BSdmp2S0pVOWZlSVZ1bWorTEZh?= =?utf-8?B?NTBjOTI5U0FDNkFNUzRyaC9rdXc3TUNHdWgrYmM5dnNyZ0haZUVxQ3JQZ08y?= =?utf-8?B?dytCU05YdUxQMjhtRmNvWXREQWxSbnk0cUNBTVhIWktXQzZEOHNIYTBiL0RP?= =?utf-8?B?bEY4eEU2VG1DOXVQVVdUVWdoTXVaUFc0SERkNk0zWXRJODdVNmY5bXBoU09N?= =?utf-8?B?MzhnNGM5aUxEaVZpOTUwUXN0Z280WGFXbzdhNFJJUjJhN3VaUHQxYk0reFpi?= =?utf-8?B?RVJ2eGdjYklKVTg4MFlnVFhUT283eEtEUFc5SHEvMnc3Qi8rVlFpUnRzUDdD?= =?utf-8?B?Z2pPK1k1NjRZM1psMm0vWTNTWHpGMjM5OXJvKzRVWUhWcElPV3U2bUhDVTYy?= =?utf-8?B?RDF0YmZJNm83anFuSVA3ZGp2RURwS3dqMXhQNmxkblJ4OHRvVmlaOGdsTmRF?= =?utf-8?B?VXcrY3FPcnBFcVNXYVM5WHpCMktoUjBsSkdoenYrWklsUTd6Um5HQ250MlJi?= =?utf-8?B?UW9TNDFvVjRRSVBLSnd3OUJwV2U1aXFUUG1EalVKaXF4eFBJQmRlVFMzQzJB?= =?utf-8?B?dFlJaUZtMnJTZG91MkZiRmU0OXJGaHZFU2dubzlRZURnRS9qdStaMWZjc09v?= =?utf-8?B?QjdaN1ZIY0Z6bDV6SU5SN1NTZnUydEt6NURFOHFFTmFzcUFMTnVqd1JmZVFK?= =?utf-8?B?QlZrUWlKa3pYTjludzJsZU11ZDVwRi9NelVjNGVaN3Z5WFFFSU50VmkycGxS?= =?utf-8?B?QVNSZVV4bkIvNFhIT2Y3RWZBNVlkMldCTG9Qa0ZkTE4rU1RxdWtZNmJRdThR?= =?utf-8?B?UHFRODFrVGsvaVB4Tk9QaFRJKzU4RnRBZnN4azBnUERkNGFlZnErSHV4MGYv?= =?utf-8?B?WW1MdVJGbXFqZ0hKWUdkME5CVE1XaHdkVEFVMXo2N09KcEN3YjJiTUR2RFQ2?= =?utf-8?B?TU5uK3ZveHZOSEI3bjNrRWErT0cxenRSdUJWOENHZnFnanFGcUFZNm9WQk9l?= =?utf-8?B?T3FnVGEvdnFwdUVNSzVPd3U2ZEF6QnVMcU9nT0U0Z3FMTGNrQWplK1VTQVRY?= =?utf-8?B?eDJvUXhKQllwQVZhYitJeUY5RGQ5YXdNeEFFSEtsdVlzdlV4S1lVS0hiVUhF?= =?utf-8?B?QjNNOWl1L2E3MUZFazRxcHJsdU9BQ3lMSUNZOS9raVk4ckJQS0Y0OGdoTHdE?= =?utf-8?B?NlJDeXRBamRrbUtaMW41MUxhQTBWbDhSRlVncU0xRG9saSttQ3lWUDlqVUJH?= =?utf-8?B?YWUvQXgrNno3RTFEZU53OFRRUERYU2FEcVBsUnl2dlpSTWorRUMvZ0ttZm0y?= =?utf-8?B?RFVFVE81Rk0raWx1NTNxOWwweXIzSGZUZ3Q4Qll1cEZPTFpYeDN6U2RMRzhT?= =?utf-8?B?YStBUWJCcVFheWFOeWpPa3ZkMlkzZENUc2JScmluVmsrR3VGZEJxaUJuNDgy?= =?utf-8?B?eTF6QmdYSlVkWWthZlp1M0FCQXdIOFpXQmhSR2k1UlFMSkIvSUhpNHBaSEFF?= =?utf-8?B?LysybTJ6dXVOQzRabnl4ZGQvclpydUdrUkZLR3VST3NXcG1BandablhUd2U4?= =?utf-8?B?NGxGNkZDN1dGV3RLaFFMVUlCR3NKUFkrQ25lelpjbEl5L2E1a3NFNXU5TTZT?= =?utf-8?B?ek9tWllOVmQzSVNOYjRZQVdESEFFOXRPUlZINkZpU3JHMWsrUVRrb3pMNVlT?= =?utf-8?B?UDlzRVlFQnppOW9neUphMTIyU29wcXhmTU1SR05IS3U0ejZHTDBVRmNMOTBl?= =?utf-8?B?SW9LTy9uM1NHWUh4ZEhIOWlreVcrSHNjMm1GT09zUkxQd2lhb2tRdTJJMFZx?= =?utf-8?B?bHdpUTQ3OUViUXVTMkt4N0JVQ1hVQ01XZUptZz09?= Content-Type: text/plain; charset="utf-8" Content-ID: <5D62355AA46D1D4BA6537013A578941B@NAMP110.PROD.OUTLOOK.COM> Content-Transfer-Encoding: quoted-printable Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-OriginatorOrg: spacex.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: PH1P110MB1186.NAMP110.PROD.OUTLOOK.COM X-MS-Exchange-CrossTenant-Network-Message-Id: 813b954d-23e7-4851-a862-08de781fd7d5 X-MS-Exchange-CrossTenant-originalarrivaltime: 02 Mar 2026 05:52:09.3539 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 70922d1c-6e01-4d95-82dd-55b449e38bd1 X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN0P110MB1676 X-Proofpoint-ORIG-GUID: XrGB2e3puRtxCzgPgmtw-4optKGCjtwX X-Proofpoint-GUID: XrGB2e3puRtxCzgPgmtw-4optKGCjtwX X-Proofpoint-Spam-Details-Enc: AW1haW4tMjYwMzAyMDA0OSBTYWx0ZWRfX7bu/up44es8P zxINi0MkxcDmLBPb7XZQOAJmVf9cGCPjp89UOXRJ+E7b5d71aBeyAzO7F92wiVtPZ+aKr+Dx/IU 6Es9+GEacUnbG758UdWMjdyIRzyNAEvquwQyr0dyEtMh2fKz41Zyc84blCS25sCz2NPQG0fexxH ZvvbKRsryciArzPt9NEbgwBAdwYsGYWH4mTuQgWhmG/RBQIHCkRETU0u5slVI8E54mhYTBGILeN pg/gtgdik/paDp2uXvCOrjiQncq/AQ0HS9D58H2EGXnHGnYTppde9FMZsQXs0vTQpSal5dtiX0X xfBXkaNHPbcndSNCKpWKlbugLbIcpfPxsVHByx5MjN2g4+q6tS8g99Yliaw+8t8UBLViTYr2s/0 laxPQB8t X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 malwarescore=0 suspectscore=0 mlxscore=0 spamscore=0 lowpriorityscore=0 phishscore=0 clxscore=1015 adultscore=0 mlxlogscore=999 bulkscore=0 impostorscore=0 priorityscore=1501 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2602130000 definitions=main-2603020049 mlx5e_xdp_xmit() currently selects the xdpsq (send queue) using smp_processor_id() (i.e. cpu id). When doing XDP_REDIRECT from a cpu with id >=3D priv->channels.num, however, mlx5e_xdp_xmit() returns -ENXIO and the redirect fails. Previous discussion proposed using modulo or while loop subtraction in mlx5e_xdp_xmit() to map cpu id to send queue, but these approaches introduce hot path overhead on modern systems where the number of logical cores >> the number of XDP send queues (xdpsq). The below approach precomputes per-cpu priv->xdpsq assignments when channels are (re)configured and does a constant-time lookup in mlx5e_xdp_xmit(). Because multiple CPUs may now map to the same xdpsq (whenever cpu count exceeds channel count), we serialize writes in xdp_xmit with a tx_lock. Link: https://lore.kernel.org/all/610D8F9E-0038-46D9-AD8A-1D596236B1EF@spac= ex.com/ Link: https://lore.kernel.org/all/474c1f71-3a5c-4fe5-a01e-80f2ba95fd7e@byte= dance.com/ Signed-off-by: Finn Dayton --- v2: - Removed unnecessary guards - Improved variable naming and placement=20 - Change mapping from cpu -> index to cpu -> xdpsq=20 - Call smp_wmb() after updates to mapping=20 drivers/net/ethernet/mellanox/mlx5/core/en.h | 4 +++ .../net/ethernet/mellanox/mlx5/core/en/xdp.c | 17 ++++++------- .../net/ethernet/mellanox/mlx5/core/en_main.c | 25 +++++++++++++++++++ 3 files changed, 36 insertions(+), 10 deletions(-) diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en.h b/drivers/net/eth= ernet/mellanox/mlx5/core/en.h index ea2cd1f5d1d0..713dc7f9bae3 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en.h +++ b/drivers/net/ethernet/mellanox/mlx5/core/en.h @@ -493,6 +493,8 @@ struct mlx5e_xdpsq { u16 pc; struct mlx5_wqe_ctrl_seg *doorbell_cseg; struct mlx5e_tx_mpwqe mpwqe; + /* serialize writes by multiple CPUs to this send queue */ + spinlock_t tx_lock; =20 struct mlx5e_cq cq; =20 @@ -898,6 +900,8 @@ struct mlx5e_priv { struct mlx5e_selq selq; struct mlx5e_txqsq **txq2sq; struct mlx5e_sq_stats **txq2sq_stats; + /* selects the xdpsq during mlx5e_xdp_xmit() */ + struct mlx5e_xdpsq * __percpu *send_queue_ptr; =20 #ifdef CONFIG_MLX5_CORE_EN_DCB struct mlx5e_dcbx_dp dcbx_dp; diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en/xdp.c b/drivers/net= /ethernet/mellanox/mlx5/core/en/xdp.c index 80f9fc10877a..1db83a69055c 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en/xdp.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/en/xdp.c @@ -845,7 +845,6 @@ int mlx5e_xdp_xmit(struct net_device *dev, int n, struc= t xdp_frame **frames, struct mlx5e_priv *priv =3D netdev_priv(dev); struct mlx5e_xdpsq *sq; int nxmit =3D 0; - int sq_num; int i; =20 /* this flag is sufficient, no need to test internal sq state */ @@ -854,14 +853,12 @@ int mlx5e_xdp_xmit(struct net_device *dev, int n, str= uct xdp_frame **frames, =20 if (unlikely(flags & ~XDP_XMIT_FLAGS_MASK)) return -EINVAL; - - sq_num =3D smp_processor_id(); - - if (unlikely(sq_num >=3D priv->channels.num)) - return -ENXIO; - - sq =3D priv->channels.c[sq_num]->xdpsq; - + /* Per-CPU xdpsq mapping, rebuilt on channel (re)configuration while XDP = TX is disabled */ + sq =3D *this_cpu_ptr(priv->send_queue_ptr); + /* The number of queues configured on a netdev may be smaller than the + * CPU pool, so two CPUs might map to this queue. We must serialize write= s. + */ + spin_lock(&sq->tx_lock); for (i =3D 0; i < n; i++) { struct mlx5e_xmit_data_frags xdptxdf =3D {}; struct xdp_frame *xdpf =3D frames[i]; @@ -941,7 +938,7 @@ int mlx5e_xdp_xmit(struct net_device *dev, int n, struc= t xdp_frame **frames, =20 if (flags & XDP_XMIT_FLUSH) mlx5e_xmit_xdp_doorbell(sq); - + spin_unlock(&sq->tx_lock); return nxmit; } =20 diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_main.c b/drivers/ne= t/ethernet/mellanox/mlx5/core/en_main.c index b6c12460b54a..434db74f096b 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en_main.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_main.c @@ -1505,6 +1505,7 @@ static int mlx5e_alloc_xdpsq(struct mlx5e_channel *c, sq->stop_room =3D param->is_mpw ? mlx5e_stop_room_for_mpwqe(mdev) : mlx5e_stop_room_for_max_wqe(mdev); sq->max_sq_mpw_wqebbs =3D mlx5e_get_max_sq_aligned_wqebbs(mdev); + spin_lock_init(&sq->tx_lock); =20 param->wq.db_numa_node =3D cpu_to_node(c->cpu); err =3D mlx5_wq_cyc_create(mdev, ¶m->wq, sqc_wq, wq, &sq->wq_ctrl); @@ -3283,10 +3284,27 @@ static void mlx5e_build_txq_maps(struct mlx5e_priv = *priv) smp_wmb(); } =20 +static void mlx5e_build_xdpsq_maps(struct mlx5e_priv *priv) +{ + /* Build mapping from CPU id to XDP send queue, used by + * mlx5e_xdp_xmit() to determine which send queue to transmit packet on. + */ + int cpu; + + for_each_possible_cpu(cpu) { + int send_queue_idx =3D cpu % priv->channels.num; + struct mlx5e_xdpsq *sq =3D priv->channels.c[send_queue_idx]->xdpsq; + *per_cpu_ptr(priv->send_queue_ptr, cpu) =3D sq; + } + /* Publish the new CPU->xdpsq map before re-enabling XDP TX */ + smp_wmb(); +} + void mlx5e_activate_priv_channels(struct mlx5e_priv *priv) { mlx5e_build_txq_maps(priv); mlx5e_activate_channels(priv, &priv->channels); + mlx5e_build_xdpsq_maps(priv); mlx5e_xdp_tx_enable(priv); =20 /* dev_watchdog() wants all TX queues to be started when the carrier is @@ -6262,8 +6280,14 @@ int mlx5e_priv_init(struct mlx5e_priv *priv, if (!priv->fec_ranges) goto err_free_channel_stats; =20 + priv->send_queue_ptr =3D alloc_percpu(struct mlx5e_xdpsq *); + if (!priv->send_queue_ptr) + goto err_free_fec_ranges; + return 0; =20 +err_free_fec_ranges: + kfree(priv->fec_ranges); err_free_channel_stats: kfree(priv->channel_stats); err_free_tx_rates: @@ -6290,6 +6314,7 @@ void mlx5e_priv_cleanup(struct mlx5e_priv *priv) if (!priv->mdev) return; =20 + free_percpu(priv->send_queue_ptr); kfree(priv->fec_ranges); for (i =3D 0; i < priv->stats_nch; i++) kvfree(priv->channel_stats[i]); --=20 2.43.0