From nobody Sat Jul 25 03:48:19 2026 Received: from rcdn-iport-1.cisco.com (rcdn-iport-1.cisco.com [173.37.86.72]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B54EE2F8E81; Sun, 19 Jul 2026 08:45:16 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=173.37.86.72 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784450718; cv=none; b=AyZs3rwKwwfBVg6T35xJ4REyXNyb/KN+q4auUkMC4UOYrNtwkV5GadMdsOB7v/Ih8EPEgU90fDAayzyeo+Ej6+Yj96SzCRn0RtR25yiPd1JlEL/Z+lMa3Qr63uVrrHVZ7y8PPPZ3tHpTQ3ox2tWvQThFUgP0kwfQKYPrXJZmt+I= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784450718; c=relaxed/simple; bh=Wz8F6wfgDYuYsjWvaq0nfBuoX9f5woDLyMTg1gEV52c=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=nlpFiGjhmRwx0X//tJXvcg3m5BbuiXeL6/tvMJrGqyRduetwCx2DYN0KRuCfq+EKo1MmOxzGHDa135X50pMyoKTP80CCmMyMC6pK88yWRMv4qHJPI3IO7GCPhxHWQpNnTvxd9wvLk58ccllcQiWLqevnih5hpPjRSxmBrP9jbEM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=cisco.com; spf=pass smtp.mailfrom=cisco.com; dkim=pass (2048-bit key) header.d=cisco.com header.i=@cisco.com header.b=i5KG9jVH; arc=none smtp.client-ip=173.37.86.72 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=cisco.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=cisco.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=cisco.com header.i=@cisco.com header.b="i5KG9jVH" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cisco.com; i=@cisco.com; l=1225; q=dns/txt; s=iport01; t=1784450716; x=1785660316; h=from:date:subject:mime-version:content-transfer-encoding: message-id:references:in-reply-to:to:cc; bh=9Vvmc0A1inRMr3zmhnwnNdpnC36sbR/RofUydw34CYw=; b=i5KG9jVHW1GOkHUgnlDGnWUy8rGNDSuDJN7r0ikSaoc93RfnsA6pnm19 GTfgr0pZGRis1P8po9yn1Ilq0EtKtXoSz5iNK+nNuVVb2IXE92nBfTfAu 4mo+bBvsFRwKHTugbnTKBxH7MQxVrymE90NyA7+tAp+Qg7yAI2l9uHzLr r7N+4M+ANxCuUX2eXylTI6SSSsh2eeGyLQ5T8QDirI6fmepIokDinoVOf Lq6GPNlQGE7XlXtfkYXWelY4MELHyH+QS/mgrPyH3aafBplul+8qkMGvd OWPZcZ7vCLRpNer0qvZj9RtRS/BlHszfHtkWUdmctp+vgxE3aPMCMCrMs w==; X-CSE-ConnectionGUID: qh7vPOGJRqqOUhWXGdwKEA== X-CSE-MsgGUID: XhRLJI28Suq31Qe7WwX7gg== X-IPAS-Result: =?us-ascii?q?A0AmAAAEjVxq/5H/Ja1aHQEBAQEJARIBBQUBgXwIAQsBg?= =?us-ascii?q?laBUkNJhFeIG4c3giGeHhSBag8BAQEPUQQBAYUFAo1TAiY0CQ4BAgQDAgMBA?= =?us-ascii?q?QEBAQEBAQEBAQsBAQUBAQECAQcFgQ4ThlyGWwIBAyMEUhAlAiYCAkYQBgESg?= =?us-ascii?q?wKCdQO9C3p/M4EBhH3ZSIFmAQsUAYEKLgGFcIJrAYFwhAc7hEEnG4FJRIR+h?= =?us-ascii?q?CkBg3SCagSDHBKRbAk/gQIcA1ksAVUTDQoLBwVhUjMDIAoLEhIYFQIULw8EF?= =?us-ascii?q?jIdcAwnEiwXNFgbBwWBHX8WBCGBAoR2Ix8DOX+BMHVKdy1pARIXgS+CFIE6A?= =?us-ascii?q?gcDCxgNSBEsNwYOGQQ9AW4HjUAjgi8JB4EOFDiCV5ZhjV9roQ+EJ4FmoA8BM?= =?us-ascii?q?pdnkwUuh10JkHQjpDeEaYFoPIFZMxoIGxWDIlMZD44tFtI6JzI9AgcCBw4DC?= =?us-ascii?q?5FoBIF6AQE?= IronPort-Data: A9a23:qd1zpq3ABNh0pCt1G/bD5Q5wkn2cJEfYwER7XKvMYLTBsI5bpzYEz WIdUGmDPf7bNmb9Lo9+Oti0800B6p/UyIQxSQo53Hw8FHgiRegpqji6wuYcGwvIc6UvmWo+t 512huHodZ5yFjmH4E/xbtANlFEkvYmQXL3wFeXYDS54QA5gWU8JhAlq8wIDqtYAbeORXUXX5 Lsen+WFYAX7g2EtYzpNg06+gEoHUMra6WtwUmMWPZinjHeG/1EJAZQWI72GLneQauF8Au6gS u/f+6qy92Xf8g1FIovNfmHTKxBirhb6ZGBiu1IOM0SQqkEqSh8ajs7XAMEhhXJ/0F1lqTzeJ OJl7vRcQS9xVkHFdX90vxNwS0mSNoUekFPLzOTWXcG7lyX7n3XQL/pGFUFpI7U/1cRNGHhpy PwnGB8BcTOurrfjqF67YrEEasULNsLnOsYb/3pn1zycVaZgSpHYSKKM7thdtNsyrpkRRrCFO IxDNGcpNUidC/FMEg9/5JYWh+60h2L/cD1wo1OOrq1x6G/WpOB0+OW0aoKPJ43QHa25mG6op lji3UWjUioHLYfG8DfazW2c2eDAyHaTtIU6UefQGuRRqFGa2GEWIBEfS1a+pf63lgi4Vs43A 0gZ+zYnqoA++UaqR9+7WAe3yFaJsgMZUshdD8U17waCzqeS6AGcbkAHUzRIQN8rrsk7QXotz FDht9rvCSZir/6TRG6R+6m8qS60P24eLQcqfSYOQA0Ey8PurIE6klTESdMLOKq0iMDlXDL92 TaHqAAgiLgJy80GzaO2+RbAmT3Ejp7EUgI4+C3JUW+/qAB0foioY8qv81ez0BpbBJySQl/Eu D0PnNKTqbheS5qMjyeKBu4KGdlF+sq4DdEVunY3d7FJythn0yfLkVx4iN2mGHpUDw== IronPort-HdrOrdr: A9a23:uTH6864+h5HKxgHaswPXwD7XdLJyesId70hD6qm+c20wTiX+ra GTdZUguyMc5wx9ZJhNo7q90ey7MBHhHP1OjrX5Q43DYOCOggLBR72Kr7GSoAEIcBeTygcy79 YET4FOTPD9ElR+i9/3+02bH8ZI+qj/zImYwcrT0HtpSxhncOVb7wl/AhuGCUEefnghOXNALu v62uN34xy9ZH8QcsO3QlMBX+TFupn3saiOW29/O/bigzP+9w9BL9XBYmOl4is= X-Talos-CUID: 9a23:mglFqGGQwDZZSvOxqmJ+9BEKIucKUEfE1VDtI3CIJmp2a5C8HAo= X-Talos-MUID: 9a23:yzy5+wgm7G3HGxWgu58oP8MpBMdSx4aPUhA2jIxd4vCJahNvC2iSg2Hi X-IronPort-Anti-Spam-Filtered: true X-IronPort-AV: E=Sophos;i="6.25,172,1779148800"; d="scan'208";a="512025920" Received: from rcdn-l-core-08.cisco.com ([173.37.255.145]) by rcdn-iport-1.cisco.com with ESMTP/TLS/TLS_AES_256_GCM_SHA384; 19 Jul 2026 08:42:06 +0000 Received: from [127.0.0.1] (unknown [10.25.130.254]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by rcdn-l-core-08.cisco.com (Postfix) with ESMTPS id ABFA818000472; Sun, 19 Jul 2026 08:42:04 +0000 (GMT) From: Satish Kharat Date: Sun, 19 Jul 2026 01:41:45 -0700 Subject: [PATCH net-next v12 01/12] enic: cancel tx_hang_reset work on device removal Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260719-enic-sriov-v2-admin-channel-v2-v12-1-6ff2c617001d@cisco.com> References: <20260719-enic-sriov-v2-admin-channel-v2-v12-0-6ff2c617001d@cisco.com> In-Reply-To: <20260719-enic-sriov-v2-admin-channel-v2-v12-0-6ff2c617001d@cisco.com> To: Andrew Lunn , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni Cc: netdev@vger.kernel.org, linux-kernel@vger.kernel.org, Breno Leitao , Sesidhar Baddela , Satish Kharat X-Mailer: b4 0.15.2 X-Developer-Signature: v=1; a=ed25519-sha256; t=1784450523; l=1239; i=satishkh@cisco.com; h=from:subject:message-id; bh=Wz8F6wfgDYuYsjWvaq0nfBuoX9f5woDLyMTg1gEV52c=; b=5XKYuKI506vAZNaNbnmHT4wQrP8Ib6GW49LUxyglbEAHgl2RMV0j0pwp4UUJ10pM8Xt2hMsnr eBuqdBk/z0EBVogVFAs7lpkgii2qcK+E7DoBwHpt8ekjio9g1DpfzhB X-Developer-Key: i=satishkh@cisco.com; a=ed25519; pk=4BxbLoCGvHeIMH/PbpiNVx8/2IcTjFvqcr0AD8K6WLY= X-Outbound-Client-TLS: ANONYMOUS;unknown [10.25.130.254];TLSv1.3;TLS_AES_256_GCM_SHA384;256 X-Outbound-SMTP-Client: 10.25.130.254, [10.25.130.254] X-Outbound-Node: rcdn-l-core-08.cisco.com enic_remove() cancels the reset and change_mtu_work items but does not cancel tx_hang_reset. A TX timeout that fires while the device is being removed can schedule enic_tx_hang_reset() so that it runs after free_netdev(), resulting in a use-after-free. Cancel tx_hang_reset alongside the other work items before unregister_netdev(). This is a pre-existing issue, not introduced by the SR-IOV V2 series; it is included here as an independent fix. Fixes: 937317c7c109 ("enic: do hang reset only in case of tx timeout") Signed-off-by: Satish Kharat --- drivers/net/ethernet/cisco/enic/enic_main.c | 1 + 1 file changed, 1 insertion(+) diff --git a/drivers/net/ethernet/cisco/enic/enic_main.c b/drivers/net/ethe= rnet/cisco/enic/enic_main.c index e7125b818087..b65796d96efc 100644 --- a/drivers/net/ethernet/cisco/enic/enic_main.c +++ b/drivers/net/ethernet/cisco/enic/enic_main.c @@ -3012,6 +3012,7 @@ static void enic_remove(struct pci_dev *pdev) struct enic *enic =3D netdev_priv(netdev); =20 cancel_work_sync(&enic->reset); + cancel_work_sync(&enic->tx_hang_reset); cancel_work_sync(&enic->change_mtu_work); unregister_netdev(netdev); enic_dev_deinit(enic); --=20 2.43.0 From nobody Sat Jul 25 03:48:19 2026 Received: from rcdn-iport-2.cisco.com (rcdn-iport-2.cisco.com [173.37.86.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A232736D4E1; Sun, 19 Jul 2026 08:43:14 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=173.37.86.73 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784450596; cv=none; b=KVG/jPxpkrVLYCpmr5fPhiMk9sBRRH5KZXar4498ptYs7jr0cC0//2gXr3iTNYzjfJtuIriFmtVdh/6IfiQaXd2x8+ouWXanSYUFlT7t8H3z2EMQVMCV2eKCHOsC5gsJa54LujQb4+6MF5DtmN8jnr36ZXu7a14mSP2Wc6oLSD4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784450596; c=relaxed/simple; bh=IsES+n+TKvyj9aUMCeVzccPUfuHCV8lfNTLLbzVDWrg=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=mEALKQdz2XHWdzyQjQwImXqvi1POLvZXDxMLkMPqn6DC1+PL1SiYXIK7KPsvsLxapPwmOxYYqTLzewuE1Pu+q0vgKKE7tkcoauk07KJ0w1tUXfPM0jRCBhpTZD6j4uATrsAYnpEk3JAGDcgnRgP+u3vSUl6WGEWAk34j2lxmS1A= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=cisco.com; spf=pass smtp.mailfrom=cisco.com; dkim=pass (2048-bit key) header.d=cisco.com header.i=@cisco.com header.b=PehwJYm2; arc=none smtp.client-ip=173.37.86.73 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=cisco.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=cisco.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=cisco.com header.i=@cisco.com header.b="PehwJYm2" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cisco.com; i=@cisco.com; l=2833; q=dns/txt; s=iport01; t=1784450594; x=1785660194; h=from:date:subject:mime-version:content-transfer-encoding: message-id:references:in-reply-to:to:cc; bh=VOP1yf0LvjOJU23ulXauD+xcRUgLPP0kWs2v2oK4H8E=; b=PehwJYm2l4N2UltXxdLhmBZoUM7YAYuhd92bFwHr5vbHemcIAmHGZaoV Rji1kBJc8+qNPIM2IYcd6SnK5j/K3RvIzaJL9E5VT0Mp42nUl8+Y5jbhK oYCMwmWOBe3Q1Kdl++cMVALMXiExObcgkGgEqwRGUvClWEfpoTN1OzJfi o7f4vBMOb7gU8xVo7iioLDCx3kc6SdFPx9gfakmrMeZraSLs3Q0F5tr70 s1qv5BibxJf9XOcLRq2zI3f51WuMXa+v5hSVeoTQcRLxzqXM+pmNgyuvW IMywUtpv/f09hwm8T2E2ZMihfB91IHfbq00qeCmNV3GVrlW2qxjkEO3rd g==; X-CSE-ConnectionGUID: DaPteFQkTKye5CXSUxZXXA== X-CSE-MsgGUID: XjZUp4fNQZ+ICat/KETKNA== X-IPAS-Result: =?us-ascii?q?A0BFAgBUjFxq/5H/Ja1aHgEBCxIMggULgleBUkNJhFePU?= =?us-ascii?q?oIhnh6Bfg8BAQEPUQQBAYUFAo1TAiY0CQ4BAgQDAgMBAQEBAQEBAQEBAQsBA?= =?us-ascii?q?QUBAQECAQcFgQ4ThlyGWwIBAyMEUhAlAiYCAkYQBgESgwKCdQO9CXp/M4EBh?= =?us-ascii?q?H3ZSIFmAQsUAYEKLoVxgmsBgXCEBzuEQScbgUlEgRWDaYQKhBSCagSDHBKRb?= =?us-ascii?q?Ak/gQIcA1ksAVUTDQoLBwVhUjMDIAoLEhIYFQIULw8EFjIdcAwnEiwXNFgbB?= =?us-ascii?q?wWBHX8WBCGBAoR2Ix8DOX+BMHVKdy1pARIXgS+CFIE6AgcDCxgNSBEsNwYOG?= =?us-ascii?q?QQ9AW4HjUAjgU5xPVKCPykpk0CDMo1fa6EPhCeBZqAPATKXZ5MFLJhcI6QAN?= =?us-ascii?q?4RpgWg8gVkzGggbFYMiUxkPji0WHNIeJzI+AQcCBw8CgXOQAIF+AQE?= IronPort-Data: A9a23:uRSOyqOjsnoIQgfvrR14lsFynXyQoLVcMsEvi/4bfWQNrUpz0TMHy WNOC2GHa/fZYGX0LookaoS380wDsJLSmNRhS3M5pCpnJ55oRWUpJjg4wmPYZX76whjrFRo/h ykmQoCeaphyFTmE+kvF3oHJ9RFUzbuPSqf3FNnKMyVwQR4MYCo6gHqPocZh6mJTqYb/WV7lV e/a+ZWFZgf6gm8sawr41orawP9RlKWq0N8nlgRWicBj5Df2i3QTBZQDEqC9R1OQapVUBOOzW 9HYx7i/+G7Dlz91Yj9yuu+mGqGiaue60Tmm0hK6aYD76vRxjnBaPpIACRYpQRw/ZwNlMDxG4 I4lWZSYEW/FN0BX8QgXe0Ew/ypWZcWq9FJbSJSymZT78qHIT5fj69lnNF07E7QbxsVmDDxlx cclOAI9XjnW0opawJrjIgVtrt4oIM+uOMYUvWttiGmJS/0nWpvEBa7N4Le03h9p2ZsIRqmYP ZdEL2MzMXwsYDUXUrsTIIk3gOazhnr/WzZZs1mS46Ew5gA/ySQtgei1bYCFJ4TiqcN9rkSqm 3qa8ULAMFI7H9WDjhmozlGKr7qa9c/8cMdIfFGizdZmiUOew0QfAQMbUF+8r+X/jEOiM/pbJ k0I8y0GoaU49UWtCN7nUHWQrXKZuxcOVsZ4HOo95wWAjKHT5m6xBXUORxZCZcYguctwQiYlv neAmd/zCCMstrCJRX+D3rOJqzX0Mih9BXcLbyICTCMf7tXjqZ11hRXKJv5nEaionpj2FCv2z jSisicznfMQgNQN2qH9+krI6xqqq4TFQxAd+AraRCSm4xl/aYrjYJangWU39t5aJ4qfC13Et 38elo3GvKYFDIqGk2qGR+Bl8KyV2stp+Qb02TZHd6TNPRz2k5J/Vei8OA1DGXo= IronPort-HdrOrdr: A9a23:C3eVvKror7bkWpYhAq8s5G0aV5rdeYIsimQD101hICG9vPbo8/ xG+8576faUslgssR4b9OxoVJPrfZqYz+8W3WBzB9iftXfd2FdARbsKheDfKn/bak7DH4VmuZ uIHZIObeHYPBxWgdn35Q+gH81l4tWWmZrY/Nv2/jNBQR5nbqd44xw8MAaUFUVqWBJLbKBJcq Z0nvA3wAaISDAwctmxAGUDUqzoot3Gk4+jXDs9biRXkDVnSViTmcTH+9/y5GZkbw9y X-Talos-CUID: =?us-ascii?q?9a23=3A4g/sSWsTHH1W7GmNgauBuHIx6IsJe0XBwVnpIHO?= =?us-ascii?q?bGGs0Y42udE+iyJNNxp8=3D?= X-Talos-MUID: 9a23:H6kdIAuAotdOV5yV982ntDNQCd807YKXOgMsupwa/OqpCQ9WEmLI X-IronPort-Anti-Spam-Filtered: true X-IronPort-AV: E=Sophos;i="6.25,172,1779148800"; d="scan'208";a="498052850" Received: from rcdn-l-core-08.cisco.com ([173.37.255.145]) by rcdn-iport-2.cisco.com with ESMTP/TLS/TLS_AES_256_GCM_SHA384; 19 Jul 2026 08:42:06 +0000 Received: from [127.0.0.1] (unknown [10.25.130.254]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by rcdn-l-core-08.cisco.com (Postfix) with ESMTPS id 7218218000475; Sun, 19 Jul 2026 08:42:05 +0000 (GMT) From: Satish Kharat Date: Sun, 19 Jul 2026 01:41:46 -0700 Subject: [PATCH net-next v12 02/12] enic: verify firmware supports V2 SR-IOV at probe time Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260719-enic-sriov-v2-admin-channel-v2-v12-2-6ff2c617001d@cisco.com> References: <20260719-enic-sriov-v2-admin-channel-v2-v12-0-6ff2c617001d@cisco.com> In-Reply-To: <20260719-enic-sriov-v2-admin-channel-v2-v12-0-6ff2c617001d@cisco.com> To: Andrew Lunn , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni Cc: netdev@vger.kernel.org, linux-kernel@vger.kernel.org, Breno Leitao , Sesidhar Baddela , Satish Kharat X-Mailer: b4 0.15.2 X-Developer-Signature: v=1; a=ed25519-sha256; t=1784450523; l=2882; i=satishkh@cisco.com; h=from:subject:message-id; bh=IsES+n+TKvyj9aUMCeVzccPUfuHCV8lfNTLLbzVDWrg=; b=S9m0uBg25fHJ3aQrrVhBaJi/zO3mfBXzmTTt1NndEONBULy/mYYNW99UAc8Z/sCM6MWSHAmhi 4kchWvJIkfOBBuEMu4TXjX52BmdJ/yXPgxFLak4fr3FJl24706P5jTV X-Developer-Key: i=satishkh@cisco.com; a=ed25519; pk=4BxbLoCGvHeIMH/PbpiNVx8/2IcTjFvqcr0AD8K6WLY= X-Outbound-Client-TLS: ANONYMOUS;unknown [10.25.130.254];TLSv1.3;TLS_AES_256_GCM_SHA384;256 X-Outbound-SMTP-Client: 10.25.130.254, [10.25.130.254] X-Outbound-Node: rcdn-l-core-08.cisco.com During PF probe, query the firmware get-supported-feature interface to verify that the running firmware supports V2 SR-IOV. Firmware version 5.3(4.72) and later report VIC_FEATURE_SRIOV via CMD_GET_SUPP_FEATURE_VER. If the firmware does not support the feature, set vf_type to ENIC_VF_TYPE_NONE and log a warning so the admin knows a firmware upgrade is needed. The V2 admin-channel and MBOX bring-up added later in this series is gated on ENIC_VF_TYPE_V2, so this downgrade keeps those paths from running on firmware that does not support V2 SR-IOV. VIC_FEATURE_SRIOV is assigned the explicit value 4 to match the firmware ABI. Slot 3 (firmware's VIC_FEATURE_PTP) is reserved with a comment rather than a placeholder enum entry, since PTP is not used by the upstream driver. Suggested-by: Breno Leitao Signed-off-by: Satish Kharat Reviewed-by: Breno Leitao --- drivers/net/ethernet/cisco/enic/enic_main.c | 21 ++++++++++++++++++++- drivers/net/ethernet/cisco/enic/vnic_devcmd.h | 2 ++ 2 files changed, 22 insertions(+), 1 deletion(-) diff --git a/drivers/net/ethernet/cisco/enic/enic_main.c b/drivers/net/ethe= rnet/cisco/enic/enic_main.c index b65796d96efc..6992411bd3b5 100644 --- a/drivers/net/ethernet/cisco/enic/enic_main.c +++ b/drivers/net/ethernet/cisco/enic/enic_main.c @@ -2641,8 +2641,10 @@ static void enic_iounmap(struct enic *enic) static void enic_sriov_detect_vf_type(struct enic *enic) { struct pci_dev *pdev =3D enic->pdev; - int pos; + u64 supported_versions, a1 =3D 0; u16 vf_dev_id; + int pos; + int err; =20 if (enic_is_sriov_vf(enic) || enic_is_dynamic(enic)) return; @@ -2669,6 +2671,23 @@ static void enic_sriov_detect_vf_type(struct enic *e= nic) enic->vf_type =3D ENIC_VF_TYPE_NONE; break; } + + if (enic->vf_type !=3D ENIC_VF_TYPE_V2) + return; + + /* A successful command means firmware recognizes + * VIC_FEATURE_SRIOV; supported_versions is available + * for sub-feature versioning in the future. + */ + err =3D vnic_dev_get_supported_feature_ver(enic->vdev, + VIC_FEATURE_SRIOV, + &supported_versions, + &a1); + if (err) { + dev_warn(&pdev->dev, + "SR-IOV V2 not supported by current firmware. Upgrade to VIC FW 5.3(4.= 72) or higher.\n"); + enic->vf_type =3D ENIC_VF_TYPE_NONE; + } } #endif =20 diff --git a/drivers/net/ethernet/cisco/enic/vnic_devcmd.h b/drivers/net/et= hernet/cisco/enic/vnic_devcmd.h index 605ef17f967e..3b6efa743dba 100644 --- a/drivers/net/ethernet/cisco/enic/vnic_devcmd.h +++ b/drivers/net/ethernet/cisco/enic/vnic_devcmd.h @@ -734,6 +734,8 @@ enum vic_feature_t { VIC_FEATURE_VXLAN, VIC_FEATURE_RDMA, VIC_FEATURE_VXLAN_PATCH, + /* slot 3 reserved for firmware VIC_FEATURE_PTP */ + VIC_FEATURE_SRIOV =3D 4, VIC_FEATURE_MAX, }; =20 --=20 2.43.0 From nobody Sat Jul 25 03:48:19 2026 Received: from rcdn-iport-1.cisco.com (rcdn-iport-1.cisco.com [173.37.86.72]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6EDC532B99F; Sun, 19 Jul 2026 08:45:18 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=173.37.86.72 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784450721; cv=none; b=t33N5N49e6MkihEXni6KTth+Q++ahRWIJXwkcglZH8cXhT3fo2cy0lVdbVStkLcNgctiezepPdlbupUUGoASXFL70D5UhuFoNbhQsZsLSR0u4eAfRqdMSW34pi31Azf2rnkvgwDs/iN8BefkLREQIGpqKd51cINKJuDOeAxlaaM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784450721; c=relaxed/simple; bh=CNyyNpBKXvdWxQ+JKqXEuJWhIy34HxDBjCH+Kpt0yts=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=atcNo5U/sROGzRZn4ogXkO8RMCpV6luM2gKZA4qv8afi40ib6EsMHIbrAIMdUW3nviDNQerCGovwr/AQKmhkUfZs7tte6v30jkmtWS/KMmm/s3K+f+RJ7aEXoMxZX8e656YxRkCZfo386mFJNzji/TbGbJvQO8oTKoy6qtVk37k= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=cisco.com; spf=pass smtp.mailfrom=cisco.com; dkim=pass (2048-bit key) header.d=cisco.com header.i=@cisco.com header.b=Bl9oV5Ek; arc=none smtp.client-ip=173.37.86.72 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=cisco.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=cisco.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=cisco.com header.i=@cisco.com header.b="Bl9oV5Ek" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cisco.com; i=@cisco.com; l=11545; q=dns/txt; s=iport01; t=1784450718; x=1785660318; h=from:date:subject:mime-version:content-transfer-encoding: message-id:references:in-reply-to:to:cc; bh=Sorf0jTZuFVvTbMwN7h92xT9arsCuGLeYoqT+B2G15s=; b=Bl9oV5EkK8s8CjA+gdf3SMCvled+n+1w7iDIlnZIhoAbSMy4vY1HMcRs 8JsaVpYzpLLSTeOSPC//K/P32ypRIyNPqKUa+gfmtLCX4Rb1AkgShWdWt HKZl8TROUKyefjHxRH20svzqp/CID8iNN0gYYBSpa+S+Hz835JMdgcOte HL/OYXP47ZK9gRwmdYlRNaZppjZd+1aH/EuVfQOPxYfxhWptie/T5u7IH 8k4SJmcA5tXYWPBfMBWDOBsfD7ys1MD3UYd1IkroJH1k2prK06miLoCQ7 X1XE+2PsG+xZ27myDxOjrJn0ZjrFnJrQFMkoi0HlUDWJrIyI4imWzdInx g==; X-CSE-ConnectionGUID: 1SmOphp+T16gD0/a33vQng== X-CSE-MsgGUID: 0Woiw2zASuOcK2VPwI8g9Q== X-IPAS-Result: =?us-ascii?q?A0CrAAAEjVxq/5H/Ja1aHQEBAQEJARIBBQUBgXwIAQsBg?= =?us-ascii?q?laBTgRDSYRXiBuHN4Ihnh4UgWoPAQEBD1EEAQGFBQKNUwImNAkOAQIEAwIDA?= =?us-ascii?q?QEBAQEBAQEBAQELAQEFAQEBAgEHBYEOE4ZchlsCAQMjBFIQJQImAgJGEAYBE?= =?us-ascii?q?oMCgnUDvQt6fzOBAYR92UiBZgELFAGBCi4BhXCCawGBcIQHO4RBJxuBSUSBF?= =?us-ascii?q?AGDaYQFAQQVC4N0gmoEgxwSgXiPdAk/gQIcA1ksAVUTDQoLBwVhUjMDIAoLE?= =?us-ascii?q?hIYFQIULw8EFjIdcAwnEiwXNFgbBwWBHX8WBCGBAoR2Ix8DOX+BMHVKdy1pA?= =?us-ascii?q?RIXgS+CFIE6AgcDCxgNSBEsNwYOGQQ9AW4HjUAjgTcXcVk1AU1vgSEcLJJgC?= =?us-ascii?q?IN2jV9rgTWfWoQngWagDwEyl2eTBZkII6NJboRpgWg8gVkzGggbFYMiUxkPj?= =?us-ascii?q?ioDFtI6JzI9AgcCBw4DC5FoEReBVgEB?= IronPort-Data: A9a23:35FFd6KsYo3ONrwRFE+RDJQlxSXFcZb7ZxGr2PjKsXjdYENS1GcEn zEdXm2AbquKYmCjfox3bd+y90tT7ZaHnIdnHVEd+CA2RRqmiyZq6fd1j6vUF3nPRiEWZBs/t 63yUvGZcoZsCCSa/kvxWlTYhSEU/bmSQbbhA/LzNCl0RAt1IA8skhsLd9QR2uaEuvDnRVnQ0 T/Oi5eHYgH9hGclajt8B5+r8XuDgtyj4Fv0gXRmDRx7lAe2v2UYCpsZOZawIxPQKqFIHvS3T vr017qw+GXU5X8FUrtJRZ6iLyXm6paLVeS/oiI+t5qK23CulQRuukoPD8fwXG8M49m/c3+d/ /0W3XC4YV9B0qQhA43xWTEAe811FfUuFLMqvRFTvOTLp3AqfUcAzN03N0QRJY0AudouX25ir bsRc2sTVxaM0rfeLLKTEoGAh+w5J8XteYdasXZ6wHSBXbAtQIvIROPB4towMDUY358VW62BI ZBENHw2MEmojx5nYj/7DLollfupnXn8cxVTqUmeouw85G27IAlZjeG3aICIK4bVLSlTth6no WPZpXbQOQw9P//F9wKa0yKj19aayEsXX6pXTtVU7MVCjFSNy2k7BBQIWF6/pvelzEizR7p3L 0EV5iMphaQ//k2iSJ/6RRLQiHOIoxsVSddMO+I97w6Jx+zf5APxLmsZRDdpa9E8ssIyAzsw2 TehltXlHzFw87uYVX6Q6J+QsDW0fyMVRUcabCUJSwot+dTvoIgvyBnIS75LFK+zk82wGjzqx T2OhDYxiq9VjsMR0ai/u1fdjFqEoJXVQgMrzhvYU3jj7Q5jYoOhIYuy5jDmAe1oNo2VSByF+ XMDgcXbtLBIBpCWnyvLS+IIdF2028u43PTnqQYHN/EcG/6FoRZPoag4DOlCGXpU IronPort-HdrOrdr: A9a23:lPgDEK1PDDyhE6ZceuRv+AqjBEQkLtp133Aq2lEZdPWaSL39qy nIppUmPH7P5gr5N0tMpTntAsO9qDbnhPxICOoqUotKIjOHhILAFugLhrcKgQeQeBEWndQw6U 4PSdkaNPTASXV3ksr+5hC1CJIDzMnvytHQuc7ui1pgUg1ubbht9ENCCgidGlBrXwUuP+teKH Pl3LshmxOQPVIKc8+8Gn4JG8LHp9HNidbaRCRuPW9e1OFL5gnYkoIT1HOjr2wjbw8= X-Talos-CUID: 9a23:E4MXcm0+fqRK0jl9XjCw2LxfQ/sgVWXd1DTsehG3JT4uQebOV2aW9/Yx X-Talos-MUID: 9a23:UtXzjwidTeRJG489h1lgfMMpbPhv4rv0Gmk2jLINmJO6DTNRazjMtWHi X-IronPort-Anti-Spam-Filtered: true X-IronPort-AV: E=Sophos;i="6.25,172,1779148800"; d="scan'208";a="512025927" Received: from rcdn-l-core-08.cisco.com ([173.37.255.145]) by rcdn-iport-1.cisco.com with ESMTP/TLS/TLS_AES_256_GCM_SHA384; 19 Jul 2026 08:42:07 +0000 Received: from [127.0.0.1] (unknown [10.25.130.254]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by rcdn-l-core-08.cisco.com (Postfix) with ESMTPS id 398AF180005B7; Sun, 19 Jul 2026 08:42:06 +0000 (GMT) From: Satish Kharat Date: Sun, 19 Jul 2026 01:41:47 -0700 Subject: [PATCH net-next v12 03/12] enic: add admin channel open and close for SR-IOV Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260719-enic-sriov-v2-admin-channel-v2-v12-3-6ff2c617001d@cisco.com> References: <20260719-enic-sriov-v2-admin-channel-v2-v12-0-6ff2c617001d@cisco.com> In-Reply-To: <20260719-enic-sriov-v2-admin-channel-v2-v12-0-6ff2c617001d@cisco.com> To: Andrew Lunn , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni Cc: netdev@vger.kernel.org, linux-kernel@vger.kernel.org, Breno Leitao , Sesidhar Baddela , Satish Kharat X-Mailer: b4 0.15.2 X-Developer-Signature: v=1; a=ed25519-sha256; t=1784450523; l=11940; i=satishkh@cisco.com; h=from:subject:message-id; bh=CNyyNpBKXvdWxQ+JKqXEuJWhIy34HxDBjCH+Kpt0yts=; b=4pRiEpZftbK4bwa2nsnKgl0SiDqpr6hhZdpvsyaceBnaA0lflru/RrEN/rHf+7hWb/+mvl9Aq 2oOYTQ8DUP3BWcmvX8Syi3HJBkN+5bnEY377mC0oJEZnbj8T7MFRt/d X-Developer-Key: i=satishkh@cisco.com; a=ed25519; pk=4BxbLoCGvHeIMH/PbpiNVx8/2IcTjFvqcr0AD8K6WLY= X-Outbound-Client-TLS: ANONYMOUS;unknown [10.25.130.254];TLSv1.3;TLS_AES_256_GCM_SHA384;256 X-Outbound-SMTP-Client: 10.25.130.254, [10.25.130.254] X-Outbound-Node: rcdn-l-core-08.cisco.com The V2 SR-IOV design uses a dedicated admin channel (WQ/RQ/CQ/INTR on separate BAR resources) for PF-VF mailbox communication rather than firmware-proxied devcmds. Introduce enic_admin_channel_open() and enic_admin_channel_close(). Open allocates and initialises the admin WQ, RQ, and two CQs (one per direction), then issues CMD_QP_TYPE_SET to tell firmware the queues are admin-type. Close reverses the sequence. enic_admin_wq_buf_clean() unmaps and frees any WQ buffers still held at close time, fixing a DMA mapping leak when a send times out. Add CMD_QP_TYPE_SET (97), QP_TYPE_ADMIN/DATA, and QP_ENABLE/QP_DISABLE defines to vnic_devcmd.h. Add VNIC_CQ_* named constants to vnic_cq.h so CQ initialisation parameters are self-documenting from their first introduction. Signed-off-by: Satish Kharat --- drivers/net/ethernet/cisco/enic/Makefile | 3 +- drivers/net/ethernet/cisco/enic/enic.h | 5 + drivers/net/ethernet/cisco/enic/enic_admin.c | 227 ++++++++++++++++++++++= ++++ drivers/net/ethernet/cisco/enic/enic_admin.h | 15 ++ drivers/net/ethernet/cisco/enic/vnic_cq.h | 9 + drivers/net/ethernet/cisco/enic/vnic_devcmd.h | 11 ++ 6 files changed, 269 insertions(+), 1 deletion(-) diff --git a/drivers/net/ethernet/cisco/enic/Makefile b/drivers/net/etherne= t/cisco/enic/Makefile index a96b8332e6e2..7ae72fefc99a 100644 --- a/drivers/net/ethernet/cisco/enic/Makefile +++ b/drivers/net/ethernet/cisco/enic/Makefile @@ -3,5 +3,6 @@ obj-$(CONFIG_ENIC) :=3D enic.o =20 enic-y :=3D enic_main.o vnic_cq.o vnic_intr.o vnic_wq.o \ enic_res.o enic_dev.o enic_pp.o vnic_dev.o vnic_rq.o vnic_vic.o \ - enic_ethtool.o enic_api.o enic_clsf.o enic_rq.o enic_wq.o + enic_ethtool.o enic_api.o enic_clsf.o enic_rq.o enic_wq.o \ + enic_admin.o =20 diff --git a/drivers/net/ethernet/cisco/enic/enic.h b/drivers/net/ethernet/= cisco/enic/enic.h index 08472420f3a1..398227448b37 100644 --- a/drivers/net/ethernet/cisco/enic/enic.h +++ b/drivers/net/ethernet/cisco/enic/enic.h @@ -292,6 +292,11 @@ struct enic { =20 /* Admin channel resources for SR-IOV MBOX */ bool has_admin_channel; + /* true only while the admin WQ/RQ/CQ are allocated and enabled; gates + * enic_admin_channel_close() so it is a no-op after a failed (re)open + * left the resources freed. + */ + bool admin_chan_up; struct vnic_wq admin_wq; struct vnic_rq admin_rq; struct vnic_cq admin_cq[2]; diff --git a/drivers/net/ethernet/cisco/enic/enic_admin.c b/drivers/net/eth= ernet/cisco/enic/enic_admin.c new file mode 100644 index 000000000000..50b46b92c88f --- /dev/null +++ b/drivers/net/ethernet/cisco/enic/enic_admin.c @@ -0,0 +1,227 @@ +// SPDX-License-Identifier: GPL-2.0-only +// Copyright 2025 Cisco Systems, Inc. All rights reserved. + +#include +#include + +#include "vnic_dev.h" +#include "vnic_wq.h" +#include "vnic_rq.h" +#include "vnic_cq.h" +#include "vnic_intr.h" +#include "vnic_resource.h" +#include "vnic_devcmd.h" +#include "enic.h" +#include "enic_admin.h" +#include "cq_desc.h" +#include "wq_enet_desc.h" +#include "rq_enet_desc.h" + +/* Clean up any admin WQ buffers still held by hardware at close time. + * Normally buffers are freed inline after send completion, but a timed-out + * send intentionally leaves the buffer live until the queue is stopped. + */ +static void enic_admin_wq_buf_clean(struct vnic_wq *wq, + struct vnic_wq_buf *buf) +{ + struct enic *enic =3D vnic_dev_priv(wq->vdev); + + if (buf->os_buf) { + dma_unmap_single(&enic->pdev->dev, buf->dma_addr, + buf->len, DMA_TO_DEVICE); + kfree(buf->os_buf); + buf->os_buf =3D NULL; + } +} + +/* No-op: admin RQ buffer teardown is handled in enic_admin_channel_close = */ +static void enic_admin_rq_buf_clean(struct vnic_rq *rq, + struct vnic_rq_buf *buf) +{ +} + +static int enic_admin_qp_type_set(struct enic *enic, u32 enable) +{ + u64 a0 =3D QP_TYPE_ADMIN, a1 =3D enable; + int wait =3D 1000; + int err; + + spin_lock_bh(&enic->devcmd_lock); + err =3D vnic_dev_cmd(enic->vdev, CMD_QP_TYPE_SET, &a0, &a1, wait); + spin_unlock_bh(&enic->devcmd_lock); + + return err; +} + +static int enic_admin_alloc_resources(struct enic *enic) +{ + int err; + + err =3D vnic_wq_alloc_with_type(enic->vdev, &enic->admin_wq, 0, + ENIC_ADMIN_DESC_COUNT, + sizeof(struct wq_enet_desc), + RES_TYPE_ADMIN_WQ); + if (err) + return err; + + err =3D vnic_rq_alloc_with_type(enic->vdev, &enic->admin_rq, 0, + ENIC_ADMIN_DESC_COUNT, + sizeof(struct rq_enet_desc), + RES_TYPE_ADMIN_RQ); + if (err) + goto free_wq; + + /* admin_cq[0] is the WQ completion queue. WQ CQEs are always + * 16 bytes wide; firmware always writes 16-byte CQEs for WQ + * completions on every WQ, including the admin channel WQ. + * Use sizeof(struct cq_desc) accordingly. + */ + err =3D vnic_cq_alloc_with_type(enic->vdev, &enic->admin_cq[0], 0, + ENIC_ADMIN_DESC_COUNT, + sizeof(struct cq_desc), + RES_TYPE_ADMIN_CQ); + if (err) + goto free_rq; + + /* admin_cq[1] is the RQ completion queue. Its descriptor size + * must match what firmware writes. enic_ext_cq() called earlier + * in probe issues CMD_CQ_ENTRY_SIZE_SET for VNIC_RQ_ALL, + * programming firmware to write CQ entries of (16 << enic->ext_cq) + * bytes for every RQ CQ on the vNIC, including the admin RQ CQ. + * Allocating with the same size keeps the host poller and + * firmware in lockstep: + * + * - The color/valid bit lives at byte (desc_size - 1) of every + * cq_enet_rq_desc[_32|_64] variant, so enic_admin_cq_color() + * reads it from the correct offset. + * - Only the first 15 bytes of the descriptor (vlan, + * bytes_written_flags, ...) are accessed by the admin path; + * these fields are identical across all three variants (see + * comment in enic_rq.c above cq_enet_rq_desc_dec()). + */ + err =3D vnic_cq_alloc_with_type(enic->vdev, &enic->admin_cq[1], 1, + ENIC_ADMIN_DESC_COUNT, + 16 << enic->ext_cq, + RES_TYPE_ADMIN_CQ); + if (err) + goto free_cq0; + + return 0; + +free_cq0: + vnic_cq_free(&enic->admin_cq[0]); +free_rq: + vnic_rq_free(&enic->admin_rq); +free_wq: + vnic_wq_free(&enic->admin_wq); + return err; +} + +static void enic_admin_free_resources(struct enic *enic) +{ + vnic_cq_free(&enic->admin_cq[1]); + vnic_cq_free(&enic->admin_cq[0]); + vnic_rq_free(&enic->admin_rq); + vnic_wq_free(&enic->admin_wq); +} + +static void enic_admin_init_resources(struct enic *enic) +{ + vnic_wq_init(&enic->admin_wq, + 0, 0, 0); /* cq_index, err_intr_enable, err_intr_offset */ + vnic_rq_init(&enic->admin_rq, + 1, 0, 0); /* cq_index, err_intr_enable, err_intr_offset */ + vnic_cq_init(&enic->admin_cq[0], + VNIC_CQ_FC_DISABLE, + VNIC_CQ_COLOR_ENABLE, + 0, 0, 1, /* cq_head, cq_tail, cq_tail_color */ + VNIC_CQ_INTR_DISABLE, + VNIC_CQ_ENTRY_ENABLE, + VNIC_CQ_MSG_DISABLE, + 0, /* interrupt_offset */ + 0 /* cq_message_addr */); + vnic_cq_init(&enic->admin_cq[1], + VNIC_CQ_FC_DISABLE, + VNIC_CQ_COLOR_ENABLE, + 0, 0, 1, /* cq_head, cq_tail, cq_tail_color */ + VNIC_CQ_INTR_DISABLE, + VNIC_CQ_ENTRY_ENABLE, + VNIC_CQ_MSG_DISABLE, + 0, /* interrupt_offset */ + 0 /* cq_message_addr */); +} + +int enic_admin_channel_open(struct enic *enic) +{ + int err; + + if (!enic->has_admin_channel) + return -ENODEV; + + err =3D enic_admin_alloc_resources(enic); + if (err) { + netdev_err(enic->netdev, + "Failed to alloc admin channel resources: %d\n", + err); + return err; + } + + enic_admin_init_resources(enic); + + vnic_wq_enable(&enic->admin_wq); + vnic_rq_enable(&enic->admin_rq); + + err =3D enic_admin_qp_type_set(enic, QP_ENABLE); + if (err) { + netdev_err(enic->netdev, + "Failed to set admin QP type: %d\n", err); + goto disable_queues; + } + + enic->admin_chan_up =3D true; + + return 0; + +disable_queues: + enic_admin_qp_type_set(enic, QP_DISABLE); + if (vnic_wq_disable(&enic->admin_wq)) + netdev_warn(enic->netdev, "Failed to disable admin WQ\n"); + if (vnic_rq_disable(&enic->admin_rq)) + netdev_warn(enic->netdev, "Failed to disable admin RQ\n"); + enic_admin_free_resources(enic); + return err; +} + +void enic_admin_channel_close(struct enic *enic) +{ + int err; + + if (!enic->has_admin_channel) + return; + + /* Nothing to tear down if the channel was never (re)opened, e.g. a + * failed enic_admin_channel_open() in probe or in the reset path; + * otherwise the disable/clean calls below dereference freed resources. + */ + if (!enic->admin_chan_up) + return; + + enic_admin_qp_type_set(enic, QP_DISABLE); + + err =3D vnic_wq_disable(&enic->admin_wq); + if (err) + netdev_warn(enic->netdev, + "Failed to disable admin WQ: %d\n", err); + err =3D vnic_rq_disable(&enic->admin_rq); + if (err) + netdev_warn(enic->netdev, + "Failed to disable admin RQ: %d\n", err); + + vnic_wq_clean(&enic->admin_wq, enic_admin_wq_buf_clean); + vnic_rq_clean(&enic->admin_rq, enic_admin_rq_buf_clean); + vnic_cq_clean(&enic->admin_cq[0]); + vnic_cq_clean(&enic->admin_cq[1]); + enic_admin_free_resources(enic); + + enic->admin_chan_up =3D false; +} diff --git a/drivers/net/ethernet/cisco/enic/enic_admin.h b/drivers/net/eth= ernet/cisco/enic/enic_admin.h new file mode 100644 index 000000000000..569aadeb9312 --- /dev/null +++ b/drivers/net/ethernet/cisco/enic/enic_admin.h @@ -0,0 +1,15 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* Copyright 2025 Cisco Systems, Inc. All rights reserved. */ + +#ifndef _ENIC_ADMIN_H_ +#define _ENIC_ADMIN_H_ + +#define ENIC_ADMIN_DESC_COUNT 64 +#define ENIC_ADMIN_BUF_SIZE 2048 + +struct enic; + +int enic_admin_channel_open(struct enic *enic); +void enic_admin_channel_close(struct enic *enic); + +#endif /* _ENIC_ADMIN_H_ */ diff --git a/drivers/net/ethernet/cisco/enic/vnic_cq.h b/drivers/net/ethern= et/cisco/enic/vnic_cq.h index d46d4d2ef6bb..35ffa3230713 100644 --- a/drivers/net/ethernet/cisco/enic/vnic_cq.h +++ b/drivers/net/ethernet/cisco/enic/vnic_cq.h @@ -76,6 +76,15 @@ int vnic_cq_alloc(struct vnic_dev *vdev, struct vnic_cq = *cq, unsigned int index, int vnic_cq_alloc_with_type(struct vnic_dev *vdev, struct vnic_cq *cq, unsigned int index, unsigned int desc_count, unsigned int desc_size, unsigned int res_type); +#define VNIC_CQ_FC_ENABLE 1 +#define VNIC_CQ_FC_DISABLE 0 +#define VNIC_CQ_COLOR_ENABLE 1 +#define VNIC_CQ_INTR_ENABLE 1 +#define VNIC_CQ_INTR_DISABLE 0 +#define VNIC_CQ_ENTRY_ENABLE 1 +#define VNIC_CQ_MSG_ENABLE 1 +#define VNIC_CQ_MSG_DISABLE 0 + void vnic_cq_init(struct vnic_cq *cq, unsigned int flow_control_enable, unsigned int color_enable, unsigned int cq_head, unsigned int cq_tail, unsigned int cq_tail_color, unsigned int interrupt_enable, diff --git a/drivers/net/ethernet/cisco/enic/vnic_devcmd.h b/drivers/net/et= hernet/cisco/enic/vnic_devcmd.h index 3b6efa743dba..90ca06691ebd 100644 --- a/drivers/net/ethernet/cisco/enic/vnic_devcmd.h +++ b/drivers/net/ethernet/cisco/enic/vnic_devcmd.h @@ -455,8 +455,19 @@ enum vnic_devcmd_cmd { */ CMD_CQ_ENTRY_SIZE_SET =3D _CMDC(_CMD_DIR_WRITE, _CMD_VTYPE_ENET, 90), =20 + /* + * Set queue pair type (admin or data) + * in: (u32) a0 =3D queue pair type (0 =3D admin, 1 =3D data) + * in: (u32) a1 =3D enable (1) / disable (0) + */ + CMD_QP_TYPE_SET =3D _CMDC(_CMD_DIR_WRITE, _CMD_VTYPE_ENET, 97), }; =20 +#define QP_TYPE_ADMIN 0 +#define QP_TYPE_DATA 1 +#define QP_ENABLE 1 +#define QP_DISABLE 0 + /* CMD_ENABLE2 flags */ #define CMD_ENABLE2_STANDBY 0x0 #define CMD_ENABLE2_ACTIVE 0x1 --=20 2.43.0 From nobody Sat Jul 25 03:48:19 2026 Received: from rcdn-iport-7.cisco.com (rcdn-iport-7.cisco.com [173.37.86.78]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9ED95376BD6; Sun, 19 Jul 2026 08:43:16 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=173.37.86.78 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784450598; cv=none; b=lOZhJ1UQB+FZwiEquTs8IngfXz7j3BanLGMBdTtwupp6Jh/N1xEtq4qS9IMEfq+xvKJJQ5RF60K6jy/p9XBtMwZqplvGJ/cthHz2SKVNwXFaYIBuscEJglduedxvsh9Alp7NNfRP+ATAUAOLqGLb6Qj7FxejqTHv1ZSDP7UX51A= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784450598; c=relaxed/simple; bh=3gL3NIFaVF+0H+Ofn5EKlW0eoZSHlup+Ozy7hX6+HN4=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=Q08rcF4wVBgF4LnmW6EZpSao+IeKAzuzN6Ev6lYbSsuKqNNpx+cyeEOYoKMGXwCJM6d12ilvz5fQD8TZ0Qr8V8Q7DddCSVIr0ERjOMdFZ9cqlY4bzyQFDHH71F92bB86z1n8LXWkpoPLPaNAkdv156h0H4Ty7hUyJvOZAkuOOGY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=cisco.com; spf=pass smtp.mailfrom=cisco.com; dkim=pass (2048-bit key) header.d=cisco.com header.i=@cisco.com header.b=JOFdnby4; arc=none smtp.client-ip=173.37.86.78 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=cisco.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=cisco.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=cisco.com header.i=@cisco.com header.b="JOFdnby4" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cisco.com; i=@cisco.com; l=3920; q=dns/txt; s=iport01; t=1784450596; x=1785660196; h=from:date:subject:mime-version:content-transfer-encoding: message-id:references:in-reply-to:to:cc; bh=PgJZJHti/xto1BSearAPGsr9py1TUyVl/DNcgdMawyo=; b=JOFdnby4C/68onLaQEv0cYbib0U4RpRv6s9TJY0WVQuSi8G/Ojrf6EOx +E94FO7Xo/HPzWHIokV+ovH9pLPooYFDYVgVnNfY0Pm8DhMNXFB3L1N8V wsJJ+55mjrIjcmPg8iivEYwShhAcBf52UpWobIDgUpIZ7QKVVzppzM2UW KhJlCpqIL0ZC5RPjj+GSxYrpN5HyfqBQeqY8LWI6tYOOSS35QY6wq+cnY FfiNBRiEH9iVOyqk2r/At2QDItWrQWY9Hdv/fJWccULas4xuMcY+eyM8p qBQkwK9GDTUgaXUimlXC6b3WUR8XADgLR7LTauaGKrcFQBxSkXi05UDWt Q==; X-CSE-ConnectionGUID: EKjcFDRDRXmRXwj5k/6aKQ== X-CSE-MsgGUID: d5qkmQh+R2upX9t978ATlg== X-IPAS-Result: =?us-ascii?q?A0BFAgAEjVxq/5H/Ja1aHgEBCxIMggULgleBUkNJhFePU?= =?us-ascii?q?oIhnh6Bfg8BAQEPUQQBAYUFAo1TAiY0CQ4BAgQDAgMBAQEBAQEBAQEBAQsBA?= =?us-ascii?q?QUBAQECAQcFgQ4ThlyGWwIBAyMEUhAlAiYCAkYQBgESgwKCdQO9C3p/M4EBh?= =?us-ascii?q?H3ZSIFmAQsUAYEKLoVxgmsBgXCEBzuEQScbgUlEgRWDaYEZgwaDf4JqBIMcE?= =?us-ascii?q?pFsCT+BAhwDWSwBVRMNCgsHBWFSMwMgCgsSEhgVAhQvDwQWMh1wDCcSLBc0W?= =?us-ascii?q?BsHBYEdfxYEIYEChHYjHwM5f4EwdUp3LWkBEheBL4IUgToCBwMLGA1IESw3B?= =?us-ascii?q?g4ZBD0BbgeNQCOBTl8SPRw1FIEpDxQZG4EPkmMIFYNhjV9rgTWfWoQngWagD?= =?us-ascii?q?wEyl2eTBZkII6Q3hGmBaDyBWTMaCBsVgyJTGQ+OKgMW0jonMj8HAgcPApIEg?= =?us-ascii?q?W0BAQ?= IronPort-Data: A9a23:9GJnEajFdC6yeuULkd4hqDeBX161vREKZh0ujC45NGQN5FlHY01je htvUWyCaamPazH9fYt+adyy90gPu8OEy4c2SVBpri4wRC9jpJueD7x1DKtf0wB+jyHnZBg6h ynLQoCYdKjYdleF+FH1dOOn9SUgvU2xbuKUIPbePSxsThNTRi4kiBZy88Y0mYcAbeKRW2thg vus5ZeCULOZ82QsaDxMtPvS8EoHUMna4Vv0gHRvPZing3eG/5UlJMp3Db28KXL+Xr5VEoaSL 87fzKu093/u5BwkDNWoiN7TKiXmlZaLYGBiIlIPM0STqkAqSh4ai87XB9JAAatjsAhlqvgqo Dl7WTNcfi9yVkHEsLx1vxC1iEiSN4UekFPMCSDXXcB+UyQqflO0q8iCAn3aMqU12eR8L0102 cYoLQo9cRLehKXoyZGCH7wEasQLdKEHPasFsX1miDWcBvE8TNWbE+PB5MRT23E7gcUm8fT2P pVCL2EwKk6dPlsWZgp/5JEWxI9EglHkeiFZs1KWpYI84nPYy0p6172F3N/9J4XTG5wJwBjBz o7A12ikGxIfDdzc82CY6kL9mr6QzQrHXI1HQdVU8dYv2jV/3Fc7BBQIWF6TrfCnh0u6XNxDb UoZ5kIGqaUz6EWsZtXwVRyxpjiDpBF0c9dcDugz+gyR4qXT5AmdCy4PSTspQN87vsYeRjEw0 FKN2dTzClRHubuZU3+CtbGZsT+/JwARMGkEIyQEJSMd6tPupoAblB/DTt9/VqWyi7XdHTD23 iDPryUkgbgXpdAE2r/9/l3dhT+o4J/TQWYd4AjLUm+7xh12aZTjZIGy71Xfq/FaI+6kokKpp nMInY2aqesJF5zIzHzLS+QWF7bv7PGAWNHBvWNS81Aa32zF0xaekUp4uVmS+G8B3h44RALU IronPort-HdrOrdr: A9a23:BNC+EKzKYZb+JYttAcIiKrPw2r1zdoMgy1knxilNoNJuA6ilfq eV/cjztCWbtN9/YhwdcLy7UpVoBEmsl6KdgrN/AV7dZmnbUQKTRekI0WKh+UyCJ8SUzIJgPM lbAs1DIey1IFRkgcP9+QWiV/Qt0LC8gdmVrNab4ntzQQRtcq16qz14BAGSD1FsSGB9dPwE/e Knl616jgvlX2gLZcKmAXRAde3Cq9rX0K/CW3c9dmYaAM3kt0LT1FY8eCLopSsjbw== X-Talos-CUID: 9a23:GJKAdmxhCUIohHHcKGmZBgU5OcIuXFHn4E3gJnWIFmQ0S/rLQHyprfY= X-Talos-MUID: =?us-ascii?q?9a23=3ARwulTA4l5CM+fRgayyEa43/Ixoxkvr2tDXkN0q9?= =?us-ascii?q?cptG0ZDAhZGe0sDC4F9o=3D?= X-IronPort-Anti-Spam-Filtered: true X-IronPort-AV: E=Sophos;i="6.25,172,1779148800"; d="scan'208";a="511614661" Received: from rcdn-l-core-08.cisco.com ([173.37.255.145]) by rcdn-iport-7.cisco.com with ESMTP/TLS/TLS_AES_256_GCM_SHA384; 19 Jul 2026 08:42:07 +0000 Received: from [127.0.0.1] (unknown [10.25.130.254]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by rcdn-l-core-08.cisco.com (Postfix) with ESMTPS id 0793218000475; Sun, 19 Jul 2026 08:42:06 +0000 (GMT) From: Satish Kharat Date: Sun, 19 Jul 2026 01:41:48 -0700 Subject: [PATCH net-next v12 04/12] enic: add admin RQ buffer management Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260719-enic-sriov-v2-admin-channel-v2-v12-4-6ff2c617001d@cisco.com> References: <20260719-enic-sriov-v2-admin-channel-v2-v12-0-6ff2c617001d@cisco.com> In-Reply-To: <20260719-enic-sriov-v2-admin-channel-v2-v12-0-6ff2c617001d@cisco.com> To: Andrew Lunn , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni Cc: netdev@vger.kernel.org, linux-kernel@vger.kernel.org, Breno Leitao , Sesidhar Baddela , Satish Kharat X-Mailer: b4 0.15.2 X-Developer-Signature: v=1; a=ed25519-sha256; t=1784450523; l=3986; i=satishkh@cisco.com; h=from:subject:message-id; bh=3gL3NIFaVF+0H+Ofn5EKlW0eoZSHlup+Ozy7hX6+HN4=; b=IircTEov9VSVE8gbqW34xCYvAFcMeILBArHr5e5YMHjUA9UxwJMIMUxX4qPd6eO2mqMiE88Jz MR6cJ+VxXGPDQJJiF+y9xa2MGijBuEZ4BWIPIRN0wcJZvQDvfSh2nj/ X-Developer-Key: i=satishkh@cisco.com; a=ed25519; pk=4BxbLoCGvHeIMH/PbpiNVx8/2IcTjFvqcr0AD8K6WLY= X-Outbound-Client-TLS: ANONYMOUS;unknown [10.25.130.254];TLSv1.3;TLS_AES_256_GCM_SHA384;256 X-Outbound-SMTP-Client: 10.25.130.254, [10.25.130.254] X-Outbound-Node: rcdn-l-core-08.cisco.com The admin receive queue needs pre-posted DMA buffers for incoming mailbox messages from VFs. Each buffer is a kmalloc'd region mapped for DMA (2048 bytes, sufficient for any MBOX message). Add enic_admin_rq_fill(gfp) to post buffers at open time, and enic_admin_rq_drain() to unmap and free them at close time. Wire both into the admin channel open/close paths. The gfp_t parameter lets the caller pass the allocation context; both current callers -- channel open and the CQ-poll work handler that refills after draining (added in the next patch) -- run in process context and use GFP_KERNEL. Signed-off-by: Satish Kharat --- drivers/net/ethernet/cisco/enic/enic_admin.c | 66 ++++++++++++++++++++++++= +++- 1 file changed, 64 insertions(+), 2 deletions(-) diff --git a/drivers/net/ethernet/cisco/enic/enic_admin.c b/drivers/net/eth= ernet/cisco/enic/enic_admin.c index 50b46b92c88f..b2be42092106 100644 --- a/drivers/net/ethernet/cisco/enic/enic_admin.c +++ b/drivers/net/ethernet/cisco/enic/enic_admin.c @@ -3,6 +3,7 @@ =20 #include #include +#include =20 #include "vnic_dev.h" #include "vnic_wq.h" @@ -34,10 +35,63 @@ static void enic_admin_wq_buf_clean(struct vnic_wq *wq, } } =20 -/* No-op: admin RQ buffer teardown is handled in enic_admin_channel_close = */ static void enic_admin_rq_buf_clean(struct vnic_rq *rq, struct vnic_rq_buf *buf) { + struct enic *enic =3D vnic_dev_priv(rq->vdev); + + if (!buf->os_buf) + return; + + dma_unmap_single(&enic->pdev->dev, buf->dma_addr, buf->len, + DMA_FROM_DEVICE); + kfree(buf->os_buf); + buf->os_buf =3D NULL; +} + +static int enic_admin_rq_post_one(struct enic *enic, gfp_t gfp) +{ + struct vnic_rq *rq =3D &enic->admin_rq; + struct rq_enet_desc *desc; + dma_addr_t dma_addr; + void *buf; + + buf =3D kmalloc(ENIC_ADMIN_BUF_SIZE, gfp); + if (!buf) + return -ENOMEM; + + dma_addr =3D dma_map_single(&enic->pdev->dev, buf, ENIC_ADMIN_BUF_SIZE, + DMA_FROM_DEVICE); + if (dma_mapping_error(&enic->pdev->dev, dma_addr)) { + kfree(buf); + return -ENOMEM; + } + + desc =3D vnic_rq_next_desc(rq); + rq_enet_desc_enc(desc, (u64)dma_addr | VNIC_PADDR_TARGET, + RQ_ENET_TYPE_ONLY_SOP, ENIC_ADMIN_BUF_SIZE); + vnic_rq_post(rq, buf, 0, dma_addr, ENIC_ADMIN_BUF_SIZE, 0); + + return 0; +} + +static int enic_admin_rq_fill(struct enic *enic, gfp_t gfp) +{ + struct vnic_rq *rq =3D &enic->admin_rq; + int err; + + while (vnic_rq_desc_avail(rq) > 0) { + err =3D enic_admin_rq_post_one(enic, gfp); + if (err) + return err; + } + + return 0; +} + +static void enic_admin_rq_drain(struct enic *enic) +{ + vnic_rq_clean(&enic->admin_rq, enic_admin_rq_buf_clean); } =20 static int enic_admin_qp_type_set(struct enic *enic, u32 enable) @@ -171,6 +225,13 @@ int enic_admin_channel_open(struct enic *enic) vnic_wq_enable(&enic->admin_wq); vnic_rq_enable(&enic->admin_rq); =20 + err =3D enic_admin_rq_fill(enic, GFP_KERNEL); + if (err) { + netdev_err(enic->netdev, + "Failed to fill admin RQ buffers: %d\n", err); + goto disable_queues; + } + err =3D enic_admin_qp_type_set(enic, QP_ENABLE); if (err) { netdev_err(enic->netdev, @@ -188,6 +249,7 @@ int enic_admin_channel_open(struct enic *enic) netdev_warn(enic->netdev, "Failed to disable admin WQ\n"); if (vnic_rq_disable(&enic->admin_rq)) netdev_warn(enic->netdev, "Failed to disable admin RQ\n"); + enic_admin_rq_drain(enic); enic_admin_free_resources(enic); return err; } @@ -218,7 +280,7 @@ void enic_admin_channel_close(struct enic *enic) "Failed to disable admin RQ: %d\n", err); =20 vnic_wq_clean(&enic->admin_wq, enic_admin_wq_buf_clean); - vnic_rq_clean(&enic->admin_rq, enic_admin_rq_buf_clean); + enic_admin_rq_drain(enic); vnic_cq_clean(&enic->admin_cq[0]); vnic_cq_clean(&enic->admin_cq[1]); enic_admin_free_resources(enic); --=20 2.43.0 From nobody Sat Jul 25 03:48:19 2026 Received: from rcdn-iport-6.cisco.com (rcdn-iport-6.cisco.com [173.37.86.77]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5054037268D; Sun, 19 Jul 2026 08:43:35 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=173.37.86.77 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784450631; cv=none; b=Eu8yA1wwbdPxK7gQvQPF2aKs0S6xD+c8KJ5IrQtuZscFkhbyDAihHo3ayKGdRoTb8+mgs8mPqrZnK7N8lv42lCvnnjeEXAZcTQBx3zenoyN9InmErR6aZDWX2nAVvX8X7e1vDxIztDb6npmjDPQmUtOs5/5lH2y+bwfej3TYAqs= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784450631; c=relaxed/simple; bh=DpeyC/QKzuqGDLhmM459SNgSjk8jVLI5AEWQcRFITRY=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=BpAYJHCL2nn5PIK5hNztIMeclB2I9rfSomZcobkop3tQrsYwDcHrizQ3ICPK5t5/u7VYaV/mhubohV6uZuuhCyTG0XUs1u9guKsNSDdckxTNr9pAb2fnFX75oM93lR2f1lkRrqrpysPGSQFrzVsZToJCprfyT0GS9VUuHp7eo/A= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=cisco.com; spf=pass smtp.mailfrom=cisco.com; dkim=pass (2048-bit key) header.d=cisco.com header.i=@cisco.com header.b=HUset5S5; arc=none smtp.client-ip=173.37.86.77 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=cisco.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=cisco.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=cisco.com header.i=@cisco.com header.b="HUset5S5" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cisco.com; i=@cisco.com; l=16063; q=dns/txt; s=iport01; t=1784450615; x=1785660215; h=from:date:subject:mime-version:content-transfer-encoding: message-id:references:in-reply-to:to:cc; bh=u5iMhBaCmpfHDvxcs9EMzRkm97Zdus8V2phYi33ryn8=; b=HUset5S5IYCDA1mG8MGTxzOv6QfWLD9NOqqjkGUgv2eRcMCdzm1DIBBN DRiwqzyu6PBw8RB963oGpZ4LLZ20GVeBZvd9sOflLjohfXDNc5fl/WA3B 9jnILhtl+mkU1PXLAAhB8nGSpJxSDjBuCPuXXqQWWyY8fJSeuKDp99k1b mcA19WdcXa08BUW9cCs7LrGORWsXvZB32BkyhaJkWdkC3KU8Q0u92FHD9 KC0x7i687e1MMp+VrHvHAsi8U6QM/M3VUn7Yu5viWXYhkgmDSaFYPyHtw foWdN2qRN/sszT3OB1AUlpAy9F/f5I1XgLIPgLLm+qDXNgHoFqiOLcWcp w==; X-CSE-ConnectionGUID: OQcqZKFQTcGIGBgq7R7hlw== X-CSE-MsgGUID: s4Up05r0SNSWnLMP6XK28A== X-IPAS-Result: =?us-ascii?q?A0ADAACSjVxq/5H/Ja1aGgEBAQEBAQEBAQEDAQEBARIBA?= =?us-ascii?q?QEBAgIBAQEBgXwFAQEBAQsBglaBUkNJhFeIG4c3giGBFpA3jFGBfg8BAQEPU?= =?us-ascii?q?QQBAYUFAo1TAiY0CQ4BAgQDAgMBAQEBAQEBAQEBAQsBAQUBAQECAQcFgQ4Th?= =?us-ascii?q?lyGWwIBAyMEQBIQJQImAgJGEAYBEoMCgnUDvHl6fzOBAd5FgWYBCxQBgQouA?= =?us-ascii?q?YVwgmsBgXCEBzuEQScbgUlEgRQBgnN2hBqEBIJqBIIiehKDdX2Megk/gQIcA?= =?us-ascii?q?1ksAVUTDQoLBwVhUjMDIAoLEhIYFQIULw8EFjIdcAwnEiwXNFgbBwWBHX8WB?= =?us-ascii?q?CGBAoR2Ix8DOX+BMHVKdy1pARIXgS+CFIE6AgcDCxgNSBEsNwYOGQQ9AW4Hj?= =?us-ascii?q?UAjgU5xATwcGwcUCnA2DFcdFBkcKQOSYBsCg2GMKYE2a4E1gRmIJZYchCeBZ?= =?us-ascii?q?qAPATKEBJNjNJJRmQgjo2cBGDeEaYFoPIFZMxoIGxU7gmdTGQ+Id4UzAxYc0?= =?us-ascii?q?h4nMj0CBwIHDgMLkWgEgXoBAQ?= IronPort-Data: A9a23:S2et6KsVwulV/IvlkCkTUkkfmOfnVItfMUV32f8akzHdYApBsoF/q tZmKWHUPP3cZDH8eogja4u/9RxX6JbWnNZmSwpsqXpnQyoXgMeUXt7xwmUckM+xwmwvaGo9s q3yv/GZdJhcokf0/0nrav666yEgiclkf5KkYMbcICd9WAR4fykojBNnioYRj5Vh6TSDK1vlV eja/YuFZTdJ5xYuajhKs/3a+Us21BjPkGpwUmIWNKgjUGD2zxH5PLpHTYmtIn3xRJVjH+LSb 47r0LGj82rFyAwmA9Wjn6yTWhVirmn6ZFXmZtJ+AsBOszAazsAA+v9T2Mk0NS+7vw60c+VZk 72hg3AfpTABZcUgkMxFO/VR/roX0aduoNcrKlDn2SCfItGvn3bEm51T4E8K0YIw3d98GFNt2 8UkIx8sQTqMnr2Mn7K/Vbw57igjBJGD0II3oHpsy3TdSP0hW52GG/WM7t5D1zB2jcdLdRrcT 5NGMnw0M1KaPkAJYwtKYH49tL/Aan3XaDREqUmVpqof6GnIxws327/oWDbQUoHTH54IzxrJ+ Qoq+UziXiwCbIGFkQGjyU2qosr3wxPYAtI7QejQGvlCxQf7KnYoIBEfS1a+ifWwlEO7X9VRN woS9zZGhaw/9VCiSfHxUhe5qXjCuQQTM/JdH/A/4ROA1oLb5AGWAmVCRTlEAPQlqcU/bT8nz FmEm5XuHzMHmLSTRWiQ6fSSoC++NDY9KXIEY2kPTWMt+9DprYcypgjCQtZqDOi+ididMTXxx S2a6SsznbMeieYV2Kihu1PKmTShot7OVAFdzgPaQm6o8Ctna4O/IY+l817W6bBHNonxc7Wal GIPl87b6KUFCouA0XTdBu4MB7quof2CNVUwnGJSInXozBz1k1bLQGyayGsWyJtBWircRQLUX Q== IronPort-HdrOrdr: A9a23:45EBy6FFeXw7b93gpLqEHseALOsnbusQ8zAXPo5KOHhom6uj9v xG/c506faaslsssR0b8+xofZPwI080lqQb3WBuB9eftWDd0QPCEGgh1/qA/9SKIUHDH4BmpM JdWpk7JdHuAV19lMrgpCu1A78bsaC62ZHtqOvAw3JqVA1wL5th4QBwFxqBHiRNNW977OICZf 2hz/sCgSa8cnIKacn+LHwEU+/f4+DvrvvdEGY779pN0nj1sdtug4SKaCSl4g== X-Talos-CUID: =?us-ascii?q?9a23=3AL1MUQGkhW+F+X4Actz3fQVJao2HXOVrkx27iAGG?= =?us-ascii?q?BNW1GaZK/RkbPwvxkl9U7zg=3D=3D?= X-Talos-MUID: 9a23:LdlyAQTm9IdP4EvVRXT32Ax6bJpyu5+IM38LtbA+5fmnb3xZbmI= X-IronPort-Anti-Spam-Filtered: true X-IronPort-AV: E=Sophos;i="6.25,172,1779148800"; d="scan'208";a="512359497" Received: from rcdn-l-core-08.cisco.com ([173.37.255.145]) by rcdn-iport-6.cisco.com with ESMTP/TLS/TLS_AES_256_GCM_SHA384; 19 Jul 2026 08:42:08 +0000 Received: from [127.0.0.1] (unknown [10.25.130.254]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by rcdn-l-core-08.cisco.com (Postfix) with ESMTPS id C17EB180005B7; Sun, 19 Jul 2026 08:42:07 +0000 (GMT) From: Satish Kharat Date: Sun, 19 Jul 2026 01:41:49 -0700 Subject: [PATCH net-next v12 05/12] enic: add admin CQ service with MSI-X interrupt and workqueue polling Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260719-enic-sriov-v2-admin-channel-v2-v12-5-6ff2c617001d@cisco.com> References: <20260719-enic-sriov-v2-admin-channel-v2-v12-0-6ff2c617001d@cisco.com> In-Reply-To: <20260719-enic-sriov-v2-admin-channel-v2-v12-0-6ff2c617001d@cisco.com> To: Andrew Lunn , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni Cc: netdev@vger.kernel.org, linux-kernel@vger.kernel.org, Breno Leitao , Sesidhar Baddela , Satish Kharat X-Mailer: b4 0.15.2 X-Developer-Signature: v=1; a=ed25519-sha256; t=1784450523; l=16555; i=satishkh@cisco.com; h=from:subject:message-id; bh=DpeyC/QKzuqGDLhmM459SNgSjk8jVLI5AEWQcRFITRY=; b=dO+a7+uUrf3eQ6oCHq8Kkutt2PC6haKwdYNOl/L0KKbV+H2eMDu8HihhEaVVc8IEgPoApR/W2 PzB2J7O86ofBGPuDnwOSi+wX0BZOfYU+UHgmIz+HTmp0QZqhUgaanI6 X-Developer-Key: i=satishkh@cisco.com; a=ed25519; pk=4BxbLoCGvHeIMH/PbpiNVx8/2IcTjFvqcr0AD8K6WLY= X-Outbound-Client-TLS: ANONYMOUS;unknown [10.25.130.254];TLSv1.3;TLS_AES_256_GCM_SHA384;256 X-Outbound-SMTP-Client: 10.25.130.254, [10.25.130.254] X-Outbound-Node: rcdn-l-core-08.cisco.com Add completion queue (CQ) service for the admin channel work queue (WQ) and receive queue (RQ), driven by a dedicated MSI-X interrupt and a workqueue-based CQ poller. The admin WQ CQ service advances the completion ring and returns the number of descriptors consumed. The admin RQ CQ service does the same for receive completions and copies each received message into a preallocated buffer. Received messages are enqueued for deferred dispatch by a separate work_struct so the CQ poller stays short. When the MSI-X interrupt fires, the ISR schedules the CQ poll work_struct. The work handler drains all pending completions, kicks message dispatch if work was done, and returns credits to unmask the interrupt. The poll handler snapshots the pending credit count before draining the CQ so it acknowledges exactly what the hardware reported for this interrupt; any credits that accrue during draining are serviced by the next interrupt. At least one credit is returned so the write always carries a non-zero acknowledgment and re-arms the vector, since the admin channel is not re-polled like the NAPI data path. Log a rate-limited warning when admin RQ buffer refill fails so that transient memory pressure is visible without flooding the log. Signed-off-by: Satish Kharat --- drivers/net/ethernet/cisco/enic/enic.h | 8 + drivers/net/ethernet/cisco/enic/enic_admin.c | 323 +++++++++++++++++++++++= +++- drivers/net/ethernet/cisco/enic/enic_admin.h | 12 + 3 files changed, 339 insertions(+), 4 deletions(-) diff --git a/drivers/net/ethernet/cisco/enic/enic.h b/drivers/net/ethernet/= cisco/enic/enic.h index 398227448b37..401123e6df1d 100644 --- a/drivers/net/ethernet/cisco/enic/enic.h +++ b/drivers/net/ethernet/cisco/enic/enic.h @@ -301,6 +301,14 @@ struct enic { struct vnic_rq admin_rq; struct vnic_cq admin_cq[2]; struct vnic_intr admin_intr; + struct work_struct admin_poll_work; + unsigned int admin_intr_index; + struct work_struct admin_msg_work; + spinlock_t admin_msg_lock; /* protects admin_msg_list */ + struct list_head admin_msg_list; + unsigned int admin_msg_count; /* current depth of admin_msg_list */ + void (*admin_rq_handler)(struct enic *enic, void *buf, + unsigned int len); }; =20 static inline struct net_device *vnic_get_netdev(struct vnic_dev *vdev) diff --git a/drivers/net/ethernet/cisco/enic/enic_admin.c b/drivers/net/eth= ernet/cisco/enic/enic_admin.c index b2be42092106..9358165263d9 100644 --- a/drivers/net/ethernet/cisco/enic/enic_admin.c +++ b/drivers/net/ethernet/cisco/enic/enic_admin.c @@ -4,6 +4,7 @@ #include #include #include +#include =20 #include "vnic_dev.h" #include "vnic_wq.h" @@ -15,6 +16,7 @@ #include "enic.h" #include "enic_admin.h" #include "cq_desc.h" +#include "cq_enet_desc.h" #include "wq_enet_desc.h" #include "rq_enet_desc.h" =20 @@ -94,6 +96,266 @@ static void enic_admin_rq_drain(struct enic *enic) vnic_rq_clean(&enic->admin_rq, enic_admin_rq_buf_clean); } =20 +static unsigned int enic_admin_cq_color(void *cq_desc, unsigned int desc_s= ize) +{ + u8 type_color =3D *((u8 *)cq_desc + desc_size - 1); + + return (type_color >> CQ_DESC_COLOR_SHIFT) & CQ_DESC_COLOR_MASK; +} + +unsigned int enic_admin_wq_cq_service(struct enic *enic) +{ + struct vnic_cq *cq =3D &enic->admin_cq[0]; + unsigned int work =3D 0; + void *desc; + + desc =3D vnic_cq_to_clean(cq); + while (enic_admin_cq_color(desc, cq->ring.desc_size) !=3D + cq->last_color) { + vnic_cq_inc_to_clean(cq); + work++; + desc =3D vnic_cq_to_clean(cq); + } + + return work; +} + +/* Upper bound on pending admin messages. A buggy or hostile VF could flo= od + * the PF admin channel faster than admin_msg_work drains it; cap the back= log + * so a guest cannot drive the host out of memory. + */ +#define ENIC_ADMIN_MSG_MAX 256 + +static void enic_admin_msg_enqueue(struct enic *enic, void *buf, + unsigned int len) +{ + struct enic_admin_msg *msg; + + msg =3D kmalloc(struct_size(msg, data, len), GFP_KERNEL); + if (!msg) + return; + + msg->len =3D len; + memcpy(msg->data, buf, len); + + spin_lock(&enic->admin_msg_lock); + if (enic->admin_msg_count >=3D ENIC_ADMIN_MSG_MAX) { + spin_unlock(&enic->admin_msg_lock); + kfree(msg); + if (net_ratelimit()) + netdev_warn(enic->netdev, + "admin msg backlog full (%u); dropping\n", + ENIC_ADMIN_MSG_MAX); + return; + } + list_add_tail(&msg->list, &enic->admin_msg_list); + enic->admin_msg_count++; + spin_unlock(&enic->admin_msg_lock); +} + +unsigned int enic_admin_rq_cq_service(struct enic *enic) +{ + struct vnic_cq *cq =3D &enic->admin_cq[1]; + struct vnic_rq *rq =3D &enic->admin_rq; + struct cq_enet_rq_desc *rq_desc; + struct vnic_rq_buf *buf; + u16 bwf, bytes_written; + unsigned int work =3D 0; + void *desc; + + /* The admin RQ and its CQ form a single in-order channel: firmware + * posts exactly one CQE per consumed RQ descriptor, in submission + * order. Each CQE therefore pairs with rq->to_clean below without a + * completed_index cross-check, mirroring the in-order assumption of + * the main enic RX path. + */ + desc =3D vnic_cq_to_clean(cq); + while (enic_admin_cq_color(desc, cq->ring.desc_size) !=3D + cq->last_color) { + /* Ensure DMA descriptor fields are read after + * the color/valid check. dma_rmb() is the + * correct barrier for DMA-written descriptors. + */ + dma_rmb(); + buf =3D rq->to_clean; + + /* Decode the actual number of bytes hardware wrote into + * the RX buffer. buf->len is the static allocation size + * (ENIC_ADMIN_BUF_SIZE) and would expose uninitialised + * heap memory beyond the real payload. bytes_written_flags + * is at the same offset in every cq_enet_rq_desc[_32|_64] + * variant. + */ + rq_desc =3D desc; + bwf =3D le16_to_cpu(rq_desc->bytes_written_flags); + bytes_written =3D bwf & CQ_ENET_RQ_DESC_BYTES_WRITTEN_MASK; + if (bytes_written > buf->len) + goto next_desc; + + dma_sync_single_for_cpu(&enic->pdev->dev, + buf->dma_addr, buf->len, + DMA_FROM_DEVICE); + + /* Drop on hardware error indications. Admin messages + * are internal to the VIC, not received over the wire. + * Firmware sets TRUNCATED when the message does not fit + * in the posted buffer, and FCS_OK is always set on + * healthy admin completions. + */ + if (bwf & CQ_ENET_RQ_DESC_FLAGS_TRUNCATED) { + netdev_warn_once(enic->netdev, + "admin RQ: truncated message dropped\n"); + goto next_desc; + } + if (!(rq_desc->flags & CQ_ENET_RQ_DESC_FLAGS_FCS_OK)) { + netdev_warn_once(enic->netdev, + "admin RQ: bad FCS, dropping message\n"); + goto next_desc; + } + + enic_admin_msg_enqueue(enic, buf->os_buf, bytes_written); + +next_desc: + enic_admin_rq_buf_clean(rq, rq->to_clean); + rq->to_clean =3D rq->to_clean->next; + rq->ring.desc_avail++; + + vnic_cq_inc_to_clean(cq); + work++; + desc =3D vnic_cq_to_clean(cq); + } + + if (enic_admin_rq_fill(enic, GFP_KERNEL) && net_ratelimit()) + netdev_warn(enic->netdev, + "admin RQ refill failed\n"); + + return work; +} + +static irqreturn_t enic_admin_isr_msix(int irq, void *data) +{ + struct enic *enic =3D data; + + schedule_work(&enic->admin_poll_work); + + return IRQ_HANDLED; +} + +static void enic_admin_msg_work_handler(struct work_struct *work) +{ + struct enic *enic =3D container_of(work, struct enic, admin_msg_work); + struct enic_admin_msg *msg, *tmp; + LIST_HEAD(local_list); + + spin_lock_bh(&enic->admin_msg_lock); + list_splice_init(&enic->admin_msg_list, &local_list); + enic->admin_msg_count =3D 0; + spin_unlock_bh(&enic->admin_msg_lock); + + list_for_each_entry_safe(msg, tmp, &local_list, list) { + if (enic->admin_rq_handler) + enic->admin_rq_handler(enic, msg->data, msg->len); + list_del(&msg->list); + kfree(msg); + } +} + +static void enic_admin_poll_work_handler(struct work_struct *work) +{ + struct enic *enic =3D container_of(work, struct enic, admin_poll_work); + unsigned int credits; + unsigned int rq_work; + + /* Snapshot the pending credit count before draining so we acknowledge + * exactly what the hardware reported for this interrupt. Credits that + * accrue while enic_admin_rq_cq_service() runs are left for the next + * interrupt, which is harmless on this low-rate control path. + */ + credits =3D vnic_intr_credits(&enic->admin_intr); + + rq_work =3D enic_admin_rq_cq_service(enic); + + if (rq_work > 0) + schedule_work(&enic->admin_msg_work); + + /* Acknowledge the snapshotted credits and unmask the vector. Unlike + * the NAPI data path, the admin channel is not re-polled, so the vector + * must be re-armed here to receive the next completion. The unmask is + * applied through the interrupt mask register independently of the + * credit count, so returning zero credits on a spurious wakeup still + * re-arms the vector. + */ + vnic_intr_return_credits(&enic->admin_intr, + credits, + 1 /* unmask */, 0); +} + +static int enic_admin_setup_intr(struct enic *enic) +{ + unsigned int intr_index =3D enic->intr_count; + int err; + + if (vnic_dev_get_intr_mode(enic->vdev) !=3D VNIC_DEV_INTR_MODE_MSIX || + intr_index >=3D enic->intr_avail) + return -ENODEV; + + /* The admin INTR uses a slot in the same RES_TYPE_INTR_CTRL + * strided array of per-vector control blocks (mask, coalescing + * timer, credit return) that the data-path IRQs occupy in BAR0. + * vnic_intr_alloc() defaults to RES_TYPE_INTR_CTRL, which is what + * we want here. + */ + err =3D vnic_intr_alloc(enic->vdev, &enic->admin_intr, intr_index); + if (err) { + netdev_warn(enic->netdev, + "Failed to alloc admin intr at index %u: %d\n", + intr_index, err); + return err; + } + + enic->admin_intr_index =3D intr_index; + + /* A V2 VF opens the admin channel during probe, before + * register_netdev() resolves the "eth%d" name template, so using + * netdev->name here would register the literal "eth%d-admin" in + * /proc/interrupts. Use the already-stable PCI device name instead. + */ + snprintf(enic->msix[intr_index].devname, + sizeof(enic->msix[intr_index].devname), + "%s-admin", pci_name(enic->pdev)); + enic->msix[intr_index].isr =3D enic_admin_isr_msix; + enic->msix[intr_index].devid =3D enic; + + err =3D request_irq(enic->msix_entry[intr_index].vector, + enic->msix[intr_index].isr, 0, + enic->msix[intr_index].devname, + enic->msix[intr_index].devid); + if (err) { + netdev_warn(enic->netdev, + "Failed to request admin MSI-X irq: %d\n", err); + vnic_intr_free(&enic->admin_intr); + return err; + } + + enic->msix[intr_index].requested =3D 1; + + netdev_dbg(enic->netdev, + "admin channel using MSI-X interrupt (index %u)\n", + intr_index); + + return 0; +} + +static void enic_admin_teardown_intr(struct enic *enic) +{ + unsigned int intr_index =3D enic->admin_intr_index; + + free_irq(enic->msix_entry[intr_index].vector, + enic->msix[intr_index].devid); + cancel_work_sync(&enic->admin_poll_work); + enic->msix[intr_index].requested =3D 0; +} + static int enic_admin_qp_type_set(struct enic *enic, u32 enable) { u64 a0 =3D QP_TYPE_ADMIN, a1 =3D enable; @@ -173,6 +435,7 @@ static int enic_admin_alloc_resources(struct enic *enic) =20 static void enic_admin_free_resources(struct enic *enic) { + vnic_intr_free(&enic->admin_intr); vnic_cq_free(&enic->admin_cq[1]); vnic_cq_free(&enic->admin_cq[0]); vnic_rq_free(&enic->admin_rq); @@ -181,6 +444,8 @@ static void enic_admin_free_resources(struct enic *enic) =20 static void enic_admin_init_resources(struct enic *enic) { + unsigned int intr_offset =3D enic->admin_intr_index; + vnic_wq_init(&enic->admin_wq, 0, 0, 0); /* cq_index, err_intr_enable, err_intr_offset */ vnic_rq_init(&enic->admin_rq, @@ -189,20 +454,35 @@ static void enic_admin_init_resources(struct enic *en= ic) VNIC_CQ_FC_DISABLE, VNIC_CQ_COLOR_ENABLE, 0, 0, 1, /* cq_head, cq_tail, cq_tail_color */ - VNIC_CQ_INTR_DISABLE, + VNIC_CQ_INTR_DISABLE, /* polled synchronously by mbox send */ VNIC_CQ_ENTRY_ENABLE, VNIC_CQ_MSG_DISABLE, - 0, /* interrupt_offset */ + intr_offset, 0 /* cq_message_addr */); vnic_cq_init(&enic->admin_cq[1], VNIC_CQ_FC_DISABLE, VNIC_CQ_COLOR_ENABLE, 0, 0, 1, /* cq_head, cq_tail, cq_tail_color */ - VNIC_CQ_INTR_DISABLE, + VNIC_CQ_INTR_ENABLE, VNIC_CQ_ENTRY_ENABLE, VNIC_CQ_MSG_DISABLE, - 0, /* interrupt_offset */ + intr_offset, 0 /* cq_message_addr */); + vnic_intr_init(&enic->admin_intr, + 0, 0, 1); /* coalescing_timer, coalescing_type, mask_on_assertion= */ +} + +static void enic_admin_msg_drain(struct enic *enic) +{ + struct enic_admin_msg *msg, *tmp; + + spin_lock_bh(&enic->admin_msg_lock); + list_for_each_entry_safe(msg, tmp, &enic->admin_msg_list, list) { + list_del(&msg->list); + kfree(msg); + } + enic->admin_msg_count =3D 0; + spin_unlock_bh(&enic->admin_msg_lock); } =20 int enic_admin_channel_open(struct enic *enic) @@ -220,6 +500,19 @@ int enic_admin_channel_open(struct enic *enic) return err; } =20 + spin_lock_init(&enic->admin_msg_lock); + INIT_LIST_HEAD(&enic->admin_msg_list); + INIT_WORK(&enic->admin_msg_work, enic_admin_msg_work_handler); + INIT_WORK(&enic->admin_poll_work, enic_admin_poll_work_handler); + + err =3D enic_admin_setup_intr(enic); + if (err) { + netdev_err(enic->netdev, + "Admin channel requires MSI-X, SR-IOV unavailable: %d\n", + err); + goto free_resources; + } + enic_admin_init_resources(enic); =20 vnic_wq_enable(&enic->admin_wq); @@ -239,17 +532,31 @@ int enic_admin_channel_open(struct enic *enic) goto disable_queues; } =20 + vnic_intr_unmask(&enic->admin_intr); + + netdev_dbg(enic->netdev, + "admin channel open: intr=3D%u wq_avail=3D%u rq_avail=3D%u cq0_color= =3D%u cq1_color=3D%u\n", + enic->admin_intr_index, + vnic_wq_desc_avail(&enic->admin_wq), + vnic_rq_desc_avail(&enic->admin_rq), + enic->admin_cq[0].last_color, + enic->admin_cq[1].last_color); + enic->admin_chan_up =3D true; =20 return 0; =20 disable_queues: + enic_admin_teardown_intr(enic); enic_admin_qp_type_set(enic, QP_DISABLE); if (vnic_wq_disable(&enic->admin_wq)) netdev_warn(enic->netdev, "Failed to disable admin WQ\n"); if (vnic_rq_disable(&enic->admin_rq)) netdev_warn(enic->netdev, "Failed to disable admin RQ\n"); + cancel_work_sync(&enic->admin_msg_work); + enic_admin_msg_drain(enic); enic_admin_rq_drain(enic); +free_resources: enic_admin_free_resources(enic); return err; } @@ -268,6 +575,13 @@ void enic_admin_channel_close(struct enic *enic) if (!enic->admin_chan_up) return; =20 + netdev_dbg(enic->netdev, "admin channel close\n"); + + vnic_intr_mask(&enic->admin_intr); + enic_admin_teardown_intr(enic); + cancel_work_sync(&enic->admin_msg_work); + enic_admin_msg_drain(enic); + enic_admin_qp_type_set(enic, QP_DISABLE); =20 err =3D vnic_wq_disable(&enic->admin_wq); @@ -283,6 +597,7 @@ void enic_admin_channel_close(struct enic *enic) enic_admin_rq_drain(enic); vnic_cq_clean(&enic->admin_cq[0]); vnic_cq_clean(&enic->admin_cq[1]); + vnic_intr_clean(&enic->admin_intr); enic_admin_free_resources(enic); =20 enic->admin_chan_up =3D false; diff --git a/drivers/net/ethernet/cisco/enic/enic_admin.h b/drivers/net/eth= ernet/cisco/enic/enic_admin.h index 569aadeb9312..62c80220b0ca 100644 --- a/drivers/net/ethernet/cisco/enic/enic_admin.h +++ b/drivers/net/ethernet/cisco/enic/enic_admin.h @@ -9,7 +9,19 @@ =20 struct enic; =20 +/* Wrapper for received admin messages queued for deferred processing. + * The admin CQ poll work handler enqueues these; a separate work handler + * processes them where sleeping (mutex, GFP_KERNEL) is safe. + */ +struct enic_admin_msg { + struct list_head list; + unsigned int len; + u8 data[] __aligned(8); +}; + int enic_admin_channel_open(struct enic *enic); void enic_admin_channel_close(struct enic *enic); +unsigned int enic_admin_wq_cq_service(struct enic *enic); +unsigned int enic_admin_rq_cq_service(struct enic *enic); =20 #endif /* _ENIC_ADMIN_H_ */ --=20 2.43.0 From nobody Sat Jul 25 03:48:19 2026 Received: from rcdn-iport-1.cisco.com (rcdn-iport-1.cisco.com [173.37.86.72]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2282437F8C9; Sun, 19 Jul 2026 08:45:18 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=173.37.86.72 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784450721; cv=none; b=qZ8S1Ik1J7AD4iAma9OcBW0bkBYkDFsFtJyzbFcfrtiFew8qSyvE2+4dQ9Eisl5cIWgIelP5ADpSY3+hHezZeFgKeZkJ7WN683hqIWwMqHBqtwrNXcVtonGRPz24BUVyXGff4xpC75OU/WU3ja57wpxUXpxxU4egIGJzZlrrtBU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784450721; c=relaxed/simple; bh=DJOEZz0LgusiJsaMcuyJ/mo5kxauoqkYFLtqBtwJznQ=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=YjL1AUQ/v13MehlsPOi1+QZvIVeROAjW4+w8lGu4Z5tH6Bnxr2hjuUJZv8Y91Wmm8Hp4pSsjKoS02q2UBUBrgko3EXkkpZReijnPFK/Rt4Tq/5ThQVxE4rmHjn1NDBjt4nKxs+x/YUwTQF6I+Wo+XuLNCVYVKvt8lasaaD0uXiI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=cisco.com; spf=pass smtp.mailfrom=cisco.com; dkim=pass (2048-bit key) header.d=cisco.com header.i=@cisco.com header.b=Y+KyIF+i; arc=none smtp.client-ip=173.37.86.72 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=cisco.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=cisco.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=cisco.com header.i=@cisco.com header.b="Y+KyIF+i" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cisco.com; i=@cisco.com; l=3222; q=dns/txt; s=iport01; t=1784450719; x=1785660319; h=from:date:subject:mime-version:content-transfer-encoding: message-id:references:in-reply-to:to:cc; bh=Ismjn8aCzPjNAuW3z5gQmVHzab5Wc6BkN3bhmJ9vQgA=; b=Y+KyIF+iUFRQf9AH9mXdtLrxQ6FO5OvSN8qKnZTGUNuGBQz1/UTymVOH 8eWHGzGT6gJeMBqZYuNBG+oBkIPKSChWPg0DJYVDs1amfCy03eFRevgj0 mHMcnTqWDYdUzdUSWSuGl4qH94HjG0NRvq4kK2ZP3DrtCX92HxvWw/iV3 ZBnYkJIc0ojgjm4BYo8tDQ2ZhDWLC9k8IrClBRPf3C+1Rda5w8TtaHQrU 2mOPX709RH+6JkNBoHH8t8hH9+WHOt3pEjacgFplEyf0AQT0eIuwALYYg QeVE972GeukgfxDya7OMblVuBi0n0xccvy+Oc/xchrUytTIOWtVO50sel A==; X-CSE-ConnectionGUID: 0yMchbEaQF29PTmFJY/jwA== X-CSE-MsgGUID: nxemRheZQ0mm38yC8hfhhQ== X-IPAS-Result: =?us-ascii?q?A0A6AAAEjVxq/5H/Ja1aHQEBAQEJARIBBQUBgXwIAQsBh?= =?us-ascii?q?ChDSYRXiBuHN4IhkU2MURSBag8BAQEPUQQBAYUFAo1TAiY0CQ4BAgQDAgMBA?= =?us-ascii?q?QEBAQEBAQEBAQsBAQUBAQECAQcFgQ4ThlyGWwIBAyNEEhAlAiYCAkYQBgESg?= =?us-ascii?q?wKCdQO9C3qBMoEBhH3ZSIFnCxQBgQouAYVwgmsBgXCEBzuEQScbgUlEgRQBg?= =?us-ascii?q?2mEBQEZg3+CagSDHBKRbAk/gQIcA1ksAVUTDQoLBwVhUjMDIAoLEhIYFQIUH?= =?us-ascii?q?BMPBBYyHXAMJxIsFzRYGwcFgR1/FgQhgQKEdiMfAzl/gTB1SnctaQESF4Evg?= =?us-ascii?q?hSBOgIHAwsYDUgRLDcGDhkEPQFuB41AI4FOcYEOfDVjGpdWjV9roQ+EJ4Fmo?= =?us-ascii?q?A8BMpdnkwWZCCOjG0MJUIRpgWg8gVkzGggbFYMiUxkPjl/SHicyPQEBBwIHD?= =?us-ascii?q?gMLgWiQEYFtAQE?= IronPort-Data: A9a23:kfLMDqNIrcEgojXvrR14lsFynXyQoLVcMsEvi/4bfWQNrUoj0mRSz jZNXWzTOPiKYjDze952at6x8k8Ev5bVm9dhQHM5pCpnJ55oRWUpJjg4wmPYZX76whjrFRo/h ykmQoCeaphyFTmE+kvF3oHJ9RFUzbuPSqf3FNnKMyVwQR4MYCo6gHqPocZh6mJTqYb/WV7lV e/a+ZWFZgf6gm8sawr41orawP9RlKWq0N8nlgRWicBj5Df2i3QTBZQDEqC9R1OQapVUBOOzW 9HYx7i/+G7Dlz91Yj9yuu+mGqGiaue60Tmm0hK6aYD76vRxjnBaPpIACRYpQRw/ZwNlMDxG4 I4lWZSYEW/FN0BX8QgXe0Ew/ypWZcWq9FJbSJSymZT78qHIT5fj661/KUo2N6ARw8gtUFly8 NI6GXcPcTnW0opawJrjIgVtrt4oIM+uOMYUvWttiGmJS/0nWpvEBa7N4Le03h9p2ZsIRqmYP ZdEL2MzMXwsYDUXUrsTIIk3gOazhnr/WzZZs1mS46Ew5gA/ySQsieK1aoeEJobiqcN9mFaVo 1jG8lbAWhQfb4yO4mqF/HKsv7qa9c/8cMdIfFGizdZmiUOew0QfAQMbUF+8r+X/jEOiM/pbJ k0I8y0GoaU49UWtCN7nUHWQrXKZuxcOVsZ4HOo95wWAjKHT5m6xBXUORxZCZcYguctwQiYlv neAmd/zCCMstrCJRX+D3rOJqzX0Mih9BXcLbyICTCMf7tXjqZ11hRXKJv5nEaionpj2FCv2z jSisicznfMQgNQN2qH9+krI6xqqq4TFQxAd+AraRCSm4xl/aYrjYJangWU39t5aJ4qfC13Et 38elo3GvKYFDIqGk2qGR+Bl8KyV2stp+Qb02TZHd6TNPRz0k5J/Vei8OA1DGXo= IronPort-HdrOrdr: A9a23:PEXRkaPKZp8nZ8BcTjejsMiBIKoaSvp037Dk7TEUdfUzSK2lfq GV8sjzuiWYtN98YhsdcJW7Scu9qBDnmqKdn7NwAV7KZmCP01dAbrsSjrcKqAeNJ8SRzJ856Y 5QN4VzF9H0C0F3loLZ7BnQKadY/PC3tI2lmuvayW5gVkVRa69m5xpkEQrzKDwReOA/P+tfKL OsouxaujGnZXMbKu6/BnUDQqz/guejruOfXfbDbCRXkjVnSliTmcXHLyQ= X-Talos-CUID: =?us-ascii?q?9a23=3AIGqkE2hOa1RfeZShRPR+VRSGMjJuKmz4x3faDka?= =?us-ascii?q?BJzw4cIGQZ32hv6F7up87?= X-Talos-MUID: 9a23:Duy9qQaJYxbWHOBTsS/hgQtiOfVRu6GrUxAcusUGtcmEDHkl X-IronPort-Anti-Spam-Filtered: true X-IronPort-AV: E=Sophos;i="6.25,172,1779148800"; d="scan'208";a="512025942" Received: from rcdn-l-core-08.cisco.com ([173.37.255.145]) by rcdn-iport-1.cisco.com with ESMTP/TLS/TLS_AES_256_GCM_SHA384; 19 Jul 2026 08:42:09 +0000 Received: from [127.0.0.1] (unknown [10.25.130.254]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by rcdn-l-core-08.cisco.com (Postfix) with ESMTPS id 8A9491800046B; Sun, 19 Jul 2026 08:42:08 +0000 (GMT) From: Satish Kharat Date: Sun, 19 Jul 2026 01:41:50 -0700 Subject: [PATCH net-next v12 06/12] enic: define MBOX message types and header structures Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260719-enic-sriov-v2-admin-channel-v2-v12-6-6ff2c617001d@cisco.com> References: <20260719-enic-sriov-v2-admin-channel-v2-v12-0-6ff2c617001d@cisco.com> In-Reply-To: <20260719-enic-sriov-v2-admin-channel-v2-v12-0-6ff2c617001d@cisco.com> To: Andrew Lunn , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni Cc: netdev@vger.kernel.org, linux-kernel@vger.kernel.org, Breno Leitao , Sesidhar Baddela , Satish Kharat X-Mailer: b4 0.15.2 X-Developer-Signature: v=1; a=ed25519-sha256; t=1784450523; l=3237; i=satishkh@cisco.com; h=from:subject:message-id; bh=DJOEZz0LgusiJsaMcuyJ/mo5kxauoqkYFLtqBtwJznQ=; b=v79hNYiTo4JD5cmwk6dCCmcNzSxvftSClxZ2ZhimllHPlDDQ6x3LcfS/DLOVW69omqvDh1Wlr CWPrX6PC6f6BzRnI32S8OpcPXhXRyPrcerveQtuKz9VDTBugHKzGZcy X-Developer-Key: i=satishkh@cisco.com; a=ed25519; pk=4BxbLoCGvHeIMH/PbpiNVx8/2IcTjFvqcr0AD8K6WLY= X-Outbound-Client-TLS: ANONYMOUS;unknown [10.25.130.254];TLSv1.3;TLS_AES_256_GCM_SHA384;256 X-Outbound-SMTP-Client: 10.25.130.254, [10.25.130.254] X-Outbound-Node: rcdn-l-core-08.cisco.com Define the mailbox protocol structures for PF-VF communication: message header, generic reply, and per-message-type payloads for capability negotiation, VF registration/unregistration, and link state notification/acknowledgment. Include linux/types.h and linux/bits.h for __le16/__le32/__le64 and BIT() used in the header. Message types use an even=3Drequest / odd=3Dreply convention. The header carries source and destination VNIC IDs, a monotonically increasing message number, and the total message length. Signed-off-by: Satish Kharat --- drivers/net/ethernet/cisco/enic/enic_mbox.h | 83 +++++++++++++++++++++++++= ++++ 1 file changed, 83 insertions(+) diff --git a/drivers/net/ethernet/cisco/enic/enic_mbox.h b/drivers/net/ethe= rnet/cisco/enic/enic_mbox.h new file mode 100644 index 000000000000..a52f1d25cb21 --- /dev/null +++ b/drivers/net/ethernet/cisco/enic/enic_mbox.h @@ -0,0 +1,83 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* Copyright 2025 Cisco Systems, Inc. All rights reserved. */ + +#ifndef _ENIC_MBOX_H_ +#define _ENIC_MBOX_H_ + +#include +#include + +/* + * Mailbox protocol for PF-VF communication over the admin channel. + * + * Even numbers are requests, odd numbers are replies/acks. + * The prefix indicates the initiator: VF_ =3D VF-initiated, PF_ =3D PF-in= itiated. + */ +enum enic_mbox_msg_type { + ENIC_MBOX_VF_CAPABILITY_REQUEST =3D 0, + ENIC_MBOX_VF_CAPABILITY_REPLY =3D 1, + ENIC_MBOX_VF_REGISTER_REQUEST =3D 2, + ENIC_MBOX_VF_REGISTER_REPLY =3D 3, + ENIC_MBOX_VF_UNREGISTER_REQUEST =3D 4, + ENIC_MBOX_VF_UNREGISTER_REPLY =3D 5, + ENIC_MBOX_PF_LINK_STATE_NOTIF =3D 6, + ENIC_MBOX_PF_LINK_STATE_ACK =3D 7, + ENIC_MBOX_MAX +}; + +struct enic_mbox_hdr { + __le16 src_vnic_id; + __le16 dst_vnic_id; + u8 msg_type; + u8 flags; + __le16 msg_len; + __le64 msg_num; +}; + +struct enic_mbox_generic_reply { + __le16 ret_major; + __le16 ret_minor; +}; + +#define ENIC_MBOX_ERR_GENERIC BIT(0) +#define ENIC_MBOX_ERR_VF_NOT_REGISTERED BIT(1) +#define ENIC_MBOX_ERR_MSG_NOT_SUPPORTED BIT(2) + +/* ENIC_MBOX_VF_CAPABILITY_REQUEST / _REPLY */ +#define ENIC_MBOX_CAP_VERSION_0 0 +#define ENIC_MBOX_CAP_VERSION_1 1 + +struct enic_mbox_vf_capability_msg { + __le32 version; + __le32 reserved[32]; +}; + +/* The embedded enic_mbox_generic_reply has 2-byte alignment, but the + * __le32 members give this struct 4-byte natural alignment. Receive + * buffers come from kmalloc (>=3D 8-byte aligned), so there is no + * misaligned access risk when casting from the receive buffer. + */ +struct enic_mbox_vf_capability_reply_msg { + struct enic_mbox_generic_reply reply; + __le32 version; + __le32 reserved[32]; +}; + +/* ENIC_MBOX_VF_REGISTER / _UNREGISTER */ +struct enic_mbox_vf_register_reply_msg { + struct enic_mbox_generic_reply reply; +}; + +/* ENIC_MBOX_PF_LINK_STATE_NOTIF / _ACK */ +#define ENIC_MBOX_LINK_STATE_DISABLE 0 +#define ENIC_MBOX_LINK_STATE_ENABLE 1 + +struct enic_mbox_pf_link_state_notif_msg { + __le32 link_state; +}; + +struct enic_mbox_pf_link_state_ack_msg { + struct enic_mbox_generic_reply ack; +}; + +#endif /* _ENIC_MBOX_H_ */ --=20 2.43.0 From nobody Sat Jul 25 03:48:19 2026 Received: from rcdn-iport-9.cisco.com (rcdn-iport-9.cisco.com [173.37.86.80]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A906237C907; Sun, 19 Jul 2026 08:43:18 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=173.37.86.80 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784450607; cv=none; b=b6fxpMqUI6/pSzOVOdvmxmNTUoND4lokliGPw5xQURuBRJ2tR2FfsCUCcsGjBfAOeKcTeRPHExxZvYQdg8xOXZgvKsxPnSq7hIckQ2X8KdhPh2oeYo6W5Iijyi+hDHTFsuL8OXdUnchW5LEb6t0RRLdZKJJcb4qnvghIuUzFX4U= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784450607; c=relaxed/simple; bh=h679FWNX5JWhmI+AdkjjNzoyMT3GuJrF5NUktANDUZI=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=OhJQXDtiwFJcadF/IGrCyomG4LaNH8OEg3RSApK5R1oJIlINdYAYmMFiRMMMY4+0qeWNQCc/paIyy/DsrwC1CQ7z378fYGtVdef+5aHs9Q5NuC6IJ7nxcmcrOHHIptcfG8cafytn6JY8ju2b53s6EH1+HbHs4bDlwfaQyHgMg0w= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=cisco.com; spf=pass smtp.mailfrom=cisco.com; dkim=pass (2048-bit key) header.d=cisco.com header.i=@cisco.com header.b=VCDJmMov; arc=none smtp.client-ip=173.37.86.80 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=cisco.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=cisco.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=cisco.com header.i=@cisco.com header.b="VCDJmMov" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cisco.com; i=@cisco.com; l=11529; q=dns/txt; s=iport01; t=1784450598; x=1785660198; h=from:date:subject:mime-version:content-transfer-encoding: message-id:references:in-reply-to:to:cc; bh=klUP5oBGUK0xklo5eiWu1C2QTKAqLmCED3ntsvsqB9I=; b=VCDJmMov0e570hixNmGZ2yY8oFA0vm1NhRjsuJXtQffQ7+onQPCDqMJT XCcQKjNmjPULHA1B9tVxpXHdjVXgEAu29R1ulSHOuDodUJnnVOYH2+ALM x+S7jbHPeoHosZ4ym1H5H4H5OV1+yStSlhWgvZOveqCwzGdFb86neHE5m CitC/EtCRlQTChmiLOdQZrnrtIRhGkRV3pAfMdLOr5Prby3q6cLz2O6yO /MR3YhfrrGY3qL2YWBfKgD/L/w6WGCwudNOgM79UWYLJwMMstEOmHKu6q PavIxEoCmV46ORT3rtVpT6jAJGvqORJ+Uy0qtoteKXa4dAfCKog7pyGLq w==; X-CSE-ConnectionGUID: ZUKX7/UvR8SqFGdCSfH6SQ== X-CSE-MsgGUID: 0il6axXPReaXyOpchkdOwA== X-IPAS-Result: =?us-ascii?q?A0BHAgAEjVxq/5H/Ja1QCh4BAQsSDIIFC4JXgVJDSYRXj?= =?us-ascii?q?1KCIYEWnQgUgWoPAQEBD1EEAQGFBQKNUwImNAkOAQIEAwIDAQEBAQEBAQEBA?= =?us-ascii?q?QELAQEFAQEBAgEHBYEOE4ZchlsCAQMjBAsBRhAlAiYCAkUBEAYBEoMCgnUDv?= =?us-ascii?q?Qt6fzOBAYR92UiBZgELFAGBCi6FcYJrAYFwhAc7hEEnG4FJRIEUAYJzdoQFA?= =?us-ascii?q?RAJC4N0gmoEgxwSgVggj3QJP4ECHANZLAFVEw0KCwcFYVIzAyAKCxISGBUCF?= =?us-ascii?q?BwTDwQWMh1wDCcSLBc0WBsHBYEdfxYEIYEChHYjHwM5f4EwdUp3LWkBEheBL?= =?us-ascii?q?4IUgToCBwMLGA1IESw3Bg4ZBD0BbgeNQCOBTmoHATwdIQkKARNoMw4jMAQxG?= =?us-ascii?q?SwXApZhjkqhD4QngWaMW5M0ATKXZ5MFmQgjo0kGaIRpgWg8gVkzGggbFYMiU?= =?us-ascii?q?xkPji0WHNIeJzI9AgcCBw4DC5FqDxcHgU8BAQ?= IronPort-Data: A9a23:X776I6BA8Ti1axVW//Tiw5YqxClBgxIJ4kV8jS/XYbTApD1x12BWx mMbCGrQa/zcZWP0et0gOoqwo0oPu5WBz99jOVdlrnsFo1CmBibm6XV1Cm+qYkt+++WaFBoPA /02M4eGdIZvCCeA+n9BC5C5xVFkz6aEW7HgP+DNPyF1VGdMRTwo4f5Zs7ZRbrVA357jXmthh fuo+5eBYAD8imYtWo4pw/vrRC1H7ayaVAww5jTSVdgT1HfCmn8cCo4oJK3ZBxPQXolOE+emc P3Ixbe/83mx109F5gSNy+uTnuUiG9Y+DCDW4pZkc/HKbitq+kTe5p0G2M80Mi+7vdkmc+dZk 72hvbToIesg0zaldO41C3G0GAkmVUFKFSOuzXWX6aSuI0P6n3TE8chNPB5xAJMiqt1uL05C3 scWCG42V0XW7w626OrTpuhEnM8vKozveYgYoHwllWGfBvc9SpeFSKLPjTNa9G5v3YYVQrCEO pdfMGYyBPjDS0Un1lM/E5sinfyohXnXeDxDo1XTrq0yi4TW5FEoieSwboCJI7RmQ+0Nn0LGr 3L2x170CyELE8SUyTmJ1W2F07qncSTTHdh6+KeD3vJjnlCW7mAaFhATUVy1vb+/h1LWc9ZSK lAd/QIioKIz/UrtScPyNzW1rGOBtwwRR/JfFOo17AzLwa3Riy6aG2INZj1MctorsIkxXzNC/ laPk8nkGn9pvaGZRGy18qqSq3W5OUA9NWYIaCkFZREI79nqvMc4iRenZtJiFrOly974Azf9x xiUoyUkwbYel8gG0+O851+vqzShupTEUCYr6QjNGGGo9AV0YMiifYPA1LTAxexLIIDcShyKu 2IJ3pDOqusPFpqK0ieKRY3hAY2U2hpMCxWE6XYHInXr323FF6KLFWyI3AxDGQ== IronPort-HdrOrdr: A9a23:KHt8LqmuHxTrbY3hvDpLuqz/alvpDfLa3DAbv31ZSRFFG/FwWf re+MjzsiWUtN9xYh8dcL+7VZVoLUm3yXcx2/hyAV7AZmbbURWTTL2KhLGKq1bd8m/Fh41gPM xbH5SWZueAaGRSvILW7Ba1FtA4zMLC0qy57N2utktFfEVGZ75t6gFhBh3eKE15QwFaGYE0fa DsgPav3wDQHkg/X4CcGmQPWfTFqpngkZLrZgNDOjscgTP+9Q9BLNXBYmKlNtB0aUI1/Ysf X-Talos-CUID: 9a23:E4/FPW2//NJMjX9V1//vXLxfKP0GQiKa8Wnqc3ShJ0llY4abaQGrwfYx X-Talos-MUID: 9a23:+ulwMwXbC6YfV1Xq/GftgG9rJPdQ2amnOFEqiZ4GuMWjKgUlbg== X-IronPort-Anti-Spam-Filtered: true X-IronPort-AV: E=Sophos;i="6.25,172,1779148800"; d="scan'208";a="511456241" Received: from rcdn-l-core-08.cisco.com ([173.37.255.145]) by rcdn-iport-9.cisco.com with ESMTP/TLS/TLS_AES_256_GCM_SHA384; 19 Jul 2026 08:42:10 +0000 Received: from [127.0.0.1] (unknown [10.25.130.254]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by rcdn-l-core-08.cisco.com (Postfix) with ESMTPS id 52B9418000595; Sun, 19 Jul 2026 08:42:09 +0000 (GMT) From: Satish Kharat Date: Sun, 19 Jul 2026 01:41:51 -0700 Subject: [PATCH net-next v12 07/12] enic: add MBOX core send and receive for admin channel Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260719-enic-sriov-v2-admin-channel-v2-v12-7-6ff2c617001d@cisco.com> References: <20260719-enic-sriov-v2-admin-channel-v2-v12-0-6ff2c617001d@cisco.com> In-Reply-To: <20260719-enic-sriov-v2-admin-channel-v2-v12-0-6ff2c617001d@cisco.com> To: Andrew Lunn , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni Cc: netdev@vger.kernel.org, linux-kernel@vger.kernel.org, Breno Leitao , Sesidhar Baddela , Satish Kharat X-Mailer: b4 0.15.2 X-Developer-Signature: v=1; a=ed25519-sha256; t=1784450523; l=11981; i=satishkh@cisco.com; h=from:subject:message-id; bh=h679FWNX5JWhmI+AdkjjNzoyMT3GuJrF5NUktANDUZI=; b=kuB/TA7/OHwRyHGb84bsHphmRQN0LQbI/XyHzqgrrZX1yFbN3YcH2sNWOCwFV2pD4s7aMMBUx FKNtd4DW+KkD8eXx1e6fyc2H8ldVcYXH29lx/mXMqytA2Alj/rKhy4i X-Developer-Key: i=satishkh@cisco.com; a=ed25519; pk=4BxbLoCGvHeIMH/PbpiNVx8/2IcTjFvqcr0AD8K6WLY= X-Outbound-Client-TLS: ANONYMOUS;unknown [10.25.130.254];TLSv1.3;TLS_AES_256_GCM_SHA384;256 X-Outbound-SMTP-Client: 10.25.130.254, [10.25.130.254] X-Outbound-Node: rcdn-l-core-08.cisco.com Implement the mailbox protocol engine used for PF-VF communication over the admin channel. The send path (enic_mbox_send_msg) builds a message with a common header, DMA-maps it, posts a single WQ descriptor with the destination vnic ID encoded in the VLAN tag field, and polls the WQ CQ for completion. MBOX sends are gated by enic->mbox_send_disabled: enic_mbox_send_msg() returns early while it is set. It is set at the very start of both enic_admin_channel_open() and enic_admin_channel_close(), and is cleared in enic_admin_channel_open() only once the admin WQ/RQ/CQ and interrupt are fully allocated, programmed and enabled. Keeping it set for the whole open sequence means an early failure that returns before the channel is ready (as well as a not-yet-ready or torn-down channel) leaves sends disabled, so a concurrent sender can never race an MBOX send against a half-open or freed admin_wq. The receive path (enic_mbox_recv_handler) is installed as the admin RQ callback and validates incoming message headers. PF/VF-specific dispatch will be added in subsequent commits. Signed-off-by: Satish Kharat --- drivers/net/ethernet/cisco/enic/Makefile | 2 +- drivers/net/ethernet/cisco/enic/enic.h | 6 + drivers/net/ethernet/cisco/enic/enic_admin.c | 42 ++++++- drivers/net/ethernet/cisco/enic/enic_mbox.c | 170 +++++++++++++++++++++++= ++++ drivers/net/ethernet/cisco/enic/enic_mbox.h | 8 ++ 5 files changed, 225 insertions(+), 3 deletions(-) diff --git a/drivers/net/ethernet/cisco/enic/Makefile b/drivers/net/etherne= t/cisco/enic/Makefile index 7ae72fefc99a..e38aaf34c148 100644 --- a/drivers/net/ethernet/cisco/enic/Makefile +++ b/drivers/net/ethernet/cisco/enic/Makefile @@ -4,5 +4,5 @@ obj-$(CONFIG_ENIC) :=3D enic.o enic-y :=3D enic_main.o vnic_cq.o vnic_intr.o vnic_wq.o \ enic_res.o enic_dev.o enic_pp.o vnic_dev.o vnic_rq.o vnic_vic.o \ enic_ethtool.o enic_api.o enic_clsf.o enic_rq.o enic_wq.o \ - enic_admin.o + enic_admin.o enic_mbox.o =20 diff --git a/drivers/net/ethernet/cisco/enic/enic.h b/drivers/net/ethernet/= cisco/enic/enic.h index 401123e6df1d..b009d87da4bd 100644 --- a/drivers/net/ethernet/cisco/enic/enic.h +++ b/drivers/net/ethernet/cisco/enic/enic.h @@ -297,6 +297,8 @@ struct enic { * left the resources freed. */ bool admin_chan_up; + /* set on send timeout; cleared on channel re-open */ + bool mbox_send_disabled; struct vnic_wq admin_wq; struct vnic_rq admin_rq; struct vnic_cq admin_cq[2]; @@ -309,6 +311,10 @@ struct enic { unsigned int admin_msg_count; /* current depth of admin_msg_list */ void (*admin_rq_handler)(struct enic *enic, void *buf, unsigned int len); + + /* MBOX protocol state =E2=80=94 mbox_lock serializes admin WQ sends */ + struct mutex mbox_lock; + u64 mbox_msg_num; }; =20 static inline struct net_device *vnic_get_netdev(struct vnic_dev *vdev) diff --git a/drivers/net/ethernet/cisco/enic/enic_admin.c b/drivers/net/eth= ernet/cisco/enic/enic_admin.c index 9358165263d9..3e7103f2afc1 100644 --- a/drivers/net/ethernet/cisco/enic/enic_admin.c +++ b/drivers/net/ethernet/cisco/enic/enic_admin.c @@ -19,6 +19,7 @@ #include "cq_enet_desc.h" #include "wq_enet_desc.h" #include "rq_enet_desc.h" +#include "enic_mbox.h" =20 /* Clean up any admin WQ buffers still held by hardware at close time. * Normally buffers are freed inline after send completion, but a timed-out @@ -213,7 +214,26 @@ unsigned int enic_admin_rq_cq_service(struct enic *eni= c) goto next_desc; } =20 - enic_admin_msg_enqueue(enic, buf->os_buf, bytes_written); + if (enic->admin_rq_handler) { + u16 sender_vlan; + + /* Firmware sets the CQ VLAN field to identify the + * sender: 0 =3D PF, 1-based =3D VF index. Overwrite + * the untrusted src_vnic_id in the MBOX header with + * the hardware-verified value. + */ + sender_vlan =3D le16_to_cpu(rq_desc->vlan); + if (bytes_written >=3D sizeof(struct enic_mbox_hdr)) { + struct enic_mbox_hdr *hdr =3D buf->os_buf; + + hdr->src_vnic_id =3D (sender_vlan =3D=3D 0) ? + cpu_to_le16(ENIC_MBOX_DST_PF) : + cpu_to_le16(sender_vlan - 1); + } + + enic_admin_msg_enqueue(enic, buf->os_buf, + bytes_written); + } =20 next_desc: enic_admin_rq_buf_clean(rq, rq->to_clean); @@ -468,8 +488,9 @@ static void enic_admin_init_resources(struct enic *enic) VNIC_CQ_MSG_DISABLE, intr_offset, 0 /* cq_message_addr */); + /* coalescing_timer, coalescing_type, mask_on_assertion */ vnic_intr_init(&enic->admin_intr, - 0, 0, 1); /* coalescing_timer, coalescing_type, mask_on_assertion= */ + 0, 0, 1); } =20 static void enic_admin_msg_drain(struct enic *enic) @@ -492,6 +513,13 @@ int enic_admin_channel_open(struct enic *enic) if (!enic->has_admin_channel) return -ENODEV; =20 + /* Keep MBOX sends disabled for the entire open sequence. It is + * cleared only after every resource is allocated and enabled below, + * so any early error return here leaves sends disabled and a + * concurrent sender cannot touch a half-open or freed admin_wq. + */ + WRITE_ONCE(enic->mbox_send_disabled, true); + err =3D enic_admin_alloc_resources(enic); if (err) { netdev_err(enic->netdev, @@ -534,6 +562,14 @@ int enic_admin_channel_open(struct enic *enic) =20 vnic_intr_unmask(&enic->admin_intr); =20 + /* Only now that the admin WQ/RQ/CQ and interrupt are fully allocated, + * programmed and enabled is it safe to allow MBOX sends. Clearing this + * earlier opened a window where a concurrent sender (e.g. link-notify + * work scheduled by a post-reset link-up) could call enic_mbox_send_msg() + * against a not-yet-allocated admin_wq and crash. + */ + WRITE_ONCE(enic->mbox_send_disabled, false); + netdev_dbg(enic->netdev, "admin channel open: intr=3D%u wq_avail=3D%u rq_avail=3D%u cq0_color= =3D%u cq1_color=3D%u\n", enic->admin_intr_index, @@ -575,6 +611,8 @@ void enic_admin_channel_close(struct enic *enic) if (!enic->admin_chan_up) return; =20 + WRITE_ONCE(enic->mbox_send_disabled, true); + netdev_dbg(enic->netdev, "admin channel close\n"); =20 vnic_intr_mask(&enic->admin_intr); diff --git a/drivers/net/ethernet/cisco/enic/enic_mbox.c b/drivers/net/ethe= rnet/cisco/enic/enic_mbox.c new file mode 100644 index 000000000000..3709704bee02 --- /dev/null +++ b/drivers/net/ethernet/cisco/enic/enic_mbox.c @@ -0,0 +1,170 @@ +// SPDX-License-Identifier: GPL-2.0-only +// Copyright 2025 Cisco Systems, Inc. All rights reserved. + +#include +#include +#include +#include + +#include "vnic_dev.h" +#include "vnic_wq.h" +#include "vnic_cq.h" +#include "enic.h" +#include "enic_admin.h" +#include "enic_mbox.h" +#include "wq_enet_desc.h" + +#define ENIC_MBOX_POLL_TIMEOUT_US 5000000 +#define ENIC_MBOX_POLL_INTERVAL_US 100 + +static void enic_mbox_fill_hdr(struct enic *enic, struct enic_mbox_hdr *hd= r, + u8 msg_type, u16 dst_vnic_id, u16 msg_len) +{ + memset(hdr, 0, sizeof(*hdr)); + hdr->dst_vnic_id =3D cpu_to_le16(dst_vnic_id); + hdr->msg_type =3D msg_type; + hdr->msg_len =3D cpu_to_le16(msg_len); + hdr->msg_num =3D cpu_to_le64(++enic->mbox_msg_num); +} + +int enic_mbox_send_msg(struct enic *enic, u8 msg_type, u16 dst_vnic_id, + void *payload, u16 payload_len) +{ + u16 total_len =3D sizeof(struct enic_mbox_hdr) + payload_len; + struct vnic_wq *wq =3D &enic->admin_wq; + struct wq_enet_desc *desc; + unsigned long timeout; + dma_addr_t dma_addr; + u16 vlan_tag; + void *buf; + int err; + + /* Serialize MBOX sends. The admin channel is a low-frequency + * control path; holding the mutex across the poll is acceptable. + */ + mutex_lock(&enic->mbox_lock); + + if (!enic->has_admin_channel || READ_ONCE(enic->mbox_send_disabled)) { + err =3D -ENODEV; + goto unlock; + } + + if (vnic_wq_desc_avail(wq) =3D=3D 0) { + err =3D -ENOSPC; + goto unlock; + } + + buf =3D kmalloc(total_len, GFP_KERNEL); + if (!buf) { + err =3D -ENOMEM; + goto unlock; + } + + enic_mbox_fill_hdr(enic, buf, msg_type, dst_vnic_id, total_len); + if (payload_len) { + void *dst =3D buf + sizeof(struct enic_mbox_hdr); + + memcpy(dst, payload, payload_len); + } + + dma_addr =3D dma_map_single(&enic->pdev->dev, buf, total_len, + DMA_TO_DEVICE); + if (dma_mapping_error(&enic->pdev->dev, dma_addr)) { + kfree(buf); + err =3D -ENOMEM; + goto unlock; + } + + /* Firmware uses vlan field for routing: 0 =3D PF, 1-based =3D VF index */ + if (dst_vnic_id =3D=3D ENIC_MBOX_DST_PF) + vlan_tag =3D 0; + else + vlan_tag =3D dst_vnic_id + 1; + + desc =3D vnic_wq_next_desc(wq); + wq_enet_desc_enc(desc, (u64)dma_addr | VNIC_PADDR_TARGET, + total_len, + 0, 0, 0, /* mss, hdr_len, offload_mode */ + 1, 1, /* eop, cq_entry */ + 0, /* fcoe_encap */ + 1, vlan_tag, /* vlan_tag_insert, vlan_tag */ + 0); /* loopback */ + vnic_wq_post(wq, buf, dma_addr, total_len, + 1, 1, /* sop, eop */ + 1, 1, /* desc_skip_cnt, cq_entry */ + 0, 0); /* compressed_send, wrid */ + vnic_wq_doorbell(wq); + + timeout =3D jiffies + usecs_to_jiffies(ENIC_MBOX_POLL_TIMEOUT_US); + err =3D -ETIMEDOUT; + while (time_before(jiffies, timeout)) { + if (enic_admin_wq_cq_service(enic)) { + err =3D 0; + break; + } + usleep_range(ENIC_MBOX_POLL_INTERVAL_US, + ENIC_MBOX_POLL_INTERVAL_US + 50); + } + /* Final check in case completion arrived during the last sleep */ + if (err && enic_admin_wq_cq_service(enic)) + err =3D 0; + + if (!err) { + wq->to_clean =3D wq->to_clean->next; + wq->ring.desc_avail++; + dma_unmap_single(&enic->pdev->dev, dma_addr, total_len, + DMA_TO_DEVICE); + kfree(buf); + } else { + netdev_err(enic->netdev, + "MBOX send timed out (type %u dst %u), disabling channel\n", + msg_type, dst_vnic_id); + /* + * The WQ descriptor is still live in hardware. Do not unmap + * or free the buffer: the device may still DMA from dma_addr. + * Mark the channel unusable so no further sends are attempted. + */ + WRITE_ONCE(enic->mbox_send_disabled, true); + } + + netdev_dbg(enic->netdev, + "MBOX send msg_type %u dst %u vlan %u err %d\n", + msg_type, dst_vnic_id, vlan_tag, err); +unlock: + mutex_unlock(&enic->mbox_lock); + return err; +} + +static void enic_mbox_recv_handler(struct enic *enic, void *buf, + unsigned int len) +{ + struct enic_mbox_hdr *hdr =3D buf; + + if (len < sizeof(*hdr)) { + if (net_ratelimit()) + netdev_warn(enic->netdev, + "MBOX: truncated message (len %u < %zu)\n", + len, sizeof(*hdr)); + return; + } + + if (hdr->msg_type >=3D ENIC_MBOX_MAX) { + if (net_ratelimit()) + netdev_warn(enic->netdev, + "MBOX: unknown msg type %u\n", + hdr->msg_type); + return; + } + + netdev_dbg(enic->netdev, + "MBOX recv: type %u from vnic %u len %u\n", + hdr->msg_type, le16_to_cpu(hdr->src_vnic_id), + le16_to_cpu(hdr->msg_len)); +} + +void enic_mbox_init(struct enic *enic) +{ + enic->mbox_msg_num =3D 0; + mutex_init(&enic->mbox_lock); + enic->admin_rq_handler =3D enic_mbox_recv_handler; +} diff --git a/drivers/net/ethernet/cisco/enic/enic_mbox.h b/drivers/net/ethe= rnet/cisco/enic/enic_mbox.h index a52f1d25cb21..73fd7f783ee2 100644 --- a/drivers/net/ethernet/cisco/enic/enic_mbox.h +++ b/drivers/net/ethernet/cisco/enic/enic_mbox.h @@ -80,4 +80,12 @@ struct enic_mbox_pf_link_state_ack_msg { struct enic_mbox_generic_reply ack; }; =20 +#define ENIC_MBOX_DST_PF 0xFFFF + +struct enic; + +void enic_mbox_init(struct enic *enic); +int enic_mbox_send_msg(struct enic *enic, u8 msg_type, u16 dst_vnic_id, + void *payload, u16 payload_len); + #endif /* _ENIC_MBOX_H_ */ --=20 2.43.0 From nobody Sat Jul 25 03:48:19 2026 Received: from rcdn-iport-1.cisco.com (rcdn-iport-1.cisco.com [173.37.86.72]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 35E7037F725; Sun, 19 Jul 2026 08:45:21 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=173.37.86.72 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784450723; cv=none; b=h1XBjnTu9xzQAzhAFzRYoG/V6Rc5qn3FYhm7xwexfhNDQGDS0i5w9pIIezhWlsNhnE7zq1HuUD4u7DbctjIZmVezEdk4TYHkKzaoyo99EOFupjI3n2x2Qus28eBdXYxgjdyjhg+e0sG54MYViieA14tEHCp8NqjL8o3ePfbvy+U= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784450723; c=relaxed/simple; bh=UVy3/H0RMuEKNl5Vs6NEqyAwXoJIsI4AzdpjTFTi9yQ=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=a28Si/492C2o8/iIazH+lrQevheZMS8L8zqRPCBcUFFIbVRqUV+FmxSaaZeRn70Pqdi8kitUAG0Kw5B4URH1apUG1Y+GsC0oSU4qnFJGWLIi+9V+11+InaZIeO+LB/GF/W3ocs0TXAP8OQ/UeXgMvrW2nX6dP4RR/ZBd7LWu12s= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=cisco.com; spf=pass smtp.mailfrom=cisco.com; dkim=pass (2048-bit key) header.d=cisco.com header.i=@cisco.com header.b=E4FPzQWH; arc=none smtp.client-ip=173.37.86.72 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=cisco.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=cisco.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=cisco.com header.i=@cisco.com header.b="E4FPzQWH" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cisco.com; i=@cisco.com; l=8939; q=dns/txt; s=iport01; t=1784450721; x=1785660321; h=from:date:subject:mime-version:content-transfer-encoding: message-id:references:in-reply-to:to:cc; bh=CXMacmSzG4Vh/h4v1BbunYRM6XQpMQSp/sageLt5vm4=; b=E4FPzQWHQ4oPhds7HNoYb8eYBZlucz2T1wucOugyw9POmXLkQVp8hlKt MqnGmu5jPN6OlbXYIICNVnY8IivxRZx4LQ+BrCRpHDAdXrTHPQXyj9Uxp FXme/hj1XMo9oE470vOxOJfGzSPMPRzZjpya/8SXZyTwiJqCxAwAdKlY4 MpvxK8f3rdLbmd3QQN5Y969sO6LMuJtVrn3cb5KJj71NW3zyZzALBooS0 nG9m1C91VXJLf4Vkfzmk2FP0UGhaOSVjGLJTEQ7SFZDGIw3R7TBDgdL4M V9HvKnHYarCSkGzSr22vlkG2JTZJRhsr0aAJ1gAzmDhZgnH1PdCv4bB5t g==; X-CSE-ConnectionGUID: VOvsOarnT9S5wK7MT/NeJA== X-CSE-MsgGUID: dtLV9NNPScmk+WuWB+bsrg== X-IPAS-Result: =?us-ascii?q?A0BHAgAEjVxq/5H/Ja1aglmCV4FSQ0mEV49SgiGRTYxRF?= =?us-ascii?q?IFqDwEBAQ9RBAEBhQUCjVMCJjQJDgECBAMCAwEBAQEBAQEBAQEBCwEBBQEBA?= =?us-ascii?q?QIBBwWBDhOGXIZbAgEDIwQLATQSECUCJgICRhAGARKDAoJ1A70Len8zgQGEf?= =?us-ascii?q?dlIgWYBCxQBgQouhXGCawGBcIQHO4RBJxuBSUSBFYNphAUag3+CagSDHBKCS?= =?us-ascii?q?I8kCT+BAhwDWSwBVRMNCgsHBWFSMwMgCgsSEhgVAhQcEw8EFjIdcAwQFxIsF?= =?us-ascii?q?zRYGwcFgR1/FgQhgQKEdiMfAzl/gTB1SnctaQESF4EvghSBOgIHAwsYDUgRL?= =?us-ascii?q?DcGDhkEPQFuB41AI4FOagc9DjAUQjkxAXsFMJMLHYN+jkqBNZ9ahCeBZqAPA?= =?us-ascii?q?TKXZ4xVhjCZCCOjTBJZhGmBaDyBWTMaCBsVgyJTGQ+OLRYc0h4nMj0CBwIHD?= =?us-ascii?q?gMLkXmBbQEB?= IronPort-Data: A9a23:Jx+1FKo6F/OZam/rvuEJ5av1GeVeBmLFZBIvgKrLsJaIsI4StFCzt garIBnVaa3cNmWgfoxwbY2x8hkOu5bSmtc2Hgc6qy1mFyIRp+PIVI+TRqvS04x+DSFioGZPt Zh2hgzodZhsJpPkjk7zdOCn9j8kif3gqoPUUIbsIjp2SRJvVBAvgBdin/9RqoNziLBVOSvV0 T/Ji5OZYgLNNwJcaDpOtfrc8UI35ZwehRtB1rAATaET1LPhvyF94KI3fcmZM3b+S49IKe+2L 86r5K255G7Q4yA2AdqjlLvhGmVSKlIFFVHT4pb+c/HKbilq/kTe4I5iXBYvQRs/ZwGyojxE4 I4lWapc5useFvakdOw1C3G0GszlVEFM0OevzXOX6aR/w6BaGpfh660GMa04AWEX0thQH3pp/ OYeFGE2Mw+g2MeKh5S2aMA506zPLOGzVG8ekmtrwTecCbMtRorOBv2ao9RZxzw3wMtJGJ4yZ eJANmEpN0uGOUASfA5LVfrSn8/w7pX7WyZRtFOOpak3y2PS1wd2lrPqNbI5f/TXHJ8NxBvJ+ zquE2LRDg0+Co257wK++XevxdDSogm4UpkIG+jtnhJtqBjJroAJMzUaXEW2pNG1g1CzXtZYJ VBS/CcyxYA2+U23R97VUxCyoXeC+BkGVLJ4HuEn7Qifx7b87AGeCWwJCDVGbbQOt9I8TBQp2 0WPktevAiZg2JWcT36H+6jXqz6uNSULBWsfYylCRgtty8Hqqow1jzrVQ9pjGbLzhdrwcRn0w jaXvG09iq8VgMojyaq25xbEjiiqq5yPSRQ6ji3RU36p4x1ReoGofcqr5ELd4PIGK5yWJmRtp 1AekMSYqeRLBpaXmWnVEKMGHaqi4LCONzi0bUNTIqTNPg+FoxaLFb28KhknTKu1Gq7ooQPUX XI= IronPort-HdrOrdr: A9a23:C4Apnq1kTFylEiwCVDaZmQqjBEQkLtp133Aq2lEZdPWaSL39qy nIppUmPH7P5gr5N0tMpTntAsO9qDbnhPxICOoqUotKIjOHhILAFugLhrcKgQeQeBEWndQw6U 4PSdkaNPTASXV3ksr+5hC1CJIDzMnvytHQuc7ui1pgUg1ubbht9ENCCgidGlBrXwUuP+teKH Pl3LshmxOQPVIKc8+8Gn4JG8LHp9HNidbaRCRuPW9e1OFL5gnYkoIT1HOjr2wjbw8= X-Talos-CUID: =?us-ascii?q?9a23=3AJxnIZWua6D8CK0Q1fHksZ8Yf6IsbVWPG637KJXP?= =?us-ascii?q?pMjt2a+KwdRi29YFNxp8=3D?= X-Talos-MUID: =?us-ascii?q?9a23=3Ayrn0bA57l5eIvgLSYD8LEPDPxoxk/4CQWWYrqa4?= =?us-ascii?q?NvsrHCjFPEXSXly64F9o=3D?= X-IronPort-Anti-Spam-Filtered: true X-IronPort-AV: E=Sophos;i="6.25,172,1779148800"; d="scan'208";a="512025950" Received: from rcdn-l-core-08.cisco.com ([173.37.255.145]) by rcdn-iport-1.cisco.com with ESMTP/TLS/TLS_AES_256_GCM_SHA384; 19 Jul 2026 08:42:10 +0000 Received: from [127.0.0.1] (unknown [10.25.130.254]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by rcdn-l-core-08.cisco.com (Postfix) with ESMTPS id 170D61800046B; Sun, 19 Jul 2026 08:42:10 +0000 (GMT) From: Satish Kharat Date: Sun, 19 Jul 2026 01:41:52 -0700 Subject: [PATCH net-next v12 08/12] enic: add MBOX PF handlers for VF register and capability Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260719-enic-sriov-v2-admin-channel-v2-v12-8-6ff2c617001d@cisco.com> References: <20260719-enic-sriov-v2-admin-channel-v2-v12-0-6ff2c617001d@cisco.com> In-Reply-To: <20260719-enic-sriov-v2-admin-channel-v2-v12-0-6ff2c617001d@cisco.com> To: Andrew Lunn , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni Cc: netdev@vger.kernel.org, linux-kernel@vger.kernel.org, Breno Leitao , Sesidhar Baddela , Satish Kharat X-Mailer: b4 0.15.2 X-Developer-Signature: v=1; a=ed25519-sha256; t=1784450523; l=9297; i=satishkh@cisco.com; h=from:subject:message-id; bh=UVy3/H0RMuEKNl5Vs6NEqyAwXoJIsI4AzdpjTFTi9yQ=; b=u6Vbx8Pd2m2dtavHAaRbp2YXq/WRA+eJjZykeqdQAjD3BLfIOK1p+L3xozKzR08XFFxbe4XRc RLdSas+9d+iDjZTesDu4OcJIbdSeesZg+e0KJo1S6PfMK1o8jlded5d X-Developer-Key: i=satishkh@cisco.com; a=ed25519; pk=4BxbLoCGvHeIMH/PbpiNVx8/2IcTjFvqcr0AD8K6WLY= X-Outbound-Client-TLS: ANONYMOUS;unknown [10.25.130.254];TLSv1.3;TLS_AES_256_GCM_SHA384;256 X-Outbound-SMTP-Client: 10.25.130.254, [10.25.130.254] X-Outbound-Node: rcdn-l-core-08.cisco.com Implement PF-side mailbox message processing for SR-IOV V2 admin channel communication. When the PF receives messages from VFs, the dispatch routes them to type-specific handlers: - VF_CAPABILITY_REQUEST: reply with protocol version 1 - VF_REGISTER_REQUEST: send the register reply, mark the VF registered on success, then send PF_LINK_STATE_NOTIF reflecting the PF's current carrier state - VF_UNREGISTER_REQUEST: mark VF unregistered, send reply - PF_LINK_STATE_ACK: log errors from VF acknowledgment Per-VF state (struct enic_vf_state) is tracked via enic->vf_state which will be allocated when SRIOV V2 is enabled. Remove the CONFIG_PCI_IOV guard from num_vfs in struct enic. The PF handlers reference enic->num_vfs for VF ID bounds checking in enic_mbox.c, which is compiled unconditionally. The field must be visible regardless of CONFIG_PCI_IOV to avoid build failures. Add enic_mbox_send_link_state() helper for PF-initiated link state notifications, also used later by ndo_set_vf_link_state. Signed-off-by: Satish Kharat --- drivers/net/ethernet/cisco/enic/enic.h | 7 +- drivers/net/ethernet/cisco/enic/enic_mbox.c | 190 ++++++++++++++++++++++++= +++- drivers/net/ethernet/cisco/enic/enic_mbox.h | 1 + 3 files changed, 194 insertions(+), 4 deletions(-) diff --git a/drivers/net/ethernet/cisco/enic/enic.h b/drivers/net/ethernet/= cisco/enic/enic.h index b009d87da4bd..d459318c46fc 100644 --- a/drivers/net/ethernet/cisco/enic/enic.h +++ b/drivers/net/ethernet/cisco/enic/enic.h @@ -256,9 +256,7 @@ struct enic { struct enic_rx_coal rx_coalesce_setting; u32 rx_coalesce_usecs; u32 tx_coalesce_usecs; -#ifdef CONFIG_PCI_IOV u16 num_vfs; -#endif enum enic_vf_type vf_type; unsigned int enable_count; spinlock_t enic_api_lock; @@ -315,6 +313,11 @@ struct enic { /* MBOX protocol state =E2=80=94 mbox_lock serializes admin WQ sends */ struct mutex mbox_lock; u64 mbox_msg_num; + + /* PF: per-VF MBOX state, allocated when SRIOV V2 is enabled */ + struct enic_vf_state { + bool registered; + } *vf_state; }; =20 static inline struct net_device *vnic_get_netdev(struct vnic_dev *vdev) diff --git a/drivers/net/ethernet/cisco/enic/enic_mbox.c b/drivers/net/ethe= rnet/cisco/enic/enic_mbox.c index 3709704bee02..b6f05b03ae26 100644 --- a/drivers/net/ethernet/cisco/enic/enic_mbox.c +++ b/drivers/net/ethernet/cisco/enic/enic_mbox.c @@ -135,10 +135,183 @@ int enic_mbox_send_msg(struct enic *enic, u8 msg_typ= e, u16 dst_vnic_id, return err; } =20 +int enic_mbox_send_link_state(struct enic *enic, u16 vf_id, u32 link_state) +{ + struct enic_mbox_pf_link_state_notif_msg notif =3D {}; + + if (!enic->vf_state || vf_id >=3D enic->num_vfs || + !enic->vf_state[vf_id].registered) { + netdev_dbg(enic->netdev, + "MBOX: skip link state to unregistered VF %u\n", + vf_id); + return 0; + } + + notif.link_state =3D cpu_to_le32(link_state); + return enic_mbox_send_msg(enic, ENIC_MBOX_PF_LINK_STATE_NOTIF, vf_id, + ¬if, sizeof(notif)); +} + +static int enic_mbox_pf_handle_capability(struct enic *enic, void *msg, + u16 vf_id, u64 msg_num) +{ + struct enic_mbox_vf_capability_reply_msg reply =3D {}; + + reply.reply.ret_major =3D cpu_to_le16(0); + reply.version =3D cpu_to_le32(ENIC_MBOX_CAP_VERSION_1); + + return enic_mbox_send_msg(enic, ENIC_MBOX_VF_CAPABILITY_REPLY, vf_id, + &reply, sizeof(reply)); +} + +static int enic_mbox_pf_handle_register(struct enic *enic, void *msg, + u16 vf_id, u64 msg_num) +{ + struct enic_mbox_vf_register_reply_msg reply =3D {}; + u32 link_state; + int err; + + if (!enic->vf_state || vf_id >=3D enic->num_vfs) { + if (net_ratelimit()) + netdev_warn(enic->netdev, + "MBOX: register from invalid VF %u\n", + vf_id); + return -EINVAL; + } + + /* VF re-registering (e.g. guest reboot without clean unregister): + * mark the previous registration inactive before accepting the new one. + */ + if (enic->vf_state[vf_id].registered) { + netdev_dbg(enic->netdev, + "MBOX: VF %u re-register, cleaning previous state\n", + vf_id); + enic->vf_state[vf_id].registered =3D false; + } + + reply.reply.ret_major =3D cpu_to_le16(0); + err =3D enic_mbox_send_msg(enic, ENIC_MBOX_VF_REGISTER_REPLY, vf_id, + &reply, sizeof(reply)); + if (err) + return err; + + enic->vf_state[vf_id].registered =3D true; + if (net_ratelimit()) + netdev_info(enic->netdev, "VF %u registered via MBOX\n", vf_id); + + link_state =3D netif_carrier_ok(enic->netdev) ? + ENIC_MBOX_LINK_STATE_ENABLE : + ENIC_MBOX_LINK_STATE_DISABLE; + err =3D enic_mbox_send_link_state(enic, vf_id, link_state); + if (err && net_ratelimit()) + netdev_warn(enic->netdev, + "VF %u: failed to send initial link state: %d\n", + vf_id, err); + /* Registration succeeded; initial link state notification sent + * above. Subsequent link state changes are sent from the PF + * when enic_link_check() detects carrier changes. + */ + return 0; +} + +static int enic_mbox_pf_handle_unregister(struct enic *enic, void *msg, + u16 vf_id, u64 msg_num) +{ + struct enic_mbox_vf_register_reply_msg reply =3D {}; + int err; + + if (!enic->vf_state || vf_id >=3D enic->num_vfs) { + if (net_ratelimit()) + netdev_warn(enic->netdev, + "MBOX: unregister from invalid VF %u\n", + vf_id); + return -EINVAL; + } + + /* VF is unloading; clear local state regardless of whether + * the reply is successfully delivered to avoid the PF treating + * a dead VF as still registered. + */ + enic->vf_state[vf_id].registered =3D false; + + reply.reply.ret_major =3D cpu_to_le16(0); + err =3D enic_mbox_send_msg(enic, ENIC_MBOX_VF_UNREGISTER_REPLY, vf_id, + &reply, sizeof(reply)); + + if (net_ratelimit()) + netdev_info(enic->netdev, + "VF %u unregistered via MBOX\n", vf_id); + + return err; +} + +static void enic_mbox_pf_process_msg(struct enic *enic, + struct enic_mbox_hdr *hdr, void *payload) +{ + u16 vf_id =3D le16_to_cpu(hdr->src_vnic_id); + u16 msg_len =3D le16_to_cpu(hdr->msg_len); + int err =3D 0; + + if (!enic->vf_state) { + netdev_dbg(enic->netdev, + "MBOX: PF received msg but SRIOV not active\n"); + return; + } + + if (vf_id >=3D enic->num_vfs) { + if (net_ratelimit()) + netdev_warn(enic->netdev, + "MBOX: PF received msg from invalid VF %u\n", + vf_id); + return; + } + + switch (hdr->msg_type) { + case ENIC_MBOX_VF_CAPABILITY_REQUEST: + err =3D enic_mbox_pf_handle_capability(enic, payload, vf_id, + le64_to_cpu(hdr->msg_num)); + break; + case ENIC_MBOX_VF_REGISTER_REQUEST: + err =3D enic_mbox_pf_handle_register(enic, payload, vf_id, + le64_to_cpu(hdr->msg_num)); + break; + case ENIC_MBOX_VF_UNREGISTER_REQUEST: + err =3D enic_mbox_pf_handle_unregister(enic, payload, vf_id, + le64_to_cpu(hdr->msg_num)); + break; + case ENIC_MBOX_PF_LINK_STATE_ACK: { + struct enic_mbox_pf_link_state_ack_msg *ack =3D payload; + + if (msg_len < sizeof(*hdr) + sizeof(*ack)) + break; + if (le16_to_cpu(ack->ack.ret_major) && net_ratelimit()) + netdev_warn(enic->netdev, + "MBOX: VF %u link state ACK error %u/%u\n", + vf_id, + le16_to_cpu(ack->ack.ret_major), + le16_to_cpu(ack->ack.ret_minor)); + break; + } + default: + netdev_dbg(enic->netdev, + "MBOX: PF unhandled msg type %u from VF %u\n", + hdr->msg_type, vf_id); + err =3D -EOPNOTSUPP; + break; + } + + if (err && net_ratelimit()) + netdev_warn(enic->netdev, + "MBOX: PF handler for msg type %u from VF %u failed: %d\n", + hdr->msg_type, vf_id, err); +} + static void enic_mbox_recv_handler(struct enic *enic, void *buf, unsigned int len) { struct enic_mbox_hdr *hdr =3D buf; + void *payload; + u16 msg_len; =20 if (len < sizeof(*hdr)) { if (net_ratelimit()) @@ -156,10 +329,23 @@ static void enic_mbox_recv_handler(struct enic *enic,= void *buf, return; } =20 + msg_len =3D le16_to_cpu(hdr->msg_len); + if (msg_len < sizeof(*hdr) || msg_len > len) { + if (net_ratelimit()) + netdev_warn(enic->netdev, + "MBOX: invalid msg_len %u (buf len %u)\n", + msg_len, len); + return; + } + netdev_dbg(enic->netdev, "MBOX recv: type %u from vnic %u len %u\n", - hdr->msg_type, le16_to_cpu(hdr->src_vnic_id), - le16_to_cpu(hdr->msg_len)); + hdr->msg_type, le16_to_cpu(hdr->src_vnic_id), msg_len); + + payload =3D buf + sizeof(*hdr); + + if (enic->vf_state) + enic_mbox_pf_process_msg(enic, hdr, payload); } =20 void enic_mbox_init(struct enic *enic) diff --git a/drivers/net/ethernet/cisco/enic/enic_mbox.h b/drivers/net/ethe= rnet/cisco/enic/enic_mbox.h index 73fd7f783ee2..f1de67db1273 100644 --- a/drivers/net/ethernet/cisco/enic/enic_mbox.h +++ b/drivers/net/ethernet/cisco/enic/enic_mbox.h @@ -87,5 +87,6 @@ struct enic; void enic_mbox_init(struct enic *enic); int enic_mbox_send_msg(struct enic *enic, u8 msg_type, u16 dst_vnic_id, void *payload, u16 payload_len); +int enic_mbox_send_link_state(struct enic *enic, u16 vf_id, u32 link_state= ); =20 #endif /* _ENIC_MBOX_H_ */ --=20 2.43.0 From nobody Sat Jul 25 03:48:19 2026 Received: from rcdn-iport-1.cisco.com (rcdn-iport-1.cisco.com [173.37.86.72]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6EC9137FF62; Sun, 19 Jul 2026 08:45:21 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=173.37.86.72 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784450727; cv=none; b=n9aNDFO7JnFuOa2iEkMD7b/huQcxC6MemjvfgwIm8hyGhZKodGvXpk6hwGJgwXoYPvo1Kr8XtLXpt2mZdM5hDV+CIqA4bT0XgWLhEEkg7/3o2n2Fxihbdfxqjuk9pUihG2YodeLl4t3ECcpkQACnXYAtcB2pEf1MqkT9E3uRlKc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784450727; c=relaxed/simple; bh=l+MNFaxJJCGeJOFAoZUQpeH9erfLp/sDcUqUDHy6R10=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=kZ3+pdoAvE6+/N7QpOqjDJ/56EmecsgqIFU4xL6kqEl2ERTaNZeczMk6B7Dncf7KH3/JlsJS64+vLyVcHOVDlxvWAYasXcCPIxE+bPBxXumY/Gy0SyNJhABFZ9Aw6DGZb57kz7r8tcUsT2HPqM02F+SqT4X85Ll7L+lVSFEJv58= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=cisco.com; spf=pass smtp.mailfrom=cisco.com; dkim=pass (2048-bit key) header.d=cisco.com header.i=@cisco.com header.b=Sv+ChjPf; arc=none smtp.client-ip=173.37.86.72 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=cisco.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=cisco.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=cisco.com header.i=@cisco.com header.b="Sv+ChjPf" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cisco.com; i=@cisco.com; l=11953; q=dns/txt; s=iport01; t=1784450721; x=1785660321; h=from:date:subject:mime-version:content-transfer-encoding: message-id:references:in-reply-to:to:cc; bh=t+rBiyTTK+1KBTgzqcDhjSivs5NNiPe+BvzYr1LOUY4=; b=Sv+ChjPfi9ZMdlTY4azkMfdGDDL9hPMieAWnbAFEJx+wk8z25y4MQ8Mj vIWOHzr3CVfjDyqWmmjlYohLcOjc2NsNtqQgDTqQR1lSj0MNIwJLjQoXI wrUZ40EAU5riGtahqiuq8UnYL3Tj816cQb5md0MxRoenGg63ZINY5g/Me G/tWcaveZVPgJWnW8Rj7tKE4SdvaxxlUwhOYdnN/stzcAPDQaeZfjN4cA 9WYcaDsx6fvGLkgXMHMOl4WOVuwThuqxZn7ffXGLqwn1k4YkNwFoIcktt 7oaGvqj2GLkPO35hKbG7nJU/56yAPZHZEHBmiare4AwqhJu54fYbFNU6+ A==; X-CSE-ConnectionGUID: FUqKGzTOQQy+R3ukLmXaLw== X-CSE-MsgGUID: kb0sv3U2QI+euu0lWD2z9w== X-IPAS-Result: =?us-ascii?q?A0BHAgAEjVxq/5H/Ja1aHgEBCxIMggULgleBUkNJhFePU?= =?us-ascii?q?oIhkU2MURSBag8BAQEPUQQBAYUFAo1TAiY0CQ4BAgQDAgMBAQEBAQEBAQEBA?= =?us-ascii?q?QsBAQUBAQECAQcFgQ4ThlyGWwIBAyMECwE0EhAlAiYCAkYQBgESgwKCdQO9C?= =?us-ascii?q?3p/M4EB3kWBZgELFAGBCi6FcYJrAYFwhAc7g0Z7JxuBSUSBFYJYgRGEBQUVg?= =?us-ascii?q?3+CagSCInoSgVuDdYwcCT+BAhwDWSwBVRMNCgsHBWFSMwMgCgsSEhgVAhQvD?= =?us-ascii?q?wQWMh1wDBAXEiwXNFgbBwWBHX8WBCGBAoR2Ix8DOX+BMHVKdy1pARIXgS+CF?= =?us-ascii?q?IE6AgcDCxgNSBEsNwYOGQQ9AW4HjUAjgU5qBxZ5TC8ELUwbFQUwLJJ8QoM8j?= =?us-ascii?q?kqBNZ9ahCeBZqAPATKEBJNjNJJRmQgjgjahKAlQhGmBaDyBWTMaCBsVgyJTG?= =?us-ascii?q?Q+OLRYc0h4nMj0CBwIHDgMLkWgRF4FWAQE?= IronPort-Data: A9a23:00BR/6Dn3nrkDhVW//Tiw5YqxClBgxIJ4kV8jS/XYbTApDwi1z0Oy 2QbCGjXOKvYYTejeYp/boSx8UJXvseDnNYwOVdlrnsFo1CmBibm6XV1Cm+qYkt+++WaFBoPA /02M4eGdIZvCCeA+n9BC5C5xVFkz6aEW7HgP+DNPyF1VGdMRTwo4f5Zs7ZRbrVA357jXmthh fuo+5eBYAD8imYtWo4pw/vrRC1H7ayaVAww5jTSVdgT1HfCmn8cCo4oJK3ZBxPQXolOE+emc P3Ixbe/83mx109F5gSNy+uTnuUiG9Y+DCDW4pZkc/HKbitq+kTe5p0G2M80Mi+7vdkmc+dZk 72hvbToIesg0zaldO41C3G0GAkmVUFKFSOuzXWX6aSuI0P6n3TE7ch3L24kAaohw/c0OzpT9 tgdGDkpcUXW7w626OrTpuhEnM8vKozveYgYoHwllWGfBvc9SpeFSKLPjTNa9G5v3YYVQrCEO pdfMGYyBPjDS0Un1lM/E5sinfyohXnXeDxDo1XTrq0yi4TW5FEoiuC3bYuIJ7RmQ+1WlB6Du Dn7pV3YQRwHKYSb9hbf4COz07qncSTTHdh6+KeD3vJjnlCW7mAaFhATUVy1vb+/h1LWc9ZSK lAd/QIioKIz/UrtScPyNzW1rGOBtwwRR/JfFOo17AzLwa3Riy6aG2INZj1MctorsIkxXzNC/ laPk8nkGn9pvaGZRGy18qqSq3W5OUA9NWYIaCkFZREI79nqvMc4iRenZtJiFrOly974Azf9x xiUoyUkwbYel8gG0+O851+vqzShupTEUCYr6QjNGGGo9AV0YMiifYPA1LTAxexLIIDcShyKu 2IJ3pDOqusPFpqK0ieKRY3hAY2U2hpMCxWE6XYHInXr323FF6KLFWyI3AxDGQ== IronPort-HdrOrdr: A9a23:qZm8jaxwFvnP3U9XG6AsKrPw2r1zdoMgy1knxilNoNJuA6ilfq eV/cjztCWbtN9/YhwdcLy7UpVoBEmsl6KdgrN/AV7dZmnbUQKTRekI0WKh+UyCJ8SUzIJgPM lbAs1DIey1IFRkgcP9+QWiV/Qt0LC8gdmVrNab4ntzQQRtcq16qz14BAGSD1FsSGB9dPwE/e Knl616jgvlX2gLZcKmAXRAde3Cq9rX0K/CW3c9dmYaAM3kt0LT1FY8eCLopSsjbw== X-Talos-CUID: =?us-ascii?q?9a23=3ADRtDcWjpOqdn+J0+FlprMK+iqjJuXEDE71jCH2i?= =?us-ascii?q?BMHcqcuaISnuVyKpAjJ87?= X-Talos-MUID: 9a23:YWVhaAg6Gi9dhNHqv2kEcsMpN80w+Lr1NRo0sNYLptTcMRNcYSXMk2Hi X-IronPort-Anti-Spam-Filtered: true X-IronPort-AV: E=Sophos;i="6.25,172,1779148800"; d="scan'208";a="512025955" Received: from rcdn-l-core-08.cisco.com ([173.37.255.145]) by rcdn-iport-1.cisco.com with ESMTP/TLS/TLS_AES_256_GCM_SHA384; 19 Jul 2026 08:42:12 +0000 Received: from [127.0.0.1] (unknown [10.25.130.254]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by rcdn-l-core-08.cisco.com (Postfix) with ESMTPS id D1F4518000475; Sun, 19 Jul 2026 08:42:10 +0000 (GMT) From: Satish Kharat Date: Sun, 19 Jul 2026 01:41:53 -0700 Subject: [PATCH net-next v12 09/12] enic: add MBOX VF handlers for capability, register and link state Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260719-enic-sriov-v2-admin-channel-v2-v12-9-6ff2c617001d@cisco.com> References: <20260719-enic-sriov-v2-admin-channel-v2-v12-0-6ff2c617001d@cisco.com> In-Reply-To: <20260719-enic-sriov-v2-admin-channel-v2-v12-0-6ff2c617001d@cisco.com> To: Andrew Lunn , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni Cc: netdev@vger.kernel.org, linux-kernel@vger.kernel.org, Breno Leitao , Sesidhar Baddela , Satish Kharat X-Mailer: b4 0.15.2 X-Developer-Signature: v=1; a=ed25519-sha256; t=1784450523; l=12397; i=satishkh@cisco.com; h=from:subject:message-id; bh=l+MNFaxJJCGeJOFAoZUQpeH9erfLp/sDcUqUDHy6R10=; b=TGQRHdIkkLui2rcW1o+l9LGzgNvdI/MnMWnyhiDV6mBLDQZDeEK4D+vGmHfvhkmQbork7EXiC 4uaNTamu17CCyzYjJzOIQewZQiDhCdt5SvojuWMsLqXwHNkLdI5Uwnb X-Developer-Key: i=satishkh@cisco.com; a=ed25519; pk=4BxbLoCGvHeIMH/PbpiNVx8/2IcTjFvqcr0AD8K6WLY= X-Outbound-Client-TLS: ANONYMOUS;unknown [10.25.130.254];TLSv1.3;TLS_AES_256_GCM_SHA384;256 X-Outbound-SMTP-Client: 10.25.130.254, [10.25.130.254] X-Outbound-Node: rcdn-l-core-08.cisco.com Implement VF-side mailbox message processing for SR-IOV V2 admin channel communication. VF receive handlers: - VF_CAPABILITY_REPLY: store PF protocol version, signal completion - VF_REGISTER_REPLY: mark VF as registered, signal completion - VF_UNREGISTER_REPLY: mark VF as unregistered, signal completion - PF_LINK_STATE_NOTIF: update carrier state via netif_carrier_on/off, send ACK back to PF VF initiation functions for the probe-time handshake: - enic_mbox_vf_capability_check: send capability request, wait for PF reply via completion - enic_mbox_vf_register: send register request, wait for PF confirmation via completion - enic_mbox_vf_unregister: send unregister request, wait for PF confirmation The wait helper (enic_mbox_wait_reply) uses wait_for_completion_timeout, signaled when the admin ISR and CQ-poll/dispatch workqueue pipeline delivers the reply message. Signed-off-by: Satish Kharat --- drivers/net/ethernet/cisco/enic/enic.h | 11 ++ drivers/net/ethernet/cisco/enic/enic_mbox.c | 277 ++++++++++++++++++++++++= +++- drivers/net/ethernet/cisco/enic/enic_mbox.h | 3 + 3 files changed, 290 insertions(+), 1 deletion(-) diff --git a/drivers/net/ethernet/cisco/enic/enic.h b/drivers/net/ethernet/= cisco/enic/enic.h index d459318c46fc..a9a376d2cf0e 100644 --- a/drivers/net/ethernet/cisco/enic/enic.h +++ b/drivers/net/ethernet/cisco/enic/enic.h @@ -258,6 +258,8 @@ struct enic { u32 tx_coalesce_usecs; u16 num_vfs; enum enic_vf_type vf_type; + bool vf_registered; + u32 pf_cap_version; unsigned int enable_count; spinlock_t enic_api_lock; bool enic_api_busy; @@ -313,6 +315,15 @@ struct enic { /* MBOX protocol state =E2=80=94 mbox_lock serializes admin WQ sends */ struct mutex mbox_lock; u64 mbox_msg_num; + /* MBOX request-reply state. Written by the process-context request + * helpers (capability/register/unregister) and read/cleared by the + * admin_msg_work receive handlers. No explicit lock is needed because + * only one request is in flight at a time: requesters run under RTNL or + * single-threaded probe/remove, so each request is serialized and its + * reply completes mbox_comp before the next request is issued. + */ + struct completion mbox_comp; + u8 mbox_expected_reply; =20 /* PF: per-VF MBOX state, allocated when SRIOV V2 is enabled */ struct enic_vf_state { diff --git a/drivers/net/ethernet/cisco/enic/enic_mbox.c b/drivers/net/ethe= rnet/cisco/enic/enic_mbox.c index b6f05b03ae26..701972f90627 100644 --- a/drivers/net/ethernet/cisco/enic/enic_mbox.c +++ b/drivers/net/ethernet/cisco/enic/enic_mbox.c @@ -5,6 +5,7 @@ #include #include #include +#include =20 #include "vnic_dev.h" #include "vnic_wq.h" @@ -135,6 +136,16 @@ int enic_mbox_send_msg(struct enic *enic, u8 msg_type,= u16 dst_vnic_id, return err; } =20 +static int enic_mbox_wait_reply(struct enic *enic, unsigned long timeout_m= s) +{ + unsigned long left; + + left =3D wait_for_completion_timeout(&enic->mbox_comp, + msecs_to_jiffies(timeout_ms)); + + return left ? 0 : -ETIMEDOUT; +} + int enic_mbox_send_link_state(struct enic *enic, u16 vf_id, u32 link_state) { struct enic_mbox_pf_link_state_notif_msg notif =3D {}; @@ -306,6 +317,166 @@ static void enic_mbox_pf_process_msg(struct enic *eni= c, hdr->msg_type, vf_id, err); } =20 +static void enic_mbox_vf_handle_capability_reply(struct enic *enic, + void *payload) +{ + struct enic_mbox_vf_capability_reply_msg *reply =3D payload; + + if (enic->mbox_expected_reply !=3D ENIC_MBOX_VF_CAPABILITY_REPLY) { + netdev_warn(enic->netdev, + "MBOX: stale capability reply (expected %u), drop\n", + enic->mbox_expected_reply); + return; + } + + if (le16_to_cpu(reply->reply.ret_major) =3D=3D 0) + enic->pf_cap_version =3D le32_to_cpu(reply->version); + else + netdev_warn(enic->netdev, + "MBOX: PF rejected capability request: %u/%u\n", + le16_to_cpu(reply->reply.ret_major), + le16_to_cpu(reply->reply.ret_minor)); + complete(&enic->mbox_comp); +} + +static void enic_mbox_vf_handle_register_reply(struct enic *enic, + void *payload) +{ + struct enic_mbox_vf_register_reply_msg *reply =3D payload; + + if (enic->mbox_expected_reply !=3D ENIC_MBOX_VF_REGISTER_REPLY) { + netdev_warn(enic->netdev, + "MBOX: stale register reply (expected %u), drop\n", + enic->mbox_expected_reply); + return; + } + + if (le16_to_cpu(reply->reply.ret_major)) { + netdev_warn(enic->netdev, + "MBOX: VF register rejected by PF: %u/%u\n", + le16_to_cpu(reply->reply.ret_major), + le16_to_cpu(reply->reply.ret_minor)); + } else { + enic->vf_registered =3D true; + } + complete(&enic->mbox_comp); +} + +static void enic_mbox_vf_handle_unregister_reply(struct enic *enic, + void *payload) +{ + struct enic_mbox_vf_register_reply_msg *reply =3D payload; + + if (enic->mbox_expected_reply !=3D ENIC_MBOX_VF_UNREGISTER_REPLY) { + netdev_warn(enic->netdev, + "MBOX: stale unregister reply (expected %u), drop\n", + enic->mbox_expected_reply); + return; + } + + if (le16_to_cpu(reply->reply.ret_major)) { + netdev_warn(enic->netdev, + "MBOX: VF unregister rejected by PF: %u/%u\n", + le16_to_cpu(reply->reply.ret_major), + le16_to_cpu(reply->reply.ret_minor)); + } else { + enic->vf_registered =3D false; + } + complete(&enic->mbox_comp); +} + +static void enic_mbox_vf_handle_link_state(struct enic *enic, void *payloa= d) +{ + struct enic_mbox_pf_link_state_notif_msg *notif =3D payload; + struct enic_mbox_pf_link_state_ack_msg ack =3D {}; + int err; + + switch (le32_to_cpu(notif->link_state)) { + case ENIC_MBOX_LINK_STATE_ENABLE: + if (!netif_carrier_ok(enic->netdev)) + netif_carrier_on(enic->netdev); + netdev_dbg(enic->netdev, "MBOX: link state -> UP\n"); + break; + case ENIC_MBOX_LINK_STATE_DISABLE: + if (netif_carrier_ok(enic->netdev)) + netif_carrier_off(enic->netdev); + netdev_dbg(enic->netdev, "MBOX: link state -> DOWN\n"); + break; + default: + netdev_warn(enic->netdev, "MBOX: unknown link state %u\n", + le32_to_cpu(notif->link_state)); + ack.ack.ret_major =3D cpu_to_le16(ENIC_MBOX_ERR_GENERIC); + break; + } + + err =3D enic_mbox_send_msg(enic, ENIC_MBOX_PF_LINK_STATE_ACK, + ENIC_MBOX_DST_PF, &ack, sizeof(ack)); + if (err && net_ratelimit()) + netdev_warn(enic->netdev, + "MBOX: failed to send link state ACK: %d\n", err); +} + +static bool enic_mbox_vf_payload_ok(struct enic *enic, u8 msg_type, + u16 payload_len, size_t min_len) +{ + if (payload_len < min_len) { + netdev_warn(enic->netdev, + "MBOX: short payload for type %u (%u < %zu)\n", + msg_type, payload_len, min_len); + return false; + } + return true; +} + +static void enic_mbox_vf_process_msg(struct enic *enic, + struct enic_mbox_hdr *hdr, void *payload, + u16 payload_len) +{ + switch (hdr->msg_type) { + case ENIC_MBOX_VF_CAPABILITY_REPLY: { + size_t exp =3D sizeof(struct enic_mbox_vf_capability_reply_msg); + + if (!enic_mbox_vf_payload_ok(enic, hdr->msg_type, + payload_len, exp)) + return; + enic_mbox_vf_handle_capability_reply(enic, payload); + break; + } + case ENIC_MBOX_VF_REGISTER_REPLY: { + size_t exp =3D sizeof(struct enic_mbox_vf_register_reply_msg); + + if (!enic_mbox_vf_payload_ok(enic, hdr->msg_type, + payload_len, exp)) + return; + enic_mbox_vf_handle_register_reply(enic, payload); + break; + } + case ENIC_MBOX_VF_UNREGISTER_REPLY: { + size_t exp =3D sizeof(struct enic_mbox_vf_register_reply_msg); + + if (!enic_mbox_vf_payload_ok(enic, hdr->msg_type, + payload_len, exp)) + return; + enic_mbox_vf_handle_unregister_reply(enic, payload); + break; + } + case ENIC_MBOX_PF_LINK_STATE_NOTIF: { + size_t exp =3D sizeof(struct enic_mbox_pf_link_state_notif_msg); + + if (!enic_mbox_vf_payload_ok(enic, hdr->msg_type, + payload_len, exp)) + return; + enic_mbox_vf_handle_link_state(enic, payload); + break; + } + default: + netdev_dbg(enic->netdev, + "MBOX: VF unhandled msg type %u\n", + hdr->msg_type); + break; + } +} + static void enic_mbox_recv_handler(struct enic *enic, void *buf, unsigned int len) { @@ -344,13 +515,117 @@ static void enic_mbox_recv_handler(struct enic *enic= , void *buf, =20 payload =3D buf + sizeof(*hdr); =20 - if (enic->vf_state) + if (enic->vf_state) { enic_mbox_pf_process_msg(enic, hdr, payload); + } else if (le16_to_cpu(hdr->src_vnic_id) =3D=3D ENIC_MBOX_DST_PF) { + /* src_vnic_id was overwritten from the hardware-verified CQ + * VLAN sender field, so a VF only accepts messages that the + * adapter attributes to the PF. Its sole admin-channel peer is + * the PF; drop anything else as a spoofed notification. + */ + enic_mbox_vf_process_msg(enic, hdr, payload, + msg_len - (u16)sizeof(*hdr)); + } else if (net_ratelimit()) { + netdev_warn(enic->netdev, + "MBOX: VF dropping non-PF message from vnic %u\n", + le16_to_cpu(hdr->src_vnic_id)); + } +} + +int enic_mbox_vf_capability_check(struct enic *enic) +{ + struct enic_mbox_vf_capability_msg req =3D {}; + int err; + + enic->pf_cap_version =3D 0; + reinit_completion(&enic->mbox_comp); + enic->mbox_expected_reply =3D ENIC_MBOX_VF_CAPABILITY_REPLY; + req.version =3D cpu_to_le32(ENIC_MBOX_CAP_VERSION_1); + + err =3D enic_mbox_send_msg(enic, ENIC_MBOX_VF_CAPABILITY_REQUEST, + ENIC_MBOX_DST_PF, &req, sizeof(req)); + if (err) { + enic->mbox_expected_reply =3D 0; + return err; + } + + err =3D enic_mbox_wait_reply(enic, 3000); + enic->mbox_expected_reply =3D 0; + if (err) { + netdev_warn(enic->netdev, + "MBOX: no capability reply from PF\n"); + return err; + } + + if (enic->pf_cap_version < ENIC_MBOX_CAP_VERSION_1) { + netdev_warn(enic->netdev, + "MBOX: PF rejected capability request or reported unsupported versi= on %u\n", + enic->pf_cap_version); + return -EOPNOTSUPP; + } + + return 0; +} + +int enic_mbox_vf_register(struct enic *enic) +{ + int err; + + enic->vf_registered =3D false; + reinit_completion(&enic->mbox_comp); + enic->mbox_expected_reply =3D ENIC_MBOX_VF_REGISTER_REPLY; + + err =3D enic_mbox_send_msg(enic, ENIC_MBOX_VF_REGISTER_REQUEST, + ENIC_MBOX_DST_PF, NULL, 0); + if (err) { + enic->mbox_expected_reply =3D 0; + return err; + } + + err =3D enic_mbox_wait_reply(enic, 3000); + enic->mbox_expected_reply =3D 0; + if (err) { + netdev_warn(enic->netdev, + "MBOX: VF registration with PF timed out\n"); + return err; + } + + if (!enic->vf_registered) + return -ENODEV; + + return 0; +} + +int enic_mbox_vf_unregister(struct enic *enic) +{ + int err; + + if (!enic->vf_registered) + return 0; + + reinit_completion(&enic->mbox_comp); + enic->mbox_expected_reply =3D ENIC_MBOX_VF_UNREGISTER_REPLY; + + err =3D enic_mbox_send_msg(enic, ENIC_MBOX_VF_UNREGISTER_REQUEST, + ENIC_MBOX_DST_PF, NULL, 0); + if (err) { + enic->mbox_expected_reply =3D 0; + return err; + } + + err =3D enic_mbox_wait_reply(enic, 3000); + enic->mbox_expected_reply =3D 0; + if (err) + return err; + if (enic->vf_registered) + return -EACCES; + return 0; } =20 void enic_mbox_init(struct enic *enic) { enic->mbox_msg_num =3D 0; mutex_init(&enic->mbox_lock); + init_completion(&enic->mbox_comp); enic->admin_rq_handler =3D enic_mbox_recv_handler; } diff --git a/drivers/net/ethernet/cisco/enic/enic_mbox.h b/drivers/net/ethe= rnet/cisco/enic/enic_mbox.h index f1de67db1273..15e30ee2b0ed 100644 --- a/drivers/net/ethernet/cisco/enic/enic_mbox.h +++ b/drivers/net/ethernet/cisco/enic/enic_mbox.h @@ -88,5 +88,8 @@ void enic_mbox_init(struct enic *enic); int enic_mbox_send_msg(struct enic *enic, u8 msg_type, u16 dst_vnic_id, void *payload, u16 payload_len); int enic_mbox_send_link_state(struct enic *enic, u16 vf_id, u32 link_state= ); +int enic_mbox_vf_capability_check(struct enic *enic); +int enic_mbox_vf_register(struct enic *enic); +int enic_mbox_vf_unregister(struct enic *enic); =20 #endif /* _ENIC_MBOX_H_ */ --=20 2.43.0 From nobody Sat Jul 25 03:48:19 2026 Received: from rcdn-iport-3.cisco.com (rcdn-iport-3.cisco.com [173.37.86.74]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A15E737D126; Sun, 19 Jul 2026 08:43:21 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=173.37.86.74 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784450607; cv=none; b=RYlP97QYef2cwXQCOMISAfrxSFMTazb1Mx8pVClpYzOAY1EVOzk7S+rktiM1l8HRPPD11Pk4kclep7IJxlO32oB6kfcEKVnstTkn4ixoSj/KO76EUUGQzz9tQ5FkPHSv88WhIgoKIqzo3n3mCfoNKiu/3e4HHuRCOWLOmwuLQ60= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784450607; c=relaxed/simple; bh=mmkH9TpyHVf+Vx2lcbQcQcIwkSaHoFUT3bk1VDT7Sr0=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=s5KebgmYavJmGnpTzodRrUwWauOjPLB6BkO4oOP6hmRRn2H+gVpQono0GQL2MAo/jd67wL+qOcWNBro77O91+wxVL1Q4sPUluLvoTqk2xgO7av3E/DUb80xwUJZ18RMHGJZI6kUzz6nRK2Au7A/eKTs2pkN3MgzXaCMGtUHW2wY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=cisco.com; spf=pass smtp.mailfrom=cisco.com; dkim=pass (2048-bit key) header.d=cisco.com header.i=@cisco.com header.b=NbeJtFNV; arc=none smtp.client-ip=173.37.86.74 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=cisco.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=cisco.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=cisco.com header.i=@cisco.com header.b="NbeJtFNV" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cisco.com; i=@cisco.com; l=20144; q=dns/txt; s=iport01; t=1784450601; x=1785660201; h=from:date:subject:mime-version:content-transfer-encoding: message-id:references:in-reply-to:to:cc; bh=gtN3yYpKzTGvqdcBUo4p0SZ/AIYyu3MhrS0LH+zf/X4=; b=NbeJtFNVlrLqMGb6AxbZHKW+rfpklVVRo1utQ+I6zuOLrAi1k78Z9ShX FM1ylANwaxv2EdS/cfWIOqbgfCRUPXvqkCVlE/2AQlSC0v3q0+k7K6eyg TIJxgwPdDhHmQNs35TZJL65ZP1WIvqSncUMlbgkXSn9rkzQO9ZSr9jwwT /tEQnTiiQKTa7DU4nPTC2ZXelZY3Hw4QOVhNZVkdHyrGcPcuKUNQ7W9cx 4yU+CNg+v6LMR2n9hIgNawnBzPpYiBnAPHW82iKzQ2dykeJhdVlptn/zY 96Jyd+0SpZu1WDyg+Kcs1d/OeqCWwGv2SSVic2cw4qiWTOHYVxuiuOXvm g==; X-CSE-ConnectionGUID: NFG9QC9pSuaYQGen54YMiQ== X-CSE-MsgGUID: 5R78wDn8QNOk8rA/E824gw== X-IPAS-Result: =?us-ascii?q?A0AnAACSjVxq/5H/Ja1aHQEBAQEJARIBBQUBgXwIAQsBg?= =?us-ascii?q?hc/gVJDSYRXiBuHN4Ihnh4UgWoPAQEBD1EEAQGFBQKNUwImNAkOAQIEAwIDA?= =?us-ascii?q?QEBAQEBAQEBAQELAQEFAQEBAgEHBYEOE4ZchlsCAQMjBEASECUCJgICRhAGA?= =?us-ascii?q?RKDAoJ1A7x5en8zgQGEfdlIgWYBCxQBgQouAYVwgmsBgXCEBzuEQScbgUlEg?= =?us-ascii?q?RQBg2mBKIMCg3SCagSDHBKRbAk/gQIcA1ksAVUTDQoLBwVhUjMDIAoLEhIYF?= =?us-ascii?q?QIULw8EFjIdcAwnEiwXNFgbBwWBHX8WBCGBAoR2Ix8DOX+BMHVKdy1pARIXg?= =?us-ascii?q?S+CFIE6AgcDCxgNSBEsNwYOGQQ9AW4HjUAjgUUCB2oHATwcARsaBwxoBC8CD?= =?us-ascii?q?DwDBQ8fAhQZQwKSYwcWg2GNX2uBNZ9ahCeBZqAPATKEBJNjkwWZCCOCNqIBh?= =?us-ascii?q?GmBaDyBWTMaCBsVgyJTGQ+OLRYceAEC0SMnMj8HAgcPApFzBIF6AQE?= IronPort-Data: A9a23:4VibVKAqZN9dxBVW//Tiw5YqxClBgxIJ4kV8jS/XYbTApDp01TIOy DMZXj2DP/jcNDfweNh/bonkoUhU6p/UmtNqOVdlrnsFo1CmBibm6XV1Cm+qYkt+++WaFBoPA /02M4eGdIZvCCeA+n9BC5C5xVFkz6aEW7HgP+DNPyF1VGdMRTwo4f5Zs7ZRbrVA357jXmthh fuo+5eBYAD8imYtWo4pw/vrRC1H7ayaVAww5jTSVdgT1HfCmn8cCo4oJK3ZBxPQXolOE+emc P3Ixbe/83mx109F5gSNy+uTnuUiG9Y+DCDW4pZkc/HKbitq+kTe5p0G2M80Mi+7vdkmc+dZk 72hvbToIesg0zaldO41C3G0GAkmVUFKFSOuzXWX6aSuI0P6n3TE5dtBXXgdbJUj5+9GOE5D8 6FEGRUMV0XW7w626OrTpuhEnM8vKozveYgYoHwllWGfBvc9SpeFSKLPjTNa9G5v3YYVQrCEO pdfMGYyBPjDS0Un1lM/E5sinfyohXnXeDxDo1XTrq0yi4TW5FEoiue9aYuIJbRmQ+0IpBnIu U7Ar13fJRJKLvXY5Grfrn2z07qncSTTHdh6+KeD3vJjnlCW7mAaFhATUVy1vb+/h1LWc9ZSK lAd/QIioKIz/UrtScPyNzW1rGOBtwwRR/JfFOo17AzLwa3Riy6aG2INZj1MctorsIkxXzNC/ laPk8nkGn9pvaGZRGy18qqSq3W5OUA9NWYIaCkFZREI79nqvMc4iRenZtJiFrOly974Azf9x xiUoyUkwbYel8gG0+O851+vqzShupTEUCYr6QjNGGGo9AV0YMiifYPA1LTAxexLIIDcShyKu 2IJ3pDOqusPFpqK0ieKRY3hAY2U2hpMCxWE6XYHInXr327FF6KLFWyI3AxDGQ== IronPort-HdrOrdr: A9a23:2fwhWaxTkcpYAcnv15K2KrPw2r1zdoMgy1knxilNoNJuA6ilfq eV/cjztCWbtN9/YhwdcLy7UpVoBEmsl6KdgrN/AV7dZmnbUQKTRekI0WKh+UyCJ8SUzIJgPM lbAs1DIey1IFRkgcP9+QWiV/Qt0LC8gdmVrNab4ntzQQRtcq16qz14BAGSD1FsSGB9dPwE/e Knl616jgvlX2gLZcKmAXRAde3Cq9rX0K/CW3c9dmYaAM3kt0LT1FY8eCLopSsjbw== X-Talos-CUID: 9a23:zoSJ322SRZGv/CWolIfI4LxfPO4KI0T2nW/rPkaJMElkTuDJeHqRwfYx X-Talos-MUID: 9a23:nR+y4gZqaeahOuBT6gWz1C5OMpdRxYOtWFok1rhW4ZWLKnkl X-IronPort-Anti-Spam-Filtered: true X-IronPort-AV: E=Sophos;i="6.25,172,1779148800"; d="scan'208";a="512781957" Received: from rcdn-l-core-08.cisco.com ([173.37.255.145]) by rcdn-iport-3.cisco.com with ESMTP/TLS/TLS_AES_256_GCM_SHA384; 19 Jul 2026 08:42:12 +0000 Received: from [127.0.0.1] (unknown [10.25.130.254]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by rcdn-l-core-08.cisco.com (Postfix) with ESMTPS id 03E32180005B7; Sun, 19 Jul 2026 08:42:11 +0000 (GMT) From: Satish Kharat Date: Sun, 19 Jul 2026 01:41:54 -0700 Subject: [PATCH net-next v12 10/12] enic: wire V2 SR-IOV enable with admin channel and MBOX Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260719-enic-sriov-v2-admin-channel-v2-v12-10-6ff2c617001d@cisco.com> References: <20260719-enic-sriov-v2-admin-channel-v2-v12-0-6ff2c617001d@cisco.com> In-Reply-To: <20260719-enic-sriov-v2-admin-channel-v2-v12-0-6ff2c617001d@cisco.com> To: Andrew Lunn , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni Cc: netdev@vger.kernel.org, linux-kernel@vger.kernel.org, Breno Leitao , Sesidhar Baddela , Satish Kharat X-Mailer: b4 0.15.2 X-Developer-Signature: v=1; a=ed25519-sha256; t=1784450523; l=20673; i=satishkh@cisco.com; h=from:subject:message-id; bh=mmkH9TpyHVf+Vx2lcbQcQcIwkSaHoFUT3bk1VDT7Sr0=; b=+V36FJ++H5GpWnay1AGmzGkyKOg31u8r4+qsYAigHymAinp/iyNAGmu7Wk58dBYStnBLkjSsb JPD4v9c0ViFDHNXMS8m/HRVXrzlWRHUh2RlrHOS7YBVJY/s4k/8Vwic X-Developer-Key: i=satishkh@cisco.com; a=ed25519; pk=4BxbLoCGvHeIMH/PbpiNVx8/2IcTjFvqcr0AD8K6WLY= X-Outbound-Client-TLS: ANONYMOUS;unknown [10.25.130.254];TLSv1.3;TLS_AES_256_GCM_SHA384;256 X-Outbound-SMTP-Client: 10.25.130.254, [10.25.130.254] X-Outbound-Node: rcdn-l-core-08.cisco.com Extend enic_sriov_configure() to handle V2 SR-IOV VFs. When the PF detects V2 VF device IDs, the enable path allocates per-VF MBOX state, opens the admin channel, initializes the MBOX protocol, and then calls pci_enable_sriov(). The admin channel must be ready before VFs are created so that VF drivers can immediately begin the MBOX capability and registration handshake during their probe. The enic_sriov_configure() dispatcher and its V2 helpers (enic_sriov_v2_enable, enic_sriov_v2_disable) are defined here but intentionally not yet wired into struct pci_driver via .sriov_configure -- hence the __maybe_unused annotations. This series introduces only the admin channel and MBOX infrastructure; sysfs-driven V2 enable/disable will be activated in a follow-up patch by adding ".sriov_configure =3D enic_sriov_configure," to enic_driver. Because .sriov_configure is not registered yet, enic_sriov_configure() cannot run concurrently with the rtnl-protected reset paths (enic_reset(), enic_tx_hang_reset()) in this series, so there is no reachable locking race between SR-IOV enable/disable and reset. The follow-up patch that wires the callback will add the necessary serialization against those paths. Note that simply taking rtnl_lock() around the enable path is not viable, because pci_enable_sriov() triggers VF probe and register_netdev(), which themselves acquire rtnl; the wiring patch therefore uses finer-grained serialization. The disable path first clears ENIC_SRIOV_ENABLED and flushes the link-notify work, so no further VF link-state broadcast can run, then calls pci_disable_sriov() (VF drivers unregister via MBOX), closes the admin channel, and frees per-VF state. Clearing the flag and flushing the work before vf_state is freed closes a use-after-free window against the link-notify path. Notify registered VFs of PF link transitions: enic_link_check() schedules link_notify_work on each carrier up/down edge, and the work handler sends PF_LINK_STATE_NOTIF to the VFs from process context. The broadcast cannot run directly in enic_link_check() because the MBOX send path may sleep and link check runs in the notify timer/ISR context. Re-establish the admin/MBOX channel across a PF reset. enic_reset() and enic_tx_hang_reset() fully close the admin channel before the soft/hang reset (which wipes all hardware queues, including the admin WQ/RQ), then reopen it and re-run enic_mbox_init() after the data path is back up, and re-push the current link state to registered VFs. Reject VF port profile requests when V2 SR-IOV is active (enic_is_valid_pp_vf), since enic->pp is not reallocated for V2 VFs and the V2 protocol uses MBOX instead of port profiles. Update enic_remove() to run enic_dev_deinit() and vnic_dev_close() after SR-IOV teardown, so the PF device remains functional while VFs are being cleaned up. This ordering applies to both V1 and V2 SR-IOV paths. Signed-off-by: Satish Kharat --- drivers/net/ethernet/cisco/enic/enic.h | 2 + drivers/net/ethernet/cisco/enic/enic_admin.c | 3 + drivers/net/ethernet/cisco/enic/enic_main.c | 260 +++++++++++++++++++++++= ++-- drivers/net/ethernet/cisco/enic/enic_mbox.c | 13 +- drivers/net/ethernet/cisco/enic/enic_pp.c | 5 + drivers/net/ethernet/cisco/enic/enic_res.c | 1 + drivers/net/ethernet/cisco/enic/vnic_enet.h | 4 +- 7 files changed, 274 insertions(+), 14 deletions(-) diff --git a/drivers/net/ethernet/cisco/enic/enic.h b/drivers/net/ethernet/= cisco/enic/enic.h index a9a376d2cf0e..b5a43fe04877 100644 --- a/drivers/net/ethernet/cisco/enic/enic.h +++ b/drivers/net/ethernet/cisco/enic/enic.h @@ -305,6 +305,7 @@ struct enic { struct vnic_intr admin_intr; struct work_struct admin_poll_work; unsigned int admin_intr_index; + struct work_struct link_notify_work; struct work_struct admin_msg_work; spinlock_t admin_msg_lock; /* protects admin_msg_list */ struct list_head admin_msg_list; @@ -324,6 +325,7 @@ struct enic { */ struct completion mbox_comp; u8 mbox_expected_reply; + bool mbox_initialized; =20 /* PF: per-VF MBOX state, allocated when SRIOV V2 is enabled */ struct enic_vf_state { diff --git a/drivers/net/ethernet/cisco/enic/enic_admin.c b/drivers/net/eth= ernet/cisco/enic/enic_admin.c index 3e7103f2afc1..6642d76cd8c8 100644 --- a/drivers/net/ethernet/cisco/enic/enic_admin.c +++ b/drivers/net/ethernet/cisco/enic/enic_admin.c @@ -617,6 +617,7 @@ void enic_admin_channel_close(struct enic *enic) =20 vnic_intr_mask(&enic->admin_intr); enic_admin_teardown_intr(enic); + cancel_work_sync(&enic->link_notify_work); cancel_work_sync(&enic->admin_msg_work); enic_admin_msg_drain(enic); =20 @@ -636,6 +637,8 @@ void enic_admin_channel_close(struct enic *enic) vnic_cq_clean(&enic->admin_cq[0]); vnic_cq_clean(&enic->admin_cq[1]); vnic_intr_clean(&enic->admin_intr); + + enic->admin_rq_handler =3D NULL; enic_admin_free_resources(enic); =20 enic->admin_chan_up =3D false; diff --git a/drivers/net/ethernet/cisco/enic/enic_main.c b/drivers/net/ethe= rnet/cisco/enic/enic_main.c index 6992411bd3b5..68a293d60ab8 100644 --- a/drivers/net/ethernet/cisco/enic/enic_main.c +++ b/drivers/net/ethernet/cisco/enic/enic_main.c @@ -60,6 +60,8 @@ #include "enic_clsf.h" #include "enic_rq.h" #include "enic_wq.h" +#include "enic_admin.h" +#include "enic_mbox.h" =20 #define ENIC_NOTIFY_TIMER_PERIOD (2 * HZ) =20 @@ -411,6 +413,24 @@ static void enic_set_rx_coal_setting(struct enic *enic) rx_coal->use_adaptive_rx_coalesce =3D 1; } =20 +static void enic_link_notify_work_handler(struct work_struct *work) +{ + struct enic *enic =3D container_of(work, struct enic, + link_notify_work); + u32 state; + u16 i; + + if (!enic_sriov_enabled(enic) || !enic->vf_state) + return; + + state =3D netif_carrier_ok(enic->netdev) ? + ENIC_MBOX_LINK_STATE_ENABLE : + ENIC_MBOX_LINK_STATE_DISABLE; + + for (i =3D 0; i < enic->num_vfs; i++) + enic_mbox_send_link_state(enic, i, state); +} + static void enic_link_check(struct enic *enic) { int link_status =3D vnic_dev_link_status(enic->vdev); @@ -420,9 +440,13 @@ static void enic_link_check(struct enic *enic) netdev_info(enic->netdev, "Link UP\n"); netif_carrier_on(enic->netdev); enic_set_rx_coal_setting(enic); + if (enic_sriov_enabled(enic) && enic->vf_state) + schedule_work(&enic->link_notify_work); } else if (!link_status && carrier_ok) { netdev_info(enic->netdev, "Link DOWN\n"); netif_carrier_off(enic->netdev); + if (enic_sriov_enabled(enic) && enic->vf_state) + schedule_work(&enic->link_notify_work); } } =20 @@ -2154,15 +2178,47 @@ static void enic_reset(struct work_struct *work) /* Stop any activity from infiniband */ enic_set_api_busy(enic, true); =20 + /* Fully tear down the V2 admin/MBOX channel before the soft reset. + * The reset wipes all hardware queues including the admin WQ/RQ; + * closing first tells firmware to stop the admin QP (so it no longer + * DMAs from the about-to-be-reset rings) and frees the admin resources + * so they are cleanly re-allocated afterwards. + */ + if (enic_sriov_enabled(enic) && + enic->vf_type =3D=3D ENIC_VF_TYPE_V2) + enic_admin_channel_close(enic); + enic_stop(enic->netdev); + enic_dev_soft_reset(enic); enic_reset_addr_lists(enic); enic_init_vnic_resources(enic); enic_set_rss_nic_cfg(enic); enic_dev_set_ig_vlan_rewrite_mode(enic); enic_ext_cq(enic); + enic_open(enic->netdev); =20 + /* Re-establish the admin/MBOX channel after the data path is back up, + * mirroring the SR-IOV enable path (channel open + mbox init). The + * channel was fully torn down by enic_admin_channel_close() above. + */ + if (enic_sriov_enabled(enic) && + enic->vf_type =3D=3D ENIC_VF_TYPE_V2) { + if (enic_admin_channel_open(enic)) { + netdev_err(enic->netdev, + "admin channel reopen after reset failed\n"); + } else { + enic_mbox_init(enic); + /* The link came back up during enic_open() above + * while MBOX sends were still disabled (channel not + * yet reopened), so that link-notify was dropped. + * Re-push current link state to registered VFs now. + */ + schedule_work(&enic->link_notify_work); + } + } + /* Allow infiniband to fiddle with the device again */ enic_set_api_busy(enic, false); =20 @@ -2180,16 +2236,46 @@ static void enic_tx_hang_reset(struct work_struct *= work) /* Stop any activity from infiniband */ enic_set_api_busy(enic, true); =20 + /* Fully tear down the V2 admin/MBOX channel before the hang reset, for + * the same reason as the soft reset path: stop the admin QP and free + * the admin resources before the hardware queues are wiped. + */ + if (enic_sriov_enabled(enic) && + enic->vf_type =3D=3D ENIC_VF_TYPE_V2) + enic_admin_channel_close(enic); + enic_dev_hang_notify(enic); enic_stop(enic->netdev); + enic_dev_hang_reset(enic); enic_reset_addr_lists(enic); enic_init_vnic_resources(enic); enic_set_rss_nic_cfg(enic); enic_dev_set_ig_vlan_rewrite_mode(enic); enic_ext_cq(enic); + enic_open(enic->netdev); =20 + /* Re-establish the admin/MBOX channel after the data path is back up, + * mirroring the SR-IOV enable path (channel open + mbox init). The + * channel was fully torn down by enic_admin_channel_close() above. + */ + if (enic_sriov_enabled(enic) && + enic->vf_type =3D=3D ENIC_VF_TYPE_V2) { + if (enic_admin_channel_open(enic)) { + netdev_err(enic->netdev, + "admin channel reopen after reset failed\n"); + } else { + enic_mbox_init(enic); + /* The link came back up during enic_open() above + * while MBOX sends were still disabled (channel not + * yet reopened), so that link-notify was dropped. + * Re-push current link state to registered VFs now. + */ + schedule_work(&enic->link_notify_work); + } + } + /* Allow infiniband to fiddle with the device again */ enic_set_api_busy(enic, false); =20 @@ -2200,6 +2286,8 @@ static void enic_tx_hang_reset(struct work_struct *wo= rk) =20 static int enic_set_intr_mode(struct enic *enic) { + unsigned int admin_reserve =3D enic->has_admin_channel ? 1 : 0; + unsigned int min_intr =3D ENIC_MSIX_MIN_INTR + admin_reserve; unsigned int i; int num_intr; =20 @@ -2210,12 +2298,12 @@ static int enic_set_intr_mode(struct enic *enic) */ =20 if (enic->config.intr_mode < 1 && - enic->intr_avail >=3D ENIC_MSIX_MIN_INTR) { + enic->intr_avail >=3D min_intr) { for (i =3D 0; i < enic->intr_avail; i++) enic->msix_entry[i].entry =3D i; =20 num_intr =3D pci_enable_msix_range(enic->pdev, enic->msix_entry, - ENIC_MSIX_MIN_INTR, + min_intr, enic->intr_avail); if (num_intr > 0) { vnic_dev_set_intr_mode(enic->vdev, @@ -2310,7 +2398,13 @@ static int enic_adjust_resources(struct enic *enic) enic->cq_count =3D 2; enic->intr_count =3D enic->intr_avail; break; - case VNIC_DEV_INTR_MODE_MSIX: + case VNIC_DEV_INTR_MODE_MSIX: { + /* Reserve one MSI-X slot for the admin channel interrupt + * when V2 SR-IOV admin channel resources are present. + */ + unsigned int admin_reserve =3D + enic->has_admin_channel ? 1 : 0; + /* Adjust the number of wqs/rqs/cqs/interrupts that will be * used based on which resource is the most constrained */ @@ -2319,7 +2413,8 @@ static int enic_adjust_resources(struct enic *enic) ENIC_RQ_MIN_DEFAULT); rq_avail =3D min3(enic->rq_avail, ENIC_RQ_MAX, rq_default); max_queues =3D min(enic->cq_avail, - enic->intr_avail - ENIC_MSIX_RESERVED_INTR); + enic->intr_avail - ENIC_MSIX_RESERVED_INTR - + admin_reserve); if (wq_avail + rq_avail <=3D max_queues) { enic->rq_count =3D rq_avail; enic->wq_count =3D wq_avail; @@ -2337,6 +2432,7 @@ static int enic_adjust_resources(struct enic *enic) enic->intr_count =3D enic->cq_count + ENIC_MSIX_RESERVED_INTR; =20 break; + } default: dev_err(enic_get_dev(enic), "Unknown interrupt mode\n"); return -EINVAL; @@ -2689,6 +2785,140 @@ static void enic_sriov_detect_vf_type(struct enic *= enic) enic->vf_type =3D ENIC_VF_TYPE_NONE; } } + +static int __maybe_unused +enic_sriov_v2_enable(struct enic *enic, int num_vfs) +{ + int err; + + if (!enic->has_admin_channel) { + netdev_err(enic->netdev, + "V2 SR-IOV requires admin channel resources\n"); + return -EOPNOTSUPP; + } + + enic->vf_state =3D kcalloc(num_vfs, sizeof(*enic->vf_state), GFP_KERNEL); + if (!enic->vf_state) + return -ENOMEM; + + err =3D enic_admin_channel_open(enic); + if (err) { + netdev_err(enic->netdev, + "Failed to open admin channel: %d\n", err); + goto free_vf_state; + } + + enic_mbox_init(enic); + + enic->num_vfs =3D num_vfs; + + err =3D pci_enable_sriov(enic->pdev, num_vfs); + if (err) { + netdev_err(enic->netdev, + "pci_enable_sriov failed: %d\n", err); + goto close_admin; + } + + enic->priv_flags |=3D ENIC_SRIOV_ENABLED; + return num_vfs; + +close_admin: + enic->num_vfs =3D 0; + enic_admin_channel_close(enic); +free_vf_state: + kfree(enic->vf_state); + enic->vf_state =3D NULL; + return err; +} + +static void enic_sriov_v2_disable(struct enic *enic) +{ + /* Stop new VF link-state broadcasts before tearing down vf_state. + * Clearing ENIC_SRIOV_ENABLED makes enic_link_check() (called from + * the notify timer/ISR) skip the VF notify path, and cancelling + * link_notify_work ensures any already-queued broadcast has finished + * before vf_state is freed, closing a use-after-free window. + */ + enic->priv_flags &=3D ~ENIC_SRIOV_ENABLED; + cancel_work_sync(&enic->link_notify_work); + + pci_disable_sriov(enic->pdev); + enic_admin_channel_close(enic); + kfree(enic->vf_state); + enic->vf_state =3D NULL; + enic->num_vfs =3D 0; +} + +/* + * enic_sriov_configure() and its V2 helpers are defined but not yet wired + * into enic_driver via .sriov_configure (see the __maybe_unused annotatio= ns); + * V2 enable/disable is activated in a follow-up series. Because the call= back + * is not registered, it cannot run concurrently with the rtnl-protected r= eset + * paths (enic_reset(), enic_tx_hang_reset()) yet. Serialization against = those + * paths is added together with the .sriov_configure wiring in that series. + */ +static int __maybe_unused +enic_sriov_configure(struct pci_dev *pdev, int num_vfs) +{ + struct net_device *netdev =3D pci_get_drvdata(pdev); + struct enic *enic =3D netdev_priv(netdev); + struct enic_port_profile *pp; + int err; + + if (num_vfs > 0) { + if (enic->config.mq_subvnic_count) { + netdev_err(netdev, + "SR-IOV not supported with multi-queue sub-vnics\n"); + return -EOPNOTSUPP; + } + + if (enic->vf_type =3D=3D ENIC_VF_TYPE_NONE) { + netdev_err(netdev, + "SR-IOV not supported on this firmware version\n"); + return -EOPNOTSUPP; + } + + if (enic->vf_type =3D=3D ENIC_VF_TYPE_V2) + return enic_sriov_v2_enable(enic, num_vfs); + + pp =3D kcalloc(num_vfs, sizeof(*pp), GFP_KERNEL); + if (!pp) + return -ENOMEM; + + err =3D pci_enable_sriov(pdev, num_vfs); + if (err) { + kfree(pp); + return err; + } + + kfree(enic->pp); + enic->pp =3D pp; + enic->num_vfs =3D num_vfs; + enic->priv_flags |=3D ENIC_SRIOV_ENABLED; + return num_vfs; + } + + if (!enic_sriov_enabled(enic)) + return 0; + + if (enic->vf_type =3D=3D ENIC_VF_TYPE_V2) { + enic_sriov_v2_disable(enic); + return 0; + } + + pp =3D kzalloc_obj(*enic->pp, GFP_KERNEL); + if (!pp) + return -ENOMEM; + + pci_disable_sriov(pdev); + enic->num_vfs =3D 0; + enic->priv_flags &=3D ~ENIC_SRIOV_ENABLED; + + kfree(enic->pp); + enic->pp =3D pp; + + return 0; +} #endif =20 static int enic_probe(struct pci_dev *pdev, const struct pci_device_id *en= t) @@ -2787,12 +3017,18 @@ static int enic_probe(struct pci_dev *pdev, const s= truct pci_device_id *ent) goto err_out_vnic_unregister; =20 #ifdef CONFIG_PCI_IOV - /* Get number of subvnics */ + enic_sriov_detect_vf_type(enic); + + /* Auto-enable SR-IOV if VFs were pre-configured (e.g. at boot). + * V2 VFs require the admin channel, which is not yet set up at probe + * time; use sysfs (enic_sriov_configure) to enable V2 SR-IOV instead. + */ pos =3D pci_find_ext_capability(pdev, PCI_EXT_CAP_ID_SRIOV); if (pos) { pci_read_config_word(pdev, pos + PCI_SRIOV_TOTAL_VF, &enic->num_vfs); - if (enic->num_vfs) { + if (enic->num_vfs && + enic->vf_type !=3D ENIC_VF_TYPE_V2) { err =3D pci_enable_sriov(pdev, enic->num_vfs); if (err) { dev_err(dev, "SRIOV enable failed, aborting." @@ -2804,7 +3040,6 @@ static int enic_probe(struct pci_dev *pdev, const str= uct pci_device_id *ent) num_pps =3D enic->num_vfs; } } - enic_sriov_detect_vf_type(enic); #endif =20 /* Allocate structure for port profiles */ @@ -2881,6 +3116,7 @@ static int enic_probe(struct pci_dev *pdev, const str= uct pci_device_id *ent) INIT_WORK(&enic->reset, enic_reset); INIT_WORK(&enic->tx_hang_reset, enic_tx_hang_reset); INIT_WORK(&enic->change_mtu_work, enic_change_mtu_work); + INIT_WORK(&enic->link_notify_work, enic_link_notify_work_handler); =20 for (i =3D 0; i < enic->wq_count; i++) spin_lock_init(&enic->wq[i].lock); @@ -3034,14 +3270,16 @@ static void enic_remove(struct pci_dev *pdev) cancel_work_sync(&enic->tx_hang_reset); cancel_work_sync(&enic->change_mtu_work); unregister_netdev(netdev); - enic_dev_deinit(enic); - vnic_dev_close(enic->vdev); #ifdef CONFIG_PCI_IOV if (enic_sriov_enabled(enic)) { - pci_disable_sriov(pdev); - enic->priv_flags &=3D ~ENIC_SRIOV_ENABLED; + if (enic->vf_type =3D=3D ENIC_VF_TYPE_V2) + enic_sriov_v2_disable(enic); + else + pci_disable_sriov(pdev); } #endif + enic_dev_deinit(enic); + vnic_dev_close(enic->vdev); kfree(enic->pp); vnic_dev_unregister(enic->vdev); enic_iounmap(enic); diff --git a/drivers/net/ethernet/cisco/enic/enic_mbox.c b/drivers/net/ethe= rnet/cisco/enic/enic_mbox.c index 701972f90627..6fb6a1027bb6 100644 --- a/drivers/net/ethernet/cisco/enic/enic_mbox.c +++ b/drivers/net/ethernet/cisco/enic/enic_mbox.c @@ -624,8 +624,17 @@ int enic_mbox_vf_unregister(struct enic *enic) =20 void enic_mbox_init(struct enic *enic) { + /* mbox_lock and mbox_comp must be initialized exactly once per + * device lifetime; the PF sriov_configure path can re-enter this + * on each enable cycle where these primitives are already set up. + */ + if (!enic->mbox_initialized) { + mutex_init(&enic->mbox_lock); + init_completion(&enic->mbox_comp); + enic->mbox_initialized =3D true; + } else { + reinit_completion(&enic->mbox_comp); + } enic->mbox_msg_num =3D 0; - mutex_init(&enic->mbox_lock); - init_completion(&enic->mbox_comp); enic->admin_rq_handler =3D enic_mbox_recv_handler; } diff --git a/drivers/net/ethernet/cisco/enic/enic_pp.c b/drivers/net/ethern= et/cisco/enic/enic_pp.c index 4720a952725d..3f611e240c25 100644 --- a/drivers/net/ethernet/cisco/enic/enic_pp.c +++ b/drivers/net/ethernet/cisco/enic/enic_pp.c @@ -25,6 +25,11 @@ int enic_is_valid_pp_vf(struct enic *enic, int vf, int *= err) if (vf !=3D PORT_SELF_VF) { #ifdef CONFIG_PCI_IOV if (enic_sriov_enabled(enic)) { + /* V2 SR-IOV uses MBOX, not port profiles */ + if (enic->vf_type =3D=3D ENIC_VF_TYPE_V2) { + *err =3D -EOPNOTSUPP; + goto err_out; + } if (vf < 0 || vf >=3D enic->num_vfs) { *err =3D -EINVAL; goto err_out; diff --git a/drivers/net/ethernet/cisco/enic/enic_res.c b/drivers/net/ether= net/cisco/enic/enic_res.c index 2b7545d6a67f..436326ace049 100644 --- a/drivers/net/ethernet/cisco/enic/enic_res.c +++ b/drivers/net/ethernet/cisco/enic/enic_res.c @@ -59,6 +59,7 @@ int enic_get_vnic_config(struct enic *enic) GET_CONFIG(intr_timer_usec); GET_CONFIG(loop_tag); GET_CONFIG(num_arfs); + GET_CONFIG(mq_subvnic_count); GET_CONFIG(max_rq_ring); GET_CONFIG(max_wq_ring); GET_CONFIG(max_cq_ring); diff --git a/drivers/net/ethernet/cisco/enic/vnic_enet.h b/drivers/net/ethe= rnet/cisco/enic/vnic_enet.h index 9e8e86262a3f..519d2969990b 100644 --- a/drivers/net/ethernet/cisco/enic/vnic_enet.h +++ b/drivers/net/ethernet/cisco/enic/vnic_enet.h @@ -21,7 +21,9 @@ struct vnic_enet_config { u16 loop_tag; u16 vf_rq_count; u16 num_arfs; - u8 reserved[66]; + u8 reserved1[32]; + u16 mq_subvnic_count; + u8 reserved2[32]; u32 max_rq_ring; // MAX RQ ring size u32 max_wq_ring; // MAX WQ ring size u32 max_cq_ring; // MAX CQ ring size --=20 2.43.0 From nobody Sat Jul 25 03:48:19 2026 Received: from rcdn-iport-9.cisco.com (rcdn-iport-9.cisco.com [173.37.86.80]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8050E375F99; Sun, 19 Jul 2026 08:43:27 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=173.37.86.80 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784450609; cv=none; b=RNumplWPV5aPpVPLfNK9kR5F5ZDJ/ko4c4T1w8XIVQxCLqoHuZwMiDoW0YESCynMvaf56ZOoZlWKse9iXA4caQBSIObWHCPvwsveUyztx4ZLEm0qw3rLTk6lau5/mK78Azv1w4nUbOXMTYAwboUF3IFE5lY3iKm5eOnT+enMAGA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784450609; c=relaxed/simple; bh=B8hX+8NYlp4nvqpweTDRHzDB20zFjnemFDEBdx6UUaY=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=RAQQIMpHv0e7qOO4asTxRFyh5B62x26muDU8cax2Hk9xO++TkVnd5FIszUq35jBdNhS0iKixyhlrmqpfVUAjUH0fESVMmKi+vnhHBOfIXFquIweeLVSPpti/OYhRMdMxA3l9tOv5Lu/oAVR7dZMSb0pX9YqYfZlQP4wFzIAHL+4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=cisco.com; spf=pass smtp.mailfrom=cisco.com; dkim=pass (2048-bit key) header.d=cisco.com header.i=@cisco.com header.b=GNLj2OFZ; arc=none smtp.client-ip=173.37.86.80 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=cisco.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=cisco.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=cisco.com header.i=@cisco.com header.b="GNLj2OFZ" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cisco.com; i=@cisco.com; l=8924; q=dns/txt; s=iport01; t=1784450607; x=1785660207; h=from:date:subject:mime-version:content-transfer-encoding: message-id:references:in-reply-to:to:cc; bh=4gwSJKlKJDUOeRYixBT9bOvQy8xbX+FlvXDjIciUTrw=; b=GNLj2OFZuASMfYlKjFal1JRuBrm7cdK1bGrmO2ho7mr5rbv5k4HcIzxl jOV7SEa4jnCJhqnNt4jwhxi6QBwb0CdBF8sZNtXQ7+vEaltppGVNS5QW4 fvkDWS4YqzsJHXGDth+fthsLtpq8wWqqdIGBAabK4cW4sxs1eZR15p88f 4pbEY3had+V9JGi0+Rf5O/RWES9XPzViIRH2VExxbqNaF2zjIigGTkv2m KFrKkVPnXxTlmC1VPPKXzC8fZcG2tvsayhjDFJnxuEAiZ0oPn7YELbjq1 e7+CgXNaJ8jCv03LdCsSzUxy5/Z34hvwmdXdCCnjXY2/CGUwY6siWpdGW g==; X-CSE-ConnectionGUID: MuXEb+sdRVS7u2vm84790A== X-CSE-MsgGUID: JY6AlI69QpGFKIsdb3H2mg== X-IPAS-Result: =?us-ascii?q?A0AmAAAEjVxq/5H/Ja1aHQEBAQEJARIBBQUBgXwIAQsBg?= =?us-ascii?q?laBUkNJhFeIG4c3giGeHhSBag8BAQEPUQQBAYUFAo1TAiY0CQ4BAgQDAgMBA?= =?us-ascii?q?QEBAQEBAQEBAQsBAQUBAQECAQcFgQ4ThlyGWwIBAyMEUhAlAiYCAkYQBgESg?= =?us-ascii?q?wKCdQO9C3p/M4EBhH3ZSIFmAQsUAYEKLgGFcIJrAYFwhAc7hEEnG4FJRIEUA?= =?us-ascii?q?YNphAUlg3SCagSCDYEPEokdiE8JP4ECHANZLAFVEw0KCwcFYVIzAyAKCxISG?= =?us-ascii?q?BUCFBwTDwQWMh1wDCcSLBc0WBsHBYEdfxYEIYEChHYjHwM5f4EwdUp3LWkBE?= =?us-ascii?q?heBL4IUgToCBwMLGA1IESw3Bg4ZBD0BbgeNQCOBTQFREAkHAVkhFEI5BC4BA?= =?us-ascii?q?gwyJRUILQk6ApJjg36NX2uBNZ9ahCeBZqAPATKXZ5MFmQgjo15ZhGmBaDyBW?= =?us-ascii?q?TMaCBsVgyJTGQ+OLRYc0h4nMj8HAgcPApFzBIF6AQE?= IronPort-Data: A9a23:JCvtOaDfZF6+wRVW//Tiw5YqxClBgxIJ4kV8jS/XYbTApGx21GNUz 2ceDT+OaamIZTfwe4skbY/k8xxXsZTdydE1OVdlrnsFo1CmBibm6XV1Cm+qYkt+++WaFBoPA /02M4eGdIZvCCeA+n9BC5C5xVFkz6aEW7HgP+DNPyF1VGdMRTwo4f5Zs7ZRbrVA357jXmthh fuo+5eBYAD8imYtWo4pw/vrRC1H7ayaVAww5jTSVdgT1HfCmn8cCo4oJK3ZBxPQXolOE+emc P3Ixbe/83mx109F5gSNy+uTnuUiG9Y+DCDW4pZkc/HKbitq+kTe5p0G2M80Mi+7vdkmc+dZk 72hvbToIesg0zaldO41C3G0GAkmVUFKFSOuzXWX6aSuI0P6n3TE5uheIUt1JoEixN0oHDtQ8 KxEd2FVR0XW7w626OrTpuhEnM8vKozveYgYoHwllWGfBvc9SpeFSKLPjTNa9G5v3YYVQrCEO pdfMGYyBPjDS0Un1lM/E5sinfyohXnXeDxDo1XTrq0yi4TW5FEoieSwboCJK7RmQ+13rxixo kOYrl3hLT85FceTw2GlqHej07qncSTTHdh6+KeD3vJjnlCW7mAaFhATUVy1vb+/h1LWc9ZSK lAd/QIioKIz/UrtScPyNzW1rGOBtwwRR/JfFOo17AzLwa3Riy6aG2INZj1MctorsIkxXzNC/ laPk8nkGn9pvaGZRGy18qqSq3W5OUA9NWYIaCkFZREI79nqvMc4iRenZtJiFrOly974Azf9x xiUoyUkwbYel8gG0+O851+vqzShupTEUCYr6QjNGGGo9AV0YMiifYPA1LTAxexLIIDcShyKu 2IJ3pDOqusPFpqK0ieKRY3hAY2U2hpMCxWE6XYHInXr323FF6KLFWyI3AxDGQ== IronPort-HdrOrdr: A9a23:oL76ZaDmjrVRy+flHela55DYdb4zR+YMi2TDGXoBLSC9Ffb4qy nOppomPHDP5Qr5NEtLpTnEAtjjfZq+z/FICPcqXYtKNTOO0ACVxcNZnOjfKlbbdxEWmNQtsJ tIQuxSBML9CF5igd2/wgWpeuxQueWvweSEifjTynF1QRovUaFv6A9lFh2We3cGIjWufaBJc6 a0145guyeqf2gRY4CQA3kIWPWGmvjw/aiWGSLvw3UcmXOzZfTC0s+ALySl X-Talos-CUID: 9a23:HLRqy25cBZrdacPzWdss0VMkN8pmaXHn8kyPPhWmDDY1F+zNYArF X-Talos-MUID: 9a23:5BdvaQuBuR0+wjV8v82npgA8HcNJvfWXEG8jsrQdt8PfFXZ1NGLI X-IronPort-Anti-Spam-Filtered: true X-IronPort-AV: E=Sophos;i="6.25,172,1779148800"; d="scan'208";a="511456249" Received: from rcdn-l-core-08.cisco.com ([173.37.255.145]) by rcdn-iport-9.cisco.com with ESMTP/TLS/TLS_AES_256_GCM_SHA384; 19 Jul 2026 08:42:13 +0000 Received: from [127.0.0.1] (unknown [10.25.130.254]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by rcdn-l-core-08.cisco.com (Postfix) with ESMTPS id CC62018000475; Sun, 19 Jul 2026 08:42:12 +0000 (GMT) From: Satish Kharat Date: Sun, 19 Jul 2026 01:41:55 -0700 Subject: [PATCH net-next v12 11/12] enic: add V2 VF probe with admin channel and PF registration Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260719-enic-sriov-v2-admin-channel-v2-v12-11-6ff2c617001d@cisco.com> References: <20260719-enic-sriov-v2-admin-channel-v2-v12-0-6ff2c617001d@cisco.com> In-Reply-To: <20260719-enic-sriov-v2-admin-channel-v2-v12-0-6ff2c617001d@cisco.com> To: Andrew Lunn , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni Cc: netdev@vger.kernel.org, linux-kernel@vger.kernel.org, Breno Leitao , Sesidhar Baddela , Satish Kharat X-Mailer: b4 0.15.2 X-Developer-Signature: v=1; a=ed25519-sha256; t=1784450523; l=9181; i=satishkh@cisco.com; h=from:subject:message-id; bh=B8hX+8NYlp4nvqpweTDRHzDB20zFjnemFDEBdx6UUaY=; b=qiOHN06d82JLsijZp+IEdrLAfrvomRLnrlawwhcwQbsrVww/eAiFUrjdTCtZWUdwbz0IGu2z5 8x6N5+k9xboAJEBjnURlUjsdE1Ej21g3vuATBZLGC85RBOCgfPHLYMV X-Developer-Key: i=satishkh@cisco.com; a=ed25519; pk=4BxbLoCGvHeIMH/PbpiNVx8/2IcTjFvqcr0AD8K6WLY= X-Outbound-Client-TLS: ANONYMOUS;unknown [10.25.130.254];TLSv1.3;TLS_AES_256_GCM_SHA384;256 X-Outbound-SMTP-Client: 10.25.130.254, [10.25.130.254] X-Outbound-Node: rcdn-l-core-08.cisco.com When a V2 SR-IOV VF probes, open the admin channel, initialize the MBOX protocol, perform the capability check with the PF, and register with the PF. This establishes the PF-VF communication path that the PF uses to send link state notifications. The admin channel and MBOX registration happen after enic_dev_init() (which discovers admin channel resources) and before register_netdev() so the VF is fully initialized before the interface is visible to userspace. A V2 VF whose firmware did not provision admin WQ/RQ/CQ resources fails probe with -ENODEV from enic_admin_channel_open(); the admin channel is a hard requirement for V2 VFs. The admin channel is opened before enic_mbox_init() installs the receive handler. This is safe because enic_admin_rq_cq_service() checks admin_rq_handler before enqueuing received buffers, so any interrupt that fires between open and mbox_init is harmlessly discarded. On remove, the VF unregisters from the PF and closes its admin channel before tearing down data path resources. V2 VFs are not provisioned with an RES_TYPE_SRIOV_INTR resource by firmware, so bypass that check in the admin channel capability detection for V2 VFs. The PF still requires this resource. The admin MSI-X vector reserved by enic_set_intr_mode() is used for the admin channel interrupt. enic_adjust_resources() ensures the reserved slot is within intr_avail bounds even at maximum queue configurations. The admin INTR uses a RES_TYPE_INTR_CTRL slot shared with the data path. Signed-off-by: Satish Kharat --- drivers/net/ethernet/cisco/enic/enic.h | 1 + drivers/net/ethernet/cisco/enic/enic_main.c | 101 ++++++++++++++++++++++++= +--- drivers/net/ethernet/cisco/enic/enic_res.c | 3 +- 3 files changed, 94 insertions(+), 11 deletions(-) diff --git a/drivers/net/ethernet/cisco/enic/enic.h b/drivers/net/ethernet/= cisco/enic/enic.h index b5a43fe04877..62b8941489d7 100644 --- a/drivers/net/ethernet/cisco/enic/enic.h +++ b/drivers/net/ethernet/cisco/enic/enic.h @@ -452,6 +452,7 @@ void enic_reset_addr_lists(struct enic *enic); int enic_sriov_enabled(struct enic *enic); int enic_is_valid_vf(struct enic *enic, int vf); int enic_is_dynamic(struct enic *enic); +int enic_is_sriov_vf_v2(struct enic *enic); void enic_set_ethtool_ops(struct net_device *netdev); int __enic_set_rsskey(struct enic *enic); void enic_ext_cq(struct enic *enic); diff --git a/drivers/net/ethernet/cisco/enic/enic_main.c b/drivers/net/ethe= rnet/cisco/enic/enic_main.c index 68a293d60ab8..99c06a049800 100644 --- a/drivers/net/ethernet/cisco/enic/enic_main.c +++ b/drivers/net/ethernet/cisco/enic/enic_main.c @@ -316,6 +316,11 @@ static int enic_is_sriov_vf(struct enic *enic) enic->pdev->device =3D=3D PCI_DEVICE_ID_CISCO_VIC_ENET_VF_V2; } =20 +int enic_is_sriov_vf_v2(struct enic *enic) +{ + return enic->pdev->device =3D=3D PCI_DEVICE_ID_CISCO_VIC_ENET_VF_V2; +} + int enic_is_valid_vf(struct enic *enic, int vf) { #ifdef CONFIG_PCI_IOV @@ -2399,15 +2404,19 @@ static int enic_adjust_resources(struct enic *enic) enic->intr_count =3D enic->intr_avail; break; case VNIC_DEV_INTR_MODE_MSIX: { - /* Reserve one MSI-X slot for the admin channel interrupt - * when V2 SR-IOV admin channel resources are present. - */ - unsigned int admin_reserve =3D - enic->has_admin_channel ? 1 : 0; - /* Adjust the number of wqs/rqs/cqs/interrupts that will be - * used based on which resource is the most constrained + * used based on which resource is the most constrained. + * Reserve one extra MSI-X slot for the admin channel INTR + * when has_admin_channel is set so that + * enic_admin_setup_intr() can allocate at intr_count + * within the intr_avail bounds even when the data queue + * count is maxed out. intr_count counts only the data-path + * IRQs (registered by enic_request_intr()); the admin INTR + * lives at msix index intr_count and is set up later by + * enic_admin_setup_intr(). */ + unsigned int admin_reserve =3D enic->has_admin_channel ? 1 : 0; + wq_avail =3D min(enic->wq_avail, ENIC_WQ_MAX); rq_default =3D max(netif_get_num_default_rss_queues(), ENIC_RQ_MIN_DEFAULT); @@ -3104,6 +3113,44 @@ static int enic_probe(struct pci_dev *pdev, const st= ruct pci_device_id *ent) goto err_out_dev_close; } =20 + /* Initialise link_notify_work before the V2-VF admin-open block below: + * its error path (err_out_admin_close -> enic_admin_channel_close() -> + * cancel_work_sync()) would otherwise act on an uninitialised work. + */ + INIT_WORK(&enic->link_notify_work, enic_link_notify_work_handler); + + /* V2 VF: open admin channel and register with PF. + * Must happen before register_netdev so the VF is fully + * initialized before the interface is visible to userspace. + * + * admin_channel_open() runs before enic_mbox_init() installs + * the receive handler. This is safe because + * enic_admin_rq_cq_service() checks admin_rq_handler before + * enqueuing any received buffer, so interrupts that fire + * between open and mbox_init are harmlessly discarded. + */ + if (enic_is_sriov_vf_v2(enic)) { + err =3D enic_admin_channel_open(enic); + if (err) { + dev_err(dev, + "Failed to open admin channel: %d\n", err); + goto err_out_dev_deinit; + } + enic_mbox_init(enic); + err =3D enic_mbox_vf_capability_check(enic); + if (err) { + dev_err(dev, + "MBOX capability check failed: %d\n", err); + goto err_out_admin_close; + } + err =3D enic_mbox_vf_register(enic); + if (err) { + dev_err(dev, + "MBOX VF registration failed: %d\n", err); + goto err_out_admin_close; + } + } + netif_set_real_num_tx_queues(netdev, enic->wq_count); netif_set_real_num_rx_queues(netdev, enic->rq_count); =20 @@ -3116,7 +3163,6 @@ static int enic_probe(struct pci_dev *pdev, const str= uct pci_device_id *ent) INIT_WORK(&enic->reset, enic_reset); INIT_WORK(&enic->tx_hang_reset, enic_tx_hang_reset); INIT_WORK(&enic->change_mtu_work, enic_change_mtu_work); - INIT_WORK(&enic->link_notify_work, enic_link_notify_work_handler); =20 for (i =3D 0; i < enic->wq_count; i++) spin_lock_init(&enic->wq[i].lock); @@ -3129,7 +3175,7 @@ static int enic_probe(struct pci_dev *pdev, const str= uct pci_device_id *ent) err =3D enic_set_mac_addr(netdev, enic->mac_addr); if (err) { dev_err(dev, "Invalid MAC address, aborting\n"); - goto err_out_dev_deinit; + goto err_out_admin_close; } =20 enic->tx_coalesce_usecs =3D enic->config.intr_timer_usec; @@ -3227,11 +3273,23 @@ static int enic_probe(struct pci_dev *pdev, const s= truct pci_device_id *ent) err =3D register_netdev(netdev); if (err) { dev_err(dev, "Cannot register net device, aborting\n"); - goto err_out_dev_deinit; + goto err_out_admin_close; } =20 return 0; =20 +err_out_admin_close: + if (enic_is_sriov_vf_v2(enic)) { + if (enic->vf_registered) { + int unreg_err =3D enic_mbox_vf_unregister(enic); + + if (unreg_err) + netdev_warn(netdev, + "Failed to unregister from PF: %d\n", + unreg_err); + } + enic_admin_channel_close(enic); + } err_out_dev_deinit: enic_dev_deinit(enic); err_out_dev_close: @@ -3269,7 +3327,30 @@ static void enic_remove(struct pci_dev *pdev) cancel_work_sync(&enic->reset); cancel_work_sync(&enic->tx_hang_reset); cancel_work_sync(&enic->change_mtu_work); + + /* Close the admin channel and unregister from the PF before + * unregister_netdev() to prevent a late PF notification from + * touching a netdev that is being torn down. + */ + if (enic_is_sriov_vf_v2(enic)) { + if (enic->vf_registered) { + int unreg_err =3D enic_mbox_vf_unregister(enic); + + if (unreg_err) + netdev_warn(netdev, + "Failed to unregister from PF: %d\n", + unreg_err); + } + enic_admin_channel_close(enic); + } + unregister_netdev(netdev); + /* unregister_netdev() -> enic_stop() stops the notify timer, so + * no new link_notify_work can be queued past this point. Cancel + * unconditionally to cover the narrow window where + * enic_link_check() scheduled it just as SR-IOV was disabled. + */ + cancel_work_sync(&enic->link_notify_work); #ifdef CONFIG_PCI_IOV if (enic_sriov_enabled(enic)) { if (enic->vf_type =3D=3D ENIC_VF_TYPE_V2) diff --git a/drivers/net/ethernet/cisco/enic/enic_res.c b/drivers/net/ether= net/cisco/enic/enic_res.c index 436326ace049..74cd2ee3af5c 100644 --- a/drivers/net/ethernet/cisco/enic/enic_res.c +++ b/drivers/net/ethernet/cisco/enic/enic_res.c @@ -211,7 +211,8 @@ void enic_get_res_counts(struct enic *enic) vnic_dev_get_res_count(enic->vdev, RES_TYPE_ADMIN_RQ) >=3D 1 && vnic_dev_get_res_count(enic->vdev, RES_TYPE_ADMIN_CQ) >=3D ARRAY_SIZE(enic->admin_cq) && - vnic_dev_get_res_count(enic->vdev, RES_TYPE_SRIOV_INTR) >=3D 1; + (enic_is_sriov_vf_v2(enic) || + vnic_dev_get_res_count(enic->vdev, RES_TYPE_SRIOV_INTR) >=3D 1); =20 dev_info(enic_get_dev(enic), "vNIC resources avail: wq %d rq %d cq %d intr %d admin %s\n", --=20 2.43.0 From nobody Sat Jul 25 03:48:19 2026 Received: from rcdn-iport-2.cisco.com (rcdn-iport-2.cisco.com [173.37.86.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A426B378806; Sun, 19 Jul 2026 08:43:16 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=173.37.86.73 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784450598; cv=none; b=d5q4Xlqm9brqpxwO8hCMoEBH5V7JPoSSlb1592nLpqeK22b1EUVIjQINEchLYi6Gq/hrWciTWVwCwhyZpwUsn0/euef/Hi9q4B0bit1od+s3AK5DGNG9F7h/xHO8xJXRrtfIbRmcZ5ePxWGgrQtPlFDFdevNY4vZ72EDs6Kl7Sc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784450598; c=relaxed/simple; bh=FdWRKYYtk0FuUUegGgtNhjQIkPVlYiHEZ4/j6/h3dCk=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=iO7Z2UzDJiPdhU9ZJwAdvdhHuTWfaCasFpwrAX6/tNGJrYPEz2JcSQXyl7D2pksZ/IRDaHpI5Gl54OD7qUTaC1PmAMNpHEMtGcSeMkbiXcOIUJbOxbTOXpGiYN/gbJ/+GX/qczeUBIwkjHIpYjBQxmWQTeAeoVp5vePYO5t6RuU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=cisco.com; spf=pass smtp.mailfrom=cisco.com; dkim=pass (2048-bit key) header.d=cisco.com header.i=@cisco.com header.b=eatJrW9Z; arc=none smtp.client-ip=173.37.86.73 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=cisco.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=cisco.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=cisco.com header.i=@cisco.com header.b="eatJrW9Z" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cisco.com; i=@cisco.com; l=6579; q=dns/txt; s=iport01; t=1784450596; x=1785660196; h=from:date:subject:mime-version:content-transfer-encoding: message-id:references:in-reply-to:to:cc; bh=r7xz+wdOM2LeRCY3jPvIhQ2Pkga6rs6A23prvlcG3s0=; b=eatJrW9ZDoU6w2w/piY+gwQAoCIDb8uYiRZ7kNiaIHhcr+2TnQcwxmty TOLNNYZalhL960nDV42n6Urqw9SOWXo9jL2SwowXlfJ4fas7JtCvJzxCw VdIzqoRe0QH41JMiurwLuqA6ZDrjVVcHWrbQN+btEgyH4DyMk0xv/BI2m M+m9Qy7SZcgc2Or2ctKw5Qzbe5J20SgKui8+hVV+vq+Mb60Zq987eKQDd 9KdobMBDUTOKkey83LL/PpnrKBycKyqqWONSw5qt5fttpR12lJ7sSD7Oq 9jUyOCPje53ZZyVdeXIQXee44ISlXPxDcMuQMav7wjNTiRq5rsIeHdB9L w==; X-CSE-ConnectionGUID: bM0ql58iSLaPOCmKVZhi/Q== X-CSE-MsgGUID: xQ1IrfExQYKSnCSeiUR+PQ== X-IPAS-Result: =?us-ascii?q?A0BFAgBUjFxq/5H/Ja1aHgEBCxIMggULgleBUkNJhFePU?= =?us-ascii?q?oIhnh4UgWoPAQEBD1EEAQGFBQKNUwImNAkOAQIEAwIDAQEBAQEBAQEBAQELA?= =?us-ascii?q?QEFAQEBAgEHBYEOE4ZchlsCAQMjBFIQJQImAgJGEAYBEoMCgnUDvQl6fzOBA?= =?us-ascii?q?d5FgWYBCxQBgQouhXGCawGBcIQHO4RBJxuBSUSBFYNphCqDdIJqBIMcEpFsC?= =?us-ascii?q?T+BAhwDWSwBVRMNCgsHBWFSMwMgCgsSEhgVAhQcEw8EFjIdcAwnEiwXNFgbB?= =?us-ascii?q?wWBHX8WBCGBAoR2Ix8DOX+BMHVKdy1pARIXgS+CFIE6AgcDCxgNSBEsNwYOG?= =?us-ascii?q?QQ9AW4HjUAjgU5xATwdNBRoMw6BZh+SRBGDbY1fa4E1n1qEJ4FmoA8BMpdnk?= =?us-ascii?q?wWZCCOkN4RpgWg8gVkzGggbFYMiUxkPji0WHNIeJzI9AQEHAgcOAwuBaJF+A?= =?us-ascii?q?QE?= IronPort-Data: A9a23:JZUai64Kovv/FEqtB2sWYQxRtOXGchMFZxGqfqrLsTDasY5as4F+v mVMWm2PbPeJZmXyeIp1a4W/900OvMWAzYVhTgNqpX9gZn8b8sCt6fZ1gavT04J+CuWZESqLO u1HMoGowPgcFyGa/lH2dOC98RGQ7InQLpLkEunIJyttcgFtTSYlmHpLlvUw6mJSqYDR7zil5 5Wo/qUzBHf/g2Qqaj1NuvrawP9SlK2aVA0w7wRWic9j5Dcyp1FNZLoDKKe4KWfPQ4U8NoaSW +bZwbilyXjS9hErB8nNuu6TnpoiG+O60aCm0xK6aoD66vRwjnVaPpUTaJLwXXxqZwChxLid/ jniWauYEm/FNoWU8AgUvoIx/ytWZcWq85efSZSzXFD6I0DuKxPRL/tS4E4eN6hA4+IqUWB10 fUgDxsITS6xh+Px+efuIgVsrpxLwMjDJogTvDRkiDreF/tjGcCFSKTR7tge1zA17ixMNa+BP IxCNnw1MUmGOkESUrsUIMpWcOOAnXDjciZYq1K9rqss6G+Vxwt0uFToGIaEKoTXH5UFxi50o ErWxj3YGB83JOWu6gGD2Umhh9L13hvCDdd6+LqQs6QCbEeo7mAaFhATfVeyv/S8jkmwR5RZJ lB80i4jrLQ782SnQ9P8Vhv+p2SL1jYRV8dVGv839CmCza3b5wvfDW8BJhZCddYvnMw7Xzon0 hmOhdyBLTVpvKeYVjGb+6uYoC2aPTUTKykJZUcsVQIP7t/iiJs+ghLGUpBoF6vdptn0Hyzgh jOHti4zg50NgsMRkaa251bKh3SrvJehZgg4+gnaQEq74Q5jIo2ofYql7R7c9/koEWqCZkOKs H5Bn42V6/oDSMnd0ieMW+4KWrqu4p5pLQHhvLKmJLF5nxzFxpJpVdk4DO1WTKuxDvs5RA== IronPort-HdrOrdr: A9a23:G8TDZaPXaBIOhsBcTjejsMiBIKoaSvp037Dk7TEUdfUzSK2lfq GV8sjzuiWYtN98YhsdcJW7Scu9qBDnmqKdn7NwAV7KZmCP01dAbrsSjrcKqAeNJ8SRzJ856Y 5QN4VzF9H0C0F3loLZ7BnQKadY/PC3tI2lmuvayW5gVkVRa69m5xpkEQrzKDwReOA/P+tfKL OsouxaujGnZXMbKu6/BnUDQqz/guejruOfXfbDbCRXkjVnSliTmcXHLyQ= X-Talos-CUID: 9a23:84WsdGOPWSXs1O5DUgw/7UpJSsEeVnTl61zVAHGKCWcucejA X-Talos-MUID: 9a23:3Dbgawr6eFZiI7MlIiUezyFbbeZy7Ym+MXEuqpMtgMSFCBYzAw7I2Q== X-IronPort-Anti-Spam-Filtered: true X-IronPort-AV: E=Sophos;i="6.25,172,1779148800"; d="scan'208";a="498052880" Received: from rcdn-l-core-08.cisco.com ([173.37.255.145]) by rcdn-iport-2.cisco.com with ESMTP/TLS/TLS_AES_256_GCM_SHA384; 19 Jul 2026 08:42:14 +0000 Received: from [127.0.0.1] (unknown [10.25.130.254]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by rcdn-l-core-08.cisco.com (Postfix) with ESMTPS id 96E5A18000595; Sun, 19 Jul 2026 08:42:13 +0000 (GMT) From: Satish Kharat Date: Sun, 19 Jul 2026 01:41:56 -0700 Subject: [PATCH net-next v12 12/12] enic: re-establish V2 VF admin channel and PF registration after reset Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260719-enic-sriov-v2-admin-channel-v2-v12-12-6ff2c617001d@cisco.com> References: <20260719-enic-sriov-v2-admin-channel-v2-v12-0-6ff2c617001d@cisco.com> In-Reply-To: <20260719-enic-sriov-v2-admin-channel-v2-v12-0-6ff2c617001d@cisco.com> To: Andrew Lunn , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni Cc: netdev@vger.kernel.org, linux-kernel@vger.kernel.org, Breno Leitao , Sesidhar Baddela , Satish Kharat X-Mailer: b4 0.15.2 X-Developer-Signature: v=1; a=ed25519-sha256; t=1784450523; l=6769; i=satishkh@cisco.com; h=from:subject:message-id; bh=FdWRKYYtk0FuUUegGgtNhjQIkPVlYiHEZ4/j6/h3dCk=; b=Ys4Vzd3hyOGUlc1pJsPpice2f3un797gaRPkn5Q1XDbUd2jEKBR9bhGavaySBS8nLJUMGRV2N zjxnTgUO4nVAyMMe5ey2SE5rvTMEc5wyR6gbrYJZDt/kQgP+1f0sS4A X-Developer-Key: i=satishkh@cisco.com; a=ed25519; pk=4BxbLoCGvHeIMH/PbpiNVx8/2IcTjFvqcr0AD8K6WLY= X-Outbound-Client-TLS: ANONYMOUS;unknown [10.25.130.254];TLSv1.3;TLS_AES_256_GCM_SHA384;256 X-Outbound-SMTP-Client: 10.25.130.254, [10.25.130.254] X-Outbound-Node: rcdn-l-core-08.cisco.com The reset paths (enic_reset/enic_tx_hang_reset) tore down and re-opened the V2 admin/MBOX channel only for the PF: the close/reopen was gated on enic_sriov_enabled() && vf_type =3D=3D ENIC_VF_TYPE_V2, which is never true= on a VF (vf_type is set only on the PF; VFs are identified by enic_is_sriov_vf_v2()). A VF-initiated reset therefore left the VF admin QP wiped by the reset but never re-opened, and the VF never re-registered with the PF, so VF<->PF MBOX traffic (link state, MAC, packet filter) stopped working until the VF was re-probed. Factor the decision into enic_has_admin_chan() (true for a V2 PF while SR-IOV is enabled and for every V2 VF) and the reopen sequence into enic_admin_chan_reopen(). For a VF the helper additionally re-runs the probe-time handshake (enic_mbox_vf_capability_check() + enic_mbox_vf_register()) so the PF learns about the VF again; for a PF it re-pushes the current link state as before. Signed-off-by: Satish Kharat --- drivers/net/ethernet/cisco/enic/enic_main.c | 105 +++++++++++++++++-------= ---- 1 file changed, 65 insertions(+), 40 deletions(-) diff --git a/drivers/net/ethernet/cisco/enic/enic_main.c b/drivers/net/ethe= rnet/cisco/enic/enic_main.c index 99c06a049800..be3ae0a57af3 100644 --- a/drivers/net/ethernet/cisco/enic/enic_main.c +++ b/drivers/net/ethernet/cisco/enic/enic_main.c @@ -2171,6 +2171,57 @@ static void enic_set_api_busy(struct enic *enic, boo= l busy) spin_unlock(&enic->enic_api_lock); } =20 +/* The admin/MBOX channel exists on a V2 PF while SR-IOV is enabled and on + * every V2 VF. A reset wipes the admin WQ/RQ/CQ, so such devices must te= ar + * the channel down before the reset and re-establish it afterwards. + */ +static bool enic_has_admin_chan(struct enic *enic) +{ + return enic_is_sriov_vf_v2(enic) || + (enic_sriov_enabled(enic) && enic->vf_type =3D=3D ENIC_VF_TYPE_V2); +} + +/* Re-establish the admin/MBOX channel after a reset has re-created the da= ta + * path. Mirrors the relevant part of the probe / SR-IOV-enable sequence: + * reopen the channel and reinitialise MBOX, then for a VF re-run the PF + * handshake (its admin QP and PF-side registration were torn down by the + * reset), or for a PF re-push the current link state to registered VFs. + */ +static void enic_admin_chan_reopen(struct enic *enic) +{ + int err; + + err =3D enic_admin_channel_open(enic); + if (err) { + netdev_err(enic->netdev, + "admin channel reopen after reset failed: %d\n", err); + return; + } + + enic_mbox_init(enic); + + if (enic_is_sriov_vf_v2(enic)) { + err =3D enic_mbox_vf_capability_check(enic); + if (err) { + netdev_err(enic->netdev, + "MBOX capability check after reset failed: %d\n", + err); + return; + } + err =3D enic_mbox_vf_register(enic); + if (err) + netdev_err(enic->netdev, + "MBOX VF re-registration after reset failed: %d\n", + err); + } else { + /* The link came back up during enic_open() above while MBOX + * sends were still disabled (channel not yet reopened), so that + * link-notify was dropped. Re-push current link state now. + */ + schedule_work(&enic->link_notify_work); + } +} + static void enic_reset(struct work_struct *work) { struct enic *enic =3D container_of(work, struct enic, reset); @@ -2189,8 +2240,7 @@ static void enic_reset(struct work_struct *work) * DMAs from the about-to-be-reset rings) and frees the admin resources * so they are cleanly re-allocated afterwards. */ - if (enic_sriov_enabled(enic) && - enic->vf_type =3D=3D ENIC_VF_TYPE_V2) + if (enic_has_admin_chan(enic)) enic_admin_channel_close(enic); =20 enic_stop(enic->netdev); @@ -2204,25 +2254,13 @@ static void enic_reset(struct work_struct *work) =20 enic_open(enic->netdev); =20 - /* Re-establish the admin/MBOX channel after the data path is back up, - * mirroring the SR-IOV enable path (channel open + mbox init). The - * channel was fully torn down by enic_admin_channel_close() above. + /* Re-establish the admin/MBOX channel after the data path is back up. + * It was fully torn down by enic_admin_channel_close() above; + * enic_admin_chan_reopen() reopens it and, for a PF re-pushes link + * state, or for a VF re-runs the probe-time PF handshake. */ - if (enic_sriov_enabled(enic) && - enic->vf_type =3D=3D ENIC_VF_TYPE_V2) { - if (enic_admin_channel_open(enic)) { - netdev_err(enic->netdev, - "admin channel reopen after reset failed\n"); - } else { - enic_mbox_init(enic); - /* The link came back up during enic_open() above - * while MBOX sends were still disabled (channel not - * yet reopened), so that link-notify was dropped. - * Re-push current link state to registered VFs now. - */ - schedule_work(&enic->link_notify_work); - } - } + if (enic_has_admin_chan(enic)) + enic_admin_chan_reopen(enic); =20 /* Allow infiniband to fiddle with the device again */ enic_set_api_busy(enic, false); @@ -2245,8 +2283,7 @@ static void enic_tx_hang_reset(struct work_struct *wo= rk) * the same reason as the soft reset path: stop the admin QP and free * the admin resources before the hardware queues are wiped. */ - if (enic_sriov_enabled(enic) && - enic->vf_type =3D=3D ENIC_VF_TYPE_V2) + if (enic_has_admin_chan(enic)) enic_admin_channel_close(enic); =20 enic_dev_hang_notify(enic); @@ -2261,25 +2298,13 @@ static void enic_tx_hang_reset(struct work_struct *= work) =20 enic_open(enic->netdev); =20 - /* Re-establish the admin/MBOX channel after the data path is back up, - * mirroring the SR-IOV enable path (channel open + mbox init). The - * channel was fully torn down by enic_admin_channel_close() above. + /* Re-establish the admin/MBOX channel after the data path is back up. + * It was fully torn down by enic_admin_channel_close() above; + * enic_admin_chan_reopen() reopens it and, for a PF re-pushes link + * state, or for a VF re-runs the probe-time PF handshake. */ - if (enic_sriov_enabled(enic) && - enic->vf_type =3D=3D ENIC_VF_TYPE_V2) { - if (enic_admin_channel_open(enic)) { - netdev_err(enic->netdev, - "admin channel reopen after reset failed\n"); - } else { - enic_mbox_init(enic); - /* The link came back up during enic_open() above - * while MBOX sends were still disabled (channel not - * yet reopened), so that link-notify was dropped. - * Re-push current link state to registered VFs now. - */ - schedule_work(&enic->link_notify_work); - } - } + if (enic_has_admin_chan(enic)) + enic_admin_chan_reopen(enic); =20 /* Allow infiniband to fiddle with the device again */ enic_set_api_busy(enic, false); --=20 2.43.0