From nobody Sun Jun 14 04:21:27 2026 Received: from alln-iport-7.cisco.com (alln-iport-7.cisco.com [173.37.142.94]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 02699346A19; Sun, 3 May 2026 11:24:04 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=173.37.142.94 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777807447; cv=none; b=Ny9qmfdPfFhqtTPvTSE8sV8U/+t81hPTu1IV0aKj8xBjcADPhmwMGqag2OWvZlw52rU7xxDh6w5Q3EkvdT2ofq6FYvx0qrobNbzsihxw+K5F/KvyI3Sl6e6dTv5AfhwYN9YzKyW1lgtwjGqIQavHjvY2DIdsmZkG5Rl52PiVQpQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777807447; c=relaxed/simple; bh=ZXSTJ7b/38rgvmYO873buvlool6i84d5a2ZJr/9ipT8=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=tPXkAKtjQMqI38CSrjEYKQhmU6RRM7YVtpVp4NBkc4FoPc3AowVE/MezzUzKdnVkkZY8MLLvFgr3S3hAaFphwdO9la1vsn9oEwhZsx760+24tQwHGB6gQYOGFpGiKzE1V16sP9Xk+/96qQMucMwhXAXitXOii7umyoXvYvs1Zl8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=cisco.com; spf=pass smtp.mailfrom=cisco.com; dkim=pass (2048-bit key) header.d=cisco.com header.i=@cisco.com header.b=MLuD6GMa; arc=none smtp.client-ip=173.37.142.94 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=cisco.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=cisco.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=cisco.com header.i=@cisco.com header.b="MLuD6GMa" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cisco.com; i=@cisco.com; l=2512; q=dns/txt; s=iport01; t=1777807445; x=1779017045; h=from:date:subject:mime-version:content-transfer-encoding: message-id:references:in-reply-to:to:cc; bh=Ks0lsgTrBdaqV3gPaDF/jXpyLgWa4qOq0Cl3j1wSwFo=; b=MLuD6GMaI8rHxvjQHupQGONv1lbgGhh97oxJC3+GwORfK7RH+hj4iIyY ObC61GwCXYS1zg6VnLhFaW7U12ZJI/40NWBvfP81cD/SA8ApPmwyCa/h7 k8jkNGuRrJ2YWWwO2kNPvgztS6N3OaQeR0tByXqQKkEOv4M/I5X3IVDLJ yI1fWyFRFBu0B3WcZ0Cbs+CzRqey/9D1UA7QyiZo/DS5ZBscBjhc24+9r DsSIcFZmyRIRcFawJqRoGmR0JAqFFUCYn14rKVk0bKO7KYjr9wPNftWQ1 72KnN2gAPQGjMWUakmiVW8VrhDM3oT+nOiUjzGKUVEjVHA45N1aVP1wlV w==; X-CSE-ConnectionGUID: lQA5TbGNStSS7yP/F4q2ug== X-CSE-MsgGUID: Lhapr4DhRfS+NUwg/wxuaQ== X-IPAS-Result: =?us-ascii?q?A0AnAADQLvdp/4v/Ja1aHQEBAQEJARIBBQUBgXwIAQsBg?= =?us-ascii?q?laBT0NJhFeIHIc3giGeHYF/DwEBAQ9RBAEBhQYCjTECJjQJDgECBAEBAQEDA?= =?us-ascii?q?gMBAQEBAQEBAQEBAQsBAQUBAQECAQcFgQ4ThlyGWwIBAyMEUhAlAiYCAkYQB?= =?us-ascii?q?gESgwKCdAO0Hnp/M4EBhH3ZQ4FkAQsUAYEKLgGFcYJmAYFwhAY7hD8nG4FJR?= =?us-ascii?q?IR9hAqEFIJpBIMcFI8bCT+BAhwDWSwBVRMNCgsHBWFSMwMgCgsSEhgVAhQvD?= =?us-ascii?q?wQWMh1wDCcSLBc0WBsHBYFLgwBtFDEEIYEFhGB4IywDTnIDCxgNSBEsNwYOG?= =?us-ascii?q?wQ9AW4HimYdD4E/cT1Sgmgpk0CDMo1da6EOhCaBZKAOATKXZpMFLJhaIqNPM?= =?us-ascii?q?TeEaIFoPIFZMxoIGxWDIlMZD44tFhzJPCYyPgEHAgcOAoFzkACBfQEB?= IronPort-Data: A9a23:EpWWr6LKuZk+I1EKFE+RDJQlxSXFcZb7ZxGr2PjKsXjdYENShT0Oy WQeXT+OOPiOY2b8f48nPo+1/UpX7JXRxtVjQFEd+CA2RRqmiyZq6fd1j6vUF3nPRiEWZBs/t 63yUvGZcoZsCCSa/kvxWlTYhSEU/bmSQbbhA/LzNCl0RAt1IA8skhsLd9QR2uaEuvDnRVnW0 T/Oi5eHYgH9hmYvajt8B5+r8XuDgtyj4Fv0gXRmDRx7lAe2v2UYCpsZOZawIxPQKqFIHvS3T vr017qw+GXU5X8FUrtJRZ6iLyXm6paLVeS/oiI+t5qK23CulQRuukoPD8fwXG8M49m/c3+d/ /0W3XC4YV9B0qQhA43xWTEAe811FfUuFLMqvRFTvOTLp3AqfUcAzN1qNWhrAYc33N1rOloR5 MRfBmwUNA2M0rfeLLKTEoGAh+w5J8XteYdasXZ6wHSBV7AtQIvIROPB4towMDUY358VW62BI ZBENHw2N0Wojx5nYj/7DLollfupnXn8cxVTqUmeouw85G27IAlZj+O0aIKPJIfULSlTtmKYr 3rco0j5OB8hd/qi1jXC7WKxj9aayEsXX6pXTtVU7MVCjFSNy2k7BBQIWF6/pvelzEizR7p3L 0EV5iMphaQ//k2iSJ/6RRLQiHOIoxsVSddMO+I97w6Jx+zf5APxLmsZRDdpa9E8ssIyAzsw2 TehltXlHzFw87uYVX6Q6J+QsDW0fyMVRUcabCUJSwot+dTvoIgvyBnIS75LFK+zk82wGjzqx T2OhDYxiq9VjsMR0ai/u1fdjFqEoJXVQgMrzhvYU3jj7Q5jYoOhIYuy5jDmAe1oNo2VSByF+ XMDgcXbtLpIBpCWnyvLS+IIdF2028u43PTnqQYHN/EcG/6FohZPoag4DOlCGXpU IronPort-HdrOrdr: A9a23:+OmI46FGbMcXnA8+pLqEHseALOsnbusQ8zAXPo5KOHhom6uj9v xG/c506faaslsssR0b8+xofZPwI080lqQb3WBuB9eftWDd0QPCEGgh1/qA/9SKIUHDH4BmpM JdWpk7JdHuAV19lMrgpCu1A78bsaC62ZHtqOvAw3JqVA1wL5th4QBwFxqBHiRNNW977OICZf 2hz/sCgSa8cnIKacn+LHwEU+/f4+DvrvvdEGY779pN0nj1sdtug4SKaCSl4g== X-Talos-CUID: 9a23:S5MAd2F92bwZ1GX1qmJjzWdJJsoKVEbB90yPMnf/JzpwEqCaHAo= X-Talos-MUID: 9a23:94MlBQaKhr73iuBTkz/lnCs+GMdS7Z/tLn0SntIZq9SNDHkl X-IronPort-Anti-Spam-Filtered: true X-IronPort-AV: E=Sophos;i="6.23,213,1770595200"; d="scan'208";a="731002636" Received: from rcdn-l-core-02.cisco.com ([173.37.255.139]) by alln-iport-7.cisco.com with ESMTP/TLS/TLS_AES_256_GCM_SHA384; 03 May 2026 11:22:56 +0000 Received: from [127.0.0.1] (unknown [10.25.128.216]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by rcdn-l-core-02.cisco.com (Postfix) with ESMTPS id EE384180007C5; Sun, 3 May 2026 11:22:55 +0000 (GMT) From: Satish Kharat Date: Sun, 03 May 2026 04:22:38 -0700 Subject: [PATCH net-next v6 01/10] enic: verify firmware supports V2 SR-IOV at probe time Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260503-enic-sriov-v2-admin-channel-v2-v6-1-0af4fbc2d86d@cisco.com> References: <20260503-enic-sriov-v2-admin-channel-v2-v6-0-0af4fbc2d86d@cisco.com> In-Reply-To: <20260503-enic-sriov-v2-admin-channel-v2-v6-0-0af4fbc2d86d@cisco.com> To: Andrew Lunn , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni Cc: netdev@vger.kernel.org, linux-kernel@vger.kernel.org, Sesidhar Baddela , Satish Kharat , Breno Leitao X-Mailer: b4 0.15.2 X-Developer-Signature: v=1; a=ed25519-sha256; t=1777807374; l=2560; i=satishkh@cisco.com; h=from:subject:message-id; bh=ZXSTJ7b/38rgvmYO873buvlool6i84d5a2ZJr/9ipT8=; b=oSM8F+SQt9tLJdFaI+2P7sK1nhrmX8yujytbRf3JpyExVl96gZ1JbjcCAO3xTd8Lf0UxHz3Ff h0gma6X5Xm3B5J8rwax/9PLpRh4A2d/nyUz8/aUR+TP1FusgHzpkSq6 X-Developer-Key: i=satishkh@cisco.com; a=ed25519; pk=4BxbLoCGvHeIMH/PbpiNVx8/2IcTjFvqcr0AD8K6WLY= X-Outbound-Client-TLS: ANONYMOUS;unknown [10.25.128.216];TLSv1.3;TLS_AES_256_GCM_SHA384;256 X-Outbound-SMTP-Client: 10.25.128.216, [10.25.128.216] X-Outbound-Node: rcdn-l-core-02.cisco.com During PF probe, query the firmware get-supported-feature interface to verify that the running firmware supports V2 SR-IOV. Firmware version 5.3(4.72) and later report VIC_FEATURE_SRIOV via CMD_GET_SUPP_FEATURE_VER. If the firmware does not support the feature, set vf_type to ENIC_VF_TYPE_NONE and log a warning so the admin knows a firmware upgrade is needed. The VIC_FEATURE_SRIOV enum value (4) matches the firmware ABI. A placeholder entry (VIC_FEATURE_PTP at position 3) is added to keep the enum in sync with firmware's feature numbering. Suggested-by: Breno Leitao Signed-off-by: Satish Kharat --- drivers/net/ethernet/cisco/enic/enic_main.c | 21 ++++++++++++++++++++- drivers/net/ethernet/cisco/enic/vnic_devcmd.h | 2 ++ 2 files changed, 22 insertions(+), 1 deletion(-) diff --git a/drivers/net/ethernet/cisco/enic/enic_main.c b/drivers/net/ethe= rnet/cisco/enic/enic_main.c index e7125b818087..53d68272d06a 100644 --- a/drivers/net/ethernet/cisco/enic/enic_main.c +++ b/drivers/net/ethernet/cisco/enic/enic_main.c @@ -2641,8 +2641,10 @@ static void enic_iounmap(struct enic *enic) static void enic_sriov_detect_vf_type(struct enic *enic) { struct pci_dev *pdev =3D enic->pdev; - int pos; + u64 supported_versions, a1 =3D 0; u16 vf_dev_id; + int pos; + int err; =20 if (enic_is_sriov_vf(enic) || enic_is_dynamic(enic)) return; @@ -2669,6 +2671,23 @@ static void enic_sriov_detect_vf_type(struct enic *e= nic) enic->vf_type =3D ENIC_VF_TYPE_NONE; break; } + + if (enic->vf_type !=3D ENIC_VF_TYPE_V2) + return; + + /* A successful command means firmware recognizes + * VIC_FEATURE_SRIOV; supported_versions is available + * for sub-feature versioning in the future. + */ + err =3D vnic_dev_get_supported_feature_ver(enic->vdev, + VIC_FEATURE_SRIOV, + &supported_versions, + &a1); + if (err) { + dev_warn(&pdev->dev, + "SR-IOV V2 not supported by current firmware. Upgrade to VIC FW 5.3(4.= 72) or higher.\n"); + enic->vf_type =3D ENIC_VF_TYPE_NONE; + } } #endif =20 diff --git a/drivers/net/ethernet/cisco/enic/vnic_devcmd.h b/drivers/net/et= hernet/cisco/enic/vnic_devcmd.h index 605ef17f967e..7a4bce736105 100644 --- a/drivers/net/ethernet/cisco/enic/vnic_devcmd.h +++ b/drivers/net/ethernet/cisco/enic/vnic_devcmd.h @@ -734,6 +734,8 @@ enum vic_feature_t { VIC_FEATURE_VXLAN, VIC_FEATURE_RDMA, VIC_FEATURE_VXLAN_PATCH, + VIC_FEATURE_PTP, + VIC_FEATURE_SRIOV, VIC_FEATURE_MAX, }; =20 --=20 2.43.0 From nobody Sun Jun 14 04:21:27 2026 Received: from alln-iport-4.cisco.com (alln-iport-4.cisco.com [173.37.142.91]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2115A25487C; Sun, 3 May 2026 11:23:20 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=173.37.142.91 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777807403; cv=none; b=SxSmo9tCtc0ol3N7D1Rkaek4lqoBZ4gHIeADrtV4SLIMgLe0Jzd/Ou/0g5f1lbNf5AmQXZ7vGQDlV6JiheOo0IikxX2HoJXdG7YbR1LLUL1hw22cEGMLRtE67RKsIYHieLcDljsov1TZ4OaojomtW/6LMl9DSDNk+RK/7muL3AE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777807403; c=relaxed/simple; bh=whURRJxU0a7V8S0T7aAuNL/DTzxt+Nmk3HDi4L19IlA=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=HZDWsgYQ3vs020e64U56uY9q2EkqKfIqQ4LFqSeUtHKaDs53FkVY3RtArufrsy1HREamh0agSuBqWqOKSX5eOswxUoYdVJV1ubAaEnQk1Yu+zptjfzbB/L/GIPspq3cGSHtPbl4G5oGNzJ5A56nJEy5a1YWNm3JOZ+xROZTaGrc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=cisco.com; spf=pass smtp.mailfrom=cisco.com; dkim=pass (2048-bit key) header.d=cisco.com header.i=@cisco.com header.b=c6x8p1j0; arc=none smtp.client-ip=173.37.142.91 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=cisco.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=cisco.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=cisco.com header.i=@cisco.com header.b="c6x8p1j0" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cisco.com; i=@cisco.com; l=9070; q=dns/txt; s=iport01; t=1777807401; x=1779017001; h=from:date:subject:mime-version:content-transfer-encoding: message-id:references:in-reply-to:to:cc; bh=NtiLDbbYQdXvwPLn9P/TF+is/CjPXYfAuDYX1dZwcLU=; b=c6x8p1j0rG4zRLKFzWKf9yXatFgq2mKzUUce0NG6VO/ic4ru7Xh+AEF7 71+rQM+hgzo1yo6w2JgG3GLOmKGUjAzXi7IcmhebFlG0zU8syvlUCcWsu CZcVHsVrmlqwKm/+V4i/zvSZ6dGp3bq/LnKE+M0P3J/9F+tGteeZG7XPE oMsiajB5DSpN85GBxgzdIKLKXUT2L/P1TPrmDS8qQ8GvlaPZ8BkalBouW k9tDq/7r2FpRw8fFbOlIrdZvbWwKSyfvPNQZhdaQdOml/1aJTf3m77A1p gDd2ruBRXfXKjS5+gXcrEgaAjzXzujTNW0CDT2R2SiH6CMfom81zEK1ks A==; X-CSE-ConnectionGUID: +DW/7vbqRNy1ftqyyom+kQ== X-CSE-MsgGUID: TfsN5muGQxSVn8pomjcfpQ== X-IPAS-Result: =?us-ascii?q?A0AaAADHL/dp/4v/Ja1aHAEBAQEBAQcBARIBAQQEAQGBf?= =?us-ascii?q?AcBAQsBglaBT0NJhFeIHIc3giGeHRSBaw8BAQEPUQQBAYUGAo0xAiY0CQ4BA?= =?us-ascii?q?gQBAQEBAwIDAQEBAQEBAQEBAQELAQEFAQEBAgEHBYEOE4ZchlsCAQMjBFIQJ?= =?us-ascii?q?QImAgJGEAYBEoMCgnQDtB56fzOBAYR92UOBZAELFAGBCi4BhXGCZgGBcIQGO?= =?us-ascii?q?4Q/JxuBSUSBFAGDaIQFAQQVg3+CaQSDHBSBfo0dCT+BAhwDWSwBVRMNCgsHB?= =?us-ascii?q?WFSMwMgCgsSEhgVAhQvDwQWMh1wDCcSLBc0WBsHBYFLgwBtFDEEIYEFhGB4I?= =?us-ascii?q?ywDTnIDCxgNSBEsNwYOGwQ9AW4HimYdD4EoF3FZNQFNb4EhHCySYAiDdo1da?= =?us-ascii?q?4E1n1mEJoFkoA4BMpdmkwWZBiKjSW6EaIFoPIFZMxoIGxWDIlMZD44qAxbJX?= =?us-ascii?q?SYyPQEBBwIHDQMLgWiQABEXgVUBAQ?= IronPort-Data: A9a23:G9SGmaisBUZUwtoQHYkhVVwFX161oBEKZh0ujC45NGQN5FlHY01je htvDTuPO6yJamqmLdtxPoS/pEoH7MLRnd4wTVM6ry5nESpjpJueD7x1DKtf0wB+jyHnZBg6h ynLQoCYdKjYdleF+FH1dOOn9SUgvU2xbuKUIPbePSxsThNTRi4kiBZy88Y0mYcAbeKRW2thg vus5ZeEULOZ82QsaDxMtvrY8EoHUMna4Vv0gHRvPZing3eG/5UlJMp3Db28KXL+Xr5VEoaSL 87fzKu093/u5BwkDNWoiN7TKiXmlZaLYGBiIlIPM0STqkAqSh4ai87XB9JAAatjsAhlqvgqo Dl7WTNcfi9yVkHEsLx1vxC1iEiSN4UekFPMCSDXXcB+UyQqflO0q8iCAn3aMqVb1tkwXn9E7 MY+OWkDcgmSluWzg7GCH7wEasQLdKEHPasFsX1miDWcBvE8TNWbGePB5MRT23E7gcUm8fT2P pVCL2EwKk6dPlsWYQZ/5JEWxI9EglHkeiFZs1KWpYI84nPYy0p6172F3N/9JYfTGJsMxRnGz o7A11j3GTdHMcGw8Cq+zHT8nujKnyj1QLtHQdVU8dYv2jV/3Fc7BBQIWF6TrfCnh0u6XNxDb UoZ5kIGoKQv8UW5Q8XVUBq/r3qJ+BUbXrJ4EPAw4SmOx7DS7gLfAXILJhZIbtA8udB1QzE22 lKXt9f0Azopu739YWqU/LqSrBuoNCQVJHNEbigBJSMB4t/+sMQwgwjJQ9JLDqG4lJv2FCv2z jTMqzIx750VgNIH2ruT41/KmXSvq4LPQwpz4R/YNl9J9Stjb4KjIojt4l/B4LMYdsCST0KKu z4PnM32AP0yMKxhXRelGI0ldIxFLd7fWNEAqTaDx6Ucygk= IronPort-HdrOrdr: A9a23:cLH1eaoQ2lX8sOnWHUxLChQaV5rdeYIsimQD101hICG9vPbo8/ xG+8576faUslgssR4b9OxoVJPrfZqYz+8W3WBzB9iftXfd2FdARbsKheDfKn/bak7DH4VmuZ uIHZIObeHYPBxWgdn35Q+gH81l4tWWmZrY/Nv2/jNBQR5nbqd44xw8MAaUFUVqWBJLbKBJcq Z0nvA3wAaISDAwctmxAGUDUqzoot3Gk4+jXDs9biRXkDVnSViTmcTH+9/y5GZkbw9y X-Talos-CUID: =?us-ascii?q?9a23=3AVbSyNGuU3jti5NIuNv3y/Yty6ItmXkKak2naOnW?= =?us-ascii?q?eGD9nVKSEWFiVu6Bdxp8=3D?= X-Talos-MUID: 9a23:y89CTApII/PtToXr8TEezw1sL+E15rmBMVA2r5NWo86BNyd+Iw7I2Q== X-IronPort-Anti-Spam-Filtered: true X-IronPort-AV: E=Sophos;i="6.23,213,1770595200"; d="scan'208";a="731561376" Received: from rcdn-l-core-02.cisco.com ([173.37.255.139]) by alln-iport-4.cisco.com with ESMTP/TLS/TLS_AES_256_GCM_SHA384; 03 May 2026 11:22:57 +0000 Received: from [127.0.0.1] (unknown [10.25.128.216]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by rcdn-l-core-02.cisco.com (Postfix) with ESMTPS id BC413180007D0; Sun, 3 May 2026 11:22:56 +0000 (GMT) From: Satish Kharat Date: Sun, 03 May 2026 04:22:39 -0700 Subject: [PATCH net-next v6 02/10] enic: add admin channel open and close for SR-IOV Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260503-enic-sriov-v2-admin-channel-v2-v6-2-0af4fbc2d86d@cisco.com> References: <20260503-enic-sriov-v2-admin-channel-v2-v6-0-0af4fbc2d86d@cisco.com> In-Reply-To: <20260503-enic-sriov-v2-admin-channel-v2-v6-0-0af4fbc2d86d@cisco.com> To: Andrew Lunn , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni Cc: netdev@vger.kernel.org, linux-kernel@vger.kernel.org, Sesidhar Baddela , Satish Kharat X-Mailer: b4 0.15.2 X-Developer-Signature: v=1; a=ed25519-sha256; t=1777807374; l=9322; i=satishkh@cisco.com; h=from:subject:message-id; bh=whURRJxU0a7V8S0T7aAuNL/DTzxt+Nmk3HDi4L19IlA=; b=SGuZBHcp7YxalRWKHT1RLxe+cYKlCMg/ntszdrVueOupiipSBzBEH14hDC+WEFhqCU4b/WIDd w1VXMuM2fAOCoLx+iH4O3qlYVO1tuvPl/SQ62ZtB15rvPtDh2AIzYCW X-Developer-Key: i=satishkh@cisco.com; a=ed25519; pk=4BxbLoCGvHeIMH/PbpiNVx8/2IcTjFvqcr0AD8K6WLY= X-Outbound-Client-TLS: ANONYMOUS;unknown [10.25.128.216];TLSv1.3;TLS_AES_256_GCM_SHA384;256 X-Outbound-SMTP-Client: 10.25.128.216, [10.25.128.216] X-Outbound-Node: rcdn-l-core-02.cisco.com The V2 SR-IOV design uses a dedicated admin channel (WQ/RQ/CQ/INTR on separate BAR resources) for PF-VF mailbox communication rather than firmware-proxied devcmds. Introduce enic_admin_channel_open() and enic_admin_channel_close(). Open allocates and initialises the admin WQ, RQ, two CQs (one per direction) and one SR-IOV interrupt, then issues CMD_QP_TYPE_SET to tell firmware the queues are admin-type. Close reverses the sequence. enic_admin_wq_buf_clean() unmaps and frees any WQ buffers still held at close time, fixing a DMA mapping leak when a send times out. Add CMD_QP_TYPE_SET (97) and QP_TYPE_ADMIN/DATA defines to vnic_devcmd.h. Signed-off-by: Satish Kharat --- drivers/net/ethernet/cisco/enic/Makefile | 3 +- drivers/net/ethernet/cisco/enic/enic_admin.c | 207 ++++++++++++++++++++++= ++++ drivers/net/ethernet/cisco/enic/enic_admin.h | 15 ++ drivers/net/ethernet/cisco/enic/vnic_devcmd.h | 9 ++ 4 files changed, 233 insertions(+), 1 deletion(-) diff --git a/drivers/net/ethernet/cisco/enic/Makefile b/drivers/net/etherne= t/cisco/enic/Makefile index a96b8332e6e2..7ae72fefc99a 100644 --- a/drivers/net/ethernet/cisco/enic/Makefile +++ b/drivers/net/ethernet/cisco/enic/Makefile @@ -3,5 +3,6 @@ obj-$(CONFIG_ENIC) :=3D enic.o =20 enic-y :=3D enic_main.o vnic_cq.o vnic_intr.o vnic_wq.o \ enic_res.o enic_dev.o enic_pp.o vnic_dev.o vnic_rq.o vnic_vic.o \ - enic_ethtool.o enic_api.o enic_clsf.o enic_rq.o enic_wq.o + enic_ethtool.o enic_api.o enic_clsf.o enic_rq.o enic_wq.o \ + enic_admin.o =20 diff --git a/drivers/net/ethernet/cisco/enic/enic_admin.c b/drivers/net/eth= ernet/cisco/enic/enic_admin.c new file mode 100644 index 000000000000..18ca23eef216 --- /dev/null +++ b/drivers/net/ethernet/cisco/enic/enic_admin.c @@ -0,0 +1,207 @@ +// SPDX-License-Identifier: GPL-2.0-only +// Copyright 2025 Cisco Systems, Inc. All rights reserved. + +#include +#include + +#include "vnic_dev.h" +#include "vnic_wq.h" +#include "vnic_rq.h" +#include "vnic_cq.h" +#include "vnic_intr.h" +#include "vnic_resource.h" +#include "vnic_devcmd.h" +#include "enic.h" +#include "enic_admin.h" +#include "cq_desc.h" +#include "wq_enet_desc.h" +#include "rq_enet_desc.h" + +/* Clean up any admin WQ buffers still held by hardware at close time. + * Normally buffers are freed inline after send completion, but a timed-out + * send intentionally leaves the buffer live until the queue is stopped. + */ +static void enic_admin_wq_buf_clean(struct vnic_wq *wq, + struct vnic_wq_buf *buf) +{ + struct enic *enic =3D vnic_dev_priv(wq->vdev); + + if (buf->os_buf) { + dma_unmap_single(&enic->pdev->dev, buf->dma_addr, + buf->len, DMA_TO_DEVICE); + kfree(buf->os_buf); + buf->os_buf =3D NULL; + } +} + +/* No-op: admin RQ buffer teardown is handled in enic_admin_channel_close = */ +static void enic_admin_rq_buf_clean(struct vnic_rq *rq, + struct vnic_rq_buf *buf) +{ +} + +static int enic_admin_qp_type_set(struct enic *enic, u32 enable) +{ + u64 a0 =3D QP_TYPE_ADMIN, a1 =3D enable; + int wait =3D 1000; + int err; + + spin_lock_bh(&enic->devcmd_lock); + err =3D vnic_dev_cmd(enic->vdev, CMD_QP_TYPE_SET, &a0, &a1, wait); + spin_unlock_bh(&enic->devcmd_lock); + + return err; +} + +static int enic_admin_alloc_resources(struct enic *enic) +{ + int err; + + err =3D vnic_wq_alloc_with_type(enic->vdev, &enic->admin_wq, 0, + ENIC_ADMIN_DESC_COUNT, + sizeof(struct wq_enet_desc), + RES_TYPE_ADMIN_WQ); + if (err) + return err; + + err =3D vnic_rq_alloc_with_type(enic->vdev, &enic->admin_rq, 0, + ENIC_ADMIN_DESC_COUNT, + sizeof(struct rq_enet_desc), + RES_TYPE_ADMIN_RQ); + if (err) + goto free_wq; + + /* admin_cq[0] is the WQ completion queue. WQ CQEs are always + * 16 bytes wide; firmware always writes 16-byte CQEs for WQ + * completions on every WQ, including the admin channel WQ. + * Use sizeof(struct cq_desc) accordingly. + */ + err =3D vnic_cq_alloc_with_type(enic->vdev, &enic->admin_cq[0], 0, + ENIC_ADMIN_DESC_COUNT, + sizeof(struct cq_desc), + RES_TYPE_ADMIN_CQ); + if (err) + goto free_rq; + + /* admin_cq[1] is the RQ completion queue. Its descriptor size + * must match what firmware writes. enic_ext_cq() called earlier + * in probe issues CMD_CQ_ENTRY_SIZE_SET for VNIC_RQ_ALL, + * programming firmware to write CQ entries of (16 << enic->ext_cq) + * bytes for every RQ CQ on the vNIC, including the admin RQ CQ. + * Allocating with the same size keeps the host poller and + * firmware in lockstep: + * + * - The color/valid bit lives at byte (desc_size - 1) of every + * cq_enet_rq_desc[_32|_64] variant, so enic_admin_cq_color() + * reads it from the correct offset. + * - Only the first 15 bytes of the descriptor (vlan, + * bytes_written_flags, ...) are accessed by the admin path; + * these fields are identical across all three variants (see + * comment in enic_rq.c above cq_enet_rq_desc_dec()). + */ + err =3D vnic_cq_alloc_with_type(enic->vdev, &enic->admin_cq[1], 1, + ENIC_ADMIN_DESC_COUNT, + 16 << enic->ext_cq, + RES_TYPE_ADMIN_CQ); + if (err) + goto free_cq0; + + /* PFs have dedicated SRIOV_INTR resources for admin channel. + * VFs lack SRIOV_INTR; use a regular INTR_CTRL slot instead. + */ + if (vnic_dev_get_res_count(enic->vdev, RES_TYPE_SRIOV_INTR) >=3D 1) + err =3D vnic_intr_alloc_with_type(enic->vdev, + &enic->admin_intr, 0, + RES_TYPE_SRIOV_INTR); + else + err =3D vnic_intr_alloc(enic->vdev, &enic->admin_intr, + enic->intr_count); + if (err) + goto free_cq1; + + return 0; + +free_cq1: + vnic_cq_free(&enic->admin_cq[1]); +free_cq0: + vnic_cq_free(&enic->admin_cq[0]); +free_rq: + vnic_rq_free(&enic->admin_rq); +free_wq: + vnic_wq_free(&enic->admin_wq); + return err; +} + +static void enic_admin_free_resources(struct enic *enic) +{ + vnic_intr_free(&enic->admin_intr); + vnic_cq_free(&enic->admin_cq[1]); + vnic_cq_free(&enic->admin_cq[0]); + vnic_rq_free(&enic->admin_rq); + vnic_wq_free(&enic->admin_wq); +} + +static void enic_admin_init_resources(struct enic *enic) +{ + vnic_wq_init(&enic->admin_wq, 0, 0, 0); + vnic_rq_init(&enic->admin_rq, 1, 0, 0); + vnic_cq_init(&enic->admin_cq[0], 0, 1, 0, 0, 1, 0, 1, 0, 0, 0); + vnic_cq_init(&enic->admin_cq[1], 0, 1, 0, 0, 1, 0, 1, 0, 0, 0); + vnic_intr_init(&enic->admin_intr, 0, 0, 1); +} + +int enic_admin_channel_open(struct enic *enic) +{ + int err; + + if (!enic->has_admin_channel) + return -ENODEV; + + err =3D enic_admin_alloc_resources(enic); + if (err) { + netdev_err(enic->netdev, + "Failed to alloc admin channel resources: %d\n", + err); + return err; + } + + enic_admin_init_resources(enic); + + vnic_wq_enable(&enic->admin_wq); + vnic_rq_enable(&enic->admin_rq); + + err =3D enic_admin_qp_type_set(enic, 1); + if (err) { + netdev_err(enic->netdev, + "Failed to set admin QP type: %d\n", err); + goto disable_queues; + } + + return 0; + +disable_queues: + enic_admin_qp_type_set(enic, 0); + vnic_wq_disable(&enic->admin_wq); + vnic_rq_disable(&enic->admin_rq); + enic_admin_free_resources(enic); + return err; +} + +void enic_admin_channel_close(struct enic *enic) +{ + if (!enic->has_admin_channel) + return; + + enic_admin_qp_type_set(enic, 0); + + vnic_wq_disable(&enic->admin_wq); + vnic_rq_disable(&enic->admin_rq); + + vnic_wq_clean(&enic->admin_wq, enic_admin_wq_buf_clean); + vnic_rq_clean(&enic->admin_rq, enic_admin_rq_buf_clean); + vnic_cq_clean(&enic->admin_cq[0]); + vnic_cq_clean(&enic->admin_cq[1]); + vnic_intr_clean(&enic->admin_intr); + + enic_admin_free_resources(enic); +} diff --git a/drivers/net/ethernet/cisco/enic/enic_admin.h b/drivers/net/eth= ernet/cisco/enic/enic_admin.h new file mode 100644 index 000000000000..569aadeb9312 --- /dev/null +++ b/drivers/net/ethernet/cisco/enic/enic_admin.h @@ -0,0 +1,15 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* Copyright 2025 Cisco Systems, Inc. All rights reserved. */ + +#ifndef _ENIC_ADMIN_H_ +#define _ENIC_ADMIN_H_ + +#define ENIC_ADMIN_DESC_COUNT 64 +#define ENIC_ADMIN_BUF_SIZE 2048 + +struct enic; + +int enic_admin_channel_open(struct enic *enic); +void enic_admin_channel_close(struct enic *enic); + +#endif /* _ENIC_ADMIN_H_ */ diff --git a/drivers/net/ethernet/cisco/enic/vnic_devcmd.h b/drivers/net/et= hernet/cisco/enic/vnic_devcmd.h index 7a4bce736105..a1c8f522c7d7 100644 --- a/drivers/net/ethernet/cisco/enic/vnic_devcmd.h +++ b/drivers/net/ethernet/cisco/enic/vnic_devcmd.h @@ -455,8 +455,17 @@ enum vnic_devcmd_cmd { */ CMD_CQ_ENTRY_SIZE_SET =3D _CMDC(_CMD_DIR_WRITE, _CMD_VTYPE_ENET, 90), =20 + /* + * Set queue pair type (admin or data) + * in: (u32) a0 =3D queue pair type (0 =3D admin, 1 =3D data) + * in: (u32) a1 =3D enable (1) / disable (0) + */ + CMD_QP_TYPE_SET =3D _CMDC(_CMD_DIR_WRITE, _CMD_VTYPE_ENET, 97), }; =20 +#define QP_TYPE_ADMIN 0 +#define QP_TYPE_DATA 1 + /* CMD_ENABLE2 flags */ #define CMD_ENABLE2_STANDBY 0x0 #define CMD_ENABLE2_ACTIVE 0x1 --=20 2.43.0 From nobody Sun Jun 14 04:21:27 2026 Received: from alln-iport-2.cisco.com (alln-iport-2.cisco.com [173.37.142.89]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 694A4319858; Sun, 3 May 2026 11:24:06 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=173.37.142.89 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777807448; cv=none; b=NjM2JqVYMK5zzvVBE4aOKw+uqRTa+WobeIYNovHgAUl5KggV8cpCu6MXdj2bjEiA2FxHKVFBMTdt0WRSByPtk+RG0Zl3bYLKt433n2jFbsla84MrygwntYSMLfoRqLaQCqoN7VqKon9n/rhOoUff1JmRZvzDzmfh65D4ZiCLJf0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777807448; c=relaxed/simple; bh=HFv8BwRk8a1zRxQAOLHI7cyn4arfkWptfVfK5vRBAYk=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=hjUcJxJfDdDm6BLfa0PYUbrjMfUGYEScCvy1bnyuRj6vpedxvO0WxhdFH0bzkspay2MBb0mrP5v2cSK3lWvdz4b5NvIBPY8DA1V52RNKOoME2fjAmforRllow7hEXWpN1FEjBtB+TUrWJ6QridGzSucFjmIfj7Ej2IDVpxCWyPI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=cisco.com; spf=pass smtp.mailfrom=cisco.com; dkim=pass (2048-bit key) header.d=cisco.com header.i=@cisco.com header.b=E52hNTt/; arc=none smtp.client-ip=173.37.142.89 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=cisco.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=cisco.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=cisco.com header.i=@cisco.com header.b="E52hNTt/" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cisco.com; i=@cisco.com; l=3582; q=dns/txt; s=iport01; t=1777807446; x=1779017046; h=from:date:subject:mime-version:content-transfer-encoding: message-id:references:in-reply-to:to:cc; bh=AylClhca1hYmZwQjZmNt6F/JHBlnW0187dEO2weuJ9w=; b=E52hNTt/CLb4db58KYC7dSICfADko9x26OudQ8pld+d8kDs+BryIohId VO5IwBgQ7Ob8NRkOmkcPE3IhYvrS+SXCpxCbr8gOZ0N4xbGKuBpzKzi8+ 4q/Fi43q1EGdqs39XUR/0FIjOjXfwIZyeRiQDs1MecWSThfUazpn1WfEN QYiYiW/Cp0W2L4A4dx40ZPbbDS6wh0oOKUuBk/I28pMYfWMX7d/Lbey5z 08judmVqvn/geTYWdfDbSA3Js86+VLYnYxEW9a8/l6IRcwKbV6NNZtn/U aGeLtnEBVWQmaGyMzRs8DoCd4KG+qtBMW0sNIqYbCEHkendhfj5Qgq1rm w==; X-CSE-ConnectionGUID: X4Tg8XpPQJazjlRERupcag== X-CSE-MsgGUID: D9boPkSoTsCAql4CoEH27A== X-IPAS-Result: =?us-ascii?q?A0AnAADQLvdp/4v/Ja1aHQEBAQEJARIBBQUBgXwIAQsBg?= =?us-ascii?q?laBT0NJhFeIHIc3giGeHYF/DwEBAQ9RBAEBhQYCjTECJjQJDgECBAEBAQEDA?= =?us-ascii?q?gMBAQEBAQEBAQEBAQsBAQUBAQECAQcFgQ4ThlyGWwIBAyMEUhAlAiYCAkYQB?= =?us-ascii?q?gESgwKCdAO0Hnp/M4EBhH3ZQ4FkAQsUAYEKLgGFcYJmAYFwhAY7hD8nG4FJR?= =?us-ascii?q?IEVg2iBGYMGg3+CaQSDHBSPGwk/gQIcA1ksAVUTDQoLBwVhUjMDIAoLEhIYF?= =?us-ascii?q?QIULw8EFjIdcAwnEiwXNFgbBwWBS4MAbRQxBCGBBYRgeCMsA05yAwsYDUgRL?= =?us-ascii?q?DcGDhsEPQFuB4pmHQ+BP3GBDhSBKSM0k3IIFYNhjV1rgTWfWYQmgWSgDgEyl?= =?us-ascii?q?2aTBZkGIqQ3hGiBaDyBWTMaCBsVgyJTGQ+OKgMWyVgmMj4BBwIHDgKBc5ARg?= =?us-ascii?q?WwBAQ?= IronPort-Data: A9a23:zD9zWq34o3Hl9GmvYvbD5RNwkn2cJEfYwER7XKvMYLTBsI5bpzwHx mVJWGuBb62ONGb0e94katzj9UxTu5HRyYVhGlNv3Hw8FHgiRegpqji6wuYcGwvIc6UvmWo+t 512huHodZ5yFjmH4E/xbtANlFEkvYmQXL3wFeXYDS54QA5gWU8JhAlq8wIDqtYAbeORXUXX4 rsen+WFYAX7g2MsaTpNg06+gEoHUMra6WtwUmMWPZinjHeG/1EJAZQWI72GLneQauF8Au6gS u/f+6qy92Xf8g1FIovNfmHTKxBirhb6ZGBiu1IOM0SQqkEqSh8ajs7XAMEhhXJ/0F1lqTzeJ OJl7vRcQS9xVkHFdX90vxNwS0mSNoUekFPLzOTWXcG7lyX7n3XQL/pGPB0KMt0o4t5OI2hc9 /giBQoQdhuKiLfjqF67YrEEasULNsLnOsYb/3pn1zycVaxgSpHYSKKM7thdtNsyrpkRRrCFO IxDNGcpNU+RC/FMEg9/5JYWh+60h2L/cD1wo1OOrq1x6G/WpOB0+Oe3YIuKIYDXGK25mG609 2jBx2fCBS0BH/WGzWau9V+M3LDmyHaTtIU6UefQGuRRqFGa2GEWIBEfS1a+pf63lgi4Vs43A 0EY5y8nv6Uu3EOsSdb5Uluzp3vslh0BUdN4EOAg7gyJjK3O7G6xD24IVDNeK9orqMM7WxQuy 1mP2djpbRR3ubeYT3O17LqYrTqufyMSKAcqYS4CUBtA4NT5pow3pgzAQ8wlE6OviNDxXzbqz FiiqiUii7gNpdAE2r/9/l3dhT+o4J/TQWYICh7/RGmp6EZ9IYWifYHtsQKd5vdbJ4HfRV6E1 JQZp/WjACk1JcnlvESwrC8lRdlFO97t3OXgvGNS IronPort-HdrOrdr: A9a23:srwi1ayHhdoU+L+ZnCH+KrPw2r1zdoMgy1knxilNoNJuA6ilfq eV/cjztCWbtN9/YhwdcLy7UpVoBEmsl6KdgrN/AV7dZmnbUQKTRekI0WKh+UyCJ8SUzIJgPM lbAs1DIey1IFRkgcP9+QWiV/Qt0LC8gdmVrNab4ntzQQRtcq16qz14BAGSD1FsSGB9dPwE/e Knl616jgvlX2gLZcKmAXRAde3Cq9rX0K/CW3c9dmYaAM3kt0LT1FY8eCLopSsjbw== X-Talos-CUID: 9a23:g5FMFm3iS7PDbGqOJOnSabxfKpwZfS3t1E/pAUaqC2JTZYeIUVSZ9/Yx X-Talos-MUID: =?us-ascii?q?9a23=3AzTJ8cw3XaHH3EgHvUUmS/kizHDUjz/WkD3gOqpI?= =?us-ascii?q?6oOKtLCgsEDCwrG7oa9py?= X-IronPort-Anti-Spam-Filtered: true X-IronPort-AV: E=Sophos;i="6.23,213,1770595200"; d="scan'208";a="728973455" Received: from rcdn-l-core-02.cisco.com ([173.37.255.139]) by alln-iport-2.cisco.com with ESMTP/TLS/TLS_AES_256_GCM_SHA384; 03 May 2026 11:22:58 +0000 Received: from [127.0.0.1] (unknown [10.25.128.216]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by rcdn-l-core-02.cisco.com (Postfix) with ESMTPS id 7C27E180007C2; Sun, 3 May 2026 11:22:57 +0000 (GMT) From: Satish Kharat Date: Sun, 03 May 2026 04:22:40 -0700 Subject: [PATCH net-next v6 03/10] enic: add admin RQ buffer management Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260503-enic-sriov-v2-admin-channel-v2-v6-3-0af4fbc2d86d@cisco.com> References: <20260503-enic-sriov-v2-admin-channel-v2-v6-0-0af4fbc2d86d@cisco.com> In-Reply-To: <20260503-enic-sriov-v2-admin-channel-v2-v6-0-0af4fbc2d86d@cisco.com> To: Andrew Lunn , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni Cc: netdev@vger.kernel.org, linux-kernel@vger.kernel.org, Sesidhar Baddela , Satish Kharat X-Mailer: b4 0.15.2 X-Developer-Signature: v=1; a=ed25519-sha256; t=1777807374; l=3640; i=satishkh@cisco.com; h=from:subject:message-id; bh=HFv8BwRk8a1zRxQAOLHI7cyn4arfkWptfVfK5vRBAYk=; b=6T8tyXSKx38GAPZt16KHhxMky//wn2+esaq4L8xXikElVhx2fgX8L82+RGJKy9whpsXXhb5vP GDtipo6JOktAMfY1poqr9b9t2ByyIQo95+XGm9yv9Q3l/UcBlTGImED X-Developer-Key: i=satishkh@cisco.com; a=ed25519; pk=4BxbLoCGvHeIMH/PbpiNVx8/2IcTjFvqcr0AD8K6WLY= X-Outbound-Client-TLS: ANONYMOUS;unknown [10.25.128.216];TLSv1.3;TLS_AES_256_GCM_SHA384;256 X-Outbound-SMTP-Client: 10.25.128.216, [10.25.128.216] X-Outbound-Node: rcdn-l-core-02.cisco.com The admin receive queue needs pre-posted DMA buffers for incoming mailbox messages from VFs. Each buffer is a kmalloc'd region mapped for DMA (2048 bytes, sufficient for any MBOX message). Add enic_admin_rq_fill() to post buffers at open time, and enic_admin_rq_drain() to unmap and free them at close time. Wire both into the admin channel open/close paths. Signed-off-by: Satish Kharat --- drivers/net/ethernet/cisco/enic/enic_admin.c | 66 ++++++++++++++++++++++++= +++- 1 file changed, 64 insertions(+), 2 deletions(-) diff --git a/drivers/net/ethernet/cisco/enic/enic_admin.c b/drivers/net/eth= ernet/cisco/enic/enic_admin.c index 18ca23eef216..9e5061ad3087 100644 --- a/drivers/net/ethernet/cisco/enic/enic_admin.c +++ b/drivers/net/ethernet/cisco/enic/enic_admin.c @@ -3,6 +3,7 @@ =20 #include #include +#include =20 #include "vnic_dev.h" #include "vnic_wq.h" @@ -34,10 +35,63 @@ static void enic_admin_wq_buf_clean(struct vnic_wq *wq, } } =20 -/* No-op: admin RQ buffer teardown is handled in enic_admin_channel_close = */ static void enic_admin_rq_buf_clean(struct vnic_rq *rq, struct vnic_rq_buf *buf) { + struct enic *enic =3D vnic_dev_priv(rq->vdev); + + if (!buf->os_buf) + return; + + dma_unmap_single(&enic->pdev->dev, buf->dma_addr, buf->len, + DMA_FROM_DEVICE); + kfree(buf->os_buf); + buf->os_buf =3D NULL; +} + +static int enic_admin_rq_post_one(struct enic *enic) +{ + struct vnic_rq *rq =3D &enic->admin_rq; + struct rq_enet_desc *desc; + dma_addr_t dma_addr; + void *buf; + + buf =3D kmalloc(ENIC_ADMIN_BUF_SIZE, GFP_KERNEL); + if (!buf) + return -ENOMEM; + + dma_addr =3D dma_map_single(&enic->pdev->dev, buf, ENIC_ADMIN_BUF_SIZE, + DMA_FROM_DEVICE); + if (dma_mapping_error(&enic->pdev->dev, dma_addr)) { + kfree(buf); + return -ENOMEM; + } + + desc =3D vnic_rq_next_desc(rq); + rq_enet_desc_enc(desc, (u64)dma_addr | VNIC_PADDR_TARGET, + RQ_ENET_TYPE_ONLY_SOP, ENIC_ADMIN_BUF_SIZE); + vnic_rq_post(rq, buf, 0, dma_addr, ENIC_ADMIN_BUF_SIZE, 0); + + return 0; +} + +static int enic_admin_rq_fill(struct enic *enic) +{ + struct vnic_rq *rq =3D &enic->admin_rq; + int err; + + while (vnic_rq_desc_avail(rq) > 0) { + err =3D enic_admin_rq_post_one(enic); + if (err) + return err; + } + + return 0; +} + +static void enic_admin_rq_drain(struct enic *enic) +{ + vnic_rq_clean(&enic->admin_rq, enic_admin_rq_buf_clean); } =20 static int enic_admin_qp_type_set(struct enic *enic, u32 enable) @@ -170,6 +224,13 @@ int enic_admin_channel_open(struct enic *enic) vnic_wq_enable(&enic->admin_wq); vnic_rq_enable(&enic->admin_rq); =20 + err =3D enic_admin_rq_fill(enic); + if (err) { + netdev_err(enic->netdev, + "Failed to fill admin RQ buffers: %d\n", err); + goto disable_queues; + } + err =3D enic_admin_qp_type_set(enic, 1); if (err) { netdev_err(enic->netdev, @@ -183,6 +244,7 @@ int enic_admin_channel_open(struct enic *enic) enic_admin_qp_type_set(enic, 0); vnic_wq_disable(&enic->admin_wq); vnic_rq_disable(&enic->admin_rq); + enic_admin_rq_drain(enic); enic_admin_free_resources(enic); return err; } @@ -198,7 +260,7 @@ void enic_admin_channel_close(struct enic *enic) vnic_rq_disable(&enic->admin_rq); =20 vnic_wq_clean(&enic->admin_wq, enic_admin_wq_buf_clean); - vnic_rq_clean(&enic->admin_rq, enic_admin_rq_buf_clean); + enic_admin_rq_drain(enic); vnic_cq_clean(&enic->admin_cq[0]); vnic_cq_clean(&enic->admin_cq[1]); vnic_intr_clean(&enic->admin_intr); --=20 2.43.0 From nobody Sun Jun 14 04:21:27 2026 Received: from alln-iport-2.cisco.com (alln-iport-2.cisco.com [173.37.142.89]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 249F3347BD7; Sun, 3 May 2026 11:24:08 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=173.37.142.89 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777807450; cv=none; b=ZwkzzZNevogdgHEPAxgTtYMUmSMM0aayFvuF4h8P2KHOaJTbvoY8L5oJbktsKwVTSGhKfGnn/vm9cdT0J0dgUH5b4KvoP5yvMg9cNPi00l961kyGyXc8pXAy+X9h9oPj1nx45YUxbRu8hV5NjVl/1871yFhbpyqg254HdNuSbho= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777807450; c=relaxed/simple; bh=9+FmhS7wb08SpsweXzk3jqUnuV+Vwd87EsjztsP8bwU=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=tVD9J5XSOhZBvK6+nyS4N1CeaX6TZGNguKVLwKkPC3gAcbDEUc2C2uL/6RaWDYF2XuSpNm+iXx47FKwitjcO4QHsRH38iiXIxdbNrhertcyXTb86qF5eFUyTyOJRUhuhqIM9RRL0fn+q28GYrfax674q2bfrgVGXGAntk75gieo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=cisco.com; spf=pass smtp.mailfrom=cisco.com; dkim=pass (2048-bit key) header.d=cisco.com header.i=@cisco.com header.b=Lk5+gd27; arc=none smtp.client-ip=173.37.142.89 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=cisco.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=cisco.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=cisco.com header.i=@cisco.com header.b="Lk5+gd27" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cisco.com; i=@cisco.com; l=16039; q=dns/txt; s=iport01; t=1777807448; x=1779017048; h=from:date:subject:mime-version:content-transfer-encoding: message-id:references:in-reply-to:to:cc; bh=viXJdgqop0YmYY0Bpyz9hXgUu8InEsfz+ezp1wkNUa0=; b=Lk5+gd27jSAdzWC0heoLY+VtZ2EJ1ILkYIYGSIFRbjgbWUfUbvIsPVNP WLJJ1wlk+2P28GLCQCs7hSM04QWSdJF+xdi7BUNup93l8AAYkZ1DcSpfG NX3D+wwndv5MoZOkZdYEWOZ5PNcGYBlKo6oTVgJ0cpYKQr69kMIE/9bJe MRRIfZ5TAN7oQiSPfQlC8lONVVZpGt+woYg360CdEDoAHivXezT5w54Ge KEiIhsG6vBGVdq6SXzBpmvaicPsQ80l5jDhK3UJQRWIWqFx9fOTTa35FQ XvU/5JJIZOw4rNgCOWI7Yd6Nq3whQn6dH/sqf0Fg68hvPVWvrveCYA+i4 Q==; X-CSE-ConnectionGUID: GFieJpsHR1WMPiM6NsM5+Q== X-CSE-MsgGUID: nJOl/UP/RLGZfjx7iLsOrw== X-IPAS-Result: =?us-ascii?q?A0ADAADQLvdp/4v/Ja1aGgEBAQEBAQEBAQEDAQEBARIBA?= =?us-ascii?q?QEBAgIBAQEBgXwFAQEBAQsBglaBT0NJhFeIHIc3giGRTYxQgX8PAQEBD1EEA?= =?us-ascii?q?QGFBgKNMQImNAkOAQIEAQEBAQMCAwEBAQEBAQEBAQEBCwEBBQEBAQIBBwWBD?= =?us-ascii?q?hOGXIZbAgEDIwRSECUCJgICRhAGARKDAoJ0A7Qeen8zgQGEfdlDgWQBCxQBg?= =?us-ascii?q?QouAYVxgmYBgXCEBjuEPycbgUlEgRQBg2iEGoQEgmkEgiJ6FIMAgzKBAIdpC?= =?us-ascii?q?T+BAhwDWSwBVRMNCgsHBWFSMwMgCgsSEhgVAhQvDwQWMh1wDCcSLBc0WBsHB?= =?us-ascii?q?YFLgwBtFDEEIYEFhGB4IywDTnIDCxgNSBEsNwYOGwQ9AW4HimYdD4E/cQFZG?= =?us-ascii?q?gcUejYMPAgTHRQZHCcCkmMbAoNhjCeBNmuBNZ9ZhCaBZKAOATKEBJNiM5JSm?= =?us-ascii?q?QYiozM0ARg3hGiBaDyBWTMaCBsVO4JnUxkPjioDFhzJPCYyPQEBBwIHDQMLg?= =?us-ascii?q?WiQAASBeQEB?= IronPort-Data: A9a23:EmLunKhNOK0KXg17HIFAv79JX161oBEKZh0ujC45NGQN5FlHY01je htvDzjQbq2JMGvzKY8kO46+9kpT78SAzNFjGVdrqi88EShjpJueD7x1DKtf0wB+jyHnZBg6h ynLQoCYdKjYdleF+FH1dOOn9SUgvU2xbuKUIPbePSxsThNTRi4kiBZy88Y0mYcAbeKRW2thg vus5ZeEULOZ82QsaDxMtvrY8EoHUMna4Vv0gHRvPZing3eG/5UlJMp3Db28KXL+Xr5VEoaSL 87fzKu093/u5BwkDNWoiN7TKiXmlZaLYGBiIlIPM0STqkAqSh4ai87XB9JAAatjsAhlqvgqo Dl7WTNcfi9yVkHEsLx1vxC1iEiSN4UekFPMCSDXXcB+UyQqflO0q8iCAn3aMqU31Od6I3lV1 cZBFxU1bzXdofmTnfGCH7wEasQLdKEHPasFsX1miDWcBvE8TNWbGePB5MRT23E7gcUm8fT2P pVCL2EwKk6dPlsWYQZ/5JEWxI9EglHkeiFZs1KWpYI84nPYy0p6172F3N/9JYbaFJoOwhvJz o7A12LbJRVdCeTY8x6qwHyulrDOuTjcQp1HQdVU8dYv2jV/3Fc7BBQIWF6TrfCnh0u6XNxDb UoZ5kIGoKQv8UW5Q8XVUBq/r3qJ+BUbXrJ4EPAw4SmOx7DS7gLfAXILJhZIbtA8udB1QzE22 lKXt9f0Azopu739YWqU/LqSrBuoNCQVJHNEbigBJSMB4t/+sMQwgwjJQ9JLDqG4lJv2FCv2z jTMqzIx750VgNIH2ruT41/KmXSvq4LPQwpz4R/YNl9J9Stjb4KjIojt4l/B4LMYdsCST0KKu z4PnM32AP0yMKxhXRelGI0ldIxFLd7fWNEAqTaDx6Ucygk= IronPort-HdrOrdr: A9a23:1PgNR68j6Y00lybJR+luk+AuI+orL9Y04lQ7vn2ZhyYlFvBw8P rOoB1773HJYVkqNU3I9errBEDEewK/yXcX2+Ys1MmZPDUO0VHAROpfBMnZsl7d8kbFmtK1u5 0QFpRWOZncAUd3iMDi7BL9MdoxqeP3iZyAtKPyw2hgSAdyZ7ol1Qd2DQ6HDlZ7LTM2Y6bRbK DsgfZvln6HZWkdaNi9Cz0uWujOoMCOqbfdCCR2fyLOLGK1/EmVAHmQKWni4isj X-Talos-CUID: 9a23:K0MppmPsDxrhg+5Ddwdm+WwMAeAfK0L49Hb1fWmjLDsqcejA X-Talos-MUID: 9a23:P5+6EQoL461Ra/oJcWgezzdAE8Mv3pzpMmQio5sEq5aFETdXISjI2Q== X-IronPort-Anti-Spam-Filtered: true X-IronPort-AV: E=Sophos;i="6.23,213,1770595200"; d="scan'208";a="728973459" Received: from rcdn-l-core-02.cisco.com ([173.37.255.139]) by alln-iport-2.cisco.com with ESMTP/TLS/TLS_AES_256_GCM_SHA384; 03 May 2026 11:22:58 +0000 Received: from [127.0.0.1] (unknown [10.25.128.216]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by rcdn-l-core-02.cisco.com (Postfix) with ESMTPS id 382B3180007C5; Sun, 3 May 2026 11:22:58 +0000 (GMT) From: Satish Kharat Date: Sun, 03 May 2026 04:22:41 -0700 Subject: [PATCH net-next v6 04/10] enic: add admin CQ service with MSI-X interrupt and NAPI polling Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260503-enic-sriov-v2-admin-channel-v2-v6-4-0af4fbc2d86d@cisco.com> References: <20260503-enic-sriov-v2-admin-channel-v2-v6-0-0af4fbc2d86d@cisco.com> In-Reply-To: <20260503-enic-sriov-v2-admin-channel-v2-v6-0-0af4fbc2d86d@cisco.com> To: Andrew Lunn , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni Cc: netdev@vger.kernel.org, linux-kernel@vger.kernel.org, Sesidhar Baddela , Satish Kharat X-Mailer: b4 0.15.2 X-Developer-Signature: v=1; a=ed25519-sha256; t=1777807374; l=16551; i=satishkh@cisco.com; h=from:subject:message-id; bh=9+FmhS7wb08SpsweXzk3jqUnuV+Vwd87EsjztsP8bwU=; b=AbZgUNBRojZC/D6u8ihVGfgwtju6MoH6nEHTvlI/s3EE59l5YkZoPGCOmWeAhVjfENBCkFusW vM994CXMXagAH3aPFOEg7tYNp4Wd3p8Wv7oJzIxIdNUznJym0lYqtJY X-Developer-Key: i=satishkh@cisco.com; a=ed25519; pk=4BxbLoCGvHeIMH/PbpiNVx8/2IcTjFvqcr0AD8K6WLY= X-Outbound-Client-TLS: ANONYMOUS;unknown [10.25.128.216];TLSv1.3;TLS_AES_256_GCM_SHA384;256 X-Outbound-SMTP-Client: 10.25.128.216, [10.25.128.216] X-Outbound-Node: rcdn-l-core-02.cisco.com Add completion queue service for the admin channel WQ and RQ, driven by an MSI-X interrupt and NAPI polling. The receive pipeline is: MSI-X ISR -> NAPI poll -> RQ CQ service -> message enqueue -> workqueue handler -> admin_rq_handler callback. NAPI drains the RQ CQ in softirq context, copying each received buffer into an enic_admin_msg and appending it to a spinlock-protected list. A system workqueue handler then processes each message in process context where sleeping (mutex, GFP_KERNEL allocations) is safe. The WQ CQ service counts transmit completions and is called from the synchronous MBOX send path. Interrupt generation is disabled on the WQ CQ (admin_cq[0]) because it is polled synchronously; only the RQ CQ (admin_cq[1]) is interrupt-driven via NAPI. RQ buffer allocation uses GFP_ATOMIC since enic_admin_rq_fill() is called from NAPI context during CQ processing. Log a rate-limited warning when admin RQ buffer refill fails in NAPI context. The admin channel open/close paths set up and tear down the MSI-X interrupt, NAPI instance, and workqueue. CQ init enables interrupt delivery on the RQ CQ and sets the interrupt offset so completions trigger the admin ISR. The admin interrupt is allocated from the general INTR_CTRL pool (index =3D=3D intr_count) rather than the RES_TYPE_SRIOV_INTR slot, which firmware reserves for its own SR-IOV signaling. Signed-off-by: Satish Kharat --- drivers/net/ethernet/cisco/enic/enic.h | 8 + drivers/net/ethernet/cisco/enic/enic_admin.c | 343 +++++++++++++++++++++++= ++-- drivers/net/ethernet/cisco/enic/enic_admin.h | 12 + 3 files changed, 341 insertions(+), 22 deletions(-) diff --git a/drivers/net/ethernet/cisco/enic/enic.h b/drivers/net/ethernet/= cisco/enic/enic.h index 08472420f3a1..1c09da3c0b1a 100644 --- a/drivers/net/ethernet/cisco/enic/enic.h +++ b/drivers/net/ethernet/cisco/enic/enic.h @@ -296,6 +296,14 @@ struct enic { struct vnic_rq admin_rq; struct vnic_cq admin_cq[2]; struct vnic_intr admin_intr; + struct napi_struct admin_napi; + unsigned int admin_intr_index; + struct work_struct admin_msg_work; + spinlock_t admin_msg_lock; /* protects admin_msg_list */ + struct list_head admin_msg_list; + u64 admin_msg_drop_cnt; + void (*admin_rq_handler)(struct enic *enic, void *buf, + unsigned int len); }; =20 static inline struct net_device *vnic_get_netdev(struct vnic_dev *vdev) diff --git a/drivers/net/ethernet/cisco/enic/enic_admin.c b/drivers/net/eth= ernet/cisco/enic/enic_admin.c index 9e5061ad3087..5445815139b5 100644 --- a/drivers/net/ethernet/cisco/enic/enic_admin.c +++ b/drivers/net/ethernet/cisco/enic/enic_admin.c @@ -4,6 +4,7 @@ #include #include #include +#include =20 #include "vnic_dev.h" #include "vnic_wq.h" @@ -15,6 +16,7 @@ #include "enic.h" #include "enic_admin.h" #include "cq_desc.h" +#include "cq_enet_desc.h" #include "wq_enet_desc.h" #include "rq_enet_desc.h" =20 @@ -49,14 +51,14 @@ static void enic_admin_rq_buf_clean(struct vnic_rq *rq, buf->os_buf =3D NULL; } =20 -static int enic_admin_rq_post_one(struct enic *enic) +static int enic_admin_rq_post_one(struct enic *enic, gfp_t gfp) { struct vnic_rq *rq =3D &enic->admin_rq; struct rq_enet_desc *desc; dma_addr_t dma_addr; void *buf; =20 - buf =3D kmalloc(ENIC_ADMIN_BUF_SIZE, GFP_KERNEL); + buf =3D kmalloc(ENIC_ADMIN_BUF_SIZE, gfp); if (!buf) return -ENOMEM; =20 @@ -75,13 +77,13 @@ static int enic_admin_rq_post_one(struct enic *enic) return 0; } =20 -static int enic_admin_rq_fill(struct enic *enic) +static int enic_admin_rq_fill(struct enic *enic, gfp_t gfp) { struct vnic_rq *rq =3D &enic->admin_rq; int err; =20 while (vnic_rq_desc_avail(rq) > 0) { - err =3D enic_admin_rq_post_one(enic); + err =3D enic_admin_rq_post_one(enic, gfp); if (err) return err; } @@ -94,6 +96,251 @@ static void enic_admin_rq_drain(struct enic *enic) vnic_rq_clean(&enic->admin_rq, enic_admin_rq_buf_clean); } =20 +static unsigned int enic_admin_cq_color(void *cq_desc, unsigned int desc_s= ize) +{ + u8 type_color =3D *((u8 *)cq_desc + desc_size - 1); + + return (type_color >> CQ_DESC_COLOR_SHIFT) & CQ_DESC_COLOR_MASK; +} + +unsigned int enic_admin_wq_cq_service(struct enic *enic) +{ + struct vnic_cq *cq =3D &enic->admin_cq[0]; + unsigned int work =3D 0; + void *desc; + + desc =3D vnic_cq_to_clean(cq); + while (enic_admin_cq_color(desc, cq->ring.desc_size) !=3D + cq->last_color) { + /* Ensure color bit is read before descriptor fields */ + rmb(); + vnic_cq_inc_to_clean(cq); + work++; + desc =3D vnic_cq_to_clean(cq); + } + + return work; +} + +static void enic_admin_msg_enqueue(struct enic *enic, void *buf, + unsigned int len) +{ + struct enic_admin_msg *msg; + + msg =3D kmalloc(struct_size(msg, data, len), GFP_ATOMIC); + if (!msg) { + enic->admin_msg_drop_cnt++; + if (net_ratelimit()) + netdev_warn(enic->netdev, + "admin msg enqueue drop (len=3D%u drops=3D%llu)\n", + len, enic->admin_msg_drop_cnt); + return; + } + + msg->len =3D len; + memcpy(msg->data, buf, len); + + spin_lock(&enic->admin_msg_lock); + list_add_tail(&msg->list, &enic->admin_msg_list); + spin_unlock(&enic->admin_msg_lock); +} + +unsigned int enic_admin_rq_cq_service(struct enic *enic, unsigned int budg= et) +{ + struct vnic_cq *cq =3D &enic->admin_cq[1]; + struct vnic_rq *rq =3D &enic->admin_rq; + struct cq_enet_rq_desc *rq_desc; + struct vnic_rq_buf *buf; + u16 bwf, bytes_written; + unsigned int work =3D 0; + void *desc; + + desc =3D vnic_cq_to_clean(cq); + while (work < budget && + enic_admin_cq_color(desc, cq->ring.desc_size) !=3D + cq->last_color) { + /* Ensure CQ descriptor fields are read after + * the color/valid check. + */ + rmb(); + buf =3D rq->to_clean; + + /* Decode the actual number of bytes hardware wrote into + * the RX buffer. buf->len is the static allocation size + * (ENIC_ADMIN_BUF_SIZE) and would expose uninitialised + * heap memory beyond the real payload. bytes_written_flags + * is at the same offset in every cq_enet_rq_desc[_32|_64] + * variant. + */ + rq_desc =3D desc; + bwf =3D le16_to_cpu(rq_desc->bytes_written_flags); + bytes_written =3D bwf & CQ_ENET_RQ_DESC_BYTES_WRITTEN_MASK; + + dma_sync_single_for_cpu(&enic->pdev->dev, + buf->dma_addr, buf->len, + DMA_FROM_DEVICE); + + /* Drop on hardware error indications. Admin messages + * are internal to the VIC, not received over the wire. + * Firmware sets TRUNCATED when the message does not fit + * in the posted buffer, and FCS_OK is always set on + * healthy admin completions. + */ + if (bwf & CQ_ENET_RQ_DESC_FLAGS_TRUNCATED) { + netdev_warn_once(enic->netdev, + "admin RQ: truncated message dropped\n"); + goto next_desc; + } + if (!(rq_desc->flags & CQ_ENET_RQ_DESC_FLAGS_FCS_OK)) { + netdev_warn_once(enic->netdev, + "admin RQ: bad FCS, dropping message\n"); + goto next_desc; + } + + enic_admin_msg_enqueue(enic, buf->os_buf, bytes_written); + +next_desc: + enic_admin_rq_buf_clean(rq, rq->to_clean); + rq->to_clean =3D rq->to_clean->next; + rq->ring.desc_avail++; + + vnic_cq_inc_to_clean(cq); + work++; + desc =3D vnic_cq_to_clean(cq); + } + + if (enic_admin_rq_fill(enic, GFP_ATOMIC) && net_ratelimit()) + netdev_warn(enic->netdev, "admin RQ refill failed\n"); + + return work; +} + +static irqreturn_t enic_admin_isr_msix(int irq, void *data) +{ + struct napi_struct *napi =3D data; + + napi_schedule_irqoff(napi); + + return IRQ_HANDLED; +} + +static void enic_admin_msg_work_handler(struct work_struct *work) +{ + struct enic *enic =3D container_of(work, struct enic, admin_msg_work); + struct enic_admin_msg *msg, *tmp; + LIST_HEAD(local_list); + + spin_lock_bh(&enic->admin_msg_lock); + list_splice_init(&enic->admin_msg_list, &local_list); + spin_unlock_bh(&enic->admin_msg_lock); + + list_for_each_entry_safe(msg, tmp, &local_list, list) { + if (enic->admin_rq_handler) + enic->admin_rq_handler(enic, msg->data, msg->len); + list_del(&msg->list); + kfree(msg); + } +} + +static int enic_admin_napi_poll(struct napi_struct *napi, int budget) +{ + struct enic *enic =3D container_of(napi, struct enic, admin_napi); + unsigned int credits; + unsigned int rq_work; + + credits =3D vnic_intr_credits(&enic->admin_intr); + + rq_work =3D enic_admin_rq_cq_service(enic, budget); + + if (rq_work > 0) + schedule_work(&enic->admin_msg_work); + + if (rq_work < budget && napi_complete_done(napi, rq_work)) { + if (credits) + vnic_intr_return_credits(&enic->admin_intr, credits, + 1 /* unmask */, 0); + } else { + if (credits) + vnic_intr_return_credits(&enic->admin_intr, credits, + 0 /* don't unmask */, 0); + } + + return rq_work; +} + +static int enic_admin_setup_intr(struct enic *enic) +{ + unsigned int intr_index =3D enic->intr_count; + int err; + + if (vnic_dev_get_intr_mode(enic->vdev) !=3D VNIC_DEV_INTR_MODE_MSIX || + intr_index >=3D enic->intr_avail) + return -ENODEV; + + /* The admin INTR uses a slot in the same RES_TYPE_INTR_CTRL + * strided array of per-vector control blocks (mask, coalescing + * timer, credit return) that the data-path IRQs occupy in BAR0. + * vnic_intr_alloc() defaults to RES_TYPE_INTR_CTRL, which is what + * we want here. + * + * RES_TYPE_SRIOV_INTR is *not* a substitute: it is a PF-side + * capability marker that counts the number of per-VF interrupt + * banks firmware has provisioned, not a usable per-vector + * register window. Firmware exposes the actual per-VF interrupt + * registers in each VF's BAR0 as RES_TYPE_INTR_CTRL. + */ + err =3D vnic_intr_alloc(enic->vdev, &enic->admin_intr, intr_index); + if (err) { + netdev_warn(enic->netdev, + "Failed to alloc admin intr at index %u: %d\n", + intr_index, err); + return err; + } + + enic->admin_intr_index =3D intr_index; + + snprintf(enic->msix[intr_index].devname, + sizeof(enic->msix[intr_index].devname), + "%s-admin", enic->netdev->name); + enic->msix[intr_index].isr =3D enic_admin_isr_msix; + enic->msix[intr_index].devid =3D &enic->admin_napi; + + err =3D request_irq(enic->msix_entry[intr_index].vector, + enic->msix[intr_index].isr, 0, + enic->msix[intr_index].devname, + enic->msix[intr_index].devid); + if (err) { + netdev_warn(enic->netdev, + "Failed to request admin MSI-X irq: %d\n", err); + vnic_intr_free(&enic->admin_intr); + return err; + } + + enic->msix[intr_index].requested =3D 1; + + netif_napi_add(enic->netdev, &enic->admin_napi, + enic_admin_napi_poll); + napi_enable(&enic->admin_napi); + + netdev_dbg(enic->netdev, + "admin channel using MSI-X interrupt (index %u)\n", + intr_index); + + return 0; +} + +static void enic_admin_teardown_intr(struct enic *enic) +{ + unsigned int intr_index =3D enic->admin_intr_index; + + napi_disable(&enic->admin_napi); + netif_napi_del(&enic->admin_napi); + + free_irq(enic->msix_entry[intr_index].vector, + enic->msix[intr_index].devid); + enic->msix[intr_index].requested =3D 0; +} + static int enic_admin_qp_type_set(struct enic *enic, u32 enable) { u64 a0 =3D QP_TYPE_ADMIN, a1 =3D enable; @@ -160,23 +407,8 @@ static int enic_admin_alloc_resources(struct enic *eni= c) if (err) goto free_cq0; =20 - /* PFs have dedicated SRIOV_INTR resources for admin channel. - * VFs lack SRIOV_INTR; use a regular INTR_CTRL slot instead. - */ - if (vnic_dev_get_res_count(enic->vdev, RES_TYPE_SRIOV_INTR) >=3D 1) - err =3D vnic_intr_alloc_with_type(enic->vdev, - &enic->admin_intr, 0, - RES_TYPE_SRIOV_INTR); - else - err =3D vnic_intr_alloc(enic->vdev, &enic->admin_intr, - enic->intr_count); - if (err) - goto free_cq1; - return 0; =20 -free_cq1: - vnic_cq_free(&enic->admin_cq[1]); free_cq0: vnic_cq_free(&enic->admin_cq[0]); free_rq: @@ -197,13 +429,47 @@ static void enic_admin_free_resources(struct enic *en= ic) =20 static void enic_admin_init_resources(struct enic *enic) { + unsigned int intr_offset =3D enic->admin_intr_index; + vnic_wq_init(&enic->admin_wq, 0, 0, 0); vnic_rq_init(&enic->admin_rq, 1, 0, 0); - vnic_cq_init(&enic->admin_cq[0], 0, 1, 0, 0, 1, 0, 1, 0, 0, 0); - vnic_cq_init(&enic->admin_cq[1], 0, 1, 0, 0, 1, 0, 1, 0, 0, 0); + vnic_cq_init(&enic->admin_cq[0], + 0 /* flow_control_enable */, + 1 /* color_enable */, + 0 /* cq_head */, + 0 /* cq_tail */, + 1 /* cq_tail_color */, + 0 /* interrupt_enable - polled synchronously by mbox send */, + 1 /* cq_entry_enable */, + 0 /* cq_message_enable */, + intr_offset, + 0 /* cq_message_addr */); + vnic_cq_init(&enic->admin_cq[1], + 0 /* flow_control_enable */, + 1 /* color_enable */, + 0 /* cq_head */, + 0 /* cq_tail */, + 1 /* cq_tail_color */, + 1 /* interrupt_enable */, + 1 /* cq_entry_enable */, + 0 /* cq_message_enable */, + intr_offset, + 0 /* cq_message_addr */); vnic_intr_init(&enic->admin_intr, 0, 0, 1); } =20 +static void enic_admin_msg_drain(struct enic *enic) +{ + struct enic_admin_msg *msg, *tmp; + + spin_lock_bh(&enic->admin_msg_lock); + list_for_each_entry_safe(msg, tmp, &enic->admin_msg_list, list) { + list_del(&msg->list); + kfree(msg); + } + spin_unlock_bh(&enic->admin_msg_lock); +} + int enic_admin_channel_open(struct enic *enic) { int err; @@ -219,12 +485,24 @@ int enic_admin_channel_open(struct enic *enic) return err; } =20 + spin_lock_init(&enic->admin_msg_lock); + INIT_LIST_HEAD(&enic->admin_msg_list); + INIT_WORK(&enic->admin_msg_work, enic_admin_msg_work_handler); + + err =3D enic_admin_setup_intr(enic); + if (err) { + netdev_err(enic->netdev, + "Admin channel requires MSI-X, SR-IOV unavailable: %d\n", + err); + goto free_resources; + } + enic_admin_init_resources(enic); =20 vnic_wq_enable(&enic->admin_wq); vnic_rq_enable(&enic->admin_rq); =20 - err =3D enic_admin_rq_fill(enic); + err =3D enic_admin_rq_fill(enic, GFP_KERNEL); if (err) { netdev_err(enic->netdev, "Failed to fill admin RQ buffers: %d\n", err); @@ -238,13 +516,27 @@ int enic_admin_channel_open(struct enic *enic) goto disable_queues; } =20 + vnic_intr_unmask(&enic->admin_intr); + + netdev_dbg(enic->netdev, + "admin channel open: intr=3D%u wq_avail=3D%u rq_avail=3D%u cq0_color= =3D%u cq1_color=3D%u\n", + enic->admin_intr_index, + vnic_wq_desc_avail(&enic->admin_wq), + vnic_rq_desc_avail(&enic->admin_rq), + enic->admin_cq[0].last_color, + enic->admin_cq[1].last_color); + return 0; =20 disable_queues: + enic_admin_teardown_intr(enic); enic_admin_qp_type_set(enic, 0); vnic_wq_disable(&enic->admin_wq); vnic_rq_disable(&enic->admin_rq); + cancel_work_sync(&enic->admin_msg_work); + enic_admin_msg_drain(enic); enic_admin_rq_drain(enic); +free_resources: enic_admin_free_resources(enic); return err; } @@ -254,6 +546,13 @@ void enic_admin_channel_close(struct enic *enic) if (!enic->has_admin_channel) return; =20 + netdev_dbg(enic->netdev, "admin channel close\n"); + + vnic_intr_mask(&enic->admin_intr); + enic_admin_teardown_intr(enic); + cancel_work_sync(&enic->admin_msg_work); + enic_admin_msg_drain(enic); + enic_admin_qp_type_set(enic, 0); =20 vnic_wq_disable(&enic->admin_wq); diff --git a/drivers/net/ethernet/cisco/enic/enic_admin.h b/drivers/net/eth= ernet/cisco/enic/enic_admin.h index 569aadeb9312..73cdd3dac7ec 100644 --- a/drivers/net/ethernet/cisco/enic/enic_admin.h +++ b/drivers/net/ethernet/cisco/enic/enic_admin.h @@ -9,7 +9,19 @@ =20 struct enic; =20 +/* Wrapper for received admin messages queued for deferred processing. + * NAPI enqueues these; a workqueue handler processes them in process cont= ext + * where sleeping (mutex, GFP_KERNEL) is safe. + */ +struct enic_admin_msg { + struct list_head list; + unsigned int len; + u8 data[]; +}; + int enic_admin_channel_open(struct enic *enic); void enic_admin_channel_close(struct enic *enic); +unsigned int enic_admin_wq_cq_service(struct enic *enic); +unsigned int enic_admin_rq_cq_service(struct enic *enic, unsigned int budg= et); =20 #endif /* _ENIC_ADMIN_H_ */ --=20 2.43.0 From nobody Sun Jun 14 04:21:27 2026 Received: from alln-iport-3.cisco.com (alln-iport-3.cisco.com [173.37.142.90]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8C7041F3BA4; Sun, 3 May 2026 11:23:07 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=173.37.142.90 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777807389; cv=none; b=CDvsPvpudo1bqZsmMj8v5w+KYTJ7w0Z2zi67ktR2RJ7sCCLIdEVOCVrXPuBCGRrDwPeWOJmhvFT0YednLLxPFWIszC0XVFGO5Nh+wKy+P8471VyoQegLoIpeyN313cNCEimID3mrCSDU+jieCC9GOt4UmrLdtJLd2ClgO1In2Ro= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777807389; c=relaxed/simple; bh=DJOEZz0LgusiJsaMcuyJ/mo5kxauoqkYFLtqBtwJznQ=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=V6aFqTWOjP9zDAeVgyWzZOVF6/gjTGIWEiQesku7+Ctsh5n1Vh8tfpvodgsfGfujoWfhP0CD0Iy+pX0hXW6O8A5N+PpbxehmqDKtL473cdxGN3o/9WBISXqqJwAgFRpGyMt6vMPFLrMgQpPWe5jTwP8fhY7gdWMzBJmd5BcL6C8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=cisco.com; spf=pass smtp.mailfrom=cisco.com; dkim=pass (2048-bit key) header.d=cisco.com header.i=@cisco.com header.b=KpM0q1AO; arc=none smtp.client-ip=173.37.142.90 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=cisco.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=cisco.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=cisco.com header.i=@cisco.com header.b="KpM0q1AO" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cisco.com; i=@cisco.com; l=3222; q=dns/txt; s=iport01; t=1777807387; x=1779016987; h=from:date:subject:mime-version:content-transfer-encoding: message-id:references:in-reply-to:to:cc; bh=Ismjn8aCzPjNAuW3z5gQmVHzab5Wc6BkN3bhmJ9vQgA=; b=KpM0q1AOqy4C48+lAWJlrLAl693i3kWamAjhs3o7XOsyrjmVq9OG088O GGhao8VmmkoYHIaF2HVrlK5/duk/WQzAa629dRfhjBmDNNuFk6c4BZ+WG rYWRC1FE84RdyzRf1sADWyEZnAgOW2IXICOaFWfdbZV4a+tBXmi3akqW1 8CwHE8Kky4dtRqZhh/AZ9TPQqxSmk79D6Xy9br6pN1vDnssCsxt3cGNlk Eq/Gy/xdN901BXIMz8AIeERKoAK9bBkkkNGdy/z3K+E6b697vYIlgsNtF sBMdsaOljBHrDU00B6jQ06cf/ekupiA32dXFm3Zqtplndi+5ZQFNb+Jvj w==; X-CSE-ConnectionGUID: iaWwy1uCSTGD8banveCnpA== X-CSE-MsgGUID: dsafLTyxSsS5Xm7KNkHijQ== X-IPAS-Result: =?us-ascii?q?A0BlAwCIL/dp/4v/Ja1aHgEBCxIMgWsQDwuEIkNJhFePU?= =?us-ascii?q?4IhkU2MUBSBaw8BAQEPUQQBAYUGAo0xAiY0CQ4BAgQBAQEBAwIDAQEBAQEBA?= =?us-ascii?q?QEBAQEBCgEBBQEBAQIBBwWBDhOGXIZbAgEDI0QSECUCJgICRhAGARKDAoJ0A?= =?us-ascii?q?7MAeoEygQGEfbMlgWULFAGBCi6FcoJmAYFwhAY7hD8nG4FJRIEUAYNohAUBG?= =?us-ascii?q?YN/gmkEgxwUjxsJP4ECHANZLAFVEw0KCwcFYVIzAyAKCxISGBUCFBwTDwQWM?= =?us-ascii?q?h1wDCcSLBc0WBsHBYFLgwBtFDEEIYEFhGB4IywDTnIDCxgNSBEsNwYOGwQ9A?= =?us-ascii?q?W4HimYdD4E/cYEOfDVjGpdWjV1roQ6EJoFkoA4BMpdmkwWZBiKjG0MJUIRog?= =?us-ascii?q?Wg8gVkzGggbFYMiUxkP2GMmMj0BAQcCBw0DC4FokBGBbAEB?= IronPort-Data: A9a23:WGR4OKPUTsa8tvPvrR1llsFynXyQoLVcMsEvi/4bfWQNrUpwg2EBy WRKWGCOPq2JZjSgKogjbIyy/UkOu8PWn9dnSnM5pCpnJ55oRWUpJjg4wmPYZX76whjrFRo/h ykmQoCeap1yFjmB+0rF3oHJ9RFUzbuPSqf3FNnKMyVwQR4MYCo6gHqPocZh6mJTqYb/WVjlV e/a+ZWFZgf4g2Usawr41orawP9RlKWq0N8nlgRWicBj5Df2i3QTBZQDEqC9R1OQapVUBOOzW 9HYx7i/+G7Dlz91Yj9yuu+mGqGiaue60Tmm0hK6aYD76vRxjnBaPpIACRYpQRw/ZwNlMDxG4 I4lWZSYEW/FN0BX8QgXe0Ew/ypWZcWq9FJbSJSymZT78qHIT5fj6/RnM14nZJAzwdpYLTFE/ PoGJRsLdjnW0opawJrjIgVtrt4oIM+uOMYUvWttiGmDS/0nWpvEBa7N4Le03h9p2ZsIRqmYP ZdEL2M0PXwsYDUXUrsTIIk3gOazhnr/WzZZs1mS46Ew5gA/ySQujOiwYYaEK4fiqcN9kl2Rq Eb67nTCGSpHBNnA6hyG0nOuv7qa9c/8cMdIfFGizdZmiUOew0QfAQMbUF+8r+X/jEOiM/pSJ 1ER8zgjsYA980ukStS7VBq9yFaHoxQVc9ldCes37EeK0KW8yw+fCnIJUX1HZcAqudEeQSEs0 BmCn7vBHTVlvbuUYWiQ+redsXW5Pi19BWsDYzIUCAgI+d/upKktgR/VCNVuCqi4ipvyAz6Y6 zaHsCQzmZ0NgsMRkaa251bKh3SrvJehc+Iuzh/cUmTg6kZyY5SoItTyr1Pa9v1Hao2eSzFto UQ5piRX18hWZbnlqcBHaLxl8G2BjxpdDADhvA== IronPort-HdrOrdr: A9a23:3l+1zqC80f1xDdnlHela55DYdb4zR+YMi2TDGXoBLSC9Ffb4qy nOppomPHDP5Qr5NEtLpTnEAtjjfZq+z/FICPcqXYtKNTOO0ACVxcNZnOjfKlbbdxEWmNQtsJ tIQuxSBML9CF5igd2/wgWpeuxQueWvweSEifjTynF1QRovUaFv6A9lFh2We3cGIjWufaBJc6 a0145guyeqf2gRY4CQA3kIWPWGmvjw/aiWGSLvw3UcmXOzZfTC0s+ALySl X-Talos-CUID: =?us-ascii?q?9a23=3A/eKbY2upxGIyaGnMYI67eUVi6Iske0LC3S7NCHS?= =?us-ascii?q?EJko5QrSTV1K7/rtNxp8=3D?= X-Talos-MUID: 9a23:/6WKSAVmYuV1/GLq/DP3jT9GCNZ42L+VVXAIzrYguOmFNxUlbg== X-IronPort-Anti-Spam-Filtered: true X-IronPort-AV: E=Sophos;i="6.23,213,1770595200"; d="scan'208";a="748594993" Received: from rcdn-l-core-02.cisco.com ([173.37.255.139]) by alln-iport-3.cisco.com with ESMTP/TLS/TLS_AES_256_GCM_SHA384; 03 May 2026 11:22:59 +0000 Received: from [127.0.0.1] (unknown [10.25.128.216]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by rcdn-l-core-02.cisco.com (Postfix) with ESMTPS id EAE74180007D0; Sun, 3 May 2026 11:22:58 +0000 (GMT) From: Satish Kharat Date: Sun, 03 May 2026 04:22:42 -0700 Subject: [PATCH net-next v6 05/10] enic: define MBOX message types and header structures Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260503-enic-sriov-v2-admin-channel-v2-v6-5-0af4fbc2d86d@cisco.com> References: <20260503-enic-sriov-v2-admin-channel-v2-v6-0-0af4fbc2d86d@cisco.com> In-Reply-To: <20260503-enic-sriov-v2-admin-channel-v2-v6-0-0af4fbc2d86d@cisco.com> To: Andrew Lunn , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni Cc: netdev@vger.kernel.org, linux-kernel@vger.kernel.org, Sesidhar Baddela , Satish Kharat X-Mailer: b4 0.15.2 X-Developer-Signature: v=1; a=ed25519-sha256; t=1777807375; l=3237; i=satishkh@cisco.com; h=from:subject:message-id; bh=DJOEZz0LgusiJsaMcuyJ/mo5kxauoqkYFLtqBtwJznQ=; b=n11UP+DdIpvGb+6r2WYtWl6dCXegUqdRas5tokuoYwpouUnj2SWnOO6nzRLkRRv1Gp+LtC0hz jvSquuv225/C+PU1ZICj5RPn9VvvLRm2SiFAzac+Qf3DqJVO7GzMt+W X-Developer-Key: i=satishkh@cisco.com; a=ed25519; pk=4BxbLoCGvHeIMH/PbpiNVx8/2IcTjFvqcr0AD8K6WLY= X-Outbound-Client-TLS: ANONYMOUS;unknown [10.25.128.216];TLSv1.3;TLS_AES_256_GCM_SHA384;256 X-Outbound-SMTP-Client: 10.25.128.216, [10.25.128.216] X-Outbound-Node: rcdn-l-core-02.cisco.com Define the mailbox protocol structures for PF-VF communication: message header, generic reply, and per-message-type payloads for capability negotiation, VF registration/unregistration, and link state notification/acknowledgment. Include linux/types.h and linux/bits.h for __le16/__le32/__le64 and BIT() used in the header. Message types use an even=3Drequest / odd=3Dreply convention. The header carries source and destination VNIC IDs, a monotonically increasing message number, and the total message length. Signed-off-by: Satish Kharat --- drivers/net/ethernet/cisco/enic/enic_mbox.h | 83 +++++++++++++++++++++++++= ++++ 1 file changed, 83 insertions(+) diff --git a/drivers/net/ethernet/cisco/enic/enic_mbox.h b/drivers/net/ethe= rnet/cisco/enic/enic_mbox.h new file mode 100644 index 000000000000..a52f1d25cb21 --- /dev/null +++ b/drivers/net/ethernet/cisco/enic/enic_mbox.h @@ -0,0 +1,83 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* Copyright 2025 Cisco Systems, Inc. All rights reserved. */ + +#ifndef _ENIC_MBOX_H_ +#define _ENIC_MBOX_H_ + +#include +#include + +/* + * Mailbox protocol for PF-VF communication over the admin channel. + * + * Even numbers are requests, odd numbers are replies/acks. + * The prefix indicates the initiator: VF_ =3D VF-initiated, PF_ =3D PF-in= itiated. + */ +enum enic_mbox_msg_type { + ENIC_MBOX_VF_CAPABILITY_REQUEST =3D 0, + ENIC_MBOX_VF_CAPABILITY_REPLY =3D 1, + ENIC_MBOX_VF_REGISTER_REQUEST =3D 2, + ENIC_MBOX_VF_REGISTER_REPLY =3D 3, + ENIC_MBOX_VF_UNREGISTER_REQUEST =3D 4, + ENIC_MBOX_VF_UNREGISTER_REPLY =3D 5, + ENIC_MBOX_PF_LINK_STATE_NOTIF =3D 6, + ENIC_MBOX_PF_LINK_STATE_ACK =3D 7, + ENIC_MBOX_MAX +}; + +struct enic_mbox_hdr { + __le16 src_vnic_id; + __le16 dst_vnic_id; + u8 msg_type; + u8 flags; + __le16 msg_len; + __le64 msg_num; +}; + +struct enic_mbox_generic_reply { + __le16 ret_major; + __le16 ret_minor; +}; + +#define ENIC_MBOX_ERR_GENERIC BIT(0) +#define ENIC_MBOX_ERR_VF_NOT_REGISTERED BIT(1) +#define ENIC_MBOX_ERR_MSG_NOT_SUPPORTED BIT(2) + +/* ENIC_MBOX_VF_CAPABILITY_REQUEST / _REPLY */ +#define ENIC_MBOX_CAP_VERSION_0 0 +#define ENIC_MBOX_CAP_VERSION_1 1 + +struct enic_mbox_vf_capability_msg { + __le32 version; + __le32 reserved[32]; +}; + +/* The embedded enic_mbox_generic_reply has 2-byte alignment, but the + * __le32 members give this struct 4-byte natural alignment. Receive + * buffers come from kmalloc (>=3D 8-byte aligned), so there is no + * misaligned access risk when casting from the receive buffer. + */ +struct enic_mbox_vf_capability_reply_msg { + struct enic_mbox_generic_reply reply; + __le32 version; + __le32 reserved[32]; +}; + +/* ENIC_MBOX_VF_REGISTER / _UNREGISTER */ +struct enic_mbox_vf_register_reply_msg { + struct enic_mbox_generic_reply reply; +}; + +/* ENIC_MBOX_PF_LINK_STATE_NOTIF / _ACK */ +#define ENIC_MBOX_LINK_STATE_DISABLE 0 +#define ENIC_MBOX_LINK_STATE_ENABLE 1 + +struct enic_mbox_pf_link_state_notif_msg { + __le32 link_state; +}; + +struct enic_mbox_pf_link_state_ack_msg { + struct enic_mbox_generic_reply ack; +}; + +#endif /* _ENIC_MBOX_H_ */ --=20 2.43.0 From nobody Sun Jun 14 04:21:27 2026 Received: from alln-iport-2.cisco.com (alln-iport-2.cisco.com [173.37.142.89]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id ABF4734D389; Sun, 3 May 2026 11:24:08 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=173.37.142.89 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777807450; cv=none; b=e3sj1A5KOkANqJf+i8H6zbNMqIA6iN77yuUBLBhgDncrtofqKlnFSeOyJoCw2gWoVWO2jBc8uOAYBNhw1UovmWRwRd4oD2TmUsnVid1Ei/Y4G6iNY406fXsg75M3IM70XEck57vtFK7ZYI1WKQHP/7AkI6LGRbBUOKaIMDcmYDk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777807450; c=relaxed/simple; bh=9zpd/txLttPW3uuD2/DhiCw+LPXtnqyXW0Y+kpn/Y1k=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=bsZ0zqhp1DmLXu8jMJzJM1cxP1P5Tj2dB8QDW1CnNHc48FBfZpEgQsy8wxmlcFrQQo+RkjQ/c6re4atBJoiW1t0fCjDsib4EB4rtOA0VHiwQi6QUHDzDrEdYL8IvLkzp/lDS2oDFTMJzor9yswoxvP/4E5s1WA5RNPqhvneBAcI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=cisco.com; spf=pass smtp.mailfrom=cisco.com; dkim=pass (2048-bit key) header.d=cisco.com header.i=@cisco.com header.b=E92Qe/Ft; arc=none smtp.client-ip=173.37.142.89 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=cisco.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=cisco.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=cisco.com header.i=@cisco.com header.b="E92Qe/Ft" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cisco.com; i=@cisco.com; l=9048; q=dns/txt; s=iport01; t=1777807448; x=1779017048; h=from:date:subject:mime-version:content-transfer-encoding: message-id:references:in-reply-to:to:cc; bh=dR68T1HVCfUDwmwTAMKpL+GC8TWYIs6Vwc/0JTzPkkg=; b=E92Qe/FtFPaW1t4CDwRRqkAtQrzD3qupJLqw5iY7rDlMKoa01j4gDxof Fnle6QS22KSXTpWde6WB5hf+9cb/H7gdttVOw1yx1gphL/J31oMtdw9Zm wpi3U7TPBGgPU4hKhj//3vXG/tESrr+znMnT8u13z07OG+PVBAmGwkLF7 Dv0Vhxg4ujS1txZdyQZUSYETLW4ztQ0HweQ9q39ubseKtdsHCm+S7oWw1 skOalGXjeO1G8mqVfNqjFKTPWYPjQPwTNFNbDd2Vg9wE32vgxZWsAhBP8 doRywLukFlE3DxpcPdeIm512ea6ssOsqG9ynQHqTa40oXL2nHQBjDZ20/ Q==; X-CSE-ConnectionGUID: bz/W7YWzTO24B2QFQ7ERYg== X-CSE-MsgGUID: p4i5KOSBRZ2mHym8aDc6kw== X-IPAS-Result: =?us-ascii?q?A0BIAgDQLvdp/4v/Ja1QCh4BAQsSDIIFC4JXgU9DSYRXj?= =?us-ascii?q?1OCIZ4dFIFrDwEBAQ9RBAEBhQYCjTECJjQJDgECBAEBAQEDAgMBAQEBAQEBA?= =?us-ascii?q?QEBAQsBAQUBAQECAQcFgQ4ThlyGWwIBAyMEUhAlAiYCAkUBEAYBEoMCgnQDt?= =?us-ascii?q?B56fzOBAYR92UOBZAELFAGBCi6FcoJmAYFwhAY7hD8nG4FJRIEUAYNohAUBE?= =?us-ascii?q?AmDf4JpBIMcFIFeIIJxiiwJP4ECHANZLAFVEw0KCwcFYVIzAyAKCxISGBUCF?= =?us-ascii?q?BwTDwQWMh1wDCcSLBc0WBsHBYFLgwBtFDEEIYEFhGB4IywDTnIDCxgNSBEsN?= =?us-ascii?q?wYOGwQ9AW4HimYdD4E/agcBgQ0BE4EpVzEZLJZ6jV1roQ6EJoFkjFqTNAEyl?= =?us-ascii?q?2aTBZkGIqNJboRogWg8gVkzGggbFYMiUxkPji0WHMk8JjI9AQEHAgcNAwuBa?= =?us-ascii?q?JACDxcHgU4BAQ?= IronPort-Data: A9a23:WVSSzqrl+5Rlhn8d5lsCbZFHSuVeBmLYZBIvgKrLsJaIsI4StFCzt garIBmCb/mDZTD3L9p0YY+/8koEvZXSmNViHVBl+CBhQSIS9+PIVI+TRqvS04x+DSFioGZPt Zh2hgzodZhsJpPkjk7zdOCn9j8kif3gqoPUUIbsIjp2SRJvVBAvgBdin/9RqoNziLBVOSvV0 T/Ji5OZYgTNNwJcaDpOtfre8Eg35ZwehRtB1rAATaET1LPhvyF94KI3fcmZM3b+S49IKe+2L 86r5K255G7Q4yA2AdqjlLvhGmVSKlIFFVHT4pb+c/HKbilq/kTe4I5iXBYvQRs/ZwGyojxE4 I4lWapc5useFvakdOw1C3G0GszlVEFM0OevzXOX6aR/w6BaGpfh660GMa04AWEX0uxlRl4Rx MMKFBdXMjrZvsyPn5+Be+Y506zPLOGzVG8ekmtrwTecCbMtRorOBv2Qo9RZxzw3wMtJGJ4yZ eJANmEpN0uGOUASfA5MWfrSn8/w7pX7WyZRtFOOpak3y2PS1wd2lrPqNbI5f/TVH5UEwR3E+ DKuE2LRGk0baf+48hC/wCippv7KzSrXU9wRC+jtnhJtqBjJroAJMzUaXEW2pNG1g1CzXtZYJ VBS/CcyxYA/+FGuR8vwQzW3p3mLuhNaUN1Ve8U59QuE4qnZ+QCUAi4DVDEpQNUguNU7Wn8s2 0OFks3BASFptvueSRq17r6eoDWzETIYIW8LeWkPSg5ty93ippwjyxHCVNBuFIargdDvXzL92 TaHqG45nbp7pcgGy6m243jZjD+24JvEVAg44kPQRG3N0+9iTJSua4rt7R3Q6uxNad7DCFKAp 3MD3cOZ6YjiEK2wqcBEe81VdJnB2hpPGGS0bYJHd3X5ywmQxg== IronPort-HdrOrdr: A9a23:eYLYFKgGbXBWFQnCmT3kXdjiB3BQXiwji2hC6mlwRA09TyVXrb HLoB19737JYVcqNk3I9ursBEDtexnhHNtOkOss1NSZMzUOxlHJEGgK1+KLqAEIWReOldK1vp 0NT0ERMqyUMXFKyev3/wW8Fc8t252k/LDAv5am815dCSxndK1k6R50EUKgEkNwTBRbHpZRLu v72iM+nUvGRV0nKuKmG3ICQ+/P4/fPlJ7gfFo6IiRP0nj3sdttg4SKaSRxGXwlIkhy/Ys= X-Talos-CUID: =?us-ascii?q?9a23=3AXF7pQGmOAjADf6Gloa4VLFpkC9vXOXfPi0iPEnC?= =?us-ascii?q?qNUEzFJfPcHGpp45eosU7zg=3D=3D?= X-Talos-MUID: 9a23:0PYkPQnpZVuL6IaVHZQjdnp6aMY1wIOVIno/y5AmqszUPAB7ZTSC2WE= X-IronPort-Anti-Spam-Filtered: true X-IronPort-AV: E=Sophos;i="6.23,213,1770595200"; d="scan'208";a="728973468" Received: from rcdn-l-core-02.cisco.com ([173.37.255.139]) by alln-iport-2.cisco.com with ESMTP/TLS/TLS_AES_256_GCM_SHA384; 03 May 2026 11:23:00 +0000 Received: from [127.0.0.1] (unknown [10.25.128.216]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by rcdn-l-core-02.cisco.com (Postfix) with ESMTPS id A7325180007C2; Sun, 3 May 2026 11:22:59 +0000 (GMT) From: Satish Kharat Date: Sun, 03 May 2026 04:22:43 -0700 Subject: [PATCH net-next v6 06/10] enic: add MBOX core send and receive for admin channel Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260503-enic-sriov-v2-admin-channel-v2-v6-6-0af4fbc2d86d@cisco.com> References: <20260503-enic-sriov-v2-admin-channel-v2-v6-0-0af4fbc2d86d@cisco.com> In-Reply-To: <20260503-enic-sriov-v2-admin-channel-v2-v6-0-0af4fbc2d86d@cisco.com> To: Andrew Lunn , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni Cc: netdev@vger.kernel.org, linux-kernel@vger.kernel.org, Sesidhar Baddela , Satish Kharat X-Mailer: b4 0.15.2 X-Developer-Signature: v=1; a=ed25519-sha256; t=1777807375; l=9303; i=satishkh@cisco.com; h=from:subject:message-id; bh=9zpd/txLttPW3uuD2/DhiCw+LPXtnqyXW0Y+kpn/Y1k=; b=8Q5gsM+eCSltrgB0Un1gZ7JJfaWGB5lI+2eUOk0Dqw6h6GPvzmTxLmCDKolQb9IDoudx/gptR hhjtOAJjtDkDQppC8rUztm/itg0v+//XVj0jbdMFhzi4qUEGLP4+1Fn X-Developer-Key: i=satishkh@cisco.com; a=ed25519; pk=4BxbLoCGvHeIMH/PbpiNVx8/2IcTjFvqcr0AD8K6WLY= X-Outbound-Client-TLS: ANONYMOUS;unknown [10.25.128.216];TLSv1.3;TLS_AES_256_GCM_SHA384;256 X-Outbound-SMTP-Client: 10.25.128.216, [10.25.128.216] X-Outbound-Node: rcdn-l-core-02.cisco.com Implement the mailbox protocol engine used for PF-VF communication over the admin channel. The send path (enic_mbox_send_msg) builds a message with a common header, DMA-maps it, posts a single WQ descriptor with the destination vnic ID encoded in the VLAN tag field, and polls the WQ CQ for completion. The receive path (enic_mbox_recv_handler) is installed as the admin RQ callback and validates incoming message headers. PF/VF-specific dispatch will be added in subsequent commits. Signed-off-by: Satish Kharat --- drivers/net/ethernet/cisco/enic/Makefile | 2 +- drivers/net/ethernet/cisco/enic/enic.h | 6 + drivers/net/ethernet/cisco/enic/enic_admin.c | 22 +++- drivers/net/ethernet/cisco/enic/enic_mbox.c | 162 +++++++++++++++++++++++= ++++ drivers/net/ethernet/cisco/enic/enic_mbox.h | 8 ++ 5 files changed, 198 insertions(+), 2 deletions(-) diff --git a/drivers/net/ethernet/cisco/enic/Makefile b/drivers/net/etherne= t/cisco/enic/Makefile index 7ae72fefc99a..e38aaf34c148 100644 --- a/drivers/net/ethernet/cisco/enic/Makefile +++ b/drivers/net/ethernet/cisco/enic/Makefile @@ -4,5 +4,5 @@ obj-$(CONFIG_ENIC) :=3D enic.o enic-y :=3D enic_main.o vnic_cq.o vnic_intr.o vnic_wq.o \ enic_res.o enic_dev.o enic_pp.o vnic_dev.o vnic_rq.o vnic_vic.o \ enic_ethtool.o enic_api.o enic_clsf.o enic_rq.o enic_wq.o \ - enic_admin.o + enic_admin.o enic_mbox.o =20 diff --git a/drivers/net/ethernet/cisco/enic/enic.h b/drivers/net/ethernet/= cisco/enic/enic.h index 1c09da3c0b1a..42f345aceced 100644 --- a/drivers/net/ethernet/cisco/enic/enic.h +++ b/drivers/net/ethernet/cisco/enic/enic.h @@ -292,6 +292,8 @@ struct enic { =20 /* Admin channel resources for SR-IOV MBOX */ bool has_admin_channel; + /* set on send timeout; cleared on channel re-open */ + bool mbox_send_disabled; struct vnic_wq admin_wq; struct vnic_rq admin_rq; struct vnic_cq admin_cq[2]; @@ -304,6 +306,10 @@ struct enic { u64 admin_msg_drop_cnt; void (*admin_rq_handler)(struct enic *enic, void *buf, unsigned int len); + + /* MBOX protocol state */ + struct mutex mbox_lock; + u64 mbox_msg_num; }; =20 static inline struct net_device *vnic_get_netdev(struct vnic_dev *vdev) diff --git a/drivers/net/ethernet/cisco/enic/enic_admin.c b/drivers/net/eth= ernet/cisco/enic/enic_admin.c index 5445815139b5..43e697b7b424 100644 --- a/drivers/net/ethernet/cisco/enic/enic_admin.c +++ b/drivers/net/ethernet/cisco/enic/enic_admin.c @@ -19,6 +19,7 @@ #include "cq_enet_desc.h" #include "wq_enet_desc.h" #include "rq_enet_desc.h" +#include "enic_mbox.h" =20 /* Clean up any admin WQ buffers still held by hardware at close time. * Normally buffers are freed inline after send completion, but a timed-out @@ -197,7 +198,25 @@ unsigned int enic_admin_rq_cq_service(struct enic *eni= c, unsigned int budget) goto next_desc; } =20 - enic_admin_msg_enqueue(enic, buf->os_buf, bytes_written); + if (enic->admin_rq_handler) { + u16 sender_vlan; + + /* Firmware sets the CQ VLAN field to identify the + * sender: 0 =3D PF, 1-based =3D VF index. Overwrite + * the untrusted src_vnic_id in the MBOX header with + * the hardware-verified value. + */ + sender_vlan =3D le16_to_cpu(rq_desc->vlan); + if (bytes_written >=3D sizeof(struct enic_mbox_hdr)) { + struct enic_mbox_hdr *hdr =3D buf->os_buf; + + hdr->src_vnic_id =3D (sender_vlan =3D=3D 0) ? + cpu_to_le16(ENIC_MBOX_DST_PF) : + cpu_to_le16(sender_vlan - 1); + } + + enic_admin_msg_enqueue(enic, buf->os_buf, bytes_written); + } =20 next_desc: enic_admin_rq_buf_clean(rq, rq->to_clean); @@ -477,6 +496,7 @@ int enic_admin_channel_open(struct enic *enic) if (!enic->has_admin_channel) return -ENODEV; =20 + enic->mbox_send_disabled =3D false; err =3D enic_admin_alloc_resources(enic); if (err) { netdev_err(enic->netdev, diff --git a/drivers/net/ethernet/cisco/enic/enic_mbox.c b/drivers/net/ethe= rnet/cisco/enic/enic_mbox.c new file mode 100644 index 000000000000..3b2fc9822176 --- /dev/null +++ b/drivers/net/ethernet/cisco/enic/enic_mbox.c @@ -0,0 +1,162 @@ +// SPDX-License-Identifier: GPL-2.0-only +// Copyright 2025 Cisco Systems, Inc. All rights reserved. + +#include +#include +#include +#include + +#include "vnic_dev.h" +#include "vnic_wq.h" +#include "vnic_cq.h" +#include "enic.h" +#include "enic_admin.h" +#include "enic_mbox.h" +#include "wq_enet_desc.h" + +#define ENIC_MBOX_POLL_TIMEOUT_US 5000000 +#define ENIC_MBOX_POLL_INTERVAL_US 100 + +static void enic_mbox_fill_hdr(struct enic *enic, struct enic_mbox_hdr *hd= r, + u8 msg_type, u16 dst_vnic_id, u16 msg_len) +{ + memset(hdr, 0, sizeof(*hdr)); + hdr->dst_vnic_id =3D cpu_to_le16(dst_vnic_id); + hdr->msg_type =3D msg_type; + hdr->msg_len =3D cpu_to_le16(msg_len); + hdr->msg_num =3D cpu_to_le64(++enic->mbox_msg_num); +} + +int enic_mbox_send_msg(struct enic *enic, u8 msg_type, u16 dst_vnic_id, + void *payload, u16 payload_len) +{ + u16 total_len =3D sizeof(struct enic_mbox_hdr) + payload_len; + struct vnic_wq *wq =3D &enic->admin_wq; + struct wq_enet_desc *desc; + unsigned long timeout; + dma_addr_t dma_addr; + u16 vlan_tag; + void *buf; + int err; + + /* Serialize MBOX sends. The admin channel is a low-frequency + * control path; holding the mutex across the poll is acceptable. + */ + mutex_lock(&enic->mbox_lock); + + if (!enic->has_admin_channel || enic->mbox_send_disabled) { + err =3D -ENODEV; + goto unlock; + } + + if (vnic_wq_desc_avail(wq) =3D=3D 0) { + err =3D -ENOSPC; + goto unlock; + } + + buf =3D kmalloc(total_len, GFP_KERNEL); + if (!buf) { + err =3D -ENOMEM; + goto unlock; + } + + enic_mbox_fill_hdr(enic, buf, msg_type, dst_vnic_id, total_len); + if (payload_len) { + void *dst =3D buf + sizeof(struct enic_mbox_hdr); + + memcpy(dst, payload, payload_len); + } + + dma_addr =3D dma_map_single(&enic->pdev->dev, buf, total_len, + DMA_TO_DEVICE); + if (dma_mapping_error(&enic->pdev->dev, dma_addr)) { + kfree(buf); + err =3D -ENOMEM; + goto unlock; + } + + /* Firmware uses vlan field for routing: 0 =3D PF, 1-based =3D VF index */ + if (dst_vnic_id =3D=3D ENIC_MBOX_DST_PF) + vlan_tag =3D 0; + else + vlan_tag =3D dst_vnic_id + 1; + + desc =3D vnic_wq_next_desc(wq); + wq_enet_desc_enc(desc, (u64)dma_addr | VNIC_PADDR_TARGET, + total_len, 0, 0, 0, 1, 1, 0, 1, vlan_tag, 0); + vnic_wq_post(wq, buf, dma_addr, total_len, 1, 1, 1, 1, 0, 0); + vnic_wq_doorbell(wq); + + timeout =3D jiffies + usecs_to_jiffies(ENIC_MBOX_POLL_TIMEOUT_US); + err =3D -ETIMEDOUT; + while (time_before(jiffies, timeout)) { + if (enic_admin_wq_cq_service(enic)) { + err =3D 0; + break; + } + usleep_range(ENIC_MBOX_POLL_INTERVAL_US, + ENIC_MBOX_POLL_INTERVAL_US + 50); + } + /* Final check in case completion arrived during the last sleep */ + if (err && enic_admin_wq_cq_service(enic)) + err =3D 0; + + if (!err) { + wq->to_clean =3D wq->to_clean->next; + wq->ring.desc_avail++; + dma_unmap_single(&enic->pdev->dev, dma_addr, total_len, + DMA_TO_DEVICE); + kfree(buf); + } else { + netdev_err(enic->netdev, + "MBOX send timed out (type %u dst %u), disabling channel\n", + msg_type, dst_vnic_id); + /* + * The WQ descriptor is still live in hardware. Do not unmap + * or free the buffer: the device may still DMA from dma_addr. + * Mark the channel unusable so no further sends are attempted. + */ + enic->mbox_send_disabled =3D true; + } + + netdev_dbg(enic->netdev, + "MBOX send msg_type %u dst %u vlan %u err %d\n", + msg_type, dst_vnic_id, vlan_tag, err); +unlock: + mutex_unlock(&enic->mbox_lock); + return err; +} + +static void enic_mbox_recv_handler(struct enic *enic, void *buf, + unsigned int len) +{ + struct enic_mbox_hdr *hdr =3D buf; + + if (len < sizeof(*hdr)) { + if (net_ratelimit()) + netdev_warn(enic->netdev, + "MBOX: truncated message (len %u < %zu)\n", + len, sizeof(*hdr)); + return; + } + + if (hdr->msg_type >=3D ENIC_MBOX_MAX) { + if (net_ratelimit()) + netdev_warn(enic->netdev, + "MBOX: unknown msg type %u\n", + hdr->msg_type); + return; + } + + netdev_dbg(enic->netdev, + "MBOX recv: type %u from vnic %u len %u\n", + hdr->msg_type, le16_to_cpu(hdr->src_vnic_id), + le16_to_cpu(hdr->msg_len)); +} + +void enic_mbox_init(struct enic *enic) +{ + enic->mbox_msg_num =3D 0; + mutex_init(&enic->mbox_lock); + enic->admin_rq_handler =3D enic_mbox_recv_handler; +} diff --git a/drivers/net/ethernet/cisco/enic/enic_mbox.h b/drivers/net/ethe= rnet/cisco/enic/enic_mbox.h index a52f1d25cb21..73fd7f783ee2 100644 --- a/drivers/net/ethernet/cisco/enic/enic_mbox.h +++ b/drivers/net/ethernet/cisco/enic/enic_mbox.h @@ -80,4 +80,12 @@ struct enic_mbox_pf_link_state_ack_msg { struct enic_mbox_generic_reply ack; }; =20 +#define ENIC_MBOX_DST_PF 0xFFFF + +struct enic; + +void enic_mbox_init(struct enic *enic); +int enic_mbox_send_msg(struct enic *enic, u8 msg_type, u16 dst_vnic_id, + void *payload, u16 payload_len); + #endif /* _ENIC_MBOX_H_ */ --=20 2.43.0 From nobody Sun Jun 14 04:21:27 2026 Received: from alln-iport-8.cisco.com (alln-iport-8.cisco.com [173.37.142.95]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8F5FB25487C; Sun, 3 May 2026 11:23:07 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=173.37.142.95 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777807389; cv=none; b=Ig+YqkJ4PTKqWjU7OEeKXbnnyJuHS6uyyXfxHi5JpnU/HcPj7iXJIyseeLHnfT8AkDRD1hUz5IIrBccGWHVPWyHMyeV45TNWgbUUd26GznavsntRIM2++1JcVqtmzOGBaiDjOoLMtqLmIXeuG9S1eN2VWEY94e16QkQgpCeN5+E= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777807389; c=relaxed/simple; bh=3UXiwEguexDSU5q9sXykOpn4fjIRMVeBycb+uHtZMiI=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=TuGuS1leXApK6qplUmc3p3nEyTjGMwNl3CLF7pj2LYftrqOS7hF+gU0pKaB6h5EfngykQyNkANHDMCoTIvZptQcrz07xo03WN+WeRcvi1oaCHEKEGhYrYWxK6Q7sGcoYbabAAiCDJxqJ3vCu1/2etv58KO5jnPO+E1XNxeeWRa0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=cisco.com; spf=pass smtp.mailfrom=cisco.com; dkim=pass (2048-bit key) header.d=cisco.com header.i=@cisco.com header.b=R3Ev4MBj; arc=none smtp.client-ip=173.37.142.95 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=cisco.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=cisco.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=cisco.com header.i=@cisco.com header.b="R3Ev4MBj" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cisco.com; i=@cisco.com; l=8529; q=dns/txt; s=iport01; t=1777807387; x=1779016987; h=from:date:subject:mime-version:content-transfer-encoding: message-id:references:in-reply-to:to:cc; bh=iD3A2YdfU8gmEgjI03DI0UYPW1uFfq1azzwfwXueZ/k=; b=R3Ev4MBjvuBICBkcTMOqM3Z7Nw9SBQhXVnmMK+WCe54QdLMFj8RmgCKI IYBS1yM3P+lDUpUbxfjMGHsCDlp8gzJvWRM6dyc6S9RKAw8R10G/egz/R M0cVhebxwWPLNZrJN24+qH76sfW8TIOti/PqC7ZRdxWwpAbVLXYalFIx+ fePQwj4FqY1INtlao7qCbCUPqFqfIz0JuTGescHn6lYEq2l3i8ISJD/CE 7hi4D1HrLqp13Rgj5KbnVNdvp2IeJaBLK1psJgOOOiclu14UOYtu7/nUJ pCZY6DEyt/b7WOQ8NMWQ3Iqf7Uu0fiMs7lBY4NSaC0MvcVbB7hCmYEu1n g==; X-CSE-ConnectionGUID: jCmLtN+xRtiPdv/eNsoKpA== X-CSE-MsgGUID: 1guhVOy0SdeG5qB4nwAdaw== X-IPAS-Result: =?us-ascii?q?A0BIAgDQLvdp/4v/Ja1aglmCGD+BT0NJhFePU4IhkU2MU?= =?us-ascii?q?BSBaw8BAQEPUQQBAYUGAo0xAiY0CQ4BAgQBAQEBAwIDAQEBAQEBAQEBAQELA?= =?us-ascii?q?QEFAQEBAgEHBYEOE4ZchlsCAQMjBFIQJQImAgJGEAYBEoIqWIJ0A7Qeen8zg?= =?us-ascii?q?QGEfdlDgWQBCxQBgQouhXKCZgGBcIQGO4Q/JxuBSUSBFYNohAUag3+CaQSDH?= =?us-ascii?q?BSEBosVCT+BAhwDWSwBVRMNCgsHBWFSMwMgCgsSEhgVAhQcEw8EFjIdcAwQF?= =?us-ascii?q?xIsFzRYGwcFgUuDAG0UMQQhgQWEYHgjLANOcgMLGA1IESw3Bg4bBD0BbgeKZ?= =?us-ascii?q?h0PgT9qB0swFEI5MQF7BTCTCx2Dfo1da4E1n1mEJoFkoA4BMpdmkwWZBiKjT?= =?us-ascii?q?GuEaIFoPIFZMxoIGxWDIlMZD44tFhzJPCYyPQEBBwIHDQMLgWiQEYFsAQE?= IronPort-Data: A9a23:OwXWta9XYWWapSP1PexxDrUDQH+TJUtcMsCJ2f8bNWPcYEJGY0x3y WRJXGCCbKrca2v3L41/Ptni9UtXucfWyIVlSQQ/+CtEQiMRo6IpJzg2wmQcns+2BpeeJK6yx 5xGMrEsFOhtEDmE4EzrauS9xZVF/fngbqLmD+LZMTxGSwZhSSMw4TpugOdRbrRA2bBVOCvT/ 4mpyyHjEAX9gWAsbzpPs/vrRC5H5ZwehhtJ5jTSWtgT1LPuvyF9JI4SI6i3M0z5TuF8dsamR /zOxa2O5WjQ+REgELuNyt4XpWVTH9Y+lSDX4pZnc/DKbipq/0Te4Y5nXBYoUnq7vh3S9zxHJ HqhgrTrIeshFvWkdO3wyHC0GQkmVUFN0OevzXRSLaV/wmWeG0YAzcmCA2k0Fog85sA0EVtS9 MQUNncASAuEpPqflefTpulE3qzPLeHxN48Z/3UlxjbDALN+G9bIQr7B4plT2zJYasJmRKmFI ZFGL2AyMVKZOEwn1lQ/UPrSmM+zj2X+aD1bqHqepLE85C7YywkZPL3Fb4GPJ43bGpsK9qqej jvJ4mTHJQBCD9CS5mec2C6qmcbDlDyuDer+E5X9rJaGmma7ymUNBRg+WVKlrPy9jUCiHdRSN yQ89yYzqKEg+VCDQd76UBm15nWDu3Y0WMdaGsU55RuLx66S5ByWbkANSDJbZcNlssIqSTE0/ luUmdWvDjwHmKWcQ3+b95+OoD+yMDRTJmgHDQcATA0Y85zgrZs1gxbnUNluCui2g8fzFDW2x CqFxAA4iqkfgNAjyaq25xbEjiiqq5yPSRQ6jjg7RUq/5Q9/IYrgbIuy5B2Dt7BLLZ2SSR+Ku 31sd9Wi0d3ixKqlzESlKNjh1pnwjxpZGFUwWWJSIqQ= IronPort-HdrOrdr: A9a23:qtNa0aFT3ZIWcjalpLqEHseALOsnbusQ8zAXPo5KOHhom6uj9v xG/c506faaslsssR0b8+xofZPwI080lqQb3WBuB9eftWDd0QPCEGgh1/qA/9SKIUHDH4BmpM JdWpk7JdHuAV19lMrgpCu1A78bsaC62ZHtqOvAw3JqVA1wL5th4QBwFxqBHiRNNW977OICZf 2hz/sCgSa8cnIKacn+LHwEU+/f4+DvrvvdEGY779pN0nj1sdtug4SKaCSl4g== X-Talos-CUID: 9a23:MBJS3m/AS+omTLpHcRaVv0w/H+Q5VD7N9mvRG0OmTmtPU7q2UXbFrQ== X-Talos-MUID: 9a23:i/HSFwhGI2tikvg0TJoIGsMpbch145qJGRg2nZgp5tCuaSpvEjuUk2Hi X-IronPort-Anti-Spam-Filtered: true X-IronPort-AV: E=Sophos;i="6.23,213,1770595200"; d="scan'208";a="732599767" Received: from rcdn-l-core-02.cisco.com ([173.37.255.139]) by alln-iport-8.cisco.com with ESMTP/TLS/TLS_AES_256_GCM_SHA384; 03 May 2026 11:23:01 +0000 Received: from [127.0.0.1] (unknown [10.25.128.216]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by rcdn-l-core-02.cisco.com (Postfix) with ESMTPS id 61DC3180007C5; Sun, 3 May 2026 11:23:00 +0000 (GMT) From: Satish Kharat Date: Sun, 03 May 2026 04:22:44 -0700 Subject: [PATCH net-next v6 07/10] enic: add MBOX PF handlers for VF register and capability Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260503-enic-sriov-v2-admin-channel-v2-v6-7-0af4fbc2d86d@cisco.com> References: <20260503-enic-sriov-v2-admin-channel-v2-v6-0-0af4fbc2d86d@cisco.com> In-Reply-To: <20260503-enic-sriov-v2-admin-channel-v2-v6-0-0af4fbc2d86d@cisco.com> To: Andrew Lunn , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni Cc: netdev@vger.kernel.org, linux-kernel@vger.kernel.org, Sesidhar Baddela , Satish Kharat X-Mailer: b4 0.15.2 X-Developer-Signature: v=1; a=ed25519-sha256; t=1777807375; l=8869; i=satishkh@cisco.com; h=from:subject:message-id; bh=3UXiwEguexDSU5q9sXykOpn4fjIRMVeBycb+uHtZMiI=; b=TVwHspcc8cUbUoy4HbJsmvGGRMwt6FSCdvfvy3ZWd5KBpJ4jtNTTTfHwGHYLuxoHmMayJ5dIp NAFfNKIbgnLBROMZHpvOFvIBWx+g2kus4Rs8PDtN2LlEv7Mnfirthcn X-Developer-Key: i=satishkh@cisco.com; a=ed25519; pk=4BxbLoCGvHeIMH/PbpiNVx8/2IcTjFvqcr0AD8K6WLY= X-Outbound-Client-TLS: ANONYMOUS;unknown [10.25.128.216];TLSv1.3;TLS_AES_256_GCM_SHA384;256 X-Outbound-SMTP-Client: 10.25.128.216, [10.25.128.216] X-Outbound-Node: rcdn-l-core-02.cisco.com Implement PF-side mailbox message processing for SR-IOV V2 admin channel communication. When the PF receives messages from VFs, the dispatch routes them to type-specific handlers: - VF_CAPABILITY_REQUEST: reply with protocol version 1 - VF_REGISTER_REQUEST: mark VF registered, reply, then send PF_LINK_STATE_NOTIF with link enabled - VF_UNREGISTER_REQUEST: mark VF unregistered, send reply - PF_LINK_STATE_ACK: log errors from VF acknowledgment Per-VF state (struct enic_vf_state) is tracked via enic->vf_state which will be allocated when SRIOV V2 is enabled. Remove the CONFIG_PCI_IOV guard from num_vfs in struct enic. The PF handlers reference enic->num_vfs for VF ID bounds checking in enic_mbox.c, which is compiled unconditionally. The field must be visible regardless of CONFIG_PCI_IOV to avoid build failures. Add enic_mbox_send_link_state() helper for PF-initiated link state notifications, also used later by ndo_set_vf_link_state. Signed-off-by: Satish Kharat --- drivers/net/ethernet/cisco/enic/enic.h | 7 +- drivers/net/ethernet/cisco/enic/enic_mbox.c | 182 ++++++++++++++++++++++++= +++- drivers/net/ethernet/cisco/enic/enic_mbox.h | 1 + 3 files changed, 186 insertions(+), 4 deletions(-) diff --git a/drivers/net/ethernet/cisco/enic/enic.h b/drivers/net/ethernet/= cisco/enic/enic.h index 42f345aceced..9b1fa3857df5 100644 --- a/drivers/net/ethernet/cisco/enic/enic.h +++ b/drivers/net/ethernet/cisco/enic/enic.h @@ -256,9 +256,7 @@ struct enic { struct enic_rx_coal rx_coalesce_setting; u32 rx_coalesce_usecs; u32 tx_coalesce_usecs; -#ifdef CONFIG_PCI_IOV u16 num_vfs; -#endif enum enic_vf_type vf_type; unsigned int enable_count; spinlock_t enic_api_lock; @@ -310,6 +308,11 @@ struct enic { /* MBOX protocol state */ struct mutex mbox_lock; u64 mbox_msg_num; + + /* PF: per-VF MBOX state, allocated when SRIOV V2 is enabled */ + struct enic_vf_state { + bool registered; + } *vf_state; }; =20 static inline struct net_device *vnic_get_netdev(struct vnic_dev *vdev) diff --git a/drivers/net/ethernet/cisco/enic/enic_mbox.c b/drivers/net/ethe= rnet/cisco/enic/enic_mbox.c index 3b2fc9822176..f3c0e94c2417 100644 --- a/drivers/net/ethernet/cisco/enic/enic_mbox.c +++ b/drivers/net/ethernet/cisco/enic/enic_mbox.c @@ -127,10 +127,175 @@ int enic_mbox_send_msg(struct enic *enic, u8 msg_typ= e, u16 dst_vnic_id, return err; } =20 +int enic_mbox_send_link_state(struct enic *enic, u16 vf_id, u32 link_state) +{ + struct enic_mbox_pf_link_state_notif_msg notif =3D {}; + + if (!enic->vf_state || vf_id >=3D enic->num_vfs || + !enic->vf_state[vf_id].registered) { + netdev_dbg(enic->netdev, + "MBOX: skip link state to unregistered VF %u\n", + vf_id); + return 0; + } + + notif.link_state =3D cpu_to_le32(link_state); + return enic_mbox_send_msg(enic, ENIC_MBOX_PF_LINK_STATE_NOTIF, vf_id, + ¬if, sizeof(notif)); +} + +static int enic_mbox_pf_handle_capability(struct enic *enic, void *msg, + u16 vf_id, u64 msg_num) +{ + struct enic_mbox_vf_capability_reply_msg reply =3D {}; + + reply.reply.ret_major =3D cpu_to_le16(0); + reply.version =3D cpu_to_le32(ENIC_MBOX_CAP_VERSION_1); + + return enic_mbox_send_msg(enic, ENIC_MBOX_VF_CAPABILITY_REPLY, vf_id, + &reply, sizeof(reply)); +} + +static int enic_mbox_pf_handle_register(struct enic *enic, void *msg, + u16 vf_id, u64 msg_num) +{ + struct enic_mbox_vf_register_reply_msg reply =3D {}; + int err; + + if (!enic->vf_state || vf_id >=3D enic->num_vfs) { + if (net_ratelimit()) + netdev_warn(enic->netdev, + "MBOX: register from invalid VF %u\n", + vf_id); + return -EINVAL; + } + + /* VF re-registering (e.g. guest reboot without clean unregister): + * mark the previous registration inactive before accepting the new one. + */ + if (enic->vf_state[vf_id].registered) { + netdev_dbg(enic->netdev, + "MBOX: VF %u re-register, cleaning previous state\n", + vf_id); + enic->vf_state[vf_id].registered =3D false; + } + + reply.reply.ret_major =3D cpu_to_le16(0); + err =3D enic_mbox_send_msg(enic, ENIC_MBOX_VF_REGISTER_REPLY, vf_id, + &reply, sizeof(reply)); + if (err) + return err; + + enic->vf_state[vf_id].registered =3D true; + netdev_info(enic->netdev, "VF %u registered via MBOX\n", vf_id); + + err =3D enic_mbox_send_link_state(enic, vf_id, + ENIC_MBOX_LINK_STATE_ENABLE); + if (err) + netdev_warn(enic->netdev, + "VF %u: failed to send initial link state: %d\n", + vf_id, err); + /* Registration succeeded; link state will be (re-)sent on next + * enic_link_check() event. + */ + return 0; +} + +static int enic_mbox_pf_handle_unregister(struct enic *enic, void *msg, + u16 vf_id, u64 msg_num) +{ + struct enic_mbox_vf_register_reply_msg reply =3D {}; + int err; + + if (!enic->vf_state || vf_id >=3D enic->num_vfs) { + if (net_ratelimit()) + netdev_warn(enic->netdev, + "MBOX: unregister from invalid VF %u\n", + vf_id); + return -EINVAL; + } + + /* VF is unloading; clear local state regardless of whether + * the reply is successfully delivered to avoid the PF treating + * a dead VF as still registered. + */ + enic->vf_state[vf_id].registered =3D false; + + reply.reply.ret_major =3D cpu_to_le16(0); + err =3D enic_mbox_send_msg(enic, ENIC_MBOX_VF_UNREGISTER_REPLY, vf_id, + &reply, sizeof(reply)); + + netdev_info(enic->netdev, "VF %u unregistered via MBOX\n", vf_id); + + return err; +} + +static void enic_mbox_pf_process_msg(struct enic *enic, + struct enic_mbox_hdr *hdr, void *payload) +{ + u16 vf_id =3D le16_to_cpu(hdr->src_vnic_id); + u16 msg_len =3D le16_to_cpu(hdr->msg_len); + int err =3D 0; + + if (!enic->vf_state) { + netdev_dbg(enic->netdev, + "MBOX: PF received msg but SRIOV not active\n"); + return; + } + + if (vf_id >=3D enic->num_vfs) { + if (net_ratelimit()) + netdev_warn(enic->netdev, + "MBOX: PF received msg from invalid VF %u\n", + vf_id); + return; + } + + switch (hdr->msg_type) { + case ENIC_MBOX_VF_CAPABILITY_REQUEST: + err =3D enic_mbox_pf_handle_capability(enic, payload, vf_id, + le64_to_cpu(hdr->msg_num)); + break; + case ENIC_MBOX_VF_REGISTER_REQUEST: + err =3D enic_mbox_pf_handle_register(enic, payload, vf_id, + le64_to_cpu(hdr->msg_num)); + break; + case ENIC_MBOX_VF_UNREGISTER_REQUEST: + err =3D enic_mbox_pf_handle_unregister(enic, payload, vf_id, + le64_to_cpu(hdr->msg_num)); + break; + case ENIC_MBOX_PF_LINK_STATE_ACK: { + struct enic_mbox_pf_link_state_ack_msg *ack =3D payload; + + if (msg_len < sizeof(*hdr) + sizeof(*ack)) + break; + if (le16_to_cpu(ack->ack.ret_major)) + netdev_warn(enic->netdev, + "MBOX: VF %u link state ACK error %u/%u\n", + vf_id, le16_to_cpu(ack->ack.ret_major), + le16_to_cpu(ack->ack.ret_minor)); + break; + } + default: + netdev_dbg(enic->netdev, + "MBOX: PF unhandled msg type %u from VF %u\n", + hdr->msg_type, vf_id); + err =3D -EOPNOTSUPP; + break; + } + + if (err) + netdev_warn(enic->netdev, + "MBOX: PF handler for msg type %u from VF %u failed: %d\n", + hdr->msg_type, vf_id, err); +} + static void enic_mbox_recv_handler(struct enic *enic, void *buf, unsigned int len) { struct enic_mbox_hdr *hdr =3D buf; + void *payload; + u16 msg_len; =20 if (len < sizeof(*hdr)) { if (net_ratelimit()) @@ -148,10 +313,23 @@ static void enic_mbox_recv_handler(struct enic *enic,= void *buf, return; } =20 + msg_len =3D le16_to_cpu(hdr->msg_len); + if (msg_len < sizeof(*hdr) || msg_len > len) { + if (net_ratelimit()) + netdev_warn(enic->netdev, + "MBOX: invalid msg_len %u (buf len %u)\n", + msg_len, len); + return; + } + netdev_dbg(enic->netdev, "MBOX recv: type %u from vnic %u len %u\n", - hdr->msg_type, le16_to_cpu(hdr->src_vnic_id), - le16_to_cpu(hdr->msg_len)); + hdr->msg_type, le16_to_cpu(hdr->src_vnic_id), msg_len); + + payload =3D buf + sizeof(*hdr); + + if (enic->vf_state) + enic_mbox_pf_process_msg(enic, hdr, payload); } =20 void enic_mbox_init(struct enic *enic) diff --git a/drivers/net/ethernet/cisco/enic/enic_mbox.h b/drivers/net/ethe= rnet/cisco/enic/enic_mbox.h index 73fd7f783ee2..f1de67db1273 100644 --- a/drivers/net/ethernet/cisco/enic/enic_mbox.h +++ b/drivers/net/ethernet/cisco/enic/enic_mbox.h @@ -87,5 +87,6 @@ struct enic; void enic_mbox_init(struct enic *enic); int enic_mbox_send_msg(struct enic *enic, u8 msg_type, u16 dst_vnic_id, void *payload, u16 payload_len); +int enic_mbox_send_link_state(struct enic *enic, u16 vf_id, u32 link_state= ); =20 #endif /* _ENIC_MBOX_H_ */ --=20 2.43.0 From nobody Sun Jun 14 04:21:27 2026 Received: from alln-iport-8.cisco.com (alln-iport-8.cisco.com [173.37.142.95]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 69B583126C0; Sun, 3 May 2026 11:23:09 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=173.37.142.95 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777807391; cv=none; b=r10i6ZuRReNxB7gWFp9WY0jYDGqTqnwIptP7hSRKyE6TYis51VD+PMLHXFhxc/jV1exPWYmIGOuxmgR8h5NNrh1VoZOiA7vLVyhrwq0KNGuVdjB5upPf2Z0t7lMPY9ZKFHBiy9kAc+YfyaO0dF/mJ4cxPIUKiWm3ryNH2RlCAIg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777807391; c=relaxed/simple; bh=j8jcnV6R/vBeZYaBxbZgAebIz9HVZjEefr0QryF5/TU=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=TFGgd3MmRjU/SMLV1JKVZUumzlwXGUtUOBxQuYnvkumFsxuQ4JUKuls7whVd51BAuBv5T2sIQ7MFlz4AkA3nrslr7uqEoF0MtCEwAa5pkMEczkcuuMnqcKpC3GigfRZKotpfSVxvJiyjtR8+ggolMfYBQtCOfzEWe3FWXbB7OyA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=cisco.com; spf=pass smtp.mailfrom=cisco.com; dkim=pass (2048-bit key) header.d=cisco.com header.i=@cisco.com header.b=M4cXih7r; arc=none smtp.client-ip=173.37.142.95 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=cisco.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=cisco.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=cisco.com header.i=@cisco.com header.b="M4cXih7r" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cisco.com; i=@cisco.com; l=11264; q=dns/txt; s=iport01; t=1777807389; x=1779016989; h=from:date:subject:mime-version:content-transfer-encoding: message-id:references:in-reply-to:to:cc; bh=8w/FShKlryzv94gx8ZTG6swlZLJNnDreKhG8F9f16+8=; b=M4cXih7r1RAA/9eOd5JizHQjSwNgr/A4XIQEFt3EewE/kVHT/kC+KyAx giHuwQBxHFtsGaQ76WjUGbwwMlwn/nzMohU34LXvx5BM4hdCKSmc5aGN3 3ZnHEc55JoDCu2j1fdmBdFINsqgo9CNZqJmV8ZadC/gAw5KS6Q4f7DaPZ W58aWOz2btsrk9jRXkveEOXxYbQuv51xYeW4z+GlWhGSqczEy4WbTyBmj 5mJ1kW1EnNxYKKx2+5dUYvTW3CKlpC9zA4tKbjrBAFEZs+A5DFAQUvQeb rKJtu+eWHVoEyeW9YBuBCGxJ4pL+LStl0gM4Lm3Qtm5q39EiB19P4tTsn g==; X-CSE-ConnectionGUID: 8GSCGoNySvaYkCJ3/9Z2jQ== X-CSE-MsgGUID: XLeIEepvQx252Lr6ZbK90w== X-IPAS-Result: =?us-ascii?q?A0AaAADQLvdp/4v/Ja1aHAEBAQEBAQcBARIBAQQEAQGBf?= =?us-ascii?q?AcBAQsBglaBT0NJhFeIHIc3giGBFp0HFIFrDwEBAQ9RBAEBhQYCjTECJjQJD?= =?us-ascii?q?gECBAEBAQEDAgMBAQEBAQEBAQEBAQsBAQUBAQECAQcFgQ4ThlyGWwIBAyMEQ?= =?us-ascii?q?BIQJQImAgJGEAYBEoMCgnQDtB56fzOBAd5AgWQBCxQBgQouAYVxgmYBgXCEB?= =?us-ascii?q?juEPycbgUlEgRWCcnaEBQUVg3+CaQSCDYEPFI8bCT+BAhwDWSwBVRMNCgsHB?= =?us-ascii?q?WFSMwMgCgsSEhgVAhQvDwQWMh1wDBAXEiwXNFgbBwWBS4MAbRQxBCGBBYRge?= =?us-ascii?q?CMsA05yAwsYDUgRLDcGDhsEPQFuB4pmHQ+BP2oHFidSezFnFQUwCZNhgzyNX?= =?us-ascii?q?WuBNZ9ZhCaBZKAOATKEBJNikwWZBiKCNqExUIRogWg8gVkzGggbFYMiUxkPj?= =?us-ascii?q?i0WHMk8JjI9AQEHAgcNAwuBaJAAEReBVQEB?= IronPort-Data: A9a23:MI7GtK/JCTUmZ4CKQPfkDrUDQH+TJUtcMsCJ2f8bNWPcYEJGY0x3y GYdCzuCbPffMWHzfNpxOYWw8BlX6JHVydM1GlFoqnhEQiMRo6IpJzg2wmQcns+2BpeeJK6yx 5xGMrEsFOhtEDmE4EzrauS9xZVF/fngbqLmD+LZMTxGSwZhSSMw4TpugOdRbrRA2bBVOCvT/ 4mpyyHjEAX9gWAsbzpPs/vrRC5H5ZwehhtJ5jTSWtgT1LPuvyF9JI4SI6i3M0z5TuF8dsamR /zOxa2O5WjQ+REgELuNyt4XpWVTH9Y+lSDX4pZnc/DKbipq/0Te4Y5nXBYoUnq7vh3S9zxHJ HqhgrTrIeshFvWkdO3wyHC0GQkmVUFN0OevzXRSLaV/wmWeG0YAzcmCA2lmErYz1eFREFpQ/ M0bAxJWKUGx3eCPlefTpulE3qzPLeHxN48Z/3UlxjbDALN+G9bIQr7B4plT2zJYasJmRKmFI ZFGL2AyMVKZOEwn1lQ/UPrSmM+zj2X+aD1bqHqepLE85C7YywkZPL3Fb4GPJ43bGpoP9qqej lLi8kXUDwkEDczFlGCh6CyejMyexTyuDer+E5X9rJaGmma7ymUNBRg+WVKlrPy9jUCiHdRSN yQ89yYzqKEg+VCDQd76UBm15nWDu3Y0WMdaGsU55RuLx66S5ByWbkANSDJbZcNlssIqSTE0/ luUmdWvDjwHmKWcQ3+b95+OoD+yMDRTJmgHDQcATA0Y85zgrZs1gxbnUNluCui2g8fzFDW2x CqFxAA4iqkfgNAjyaq25xbEjiiqq5yPSRQ6jjg7RUq/5Q9/IYrgbIuy5B2Dt7BLLZ2SSR+Ku 31sd9Wi0d3ixKqlzESlKNjh1pnwjxpZGFUwWWJSIqQ= IronPort-HdrOrdr: A9a23:Pk+jVak52INoU8mO4D8fiR2MpVLpDfLa3DAbv31ZSRFFG/FwWf re+MjzsiWUtN9xYh8dcL+7VZVoLUm3yXcx2/hyAV7AZmbbURWTTL2KhLGKq1bd8m/Fh41gPM xbH5SWZueAaGRSvILW7Ba1FtA4zMLC0qy57N2utktFfEVGZ75t6gFhBh3eKE15QwFaGYE0fa DsgPav3wDQHkg/X4CcGmQPWfTFqpngkZLrZgNDOjscgTP+9Q9BLNXBYmKlNtB0aUI1/Ysf X-Talos-CUID: =?us-ascii?q?9a23=3A0vEQpWse5GESIQMNh/JF4khf6It1XUb243D2MnS?= =?us-ascii?q?mAFdqY5/ODg6ioqFdxp8=3D?= X-Talos-MUID: 9a23:OSlCYgnha1mCzZ9iAx1ndnpSEMpJzqm2Fnkdy8hcn9PaByZTam+C2WE= X-IronPort-Anti-Spam-Filtered: true X-IronPort-AV: E=Sophos;i="6.23,213,1770595200"; d="scan'208";a="732599772" Received: from rcdn-l-core-02.cisco.com ([173.37.255.139]) by alln-iport-8.cisco.com with ESMTP/TLS/TLS_AES_256_GCM_SHA384; 03 May 2026 11:23:01 +0000 Received: from [127.0.0.1] (unknown [10.25.128.216]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by rcdn-l-core-02.cisco.com (Postfix) with ESMTPS id 1E608180007D0; Sun, 3 May 2026 11:23:01 +0000 (GMT) From: Satish Kharat Date: Sun, 03 May 2026 04:22:45 -0700 Subject: [PATCH net-next v6 08/10] enic: add MBOX VF handlers for capability, register and link state Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260503-enic-sriov-v2-admin-channel-v2-v6-8-0af4fbc2d86d@cisco.com> References: <20260503-enic-sriov-v2-admin-channel-v2-v6-0-0af4fbc2d86d@cisco.com> In-Reply-To: <20260503-enic-sriov-v2-admin-channel-v2-v6-0-0af4fbc2d86d@cisco.com> To: Andrew Lunn , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni Cc: netdev@vger.kernel.org, linux-kernel@vger.kernel.org, Sesidhar Baddela , Satish Kharat X-Mailer: b4 0.15.2 X-Developer-Signature: v=1; a=ed25519-sha256; t=1777807375; l=11669; i=satishkh@cisco.com; h=from:subject:message-id; bh=j8jcnV6R/vBeZYaBxbZgAebIz9HVZjEefr0QryF5/TU=; b=Xeej+NpjOYpRxAtQ14XFeKWRhzQiB9paC5/nfmicB6XP/e1xgDW4L7SqQOZwUEvVTblP9578f 0szD/9PBPeNBoJZ3pZfStUpUZgXkeX/47tgCtjfJAH9kJFPRCx0QMZr X-Developer-Key: i=satishkh@cisco.com; a=ed25519; pk=4BxbLoCGvHeIMH/PbpiNVx8/2IcTjFvqcr0AD8K6WLY= X-Outbound-Client-TLS: ANONYMOUS;unknown [10.25.128.216];TLSv1.3;TLS_AES_256_GCM_SHA384;256 X-Outbound-SMTP-Client: 10.25.128.216, [10.25.128.216] X-Outbound-Node: rcdn-l-core-02.cisco.com Implement VF-side mailbox message processing for SR-IOV V2 admin channel communication. VF receive handlers: - VF_CAPABILITY_REPLY: store PF protocol version, signal completion - VF_REGISTER_REPLY: mark VF as registered, signal completion - VF_UNREGISTER_REPLY: mark VF as unregistered, signal completion - PF_LINK_STATE_NOTIF: update carrier state via netif_carrier_on/off, send ACK back to PF VF initiation functions for the probe-time handshake: - enic_mbox_vf_capability_check: send capability request, wait for PF reply via completion - enic_mbox_vf_register: send register request, wait for PF confirmation via completion - enic_mbox_vf_unregister: send unregister request, wait for PF confirmation The wait helper (enic_mbox_wait_reply) uses wait_for_completion_timeout, signaled when the admin ISR/NAPI/ workqueue pipeline delivers the reply message. Signed-off-by: Satish Kharat --- drivers/net/ethernet/cisco/enic/enic.h | 17 +- drivers/net/ethernet/cisco/enic/enic_mbox.c | 256 ++++++++++++++++++++++++= ++++ drivers/net/ethernet/cisco/enic/enic_mbox.h | 3 + 3 files changed, 275 insertions(+), 1 deletion(-) diff --git a/drivers/net/ethernet/cisco/enic/enic.h b/drivers/net/ethernet/= cisco/enic/enic.h index 9b1fa3857df5..483053c781df 100644 --- a/drivers/net/ethernet/cisco/enic/enic.h +++ b/drivers/net/ethernet/cisco/enic/enic.h @@ -258,6 +258,8 @@ struct enic { u32 tx_coalesce_usecs; u16 num_vfs; enum enic_vf_type vf_type; + bool vf_registered; + u32 pf_cap_version; unsigned int enable_count; spinlock_t enic_api_lock; bool enic_api_busy; @@ -305,9 +307,22 @@ struct enic { void (*admin_rq_handler)(struct enic *enic, void *buf, unsigned int len); =20 - /* MBOX protocol state */ + /* MBOX protocol state -- single-flight: on the VF, all callers + * that wait on mbox_comp run under RTNL or during probe/remove, + * so only one completion is outstanding at a time. mbox_lock + * protects the shared admin WQ from concurrent senders. + */ struct mutex mbox_lock; u64 mbox_msg_num; + struct completion mbox_comp; + /* Type of reply the current waiter on mbox_comp expects. Set + * under mbox_lock before reinit_completion(); cleared after + * wait_reply returns. Reply handlers compare against the + * incoming reply type and drop stale replies from previously + * timed-out requests instead of waking the unrelated current + * waiter. + */ + u8 mbox_expected_reply; =20 /* PF: per-VF MBOX state, allocated when SRIOV V2 is enabled */ struct enic_vf_state { diff --git a/drivers/net/ethernet/cisco/enic/enic_mbox.c b/drivers/net/ethe= rnet/cisco/enic/enic_mbox.c index f3c0e94c2417..7680baece802 100644 --- a/drivers/net/ethernet/cisco/enic/enic_mbox.c +++ b/drivers/net/ethernet/cisco/enic/enic_mbox.c @@ -5,6 +5,7 @@ #include #include #include +#include =20 #include "vnic_dev.h" #include "vnic_wq.h" @@ -127,6 +128,16 @@ int enic_mbox_send_msg(struct enic *enic, u8 msg_type,= u16 dst_vnic_id, return err; } =20 +static int enic_mbox_wait_reply(struct enic *enic, unsigned long timeout_m= s) +{ + unsigned long left; + + left =3D wait_for_completion_timeout(&enic->mbox_comp, + msecs_to_jiffies(timeout_ms)); + + return left ? 0 : -ETIMEDOUT; +} + int enic_mbox_send_link_state(struct enic *enic, u16 vf_id, u32 link_state) { struct enic_mbox_pf_link_state_notif_msg notif =3D {}; @@ -290,6 +301,157 @@ static void enic_mbox_pf_process_msg(struct enic *eni= c, hdr->msg_type, vf_id, err); } =20 +static void enic_mbox_vf_handle_capability_reply(struct enic *enic, + void *payload) +{ + struct enic_mbox_vf_capability_reply_msg *reply =3D payload; + + if (enic->mbox_expected_reply !=3D ENIC_MBOX_VF_CAPABILITY_REPLY) { + netdev_warn(enic->netdev, + "MBOX: stale capability reply (expected %u), drop\n", + enic->mbox_expected_reply); + return; + } + + if (le16_to_cpu(reply->reply.ret_major) =3D=3D 0) + enic->pf_cap_version =3D le32_to_cpu(reply->version); + complete(&enic->mbox_comp); +} + +static void enic_mbox_vf_handle_register_reply(struct enic *enic, + void *payload) +{ + struct enic_mbox_vf_register_reply_msg *reply =3D payload; + + if (enic->mbox_expected_reply !=3D ENIC_MBOX_VF_REGISTER_REPLY) { + netdev_warn(enic->netdev, + "MBOX: stale register reply (expected %u), drop\n", + enic->mbox_expected_reply); + return; + } + + if (le16_to_cpu(reply->reply.ret_major)) { + netdev_warn(enic->netdev, + "MBOX: VF register rejected by PF: %u/%u\n", + le16_to_cpu(reply->reply.ret_major), + le16_to_cpu(reply->reply.ret_minor)); + } else { + enic->vf_registered =3D true; + } + complete(&enic->mbox_comp); +} + +static void enic_mbox_vf_handle_unregister_reply(struct enic *enic, + void *payload) +{ + struct enic_mbox_vf_register_reply_msg *reply =3D payload; + + if (enic->mbox_expected_reply !=3D ENIC_MBOX_VF_UNREGISTER_REPLY) { + netdev_warn(enic->netdev, + "MBOX: stale unregister reply (expected %u), drop\n", + enic->mbox_expected_reply); + return; + } + + if (le16_to_cpu(reply->reply.ret_major)) { + netdev_warn(enic->netdev, + "MBOX: VF unregister rejected by PF: %u/%u\n", + le16_to_cpu(reply->reply.ret_major), + le16_to_cpu(reply->reply.ret_minor)); + } else { + enic->vf_registered =3D false; + } + complete(&enic->mbox_comp); +} + +static void enic_mbox_vf_handle_link_state(struct enic *enic, void *payloa= d) +{ + struct enic_mbox_pf_link_state_notif_msg *notif =3D payload; + struct enic_mbox_pf_link_state_ack_msg ack =3D {}; + + switch (le32_to_cpu(notif->link_state)) { + case ENIC_MBOX_LINK_STATE_ENABLE: + if (!netif_carrier_ok(enic->netdev)) + netif_carrier_on(enic->netdev); + netdev_dbg(enic->netdev, "MBOX: link state -> UP\n"); + break; + case ENIC_MBOX_LINK_STATE_DISABLE: + if (netif_carrier_ok(enic->netdev)) + netif_carrier_off(enic->netdev); + netdev_dbg(enic->netdev, "MBOX: link state -> DOWN\n"); + break; + default: + netdev_warn(enic->netdev, "MBOX: unknown link state %u\n", + le32_to_cpu(notif->link_state)); + ack.ack.ret_major =3D cpu_to_le16(ENIC_MBOX_ERR_GENERIC); + break; + } + + enic_mbox_send_msg(enic, ENIC_MBOX_PF_LINK_STATE_ACK, ENIC_MBOX_DST_PF, + &ack, sizeof(ack)); +} + +static bool enic_mbox_vf_payload_ok(struct enic *enic, u8 msg_type, + u16 payload_len, size_t min_len) +{ + if (payload_len < min_len) { + netdev_warn(enic->netdev, + "MBOX: short payload for type %u (%u < %zu)\n", + msg_type, payload_len, min_len); + return false; + } + return true; +} + +static void enic_mbox_vf_process_msg(struct enic *enic, + struct enic_mbox_hdr *hdr, void *payload, + u16 payload_len) +{ + switch (hdr->msg_type) { + case ENIC_MBOX_VF_CAPABILITY_REPLY: { + size_t exp =3D sizeof(struct enic_mbox_vf_capability_reply_msg); + + if (!enic_mbox_vf_payload_ok(enic, hdr->msg_type, + payload_len, exp)) + return; + enic_mbox_vf_handle_capability_reply(enic, payload); + break; + } + case ENIC_MBOX_VF_REGISTER_REPLY: { + size_t exp =3D sizeof(struct enic_mbox_vf_register_reply_msg); + + if (!enic_mbox_vf_payload_ok(enic, hdr->msg_type, + payload_len, exp)) + return; + enic_mbox_vf_handle_register_reply(enic, payload); + break; + } + case ENIC_MBOX_VF_UNREGISTER_REPLY: { + size_t exp =3D sizeof(struct enic_mbox_vf_register_reply_msg); + + if (!enic_mbox_vf_payload_ok(enic, hdr->msg_type, + payload_len, exp)) + return; + enic_mbox_vf_handle_unregister_reply(enic, payload); + break; + } + case ENIC_MBOX_PF_LINK_STATE_NOTIF: { + size_t exp =3D sizeof(struct enic_mbox_pf_link_state_notif_msg); + + if (!enic_mbox_vf_payload_ok(enic, hdr->msg_type, + payload_len, exp)) + return; + enic_mbox_vf_handle_link_state(enic, payload); + break; + } + default: + netdev_dbg(enic->netdev, + "MBOX: VF unhandled msg type %u\n", + hdr->msg_type); + break; + } +} + static void enic_mbox_recv_handler(struct enic *enic, void *buf, unsigned int len) { @@ -330,11 +492,105 @@ static void enic_mbox_recv_handler(struct enic *enic= , void *buf, =20 if (enic->vf_state) enic_mbox_pf_process_msg(enic, hdr, payload); + else + enic_mbox_vf_process_msg(enic, hdr, payload, + msg_len - (u16)sizeof(*hdr)); +} + +int enic_mbox_vf_capability_check(struct enic *enic) +{ + struct enic_mbox_vf_capability_msg req =3D {}; + int err; + + enic->pf_cap_version =3D 0; + enic->mbox_expected_reply =3D ENIC_MBOX_VF_CAPABILITY_REPLY; + reinit_completion(&enic->mbox_comp); + req.version =3D cpu_to_le32(ENIC_MBOX_CAP_VERSION_1); + + err =3D enic_mbox_send_msg(enic, ENIC_MBOX_VF_CAPABILITY_REQUEST, + ENIC_MBOX_DST_PF, &req, sizeof(req)); + if (err) { + enic->mbox_expected_reply =3D 0; + return err; + } + + err =3D enic_mbox_wait_reply(enic, 3000); + enic->mbox_expected_reply =3D 0; + if (err) { + netdev_warn(enic->netdev, + "MBOX: no capability reply from PF\n"); + return err; + } + + if (enic->pf_cap_version < ENIC_MBOX_CAP_VERSION_1) { + netdev_warn(enic->netdev, + "MBOX: PF version %u too old\n", + enic->pf_cap_version); + return -EOPNOTSUPP; + } + + return 0; +} + +int enic_mbox_vf_register(struct enic *enic) +{ + int err; + + enic->vf_registered =3D false; + enic->mbox_expected_reply =3D ENIC_MBOX_VF_REGISTER_REPLY; + reinit_completion(&enic->mbox_comp); + + err =3D enic_mbox_send_msg(enic, ENIC_MBOX_VF_REGISTER_REQUEST, + ENIC_MBOX_DST_PF, NULL, 0); + if (err) { + enic->mbox_expected_reply =3D 0; + return err; + } + + err =3D enic_mbox_wait_reply(enic, 3000); + enic->mbox_expected_reply =3D 0; + if (err) { + netdev_warn(enic->netdev, + "MBOX: VF registration with PF timed out\n"); + return err; + } + + if (!enic->vf_registered) + return -ENODEV; + + return 0; +} + +int enic_mbox_vf_unregister(struct enic *enic) +{ + int err; + + if (!enic->vf_registered) + return 0; + + enic->mbox_expected_reply =3D ENIC_MBOX_VF_UNREGISTER_REPLY; + reinit_completion(&enic->mbox_comp); + + err =3D enic_mbox_send_msg(enic, ENIC_MBOX_VF_UNREGISTER_REQUEST, + ENIC_MBOX_DST_PF, NULL, 0); + if (err) { + enic->mbox_expected_reply =3D 0; + return err; + } + + err =3D enic_mbox_wait_reply(enic, 3000); + enic->mbox_expected_reply =3D 0; + if (err) + return err; + if (enic->vf_registered) + return -EACCES; + return 0; } =20 void enic_mbox_init(struct enic *enic) { enic->mbox_msg_num =3D 0; mutex_init(&enic->mbox_lock); + init_completion(&enic->mbox_comp); enic->admin_rq_handler =3D enic_mbox_recv_handler; } diff --git a/drivers/net/ethernet/cisco/enic/enic_mbox.h b/drivers/net/ethe= rnet/cisco/enic/enic_mbox.h index f1de67db1273..15e30ee2b0ed 100644 --- a/drivers/net/ethernet/cisco/enic/enic_mbox.h +++ b/drivers/net/ethernet/cisco/enic/enic_mbox.h @@ -88,5 +88,8 @@ void enic_mbox_init(struct enic *enic); int enic_mbox_send_msg(struct enic *enic, u8 msg_type, u16 dst_vnic_id, void *payload, u16 payload_len); int enic_mbox_send_link_state(struct enic *enic, u16 vf_id, u32 link_state= ); +int enic_mbox_vf_capability_check(struct enic *enic); +int enic_mbox_vf_register(struct enic *enic); +int enic_mbox_vf_unregister(struct enic *enic); =20 #endif /* _ENIC_MBOX_H_ */ --=20 2.43.0 From nobody Sun Jun 14 04:21:27 2026 Received: from alln-iport-5.cisco.com (alln-iport-5.cisco.com [173.37.142.92]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 978B8347FFE; Sun, 3 May 2026 11:24:10 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=173.37.142.92 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777807454; cv=none; b=mCOV4FC1b1cZWWkCaualpBMwj9lCqz8mIx4FRB8WG+7egbHcKvNjvomKZHvtoCufmEGK3PvRx2xY97txPqlf+59ctjgCR+miAO/eOFYuaVurr7+pSZwoh8NEt+W/XT8HqNbpWemDDlgcFT79pc7k0VKkJshXEmTBu+ZPL7Wc5BQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777807454; c=relaxed/simple; bh=98HiIa1W+ZDwSMfxvLFC1y+jPE25eZ8OwNCbntSWJX0=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=LOHj/vYTB4zWkDuOZzNIkklUDC7k5JFQgB1yE6UlmvWQixhOUeEdmhLd+ZrI95a/oS7fe/auqw7c7psKsQmuugxhcAgZsuPg1I0bDZV1w6FcFnI72GrknWVxfKqw5PrlGP+EXaTd6PMQlHGNwOGUCFkfDCG21CFTozjnHlFF9xU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=cisco.com; spf=pass smtp.mailfrom=cisco.com; dkim=pass (2048-bit key) header.d=cisco.com header.i=@cisco.com header.b=NKqpz1wa; arc=none smtp.client-ip=173.37.142.92 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=cisco.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=cisco.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=cisco.com header.i=@cisco.com header.b="NKqpz1wa" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cisco.com; i=@cisco.com; l=9758; q=dns/txt; s=iport01; t=1777807450; x=1779017050; h=from:date:subject:mime-version:content-transfer-encoding: message-id:references:in-reply-to:to:cc; bh=wr9E2hgyq5hI01dxAyTO4C42w+m31gJequVXo2bhuqU=; b=NKqpz1waYSZMKu3BjPU1dMjmonS37iImF9uYimg54srd/KEGNPr4IUye QkQiOuiWe42tN2Dqbm6RLUqQfqNLmsPxI6FfldEn9YE3ifrD1eOG+RstG 6GNHlEzp7rnuaS/MkM/zaMzi/zkTyi0wdI7MKhBhhVzimrQpx3mSowklS oy0I+BEEPzue93NJDt7skS7NvzkdwiiypD84O3c0Hq+krpaU54XafnRZ8 MsMXKKN9XT1wokb5K74Y9V8d8jk4Nr8kJVPjstW+Cx5Ya9bG9TBWtISvK OlD5CH1TXyxLjrEiulbZpP4abVbCUGtcbZD6w1EUlNYSHwC6DGT43qjJS w==; X-CSE-ConnectionGUID: B43om+E5T5mKVBR7fSFlTg== X-CSE-MsgGUID: WTFrMjVbRyy4Z5X1dLckcA== X-IPAS-Result: =?us-ascii?q?A0AaAADQLvdp/4v/Ja1aHAEBAQEBAQcBARIBAQQEAQGBf?= =?us-ascii?q?AcBAQsBghc/gU9DSYRXiByHN4Ihnh0UgWsPAQEBD1EEAQGFBgKNMQImNAkOA?= =?us-ascii?q?QIEAQEBAQMCAwEBAQEBAQEBAQEBCwEBBQEBAQIBBwWBDhOGXIZbAgEDIwRSE?= =?us-ascii?q?CUCJgICRhAGARKDAoJ0A7Qeen8zgQGEfdlDgWQBCxQBgQouAYVxgmYBgXCEB?= =?us-ascii?q?juEPycbgUlEgRWDaIEogwKDdIJpBIMcFI8bCT+BAhwDWSwBVRMNCgsHBWFSM?= =?us-ascii?q?wMgCgsSEhgVAhQvDwQWMh1wDCcSLBc0WBsHBYFLgwBtFDEEIYEFhGB4IywDT?= =?us-ascii?q?nIDCxgNSBEsNwYOGwQ9AW4HimYdD4E2CXEOSxwaB4EpDD8UHwIUk0EHFoNhj?= =?us-ascii?q?V1rgTWfWYQmgWSgDgEyhASTYpMFmQYigjaiAYRogWg8gVkzGggbFYMiUxkPj?= =?us-ascii?q?i0WHHgBAshBJjI+AQcCBw4CgXOQAASBeQEB?= IronPort-Data: A9a23:nopz3Ktpt9irghr7M8kjcCiINOfnVJZfMUV32f8akzHdYApBsoF/q tZmKWiFa/2LZDDyeN10bI+woUhQ6MWGndRgS1ds+ypjQyoVgMeUXt7xwmUckM+xwmwvaGo9s q3yv/GZdJhcokf0/0nrav666yEgiclkf5KkYMbcICd9WAR4fykojBNnioYRj5Vh6TSDK1vlV eja/YuFYzdJ5xYuajhKs//b80s21BjPkGpwUmIWNKgjUGD2zxH5PLpHTYmtIn3xRJVjH+LSb 47r0LGj82rFyAwmA9Wjn6yTWhVirmn6ZFXmZtJ+AsBOszAazsAA+v9T2Mk0NS+7vw60c+VZk 72hg3AfpTABZcUgkMxFO/VR/roX0aduoNcrKlDn2SCfItGvn3bEm51T4E8K0YIw0LosBmQN2 KEkdTUuUDq52OyN7raHRbw57igjBJGD0II3oHpsy3TdSP0hW52GG/+M7t5D1zB2jcdLdRrcT 5NGMnw0M1KaPkAJYwxGYH49tL/Aan3XaDREqUmVpqof6GnIxws327/oWDbQUoPRH5QNzh3I+ Qoq+UzGPj8AEd6hwwCU4y6e29+WkQfwW4k7QejQGvlCxQf7KnYoIBEfS1a+ifWwlEO7X9VRN woS9zZGhaU+6UmiXNThdxK/p3GAs1gXXN84O+4n4gilyafO5QudQG8eQVZpYdoqrsIpBjony lOEgfvtGDpp9raVIVqF/72ZqzKaIyUZLWYeIyQDSGMt59jlvZF2jR/VSNtnOLC6g8ezGjzqx T2O6i8kiN07i8MX26il1U7IjijqpZXTSAMxoALNUQqYAhhRfoWpYcmsrFPc9/sFdNzfRViat 39CkM+bhAwTMayweOW2aL1lNNmUCzytaVUwXXYH80EdygmQ IronPort-HdrOrdr: A9a23:S+uo46tF2/oDG8UQp5E7YnHQ7skDgNV00zEX/kB9WHVpmwKj5q STdZMgtCMc6Qx+ZJhOo7290cW7K080sKQFg7X5Xo3MYOCZgguVxehZhOOI/9SjIVycygc378 tdmsZFebjNJGk/oMrk7Ay/Cto6hPuK4MmT9J7j5kYoYA10Z6Rn9gtjTjyaHEp/WRVcCfMCZf yhD7J81l2dUEVSSNinDX8YWOWGjdjKmJX6JSMiPXccmXGzZffC0s+BL/BetS1uKA9y/Q== X-Talos-CUID: 9a23:tM0tqG3a9q0bMJv2ykm7zrxfMJx+dXie4SrsfkqbN0xTF7OuZ1us9/Yx X-Talos-MUID: =?us-ascii?q?9a23=3ABsgvrwwxjnvx047zwT/iMIpEdkuaqJuSJlook6s?= =?us-ascii?q?MgdiVaR4tC26RqD+0SaZyfw=3D=3D?= X-IronPort-Anti-Spam-Filtered: true X-IronPort-AV: E=Sophos;i="6.23,213,1770595200"; d="scan'208";a="732908387" Received: from rcdn-l-core-02.cisco.com ([173.37.255.139]) by alln-iport-5.cisco.com with ESMTP/TLS/TLS_AES_256_GCM_SHA384; 03 May 2026 11:23:02 +0000 Received: from [127.0.0.1] (unknown [10.25.128.216]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by rcdn-l-core-02.cisco.com (Postfix) with ESMTPS id D458818000367; Sun, 3 May 2026 11:23:01 +0000 (GMT) From: Satish Kharat Date: Sun, 03 May 2026 04:22:46 -0700 Subject: [PATCH net-next v6 09/10] enic: wire V2 SR-IOV enable with admin channel and MBOX Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260503-enic-sriov-v2-admin-channel-v2-v6-9-0af4fbc2d86d@cisco.com> References: <20260503-enic-sriov-v2-admin-channel-v2-v6-0-0af4fbc2d86d@cisco.com> In-Reply-To: <20260503-enic-sriov-v2-admin-channel-v2-v6-0-0af4fbc2d86d@cisco.com> To: Andrew Lunn , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni Cc: netdev@vger.kernel.org, linux-kernel@vger.kernel.org, Sesidhar Baddela , Satish Kharat X-Mailer: b4 0.15.2 X-Developer-Signature: v=1; a=ed25519-sha256; t=1777807375; l=9981; i=satishkh@cisco.com; h=from:subject:message-id; bh=98HiIa1W+ZDwSMfxvLFC1y+jPE25eZ8OwNCbntSWJX0=; b=99JUCy4YpKCFPGEVWgaHKDsKemriwDJF94SQNj8T09t3W/sg0bXeGhIrfyMD9Sdb2eWw0b10J L6+I36BqkTeDVIncuwCyJSRs2v82iXcq+IXxscbkrStWZV5uKA/XCfc X-Developer-Key: i=satishkh@cisco.com; a=ed25519; pk=4BxbLoCGvHeIMH/PbpiNVx8/2IcTjFvqcr0AD8K6WLY= X-Outbound-Client-TLS: ANONYMOUS;unknown [10.25.128.216];TLSv1.3;TLS_AES_256_GCM_SHA384;256 X-Outbound-SMTP-Client: 10.25.128.216, [10.25.128.216] X-Outbound-Node: rcdn-l-core-02.cisco.com Extend enic_sriov_configure() to handle V2 SR-IOV VFs. When the PF detects V2 VF device IDs, the enable path allocates per-VF MBOX state, opens the admin channel, initializes the MBOX protocol, and then calls pci_enable_sriov(). The admin channel must be ready before VFs are created so that VF drivers can immediately begin the MBOX capability and registration handshake during their probe. The enic_sriov_configure() dispatcher and its V2 helpers (enic_sriov_v2_enable, enic_sriov_v2_disable) are defined here but intentionally not yet wired into struct pci_driver via .sriov_configure -- hence the __maybe_unused annotations. This series introduces only the admin channel and MBOX infrastructure; sysfs-driven V2 enable/disable will be activated in a follow-up patch by adding ".sriov_configure =3D enic_sriov_configure," to enic_driver. The disable path reverses this order: pci_disable_sriov() first (so VF drivers unregister via MBOX), then the admin channel is closed and per-VF state is freed. Reject VF port profile requests when V2 SR-IOV is active (enic_is_valid_pp_vf), since enic->pp is not reallocated for V2 VFs and the V2 protocol uses MBOX instead of port profiles. Update enic_remove() to run enic_dev_deinit() and vnic_dev_close() after SR-IOV teardown, so the PF device remains functional while VFs are being cleaned up. This ordering applies to both V1 and V2 SR-IOV paths. Signed-off-by: Satish Kharat --- drivers/net/ethernet/cisco/enic/enic.h | 1 + drivers/net/ethernet/cisco/enic/enic_main.c | 139 ++++++++++++++++++++++++= ++-- drivers/net/ethernet/cisco/enic/enic_mbox.c | 13 ++- drivers/net/ethernet/cisco/enic/enic_pp.c | 5 + drivers/net/ethernet/cisco/enic/enic_res.c | 1 + drivers/net/ethernet/cisco/enic/vnic_enet.h | 4 +- 6 files changed, 153 insertions(+), 10 deletions(-) diff --git a/drivers/net/ethernet/cisco/enic/enic.h b/drivers/net/ethernet/= cisco/enic/enic.h index 483053c781df..1bf7a91ad915 100644 --- a/drivers/net/ethernet/cisco/enic/enic.h +++ b/drivers/net/ethernet/cisco/enic/enic.h @@ -323,6 +323,7 @@ struct enic { * waiter. */ u8 mbox_expected_reply; + bool mbox_initialized; =20 /* PF: per-VF MBOX state, allocated when SRIOV V2 is enabled */ struct enic_vf_state { diff --git a/drivers/net/ethernet/cisco/enic/enic_main.c b/drivers/net/ethe= rnet/cisco/enic/enic_main.c index 53d68272d06a..057716ccc283 100644 --- a/drivers/net/ethernet/cisco/enic/enic_main.c +++ b/drivers/net/ethernet/cisco/enic/enic_main.c @@ -60,6 +60,8 @@ #include "enic_clsf.h" #include "enic_rq.h" #include "enic_wq.h" +#include "enic_admin.h" +#include "enic_mbox.h" =20 #define ENIC_NOTIFY_TIMER_PERIOD (2 * HZ) =20 @@ -2689,6 +2691,122 @@ static void enic_sriov_detect_vf_type(struct enic *= enic) enic->vf_type =3D ENIC_VF_TYPE_NONE; } } + +static int __maybe_unused +enic_sriov_v2_enable(struct enic *enic, int num_vfs) +{ + int err; + + if (!enic->has_admin_channel) { + netdev_err(enic->netdev, + "V2 SR-IOV requires admin channel resources\n"); + return -EOPNOTSUPP; + } + + enic->vf_state =3D kcalloc(num_vfs, sizeof(*enic->vf_state), GFP_KERNEL); + if (!enic->vf_state) + return -ENOMEM; + + err =3D enic_admin_channel_open(enic); + if (err) { + netdev_err(enic->netdev, + "Failed to open admin channel: %d\n", err); + goto free_vf_state; + } + + enic_mbox_init(enic); + + enic->num_vfs =3D num_vfs; + + err =3D pci_enable_sriov(enic->pdev, num_vfs); + if (err) { + netdev_err(enic->netdev, + "pci_enable_sriov failed: %d\n", err); + goto close_admin; + } + + enic->priv_flags |=3D ENIC_SRIOV_ENABLED; + return num_vfs; + +close_admin: + enic->num_vfs =3D 0; + enic_admin_channel_close(enic); +free_vf_state: + kfree(enic->vf_state); + enic->vf_state =3D NULL; + return err; +} + +static void enic_sriov_v2_disable(struct enic *enic) +{ + pci_disable_sriov(enic->pdev); + enic_admin_channel_close(enic); + kfree(enic->vf_state); + enic->vf_state =3D NULL; + enic->num_vfs =3D 0; + enic->priv_flags &=3D ~ENIC_SRIOV_ENABLED; +} + +static int __maybe_unused +enic_sriov_configure(struct pci_dev *pdev, int num_vfs) +{ + struct net_device *netdev =3D pci_get_drvdata(pdev); + struct enic *enic =3D netdev_priv(netdev); + struct enic_port_profile *pp; + int err; + + if (num_vfs > 0) { + if (enic->config.mq_subvnic_count) { + netdev_err(netdev, + "SR-IOV not supported with multi-queue sub-vnics\n"); + return -EOPNOTSUPP; + } + + if (enic->vf_type =3D=3D ENIC_VF_TYPE_NONE) { + netdev_err(netdev, + "SR-IOV not supported on this firmware version\n"); + return -EOPNOTSUPP; + } + + if (enic->vf_type =3D=3D ENIC_VF_TYPE_V2) + return enic_sriov_v2_enable(enic, num_vfs); + + pp =3D kcalloc(num_vfs, sizeof(*pp), GFP_KERNEL); + if (!pp) + return -ENOMEM; + + err =3D pci_enable_sriov(pdev, num_vfs); + if (err) { + kfree(pp); + return err; + } + + kfree(enic->pp); + enic->pp =3D pp; + enic->num_vfs =3D num_vfs; + enic->priv_flags |=3D ENIC_SRIOV_ENABLED; + return num_vfs; + } + + if (!enic_sriov_enabled(enic)) + return 0; + + if (enic->vf_type =3D=3D ENIC_VF_TYPE_V2) { + enic_sriov_v2_disable(enic); + return 0; + } + + pci_disable_sriov(pdev); + enic->num_vfs =3D 0; + enic->priv_flags &=3D ~ENIC_SRIOV_ENABLED; + + kfree(enic->pp); + enic->pp =3D kzalloc_obj(*enic->pp, GFP_KERNEL); + if (!enic->pp) + return -ENOMEM; + + return 0; +} #endif =20 static int enic_probe(struct pci_dev *pdev, const struct pci_device_id *en= t) @@ -2787,12 +2905,18 @@ static int enic_probe(struct pci_dev *pdev, const s= truct pci_device_id *ent) goto err_out_vnic_unregister; =20 #ifdef CONFIG_PCI_IOV - /* Get number of subvnics */ + enic_sriov_detect_vf_type(enic); + + /* Auto-enable SR-IOV if VFs were pre-configured (e.g. at boot). + * V2 VFs require the admin channel, which is not yet set up at probe + * time; use sysfs (enic_sriov_configure) to enable V2 SR-IOV instead. + */ pos =3D pci_find_ext_capability(pdev, PCI_EXT_CAP_ID_SRIOV); if (pos) { pci_read_config_word(pdev, pos + PCI_SRIOV_TOTAL_VF, &enic->num_vfs); - if (enic->num_vfs) { + if (enic->num_vfs && + enic->vf_type !=3D ENIC_VF_TYPE_V2) { err =3D pci_enable_sriov(pdev, enic->num_vfs); if (err) { dev_err(dev, "SRIOV enable failed, aborting." @@ -2804,7 +2928,6 @@ static int enic_probe(struct pci_dev *pdev, const str= uct pci_device_id *ent) num_pps =3D enic->num_vfs; } } - enic_sriov_detect_vf_type(enic); #endif =20 /* Allocate structure for port profiles */ @@ -3033,14 +3156,16 @@ static void enic_remove(struct pci_dev *pdev) cancel_work_sync(&enic->reset); cancel_work_sync(&enic->change_mtu_work); unregister_netdev(netdev); - enic_dev_deinit(enic); - vnic_dev_close(enic->vdev); #ifdef CONFIG_PCI_IOV if (enic_sriov_enabled(enic)) { - pci_disable_sriov(pdev); - enic->priv_flags &=3D ~ENIC_SRIOV_ENABLED; + if (enic->vf_type =3D=3D ENIC_VF_TYPE_V2) + enic_sriov_v2_disable(enic); + else + pci_disable_sriov(pdev); } #endif + enic_dev_deinit(enic); + vnic_dev_close(enic->vdev); kfree(enic->pp); vnic_dev_unregister(enic->vdev); enic_iounmap(enic); diff --git a/drivers/net/ethernet/cisco/enic/enic_mbox.c b/drivers/net/ethe= rnet/cisco/enic/enic_mbox.c index 7680baece802..71da7f05d4a0 100644 --- a/drivers/net/ethernet/cisco/enic/enic_mbox.c +++ b/drivers/net/ethernet/cisco/enic/enic_mbox.c @@ -589,8 +589,17 @@ int enic_mbox_vf_unregister(struct enic *enic) =20 void enic_mbox_init(struct enic *enic) { + /* mbox_lock and mbox_comp must be initialized exactly once per + * device lifetime; the PF sriov_configure path can re-enter this + * on each enable cycle where these primitives are already set up. + */ + if (!enic->mbox_initialized) { + mutex_init(&enic->mbox_lock); + init_completion(&enic->mbox_comp); + enic->mbox_initialized =3D true; + } else { + reinit_completion(&enic->mbox_comp); + } enic->mbox_msg_num =3D 0; - mutex_init(&enic->mbox_lock); - init_completion(&enic->mbox_comp); enic->admin_rq_handler =3D enic_mbox_recv_handler; } diff --git a/drivers/net/ethernet/cisco/enic/enic_pp.c b/drivers/net/ethern= et/cisco/enic/enic_pp.c index 4720a952725d..3f611e240c25 100644 --- a/drivers/net/ethernet/cisco/enic/enic_pp.c +++ b/drivers/net/ethernet/cisco/enic/enic_pp.c @@ -25,6 +25,11 @@ int enic_is_valid_pp_vf(struct enic *enic, int vf, int *= err) if (vf !=3D PORT_SELF_VF) { #ifdef CONFIG_PCI_IOV if (enic_sriov_enabled(enic)) { + /* V2 SR-IOV uses MBOX, not port profiles */ + if (enic->vf_type =3D=3D ENIC_VF_TYPE_V2) { + *err =3D -EOPNOTSUPP; + goto err_out; + } if (vf < 0 || vf >=3D enic->num_vfs) { *err =3D -EINVAL; goto err_out; diff --git a/drivers/net/ethernet/cisco/enic/enic_res.c b/drivers/net/ether= net/cisco/enic/enic_res.c index 2b7545d6a67f..436326ace049 100644 --- a/drivers/net/ethernet/cisco/enic/enic_res.c +++ b/drivers/net/ethernet/cisco/enic/enic_res.c @@ -59,6 +59,7 @@ int enic_get_vnic_config(struct enic *enic) GET_CONFIG(intr_timer_usec); GET_CONFIG(loop_tag); GET_CONFIG(num_arfs); + GET_CONFIG(mq_subvnic_count); GET_CONFIG(max_rq_ring); GET_CONFIG(max_wq_ring); GET_CONFIG(max_cq_ring); diff --git a/drivers/net/ethernet/cisco/enic/vnic_enet.h b/drivers/net/ethe= rnet/cisco/enic/vnic_enet.h index 9e8e86262a3f..519d2969990b 100644 --- a/drivers/net/ethernet/cisco/enic/vnic_enet.h +++ b/drivers/net/ethernet/cisco/enic/vnic_enet.h @@ -21,7 +21,9 @@ struct vnic_enet_config { u16 loop_tag; u16 vf_rq_count; u16 num_arfs; - u8 reserved[66]; + u8 reserved1[32]; + u16 mq_subvnic_count; + u8 reserved2[32]; u32 max_rq_ring; // MAX RQ ring size u32 max_wq_ring; // MAX WQ ring size u32 max_cq_ring; // MAX CQ ring size --=20 2.43.0 From nobody Sun Jun 14 04:21:27 2026 Received: from alln-iport-2.cisco.com (alln-iport-2.cisco.com [173.37.142.89]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7D418346A11; Sun, 3 May 2026 11:24:10 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=173.37.142.89 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777807462; cv=none; b=heXyoTrldBMpaibC90S8lvCgSunZkYXKMDkHDHGolL9i1KQjv5hW6pIURIM79vzfNGx046Pnhh5ihbDq65mLpk5lQHDibYGhXv4cOMJE36k+30G64Vtss8Xm06R6o+rq1UkKfDdYqvUzHptM/0lkiOE6Dz/HW8IzTZTr3F8QK2M= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777807462; c=relaxed/simple; bh=wk2VuoyHYnCiuHjTqgryUfIp3/LN3R370pEfXCEaHm4=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=JOIs7ojJlnlP0YYEm7VjElJA4IWQlWa0RclASWga+75Tj/Xt4mwrkpDre7Y37AcoaYPmOmjzknUiKWXhp4icauSbdpjqYeY2khkjPnqCvknIVyZaCiTKfIqsE5/sOZBv40xjjFmk9fA2ialruFeWw4XSg5uMM7kKU1UvLHUTKP8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=cisco.com; spf=pass smtp.mailfrom=cisco.com; dkim=pass (2048-bit key) header.d=cisco.com header.i=@cisco.com header.b=Do2mLEx0; arc=none smtp.client-ip=173.37.142.89 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=cisco.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=cisco.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=cisco.com header.i=@cisco.com header.b="Do2mLEx0" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cisco.com; i=@cisco.com; l=8933; q=dns/txt; s=iport01; t=1777807450; x=1779017050; h=from:date:subject:mime-version:content-transfer-encoding: message-id:references:in-reply-to:to:cc; bh=X7yfeTGXmWZIPEqxNobBua0eFKKXPYEsDdSjpQuVKlg=; b=Do2mLEx01FXUslYZ8XXVO1w7SzkGenwBBkuIH4W8zhj2x3bFx1/KvfQl pXgQxVFSLLPhhfKGXoCH90KVgWygx46yNQLpu+g4ZRlhczlFfI8as9Kr9 9WHpuer/QTzn/ClIuw0G0fQHhO77yiNIadDJvibqkE2n4IbUHTib4SeEN mxELA+f3VoEOX2lbXxMX7LNIAu3qnxyJKDt9rLmpi+FBKnBBONZk9YiaY WbENmtHeVFyJO0q6MvUlewsQlQtcGBI26ooE00ckpKy2UErhS9XQsDinz n5xlAImNRZSz1e1ehH+Hzgis33Miqyu+1ORKHYcf3c8ssgWsTbNTR6+I6 g==; X-CSE-ConnectionGUID: k+CLW1VySi27n3CrDJc/7Q== X-CSE-MsgGUID: q3PxNGBMQZK7UwGl0xGqjQ== X-IPAS-Result: =?us-ascii?q?A0BHAgDQLvdp/4v/Ja1aglmCV4FPQ0mEV49TgiGeHRSBa?= =?us-ascii?q?w8BAQEPUQQBAYUGAo0xAiY0CQ4BAgQBAQEBAwIDAQEBAQEBAQEBAQELAQEFA?= =?us-ascii?q?QEBAgEHBYEOE4ZchlsCAQMjBFIQJQImAgJGEAYBEoMCgnQDtB56fzOBAYR92?= =?us-ascii?q?UOBZAELFAGBCi6FcoJmAYFwhAY7hD8nG4FJRIEUAYJ5b4QFJYN0gmkEgg2BD?= =?us-ascii?q?xSPGwk/gQIcA1ksAVUTDQoLBwVhUjMDIAoLEhIYFQIUHBMPBBYyHXAMJxIsF?= =?us-ascii?q?zRYGwcFgUuDAG0UMQQhgQWEYHgjLANOcgMLGA1IESw3Bg4bBD0BbgeKZh0Pg?= =?us-ascii?q?T0BAWoHAVk1Bzs5IBIDDDIlFQgtCTySY4N+jV1rgTWfWYQmgWSgDgEyl2aTB?= =?us-ascii?q?ZkGIqNeWYRogWg8gVkzGggbFYMiUxkPji0WHMk8JjI+AQcCBw4CgXOQAASBe?= =?us-ascii?q?QEB?= IronPort-Data: A9a23:eHo5LK7gTEMrvtPhwhxjQAxRtPjGchMFZxGqfqrLsTDasY5as4F+v mpKCmqOa/uMMWfxLd5yOd7k/RkOuJGHxtZlG1NqrypnZn8b8sCt6fZ1gavT04J+CuWZESqLO u1HMoGowPgcFyGa/lH2dOC98RGQ7InQLpLkEunIJyttcgFtTSYlmHpLlvUw6mJSqYDR7zil5 5Wo+KUzBHf/g2Qqaj9MsPrawP9SlK2aVA0w7wRWic9j5Dcyp1FNZLoDKKe4KWfPQ4U8NoaSW +bZwbilyXjS9hErB8nNuu6TnpoiG+O60aCm0xK6aoD66vRwjnVaPpUTaJLwXXxqZwChxLid/ jniWauYEm/FNoWU8AgUvoIx/ytWZcWq85efSZSzXFD6I0DuKxPRL/tS4E4ePs4z3tkuP3B19 KZHLmsmdDyhjKXp+efuIgVsrpxLwMjDJogTvDRkiDreF/tjGcqFSKTR7tge1zA17ixMNa+BP IxCNnw1MUmGOkYeUrsUIMpWcOOAnXDjciZYq1K9rqss6G+Vxwt0uFToGIWPKo3VHpkFzi50o ErfpFzpJD4wEeWt6mW47V2o3v/ghCHCDdd6+LqQs6QCbEeo7mAaFhATfVeyv/S8jkmwR5RZJ lB80icisK075kG3Ztb6WBK8rTiPuRt0c9lNGeQS6wyXzKfQpQGDCQAsRzhNctE598k7WTAny HeNgtXvQzdv2JWNQHiQ8La8tz6+OSEJa2QFYEcsSQoD/smmo4wpiB/LZshsHbTzjdDvHzz0h TeQo0AWgbgPgcMV/7u08EqBgD+2oJXNCAkv6W3qsnmN9Ah1YsuhIoev81WetawGJ4eCRV7Ht 38B8ySD0N0z4Vi2vHTlaI0w8HuBvp5p7BW0bYZTIqQc IronPort-HdrOrdr: A9a23:7XfIT6/I4YnWW+fa7khuk+AuI+orL9Y04lQ7vn2ZhyYlFvBw8P rOoB1773HJYVkqNU3I9errBEDEewK/yXcX2+Ys1MmZPDUO0VHAROpfBMnZsl7d8kbFmtK1u5 0QFpRWOZncAUd3iMDi7BL9MdoxqeP3iZyAtKPyw2hgSAdyZ7ol1Qd2DQ6HDlZ7LTM2Y6bRbK DsgfZvln6HZWkdaNi9Cz0uWujOoMCOqbfdCCR2fyLOLGK1/EmVAHmQKWni4isj X-Talos-CUID: =?us-ascii?q?9a23=3AiSSyB2iYJFI/crcWCgAwfsoMYTJucT726EiJHV6?= =?us-ascii?q?DCDw2Tua+U3Oy8+Q7qp87?= X-Talos-MUID: 9a23:iUzpqAZ7oGgDtOBTrmTUlBdNHuJR0YfxNV8ilc0WlNCGDHkl X-IronPort-Anti-Spam-Filtered: true X-IronPort-AV: E=Sophos;i="6.23,213,1770595200"; d="scan'208";a="728973488" Received: from rcdn-l-core-02.cisco.com ([173.37.255.139]) by alln-iport-2.cisco.com with ESMTP/TLS/TLS_AES_256_GCM_SHA384; 03 May 2026 11:23:03 +0000 Received: from [127.0.0.1] (unknown [10.25.128.216]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by rcdn-l-core-02.cisco.com (Postfix) with ESMTPS id 91A0918000341; Sun, 3 May 2026 11:23:02 +0000 (GMT) From: Satish Kharat Date: Sun, 03 May 2026 04:22:47 -0700 Subject: [PATCH net-next v6 10/10] enic: add V2 VF probe with admin channel and PF registration Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260503-enic-sriov-v2-admin-channel-v2-v6-10-0af4fbc2d86d@cisco.com> References: <20260503-enic-sriov-v2-admin-channel-v2-v6-0-0af4fbc2d86d@cisco.com> In-Reply-To: <20260503-enic-sriov-v2-admin-channel-v2-v6-0-0af4fbc2d86d@cisco.com> To: Andrew Lunn , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni Cc: netdev@vger.kernel.org, linux-kernel@vger.kernel.org, Sesidhar Baddela , Satish Kharat X-Mailer: b4 0.15.2 X-Developer-Signature: v=1; a=ed25519-sha256; t=1777807375; l=9204; i=satishkh@cisco.com; h=from:subject:message-id; bh=wk2VuoyHYnCiuHjTqgryUfIp3/LN3R370pEfXCEaHm4=; b=+n0RKO3RYTx9xe9on0prM4rHWmp6WhcduXhYiurVmsH/CVdHo2LQ7p8FSRNsJi4GXTCegv0xw APBUh56EBawBENN/5gm/wcsOu3ZFmjr9g9j/U2VTVeAwKUqJJ80yYe8 X-Developer-Key: i=satishkh@cisco.com; a=ed25519; pk=4BxbLoCGvHeIMH/PbpiNVx8/2IcTjFvqcr0AD8K6WLY= X-Outbound-Client-TLS: ANONYMOUS;unknown [10.25.128.216];TLSv1.3;TLS_AES_256_GCM_SHA384;256 X-Outbound-SMTP-Client: 10.25.128.216, [10.25.128.216] X-Outbound-Node: rcdn-l-core-02.cisco.com When a V2 SR-IOV VF probes, open the admin channel, initialize the MBOX protocol, perform the capability check with the PF, and register with the PF. This establishes the PF-VF communication path that the PF uses to send link state notifications. The admin channel and MBOX registration happen after enic_dev_init() (which discovers admin channel resources) and before register_netdev() so the VF is fully initialized before the interface is visible to userspace. The admin channel is opened before enic_mbox_init() installs the receive handler. This is safe because enic_admin_rq_cq_service() checks admin_rq_handler before enqueuing received buffers, so any interrupt that fires between open and mbox_init is harmlessly discarded. On remove, the VF unregisters from the PF and closes its admin channel before tearing down data path resources. V2 VFs are not provisioned with an RES_TYPE_SRIOV_INTR resource by firmware, so bypass that check in the admin channel capability detection for V2 VFs. The PF still requires this resource. Reserve an additional MSI-X interrupt for the admin channel when has_admin_channel is set in enic_set_intr_mode(), so enic_admin_setup_intr()'s intr_index =3D intr_count slot is guaranteed to be within intr_avail bounds even at maximum queue configurations. The admin INTR is currently allocated from the RES_TYPE_INTR_CTRL pool shared with the data path. Firmware provisions a dedicated RES_TYPE_SRIOV_INTR resource for admin channel use; migrating the admin INTR to vnic_intr_alloc_with_type(... RES_TYPE_SRIOV_INTR) is planned as a follow-up. Signed-off-by: Satish Kharat --- drivers/net/ethernet/cisco/enic/enic.h | 1 + drivers/net/ethernet/cisco/enic/enic_main.c | 95 +++++++++++++++++++++++++= ++-- drivers/net/ethernet/cisco/enic/enic_res.c | 3 +- 3 files changed, 93 insertions(+), 6 deletions(-) diff --git a/drivers/net/ethernet/cisco/enic/enic.h b/drivers/net/ethernet/= cisco/enic/enic.h index 1bf7a91ad915..89cf412cec1d 100644 --- a/drivers/net/ethernet/cisco/enic/enic.h +++ b/drivers/net/ethernet/cisco/enic/enic.h @@ -450,6 +450,7 @@ void enic_reset_addr_lists(struct enic *enic); int enic_sriov_enabled(struct enic *enic); int enic_is_valid_vf(struct enic *enic, int vf); int enic_is_dynamic(struct enic *enic); +int enic_is_sriov_vf_v2(struct enic *enic); void enic_set_ethtool_ops(struct net_device *netdev); int __enic_set_rsskey(struct enic *enic); void enic_ext_cq(struct enic *enic); diff --git a/drivers/net/ethernet/cisco/enic/enic_main.c b/drivers/net/ethe= rnet/cisco/enic/enic_main.c index 057716ccc283..643e547f3d48 100644 --- a/drivers/net/ethernet/cisco/enic/enic_main.c +++ b/drivers/net/ethernet/cisco/enic/enic_main.c @@ -316,6 +316,11 @@ static int enic_is_sriov_vf(struct enic *enic) enic->pdev->device =3D=3D PCI_DEVICE_ID_CISCO_VIC_ENET_VF_V2; } =20 +int enic_is_sriov_vf_v2(struct enic *enic) +{ + return enic->pdev->device =3D=3D PCI_DEVICE_ID_CISCO_VIC_ENET_VF_V2; +} + int enic_is_valid_vf(struct enic *enic, int vf) { #ifdef CONFIG_PCI_IOV @@ -2157,6 +2162,13 @@ static void enic_reset(struct work_struct *work) enic_set_api_busy(enic, true); =20 enic_stop(enic->netdev); + /* CMD_SOFT_RESET disables all hardware queues including the + * admin channel queues (admin_wq, admin_rq, admin_cq). The + * recovery path below only reinitializes the data path queues. + * If the admin channel was active (V2 SR-IOV), it will be left + * in a disabled state after soft reset. Full admin channel + * recovery is planned as a future enhancement. + */ enic_dev_soft_reset(enic); enic_reset_addr_lists(enic); enic_init_vnic_resources(enic); @@ -2312,16 +2324,27 @@ static int enic_adjust_resources(struct enic *enic) enic->cq_count =3D 2; enic->intr_count =3D enic->intr_avail; break; - case VNIC_DEV_INTR_MODE_MSIX: + case VNIC_DEV_INTR_MODE_MSIX: { /* Adjust the number of wqs/rqs/cqs/interrupts that will be - * used based on which resource is the most constrained + * used based on which resource is the most constrained. + * Reserve one extra MSI-X slot for the admin channel INTR + * when has_admin_channel is set so that + * enic_admin_setup_intr() can allocate at intr_count + * within the intr_avail bounds even when the data queue + * count is maxed out. intr_count counts only the data-path + * IRQs (registered by enic_request_intr()); the admin INTR + * lives at msix index intr_count and is set up later by + * enic_admin_setup_intr(). */ + unsigned int admin_reserve =3D enic->has_admin_channel ? 1 : 0; + wq_avail =3D min(enic->wq_avail, ENIC_WQ_MAX); rq_default =3D max(netif_get_num_default_rss_queues(), ENIC_RQ_MIN_DEFAULT); rq_avail =3D min3(enic->rq_avail, ENIC_RQ_MAX, rq_default); max_queues =3D min(enic->cq_avail, - enic->intr_avail - ENIC_MSIX_RESERVED_INTR); + enic->intr_avail - ENIC_MSIX_RESERVED_INTR - + admin_reserve); if (wq_avail + rq_avail <=3D max_queues) { enic->rq_count =3D rq_avail; enic->wq_count =3D wq_avail; @@ -2339,6 +2362,7 @@ static int enic_adjust_resources(struct enic *enic) enic->intr_count =3D enic->cq_count + ENIC_MSIX_RESERVED_INTR; =20 break; + } default: dev_err(enic_get_dev(enic), "Unknown interrupt mode\n"); return -EINVAL; @@ -2992,6 +3016,38 @@ static int enic_probe(struct pci_dev *pdev, const st= ruct pci_device_id *ent) goto err_out_dev_close; } =20 + /* V2 VF: open admin channel and register with PF. + * Must happen before register_netdev so the VF is fully + * initialized before the interface is visible to userspace. + * + * admin_channel_open() runs before enic_mbox_init() installs + * the receive handler. This is safe because + * enic_admin_rq_cq_service() checks admin_rq_handler before + * enqueuing any received buffer, so interrupts that fire + * between open and mbox_init are harmlessly discarded. + */ + if (enic_is_sriov_vf_v2(enic)) { + err =3D enic_admin_channel_open(enic); + if (err) { + dev_err(dev, + "Failed to open admin channel: %d\n", err); + goto err_out_dev_deinit; + } + enic_mbox_init(enic); + err =3D enic_mbox_vf_capability_check(enic); + if (err) { + dev_err(dev, + "MBOX capability check failed: %d\n", err); + goto err_out_admin_close; + } + err =3D enic_mbox_vf_register(enic); + if (err) { + dev_err(dev, + "MBOX VF registration failed: %d\n", err); + goto err_out_admin_close; + } + } + netif_set_real_num_tx_queues(netdev, enic->wq_count); netif_set_real_num_rx_queues(netdev, enic->rq_count); =20 @@ -3016,7 +3072,7 @@ static int enic_probe(struct pci_dev *pdev, const str= uct pci_device_id *ent) err =3D enic_set_mac_addr(netdev, enic->mac_addr); if (err) { dev_err(dev, "Invalid MAC address, aborting\n"); - goto err_out_dev_deinit; + goto err_out_admin_close; } =20 enic->tx_coalesce_usecs =3D enic->config.intr_timer_usec; @@ -3114,11 +3170,23 @@ static int enic_probe(struct pci_dev *pdev, const s= truct pci_device_id *ent) err =3D register_netdev(netdev); if (err) { dev_err(dev, "Cannot register net device, aborting\n"); - goto err_out_dev_deinit; + goto err_out_admin_close; } =20 return 0; =20 +err_out_admin_close: + if (enic_is_sriov_vf_v2(enic)) { + if (enic->vf_registered) { + int unreg_err =3D enic_mbox_vf_unregister(enic); + + if (unreg_err) + netdev_warn(netdev, + "Failed to unregister from PF: %d\n", + unreg_err); + } + enic_admin_channel_close(enic); + } err_out_dev_deinit: enic_dev_deinit(enic); err_out_dev_close: @@ -3155,6 +3223,23 @@ static void enic_remove(struct pci_dev *pdev) =20 cancel_work_sync(&enic->reset); cancel_work_sync(&enic->change_mtu_work); + + /* Close the admin channel and unregister from the PF before + * unregister_netdev() to prevent a late PF notification from + * touching a netdev that has been freed. + */ + if (enic_is_sriov_vf_v2(enic)) { + if (enic->vf_registered) { + int unreg_err =3D enic_mbox_vf_unregister(enic); + + if (unreg_err) + netdev_warn(netdev, + "Failed to unregister from PF: %d\n", + unreg_err); + } + enic_admin_channel_close(enic); + } + unregister_netdev(netdev); #ifdef CONFIG_PCI_IOV if (enic_sriov_enabled(enic)) { diff --git a/drivers/net/ethernet/cisco/enic/enic_res.c b/drivers/net/ether= net/cisco/enic/enic_res.c index 436326ace049..74cd2ee3af5c 100644 --- a/drivers/net/ethernet/cisco/enic/enic_res.c +++ b/drivers/net/ethernet/cisco/enic/enic_res.c @@ -211,7 +211,8 @@ void enic_get_res_counts(struct enic *enic) vnic_dev_get_res_count(enic->vdev, RES_TYPE_ADMIN_RQ) >=3D 1 && vnic_dev_get_res_count(enic->vdev, RES_TYPE_ADMIN_CQ) >=3D ARRAY_SIZE(enic->admin_cq) && - vnic_dev_get_res_count(enic->vdev, RES_TYPE_SRIOV_INTR) >=3D 1; + (enic_is_sriov_vf_v2(enic) || + vnic_dev_get_res_count(enic->vdev, RES_TYPE_SRIOV_INTR) >=3D 1); =20 dev_info(enic_get_dev(enic), "vNIC resources avail: wq %d rq %d cq %d intr %d admin %s\n", --=20 2.43.0