From nobody Sun Jun 14 23:04:29 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5D11D1A4F3C; Tue, 7 Apr 2026 00:37:07 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1775522227; cv=none; b=Yteq/tKwrKehHWmrYB3OjWjA5/L8SgF45KTkFivTJXRKWmGMotLmEVdaQxOsKq59h/OmoNU4CW2RHt6/k0/OAuZrKdpIJij4Q6MvRumsfe5b0L7sunZgwrKLFnwj4VuZefI6FmBXOKLxHlr5/pEdeAIU73jMbbkiYyQEuEFxKeo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1775522227; c=relaxed/simple; bh=JWqtaasnuEKPR7/WgytTAsrc+MMH0s97iQTbPPqLDHA=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=pn/3VhSBk12gKHZqIHAZmCee8so5eAH2UIaxqva9+25eQJz0c+Jg3qNvEfKmmHytXnsZHv4tIWhSvzERZk8wWt2bELuNHIipLJvF3lzPtP1tDfcnMZpu2eX6L53SQ26r8BhYbxZ6m3mNwZxBB0x/KFuFad/rFkiEfYbmWSgtBhw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=KDJs+YuN; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="KDJs+YuN" Received: by smtp.kernel.org (Postfix) with ESMTPS id 1FBE8C2BCB0; Tue, 7 Apr 2026 00:37:07 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1775522227; bh=JWqtaasnuEKPR7/WgytTAsrc+MMH0s97iQTbPPqLDHA=; h=From:Date:Subject:References:In-Reply-To:To:Cc:Reply-To:From; b=KDJs+YuNrJp44NYWxMqkUu8QXLoAKLBDBuuzkysUsW/Nt1+lzACqB4Zi5htv7YQTx +/z3SrM3acuNSUz+mkV9FEBUoejOUjEKRm7KzaqDYIlmb1ZKy2H31LLOrPJNz0XVnX q12cReOUHeTRGPJuHM93YdfjzQ1sw8xemjbugpa6RqAKnwLgA30dplmwRFaEez2zpi Lt534I7KmW/1/GkH5TbFRCHcol2s1VXOysuUajV2uC8zJqJ4cwFQu87S6A2d/aC8+z Q9oRRb/HmwQ7U0ZQe6FzoHB6uzfhXT9JtGvMvykX4kBDPNxTD+9qvf5vkCL21vkvUS iQD+X72diveDw== Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0DAE1FB5175; Tue, 7 Apr 2026 00:37:07 +0000 (UTC) From: Satish Kharat via B4 Relay Date: Mon, 06 Apr 2026 17:36:59 -0700 Subject: [PATCH 01/10] enic: verify firmware supports V2 SR-IOV at probe time Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260406-enic-sriov-v2-admin-channel-v2-v1-1-82cc47636a78@cisco.com> References: <20260406-enic-sriov-v2-admin-channel-v2-v1-0-82cc47636a78@cisco.com> In-Reply-To: <20260406-enic-sriov-v2-admin-channel-v2-v1-0-82cc47636a78@cisco.com> To: Andrew Lunn , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni Cc: netdev@vger.kernel.org, linux-kernel@vger.kernel.org, 20260401-enic-sriov-v2-prep-v4-0-d5834b2ef1b9@cisco.com, Satish Kharat X-Mailer: b4 0.13.0 X-Developer-Signature: v=1; a=ed25519-sha256; t=1775522225; l=2487; i=satishkh@cisco.com; h=from:subject:message-id; bh=U+7cccRNUL3xFmQmWMuuDwvDAzXNoaVbS53PO5MaWOY=; b=CW+AiksAfYFgg7FmNIrrGZRIjj6hBnMPQKaFYb5kJfWnBQHbsrFmvroXpUZieFCPiuzVTJ9W/ hBkPBcDaOoNCcJ5Qa5a/qJlGS2ZPkAfJab7Tgx0MMLrvutKEvmq6Ux3 X-Developer-Key: i=satishkh@cisco.com; a=ed25519; pk=4BxbLoCGvHeIMH/PbpiNVx8/2IcTjFvqcr0AD8K6WLY= X-Endpoint-Received: by B4 Relay for satishkh@cisco.com/default with auth_id=639 X-Original-From: Satish Kharat Reply-To: satishkh@cisco.com From: Satish Kharat During PF probe, query the firmware get-supported-feature interface to verify that the running firmware supports V2 SR-IOV. Firmware version 5.3(4.72) and later report VIC_FEATURE_SRIOV via CMD_GET_SUPP_FEATURE_VER. If the firmware does not support the feature, set vf_type to ENIC_VF_TYPE_NONE and log a warning so the admin knows a firmware upgrade is needed. The VIC_FEATURE_SRIOV enum value (4) matches the firmware ABI. A placeholder entry (VIC_FEATURE_PTP at position 3) is added to keep the enum in sync with firmware's feature numbering. Signed-off-by: Satish Kharat --- drivers/net/ethernet/cisco/enic/enic_main.c | 18 ++++++++++++++++++ drivers/net/ethernet/cisco/enic/vnic_devcmd.h | 2 ++ 2 files changed, 20 insertions(+) diff --git a/drivers/net/ethernet/cisco/enic/enic_main.c b/drivers/net/ethe= rnet/cisco/enic/enic_main.c index e7125b818087..3a4afd6da41f 100644 --- a/drivers/net/ethernet/cisco/enic/enic_main.c +++ b/drivers/net/ethernet/cisco/enic/enic_main.c @@ -2641,8 +2641,10 @@ static void enic_iounmap(struct enic *enic) static void enic_sriov_detect_vf_type(struct enic *enic) { struct pci_dev *pdev =3D enic->pdev; + u64 supported_versions, a1 =3D 0; int pos; u16 vf_dev_id; + int err; =20 if (enic_is_sriov_vf(enic) || enic_is_dynamic(enic)) return; @@ -2669,6 +2671,22 @@ static void enic_sriov_detect_vf_type(struct enic *e= nic) enic->vf_type =3D ENIC_VF_TYPE_NONE; break; } + + if (enic->vf_type =3D=3D ENIC_VF_TYPE_V2) { + /* A successful command means firmware recognizes + * VIC_FEATURE_SRIOV; supported_versions is available + * for sub-feature versioning in the future. + */ + err =3D vnic_dev_get_supported_feature_ver(enic->vdev, + VIC_FEATURE_SRIOV, + &supported_versions, + &a1); + if (err) { + dev_warn(&pdev->dev, + "SR-IOV V2 not supported by current firmware. Upgrade to VIC FW 5.3(4= .72) or higher.\n"); + enic->vf_type =3D ENIC_VF_TYPE_NONE; + } + } } #endif =20 diff --git a/drivers/net/ethernet/cisco/enic/vnic_devcmd.h b/drivers/net/et= hernet/cisco/enic/vnic_devcmd.h index 605ef17f967e..7a4bce736105 100644 --- a/drivers/net/ethernet/cisco/enic/vnic_devcmd.h +++ b/drivers/net/ethernet/cisco/enic/vnic_devcmd.h @@ -734,6 +734,8 @@ enum vic_feature_t { VIC_FEATURE_VXLAN, VIC_FEATURE_RDMA, VIC_FEATURE_VXLAN_PATCH, + VIC_FEATURE_PTP, + VIC_FEATURE_SRIOV, VIC_FEATURE_MAX, }; =20 --=20 2.43.0 From nobody Sun Jun 14 23:04:29 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5D0C519B5A3; Tue, 7 Apr 2026 00:37:07 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1775522227; cv=none; b=QzS/xFBFpJiBJstn/JsmvUUfNcA92F8XbcD9uq7u9sn/CuSqDrJokObUcQgmtiBXY+QmeJMHm6JtX7DbG9arjxmJ1J8z7B2OoSTHg54+hPLb0RyRvwBsGhORZAzthCZfhOIaesdoMjA7qH0WdTw9INQBOqFHgbrAdfZxcpcDlFc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1775522227; c=relaxed/simple; bh=05ZM9TgEqk5dcoGAhAKAEkr6lUqgY0+lE0WtMJW32A8=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=WGXXglintQ63yjQ5bCYF8k/7WdaZGHm9OdDJecyvtCbxSAQZmf1oj6QtV9QfoiE4nq9ZMcV/F10HnrBeZpoEFTEQ6dLz1M4EL9042jWdZvyFWoo8l9trE4RKbi2XhQPB9sKccbJE7V3vBb2GEKJXRRrK/7sc4Lpcf3aOpOHJ7n4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=MzlNqGAR; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="MzlNqGAR" Received: by smtp.kernel.org (Postfix) with ESMTPS id 2D466C2BCAF; Tue, 7 Apr 2026 00:37:07 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1775522227; bh=05ZM9TgEqk5dcoGAhAKAEkr6lUqgY0+lE0WtMJW32A8=; h=From:Date:Subject:References:In-Reply-To:To:Cc:Reply-To:From; b=MzlNqGARP60BQrLNTlQdySbkdYiaiI4oqSgWTjN7Lty7s+PnlXqJ5vEiA0RzKyKDb Mbcm4RyD+ChsBQGaBbg2pUkvGYt03ulg/txw6OJu5c4nY0Bd0B4jz6+/NGX7xpJMnS AUV/0sopkNrdgqJfr7GL1fO77Jg5SWBnaIz0A00oXbX0k5acAxh+E24VjOXtnre2z6 R3MeErnO1pYPvXxbFABJPChG2eo025XGy9NJyIBo0FQs/ZRl3hJBJW37wZuUEaFEDf dqrJjvdf+YxSqidPkKdd6TGawomkm9LFMl2hhF5/G5O4PyzsBUBv0e0SHvr0kGSGql +Hi89wOQklrhQ== Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1FD9FFB5178; Tue, 7 Apr 2026 00:37:07 +0000 (UTC) From: Satish Kharat via B4 Relay Date: Mon, 06 Apr 2026 17:37:00 -0700 Subject: [PATCH 02/10] enic: add admin channel open and close for SR-IOV Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260406-enic-sriov-v2-admin-channel-v2-v1-2-82cc47636a78@cisco.com> References: <20260406-enic-sriov-v2-admin-channel-v2-v1-0-82cc47636a78@cisco.com> In-Reply-To: <20260406-enic-sriov-v2-admin-channel-v2-v1-0-82cc47636a78@cisco.com> To: Andrew Lunn , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni Cc: netdev@vger.kernel.org, linux-kernel@vger.kernel.org, 20260401-enic-sriov-v2-prep-v4-0-d5834b2ef1b9@cisco.com, Satish Kharat X-Mailer: b4 0.13.0 X-Developer-Signature: v=1; a=ed25519-sha256; t=1775522225; l=7692; i=satishkh@cisco.com; h=from:subject:message-id; bh=MSrD6SHhLimV7ZBBHtD+FMA9ZuwQh8TK6KZclm5ORVk=; b=pUwFQmKBe/7Y2wYPKJi32LKiSurCVuK/TPjnmV/szNXc1c4xoeRE9W0sUonuuuwHdgCYktetO Srggs+mtqwWCGCx/xpGnH+2Q8rpX3ErkHn31IxyeQcU7pqtfxYv05B4 X-Developer-Key: i=satishkh@cisco.com; a=ed25519; pk=4BxbLoCGvHeIMH/PbpiNVx8/2IcTjFvqcr0AD8K6WLY= X-Endpoint-Received: by B4 Relay for satishkh@cisco.com/default with auth_id=639 X-Original-From: Satish Kharat Reply-To: satishkh@cisco.com From: Satish Kharat The V2 SR-IOV design uses a dedicated admin channel (WQ/RQ/CQ/INTR on separate BAR resources) for PF-VF mailbox communication rather than firmware-proxied devcmds. Introduce enic_admin_channel_open() and enic_admin_channel_close(). Open allocates and initialises the admin WQ, RQ, two CQs (one per direction) and one SR-IOV interrupt, then issues CMD_QP_TYPE_SET to tell firmware the queues are admin-type. Close reverses the sequence. Add CMD_QP_TYPE_SET (97) and QP_TYPE_ADMIN/DATA defines to vnic_devcmd.h. Signed-off-by: Satish Kharat --- drivers/net/ethernet/cisco/enic/Makefile | 3 +- drivers/net/ethernet/cisco/enic/enic_admin.c | 175 ++++++++++++++++++++++= ++++ drivers/net/ethernet/cisco/enic/enic_admin.h | 15 +++ drivers/net/ethernet/cisco/enic/vnic_devcmd.h | 9 ++ 4 files changed, 201 insertions(+), 1 deletion(-) diff --git a/drivers/net/ethernet/cisco/enic/Makefile b/drivers/net/etherne= t/cisco/enic/Makefile index a96b8332e6e2..7ae72fefc99a 100644 --- a/drivers/net/ethernet/cisco/enic/Makefile +++ b/drivers/net/ethernet/cisco/enic/Makefile @@ -3,5 +3,6 @@ obj-$(CONFIG_ENIC) :=3D enic.o =20 enic-y :=3D enic_main.o vnic_cq.o vnic_intr.o vnic_wq.o \ enic_res.o enic_dev.o enic_pp.o vnic_dev.o vnic_rq.o vnic_vic.o \ - enic_ethtool.o enic_api.o enic_clsf.o enic_rq.o enic_wq.o + enic_ethtool.o enic_api.o enic_clsf.o enic_rq.o enic_wq.o \ + enic_admin.o =20 diff --git a/drivers/net/ethernet/cisco/enic/enic_admin.c b/drivers/net/eth= ernet/cisco/enic/enic_admin.c new file mode 100644 index 000000000000..d1abe6a50095 --- /dev/null +++ b/drivers/net/ethernet/cisco/enic/enic_admin.c @@ -0,0 +1,175 @@ +// SPDX-License-Identifier: GPL-2.0-only +// Copyright 2025 Cisco Systems, Inc. All rights reserved. + +#include +#include + +#include "vnic_dev.h" +#include "vnic_wq.h" +#include "vnic_rq.h" +#include "vnic_cq.h" +#include "vnic_intr.h" +#include "vnic_resource.h" +#include "vnic_devcmd.h" +#include "enic.h" +#include "enic_admin.h" +#include "cq_desc.h" +#include "wq_enet_desc.h" +#include "rq_enet_desc.h" + +/* No-op: admin WQ buffers are freed inline after completion polling */ +static void enic_admin_wq_buf_clean(struct vnic_wq *wq, + struct vnic_wq_buf *buf) +{ +} + +/* No-op: admin RQ buffer teardown is handled in enic_admin_channel_close = */ +static void enic_admin_rq_buf_clean(struct vnic_rq *rq, + struct vnic_rq_buf *buf) +{ +} + +static int enic_admin_qp_type_set(struct enic *enic, u32 enable) +{ + u64 a0 =3D QP_TYPE_ADMIN, a1 =3D enable; + int wait =3D 1000; + int err; + + spin_lock_bh(&enic->devcmd_lock); + err =3D vnic_dev_cmd(enic->vdev, CMD_QP_TYPE_SET, &a0, &a1, wait); + spin_unlock_bh(&enic->devcmd_lock); + + return err; +} + +static int enic_admin_alloc_resources(struct enic *enic) +{ + int err; + + err =3D vnic_wq_alloc_with_type(enic->vdev, &enic->admin_wq, 0, + ENIC_ADMIN_DESC_COUNT, + sizeof(struct wq_enet_desc), + RES_TYPE_ADMIN_WQ); + if (err) + return err; + + err =3D vnic_rq_alloc_with_type(enic->vdev, &enic->admin_rq, 0, + ENIC_ADMIN_DESC_COUNT, + sizeof(struct rq_enet_desc), + RES_TYPE_ADMIN_RQ); + if (err) + goto free_wq; + + err =3D vnic_cq_alloc_with_type(enic->vdev, &enic->admin_cq[0], 0, + ENIC_ADMIN_DESC_COUNT, + sizeof(struct cq_desc), + RES_TYPE_ADMIN_CQ); + if (err) + goto free_rq; + + err =3D vnic_cq_alloc_with_type(enic->vdev, &enic->admin_cq[1], 1, + ENIC_ADMIN_DESC_COUNT, + 16 << enic->ext_cq, + RES_TYPE_ADMIN_CQ); + if (err) + goto free_cq0; + + /* PFs have dedicated SRIOV_INTR resources for admin channel. + * VFs lack SRIOV_INTR; use a regular INTR_CTRL slot instead. + */ + if (vnic_dev_get_res_count(enic->vdev, RES_TYPE_SRIOV_INTR) >=3D 1) + err =3D vnic_intr_alloc_with_type(enic->vdev, + &enic->admin_intr, 0, + RES_TYPE_SRIOV_INTR); + else + err =3D vnic_intr_alloc(enic->vdev, &enic->admin_intr, + enic->intr_count); + if (err) + goto free_cq1; + + return 0; + +free_cq1: + vnic_cq_free(&enic->admin_cq[1]); +free_cq0: + vnic_cq_free(&enic->admin_cq[0]); +free_rq: + vnic_rq_free(&enic->admin_rq); +free_wq: + vnic_wq_free(&enic->admin_wq); + return err; +} + +static void enic_admin_free_resources(struct enic *enic) +{ + vnic_intr_free(&enic->admin_intr); + vnic_cq_free(&enic->admin_cq[1]); + vnic_cq_free(&enic->admin_cq[0]); + vnic_rq_free(&enic->admin_rq); + vnic_wq_free(&enic->admin_wq); +} + +static void enic_admin_init_resources(struct enic *enic) +{ + vnic_wq_init(&enic->admin_wq, 0, 0, 0); + vnic_rq_init(&enic->admin_rq, 1, 0, 0); + vnic_cq_init(&enic->admin_cq[0], 0, 1, 0, 0, 1, 0, 1, 0, 0, 0); + vnic_cq_init(&enic->admin_cq[1], 0, 1, 0, 0, 1, 0, 1, 0, 0, 0); + vnic_intr_init(&enic->admin_intr, 0, 0, 1); +} + +int enic_admin_channel_open(struct enic *enic) +{ + int err; + + if (!enic->has_admin_channel) + return -ENODEV; + + err =3D enic_admin_alloc_resources(enic); + if (err) { + netdev_err(enic->netdev, + "Failed to alloc admin channel resources: %d\n", + err); + return err; + } + + enic_admin_init_resources(enic); + + vnic_wq_enable(&enic->admin_wq); + vnic_rq_enable(&enic->admin_rq); + + err =3D enic_admin_qp_type_set(enic, 1); + if (err) { + netdev_err(enic->netdev, + "Failed to set admin QP type: %d\n", err); + goto disable_queues; + } + + return 0; + +disable_queues: + vnic_wq_disable(&enic->admin_wq); + vnic_rq_disable(&enic->admin_rq); + enic_admin_qp_type_set(enic, 0); + enic_admin_free_resources(enic); + return err; +} + +void enic_admin_channel_close(struct enic *enic) +{ + if (!enic->has_admin_channel) + return; + + vnic_wq_disable(&enic->admin_wq); + vnic_rq_disable(&enic->admin_rq); + + enic_admin_qp_type_set(enic, 0); + + vnic_wq_clean(&enic->admin_wq, enic_admin_wq_buf_clean); + vnic_rq_clean(&enic->admin_rq, enic_admin_rq_buf_clean); + vnic_cq_clean(&enic->admin_cq[0]); + vnic_cq_clean(&enic->admin_cq[1]); + vnic_intr_clean(&enic->admin_intr); + + enic_admin_free_resources(enic); +} diff --git a/drivers/net/ethernet/cisco/enic/enic_admin.h b/drivers/net/eth= ernet/cisco/enic/enic_admin.h new file mode 100644 index 000000000000..569aadeb9312 --- /dev/null +++ b/drivers/net/ethernet/cisco/enic/enic_admin.h @@ -0,0 +1,15 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* Copyright 2025 Cisco Systems, Inc. All rights reserved. */ + +#ifndef _ENIC_ADMIN_H_ +#define _ENIC_ADMIN_H_ + +#define ENIC_ADMIN_DESC_COUNT 64 +#define ENIC_ADMIN_BUF_SIZE 2048 + +struct enic; + +int enic_admin_channel_open(struct enic *enic); +void enic_admin_channel_close(struct enic *enic); + +#endif /* _ENIC_ADMIN_H_ */ diff --git a/drivers/net/ethernet/cisco/enic/vnic_devcmd.h b/drivers/net/et= hernet/cisco/enic/vnic_devcmd.h index 7a4bce736105..a1c8f522c7d7 100644 --- a/drivers/net/ethernet/cisco/enic/vnic_devcmd.h +++ b/drivers/net/ethernet/cisco/enic/vnic_devcmd.h @@ -455,8 +455,17 @@ enum vnic_devcmd_cmd { */ CMD_CQ_ENTRY_SIZE_SET =3D _CMDC(_CMD_DIR_WRITE, _CMD_VTYPE_ENET, 90), =20 + /* + * Set queue pair type (admin or data) + * in: (u32) a0 =3D queue pair type (0 =3D admin, 1 =3D data) + * in: (u32) a1 =3D enable (1) / disable (0) + */ + CMD_QP_TYPE_SET =3D _CMDC(_CMD_DIR_WRITE, _CMD_VTYPE_ENET, 97), }; =20 +#define QP_TYPE_ADMIN 0 +#define QP_TYPE_DATA 1 + /* CMD_ENABLE2 flags */ #define CMD_ENABLE2_STANDBY 0x0 #define CMD_ENABLE2_ACTIVE 0x1 --=20 2.43.0 From nobody Sun Jun 14 23:04:29 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 613731A6835; Tue, 7 Apr 2026 00:37:07 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1775522227; cv=none; b=bRfUlumCk4T/tDm8MUJfGjNYQn/ks+CTiJmaQ19571pU5jcnU0r4a+F4WFLIjruucnx8b9MXyxQZwCPvng82L60ZVGBXGCW5HNKQmm0KXDtLJmzytSQw6/HcTWr/rpwEuSKpJnczKwzzKjDusWRzPoh9l1fxFJ5BO8R15JoqwFY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1775522227; c=relaxed/simple; bh=qoiuiRACX+7fyGd6PmgsS4FBGmhMwOSqH3IFoLmmOYA=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=ryZpRmo0UO7rSoEHOdhgkZ1kH1ut0/yPBJfUoJJNTDXrXosPtPkvpLzYf4RaWSXUD8ACwBwNSYFEeO1pn0lRZAWv1X3s05Y5DKxCrHGQGy8w+4CD2AyKlb3742Lxkf6b402Isz/BiEqjjBEr+5u04MOoBk03brK8tG/z5/JB7R8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=G+J/eJGv; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="G+J/eJGv" Received: by smtp.kernel.org (Postfix) with ESMTPS id 3CAE5C2BCB2; Tue, 7 Apr 2026 00:37:07 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1775522227; bh=qoiuiRACX+7fyGd6PmgsS4FBGmhMwOSqH3IFoLmmOYA=; h=From:Date:Subject:References:In-Reply-To:To:Cc:Reply-To:From; b=G+J/eJGvpbXjh7cYf8iAVwdK/x9KjQ9h2OKSPZtp+v9RxxHE2ED+RQGOxjW4hq0By GpPRURxN0QtNv/nF3yU7TaeZOnVluF8doH0NcsX/whKg7XJczMmw5rYew5c85Z9zeW SxMS27XC+p1fDTkL8Q2IM4yJo/wyVCAUr0HAeAZHsTZhCWhtQN9PdOntf55APRY4we WYyaqnpS2TbQgYYMkHkAV/xS26bi5LqWmauSl3eI2luIz8MLOLGeyk2rYi7ebSlIK7 AbXJVVEd8s123PVw0yIypG1YfAM4LCZHbH5Yay5Ivwhk1aQKB7OIE0WTYcQtDG6slD SVw6pJ9ecjbTw== Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2FB21FB517D; Tue, 7 Apr 2026 00:37:07 +0000 (UTC) From: Satish Kharat via B4 Relay Date: Mon, 06 Apr 2026 17:37:01 -0700 Subject: [PATCH 03/10] enic: add admin RQ buffer management Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260406-enic-sriov-v2-admin-channel-v2-v1-3-82cc47636a78@cisco.com> References: <20260406-enic-sriov-v2-admin-channel-v2-v1-0-82cc47636a78@cisco.com> In-Reply-To: <20260406-enic-sriov-v2-admin-channel-v2-v1-0-82cc47636a78@cisco.com> To: Andrew Lunn , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni Cc: netdev@vger.kernel.org, linux-kernel@vger.kernel.org, 20260401-enic-sriov-v2-prep-v4-0-d5834b2ef1b9@cisco.com, Satish Kharat X-Mailer: b4 0.13.0 X-Developer-Signature: v=1; a=ed25519-sha256; t=1775522225; l=3638; i=satishkh@cisco.com; h=from:subject:message-id; bh=ZMpfBevBZMtAG95G26rzqtWY0Kge40y6XyZgAR1yrjY=; b=ziH7ce5EreMXy0bLfSJ9nTBmN2dwacVTIhq+biCWLOUbwn7gYciZTZj4sLUY8dBtTRC29BLuy EDpU/nK7jP1C8l6jclfn2dTXD9Kb5434TRrvUlKE/gRauMNgWxvqGFY X-Developer-Key: i=satishkh@cisco.com; a=ed25519; pk=4BxbLoCGvHeIMH/PbpiNVx8/2IcTjFvqcr0AD8K6WLY= X-Endpoint-Received: by B4 Relay for satishkh@cisco.com/default with auth_id=639 X-Original-From: Satish Kharat Reply-To: satishkh@cisco.com From: Satish Kharat The admin receive queue needs pre-posted DMA buffers for incoming mailbox messages from VFs. Each buffer is a kmalloc'd region mapped for DMA (2048 bytes, sufficient for any MBOX message). Add enic_admin_rq_fill() to post buffers at open time, and enic_admin_rq_drain() to unmap and free them at close time. Wire both into the admin channel open/close paths. Signed-off-by: Satish Kharat --- drivers/net/ethernet/cisco/enic/enic_admin.c | 66 ++++++++++++++++++++++++= +++- 1 file changed, 64 insertions(+), 2 deletions(-) diff --git a/drivers/net/ethernet/cisco/enic/enic_admin.c b/drivers/net/eth= ernet/cisco/enic/enic_admin.c index d1abe6a50095..a8fcd5f116d1 100644 --- a/drivers/net/ethernet/cisco/enic/enic_admin.c +++ b/drivers/net/ethernet/cisco/enic/enic_admin.c @@ -3,6 +3,7 @@ =20 #include #include +#include =20 #include "vnic_dev.h" #include "vnic_wq.h" @@ -23,10 +24,63 @@ static void enic_admin_wq_buf_clean(struct vnic_wq *wq, { } =20 -/* No-op: admin RQ buffer teardown is handled in enic_admin_channel_close = */ static void enic_admin_rq_buf_clean(struct vnic_rq *rq, struct vnic_rq_buf *buf) { + struct enic *enic =3D vnic_dev_priv(rq->vdev); + + if (!buf->os_buf) + return; + + dma_unmap_single(&enic->pdev->dev, buf->dma_addr, buf->len, + DMA_FROM_DEVICE); + kfree(buf->os_buf); + buf->os_buf =3D NULL; +} + +static int enic_admin_rq_post_one(struct enic *enic) +{ + struct vnic_rq *rq =3D &enic->admin_rq; + struct rq_enet_desc *desc; + dma_addr_t dma_addr; + void *buf; + + buf =3D kmalloc(ENIC_ADMIN_BUF_SIZE, GFP_KERNEL); + if (!buf) + return -ENOMEM; + + dma_addr =3D dma_map_single(&enic->pdev->dev, buf, ENIC_ADMIN_BUF_SIZE, + DMA_FROM_DEVICE); + if (dma_mapping_error(&enic->pdev->dev, dma_addr)) { + kfree(buf); + return -ENOMEM; + } + + desc =3D vnic_rq_next_desc(rq); + rq_enet_desc_enc(desc, (u64)dma_addr | VNIC_PADDR_TARGET, + RQ_ENET_TYPE_ONLY_SOP, ENIC_ADMIN_BUF_SIZE); + vnic_rq_post(rq, buf, 0, dma_addr, ENIC_ADMIN_BUF_SIZE, 0); + + return 0; +} + +static int enic_admin_rq_fill(struct enic *enic) +{ + struct vnic_rq *rq =3D &enic->admin_rq; + int err; + + while (vnic_rq_desc_avail(rq) > 0) { + err =3D enic_admin_rq_post_one(enic); + if (err) + return err; + } + + return 0; +} + +static void enic_admin_rq_drain(struct enic *enic) +{ + vnic_rq_clean(&enic->admin_rq, enic_admin_rq_buf_clean); } =20 static int enic_admin_qp_type_set(struct enic *enic, u32 enable) @@ -138,6 +192,13 @@ int enic_admin_channel_open(struct enic *enic) vnic_wq_enable(&enic->admin_wq); vnic_rq_enable(&enic->admin_rq); =20 + err =3D enic_admin_rq_fill(enic); + if (err) { + netdev_err(enic->netdev, + "Failed to fill admin RQ buffers: %d\n", err); + goto disable_queues; + } + err =3D enic_admin_qp_type_set(enic, 1); if (err) { netdev_err(enic->netdev, @@ -151,6 +212,7 @@ int enic_admin_channel_open(struct enic *enic) vnic_wq_disable(&enic->admin_wq); vnic_rq_disable(&enic->admin_rq); enic_admin_qp_type_set(enic, 0); + enic_admin_rq_drain(enic); enic_admin_free_resources(enic); return err; } @@ -166,7 +228,7 @@ void enic_admin_channel_close(struct enic *enic) enic_admin_qp_type_set(enic, 0); =20 vnic_wq_clean(&enic->admin_wq, enic_admin_wq_buf_clean); - vnic_rq_clean(&enic->admin_rq, enic_admin_rq_buf_clean); + enic_admin_rq_drain(enic); vnic_cq_clean(&enic->admin_cq[0]); vnic_cq_clean(&enic->admin_cq[1]); vnic_intr_clean(&enic->admin_intr); --=20 2.43.0 From nobody Sun Jun 14 23:04:29 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6C6371A6839; Tue, 7 Apr 2026 00:37:07 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1775522227; cv=none; b=lYSSTCfg6Ti76MyQeF44vNmeLM5qlfmsFws+ePv5N9fVzl4iOfe+slRLfHO/IxUkgOjh0nYeJ81jrNB+kpxN07jY9ZHZYRK/vhdTthX3EkCwmCGdreQz+d4Ae8iBmVjH9OTiY+NASJMdsunHdwu+zAlEfvdaxhGQm4U1BQ5jM0w= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1775522227; c=relaxed/simple; bh=4NgpsWlCdsIAIC5xIcBsVJKQpyg88AjADIh6WN963i8=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=jVPn793BWHoqBbEDUF/PAcxPMTv8HFN0NM1vP0rVKveDeox159cFObev/01ecqxTWKs9mVNwv9DIVIlK+As8hM0BNFwgeZ7cMbB0v4j/CvFulN6N/RcoPVVNB6MFiwvENIFMxn2dzWQ1EKqvXXx9ePCbfYHD/pKPcwskLHVxaWs= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=JKZxsSL/; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="JKZxsSL/" Received: by smtp.kernel.org (Postfix) with ESMTPS id 4A63EC2BCB5; Tue, 7 Apr 2026 00:37:07 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1775522227; bh=4NgpsWlCdsIAIC5xIcBsVJKQpyg88AjADIh6WN963i8=; h=From:Date:Subject:References:In-Reply-To:To:Cc:Reply-To:From; b=JKZxsSL/JHnvqLwBn1eiXZyLvcg7HtV2ZHz7K1nl7Waf+TzhfLiz14NkrRdpdoPkj 0TEbZtn/HjVEWuWQgqgBauMvjfQK9jW7O4OCkbolS6JDLeIYL7aWEHsg5FZlare1cZ bD6QpWr3xP487ClwBc879tzQ35sWU76QyD/EQ9PlAoSFDYzZV//ArT57XEjvKPugpD P+ljjKwubgbjHrRyCla1XaPrHyBlQVwJEUlc2pt04t2MHscwwJ2sUtCrmJGdRJjXIj znPd129fx5FQS+YnC1mhxVyjSQitus8dx/YSNJg8XsIxQDa9bW/5zCSlPkUo5XlPBm gWLT6yrwX7SfA== Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 41BD4FB516E; Tue, 7 Apr 2026 00:37:07 +0000 (UTC) From: Satish Kharat via B4 Relay Date: Mon, 06 Apr 2026 17:37:02 -0700 Subject: [PATCH 04/10] enic: add admin CQ service with MSI-X interrupt and NAPI polling Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260406-enic-sriov-v2-admin-channel-v2-v1-4-82cc47636a78@cisco.com> References: <20260406-enic-sriov-v2-admin-channel-v2-v1-0-82cc47636a78@cisco.com> In-Reply-To: <20260406-enic-sriov-v2-admin-channel-v2-v1-0-82cc47636a78@cisco.com> To: Andrew Lunn , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni Cc: netdev@vger.kernel.org, linux-kernel@vger.kernel.org, 20260401-enic-sriov-v2-prep-v4-0-d5834b2ef1b9@cisco.com, Satish Kharat X-Mailer: b4 0.13.0 X-Developer-Signature: v=1; a=ed25519-sha256; t=1775522225; l=14030; i=satishkh@cisco.com; h=from:subject:message-id; bh=3WUrNO7kGfpAIJkN06TsC/hhQmW3Yd+7+tJKYEj0r7U=; b=Q5LhSP9t5AOC9l+OycX1JkBiiczjdBQmqD8fE+DvTi6Jg0aKWoExG1OKNPaPSM0juRuiBhKCN Unmr5K12xQvDn/Byh8kP9Rncm8wU/vvrR95LGuui4VCP1BwBjKyPF/+ X-Developer-Key: i=satishkh@cisco.com; a=ed25519; pk=4BxbLoCGvHeIMH/PbpiNVx8/2IcTjFvqcr0AD8K6WLY= X-Endpoint-Received: by B4 Relay for satishkh@cisco.com/default with auth_id=639 X-Original-From: Satish Kharat Reply-To: satishkh@cisco.com From: Satish Kharat Add completion queue service for the admin channel WQ and RQ, driven by an MSI-X interrupt and NAPI polling. The receive pipeline is: MSI-X ISR -> NAPI poll -> RQ CQ service -> message enqueue -> workqueue handler -> admin_rq_handler callback. NAPI drains the RQ CQ in softirq context, copying each received buffer into an enic_admin_msg and appending it to a spinlock-protected list. A system workqueue handler then processes each message in process context where sleeping (mutex, GFP_KERNEL allocations) is safe. The WQ CQ service counts transmit completions and is called from the synchronous MBOX send path. RQ buffer allocation uses GFP_ATOMIC since enic_admin_rq_fill() is called from NAPI context during CQ processing. The admin channel open/close paths set up and tear down the MSI-X interrupt, NAPI instance, and workqueue. CQ init enables interrupt delivery and sets the interrupt offset so completions trigger the admin ISR. Signed-off-by: Satish Kharat --- drivers/net/ethernet/cisco/enic/enic.h | 7 + drivers/net/ethernet/cisco/enic/enic_admin.c | 295 +++++++++++++++++++++++= ++-- drivers/net/ethernet/cisco/enic/enic_admin.h | 12 ++ 3 files changed, 292 insertions(+), 22 deletions(-) diff --git a/drivers/net/ethernet/cisco/enic/enic.h b/drivers/net/ethernet/= cisco/enic/enic.h index 08472420f3a1..9700fe14e51f 100644 --- a/drivers/net/ethernet/cisco/enic/enic.h +++ b/drivers/net/ethernet/cisco/enic/enic.h @@ -296,6 +296,13 @@ struct enic { struct vnic_rq admin_rq; struct vnic_cq admin_cq[2]; struct vnic_intr admin_intr; + struct napi_struct admin_napi; + unsigned int admin_intr_index; + struct work_struct admin_msg_work; + spinlock_t admin_msg_lock; /* protects admin_msg_list */ + struct list_head admin_msg_list; + u64 admin_msg_drop_cnt; + void (*admin_rq_handler)(struct enic *enic, void *buf, unsigned int len); }; =20 static inline struct net_device *vnic_get_netdev(struct vnic_dev *vdev) diff --git a/drivers/net/ethernet/cisco/enic/enic_admin.c b/drivers/net/eth= ernet/cisco/enic/enic_admin.c index a8fcd5f116d1..31f9f754941e 100644 --- a/drivers/net/ethernet/cisco/enic/enic_admin.c +++ b/drivers/net/ethernet/cisco/enic/enic_admin.c @@ -4,6 +4,7 @@ #include #include #include +#include =20 #include "vnic_dev.h" #include "vnic_wq.h" @@ -15,6 +16,7 @@ #include "enic.h" #include "enic_admin.h" #include "cq_desc.h" +#include "cq_enet_desc.h" #include "wq_enet_desc.h" #include "rq_enet_desc.h" =20 @@ -38,14 +40,14 @@ static void enic_admin_rq_buf_clean(struct vnic_rq *rq, buf->os_buf =3D NULL; } =20 -static int enic_admin_rq_post_one(struct enic *enic) +static int enic_admin_rq_post_one(struct enic *enic, gfp_t gfp) { struct vnic_rq *rq =3D &enic->admin_rq; struct rq_enet_desc *desc; dma_addr_t dma_addr; void *buf; =20 - buf =3D kmalloc(ENIC_ADMIN_BUF_SIZE, GFP_KERNEL); + buf =3D kmalloc(ENIC_ADMIN_BUF_SIZE, gfp); if (!buf) return -ENOMEM; =20 @@ -64,13 +66,13 @@ static int enic_admin_rq_post_one(struct enic *enic) return 0; } =20 -static int enic_admin_rq_fill(struct enic *enic) +static int enic_admin_rq_fill(struct enic *enic, gfp_t gfp) { struct vnic_rq *rq =3D &enic->admin_rq; int err; =20 while (vnic_rq_desc_avail(rq) > 0) { - err =3D enic_admin_rq_post_one(enic); + err =3D enic_admin_rq_post_one(enic, gfp); if (err) return err; } @@ -83,6 +85,205 @@ static void enic_admin_rq_drain(struct enic *enic) vnic_rq_clean(&enic->admin_rq, enic_admin_rq_buf_clean); } =20 +static unsigned int enic_admin_cq_color(void *cq_desc, unsigned int desc_s= ize) +{ + u8 type_color =3D *((u8 *)cq_desc + desc_size - 1); + + return (type_color >> CQ_DESC_COLOR_SHIFT) & CQ_DESC_COLOR_MASK; +} + +unsigned int enic_admin_wq_cq_service(struct enic *enic) +{ + struct vnic_cq *cq =3D &enic->admin_cq[0]; + unsigned int work =3D 0; + void *desc; + + desc =3D vnic_cq_to_clean(cq); + while (enic_admin_cq_color(desc, cq->ring.desc_size) !=3D + cq->last_color) { + /* Ensure color bit is read before descriptor fields */ + rmb(); + vnic_cq_inc_to_clean(cq); + work++; + desc =3D vnic_cq_to_clean(cq); + } + + return work; +} + +static void enic_admin_msg_enqueue(struct enic *enic, void *buf, + unsigned int len) +{ + struct enic_admin_msg *msg; + + msg =3D kmalloc(struct_size(msg, data, len), GFP_ATOMIC); + if (!msg) { + enic->admin_msg_drop_cnt++; + if (net_ratelimit()) + netdev_warn(enic->netdev, + "admin msg enqueue drop (len=3D%u drops=3D%llu)\n", + len, enic->admin_msg_drop_cnt); + return; + } + + msg->len =3D len; + memcpy(msg->data, buf, len); + + spin_lock(&enic->admin_msg_lock); + list_add_tail(&msg->list, &enic->admin_msg_list); + spin_unlock(&enic->admin_msg_lock); +} + +unsigned int enic_admin_rq_cq_service(struct enic *enic, unsigned int budg= et) +{ + struct vnic_cq *cq =3D &enic->admin_cq[1]; + struct vnic_rq *rq =3D &enic->admin_rq; + struct vnic_rq_buf *buf; + unsigned int work =3D 0; + void *desc; + + desc =3D vnic_cq_to_clean(cq); + while (work < budget && + enic_admin_cq_color(desc, cq->ring.desc_size) !=3D + cq->last_color) { + /* Ensure CQ descriptor fields are read after the color/valid check */ + rmb(); + buf =3D rq->to_clean; + + dma_sync_single_for_cpu(&enic->pdev->dev, + buf->dma_addr, buf->len, + DMA_FROM_DEVICE); + + enic_admin_msg_enqueue(enic, buf->os_buf, buf->len); + + enic_admin_rq_buf_clean(rq, rq->to_clean); + rq->to_clean =3D rq->to_clean->next; + rq->ring.desc_avail++; + + vnic_cq_inc_to_clean(cq); + work++; + desc =3D vnic_cq_to_clean(cq); + } + + enic_admin_rq_fill(enic, GFP_ATOMIC); + + return work; +} + +static irqreturn_t enic_admin_isr_msix(int irq, void *data) +{ + struct napi_struct *napi =3D data; + + napi_schedule_irqoff(napi); + + return IRQ_HANDLED; +} + +static void enic_admin_msg_work_handler(struct work_struct *work) +{ + struct enic *enic =3D container_of(work, struct enic, admin_msg_work); + struct enic_admin_msg *msg, *tmp; + LIST_HEAD(local_list); + + spin_lock_bh(&enic->admin_msg_lock); + list_splice_init(&enic->admin_msg_list, &local_list); + spin_unlock_bh(&enic->admin_msg_lock); + + list_for_each_entry_safe(msg, tmp, &local_list, list) { + if (enic->admin_rq_handler) + enic->admin_rq_handler(enic, msg->data, msg->len); + list_del(&msg->list); + kfree(msg); + } +} + +static int enic_admin_napi_poll(struct napi_struct *napi, int budget) +{ + struct enic *enic =3D container_of(napi, struct enic, admin_napi); + unsigned int credits; + unsigned int rq_work; + + credits =3D vnic_intr_credits(&enic->admin_intr); + + rq_work =3D enic_admin_rq_cq_service(enic, budget); + + if (rq_work > 0) + schedule_work(&enic->admin_msg_work); + + if (rq_work < budget && napi_complete_done(napi, rq_work)) { + if (credits) + vnic_intr_return_credits(&enic->admin_intr, credits, + 1 /* unmask */, 0); + } else { + if (credits) + vnic_intr_return_credits(&enic->admin_intr, credits, + 0 /* don't unmask */, 0); + } + + return rq_work; +} + +static int enic_admin_setup_intr(struct enic *enic) +{ + unsigned int intr_index =3D enic->intr_count; + int err; + + if (vnic_dev_get_intr_mode(enic->vdev) !=3D VNIC_DEV_INTR_MODE_MSIX || + intr_index >=3D enic->intr_avail) + return -ENODEV; + + err =3D vnic_intr_alloc(enic->vdev, &enic->admin_intr, intr_index); + if (err) { + netdev_warn(enic->netdev, + "Failed to alloc admin intr at index %u: %d\n", + intr_index, err); + return err; + } + + enic->admin_intr_index =3D intr_index; + + snprintf(enic->msix[intr_index].devname, + sizeof(enic->msix[intr_index].devname), + "%s-admin", enic->netdev->name); + enic->msix[intr_index].isr =3D enic_admin_isr_msix; + enic->msix[intr_index].devid =3D &enic->admin_napi; + + err =3D request_irq(enic->msix_entry[intr_index].vector, + enic->msix[intr_index].isr, 0, + enic->msix[intr_index].devname, + enic->msix[intr_index].devid); + if (err) { + netdev_warn(enic->netdev, + "Failed to request admin MSI-X irq: %d\n", err); + vnic_intr_free(&enic->admin_intr); + return err; + } + + enic->msix[intr_index].requested =3D 1; + + netif_napi_add(enic->netdev, &enic->admin_napi, + enic_admin_napi_poll); + napi_enable(&enic->admin_napi); + + netdev_dbg(enic->netdev, + "admin channel using MSI-X interrupt (index %u)\n", + intr_index); + + return 0; +} + +static void enic_admin_teardown_intr(struct enic *enic) +{ + unsigned int intr_index =3D enic->admin_intr_index; + + napi_disable(&enic->admin_napi); + netif_napi_del(&enic->admin_napi); + + free_irq(enic->msix_entry[intr_index].vector, + enic->msix[intr_index].devid); + enic->msix[intr_index].requested =3D 0; +} + static int enic_admin_qp_type_set(struct enic *enic, u32 enable) { u64 a0 =3D QP_TYPE_ADMIN, a1 =3D enable; @@ -128,23 +329,8 @@ static int enic_admin_alloc_resources(struct enic *eni= c) if (err) goto free_cq0; =20 - /* PFs have dedicated SRIOV_INTR resources for admin channel. - * VFs lack SRIOV_INTR; use a regular INTR_CTRL slot instead. - */ - if (vnic_dev_get_res_count(enic->vdev, RES_TYPE_SRIOV_INTR) >=3D 1) - err =3D vnic_intr_alloc_with_type(enic->vdev, - &enic->admin_intr, 0, - RES_TYPE_SRIOV_INTR); - else - err =3D vnic_intr_alloc(enic->vdev, &enic->admin_intr, - enic->intr_count); - if (err) - goto free_cq1; - return 0; =20 -free_cq1: - vnic_cq_free(&enic->admin_cq[1]); free_cq0: vnic_cq_free(&enic->admin_cq[0]); free_rq: @@ -165,10 +351,32 @@ static void enic_admin_free_resources(struct enic *en= ic) =20 static void enic_admin_init_resources(struct enic *enic) { + unsigned int intr_offset =3D enic->admin_intr_index; + vnic_wq_init(&enic->admin_wq, 0, 0, 0); vnic_rq_init(&enic->admin_rq, 1, 0, 0); - vnic_cq_init(&enic->admin_cq[0], 0, 1, 0, 0, 1, 0, 1, 0, 0, 0); - vnic_cq_init(&enic->admin_cq[1], 0, 1, 0, 0, 1, 0, 1, 0, 0, 0); + vnic_cq_init(&enic->admin_cq[0], + 0 /* flow_control_enable */, + 1 /* color_enable */, + 0 /* cq_head */, + 0 /* cq_tail */, + 1 /* cq_tail_color */, + 1 /* interrupt_enable */, + 1 /* cq_entry_enable */, + 0 /* cq_message_enable */, + intr_offset, + 0 /* cq_message_addr */); + vnic_cq_init(&enic->admin_cq[1], + 0 /* flow_control_enable */, + 1 /* color_enable */, + 0 /* cq_head */, + 0 /* cq_tail */, + 1 /* cq_tail_color */, + 1 /* interrupt_enable */, + 1 /* cq_entry_enable */, + 0 /* cq_message_enable */, + intr_offset, + 0 /* cq_message_addr */); vnic_intr_init(&enic->admin_intr, 0, 0, 1); } =20 @@ -187,12 +395,24 @@ int enic_admin_channel_open(struct enic *enic) return err; } =20 + err =3D enic_admin_setup_intr(enic); + if (err) { + netdev_err(enic->netdev, + "Admin channel requires MSI-X, SR-IOV unavailable: %d\n", + err); + goto free_resources; + } + + spin_lock_init(&enic->admin_msg_lock); + INIT_LIST_HEAD(&enic->admin_msg_list); + INIT_WORK(&enic->admin_msg_work, enic_admin_msg_work_handler); + enic_admin_init_resources(enic); =20 vnic_wq_enable(&enic->admin_wq); vnic_rq_enable(&enic->admin_rq); =20 - err =3D enic_admin_rq_fill(enic); + err =3D enic_admin_rq_fill(enic, GFP_KERNEL); if (err) { netdev_err(enic->netdev, "Failed to fill admin RQ buffers: %d\n", err); @@ -206,22 +426,53 @@ int enic_admin_channel_open(struct enic *enic) goto disable_queues; } =20 + vnic_intr_unmask(&enic->admin_intr); + + netdev_dbg(enic->netdev, + "admin channel open: intr=3D%u wq_avail=3D%u rq_avail=3D%u cq0_color= =3D%u cq1_color=3D%u\n", + enic->admin_intr_index, + vnic_wq_desc_avail(&enic->admin_wq), + vnic_rq_desc_avail(&enic->admin_rq), + enic->admin_cq[0].last_color, + enic->admin_cq[1].last_color); + return 0; =20 disable_queues: + enic_admin_teardown_intr(enic); vnic_wq_disable(&enic->admin_wq); vnic_rq_disable(&enic->admin_rq); enic_admin_qp_type_set(enic, 0); enic_admin_rq_drain(enic); +free_resources: enic_admin_free_resources(enic); return err; } =20 +static void enic_admin_msg_drain(struct enic *enic) +{ + struct enic_admin_msg *msg, *tmp; + + spin_lock_bh(&enic->admin_msg_lock); + list_for_each_entry_safe(msg, tmp, &enic->admin_msg_list, list) { + list_del(&msg->list); + kfree(msg); + } + spin_unlock_bh(&enic->admin_msg_lock); +} + void enic_admin_channel_close(struct enic *enic) { if (!enic->has_admin_channel) return; =20 + netdev_dbg(enic->netdev, "admin channel close\n"); + + vnic_intr_mask(&enic->admin_intr); + enic_admin_teardown_intr(enic); + cancel_work_sync(&enic->admin_msg_work); + enic_admin_msg_drain(enic); + vnic_wq_disable(&enic->admin_wq); vnic_rq_disable(&enic->admin_rq); =20 diff --git a/drivers/net/ethernet/cisco/enic/enic_admin.h b/drivers/net/eth= ernet/cisco/enic/enic_admin.h index 569aadeb9312..73cdd3dac7ec 100644 --- a/drivers/net/ethernet/cisco/enic/enic_admin.h +++ b/drivers/net/ethernet/cisco/enic/enic_admin.h @@ -9,7 +9,19 @@ =20 struct enic; =20 +/* Wrapper for received admin messages queued for deferred processing. + * NAPI enqueues these; a workqueue handler processes them in process cont= ext + * where sleeping (mutex, GFP_KERNEL) is safe. + */ +struct enic_admin_msg { + struct list_head list; + unsigned int len; + u8 data[]; +}; + int enic_admin_channel_open(struct enic *enic); void enic_admin_channel_close(struct enic *enic); +unsigned int enic_admin_wq_cq_service(struct enic *enic); +unsigned int enic_admin_rq_cq_service(struct enic *enic, unsigned int budg= et); =20 #endif /* _ENIC_ADMIN_H_ */ --=20 2.43.0 From nobody Sun Jun 14 23:04:29 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 834471C2324; Tue, 7 Apr 2026 00:37:07 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1775522227; cv=none; b=L0o0I6aVx8elqQvY46uvhKZNvmrI8IpOhucvVeZEIeHEZBzOvn9SOEd3tRrcafOBfEtEZkmPqn25+7cL9gA0Y2ur7N45fWwy0EiodSmoMoKudSSc8IiCQS1llbf2P48s1uA8gXun8i/2eXztilZeG2C8mmVHOZnRPMWHx95AyBI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1775522227; c=relaxed/simple; bh=CHD2sGmHFADoajsajVO5DHFLgnLfs7K8NFH7Cqwp6P0=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=RIxPIBkzpfpuFhJfe/k86qAdSsP7YPRTp0aGdC24wy0jRvhLlXGi8S9ZzFw57J7o4HQDJHXvX5F3zO0zRXcJKrxvNpN6YJmI5CTuuYqg9GWLzIzgvMFg0CcaUqa11MCnD5bwfBQOFVcJfvbAlJJ+BnRt5RCKWTguP9QNwNIMnKs= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=qcQRUstJ; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="qcQRUstJ" Received: by smtp.kernel.org (Postfix) with ESMTPS id 5D091C2BCB6; Tue, 7 Apr 2026 00:37:07 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1775522227; bh=CHD2sGmHFADoajsajVO5DHFLgnLfs7K8NFH7Cqwp6P0=; h=From:Date:Subject:References:In-Reply-To:To:Cc:Reply-To:From; b=qcQRUstJE5YdZoejDNp+7rn6x/ze6ftVNY9ZOqweQHegm3sM+b4hseGfQblZpesbt nACehzVx35IKsaAh/esbQdvaRDnp4zP4I7BzoBkKqWdS3WGGTcc9YHbXJKz9BufsYC jwUvietQIUtyK7syzqdpBSni5LzYBbVuhCKWhA1e+Fr904Bano91I7MYBYY6/+GlUD ais8vQuuRiHzxfQzMOEw2ZqyI20On/+LlmtuufM5WusG4c4l/kg7u2JG4L/Te8efvG U8f7V/tsuzPfYf0T+rjjtingq75RwgesxxK4Lv5YKGm0YRBKXg8pvvU53HaZu7+sIF BvBn3RTiHtvXA== Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 539B0FB5178; Tue, 7 Apr 2026 00:37:07 +0000 (UTC) From: Satish Kharat via B4 Relay Date: Mon, 06 Apr 2026 17:37:03 -0700 Subject: [PATCH 05/10] enic: define MBOX message types and header structures Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260406-enic-sriov-v2-admin-channel-v2-v1-5-82cc47636a78@cisco.com> References: <20260406-enic-sriov-v2-admin-channel-v2-v1-0-82cc47636a78@cisco.com> In-Reply-To: <20260406-enic-sriov-v2-admin-channel-v2-v1-0-82cc47636a78@cisco.com> To: Andrew Lunn , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni Cc: netdev@vger.kernel.org, linux-kernel@vger.kernel.org, 20260401-enic-sriov-v2-prep-v4-0-d5834b2ef1b9@cisco.com, Satish Kharat X-Mailer: b4 0.13.0 X-Developer-Signature: v=1; a=ed25519-sha256; t=1775522225; l=2895; i=satishkh@cisco.com; h=from:subject:message-id; bh=QgPYsFppAoUswndbEnI+khJeTR1Tn6EJVQstlZ2VNn8=; b=z/2/f+Nvt+Qn1oQEZWZO3RC83AE3LkVK33tT1oY42MT9B/sluF5nMKKUVilAdpK7d5PY042QC DMrRLjcGcyMCDo+CFqO6nGXV0X5wYIUYIv5FCJphqmB4di/Go0cV2bG X-Developer-Key: i=satishkh@cisco.com; a=ed25519; pk=4BxbLoCGvHeIMH/PbpiNVx8/2IcTjFvqcr0AD8K6WLY= X-Endpoint-Received: by B4 Relay for satishkh@cisco.com/default with auth_id=639 X-Original-From: Satish Kharat Reply-To: satishkh@cisco.com From: Satish Kharat Define the mailbox protocol used for PF-VF communication over the admin channel. The protocol uses request/reply pairs where even message types are requests and odd are replies. Initial message types cover the core SR-IOV handshake: - VF_CAPABILITY: version negotiation - VF_REGISTER/UNREGISTER: VF lifecycle management - PF_LINK_STATE_NOTIF: PF-initiated link state changes Each message carries a common header (src/dst vnic ID, type, length, sequence number) followed by a type-specific payload. Signed-off-by: Satish Kharat --- drivers/net/ethernet/cisco/enic/enic_mbox.h | 75 +++++++++++++++++++++++++= ++++ 1 file changed, 75 insertions(+) diff --git a/drivers/net/ethernet/cisco/enic/enic_mbox.h b/drivers/net/ethe= rnet/cisco/enic/enic_mbox.h new file mode 100644 index 000000000000..84cb6bbc1ead --- /dev/null +++ b/drivers/net/ethernet/cisco/enic/enic_mbox.h @@ -0,0 +1,75 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* Copyright 2025 Cisco Systems, Inc. All rights reserved. */ + +#ifndef _ENIC_MBOX_H_ +#define _ENIC_MBOX_H_ + +/* + * Mailbox protocol for PF-VF communication over the admin channel. + * + * Even numbers are requests, odd numbers are replies/acks. + * The prefix indicates the initiator: VF_ =3D VF-initiated, PF_ =3D PF-in= itiated. + */ +enum enic_mbox_msg_type { + ENIC_MBOX_VF_CAPABILITY_REQUEST =3D 0, + ENIC_MBOX_VF_CAPABILITY_REPLY =3D 1, + ENIC_MBOX_VF_REGISTER_REQUEST =3D 2, + ENIC_MBOX_VF_REGISTER_REPLY =3D 3, + ENIC_MBOX_VF_UNREGISTER_REQUEST =3D 4, + ENIC_MBOX_VF_UNREGISTER_REPLY =3D 5, + ENIC_MBOX_PF_LINK_STATE_NOTIF =3D 6, + ENIC_MBOX_PF_LINK_STATE_ACK =3D 7, + ENIC_MBOX_MAX +}; + +struct enic_mbox_hdr { + __le16 src_vnic_id; + __le16 dst_vnic_id; + u8 msg_type; + u8 flags; + __le16 msg_len; + __le64 msg_num; +}; + +struct enic_mbox_generic_reply { + __le16 ret_major; + __le16 ret_minor; +}; + +#define ENIC_MBOX_ERR_GENERIC BIT(0) +#define ENIC_MBOX_ERR_VF_NOT_REGISTERED BIT(1) +#define ENIC_MBOX_ERR_MSG_NOT_SUPPORTED BIT(2) + +/* ENIC_MBOX_VF_CAPABILITY_REQUEST / _REPLY */ +#define ENIC_MBOX_CAP_VERSION_0 0 +#define ENIC_MBOX_CAP_VERSION_1 1 + +struct enic_mbox_vf_capability_msg { + __le32 version; + __le32 reserved[32]; +}; + +struct enic_mbox_vf_capability_reply_msg { + struct enic_mbox_generic_reply reply; + __le32 version; + __le32 reserved[32]; +}; + +/* ENIC_MBOX_VF_REGISTER / _UNREGISTER */ +struct enic_mbox_vf_register_reply_msg { + struct enic_mbox_generic_reply reply; +}; + +/* ENIC_MBOX_PF_LINK_STATE_NOTIF / _ACK */ +#define ENIC_MBOX_LINK_STATE_DISABLE 0 +#define ENIC_MBOX_LINK_STATE_ENABLE 1 + +struct enic_mbox_pf_link_state_notif_msg { + __le32 link_state; +}; + +struct enic_mbox_pf_link_state_ack_msg { + struct enic_mbox_generic_reply ack; +}; + +#endif /* _ENIC_MBOX_H_ */ --=20 2.43.0 From nobody Sun Jun 14 23:04:29 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 93B241CAA7D; Tue, 7 Apr 2026 00:37:07 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1775522227; cv=none; b=e3mSM1H/Z6sPc019/GUuuApen2wUuZ+YhBb6Gzjgnp2NkutFGEzQ3ijO3wb5zxyy8Fg+OHgNrzr+3vmSPXo8PftKcEgWjiMe1niluhpd7f69rcI8yPVG79BgTfPtr47qpkc0PWphbvipo8yH3Fvy2xM76MfJESHavcZWUYmdXPE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1775522227; c=relaxed/simple; bh=ObRwmo5R6YAqCQcqHZqIrEwBuFjLvD6CPLpkA/DcCQI=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=PUSMIqSYQOPyCQrnLtYNVqG2kYDAkh/mDHkxQIbmfluN9BoNleWNtSKVEo3lPPdD0SJmSgmTRugtOwk9/OTrJriUPVm4q+j2hARt9Gl2hAKs8hbmm4wQKCEM2i3/Sq+9fkWBKLQMkAoEUL0+F4VSfkTR2EHJDUooNStvvFFTKYw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=EJs29Lll; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="EJs29Lll" Received: by smtp.kernel.org (Postfix) with ESMTPS id 7653DC2BC9E; Tue, 7 Apr 2026 00:37:07 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1775522227; bh=ObRwmo5R6YAqCQcqHZqIrEwBuFjLvD6CPLpkA/DcCQI=; h=From:Date:Subject:References:In-Reply-To:To:Cc:Reply-To:From; b=EJs29Lll3OJX/Fq6zK+6AN/LkrRkdXHNMK0zBtjyQmdLzL5JnSL/Qdw/r3Xrg9Ija LzDqVxJ6y2YEuES3ZCjoudB8J58yc7Af5eLgDnu22EhfI0sIBZ2Vn/EjIOqgtkGH/+ pcLu0RFi8og4ZEaxoFU9gQ8kUwGtm7gJF782r9ttwzvenHrxni8r2SuqVSjLHIxd9o +nKT80ab+wsG/MgoG6WC695OTjminS/940HxDp4f+9OXKgfaT8GCRaOnIwvcRommqF AK3D0321Xbv+jKoinT3SiPtLpg2bPQcCC/qDPeS7Wg5vqnCvqcNXaOo3MQsPHmtABs +j/c3vppPClWg== Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6B455FB517B; Tue, 7 Apr 2026 00:37:07 +0000 (UTC) From: Satish Kharat via B4 Relay Date: Mon, 06 Apr 2026 17:37:04 -0700 Subject: [PATCH 06/10] enic: add MBOX core send and receive for admin channel Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260406-enic-sriov-v2-admin-channel-v2-v1-6-82cc47636a78@cisco.com> References: <20260406-enic-sriov-v2-admin-channel-v2-v1-0-82cc47636a78@cisco.com> In-Reply-To: <20260406-enic-sriov-v2-admin-channel-v2-v1-0-82cc47636a78@cisco.com> To: Andrew Lunn , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni Cc: netdev@vger.kernel.org, linux-kernel@vger.kernel.org, 20260401-enic-sriov-v2-prep-v4-0-d5834b2ef1b9@cisco.com, Satish Kharat X-Mailer: b4 0.13.0 X-Developer-Signature: v=1; a=ed25519-sha256; t=1775522225; l=9171; i=satishkh@cisco.com; h=from:subject:message-id; bh=KyHgU2YHxewnHE1Bfge3P6sARAqORdBDAgbx+yBXJR4=; b=zpjNGWcRI7TDsQKO7euLseFryzp64crMGJL2doQnb1/ja379K3iKms7Wmu9pMS9ca6lMmIt2U bZpkWz1Hm5/Cwp1siIWVbdRYvlTzVsjRwJmLOxYx6EsVnCyd0fplbpR X-Developer-Key: i=satishkh@cisco.com; a=ed25519; pk=4BxbLoCGvHeIMH/PbpiNVx8/2IcTjFvqcr0AD8K6WLY= X-Endpoint-Received: by B4 Relay for satishkh@cisco.com/default with auth_id=639 X-Original-From: Satish Kharat Reply-To: satishkh@cisco.com From: Satish Kharat Implement the mailbox protocol engine used for PF-VF communication over the admin channel. The send path (enic_mbox_send_msg) builds a message with a common header, DMA-maps it, posts a single WQ descriptor with the destination vnic ID encoded in the VLAN tag field, and polls the WQ CQ for completion. The receive path (enic_mbox_recv_handler) is installed as the admin RQ callback and validates incoming message headers. PF/VF-specific dispatch will be added in subsequent commits. Signed-off-by: Satish Kharat --- drivers/net/ethernet/cisco/enic/Makefile | 2 +- drivers/net/ethernet/cisco/enic/enic.h | 5 + drivers/net/ethernet/cisco/enic/enic_admin.c | 23 +++- drivers/net/ethernet/cisco/enic/enic_mbox.c | 153 +++++++++++++++++++++++= ++++ drivers/net/ethernet/cisco/enic/enic_mbox.h | 8 ++ 5 files changed, 189 insertions(+), 2 deletions(-) diff --git a/drivers/net/ethernet/cisco/enic/Makefile b/drivers/net/etherne= t/cisco/enic/Makefile index 7ae72fefc99a..e38aaf34c148 100644 --- a/drivers/net/ethernet/cisco/enic/Makefile +++ b/drivers/net/ethernet/cisco/enic/Makefile @@ -4,5 +4,5 @@ obj-$(CONFIG_ENIC) :=3D enic.o enic-y :=3D enic_main.o vnic_cq.o vnic_intr.o vnic_wq.o \ enic_res.o enic_dev.o enic_pp.o vnic_dev.o vnic_rq.o vnic_vic.o \ enic_ethtool.o enic_api.o enic_clsf.o enic_rq.o enic_wq.o \ - enic_admin.o + enic_admin.o enic_mbox.o =20 diff --git a/drivers/net/ethernet/cisco/enic/enic.h b/drivers/net/ethernet/= cisco/enic/enic.h index 9700fe14e51f..c2ca8d6a4a1c 100644 --- a/drivers/net/ethernet/cisco/enic/enic.h +++ b/drivers/net/ethernet/cisco/enic/enic.h @@ -292,6 +292,7 @@ struct enic { =20 /* Admin channel resources for SR-IOV MBOX */ bool has_admin_channel; + bool mbox_send_disabled; /* set on send timeout; cleared on channel re-op= en */ struct vnic_wq admin_wq; struct vnic_rq admin_rq; struct vnic_cq admin_cq[2]; @@ -303,6 +304,10 @@ struct enic { struct list_head admin_msg_list; u64 admin_msg_drop_cnt; void (*admin_rq_handler)(struct enic *enic, void *buf, unsigned int len); + + /* MBOX protocol state */ + struct mutex mbox_lock; + u64 mbox_msg_num; }; =20 static inline struct net_device *vnic_get_netdev(struct vnic_dev *vdev) diff --git a/drivers/net/ethernet/cisco/enic/enic_admin.c b/drivers/net/eth= ernet/cisco/enic/enic_admin.c index 31f9f754941e..d3ba2cf8f9d3 100644 --- a/drivers/net/ethernet/cisco/enic/enic_admin.c +++ b/drivers/net/ethernet/cisco/enic/enic_admin.c @@ -19,6 +19,7 @@ #include "cq_enet_desc.h" #include "wq_enet_desc.h" #include "rq_enet_desc.h" +#include "enic_mbox.h" =20 /* No-op: admin WQ buffers are freed inline after completion polling */ static void enic_admin_wq_buf_clean(struct vnic_wq *wq, @@ -154,7 +155,26 @@ unsigned int enic_admin_rq_cq_service(struct enic *eni= c, unsigned int budget) buf->dma_addr, buf->len, DMA_FROM_DEVICE); =20 - enic_admin_msg_enqueue(enic, buf->os_buf, buf->len); + if (enic->admin_rq_handler) { + struct cq_enet_rq_desc *rq_desc =3D desc; + u16 sender_vlan; + + /* Firmware sets the CQ VLAN field to identify the + * sender: 0 =3D PF, 1-based =3D VF index. Overwrite + * the untrusted src_vnic_id in the MBOX header with + * the hardware-verified value. + */ + sender_vlan =3D le16_to_cpu(rq_desc->vlan); + if (buf->len >=3D sizeof(struct enic_mbox_hdr)) { + struct enic_mbox_hdr *hdr =3D buf->os_buf; + + hdr->src_vnic_id =3D (sender_vlan =3D=3D 0) ? + cpu_to_le16(ENIC_MBOX_DST_PF) : + cpu_to_le16(sender_vlan - 1); + } + + enic_admin_msg_enqueue(enic, buf->os_buf, buf->len); + } =20 enic_admin_rq_buf_clean(rq, rq->to_clean); rq->to_clean =3D rq->to_clean->next; @@ -387,6 +407,7 @@ int enic_admin_channel_open(struct enic *enic) if (!enic->has_admin_channel) return -ENODEV; =20 + enic->mbox_send_disabled =3D false; err =3D enic_admin_alloc_resources(enic); if (err) { netdev_err(enic->netdev, diff --git a/drivers/net/ethernet/cisco/enic/enic_mbox.c b/drivers/net/ethe= rnet/cisco/enic/enic_mbox.c new file mode 100644 index 000000000000..d7fee4353e14 --- /dev/null +++ b/drivers/net/ethernet/cisco/enic/enic_mbox.c @@ -0,0 +1,153 @@ +// SPDX-License-Identifier: GPL-2.0-only +// Copyright 2025 Cisco Systems, Inc. All rights reserved. + +#include +#include +#include +#include + +#include "vnic_dev.h" +#include "vnic_wq.h" +#include "vnic_cq.h" +#include "enic.h" +#include "enic_admin.h" +#include "enic_mbox.h" +#include "wq_enet_desc.h" + +#define ENIC_MBOX_POLL_TIMEOUT_US 5000000 +#define ENIC_MBOX_POLL_INTERVAL_US 100 + +static void enic_mbox_fill_hdr(struct enic *enic, struct enic_mbox_hdr *hd= r, + u8 msg_type, u16 dst_vnic_id, u16 msg_len) +{ + memset(hdr, 0, sizeof(*hdr)); + hdr->dst_vnic_id =3D cpu_to_le16(dst_vnic_id); + hdr->msg_type =3D msg_type; + hdr->msg_len =3D cpu_to_le16(msg_len); + hdr->msg_num =3D cpu_to_le64(++enic->mbox_msg_num); +} + +int enic_mbox_send_msg(struct enic *enic, u8 msg_type, u16 dst_vnic_id, + void *payload, u16 payload_len) +{ + u16 total_len =3D sizeof(struct enic_mbox_hdr) + payload_len; + struct vnic_wq *wq =3D &enic->admin_wq; + struct wq_enet_desc *desc; + dma_addr_t dma_addr; + unsigned long timeout; + u16 vlan_tag; + void *buf; + int err; + + /* Serialize MBOX sends. The admin channel is a low-frequency + * control path; holding the mutex across the poll is acceptable. + */ + mutex_lock(&enic->mbox_lock); + + if (!enic->has_admin_channel || enic->mbox_send_disabled) { + err =3D -ENODEV; + goto unlock; + } + + if (vnic_wq_desc_avail(wq) =3D=3D 0) { + err =3D -ENOSPC; + goto unlock; + } + + buf =3D kmalloc(total_len, GFP_KERNEL); + if (!buf) { + err =3D -ENOMEM; + goto unlock; + } + + enic_mbox_fill_hdr(enic, buf, msg_type, dst_vnic_id, total_len); + if (payload_len) + memcpy(buf + sizeof(struct enic_mbox_hdr), payload, payload_len); + + dma_addr =3D dma_map_single(&enic->pdev->dev, buf, total_len, + DMA_TO_DEVICE); + if (dma_mapping_error(&enic->pdev->dev, dma_addr)) { + kfree(buf); + err =3D -ENOMEM; + goto unlock; + } + + /* Firmware uses vlan field for routing: 0 =3D PF, 1-based =3D VF index */ + if (dst_vnic_id =3D=3D ENIC_MBOX_DST_PF) + vlan_tag =3D 0; + else + vlan_tag =3D dst_vnic_id + 1; + + desc =3D vnic_wq_next_desc(wq); + wq_enet_desc_enc(desc, (u64)dma_addr | VNIC_PADDR_TARGET, + total_len, 0, 0, 0, 1, 1, 0, 1, vlan_tag, 0); + vnic_wq_post(wq, buf, dma_addr, total_len, 1, 1, 1, 1, 0, 0); + vnic_wq_doorbell(wq); + + timeout =3D jiffies + usecs_to_jiffies(ENIC_MBOX_POLL_TIMEOUT_US); + err =3D -ETIMEDOUT; + while (time_before(jiffies, timeout)) { + if (enic_admin_wq_cq_service(enic)) { + err =3D 0; + break; + } + usleep_range(ENIC_MBOX_POLL_INTERVAL_US, + ENIC_MBOX_POLL_INTERVAL_US + 50); + } + + if (!err) { + wq->to_clean =3D wq->to_clean->next; + wq->ring.desc_avail++; + dma_unmap_single(&enic->pdev->dev, dma_addr, total_len, + DMA_TO_DEVICE); + kfree(buf); + } else { + netdev_err(enic->netdev, + "MBOX send timed out (type %u dst %u), disabling channel\n", + msg_type, dst_vnic_id); + /* + * The WQ descriptor is still live in hardware. Do not unmap + * or free the buffer: the device may still DMA from dma_addr. + * Mark the channel unusable so no further sends are attempted. + */ + enic->mbox_send_disabled =3D true; + } + + netdev_dbg(enic->netdev, + "MBOX send msg_type %u dst %u vlan %u err %d\n", + msg_type, dst_vnic_id, vlan_tag, err); +unlock: + mutex_unlock(&enic->mbox_lock); + return err; +} + +static void enic_mbox_recv_handler(struct enic *enic, void *buf, + unsigned int len) +{ + struct enic_mbox_hdr *hdr =3D buf; + + if (len < sizeof(*hdr)) { + netdev_warn(enic->netdev, + "MBOX: truncated message (len %u < %zu)\n", + len, sizeof(*hdr)); + return; + } + + if (hdr->msg_type >=3D ENIC_MBOX_MAX) { + netdev_warn(enic->netdev, "MBOX: unknown msg type %u\n", + hdr->msg_type); + return; + } + + netdev_dbg(enic->netdev, + "MBOX recv: type %u from vnic %u len %u\n", + hdr->msg_type, le16_to_cpu(hdr->src_vnic_id), + le16_to_cpu(hdr->msg_len)); +} + +void enic_mbox_init(struct enic *enic) +{ + enic->mbox_msg_num =3D 0; + mutex_init(&enic->mbox_lock); + enic->admin_rq_handler =3D enic_mbox_recv_handler; +} diff --git a/drivers/net/ethernet/cisco/enic/enic_mbox.h b/drivers/net/ethe= rnet/cisco/enic/enic_mbox.h index 84cb6bbc1ead..554269b78780 100644 --- a/drivers/net/ethernet/cisco/enic/enic_mbox.h +++ b/drivers/net/ethernet/cisco/enic/enic_mbox.h @@ -72,4 +72,12 @@ struct enic_mbox_pf_link_state_ack_msg { struct enic_mbox_generic_reply ack; }; =20 +#define ENIC_MBOX_DST_PF 0xFFFF + +struct enic; + +void enic_mbox_init(struct enic *enic); +int enic_mbox_send_msg(struct enic *enic, u8 msg_type, u16 dst_vnic_id, + void *payload, u16 payload_len); + #endif /* _ENIC_MBOX_H_ */ --=20 2.43.0 From nobody Sun Jun 14 23:04:29 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A48831CD1E4; Tue, 7 Apr 2026 00:37:07 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1775522227; cv=none; b=m37L2ekF8d+SXqgYtNeeymzz+c2sTbbmNkz3jFmIPymMZEnXIS1I08lFTjjanSCtOuAigULOQVuKOygld5DgrYzf5VhH1wGF/y7WIQjDxqGUoenFuLGxmB9dfpgMJZV9G1nCxvVC2O3MqisQ7nnyJ3ZFAFKp/x89+P0/Bw99bPM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1775522227; c=relaxed/simple; bh=GH158LzLpBVNfOWPcyEUDIANrRs5yCh8tBXxzuZ+4zg=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=tiGKYs3ny1k+b6f2LsYVJ9bnhmmgEKnlL3vRuNThalrfqB1CfRct5zNjkoNnRpK2y81Bob8DCQGpk9uDZCnWde4adoPFaUumntVZLTZ3Yg6FJsVtksMaFZ/KJjIBmnURIsse3N49vDsH81XH1pna1OKptHms97198cThxph+yAg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=ZGlPjY1T; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="ZGlPjY1T" Received: by smtp.kernel.org (Postfix) with ESMTPS id 89AD3C4CEF7; Tue, 7 Apr 2026 00:37:07 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1775522227; bh=GH158LzLpBVNfOWPcyEUDIANrRs5yCh8tBXxzuZ+4zg=; h=From:Date:Subject:References:In-Reply-To:To:Cc:Reply-To:From; b=ZGlPjY1TV32VUAp+W9gVPKNmw9oCGcSi/q4uiicloVl+6+dT5ue5QhEwz9kgPaI8N Gp4nnnhrKlUIblAgZQnogtbJNNtJWlOodezw6U4/SDFx2iSVOuPlpe/cFWaU7/RbaJ hDW25zmJCbRmryJVSnU2jPAfs+DyqPZcGCjgFHHcIrYMeiDi/ixfjfbNMc/yQT0cd8 05eBRNXKB4y/1Jk6kf7E1Xw7oVMHXjyhyiBDzTc1/ZQsETdF9UwOkti0nm3wADkjqQ k+5lEqZy+Zxx+KeGO0X0w5Zbv1bC+wTAOpHV2zBN4amzSqCan4noaxLaXWuVaxxo7h elC5qr/myQ3dQ== Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 821DAFB5175; Tue, 7 Apr 2026 00:37:07 +0000 (UTC) From: Satish Kharat via B4 Relay Date: Mon, 06 Apr 2026 17:37:05 -0700 Subject: [PATCH 07/10] enic: add MBOX PF handlers for VF register and capability Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260406-enic-sriov-v2-admin-channel-v2-v1-7-82cc47636a78@cisco.com> References: <20260406-enic-sriov-v2-admin-channel-v2-v1-0-82cc47636a78@cisco.com> In-Reply-To: <20260406-enic-sriov-v2-admin-channel-v2-v1-0-82cc47636a78@cisco.com> To: Andrew Lunn , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni Cc: netdev@vger.kernel.org, linux-kernel@vger.kernel.org, 20260401-enic-sriov-v2-prep-v4-0-d5834b2ef1b9@cisco.com, Satish Kharat X-Mailer: b4 0.13.0 X-Developer-Signature: v=1; a=ed25519-sha256; t=1775522225; l=8589; i=satishkh@cisco.com; h=from:subject:message-id; bh=N0TNWGDTjM2CEZA/Hbz6xS/2j02o2016QDONXhR2ycs=; b=uZ4KgcWlVLrlBX3waD3PHco4iHpB+y2RclggR3jMtLk+0+OY7i+3x6UngVZAEBcKeBIybfHiP XAhgchppVx+BQ7tuX+AgEPTOCgHzB48YAu2vqSVPNvfJzMShyBH12uD X-Developer-Key: i=satishkh@cisco.com; a=ed25519; pk=4BxbLoCGvHeIMH/PbpiNVx8/2IcTjFvqcr0AD8K6WLY= X-Endpoint-Received: by B4 Relay for satishkh@cisco.com/default with auth_id=639 X-Original-From: Satish Kharat Reply-To: satishkh@cisco.com From: Satish Kharat Implement PF-side mailbox message processing for SR-IOV V2 admin channel communication. When the PF receives messages from VFs, the dispatch routes them to type-specific handlers: - VF_CAPABILITY_REQUEST: reply with protocol version 1 - VF_REGISTER_REQUEST: mark VF registered, reply, then send PF_LINK_STATE_NOTIF with link enabled - VF_UNREGISTER_REQUEST: mark VF unregistered, send reply - PF_LINK_STATE_ACK: log errors from VF acknowledgment Per-VF state (struct enic_vf_state) is tracked via enic->vf_state which will be allocated when SRIOV V2 is enabled. Remove the CONFIG_PCI_IOV guard from num_vfs in struct enic. The PF handlers reference enic->num_vfs for VF ID bounds checking in enic_mbox.c, which is compiled unconditionally. The field must be visible regardless of CONFIG_PCI_IOV to avoid build failures. Add enic_mbox_send_link_state() helper for PF-initiated link state notifications, also used later by ndo_set_vf_link_state. Signed-off-by: Satish Kharat --- drivers/net/ethernet/cisco/enic/enic.h | 7 +- drivers/net/ethernet/cisco/enic/enic_mbox.c | 173 ++++++++++++++++++++++++= +++- drivers/net/ethernet/cisco/enic/enic_mbox.h | 1 + 3 files changed, 177 insertions(+), 4 deletions(-) diff --git a/drivers/net/ethernet/cisco/enic/enic.h b/drivers/net/ethernet/= cisco/enic/enic.h index c2ca8d6a4a1c..64086f172df5 100644 --- a/drivers/net/ethernet/cisco/enic/enic.h +++ b/drivers/net/ethernet/cisco/enic/enic.h @@ -256,9 +256,7 @@ struct enic { struct enic_rx_coal rx_coalesce_setting; u32 rx_coalesce_usecs; u32 tx_coalesce_usecs; -#ifdef CONFIG_PCI_IOV u16 num_vfs; -#endif enum enic_vf_type vf_type; unsigned int enable_count; spinlock_t enic_api_lock; @@ -308,6 +306,11 @@ struct enic { /* MBOX protocol state */ struct mutex mbox_lock; u64 mbox_msg_num; + + /* PF: per-VF MBOX state, allocated when SRIOV V2 is enabled */ + struct enic_vf_state { + bool registered; + } *vf_state; }; =20 static inline struct net_device *vnic_get_netdev(struct vnic_dev *vdev) diff --git a/drivers/net/ethernet/cisco/enic/enic_mbox.c b/drivers/net/ethe= rnet/cisco/enic/enic_mbox.c index d7fee4353e14..799c87082457 100644 --- a/drivers/net/ethernet/cisco/enic/enic_mbox.c +++ b/drivers/net/ethernet/cisco/enic/enic_mbox.c @@ -121,10 +121,167 @@ int enic_mbox_send_msg(struct enic *enic, u8 msg_typ= e, u16 dst_vnic_id, return err; } =20 +int enic_mbox_send_link_state(struct enic *enic, u16 vf_id, u32 link_state) +{ + struct enic_mbox_pf_link_state_notif_msg notif =3D {}; + + if (!enic->vf_state || vf_id >=3D enic->num_vfs || + !enic->vf_state[vf_id].registered) { + netdev_dbg(enic->netdev, + "MBOX: skip link state to unregistered VF %u\n", + vf_id); + return 0; + } + + notif.link_state =3D cpu_to_le32(link_state); + return enic_mbox_send_msg(enic, ENIC_MBOX_PF_LINK_STATE_NOTIF, vf_id, + ¬if, sizeof(notif)); +} + +static int enic_mbox_pf_handle_capability(struct enic *enic, void *msg, + u16 vf_id, u64 msg_num) +{ + struct enic_mbox_vf_capability_reply_msg reply =3D {}; + + reply.reply.ret_major =3D cpu_to_le16(0); + reply.version =3D cpu_to_le32(ENIC_MBOX_CAP_VERSION_1); + + return enic_mbox_send_msg(enic, ENIC_MBOX_VF_CAPABILITY_REPLY, vf_id, + &reply, sizeof(reply)); +} + +static int enic_mbox_pf_handle_register(struct enic *enic, void *msg, + u16 vf_id, u64 msg_num) +{ + struct enic_mbox_vf_register_reply_msg reply =3D {}; + int err; + + if (!enic->vf_state || vf_id >=3D enic->num_vfs) { + netdev_warn(enic->netdev, + "MBOX: register from invalid VF %u\n", vf_id); + return -EINVAL; + } + + /* VF re-registering (e.g. guest reboot without clean unregister): + * mark the previous registration inactive before accepting the new one. + */ + if (enic->vf_state[vf_id].registered) { + netdev_dbg(enic->netdev, + "MBOX: VF %u re-register, cleaning previous state\n", + vf_id); + enic->vf_state[vf_id].registered =3D false; + } + + reply.reply.ret_major =3D cpu_to_le16(0); + err =3D enic_mbox_send_msg(enic, ENIC_MBOX_VF_REGISTER_REPLY, vf_id, + &reply, sizeof(reply)); + if (err) + return err; + + enic->vf_state[vf_id].registered =3D true; + netdev_info(enic->netdev, "VF %u registered via MBOX\n", vf_id); + + err =3D enic_mbox_send_link_state(enic, vf_id, ENIC_MBOX_LINK_STATE_ENABL= E); + if (err) + netdev_warn(enic->netdev, + "VF %u: failed to send initial link state: %d\n", + vf_id, err); + /* Registration succeeded; link state will be (re-)sent on next + * enic_link_check() event. + */ + return 0; +} + +static int enic_mbox_pf_handle_unregister(struct enic *enic, void *msg, + u16 vf_id, u64 msg_num) +{ + struct enic_mbox_vf_register_reply_msg reply =3D {}; + int err; + + if (!enic->vf_state || vf_id >=3D enic->num_vfs) { + netdev_warn(enic->netdev, + "MBOX: unregister from invalid VF %u\n", vf_id); + return -EINVAL; + } + + reply.reply.ret_major =3D cpu_to_le16(0); + err =3D enic_mbox_send_msg(enic, ENIC_MBOX_VF_UNREGISTER_REPLY, vf_id, + &reply, sizeof(reply)); + if (err) + return err; + + enic->vf_state[vf_id].registered =3D false; + + netdev_info(enic->netdev, "VF %u unregistered via MBOX\n", vf_id); + + return 0; +} + +static void enic_mbox_pf_process_msg(struct enic *enic, + struct enic_mbox_hdr *hdr, void *payload) +{ + u16 vf_id =3D le16_to_cpu(hdr->src_vnic_id); + u16 msg_len =3D le16_to_cpu(hdr->msg_len); + int err =3D 0; + + if (!enic->vf_state) { + netdev_dbg(enic->netdev, + "MBOX: PF received msg but SRIOV not active\n"); + return; + } + + if (vf_id >=3D enic->num_vfs) { + netdev_warn(enic->netdev, + "MBOX: PF received msg from invalid VF %u\n", + vf_id); + return; + } + + switch (hdr->msg_type) { + case ENIC_MBOX_VF_CAPABILITY_REQUEST: + err =3D enic_mbox_pf_handle_capability(enic, payload, vf_id, + le64_to_cpu(hdr->msg_num)); + break; + case ENIC_MBOX_VF_REGISTER_REQUEST: + err =3D enic_mbox_pf_handle_register(enic, payload, vf_id, + le64_to_cpu(hdr->msg_num)); + break; + case ENIC_MBOX_VF_UNREGISTER_REQUEST: + err =3D enic_mbox_pf_handle_unregister(enic, payload, vf_id, + le64_to_cpu(hdr->msg_num)); + break; + case ENIC_MBOX_PF_LINK_STATE_ACK: { + struct enic_mbox_pf_link_state_ack_msg *ack =3D payload; + + if (msg_len < sizeof(*hdr) + sizeof(*ack)) + break; + if (le16_to_cpu(ack->ack.ret_major)) + netdev_warn(enic->netdev, + "MBOX: VF %u link state ACK error %u/%u\n", + vf_id, le16_to_cpu(ack->ack.ret_major), + le16_to_cpu(ack->ack.ret_minor)); + break; + } + default: + netdev_dbg(enic->netdev, + "MBOX: PF unhandled msg type %u from VF %u\n", + hdr->msg_type, vf_id); + err =3D -EOPNOTSUPP; + break; + } + + if (err) + netdev_warn(enic->netdev, + "MBOX: PF handler for msg type %u from VF %u failed: %d\n", + hdr->msg_type, vf_id, err); +} + static void enic_mbox_recv_handler(struct enic *enic, void *buf, unsigned int len) { struct enic_mbox_hdr *hdr =3D buf; + void *payload; + u16 msg_len; =20 if (len < sizeof(*hdr)) { netdev_warn(enic->netdev, @@ -139,10 +296,22 @@ static void enic_mbox_recv_handler(struct enic *enic,= void *buf, return; } =20 + msg_len =3D le16_to_cpu(hdr->msg_len); + if (msg_len < sizeof(*hdr) || msg_len > len) { + netdev_warn(enic->netdev, + "MBOX: invalid msg_len %u (buf len %u)\n", + msg_len, len); + return; + } + netdev_dbg(enic->netdev, "MBOX recv: type %u from vnic %u len %u\n", - hdr->msg_type, le16_to_cpu(hdr->src_vnic_id), - le16_to_cpu(hdr->msg_len)); + hdr->msg_type, le16_to_cpu(hdr->src_vnic_id), msg_len); + + payload =3D buf + sizeof(*hdr); + + if (enic->vf_state) + enic_mbox_pf_process_msg(enic, hdr, payload); } =20 void enic_mbox_init(struct enic *enic) diff --git a/drivers/net/ethernet/cisco/enic/enic_mbox.h b/drivers/net/ethe= rnet/cisco/enic/enic_mbox.h index 554269b78780..a6f6798d14f4 100644 --- a/drivers/net/ethernet/cisco/enic/enic_mbox.h +++ b/drivers/net/ethernet/cisco/enic/enic_mbox.h @@ -79,5 +79,6 @@ struct enic; void enic_mbox_init(struct enic *enic); int enic_mbox_send_msg(struct enic *enic, u8 msg_type, u16 dst_vnic_id, void *payload, u16 payload_len); +int enic_mbox_send_link_state(struct enic *enic, u16 vf_id, u32 link_state= ); =20 #endif /* _ENIC_MBOX_H_ */ --=20 2.43.0 From nobody Sun Jun 14 23:04:29 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B5CBF1DE2D8; Tue, 7 Apr 2026 00:37:07 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1775522227; cv=none; b=sugsd0idtJBmx6RP8nitOOB/1onr2phiQKZuMNScINT36k/PmJorz+y+W37/XCN/6UHVddIz6C7lW9bqGiLLoek01BeHbCtZHTam5y/QKi330l7e28GaaAF9BJUqG53UcBdd3KosBjHsljo9qkxJgOrzM4i+Jf6nMiVxieSHIbk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1775522227; c=relaxed/simple; bh=j1zrujaGHMx9vg7URAMd++7ZX0ZIK1Y1QUc0RQNoq/I=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=WKzV+hsD6e4xm5xI+B2SFiPDFMGwawFnaIKDtyJcN+ImNCH4LwNR5WKmJtxK+2ZW5/nV+XeW0Vv/x0UULDqTrN4ZRFERHMhHkGG03orVvvmCzzZxYS0n5gJJCeHc0LGlzXoN/6V3ApPCA26I3WUI9X/9p4gHE150siDQh9wedcg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=X6w5TIFT; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="X6w5TIFT" Received: by smtp.kernel.org (Postfix) with ESMTPS id 9A787C2BCB0; Tue, 7 Apr 2026 00:37:07 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1775522227; bh=j1zrujaGHMx9vg7URAMd++7ZX0ZIK1Y1QUc0RQNoq/I=; h=From:Date:Subject:References:In-Reply-To:To:Cc:Reply-To:From; b=X6w5TIFTaI8dKAQmUywt2Ze9FPsS/F/uRsb4mb1bl631Brk/zBdReZ4XW/44noZFC KANKzJc7/mz7+CzgR1ovC9/szCDm3Mgrrfhc+9ypNk4Om/4i2fZePVvpyJu+ojr+6d g54suxDJ7qb+FYMzd9NEfEuE1MnCZWdlUKaDUCccIhmGIyrTD6eW4W7zCNfYeg9BO6 dT4Z8NvnLBen1roshlRcpH2vLhry39bUUagCRQeWUyqU4D6u7mMX/eYQdaIZO3OeaH iwUePWvWfAIWwPajnKAsTuwJtIZ+4FsKt9o23XJb9i+Vqs9H8E7TOp3tcbaABf8kZL Ap4NNR6nS35bw== Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9252FFB516E; Tue, 7 Apr 2026 00:37:07 +0000 (UTC) From: Satish Kharat via B4 Relay Date: Mon, 06 Apr 2026 17:37:06 -0700 Subject: [PATCH 08/10] enic: add MBOX VF handlers for capability, register and link state Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260406-enic-sriov-v2-admin-channel-v2-v1-8-82cc47636a78@cisco.com> References: <20260406-enic-sriov-v2-admin-channel-v2-v1-0-82cc47636a78@cisco.com> In-Reply-To: <20260406-enic-sriov-v2-admin-channel-v2-v1-0-82cc47636a78@cisco.com> To: Andrew Lunn , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni Cc: netdev@vger.kernel.org, linux-kernel@vger.kernel.org, 20260401-enic-sriov-v2-prep-v4-0-d5834b2ef1b9@cisco.com, Satish Kharat X-Mailer: b4 0.13.0 X-Developer-Signature: v=1; a=ed25519-sha256; t=1775522225; l=10087; i=satishkh@cisco.com; h=from:subject:message-id; bh=DmJ4teZg59FAWsrjKm/5UAquskfZPDrYNwMERalPXx4=; b=0zSIBeTp0mxVVN2lqBcK68d/HJXvlrJrxGTjvQ1+s1Bd08hBl45vv1dRM4qW885+adfB41RNp j+KLdWlIocuCqsPpIlFOYLa9QRmoy8fJxxL5oHbGz6xADsMyxz0FLSF X-Developer-Key: i=satishkh@cisco.com; a=ed25519; pk=4BxbLoCGvHeIMH/PbpiNVx8/2IcTjFvqcr0AD8K6WLY= X-Endpoint-Received: by B4 Relay for satishkh@cisco.com/default with auth_id=639 X-Original-From: Satish Kharat Reply-To: satishkh@cisco.com From: Satish Kharat Implement VF-side mailbox message processing for SR-IOV V2 admin channel communication. VF receive handlers: - VF_CAPABILITY_REPLY: store PF protocol version, signal completion - VF_REGISTER_REPLY: mark VF as registered, signal completion - VF_UNREGISTER_REPLY: mark VF as unregistered, signal completion - PF_LINK_STATE_NOTIF: update carrier state via netif_carrier_on/off, send ACK back to PF VF initiation functions for the probe-time handshake: - enic_mbox_vf_capability_check: send capability request, wait for PF reply via completion - enic_mbox_vf_register: send register request, wait for PF confirmation via completion - enic_mbox_vf_unregister: send unregister request, wait for PF confirmation The wait helper (enic_mbox_wait_reply) uses wait_for_completion_timeout, signaled when the admin ISR/NAPI/ workqueue pipeline delivers the reply message. Signed-off-by: Satish Kharat --- drivers/net/ethernet/cisco/enic/enic.h | 9 +- drivers/net/ethernet/cisco/enic/enic_mbox.c | 208 ++++++++++++++++++++++++= ++++ drivers/net/ethernet/cisco/enic/enic_mbox.h | 3 + 3 files changed, 219 insertions(+), 1 deletion(-) diff --git a/drivers/net/ethernet/cisco/enic/enic.h b/drivers/net/ethernet/= cisco/enic/enic.h index 64086f172df5..5ef4ed86174f 100644 --- a/drivers/net/ethernet/cisco/enic/enic.h +++ b/drivers/net/ethernet/cisco/enic/enic.h @@ -258,6 +258,8 @@ struct enic { u32 tx_coalesce_usecs; u16 num_vfs; enum enic_vf_type vf_type; + bool vf_registered; + u32 pf_cap_version; unsigned int enable_count; spinlock_t enic_api_lock; bool enic_api_busy; @@ -303,9 +305,14 @@ struct enic { u64 admin_msg_drop_cnt; void (*admin_rq_handler)(struct enic *enic, void *buf, unsigned int len); =20 - /* MBOX protocol state */ + /* MBOX protocol state -- single-flight: on the VF, all callers + * that wait on mbox_comp run under RTNL or during probe/remove, + * so only one completion is outstanding at a time. mbox_lock + * protects the shared admin WQ from concurrent senders. + */ struct mutex mbox_lock; u64 mbox_msg_num; + struct completion mbox_comp; =20 /* PF: per-VF MBOX state, allocated when SRIOV V2 is enabled */ struct enic_vf_state { diff --git a/drivers/net/ethernet/cisco/enic/enic_mbox.c b/drivers/net/ethe= rnet/cisco/enic/enic_mbox.c index 799c87082457..ad74669d7b2a 100644 --- a/drivers/net/ethernet/cisco/enic/enic_mbox.c +++ b/drivers/net/ethernet/cisco/enic/enic_mbox.c @@ -5,6 +5,7 @@ #include #include #include +#include =20 #include "vnic_dev.h" #include "vnic_wq.h" @@ -121,6 +122,16 @@ int enic_mbox_send_msg(struct enic *enic, u8 msg_type,= u16 dst_vnic_id, return err; } =20 +static int enic_mbox_wait_reply(struct enic *enic, unsigned long timeout_m= s) +{ + unsigned long left; + + left =3D wait_for_completion_timeout(&enic->mbox_comp, + msecs_to_jiffies(timeout_ms)); + + return left ? 0 : -ETIMEDOUT; +} + int enic_mbox_send_link_state(struct enic *enic, u16 vf_id, u32 link_state) { struct enic_mbox_pf_link_state_notif_msg notif =3D {}; @@ -276,6 +287,124 @@ static void enic_mbox_pf_process_msg(struct enic *eni= c, hdr->msg_type, vf_id, err); } =20 +static void enic_mbox_vf_handle_capability_reply(struct enic *enic, + void *payload) +{ + struct enic_mbox_vf_capability_reply_msg *reply =3D payload; + + if (le16_to_cpu(reply->reply.ret_major) =3D=3D 0) + enic->pf_cap_version =3D le32_to_cpu(reply->version); + complete(&enic->mbox_comp); +} + +static void enic_mbox_vf_handle_register_reply(struct enic *enic, + void *payload) +{ + struct enic_mbox_vf_register_reply_msg *reply =3D payload; + + if (le16_to_cpu(reply->reply.ret_major)) { + netdev_warn(enic->netdev, + "MBOX: VF register rejected by PF: %u/%u\n", + le16_to_cpu(reply->reply.ret_major), + le16_to_cpu(reply->reply.ret_minor)); + } else { + enic->vf_registered =3D true; + } + complete(&enic->mbox_comp); +} + +static void enic_mbox_vf_handle_unregister_reply(struct enic *enic, + void *payload) +{ + struct enic_mbox_vf_register_reply_msg *reply =3D payload; + + if (le16_to_cpu(reply->reply.ret_major)) { + netdev_warn(enic->netdev, + "MBOX: VF unregister rejected by PF: %u/%u\n", + le16_to_cpu(reply->reply.ret_major), + le16_to_cpu(reply->reply.ret_minor)); + } else { + enic->vf_registered =3D false; + } + complete(&enic->mbox_comp); +} + +static void enic_mbox_vf_handle_link_state(struct enic *enic, void *payloa= d) +{ + struct enic_mbox_pf_link_state_notif_msg *notif =3D payload; + struct enic_mbox_pf_link_state_ack_msg ack =3D {}; + + switch (le32_to_cpu(notif->link_state)) { + case ENIC_MBOX_LINK_STATE_ENABLE: + if (!netif_carrier_ok(enic->netdev)) + netif_carrier_on(enic->netdev); + netdev_dbg(enic->netdev, "MBOX: link state -> UP\n"); + break; + case ENIC_MBOX_LINK_STATE_DISABLE: + if (netif_carrier_ok(enic->netdev)) + netif_carrier_off(enic->netdev); + netdev_dbg(enic->netdev, "MBOX: link state -> DOWN\n"); + break; + default: + netdev_warn(enic->netdev, "MBOX: unknown link state %u\n", + le32_to_cpu(notif->link_state)); + ack.ack.ret_major =3D cpu_to_le16(ENIC_MBOX_ERR_GENERIC); + break; + } + + enic_mbox_send_msg(enic, ENIC_MBOX_PF_LINK_STATE_ACK, ENIC_MBOX_DST_PF, + &ack, sizeof(ack)); +} + +static bool enic_mbox_vf_payload_ok(struct enic *enic, u8 msg_type, + u16 payload_len, size_t min_len) +{ + if (payload_len < min_len) { + netdev_warn(enic->netdev, + "MBOX: short payload for type %u (%u < %zu)\n", + msg_type, payload_len, min_len); + return false; + } + return true; +} + +static void enic_mbox_vf_process_msg(struct enic *enic, + struct enic_mbox_hdr *hdr, void *payload, + u16 payload_len) +{ + switch (hdr->msg_type) { + case ENIC_MBOX_VF_CAPABILITY_REPLY: + if (!enic_mbox_vf_payload_ok(enic, hdr->msg_type, payload_len, + sizeof(struct enic_mbox_vf_capability_reply_msg))) + return; + enic_mbox_vf_handle_capability_reply(enic, payload); + break; + case ENIC_MBOX_VF_REGISTER_REPLY: + if (!enic_mbox_vf_payload_ok(enic, hdr->msg_type, payload_len, + sizeof(struct enic_mbox_vf_register_reply_msg))) + return; + enic_mbox_vf_handle_register_reply(enic, payload); + break; + case ENIC_MBOX_VF_UNREGISTER_REPLY: + if (!enic_mbox_vf_payload_ok(enic, hdr->msg_type, payload_len, + sizeof(struct enic_mbox_vf_register_reply_msg))) + return; + enic_mbox_vf_handle_unregister_reply(enic, payload); + break; + case ENIC_MBOX_PF_LINK_STATE_NOTIF: + if (!enic_mbox_vf_payload_ok(enic, hdr->msg_type, payload_len, + sizeof(struct enic_mbox_pf_link_state_notif_msg))) + return; + enic_mbox_vf_handle_link_state(enic, payload); + break; + default: + netdev_dbg(enic->netdev, + "MBOX: VF unhandled msg type %u\n", + hdr->msg_type); + break; + } +} + static void enic_mbox_recv_handler(struct enic *enic, void *buf, unsigned int len) { @@ -312,11 +441,90 @@ static void enic_mbox_recv_handler(struct enic *enic,= void *buf, =20 if (enic->vf_state) enic_mbox_pf_process_msg(enic, hdr, payload); + else + enic_mbox_vf_process_msg(enic, hdr, payload, + msg_len - (u16)sizeof(*hdr)); +} + +int enic_mbox_vf_capability_check(struct enic *enic) +{ + struct enic_mbox_vf_capability_msg req =3D {}; + int err; + + enic->pf_cap_version =3D 0; + reinit_completion(&enic->mbox_comp); + req.version =3D cpu_to_le32(ENIC_MBOX_CAP_VERSION_1); + + err =3D enic_mbox_send_msg(enic, ENIC_MBOX_VF_CAPABILITY_REQUEST, + ENIC_MBOX_DST_PF, &req, sizeof(req)); + if (err) + return err; + + err =3D enic_mbox_wait_reply(enic, 3000); + if (err) { + netdev_warn(enic->netdev, + "MBOX: no capability reply from PF\n"); + return err; + } + + if (enic->pf_cap_version < ENIC_MBOX_CAP_VERSION_1) { + netdev_warn(enic->netdev, + "MBOX: PF version %u too old\n", + enic->pf_cap_version); + return -EOPNOTSUPP; + } + + return 0; +} + +int enic_mbox_vf_register(struct enic *enic) +{ + int err; + + enic->vf_registered =3D false; + reinit_completion(&enic->mbox_comp); + + err =3D enic_mbox_send_msg(enic, ENIC_MBOX_VF_REGISTER_REQUEST, + ENIC_MBOX_DST_PF, NULL, 0); + if (err) + return err; + + err =3D enic_mbox_wait_reply(enic, 3000); + if (err) { + netdev_warn(enic->netdev, + "MBOX: VF registration with PF timed out\n"); + return err; + } + + if (!enic->vf_registered) + return -ENODEV; + + return 0; +} + +int enic_mbox_vf_unregister(struct enic *enic) +{ + int err; + + if (!enic->vf_registered) + return 0; + + reinit_completion(&enic->mbox_comp); + + err =3D enic_mbox_send_msg(enic, ENIC_MBOX_VF_UNREGISTER_REQUEST, + ENIC_MBOX_DST_PF, NULL, 0); + if (err) + return err; + + err =3D enic_mbox_wait_reply(enic, 3000); + + return enic->vf_registered ? -ETIMEDOUT : 0; } =20 void enic_mbox_init(struct enic *enic) { enic->mbox_msg_num =3D 0; mutex_init(&enic->mbox_lock); + init_completion(&enic->mbox_comp); enic->admin_rq_handler =3D enic_mbox_recv_handler; } diff --git a/drivers/net/ethernet/cisco/enic/enic_mbox.h b/drivers/net/ethe= rnet/cisco/enic/enic_mbox.h index a6f6798d14f4..fa2fb08bf7d0 100644 --- a/drivers/net/ethernet/cisco/enic/enic_mbox.h +++ b/drivers/net/ethernet/cisco/enic/enic_mbox.h @@ -80,5 +80,8 @@ void enic_mbox_init(struct enic *enic); int enic_mbox_send_msg(struct enic *enic, u8 msg_type, u16 dst_vnic_id, void *payload, u16 payload_len); int enic_mbox_send_link_state(struct enic *enic, u16 vf_id, u32 link_state= ); +int enic_mbox_vf_capability_check(struct enic *enic); +int enic_mbox_vf_register(struct enic *enic); +int enic_mbox_vf_unregister(struct enic *enic); =20 #endif /* _ENIC_MBOX_H_ */ --=20 2.43.0 From nobody Sun Jun 14 23:04:29 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C78581DF736; Tue, 7 Apr 2026 00:37:07 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1775522227; cv=none; b=ME/WldMmiCmbqm3T4DXpqfF7m19IDpefvLvEsGKzBttLnHlkBFnjeKBiA9+f8xdJ31IJ2Hyt+uXAd7d4zPCDHJwBIExUHzZYK30XVPVVAJf/lJ9z/d3pe8e3yoUfJ3bxhNyQCzv5O2XInVTN39hBmKBGxnWVkFQitao3Ekqfob8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1775522227; c=relaxed/simple; bh=ms+J4N9O80S4ubcp2liVfP2bJCCZouxMic+lySYVq5M=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=Lf0wfgccqBP+x6nW+89MB2puYszlhdUM4weeKBWW0u7krC3Z7l5FSptVm4oGdJp3uP5r37bc1b4hG7Zxg3srA3bvKvgCzev01BvzDqvcz9zeUDbAWGP0O/T3fYWWUTQb6/wfmSrMxMxHFjUY9e72kOQ7cmLjaI6eUCjDCezTp2w= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=JrSb+AkB; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="JrSb+AkB" Received: by smtp.kernel.org (Postfix) with ESMTPS id ACF64C2BCB2; Tue, 7 Apr 2026 00:37:07 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1775522227; bh=ms+J4N9O80S4ubcp2liVfP2bJCCZouxMic+lySYVq5M=; h=From:Date:Subject:References:In-Reply-To:To:Cc:Reply-To:From; b=JrSb+AkBA+cGNpM0JporyBMKVmv3hN7qZ3bs0dh4FcXXYMlQlRABb8QE0bz7wZUr2 sYZaBCzaA1nXlDQ2cWqE3HUGZDIkv5qo/y/MGR1VU5sB7Fyug2DfIxBW8PD5CFgXHG dMeKbP0KIlTvPXcDFgDUpUKT9oKA6kkWW89ehzC4jGh6CpJ2LBnrGHTgarAqLm7bBL Yxp58BlfY89/sWcqfEk/kG22IbSS4FYUCjB9UMdi/7lJzZ8o3RN8/tXh/NroWFXjUY gwmFzXcG3YM3IQAeashJukZWOV8AsszHlWcgAKDbh7txs6mQnQGpND4MrPWuF/kIsh /ZBpo1VPWcvHg== Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id A4CA1FB5178; Tue, 7 Apr 2026 00:37:07 +0000 (UTC) From: Satish Kharat via B4 Relay Date: Mon, 06 Apr 2026 17:37:07 -0700 Subject: [PATCH 09/10] enic: wire V2 SR-IOV enable with admin channel and MBOX Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260406-enic-sriov-v2-admin-channel-v2-v1-9-82cc47636a78@cisco.com> References: <20260406-enic-sriov-v2-admin-channel-v2-v1-0-82cc47636a78@cisco.com> In-Reply-To: <20260406-enic-sriov-v2-admin-channel-v2-v1-0-82cc47636a78@cisco.com> To: Andrew Lunn , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni Cc: netdev@vger.kernel.org, linux-kernel@vger.kernel.org, 20260401-enic-sriov-v2-prep-v4-0-d5834b2ef1b9@cisco.com, Satish Kharat X-Mailer: b4 0.13.0 X-Developer-Signature: v=1; a=ed25519-sha256; t=1775522225; l=6666; i=satishkh@cisco.com; h=from:subject:message-id; bh=pYDQwwmIn2nak9DSjQMEZuHxqsKqwmTS/K+DiofNc/k=; b=NAiKzr57YAIadVeQYEOYunFj0eIRb8JYpKCv4AiMw3Yl3hXtaBEdwhtgGoQkb0twQ11BkxPrQ MgB2T9Q2TN0BSVDHdHT6qerNdCvVmranTeaINP7feMIF6dRS2+W9vfv X-Developer-Key: i=satishkh@cisco.com; a=ed25519; pk=4BxbLoCGvHeIMH/PbpiNVx8/2IcTjFvqcr0AD8K6WLY= X-Endpoint-Received: by B4 Relay for satishkh@cisco.com/default with auth_id=639 X-Original-From: Satish Kharat Reply-To: satishkh@cisco.com From: Satish Kharat Extend enic_sriov_configure() to handle V2 SR-IOV VFs. When the PF detects V2 VF device IDs, the enable path allocates per-VF MBOX state, opens the admin channel, initializes the MBOX protocol, and then calls pci_enable_sriov(). The admin channel must be ready before VFs are created so that VF drivers can immediately begin the MBOX capability and registration handshake during their probe. The disable path reverses this order: pci_disable_sriov() first (so VF drivers unregister via MBOX), then the admin channel is closed and per-VF state is freed. The existing V1/USNIC SR-IOV paths are unchanged. Signed-off-by: Satish Kharat --- drivers/net/ethernet/cisco/enic/enic_main.c | 132 ++++++++++++++++++++++++= ++-- drivers/net/ethernet/cisco/enic/enic_res.c | 1 + drivers/net/ethernet/cisco/enic/vnic_enet.h | 4 +- 3 files changed, 130 insertions(+), 7 deletions(-) diff --git a/drivers/net/ethernet/cisco/enic/enic_main.c b/drivers/net/ethe= rnet/cisco/enic/enic_main.c index 3a4afd6da41f..4328e2a79619 100644 --- a/drivers/net/ethernet/cisco/enic/enic_main.c +++ b/drivers/net/ethernet/cisco/enic/enic_main.c @@ -60,6 +60,8 @@ #include "enic_clsf.h" #include "enic_rq.h" #include "enic_wq.h" +#include "enic_admin.h" +#include "enic_mbox.h" =20 #define ENIC_NOTIFY_TIMER_PERIOD (2 * HZ) =20 @@ -2688,6 +2690,118 @@ static void enic_sriov_detect_vf_type(struct enic *= enic) } } } + +static int enic_sriov_v2_enable(struct enic *enic, int num_vfs) +{ + int err; + + if (!enic->has_admin_channel) { + netdev_err(enic->netdev, + "V2 SR-IOV requires admin channel resources\n"); + return -EOPNOTSUPP; + } + + enic->vf_state =3D kcalloc(num_vfs, sizeof(*enic->vf_state), GFP_KERNEL); + if (!enic->vf_state) + return -ENOMEM; + + err =3D enic_admin_channel_open(enic); + if (err) { + netdev_err(enic->netdev, + "Failed to open admin channel: %d\n", err); + goto free_vf_state; + } + + enic_mbox_init(enic); + + enic->num_vfs =3D num_vfs; + + err =3D pci_enable_sriov(enic->pdev, num_vfs); + if (err) { + netdev_err(enic->netdev, + "pci_enable_sriov failed: %d\n", err); + goto close_admin; + } + + enic->priv_flags |=3D ENIC_SRIOV_ENABLED; + return num_vfs; + +close_admin: + enic->num_vfs =3D 0; + enic_admin_channel_close(enic); +free_vf_state: + kfree(enic->vf_state); + enic->vf_state =3D NULL; + return err; +} + +static void enic_sriov_v2_disable(struct enic *enic) +{ + pci_disable_sriov(enic->pdev); + enic_admin_channel_close(enic); + kfree(enic->vf_state); + enic->vf_state =3D NULL; + enic->num_vfs =3D 0; + enic->priv_flags &=3D ~ENIC_SRIOV_ENABLED; +} + +static int enic_sriov_configure(struct pci_dev *pdev, int num_vfs) +{ + struct net_device *netdev =3D pci_get_drvdata(pdev); + struct enic *enic =3D netdev_priv(netdev); + struct enic_port_profile *pp; + int err; + + if (num_vfs > 0) { + if (enic->config.mq_subvnic_count) { + netdev_err(netdev, + "SR-IOV not supported with multi-queue sub-vnics\n"); + return -EOPNOTSUPP; + } + + if (enic->vf_type =3D=3D ENIC_VF_TYPE_NONE) { + netdev_err(netdev, + "SR-IOV not supported on this firmware version\n"); + return -EOPNOTSUPP; + } + + if (enic->vf_type =3D=3D ENIC_VF_TYPE_V2) + return enic_sriov_v2_enable(enic, num_vfs); + + pp =3D kcalloc(num_vfs, sizeof(*pp), GFP_KERNEL); + if (!pp) + return -ENOMEM; + + err =3D pci_enable_sriov(pdev, num_vfs); + if (err) { + kfree(pp); + return err; + } + + kfree(enic->pp); + enic->pp =3D pp; + enic->num_vfs =3D num_vfs; + enic->priv_flags |=3D ENIC_SRIOV_ENABLED; + return num_vfs; + } + + if (!enic_sriov_enabled(enic)) + return 0; + + if (enic->vf_type =3D=3D ENIC_VF_TYPE_V2) { + enic_sriov_v2_disable(enic); + return 0; + } + + pci_disable_sriov(pdev); + enic->num_vfs =3D 0; + enic->priv_flags &=3D ~ENIC_SRIOV_ENABLED; + + kfree(enic->pp); + enic->pp =3D kzalloc(sizeof(*enic->pp), GFP_KERNEL); + + return 0; +} #endif =20 static int enic_probe(struct pci_dev *pdev, const struct pci_device_id *en= t) @@ -2786,7 +2900,12 @@ static int enic_probe(struct pci_dev *pdev, const st= ruct pci_device_id *ent) goto err_out_vnic_unregister; =20 #ifdef CONFIG_PCI_IOV - /* Get number of subvnics */ + enic_sriov_detect_vf_type(enic); + + /* Auto-enable SR-IOV if VFs were pre-configured (e.g. at boot). + * V2 VFs require the admin channel, which is not yet set up at probe + * time; use sysfs (enic_sriov_configure) to enable V2 SR-IOV instead. + */ pos =3D pci_find_ext_capability(pdev, PCI_EXT_CAP_ID_SRIOV); if (pos) { pci_read_config_word(pdev, pos + PCI_SRIOV_TOTAL_VF, @@ -2803,7 +2922,6 @@ static int enic_probe(struct pci_dev *pdev, const str= uct pci_device_id *ent) num_pps =3D enic->num_vfs; } } - enic_sriov_detect_vf_type(enic); #endif =20 /* Allocate structure for port profiles */ @@ -3032,14 +3150,16 @@ static void enic_remove(struct pci_dev *pdev) cancel_work_sync(&enic->reset); cancel_work_sync(&enic->change_mtu_work); unregister_netdev(netdev); - enic_dev_deinit(enic); - vnic_dev_close(enic->vdev); #ifdef CONFIG_PCI_IOV if (enic_sriov_enabled(enic)) { - pci_disable_sriov(pdev); - enic->priv_flags &=3D ~ENIC_SRIOV_ENABLED; + if (enic->vf_type =3D=3D ENIC_VF_TYPE_V2) + enic_sriov_v2_disable(enic); + else + pci_disable_sriov(pdev); } #endif + enic_dev_deinit(enic); + vnic_dev_close(enic->vdev); kfree(enic->pp); vnic_dev_unregister(enic->vdev); enic_iounmap(enic); diff --git a/drivers/net/ethernet/cisco/enic/enic_res.c b/drivers/net/ether= net/cisco/enic/enic_res.c index 2b7545d6a67f..436326ace049 100644 --- a/drivers/net/ethernet/cisco/enic/enic_res.c +++ b/drivers/net/ethernet/cisco/enic/enic_res.c @@ -59,6 +59,7 @@ int enic_get_vnic_config(struct enic *enic) GET_CONFIG(intr_timer_usec); GET_CONFIG(loop_tag); GET_CONFIG(num_arfs); + GET_CONFIG(mq_subvnic_count); GET_CONFIG(max_rq_ring); GET_CONFIG(max_wq_ring); GET_CONFIG(max_cq_ring); diff --git a/drivers/net/ethernet/cisco/enic/vnic_enet.h b/drivers/net/ethe= rnet/cisco/enic/vnic_enet.h index 9e8e86262a3f..519d2969990b 100644 --- a/drivers/net/ethernet/cisco/enic/vnic_enet.h +++ b/drivers/net/ethernet/cisco/enic/vnic_enet.h @@ -21,7 +21,9 @@ struct vnic_enet_config { u16 loop_tag; u16 vf_rq_count; u16 num_arfs; - u8 reserved[66]; + u8 reserved1[32]; + u16 mq_subvnic_count; + u8 reserved2[32]; u32 max_rq_ring; // MAX RQ ring size u32 max_wq_ring; // MAX WQ ring size u32 max_cq_ring; // MAX CQ ring size --=20 2.43.0 From nobody Sun Jun 14 23:04:29 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id DDC081EA7CB; Tue, 7 Apr 2026 00:37:07 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1775522227; cv=none; b=ZCWyk0zCMPY6EXUr5NuxKaItUe613zJOfKWYG4phk6zfMRPghh1pP+buBFPddQ8ahR2aqSC2yKVnNB5AP06Jw/Grnt0RKUgh8msC6IUAJEFJ9tWESMcVo/PbDbXdoHXyNeYRMctemwwldS7NVxNMbu6leqvoQbatz8PzLI34tZ0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1775522227; c=relaxed/simple; bh=SM+RaBVh99fSkYLMMUl7uYCf15QIGheRSH5dXb45XeE=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=GRAqS5GZi/6GMd4jya1uxMm5ls9vNn7vaD/cKj1fcr6Ya27/96qR0Zh5WFonDaFUo2NWdb8RN68qmQofv5q9c9cgd4irrZI8nc3QbqzwLpBN6g94AaVflHnNb+CthriVj5soux7YC/iuJF/4Op39DpvlnvaVyNfJMMk7dbkZLpI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=Rrj+tNos; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="Rrj+tNos" Received: by smtp.kernel.org (Postfix) with ESMTPS id BD895C2BCB8; Tue, 7 Apr 2026 00:37:07 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1775522227; bh=SM+RaBVh99fSkYLMMUl7uYCf15QIGheRSH5dXb45XeE=; h=From:Date:Subject:References:In-Reply-To:To:Cc:Reply-To:From; b=Rrj+tNos8IFNTvI5usUERek3d9KwqLBOvALsva5YzesCIG3jebGhSMmS2BB4jQK6Z B/cAoq8Tu8Lx+AptGcq/dwk8Dw3IeEeRfXZmKx4cvabF6JBDjEV5xa2vYip/z2wn8Y 8FUsQdnV/e+CmudjFztxPDbN1k0TD2sAVJurKsU0qyVZ92kppwhF3yyyooa6kjlVHy fmDjRjFp0/5iwu86lMNGi9AiJ7Tmp84Ww/aMX6q1CRyS/yNZ73hueATSIj3Si1Tf1L AWVcg7pAlBKGWvscIUnKxtsk8W3jBKWj0Q0UW7KSHrHe76VsVof1LuCc2D/kJDNQEi L8iKUjxmQTAtg== Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id B4C56FB5175; Tue, 7 Apr 2026 00:37:07 +0000 (UTC) From: Satish Kharat via B4 Relay Date: Mon, 06 Apr 2026 17:37:08 -0700 Subject: [PATCH 10/10] enic: add V2 VF probe with admin channel and PF registration Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260406-enic-sriov-v2-admin-channel-v2-v1-10-82cc47636a78@cisco.com> References: <20260406-enic-sriov-v2-admin-channel-v2-v1-0-82cc47636a78@cisco.com> In-Reply-To: <20260406-enic-sriov-v2-admin-channel-v2-v1-0-82cc47636a78@cisco.com> To: Andrew Lunn , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni Cc: netdev@vger.kernel.org, linux-kernel@vger.kernel.org, 20260401-enic-sriov-v2-prep-v4-0-d5834b2ef1b9@cisco.com, Satish Kharat X-Mailer: b4 0.13.0 X-Developer-Signature: v=1; a=ed25519-sha256; t=1775522225; l=5643; i=satishkh@cisco.com; h=from:subject:message-id; bh=dGwC+ltN2vIFndaVGoadwoJkfe40tD8MwvsVFbzhE8w=; b=gR56pX3zYq1V0tgUKHaQdrf7ldLB371/wP4Q4gT7qLOUAswfRYwZJEz0o6K4PkWaiteOed/gx d8ZN66MrIulAwcsY2N7q2ktJTKVwyczHJuoU/JPeppLkM08FAF+mK1y X-Developer-Key: i=satishkh@cisco.com; a=ed25519; pk=4BxbLoCGvHeIMH/PbpiNVx8/2IcTjFvqcr0AD8K6WLY= X-Endpoint-Received: by B4 Relay for satishkh@cisco.com/default with auth_id=639 X-Original-From: Satish Kharat Reply-To: satishkh@cisco.com From: Satish Kharat When a V2 SR-IOV VF probes, open the admin channel, initialize the MBOX protocol, perform the capability check with the PF, and register with the PF. This establishes the PF-VF communication path that the PF uses to send link state notifications. The admin channel and MBOX registration happen after enic_dev_init() (which discovers admin channel resources) and before register_netdev() so the VF is fully initialized before the interface is visible to userspace. On remove, the VF unregisters from the PF and closes its admin channel before tearing down data path resources. V2 VFs are not provisioned with an RES_TYPE_SRIOV_INTR resource by firmware, so bypass that check in the admin channel capability detection for V2 VFs. The PF still requires this resource. Signed-off-by: Satish Kharat --- drivers/net/ethernet/cisco/enic/enic.h | 1 + drivers/net/ethernet/cisco/enic/enic_main.c | 58 +++++++++++++++++++++++++= +++- drivers/net/ethernet/cisco/enic/enic_res.c | 3 +- 3 files changed, 59 insertions(+), 3 deletions(-) diff --git a/drivers/net/ethernet/cisco/enic/enic.h b/drivers/net/ethernet/= cisco/enic/enic.h index 5ef4ed86174f..a8c622057b17 100644 --- a/drivers/net/ethernet/cisco/enic/enic.h +++ b/drivers/net/ethernet/cisco/enic/enic.h @@ -439,6 +439,7 @@ void enic_reset_addr_lists(struct enic *enic); int enic_sriov_enabled(struct enic *enic); int enic_is_valid_vf(struct enic *enic, int vf); int enic_is_dynamic(struct enic *enic); +int enic_is_sriov_vf_v2(struct enic *enic); void enic_set_ethtool_ops(struct net_device *netdev); int __enic_set_rsskey(struct enic *enic); void enic_ext_cq(struct enic *enic); diff --git a/drivers/net/ethernet/cisco/enic/enic_main.c b/drivers/net/ethe= rnet/cisco/enic/enic_main.c index 4328e2a79619..cdfbac474986 100644 --- a/drivers/net/ethernet/cisco/enic/enic_main.c +++ b/drivers/net/ethernet/cisco/enic/enic_main.c @@ -316,6 +316,11 @@ static int enic_is_sriov_vf(struct enic *enic) enic->pdev->device =3D=3D PCI_DEVICE_ID_CISCO_VIC_ENET_VF_V2; } =20 +int enic_is_sriov_vf_v2(struct enic *enic) +{ + return enic->pdev->device =3D=3D PCI_DEVICE_ID_CISCO_VIC_ENET_VF_V2; +} + int enic_is_valid_vf(struct enic *enic, int vf) { #ifdef CONFIG_PCI_IOV @@ -2986,6 +2991,32 @@ static int enic_probe(struct pci_dev *pdev, const st= ruct pci_device_id *ent) goto err_out_dev_close; } =20 + /* V2 VF: open admin channel and register with PF. + * Must happen before register_netdev so the VF is fully + * initialized before the interface is visible to userspace. + */ + if (enic_is_sriov_vf_v2(enic)) { + err =3D enic_admin_channel_open(enic); + if (err) { + dev_err(dev, + "Failed to open admin channel: %d\n", err); + goto err_out_dev_deinit; + } + enic_mbox_init(enic); + err =3D enic_mbox_vf_capability_check(enic); + if (err) { + dev_err(dev, + "MBOX capability check failed: %d\n", err); + goto err_out_admin_close; + } + err =3D enic_mbox_vf_register(enic); + if (err) { + dev_err(dev, + "MBOX VF registration failed: %d\n", err); + goto err_out_admin_close; + } + } + netif_set_real_num_tx_queues(netdev, enic->wq_count); netif_set_real_num_rx_queues(netdev, enic->rq_count); =20 @@ -3010,7 +3041,7 @@ static int enic_probe(struct pci_dev *pdev, const str= uct pci_device_id *ent) err =3D enic_set_mac_addr(netdev, enic->mac_addr); if (err) { dev_err(dev, "Invalid MAC address, aborting\n"); - goto err_out_dev_deinit; + goto err_out_admin_close; } =20 enic->tx_coalesce_usecs =3D enic->config.intr_timer_usec; @@ -3108,11 +3139,23 @@ static int enic_probe(struct pci_dev *pdev, const s= truct pci_device_id *ent) err =3D register_netdev(netdev); if (err) { dev_err(dev, "Cannot register net device, aborting\n"); - goto err_out_dev_deinit; + goto err_out_admin_close; } =20 return 0; =20 +err_out_admin_close: + if (enic_is_sriov_vf_v2(enic)) { + if (enic->vf_registered) { + int unreg_err =3D enic_mbox_vf_unregister(enic); + + if (unreg_err) + netdev_warn(netdev, + "Failed to unregister from PF: %d\n", + unreg_err); + } + enic_admin_channel_close(enic); + } err_out_dev_deinit: enic_dev_deinit(enic); err_out_dev_close: @@ -3150,6 +3193,17 @@ static void enic_remove(struct pci_dev *pdev) cancel_work_sync(&enic->reset); cancel_work_sync(&enic->change_mtu_work); unregister_netdev(netdev); + if (enic_is_sriov_vf_v2(enic)) { + if (enic->vf_registered) { + int unreg_err =3D enic_mbox_vf_unregister(enic); + + if (unreg_err) + netdev_warn(netdev, + "Failed to unregister from PF: %d\n", + unreg_err); + } + enic_admin_channel_close(enic); + } #ifdef CONFIG_PCI_IOV if (enic_sriov_enabled(enic)) { if (enic->vf_type =3D=3D ENIC_VF_TYPE_V2) diff --git a/drivers/net/ethernet/cisco/enic/enic_res.c b/drivers/net/ether= net/cisco/enic/enic_res.c index 436326ace049..74cd2ee3af5c 100644 --- a/drivers/net/ethernet/cisco/enic/enic_res.c +++ b/drivers/net/ethernet/cisco/enic/enic_res.c @@ -211,7 +211,8 @@ void enic_get_res_counts(struct enic *enic) vnic_dev_get_res_count(enic->vdev, RES_TYPE_ADMIN_RQ) >=3D 1 && vnic_dev_get_res_count(enic->vdev, RES_TYPE_ADMIN_CQ) >=3D ARRAY_SIZE(enic->admin_cq) && - vnic_dev_get_res_count(enic->vdev, RES_TYPE_SRIOV_INTR) >=3D 1; + (enic_is_sriov_vf_v2(enic) || + vnic_dev_get_res_count(enic->vdev, RES_TYPE_SRIOV_INTR) >=3D 1); =20 dev_info(enic_get_dev(enic), "vNIC resources avail: wq %d rq %d cq %d intr %d admin %s\n", --=20 2.43.0