From nobody Sun Feb 8 18:49:34 2026 Received: from mail-qk1-f227.google.com (mail-qk1-f227.google.com [209.85.222.227]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9E508236453 for ; Mon, 26 Jan 2026 05:44:13 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.222.227 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1769406255; cv=none; b=qlVjhsZ87OXARVkDCmE69wA3leNdMMzRsrrQmjzKKmkj+FPAjsc71udPyeBarx7EnNaHhg5qY0tQ8Sg6/gyGVDrf+z9dmRKRf+JtGJ7kEBo8XkBI8ml1aGUbp4YPTzo0BL5CcmG+xpaMl9f4fGsZuIoCBZ/hf9HDKDn+ZSQlB4Q= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1769406255; c=relaxed/simple; bh=AlDSmdLtThNTvMg5cKevGz/HN6FyPnz9oRw5Fwtd3zI=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=q7ZlIZMwWcCFOBgBRX5DXWw/ZOKBaoaefHV01wRIUldFun5X9NFsykHHuJLeJnP5oyJtwPwYrqf4gL6LxhPVG2sJTxo/OwUxn3wQhRNb3V40G3tANrrftgzJhvObl3ayouIF0FpzYzeFX8zDbltpdeWOlvPvNX+cxrwBnGdoxAc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=broadcom.com; spf=fail smtp.mailfrom=broadcom.com; dkim=pass (1024-bit key) header.d=broadcom.com header.i=@broadcom.com header.b=OjIHYFtN; arc=none smtp.client-ip=209.85.222.227 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=broadcom.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=broadcom.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=broadcom.com header.i=@broadcom.com header.b="OjIHYFtN" Received: by mail-qk1-f227.google.com with SMTP id af79cd13be357-8c538d17816so581000585a.0 for ; Sun, 25 Jan 2026 21:44:13 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1769406252; x=1770011052; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:dkim-signature:x-gm-gg :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=5Vo8GbKpsK7bLtMuDfhqrExL3Ad6QrF51zG2vtYVAaA=; b=tL7+Pl976zColKRg806JqhSHc1/GOJhih8jbq1BFoXGxklEirkTcI5zmYbjEoYXXot uLVfu+L1bB3PWhGcCoP9g9agi2lpHza7oTCHSfemNSkd/t4EzaTeJQdiwqkTCCxVNd3w 0s8IXepZFGvcLzQqaejBRydsQOA5+jXL5a+mapcnQo3WpiHUAepf9atYxSCSjK4A/R8z cNRF5cH9Wcp1KLnoyW0m/E9RzId8ffbbP4BpxKOXgcZShfT0h8GVzjy3BRhKofEfIYIP m47MpoGi446olLwxSaoGrY6Lyg21EEFqQH72c301P34kuneKMRwi+adjrds90AUz3miD vIbQ== X-Gm-Message-State: AOJu0YycaPc2hhJIJDJfMuL1qTFZTUK8FYgWh/mHLCivwR6d2fMPEmoJ vef91AMwx6igeKfQVjIE3kzNA3Krft/oJasqy5uUiMBuOQ3kR8T9YCgOiwraUSS+4Rt97bML6VB TQQy5v++gWMFK76RQUmjrujBZGd4ShJ4xmxBPSFm/XIMioXtjqH+jqVJPvfRvznL0NOV4LBvftu 9ZIUYzPqfdY2RiDb0MmZkgAI5qkez8QDV+OkeMf8/yVylxwVx8MKFYPZ06MiBpEBwZkUKrHrMwT uvGp9CJgCojoFTKhZs= X-Gm-Gg: AZuq6aLqXn4BWuJW9D0Ei+Xk5CSWwKkg/X/fCYEsNxEf0UEgfuIBBF6/mrgmnbn2mM+ 253MdSEwDHhkwLgZrK0QIGUOB1WIZk8oqQyINo7rLHKBopOTCs4gP5EVE2yUOPmj26DU1W0V3Dz 8hlU56Qo1ZsBb2DnemdEIWZtnfK7XcfeJ76VRtep8jjOO1jpjksz0JS1bqkZjtzlr2C/zgY/1Lg s42g4SqoJD79getiVS2OS5Y67ZARyF0XpbSvqOKeJDCX+DWGZS551BQpRMsAhaVYCoR6fPgyaJl BjxrzSkuJpFi/ZUpQmOoFGY9oXEYmAtL/XqPMYNSu6xVcwycD/ZbVfyJ8+1s/88teRjObd6pffE HXi2k0iCIQJl5N5UKlUAw+Nk1StMLCnaHvGRMkQPchglmjjamJ4ouWLy3uJ8IzT9pvm7wN4mppG U0ErBsmHMmmPkfo+mZtqeM1vcYuK0+rF0ZcT9+r5UVVr2LKKY= X-Received: by 2002:a05:620a:1712:b0:8c6:e11c:5ec4 with SMTP id af79cd13be357-8c6f95c72a2mr358324385a.35.1769406252367; Sun, 25 Jan 2026 21:44:12 -0800 (PST) Received: from smtp-us-east1-p01-i01-si01.dlp.protect.broadcom.com (address-144-49-247-102.dlp.protect.broadcom.com. [144.49.247.102]) by smtp-relay.gmail.com with ESMTPS id 6a1803df08f44-8949188f1d0sm12894046d6.27.2026.01.25.21.44.12 for (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Sun, 25 Jan 2026 21:44:12 -0800 (PST) X-Relaying-Domain: broadcom.com X-CFilter-Loop: Reflected Received: by mail-pg1-f198.google.com with SMTP id 41be03b00d2f7-c5454bf50e0so7425962a12.2 for ; Sun, 25 Jan 2026 21:44:11 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=broadcom.com; s=google; t=1769406251; x=1770011051; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=5Vo8GbKpsK7bLtMuDfhqrExL3Ad6QrF51zG2vtYVAaA=; b=OjIHYFtNflphdFge4Ru2awHVfJ8NARs7cfUTdElvp5GO2aGburSZwJFV/zxw4s9uT9 iE1bWXyzysQpgsT9iSa9vrcFQG0uAF4nEAj3Zo8ypyLaqnlNfEI9EV84G+H00x08O8rv kB2XMS2dDDHHk4bshVkkC9N/XmAcZaSIgx+r4= X-Received: by 2002:a05:6a20:c91c:b0:38d:f988:613f with SMTP id adf61e73a8af0-38e9f284ec0mr2700078637.80.1769406250637; Sun, 25 Jan 2026 21:44:10 -0800 (PST) X-Received: by 2002:a05:6a20:c91c:b0:38d:f988:613f with SMTP id adf61e73a8af0-38e9f284ec0mr2700060637.80.1769406250122; Sun, 25 Jan 2026 21:44:10 -0800 (PST) Received: from PC-MID-R740.dhcp.broadcom.net ([192.19.234.250]) by smtp.gmail.com with ESMTPSA id 41be03b00d2f7-c635a3f2ddcsm7427213a12.17.2026.01.25.21.44.06 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 25 Jan 2026 21:44:09 -0800 (PST) From: Pavan Chebbi To: jgg@ziepe.ca, michael.chan@broadcom.com Cc: linux-kernel@vger.kernel.org, dave.jiang@intel.com, saeedm@nvidia.com, Jonathan.Cameron@huawei.com, gospo@broadcom.com, selvin.xavier@broadcom.com, leon@kernel.org, kalesh-anakkur.purayil@broadcom.com, Pavan Chebbi Subject: [PATCH v2 fwctl 2/5] fwctl/bnxt_en: Refactor aux bus functions to be more generic Date: Sun, 25 Jan 2026 21:37:07 -0800 Message-Id: <20260126053710.3474483-3-pavan.chebbi@broadcom.com> X-Mailer: git-send-email 2.39.1 In-Reply-To: <20260126053710.3474483-1-pavan.chebbi@broadcom.com> References: <20260126053710.3474483-1-pavan.chebbi@broadcom.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-DetectorID-Processed: b00c1d49-9d2e-4205-b15f-d015386d3d5e Content-Type: text/plain; charset="utf-8" Up until now there was only one auxiliary device that bnxt created and that was for RoCE driver. bnxt fwctl is also going to use an aux bus device that bnxt should create. This requires some nomenclature changes and refactoring of the existing bnxt aux dev functions. Convert 'aux_priv' and 'edev' members of struct bnxt into arrays where each element contains supported auxbus device's data. Move struct bnxt_aux_priv from bnxt.h to ulp.h because that is where it belongs. Make aux bus init/uninit/add/del functions more generic which will loop through all the aux device types. Make bnxt_ulp_start/stop functions (the only other common functions applicable to any aux device) loop through the aux devices to update their config and states. Make callers of bnxt_ulp_start() call it only when there are no errors. Also, as an improvement in code, bnxt_register_dev() can skip unnecessary dereferencing of edev from bp, instead use the edev pointer from the function parameter. Future patches will reuse these functions to add an aux bus device for fwctl. Reviewed-by: Andy Gospodarek Signed-off-by: Pavan Chebbi --- drivers/net/ethernet/broadcom/bnxt/bnxt.c | 47 ++- drivers/net/ethernet/broadcom/bnxt/bnxt.h | 19 +- .../net/ethernet/broadcom/bnxt/bnxt_devlink.c | 8 +- .../net/ethernet/broadcom/bnxt/bnxt_ethtool.c | 2 +- drivers/net/ethernet/broadcom/bnxt/bnxt_ulp.c | 334 +++++++++++------- include/linux/bnxt/ulp.h | 25 +- 6 files changed, 271 insertions(+), 164 deletions(-) diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt.c b/drivers/net/ethern= et/broadcom/bnxt/bnxt.c index 4481d80cdfc2..a9bfbfabf121 100644 --- a/drivers/net/ethernet/broadcom/bnxt/bnxt.c +++ b/drivers/net/ethernet/broadcom/bnxt/bnxt.c @@ -6859,7 +6859,8 @@ int bnxt_hwrm_vnic_cfg(struct bnxt *bp, struct bnxt_v= nic_info *vnic) #endif if ((bp->flags & BNXT_FLAG_STRIP_VLAN) || def_vlan) req->flags |=3D cpu_to_le32(VNIC_CFG_REQ_FLAGS_VLAN_STRIP_MODE); - if (vnic->vnic_id =3D=3D BNXT_VNIC_DEFAULT && bnxt_ulp_registered(bp->ede= v)) + if (vnic->vnic_id =3D=3D BNXT_VNIC_DEFAULT && + bnxt_ulp_registered(bp->edev[BNXT_AUXDEV_RDMA])) req->flags |=3D cpu_to_le32(bnxt_get_roce_vnic_mode(bp)); =20 return hwrm_req_send(bp, req); @@ -7974,6 +7975,7 @@ static int bnxt_get_avail_msix(struct bnxt *bp, int n= um); =20 static int __bnxt_reserve_rings(struct bnxt *bp) { + struct bnxt_en_dev *edev =3D bp->edev[BNXT_AUXDEV_RDMA]; struct bnxt_hw_rings hwr =3D {0}; int rx_rings, old_rx_rings, rc; int cp =3D bp->cp_nr_rings; @@ -7984,7 +7986,7 @@ static int __bnxt_reserve_rings(struct bnxt *bp) if (!bnxt_need_reserve_rings(bp)) return 0; =20 - if (BNXT_NEW_RM(bp) && !bnxt_ulp_registered(bp->edev)) { + if (BNXT_NEW_RM(bp) && !bnxt_ulp_registered(edev)) { ulp_msix =3D bnxt_get_avail_msix(bp, bp->ulp_num_msix_want); if (!ulp_msix) bnxt_set_ulp_stat_ctxs(bp, 0); @@ -8035,8 +8037,7 @@ static int __bnxt_reserve_rings(struct bnxt *bp) } rx_rings =3D min_t(int, rx_rings, hwr.grp); hwr.cp =3D min_t(int, hwr.cp, bp->cp_nr_rings); - if (bnxt_ulp_registered(bp->edev) && - hwr.stat > bnxt_get_ulp_stat_ctxs(bp)) + if (bnxt_ulp_registered(edev) && hwr.stat > bnxt_get_ulp_stat_ctxs(bp)) hwr.stat -=3D bnxt_get_ulp_stat_ctxs(bp); hwr.cp =3D min_t(int, hwr.cp, hwr.stat); rc =3D bnxt_trim_rings(bp, &rx_rings, &hwr.tx, hwr.cp, sh); @@ -8075,7 +8076,7 @@ static int __bnxt_reserve_rings(struct bnxt *bp) !netif_is_rxfh_configured(bp->dev)) bnxt_set_dflt_rss_indir_tbl(bp, NULL); =20 - if (!bnxt_ulp_registered(bp->edev) && BNXT_NEW_RM(bp)) { + if (!bnxt_ulp_registered(edev) && BNXT_NEW_RM(bp)) { int resv_msix, resv_ctx, ulp_ctxs; struct bnxt_hw_resc *hw_resc; =20 @@ -11430,6 +11431,7 @@ static void bnxt_clear_int_mode(struct bnxt *bp) =20 int bnxt_reserve_rings(struct bnxt *bp, bool irq_re_init) { + struct bnxt_en_dev *edev =3D bp->edev[BNXT_AUXDEV_RDMA]; bool irq_cleared =3D false; bool irq_change =3D false; int tcs =3D bp->num_tc; @@ -11439,7 +11441,7 @@ int bnxt_reserve_rings(struct bnxt *bp, bool irq_re= _init) if (!bnxt_need_reserve_rings(bp)) return 0; =20 - if (BNXT_NEW_RM(bp) && !bnxt_ulp_registered(bp->edev)) { + if (BNXT_NEW_RM(bp) && !bnxt_ulp_registered(edev)) { int ulp_msix =3D bnxt_get_avail_msix(bp, bp->ulp_num_msix_want); =20 if (ulp_msix > bp->ulp_num_msix_want) @@ -14522,7 +14524,7 @@ static void bnxt_fw_echo_reply(struct bnxt *bp) static void bnxt_ulp_restart(struct bnxt *bp) { bnxt_ulp_stop(bp); - bnxt_ulp_start(bp, 0); + bnxt_ulp_start(bp); } =20 static void bnxt_sp_task(struct work_struct *work) @@ -14679,7 +14681,7 @@ int bnxt_check_rings(struct bnxt *bp, int tx, int r= x, bool sh, int tcs, hwr.cp_p5 =3D hwr.tx + rx; rc =3D bnxt_hwrm_check_rings(bp, &hwr); if (!rc && pci_msix_can_alloc_dyn(bp->pdev)) { - if (!bnxt_ulp_registered(bp->edev)) { + if (!bnxt_ulp_registered(bp->edev[BNXT_AUXDEV_RDMA])) { hwr.cp +=3D bnxt_get_ulp_msix_num(bp); hwr.cp =3D min_t(int, hwr.cp, bnxt_get_max_func_irqs(bp)); } @@ -15199,7 +15201,7 @@ static void bnxt_fw_reset_task(struct work_struct *= work) bnxt_dl_health_fw_status_update(bp, true); } netdev_unlock(bp->dev); - bnxt_ulp_start(bp, 0); + bnxt_ulp_start(bp); bnxt_reenable_sriov(bp); netdev_lock(bp->dev); bnxt_vf_reps_alloc(bp); @@ -15221,7 +15223,8 @@ static void bnxt_fw_reset_task(struct work_struct *= work) bnxt_fw_reset_abort(bp, rc); netdev_unlock(bp->dev); ulp_start: - bnxt_ulp_start(bp, rc); + if (!rc) + bnxt_ulp_start(bp); } =20 static int bnxt_init_board(struct pci_dev *pdev, struct net_device *dev) @@ -16211,12 +16214,13 @@ static void bnxt_remove_one(struct pci_dev *pdev) if (BNXT_PF(bp)) __bnxt_sriov_disable(bp); =20 - bnxt_rdma_aux_device_del(bp); + bnxt_aux_devices_del(bp); =20 unregister_netdev(dev); bnxt_ptp_clear(bp); =20 - bnxt_rdma_aux_device_uninit(bp); + bnxt_aux_devices_uninit(bp); + bnxt_auxdev_id_free(bp, bp->auxdev_id); =20 bnxt_free_l2_filters(bp, true); bnxt_free_ntp_fltrs(bp, true); @@ -16802,7 +16806,9 @@ static int bnxt_init_one(struct pci_dev *pdev, cons= t struct pci_device_id *ent) bnxt_set_tpa_flags(bp); bnxt_init_ring_params(bp); bnxt_set_ring_params(bp); - bnxt_rdma_aux_device_init(bp); + mutex_init(&bp->auxdev_lock); + if (!bnxt_auxdev_id_alloc(bp)) + bnxt_aux_devices_init(bp); rc =3D bnxt_set_dflt_rings(bp, true); if (rc) { if (BNXT_VF(bp) && rc =3D=3D -ENODEV) { @@ -16866,7 +16872,7 @@ static int bnxt_init_one(struct pci_dev *pdev, cons= t struct pci_device_id *ent) =20 bnxt_dl_fw_reporters_create(bp); =20 - bnxt_rdma_aux_device_add(bp); + bnxt_aux_devices_add(bp); =20 bnxt_print_device_info(bp); =20 @@ -16874,7 +16880,8 @@ static int bnxt_init_one(struct pci_dev *pdev, cons= t struct pci_device_id *ent) =20 return 0; init_err_cleanup: - bnxt_rdma_aux_device_uninit(bp); + bnxt_aux_devices_uninit(bp); + bnxt_auxdev_id_free(bp, bp->auxdev_id); bnxt_dl_unregister(bp); init_err_dl: bnxt_shutdown_tc(bp); @@ -17008,9 +17015,10 @@ static int bnxt_resume(struct device *device) =20 resume_exit: netdev_unlock(bp->dev); - bnxt_ulp_start(bp, rc); - if (!rc) + if (!rc) { + bnxt_ulp_start(bp); bnxt_reenable_sriov(bp); + } return rc; } =20 @@ -17190,9 +17198,10 @@ static void bnxt_io_resume(struct pci_dev *pdev) netif_device_attach(netdev); =20 netdev_unlock(netdev); - bnxt_ulp_start(bp, err); - if (!err) + if (!err) { + bnxt_ulp_start(bp); bnxt_reenable_sriov(bp); + } } =20 static const struct pci_error_handlers bnxt_err_handler =3D { diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt.h b/drivers/net/ethern= et/broadcom/bnxt/bnxt.h index f5f07a7e6b29..a739e7d52bdc 100644 --- a/drivers/net/ethernet/broadcom/bnxt/bnxt.h +++ b/drivers/net/ethernet/broadcom/bnxt/bnxt.h @@ -24,12 +24,12 @@ #include #include #include -#include #include #include #include #include #include +#include #ifdef CONFIG_TEE_BNXT_FW #include #endif @@ -2076,12 +2076,6 @@ struct bnxt_fw_health { #define BNXT_FW_IF_RETRY 10 #define BNXT_FW_SLOT_RESET_RETRY 4 =20 -struct bnxt_aux_priv { - struct auxiliary_device aux_dev; - struct bnxt_en_dev *edev; - int id; -}; - enum board_idx { BCM57301, BCM57302, @@ -2341,8 +2335,8 @@ struct bnxt { #define BNXT_CHIP_P5_AND_MINUS(bp) \ (BNXT_CHIP_P3(bp) || BNXT_CHIP_P4(bp) || BNXT_CHIP_P5(bp)) =20 - struct bnxt_aux_priv *aux_priv; - struct bnxt_en_dev *edev; + struct bnxt_aux_priv *aux_priv[__BNXT_AUXDEV_MAX]; + struct bnxt_en_dev *edev[__BNXT_AUXDEV_MAX]; =20 struct bnxt_napi **bnapi; =20 @@ -2751,6 +2745,13 @@ struct bnxt { struct bnxt_ctx_pg_info *fw_crash_mem; u32 fw_crash_len; struct bnxt_bs_trace_info bs_trace[BNXT_TRACE_MAX]; + int auxdev_id; + /* synchronize validity checks of available aux devices */ + struct mutex auxdev_lock; + u8 auxdev_state[__BNXT_AUXDEV_MAX]; +#define BNXT_ADEV_STATE_NONE 0 +#define BNXT_ADEV_STATE_INIT 1 +#define BNXT_ADEV_STATE_ADD 2 }; =20 #define BNXT_NUM_RX_RING_STATS 8 diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt_devlink.c b/drivers/ne= t/ethernet/broadcom/bnxt/bnxt_devlink.c index 230cd95d30a2..835f2b413931 100644 --- a/drivers/net/ethernet/broadcom/bnxt/bnxt_devlink.c +++ b/drivers/net/ethernet/broadcom/bnxt/bnxt_devlink.c @@ -440,13 +440,13 @@ static int bnxt_dl_reload_down(struct devlink *dl, bo= ol netns_change, "reload is unsupported while VFs are allocated or being configured= "); netdev_unlock(bp->dev); rtnl_unlock(); - bnxt_ulp_start(bp, 0); + bnxt_ulp_start(bp); return -EOPNOTSUPP; } if (bp->dev->reg_state =3D=3D NETREG_UNREGISTERED) { netdev_unlock(bp->dev); rtnl_unlock(); - bnxt_ulp_start(bp, 0); + bnxt_ulp_start(bp); return -ENODEV; } if (netif_running(bp->dev)) @@ -578,8 +578,8 @@ static int bnxt_dl_reload_up(struct devlink *dl, enum d= evlink_reload_action acti } netdev_unlock(bp->dev); rtnl_unlock(); - if (action =3D=3D DEVLINK_RELOAD_ACTION_DRIVER_REINIT) - bnxt_ulp_start(bp, rc); + if (!rc && action =3D=3D DEVLINK_RELOAD_ACTION_DRIVER_REINIT) + bnxt_ulp_start(bp); return rc; } =20 diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt_ethtool.c b/drivers/ne= t/ethernet/broadcom/bnxt/bnxt_ethtool.c index 8cad7b982664..064d7bc4ce8d 100644 --- a/drivers/net/ethernet/broadcom/bnxt/bnxt_ethtool.c +++ b/drivers/net/ethernet/broadcom/bnxt/bnxt_ethtool.c @@ -5100,7 +5100,7 @@ static void bnxt_self_test(struct net_device *dev, st= ruct ethtool_test *etest, =20 memset(buf, 0, sizeof(u64) * bp->num_tests); if (etest->flags & ETH_TEST_FL_OFFLINE && - bnxt_ulp_registered(bp->edev)) { + bnxt_ulp_registered(bp->edev[BNXT_AUXDEV_RDMA])) { etest->flags |=3D ETH_TEST_FL_FAILED; netdev_warn(dev, "Offline tests cannot be run with RoCE driver loaded\n"= ); return; diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt_ulp.c b/drivers/net/et= hernet/broadcom/bnxt/bnxt_ulp.c index fa513892db50..4cdfc9641639 100644 --- a/drivers/net/ethernet/broadcom/bnxt/bnxt_ulp.c +++ b/drivers/net/ethernet/broadcom/bnxt/bnxt_ulp.c @@ -29,9 +29,44 @@ =20 static DEFINE_IDA(bnxt_aux_dev_ids); =20 +struct bnxt_aux_device { + const char *name; +}; + +static void bnxt_auxdev_set_state(struct bnxt *bp, int idx, int state) +{ + mutex_lock(&bp->auxdev_lock); + bp->auxdev_state[idx] =3D state; + mutex_unlock(&bp->auxdev_lock); +} + +static bool bnxt_auxdev_is_init(struct bnxt *bp, int idx) +{ + bool ret; + + mutex_lock(&bp->auxdev_lock); + ret =3D (bp->auxdev_state[idx] =3D=3D BNXT_ADEV_STATE_INIT); + mutex_unlock(&bp->auxdev_lock); + return ret; +} + +static bool bnxt_auxdev_is_active(struct bnxt *bp, int idx) +{ + bool ret; + + mutex_lock(&bp->auxdev_lock); + ret =3D (bp->auxdev_state[idx] =3D=3D BNXT_ADEV_STATE_ADD); + mutex_unlock(&bp->auxdev_lock); + return ret; +} + +static struct bnxt_aux_device bnxt_aux_devices[__BNXT_AUXDEV_MAX] =3D {{ + .name =3D "rdma", +}}; + static void bnxt_fill_msix_vecs(struct bnxt *bp, struct bnxt_msix_entry *e= nt) { - struct bnxt_en_dev *edev =3D bp->edev; + struct bnxt_en_dev *edev =3D bp->edev[BNXT_AUXDEV_RDMA]; int num_msix, i; =20 if (!edev->ulp_tbl->msix_requested) { @@ -51,61 +86,75 @@ static void bnxt_fill_msix_vecs(struct bnxt *bp, struct= bnxt_msix_entry *ent) =20 int bnxt_get_ulp_msix_num(struct bnxt *bp) { - if (bp->edev) - return bp->edev->ulp_num_msix_vec; + struct bnxt_en_dev *edev =3D bp->edev[BNXT_AUXDEV_RDMA]; + + if (edev) + return edev->ulp_num_msix_vec; return 0; } =20 void bnxt_set_ulp_msix_num(struct bnxt *bp, int num) { - if (bp->edev) - bp->edev->ulp_num_msix_vec =3D num; + struct bnxt_en_dev *edev =3D bp->edev[BNXT_AUXDEV_RDMA]; + + if (edev) + edev->ulp_num_msix_vec =3D num; } =20 int bnxt_get_ulp_msix_num_in_use(struct bnxt *bp) { - if (bnxt_ulp_registered(bp->edev)) - return bp->edev->ulp_num_msix_vec; + struct bnxt_en_dev *edev =3D bp->edev[BNXT_AUXDEV_RDMA]; + + if (bnxt_ulp_registered(edev)) + return edev->ulp_num_msix_vec; return 0; } =20 int bnxt_get_ulp_stat_ctxs(struct bnxt *bp) { - if (bp->edev) - return bp->edev->ulp_num_ctxs; + struct bnxt_en_dev *edev =3D bp->edev[BNXT_AUXDEV_RDMA]; + + if (edev) + return edev->ulp_num_ctxs; return 0; } =20 void bnxt_set_ulp_stat_ctxs(struct bnxt *bp, int num_ulp_ctx) { - if (bp->edev) - bp->edev->ulp_num_ctxs =3D num_ulp_ctx; + struct bnxt_en_dev *edev =3D bp->edev[BNXT_AUXDEV_RDMA]; + + if (edev) + edev->ulp_num_ctxs =3D num_ulp_ctx; } =20 int bnxt_get_ulp_stat_ctxs_in_use(struct bnxt *bp) { - if (bnxt_ulp_registered(bp->edev)) - return bp->edev->ulp_num_ctxs; + struct bnxt_en_dev *edev =3D bp->edev[BNXT_AUXDEV_RDMA]; + + if (bnxt_ulp_registered(edev)) + return edev->ulp_num_ctxs; return 0; } =20 void bnxt_set_dflt_ulp_stat_ctxs(struct bnxt *bp) { - if (bp->edev) { - bp->edev->ulp_num_ctxs =3D BNXT_MIN_ROCE_STAT_CTXS; + struct bnxt_en_dev *edev =3D bp->edev[BNXT_AUXDEV_RDMA]; + + if (edev) { + edev->ulp_num_ctxs =3D BNXT_MIN_ROCE_STAT_CTXS; /* Reserve one additional stat_ctx for PF0 (except * on 1-port NICs) as it also creates one stat_ctx * for PF1 in case of RoCE bonding. */ if (BNXT_PF(bp) && !bp->pf.port_id && bp->port_count > 1) - bp->edev->ulp_num_ctxs++; + edev->ulp_num_ctxs++; =20 /* Reserve one additional stat_ctx when the device is capable * of supporting port mirroring on RDMA device. */ if (BNXT_MIRROR_ON_ROCE_CAP(bp)) - bp->edev->ulp_num_ctxs++; + edev->ulp_num_ctxs++; } } =20 @@ -141,7 +190,7 @@ int bnxt_register_dev(struct bnxt_en_dev *edev, =20 edev->ulp_tbl->msix_requested =3D bnxt_get_ulp_msix_num(bp); =20 - bnxt_fill_msix_vecs(bp, bp->edev->msix_entries); + bnxt_fill_msix_vecs(bp, edev->msix_entries); exit: mutex_unlock(&edev->en_dev_lock); netdev_unlock(dev); @@ -227,20 +276,26 @@ EXPORT_SYMBOL(bnxt_send_msg); =20 void bnxt_ulp_stop(struct bnxt *bp) { - struct bnxt_aux_priv *aux_priv =3D bp->aux_priv; - struct bnxt_en_dev *edev =3D bp->edev; - - if (!edev) - return; + int i; =20 - mutex_lock(&edev->en_dev_lock); - if (!bnxt_ulp_registered(edev) || - (edev->flags & BNXT_EN_FLAG_ULP_STOPPED)) - goto ulp_stop_exit; - - edev->flags |=3D BNXT_EN_FLAG_ULP_STOPPED; - if (aux_priv) { + for (i =3D 0; i < __BNXT_AUXDEV_MAX; i++) { + struct bnxt_aux_priv *aux_priv; struct auxiliary_device *adev; + struct bnxt_en_dev *edev; + + if (!bnxt_auxdev_is_active(bp, i)) + continue; + + aux_priv =3D bp->aux_priv[i]; + edev =3D bp->edev[i]; + mutex_lock(&edev->en_dev_lock); + if (!bnxt_ulp_registered(edev) || + (edev->flags & BNXT_EN_FLAG_ULP_STOPPED)) { + mutex_unlock(&edev->en_dev_lock); + continue; + } + + edev->flags |=3D BNXT_EN_FLAG_ULP_STOPPED; =20 adev =3D &aux_priv->aux_dev; if (adev->dev.driver) { @@ -251,29 +306,33 @@ void bnxt_ulp_stop(struct bnxt *bp) edev->en_state =3D bp->state; adrv->suspend(adev, pm); } + mutex_unlock(&edev->en_dev_lock); } -ulp_stop_exit: - mutex_unlock(&edev->en_dev_lock); } =20 -void bnxt_ulp_start(struct bnxt *bp, int err) +void bnxt_ulp_start(struct bnxt *bp) { - struct bnxt_aux_priv *aux_priv =3D bp->aux_priv; - struct bnxt_en_dev *edev =3D bp->edev; + int i; =20 - if (!edev || err) - return; + for (i =3D 0; i < __BNXT_AUXDEV_MAX; i++) { + struct bnxt_aux_priv *aux_priv; + struct auxiliary_device *adev; + struct bnxt_en_dev *edev; =20 - mutex_lock(&edev->en_dev_lock); - if (!bnxt_ulp_registered(edev) || - !(edev->flags & BNXT_EN_FLAG_ULP_STOPPED)) - goto ulp_start_exit; + if (!bnxt_auxdev_is_active(bp, i)) + continue; =20 - if (edev->ulp_tbl->msix_requested) - bnxt_fill_msix_vecs(bp, edev->msix_entries); + aux_priv =3D bp->aux_priv[i]; + edev =3D bp->edev[i]; + mutex_lock(&edev->en_dev_lock); + if (!bnxt_ulp_registered(edev) || + !(edev->flags & BNXT_EN_FLAG_ULP_STOPPED)) { + goto clear_flag_continue; + } + + if (edev->ulp_tbl->msix_requested) + bnxt_fill_msix_vecs(bp, edev->msix_entries); =20 - if (aux_priv) { - struct auxiliary_device *adev; =20 adev =3D &aux_priv->aux_dev; if (adev->dev.driver) { @@ -283,22 +342,22 @@ void bnxt_ulp_start(struct bnxt *bp, int err) edev->en_state =3D bp->state; adrv->resume(adev); } +clear_flag_continue: + edev->flags &=3D ~BNXT_EN_FLAG_ULP_STOPPED; + mutex_unlock(&edev->en_dev_lock); } -ulp_start_exit: - edev->flags &=3D ~BNXT_EN_FLAG_ULP_STOPPED; - mutex_unlock(&edev->en_dev_lock); } =20 void bnxt_ulp_irq_stop(struct bnxt *bp) { - struct bnxt_en_dev *edev =3D bp->edev; + struct bnxt_en_dev *edev =3D bp->edev[BNXT_AUXDEV_RDMA]; struct bnxt_ulp_ops *ops; bool reset =3D false; =20 if (!edev) return; =20 - if (bnxt_ulp_registered(bp->edev)) { + if (bnxt_ulp_registered(edev)) { struct bnxt_ulp *ulp =3D edev->ulp_tbl; =20 if (!ulp->msix_requested) @@ -315,13 +374,13 @@ void bnxt_ulp_irq_stop(struct bnxt *bp) =20 void bnxt_ulp_irq_restart(struct bnxt *bp, int err) { - struct bnxt_en_dev *edev =3D bp->edev; + struct bnxt_en_dev *edev =3D bp->edev[BNXT_AUXDEV_RDMA]; struct bnxt_ulp_ops *ops; =20 if (!edev) return; =20 - if (bnxt_ulp_registered(bp->edev)) { + if (bnxt_ulp_registered(edev)) { struct bnxt_ulp *ulp =3D edev->ulp_tbl; struct bnxt_msix_entry *ent =3D NULL; =20 @@ -347,7 +406,7 @@ void bnxt_ulp_irq_restart(struct bnxt *bp, int err) void bnxt_ulp_async_events(struct bnxt *bp, struct hwrm_async_event_cmpl *= cmpl) { u16 event_id =3D le16_to_cpu(cmpl->event_id); - struct bnxt_en_dev *edev =3D bp->edev; + struct bnxt_en_dev *edev =3D bp->edev[BNXT_AUXDEV_RDMA]; struct bnxt_ulp_ops *ops; struct bnxt_ulp *ulp; =20 @@ -388,18 +447,19 @@ void bnxt_register_async_events(struct bnxt_en_dev *e= dev, } EXPORT_SYMBOL(bnxt_register_async_events); =20 -void bnxt_rdma_aux_device_uninit(struct bnxt *bp) +void bnxt_aux_devices_uninit(struct bnxt *bp) { struct bnxt_aux_priv *aux_priv; struct auxiliary_device *adev; + int idx; =20 - /* Skip if no auxiliary device init was done. */ - if (!bp->aux_priv) - return; - - aux_priv =3D bp->aux_priv; - adev =3D &aux_priv->aux_dev; - auxiliary_device_uninit(adev); + for (idx =3D 0; idx < __BNXT_AUXDEV_MAX; idx++) { + if (bnxt_auxdev_is_init(bp, idx)) { + aux_priv =3D bp->aux_priv[idx]; + adev =3D &aux_priv->aux_dev; + auxiliary_device_uninit(adev); + } + } } =20 static void bnxt_aux_dev_release(struct device *dev) @@ -408,20 +468,24 @@ static void bnxt_aux_dev_release(struct device *dev) container_of(dev, struct bnxt_aux_priv, aux_dev.dev); struct bnxt *bp =3D netdev_priv(aux_priv->edev->net); =20 - ida_free(&bnxt_aux_dev_ids, aux_priv->id); kfree(aux_priv->edev->ulp_tbl); - bp->edev =3D NULL; + bp->edev[aux_priv->id] =3D NULL; kfree(aux_priv->edev); kfree(aux_priv); - bp->aux_priv =3D NULL; + bp->aux_priv[aux_priv->id] =3D NULL; + bnxt_auxdev_set_state(bp, aux_priv->id, BNXT_ADEV_STATE_NONE); } =20 -void bnxt_rdma_aux_device_del(struct bnxt *bp) +void bnxt_aux_devices_del(struct bnxt *bp) { - if (!bp->edev) - return; + int idx; =20 - auxiliary_device_delete(&bp->aux_priv->aux_dev); + for (idx =3D 0; idx < __BNXT_AUXDEV_MAX; idx++) { + if (bnxt_auxdev_is_active(bp, idx)) { + auxiliary_device_delete(&bp->aux_priv[idx]->aux_dev); + bnxt_auxdev_set_state(bp, idx, BNXT_ADEV_STATE_INIT); + } + } } =20 static void bnxt_set_edev_info(struct bnxt_en_dev *edev, struct bnxt *bp) @@ -451,83 +515,101 @@ static void bnxt_set_edev_info(struct bnxt_en_dev *e= dev, struct bnxt *bp) edev->bar0 =3D bp->bar0; } =20 -void bnxt_rdma_aux_device_add(struct bnxt *bp) +void bnxt_aux_devices_add(struct bnxt *bp) { struct auxiliary_device *aux_dev; - int rc; - - if (!bp->edev) - return; - - aux_dev =3D &bp->aux_priv->aux_dev; - rc =3D auxiliary_device_add(aux_dev); - if (rc) { - netdev_warn(bp->dev, "Failed to add auxiliary device for ROCE\n"); - auxiliary_device_uninit(aux_dev); - bp->flags &=3D ~BNXT_FLAG_ROCE_CAP; + int rc, idx; + + for (idx =3D 0; idx < __BNXT_AUXDEV_MAX; idx++) { + if (bnxt_auxdev_is_init(bp, idx)) { + aux_dev =3D &bp->aux_priv[idx]->aux_dev; + rc =3D auxiliary_device_add(aux_dev); + if (rc) { + netdev_warn(bp->dev, "Failed to add auxiliary device for ROCE\n"); + auxiliary_device_uninit(aux_dev); + if (idx =3D=3D BNXT_AUXDEV_RDMA) + bp->flags &=3D ~BNXT_FLAG_ROCE_CAP; + continue; + } + bnxt_auxdev_set_state(bp, idx, BNXT_ADEV_STATE_ADD); + } } } =20 -void bnxt_rdma_aux_device_init(struct bnxt *bp) +void bnxt_aux_devices_init(struct bnxt *bp) { struct auxiliary_device *aux_dev; struct bnxt_aux_priv *aux_priv; struct bnxt_en_dev *edev; struct bnxt_ulp *ulp; - int rc; + int rc, idx; =20 - if (!(bp->flags & BNXT_FLAG_ROCE_CAP)) - return; + for (idx =3D 0; idx < __BNXT_AUXDEV_MAX; idx++) { + bnxt_auxdev_set_state(bp, idx, BNXT_ADEV_STATE_NONE); =20 - aux_priv =3D kzalloc(sizeof(*bp->aux_priv), GFP_KERNEL); - if (!aux_priv) - goto exit; - - aux_priv->id =3D ida_alloc(&bnxt_aux_dev_ids, GFP_KERNEL); - if (aux_priv->id < 0) { - netdev_warn(bp->dev, - "ida alloc failed for ROCE auxiliary device\n"); - kfree(aux_priv); - goto exit; - } + if (idx =3D=3D BNXT_AUXDEV_RDMA && + !(bp->flags & BNXT_FLAG_ROCE_CAP)) + continue; =20 - aux_dev =3D &aux_priv->aux_dev; - aux_dev->id =3D aux_priv->id; - aux_dev->name =3D "rdma"; - aux_dev->dev.parent =3D &bp->pdev->dev; - aux_dev->dev.release =3D bnxt_aux_dev_release; + aux_priv =3D kzalloc(sizeof(*aux_priv), GFP_KERNEL); + if (!aux_priv) + goto next_auxdev; =20 - rc =3D auxiliary_device_init(aux_dev); - if (rc) { - ida_free(&bnxt_aux_dev_ids, aux_priv->id); - kfree(aux_priv); - goto exit; - } - bp->aux_priv =3D aux_priv; + aux_dev =3D &aux_priv->aux_dev; + aux_dev->id =3D bp->auxdev_id; + aux_dev->name =3D bnxt_aux_devices[idx].name; + aux_dev->dev.parent =3D &bp->pdev->dev; + aux_dev->dev.release =3D bnxt_aux_dev_release; =20 - /* From this point, all cleanup will happen via the .release callback & - * any error unwinding will need to include a call to - * auxiliary_device_uninit. - */ - edev =3D kzalloc(sizeof(*edev), GFP_KERNEL); - if (!edev) - goto aux_dev_uninit; + rc =3D auxiliary_device_init(aux_dev); + if (rc) { + kfree(aux_priv); + goto next_auxdev; + } + bp->aux_priv[idx] =3D aux_priv; =20 - aux_priv->edev =3D edev; + /* From this point, all cleanup will happen via the .release + * callback & any error unwinding will need to include a call + * to auxiliary_device_uninit. + */ + edev =3D kzalloc(sizeof(*edev), GFP_KERNEL); + if (!edev) + goto aux_dev_uninit; =20 - ulp =3D kzalloc(sizeof(*ulp), GFP_KERNEL); - if (!ulp) - goto aux_dev_uninit; + aux_priv->edev =3D edev; + bnxt_set_edev_info(edev, bp); =20 - edev->ulp_tbl =3D ulp; - bp->edev =3D edev; - bnxt_set_edev_info(edev, bp); - bp->ulp_num_msix_want =3D bnxt_set_dflt_ulp_msix(bp); + ulp =3D kzalloc(sizeof(*ulp), GFP_KERNEL); + if (!ulp) + goto aux_dev_uninit; =20 - return; + edev->ulp_tbl =3D ulp; + bp->edev[idx] =3D edev; + if (idx =3D=3D BNXT_AUXDEV_RDMA) + bp->ulp_num_msix_want =3D bnxt_set_dflt_ulp_msix(bp); + aux_priv->id =3D idx; + bnxt_auxdev_set_state(bp, idx, BNXT_ADEV_STATE_INIT); =20 + continue; aux_dev_uninit: - auxiliary_device_uninit(aux_dev); -exit: - bp->flags &=3D ~BNXT_FLAG_ROCE_CAP; + auxiliary_device_uninit(aux_dev); +next_auxdev: + if (idx =3D=3D BNXT_AUXDEV_RDMA) + bp->flags &=3D ~BNXT_FLAG_ROCE_CAP; + } +} + +int bnxt_auxdev_id_alloc(struct bnxt *bp) +{ + bp->auxdev_id =3D ida_alloc(&bnxt_aux_dev_ids, GFP_KERNEL); + if (bp->auxdev_id < 0) + return bp->auxdev_id; + + return 0; +} + +void bnxt_auxdev_id_free(struct bnxt *bp, int id) +{ + if (bp->auxdev_id >=3D 0) + ida_free(&bnxt_aux_dev_ids, id); } diff --git a/include/linux/bnxt/ulp.h b/include/linux/bnxt/ulp.h index 3c5b8a53f715..1a4643c46f86 100644 --- a/include/linux/bnxt/ulp.h +++ b/include/linux/bnxt/ulp.h @@ -10,6 +10,8 @@ #ifndef BNXT_ULP_H #define BNXT_ULP_H =20 +#include + #define BNXT_MIN_ROCE_CP_RINGS 2 #define BNXT_MIN_ROCE_STAT_CTXS 1 =20 @@ -20,6 +22,17 @@ struct hwrm_async_event_cmpl; struct bnxt; =20 +enum bnxt_auxdev_type { + BNXT_AUXDEV_RDMA =3D 0, + __BNXT_AUXDEV_MAX +}; + +struct bnxt_aux_priv { + struct auxiliary_device aux_dev; + struct bnxt_en_dev *edev; + int id; +}; + struct bnxt_msix_entry { u32 vector; u32 ring_idx; @@ -110,19 +123,21 @@ void bnxt_set_ulp_stat_ctxs(struct bnxt *bp, int num_= ctxs); int bnxt_get_ulp_stat_ctxs_in_use(struct bnxt *bp); void bnxt_set_dflt_ulp_stat_ctxs(struct bnxt *bp); void bnxt_ulp_stop(struct bnxt *bp); -void bnxt_ulp_start(struct bnxt *bp, int err); +void bnxt_ulp_start(struct bnxt *bp); void bnxt_ulp_sriov_cfg(struct bnxt *bp, int num_vfs); void bnxt_ulp_irq_stop(struct bnxt *bp); void bnxt_ulp_irq_restart(struct bnxt *bp, int err); void bnxt_ulp_async_events(struct bnxt *bp, struct hwrm_async_event_cmpl *= cmpl); -void bnxt_rdma_aux_device_uninit(struct bnxt *bp); -void bnxt_rdma_aux_device_del(struct bnxt *bp); -void bnxt_rdma_aux_device_add(struct bnxt *bp); -void bnxt_rdma_aux_device_init(struct bnxt *bp); +void bnxt_aux_devices_uninit(struct bnxt *bp); +void bnxt_aux_devices_del(struct bnxt *bp); +void bnxt_aux_devices_add(struct bnxt *bp); +void bnxt_aux_devices_init(struct bnxt *bp); int bnxt_register_dev(struct bnxt_en_dev *edev, struct bnxt_ulp_ops *ulp_o= ps, void *handle); void bnxt_unregister_dev(struct bnxt_en_dev *edev); int bnxt_send_msg(struct bnxt_en_dev *edev, struct bnxt_fw_msg *fw_msg); void bnxt_register_async_events(struct bnxt_en_dev *edev, unsigned long *events_bmap, u16 max_id); +int bnxt_auxdev_id_alloc(struct bnxt *bp); +void bnxt_auxdev_id_free(struct bnxt *bp, int id); #endif --=20 2.39.1