From nobody Sat Oct 24 13:21:15 2026 Received: from mail-oa1-f98.google.com (mail-oa1-f98.google.com [209.85.160.98]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 97865390609 for ; Tue, 13 Oct 2026 20:08:08 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.160.98 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1791922091; cv=none; b=WLHvkm7t1LjkF577T9L5aLo0PUq1O8hbWojD+Gf1iMx7CguSvJVL1JQ7G1m7y5FLHsa+hrTYdAWj6xnbd8qQ7/gRsoQjm5EDG36ga0i1co6ehEiePw35VYgKDxPzrtu588FKqU/cGjPi3k6zVNdUjkVQPL0onZsEV9KfFGNbNtg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1791922091; c=relaxed/simple; bh=lpjZhT6XXWw6XX138SjuWWgMvV1BnmW4k7hdFt8UpY8=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=uwYdDLxpf85XW5HpYBNsCTo8VXKo3X1rCiLyg352UW5lYdU+vLeUUIKkiiu4GaB3uFUqc/x2FmsPeitCwuarVw5d1dRBsb+eEYi+awPISkycMsyKKd7uYdW9sLRHF1WqqFryAuUmEM5RhJUZZVSpcLGosvTEwvqnboNgocK8IL4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=broadcom.com; spf=fail smtp.mailfrom=broadcom.com; dkim=pass (1024-bit key) header.d=broadcom.com header.i=@broadcom.com header.b=OFLrgXD8; arc=none smtp.client-ip=209.85.160.98 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=broadcom.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=broadcom.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=broadcom.com header.i=@broadcom.com header.b="OFLrgXD8" Received: by mail-oa1-f98.google.com with SMTP id 586e51a60fabf-4a69499cefaso1195184fac.0 for ; Tue, 13 Oct 2026 13:08:08 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20260707; t=1791922087; x=1792526887; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:dkim-signature:x-gm-gg :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to :content-type; bh=3hPRXO6sFueqWIi4s/FdTD+h3BsbEISg9BIgMP6UGWc=; b=e39S49GK8ibp73GX84Eoh6r7tDWsK8ur5MJON1lubC7l0kB2jFfL0sAUSm9m/9C6Yx Me22WG7hTCj4y3k/I8geYeXXbNCeeuxman7k9aJTyDJRxHoW0D6S3qjAI6IJj2pNrCua iO2SJNmy2lVKfw1n26xR7qZOAfi75sFhSGT0sap39HjGkYMO4M7Z9Lw/8sr0v0ffoDI+ aqhiU72QYI7mBi3g4ds3PkBlNre4QGlkWN/VFj8I9r4cfusQUsl2ag5+T3DnnW7poSvN wgozCUexMcN6UEmIHYh4unPtbTgTBnGj/EfJy72yvzwET7F51p4UFXH6w9vieaEWCg/B LwHA== X-Forwarded-Encrypted: i=1; AKwUvBw1yu9TEqXOWjH/hJALWBNoLBIuMUloH0/VXTe9fAzChWotjT5Zck2gIcwRoqE2EufFKB8jdQTyCzQUfYA=@vger.kernel.org X-Gm-Message-State: AFq9FYKwZ2plgChks7ovsHyUKm4kweBUCRdLBji38PxA5ktTgvBSwlNE MOkKI79AEOhd2yh2VjorQxmbtjYS9Ne5iYm0dw1GsrMD57LyCyaDk5Tt4wXO4ZNOSzlACURkSKh pnJbKIFwSyJFWAZx8hWYeKF5VTjIl5LBS/aEgqzw5BOKe2FzmQZwXdT2XIquScXngFkHVJYay7F YFhsr7mvarspNUwNZoZd3KshUBjcnrsqoLsFK99rqIc7sT6hLZKsgFStxhKDCplVID/5lfZ8H8C WxUX3Zq2+dJbgXFEEDe5uQHFHHo9mc= X-Gm-Gg: AYBFou3Bl6TQptk000R3fYd64ymk+YWou22EFi9KrEpdgwx8Byt/Ue2HetfPhOje1ET lDztWrr2oAgqwFdBmdy6WVcmsHUpL7SJHe+IfP28kBrsJwsPNg8M/C4jZHbVXUe25by1t9EbbK2 ZuE0A6cZ1wXcWt7U5DvJgA+BgEm9wk82dgZ0Iw1tkc2QCHSCPENDgU2z/KafgjFUKv84SZFR18D xZpkYJH+IHI7lXraF+AtMJWnqFlfPvTfzp3/e1/P2uAd7dzIzQBL11IQkHU3E6Jk+zIfC5CrwDB 5rxSE0d1PO6r6i75eM7omdJFS3Qy9RRNkimmaTGhfWxLjoABU5iN0sz0MSTtOGQrV8dYr0HqMeN BD3Q8aZOTBpNMHbq+0BFMPJLf5LHsRtTE4LJDEElQbOUjiKAVD4ZO7QaR9qZtBoR1TqUAK4ogJ+ 2ngfww3bc8GN3UN+BuBsOezduS+Pr5rXeZiNcTUDNpUY3n X-Received: by 2002:a05:6870:248c:b0:49e:241:600e with SMTP id 586e51a60fabf-4a2a4b4676amr10074470fac.36.1791922087037; Tue, 13 Oct 2026 13:08:07 -0700 (PDT) Received: from smtp-us-east1-p01-i01-si01.dlp.protect.broadcom.com (address-144-49-247-120.dlp.protect.broadcom.com. [144.49.247.120]) by smtp-relay.gmail.com with ESMTPS id 586e51a60fabf-4a6dfbbb573sm23804fac.5.2026.10.13.13.08.06 for (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Tue, 13 Oct 2026 13:08:07 -0700 (PDT) X-Relaying-Domain: broadcom.com X-CFilter-Loop: Reflected Received: by mail-dl1-f70.google.com with SMTP id a92af1059eb24-14364a88479so149120c88.1 for ; Tue, 13 Oct 2026 13:08:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=broadcom.com; s=google; t=1791922085; x=1792526885; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=3hPRXO6sFueqWIi4s/FdTD+h3BsbEISg9BIgMP6UGWc=; b=OFLrgXD8NFLCwhX+2hhCUs1WVkVQMPd/igrBmkZgEfTJkMEyH6+abisl6wMbvBXZIa +z7a6YZB7nAYqP61pxpeAwFrGPq5iaG55qAtk1eDEdlSoRBPj/nRD1sd44040yFe4DWU GJxqNoBYgDe7S+tgJHVoyU0MzrCstu8bsdUzA= X-Forwarded-Encrypted: i=1; AKwUvBy+G+eVvVMdTVLBar8bBfH6y7ZHwSNoCBKogwSxcXZt+7vMfBdhwYrP1kfeM+yXDMCESXbamlyFfOLv24U=@vger.kernel.org X-Received: by 2002:a05:701b:4284:10b0:146:84fd:7b58 with SMTP id a92af1059eb24-16a10f2a74fmr18600553c88.9.1791922084924; Tue, 13 Oct 2026 13:08:04 -0700 (PDT) X-Received: by 2002:a05:701b:4284:10b0:146:84fd:7b58 with SMTP id a92af1059eb24-16a10f2a74fmr18600506c88.9.1791922083973; Tue, 13 Oct 2026 13:08:03 -0700 (PDT) Received: from localhost.localdomain ([192.19.203.250]) by smtp.gmail.com with ESMTPSA id a92af1059eb24-169a31f6f5csm39526585c88.11.2026.10.13.13.07.58 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 13 Oct 2026 13:08:03 -0700 (PDT) From: Bhargava Marreddy To: davem@davemloft.net, edumazet@google.com, kuba@kernel.org, pabeni@redhat.com, andrew+netdev@lunn.ch, horms@kernel.org Cc: netdev@vger.kernel.org, linux-kernel@vger.kernel.org, michael.chan@broadcom.com, pavan.chebbi@broadcom.com, vsrama-krishna.nemani@broadcom.com, vikas.gupta@broadcom.com, Bhargava Marreddy , Dharmender Garg , Rajashekar Hudumula , Ramakrishna Koduri Subject: [PATCH net-next v3 1/5] bnge: factor out per-NQ alloc/free helpers Date: Wed, 14 Oct 2026 01:36:41 +0530 Message-ID: <20261013200645.1346299-2-bhargava.marreddy@broadcom.com> X-Mailer: git-send-email 2.47.3 In-Reply-To: <20261013200645.1346299-1-bhargava.marreddy@broadcom.com> References: <20261013200645.1346299-1-bhargava.marreddy@broadcom.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-DetectorID-Processed: b00c1d49-9d2e-4205-b15f-d015386d3d5e Content-Type: text/plain; charset="utf-8" The descriptor array, ring memory, firmware ring and IRQ of an NQ are allocated and released inside loops over all NQs, and the NQ ring memory is set up with the completion rings. Factor the per-NQ parts out into bnge_alloc_nq()/bnge_free_nq(), bnge_hwrm_nq_alloc()/bnge_hwrm_nq_free() and bnge_nq_request_irq()/bnge_nq_free_irq() so a single NQ can be created and destroyed independently of the others. Rename bnge_{alloc,free}_nq_arrays() to bnge_{alloc,free}_nqs(). The NQ ring_mem setup, ring allocation and fw_ring_id initialization move into bnge_alloc_nq(), so the NQ ring memory is allocated with the NQ descriptor arrays rather than with the completion rings. The fw_ring_id initialization thus moves from bnge_init_nic(), which runs on every open, to bnge_alloc_core(), which also runs on every open. The IRQ map index is also set there; the value is unchanged. The affinity hint is now derived from bnapi->index, which equals the loop index used before. The error path of bnge_alloc_nqs() now unwinds only the NQs allocated so far. Also rename bnge_{alloc,free,init}_nq_tree() to bnge_{alloc,free,init}_cp_rings(), since they no longer touch the NQ itself, only the completion rings hanging off it. One behavioral difference: bnge_hwrm_ring_free() now resets grp_info[i].nq_fw_ring_id to INVALID_HW_RING_ID for every NQ, not only for the NQs that had a valid firmware ring ID. Otherwise there is no functional change. Signed-off-by: Bhargava Marreddy Reviewed-by: Dharmender Garg Reviewed-by: Rajashekar Hudumula Reviewed-by: Ramakrishna Koduri --- .../net/ethernet/broadcom/bnge/bnge_netdev.c | 284 ++++++++++-------- .../net/ethernet/broadcom/bnge/bnge_rmem.c | 10 - 2 files changed, 163 insertions(+), 131 deletions(-) diff --git a/drivers/net/ethernet/broadcom/bnge/bnge_netdev.c b/drivers/net= /ethernet/broadcom/bnge/bnge_netdev.c index 25f844e6f6d3..8cd0d8bd68c2 100644 --- a/drivers/net/ethernet/broadcom/bnge/bnge_netdev.c +++ b/drivers/net/ethernet/broadcom/bnge/bnge_netdev.c @@ -503,61 +503,89 @@ static int bnge_alloc_cp_desc_arr(struct bnge_cp_ring= _info *cpr, int n) return -ENOMEM; } =20 -static void bnge_free_nq_arrays(struct bnge_net *bn) +static void bnge_free_nq(struct bnge_net *bn, struct bnge_napi *bnapi) { - struct bnge_dev *bd =3D bn->bd; - int i; + struct bnge_nq_ring_info *nqr =3D &bnapi->nq_ring; =20 - for (i =3D 0; i < bd->nq_nr_rings; i++) { - struct bnge_napi *bnapi =3D bn->bnapi[i]; + bnge_free_ring(bn->bd, &nqr->ring_struct.ring_mem); + bnge_free_nq_desc_arr(nqr); +} + +static int bnge_alloc_nq(struct bnge_net *bn, struct bnge_napi *bnapi) +{ + struct bnge_nq_ring_info *nqr =3D &bnapi->nq_ring; + struct bnge_ring_mem_info *rmem; + struct bnge_ring_struct *ring; + int rc; + + ring =3D &nqr->ring_struct; + rmem =3D &ring->ring_mem; =20 - bnge_free_nq_desc_arr(&bnapi->nq_ring); + rc =3D bnge_alloc_nq_desc_arr(nqr, bn->cp_nr_pages); + if (rc) + return rc; + + nqr->bnapi =3D bnapi; + rmem->nr_pages =3D bn->cp_nr_pages; + rmem->page_size =3D HW_CMPD_RING_SIZE; + rmem->pg_arr =3D (void **)nqr->desc_ring; + rmem->dma_arr =3D nqr->desc_mapping; + rmem->vmem_size =3D 0; + rmem->flags =3D BNGE_RMEM_RING_PTE_FLAG; + rc =3D bnge_alloc_ring(bn->bd, rmem); + if (rc) { + bnge_free_nq_desc_arr(nqr); + return rc; } + + ring->fw_ring_id =3D INVALID_HW_RING_ID_32BIT; + ring->map_idx =3D bnge_aux_get_msix(bn->bd) + bnapi->index; + return 0; } =20 -static int bnge_alloc_nq_arrays(struct bnge_net *bn) +static void bnge_free_nqs(struct bnge_net *bn) +{ + struct bnge_dev *bd =3D bn->bd; + int i; + + for (i =3D 0; i < bd->nq_nr_rings; i++) + bnge_free_nq(bn, bn->bnapi[i]); +} + +static int bnge_alloc_nqs(struct bnge_net *bn) { struct bnge_dev *bd =3D bn->bd; int i, rc; =20 for (i =3D 0; i < bd->nq_nr_rings; i++) { - struct bnge_napi *bnapi =3D bn->bnapi[i]; - - rc =3D bnge_alloc_nq_desc_arr(&bnapi->nq_ring, bn->cp_nr_pages); + rc =3D bnge_alloc_nq(bn, bn->bnapi[i]); if (rc) - goto err_free_nq_arrays; + goto err_free_nqs; } return 0; =20 -err_free_nq_arrays: - bnge_free_nq_arrays(bn); +err_free_nqs: + while (--i >=3D 0) + bnge_free_nq(bn, bn->bnapi[i]); return rc; } =20 -static void bnge_free_nq_tree(struct bnge_net *bn) +static void bnge_free_cp_rings(struct bnge_net *bn) { struct bnge_dev *bd =3D bn->bd; int i; =20 for (i =3D 0; i < bd->nq_nr_rings; i++) { - struct bnge_napi *bnapi =3D bn->bnapi[i]; - struct bnge_nq_ring_info *nqr; - struct bnge_ring_struct *ring; + struct bnge_nq_ring_info *nqr =3D &bn->bnapi[i]->nq_ring; int j; =20 - nqr =3D &bnapi->nq_ring; - ring =3D &nqr->ring_struct; - - bnge_free_ring(bd, &ring->ring_mem); - if (!nqr->cp_ring_arr) continue; =20 for (j =3D 0; j < nqr->cp_ring_count; j++) { struct bnge_cp_ring_info *cpr =3D &nqr->cp_ring_arr[j]; =20 - ring =3D &cpr->ring_struct; - bnge_free_ring(bd, &ring->ring_mem); + bnge_free_ring(bd, &cpr->ring_struct.ring_mem); bnge_free_cp_desc_arr(cpr); } kfree(nqr->cp_ring_arr); @@ -594,31 +622,21 @@ static int alloc_one_cp_ring(struct bnge_net *bn, return rc; } =20 -static int bnge_alloc_nq_tree(struct bnge_net *bn) +static int bnge_alloc_cp_rings(struct bnge_net *bn) { struct bnge_dev *bd =3D bn->bd; - int i, j, ulp_msix, rc; + int i, j, rc; int tcs =3D 1; =20 - ulp_msix =3D bnge_aux_get_msix(bd); for (i =3D 0, j =3D 0; i < bd->nq_nr_rings; i++) { bool sh =3D !!(bd->flags & BNGE_EN_SHARED_CHNL); struct bnge_napi *bnapi =3D bn->bnapi[i]; struct bnge_nq_ring_info *nqr; struct bnge_cp_ring_info *cpr; - struct bnge_ring_struct *ring; int cp_count =3D 0, k; int rx =3D 0, tx =3D 0; =20 nqr =3D &bnapi->nq_ring; - nqr->bnapi =3D bnapi; - ring =3D &nqr->ring_struct; - - rc =3D bnge_alloc_ring(bd, &ring->ring_mem); - if (rc) - goto err_free_nq_tree; - - ring->map_idx =3D ulp_msix + i; =20 if (i < bd->rx_nr_rings) { cp_count++; @@ -634,7 +652,7 @@ static int bnge_alloc_nq_tree(struct bnge_net *bn) nqr->cp_ring_arr =3D kzalloc_objs(*cpr, cp_count); if (!nqr->cp_ring_arr) { rc =3D -ENOMEM; - goto err_free_nq_tree; + goto err_free_cp_rings; } =20 nqr->cp_ring_count =3D cp_count; @@ -643,7 +661,7 @@ static int bnge_alloc_nq_tree(struct bnge_net *bn) cpr =3D &nqr->cp_ring_arr[k]; rc =3D alloc_one_cp_ring(bn, cpr); if (rc) - goto err_free_nq_tree; + goto err_free_cp_rings; =20 cpr->bnapi =3D bnapi; cpr->cp_idx =3D k; @@ -663,8 +681,8 @@ static int bnge_alloc_nq_tree(struct bnge_net *bn) } return 0; =20 -err_free_nq_tree: - bnge_free_nq_tree(bn); +err_free_cp_rings: + bnge_free_cp_rings(bn); return rc; } =20 @@ -1200,8 +1218,8 @@ static void bnge_free_core(struct bnge_net *bn) bnge_free_vnic_attributes(bn); bnge_free_tx_rings(bn); bnge_free_rx_rings(bn); - bnge_free_nq_tree(bn); - bnge_free_nq_arrays(bn); + bnge_free_cp_rings(bn); + bnge_free_nqs(bn); bnge_free_ring_stats(bn); bnge_free_ring_grps(bn); bnge_free_vnics(bn); @@ -1232,13 +1250,9 @@ static int bnge_alloc_core(struct bnge_net *bn) bn->bnapi =3D bnapi; bnapi +=3D arr_size; for (i =3D 0; i < bd->nq_nr_rings; i++, bnapi +=3D size) { - struct bnge_nq_ring_info *nqr; - bn->bnapi[i] =3D bnapi; bn->bnapi[i]->index =3D i; bn->bnapi[i]->bn =3D bn; - nqr =3D &bn->bnapi[i]->nq_ring; - nqr->ring_struct.ring_mem.flags =3D BNGE_RMEM_RING_PTE_FLAG; } =20 bn->rx_ring =3D kzalloc_objs(struct bnge_rx_ring_info, bd->rx_nr_rings); @@ -1297,7 +1311,7 @@ static int bnge_alloc_core(struct bnge_net *bn) if (rc) goto err_free_ring_stats; =20 - rc =3D bnge_alloc_nq_arrays(bn); + rc =3D bnge_alloc_nqs(bn); if (rc) goto err_free_vnics; =20 @@ -1305,13 +1319,13 @@ static int bnge_alloc_core(struct bnge_net *bn) =20 rc =3D bnge_alloc_rx_rings(bn); if (rc) - goto err_free_nq_arrays; + goto err_free_nqs; =20 rc =3D bnge_alloc_tx_rings(bn); if (rc) goto err_free_rx_rings; =20 - rc =3D bnge_alloc_nq_tree(bn); + rc =3D bnge_alloc_cp_rings(bn); if (rc) goto err_free_tx_rings; =20 @@ -1320,17 +1334,17 @@ static int bnge_alloc_core(struct bnge_net *bn) BNGE_VNIC_UCAST_FLAG; rc =3D bnge_alloc_vnic_attributes(bn); if (rc) - goto err_free_nq_tree; + goto err_free_cp_rings; return 0; =20 -err_free_nq_tree: - bnge_free_nq_tree(bn); +err_free_cp_rings: + bnge_free_cp_rings(bn); err_free_tx_rings: bnge_free_tx_rings(bn); err_free_rx_rings: bnge_free_rx_rings(bn); -err_free_nq_arrays: - bnge_free_nq_arrays(bn); +err_free_nqs: + bnge_free_nqs(bn); err_free_vnics: bnge_free_vnics(bn); err_free_ring_stats: @@ -1389,21 +1403,18 @@ static int bnge_cp_num_to_irq_num(struct bnge_net *= bn, int n) return nqr->ring_struct.map_idx; } =20 -static void bnge_init_nq_tree(struct bnge_net *bn) +static void bnge_init_cp_rings(struct bnge_net *bn) { struct bnge_dev *bd =3D bn->bd; int i, j; =20 for (i =3D 0; i < bd->nq_nr_rings; i++) { struct bnge_nq_ring_info *nqr =3D &bn->bnapi[i]->nq_ring; - struct bnge_ring_struct *ring =3D &nqr->ring_struct; =20 - ring->fw_ring_id =3D INVALID_HW_RING_ID_32BIT; for (j =3D 0; j < nqr->cp_ring_count; j++) { struct bnge_cp_ring_info *cpr =3D &nqr->cp_ring_arr[j]; =20 - ring =3D &cpr->ring_struct; - ring->fw_ring_id =3D INVALID_HW_RING_ID_32BIT; + cpr->ring_struct.fw_ring_id =3D INVALID_HW_RING_ID_32BIT; } } } @@ -1914,6 +1925,44 @@ static int bnge_hwrm_rx_ring_alloc(struct bnge_net *= bn, return 0; } =20 +static int bnge_hwrm_nq_alloc(struct bnge_net *bn, struct bnge_napi *bnapi) +{ + struct bnge_nq_ring_info *nqr =3D &bnapi->nq_ring; + u32 type =3D HWRM_RING_ALLOC_NQ; + struct bnge_ring_struct *ring; + unsigned int vector; + u32 map_idx; + int rc; + + ring =3D &nqr->ring_struct; + map_idx =3D ring->map_idx; + + vector =3D bn->bd->irq_tbl[map_idx].vector; + disable_irq_nosync(vector); + rc =3D hwrm_ring_alloc_send_msg(bn, ring, type, map_idx); + if (rc) { + enable_irq(vector); + return rc; + } + bnge_set_db(bn, &nqr->nq_db, type, map_idx, ring->fw_ring_id); + bnge_db_nq(bn, &nqr->nq_db, nqr->nq_raw_cons); + enable_irq(vector); + + return 0; +} + +static void bnge_hwrm_nq_free(struct bnge_net *bn, struct bnge_napi *bnapi) +{ + struct bnge_ring_struct *ring =3D &bnapi->nq_ring.ring_struct; + + if (ring->fw_ring_id =3D=3D INVALID_HW_RING_ID_32BIT) + return; + + hwrm_ring_free_send_msg(bn, ring, RING_FREE_REQ_RING_TYPE_NQ, + INVALID_HW_RING_ID); + ring->fw_ring_id =3D INVALID_HW_RING_ID_32BIT; +} + static int bnge_hwrm_ring_alloc(struct bnge_net *bn) { struct bnge_dev *bd =3D bn->bd; @@ -1923,22 +1972,11 @@ static int bnge_hwrm_ring_alloc(struct bnge_net *bn) agg_rings =3D !!(bnge_is_agg_reqd(bd)); for (i =3D 0; i < bd->nq_nr_rings; i++) { struct bnge_napi *bnapi =3D bn->bnapi[i]; - struct bnge_nq_ring_info *nqr =3D &bnapi->nq_ring; - struct bnge_ring_struct *ring =3D &nqr->ring_struct; - u32 type =3D HWRM_RING_ALLOC_NQ; - u32 map_idx =3D ring->map_idx; - unsigned int vector; + struct bnge_ring_struct *ring =3D &bnapi->nq_ring.ring_struct; =20 - vector =3D bd->irq_tbl[map_idx].vector; - disable_irq_nosync(vector); - rc =3D hwrm_ring_alloc_send_msg(bn, ring, type, map_idx); - if (rc) { - enable_irq(vector); + rc =3D bnge_hwrm_nq_alloc(bn, bnapi); + if (rc) goto err_out; - } - bnge_set_db(bn, &nqr->nq_db, type, map_idx, ring->fw_ring_id); - bnge_db_nq(bn, &nqr->nq_db, nqr->nq_raw_cons); - enable_irq(vector); bn->grp_info[i].nq_fw_ring_id =3D (u16)ring->fw_ring_id; =20 if (!i) { @@ -2566,23 +2604,14 @@ static void bnge_hwrm_ring_free(struct bnge_net *bn= , bool close_path) bnge_disable_int_sync(bn); =20 for (i =3D 0; i < bd->nq_nr_rings; i++) { - struct bnge_napi *bnapi =3D bn->bnapi[i]; - struct bnge_nq_ring_info *nqr; - struct bnge_ring_struct *ring; + struct bnge_nq_ring_info *nqr =3D &bn->bnapi[i]->nq_ring; int j; =20 - nqr =3D &bnapi->nq_ring; for (j =3D 0; j < nqr->cp_ring_count && nqr->cp_ring_arr; j++) bnge_hwrm_cp_ring_free(bn, &nqr->cp_ring_arr[j]); =20 - ring =3D &nqr->ring_struct; - if (ring->fw_ring_id !=3D INVALID_HW_RING_ID_32BIT) { - hwrm_ring_free_send_msg(bn, ring, - RING_FREE_REQ_RING_TYPE_NQ, - INVALID_HW_RING_ID); - ring->fw_ring_id =3D INVALID_HW_RING_ID_32BIT; - bn->grp_info[i].nq_fw_ring_id =3D INVALID_HW_RING_ID; - } + bnge_hwrm_nq_free(bn, bn->bnapi[i]); + bn->grp_info[i].nq_fw_ring_id =3D INVALID_HW_RING_ID; } } =20 @@ -2627,27 +2656,60 @@ static void bnge_hwrm_resource_free(struct bnge_net= *bn, bool close_path) bnge_hwrm_stat_ctx_free(bn); } =20 -static void bnge_free_irq(struct bnge_net *bn) +static void bnge_nq_free_irq(struct bnge_net *bn, struct bnge_napi *bnapi) +{ + int map_idx =3D bnapi->nq_ring.ring_struct.map_idx; + struct bnge_irq *irq =3D &bn->bd->irq_tbl[map_idx]; + + if (irq->requested) { + if (irq->have_cpumask) { + irq_set_affinity_hint(irq->vector, NULL); + free_cpumask_var(irq->cpu_mask); + irq->have_cpumask =3D 0; + } + free_irq(irq->vector, bnapi); + } + + irq->requested =3D 0; +} + +static int bnge_nq_request_irq(struct bnge_net *bn, struct bnge_napi *bnap= i) { + int map_idx =3D bnapi->nq_ring.ring_struct.map_idx; struct bnge_dev *bd =3D bn->bd; struct bnge_irq *irq; - int i; + int rc; =20 - for (i =3D 0; i < bd->nq_nr_rings; i++) { - int map_idx =3D bnge_cp_num_to_irq_num(bn, i); + irq =3D &bd->irq_tbl[map_idx]; =20 - irq =3D &bd->irq_tbl[map_idx]; - if (irq->requested) { - if (irq->have_cpumask) { - irq_set_affinity_hint(irq->vector, NULL); - free_cpumask_var(irq->cpu_mask); - irq->have_cpumask =3D 0; - } - free_irq(irq->vector, bn->bnapi[i]); - } + rc =3D request_irq(irq->vector, irq->handler, 0, irq->name, bnapi); + if (rc) + return rc; + + netif_napi_set_irq_locked(&bnapi->napi, irq->vector); + irq->requested =3D 1; + + if (zalloc_cpumask_var(&irq->cpu_mask, GFP_KERNEL)) { + int numa_node =3D dev_to_node(&bd->pdev->dev); =20 - irq->requested =3D 0; + irq->have_cpumask =3D 1; + cpumask_set_cpu(cpumask_local_spread(bnapi->index, numa_node), + irq->cpu_mask); + if (irq_set_affinity_hint(irq->vector, irq->cpu_mask)) + netdev_warn(bn->netdev, + "Set affinity failed, IRQ =3D %d\n", + irq->vector); } + return 0; +} + +static void bnge_free_irq(struct bnge_net *bn) +{ + struct bnge_dev *bd =3D bn->bd; + int i; + + for (i =3D 0; i < bd->nq_nr_rings; i++) + bnge_nq_free_irq(bn, bn->bnapi[i]); } =20 static int bnge_request_irq(struct bnge_net *bn) @@ -2661,29 +2723,9 @@ static int bnge_request_irq(struct bnge_net *bn) return rc; } for (i =3D 0; i < bd->nq_nr_rings; i++) { - int map_idx =3D bnge_cp_num_to_irq_num(bn, i); - struct bnge_irq *irq =3D &bd->irq_tbl[map_idx]; - - rc =3D request_irq(irq->vector, irq->handler, 0, irq->name, - bn->bnapi[i]); + rc =3D bnge_nq_request_irq(bn, bn->bnapi[i]); if (rc) goto err_free_irq; - - netif_napi_set_irq_locked(&bn->bnapi[i]->napi, irq->vector); - irq->requested =3D 1; - - if (zalloc_cpumask_var(&irq->cpu_mask, GFP_KERNEL)) { - int numa_node =3D dev_to_node(&bd->pdev->dev); - - irq->have_cpumask =3D 1; - cpumask_set_cpu(cpumask_local_spread(i, numa_node), - irq->cpu_mask); - if (irq_set_affinity_hint(irq->vector, irq->cpu_mask)) { - netdev_warn(bn->netdev, - "Set affinity failed, IRQ =3D %d\n", - irq->vector); - } - } } return 0; =20 @@ -2830,7 +2872,7 @@ static int bnge_init_nic(struct bnge_net *bn) { int rc; =20 - bnge_init_nq_tree(bn); + bnge_init_cp_rings(bn); =20 bnge_init_rx_rings(bn); rc =3D bnge_alloc_rx_ring_pair_bufs(bn); diff --git a/drivers/net/ethernet/broadcom/bnge/bnge_rmem.c b/drivers/net/e= thernet/broadcom/bnge/bnge_rmem.c index b066ee887a09..bbae84b4157e 100644 --- a/drivers/net/ethernet/broadcom/bnge/bnge_rmem.c +++ b/drivers/net/ethernet/broadcom/bnge/bnge_rmem.c @@ -431,20 +431,10 @@ void bnge_init_ring_struct(struct bnge_net *bn) for (i =3D 0; i < bd->nq_nr_rings; i++) { struct bnge_napi *bnapi =3D bn->bnapi[i]; struct bnge_ring_mem_info *rmem; - struct bnge_nq_ring_info *nqr; struct bnge_rx_ring_info *rxr; struct bnge_tx_ring_info *txr; struct bnge_ring_struct *ring; =20 - nqr =3D &bnapi->nq_ring; - ring =3D &nqr->ring_struct; - rmem =3D &ring->ring_mem; - rmem->nr_pages =3D bn->cp_nr_pages; - rmem->page_size =3D HW_CMPD_RING_SIZE; - rmem->pg_arr =3D (void **)nqr->desc_ring; - rmem->dma_arr =3D nqr->desc_mapping; - rmem->vmem_size =3D 0; - rxr =3D bnapi->rx_ring; if (!rxr) goto skip_rx; --=20 2.47.3 From nobody Sat Oct 24 13:21:15 2026 Received: from mail-yw1-f225.google.com (mail-yw1-f225.google.com [209.85.128.225]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 235735476E6 for ; Tue, 13 Oct 2026 20:08:12 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.225 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1791922095; cv=none; b=g7SDFto3POJ4x2OZSAkoYkJxOS/Pe2mPUJz3IQPWzUKuoJ0+QTCnHdCySf2ua1c2c8vPDYbd0bnJ4DDoQKDtBBSRTikeRWkdGfPEYPMbH5BUW+/MNuiOhKaA8MI+87vRNDvrOiF5Z7rcGQFo5dP245xxSnhFHw3e/tt9cK/PsCo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1791922095; c=relaxed/simple; bh=nuj98LMHJV7YXMvZFP0mgsnkil3gxXP9Diu5IS6CATg=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=mofoccI63eRHEzziW3FhoQTPHw/Ov6HMVbO3brOv787X7kR9+lP8nAoeOstADgt2SfaZDlvNOyEA/b98NxWUPus0AyGL4i2/MJ2pdHOo7cnD7jMpLNWfk9H9RrQ0Iz2wC26lW36+rdVeZPDlYQSzcnVtR95V+lsPDcP+C21Tf14= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=broadcom.com; spf=fail smtp.mailfrom=broadcom.com; dkim=pass (1024-bit key) header.d=broadcom.com header.i=@broadcom.com header.b=Z91HPY7z; arc=none smtp.client-ip=209.85.128.225 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=broadcom.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=broadcom.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=broadcom.com header.i=@broadcom.com header.b="Z91HPY7z" Received: by mail-yw1-f225.google.com with SMTP id 00721157ae682-8ac551be4d3so27021527b3.3 for ; Tue, 13 Oct 2026 13:08:12 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20260707; t=1791922092; x=1792526892; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:dkim-signature:x-gm-gg :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to :content-type; bh=UOX1fnqCCqvGPeGhBPtjplBZCMxVrAi4/APZ6zwxUvY=; b=vIGW9aCrsBJTneBOIcHzZ+3diuwy8CEjmo70MMbRqrTL677E8pmG4PZ9hP9qVw3Dgs Cc40PefYDQDTqskhvB1lHlla2iUlESrK75IGq6QxikNzoh7Glb+lTe1UBLy6NP51PrEW 0OpZs5pdZ6vCyL22mntHg1SoHjkQkuuzGv9I5ekR0TKFOIVS519I882IWdvTJv75Cl0M KhFWQqQBj9fhL3qQ89px/utjVZEn98P7gSMUYT1BbSiax+5kksvxzb0GHdQzYjfL7qHb /sXRQ0NrU0Nf9VQ/VcRg9L8HYgzdUUukviTG275rkX37iihk4MEVbBMipl8D4AnJAO6v 5VqQ== X-Forwarded-Encrypted: i=1; AKwUvBwemuZlt4prCsbPjBY1oXgzl0BYDZF/6if4zkAPcuF4Zkkl7/CDuoBTSm7V2iSM7oRKaHeiDRKuTAnPkxk=@vger.kernel.org X-Gm-Message-State: AFq9FYLiFAVd276CgiWNwjo2BpwZSFFh6+YhjMt5kcorQNGmkjQ0Ugy7 oJDujbdeC4dgNKD+mwTu9/vPDGaDCXqNCMqLYFCKxngSIbWVha1D1uzRXfaYM/yLwx27Penqiu9 MeK6wALPFMvP9gmzCDyhlkUGm5DrMYsB8Bm7v/e9CgVrWPnBI7AJgPTmSCPVCTZ9uSLI8aM4M/I 4uKfXFvFqFuq0BXm06JcOQ3mtJOZIKXlwn4ejwZ+rA+fl66EEMfDDIGgSaMDhtMxCRrxgCOMtbw jGwBR1vVpBei5mGZH2I+rVydto6jrg= X-Gm-Gg: AYBFou03Ek7Tfukx83FZvpslJIx1f6W0FDv8iQLNYmCs4XydEGEocQ/HfeXKt35AHok GoOo1z1DklC5k+XvuTFGCBLVxi+x8JbMg6e0A+gor+aCepTaY8/WLeAWCrUXekMiOlT39b7FaOd 0np1Z9zY9mr1STwR93PYP3TFKTgGJfUzKaVVnnFB1UPJElT6oeKZxNDbqLnCaMY24DIIDpGUUyQ XGsaKJi0ZuOatUzVoMPZv3/5YhO6+DKYdSFuhe7+lMTa9+VMlReDI+su+WCdIODNsm7ILsmUaOd ye+7Eg9plQcay0ZmE8HSsys3Ob76cCpkazKeLPhG4ISmVliwbOaZA0V0oMvd45EhRtVusMA7NsN ZexdeTq9Mwtd4DSsxD/ETjYvOZJ1OuqZUP+TgapNcuFa4SztK5EKanA8JCea7EoVn+Mh3GDZboa JiitSdnDLFoDIOMTq+aHYfpDleWXSnnKoqej/nawy41WyU X-Received: by 2002:a05:690c:6d84:b0:8b0:8b34:edc2 with SMTP id 00721157ae682-8b08b34f047mr35035717b3.19.1791922091867; Tue, 13 Oct 2026 13:08:11 -0700 (PDT) Received: from smtp-us-east1-p01-i01-si01.dlp.protect.broadcom.com (address-144-49-247-120.dlp.protect.broadcom.com. [144.49.247.120]) by smtp-relay.gmail.com with ESMTPS id 00721157ae682-8b07f30c569sm696567b3.37.2026.10.13.13.08.11 for (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Tue, 13 Oct 2026 13:08:11 -0700 (PDT) X-Relaying-Domain: broadcom.com X-CFilter-Loop: Reflected Received: by mail-dl1-f69.google.com with SMTP id a92af1059eb24-152d1363a29so12558478c88.1 for ; Tue, 13 Oct 2026 13:08:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=broadcom.com; s=google; t=1791922091; x=1792526891; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=UOX1fnqCCqvGPeGhBPtjplBZCMxVrAi4/APZ6zwxUvY=; b=Z91HPY7zRaFWXBohwshKSeOvqCw5QU0xXoYnVzee/XF3Ai2ERedjUDYHnTdhnpyEJz eObThpIIG/bG6Z9XWNwVCD2S5kGean0wUjpApvGjsX7W2rIXpCyeNhHnmhcIEmnnhW+d VJtaPWOc/hLOCd9h9fb7jNFPb/NIk659repFs= X-Forwarded-Encrypted: i=1; AKwUvBxPxSDLl5VX8OTKc+bupibAqzViC519+ytaxdpnst8fEfc9/HCGzpk8TAVJJhNc8Xl7SGTSNpSgPnD8J/0=@vger.kernel.org X-Received: by 2002:a05:701b:2707:b0:144:c127:dd1b with SMTP id a92af1059eb24-16a5fe7c188mr20179186c88.39.1791922090230; Tue, 13 Oct 2026 13:08:10 -0700 (PDT) X-Received: by 2002:a05:701b:2707:b0:144:c127:dd1b with SMTP id a92af1059eb24-16a5fe7c188mr20179129c88.39.1791922089345; Tue, 13 Oct 2026 13:08:09 -0700 (PDT) Received: from localhost.localdomain ([192.19.203.250]) by smtp.gmail.com with ESMTPSA id a92af1059eb24-169a31f6f5csm39526585c88.11.2026.10.13.13.08.04 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 13 Oct 2026 13:08:08 -0700 (PDT) From: Bhargava Marreddy To: davem@davemloft.net, edumazet@google.com, kuba@kernel.org, pabeni@redhat.com, andrew+netdev@lunn.ch, horms@kernel.org Cc: netdev@vger.kernel.org, linux-kernel@vger.kernel.org, michael.chan@broadcom.com, pavan.chebbi@broadcom.com, vsrama-krishna.nemani@broadcom.com, vikas.gupta@broadcom.com, Bhargava Marreddy , Dharmender Garg , Kumar Mudide , Ramakrishna Koduri Subject: [PATCH net-next v3 2/5] bnge: add bnge_err()/bnge_warn()/bnge_dbg() logging helpers Date: Wed, 14 Oct 2026 01:36:42 +0530 Message-ID: <20261013200645.1346299-3-bhargava.marreddy@broadcom.com> X-Mailer: git-send-email 2.47.3 In-Reply-To: <20261013200645.1346299-1-bhargava.marreddy@broadcom.com> References: <20261013200645.1346299-1-bhargava.marreddy@broadcom.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-DetectorID-Processed: b00c1d49-9d2e-4205-b15f-d015386d3d5e Content-Type: text/plain; charset="utf-8" netdev_err() and netdev_warn() print "(unnamed net_device) (uninitialized)" instead of the interface name while the netdev has not been registered yet. Later patches in this series run NQ0 init before register_netdev() in probe, so messages from the HWRM ring alloc/free paths and from NQ0's NAPI would be logged that way. Add bnge_err(), bnge_warn() and bnge_dbg(), which take a struct bnge_dev and log through the netdev once it has been registered, including while it is being unregistered and after, where it keeps its name. Before that, fall back to dev_err()/dev_warn()/dev_dbg() on the PCI device. Convert the HWRM ring alloc/free error messages, the IRQ affinity hint warning and the async event debug message, which are reachable from those paths. No other message is changed. No functional change. Signed-off-by: Bhargava Marreddy Reviewed-by: Dharmender Garg Reviewed-by: Kumar Mudide Reviewed-by: Ramakrishna Koduri --- drivers/net/ethernet/broadcom/bnge/bnge.h | 18 ++++++++++++++++++ .../net/ethernet/broadcom/bnge/bnge_hwrm_lib.c | 8 +++++--- .../net/ethernet/broadcom/bnge/bnge_netdev.c | 5 ++--- drivers/net/ethernet/broadcom/bnge/bnge_txrx.c | 4 ++-- 4 files changed, 27 insertions(+), 8 deletions(-) diff --git a/drivers/net/ethernet/broadcom/bnge/bnge.h b/drivers/net/ethern= et/broadcom/bnge/bnge.h index 4479ccd071f5..e76ba4e063bb 100644 --- a/drivers/net/ethernet/broadcom/bnge/bnge.h +++ b/drivers/net/ethernet/broadcom/bnge/bnge.h @@ -220,6 +220,24 @@ struct bnge_dev { u32 phy_flags; }; =20 +/* The netdev has no name until register_netdev(), so log through the PCI + * device before that (e.g. during probe). + */ +#define bnge_log(level, bd, fmt, ...) \ + do { \ + struct bnge_dev *__bd =3D (bd); \ + \ + if (__bd->netdev && \ + READ_ONCE(__bd->netdev->reg_state) !=3D NETREG_UNINITIALIZED)\ + netdev_##level(__bd->netdev, fmt, ##__VA_ARGS__);\ + else \ + dev_##level(__bd->dev, fmt, ##__VA_ARGS__); \ + } while (0) + +#define bnge_err(bd, fmt, ...) bnge_log(err, bd, fmt, ##__VA_ARGS__) +#define bnge_warn(bd, fmt, ...) bnge_log(warn, bd, fmt, ##__VA_ARGS__) +#define bnge_dbg(bd, fmt, ...) bnge_log(dbg, bd, fmt, ##__VA_ARGS__) + static inline bool bnge_is_roce_en(struct bnge_dev *bd) { return bd->flags & BNGE_EN_ROCE; diff --git a/drivers/net/ethernet/broadcom/bnge/bnge_hwrm_lib.c b/drivers/n= et/ethernet/broadcom/bnge/bnge_hwrm_lib.c index 651c5e783516..fb960bff85fa 100644 --- a/drivers/net/ethernet/broadcom/bnge/bnge_hwrm_lib.c +++ b/drivers/net/ethernet/broadcom/bnge/bnge_hwrm_lib.c @@ -1302,7 +1302,8 @@ int hwrm_ring_free_send_msg(struct bnge_net *bn, bnge_hwrm_req_drop(bd, req); exit: if (rc) { - netdev_err(bd->netdev, "hwrm_ring_free type %d failed. rc:%d\n", ring_ty= pe, rc); + bnge_err(bd, "hwrm_ring_free type %d failed. rc:%d\n", + ring_type, rc); return -EIO; } return 0; @@ -1395,7 +1396,7 @@ int hwrm_ring_alloc_send_msg(struct bnge_net *bn, req->int_mode =3D RING_ALLOC_REQ_INT_MODE_MSIX; break; default: - netdev_err(bn->netdev, "hwrm alloc invalid ring type %d\n", ring_type); + bnge_err(bd, "hwrm alloc invalid ring type %d\n", ring_type); return -EINVAL; } =20 @@ -1406,7 +1407,8 @@ int hwrm_ring_alloc_send_msg(struct bnge_net *bn, =20 exit: if (rc) { - netdev_err(bd->netdev, "hwrm_ring_alloc type %d failed. rc:%d\n", ring_t= ype, rc); + bnge_err(bd, "hwrm_ring_alloc type %d failed. rc:%d\n", + ring_type, rc); return -EIO; } ring->fw_ring_id =3D ring_id; diff --git a/drivers/net/ethernet/broadcom/bnge/bnge_netdev.c b/drivers/net= /ethernet/broadcom/bnge/bnge_netdev.c index 8cd0d8bd68c2..f8f81f3f7cd9 100644 --- a/drivers/net/ethernet/broadcom/bnge/bnge_netdev.c +++ b/drivers/net/ethernet/broadcom/bnge/bnge_netdev.c @@ -2696,9 +2696,8 @@ static int bnge_nq_request_irq(struct bnge_net *bn, s= truct bnge_napi *bnapi) cpumask_set_cpu(cpumask_local_spread(bnapi->index, numa_node), irq->cpu_mask); if (irq_set_affinity_hint(irq->vector, irq->cpu_mask)) - netdev_warn(bn->netdev, - "Set affinity failed, IRQ =3D %d\n", - irq->vector); + bnge_warn(bd, "Set affinity failed, IRQ =3D %d\n", + irq->vector); } return 0; } diff --git a/drivers/net/ethernet/broadcom/bnge/bnge_txrx.c b/drivers/net/e= thernet/broadcom/bnge/bnge_txrx.c index 472b02cc43b3..2f382514c6ea 100644 --- a/drivers/net/ethernet/broadcom/bnge/bnge_txrx.c +++ b/drivers/net/ethernet/broadcom/bnge/bnge_txrx.c @@ -1135,8 +1135,8 @@ static void bnge_async_event_process(struct bnge_net = *bn, u32 data1 =3D le32_to_cpu(cmpl->event_data1); u32 data2 =3D le32_to_cpu(cmpl->event_data2); =20 - netdev_dbg(bn->netdev, "hwrm event 0x%x {0x%x, 0x%x}\n", - event_id, data1, data2); + bnge_dbg(bn->bd, "hwrm event 0x%x {0x%x, 0x%x}\n", + event_id, data1, data2); =20 switch (event_id) { case ASYNC_EVENT_CMPL_EVENT_ID_LINK_SPEED_CFG_CHANGE: --=20 2.47.3 From nobody Sat Oct 24 13:21:15 2026 Received: from mail-ot1-f99.google.com (mail-ot1-f99.google.com [209.85.210.99]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 17CBD5476D8 for ; Tue, 13 Oct 2026 20:08:20 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.99 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1791922103; cv=none; b=tj0+mRC4gu4PkYE5s0j4Gng6+4RxVxtjamME4sh9Z+YZYWhUHvVscBCemTvRLyJ/J0470u1ckxSS14LofZWGGenRFHruqtrTLkEwxZtnhaDm0sI55zl4R0TU+zzwP6TYnm4i9M9x/35OJx4Mwd4ygsdJMI+ByU9LFjkJqmtAYSU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1791922103; c=relaxed/simple; bh=9yo9xv6KxKaLJOPj8QN0fOXVqpQCRiX2rAK9W0phauE=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=Lh9J13wK18yh7InKsnzjRp7oK13ySSGZgCpDUtpx35Sk2p0RwFwus2e2XcLzmRWXxtdMIzQU+YA7SzEXYdCdesRw1yBaZ6lXsTnVIIv/Li7Re3jiaKbOxPbuPvSekefpjN5z+MYY7hA2xwlM+myhJ5pBidT3dgshUbG7gI2cpMY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=broadcom.com; spf=fail smtp.mailfrom=broadcom.com; dkim=pass (1024-bit key) header.d=broadcom.com header.i=@broadcom.com header.b=NBpipssB; arc=none smtp.client-ip=209.85.210.99 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=broadcom.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=broadcom.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=broadcom.com header.i=@broadcom.com header.b="NBpipssB" Received: by mail-ot1-f99.google.com with SMTP id 46e09a7af769-822ddb835fbso2486764a34.0 for ; Tue, 13 Oct 2026 13:08:20 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20260707; t=1791922099; x=1792526899; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:dkim-signature:x-gm-gg :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to :content-type; bh=rbjWRwvBceXDUpKmVzvJBj3QPpFh+Fowktv6+6Nd1Uc=; b=K18i1vxizmpOy2UJ7qSXxeX7n89mUDtrbUtUgdJWObMW+mB2uZKvrskjhx3OLb5uBN YJ7/LBrJITnCqY2x23JLhul5fTUc2GqIWdvLU33ZMvhXBuordugvBo3p12EMwcvWIUS9 nOS1Yt7VEVir+ADGXohiFcWJBzgcVw/SPItpAlseNdTZAIBSl8XxQ04HxUMJLzx8MZjB JvpErD9pHUG/6VrzvLxThoQ9LzQj/al8tD79GmiP04EL8/Rqx/AHM0tFPuJUVaZTLzZT nyVYEHiyvLRPrkO9Znwnn4iZwqfwqCVggY0wHFxW3pvfIN+eCtcqhh2rmUXyjiSNhBJj OuWg== X-Forwarded-Encrypted: i=1; AKwUvBzygCBpZe6+4EnC660XjqEYUbY/x+7JsV16zcHhmMSGDKncReOuOp3jPIbGgFwmBvSXGovRfpeKmM9dTN0=@vger.kernel.org X-Gm-Message-State: AFq9FYL8QKfVw+nbFvz7uiRNST8OKOwsxrrtmkCH1U3OQEborTE+qYQU ipQvwrwMS1egzG/SqkkLR0kjffJsGjrxMoHH+2BaIrC2isXoMRO9aEms6WZHEvWTiorPXQ64iq4 ylB9XpfduRW7V1aZ6AkAh8GF33JdQWEhjn7PoJ51ptNX8UrikVP1NaiNsLhUx2eO8GCeCpxqXfb 8MB2qegyvIOURnQSR4T5UrUqeHEqblvz38zO7xtsNiq84Bnt4RgqokpzYFq1jjVOdCEcs+nVd8Q CfHqqlQCYSAboU07VJV7ZfMewwRvsE= X-Gm-Gg: AYBFou0Ln6ucsGkFZheERjd1uhvsOqvzXvaWNfP7Y+Y6NGdxV6Ng3n2bbpoJnL0vXfD dla3QcoXbXtQ6A6GPoT+CINWbs6GrAwlE7ZmHymFqTTfiDI8Z3eCuiHpunfih1+UFNHnscsQWkN nocMPS5vvjTJFIJMelBzKWu/FmQdzjWCJggksgr6PEdvnRX5cEw9cechyuYxJZBQAfUwtqifKhu 9S69j3ePayhpUrHaSb7QPrAcerE0I6BpKcc+/hpbypoaLuqesGfC6MXr0CCz2V1HfIe0c2zCS+P ykKZqwurOQL658UFio6YuEShJc3T1OCHSvLxT3flLLrP+o52yqN7QBszxHHVmTfc07RWxYSEWlS nP2znKLtOXdyDz2h3MsHj0DE0Iy3/IixsQG9gofV9gUanL3lhs+YkEwFbmEtOImXu2Z+RPykc77 CeD5pmjwBZxbcnZwVxXOnKp/ghxPo8H/HRZOgc7dD7VPKE X-Received: by 2002:a05:6808:6f94:b0:4c3:b290:cf55 with SMTP id 5614622812f47-50c42a26c1dmr11553219b6e.7.1791922099563; Tue, 13 Oct 2026 13:08:19 -0700 (PDT) Received: from smtp-us-east1-p01-i01-si01.dlp.protect.broadcom.com (address-144-49-247-125.dlp.protect.broadcom.com. [144.49.247.125]) by smtp-relay.gmail.com with ESMTPS id 586e51a60fabf-4a6dfcb178csm24633fac.18.2026.10.13.13.08.17 for (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Tue, 13 Oct 2026 13:08:19 -0700 (PDT) X-Relaying-Domain: broadcom.com X-CFilter-Loop: Reflected Received: by mail-dl1-f71.google.com with SMTP id a92af1059eb24-156b5338d4bso563323c88.1 for ; Tue, 13 Oct 2026 13:08:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=broadcom.com; s=google; t=1791922096; x=1792526896; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=rbjWRwvBceXDUpKmVzvJBj3QPpFh+Fowktv6+6Nd1Uc=; b=NBpipssBI1awyaBfO9yTIZUYsKB0o2atQJbjNxnjQDti43juYDPnt9AIV2OqbAvv2P TimCUYaajWghjOg2F0ps/xk0UrV2OqnfsbXUwOBxsPI24tv0hI4A2UkPV5BPh4QCp6KD DBXxIaAYqT8q01ouRUPWa1MdIrBTKCNkXXbp8= X-Forwarded-Encrypted: i=1; AKwUvBxQea9OzoEFmA3O2h66X/esTwvI5Q9AAE4zFT4TkCvIlrBXiRyYYZCPdPeSUJ/OoMAt/QiYh2uXH680LMA=@vger.kernel.org X-Received: by 2002:a05:701b:21db:b0:149:ef22:2fa2 with SMTP id a92af1059eb24-16a5dcf5a62mr18926970c88.8.1791922096211; Tue, 13 Oct 2026 13:08:16 -0700 (PDT) X-Received: by 2002:a05:701b:21db:b0:149:ef22:2fa2 with SMTP id a92af1059eb24-16a5dcf5a62mr18926890c88.8.1791922094795; Tue, 13 Oct 2026 13:08:14 -0700 (PDT) Received: from localhost.localdomain ([192.19.203.250]) by smtp.gmail.com with ESMTPSA id a92af1059eb24-169a31f6f5csm39526585c88.11.2026.10.13.13.08.09 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 13 Oct 2026 13:08:14 -0700 (PDT) From: Bhargava Marreddy To: davem@davemloft.net, edumazet@google.com, kuba@kernel.org, pabeni@redhat.com, andrew+netdev@lunn.ch, horms@kernel.org Cc: netdev@vger.kernel.org, linux-kernel@vger.kernel.org, michael.chan@broadcom.com, pavan.chebbi@broadcom.com, vsrama-krishna.nemani@broadcom.com, vikas.gupta@broadcom.com, Bhargava Marreddy , Dharmender Garg , Kumar Mudide , Ramakrishna Koduri Subject: [PATCH net-next v3 3/5] bnge: add bnge_for_each_open_nq() iterator Date: Wed, 14 Oct 2026 01:36:43 +0530 Message-ID: <20261013200645.1346299-4-bhargava.marreddy@broadcom.com> X-Mailer: git-send-email 2.47.3 In-Reply-To: <20261013200645.1346299-1-bhargava.marreddy@broadcom.com> References: <20261013200645.1346299-1-bhargava.marreddy@broadcom.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-DetectorID-Processed: b00c1d49-9d2e-4205-b15f-d015386d3d5e Content-Type: text/plain; charset="utf-8" Add BNGE_FIRST_OPEN_NQ and bnge_for_each_open_nq(), and use them in the loops that will need to skip the NQ that is created at probe once it is split out. The "open" NQs are the ones created in ndo_open() and destroyed in ndo_stop(), i.e. all NQs from index BNGE_FIRST_OPEN_NQ on. The loop that carves the bnge_napi instances out of the bn->bnapi allocation in bnge_alloc_core() is converted as well; its allocation size will be reduced by BNGE_FIRST_OPEN_NQ when the probe-time NQ moves out of it. The loops over the completion rings, over the per-NQ stats and ring group state, and the interrupt disable/sync loops are intentionally left alone, as they still cover all NQs. The latter must mask and synchronize the IRQ of the probe-time NQ before its completion rings are freed. In bnge_hwrm_ring_free(), split the NQ free out of the completion ring loop so that it can use the iterator, mirroring bnge_hwrm_ring_alloc(). This changes the order of the firmware commands: previously each NQ was freed right after its own completion rings, now all completion rings are freed first and all NQs after. Each completion ring is still freed before the NQ it belongs to. BNGE_FIRST_OPEN_NQ is 0 for now, i.e. every NQ is an open NQ, so the macro expands to the loops it replaces. Apart from the firmware command order above, no functional change. Signed-off-by: Bhargava Marreddy Reviewed-by: Dharmender Garg Reviewed-by: Kumar Mudide Reviewed-by: Ramakrishna Koduri --- drivers/net/ethernet/broadcom/bnge/bnge.h | 9 ++++++ .../net/ethernet/broadcom/bnge/bnge_netdev.c | 29 ++++++++++--------- 2 files changed, 25 insertions(+), 13 deletions(-) diff --git a/drivers/net/ethernet/broadcom/bnge/bnge.h b/drivers/net/ethern= et/broadcom/bnge/bnge.h index e76ba4e063bb..fdd8bca01440 100644 --- a/drivers/net/ethernet/broadcom/bnge/bnge.h +++ b/drivers/net/ethernet/broadcom/bnge/bnge.h @@ -238,6 +238,15 @@ struct bnge_dev { #define bnge_warn(bd, fmt, ...) bnge_log(warn, bd, fmt, ##__VA_ARGS__) #define bnge_dbg(bd, fmt, ...) bnge_log(dbg, bd, fmt, ##__VA_ARGS__) =20 +/* Index of the first NQ created in ndo_open() and destroyed in ndo_stop(). + * NQs below it are created at probe and destroyed at remove. It starts at= 0, + * i.e. all NQs are open NQs, until NQ0 moves to probe. + */ +#define BNGE_FIRST_OPEN_NQ 0 + +#define bnge_for_each_open_nq(i, bd) \ + for ((i) =3D BNGE_FIRST_OPEN_NQ; (i) < (bd)->nq_nr_rings; (i)++) + static inline bool bnge_is_roce_en(struct bnge_dev *bd) { return bd->flags & BNGE_EN_ROCE; diff --git a/drivers/net/ethernet/broadcom/bnge/bnge_netdev.c b/drivers/net= /ethernet/broadcom/bnge/bnge_netdev.c index f8f81f3f7cd9..0d98223cd703 100644 --- a/drivers/net/ethernet/broadcom/bnge/bnge_netdev.c +++ b/drivers/net/ethernet/broadcom/bnge/bnge_netdev.c @@ -548,7 +548,7 @@ static void bnge_free_nqs(struct bnge_net *bn) struct bnge_dev *bd =3D bn->bd; int i; =20 - for (i =3D 0; i < bd->nq_nr_rings; i++) + bnge_for_each_open_nq(i, bd) bnge_free_nq(bn, bn->bnapi[i]); } =20 @@ -557,7 +557,7 @@ static int bnge_alloc_nqs(struct bnge_net *bn) struct bnge_dev *bd =3D bn->bd; int i, rc; =20 - for (i =3D 0; i < bd->nq_nr_rings; i++) { + bnge_for_each_open_nq(i, bd) { rc =3D bnge_alloc_nq(bn, bn->bnapi[i]); if (rc) goto err_free_nqs; @@ -565,7 +565,7 @@ static int bnge_alloc_nqs(struct bnge_net *bn) return 0; =20 err_free_nqs: - while (--i >=3D 0) + while (--i >=3D BNGE_FIRST_OPEN_NQ) bnge_free_nq(bn, bn->bnapi[i]); return rc; } @@ -1249,10 +1249,11 @@ static int bnge_alloc_core(struct bnge_net *bn) =20 bn->bnapi =3D bnapi; bnapi +=3D arr_size; - for (i =3D 0; i < bd->nq_nr_rings; i++, bnapi +=3D size) { + bnge_for_each_open_nq(i, bd) { bn->bnapi[i] =3D bnapi; bn->bnapi[i]->index =3D i; bn->bnapi[i]->bn =3D bn; + bnapi +=3D size; } =20 bn->rx_ring =3D kzalloc_objs(struct bnge_rx_ring_info, bd->rx_nr_rings); @@ -1970,7 +1971,7 @@ static int bnge_hwrm_ring_alloc(struct bnge_net *bn) int i, rc =3D 0; =20 agg_rings =3D !!(bnge_is_agg_reqd(bd)); - for (i =3D 0; i < bd->nq_nr_rings; i++) { + bnge_for_each_open_nq(i, bd) { struct bnge_napi *bnapi =3D bn->bnapi[i]; struct bnge_ring_struct *ring =3D &bnapi->nq_ring.ring_struct; =20 @@ -2426,7 +2427,7 @@ static void bnge_enable_int(struct bnge_net *bn) struct bnge_dev *bd =3D bn->bd; int i; =20 - for (i =3D 0; i < bd->nq_nr_rings; i++) { + bnge_for_each_open_nq(i, bd) { struct bnge_napi *bnapi =3D bn->bnapi[i]; struct bnge_nq_ring_info *nqr; =20 @@ -2443,7 +2444,7 @@ static void bnge_disable_napi(struct bnge_net *bn) if (test_and_set_bit(BNGE_STATE_NAPI_DISABLED, &bn->state)) return; =20 - for (i =3D 0; i < bd->nq_nr_rings; i++) { + bnge_for_each_open_nq(i, bd) { struct bnge_napi *bnapi =3D bn->bnapi[i]; =20 napi_disable_locked(&bnapi->napi); @@ -2456,7 +2457,7 @@ static void bnge_enable_napi(struct bnge_net *bn) int i; =20 clear_bit(BNGE_STATE_NAPI_DISABLED, &bn->state); - for (i =3D 0; i < bd->nq_nr_rings; i++) { + bnge_for_each_open_nq(i, bd) { struct bnge_napi *bnapi =3D bn->bnapi[i]; =20 bnapi->in_reset =3D false; @@ -2609,7 +2610,9 @@ static void bnge_hwrm_ring_free(struct bnge_net *bn, = bool close_path) =20 for (j =3D 0; j < nqr->cp_ring_count && nqr->cp_ring_arr; j++) bnge_hwrm_cp_ring_free(bn, &nqr->cp_ring_arr[j]); + } =20 + bnge_for_each_open_nq(i, bd) { bnge_hwrm_nq_free(bn, bn->bnapi[i]); bn->grp_info[i].nq_fw_ring_id =3D INVALID_HW_RING_ID; } @@ -2622,7 +2625,7 @@ static void bnge_setup_msix(struct bnge_net *bn) int len, i; =20 len =3D sizeof(bd->irq_tbl[0].name); - for (i =3D 0; i < bd->nq_nr_rings; i++) { + bnge_for_each_open_nq(i, bd) { int map_idx =3D bnge_cp_num_to_irq_num(bn, i); char *attr; =20 @@ -2707,7 +2710,7 @@ static void bnge_free_irq(struct bnge_net *bn) struct bnge_dev *bd =3D bn->bd; int i; =20 - for (i =3D 0; i < bd->nq_nr_rings; i++) + bnge_for_each_open_nq(i, bd) bnge_nq_free_irq(bn, bn->bnapi[i]); } =20 @@ -2721,7 +2724,7 @@ static int bnge_request_irq(struct bnge_net *bn) netdev_err(bn->netdev, "bnge_setup_interrupts err: %d\n", rc); return rc; } - for (i =3D 0; i < bd->nq_nr_rings; i++) { + bnge_for_each_open_nq(i, bd) { rc =3D bnge_nq_request_irq(bn, bn->bnapi[i]); if (rc) goto err_free_irq; @@ -2840,7 +2843,7 @@ static void bnge_init_napi(struct bnge_net *bn) struct bnge_napi *bnapi; int i; =20 - for (i =3D 0; i < bd->nq_nr_rings; i++) { + bnge_for_each_open_nq(i, bd) { bnapi =3D bn->bnapi[i]; netif_napi_add_config_locked(bn->netdev, &bnapi->napi, bnge_napi_poll, bnapi->index); @@ -2857,7 +2860,7 @@ static void bnge_del_napi(struct bnge_net *bn) for (i =3D 0; i < bd->tx_nr_rings; i++) netif_queue_set_napi(bn->netdev, i, NETDEV_QUEUE_TYPE_TX, NULL); =20 - for (i =3D 0; i < bd->nq_nr_rings; i++) { + bnge_for_each_open_nq(i, bd) { struct bnge_napi *bnapi =3D bn->bnapi[i]; =20 __netif_napi_del_locked(&bnapi->napi); --=20 2.47.3 From nobody Sat Oct 24 13:21:15 2026 Received: from mail-pj1-f97.google.com (mail-pj1-f97.google.com [209.85.216.97]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 172EE548575 for ; Tue, 13 Oct 2026 20:08:23 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.97 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1791922106; cv=none; b=EqLygeZLaHNJsLzeUjTuUeIO8KZPi8BDvrU0jUHlsb2+qYhcoYVl+D/fympDMxanMeAHyy05qaj6Qgx/nevq0T+hqJ4eU9bYCWSwdoIIifkDfHssTq7jD2hUiVviDgC0OZO3aMOOmhq+3N9+BUHndfXneaYxDuUJgBIESsFZ8gA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1791922106; c=relaxed/simple; bh=HRGi3+8s+xZ0Pv2YLVltUZU5MS3BORmziEe4ICaxj2E=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=HUPTXHaCuVuRHJh8otT9eCBt11Vd/G+jPOiEqlpzTTqRmfpkf9b7YeER+6utyFlte+1u2zZ/HWSgHPt8Bvrpran47U57Br9J9c6j/pA7+Wlvzs0vmzBBlZUJE4X8xC3bDqNR6moUV74UyqV6Aml1huYWnoMd8Lwwyqg933qO7ys= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=broadcom.com; spf=fail smtp.mailfrom=broadcom.com; dkim=pass (1024-bit key) header.d=broadcom.com header.i=@broadcom.com header.b=T2mVNeGo; arc=none smtp.client-ip=209.85.216.97 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=broadcom.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=broadcom.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=broadcom.com header.i=@broadcom.com header.b="T2mVNeGo" Received: by mail-pj1-f97.google.com with SMTP id 98e67ed59e1d1-3a4b3ac8fb7so1534751a91.0 for ; Tue, 13 Oct 2026 13:08:23 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20260707; t=1791922103; x=1792526903; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:dkim-signature:x-gm-gg :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to :content-type; bh=M2ErZj7OvfiHzwXbTEuEIcm9vf6VpN09VWgyrHzyJFw=; b=uupsgH4fUxTPq/MejM9YCWEJJwGs0q9DQsGUJaErkGVkl0DCDHjJxEGuUNgCOaM5z4 k7UtgOwduT67x30H/KeSGG4UFWpBXkerNp+sAdwCP7KhaoyJyZ5eXCFp/SsUlhWk+BHF Rx09r9BvAILcx290THQHDLnu7T6Pzm35zTF479ypxe3WZmCgASDe7IsTK3npIqg9XuWY tNd2w/ghAFXV9MaQ7yGKRY2BjyovbSA2LRyWF1PwQD7Lj4VF+aqaOMOEYn6KlKwHmGHB KC4ImSJquAYrUu03dG/66q07g9AoLWFw7/rofzYiOx51+3lTf86RZKCmDgSyIhBqBrNc lFbQ== X-Forwarded-Encrypted: i=1; AKwUvBwhzq7XedZkdBUcyiSdlq7q3NeVPLev6ZwJC18+jCGDNXDrxGhR3O3vyo2y0Vr9WKV+DAQho7rgJKALXtQ=@vger.kernel.org X-Gm-Message-State: AFq9FYJascO7zhfJsETgxdsQElu+8PC4c25CdqqouV8kVqP6ImMh+9Rp tlMf0zg3xxSiuXX8UqQ67D4hEZTKssBaKXSgLa2Ievg8ZpnCJeXNLGoxgAnfqIGZWuEbXwbtctS qv+2BeUcz4PzSiJrwPIwTDzW2OQAwMh0/HiqA6UULrkX1N6tbc6zt8/rLARF+RqZv1Fu71yuJDV caBurgcPgITl8XtpWxuZoOhNaqVVyVBC3QzLGRydxPZs7qsVK7pJXzEy/Ny+o59OkZfgsssJDVC 1vRwPtMlZmy6Pfhxg3+43AFpQ== X-Gm-Gg: AYBFou1VtyMyHFXhzqu6xT3D4IWfbwfox7fGvdLKDIGdidpt38Gc8DZzrPN458bgE2o ECWTsZThheYEIXeP+KkNisLdY/x98x8uC5rZOVbxKtCULtGQ3ceBHOJjtKVjtUVAbnmBXGVGyGt mMFWe64nxvti0bhwhTaaDJwr10n9CwUJO8rNVmeHiuvqxT6RgNmogUIMGpVeHmCFJ5buCvLWtnb c0ACBv2KaVRNCA4wZSU2ANjUeIGZlA/kmTWJzfqkaBqE2lhAMUQtlE27HGkv5CymufCJuaPYLzG 4ImvgAHS5P7MBdoB7RXD1lvX+Q9Sf9hYJY3Rml/WYhC6ZIiJHGASy3PNxqjTX6uqffYutNNpoIX TTCH3b1LHJpT3d7rDIY73jKMV4pJH7ogkMY+FQFlS4RYWD6Af+Sk+kjv4RCmtLQdTcb30tevIWU jFONuQjnwDs+ZEKu0sw+PP3hNiDn1fYCEMpRJSdY4LvdOQ X-Received: by 2002:a17:90b:524f:b0:3a8:c0cd:57f8 with SMTP id 98e67ed59e1d1-3ab3a239c2fmr10577684a91.4.1791922102992; Tue, 13 Oct 2026 13:08:22 -0700 (PDT) Received: from smtp-us-east1-p01-i01-si01.dlp.protect.broadcom.com (address-144-49-247-120.dlp.protect.broadcom.com. [144.49.247.120]) by smtp-relay.gmail.com with ESMTPS id 98e67ed59e1d1-3ac60395c4fsm310244a91.6.2026.10.13.13.08.22 for (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Tue, 13 Oct 2026 13:08:22 -0700 (PDT) X-Relaying-Domain: broadcom.com X-CFilter-Loop: Reflected Received: by mail-dl1-f69.google.com with SMTP id a92af1059eb24-147be78cd56so13115406c88.1 for ; Tue, 13 Oct 2026 13:08:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=broadcom.com; s=google; t=1791922101; x=1792526901; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=M2ErZj7OvfiHzwXbTEuEIcm9vf6VpN09VWgyrHzyJFw=; b=T2mVNeGoIkhbtajAiAk8t2Yan2lNEPw5td6IHwmTyso52eE0XRx6VgrD59Zdm5k0Uc Y7OkFbAOnCW7If826p2Q9fMrnIGhdID563YCgnT+H0xodLs8UNFdfiqRIpcoqq1MuqKC AmXRs6F3XjgEUMUwq9GMNoLkJ16uD8NjT6MHc= X-Forwarded-Encrypted: i=1; AKwUvBy/N8DHfWzXgpl23R27SzNMZDu8wv4sJT84YZzZClfhuA6YvcxpGo6MENwbEWxVg0SfNySE8EUaxz2Urpo=@vger.kernel.org X-Received: by 2002:a05:701b:4583:10b0:149:cfa4:9b3b with SMTP id a92af1059eb24-16a5b2c7120mr18196181c88.3.1791922101133; Tue, 13 Oct 2026 13:08:21 -0700 (PDT) X-Received: by 2002:a05:701b:4583:10b0:149:cfa4:9b3b with SMTP id a92af1059eb24-16a5b2c7120mr18196156c88.3.1791922100434; Tue, 13 Oct 2026 13:08:20 -0700 (PDT) Received: from localhost.localdomain ([192.19.203.250]) by smtp.gmail.com with ESMTPSA id a92af1059eb24-169a31f6f5csm39526585c88.11.2026.10.13.13.08.15 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 13 Oct 2026 13:08:19 -0700 (PDT) From: Bhargava Marreddy To: davem@davemloft.net, edumazet@google.com, kuba@kernel.org, pabeni@redhat.com, andrew+netdev@lunn.ch, horms@kernel.org Cc: netdev@vger.kernel.org, linux-kernel@vger.kernel.org, michael.chan@broadcom.com, pavan.chebbi@broadcom.com, vsrama-krishna.nemani@broadcom.com, vikas.gupta@broadcom.com, Bhargava Marreddy , Dharmender Garg , Rajashekar Hudumula , Ramakrishna Koduri Subject: [PATCH net-next v3 4/5] bnge: make irq_tbl layout depend on BNGE_FIRST_OPEN_NQ Date: Wed, 14 Oct 2026 01:36:44 +0530 Message-ID: <20261013200645.1346299-5-bhargava.marreddy@broadcom.com> X-Mailer: git-send-email 2.47.3 In-Reply-To: <20261013200645.1346299-1-bhargava.marreddy@broadcom.com> References: <20261013200645.1346299-1-bhargava.marreddy@broadcom.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-DetectorID-Processed: b00c1d49-9d2e-4205-b15f-d015386d3d5e Content-Type: text/plain; charset="utf-8" NQ0 will be created at probe, before the aux (RoCE) device has been given its MSI-X vectors, and must keep its vector for the lifetime of the driver. So the probe NQs must come first in irq_tbl, followed by the aux vectors and then the open NQs, instead of putting all NQs after the aux vectors. Describe the irq_tbl layout once, next to BNGE_FIRST_OPEN_NQ, and add bnge_nq_irq_idx() and bnge_aux_irq_idx() to compute the irq_tbl index of an NQ and of an aux vector from it. bnge_fill_msix_vecs() uses the latter to hand the aux device the vectors that follow the probe NQs. The ring_idx passed to the aux device changes meaning accordingly: it is the irq_tbl index of the vector, not an index within the aux range. This matches how NQ rings are created, where the ring index is used as the logical ID that ties the ring to its doorbell and MSI-X vector. The aux driver uses the values it is given, so it needs no change. The total number of vectors is unchanged. BNGE_FIRST_OPEN_NQ is 0 for now, so the layout is unchanged. No functional change. Signed-off-by: Bhargava Marreddy Reviewed-by: Dharmender Garg Reviewed-by: Rajashekar Hudumula Reviewed-by: Ramakrishna Koduri --- drivers/net/ethernet/broadcom/bnge/bnge.h | 19 +++++++++++++++++++ .../net/ethernet/broadcom/bnge/bnge_auxr.c | 4 ++-- .../net/ethernet/broadcom/bnge/bnge_netdev.c | 2 +- 3 files changed, 22 insertions(+), 3 deletions(-) diff --git a/drivers/net/ethernet/broadcom/bnge/bnge.h b/drivers/net/ethern= et/broadcom/bnge/bnge.h index fdd8bca01440..13452caa9081 100644 --- a/drivers/net/ethernet/broadcom/bnge/bnge.h +++ b/drivers/net/ethernet/broadcom/bnge/bnge.h @@ -241,6 +241,10 @@ struct bnge_dev { /* Index of the first NQ created in ndo_open() and destroyed in ndo_stop(). * NQs below it are created at probe and destroyed at remove. It starts at= 0, * i.e. all NQs are open NQs, until NQ0 moves to probe. + * + * Layout of bd->irq_tbl: the NQs created at probe (none yet) come first, + * then the aux (RoCE) vectors, then the NQs created in open. The ring_idx + * the aux device receives is its irq_tbl index. */ #define BNGE_FIRST_OPEN_NQ 0 =20 @@ -290,4 +294,19 @@ static inline void bnge_db_write(struct bnge_dev *bd, = struct bnge_db_info *db, bool bnge_aux_registered(struct bnge_dev *bd); u16 bnge_aux_get_msix(struct bnge_dev *bd); =20 +/* irq_tbl index of the i-th aux vector, also used as its ring_idx */ +static inline int bnge_aux_irq_idx(int i) +{ + return BNGE_FIRST_OPEN_NQ + i; +} + +/* irq_tbl index of NQ nq_idx */ +static inline int bnge_nq_irq_idx(struct bnge_dev *bd, int nq_idx) +{ + if (nq_idx < BNGE_FIRST_OPEN_NQ) + return nq_idx; + + return bnge_aux_get_msix(bd) + nq_idx; +} + #endif /* _BNGE_H_ */ diff --git a/drivers/net/ethernet/broadcom/bnge/bnge_auxr.c b/drivers/net/e= thernet/broadcom/bnge/bnge_auxr.c index 0955b488b6fe..0393bfea7b65 100644 --- a/drivers/net/ethernet/broadcom/bnge/bnge_auxr.c +++ b/drivers/net/ethernet/broadcom/bnge/bnge_auxr.c @@ -34,9 +34,9 @@ static void bnge_fill_msix_vecs(struct bnge_dev *bd, } num_msix =3D auxr_dev->auxr_info->msix_requested; for (i =3D 0; i < num_msix; i++) { - info[i].vector =3D bd->irq_tbl[i].vector; + info[i].vector =3D bd->irq_tbl[bnge_aux_irq_idx(i)].vector; info[i].db_offset =3D bd->db_offset; - info[i].ring_idx =3D i; + info[i].ring_idx =3D bnge_aux_irq_idx(i); } } =20 diff --git a/drivers/net/ethernet/broadcom/bnge/bnge_netdev.c b/drivers/net= /ethernet/broadcom/bnge/bnge_netdev.c index 0d98223cd703..dc6cf3525034 100644 --- a/drivers/net/ethernet/broadcom/bnge/bnge_netdev.c +++ b/drivers/net/ethernet/broadcom/bnge/bnge_netdev.c @@ -539,7 +539,7 @@ static int bnge_alloc_nq(struct bnge_net *bn, struct bn= ge_napi *bnapi) } =20 ring->fw_ring_id =3D INVALID_HW_RING_ID_32BIT; - ring->map_idx =3D bnge_aux_get_msix(bn->bd) + bnapi->index; + ring->map_idx =3D bnge_nq_irq_idx(bn->bd, bnapi->index); return 0; } =20 --=20 2.47.3 From nobody Sat Oct 24 13:21:15 2026 Received: from mail-qv1-f99.google.com (mail-qv1-f99.google.com [209.85.219.99]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 742505476D8 for ; Tue, 13 Oct 2026 20:08:30 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.219.99 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1791922113; cv=none; b=WFSlTxwGAF2ULqhUT8oM/b5M1Bf9Vu/7e0wY9M4G7OaQZ6xo24kVgfHKnNfxK3jD2EHfpOLCXM/Av/HUq+1tY7oH74/LiLpY8lmTBpmgZQjXmNyH6yW+nNo3+tUaC6KIN3mQailtZcqLHLEt/jea6yikMQwKigFdzY2HIjEZo9E= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1791922113; c=relaxed/simple; bh=nkCXaHyulsci4+25FlzL1lifaVujHsBgtU1uXH+HSjM=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=F5Pnq9pU1pyhu7L4SsTZK/WrM/ugdN1D6E6IG8MTV0PTzDMV+aNLMia906FQ2DXq6EmjI9rI4zHoA7RV/XEzk2f9JtKcZCZesjLtS60ON5d1GeWIa8l+0QOjDEgOT5LiqrbZ+Gqgl8pUgSeHn2RjR4JZF1y4fsErRcB/UasGTu0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=broadcom.com; spf=fail smtp.mailfrom=broadcom.com; dkim=pass (1024-bit key) header.d=broadcom.com header.i=@broadcom.com header.b=CAli1DF+; arc=none smtp.client-ip=209.85.219.99 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=broadcom.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=broadcom.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=broadcom.com header.i=@broadcom.com header.b="CAli1DF+" Received: by mail-qv1-f99.google.com with SMTP id 6a1803df08f44-917ad52d7b6so40662496d6.2 for ; Tue, 13 Oct 2026 13:08:30 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20260707; t=1791922109; x=1792526909; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:dkim-signature:x-gm-gg :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to :content-type; bh=HPTAkjLklbTZeRakkGsbkrTRsLZf0cNlseZNQYjTzSM=; b=ndeYV56g2FaRrao2nGiKL0XSHyksRkFB9CTMbCQslcF07NE3W6uhAcWqIWTE+0yaO7 zjLBCvPAvKobYo1VnKWfDe9k4L8Z5kGWhfTzl4klAF8spwwbC1hxYW9q9Lx7baxW0QHy LygIjcWQpGX3pwAs5iB3UJLVuPUZvPdMqkkG7t2OCVhxmu+pDPQwsIb9ZkeVEFC1b8DX Y0vg5NUImhqgPdYAi9rcLpBY7lqax3RhSMVmGFUCEONhanIbNYNJld9XLCPyl5kHffnr FwfEKhBK3DkqyhtOFzhyRVkX8MQ0ADtAelHQhqgcTJvVnVa7ZG5gK6ORwvTuMPq41QYx Vg3w== X-Forwarded-Encrypted: i=1; AKwUvBxSveu+yQtC963Zx+VeUcxBfTM6TT48b1D7loUiF3KcCG9qSRBsN9zfy4VzHLL0MOM0n2XN/ivEKa0Nyd0=@vger.kernel.org X-Gm-Message-State: AFq9FYKSogKvrWEugWIh0x8Nv8upqgnlaFLCDeRmMYVbts9QpW/IFMZz KC8m8CqQfXH2scYzxoA1m0ZHjSQRZ6NtYC3XjGdlWhyfYIFhwickSBBv5s9UhLPx9sY8E9f0NK5 avdWJuwejfEDG3EektwUKF6g6ujk2t0iOvMDPw7YNPClEyFWnONnQAiAHUobc+T5v2OxQNWM2Ci QxHjzKx36TLU6vKCBUKBYXumBE9EgsGACPuphK6VeTxhv81+GeOAWK5EP7ys17raZraAsO8cqnu 2MV7sI/QVwoTOIvBy4TvDj8jCnzqfM= X-Gm-Gg: AYBFou0JtanqrGXw3ViqnZPvfKMwSIVjwTHSEVDplTkEkM2p2nIyd3nKqqUjP4HU9OE dHAzS2rnN7B83jPzoDAYh9E//MmjNu4S/9fh2ayXzRBwo5+yZuxP26/nmAnt9iS2+206zxvuEJ1 sgOL5O1gLcqOLFjnJvgrCuvJPwkoj45vcz2q68nP50anQvDZtWDfxRlo2kTKWkzT9tAPRbNFbSW VwnrfwBENkJ7qah7tQnijX0Ig0T7+103lmH2bZ3KnBx58gwdk/d/DdKJkRp5lv6zLNco6/RlatF l7Q1mpA22IKzYWTQRNFl/VvEVHIPAVRhIklwztq2P9W89ML+z06k1b5XGPJhyUNogsCLL7F0JXf hEGJQDqw46yyjlsbYoToR1p8eW0PNpyC1MdDHqq8za+noUe7f4GvjqzhbZPNmkOsaM77wFDKnZO gF5S9v0zl+1X/1OHqBo/Nx7i4yHasIyP/lrLXDIQBWb5WF X-Received: by 2002:a05:620a:a182:20b0:93e:c12c:9b78 with SMTP id af79cd13be357-93ec12c9f48mr1540182085a.83.1791922109009; Tue, 13 Oct 2026 13:08:29 -0700 (PDT) Received: from smtp-us-east1-p01-i01-si01.dlp.protect.broadcom.com (address-144-49-247-120.dlp.protect.broadcom.com. [144.49.247.120]) by smtp-relay.gmail.com with ESMTPS id af79cd13be357-93ef4d33985sm83530385a.0.2026.10.13.13.08.28 for (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Tue, 13 Oct 2026 13:08:29 -0700 (PDT) X-Relaying-Domain: broadcom.com X-CFilter-Loop: Reflected Received: by mail-dy1-f197.google.com with SMTP id 5a478bee46e88-359b81ad4e8so362358eec.0 for ; Tue, 13 Oct 2026 13:08:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=broadcom.com; s=google; t=1791922108; x=1792526908; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=HPTAkjLklbTZeRakkGsbkrTRsLZf0cNlseZNQYjTzSM=; b=CAli1DF+oqZyT5qZ27fa54EO7fB54SA/7KiNP//j84RoGP6gWviGYyaNwjCQ2FsYJF AUn4W97c+rz1WoXUoGVJgvUZ7nPwwxdrcUzg4Z/0bPdvLVcHUMpDndbxjPrc3AIkDbc8 pauCaI9doLQplw53a2Tz78AwDPmwIz/bCgMYQ= X-Forwarded-Encrypted: i=1; AKwUvBwx4r6OPLaQHm95HlWzGerBXyhPxk1KUyDOmEdMaFwLMqyKspGPAxspVslXQyVlOJOaCyLr91vxeny4miI=@vger.kernel.org X-Received: by 2002:a05:701b:2910:b0:14a:a6dc:9ca6 with SMTP id a92af1059eb24-16a5888bf17mr16919540c88.3.1791922107153; Tue, 13 Oct 2026 13:08:27 -0700 (PDT) X-Received: by 2002:a05:701b:2910:b0:14a:a6dc:9ca6 with SMTP id a92af1059eb24-16a5888bf17mr16919470c88.3.1791922105865; Tue, 13 Oct 2026 13:08:25 -0700 (PDT) Received: from localhost.localdomain ([192.19.203.250]) by smtp.gmail.com with ESMTPSA id a92af1059eb24-169a31f6f5csm39526585c88.11.2026.10.13.13.08.20 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 13 Oct 2026 13:08:25 -0700 (PDT) From: Bhargava Marreddy To: davem@davemloft.net, edumazet@google.com, kuba@kernel.org, pabeni@redhat.com, andrew+netdev@lunn.ch, horms@kernel.org Cc: netdev@vger.kernel.org, linux-kernel@vger.kernel.org, michael.chan@broadcom.com, pavan.chebbi@broadcom.com, vsrama-krishna.nemani@broadcom.com, vikas.gupta@broadcom.com, Bhargava Marreddy , Dharmender Garg , Rajashekar Hudumula , Ramakrishna Koduri Subject: [PATCH net-next v3 5/5] bnge: make NQ0 persistent across open/stop cycles Date: Wed, 14 Oct 2026 01:36:45 +0530 Message-ID: <20261013200645.1346299-6-bhargava.marreddy@broadcom.com> X-Mailer: git-send-email 2.47.3 In-Reply-To: <20261013200645.1346299-1-bhargava.marreddy@broadcom.com> References: <20261013200645.1346299-1-bhargava.marreddy@broadcom.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-DetectorID-Processed: b00c1d49-9d2e-4205-b15f-d015386d3d5e Content-Type: text/plain; charset="utf-8" NQ0 is currently created in ndo_open(), so firmware async events are not received while the interface is down. This change lets NQ0 receive async events in the down state, but sp_task bails out unless BNGE_STATE_OPEN is set. Down-state event processing is left to subsequent patches. Focus this patch strictly on making NQ0 persistent across ndo_stop/ndo_open cycles: - probe: Allocate NQ0, register its NAPI, request its IRQ, allocate the HWRM ring, configure it as the async event completion ring (a failure is only warned about, as before), then enable its NAPI and arm it. - open: Allocate/create NQ1..N and attach queue 0's RX/TX and completion rings to NQ0. - close: Destroy NQ1..N and detach rings from NQ0, leaving NQ0 to handle async events only. - remove: Destroy NQ0 before the timer and workqueue, as NQ0 queues sp_task on async events. NQ0's NAPI keeps running while the interface is down, so quiesce it while the host ring structures (Rx, Tx, etc.) are allocated on open and freed on close. Async events NQ0 receives while down are ignored by sp_task, so open also clears sp_event while NQ0 is quiesced, and resumes NQ0 only after BNGE_STATE_OPEN is set, together with the other NQs. Introduce bnge_nq_drain() to drop pending NQEs on NQ0 once the completion rings they refer to are gone, while still handling HWRM and async event completions. Call it with NQ0's NAPI disabled. For the same reason, bnge_napi_poll() now ignores a CQ notification whose index is beyond the current completion ring count, instead of indexing the freed cp_ring_arr. Only the NQ, its IRQ and its NAPI persist. NQ0 is skipped, via bnge_for_each_open_nq(), when enabling NQ interrupts, adding/deleting and enabling/disabling NAPI, requesting/freeing IRQs, naming IRQs, allocating/freeing NQ rings and the firmware NQ, and its firmware ring ID is carried over in grp_info[0]. Its completion rings, cp_ring_arr and stats remain per-open. Give NQ0 a stable -nq0 IRQ name and retain the existing names for NQs created during open. As a result, the vector serving queue 0 is listed in /proc/interrupts under the -nq0 name instead of -TxRx-0, and stays requested while the interface is down. Set BNGE_FIRST_OPEN_NQ to 1, shifting aux (RoCE) vectors to irq_tbl[1..] as prepared by earlier patches. The async event completion ring is now configured once at probe rather than on every open. This relies on firmware keeping NQ0 allocated, and the async event completion ring configured, across FUNC_DRV_IF_CHANGE down/up. Recreating NQ0 after a firmware reset is not handled here. Signed-off-by: Bhargava Marreddy Reviewed-by: Dharmender Garg Reviewed-by: Rajashekar Hudumula Reviewed-by: Ramakrishna Koduri --- drivers/net/ethernet/broadcom/bnge/bnge.h | 13 +- .../net/ethernet/broadcom/bnge/bnge_netdev.c | 185 ++++++++++++++++-- .../net/ethernet/broadcom/bnge/bnge_netdev.h | 2 + .../net/ethernet/broadcom/bnge/bnge_txrx.c | 42 ++++ .../net/ethernet/broadcom/bnge/bnge_txrx.h | 1 + 5 files changed, 225 insertions(+), 18 deletions(-) diff --git a/drivers/net/ethernet/broadcom/bnge/bnge.h b/drivers/net/ethern= et/broadcom/bnge/bnge.h index 13452caa9081..950f40d69916 100644 --- a/drivers/net/ethernet/broadcom/bnge/bnge.h +++ b/drivers/net/ethernet/broadcom/bnge/bnge.h @@ -239,14 +239,15 @@ struct bnge_dev { #define bnge_dbg(bd, fmt, ...) bnge_log(dbg, bd, fmt, ##__VA_ARGS__) =20 /* Index of the first NQ created in ndo_open() and destroyed in ndo_stop(). - * NQs below it are created at probe and destroyed at remove. It starts at= 0, - * i.e. all NQs are open NQs, until NQ0 moves to probe. + * NQs below it are created at probe and destroyed at remove. Exactly one = NQ, + * NQ0 (async events and queue 0), is created at probe. * - * Layout of bd->irq_tbl: the NQs created at probe (none yet) come first, - * then the aux (RoCE) vectors, then the NQs created in open. The ring_idx - * the aux device receives is its irq_tbl index. + * Layout of bd->irq_tbl, with A =3D aux MSI-X count: NQ0 uses irq_tbl[0], + * the aux (RoCE) device irq_tbl[1..A], and the NQs created in open + * irq_tbl[A + 1..]. The ring_idx the aux device receives is its irq_tbl + * index. */ -#define BNGE_FIRST_OPEN_NQ 0 +#define BNGE_FIRST_OPEN_NQ 1 =20 #define bnge_for_each_open_nq(i, bd) \ for ((i) =3D BNGE_FIRST_OPEN_NQ; (i) < (bd)->nq_nr_rings; (i)++) diff --git a/drivers/net/ethernet/broadcom/bnge/bnge_netdev.c b/drivers/net= /ethernet/broadcom/bnge/bnge_netdev.c index dc6cf3525034..c6649a2c5876 100644 --- a/drivers/net/ethernet/broadcom/bnge/bnge_netdev.c +++ b/drivers/net/ethernet/broadcom/bnge/bnge_netdev.c @@ -1209,10 +1209,20 @@ static int bnge_init_ring_grps(struct bnge_net *bn) bn->grp_info[i].agg_fw_ring_id =3D INVALID_HW_RING_ID; bn->grp_info[i].nq_fw_ring_id =3D INVALID_HW_RING_ID; } + /* NQ0 stays allocated in FW across open/close */ + bn->grp_info[0].nq_fw_ring_id =3D + (u16)bn->nq0->nq_ring.ring_struct.fw_ring_id; =20 return 0; } =20 +static void bnge_nq0_clear_rings(struct bnge_net *bn) +{ + bn->nq0->rx_ring =3D NULL; + memset(bn->nq0->tx_ring, 0, sizeof(bn->nq0->tx_ring)); + bn->nq0->events =3D 0; +} + static void bnge_free_core(struct bnge_net *bn) { bnge_free_vnic_attributes(bn); @@ -1229,6 +1239,7 @@ static void bnge_free_core(struct bnge_net *bn) bn->tx_ring =3D NULL; kfree(bn->rx_ring); bn->rx_ring =3D NULL; + bnge_nq0_clear_rings(bn); kfree(bn->bnapi); bn->bnapi =3D NULL; } @@ -1243,12 +1254,15 @@ static int bnge_alloc_core(struct bnge_net *bn) arr_size =3D L1_CACHE_ALIGN(sizeof(struct bnge_napi *) * bd->nq_nr_rings); size =3D L1_CACHE_ALIGN(sizeof(struct bnge_napi)); - bnapi =3D kzalloc(arr_size + size * bd->nq_nr_rings, GFP_KERNEL); + bnapi =3D kzalloc(arr_size + + size * (bd->nq_nr_rings - BNGE_FIRST_OPEN_NQ), + GFP_KERNEL); if (!bnapi) return rc; =20 bn->bnapi =3D bnapi; bnapi +=3D arr_size; + bn->bnapi[0] =3D bn->nq0; bnge_for_each_open_nq(i, bd) { bn->bnapi[i] =3D bnapi; bn->bnapi[i]->index =3D i; @@ -1360,6 +1374,7 @@ static int bnge_alloc_core(struct bnge_net *bn) kfree(bn->rx_ring); bn->rx_ring =3D NULL; err_free_bnapi: + bnge_nq0_clear_rings(bn); kfree(bn->bnapi); bn->bnapi =3D NULL; return rc; @@ -1973,18 +1988,12 @@ static int bnge_hwrm_ring_alloc(struct bnge_net *bn) agg_rings =3D !!(bnge_is_agg_reqd(bd)); bnge_for_each_open_nq(i, bd) { struct bnge_napi *bnapi =3D bn->bnapi[i]; - struct bnge_ring_struct *ring =3D &bnapi->nq_ring.ring_struct; =20 rc =3D bnge_hwrm_nq_alloc(bn, bnapi); if (rc) goto err_out; - bn->grp_info[i].nq_fw_ring_id =3D (u16)ring->fw_ring_id; - - if (!i) { - rc =3D bnge_hwrm_set_async_event_cr(bd, ring->fw_ring_id); - if (rc) - netdev_warn(bn->netdev, "Failed to set async event completion ring.\n"= ); - } + bn->grp_info[i].nq_fw_ring_id =3D + (u16)bnapi->nq_ring.ring_struct.fw_ring_id; } =20 for (i =3D 0; i < bd->tx_nr_rings; i++) { @@ -2625,6 +2634,9 @@ static void bnge_setup_msix(struct bnge_net *bn) int len, i; =20 len =3D sizeof(bd->irq_tbl[0].name); + /* NQ0 keeps the "-nq0" name set in bnge_nq0_init(). Only the + * NQs created in open are named after the netdev here. + */ bnge_for_each_open_nq(i, bd) { int map_idx =3D bnge_cp_num_to_irq_num(bn, i); char *attr; @@ -2870,6 +2882,129 @@ static void bnge_del_napi(struct bnge_net *bn) synchronize_net(); } =20 +static void bnge_nq0_quiesce(struct bnge_net *bn) +{ + napi_disable_locked(&bn->nq0->napi); + /* Wait for RCU grace period after napi_disable_locked(): a poll can + * still be arming the NQ doorbell after it clears NAPI_STATE_SCHED. + */ + synchronize_net(); + bnge_db_nq(bn, &bn->nq0->nq_ring.nq_db, bn->nq0->nq_ring.nq_raw_cons); +} + +static void bnge_nq0_resume(struct bnge_net *bn) +{ + struct bnge_nq_ring_info *nqr =3D &bn->nq0->nq_ring; + + bn->nq0->in_reset =3D false; + bn->nq0->tx_fault =3D 0; + napi_enable_locked(&bn->nq0->napi); + bnge_db_nq_arm(bn, &nqr->nq_db, nqr->nq_raw_cons); +} + +static int bnge_nq0_init(struct bnge_net *bn) +{ + struct net_device *dev =3D bn->netdev; + struct bnge_nq_ring_info *nqr; + struct bnge_dev *bd =3D bn->bd; + struct bnge_napi *bnapi; + struct bnge_irq *irq; + int rc; + + bnapi =3D kzalloc_obj(*bnapi); + if (!bnapi) + return -ENOMEM; + + /* NQ0 helpers hard-code NQ index 0 */ + BUILD_BUG_ON(BNGE_FIRST_OPEN_NQ !=3D 1); + + bnapi->bn =3D bn; + bnapi->index =3D 0; + bn->nq0 =3D bnapi; + nqr =3D &bnapi->nq_ring; + + rc =3D bnge_alloc_nq(bn, bnapi); + if (rc) + goto err_free_bnapi; + + irq =3D &bd->irq_tbl[nqr->ring_struct.map_idx]; + irq->handler =3D bnge_msix; + snprintf(irq->name, sizeof(irq->name), "%s-nq0", pci_name(bd->pdev)); + + /* The netdev is not registered yet, but the *_locked() NAPI helpers + * expect the instance lock to be held. + */ + netdev_lock(dev); + netif_napi_add_config_locked(dev, &bnapi->napi, bnge_napi_poll, + bnapi->index); + + rc =3D bnge_nq_request_irq(bn, bnapi); + if (rc) + goto err_del_napi; + + rc =3D bnge_hwrm_nq_alloc(bn, bnapi); + if (rc) + goto err_free_irq; + + rc =3D bnge_hwrm_set_async_event_cr(bd, nqr->ring_struct.fw_ring_id); + if (rc) { + bnge_err(bd, "Failed to set async event completion ring: %d\n", + rc); + goto err_free_irq; + } + + /* Enable NAPI only now that the NQ doorbell is set up, so an early + * interrupt cannot make the poll arm an uninitialized doorbell. + */ + napi_enable_locked(&bnapi->napi); + bnge_db_nq_arm(bn, &nqr->nq_db, nqr->nq_raw_cons); + netdev_unlock(dev); + return 0; + +err_free_irq: + bnge_nq_free_irq(bn, bnapi); + __netif_napi_del_locked(&bnapi->napi); + netdev_unlock(dev); + /* Same order as bnge_nq0_uninit(): no poll may touch the NQ once the + * FW ring is freed. + */ + synchronize_net(); + bnge_hwrm_nq_free(bn, bnapi); + bnge_free_nq(bn, bnapi); + goto err_free_bnapi; +err_del_napi: + __netif_napi_del_locked(&bnapi->napi); + netdev_unlock(dev); + synchronize_net(); + bnge_free_nq(bn, bnapi); +err_free_bnapi: + kfree(bnapi); + bn->nq0 =3D NULL; + return rc; +} + +static void bnge_nq0_uninit(struct bnge_net *bn) +{ + struct net_device *dev =3D bn->netdev; + struct bnge_napi *bnapi =3D bn->nq0; + + netdev_lock(dev); + napi_disable_locked(&bnapi->napi); + bnge_nq_free_irq(bn, bnapi); + __netif_napi_del_locked(&bnapi->napi); + netdev_unlock(dev); + /* Wait for RCU grace period after removing the NAPI instance, and for + * a poll that may still be arming the NQ doorbell, before destroying + * the FW ring. + */ + synchronize_net(); + + bnge_hwrm_nq_free(bn, bnapi); + bnge_free_nq(bn, bnapi); + kfree(bnapi); + bn->nq0 =3D NULL; +} + static int bnge_init_nic(struct bnge_net *bn) { int rc; @@ -2976,10 +3111,17 @@ static int bnge_open_core(struct bnge_net *bn) goto err_if_change; } =20 + bnge_nq0_quiesce(bn); + + /* Drop async events NQ0 received while down. Its NAPI is quiesced, so + * nothing can set bits concurrently. + */ + WRITE_ONCE(bn->sp_event, 0); + rc =3D bnge_alloc_core(bn); if (rc) { netdev_err(bn->netdev, "bnge_alloc_core err: %d\n", rc); - goto err_if_change; + goto err_nq0_resume; } =20 bnge_init_napi(bn); @@ -3011,6 +3153,8 @@ static int bnge_open_core(struct bnge_net *bn) set_bit(BNGE_STATE_STATS_ENABLE, &bn->state); =20 bnge_enable_int(bn); + /* Resume NQ0 with the other NQs, once BNGE_STATE_OPEN is set */ + bnge_nq0_resume(bn); =20 bnge_tx_enable(bn); =20 @@ -3022,10 +3166,13 @@ static int bnge_open_core(struct bnge_net *bn) return 0; =20 err_free_irq: + bnge_nq_drain(bn, bn->nq0); bnge_free_irq(bn); err_del_napi: bnge_del_napi(bn); bnge_free_core(bn); +err_nq0_resume: + bnge_nq0_resume(bn); err_if_change: bnge_hwrm_if_change(bd, false); return rc; @@ -3238,6 +3385,8 @@ static void bnge_close_core(struct bnge_net *bn) timer_delete_sync(&bn->timer); bnge_shutdown_nic(bn); bnge_disable_napi(bn); + bnge_nq0_quiesce(bn); + bnge_nq_drain(bn, bn->nq0); =20 /* Save ring stats before shutdown */ spin_lock_bh(&bn->stats_lock); @@ -3250,6 +3399,7 @@ static void bnge_close_core(struct bnge_net *bn) bnge_del_napi(bn); =20 bnge_free_core(bn); + bnge_nq0_resume(bn); } =20 static int bnge_close(struct net_device *dev) @@ -3259,7 +3409,6 @@ static int bnge_close(struct net_device *dev) bnge_close_core(bn); bnge_hwrm_shutdown_link(bn->bd); bnge_hwrm_if_change(bn->bd, false); - bn->sp_event =3D 0; =20 return 0; } @@ -3593,14 +3742,23 @@ int bnge_netdev_alloc(struct bnge_dev *bd, int max_= irqs) spin_lock_init(&bn->stats_lock); =20 netdev->request_ops_lock =3D true; + + rc =3D bnge_nq0_init(bn); + if (rc) { + dev_err(bd->dev, "NQ0 init failed rc: %d\n", rc); + goto err_free_port_stats; + } + rc =3D register_netdev(netdev); if (rc) { dev_err(bd->dev, "Register netdev failed rc: %d\n", rc); - goto err_free_port_stats; + goto err_uninit_nq0; } =20 return 0; =20 +err_uninit_nq0: + bnge_nq0_uninit(bn); err_free_port_stats: bnge_free_port_stats(bn); err_free_workq: @@ -3620,6 +3778,9 @@ void bnge_netdev_free(struct bnge_dev *bd) =20 unregister_netdev(netdev); =20 + /* NQ0 can queue sp_task; destroy it before the timer and workqueue */ + bnge_nq0_uninit(bn); + timer_shutdown_sync(&bn->timer); cancel_work_sync(&bn->sp_task); bn->sp_event =3D 0; diff --git a/drivers/net/ethernet/broadcom/bnge/bnge_netdev.h b/drivers/net= /ethernet/broadcom/bnge/bnge_netdev.h index 476b5bab96fe..118a2e4f5331 100644 --- a/drivers/net/ethernet/broadcom/bnge/bnge_netdev.h +++ b/drivers/net/ethernet/broadcom/bnge/bnge_netdev.h @@ -315,6 +315,8 @@ struct bnge_net { u32 hash_seed; u64 toeplitz_prefix; =20 + /* Lives from probe to remove. bnapi[0] points to it. */ + struct bnge_napi *nq0; struct bnge_napi **bnapi; =20 struct bnge_rx_ring_info *rx_ring; diff --git a/drivers/net/ethernet/broadcom/bnge/bnge_txrx.c b/drivers/net/e= thernet/broadcom/bnge/bnge_txrx.c index 2f382514c6ea..9324ee889034 100644 --- a/drivers/net/ethernet/broadcom/bnge/bnge_txrx.c +++ b/drivers/net/ethernet/broadcom/bnge/bnge_txrx.c @@ -1375,6 +1375,13 @@ int bnge_napi_poll(struct napi_struct *napi, int bud= get) break; =20 idx =3D BNGE_NQ_HDL_IDX(idx); + /* NQ0 outlives its completion rings: ignore a stale + * notification for a CQ that no longer exists. + */ + if (unlikely(idx >=3D nqr->cp_ring_count)) { + raw_cons =3D NEXT_RAW_CMP(raw_cons); + continue; + } cpr =3D &nqr->cp_ring_arr[idx]; cpr->had_nqe_notify =3D 1; cpr->toggle =3D NQE_CN_TOGGLE(type); @@ -1396,6 +1403,41 @@ int bnge_napi_poll(struct napi_struct *napi, int bud= get) return work_done; } =20 +/* Consume pending NQ entries without polling the CP rings. The NQ's NAPI = must + * be disabled (or quiesced) by the caller. HWRM events are handled, CQ + * notifications are dropped. cpr->had_nqe_notify is not reset, so the CP = rings + * must be freed or re-initialised afterwards. + */ +void bnge_nq_drain(struct bnge_net *bn, struct bnge_napi *bnapi) +{ + struct bnge_nq_ring_info *nqr =3D &bnapi->nq_ring; + u32 raw_cons =3D nqr->nq_raw_cons; + + while (1) { + u32 cons =3D RING_CMP(bn, raw_cons); + struct nqe_cn *nqcmp; + u16 type; + + nqcmp =3D &nqr->desc_ring[CP_RING(cons)][CP_IDX(cons)]; + if (!NQ_CMP_VALID(bn, nqcmp, raw_cons)) + break; + + /* The valid test of the entry must be done first before + * reading any further. + */ + dma_rmb(); + + type =3D le16_to_cpu(nqcmp->type); + if (NQE_CN_TYPE(type) !=3D NQ_CN_TYPE_CQ_NOTIFICATION) + bnge_hwrm_handler(bn, (struct tx_cmp *)nqcmp); + raw_cons =3D NEXT_RAW_CMP(raw_cons); + } + + nqr->has_more_work =3D 0; + nqr->nq_raw_cons =3D raw_cons; + BNGE_DB_NQ(bn->bd, &nqr->nq_db, raw_cons); +} + static u16 bnge_xmit_get_cfa_action(struct sk_buff *skb) { struct metadata_dst *md_dst =3D skb_metadata_dst(skb); diff --git a/drivers/net/ethernet/broadcom/bnge/bnge_txrx.h b/drivers/net/e= thernet/broadcom/bnge/bnge_txrx.h index bd0aa6c221a4..5779b78fb6a2 100644 --- a/drivers/net/ethernet/broadcom/bnge/bnge_txrx.h +++ b/drivers/net/ethernet/broadcom/bnge/bnge_txrx.h @@ -120,6 +120,7 @@ irqreturn_t bnge_msix(int irq, void *dev_instance); netdev_tx_t bnge_start_xmit(struct sk_buff *skb, struct net_device *dev); void bnge_reuse_rx_data(struct bnge_rx_ring_info *rxr, u16 cons, void *dat= a); int bnge_napi_poll(struct napi_struct *napi, int budget); +void bnge_nq_drain(struct bnge_net *bn, struct bnge_napi *bnapi); netdev_features_t bnge_features_check(struct sk_buff *skb, struct net_device *dev, netdev_features_t features); --=20 2.47.3