From nobody Sat Feb 7 06:20:57 2026 Received: from mail-pl1-f227.google.com (mail-pl1-f227.google.com [209.85.214.227]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5DDDC36CC9D for ; Thu, 11 Sep 2025 19:35:59 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.227 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1757619361; cv=none; b=Uyx59LvEGpQb+FW9LVufsWnmNXNW5cN5jiIfP2g7/twN9snUeP6YeNw0jzlAqtImr9Rvs6ibOmdm81XFCOP5hGI91TWy624QyEPhoyTMbcJbJGiE7SGxhqUMWsDxbgsQS4oWAWeHbGX1gaLOSBXETexprA0JMgJAThdUU50o2OA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1757619361; c=relaxed/simple; bh=LrxC4ZNVNmSq4ZS92qwzzTLyemLLlV4RH3+E8FX/Nq4=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=Vy5t4yqU0Y6Ifb75Zci7uefMgBQMEkdhJMr1h1CDbsBkbEjZi4Nad+4YCqzed4acMYfUs6lsrMSQWF4vJkKpI+g1qkF08pD985DIjLyruDmjq0hjGjhjcesW/4tsTI6trAChiboH4P9opTscMn0Urq7gt8mKAR8CLSob6IuOxeo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=broadcom.com; spf=fail smtp.mailfrom=broadcom.com; dkim=pass (1024-bit key) header.d=broadcom.com header.i=@broadcom.com header.b=EKs0wQgQ; arc=none smtp.client-ip=209.85.214.227 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=broadcom.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=broadcom.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=broadcom.com header.i=@broadcom.com header.b="EKs0wQgQ" Received: by mail-pl1-f227.google.com with SMTP id d9443c01a7336-2570bf605b1so10856505ad.2 for ; Thu, 11 Sep 2025 12:35:59 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1757619358; x=1758224158; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:dkim-signature :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=imG5pN0GTv13vxSJjHPy14bFiRuvwF0ZssLFfjreBEc=; b=RqUtfQb4bZcwYXP/CYbaNGxokLW4zURtmr36PSpqFwVG25kytvkVW9mBUbSb+XKud+ 1WWsX+v7O1xHeLcI46aLotIkHAiG5ViDNEzhtKFMgcVq5W7e30jRKw4Perb5Dey/7gj0 XtHzCIzqlrtOiDGQSaGktzxEjJvRcGqJnIfRpHAG0JSMBwLL1dwYDJvNQ7uabo8v+QZz daSKuUD2ViH1gxluDhpLRF13uTLQTcKMmU4Hxt5kyLR/bRoTbBg4NdeiNwrcX4lyW4Kz QfJgw70voibgDvOBhKcap4b83sEKaJnE+QYb8G8v5L6c/6g4xtbHpuOEhWXyWCl6Dx9z Md3A== X-Forwarded-Encrypted: i=1; AJvYcCXv9VGS6N+A86QC5x1sVgRbSkOtIfACvKl2jEIV5Zjpy4O26X+gLYumvDBt+YskZu/04ESuB/IWaa3lk5o=@vger.kernel.org X-Gm-Message-State: AOJu0YyEI+sBxvsO4nsQBmx8i/2f8i7OdWA+8TkFhicQzIpoIIvp2F9O 65GKafnxywzNXjKT5eByTh4lJePv8IVeIuVSIOife4tNh9o+bHli18pamgwG2at0y5X2eUDtFNl 6eVsb0XQQvVTbPtF4ZUnta+ffkj1/4fWnj/8H0qk6sC6I1uvwUGI549wlTORQqkEkAAbxPgIjyk PBJkwaQCtAyvqsqKV/lMVNZo86wfs4fqjvwZR3379xmLfr4J74VTZrb/IyFL4rF8QoPKRa1OlOq 4mRhHaFRWfnxcPJcsmuQ6jzHs3J X-Gm-Gg: ASbGncuHiZRtYEESpJOHQFhrZWkYAtPSAcog0zc54Ec2IYZvgxBCaFyYI06Tsx5erRh Nu3IlFkPztr0w4Wcew1HW8kr0N36VBTwYX7gdtfwnXtIwjSYsX4n4DwAeeZ4Yezkke6HGuXXzEB THc2MV1/VcJfNX0UNA75wYdNA77muX/21q043aYtQi09WAd0WU1zsnuM2LeIhbg5cMI/g4mW1yp 4nqGNg6AmDMtft7BscUB4TctNVaFPJ7L+j2ptSqvMSXGfkI+A4LbPLXBRysiKRwS9C8Kv2iThxp i0o7yO4flgyZpuf0Bdfh0Mov/oczTtNaP3tSPOzvzx1T1zarr+F8kt0NySWIAfk3t9GG0Kci+j5 MBQbJgUL3UfKCbi1C2Fz8TTHGBjTq+ks3eEnBpqX/R2H3xsFfz9NJo+hbGSdUwB79m/VI6zYHWV CpkSavmOr8 X-Google-Smtp-Source: AGHT+IFtYAWDJ7Pr1WCiIJeu5/dqTqeOTpptsUO9Ak25hihe82YdMdikIzwORfuHsMxn4AbnGxWLAWoFkJaE X-Received: by 2002:a17:903:2f06:b0:24e:229e:204e with SMTP id d9443c01a7336-25d25291ccdmr4843495ad.16.1757619358565; Thu, 11 Sep 2025 12:35:58 -0700 (PDT) Received: from smtp-us-east1-p01-i01-si01.dlp.protect.broadcom.com (address-144-49-247-101.dlp.protect.broadcom.com. [144.49.247.101]) by smtp-relay.gmail.com with ESMTPS id d9443c01a7336-25c36bc2c4bsm1976485ad.15.2025.09.11.12.35.58 for (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Thu, 11 Sep 2025 12:35:58 -0700 (PDT) X-Relaying-Domain: broadcom.com X-CFilter-Loop: Reflected Received: by mail-pl1-f199.google.com with SMTP id d9443c01a7336-25c1dba8d26so13641865ad.1 for ; Thu, 11 Sep 2025 12:35:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=broadcom.com; s=google; t=1757619357; x=1758224157; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=imG5pN0GTv13vxSJjHPy14bFiRuvwF0ZssLFfjreBEc=; b=EKs0wQgQhqN7aVyWewx0qojUP2xNOzA5QyAxlIBnX/wfRQxEDJhBlriNcKFikmtmWi q3JU7ymKx5btymvAkp5xE7ujQHedoK5UQFgaYJelcqCOf+U1IK7fbjxTviFI7DGMTSEM zyXkQ9vDDqabkwtRCgsqivUXsR+GFJY/xYkME= X-Forwarded-Encrypted: i=1; AJvYcCVQ7IOYTTZVwBwS50Roc2TP8HUOBBIcvxV2aQIuR81cx321oIuWTIkQIX2fDjIy5ncNFbqGM6nnVSMgNXU=@vger.kernel.org X-Received: by 2002:a17:902:7b85:b0:258:c13d:9b1a with SMTP id d9443c01a7336-25d274340b0mr3380885ad.41.1757619356715; Thu, 11 Sep 2025 12:35:56 -0700 (PDT) X-Received: by 2002:a17:902:7b85:b0:258:c13d:9b1a with SMTP id d9443c01a7336-25d274340b0mr3380725ad.41.1757619356179; Thu, 11 Sep 2025 12:35:56 -0700 (PDT) Received: from hyd-csg-thor2-h1-server2.dhcp.broadcom.net ([192.19.203.250]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-25c3ad3404csm25839285ad.113.2025.09.11.12.35.51 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 11 Sep 2025 12:35:55 -0700 (PDT) From: Bhargava Marreddy To: davem@davemloft.net, edumazet@google.com, kuba@kernel.org, pabeni@redhat.com, andrew+netdev@lunn.ch, horms@kernel.org Cc: netdev@vger.kernel.org, linux-kernel@vger.kernel.org, michael.chan@broadcom.com, pavan.chebbi@broadcom.com, vsrama-krishna.nemani@broadcom.com, vikas.gupta@broadcom.com, Bhargava Marreddy , Rajashekar Hudumula Subject: [v7, net-next 08/10] bng_en: Register rings with the firmware Date: Fri, 12 Sep 2025 01:05:03 +0530 Message-ID: <20250911193505.24068-9-bhargava.marreddy@broadcom.com> X-Mailer: git-send-email 2.47.3 In-Reply-To: <20250911193505.24068-1-bhargava.marreddy@broadcom.com> References: <20250911193505.24068-1-bhargava.marreddy@broadcom.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-DetectorID-Processed: b00c1d49-9d2e-4205-b15f-d015386d3d5e Content-Type: text/plain; charset="utf-8" Enable ring functionality by registering RX, AGG, TX, CMPL, and NQ rings with the firmware. Initialise the doorbells associated with the rings. Signed-off-by: Bhargava Marreddy Reviewed-by: Vikas Gupta Reviewed-by: Rajashekar Hudumula --- drivers/net/ethernet/broadcom/bnge/bnge.h | 24 ++ .../net/ethernet/broadcom/bnge/bnge_core.c | 4 + drivers/net/ethernet/broadcom/bnge/bnge_db.h | 34 ++ .../ethernet/broadcom/bnge/bnge_hwrm_lib.c | 147 ++++++++ .../ethernet/broadcom/bnge/bnge_hwrm_lib.h | 6 + .../net/ethernet/broadcom/bnge/bnge_netdev.c | 349 ++++++++++++++++++ .../net/ethernet/broadcom/bnge/bnge_netdev.h | 16 + .../net/ethernet/broadcom/bnge/bnge_rmem.h | 1 + 8 files changed, 581 insertions(+) create mode 100644 drivers/net/ethernet/broadcom/bnge/bnge_db.h diff --git a/drivers/net/ethernet/broadcom/bnge/bnge.h b/drivers/net/ethern= et/broadcom/bnge/bnge.h index c536c0cc66e..aee65a6c980 100644 --- a/drivers/net/ethernet/broadcom/bnge/bnge.h +++ b/drivers/net/ethernet/broadcom/bnge/bnge.h @@ -102,6 +102,10 @@ struct bnge_dev { u16 chip_num; u8 chip_rev; =20 +#if BITS_PER_LONG =3D=3D 32 + /* ensure atomic 64-bit doorbell writes on 32-bit systems. */ + spinlock_t db_lock; +#endif int db_offset; /* db_offset within db_size */ int db_size; =20 @@ -214,6 +218,26 @@ static inline bool bnge_is_agg_reqd(struct bnge_dev *b= d) return true; } =20 +static inline void bnge_writeq(struct bnge_dev *bd, u64 val, + void __iomem *addr) +{ +#if BITS_PER_LONG =3D=3D 32 + spin_lock(&bd->db_lock); + lo_hi_writeq(val, addr); + spin_unlock(&bd->db_lock); +#else + writeq(val, addr); +#endif +} + +/* For TX and RX ring doorbells */ +static inline void bnge_db_write(struct bnge_dev *bd, struct bnge_db_info = *db, + u32 idx) +{ + bnge_writeq(bd, db->db_key64 | DB_RING_IDX(db, idx), + db->doorbell); +} + bool bnge_aux_registered(struct bnge_dev *bd); u16 bnge_aux_get_msix(struct bnge_dev *bd); =20 diff --git a/drivers/net/ethernet/broadcom/bnge/bnge_core.c b/drivers/net/e= thernet/broadcom/bnge/bnge_core.c index 68da656f289..304b1e4d5a0 100644 --- a/drivers/net/ethernet/broadcom/bnge/bnge_core.c +++ b/drivers/net/ethernet/broadcom/bnge/bnge_core.c @@ -296,6 +296,10 @@ static int bnge_probe_one(struct pci_dev *pdev, const = struct pci_device_id *ent) goto err_config_uninit; } =20 +#if BITS_PER_LONG =3D=3D 32 + spin_lock_init(&bd->db_lock); +#endif + rc =3D bnge_alloc_irqs(bd); if (rc) { dev_err(&pdev->dev, "Error IRQ allocation rc =3D %d\n", rc); diff --git a/drivers/net/ethernet/broadcom/bnge/bnge_db.h b/drivers/net/eth= ernet/broadcom/bnge/bnge_db.h new file mode 100644 index 00000000000..950ed582f1d --- /dev/null +++ b/drivers/net/ethernet/broadcom/bnge/bnge_db.h @@ -0,0 +1,34 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* Copyright (c) 2025 Broadcom */ + +#ifndef _BNGE_DB_H_ +#define _BNGE_DB_H_ + +/* 64-bit doorbell */ +#define DBR_EPOCH_SFT 24 +#define DBR_TOGGLE_SFT 25 +#define DBR_XID_SFT 32 +#define DBR_PATH_L2 (0x1ULL << 56) +#define DBR_VALID (0x1ULL << 58) +#define DBR_TYPE_SQ (0x0ULL << 60) +#define DBR_TYPE_SRQ (0x2ULL << 60) +#define DBR_TYPE_CQ (0x4ULL << 60) +#define DBR_TYPE_CQ_ARMALL (0x6ULL << 60) +#define DBR_TYPE_NQ (0xaULL << 60) +#define DBR_TYPE_NQ_ARM (0xbULL << 60) +#define DBR_TYPE_NQ_MASK (0xeULL << 60) + +struct bnge_db_info { + void __iomem *doorbell; + u64 db_key64; + u32 db_ring_mask; + u32 db_epoch_mask; + u8 db_epoch_shift; +}; + +#define DB_EPOCH(db, idx) (((idx) & (db)->db_epoch_mask) << \ + ((db)->db_epoch_shift)) +#define DB_RING_IDX(db, idx) (((idx) & (db)->db_ring_mask) | \ + DB_EPOCH(db, idx)) + +#endif /* _BNGE_DB_H_ */ diff --git a/drivers/net/ethernet/broadcom/bnge/bnge_hwrm_lib.c b/drivers/n= et/ethernet/broadcom/bnge/bnge_hwrm_lib.c index 8f20b880c11..b44e0f4ed7c 100644 --- a/drivers/net/ethernet/broadcom/bnge/bnge_hwrm_lib.c +++ b/drivers/net/ethernet/broadcom/bnge/bnge_hwrm_lib.c @@ -757,3 +757,150 @@ int bnge_hwrm_stat_ctx_alloc(struct bnge_net *bn) bnge_hwrm_req_drop(bd, req); return rc; } + +int hwrm_ring_free_send_msg(struct bnge_net *bn, + struct bnge_ring_struct *ring, + u32 ring_type, int cmpl_ring_id) +{ + struct hwrm_ring_free_input *req; + struct bnge_dev *bd =3D bn->bd; + int rc; + + rc =3D bnge_hwrm_req_init(bd, req, HWRM_RING_FREE); + if (rc) + goto exit; + + req->cmpl_ring =3D cpu_to_le16(cmpl_ring_id); + req->ring_type =3D ring_type; + req->ring_id =3D cpu_to_le16(ring->fw_ring_id); + + bnge_hwrm_req_hold(bd, req); + rc =3D bnge_hwrm_req_send(bd, req); + bnge_hwrm_req_drop(bd, req); +exit: + if (rc) { + netdev_err(bd->netdev, "hwrm_ring_free type %d failed. rc:%d\n", ring_ty= pe, rc); + return -EIO; + } + return 0; +} + +int hwrm_ring_alloc_send_msg(struct bnge_net *bn, + struct bnge_ring_struct *ring, + u32 ring_type, u32 map_index) +{ + struct bnge_ring_mem_info *rmem =3D &ring->ring_mem; + struct bnge_ring_grp_info *grp_info; + struct hwrm_ring_alloc_output *resp; + struct hwrm_ring_alloc_input *req; + struct bnge_dev *bd =3D bn->bd; + u16 ring_id, flags =3D 0; + int rc; + + rc =3D bnge_hwrm_req_init(bd, req, HWRM_RING_ALLOC); + if (rc) + goto exit; + + req->enables =3D 0; + if (rmem->nr_pages > 1) { + req->page_tbl_addr =3D cpu_to_le64(rmem->dma_pg_tbl); + /* Page size is in log2 units */ + req->page_size =3D BNGE_PAGE_SHIFT; + req->page_tbl_depth =3D 1; + } else { + req->page_tbl_addr =3D cpu_to_le64(rmem->dma_arr[0]); + } + req->fbo =3D 0; + /* Association of ring index with doorbell index and MSIX number */ + req->logical_id =3D cpu_to_le16(map_index); + + switch (ring_type) { + case HWRM_RING_ALLOC_TX: { + struct bnge_tx_ring_info *txr; + + txr =3D container_of(ring, struct bnge_tx_ring_info, + tx_ring_struct); + req->ring_type =3D RING_ALLOC_REQ_RING_TYPE_TX; + /* Association of transmit ring with completion ring */ + grp_info =3D &bn->grp_info[ring->grp_idx]; + req->cmpl_ring_id =3D cpu_to_le16(bnge_cp_ring_for_tx(txr)); + req->length =3D cpu_to_le32(bn->tx_ring_mask + 1); + req->stat_ctx_id =3D cpu_to_le32(grp_info->fw_stats_ctx); + req->queue_id =3D cpu_to_le16(ring->queue_id); + req->flags =3D cpu_to_le16(flags); + break; + } + case HWRM_RING_ALLOC_RX: + req->ring_type =3D RING_ALLOC_REQ_RING_TYPE_RX; + req->length =3D cpu_to_le32(bn->rx_ring_mask + 1); + + /* Association of rx ring with stats context */ + grp_info =3D &bn->grp_info[ring->grp_idx]; + req->rx_buf_size =3D cpu_to_le16(bn->rx_buf_use_size); + req->stat_ctx_id =3D cpu_to_le32(grp_info->fw_stats_ctx); + req->enables |=3D + cpu_to_le32(RING_ALLOC_REQ_ENABLES_RX_BUF_SIZE_VALID); + if (NET_IP_ALIGN =3D=3D 2) + flags =3D RING_ALLOC_REQ_FLAGS_RX_SOP_PAD; + req->flags =3D cpu_to_le16(flags); + break; + case HWRM_RING_ALLOC_AGG: + req->ring_type =3D RING_ALLOC_REQ_RING_TYPE_RX_AGG; + /* Association of agg ring with rx ring */ + grp_info =3D &bn->grp_info[ring->grp_idx]; + req->rx_ring_id =3D cpu_to_le16(grp_info->rx_fw_ring_id); + req->rx_buf_size =3D cpu_to_le16(BNGE_RX_PAGE_SIZE); + req->stat_ctx_id =3D cpu_to_le32(grp_info->fw_stats_ctx); + req->enables |=3D + cpu_to_le32(RING_ALLOC_REQ_ENABLES_RX_RING_ID_VALID | + RING_ALLOC_REQ_ENABLES_RX_BUF_SIZE_VALID); + req->length =3D cpu_to_le32(bn->rx_agg_ring_mask + 1); + break; + case HWRM_RING_ALLOC_CMPL: + req->ring_type =3D RING_ALLOC_REQ_RING_TYPE_L2_CMPL; + req->length =3D cpu_to_le32(bn->cp_ring_mask + 1); + /* Association of cp ring with nq */ + grp_info =3D &bn->grp_info[map_index]; + req->nq_ring_id =3D cpu_to_le16(grp_info->nq_fw_ring_id); + req->cq_handle =3D cpu_to_le64(ring->handle); + req->enables |=3D + cpu_to_le32(RING_ALLOC_REQ_ENABLES_NQ_RING_ID_VALID); + break; + case HWRM_RING_ALLOC_NQ: + req->ring_type =3D RING_ALLOC_REQ_RING_TYPE_NQ; + req->length =3D cpu_to_le32(bn->cp_ring_mask + 1); + req->int_mode =3D RING_ALLOC_REQ_INT_MODE_MSIX; + break; + default: + netdev_err(bn->netdev, "hwrm alloc invalid ring type %d\n", ring_type); + return -EINVAL; + } + + resp =3D bnge_hwrm_req_hold(bd, req); + rc =3D bnge_hwrm_req_send(bd, req); + ring_id =3D le16_to_cpu(resp->ring_id); + bnge_hwrm_req_drop(bd, req); + +exit: + if (rc) { + netdev_err(bd->netdev, "hwrm_ring_alloc type %d failed. rc:%d\n", ring_t= ype, rc); + return -EIO; + } + ring->fw_ring_id =3D ring_id; + return rc; +} + +int bnge_hwrm_set_async_event_cr(struct bnge_dev *bd, int idx) +{ + struct hwrm_func_cfg_input *req; + int rc; + + rc =3D bnge_hwrm_req_init(bd, req, HWRM_FUNC_CFG); + if (rc) + return rc; + + req->fid =3D cpu_to_le16(0xffff); + req->enables =3D cpu_to_le32(FUNC_CFG_REQ_ENABLES_ASYNC_EVENT_CR); + req->async_event_cr =3D cpu_to_le16(idx); + return bnge_hwrm_req_send(bd, req); +} diff --git a/drivers/net/ethernet/broadcom/bnge/bnge_hwrm_lib.h b/drivers/n= et/ethernet/broadcom/bnge/bnge_hwrm_lib.h index 1c3fd02d7e0..b2e2ec47be2 100644 --- a/drivers/net/ethernet/broadcom/bnge/bnge_hwrm_lib.h +++ b/drivers/net/ethernet/broadcom/bnge/bnge_hwrm_lib.h @@ -26,4 +26,10 @@ int bnge_hwrm_queue_qportcfg(struct bnge_dev *bd); =20 void bnge_hwrm_stat_ctx_free(struct bnge_net *bn); int bnge_hwrm_stat_ctx_alloc(struct bnge_net *bn); +int hwrm_ring_free_send_msg(struct bnge_net *bn, struct bnge_ring_struct *= ring, + u32 ring_type, int cmpl_ring_id); +int hwrm_ring_alloc_send_msg(struct bnge_net *bn, + struct bnge_ring_struct *ring, + u32 ring_type, u32 map_index); +int bnge_hwrm_set_async_event_cr(struct bnge_dev *bd, int idx); #endif /* _BNGE_HWRM_LIB_H_ */ diff --git a/drivers/net/ethernet/broadcom/bnge/bnge_netdev.c b/drivers/net= /ethernet/broadcom/bnge/bnge_netdev.c index 1d506e36af9..dbfb4516933 100644 --- a/drivers/net/ethernet/broadcom/bnge/bnge_netdev.c +++ b/drivers/net/ethernet/broadcom/bnge/bnge_netdev.c @@ -844,6 +844,28 @@ static int bnge_alloc_core(struct bnge_net *bn) return rc; } =20 +u16 bnge_cp_ring_for_rx(struct bnge_rx_ring_info *rxr) +{ + return rxr->rx_cpr->ring_struct.fw_ring_id; +} + +u16 bnge_cp_ring_for_tx(struct bnge_tx_ring_info *txr) +{ + return txr->tx_cpr->ring_struct.fw_ring_id; +} + +static void bnge_db_nq(struct bnge_net *bn, struct bnge_db_info *db, u32 i= dx) +{ + bnge_writeq(bn->bd, db->db_key64 | DBR_TYPE_NQ_MASK | + DB_RING_IDX(db, idx), db->doorbell); +} + +static void bnge_db_cq(struct bnge_net *bn, struct bnge_db_info *db, u32 i= dx) +{ + bnge_writeq(bn->bd, db->db_key64 | DBR_TYPE_CQ_ARMALL | + DB_RING_IDX(db, idx), db->doorbell); +} + static int bnge_cp_num_to_irq_num(struct bnge_net *bn, int n) { struct bnge_napi *bnapi =3D bn->bnapi[n]; @@ -1176,6 +1198,326 @@ static void bnge_init_vnics(struct bnge_net *bn) } } =20 +static void bnge_set_db_mask(struct bnge_net *bn, struct bnge_db_info *db, + u32 ring_type) +{ + switch (ring_type) { + case HWRM_RING_ALLOC_TX: + db->db_ring_mask =3D bn->tx_ring_mask; + break; + case HWRM_RING_ALLOC_RX: + db->db_ring_mask =3D bn->rx_ring_mask; + break; + case HWRM_RING_ALLOC_AGG: + db->db_ring_mask =3D bn->rx_agg_ring_mask; + break; + case HWRM_RING_ALLOC_CMPL: + case HWRM_RING_ALLOC_NQ: + db->db_ring_mask =3D bn->cp_ring_mask; + break; + } + db->db_epoch_mask =3D db->db_ring_mask + 1; + db->db_epoch_shift =3D DBR_EPOCH_SFT - ilog2(db->db_epoch_mask); +} + +static void bnge_set_db(struct bnge_net *bn, struct bnge_db_info *db, + u32 ring_type, u32 map_idx, u32 xid) +{ + struct bnge_dev *bd =3D bn->bd; + + switch (ring_type) { + case HWRM_RING_ALLOC_TX: + db->db_key64 =3D DBR_PATH_L2 | DBR_TYPE_SQ; + break; + case HWRM_RING_ALLOC_RX: + case HWRM_RING_ALLOC_AGG: + db->db_key64 =3D DBR_PATH_L2 | DBR_TYPE_SRQ; + break; + case HWRM_RING_ALLOC_CMPL: + db->db_key64 =3D DBR_PATH_L2; + break; + case HWRM_RING_ALLOC_NQ: + db->db_key64 =3D DBR_PATH_L2; + break; + } + db->db_key64 |=3D ((u64)xid << DBR_XID_SFT) | DBR_VALID; + + db->doorbell =3D bd->bar1 + bd->db_offset; + bnge_set_db_mask(bn, db, ring_type); +} + +static int bnge_hwrm_cp_ring_alloc(struct bnge_net *bn, + struct bnge_cp_ring_info *cpr) +{ + const u32 type =3D HWRM_RING_ALLOC_CMPL; + struct bnge_napi *bnapi =3D cpr->bnapi; + struct bnge_ring_struct *ring; + u32 map_idx =3D bnapi->index; + int rc; + + ring =3D &cpr->ring_struct; + ring->handle =3D BNGE_SET_NQ_HDL(cpr); + rc =3D hwrm_ring_alloc_send_msg(bn, ring, type, map_idx); + if (rc) + return rc; + + bnge_set_db(bn, &cpr->cp_db, type, map_idx, ring->fw_ring_id); + bnge_db_cq(bn, &cpr->cp_db, cpr->cp_raw_cons); + + return 0; +} + +static int bnge_hwrm_tx_ring_alloc(struct bnge_net *bn, + struct bnge_tx_ring_info *txr, u32 tx_idx) +{ + struct bnge_ring_struct *ring =3D &txr->tx_ring_struct; + const u32 type =3D HWRM_RING_ALLOC_TX; + int rc; + + rc =3D hwrm_ring_alloc_send_msg(bn, ring, type, tx_idx); + if (rc) + return rc; + + bnge_set_db(bn, &txr->tx_db, type, tx_idx, ring->fw_ring_id); + + return 0; +} + +static int bnge_hwrm_rx_agg_ring_alloc(struct bnge_net *bn, + struct bnge_rx_ring_info *rxr) +{ + struct bnge_ring_struct *ring =3D &rxr->rx_agg_ring_struct; + u32 type =3D HWRM_RING_ALLOC_AGG; + struct bnge_dev *bd =3D bn->bd; + u32 grp_idx =3D ring->grp_idx; + u32 map_idx; + int rc; + + map_idx =3D grp_idx + bd->rx_nr_rings; + rc =3D hwrm_ring_alloc_send_msg(bn, ring, type, map_idx); + if (rc) + return rc; + + bnge_set_db(bn, &rxr->rx_agg_db, type, map_idx, + ring->fw_ring_id); + bnge_db_write(bn->bd, &rxr->rx_agg_db, rxr->rx_agg_prod); + bnge_db_write(bn->bd, &rxr->rx_db, rxr->rx_prod); + bn->grp_info[grp_idx].agg_fw_ring_id =3D ring->fw_ring_id; + + return 0; +} + +static int bnge_hwrm_rx_ring_alloc(struct bnge_net *bn, + struct bnge_rx_ring_info *rxr) +{ + struct bnge_ring_struct *ring =3D &rxr->rx_ring_struct; + struct bnge_napi *bnapi =3D rxr->bnapi; + u32 type =3D HWRM_RING_ALLOC_RX; + u32 map_idx =3D bnapi->index; + int rc; + + rc =3D hwrm_ring_alloc_send_msg(bn, ring, type, map_idx); + if (rc) + return rc; + + bnge_set_db(bn, &rxr->rx_db, type, map_idx, ring->fw_ring_id); + bn->grp_info[map_idx].rx_fw_ring_id =3D ring->fw_ring_id; + + return 0; +} + +static int bnge_hwrm_ring_alloc(struct bnge_net *bn) +{ + struct bnge_dev *bd =3D bn->bd; + bool agg_rings; + int i, rc =3D 0; + + agg_rings =3D !!(bnge_is_agg_reqd(bd)); + for (i =3D 0; i < bd->nq_nr_rings; i++) { + struct bnge_napi *bnapi =3D bn->bnapi[i]; + struct bnge_nq_ring_info *nqr =3D &bnapi->nq_ring; + struct bnge_ring_struct *ring =3D &nqr->ring_struct; + u32 type =3D HWRM_RING_ALLOC_NQ; + u32 map_idx =3D ring->map_idx; + unsigned int vector; + + vector =3D bd->irq_tbl[map_idx].vector; + disable_irq_nosync(vector); + rc =3D hwrm_ring_alloc_send_msg(bn, ring, type, map_idx); + if (rc) { + enable_irq(vector); + goto err_out; + } + bnge_set_db(bn, &nqr->nq_db, type, map_idx, ring->fw_ring_id); + bnge_db_nq(bn, &nqr->nq_db, nqr->nq_raw_cons); + enable_irq(vector); + bn->grp_info[i].nq_fw_ring_id =3D ring->fw_ring_id; + + if (!i) { + rc =3D bnge_hwrm_set_async_event_cr(bd, ring->fw_ring_id); + if (rc) + netdev_warn(bn->netdev, "Failed to set async event completion ring.\n"= ); + } + } + + for (i =3D 0; i < bd->tx_nr_rings; i++) { + struct bnge_tx_ring_info *txr =3D &bn->tx_ring[i]; + + rc =3D bnge_hwrm_cp_ring_alloc(bn, txr->tx_cpr); + if (rc) + goto err_out; + rc =3D bnge_hwrm_tx_ring_alloc(bn, txr, i); + if (rc) + goto err_out; + } + + for (i =3D 0; i < bd->rx_nr_rings; i++) { + struct bnge_rx_ring_info *rxr =3D &bn->rx_ring[i]; + struct bnge_cp_ring_info *cpr; + struct bnge_ring_struct *ring; + struct bnge_napi *bnapi; + u32 map_idx, type; + + rc =3D bnge_hwrm_rx_ring_alloc(bn, rxr); + if (rc) + goto err_out; + /* If we have agg rings, post agg buffers first. */ + if (!agg_rings) + bnge_db_write(bn->bd, &rxr->rx_db, rxr->rx_prod); + + cpr =3D rxr->rx_cpr; + bnapi =3D rxr->bnapi; + type =3D HWRM_RING_ALLOC_CMPL; + map_idx =3D bnapi->index; + + ring =3D &cpr->ring_struct; + ring->handle =3D BNGE_SET_NQ_HDL(cpr); + rc =3D hwrm_ring_alloc_send_msg(bn, ring, type, map_idx); + if (rc) + goto err_out; + bnge_set_db(bn, &cpr->cp_db, type, map_idx, + ring->fw_ring_id); + bnge_db_cq(bn, &cpr->cp_db, cpr->cp_raw_cons); + } + + if (agg_rings) { + for (i =3D 0; i < bd->rx_nr_rings; i++) { + rc =3D bnge_hwrm_rx_agg_ring_alloc(bn, &bn->rx_ring[i]); + if (rc) + goto err_out; + } + } +err_out: + return rc; +} + +static void bnge_hwrm_rx_ring_free(struct bnge_net *bn, + struct bnge_rx_ring_info *rxr, + bool close_path) +{ + struct bnge_ring_struct *ring =3D &rxr->rx_ring_struct; + u32 grp_idx =3D rxr->bnapi->index; + u32 cmpl_ring_id; + + if (ring->fw_ring_id =3D=3D INVALID_HW_RING_ID) + return; + + cmpl_ring_id =3D bnge_cp_ring_for_rx(rxr); + hwrm_ring_free_send_msg(bn, ring, + RING_FREE_REQ_RING_TYPE_RX, + close_path ? cmpl_ring_id : + INVALID_HW_RING_ID); + ring->fw_ring_id =3D INVALID_HW_RING_ID; + bn->grp_info[grp_idx].rx_fw_ring_id =3D INVALID_HW_RING_ID; +} + +static void bnge_hwrm_rx_agg_ring_free(struct bnge_net *bn, + struct bnge_rx_ring_info *rxr, + bool close_path) +{ + struct bnge_ring_struct *ring =3D &rxr->rx_agg_ring_struct; + u32 grp_idx =3D rxr->bnapi->index; + u32 cmpl_ring_id; + + if (ring->fw_ring_id =3D=3D INVALID_HW_RING_ID) + return; + + cmpl_ring_id =3D bnge_cp_ring_for_rx(rxr); + hwrm_ring_free_send_msg(bn, ring, RING_FREE_REQ_RING_TYPE_RX_AGG, + close_path ? cmpl_ring_id : + INVALID_HW_RING_ID); + ring->fw_ring_id =3D INVALID_HW_RING_ID; + bn->grp_info[grp_idx].agg_fw_ring_id =3D INVALID_HW_RING_ID; +} + +static void bnge_hwrm_tx_ring_free(struct bnge_net *bn, + struct bnge_tx_ring_info *txr, + bool close_path) +{ + struct bnge_ring_struct *ring =3D &txr->tx_ring_struct; + u32 cmpl_ring_id; + + if (ring->fw_ring_id =3D=3D INVALID_HW_RING_ID) + return; + + cmpl_ring_id =3D close_path ? bnge_cp_ring_for_tx(txr) : + INVALID_HW_RING_ID; + hwrm_ring_free_send_msg(bn, ring, RING_FREE_REQ_RING_TYPE_TX, + cmpl_ring_id); + ring->fw_ring_id =3D INVALID_HW_RING_ID; +} + +static void bnge_hwrm_cp_ring_free(struct bnge_net *bn, + struct bnge_cp_ring_info *cpr) +{ + struct bnge_ring_struct *ring; + + ring =3D &cpr->ring_struct; + if (ring->fw_ring_id =3D=3D INVALID_HW_RING_ID) + return; + + hwrm_ring_free_send_msg(bn, ring, RING_FREE_REQ_RING_TYPE_L2_CMPL, + INVALID_HW_RING_ID); + ring->fw_ring_id =3D INVALID_HW_RING_ID; +} + +static void bnge_hwrm_ring_free(struct bnge_net *bn, bool close_path) +{ + struct bnge_dev *bd =3D bn->bd; + int i; + + if (!bn->bnapi) + return; + + for (i =3D 0; i < bd->tx_nr_rings; i++) + bnge_hwrm_tx_ring_free(bn, &bn->tx_ring[i], close_path); + + for (i =3D 0; i < bd->rx_nr_rings; i++) { + bnge_hwrm_rx_ring_free(bn, &bn->rx_ring[i], close_path); + bnge_hwrm_rx_agg_ring_free(bn, &bn->rx_ring[i], close_path); + } + + for (i =3D 0; i < bd->nq_nr_rings; i++) { + struct bnge_napi *bnapi =3D bn->bnapi[i]; + struct bnge_nq_ring_info *nqr; + struct bnge_ring_struct *ring; + int j; + + nqr =3D &bnapi->nq_ring; + for (j =3D 0; j < nqr->cp_ring_count && nqr->cp_ring_arr; j++) + bnge_hwrm_cp_ring_free(bn, &nqr->cp_ring_arr[j]); + + ring =3D &nqr->ring_struct; + if (ring->fw_ring_id !=3D INVALID_HW_RING_ID) { + hwrm_ring_free_send_msg(bn, ring, + RING_FREE_REQ_RING_TYPE_NQ, + INVALID_HW_RING_ID); + ring->fw_ring_id =3D INVALID_HW_RING_ID; + bn->grp_info[i].nq_fw_ring_id =3D INVALID_HW_RING_ID; + } + } +} + static void bnge_setup_msix(struct bnge_net *bn) { struct net_device *dev =3D bn->netdev; @@ -1212,6 +1554,7 @@ static int bnge_setup_interrupts(struct bnge_net *bn) =20 static void bnge_hwrm_resource_free(struct bnge_net *bn, bool close_path) { + bnge_hwrm_ring_free(bn, close_path); bnge_hwrm_stat_ctx_free(bn); } =20 @@ -1294,6 +1637,12 @@ static int bnge_init_chip(struct bnge_net *bn) netdev_err(bn->netdev, "hwrm stat ctx alloc failure rc: %d\n", rc); goto err_out; } + + rc =3D bnge_hwrm_ring_alloc(bn); + if (rc) { + netdev_err(bn->netdev, "hwrm ring alloc failure rc: %d\n", rc); + goto err_out; + } return 0; =20 err_out: diff --git a/drivers/net/ethernet/broadcom/bnge/bnge_netdev.h b/drivers/net= /ethernet/broadcom/bnge/bnge_netdev.h index 56df0765bf0..ba0dd2202fb 100644 --- a/drivers/net/ethernet/broadcom/bnge/bnge_netdev.h +++ b/drivers/net/ethernet/broadcom/bnge/bnge_netdev.h @@ -5,6 +5,8 @@ #define _BNGE_NETDEV_H_ =20 #include +#include +#include "bnge_db.h" =20 struct tx_bd { __le32 tx_bd_len_flags_type; @@ -169,6 +171,7 @@ enum { #define RING_RX_AGG(bn, idx) ((idx) & (bn)->rx_agg_ring_mask) #define NEXT_RX_AGG(idx) ((idx) + 1) =20 +#define BNGE_NQ_HDL_TYPE_SHIFT 24 #define BNGE_NQ_HDL_TYPE_RX 0x00 #define BNGE_NQ_HDL_TYPE_TX 0x01 =20 @@ -272,6 +275,9 @@ void bnge_set_ring_params(struct bnge_dev *bd); txr =3D (iter < BNGE_MAX_TXR_PER_NAPI - 1) ? \ (bnapi)->tx_ring[++iter] : NULL) =20 +#define BNGE_SET_NQ_HDL(cpr) \ + (((cpr)->cp_ring_type << BNGE_NQ_HDL_TYPE_SHIFT) | (cpr)->cp_idx) + struct bnge_stats_mem { u64 *sw_stats; u64 *hw_masks; @@ -287,6 +293,8 @@ struct bnge_cp_ring_info { struct bnge_ring_struct ring_struct; u8 cp_ring_type; u8 cp_idx; + u32 cp_raw_cons; + struct bnge_db_info cp_db; }; =20 struct bnge_nq_ring_info { @@ -294,6 +302,8 @@ struct bnge_nq_ring_info { dma_addr_t *desc_mapping; struct nqe_cn **desc_ring; struct bnge_ring_struct ring_struct; + u32 nq_raw_cons; + struct bnge_db_info nq_db; =20 struct bnge_stats_mem stats; u32 hw_stats_ctx_id; @@ -309,6 +319,8 @@ struct bnge_rx_ring_info { u16 rx_agg_prod; u16 rx_sw_agg_prod; u16 rx_next_cons; + struct bnge_db_info rx_db; + struct bnge_db_info rx_agg_db; =20 struct rx_bd *rx_desc_ring[MAX_RX_PAGES]; struct bnge_sw_rx_bd *rx_buf_ring; @@ -338,6 +350,7 @@ struct bnge_tx_ring_info { u16 txq_index; u8 tx_napi_idx; u8 kick_pending; + struct bnge_db_info tx_db; =20 struct tx_bd *tx_desc_ring[MAX_TX_PAGES]; struct bnge_sw_tx_bd *tx_buf_ring; @@ -392,4 +405,7 @@ struct bnge_vnic_info { #define BNGE_VNIC_UCAST_FLAG 8 u32 vnic_id; }; + +u16 bnge_cp_ring_for_rx(struct bnge_rx_ring_info *rxr); +u16 bnge_cp_ring_for_tx(struct bnge_tx_ring_info *txr); #endif /* _BNGE_NETDEV_H_ */ diff --git a/drivers/net/ethernet/broadcom/bnge/bnge_rmem.h b/drivers/net/e= thernet/broadcom/bnge/bnge_rmem.h index 0e7684e2071..341c7f81ed0 100644 --- a/drivers/net/ethernet/broadcom/bnge/bnge_rmem.h +++ b/drivers/net/ethernet/broadcom/bnge/bnge_rmem.h @@ -189,6 +189,7 @@ struct bnge_ring_struct { u16 grp_idx; u16 map_idx; /* Used by NQs */ }; + u32 handle; u8 queue_id; }; =20 --=20 2.47.3