From nobody Mon Feb 9 17:35:32 2026 Received: from mail-pl1-f228.google.com (mail-pl1-f228.google.com [209.85.214.228]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B38FA34C140 for ; Fri, 16 Jan 2026 19:38:19 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.228 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1768592306; cv=none; b=HJ3FJ32Af/3LVkW3XFNeouAtlfHrx7xeEcMjYLjlG4OOIVuWPTvOXHDAZebFWmZ8ekLqmF2sgE0lPJ8sHj5/zjKdLgWXlz52wPSsiZtyesS/nhjDC6s404ASnls9c6WNC8N+p8XgC6zEaVUtOaNex9opHSmW4KQSDSXFzuQ/Sss= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1768592306; c=relaxed/simple; bh=H4d1GmIpDpk+qEEU1o/FcreRLGh2PG8degWZnE9mrEY=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=Ln1fNEQBhlDX2PlYCrG0YRqZB8gXz0DI1S65UApy2pWxVpVV6wBg31F1UtGmH/EwcrUBKQMqLpvkOeZv81JOVpnDn5zYCEnpdKFwKpZkwn90e79P/eLgRrNwLh0eY7xnqLiuU/IrczdC3k5DcEHV7KW1V2Y7xZL3tO8hQUG5LC4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=broadcom.com; spf=fail smtp.mailfrom=broadcom.com; dkim=pass (1024-bit key) header.d=broadcom.com header.i=@broadcom.com header.b=K7s0puAs; arc=none smtp.client-ip=209.85.214.228 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=broadcom.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=broadcom.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=broadcom.com header.i=@broadcom.com header.b="K7s0puAs" Received: by mail-pl1-f228.google.com with SMTP id d9443c01a7336-29f1bc40b35so25072755ad.2 for ; Fri, 16 Jan 2026 11:38:19 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1768592299; x=1769197099; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:dkim-signature:x-gm-gg :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=5RE1Ydonxn/EbzGQfkPi595Jg1sH7wQCQVQgQmIM5tc=; b=HVj8MIjfeNiy9pIh40kmhQEANMa4gDP+rY6noS2khlfyEMiLGLF3mjF6y9yrHE17hz dobd6v7DebtFcPXCaLUWOaQD95AuPjKAmvhkLDBy6OSqekwZxN9JHg+fKT3aFAPJR8oI 3Pfj7oUEYjBDg0NwSoekbSiZ5CeRidc4v+ntalMbuZac0azL4OsrZ1w5NQBRGu/9zTqV 1oRizzPyZFtG+hmKlDd5ZtyUuxslb3emdAQBO6dZmZbOkjBFMs5kVoADWyGeH0yvPAdA 47fMccz49kMZYxTJzA9cGtI8DMO/HxXfevZTWHQkOQgIjXe4FG5j5lvvi0jX2jJFG+ug Hvfw== X-Forwarded-Encrypted: i=1; AJvYcCXwTJI/vzOY/1kQ58WiiFPSlwDRsKEoedjPI2n5ZzcHZz0o0FvPSD7FMO3K1/SZ6IXV3pxbp26mcVKuD24=@vger.kernel.org X-Gm-Message-State: AOJu0YxUsn1gQld3zrg6rvPSJziIDmrhuR+pfC7LoWNQWXSmohYaoVPo 7oSI7ug0iW4I8NJMYOlMzkCQyt2hp+V+zBrNVybQiH6kD0cjYpgiGMCdh7Cn7J5WzzKnhY5fdEB i5SZIFMd4CTMvC5oUlhAsD04Ko9Ntguap3kWNYWxb8Ws2HP2RBMquPUgErJ5WRITfrrMu77BloX PDwssV3Qsj17U4+db6ufLh8dzLhEID2ZB9Fg2pfwWXxXEEkbiRXAFTdMmKEdhMyksZLyjuN8nea hR98HIHVsty2UweKAKmdlNqkA== X-Gm-Gg: AY/fxX7e5w6xQlN2SIFqZJVy6tGAHBkv5ahPLw+GYuB48KXbzANmdVVFcGM+Ed/nnGc 5iSXGkGknRUbXZBR0dS8l8skGxyPJw/TvdngbfhW5nD2pgd9xZQSknfpc4WL2Ph+bEQ5hADVYvi Ig1o14/puOi5qwJCm8OeEjOuu1jqPUDV9Xaa7cx65l+OOIvjc5QeinZs4gcD/VIONuhckHgV0nb SDp8godITjP+TcLhguI4qozsIkfNC8aKMZcL8q7X74NfLtMK5DnfjV88YPZUM2UyMaZ4I5YKfpo nDCQqDfX4n1dPkP7eOMCouycbm1lBStdX1AADG4x6zrj7f/H5l+5pPhfaUo9kUUnYQK34yjCDNB SJ1b+Trrndmr/iQYJ8xuBZATlZ5oyj2OZ480PPXlQpQyuKBM3d2XCH1y1FNzqW9OpB0y9pF3CpY CFFovm2jmZImdfazuai7Ud1S95xFtH3HsrV28g/1vf8sPWR4se/hDAWQ== X-Received: by 2002:a17:903:1210:b0:2a0:d692:5681 with SMTP id d9443c01a7336-2a717555332mr42111615ad.24.1768592299031; Fri, 16 Jan 2026 11:38:19 -0800 (PST) Received: from smtp-us-east1-p01-i01-si01.dlp.protect.broadcom.com (address-144-49-247-118.dlp.protect.broadcom.com. [144.49.247.118]) by smtp-relay.gmail.com with ESMTPS id d9443c01a7336-2a7193ac321sm4117515ad.60.2026.01.16.11.38.18 for (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Fri, 16 Jan 2026 11:38:19 -0800 (PST) X-Relaying-Domain: broadcom.com X-CFilter-Loop: Reflected Received: by mail-pj1-f69.google.com with SMTP id 98e67ed59e1d1-34c21341f56so4507832a91.2 for ; Fri, 16 Jan 2026 11:38:18 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=broadcom.com; s=google; t=1768592297; x=1769197097; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=5RE1Ydonxn/EbzGQfkPi595Jg1sH7wQCQVQgQmIM5tc=; b=K7s0puAsd4bfRxja1j2fw0QkNeMqGp/1bjhILnvCEDkBtbChCPDqa4fDHzFwekIQmn WcCdhdB0LNGn4Xpd/HtBvWprsldL8z3Xu3sIGy/mUhcIuTgLz8xAmNYBl4an6HWWnU5N KIJsBUcjX+Kfu9UGMZXA8P5fdiw1QxrfU85LE= X-Forwarded-Encrypted: i=1; AJvYcCVH0LfD/5tQTrVXW1Zb7Kz7PxuGR6GZ5iWJq4EcNsrggJqAbYvYKvm9qAUGqYRWlcvttlXpmz6Fshe4PQI=@vger.kernel.org X-Received: by 2002:a17:90b:3890:b0:34c:99d6:175d with SMTP id 98e67ed59e1d1-35272ec5844mr3472094a91.2.1768592297252; Fri, 16 Jan 2026 11:38:17 -0800 (PST) X-Received: by 2002:a17:90b:3890:b0:34c:99d6:175d with SMTP id 98e67ed59e1d1-35272ec5844mr3472078a91.2.1768592296857; Fri, 16 Jan 2026 11:38:16 -0800 (PST) Received: from localhost.localdomain ([192.19.203.250]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-35273121856sm2764909a91.15.2026.01.16.11.38.12 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 16 Jan 2026 11:38:16 -0800 (PST) From: Bhargava Marreddy To: davem@davemloft.net, edumazet@google.com, kuba@kernel.org, pabeni@redhat.com, andrew+netdev@lunn.ch, horms@kernel.org Cc: netdev@vger.kernel.org, linux-kernel@vger.kernel.org, michael.chan@broadcom.com, pavan.chebbi@broadcom.com, vsrama-krishna.nemani@broadcom.com, vikas.gupta@broadcom.com, ajit.khaparde@broadcom.com, Bhargava Marreddy , Rajashekar Hudumula , Rahul Gupta Subject: [v5, net-next 4/8] bng_en: Add TX support Date: Sat, 17 Jan 2026 01:07:28 +0530 Message-ID: <20260116193732.157898-5-bhargava.marreddy@broadcom.com> X-Mailer: git-send-email 2.47.3 In-Reply-To: <20260116193732.157898-1-bhargava.marreddy@broadcom.com> References: <20260116193732.157898-1-bhargava.marreddy@broadcom.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-DetectorID-Processed: b00c1d49-9d2e-4205-b15f-d015386d3d5e Content-Type: text/plain; charset="utf-8" Add functions to support xmit along with TSO/GSO. Also, add functions to handle TX completion events in the NAPI context. This commit introduces the fundamental transmit data path Signed-off-by: Bhargava Marreddy Reviewed-by: Vikas Gupta Reviewed-by: Rajashekar Hudumula Reviewed-by: Ajit Kumar Khaparde Reviewed-by: Rahul Gupta --- .../net/ethernet/broadcom/bnge/bnge_netdev.c | 100 ++++- .../net/ethernet/broadcom/bnge/bnge_netdev.h | 3 + .../net/ethernet/broadcom/bnge/bnge_txrx.c | 366 +++++++++++++++++- .../net/ethernet/broadcom/bnge/bnge_txrx.h | 36 +- 4 files changed, 493 insertions(+), 12 deletions(-) diff --git a/drivers/net/ethernet/broadcom/bnge/bnge_netdev.c b/drivers/net= /ethernet/broadcom/bnge/bnge_netdev.c index 594e83759802..2c1df5d48b5e 100644 --- a/drivers/net/ethernet/broadcom/bnge/bnge_netdev.c +++ b/drivers/net/ethernet/broadcom/bnge/bnge_netdev.c @@ -393,9 +393,60 @@ static void bnge_free_rx_ring_pair_bufs(struct bnge_ne= t *bn) bnge_free_one_rx_ring_pair_bufs(bn, &bn->rx_ring[i]); } =20 +static void bnge_free_tx_skbs(struct bnge_net *bn) +{ + struct bnge_dev *bd =3D bn->bd; + u16 max_idx; + int i; + + max_idx =3D bn->tx_nr_pages * TX_DESC_CNT; + for (i =3D 0; i < bd->tx_nr_rings; i++) { + struct bnge_tx_ring_info *txr =3D &bn->tx_ring[i]; + int j; + + if (!txr->tx_buf_ring) + continue; + + for (j =3D 0; j < max_idx;) { + struct bnge_sw_tx_bd *tx_buf =3D &txr->tx_buf_ring[j]; + struct sk_buff *skb; + int k, last; + + skb =3D tx_buf->skb; + if (!skb) { + j++; + continue; + } + + tx_buf->skb =3D NULL; + + dma_unmap_single(bd->dev, + dma_unmap_addr(tx_buf, mapping), + skb_headlen(skb), + DMA_TO_DEVICE); + + last =3D tx_buf->nr_frags; + j +=3D 2; + for (k =3D 0; k < last; k++, j++) { + int ring_idx =3D j & bn->tx_ring_mask; + skb_frag_t *frag =3D &skb_shinfo(skb)->frags[k]; + + tx_buf =3D &txr->tx_buf_ring[ring_idx]; + dma_unmap_page(bd->dev, + dma_unmap_addr(tx_buf, mapping), + skb_frag_size(frag), + DMA_TO_DEVICE); + } + dev_kfree_skb(skb); + } + netdev_tx_reset_queue(netdev_get_tx_queue(bd->netdev, i)); + } +} + static void bnge_free_all_rings_bufs(struct bnge_net *bn) { bnge_free_rx_ring_pair_bufs(bn); + bnge_free_tx_skbs(bn); } =20 static void bnge_free_rx_rings(struct bnge_net *bn) @@ -1829,6 +1880,8 @@ static void bnge_enable_napi(struct bnge_net *bn) for (i =3D 0; i < bd->nq_nr_rings; i++) { struct bnge_napi *bnapi =3D bn->bnapi[i]; =20 + bnapi->tx_fault =3D 0; + napi_enable_locked(&bnapi->napi); } } @@ -2235,6 +2288,42 @@ static int bnge_init_nic(struct bnge_net *bn) return rc; } =20 +static void bnge_tx_disable(struct bnge_net *bn) +{ + struct bnge_tx_ring_info *txr; + int i; + + if (bn->tx_ring) { + for (i =3D 0; i < bn->bd->tx_nr_rings; i++) { + txr =3D &bn->tx_ring[i]; + WRITE_ONCE(txr->dev_state, BNGE_DEV_STATE_CLOSING); + } + } + /* Make sure napi polls see @dev_state change */ + synchronize_net(); + + if (!bn->netdev) + return; + /* Drop carrier first to prevent TX timeout */ + netif_carrier_off(bn->netdev); + /* Stop all TX queues */ + netif_tx_disable(bn->netdev); +} + +static void bnge_tx_enable(struct bnge_net *bn) +{ + struct bnge_tx_ring_info *txr; + int i; + + for (i =3D 0; i < bn->bd->tx_nr_rings; i++) { + txr =3D &bn->tx_ring[i]; + WRITE_ONCE(txr->dev_state, 0); + } + /* Make sure napi polls see @dev_state change */ + synchronize_net(); + netif_tx_wake_all_queues(bn->netdev); +} + static int bnge_open_core(struct bnge_net *bn) { struct bnge_dev *bd =3D bn->bd; @@ -2272,6 +2361,8 @@ static int bnge_open_core(struct bnge_net *bn) set_bit(BNGE_STATE_OPEN, &bd->state); =20 bnge_enable_int(bn); + + bnge_tx_enable(bn); return 0; =20 err_free_irq: @@ -2282,13 +2373,6 @@ static int bnge_open_core(struct bnge_net *bn) return rc; } =20 -static netdev_tx_t bnge_start_xmit(struct sk_buff *skb, struct net_device = *dev) -{ - dev_kfree_skb_any(skb); - - return NETDEV_TX_OK; -} - static int bnge_open(struct net_device *dev) { struct bnge_net *bn =3D netdev_priv(dev); @@ -2311,6 +2395,8 @@ static void bnge_close_core(struct bnge_net *bn) { struct bnge_dev *bd =3D bn->bd; =20 + bnge_tx_disable(bn); + clear_bit(BNGE_STATE_OPEN, &bd->state); bnge_shutdown_nic(bn); bnge_disable_napi(bn); diff --git a/drivers/net/ethernet/broadcom/bnge/bnge_netdev.h b/drivers/net= /ethernet/broadcom/bnge/bnge_netdev.h index 4cc69b6cf30c..d7cef5c5ba68 100644 --- a/drivers/net/ethernet/broadcom/bnge/bnge_netdev.h +++ b/drivers/net/ethernet/broadcom/bnge/bnge_netdev.h @@ -243,6 +243,8 @@ struct bnge_net { =20 unsigned long state; #define BNGE_STATE_NAPI_DISABLED 0 + + u32 msg_enable; }; =20 #define BNGE_DEFAULT_RX_RING_SIZE 511 @@ -431,6 +433,7 @@ struct bnge_napi { #define BNGE_TX_EVENT 4 #define BNGE_REDIRECT_EVENT 8 #define BNGE_TX_CMP_EVENT 0x10 + u8 tx_fault:1; }; =20 #define INVALID_STATS_CTX_ID -1 diff --git a/drivers/net/ethernet/broadcom/bnge/bnge_txrx.c b/drivers/net/e= thernet/broadcom/bnge/bnge_txrx.c index 360ff6e2fa58..9e069faf2a58 100644 --- a/drivers/net/ethernet/broadcom/bnge/bnge_txrx.c +++ b/drivers/net/ethernet/broadcom/bnge/bnge_txrx.c @@ -50,6 +50,23 @@ static void bnge_sched_reset_rxr(struct bnge_net *bn, rxr->rx_next_cons =3D 0xffff; } =20 +static void bnge_sched_reset_txr(struct bnge_net *bn, + struct bnge_tx_ring_info *txr, + u16 curr) +{ + struct bnge_napi *bnapi =3D txr->bnapi; + + if (bnapi->tx_fault) + return; + + netdev_err(bn->netdev, "Invalid Tx completion (ring:%d tx_hw_cons:%u cons= :%u prod:%u curr:%u)", + txr->txq_index, txr->tx_hw_cons, + txr->tx_cons, txr->tx_prod, curr); + WARN_ON_ONCE(1); + bnapi->tx_fault =3D 1; + /* TODO: Initiate reset task */ +} + void bnge_reuse_rx_data(struct bnge_rx_ring_info *rxr, u16 cons, void *dat= a) { struct bnge_sw_rx_bd *cons_rx_buf, *prod_rx_buf; @@ -366,11 +383,86 @@ static int bnge_force_rx_discard(struct bnge_net *bn, return rc; } =20 +static void __bnge_tx_int(struct bnge_net *bn, struct bnge_tx_ring_info *t= xr, + int budget) +{ + u16 hw_cons =3D txr->tx_hw_cons; + struct bnge_dev *bd =3D bn->bd; + unsigned int tx_bytes =3D 0; + unsigned int tx_pkts =3D 0; + struct netdev_queue *txq; + u16 cons =3D txr->tx_cons; + skb_frag_t *frag; + + txq =3D netdev_get_tx_queue(bn->netdev, txr->txq_index); + + while (SW_TX_RING(bn, cons) !=3D hw_cons) { + struct bnge_sw_tx_bd *tx_buf; + struct sk_buff *skb; + int j, last; + + tx_buf =3D &txr->tx_buf_ring[SW_TX_RING(bn, cons)]; + skb =3D tx_buf->skb; + if (unlikely(!skb)) { + bnge_sched_reset_txr(bn, txr, cons); + return; + } + + cons =3D NEXT_TX(cons); + tx_pkts++; + tx_bytes +=3D skb->len; + tx_buf->skb =3D NULL; + + dma_unmap_single(bd->dev, dma_unmap_addr(tx_buf, mapping), + skb_headlen(skb), DMA_TO_DEVICE); + last =3D tx_buf->nr_frags; + + for (j =3D 0; j < last; j++) { + frag =3D &skb_shinfo(skb)->frags[j]; + cons =3D NEXT_TX(cons); + tx_buf =3D &txr->tx_buf_ring[SW_TX_RING(bn, cons)]; + netmem_dma_unmap_page_attrs(bd->dev, + dma_unmap_addr(tx_buf, + mapping), + skb_frag_size(frag), + DMA_TO_DEVICE, 0); + } + + cons =3D NEXT_TX(cons); + + napi_consume_skb(skb, budget); + } + + WRITE_ONCE(txr->tx_cons, cons); + + __netif_txq_completed_wake(txq, tx_pkts, tx_bytes, + bnge_tx_avail(bn, txr), bn->tx_wake_thresh, + (READ_ONCE(txr->dev_state) =3D=3D + BNGE_DEV_STATE_CLOSING)); +} + +static void bnge_tx_int(struct bnge_net *bn, struct bnge_napi *bnapi, + int budget) +{ + struct bnge_tx_ring_info *txr; + int i; + + bnge_for_each_napi_tx(i, bnapi, txr) { + if (txr->tx_hw_cons !=3D SW_TX_RING(bn, txr->tx_cons)) + __bnge_tx_int(bn, txr, budget); + } + + bnapi->events &=3D ~BNGE_TX_CMP_EVENT; +} + static void __bnge_poll_work_done(struct bnge_net *bn, struct bnge_napi *b= napi, int budget) { struct bnge_rx_ring_info *rxr =3D bnapi->rx_ring; =20 + if ((bnapi->events & BNGE_TX_CMP_EVENT) && !bnapi->tx_fault) + bnge_tx_int(bn, bnapi, budget); + if ((bnapi->events & BNGE_RX_EVENT)) { bnge_db_write(bn->bd, &rxr->rx_db, rxr->rx_prod); bnapi->events &=3D ~BNGE_RX_EVENT; @@ -443,9 +535,26 @@ static int __bnge_poll_work(struct bnge_net *bn, struc= t bnge_cp_ring_info *cpr, cmp_type =3D TX_CMP_TYPE(txcmp); if (cmp_type =3D=3D CMP_TYPE_TX_L2_CMP || cmp_type =3D=3D CMP_TYPE_TX_L2_COAL_CMP) { - /* - * Tx Compl Processing - */ + u32 opaque =3D txcmp->tx_cmp_opaque; + struct bnge_tx_ring_info *txr; + u16 tx_freed; + + txr =3D bnapi->tx_ring[TX_OPAQUE_RING(opaque)]; + event |=3D BNGE_TX_CMP_EVENT; + if (cmp_type =3D=3D CMP_TYPE_TX_L2_COAL_CMP) + txr->tx_hw_cons =3D TX_CMP_SQ_CONS_IDX(txcmp); + else + txr->tx_hw_cons =3D TX_OPAQUE_PROD(bn, opaque); + tx_freed =3D ((txr->tx_hw_cons - txr->tx_cons) & + bn->tx_ring_mask); + /* return full budget so NAPI will complete. */ + if (unlikely(tx_freed >=3D bn->tx_wake_thresh)) { + rx_pkts =3D budget; + raw_cons =3D NEXT_RAW_CMP(raw_cons); + if (budget) + cpr->has_more_work =3D 1; + break; + } } else if (cmp_type >=3D CMP_TYPE_RX_L2_CMP && cmp_type <=3D CMP_TYPE_RX_L2_TPA_START_V3_CMP) { if (likely(budget)) @@ -600,3 +709,254 @@ int bnge_napi_poll(struct napi_struct *napi, int budg= et) poll_done: return work_done; } + +static u16 bnge_xmit_get_cfa_action(struct sk_buff *skb) +{ + struct metadata_dst *md_dst =3D skb_metadata_dst(skb); + + if (!md_dst || md_dst->type !=3D METADATA_HW_PORT_MUX) + return 0; + + return md_dst->u.port_info.port_id; +} + +static const u16 bnge_lhint_arr[] =3D { + TX_BD_FLAGS_LHINT_512_AND_SMALLER, + TX_BD_FLAGS_LHINT_512_TO_1023, + TX_BD_FLAGS_LHINT_1024_TO_2047, + TX_BD_FLAGS_LHINT_1024_TO_2047, + TX_BD_FLAGS_LHINT_2048_AND_LARGER, + TX_BD_FLAGS_LHINT_2048_AND_LARGER, + TX_BD_FLAGS_LHINT_2048_AND_LARGER, + TX_BD_FLAGS_LHINT_2048_AND_LARGER, + TX_BD_FLAGS_LHINT_2048_AND_LARGER, + TX_BD_FLAGS_LHINT_2048_AND_LARGER, + TX_BD_FLAGS_LHINT_2048_AND_LARGER, + TX_BD_FLAGS_LHINT_2048_AND_LARGER, + TX_BD_FLAGS_LHINT_2048_AND_LARGER, + TX_BD_FLAGS_LHINT_2048_AND_LARGER, + TX_BD_FLAGS_LHINT_2048_AND_LARGER, + TX_BD_FLAGS_LHINT_2048_AND_LARGER, + TX_BD_FLAGS_LHINT_2048_AND_LARGER, + TX_BD_FLAGS_LHINT_2048_AND_LARGER, + TX_BD_FLAGS_LHINT_2048_AND_LARGER, +}; + +static void bnge_txr_db_kick(struct bnge_net *bn, struct bnge_tx_ring_info= *txr, + u16 prod) +{ + /* Sync BD data before updating doorbell */ + wmb(); + bnge_db_write(bn->bd, &txr->tx_db, prod); + txr->kick_pending =3D 0; +} + +static u32 bnge_get_gso_hdr_len(struct sk_buff *skb) +{ + bool udp_gso =3D !!(skb_shinfo(skb)->gso_type & SKB_GSO_UDP_L4); + u32 hdr_len; + + if (skb->encapsulation) { + if (udp_gso) + hdr_len =3D skb_inner_transport_offset(skb) + + sizeof(struct udphdr); + else + hdr_len =3D skb_inner_tcp_all_headers(skb); + } else if (udp_gso) { + hdr_len =3D skb_transport_offset(skb) + sizeof(struct udphdr); + } else { + hdr_len =3D skb_tcp_all_headers(skb); + } + + return hdr_len; +} + +netdev_tx_t bnge_start_xmit(struct sk_buff *skb, struct net_device *dev) +{ + u32 len, free_size, vlan_tag_flags, cfa_action, flags; + struct bnge_net *bn =3D netdev_priv(dev); + struct bnge_tx_ring_info *txr; + struct bnge_dev *bd =3D bn->bd; + struct bnge_sw_tx_bd *tx_buf; + struct tx_bd *txbd, *txbd0; + struct netdev_queue *txq; + struct tx_bd_ext *txbd1; + u16 prod, last_frag; + unsigned int length; + dma_addr_t mapping; + __le32 lflags =3D 0; + skb_frag_t *frag; + int i; + + i =3D skb_get_queue_mapping(skb); + txq =3D netdev_get_tx_queue(dev, i); + txr =3D &bn->tx_ring[bn->tx_ring_map[i]]; + prod =3D txr->tx_prod; + + free_size =3D bnge_tx_avail(bn, txr); + if (unlikely(free_size < skb_shinfo(skb)->nr_frags + 2)) { + /* We must have raced with NAPI cleanup */ + if (net_ratelimit() && txr->kick_pending) + netif_warn(bn, tx_err, dev, + "bnge: ring busy w/ flush pending!\n"); + if (!netif_txq_try_stop(txq, bnge_tx_avail(bn, txr), + bn->tx_wake_thresh)) + return NETDEV_TX_BUSY; + } + + if (unlikely(ipv6_hopopt_jumbo_remove(skb))) + goto tx_free; + + last_frag =3D skb_shinfo(skb)->nr_frags; + + txbd =3D &txr->tx_desc_ring[TX_RING(bn, prod)][TX_IDX(prod)]; + + tx_buf =3D &txr->tx_buf_ring[SW_TX_RING(bn, prod)]; + tx_buf->skb =3D skb; + tx_buf->nr_frags =3D last_frag; + + vlan_tag_flags =3D 0; + cfa_action =3D bnge_xmit_get_cfa_action(skb); + if (skb_vlan_tag_present(skb)) { + vlan_tag_flags =3D TX_BD_CFA_META_KEY_VLAN | + skb_vlan_tag_get(skb); + /* Currently supports 8021Q, 8021AD vlan offloads + * QINQ1, QINQ2, QINQ3 vlan headers are deprecated + */ + if (skb->vlan_proto =3D=3D htons(ETH_P_8021Q)) + vlan_tag_flags |=3D 1 << TX_BD_CFA_META_TPID_SHIFT; + } + + if (unlikely(skb->no_fcs)) + lflags |=3D cpu_to_le32(TX_BD_FLAGS_NO_CRC); + + if (eth_skb_pad(skb)) + goto tx_kick_pending; + + len =3D skb_headlen(skb); + + mapping =3D dma_map_single(bd->dev, skb->data, len, DMA_TO_DEVICE); + + if (unlikely(dma_mapping_error(bd->dev, mapping))) + goto tx_free; + + dma_unmap_addr_set(tx_buf, mapping, mapping); + flags =3D (len << TX_BD_LEN_SHIFT) | TX_BD_TYPE_LONG_TX_BD | + TX_BD_CNT(last_frag + 2); + + txbd->tx_bd_haddr =3D cpu_to_le64(mapping); + txbd->tx_bd_opaque =3D SET_TX_OPAQUE(bn, txr, prod, 2 + last_frag); + + prod =3D NEXT_TX(prod); + txbd1 =3D (struct tx_bd_ext *) + &txr->tx_desc_ring[TX_RING(bn, prod)][TX_IDX(prod)]; + + if (skb_is_gso(skb)) { + u32 hdr_len =3D bnge_get_gso_hdr_len(skb); + + lflags |=3D cpu_to_le32(TX_BD_FLAGS_LSO | TX_BD_FLAGS_T_IPID | + (hdr_len << (TX_BD_HSIZE_SHIFT - 1))); + length =3D skb_shinfo(skb)->gso_size; + txbd1->tx_bd_mss =3D cpu_to_le32(length); + length +=3D hdr_len; + } else { + length =3D skb->len; + if (skb->ip_summed =3D=3D CHECKSUM_PARTIAL) { + lflags |=3D cpu_to_le32(TX_BD_FLAGS_TCP_UDP_CHKSUM); + txbd1->tx_bd_mss =3D 0; + } + } + + flags |=3D bnge_lhint_arr[length >> 9]; + + txbd->tx_bd_len_flags_type =3D cpu_to_le32(flags); + txbd1->tx_bd_hsize_lflags =3D lflags; + txbd1->tx_bd_cfa_meta =3D cpu_to_le32(vlan_tag_flags); + txbd1->tx_bd_cfa_action =3D + cpu_to_le32(cfa_action << TX_BD_CFA_ACTION_SHIFT); + txbd0 =3D txbd; + for (i =3D 0; i < last_frag; i++) { + frag =3D &skb_shinfo(skb)->frags[i]; + + prod =3D NEXT_TX(prod); + txbd =3D &txr->tx_desc_ring[TX_RING(bn, prod)][TX_IDX(prod)]; + + len =3D skb_frag_size(frag); + mapping =3D skb_frag_dma_map(bd->dev, frag, 0, len, + DMA_TO_DEVICE); + + if (unlikely(dma_mapping_error(bd->dev, mapping))) + goto tx_dma_error; + + tx_buf =3D &txr->tx_buf_ring[SW_TX_RING(bn, prod)]; + netmem_dma_unmap_addr_set(skb_frag_netmem(frag), tx_buf, + mapping, mapping); + + txbd->tx_bd_haddr =3D cpu_to_le64(mapping); + + flags =3D len << TX_BD_LEN_SHIFT; + txbd->tx_bd_len_flags_type =3D cpu_to_le32(flags); + } + + flags &=3D ~TX_BD_LEN; + txbd->tx_bd_len_flags_type =3D + cpu_to_le32(((len) << TX_BD_LEN_SHIFT) | flags | + TX_BD_FLAGS_PACKET_END); + + netdev_tx_sent_queue(txq, skb->len); + + prod =3D NEXT_TX(prod); + WRITE_ONCE(txr->tx_prod, prod); + + if (!netdev_xmit_more() || netif_xmit_stopped(txq)) { + bnge_txr_db_kick(bn, txr, prod); + } else { + if (free_size >=3D bn->tx_wake_thresh) + txbd0->tx_bd_len_flags_type |=3D + cpu_to_le32(TX_BD_FLAGS_NO_CMPL); + txr->kick_pending =3D 1; + } + + if (unlikely(bnge_tx_avail(bn, txr) <=3D MAX_SKB_FRAGS + 1)) { + if (netdev_xmit_more()) { + txbd0->tx_bd_len_flags_type &=3D + cpu_to_le32(~TX_BD_FLAGS_NO_CMPL); + bnge_txr_db_kick(bn, txr, prod); + } + + netif_txq_try_stop(txq, bnge_tx_avail(bn, txr), + bn->tx_wake_thresh); + } + return NETDEV_TX_OK; + +tx_dma_error: + last_frag =3D i; + + /* start back at beginning and unmap skb */ + prod =3D txr->tx_prod; + tx_buf =3D &txr->tx_buf_ring[SW_TX_RING(bn, prod)]; + dma_unmap_single(bd->dev, dma_unmap_addr(tx_buf, mapping), + skb_headlen(skb), DMA_TO_DEVICE); + prod =3D NEXT_TX(prod); + + /* unmap remaining mapped pages */ + for (i =3D 0; i < last_frag; i++) { + prod =3D NEXT_TX(prod); + tx_buf =3D &txr->tx_buf_ring[SW_TX_RING(bn, prod)]; + frag =3D &skb_shinfo(skb)->frags[i]; + netmem_dma_unmap_page_attrs(bd->dev, + dma_unmap_addr(tx_buf, mapping), + skb_frag_size(frag), + DMA_TO_DEVICE, 0); + } + +tx_free: + dev_kfree_skb_any(skb); + +tx_kick_pending: + if (txr->kick_pending) + bnge_txr_db_kick(bn, txr, txr->tx_prod); + txr->tx_buf_ring[SW_TX_RING(bn, txr->tx_prod)].skb =3D NULL; + dev_core_stats_tx_dropped_inc(dev); + return NETDEV_TX_OK; +} diff --git a/drivers/net/ethernet/broadcom/bnge/bnge_txrx.h b/drivers/net/e= thernet/broadcom/bnge/bnge_txrx.h index f5aa5ac888a9..81a24d8f9689 100644 --- a/drivers/net/ethernet/broadcom/bnge/bnge_txrx.h +++ b/drivers/net/ethernet/broadcom/bnge/bnge_txrx.h @@ -7,7 +7,33 @@ #include #include "bnge_netdev.h" =20 -#define BNGE_MIN_PKT_SIZE 52 +static inline u32 bnge_tx_avail(struct bnge_net *bn, + const struct bnge_tx_ring_info *txr) +{ + u32 used =3D READ_ONCE(txr->tx_prod) - READ_ONCE(txr->tx_cons); + + return bn->tx_ring_size - (used & bn->tx_ring_mask); +} + +static inline void bnge_writeq_relaxed(struct bnge_dev *bd, u64 val, + void __iomem *addr) +{ +#if BITS_PER_LONG =3D=3D 32 + spin_lock(&bd->db_lock); + lo_hi_writeq_relaxed(val, addr); + spin_unlock(&bd->db_lock); +#else + writeq_relaxed(val, addr); +#endif +} + +/* For TX and RX ring doorbells with no ordering guarantee*/ +static inline void bnge_db_write_relaxed(struct bnge_net *bn, + struct bnge_db_info *db, u32 idx) +{ + bnge_writeq_relaxed(bn->bd, db->db_key64 | DB_RING_IDX(db, idx), + db->doorbell); +} =20 #define TX_OPAQUE_IDX_MASK 0x0000ffff #define TX_OPAQUE_BDS_MASK 0x00ff0000 @@ -26,6 +52,11 @@ TX_OPAQUE_BDS_SHIFT) #define TX_OPAQUE_PROD(bn, opq) ((TX_OPAQUE_IDX(opq) + TX_OPAQUE_BDS(opq))= &\ (bn)->tx_ring_mask) +#define TX_BD_CNT(n) (((n) << TX_BD_FLAGS_BD_CNT_SHIFT) & TX_BD_FLAGS_BD_C= NT) + +#define TX_MAX_BD_CNT 32 + +#define TX_MAX_FRAGS (TX_MAX_BD_CNT - 2) =20 /* Minimum TX BDs for a TX packet with MAX_SKB_FRAGS + 1. We need one ext= ra * BD because the first TX BD is always a long BD. @@ -70,7 +101,7 @@ #define RING_RX_AGG(bn, idx) ((idx) & (bn)->rx_agg_ring_mask) #define NEXT_RX_AGG(idx) ((idx) + 1) =20 -#define RING_TX(bn, idx) ((idx) & (bn)->tx_ring_mask) +#define SW_TX_RING(bn, idx) ((idx) & (bn)->tx_ring_mask) #define NEXT_TX(idx) ((idx) + 1) =20 #define ADV_RAW_CMP(idx, n) ((idx) + (n)) @@ -85,6 +116,7 @@ RX_CMPL_CFA_CODE_MASK) >> RX_CMPL_CFA_CODE_SFT) =20 irqreturn_t bnge_msix(int irq, void *dev_instance); +netdev_tx_t bnge_start_xmit(struct sk_buff *skb, struct net_device *dev); void bnge_reuse_rx_data(struct bnge_rx_ring_info *rxr, u16 cons, void *dat= a); int bnge_napi_poll(struct napi_struct *napi, int budget); #endif /* _BNGE_TXRX_H_ */ --=20 2.47.3