From nobody Tue Dec 16 22:20:53 2025 Received: from lelvem-ot01.ext.ti.com (lelvem-ot01.ext.ti.com [198.47.23.234]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A44941DFE3A; Mon, 10 Feb 2025 10:34:53 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.47.23.234 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739183695; cv=none; b=Scch/Px3JOdIkX02ZUgZoXZXOkRr8JguGPyKgOU8paONltfZpTx3AE6MM9Q3RLKRfD1JCtSFwBrx9qTFXmO8rZatFzttAdfrTHRUJpz135BUMTpVO5xLHY4fuE770SwPI3IDeAzq/+2OgQd6ctYfnvgWAOhFWDiXEHqbr1zPN2E= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739183695; c=relaxed/simple; bh=Z6A6zF8Hp2h1bFsk5192yZ4ox6u4NpMWcR7fNQ7REK8=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=afrLZkrZ29U125J3qwaw+nNCQ/Q+O2UugHaPFJjxc/Eia6EIN6iL4bwoDWSKuvatTNKnzlaVCwlTEmAMGbQB4LtogREmktStyMQzoheYyklC3L9zkPYKMd+614+vA0l99hEgg9oFRnvwArB20i6ib/BQPDcRWs2Dqk1O9Km7mC4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=ti.com; spf=pass smtp.mailfrom=ti.com; dkim=pass (1024-bit key) header.d=ti.com header.i=@ti.com header.b=nLT2DV2C; arc=none smtp.client-ip=198.47.23.234 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=ti.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=ti.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=ti.com header.i=@ti.com header.b="nLT2DV2C" Received: from fllv0034.itg.ti.com ([10.64.40.246]) by lelvem-ot01.ext.ti.com (8.15.2/8.15.2) with ESMTPS id 51AAY110110659 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 10 Feb 2025 04:34:01 -0600 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ti.com; s=ti-com-17Q1; t=1739183642; bh=SMSCCsPVLl4FCkEwkABBDUvCPXaDMGUKXTpMQV6GSNw=; h=From:To:CC:Subject:Date:In-Reply-To:References; b=nLT2DV2C9wYS+zEnCrhbKSSOpkwdq+vxUIkLvI9clafzY+sWag9mv3XZq7F0nyC/T DCYye7/vjI0eJ1Yg5xKBzQjUDOJUuKSkQWYQq5NecFWQ3iKFMythCKlkKpduny2Af1 cT+r7DbFyv0TGM8Sc7KJrtdoSIIaOp+sd3hynBxg= Received: from DLEE103.ent.ti.com (dlee103.ent.ti.com [157.170.170.33]) by fllv0034.itg.ti.com (8.15.2/8.15.2) with ESMTPS id 51AAY1wr108079 (version=TLSv1.2 cipher=AES256-GCM-SHA384 bits=256 verify=FAIL); Mon, 10 Feb 2025 04:34:01 -0600 Received: from DLEE112.ent.ti.com (157.170.170.23) by DLEE103.ent.ti.com (157.170.170.33) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.2507.23; Mon, 10 Feb 2025 04:34:01 -0600 Received: from lelvsmtp6.itg.ti.com (10.180.75.249) by DLEE112.ent.ti.com (157.170.170.23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.2507.23 via Frontend Transport; Mon, 10 Feb 2025 04:34:01 -0600 Received: from fllv0122.itg.ti.com (fllv0122.itg.ti.com [10.247.120.72]) by lelvsmtp6.itg.ti.com (8.15.2/8.15.2) with ESMTP id 51AAY1au123055; Mon, 10 Feb 2025 04:34:01 -0600 Received: from localhost (meghana-pc.dhcp.ti.com [10.24.69.13] (may be forged)) by fllv0122.itg.ti.com (8.14.7/8.14.7) with ESMTP id 51AAY0mD021832; Mon, 10 Feb 2025 04:34:00 -0600 From: Meghana Malladi To: , , , , , , CC: , , , , , , , , , , , , , , , , , , Vignesh Raghavendra Subject: [PATCH net-next v2 1/3] net: ti: icssg-prueth: Use page_pool API for RX buffer allocation Date: Mon, 10 Feb 2025 16:03:50 +0530 Message-ID: <20250210103352.541052-2-m-malladi@ti.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250210103352.541052-1-m-malladi@ti.com> References: <20250210103352.541052-1-m-malladi@ti.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-C2ProcessedOrg: 333ef613-75bf-4e12-a4b1-8e3623f5dcea Content-Type: text/plain; charset="utf-8" From: Roger Quadros This is to prepare for native XDP support. The page pool API is more faster in allocating pages than __alloc_skb(). Drawback is that it works at PAGE_SIZE granularity so we are not efficient in memory usage. i.e. we are using PAGE_SIZE (4KB) memory for 1.5KB max packet size. Signed-off-by: Roger Quadros Signed-off-by: MD Danish Anwar Signed-off-by: Meghana Malladi --- v1: https://lore.kernel.org/all/20250122124951.3072410-1-m-malladi@ti.com/ Changes since v1 (v2-v1): - Recycle pages wherever necessary using skb_mark_for_recycle() - Use napi_build_skb() instead of build_skb() - Update with correct frag_size argument in napi_build_skb() - Use napi_gro_receive() instead of netif_receive_skb() - Use PP_FLAG_DMA_SYNC_DEV to enable DMA sync with device - Use page_pool_dma_sync_for_cpu() to sync Rx page pool for CPU All the above changes have been suggested by Ido Schimmel drivers/net/ethernet/ti/Kconfig | 1 + drivers/net/ethernet/ti/icssg/icssg_common.c | 174 ++++++++++++------ drivers/net/ethernet/ti/icssg/icssg_prueth.h | 14 +- .../net/ethernet/ti/icssg/icssg_prueth_sr1.c | 21 ++- 4 files changed, 140 insertions(+), 70 deletions(-) diff --git a/drivers/net/ethernet/ti/Kconfig b/drivers/net/ethernet/ti/Kcon= fig index 0d5a862cd78a..b461281d31b6 100644 --- a/drivers/net/ethernet/ti/Kconfig +++ b/drivers/net/ethernet/ti/Kconfig @@ -204,6 +204,7 @@ config TI_ICSSG_PRUETH_SR1 select PHYLIB select TI_ICSS_IEP select TI_K3_CPPI_DESC_POOL + select PAGE_POOL depends on PRU_REMOTEPROC depends on NET_SWITCHDEV depends on ARCH_K3 && OF && TI_K3_UDMA_GLUE_LAYER diff --git a/drivers/net/ethernet/ti/icssg/icssg_common.c b/drivers/net/eth= ernet/ti/icssg/icssg_common.c index 74f0f200a89d..c3c1e2bf461e 100644 --- a/drivers/net/ethernet/ti/icssg/icssg_common.c +++ b/drivers/net/ethernet/ti/icssg/icssg_common.c @@ -45,6 +45,11 @@ void prueth_cleanup_rx_chns(struct prueth_emac *emac, struct prueth_rx_chn *rx_chn, int max_rflows) { + if (rx_chn->pg_pool) { + page_pool_destroy(rx_chn->pg_pool); + rx_chn->pg_pool =3D NULL; + } + if (rx_chn->desc_pool) k3_cppi_desc_pool_destroy(rx_chn->desc_pool); =20 @@ -461,17 +466,17 @@ int prueth_init_rx_chns(struct prueth_emac *emac, } EXPORT_SYMBOL_GPL(prueth_init_rx_chns); =20 -int prueth_dma_rx_push(struct prueth_emac *emac, - struct sk_buff *skb, - struct prueth_rx_chn *rx_chn) +int prueth_dma_rx_push_mapped(struct prueth_emac *emac, + struct prueth_rx_chn *rx_chn, + struct page *page, u32 buf_len) { struct net_device *ndev =3D emac->ndev; struct cppi5_host_desc_t *desc_rx; - u32 pkt_len =3D skb_tailroom(skb); dma_addr_t desc_dma; dma_addr_t buf_dma; void **swdata; =20 + buf_dma =3D page_pool_get_dma_addr(page) + PRUETH_HEADROOM; desc_rx =3D k3_cppi_desc_pool_alloc(rx_chn->desc_pool); if (!desc_rx) { netdev_err(ndev, "rx push: failed to allocate descriptor\n"); @@ -479,25 +484,18 @@ int prueth_dma_rx_push(struct prueth_emac *emac, } desc_dma =3D k3_cppi_desc_pool_virt2dma(rx_chn->desc_pool, desc_rx); =20 - buf_dma =3D dma_map_single(rx_chn->dma_dev, skb->data, pkt_len, DMA_FROM_= DEVICE); - if (unlikely(dma_mapping_error(rx_chn->dma_dev, buf_dma))) { - k3_cppi_desc_pool_free(rx_chn->desc_pool, desc_rx); - netdev_err(ndev, "rx push: failed to map rx pkt buffer\n"); - return -EINVAL; - } - cppi5_hdesc_init(desc_rx, CPPI5_INFO0_HDESC_EPIB_PRESENT, PRUETH_NAV_PS_DATA_SIZE); k3_udma_glue_rx_dma_to_cppi5_addr(rx_chn->rx_chn, &buf_dma); - cppi5_hdesc_attach_buf(desc_rx, buf_dma, skb_tailroom(skb), buf_dma, skb_= tailroom(skb)); + cppi5_hdesc_attach_buf(desc_rx, buf_dma, buf_len, buf_dma, buf_len); =20 swdata =3D cppi5_hdesc_get_swdata(desc_rx); - *swdata =3D skb; + *swdata =3D page; =20 - return k3_udma_glue_push_rx_chn(rx_chn->rx_chn, 0, + return k3_udma_glue_push_rx_chn(rx_chn->rx_chn, PRUETH_RX_FLOW_DATA, desc_rx, desc_dma); } -EXPORT_SYMBOL_GPL(prueth_dma_rx_push); +EXPORT_SYMBOL_GPL(prueth_dma_rx_push_mapped); =20 u64 icssg_ts_to_ns(u32 hi_sw, u32 hi, u32 lo, u32 cycle_time_ns) { @@ -541,12 +539,16 @@ static int emac_rx_packet(struct prueth_emac *emac, u= 32 flow_id) u32 buf_dma_len, pkt_len, port_id =3D 0; struct net_device *ndev =3D emac->ndev; struct cppi5_host_desc_t *desc_rx; - struct sk_buff *skb, *new_skb; dma_addr_t desc_dma, buf_dma; + struct page *page, *new_page; + struct page_pool *pool; + struct sk_buff *skb; void **swdata; u32 *psdata; + void *pa; int ret; =20 + pool =3D rx_chn->pg_pool; ret =3D k3_udma_glue_pop_rx_chn(rx_chn->rx_chn, flow_id, &desc_dma); if (ret) { if (ret !=3D -ENODATA) @@ -558,15 +560,10 @@ static int emac_rx_packet(struct prueth_emac *emac, u= 32 flow_id) return 0; =20 desc_rx =3D k3_cppi_desc_pool_dma2virt(rx_chn->desc_pool, desc_dma); - swdata =3D cppi5_hdesc_get_swdata(desc_rx); - skb =3D *swdata; - - psdata =3D cppi5_hdesc_get_psdata(desc_rx); - /* RX HW timestamp */ - if (emac->rx_ts_enabled) - emac_rx_timestamp(emac, skb, psdata); + page =3D *swdata; =20 + page_pool_dma_sync_for_cpu(pool, page, 0, PAGE_SIZE); cppi5_hdesc_get_obuf(desc_rx, &buf_dma, &buf_dma_len); k3_udma_glue_rx_cppi5_to_dma_addr(rx_chn->rx_chn, &buf_dma); pkt_len =3D cppi5_hdesc_get_pktlen(desc_rx); @@ -574,32 +571,51 @@ static int emac_rx_packet(struct prueth_emac *emac, u= 32 flow_id) pkt_len -=3D 4; cppi5_desc_get_tags_ids(&desc_rx->hdr, &port_id, NULL); =20 - dma_unmap_single(rx_chn->dma_dev, buf_dma, buf_dma_len, DMA_FROM_DEVICE); k3_cppi_desc_pool_free(rx_chn->desc_pool, desc_rx); =20 - skb->dev =3D ndev; - new_skb =3D netdev_alloc_skb_ip_align(ndev, PRUETH_MAX_PKT_SIZE); /* if allocation fails we drop the packet but push the - * descriptor back to the ring with old skb to prevent a stall + * descriptor back to the ring with old page to prevent a stall */ - if (!new_skb) { + new_page =3D page_pool_dev_alloc_pages(pool); + if (unlikely(!new_page)) { + new_page =3D page; ndev->stats.rx_dropped++; - new_skb =3D skb; - } else { - /* send the filled skb up the n/w stack */ - skb_put(skb, pkt_len); - if (emac->prueth->is_switch_mode) - skb->offload_fwd_mark =3D emac->offload_fwd_mark; - skb->protocol =3D eth_type_trans(skb, ndev); - napi_gro_receive(&emac->napi_rx, skb); - ndev->stats.rx_bytes +=3D pkt_len; - ndev->stats.rx_packets++; + goto requeue; + } + + /* prepare skb and send to n/w stack */ + pa =3D page_address(page); + skb =3D napi_build_skb(pa, PAGE_SIZE); + if (!skb) { + ndev->stats.rx_dropped++; + page_pool_recycle_direct(pool, page); + goto requeue; } =20 + skb_reserve(skb, PRUETH_HEADROOM); + skb_put(skb, pkt_len); + skb->dev =3D ndev; + + psdata =3D cppi5_hdesc_get_psdata(desc_rx); + /* RX HW timestamp */ + if (emac->rx_ts_enabled) + emac_rx_timestamp(emac, skb, psdata); + + if (emac->prueth->is_switch_mode) + skb->offload_fwd_mark =3D emac->offload_fwd_mark; + skb->protocol =3D eth_type_trans(skb, ndev); + + skb_mark_for_recycle(skb); + napi_gro_receive(&emac->napi_rx, skb); + ndev->stats.rx_bytes +=3D pkt_len; + ndev->stats.rx_packets++; + +requeue: /* queue another RX DMA */ - ret =3D prueth_dma_rx_push(emac, new_skb, &emac->rx_chns); + ret =3D prueth_dma_rx_push_mapped(emac, &emac->rx_chns, new_page, + PRUETH_MAX_PKT_SIZE); if (WARN_ON(ret < 0)) { - dev_kfree_skb_any(new_skb); + page_pool_recycle_direct(pool, new_page); ndev->stats.rx_errors++; ndev->stats.rx_dropped++; } @@ -611,22 +627,17 @@ static void prueth_rx_cleanup(void *data, dma_addr_t = desc_dma) { struct prueth_rx_chn *rx_chn =3D data; struct cppi5_host_desc_t *desc_rx; - struct sk_buff *skb; - dma_addr_t buf_dma; - u32 buf_dma_len; + struct page_pool *pool; + struct page *page; void **swdata; =20 + pool =3D rx_chn->pg_pool; + desc_rx =3D k3_cppi_desc_pool_dma2virt(rx_chn->desc_pool, desc_dma); swdata =3D cppi5_hdesc_get_swdata(desc_rx); - skb =3D *swdata; - cppi5_hdesc_get_obuf(desc_rx, &buf_dma, &buf_dma_len); - k3_udma_glue_rx_cppi5_to_dma_addr(rx_chn->rx_chn, &buf_dma); - - dma_unmap_single(rx_chn->dma_dev, buf_dma, buf_dma_len, - DMA_FROM_DEVICE); + page =3D *swdata; + page_pool_recycle_direct(pool, page); k3_cppi_desc_pool_free(rx_chn->desc_pool, desc_rx); - - dev_kfree_skb_any(skb); } =20 static int prueth_tx_ts_cookie_get(struct prueth_emac *emac) @@ -907,29 +918,71 @@ int icssg_napi_rx_poll(struct napi_struct *napi_rx, i= nt budget) } EXPORT_SYMBOL_GPL(icssg_napi_rx_poll); =20 +static struct page_pool *prueth_create_page_pool(struct prueth_emac *emac, + struct device *dma_dev, + int size) +{ + struct page_pool_params pp_params =3D { 0 }; + struct page_pool *pool; + + pp_params.order =3D 0; + pp_params.flags =3D PP_FLAG_DMA_MAP | PP_FLAG_DMA_SYNC_DEV; + pp_params.pool_size =3D size; + pp_params.nid =3D dev_to_node(emac->prueth->dev); + pp_params.dma_dir =3D DMA_BIDIRECTIONAL; + pp_params.dev =3D dma_dev; + pp_params.napi =3D &emac->napi_rx; + pp_params.max_len =3D PAGE_SIZE; + + pool =3D page_pool_create(&pp_params); + if (IS_ERR(pool)) + netdev_err(emac->ndev, "cannot create rx page pool\n"); + + return pool; +} + int prueth_prepare_rx_chan(struct prueth_emac *emac, struct prueth_rx_chn *chn, int buf_size) { - struct sk_buff *skb; + struct page_pool *pool; + struct page *page; int i, ret; =20 + pool =3D prueth_create_page_pool(emac, chn->dma_dev, chn->descs_num); + if (IS_ERR(pool)) + return PTR_ERR(pool); + + chn->pg_pool =3D pool; + for (i =3D 0; i < chn->descs_num; i++) { - skb =3D __netdev_alloc_skb_ip_align(NULL, buf_size, GFP_KERNEL); - if (!skb) - return -ENOMEM; + /* NOTE: we're not using memory efficiently here. + * 1 full page (4KB?) used here instead of + * PRUETH_MAX_PKT_SIZE (~1.5KB?) + */ + page =3D page_pool_dev_alloc_pages(pool); + if (!page) { + netdev_err(emac->ndev, "couldn't allocate rx page\n"); + ret =3D -ENOMEM; + goto recycle_alloc_pg; + } =20 - ret =3D prueth_dma_rx_push(emac, skb, chn); + ret =3D prueth_dma_rx_push_mapped(emac, chn, page, buf_size); if (ret < 0) { netdev_err(emac->ndev, - "cannot submit skb for rx chan %s ret %d\n", + "cannot submit page for rx chan %s ret %d\n", chn->name, ret); - kfree_skb(skb); - return ret; + page_pool_recycle_direct(pool, page); + goto recycle_alloc_pg; } } =20 return 0; + +recycle_alloc_pg: + prueth_reset_rx_chan(&emac->rx_chns, PRUETH_MAX_RX_FLOWS, false); + + return ret; } EXPORT_SYMBOL_GPL(prueth_prepare_rx_chan); =20 @@ -958,6 +1011,9 @@ void prueth_reset_rx_chan(struct prueth_rx_chn *chn, prueth_rx_cleanup, !!i); if (disable) k3_udma_glue_disable_rx_chn(chn->rx_chn); + + page_pool_destroy(chn->pg_pool); + chn->pg_pool =3D NULL; } EXPORT_SYMBOL_GPL(prueth_reset_rx_chan); =20 diff --git a/drivers/net/ethernet/ti/icssg/icssg_prueth.h b/drivers/net/eth= ernet/ti/icssg/icssg_prueth.h index 329b46e9ee53..c7b906de18af 100644 --- a/drivers/net/ethernet/ti/icssg/icssg_prueth.h +++ b/drivers/net/ethernet/ti/icssg/icssg_prueth.h @@ -33,6 +33,8 @@ #include =20 #include +#include +#include =20 #include "icssg_config.h" #include "icss_iep.h" @@ -131,6 +133,7 @@ struct prueth_rx_chn { u32 descs_num; unsigned int irq[ICSSG_MAX_RFLOWS]; /* separate irq per flow */ char name[32]; + struct page_pool *pg_pool; }; =20 /* There are 4 Tx DMA channels, but the highest priority is CH3 (thread 3) @@ -210,6 +213,10 @@ struct prueth_emac { struct netdev_hw_addr_list vlan_mcast_list[MAX_VLAN_ID]; }; =20 +/* The buf includes headroom compatible with both skb and xdpf */ +#define PRUETH_HEADROOM_NA (max(XDP_PACKET_HEADROOM, NET_SKB_PAD) + NET_IP= _ALIGN) +#define PRUETH_HEADROOM ALIGN(PRUETH_HEADROOM_NA, sizeof(long)) + /** * struct prueth_pdata - PRUeth platform data * @fdqring_mode: Free desc queue mode @@ -410,9 +417,10 @@ int prueth_init_rx_chns(struct prueth_emac *emac, struct prueth_rx_chn *rx_chn, char *name, u32 max_rflows, u32 max_desc_num); -int prueth_dma_rx_push(struct prueth_emac *emac, - struct sk_buff *skb, - struct prueth_rx_chn *rx_chn); +int prueth_dma_rx_push_mapped(struct prueth_emac *emac, + struct prueth_rx_chn *rx_chn, + struct page *page, u32 buf_len); +unsigned int prueth_rxbuf_total_len(unsigned int len); void emac_rx_timestamp(struct prueth_emac *emac, struct sk_buff *skb, u32 *psdata); enum netdev_tx icssg_ndo_start_xmit(struct sk_buff *skb, struct net_device= *ndev); diff --git a/drivers/net/ethernet/ti/icssg/icssg_prueth_sr1.c b/drivers/net= /ethernet/ti/icssg/icssg_prueth_sr1.c index 64a19ff39562..aeeb8a50376b 100644 --- a/drivers/net/ethernet/ti/icssg/icssg_prueth_sr1.c +++ b/drivers/net/ethernet/ti/icssg/icssg_prueth_sr1.c @@ -274,10 +274,12 @@ static struct sk_buff *prueth_process_rx_mgm(struct p= rueth_emac *emac, struct prueth_rx_chn *rx_chn =3D &emac->rx_mgm_chn; struct net_device *ndev =3D emac->ndev; struct cppi5_host_desc_t *desc_rx; - struct sk_buff *skb, *new_skb; + struct page *page, *new_page; dma_addr_t desc_dma, buf_dma; u32 buf_dma_len, pkt_len; + struct sk_buff *skb; void **swdata; + void *pa; int ret; =20 ret =3D k3_udma_glue_pop_rx_chn(rx_chn->rx_chn, flow_id, &desc_dma); @@ -299,32 +301,35 @@ static struct sk_buff *prueth_process_rx_mgm(struct p= rueth_emac *emac, } =20 swdata =3D cppi5_hdesc_get_swdata(desc_rx); - skb =3D *swdata; + page =3D *swdata; cppi5_hdesc_get_obuf(desc_rx, &buf_dma, &buf_dma_len); pkt_len =3D cppi5_hdesc_get_pktlen(desc_rx); =20 dma_unmap_single(rx_chn->dma_dev, buf_dma, buf_dma_len, DMA_FROM_DEVICE); k3_cppi_desc_pool_free(rx_chn->desc_pool, desc_rx); =20 - new_skb =3D netdev_alloc_skb_ip_align(ndev, PRUETH_MAX_PKT_SIZE); + new_page =3D page_pool_dev_alloc_pages(rx_chn->pg_pool); /* if allocation fails we drop the packet but push the * descriptor back to the ring with old skb to prevent a stall */ - if (!new_skb) { + if (!new_page) { netdev_err(ndev, - "skb alloc failed, dropped mgm pkt from flow %d\n", + "page alloc failed, dropped mgm pkt from flow %d\n", flow_id); - new_skb =3D skb; + new_page =3D page; skb =3D NULL; /* return NULL */ } else { /* return the filled skb */ + pa =3D page_address(page); + skb =3D napi_build_skb(pa, PAGE_SIZE); skb_put(skb, pkt_len); } =20 /* queue another DMA */ - ret =3D prueth_dma_rx_push(emac, new_skb, &emac->rx_mgm_chn); + ret =3D prueth_dma_rx_push_mapped(emac, &emac->rx_chns, new_page, + PRUETH_MAX_PKT_SIZE); if (WARN_ON(ret < 0)) - dev_kfree_skb_any(new_skb); + page_pool_recycle_direct(rx_chn->pg_pool, new_page); =20 return skb; } --=20 2.43.0 From nobody Tue Dec 16 22:20:53 2025 Received: from fllvem-ot04.ext.ti.com (fllvem-ot04.ext.ti.com [198.47.19.246]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 00FF117591; Mon, 10 Feb 2025 10:34:41 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.47.19.246 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739183684; cv=none; b=TpnHVf6lYUsJ0ehG1uO6RIZ6bIC1Gpv8wsOBb71bTQ5ySCIQ2Dth+SquOl/g/B+QiFb12Vw9kSAOQGLRbI27Bh73lOo7/OufvrZC3eVZy31KbOGPCHpGb9Oq5g8z9l9/d/ntPwdgbAyYNjBuWH6WmaPjsJkprlmX9qMUyGEMryM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739183684; c=relaxed/simple; bh=6xhQe58apX1iidhAWVxgEcf4RPIt8ZHWi7vpEI++4pQ=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=gGNWgyDAQqf0DZXe8+BGrPSi83OxaQt6MbTxHtMX56a1w8tcI/7LvFsBbQfC5JA2CLP4bhXuYueBQVYg8TQDphhvO86UDXGfMJHygeuLpZUO4HsS/2mvo8sbFHJeoF29lmuIgjqSP3u/gwHjZeyfCgjdgL6xVXaQl+ismbXp5Vw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=ti.com; spf=pass smtp.mailfrom=ti.com; dkim=pass (1024-bit key) header.d=ti.com header.i=@ti.com header.b=GSNf8YbI; arc=none smtp.client-ip=198.47.19.246 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=ti.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=ti.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=ti.com header.i=@ti.com header.b="GSNf8YbI" Received: from fllv0035.itg.ti.com ([10.64.41.0]) by fllvem-ot04.ext.ti.com (8.15.2/8.15.2) with ESMTPS id 51AAY5Vj3393387 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 10 Feb 2025 04:34:05 -0600 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ti.com; s=ti-com-17Q1; t=1739183645; bh=Sv4b5NVpXzCHisqEvUyRD23v7ufs1KnFG/uyf9a3DXQ=; h=From:To:CC:Subject:Date:In-Reply-To:References; b=GSNf8YbICAVn6Q7vYZf75yUMJUN30n/Bg5jSBe7QxwkC7iAEXRdlWc3vrZJof7fJb KOnHdYSvj/8Mu0STRRKCqaAG5mww9bV/gnSyMLlq1LlnbCEVoJ9jZmobn04YOkajhy XIgyvTNzyesiH30noEWvZmzURP/gFHmsLm/a6TFs= Received: from DFLE105.ent.ti.com (dfle105.ent.ti.com [10.64.6.26]) by fllv0035.itg.ti.com (8.15.2/8.15.2) with ESMTPS id 51AAY5PF074085 (version=TLSv1.2 cipher=AES256-GCM-SHA384 bits=256 verify=FAIL); Mon, 10 Feb 2025 04:34:05 -0600 Received: from DFLE107.ent.ti.com (10.64.6.28) by DFLE105.ent.ti.com (10.64.6.26) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.2507.23; Mon, 10 Feb 2025 04:34:04 -0600 Received: from fllvsmtp8.itg.ti.com (10.64.41.158) by DFLE107.ent.ti.com (10.64.6.28) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.2507.23 via Frontend Transport; Mon, 10 Feb 2025 04:34:04 -0600 Received: from fllv0122.itg.ti.com (fllv0122.itg.ti.com [10.247.120.72]) by fllvsmtp8.itg.ti.com (8.15.2/8.15.2) with ESMTP id 51AAY4hp109014; Mon, 10 Feb 2025 04:34:04 -0600 Received: from localhost (meghana-pc.dhcp.ti.com [10.24.69.13] (may be forged)) by fllv0122.itg.ti.com (8.14.7/8.14.7) with ESMTP id 51AAY4BV021838; Mon, 10 Feb 2025 04:34:04 -0600 From: Meghana Malladi To: , , , , , , CC: , , , , , , , , , , , , , , , , , , Vignesh Raghavendra Subject: [PATCH net-next v2 2/3] net: ti: icssg-prueth: introduce and use prueth_swdata struct for SWDATA Date: Mon, 10 Feb 2025 16:03:51 +0530 Message-ID: <20250210103352.541052-3-m-malladi@ti.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250210103352.541052-1-m-malladi@ti.com> References: <20250210103352.541052-1-m-malladi@ti.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-C2ProcessedOrg: 333ef613-75bf-4e12-a4b1-8e3623f5dcea Content-Type: text/plain; charset="utf-8" From: Roger Quadros We have different cases for SWDATA (skb, page, cmd, etc) so it is better to have a dedicated data structure for that. We can embed the type field inside the struct and use it to interpret the data in completion handlers. Increase SWDATA size to 48 so we have some room to add more data if required. Signed-off-by: Roger Quadros Signed-off-by: MD Danish Anwar Signed-off-by: Meghana Malladi --- v1: https://lore.kernel.org/all/20250122124951.3072410-1-m-malladi@ti.com/ Changes since v1 (v2-v1): None drivers/net/ethernet/ti/icssg/icssg_common.c | 47 ++++++++++++------- drivers/net/ethernet/ti/icssg/icssg_config.h | 2 +- drivers/net/ethernet/ti/icssg/icssg_prueth.c | 6 +++ drivers/net/ethernet/ti/icssg/icssg_prueth.h | 18 +++++++ .../net/ethernet/ti/icssg/icssg_prueth_sr1.c | 4 +- 5 files changed, 58 insertions(+), 19 deletions(-) diff --git a/drivers/net/ethernet/ti/icssg/icssg_common.c b/drivers/net/eth= ernet/ti/icssg/icssg_common.c index c3c1e2bf461e..a124c5773551 100644 --- a/drivers/net/ethernet/ti/icssg/icssg_common.c +++ b/drivers/net/ethernet/ti/icssg/icssg_common.c @@ -136,12 +136,12 @@ int emac_tx_complete_packets(struct prueth_emac *emac= , int chn, struct net_device *ndev =3D emac->ndev; struct cppi5_host_desc_t *desc_tx; struct netdev_queue *netif_txq; + struct prueth_swdata *swdata; struct prueth_tx_chn *tx_chn; unsigned int total_bytes =3D 0; struct sk_buff *skb; dma_addr_t desc_dma; int res, num_tx =3D 0; - void **swdata; =20 tx_chn =3D &emac->tx_chns[chn]; =20 @@ -163,12 +163,18 @@ int emac_tx_complete_packets(struct prueth_emac *emac= , int chn, swdata =3D cppi5_hdesc_get_swdata(desc_tx); =20 /* was this command's TX complete? */ - if (emac->is_sr1 && *(swdata) =3D=3D emac->cmd_data) { + if (emac->is_sr1 && (void *)(swdata) =3D=3D emac->cmd_data) { prueth_xmit_free(tx_chn, desc_tx); continue; } =20 - skb =3D *(swdata); + if (swdata->type !=3D PRUETH_SWDATA_SKB) { + netdev_err(ndev, "tx_complete: invalid swdata type %d\n", swdata->type); + budget++; + continue; + } + + skb =3D swdata->data.skb; prueth_xmit_free(tx_chn, desc_tx); =20 ndev =3D skb->dev; @@ -472,9 +478,9 @@ int prueth_dma_rx_push_mapped(struct prueth_emac *emac, { struct net_device *ndev =3D emac->ndev; struct cppi5_host_desc_t *desc_rx; + struct prueth_swdata *swdata; dma_addr_t desc_dma; dma_addr_t buf_dma; - void **swdata; =20 buf_dma =3D page_pool_get_dma_addr(page) + PRUETH_HEADROOM; desc_rx =3D k3_cppi_desc_pool_alloc(rx_chn->desc_pool); @@ -490,7 +496,8 @@ int prueth_dma_rx_push_mapped(struct prueth_emac *emac, cppi5_hdesc_attach_buf(desc_rx, buf_dma, buf_len, buf_dma, buf_len); =20 swdata =3D cppi5_hdesc_get_swdata(desc_rx); - *swdata =3D page; + swdata->type =3D PRUETH_SWDATA_PAGE; + swdata->data.page =3D page; =20 return k3_udma_glue_push_rx_chn(rx_chn->rx_chn, PRUETH_RX_FLOW_DATA, desc_rx, desc_dma); @@ -539,11 +546,11 @@ static int emac_rx_packet(struct prueth_emac *emac, u= 32 flow_id) u32 buf_dma_len, pkt_len, port_id =3D 0; struct net_device *ndev =3D emac->ndev; struct cppi5_host_desc_t *desc_rx; + struct prueth_swdata *swdata; dma_addr_t desc_dma, buf_dma; struct page *page, *new_page; struct page_pool *pool; struct sk_buff *skb; - void **swdata; u32 *psdata; void *pa; int ret; @@ -561,7 +568,11 @@ static int emac_rx_packet(struct prueth_emac *emac, u3= 2 flow_id) =20 desc_rx =3D k3_cppi_desc_pool_dma2virt(rx_chn->desc_pool, desc_dma); swdata =3D cppi5_hdesc_get_swdata(desc_rx); - page =3D *swdata; + if (swdata->type !=3D PRUETH_SWDATA_PAGE) { + netdev_err(ndev, "rx_pkt: invalid swdata->type %d\n", swdata->type); + return 0; + } + page =3D swdata->data.page; =20 page_pool_dma_sync_for_cpu(pool, page, 0, PAGE_SIZE); cppi5_hdesc_get_obuf(desc_rx, &buf_dma, &buf_dma_len); @@ -627,16 +638,18 @@ static void prueth_rx_cleanup(void *data, dma_addr_t = desc_dma) { struct prueth_rx_chn *rx_chn =3D data; struct cppi5_host_desc_t *desc_rx; + struct prueth_swdata *swdata; struct page_pool *pool; struct page *page; - void **swdata; =20 pool =3D rx_chn->pg_pool; =20 desc_rx =3D k3_cppi_desc_pool_dma2virt(rx_chn->desc_pool, desc_dma); swdata =3D cppi5_hdesc_get_swdata(desc_rx); - page =3D *swdata; - page_pool_recycle_direct(pool, page); + if (swdata->type =3D=3D PRUETH_SWDATA_PAGE) { + page =3D swdata->data.page; + page_pool_recycle_direct(pool, page); + } k3_cppi_desc_pool_free(rx_chn->desc_pool, desc_rx); } =20 @@ -673,13 +686,13 @@ enum netdev_tx icssg_ndo_start_xmit(struct sk_buff *s= kb, struct net_device *ndev struct prueth_emac *emac =3D netdev_priv(ndev); struct prueth *prueth =3D emac->prueth; struct netdev_queue *netif_txq; + struct prueth_swdata *swdata; struct prueth_tx_chn *tx_chn; dma_addr_t desc_dma, buf_dma; u32 pkt_len, dst_tag_id; int i, ret =3D 0, q_idx; bool in_tx_ts =3D 0; int tx_ts_cookie; - void **swdata; u32 *epib; =20 pkt_len =3D skb_headlen(skb); @@ -741,7 +754,8 @@ enum netdev_tx icssg_ndo_start_xmit(struct sk_buff *skb= , struct net_device *ndev k3_udma_glue_tx_dma_to_cppi5_addr(tx_chn->tx_chn, &buf_dma); cppi5_hdesc_attach_buf(first_desc, buf_dma, pkt_len, buf_dma, pkt_len); swdata =3D cppi5_hdesc_get_swdata(first_desc); - *swdata =3D skb; + swdata->type =3D PRUETH_SWDATA_SKB; + swdata->data.skb =3D skb; =20 /* Handle the case where skb is fragmented in pages */ cur_desc =3D first_desc; @@ -844,15 +858,16 @@ static void prueth_tx_cleanup(void *data, dma_addr_t = desc_dma) { struct prueth_tx_chn *tx_chn =3D data; struct cppi5_host_desc_t *desc_tx; + struct prueth_swdata *swdata; struct sk_buff *skb; - void **swdata; =20 desc_tx =3D k3_cppi_desc_pool_dma2virt(tx_chn->desc_pool, desc_dma); swdata =3D cppi5_hdesc_get_swdata(desc_tx); - skb =3D *(swdata); + if (swdata->type =3D=3D PRUETH_SWDATA_SKB) { + skb =3D swdata->data.skb; + dev_kfree_skb_any(skb); + } prueth_xmit_free(tx_chn, desc_tx); - - dev_kfree_skb_any(skb); } =20 irqreturn_t prueth_rx_irq(int irq, void *dev_id) diff --git a/drivers/net/ethernet/ti/icssg/icssg_config.h b/drivers/net/eth= ernet/ti/icssg/icssg_config.h index c884e9fa099e..eab84e11d80e 100644 --- a/drivers/net/ethernet/ti/icssg/icssg_config.h +++ b/drivers/net/ethernet/ti/icssg/icssg_config.h @@ -20,7 +20,7 @@ struct icssg_flow_cfg { =20 #define PRUETH_PKT_TYPE_CMD 0x10 #define PRUETH_NAV_PS_DATA_SIZE 16 /* Protocol specific data size */ -#define PRUETH_NAV_SW_DATA_SIZE 16 /* SW related data size */ +#define PRUETH_NAV_SW_DATA_SIZE 48 /* SW related data size */ #define PRUETH_MAX_TX_DESC 512 #define PRUETH_MAX_RX_DESC 512 #define PRUETH_MAX_RX_FLOWS 1 /* excluding default flow */ diff --git a/drivers/net/ethernet/ti/icssg/icssg_prueth.c b/drivers/net/eth= ernet/ti/icssg/icssg_prueth.c index 00ed97860547..e5e4efe485f6 100644 --- a/drivers/net/ethernet/ti/icssg/icssg_prueth.c +++ b/drivers/net/ethernet/ti/icssg/icssg_prueth.c @@ -1522,6 +1522,12 @@ static int prueth_probe(struct platform_device *pdev) =20 np =3D dev->of_node; =20 + if (sizeof(struct prueth_swdata) > PRUETH_NAV_SW_DATA_SIZE) { + dev_err(dev, "insufficient SW_DATA size: %d vs %ld\n", + PRUETH_NAV_SW_DATA_SIZE, sizeof(struct prueth_swdata)); + return -ENOMEM; + } + prueth =3D devm_kzalloc(dev, sizeof(*prueth), GFP_KERNEL); if (!prueth) return -ENOMEM; diff --git a/drivers/net/ethernet/ti/icssg/icssg_prueth.h b/drivers/net/eth= ernet/ti/icssg/icssg_prueth.h index c7b906de18af..2c8585255b7c 100644 --- a/drivers/net/ethernet/ti/icssg/icssg_prueth.h +++ b/drivers/net/ethernet/ti/icssg/icssg_prueth.h @@ -136,6 +136,24 @@ struct prueth_rx_chn { struct page_pool *pg_pool; }; =20 +enum prueth_swdata_type { + PRUETH_SWDATA_INVALID =3D 0, + PRUETH_SWDATA_SKB, + PRUETH_SWDATA_PAGE, + PRUETH_SWDATA_CMD, +}; + +union prueth_data { + struct sk_buff *skb; + struct page *page; + u32 cmd; +}; + +struct prueth_swdata { + union prueth_data data; + enum prueth_swdata_type type; +}; + /* There are 4 Tx DMA channels, but the highest priority is CH3 (thread 3) * and lower three are lower priority channels or threads. */ diff --git a/drivers/net/ethernet/ti/icssg/icssg_prueth_sr1.c b/drivers/net= /ethernet/ti/icssg/icssg_prueth_sr1.c index aeeb8a50376b..7bbe0808b3ec 100644 --- a/drivers/net/ethernet/ti/icssg/icssg_prueth_sr1.c +++ b/drivers/net/ethernet/ti/icssg/icssg_prueth_sr1.c @@ -275,10 +275,10 @@ static struct sk_buff *prueth_process_rx_mgm(struct p= rueth_emac *emac, struct net_device *ndev =3D emac->ndev; struct cppi5_host_desc_t *desc_rx; struct page *page, *new_page; + struct prueth_swdata *swdata; dma_addr_t desc_dma, buf_dma; u32 buf_dma_len, pkt_len; struct sk_buff *skb; - void **swdata; void *pa; int ret; =20 @@ -301,7 +301,7 @@ static struct sk_buff *prueth_process_rx_mgm(struct pru= eth_emac *emac, } =20 swdata =3D cppi5_hdesc_get_swdata(desc_rx); - page =3D *swdata; + page =3D swdata->data.page; cppi5_hdesc_get_obuf(desc_rx, &buf_dma, &buf_dma_len); pkt_len =3D cppi5_hdesc_get_pktlen(desc_rx); =20 --=20 2.43.0 From nobody Tue Dec 16 22:20:53 2025 Received: from fllvem-ot04.ext.ti.com (fllvem-ot04.ext.ti.com [198.47.19.246]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 994941D47A2; Mon, 10 Feb 2025 10:34:45 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.47.19.246 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739183687; cv=none; b=VrvT9DDor6abb9t2YtiE8h+31eJx68Y3cxZillQI6zYkK2hdt1QSsd8Zc1ZVmT/PnM1MXOSGFjiRvbnXrodBSBrdLfk8CSVOhyvcHjgjeKd2tHtgF5hZH/FO5J8ZaAu2qApdEkJYuq/EzO43yHvq4Ii+LtnuPugjgI4juyB1EBY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739183687; c=relaxed/simple; bh=IUPT24Q+s48dw9/F9k+BRM87cH7aYU83zaKEHP2kGho=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=r9s9VR5GBkzY8wJfK8v82rezfC8qObTVifa/2t9v8q7fsmLflGVKa4lbihk6iWuKJhwPkZdqK8Xcj3TesEuu3txgRDhQSAKLULEmtjA3J7ULEO/bRy/nHGU5M3EQNUXBd946Aw/GnOjdrkF9PGh4SS4JVXk5nEPALmHdi8dTLWw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=ti.com; spf=pass smtp.mailfrom=ti.com; dkim=pass (1024-bit key) header.d=ti.com header.i=@ti.com header.b=c2usI9fn; arc=none smtp.client-ip=198.47.19.246 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=ti.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=ti.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=ti.com header.i=@ti.com header.b="c2usI9fn" Received: from fllv0034.itg.ti.com ([10.64.40.246]) by fllvem-ot04.ext.ti.com (8.15.2/8.15.2) with ESMTPS id 51AAY8g23393391 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 10 Feb 2025 04:34:08 -0600 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ti.com; s=ti-com-17Q1; t=1739183648; bh=DqB8wJR2br4+di0LrgJHaoGCqfgMCKWXnXmvDGStkzk=; h=From:To:CC:Subject:Date:In-Reply-To:References; b=c2usI9fnbm4FRZHEVwgxRqUUdg5R09+/FXdVqkG5sW605O2IxR+sDBIhz2h+xy3hA Hic+ICqPwJtFuKmhEFGwfOiEbuWbcO2pZSpk3oYxWtVMEQssY8CuTa0jgXbkuVPXAN +0drTXXoArkE0smJPRSddkKDb70kwUbmJJdPwA9M= Received: from DLEE105.ent.ti.com (dlee105.ent.ti.com [157.170.170.35]) by fllv0034.itg.ti.com (8.15.2/8.15.2) with ESMTPS id 51AAY88f108212 (version=TLSv1.2 cipher=AES256-GCM-SHA384 bits=256 verify=FAIL); Mon, 10 Feb 2025 04:34:08 -0600 Received: from DLEE106.ent.ti.com (157.170.170.36) by DLEE105.ent.ti.com (157.170.170.35) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.2507.23; Mon, 10 Feb 2025 04:34:07 -0600 Received: from lelvsmtp5.itg.ti.com (10.180.75.250) by DLEE106.ent.ti.com (157.170.170.36) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.2507.23 via Frontend Transport; Mon, 10 Feb 2025 04:34:07 -0600 Received: from fllv0122.itg.ti.com (fllv0122.itg.ti.com [10.247.120.72]) by lelvsmtp5.itg.ti.com (8.15.2/8.15.2) with ESMTP id 51AAY7PX100471; Mon, 10 Feb 2025 04:34:07 -0600 Received: from localhost (meghana-pc.dhcp.ti.com [10.24.69.13] (may be forged)) by fllv0122.itg.ti.com (8.14.7/8.14.7) with ESMTP id 51AAY6gf021846; Mon, 10 Feb 2025 04:34:07 -0600 From: Meghana Malladi To: , , , , , , CC: , , , , , , , , , , , , , , , , , , Vignesh Raghavendra Subject: [PATCH net-next v2 3/3] net: ti: icssg-prueth: Add XDP support Date: Mon, 10 Feb 2025 16:03:52 +0530 Message-ID: <20250210103352.541052-4-m-malladi@ti.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250210103352.541052-1-m-malladi@ti.com> References: <20250210103352.541052-1-m-malladi@ti.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-C2ProcessedOrg: 333ef613-75bf-4e12-a4b1-8e3623f5dcea Content-Type: text/plain; charset="utf-8" From: Roger Quadros Add native XDP support. We do not support zero copy yet. Signed-off-by: Roger Quadros Signed-off-by: MD Danish Anwar Signed-off-by: Meghana Malladi --- v1: https://lore.kernel.org/all/20250122124951.3072410-1-m-malladi@ti.com/ Changes since v1 (v2-v1): - Fix XDP typo in the commit message - Add XDP feature flags using xdp_set_features_flag() - Use xdp_build_skb_from_buff() when XDP ran All the above changes have been suggested by Ido Schimmel drivers/net/ethernet/ti/icssg/icssg_common.c | 226 +++++++++++++++++-- drivers/net/ethernet/ti/icssg/icssg_prueth.c | 123 +++++++++- drivers/net/ethernet/ti/icssg/icssg_prueth.h | 18 ++ 3 files changed, 353 insertions(+), 14 deletions(-) diff --git a/drivers/net/ethernet/ti/icssg/icssg_common.c b/drivers/net/eth= ernet/ti/icssg/icssg_common.c index a124c5773551..b01750a2d57e 100644 --- a/drivers/net/ethernet/ti/icssg/icssg_common.c +++ b/drivers/net/ethernet/ti/icssg/icssg_common.c @@ -98,11 +98,19 @@ void prueth_xmit_free(struct prueth_tx_chn *tx_chn, { struct cppi5_host_desc_t *first_desc, *next_desc; dma_addr_t buf_dma, next_desc_dma; + struct prueth_swdata *swdata; u32 buf_dma_len; =20 first_desc =3D desc; next_desc =3D first_desc; =20 + swdata =3D cppi5_hdesc_get_swdata(desc); + if (swdata->type =3D=3D PRUETH_SWDATA_PAGE) { + page_pool_recycle_direct(swdata->rx_chn->pg_pool, + swdata->data.page); + goto free_desc; + } + cppi5_hdesc_get_obuf(first_desc, &buf_dma, &buf_dma_len); k3_udma_glue_tx_cppi5_to_dma_addr(tx_chn->tx_chn, &buf_dma); =20 @@ -126,6 +134,7 @@ void prueth_xmit_free(struct prueth_tx_chn *tx_chn, k3_cppi_desc_pool_free(tx_chn->desc_pool, next_desc); } =20 +free_desc: k3_cppi_desc_pool_free(tx_chn->desc_pool, first_desc); } EXPORT_SYMBOL_GPL(prueth_xmit_free); @@ -139,6 +148,7 @@ int emac_tx_complete_packets(struct prueth_emac *emac, = int chn, struct prueth_swdata *swdata; struct prueth_tx_chn *tx_chn; unsigned int total_bytes =3D 0; + struct xdp_frame *xdpf; struct sk_buff *skb; dma_addr_t desc_dma; int res, num_tx =3D 0; @@ -168,20 +178,29 @@ int emac_tx_complete_packets(struct prueth_emac *emac= , int chn, continue; } =20 - if (swdata->type !=3D PRUETH_SWDATA_SKB) { + switch (swdata->type) { + case PRUETH_SWDATA_SKB: + skb =3D swdata->data.skb; + ndev->stats.tx_bytes +=3D skb->len; + ndev->stats.tx_packets++; + total_bytes +=3D skb->len; + napi_consume_skb(skb, budget); + break; + case PRUETH_SWDATA_XDPF: + xdpf =3D swdata->data.xdpf; + ndev->stats.tx_bytes +=3D xdpf->len; + ndev->stats.tx_packets++; + total_bytes +=3D xdpf->len; + xdp_return_frame(xdpf); + break; + default: netdev_err(ndev, "tx_complete: invalid swdata type %d\n", swdata->type); + prueth_xmit_free(tx_chn, desc_tx); budget++; continue; } =20 - skb =3D swdata->data.skb; prueth_xmit_free(tx_chn, desc_tx); - - ndev =3D skb->dev; - ndev->stats.tx_packets++; - ndev->stats.tx_bytes +=3D skb->len; - total_bytes +=3D skb->len; - napi_consume_skb(skb, budget); num_tx++; } =20 @@ -498,6 +517,7 @@ int prueth_dma_rx_push_mapped(struct prueth_emac *emac, swdata =3D cppi5_hdesc_get_swdata(desc_rx); swdata->type =3D PRUETH_SWDATA_PAGE; swdata->data.page =3D page; + swdata->rx_chn =3D rx_chn; =20 return k3_udma_glue_push_rx_chn(rx_chn->rx_chn, PRUETH_RX_FLOW_DATA, desc_rx, desc_dma); @@ -540,7 +560,156 @@ void emac_rx_timestamp(struct prueth_emac *emac, ssh->hwtstamp =3D ns_to_ktime(ns); } =20 -static int emac_rx_packet(struct prueth_emac *emac, u32 flow_id) +/** + * emac_xmit_xdp_frame - transmits an XDP frame + * @emac: emac device + * @xdpf: data to transmit + * @page: page from page pool if already DMA mapped + * @q_idx: queue id + * + * Return: XDP state + */ +int emac_xmit_xdp_frame(struct prueth_emac *emac, + struct xdp_frame *xdpf, + struct page *page, + unsigned int q_idx) +{ + struct cppi5_host_desc_t *first_desc; + struct net_device *ndev =3D emac->ndev; + struct prueth_tx_chn *tx_chn; + dma_addr_t desc_dma, buf_dma; + struct prueth_swdata *swdata; + u32 *epib; + int ret; + + void *data =3D xdpf->data; + u32 pkt_len =3D xdpf->len; + + if (q_idx >=3D PRUETH_MAX_TX_QUEUES) { + netdev_err(ndev, "xdp tx: invalid q_id %d\n", q_idx); + return ICSSG_XDP_CONSUMED; /* drop */ + } + + tx_chn =3D &emac->tx_chns[q_idx]; + + if (page) { /* already DMA mapped by page_pool */ + buf_dma =3D page_pool_get_dma_addr(page); + buf_dma +=3D xdpf->headroom + sizeof(struct xdp_frame); + } else { /* Map the linear buffer */ + buf_dma =3D dma_map_single(tx_chn->dma_dev, data, pkt_len, DMA_TO_DEVICE= ); + if (dma_mapping_error(tx_chn->dma_dev, buf_dma)) { + netdev_err(ndev, "xdp tx: failed to map data buffer\n"); + return ICSSG_XDP_CONSUMED; /* drop */ + } + } + + first_desc =3D k3_cppi_desc_pool_alloc(tx_chn->desc_pool); + if (!first_desc) { + netdev_dbg(ndev, "xdp tx: failed to allocate descriptor\n"); + if (!page) + dma_unmap_single(tx_chn->dma_dev, buf_dma, pkt_len, DMA_TO_DEVICE); + goto drop_free_descs; /* drop */ + } + + cppi5_hdesc_init(first_desc, CPPI5_INFO0_HDESC_EPIB_PRESENT, + PRUETH_NAV_PS_DATA_SIZE); + cppi5_hdesc_set_pkttype(first_desc, 0); + epib =3D first_desc->epib; + epib[0] =3D 0; + epib[1] =3D 0; + + /* set dst tag to indicate internal qid at the firmware which is at + * bit8..bit15. bit0..bit7 indicates port num for directed + * packets in case of switch mode operation + */ + cppi5_desc_set_tags_ids(&first_desc->hdr, 0, (emac->port_id | (q_idx << 8= ))); + k3_udma_glue_tx_dma_to_cppi5_addr(tx_chn->tx_chn, &buf_dma); + cppi5_hdesc_attach_buf(first_desc, buf_dma, pkt_len, buf_dma, pkt_len); + swdata =3D cppi5_hdesc_get_swdata(first_desc); + if (page) { + swdata->type =3D PRUETH_SWDATA_PAGE; + swdata->data.page =3D page; + /* we assume page came from RX channel page pool */ + swdata->rx_chn =3D &emac->rx_chns; + } else { + swdata->type =3D PRUETH_SWDATA_XDPF; + swdata->data.xdpf =3D xdpf; + } + + cppi5_hdesc_set_pktlen(first_desc, pkt_len); + desc_dma =3D k3_cppi_desc_pool_virt2dma(tx_chn->desc_pool, first_desc); + + ret =3D k3_udma_glue_push_tx_chn(tx_chn->tx_chn, first_desc, desc_dma); + if (ret) { + netdev_err(ndev, "xdp tx: push failed: %d\n", ret); + goto drop_free_descs; + } + + return ICSSG_XDP_TX; + +drop_free_descs: + prueth_xmit_free(tx_chn, first_desc); + return ICSSG_XDP_CONSUMED; +} +EXPORT_SYMBOL_GPL(emac_xmit_xdp_frame); + +/** + * emac_run_xdp - run an XDP program + * @emac: emac device + * @xdp: XDP buffer containing the frame + * @page: page with RX data if already DMA mapped + * + * Return: XDP state + */ +static int emac_run_xdp(struct prueth_emac *emac, struct xdp_buff *xdp, + struct page *page) +{ + int err, result =3D ICSSG_XDP_PASS; + struct bpf_prog *xdp_prog; + struct xdp_frame *xdpf; + int q_idx; + u32 act; + + xdp_prog =3D READ_ONCE(emac->xdp_prog); + + act =3D bpf_prog_run_xdp(xdp_prog, xdp); + switch (act) { + case XDP_PASS: + break; + case XDP_TX: + /* Send packet to TX ring for immediate transmission */ + xdpf =3D xdp_convert_buff_to_frame(xdp); + if (unlikely(!xdpf)) + goto drop; + + q_idx =3D smp_processor_id() % emac->tx_ch_num; + result =3D emac_xmit_xdp_frame(emac, xdpf, page, q_idx); + if (result =3D=3D ICSSG_XDP_CONSUMED) + goto drop; + break; + case XDP_REDIRECT: + err =3D xdp_do_redirect(emac->ndev, xdp, xdp_prog); + if (err) + goto drop; + result =3D ICSSG_XDP_REDIR; + break; + default: + bpf_warn_invalid_xdp_action(emac->ndev, xdp_prog, act); + fallthrough; + case XDP_ABORTED: +drop: + trace_xdp_exception(emac->ndev, xdp_prog, act); + fallthrough; /* handle aborts by dropping packet */ + case XDP_DROP: + result =3D ICSSG_XDP_CONSUMED; + page_pool_recycle_direct(emac->rx_chns.pg_pool, page); + break; + } + + return result; +} + +static int emac_rx_packet(struct prueth_emac *emac, u32 flow_id, int *xdp_= state) { struct prueth_rx_chn *rx_chn =3D &emac->rx_chns; u32 buf_dma_len, pkt_len, port_id =3D 0; @@ -551,10 +720,12 @@ static int emac_rx_packet(struct prueth_emac *emac, u= 32 flow_id) struct page *page, *new_page; struct page_pool *pool; struct sk_buff *skb; + struct xdp_buff xdp; u32 *psdata; void *pa; int ret; =20 + *xdp_state =3D 0; pool =3D rx_chn->pg_pool; ret =3D k3_udma_glue_pop_rx_chn(rx_chn->rx_chn, flow_id, &desc_dma); if (ret) { @@ -594,9 +765,21 @@ static int emac_rx_packet(struct prueth_emac *emac, u3= 2 flow_id) goto requeue; } =20 - /* prepare skb and send to n/w stack */ pa =3D page_address(page); - skb =3D napi_build_skb(pa, PAGE_SIZE); + if (emac->xdp_prog) { + xdp_init_buff(&xdp, PAGE_SIZE, &rx_chn->xdp_rxq); + xdp_prepare_buff(&xdp, pa, PRUETH_HEADROOM, pkt_len, false); + + *xdp_state =3D emac_run_xdp(emac, &xdp, page); + if (*xdp_state =3D=3D ICSSG_XDP_PASS) + skb =3D xdp_build_skb_from_buff(&xdp); + else + goto requeue; + } else { + /* prepare skb and send to n/w stack */ + skb =3D napi_build_skb(pa, PAGE_SIZE); + } + if (!skb) { ndev->stats.rx_dropped++; page_pool_recycle_direct(pool, page); @@ -859,14 +1042,25 @@ static void prueth_tx_cleanup(void *data, dma_addr_t= desc_dma) struct prueth_tx_chn *tx_chn =3D data; struct cppi5_host_desc_t *desc_tx; struct prueth_swdata *swdata; + struct xdp_frame *xdpf; struct sk_buff *skb; =20 desc_tx =3D k3_cppi_desc_pool_dma2virt(tx_chn->desc_pool, desc_dma); swdata =3D cppi5_hdesc_get_swdata(desc_tx); - if (swdata->type =3D=3D PRUETH_SWDATA_SKB) { + + switch (swdata->type) { + case PRUETH_SWDATA_SKB: skb =3D swdata->data.skb; dev_kfree_skb_any(skb); + break; + case PRUETH_SWDATA_XDPF: + xdpf =3D swdata->data.xdpf; + xdp_return_frame(xdpf); + break; + default: + break; } + prueth_xmit_free(tx_chn, desc_tx); } =20 @@ -901,15 +1095,18 @@ int icssg_napi_rx_poll(struct napi_struct *napi_rx, = int budget) PRUETH_RX_FLOW_DATA_SR1 : PRUETH_RX_FLOW_DATA; int flow =3D emac->is_sr1 ? PRUETH_MAX_RX_FLOWS_SR1 : PRUETH_MAX_RX_FLOWS; + int xdp_state_or =3D 0; int num_rx =3D 0; int cur_budget; + int xdp_state; int ret; =20 while (flow--) { cur_budget =3D budget - num_rx; =20 while (cur_budget--) { - ret =3D emac_rx_packet(emac, flow); + ret =3D emac_rx_packet(emac, flow, &xdp_state); + xdp_state_or |=3D xdp_state; if (ret) break; num_rx++; @@ -919,6 +1116,9 @@ int icssg_napi_rx_poll(struct napi_struct *napi_rx, in= t budget) break; } =20 + if (xdp_state_or & ICSSG_XDP_REDIR) + xdp_do_flush(); + if (num_rx < budget && napi_complete_done(napi_rx, num_rx)) { if (unlikely(emac->rx_pace_timeout_ns)) { hrtimer_start(&emac->rx_hrtimer, diff --git a/drivers/net/ethernet/ti/icssg/icssg_prueth.c b/drivers/net/eth= ernet/ti/icssg/icssg_prueth.c index e5e4efe485f6..a360a1d6f8d7 100644 --- a/drivers/net/ethernet/ti/icssg/icssg_prueth.c +++ b/drivers/net/ethernet/ti/icssg/icssg_prueth.c @@ -559,6 +559,33 @@ const struct icss_iep_clockops prueth_iep_clockops =3D= { .perout_enable =3D prueth_perout_enable, }; =20 +static int prueth_create_xdp_rxqs(struct prueth_emac *emac) +{ + struct xdp_rxq_info *rxq =3D &emac->rx_chns.xdp_rxq; + struct page_pool *pool =3D emac->rx_chns.pg_pool; + int ret; + + ret =3D xdp_rxq_info_reg(rxq, emac->ndev, 0, rxq->napi_id); + if (ret) + return ret; + + ret =3D xdp_rxq_info_reg_mem_model(rxq, MEM_TYPE_PAGE_POOL, pool); + if (ret) + xdp_rxq_info_unreg(rxq); + + return ret; +} + +static void prueth_destroy_xdp_rxqs(struct prueth_emac *emac) +{ + struct xdp_rxq_info *rxq =3D &emac->rx_chns.xdp_rxq; + + if (!xdp_rxq_info_is_reg(rxq)) + return; + + xdp_rxq_info_unreg(rxq); +} + static int icssg_prueth_add_mcast(struct net_device *ndev, const u8 *addr) { struct net_device *real_dev; @@ -780,10 +807,14 @@ static int emac_ndo_open(struct net_device *ndev) if (ret) goto free_tx_ts_irq; =20 - ret =3D k3_udma_glue_enable_rx_chn(emac->rx_chns.rx_chn); + ret =3D prueth_create_xdp_rxqs(emac); if (ret) goto reset_rx_chn; =20 + ret =3D k3_udma_glue_enable_rx_chn(emac->rx_chns.rx_chn); + if (ret) + goto destroy_xdp_rxqs; + for (i =3D 0; i < emac->tx_ch_num; i++) { ret =3D k3_udma_glue_enable_tx_chn(emac->tx_chns[i].tx_chn); if (ret) @@ -809,6 +840,8 @@ static int emac_ndo_open(struct net_device *ndev) * any SKB for completion. So set false to free_skb */ prueth_reset_tx_chan(emac, i, false); +destroy_xdp_rxqs: + prueth_destroy_xdp_rxqs(emac); reset_rx_chn: prueth_reset_rx_chan(&emac->rx_chns, max_rx_flows, false); free_tx_ts_irq: @@ -880,6 +913,8 @@ static int emac_ndo_stop(struct net_device *ndev) =20 prueth_reset_rx_chan(&emac->rx_chns, max_rx_flows, true); =20 + prueth_destroy_xdp_rxqs(emac); + napi_disable(&emac->napi_rx); hrtimer_cancel(&emac->rx_hrtimer); =20 @@ -1024,6 +1059,90 @@ static int emac_ndo_vlan_rx_del_vid(struct net_devic= e *ndev, return 0; } =20 +/** + * emac_xdp_xmit - Implements ndo_xdp_xmit + * @dev: netdev + * @n: number of frames + * @frames: array of XDP buffer pointers + * @flags: XDP extra info + * + * Return: number of frames successfully sent. Failed frames + * will be free'ed by XDP core. + * + * For error cases, a negative errno code is returned and no-frames + * are transmitted (caller must handle freeing frames). + **/ +static int emac_xdp_xmit(struct net_device *dev, int n, struct xdp_frame *= *frames, + u32 flags) +{ + struct prueth_emac *emac =3D netdev_priv(dev); + unsigned int q_idx; + int nxmit =3D 0; + int i; + + q_idx =3D smp_processor_id() % emac->tx_ch_num; + + if (unlikely(flags & ~XDP_XMIT_FLAGS_MASK)) + return -EINVAL; + + for (i =3D 0; i < n; i++) { + struct xdp_frame *xdpf =3D frames[i]; + int err; + + err =3D emac_xmit_xdp_frame(emac, xdpf, NULL, q_idx); + if (err !=3D ICSSG_XDP_TX) + break; + nxmit++; + } + + return nxmit; +} + +/** + * emac_xdp_setup - add/remove an XDP program + * @emac: emac device + * @bpf: XDP program + * + * Return: Always 0 (Success) + **/ +static int emac_xdp_setup(struct prueth_emac *emac, struct netdev_bpf *bpf) +{ + struct bpf_prog *prog =3D bpf->prog; + xdp_features_t val; + + val =3D NETDEV_XDP_ACT_BASIC | NETDEV_XDP_ACT_REDIRECT | + NETDEV_XDP_ACT_NDO_XMIT; + xdp_set_features_flag(emac->ndev, val); + + if (!emac->xdpi.prog && !prog) + return 0; + + WRITE_ONCE(emac->xdp_prog, prog); + + xdp_attachment_setup(&emac->xdpi, bpf); + + return 0; +} + +/** + * emac_ndo_bpf - implements ndo_bpf for icssg_prueth + * @ndev: network adapter device + * @bpf: XDP program + * + * Return: 0 on success, error code on failure. + **/ +static int emac_ndo_bpf(struct net_device *ndev, struct netdev_bpf *bpf) +{ + struct prueth_emac *emac =3D netdev_priv(ndev); + + switch (bpf->command) { + case XDP_SETUP_PROG: + return emac_xdp_setup(emac, bpf); + default: + return -EINVAL; + } +} + static const struct net_device_ops emac_netdev_ops =3D { .ndo_open =3D emac_ndo_open, .ndo_stop =3D emac_ndo_stop, @@ -1038,6 +1157,8 @@ static const struct net_device_ops emac_netdev_ops = =3D { .ndo_fix_features =3D emac_ndo_fix_features, .ndo_vlan_rx_add_vid =3D emac_ndo_vlan_rx_add_vid, .ndo_vlan_rx_kill_vid =3D emac_ndo_vlan_rx_del_vid, + .ndo_bpf =3D emac_ndo_bpf, + .ndo_xdp_xmit =3D emac_xdp_xmit, }; =20 static int prueth_netdev_init(struct prueth *prueth, diff --git a/drivers/net/ethernet/ti/icssg/icssg_prueth.h b/drivers/net/eth= ernet/ti/icssg/icssg_prueth.h index 2c8585255b7c..fb8dc8e12c19 100644 --- a/drivers/net/ethernet/ti/icssg/icssg_prueth.h +++ b/drivers/net/ethernet/ti/icssg/icssg_prueth.h @@ -8,6 +8,8 @@ #ifndef __NET_TI_ICSSG_PRUETH_H #define __NET_TI_ICSSG_PRUETH_H =20 +#include +#include #include #include #include @@ -134,6 +136,7 @@ struct prueth_rx_chn { unsigned int irq[ICSSG_MAX_RFLOWS]; /* separate irq per flow */ char name[32]; struct page_pool *pg_pool; + struct xdp_rxq_info xdp_rxq; }; =20 enum prueth_swdata_type { @@ -141,16 +144,19 @@ enum prueth_swdata_type { PRUETH_SWDATA_SKB, PRUETH_SWDATA_PAGE, PRUETH_SWDATA_CMD, + PRUETH_SWDATA_XDPF, }; =20 union prueth_data { struct sk_buff *skb; struct page *page; u32 cmd; + struct xdp_frame *xdpf; }; =20 struct prueth_swdata { union prueth_data data; + struct prueth_rx_chn *rx_chn; enum prueth_swdata_type type; }; =20 @@ -161,6 +167,12 @@ struct prueth_swdata { =20 #define PRUETH_MAX_TX_TS_REQUESTS 50 /* Max simultaneous TX_TS requests */ =20 +/* XDP BPF state */ +#define ICSSG_XDP_PASS 0 +#define ICSSG_XDP_CONSUMED BIT(0) +#define ICSSG_XDP_TX BIT(1) +#define ICSSG_XDP_REDIR BIT(2) + /* Minimum coalesce time in usecs for both Tx and Rx */ #define ICSSG_MIN_COALESCE_USECS 20 =20 @@ -229,6 +241,8 @@ struct prueth_emac { unsigned long rx_pace_timeout_ns; =20 struct netdev_hw_addr_list vlan_mcast_list[MAX_VLAN_ID]; + struct bpf_prog *xdp_prog; + struct xdp_attachment_info xdpi; }; =20 /* The buf includes headroom compatible with both skb and xdpf */ @@ -467,5 +481,9 @@ void prueth_put_cores(struct prueth *prueth, int slice); =20 /* Revision specific helper */ u64 icssg_ts_to_ns(u32 hi_sw, u32 hi, u32 lo, u32 cycle_time_ns); +int emac_xmit_xdp_frame(struct prueth_emac *emac, + struct xdp_frame *xdpf, + struct page *page, + unsigned int q_idx); =20 #endif /* __NET_TI_ICSSG_PRUETH_H */ --=20 2.43.0