From nobody Wed Dec 17 10:47:49 2025 Received: from invmail4.hynix.com (exvmail4.skhynix.com [166.125.252.92]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 83A9928FFEE; Fri, 9 May 2025 11:51:44 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=166.125.252.92 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1746791507; cv=none; b=sxczGiWC7YawfeVVBvnmkMZkpoDuLM8RBYdODWhMYj1qTS3hTxQsw39QLdWqcrSf2ktNKB/yFPdEXBVAPP2cuTqaTZjHuqNsJ8YzRXvV1S84CzM9B23NwnDhnwNWwgZJkHMdDO6tEGVqnuVrWQ3xos9UfJ2Vnq/5Po2v38xkRoo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1746791507; c=relaxed/simple; bh=40hhzcReE9lzoseTEMmVCYl1hIewehLeKTS2dWTVKHc=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References; b=t3NJFQlnFyP1x5FBA4pH897HSk3DkemV416aXVJY//s7nv0JLMhXwD5XOdeggdk5yVmCpDTIZL2tz9JaAL43I2pM/BtVTNNJBNjo4eJntAoWHYlJ9eeFeMmdSCoQksls4EwkhgAe31Fz3eP/qo/aQWMlCaAEt6l0vIeSklgUv/A= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com; spf=pass smtp.mailfrom=sk.com; arc=none smtp.client-ip=166.125.252.92 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=sk.com X-AuditID: a67dfc5b-669ff7000002311f-68-681dec49076f From: Byungchul Park To: willy@infradead.org, netdev@vger.kernel.org Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, kernel_team@skhynix.com, kuba@kernel.org, almasrymina@google.com, ilias.apalodimas@linaro.org, harry.yoo@oracle.com, hawk@kernel.org, akpm@linux-foundation.org, ast@kernel.org, daniel@iogearbox.net, davem@davemloft.net, john.fastabend@gmail.com, andrew+netdev@lunn.ch, edumazet@google.com, pabeni@redhat.com, vishal.moola@gmail.com Subject: [RFC 11/19] mlx4: use netmem descriptor and API for page pool Date: Fri, 9 May 2025 20:51:18 +0900 Message-Id: <20250509115126.63190-12-byungchul@sk.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20250509115126.63190-1-byungchul@sk.com> References: <20250509115126.63190-1-byungchul@sk.com> X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFnrNLMWRmVeSWpSXmKPExsXC9ZZnka7nG9kMgy2rtS3mrF/DZrH6R4XF 8gc7WC2+/LzNbrF44TdmiznnW1gsnh57xG5xf9kzFos97duZLXpbfjNbNO1YwWRxYVsfq8Xl XXPYLO6t+c9qcWyBmMW3028YLdbvu8Fq8fvHHDYHIY8tK28yeeycdZfdY8GmUo/NK7Q8um5c YvbYtKqTzWPTp0nsHneu7WHzODHjN4vHzh2fmTw+Pr3F4vF+31U2j8+b5AJ4o7hsUlJzMstS i/TtErgyjq9rZS6YaFXxbO4/9gbGRv0uRk4OCQETiaalu1hg7Gvd/cwgNpuAusSNGz/BbBEB Q4nPj44D1XBxMAssZJa4svgnO0hCWMBN4tPvBrAiFgFVicWz1jOB2LwCZhKb9j1lhRgqL7F6 wwGgGg4OTqB4/0d1kLCQgKnEsikL2EBmSgj8Z5PomjYXql5S4uCKGywTGHkXMDKsYhTKzCvL TczMMdHLqMzLrNBLzs/dxAiMgmW1f6J3MH66EHyIUYCDUYmH1+K5bIYQa2JZcWXuIUYJDmYl Ed7nnTIZQrwpiZVVqUX58UWlOanFhxilOViUxHmNvpWnCAmkJ5akZqemFqQWwWSZODilGhjF dzw9wHBwWbvYtv6Q2QJn0hbzHI18q2Ti1X/4fs+3SU0mtXs1/jw9dPRCyrpfV6IzW+59Mgnu X87vYFGr4nw+MfD5laSfWwWqOc/9rBLhntHczxwZtSnLdaZinctp6y8vf53l4dw8Zb9OTOQ8 82k2UgIh0bLXZh85bJAp/3zXN4cJQn3OZjuUWIozEg21mIuKEwH3hxtkfgIAAA== X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFrrELMWRmVeSWpSXmKPExsXC5WfdrOv5RjbD4GqPvMWc9WvYLFb/qLBY /mAHq8WXn7fZLRYv/MZsMed8C4vF02OP2C3uL3vGYrGnfTuzRW/Lb2aLph0rmCwOzz3JanFh Wx+rxeVdc9gs7q35z2pxbIGYxbfTbxgt1u+7wWrx+8ccNgdhjy0rbzJ57Jx1l91jwaZSj80r tDy6blxi9ti0qpPNY9OnSewed67tYfM4MeM3i8fOHZ+ZPD4+vcXi8X7fVTaPxS8+MHl83iQX wBfFZZOSmpNZllqkb5fAlXF8XStzwUSrimdz/7E3MDbqdzFyckgImEhc6+5nBrHZBNQlbtz4 CWaLCBhKfH50nKWLkYuDWWAhs8SVxT/ZQRLCAm4Sn343gBWxCKhKLJ61ngnE5hUwk9i07ykr xFB5idUbDgDVcHBwAsX7P6qDhIUETCWWTVnANoGRawEjwypGkcy8stzEzBxTveLsjMq8zAq9 5PzcTYzAkF5W+2fiDsYvl90PMQpwMCrx8Fo8l80QYk0sK67MPcQowcGsJML7vFMmQ4g3JbGy KrUoP76oNCe1+BCjNAeLkjivV3hqgpBAemJJanZqakFqEUyWiYNTqoFRKferf2BsU9GK1JP+ N5/qT54W8bPY/HnzQ2um3g0sfvJf52zs1XjVrKmh9W7d4ycZvz2Vyz0Mo0/rdv1SvjtZdPJt q0rGJrVtpbdWrpGeJCMuyv26sdnS5fy/HBfb5Ue4+JI/R7I5hWX6R7Y5rLPj/e7za+Htjup9 PQfT/ukGfT+9d46ukKcSS3FGoqEWc1FxIgCbjJVbZQIAAA== X-CFilter-Loop: Reflected Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" To simplify struct page, the effort to seperate its own descriptor from struct page is required and the work for page pool is on going. Use netmem descriptor and API for page pool in mlx4 code. Signed-off-by: Byungchul Park --- drivers/net/ethernet/mellanox/mlx4/en_rx.c | 46 +++++++++++--------- drivers/net/ethernet/mellanox/mlx4/en_tx.c | 8 ++-- drivers/net/ethernet/mellanox/mlx4/mlx4_en.h | 4 +- 3 files changed, 31 insertions(+), 27 deletions(-) diff --git a/drivers/net/ethernet/mellanox/mlx4/en_rx.c b/drivers/net/ether= net/mellanox/mlx4/en_rx.c index b33285d755b90..82c24931fa443 100644 --- a/drivers/net/ethernet/mellanox/mlx4/en_rx.c +++ b/drivers/net/ethernet/mellanox/mlx4/en_rx.c @@ -62,18 +62,18 @@ static int mlx4_en_alloc_frags(struct mlx4_en_priv *pri= v, int i; =20 for (i =3D 0; i < priv->num_frags; i++, frags++) { - if (!frags->page) { - frags->page =3D page_pool_alloc_pages(ring->pp, gfp); - if (!frags->page) { + if (!frags->netmem) { + frags->netmem =3D page_pool_alloc_netmems(ring->pp, gfp); + if (!frags->netmem) { ring->alloc_fail++; return -ENOMEM; } - page_pool_fragment_page(frags->page, 1); + page_pool_fragment_netmem(frags->netmem, 1); frags->page_offset =3D priv->rx_headroom; =20 ring->rx_alloc_pages++; } - dma =3D page_pool_get_dma_addr(frags->page); + dma =3D page_pool_get_dma_addr_netmem(frags->netmem); rx_desc->data[i].addr =3D cpu_to_be64(dma + frags->page_offset); } return 0; @@ -83,10 +83,10 @@ static void mlx4_en_free_frag(const struct mlx4_en_priv= *priv, struct mlx4_en_rx_ring *ring, struct mlx4_en_rx_alloc *frag) { - if (frag->page) - page_pool_put_full_page(ring->pp, frag->page, false); + if (frag->netmem) + page_pool_put_full_netmem(ring->pp, frag->netmem, false); /* We need to clear all fields, otherwise a change of priv->log_rx_info - * could lead to see garbage later in frag->page. + * could lead to see garbage later in frag->netmem. */ memset(frag, 0, sizeof(*frag)); } @@ -440,29 +440,33 @@ static int mlx4_en_complete_rx_desc(struct mlx4_en_pr= iv *priv, unsigned int truesize =3D 0; bool release =3D true; int nr, frag_size; - struct page *page; + netmem_ref netmem; dma_addr_t dma; =20 /* Collect used fragments while replacing them in the HW descriptors */ for (nr =3D 0;; frags++) { frag_size =3D min_t(int, length, frag_info->frag_size); =20 - page =3D frags->page; - if (unlikely(!page)) + netmem =3D frags->netmem; + if (unlikely(!netmem)) goto fail; =20 - dma =3D page_pool_get_dma_addr(page); + dma =3D page_pool_get_dma_addr_netmem(netmem); dma_sync_single_range_for_cpu(priv->ddev, dma, frags->page_offset, frag_size, priv->dma_dir); =20 - __skb_fill_page_desc(skb, nr, page, frags->page_offset, + __skb_fill_netmem_desc(skb, nr, netmem, frags->page_offset, frag_size); =20 truesize +=3D frag_info->frag_stride; if (frag_info->frag_stride =3D=3D PAGE_SIZE / 2) { + struct page *page =3D netmem_to_page(netmem); + atomic_long_t *pp_ref_count =3D + netmem_get_pp_ref_count_ref(netmem); + frags->page_offset ^=3D PAGE_SIZE / 2; release =3D page_count(page) !=3D 1 || - atomic_long_read(&page->pp_ref_count) !=3D 1 || + atomic_long_read(pp_ref_count) !=3D 1 || page_is_pfmemalloc(page) || page_to_nid(page) !=3D numa_mem_id(); } else if (!priv->rx_headroom) { @@ -476,9 +480,9 @@ static int mlx4_en_complete_rx_desc(struct mlx4_en_priv= *priv, release =3D frags->page_offset + frag_info->frag_size > PAGE_SIZE; } if (release) { - frags->page =3D NULL; + frags->netmem =3D 0; } else { - page_pool_ref_page(page); + page_pool_ref_netmem(netmem); } =20 nr++; @@ -719,7 +723,7 @@ int mlx4_en_process_rx_cq(struct net_device *dev, struc= t mlx4_en_cq *cq, int bud int nr; =20 frags =3D ring->rx_info + (index << priv->log_rx_info); - va =3D page_address(frags[0].page) + frags[0].page_offset; + va =3D netmem_address(frags[0].netmem) + frags[0].page_offset; net_prefetchw(va); /* * make sure we read the CQE after we read the ownership bit @@ -748,7 +752,7 @@ int mlx4_en_process_rx_cq(struct net_device *dev, struc= t mlx4_en_cq *cq, int bud /* Get pointer to first fragment since we haven't * skb yet and cast it to ethhdr struct */ - dma =3D page_pool_get_dma_addr(frags[0].page); + dma =3D page_pool_get_dma_addr_netmem(frags[0].netmem); dma +=3D frags[0].page_offset; dma_sync_single_for_cpu(priv->ddev, dma, sizeof(*ethh), DMA_FROM_DEVICE); @@ -788,7 +792,7 @@ int mlx4_en_process_rx_cq(struct net_device *dev, struc= t mlx4_en_cq *cq, int bud void *orig_data; u32 act; =20 - dma =3D page_pool_get_dma_addr(frags[0].page); + dma =3D page_pool_get_dma_addr_netmem(frags[0].netmem); dma +=3D frags[0].page_offset; dma_sync_single_for_cpu(priv->ddev, dma, priv->frag_info[0].frag_size, @@ -818,7 +822,7 @@ int mlx4_en_process_rx_cq(struct net_device *dev, struc= t mlx4_en_cq *cq, int bud if (likely(!xdp_do_redirect(dev, &mxbuf.xdp, xdp_prog))) { ring->xdp_redirect++; xdp_redir_flush =3D true; - frags[0].page =3D NULL; + frags[0].netmem =3D 0; goto next; } ring->xdp_redirect_fail++; @@ -828,7 +832,7 @@ int mlx4_en_process_rx_cq(struct net_device *dev, struc= t mlx4_en_cq *cq, int bud if (likely(!mlx4_en_xmit_frame(ring, frags, priv, length, cq_ring, &doorbell_pending))) { - frags[0].page =3D NULL; + frags[0].netmem =3D 0; goto next; } trace_xdp_exception(dev, xdp_prog, act); diff --git a/drivers/net/ethernet/mellanox/mlx4/en_tx.c b/drivers/net/ether= net/mellanox/mlx4/en_tx.c index 87f35bcbeff8f..b564a953da09b 100644 --- a/drivers/net/ethernet/mellanox/mlx4/en_tx.c +++ b/drivers/net/ethernet/mellanox/mlx4/en_tx.c @@ -354,7 +354,7 @@ u32 mlx4_en_recycle_tx_desc(struct mlx4_en_priv *priv, struct page_pool *pool =3D ring->recycle_ring->pp; =20 /* Note that napi_mode =3D 0 means ndo_close() path, not budget =3D 0 */ - page_pool_put_full_page(pool, tx_info->page, !!napi_mode); + page_pool_put_full_netmem(pool, tx_info->netmem, !!napi_mode); =20 return tx_info->nr_txbb; } @@ -1191,10 +1191,10 @@ netdev_tx_t mlx4_en_xmit_frame(struct mlx4_en_rx_ri= ng *rx_ring, tx_desc =3D ring->buf + (index << LOG_TXBB_SIZE); data =3D &tx_desc->data; =20 - dma =3D page_pool_get_dma_addr(frame->page); + dma =3D page_pool_get_dma_addr_netmem(frame->netmem); =20 - tx_info->page =3D frame->page; - frame->page =3D NULL; + tx_info->netmem =3D frame->netmem; + frame->netmem =3D 0; tx_info->map0_dma =3D dma; tx_info->nr_bytes =3D max_t(unsigned int, length, ETH_ZLEN); =20 diff --git a/drivers/net/ethernet/mellanox/mlx4/mlx4_en.h b/drivers/net/eth= ernet/mellanox/mlx4/mlx4_en.h index ad0d91a751848..3ef9a0a1f783d 100644 --- a/drivers/net/ethernet/mellanox/mlx4/mlx4_en.h +++ b/drivers/net/ethernet/mellanox/mlx4/mlx4_en.h @@ -213,7 +213,7 @@ enum cq_type { struct mlx4_en_tx_info { union { struct sk_buff *skb; - struct page *page; + netmem_ref netmem; }; dma_addr_t map0_dma; u32 map0_byte_count; @@ -246,7 +246,7 @@ struct mlx4_en_tx_desc { #define MLX4_EN_CX3_HIGH_ID 0x1005 =20 struct mlx4_en_rx_alloc { - struct page *page; + netmem_ref netmem; u32 page_offset; }; =20 --=20 2.17.1