From nobody Wed Dec 17 08:56:40 2025 Received: from invmail4.hynix.com (exvmail4.skhynix.com [166.125.252.92]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 993BE28FA98; Fri, 9 May 2025 11:51:38 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=166.125.252.92 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1746791504; cv=none; b=OcjZuiPbfH9nvzjCMzxWy43knCyQmIKQkVb/CTCN/PVUWrtBx8eBI2BWFnXotbjzAh3vLVLroqrARr8ZDVCGi+md+TXv4g7m9dzjdkFpNVrVJF9XKW8Bw/YLs/H/QFW9/rJHnIZh0kf8xtdYFhhNXIwACqthDrGhDC7sFIIVZng= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1746791504; c=relaxed/simple; bh=WTci3PN2uM1LSRFgheNdfLrJAz16t/cIFt6CBGIFCQE=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References; b=ko+dqCkS6GGvOQ+YijoQ73deEFg6WSO7X4OxSGVvBofYGU0BTXqadtg9ebolye5MyL5UGrZ/4KTFEVsdiqZDkvvbppl9U3duBSS8uNECDsmXW0pc4xGXMn+EyZ3q9m1sJmT+RFFLehZbmZM4H4Ks+Vef6aG4d6JX4JwoaLYRCB8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com; spf=pass smtp.mailfrom=sk.com; arc=none smtp.client-ip=166.125.252.92 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=sk.com X-AuditID: a67dfc5b-669ff7000002311f-2c-681dec49ee25 From: Byungchul Park To: willy@infradead.org, netdev@vger.kernel.org Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, kernel_team@skhynix.com, kuba@kernel.org, almasrymina@google.com, ilias.apalodimas@linaro.org, harry.yoo@oracle.com, hawk@kernel.org, akpm@linux-foundation.org, ast@kernel.org, daniel@iogearbox.net, davem@davemloft.net, john.fastabend@gmail.com, andrew+netdev@lunn.ch, edumazet@google.com, pabeni@redhat.com, vishal.moola@gmail.com Subject: [RFC 01/19] netmem: rename struct net_iov to struct netmem_desc Date: Fri, 9 May 2025 20:51:08 +0900 Message-Id: <20250509115126.63190-2-byungchul@sk.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20250509115126.63190-1-byungchul@sk.com> References: <20250509115126.63190-1-byungchul@sk.com> X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFnrFLMWRmVeSWpSXmKPExsXC9ZZnka7nG9kMgwmHOC3mrF/DZrH6R4XF 8gc7WC2+/LzNbrF44TdmiznnW1gsnh57xG5xf9kzFos97duZLXpbfjNbNO1YwWRxYVsfq8Xl XXPYLO6t+c9qcWyBmMW3028YLdbvu8Fq8fvHHDYHIY8tK28yeeycdZfdY8GmUo/NK7Q8um5c YvbYtKqTzWPTp0nsHneu7WHzODHjN4vHzh2fmTw+Pr3F4vF+31U2j8+b5AJ4o7hsUlJzMstS i/TtErgyJj74xFrQPoGx4tyD9AbGu+VdjJwcEgImEhc+LGCCsRd+ncQGYrMJqEvcuPGTGcQW ETCU+PzoOEsXIxcHs8BCZokri3+ygySEBTwkvp9rBmrg4GARUJVYtqIYJMwrYCpx/OciVoiZ 8hKrNxxgBinhFDCT6P+oDhIWAipZNmUBG8hICYFmdokp5w8xQ9RLShxccYNlAiPvAkaGVYxC mXlluYmZOSZ6GZV5mRV6yfm5mxiBMbCs9k/0DsZPF4IPMQpwMCrx8Fo8l80QYk0sK67MPcQo wcGsJML7vFMmQ4g3JbGyKrUoP76oNCe1+BCjNAeLkjiv0bfyFCGB9MSS1OzU1ILUIpgsEwen VAOj9bH/O1IW3lect3j1/1Xe5l4+jRZsNYY7580KOsi598b5Yz/CXHUkbKf89I7XPf/kMZfM 1wulPjHxfDu29m1cMe+l0+vQdF8jXo9la1auqTqWcvdW0rXQqaG+LyySfHsS6u5cXyB45hFn SV1ofswb9mn3zR0s+BXF1viU5rVOtBN0WR/8Sm6BEktxRqKhFnNRcSIANm6nWH0CAAA= X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFrrILMWRmVeSWpSXmKPExsXC5WfdrOvxRjbDYPVCK4s569ewWaz+UWGx /MEOVosvP2+zWyxe+I3ZYs75FhaLp8cesVvcX/aMxWJP+3Zmi96W38wWTTtWMFkcnnuS1eLC tj5Wi8u75rBZ3Fvzn9Xi2AIxi2+n3zBarN93g9Xi9485bA7CHltW3mTy2DnrLrvHgk2lHptX aHl03bjE7LFpVSebx6ZPk9g97lzbw+ZxYsZvFo+dOz4zeXx8eovF4/2+q2wei198YPL4vEku gC+KyyYlNSezLLVI3y6BK2Pig0+sBe0TGCvOPUhvYLxb3sXIySEhYCKx8OskNhCbTUBd4saN n8wgtoiAocTnR8dZuhi5OJgFFjJLXFn8kx0kISzgIfH9XDNQAwcHi4CqxLIVxSBhXgFTieM/ F7FCzJSXWL3hADNICaeAmUT/R3WQsBBQybIpC9gmMHItYGRYxSiSmVeWm5iZY6pXnJ1RmZdZ oZecn7uJERjQy2r/TNzB+OWy+yFGAQ5GJR5ei+eyGUKsiWXFlbmHGCU4mJVEeJ93ymQI8aYk VlalFuXHF5XmpBYfYpTmYFES5/UKT00QEkhPLEnNTk0tSC2CyTJxcEo1MC5KF7gbe+fe1u62 rTYdvstNFU1Ph4q9E5k5W/H4vTSldb32T4U2HuWeFNtnvnhh8OKYCSEfv3PP7uD0PfLKPXB6 beBTFY5TcaKynTdPTJxwRsf+i57O80VHH7u4vC8sZ5TcK/i3kzMoOqaUraaa49j8E19XHsv7 uk2Btbj454IXj2Y0TNx564oSS3FGoqEWc1FxIgCaEHMmZAIAAA== X-CFilter-Loop: Reflected Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" To simplify struct page, the page pool members of struct page should be moved to other, allowing these members to be removed from struct page. Reuse struct net_iov for also system memory, that already mirrored the page pool members. Signed-off-by: Byungchul Park --- include/linux/skbuff.h | 4 +-- include/net/netmem.h | 20 ++++++------ include/net/page_pool/memory_provider.h | 6 ++-- io_uring/zcrx.c | 42 ++++++++++++------------- net/core/devmem.c | 14 ++++----- net/core/devmem.h | 24 +++++++------- net/core/page_pool.c | 6 ++-- net/ipv4/tcp.c | 2 +- 8 files changed, 59 insertions(+), 59 deletions(-) diff --git a/include/linux/skbuff.h b/include/linux/skbuff.h index b974a277975a8..bf67c47319a56 100644 --- a/include/linux/skbuff.h +++ b/include/linux/skbuff.h @@ -3598,10 +3598,10 @@ static inline bool skb_frag_is_net_iov(const skb_fr= ag_t *frag) * skb_frag_net_iov - retrieve the net_iov referred to by fragment * @frag: the fragment * - * Return: the &struct net_iov associated with @frag. Returns NULL if this + * Return: the &struct netmem_desc associated with @frag. Returns NULL if = this * frag has no associated net_iov. */ -static inline struct net_iov *skb_frag_net_iov(const skb_frag_t *frag) +static inline struct netmem_desc *skb_frag_net_iov(const skb_frag_t *frag) { if (!skb_frag_is_net_iov(frag)) return NULL; diff --git a/include/net/netmem.h b/include/net/netmem.h index c61d5b21e7b42..45c209d6cc689 100644 --- a/include/net/netmem.h +++ b/include/net/netmem.h @@ -20,7 +20,7 @@ DECLARE_STATIC_KEY_FALSE(page_pool_mem_providers); */ #define NET_IOV 0x01UL =20 -struct net_iov { +struct netmem_desc { unsigned long __unused_padding; unsigned long pp_magic; struct page_pool *pp; @@ -31,7 +31,7 @@ struct net_iov { =20 struct net_iov_area { /* Array of net_iovs for this area. */ - struct net_iov *niovs; + struct netmem_desc *niovs; size_t num_niovs; =20 /* Offset into the dma-buf where this chunk starts. */ @@ -56,19 +56,19 @@ struct net_iov_area { */ #define NET_IOV_ASSERT_OFFSET(pg, iov) \ static_assert(offsetof(struct page, pg) =3D=3D \ - offsetof(struct net_iov, iov)) + offsetof(struct netmem_desc, iov)) NET_IOV_ASSERT_OFFSET(pp_magic, pp_magic); NET_IOV_ASSERT_OFFSET(pp, pp); NET_IOV_ASSERT_OFFSET(dma_addr, dma_addr); NET_IOV_ASSERT_OFFSET(pp_ref_count, pp_ref_count); #undef NET_IOV_ASSERT_OFFSET =20 -static inline struct net_iov_area *net_iov_owner(const struct net_iov *nio= v) +static inline struct net_iov_area *net_iov_owner(const struct netmem_desc = *niov) { return niov->owner; } =20 -static inline unsigned int net_iov_idx(const struct net_iov *niov) +static inline unsigned int net_iov_idx(const struct netmem_desc *niov) { return niov - net_iov_owner(niov)->niovs; } @@ -118,17 +118,17 @@ static inline struct page *netmem_to_page(netmem_ref = netmem) return __netmem_to_page(netmem); } =20 -static inline struct net_iov *netmem_to_net_iov(netmem_ref netmem) +static inline struct netmem_desc *netmem_to_net_iov(netmem_ref netmem) { if (netmem_is_net_iov(netmem)) - return (struct net_iov *)((__force unsigned long)netmem & + return (struct netmem_desc *)((__force unsigned long)netmem & ~NET_IOV); =20 DEBUG_NET_WARN_ON_ONCE(true); return NULL; } =20 -static inline netmem_ref net_iov_to_netmem(struct net_iov *niov) +static inline netmem_ref net_iov_to_netmem(struct netmem_desc *niov) { return (__force netmem_ref)((unsigned long)niov | NET_IOV); } @@ -168,9 +168,9 @@ static inline unsigned long netmem_pfn_trace(netmem_ref= netmem) return page_to_pfn(netmem_to_page(netmem)); } =20 -static inline struct net_iov *__netmem_clear_lsb(netmem_ref netmem) +static inline struct netmem_desc *__netmem_clear_lsb(netmem_ref netmem) { - return (struct net_iov *)((__force unsigned long)netmem & ~NET_IOV); + return (struct netmem_desc *)((__force unsigned long)netmem & ~NET_IOV); } =20 /** diff --git a/include/net/page_pool/memory_provider.h b/include/net/page_poo= l/memory_provider.h index ada4f968960ae..83aac2e9692c0 100644 --- a/include/net/page_pool/memory_provider.h +++ b/include/net/page_pool/memory_provider.h @@ -19,9 +19,9 @@ struct memory_provider_ops { void (*uninstall)(void *mp_priv, struct netdev_rx_queue *rxq); }; =20 -bool net_mp_niov_set_dma_addr(struct net_iov *niov, dma_addr_t addr); -void net_mp_niov_set_page_pool(struct page_pool *pool, struct net_iov *nio= v); -void net_mp_niov_clear_page_pool(struct net_iov *niov); +bool net_mp_niov_set_dma_addr(struct netmem_desc *niov, dma_addr_t addr); +void net_mp_niov_set_page_pool(struct page_pool *pool, struct netmem_desc = *niov); +void net_mp_niov_clear_page_pool(struct netmem_desc *niov); =20 int net_mp_open_rxq(struct net_device *dev, unsigned ifq_idx, struct pp_memory_provider_params *p); diff --git a/io_uring/zcrx.c b/io_uring/zcrx.c index 0f46e0404c045..c0b66039d43df 100644 --- a/io_uring/zcrx.c +++ b/io_uring/zcrx.c @@ -34,7 +34,7 @@ static void __io_zcrx_unmap_area(struct io_zcrx_ifq *ifq, int i; =20 for (i =3D 0; i < nr_mapped; i++) { - struct net_iov *niov =3D &area->nia.niovs[i]; + struct netmem_desc *niov =3D &area->nia.niovs[i]; dma_addr_t dma; =20 dma =3D page_pool_get_dma_addr_netmem(net_iov_to_netmem(niov)); @@ -55,7 +55,7 @@ static int io_zcrx_map_area(struct io_zcrx_ifq *ifq, stru= ct io_zcrx_area *area) int i; =20 for (i =3D 0; i < area->nia.num_niovs; i++) { - struct net_iov *niov =3D &area->nia.niovs[i]; + struct netmem_desc *niov =3D &area->nia.niovs[i]; dma_addr_t dma; =20 dma =3D dma_map_page_attrs(ifq->dev, area->pages[i], 0, PAGE_SIZE, @@ -79,7 +79,7 @@ static int io_zcrx_map_area(struct io_zcrx_ifq *ifq, stru= ct io_zcrx_area *area) } =20 static void io_zcrx_sync_for_device(const struct page_pool *pool, - struct net_iov *niov) + struct netmem_desc *niov) { #if defined(CONFIG_HAS_DMA) && defined(CONFIG_DMA_NEED_SYNC) dma_addr_t dma_addr; @@ -106,21 +106,21 @@ struct io_zcrx_args { =20 static const struct memory_provider_ops io_uring_pp_zc_ops; =20 -static inline struct io_zcrx_area *io_zcrx_iov_to_area(const struct net_io= v *niov) +static inline struct io_zcrx_area *io_zcrx_iov_to_area(const struct netmem= _desc *niov) { struct net_iov_area *owner =3D net_iov_owner(niov); =20 return container_of(owner, struct io_zcrx_area, nia); } =20 -static inline atomic_t *io_get_user_counter(struct net_iov *niov) +static inline atomic_t *io_get_user_counter(struct netmem_desc *niov) { struct io_zcrx_area *area =3D io_zcrx_iov_to_area(niov); =20 return &area->user_refs[net_iov_idx(niov)]; } =20 -static bool io_zcrx_put_niov_uref(struct net_iov *niov) +static bool io_zcrx_put_niov_uref(struct netmem_desc *niov) { atomic_t *uref =3D io_get_user_counter(niov); =20 @@ -130,12 +130,12 @@ static bool io_zcrx_put_niov_uref(struct net_iov *nio= v) return true; } =20 -static void io_zcrx_get_niov_uref(struct net_iov *niov) +static void io_zcrx_get_niov_uref(struct netmem_desc *niov) { atomic_inc(io_get_user_counter(niov)); } =20 -static inline struct page *io_zcrx_iov_page(const struct net_iov *niov) +static inline struct page *io_zcrx_iov_page(const struct netmem_desc *niov) { struct io_zcrx_area *area =3D io_zcrx_iov_to_area(niov); =20 @@ -242,7 +242,7 @@ static int io_zcrx_create_area(struct io_zcrx_ifq *ifq, goto err; =20 for (i =3D 0; i < nr_iovs; i++) { - struct net_iov *niov =3D &area->nia.niovs[i]; + struct netmem_desc *niov =3D &area->nia.niovs[i]; =20 niov->owner =3D &area->nia; area->freelist[i] =3D i; @@ -435,7 +435,7 @@ void io_unregister_zcrx_ifqs(struct io_ring_ctx *ctx) io_zcrx_ifq_free(ifq); } =20 -static struct net_iov *__io_zcrx_get_free_niov(struct io_zcrx_area *area) +static struct netmem_desc *__io_zcrx_get_free_niov(struct io_zcrx_area *ar= ea) { unsigned niov_idx; =20 @@ -445,7 +445,7 @@ static struct net_iov *__io_zcrx_get_free_niov(struct i= o_zcrx_area *area) return &area->nia.niovs[niov_idx]; } =20 -static void io_zcrx_return_niov_freelist(struct net_iov *niov) +static void io_zcrx_return_niov_freelist(struct netmem_desc *niov) { struct io_zcrx_area *area =3D io_zcrx_iov_to_area(niov); =20 @@ -454,7 +454,7 @@ static void io_zcrx_return_niov_freelist(struct net_iov= *niov) spin_unlock_bh(&area->freelist_lock); } =20 -static void io_zcrx_return_niov(struct net_iov *niov) +static void io_zcrx_return_niov(struct netmem_desc *niov) { netmem_ref netmem =3D net_iov_to_netmem(niov); =20 @@ -476,7 +476,7 @@ static void io_zcrx_scrub(struct io_zcrx_ifq *ifq) =20 /* Reclaim back all buffers given to the user space. */ for (i =3D 0; i < area->nia.num_niovs; i++) { - struct net_iov *niov =3D &area->nia.niovs[i]; + struct netmem_desc *niov =3D &area->nia.niovs[i]; int nr; =20 if (!atomic_read(io_get_user_counter(niov))) @@ -532,7 +532,7 @@ static void io_zcrx_ring_refill(struct page_pool *pp, do { struct io_uring_zcrx_rqe *rqe =3D io_zcrx_get_rqe(ifq, mask); struct io_zcrx_area *area; - struct net_iov *niov; + struct netmem_desc *niov; unsigned niov_idx, area_idx; =20 area_idx =3D rqe->off >> IORING_ZCRX_AREA_SHIFT; @@ -573,7 +573,7 @@ static void io_zcrx_refill_slow(struct page_pool *pp, s= truct io_zcrx_ifq *ifq) =20 spin_lock_bh(&area->freelist_lock); while (area->free_count && pp->alloc.count < PP_ALLOC_CACHE_REFILL) { - struct net_iov *niov =3D __io_zcrx_get_free_niov(area); + struct netmem_desc *niov =3D __io_zcrx_get_free_niov(area); netmem_ref netmem =3D net_iov_to_netmem(niov); =20 net_mp_niov_set_page_pool(pp, niov); @@ -604,7 +604,7 @@ static netmem_ref io_pp_zc_alloc_netmems(struct page_po= ol *pp, gfp_t gfp) =20 static bool io_pp_zc_release_netmem(struct page_pool *pp, netmem_ref netme= m) { - struct net_iov *niov; + struct netmem_desc *niov; =20 if (WARN_ON_ONCE(!netmem_is_net_iov(netmem))) return false; @@ -678,7 +678,7 @@ static const struct memory_provider_ops io_uring_pp_zc_= ops =3D { .uninstall =3D io_pp_uninstall, }; =20 -static bool io_zcrx_queue_cqe(struct io_kiocb *req, struct net_iov *niov, +static bool io_zcrx_queue_cqe(struct io_kiocb *req, struct netmem_desc *ni= ov, struct io_zcrx_ifq *ifq, int off, int len) { struct io_uring_zcrx_cqe *rcqe; @@ -701,9 +701,9 @@ static bool io_zcrx_queue_cqe(struct io_kiocb *req, str= uct net_iov *niov, return true; } =20 -static struct net_iov *io_zcrx_alloc_fallback(struct io_zcrx_area *area) +static struct netmem_desc *io_zcrx_alloc_fallback(struct io_zcrx_area *are= a) { - struct net_iov *niov =3D NULL; + struct netmem_desc *niov =3D NULL; =20 spin_lock_bh(&area->freelist_lock); if (area->free_count) @@ -726,7 +726,7 @@ static ssize_t io_zcrx_copy_chunk(struct io_kiocb *req,= struct io_zcrx_ifq *ifq, while (len) { size_t copy_size =3D min_t(size_t, PAGE_SIZE, len); const int dst_off =3D 0; - struct net_iov *niov; + struct netmem_desc *niov; struct page *dst_page; void *dst_addr; =20 @@ -784,7 +784,7 @@ static int io_zcrx_copy_frag(struct io_kiocb *req, stru= ct io_zcrx_ifq *ifq, static int io_zcrx_recv_frag(struct io_kiocb *req, struct io_zcrx_ifq *ifq, const skb_frag_t *frag, int off, int len) { - struct net_iov *niov; + struct netmem_desc *niov; =20 if (unlikely(!skb_frag_is_net_iov(frag))) return io_zcrx_copy_frag(req, ifq, frag, off, len); diff --git a/net/core/devmem.c b/net/core/devmem.c index 6e27a47d04935..5568478dd2ff6 100644 --- a/net/core/devmem.c +++ b/net/core/devmem.c @@ -28,7 +28,7 @@ static DEFINE_XARRAY_FLAGS(net_devmem_dmabuf_bindings, XA= _FLAGS_ALLOC1); =20 static const struct memory_provider_ops dmabuf_devmem_ops; =20 -bool net_is_devmem_iov(struct net_iov *niov) +bool net_is_devmem_iov(struct netmem_desc *niov) { return niov->pp->mp_ops =3D=3D &dmabuf_devmem_ops; } @@ -43,7 +43,7 @@ static void net_devmem_dmabuf_free_chunk_owner(struct gen= _pool *genpool, kfree(owner); } =20 -static dma_addr_t net_devmem_get_dma_addr(const struct net_iov *niov) +static dma_addr_t net_devmem_get_dma_addr(const struct netmem_desc *niov) { struct dmabuf_genpool_chunk_owner *owner; =20 @@ -74,12 +74,12 @@ void __net_devmem_dmabuf_binding_free(struct net_devmem= _dmabuf_binding *binding) kfree(binding); } =20 -struct net_iov * +struct netmem_desc * net_devmem_alloc_dmabuf(struct net_devmem_dmabuf_binding *binding) { struct dmabuf_genpool_chunk_owner *owner; unsigned long dma_addr; - struct net_iov *niov; + struct netmem_desc *niov; ssize_t offset; ssize_t index; =20 @@ -99,7 +99,7 @@ net_devmem_alloc_dmabuf(struct net_devmem_dmabuf_binding = *binding) return niov; } =20 -void net_devmem_free_dmabuf(struct net_iov *niov) +void net_devmem_free_dmabuf(struct netmem_desc *niov) { struct net_devmem_dmabuf_binding *binding =3D net_devmem_iov_binding(niov= ); unsigned long dma_addr =3D net_devmem_get_dma_addr(niov); @@ -233,7 +233,7 @@ net_devmem_bind_dmabuf(struct net_device *dev, unsigned= int dmabuf_fd, dma_addr_t dma_addr =3D sg_dma_address(sg); struct dmabuf_genpool_chunk_owner *owner; size_t len =3D sg_dma_len(sg); - struct net_iov *niov; + struct netmem_desc *niov; =20 owner =3D kzalloc_node(sizeof(*owner), GFP_KERNEL, dev_to_node(&dev->dev)); @@ -319,7 +319,7 @@ int mp_dmabuf_devmem_init(struct page_pool *pool) netmem_ref mp_dmabuf_devmem_alloc_netmems(struct page_pool *pool, gfp_t gf= p) { struct net_devmem_dmabuf_binding *binding =3D pool->mp_priv; - struct net_iov *niov; + struct netmem_desc *niov; netmem_ref netmem; =20 niov =3D net_devmem_alloc_dmabuf(binding); diff --git a/net/core/devmem.h b/net/core/devmem.h index 7fc158d527293..314ab0acdf716 100644 --- a/net/core/devmem.h +++ b/net/core/devmem.h @@ -70,7 +70,7 @@ int net_devmem_bind_dmabuf_to_queue(struct net_device *de= v, u32 rxq_idx, struct netlink_ext_ack *extack); =20 static inline struct dmabuf_genpool_chunk_owner * -net_devmem_iov_to_chunk_owner(const struct net_iov *niov) +net_devmem_iov_to_chunk_owner(const struct netmem_desc *niov) { struct net_iov_area *owner =3D net_iov_owner(niov); =20 @@ -78,17 +78,17 @@ net_devmem_iov_to_chunk_owner(const struct net_iov *nio= v) } =20 static inline struct net_devmem_dmabuf_binding * -net_devmem_iov_binding(const struct net_iov *niov) +net_devmem_iov_binding(const struct netmem_desc *niov) { return net_devmem_iov_to_chunk_owner(niov)->binding; } =20 -static inline u32 net_devmem_iov_binding_id(const struct net_iov *niov) +static inline u32 net_devmem_iov_binding_id(const struct netmem_desc *niov) { return net_devmem_iov_binding(niov)->id; } =20 -static inline unsigned long net_iov_virtual_addr(const struct net_iov *nio= v) +static inline unsigned long net_iov_virtual_addr(const struct netmem_desc = *niov) { struct net_iov_area *owner =3D net_iov_owner(niov); =20 @@ -111,11 +111,11 @@ net_devmem_dmabuf_binding_put(struct net_devmem_dmabu= f_binding *binding) __net_devmem_dmabuf_binding_free(binding); } =20 -struct net_iov * +struct netmem_desc * net_devmem_alloc_dmabuf(struct net_devmem_dmabuf_binding *binding); -void net_devmem_free_dmabuf(struct net_iov *ppiov); +void net_devmem_free_dmabuf(struct netmem_desc *ppiov); =20 -bool net_is_devmem_iov(struct net_iov *niov); +bool net_is_devmem_iov(struct netmem_desc *niov); =20 #else struct net_devmem_dmabuf_binding; @@ -146,27 +146,27 @@ net_devmem_bind_dmabuf_to_queue(struct net_device *de= v, u32 rxq_idx, return -EOPNOTSUPP; } =20 -static inline struct net_iov * +static inline struct netmem_desc * net_devmem_alloc_dmabuf(struct net_devmem_dmabuf_binding *binding) { return NULL; } =20 -static inline void net_devmem_free_dmabuf(struct net_iov *ppiov) +static inline void net_devmem_free_dmabuf(struct netmem_desc *ppiov) { } =20 -static inline unsigned long net_iov_virtual_addr(const struct net_iov *nio= v) +static inline unsigned long net_iov_virtual_addr(const struct netmem_desc = *niov) { return 0; } =20 -static inline u32 net_devmem_iov_binding_id(const struct net_iov *niov) +static inline u32 net_devmem_iov_binding_id(const struct netmem_desc *niov) { return 0; } =20 -static inline bool net_is_devmem_iov(struct net_iov *niov) +static inline bool net_is_devmem_iov(struct netmem_desc *niov) { return false; } diff --git a/net/core/page_pool.c b/net/core/page_pool.c index 7745ad924ae2d..575fdab337414 100644 --- a/net/core/page_pool.c +++ b/net/core/page_pool.c @@ -1198,7 +1198,7 @@ void page_pool_update_nid(struct page_pool *pool, int= new_nid) } EXPORT_SYMBOL(page_pool_update_nid); =20 -bool net_mp_niov_set_dma_addr(struct net_iov *niov, dma_addr_t addr) +bool net_mp_niov_set_dma_addr(struct netmem_desc *niov, dma_addr_t addr) { return page_pool_set_dma_addr_netmem(net_iov_to_netmem(niov), addr); } @@ -1206,7 +1206,7 @@ bool net_mp_niov_set_dma_addr(struct net_iov *niov, d= ma_addr_t addr) /* Associate a niov with a page pool. Should follow with a matching * net_mp_niov_clear_page_pool() */ -void net_mp_niov_set_page_pool(struct page_pool *pool, struct net_iov *nio= v) +void net_mp_niov_set_page_pool(struct page_pool *pool, struct netmem_desc = *niov) { netmem_ref netmem =3D net_iov_to_netmem(niov); =20 @@ -1219,7 +1219,7 @@ void net_mp_niov_set_page_pool(struct page_pool *pool= , struct net_iov *niov) /* Disassociate a niov from a page pool. Should only be used in the * ->release_netmem() path. */ -void net_mp_niov_clear_page_pool(struct net_iov *niov) +void net_mp_niov_clear_page_pool(struct netmem_desc *niov) { netmem_ref netmem =3D net_iov_to_netmem(niov); =20 diff --git a/net/ipv4/tcp.c b/net/ipv4/tcp.c index 6edc441b37023..76199c832ef82 100644 --- a/net/ipv4/tcp.c +++ b/net/ipv4/tcp.c @@ -2483,7 +2483,7 @@ static int tcp_recvmsg_dmabuf(struct sock *sk, const = struct sk_buff *skb, */ for (i =3D 0; i < skb_shinfo(skb)->nr_frags; i++) { skb_frag_t *frag =3D &skb_shinfo(skb)->frags[i]; - struct net_iov *niov; + struct netmem_desc *niov; u64 frag_offset; int end; =20 --=20 2.17.1 From nobody Wed Dec 17 08:56:40 2025 Received: from invmail4.hynix.com (exvmail4.skhynix.com [166.125.252.92]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 992F428F930; Fri, 9 May 2025 11:51:38 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=166.125.252.92 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1746791505; cv=none; b=ematUwLqL1XLLl5OjUXv24TTf2O5a6ygiuMEltAxHBfhPB46UJg0IHQSRpigHrNwKqvqIDgn1q9pZnEI358W7kUowZYooy5LZVt/HG1hL582VOAIZY2KN35pdFAanKVbRGEDrQOgwz1KOPA/OgSwUeIj5qGuRHCQpQwx82Wbfuc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1746791505; c=relaxed/simple; bh=MaWnivG0bkyZnL3UV4hhAq5uTzeVjyZQ1dynQ9IhHto=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References; b=tQLkh3SXarXC9+J9pWS7TqVkzVV9DAt4MsaV4FA07xiiPieX1bSewccx8xMd6j7VIYuZ7mf89bmiOzuxE5pQ6nvLvrL1stwnC9NGZ/tudLwJDZVUm0BkSzfQVT1g3kv6xpi/C6lwXQNbLqQ0JBzGHPdQEgiYFq2KhlUWnySqHBg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com; spf=pass smtp.mailfrom=sk.com; arc=none smtp.client-ip=166.125.252.92 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=sk.com X-AuditID: a67dfc5b-669ff7000002311f-32-681dec49c7b9 From: Byungchul Park To: willy@infradead.org, netdev@vger.kernel.org Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, kernel_team@skhynix.com, kuba@kernel.org, almasrymina@google.com, ilias.apalodimas@linaro.org, harry.yoo@oracle.com, hawk@kernel.org, akpm@linux-foundation.org, ast@kernel.org, daniel@iogearbox.net, davem@davemloft.net, john.fastabend@gmail.com, andrew+netdev@lunn.ch, edumazet@google.com, pabeni@redhat.com, vishal.moola@gmail.com Subject: [RFC 02/19] netmem: introduce netmem alloc/put API to wrap page alloc/put API Date: Fri, 9 May 2025 20:51:09 +0900 Message-Id: <20250509115126.63190-3-byungchul@sk.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20250509115126.63190-1-byungchul@sk.com> References: <20250509115126.63190-1-byungchul@sk.com> X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFnrFLMWRmVeSWpSXmKPExsXC9ZZnka7nG9kMg5W/uSzmrF/DZrH6R4XF 8gc7WC2+/LzNbrF44TdmiznnW1gsnh57xG5xf9kzFos97duZLXpbfjNbNO1YwWRxYVsfq8Xl XXPYLO6t+c9qcWyBmMW3028YLdbvu8Fq8fvHHDYHIY8tK28yeeycdZfdY8GmUo/NK7Q8um5c YvbYtKqTzWPTp0nsHneu7WHzODHjN4vHzh2fmTw+Pr3F4vF+31U2j8+b5AJ4o7hsUlJzMstS i/TtErgylnxqZC2YxFUxa9cP1gbGbRxdjJwcEgImEpc7+lm7GDnA7L8rA0DCbALqEjdu/GQG sUUEDCU+PzrO0sXIxcEssJBZ4srin+wgCWGBMIkty6YygtgsAqoS/aeesoHYvAKmEkeunGaD mC8vsXrDAWaQ+ZwCZhL9H9VBwkJAJcumLGADmSkh0MwucXvbOkaIekmJgytusExg5F3AyLCK USgzryw3MTPHRC+jMi+zQi85P3cTIzAGltX+id7B+OlC8CFGAQ5GJR5ei+eyGUKsiWXFlbmH GCU4mJVEeJ93ymQI8aYkVlalFuXHF5XmpBYfYpTmYFES5zX6Vp4iJJCeWJKanZpakFoEk2Xi 4JRqYFSP/Ly26jtTRF/ejLwtfvH2m93tRR7a3PsS9fdVUOauFxdLbTlvBq7zeD7hWbZ5+Yzq 2om+9p+vnQzKEg9LT8hzXS6nEujSdMZAyPWVNM+a/+9t7P/LnZecrP1xU/WGlCg+448mV3c9 vbJbWG3z3hq3y8mPv4W6rZ6jVpp2wsdLUr28Zw+bqBJLcUaioRZzUXEiAOuMkL99AgAA X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFrrILMWRmVeSWpSXmKPExsXC5WfdrOv5RjbD4MgNRos569ewWaz+UWGx /MEOVosvP2+zWyxe+I3ZYs75FhaLp8cesVvcX/aMxWJP+3Zmi96W38wWTTtWMFkcnnuS1eLC tj5Wi8u75rBZ3Fvzn9Xi2AIxi2+n3zBarN93g9Xi9485bA7CHltW3mTy2DnrLrvHgk2lHptX aHl03bjE7LFpVSebx6ZPk9g97lzbw+ZxYsZvFo+dOz4zeXx8eovF4/2+q2wei198YPL4vEku gC+KyyYlNSezLLVI3y6BK2PJp0bWgklcFbN2/WBtYNzG0cXIwSEhYCLxd2VAFyMnB5uAusSN Gz+ZQWwRAUOJz4+Os3QxcnEwCyxklriy+Cc7SEJYIExiy7KpjCA2i4CqRP+pp2wgNq+AqcSR K6fBbAkBeYnVGw4wg8znFDCT6P+oDhIWAipZNmUB2wRGrgWMDKsYRTLzynITM3NM9YqzMyrz Miv0kvNzNzECA3pZ7Z+JOxi/XHY/xCjAwajEw2vxXDZDiDWxrLgy9xCjBAezkgjv806ZDCHe lMTKqtSi/Pii0pzU4kOM0hwsSuK8XuGpCUIC6YklqdmpqQWpRTBZJg5OqQbGubUxFo9ePvLO K3HPCbuaWNCZpntlugdbiJwqw88jJvw+VRvNtGX2Si3yMNkqWHhhz+x30apTOb8+27tnM7fS BTOnulbWqy6+vpZ1wosuhMcny3rH82u4sJ9jspu27kPEV9ktdfUKQfPbz0a/7ZjQInzq4Yue kumvFeQXfMy/PfdKpbyawRclluKMREMt5qLiRADMbW5KZAIAAA== X-CFilter-Loop: Reflected Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" To eliminate the use of struct page in page pool, the page pool code should use netmem descriptor and API instead. As part of the work, introduce netmem alloc/put API allowing the code to use them rather than struct page things. Signed-off-by: Byungchul Park --- include/net/netmem.h | 18 ++++++++++++++++++ 1 file changed, 18 insertions(+) diff --git a/include/net/netmem.h b/include/net/netmem.h index 45c209d6cc689..c87a604e980b9 100644 --- a/include/net/netmem.h +++ b/include/net/netmem.h @@ -138,6 +138,24 @@ static inline netmem_ref page_to_netmem(struct page *p= age) return (__force netmem_ref)page; } =20 +static inline netmem_ref alloc_netmems_node(int nid, gfp_t gfp_mask, + unsigned int order) +{ + return page_to_netmem(alloc_pages_node(nid, gfp_mask, order)); +} + +static inline unsigned long alloc_netmems_bulk_node(gfp_t gfp, int nid, + unsigned long nr_netmems, netmem_ref *netmem_array) +{ + return alloc_pages_bulk_node(gfp, nid, nr_netmems, + (struct page **)netmem_array); +} + +static inline void put_netmem(netmem_ref netmem) +{ + put_page(netmem_to_page(netmem)); +} + /** * virt_to_netmem - convert virtual memory pointer to a netmem reference * @data: host memory pointer to convert --=20 2.17.1 From nobody Wed Dec 17 08:56:40 2025 Received: from invmail4.hynix.com (exvmail4.skhynix.com [166.125.252.92]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 9F7E828FA9F; Fri, 9 May 2025 11:51:38 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=166.125.252.92 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1746791504; cv=none; b=N44R7r83qmiikHrtZWvHbGAtHYgKX4UrJHp8BGI299h2WvObCgFDvNgsC+ckFOXfss+9J5LntYEe7IahQvLRejjK3ejaTr7gZH9XtinTIZXNjWsCZmzGvVyjy6hfZJOHhxlGIV2vIoEcuVgSbmK0+dzmFnKp4IYYpu5wXa+IJqk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1746791504; c=relaxed/simple; bh=AfghNN/wTX/92J/6suI/Y/6yoO250NFXx+K8SJzJryk=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References; b=bMhnNN2GmcyPEtMPsdk8abnGhgrQinJhrvvuMI36cjceQwZ0HbXj1wUJxIjfcAQj9ypAEvXZQ4ISEgzrDqwt229HhRyVTdQlo0I7SDxjnNcB4V3iun3W93/VvUheRrprH6qNjKxZbl/CTkykNdKn8BMEUIu00JwB+qeU7bpuvtk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com; spf=pass smtp.mailfrom=sk.com; arc=none smtp.client-ip=166.125.252.92 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=sk.com X-AuditID: a67dfc5b-669ff7000002311f-38-681dec495411 From: Byungchul Park To: willy@infradead.org, netdev@vger.kernel.org Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, kernel_team@skhynix.com, kuba@kernel.org, almasrymina@google.com, ilias.apalodimas@linaro.org, harry.yoo@oracle.com, hawk@kernel.org, akpm@linux-foundation.org, ast@kernel.org, daniel@iogearbox.net, davem@davemloft.net, john.fastabend@gmail.com, andrew+netdev@lunn.ch, edumazet@google.com, pabeni@redhat.com, vishal.moola@gmail.com Subject: [RFC 03/19] page_pool: use netmem alloc/put API in __page_pool_alloc_page_order() Date: Fri, 9 May 2025 20:51:10 +0900 Message-Id: <20250509115126.63190-4-byungchul@sk.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20250509115126.63190-1-byungchul@sk.com> References: <20250509115126.63190-1-byungchul@sk.com> X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFnrBLMWRmVeSWpSXmKPExsXC9ZZnoa7nG9kMg+Pz+SzmrF/DZrH6R4XF 8gc7WC2+/LzNbrF44TdmiznnW1gsnh57xG5xf9kzFos97duZLXpbfjNbNO1YwWRxYVsfq8Xl XXPYLO6t+c9qcWyBmMW3028YLdbvu8Fq8fvHHDYHIY8tK28yeeycdZfdY8GmUo/NK7Q8um5c YvbYtKqTzWPTp0nsHneu7WHzODHjN4vHzh2fmTw+Pr3F4vF+31U2j8+b5AJ4o7hsUlJzMstS i/TtErgy5nXLFVwXqOhq7WNtYPzK28XIySEhYCLxtn8BI4y9+GAfC4jNJqAucePGT2YQW0TA UOLzo+NAcS4OZoGFzBJXFv9k72Lk4BAWiJLY05AKYrIIqErM/S0BUs4rYCrxaMUHJoiR8hKr NxxgBinhFDCT6P+oDhIWAipZNmUBG8hECYHfbBKLn3WxQ9RLShxccYNlAiPvAkaGVYxCmXll uYmZOSZ6GZV5mRV6yfm5mxiB4b+s9k/0DsZPF4IPMQpwMCrx8Fo8l80QYk0sK67MPcQowcGs JML7vFMmQ4g3JbGyKrUoP76oNCe1+BCjNAeLkjiv0bfyFCGB9MSS1OzU1ILUIpgsEwenVAMj /9api2/dbyr88H+mutBp2U7WR5Kdy+If1Ja1ti42PVBcLf/mYu5lf5+lKxpmxqc8XRhkf5Xf xflrXMjcd7ZV/W8aJk8Q/RMXsDul6vB8BfXFwft8Pv093cUwWWr3kfe2py6GOv2zCzrArDp3 rlj1xBWH7Nserwr2XjVbbZ6TH+/hmP7Lmp2eSizFGYmGWsxFxYkA7TKgs3sCAAA= X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFjrPLMWRmVeSWpSXmKPExsXC5WfdrOv5RjbD4OMfZos569ewWaz+UWGx /MEOVosvP2+zWyxe+A0ofr6FxeLpsUfsFveXPWOx2NO+ndmit+U3s0XTjhVMFofnnmS1uLCt j9Xi8q45bBb31vxntTi2QMzi2+k3jBbr991gtfj9Yw6bg7DHlpU3mTx2zrrL7rFgU6nH5hVa Hl03LjF7bFrVyeax6dMkdo871/aweZyY8ZvFY+eOz0weH5/eYvF4v+8qm8fiFx+YPD5vkgvg i+KySUnNySxLLdK3S+DKmNctV3BdoKKrtY+1gfErbxcjJ4eEgInE4oN9LCA2m4C6xI0bP5lB bBEBQ4nPj44Dxbk4mAUWMktcWfyTvYuRg0NYIEpiT0MqiMkioCox97cESDmvgKnEoxUfmCBG ykus3nCAGaSEU8BMov+jOkhYCKhk2ZQFbBMYuRYwMqxiFMnMK8tNzMwx1SvOzqjMy6zQS87P 3cQIDOZltX8m7mD8ctn9EKMAB6MSD6/Fc9kMIdbEsuLK3EOMEhzMSiK8zztlMoR4UxIrq1KL 8uOLSnNSiw8xSnOwKInzeoWnJggJpCeWpGanphakFsFkmTg4pRoYJzeo6l5kX37GX2bH8+U5 W6ITO49EqHzesGnGss+vTqnLaYnO357740/Zr10xSzmaX07gXt159pzMymTrODb2h60lUV/d 1kUfNXwt5Rfz59GWLzq3lb37bxj2TNzYsNTwwcVrceKfnph/YU+IuhLWdaaxItWEqV/z+IMH LzbduSfgXszyQVQvRomlOCPRUIu5qDgRAKiZ4UZiAgAA X-CFilter-Loop: Reflected Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Use netmem alloc/put API instead of page alloc/put API and make it return netmem_ref instead of struct page * in __page_pool_alloc_page_order(). Signed-off-by: Byungchul Park --- net/core/page_pool.c | 24 ++++++++++++------------ 1 file changed, 12 insertions(+), 12 deletions(-) diff --git a/net/core/page_pool.c b/net/core/page_pool.c index 575fdab337414..9f5e07a15f707 100644 --- a/net/core/page_pool.c +++ b/net/core/page_pool.c @@ -498,29 +498,29 @@ static bool page_pool_dma_map(struct page_pool *pool,= netmem_ref netmem) return false; } =20 -static struct page *__page_pool_alloc_page_order(struct page_pool *pool, +static netmem_ref __page_pool_alloc_page_order(struct page_pool *pool, gfp_t gfp) { - struct page *page; + netmem_ref netmem; =20 gfp |=3D __GFP_COMP; - page =3D alloc_pages_node(pool->p.nid, gfp, pool->p.order); - if (unlikely(!page)) - return NULL; + netmem =3D alloc_netmems_node(pool->p.nid, gfp, pool->p.order); + if (unlikely(!netmem)) + return 0; =20 - if (pool->dma_map && unlikely(!page_pool_dma_map(pool, page_to_netmem(pag= e)))) { - put_page(page); - return NULL; + if (pool->dma_map && unlikely(!page_pool_dma_map(pool, netmem))) { + put_netmem(netmem); + return 0; } =20 alloc_stat_inc(pool, slow_high_order); - page_pool_set_pp_info(pool, page_to_netmem(page)); + page_pool_set_pp_info(pool, netmem); =20 /* Track how many pages are held 'in-flight' */ pool->pages_state_hold_cnt++; - trace_page_pool_state_hold(pool, page_to_netmem(page), + trace_page_pool_state_hold(pool, netmem, pool->pages_state_hold_cnt); - return page; + return netmem; } =20 /* slow path */ @@ -535,7 +535,7 @@ static noinline netmem_ref __page_pool_alloc_pages_slow= (struct page_pool *pool, =20 /* Don't support bulk alloc for high-order pages */ if (unlikely(pp_order)) - return page_to_netmem(__page_pool_alloc_page_order(pool, gfp)); + return __page_pool_alloc_page_order(pool, gfp); =20 /* Unnecessary as alloc cache is empty, but guarantees zero count */ if (unlikely(pool->alloc.count > 0)) --=20 2.17.1 From nobody Wed Dec 17 08:56:40 2025 Received: from invmail4.hynix.com (exvmail4.skhynix.com [166.125.252.92]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 9935828F95E; Fri, 9 May 2025 11:51:38 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=166.125.252.92 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1746791504; cv=none; b=G0LvArfoDnp2xChziuQ214t2ZVxWgMXzYXJNdTAcqZd2w9Ov5/big7m/ucogmuFTQ+K0h+xPyCu1Fh0iBsD7G1x66xfElfeSjKbqCINZqk3PGhufu2hubPvHnADd2PZr+tUqMyCvFF2D7lWhgfJZzU5IRVC+ffrsFMx/qRlrJEk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1746791504; c=relaxed/simple; bh=gRcToev8ITqpNNNz/rVPb36ZaDdlnPkMef4UC4TXKSU=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References; b=eLKp+kjRTNMEBgnxcI7uWXUvDMnS9Q0S8mh0d4nKiW4ke9WPg0qn14m1QWCeMNabMJhZ2pFSOfnxBgSP62UjIllOnQUcpMuqE8MMOYdPpr3hPxb4mv2+ZnIgQqe+wr0TrQaIWcX+HMMwcvqv1RgtT1hCCebhsYaExnhN2MqKkdY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com; spf=pass smtp.mailfrom=sk.com; arc=none smtp.client-ip=166.125.252.92 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=sk.com X-AuditID: a67dfc5b-669ff7000002311f-3e-681dec496f14 From: Byungchul Park To: willy@infradead.org, netdev@vger.kernel.org Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, kernel_team@skhynix.com, kuba@kernel.org, almasrymina@google.com, ilias.apalodimas@linaro.org, harry.yoo@oracle.com, hawk@kernel.org, akpm@linux-foundation.org, ast@kernel.org, daniel@iogearbox.net, davem@davemloft.net, john.fastabend@gmail.com, andrew+netdev@lunn.ch, edumazet@google.com, pabeni@redhat.com, vishal.moola@gmail.com Subject: [RFC 04/19] page_pool: rename __page_pool_alloc_page_order() to __page_pool_alloc_large_netmem() Date: Fri, 9 May 2025 20:51:11 +0900 Message-Id: <20250509115126.63190-5-byungchul@sk.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20250509115126.63190-1-byungchul@sk.com> References: <20250509115126.63190-1-byungchul@sk.com> X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFnrFLMWRmVeSWpSXmKPExsXC9ZZnka7nG9kMg3sLRCzmrF/DZrH6R4XF 8gc7WC2+/LzNbrF44TdmiznnW1gsnh57xG5xf9kzFos97duZLXpbfjNbNO1YwWRxYVsfq8Xl XXPYLO6t+c9qcWyBmMW3028YLdbvu8Fq8fvHHDYHIY8tK28yeeycdZfdY8GmUo/NK7Q8um5c YvbYtKqTzWPTp0nsHneu7WHzODHjN4vHzh2fmTw+Pr3F4vF+31U2j8+b5AJ4o7hsUlJzMstS i/TtErgyej5KF6zlrDjw7j5zA+Mb9i5GTg4JAROJtZtfsMHY7yf1MYLYbALqEjdu/GQGsUUE DCU+PzrO0sXIxcEssJBZ4srin2DNwgKZEktuLQFrZhFQlXi0bT2YzStgKrGi/yYTxFB5idUb DgAN4uDgFDCT6P+oDhIWAipZNmUBG8hMCYH/bBJTO5awQNRLShxccYNlAiPvAkaGVYxCmXll uYmZOSZ6GZV5mRV6yfm5mxiBMbCs9k/0DsZPF4IPMQpwMCrx8Fo8l80QYk0sK67MPcQowcGs JML7vFMmQ4g3JbGyKrUoP76oNCe1+BCjNAeLkjiv0bfyFCGB9MSS1OzU1ILUIpgsEwenVANj yGf9koJz3woLPN5kubRNlP5sKzNH+2ppIoPTpispc0u6ZiQ4rixN4Xky+e5D/x32WZvOT9or Lv/oln9o8+6nf15prLAU5tyn/HvLvJzugrSn5ssTZyrMe2d0cIbL5Hm72K/p3ruzJ6Q6N3nC fuWJtR9WKe99qR7/4/hn/QmL5c40vNx5o+XPSSWW4oxEQy3mouJEAAE+uah9AgAA X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFrrILMWRmVeSWpSXmKPExsXC5WfdrOv5RjbD4ORODos569ewWaz+UWGx /MEOVosvP2+zWyxe+I3ZYs75FhaLp8cesVvcX/aMxWJP+3Zmi96W38wWTTtWMFkcnnuS1eLC tj5Wi8u75rBZ3Fvzn9Xi2AIxi2+n3zBarN93g9Xi9485bA7CHltW3mTy2DnrLrvHgk2lHptX aHl03bjE7LFpVSebx6ZPk9g97lzbw+ZxYsZvFo+dOz4zeXx8eovF4/2+q2wei198YPL4vEku gC+KyyYlNSezLLVI3y6BK6Pno3TBWs6KA+/uMzcwvmHvYuTkkBAwkXg/qY8RxGYTUJe4ceMn M4gtImAo8fnRcZYuRi4OZoGFzBJXFv8EaxAWyJRYcmsJG4jNIqAq8WjbejCbV8BUYkX/TSaI ofISqzccABrEwcEpYCbR/1EdJCwEVLJsygK2CYxcCxgZVjGKZOaV5SZm5pjqFWdnVOZlVugl 5+duYgQG9LLaPxN3MH657H6IUYCDUYmH1+K5bIYQa2JZcWXuIUYJDmYlEd7nnTIZQrwpiZVV qUX58UWlOanFhxilOViUxHm9wlMThATSE0tSs1NTC1KLYLJMHJxSDYyC673nR3iypd/h1Kwz 2754d8S7ZoaoC7ciDf2/sGWox2x6cu+lU+emm2K/j5zave4Sr0+DfY7EO0mv/Rk6juY3Dlh5 dqlsWMuoIfTouvq1Oz2GLFkezx+2NX56daP/93vNvwuLFi1wFNiucuRCx9XfDzg4w3Z9Wbou Q77RR/Bbq9ekDPujj6YosRRnJBpqMRcVJwIAwB70d2QCAAA= X-CFilter-Loop: Reflected Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Now that __page_pool_alloc_page_order() uses netmem alloc/put API, not page alloc/put API, rename it to __page_pool_alloc_large_netmem() to reflect what it does. Signed-off-by: Byungchul Park --- net/core/page_pool.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/net/core/page_pool.c b/net/core/page_pool.c index 9f5e07a15f707..c03caa11fc606 100644 --- a/net/core/page_pool.c +++ b/net/core/page_pool.c @@ -498,7 +498,7 @@ static bool page_pool_dma_map(struct page_pool *pool, n= etmem_ref netmem) return false; } =20 -static netmem_ref __page_pool_alloc_page_order(struct page_pool *pool, +static netmem_ref __page_pool_alloc_large_netmem(struct page_pool *pool, gfp_t gfp) { netmem_ref netmem; @@ -535,7 +535,7 @@ static noinline netmem_ref __page_pool_alloc_pages_slow= (struct page_pool *pool, =20 /* Don't support bulk alloc for high-order pages */ if (unlikely(pp_order)) - return __page_pool_alloc_page_order(pool, gfp); + return __page_pool_alloc_large_netmem(pool, gfp); =20 /* Unnecessary as alloc cache is empty, but guarantees zero count */ if (unlikely(pool->alloc.count > 0)) --=20 2.17.1 From nobody Wed Dec 17 08:56:40 2025 Received: from invmail4.hynix.com (exvmail4.skhynix.com [166.125.252.92]) by smtp.subspace.kernel.org (Postfix) with ESMTP id A0E6B28FAA7; Fri, 9 May 2025 11:51:38 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=166.125.252.92 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1746791504; cv=none; b=UBO3CYaGplxFSVyxRnBwKQEHoo77xt2SgeZyLDixUBaq8W95tj+g/2I+89/bO4Xkvwpeya4F2K+tCAOzmuIi5eLvUl/vALud1QPKhleEdfay5BGe/SDqN6V4v7ZsdD6wdDDWs6tyl3/JjPQNBA1hjDj0M9QJ7FYUEkyRwFVj/9g= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1746791504; c=relaxed/simple; bh=NvxFm06XQPMr5ZW9JmyqSkOq3f4umYXIgqYFpOzJOOg=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References; b=NRyBH/S83pz33fv4KkGqkURrkzZWnshAolElJadW1/mhGwqufc7T7kCnp4pcKhTQEdGfysmDmW/rmGjFQmB7mFv1t8NF4SyHmkZuVvQ8ioxN51fGhoCWUbYdMy+hZbmcL1MHLe41sWUDaCmG9U3Q0wAeAXqvqjOjM7AdoRHaZVM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com; spf=pass smtp.mailfrom=sk.com; arc=none smtp.client-ip=166.125.252.92 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=sk.com X-AuditID: a67dfc5b-669ff7000002311f-44-681dec49fa7f From: Byungchul Park To: willy@infradead.org, netdev@vger.kernel.org Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, kernel_team@skhynix.com, kuba@kernel.org, almasrymina@google.com, ilias.apalodimas@linaro.org, harry.yoo@oracle.com, hawk@kernel.org, akpm@linux-foundation.org, ast@kernel.org, daniel@iogearbox.net, davem@davemloft.net, john.fastabend@gmail.com, andrew+netdev@lunn.ch, edumazet@google.com, pabeni@redhat.com, vishal.moola@gmail.com Subject: [RFC 05/19] page_pool: use netmem alloc/put API in __page_pool_alloc_pages_slow() Date: Fri, 9 May 2025 20:51:12 +0900 Message-Id: <20250509115126.63190-6-byungchul@sk.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20250509115126.63190-1-byungchul@sk.com> References: <20250509115126.63190-1-byungchul@sk.com> X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFnrNLMWRmVeSWpSXmKPExsXC9ZZnoa7nG9kMgwlvRC3mrF/DZrH6R4XF 8gc7WC2+/LzNbrF44TdmiznnW1gsnh57xG5xf9kzFos97duZLXpbfjNbNO1YwWRxYVsfq8Xl XXPYLO6t+c9qcWyBmMW3028YLdbvu8Fq8fvHHDYHIY8tK28yeeycdZfdY8GmUo/NK7Q8um5c YvbYtKqTzWPTp0nsHneu7WHzODHjN4vHzh2fmTw+Pr3F4vF+31U2j8+b5AJ4o7hsUlJzMstS i/TtErgy/q1ZxVKwT6Ti8s3FbA2MZwS6GDk5JARMJG7dbmKCsR++38QKYrMJqEvcuPGTGcQW ETCU+PzoOEsXIxcHs8BCZokri3+ygySEBaIkvh/7D9bMIqAqsehSA1icV8BUYvOBy6wQQ+Ul Vm84ADSIg4NTwEyi/6M6SFgIqGTZlAVsIDMlBP6zSUzaeJYRol5S4uCKGywTGHkXMDKsYhTK zCvLTczMMdHLqMzLrNBLzs/dxAiMgmW1f6J3MH66EHyIUYCDUYmH1+K5bIYQa2JZcWXuIUYJ DmYlEd7nnTIZQrwpiZVVqUX58UWlOanFhxilOViUxHmNvpWnCAmkJ5akZqemFqQWwWSZODil GhibZdZLsfFFMNzx9K/5KN2244/KEi9vsSlTfP/wzFpgL8TeVflMJeJt1t4TV6sd2o4G8/e/ vCiUrWyZOVHo2WmZfuXAtycaz94/fCkiNKtJu85l1ovYh++a2Dl5M938M5tZ2bV/Vp4ScQ9L 9L69aq3vmgnqyyN2rDbi/PxV92i4irvhxugVa5RYijMSDbWYi4oTAWaa+r5+AgAA X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFrrELMWRmVeSWpSXmKPExsXC5WfdrOv5RjbD4NYzHos569ewWaz+UWGx /MEOVosvP2+zWyxe+I3ZYs75FhaLp8cesVvcX/aMxWJP+3Zmi96W38wWTTtWMFkcnnuS1eLC tj5Wi8u75rBZ3Fvzn9Xi2AIxi2+n3zBarN93g9Xi9485bA7CHltW3mTy2DnrLrvHgk2lHptX aHl03bjE7LFpVSebx6ZPk9g97lzbw+ZxYsZvFo+dOz4zeXx8eovF4/2+q2wei198YPL4vEku gC+KyyYlNSezLLVI3y6BK+PfmlUsBftEKi7fXMzWwHhGoIuRk0NCwETi4ftNrCA2m4C6xI0b P5lBbBEBQ4nPj46zdDFycTALLGSWuLL4JztIQlggSuL7sf9MIDaLgKrEoksNYHFeAVOJzQcu s0IMlZdYveEA0CAODk4BM4n+j+ogYSGgkmVTFrBNYORawMiwilEkM68sNzEzx1SvODujMi+z Qi85P3cTIzCkl9X+mbiD8ctl90OMAhyMSjy8Fs9lM4RYE8uKK3MPMUpwMCuJ8D7vlMkQ4k1J rKxKLcqPLyrNSS0+xCjNwaIkzusVnpogJJCeWJKanZpakFoEk2Xi4JRqYJwUEpsmcfZm1Jq7 R2ZMNvyRWqAmpC11cdvBAmexqvWivfY+R5x6AyOaktoLN1QlPmzqMIsUPLPickr39BN8W8/t UTjO93s9z46CP7axeYYLGsR5pOxD1SpS1C/VPtab80Pp/sFU/U1MzgYXhH862+05uc5959J9 5x6pH9GUXf542g5Ovssp25RYijMSDbWYi4oTARRwE2xlAgAA X-CFilter-Loop: Reflected Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Use netmem alloc/put API instead of page alloc/put API in __page_pool_alloc_pages_slow(). While at it, improved some comments. Signed-off-by: Byungchul Park --- net/core/page_pool.c | 23 ++++++++++++----------- 1 file changed, 12 insertions(+), 11 deletions(-) diff --git a/net/core/page_pool.c b/net/core/page_pool.c index c03caa11fc606..57ad133e6dc8c 100644 --- a/net/core/page_pool.c +++ b/net/core/page_pool.c @@ -531,7 +531,7 @@ static noinline netmem_ref __page_pool_alloc_pages_slow= (struct page_pool *pool, unsigned int pp_order =3D pool->p.order; bool dma_map =3D pool->dma_map; netmem_ref netmem; - int i, nr_pages; + int i, nr_netmems; =20 /* Don't support bulk alloc for high-order pages */ if (unlikely(pp_order)) @@ -541,21 +541,21 @@ static noinline netmem_ref __page_pool_alloc_pages_sl= ow(struct page_pool *pool, if (unlikely(pool->alloc.count > 0)) return pool->alloc.cache[--pool->alloc.count]; =20 - /* Mark empty alloc.cache slots "empty" for alloc_pages_bulk */ + /* Mark empty alloc.cache slots "empty" for alloc_netmems_bulk_node() */ memset(&pool->alloc.cache, 0, sizeof(void *) * bulk); =20 - nr_pages =3D alloc_pages_bulk_node(gfp, pool->p.nid, bulk, - (struct page **)pool->alloc.cache); - if (unlikely(!nr_pages)) + nr_netmems =3D alloc_netmems_bulk_node(gfp, pool->p.nid, bulk, + pool->alloc.cache); + if (unlikely(!nr_netmems)) return 0; =20 - /* Pages have been filled into alloc.cache array, but count is zero and - * page element have not been (possibly) DMA mapped. + /* Netmems have been filled into alloc.cache array, but count is + * zero and elements have not been (possibly) DMA mapped. */ - for (i =3D 0; i < nr_pages; i++) { + for (i =3D 0; i < nr_netmems; i++) { netmem =3D pool->alloc.cache[i]; if (dma_map && unlikely(!page_pool_dma_map(pool, netmem))) { - put_page(netmem_to_page(netmem)); + put_netmem(netmem); continue; } =20 @@ -567,7 +567,7 @@ static noinline netmem_ref __page_pool_alloc_pages_slow= (struct page_pool *pool, pool->pages_state_hold_cnt); } =20 - /* Return last page */ + /* Return the last netmem */ if (likely(pool->alloc.count > 0)) { netmem =3D pool->alloc.cache[--pool->alloc.count]; alloc_stat_inc(pool, slow); @@ -575,7 +575,8 @@ static noinline netmem_ref __page_pool_alloc_pages_slow= (struct page_pool *pool, netmem =3D 0; } =20 - /* When page just alloc'ed is should/must have refcnt 1. */ + /* When a netmem has been just allocated, it should/must have + * refcnt 1. */ return netmem; } =20 --=20 2.17.1 From nobody Wed Dec 17 08:56:40 2025 Received: from invmail4.hynix.com (exvmail4.hynix.com [166.125.252.92]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 46E1028F94C; Fri, 9 May 2025 11:51:38 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=166.125.252.92 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1746791504; cv=none; b=C4Uc/4Q7aGI+DRw7mlhqUEoJ+VLQ8a7ATDLz/PYP73YcewS5rHcDyxdCm3DGwzmXgvkSKq1pZqYG8euOzPDqOmbdBmRp0xnhppm/W1y2jnIiW0JdHuFQsrLLMzMUvmf6N3ZVpPim9hCvODlTs0HORpAEZHb2By6Nw67n3GishvE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1746791504; c=relaxed/simple; bh=zucHbcOJB1t3QTGQzyUIjDjepX5J6BUr5/m7zjQc1B0=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References; b=RxI5MKIDLgyprMntIr4AdvKoqr8kPc17qcghYAy43s6+hmHJroOEm8KmNBPN0x00th3sKsJFRjfx+m3VyAY7AfQhqAaXrKvUsyKIfDcu/3LgyL0/Q1R3Cseur82g6ISZviuLDuxpFO1iBBI8S1gZ+8ISTnq9kUEXC0LJVeVcIn8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com; spf=pass smtp.mailfrom=sk.com; arc=none smtp.client-ip=166.125.252.92 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=sk.com X-AuditID: a67dfc5b-669ff7000002311f-4a-681dec4954fc From: Byungchul Park To: willy@infradead.org, netdev@vger.kernel.org Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, kernel_team@skhynix.com, kuba@kernel.org, almasrymina@google.com, ilias.apalodimas@linaro.org, harry.yoo@oracle.com, hawk@kernel.org, akpm@linux-foundation.org, ast@kernel.org, daniel@iogearbox.net, davem@davemloft.net, john.fastabend@gmail.com, andrew+netdev@lunn.ch, edumazet@google.com, pabeni@redhat.com, vishal.moola@gmail.com Subject: [RFC 06/19] page_pool: rename page_pool_return_page() to page_pool_return_netmem() Date: Fri, 9 May 2025 20:51:13 +0900 Message-Id: <20250509115126.63190-7-byungchul@sk.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20250509115126.63190-1-byungchul@sk.com> References: <20250509115126.63190-1-byungchul@sk.com> X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFnrFLMWRmVeSWpSXmKPExsXC9ZZnka7nG9kMg9bHEhZz1q9hs1j9o8Ji +YMdrBZfft5mt1i88BuzxZzzLSwWT489Yre4v+wZi8We9u3MFr0tv5ktmnasYLK4sK2P1eLy rjlsFvfW/Ge1OLZAzOLb6TeMFuv33WC1+P1jDpuDkMeWlTeZPHbOusvusWBTqcfmFVoeXTcu MXtsWtXJ5rHp0yR2jzvX9rB5nJjxm8Vj547PTB4fn95i8Xi/7yqbx+dNcgG8UVw2Kak5mWWp Rfp2CVwZX7tPsxWsla/4dnILewNjt1QXIyeHhICJxNwpk1hh7Ec/3rKB2GwC6hI3bvxkBrFF BAwlPj86ztLFyMXBLLCQWeLK4p/sIAlhgWiJW33HgRo4OFgEVCV+9KSDhHkFTCV+906Gmikv sXrDAWaQEk4BM4n+j+ogYSGgkmVTFrCBjJQQ+M8mcePMJEaIekmJgytusExg5F3AyLCKUSgz ryw3MTPHRC+jMi+zQi85P3cTIzAGltX+id7B+OlC8CFGAQ5GJR5ei+eyGUKsiWXFlbmHGCU4 mJVEeJ93ymQI8aYkVlalFuXHF5XmpBYfYpTmYFES5zX6Vp4iJJCeWJKanZpakFoEk2Xi4JRq YHQ/sfrZobNPp9RU8uxUeRGfzPyjJ+PHtL/rP51//3DSzrzGmTYWX7bUpwf+f3P20dvjgZx9 83ON1315K/mX0ZnF9M/jSpHLG1YcK2Lb4c60s9eX8eWSK/Mjt06rL/gZ36jUPMXh2Pw1N62+ CLK0qhtOXcfP5xvWWBcQMGHr5zdTpE+w7ajnUnJXYinOSDTUYi4qTgQAQbk+ZX0CAAA= X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFrrILMWRmVeSWpSXmKPExsXC5WfdrOv5RjbDYMMkfos569ewWaz+UWGx /MEOVosvP2+zWyxe+I3ZYs75FhaLp8cesVvcX/aMxWJP+3Zmi96W38wWTTtWMFkcnnuS1eLC tj5Wi8u75rBZ3Fvzn9Xi2AIxi2+n3zBarN93g9Xi9485bA7CHltW3mTy2DnrLrvHgk2lHptX aHl03bjE7LFpVSebx6ZPk9g97lzbw+ZxYsZvFo+dOz4zeXx8eovF4/2+q2wei198YPL4vEku gC+KyyYlNSezLLVI3y6BK+Nr92m2grXyFd9ObmFvYOyW6mLk5JAQMJF49OMtG4jNJqAucePG T2YQW0TAUOLzo+MsXYxcHMwCC5klriz+yQ6SEBaIlrjVdxyogYODRUBV4kdPOkiYV8BU4nfv ZFaImfISqzccYAYp4RQwk+j/qA4SFgIqWTZlAdsERq4FjAyrGEUy88pyEzNzTPWKszMq8zIr 9JLzczcxAgN6We2fiTsYv1x2P8QowMGoxMNr8Vw2Q4g1say4MvcQowQHs5II7/NOmQwh3pTE yqrUovz4otKc1OJDjNIcLErivF7hqQlCAumJJanZqakFqUUwWSYOTqkGxm0SCqZedm7vrh/n e3Iy98R94ZWt0foT/i7enWol2Zpavf1OXQmnl//T4tJJWVUW67zE3hrGKLTENT3YMclt6zTe XQL+bxq3/7d2rnt+rGXV7GOMTt5RvUbrzZT6eJ5NOh2X8zN94qyc21uW1N+p6X4Y+X3vbZZT V1am/kzUOxDaw3TBmp/5rhJLcUaioRZzUXEiAN75Ub5kAgAA X-CFilter-Loop: Reflected Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Now that page_pool_return_page() is for returning netmem, not struct page, rename it to page_pool_return_netmem() to reflect what it does. Signed-off-by: Byungchul Park --- net/core/page_pool.c | 22 +++++++++++----------- 1 file changed, 11 insertions(+), 11 deletions(-) diff --git a/net/core/page_pool.c b/net/core/page_pool.c index 57ad133e6dc8c..bd43026ed7232 100644 --- a/net/core/page_pool.c +++ b/net/core/page_pool.c @@ -374,7 +374,7 @@ struct page_pool *page_pool_create(const struct page_po= ol_params *params) } EXPORT_SYMBOL(page_pool_create); =20 -static void page_pool_return_page(struct page_pool *pool, netmem_ref netme= m); +static void page_pool_return_netmem(struct page_pool *pool, netmem_ref net= mem); =20 static noinline netmem_ref page_pool_refill_alloc_cache(struct page_pool *= pool) { @@ -412,7 +412,7 @@ static noinline netmem_ref page_pool_refill_alloc_cache= (struct page_pool *pool) * (2) break out to fallthrough to alloc_pages_node. * This limit stress on page buddy alloactor. */ - page_pool_return_page(pool, netmem); + page_pool_return_netmem(pool, netmem); alloc_stat_inc(pool, waive); netmem =3D 0; break; @@ -679,7 +679,7 @@ static __always_inline void __page_pool_release_page_dm= a(struct page_pool *pool, * a regular page (that will eventually be returned to the normal * page-allocator via put_page). */ -void page_pool_return_page(struct page_pool *pool, netmem_ref netmem) +static void page_pool_return_netmem(struct page_pool *pool, netmem_ref net= mem) { int count; bool put; @@ -796,7 +796,7 @@ __page_pool_put_page(struct page_pool *pool, netmem_ref= netmem, * will be invoking put_page. */ recycle_stat_inc(pool, released_refcnt); - page_pool_return_page(pool, netmem); + page_pool_return_netmem(pool, netmem); =20 return 0; } @@ -835,7 +835,7 @@ void page_pool_put_unrefed_netmem(struct page_pool *poo= l, netmem_ref netmem, if (netmem && !page_pool_recycle_in_ring(pool, netmem)) { /* Cache full, fallback to free pages */ recycle_stat_inc(pool, ring_full); - page_pool_return_page(pool, netmem); + page_pool_return_netmem(pool, netmem); } } EXPORT_SYMBOL(page_pool_put_unrefed_netmem); @@ -878,7 +878,7 @@ static void page_pool_recycle_ring_bulk(struct page_poo= l *pool, * since put_page() with refcnt =3D=3D 1 can be an expensive operation. */ for (; i < bulk_len; i++) - page_pool_return_page(pool, bulk[i]); + page_pool_return_netmem(pool, bulk[i]); } =20 /** @@ -961,7 +961,7 @@ static netmem_ref page_pool_drain_frag(struct page_pool= *pool, return netmem; } =20 - page_pool_return_page(pool, netmem); + page_pool_return_netmem(pool, netmem); return 0; } =20 @@ -975,7 +975,7 @@ static void page_pool_free_frag(struct page_pool *pool) if (!netmem || page_pool_unref_netmem(netmem, drain_count)) return; =20 - page_pool_return_page(pool, netmem); + page_pool_return_netmem(pool, netmem); } =20 netmem_ref page_pool_alloc_frag_netmem(struct page_pool *pool, @@ -1042,7 +1042,7 @@ static void page_pool_empty_ring(struct page_pool *po= ol) pr_crit("%s() page_pool refcnt %d violation\n", __func__, netmem_ref_count(netmem)); =20 - page_pool_return_page(pool, netmem); + page_pool_return_netmem(pool, netmem); } } =20 @@ -1075,7 +1075,7 @@ static void page_pool_empty_alloc_cache_once(struct p= age_pool *pool) */ while (pool->alloc.count) { netmem =3D pool->alloc.cache[--pool->alloc.count]; - page_pool_return_page(pool, netmem); + page_pool_return_netmem(pool, netmem); } } =20 @@ -1194,7 +1194,7 @@ void page_pool_update_nid(struct page_pool *pool, int= new_nid) /* Flush pool alloc cache, as refill will check NUMA node */ while (pool->alloc.count) { netmem =3D pool->alloc.cache[--pool->alloc.count]; - page_pool_return_page(pool, netmem); + page_pool_return_netmem(pool, netmem); } } EXPORT_SYMBOL(page_pool_update_nid); --=20 2.17.1 From nobody Wed Dec 17 08:56:40 2025 Received: from invmail4.hynix.com (exvmail4.skhynix.com [166.125.252.92]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 782BE28F95D; Fri, 9 May 2025 11:51:38 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=166.125.252.92 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1746791504; cv=none; b=tmRSCh3kgPnA3y26YR7Q7AWoPQEftrLX7k+RkE9SMrVQntUm2jOvcWpnuXolHaXGszoqNn3HEMl7GeTBETXQPJFVJjX00z261arp9XbI1n7dChZKfd4yfjy5Pj8XoHAX+4Hqbp5/ql6oB5O+uXmz1CI/DaBcC1zAn9ppNVBOzLc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1746791504; c=relaxed/simple; bh=uNZXALa5rN0NixaHl8NdUd9UmqartCfTHgM9I0a+/wA=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References; b=mRYxIvrLiet5sNaWEG+RSqqAnlo5xiEVSBtNsUCt3BDfznuWGh0pRcSPO3D3WO0oY4RfsqBY9kZSuvIwhZRsXBIiFPt4wY3YlcAGrQniaDtFyPotaiFrO8DdcvQuOoIdm1D13IuMQ7qPd5Wo6SYmSxnvE0HXsqKZglyEb5ftcnA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com; spf=pass smtp.mailfrom=sk.com; arc=none smtp.client-ip=166.125.252.92 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=sk.com X-AuditID: a67dfc5b-669ff7000002311f-50-681dec491105 From: Byungchul Park To: willy@infradead.org, netdev@vger.kernel.org Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, kernel_team@skhynix.com, kuba@kernel.org, almasrymina@google.com, ilias.apalodimas@linaro.org, harry.yoo@oracle.com, hawk@kernel.org, akpm@linux-foundation.org, ast@kernel.org, daniel@iogearbox.net, davem@davemloft.net, john.fastabend@gmail.com, andrew+netdev@lunn.ch, edumazet@google.com, pabeni@redhat.com, vishal.moola@gmail.com Subject: [RFC 07/19] page_pool: use netmem alloc/put API in page_pool_return_netmem() Date: Fri, 9 May 2025 20:51:14 +0900 Message-Id: <20250509115126.63190-8-byungchul@sk.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20250509115126.63190-1-byungchul@sk.com> References: <20250509115126.63190-1-byungchul@sk.com> X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFnrBLMWRmVeSWpSXmKPExsXC9ZZnka7nG9kMgw1fpSzmrF/DZrH6R4XF 8gc7WC2+/LzNbrF44TdmiznnW1gsnh57xG5xf9kzFos97duZLXpbfjNbNO1YwWRxYVsfq8Xl XXPYLO6t+c9qcWyBmMW3028YLdbvu8Fq8fvHHDYHIY8tK28yeeycdZfdY8GmUo/NK7Q8um5c YvbYtKqTzWPTp0nsHneu7WHzODHjN4vHzh2fmTw+Pr3F4vF+31U2j8+b5AJ4o7hsUlJzMstS i/TtErgyrjzcxlLwgqdi2fQbbA2MV7m6GDk4JARMJFaezYQx7/yK72Lk5GATUJe4ceMnM4gt ImAo8fnRcZYuRi4OZoGFzBJXFv9kB6kXFgiV+PXSC6SGRUBVYtHNu+wgNq+AqcTHVadZQWwJ AXmJ1RsOMIOUcwqYSfR/VAcJCwGVLJuygA1kpIRAO7vEvT39LBD1khIHV9xgmcDIu4CRYRWj UGZeWW5iZo6JXkZlXmaFXnJ+7iZGYPgvq/0TvYPx04XgQ4wCHIxKPLwWz2UzhFgTy4orcw8x SnAwK4nwPu+UyRDiTUmsrEotyo8vKs1JLT7EKM3BoiTOa/StPEVIID2xJDU7NbUgtQgmy8TB KdXAmPj0YvUv7YZA8addS5sLRD9HlL9es/vR5jaV8Ps6c8w4zr/Wt38p9VzZrT252U2qf8av Fynasva7/8edm3z7JsfLqrTb0Ws0hXYH7OTUf3v9zcJUtRkZ7yw3GS6py7oiMD0s6L9spVZL 4jnlw+cO627I/LZgP8Maph1hp6V0VzW2fV7CN/fKCyWW4oxEQy3mouJEAJBbyGR7AgAA X-Brightmail-Tracker: H4sIAAAAAAAAAwFjApz9CAMS0AMaCGludGVybmFsIgYKBApOO4MtSewdaDDYoBE4nK+sBjir +Hg4p+C4BTj0+dsHOKOh9gM4nM+EBDjlxuIHON+m5gQ4vIe3AziNhPsDOIK4qAI4w53JBTjQ to4FONO6nAY43qz/BTjGoBY49svsATivvtgFOPv4nAZAE0i0qdkCSLma3QdIoLJ1SLOoKkiK 2NIDSLKqiQZIsvKSB0jc1rwGSMiY+wRIubjzAkjx5doESO++1QZIo+jwAkjzsh5QDloKPGRl bGl2ZXIvPmAKaNThtgRw6Ax4ppfYBoAB1QqKAQgIGBA0GNz6X4oBCQgGECcY2Nj5A4oBCQgU EDEY8+LHBIoBCggDEKEDGNSj+QeKAQgIExBVGPrpSooBCQgEECUYotndB4oBCQgNEDUY8arL BYoBCQgYEB8Yq7DAA4oBCAgJEDYYj/EnigEJCBIQNRimlKAGkAEKoAEAqgEUaW52bWFpbDUu c2toeW5peC5jb22yAQYKBKZ9/JG4AfTTR8IBEAgBIgwNOOcdaBIFYXZzeW3CARgIAyIUDeeJ HGgSDWRheXplcm9fcnVsZXPCARsIBCIXDUpXZWASEGdhdGVrZWVwZXJfcnVsZXPCAQIICRqA ATYXuV6NccGolACrf1G/cIxtw+DqsSK/D+0AbNepx4E7LVCq5jD8yReqzgN630n3EfFrWL0U f7gD0gsl7ciN0iSf18wiWJighMvo6AllftCb07jLH65F+hMr+8mQU8mc78ioy7Zw1H9fxDg6 Ok0/+9jparGWMoA3O5kon5FSOZvBIgRzaGExKgNyc2HPK+YAYwIAAA== X-CFilter-Loop: Reflected Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Use netmem alloc/put API, put_netmem(), instead of put_page() in page_pool_return_netmem(). While at it, delete #include since the last put_page() in page_pool.c has been just removed with this patch. Signed-off-by: Byungchul Park --- net/core/page_pool.c | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/net/core/page_pool.c b/net/core/page_pool.c index bd43026ed7232..311d0ef620ea1 100644 --- a/net/core/page_pool.c +++ b/net/core/page_pool.c @@ -20,7 +20,6 @@ #include #include #include -#include /* for put_page() */ #include #include #include @@ -677,7 +676,7 @@ static __always_inline void __page_pool_release_page_dm= a(struct page_pool *pool, /* Disconnects a page (from a page_pool). API users can have a need * to disconnect a page (from a page_pool), to allow it to be used as * a regular page (that will eventually be returned to the normal - * page-allocator via put_page). + * page-allocator via put_netmem() and then put_page()). */ static void page_pool_return_netmem(struct page_pool *pool, netmem_ref net= mem) { @@ -698,7 +697,7 @@ static void page_pool_return_netmem(struct page_pool *p= ool, netmem_ref netmem) =20 if (put) { page_pool_clear_pp_info(netmem); - put_page(netmem_to_page(netmem)); + put_netmem(netmem); } /* An optimization would be to call __free_pages(page, pool->p.order) * knowing page is not part of page-cache (thus avoiding a --=20 2.17.1 From nobody Wed Dec 17 08:56:40 2025 Received: from invmail4.hynix.com (exvmail4.skhynix.com [166.125.252.92]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 837A728FFEC; Fri, 9 May 2025 11:51:44 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=166.125.252.92 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1746791507; cv=none; b=V59zYtRRjKywWXXKV9zf14DTz5qUxu4z4MbDmm+vaCTSNw1ZliMxzfqb4GNhpInb9R1z85YXEcDWeC4tUQT6gro2mPWqfbQctW78ifMgdBYrbE+X5H3x4cI0cE3jvWDPNtub1gJYEG/wSpfg5AsN7zjqy4uSFb0rjDt7JgN6+aU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1746791507; c=relaxed/simple; bh=DA9jKs0Qda8IqZ65NNV2UZpPRNKW+Kk9K4H926UA2rU=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References; b=Whk+25ir61+//RueZK9a8cCaR9StcvZ7DccvICnzmXl5FCBVM+uR3cY5lflbGJSodBRz5hDoPn8JOFckGBMKdzY8J5lq8wArZEi7fQ5S0UzLcneSdv/ekCkDqhGFqKbQCc+HrnlNDavaxbjihqvlpCwlRTUy8/YwwNV3h8lTO94= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com; spf=pass smtp.mailfrom=sk.com; arc=none smtp.client-ip=166.125.252.92 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=sk.com X-AuditID: a67dfc5b-669ff7000002311f-56-681dec4911e8 From: Byungchul Park To: willy@infradead.org, netdev@vger.kernel.org Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, kernel_team@skhynix.com, kuba@kernel.org, almasrymina@google.com, ilias.apalodimas@linaro.org, harry.yoo@oracle.com, hawk@kernel.org, akpm@linux-foundation.org, ast@kernel.org, daniel@iogearbox.net, davem@davemloft.net, john.fastabend@gmail.com, andrew+netdev@lunn.ch, edumazet@google.com, pabeni@redhat.com, vishal.moola@gmail.com Subject: [RFC 08/19] page_pool: rename __page_pool_release_page_dma() to __page_pool_release_netmem_dma() Date: Fri, 9 May 2025 20:51:15 +0900 Message-Id: <20250509115126.63190-9-byungchul@sk.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20250509115126.63190-1-byungchul@sk.com> References: <20250509115126.63190-1-byungchul@sk.com> X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFnrNLMWRmVeSWpSXmKPExsXC9ZZnoa7nG9kMg3vN8hZz1q9hs1j9o8Ji +YMdrBZfft5mt1i88BuzxZzzLSwWT489Yre4v+wZi8We9u3MFr0tv5ktmnasYLK4sK2P1eLy rjlsFvfW/Ge1OLZAzOLb6TeMFuv33WC1+P1jDpuDkMeWlTeZPHbOusvusWBTqcfmFVoeXTcu MXtsWtXJ5rHp0yR2jzvX9rB5nJjxm8Vj547PTB4fn95i8Xi/7yqbx+dNcgG8UVw2Kak5mWWp Rfp2CVwZay4vZSt4yFmxZ8MZtgbGeRxdjJwcEgImEp9PTWSEsZd8uc0GYrMJqEvcuPGTGcQW ETCU+PzoOEsXIxcHs8BCZokri3+ygySEBTIlvty/BdbMIqAq8ezgNlYQm1fAVOL0k6WsEEPl JVZvOAA0iIODU8BMov+jOkhYCKhk2ZQFbCAzJQSa2SXOLVjBBlEvKXFwxQ2WCYy8CxgZVjEK ZeaV5SZm5pjoZVTmZVboJefnbmIERsGy2j/ROxg/XQg+xCjAwajEw2vxXDZDiDWxrLgy9xCj BAezkgjv806ZDCHelMTKqtSi/Pii0pzU4kOM0hwsSuK8Rt/KU4QE0hNLUrNTUwtSi2CyTByc Ug2M4dG8Pi/iLucUyVf4HvtU7/pmD+vWc1W6Fpw2R44IP82Peic8eetBRh2bvhtCdtWvj7B/ Mv8p9vTrlntT5u0P47zLZj/NQfGA/KKtaaqxQkIObxVeJK2781FC5ZBfu7ejn1v0f5dH+cYe wYVphnm3tTirJ86+6e3Om3KIT/EX7/LdIWw760qUWIozEg21mIuKEwG94xq1fgIAAA== X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFrrELMWRmVeSWpSXmKPExsXC5WfdrOv5RjbD4PYPUYs569ewWaz+UWGx /MEOVosvP2+zWyxe+I3ZYs75FhaLp8cesVvcX/aMxWJP+3Zmi96W38wWTTtWMFkcnnuS1eLC tj5Wi8u75rBZ3Fvzn9Xi2AIxi2+n3zBarN93g9Xi9485bA7CHltW3mTy2DnrLrvHgk2lHptX aHl03bjE7LFpVSebx6ZPk9g97lzbw+ZxYsZvFo+dOz4zeXx8eovF4/2+q2wei198YPL4vEku gC+KyyYlNSezLLVI3y6BK2PN5aVsBQ85K/ZsOMPWwDiPo4uRk0NCwERiyZfbbCA2m4C6xI0b P5lBbBEBQ4nPj46zdDFycTALLGSWuLL4JztIQlggU+LL/VuMIDaLgKrEs4PbWEFsXgFTidNP lrJCDJWXWL3hANAgDg5OATOJ/o/qIGEhoJJlUxawTWDkWsDIsIpRJDOvLDcxM8dUrzg7ozIv s0IvOT93EyMwpJfV/pm4g/HLZfdDjAIcjEo8vBbPZTOEWBPLiitzDzFKcDArifA+75TJEOJN SaysSi3Kjy8qzUktPsQozcGiJM7rFZ6aICSQnliSmp2aWpBaBJNl4uCUamDkE1FWNdsfGegT fJhfLODmCjnJp9miH4NnVr02056ztzx+lQS/SFfP/IcuNw78WLw2cJEm56X6/T3T1jSLz33h K6W8xHdRrdDluoeXJTaxsrTzlzEoxvY9vJrEMz/g5+ctt41Y7nq/FZw0K2yTZ/WVyIZ/Wazv lh8OvPR2qdnhZoO4WM1V2/4psRRnJBpqMRcVJwIAp1mvwmUCAAA= X-CFilter-Loop: Reflected Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Now that __page_pool_release_page_dma() is for releasing netmem, not struct page, rename it to __page_pool_release_netmem_dma() to reflect what it does. Signed-off-by: Byungchul Park --- net/core/page_pool.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/net/core/page_pool.c b/net/core/page_pool.c index 311d0ef620ea1..47164d561d1aa 100644 --- a/net/core/page_pool.c +++ b/net/core/page_pool.c @@ -653,7 +653,7 @@ void page_pool_clear_pp_info(netmem_ref netmem) netmem_set_pp(netmem, NULL); } =20 -static __always_inline void __page_pool_release_page_dma(struct page_pool = *pool, +static __always_inline void __page_pool_release_netmem_dma(struct page_poo= l *pool, netmem_ref netmem) { dma_addr_t dma; @@ -687,7 +687,7 @@ static void page_pool_return_netmem(struct page_pool *p= ool, netmem_ref netmem) if (static_branch_unlikely(&page_pool_mem_providers) && pool->mp_ops) put =3D pool->mp_ops->release_netmem(pool, netmem); else - __page_pool_release_page_dma(pool, netmem); + __page_pool_release_netmem_dma(pool, netmem); =20 /* This may be the last page returned, releasing the pool, so * it is not safe to reference pool afterwards. --=20 2.17.1 From nobody Wed Dec 17 08:56:40 2025 Received: from invmail4.hynix.com (exvmail4.skhynix.com [166.125.252.92]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 8370828FAA8; Fri, 9 May 2025 11:51:44 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=166.125.252.92 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1746791507; cv=none; b=Hsj+jpDwJnCEwSd+vk7cdtQ4xo4d25xS/N6x7ntmDUP8KpaXD/eHnyKVRVV3wdwckV+hriuOgHX8BujeMPWoQoWDooZWu02N/9TwHrisklG5VBcR3V4F0G3npb4h7+K0bCTcv/YS+mZjwAY2AMqFnLJ0wIGZX3ixkFc5wXUwJMY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1746791507; c=relaxed/simple; bh=cVoQg6VLoUphUFismE0rH5IYGlskFI/Je6TNOOVwkqA=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References; b=CmQRVjLtKisQDzlRrC3rXP8aCpmbNDnf3qIyXBnek7R/Eoq0RyU7hQUiZNqo+QFWvzQcKbqyjUZutoWExr1jlp9PSEB5cCoxvb/GJk1JSZTXGOqsEfKykA1RVAIyJVxa8GbRmoutTrqjokjHPWeZXjGXioxfiLsbEtnoZzEfVRw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com; spf=pass smtp.mailfrom=sk.com; arc=none smtp.client-ip=166.125.252.92 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=sk.com X-AuditID: a67dfc5b-669ff7000002311f-5c-681dec495ced From: Byungchul Park To: willy@infradead.org, netdev@vger.kernel.org Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, kernel_team@skhynix.com, kuba@kernel.org, almasrymina@google.com, ilias.apalodimas@linaro.org, harry.yoo@oracle.com, hawk@kernel.org, akpm@linux-foundation.org, ast@kernel.org, daniel@iogearbox.net, davem@davemloft.net, john.fastabend@gmail.com, andrew+netdev@lunn.ch, edumazet@google.com, pabeni@redhat.com, vishal.moola@gmail.com Subject: [RFC 09/19] page_pool: rename __page_pool_put_page() to __page_pool_put_netmem() Date: Fri, 9 May 2025 20:51:16 +0900 Message-Id: <20250509115126.63190-10-byungchul@sk.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20250509115126.63190-1-byungchul@sk.com> References: <20250509115126.63190-1-byungchul@sk.com> X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFnrFLMWRmVeSWpSXmKPExsXC9ZZnka7nG9kMgxNbVS3mrF/DZrH6R4XF 8gc7WC2+/LzNbrF44TdmiznnW1gsnh57xG5xf9kzFos97duZLXpbfjNbNO1YwWRxYVsfq8Xl XXPYLO6t+c9qcWyBmMW3028YLdbvu8Fq8fvHHDYHIY8tK28yeeycdZfdY8GmUo/NK7Q8um5c YvbYtKqTzWPTp0nsHneu7WHzODHjN4vHzh2fmTw+Pr3F4vF+31U2j8+b5AJ4o7hsUlJzMstS i/TtErgyHi1ewFjQJVAxd8d61gbGGbxdjJwcEgImEn+eLGLtYuQAs/unK4GE2QTUJW7c+MkM YosIGEp8fnScpYuRi4NZYCGzxJXFP9lBEsICkRK73p0Hs1kEVCXmLbrCCmLzCphJvNkymwVi vrzE6g0HmEHmcwLF+z+qg4SFBEwllk1ZwAYyU0KgmV3i5JQFjBD1khIHV9xgmcDIu4CRYRWj UGZeWW5iZo6JXkZlXmaFXnJ+7iZGYAwsq/0TvYPx04XgQ4wCHIxKPLwWz2UzhFgTy4orcw8x SnAwK4nwPu+UyRDiTUmsrEotyo8vKs1JLT7EKM3BoiTOa/StPEVIID2xJDU7NbUgtQgmy8TB KdXAWLtd3F1OJtP9pxjvD+0PZu3b+868TJslsDLHusr8n3MO08K2i3LqIVeCqqfoe6yfonOg K/CtyY0/7TI8bT/84j5/l3I5lNGWaen8XlDl2/euNB6lrNJXvy05TVtvPL/yv/em+d2aE0zL Pna7Tt/TfMdI4OmJgyUm8pfm/hB9qXucKyDR+dEJJZbijERDLeai4kQAy4ET9n0CAAA= X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFrrILMWRmVeSWpSXmKPExsXC5WfdrOv5RjbD4MQ/cYs569ewWaz+UWGx /MEOVosvP2+zWyxe+I3ZYs75FhaLp8cesVvcX/aMxWJP+3Zmi96W38wWTTtWMFkcnnuS1eLC tj5Wi8u75rBZ3Fvzn9Xi2AIxi2+n3zBarN93g9Xi9485bA7CHltW3mTy2DnrLrvHgk2lHptX aHl03bjE7LFpVSebx6ZPk9g97lzbw+ZxYsZvFo+dOz4zeXx8eovF4/2+q2wei198YPL4vEku gC+KyyYlNSezLLVI3y6BK+PR4gWMBV0CFXN3rGdtYJzB28XIwSEhYCLRP12pi5GTg01AXeLG jZ/MILaIgKHE50fHWboYuTiYBRYyS1xZ/JMdJCEsECmx6915MJtFQFVi3qIrrCA2r4CZxJst s1lAbAkBeYnVGw4wg8znBIr3f1QHCQsJmEosm7KAbQIj1wJGhlWMIpl5ZbmJmTmmesXZGZV5 mRV6yfm5mxiBAb2s9s/EHYxfLrsfYhTgYFTi4bV4LpshxJpYVlyZe4hRgoNZSYT3eadMhhBv SmJlVWpRfnxRaU5q8SFGaQ4WJXFer/DUBCGB9MSS1OzU1ILUIpgsEwenVAOj6vr0/LK0BRe/ iOUuqO84M11OZ529OM8E/R+/Vh1jZm2+EyAUI9HD+eG/w7kv67Qthd2LbBl+bnTpyztS2Mt4 NbhKr7g4M5XtfVFMY7L7O6nfO6Unx21Ycvh99MOkudHmqZwP166bfEVEUj1xNkPVm4idvasX KOQkxnSG+7kfr9+s87nwSZ4SS3FGoqEWc1FxIgCHxaVyZAIAAA== X-CFilter-Loop: Reflected Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Now that __page_pool_put_page() puts netmem, not struct page, rename it to __page_pool_put_netmem() to reflect what it does. Signed-off-by: Byungchul Park --- net/core/page_pool.c | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/net/core/page_pool.c b/net/core/page_pool.c index 47164d561d1aa..f858a5518b7a4 100644 --- a/net/core/page_pool.c +++ b/net/core/page_pool.c @@ -755,7 +755,7 @@ static bool __page_pool_page_can_be_recycled(netmem_ref= netmem) * subsystem. */ static __always_inline netmem_ref -__page_pool_put_page(struct page_pool *pool, netmem_ref netmem, +__page_pool_put_netmem(struct page_pool *pool, netmem_ref netmem, unsigned int dma_sync_size, bool allow_direct) { lockdep_assert_no_hardirq(); @@ -811,7 +811,7 @@ static bool page_pool_napi_local(const struct page_pool= *pool) /* Allow direct recycle if we have reasons to believe that we are * in the same context as the consumer would run, so there's * no possible race. - * __page_pool_put_page() makes sure we're not in hardirq context + * __page_pool_put_netmem() makes sure we're not in hardirq context * and interrupts are enabled prior to accessing the cache. */ cpuid =3D smp_processor_id(); @@ -830,7 +830,7 @@ void page_pool_put_unrefed_netmem(struct page_pool *poo= l, netmem_ref netmem, allow_direct =3D page_pool_napi_local(pool); =20 netmem =3D - __page_pool_put_page(pool, netmem, dma_sync_size, allow_direct); + __page_pool_put_netmem(pool, netmem, dma_sync_size, allow_direct); if (netmem && !page_pool_recycle_in_ring(pool, netmem)) { /* Cache full, fallback to free pages */ recycle_stat_inc(pool, ring_full); @@ -931,7 +931,7 @@ void page_pool_put_netmem_bulk(netmem_ref *data, u32 co= unt) continue; } =20 - netmem =3D __page_pool_put_page(pool, netmem, -1, + netmem =3D __page_pool_put_netmem(pool, netmem, -1, allow_direct); /* Approved for bulk recycling in ptr_ring cache */ if (netmem) --=20 2.17.1 From nobody Wed Dec 17 08:56:40 2025 Received: from invmail4.hynix.com (exvmail4.hynix.com [166.125.252.92]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 838F528FFED; Fri, 9 May 2025 11:51:44 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=166.125.252.92 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1746791507; cv=none; b=STBnbqrlqU/YJGGVK0jCQnT574fK833OQyqkBs3taipzS0ifgANPhQEgpq/lleTG0iwG/i2XkMv/zqz1yLtZeuQkEe6KhDhb7aaDqqS1GUabYHZtTVos2NfWDL1GQVQFjzRnckqB6JHg6w+WMaOJM3dd09lgu7aH6iJze7sX2Vg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1746791507; c=relaxed/simple; bh=Bad1PLduXW27p+j9Xa6JrsTsCSPwcFD0YvcDAM5wysQ=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References; b=FJ1KTpP+W653rYHdM/Lh9pl0/bcxen+lHCAR2QFUmxyaWrCfRblRVtDPANPHTX/4Yj9BT/dgkOnBUz543Ouruh7vbCOhBJRyCenVf01vcXvwJGQyBeTWDtFZx2PxhW/fAemHbbj1Gx96P8CFW8SHH9Wf807z0X3H3vChF/UNpNA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com; spf=pass smtp.mailfrom=sk.com; arc=none smtp.client-ip=166.125.252.92 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=sk.com X-AuditID: a67dfc5b-669ff7000002311f-62-681dec49e1aa From: Byungchul Park To: willy@infradead.org, netdev@vger.kernel.org Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, kernel_team@skhynix.com, kuba@kernel.org, almasrymina@google.com, ilias.apalodimas@linaro.org, harry.yoo@oracle.com, hawk@kernel.org, akpm@linux-foundation.org, ast@kernel.org, daniel@iogearbox.net, davem@davemloft.net, john.fastabend@gmail.com, andrew+netdev@lunn.ch, edumazet@google.com, pabeni@redhat.com, vishal.moola@gmail.com Subject: [RFC 10/19] page_pool: rename __page_pool_alloc_pages_slow() to __page_pool_alloc_netmems_slow() Date: Fri, 9 May 2025 20:51:17 +0900 Message-Id: <20250509115126.63190-11-byungchul@sk.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20250509115126.63190-1-byungchul@sk.com> References: <20250509115126.63190-1-byungchul@sk.com> X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFnrBLMWRmVeSWpSXmKPExsXC9ZZnoa7nG9kMg2/NahZz1q9hs1j9o8Ji +YMdrBZfft5mt1i88BuzxZzzLSwWT489Yre4v+wZi8We9u3MFr0tv5ktmnasYLK4sK2P1eLy rjlsFvfW/Ge1OLZAzOLb6TeMFuv33WC1+P1jDpuDkMeWlTeZPHbOusvusWBTqcfmFVoeXTcu MXtsWtXJ5rHp0yR2jzvX9rB5nJjxm8Vj547PTB4fn95i8Xi/7yqbx+dNcgG8UVw2Kak5mWWp Rfp2CVwZvW1zGAuOclbMXp/RwNjH0cXIySEhYCJx6/c7dhi77+5iZhCbTUBd4saNn2C2iICh xOdHx1m6GLk4mAUWMktcWfwTrEFYIFPi6sTnTF2MHBwsAqoSt9aAzeQVMJNYfPIyE8RMeYnV Gw4wg5RwAsX7P6qDhIUETCWWTVnABlHyn03iySkvCFtS4uCKGywTGHkXMDKsYhTKzCvLTczM MdHLqMzLrNBLzs/dxAgM/2W1f6J3MH66EHyIUYCDUYmH1+K5bIYQa2JZcWXuIUYJDmYlEd7n nTIZQrwpiZVVqUX58UWlOanFhxilOViUxHmNvpWnCAmkJ5akZqemFqQWwWSZODilGhi1FQ+L rLObE71FfILzw3O/zL+nzawseGfKe1u79Geql5Bt/a52ycymMs2rZwRvZNnLfJavr7KPqFJz 7Lt4vG6uRsf9huMn8kXvNdy8mh51Wt6k/6blzb6IPT2ves/48myefrDo7gMTH9PY4xuX7OJa MuNi7c3GBXJ2Jhw33ma9mpjblWirmK3EUpyRaKjFXFScCACh66yiewIAAA== X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFrrALMWRmVeSWpSXmKPExsXC5WfdrOv5RjbDoOmajMWc9WvYLFb/qLBY /mAHq8WXn7fZLRYv/MZsMed8C4vF02OP2C3uL3vGYrGnfTuzRW/Lb2aLph0rmCwOzz3JanFh Wx+rxeVdc9gs7q35z2pxbIGYxbfTbxgt1u+7wWrx+8ccNgdhjy0rbzJ57Jx1l91jwaZSj80r tDy6blxi9ti0qpPNY9OnSewed67tYfM4MeM3i8fOHZ+ZPD4+vcXi8X7fVTaPxS8+MHl83iQX wBfFZZOSmpNZllqkb5fAldHbNoex4Chnxez1GQ2MfRxdjJwcEgImEn13FzOD2GwC6hI3bvwE s0UEDCU+PzrO0sXIxcEssJBZ4srin+wgCWGBTImrE58zdTFycLAIqErcWgM2h1fATGLxyctM EDPlJVZvOMAMUsIJFO//qA4SFhIwlVg2ZQHbBEauBYwMqxhFMvPKchMzc0z1irMzKvMyK/SS 83M3MQLDeVntn4k7GL9cdj/EKMDBqMTDa/FcNkOINbGsuDL3EKMEB7OSCO/zTpkMId6UxMqq 1KL8+KLSnNTiQ4zSHCxK4rxe4akJQgLpiSWp2ampBalFMFkmDk6pBkbvt0efW76vldkv2vjm QEjCkUtXFus9ZU51ZNg8i7fP8PfF58p1V3a5P1R4YBF5+H61Cc/v1c7SF2eGXzkxwzBDrezq pIX7fk/atf8f38cVW+oYlFo2TX9t4huwgTuJT7iGXe5f0KSGPVVTbYMNvk6zc2Rj0eG2n7Ni h7PDjPBH51NXZynqtvxVYinOSDTUYi4qTgQAAsLWhmMCAAA= X-CFilter-Loop: Reflected Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Now that __page_pool_alloc_pages_slow() is for allocating netmem, not struct page, rename it to __page_pool_alloc_netmems_slow() to reflect what it does. Signed-off-by: Byungchul Park --- net/core/page_pool.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/net/core/page_pool.c b/net/core/page_pool.c index f858a5518b7a4..b61c1038f4c68 100644 --- a/net/core/page_pool.c +++ b/net/core/page_pool.c @@ -523,7 +523,7 @@ static netmem_ref __page_pool_alloc_large_netmem(struct= page_pool *pool, } =20 /* slow path */ -static noinline netmem_ref __page_pool_alloc_pages_slow(struct page_pool *= pool, +static noinline netmem_ref __page_pool_alloc_netmems_slow(struct page_pool= *pool, gfp_t gfp) { const int bulk =3D PP_ALLOC_CACHE_REFILL; @@ -595,7 +595,7 @@ netmem_ref page_pool_alloc_netmems(struct page_pool *po= ol, gfp_t gfp) if (static_branch_unlikely(&page_pool_mem_providers) && pool->mp_ops) netmem =3D pool->mp_ops->alloc_netmems(pool, gfp); else - netmem =3D __page_pool_alloc_pages_slow(pool, gfp); + netmem =3D __page_pool_alloc_netmems_slow(pool, gfp); return netmem; } EXPORT_SYMBOL(page_pool_alloc_netmems); --=20 2.17.1 From nobody Wed Dec 17 08:56:40 2025 Received: from invmail4.hynix.com (exvmail4.skhynix.com [166.125.252.92]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 83A9928FFEE; Fri, 9 May 2025 11:51:44 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=166.125.252.92 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1746791507; cv=none; b=sxczGiWC7YawfeVVBvnmkMZkpoDuLM8RBYdODWhMYj1qTS3hTxQsw39QLdWqcrSf2ktNKB/yFPdEXBVAPP2cuTqaTZjHuqNsJ8YzRXvV1S84CzM9B23NwnDhnwNWwgZJkHMdDO6tEGVqnuVrWQ3xos9UfJ2Vnq/5Po2v38xkRoo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1746791507; c=relaxed/simple; bh=40hhzcReE9lzoseTEMmVCYl1hIewehLeKTS2dWTVKHc=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References; b=t3NJFQlnFyP1x5FBA4pH897HSk3DkemV416aXVJY//s7nv0JLMhXwD5XOdeggdk5yVmCpDTIZL2tz9JaAL43I2pM/BtVTNNJBNjo4eJntAoWHYlJ9eeFeMmdSCoQksls4EwkhgAe31Fz3eP/qo/aQWMlCaAEt6l0vIeSklgUv/A= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com; spf=pass smtp.mailfrom=sk.com; arc=none smtp.client-ip=166.125.252.92 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=sk.com X-AuditID: a67dfc5b-669ff7000002311f-68-681dec49076f From: Byungchul Park To: willy@infradead.org, netdev@vger.kernel.org Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, kernel_team@skhynix.com, kuba@kernel.org, almasrymina@google.com, ilias.apalodimas@linaro.org, harry.yoo@oracle.com, hawk@kernel.org, akpm@linux-foundation.org, ast@kernel.org, daniel@iogearbox.net, davem@davemloft.net, john.fastabend@gmail.com, andrew+netdev@lunn.ch, edumazet@google.com, pabeni@redhat.com, vishal.moola@gmail.com Subject: [RFC 11/19] mlx4: use netmem descriptor and API for page pool Date: Fri, 9 May 2025 20:51:18 +0900 Message-Id: <20250509115126.63190-12-byungchul@sk.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20250509115126.63190-1-byungchul@sk.com> References: <20250509115126.63190-1-byungchul@sk.com> X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFnrNLMWRmVeSWpSXmKPExsXC9ZZnka7nG9kMgy2rtS3mrF/DZrH6R4XF 8gc7WC2+/LzNbrF44TdmiznnW1gsnh57xG5xf9kzFos97duZLXpbfjNbNO1YwWRxYVsfq8Xl XXPYLO6t+c9qcWyBmMW3028YLdbvu8Fq8fvHHDYHIY8tK28yeeycdZfdY8GmUo/NK7Q8um5c YvbYtKqTzWPTp0nsHneu7WHzODHjN4vHzh2fmTw+Pr3F4vF+31U2j8+b5AJ4o7hsUlJzMstS i/TtErgyjq9rZS6YaFXxbO4/9gbGRv0uRk4OCQETiaalu1hg7Gvd/cwgNpuAusSNGz/BbBEB Q4nPj44D1XBxMAssZJa4svgnO0hCWMBN4tPvBrAiFgFVicWz1jOB2LwCZhKb9j1lhRgqL7F6 wwGgGg4OTqB4/0d1kLCQgKnEsikL2EBmSgj8Z5PomjYXql5S4uCKGywTGHkXMDKsYhTKzCvL TczMMdHLqMzLrNBLzs/dxAiMgmW1f6J3MH66EHyIUYCDUYmH1+K5bIYQa2JZcWXuIUYJDmYl Ed7nnTIZQrwpiZVVqUX58UWlOanFhxilOViUxHmNvpWnCAmkJ5akZqemFqQWwWSZODilGhjF dzw9wHBwWbvYtv6Q2QJn0hbzHI18q2Ti1X/4fs+3SU0mtXs1/jw9dPRCyrpfV6IzW+59Mgnu X87vYFGr4nw+MfD5laSfWwWqOc/9rBLhntHczxwZtSnLdaZinctp6y8vf53l4dw8Zb9OTOQ8 82k2UgIh0bLXZh85bJAp/3zXN4cJQn3OZjuUWIozEg21mIuKEwH3hxtkfgIAAA== X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFrrELMWRmVeSWpSXmKPExsXC5WfdrOv5RjbD4GqPvMWc9WvYLFb/qLBY /mAHq8WXn7fZLRYv/MZsMed8C4vF02OP2C3uL3vGYrGnfTuzRW/Lb2aLph0rmCwOzz3JanFh Wx+rxeVdc9gs7q35z2pxbIGYxbfTbxgt1u+7wWrx+8ccNgdhjy0rbzJ57Jx1l91jwaZSj80r tDy6blxi9ti0qpPNY9OnSewed67tYfM4MeM3i8fOHZ+ZPD4+vcXi8X7fVTaPxS8+MHl83iQX wBfFZZOSmpNZllqkb5fAlXF8XStzwUSrimdz/7E3MDbqdzFyckgImEhc6+5nBrHZBNQlbtz4 CWaLCBhKfH50nKWLkYuDWWAhs8SVxT/ZQRLCAm4Sn343gBWxCKhKLJ61ngnE5hUwk9i07ykr xFB5idUbDgDVcHBwAsX7P6qDhIUETCWWTVnANoGRawEjwypGkcy8stzEzBxTveLsjMq8zAq9 5PzcTYzAkF5W+2fiDsYvl90PMQpwMCrx8Fo8l80QYk0sK67MPcQowcGsJML7vFMmQ4g3JbGy KrUoP76oNCe1+BCjNAeLkjivV3hqgpBAemJJanZqakFqEUyWiYNTqoFRKferf2BsU9GK1JP+ N5/qT54W8bPY/HnzQ2um3g0sfvJf52zs1XjVrKmh9W7d4ycZvz2Vyz0Mo0/rdv1SvjtZdPJt q0rGJrVtpbdWrpGeJCMuyv26sdnS5fy/HBfb5Ue4+JI/R7I5hWX6R7Y5rLPj/e7za+Htjup9 PQfT/ukGfT+9d46ukKcSS3FGoqEWc1FxIgCbjJVbZQIAAA== X-CFilter-Loop: Reflected Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" To simplify struct page, the effort to seperate its own descriptor from struct page is required and the work for page pool is on going. Use netmem descriptor and API for page pool in mlx4 code. Signed-off-by: Byungchul Park --- drivers/net/ethernet/mellanox/mlx4/en_rx.c | 46 +++++++++++--------- drivers/net/ethernet/mellanox/mlx4/en_tx.c | 8 ++-- drivers/net/ethernet/mellanox/mlx4/mlx4_en.h | 4 +- 3 files changed, 31 insertions(+), 27 deletions(-) diff --git a/drivers/net/ethernet/mellanox/mlx4/en_rx.c b/drivers/net/ether= net/mellanox/mlx4/en_rx.c index b33285d755b90..82c24931fa443 100644 --- a/drivers/net/ethernet/mellanox/mlx4/en_rx.c +++ b/drivers/net/ethernet/mellanox/mlx4/en_rx.c @@ -62,18 +62,18 @@ static int mlx4_en_alloc_frags(struct mlx4_en_priv *pri= v, int i; =20 for (i =3D 0; i < priv->num_frags; i++, frags++) { - if (!frags->page) { - frags->page =3D page_pool_alloc_pages(ring->pp, gfp); - if (!frags->page) { + if (!frags->netmem) { + frags->netmem =3D page_pool_alloc_netmems(ring->pp, gfp); + if (!frags->netmem) { ring->alloc_fail++; return -ENOMEM; } - page_pool_fragment_page(frags->page, 1); + page_pool_fragment_netmem(frags->netmem, 1); frags->page_offset =3D priv->rx_headroom; =20 ring->rx_alloc_pages++; } - dma =3D page_pool_get_dma_addr(frags->page); + dma =3D page_pool_get_dma_addr_netmem(frags->netmem); rx_desc->data[i].addr =3D cpu_to_be64(dma + frags->page_offset); } return 0; @@ -83,10 +83,10 @@ static void mlx4_en_free_frag(const struct mlx4_en_priv= *priv, struct mlx4_en_rx_ring *ring, struct mlx4_en_rx_alloc *frag) { - if (frag->page) - page_pool_put_full_page(ring->pp, frag->page, false); + if (frag->netmem) + page_pool_put_full_netmem(ring->pp, frag->netmem, false); /* We need to clear all fields, otherwise a change of priv->log_rx_info - * could lead to see garbage later in frag->page. + * could lead to see garbage later in frag->netmem. */ memset(frag, 0, sizeof(*frag)); } @@ -440,29 +440,33 @@ static int mlx4_en_complete_rx_desc(struct mlx4_en_pr= iv *priv, unsigned int truesize =3D 0; bool release =3D true; int nr, frag_size; - struct page *page; + netmem_ref netmem; dma_addr_t dma; =20 /* Collect used fragments while replacing them in the HW descriptors */ for (nr =3D 0;; frags++) { frag_size =3D min_t(int, length, frag_info->frag_size); =20 - page =3D frags->page; - if (unlikely(!page)) + netmem =3D frags->netmem; + if (unlikely(!netmem)) goto fail; =20 - dma =3D page_pool_get_dma_addr(page); + dma =3D page_pool_get_dma_addr_netmem(netmem); dma_sync_single_range_for_cpu(priv->ddev, dma, frags->page_offset, frag_size, priv->dma_dir); =20 - __skb_fill_page_desc(skb, nr, page, frags->page_offset, + __skb_fill_netmem_desc(skb, nr, netmem, frags->page_offset, frag_size); =20 truesize +=3D frag_info->frag_stride; if (frag_info->frag_stride =3D=3D PAGE_SIZE / 2) { + struct page *page =3D netmem_to_page(netmem); + atomic_long_t *pp_ref_count =3D + netmem_get_pp_ref_count_ref(netmem); + frags->page_offset ^=3D PAGE_SIZE / 2; release =3D page_count(page) !=3D 1 || - atomic_long_read(&page->pp_ref_count) !=3D 1 || + atomic_long_read(pp_ref_count) !=3D 1 || page_is_pfmemalloc(page) || page_to_nid(page) !=3D numa_mem_id(); } else if (!priv->rx_headroom) { @@ -476,9 +480,9 @@ static int mlx4_en_complete_rx_desc(struct mlx4_en_priv= *priv, release =3D frags->page_offset + frag_info->frag_size > PAGE_SIZE; } if (release) { - frags->page =3D NULL; + frags->netmem =3D 0; } else { - page_pool_ref_page(page); + page_pool_ref_netmem(netmem); } =20 nr++; @@ -719,7 +723,7 @@ int mlx4_en_process_rx_cq(struct net_device *dev, struc= t mlx4_en_cq *cq, int bud int nr; =20 frags =3D ring->rx_info + (index << priv->log_rx_info); - va =3D page_address(frags[0].page) + frags[0].page_offset; + va =3D netmem_address(frags[0].netmem) + frags[0].page_offset; net_prefetchw(va); /* * make sure we read the CQE after we read the ownership bit @@ -748,7 +752,7 @@ int mlx4_en_process_rx_cq(struct net_device *dev, struc= t mlx4_en_cq *cq, int bud /* Get pointer to first fragment since we haven't * skb yet and cast it to ethhdr struct */ - dma =3D page_pool_get_dma_addr(frags[0].page); + dma =3D page_pool_get_dma_addr_netmem(frags[0].netmem); dma +=3D frags[0].page_offset; dma_sync_single_for_cpu(priv->ddev, dma, sizeof(*ethh), DMA_FROM_DEVICE); @@ -788,7 +792,7 @@ int mlx4_en_process_rx_cq(struct net_device *dev, struc= t mlx4_en_cq *cq, int bud void *orig_data; u32 act; =20 - dma =3D page_pool_get_dma_addr(frags[0].page); + dma =3D page_pool_get_dma_addr_netmem(frags[0].netmem); dma +=3D frags[0].page_offset; dma_sync_single_for_cpu(priv->ddev, dma, priv->frag_info[0].frag_size, @@ -818,7 +822,7 @@ int mlx4_en_process_rx_cq(struct net_device *dev, struc= t mlx4_en_cq *cq, int bud if (likely(!xdp_do_redirect(dev, &mxbuf.xdp, xdp_prog))) { ring->xdp_redirect++; xdp_redir_flush =3D true; - frags[0].page =3D NULL; + frags[0].netmem =3D 0; goto next; } ring->xdp_redirect_fail++; @@ -828,7 +832,7 @@ int mlx4_en_process_rx_cq(struct net_device *dev, struc= t mlx4_en_cq *cq, int bud if (likely(!mlx4_en_xmit_frame(ring, frags, priv, length, cq_ring, &doorbell_pending))) { - frags[0].page =3D NULL; + frags[0].netmem =3D 0; goto next; } trace_xdp_exception(dev, xdp_prog, act); diff --git a/drivers/net/ethernet/mellanox/mlx4/en_tx.c b/drivers/net/ether= net/mellanox/mlx4/en_tx.c index 87f35bcbeff8f..b564a953da09b 100644 --- a/drivers/net/ethernet/mellanox/mlx4/en_tx.c +++ b/drivers/net/ethernet/mellanox/mlx4/en_tx.c @@ -354,7 +354,7 @@ u32 mlx4_en_recycle_tx_desc(struct mlx4_en_priv *priv, struct page_pool *pool =3D ring->recycle_ring->pp; =20 /* Note that napi_mode =3D 0 means ndo_close() path, not budget =3D 0 */ - page_pool_put_full_page(pool, tx_info->page, !!napi_mode); + page_pool_put_full_netmem(pool, tx_info->netmem, !!napi_mode); =20 return tx_info->nr_txbb; } @@ -1191,10 +1191,10 @@ netdev_tx_t mlx4_en_xmit_frame(struct mlx4_en_rx_ri= ng *rx_ring, tx_desc =3D ring->buf + (index << LOG_TXBB_SIZE); data =3D &tx_desc->data; =20 - dma =3D page_pool_get_dma_addr(frame->page); + dma =3D page_pool_get_dma_addr_netmem(frame->netmem); =20 - tx_info->page =3D frame->page; - frame->page =3D NULL; + tx_info->netmem =3D frame->netmem; + frame->netmem =3D 0; tx_info->map0_dma =3D dma; tx_info->nr_bytes =3D max_t(unsigned int, length, ETH_ZLEN); =20 diff --git a/drivers/net/ethernet/mellanox/mlx4/mlx4_en.h b/drivers/net/eth= ernet/mellanox/mlx4/mlx4_en.h index ad0d91a751848..3ef9a0a1f783d 100644 --- a/drivers/net/ethernet/mellanox/mlx4/mlx4_en.h +++ b/drivers/net/ethernet/mellanox/mlx4/mlx4_en.h @@ -213,7 +213,7 @@ enum cq_type { struct mlx4_en_tx_info { union { struct sk_buff *skb; - struct page *page; + netmem_ref netmem; }; dma_addr_t map0_dma; u32 map0_byte_count; @@ -246,7 +246,7 @@ struct mlx4_en_tx_desc { #define MLX4_EN_CX3_HIGH_ID 0x1005 =20 struct mlx4_en_rx_alloc { - struct page *page; + netmem_ref netmem; u32 page_offset; }; =20 --=20 2.17.1 From nobody Wed Dec 17 08:56:40 2025 Received: from invmail4.hynix.com (exvmail4.skhynix.com [166.125.252.92]) by smtp.subspace.kernel.org (Postfix) with ESMTP id CAD48290BA3; Fri, 9 May 2025 11:51:44 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=166.125.252.92 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1746791507; cv=none; b=Dpxb6Eav5sTO3i8NQgP2Zpl57yqSrpKCnA/JAMp8wf78Qhfk8wU9AXEYvSg/acg4mEIOv63RL+NMb88Lj0ToKAW7BB9R+wcYerDnEFYm1X3dcOxZIFZqRQ/WYvJIrmgSl+ZYFCvZdI2zBKCYZnVkbUHjDIn+XM9RLM9gpObd1RY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1746791507; c=relaxed/simple; bh=M9cEjEFmrsQNR82PG/c3T/1k29MAQSnNtwed2DDOtHw=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References; b=JJ8BIJLQ6FhUzD4RwkOPLY7P+98jsHkf4x93DNW58usTSA+VVkFFF3ViQXFOrI4+/XCnaZ3IPckQG8lFNbl4BHjZ9DEwPehOwE2blPVpsQAJxPcvjd4b/G8GnWPTGLJGB034vanLI7upioZvIOE8e0guBTP0bSarVArhgdJOqt8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com; spf=pass smtp.mailfrom=sk.com; arc=none smtp.client-ip=166.125.252.92 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=sk.com X-AuditID: a67dfc5b-669ff7000002311f-6e-681dec49e29b From: Byungchul Park To: willy@infradead.org, netdev@vger.kernel.org Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, kernel_team@skhynix.com, kuba@kernel.org, almasrymina@google.com, ilias.apalodimas@linaro.org, harry.yoo@oracle.com, hawk@kernel.org, akpm@linux-foundation.org, ast@kernel.org, daniel@iogearbox.net, davem@davemloft.net, john.fastabend@gmail.com, andrew+netdev@lunn.ch, edumazet@google.com, pabeni@redhat.com, vishal.moola@gmail.com Subject: [RFC 12/19] netmem: introduce page_pool_recycle_direct_netmem() Date: Fri, 9 May 2025 20:51:19 +0900 Message-Id: <20250509115126.63190-13-byungchul@sk.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20250509115126.63190-1-byungchul@sk.com> References: <20250509115126.63190-1-byungchul@sk.com> X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFnrJLMWRmVeSWpSXmKPExsXC9ZZnoa7nG9kMgxn79S3mrF/DZrH6R4XF 8gc7WC2+/LzNbrF44TdmiznnW1gsnh57xG5xf9kzFos97duZLXpbfjNbNO1YwWRxYVsfq8Xl XXPYLO6t+c9qcWyBmMW3028YLdbvu8Fq8fvHHDYHIY8tK28yeeycdZfdY8GmUo/NK7Q8um5c YvbYtKqTzWPTp0nsHneu7WHzODHjN4vHzh2fmTw+Pr3F4vF+31U2j8+b5AJ4o7hsUlJzMstS i/TtErgyrv46yl7wj73ixmLFBsaHbF2MnBwSAiYSuw58Z4exZ6/tZgSx2QTUJW7c+MkMYosI GEp8fnScpYuRi4NZYCGzxJXFP4EaODiEBTwkNvy3BqlhEVCVWHtpI1g9r4CZxL/OdcwQM+Ul Vm84wAxSzgkU7/+oDhIWEjCVWDZlARvISAmB/2wS36b9Z4Kol5Q4uOIGywRG3gWMDKsYhTLz ynITM3NM9DIq8zIr9JLzczcxAiNgWe2f6B2Mny4EH2IU4GBU4uG1eC6bIcSaWFZcmXuIUYKD WUmE93mnTIYQb0piZVVqUX58UWlOavEhRmkOFiVxXqNv5SlCAumJJanZqakFqUUwWSYOTqkG xlqFd+duFOZr+GxKtC3OPjTV+kd7tvDtCzIxvysnZ0XwZjaqHy/YWyieErNh5bMHIhdNz6ZN 7Hr3Z/avj/YzTS/+MX33QbbyiNvyJ6ka6sqJk5dMSGTk5mZ+yJB0suuYlrrG60MmTxW0nGRM zu9tk1u15/f6hYdK9608t37ujrSzjRMLpv3PKlFiKc5INNRiLipOBABInBU4fAIAAA== X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFrrALMWRmVeSWpSXmKPExsXC5WfdrOv5RjbDYO5bZYs569ewWaz+UWGx /MEOVosvP2+zWyxe+I3ZYs75FhaLp8cesVvcX/aMxWJP+3Zmi96W38wWTTtWMFkcnnuS1eLC tj5Wi8u75rBZ3Fvzn9Xi2AIxi2+n3zBarN93g9Xi9485bA7CHltW3mTy2DnrLrvHgk2lHptX aHl03bjE7LFpVSebx6ZPk9g97lzbw+ZxYsZvFo+dOz4zeXx8eovF4/2+q2wei198YPL4vEku gC+KyyYlNSezLLVI3y6BK+Pqr6PsBf/YK24sVmxgfMjWxcjJISFgIjF7bTcjiM0moC5x48ZP ZhBbRMBQ4vOj4yxdjFwczAILmSWuLP7J3sXIwSEs4CGx4b81SA2LgKrE2ksbwep5Bcwk/nWu Y4aYKS+xesMBZpByTqB4/0d1kLCQgKnEsikL2CYwci1gZFjFKJKZV5abmJljqlecnVGZl1mh l5yfu4kRGM7Lav9M3MH45bL7IUYBDkYlHl6L57IZQqyJZcWVuYcYJTiYlUR4n3fKZAjxpiRW VqUW5ccXleakFh9ilOZgURLn9QpPTRASSE8sSc1OTS1ILYLJMnFwSjUw7r6+07+PoXHHRDXt I5HnVJb2iPUYaRs/d9tYEVJz/f6/cxfLe61+n+ywtaqZc7o49mqf3pbQ4IR/Bi8tv7/a4GHX 2Gzbv27T5Pq/ma/CPV3Os89TOTkt7bi1V/g/811n1i+8mKh/b+Uvew4HpieLYwW9Z96ztz0y d/2hG2msWsdmZzCIbbWvVmIpzkg01GIuKk4EACkoVnBjAgAA X-CFilter-Loop: Reflected Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" All the callers to page_pool_recycle_direct() need to be converted to use netmem descriptor and API instead of struct page things. As part of the work, introduce page_pool_recycle_direct_netmem() allowing the callers to use netmem descriptor and API. Signed-off-by: Byungchul Park --- include/net/page_pool/helpers.h | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/include/net/page_pool/helpers.h b/include/net/page_pool/helper= s.h index 582a3d00cbe23..9b7a3a996bbea 100644 --- a/include/net/page_pool/helpers.h +++ b/include/net/page_pool/helpers.h @@ -395,6 +395,12 @@ static inline void page_pool_recycle_direct(struct pag= e_pool *pool, page_pool_put_full_page(pool, page, true); } =20 +static inline void page_pool_recycle_direct_netmem(struct page_pool *pool, + netmem_ref netmem) +{ + page_pool_put_full_netmem(pool, netmem, true); +} + #define PAGE_POOL_32BIT_ARCH_WITH_64BIT_DMA \ (sizeof(dma_addr_t) > sizeof(unsigned long)) =20 --=20 2.17.1 From nobody Wed Dec 17 08:56:40 2025 Received: from invmail4.hynix.com (exvmail4.skhynix.com [166.125.252.92]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 164AF28FAB0; Fri, 9 May 2025 11:51:44 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=166.125.252.92 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1746791508; cv=none; b=P8AshW8v0rJefNmyc3/QL/88iHjfnYLs7q/qUMIv4Zs4QrWaSKqKY4qYespQxnWu+vS6Q1cg/mmlib+5bj+DBRjoJ0Jg9VXLW2dg/KsDohY5s2ogRSPO8MacU9T/awin8ZEc+7zpbKJZ32PgMDIj3uYPIqPnsaONNxNxMdJWcH8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1746791508; c=relaxed/simple; bh=IZDwWSa7tYG0AzeeZd4fNbcoiMSc2mSeAQfzyNwQjpc=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References; b=Kv+6MwNpHI6Y4amnNmXfebuY+DVSrSPEfalzXCYhnA3ILGZ9ldYiE5zIFd0wGyY9RKa5dBzWOOrb5GQrVkTgj/I7uHzjES580Hqf7sSkKRopg27QH/HhrQLg10qeCjDBg5azkPTFEAbXIEL/ar4ZmorQSqwJ+CO9uFaEM0Ynqfw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com; spf=pass smtp.mailfrom=sk.com; arc=none smtp.client-ip=166.125.252.92 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=sk.com X-AuditID: a67dfc5b-669ff7000002311f-74-681dec49a92a From: Byungchul Park To: willy@infradead.org, netdev@vger.kernel.org Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, kernel_team@skhynix.com, kuba@kernel.org, almasrymina@google.com, ilias.apalodimas@linaro.org, harry.yoo@oracle.com, hawk@kernel.org, akpm@linux-foundation.org, ast@kernel.org, daniel@iogearbox.net, davem@davemloft.net, john.fastabend@gmail.com, andrew+netdev@lunn.ch, edumazet@google.com, pabeni@redhat.com, vishal.moola@gmail.com Subject: [RFC 13/19] page_pool: expand scope of is_pp_{netmem,page}() to global Date: Fri, 9 May 2025 20:51:20 +0900 Message-Id: <20250509115126.63190-14-byungchul@sk.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20250509115126.63190-1-byungchul@sk.com> References: <20250509115126.63190-1-byungchul@sk.com> X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFnrNLMWRmVeSWpSXmKPExsXC9ZZnka7nG9kMg8b9hhZz1q9hs1j9o8Ji +YMdrBZfft5mt1i88BuzxZzzLSwWT489Yre4v+wZi8We9u3MFr0tv5ktmnasYLK4sK2P1eLy rjlsFvfW/Ge1OLZAzOLb6TeMFuv33WC1+P1jDpuDkMeWlTeZPHbOusvusWBTqcfmFVoeXTcu MXtsWtXJ5rHp0yR2jzvX9rB5nJjxm8Vj547PTB4fn95i8Xi/7yqbx+dNcgG8UVw2Kak5mWWp Rfp2CVwZvecnshRcE6houDmLuYHxJ28XIyeHhICJxOQT59hh7Bn/e5lBbDYBdYkbN36C2SIC hhKfHx1n6WLk4mAWWMgscWXxT7AGYQF/iZ87vjOC2CwCqhLH7nWwgNi8AmYSc75dY4MYKi+x esMBoEEcHJxA8f6P6iBhIQFTiWVTFrCBzJQQ+M8msfLDVSaIekmJgytusExg5F3AyLCKUSgz ryw3MTPHRC+jMi+zQi85P3cTIzAKltX+id7B+OlC8CFGAQ5GJR5ei+eyGUKsiWXFlbmHGCU4 mJVEeJ93ymQI8aYkVlalFuXHF5XmpBYfYpTmYFES5zX6Vp4iJJCeWJKanZpakFoEk2Xi4JRq YGy6FcNzyfLWP6uLmY/PlPyvzq5OVsyflHLn9zTN8+KPSkQXltzrfymcu19Jc/JngZIj5+6K iE3cqdi0brfMVWm3vlSTeUJtHq9OBl7xa2Dd2hay90FSpHW3iZf5sRaODco3biU33SkplyrP yXvGwan0JdE/WebP1X9/0k+VHtXeXdVtuPqAqxJLcUaioRZzUXEiAL7nXy9+AgAA X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFrrELMWRmVeSWpSXmKPExsXC5WfdrOv5RjbDYPoxDYs569ewWaz+UWGx /MEOVosvP2+zWyxe+I3ZYs75FhaLp8cesVvcX/aMxWJP+3Zmi96W38wWTTtWMFkcnnuS1eLC tj5Wi8u75rBZ3Fvzn9Xi2AIxi2+n3zBarN93g9Xi9485bA7CHltW3mTy2DnrLrvHgk2lHptX aHl03bjE7LFpVSebx6ZPk9g97lzbw+ZxYsZvFo+dOz4zeXx8eovF4/2+q2wei198YPL4vEku gC+KyyYlNSezLLVI3y6BK6P3/ESWgmsCFQ03ZzE3MP7k7WLk5JAQMJGY8b+XGcRmE1CXuHHj J5gtImAo8fnRcZYuRi4OZoGFzBJXFv9kB0kIC/hL/NzxnRHEZhFQlTh2r4MFxOYVMJOY8+0a G8RQeYnVGw4ADeLg4ASK939UBwkLCZhKLJuygG0CI9cCRoZVjCKZeWW5iZk5pnrF2RmVeZkV esn5uZsYgSG9rPbPxB2MXy67H2IU4GBU4uG1eC6bIcSaWFZcmXuIUYKDWUmE93mnTIYQb0pi ZVVqUX58UWlOavEhRmkOFiVxXq/w1AQhgfTEktTs1NSC1CKYLBMHp1QDI4tcX5xwzgmdQ46q AVvmhL5vdZdObP0rqHhi6cJFP7ZVc1U9v7YqmzGGc4l8XEe5477Ct83vlPgz+RZFP7m5ep5A 3aPDG7ecPM57Zv4ltjqruM+STIeDWC6s/3Q2WzKb/XiXZb+vruHvOYXK7a/8c5e9fNjq97ym sP0hd5eUQV/rWdYX0jc5lViKMxINtZiLihMB7kIiDmUCAAA= X-CFilter-Loop: Reflected Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Other than skbuff.c might need to check if a page or netmem is for page pool, for example, page_alloc.c needs to check the page state, whether it comes from page pool or not for their own purpose. Expand the scope of is_pp_netmem() and introduce is_pp_page() newly, so that those who want to check the source can achieve the checking without accessing page pool member, page->pp_magic, directly. Signed-off-by: Byungchul Park --- include/net/page_pool/types.h | 2 ++ net/core/page_pool.c | 10 ++++++++++ net/core/skbuff.c | 5 ----- 3 files changed, 12 insertions(+), 5 deletions(-) diff --git a/include/net/page_pool/types.h b/include/net/page_pool/types.h index 36eb57d73abc6..d3e1a52f01e09 100644 --- a/include/net/page_pool/types.h +++ b/include/net/page_pool/types.h @@ -299,4 +299,6 @@ static inline bool is_page_pool_compiled_in(void) /* Caller must provide appropriate safe context, e.g. NAPI. */ void page_pool_update_nid(struct page_pool *pool, int new_nid); =20 +bool is_pp_netmem(netmem_ref netmem); +bool is_pp_page(struct page *page); #endif /* _NET_PAGE_POOL_H */ diff --git a/net/core/page_pool.c b/net/core/page_pool.c index b61c1038f4c68..9c553e5a1b555 100644 --- a/net/core/page_pool.c +++ b/net/core/page_pool.c @@ -1225,3 +1225,13 @@ void net_mp_niov_clear_page_pool(struct netmem_desc = *niov) =20 page_pool_clear_pp_info(netmem); } + +bool is_pp_netmem(netmem_ref netmem) +{ + return (netmem_get_pp_magic(netmem) & ~0x3UL) =3D=3D PP_SIGNATURE; +} + +bool is_pp_page(struct page *page) +{ + return is_pp_netmem(page_to_netmem(page)); +} diff --git a/net/core/skbuff.c b/net/core/skbuff.c index 6cbf77bc61fce..11098c204fe3e 100644 --- a/net/core/skbuff.c +++ b/net/core/skbuff.c @@ -893,11 +893,6 @@ static void skb_clone_fraglist(struct sk_buff *skb) skb_get(list); } =20 -static bool is_pp_netmem(netmem_ref netmem) -{ - return (netmem_get_pp_magic(netmem) & ~0x3UL) =3D=3D PP_SIGNATURE; -} - int skb_pp_cow_data(struct page_pool *pool, struct sk_buff **pskb, unsigned int headroom) { --=20 2.17.1 From nobody Wed Dec 17 08:56:40 2025 Received: from invmail4.hynix.com (exvmail4.skhynix.com [166.125.252.92]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 43127290BC4; Fri, 9 May 2025 11:51:45 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=166.125.252.92 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1746791508; cv=none; b=L9EF0tUfBj2ZIEuzk6n/wq5AQX5VelNZ8SCH7H2TRt9GPe3s+yCwnZEgkN+TRVlSx+lswptrcxd6ACOFLsaghGGOcJvBM/t/CWJ+1tguoIzVHBU2RzFW6AhiL9DA6hQ/3Z7iOqeJKrXhdkCtot8IlrHqgJGRESJNgYUjJl+C1bU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1746791508; c=relaxed/simple; bh=2iMzVdlFrzD3Fj/Ndyc9URuSD+ZMGnQXSYGq+gYu9Fk=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References; b=IEAAgN1JvEDjQ3b0lBKX+qPfM79lqz5prAuoS66WHl4tyBShOhUcP7BXi2beL4qRJE0i6UUqqYI7cnjxWuq8OHdRWJUaCd9P2ztm7et0B+CU6dqfcTT67xrrOVtJkjFW2ExAz0tDuPBHNc0OADb5ZhQm/qb5hlzjJP/+PVhvRzA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com; spf=pass smtp.mailfrom=sk.com; arc=none smtp.client-ip=166.125.252.92 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=sk.com X-AuditID: a67dfc5b-669ff7000002311f-7a-681dec491fdd From: Byungchul Park To: willy@infradead.org, netdev@vger.kernel.org Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, kernel_team@skhynix.com, kuba@kernel.org, almasrymina@google.com, ilias.apalodimas@linaro.org, harry.yoo@oracle.com, hawk@kernel.org, akpm@linux-foundation.org, ast@kernel.org, daniel@iogearbox.net, davem@davemloft.net, john.fastabend@gmail.com, andrew+netdev@lunn.ch, edumazet@google.com, pabeni@redhat.com, vishal.moola@gmail.com Subject: [RFC 14/19] mm: page_alloc: do not directly access page->pp_magic but use is_pp_page() Date: Fri, 9 May 2025 20:51:21 +0900 Message-Id: <20250509115126.63190-15-byungchul@sk.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20250509115126.63190-1-byungchul@sk.com> References: <20250509115126.63190-1-byungchul@sk.com> X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFnrNLMWRmVeSWpSXmKPExsXC9ZZnka7nG9kMg9bXxhZz1q9hs1j9o8Ji +YMdrBZfft5mt1i88BuzxZzzLSwWT489Yre4v+wZi8We9u3MFr0tv5ktmnasYLK4sK2P1eLy rjlsFvfW/Ge1OLZAzOLb6TeMFuv33WC1+P1jDpuDkMeWlTeZPHbOusvusWBTqcfmFVoeXTcu MXtsWtXJ5rHp0yR2jzvX9rB5nJjxm8Vj547PTB4fn95i8Xi/7yqbx+dNcgG8UVw2Kak5mWWp Rfp2CVwZS76/ZyvYw1Nx4NE69gbG2VxdjJwcEgImEiev/WWDsc80XmIBsdkE1CVu3PjJDGKL CBhKfH50HCjOxcEssJBZ4srin+wgCWGBeIkjm14wgtgsAqoSDzafA4vzCphJnLq5kRFiqLzE 6g0HgAZxcHACxfs/qoOEhQRMJZZNWcAGMlNC4D+bxPlXs6COkJQ4uOIGywRG3gWMDKsYhTLz ynITM3NM9DIq8zIr9JLzczcxAqNgWe2f6B2Mny4EH2IU4GBU4uG1eC6bIcSaWFZcmXuIUYKD WUmE93mnTIYQb0piZVVqUX58UWlOavEhRmkOFiVxXqNv5SlCAumJJanZqakFqUUwWSYOTqkG RskU7jcWaTXXs743zfPIkItqm+E/VXpF4S7Tt4d41euud9uVCbwM79Pr/LKwfdE88YnKb4V6 i1R63klctvr+0ON//pvFpf3yBa86RLKP+zndiPtb/UzfIl0r2WGpncKfyhhdh3K2Fi3L87Vm TP9tG7dzfdkWNyWreI7n28KQRedtL208IlSnxFKckWioxVxUnAgA7xMAv34CAAA= X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFrrELMWRmVeSWpSXmKPExsXC5WfdrOv5RjbD4MAdLYs569ewWaz+UWGx /MEOVosvP2+zWyxe+I3ZYs75FhaLp8cesVvcX/aMxWJP+3Zmi96W38wWTTtWMFkcnnuS1eLC tj5Wi8u75rBZ3Fvzn9Xi2AIxi2+n3zBarN93g9Xi9485bA7CHltW3mTy2DnrLrvHgk2lHptX aHl03bjE7LFpVSebx6ZPk9g97lzbw+ZxYsZvFo+dOz4zeXx8eovF4/2+q2wei198YPL4vEku gC+KyyYlNSezLLVI3y6BK2PJ9/dsBXt4Kg48WsfewDibq4uRk0NCwETiTOMlFhCbTUBd4saN n8wgtoiAocTnR8eB4lwczAILmSWuLP7JDpIQFoiXOLLpBSOIzSKgKvFg8zmwOK+AmcSpmxsZ IYbKS6zecABoEAcHJ1C8/6M6SFhIwFRi2ZQFbBMYuRYwMqxiFMnMK8tNzMwx1SvOzqjMy6zQ S87P3cQIDOlltX8m7mD8ctn9EKMAB6MSD6/Fc9kMIdbEsuLK3EOMEhzMSiK8zztlMoR4UxIr q1KL8uOLSnNSiw8xSnOwKInzeoWnJggJpCeWpGanphakFsFkmTg4pRoYH3H7eh/2fvG5VPpo 98FrtuKWQsekpwUaWn/XvmDimrkoX+1Vy47a/rgH73Y6b2mYx6A7c83X7FSlZu72P8LV0c9v PkwV2ZvtvnPBut3ZlbcX6agqrXSZzXxb+9JD/Sz1L/8nHUkWeBn255zU59eRf9/zBGRdZPi5 1fXGi806J7mUvWQTnlnNVGIpzkg01GIuKk4EANsHcBBlAgAA X-CFilter-Loop: Reflected Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" To simplify struct page, the effort to seperate its own descriptor from struct page is required and the work for page pool is on going. To achieve that, all the code should avoid accessing page pool members of struct page directly, but use safe API for the corresponding purpose, that is is_pp_page() in this case. Use is_pp_page() instead of accessing the members directly, when checking if a page comes from page pool or not. Signed-off-by: Byungchul Park --- mm/page_alloc.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/mm/page_alloc.c b/mm/page_alloc.c index a6fe1e9b95941..cf672b9ab7086 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -55,6 +55,7 @@ #include #include #include +#include /* for page pool checking */ #include #include "internal.h" #include "shuffle.h" @@ -899,7 +900,7 @@ static inline bool page_expected_state(struct page *pag= e, page->memcg_data | #endif #ifdef CONFIG_PAGE_POOL - ((page->pp_magic & ~0x3UL) =3D=3D PP_SIGNATURE) | + (is_pp_page(page)) | #endif (page->flags & check_flags))) return false; @@ -928,7 +929,7 @@ static const char *page_bad_reason(struct page *page, u= nsigned long flags) bad_reason =3D "page still charged to cgroup"; #endif #ifdef CONFIG_PAGE_POOL - if (unlikely((page->pp_magic & ~0x3UL) =3D=3D PP_SIGNATURE)) + if (unlikely(is_pp_page(page))) bad_reason =3D "page_pool leak"; #endif return bad_reason; --=20 2.17.1 From nobody Wed Dec 17 08:56:40 2025 Received: from invmail4.hynix.com (exvmail4.skhynix.com [166.125.252.92]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 06CEF291161; Fri, 9 May 2025 11:51:45 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=166.125.252.92 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1746791509; cv=none; b=YBsuAJawphVz0+ht6v1kbMZ/XQCyM5wojxcUH+Z/5pHtWWW1EcQAUimuT1PMI4yCmBDbLwIiQzmfAJk5+No3y6ZPw0oWEPzNgdgZ5byRp/oQZbO0NH6JaAnKznUhMpIAu3rdB73QoGa7DSdpITuu/RTd15/V9aw0YvTR+RQEbb4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1746791509; c=relaxed/simple; bh=DkJp7wAjvMrUDAYuBwcYLyJ+TyU1S0lf69KyTOnLseo=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References; b=enF7NwD8e2jD4kBxPNcAIWL0OzouaJxHZJpFZMONcRcL9V2p7E9zV+J/SdgxKe9v1FKZ/TaPnztdGBWqZdfq3AT1gFqI6/U/zXfTdnkwfRmkq9HZ+w9LdzKan+mqYXAkmdC+VzmuBK89GWwwxwlJ8G6E2ktUAK7b6u0M+q+r26w= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com; spf=pass smtp.mailfrom=sk.com; arc=none smtp.client-ip=166.125.252.92 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=sk.com X-AuditID: a67dfc5b-669ff7000002311f-80-681dec49033a From: Byungchul Park To: willy@infradead.org, netdev@vger.kernel.org Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, kernel_team@skhynix.com, kuba@kernel.org, almasrymina@google.com, ilias.apalodimas@linaro.org, harry.yoo@oracle.com, hawk@kernel.org, akpm@linux-foundation.org, ast@kernel.org, daniel@iogearbox.net, davem@davemloft.net, john.fastabend@gmail.com, andrew+netdev@lunn.ch, edumazet@google.com, pabeni@redhat.com, vishal.moola@gmail.com Subject: [RFC 15/19] mlx5: use netmem descriptor and API for page pool Date: Fri, 9 May 2025 20:51:22 +0900 Message-Id: <20250509115126.63190-16-byungchul@sk.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20250509115126.63190-1-byungchul@sk.com> References: <20250509115126.63190-1-byungchul@sk.com> X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFnrDLMWRmVeSWpSXmKPExsXC9ZZnoa7nG9kMgw9XzC3mrF/DZrH6R4XF 8gc7WC2+/LzNbrF44TdmiznnW1gsnh57xG5xf9kzFos97duZLXpbfjNbNO1YwWRxYVsfq8Xl XXPYLO6t+c9qcWyBmMW3028YLdbvu8Fq8fvHHDYHIY8tK28yeeycdZfdY8GmUo/NK7Q8um5c YvbYtKqTzWPTp0nsHneu7WHzODHjN4vHzh2fmTw+Pr3F4vF+31U2j8+b5AJ4o7hsUlJzMstS i/TtErgyPhzdwFZwuoWx4vr2bsYGxjO5XYycHBICJhItJ34zwthv+94xg9hsAuoSN278BLNF BAwlPj86ztLFyMXBLLCQWeLK4p/sIAlhATeJm8tusoDYLAKqEn+/fGQFsXkFzCTu7frCBDFU XmL1hgNAgzg4OIHi/R/VQcJCAqYSy6YsYAOZKSHQzC4x8d5/qCMkJQ6uuMEygZF3ASPDKkah zLyy3MTMHBO9jMq8zAq95PzcTYzAOFhW+yd6B+OnC8GHGAU4GJV4eC2ey2YIsSaWFVfmHmKU 4GBWEuF93imTIcSbklhZlVqUH19UmpNafIhRmoNFSZzX6Ft5ipBAemJJanZqakFqEUyWiYNT qoExcWeg6F7rYj8t4eOzrxbv/s7azj05osxDy+GPbPYrkXqn3tcrsl5mdOfsZrfvufDZufB9 wdO6ix0H5Fi8D5q+Vsr2Do82m3p98pNNPzduiluXedI64f/2HJnNd8o/x5bzBhh33e3gcRM+ kGfkka8v8XvV0vVxpy8biN356BfKLXL46AXF1rNKLMUZiYZazEXFiQAHayQrfwIAAA== X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFrrMLMWRmVeSWpSXmKPExsXC5WfdrOv5RjbDYMMCXYs569ewWaz+UWGx /MEOVosvP2+zWyxe+I3ZYs75FhaLp8cesVvcX/aMxWJP+3Zmi96W38wWTTtWMFkcnnuS1eLC tj5Wi8u75rBZ3Fvzn9Xi2AIxi2+n3zBarN93g9Xi9485bA7CHltW3mTy2DnrLrvHgk2lHptX aHl03bjE7LFpVSebx6ZPk9g97lzbw+ZxYsZvFo+dOz4zeXx8eovF4/2+q2wei198YPL4vEku gC+KyyYlNSezLLVI3y6BK+PD0Q1sBadbGCuub+9mbGA8k9vFyMkhIWAi8bbvHTOIzSagLnHj xk8wW0TAUOLzo+MsXYxcHMwCC5klriz+yQ6SEBZwk7i57CYLiM0ioCrx98tHVhCbV8BM4t6u L0wQQ+UlVm84ADSIg4MTKN7/UR0kLCRgKrFsygK2CYxcCxgZVjGKZOaV5SZm5pjqFWdnVOZl Vugl5+duYgQG9bLaPxN3MH657H6IUYCDUYmH1+K5bIYQa2JZcWXuIUYJDmYlEd7nnTIZQrwp iZVVqUX58UWlOanFhxilOViUxHm9wlMThATSE0tSs1NTC1KLYLJMHJxSDYxPb+dGaoUUHZbJ eLCUOT1I+8Sqf/ufLD20Z//7LO7lb3c4XjNySl11e+2Cq60z+l24Bd6tcvSWOd7w5Vfo9/ad ff8kWULbCvl89u3TWdjyTNK4emqD8MOz+xoOWCutCPO3+Gvp2LnJTs2H83yyZvaUEvbX7/ZV HV58bH9x7a3FF+00ts3z61mixFKckWioxVxUnAgAe95HIWYCAAA= X-CFilter-Loop: Reflected Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" To simplify struct page, the effort to seperate its own descriptor from struct page is required and the work for page pool is on going. Use netmem descriptor and API for page pool in mlx5 code. Signed-off-by: Byungchul Park --- drivers/net/ethernet/mellanox/mlx5/core/en.h | 4 +- .../net/ethernet/mellanox/mlx5/core/en/xdp.c | 18 ++--- .../net/ethernet/mellanox/mlx5/core/en/xdp.h | 2 +- .../net/ethernet/mellanox/mlx5/core/en_main.c | 15 +++-- .../net/ethernet/mellanox/mlx5/core/en_rx.c | 66 +++++++++---------- include/linux/skbuff.h | 14 ++++ include/net/page_pool/helpers.h | 4 ++ 7 files changed, 73 insertions(+), 50 deletions(-) diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en.h b/drivers/net/eth= ernet/mellanox/mlx5/core/en.h index 32ed4963b8ada..c3992b9961540 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en.h +++ b/drivers/net/ethernet/mellanox/mlx5/core/en.h @@ -551,7 +551,7 @@ struct mlx5e_icosq { } ____cacheline_aligned_in_smp; =20 struct mlx5e_frag_page { - struct page *page; + netmem_ref netmem; u16 frags; }; =20 @@ -623,7 +623,7 @@ struct mlx5e_dma_info { dma_addr_t addr; union { struct mlx5e_frag_page *frag_page; - struct page *page; + netmem_ref netmem; }; }; =20 diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en/xdp.c b/drivers/net= /ethernet/mellanox/mlx5/core/en/xdp.c index f803e1c935900..886ed930d6a1f 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en/xdp.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/en/xdp.c @@ -61,7 +61,7 @@ static inline bool mlx5e_xmit_xdp_buff(struct mlx5e_xdpsq *sq, struct mlx5e_rq *rq, struct xdp_buff *xdp) { - struct page *page =3D virt_to_page(xdp->data); + netmem_ref netmem =3D virt_to_netmem(xdp->data); struct mlx5e_xmit_data_frags xdptxdf =3D {}; struct mlx5e_xmit_data *xdptxd; struct xdp_frame *xdpf; @@ -122,7 +122,7 @@ mlx5e_xmit_xdp_buff(struct mlx5e_xdpsq *sq, struct mlx5= e_rq *rq, * mode. */ =20 - dma_addr =3D page_pool_get_dma_addr(page) + (xdpf->data - (void *)xdpf); + dma_addr =3D page_pool_get_dma_addr_netmem(netmem) + (xdpf->data - (void = *)xdpf); dma_sync_single_for_device(sq->pdev, dma_addr, xdptxd->len, DMA_BIDIRECTI= ONAL); =20 if (xdptxd->has_frags) { @@ -134,7 +134,7 @@ mlx5e_xmit_xdp_buff(struct mlx5e_xdpsq *sq, struct mlx5= e_rq *rq, dma_addr_t addr; u32 len; =20 - addr =3D page_pool_get_dma_addr(skb_frag_page(frag)) + + addr =3D page_pool_get_dma_addr_netmem(skb_frag_netmem(frag)) + skb_frag_off(frag); len =3D skb_frag_size(frag); dma_sync_single_for_device(sq->pdev, addr, len, @@ -157,19 +157,19 @@ mlx5e_xmit_xdp_buff(struct mlx5e_xdpsq *sq, struct ml= x5e_rq *rq, (union mlx5e_xdp_info) { .page.num =3D 1 + xdptxdf.sinfo->nr_frags }); mlx5e_xdpi_fifo_push(&sq->db.xdpi_fifo, - (union mlx5e_xdp_info) { .page.page =3D page }); + (union mlx5e_xdp_info) { .page.netmem =3D netmem }); for (i =3D 0; i < xdptxdf.sinfo->nr_frags; i++) { skb_frag_t *frag =3D &xdptxdf.sinfo->frags[i]; =20 mlx5e_xdpi_fifo_push(&sq->db.xdpi_fifo, (union mlx5e_xdp_info) - { .page.page =3D skb_frag_page(frag) }); + { .page.netmem =3D skb_frag_netmem(frag) }); } } else { mlx5e_xdpi_fifo_push(&sq->db.xdpi_fifo, (union mlx5e_xdp_info) { .page.num =3D 1 }); mlx5e_xdpi_fifo_push(&sq->db.xdpi_fifo, - (union mlx5e_xdp_info) { .page.page =3D page }); + (union mlx5e_xdp_info) { .page.netmem =3D netmem }); } =20 return true; @@ -702,15 +702,15 @@ static void mlx5e_free_xdpsq_desc(struct mlx5e_xdpsq = *sq, num =3D xdpi.page.num; =20 do { - struct page *page; + netmem_ref netmem; =20 xdpi =3D mlx5e_xdpi_fifo_pop(xdpi_fifo); - page =3D xdpi.page.page; + netmem =3D xdpi.page.netmem; =20 /* No need to check ((page->pp_magic & ~0x3UL) =3D=3D PP_SIGNATURE) * as we know this is a page_pool page. */ - page_pool_recycle_direct(page->pp, page); + page_pool_recycle_direct_netmem(netmem_get_pp(netmem), netmem); } while (++n < num); =20 break; diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en/xdp.h b/drivers/net= /ethernet/mellanox/mlx5/core/en/xdp.h index 446e492c6bb8e..b37541837efba 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en/xdp.h +++ b/drivers/net/ethernet/mellanox/mlx5/core/en/xdp.h @@ -96,7 +96,7 @@ union mlx5e_xdp_info { union { struct mlx5e_rq *rq; u8 num; - struct page *page; + netmem_ref netmem; } page; struct xsk_tx_metadata_compl xsk_meta; }; diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_main.c b/drivers/ne= t/ethernet/mellanox/mlx5/core/en_main.c index 3506024c24539..c152fd454f605 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en_main.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_main.c @@ -708,24 +708,29 @@ static void mlx5e_rq_err_cqe_work(struct work_struct = *recover_work) =20 static int mlx5e_alloc_mpwqe_rq_drop_page(struct mlx5e_rq *rq) { - rq->wqe_overflow.page =3D alloc_page(GFP_KERNEL); - if (!rq->wqe_overflow.page) + struct page *page =3D alloc_page(GFP_KERNEL); + + if (!page) return -ENOMEM; =20 - rq->wqe_overflow.addr =3D dma_map_page(rq->pdev, rq->wqe_overflow.page, 0, + rq->wqe_overflow.addr =3D dma_map_page(rq->pdev, page, 0, PAGE_SIZE, rq->buff.map_dir); if (dma_mapping_error(rq->pdev, rq->wqe_overflow.addr)) { - __free_page(rq->wqe_overflow.page); + __free_page(page); return -ENOMEM; } + + rq->wqe_overflow.netmem =3D page_to_netmem(page); return 0; } =20 static void mlx5e_free_mpwqe_rq_drop_page(struct mlx5e_rq *rq) { + struct page *page =3D netmem_to_page(rq->wqe_overflow.netmem); + dma_unmap_page(rq->pdev, rq->wqe_overflow.addr, PAGE_SIZE, rq->buff.map_dir); - __free_page(rq->wqe_overflow.page); + __free_page(page); } =20 static int mlx5e_init_rxq_rq(struct mlx5e_channel *c, struct mlx5e_params = *params, diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_rx.c b/drivers/net/= ethernet/mellanox/mlx5/core/en_rx.c index 5fd70b4d55beb..ce7052287b2ce 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en_rx.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_rx.c @@ -276,16 +276,16 @@ static inline u32 mlx5e_decompress_cqes_start(struct = mlx5e_rq *rq, static int mlx5e_page_alloc_fragmented(struct mlx5e_rq *rq, struct mlx5e_frag_page *frag_page) { - struct page *page; + netmem_ref netmem; =20 - page =3D page_pool_dev_alloc_pages(rq->page_pool); - if (unlikely(!page)) + netmem =3D page_pool_dev_alloc_netmem(rq->page_pool, NULL, NULL); + if (unlikely(!netmem)) return -ENOMEM; =20 - page_pool_fragment_page(page, MLX5E_PAGECNT_BIAS_MAX); + page_pool_fragment_netmem(netmem, MLX5E_PAGECNT_BIAS_MAX); =20 *frag_page =3D (struct mlx5e_frag_page) { - .page =3D page, + .netmem =3D netmem, .frags =3D 0, }; =20 @@ -296,10 +296,10 @@ static void mlx5e_page_release_fragmented(struct mlx5= e_rq *rq, struct mlx5e_frag_page *frag_page) { u16 drain_count =3D MLX5E_PAGECNT_BIAS_MAX - frag_page->frags; - struct page *page =3D frag_page->page; + netmem_ref netmem =3D frag_page->netmem; =20 - if (page_pool_unref_page(page, drain_count) =3D=3D 0) - page_pool_put_unrefed_page(rq->page_pool, page, -1, true); + if (page_pool_unref_netmem(netmem, drain_count) =3D=3D 0) + page_pool_put_unrefed_netmem(rq->page_pool, netmem, -1, true); } =20 static inline int mlx5e_get_rx_frag(struct mlx5e_rq *rq, @@ -358,7 +358,7 @@ static int mlx5e_alloc_rx_wqe(struct mlx5e_rq *rq, stru= ct mlx5e_rx_wqe_cyc *wqe, frag->flags &=3D ~BIT(MLX5E_WQE_FRAG_SKIP_RELEASE); =20 headroom =3D i =3D=3D 0 ? rq->buff.headroom : 0; - addr =3D page_pool_get_dma_addr(frag->frag_page->page); + addr =3D page_pool_get_dma_addr_netmem(frag->frag_page->netmem); wqe->data[i].addr =3D cpu_to_be64(addr + frag->offset + headroom); } =20 @@ -501,7 +501,7 @@ mlx5e_add_skb_shared_info_frag(struct mlx5e_rq *rq, str= uct skb_shared_info *sinf { skb_frag_t *frag; =20 - dma_addr_t addr =3D page_pool_get_dma_addr(frag_page->page); + dma_addr_t addr =3D page_pool_get_dma_addr_netmem(frag_page->netmem); =20 dma_sync_single_for_cpu(rq->pdev, addr + frag_offset, len, rq->buff.map_d= ir); if (!xdp_buff_has_frags(xdp)) { @@ -514,9 +514,9 @@ mlx5e_add_skb_shared_info_frag(struct mlx5e_rq *rq, str= uct skb_shared_info *sinf } =20 frag =3D &sinfo->frags[sinfo->nr_frags++]; - skb_frag_fill_page_desc(frag, frag_page->page, frag_offset, len); + skb_frag_fill_netmem_desc(frag, frag_page->netmem, frag_offset, len); =20 - if (page_is_pfmemalloc(frag_page->page)) + if (netmem_is_pfmemalloc(frag_page->netmem)) xdp_buff_set_frag_pfmemalloc(xdp); sinfo->xdp_frags_size +=3D len; } @@ -527,27 +527,27 @@ mlx5e_add_skb_frag(struct mlx5e_rq *rq, struct sk_buf= f *skb, u32 frag_offset, u32 len, unsigned int truesize) { - dma_addr_t addr =3D page_pool_get_dma_addr(frag_page->page); + dma_addr_t addr =3D page_pool_get_dma_addr_netmem(frag_page->netmem); u8 next_frag =3D skb_shinfo(skb)->nr_frags; =20 dma_sync_single_for_cpu(rq->pdev, addr + frag_offset, len, rq->buff.map_dir); =20 - if (skb_can_coalesce(skb, next_frag, frag_page->page, frag_offset)) { + if (skb_can_coalesce_netmem(skb, next_frag, frag_page->netmem, frag_offse= t)) { skb_coalesce_rx_frag(skb, next_frag - 1, len, truesize); } else { frag_page->frags++; - skb_add_rx_frag(skb, next_frag, frag_page->page, + skb_add_rx_frag_netmem(skb, next_frag, frag_page->netmem, frag_offset, len, truesize); } } =20 static inline void mlx5e_copy_skb_header(struct mlx5e_rq *rq, struct sk_buff *skb, - struct page *page, dma_addr_t addr, + netmem_ref netmem, dma_addr_t addr, int offset_from, int dma_offset, u32 headlen) { - const void *from =3D page_address(page) + offset_from; + const void *from =3D netmem_address(netmem) + offset_from; /* Aligning len to sizeof(long) optimizes memcpy performance */ unsigned int len =3D ALIGN(headlen, sizeof(long)); =20 @@ -684,7 +684,7 @@ static int mlx5e_build_shampo_hd_umr(struct mlx5e_rq *r= q, goto err_unmap; =20 =20 - addr =3D page_pool_get_dma_addr(frag_page->page); + addr =3D page_pool_get_dma_addr_netmem(frag_page->netmem); =20 for (int j =3D 0; j < MLX5E_SHAMPO_WQ_HEADER_PER_PAGE; j++) { header_offset =3D mlx5e_shampo_hd_offset(index++); @@ -794,7 +794,7 @@ static int mlx5e_alloc_rx_mpwqe(struct mlx5e_rq *rq, u1= 6 ix) err =3D mlx5e_page_alloc_fragmented(rq, frag_page); if (unlikely(err)) goto err_unmap; - addr =3D page_pool_get_dma_addr(frag_page->page); + addr =3D page_pool_get_dma_addr_netmem(frag_page->netmem); umr_wqe->inline_mtts[i] =3D (struct mlx5_mtt) { .ptag =3D cpu_to_be64(addr | MLX5_EN_WR), }; @@ -1212,7 +1212,7 @@ static void *mlx5e_shampo_get_packet_hd(struct mlx5e_= rq *rq, u16 header_index) struct mlx5e_frag_page *frag_page =3D mlx5e_shampo_hd_to_frag_page(rq, he= ader_index); u16 head_offset =3D mlx5e_shampo_hd_offset(header_index) + rq->buff.headr= oom; =20 - return page_address(frag_page->page) + head_offset; + return netmem_address(frag_page->netmem) + head_offset; } =20 static void mlx5e_shampo_update_ipv4_udp_hdr(struct mlx5e_rq *rq, struct i= phdr *ipv4) @@ -1673,11 +1673,11 @@ mlx5e_skb_from_cqe_linear(struct mlx5e_rq *rq, stru= ct mlx5e_wqe_frag_info *wi, dma_addr_t addr; u32 frag_size; =20 - va =3D page_address(frag_page->page) + wi->offset; + va =3D netmem_address(frag_page->netmem) + wi->offset; data =3D va + rx_headroom; frag_size =3D MLX5_SKB_FRAG_SZ(rx_headroom + cqe_bcnt); =20 - addr =3D page_pool_get_dma_addr(frag_page->page); + addr =3D page_pool_get_dma_addr_netmem(frag_page->netmem); dma_sync_single_range_for_cpu(rq->pdev, addr, wi->offset, frag_size, rq->buff.map_dir); net_prefetch(data); @@ -1727,10 +1727,10 @@ mlx5e_skb_from_cqe_nonlinear(struct mlx5e_rq *rq, s= truct mlx5e_wqe_frag_info *wi =20 frag_page =3D wi->frag_page; =20 - va =3D page_address(frag_page->page) + wi->offset; + va =3D netmem_address(frag_page->netmem) + wi->offset; frag_consumed_bytes =3D min_t(u32, frag_info->frag_size, cqe_bcnt); =20 - addr =3D page_pool_get_dma_addr(frag_page->page); + addr =3D page_pool_get_dma_addr_netmem(frag_page->netmem); dma_sync_single_range_for_cpu(rq->pdev, addr, wi->offset, rq->buff.frame0_sz, rq->buff.map_dir); net_prefetchw(va); /* xdp_frame data area */ @@ -2000,12 +2000,12 @@ mlx5e_skb_from_cqe_mpwrq_nonlinear(struct mlx5e_rq = *rq, struct mlx5e_mpw_info *w =20 if (prog) { /* area for bpf_xdp_[store|load]_bytes */ - net_prefetchw(page_address(frag_page->page) + frag_offset); + net_prefetchw(netmem_address(frag_page->netmem) + frag_offset); if (unlikely(mlx5e_page_alloc_fragmented(rq, &wi->linear_page))) { rq->stats->buff_alloc_err++; return NULL; } - va =3D page_address(wi->linear_page.page); + va =3D netmem_address(wi->linear_page.netmem); net_prefetchw(va); /* xdp_frame data area */ linear_hr =3D XDP_PACKET_HEADROOM; linear_data_len =3D 0; @@ -2110,8 +2110,8 @@ mlx5e_skb_from_cqe_mpwrq_nonlinear(struct mlx5e_rq *r= q, struct mlx5e_mpw_info *w while (++pagep < frag_page); } /* copy header */ - addr =3D page_pool_get_dma_addr(head_page->page); - mlx5e_copy_skb_header(rq, skb, head_page->page, addr, + addr =3D page_pool_get_dma_addr_netmem(head_page->netmem); + mlx5e_copy_skb_header(rq, skb, head_page->netmem, addr, head_offset, head_offset, headlen); /* skb linear part was allocated with headlen and aligned to long */ skb->tail +=3D headlen; @@ -2141,11 +2141,11 @@ mlx5e_skb_from_cqe_mpwrq_linear(struct mlx5e_rq *rq= , struct mlx5e_mpw_info *wi, return NULL; } =20 - va =3D page_address(frag_page->page) + head_offset; + va =3D netmem_address(frag_page->netmem) + head_offset; data =3D va + rx_headroom; frag_size =3D MLX5_SKB_FRAG_SZ(rx_headroom + cqe_bcnt); =20 - addr =3D page_pool_get_dma_addr(frag_page->page); + addr =3D page_pool_get_dma_addr_netmem(frag_page->netmem); dma_sync_single_range_for_cpu(rq->pdev, addr, head_offset, frag_size, rq->buff.map_dir); net_prefetch(data); @@ -2184,7 +2184,7 @@ mlx5e_skb_from_cqe_shampo(struct mlx5e_rq *rq, struct= mlx5e_mpw_info *wi, struct mlx5_cqe64 *cqe, u16 header_index) { struct mlx5e_frag_page *frag_page =3D mlx5e_shampo_hd_to_frag_page(rq, he= ader_index); - dma_addr_t page_dma_addr =3D page_pool_get_dma_addr(frag_page->page); + dma_addr_t page_dma_addr =3D page_pool_get_dma_addr_netmem(frag_page->net= mem); u16 head_offset =3D mlx5e_shampo_hd_offset(header_index); dma_addr_t dma_addr =3D page_dma_addr + head_offset; u16 head_size =3D cqe->shampo.header_size; @@ -2193,7 +2193,7 @@ mlx5e_skb_from_cqe_shampo(struct mlx5e_rq *rq, struct= mlx5e_mpw_info *wi, void *hdr, *data; u32 frag_size; =20 - hdr =3D page_address(frag_page->page) + head_offset; + hdr =3D netmem_address(frag_page->netmem) + head_offset; data =3D hdr + rx_headroom; frag_size =3D MLX5_SKB_FRAG_SZ(rx_headroom + head_size); =20 @@ -2218,7 +2218,7 @@ mlx5e_skb_from_cqe_shampo(struct mlx5e_rq *rq, struct= mlx5e_mpw_info *wi, } =20 net_prefetchw(skb->data); - mlx5e_copy_skb_header(rq, skb, frag_page->page, dma_addr, + mlx5e_copy_skb_header(rq, skb, frag_page->netmem, dma_addr, head_offset + rx_headroom, rx_headroom, head_size); /* skb linear part was allocated with headlen and aligned to long */ diff --git a/include/linux/skbuff.h b/include/linux/skbuff.h index bf67c47319a56..afec5ebed4372 100644 --- a/include/linux/skbuff.h +++ b/include/linux/skbuff.h @@ -3882,6 +3882,20 @@ static inline bool skb_can_coalesce(struct sk_buff *= skb, int i, return false; } =20 +static inline bool skb_can_coalesce_netmem(struct sk_buff *skb, int i, + const netmem_ref netmem, int off) +{ + if (skb_zcopy(skb)) + return false; + if (i) { + const skb_frag_t *frag =3D &skb_shinfo(skb)->frags[i - 1]; + + return netmem =3D=3D skb_frag_netmem(frag) && + off =3D=3D skb_frag_off(frag) + skb_frag_size(frag); + } + return false; +} + static inline int __skb_linearize(struct sk_buff *skb) { return __pskb_pull_tail(skb, skb->data_len) ? 0 : -ENOMEM; diff --git a/include/net/page_pool/helpers.h b/include/net/page_pool/helper= s.h index 9b7a3a996bbea..4deb0b32e4bac 100644 --- a/include/net/page_pool/helpers.h +++ b/include/net/page_pool/helpers.h @@ -150,6 +150,10 @@ static inline netmem_ref page_pool_dev_alloc_netmem(st= ruct page_pool *pool, { gfp_t gfp =3D GFP_ATOMIC | __GFP_NOWARN; =20 + WARN_ON((!offset && size) || (offset && !size)); + if (!offset || !size) + return page_pool_alloc_netmems(pool, gfp); + return page_pool_alloc_netmem(pool, offset, size, gfp); } =20 --=20 2.17.1 From nobody Wed Dec 17 08:56:40 2025 Received: from invmail4.hynix.com (exvmail4.skhynix.com [166.125.252.92]) by smtp.subspace.kernel.org (Postfix) with ESMTP id D00062918D9; Fri, 9 May 2025 11:51:47 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=166.125.252.92 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1746791510; cv=none; b=A/6bNIRkE5p+kWHSSWLmjTecSL/uWegOpWZ/nxOUn36DEla0s7ORD3fTNFGBXJrbdAYUohuz+5s+NpLRAJGRat5pi4yfrKCUwgAc95hAjIzRe7GNeRZV73SrAi6wNdz5fBn+QPGimVj/25qwfiXrKzM9y8nCMrdXW4403oQ3A5s= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1746791510; c=relaxed/simple; bh=8uQx4Xxlw13MVRH76cgZ+D3TL/9RtWBA0M1yU0nr2pc=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References; b=JhU49Ua41tpb/t9S1vO3WYnQfvlq/3JIUgzqyi2pRO9aMlfoGhBLtM7uYFy+MdgsVyP4USGeqfHT/NESzOble/7jf5TyBxqOAiP9/88bS2Mpl/QExd1GVfpZClTnJvOislhz9OORKKyB/3amk3eEnfbsiN4hKhyuCYY3mmZTaOM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com; spf=pass smtp.mailfrom=sk.com; arc=none smtp.client-ip=166.125.252.92 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=sk.com X-AuditID: a67dfc5b-669ff7000002311f-86-681dec498798 From: Byungchul Park To: willy@infradead.org, netdev@vger.kernel.org Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, kernel_team@skhynix.com, kuba@kernel.org, almasrymina@google.com, ilias.apalodimas@linaro.org, harry.yoo@oracle.com, hawk@kernel.org, akpm@linux-foundation.org, ast@kernel.org, daniel@iogearbox.net, davem@davemloft.net, john.fastabend@gmail.com, andrew+netdev@lunn.ch, edumazet@google.com, pabeni@redhat.com, vishal.moola@gmail.com Subject: [RFC 16/19] netmem: use _Generic to cover const casting for page_to_netmem() Date: Fri, 9 May 2025 20:51:23 +0900 Message-Id: <20250509115126.63190-17-byungchul@sk.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20250509115126.63190-1-byungchul@sk.com> References: <20250509115126.63190-1-byungchul@sk.com> X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFnrJLMWRmVeSWpSXmKPExsXC9ZZnoa7XG9kMgzNPGS3mrF/DZrH6R4XF 8gc7WC2+/LzNbrF44TdmiznnW1gsnh57xG5xf9kzFos97duZLXpbfjNbNO1YwWRxYVsfq8Xl XXPYLO6t+c9qcWyBmMW3028YLdbvu8Fq8fvHHDYHIY8tK28yeeycdZfdY8GmUo/NK7Q8um5c YvbYtKqTzWPTp0nsHneu7WHzODHjN4vHzh2fmTw+Pr3F4vF+31U2j8+b5AJ4o7hsUlJzMstS i/TtErgyPj1MLejhqFjR5djA+Iiti5GTQ0LAROLC0leMMHbjvC/sIDabgLrEjRs/mUFsEQFD ic+PjrN0MXJxMAssZJa4svgnWJGwQKjEt5eHwAaxCKhKdDTOB4vzCphJfPixmBViqLzE6g0H gAZxcHACxfs/qoOEhQRMJZZNWcAGMlNC4DebxNbpX6GOkJQ4uOIGywRG3gWMDKsYhTLzynIT M3NM9DIq8zIr9JLzczcxAiNgWe2f6B2Mny4EH2IU4GBU4uG1eC6bIcSaWFZcmXuIUYKDWUmE 93mnTIYQb0piZVVqUX58UWlOavEhRmkOFiVxXqNv5SlCAumJJanZqakFqUUwWSYOTqkGxubN 64MPyu7z26U67/UZjvWqx+0LhNLO1qZuZt8VNDfN4Wrqvunb4j0rNvntapzkwN0qKZc9YdGv a2q3JtSbCa/ifJAarPosSi/ryL6u7h/ie6o+hT5L4fX/+H/9RAWnoxKSn45bqqRN/N37dNP1 7nv/SoSLt5v+6MyIetCn/NtsqfOa94fmtimxFGckGmoxFxUnAgDQFSMYfAIAAA== X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFrrALMWRmVeSWpSXmKPExsXC5WfdrOv5RjbDYN85Q4s569ewWaz+UWGx /MEOVosvP2+zWyxe+I3ZYs75FhaLp8cesVvcX/aMxWJP+3Zmi96W38wWTTtWMFkcnnuS1eLC tj5Wi8u75rBZ3Fvzn9Xi2AIxi2+n3zBarN93g9Xi9485bA7CHltW3mTy2DnrLrvHgk2lHptX aHl03bjE7LFpVSebx6ZPk9g97lzbw+ZxYsZvFo+dOz4zeXx8eovF4/2+q2wei198YPL4vEku gC+KyyYlNSezLLVI3y6BK+PTw9SCHo6KFV2ODYyP2LoYOTkkBEwkGud9YQex2QTUJW7c+MkM YosIGEp8fnScpYuRi4NZYCGzxJXFP8GKhAVCJb69PATWzCKgKtHROB8szitgJvHhx2JWiKHy Eqs3HAAaxMHBCRTv/6gOEhYSMJVYNmUB2wRGrgWMDKsYRTLzynITM3NM9YqzMyrzMiv0kvNz NzECw3lZ7Z+JOxi/XHY/xCjAwajEw2vxXDZDiDWxrLgy9xCjBAezkgjv806ZDCHelMTKqtSi /Pii0pzU4kOM0hwsSuK8XuGpCUIC6YklqdmpqQWpRTBZJg5OqQbGyJ1GAfLzGK3Xue+cv614 2pLoX2VfeSpOvBIp+pTqz5Fmt9PhtlffwprIs15sM/fbKl7bs3/bs29zr5w66L79oNzpHeKd TocywlYwXbos5M3AmmNZEBVzdbO4nHScmFWr/Aqp5cXbN6fNCwsuEv2YYCX9hXPNh9I5nHYh 257Imp74M//sofkXlFiKMxINtZiLihMBB5a/ZmMCAAA= X-CFilter-Loop: Reflected Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" The current page_to_netmem() doesn't cover const casting resulting in trying to cast const struct page * to const netmem_ref fails. To cover the case, change page_to_netmem() to use macro and _Generic. Signed-off-by: Byungchul Park --- include/net/netmem.h | 7 +++---- 1 file changed, 3 insertions(+), 4 deletions(-) diff --git a/include/net/netmem.h b/include/net/netmem.h index c87a604e980b9..ce3765e675d19 100644 --- a/include/net/netmem.h +++ b/include/net/netmem.h @@ -133,10 +133,9 @@ static inline netmem_ref net_iov_to_netmem(struct netm= em_desc *niov) return (__force netmem_ref)((unsigned long)niov | NET_IOV); } =20 -static inline netmem_ref page_to_netmem(struct page *page) -{ - return (__force netmem_ref)page; -} +#define page_to_netmem(p) (_Generic((p), \ + const struct page *: (__force const netmem_ref)(p), \ + struct page *: (__force netmem_ref)(p))) =20 static inline netmem_ref alloc_netmems_node(int nid, gfp_t gfp_mask, unsigned int order) --=20 2.17.1 From nobody Wed Dec 17 08:56:40 2025 Received: from invmail4.hynix.com (exvmail4.skhynix.com [166.125.252.92]) by smtp.subspace.kernel.org (Postfix) with ESMTP id E9E9A2918DC; Fri, 9 May 2025 11:51:47 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=166.125.252.92 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1746791510; cv=none; b=g58jOhqCjQA3T0PAG62RYbcA7erifuI9gyLiYtDyLP6MXQkN/+F61ne8UKSFaVSMWtsaIl4NQ4NUxmojKXM6F7By+BWp5L3sItjJb1CzN0uajiOcp02ZhQTJD4UPUOyI4dB5y8oF/UgW3uzXc2SpUp7+4M8fMYNgKJ6uB3OZTmI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1746791510; c=relaxed/simple; bh=P4lYjrKn+hBN6dkHzEPzkQ/tgiyPOVAU0Xr1rVFaLVQ=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References; b=J4GirDSsHbCyOOU923tYEhy9fPtkWnGv9TrreCAhJpeYo4E9bOFDBrzxynd6CxkptsEQNSIYqSzfre6vqzVexdRLdQGtectwn7PPEqGJUUKv/k/TzArSqidPJ2os53HO9VpSXjZwA3m3WfFMLSBxprb/F4EAkjQi8U48MqP9qG8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com; spf=pass smtp.mailfrom=sk.com; arc=none smtp.client-ip=166.125.252.92 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=sk.com X-AuditID: a67dfc5b-669ff7000002311f-8c-681dec494b65 From: Byungchul Park To: willy@infradead.org, netdev@vger.kernel.org Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, kernel_team@skhynix.com, kuba@kernel.org, almasrymina@google.com, ilias.apalodimas@linaro.org, harry.yoo@oracle.com, hawk@kernel.org, akpm@linux-foundation.org, ast@kernel.org, daniel@iogearbox.net, davem@davemloft.net, john.fastabend@gmail.com, andrew+netdev@lunn.ch, edumazet@google.com, pabeni@redhat.com, vishal.moola@gmail.com Subject: [RFC 17/19] netmem: remove __netmem_get_pp() Date: Fri, 9 May 2025 20:51:24 +0900 Message-Id: <20250509115126.63190-18-byungchul@sk.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20250509115126.63190-1-byungchul@sk.com> References: <20250509115126.63190-1-byungchul@sk.com> X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFnrNLMWRmVeSWpSXmKPExsXC9ZZnka7XG9kMg+N3OC3mrF/DZrH6R4XF 8gc7WC2+/LzNbrF44TdmiznnW1gsnh57xG5xf9kzFos97duZLXpbfjNbNO1YwWRxYVsfq8Xl XXPYLO6t+c9qcWyBmMW3028YLdbvu8Fq8fvHHDYHIY8tK28yeeycdZfdY8GmUo/NK7Q8um5c YvbYtKqTzWPTp0nsHneu7WHzODHjN4vHzh2fmTw+Pr3F4vF+31U2j8+b5AJ4o7hsUlJzMstS i/TtErgyZv2dyVLwlbNiw6uXjA2MCzi6GDk5JARMJJ68+8EGY3fP/8YCYrMJqEvcuPGTGcQW ETCU+PzoOFCci4NZYCGzxJXFP9lBEsICphKPlvcxgtgsAqoSTZPPgzXzCphJLO+6DzVUXmL1 hgNAgzg4OIHi/R/VQcJCQK3LpixgA5kpIfCfTWL+t8PsEPWSEgdX3GCZwMi7gJFhFaNQZl5Z bmJmjoleRmVeZoVecn7uJkZgFCyr/RO9g/HTheBDjAIcjEo8vBbPZTOEWBPLiitzDzFKcDAr ifA+75TJEOJNSaysSi3Kjy8qzUktPsQozcGiJM5r9K08RUggPbEkNTs1tSC1CCbLxMEp1cDI 7WvWfvzH0mcpoa0v2btlTzznKOa0ytKXScnN2DZznxJL69NTb2/npZnsnsJhVpB4ZZNvF+sT LgGPw9eX2P4+c/yN4I4zVu1NH5iYom3zDVRcEpZUxZR4Ogn/VmR8tzbbwepHtmzp5CePvTQN t20v36LstEzooaBqn/5dxw0+8qtk34lHHVJiKc5INNRiLipOBAD2TInOfgIAAA== X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFrrELMWRmVeSWpSXmKPExsXC5WfdrOv5RjbD4NsNC4s569ewWaz+UWGx /MEOVosvP2+zWyxe+I3ZYs75FhaLp8cesVvcX/aMxWJP+3Zmi96W38wWTTtWMFkcnnuS1eLC tj5Wi8u75rBZ3Fvzn9Xi2AIxi2+n3zBarN93g9Xi9485bA7CHltW3mTy2DnrLrvHgk2lHptX aHl03bjE7LFpVSebx6ZPk9g97lzbw+ZxYsZvFo+dOz4zeXx8eovF4/2+q2wei198YPL4vEku gC+KyyYlNSezLLVI3y6BK2PW35ksBV85Kza8esnYwLiAo4uRk0NCwESie/43FhCbTUBd4saN n8wgtoiAocTnR8eB4lwczAILmSWuLP7JDpIQFjCVeLS8jxHEZhFQlWiafB6smVfATGJ51302 iKHyEqs3HAAaxMHBCRTv/6gOEhYCal02ZQHbBEauBYwMqxhFMvPKchMzc0z1irMzKvMyK/SS 83M3MQJDelntn4k7GL9cdj/EKMDBqMTDa/FcNkOINbGsuDL3EKMEB7OSCO/zTpkMId6UxMqq 1KL8+KLSnNTiQ4zSHCxK4rxe4akJQgLpiSWp2ampBalFMFkmDk6pBsZ+iYUbfFVOLEpT+CZp YDR11bEL5zkK0nl0m4xF+My4ld2i1/5N1DqhLlbRaLLo6aMDAttjGRd0BossU/jEHlC/gHNR zNl/pbwrPh0Lt2BlFbypFK6wNO1ZxIxdPxtPPPb/WrvxjLOQ4r5b23b4ntld6je14/vCWU+3 fD4/ZR7Dy3Je1t2cx5cpsRRnJBpqMRcVJwIA+dmTaWUCAAA= X-CFilter-Loop: Reflected Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" There are no users of __netmem_get_pp(). Remove it. Signed-off-by: Byungchul Park Reviewed-by: Mina Almasry --- include/net/netmem.h | 16 ---------------- 1 file changed, 16 deletions(-) diff --git a/include/net/netmem.h b/include/net/netmem.h index ce3765e675d19..00064e766b889 100644 --- a/include/net/netmem.h +++ b/include/net/netmem.h @@ -190,22 +190,6 @@ static inline struct netmem_desc *__netmem_clear_lsb(n= etmem_ref netmem) return (struct netmem_desc *)((__force unsigned long)netmem & ~NET_IOV); } =20 -/** - * __netmem_get_pp - unsafely get pointer to the &page_pool backing @netmem - * @netmem: netmem reference to get the pointer from - * - * Unsafe version of netmem_get_pp(). When @netmem is always page-backed, - * e.g. when it's a header buffer, performs faster and generates smaller - * object code (avoids clearing the LSB). When @netmem points to IOV, - * provokes invalid memory access. - * - * Return: pointer to the &page_pool (garbage if @netmem is not page-backe= d). - */ -static inline struct page_pool *__netmem_get_pp(netmem_ref netmem) -{ - return __netmem_to_page(netmem)->pp; -} - static inline struct page_pool *netmem_get_pp(netmem_ref netmem) { return __netmem_clear_lsb(netmem)->pp; --=20 2.17.1 From nobody Wed Dec 17 08:56:40 2025 Received: from invmail4.hynix.com (exvmail4.hynix.com [166.125.252.92]) by smtp.subspace.kernel.org (Postfix) with ESMTP id CE5EA2918D5; Fri, 9 May 2025 11:51:47 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=166.125.252.92 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1746791512; cv=none; b=eFOWpuduJzlCyJ85jv1pN02eMfI3RcuO578Gk1Fqm4bLok2LHYVpLZ5R5LkgZedEz1lxxWVzCLnjGMrx4fBiXyBCSak2lbzO3UxoTrzSCec2V0KnleWYiZNsVDr0ZxvbBmfFKUXYz7eXENUySDIatmVLBczvrrqLmqfopFd/YW0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1746791512; c=relaxed/simple; bh=sibsT374OtjUcS3jJqLxxETwOcudQO7Eq4S6H+J68sQ=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References; b=gSKafUZ9CE5LkjNvpT8d5EVRBLQyPhpRQDFcOxgsdpTzlCGzAU62udNAw2Mo180+D79BYOA7UNyA6ZPs5IRdE7eKRy6QxAil8BcKEEWpDfYkqnr+BXRY69wa1qOsSRoHHU+7PGyd4pgrb4+hOhg48RKIHCx5jLG5+P21VyIp4Tg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com; spf=pass smtp.mailfrom=sk.com; arc=none smtp.client-ip=166.125.252.92 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=sk.com X-AuditID: a67dfc5b-669ff7000002311f-92-681dec4a662d From: Byungchul Park To: willy@infradead.org, netdev@vger.kernel.org Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, kernel_team@skhynix.com, kuba@kernel.org, almasrymina@google.com, ilias.apalodimas@linaro.org, harry.yoo@oracle.com, hawk@kernel.org, akpm@linux-foundation.org, ast@kernel.org, daniel@iogearbox.net, davem@davemloft.net, john.fastabend@gmail.com, andrew+netdev@lunn.ch, edumazet@google.com, pabeni@redhat.com, vishal.moola@gmail.com Subject: [RFC 18/19] page_pool: make page_pool_get_dma_addr() just wrap page_pool_get_dma_addr_netmem() Date: Fri, 9 May 2025 20:51:25 +0900 Message-Id: <20250509115126.63190-19-byungchul@sk.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20250509115126.63190-1-byungchul@sk.com> References: <20250509115126.63190-1-byungchul@sk.com> X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFnrJLMWRmVeSWpSXmKPExsXC9ZZnoa7XG9kMg1svOCzmrF/DZrH6R4XF 8gc7WC2+/LzNbrF44TdmiznnW1gsnh57xG5xf9kzFos97duZLXpbfjNbNO1YwWRxYVsfq8Xl XXPYLO6t+c9qcWyBmMW3028YLdbvu8Fq8fvHHDYHIY8tK28yeeycdZfdY8GmUo/NK7Q8um5c YvbYtKqTzWPTp0nsHneu7WHzODHjN4vHzh2fmTw+Pr3F4vF+31U2j8+b5AJ4o7hsUlJzMstS i/TtErgyFp9fxl6wgaPiSecfxgbGyexdjJwcEgImEhvffIOzXzw6yARiswmoS9y48ZMZxBYR MJT4/Og4SxcjFwezwEJmiSuLf4I1CAukS8y+tIixi5GDg0VAVeL1RW4Qk1fATOLR4iCIkfIS qzccYAYJcwKF+z+qg4SFBEwllk1ZwAYyUULgP5vEiZsf2CDqJSUOrrjBMoGRdwEjwypGocy8 stzEzBwTvYzKvMwKveT83E2MwAhYVvsnegfjpwvBhxgFOBiVeHgtnstmCLEmlhVX5h5ilOBg VhLhfd4pkyHEm5JYWZValB9fVJqTWnyIUZqDRUmc1+hbeYqQQHpiSWp2ampBahFMlomDU6qB sb9MV1H8FVOY+WFf7xa3KWeqz/NYvjoqYdubuuLEtg/Tzu+UTdb/bNuqHmKoFuj/8t2kaZ1n 5r+PLwlZw3O/8EWYzY5bU5vnFlY4tb5smFwlaCzgeF9qemU2f++xwBMvb6uevvu5Y4PkkcO/ o7cLvZnouSXp6Ob8M0tS52Zd/7mxS6NTaELfLCWW4oxEQy3mouJEADmA7818AgAA X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFjrPLMWRmVeSWpSXmKPExsXC5WfdrOv1RjbDoFvDYs76NWwWq39UWCx/ sIPV4svP2+wWixd+Y7aYc76FxeLpsUfsFveXPWOx2NO+ndmit+U3s0XTjhVMFofnnmS1uLCt j9Xi8q45bBb31vxntTi2QMzi2+k3jBbr991gtfj9Yw6bg7DHlpU3mTx2zrrL7rFgU6nH5hVa Hl03LjF7bFrVyeax6dMkdo871/aweZyY8ZvFY+eOz0weH5/eYvF4v+8qm8fiFx+YPD5vkgvg i+KySUnNySxLLdK3S+DKWHx+GXvBBo6KJ51/GBsYJ7N3MXJySAiYSLx4dJAJxGYTUJe4ceMn M4gtImAo8fnRcZYuRi4OZoGFzBJXFv8EaxAWSJeYfWkRYxcjBweLgKrE64vcICavgJnEo8VB ECPlJVZvOMAMEuYECvd/VAcJCwmYSiybsoBtAiPXAkaGVYwimXlluYmZOaZ6xdkZlXmZFXrJ +bmbGIHBvKz2z8QdjF8uux9iFOBgVOLhtXgumyHEmlhWXJl7iFGCg1lJhPd5p0yGEG9KYmVV alF+fFFpTmrxIUZpDhYlcV6v8NQEIYH0xJLU7NTUgtQimCwTB6dUA+OysyttVlSeF/0fwDj7 +JncK5zTZjdynjnPlfdg0jV+4/vZ+89rcLtbSi08UKP2x+hS3eopZuf80zex/Y3lS1764VWP ysNyZcceHenfnw50MhptaZyo0p0suf/3rrvTvC7xq6Vef9+Xb9t/R2xxCfOp8mONWkrn6+2e haSK1xaF3az3OPHmQpQSS3FGoqEWc1FxIgATlcwqYgIAAA== X-CFilter-Loop: Reflected Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" The page pool members in struct page cannot be removed unless it's not allowed to access any of them via struct page. Do not access 'page->dma_addr' directly in page_pool_get_dma_addr() but just wrap page_pool_get_dma_addr_netmem() safely. Signed-off-by: Byungchul Park Reviewed-by: Ilias Apalodimas Reviewed-by: Mina Almasry --- include/net/page_pool/helpers.h | 7 +------ 1 file changed, 1 insertion(+), 6 deletions(-) diff --git a/include/net/page_pool/helpers.h b/include/net/page_pool/helper= s.h index 4deb0b32e4bac..7e0395c70bfa2 100644 --- a/include/net/page_pool/helpers.h +++ b/include/net/page_pool/helpers.h @@ -441,12 +441,7 @@ static inline dma_addr_t page_pool_get_dma_addr_netmem= (netmem_ref netmem) */ static inline dma_addr_t page_pool_get_dma_addr(const struct page *page) { - dma_addr_t ret =3D page->dma_addr; - - if (PAGE_POOL_32BIT_ARCH_WITH_64BIT_DMA) - ret <<=3D PAGE_SHIFT; - - return ret; + return page_pool_get_dma_addr_netmem(page_to_netmem(page)); } =20 static inline void __page_pool_dma_sync_for_cpu(const struct page_pool *po= ol, --=20 2.17.1 From nobody Wed Dec 17 08:56:40 2025 Received: from invmail4.hynix.com (exvmail4.skhynix.com [166.125.252.92]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 1DB602918F1; Fri, 9 May 2025 11:51:47 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=166.125.252.92 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1746791511; cv=none; b=MpTmkfpVHNY1b74axei+Yiyo0DvBn3Ldv2JGD/fkXJBtBEqSDr5edcaTNL+Jcvt0OwBgLeZllhGsx6gWkgLOigNblEqaNEzbrZBJjQ4q7jZXwiQsQxyATzTFrf/XHn1QzneQNIprdcfnqPCIj+IGX+0QSHtCHT996uNXrzT4vz4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1746791511; c=relaxed/simple; bh=5HbkRADnH0AfURkYRtpP5zV79eZ3YdBexWxwI/8bZw8=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References; b=VqqfAUSSmeqVslFr1c3LD4Zr4+D0qbM5miz+YrRAz47NP0uife7O2BEmVsRO8qVhlGTiF1/E2Yj2Tu0IMc4xfXv6W3h1Nv+cnkaFFcAi/Urla8QLJGnld+HOpfavAstwNkuaQfk72kfxAOdJ5LwYVppXE9ltUxku5uayuLmoATE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com; spf=pass smtp.mailfrom=sk.com; arc=none smtp.client-ip=166.125.252.92 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=sk.com X-AuditID: a67dfc5b-669ff7000002311f-98-681dec4a7ecf From: Byungchul Park To: willy@infradead.org, netdev@vger.kernel.org Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, kernel_team@skhynix.com, kuba@kernel.org, almasrymina@google.com, ilias.apalodimas@linaro.org, harry.yoo@oracle.com, hawk@kernel.org, akpm@linux-foundation.org, ast@kernel.org, daniel@iogearbox.net, davem@davemloft.net, john.fastabend@gmail.com, andrew+netdev@lunn.ch, edumazet@google.com, pabeni@redhat.com, vishal.moola@gmail.com Subject: [RFC 19/19] mm, netmem: remove the page pool members in struct page Date: Fri, 9 May 2025 20:51:26 +0900 Message-Id: <20250509115126.63190-20-byungchul@sk.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20250509115126.63190-1-byungchul@sk.com> References: <20250509115126.63190-1-byungchul@sk.com> X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFnrFLMWRmVeSWpSXmKPExsXC9ZZnka7XG9kMg2PX+SzmrF/DZrH6R4XF 8gc7WC2+/LzNbrF44TdmiznnW1gsnh57xG5xf9kzFos97duZLXpbfjNbNO1YwWRxYVsfq8Xl XXPYLO6t+c9qcWyBmMW3028YLdbvu8Fq8fvHHDYHIY8tK28yeeycdZfdY8GmUo/NK7Q8um5c YvbYtKqTzWPTp0nsHneu7WHzODHjN4vHzh2fmTw+Pr3F4vF+31U2j8+b5AJ4o7hsUlJzMstS i/TtErgydh+7xl4wW6ni0dd+9gbG2dJdjJwcEgImEu2nD7LA2Fu2vgCz2QTUJW7c+MkMYosI GEp8fnQcKM7FwSywkFniyuKf7CAJYQEfidZrd4CKODhYBFQldjdEgIR5Bcwkth7ZxQoxU15i 9YYDYCWcQPH+j+ogYSEBU4llUxawgYyUEPjPJtH/5zMjRL2kxMEVN1gmMPIuYGRYxSiUmVeW m5iZY6KXUZmXWaGXnJ+7iREYA8tq/0TvYPx0IfgQowAHoxIPr8Vz2Qwh1sSy4srcQ4wSHMxK IrzPO2UyhHhTEiurUovy44tKc1KLDzFKc7AoifMafStPERJITyxJzU5NLUgtgskycXBKNTB6 lPLrf9nrr5vIZdsyx10m80rK37Tfu2dtUZza3LfQ+UjHRsfkqom8E3bYXbiTIb0mt2HXzbOK bm2mPQk7rvP/7i09XP/yu3qej/zkK+LOzK93s9vuOW4Zmru5v4nvIC/3xvBWgzqNhiPcUp6H 94bw8guu59dcV3xpe1fIjCcrA7dIr19fu1KJpTgj0VCLuag4EQBHz4cXfQIAAA== X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFrrILMWRmVeSWpSXmKPExsXC5WfdrOv1RjbD4OhNJos569ewWaz+UWGx /MEOVosvP2+zWyxe+I3ZYs75FhaLp8cesVvcX/aMxWJP+3Zmi96W38wWTTtWMFkcnnuS1eLC tj5Wi8u75rBZ3Fvzn9Xi2AIxi2+n3zBarN93g9Xi9485bA7CHltW3mTy2DnrLrvHgk2lHptX aHl03bjE7LFpVSebx6ZPk9g97lzbw+ZxYsZvFo+dOz4zeXx8eovF4/2+q2wei198YPL4vEku gC+KyyYlNSezLLVI3y6BK2P3sWvsBbOVKh597WdvYJwt3cXIySEhYCKxZesLFhCbTUBd4saN n8wgtoiAocTnR8eB4lwczAILmSWuLP7JDpIQFvCRaL12B6iIg4NFQFVid0MESJhXwExi65Fd rBAz5SVWbzgAVsIJFO//qA4SFhIwlVg2ZQHbBEauBYwMqxhFMvPKchMzc0z1irMzKvMyK/SS 83M3MQIDelntn4k7GL9cdj/EKMDBqMTDa/FcNkOINbGsuDL3EKMEB7OSCO/zTpkMId6UxMqq 1KL8+KLSnNTiQ4zSHCxK4rxe4akJQgLpiSWp2ampBalFMFkmDk6pBkaWQ4KSvseWGcZPqJp5 WuZr8on9agrFqTKPix26MrzU9+kvX9C8xmnFriwF0RcC+2JPZbBtK2LYN6NNsaAkQZI70Itp 1Wa2xnudRZeW3r03+ZvyDIsnIeuyJiXFf+VS7Jq6dMnEfIPpinsYzU48ufT42MeopZ+PVYtc CA70eS3D0WxxX2u24E4lluKMREMt5qLiRACPzzROZAIAAA== X-CFilter-Loop: Reflected Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Now that all the users of the page pool members in struct page have been gone, the members can be removed from struct page. However, the space in struct page needs to be kept using a place holder with the same size, until struct netmem_desc has its own instance, not overlayed onto struct page, to avoid conficting with other members within struct page. Remove the page pool members in struct page and replace with a place holder. The place holder should be removed once struct netmem_desc has its own instance. Signed-off-by: Byungchul Park --- include/linux/mm_types.h | 13 ++----------- include/net/netmem.h | 35 +---------------------------------- include/net/netmem_type.h | 22 ++++++++++++++++++++++ 3 files changed, 25 insertions(+), 45 deletions(-) create mode 100644 include/net/netmem_type.h diff --git a/include/linux/mm_types.h b/include/linux/mm_types.h index e76bade9ebb12..69904a0855358 100644 --- a/include/linux/mm_types.h +++ b/include/linux/mm_types.h @@ -20,6 +20,7 @@ #include #include #include +#include /* for page pool */ =20 #include =20 @@ -118,17 +119,7 @@ struct page { */ unsigned long private; }; - struct { /* page_pool used by netstack */ - /** - * @pp_magic: magic value to avoid recycling non - * page_pool allocated pages. - */ - unsigned long pp_magic; - struct page_pool *pp; - unsigned long _pp_mapping_pad; - unsigned long dma_addr; - atomic_long_t pp_ref_count; - }; + struct __netmem_desc place_holder_1; /* for page pool */ struct { /* Tail pages of compound page */ unsigned long compound_head; /* Bit zero is set */ }; diff --git a/include/net/netmem.h b/include/net/netmem.h index 00064e766b889..c414de6c6ab0d 100644 --- a/include/net/netmem.h +++ b/include/net/netmem.h @@ -10,6 +10,7 @@ =20 #include #include +#include =20 /* net_iov */ =20 @@ -20,15 +21,6 @@ DECLARE_STATIC_KEY_FALSE(page_pool_mem_providers); */ #define NET_IOV 0x01UL =20 -struct netmem_desc { - unsigned long __unused_padding; - unsigned long pp_magic; - struct page_pool *pp; - struct net_iov_area *owner; - unsigned long dma_addr; - atomic_long_t pp_ref_count; -}; - struct net_iov_area { /* Array of net_iovs for this area. */ struct netmem_desc *niovs; @@ -38,31 +30,6 @@ struct net_iov_area { unsigned long base_virtual; }; =20 -/* These fields in struct page are used by the page_pool and net stack: - * - * struct { - * unsigned long pp_magic; - * struct page_pool *pp; - * unsigned long _pp_mapping_pad; - * unsigned long dma_addr; - * atomic_long_t pp_ref_count; - * }; - * - * We mirror the page_pool fields here so the page_pool can access these f= ields - * without worrying whether the underlying fields belong to a page or net_= iov. - * - * The non-net stack fields of struct page are private to the mm stack and= must - * never be mirrored to net_iov. - */ -#define NET_IOV_ASSERT_OFFSET(pg, iov) \ - static_assert(offsetof(struct page, pg) =3D=3D \ - offsetof(struct netmem_desc, iov)) -NET_IOV_ASSERT_OFFSET(pp_magic, pp_magic); -NET_IOV_ASSERT_OFFSET(pp, pp); -NET_IOV_ASSERT_OFFSET(dma_addr, dma_addr); -NET_IOV_ASSERT_OFFSET(pp_ref_count, pp_ref_count); -#undef NET_IOV_ASSERT_OFFSET - static inline struct net_iov_area *net_iov_owner(const struct netmem_desc = *niov) { return niov->owner; diff --git a/include/net/netmem_type.h b/include/net/netmem_type.h new file mode 100644 index 0000000000000..6a3ac8e908515 --- /dev/null +++ b/include/net/netmem_type.h @@ -0,0 +1,22 @@ +/* SPDX-License-Identifier: GPL-2.0 + * + * Author: Byungchul Park + */ + +#ifndef _NET_NETMEM_TYPE_H +#define _NET_NETMEM_TYPE_H + +#include + +struct netmem_desc { + unsigned long __unused_padding; + struct_group_tagged(__netmem_desc, actual_data, + unsigned long pp_magic; + struct page_pool *pp; + struct net_iov_area *owner; + unsigned long dma_addr; + atomic_long_t pp_ref_count; + ); +}; + +#endif /* _NET_NETMEM_TYPE_H */ --=20 2.17.1