drivers/net/ethernet/mellanox/mlx4/en_rx.c | 46 ++++---- drivers/net/ethernet/mellanox/mlx4/en_tx.c | 8 +- drivers/net/ethernet/mellanox/mlx4/mlx4_en.h | 4 +- drivers/net/ethernet/mellanox/mlx5/core/en.h | 4 +- .../net/ethernet/mellanox/mlx5/core/en/xdp.c | 18 ++-- .../net/ethernet/mellanox/mlx5/core/en/xdp.h | 2 +- .../net/ethernet/mellanox/mlx5/core/en_main.c | 15 ++- .../net/ethernet/mellanox/mlx5/core/en_rx.c | 66 ++++++------ drivers/net/netdevsim/netdev.c | 18 ++-- drivers/net/netdevsim/netdevsim.h | 2 +- include/linux/mm.h | 5 +- include/linux/mm_types.h | 11 -- include/linux/skbuff.h | 14 +++ include/net/netmem.h | 101 ++++++++++-------- include/net/page_pool/helpers.h | 11 +- net/core/page_pool.c | 97 +++++++++-------- 16 files changed, 221 insertions(+), 201 deletions(-)
The MM subsystem is trying to reduce struct page to a single pointer.
The first step towards that is splitting struct page by its individual
users, as has already been done with folio and slab. This patchset does
that for netmem which is used for page pools.
Matthew Wilcox tried and stopped the same work, you can see in:
https://lore.kernel.org/linux-mm/20230111042214.907030-1-willy@infradead.org/
Mina Almasry already has done a lot fo prerequisite works by luck, he
said :). I stacked my patches on the top of his work e.i. netmem.
I focused on removing the page pool members in struct page this time,
not moving the allocation code of page pool from net to mm. It can be
done later if needed.
My rfc version of this work is:
https://lore.kernel.org/all/20250509115126.63190-1-byungchul@sk.com/
There are still a lot of works to do, to remove the dependency on struct
page in the network subsystem. I will continue to work on this after
this base patchset is merged.
---
Changes from rfc:
1. Rebase on net-next's main branch
https://git.kernel.org/pub/scm/linux/kernel/git/netdev/net-next.git/
2. Fix a build error reported by kernel test robot
https://lore.kernel.org/all/202505100932.uzAMBW1y-lkp@intel.com/
3. Add given 'Reviewed-by's, thanks to Mina and Ilias
4. Do static_assert() on the size of struct netmem_desc instead
of placing place-holder in struct page, feedbacked by Matthew
5. Do struct_group_tagged(netmem_desc) on struct net_iov instead
of wholly renaming it to strcut netmem_desc, feedbacked by
Mina and Pavel
Byungchul Park (18):
netmem: introduce struct netmem_desc struct_group_tagged()'ed on
struct net_iov
netmem: introduce netmem alloc APIs to wrap page alloc APIs
page_pool: use netmem alloc/put APIs in __page_pool_alloc_page_order()
page_pool: rename __page_pool_alloc_page_order() to
__page_pool_alloc_large_netmem()
page_pool: use netmem alloc/put APIs in __page_pool_alloc_pages_slow()
page_pool: rename page_pool_return_page() to page_pool_return_netmem()
page_pool: use netmem put API in page_pool_return_netmem()
page_pool: rename __page_pool_release_page_dma() to
__page_pool_release_netmem_dma()
page_pool: rename __page_pool_put_page() to __page_pool_put_netmem()
page_pool: rename __page_pool_alloc_pages_slow() to
__page_pool_alloc_netmems_slow()
mlx4: use netmem descriptor and APIs for page pool
page_pool: use netmem APIs to access page->pp_magic in
page_pool_page_is_pp()
mlx5: use netmem descriptor and APIs for page pool
netmem: use _Generic to cover const casting for page_to_netmem()
netmem: remove __netmem_get_pp()
page_pool: make page_pool_get_dma_addr() just wrap
page_pool_get_dma_addr_netmem()
netdevsim: use netmem descriptor and APIs for page pool
mm, netmem: remove the page pool members in struct page
drivers/net/ethernet/mellanox/mlx4/en_rx.c | 46 ++++----
drivers/net/ethernet/mellanox/mlx4/en_tx.c | 8 +-
drivers/net/ethernet/mellanox/mlx4/mlx4_en.h | 4 +-
drivers/net/ethernet/mellanox/mlx5/core/en.h | 4 +-
.../net/ethernet/mellanox/mlx5/core/en/xdp.c | 18 ++--
.../net/ethernet/mellanox/mlx5/core/en/xdp.h | 2 +-
.../net/ethernet/mellanox/mlx5/core/en_main.c | 15 ++-
.../net/ethernet/mellanox/mlx5/core/en_rx.c | 66 ++++++------
drivers/net/netdevsim/netdev.c | 18 ++--
drivers/net/netdevsim/netdevsim.h | 2 +-
include/linux/mm.h | 5 +-
include/linux/mm_types.h | 11 --
include/linux/skbuff.h | 14 +++
include/net/netmem.h | 101 ++++++++++--------
include/net/page_pool/helpers.h | 11 +-
net/core/page_pool.c | 97 +++++++++--------
16 files changed, 221 insertions(+), 201 deletions(-)
base-commit: f44092606a3f153bb7e6b277006b1f4a5b914cfc
--
2.17.1
On Fri, May 23, 2025 at 12:36 PM Byungchul Park <byungchul@sk.com> wrote:
>
Hi Byungchul,
Thanks a lot for this work!
> The MM subsystem is trying to reduce struct page to a single pointer.
> The first step towards that is splitting struct page by its individual
> users, as has already been done with folio and slab. This patchset does
> that for netmem which is used for page pools.
>
> Matthew Wilcox tried and stopped the same work, you can see in:
>
> https://lore.kernel.org/linux-mm/20230111042214.907030-1-willy@infradead.org/
>
> Mina Almasry already has done a lot fo prerequisite works by luck, he
> said :). I stacked my patches on the top of his work e.i. netmem.
>
> I focused on removing the page pool members in struct page this time,
> not moving the allocation code of page pool from net to mm. It can be
> done later if needed.
>
> My rfc version of this work is:
>
> https://lore.kernel.org/all/20250509115126.63190-1-byungchul@sk.com/
>
> There are still a lot of works to do, to remove the dependency on struct
> page in the network subsystem. I will continue to work on this after
> this base patchset is merged.
There is a compile failure.
In file included from drivers/net/ethernet/intel/libeth/rx.c:4:
./include/net/libeth/rx.h: In function ‘libeth_rx_sync_for_cpu’:
./include/net/libeth/rx.h:140:40: error: ‘struct page’ has no member named ‘pp’
140 | page_pool_dma_sync_for_cpu(page->pp, page, fqe->offset, len);
| ^~
drivers/net/ethernet/intel/libeth/rx.c: In function ‘libeth_rx_recycle_slow’:
drivers/net/ethernet/intel/libeth/rx.c:210:38: error: ‘struct page’
has no member named ‘pp’
210 | page_pool_recycle_direct(page->pp, page);
| ^~
make[7]: *** [scripts/Makefile.build:203:
drivers/net/ethernet/intel/libeth/rx.o] Error 1
make[6]: *** [scripts/Makefile.build:461:
drivers/net/ethernet/intel/libeth] Error 2
make[5]: *** [scripts/Makefile.build:461: drivers/net/ethernet/intel] Error 2
make[5]: *** Waiting for unfinished jobs....
There are page->pp usecases in drivers/net
./drivers/net/ethernet/marvell/octeontx2/nic/otx2_txrx.c:1574:
} else if (page->pp) {
./drivers/net/ethernet/freescale/fec_main.c:1046:
page_pool_put_page(page->pp, page, 0, false);
./drivers/net/ethernet/freescale/fec_main.c:1584:
page_pool_put_page(page->pp, page, 0, true);
./drivers/net/ethernet/freescale/fec_main.c:3351:
page_pool_put_page(page->pp, page, 0, false);
./drivers/net/ethernet/ti/icssg/icssg_prueth_sr1.c:370:
page_pool_recycle_direct(page->pp, page);
./drivers/net/ethernet/ti/icssg/icssg_prueth_sr1.c:395:
page_pool_recycle_direct(page->pp, page);
./drivers/net/ethernet/ti/icssg/icssg_common.c:111:
page_pool_recycle_direct(page->pp, swdata->data.page);
./drivers/net/ethernet/intel/idpf/idpf_txrx.c:389:
page_pool_put_full_page(rx_buf->page->pp, rx_buf->page, false);
./drivers/net/ethernet/intel/idpf/idpf_txrx.c:3254: u32 hr =
rx_buf->page->pp->p.offset;
./drivers/net/ethernet/intel/idpf/idpf_txrx.c:3286: dst =
page_address(hdr->page) + hdr->offset + hdr->page->pp->p.offset;
./drivers/net/ethernet/intel/idpf/idpf_txrx.c:3287: src =
page_address(buf->page) + buf->offset + buf->page->pp->p.offset;
./drivers/net/ethernet/intel/idpf/idpf_txrx.c:3305: u32 hr =
buf->page->pp->p.offset;
./drivers/net/ethernet/intel/libeth/rx.c:210:
page_pool_recycle_direct(page->pp, page);
./drivers/net/ethernet/intel/iavf/iavf_txrx.c:1200: u32 hr =
rx_buffer->page->pp->p.offset;
./drivers/net/ethernet/intel/iavf/iavf_txrx.c:1217: u32 hr =
rx_buffer->page->pp->p.offset;
./drivers/net/wireless/mediatek/mt76/mt76.h:1800:
page_pool_put_full_page(page->pp, page, allow_direct);
./include/net/libeth/rx.h:140: page_pool_dma_sync_for_cpu(page->pp,
page, fqe->offset, len);
Thanks a lot!
Taehee Yoo
>
> ---
>
> Changes from rfc:
> 1. Rebase on net-next's main branch
> https://git.kernel.org/pub/scm/linux/kernel/git/netdev/net-next.git/
> 2. Fix a build error reported by kernel test robot
> https://lore.kernel.org/all/202505100932.uzAMBW1y-lkp@intel.com/
> 3. Add given 'Reviewed-by's, thanks to Mina and Ilias
> 4. Do static_assert() on the size of struct netmem_desc instead
> of placing place-holder in struct page, feedbacked by Matthew
> 5. Do struct_group_tagged(netmem_desc) on struct net_iov instead
> of wholly renaming it to strcut netmem_desc, feedbacked by
> Mina and Pavel
>
> Byungchul Park (18):
> netmem: introduce struct netmem_desc struct_group_tagged()'ed on
> struct net_iov
> netmem: introduce netmem alloc APIs to wrap page alloc APIs
> page_pool: use netmem alloc/put APIs in __page_pool_alloc_page_order()
> page_pool: rename __page_pool_alloc_page_order() to
> __page_pool_alloc_large_netmem()
> page_pool: use netmem alloc/put APIs in __page_pool_alloc_pages_slow()
> page_pool: rename page_pool_return_page() to page_pool_return_netmem()
> page_pool: use netmem put API in page_pool_return_netmem()
> page_pool: rename __page_pool_release_page_dma() to
> __page_pool_release_netmem_dma()
> page_pool: rename __page_pool_put_page() to __page_pool_put_netmem()
> page_pool: rename __page_pool_alloc_pages_slow() to
> __page_pool_alloc_netmems_slow()
> mlx4: use netmem descriptor and APIs for page pool
> page_pool: use netmem APIs to access page->pp_magic in
> page_pool_page_is_pp()
> mlx5: use netmem descriptor and APIs for page pool
> netmem: use _Generic to cover const casting for page_to_netmem()
> netmem: remove __netmem_get_pp()
> page_pool: make page_pool_get_dma_addr() just wrap
> page_pool_get_dma_addr_netmem()
> netdevsim: use netmem descriptor and APIs for page pool
> mm, netmem: remove the page pool members in struct page
>
> drivers/net/ethernet/mellanox/mlx4/en_rx.c | 46 ++++----
> drivers/net/ethernet/mellanox/mlx4/en_tx.c | 8 +-
> drivers/net/ethernet/mellanox/mlx4/mlx4_en.h | 4 +-
> drivers/net/ethernet/mellanox/mlx5/core/en.h | 4 +-
> .../net/ethernet/mellanox/mlx5/core/en/xdp.c | 18 ++--
> .../net/ethernet/mellanox/mlx5/core/en/xdp.h | 2 +-
> .../net/ethernet/mellanox/mlx5/core/en_main.c | 15 ++-
> .../net/ethernet/mellanox/mlx5/core/en_rx.c | 66 ++++++------
> drivers/net/netdevsim/netdev.c | 18 ++--
> drivers/net/netdevsim/netdevsim.h | 2 +-
> include/linux/mm.h | 5 +-
> include/linux/mm_types.h | 11 --
> include/linux/skbuff.h | 14 +++
> include/net/netmem.h | 101 ++++++++++--------
> include/net/page_pool/helpers.h | 11 +-
> net/core/page_pool.c | 97 +++++++++--------
> 16 files changed, 221 insertions(+), 201 deletions(-)
>
>
> base-commit: f44092606a3f153bb7e6b277006b1f4a5b914cfc
> --
> 2.17.1
>
>
On Fri, May 23, 2025 at 03:20:27PM +0900, Taehee Yoo wrote:
> On Fri, May 23, 2025 at 12:36 PM Byungchul Park <byungchul@sk.com> wrote:
> >
>
> Hi Byungchul,
> Thanks a lot for this work!
>
> > The MM subsystem is trying to reduce struct page to a single pointer.
> > The first step towards that is splitting struct page by its individual
> > users, as has already been done with folio and slab. This patchset does
> > that for netmem which is used for page pools.
> >
> > Matthew Wilcox tried and stopped the same work, you can see in:
> >
> > https://lore.kernel.org/linux-mm/20230111042214.907030-1-willy@infradead.org/
> >
> > Mina Almasry already has done a lot fo prerequisite works by luck, he
> > said :). I stacked my patches on the top of his work e.i. netmem.
> >
> > I focused on removing the page pool members in struct page this time,
> > not moving the allocation code of page pool from net to mm. It can be
> > done later if needed.
> >
> > My rfc version of this work is:
> >
> > https://lore.kernel.org/all/20250509115126.63190-1-byungchul@sk.com/
> >
> > There are still a lot of works to do, to remove the dependency on struct
> > page in the network subsystem. I will continue to work on this after
> > this base patchset is merged.
>
> There is a compile failure.
Thanks a lot. I will fix it.
Byungchul
>
> In file included from drivers/net/ethernet/intel/libeth/rx.c:4:
> ./include/net/libeth/rx.h: In function ‘libeth_rx_sync_for_cpu’:
> ./include/net/libeth/rx.h:140:40: error: ‘struct page’ has no member named ‘pp’
> 140 | page_pool_dma_sync_for_cpu(page->pp, page, fqe->offset, len);
> | ^~
> drivers/net/ethernet/intel/libeth/rx.c: In function ‘libeth_rx_recycle_slow’:
> drivers/net/ethernet/intel/libeth/rx.c:210:38: error: ‘struct page’
> has no member named ‘pp’
> 210 | page_pool_recycle_direct(page->pp, page);
> | ^~
> make[7]: *** [scripts/Makefile.build:203:
> drivers/net/ethernet/intel/libeth/rx.o] Error 1
> make[6]: *** [scripts/Makefile.build:461:
> drivers/net/ethernet/intel/libeth] Error 2
> make[5]: *** [scripts/Makefile.build:461: drivers/net/ethernet/intel] Error 2
> make[5]: *** Waiting for unfinished jobs....
>
> There are page->pp usecases in drivers/net
> ./drivers/net/ethernet/marvell/octeontx2/nic/otx2_txrx.c:1574:
> } else if (page->pp) {
> ./drivers/net/ethernet/freescale/fec_main.c:1046:
> page_pool_put_page(page->pp, page, 0, false);
> ./drivers/net/ethernet/freescale/fec_main.c:1584:
> page_pool_put_page(page->pp, page, 0, true);
> ./drivers/net/ethernet/freescale/fec_main.c:3351:
> page_pool_put_page(page->pp, page, 0, false);
> ./drivers/net/ethernet/ti/icssg/icssg_prueth_sr1.c:370:
> page_pool_recycle_direct(page->pp, page);
> ./drivers/net/ethernet/ti/icssg/icssg_prueth_sr1.c:395:
> page_pool_recycle_direct(page->pp, page);
> ./drivers/net/ethernet/ti/icssg/icssg_common.c:111:
> page_pool_recycle_direct(page->pp, swdata->data.page);
> ./drivers/net/ethernet/intel/idpf/idpf_txrx.c:389:
> page_pool_put_full_page(rx_buf->page->pp, rx_buf->page, false);
> ./drivers/net/ethernet/intel/idpf/idpf_txrx.c:3254: u32 hr =
> rx_buf->page->pp->p.offset;
> ./drivers/net/ethernet/intel/idpf/idpf_txrx.c:3286: dst =
> page_address(hdr->page) + hdr->offset + hdr->page->pp->p.offset;
> ./drivers/net/ethernet/intel/idpf/idpf_txrx.c:3287: src =
> page_address(buf->page) + buf->offset + buf->page->pp->p.offset;
> ./drivers/net/ethernet/intel/idpf/idpf_txrx.c:3305: u32 hr =
> buf->page->pp->p.offset;
> ./drivers/net/ethernet/intel/libeth/rx.c:210:
> page_pool_recycle_direct(page->pp, page);
> ./drivers/net/ethernet/intel/iavf/iavf_txrx.c:1200: u32 hr =
> rx_buffer->page->pp->p.offset;
> ./drivers/net/ethernet/intel/iavf/iavf_txrx.c:1217: u32 hr =
> rx_buffer->page->pp->p.offset;
> ./drivers/net/wireless/mediatek/mt76/mt76.h:1800:
> page_pool_put_full_page(page->pp, page, allow_direct);
> ./include/net/libeth/rx.h:140: page_pool_dma_sync_for_cpu(page->pp,
> page, fqe->offset, len);
>
> Thanks a lot!
> Taehee Yoo
>
> >
> > ---
> >
> > Changes from rfc:
> > 1. Rebase on net-next's main branch
> > https://git.kernel.org/pub/scm/linux/kernel/git/netdev/net-next.git/
> > 2. Fix a build error reported by kernel test robot
> > https://lore.kernel.org/all/202505100932.uzAMBW1y-lkp@intel.com/
> > 3. Add given 'Reviewed-by's, thanks to Mina and Ilias
> > 4. Do static_assert() on the size of struct netmem_desc instead
> > of placing place-holder in struct page, feedbacked by Matthew
> > 5. Do struct_group_tagged(netmem_desc) on struct net_iov instead
> > of wholly renaming it to strcut netmem_desc, feedbacked by
> > Mina and Pavel
> >
> > Byungchul Park (18):
> > netmem: introduce struct netmem_desc struct_group_tagged()'ed on
> > struct net_iov
> > netmem: introduce netmem alloc APIs to wrap page alloc APIs
> > page_pool: use netmem alloc/put APIs in __page_pool_alloc_page_order()
> > page_pool: rename __page_pool_alloc_page_order() to
> > __page_pool_alloc_large_netmem()
> > page_pool: use netmem alloc/put APIs in __page_pool_alloc_pages_slow()
> > page_pool: rename page_pool_return_page() to page_pool_return_netmem()
> > page_pool: use netmem put API in page_pool_return_netmem()
> > page_pool: rename __page_pool_release_page_dma() to
> > __page_pool_release_netmem_dma()
> > page_pool: rename __page_pool_put_page() to __page_pool_put_netmem()
> > page_pool: rename __page_pool_alloc_pages_slow() to
> > __page_pool_alloc_netmems_slow()
> > mlx4: use netmem descriptor and APIs for page pool
> > page_pool: use netmem APIs to access page->pp_magic in
> > page_pool_page_is_pp()
> > mlx5: use netmem descriptor and APIs for page pool
> > netmem: use _Generic to cover const casting for page_to_netmem()
> > netmem: remove __netmem_get_pp()
> > page_pool: make page_pool_get_dma_addr() just wrap
> > page_pool_get_dma_addr_netmem()
> > netdevsim: use netmem descriptor and APIs for page pool
> > mm, netmem: remove the page pool members in struct page
> >
> > drivers/net/ethernet/mellanox/mlx4/en_rx.c | 46 ++++----
> > drivers/net/ethernet/mellanox/mlx4/en_tx.c | 8 +-
> > drivers/net/ethernet/mellanox/mlx4/mlx4_en.h | 4 +-
> > drivers/net/ethernet/mellanox/mlx5/core/en.h | 4 +-
> > .../net/ethernet/mellanox/mlx5/core/en/xdp.c | 18 ++--
> > .../net/ethernet/mellanox/mlx5/core/en/xdp.h | 2 +-
> > .../net/ethernet/mellanox/mlx5/core/en_main.c | 15 ++-
> > .../net/ethernet/mellanox/mlx5/core/en_rx.c | 66 ++++++------
> > drivers/net/netdevsim/netdev.c | 18 ++--
> > drivers/net/netdevsim/netdevsim.h | 2 +-
> > include/linux/mm.h | 5 +-
> > include/linux/mm_types.h | 11 --
> > include/linux/skbuff.h | 14 +++
> > include/net/netmem.h | 101 ++++++++++--------
> > include/net/page_pool/helpers.h | 11 +-
> > net/core/page_pool.c | 97 +++++++++--------
> > 16 files changed, 221 insertions(+), 201 deletions(-)
> >
> >
> > base-commit: f44092606a3f153bb7e6b277006b1f4a5b914cfc
> > --
> > 2.17.1
> >
> >
Hi Byungchul, On Fri, 23 May 2025 12:25:51 +0900 Byungchul Park <byungchul@sk.com> wrote: > The MM subsystem is trying to reduce struct page to a single pointer. > The first step towards that is splitting struct page by its individual > users, as has already been done with folio and slab. This patchset does > that for netmem which is used for page pools. I found checkpatch.pl outputs some complaints to a few patches of this patch series. Most warnings and errors look not critical or even unnecessary, but seems some of those would better to be reduced in my opinion. Thanks, SJ [...]
On Fri, May 23, 2025 at 10:47:48AM -0700, SeongJae Park wrote: > Hi Byungchul, > > On Fri, 23 May 2025 12:25:51 +0900 Byungchul Park <byungchul@sk.com> wrote: > > > The MM subsystem is trying to reduce struct page to a single pointer. > > The first step towards that is splitting struct page by its individual > > users, as has already been done with folio and slab. This patchset does > > that for netmem which is used for page pools. > > I found checkpatch.pl outputs some complaints to a few patches of this > patch series. Most warnings and errors look not critical or even unnecessary, > but seems some of those would better to be reduced in my opinion. Thanks for the suggestion. I will check it. Byungchul > > > Thanks, > SJ > > [...]
© 2016 - 2025 Red Hat, Inc.