As more drivers begin to use the frag API, update the
document about how to decide which API to use for the
driver author.

Also it seems there is a similar document in page_pool.h,
so remove it to avoid the duplication.

Signed-off-by: Yunsheng Lin <linyunsheng@huawei.com>
CC: Lorenzo Bianconi <lorenzo@kernel.org>
CC: Alexander Duyck <alexander.duyck@gmail.com>
CC: Liang Chen <liangchen.linux@gmail.com>
CC: Alexander Lobakin <aleksander.lobakin@intel.com>
---
 Documentation/networking/page_pool.rst | 34 ++++++++++++++++++++++----
 include/net/page_pool.h                | 22 -----------------
 2 files changed, 29 insertions(+), 27 deletions(-)

diff --git a/Documentation/networking/page_pool.rst b/Documentation/networking/page_pool.rst
index 873efd97f822..18b13d659c98 100644
--- a/Documentation/networking/page_pool.rst
+++ b/Documentation/networking/page_pool.rst
@@ -4,12 +4,27 @@
 Page Pool API
 =============
 
-The page_pool allocator is optimized for the XDP mode that uses one frame
-per-page, but it can fallback on the regular page allocator APIs.
+The page_pool allocator is optimized for recycling page or page frag used by skb
+packet and xdp frame.
 
-Basic use involves replacing alloc_pages() calls with the
-page_pool_alloc_pages() call.  Drivers should use page_pool_dev_alloc_pages()
-replacing dev_alloc_pages().
+Basic use involves replacing napi_alloc_frag() and alloc_pages() calls with
+page_pool_cache_alloc() and page_pool_alloc(), which allocate memory with or
+without page splitting depending on the requested memory size.
+
+If the driver knows that it always requires full pages or its allocates are
+always smaller than half a page, it can use one of the more specific API calls:
+
+1. page_pool_alloc_pages(): allocate memory without page splitting when driver
+   knows that the memory it need is always bigger than half of the page
+   allocated from page pool. There is no cache line dirtying for 'struct page'
+   when a page is recycled back to the page pool.
+
+2. page_pool_alloc_frag(): allocate memory with page splitting when driver knows
+   that the memory it need is always smaller than or equal to half of the page
+   allocated from page pool. Page splitting enables memory saving and thus avoid
+   TLB/cache miss for data access, but there also is some cost to implement page
+   splitting, mainly some cache line dirtying/bouncing for 'struct page' and
+   atomic operation for page->pp_frag_count.
 
 API keeps track of in-flight pages, in order to let API user know
 when it is safe to free a page_pool object.  Thus, API users
@@ -93,6 +108,15 @@ a page will cause no race conditions is enough.
 * page_pool_dev_alloc_pages(): Get a page from the page allocator or page_pool
   caches.
 
+* page_pool_dev_alloc_frag(): Get a page frag from the page allocator or
+  page_pool caches.
+
+* page_pool_dev_alloc(): Get a page or page frag from the page allocator or
+  page_pool caches.
+
+* page_pool_dev_cache_alloc(): Get a cache from the page allocator or page_pool
+  caches.
+
 * page_pool_get_dma_addr(): Retrieve the stored DMA address.
 
 * page_pool_get_dma_dir(): Retrieve the stored DMA direction.
diff --git a/include/net/page_pool.h b/include/net/page_pool.h
index e9fb95d62ed5..2b7db9992fc0 100644
--- a/include/net/page_pool.h
+++ b/include/net/page_pool.h
@@ -5,28 +5,6 @@
  *	Copyright (C) 2016 Red Hat, Inc.
  */
 
-/**
- * DOC: page_pool allocator
- *
- * This page_pool allocator is optimized for the XDP mode that
- * uses one-frame-per-page, but have fallbacks that act like the
- * regular page allocator APIs.
- *
- * Basic use involve replacing alloc_pages() calls with the
- * page_pool_alloc_pages() call.  Drivers should likely use
- * page_pool_dev_alloc_pages() replacing dev_alloc_pages().
- *
- * API keeps track of in-flight pages, in-order to let API user know
- * when it is safe to dealloactor page_pool object.  Thus, API users
- * must make sure to call page_pool_release_page() when a page is
- * "leaving" the page_pool.  Or call page_pool_put_page() where
- * appropiate.  For maintaining correct accounting.
- *
- * API user must only call page_pool_put_page() once on a page, as it
- * will either recycle the page, or in case of elevated refcnt, it
- * will release the DMA mapping and in-flight state accounting.  We
- * hope to lift this requirement in the future.
- */
 #ifndef _NET_PAGE_POOL_H
 #define _NET_PAGE_POOL_H
 
-- 
2.33.0

Hi--

On 6/29/23 05:02, Yunsheng Lin wrote:
> As more drivers begin to use the frag API, update the
> document about how to decide which API to use for the
> driver author.
> 
> Also it seems there is a similar document in page_pool.h,
> so remove it to avoid the duplication.
> 
> Signed-off-by: Yunsheng Lin <linyunsheng@huawei.com>
> CC: Lorenzo Bianconi <lorenzo@kernel.org>
> CC: Alexander Duyck <alexander.duyck@gmail.com>
> CC: Liang Chen <liangchen.linux@gmail.com>
> CC: Alexander Lobakin <aleksander.lobakin@intel.com>
> ---
>  Documentation/networking/page_pool.rst | 34 ++++++++++++++++++++++----
>  include/net/page_pool.h                | 22 -----------------
>  2 files changed, 29 insertions(+), 27 deletions(-)
> 
> diff --git a/Documentation/networking/page_pool.rst b/Documentation/networking/page_pool.rst
> index 873efd97f822..18b13d659c98 100644
> --- a/Documentation/networking/page_pool.rst
> +++ b/Documentation/networking/page_pool.rst
> @@ -4,12 +4,27 @@
>  Page Pool API
>  =============
>  
> -The page_pool allocator is optimized for the XDP mode that uses one frame
> -per-page, but it can fallback on the regular page allocator APIs.
> +The page_pool allocator is optimized for recycling page or page frag used by skb
> +packet and xdp frame.

That sentence could use some adjectives. Choose singular or plural:

> +The page_pool allocator is optimized for recycling a page or page frag used by an skb
> +packet or xdp frame.

or

> +The page_pool allocator is optimized for recycling pages or page frags used by skb
> +packets or xdp frames.

Now that I have written them, I prefer the latter one (plural). FWIW.

>  
> -Basic use involves replacing alloc_pages() calls with the
> -page_pool_alloc_pages() call.  Drivers should use page_pool_dev_alloc_pages()
> -replacing dev_alloc_pages().
> +Basic use involves replacing napi_alloc_frag() and alloc_pages() calls with
> +page_pool_cache_alloc() and page_pool_alloc(), which allocate memory with or
> +without page splitting depending on the requested memory size.
> +
> +If the driver knows that it always requires full pages or its allocates are

                                                                 allocations are

> +always smaller than half a page, it can use one of the more specific API calls:
> +
> +1. page_pool_alloc_pages(): allocate memory without page splitting when driver
> +   knows that the memory it need is always bigger than half of the page
> +   allocated from page pool. There is no cache line dirtying for 'struct page'
> +   when a page is recycled back to the page pool.
> +
> +2. page_pool_alloc_frag(): allocate memory with page splitting when driver knows
> +   that the memory it need is always smaller than or equal to half of the page
> +   allocated from page pool. Page splitting enables memory saving and thus avoid

                                                                     and thus avoids

> +   TLB/cache miss for data access, but there also is some cost to implement page
> +   splitting, mainly some cache line dirtying/bouncing for 'struct page' and
> +   atomic operation for page->pp_frag_count.
>  
>  API keeps track of in-flight pages, in order to let API user know
>  when it is safe to free a page_pool object.  Thus, API users
> @@ -93,6 +108,15 @@ a page will cause no race conditions is enough.
>  * page_pool_dev_alloc_pages(): Get a page from the page allocator or page_pool
>    caches.
>  
> +* page_pool_dev_alloc_frag(): Get a page frag from the page allocator or
> +  page_pool caches.
> +
> +* page_pool_dev_alloc(): Get a page or page frag from the page allocator or
> +  page_pool caches.
> +
> +* page_pool_dev_cache_alloc(): Get a cache from the page allocator or page_pool
> +  caches.
> +
>  * page_pool_get_dma_addr(): Retrieve the stored DMA address.
>  
>  * page_pool_get_dma_dir(): Retrieve the stored DMA direction.

Thanks for adding the documentation.

-- 
~Randy