From nobody Sun Feb 8 04:51:31 2026 Received: from invmail4.hynix.com (exvmail4.skhynix.com [166.125.252.92]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 3B2B61411DE; Mon, 3 Nov 2025 07:51:25 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=166.125.252.92 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1762156292; cv=none; b=LxZu1v9Dj72t2bmQdUkGBgFuMMKzVne1rp3dv5fQgg9rJia1ByfEy0Ovc1PvHS2SbTaFR+u0RBcIjuUWsez5tDU9mhiER3m/RtGZUVzXvUdnjFPhM94+cmbmIUAPI+YqPMbQ27N9oI+2NXlpoHSy4rv+tMKCh1rohQfjinDfxao= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1762156292; c=relaxed/simple; bh=hfqpfBfHxBxqAMWf51/e9m+GyU8Dfc8f0Ko1up50BfE=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References; b=Z+Hd882BjU6FpZa3tjEiYdkx5PBuWCFsh4VIIqYDD7BIguFUf8dAo95cnQfNM0WMpaWHd7LfZH4gAhQW3jivXa/f9q++IP8RjGtaFuycoZmp+rUDlvnlFpPaU4qPGIuQXZqyfybVG+e3Ca8fddCoTBK70FzHqW0JChzCnjWC7K8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com; spf=pass smtp.mailfrom=sk.com; arc=none smtp.client-ip=166.125.252.92 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=sk.com X-AuditID: a67dfc5b-c45ff70000001609-ff-69085ef70e8f From: Byungchul Park To: linux-mm@kvack.org, netdev@vger.kernel.org Cc: linux-kernel@vger.kernel.org, kernel_team@skhynix.com, harry.yoo@oracle.com, ast@kernel.org, daniel@iogearbox.net, davem@davemloft.net, kuba@kernel.org, hawk@kernel.org, john.fastabend@gmail.com, sdf@fomichev.me, saeedm@nvidia.com, leon@kernel.org, tariqt@nvidia.com, mbloch@nvidia.com, andrew+netdev@lunn.ch, edumazet@google.com, pabeni@redhat.com, akpm@linux-foundation.org, david@redhat.com, lorenzo.stoakes@oracle.com, Liam.Howlett@oracle.com, vbabka@suse.cz, rppt@kernel.org, surenb@google.com, mhocko@suse.com, horms@kernel.org, jackmanb@google.com, hannes@cmpxchg.org, ziy@nvidia.com, ilias.apalodimas@linaro.org, willy@infradead.org, brauner@kernel.org, kas@kernel.org, yuzhao@google.com, usamaarif642@gmail.com, baolin.wang@linux.alibaba.com, almasrymina@google.com, toke@redhat.com, asml.silence@gmail.com, bpf@vger.kernel.org, linux-rdma@vger.kernel.org, sfr@canb.auug.org.au, dw@davidwei.uk, ap420073@gmail.com, dtatulea@nvidia.com Subject: [RFC mm v5 1/2] page_pool: check nmdesc->pp to see its usage as page pool for net_iov not page-backed Date: Mon, 3 Nov 2025 16:51:07 +0900 Message-Id: <20251103075108.26437-2-byungchul@sk.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20251103075108.26437-1-byungchul@sk.com> References: <20251103075108.26437-1-byungchul@sk.com> X-Brightmail-Tracker: H4sIAAAAAAAAAzXSb0gTcRwG8H53t7tzOTqW5qWRMIrASE2tvkGJIMSvIgh6VUE53JFXumTL pYG0apJlmlmW6QTtj82pWLN0LldrM80yCkudVP4vo8zMfzk1bBq9+8DzfJ9XX5aUZ0gCWVF9 QtColYkKWkpJf/iWbvh9iBXDPxX4g7G6koaK6VS412uVgKdyiACjuRbBhOcDA/P2JgTjjc00 fHeNIbhdOkWC8Y2BgsnqGRLqbUMIvhVU0fC5qZ+BCsse6Cn7QkHD+ToS+i+/oCHbMEuC3TPC wFmryTtco2fgbW2OBK7N3CWhTt/LwDubkYbuynkJfHFmU9BSWE7BaH4jCT05MdBUsgKmXg0j aKyuI2DqUjEN7TdtBDyytzNwta2EhgFDD4I2Vz8F+XOZNBSdyUEwO+2dHMmdkEDR824mJhSf cbtp7Br+SeKH5V0E7ii4QmH3k5cEri/8xOASSwquMYXgi+42ElvMF2hsGctj8MeOBhq/KJil cH3fVlxvHSdw9rkReq//Aek2lZAo6gRNWHScNGHSOk0lvw9Kbfp1h9KjwRUXkQ/Lc1H8ecco 89+u/kx6wTS3jne7PeSC/bhw3pQ/4bWUJbkslv/w5MFisJxT8+Nd9kVT3Fr+a187tWAZt4kv zXLS/0aD+Yr7jsWOD7eZt7eWEQuWezvmh4bFUZ5zsHzDn9fEv4OV/DOTm8pFshK0xIzkolqX pBQTo0IT0tRiamj88SQL8j5HWfrcQSsae7vPiTgWKXxlvfGMKJcoddq0JCfiWVLhJ+syUKJc plKmnRI0xw9rUhIFrRMFsZQiQBYxdVIl544oTwjHBCFZ0PxPCdYnUI8yVq/KIxMC4pfoHN92 GiPiM6W7zcpy/10hP/qiTjuyRm/tCFb1SIa3momc2uT9nZE3tumwQttS7HkfeW9+YF2k+LT5 +tItsdHLgh5v0LoqhvIybMGDN7pjYp8ubz+5ptOuXa/YblN/3dWrzy+Kk6iC3h0++tmvqkbb WpoVEd6RHqagtAnKjSGkRqv8C9711w0YAwAA X-Brightmail-Tracker: H4sIAAAAAAAAAzXRa0hTYRwG8N6ds/ccZ5PDMjtoEY3EEtKkrH9m0ZfoIBjdi6B01CFP6qxN TaNCc5BmLrVWZkpqZd5Cm7WLOKnN7KKQeMmZ5a20m1le84Y1jb794Hl4vjw0Iesn3WlBGc2r lIoIOZaQkp2bk9b8PkoLa40nIae8DEPpRBw86DaJYbLsswhySgwIRic7KPhjqUMwUvsCw3fb MIK7+eME5LzRkDBWPkWAueozgm9ZDzH01fVSUKoPhq7CfhKqLxkJ6L36EkOaZpoAy+QgBRdN RY7hygQKbLmvxNBo0Irh+tR9AowJ3RQ0V+Vg6Cz7I4Z+axoJr7KLSfilqyWgS7sN6vLcYLx+ AEFtuVEE41dyMbTeqhLBE0srBdea8jB81HQhaLL1kqCbScZwO1GLYHrCMTmYPiqG2887qW2+ XKLdjjnbwE+Ce1zcLuLeZmWQnL3mtYgzZ3+guDx9DFdZ5M1dtjcRnL4kBXP64UyKe/+2GnMv s6ZJztyziTObRkRcWtIg3uV2WBJ4nI8QYnmV79ZQSdiYaYI81eIRVzd0j0xAn9wuIyeaZdaz tt5kPGfMeLF2+yQxZ1dmLVukG3VYQhNMKs121DyaDxYxSnak3TJvkvFkv/S0knOWMv5sfqoV /xtdzpZWPJ3vODEbWEtDoWjOMken5LGGSEeSPLSgBLkKythIhRDh76MOD4tXCnE+x6Ii9cjx fuH5mQwTGm3eYUUMjeQLpd3HKEEmVsSq4yOtiKUJuau0XUMKMulxRfxZXhUVooqJ4NVW5EGT 8iXSoIN8qIw5oYjmw3n+FK/6n4poJ/cE5KI7NBV0kvBKCjQcaeGl7+SNPhVts8taVukMG73c tWJ1wOr7yb/PZq+Oly/NFG4WnCvff+DJ3oGxNkr4keiyx7zYvq9ruu+0R+DKNhd2e2a+5Wt6 2ooLdz1t2pAr6y5h3wZnv6gthcGzBS6Rht1nhooDntWv2G/sTFHdCM24lX7HWU6qwxR+3oRK rfgLYMXzOvkCAAA= X-CFilter-Loop: Reflected Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Currently, the condition 'page->pp_magic =3D=3D PP_SIGNATURE' is used to determine if a page belongs to a page pool. However, with the planned removal of ->pp_magic, we will instead leverage the page_type in struct page, such as PGTY_netpp, for this purpose. That works for page-backed network memory. However, for net_iov not page-backed, the identification cannot be based on the page_type. Instead, nmdesc->pp can be used to see if it belongs to a page pool, by making sure nmdesc->pp is NULL otherwise. For net_iov not page-backed, initialize it using nmdesc->pp =3D NULL in net_devmem_bind_dmabuf() and using kvmalloc_array(__GFP_ZERO) in io_zcrx_create_area() so that netmem_is_pp() can check if nmdesc->pp is !NULL to confirm its usage as page pool. Signed-off-by: Byungchul Park Reviewed-by: Mina Almasry Reviewed-by: Pavel Begunkov Reviewed-by: Toke H=C3=B8iland-J=C3=B8rgensen --- net/core/devmem.c | 1 + net/core/netmem_priv.h | 8 ++++++++ net/core/page_pool.c | 16 ++++++++++++++-- 3 files changed, 23 insertions(+), 2 deletions(-) diff --git a/net/core/devmem.c b/net/core/devmem.c index d9de31a6cc7f..f81b700f1fd1 100644 --- a/net/core/devmem.c +++ b/net/core/devmem.c @@ -291,6 +291,7 @@ net_devmem_bind_dmabuf(struct net_device *dev, niov =3D &owner->area.niovs[i]; niov->type =3D NET_IOV_DMABUF; niov->owner =3D &owner->area; + niov->desc.pp =3D NULL; page_pool_set_dma_addr_netmem(net_iov_to_netmem(niov), net_devmem_get_dma_addr(niov)); if (direction =3D=3D DMA_TO_DEVICE) diff --git a/net/core/netmem_priv.h b/net/core/netmem_priv.h index 23175cb2bd86..5561fd556bc5 100644 --- a/net/core/netmem_priv.h +++ b/net/core/netmem_priv.h @@ -22,6 +22,14 @@ static inline void netmem_clear_pp_magic(netmem_ref netm= em) =20 static inline bool netmem_is_pp(netmem_ref netmem) { + /* net_iov may be part of a page pool. For net_iov, ->pp in + * net_iov.desc can be used to determine if the pages belong to + * a page pool. Ensure that the ->pp either points to its page + * pool or is set to NULL if it does not. + */ + if (netmem_is_net_iov(netmem)) + return !!netmem_to_nmdesc(netmem)->pp; + return (netmem_get_pp_magic(netmem) & PP_MAGIC_MASK) =3D=3D PP_SIGNATURE; } =20 diff --git a/net/core/page_pool.c b/net/core/page_pool.c index 1a5edec485f1..2756b78754b0 100644 --- a/net/core/page_pool.c +++ b/net/core/page_pool.c @@ -699,7 +699,13 @@ s32 page_pool_inflight(const struct page_pool *pool, b= ool strict) void page_pool_set_pp_info(struct page_pool *pool, netmem_ref netmem) { netmem_set_pp(netmem, pool); - netmem_or_pp_magic(netmem, PP_SIGNATURE); + + /* For page-backed, pp_magic is used to identify if it's pp. + * For net_iov, it's ensured nmdesc->pp is non-NULL if it's pp + * and nmdesc->pp is NULL if it's not. + */ + if (!netmem_is_net_iov(netmem)) + netmem_or_pp_magic(netmem, PP_SIGNATURE); =20 /* Ensuring all pages have been split into one fragment initially: * page_pool_set_pp_info() is only called once for every page when it @@ -714,7 +720,13 @@ void page_pool_set_pp_info(struct page_pool *pool, net= mem_ref netmem) =20 void page_pool_clear_pp_info(netmem_ref netmem) { - netmem_clear_pp_magic(netmem); + /* For page-backed, pp_magic is used to identify if it's pp. + * For net_iov, it's ensured nmdesc->pp is non-NULL if it's pp + * and nmdesc->pp is NULL if it's not. + */ + if (!netmem_is_net_iov(netmem)) + netmem_clear_pp_magic(netmem); + netmem_set_pp(netmem, NULL); } =20 --=20 2.17.1 From nobody Sun Feb 8 04:51:31 2026 Received: from invmail4.hynix.com (exvmail4.skhynix.com [166.125.252.92]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 355361411DE; Mon, 3 Nov 2025 07:51:45 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=166.125.252.92 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1762156309; cv=none; b=da1wpnisq30793+dxCbf2hUnDEmZxV3/jGtrqtlNeojdM0523bD1m/duobWGfQQpMGu/qBqkNXQ/aZgi4HSnsQcJ6lawwa5lXLxgn13MSNuun21/nj6yDPIXIksJmYBcPBUgZDnrZiGxjyhUPrgDQ1WXURf8qPBRwI0shaz2CQc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1762156309; c=relaxed/simple; bh=HZcjv4SLWrRkO22qgp4GIaRAxK0/5cYFlWxwMRi41lk=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References; b=oJd6go75LbQh2+A0qorulfpCvU87rcvQxxVg6yc6jXMBNmEaM05XV5+FSan7u7Noq1q+AuODg0Qs1o+MRv8ls2esmcpCbP+pRoMXpdiLEJpnUkSuAEk95SjKxtiRxGGZdwsxiqA5NLID7D/8Gm8VxyiWfjTrXauoOTK5xmoSKs8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com; spf=pass smtp.mailfrom=sk.com; arc=none smtp.client-ip=166.125.252.92 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=sk.com X-AuditID: a67dfc5b-c45ff70000001609-5f-69085f0c2fe6 From: Byungchul Park To: linux-mm@kvack.org, netdev@vger.kernel.org Cc: linux-kernel@vger.kernel.org, kernel_team@skhynix.com, harry.yoo@oracle.com, ast@kernel.org, daniel@iogearbox.net, davem@davemloft.net, kuba@kernel.org, hawk@kernel.org, john.fastabend@gmail.com, sdf@fomichev.me, saeedm@nvidia.com, leon@kernel.org, tariqt@nvidia.com, mbloch@nvidia.com, andrew+netdev@lunn.ch, edumazet@google.com, pabeni@redhat.com, akpm@linux-foundation.org, david@redhat.com, lorenzo.stoakes@oracle.com, Liam.Howlett@oracle.com, vbabka@suse.cz, rppt@kernel.org, surenb@google.com, mhocko@suse.com, horms@kernel.org, jackmanb@google.com, hannes@cmpxchg.org, ziy@nvidia.com, ilias.apalodimas@linaro.org, willy@infradead.org, brauner@kernel.org, kas@kernel.org, yuzhao@google.com, usamaarif642@gmail.com, baolin.wang@linux.alibaba.com, almasrymina@google.com, toke@redhat.com, asml.silence@gmail.com, bpf@vger.kernel.org, linux-rdma@vger.kernel.org, sfr@canb.auug.org.au, dw@davidwei.uk, ap420073@gmail.com, dtatulea@nvidia.com Subject: [RFC mm v5 2/2] mm: introduce a new page type for page pool in page type Date: Mon, 3 Nov 2025 16:51:08 +0900 Message-Id: <20251103075108.26437-3-byungchul@sk.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20251103075108.26437-1-byungchul@sk.com> References: <20251103075108.26437-1-byungchul@sk.com> X-Brightmail-Tracker: H4sIAAAAAAAAAzWSbUhTYRTHe+69u/e6XNym1U0/WCMJjNTK4kQRURFPlGBEH8rCRl5ypFbz JY2CaUPL8q0yzRYY1ZzTUKeps810W8vemBnqlqZmZGTLwsxSBzYtv/0O58/vnA9/lpSqRQGs IjFZUCbK42W0mBJ/8727dnEMqwjvKWdBU11FQ+WfNCgfbBLBZNVnAjT6BgTjk70MzJjtCH7a ntHw1TqG4N7dCRI0DjUFv6qnSDA2f0YwUvKQhk/2IQYqDZEwoB2mwJTdSMJQfjsNueppEsyT owxkNum84joVAx0NeSK4MfWAhEbVIANvmzU09FfNiGDYkkvB89IKCn4U2UgYyNsO9rKlMPHS jcBW3UjAxNU7NHTdaibgkbmLgeudZTR8VA8g6LQOUVDkuUTD7Yw8BNN/vMrRgnER3H7az2wP xRlOJ42t7u8krq9wEbi7pJDCzpYXBDaWvmdwmSEF1+lCcI6zk8QG/WUaG8auMbiv20Tj9pJp Chs/bMbGpp8Ezr04SkctOSzeGivEK1IFZdi2Y+K4b65i4vQ7nOZqNJMqVLMlB/mwPBfB6z/V EfOcr/9BzjLNreadzsk59ufCeV3RuJfFLMldYfnellrvwLJ+3H4+qyNtNkNxwXx/wVN6liXc Rr73+j3ynzOIr6xpnWMfbhNvfqWduyX1ZvT16jknz7Wy/Ps37v9PLOfbdE6qAEnK0AI9kioS UxPkiviI0Lj0REVa6PFTCQbk7Yb2gie6CY11HLAgjkUyX8ngcUYhFclTk9ITLIhnSZm/xKWm FFJJrDz9nKA8FaNMiReSLCiQpWTLJOsnzsZKuRPyZOGkIJwWlPNbgvUJUKElrwOD/J2E/xPd a0dA6XDQoS87FrWP1LTZ8k2vlHv3rYjwXPJ0LnRFa0cK17LTUz3Lbg6sctvDznTHRDvOX9Pe dNW61w/tPWfxcxDFUVbPysepGtfurX0FYa17JKrfnoORa8j7UVmGda6d2drwI/lt33fVFoYF b9BnHtUl+5o+rpRRSXHydSGkMkn+F6OI+pUXAwAA X-Brightmail-Tracker: H4sIAAAAAAAAAzXRa0hTcRgG8P7nnJ1zXC2OS+pglLQIycyKtF4xxCTwX1TYp6AUHXnK4bxw pqaBOXNkWt6VzEsolum0tGnqxFnNS7OERDGmljMjlVIzNcksahZ9+z28D8+XlyXlk5Qzq4qK FcQopVpBSynpaZ/UvRtCWNX+4mwvKK2vo6H2ewI8GG+VwErdFAGl+mYESyujDPw29SBY7HpB w+fOBQSVFcsklL7WUfCt/gcJxrYpBJ+KHtLwsWeCgVrDKbBVTVLQntZCwkS2hYZM3SoJppU5 Bq61VtuHG7UMdJb1SqC/OUsCBT/uk9CiHWdgsK2UhrG63xKYNGdS0FtcQ8F8YRcJtiw/6Cnf DMuvZhB01bcQsHyrjIahO20EPDENMZA/UE7DB50NwUDnBAWFP2/QUJKShWD1u31yLmdJAiXd Y4zfPpxitdK4c+YLiZtqhgn8piiXwtaOlwQ2Fr9jcLkhDjdWu+EM6wCJDfp0GhsW8hj89k07 jS1FqxQ2vvfGxtZFAmemztGBm89Jj4QJalW8IO7zDZWGzw7fJmJGcMJwi4nUogafDOTA8pwn n62fJ9dMc6681bry107cfr66cMluKUtyN1l+tOOxPbDsJu4Mf70/Ya1Dcbv4sZxues0yzosf za8k/2268LUNz/7agTvEm/qqiDXL7R19k47MQdJytE6PnFRR8ZFKldrLQxMRnhilSvC4EB1p QPbvVyX9zG1FS4MBZsSxSLFBNn6BUcklynhNYqQZ8SypcJIN6yiVXBamTLwiiNEhYpxa0JjR VpZSbJGdOCuEyrlLylghQhBiBPH/lWAdnLXo6szu85XyiqD1zxn3oxYLk2zKCzusdXb3df0g ZriUbHMema4NPhhw8Wmgt21+4+ysecExtKbsuGPQZdFHf5c6pldu9/cueLKn11WdV5b2y/LV Nb5yTGMTtuiCj/pORx5xFANS0pqpzJ2e/v7pJx/twJJjXcaOJPfqkbiB5Ma+ewpKE6484EaK GuUffCXdMvkCAAA= X-CFilter-Loop: Reflected Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Currently, the condition 'page->pp_magic =3D=3D PP_SIGNATURE' is used to determine if a page belongs to a page pool. However, with the planned removal of ->pp_magic, we should instead leverage the page_type in struct page, such as PGTY_netpp, for this purpose. Introduce and use the page type APIs e.g. PageNetpp(), __SetPageNetpp(), and __ClearPageNetpp() instead, and remove the existing APIs accessing ->pp_magic e.g. page_pool_page_is_pp(), netmem_or_pp_magic(), and netmem_clear_pp_magic(). This work was inspired by the following link: [1] https://lore.kernel.org/all/582f41c0-2742-4400-9c81-0d46bf4e8314@gmail.= com/ While at it, move the sanity check for page pool to on free. Suggested-by: David Hildenbrand Co-developed-by: Pavel Begunkov Signed-off-by: Pavel Begunkov Signed-off-by: Byungchul Park Acked-by: David Hildenbrand Acked-by: Zi Yan Acked-by: Mina Almasry Acked-by: Toke H=C3=B8iland-J=C3=B8rgensen Reviewed-by: Pavel Begunkov --- .../net/ethernet/mellanox/mlx5/core/en/xdp.c | 2 +- include/linux/mm.h | 27 +++---------------- include/linux/page-flags.h | 6 +++++ include/net/netmem.h | 2 +- mm/page_alloc.c | 8 +++--- net/core/netmem_priv.h | 17 +++--------- net/core/page_pool.c | 14 +++++----- 7 files changed, 25 insertions(+), 51 deletions(-) diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en/xdp.c b/drivers/net= /ethernet/mellanox/mlx5/core/en/xdp.c index 5d51600935a6..def274f5c1ca 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en/xdp.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/en/xdp.c @@ -707,7 +707,7 @@ static void mlx5e_free_xdpsq_desc(struct mlx5e_xdpsq *s= q, xdpi =3D mlx5e_xdpi_fifo_pop(xdpi_fifo); page =3D xdpi.page.page; =20 - /* No need to check page_pool_page_is_pp() as we + /* No need to check PageNetpp() as we * know this is a page_pool page. */ page_pool_recycle_direct(pp_page_to_nmdesc(page)->pp, diff --git a/include/linux/mm.h b/include/linux/mm.h index a3f97c551ad8..081e365caa1a 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -4252,10 +4252,9 @@ int arch_lock_shadow_stack_status(struct task_struct= *t, unsigned long status); * DMA mapping IDs for page_pool * * When DMA-mapping a page, page_pool allocates an ID (from an xarray) and - * stashes it in the upper bits of page->pp_magic. We always want to be ab= le to - * unambiguously identify page pool pages (using page_pool_page_is_pp()). = Non-PP - * pages can have arbitrary kernel pointers stored in the same field as pp= _magic - * (since it overlaps with page->lru.next), so we must ensure that we cann= ot + * stashes it in the upper bits of page->pp_magic. Non-PP pages can have + * arbitrary kernel pointers stored in the same field as pp_magic (since + * it overlaps with page->lru.next), so we must ensure that we cannot * mistake a valid kernel pointer with any of the values we write into this * field. * @@ -4290,26 +4289,6 @@ int arch_lock_shadow_stack_status(struct task_struct= *t, unsigned long status); #define PP_DMA_INDEX_MASK GENMASK(PP_DMA_INDEX_BITS + PP_DMA_INDEX_SHIFT -= 1, \ PP_DMA_INDEX_SHIFT) =20 -/* Mask used for checking in page_pool_page_is_pp() below. page->pp_magic = is - * OR'ed with PP_SIGNATURE after the allocation in order to preserve bit 0= for - * the head page of compound page and bit 1 for pfmemalloc page, as well a= s the - * bits used for the DMA index. page_is_pfmemalloc() is checked in - * __page_pool_put_page() to avoid recycling the pfmemalloc page. - */ -#define PP_MAGIC_MASK ~(PP_DMA_INDEX_MASK | 0x3UL) - -#ifdef CONFIG_PAGE_POOL -static inline bool page_pool_page_is_pp(const struct page *page) -{ - return (page->pp_magic & PP_MAGIC_MASK) =3D=3D PP_SIGNATURE; -} -#else -static inline bool page_pool_page_is_pp(const struct page *page) -{ - return false; -} -#endif - #define PAGE_SNAPSHOT_FAITHFUL (1 << 0) #define PAGE_SNAPSHOT_PG_BUDDY (1 << 1) #define PAGE_SNAPSHOT_PG_IDLE (1 << 2) diff --git a/include/linux/page-flags.h b/include/linux/page-flags.h index 0091ad1986bf..edf5418c91dd 100644 --- a/include/linux/page-flags.h +++ b/include/linux/page-flags.h @@ -934,6 +934,7 @@ enum pagetype { PGTY_zsmalloc =3D 0xf6, PGTY_unaccepted =3D 0xf7, PGTY_large_kmalloc =3D 0xf8, + PGTY_netpp =3D 0xf9, =20 PGTY_mapcount_underflow =3D 0xff }; @@ -1078,6 +1079,11 @@ PAGE_TYPE_OPS(Zsmalloc, zsmalloc, zsmalloc) PAGE_TYPE_OPS(Unaccepted, unaccepted, unaccepted) FOLIO_TYPE_OPS(large_kmalloc, large_kmalloc) =20 +/* + * Marks page_pool allocated pages. + */ +PAGE_TYPE_OPS(Netpp, netpp, netpp) + /** * PageHuge - Determine if the page belongs to hugetlbfs * @page: The page to test. diff --git a/include/net/netmem.h b/include/net/netmem.h index 651e2c62d1dd..0ec4c7561081 100644 --- a/include/net/netmem.h +++ b/include/net/netmem.h @@ -260,7 +260,7 @@ static inline unsigned long netmem_pfn_trace(netmem_ref= netmem) */ #define pp_page_to_nmdesc(p) \ ({ \ - DEBUG_NET_WARN_ON_ONCE(!page_pool_page_is_pp(p)); \ + DEBUG_NET_WARN_ON_ONCE(!PageNetpp(p)); \ __pp_page_to_nmdesc(p); \ }) =20 diff --git a/mm/page_alloc.c b/mm/page_alloc.c index 600d9e981c23..01dd14123065 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -1041,7 +1041,6 @@ static inline bool page_expected_state(struct page *p= age, #ifdef CONFIG_MEMCG page->memcg_data | #endif - page_pool_page_is_pp(page) | (page->flags.f & check_flags))) return false; =20 @@ -1068,8 +1067,6 @@ static const char *page_bad_reason(struct page *page,= unsigned long flags) if (unlikely(page->memcg_data)) bad_reason =3D "page still charged to cgroup"; #endif - if (unlikely(page_pool_page_is_pp(page))) - bad_reason =3D "page_pool leak"; return bad_reason; } =20 @@ -1378,9 +1375,12 @@ __always_inline bool free_pages_prepare(struct page = *page, mod_mthp_stat(order, MTHP_STAT_NR_ANON, -1); folio->mapping =3D NULL; } - if (unlikely(page_has_type(page))) + if (unlikely(page_has_type(page))) { + /* networking expects to clear its page type before releasing */ + WARN_ON_ONCE(PageNetpp(page)); /* Reset the page_type (which overlays _mapcount) */ page->page_type =3D UINT_MAX; + } =20 if (is_check_pages_enabled()) { if (free_page_is_bad(page)) diff --git a/net/core/netmem_priv.h b/net/core/netmem_priv.h index 5561fd556bc5..664a9fe87c66 100644 --- a/net/core/netmem_priv.h +++ b/net/core/netmem_priv.h @@ -8,18 +8,6 @@ static inline unsigned long netmem_get_pp_magic(netmem_ref= netmem) return netmem_to_nmdesc(netmem)->pp_magic & ~PP_DMA_INDEX_MASK; } =20 -static inline void netmem_or_pp_magic(netmem_ref netmem, unsigned long pp_= magic) -{ - netmem_to_nmdesc(netmem)->pp_magic |=3D pp_magic; -} - -static inline void netmem_clear_pp_magic(netmem_ref netmem) -{ - WARN_ON_ONCE(netmem_to_nmdesc(netmem)->pp_magic & PP_DMA_INDEX_MASK); - - netmem_to_nmdesc(netmem)->pp_magic =3D 0; -} - static inline bool netmem_is_pp(netmem_ref netmem) { /* net_iov may be part of a page pool. For net_iov, ->pp in @@ -30,7 +18,10 @@ static inline bool netmem_is_pp(netmem_ref netmem) if (netmem_is_net_iov(netmem)) return !!netmem_to_nmdesc(netmem)->pp; =20 - return (netmem_get_pp_magic(netmem) & PP_MAGIC_MASK) =3D=3D PP_SIGNATURE; + /* For system memory, page type in struct page can be used to + * determine if the pages belong to a page pool. + */ + return PageNetpp(__netmem_to_page(netmem)); } =20 static inline void netmem_set_pp(netmem_ref netmem, struct page_pool *pool) diff --git a/net/core/page_pool.c b/net/core/page_pool.c index 2756b78754b0..c43a0f4479d4 100644 --- a/net/core/page_pool.c +++ b/net/core/page_pool.c @@ -700,12 +700,11 @@ void page_pool_set_pp_info(struct page_pool *pool, ne= tmem_ref netmem) { netmem_set_pp(netmem, pool); =20 - /* For page-backed, pp_magic is used to identify if it's pp. - * For net_iov, it's ensured nmdesc->pp is non-NULL if it's pp - * and nmdesc->pp is NULL if it's not. + /* For system memory, page type in struct page is used to + * determine if the pages belong to a page pool. */ if (!netmem_is_net_iov(netmem)) - netmem_or_pp_magic(netmem, PP_SIGNATURE); + __SetPageNetpp(__netmem_to_page(netmem)); =20 /* Ensuring all pages have been split into one fragment initially: * page_pool_set_pp_info() is only called once for every page when it @@ -720,12 +719,11 @@ void page_pool_set_pp_info(struct page_pool *pool, ne= tmem_ref netmem) =20 void page_pool_clear_pp_info(netmem_ref netmem) { - /* For page-backed, pp_magic is used to identify if it's pp. - * For net_iov, it's ensured nmdesc->pp is non-NULL if it's pp - * and nmdesc->pp is NULL if it's not. + /* For system memory, page type in struct page is used to + * determine if the pages belong to a page pool. */ if (!netmem_is_net_iov(netmem)) - netmem_clear_pp_magic(netmem); + __ClearPageNetpp(__netmem_to_page(netmem)); =20 netmem_set_pp(netmem, NULL); } --=20 2.17.1