fs/nfs/direct.c | 139 ++++++++++++++++++++------------------- fs/nfs/pagelist.c | 66 ++++++++++++++++--- fs/nfs/read.c | 2 +- fs/nfs/write.c | 2 +- include/linux/nfs_page.h | 8 ++- 5 files changed, 134 insertions(+), 83 deletions(-)
Modernize the NFS Direct I/O path as a preparatory step to enable PCI Peer-to-Peer DMA (P2PDMA) support. Following feedback on the initial RFC [1], the modernization and architectural changes are split into different series. Additionally, based on the discussion in the v2 [2] of this series, the migration of NFS Direct I/O to folios would be handled in a separate follow-up series. Currently, NFS O_DIRECT relies on the legacy iov_iter_get_pages_alloc2() API which does not support the pinning requirements for P2P memory. The implementation moves NFS to the modern iov_iter_extract_pages() API. Design ====== 1. Pin-Awareness Standard NFS requests use get_page() and put_page() for memory management. However, memory extracted via iov_iter_extract_pages() requires explicit pinning. Introduce a PG_PINNED flag and a wb_nr_pinned count to struct nfs_page. This allows the request lifecycle to track ownership of physical pins and ensure that unpinning is performed only when the I/O is complete. 2. API Migration Migrate the Direct I/O path to the modern iov_iter_extract_pages() API. This aligns NFS with the modern extraction model and serves as the foundation for passing ITER_ALLOW_P2PDMA in a follow-up series. Upcoming Work / Roadmap ================================ As decided in the RFC [1] & v2 [2], there will be separate series for P2PDMA Enablement and Migrating NFS Direct I/O to use folios. This series lays the necessary groundwork for the upcoming work. Following this, two additional series are planned: 1. Migrating NFS Direct I/O to Folios A series that introduces and exports helper from iov_iter.c to allow the nfs_direct_extract_pages() helper introduced here to aggregate multiple pages into a single large folio-based request, aiming to reduce the RPC overhead for hugepage I/O. 2. P2PDMA Enablement for NFS Enabling ITER_ALLOW_P2PDMA for the Direct I/O path and introducing transport-level negotiation (discovery of P2P-capable RDMA/NVMe devices). This will build upon the PG_PINNED infrastructure introduced by this series. Testing ======= This series has been tested with xfstests [3] on RDMA & TCP transports by running the quick test suite for each transport vs. version combo: ./check -g quick -s rdma3 -s rdma40 -s rdma41 -s rdma42 -s tcp3 -s tcp40 -s tcp41 -s tcp42 The tests were run before & after applying the series. No regressions were observed. The following summary was tabulated via a custom script [4] (on github) to depict that the tests failing in v1 [5] are now passing. python3 display.py results/*/check.log +--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+ | testcase | rdma-sys-3 | rdma-sys-4.0 | rdma-sys-4.1 | rdma-sys-4.2 | tcp-sys-3 | tcp-sys-4.0 | tcp-sys-4.1 | tcp-sys-4.2 | +--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+ | generic/091 | pass | pass | pass | pass | pass | pass | pass | pass | | generic/130 | pass | pass | pass | pass | pass | pass | pass | pass | | generic/139 | pass | pass | pass | pass | pass | pass | pass | pass | | generic/143 | pass | pass | pass | pass | pass | pass | pass | pass | | generic/154 | pass | pass | pass | pass | pass | pass | pass | pass | | generic/155 | pass | pass | pass | pass | pass | pass | pass | pass | | generic/183 | skipped | skipped | skipped | pass | skipped | skipped | skipped | pass | | generic/188 | skipped | skipped | skipped | pass | skipped | skipped | skipped | pass | | generic/190 | skipped | skipped | skipped | pass | skipped | skipped | skipped | pass | | generic/196 | skipped | skipped | skipped | pass | skipped | skipped | skipped | pass | | generic/198 | pass | pass | pass | pass | pass | pass | pass | pass | | generic/203 | skipped | skipped | skipped | pass | skipped | skipped | skipped | pass | | generic/214 | pass | pass | pass | pass | pass | pass | pass | pass | | generic/240 | pass | pass | pass | pass | pass | pass | pass | pass | | generic/263 | pass | pass | pass | pass | pass | pass | pass | pass | | generic/287 | skipped | skipped | skipped | pass | skipped | skipped | skipped | pass | | generic/290 | skipped | skipped | skipped | pass | skipped | skipped | skipped | pass | | generic/292 | skipped | skipped | skipped | pass | skipped | skipped | skipped | pass | | generic/330 | skipped | skipped | skipped | pass | skipped | skipped | skipped | pass | | generic/444 | skipped | skipped | skipped | skipped | skipped | skipped | skipped | skipped | | generic/450 | pass | pass | pass | pass | pass | pass | pass | pass | | generic/451 | pass | pass | pass | pass | pass | pass | pass | pass | | generic/586 | pass | pass | pass | pass | pass | pass | pass | pass | | generic/647 | pass | pass | pass | pass | pass | pass | pass | pass | | generic/708 | pass | pass | pass | pass | pass | pass | pass | pass | | generic/729 | pass | pass | pass | pass | pass | pass | pass | pass | | generic/760 | pass | pass | pass | pass | pass | pass | pass | pass | +--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+ Thanks, Praan [1] https://lore.kernel.org/all/20260401194501.2269200-1-praan@google.com/ [2] https://lore.kernel.org/all/ak8-NMsNPOB3zpF-@infradead.org/ [3] https://git.kernel.org/pub/scm/fs/xfs/xfstests-dev.git [4] https://github.com/pran005/tools/blob/main/display.py [5] https://lore.kernel.org/all/29a0511d-5216-46f2-a7e4-9c04ae9b1890@app.fastmail.com/ [v6] - Fixed double unpin bugs pointed out by Sashiko: https://sashiko.dev/#/patchset/20260810195040.2877784-1-praan%40google.com - Added R-bs from Christoph (due since v4) [v5] - https://lore.kernel.org/all/20260810195040.2877784-1-praan@google.com/ - Fixed an iov_iter_revert oops in nfs_direct_extract_pages(). - Fixed a refcount underflow in nfs_direct_release_pages() during swap-over-NFS by ensuring we do not call put_page() on unpinned memory. - Rebased on fs-next [v4] - https://lore.kernel.org/all/20260720150601.2702700-1-praan@google.com/ - Dropped superfluous externs in include/linux/nfs_page.h - Scoped `req` locally inside the loop in nfs_release_request_list() - Wrapped >80 character lines in Patch 1 to adhere to coding style. - Fixed typos and capitalization convention in the commit msgs - Collected Reviewed-by tags from Christoph and Shivaji - Rebased on fs-next [v3] - https://lore.kernel.org/all/20260715143540.3597616-1-praan@google.com/ - Dropped patches that added folio support for NFS Direct I/O - Folded requested_bytes accounting in patch 5 due to dropped folio support - Rebased on fs-next [v2] - https://lore.kernel.org/all/20260616134000.2733403-1-praan@google.com/ - Fix data corruption in nfs_direct_extract_pages() by correctly calculating intra-page offsets using offset_in_page(). - Fix requested_bytes accounting in direct read/write paths to only increment after successful RPC scheduling. - Add missing kernel-doc descriptions for the @pinned parameter in nfs_page_create_from_page() and nfs_page_create_from_folio(). - Rebase on fs-next/ [v1] https://lore.kernel.org/all/20260603053033.3300318-1-praan@google.com/ Pranjal Shrivastava (5): nfs: make nfs_page pin-aware nfs: track number of pinned pages in nfs_page nfs: introduce nfs_release_request_list helper nfs: migrate direct I/O to iov_iter_extract_pages nfs: introduce nfs_direct_extract_pages helper fs/nfs/direct.c | 139 ++++++++++++++++++++------------------- fs/nfs/pagelist.c | 66 ++++++++++++++++--- fs/nfs/read.c | 2 +- fs/nfs/write.c | 2 +- include/linux/nfs_page.h | 8 ++- 5 files changed, 134 insertions(+), 83 deletions(-) base-commit: 15db403711f3cd9b7b764716ddf5909091b604bf -- 2.55.0.691.gc56d675ccc-goog
On Fri, Aug 14, 2026 at 02:32:50PM +0000, Pranjal Shrivastava wrote: > Modernize the NFS Direct I/O path as a preparatory step to enable PCI > Peer-to-Peer DMA (P2PDMA) support. Following feedback on the initial > RFC [1], the modernization and architectural changes are split into > different series. Additionally, based on the discussion in the v2 [2] > of this series, the migration of NFS Direct I/O to folios would be > handled in a separate follow-up series. > > Currently, NFS O_DIRECT relies on the legacy iov_iter_get_pages_alloc2() > API which does not support the pinning requirements for P2P memory. > The implementation moves NFS to the modern iov_iter_extract_pages() API. > Gentle ping on this series. (I realized that I sent it close to the merge window, Apologies). Please let me know if anything else needs to be done for this.. [...] > > [v6] > - Fixed double unpin bugs pointed out by Sashiko: > https://sashiko.dev/#/patchset/20260810195040.2877784-1-praan%40google.com > - Added R-bs from Christoph (due since v4) > > [v5] > - https://lore.kernel.org/all/20260810195040.2877784-1-praan@google.com/ > - Fixed an iov_iter_revert oops in nfs_direct_extract_pages(). > - Fixed a refcount underflow in nfs_direct_release_pages() during > swap-over-NFS by ensuring we do not call put_page() on unpinned memory. > - Rebased on fs-next > [...] Thanks, Praan
Hi Pranjal, On Fri, Sep 11, 2026, at 8:57 AM, Pranjal Shrivastava wrote: > On Fri, Aug 14, 2026 at 02:32:50PM +0000, Pranjal Shrivastava wrote: >> Modernize the NFS Direct I/O path as a preparatory step to enable PCI >> Peer-to-Peer DMA (P2PDMA) support. Following feedback on the initial >> RFC [1], the modernization and architectural changes are split into >> different series. Additionally, based on the discussion in the v2 [2] >> of this series, the migration of NFS Direct I/O to folios would be >> handled in a separate follow-up series. >> >> Currently, NFS O_DIRECT relies on the legacy iov_iter_get_pages_alloc2() >> API which does not support the pinning requirements for P2P memory. >> The implementation moves NFS to the modern iov_iter_extract_pages() API. >> > > Gentle ping on this series. > (I realized that I sent it close to the merge window, Apologies). > > Please let me know if anything else needs to be done for this.. Thanks for the ping! The plan right now is to send it along as part of the 7.4 merge window. I haven't pushed out a public linux-next branch yet, but when I do these patches will be included. I hope this helps! Anna > > [...] > >> >> [v6] >> - Fixed double unpin bugs pointed out by Sashiko: >> https://sashiko.dev/#/patchset/20260810195040.2877784-1-praan%40google.com >> - Added R-bs from Christoph (due since v4) >> >> [v5] >> - https://lore.kernel.org/all/20260810195040.2877784-1-praan@google.com/ >> - Fixed an iov_iter_revert oops in nfs_direct_extract_pages(). >> - Fixed a refcount underflow in nfs_direct_release_pages() during >> swap-over-NFS by ensuring we do not call put_page() on unpinned memory. >> - Rebased on fs-next >> > > [...] > > Thanks, > Praan
On Fri, Sep 11, 2026 at 10:41:57AM -0400, Anna Schumaker wrote: > Hi Pranjal, > > On Fri, Sep 11, 2026, at 8:57 AM, Pranjal Shrivastava wrote: > > On Fri, Aug 14, 2026 at 02:32:50PM +0000, Pranjal Shrivastava wrote: > >> Modernize the NFS Direct I/O path as a preparatory step to enable PCI > >> Peer-to-Peer DMA (P2PDMA) support. Following feedback on the initial > >> RFC [1], the modernization and architectural changes are split into > >> different series. Additionally, based on the discussion in the v2 [2] > >> of this series, the migration of NFS Direct I/O to folios would be > >> handled in a separate follow-up series. > >> > >> Currently, NFS O_DIRECT relies on the legacy iov_iter_get_pages_alloc2() > >> API which does not support the pinning requirements for P2P memory. > >> The implementation moves NFS to the modern iov_iter_extract_pages() API. > >> > > > > Gentle ping on this series. > > (I realized that I sent it close to the merge window, Apologies). > > > > Please let me know if anything else needs to be done for this.. > > Thanks for the ping! The plan right now is to send it along as part of > the 7.4 merge window. I haven't pushed out a public linux-next branch > yet, but when I do these patches will be included. > > I hope this helps! > Anna > Thanks a lot for responding, quite helpful! :) Best, Praan
© 2016 - 2026 Red Hat, Inc.