[v3] mm: handle device-private PMDs in walk callbacks

[PATCH v3 0/3] mm: handle device-private PMDs in walk callbacks

Posted by Usama Arif 2 weeks, 1 day ago

Since commit 368076f52ebe ("mm/huge_memory: add device-private THP support
to PMD operations") a PMD may hold a device-private swap entry whenever
an HMM-based GPU driver migrates an anonymous THP folio to device memory
via migrate_vma_pages().

pmd_trans_huge_lock() succeeds for such PMDs (pmd_is_huge() returns true
for any non-present, non-none huge PMD), so several MM walk callbacks
that used to assume present THP or migration entry are now reachable with
a device-private PMD. The results range from a VM_BUG_ON() firing on debug
kernels, to an oops on a bogus vmemmap dereference, to silently isolating
an unrelated live folio from LRU in the aliasing case.

---
v2 -> v3: https://lore.kernel.org/all/20260708122040.861335-1-usama.arif@linux.dev/
- Patch 1: use pmdp_get() instead of *pmd. (Zi)
- Patches 2 and 3: drop the thp_migration_supported() guard and
  downgrade VM_BUG_ON to VM_WARN_ON_ONCE(). thp_migration_supported()
  expands to IS_ENABLED(CONFIG_ARCH_SUPPORTS_PMD_SOFTLEAF), and both
  pmd_is_migration_entry() and pmd_is_device_private_entry() already
  return false without that config, so the guard suppresses only the
  case where the warning would already be silent. (David)

v1 -> v2: https://lore.kernel.org/all/20260707135255.292870-1-usama.arif@linux.dev/
- Patch 1 now gates queue_folios_pmd() on !pmd_present() instead of
  checking only pmd_is_device_private_entry(). This matches the PTE path,
  keeps migration entries counted in qp->nr_failed, and skips other
  non-present PMDs such as device-private entries before calling
  pmd_folio(). (Joshua)
- Patches 2 and 3 now fold device-private PMD handling into the existing
  !pmd_present() VM_BUG_ON() condition instead of using a separate early
  pmd_is_device_private_entry() check. (Joshua)
- cc stable (Zi)
 
Usama Arif (3):
  mm/mempolicy: skip non-present PMDs when queueing folios
  mm/madvise: skip device-private PMDs in cold and pageout walks
  mm/huge_memory: skip device-private PMDs in madvise_free_huge_pmd

 mm/huge_memory.c | 4 ++--
 mm/madvise.c     | 4 ++--
 mm/mempolicy.c   | 8 +++++---
 3 files changed, 9 insertions(+), 7 deletions(-)

-- 
2.53.0-Meta

Re: [PATCH v3 0/3] mm: handle device-private PMDs in walk callbacks

Posted by Andrew Morton 2 weeks, 1 day ago

On Fri, 10 Jul 2026 03:55:20 -0700 Usama Arif <usama.arif@linux.dev> wrote:

> Since commit 368076f52ebe ("mm/huge_memory: add device-private THP support
> to PMD operations") a PMD may hold a device-private swap entry whenever
> an HMM-based GPU driver migrates an anonymous THP folio to device memory
> via migrate_vma_pages().
> 
> pmd_trans_huge_lock() succeeds for such PMDs (pmd_is_huge() returns true
> for any non-present, non-none huge PMD), so several MM walk callbacks
> that used to assume present THP or migration entry are now reachable with
> a device-private PMD. The results range from a VM_BUG_ON() firing on debug
> kernels, to an oops on a bogus vmemmap dereference, to silently isolating
> an unrelated live folio from LRU in the aliasing case.

Added, thanks.

I didn't add these as hotfixes - the Fixes: commit is somewhat old. 
Feel free to disagree with this!

Sashiko went nuts over possible pre-existing issues:

	https://sashiko.dev/#/patchset/20260710105557.1987433-1-usama.arif@linux.dev

Re: [PATCH v3 0/3] mm: handle device-private PMDs in walk callbacks

Posted by Balbir Singh 1 week, 5 days ago

On Fri, Jul 10, 2026 at 04:53:48PM -0700, Andrew Morton wrote:
> On Fri, 10 Jul 2026 03:55:20 -0700 Usama Arif <usama.arif@linux.dev> wrote:
> 
> > Since commit 368076f52ebe ("mm/huge_memory: add device-private THP support
> > to PMD operations") a PMD may hold a device-private swap entry whenever
> > an HMM-based GPU driver migrates an anonymous THP folio to device memory
> > via migrate_vma_pages().
> > 
> > pmd_trans_huge_lock() succeeds for such PMDs (pmd_is_huge() returns true
> > for any non-present, non-none huge PMD), so several MM walk callbacks
> > that used to assume present THP or migration entry are now reachable with
> > a device-private PMD. The results range from a VM_BUG_ON() firing on debug
> > kernels, to an oops on a bogus vmemmap dereference, to silently isolating
> > an unrelated live folio from LRU in the aliasing case.
> 
> Added, thanks.
>

Thanks Andrew!

> I didn't add these as hotfixes - the Fixes: commit is somewhat old. 
> Feel free to disagree with this!
> 
> Sashiko went nuts over possible pre-existing issues:
> 
> 	https://sashiko.dev/#/patchset/20260710105557.1987433-1-usama.arif@linux.dev
> 

The pointed out seems plausible? I am working on some tests to catch
more errors via hmm-tests.

Balbir