mm/huge_memory.c | 4 ++++ mm/madvise.c | 3 +++ mm/mempolicy.c | 2 ++ 3 files changed, 9 insertions(+)
Several LRU-oriented mm walkers resolve the folio backing a PMD entry (or a physical pfn) and then reclaim, age, migrate, or lazyfree it without ever checking for ZONE_DEVICE memory. This series adds missing folio_is_zone_device() rejections, matching the checks that comparable walkers already perform. - mm/huge_memory, mm/madvise: the !pmd_present branch above these sites only filters device-private entries (which are non-present). A present zone device PMD (e.g. device-coherent) would still reach the folio and be lazyfreed / aged / paged out. Add an explicit check. - mm/mempolicy: queue_folios_pmd() can see a present zone device PMD (e.g. device-coherent) and queue it for migration. No crash reproducer - this is a correctness/hardening cleanup found by inspection. All checks are placed after the folio is resolved and before it is acted upon, on paths that already hold the relevant page-table lock, so no locking or refcount changes are involved. Gregory Price (3): mm/huge_memory: skip zone device folios in madvise_free_huge_pmd() mm/madvise: skip zone device folios in cold/pageout PMD range mm/mempolicy: skip zone device folios when queueing folios mm/huge_memory.c | 4 ++++ mm/madvise.c | 3 +++ mm/mempolicy.c | 2 ++ 3 files changed, 9 insertions(+) --- v2 - drop hugetlb, move checks earlier in queue migration route - fixes tags for all 3 -- 2.53.0-Meta
On Mon, Aug 17, 2026 at 06:08:07PM -0400, Gregory Price wrote: >Several LRU-oriented mm walkers resolve the folio backing a PMD entry >(or a physical pfn) and then reclaim, age, migrate, or lazyfree it >without ever checking for ZONE_DEVICE memory. > >This series adds missing folio_is_zone_device() rejections, matching >the checks that comparable walkers already perform. > >- mm/huge_memory, mm/madvise: the !pmd_present branch above these sites > only filters device-private entries (which are non-present). > > A present zone device PMD (e.g. device-coherent) would still reach the > folio and be lazyfreed / aged / paged out. Add an explicit check. > >- mm/mempolicy: queue_folios_pmd() can see a present zone device PMD > (e.g. device-coherent) and queue it for migration. > >No crash reproducer - this is a correctness/hardening cleanup found by >inspection. All checks are placed after the folio is resolved and before >it is acted upon, on paths that already hold the relevant page-table lock, >so no locking or refcount changes are involved. Cool! Gave the whole series a spin on x86_64 QEMU with a PMD-mapped device-coherent THP. Without these patches, partial MADV_FREE and MADV_COLD reliably hit a kernel panic in remove_migration_pte(), while mbind(MPOL_MF_MOVE | MPOL_MF_STRICT) returned -EIO. With v2, all three worked fine, PMD mapping stayed intact, and data checked out :) Note that both kernels used the same small change to the in-kernel HMM test driver, allowing its coherent device memory to be allocated as 2 MB folios so the PMD-mapped test case could be exercised. Tested-by: Lance Yang <lance.yang@linux.dev>
On Tue, 18 Aug 2026 12:31:43 +0800 Lance Yang <lance.yang@linux.dev> wrote: > > On Mon, Aug 17, 2026 at 06:08:07PM -0400, Gregory Price wrote: > >Several LRU-oriented mm walkers resolve the folio backing a PMD entry > >(or a physical pfn) and then reclaim, age, migrate, or lazyfree it > >without ever checking for ZONE_DEVICE memory. > > > >This series adds missing folio_is_zone_device() rejections, matching > >the checks that comparable walkers already perform. > > > >- mm/huge_memory, mm/madvise: the !pmd_present branch above these sites > > only filters device-private entries (which are non-present). > > > > A present zone device PMD (e.g. device-coherent) would still reach the > > folio and be lazyfreed / aged / paged out. Add an explicit check. > > > >- mm/mempolicy: queue_folios_pmd() can see a present zone device PMD > > (e.g. device-coherent) and queue it for migration. > > > >No crash reproducer - this is a correctness/hardening cleanup found by > >inspection. All checks are placed after the folio is resolved and before > >it is acted upon, on paths that already hold the relevant page-table lock, > >so no locking or refcount changes are involved. > > Cool! > > Gave the whole series a spin on x86_64 QEMU with a PMD-mapped > device-coherent THP. Without these patches, partial MADV_FREE and > MADV_COLD reliably hit a kernel panic in remove_migration_pte(), while > mbind(MPOL_MF_MOVE | MPOL_MF_STRICT) returned -EIO. > > With v2, all three worked fine, PMD mapping stayed intact, and data > checked out :) > > Note that both kernels used the same small change to the in-kernel HMM > test driver, allowing its coherent device memory to be allocated as 2 MB > folios so the PMD-mapped test case could be exercised. > > Tested-by: Lance Yang <lance.yang@linux.dev> Thanks Lance, you're so diligent. I'm wondering what to do here. Gregory told us : No crash reproducer - this is a correctness/hardening cleanup found by : inspection. All checks are placed after the folio is resolved and before : it is acted upon, on paths that already hold the relevant page-table lock, : so no locking or refcount changes are involved. And you had to tweak the hmm-test driver to reproduce the bug(s). So when do we push this series out to -stable? As a hair-on-fire hotfix, or as a leisurely next-merge-window thing? And Sashiko was clearly having a bad day, able to find only nine pre-existing things to shout about: https://sashiko.dev/#/patchset/20260817220810.1175596-1-gourry@gourry.net
On 2026/8/30 08:18, Andrew Morton wrote: > On Tue, 18 Aug 2026 12:31:43 +0800 Lance Yang <lance.yang@linux.dev> wrote: > >> >> On Mon, Aug 17, 2026 at 06:08:07PM -0400, Gregory Price wrote: >>> Several LRU-oriented mm walkers resolve the folio backing a PMD entry >>> (or a physical pfn) and then reclaim, age, migrate, or lazyfree it >>> without ever checking for ZONE_DEVICE memory. >>> >>> This series adds missing folio_is_zone_device() rejections, matching >>> the checks that comparable walkers already perform. >>> >>> - mm/huge_memory, mm/madvise: the !pmd_present branch above these sites >>> only filters device-private entries (which are non-present). >>> >>> A present zone device PMD (e.g. device-coherent) would still reach the >>> folio and be lazyfreed / aged / paged out. Add an explicit check. >>> >>> - mm/mempolicy: queue_folios_pmd() can see a present zone device PMD >>> (e.g. device-coherent) and queue it for migration. >>> >>> No crash reproducer - this is a correctness/hardening cleanup found by >>> inspection. All checks are placed after the folio is resolved and before >>> it is acted upon, on paths that already hold the relevant page-table lock, >>> so no locking or refcount changes are involved. >> >> Cool! >> >> Gave the whole series a spin on x86_64 QEMU with a PMD-mapped >> device-coherent THP. Without these patches, partial MADV_FREE and >> MADV_COLD reliably hit a kernel panic in remove_migration_pte(), while >> mbind(MPOL_MF_MOVE | MPOL_MF_STRICT) returned -EIO. >> >> With v2, all three worked fine, PMD mapping stayed intact, and data >> checked out :) >> >> Note that both kernels used the same small change to the in-kernel HMM >> test driver, allowing its coherent device memory to be allocated as 2 MB >> folios so the PMD-mapped test case could be exercised. >> >> Tested-by: Lance Yang <lance.yang@linux.dev> > > Thanks Lance, you're so diligent. > > I'm wondering what to do here. Gregory told us > > : No crash reproducer - this is a correctness/hardening cleanup found by > : inspection. All checks are placed after the folio is resolved and before > : it is acted upon, on paths that already hold the relevant page-table lock, > : so no locking or refcount changes are involved. > > And you had to tweak the hmm-test driver to reproduce the bug(s). > > So when do we push this series out to -stable? As a hair-on-fire > hotfix, or as a leisurely next-merge-window thing? Thanks, Andrew :) Yeah, I'd say next merge window should be fine :) The crash is real once the mapping exists, but I had to tweak test_hmm to create that PMD-mapped device-coherent folio, and I couldn't find any in-tree production driver doing that today. So no need to rush this one, I guess. > > And Sashiko was clearly having a bad day, able to find only nine > pre-existing things to shout about: > https://sashiko.dev/#/patchset/20260817220810.1175596-1-gourry@gourry.net >
On Sun, Aug 30, 2026 at 01:18:24PM +0800, Lance Yang wrote:
>
>
>On 2026/8/30 08:18, Andrew Morton wrote:
>> On Tue, 18 Aug 2026 12:31:43 +0800 Lance Yang <lance.yang@linux.dev> wrote:
>>
>>>
>>> On Mon, Aug 17, 2026 at 06:08:07PM -0400, Gregory Price wrote:
>>>> Several LRU-oriented mm walkers resolve the folio backing a PMD entry
>>>> (or a physical pfn) and then reclaim, age, migrate, or lazyfree it
>>>> without ever checking for ZONE_DEVICE memory.
>>>>
>>>> This series adds missing folio_is_zone_device() rejections, matching
>>>> the checks that comparable walkers already perform.
>>>>
>>>> - mm/huge_memory, mm/madvise: the !pmd_present branch above these sites
>>>> only filters device-private entries (which are non-present).
>>>>
>>>> A present zone device PMD (e.g. device-coherent) would still reach the
>>>> folio and be lazyfreed / aged / paged out. Add an explicit check.
>>>>
>>>> - mm/mempolicy: queue_folios_pmd() can see a present zone device PMD
>>>> (e.g. device-coherent) and queue it for migration.
>>>>
>>>> No crash reproducer - this is a correctness/hardening cleanup found by
>>>> inspection. All checks are placed after the folio is resolved and before
>>>> it is acted upon, on paths that already hold the relevant page-table lock,
>>>> so no locking or refcount changes are involved.
>>>
>>> Cool!
>>>
>>> Gave the whole series a spin on x86_64 QEMU with a PMD-mapped
>>> device-coherent THP. Without these patches, partial MADV_FREE and
>>> MADV_COLD reliably hit a kernel panic in remove_migration_pte(), while
>>> mbind(MPOL_MF_MOVE | MPOL_MF_STRICT) returned -EIO.
>>>
>>> With v2, all three worked fine, PMD mapping stayed intact, and data
>>> checked out :)
>>>
>>> Note that both kernels used the same small change to the in-kernel HMM
>>> test driver, allowing its coherent device memory to be allocated as 2 MB
>>> folios so the PMD-mapped test case could be exercised.
>>>
>>> Tested-by: Lance Yang <lance.yang@linux.dev>
>>
>> Thanks Lance, you're so diligent.
>>
>> I'm wondering what to do here. Gregory told us
>>
>> : No crash reproducer - this is a correctness/hardening cleanup found by
>> : inspection. All checks are placed after the folio is resolved and before
>> : it is acted upon, on paths that already hold the relevant page-table lock,
>> : so no locking or refcount changes are involved.
>>
>> And you had to tweak the hmm-test driver to reproduce the bug(s).
>>
>> So when do we push this series out to -stable? As a hair-on-fire
>> hotfix, or as a leisurely next-merge-window thing?
>
>Thanks, Andrew :) Yeah, I'd say next merge window should be fine :)
>
>The crash is real once the mapping exists, but I had to tweak test_hmm
>to create that PMD-mapped device-coherent folio, and I couldn't find
>any in-tree production driver doing that today.
>
>So no need to rush this one, I guess.
BTW, noticed that the ZONE_DEVICE split handling only covers
device-private folios, so device-coherent folios aren't supported ...
The call chains are:
split_folio()
-> __folio_split()
-> folio_check_splittable()
-> __folio_freeze_and_split_unmapped()
migrate_vma_pages()
-> __migrate_device_pages()
-> migrate_vma_split_unmapped_folio()
-> folio_split_unmapped()
-> __folio_freeze_and_split_unmapped()
And I added the device-coherent check to folio_check_splittable() and
folio_split_unmapped(). See below. They can go away once device-coherent
folio splitting is supported :)
If folks think it's worth having, I can send it as a follow-up :)
---8<---
Subject: [PATCH] mm/huge_memory: don't split device-coherent folios
From: Lance Yang <lance.yang@linux.dev>
The ZONE_DEVICE split handling only covers device-private folios.
Device-coherent folios are not supported.
The call chains are:
split_folio()
-> __folio_split()
-> folio_check_splittable()
-> __folio_freeze_and_split_unmapped()
migrate_vma_pages()
-> __migrate_device_pages()
-> migrate_vma_split_unmapped_folio()
-> folio_split_unmapped()
-> __folio_freeze_and_split_unmapped()
Reject device-coherent folios in folio_check_splittable() and
folio_split_unmapped().
Fixes: a30b48bf1b24 ("mm/migrate_device: implement THP migration of zone device pages")
Signed-off-by: Lance Yang <lance.yang@linux.dev>
---
mm/huge_memory.c | 14 +++++++++++---
1 file changed, 11 insertions(+), 3 deletions(-)
diff --git a/mm/huge_memory.c b/mm/huge_memory.c
index 54494c3fa983..a5dd38e9a8de 100644
--- a/mm/huge_memory.c
+++ b/mm/huge_memory.c
@@ -3937,6 +3937,10 @@ int folio_check_splittable(struct folio *folio, unsigned int new_order,
if (!folio->mapping && !folio_test_anon(folio))
return -EBUSY;
+ /* TODO: Support splitting device-coherent folios. */
+ if (folio_is_device_coherent(folio))
+ return -EOPNOTSUPP;
+
/* order-1 is not supported for anonymous THP. */
if (folio_test_anon(folio) && new_order == 1)
return -EINVAL;
@@ -4354,15 +4358,16 @@ static int __folio_split(struct folio *folio, unsigned int new_order,
*
* anon_vma_lock is not required to be held, mmap_read_lock() or
* mmap_write_lock() should be held. @folio is expected to be locked by the
- * caller. device-private and non device-private folios are supported along
+ * caller. device-private and non-ZONE_DEVICE folios are supported along
* with folios that are in the swapcache. @folio should also be unmapped and
* isolated from LRU (if applicable)
*
* Upon return, the folio is not remapped, split folios are not added to LRU,
* free_folio_and_swap_cache() is not called, and new folios remain locked.
*
- * Return: 0 on success, -EAGAIN if the folio cannot be split (e.g., due to
- * insufficient reference count or extra pins).
+ * Return: 0 on success, -EOPNOTSUPP for device-coherent folios, or -EAGAIN if
+ * the folio cannot be split (e.g., due to insufficient reference
+ * count or extra pins).
*/
int folio_split_unmapped(struct folio *folio, unsigned int new_order)
{
@@ -4373,6 +4378,9 @@ int folio_split_unmapped(struct folio *folio, unsigned int new_order)
VM_WARN_ON_ONCE_FOLIO(!folio_test_large(folio), folio);
VM_WARN_ON_ONCE_FOLIO(!folio_test_anon(folio), folio);
+ if (folio_is_device_coherent(folio))
+ return -EOPNOTSUPP;
+
if (folio_expected_ref_count(folio) != folio_ref_count(folio) - 1)
return -EAGAIN;
--
Cheers, Lance
On 8/30/26 5:47 PM, Lance Yang wrote:
>
> On Sun, Aug 30, 2026 at 01:18:24PM +0800, Lance Yang wrote:
>>
>>
>> On 2026/8/30 08:18, Andrew Morton wrote:
>>> On Tue, 18 Aug 2026 12:31:43 +0800 Lance Yang <lance.yang@linux.dev> wrote:
>>>
>>>>
>>>> On Mon, Aug 17, 2026 at 06:08:07PM -0400, Gregory Price wrote:
>>>>> Several LRU-oriented mm walkers resolve the folio backing a PMD entry
>>>>> (or a physical pfn) and then reclaim, age, migrate, or lazyfree it
>>>>> without ever checking for ZONE_DEVICE memory.
>>>>>
>>>>> This series adds missing folio_is_zone_device() rejections, matching
>>>>> the checks that comparable walkers already perform.
>>>>>
>>>>> - mm/huge_memory, mm/madvise: the !pmd_present branch above these sites
>>>>> only filters device-private entries (which are non-present).
>>>>>
>>>>> A present zone device PMD (e.g. device-coherent) would still reach the
>>>>> folio and be lazyfreed / aged / paged out. Add an explicit check.
>>>>>
>>>>> - mm/mempolicy: queue_folios_pmd() can see a present zone device PMD
>>>>> (e.g. device-coherent) and queue it for migration.
>>>>>
>>>>> No crash reproducer - this is a correctness/hardening cleanup found by
>>>>> inspection. All checks are placed after the folio is resolved and before
>>>>> it is acted upon, on paths that already hold the relevant page-table lock,
>>>>> so no locking or refcount changes are involved.
>>>>
>>>> Cool!
>>>>
>>>> Gave the whole series a spin on x86_64 QEMU with a PMD-mapped
>>>> device-coherent THP. Without these patches, partial MADV_FREE and
>>>> MADV_COLD reliably hit a kernel panic in remove_migration_pte(), while
>>>> mbind(MPOL_MF_MOVE | MPOL_MF_STRICT) returned -EIO.
>>>>
>>>> With v2, all three worked fine, PMD mapping stayed intact, and data
>>>> checked out :)
>>>>
>>>> Note that both kernels used the same small change to the in-kernel HMM
>>>> test driver, allowing its coherent device memory to be allocated as 2 MB
>>>> folios so the PMD-mapped test case could be exercised.
>>>>
>>>> Tested-by: Lance Yang <lance.yang@linux.dev>
>>>
>>> Thanks Lance, you're so diligent.
>>>
>>> I'm wondering what to do here. Gregory told us
>>>
>>> : No crash reproducer - this is a correctness/hardening cleanup found by
>>> : inspection. All checks are placed after the folio is resolved and before
>>> : it is acted upon, on paths that already hold the relevant page-table lock,
>>> : so no locking or refcount changes are involved.
>>>
>>> And you had to tweak the hmm-test driver to reproduce the bug(s).
>>>
>>> So when do we push this series out to -stable? As a hair-on-fire
>>> hotfix, or as a leisurely next-merge-window thing?
>>
>> Thanks, Andrew :) Yeah, I'd say next merge window should be fine :)
>>
>> The crash is real once the mapping exists, but I had to tweak test_hmm
>> to create that PMD-mapped device-coherent folio, and I couldn't find
>> any in-tree production driver doing that today.
>>
>> So no need to rush this one, I guess.
>
> BTW, noticed that the ZONE_DEVICE split handling only covers
> device-private folios, so device-coherent folios aren't supported ...
>
> The call chains are:
>
> split_folio()
> -> __folio_split()
> -> folio_check_splittable()
> -> __folio_freeze_and_split_unmapped()
>
> migrate_vma_pages()
> -> __migrate_device_pages()
> -> migrate_vma_split_unmapped_folio()
> -> folio_split_unmapped()
> -> __folio_freeze_and_split_unmapped()
>
> And I added the device-coherent check to folio_check_splittable() and
> folio_split_unmapped(). See below. They can go away once device-coherent
> folio splitting is supported :)
>
> If folks think it's worth having, I can send it as a follow-up :)
>
> ---8<---
> Subject: [PATCH] mm/huge_memory: don't split device-coherent folios
>
> From: Lance Yang <lance.yang@linux.dev>
>
> The ZONE_DEVICE split handling only covers device-private folios.
> Device-coherent folios are not supported.
>
> The call chains are:
>
> split_folio()
> -> __folio_split()
> -> folio_check_splittable()
> -> __folio_freeze_and_split_unmapped()
>
> migrate_vma_pages()
> -> __migrate_device_pages()
> -> migrate_vma_split_unmapped_folio()
> -> folio_split_unmapped()
> -> __folio_freeze_and_split_unmapped()
>
> Reject device-coherent folios in folio_check_splittable() and
> folio_split_unmapped().
>
> Fixes: a30b48bf1b24 ("mm/migrate_device: implement THP migration of zone device pages")
> Signed-off-by: Lance Yang <lance.yang@linux.dev>
> ---
> mm/huge_memory.c | 14 +++++++++++---
> 1 file changed, 11 insertions(+), 3 deletions(-)
>
> diff --git a/mm/huge_memory.c b/mm/huge_memory.c
> index 54494c3fa983..a5dd38e9a8de 100644
> --- a/mm/huge_memory.c
> +++ b/mm/huge_memory.c
> @@ -3937,6 +3937,10 @@ int folio_check_splittable(struct folio *folio, unsigned int new_order,
> if (!folio->mapping && !folio_test_anon(folio))
> return -EBUSY;
>
> + /* TODO: Support splitting device-coherent folios. */
> + if (folio_is_device_coherent(folio))
> + return -EOPNOTSUPP;
> +
> /* order-1 is not supported for anonymous THP. */
> if (folio_test_anon(folio) && new_order == 1)
> return -EINVAL;
> @@ -4354,15 +4358,16 @@ static int __folio_split(struct folio *folio, unsigned int new_order,
> *
> * anon_vma_lock is not required to be held, mmap_read_lock() or
> * mmap_write_lock() should be held. @folio is expected to be locked by the
> - * caller. device-private and non device-private folios are supported along
> + * caller. device-private and non-ZONE_DEVICE folios are supported along
> * with folios that are in the swapcache. @folio should also be unmapped and
> * isolated from LRU (if applicable)
> *
> * Upon return, the folio is not remapped, split folios are not added to LRU,
> * free_folio_and_swap_cache() is not called, and new folios remain locked.
> *
> - * Return: 0 on success, -EAGAIN if the folio cannot be split (e.g., due to
> - * insufficient reference count or extra pins).
> + * Return: 0 on success, -EOPNOTSUPP for device-coherent folios, or -EAGAIN if
> + * the folio cannot be split (e.g., due to insufficient reference
> + * count or extra pins).
> */
> int folio_split_unmapped(struct folio *folio, unsigned int new_order)
> {
> @@ -4373,6 +4378,9 @@ int folio_split_unmapped(struct folio *folio, unsigned int new_order)
> VM_WARN_ON_ONCE_FOLIO(!folio_test_large(folio), folio);
> VM_WARN_ON_ONCE_FOLIO(!folio_test_anon(folio), folio);
>
> + if (folio_is_device_coherent(folio))
> + return -EOPNOTSUPP;
> +
> if (folio_expected_ref_count(folio) != folio_ref_count(folio) - 1)
> return -EAGAIN;
>
> --
>
FYI: Device Coherent THP is not yet supported, support should be easy to add. it
is definitely desirable, we should get it working along with mTHP support as
well (TODO).
Balbir
On Sun, Aug 30, 2026 at 03:47:51PM +0800, Lance Yang wrote: > > If folks think it's worth having, I can send it as a follow-up :) > > ---8<--- > Subject: [PATCH] mm/huge_memory: don't split device-coherent folios > First - thank you for testing, I was struggling to figure out a way to test this and I somehow missed your email where you managed it. Second - I think the split checks are worth it. As I've worked through my private node series, I've learned there are at least a handful of missing zone device hooks throughout mm - but there is a lot of nuance between the modes that is frustrating to keep straight. Every zone device patch helps clarify how coherent-link accelerators might manage its memory as a numa node, so at the very least I appreciate it. ~Gregory
© 2016 - 2026 Red Hat, Inc.