From nobody Mon Jun 8 06:35:51 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3C1752367CF for ; Mon, 1 Jun 2026 08:30:54 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780302655; cv=none; b=hplH1WfR8rEPxgWJCJP3/AHBYYZZnsUZU2ad9dUFp+6xF01ygGLkkkvVR9UbEmWP0BALmJtsJJf3PK1g6O92kDfQE+Ljdr3uLwpNQWBhdfx82pgdmmWVlnbCncw/uw/Nc/uGpVsgTAARNwi9GrbeQtUbejH4+5irWMTZ1Srx5hc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780302655; c=relaxed/simple; bh=cWyPAze5e+ZuvNa2kgU+iFMSHa3YVBSGLugbEZHPVtI=; h=From:To:Cc:Subject:Date:Message-ID:MIME-Version; b=oobhkkna964TFujxdSzeHcIpKFK+o5rUbV/8TgU2u8yb7e99S+RiUki3o+/bSSDvkRNH3QV9K3dqj9wBIoJhTsdLDRg/AfkG6RxB9+gkvidaRp2JH3Cy6PSg86H/Kx3hQ/V+bukg8xLgJjrOHJux/QPymUvycUvY0Idk5SKiwRE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=EYJarc1W; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="EYJarc1W" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 4198A1F00893; Mon, 1 Jun 2026 08:30:53 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1780302653; bh=IOoxh1i/NfGyHNHYUj44iC+SXTNiuSmyXkF+hDZN9Zs=; h=From:To:Cc:Subject:Date; b=EYJarc1WBjXEjFcFJvxmPWAAbf+3HdsrSPO0J7pvPWcjchQN0JV3z+pCapSrtZ3GX pZb44/6YDwpQGknU2DONJ4kZLcBe5Zljv6MSY2cXMDOhG/wiDvNMl4xBtXNKmqvKeW zd1xzTWFbS+g/kFAC5lNvZlRGW63u0QVWeXhTdUAfdXV9L0llD+sWKuyaXebkhnE/P p/1OhIoMCovAJiQV/SiX9hviMx/Ito0SvZjgqU2hD0u3ty7a7geFdFPxuFwMDB/aH0 Xy0i7n4DWYwW94kO7k5euMCt7XNPytzqvkLoZzm1+A0CZx/Sp0CxWdlehWEUPt6BXE r9HfIT+axA/mg== From: Lorenzo Stoakes To: Andrew Morton Cc: David Hildenbrand , Zi Yan , Baolin Wang , "Liam R . Howlett" , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Lance Yang , SeongJae Park , Balbir Singh , linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH mm-hotfixes] mm/huge_memory: use correct flags for device private PMD entry Date: Mon, 1 Jun 2026 09:30:44 +0100 Message-ID: <20260601083044.57132-1-ljs@kernel.org> X-Mailer: git-send-email 2.54.0 Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Commit 65edfda6f3f2 ("mm/rmap: extend rmap and migration support device-private entries") updated set_pmd_migration_entry() to use pmdp_huge_get_and_clear() in the softleaf case, but made no further adjustments to the function itself. Therefore this function continues to incorrectly use pmd_write(), pmd_soft_dirty() and pmd_uffd_wp() to determine whether the installed migration entry should be marked writable, softdirty or uffd-wp respectively. Whilst all are incorrect, the most problematic of these is pmd_write(), as this can lead to corrupted rmap state. On x86-64 _PAGE_SWP_SOFT_DIRTY is aliased to _PAGE_RW. So calling pmd_write() on a softleaf will return the softdirty state encoded in the entry, assuming CONFIG_MEM_SOFT_DIRTY was enabled. This was observed when running the hmm.hmm_device_private.anon_write_child selftest: 1. The test faults in a range then migrates it such that a device-private THP range is established. 2. The parent then migrates it to a device-private writable PMD entry whose folio is entirely AnonExclusive with entire_mapcount=3D1, softdirty set (accidentally correct write state). 3. The parent forks and the PMD entries are set to device-private read only entries, entire_mapcount=3D2, softdirty still set. 4. [BUG] The child writes to the range then migrates to RAM - intending to install non-writable migration entries - but replacing parent and child PMD mappings with WRITABLE entries due to misinterpreting the softdirty bit. 5. In remove_migration_pmd(), if !softleaf_is_migration_read(entry) we set the RMAP_EXCLUSIVE flag when calling folio_add_anon_rmap_pmd() for both parent and child, which are therefore AnonExclusive. 6. [SPLAT] Child sets migrated folio entire_mapcount=3D1, parent sets entire_mapcount=3D2 and we end up with an AnonExclusive folio with entire_mapcount=3D2! Assert fires in __folio_add_anon_rmap(): VM_WARN_ON_FOLIO(folio_test_large(folio) && folio_entire_mapcount(folio) > 1 && PageAnonExclusive(cur_page), folio) This patch fixes the issue by correctly referencing the softleaf entry fields for writable, softdirty and uffd-wp in set_pmd_migration_entry(). It also only updates A/D flags if the entry is present as these are otherwise not meaningful for a softleaf entry. This patch also flips the if (!present) { ... } else { ... } logic in set_pmd_migration_entry() so it is easier to understand, and adds some comments to make things clearer. I was able to bisect this to commit 775465fd26a3 ("lib/test_hmm: add zone device private THP test infrastructure") which first exposes this bug as it was the commit that permitted test_hmm to generate the test. However commit 65edfda6f3f2 ("mm/rmap: extend rmap and migration support device-private entries") is the commit that actually enabled this behaviour. Fixes: 65edfda6f3f2 ("mm/rmap: extend rmap and migration support device-pri= vate entries") Cc: stable@vger.kernel.org Signed-off-by: Lorenzo Stoakes Acked-by: David Hildenbrand (Arm) Reviewed-by: Balbir Singh Reviewed-by: Baolin Wang Reviewed-by: Barry Song Reviewed-by: Dev Jain Reviewed-by: Lance Yang Reviewed-by: Oscar Salvador (SUSE) Reviewed-by: Zi Yan --- mm/huge_memory.c | 45 +++++++++++++++++++++++++++++++++------------ 1 file changed, 33 insertions(+), 12 deletions(-) diff --git a/mm/huge_memory.c b/mm/huge_memory.c index bf9b480bb3b0..79463c709c98 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -4982,7 +4982,7 @@ int set_pmd_migration_entry(struct page_vma_mapped_wa= lk *pvmw, struct vm_area_struct *vma =3D pvmw->vma; struct mm_struct *mm =3D vma->vm_mm; unsigned long address =3D pvmw->address; - bool anon_exclusive; + bool anon_exclusive, present, writable, softdirty, uffd_wp; pmd_t pmdval; swp_entry_t entry; pmd_t pmdswp; @@ -4990,12 +4990,26 @@ int set_pmd_migration_entry(struct page_vma_mapped_= walk *pvmw, if (!(pvmw->pmd && !pvmw->pte)) return 0; - flush_cache_range(vma, address, address + HPAGE_PMD_SIZE); - if (unlikely(!pmd_present(*pvmw->pmd))) - pmdval =3D pmdp_huge_get_and_clear(vma->vm_mm, address, pvmw->pmd); - else + present =3D pmd_present(*pvmw->pmd); + if (likely(present)) { + flush_cache_range(vma, address, address + HPAGE_PMD_SIZE); + pmdval =3D pmdp_invalidate(vma, address, pvmw->pmd); + writable =3D pmd_write(pmdval); + softdirty =3D pmd_soft_dirty(pmdval); + uffd_wp =3D pmd_uffd_wp(pmdval); + } else { + softleaf_t old_entry; + + pmdval =3D pmdp_huge_get_and_clear(vma->vm_mm, address, pvmw->pmd); + old_entry =3D softleaf_from_pmd(pmdval); + + writable =3D softleaf_is_device_private_write(old_entry); + softdirty =3D pmd_swp_soft_dirty(pmdval); + uffd_wp =3D pmd_swp_uffd_wp(pmdval); + } + /* See folio_try_share_anon_rmap_pmd(): invalidate PMD first. */ anon_exclusive =3D folio_test_anon(folio) && PageAnonExclusive(page); if (anon_exclusive && folio_try_share_anon_rmap_pmd(folio, page)) { @@ -5003,24 +5017,31 @@ int set_pmd_migration_entry(struct page_vma_mapped_= walk *pvmw, return -EBUSY; } - if (pmd_dirty(pmdval)) - folio_mark_dirty(folio); - if (pmd_write(pmdval)) + /* Determine type of migration entry. */ + if (writable) entry =3D make_writable_migration_entry(page_to_pfn(page)); else if (anon_exclusive) entry =3D make_readable_exclusive_migration_entry(page_to_pfn(page)); else entry =3D make_readable_migration_entry(page_to_pfn(page)); - if (pmd_young(pmdval)) + + /* Set A/D bits as necessary. */ + if (present && pmd_young(pmdval)) entry =3D make_migration_entry_young(entry); - if (pmd_dirty(pmdval)) + if (present && pmd_dirty(pmdval)) { + folio_mark_dirty(folio); entry =3D make_migration_entry_dirty(entry); + } + + /* Set PMD. */ pmdswp =3D swp_entry_to_pmd(entry); - if (pmd_soft_dirty(pmdval)) + if (softdirty) pmdswp =3D pmd_swp_mksoft_dirty(pmdswp); - if (pmd_uffd_wp(pmdval)) + if (uffd_wp) pmdswp =3D pmd_swp_mkuffd_wp(pmdswp); set_pmd_at(mm, address, pvmw->pmd, pmdswp); + + /* Migration entry installed: cleanup rmap, folio. */ folio_remove_rmap_pmd(folio, page, vma); folio_put(folio); trace_set_migration_pmd(address, pmd_val(pmdswp)); -- 2.54.0