From nobody Sat Sep 26 22:53:02 2026 Received: from mail-pf1-f170.google.com (mail-pf1-f170.google.com [209.85.210.170]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 26A41383993 for ; Fri, 28 Aug 2026 18:37:03 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.170 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1787942227; cv=none; b=iJY/OqyYqma4SMzY3qAmN1BP/AL/4t8sKVXQHp7TzG/X3SRKNqtkKLi/TV9ElOTbI/1NtrspFvcAsqaffKA9zChEXWKMtJ2Qz6OXwuvsi5fIfm4llxszlgtwTEmo3+Y5SnHwWJwm2RFbOVQvPX1N4jBRFl34PmjgoNbitXIxpl0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1787942227; c=relaxed/simple; bh=I25lWYt6cViseaUZtP8CeKwfUIj5/ciNKEk4etFYL1w=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=fH92yCBLt5yM5nFGkcQVyohlKEfRSkj3sen3px4/wwBVlbIQaaBEJOqFn2iiRDcOq6Z5jsFaUB1z7aL2tpEqMlTVt2zQMJM6KBd5wKHzc/rvsPmbMQi08jOWi73v+lGLsaio7Iv7DKeUMg62Wzw4nMMtyd/zsHe4h/29FCqKHw8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=CZ00InjO; arc=none smtp.client-ip=209.85.210.170 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="CZ00InjO" Received: by mail-pf1-f170.google.com with SMTP id d2e1a72fcca58-84f3ab8750cso1072064b3a.0 for ; Fri, 28 Aug 2026 11:37:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1787942223; x=1788547023; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=7mVEURHMP319Byv9p6qmQOcjq3woNvQVkr+r4ShGABI=; b=CZ00InjOHXHX/H8HcWuRUtfMdLDaC6jCwr9GCNYajPWW77vuhiTUse7xrlq4zJqtLB QFxZIEeW/DbD+olttDXAw276gLg4snaZYwZlYc1owsTp93eoN6QRnn4sjK4e9QSCzcsa SGsI2S0358nrAiDAGw+j2EQhKdzZOORh86tK08y57cmX65DMBh1IJ/7OIT5USaT2wTUs uYf1OOYlOMF9Uej0Di7MQfyQFsZwQvbsx2zUluBkhEHnAOjnHI/KRro1boSz7CW5gnNv 7hH6OppQmfWglAyuHVlcruCELwFfTIUw8myLyJsnG9cf6DXdOsnmGMw7ptg3aYNVmpRH XdvA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1787942223; x=1788547023; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=7mVEURHMP319Byv9p6qmQOcjq3woNvQVkr+r4ShGABI=; b=l7rkA2V0uQKrB5Y+3nO08y/hGjEX7Xd1XLiv//HVNIhGcimZQoPBIlkmcauc3+Gg5U TgjEoUw3+skSiLOmERBQ5lWqxDsjR9oed8KWwq6Ql40+yjUyyeSYSUr7Pst/EwIEzAlV RA4YVCpOws+YG8iYDSBjuRzoXW1eaFlr0UQnOpl56bvDfTHpMLEKHaiOBBvyNEeEiwY/ IHytPg7E3IWjFMYmQSG5aqgGhUwFj/27hOddhkt86i6ULV3TRokTY5lA7k7YnKrJ/0Ze sKgjENVcnrJjDKfqxdY0njbhbA7G93JC14nvmeZoPlU1PI9wy+Asf8GaTr1XyLITARQ1 3UOA== X-Forwarded-Encrypted: i=1; AHgh+Rp0lScUembjdohgk8l4nSYvTgm+f+iqNdQxsjgvIS804rTpo4vx5ejPCACedNeNcqESk5q6PzftBvfMY4w=@vger.kernel.org X-Gm-Message-State: AFuF++m68kICtTCjv+slj7LsA60qkB0LstdgRR2obE6DMkqKDZ92MWGf v5h1tVLDsndhIawHc2O3YNrIvHJvQN9xSMGPRMDbXrPs0c7nzAmOSXMc X-Gm-Gg: AR+sD11LVv2ZsuKUVWz8xohn82Cu+bua2GHDf71y8YhlLrUWus/X9K7V7NJ6+4Tp7yB IdI4urZ++4XQpLm7VntqjjWI0rmt/OwYj76OuA++sNtRzw0Xt+XkpA9TxHf2HRwyLDeF6ZitNpk RQaGzHdnb65nGLF/CpnmaFJHOYhonigj4SLQPTI/Z0ONcT5wK+ytBzFUbnqVxzC+gWFIwGewpkB KLfBDt6Et1CfjCzEwxKia2GNqqDDtvqO2dKZ2dQ97ozXz1p3XcTpfkB3YtIX3Vv1PJb5vJOmJ9E 4y+xeZd9ZQ1kiPyh7RArjfm8iBSJoA3rHiPvz32LMfEvFsrEE9Xcrz7k0K6hY24NduFLpakX3Nx AFP1W/qSgxk4iExgFuJ/V/eLf0vR0h6RRxME5sRoXfmfh+vnU7fxn9ycxeekQlSidIjfJhiCJim k8iaPl/Os/liEuEk7YE8CUIyCjAKAgZ3RUwJLzTlSDygv9tDENXoZ0PXIxxtYuWjX2hyzWFnGIj 4Hw5r/AY1D6DjClL2HRGJEoZ7KR X-Received: by 2002:a05:6a00:2309:b0:857:7384:b5fc with SMTP id d2e1a72fcca58-8577384c2c3mr2082303b3a.24.1787942223100; Fri, 28 Aug 2026 11:37:03 -0700 (PDT) Received: from aig-kmd-01.. ([120.133.49.20]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-856a2ea6fa3sm844230b3a.37.2026.08.28.11.36.53 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 28 Aug 2026 11:37:02 -0700 (PDT) From: Yin Tirui To: Andrew Morton , linux-mm@kvack.org Cc: David Hildenbrand , Lorenzo Stoakes , Dev Jain , Zi Yan , Baolin Wang , Barry Song , Lance Yang , Ryan Roberts , Nico Pache , Usama Arif , "Liam R . Howlett" , wangkefeng.wang@huawei.com, chenjun102@huawei.com, linux-kernel@vger.kernel.org, Yin Tirui Subject: [PATCH RFC 1/9] mm/huge_memory: read the huge PMD entry once when splitting it Date: Sat, 29 Aug 2026 02:33:11 +0800 Message-Id: <1b11fdaa4f5819959505ad72e3d70bad5fc1d343.1787941780.git.yintirui@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" __split_huge_pmd_locked() re-reads *pmd several times to classify the same entry. The PMD page table lock is held throughout, so read it once into old_pmd and classify from that. No functional change intended. Signed-off-by: Yin Tirui --- mm/huge_memory.c | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/mm/huge_memory.c b/mm/huge_memory.c index afbb5974bd22..8feabdcf6307 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -3143,10 +3143,11 @@ static void __split_huge_pmd_locked(struct vm_area_= struct *vma, pmd_t *pmd, unsigned long haddr, bool freeze) { struct mm_struct *mm =3D vma->vm_mm; + pmd_t old_pmd =3D *pmd; struct folio *folio; struct page *page; pgtable_t pgtable; - pmd_t old_pmd, _pmd; + pmd_t _pmd; bool soft_dirty, uffd_wp =3D false, young =3D false, write =3D false; bool anon_exclusive =3D false, dirty =3D false; unsigned long addr; @@ -3157,7 +3158,8 @@ static void __split_huge_pmd_locked(struct vm_area_st= ruct *vma, pmd_t *pmd, VM_BUG_ON_VMA(vma->vm_start > haddr, vma); VM_BUG_ON_VMA(vma->vm_end < haddr + HPAGE_PMD_SIZE, vma); =20 - VM_WARN_ON_ONCE(!pmd_is_valid_softleaf(*pmd) && !pmd_trans_huge(*pmd)); + VM_WARN_ON_ONCE(!pmd_is_valid_softleaf(old_pmd) && + !pmd_trans_huge(old_pmd)); =20 count_vm_event(THP_SPLIT_PMD); =20 @@ -3193,7 +3195,7 @@ static void __split_huge_pmd_locked(struct vm_area_st= ruct *vma, pmd_t *pmd, return; } =20 - if (is_huge_zero_pmd(*pmd)) { + if (is_huge_zero_pmd(old_pmd)) { /* * FIXME: Do we want to invalidate secondary mmu by calling * mmu_notifier_arch_invalidate_secondary_tlbs() see comments below @@ -3206,10 +3208,9 @@ static void __split_huge_pmd_locked(struct vm_area_s= truct *vma, pmd_t *pmd, return __split_huge_zero_page_pmd(vma, haddr, pmd); } =20 - if (pmd_is_migration_entry(*pmd)) { + if (pmd_is_migration_entry(old_pmd)) { softleaf_t entry; =20 - old_pmd =3D *pmd; entry =3D softleaf_from_pmd(old_pmd); page =3D softleaf_to_page(entry); folio =3D page_folio(page); @@ -3222,10 +3223,9 @@ static void __split_huge_pmd_locked(struct vm_area_s= truct *vma, pmd_t *pmd, anon_exclusive =3D softleaf_is_migration_read_exclusive(entry); young =3D softleaf_is_migration_young(entry); dirty =3D softleaf_is_migration_dirty(entry); - } else if (pmd_is_device_private_entry(*pmd)) { + } else if (pmd_is_device_private_entry(old_pmd)) { softleaf_t entry; =20 - old_pmd =3D *pmd; entry =3D softleaf_from_pmd(old_pmd); page =3D softleaf_to_page(entry); folio =3D page_folio(page); @@ -3417,7 +3417,7 @@ static void __split_huge_pmd_locked(struct vm_area_st= ruct *vma, pmd_t *pmd, } pte_unmap(pte); =20 - if (!pmd_is_migration_entry(*pmd)) + if (!pmd_is_migration_entry(old_pmd)) folio_remove_rmap_pmd(folio, page, vma); if (freeze) put_page(page); --=20 2.34.1 From nobody Sat Sep 26 22:53:02 2026 Received: from mail-pf1-f181.google.com (mail-pf1-f181.google.com [209.85.210.181]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5044E3BB66B for ; Fri, 28 Aug 2026 18:37:13 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.181 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1787942234; cv=none; b=HH1jWwpywn5anpAuI/R5+m25yF/0drLFSlgLaYXpmxeOf9cV/4UpkXJ6XFZU3tp6eu4flvIDlDFPLL7gcs2i+sG0yho38/JCu2NoPQtzJ3oPUy9sB39nXZWKrcDcZBQ7eixrOkUwrz6Nwn5NT+VKFSsjIz6UmlLRgRlfadwVZdk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1787942234; c=relaxed/simple; bh=Y/c5kVLFC0JKlo2iGPxBLFu39HOkVg1K1rkWGt9s8nw=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=fHov/g6jPcUVUlnfYV8VetYgGwqZpGlk3sPBc9HWu/+iPEU/s0VnBP8Luo1Bvo7o2R98aUJcsBmyT/oSrx0wWrtEEqMz03h9soVUy3ifPOrI1os9E7AYjwax/PVlLA9nk8ACyd2cdmDL5wSMxdSJduwByyq8v0J2jCgfh6FcyJ4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=RhfFRFOV; arc=none smtp.client-ip=209.85.210.181 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="RhfFRFOV" Received: by mail-pf1-f181.google.com with SMTP id d2e1a72fcca58-848643382fcso1516806b3a.1 for ; Fri, 28 Aug 2026 11:37:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1787942232; x=1788547032; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=1Jcn7GGjqrVUI/71BI99aJS+k5gR0ZeEnxqhpDzyiDE=; b=RhfFRFOVL97Uy2Qhc6vyB9PZ4+jAgG+tXH5p0c7x33/GnnJ7/B5VhQT8Iu0EDwHngj BWQb/WQ/s8ku2KUKgNBqKIbKh4UyAAf8+dP5Jz1nlNknQ97jBNO55LIUNMgagDyqxNWy uMqsBvlTdZLMr8N++mIlojHe0Fr/rBD3zydvp7ge7FT0rY7iMSK8ORdnOFpEU3iaBewE uaM5VtIOCyMWx7iHNJgeL0T2NvBK+ARzvqWX0OXWjBPEwBxjLH5GdvTQC6xIkYT1MfoM Bh83tmEUNOWj2Y+shlOQqRQ4cuasSa2SXuyKfAxAmOJEF21WmuGdC2lKRP7/rCY/HYdf nNEg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1787942232; x=1788547032; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=1Jcn7GGjqrVUI/71BI99aJS+k5gR0ZeEnxqhpDzyiDE=; b=Zr7MPPfTIZS6M1yZpRcnzb6VIcTQNhN0VOwOQNo+Zb6UqTOPulIDQmIPjbSMujviLN vKE6uX7AzWsqIpSQLMLtyN+wwMTr0GGoYugO7ZhoEVGUmcA35CB2lIkLz3tCNfGDfFys VlanYDsYtFXLI9uiY3cvvLcqlfoihH7qfphkDB36SgPK7b2dGlePBIs1Df8wOzUhefBe 5Bhxp45Tye/j1ZknRNxLuWpyeH8EOCfMewn+OnBgDxe3cyPCbVbyNct4in4qHstErrOW +hD7UnqfKu0X6fiGNrOdkLXt3o2LQdVYKwYMPk1zw6khYwbH7oHZRiqlBYAfOEbj5ZVh VEfg== X-Forwarded-Encrypted: i=1; AHgh+RpT/H8RaTcU0xJzE7QigjfradMQBS9FAmDhR53EEtD/hNg1gmbDKsCbyNRgr6GqAJKqIQZn4SFMZ3uIBTQ=@vger.kernel.org X-Gm-Message-State: AFuF++mp/HfUYwbBjiv38IyYcZsT0idLDRKabsNFOQCM3uOjd8VWUFKz k7CMtsdoffRUfoyXoN5nOfHnJLOCh5CdzvH0XBiTFjWu6OrBOzB/3O7+ X-Gm-Gg: AR+sD13oFYpic6/LVmQr5RU3XdbWlwUzrsXnETTp3cBT/PosHubF6jb+3kVVXuiLhDR ElTSoVUAdI/Bu7IzR9jhYPKfjrCVla+AtY92rNHC1+l94Cv2Myc7aRbfhWfH6he1x7z6qdMp5vV a80edsVlsFctbYhTvNuf0K1IQtiXTq4oEUj45T46xJAc/Z/2uxU6/ULu+UaRsuOWYyThxujzBqB MZbJodCeQbKyqlfQjsW0z1Fd2k9LcMnUJgf4RWhlDQLjhlyl5+3sM0uu2EwukbYlDSeGoKZi2LW Q/PIVuirgHw/a7+V1CQTRf4G8FbgepAvRbGsXeOKCtnsZzErxVAChbtaopgXBFDb7N+u4ewoEih cttLZP+51E9FUSgd0wAP1NkFpOCcfnpcUdHsAVy32X+H+0SvxcFAvW/vPb23k+KkWEWuUC91hH6 fW8OqFLvo0K5tDJ4AtUasp1VuBQjFNs7y7G00NXeIiQ/FNpB7YfNwkdRGkAiATdDPBnS85ra+Ol GOjP1qwofYGozv2DQ== X-Received: by 2002:a05:6a00:3311:b0:857:72ba:ff0f with SMTP id d2e1a72fcca58-85772bafff7mr2808955b3a.23.1787942232370; Fri, 28 Aug 2026 11:37:12 -0700 (PDT) Received: from aig-kmd-01.. ([120.133.49.20]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-856a2ea6fa3sm844230b3a.37.2026.08.28.11.37.03 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 28 Aug 2026 11:37:11 -0700 (PDT) From: Yin Tirui To: Andrew Morton , linux-mm@kvack.org Cc: David Hildenbrand , Lorenzo Stoakes , Dev Jain , Zi Yan , Baolin Wang , Barry Song , Lance Yang , Ryan Roberts , Nico Pache , Usama Arif , "Liam R . Howlett" , wangkefeng.wang@huawei.com, chenjun102@huawei.com, linux-kernel@vger.kernel.org, Yin Tirui Subject: [PATCH RFC 2/9] mm/huge_memory: add and use huge_zero_pmd_can_split() Date: Sat, 29 Aug 2026 02:33:12 +0800 Message-Id: <5e7add9f8103d3ebcc6edec70a8449e5336cb18b.1787941780.git.yintirui@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Only a huge zero PMD in an anonymous VMA is split, into a page table of shared zero page mappings. Any other huge zero PMD is simply unmapped. vm_normal_folio_pmd() returns NULL for a huge zero PMD, and the split path unmaps any entry which has no folio. So make the decision before the folio is looked up, in huge_zero_pmd_can_split(). No functional change intended. Signed-off-by: Yin Tirui --- mm/huge_memory.c | 36 +++++++++++++++++++++++------------- 1 file changed, 23 insertions(+), 13 deletions(-) diff --git a/mm/huge_memory.c b/mm/huge_memory.c index 8feabdcf6307..90d84f761619 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -3139,6 +3139,15 @@ static void __split_huge_zero_page_pmd(struct vm_are= a_struct *vma, pmd_populate(mm, pmd, pgtable); } =20 +/* + * Only a huge zero PMD in an anonymous VMA is split, into a page table of + * shared zero page mappings. Any other huge zero PMD is simply unmapped. + */ +static bool huge_zero_pmd_can_split(struct vm_area_struct *vma, pmd_t pmdv= al) +{ + return is_huge_zero_pmd(pmdval) && vma_is_anonymous(vma); +} + static void __split_huge_pmd_locked(struct vm_area_struct *vma, pmd_t *pmd, unsigned long haddr, bool freeze) { @@ -3163,6 +3172,20 @@ static void __split_huge_pmd_locked(struct vm_area_s= truct *vma, pmd_t *pmd, =20 count_vm_event(THP_SPLIT_PMD); =20 + /* + * FIXME: Do we want to invalidate secondary mmu by calling + * mmu_notifier_arch_invalidate_secondary_tlbs() see comments below + * inside __split_huge_pmd() ? + * + * We are going from a zero huge page write protected to zero small + * page also write protected so it does not seems useful to invalidate + * secondary mmu at this time. + */ + if (huge_zero_pmd_can_split(vma, old_pmd)) { + __split_huge_zero_page_pmd(vma, haddr, pmd); + return; + } + if (!vma_is_anonymous(vma)) { old_pmd =3D pmdp_huge_clear_flush(vma, haddr, pmd); /* @@ -3195,19 +3218,6 @@ static void __split_huge_pmd_locked(struct vm_area_s= truct *vma, pmd_t *pmd, return; } =20 - if (is_huge_zero_pmd(old_pmd)) { - /* - * FIXME: Do we want to invalidate secondary mmu by calling - * mmu_notifier_arch_invalidate_secondary_tlbs() see comments below - * inside __split_huge_pmd() ? - * - * We are going from a zero huge page write protected to zero - * small page also write protected so it does not seems useful - * to invalidate secondary mmu at this time. - */ - return __split_huge_zero_page_pmd(vma, haddr, pmd); - } - if (pmd_is_migration_entry(old_pmd)) { softleaf_t entry; =20 --=20 2.34.1 From nobody Sat Sep 26 22:53:02 2026 Received: from mail-pf1-f171.google.com (mail-pf1-f171.google.com [209.85.210.171]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7FDEA32D0E3 for ; Fri, 28 Aug 2026 18:37:22 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.171 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1787942243; cv=none; b=VyVCYL2U0exRCt/9fenYopkAVRJ6sbCi2sFYjim/78+jaNewe44nfRd83QXUpHkVB5u1yZqjcVjb5lmxzfJkpHaVK5ZC9mTsVmzkUGlLX9ZOFQSMSJtcjCgWDcSp0PTqSgIin74DxjDBivCoYpqueIt6bow1LJIzQechiz5dOG8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1787942243; c=relaxed/simple; bh=lVioQX+VsW1vNtJcHZ2TqlH0YfV3EGIpr3cLnID2MDs=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=qpruy8KQiDFta7tuhN78Gz2TGrsuxhn++CtZQO+pvdI8yZ3NTnUlTDG15b0taUwMpF1PPD5t7wLAbFPMCPYImVsSiV38410Ydq5YuPw6IezhnnRyajpmivK6Flk60amsg7USZYKxL9hug9IsiLhoPcnS+wfxjb63XhljNQdDBpw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=I8zSqy1d; arc=none smtp.client-ip=209.85.210.171 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="I8zSqy1d" Received: by mail-pf1-f171.google.com with SMTP id d2e1a72fcca58-854f8068301so901792b3a.0 for ; Fri, 28 Aug 2026 11:37:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1787942242; x=1788547042; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=zfcyzCAqaQXbJL0WVp7G3Q9umeDsMlNHg+F0EE9N96I=; b=I8zSqy1dmLHUWhSbHK8aXWYatE0x/O8B2zOpgzMGTYBBkJV6gHbkyEXgxA3JSHfzCN 9+NGDZlLASN684eZgcpB6yZiqQBRw7l5Wb/iZRc9t230DIAkO2/Vj2fHih59OFt2L7R7 9R4n085SCLCZR4J4LbFJ7H8FES0gHs/f/6mN/DSeACLJrVIEmJSlK0+nKN4C3+q4pw0s NRj5r4HZhTJp0wypBGqCtxUNKRr4eD0oAASYy74WmefBhNZspmX3p78jBasdf1otuyEK PLhJnN/mXNttE6VsZuL8jfLrnxdBVW3t5TPIxcRRX8vAhH6XBsDG7d8QIP1lE7qaAqGv PHIw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1787942242; x=1788547042; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=zfcyzCAqaQXbJL0WVp7G3Q9umeDsMlNHg+F0EE9N96I=; b=rn5QkcR3xC/bWWzMtvpSUY495SudZkGl3gBzSguz/zXVkc0uvuQ1jyraMgq/E73kZ+ ftPvsd0tkRkBHfrAXCEVvTSFqQ9FqF2z+++nmju1xB21/Q52mGH/FtTiEaIKbKDKgRMM FQaUx5Jp3qKCkpqdt8Yli6RrxsZusRO6mkJarz+cQeBAYdtm9UScgBkTIvC0VTOsYnYz sKsWBpS/L4Rh1IDQnB4sBzOw3TXPmSB9nHc159lEeR7oKbpp1CAc8zsmCDAFcUuSKrK3 tn3WT1d1Ov8xZycj+aK2PlVQqL25WpMA12KNAPVbxtrB8XWAt+/lbvya3fNS5xmV1m+7 Q7WQ== X-Forwarded-Encrypted: i=1; AHgh+RrFvVZ2uj0Zqy6wIhEpi35Z37G9PiHOaE7h77lZ04WaEAIbe7+C8EX923Yle5iy9+t17/5jXebts9C+kBc=@vger.kernel.org X-Gm-Message-State: AFuF++kVdxhEfHmDkY8vBo9lSEd/7j7CQPYkYNQ38U/S0ttloNQjx5II 1wlbGLr4Uy81/vcaQkGJ7geuQF8cvEvXKDchVIIj2Kz7Av4hsiKcfbCq X-Gm-Gg: AR+sD11oJf37eOB0z03zm0Qfp3xP8LVP541FkdoQlcmecBARYX0G1X6ciZWmRM+Ijco ecEm1QTJ47PdhXVrxmCRj7ZO90BPVM9v6ky5lZimgsXdPlr3bU4MToEKxjbON3poieWbgy09/wq QRtqA8GWuOyrkvLCezDAGEYE1sfINwnEAZSJumwAVpCKnr5t8C84lbE+V5wogdi4dOpm90uy/xe 3KF5/TC0l3RZDaWfs+CsfnDYJ1bFkKBMnu8+B3+uvEyqhHLiC+tmZy16TVO9MkFT+PbQD0cZaUe hZLqpyPwpnIecs1BSOjfBg6ouBCzbyKFIvRpuepzclhHOzH6rblu9/FhmWG+AxpezAjXsvaAVaY TF++ym8wEqoPJDxf/qfavQoVav0czkNRxEEJ0+vPUz6DHs0v++Mw4EFbBBYpS+Vv6q8fAGuNONB MY6r8gdDeTGBn02ZdXF0zDYpe0Ba5lXF+UT65ysOHaCBw+NQYgxLBpiC2r9GslYEyE32Gg8LCHO aRxbyKHFnCm5NY+EPE= X-Received: by 2002:a05:6a00:9459:b0:857:727c:a1f8 with SMTP id d2e1a72fcca58-857727ca426mr2412271b3a.26.1787942241652; Fri, 28 Aug 2026 11:37:21 -0700 (PDT) Received: from aig-kmd-01.. ([120.133.49.20]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-856a2ea6fa3sm844230b3a.37.2026.08.28.11.37.12 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 28 Aug 2026 11:37:21 -0700 (PDT) From: Yin Tirui To: Andrew Morton , linux-mm@kvack.org Cc: David Hildenbrand , Lorenzo Stoakes , Dev Jain , Zi Yan , Baolin Wang , Barry Song , Lance Yang , Ryan Roberts , Nico Pache , Usama Arif , "Liam R . Howlett" , wangkefeng.wang@huawei.com, chenjun102@huawei.com, linux-kernel@vger.kernel.org, Yin Tirui Subject: [PATCH RFC 3/9] mm/huge_memory: add and use unmap_huge_pmd_entry() Date: Sat, 29 Aug 2026 02:33:13 +0800 Message-Id: X-Mailer: git-send-email 2.34.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Only a huge PMD entry mapping an anonymous folio is split. Everything else is unmapped and comes back on the next fault. Put that in unmap_huge_pmd_entry(). Old and new differ only for a device private entry in a non-anonymous VMA, which nothing in the tree can produce. Signed-off-by: Yin Tirui --- mm/huge_memory.c | 84 ++++++++++++++++++++++++++++++++---------------- 1 file changed, 56 insertions(+), 28 deletions(-) diff --git a/mm/huge_memory.c b/mm/huge_memory.c index 90d84f761619..aefd62827139 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -3148,6 +3148,52 @@ static bool huge_zero_pmd_can_split(struct vm_area_s= truct *vma, pmd_t pmdval) return is_huge_zero_pmd(pmdval) && vma_is_anonymous(vma); } =20 +/** + * unmap_huge_pmd_entry() - Unmap a huge PMD entry rather than splitting i= t. + * @vma: The VMA @pmd belongs to. + * @haddr: The PMD-aligned address @pmd maps. + * @pmd: Pointer to the huge PMD entry. + * @folio: The folio @pmd describes, or NULL if it describes none. + * @is_present: Is @pmd a present entry rather than a softleaf entry? + * + * Only anonymous folios are rebuilt at PTE level when a huge PMD entry is + * split. Everything else is unmapped here and faulted back in on the next + * access. + */ +static void unmap_huge_pmd_entry(struct vm_area_struct *vma, + unsigned long haddr, pmd_t *pmd, struct folio *folio, + bool is_present) +{ + struct mm_struct *mm =3D vma->vm_mm; + pmd_t old_pmd; + + old_pmd =3D pmdp_huge_clear_flush(vma, haddr, pmd); + /* + * We are going to unmap this huge page. So + * just go ahead and zap it + */ + if (arch_needs_pgtable_deposit()) + zap_deposited_table(mm, pmd); + + if (!folio) + return; + + if (is_present) { + struct page *page =3D pmd_page(old_pmd); + + if (!folio_test_dirty(folio) && pmd_dirty(old_pmd)) + folio_mark_dirty(folio); + if (!folio_test_referenced(folio) && pmd_young(old_pmd)) + folio_set_referenced(folio); + folio_remove_rmap_pmd(folio, page, vma); + } + + add_mm_counter(mm, mm_counter_file(folio), -HPAGE_PMD_NR); + + if (is_present) + folio_put(folio); +} + static void __split_huge_pmd_locked(struct vm_area_struct *vma, pmd_t *pmd, unsigned long haddr, bool freeze) { @@ -3187,34 +3233,16 @@ static void __split_huge_pmd_locked(struct vm_area_= struct *vma, pmd_t *pmd, } =20 if (!vma_is_anonymous(vma)) { - old_pmd =3D pmdp_huge_clear_flush(vma, haddr, pmd); - /* - * We are going to unmap this huge page. So - * just go ahead and zap it - */ - if (arch_needs_pgtable_deposit()) - zap_deposited_table(mm, pmd); - if (vma_is_special_huge(vma)) - return; - if (unlikely(pmd_is_migration_entry(old_pmd))) { - const softleaf_t old_entry =3D softleaf_from_pmd(old_pmd); - - folio =3D softleaf_to_folio(old_entry); - } else if (is_huge_zero_pmd(old_pmd)) { - return; - } else { - page =3D pmd_page(old_pmd); - folio =3D page_folio(page); - if (!folio_test_dirty(folio) && pmd_dirty(old_pmd)) - folio_mark_dirty(folio); - if (!folio_test_referenced(folio) && pmd_young(old_pmd)) - folio_set_referenced(folio); - folio_remove_rmap_pmd(folio, page, vma); - add_mm_counter(mm, mm_counter_file(folio), -HPAGE_PMD_NR); - folio_put(folio); - return; - } - add_mm_counter(mm, mm_counter_file(folio), -HPAGE_PMD_NR); + const bool is_present =3D pmd_present(old_pmd); + + if (vma_is_special_huge(vma) || is_huge_zero_pmd(old_pmd)) + folio =3D NULL; + else if (is_present) + folio =3D page_folio(pmd_page(old_pmd)); + else + folio =3D softleaf_to_folio(softleaf_from_pmd(old_pmd)); + + unmap_huge_pmd_entry(vma, haddr, pmd, folio, is_present); return; } =20 --=20 2.34.1 From nobody Sat Sep 26 22:53:02 2026 Received: from mail-pf1-f176.google.com (mail-pf1-f176.google.com [209.85.210.176]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D61693AE6FD for ; Fri, 28 Aug 2026 18:37:31 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.176 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1787942253; cv=none; b=WsuoUwVz5Vmw7VzPFNaEX/BjG1bk50rNn1TZ+2TINrJMyXOcOtLA5OuVYXVz3CK8JsYhgdiN8aX0dYYZBymtDIeaV0VmSIwtsNsGoJFvaYSCrupqd3FUYrueH3sBQloOA6tgyS+YZYeThxL4PEBz16D7IjEyXdvJVRW9xRXyi1U= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1787942253; c=relaxed/simple; bh=aNdGxo5FUbZ/b3hdJ54tgwnPkKj2RIxatu8EIqBxmec=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=uHVM18Avf+m0MB+bUSMdqmJZXmgxL3dJgMfvkeq9SdpXNwjAU6locc43mBwqdDUg1Rm5AllwskshZbguBFXjlx177+4LXMtysn5dJ2FFi7Ondxg36gXh9Q7hr/pkAeCPP8eboEsivomaok6J/cS6pf0UM8dlhB7xoKF713XzUIc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=JUMW65Hb; arc=none smtp.client-ip=209.85.210.176 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="JUMW65Hb" Received: by mail-pf1-f176.google.com with SMTP id d2e1a72fcca58-85339ed040aso1257315b3a.1 for ; Fri, 28 Aug 2026 11:37:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1787942251; x=1788547051; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=qYGUS2yq4w52aMYmHXMMhpQA8K1oYL7onWQGdlT1JgM=; b=JUMW65HbDG1vrqIJpaw2S8tkCbz5qkubgXnW2Ct9qdo7e9JYFVTba4E+sWzXFQFHYz jNbZd5pCcU89JCAEv+5WZZgQ1HSG5M2Sl+gTvnIKVjRqwxjUjiXTpid8IqYUpeT9GYEt jI8xYuFuBaC3GSmMQ5pfTf2dsWlytKyzbEv0B9PAPCRWrfU/s4n0uI/Zp+ykokNyZgRZ 2IKwObTnRHgY8YR34odt/SLumY+xpQ+Nslp06WsaeFafAqERUJeosqerRxpp3UuOA8VB n0+eNTRnbK5NiCQDtAgtqztetuHm4qpM3PUJJ1XwLT/hUGfaqNiEFWzX7GPEqLabDTWK Bq7g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1787942251; x=1788547051; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=qYGUS2yq4w52aMYmHXMMhpQA8K1oYL7onWQGdlT1JgM=; b=L4TEmnFwhw78yatbyUedd0SlE0SqQgsQvt2BbqAZJE3ALe/m7I9QQWjUCSce+uymrX bbv5TOuFYVCahelak0lQLgC2sXpixl2r9g4W8mj5GjT2Pygmf6SjmvA9xDc4Ue6rUKdu P4TLrZAl7nW9MBTLiCoNYj/Qpj4CTpl1JvzF9LPvo2dZfYUh871/WiQyrGVfTHRn2NEg wjvQAPbWan/PeolnDufwuuwhfz24HUJrai0yY5IvYgB53axu8bFSxH9J774ljwwUZlgL oUpS2W6Uxg83DyE3QeUtC9phIu9fcnuJkAtGT2DQDlmWYhdW580Il9uUuNPJLxXh5ijH SHIg== X-Forwarded-Encrypted: i=1; AHgh+Ro/j9MLWgABON2d0aMyya/GrZgOSN9E/9ksz5pB2K+8mEq6H9a5CrngatrdG8q6fgYh3ZkPm2tQo5g0Pa4=@vger.kernel.org X-Gm-Message-State: AFuF++lpPbnkYnGWrIE+vftRyxyafMA2uwu89KHRr+7A4LinpLwrR/qt ahHI5/wl/7Vk7AlCgWwM0jRpP4meQvCqDn1pHxxZAYMlhUkLKM9tdZXI X-Gm-Gg: AR+sD11tlsrEtasfjWOBpjcbO53IhU9DPESjFAhE70sjQr7zFO0/8PunKz1taF8Na8z hxap1n/TU3fCnzRtbjj7K0OyTZUB7csgYW8l+0A/z4coHCqzhn3BeJgK4gpxmylj3HHgK4PbnPI TDPOGI362wmxuPDiLVecmAIjsarteftIumoBN7X+6XVzDhMRteso3Tl6Mfbl9MdzraS8QtLcLK1 0m88sv0mnnL05wwgTeXShPa9aoRaVa7PZdXoBbiRDplOmU2FUVaaE3uCV4Q5TaChYTvibrvvwbr V0fu5zu/B1EHT7uvQJv5p34EJ+35miG5+B3UiAcw0dnrGYJbOVFzoDe+G3hfVVxozyX9xbn96Uk e8288eutOGGXbPj8OTtKguybowokEH7P9MpflgRzD+aGI8zcAXwB7d2mUnP/WRTxoo6WIJLYTEL dtPXENeFQ6WXemb/OAfuRdZAEMWS+BFmhDuvT3pkmrUSZCXG2muDDB+Ccbv6VE3XPdxeOLIOQ61 DDQ+SX3bTZju1X5Uw== X-Received: by 2002:a05:6a00:368b:b0:851:8baf:5b29 with SMTP id d2e1a72fcca58-8562a6deb93mr17519619b3a.10.1787942250996; Fri, 28 Aug 2026 11:37:30 -0700 (PDT) Received: from aig-kmd-01.. ([120.133.49.20]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-856a2ea6fa3sm844230b3a.37.2026.08.28.11.37.22 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 28 Aug 2026 11:37:30 -0700 (PDT) From: Yin Tirui To: Andrew Morton , linux-mm@kvack.org Cc: David Hildenbrand , Lorenzo Stoakes , Dev Jain , Zi Yan , Baolin Wang , Barry Song , Lance Yang , Ryan Roberts , Nico Pache , Usama Arif , "Liam R . Howlett" , wangkefeng.wang@huawei.com, chenjun102@huawei.com, linux-kernel@vger.kernel.org, Yin Tirui Subject: [PATCH RFC 4/9] mm/huge_memory: use normal_or_softleaf_folio_pmd() in the PMD split path Date: Sat, 29 Aug 2026 02:33:14 +0800 Message-Id: X-Mailer: git-send-email 2.34.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Get the folio once with normal_or_softleaf_folio_pmd() and decide the deposit once with has_deposited_pgtable(), as zap_huge_pmd() does. That makes split and zap classify an entry the same way, and drops vma_is_special_huge() from this path. Behaviour changes only where the entry and the VMA flags disagree, which no in-tree path produces. Signed-off-by: Yin Tirui --- mm/huge_memory.c | 21 ++++++++++++--------- 1 file changed, 12 insertions(+), 9 deletions(-) diff --git a/mm/huge_memory.c b/mm/huge_memory.c index aefd62827139..b2ede9a6ae5d 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -3172,7 +3172,7 @@ static void unmap_huge_pmd_entry(struct vm_area_struc= t *vma, * We are going to unmap this huge page. So * just go ahead and zap it */ - if (arch_needs_pgtable_deposit()) + if (has_deposited_pgtable(vma, old_pmd, folio)) zap_deposited_table(mm, pmd); =20 if (!folio) @@ -3199,6 +3199,7 @@ static void __split_huge_pmd_locked(struct vm_area_st= ruct *vma, pmd_t *pmd, { struct mm_struct *mm =3D vma->vm_mm; pmd_t old_pmd =3D *pmd; + const bool is_present =3D pmd_present(old_pmd); struct folio *folio; struct page *page; pgtable_t pgtable; @@ -3232,16 +3233,18 @@ static void __split_huge_pmd_locked(struct vm_area_= struct *vma, pmd_t *pmd, return; } =20 - if (!vma_is_anonymous(vma)) { - const bool is_present =3D pmd_present(old_pmd); + folio =3D normal_or_softleaf_folio_pmd(vma, haddr, old_pmd, is_present); =20 - if (vma_is_special_huge(vma) || is_huge_zero_pmd(old_pmd)) - folio =3D NULL; - else if (is_present) - folio =3D page_folio(pmd_page(old_pmd)); - else - folio =3D softleaf_to_folio(softleaf_from_pmd(old_pmd)); + /* + * A non-present entry which is neither a migration nor a device + * private entry is corrupt, and pmd_to_softleaf_folio() has already + * warned about it. Leave it alone rather than act on a PFN which + * means nothing. + */ + if (unlikely(!is_present && !folio)) + return; =20 + if (!vma_is_anonymous(vma)) { unmap_huge_pmd_entry(vma, haddr, pmd, folio, is_present); return; } --=20 2.34.1 From nobody Sat Sep 26 22:53:02 2026 Received: from mail-pf1-f178.google.com (mail-pf1-f178.google.com [209.85.210.178]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 22B0F383993 for ; Fri, 28 Aug 2026 18:37:40 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.178 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1787942262; cv=none; b=U3vOgaC8b86zPYFWbZQnr85XNfwM8iX4CUx1NfYOE4tTsJzBdOD6SGkg1yZUqHyHkX7CXWXJx7y5gmnzPCgzG+WithO0voMJRdsJyrPNuy8uWWMiBdaztw7+bYYrECo7TIi9n/o825ctiWaz0tZ2Zgpb5P4o+O/Hnd72+7ZefD0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1787942262; c=relaxed/simple; bh=ud+62vNrtpRD4J3vyDMNpiYp6k5bWYTZ9esKw8TJHoE=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=fLxJYnvImxJyzNQqFySJDegkaOlMttaDmLxfzwqT8C0DAdxyYjcinIkdQLsHGtbM0bdRCe/kwHUqIRL2hfjSS6/58mFkSmCSUXQH4muO8QhFjo5WS0sw85azCXJ4uNOKl1GkKTfSrgAPNh3vlSDSzgUvmdIXMEjspm+6JnYRlZc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=pGmDBETA; arc=none smtp.client-ip=209.85.210.178 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="pGmDBETA" Received: by mail-pf1-f178.google.com with SMTP id d2e1a72fcca58-84f38f3b36eso1289106b3a.1 for ; Fri, 28 Aug 2026 11:37:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1787942260; x=1788547060; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=p42zDyAV7UZaUy//KUdiOCk0RqqsUcLlIr4kopLObvw=; b=pGmDBETAGqQmRuswFpfy0glDWr3lQvFTVJH34mSK6oZ9IsmP+dpw4lyGIzQkOuiE79 8SQ7AOglZG6FUhavKhOu8H5R7avIz7ujwx0Km0BDZlzlYu90/UyV1sjDr7E+QmCjLTv+ PfsJGUSyxCMd0uSnxVXpnhWCVrAT+cmgWOKVVMrk41gHuqqizqkeTDD9hx/0gwKi6vQg WoF6TLFYeukr95Bmsz6pjmyRcLYuTkC4vQN//x5fk7WKUosfjoFEPH6QggRaxEP/AG30 R4o36WmOnfamqLqOn4JNW6sf8Tj0PW02vtH3h6+jneQxMHOy2UmkdfbRsSH0FS/iZntc XtQg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1787942260; x=1788547060; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=p42zDyAV7UZaUy//KUdiOCk0RqqsUcLlIr4kopLObvw=; b=icawQfmw9pxNMFuAJqAV8q9zJRl3W0aODzEPQkRmXAdWxfwwWYo3V/8e+pfA3jmZ2k QnLgIZqGmgXgHJLx4gz22lan4QMFrfMM1CYCNUlbyW7mF5CDC/+0ai4HksfJJC+p+AW/ bNFWgWuy+BZfaxt/UiDQdmkjhIpC0sJDfbGZqrWYlCtt9YAyifbwQ9jonehrw/BfwcS7 Zm5kj9D8jcHh/Iux4/D2ZMJMOhYddY+3mPD6frAnIHUwB5CAhEc8J8w2osA8WM6XiX9Y 1QyKgyVV66UX4cxo3K22E2pWHqbUOiuuc+F7QFT+5y6BpZzZF1RbZKYJ8FTQOSPmBUAE JS/w== X-Forwarded-Encrypted: i=1; AHgh+RqzorpURD+m6z3HoanBy2vENHYTXez+XV7QM0b/T1zDQ7U98hKQeWCo0wUxZaK2ptG/P5BxYG5fau7BoPw=@vger.kernel.org X-Gm-Message-State: AFuF++lylEusnxRuoe8IltxhN/b+7d+VDRFieYfFbiyDuWo7JPjY9LCQ j99HG9ib57HxxkQSLr8WfZDmIrpeWd11PPL788W1EVmR31DpweQIn0sm X-Gm-Gg: AR+sD12HcvWunCmWYl6cDQTgO9DTHEHZ3p6xTqlDorrgxjzPsyXbEc0VAwRck08gPeR V9ZMyLCFd1Vs627oYKBAWwN1pYzUq40STEtzZon3r3h7t0WMqNY6IQ3NKQYlE1P/CwA1aIRluyZ +pUXZgnjLeBc/iP7SQYMq6mQZRx2PiXwTWVeCGHZo3trJEUfqlFq8AgEUDw86p0AQ+r6JzKZbZX tN+JGRbu7qUqMfcOb4olKot0bE403t2aKsRLC1klopWDwFuZ0ezjwdPhr8GLRz9Y5nB/AULQRos Hd6DW5yKwfFLiUHh7txvi+XICLDBF59falzbYDsSGqq4Jz7mHStLN/5p63yCQT4yxNI7wLVEp47 Jh02t0PYIC0gJMsRjG87aH/Bb5gV5q4pnJblYq39MQgDZn+kOS9u2/7Jll7I6WxP+eXmd6vv92a 5UNWVeUU2Fz/vqXszj4FgI/Rjoz5XENY5Rk6KV/5WAbFOxHWXDX8Puw482fkXOmzOfjlbMUeGDt C3u1OxjQlmOGzA89/A= X-Received: by 2002:a05:6a00:4f92:b0:845:dfef:75b8 with SMTP id d2e1a72fcca58-8562c04ef5emr16403554b3a.15.1787942260279; Fri, 28 Aug 2026 11:37:40 -0700 (PDT) Received: from aig-kmd-01.. ([120.133.49.20]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-856a2ea6fa3sm844230b3a.37.2026.08.28.11.37.31 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 28 Aug 2026 11:37:39 -0700 (PDT) From: Yin Tirui To: Andrew Morton , linux-mm@kvack.org Cc: David Hildenbrand , Lorenzo Stoakes , Dev Jain , Zi Yan , Baolin Wang , Barry Song , Lance Yang , Ryan Roberts , Nico Pache , Usama Arif , "Liam R . Howlett" , wangkefeng.wang@huawei.com, chenjun102@huawei.com, linux-kernel@vger.kernel.org, Yin Tirui Subject: [PATCH RFC 5/9] mm/huge_memory: dispatch on the folio when splitting a huge PMD Date: Sat, 29 Aug 2026 02:33:15 +0800 Message-Id: X-Mailer: git-send-email 2.34.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Only anonymous folios carry the anon rmap and PageAnonExclusive() state needed when rebuilding PTEs, so decide the split path from folio_test_anon() rather than vma_is_anonymous(). Treat a NULL folio as nothing to rebuild. This only changes behavior for folio/VMA mismatches, which are not produced by any in-tree path. Signed-off-by: Yin Tirui --- mm/huge_memory.c | 9 ++------- 1 file changed, 2 insertions(+), 7 deletions(-) diff --git a/mm/huge_memory.c b/mm/huge_memory.c index b2ede9a6ae5d..1cd8878edc1d 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -3244,7 +3244,7 @@ static void __split_huge_pmd_locked(struct vm_area_st= ruct *vma, pmd_t *pmd, if (unlikely(!is_present && !folio)) return; =20 - if (!vma_is_anonymous(vma)) { + if (!folio || !folio_test_anon(folio)) { unmap_huge_pmd_entry(vma, haddr, pmd, folio, is_present); return; } @@ -3254,14 +3254,12 @@ static void __split_huge_pmd_locked(struct vm_area_= struct *vma, pmd_t *pmd, =20 entry =3D softleaf_from_pmd(old_pmd); page =3D softleaf_to_page(entry); - folio =3D page_folio(page); =20 soft_dirty =3D pmd_swp_soft_dirty(old_pmd); uffd_wp =3D pmd_swp_uffd(old_pmd); =20 write =3D softleaf_is_migration_write(entry); - if (PageAnon(page)) - anon_exclusive =3D softleaf_is_migration_read_exclusive(entry); + anon_exclusive =3D softleaf_is_migration_read_exclusive(entry); young =3D softleaf_is_migration_young(entry); dirty =3D softleaf_is_migration_dirty(entry); } else if (pmd_is_device_private_entry(old_pmd)) { @@ -3269,7 +3267,6 @@ static void __split_huge_pmd_locked(struct vm_area_st= ruct *vma, pmd_t *pmd, =20 entry =3D softleaf_from_pmd(old_pmd); page =3D softleaf_to_page(entry); - folio =3D page_folio(page); =20 soft_dirty =3D pmd_swp_soft_dirty(old_pmd); uffd_wp =3D pmd_swp_uffd(old_pmd); @@ -3321,7 +3318,6 @@ static void __split_huge_pmd_locked(struct vm_area_st= ruct *vma, pmd_t *pmd, */ old_pmd =3D pmdp_invalidate(vma, haddr, pmd); page =3D pmd_page(old_pmd); - folio =3D page_folio(page); if (pmd_dirty(old_pmd)) { dirty =3D true; folio_set_dirty(folio); @@ -3332,7 +3328,6 @@ static void __split_huge_pmd_locked(struct vm_area_st= ruct *vma, pmd_t *pmd, uffd_wp =3D pmd_uffd(old_pmd); =20 VM_WARN_ON_FOLIO(!folio_ref_count(folio), folio); - VM_WARN_ON_FOLIO(!folio_test_anon(folio), folio); =20 /* * Without "freeze", we'll simply split the PMD, propagating the --=20 2.34.1 From nobody Sat Sep 26 22:53:02 2026 Received: from mail-pf1-f175.google.com (mail-pf1-f175.google.com [209.85.210.175]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7D5BF3B894B for ; Fri, 28 Aug 2026 18:37:50 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.175 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1787942271; cv=none; b=UwQWd5CnBwzghIljkmjPjm7wAHV8qN0Sy1htZ7XmfMIXfJpHS2fQvv/1bHz9CnZf/inm1Zm1beJnZ2/Z9SJ3mAOoHvmkK/UlD7fOYnPk9BwMflpNyevFNrNhzrtpEE7NR7ibBlDAcwGY9m0xStsQ/tu7BxwS9fjbqcm+Jy9n5u4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1787942271; c=relaxed/simple; bh=6NqEHEqHyjECT3AL8PMFJmkyHVP8IPxNQJWyBT5Jo2c=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=smlctoi2WHSVGvdfrlkd5TrIDCMBIHxu0rbE2FB2h1kHp2ZgQdp6c7QUCkAjryxiojIEk1FDlRa7C6qIGJUwzlOdx1wHPf+wWBdq1ACskV5qstugcahs/fc7oltzPGBYubOP9m2pKxJqMnFiyl5UXJ2cQgpXP12iclxgCY4w6rs= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=O08xongu; arc=none smtp.client-ip=209.85.210.175 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="O08xongu" Received: by mail-pf1-f175.google.com with SMTP id d2e1a72fcca58-84f38f3b36eso1289247b3a.1 for ; Fri, 28 Aug 2026 11:37:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1787942270; x=1788547070; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=Zk9fbsTtY0uQCDGritDt2l0ASVNz5DfiHm7Yt38aS10=; b=O08xonguB+uUl6V8EXOKsSRZpUUumS6mRB3rMmVMRzV6Via6obL6TMxIjMFFDymdPH HgOkYxzMnYuuPdsZA6zkGo2+SWvOsqgRW+iKleEv/87/nv54uTcB1+PsCNa7SuCQ9j0C vY6iHEu9jzp3NlEM6d5tQ15TJbcmFrbEfUgUyaU+A7yZO08hilWa4CXQ1oXEMUVYudsD z06V3WaTD21Kq/nPUxWk3roXg0h8sdYk/ls4nSh2InDItuFmPvTaZTx7QffMWWtkN8F8 hGDqc8+mPPM/tr6jBrawdYsvC3x6a9grji9TyB0IGH+/gbLTuLc9rYd/MusmsY3x0LCv mXMA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1787942270; x=1788547070; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=Zk9fbsTtY0uQCDGritDt2l0ASVNz5DfiHm7Yt38aS10=; b=DaPWrWBVIVkNkrJhaHOXIyOGaeeQkSnF4+IDZrS+EYFe5HdYrEG58gAShtgSQlc/69 nMR+GFaQJufoZSIz9A55Q+8UXPBKMxwtNRrlT1tI5m4bD7z0aWeNujoiGzT7usyEkzAa /5Ow1rHKoNTtMeiJLx0E0SFwiDhaKRR+EXAihohkT97d1umw2AGdKSaHHbZ6hvTmowAq lIJvy7AZyrbIJ2E2F/02roCZstiDHIBlU2AdMNUMuUbjT6cVCuRIB2XUpAz43gsuzc+y MfLA/G/yqAExGyVFJEi3/ig4OCkvM0RFlAddgttqbwxcdvOpUPe/gH+u1/azqu13iGIO c2hQ== X-Forwarded-Encrypted: i=1; AHgh+RpMNdiXamby0gL9q5ODKKkBpUOHXQrR8rzkTCKUFw6gJIAKTp1DMtQ4lcU7HZPb1PZFHuSg7na+yxTYfcA=@vger.kernel.org X-Gm-Message-State: AFuF++ndcQX5ipl6UCLnCV1ii/UeNcu/dlYXLyT2i0yNGjzn6nZyNiMX 3ueVtkxvC+UH2ME9A+3qruDmusO50zoA2xSmQl5+PABbt6xcoFwkHHZu X-Gm-Gg: AR+sD130ABEhIcIxTUlf8Ddl74mR52XUqZj+pEK72vXp5rLIlyJ+tatEynigPNY5m2p 630VdOIoZltrPPXKKqSNpzgNemUCmeqtel2V1k1vVIO472/yBQn5r6luL5CRy6/nYBzzB9OFlU5 FmjpIh5kEyp8fc9+5/kwT3CENAfxsUzQrXyfjACI734BxsGjvj4IdJYesg1gFUcA6UC/erwH5NW VFAFTCYbS0fRCtJcmlpARSwOFW7z9X9WQzVyCkrBfyWaCvREorHbkcEw5Z66dqlUUWjSQAfxGwn wtugffQCFPc/W2wzUpnL4tdmQ7cWA4QakcG8bMO54bpBlwdk01iuyeD9WY2POZokJUrvUiOmAvJ jHYpuSwt5kMjddGLaIljnXmIHlDdPFxgog7zOA7cwELwHuIDv2HamIl9ZlI1hSQQ6mKw1TlI2rd SW8Q3rZCwPVkRuFA4F0GsyJSDs8FQLOO4NVaU7mYJddFt241Eoz7FyUFcX5bmDH7JCd412lmL8J MYlMoiaohHzJMnP4g== X-Received: by 2002:a05:6a00:3cd6:b0:857:7317:cff2 with SMTP id d2e1a72fcca58-8577317d17cmr2655096b3a.19.1787942269557; Fri, 28 Aug 2026 11:37:49 -0700 (PDT) Received: from aig-kmd-01.. ([120.133.49.20]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-856a2ea6fa3sm844230b3a.37.2026.08.28.11.37.40 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 28 Aug 2026 11:37:49 -0700 (PDT) From: Yin Tirui To: Andrew Morton , linux-mm@kvack.org Cc: David Hildenbrand , Lorenzo Stoakes , Dev Jain , Zi Yan , Baolin Wang , Barry Song , Lance Yang , Ryan Roberts , Nico Pache , Usama Arif , "Liam R . Howlett" , wangkefeng.wang@huawei.com, chenjun102@huawei.com, linux-kernel@vger.kernel.org, Yin Tirui Subject: [PATCH RFC 6/9] mm/huge_memory: add and use split_huge_pmd_anon_rmap() Date: Sat, 29 Aug 2026 02:33:16 +0800 Message-Id: <1022e5cfad8bb7e32946da7e881534730757bbbd.1787941780.git.yintirui@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" The anon-exclusive handling and the PTE-level rmap conversion are written twice, once for present entries and once for device private ones. Factor them into one helper. It returns whether the mapping may still be frozen instead of writing the caller's freeze back. No functional change intended. Signed-off-by: Yin Tirui --- mm/huge_memory.c | 90 +++++++++++++++++++++++++----------------------- 1 file changed, 47 insertions(+), 43 deletions(-) diff --git a/mm/huge_memory.c b/mm/huge_memory.c index 1cd8878edc1d..e0083a9e89b8 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -3194,6 +3194,43 @@ static void unmap_huge_pmd_entry(struct vm_area_stru= ct *vma, folio_put(folio); } =20 +/* + * Convert the folio's PMD-level anonymous rmap into PTE-level ones. + * + * Without "freeze", we'll simply split the PMD, propagating the + * PageAnonExclusive() flag for each PTE by setting it for + * each subpage -- no need to (temporarily) clear. + * + * With "freeze" we want to replace mapped pages by + * migration entries right away. This is only possible if we + * managed to clear PageAnonExclusive() -- see + * set_pmd_migration_entry(). + * + * In case we cannot clear PageAnonExclusive(), split the PMD + * only and let try_to_migrate_one() fail later. + * + * See folio_try_share_anon_rmap_pmd(): invalidate PMD first. + * + * Returns: whether the mapping may still be frozen. + */ +static bool split_huge_pmd_anon_rmap(struct folio *folio, struct page *pag= e, + struct vm_area_struct *vma, unsigned long haddr, bool freeze, + bool anon_exclusive) +{ + rmap_t rmap_flags =3D RMAP_NONE; + + if (freeze && + (!anon_exclusive || !folio_try_share_anon_rmap_pmd(folio, page))) + return true; + + folio_ref_add(folio, HPAGE_PMD_NR - 1); + if (anon_exclusive) + rmap_flags |=3D RMAP_EXCLUSIVE; + folio_add_anon_rmap_ptes(folio, page, HPAGE_PMD_NR, vma, haddr, + rmap_flags); + return false; +} + static void __split_huge_pmd_locked(struct vm_area_struct *vma, pmd_t *pmd, unsigned long haddr, bool freeze) { @@ -3275,23 +3312,12 @@ static void __split_huge_pmd_locked(struct vm_area_= struct *vma, pmd_t *pmd, anon_exclusive =3D PageAnonExclusive(page); =20 /* - * Device private THP should be treated the same as regular - * folios w.r.t anon exclusive handling. See the comments for - * folio handling and anon_exclusive below. + * Device private folios are treated the same as regular folios + * w.r.t. anon exclusive handling, see + * split_huge_pmd_anon_rmap(). */ - if (freeze && anon_exclusive && - folio_try_share_anon_rmap_pmd(folio, page)) - freeze =3D false; - if (!freeze) { - rmap_t rmap_flags =3D RMAP_NONE; - - folio_ref_add(folio, HPAGE_PMD_NR - 1); - if (anon_exclusive) - rmap_flags |=3D RMAP_EXCLUSIVE; - - folio_add_anon_rmap_ptes(folio, page, HPAGE_PMD_NR, - vma, haddr, rmap_flags); - } + freeze =3D split_huge_pmd_anon_rmap(folio, page, vma, haddr, + freeze, anon_exclusive); } else { /* * Up to this point the pmd is present and huge and userland has @@ -3315,6 +3341,9 @@ static void __split_huge_pmd_locked(struct vm_area_st= ruct *vma, pmd_t *pmd, * complete for this pmd), then we flush the SMP TLB and finally * we write the non-huge version of the pmd entry with * pmd_populate. + * + * This must also happen before PageAnonExclusive() is read + * below, see folio_try_share_anon_rmap_pmd(). */ old_pmd =3D pmdp_invalidate(vma, haddr, pmd); page =3D pmd_page(old_pmd); @@ -3329,34 +3358,9 @@ static void __split_huge_pmd_locked(struct vm_area_s= truct *vma, pmd_t *pmd, =20 VM_WARN_ON_FOLIO(!folio_ref_count(folio), folio); =20 - /* - * Without "freeze", we'll simply split the PMD, propagating the - * PageAnonExclusive() flag for each PTE by setting it for - * each subpage -- no need to (temporarily) clear. - * - * With "freeze" we want to replace mapped pages by - * migration entries right away. This is only possible if we - * managed to clear PageAnonExclusive() -- see - * set_pmd_migration_entry(). - * - * In case we cannot clear PageAnonExclusive(), split the PMD - * only and let try_to_migrate_one() fail later. - * - * See folio_try_share_anon_rmap_pmd(): invalidate PMD first. - */ anon_exclusive =3D PageAnonExclusive(page); - if (freeze && anon_exclusive && - folio_try_share_anon_rmap_pmd(folio, page)) - freeze =3D false; - if (!freeze) { - rmap_t rmap_flags =3D RMAP_NONE; - - folio_ref_add(folio, HPAGE_PMD_NR - 1); - if (anon_exclusive) - rmap_flags |=3D RMAP_EXCLUSIVE; - folio_add_anon_rmap_ptes(folio, page, HPAGE_PMD_NR, - vma, haddr, rmap_flags); - } + freeze =3D split_huge_pmd_anon_rmap(folio, page, vma, haddr, + freeze, anon_exclusive); } =20 /* --=20 2.34.1 From nobody Sat Sep 26 22:53:02 2026 Received: from mail-pf1-f172.google.com (mail-pf1-f172.google.com [209.85.210.172]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id AF6CD2BE03C for ; Fri, 28 Aug 2026 18:37:59 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.172 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1787942281; cv=none; b=TA+KZmL19cWfPmbDxg8s4wowOScDp/3a976w/aW7TAwsn3Qtzr61L3mJ+GFY9gp1qgBbY8H+PW32NggGp+mLBuORCQDxbUreyvcafGWyXMNiCohSxp+JUHL/t+3gyRPJ4LbKWNV7DS3rCD2zzuHJ7ABxmPrt65aJnz1tNU6Nxd0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1787942281; c=relaxed/simple; bh=ZzKclsIsip5W4tkk6HLyJzigiyvSRlS9zTcT1o8eEH8=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=MYNRbo9C+b2IzrwRK9WNoyRZKgyaHfMNih25pAfoy5ZFJflHS7KO2ZLPcMQyuljmMvA/dHwGb4pl6rge5GNAW0DnTe9AcLXOuIebUigmz1qvZcCRQGcG0YkRoYCCW6zwxvmNtWN96t9S/qsitaKGn/PzYPS52nkP26lsTUQ1kTw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=cStLlI0i; arc=none smtp.client-ip=209.85.210.172 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="cStLlI0i" Received: by mail-pf1-f172.google.com with SMTP id d2e1a72fcca58-84eb992a881so1296983b3a.2 for ; Fri, 28 Aug 2026 11:37:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1787942279; x=1788547079; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=/ohz0fKfBZAA1QqDfzvlZw08AYcaw1ZYCfP9nCow3TI=; b=cStLlI0igaZ7Fw0pIJSD1CtljAjE+r2Dft8F+IdAybZ++o/LZtQDPLUmIDsIbWTaLW iHc/qWfaOhOCJYcjtd1MbhVtwOyj49Qom/+YYrgADC/v1HRtC2rSZs8ea6TLB/7cNlVz bW+V1YxSlI1+/q4mM4i5CYNJtHtUqPxdJPxCcYp4Qz7sAjA1/lHGwNrHa47w0ogy1H17 JVNtiXVROwv0IfeCjrAF+h2CwkMaknWFYA3tXD8QoLUq1VPJbWzpO14psfFp3LPrbeiK uf2LkuLt93qMpjsy3wYLnKJ2fqLXqSgpu1C3Zv7LwTDR3zlAm9Rzi4Bf7t3Ibv/RWDLk y3pQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1787942279; x=1788547079; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=/ohz0fKfBZAA1QqDfzvlZw08AYcaw1ZYCfP9nCow3TI=; b=K2xMEEjR1oiFb0PEd+Dlh0sp1MdYGMdY4hPtKzmSBUwYu+TzRZHeegJuVmlxjHZDCj 2Bvdge4fwSjqfzmK/S8cVjbFgqWrDxMp6CNFsfjrIZV9AldVi9jdCXj4l5ePft6/5C/D tJE6qSTOEZFdCGKsOUId7LERK84YHL3Qp8OzOkcp+jg8dkG1AKvA0CK9gYAqIf+k7HkV 0W99FhOXG949bjaATvFp/khrU6HAf8LUBapnzmHQz/bCaIPz0WNKr7izxgQ3zEu87ifH Q1IpkqmspC5ALQ0RBN/EcY+TXRljK+i5At10HqKjaNHTNLThyFf2ZdpLR5gz2APbuVXJ ITDg== X-Forwarded-Encrypted: i=1; AHgh+RqJpRPYv5TQzlydpSXXVQnxp/SbhMWhckTmGiZQcRGusTi8gTiaCFxsRXLJCUQaXyIZdhrhONaon5TTh1U=@vger.kernel.org X-Gm-Message-State: AFuF++kLqSBrta6HRGiYn4fz8zbWDiQhramG4tV7WzqVUDUWL3bt0Iac bpTXEWwWGygDt0dWxBR1wB5q5/czuDhwxwkkd6zAHROEh5KCRYXDnUQU X-Gm-Gg: AR+sD12yDvjTaFzUP0qjfPTXUqGE9W5mq2IonlURTnGKEsgTB2luooTNtXr/UqrV76u hxIIC4w0BpPvxjEE58TjQEmrP2cuON+7nBfUwzcWXd08n4VTa2QgRAordaroEUHHiP5nW08wMww V3wOi1U+o4Dp23kDVBAD2ZNE1wlZNkHtGwZsBF7D+wvWFIots/NLC7rPnCcmdHysSSSv9+SkJiV C7v/k31X0ijGz/kJH3e4yx3zydVBQxerT5SNKzPTCV/Ahz8SKbDnEIZ6kDo3Fami2wxAVDWvpkW bi2vjwugmCzf4RGXKzaJ9VDMsh3+RdUsxTIKLQQvEbJOS0RkyODikdVdKQQv7xruqeTLen3E6fd OwxMbvVeGsyK7ToCa1xxGXwdddz6nP1iiOpf9LNz1m1mSnjL+cB/ql/uPQn1Z0+mUdycnBQewbP jBnUCZ16UPDx6rL8kmj0yG26sBnYZ+J1xXOfhbCm0G55/KEOP9hXsKZsmF4V1bw40+F/ndRBU9s z1fUawXPHtPiMrPGA== X-Received: by 2002:a05:6a00:3694:b0:857:7317:cff7 with SMTP id d2e1a72fcca58-8577317d16dmr2447151b3a.24.1787942278878; Fri, 28 Aug 2026 11:37:58 -0700 (PDT) Received: from aig-kmd-01.. ([120.133.49.20]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-856a2ea6fa3sm844230b3a.37.2026.08.28.11.37.50 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 28 Aug 2026 11:37:58 -0700 (PDT) From: Yin Tirui To: Andrew Morton , linux-mm@kvack.org Cc: David Hildenbrand , Lorenzo Stoakes , Dev Jain , Zi Yan , Baolin Wang , Barry Song , Lance Yang , Ryan Roberts , Nico Pache , Usama Arif , "Liam R . Howlett" , wangkefeng.wang@huawei.com, chenjun102@huawei.com, linux-kernel@vger.kernel.org, Yin Tirui Subject: [PATCH RFC 7/9] mm/huge_memory: add struct split_pmd_state Date: Sat, 29 Aug 2026 02:33:17 +0800 Message-Id: <0ef68612bfdbe340e8c7f142c12b7f9998181a54.1787941780.git.yintirui@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Put the state read out of the entry being split into one descriptor, so the read and write paths can be separated without passing a long argument list between them. No functional change intended. Signed-off-by: Yin Tirui --- mm/huge_memory.c | 160 ++++++++++++++++++++++++++--------------------- 1 file changed, 88 insertions(+), 72 deletions(-) diff --git a/mm/huge_memory.c b/mm/huge_memory.c index e0083a9e89b8..72e2cd1d7672 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -3194,6 +3194,20 @@ static void unmap_huge_pmd_entry(struct vm_area_stru= ct *vma, folio_put(folio); } =20 +struct split_pmd_state { + struct folio *folio; + struct page *page; + bool is_present; + bool is_device_private; + bool freeze; + bool write; + bool young; + bool dirty; + bool soft_dirty; + bool uffd; + bool anon_exclusive; +}; + /* * Convert the folio's PMD-level anonymous rmap into PTE-level ones. * @@ -3213,37 +3227,38 @@ static void unmap_huge_pmd_entry(struct vm_area_str= uct *vma, * * Returns: whether the mapping may still be frozen. */ -static bool split_huge_pmd_anon_rmap(struct folio *folio, struct page *pag= e, - struct vm_area_struct *vma, unsigned long haddr, bool freeze, - bool anon_exclusive) +static bool split_huge_pmd_anon_rmap(const struct split_pmd_state *state, + struct vm_area_struct *vma, unsigned long haddr) { rmap_t rmap_flags =3D RMAP_NONE; =20 - if (freeze && - (!anon_exclusive || !folio_try_share_anon_rmap_pmd(folio, page))) + if (state->freeze && + (!state->anon_exclusive || + !folio_try_share_anon_rmap_pmd(state->folio, state->page))) return true; =20 - folio_ref_add(folio, HPAGE_PMD_NR - 1); - if (anon_exclusive) + folio_ref_add(state->folio, HPAGE_PMD_NR - 1); + if (state->anon_exclusive) rmap_flags |=3D RMAP_EXCLUSIVE; - folio_add_anon_rmap_ptes(folio, page, HPAGE_PMD_NR, vma, haddr, - rmap_flags); + folio_add_anon_rmap_ptes(state->folio, state->page, HPAGE_PMD_NR, vma, + haddr, rmap_flags); return false; } =20 static void __split_huge_pmd_locked(struct vm_area_struct *vma, pmd_t *pmd, unsigned long haddr, bool freeze) { - struct mm_struct *mm =3D vma->vm_mm; - pmd_t old_pmd =3D *pmd; + const pmd_t old_pmd =3D *pmd; const bool is_present =3D pmd_present(old_pmd); + struct mm_struct *mm =3D vma->vm_mm; + struct split_pmd_state state =3D { + .is_present =3D is_present, + .freeze =3D freeze, + }; struct folio *folio; - struct page *page; + unsigned long addr; pgtable_t pgtable; pmd_t _pmd; - bool soft_dirty, uffd_wp =3D false, young =3D false, write =3D false; - bool anon_exclusive =3D false, dirty =3D false; - unsigned long addr; pte_t *pte; int i; =20 @@ -3286,38 +3301,39 @@ static void __split_huge_pmd_locked(struct vm_area_= struct *vma, pmd_t *pmd, return; } =20 + state.folio =3D folio; + if (pmd_is_migration_entry(old_pmd)) { - softleaf_t entry; + const softleaf_t entry =3D softleaf_from_pmd(old_pmd); =20 - entry =3D softleaf_from_pmd(old_pmd); - page =3D softleaf_to_page(entry); + state.page =3D softleaf_to_page(entry); =20 - soft_dirty =3D pmd_swp_soft_dirty(old_pmd); - uffd_wp =3D pmd_swp_uffd(old_pmd); + state.soft_dirty =3D pmd_swp_soft_dirty(old_pmd); + state.uffd =3D pmd_swp_uffd(old_pmd); =20 - write =3D softleaf_is_migration_write(entry); - anon_exclusive =3D softleaf_is_migration_read_exclusive(entry); - young =3D softleaf_is_migration_young(entry); - dirty =3D softleaf_is_migration_dirty(entry); + state.write =3D softleaf_is_migration_write(entry); + state.anon_exclusive =3D + softleaf_is_migration_read_exclusive(entry); + state.young =3D softleaf_is_migration_young(entry); + state.dirty =3D softleaf_is_migration_dirty(entry); } else if (pmd_is_device_private_entry(old_pmd)) { - softleaf_t entry; + const softleaf_t entry =3D softleaf_from_pmd(old_pmd); =20 - entry =3D softleaf_from_pmd(old_pmd); - page =3D softleaf_to_page(entry); + state.is_device_private =3D true; + state.page =3D softleaf_to_page(entry); =20 - soft_dirty =3D pmd_swp_soft_dirty(old_pmd); - uffd_wp =3D pmd_swp_uffd(old_pmd); + state.soft_dirty =3D pmd_swp_soft_dirty(old_pmd); + state.uffd =3D pmd_swp_uffd(old_pmd); =20 - write =3D softleaf_is_device_private_write(entry); - anon_exclusive =3D PageAnonExclusive(page); + state.write =3D softleaf_is_device_private_write(entry); + state.anon_exclusive =3D PageAnonExclusive(state.page); =20 /* * Device private folios are treated the same as regular folios * w.r.t. anon exclusive handling, see * split_huge_pmd_anon_rmap(). */ - freeze =3D split_huge_pmd_anon_rmap(folio, page, vma, haddr, - freeze, anon_exclusive); + state.freeze =3D split_huge_pmd_anon_rmap(&state, vma, haddr); } else { /* * Up to this point the pmd is present and huge and userland has @@ -3345,22 +3361,22 @@ static void __split_huge_pmd_locked(struct vm_area_= struct *vma, pmd_t *pmd, * This must also happen before PageAnonExclusive() is read * below, see folio_try_share_anon_rmap_pmd(). */ - old_pmd =3D pmdp_invalidate(vma, haddr, pmd); - page =3D pmd_page(old_pmd); - if (pmd_dirty(old_pmd)) { - dirty =3D true; + const pmd_t pmdval =3D pmdp_invalidate(vma, haddr, pmd); + + state.page =3D pmd_page(pmdval); + state.write =3D pmd_write(pmdval); + state.young =3D pmd_young(pmdval); + state.dirty =3D pmd_dirty(pmdval); + state.soft_dirty =3D pmd_soft_dirty(pmdval); + state.uffd =3D pmd_uffd(pmdval); + state.anon_exclusive =3D PageAnonExclusive(state.page); + + if (state.dirty) folio_set_dirty(folio); - } - write =3D pmd_write(old_pmd); - young =3D pmd_young(old_pmd); - soft_dirty =3D pmd_soft_dirty(old_pmd); - uffd_wp =3D pmd_uffd(old_pmd); =20 VM_WARN_ON_FOLIO(!folio_ref_count(folio), folio); =20 - anon_exclusive =3D PageAnonExclusive(page); - freeze =3D split_huge_pmd_anon_rmap(folio, page, vma, haddr, - freeze, anon_exclusive); + state.freeze =3D split_huge_pmd_anon_rmap(&state, vma, haddr); } =20 /* @@ -3377,33 +3393,33 @@ static void __split_huge_pmd_locked(struct vm_area_= struct *vma, pmd_t *pmd, * Note that NUMA hinting access restrictions are not transferred to * avoid any possibility of altering permissions across VMAs. */ - if (freeze || pmd_is_migration_entry(old_pmd)) { + if (state.freeze || (!state.is_present && !state.is_device_private)) { pte_t entry; swp_entry_t swp_entry; =20 for (i =3D 0, addr =3D haddr; i < HPAGE_PMD_NR; i++, addr +=3D PAGE_SIZE= ) { - if (write) + if (state.write) swp_entry =3D make_writable_migration_entry( - page_to_pfn(page + i)); - else if (anon_exclusive) + page_to_pfn(state.page + i)); + else if (state.anon_exclusive) swp_entry =3D make_readable_exclusive_migration_entry( - page_to_pfn(page + i)); + page_to_pfn(state.page + i)); else swp_entry =3D make_readable_migration_entry( - page_to_pfn(page + i)); - if (young) + page_to_pfn(state.page + i)); + if (state.young) swp_entry =3D make_migration_entry_young(swp_entry); - if (dirty) + if (state.dirty) swp_entry =3D make_migration_entry_dirty(swp_entry); entry =3D swp_entry_to_pte(swp_entry); - if (soft_dirty) + if (state.soft_dirty) entry =3D pte_swp_mksoft_dirty(entry); - if (uffd_wp) + if (state.uffd) entry =3D pte_swp_mkuffd(entry); VM_WARN_ON(!pte_none(ptep_get(pte + i))); set_pte_at(mm, addr, pte + i, entry); } - } else if (pmd_is_device_private_entry(old_pmd)) { + } else if (state.is_device_private) { pte_t entry; swp_entry_t swp_entry; =20 @@ -3413,19 +3429,19 @@ static void __split_huge_pmd_locked(struct vm_area_= struct *vma, pmd_t *pmd, * pages corresponding to the pte entries when freeze * is false. */ - if (write) + if (state.write) swp_entry =3D make_writable_device_private_entry( - page_to_pfn(page + i)); + page_to_pfn(state.page + i)); else swp_entry =3D make_readable_device_private_entry( - page_to_pfn(page + i)); + page_to_pfn(state.page + i)); /* * Young and dirty bits are not progated via swp_entry */ entry =3D swp_entry_to_pte(swp_entry); - if (soft_dirty) + if (state.soft_dirty) entry =3D pte_swp_mksoft_dirty(entry); - if (uffd_wp) + if (state.uffd) entry =3D pte_swp_mkuffd(entry); VM_WARN_ON(!pte_none(ptep_get(pte + i))); set_pte_at(mm, addr, pte + i, entry); @@ -3433,21 +3449,21 @@ static void __split_huge_pmd_locked(struct vm_area_= struct *vma, pmd_t *pmd, } else { pte_t entry; =20 - entry =3D mk_pte(page, READ_ONCE(vma->vm_page_prot)); - if (write) + entry =3D mk_pte(state.page, READ_ONCE(vma->vm_page_prot)); + if (state.write) entry =3D pte_mkwrite(entry, vma); - if (!young) + if (!state.young) entry =3D pte_mkold(entry); /* NOTE: this may set soft-dirty too on some archs */ - if (dirty) + if (state.dirty) entry =3D pte_mkdirty(entry); - if (soft_dirty) + if (state.soft_dirty) entry =3D pte_mksoft_dirty(entry); - if (uffd_wp) + if (state.uffd) entry =3D pte_mkuffd(entry); =20 /* Restore PAGE_NONE so an RWP marker keeps trapping */ - if (userfaultfd_rwp(vma) && uffd_wp) + if (userfaultfd_rwp(vma) && state.uffd) entry =3D pte_modify(entry, PAGE_NONE); =20 for (i =3D 0; i < HPAGE_PMD_NR; i++) @@ -3457,10 +3473,10 @@ static void __split_huge_pmd_locked(struct vm_area_= struct *vma, pmd_t *pmd, } pte_unmap(pte); =20 - if (!pmd_is_migration_entry(old_pmd)) - folio_remove_rmap_pmd(folio, page, vma); - if (freeze) - put_page(page); + if (state.is_present || state.is_device_private) + folio_remove_rmap_pmd(state.folio, state.page, vma); + if (state.freeze) + put_page(state.page); =20 smp_wmb(); /* make pte visible before pmd */ pmd_populate(mm, pmd, pgtable); --=20 2.34.1 From nobody Sat Sep 26 22:53:02 2026 Received: from mail-pf1-f180.google.com (mail-pf1-f180.google.com [209.85.210.180]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6C0833B9DA2 for ; Fri, 28 Aug 2026 18:38:09 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.180 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1787942291; cv=none; b=aaubqVwPvi0Dc1RBo3YU3cOQkGz8gSLjI9s/3tjp9WqHPlpEBxpYmIZvgF4wKZRG43SmKM2Oq9RtxBDTccSUOLSW4RylSzaK4ZCAQNbVSTxquDtoyF0GzhCibyaI5PPkEoojkl7ThHxV0hzxqV25LaNa3fm4ZzkGbtPuy/cEmtY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1787942291; c=relaxed/simple; bh=H2giclNVNkGHGU3dGxNEn2Np1ew1ExNFD872nl41krY=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=DWL3qd6PAXYi513mvMYCkwo1hnJJOeiuCykZ4yPMiuSnxBl4/i/xBHOUX7NIK4BuxrsXIW6RS8JLaA3Ezeii9zfbjWN/5sEGAXcr73LZLImxuhJ6vZHwt0ADQQhf6U6fbMpKD6331B0cERFKmDsJNTNxI3nZ7Zel+C1Q0qLt0Wo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=NwnWMhHU; arc=none smtp.client-ip=209.85.210.180 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="NwnWMhHU" Received: by mail-pf1-f180.google.com with SMTP id d2e1a72fcca58-84e27035206so1241939b3a.3 for ; Fri, 28 Aug 2026 11:38:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1787942289; x=1788547089; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=CvYR8hSj3HruxGRZAeIbxinhDfyfRxH1dMC6OkZYttE=; b=NwnWMhHUp98EKb79G6Fs1ZX/AjeNAxFo76NO//p2ubLUlFBI2Fuu2y+gbODTUwAcqI jPcPKns+FwgzwQ1PSRLdwrEqP3KuBHmg3527gnlpVsx4lQRMdA3GbyhLenjoG/yJ/Hwb 4CyQU4SbGAgmiOVCaTT3Uysfo70/vm68m2avOQsg9Ko5amVEPOXq/WlEPUM19UH9Pd6y 4TV5T1nuvLoNqYZUS9AolsDic81M+ZvHi3hcFT9SKv/Yfeh0+qIQDbn8siZMxv3ZGtBQ kLsEaiubH9JfH4m3Am6HK1KKMVrbupT9LgY2QZSJkbUurfhMg0Xcm8COlJNO1Vm9bgCj eenw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1787942289; x=1788547089; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=CvYR8hSj3HruxGRZAeIbxinhDfyfRxH1dMC6OkZYttE=; b=Po4YPo4gSqy2dO4hS8a2Dh+xOgegssALfBdhEnT4FvwQcB0OSPK5nHyuX9btWj2cRJ RfF3j2ocu7KGJiD+2VI1+OTMpQu6RanlDIDXmv4aRIw/GV6Q1Ec9OIO8cQCLStFdRQEb DCrR81r1K0z2ccLQ8NVrnnqGqhYerkXSDGpDi9cyPIP7FtkW3bEnMKyeVhRYWhaRfCfc Gq2cDhtcaqHqmX9NOeD+j5O3DND93B8TecWSxNc+3yf/RLyKfBsZ7LvG8atynaibTwCG 4Nm6kUiFs+Ld4G+cJ2a9BiA2zA9Ky8KnmfyIv6NG8MxxEyVixwX4xIoU0TS8+SJ6tUFI 64UQ== X-Forwarded-Encrypted: i=1; AHgh+Rq1OJY4D2mxDn8zi9XRkAPEzpBcuCi82vcCsJQUGps+D7TX6RLDhwNQFiqiGGPDrn7ZPTk1mnvwrhxbnxI=@vger.kernel.org X-Gm-Message-State: AFuF++ncX7gkTtYqTqsOrJv2c7inYmP0WRgO7ND1V0YfhqpJMOpyeKGb RaEpdjHSlceQIT9gAvkLWSgDPGlHwCCtDpO2GtTimxpej/o4McvEV0vK X-Gm-Gg: AR+sD13blOIgLBDfORjEA/LB3Y5ihnKgR53syoARok9AAIZA4gTxwILvILwJK9ocasy s/BBTcJkHRVw8xpmKUJQjTsax3W9uG2jXoFzlQb1DdKg8vf3lAT/wxwuw+FkeK7mCXO1GoqtvmN lBJfbVCCu4SUBv4WTN9Dm+yYDKUEC6bg2tQTLzY7hP0SX4yDSIlzESH/ojwLsxaUwtMO4iKHLSU dnrSRq3QR0T5kn1BqApgjHDQlKBISlvhxPx0gMetbMA6fs8Rkkw/2RLAnonDYiEbs7Lhg5LP0rs 50u5hSsCpsdznSVqXzKcCPAt2Jo3anEcv4iVHONyeIJFB8PC7AYWnyiATAHkvJ2bYgOVPufzqQe LZjKkFQRLXytwwVOorX2wzU0f/4qSh4usY2/EOqIft2LbtfqOWNERsJcHOAAyEchEvvZWW7J0/L Kv7b2sve/HhZMCDxQWc+M91fhHha24hah0SH1o5FRU14VWCYUTIAIOtjFMj/QK3AHAyaKcSGO9R PsP4rrnie+teovYBnvjrYc09yFV X-Received: by 2002:a05:6a00:414c:b0:848:4754:28e4 with SMTP id d2e1a72fcca58-8562abbefa2mr18993315b3a.15.1787942288449; Fri, 28 Aug 2026 11:38:08 -0700 (PDT) Received: from aig-kmd-01.. ([120.133.49.20]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-856a2ea6fa3sm844230b3a.37.2026.08.28.11.37.59 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 28 Aug 2026 11:38:07 -0700 (PDT) From: Yin Tirui To: Andrew Morton , linux-mm@kvack.org Cc: David Hildenbrand , Lorenzo Stoakes , Dev Jain , Zi Yan , Baolin Wang , Barry Song , Lance Yang , Ryan Roberts , Nico Pache , Usama Arif , "Liam R . Howlett" , wangkefeng.wang@huawei.com, chenjun102@huawei.com, linux-kernel@vger.kernel.org, Yin Tirui Subject: [PATCH RFC 8/9] mm/huge_memory: split present and non-present huge PMDs separately Date: Sat, 29 Aug 2026 02:33:18 +0800 Message-Id: <46d6e131bcae6f4f2c8716c480aab87750a52c76.1787941780.git.yintirui@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Move the shared PTE rebuild into split_huge_pmd_to_ptes(), then add split_present_huge_pmd() and split_non_present_huge_pmd() on top of it to separate the present and non-present cases in __split_huge_pmd_locked(). No functional change intended. Suggested-by: David Hildenbrand Link: https://lore.kernel.org/linux-mm/67a655e3-fa23-4d2a-9685-14e6221d5d26= @kernel.org/ Signed-off-by: Yin Tirui --- mm/huge_memory.c | 334 +++++++++++++++++++++++++---------------------- 1 file changed, 180 insertions(+), 154 deletions(-) diff --git a/mm/huge_memory.c b/mm/huge_memory.c index 72e2cd1d7672..fdb751a1e525 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -3245,143 +3245,31 @@ static bool split_huge_pmd_anon_rmap(const struct = split_pmd_state *state, return false; } =20 -static void __split_huge_pmd_locked(struct vm_area_struct *vma, pmd_t *pmd, - unsigned long haddr, bool freeze) +/* + * Replace an anonymous huge PMD entry with a page table mapping the same + * folio at PTE granularity. + */ +static void split_huge_pmd_to_ptes(struct vm_area_struct *vma, + unsigned long haddr, pmd_t *pmd, struct split_pmd_state *state) { - const pmd_t old_pmd =3D *pmd; - const bool is_present =3D pmd_present(old_pmd); + /* Present mappings and device private entries hold a PMD-level rmap. */ + const bool rmapped =3D state->is_present || state->is_device_private; struct mm_struct *mm =3D vma->vm_mm; - struct split_pmd_state state =3D { - .is_present =3D is_present, - .freeze =3D freeze, - }; - struct folio *folio; + struct page *page =3D state->page; unsigned long addr; pgtable_t pgtable; pmd_t _pmd; pte_t *pte; int i; =20 - VM_BUG_ON(haddr & ~HPAGE_PMD_MASK); - VM_BUG_ON_VMA(vma->vm_start > haddr, vma); - VM_BUG_ON_VMA(vma->vm_end < haddr + HPAGE_PMD_SIZE, vma); - - VM_WARN_ON_ONCE(!pmd_is_valid_softleaf(old_pmd) && - !pmd_trans_huge(old_pmd)); - - count_vm_event(THP_SPLIT_PMD); + if (rmapped) + state->freeze =3D split_huge_pmd_anon_rmap(state, vma, haddr); =20 /* - * FIXME: Do we want to invalidate secondary mmu by calling - * mmu_notifier_arch_invalidate_secondary_tlbs() see comments below - * inside __split_huge_pmd() ? - * - * We are going from a zero huge page write protected to zero small - * page also write protected so it does not seems useful to invalidate - * secondary mmu at this time. - */ - if (huge_zero_pmd_can_split(vma, old_pmd)) { - __split_huge_zero_page_pmd(vma, haddr, pmd); - return; - } - - folio =3D normal_or_softleaf_folio_pmd(vma, haddr, old_pmd, is_present); - - /* - * A non-present entry which is neither a migration nor a device - * private entry is corrupt, and pmd_to_softleaf_folio() has already - * warned about it. Leave it alone rather than act on a PFN which - * means nothing. - */ - if (unlikely(!is_present && !folio)) - return; - - if (!folio || !folio_test_anon(folio)) { - unmap_huge_pmd_entry(vma, haddr, pmd, folio, is_present); - return; - } - - state.folio =3D folio; - - if (pmd_is_migration_entry(old_pmd)) { - const softleaf_t entry =3D softleaf_from_pmd(old_pmd); - - state.page =3D softleaf_to_page(entry); - - state.soft_dirty =3D pmd_swp_soft_dirty(old_pmd); - state.uffd =3D pmd_swp_uffd(old_pmd); - - state.write =3D softleaf_is_migration_write(entry); - state.anon_exclusive =3D - softleaf_is_migration_read_exclusive(entry); - state.young =3D softleaf_is_migration_young(entry); - state.dirty =3D softleaf_is_migration_dirty(entry); - } else if (pmd_is_device_private_entry(old_pmd)) { - const softleaf_t entry =3D softleaf_from_pmd(old_pmd); - - state.is_device_private =3D true; - state.page =3D softleaf_to_page(entry); - - state.soft_dirty =3D pmd_swp_soft_dirty(old_pmd); - state.uffd =3D pmd_swp_uffd(old_pmd); - - state.write =3D softleaf_is_device_private_write(entry); - state.anon_exclusive =3D PageAnonExclusive(state.page); - - /* - * Device private folios are treated the same as regular folios - * w.r.t. anon exclusive handling, see - * split_huge_pmd_anon_rmap(). - */ - state.freeze =3D split_huge_pmd_anon_rmap(&state, vma, haddr); - } else { - /* - * Up to this point the pmd is present and huge and userland has - * the whole access to the hugepage during the split (which - * happens in place). If we overwrite the pmd with the not-huge - * version pointing to the pte here (which of course we could if - * all CPUs were bug free), userland could trigger a small page - * size TLB miss on the small sized TLB while the hugepage TLB - * entry is still established in the huge TLB. Some CPU doesn't - * like that. See - * http://support.amd.com/TechDocs/41322_10h_Rev_Gd.pdf, Erratum - * 383 on page 105. Intel should be safe but is also warns that - * it's only safe if the permission and cache attributes of the - * two entries loaded in the two TLB is identical (which should - * be the case here). But it is generally safer to never allow - * small and huge TLB entries for the same virtual address to be - * loaded simultaneously. So instead of doing "pmd_populate(); - * flush_pmd_tlb_range();" we first mark the current pmd - * notpresent (atomically because here the pmd_trans_huge must - * remain set at all times on the pmd until the split is - * complete for this pmd), then we flush the SMP TLB and finally - * we write the non-huge version of the pmd entry with - * pmd_populate. - * - * This must also happen before PageAnonExclusive() is read - * below, see folio_try_share_anon_rmap_pmd(). - */ - const pmd_t pmdval =3D pmdp_invalidate(vma, haddr, pmd); - - state.page =3D pmd_page(pmdval); - state.write =3D pmd_write(pmdval); - state.young =3D pmd_young(pmdval); - state.dirty =3D pmd_dirty(pmdval); - state.soft_dirty =3D pmd_soft_dirty(pmdval); - state.uffd =3D pmd_uffd(pmdval); - state.anon_exclusive =3D PageAnonExclusive(state.page); - - if (state.dirty) - folio_set_dirty(folio); - - VM_WARN_ON_FOLIO(!folio_ref_count(folio), folio); - - state.freeze =3D split_huge_pmd_anon_rmap(&state, vma, haddr); - } - - /* - * Withdraw the table only after we mark the pmd entry invalid. - * This's critical for some architectures (Power). + * The caller has already invalidated a present entry, and a softleaf + * entry is not present to begin with. Either way the entry is out of + * service before we withdraw the deposited page table, which is + * critical for some architectures (Power). */ pgtable =3D pgtable_trans_huge_withdraw(mm, pmd); pmd_populate(mm, &_pmd, pgtable); @@ -3393,33 +3281,34 @@ static void __split_huge_pmd_locked(struct vm_area_= struct *vma, pmd_t *pmd, * Note that NUMA hinting access restrictions are not transferred to * avoid any possibility of altering permissions across VMAs. */ - if (state.freeze || (!state.is_present && !state.is_device_private)) { + if (state->freeze || + (!state->is_present && !state->is_device_private)) { pte_t entry; swp_entry_t swp_entry; =20 for (i =3D 0, addr =3D haddr; i < HPAGE_PMD_NR; i++, addr +=3D PAGE_SIZE= ) { - if (state.write) + if (state->write) swp_entry =3D make_writable_migration_entry( - page_to_pfn(state.page + i)); - else if (state.anon_exclusive) + page_to_pfn(page + i)); + else if (state->anon_exclusive) swp_entry =3D make_readable_exclusive_migration_entry( - page_to_pfn(state.page + i)); + page_to_pfn(page + i)); else swp_entry =3D make_readable_migration_entry( - page_to_pfn(state.page + i)); - if (state.young) + page_to_pfn(page + i)); + if (state->young) swp_entry =3D make_migration_entry_young(swp_entry); - if (state.dirty) + if (state->dirty) swp_entry =3D make_migration_entry_dirty(swp_entry); entry =3D swp_entry_to_pte(swp_entry); - if (state.soft_dirty) + if (state->soft_dirty) entry =3D pte_swp_mksoft_dirty(entry); - if (state.uffd) + if (state->uffd) entry =3D pte_swp_mkuffd(entry); VM_WARN_ON(!pte_none(ptep_get(pte + i))); set_pte_at(mm, addr, pte + i, entry); } - } else if (state.is_device_private) { + } else if (state->is_device_private) { pte_t entry; swp_entry_t swp_entry; =20 @@ -3429,19 +3318,19 @@ static void __split_huge_pmd_locked(struct vm_area_= struct *vma, pmd_t *pmd, * pages corresponding to the pte entries when freeze * is false. */ - if (state.write) + if (state->write) swp_entry =3D make_writable_device_private_entry( - page_to_pfn(state.page + i)); + page_to_pfn(page + i)); else swp_entry =3D make_readable_device_private_entry( - page_to_pfn(state.page + i)); + page_to_pfn(page + i)); /* * Young and dirty bits are not progated via swp_entry */ entry =3D swp_entry_to_pte(swp_entry); - if (state.soft_dirty) + if (state->soft_dirty) entry =3D pte_swp_mksoft_dirty(entry); - if (state.uffd) + if (state->uffd) entry =3D pte_swp_mkuffd(entry); VM_WARN_ON(!pte_none(ptep_get(pte + i))); set_pte_at(mm, addr, pte + i, entry); @@ -3449,21 +3338,21 @@ static void __split_huge_pmd_locked(struct vm_area_= struct *vma, pmd_t *pmd, } else { pte_t entry; =20 - entry =3D mk_pte(state.page, READ_ONCE(vma->vm_page_prot)); - if (state.write) + entry =3D mk_pte(page, READ_ONCE(vma->vm_page_prot)); + if (state->write) entry =3D pte_mkwrite(entry, vma); - if (!state.young) + if (!state->young) entry =3D pte_mkold(entry); /* NOTE: this may set soft-dirty too on some archs */ - if (state.dirty) + if (state->dirty) entry =3D pte_mkdirty(entry); - if (state.soft_dirty) + if (state->soft_dirty) entry =3D pte_mksoft_dirty(entry); - if (state.uffd) + if (state->uffd) entry =3D pte_mkuffd(entry); =20 /* Restore PAGE_NONE so an RWP marker keeps trapping */ - if (userfaultfd_rwp(vma) && state.uffd) + if (userfaultfd_rwp(vma) && state->uffd) entry =3D pte_modify(entry, PAGE_NONE); =20 for (i =3D 0; i < HPAGE_PMD_NR; i++) @@ -3473,15 +3362,152 @@ static void __split_huge_pmd_locked(struct vm_area= _struct *vma, pmd_t *pmd, } pte_unmap(pte); =20 - if (state.is_present || state.is_device_private) - folio_remove_rmap_pmd(state.folio, state.page, vma); - if (state.freeze) - put_page(state.page); + if (rmapped) + folio_remove_rmap_pmd(state->folio, page, vma); + if (state->freeze) + put_page(page); =20 smp_wmb(); /* make pte visible before pmd */ pmd_populate(mm, pmd, pgtable); } =20 +static void split_present_huge_pmd(struct vm_area_struct *vma, + unsigned long haddr, pmd_t *pmd, struct folio *folio, + bool freeze) +{ + struct split_pmd_state state =3D { + .folio =3D folio, + .is_present =3D true, + .freeze =3D freeze, + }; + + /* + * Up to this point the pmd is present and huge and userland has the + * whole access to the hugepage during the split (which happens in + * place). If we overwrite the pmd with the not-huge version pointing + * to the pte here (which of course we could if all CPUs were bug + * free), userland could trigger a small page size TLB miss on the + * small sized TLB while the hugepage TLB entry is still established in + * the huge TLB. Some CPU doesn't like that. See + * http://support.amd.com/TechDocs/41322_10h_Rev_Gd.pdf, Erratum 383 on + * page 105. Intel should be safe but is also warns that it's only safe + * if the permission and cache attributes of the two entries loaded in + * the two TLB is identical (which should be the case here). But it is + * generally safer to never allow small and huge TLB entries for the + * same virtual address to be loaded simultaneously. So instead of + * doing "pmd_populate(); flush_pmd_tlb_range();" we first mark the + * current pmd notpresent (atomically because here the pmd_trans_huge + * must remain set at all times on the pmd until the split is complete + * for this pmd), then we flush the SMP TLB and finally we write the + * non-huge version of the pmd entry with pmd_populate. + * + * This must also happen before PageAnonExclusive() is read below, see + * folio_try_share_anon_rmap_pmd(). + */ + const pmd_t pmdval =3D pmdp_invalidate(vma, haddr, pmd); + + state.page =3D pmd_page(pmdval); + state.write =3D pmd_write(pmdval); + state.young =3D pmd_young(pmdval); + state.dirty =3D pmd_dirty(pmdval); + state.soft_dirty =3D pmd_soft_dirty(pmdval); + state.uffd =3D pmd_uffd(pmdval); + state.anon_exclusive =3D PageAnonExclusive(state.page); + + if (state.dirty) + folio_set_dirty(folio); + + VM_WARN_ON_FOLIO(!folio_ref_count(folio), folio); + + split_huge_pmd_to_ptes(vma, haddr, pmd, &state); +} + +static void split_non_present_huge_pmd(struct vm_area_struct *vma, + unsigned long haddr, pmd_t *pmd, pmd_t old_pmd, + struct folio *folio, bool freeze) +{ + const softleaf_t entry =3D softleaf_from_pmd(old_pmd); + struct split_pmd_state state =3D { + .folio =3D folio, + .page =3D softleaf_to_page(entry), + .is_device_private =3D softleaf_is_device_private(entry), + .freeze =3D freeze, + .soft_dirty =3D pmd_swp_soft_dirty(old_pmd), + .uffd =3D pmd_swp_uffd(old_pmd), + }; + + if (state.is_device_private) { + /* + * Device private folios are treated the same as regular folios + * w.r.t. anon exclusive handling, see + * split_huge_pmd_anon_rmap(). + */ + state.write =3D softleaf_is_device_private_write(entry); + state.anon_exclusive =3D PageAnonExclusive(state.page); + } else { + state.write =3D softleaf_is_migration_write(entry); + state.young =3D softleaf_is_migration_young(entry); + state.dirty =3D softleaf_is_migration_dirty(entry); + state.anon_exclusive =3D + softleaf_is_migration_read_exclusive(entry); + } + + split_huge_pmd_to_ptes(vma, haddr, pmd, &state); +} + +static void __split_huge_pmd_locked(struct vm_area_struct *vma, pmd_t *pmd, + unsigned long haddr, bool freeze) +{ + const pmd_t old_pmd =3D *pmd; + const bool is_present =3D pmd_present(old_pmd); + struct folio *folio; + + VM_BUG_ON(haddr & ~HPAGE_PMD_MASK); + VM_BUG_ON_VMA(vma->vm_start > haddr, vma); + VM_BUG_ON_VMA(vma->vm_end < haddr + HPAGE_PMD_SIZE, vma); + + VM_WARN_ON_ONCE(!pmd_is_valid_softleaf(old_pmd) && + !pmd_trans_huge(old_pmd)); + + count_vm_event(THP_SPLIT_PMD); + + /* + * FIXME: Do we want to invalidate secondary mmu by calling + * mmu_notifier_arch_invalidate_secondary_tlbs() see comments below + * inside __split_huge_pmd() ? + * + * We are going from a zero huge page write protected to zero small + * page also write protected so it does not seems useful to invalidate + * secondary mmu at this time. + */ + if (huge_zero_pmd_can_split(vma, old_pmd)) { + __split_huge_zero_page_pmd(vma, haddr, pmd); + return; + } + + folio =3D normal_or_softleaf_folio_pmd(vma, haddr, old_pmd, is_present); + + /* + * A non-present entry which is neither a migration nor a device + * private entry is corrupt, and pmd_to_softleaf_folio() has already + * warned about it. Leave it alone rather than act on a PFN which + * means nothing. + */ + if (unlikely(!is_present && !folio)) + return; + + if (!folio || !folio_test_anon(folio)) { + unmap_huge_pmd_entry(vma, haddr, pmd, folio, is_present); + return; + } + + if (is_present) + split_present_huge_pmd(vma, haddr, pmd, folio, freeze); + else + split_non_present_huge_pmd(vma, haddr, pmd, old_pmd, folio, + freeze); +} + void split_huge_pmd_locked(struct vm_area_struct *vma, unsigned long addre= ss, pmd_t *pmd, bool freeze) { --=20 2.34.1 From nobody Sat Sep 26 22:53:02 2026 Received: from mail-pf1-f182.google.com (mail-pf1-f182.google.com [209.85.210.182]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 67E25372B5A for ; Fri, 28 Aug 2026 18:38:18 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.182 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1787942299; cv=none; b=dLtjSiWUch3Ro9PiXPvhAiS0fb+dGEMMDBigcqnYehKv2c74eHfH4uCd4WBpYQdSAFZomKNODuvbLWXgkEqnIrNbLEPmB8dxentnalqVdc0trCsARX3Iz0GYi4g+po7Ym/b5GzPuq/YQhi7qDFecvPycPptICGpSk/J+eMwOWfs= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1787942299; c=relaxed/simple; bh=Rf485JwcWlHF+00e9hXUSMYaTdPU2gja5rXNVnGjo10=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=c80LTaw5ifax42GMc37QpJwqVMT1kDxkDQvcLcYFetDWghBSL/qltJyqJ6/C8c9KplZtvyuOQ5/cNacB2k6wt8MLgNQ6ehDqFroV4zWXa/IauuyKZTtLx25xnPtZRzyjggnGc0YIbeJL+8DUMpWM4ZelYW1JMv77Fw+PG1kr8HQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=hNV0OYvz; arc=none smtp.client-ip=209.85.210.182 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="hNV0OYvz" Received: by mail-pf1-f182.google.com with SMTP id d2e1a72fcca58-853f8c34ba4so1832334b3a.0 for ; Fri, 28 Aug 2026 11:38:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1787942298; x=1788547098; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=v3nAcDLCkfTKlmodmQ0boAx10KLIo6Vy0ud2mfcvPW0=; b=hNV0OYvzATsAEJkY7fPSbN1Hi2uqcSrGqR5Xc3YItCPr5DwEeKKng8VFro5qM1Ixet EaEHW5yC7XNa4MqgonZC4qpfiDUmgBDNa3cMHQuL9HidbxOMpP7Xc0eeRflPvA9HQFew 47zJDhyJV4KhpvCX4bzO9PO/2f47MATpgyzCZsiemoVUJ2rW1Q4HZACzmoqNbFaL28oU iwvKKhSLW9bH/c5puuaXqom+WoIb/yMAIFwh2O8klSGfzLZu4c4VM2vj6in1hZhMxgww 5LR0/tSodgjNpBhxiw6keYGz7u2nmsIOlueSixwUHS7pewkDDchYlFjbanpdguA9Ennq yqJA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1787942298; x=1788547098; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=v3nAcDLCkfTKlmodmQ0boAx10KLIo6Vy0ud2mfcvPW0=; b=BX1L0s7Gam3YnMGiPBUiGVrS4A5jb9wPMLzPPKkHKFQZB+iX8sK7j/xtL55N5JisCj cn8+FWmfpQvDqFF3bi6iieQ6Kkn1buuIPa1RpeWf6nSV/TpWpaUoyHz3kmtvLEXmVETB fGUZTkD8NkJWBZ8pQCoaHT/VuJvpRkeSVse8X+E2n4j4wG0bH/fe8jLgpTs21DuMdk+j YTG153Gsfm2hIttuvRk6pFe1cq0waa0ElO1wTBphpjJPFoDeZCKK20FB0042d9AWe9+/ KNsTY5ClUyhFhf7EdVizf0Neja8SXhi+0LUIDm3XhHHEUtyipPyGsQ9BtaOTM9evUgpF esLQ== X-Forwarded-Encrypted: i=1; AHgh+RrHFLwZTWwUw9xxT9DFQkgYE3lNYf6iTJFdT4vrdfBfw1I6L3Nn5oYz0BCXoIx8z724BjOADk3SoTE/5cg=@vger.kernel.org X-Gm-Message-State: AFuF++mkjqaV69dPXCds2eKXIGjbGznTXXg0Uh9YVk25rKWSE2MoadH7 ZtJ2IbLluBW0n5gvS3WkU5gMF5v7VSD3ISBmo7hY9lXZbw0asEOLNgSq X-Gm-Gg: AR+sD13oXPTEcF01eJEV/IaAwboSdHkm2D7KuATWY8kznCgz+yOaxxcTW1LLTH1priX 6Qe3FJSqHbzFQ+lEtEBS2UtpYDAqwESIOEA1fP4LN1YMU8L7hdMcB6QrQTIBecXM8vCWxVyIMjS evQPRf5JI4ezyA7sFVCpUr3vl52WKk714TdJWglrlTD9wV4rlzww3ADVJ12PxoGA/X2mOGyAzM2 UwKD0vn8Vc4k0XSPwdNZPIduUn0Ncuqdx5Lir/LN7Z09vlTbHLASq7KxL69aTB3hVLLIsIikfHI 1yVDPJIMdhQKFoA0+aa7yUM16qc6zlJMqihlvabCJZZMPtrvyM1iwcd/t1Jf8Lu0BLYdcywn2ce 2E7M9kDPRX1SZq+XNddo7oxd1++uUuLxsC8aPEA2zLZ1dDeFwtfXQpoR0ycym0Qmthfjkl5aEpv +55D1uxsZWwZu2nEOoJpK9Y6T+Y3UGtclzWT/l1zX29tb2LoRnMyq1kE0nqkbEq445spYGIH6Tt 3seKT7zHBFaTjTisA== X-Received: by 2002:aa7:88c1:0:b0:857:726d:2708 with SMTP id d2e1a72fcca58-857726d2e5amr2500397b3a.20.1787942297709; Fri, 28 Aug 2026 11:38:17 -0700 (PDT) Received: from aig-kmd-01.. ([120.133.49.20]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-856a2ea6fa3sm844230b3a.37.2026.08.28.11.38.08 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 28 Aug 2026 11:38:17 -0700 (PDT) From: Yin Tirui To: Andrew Morton , linux-mm@kvack.org Cc: David Hildenbrand , Lorenzo Stoakes , Dev Jain , Zi Yan , Baolin Wang , Barry Song , Lance Yang , Ryan Roberts , Nico Pache , Usama Arif , "Liam R . Howlett" , wangkefeng.wang@huawei.com, chenjun102@huawei.com, linux-kernel@vger.kernel.org, Yin Tirui Subject: [PATCH RFC 9/9] mm/huge_memory: unify the migration and device private PTE loops Date: Sat, 29 Aug 2026 02:33:19 +0800 Message-Id: <23d7c10efec57be848fd158f786473aa3b2f6e5c.1787941780.git.yintirui@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" When splitting a huge PMD into non-present PTEs, one loop installs migration entries while the other installs device private entries. They differ only in the leaf entry they build. Build the entry in split_pmd_make_softleaf() and use one loop. Freezing still installs migration entries even for a device private mapping, as before. No functional change intended. Signed-off-by: Yin Tirui --- mm/huge_memory.c | 89 +++++++++++++++++++++++------------------------- 1 file changed, 43 insertions(+), 46 deletions(-) diff --git a/mm/huge_memory.c b/mm/huge_memory.c index fdb751a1e525..8e0fd11da3d7 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -3245,6 +3245,42 @@ static bool split_huge_pmd_anon_rmap(const struct sp= lit_pmd_state *state, return false; } =20 +/* + * Build the leaf entry for the PTE entry describing @pfn, for a huge PMD = entry + * which is not restored as a present mapping. + */ +static softleaf_t split_pmd_make_softleaf(const struct split_pmd_state *st= ate, + unsigned long pfn) +{ + softleaf_t entry; + + if (state->is_device_private && !state->freeze) { + /* + * anon_exclusive was already propagated to the pages backing + * the PTE entries by split_huge_pmd_anon_rmap(), and accessed + * and dirty bits are not propagated via device private + * entries. + */ + if (state->write) + return make_writable_device_private_entry(pfn); + return make_readable_device_private_entry(pfn); + } + + if (state->write) + entry =3D make_writable_migration_entry(pfn); + else if (state->anon_exclusive) + entry =3D make_readable_exclusive_migration_entry(pfn); + else + entry =3D make_readable_migration_entry(pfn); + + if (state->young) + entry =3D make_migration_entry_young(entry); + if (state->dirty) + entry =3D make_migration_entry_dirty(entry); + + return entry; +} + /* * Replace an anonymous huge PMD entry with a page table mapping the same * folio at PTE granularity. @@ -3281,53 +3317,14 @@ static void split_huge_pmd_to_ptes(struct vm_area_s= truct *vma, * Note that NUMA hinting access restrictions are not transferred to * avoid any possibility of altering permissions across VMAs. */ - if (state->freeze || - (!state->is_present && !state->is_device_private)) { - pte_t entry; - swp_entry_t swp_entry; - - for (i =3D 0, addr =3D haddr; i < HPAGE_PMD_NR; i++, addr +=3D PAGE_SIZE= ) { - if (state->write) - swp_entry =3D make_writable_migration_entry( - page_to_pfn(page + i)); - else if (state->anon_exclusive) - swp_entry =3D make_readable_exclusive_migration_entry( - page_to_pfn(page + i)); - else - swp_entry =3D make_readable_migration_entry( - page_to_pfn(page + i)); - if (state->young) - swp_entry =3D make_migration_entry_young(swp_entry); - if (state->dirty) - swp_entry =3D make_migration_entry_dirty(swp_entry); - entry =3D swp_entry_to_pte(swp_entry); - if (state->soft_dirty) - entry =3D pte_swp_mksoft_dirty(entry); - if (state->uffd) - entry =3D pte_swp_mkuffd(entry); - VM_WARN_ON(!pte_none(ptep_get(pte + i))); - set_pte_at(mm, addr, pte + i, entry); - } - } else if (state->is_device_private) { - pte_t entry; - swp_entry_t swp_entry; + if (state->freeze || !state->is_present) { + for (i =3D 0, addr =3D haddr; i < HPAGE_PMD_NR; + i++, addr +=3D PAGE_SIZE) { + const unsigned long pfn =3D page_to_pfn(page + i); + const softleaf_t leaf =3D + split_pmd_make_softleaf(state, pfn); + pte_t entry =3D softleaf_to_pte(leaf); =20 - for (i =3D 0, addr =3D haddr; i < HPAGE_PMD_NR; i++, addr +=3D PAGE_SIZE= ) { - /* - * anon_exclusive was already propagated to the relevant - * pages corresponding to the pte entries when freeze - * is false. - */ - if (state->write) - swp_entry =3D make_writable_device_private_entry( - page_to_pfn(page + i)); - else - swp_entry =3D make_readable_device_private_entry( - page_to_pfn(page + i)); - /* - * Young and dirty bits are not progated via swp_entry - */ - entry =3D swp_entry_to_pte(swp_entry); if (state->soft_dirty) entry =3D pte_swp_mksoft_dirty(entry); if (state->uffd) --=20 2.34.1