From nobody Sat Sep 26 07:57:19 2026 Received: from mail-pf1-f181.google.com (mail-pf1-f181.google.com [209.85.210.181]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7092F4A260C for ; Thu, 3 Sep 2026 12:21:51 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.181 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788438117; cv=none; b=lcoZtojVkPD79as7ep3/W/WVucQONiaAQIL/WNIw2dEGUowxZqbQcnVSyGBi9eKLTUQveHsoCP9dUlM0Pf5ZnicV0rLBPKmYeVQugda5cxW2P5HKvKpemnhiyhu6gs6NZeaQaQLZ4YcTMdconi9Gw65d6SrOnsvsAxKfdhZ5tpI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788438117; c=relaxed/simple; bh=gSp03gDMBE517M+sfYMxfrdEqp8BxtiQyag0D0cdDZY=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=E3LPFbANuDT+BDZ31XQCFW1jDRQ4k5Ntz9yTnlvT/GXPEy+VScC9PuXM/7vZXSQ9TKo0kTdLCz5NnbanWrA+/nkiTIE1K6SbK3JAkekKSZ249lSbGDyCmKO1deomKiUuo4uglB0rm/usz1cRF5On0ccRlWSAFlEGpICEHj5ZdvY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com; spf=pass smtp.mailfrom=bytedance.com; dkim=pass (2048-bit key) header.d=bytedance.com header.i=@bytedance.com header.b=K/FkTuR+; arc=none smtp.client-ip=209.85.210.181 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=bytedance.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=bytedance.com header.i=@bytedance.com header.b="K/FkTuR+" Received: by mail-pf1-f181.google.com with SMTP id d2e1a72fcca58-84f38f3b36eso1949869b3a.1 for ; Thu, 03 Sep 2026 05:21:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1788438108; x=1789042908; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=pJ73qBOQiTpXKwSrjAZqmWemxXwNOfQBE65lRVHowjY=; b=K/FkTuR+tFUmn6R5hMiyhpSkfSjGDW8McKFqlDSYtKc6Yfi9iN3NmwHs8C1hFilCCB jwMPQB353w2D7zxkOqxaeyRW1dBnkaeFxu+bXaR21KNmor0K6gdukCjxwJwJCeL+sV41 FMKYaLW8bku3mfCDX7TA3QLEBEiqGNTx/Xoerxgvz2zlL/0I8GiZv9Ju1BukErzsqki+ 1fHLheoSghIJAAbX/YCGgqTKlHbeZ9CVlEiSnrADDoLG4nsO62lOZg/rYYzBNLHj+9lz TI8NivZWb8wTxSOYFnot29LvmNN2ZkWFb5/eHWq/Ium48lccpaafhMv3Aw2hRN35XhwI iE+g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1788438108; x=1789042908; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=pJ73qBOQiTpXKwSrjAZqmWemxXwNOfQBE65lRVHowjY=; b=f7rwInjlT55BPSOIgKszir9FSYQd2Q8myvXWF+P4IUge5CRtjf5qX1k2FadTdTh2R7 2cPTjEevy4Utw+j2TLCm4lm02ugzODtOO5BiItfKU1v0kh0Vi5Zyq2U6Le3kQBtYdJ7n 2B/Jk/qOHYtORETPUpIJ+MqTX8sP9pGK9zwglegGJFvh0uqv66gdl7H2d9XFUd3f572x YLxmX9Qv7N3+7iBdPw4rWbQJ5GBIzjQWHdBk6kWPUSa7o3itqG9mulcZc5p2j1gnY56Q qiF7asdbQX6wdtP0lczXp2LRzCWgKa0qeM8Sz5j/rnvGxraSjBXfvukfObSIiPkwAax+ sRww== X-Forwarded-Encrypted: i=1; AKwUvBxvej2wOEuTyaVi6dHXU/7Ntm5OreQUBUaj867E0Y0fbc9k0K9H5rA0LPjXnjHlM81W/H6/fG57P7MSmJo=@vger.kernel.org X-Gm-Message-State: AFuF++m//OSrjsNBs94fYX4d+KFPYLYEEvV1jL8etexmpCtulU1euc1O 24KgLJO9GA35i3+RCdYDE7l7ZDxK2ggUZlnc+DrmThl2AZddE8LXsCRcEN3VdWGvaLM= X-Gm-Gg: AYBFou2REy61HZYsv6Rpk0Kefajmf29xtNM5dJEjCuGTNh7WZR1VOxZ7R3YS6VTrFXZ E2mMr5G+AYGX/LVebT21YWpHy+a4hDb67l7qWQDioKgXDxDhIdVDv//D5R523KRMq+bcKEpRYdv I6qRTvO92hDn4BC0NZ4yRIFKlYJSG0hQBqz+B5Rzm/xVFL17Em4PvfyLIehYTkRS1OhuEVOEllG wwLT/QI9ZTXuHnBYrqF+tPhe9RswWQkvnMr+pXKwHlkR6E0oLIIPBfFLWIfrSx4oRKnqVdAOHwg 2PFO3EPaIJp3KNt5fp9/1p1l5OUGpaoHuPXtj8wwIQF0XwOA2t0hMUAo0Wl8KkKfpjjLSoz3zHO IkgwMN3JE4zAnK3LbcscwNEdYz29pvbKvtNZR7JTm32zc3Bk/wruJBPFi5EbyYYgKAMS/MV+keT Kce7SDgVUvl9gywIPO6f0I05VOOtHA7h8jJd7T/KjNuhM+4ZFYT83ecRlV0fAc0Ousf50D5FIK2 Cxvu1ZMC1JDMrD0WNw2YryLgA== X-Received: by 2002:a05:6a00:3021:b0:848:30c3:45dd with SMTP id d2e1a72fcca58-85ed2cbc58amr18768996b3a.11.1788438108017; Thu, 03 Sep 2026 05:21:48 -0700 (PDT) Received: from G6L4RL2QG9.bytedance.net ([61.213.176.13]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-85db24f2076sm2868126b3a.4.2026.09.03.05.21.43 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Thu, 03 Sep 2026 05:21:47 -0700 (PDT) From: Muchun Song To: Andrew Morton , Dan Williams , David Hildenbrand Cc: linux-mm@kvack.org, nvdimm@lists.linux.dev, linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-cxl@vger.kernel.org, Vishal Verma , Dave Jiang , Alison Schofield , Mike Rapoport , Oscar Salvador , Ira Weiny , Jan Kara , Matthew Wilcox , Lorenzo Stoakes , Vlastimil Babka , Michal Hocko , Qi Zheng , Muchun Song , muchun.song@linux.dev Subject: [PATCH 1/4] mm: generalize vmemmap remap architecture support Date: Thu, 3 Sep 2026 20:21:24 +0800 Message-ID: <20260903122128.12264-2-songmuchun@bytedance.com> X-Mailer: git-send-email 2.54.0 In-Reply-To: <20260903122128.12264-1-songmuchun@bytedance.com> References: <20260903122128.12264-1-songmuchun@bytedance.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" ARCH_WANT_OPTIMIZE_HUGETLB_VMEMMAP is named after its first user, but the underlying requirement is not HugeTLB-specific. Generic MM only needs to know whether an architecture can safely remap populated vmemmap PTEs at run time. Rename the architecture opt-in to ARCH_SUPPORTS_VMEMMAP_REMAP and make HugeTLB vmemmap optimization depend on that generic capability. This also prepares for FS-DAX vmemmap optimization, which relies on the same runtime remap support. Signed-off-by: Muchun Song Reviewed-by: Oscar Salvador --- arch/loongarch/Kconfig | 2 +- arch/riscv/Kconfig | 2 +- arch/x86/Kconfig | 2 +- fs/Kconfig | 2 +- mm/Kconfig | 6 ++++-- 5 files changed, 8 insertions(+), 6 deletions(-) diff --git a/arch/loongarch/Kconfig b/arch/loongarch/Kconfig index 9c5def706222..e9bf19172e86 100644 --- a/arch/loongarch/Kconfig +++ b/arch/loongarch/Kconfig @@ -72,6 +72,7 @@ config LOONGARCH select ARCH_SUPPORTS_RT select ARCH_SUPPORTS_SCHED_SMT if SMP select ARCH_SUPPORTS_SCHED_MC if SMP + select ARCH_SUPPORTS_VMEMMAP_REMAP if 64BIT select ARCH_USE_BUILTIN_BSWAP select ARCH_USE_CMPXCHG_LOCKREF select ARCH_USE_MEMTEST @@ -80,7 +81,6 @@ config LOONGARCH select ARCH_WANT_DEFAULT_BPF_JIT if HAVE_EBPF_JIT select ARCH_WANT_DEFAULT_TOPDOWN_MMAP_LAYOUT select ARCH_WANT_LD_ORPHAN_WARN - select ARCH_WANT_OPTIMIZE_HUGETLB_VMEMMAP if 64BIT select ARCH_WANTS_NO_INSTR select ARCH_WANTS_THP_SWAP if HAVE_ARCH_TRANSPARENT_HUGEPAGE select BUILDTIME_TABLE_SORT diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig index 5965666194b0..3614001cafdb 100644 --- a/arch/riscv/Kconfig +++ b/arch/riscv/Kconfig @@ -76,6 +76,7 @@ config RISCV select ARCH_SUPPORTS_RT select ARCH_SUPPORTS_SHADOW_CALL_STACK if HAVE_SHADOW_CALL_STACK select ARCH_SUPPORTS_SCHED_MC if SMP + select ARCH_SUPPORTS_VMEMMAP_REMAP select ARCH_USE_CMPXCHG_LOCKREF if 64BIT select ARCH_USE_MEMTEST select ARCH_USE_QUEUED_RWLOCKS @@ -88,7 +89,6 @@ config RISCV select ARCH_WANT_HUGE_PMD_SHARE if 64BIT select ARCH_WANT_LD_ORPHAN_WARN select ARCH_WANT_OPTIMIZE_DAX_VMEMMAP - select ARCH_WANT_OPTIMIZE_HUGETLB_VMEMMAP select ARCH_WANTS_NO_INSTR select ARCH_WANTS_THP_SWAP if HAVE_ARCH_TRANSPARENT_HUGEPAGE select ARCH_WEAK_RELEASE_ACQUIRE if ARCH_USE_QUEUED_SPINLOCKS diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig index a8c3b3d31a27..53f72d1500c5 100644 --- a/arch/x86/Kconfig +++ b/arch/x86/Kconfig @@ -133,6 +133,7 @@ config X86 select ARCH_SUPPORTS_LTO_CLANG select ARCH_SUPPORTS_LTO_CLANG_THIN select ARCH_SUPPORTS_RT + select ARCH_SUPPORTS_VMEMMAP_REMAP if X86_64 select ARCH_USE_BUILTIN_BSWAP select ARCH_USE_CMPXCHG_LOCKREF select ARCH_USE_MEMTEST @@ -148,7 +149,6 @@ config X86 select ARCH_WANT_HUGE_PMD_SHARE if X86_64 select ARCH_WANT_LD_ORPHAN_WARN select ARCH_WANT_OPTIMIZE_DAX_VMEMMAP if X86_64 - select ARCH_WANT_OPTIMIZE_HUGETLB_VMEMMAP if X86_64 select ARCH_WANTS_THP_SWAP if X86_64 select ARCH_HAS_PARANOID_L1D_FLUSH select ARCH_WANT_IRQS_OFF_ACTIVATE_MM diff --git a/fs/Kconfig b/fs/Kconfig index d1c210c6508f..655795fbbd1b 100644 --- a/fs/Kconfig +++ b/fs/Kconfig @@ -276,7 +276,7 @@ config HUGETLB_PAGE =20 config HUGETLB_PAGE_OPTIMIZE_VMEMMAP def_bool HUGETLB_PAGE - depends on ARCH_WANT_OPTIMIZE_HUGETLB_VMEMMAP + depends on ARCH_SUPPORTS_VMEMMAP_REMAP depends on SPARSEMEM_VMEMMAP =20 config HUGETLB_PMD_PAGE_TABLE_SHARING diff --git a/mm/Kconfig b/mm/Kconfig index c1ddf59c0d71..7ec734d89beb 100644 --- a/mm/Kconfig +++ b/mm/Kconfig @@ -463,12 +463,14 @@ config SPARSEMEM_VMEMMAP =20 # # Select this config option from the architecture Kconfig, if it is prefer= red -# to enable the feature of HugeTLB/dev_dax vmemmap optimization. +# to enable the feature of dev_dax vmemmap optimization. # config ARCH_WANT_OPTIMIZE_DAX_VMEMMAP bool =20 -config ARCH_WANT_OPTIMIZE_HUGETLB_VMEMMAP +# Select this from architectures where generic MM can safely remap populat= ed +# vmemmap PTEs at run time. +config ARCH_SUPPORTS_VMEMMAP_REMAP bool =20 config HAVE_MEMBLOCK_PHYS_MAP --=20 2.54.0 From nobody Sat Sep 26 07:57:19 2026 Received: from mail-pf1-f180.google.com (mail-pf1-f180.google.com [209.85.210.180]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 23A3E4A387D for ; Thu, 3 Sep 2026 12:21:54 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.180 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788438119; cv=none; b=PUUnZzK3Ku3thWikkG1NCjnw0crcFjdSqWChmXt+OyaekW/qEp29kxbjjctPHH5WlAfWihvmOlyMdfgdKMKqfR00L8oqncrQUqT4aJd2L+d4TAJgJdnBazF8DyqAxvEzeiBP/mQWzIq9Sn7UbY3FRGA4sq53jeSsqofR8+zq3QQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788438119; c=relaxed/simple; bh=iUjs+EiXrSNpfuKC5pykyao7Ehz6dcFc1lBNJ06asU4=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=fM5b8scbPb1IA7Svk1ohqshkonsdrPgd8izdolQuFy1VOp9WJ9cT9lCykvB3z2L6vXRYMe/KO62HRBMnWzX3jX31f9Xcil6zM95XMNfOV0cy6Sz/Iu89oALkWYJuG6O+Q/bREtjgN4ql12WanPkK3gP18q+aIjRJlNjKx6eMzlo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com; spf=pass smtp.mailfrom=bytedance.com; dkim=pass (2048-bit key) header.d=bytedance.com header.i=@bytedance.com header.b=HM8P+JNe; arc=none smtp.client-ip=209.85.210.180 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=bytedance.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=bytedance.com header.i=@bytedance.com header.b="HM8P+JNe" Received: by mail-pf1-f180.google.com with SMTP id d2e1a72fcca58-84e0688b7e8so2201899b3a.1 for ; Thu, 03 Sep 2026 05:21:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1788438113; x=1789042913; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=8wCVi0RPQ/1w5scAtdtuQVBMayOKGUht0GU0fLrik4o=; b=HM8P+JNeXozKprIxH883rNenR/o9M2W+wBj9fdZuwElwFzOkyitwEx5Mt0YhG5NWRK inKZgpUnbZGWPn1WoBGBmEwxh2wIyIRHgsoO1ZOCrl/vitP/SJfYocCbzZaxW+53kM4k UEbwW093laLXhGKAuHqmeVPFSqBG1YiqIcrvezNM/yp/xH4euQaRNyixTz1xB84LArZ2 BpywZzOOmqgKcDYishQLzhNw7qZX62830zj7cETarR3To3nmLmMA/Yo4ZdLbnm6XNFrR S2qXTOW0U1pKmPVhapjTTxhNOmNMVHsK1OJAPhMLWlwUrogk5QJYCLwYd92aB41O3gRY SJ4A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1788438113; x=1789042913; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=8wCVi0RPQ/1w5scAtdtuQVBMayOKGUht0GU0fLrik4o=; b=EmF6sCCB+Zjokgr3H5fmH2z08ZkVayzHMbE2+lhvCMD9FoDW/GysvThAz4I+NH7qx9 jqwMEp4Ysp88lk6+7vA+qOQjZsPPGjI4aKdlL63fvbvdei7dVpWquWsLRe93eE8e6bye 5/taplZeulnqoZdUmo8Ihr1EmtdD7vXP7nnGtERpPkbolPZNRA4gE+Atnx12mMgDgdO2 Lz60g7u6EyHY64zM7HgSd8yaEVUTqWkWKdZ49NLOMhZ+TgbjY37csjoRS+ncUeaL/Y2p T2V0rppygz7fOfi0WVXOqHL/gefOxP/kFloObq7AzsklbrzZTR4LKIyBP1MTzRkWfIO8 tVDQ== X-Forwarded-Encrypted: i=1; AKwUvBxPzeJJOVm/O6I4T6As36UDW5bNsmZdAlvyFNWk4EdjBHawScEE/v/JHdyTn++TjDukybMzwuUxabTWNvo=@vger.kernel.org X-Gm-Message-State: AFuF++nmU84rLqxTpNSqr0Fs7fzgSpesDe+gAwQsR8jqumuuZ5ANDkfV KvWJN5JGooouJ6MJln7j0e+rIy8Q/xDKBQ8ZYK2wK9pmanN+j2TvjjUGJHeMnDapFyM= X-Gm-Gg: AYBFou2FUtMxEp9UzFxRAX3keDpgLu6UYMnKw5SZ/ewlf8Cmvk5BK0tPVZ4Vxalt/I/ Sglxz5koZFbeHwwtIkW/okHZ26F8lcTOK3/pTgMpiKi7reWj1ynuJpDNM7HAUFnBWlOE4cU6FDr q0zU5qn5lvUM95Pm6cIoopUsOGvX+yLIBWIyKF2OE2yRd+mIxzu9nrfhrrASLhM56WprJd1uy1n mM94KWk+b0Jh4ba4bnJRYJhohN11IYnnmPKXlSpbD4R3DtmktW1lZqRN7yEwOTTyA1CWy5FZYD+ xyNEQt9xChzBF+8yf8iJajRCkz7SKh2FQTue1ZZWn9h92DMs7lpAZ4r+ZyuUiZX7ABUJRHmOpnn 8IBMY8YS8CBsPobezRooW4yZE9BNlfRlS/sw7D/B7yiIcoH+HV5/T73+x9+1rvo8W+Ufr2pSvH8 HGqmY/pMV4w02OnwGzr+O9LAbo6jYteRVxjdS7y0MekEBCMH9ZC7V1lSfgtL5hh/QZ/mU+RWRwz 8WXmrsXlSyr1HaDAzz8PsneMA== X-Received: by 2002:a05:6a00:e8f:b0:857:72f8:dc91 with SMTP id d2e1a72fcca58-85ed850a6d7mr17640794b3a.18.1788438112769; Thu, 03 Sep 2026 05:21:52 -0700 (PDT) Received: from G6L4RL2QG9.bytedance.net ([61.213.176.13]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-85db24f2076sm2868126b3a.4.2026.09.03.05.21.48 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Thu, 03 Sep 2026 05:21:52 -0700 (PDT) From: Muchun Song To: Andrew Morton , Dan Williams , David Hildenbrand Cc: linux-mm@kvack.org, nvdimm@lists.linux.dev, linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-cxl@vger.kernel.org, Vishal Verma , Dave Jiang , Alison Schofield , Mike Rapoport , Oscar Salvador , Ira Weiny , Jan Kara , Matthew Wilcox , Lorenzo Stoakes , Vlastimil Babka , Michal Hocko , Qi Zheng , Muchun Song , muchun.song@linux.dev Subject: [PATCH 2/4] nvdimm/pmem: avoid HWPoison flag updates for clean pages Date: Thu, 3 Sep 2026 20:21:25 +0800 Message-ID: <20260903122128.12264-3-songmuchun@bytedance.com> X-Mailer: git-send-email 2.54.0 In-Reply-To: <20260903122128.12264-1-songmuchun@bytedance.com> References: <20260903122128.12264-1-songmuchun@bytedance.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" pmem_mkpage_present() is called after persistent-memory poison has been cleared. It only needs to clear PG_hwpoison and undo the MCE nospec state for PFNs that are still marked poisoned. For clean PFNs, test_and_clear_pmem_poison() has no semantic effect, but it still performs an atomic clear operation against struct page flags. That is unnecessary today and conflicts with the FS-DAX vmemmap optimization added later in the series, where clean PFNs may be backed by shared read-only metadata. Check PageHWPoison() first so only poisoned PFNs update the page state. Poison recovery keeps the same behavior, while clean PFNs no longer require writable per-PFN metadata. Signed-off-by: Muchun Song Reviewed-by: Oscar Salvador Reviewed-by: Pankaj Gupta --- drivers/nvdimm/pmem.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/drivers/nvdimm/pmem.c b/drivers/nvdimm/pmem.c index 30a51c365ce8..b14f75daeda7 100644 --- a/drivers/nvdimm/pmem.c +++ b/drivers/nvdimm/pmem.c @@ -80,7 +80,7 @@ static void pmem_mkpage_present(struct pmem_device *pmem,= phys_addr_t offset, * here since we're in the driver I/O path and * outstanding I/O requests pin the dev_pagemap. */ - if (test_and_clear_pmem_poison(page)) + if (PageHWPoison(page) && test_and_clear_pmem_poison(page)) clear_mce_nospec(pfn); } } --=20 2.54.0 From nobody Sat Sep 26 07:57:19 2026 Received: from mail-pf1-f181.google.com (mail-pf1-f181.google.com [209.85.210.181]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D645C4A3F0D for ; Thu, 3 Sep 2026 12:21:59 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.181 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788438123; cv=none; b=nHx9WUJCMS+QN1t/r7CklnMBqQerhYziGplRELsgfKyWBRU+Mdov7YTcZWpHL/pMLY2TLBtqqh2ivnG9DA8BnXROmEoVQU6Q46N3A3PEhxj13Dq2lLrOSEEq3N/090QGqsm3kydA+UL5T2rBkSxBnyHCXU0OSicRvvdsDpHzWHY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788438123; c=relaxed/simple; bh=G8uv8NqcjN/pJXcO5D6thq5P01P4lWTMPeOJansg2MY=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=H6d6VZWZ44qwq4HVQbMkBElm7IM+15sNDe/7adbazmGXQMqkvFcTab9cOQHfuLUDJpeNBPUPy7NFDAFQTfxiPN3AzoibRFl6QP4avj6xHiM/1zbgBzjKaXTkiTWbQL3rek/sSRECMYIEAqtNSxvQZJVkZqfeio6p2HRw8RpkM5w= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com; spf=pass smtp.mailfrom=bytedance.com; dkim=pass (2048-bit key) header.d=bytedance.com header.i=@bytedance.com header.b=S+xTDtGC; arc=none smtp.client-ip=209.85.210.181 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=bytedance.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=bytedance.com header.i=@bytedance.com header.b="S+xTDtGC" Received: by mail-pf1-f181.google.com with SMTP id d2e1a72fcca58-84faf0fa17eso2540105b3a.2 for ; Thu, 03 Sep 2026 05:21:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1788438118; x=1789042918; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=6oJC5ERX01rpyohZHkG07vcg33WWq46jJVtIFvSG5mY=; b=S+xTDtGCU7/2kAp57h0DuBfGxsUpEf6pZfGG7xDiqcvEJUmDgeBzrOaEI34WGZEzXB hvuqVkbkRY+q7PiKPcf0UnTxqrq4gWtDIbNJ8Q2CyKpO9EYogPMoUuE502xDjqerh3hM gA7a58P/LZhWBC5/w1W539Vdxqu0o6UeVkA+JRvuahXHPWA0x0lmcI86GQXl0y4GMhuo YtnGEA4dbaPs7Dq3Zzc1L4Cs0OXAZi6e3OPt2h5CQEgxWxBXx1typibxA8rSvYfaKxUj ozeV67Km9fNInQoAlxqAanmH6AQcChbdfnHXuU/SG3xDRhxrLLWev95Zinzsswqh2iHc a9yw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1788438118; x=1789042918; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=6oJC5ERX01rpyohZHkG07vcg33WWq46jJVtIFvSG5mY=; b=Y2IPiPlhcJNv5iVtJW/gxvunKq2+BXPKcQbtsUk92s2NVlxlYTLk1Lr66r2WUjfZy9 t3W89eiCBKFaawUHkE6nXnnoP2r/c1eSunzL8av1kDN+83eBXJq4PxOXjcMydnFlLkgk gqtG7kl/5O2JdvU9phVn8wADOQkUv8/ZM6O3UfGQ4eKCiu760m5dQZhkmES5ohnC8stH PsXrkf1V2MSCvGwTh3qDzdzbh3oz/836ygV/v+3OZ9c+n+IS8OD5q+W2ZGhSo2oX8pIh Luy7OCxuY6fiIE9SXTEOWOJeNQQXX6SQBG5c0VQfnTfwOKvJI1fWMOzn1AXbhdocxMK1 iBtw== X-Forwarded-Encrypted: i=1; AKwUvBycCPyE7CHbKQTSfsD39Sh+eVvgh8SPLvEZ/0aKP/TRcqDAIQUyyhMwDkiwqg4GP4XqWlIQDumOQpTpK+M=@vger.kernel.org X-Gm-Message-State: AFuF++kSVuI4SV041+dJckoKmhH2dtR1NKIFAWNDFVVRb3A6dgFkZ/s1 R5Gh2kkdV1PQ2U1YvkCG3CeMW5EYvlaOa397oAKChZY+y1HPT1tNfsJwo3/t145YvCU= X-Gm-Gg: AYBFou3nP4Tw8/4pjf4XVgHlHqLNrs4nbSUpjoGHuZDUkBAblYJ+6fbJiXli5/AiOvV ac0HL3P/t/dGvs9rvQWyeDps3r7Yeeulz1n6hBYPQEkVkOnr7wvCAf0fan4bs9y+uhSPrazrEpj 4JZTY41QBovdE7lYtdrSOTHLidWjTtlLAjELve5vJYuJplkPkWl5Vz1Gjya+ViMWglAT3PzquGh aCDsD97pvyr3sKXgspYxGBApKGVOV+38vWcQUC3h4fABzErwjBBeG9cbFAi+ZFzdX5xN8qDMhHK f0CWFq76av5ANTo9EjgkRdktEXYIcdohCmD38vdoZvWeFO8djmmyirMLEJ+xv0i7RppAiDTmcIp /uPNTliK9TF50+gZaiK4FqX+InEwVNUl0YVh/aIOyHHirf5mwbYgbAooFidp+HBVhHlLLZOF91z 13jgPt5KkzqMo0zHMxQGqYVhqHowktGv5w+bZxnpqdsBpurF4S13I1h6o+RKRHzoPWNcMp7PQMA sc4havBRxdgfmIoA3yBn5T7/g== X-Received: by 2002:a05:6a00:4601:b0:84f:a7bf:8fb9 with SMTP id d2e1a72fcca58-85ed7743221mr15576091b3a.3.1788438117860; Thu, 03 Sep 2026 05:21:57 -0700 (PDT) Received: from G6L4RL2QG9.bytedance.net ([61.213.176.13]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-85db24f2076sm2868126b3a.4.2026.09.03.05.21.53 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Thu, 03 Sep 2026 05:21:57 -0700 (PDT) From: Muchun Song To: Andrew Morton , Dan Williams , David Hildenbrand Cc: linux-mm@kvack.org, nvdimm@lists.linux.dev, linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-cxl@vger.kernel.org, Vishal Verma , Dave Jiang , Alison Schofield , Mike Rapoport , Oscar Salvador , Ira Weiny , Jan Kara , Matthew Wilcox , Lorenzo Stoakes , Vlastimil Babka , Michal Hocko , Qi Zheng , Muchun Song , muchun.song@linux.dev Subject: [PATCH 3/4] mm: add shared read-only vmemmap support for FS-DAX Date: Thu, 3 Sep 2026 20:21:26 +0800 Message-ID: <20260903122128.12264-4-songmuchun@bytedance.com> X-Mailer: git-send-email 2.54.0 In-Reply-To: <20260903122128.12264-1-songmuchun@bytedance.com> References: <20260903122128.12264-1-songmuchun@bytedance.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" FS-DAX registers persistent-memory ranges as ZONE_DEVICE memory, and the kernel normally allocates and initializes vmemmap storage for every advertised PFN up front. Sparse pmem images and workloads that only use the DAX direct-access path may never need writable per-PFN state for most of that range, but still pay the memory and initialization cost. Add an opt-in dev_pagemap mode that populates FS-DAX vmemmap PTEs from a shared read-only metadata page. The shared page is initialized with the common ZONE_DEVICE and dev_pagemap state, so every PFN still has a valid struct page representation while private metadata allocation is deferred. This relies on sizeof(struct page) being a power of two, so each vmemmap page contains a naturally aligned and repeatable set of struct page slots. It also requires architecture support for runtime vmemmap remapping, because shared mappings must be replaced with private writable pages before a PFN can enter userspace mappings. The initial implementation is deliberately limited to a single memory-block-aligned range. That is not a fundamental requirement, but keeps the registration and teardown paths simple; support for multiple ranges or less strict alignment can be added later. Provide vmemmap_materialize_page() to replace shared mappings in the requested metadata range with private writable copies. A later patch will call it from the FS-DAX fault path. No caller enables the mode yet. Signed-off-by: Muchun Song --- include/linux/memremap.h | 11 ++++++- mm/memremap.c | 38 ++++++++++++++++++++++-- mm/mm_init.c | 11 +++++++ mm/sparse-vmemmap.c | 64 +++++++++++++++++++++++++++++++++++++--- 4 files changed, 116 insertions(+), 8 deletions(-) diff --git a/include/linux/memremap.h b/include/linux/memremap.h index e3c2ccf872a8..21c9b6aeef67 100644 --- a/include/linux/memremap.h +++ b/include/linux/memremap.h @@ -9,6 +9,7 @@ =20 struct resource; struct device; +struct page; =20 /** * struct vmem_altmap - pre-allocated storage for vmemmap_populate @@ -108,7 +109,8 @@ struct dev_pagemap_ops { void (*folio_split)(struct folio *head, struct folio *tail); }; =20 -#define PGMAP_ALTMAP_VALID (1 << 0) +#define PGMAP_ALTMAP_VALID BIT(0) +#define PGMAP_VMEMMAP_OPTIMIZATION BIT(1) =20 /** * struct dev_pagemap - metadata for ZONE_DEVICE mappings @@ -122,6 +124,7 @@ struct dev_pagemap_ops { * A zero value (default) uses base pages as the vmemmap metadata * representation. A bigger value will set up compound struct pages * of the requested order value. + * @vmemmap_shared_page: shared read-only vmemmap page for optimized FS-DAX * @ops: method table * @owner: an opaque pointer identifying the entity that manages this * instance. Used by various helpers to make sure that no @@ -137,6 +140,7 @@ struct dev_pagemap { enum memory_type type; unsigned int flags; unsigned long vmemmap_shift; + struct page *vmemmap_shared_page; const struct dev_pagemap_ops *ops; void *owner; int nr_range; @@ -232,6 +236,7 @@ void *devm_memremap_pages(struct device *dev, struct de= v_pagemap *pgmap); void devm_memunmap_pages(struct device *dev, struct dev_pagemap *pgmap); struct dev_pagemap *get_dev_pagemap(unsigned long pfn); bool pgmap_pfn_valid(struct dev_pagemap *pgmap, unsigned long pfn); +int vmemmap_materialize_page(struct page *page, unsigned int order); =20 unsigned long memremap_compat_align(void); =20 @@ -307,4 +312,8 @@ static inline void put_dev_pagemap(struct dev_pagemap *= pgmap) percpu_ref_put(&pgmap->ref); } =20 +static inline bool pgmap_vmemmap_optimizable(struct dev_pagemap *pgmap) +{ + return pgmap && pgmap->vmemmap_shared_page !=3D NULL; +} #endif /* _LINUX_MEMREMAP_H_ */ diff --git a/mm/memremap.c b/mm/memremap.c index accba23aef28..a53d09b84eaa 100644 --- a/mm/memremap.c +++ b/mm/memremap.c @@ -3,6 +3,7 @@ #include #include #include +#include #include #include #include @@ -83,6 +84,30 @@ static unsigned long pfn_len(struct dev_pagemap *pgmap, = unsigned long range_id) pfn_first(pgmap, range_id)) >> pgmap->vmemmap_shift; } =20 +static int pgmap_vmemmap_shared_page_alloc(struct dev_pagemap *pgmap, int = nid) +{ + const struct range *range =3D &pgmap->range; + + if (!is_power_of_2(sizeof(struct page)) || + !IS_ENABLED(CONFIG_ARCH_SUPPORTS_VMEMMAP_REMAP) || + !(pgmap->flags & PGMAP_VMEMMAP_OPTIMIZATION)) + return 0; + + if (pgmap->nr_range !=3D 1 || + !IS_ALIGNED(range->start | range_len(range), MIN_MEMORY_BLOCK_SIZE)) + return 0; + + pgmap->vmemmap_shared_page =3D alloc_pages_node(nid, GFP_KERNEL, 0); + + return pgmap->vmemmap_shared_page ? 0 : -ENOMEM; +} + +static inline void pgmap_vmemmap_shared_page_free(struct dev_pagemap *pgma= p) +{ + if (pgmap->vmemmap_shared_page) + put_page(pgmap->vmemmap_shared_page); +} + static void pageunmap_range(struct dev_pagemap *pgmap, int range_id) { struct range *range =3D &pgmap->ranges[range_id]; @@ -93,8 +118,9 @@ static void pageunmap_range(struct dev_pagemap *pgmap, i= nt range_id) =20 /* pages are dead and unused, undo the arch mapping */ mem_hotplug_begin(); - remove_pfn_range_from_zone(page_zone(first_page), PHYS_PFN(range->start), - PHYS_PFN(range_len(range))); + if (!pgmap_vmemmap_optimizable(pgmap)) + remove_pfn_range_from_zone(page_zone(first_page), PHYS_PFN(range->start), + PHYS_PFN(range_len(range))); if (pgmap->type =3D=3D MEMORY_DEVICE_PRIVATE) { __remove_pages(PHYS_PFN(range->start), PHYS_PFN(range_len(range)), NULL, pgmap); @@ -123,6 +149,7 @@ void memunmap_pages(struct dev_pagemap *pgmap) =20 for (i =3D 0; i < pgmap->nr_range; i++) pageunmap_range(pgmap, i); + pgmap_vmemmap_shared_page_free(pgmap); percpu_ref_exit(&pgmap->ref); =20 WARN_ONCE(pgmap->altmap.alloc, "failed to free all reserved pages\n"); @@ -310,6 +337,9 @@ void *memremap_pages(struct dev_pagemap *pgmap, int nid) break; case MEMORY_DEVICE_FS_DAX: params.pgprot =3D pgprot_decrypted(params.pgprot); + error =3D pgmap_vmemmap_shared_page_alloc(pgmap, nid); + if (error) + return ERR_PTR(error); break; case MEMORY_DEVICE_GENERIC: break; @@ -324,8 +354,10 @@ void *memremap_pages(struct dev_pagemap *pgmap, int ni= d) init_completion(&pgmap->done); error =3D percpu_ref_init(&pgmap->ref, dev_pagemap_percpu_release, 0, GFP_KERNEL); - if (error) + if (error) { + pgmap_vmemmap_shared_page_free(pgmap); return ERR_PTR(error); + } =20 /* * Clear the pgmap nr_range as it will be incremented for each diff --git a/mm/mm_init.c b/mm/mm_init.c index 2ed17cc707ed..7dd03b8a8d28 100644 --- a/mm/mm_init.c +++ b/mm/mm_init.c @@ -33,6 +33,7 @@ #include #include #include +#include #include "internal.h" #include "mm_init.h" #include "page_alloc.h" @@ -1133,6 +1134,15 @@ void __ref memmap_init_zone_device(struct zone *zone, if (!nr_pages) return; =20 + if (pgmap_vmemmap_optimizable(pgmap)) { + struct page *page =3D page_address(pgmap->vmemmap_shared_page); + + for (int i =3D 0; i < PAGE_SIZE / sizeof(struct page); i++) + __init_zone_device_page(page + i, start_pfn + i, + ZONE_DEVICE, nid, pgmap); + goto pageblock_init; + } + /* * Seed the reusable head-page template from the first real struct * page. The normal page-init and refcount helpers must operate on @@ -1164,6 +1174,7 @@ void __ref memmap_init_zone_device(struct zone *zone, compound_nr_pages(pfn, altmap, pgmap)); } =20 +pageblock_init: pageblock_migratetype_init_range(start_pfn, nr_pages, MIGRATE_MOVABLE, /* isolate */ false, /* atomic */ false); =20 diff --git a/mm/sparse-vmemmap.c b/mm/sparse-vmemmap.c index e62e6aa07f12..4dc7f020ed10 100644 --- a/mm/sparse-vmemmap.c +++ b/mm/sparse-vmemmap.c @@ -37,6 +37,8 @@ */ /* Get a ref on the head page struct page, for ZONE_DEVICE compound pages = */ #define VMEMMAP_POPULATE_PAGEREF 0x0001 +/* Read-only shared vmemmap mappings for FS-DAX base pages */ +#define VMEMMAP_POPULATE_FSDAX_SHARED 0x0002 =20 #include "internal.h" #include "mm_init.h" @@ -262,10 +264,11 @@ static pte_t * __meminit vmemmap_pte_populate(pmd_t *= pmd, unsigned long addr, in * and through vmemmap_populate_compound_pages() when * slab is available. */ - if (flags & VMEMMAP_POPULATE_PAGEREF) + if (flags & (VMEMMAP_POPULATE_PAGEREF | VMEMMAP_POPULATE_FSDAX_SHARED)) get_page(pfn_to_page(ptpfn)); } - entry =3D pfn_pte(ptpfn, PAGE_KERNEL); + entry =3D pfn_pte(ptpfn, flags & VMEMMAP_POPULATE_FSDAX_SHARED ? + PAGE_KERNEL_RO : PAGE_KERNEL); set_pte_at(&init_mm, addr, pte, entry); } else if (WARN_ON_ONCE(vmemmap_optimizable_pfn(pfn))) return NULL; @@ -379,6 +382,54 @@ int __meminit vmemmap_populate_basepages(unsigned long= start, unsigned long end, return vmemmap_populate_range(start, end, node, altmap, -1, 0); } =20 +#ifdef CONFIG_ZONE_DEVICE +static int __vmemmap_materialize_page(struct page *page) +{ + unsigned long addr =3D PAGE_ALIGN_DOWN((unsigned long)page); + struct dev_pagemap *pgmap =3D page_pgmap(page); + struct page *candidate, *template =3D pgmap->vmemmap_shared_page; + pte_t *pte =3D virt_to_kpte(addr); + + if (pte_page(ptep_get(pte)) !=3D template) + return 0; + + candidate =3D alloc_pages_node(page_to_nid(page), GFP_KERNEL, 0); + if (!candidate) + return -ENOMEM; + copy_page(page_address(candidate), page_address(template)); + + spin_lock(&init_mm.page_table_lock); + if (pte_page(ptep_get(pte)) !=3D template) { + __free_page(candidate); + goto out; + } + /* Make the copied struct page contents visible before the PTE update. */ + smp_wmb(); + set_pte_at(&init_mm, addr, pte, mk_pte(candidate, PAGE_KERNEL)); + flush_tlb_kernel_range(addr, addr + PAGE_SIZE); + put_page(template); +out: + spin_unlock(&init_mm.page_table_lock); + + return 0; +} + +int vmemmap_materialize_page(struct page *page, unsigned int order) +{ + struct dev_pagemap *pgmap =3D page_pgmap(page); + unsigned long end =3D (unsigned long)(page + (1UL << order)); + + if (!pgmap_vmemmap_optimizable(pgmap)) + return 0; + + for (unsigned long addr =3D (unsigned long)page; addr < end; addr +=3D PA= GE_SIZE) + if (__vmemmap_materialize_page((struct page *)addr)) + return -ENOMEM; + + return 0; +} +#endif /* CONFIG_ZONE_DEVICE */ + /* * Write protect the mirrored tail page structs for HVO. This will be * called from the hugetlb code when gathering and initializing the @@ -581,7 +632,11 @@ struct page * __meminit __populate_section_memmap(unsi= gned long pfn, !IS_ALIGNED(nr_pages, PAGES_PER_SUBSECTION))) return NULL; =20 - if (vmemmap_can_optimize(altmap, pgmap)) + if (pgmap_vmemmap_optimizable(pgmap)) + r =3D vmemmap_populate_range(start, end, nid, NULL, + page_to_pfn(pgmap->vmemmap_shared_page), + VMEMMAP_POPULATE_FSDAX_SHARED); + else if (vmemmap_can_optimize(altmap, pgmap)) r =3D vmemmap_populate_compound_pages(pfn, start, end, nid, pgmap); else r =3D vmemmap_populate(start, end, nid, altmap); @@ -887,7 +942,8 @@ int __meminit sparse_add_section(int nid, unsigned long= start_pfn, * Poison uninitialized struct pages in order to catch invalid flags * combinations. */ - page_init_poison(memmap, sizeof(struct page) * nr_pages); + if (!pgmap_vmemmap_optimizable(pgmap)) + page_init_poison(memmap, sizeof(struct page) * nr_pages); =20 ms =3D __nr_to_section(section_nr); __section_mark_present(ms, section_nr); --=20 2.54.0 From nobody Sat Sep 26 07:57:19 2026 Received: from mail-pf1-f182.google.com (mail-pf1-f182.google.com [209.85.210.182]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id ACA144A4414 for ; Thu, 3 Sep 2026 12:22:05 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.182 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788438130; cv=none; b=UFSkx9ZTz/RjwugyjhtsreDqLFQCkRG81AbnsQtYAZnMuvGvQEYWpvDfvyJiVWoRSNjwcvIHneOul9ih/1gC/0NJoD2/r3SXky73haPWC03BgcFiNtuki0zDY2r5LiWJkM3wpjLv8ATInFOBG49go3MliCo+FUDnQ+pNTfy1iXY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788438130; c=relaxed/simple; bh=Hr4ffWVPhBSeZxC4Gy1Y+eU1zBxu0o3ECl+FjVnr0+4=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=mhwH0/35j84R7B6bIb6o0v6fhPAdg1B4g2y+n0ZrhbcnBZwSWtyynTNSzFSfV6nrnP+09nc6BsvL2hyZngjQbXW4YHCC6sOkXqSyU+cOESo3+kAZvH04vA0lZNh0GUsm60KMufdtBdvu6CtLFgtlqizG2T0CL0k8jQvU50xaLr0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com; spf=pass smtp.mailfrom=bytedance.com; dkim=pass (2048-bit key) header.d=bytedance.com header.i=@bytedance.com header.b=fFO7tnnD; arc=none smtp.client-ip=209.85.210.182 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=bytedance.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=bytedance.com header.i=@bytedance.com header.b="fFO7tnnD" Received: by mail-pf1-f182.google.com with SMTP id d2e1a72fcca58-84e84a6c4bfso961076b3a.1 for ; Thu, 03 Sep 2026 05:22:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1788438124; x=1789042924; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=M9z1xem6qEcuUVJMDQZaw5sCruZ2xCngKTtcTUgfBOA=; b=fFO7tnnDxTTCUkjZOG4JvdCV0i/1Zz3lsZuFqgBFxgNldaUN6MLzVcl5zIDpiA3xnN VxWl2cMTtWWoD+dHP8V79AJuchoWxjsked/qWQNHoLg8K9MWjoFlBPX3TO3W9fjikA7h 4DxC02UULyhhor4jil23ePTAXAuMCndbYkatmY9H1vPhlnC3Ctx6z6vLTnaunZZGiRmA 3mNBeK7JnT8lpyVw/WzRViXqVuZHFimCMZ/56HEDEDfcYASDG1ihOAtcgC1q19t9QotA GeZJjWrkbo8UIZqynnfzjYAcgzVDVGTVWNyIFcSDrTo10FE2kHHWfTwaoZNnI7dFI0lr iwQg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1788438124; x=1789042924; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=M9z1xem6qEcuUVJMDQZaw5sCruZ2xCngKTtcTUgfBOA=; b=e0IjfLQjC0dnzrK9Ffss6AvnJ/8FUQNvVjqqwNCT+Q42+3/90tZlP+9CGoAvvRhBqo m7//FpdQ/qmOSpw+Nio+arFx8VbHAGqhOy02uXfBWrWJ6B52MR6HQ1GfwKmnk8ewPNwW Y0lK/ux/I1pDVAjkvv/n/Q1GUORIoyO7x3p0yHbpyqTXX+Vh25dMPtq01lpGfdqkqmrj pC84Ra16KhaJE6h0EUjHLeYvXjUF/bpJaRx3iYvjy9uLQT8PJe3XsAHmlyWgiX5CZaLc yaV15w095s73Qw/4l16TZDP7XoCPcz2wTUgdF02BFSuDelKwFYq9FM82eQipGyMKZZXO IVYQ== X-Forwarded-Encrypted: i=1; AKwUvBygAxZ/b6drLT0DV1DYi3w4rZjhdNJMYNs4o9hUcy8KL8oINl72Q6aAHjYtXeeoHKG/gc3MV3ASbox+pGE=@vger.kernel.org X-Gm-Message-State: AFuF++kUEVFhD8zwmHFUyAPcW9c/3CZbjxa602Ci+MAVmDkYDFr1PNLP am+oq8IOOS3yXfS/kv/JwyIUxtKrrTE0dUFg/EkaJtYwG6gO4alFAwZjf9f/EP9QFFA= X-Gm-Gg: AYBFou1Nq8uQEJtHsGPYPLPcfTdE+rKp6DtjUp3rBOd97W05jnt+aPyzUJcmtmoirTZ 6pIy7p7PrdG4wB6J3SabeRXiSyM9gH4BxiHH8+oHWBizX2GZ2AkIm/y+MsnwtjMfD5khkxKH9z6 vXDK5fcqncd1bj9e6+Zr+7Wjzk3xl2uk57y0yMsQmX8lCwNwarXGncrr/UHDbCMtkM8cdK/vnDD 5gRn4/DG0YFB9M0WZD7jT6ToI0dkzkisbAnwDBUlGPEDh+UCeV1C1LZVSDns7af+qveWE4MvsYs 9rBh4liuwGZE/h4JOHtD7nrie751YNTGugbi+qtmoZPIkh7Ia6be2Ijf/5Zt/pN+xSOL8/RlOds XmwNg9VtKg+kV/XMamSb/jz4GVAWVpHpxzbVY31+cBgWET4pJIckYQvzaiRWbVOXvShwTSQQUst 6X/XYo6+ZTuyj3x25XZv/QicbqZ7wzLzqloD3wWFsxf87NJ/klxGhESRgXSxMI+9MfppkdDrRBI nAUMREPNSbRYqhn4rgBTliKog== X-Received: by 2002:a05:6a00:17aa:b0:84e:c851:a058 with SMTP id d2e1a72fcca58-85ffcb616aamr4808528b3a.10.1788438123307; Thu, 03 Sep 2026 05:22:03 -0700 (PDT) Received: from G6L4RL2QG9.bytedance.net ([61.213.176.13]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-85db24f2076sm2868126b3a.4.2026.09.03.05.21.58 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Thu, 03 Sep 2026 05:22:02 -0700 (PDT) From: Muchun Song To: Andrew Morton , Dan Williams , David Hildenbrand Cc: linux-mm@kvack.org, nvdimm@lists.linux.dev, linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-cxl@vger.kernel.org, Vishal Verma , Dave Jiang , Alison Schofield , Mike Rapoport , Oscar Salvador , Ira Weiny , Jan Kara , Matthew Wilcox , Lorenzo Stoakes , Vlastimil Babka , Michal Hocko , Qi Zheng , Muchun Song , muchun.song@linux.dev Subject: [PATCH 4/4] fsdax: materialize pmem vmemmap metadata on faults Date: Thu, 3 Sep 2026 20:21:27 +0800 Message-ID: <20260903122128.12264-5-songmuchun@bytedance.com> X-Mailer: git-send-email 2.54.0 In-Reply-To: <20260903122128.12264-1-songmuchun@bytedance.com> References: <20260903122128.12264-1-songmuchun@bytedance.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Pmem FS-DAX registers the device range as ZONE_DEVICE memory, and the kernel normally allocates and initializes vmemmap storage for every advertised PFN up front. Sparse backing storage and workloads that only use the DAX direct-access path may never need writable per-PFN state for most of that range, but still pay the memory and initialization cost. Opt pmem FS-DAX into the shared read-only vmemmap mode and materialize private metadata before inserting a PFN into a userspace mapping. PMD faults materialize the whole PMD-sized metadata range. PFNs that are never faulted continue to use the shared metadata. This shifts private vmemmap allocation from device registration to the first DAX fault for each metadata page, so the struct page overhead tracks the faulted DAX working set instead of the full advertised pmem device size. Signed-off-by: Muchun Song --- drivers/nvdimm/pmem.c | 1 + fs/dax.c | 5 +++++ 2 files changed, 6 insertions(+) diff --git a/drivers/nvdimm/pmem.c b/drivers/nvdimm/pmem.c index b14f75daeda7..5530b32163d3 100644 --- a/drivers/nvdimm/pmem.c +++ b/drivers/nvdimm/pmem.c @@ -543,6 +543,7 @@ static int pmem_attach_disk(struct device *dev, pmem->pgmap.nr_range =3D 1; pmem->pgmap.type =3D MEMORY_DEVICE_FS_DAX; pmem->pgmap.ops =3D &fsdax_pagemap_ops; + pmem->pgmap.flags |=3D PGMAP_VMEMMAP_OPTIMIZATION; addr =3D devm_memremap_pages(dev, &pmem->pgmap); bb_range =3D pmem->pgmap.range; } else { diff --git a/fs/dax.c b/fs/dax.c index 1fbba0d21c13..89377301d51b 100644 --- a/fs/dax.c +++ b/fs/dax.c @@ -13,6 +13,7 @@ #include #include #include +#include #include #include #include @@ -1877,6 +1878,10 @@ static vm_fault_t dax_fault_iter(struct vm_fault *vm= f, if (err) return pmd ? VM_FAULT_FALLBACK : dax_fault_return(err); =20 + err =3D vmemmap_materialize_page(pfn_to_page(pfn), pmd ? PMD_ORDER : 0); + if (err) + return dax_fault_return(err); + *entry =3D dax_insert_entry(xas, vmf, iter, *entry, pfn, entry_flags); =20 if (write && iomap->flags & IOMAP_F_SHARED) { --=20 2.54.0