From nobody Thu Oct 2 13:00:44 2025 Received: from cstnet.cn (smtp81.cstnet.cn [159.226.251.81]) (using TLSv1.2 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A828D1D5147; Wed, 17 Sep 2025 03:38:03 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=159.226.251.81 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1758080286; cv=none; b=nWak4XIFu7FbUwWIAegtLB/8YYDoR54bDofzTyQz4IS5kHI6cH7ov5ImMcgHOoL4Bmntih2mWdfdtcMOchZc49l/LEL1RCMOfRz9ovoMUaUCYn9ihw98cs8Z4x9JfUQ4vQlJSH5bUC2VF5GiTL23YCpZ33l/NdjxTywHNXbozHA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1758080286; c=relaxed/simple; bh=koc6EEdLy4wa5XJQGPSoGwU9yO2X6LPF2uqMad3vvaA=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=lxH6sud/pA13pdUefLSp6EzQYjJkzyzI6BAZ4Au4bdRAQCfDPmcmeqpoJizzJGBNJNVewhTioYcxPd9tWK9vs4/Frfp6rywrd9HqeMPxIwcsAk07+JR1lQoarbm4NepCq1tFJDTvYbI5DsmfBtwSBy0FsC4b8laT2Lk/c/a/U0k= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=iscas.ac.cn; spf=pass smtp.mailfrom=iscas.ac.cn; arc=none smtp.client-ip=159.226.251.81 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=iscas.ac.cn Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=iscas.ac.cn Received: from ubt.. (unknown [210.73.43.101]) by APP-03 (Coremail) with SMTP id rQCowABnsXvsLMpojtxAAw--.607S4; Wed, 17 Sep 2025 11:37:24 +0800 (CST) From: Chunyan Zhang To: linux-riscv@lists.infradead.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org Cc: Paul Walmsley , Palmer Dabbelt , Albert Ou , Alexandre Ghiti , Conor Dooley , Deepak Gupta , Ved Shanbhogue , Alexander Viro , Christian Brauner , Jan Kara , Andrew Morton , Peter Xu , Arnd Bergmann , David Hildenbrand , Lorenzo Stoakes , "Liam R . Howlett" , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Axel Rasmussen , Yuanchu Xie , Chunyan Zhang Subject: [PATCH V13 2/6] mm: userfaultfd: Add pgtable_supports_uffd_wp() Date: Wed, 17 Sep 2025 11:36:59 +0800 Message-Id: <20250917033703.1695933-3-zhangchunyan@iscas.ac.cn> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20250917033703.1695933-1-zhangchunyan@iscas.ac.cn> References: <20250917033703.1695933-1-zhangchunyan@iscas.ac.cn> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-CM-TRANSID: rQCowABnsXvsLMpojtxAAw--.607S4 X-Coremail-Antispam: 1UD129KBjvJXoW3Gr1DGFWDXr1kZF4UCr1rZwb_yoWfKryxpF 4rGw4UXw4ktFy8Ja97AF48A3s5Zw4SgryDGryF93WkAa13t390vryFkF4rKr93Jr48Wry7 tF4UtrZ5ur42yFUanT9S1TB71UUUUU7qnTZGkaVYY2UrUUUUjbIjqfuFe4nvWSU5nxnvy2 9KBjDU0xBIdaVrnRJUUUmmb7Iv0xC_Kw4lb4IE77IF4wAFF20E14v26rWj6s0DM7CY07I2 0VC2zVCF04k26cxKx2IYs7xG6rWj6s0DM7CIcVAFz4kK6r1j6r18M28IrcIa0xkI8VA2jI 8067AKxVWUXwA2048vs2IY020Ec7CjxVAFwI0_Xr0E3s1l8cAvFVAK0II2c7xJM28CjxkF 64kEwVA0rcxSw2x7M28EF7xvwVC0I7IYx2IY67AKxVW8JVW5JwA2z4x0Y4vE2Ix0cI8IcV CY1x0267AKxVWxJVW8Jr1l84ACjcxK6I8E87Iv67AKxVWxJr0_GcWl84ACjcxK6I8E87Iv 6xkF7I0E14v26rxl6s0DM2AIxVAIcxkEcVAq07x20xvEncxIr21l5I8CrVACY4xI64kE6c 02F40Ex7xfMcIj6xIIjxv20xvE14v26r106r15McIj6I8E87Iv67AKxVWUJVW8JwAm72CE 4IkC6x0Yz7v_Jr0_Gr1lF7xvr2IYc2Ij64vIr41lFIxGxcIEc7CjxVA2Y2ka0xkIwI1lc7 CjxVAaw2AFwI0_GFv_Wrylc2xSY4AK67AK6r48MxAIw28IcxkI7VAKI48JMxC20s026xCa FVCjc4AY6r1j6r4UMI8I3I0E5I8CrVAFwI0_Jr0_Jr4lx2IqxVCjr7xvwVAFwI0_JrI_Jr Wlx4CE17CEb7AF67AKxVW8ZVWrXwCIc40Y0x0EwIxGrwCI42IY6xIIjxv20xvE14v26r1j 6r1xMIIF0xvE2Ix0cI8IcVCY1x0267AKxVWxJVW8Jr1lIxAIcVCF04k26cxKx2IYs7xG6r 1j6r1xMIIF0xvEx4A2jsIE14v26r1j6r4UMIIF0xvEx4A2jsIEc7CjxVAFwI0_Gr0_Gr1U YxBIdaVFxhVjvjDU0xZFpf9x07bVkuxUUUUU= X-CM-SenderInfo: x2kd0wxfkx051dq6x2xfdvhtffof0/1tbiCQ8IB2jJ8TDyAgAAsZ Content-Type: text/plain; charset="utf-8" Some platforms can customize the PTE/PMD entry uffd-wp bit making it unavailable even if the architecture provides the resource. This patch adds a macro API that allows architectures to define their specific implementations to check if the uffd-wp bit is available on which device the kernel is running. Also this patch is removing "ifdef CONFIG_HAVE_ARCH_USERFAULTFD_WP" and "ifdef CONFIG_PTE_MARKER_UFFD_WP" in favor of pgtable_supports_uffd_wp() and uffd_supports_wp_marker() checks respectively that default to IS_ENABLED(CONFIG_HAVE_ARCH_USERFAULTFD_WP) and "IS_ENABLED(CONFIG_HAVE_ARCH_USERFAULTFD_WP) && IS_ENABLED(CONFIG_PTE_MARKE= R_UFFD_WP)" if not overridden by the architecture, no change in behavior is expected. Acked-by: David Hildenbrand Signed-off-by: Chunyan Zhang --- fs/userfaultfd.c | 22 +++--- include/asm-generic/pgtable_uffd.h | 17 +++++ include/linux/mm_inline.h | 12 ++- include/linux/userfaultfd_k.h | 114 ++++++++++++++++------------- mm/memory.c | 6 +- 5 files changed, 106 insertions(+), 65 deletions(-) diff --git a/fs/userfaultfd.c b/fs/userfaultfd.c index 54c6cc7fe9c6..e41736ffa202 100644 --- a/fs/userfaultfd.c +++ b/fs/userfaultfd.c @@ -1270,9 +1270,9 @@ static int userfaultfd_register(struct userfaultfd_ct= x *ctx, if (uffdio_register.mode & UFFDIO_REGISTER_MODE_MISSING) vm_flags |=3D VM_UFFD_MISSING; if (uffdio_register.mode & UFFDIO_REGISTER_MODE_WP) { -#ifndef CONFIG_HAVE_ARCH_USERFAULTFD_WP - goto out; -#endif + if (!pgtable_supports_uffd_wp()) + goto out; + vm_flags |=3D VM_UFFD_WP; } if (uffdio_register.mode & UFFDIO_REGISTER_MODE_MINOR) { @@ -1980,14 +1980,14 @@ static int userfaultfd_api(struct userfaultfd_ctx *= ctx, uffdio_api.features &=3D ~(UFFD_FEATURE_MINOR_HUGETLBFS | UFFD_FEATURE_MINOR_SHMEM); #endif -#ifndef CONFIG_HAVE_ARCH_USERFAULTFD_WP - uffdio_api.features &=3D ~UFFD_FEATURE_PAGEFAULT_FLAG_WP; -#endif -#ifndef CONFIG_PTE_MARKER_UFFD_WP - uffdio_api.features &=3D ~UFFD_FEATURE_WP_HUGETLBFS_SHMEM; - uffdio_api.features &=3D ~UFFD_FEATURE_WP_UNPOPULATED; - uffdio_api.features &=3D ~UFFD_FEATURE_WP_ASYNC; -#endif + if (!pgtable_supports_uffd_wp()) + uffdio_api.features &=3D ~UFFD_FEATURE_PAGEFAULT_FLAG_WP; + + if (!uffd_supports_wp_marker()) { + uffdio_api.features &=3D ~UFFD_FEATURE_WP_HUGETLBFS_SHMEM; + uffdio_api.features &=3D ~UFFD_FEATURE_WP_UNPOPULATED; + uffdio_api.features &=3D ~UFFD_FEATURE_WP_ASYNC; + } =20 ret =3D -EINVAL; if (features & ~uffdio_api.features) diff --git a/include/asm-generic/pgtable_uffd.h b/include/asm-generic/pgtab= le_uffd.h index 828966d4c281..0d85791efdf7 100644 --- a/include/asm-generic/pgtable_uffd.h +++ b/include/asm-generic/pgtable_uffd.h @@ -1,6 +1,23 @@ #ifndef _ASM_GENERIC_PGTABLE_UFFD_H #define _ASM_GENERIC_PGTABLE_UFFD_H =20 +/* + * Some platforms can customize the uffd-wp bit, making it unavailable + * even if the architecture provides the resource. + * Adding this API allows architectures to add their own checks for the + * devices on which the kernel is running. + * Note: When overriding it, please make sure the + * CONFIG_HAVE_ARCH_USERFAULTFD_WP is part of this macro. + */ +#ifndef pgtable_supports_uffd_wp +#define pgtable_supports_uffd_wp() IS_ENABLED(CONFIG_HAVE_ARCH_USERFAULTFD= _WP) +#endif + +static inline bool uffd_supports_wp_marker(void) +{ + return pgtable_supports_uffd_wp() && IS_ENABLED(CONFIG_PTE_MARKER_UFFD_WP= ); +} + #ifndef CONFIG_HAVE_ARCH_USERFAULTFD_WP static __always_inline int pte_uffd_wp(pte_t pte) { diff --git a/include/linux/mm_inline.h b/include/linux/mm_inline.h index d6c1011b38f2..c69162812ba6 100644 --- a/include/linux/mm_inline.h +++ b/include/linux/mm_inline.h @@ -553,7 +553,6 @@ static inline pte_marker copy_pte_marker( =20 return dstm; } -#endif =20 /* * If this pte is wr-protected by uffd-wp in any form, arm the special pte= to @@ -571,9 +570,15 @@ static inline bool pte_install_uffd_wp_if_needed(struct vm_area_struct *vma, unsigned long ad= dr, pte_t *pte, pte_t pteval) { -#ifdef CONFIG_PTE_MARKER_UFFD_WP bool arm_uffd_pte =3D false; =20 + /* + * Some platforms can customize the PTE uffd-wp bit, making it unavailable + * even if the architecture allows providing the PTE resource. + */ + if (!uffd_supports_wp_marker()) + return false; + /* The current status of the pte should be "cleared" before calling */ WARN_ON_ONCE(!pte_none(ptep_get(pte))); =20 @@ -602,7 +607,7 @@ pte_install_uffd_wp_if_needed(struct vm_area_struct *vm= a, unsigned long addr, make_pte_marker(PTE_MARKER_UFFD_WP)); return true; } -#endif + return false; } =20 @@ -616,5 +621,6 @@ static inline bool vma_has_recency(const struct vm_area= _struct *vma) =20 return true; } +#endif =20 #endif diff --git a/include/linux/userfaultfd_k.h b/include/linux/userfaultfd_k.h index c0e716aec26a..4ccc79b5731e 100644 --- a/include/linux/userfaultfd_k.h +++ b/include/linux/userfaultfd_k.h @@ -228,15 +228,14 @@ static inline bool vma_can_userfault(struct vm_area_s= truct *vma, if (wp_async && (vm_flags =3D=3D VM_UFFD_WP)) return true; =20 -#ifndef CONFIG_PTE_MARKER_UFFD_WP /* * If user requested uffd-wp but not enabled pte markers for * uffd-wp, then shmem & hugetlbfs are not supported but only * anonymous. */ - if ((vm_flags & VM_UFFD_WP) && !vma_is_anonymous(vma)) + if (!uffd_supports_wp_marker() && (vm_flags & VM_UFFD_WP) && + !vma_is_anonymous(vma)) return false; -#endif =20 /* By default, allow any of anon|shmem|hugetlb */ return vma_is_anonymous(vma) || is_vm_hugetlb_page(vma) || @@ -291,6 +290,67 @@ void userfaultfd_release_new(struct userfaultfd_ctx *c= tx); void userfaultfd_release_all(struct mm_struct *mm, struct userfaultfd_ctx *ctx); =20 +static inline bool userfaultfd_wp_use_markers(struct vm_area_struct *vma) +{ + /* Only wr-protect mode uses pte markers */ + if (!userfaultfd_wp(vma)) + return false; + + /* File-based uffd-wp always need markers */ + if (!vma_is_anonymous(vma)) + return true; + + /* + * Anonymous uffd-wp only needs the markers if WP_UNPOPULATED + * enabled (to apply markers on zero pages). + */ + return userfaultfd_wp_unpopulated(vma); +} + +static inline bool pte_marker_entry_uffd_wp(swp_entry_t entry) +{ + if (!uffd_supports_wp_marker()) + return false; + + return is_pte_marker_entry(entry) && + (pte_marker_get(entry) & PTE_MARKER_UFFD_WP); +} + +static inline bool pte_marker_uffd_wp(pte_t pte) +{ + swp_entry_t entry; + + if (!uffd_supports_wp_marker()) + return false; + + if (!is_swap_pte(pte)) + return false; + + entry =3D pte_to_swp_entry(pte); + + return pte_marker_entry_uffd_wp(entry); +} + +/* + * Returns true if this is a swap pte and was uffd-wp wr-protected in eith= er + * forms (pte marker or a normal swap pte), false otherwise. + */ +static inline bool pte_swp_uffd_wp_any(pte_t pte) +{ + if (!uffd_supports_wp_marker()) + return false; + + if (!is_swap_pte(pte)) + return false; + + if (pte_swp_uffd_wp(pte)) + return true; + + if (pte_marker_uffd_wp(pte)) + return true; + + return false; +} #else /* CONFIG_USERFAULTFD */ =20 /* mm helpers */ @@ -415,68 +475,24 @@ static inline bool vma_has_uffd_without_event_remap(s= truct vm_area_struct *vma) return false; } =20 -#endif /* CONFIG_USERFAULTFD */ - static inline bool userfaultfd_wp_use_markers(struct vm_area_struct *vma) { - /* Only wr-protect mode uses pte markers */ - if (!userfaultfd_wp(vma)) return false; - - /* File-based uffd-wp always need markers */ - if (!vma_is_anonymous(vma)) - return true; - - /* - * Anonymous uffd-wp only needs the markers if WP_UNPOPULATED - * enabled (to apply markers on zero pages). - */ - return userfaultfd_wp_unpopulated(vma); } =20 static inline bool pte_marker_entry_uffd_wp(swp_entry_t entry) { -#ifdef CONFIG_PTE_MARKER_UFFD_WP - return is_pte_marker_entry(entry) && - (pte_marker_get(entry) & PTE_MARKER_UFFD_WP); -#else - return false; -#endif + return false; } =20 static inline bool pte_marker_uffd_wp(pte_t pte) { -#ifdef CONFIG_PTE_MARKER_UFFD_WP - swp_entry_t entry; - - if (!is_swap_pte(pte)) return false; - - entry =3D pte_to_swp_entry(pte); - - return pte_marker_entry_uffd_wp(entry); -#else - return false; -#endif } =20 -/* - * Returns true if this is a swap pte and was uffd-wp wr-protected in eith= er - * forms (pte marker or a normal swap pte), false otherwise. - */ static inline bool pte_swp_uffd_wp_any(pte_t pte) { -#ifdef CONFIG_PTE_MARKER_UFFD_WP - if (!is_swap_pte(pte)) return false; - - if (pte_swp_uffd_wp(pte)) - return true; - - if (pte_marker_uffd_wp(pte)) - return true; -#endif - return false; } - +#endif /* CONFIG_USERFAULTFD */ #endif /* _LINUX_USERFAULTFD_K_H */ diff --git a/mm/memory.c b/mm/memory.c index 39ed698dfc37..a47621b35194 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -1593,7 +1593,9 @@ zap_install_uffd_wp_if_needed(struct vm_area_struct *= vma, { bool was_installed =3D false; =20 -#ifdef CONFIG_PTE_MARKER_UFFD_WP + if (!uffd_supports_wp_marker()) + return false; + /* Zap on anonymous always means dropping everything */ if (vma_is_anonymous(vma)) return false; @@ -1610,7 +1612,7 @@ zap_install_uffd_wp_if_needed(struct vm_area_struct *= vma, pte++; addr +=3D PAGE_SIZE; } -#endif + return was_installed; } =20 --=20 2.34.1