From nobody Thu Oct 9 13:18:00 2025 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B698E2ED843 for ; Tue, 17 Jun 2025 15:43:53 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1750175035; cv=none; b=dZK0OVMbV9HTrzdaVrHoSuYLXVYvAPwZ6jgZjjkS5QHXnm8LsN4P1ZOc1Sz0dJuLf6vbwySjPze+ukBa9l7pU1Ss32m5gZGl/UoBfhWjJWn7FribukVqB2lXHKZsLf3zbi/nsB3wfRkIgExilvES67YHELP7KgkBacSZ7+PQNNM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1750175035; c=relaxed/simple; bh=gfOK1w8At4LCO+SEz2YxE1sL/b3knNVAuIBGQzCExJ8=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=I3mQyGGQVoszh1BDZucy+txuNzsLaMKKkGJYfONXjg1v0KSVyc+Bo3mQJZOJmlA5HBOlSYjT9WZLW7/F0TiERV2/sd2RKe7SSb/io/gd3/DLnaBLlzYD8Lvt5nKtcrLHRsujFkupZIWeBV07cqotbeDqxBOGup/R4x+kijQ45r4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=LWRBb6bZ; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="LWRBb6bZ" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1750175032; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=t4hXdfl2BxaEs0JquHL/H5jXrrSdl8LRqFfGRUF6aUs=; b=LWRBb6bZ2Z5q5H37/6a19Fy8rnQnzW6tO4tCL9e2vYVTJtAK3pfUIPqpIPWiN5DUod0t5c Q6GhzxzS8cH4Wo9uTxOqDCH71hQWgQVh0eecxXuCgAykEotKypxhAWDovfwGPFoWlrOb9o ruWOGk+yJh5IhdpydUZO4RH4cpt1Tbw= Received: from mail-wr1-f70.google.com (mail-wr1-f70.google.com [209.85.221.70]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-320-hRgb-HucOey_TeIhho_60A-1; Tue, 17 Jun 2025 11:43:51 -0400 X-MC-Unique: hRgb-HucOey_TeIhho_60A-1 X-Mimecast-MFC-AGG-ID: hRgb-HucOey_TeIhho_60A_1750175030 Received: by mail-wr1-f70.google.com with SMTP id ffacd0b85a97d-3a5780e8137so1596753f8f.1 for ; Tue, 17 Jun 2025 08:43:51 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1750175030; x=1750779830; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=t4hXdfl2BxaEs0JquHL/H5jXrrSdl8LRqFfGRUF6aUs=; b=aAcZbliPIr16hCPE/PFyZYrF8aPDK/TG04UCnNRbv01peWxcqjEEYrdvVtpk0t2wky V08hgA+VBtdOTcCndZ8eNxpWNe5THCzTvSABRWvcQ1aleWVFo4unFuFmdX/xZRk6hZMB R5RjK5zJUh1zTGlZVJVfAs87ay2gkiabYAaCZ3l+0pjooU+8yNXpgNY11+N4h5aGmF0i N5QFsBfNHvnWR1lVBMwvkQcryzOsuRC7tWtNj9R6Erxc0pOJiDCAFyrwGqNPjPyYmpFc ZjiZljYfXXHKx59L3GEmwcKjplj6+rzhHagAPh+vG7gwUr90mL/rA2TziHyZGTVL8eMW mu8g== X-Gm-Message-State: AOJu0Yw79GUlp2fUvpiHGGoVem6l0aqtbAWPh7TIvCjWmtjz1W8/pGL5 7+ZMLVETDgCp8AoNQgSTo976S2Nm/hx8Iv8x9CI8F+NGjUQipmx7ILMmUIuGNGyG2U7N8LZLWOE D5iYNPSCod9FvxmgNLBYjeAknrJubzQmG420+1qAMOWnrQvDDqkSVmJS6w5IWrmRnt16hWSJLZx ayPWvQRrcv2l7kC15Skn5+0lNsi+jryzDXVde3xgwEZKmmBsIC X-Gm-Gg: ASbGncumaoByK+CmLR6KSQ0DuuJMhp/tfXERsPI0BvJyZRbm0+qXfzZvv/rc39Y2gON VuzICtNfG3jKulo3bEaZ9H8AJ8Z2790A01iWSKy610iLGhD4tsPIloli+3624B/pH7gFxcBgTP8 mIPVz0N+CtyjSm81IFg0Y4pSqC1wGaF66yVOCR6orjyE8ae7voT5cWlqt+SYor9VQYUD1HQIjE5 +V3Khqw2fGIkt9LmegsDMPQtvtmlXF2I0967DzPI5T9ARpLXbIVHewIg/DQkr+xjsU30jTYp8cv tgAIp9Z/tU31HmPrrX1CbyXd0Dt/l+wEuwIlLdySU1+94/dum4ZJ1YAYGvQr8T2nhmi2d/0KKjN qNN5ByA== X-Received: by 2002:a05:6000:1884:b0:3a4:eb7a:2ccb with SMTP id ffacd0b85a97d-3a56d821e4dmr12961334f8f.16.1750175030047; Tue, 17 Jun 2025 08:43:50 -0700 (PDT) X-Google-Smtp-Source: AGHT+IFcoD8JqLOB4EjbN+k5qD0zJS7SAB63dB1ZbzMQjH6b78ztWbJgN3nPYj9/mZ7cyKIKJjAtyA== X-Received: by 2002:a05:6000:1884:b0:3a4:eb7a:2ccb with SMTP id ffacd0b85a97d-3a56d821e4dmr12961278f8f.16.1750175029492; Tue, 17 Jun 2025 08:43:49 -0700 (PDT) Received: from localhost (p200300d82f3107003851c66ab6b93490.dip0.t-ipconnect.de. [2003:d8:2f31:700:3851:c66a:b6b9:3490]) by smtp.gmail.com with UTF8SMTPSA id ffacd0b85a97d-3a568a54a36sm14542075f8f.15.2025.06.17.08.43.48 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Tue, 17 Jun 2025 08:43:49 -0700 (PDT) From: David Hildenbrand To: linux-kernel@vger.kernel.org Cc: linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, nvdimm@lists.linux.dev, David Hildenbrand , Andrew Morton , Juergen Gross , Stefano Stabellini , Oleksandr Tyshchenko , Dan Williams , Alistair Popple , Matthew Wilcox , Jan Kara , Alexander Viro , Christian Brauner , Zi Yan , Baolin Wang , Lorenzo Stoakes , "Liam R. Howlett" , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Jann Horn , Pedro Falcato Subject: [PATCH RFC 01/14] mm/memory: drop highest_memmap_pfn sanity check in vm_normal_page() Date: Tue, 17 Jun 2025 17:43:32 +0200 Message-ID: <20250617154345.2494405-2-david@redhat.com> X-Mailer: git-send-email 2.49.0 In-Reply-To: <20250617154345.2494405-1-david@redhat.com> References: <20250617154345.2494405-1-david@redhat.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" In 2009, we converted a VM_BUG_ON(!pfn_valid(pfn)) to the current highest_memmap_pfn sanity check in commit 22b31eec63e5 ("badpage: vm_normal_page use print_bad_pte"), because highest_memmap_pfn was readily available. Nowadays, this is the last remaining highest_memmap_pfn user, and this sanity check is not really triggering ... frequently. Let's convert it to VM_WARN_ON_ONCE(!pfn_valid(pfn)), so we can simplify and get rid of highest_memmap_pfn. Checking for pfn_to_online_page() might be even better, but it would not handle ZONE_DEVICE properly. Do the same in vm_normal_page_pmd(), where we don't even report a problem at all ... What might be better in the future is having a runtime option like page-table-check to enable such checks dynamically on-demand. Something for the future. Signed-off-by: David Hildenbrand Reviewed-by: Lance Yang Reviewed-by: Oscar Salvador --- mm/memory.c | 15 +++------------ 1 file changed, 3 insertions(+), 12 deletions(-) diff --git a/mm/memory.c b/mm/memory.c index 0163d127cece9..188b84ebf479a 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -590,7 +590,7 @@ struct page *vm_normal_page(struct vm_area_struct *vma,= unsigned long addr, =20 if (IS_ENABLED(CONFIG_ARCH_HAS_PTE_SPECIAL)) { if (likely(!pte_special(pte))) - goto check_pfn; + goto out; if (vma->vm_ops && vma->vm_ops->find_special_page) return vma->vm_ops->find_special_page(vma, addr); if (vma->vm_flags & (VM_PFNMAP | VM_MIXEDMAP)) @@ -608,9 +608,6 @@ struct page *vm_normal_page(struct vm_area_struct *vma,= unsigned long addr, if (vma->vm_flags & VM_MIXEDMAP) { if (!pfn_valid(pfn)) return NULL; - if (is_zero_pfn(pfn)) - return NULL; - goto out; } else { unsigned long off; off =3D (addr - vma->vm_start) >> PAGE_SHIFT; @@ -624,17 +621,12 @@ struct page *vm_normal_page(struct vm_area_struct *vm= a, unsigned long addr, if (is_zero_pfn(pfn)) return NULL; =20 -check_pfn: - if (unlikely(pfn > highest_memmap_pfn)) { - print_bad_pte(vma, addr, pte, NULL); - return NULL; - } - /* * NOTE! We still have PageReserved() pages in the page tables. * eg. VDSO mappings can cause them to exist. */ out: + VM_WARN_ON_ONCE(!pfn_valid(pfn)); VM_WARN_ON_ONCE(is_zero_pfn(pfn)); return pfn_to_page(pfn); } @@ -676,14 +668,13 @@ struct page *vm_normal_page_pmd(struct vm_area_struct= *vma, unsigned long addr, =20 if (is_huge_zero_pmd(pmd)) return NULL; - if (unlikely(pfn > highest_memmap_pfn)) - return NULL; =20 /* * NOTE! We still have PageReserved() pages in the page tables. * eg. VDSO mappings can cause them to exist. */ out: + VM_WARN_ON_ONCE(!pfn_valid(pfn)); return pfn_to_page(pfn); } =20 --=20 2.49.0 From nobody Thu Oct 9 13:18:00 2025 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D1B302EE602 for ; Tue, 17 Jun 2025 15:43:57 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1750175040; cv=none; b=bBCuMSEhsKY/R1pAmw3WYhFtsELR1DDGqFhnDizMbzv+FrZXg8DaPF7VRpgTKN976sMWE657S3+aqbNseBEnRxlHeW9sK3rQuhsUjQLIfUFcHyZbwFMDnDcOxc8hQTtcLPzmIe9+3xwDhMhI8Hnhi2voQTxwDTYWreMRzYFDdzw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1750175040; c=relaxed/simple; bh=N+sqTwgnUIZg4czKbr4/JT7TGmW4bJe3WXX3qzBFdJI=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=Jgh1igekPvtvMT2SzQ6b1Yv1w7Z0MvWkjYc2OXdFWhg3eqwwm48kaK/EmMZnri56ulqr7IOqxD82zRzPUFPIvDoe8ef7+AWnfa3gUPMNACJTM7VTkq0DqVQvQpS1JNLpNY5Rw6Is3JUbSoaGXR2SScsLnk2WEA4vJrPqwmx4ghk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=BUrxcXAv; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="BUrxcXAv" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1750175036; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=n+enD/DBO3lH97hAndG2Fd7ZdK35XEPxGRz1KxKwpT4=; b=BUrxcXAvh1yUzsLJhN5hN3tYddRQmbNUxMyaQDIALcPrapUH6JcnVyNbtzr+96w7lRxY6A 4gQo3mGIN908YLw29+mv2KLXFDVtKyYDlGNQuW7mMWJHdA11KgQ3O3TKj89vZIfnh1jI9k 4iSG3NXhXmrgQUwvtD2ExDm3LonLEOI= Received: from mail-wr1-f69.google.com (mail-wr1-f69.google.com [209.85.221.69]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-240-5KnkyIbsNYqe3wLS2h_38g-1; Tue, 17 Jun 2025 11:43:53 -0400 X-MC-Unique: 5KnkyIbsNYqe3wLS2h_38g-1 X-Mimecast-MFC-AGG-ID: 5KnkyIbsNYqe3wLS2h_38g_1750175032 Received: by mail-wr1-f69.google.com with SMTP id ffacd0b85a97d-3a4eb6fcd88so3507350f8f.1 for ; Tue, 17 Jun 2025 08:43:53 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1750175032; x=1750779832; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=n+enD/DBO3lH97hAndG2Fd7ZdK35XEPxGRz1KxKwpT4=; b=QWFUMbqF19bZvooJhUBviSkWsQtjS5DTO1uZNH0GMuVt7MViEmr3nzHqZALVNPhbaB GLcNDLrocsE0Qk+w8aC1RTnnxqvWVYed3+5+xFmo3A812QYvCReGQcbh/fDsUnrRR/Fk cHEHaR2Vc+p10tUKLe9wOVPuCFKx53Ee1bNDkXSMmhwEiL1lzR6abhjgAdYGxdBgneMJ 1yiT+pyDnu07Jg6LlNzrVAm9Sao9WoqpC6RmtoqRCq1btGmINNamQ/sRt9h8RfWqVvkw zVh1lKeWpzcy9QarYajL4SQJHId+9wJ0A2WmTcuww8mDgOzm8gLuhAWHNxOh0Wfkre2l HimA== X-Gm-Message-State: AOJu0YxYT7mFZ9qcDtYqulcdPQkuvglo6U+RpdGdUolKlnIIz4D+cglD RLovIcQsLt4J8xRC57JbjW2vXJslLg4yEnb4HWozWcgJIaoV8uN1YSw5rsAX2iI1LMh58iCBD5F BrX1aryATAoNcXnhr+wYOGOm/khB6flaBqQcaRO3FoVyZjVWDU9+P9hiswtHrDBKuqtmjVwqyzO oom2LqXPC3wejXn/oIEPPw2LC6CAI9AnmEzZU/foxEP12NaDUO X-Gm-Gg: ASbGncu5xoUwR9kEYOBZM4xNFRAMEWzNwxpYkTXT66l4qaLo82lESUBcGCFWlVXrRu4 DSyXn1a32b2RRlYlHjkmZjCgSTfAmBD/F0fzWdQrhf2LAJ7fZ8cXFSqSuQi3ffXHwt9QkVfWJJa nMS+/QVtvt5ANxroijynPgT6IGXquFzHYDA1/NMC3KvI6vXPBsq5JxtjJ+cNbuhhdj8XfBvBuPC CqB7gM6ZkOwIlo0FkSbsl6EBDDa0S811HEX5SI3SUEZixJ+C+AjiAelHATNOFpTv0Hj9T5kEHUS aNh9/kkjE22rd0xBdUcaPB4cdoUzYZa9IaxW6ARiS+j0skKMSREnT/FSF55YuTX1uh6TmluIKLE DK76Yfw== X-Received: by 2002:a5d:5847:0:b0:3a5:1c0d:85e8 with SMTP id ffacd0b85a97d-3a57237797bmr10355657f8f.22.1750175032166; Tue, 17 Jun 2025 08:43:52 -0700 (PDT) X-Google-Smtp-Source: AGHT+IGgGKPI0HPFQFNSDiU1UoCT4KWg9jjW+HS33TdpBy2263RBIYIhTEc5b/vsOH/vxH2erO7KQw== X-Received: by 2002:a5d:5847:0:b0:3a5:1c0d:85e8 with SMTP id ffacd0b85a97d-3a57237797bmr10355602f8f.22.1750175031623; Tue, 17 Jun 2025 08:43:51 -0700 (PDT) Received: from localhost (p200300d82f3107003851c66ab6b93490.dip0.t-ipconnect.de. [2003:d8:2f31:700:3851:c66a:b6b9:3490]) by smtp.gmail.com with UTF8SMTPSA id ffacd0b85a97d-3a568a547ecsm14548596f8f.17.2025.06.17.08.43.50 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Tue, 17 Jun 2025 08:43:51 -0700 (PDT) From: David Hildenbrand To: linux-kernel@vger.kernel.org Cc: linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, nvdimm@lists.linux.dev, David Hildenbrand , Andrew Morton , Juergen Gross , Stefano Stabellini , Oleksandr Tyshchenko , Dan Williams , Alistair Popple , Matthew Wilcox , Jan Kara , Alexander Viro , Christian Brauner , Zi Yan , Baolin Wang , Lorenzo Stoakes , "Liam R. Howlett" , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Jann Horn , Pedro Falcato Subject: [PATCH RFC 02/14] mm: drop highest_memmap_pfn Date: Tue, 17 Jun 2025 17:43:33 +0200 Message-ID: <20250617154345.2494405-3-david@redhat.com> X-Mailer: git-send-email 2.49.0 In-Reply-To: <20250617154345.2494405-1-david@redhat.com> References: <20250617154345.2494405-1-david@redhat.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Now unused, so let's drop it. Signed-off-by: David Hildenbrand Reviewed-by: Oscar Salvador Reviewed-by: Pedro Falcato --- mm/internal.h | 2 -- mm/memory.c | 2 -- mm/mm_init.c | 3 --- mm/nommu.c | 1 - 4 files changed, 8 deletions(-) diff --git a/mm/internal.h b/mm/internal.h index f519eb7217c26..703871905fd6d 100644 --- a/mm/internal.h +++ b/mm/internal.h @@ -503,8 +503,6 @@ static inline bool folio_needs_release(struct folio *fo= lio) (mapping && mapping_release_always(mapping)); } =20 -extern unsigned long highest_memmap_pfn; - /* * Maximum number of reclaim retries without progress before the OOM * killer is consider the only way forward. diff --git a/mm/memory.c b/mm/memory.c index 188b84ebf479a..a1b5575db52ac 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -146,8 +146,6 @@ __setup("norandmaps", disable_randmaps); unsigned long zero_pfn __read_mostly; EXPORT_SYMBOL(zero_pfn); =20 -unsigned long highest_memmap_pfn __read_mostly; - /* * CONFIG_MMU architectures set up ZERO_PAGE in their paging_init() */ diff --git a/mm/mm_init.c b/mm/mm_init.c index 5c21b3af216b2..1dac66c209984 100644 --- a/mm/mm_init.c +++ b/mm/mm_init.c @@ -881,9 +881,6 @@ void __meminit memmap_init_range(unsigned long size, in= t nid, unsigned long zone unsigned long pfn, end_pfn =3D start_pfn + size; struct page *page; =20 - if (highest_memmap_pfn < end_pfn - 1) - highest_memmap_pfn =3D end_pfn - 1; - #ifdef CONFIG_ZONE_DEVICE /* * Honor reservation requested by the driver for this ZONE_DEVICE diff --git a/mm/nommu.c b/mm/nommu.c index 38c22ea0a95c6..cd9ddbfe1af80 100644 --- a/mm/nommu.c +++ b/mm/nommu.c @@ -42,7 +42,6 @@ #include #include "internal.h" =20 -unsigned long highest_memmap_pfn; int heap_stack_gap =3D 0; =20 atomic_long_t mmap_pages_allocated; --=20 2.49.0 From nobody Thu Oct 9 13:18:00 2025 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id EA2152EE603 for ; Tue, 17 Jun 2025 15:43:57 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1750175040; cv=none; b=Iq0m6gLS+NBhdVkqzXd0Wwfq/8b23ho47J81TBbQbo1DQCdBJkWuToRRPkx4CvQZq6WwUpBoOYhfCW37NdXi+Q/ykxHl1X//CjdMAEhqqBAOIN4lqIFVQ/tCJtVXNf2waYZDSXCpMJEcPvXGLiaz5evNrizJWYVTGSQZIx7Nlm0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1750175040; c=relaxed/simple; bh=QWNOR+1eFkXIAwi4188jXafhSE+uLJ9EpozLL4/g7bA=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=rPu5ixi7NV2+wNmqmHe4XW7pvwjtu5tASAFSkIF6m2g3E7U0OlK9QXhR/HCJ+E5GvDGpqzQwYMR9HeCmsbLx07eMu7iX08DJS23mJoPySZFM3N6BNLCooCBJIYeXsw3x3KY6v2mNj61qkGAavmBinaHj59YJEW4V0AhYPrd7k58= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=fpMMW94D; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="fpMMW94D" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1750175037; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=VIojkghjiOfK3qS9mo4araH3tv/iIJHq9NK1Pu9RhVw=; b=fpMMW94DQPqVLGF7wHVQtxI2EHwx/DaQmeyHf7di0xl8BLAhzMiE5u5eBWH0F8uGr0KYHm 92dhc6E6OYy36nDBrIrGztLH+v5njslOVd1EZYPgWXUiJVlL4IEdzMfbLWo0i8eKTxVEZJ fpklndtunzyQD+Mzu3vZLmlheKUsBH4= Received: from mail-wr1-f70.google.com (mail-wr1-f70.google.com [209.85.221.70]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-655-Rwexx-4UO8evmcYEWY56kQ-1; Tue, 17 Jun 2025 11:43:55 -0400 X-MC-Unique: Rwexx-4UO8evmcYEWY56kQ-1 X-Mimecast-MFC-AGG-ID: Rwexx-4UO8evmcYEWY56kQ_1750175034 Received: by mail-wr1-f70.google.com with SMTP id ffacd0b85a97d-3a50049f8eeso2412962f8f.3 for ; Tue, 17 Jun 2025 08:43:55 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1750175034; x=1750779834; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=VIojkghjiOfK3qS9mo4araH3tv/iIJHq9NK1Pu9RhVw=; b=kz26qZBEpGo1YW3YsonpqDcbWA7Q8Mpyk+mD9suwP/ZpCsXvKqUM/oLnNiODTPm6nu iiLkbB92kUzTUJM1qK97UtgUjN33UDUsg6TSWYvGUimZUlBbTJ6SkY66GKSa4DDEw8K3 61pWlIjjUBX2xjWcWp5Xpq7vzPa8/LZDNW6OcpuB1ucfuq3+ppp7C0YdE6zPi1QdjQeE k1B/eRpH4s+UM+FbiuFGhGxOJ1m+YbhqfZp6g1rbRiQlnJHyAkX3lGM+qN4LZiGlFC96 IIpmm0RGLk68HqdCdfL8QjJuNCwsdzArLMNUmOFRU557CWvFTW7K1wZ43qzawxtk91y6 GuLw== X-Gm-Message-State: AOJu0YwJLSFEZr2lNNWPnwnmQ3+S28SLlJ1A9R2OA8Ds8fhenIOj3mps 9Umgcp8JyT7pkA4Niv84T+f3agt0J5S6B6o6V0ka0i6gUwpZv46kwF4gG52t/lOafw3m9c+xku3 9Zs6gJN9hcO3a+kLLEDzt3I8HeOZWjbaiL5qXG0e7pnUEJW19dRk1ia0VLn6FODcCovL5qMrsiv 7pxq5uSIFjNa1gPq+g1AA1N4hvqRU6lPEMCdu1IN17xayewYDp X-Gm-Gg: ASbGncueNKD9WeeDVRymwrt/WD8xqsClliNnRDEWHx8ZfcAjzoFGk2ap2keeCtxmrVV OGJVtYQvGYBX9CDMXWFCtZDyo4ZayqGWrcmJ5nuXkEZ/6f30G5YxieIavtiCdK6r1Sdcut7KXx2 WKf/N854OzeQtqZrtOUQCX3+0diHyd4Zqzz4PYRuaWx/3Jc5cBt2XbelsXU2e3DEp4w78wuO+Q1 21oDP2oaueyhIhEywvGXUe9pw7pez2FMSn9l9K4el2XiqTOxbP7e3Hr7JFUJUaNkYBaEVIVz/uk iAg3wB2wUwei1LMdYHjSPRDKpOVcGLFtsi0Snq+MnDOtvY1uGovZh7BI3RW4k5UhAwH+choKQSA PNZiTYQ== X-Received: by 2002:a05:6000:178e:b0:3a4:c909:ce16 with SMTP id ffacd0b85a97d-3a572e92862mr10792713f8f.49.1750175034398; Tue, 17 Jun 2025 08:43:54 -0700 (PDT) X-Google-Smtp-Source: AGHT+IEuUUJS22/cv8lxbktZwrzLsxc7yFEH6CvMGkxQ5Sx5aQp7V5zrRMyeFF5jQJ+MyIsqVh3uOw== X-Received: by 2002:a05:6000:178e:b0:3a4:c909:ce16 with SMTP id ffacd0b85a97d-3a572e92862mr10792665f8f.49.1750175033907; Tue, 17 Jun 2025 08:43:53 -0700 (PDT) Received: from localhost (p200300d82f3107003851c66ab6b93490.dip0.t-ipconnect.de. [2003:d8:2f31:700:3851:c66a:b6b9:3490]) by smtp.gmail.com with UTF8SMTPSA id ffacd0b85a97d-3a568a543d9sm14118342f8f.5.2025.06.17.08.43.52 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Tue, 17 Jun 2025 08:43:53 -0700 (PDT) From: David Hildenbrand To: linux-kernel@vger.kernel.org Cc: linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, nvdimm@lists.linux.dev, David Hildenbrand , Andrew Morton , Juergen Gross , Stefano Stabellini , Oleksandr Tyshchenko , Dan Williams , Alistair Popple , Matthew Wilcox , Jan Kara , Alexander Viro , Christian Brauner , Zi Yan , Baolin Wang , Lorenzo Stoakes , "Liam R. Howlett" , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Jann Horn , Pedro Falcato Subject: [PATCH RFC 03/14] mm: compare pfns only if the entry is present when inserting pfns/pages Date: Tue, 17 Jun 2025 17:43:34 +0200 Message-ID: <20250617154345.2494405-4-david@redhat.com> X-Mailer: git-send-email 2.49.0 In-Reply-To: <20250617154345.2494405-1-david@redhat.com> References: <20250617154345.2494405-1-david@redhat.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Doing a pte_pfn() etc. of something that is not a present page table entry is wrong. Let's check in all relevant cases where we want to upgrade write permissions when inserting pfns/pages whether the entry is actually present. It's not expected to have caused real harm in practice, so this is more a cleanup than a fix for something that would likely trigger in some weird circumstances. At some point, we should likely unify the two pte handling paths, similar to how we did it for pmds/puds. Signed-off-by: David Hildenbrand Reviewed-by: Oscar Salvador Reviewed-by: Pedro Falcato --- mm/huge_memory.c | 4 ++-- mm/memory.c | 4 ++-- 2 files changed, 4 insertions(+), 4 deletions(-) diff --git a/mm/huge_memory.c b/mm/huge_memory.c index 8e0e3cfd9f223..e52360df87d15 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -1392,7 +1392,7 @@ static int insert_pmd(struct vm_area_struct *vma, uns= igned long addr, const unsigned long pfn =3D fop.is_folio ? folio_pfn(fop.folio) : fop.pfn; =20 - if (write) { + if (write && pmd_present(*pmd)) { if (pmd_pfn(*pmd) !=3D pfn) { WARN_ON_ONCE(!is_huge_zero_pmd(*pmd)); return -EEXIST; @@ -1541,7 +1541,7 @@ static void insert_pud(struct vm_area_struct *vma, un= signed long addr, const unsigned long pfn =3D fop.is_folio ? folio_pfn(fop.folio) : fop.pfn; =20 - if (write) { + if (write && pud_present(*pud)) { if (WARN_ON_ONCE(pud_pfn(*pud) !=3D pfn)) return; entry =3D pud_mkyoung(*pud); diff --git a/mm/memory.c b/mm/memory.c index a1b5575db52ac..9a1acd057ce59 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -2137,7 +2137,7 @@ static int insert_page_into_pte_locked(struct vm_area= _struct *vma, pte_t *pte, pte_t pteval =3D ptep_get(pte); =20 if (!pte_none(pteval)) { - if (!mkwrite) + if (!mkwrite || !pte_present(pteval)) return -EBUSY; =20 /* see insert_pfn(). */ @@ -2434,7 +2434,7 @@ static vm_fault_t insert_pfn(struct vm_area_struct *v= ma, unsigned long addr, return VM_FAULT_OOM; entry =3D ptep_get(pte); if (!pte_none(entry)) { - if (mkwrite) { + if (mkwrite && pte_present(entry)) { /* * For read faults on private mappings the PFN passed * in may not match the PFN we have mapped if the --=20 2.49.0 From nobody Thu Oct 9 13:18:00 2025 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0B71D2F2C46 for ; Tue, 17 Jun 2025 15:44:00 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1750175042; cv=none; b=cVDUP7hYOIxr80AtSzLWInRCCdj4ozgy1k8Orudr7yU6giAyqdF+GatSThWDrjWpJhIks9EayZFz2v4BevAmL4hsjVUeDeAdpoiHHbvIyujsFUHwSV562QII6E0rk7rJPvHH4AkRn0PGZbeN19o/wJfu3zFrLKqqleFLT/5EwsU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1750175042; c=relaxed/simple; bh=iM0Xdm4e3e5wygmWbk96k2XyIm1O0G4shrhlooh7yec=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=WumohrcTsK7FPRZB9dJHJgf1i/5rgh/zMAZ3PzX3CP5gXp228+7oB6I6EZFhv0O83OaoNrHtt9rjGkN5quWfSPPVPDkPDNGJlBmJv1qGcpQXxSVssYqGBqzCn0RZ3t6D3c3yf6POfcIaQQTW/UsPyxJihSCdFdzzKJphOTEUIDY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=Zcdmn5eN; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="Zcdmn5eN" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1750175040; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=5t+2oURhuR4fay2j7MSZreHj5RBEUrxIhgwh9uqVL44=; b=Zcdmn5eNXyUO57DWLH9o+eH/vggx9e39YSufqXEqw7b0UnpEv19qttJ5e9P/aEXMd6xL47 dkQkIRQftKWI8pHR3HSiEeiloeWHjyS5j+Dvbe1KzDy6V3O5tuaHPjF3k8x/hZSmqISG2a csVVNIvZjWVL+Cb4rCZSwh3+7Zybhr0= Received: from mail-wm1-f72.google.com (mail-wm1-f72.google.com [209.85.128.72]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-2-DkXR1qRjOY-9L5I5Wt0mQQ-1; Tue, 17 Jun 2025 11:43:58 -0400 X-MC-Unique: DkXR1qRjOY-9L5I5Wt0mQQ-1 X-Mimecast-MFC-AGG-ID: DkXR1qRjOY-9L5I5Wt0mQQ_1750175037 Received: by mail-wm1-f72.google.com with SMTP id 5b1f17b1804b1-4532ff43376so45620825e9.3 for ; Tue, 17 Jun 2025 08:43:58 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1750175037; x=1750779837; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=5t+2oURhuR4fay2j7MSZreHj5RBEUrxIhgwh9uqVL44=; b=bssWAquhvxA2gYjcShqyh5cP/FSpGNgk3PEToXQmqiEGEdr0hqrbxzMTvnjGBGW5/I LOyBOD41tZuimb0OIMLIOYQnZC59OUo32SJChNNHkxDdl1IX2hDWmXyrcvGs+VV3MFuE 6Iy2NQfS1qI4JPv1KwXgk+fPyrB3alqJsOkT98c7L/RdcSEaSBCqbOb7Qmn5xHecfPLj r3rDM/qgB7UTnUVGr3Powe29ABWbkNmjElHd65zWMUeMufGp8DHRLk7+57cAgfb7nL6X nFRbsV9MWjXIhEOBmVDtkjS2S/+UBgSfyPr7XoZmQETUltypwYjyzwy0+rGm9hrDHm+K tc5Q== X-Gm-Message-State: AOJu0Yy23HHgU2XriNfTwHca7otNGZxHEuQYjmfXwY6/332MLBeLbjRA O5kA1fdB9AXkCHeuEdsTDSXFTSo2Ep9rCU0WZNSO6vIxUEajW+wyjji3TieI045naDkkUxQKM3B 2ZQhFcI3ph8akHsn/nGj1vfM/ZknWf5I41p5a11OIY5cn4CrN9kMFxxq0SxspV9uGKgFcmoyZkk EvEOLdco9i/z990QRUSI1tsr4WpAkeVLgWD97/VOgdVuqf7DDz X-Gm-Gg: ASbGncswkwlOxid5Yi0mtvzp3ovC0DfHVe9TR9ZcvucOIdj0PHZmb98UZqXkdwomGfj 250jGRyjKH3JUPAK+QDZqqDCik9x/q35vypSO7p3uCwZ3NldxGBNDx5OrC9plARSx8k7kDPznt3 ei/tf7nKQnNmloGnxfea4zHwrndBY9Sz8Yo0UVdSv9rVxttHQRGTfyCDidpKNSFS3h29544yRFP +1eddnylCmPcNRur+xbp6fAL8LXptoOOjtWnovJvOEy1DNSKZQvVEkXX1vq058Bf8jqyrnXM5tQ IbU3a7+hBi9N7yPK9nmtRJbESBT3TnuXAl5kfbUpDFxp4PO6wQ== X-Received: by 2002:a05:600c:8509:b0:441:b19c:96fe with SMTP id 5b1f17b1804b1-4533caa3d54mr172176645e9.10.1750175037130; Tue, 17 Jun 2025 08:43:57 -0700 (PDT) X-Google-Smtp-Source: AGHT+IEKe6joB8XUUUHr9L6o0Yf1elMxRlblx8Zso9tX4QaAEAaXQGULg7L379TE9a6MvFpj5CfDTQ== X-Received: by 2002:a05:600c:8509:b0:441:b19c:96fe with SMTP id 5b1f17b1804b1-4533caa3d54mr172175925e9.10.1750175036588; Tue, 17 Jun 2025 08:43:56 -0700 (PDT) Received: from localhost (p57a1a266.dip0.t-ipconnect.de. [87.161.162.102]) by smtp.gmail.com with UTF8SMTPSA id 5b1f17b1804b1-4534226aa44sm108387495e9.13.2025.06.17.08.43.55 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Tue, 17 Jun 2025 08:43:56 -0700 (PDT) From: David Hildenbrand To: linux-kernel@vger.kernel.org Cc: linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, nvdimm@lists.linux.dev, David Hildenbrand , Andrew Morton , Juergen Gross , Stefano Stabellini , Oleksandr Tyshchenko , Dan Williams , Alistair Popple , Matthew Wilcox , Jan Kara , Alexander Viro , Christian Brauner , Zi Yan , Baolin Wang , Lorenzo Stoakes , "Liam R. Howlett" , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Jann Horn , Pedro Falcato Subject: [PATCH RFC 04/14] mm/huge_memory: move more common code into insert_pmd() Date: Tue, 17 Jun 2025 17:43:35 +0200 Message-ID: <20250617154345.2494405-5-david@redhat.com> X-Mailer: git-send-email 2.49.0 In-Reply-To: <20250617154345.2494405-1-david@redhat.com> References: <20250617154345.2494405-1-david@redhat.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Let's clean it all further up. Signed-off-by: David Hildenbrand Reviewed-by: Alistair Popple Reviewed-by: Oscar Salvador --- mm/huge_memory.c | 72 ++++++++++++++++-------------------------------- 1 file changed, 24 insertions(+), 48 deletions(-) diff --git a/mm/huge_memory.c b/mm/huge_memory.c index e52360df87d15..a85e0cd455109 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -1379,15 +1379,25 @@ struct folio_or_pfn { bool is_folio; }; =20 -static int insert_pmd(struct vm_area_struct *vma, unsigned long addr, +static vm_fault_t insert_pmd(struct vm_area_struct *vma, unsigned long add= r, pmd_t *pmd, struct folio_or_pfn fop, pgprot_t prot, - bool write, pgtable_t pgtable) + bool write) { struct mm_struct *mm =3D vma->vm_mm; + pgtable_t pgtable =3D NULL; + spinlock_t *ptl; pmd_t entry; =20 - lockdep_assert_held(pmd_lockptr(mm, pmd)); + if (addr < vma->vm_start || addr >=3D vma->vm_end) + return VM_FAULT_SIGBUS; =20 + if (arch_needs_pgtable_deposit()) { + pgtable =3D pte_alloc_one(vma->vm_mm); + if (!pgtable) + return VM_FAULT_OOM; + } + + ptl =3D pmd_lock(mm, pmd); if (!pmd_none(*pmd)) { const unsigned long pfn =3D fop.is_folio ? folio_pfn(fop.folio) : fop.pfn; @@ -1395,15 +1405,14 @@ static int insert_pmd(struct vm_area_struct *vma, u= nsigned long addr, if (write && pmd_present(*pmd)) { if (pmd_pfn(*pmd) !=3D pfn) { WARN_ON_ONCE(!is_huge_zero_pmd(*pmd)); - return -EEXIST; + goto out_unlock; } entry =3D pmd_mkyoung(*pmd); entry =3D maybe_pmd_mkwrite(pmd_mkdirty(entry), vma); if (pmdp_set_access_flags(vma, addr, pmd, entry, 1)) update_mmu_cache_pmd(vma, addr, pmd); } - - return -EEXIST; + goto out_unlock; } =20 if (fop.is_folio) { @@ -1424,11 +1433,17 @@ static int insert_pmd(struct vm_area_struct *vma, u= nsigned long addr, if (pgtable) { pgtable_trans_huge_deposit(mm, pmd, pgtable); mm_inc_nr_ptes(mm); + pgtable =3D NULL; } =20 set_pmd_at(mm, addr, pmd, entry); update_mmu_cache_pmd(vma, addr, pmd); - return 0; + +out_unlock: + spin_unlock(ptl); + if (pgtable) + pte_free(mm, pgtable); + return VM_FAULT_NOPAGE; } =20 /** @@ -1450,9 +1465,6 @@ vm_fault_t vmf_insert_pfn_pmd(struct vm_fault *vmf, u= nsigned long pfn, struct folio_or_pfn fop =3D { .pfn =3D pfn, }; - pgtable_t pgtable =3D NULL; - spinlock_t *ptl; - int error; =20 /* * If we had pmd_special, we could avoid all these restrictions, @@ -1464,25 +1476,9 @@ vm_fault_t vmf_insert_pfn_pmd(struct vm_fault *vmf, = unsigned long pfn, (VM_PFNMAP|VM_MIXEDMAP)); BUG_ON((vma->vm_flags & VM_PFNMAP) && is_cow_mapping(vma->vm_flags)); =20 - if (addr < vma->vm_start || addr >=3D vma->vm_end) - return VM_FAULT_SIGBUS; - - if (arch_needs_pgtable_deposit()) { - pgtable =3D pte_alloc_one(vma->vm_mm); - if (!pgtable) - return VM_FAULT_OOM; - } - pfnmap_setup_cachemode_pfn(pfn, &pgprot); =20 - ptl =3D pmd_lock(vma->vm_mm, vmf->pmd); - error =3D insert_pmd(vma, addr, vmf->pmd, fop, pgprot, write, - pgtable); - spin_unlock(ptl); - if (error && pgtable) - pte_free(vma->vm_mm, pgtable); - - return VM_FAULT_NOPAGE; + return insert_pmd(vma, addr, vmf->pmd, fop, pgprot, write); } EXPORT_SYMBOL_GPL(vmf_insert_pfn_pmd); =20 @@ -1491,35 +1487,15 @@ vm_fault_t vmf_insert_folio_pmd(struct vm_fault *vm= f, struct folio *folio, { struct vm_area_struct *vma =3D vmf->vma; unsigned long addr =3D vmf->address & PMD_MASK; - struct mm_struct *mm =3D vma->vm_mm; struct folio_or_pfn fop =3D { .folio =3D folio, .is_folio =3D true, }; - spinlock_t *ptl; - pgtable_t pgtable =3D NULL; - int error; - - if (addr < vma->vm_start || addr >=3D vma->vm_end) - return VM_FAULT_SIGBUS; =20 if (WARN_ON_ONCE(folio_order(folio) !=3D PMD_ORDER)) return VM_FAULT_SIGBUS; =20 - if (arch_needs_pgtable_deposit()) { - pgtable =3D pte_alloc_one(vma->vm_mm); - if (!pgtable) - return VM_FAULT_OOM; - } - - ptl =3D pmd_lock(mm, vmf->pmd); - error =3D insert_pmd(vma, addr, vmf->pmd, fop, vma->vm_page_prot, - write, pgtable); - spin_unlock(ptl); - if (error && pgtable) - pte_free(mm, pgtable); - - return VM_FAULT_NOPAGE; + return insert_pmd(vma, addr, vmf->pmd, fop, vma->vm_page_prot, write); } EXPORT_SYMBOL_GPL(vmf_insert_folio_pmd); =20 --=20 2.49.0 From nobody Thu Oct 9 13:18:00 2025 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7BF112F362C for ; Tue, 17 Jun 2025 15:44:03 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1750175045; cv=none; b=Kqjt/3zmtK+dBJcn6PlW+jmC0rV+1Y5q1wL2V4EvNORKsCuAyo7YaFjwq91ZnMxvdrh2H8V5vlWy/u5kO7GK1E8TqMGqPV/BroU4e+cubSGZhlQpKThog7m/2EhmojkF2QXwKLs2siamNk3+heu+BX9TqLMDTlTwE2/1gXZ2xCM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1750175045; c=relaxed/simple; bh=Lmnd5ZdzMmI5UAgfdf1SJgbeKDVj30wC0SI7xpkFiMk=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=JDTX3jK/yAFynCcCdp4/pdp6XbetqiGxhdM6c/Kc4MhhS+8QMRdbv/F2CmNGRiMblAHHxCwfRspTn/nO5nr2iElpe9UIcRkhPACEtp6dnUgJVgrE7umyDG1kvoz8JJ5xq7FdHOA1oXTaD1LOj68P+ta3X+RA9PYyDIrkcR3PrFE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=J3vsOxw4; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="J3vsOxw4" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1750175042; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=5L8RMUDCiRiYUMLt7aYvOaTdy74+EjgVDuyzYaNy+k8=; b=J3vsOxw4T6/7W98lteezdAZpJffa86VAE9b+YbyloGvxcK46PzZ4hDhQHMxr1SZ34IdLO5 JcGSdhWfUKIs/TXtoLmq1MFiOjAjPhuqTiqv9EEFRwiu4TGUJxu9J3t78+FZD+Xa/9Hy+C unkN3XWQFfWabztB3L0OH+5EZhq9JSU= Received: from mail-wm1-f70.google.com (mail-wm1-f70.google.com [209.85.128.70]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-37-BUtSATPSOcOwPt-u2o-YRA-1; Tue, 17 Jun 2025 11:44:01 -0400 X-MC-Unique: BUtSATPSOcOwPt-u2o-YRA-1 X-Mimecast-MFC-AGG-ID: BUtSATPSOcOwPt-u2o-YRA_1750175040 Received: by mail-wm1-f70.google.com with SMTP id 5b1f17b1804b1-451d2037f1eso36347315e9.0 for ; Tue, 17 Jun 2025 08:44:01 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1750175040; x=1750779840; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=5L8RMUDCiRiYUMLt7aYvOaTdy74+EjgVDuyzYaNy+k8=; b=JXEqw2k+WU257YbDzxyWss8bvJ1HrVrjx0U9I+354K5bOra3sSykTDj6C7XdK35oWs PgTaswyYL5wP0n3KL/qaPedn54Oy+nsdDzCLIutfNGS+9j3q13jXKU5QvKlLTnKUgBTv IvvlRDZ/iqLvygWM4oPAvokLlGESsPrQJ+kla4KTdvL3rITU67Poa6l0bYUR7ssCAHwh ltb/hc9AamTNI0mqWuaZc4xYyQQr0SZspaTwOhgzRgAyznMg5P8nCUO3ftCHdJ7eZFbZ yzqr4LpP9qIEiYobnhPJZilj1OQtrgIDNuJhQJ30qBdLZ0t0H10Vy/bfjjFtQDnDbgjt 5gZw== X-Gm-Message-State: AOJu0YxrFml+JlHzts4xt0l7QujJn4TEwDfozOy3CVVrh5vITwYA5szB /bGsYMYk/FasCUcYIEjKXeJl8emY0OqRrLoq1ZxuWWQn11KQ1E0POj4D9io9TZNFSbdtueEVVdK 3eHOzDjqewNg8C0S1izcgX5fa/KGhmxweLYkOa/hSaBlVu5sha5I0cBrhfMEvGppLGOT6L7U39U 4D9cepKN5pZkjLgw1Gm0zatBuFS5L4GhxwmU4I9TRNXpReq/n6 X-Gm-Gg: ASbGnctB4swKkAaIxbjsUQUIsSnbFO6c/3OfauufaxrQCKQ6hFA+uSeywz6Vy0PsbLo 3tTc9ZR4zi9y5soqO7EVadwfZHg4gEROnA4FbXaAFadGmypBbbuz71TNUU4ufNMfkavdwfyQRVN rY7NqADwNc4Z332BdqE5xaRXKmfSgUIOgk1nFmu4n32UjN97MBksrXmFXbIFfY3IN6km6savGwy PKt+L3frvYJitOLo4rgEeBegoQNQo5iNXh/b4jOUBQgokbMHy9a5Ef1JKiLJt4dGtlUpKTkpZ4U 8S7zOGPsmPrrkfoSf4AP3Bz1D/s+juu9Pe0f2Yu6nZh8I/RCOGeUZT7/FNLC//jo0aQjfV19eCl cFpEoUw== X-Received: by 2002:a05:600c:8710:b0:44a:775d:b5e8 with SMTP id 5b1f17b1804b1-4533cadf840mr119660395e9.1.1750175039997; Tue, 17 Jun 2025 08:43:59 -0700 (PDT) X-Google-Smtp-Source: AGHT+IG6BFmn+0lN3d+8pZJ+Q6QIbSjyAC/18B9P+xMU7OTx196gniDr43GvtKhKD3/WRsO3OGi0Uw== X-Received: by 2002:a05:600c:8710:b0:44a:775d:b5e8 with SMTP id 5b1f17b1804b1-4533cadf840mr119659825e9.1.1750175039526; Tue, 17 Jun 2025 08:43:59 -0700 (PDT) Received: from localhost (p200300d82f3107003851c66ab6b93490.dip0.t-ipconnect.de. [2003:d8:2f31:700:3851:c66a:b6b9:3490]) by smtp.gmail.com with UTF8SMTPSA id 5b1f17b1804b1-4532e25f207sm178703705e9.35.2025.06.17.08.43.57 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Tue, 17 Jun 2025 08:43:59 -0700 (PDT) From: David Hildenbrand To: linux-kernel@vger.kernel.org Cc: linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, nvdimm@lists.linux.dev, David Hildenbrand , Andrew Morton , Juergen Gross , Stefano Stabellini , Oleksandr Tyshchenko , Dan Williams , Alistair Popple , Matthew Wilcox , Jan Kara , Alexander Viro , Christian Brauner , Zi Yan , Baolin Wang , Lorenzo Stoakes , "Liam R. Howlett" , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Jann Horn , Pedro Falcato Subject: [PATCH RFC 05/14] mm/huge_memory: move more common code into insert_pud() Date: Tue, 17 Jun 2025 17:43:36 +0200 Message-ID: <20250617154345.2494405-6-david@redhat.com> X-Mailer: git-send-email 2.49.0 In-Reply-To: <20250617154345.2494405-1-david@redhat.com> References: <20250617154345.2494405-1-david@redhat.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Let's clean it all further up. Signed-off-by: David Hildenbrand Reviewed-by: Alistair Popple Reviewed-by: Oscar Salvador --- mm/huge_memory.c | 36 +++++++++++++----------------------- 1 file changed, 13 insertions(+), 23 deletions(-) diff --git a/mm/huge_memory.c b/mm/huge_memory.c index a85e0cd455109..1ea23900b5adb 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -1507,25 +1507,30 @@ static pud_t maybe_pud_mkwrite(pud_t pud, struct vm= _area_struct *vma) return pud; } =20 -static void insert_pud(struct vm_area_struct *vma, unsigned long addr, +static vm_fault_t insert_pud(struct vm_area_struct *vma, unsigned long add= r, pud_t *pud, struct folio_or_pfn fop, pgprot_t prot, bool write) { struct mm_struct *mm =3D vma->vm_mm; + spinlock_t *ptl; pud_t entry; =20 + if (addr < vma->vm_start || addr >=3D vma->vm_end) + return VM_FAULT_SIGBUS; + + ptl =3D pud_lock(mm, pud); if (!pud_none(*pud)) { const unsigned long pfn =3D fop.is_folio ? folio_pfn(fop.folio) : fop.pfn; =20 if (write && pud_present(*pud)) { if (WARN_ON_ONCE(pud_pfn(*pud) !=3D pfn)) - return; + goto out_unlock; entry =3D pud_mkyoung(*pud); entry =3D maybe_pud_mkwrite(pud_mkdirty(entry), vma); if (pudp_set_access_flags(vma, addr, pud, entry, 1)) update_mmu_cache_pud(vma, addr, pud); } - return; + goto out_unlock; } =20 if (fop.is_folio) { @@ -1544,6 +1549,9 @@ static void insert_pud(struct vm_area_struct *vma, un= signed long addr, } set_pud_at(mm, addr, pud, entry); update_mmu_cache_pud(vma, addr, pud); +out_unlock: + spin_unlock(ptl); + return VM_FAULT_NOPAGE; } =20 /** @@ -1565,7 +1573,6 @@ vm_fault_t vmf_insert_pfn_pud(struct vm_fault *vmf, u= nsigned long pfn, struct folio_or_pfn fop =3D { .pfn =3D pfn, }; - spinlock_t *ptl; =20 /* * If we had pud_special, we could avoid all these restrictions, @@ -1577,16 +1584,9 @@ vm_fault_t vmf_insert_pfn_pud(struct vm_fault *vmf, = unsigned long pfn, (VM_PFNMAP|VM_MIXEDMAP)); BUG_ON((vma->vm_flags & VM_PFNMAP) && is_cow_mapping(vma->vm_flags)); =20 - if (addr < vma->vm_start || addr >=3D vma->vm_end) - return VM_FAULT_SIGBUS; - pfnmap_setup_cachemode_pfn(pfn, &pgprot); =20 - ptl =3D pud_lock(vma->vm_mm, vmf->pud); - insert_pud(vma, addr, vmf->pud, fop, pgprot, write); - spin_unlock(ptl); - - return VM_FAULT_NOPAGE; + return insert_pud(vma, addr, vmf->pud, fop, pgprot, write); } EXPORT_SYMBOL_GPL(vmf_insert_pfn_pud); =20 @@ -1603,25 +1603,15 @@ vm_fault_t vmf_insert_folio_pud(struct vm_fault *vm= f, struct folio *folio, { struct vm_area_struct *vma =3D vmf->vma; unsigned long addr =3D vmf->address & PUD_MASK; - pud_t *pud =3D vmf->pud; - struct mm_struct *mm =3D vma->vm_mm; struct folio_or_pfn fop =3D { .folio =3D folio, .is_folio =3D true, }; - spinlock_t *ptl; - - if (addr < vma->vm_start || addr >=3D vma->vm_end) - return VM_FAULT_SIGBUS; =20 if (WARN_ON_ONCE(folio_order(folio) !=3D PUD_ORDER)) return VM_FAULT_SIGBUS; =20 - ptl =3D pud_lock(mm, pud); - insert_pud(vma, addr, vmf->pud, fop, vma->vm_page_prot, write); - spin_unlock(ptl); - - return VM_FAULT_NOPAGE; + return insert_pud(vma, addr, vmf->pud, fop, vma->vm_page_prot, write); } EXPORT_SYMBOL_GPL(vmf_insert_folio_pud); #endif /* CONFIG_HAVE_ARCH_TRANSPARENT_HUGEPAGE_PUD */ --=20 2.49.0 From nobody Thu Oct 9 13:18:00 2025 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D44A82F4309 for ; Tue, 17 Jun 2025 15:44:06 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1750175048; cv=none; b=IR6lp6+E03/IBsc5rgxPFCXTZh8RbJzMEEjQW0TL1smngr53KLIqgk3Tsl4eH2If1zkDMiqROPjQ7k+Tyw6mPn9wlB3/RbjvTqdT8pgputOLiAGZUNNth94M4lxSMPjMfj0VH+6DiZhKE1JC87QPqKl/jX7yoWisTzkPMVd+d4U= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1750175048; c=relaxed/simple; bh=AaLRVPcfUejWvJOHKekM/3Bd9o1DBMGCz5AGg2IT7Eo=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=Y3NluLQONVYN7mDK/oXBdPQnvhBK31Q328n8f15lPcedboVZgJEFyGrMCHnLHFacPlg4VGbwl77vIhuXVQqDg9ZSGEqIrDQ1B2HQgCPv5Y8IoZVA937MxiqDZXPz6xMHaHiudbpeLlXHdK5o4AnwHhAi1Z3Ix/W8EAHm0UGcMhI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=ghhlphs1; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="ghhlphs1" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1750175046; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=IZKss+ViQ3XsGRfMnfUZIAkjKJYTTy4h8nMBjUoSR6g=; b=ghhlphs1tkT2NyItL67FD25ANVqM5Fw3POoV8UI83m1DgqE1z5gZLQvozrgQJ3UyEXnXUp 630d5zW2Crtt76tXEtmfcLT7JCuTffnZX1cBaTCbmaW4YGiKJs3agj6z7U3/nwRKLd4Uyu /vLtKqjXIZ7u3m9A8odZHomp7Ar1zGU= Received: from mail-wm1-f72.google.com (mail-wm1-f72.google.com [209.85.128.72]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-224-SVEpRpw5Nbeppqa2RRkpjg-1; Tue, 17 Jun 2025 11:44:03 -0400 X-MC-Unique: SVEpRpw5Nbeppqa2RRkpjg-1 X-Mimecast-MFC-AGG-ID: SVEpRpw5Nbeppqa2RRkpjg_1750175042 Received: by mail-wm1-f72.google.com with SMTP id 5b1f17b1804b1-43eed325461so37322285e9.3 for ; Tue, 17 Jun 2025 08:44:03 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1750175042; x=1750779842; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=IZKss+ViQ3XsGRfMnfUZIAkjKJYTTy4h8nMBjUoSR6g=; b=p8dj3py2/G70K6BwCdF1uMWssPe+WKSCKcHxEGaq+tPOQ9/QdEi98f56tqaLSL2oFh AC1ni8o4+U8Wu/3Yu4ue64AmFf8uHdOuEfyF9zr/0mO4CKhzgFdSdorgPgJZh8LGp5H9 UtsfEnr3JgRdJ9zM24pPmHglK7rc9JXk+oPnkUmCnLQXCqn6YA1b7fKrMSARuGKzUwbo McUqAX74I9RhgO7kNpD3lhPlvaMrXgcpqP2E4iaD3EQINGf9xwkOHrrBN21cBG2sBFxK CfekBYghF3D+L/aj4aTxkvQYlYxxZVPrS8bJC/9k0rm4SGqFhkzXRlJiWf6ZTBnoBkFn 6e9w== X-Gm-Message-State: AOJu0YxvAqpBlPibhqKE+i1wT3DZsxazVEoTeuYnmpuZ7Yv76GdWUBYN fb5va+YmebdKGFZGWrGHUmtvGCGQGo80Jv5rWGLFGkyBN0ZdO0Ojj0UVpvUoDXs53XhEYPgO2t3 Lg0cFrwoi4OV4qgs9bobdt307ZSyWYfXt2A50YDEDe5e901rU2JbmfiEcHBlgOOsvIV3DZmAVj5 kWbvaaExSuBWh0dPS0cGSZUcaw9bnmC7bQeXM6m3+NNu4D/wjH X-Gm-Gg: ASbGncsbaK600qdKkPMdnPNw7634WZkaKfQlkQ4ZsntINlYNWLfhm7ktj+N/daa2UZt f7q5ZZyxTxiAAi9ZhudUq3Fki8jyJwbg596zRAJ6KoyWGRtePOUCuJQrpxeyoisTVF9xrYOAcR4 NYb7lY3CRJ858M22HugYK4JUX+Jp/sn/fZIf+llXUQrQi+mhHQCO/BgDiHTxWYXpCY4XcK2cSgZ gR0UNZ3AvewDzKBoSJKuGVircDJDmZa06SeFFpwAG0seiYp5PKuni0MtiA/pw1ov42lUkFYeZwI +KmpUyt9Geo2urSVA6Osm+MCcuQoe3n1C3LorbVMiDPPYs6pYY94b5kG9KJSssRFQBpup0klYZW fGZX8Zg== X-Received: by 2002:a05:6000:40ca:b0:3a4:eb80:762d with SMTP id ffacd0b85a97d-3a572e9e7bfmr10544499f8f.56.1750175042219; Tue, 17 Jun 2025 08:44:02 -0700 (PDT) X-Google-Smtp-Source: AGHT+IEizYNw7dc5qMU4ZYxHKAQ+J5Vp2J4wukFj23PAnuBxSQs+o+0P3zEnBC8W1ti7paRppqlPmw== X-Received: by 2002:a05:6000:40ca:b0:3a4:eb80:762d with SMTP id ffacd0b85a97d-3a572e9e7bfmr10544452f8f.56.1750175041775; Tue, 17 Jun 2025 08:44:01 -0700 (PDT) Received: from localhost (p200300d82f3107003851c66ab6b93490.dip0.t-ipconnect.de. [2003:d8:2f31:700:3851:c66a:b6b9:3490]) by smtp.gmail.com with UTF8SMTPSA id 5b1f17b1804b1-4532e25ec9fsm175719105e9.34.2025.06.17.08.44.00 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Tue, 17 Jun 2025 08:44:01 -0700 (PDT) From: David Hildenbrand To: linux-kernel@vger.kernel.org Cc: linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, nvdimm@lists.linux.dev, David Hildenbrand , Andrew Morton , Juergen Gross , Stefano Stabellini , Oleksandr Tyshchenko , Dan Williams , Alistair Popple , Matthew Wilcox , Jan Kara , Alexander Viro , Christian Brauner , Zi Yan , Baolin Wang , Lorenzo Stoakes , "Liam R. Howlett" , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Jann Horn , Pedro Falcato Subject: [PATCH RFC 06/14] mm/huge_memory: support huge zero folio in vmf_insert_folio_pmd() Date: Tue, 17 Jun 2025 17:43:37 +0200 Message-ID: <20250617154345.2494405-7-david@redhat.com> X-Mailer: git-send-email 2.49.0 In-Reply-To: <20250617154345.2494405-1-david@redhat.com> References: <20250617154345.2494405-1-david@redhat.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Just like we do for vmf_insert_page_mkwrite() -> ... -> insert_page_into_pte_locked(), support the huge zero folio. Signed-off-by: David Hildenbrand Reviewed-by: Oscar Salvador --- mm/huge_memory.c | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/mm/huge_memory.c b/mm/huge_memory.c index 1ea23900b5adb..92400f3baa9ff 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -1418,9 +1418,11 @@ static vm_fault_t insert_pmd(struct vm_area_struct *= vma, unsigned long addr, if (fop.is_folio) { entry =3D folio_mk_pmd(fop.folio, vma->vm_page_prot); =20 - folio_get(fop.folio); - folio_add_file_rmap_pmd(fop.folio, &fop.folio->page, vma); - add_mm_counter(mm, mm_counter_file(fop.folio), HPAGE_PMD_NR); + if (!is_huge_zero_folio(fop.folio)) { + folio_get(fop.folio); + folio_add_file_rmap_pmd(fop.folio, &fop.folio->page, vma); + add_mm_counter(mm, mm_counter_file(fop.folio), HPAGE_PMD_NR); + } } else { entry =3D pmd_mkhuge(pfn_pmd(fop.pfn, prot)); entry =3D pmd_mkspecial(entry); --=20 2.49.0 From nobody Thu Oct 9 13:18:00 2025 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E9E932F5467 for ; Tue, 17 Jun 2025 15:44:07 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1750175049; cv=none; b=hQeCJSzej7V/28tPpg4/5ouxdftr/P4bOxbO1Ex9wLB8dRk5Iwqh6oTCVdp/ebLI8iMP3f5AiE79+8NQZ3JqEPGm79p8FsroaOaT35iC2InQDms5ZXDbrULxJZwXuEgmWHjDLKh5paTy2BTeh2qn38oJGvEycjx31aIetdiKRHA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1750175049; c=relaxed/simple; bh=3FCxZ8zDcceG27Pi1n4ipPgji5pZnfBRAxjPk2GRm/A=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=W5MwmX8+RLxTo/4JWMBw3SSMzwwXwK0ASfUpcR9zUJGfHWwwMOxjIaFZ0uCJ0E/twO+F5IwFPaHSOIYYwTSabD1Jn4bLmVV/l5YN3Yn4g/2B33CEVvGg9qChE8/pT+zZNGCz8NHELOcdObVBXs/rQh/7NxrcMGacLM+EgEIjCiA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=IGyuA4lb; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="IGyuA4lb" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1750175047; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=gs4285BVc9AdSpQGnU9ous2Jq7UzfwNWKMh/gkXW9pc=; b=IGyuA4lb6EyOZvI83QJ6eqBP7UXjqYJMk+pK+JTY1B4MbJAPXBhhgAeYMXf+XQe4K8ACkM 7dhZ5zw7HhVr3aYwn/IJzGJonIFRw19dVfkUN4v/bcXHNc087kuaIs5CD2O8gglPclJfBN 65c/njgF3jsBXVbsV2aAdlSX16jFaqo= Received: from mail-wr1-f69.google.com (mail-wr1-f69.google.com [209.85.221.69]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-518-Lt9pj-g1PSyPASj2ycDhvg-1; Tue, 17 Jun 2025 11:44:06 -0400 X-MC-Unique: Lt9pj-g1PSyPASj2ycDhvg-1 X-Mimecast-MFC-AGG-ID: Lt9pj-g1PSyPASj2ycDhvg_1750175045 Received: by mail-wr1-f69.google.com with SMTP id ffacd0b85a97d-3a37a0d1005so3556376f8f.3 for ; Tue, 17 Jun 2025 08:44:05 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1750175044; x=1750779844; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=gs4285BVc9AdSpQGnU9ous2Jq7UzfwNWKMh/gkXW9pc=; b=avVEaeOTIdvcLuEDWgpjBY/6wggdOJ4/TdSugQ2B/YRslQKgWDsr/jVdPAkV4ucdyI 0FNW0PE41d28ofsfpmK+R3iF2FqcctC+Y4zpb8tqyVdhW5dJ+nnOFgUzAn9wwP5brWS4 tEwNndVGxHbu2HXiELDuWnKaqUhXMHT6HYEf5GpYtPZWpee0Hd1xN/RlWMSV8RWbw5Mm VhNYPlBbJ7NsSSDIT7C/xmP949cEavfKwUBfVizNB+m3M5CF+hKaiY7jjJ0sESEAyAiC 4YUSkaVHnx/FqqNe84uN7zh/MGzfIaxAaqmW+wniaGscdAJYGt8rZNLpT0knrFNs/Zjz qbPQ== X-Gm-Message-State: AOJu0YyIl5Ut/2qC5q/CQc97yJMKQsxLNuz38dS3HUf2gF1lHNy7Zh/i x3BGwUGbRj3GZC0QuXZ71WNZek5blWn0f1HUpBOn1p7cCZ0QnhfTikisA9YRsyV8ZzZd/p0qqJU TfNR0n/Tms1jU7g9/dPcccoVn7lKPaxAeSzNkDP/Xb9W3uGuMzw+ufBkcwMq7iWU1rRNsutjc+E imVarp9wlfuWlGifBhtA5HXXiJWEroEjchhB+IwhWFViJTldS0 X-Gm-Gg: ASbGnctZcfP4r+8ejMa/EMdEsU9RyWugqv05/xDJT/MQMnLrd82NIA3bNljOx1OEaUO wzTUbVss6Mp+/a4aad9a3yn/hcZ+QItXc/gwC2YKqlAu7M/GvoTm8n+JJ5Trx2lIN52v6CO4cM4 WMBrqwxN7pq6nqGpkoxi1ZWNvD0r716mKBna0UvXAlLS12XXmQhLI5qa/EKutW5Xauahrb25Pp9 K209r44o8NY3wLjrFpsshIF9+RxToeHsYJ6pEM87N81jg7LZjjLlE6vh4TRi5G3qOVdQiJct03I JESJs0fj6P0in3Kch7/v9NGQlM+br2JmClXe5miC3xFiLr7eT3l1SIroszznrIOBHSHnM1fV6Fl dklFTgg== X-Received: by 2002:a05:6000:1acb:b0:3a4:f038:af76 with SMTP id ffacd0b85a97d-3a572e886ddmr11868390f8f.53.1750175044624; Tue, 17 Jun 2025 08:44:04 -0700 (PDT) X-Google-Smtp-Source: AGHT+IFo66SwAdCOqzoQeskB1+JNa+FajMdh1o2TiDbiY6YeDDtEJko8k8Xcyqba8MVeHhWH2HTahA== X-Received: by 2002:a05:6000:1acb:b0:3a4:f038:af76 with SMTP id ffacd0b85a97d-3a572e886ddmr11868335f8f.53.1750175044129; Tue, 17 Jun 2025 08:44:04 -0700 (PDT) Received: from localhost (p200300d82f3107003851c66ab6b93490.dip0.t-ipconnect.de. [2003:d8:2f31:700:3851:c66a:b6b9:3490]) by smtp.gmail.com with UTF8SMTPSA id ffacd0b85a97d-3a568b28240sm14420724f8f.72.2025.06.17.08.44.02 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Tue, 17 Jun 2025 08:44:03 -0700 (PDT) From: David Hildenbrand To: linux-kernel@vger.kernel.org Cc: linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, nvdimm@lists.linux.dev, David Hildenbrand , Andrew Morton , Juergen Gross , Stefano Stabellini , Oleksandr Tyshchenko , Dan Williams , Alistair Popple , Matthew Wilcox , Jan Kara , Alexander Viro , Christian Brauner , Zi Yan , Baolin Wang , Lorenzo Stoakes , "Liam R. Howlett" , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Jann Horn , Pedro Falcato Subject: [PATCH RFC 07/14] fs/dax: use vmf_insert_folio_pmd() to insert the huge zero folio Date: Tue, 17 Jun 2025 17:43:38 +0200 Message-ID: <20250617154345.2494405-8-david@redhat.com> X-Mailer: git-send-email 2.49.0 In-Reply-To: <20250617154345.2494405-1-david@redhat.com> References: <20250617154345.2494405-1-david@redhat.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Let's convert to vmf_insert_folio_pmd(). In the unlikely case there is already something mapped, we'll now still call trace_dax_pmd_load_hole() and return VM_FAULT_NOPAGE. That should probably be fine, no need to add special cases for that. Signed-off-by: David Hildenbrand --- fs/dax.c | 47 ++++++++++------------------------------------- 1 file changed, 10 insertions(+), 37 deletions(-) diff --git a/fs/dax.c b/fs/dax.c index 4229513806bea..ae90706674a3f 100644 --- a/fs/dax.c +++ b/fs/dax.c @@ -1375,51 +1375,24 @@ static vm_fault_t dax_pmd_load_hole(struct xa_state= *xas, struct vm_fault *vmf, const struct iomap_iter *iter, void **entry) { struct address_space *mapping =3D vmf->vma->vm_file->f_mapping; - unsigned long pmd_addr =3D vmf->address & PMD_MASK; - struct vm_area_struct *vma =3D vmf->vma; struct inode *inode =3D mapping->host; - pgtable_t pgtable =3D NULL; struct folio *zero_folio; - spinlock_t *ptl; - pmd_t pmd_entry; - unsigned long pfn; + vm_fault_t ret; =20 zero_folio =3D mm_get_huge_zero_folio(vmf->vma->vm_mm); =20 - if (unlikely(!zero_folio)) - goto fallback; - - pfn =3D page_to_pfn(&zero_folio->page); - *entry =3D dax_insert_entry(xas, vmf, iter, *entry, pfn, - DAX_PMD | DAX_ZERO_PAGE); - - if (arch_needs_pgtable_deposit()) { - pgtable =3D pte_alloc_one(vma->vm_mm); - if (!pgtable) - return VM_FAULT_OOM; - } - - ptl =3D pmd_lock(vmf->vma->vm_mm, vmf->pmd); - if (!pmd_none(*(vmf->pmd))) { - spin_unlock(ptl); - goto fallback; + if (unlikely(!zero_folio)) { + trace_dax_pmd_load_hole_fallback(inode, vmf, zero_folio, *entry); + return VM_FAULT_FALLBACK; } =20 - if (pgtable) { - pgtable_trans_huge_deposit(vma->vm_mm, vmf->pmd, pgtable); - mm_inc_nr_ptes(vma->vm_mm); - } - pmd_entry =3D folio_mk_pmd(zero_folio, vmf->vma->vm_page_prot); - set_pmd_at(vmf->vma->vm_mm, pmd_addr, vmf->pmd, pmd_entry); - spin_unlock(ptl); - trace_dax_pmd_load_hole(inode, vmf, zero_folio, *entry); - return VM_FAULT_NOPAGE; + *entry =3D dax_insert_entry(xas, vmf, iter, *entry, folio_pfn(zero_folio), + DAX_PMD | DAX_ZERO_PAGE); =20 -fallback: - if (pgtable) - pte_free(vma->vm_mm, pgtable); - trace_dax_pmd_load_hole_fallback(inode, vmf, zero_folio, *entry); - return VM_FAULT_FALLBACK; + ret =3D vmf_insert_folio_pmd(vmf, zero_folio, false); + if (ret =3D=3D VM_FAULT_NOPAGE) + trace_dax_pmd_load_hole(inode, vmf, zero_folio, *entry); + return ret; } #else static vm_fault_t dax_pmd_load_hole(struct xa_state *xas, struct vm_fault = *vmf, --=20 2.49.0 From nobody Thu Oct 9 13:18:00 2025 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 211F12F5483 for ; Tue, 17 Jun 2025 15:44:10 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1750175051; cv=none; b=X2+NHNIBDbuoAl8rGx+2GmOsuaSvNtgn+JfEf0W3B+jmS0e49vMBaWV2rO/uMyFd//0J0mRKCS2xNCQzL7ztDfensUvOHSOYWgnrxt7UHj7KD6uYX1jO3sCWM6gx2DRzUs29i815gVKtGiL3Mf1eEuGhfZM+rk3etAeOe3+cgkY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1750175051; c=relaxed/simple; bh=Qesw7FWBdB790FSeVsoF7pLTin07jZCKibjTFQ1N/4o=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=pTLoRZn6iZZwWV7lZ8RwGoaV9/dRrGv47DmDjffkQZMVtneRHnopfjH1lQFz3YOUHdbUvkF6wAKbjoOQ5iYBMSsofPqtHfeaTsaBxDM07JxlUIcfx4XOJzRhz4TrRuHYzKCFwZul8+YAhSPebWdqpnMFW/OuibMrhNWWrt8ar1w= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=LA5nN60J; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="LA5nN60J" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1750175049; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=u8pCpCuV3SHMatGSJCijHr+IJztx1AQFgmUpAFfOuvk=; b=LA5nN60J85cVeuPFax7G1hLDhovss6toL7z3FuK/B/0nfvjQEr4AYa+NzdJ5vy9PABVyfU 69QPGC0845mH9l+dtoMA9txw8MCdt+m0xWYlEpW5qzB2xuJcask76bbacfHEvUmQMo+yAC xAm+o0MKmyPsXhsnA2AQOdSkHUT+WOQ= Received: from mail-wm1-f69.google.com (mail-wm1-f69.google.com [209.85.128.69]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-686-c1wfI6IiPn6QSncETxJxXg-1; Tue, 17 Jun 2025 11:44:08 -0400 X-MC-Unique: c1wfI6IiPn6QSncETxJxXg-1 X-Mimecast-MFC-AGG-ID: c1wfI6IiPn6QSncETxJxXg_1750175047 Received: by mail-wm1-f69.google.com with SMTP id 5b1f17b1804b1-451ac1b43c4so32906885e9.0 for ; Tue, 17 Jun 2025 08:44:08 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1750175047; x=1750779847; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=u8pCpCuV3SHMatGSJCijHr+IJztx1AQFgmUpAFfOuvk=; b=f9mKe3UWzs6j6Qg0Cx4bxFDnU4sJKLRNx128cTohL7mZbNSY5oBUKU9W4FCOijmkQu 5YkprACmi+kH2z23Kj3RansCma2PNE2WPGm3iK56UDk7Ls1dGibXgujgm+3tHEUlPGzm UWH/KLTWMC1L1zkLWEkhi9HRZp8bOGHy+Gp32sDdDYnYmE4vUO9OZcB8hkFmpgxYdBkZ ED9J7OQTFUp2Mpch3SNWUgD3MC4yCNtkxEqXgIm+xwfVaqTXc9tutZEYYs2gvqiyUx8J nX2pR6lVYsaV/vBxAZUMSbx3yXcyTjuLXSIO3zdGaHL8KKAaH4SLMqFJQFoJXyRa0Fnp jQqw== X-Gm-Message-State: AOJu0YwG4LcBwA8Y8T3Q+40mCd5U8mFwi/tGRY+Qtja6OlInr/6ejg0F HX6JEYwYWSopOlxuf39xrLi+T1yQ6K8gjWr5gTEf/VlSNoiBtE6luw3ZHU1bqK0LRE31A/Zp7mX FzvVxFo6t8zjAdxAKu1yumR7Dg8hwUqAfay/c1jTXvUazOPwhCHfjmNp25mIlPBahySY5bG9ASA HD5WW2CqUbLV4G9FCnNohrks2Y1FNYoVMgTl6kX4m1pYnRTOJ4 X-Gm-Gg: ASbGncsXOO0dxKh8uz55MjcQEBmoRxaAD+YIo8qhFxErzn5MF8bBXHQi2EnwRH256aN C52ys0kQRU0XJMKezUex8gA+hdt9Q5pHFR/yxmsfTq2EylRApHaYGlGFvyGeUZ1PgCTn/WoNHjd /RCAitCL1RVIgyjFntes5R+RnEgmt5e0bV0JJ4y+jsG6Fn35rJIo5xPCSAbiexfxu1ZAb8iAsrL BJUgQCaZAej/aAN1YpOjR0FE1cWsLPy1xHWqODlZSV6IJ552AUkx1Vdo2N5myIgabmSV4PY5e+3 1jfvMSidx8DhKvDdQVE8K3Yfs8f0vzXbXpdFXwkK8PvA2D4XD3XEaK8kEK13z67TfjXMdO3qbBJ 85emFRw== X-Received: by 2002:a05:6000:2c13:b0:3a5:1241:ce99 with SMTP id ffacd0b85a97d-3a5723ad5dfmr10866456f8f.24.1750175047024; Tue, 17 Jun 2025 08:44:07 -0700 (PDT) X-Google-Smtp-Source: AGHT+IEHpBD+dKVpYruKbhanJPmGuA69TQLBLGeutEiQZdQL6alvi0yDoDZEPOFHzDtmskCoeBKHRQ== X-Received: by 2002:a05:6000:2c13:b0:3a5:1241:ce99 with SMTP id ffacd0b85a97d-3a5723ad5dfmr10866409f8f.24.1750175046528; Tue, 17 Jun 2025 08:44:06 -0700 (PDT) Received: from localhost (p200300d82f3107003851c66ab6b93490.dip0.t-ipconnect.de. [2003:d8:2f31:700:3851:c66a:b6b9:3490]) by smtp.gmail.com with UTF8SMTPSA id ffacd0b85a97d-3a568b4e4f1sm14054457f8f.87.2025.06.17.08.44.04 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Tue, 17 Jun 2025 08:44:06 -0700 (PDT) From: David Hildenbrand To: linux-kernel@vger.kernel.org Cc: linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, nvdimm@lists.linux.dev, David Hildenbrand , Andrew Morton , Juergen Gross , Stefano Stabellini , Oleksandr Tyshchenko , Dan Williams , Alistair Popple , Matthew Wilcox , Jan Kara , Alexander Viro , Christian Brauner , Zi Yan , Baolin Wang , Lorenzo Stoakes , "Liam R. Howlett" , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Jann Horn , Pedro Falcato Subject: [PATCH RFC 08/14] mm/huge_memory: mark PMD mappings of the huge zero folio special Date: Tue, 17 Jun 2025 17:43:39 +0200 Message-ID: <20250617154345.2494405-9-david@redhat.com> X-Mailer: git-send-email 2.49.0 In-Reply-To: <20250617154345.2494405-1-david@redhat.com> References: <20250617154345.2494405-1-david@redhat.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" The huge zero folio is refcounted (+mapcounted -- is that a word?) differently than "normal" folios, similarly (but different) to the ordinary shared zeropage. For this reason, we special-case these pages in vm_normal_page*/vm_normal_folio*, and only allow selected callers to still use them (e.g., GUP can still take a reference on them). vm_normal_page_pmd() already filters out the huge zero folio. However, so far we are not marking it as special like we do with the ordinary shared zeropage. Let's mark it as special, so we can further refactor vm_normal_page_pmd() and vm_normal_page(). While at it, update the doc regarding the shared zero folios. Signed-off-by: David Hildenbrand Reviewed-by: Oscar Salvador --- mm/huge_memory.c | 5 ++++- mm/memory.c | 13 +++++++++---- 2 files changed, 13 insertions(+), 5 deletions(-) diff --git a/mm/huge_memory.c b/mm/huge_memory.c index 92400f3baa9ff..8f03cd4e40397 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -1309,6 +1309,7 @@ static void set_huge_zero_folio(pgtable_t pgtable, st= ruct mm_struct *mm, { pmd_t entry; entry =3D folio_mk_pmd(zero_folio, vma->vm_page_prot); + entry =3D pmd_mkspecial(entry); pgtable_trans_huge_deposit(mm, pmd, pgtable); set_pmd_at(mm, haddr, pmd, entry); mm_inc_nr_ptes(mm); @@ -1418,7 +1419,9 @@ static vm_fault_t insert_pmd(struct vm_area_struct *v= ma, unsigned long addr, if (fop.is_folio) { entry =3D folio_mk_pmd(fop.folio, vma->vm_page_prot); =20 - if (!is_huge_zero_folio(fop.folio)) { + if (is_huge_zero_folio(fop.folio)) { + entry =3D pmd_mkspecial(entry); + } else { folio_get(fop.folio); folio_add_file_rmap_pmd(fop.folio, &fop.folio->page, vma); add_mm_counter(mm, mm_counter_file(fop.folio), HPAGE_PMD_NR); diff --git a/mm/memory.c b/mm/memory.c index 9a1acd057ce59..ef277dab69e33 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -541,7 +541,13 @@ static void print_bad_pte(struct vm_area_struct *vma, = unsigned long addr, * * "Special" mappings do not wish to be associated with a "struct page" (e= ither * it doesn't exist, or it exists but they don't want to touch it). In this - * case, NULL is returned here. "Normal" mappings do have a struct page. + * case, NULL is returned here. "Normal" mappings do have a struct page and + * are ordinarily refcounted. + * + * Page mappings of the shared zero folios are always considered "special"= , as + * they are not ordinarily refcounted. However, selected page table walkers + * (such as GUP) can still identify these mappings and work with the + * underlying "struct page". * * There are 2 broad cases. Firstly, an architecture may define a pte_spec= ial() * pte bit, in which case this function is trivial. Secondly, an architect= ure @@ -571,9 +577,8 @@ static void print_bad_pte(struct vm_area_struct *vma, u= nsigned long addr, * * VM_MIXEDMAP mappings can likewise contain memory with or without "struct * page" backing, however the difference is that _all_ pages with a struct - * page (that is, those where pfn_valid is true) are refcounted and consid= ered - * normal pages by the VM. The only exception are zeropages, which are - * *never* refcounted. + * page (that is, those where pfn_valid is true, except the shared zero + * folios) are refcounted and considered normal pages by the VM. * * The disadvantage is that pages are refcounted (which can be slower and * simply not an option for some PFNMAP users). The advantage is that we --=20 2.49.0 From nobody Thu Oct 9 13:18:00 2025 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 66D362DFF05 for ; Tue, 17 Jun 2025 15:44:12 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1750175053; cv=none; b=oOSwrTNxBWkyHGXd1IaQIX12fwVElvyedy7S08xzm870Mn0Tew3HFqdm1kt4qfWfUzu7NGUvsyGM0O5gb5G46sqVlHm56flf4YU5r67+H243Ooc4ZBueatyTCJI+LkfiQCPRWbY5/7Pwg69KiOIIPcwUv67+C/kLEwjRwjhI4OA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1750175053; c=relaxed/simple; bh=8LbU7WHsSEUDC9XyX0d6DQNa5JIUja8IK4JgIC3/fqo=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=eAgBCu0f1bsqZ1vf1mWP5p3GNFKgV9EF7Fz0kEcIXPJfj7qn8iZUwUJOz04zVSZFhgtUh85vkiF/JW/ei+gA1S5I2WaEI+GLzv98qLcxmxyFAn7rUTDDZjjZgNbIaNkN9ZNrEADzRjUVph3xOjhNzjpwfQlQX1eeg+S/qRhOJGk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=eQZv2OlI; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="eQZv2OlI" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1750175051; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=VPSX3LBtUNEGTHSkguHJQ+rqsVYvGpTDP2rR8uKdQH8=; b=eQZv2OlIC1FKlMF7ram8GaWB464GI5OTZj9cu4zaY660Y0vAb8tPPv6t7BqxXcQnuLwGFe cVH/vOxOCqAigFNEHzLpPfs6hCEAJwKlyfhhnzWvQBO4MOp8jbI1b2fZg7Q8JiKnSTnKq3 MO7ftmkzMDwUTAVbXOuUqKtLFU1pEV0= Received: from mail-wr1-f69.google.com (mail-wr1-f69.google.com [209.85.221.69]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-361-_7AhdcPMPgWckoR3IWKGVw-1; Tue, 17 Jun 2025 11:44:10 -0400 X-MC-Unique: _7AhdcPMPgWckoR3IWKGVw-1 X-Mimecast-MFC-AGG-ID: _7AhdcPMPgWckoR3IWKGVw_1750175049 Received: by mail-wr1-f69.google.com with SMTP id ffacd0b85a97d-3a4f858bc5eso3623603f8f.0 for ; Tue, 17 Jun 2025 08:44:10 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1750175049; x=1750779849; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=VPSX3LBtUNEGTHSkguHJQ+rqsVYvGpTDP2rR8uKdQH8=; b=CIesRVM8c02lsKGBW3E4ZQgSFxbi5i8kNWhHH/4XXV1m3ayxijyoVUxOJm3M5Athb5 PH1GWCcGZRhVDmbQdQ7SK4JueIwo2iBghEanR0KgzotObDaEcRhXGx15a1IDuP+D91Fg Hug+FPNEp8E+Ote4HjSn6kuJ2Iz1zVJfzc9ZhaaZkyMJkzLlOTiAuG2qMET6nem7xg2e dZQAkSmK2zMh/0qQUDw3kDlkwsD1UVlOrHkkngvgN7o/iGsbuO44oG3aqYSj/0s2PlNe 3ghH/fcmDiDZB886tIrLzIVyUSiMdp6vEH3/QBwwxV2m0jZnNXfjwN2MV0Rkql0U+WsL nwjA== X-Gm-Message-State: AOJu0YwuOCJ/KMBbMLpALLBO7AY+yyM+lHzb7AF6/AqNj9yZLmg4K52Y /6XHfVSbCCXRPgdBOjQEyer/6GOV11e8oMPR+ipoy7jDYLc6+YnR5y1ZG/TfIJktw/RIUNSKRgG t2Vt8ogmiL7y4it5TQnkpnp+RkS5wNl8Tl3jHHqePqJo5jyykW35MoQGj86chCXW3iAhSmx7ldp Br+dwBRaURpEYp2SViotUaVTFuHLctj7ggn3HG/JUG/iOq0jeZ X-Gm-Gg: ASbGncsLdn1b5BEL+ezD+J5C2Y5WpeCfDEDW+fkB3J3Peixn949C49C87WXE+JTPzJk mdum8mAT8QELQYBCDvznlspNZ4YpJ1Z05rRAi5UZu9oei8kuQuoUAaUYPztRIB48pZ1XGmX/f9m 5LHGMF/Xt4w6vDPMPVagC14jMCxtzSVRV9LJQ8/5insrw/GahRMhlfrpgVg/aqLFlarTNrwMauw ef9SaXu1TkHzjFindODbcSIrwJ6lRFUFdohJInmtrXtiAcqvZy17nYUZL+RZ4zJSg/QB5rvLAo/ 1MIahcoGVJu9E0s8ZLvHgpCaILfJSlptfrHG0NgSLEE0hVyYAJgXDjXjO4YeWGyDOWFRqjYapCn qVsOSSQ== X-Received: by 2002:a5d:64c5:0:b0:3a5:2670:e220 with SMTP id ffacd0b85a97d-3a572e6b9ffmr8898823f8f.32.1750175049144; Tue, 17 Jun 2025 08:44:09 -0700 (PDT) X-Google-Smtp-Source: AGHT+IHM5m+wJRDUa8DrC/K/257QBXTR6G19KsTYyK8OpbCkrMRqwW7zfUBGXG+Jg5gfXSGV5z4C0A== X-Received: by 2002:a5d:64c5:0:b0:3a5:2670:e220 with SMTP id ffacd0b85a97d-3a572e6b9ffmr8898776f8f.32.1750175048645; Tue, 17 Jun 2025 08:44:08 -0700 (PDT) Received: from localhost (p200300d82f3107003851c66ab6b93490.dip0.t-ipconnect.de. [2003:d8:2f31:700:3851:c66a:b6b9:3490]) by smtp.gmail.com with UTF8SMTPSA id ffacd0b85a97d-3a568b1eb9bsm14491105f8f.69.2025.06.17.08.44.07 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Tue, 17 Jun 2025 08:44:08 -0700 (PDT) From: David Hildenbrand To: linux-kernel@vger.kernel.org Cc: linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, nvdimm@lists.linux.dev, David Hildenbrand , Andrew Morton , Juergen Gross , Stefano Stabellini , Oleksandr Tyshchenko , Dan Williams , Alistair Popple , Matthew Wilcox , Jan Kara , Alexander Viro , Christian Brauner , Zi Yan , Baolin Wang , Lorenzo Stoakes , "Liam R. Howlett" , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Jann Horn , Pedro Falcato Subject: [PATCH RFC 09/14] mm/memory: introduce is_huge_zero_pfn() and use it in vm_normal_page_pmd() Date: Tue, 17 Jun 2025 17:43:40 +0200 Message-ID: <20250617154345.2494405-10-david@redhat.com> X-Mailer: git-send-email 2.49.0 In-Reply-To: <20250617154345.2494405-1-david@redhat.com> References: <20250617154345.2494405-1-david@redhat.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Let's avoid working with the PMD when not required. If vm_normal_page_pmd() would be called on something that is not a present pmd, it would already be a bug (pfn possibly garbage). While at it, let's support passing in any pfn covered by the huge zero folio by masking off PFN bits -- which should be rather cheap. Signed-off-by: David Hildenbrand Reviewed-by: Oscar Salvador --- include/linux/huge_mm.h | 12 +++++++++++- mm/memory.c | 2 +- 2 files changed, 12 insertions(+), 2 deletions(-) diff --git a/include/linux/huge_mm.h b/include/linux/huge_mm.h index 35e34e6a98a27..b260f9a1fd3f2 100644 --- a/include/linux/huge_mm.h +++ b/include/linux/huge_mm.h @@ -483,9 +483,14 @@ static inline bool is_huge_zero_folio(const struct fol= io *folio) return READ_ONCE(huge_zero_folio) =3D=3D folio; } =20 +static inline bool is_huge_zero_pfn(unsigned long pfn) +{ + return READ_ONCE(huge_zero_pfn) =3D=3D (pfn & ~(HPAGE_PMD_NR - 1)); +} + static inline bool is_huge_zero_pmd(pmd_t pmd) { - return pmd_present(pmd) && READ_ONCE(huge_zero_pfn) =3D=3D pmd_pfn(pmd); + return pmd_present(pmd) && is_huge_zero_pfn(pmd_pfn(pmd)); } =20 struct folio *mm_get_huge_zero_folio(struct mm_struct *mm); @@ -633,6 +638,11 @@ static inline bool is_huge_zero_folio(const struct fol= io *folio) return false; } =20 +static inline bool is_huge_zero_pfn(unsigned long pfn) +{ + return false; +} + static inline bool is_huge_zero_pmd(pmd_t pmd) { return false; diff --git a/mm/memory.c b/mm/memory.c index ef277dab69e33..b6c069f4ad11f 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -669,7 +669,7 @@ struct page *vm_normal_page_pmd(struct vm_area_struct *= vma, unsigned long addr, } } =20 - if (is_huge_zero_pmd(pmd)) + if (is_huge_zero_pfn(pfn)) return NULL; =20 /* --=20 2.49.0 From nobody Thu Oct 9 13:18:00 2025 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9458F2DFF31 for ; Tue, 17 Jun 2025 15:44:14 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1750175056; cv=none; b=uBsGiqfiSUZ5EzbUgyoQZQh3xxegdDfQr0DfmfTlhHF+BQRhlqbsfFu/JJCMLnCRNFuXY2GzU9HeYw3vf+GsO3rb0OfeOIi8fEDc9pJJt45m4BpQPhXDrjx1fn74PlL03Mnjnv/vvBO56r8krgn5Oo6K2bqOUZx/2VtZq0RtDsw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1750175056; c=relaxed/simple; bh=CpTPvCWCiY6s8YcaOvOWvnzj3sq5tS5GAvvZ9Q2kmdk=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=qKHhjRgp6cNQxvL+70rpW8RhN4p2QZrPPcWXen10LWYHzQPhBkruuLNjpmi3Zj/bLj1E/Fl1qa+x2qpyU+57KplWGQkrHQ3KUrvKpZmwIQUNKimIJHOKvYaKSiwaA6ECqYu3oH01eHtWFJAWZNxy90Wvuzgl/Ja6CKb/PHCBFts= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=a//OnxY0; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="a//OnxY0" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1750175053; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=8nQwds7Yyk6JA3H6HqbSBsBU+gjLOchhVzRWSpa3UwQ=; b=a//OnxY0FCBATWUxqMkpC/EiTuBKMVUCn1l+gvh5VvgXTS8WglmiKH7Wli72qQNHQe0AfR nGZ0/Sj0aiFrhvS9d8XRxehCgR+qTagxIkMaSEchr3jj8Wfj8OA+rp2M6TJrfAPptnjKgL V/4lK0klPPOUv9sCBtkCnkcxyaYXzII= Received: from mail-wr1-f70.google.com (mail-wr1-f70.google.com [209.85.221.70]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-595-3_TfplarMVCLawjbsfdB0Q-1; Tue, 17 Jun 2025 11:44:12 -0400 X-MC-Unique: 3_TfplarMVCLawjbsfdB0Q-1 X-Mimecast-MFC-AGG-ID: 3_TfplarMVCLawjbsfdB0Q_1750175051 Received: by mail-wr1-f70.google.com with SMTP id ffacd0b85a97d-3a4f8fd1856so2864348f8f.2 for ; Tue, 17 Jun 2025 08:44:12 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1750175051; x=1750779851; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=8nQwds7Yyk6JA3H6HqbSBsBU+gjLOchhVzRWSpa3UwQ=; b=njfyQNTx98V/0nOmpg45JO49gCjRRrcqlTQer/G/diryRiRwmuFnmsA54awce2eIIf X7tjU/linfHw9P2F6R30B1K4bzxfIAdJpO+aomJJJjKRGvx6UXzuqFgVrHfyZLa6jSoz dvpNZ1fNI9uE6SDtY0Tw/cRQenrp8fTYlpjswS/sJ/h9vbpo9WOOMjNxpF/6z02wuI9M Ll8NzBq5eKyRCAfVSbmbDFPRWk1eQVGTNDOH3+FIwtWtpd5ysGDd2M+fclEp8vlobAb9 XrQ4JTbHfhxrFspJpDoO12YIJF2IAwrmx2T0rZx3vu3PsvtU5CldXVRtQ6UZGKkAu5xt CkMQ== X-Gm-Message-State: AOJu0YzoeAPbukijp3LG4BUGWqc/CYLkQzuqvjwxBPpYc/UiwLuJmLLj ft6l3tEiPTIndBXzFissOARpKSpLOAmhfkSfmNm1ExyLGoOhl6PJejCYq1x+VuwkoSodfJeyEbC IO/kLawl1jCb9T2bj15ezKAyQ0jHcQb0K/8MVMTzOX6TioPRXXZwmm2VOZRQrwXgKOt2uVDMIEl 2UhoSAd190Reu2wo+vPKE4ucLt018GZjmb8fIugm6s0fiDf/W2 X-Gm-Gg: ASbGncurPgMKXOlq6QfmFvCrES+YO9anZHz3C2J/vSgMA7wydHcedI08kfPRhPJ9hmN 3qdTRrSPDPojTKAOIvTjJSbGEO9SZUy8E1zeB8lsra9cY5uNpHINTqYjcS23+ik6HmsPDtROd8q 8Xpqvl700nKj+tlRVJ/LgcI2DAj03kCVde0xOSyRC97l8Q3GsQKcw5w2ELWrolpyAU8HqZXidhO 048mCgQxgQLpoUrM9qaO/yiF+FtVOTFd6sznfGiqHx/oF48asCgSU9hINaQt4/F67/Fp58/7k7O u2a9SldHcPqHHrG2L8x+nSRZASbCbD4bTlNgOVYeqcb1mK+ItebHEf5RB94SDT8JyxBFjWIfbW5 sa4G8yA== X-Received: by 2002:a05:6000:4282:b0:3a4:fefb:c8d3 with SMTP id ffacd0b85a97d-3a5723af0aamr11566067f8f.40.1750175051338; Tue, 17 Jun 2025 08:44:11 -0700 (PDT) X-Google-Smtp-Source: AGHT+IF/V6oFuqHOfr5zNP3xy/su55zH7TFZMZO7NjqgYAqVSSsBW8t2gqFSXD3CPoL3K9BbxYOdvQ== X-Received: by 2002:a05:6000:4282:b0:3a4:fefb:c8d3 with SMTP id ffacd0b85a97d-3a5723af0aamr11566020f8f.40.1750175050826; Tue, 17 Jun 2025 08:44:10 -0700 (PDT) Received: from localhost (p200300d82f3107003851c66ab6b93490.dip0.t-ipconnect.de. [2003:d8:2f31:700:3851:c66a:b6b9:3490]) by smtp.gmail.com with UTF8SMTPSA id ffacd0b85a97d-3a568b087f8sm14566243f8f.53.2025.06.17.08.44.09 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Tue, 17 Jun 2025 08:44:10 -0700 (PDT) From: David Hildenbrand To: linux-kernel@vger.kernel.org Cc: linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, nvdimm@lists.linux.dev, David Hildenbrand , Andrew Morton , Juergen Gross , Stefano Stabellini , Oleksandr Tyshchenko , Dan Williams , Alistair Popple , Matthew Wilcox , Jan Kara , Alexander Viro , Christian Brauner , Zi Yan , Baolin Wang , Lorenzo Stoakes , "Liam R. Howlett" , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Jann Horn , Pedro Falcato Subject: [PATCH RFC 10/14] mm/memory: factor out common code from vm_normal_page_*() Date: Tue, 17 Jun 2025 17:43:41 +0200 Message-ID: <20250617154345.2494405-11-david@redhat.com> X-Mailer: git-send-email 2.49.0 In-Reply-To: <20250617154345.2494405-1-david@redhat.com> References: <20250617154345.2494405-1-david@redhat.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Let's reduce the code duplication and factor out the non-pte/pmd related magic into vm_normal_page_pfn(). To keep it simpler, check the pfn against both zero folios. We could optimize this, but as it's only for the !CONFIG_ARCH_HAS_PTE_SPECIAL case, it's not a compelling micro-optimization. With CONFIG_ARCH_HAS_PTE_SPECIAL we don't have to check anything else, really. It's a good question if we can even hit the !CONFIG_ARCH_HAS_PTE_SPECIAL scenario in the PMD case in practice: but doesn't really matter, as it's now all unified in vm_normal_page_pfn(). While at it, add a check that pmd_special() is really only set where we would expect it. No functional change intended. Signed-off-by: David Hildenbrand Reviewed-by: Oscar Salvador --- mm/memory.c | 104 +++++++++++++++++++++++----------------------------- 1 file changed, 46 insertions(+), 58 deletions(-) diff --git a/mm/memory.c b/mm/memory.c index b6c069f4ad11f..3d3fa01cd217e 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -536,6 +536,46 @@ static void print_bad_pte(struct vm_area_struct *vma, = unsigned long addr, add_taint(TAINT_BAD_PAGE, LOCKDEP_NOW_UNRELIABLE); } =20 +/* Called only if the page table entry is not marked special. */ +static inline struct page *vm_normal_page_pfn(struct vm_area_struct *vma, + unsigned long addr, unsigned long pfn) +{ + /* + * With CONFIG_ARCH_HAS_PTE_SPECIAL, any special page table mappings + * (incl. shared zero folios) are marked accordingly. + */ + if (IS_ENABLED(CONFIG_ARCH_HAS_PTE_SPECIAL)) + goto normal_page; + + if (unlikely(vma->vm_flags & (VM_PFNMAP | VM_MIXEDMAP))) { + if (vma->vm_flags & VM_MIXEDMAP) { + /* If it has a "struct page", it's "normal". */ + if (!pfn_valid(pfn)) + return NULL; + } else { + unsigned long off =3D (addr - vma->vm_start) >> PAGE_SHIFT; + + /* Only CoW'ed anon folios are "normal". */ + if (pfn =3D=3D vma->vm_pgoff + off) + return NULL; + if (!is_cow_mapping(vma->vm_flags)) + return NULL; + } + } + + if (is_zero_pfn(pfn) || is_huge_zero_pfn(pfn)) + return NULL; + +normal_page: + /* + * NOTE! We still have PageReserved() pages in the page tables. + * For example, VDSO mappings can cause them to exist. + */ + VM_WARN_ON_ONCE(!pfn_valid(pfn)); + VM_WARN_ON_ONCE(is_zero_pfn(pfn) || is_huge_zero_pfn(pfn)); + return pfn_to_page(pfn); +} + /* * vm_normal_page -- This function gets the "struct page" associated with = a pte. * @@ -591,9 +631,7 @@ struct page *vm_normal_page(struct vm_area_struct *vma,= unsigned long addr, { unsigned long pfn =3D pte_pfn(pte); =20 - if (IS_ENABLED(CONFIG_ARCH_HAS_PTE_SPECIAL)) { - if (likely(!pte_special(pte))) - goto out; + if (unlikely(pte_special(pte))) { if (vma->vm_ops && vma->vm_ops->find_special_page) return vma->vm_ops->find_special_page(vma, addr); if (vma->vm_flags & (VM_PFNMAP | VM_MIXEDMAP)) @@ -604,34 +642,7 @@ struct page *vm_normal_page(struct vm_area_struct *vma= , unsigned long addr, print_bad_pte(vma, addr, pte, NULL); return NULL; } - - /* !CONFIG_ARCH_HAS_PTE_SPECIAL case follows: */ - - if (unlikely(vma->vm_flags & (VM_PFNMAP|VM_MIXEDMAP))) { - if (vma->vm_flags & VM_MIXEDMAP) { - if (!pfn_valid(pfn)) - return NULL; - } else { - unsigned long off; - off =3D (addr - vma->vm_start) >> PAGE_SHIFT; - if (pfn =3D=3D vma->vm_pgoff + off) - return NULL; - if (!is_cow_mapping(vma->vm_flags)) - return NULL; - } - } - - if (is_zero_pfn(pfn)) - return NULL; - - /* - * NOTE! We still have PageReserved() pages in the page tables. - * eg. VDSO mappings can cause them to exist. - */ -out: - VM_WARN_ON_ONCE(!pfn_valid(pfn)); - VM_WARN_ON_ONCE(is_zero_pfn(pfn)); - return pfn_to_page(pfn); + return vm_normal_page_pfn(vma, addr, pfn); } =20 struct folio *vm_normal_folio(struct vm_area_struct *vma, unsigned long ad= dr, @@ -650,35 +661,12 @@ struct page *vm_normal_page_pmd(struct vm_area_struct= *vma, unsigned long addr, { unsigned long pfn =3D pmd_pfn(pmd); =20 - /* Currently it's only used for huge pfnmaps */ - if (unlikely(pmd_special(pmd))) + if (unlikely(pmd_special(pmd))) { + VM_WARN_ON_ONCE(!(vma->vm_flags & (VM_PFNMAP | VM_MIXEDMAP)) && + !is_huge_zero_pfn(pfn)); return NULL; - - if (unlikely(vma->vm_flags & (VM_PFNMAP|VM_MIXEDMAP))) { - if (vma->vm_flags & VM_MIXEDMAP) { - if (!pfn_valid(pfn)) - return NULL; - goto out; - } else { - unsigned long off; - off =3D (addr - vma->vm_start) >> PAGE_SHIFT; - if (pfn =3D=3D vma->vm_pgoff + off) - return NULL; - if (!is_cow_mapping(vma->vm_flags)) - return NULL; - } } - - if (is_huge_zero_pfn(pfn)) - return NULL; - - /* - * NOTE! We still have PageReserved() pages in the page tables. - * eg. VDSO mappings can cause them to exist. - */ -out: - VM_WARN_ON_ONCE(!pfn_valid(pfn)); - return pfn_to_page(pfn); + return vm_normal_page_pfn(vma, addr, pfn); } =20 struct folio *vm_normal_folio_pmd(struct vm_area_struct *vma, --=20 2.49.0 From nobody Thu Oct 9 13:18:00 2025 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7B4222EA150 for ; Tue, 17 Jun 2025 15:44:17 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1750175060; cv=none; b=ObGQ7DXBSkXyXMnok/o535DMxw6BKM4kBsA+ixk60WCG78vrv2jCe+85w0MHqCfm+rIN2tS2+JJ7vV+yDl72Wk7/UA/HewA45YQpA49X3HxfEa7VfC1DrojZah9CnLy3ttOSlzct+gnAYHszenn6tXTTOqVOWbehnkPn7rzPLaU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1750175060; c=relaxed/simple; bh=OJI5f+1+NYHuwxH37IlrE7TLESeijQhASgWBHOjp2gw=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=HVd7qp4gQ1jRyjLVUoed1breKq9JOn+MOMRq+WMPUWkD1Fc0LSyV8eoXeQZ6UwruuGIEC74zSVTNvKgAbNEoWBLluCly7fNcuZmqlFvHUb2Hl9nmRU8ql+CL5AIzQpXhW7QXqxbkTwJbmjcxxCmhKnOkNz19lCSmQdNfFRfuoXE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=Gc+bm97I; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="Gc+bm97I" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1750175056; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=EH5nVYJRAiyh61U5bFJOLvxZlURX1rOpBEXY4zGSahY=; b=Gc+bm97IFsT+ZwcDQdk3Q70EOYTGQ3dg0FHYXyJPP5DUd7ftIiC8x3ow4S1FlsX02z3sfx fgp7WM7h/BFwzZCc974e5VxOOPKiCEAcf3z08GKkStmYZjRPwsRCMTPhKXos4WUeOcUDPd TIbgC1nsZB5cihp5F7Eh69p3EGsH3qM= Received: from mail-wr1-f71.google.com (mail-wr1-f71.google.com [209.85.221.71]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-426-xPemlLcGPL-zdzkAj12MfA-1; Tue, 17 Jun 2025 11:44:15 -0400 X-MC-Unique: xPemlLcGPL-zdzkAj12MfA-1 X-Mimecast-MFC-AGG-ID: xPemlLcGPL-zdzkAj12MfA_1750175054 Received: by mail-wr1-f71.google.com with SMTP id ffacd0b85a97d-3a5780e8137so1596901f8f.1 for ; Tue, 17 Jun 2025 08:44:15 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1750175054; x=1750779854; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=EH5nVYJRAiyh61U5bFJOLvxZlURX1rOpBEXY4zGSahY=; b=TdGd3lJl+S/jWLvhlbYXX6Y/ArwytESl7wmObWlssFgrBgl6PKLDBNgxZQgZJaitEt 9xzuV5+gWuLZFnqtbRDbaziN8/6qK7/DcMX4is//gwsdAscKj0KWTl0QWxA0Rrn+EUoL HvOsxOprVzp9rosRZpEU4v5v1+zgCex26+/Gnqj06/jpnoNHSx9zeCyDtpiKDfIdxp5k PpZfRvr46wcepm/ukp/bZwcoNc/XUv9i2sj5NC1PQl6HaFOvdrJpoek82CZ0O5VkFE+a 8zuW/gnMY1Youc0otMo/0fFU4J/PQ9iLQkDwuhkDYV5+cI622RCWe5Fn+HnP8BUfKzhl UDow== X-Gm-Message-State: AOJu0YyaDTyxr3Xz7Uw8t/1PGlgHYEaRV5VxmIi0eKhTwtK9Ldbcn/2s vheTLbwVJUPm0GdUA7i1A2F0eKsJZTt8ex5kzq5ASIJSrPckPSw50p7B5eYGKcLoRXvC2xDBw+S 00IK4vJETuEd+JmE3IHIeIsbLsVckIigbATI8eoG8ODXUvrkcY1nbx7VMqdrFsSs0MNK8SKeoNz 0ZyiHmzbwGc6n0Mus7dPAc6YMknwdUwTiyLunM4RCecvKLvNag X-Gm-Gg: ASbGncvH36upOnt8zPpwQdHjh+w2EJEdvIln6BS8f7tH1LkaSNk4DsSmRH7AvHYcRM4 Lc53SfYynYFJ3uYZwsYepfYWQ9F8eN1HZRCeXk2+pJuvRd7jYDk62A2Fkk6i4Moe0dKT/Tk980q N4r6Qo7HymyPF84coKpp1RP4UYp1MObryax+v3YhwM5bOnTwD+A2BX62zKxcNKp7+c8DovU+0sB qU/ylaWNxSkL9zTVS3SwbqKaw0LqGDP8kKVb+SUqRE+tqH9jGylnTv38hvijxL+eHaloDh6othc UwjpjZWuxs0ml8Dw1ULAuqyL5vMrCN2g6erjU/eDrEWLCABnBerNMEEyDFJVS4uWmk+eD76Lkwt +4eiyIg== X-Received: by 2002:a05:6000:2c0d:b0:3a4:edf5:8a41 with SMTP id ffacd0b85a97d-3a56d7bad5emr13168076f8f.4.1750175053836; Tue, 17 Jun 2025 08:44:13 -0700 (PDT) X-Google-Smtp-Source: AGHT+IFJKGvzcDxbJ2BQdcd7rWQnrxwh5Qby5YdaZuNvO87Urq3koTuTn6k4xY4C82ZJ429DbH36HQ== X-Received: by 2002:a05:6000:2c0d:b0:3a4:edf5:8a41 with SMTP id ffacd0b85a97d-3a56d7bad5emr13168003f8f.4.1750175053036; Tue, 17 Jun 2025 08:44:13 -0700 (PDT) Received: from localhost (p200300d82f3107003851c66ab6b93490.dip0.t-ipconnect.de. [2003:d8:2f31:700:3851:c66a:b6b9:3490]) by smtp.gmail.com with UTF8SMTPSA id ffacd0b85a97d-3a568b089d8sm14553679f8f.57.2025.06.17.08.44.11 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Tue, 17 Jun 2025 08:44:12 -0700 (PDT) From: David Hildenbrand To: linux-kernel@vger.kernel.org Cc: linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, nvdimm@lists.linux.dev, David Hildenbrand , Andrew Morton , Juergen Gross , Stefano Stabellini , Oleksandr Tyshchenko , Dan Williams , Alistair Popple , Matthew Wilcox , Jan Kara , Alexander Viro , Christian Brauner , Zi Yan , Baolin Wang , Lorenzo Stoakes , "Liam R. Howlett" , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Jann Horn , Pedro Falcato Subject: [PATCH RFC 11/14] mm: remove "horrible special case to handle copy-on-write behaviour" Date: Tue, 17 Jun 2025 17:43:42 +0200 Message-ID: <20250617154345.2494405-12-david@redhat.com> X-Mailer: git-send-email 2.49.0 In-Reply-To: <20250617154345.2494405-1-david@redhat.com> References: <20250617154345.2494405-1-david@redhat.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Let's make the kernel a bit less horrible, by removing the linearity requirement in CoW PFNMAP mappings with !CONFIG_ARCH_HAS_PTE_SPECIAL. In particular, stop messing with vma->vm_pgoff in weird ways. Simply lookup in applicable (i.e., CoW PFNMAP) mappings whether we have an anon folio. Nobody should ever try mapping anon folios using PFNs, that just screams for other possible issues. To be sure, let's sanity-check when inserting PFNs. Are they really required? Probably not, but it's a good safety net at least for now. The runtime overhead should be limited: there is nothing to do for !CoW mappings (common case), and archs that care about performance (i.e., GUP-fast) should be supporting CONFIG_ARCH_HAS_PTE_SPECIAL either way. Likely the sanity checks added in mm/huge_memory.c are not required for now, because that code is probably only wired up with CONFIG_ARCH_HAS_PTE_SPECIAL, but this way is certainly cleaner and more consistent -- and doesn't really cost us anything in the cases we really care about. Signed-off-by: David Hildenbrand --- include/linux/mm.h | 16 ++++++ mm/huge_memory.c | 16 +++++- mm/memory.c | 118 +++++++++++++++++++++++++-------------------- 3 files changed, 96 insertions(+), 54 deletions(-) diff --git a/include/linux/mm.h b/include/linux/mm.h index 98a606908307b..3f52871becd3f 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -2339,6 +2339,22 @@ static inline bool can_do_mlock(void) { return false= ; } extern int user_shm_lock(size_t, struct ucounts *); extern void user_shm_unlock(size_t, struct ucounts *); =20 +#ifdef CONFIG_ARCH_HAS_PTE_SPECIAL +static inline struct page *vm_pfnmap_normal_page_pfn(struct vm_area_struct= *vma, + unsigned long pfn) +{ + /* + * We don't identify normal pages using PFNs. So if we reach + * this point, it's just for sanity checks that don't apply with + * pte_special() etc. + */ + return NULL; +} +#else +struct page *vm_pfnmap_normal_page_pfn(struct vm_area_struct *vma, + unsigned long pfn); +#endif + struct folio *vm_normal_folio(struct vm_area_struct *vma, unsigned long ad= dr, pte_t pte); struct page *vm_normal_page(struct vm_area_struct *vma, unsigned long addr, diff --git a/mm/huge_memory.c b/mm/huge_memory.c index 8f03cd4e40397..67220c30e7818 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -1479,7 +1479,13 @@ vm_fault_t vmf_insert_pfn_pmd(struct vm_fault *vmf, = unsigned long pfn, BUG_ON(!(vma->vm_flags & (VM_PFNMAP|VM_MIXEDMAP))); BUG_ON((vma->vm_flags & (VM_PFNMAP|VM_MIXEDMAP)) =3D=3D (VM_PFNMAP|VM_MIXEDMAP)); - BUG_ON((vma->vm_flags & VM_PFNMAP) && is_cow_mapping(vma->vm_flags)); + + /* + * Refuse this pfn if we could mistake it as a refcounted folio + * in a CoW mapping later in vm_normal_page_pmd(). + */ + if ((vma->vm_flags & VM_PFNMAP) && vm_pfnmap_normal_page_pfn(vma, pfn)) + return VM_FAULT_SIGBUS; =20 pfnmap_setup_cachemode_pfn(pfn, &pgprot); =20 @@ -1587,7 +1593,13 @@ vm_fault_t vmf_insert_pfn_pud(struct vm_fault *vmf, = unsigned long pfn, BUG_ON(!(vma->vm_flags & (VM_PFNMAP|VM_MIXEDMAP))); BUG_ON((vma->vm_flags & (VM_PFNMAP|VM_MIXEDMAP)) =3D=3D (VM_PFNMAP|VM_MIXEDMAP)); - BUG_ON((vma->vm_flags & VM_PFNMAP) && is_cow_mapping(vma->vm_flags)); + + /* + * Refuse this pfn if we could mistake it as a refcounted folio + * in a CoW mapping later in vm_normal_page_pud(). + */ + if ((vma->vm_flags & VM_PFNMAP) && vm_pfnmap_normal_page_pfn(vma, pfn)) + return VM_FAULT_SIGBUS; =20 pfnmap_setup_cachemode_pfn(pfn, &pgprot); =20 diff --git a/mm/memory.c b/mm/memory.c index 3d3fa01cd217e..ace9c59e97181 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -536,9 +536,35 @@ static void print_bad_pte(struct vm_area_struct *vma, = unsigned long addr, add_taint(TAINT_BAD_PAGE, LOCKDEP_NOW_UNRELIABLE); } =20 +#ifndef CONFIG_ARCH_HAS_PTE_SPECIAL +struct page *vm_pfnmap_normal_page_pfn(struct vm_area_struct *vma, + unsigned long pfn) +{ + struct folio *folio; + struct page *page; + + VM_WARN_ON_ONCE(!(vma->vm_flags & VM_PFNMAP)); + + /* + * If we have a CoW mapping and spot an anon folio, then it can + * only be due to CoW: the page is "normal". + */ + if (likely(!is_cow_mapping(vma->vm_flags))) + return NULL; + if (likely(!pfn_valid(pfn))) + return NULL; + + page =3D pfn_to_page(pfn); + folio =3D page_folio(page); + if (folio_test_slab(folio) || !folio_test_anon(folio)) + return NULL; + return page; +} +#endif /* !CONFIG_ARCH_HAS_PTE_SPECIAL */ + /* Called only if the page table entry is not marked special. */ static inline struct page *vm_normal_page_pfn(struct vm_area_struct *vma, - unsigned long addr, unsigned long pfn) + unsigned long pfn) { /* * With CONFIG_ARCH_HAS_PTE_SPECIAL, any special page table mappings @@ -553,13 +579,8 @@ static inline struct page *vm_normal_page_pfn(struct v= m_area_struct *vma, if (!pfn_valid(pfn)) return NULL; } else { - unsigned long off =3D (addr - vma->vm_start) >> PAGE_SHIFT; - /* Only CoW'ed anon folios are "normal". */ - if (pfn =3D=3D vma->vm_pgoff + off) - return NULL; - if (!is_cow_mapping(vma->vm_flags)) - return NULL; + return vm_pfnmap_normal_page_pfn(vma, pfn); } } =20 @@ -589,30 +610,19 @@ static inline struct page *vm_normal_page_pfn(struct = vm_area_struct *vma, * (such as GUP) can still identify these mappings and work with the * underlying "struct page". * - * There are 2 broad cases. Firstly, an architecture may define a pte_spec= ial() - * pte bit, in which case this function is trivial. Secondly, an architect= ure - * may not have a spare pte bit, which requires a more complicated scheme, - * described below. + * An architecture may support pte_special() to distinguish "special" + * from "normal" mappings more efficiently, and even without the VMA at ha= nd. + * For example, in order to support GUP-fast, whereby we don't have the VMA + * available when walking the page tables, support for pte_special() is + * crucial. + * + * If an architecture does not support pte_special(), this function is less + * trivial and more expensive in some cases. * * A raw VM_PFNMAP mapping (ie. one that is not COWed) is always considere= d a * special mapping (even if there are underlying and valid "struct pages"). * COWed pages of a VM_PFNMAP are always normal. * - * The way we recognize COWed pages within VM_PFNMAP mappings is through t= he - * rules set up by "remap_pfn_range()": the vma will have the VM_PFNMAP bit - * set, and the vm_pgoff will point to the first PFN mapped: thus every sp= ecial - * mapping will always honor the rule - * - * pfn_of_page =3D=3D vma->vm_pgoff + ((addr - vma->vm_start) >> PAGE_SHIF= T) - * - * And for normal mappings this is false. - * - * This restricts such mappings to be a linear translation from virtual ad= dress - * to pfn. To get around this restriction, we allow arbitrary mappings so = long - * as the vma is not a COW mapping; in that case, we know that all ptes are - * special (because none can have been COWed). - * - * * In order to support COW of arbitrary special mappings, we have VM_MIXED= MAP. * * VM_MIXEDMAP mappings can likewise contain memory with or without "struct @@ -621,10 +631,7 @@ static inline struct page *vm_normal_page_pfn(struct v= m_area_struct *vma, * folios) are refcounted and considered normal pages by the VM. * * The disadvantage is that pages are refcounted (which can be slower and - * simply not an option for some PFNMAP users). The advantage is that we - * don't have to follow the strict linearity rule of PFNMAP mappings in - * order to support COWable mappings. - * + * simply not an option for some PFNMAP users). */ struct page *vm_normal_page(struct vm_area_struct *vma, unsigned long addr, pte_t pte) @@ -642,7 +649,7 @@ struct page *vm_normal_page(struct vm_area_struct *vma,= unsigned long addr, print_bad_pte(vma, addr, pte, NULL); return NULL; } - return vm_normal_page_pfn(vma, addr, pfn); + return vm_normal_page_pfn(vma, pfn); } =20 struct folio *vm_normal_folio(struct vm_area_struct *vma, unsigned long ad= dr, @@ -666,7 +673,7 @@ struct page *vm_normal_page_pmd(struct vm_area_struct *= vma, unsigned long addr, !is_huge_zero_pfn(pfn)); return NULL; } - return vm_normal_page_pfn(vma, addr, pfn); + return vm_normal_page_pfn(vma, pfn); } =20 struct folio *vm_normal_folio_pmd(struct vm_area_struct *vma, @@ -2422,6 +2429,13 @@ static vm_fault_t insert_pfn(struct vm_area_struct *= vma, unsigned long addr, pte_t *pte, entry; spinlock_t *ptl; =20 + /* + * Refuse this pfn if we could mistake it as a refcounted folio + * in a CoW mapping later in vm_normal_page(). + */ + if ((vma->vm_flags & VM_PFNMAP) && vm_pfnmap_normal_page_pfn(vma, pfn)) + return VM_FAULT_SIGBUS; + pte =3D get_locked_pte(mm, addr, &ptl); if (!pte) return VM_FAULT_OOM; @@ -2511,7 +2525,6 @@ vm_fault_t vmf_insert_pfn_prot(struct vm_area_struct = *vma, unsigned long addr, BUG_ON(!(vma->vm_flags & (VM_PFNMAP|VM_MIXEDMAP))); BUG_ON((vma->vm_flags & (VM_PFNMAP|VM_MIXEDMAP)) =3D=3D (VM_PFNMAP|VM_MIXEDMAP)); - BUG_ON((vma->vm_flags & VM_PFNMAP) && is_cow_mapping(vma->vm_flags)); BUG_ON((vma->vm_flags & VM_MIXEDMAP) && pfn_valid(pfn)); =20 if (addr < vma->vm_start || addr >=3D vma->vm_end) @@ -2656,10 +2669,11 @@ vm_fault_t vmf_insert_mixed_mkwrite(struct vm_area_= struct *vma, * mappings are removed. any references to nonexistent pages results * in null mappings (currently treated as "copy-on-access") */ -static int remap_pte_range(struct mm_struct *mm, pmd_t *pmd, +static int remap_pte_range(struct vm_area_struct *vma, pmd_t *pmd, unsigned long addr, unsigned long end, unsigned long pfn, pgprot_t prot) { + struct mm_struct *mm =3D vma->vm_mm; pte_t *pte, *mapped_pte; spinlock_t *ptl; int err =3D 0; @@ -2674,6 +2688,14 @@ static int remap_pte_range(struct mm_struct *mm, pmd= _t *pmd, err =3D -EACCES; break; } + /* + * Refuse this pfn if we could mistake it as a refcounted folio + * in a CoW mapping later in vm_normal_page(). + */ + if (vm_pfnmap_normal_page_pfn(vma, pfn)) { + err =3D -EINVAL; + break; + } set_pte_at(mm, addr, pte, pte_mkspecial(pfn_pte(pfn, prot))); pfn++; } while (pte++, addr +=3D PAGE_SIZE, addr !=3D end); @@ -2682,10 +2704,11 @@ static int remap_pte_range(struct mm_struct *mm, pm= d_t *pmd, return err; } =20 -static inline int remap_pmd_range(struct mm_struct *mm, pud_t *pud, +static inline int remap_pmd_range(struct vm_area_struct *vma, pud_t *pud, unsigned long addr, unsigned long end, unsigned long pfn, pgprot_t prot) { + struct mm_struct *mm =3D vma->vm_mm; pmd_t *pmd; unsigned long next; int err; @@ -2697,7 +2720,7 @@ static inline int remap_pmd_range(struct mm_struct *m= m, pud_t *pud, VM_BUG_ON(pmd_trans_huge(*pmd)); do { next =3D pmd_addr_end(addr, end); - err =3D remap_pte_range(mm, pmd, addr, next, + err =3D remap_pte_range(vma, pmd, addr, next, pfn + (addr >> PAGE_SHIFT), prot); if (err) return err; @@ -2705,10 +2728,11 @@ static inline int remap_pmd_range(struct mm_struct = *mm, pud_t *pud, return 0; } =20 -static inline int remap_pud_range(struct mm_struct *mm, p4d_t *p4d, +static inline int remap_pud_range(struct vm_area_struct *vma, p4d_t *p4d, unsigned long addr, unsigned long end, unsigned long pfn, pgprot_t prot) { + struct mm_struct *mm =3D vma->vm_mm; pud_t *pud; unsigned long next; int err; @@ -2719,7 +2743,7 @@ static inline int remap_pud_range(struct mm_struct *m= m, p4d_t *p4d, return -ENOMEM; do { next =3D pud_addr_end(addr, end); - err =3D remap_pmd_range(mm, pud, addr, next, + err =3D remap_pmd_range(vma, pud, addr, next, pfn + (addr >> PAGE_SHIFT), prot); if (err) return err; @@ -2727,10 +2751,11 @@ static inline int remap_pud_range(struct mm_struct = *mm, p4d_t *p4d, return 0; } =20 -static inline int remap_p4d_range(struct mm_struct *mm, pgd_t *pgd, +static inline int remap_p4d_range(struct vm_area_struct *vma, pgd_t *pgd, unsigned long addr, unsigned long end, unsigned long pfn, pgprot_t prot) { + struct mm_struct *mm =3D vma->vm_mm; p4d_t *p4d; unsigned long next; int err; @@ -2741,7 +2766,7 @@ static inline int remap_p4d_range(struct mm_struct *m= m, pgd_t *pgd, return -ENOMEM; do { next =3D p4d_addr_end(addr, end); - err =3D remap_pud_range(mm, p4d, addr, next, + err =3D remap_pud_range(vma, p4d, addr, next, pfn + (addr >> PAGE_SHIFT), prot); if (err) return err; @@ -2773,18 +2798,7 @@ static int remap_pfn_range_internal(struct vm_area_s= truct *vma, unsigned long ad * Disable vma merging and expanding with mremap(). * VM_DONTDUMP * Omit vma from core dump, even when VM_IO turned off. - * - * There's a horrible special case to handle copy-on-write - * behaviour that some programs depend on. We mark the "original" - * un-COW'ed pages by matching them up with "vma->vm_pgoff". - * See vm_normal_page() for details. */ - if (is_cow_mapping(vma->vm_flags)) { - if (addr !=3D vma->vm_start || end !=3D vma->vm_end) - return -EINVAL; - vma->vm_pgoff =3D pfn; - } - vm_flags_set(vma, VM_IO | VM_PFNMAP | VM_DONTEXPAND | VM_DONTDUMP); =20 BUG_ON(addr >=3D end); @@ -2793,7 +2807,7 @@ static int remap_pfn_range_internal(struct vm_area_st= ruct *vma, unsigned long ad flush_cache_range(vma, addr, end); do { next =3D pgd_addr_end(addr, end); - err =3D remap_p4d_range(mm, pgd, addr, next, + err =3D remap_p4d_range(vma, pgd, addr, next, pfn + (addr >> PAGE_SHIFT), prot); if (err) return err; --=20 2.49.0 From nobody Thu Oct 9 13:18:00 2025 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 1969F2EA166 for ; Tue, 17 Jun 2025 15:44:18 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1750175060; cv=none; b=POVGr3sLHlneLEBSZNTPlRkuRGj1kkkmfmcMNJbla38BGIKdOFPHmvO6fmcmdIXAqyUgYaIKTJWFgnbNyQaLtHHeJX76Wmn1af6g+Vl6dcu69w7NZoQ7BPjQBKWfVw23mT9X5TJJKBd1NFBc2N22btCh/mATJQSjGPEcBO9j6wc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1750175060; c=relaxed/simple; bh=uAEe2Dwzev9r3qmzV6kZ+4qAK/kSGOMAxFtOCuL/PmQ=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=j9kxnPHC4JWdGh1x4B0QDZDOcEK0x4fInCekGpNIcECqIVGhzN63EbA7Nz6cGgxzRBGyBwf1rEDJxsdp6NC4xIu4Ouan6aIx6LvGCy1Q4cZ4gbw53dr5c6M+nfk6zqu+51ybkfBXVT4pK1xjUkk4Og+ZuyruBM+Ysj2trCy6uKU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=JGlHWf+p; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="JGlHWf+p" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1750175058; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=mWxrDUiAyufMO24k5ZrEJY14fbMO3RlCzXis3T5vcIs=; b=JGlHWf+pBDPfsEWNwq8erpUnxz+QYUdFWW2mMRDBYrLB/zJWpT3EhFXVB9Sm921bWh+XXS 3xXntK7FZ+VflmRjdqoAASPcrpbS3WkZNrfmcINRgouRCLBo5k+Yn8C7vrlPH6Gc0Ekxv0 1E9vvNbOd+W2RBBMgIkNfMVlwxBXfc0= Received: from mail-wm1-f72.google.com (mail-wm1-f72.google.com [209.85.128.72]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-108-jOkp9y4lMHOapBLp-uaOLw-1; Tue, 17 Jun 2025 11:44:16 -0400 X-MC-Unique: jOkp9y4lMHOapBLp-uaOLw-1 X-Mimecast-MFC-AGG-ID: jOkp9y4lMHOapBLp-uaOLw_1750175056 Received: by mail-wm1-f72.google.com with SMTP id 5b1f17b1804b1-450d64026baso40563265e9.1 for ; Tue, 17 Jun 2025 08:44:16 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1750175055; x=1750779855; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=mWxrDUiAyufMO24k5ZrEJY14fbMO3RlCzXis3T5vcIs=; b=fTKIa4w9eD52osLXduFq02ukvrwLvgvH0CVdsipjC5dRFM7QJdPC2+h+eurvk+PEPQ D/7EFa7hJMWw1hx1W5yyH+pmxthnFy+O8+jmucX6OvgA6okV66N3i+5HVjFYTzLZrN2o AUGZfjSBKfzIWzyQwMvu4fSUN8sWABne39qISU4uRtOa9GVRrJOlHfGWPhEETdvRgr6F MRsAoNXLJSnO7a1TB7xrRC2aPQtBhAyMUeEHidm2bwTBSHKDvl/k5w8eettkHuUcuTel kZ8unOtYwQ9M7CcEZArbwKboCi+EKwqODoMDX+hYKsVtvyMiRLvsYQk0xCZaIwk7sxCu v0Bg== X-Gm-Message-State: AOJu0YxBDT+uTs6WdI0z4V6ZG8cknu2QOGa69ir7k7lO1tPTAPv5ARfM Q2QJ2KOpye8SNSboY79thbvwgEUFT5ag4/llAvyMQV25xaWeL/WDZV2YSCNRhPukKLg04pRmpPu ulDQScFWCrLtbzdJZPAH7z3fBJcAnWm4EcmEak+Re5D25kczqLCEV9Q3bUly10lWFwAFX7wN9oV 7N/R1uxN748/wkNz11k9uzHor3VmupsZ7fe58UVJDrrDOLHtfF X-Gm-Gg: ASbGnctQHoDjCoqxgJoko09jggBDOad2ib7YnFRny2YS3EDKxjzA1CqS7gKVy5OfzU9 IX0LlWagTN3S4dR9F1LMzWjeufWws8yl7UmLSNeoaD6oVbJzkHNj+YB1SHZmf10uxJ7rxH9tz/E qvlkqXmLe0N6nR6cyBXD93tHN2/3J/obzb0QZkKGKrVbdK9q0u4DhH2wrsS1s1hV+tTVyNPWjZX 37ENb+rfmRidOieqDf1IPIcipRj1inQSLBN334pA+qSaTL6qmA37eXdYgUtneyK9if4aeax+cpt NAqmN6HGtRMKhj0UZQ9Yl28qlbbUEDonfQRWyCH70o5Pmz4AMB5CnGzUPH9P+6NrL2W75ROLhyr uWagoqg== X-Received: by 2002:a05:600c:35cc:b0:453:23fe:ca86 with SMTP id 5b1f17b1804b1-4533ca466e5mr128888235e9.4.1750175055597; Tue, 17 Jun 2025 08:44:15 -0700 (PDT) X-Google-Smtp-Source: AGHT+IFxLp1jc85N2QDOuW7BFdm+pgJpsU2NEtjPimOvIk95HaJi7jSUq03BgBarqHz3clTjH8a/dA== X-Received: by 2002:a05:600c:35cc:b0:453:23fe:ca86 with SMTP id 5b1f17b1804b1-4533ca466e5mr128887725e9.4.1750175055093; Tue, 17 Jun 2025 08:44:15 -0700 (PDT) Received: from localhost (p200300d82f3107003851c66ab6b93490.dip0.t-ipconnect.de. [2003:d8:2f31:700:3851:c66a:b6b9:3490]) by smtp.gmail.com with UTF8SMTPSA id 5b1f17b1804b1-4532e14fc8bsm179773215e9.28.2025.06.17.08.44.13 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Tue, 17 Jun 2025 08:44:14 -0700 (PDT) From: David Hildenbrand To: linux-kernel@vger.kernel.org Cc: linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, nvdimm@lists.linux.dev, David Hildenbrand , Andrew Morton , Juergen Gross , Stefano Stabellini , Oleksandr Tyshchenko , Dan Williams , Alistair Popple , Matthew Wilcox , Jan Kara , Alexander Viro , Christian Brauner , Zi Yan , Baolin Wang , Lorenzo Stoakes , "Liam R. Howlett" , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Jann Horn , Pedro Falcato Subject: [PATCH RFC 12/14] mm: drop addr parameter from vm_normal_*_pmd() Date: Tue, 17 Jun 2025 17:43:43 +0200 Message-ID: <20250617154345.2494405-13-david@redhat.com> X-Mailer: git-send-email 2.49.0 In-Reply-To: <20250617154345.2494405-1-david@redhat.com> References: <20250617154345.2494405-1-david@redhat.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" No longer required, let's drop it. Signed-off-by: David Hildenbrand --- fs/proc/task_mmu.c | 6 +++--- include/linux/mm.h | 6 ++---- mm/huge_memory.c | 4 ++-- mm/memory.c | 8 +++----- mm/pagewalk.c | 2 +- 5 files changed, 11 insertions(+), 15 deletions(-) diff --git a/fs/proc/task_mmu.c b/fs/proc/task_mmu.c index c4ad3083bbfa0..36ef67cdf7a3b 100644 --- a/fs/proc/task_mmu.c +++ b/fs/proc/task_mmu.c @@ -861,7 +861,7 @@ static void smaps_pmd_entry(pmd_t *pmd, unsigned long a= ddr, struct folio *folio; =20 if (pmd_present(*pmd)) { - page =3D vm_normal_page_pmd(vma, addr, *pmd); + page =3D vm_normal_page_pmd(vma, *pmd); present =3D true; } else if (unlikely(thp_migration_supported() && is_swap_pmd(*pmd))) { swp_entry_t entry =3D pmd_to_swp_entry(*pmd); @@ -2177,7 +2177,7 @@ static unsigned long pagemap_thp_category(struct page= map_scan_private *p, categories |=3D PAGE_IS_WRITTEN; =20 if (p->masks_of_interest & PAGE_IS_FILE) { - page =3D vm_normal_page_pmd(vma, addr, pmd); + page =3D vm_normal_page_pmd(vma, pmd); if (page && !PageAnon(page)) categories |=3D PAGE_IS_FILE; } @@ -2942,7 +2942,7 @@ static struct page *can_gather_numa_stats_pmd(pmd_t p= md, if (!pmd_present(pmd)) return NULL; =20 - page =3D vm_normal_page_pmd(vma, addr, pmd); + page =3D vm_normal_page_pmd(vma, pmd); if (!page) return NULL; =20 diff --git a/include/linux/mm.h b/include/linux/mm.h index 3f52871becd3f..ef709457c7076 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -2359,10 +2359,8 @@ struct folio *vm_normal_folio(struct vm_area_struct = *vma, unsigned long addr, pte_t pte); struct page *vm_normal_page(struct vm_area_struct *vma, unsigned long addr, pte_t pte); -struct folio *vm_normal_folio_pmd(struct vm_area_struct *vma, - unsigned long addr, pmd_t pmd); -struct page *vm_normal_page_pmd(struct vm_area_struct *vma, unsigned long = addr, - pmd_t pmd); +struct folio *vm_normal_folio_pmd(struct vm_area_struct *vma, pmd_t pmd); +struct page *vm_normal_page_pmd(struct vm_area_struct *vma, pmd_t pmd); =20 void zap_vma_ptes(struct vm_area_struct *vma, unsigned long address, unsigned long size); diff --git a/mm/huge_memory.c b/mm/huge_memory.c index 67220c30e7818..bf2aed8d92ec2 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -1994,7 +1994,7 @@ static inline bool can_change_pmd_writable(struct vm_= area_struct *vma, =20 if (!(vma->vm_flags & VM_SHARED)) { /* See can_change_pte_writable(). */ - page =3D vm_normal_page_pmd(vma, addr, pmd); + page =3D vm_normal_page_pmd(vma, pmd); return page && PageAnon(page) && PageAnonExclusive(page); } =20 @@ -2033,7 +2033,7 @@ vm_fault_t do_huge_pmd_numa_page(struct vm_fault *vmf) can_change_pmd_writable(vma, vmf->address, pmd)) writable =3D true; =20 - folio =3D vm_normal_folio_pmd(vma, haddr, pmd); + folio =3D vm_normal_folio_pmd(vma, pmd); if (!folio) goto out_map; =20 diff --git a/mm/memory.c b/mm/memory.c index ace9c59e97181..34f961024e8e6 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -663,8 +663,7 @@ struct folio *vm_normal_folio(struct vm_area_struct *vm= a, unsigned long addr, } =20 #ifdef CONFIG_PGTABLE_HAS_HUGE_LEAVES -struct page *vm_normal_page_pmd(struct vm_area_struct *vma, unsigned long = addr, - pmd_t pmd) +struct page *vm_normal_page_pmd(struct vm_area_struct *vma, pmd_t pmd) { unsigned long pfn =3D pmd_pfn(pmd); =20 @@ -676,10 +675,9 @@ struct page *vm_normal_page_pmd(struct vm_area_struct = *vma, unsigned long addr, return vm_normal_page_pfn(vma, pfn); } =20 -struct folio *vm_normal_folio_pmd(struct vm_area_struct *vma, - unsigned long addr, pmd_t pmd) +struct folio *vm_normal_folio_pmd(struct vm_area_struct *vma, pmd_t pmd) { - struct page *page =3D vm_normal_page_pmd(vma, addr, pmd); + struct page *page =3D vm_normal_page_pmd(vma, pmd); =20 if (page) return page_folio(page); diff --git a/mm/pagewalk.c b/mm/pagewalk.c index 648038247a8d2..0edb7240d090c 100644 --- a/mm/pagewalk.c +++ b/mm/pagewalk.c @@ -944,7 +944,7 @@ struct folio *folio_walk_start(struct folio_walk *fw, spin_unlock(ptl); goto pte_table; } else if (pmd_present(pmd)) { - page =3D vm_normal_page_pmd(vma, addr, pmd); + page =3D vm_normal_page_pmd(vma, pmd); if (page) { goto found; } else if ((flags & FW_ZEROPAGE) && --=20 2.49.0 From nobody Thu Oct 9 13:18:00 2025 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id DA0902ED14B for ; Tue, 17 Jun 2025 15:44:20 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1750175062; cv=none; b=X7Ppl4Yz9UiS5hXp/WEgnbll2UxIPiqu9VGQt3pfhns9MVcGrQVNR/dU0z1BfyOGhOsqxxRPUqdTVjl+8vEeAKHyRxfipi0sHnUAejKI6vOfMZV47YSEv4+HZetO85O/Opnyz54JGn0rBTa5Qpb0cxN1Iki2wY0P0oGfDTZyhFI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1750175062; c=relaxed/simple; bh=BsU24r5xgG+FQtB1entGNmMfL4hNbq4o2pzMYPuggy8=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=ceJPxgzCP5D4Y++Fb6yzS7fya7nJEYc34fL7Os4oeYpEGn22gP7onvOjZA9VSJLnEb2YoYo9e5gxvBypAf0q1wnX2QpdrAG2SJHnSJpJEAGzPVyQdFQkYJk0vFs5p9SmAh4QtJYdqKE1R+TyuHDZF+r96UijGhYQFwOTiQbj98k= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=OTSEQItz; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="OTSEQItz" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1750175060; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Lq97cexcfDQMKZRu4/MkS8aw3BkFIIdya9sxYwoOveM=; b=OTSEQItzzDCSl64BFu5ztl+f4vfOA8uC88Az/c01utid7+4pP10fAvtt3L8PrjhggdjXZy B55CqBxPE2uvRUCoQcQYDs3g1N051aqsBBCHRL3DyHaE3CnxKNC3wnCSmyGJuxRvwycGae m13e2e0sBhUtm7p+keZZkRA/5GeQUhE= Received: from mail-wm1-f69.google.com (mail-wm1-f69.google.com [209.85.128.69]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-130-fafsi3KYMmetSqwauRBCVA-1; Tue, 17 Jun 2025 11:44:18 -0400 X-MC-Unique: fafsi3KYMmetSqwauRBCVA-1 X-Mimecast-MFC-AGG-ID: fafsi3KYMmetSqwauRBCVA_1750175058 Received: by mail-wm1-f69.google.com with SMTP id 5b1f17b1804b1-450df53d461so48298615e9.1 for ; Tue, 17 Jun 2025 08:44:18 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1750175057; x=1750779857; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Lq97cexcfDQMKZRu4/MkS8aw3BkFIIdya9sxYwoOveM=; b=WAx2Ie477fXyv6zE/kIy4Ds8d5V0Lqp0WRkAA9WFN6ELy57VH1W03l9mgKmXBLRuSB kHILiCB1FMGgX2iUr+jctb2yOJrbc5KOQ9+qKeMHkx4FKBWAgwUcvxenb3OqB62XSiDR twsF/OyM4uodwBHcWBtpcshXD9ZjEVhSXknaHP6HG9EdmZHK3WaAbrtDqukWCAKR0fUr /E6ZNj6OGwoNeNvUxC4YdLb7788Vjmts0gXtxH66AmPSiEzhouF4TNhPtb+HD0E7aKYm YcneJBtFLHyL3y2nx99n2toFagiznm4ZUgtc2cyH/sW/70dYA3qWENXwy9TFtqi0g5uv lDJg== X-Gm-Message-State: AOJu0YwNl5cz1mHzFXcp90PGfG2t59nvaUFGzPH3GpjmF5/KVq/xsXMc Q4caYewEJZwH9AmL+8sBSI8OO3Obl4KSJ0RhIIIbMGWXjDChG//IxE8s2qfFNLMpxoofgCueN1D 5KK8uzweba75/SBdQ1UHtjCyRsAv//d/QHfvPFOqP18bQgP6Vn+xiry1un1bl43DWMS4NL51GNR 88xST9F+4SVYOjDjgX90ia4Gv9i3j69kty5v3qtWVXtXKREPEb X-Gm-Gg: ASbGncuUcV5/eDdmkvDz+TzN6XT+n2aVA3M1oHzGekeDYj/Bf9Pxc51Edctu7gIBZn2 ZpWKS4dG8JZ1Ptdmelvq0K4Z8ECjA72NKcHjojOxr+j9wS+UyhrVgd/36NFlIsW7F3aILY0QtEw KLZWoGppFbA3681ZHBRCWyd8dTIrx8VFWpta2BavctZ/t4KxE/X/a88TEqDWqyx3z9n2ztJGV6v 4I0WOBIHxcLV1Cnl7B/KMgT0oAX19QM6woc0OYUT+Ep0lKtNXslU+VvRJrvU5Hm6k9yPAjP15I0 oiYSnWN/WVWmRZFq2EOh+kVV6ViBmts8XOiOHWNmZRM7Tcny6xYMmlxyu/I+iKVQkA+SGxydkU8 6x+uwbQ== X-Received: by 2002:a05:600c:1e02:b0:43d:b85:1831 with SMTP id 5b1f17b1804b1-4533c9dbbfemr142093305e9.0.1750175057555; Tue, 17 Jun 2025 08:44:17 -0700 (PDT) X-Google-Smtp-Source: AGHT+IFmLbqQvk/KforjobulOMd7Z8HmizRkZJwsjious12VlfKySMsgRFEBQ04fbeKOdJYcWkDwbQ== X-Received: by 2002:a05:600c:1e02:b0:43d:b85:1831 with SMTP id 5b1f17b1804b1-4533c9dbbfemr142092735e9.0.1750175057156; Tue, 17 Jun 2025 08:44:17 -0700 (PDT) Received: from localhost (p200300d82f3107003851c66ab6b93490.dip0.t-ipconnect.de. [2003:d8:2f31:700:3851:c66a:b6b9:3490]) by smtp.gmail.com with UTF8SMTPSA id 5b1f17b1804b1-4533fc6578csm110483905e9.19.2025.06.17.08.44.15 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Tue, 17 Jun 2025 08:44:16 -0700 (PDT) From: David Hildenbrand To: linux-kernel@vger.kernel.org Cc: linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, nvdimm@lists.linux.dev, David Hildenbrand , Andrew Morton , Juergen Gross , Stefano Stabellini , Oleksandr Tyshchenko , Dan Williams , Alistair Popple , Matthew Wilcox , Jan Kara , Alexander Viro , Christian Brauner , Zi Yan , Baolin Wang , Lorenzo Stoakes , "Liam R. Howlett" , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Jann Horn , Pedro Falcato Subject: [PATCH RFC 13/14] mm: introduce and use vm_normal_page_pud() Date: Tue, 17 Jun 2025 17:43:44 +0200 Message-ID: <20250617154345.2494405-14-david@redhat.com> X-Mailer: git-send-email 2.49.0 In-Reply-To: <20250617154345.2494405-1-david@redhat.com> References: <20250617154345.2494405-1-david@redhat.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Let's introduce vm_normal_page_pud(), which ends up being fairly simple because of our new common helpers and there not being a PUD-sized zero folio. Use vm_normal_page_pud() in folio_walk_start() to resolve a TODO, structuring the code like the other (pmd/pte) cases. Defer introducing vm_normal_folio_pud() until really used. Signed-off-by: David Hildenbrand Reviewed-by: Oscar Salvador --- include/linux/mm.h | 1 + mm/memory.c | 11 +++++++++++ mm/pagewalk.c | 20 ++++++++++---------- 3 files changed, 22 insertions(+), 10 deletions(-) diff --git a/include/linux/mm.h b/include/linux/mm.h index ef709457c7076..022e8ef2c78ef 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -2361,6 +2361,7 @@ struct page *vm_normal_page(struct vm_area_struct *vm= a, unsigned long addr, pte_t pte); struct folio *vm_normal_folio_pmd(struct vm_area_struct *vma, pmd_t pmd); struct page *vm_normal_page_pmd(struct vm_area_struct *vma, pmd_t pmd); +struct page *vm_normal_page_pud(struct vm_area_struct *vma, pud_t pud); =20 void zap_vma_ptes(struct vm_area_struct *vma, unsigned long address, unsigned long size); diff --git a/mm/memory.c b/mm/memory.c index 34f961024e8e6..6c65f51248250 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -683,6 +683,17 @@ struct folio *vm_normal_folio_pmd(struct vm_area_struc= t *vma, pmd_t pmd) return page_folio(page); return NULL; } + +struct page *vm_normal_page_pud(struct vm_area_struct *vma, pud_t pud) +{ + unsigned long pfn =3D pud_pfn(pud); + + if (unlikely(pud_special(pud))) { + VM_WARN_ON_ONCE(!(vma->vm_flags & (VM_PFNMAP | VM_MIXEDMAP))); + return NULL; + } + return vm_normal_page_pfn(vma, pfn); +} #endif =20 /** diff --git a/mm/pagewalk.c b/mm/pagewalk.c index 0edb7240d090c..8bd95cf326872 100644 --- a/mm/pagewalk.c +++ b/mm/pagewalk.c @@ -902,23 +902,23 @@ struct folio *folio_walk_start(struct folio_walk *fw, fw->pudp =3D pudp; fw->pud =3D pud; =20 - /* - * TODO: FW_MIGRATION support for PUD migration entries - * once there are relevant users. - */ - if (!pud_present(pud) || pud_special(pud)) { + if (pud_none(pud)) { spin_unlock(ptl); goto not_found; - } else if (!pud_leaf(pud)) { + } else if (pud_present(pud) && !pud_leaf(pud)) { spin_unlock(ptl); goto pmd_table; + } else if (pud_present(pud)) { + page =3D vm_normal_page_pud(vma, pud); + if (page) + goto found; } /* - * TODO: vm_normal_page_pud() will be handy once we want to - * support PUD mappings in VM_PFNMAP|VM_MIXEDMAP VMAs. + * TODO: FW_MIGRATION support for PUD migration entries + * once there are relevant users. */ - page =3D pud_page(pud); - goto found; + spin_unlock(ptl); + goto not_found; } =20 pmd_table: --=20 2.49.0 From nobody Thu Oct 9 13:18:00 2025 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 96E622EF2AB for ; Tue, 17 Jun 2025 15:44:23 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1750175065; cv=none; b=DfJmEBZ/31l1W/jTOCaCGQniFQjxYbHNghO5IxH1YvttdDMPLfGMq/2z32EeG2k5lgoN3aOijpbzLAzNt8RuFZ6j0UDEUuc6uSleWeNzK1ZqOIiHvuCInAL5JdutBxjAdfrU86SpcR+P/0MvMNJSSqelAWxyIx/ceodRiOdY2Uw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1750175065; c=relaxed/simple; bh=JoPCIxNFhISa/uoEZhmjVFXMY/aHB9zZc630LouBJb0=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=fyjAYGuhBnZ40R8AHAldKMcitJaanC588tYt0l/H+v7Znbi3YRqr03AF9KKOKlFFV6w5n1gDFUuV/Ajhfodc8XPEGhSJoeo96EUNghORLzcDXobJRITiDWyy15G6qZBwMcc3iuQK9EDHE3EzUqrduZC3V0xv/4S+bgHKZioNZuM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=awr/9N48; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="awr/9N48" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1750175062; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=YLVJGNARtVFI5B2J8se/8/6UXe4Uo6I4yV7PyVAOlF8=; b=awr/9N48N1K0HXhB5k6H7unYbZeJnF6338pk0FA99D2G5jnW4Td4mpghU0VfVEseMf+dyK XxGJbPFVukkkQQ9bb+4z69zNUlbG5+wWKVX2zcSHJqlUnjKhAMUCpGWnlQAvdi89UlWW1N iaBl6+j5CIuyiVnB7p34X8UxbtxXOos= Received: from mail-wr1-f71.google.com (mail-wr1-f71.google.com [209.85.221.71]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-269-3cOI401oPlmvkrC-6-CeUw-1; Tue, 17 Jun 2025 11:44:21 -0400 X-MC-Unique: 3cOI401oPlmvkrC-6-CeUw-1 X-Mimecast-MFC-AGG-ID: 3cOI401oPlmvkrC-6-CeUw_1750175060 Received: by mail-wr1-f71.google.com with SMTP id ffacd0b85a97d-3a503f28b09so527105f8f.0 for ; Tue, 17 Jun 2025 08:44:21 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1750175060; x=1750779860; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=YLVJGNARtVFI5B2J8se/8/6UXe4Uo6I4yV7PyVAOlF8=; b=eoLVjaZS0256OAWxRkSmZXTBaio/AYBcS2WOWBx8rKDdLNC0nShOtkGkUylKbdcDpV V54HQD0VS4xPy6BIZhD5YoHzUB3xJ9M6UdpSXyix/HGA9gTmrdZPG6iJIWwj3r7iN2hn +3uSzPmUtTLJpjA9xt4Y1HVFMDomIyXILX5lD7m5KY4030zdfDmAyIIVHTioJ6PNjaR8 04JDMPx5wmqaHgC/tRligZ9v5btjzwT1z0IKDVrxBGYOLiO5ovjxxkeUWlAkJo5Am0qA 1XJ7Ygm3MDZscydyka5PleNk/asN3bFjPP6UpgozAsbm6eln1TP3o3q+QV5R0280ttzX adjA== X-Gm-Message-State: AOJu0Yxe2Ou7nSliBpPnnLhlhMBIA0UmpRhdIMbRi35Bmdq1VkRXJmYZ Cizh9ovlUBLzRYh1PseIo/sf6t/yC+1Y/wGS3V1BT4Mvbm9CUkD+Qmngn8e2N4l3Nj4Ic86BOCe tf4IzRe3L5/gRWX6NFzX1ZGTTFm0o/3pu+nFOvWnn9bJPOIhiBKvGVVZHRdr/IJTE6efLWZ448n sVk7fOYuyKeQsJJGgLd+ZGR3+MddyijNaHkI+4nq35j91JJ3lB X-Gm-Gg: ASbGncut3JEDwkiImOtQKR7m60+WCdpFj/YNXWlt2VNVKbunZUWS1/JJ3g0J0Wh1hJ7 0KE8OseRfrDQeSuiYG7t/0NGzSzXP+BRWpOC1e/dkvN7FIUCdvNusjb3TuaySdZj7F2ee99BJGO v4+0FJ27klNImIWcovI3/2zbFl/k5HEPsScqoatKBsiEhFLqT0/Z//gq20XFrmB6u405yKcqrIc ZETY88FHED+rCiELf2TxescAJqwXhmHh84Un25ZaKxnN7zS/1MC//HBnpgu8EiT0eKM7AXua/e3 KHxXusfDvgIpoKaPtrFjs7XVOtCFjZZk2s5gJaxNWopAF2VH54la8oBM4Sam2xy+NWBrVRgPD+b vZEcU1g== X-Received: by 2002:a05:6000:40d9:b0:3a4:e238:6496 with SMTP id ffacd0b85a97d-3a56d827f55mr10918547f8f.18.1750175059960; Tue, 17 Jun 2025 08:44:19 -0700 (PDT) X-Google-Smtp-Source: AGHT+IGQg7GY1Gsteo7GdT4ckbEy0SNxYC91KxvGox3B/Jtg2LOmOg//Pt1oxMAZKf077YGN9lBupw== X-Received: by 2002:a05:6000:40d9:b0:3a4:e238:6496 with SMTP id ffacd0b85a97d-3a56d827f55mr10918494f8f.18.1750175059294; Tue, 17 Jun 2025 08:44:19 -0700 (PDT) Received: from localhost (p200300d82f3107003851c66ab6b93490.dip0.t-ipconnect.de. [2003:d8:2f31:700:3851:c66a:b6b9:3490]) by smtp.gmail.com with UTF8SMTPSA id ffacd0b85a97d-3a568b4c969sm14100737f8f.85.2025.06.17.08.44.17 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Tue, 17 Jun 2025 08:44:18 -0700 (PDT) From: David Hildenbrand To: linux-kernel@vger.kernel.org Cc: linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, nvdimm@lists.linux.dev, David Hildenbrand , Andrew Morton , Juergen Gross , Stefano Stabellini , Oleksandr Tyshchenko , Dan Williams , Alistair Popple , Matthew Wilcox , Jan Kara , Alexander Viro , Christian Brauner , Zi Yan , Baolin Wang , Lorenzo Stoakes , "Liam R. Howlett" , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Jann Horn , Pedro Falcato , David Vrabel Subject: [PATCH RFC 14/14] mm: rename vm_ops->find_special_page() to vm_ops->find_normal_page() Date: Tue, 17 Jun 2025 17:43:45 +0200 Message-ID: <20250617154345.2494405-15-david@redhat.com> X-Mailer: git-send-email 2.49.0 In-Reply-To: <20250617154345.2494405-1-david@redhat.com> References: <20250617154345.2494405-1-david@redhat.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" ... and hide it behind a kconfig option. There is really no need for any !xen code to perform this check. The naming is a bit off: we want to find the "normal" page when a PTE was marked "special". So it's really not "finding a special" page. Improve the documentation, and add a comment in the code where XEN ends up performing the pte_mkspecial() through a hypercall. More details can be found in commit 923b2919e2c3 ("xen/gntdev: mark userspace PTEs as special on x86 PV guests"). Cc: David Vrabel Signed-off-by: David Hildenbrand --- drivers/xen/Kconfig | 1 + drivers/xen/gntdev.c | 5 +++-- include/linux/mm.h | 18 +++++++++++++----- mm/Kconfig | 2 ++ mm/memory.c | 10 ++++++++-- tools/testing/vma/vma_internal.h | 18 +++++++++++++----- 6 files changed, 40 insertions(+), 14 deletions(-) diff --git a/drivers/xen/Kconfig b/drivers/xen/Kconfig index 24f485827e039..f9a35ed266ecf 100644 --- a/drivers/xen/Kconfig +++ b/drivers/xen/Kconfig @@ -138,6 +138,7 @@ config XEN_GNTDEV depends on XEN default m select MMU_NOTIFIER + select FIND_NORMAL_PAGE help Allows userspace processes to use grants. =20 diff --git a/drivers/xen/gntdev.c b/drivers/xen/gntdev.c index 61faea1f06630..d1bc0dae2cdf9 100644 --- a/drivers/xen/gntdev.c +++ b/drivers/xen/gntdev.c @@ -309,6 +309,7 @@ static int find_grant_ptes(pte_t *pte, unsigned long ad= dr, void *data) BUG_ON(pgnr >=3D map->count); pte_maddr =3D arbitrary_virt_to_machine(pte).maddr; =20 + /* Note: this will perform a pte_mkspecial() through the hypercall. */ gnttab_set_map_op(&map->map_ops[pgnr], pte_maddr, flags, map->grants[pgnr].ref, map->grants[pgnr].domid); @@ -516,7 +517,7 @@ static void gntdev_vma_close(struct vm_area_struct *vma) gntdev_put_map(priv, map); } =20 -static struct page *gntdev_vma_find_special_page(struct vm_area_struct *vm= a, +static struct page *gntdev_vma_find_normal_page(struct vm_area_struct *vma, unsigned long addr) { struct gntdev_grant_map *map =3D vma->vm_private_data; @@ -527,7 +528,7 @@ static struct page *gntdev_vma_find_special_page(struct= vm_area_struct *vma, static const struct vm_operations_struct gntdev_vmops =3D { .open =3D gntdev_vma_open, .close =3D gntdev_vma_close, - .find_special_page =3D gntdev_vma_find_special_page, + .find_normal_page =3D gntdev_vma_find_normal_page, }; =20 /* ------------------------------------------------------------------ */ diff --git a/include/linux/mm.h b/include/linux/mm.h index 022e8ef2c78ef..b01475f3dca99 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -646,13 +646,21 @@ struct vm_operations_struct { struct mempolicy *(*get_policy)(struct vm_area_struct *vma, unsigned long addr, pgoff_t *ilx); #endif +#ifdef CONFIG_FIND_NORMAL_PAGE /* - * Called by vm_normal_page() for special PTEs to find the - * page for @addr. This is useful if the default behavior - * (using pte_page()) would not find the correct page. + * Called by vm_normal_page() for special PTEs in @vma at @addr. This + * allows for returning a "normal" page from vm_normal_page() even + * though the PTE indicates that the "struct page" either does not exist + * or should not be touched: "special". + * + * Do not add new users: this really only works when a "normal" page + * was mapped, but then the PTE got changed to something weird (+ + * marked special) that would not make pte_pfn() identify the originally + * inserted page. */ - struct page *(*find_special_page)(struct vm_area_struct *vma, - unsigned long addr); + struct page *(*find_normal_page)(struct vm_area_struct *vma, + unsigned long addr); +#endif /* CONFIG_FIND_NORMAL_PAGE */ }; =20 #ifdef CONFIG_NUMA_BALANCING diff --git a/mm/Kconfig b/mm/Kconfig index c6194d1f9d170..607a3f9672bdb 100644 --- a/mm/Kconfig +++ b/mm/Kconfig @@ -1390,6 +1390,8 @@ config PT_RECLAIM =20 Note: now only empty user PTE page table pages will be reclaimed. =20 +config FIND_NORMAL_PAGE + def_bool n =20 source "mm/damon/Kconfig" =20 diff --git a/mm/memory.c b/mm/memory.c index 6c65f51248250..1eba95fcde096 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -619,6 +619,10 @@ static inline struct page *vm_normal_page_pfn(struct v= m_area_struct *vma, * If an architecture does not support pte_special(), this function is less * trivial and more expensive in some cases. * + * With CONFIG_FIND_NORMAL_PAGE, we might have pte_special() set on PTEs t= hat + * actually map "normal" pages: however, that page cannot be looked up thr= ough + * pte_pfn(), but instead will be looked up through vm_ops->find_normal_pa= ge(). + * * A raw VM_PFNMAP mapping (ie. one that is not COWed) is always considere= d a * special mapping (even if there are underlying and valid "struct pages"). * COWed pages of a VM_PFNMAP are always normal. @@ -639,8 +643,10 @@ struct page *vm_normal_page(struct vm_area_struct *vma= , unsigned long addr, unsigned long pfn =3D pte_pfn(pte); =20 if (unlikely(pte_special(pte))) { - if (vma->vm_ops && vma->vm_ops->find_special_page) - return vma->vm_ops->find_special_page(vma, addr); +#ifdef CONFIG_FIND_NORMAL_PAGE + if (vma->vm_ops && vma->vm_ops->find_normal_page) + return vma->vm_ops->find_normal_page(vma, addr); +#endif /* CONFIG_FIND_NORMAL_PAGE */ if (vma->vm_flags & (VM_PFNMAP | VM_MIXEDMAP)) return NULL; if (is_zero_pfn(pfn)) diff --git a/tools/testing/vma/vma_internal.h b/tools/testing/vma/vma_inter= nal.h index 51dd122b8d501..c5bf041036dd7 100644 --- a/tools/testing/vma/vma_internal.h +++ b/tools/testing/vma/vma_internal.h @@ -470,13 +470,21 @@ struct vm_operations_struct { struct mempolicy *(*get_policy)(struct vm_area_struct *vma, unsigned long addr, pgoff_t *ilx); #endif +#ifdef CONFIG_FIND_NORMAL_PAGE /* - * Called by vm_normal_page() for special PTEs to find the - * page for @addr. This is useful if the default behavior - * (using pte_page()) would not find the correct page. + * Called by vm_normal_page() for special PTEs in @vma at @addr. This + * allows for returning a "normal" page from vm_normal_page() even + * though the PTE indicates that the "struct page" either does not exist + * or should not be touched: "special". + * + * Do not add new users: this really only works when a "normal" page + * was mapped, but then the PTE got changed to something weird (+ + * marked special) that would not make pte_pfn() identify the originally + * inserted page. */ - struct page *(*find_special_page)(struct vm_area_struct *vma, - unsigned long addr); + struct page *(*find_normal_page)(struct vm_area_struct *vma, + unsigned long addr); +#endif /* CONFIG_FIND_NORMAL_PAGE */ }; =20 struct vm_unmapped_area_info { --=20 2.49.0