From nobody Mon Oct 6 20:56:44 2025 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass(p=quarantine dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1752753173; cv=none; d=zohomail.com; s=zohoarc; b=fo0PaTYFwXgaUQM3rdCn38jH/TWzeRwGoU2S9MdxokmhDAyCSx01PbSHMxIdAPHt5/LssW5cs6d8n9NhNqD9ICF4DQzIds8CLTrppOychVhbOHOkoQ6CbId8QCjXvm+PwdXN426AVFdHUeRY5CQPwLAh8SgUEUH01xdQbfamGpw= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1752753173; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=zIbOZw1x2GVZdwaR5iaeVsCQGMgTtSbEtHQ5YhAco0w=; b=YRKZ0O9rean9uQXJV5Lcs28moWqorkV2h2hzL5i8j9KNOCzMhErKOZUYzEXAnNo3yeRrjhXQgusiwaUVcFf3LvG62h7WP9k0DGQeJMdI1yJ4MYNCVSzGcJL8bZkovEo0ux35cTjEn7vWw1My9sTYTXqCZqmH0Wz6eP5PZ8xwVYA= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1752753173951417.7243802698198; Thu, 17 Jul 2025 04:52:53 -0700 (PDT) Received: from list by lists.xenproject.org with outflank-mailman.1046606.1416961 (Exim 4.92) (envelope-from ) id 1ucNA3-0001Tx-DS; Thu, 17 Jul 2025 11:52:27 +0000 Received: by outflank-mailman (output) from mailman id 1046606.1416961; Thu, 17 Jul 2025 11:52:27 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1ucNA3-0001Tq-AB; Thu, 17 Jul 2025 11:52:27 +0000 Received: by outflank-mailman (input) for mailman id 1046606; Thu, 17 Jul 2025 11:52:26 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1ucNA2-0001TV-62 for xen-devel@lists.xenproject.org; Thu, 17 Jul 2025 11:52:26 +0000 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id 8007e854-6304-11f0-b894-0df219b8e170; Thu, 17 Jul 2025 13:52:23 +0200 (CEST) Received: from mail-wm1-f72.google.com (mail-wm1-f72.google.com [209.85.128.72]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-653-9EPw4LYkOxGXkgnnTsYDHQ-1; Thu, 17 Jul 2025 07:52:19 -0400 Received: by mail-wm1-f72.google.com with SMTP id 5b1f17b1804b1-455eda09c57so5373275e9.2 for ; Thu, 17 Jul 2025 04:52:19 -0700 (PDT) Received: from localhost (p200300d82f1f36000dc826ee9aa9fdc7.dip0.t-ipconnect.de. [2003:d8:2f1f:3600:dc8:26ee:9aa9:fdc7]) by smtp.gmail.com with UTF8SMTPSA id 5b1f17b1804b1-45634f8cc6esm20532295e9.26.2025.07.17.04.52.15 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Thu, 17 Jul 2025 04:52:16 -0700 (PDT) X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 8007e854-6304-11f0-b894-0df219b8e170 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1752753142; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=zIbOZw1x2GVZdwaR5iaeVsCQGMgTtSbEtHQ5YhAco0w=; b=Er+W8918I+6DeyQSgGRINmqVTLzcoiw0nJSNtrfqUACAYHeMQDnb35KN2n/+rl3REYZ7Bf KMuYjOXGYTfC3KkCCEm9+d1WLjN4Y2KOM6iQHk3+KkUB8Me3D19xvDRh3P2l8XXIHJHJle Lr0rzV1pGhLsY1aiUdQ54I4zWP7v1u4= X-MC-Unique: 9EPw4LYkOxGXkgnnTsYDHQ-1 X-Mimecast-MFC-AGG-ID: 9EPw4LYkOxGXkgnnTsYDHQ_1752753138 X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1752753138; x=1753357938; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=zIbOZw1x2GVZdwaR5iaeVsCQGMgTtSbEtHQ5YhAco0w=; b=oLanVW4Ru7/EPKY7MCb4gq4/+yt4Byehx08AQ3Gy0YzEC8PHrtnKuFpaFKOZ17BXj8 /GjQE3keO6EYgWj59t81Np4oQ0qQJ1shZNh5sYxRI49w0MRWTkAC0afpnBiX88tieyjX 1wsntOfXORL49Tb+RxkOiEEXNQw1wppNkrfVpX+DxaM/CYjE5V5yrPwjruD/jGBslCYn 5DZHD1Oulfx16Dn++1/kHeWIejicl6TMrI/3CG4pxbk3w+2eSvE79Uz5vG2JXPP4GWmS GzbtgXMQDNcwRlTJOI0o8eC/pNng4Wn80aH9VdYzLg1zVcNXAvdejmIxeCPq9ACz/WXo +mtw== X-Forwarded-Encrypted: i=1; AJvYcCXotnrsPZq1UQLGYiGDV6zXy3RKR5tIXZGjP+ueJvruwgGtm0d8AHAdJSFE/gOAHBRXyyoUYaMUats=@lists.xenproject.org X-Gm-Message-State: AOJu0YzNZhJRPljHeCL7BmorGcWCfSj8+3E6p24WGiWX7nBNuRtcvqcO lJ6bFR1asSXSNPIyheU9UFiO64JQwQclfCo55XtDaFvoMfE7WQthkSoDN+d4tma6jMSV8vYgdfK XDsBpmxylYsf5hWnLzzSBo1jl9gKwXPQv9j/GFxxrkcISoyDSxG0KBu+d3g/0PPLCimmt X-Gm-Gg: ASbGncsDvV+mwOcG+G76NEU81yY7hURbf4aHj358pHPiByhLtGVFH3nZ2+BiJvOWD68 JL4QM5GGmgtdCLZc2/Am+J3adkFDBv7no8u+fM92BEUjbImUbcJNQLbZamOIOj2H/ew7zpTpA60 39n3yHrhtVJfqSF4jAoF4xtcGj18ViM5VZvOolRM6NT6MAI5Srn7CLpdY4wWG6+2AQCmKYgzx8a bm1rriuCA14bS9/QthxglKqypNmotm5cz0ZDOoXLbTblqXX4HDETaP37I4GTEB2xKYNTuqni9XP z77QlWRrpJIKwUhE7voKM69CNh9ux61sIPLiAJxU2ea5Okp1/C19cyXtVhKdXEx2XmVruKT68zR gau7hh/h2LwWqPYjCBO8c6CQ= X-Received: by 2002:a05:600c:3e15:b0:439:86fb:7340 with SMTP id 5b1f17b1804b1-4562e39865dmr69602235e9.30.1752753137844; Thu, 17 Jul 2025 04:52:17 -0700 (PDT) X-Google-Smtp-Source: AGHT+IExNej71yyxo4EvVsey/pvqYl543s0okqsAtiv/nbzqOk+Nei3ADHbojZnjuyE7SwluE3g0Pw== X-Received: by 2002:a05:600c:3e15:b0:439:86fb:7340 with SMTP id 5b1f17b1804b1-4562e39865dmr69601365e9.30.1752753137339; Thu, 17 Jul 2025 04:52:17 -0700 (PDT) From: David Hildenbrand To: linux-kernel@vger.kernel.org Cc: linux-mm@kvack.org, xen-devel@lists.xenproject.org, linux-fsdevel@vger.kernel.org, nvdimm@lists.linux.dev, David Hildenbrand , Andrew Morton , Juergen Gross , Stefano Stabellini , Oleksandr Tyshchenko , Dan Williams , Matthew Wilcox , Jan Kara , Alexander Viro , Christian Brauner , Lorenzo Stoakes , "Liam R. Howlett" , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Zi Yan , Baolin Wang , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Jann Horn , Pedro Falcato , Hugh Dickins , Oscar Salvador , Lance Yang , Alistair Popple Subject: [PATCH v2 1/9] mm/huge_memory: move more common code into insert_pmd() Date: Thu, 17 Jul 2025 13:52:04 +0200 Message-ID: <20250717115212.1825089-2-david@redhat.com> X-Mailer: git-send-email 2.50.1 In-Reply-To: <20250717115212.1825089-1-david@redhat.com> References: <20250717115212.1825089-1-david@redhat.com> MIME-Version: 1.0 X-Mimecast-Spam-Score: 0 X-Mimecast-MFC-PROC-ID: EIvvH-GUHdxIYUW9xo6Zu1aCKZtg_UHT9g1S034au28_1752753138 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1752753174657116600 Content-Type: text/plain; charset="utf-8"; x-default="true" Let's clean it all further up. No functional change intended. Reviewed-by: Oscar Salvador Reviewed-by: Alistair Popple Signed-off-by: David Hildenbrand Reviewed-by: Lorenzo Stoakes Reviewed-by: Wei Yang --- mm/huge_memory.c | 72 ++++++++++++++++-------------------------------- 1 file changed, 24 insertions(+), 48 deletions(-) diff --git a/mm/huge_memory.c b/mm/huge_memory.c index fe17b0a157cda..1178760d2eda4 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -1390,15 +1390,25 @@ struct folio_or_pfn { bool is_folio; }; =20 -static int insert_pmd(struct vm_area_struct *vma, unsigned long addr, +static vm_fault_t insert_pmd(struct vm_area_struct *vma, unsigned long add= r, pmd_t *pmd, struct folio_or_pfn fop, pgprot_t prot, - bool write, pgtable_t pgtable) + bool write) { struct mm_struct *mm =3D vma->vm_mm; + pgtable_t pgtable =3D NULL; + spinlock_t *ptl; pmd_t entry; =20 - lockdep_assert_held(pmd_lockptr(mm, pmd)); + if (addr < vma->vm_start || addr >=3D vma->vm_end) + return VM_FAULT_SIGBUS; =20 + if (arch_needs_pgtable_deposit()) { + pgtable =3D pte_alloc_one(vma->vm_mm); + if (!pgtable) + return VM_FAULT_OOM; + } + + ptl =3D pmd_lock(mm, pmd); if (!pmd_none(*pmd)) { const unsigned long pfn =3D fop.is_folio ? folio_pfn(fop.folio) : fop.pfn; @@ -1406,15 +1416,14 @@ static int insert_pmd(struct vm_area_struct *vma, u= nsigned long addr, if (write) { if (pmd_pfn(*pmd) !=3D pfn) { WARN_ON_ONCE(!is_huge_zero_pmd(*pmd)); - return -EEXIST; + goto out_unlock; } entry =3D pmd_mkyoung(*pmd); entry =3D maybe_pmd_mkwrite(pmd_mkdirty(entry), vma); if (pmdp_set_access_flags(vma, addr, pmd, entry, 1)) update_mmu_cache_pmd(vma, addr, pmd); } - - return -EEXIST; + goto out_unlock; } =20 if (fop.is_folio) { @@ -1435,11 +1444,17 @@ static int insert_pmd(struct vm_area_struct *vma, u= nsigned long addr, if (pgtable) { pgtable_trans_huge_deposit(mm, pmd, pgtable); mm_inc_nr_ptes(mm); + pgtable =3D NULL; } =20 set_pmd_at(mm, addr, pmd, entry); update_mmu_cache_pmd(vma, addr, pmd); - return 0; + +out_unlock: + spin_unlock(ptl); + if (pgtable) + pte_free(mm, pgtable); + return VM_FAULT_NOPAGE; } =20 /** @@ -1461,9 +1476,6 @@ vm_fault_t vmf_insert_pfn_pmd(struct vm_fault *vmf, u= nsigned long pfn, struct folio_or_pfn fop =3D { .pfn =3D pfn, }; - pgtable_t pgtable =3D NULL; - spinlock_t *ptl; - int error; =20 /* * If we had pmd_special, we could avoid all these restrictions, @@ -1475,25 +1487,9 @@ vm_fault_t vmf_insert_pfn_pmd(struct vm_fault *vmf, = unsigned long pfn, (VM_PFNMAP|VM_MIXEDMAP)); BUG_ON((vma->vm_flags & VM_PFNMAP) && is_cow_mapping(vma->vm_flags)); =20 - if (addr < vma->vm_start || addr >=3D vma->vm_end) - return VM_FAULT_SIGBUS; - - if (arch_needs_pgtable_deposit()) { - pgtable =3D pte_alloc_one(vma->vm_mm); - if (!pgtable) - return VM_FAULT_OOM; - } - pfnmap_setup_cachemode_pfn(pfn, &pgprot); =20 - ptl =3D pmd_lock(vma->vm_mm, vmf->pmd); - error =3D insert_pmd(vma, addr, vmf->pmd, fop, pgprot, write, - pgtable); - spin_unlock(ptl); - if (error && pgtable) - pte_free(vma->vm_mm, pgtable); - - return VM_FAULT_NOPAGE; + return insert_pmd(vma, addr, vmf->pmd, fop, pgprot, write); } EXPORT_SYMBOL_GPL(vmf_insert_pfn_pmd); =20 @@ -1502,35 +1498,15 @@ vm_fault_t vmf_insert_folio_pmd(struct vm_fault *vm= f, struct folio *folio, { struct vm_area_struct *vma =3D vmf->vma; unsigned long addr =3D vmf->address & PMD_MASK; - struct mm_struct *mm =3D vma->vm_mm; struct folio_or_pfn fop =3D { .folio =3D folio, .is_folio =3D true, }; - spinlock_t *ptl; - pgtable_t pgtable =3D NULL; - int error; - - if (addr < vma->vm_start || addr >=3D vma->vm_end) - return VM_FAULT_SIGBUS; =20 if (WARN_ON_ONCE(folio_order(folio) !=3D PMD_ORDER)) return VM_FAULT_SIGBUS; =20 - if (arch_needs_pgtable_deposit()) { - pgtable =3D pte_alloc_one(vma->vm_mm); - if (!pgtable) - return VM_FAULT_OOM; - } - - ptl =3D pmd_lock(mm, vmf->pmd); - error =3D insert_pmd(vma, addr, vmf->pmd, fop, vma->vm_page_prot, - write, pgtable); - spin_unlock(ptl); - if (error && pgtable) - pte_free(mm, pgtable); - - return VM_FAULT_NOPAGE; + return insert_pmd(vma, addr, vmf->pmd, fop, vma->vm_page_prot, write); } EXPORT_SYMBOL_GPL(vmf_insert_folio_pmd); =20 --=20 2.50.1 From nobody Mon Oct 6 20:56:44 2025 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass(p=quarantine dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1752753169; cv=none; d=zohomail.com; s=zohoarc; b=JO6KanEcHehhc78/Fb/BTQGnRThNoXWnrv9CmGNQmoLAIQnbOsMuQTtOdPphtGhGngGlRwwyt3TxIp7Qmot/oBDSQWkZz2vVPnMPjQZtbuRvXrTb3UlWdUtQqaTVOtxVpZeAzQ8x9XfRzP9nY70EhZ7DXmv+b6FXnAwjp3rQ4zM= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1752753169; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=8U/VSSfKctO+2Q0R03xR9syW7Y0P/UCdJOxdygVgan0=; b=XNqzRGs0pDh9g1h40fVlWNcSdJqbGNI8kooDgj+L5mO1PNngm2DuJ1hkbfVuLD9vNNyhnGQoeRyETZV0h9bRLnpAwRVG6UHVGp9bQ1Is27yt53TenBMkGG/HJOtRKjQr4Jahw7XB9c7mKAXCPPmBC6IThKyFWzo3WC8GI46Zq3k= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1752753169763563.646081490344; Thu, 17 Jul 2025 04:52:49 -0700 (PDT) Received: from list by lists.xenproject.org with outflank-mailman.1046607.1416971 (Exim 4.92) (envelope-from ) id 1ucNA4-0001il-MW; Thu, 17 Jul 2025 11:52:28 +0000 Received: by outflank-mailman (output) from mailman id 1046607.1416971; Thu, 17 Jul 2025 11:52:28 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1ucNA4-0001iZ-JL; Thu, 17 Jul 2025 11:52:28 +0000 Received: by outflank-mailman (input) for mailman id 1046607; Thu, 17 Jul 2025 11:52:27 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1ucNA3-0001Ft-2j for xen-devel@lists.xenproject.org; Thu, 17 Jul 2025 11:52:27 +0000 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id 81850709-6304-11f0-a319-13f23c93f187; Thu, 17 Jul 2025 13:52:26 +0200 (CEST) Received: from mail-wm1-f69.google.com (mail-wm1-f69.google.com [209.85.128.69]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-326-v7c1l8Q7NRCjjUYTXVLA7g-1; Thu, 17 Jul 2025 07:52:21 -0400 Received: by mail-wm1-f69.google.com with SMTP id 5b1f17b1804b1-43e9b0fd00cso5757065e9.0 for ; Thu, 17 Jul 2025 04:52:21 -0700 (PDT) Received: from localhost (p200300d82f1f36000dc826ee9aa9fdc7.dip0.t-ipconnect.de. [2003:d8:2f1f:3600:dc8:26ee:9aa9:fdc7]) by smtp.gmail.com with UTF8SMTPSA id 5b1f17b1804b1-45634f82f29sm20129705e9.23.2025.07.17.04.52.18 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Thu, 17 Jul 2025 04:52:19 -0700 (PDT) X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 81850709-6304-11f0-a319-13f23c93f187 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1752753145; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=8U/VSSfKctO+2Q0R03xR9syW7Y0P/UCdJOxdygVgan0=; b=hhHUd2xh8etESKHce97p4E7wO2CJZYs60r+Y9VRw9U4fk+PgL2pR0Aq66VOKD3NuLQvji1 KGvWlaBFCOdmMAqvQCeNxMbvh1ToEZDdnjAIp9k09gHmwtenqryeDzFpPxrjly9Z/El4M0 Ndcmy2lb2eFXoBfUNjEKzvMptuw23DE= X-MC-Unique: v7c1l8Q7NRCjjUYTXVLA7g-1 X-Mimecast-MFC-AGG-ID: v7c1l8Q7NRCjjUYTXVLA7g_1752753140 X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1752753140; x=1753357940; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=8U/VSSfKctO+2Q0R03xR9syW7Y0P/UCdJOxdygVgan0=; b=SFPKy8z3Nz2js3k5ipncw680uaEiEtbTA0J2e5mnd7oA1S1hYoIUd+21JGuH17j7Ls mRLuMsUrqhPHCS7fDnQSc8EknItNW/hkCMKdQOlQKWWHji0GbjPKot93d0a1i4zstnB1 ThmiU74xvvS2nUT91pIKmtmULwY38CVXPjg6O5ehenelelPYARDLqeClnvHKoFJO4vCz j1beNDEaq30tSNt+dWmWPcfRDwVoPS5+EnGAk0HA7t41OoTq/LWz8lM2/aJQEUVtfTK7 RUfxZu4zVS6h1llo/zIuZxcJQRd0aLF6cOkVDD7j4VT4p+kTxImfY+BeTe4SW7we9NLG Ly8w== X-Forwarded-Encrypted: i=1; AJvYcCW3gLDeO4Ap/oMrHeYwJDM0EWgtBisdZmr7rTIu+KfBRgyFMQdLHEEPcbtrJyCiAMTFa2CE57Q/f70=@lists.xenproject.org X-Gm-Message-State: AOJu0Yw5q1KfpfgDLC1kMteD0vqhK72XFdIa1vNV5F7KqBFkxgLYCxHP OMp+YYu0rpSt/1IRjFqJ7+xKcmbLW37oRIIQaeo+WrLo0b7mb+8x4kJQX7Xxwbvb67DMu4SRSo3 aldIC/2LyJFm71+keo2zg6d7FFXydg6gavqogKM6al+4GOPR6EElfIyxKWx1zepokCUF+ X-Gm-Gg: ASbGncsOMPQSZ7bFDCtQ36ilhs+DkePi4/WjLt8QL0nSeXR6qRmlyCNc4Ccy245D7IL UHcTldglYykO+4pJoz11uym36+1YOhFd6T3IBB2D7/VgjCqLt64KVDQg+6X4gdBpTyDgNo/Z22K FezjvdUD7yI6Jrr7euRvmFbArreVIhCYUUBvLGR0vZzwu+TpcoxfJ5eXKpgEqj6DGRCd91O45BV Ne2q/KA2KOskAM/WszL0SJW3g2k9MmnMuuvtCHvekcF92j4XWSG6HoI1A1fm485RRpEnZExS0Yt pT1n+sTCM03Fk/UkW7pzhUs8LYV5g2cMF6Et8jykSbOLw/0+RlcgUqUi4RGgfBCA6VSxjMJLHNK cLoQoNVdzMbU69xy+k2Vbh8s= X-Received: by 2002:a05:600c:c0ce:b0:450:c9e3:91fe with SMTP id 5b1f17b1804b1-456346e2767mr22597955e9.0.1752753140276; Thu, 17 Jul 2025 04:52:20 -0700 (PDT) X-Google-Smtp-Source: AGHT+IHRhI54TwDUdDBNIeRKh7AVS/hfT6FB+wtUWYU/lc2VhuDIhVT1QHk+Jezb55nN35RC/TrzYQ== X-Received: by 2002:a05:600c:c0ce:b0:450:c9e3:91fe with SMTP id 5b1f17b1804b1-456346e2767mr22597745e9.0.1752753139781; Thu, 17 Jul 2025 04:52:19 -0700 (PDT) From: David Hildenbrand To: linux-kernel@vger.kernel.org Cc: linux-mm@kvack.org, xen-devel@lists.xenproject.org, linux-fsdevel@vger.kernel.org, nvdimm@lists.linux.dev, David Hildenbrand , Andrew Morton , Juergen Gross , Stefano Stabellini , Oleksandr Tyshchenko , Dan Williams , Matthew Wilcox , Jan Kara , Alexander Viro , Christian Brauner , Lorenzo Stoakes , "Liam R. Howlett" , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Zi Yan , Baolin Wang , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Jann Horn , Pedro Falcato , Hugh Dickins , Oscar Salvador , Lance Yang , Alistair Popple Subject: [PATCH v2 2/9] mm/huge_memory: move more common code into insert_pud() Date: Thu, 17 Jul 2025 13:52:05 +0200 Message-ID: <20250717115212.1825089-3-david@redhat.com> X-Mailer: git-send-email 2.50.1 In-Reply-To: <20250717115212.1825089-1-david@redhat.com> References: <20250717115212.1825089-1-david@redhat.com> MIME-Version: 1.0 X-Mimecast-Spam-Score: 0 X-Mimecast-MFC-PROC-ID: fnLFrf0_IOW8Cyiyu9eSCUjiuSTbteqUatPOmatIhfk_1752753140 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1752753170412116600 Content-Type: text/plain; charset="utf-8"; x-default="true" Let's clean it all further up. No functional change intended. Reviewed-by: Oscar Salvador Reviewed-by: Alistair Popple Signed-off-by: David Hildenbrand Reviewed-by: Lorenzo Stoakes Reviewed-by: Wei Yang --- mm/huge_memory.c | 36 +++++++++++++----------------------- 1 file changed, 13 insertions(+), 23 deletions(-) diff --git a/mm/huge_memory.c b/mm/huge_memory.c index 1178760d2eda4..849feacaf8064 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -1518,25 +1518,30 @@ static pud_t maybe_pud_mkwrite(pud_t pud, struct vm= _area_struct *vma) return pud; } =20 -static void insert_pud(struct vm_area_struct *vma, unsigned long addr, +static vm_fault_t insert_pud(struct vm_area_struct *vma, unsigned long add= r, pud_t *pud, struct folio_or_pfn fop, pgprot_t prot, bool write) { struct mm_struct *mm =3D vma->vm_mm; + spinlock_t *ptl; pud_t entry; =20 + if (addr < vma->vm_start || addr >=3D vma->vm_end) + return VM_FAULT_SIGBUS; + + ptl =3D pud_lock(mm, pud); if (!pud_none(*pud)) { const unsigned long pfn =3D fop.is_folio ? folio_pfn(fop.folio) : fop.pfn; =20 if (write) { if (WARN_ON_ONCE(pud_pfn(*pud) !=3D pfn)) - return; + goto out_unlock; entry =3D pud_mkyoung(*pud); entry =3D maybe_pud_mkwrite(pud_mkdirty(entry), vma); if (pudp_set_access_flags(vma, addr, pud, entry, 1)) update_mmu_cache_pud(vma, addr, pud); } - return; + goto out_unlock; } =20 if (fop.is_folio) { @@ -1555,6 +1560,9 @@ static void insert_pud(struct vm_area_struct *vma, un= signed long addr, } set_pud_at(mm, addr, pud, entry); update_mmu_cache_pud(vma, addr, pud); +out_unlock: + spin_unlock(ptl); + return VM_FAULT_NOPAGE; } =20 /** @@ -1576,7 +1584,6 @@ vm_fault_t vmf_insert_pfn_pud(struct vm_fault *vmf, u= nsigned long pfn, struct folio_or_pfn fop =3D { .pfn =3D pfn, }; - spinlock_t *ptl; =20 /* * If we had pud_special, we could avoid all these restrictions, @@ -1588,16 +1595,9 @@ vm_fault_t vmf_insert_pfn_pud(struct vm_fault *vmf, = unsigned long pfn, (VM_PFNMAP|VM_MIXEDMAP)); BUG_ON((vma->vm_flags & VM_PFNMAP) && is_cow_mapping(vma->vm_flags)); =20 - if (addr < vma->vm_start || addr >=3D vma->vm_end) - return VM_FAULT_SIGBUS; - pfnmap_setup_cachemode_pfn(pfn, &pgprot); =20 - ptl =3D pud_lock(vma->vm_mm, vmf->pud); - insert_pud(vma, addr, vmf->pud, fop, pgprot, write); - spin_unlock(ptl); - - return VM_FAULT_NOPAGE; + return insert_pud(vma, addr, vmf->pud, fop, pgprot, write); } EXPORT_SYMBOL_GPL(vmf_insert_pfn_pud); =20 @@ -1614,25 +1614,15 @@ vm_fault_t vmf_insert_folio_pud(struct vm_fault *vm= f, struct folio *folio, { struct vm_area_struct *vma =3D vmf->vma; unsigned long addr =3D vmf->address & PUD_MASK; - pud_t *pud =3D vmf->pud; - struct mm_struct *mm =3D vma->vm_mm; struct folio_or_pfn fop =3D { .folio =3D folio, .is_folio =3D true, }; - spinlock_t *ptl; - - if (addr < vma->vm_start || addr >=3D vma->vm_end) - return VM_FAULT_SIGBUS; =20 if (WARN_ON_ONCE(folio_order(folio) !=3D PUD_ORDER)) return VM_FAULT_SIGBUS; =20 - ptl =3D pud_lock(mm, pud); - insert_pud(vma, addr, vmf->pud, fop, vma->vm_page_prot, write); - spin_unlock(ptl); - - return VM_FAULT_NOPAGE; + return insert_pud(vma, addr, vmf->pud, fop, vma->vm_page_prot, write); } EXPORT_SYMBOL_GPL(vmf_insert_folio_pud); #endif /* CONFIG_HAVE_ARCH_TRANSPARENT_HUGEPAGE_PUD */ --=20 2.50.1 From nobody Mon Oct 6 20:56:44 2025 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass(p=quarantine dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1752753168; cv=none; d=zohomail.com; s=zohoarc; b=FEE/tWH4aFbw5S60tO9c7h6OUHrielsFqJwYhB/lG5t8HlNJrNx80rx0tAYd3L38T3FCuVhFRE1/q/sY9O9Q20i5C9GZQSAs9dg2y+mnFGmald//Ol4BYgQ9dzte69nMgUTFJxwB9DdFGGOW1DVBfSJbejMpAnJtEnUHx2pivjc= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1752753168; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=oAAYYXy+5qwpASBfNrlxr8guN8G2i706wjpGFlltV74=; b=Y0gLFLTIjhdoi0r58g/etbjBL0o9LQmCKJKC9APcHDhII+saNtFen2Nm72hZVnqxCbFsq+6k8cSDBeYx+/oKfuoA2UM0/Sj2nOI/s01gKmhlgSbjdO1hLOrVcWEDXTypstt37bIX2d1m8TiTvLmizGj1J4J/bUuc9EwJTqf6mgY= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1752753168872421.11226540794917; Thu, 17 Jul 2025 04:52:48 -0700 (PDT) Received: from list by lists.xenproject.org with outflank-mailman.1046608.1416981 (Exim 4.92) (envelope-from ) id 1ucNA6-0001yA-Uz; Thu, 17 Jul 2025 11:52:30 +0000 Received: by outflank-mailman (output) from mailman id 1046608.1416981; Thu, 17 Jul 2025 11:52:30 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1ucNA6-0001y1-RU; Thu, 17 Jul 2025 11:52:30 +0000 Received: by outflank-mailman (input) for mailman id 1046608; Thu, 17 Jul 2025 11:52:30 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1ucNA6-0001TV-8b for xen-devel@lists.xenproject.org; Thu, 17 Jul 2025 11:52:30 +0000 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id 831bfff8-6304-11f0-b894-0df219b8e170; Thu, 17 Jul 2025 13:52:28 +0200 (CEST) Received: from mail-wr1-f72.google.com (mail-wr1-f72.google.com [209.85.221.72]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-669-d2QeHBjPMR-28s-cTvWR9A-1; Thu, 17 Jul 2025 07:52:23 -0400 Received: by mail-wr1-f72.google.com with SMTP id ffacd0b85a97d-3a5281ba3a4so517464f8f.0 for ; Thu, 17 Jul 2025 04:52:23 -0700 (PDT) Received: from localhost (p200300d82f1f36000dc826ee9aa9fdc7.dip0.t-ipconnect.de. [2003:d8:2f1f:3600:dc8:26ee:9aa9:fdc7]) by smtp.gmail.com with UTF8SMTPSA id 5b1f17b1804b1-45634f4c546sm20298345e9.7.2025.07.17.04.52.20 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Thu, 17 Jul 2025 04:52:21 -0700 (PDT) X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 831bfff8-6304-11f0-b894-0df219b8e170 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1752753147; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=oAAYYXy+5qwpASBfNrlxr8guN8G2i706wjpGFlltV74=; b=PowJqas7tpfMmsMJaVxOmZnEmQPtsKHWnYCHCm4aEYoT9Aw2OaefEcltD1hxTxxQkn6Bov g0O403vR4QPDWg7P0Ucrsj3KaP0PQB4jBOQVAGTZ+umBd1UD14S7oGsQBeZiXKnQLBjNxc phvY48Te2VAiSA9cidoTnvC96mzq6hQ= X-MC-Unique: d2QeHBjPMR-28s-cTvWR9A-1 X-Mimecast-MFC-AGG-ID: d2QeHBjPMR-28s-cTvWR9A_1752753143 X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1752753143; x=1753357943; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=oAAYYXy+5qwpASBfNrlxr8guN8G2i706wjpGFlltV74=; b=E8PKKuhKOtVU6zgTXPv1WNnHGYONpkFc3hl3nh4W4ZyKv9gUJvge11eHO4iZ3Fb5ED ReMU6R6km9ZzkHYnyIoCHw6tvucy7W06ZV0ryj0Hz/vtNBfMHftzD7kthD6upTt7ZvbJ 6jHp4uAENMr49WKk/RM2Uj02zfJFD+cf5rBclSTSnKJwNJMrCCBIvnU/bTBetUNFrOMt InyFa2+JPVrxAId3C9esoFyR4BgeENV1lElQ2LwWAa5FLdVK4hduHRIjCXVdrgxTPwDS IQVfLQsTOTls8nvWIYl9+Ej6hE94Lq7q0vtRXeBmXsJPtRp23zr2KDdwUQNxsc7oGvoP LDoA== X-Forwarded-Encrypted: i=1; AJvYcCU3FOGkgh6J9GlliGBUUgMrK2thSA8TxoekLbhnCcweJIslnf0MLJEpuleWyW5EaNp3OT1vw815Rj4=@lists.xenproject.org X-Gm-Message-State: AOJu0YxoMt7grY09Wq5d1bDSF6YPf7ujlBp7YGXx29j4nUm2U1xHr0KX UnD3S3vU7N0Eczje28R10kfkzmHJmAOvoXrSMZRYcemP2iEu/BAHqCjxz4cIEci680HBQqwWVXS tBRLjKSqLRTythvmAx6lOoUeH9KUwubA45Upw6GLibKKO6Ef3fv39udAabH1lNK2GO4i1 X-Gm-Gg: ASbGncsJQ0LeVTLG83lYs6QSNBtIdZbMM/XamGrrv04vbQEUDKAHlCKkDgHqcFEK6zI hRDeBgnOBGWNoKD3YnhlgcesYh+T7/mZbOx9wVBUnha2RWE7Ady8puBmDDpgdrrkulIcS+6qqZA vCLa4dn0KmvKdnOKkBreQH0Av8QjND8KeQI27JEdrryq/5pA6kCWxLh6Dmj0nsFd5QYqZk/f61r Px0mx3Wx882JOSi2hbz1p42vAeovb5O1H8U6Pn+AXuR7t+6xpZkEiCsq271uKF4r1alRj36KKvZ TxdH6Pk1va5QCzTdSJWMMmtzlH+oSjbypMX3n30x/PMy5pWLYBOF1xcAfMQFL+/FW7R+IaTqv+z Df+vxCTC+oytRIsFLkuruBHA= X-Received: by 2002:a05:6000:23c8:b0:3a6:d95c:5e8 with SMTP id ffacd0b85a97d-3b60dd7aac2mr3879076f8f.35.1752753142626; Thu, 17 Jul 2025 04:52:22 -0700 (PDT) X-Google-Smtp-Source: AGHT+IGLqq0HSziphIF4gY7jkSa/7zMDAewp7/5b6DkIKpjYL1yIcxKWQwPQ10nRUjdRU/WgBdtqyw== X-Received: by 2002:a05:6000:23c8:b0:3a6:d95c:5e8 with SMTP id ffacd0b85a97d-3b60dd7aac2mr3879028f8f.35.1752753142142; Thu, 17 Jul 2025 04:52:22 -0700 (PDT) From: David Hildenbrand To: linux-kernel@vger.kernel.org Cc: linux-mm@kvack.org, xen-devel@lists.xenproject.org, linux-fsdevel@vger.kernel.org, nvdimm@lists.linux.dev, David Hildenbrand , Andrew Morton , Juergen Gross , Stefano Stabellini , Oleksandr Tyshchenko , Dan Williams , Matthew Wilcox , Jan Kara , Alexander Viro , Christian Brauner , Lorenzo Stoakes , "Liam R. Howlett" , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Zi Yan , Baolin Wang , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Jann Horn , Pedro Falcato , Hugh Dickins , Oscar Salvador , Lance Yang Subject: [PATCH v2 3/9] mm/huge_memory: support huge zero folio in vmf_insert_folio_pmd() Date: Thu, 17 Jul 2025 13:52:06 +0200 Message-ID: <20250717115212.1825089-4-david@redhat.com> X-Mailer: git-send-email 2.50.1 In-Reply-To: <20250717115212.1825089-1-david@redhat.com> References: <20250717115212.1825089-1-david@redhat.com> MIME-Version: 1.0 X-Mimecast-Spam-Score: 0 X-Mimecast-MFC-PROC-ID: Y5TXmE1qdGBfuo4UYH0z0C4uxuZTlxmI_yGfLIZFYqA_1752753143 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1752753170328116600 Content-Type: text/plain; charset="utf-8"; x-default="true" Just like we do for vmf_insert_page_mkwrite() -> ... -> insert_page_into_pte_locked() with the shared zeropage, support the huge zero folio in vmf_insert_folio_pmd(). When (un)mapping the huge zero folio in page tables, we neither adjust the refcount nor the mapcount, just like for the shared zeropage. For now, the huge zero folio is not marked as special yet, although vm_normal_page_pmd() really wants to treat it as special. We'll change that next. Reviewed-by: Oscar Salvador Signed-off-by: David Hildenbrand Reviewed-by: Lorenzo Stoakes Reviewed-by: Wei Yang --- mm/huge_memory.c | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/mm/huge_memory.c b/mm/huge_memory.c index 849feacaf8064..db08c37b87077 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -1429,9 +1429,11 @@ static vm_fault_t insert_pmd(struct vm_area_struct *= vma, unsigned long addr, if (fop.is_folio) { entry =3D folio_mk_pmd(fop.folio, vma->vm_page_prot); =20 - folio_get(fop.folio); - folio_add_file_rmap_pmd(fop.folio, &fop.folio->page, vma); - add_mm_counter(mm, mm_counter_file(fop.folio), HPAGE_PMD_NR); + if (!is_huge_zero_folio(fop.folio)) { + folio_get(fop.folio); + folio_add_file_rmap_pmd(fop.folio, &fop.folio->page, vma); + add_mm_counter(mm, mm_counter_file(fop.folio), HPAGE_PMD_NR); + } } else { entry =3D pmd_mkhuge(pfn_pmd(fop.pfn, prot)); entry =3D pmd_mkspecial(entry); --=20 2.50.1 From nobody Mon Oct 6 20:56:44 2025 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass(p=quarantine dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1752753171; cv=none; d=zohomail.com; s=zohoarc; b=K0tPISrd/aaPzXeEsn8/t1/nBxOWC6srPWMow22htQvH/25PPhzoYa4xJh4XIzgJqFTcaL4Ngf+yC8iaEX4yBafphacbf71dAuclCQdAz5uILXSbTF2/Jc4D5+ApLmk81qlMAHA4C2c2kojuyNWBCcUnNNJGr3dqi2ZxJXsoQjY= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1752753171; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=WawR8xzOLp+WGO3JwZK8yJOVf+K8h9KX2DcE6xt9pRU=; b=RSKfDbylF/bs6tDXtij0Wa7w5x8y6sxcdR4xicojs/swc7m9Hg1w6izW1kvDwbcBlS/Gf79vyl5fxwTOgyklf0/BoxOFLPHI8TiOUJtWL8FNo/nRR85agqyvVlPfSZXADYclw8A0y/qtiWwyBqQZH/jrsoQMGS7GZn4dsisiWPc= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1752753171092305.0706266640873; Thu, 17 Jul 2025 04:52:51 -0700 (PDT) Received: from list by lists.xenproject.org with outflank-mailman.1046609.1416991 (Exim 4.92) (envelope-from ) id 1ucNA8-0002D1-5Q; Thu, 17 Jul 2025 11:52:32 +0000 Received: by outflank-mailman (output) from mailman id 1046609.1416991; Thu, 17 Jul 2025 11:52:32 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1ucNA8-0002Cs-2c; Thu, 17 Jul 2025 11:52:32 +0000 Received: by outflank-mailman (input) for mailman id 1046609; Thu, 17 Jul 2025 11:52:31 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1ucNA7-0001TV-8e for xen-devel@lists.xenproject.org; Thu, 17 Jul 2025 11:52:31 +0000 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id 83145ad0-6304-11f0-b894-0df219b8e170; Thu, 17 Jul 2025 13:52:29 +0200 (CEST) Received: from mail-wr1-f70.google.com (mail-wr1-f70.google.com [209.85.221.70]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-111-kjiagve7N_6QEj5t06ikXg-1; Thu, 17 Jul 2025 07:52:26 -0400 Received: by mail-wr1-f70.google.com with SMTP id ffacd0b85a97d-3a4f6ff23ccso739374f8f.2 for ; Thu, 17 Jul 2025 04:52:26 -0700 (PDT) Received: from localhost (p200300d82f1f36000dc826ee9aa9fdc7.dip0.t-ipconnect.de. [2003:d8:2f1f:3600:dc8:26ee:9aa9:fdc7]) by smtp.gmail.com with UTF8SMTPSA id 5b1f17b1804b1-45634f9d599sm19697745e9.33.2025.07.17.04.52.23 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Thu, 17 Jul 2025 04:52:24 -0700 (PDT) X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 83145ad0-6304-11f0-b894-0df219b8e170 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1752753147; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=WawR8xzOLp+WGO3JwZK8yJOVf+K8h9KX2DcE6xt9pRU=; b=hovra43iMTyf0y8y9ueFmfIlzTlKFSW85ZKj24pk26m2sk/zCuGwdFc1b9n+0nwL1LGXPv YRqpIjXYYzNu5384WCA3GLgRoiuY3IEAiPQTcJLRu0Y+C0krfTbk7RHKcjCApe9uh/Hk/a GbduXrK3iB9/sGKyvP/BdRv1AkNAEqU= X-MC-Unique: kjiagve7N_6QEj5t06ikXg-1 X-Mimecast-MFC-AGG-ID: kjiagve7N_6QEj5t06ikXg_1752753145 X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1752753145; x=1753357945; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=WawR8xzOLp+WGO3JwZK8yJOVf+K8h9KX2DcE6xt9pRU=; b=WuXmcp7kg0umeDQBlK36LSHc166ZvB2rHoG/oOkM/ktQaeE50tBIk72rWCcM6YXcPw DtVkwtOVt+S0cn7qVA4G19M+yc4ZGjYvfAcwWtXhkmEKZ0sBin0e+yVyu3Sm8ZVidTE3 k1PBwHBrTzV9JJWuTmKlp5Sn1BRNX009n92KwW49V4Ig2pbfMmpbOCVPGHAUIbn/u+1A 2C70FgwWZQKUmZHB/BKHBGh3vaKe9UvKvRxNQopOAU81oaWw786C5CorViDJ+8O67J2w BEL4rao77Fj2OcZ7iUUAfcoLm1fcknY4an+80wSLsxIkOsjpBO/rb57TF8IX4pfearQ+ j5aw== X-Forwarded-Encrypted: i=1; AJvYcCUoQOCK2GM+owTWD5DUFPim7QMTe1Yrse+nouBCzO1WxfGhGovL9E7u4RQDlV4vI0XGPDMvJAwfkVQ=@lists.xenproject.org X-Gm-Message-State: AOJu0YygUDHNqpaTATXamXmGGT2/MbedWjsnrOAJ+nscf8x9zmLH9BN5 UKbgj1qxBqQWu07mArMpsdc023nUslG6n8uIDG7m02DkyVk4YBXreYxGDFMDXOTo99XZdpJnTFb LTJyORV9/EmeGRBW4poQVraQf/FjY0ZOP4y0IM13afa3fWJnQdxsIzvEDwHZhOy4j77oS X-Gm-Gg: ASbGnct0ASP8utII9vhaFtc6pQY7A8uDVPy23qGHUyF7iVVweFIza+Rfr2GfYDdWTIC /2g5htGyH+DdhTHNFGkDjXweI1GHsK4WeM84cJfJtPqakohxswiuWubxsNEoe+Wm21MSIeMgGUi cpdsY4mOB6dOxjwLqg5CEJ7gahebP3V3WiI5YYHa5sW7RqxghqnG7veHOjtbVOc6OdS6aUa0DDU uRCMVQETDl0sXE0FWHzwBDIE6Z+qvQe8Mjb91i0yEzaCu15Uh1c8MBeiZ4FaRjKMOlbNycEFdIV mDic/sfXcmm7wEVKM4mz07CCnOdW9dzL25CVRQwUt42z2axuWoaQHJ5ieP5zOP2Ik79G3I5sc2B vvrLjY+fbhD+WPI92ZdAV5Pw= X-Received: by 2002:a05:6000:4b02:b0:3a5:52d4:5b39 with SMTP id ffacd0b85a97d-3b60dd4ab27mr4713882f8f.8.1752753145080; Thu, 17 Jul 2025 04:52:25 -0700 (PDT) X-Google-Smtp-Source: AGHT+IFDhmg9Ls7QQA6yWIIBzDvc6abI92P/fYxcVRDFmyoxlxNBqWLFwcd4nrVInyOX88jAc9Q7DA== X-Received: by 2002:a05:6000:4b02:b0:3a5:52d4:5b39 with SMTP id ffacd0b85a97d-3b60dd4ab27mr4713858f8f.8.1752753144606; Thu, 17 Jul 2025 04:52:24 -0700 (PDT) From: David Hildenbrand To: linux-kernel@vger.kernel.org Cc: linux-mm@kvack.org, xen-devel@lists.xenproject.org, linux-fsdevel@vger.kernel.org, nvdimm@lists.linux.dev, David Hildenbrand , Andrew Morton , Juergen Gross , Stefano Stabellini , Oleksandr Tyshchenko , Dan Williams , Matthew Wilcox , Jan Kara , Alexander Viro , Christian Brauner , Lorenzo Stoakes , "Liam R. Howlett" , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Zi Yan , Baolin Wang , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Jann Horn , Pedro Falcato , Hugh Dickins , Oscar Salvador , Lance Yang , Alistair Popple Subject: [PATCH v2 4/9] fs/dax: use vmf_insert_folio_pmd() to insert the huge zero folio Date: Thu, 17 Jul 2025 13:52:07 +0200 Message-ID: <20250717115212.1825089-5-david@redhat.com> X-Mailer: git-send-email 2.50.1 In-Reply-To: <20250717115212.1825089-1-david@redhat.com> References: <20250717115212.1825089-1-david@redhat.com> MIME-Version: 1.0 X-Mimecast-Spam-Score: 0 X-Mimecast-MFC-PROC-ID: auIxs0mUXqhVbQcRUmwxmDAXCs5ylYdjA84hB6aTJH0_1752753145 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1752753172466116600 Content-Type: text/plain; charset="utf-8"; x-default="true" Let's convert to vmf_insert_folio_pmd(). There is a theoretical change in behavior: in the unlikely case there is already something mapped, we'll now still call trace_dax_pmd_load_hole() and return VM_FAULT_NOPAGE. Previously, we would have returned VM_FAULT_FALLBACK, and the caller would have zapped the PMD to try a PTE fault. However, that behavior was different to other PTE+PMD faults, when there would already be something mapped, and it's not even clear if it could be triggered. Assuming the huge zero folio is already mapped, all good, no need to fallback to PTEs. Assuming there is already a leaf page table ... the behavior would be just like when trying to insert a PMD mapping a folio through dax_fault_iter()->vmf_insert_folio_pmd(). Assuming there is already something else mapped as PMD? It sounds like a BUG, and the behavior would be just like when trying to insert a PMD mapping a folio through dax_fault_iter()->vmf_insert_folio_pmd(). So, it sounds reasonable to not handle huge zero folios differently to inserting PMDs mapping folios when there already is something mapped. Reviewed-by: Alistair Popple Signed-off-by: David Hildenbrand Reviewed-by: Lorenzo Stoakes --- fs/dax.c | 47 ++++++++++------------------------------------- 1 file changed, 10 insertions(+), 37 deletions(-) diff --git a/fs/dax.c b/fs/dax.c index 4229513806bea..ae90706674a3f 100644 --- a/fs/dax.c +++ b/fs/dax.c @@ -1375,51 +1375,24 @@ static vm_fault_t dax_pmd_load_hole(struct xa_state= *xas, struct vm_fault *vmf, const struct iomap_iter *iter, void **entry) { struct address_space *mapping =3D vmf->vma->vm_file->f_mapping; - unsigned long pmd_addr =3D vmf->address & PMD_MASK; - struct vm_area_struct *vma =3D vmf->vma; struct inode *inode =3D mapping->host; - pgtable_t pgtable =3D NULL; struct folio *zero_folio; - spinlock_t *ptl; - pmd_t pmd_entry; - unsigned long pfn; + vm_fault_t ret; =20 zero_folio =3D mm_get_huge_zero_folio(vmf->vma->vm_mm); =20 - if (unlikely(!zero_folio)) - goto fallback; - - pfn =3D page_to_pfn(&zero_folio->page); - *entry =3D dax_insert_entry(xas, vmf, iter, *entry, pfn, - DAX_PMD | DAX_ZERO_PAGE); - - if (arch_needs_pgtable_deposit()) { - pgtable =3D pte_alloc_one(vma->vm_mm); - if (!pgtable) - return VM_FAULT_OOM; - } - - ptl =3D pmd_lock(vmf->vma->vm_mm, vmf->pmd); - if (!pmd_none(*(vmf->pmd))) { - spin_unlock(ptl); - goto fallback; + if (unlikely(!zero_folio)) { + trace_dax_pmd_load_hole_fallback(inode, vmf, zero_folio, *entry); + return VM_FAULT_FALLBACK; } =20 - if (pgtable) { - pgtable_trans_huge_deposit(vma->vm_mm, vmf->pmd, pgtable); - mm_inc_nr_ptes(vma->vm_mm); - } - pmd_entry =3D folio_mk_pmd(zero_folio, vmf->vma->vm_page_prot); - set_pmd_at(vmf->vma->vm_mm, pmd_addr, vmf->pmd, pmd_entry); - spin_unlock(ptl); - trace_dax_pmd_load_hole(inode, vmf, zero_folio, *entry); - return VM_FAULT_NOPAGE; + *entry =3D dax_insert_entry(xas, vmf, iter, *entry, folio_pfn(zero_folio), + DAX_PMD | DAX_ZERO_PAGE); =20 -fallback: - if (pgtable) - pte_free(vma->vm_mm, pgtable); - trace_dax_pmd_load_hole_fallback(inode, vmf, zero_folio, *entry); - return VM_FAULT_FALLBACK; + ret =3D vmf_insert_folio_pmd(vmf, zero_folio, false); + if (ret =3D=3D VM_FAULT_NOPAGE) + trace_dax_pmd_load_hole(inode, vmf, zero_folio, *entry); + return ret; } #else static vm_fault_t dax_pmd_load_hole(struct xa_state *xas, struct vm_fault = *vmf, --=20 2.50.1 From nobody Mon Oct 6 20:56:44 2025 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass(p=quarantine dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1752753173; cv=none; d=zohomail.com; s=zohoarc; b=WmU9V3lIptb+RiGi28FWhgsJh5VKh3w3Ub/EwZuRuCHwkUJr1OguDjHmtOcGJWqisDjo4ttqmxjq+TG/tpVSdGZ6xGR9h4kcqL6bW1FNSnXZoWbcO1wT4XPUBwNAByOh+vkJvkcVJhXdU2e5/8axbTbirvUWuRv8TfupUs1XUoo= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1752753173; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=WXxbBdWkSFir8qoOw7q+QMZqWjLZWA9TcFjO3xeUJH4=; b=nifqtrl9VvPXTxSo+pnBY+cTo5Bh09nB+okuzCHA9noaWXHWU9mOQVwnP9FFgwRYdti8/LIiuc2Zv05BsvOadUMIf7ii2IxcKW+klicP2eEcTFM4vQoLhslX+GDcDy9LlW3niuZhiXvK19VfeVrMNkIPsG2OTm+S15TW8AT6F5g= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1752753173317782.1088983523138; Thu, 17 Jul 2025 04:52:53 -0700 (PDT) Received: from list by lists.xenproject.org with outflank-mailman.1046611.1417000 (Exim 4.92) (envelope-from ) id 1ucNAA-0002Uz-CF; Thu, 17 Jul 2025 11:52:34 +0000 Received: by outflank-mailman (output) from mailman id 1046611.1417000; Thu, 17 Jul 2025 11:52:34 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1ucNAA-0002Uq-96; Thu, 17 Jul 2025 11:52:34 +0000 Received: by outflank-mailman (input) for mailman id 1046611; Thu, 17 Jul 2025 11:52:33 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1ucNA9-0001TV-8h for xen-devel@lists.xenproject.org; Thu, 17 Jul 2025 11:52:33 +0000 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id 84c4110c-6304-11f0-b894-0df219b8e170; Thu, 17 Jul 2025 13:52:31 +0200 (CEST) Received: from mail-wr1-f71.google.com (mail-wr1-f71.google.com [209.85.221.71]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-444-MVqQcbgHNFWQmrKk00w84w-1; Thu, 17 Jul 2025 07:52:29 -0400 Received: by mail-wr1-f71.google.com with SMTP id ffacd0b85a97d-3a52bfda108so444016f8f.3 for ; Thu, 17 Jul 2025 04:52:28 -0700 (PDT) Received: from localhost (p200300d82f1f36000dc826ee9aa9fdc7.dip0.t-ipconnect.de. [2003:d8:2f1f:3600:dc8:26ee:9aa9:fdc7]) by smtp.gmail.com with UTF8SMTPSA id ffacd0b85a97d-3b5e8e25e75sm20438446f8f.87.2025.07.17.04.52.25 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Thu, 17 Jul 2025 04:52:26 -0700 (PDT) X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 84c4110c-6304-11f0-b894-0df219b8e170 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1752753150; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=WXxbBdWkSFir8qoOw7q+QMZqWjLZWA9TcFjO3xeUJH4=; b=dzFJrD9BXtOo90nvysQW+6hWrxvznyBjLpwndhVXoQOD3gweHcP6iZiH9RLPvnSL3YuYbe f+epGBITaMKXV1MAALtRUrTZx7IHoRoxSHC2//jsxCS0ZVOC3mud6OpvCoSDayI741UcTq XV3zGdcTOG+nZyZHh425dsySe4wwWTY= X-MC-Unique: MVqQcbgHNFWQmrKk00w84w-1 X-Mimecast-MFC-AGG-ID: MVqQcbgHNFWQmrKk00w84w_1752753148 X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1752753148; x=1753357948; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=WXxbBdWkSFir8qoOw7q+QMZqWjLZWA9TcFjO3xeUJH4=; b=Nv0CB8V3JeFdt2XYfrdSQT3v8aKlOGrehON/h/CTGOXdBhUSK2cCBLrvRZHTV7mdPR 9FgM5qRKOofCZkhFwODCrf5PNGrUx20kuIlQnOYJdk1dILZ0tEEKRuzGAH69xm9G/4ys itr7Tu9OdNqCfVVFOVNHkukN8vQ/XGeaB+ajB2RjJlWnmxfJPT8KFOjv07OIrYNF0vaj RUN1+fZQmHLauavivYVw1xCK5taox4QDjwRIhXW8DnkDeLEls62Jj6YtTuuzuR1bEmOA aKtdO9pTYnazzlngIds2ReaWt6LbDKEvEc+AYyofbj+f0jwXeJ0hqWGVyu4GTcKdIBms KJLg== X-Forwarded-Encrypted: i=1; AJvYcCUl3NgHPNrFSmaEnPs1qxk3vwP9H3TrPV/PW+0eLzkoKi+6EH3n5TtNnyk6Ye5LBLAapaiyDNViYvc=@lists.xenproject.org X-Gm-Message-State: AOJu0YweSDaXGYtSvZDfaPKZHWSzg9glU2o9Nt6vXnxB/ZAHP10+QAco onIb3KFs+L9nV7clMKsQiZIsTsPDKzRBbcDCcNkXMi72MVh1/lMTYHiuHlKRPQsCJFjWozZ05SJ iGDWI1zrUJ+cCr8nsyUnGmqOqNAm78/MtwpsQ8yNinM0IAyYu0bowD4OOAHpAYjBUvoug X-Gm-Gg: ASbGncsjpXN3G74JZPw8/9IhsJg/WQEOFsafM7YlF03i0xVkN+7/2XEUwdOWwxTlJ/w rsNm4vT2g370tVuFm5XtsDZA3hFHHV9L04+IyKiLTebrVilgaxPhwIv2fWA2orkEbLtmB/A5ezd RCCb8oSb117/spGf92V8Z28m9c/+dvDrt0LJUB2Hq9xlGtjDygP9m1NYRg6ZJ9OS+N2EoHF35/i ZDy2vmtldlghyQtxopW/SEepK7xNRkz4O7QB49scD1SYw0bjyCWzhKsR/DKmgeoCdyAXfm2kVMm 1n8KzUcw0cHTW/96xu67jIw3P8OFcUnzduAOmjiHO4s2RPC+NvmRskgQ/cqaXuzmv6eGnIetzle abcTgi2OpnoyQMkJc187xGhs= X-Received: by 2002:a05:6000:4b05:b0:3a4:e4ee:4ca9 with SMTP id ffacd0b85a97d-3b60dd72378mr5333948f8f.23.1752753147609; Thu, 17 Jul 2025 04:52:27 -0700 (PDT) X-Google-Smtp-Source: AGHT+IEVug96IdPAxHxRCwLhhCK8Tw1f2oOZhwD0absH2YXaDMF3tjDXL2HLEbyWszstU7d4tmbzEQ== X-Received: by 2002:a05:6000:4b05:b0:3a4:e4ee:4ca9 with SMTP id ffacd0b85a97d-3b60dd72378mr5333908f8f.23.1752753147033; Thu, 17 Jul 2025 04:52:27 -0700 (PDT) From: David Hildenbrand To: linux-kernel@vger.kernel.org Cc: linux-mm@kvack.org, xen-devel@lists.xenproject.org, linux-fsdevel@vger.kernel.org, nvdimm@lists.linux.dev, David Hildenbrand , Andrew Morton , Juergen Gross , Stefano Stabellini , Oleksandr Tyshchenko , Dan Williams , Matthew Wilcox , Jan Kara , Alexander Viro , Christian Brauner , Lorenzo Stoakes , "Liam R. Howlett" , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Zi Yan , Baolin Wang , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Jann Horn , Pedro Falcato , Hugh Dickins , Oscar Salvador , Lance Yang Subject: [PATCH v2 5/9] mm/huge_memory: mark PMD mappings of the huge zero folio special Date: Thu, 17 Jul 2025 13:52:08 +0200 Message-ID: <20250717115212.1825089-6-david@redhat.com> X-Mailer: git-send-email 2.50.1 In-Reply-To: <20250717115212.1825089-1-david@redhat.com> References: <20250717115212.1825089-1-david@redhat.com> MIME-Version: 1.0 X-Mimecast-Spam-Score: 0 X-Mimecast-MFC-PROC-ID: BGHfqqQjQBNTckfb2hV6vPxv6QBdEJZRNAXTRBi-8tg_1752753148 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1752753174619116600 Content-Type: text/plain; charset="utf-8"; x-default="true" The huge zero folio is refcounted (+mapcounted -- is that a word?) differently than "normal" folios, similarly (but different) to the ordinary shared zeropage. For this reason, we special-case these pages in vm_normal_page*/vm_normal_folio*, and only allow selected callers to still use them (e.g., GUP can still take a reference on them). vm_normal_page_pmd() already filters out the huge zero folio. However, so far we are not marking it as special like we do with the ordinary shared zeropage. Let's mark it as special, so we can further refactor vm_normal_page_pmd() and vm_normal_page(). While at it, update the doc regarding the shared zero folios. Reviewed-by: Oscar Salvador Signed-off-by: David Hildenbrand Reviewed-by: Lorenzo Stoakes Reviewed-by: Wei Yang --- mm/huge_memory.c | 5 ++++- mm/memory.c | 14 +++++++++----- 2 files changed, 13 insertions(+), 6 deletions(-) diff --git a/mm/huge_memory.c b/mm/huge_memory.c index db08c37b87077..3f9a27812a590 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -1320,6 +1320,7 @@ static void set_huge_zero_folio(pgtable_t pgtable, st= ruct mm_struct *mm, { pmd_t entry; entry =3D folio_mk_pmd(zero_folio, vma->vm_page_prot); + entry =3D pmd_mkspecial(entry); pgtable_trans_huge_deposit(mm, pmd, pgtable); set_pmd_at(mm, haddr, pmd, entry); mm_inc_nr_ptes(mm); @@ -1429,7 +1430,9 @@ static vm_fault_t insert_pmd(struct vm_area_struct *v= ma, unsigned long addr, if (fop.is_folio) { entry =3D folio_mk_pmd(fop.folio, vma->vm_page_prot); =20 - if (!is_huge_zero_folio(fop.folio)) { + if (is_huge_zero_folio(fop.folio)) { + entry =3D pmd_mkspecial(entry); + } else { folio_get(fop.folio); folio_add_file_rmap_pmd(fop.folio, &fop.folio->page, vma); add_mm_counter(mm, mm_counter_file(fop.folio), HPAGE_PMD_NR); diff --git a/mm/memory.c b/mm/memory.c index 92fd18a5d8d1f..173eb6267e0ac 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -537,7 +537,13 @@ static void print_bad_pte(struct vm_area_struct *vma, = unsigned long addr, * * "Special" mappings do not wish to be associated with a "struct page" (e= ither * it doesn't exist, or it exists but they don't want to touch it). In this - * case, NULL is returned here. "Normal" mappings do have a struct page. + * case, NULL is returned here. "Normal" mappings do have a struct page and + * are ordinarily refcounted. + * + * Page mappings of the shared zero folios are always considered "special"= , as + * they are not ordinarily refcounted. However, selected page table walkers + * (such as GUP) can still identify these mappings and work with the + * underlying "struct page". * * There are 2 broad cases. Firstly, an architecture may define a pte_spec= ial() * pte bit, in which case this function is trivial. Secondly, an architect= ure @@ -567,9 +573,8 @@ static void print_bad_pte(struct vm_area_struct *vma, u= nsigned long addr, * * VM_MIXEDMAP mappings can likewise contain memory with or without "struct * page" backing, however the difference is that _all_ pages with a struct - * page (that is, those where pfn_valid is true) are refcounted and consid= ered - * normal pages by the VM. The only exception are zeropages, which are - * *never* refcounted. + * page (that is, those where pfn_valid is true, except the shared zero + * folios) are refcounted and considered normal pages by the VM. * * The disadvantage is that pages are refcounted (which can be slower and * simply not an option for some PFNMAP users). The advantage is that we @@ -649,7 +654,6 @@ struct page *vm_normal_page_pmd(struct vm_area_struct *= vma, unsigned long addr, { unsigned long pfn =3D pmd_pfn(pmd); =20 - /* Currently it's only used for huge pfnmaps */ if (unlikely(pmd_special(pmd))) return NULL; =20 --=20 2.50.1 From nobody Mon Oct 6 20:56:44 2025 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass(p=quarantine dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1752753179; cv=none; d=zohomail.com; s=zohoarc; b=mLfj+EPCxlATMLNoiwmjTR59l94ACprIjR9nS2XjyTnRDAygzVCP9HgVRXf4eTP5moM6AjGAZfWBld2MK6mzMSJRiJ/qi2kN/ywQrMBeBNg7mVV4EOGK1r0u7kJHx8wBGw9xb7HAbGrBzSvDZuS9t9pFG8rfivbeqic3D0onTyU= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1752753179; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=GbR0phC2X7fIX30bxtjDJxvlOXqw8t3nlah85Ck4RFM=; b=nyvs5/QKT8EAFFHlDxkni1T/E6waJlgOJJkMsfDQLLrcqDfuNipUJDsjvHNbbJF+MDoja6V/lvEjJBKjQfHhD5dGITONebDjQG2yaU7eazWekKgLB142ckYTnrdyeVXoQWeUIDz+BH4Q0gtnCS8oYq8qbwvpnwpdDa0gtG1961s= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1752753179331279.1881036322869; Thu, 17 Jul 2025 04:52:59 -0700 (PDT) Received: from list by lists.xenproject.org with outflank-mailman.1046612.1417011 (Exim 4.92) (envelope-from ) id 1ucNAE-0002uW-ST; Thu, 17 Jul 2025 11:52:38 +0000 Received: by outflank-mailman (output) from mailman id 1046612.1417011; Thu, 17 Jul 2025 11:52:38 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1ucNAE-0002u8-NR; Thu, 17 Jul 2025 11:52:38 +0000 Received: by outflank-mailman (input) for mailman id 1046612; Thu, 17 Jul 2025 11:52:37 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1ucNAD-0001TV-Ji for xen-devel@lists.xenproject.org; Thu, 17 Jul 2025 11:52:37 +0000 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id 876bfdd8-6304-11f0-b894-0df219b8e170; Thu, 17 Jul 2025 13:52:36 +0200 (CEST) Received: from mail-wm1-f69.google.com (mail-wm1-f69.google.com [209.85.128.69]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-403-gQ-9PncEOP2ucXOPAkRmUA-1; Thu, 17 Jul 2025 07:52:31 -0400 Received: by mail-wm1-f69.google.com with SMTP id 5b1f17b1804b1-45597cc95d5so4557355e9.1 for ; Thu, 17 Jul 2025 04:52:31 -0700 (PDT) Received: from localhost (p200300d82f1f36000dc826ee9aa9fdc7.dip0.t-ipconnect.de. [2003:d8:2f1f:3600:dc8:26ee:9aa9:fdc7]) by smtp.gmail.com with UTF8SMTPSA id 5b1f17b1804b1-45626c7b1a9sm45953235e9.0.2025.07.17.04.52.28 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Thu, 17 Jul 2025 04:52:29 -0700 (PDT) X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 876bfdd8-6304-11f0-b894-0df219b8e170 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1752753154; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=GbR0phC2X7fIX30bxtjDJxvlOXqw8t3nlah85Ck4RFM=; b=U11oFUPwSgFforW31IQ3ERgYF9VPlP23HVKfearb2qtijLkm1Vd9BisAsKGTxvCdeCAlzX D2EFa0NU2IKTInvt0v3G2DTM7kF8iMpS4k8a2dwjABlYuw0kV16mwiHsj6nyVgR3UdtuN0 3K0NwrKrMsvFx5zBjdSv+aoJ5+abxZE= X-MC-Unique: gQ-9PncEOP2ucXOPAkRmUA-1 X-Mimecast-MFC-AGG-ID: gQ-9PncEOP2ucXOPAkRmUA_1752753150 X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1752753150; x=1753357950; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=GbR0phC2X7fIX30bxtjDJxvlOXqw8t3nlah85Ck4RFM=; b=T63m+ZuDn3HpkBDSz2hPucdZ4xa1MalhvlXaq0KUXf0xsgDlyQtwdPTNF3K6/U4C0k 6Z400kwJX96qw6bj3bBO6gI2p446R0Oof8Zkfvg8n4T+t8htY+rncCEqo7pIQinIjC5u iPhmUQyHGzh/RstzpPqnfE/UV+LygsTFWsT/NYRg6KXSlzFmV5U/vMTms7sU0a6XrxNJ UQf/vFpY3mzF3WnakezaQMPUme/BRTdaoV5WXvYOgs7cY8rZ0rSqDWdgDNGdt4fJ0sPA DlpVu8B01fvhF6KBNv9NSERnxW5Iz0mc6We7I2SCnH/w5kXDBjxbZlxXWTLbbe6S+/7d rYjA== X-Forwarded-Encrypted: i=1; AJvYcCWa6h5Q783eQUSwotlHMStLuPVtUBM3kZ6LTUiSFaeKL8Pqrnw9iA0rtH6pi1vcEHSDKRxZS3zPDVw=@lists.xenproject.org X-Gm-Message-State: AOJu0Yyy0OFxrMdswQN/Li9Wv6urM5iq+L4fv4aldvn8PfJGfgSAXoQx bdOpF33c3K08ByCAVNZ60k1Oen0pZXxBUcHyQlc4OyiRCHDjHA4ylBuHbAlvMfBs+6y0JPlHI1T g+m74162hsqDppKAskJzon0Um6iP6LikK7gFyCyofv+t+Fu9GELQgPnxfHYyhkHtjlH2V X-Gm-Gg: ASbGncutk1ZVZh8q9UvpU6v5rPoBLeHIWmQXvq1AGHPxpv+4n1XqyS8q5gX3Ucf1wFa htkJEaDxWfcRqL+dN0NsD234zXTbmU8eaC4utWIGK4BIC3MwqIWK8tvQXUm6s2lU6/ZQVYwuAMm TfMyeuj3l6Ug4RrB64zccmO12tARo6eoPuz4rhNKSYTtzR9zFT/ifcXhU7YAuFSeXJx9UttDs7n VcXeeQx5E2wGJGjEFHvtgv76GrTd+AnX/Fo7uh7lfvucucuvTAz7nZE0atM9H1Pp9G+rATfJS0F wKoGFwoYSUEaSwubIPlKHEnGeyRL5ZZBSiNK1134b+6F6odAaGg9CXp3rBOQgvMsNEzwqECgp3N VE3rHL1P6ntJ3yQDzGTDX+2A= X-Received: by 2002:a05:600c:4706:b0:456:19eb:2e09 with SMTP id 5b1f17b1804b1-4562edaa08cmr59716035e9.8.1752753150340; Thu, 17 Jul 2025 04:52:30 -0700 (PDT) X-Google-Smtp-Source: AGHT+IFxGF6l6GEGtw18Sa0l+0UWiR0a/WM3dG6MbtHwia3ILIrxrhuz79ecySj7OKZMvEeK7lnwGw== X-Received: by 2002:a05:600c:4706:b0:456:19eb:2e09 with SMTP id 5b1f17b1804b1-4562edaa08cmr59715535e9.8.1752753149770; Thu, 17 Jul 2025 04:52:29 -0700 (PDT) From: David Hildenbrand To: linux-kernel@vger.kernel.org Cc: linux-mm@kvack.org, xen-devel@lists.xenproject.org, linux-fsdevel@vger.kernel.org, nvdimm@lists.linux.dev, David Hildenbrand , Andrew Morton , Juergen Gross , Stefano Stabellini , Oleksandr Tyshchenko , Dan Williams , Matthew Wilcox , Jan Kara , Alexander Viro , Christian Brauner , Lorenzo Stoakes , "Liam R. Howlett" , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Zi Yan , Baolin Wang , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Jann Horn , Pedro Falcato , Hugh Dickins , Oscar Salvador , Lance Yang Subject: [PATCH v2 6/9] mm/memory: convert print_bad_pte() to print_bad_page_map() Date: Thu, 17 Jul 2025 13:52:09 +0200 Message-ID: <20250717115212.1825089-7-david@redhat.com> X-Mailer: git-send-email 2.50.1 In-Reply-To: <20250717115212.1825089-1-david@redhat.com> References: <20250717115212.1825089-1-david@redhat.com> MIME-Version: 1.0 X-Mimecast-Spam-Score: 0 X-Mimecast-MFC-PROC-ID: GMMhRS21kUgNOgN3Mp_r7bU0_EavYb4nvOtTUVGE2PA_1752753150 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1752753180816116600 Content-Type: text/plain; charset="utf-8"; x-default="true" print_bad_pte() looks like something that should actually be a WARN or similar, but historically it apparently has proven to be useful to detect corruption of page tables even on production systems -- report the issue and keep the system running to make it easier to actually detect what is going wrong (e.g., multiple such messages might shed a light). As we want to unify vm_normal_page_*() handling for PTE/PMD/PUD, we'll have to take care of print_bad_pte() as well. Let's prepare for using print_bad_pte() also for non-PTEs by adjusting the implementation and renaming the function -- we'll rename it to what we actually print: bad (page) mappings. Maybe it should be called "print_bad_table_entry()"? We'll just call it "print_bad_page_map()" because the assumption is that we are dealing with some (previously) present page table entry that got corrupted in weird ways. Whether it is a PTE or something else will usually become obvious from the page table dump or from the dumped stack. If ever required in the future, we could pass the entry level type similar to "enum rmap_level". For now, let's keep it simple. To make the function a bit more readable, factor out the ratelimit check into is_bad_page_map_ratelimited() and place the dumping of page table content into __dump_bad_page_map_pgtable(). We'll now dump information from each level in a single line, and just stop the table walk once we hit something that is not a present page table. Use print_bad_page_map() in vm_normal_page_pmd() similar to how we do it for vm_normal_page(), now that we have a function that can handle it. The report will now look something like (dumping pgd to pmd values): [ 77.943408] BUG: Bad page map in process XXX entry:80000001233f5867 [ 77.944077] addr:00007fd84bb1c000 vm_flags:08100071 anon_vma: ... [ 77.945186] pgd:10a89f067 p4d:10a89f067 pud:10e5a2067 pmd:105327067 Not using pgdp_get(), because that does not work properly on some arm configs where pgd_t is an array. Note that we are dumping all levels even when levels are folded for simplicity. Signed-off-by: David Hildenbrand --- mm/memory.c | 120 ++++++++++++++++++++++++++++++++++++++++------------ 1 file changed, 94 insertions(+), 26 deletions(-) diff --git a/mm/memory.c b/mm/memory.c index 173eb6267e0ac..08d16ed7b4cc7 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -473,22 +473,8 @@ static inline void add_mm_rss_vec(struct mm_struct *mm= , int *rss) add_mm_counter(mm, i, rss[i]); } =20 -/* - * This function is called to print an error when a bad pte - * is found. For example, we might have a PFN-mapped pte in - * a region that doesn't allow it. - * - * The calling function must still handle the error. - */ -static void print_bad_pte(struct vm_area_struct *vma, unsigned long addr, - pte_t pte, struct page *page) +static bool is_bad_page_map_ratelimited(void) { - pgd_t *pgd =3D pgd_offset(vma->vm_mm, addr); - p4d_t *p4d =3D p4d_offset(pgd, addr); - pud_t *pud =3D pud_offset(p4d, addr); - pmd_t *pmd =3D pmd_offset(pud, addr); - struct address_space *mapping; - pgoff_t index; static unsigned long resume; static unsigned long nr_shown; static unsigned long nr_unshown; @@ -500,7 +486,7 @@ static void print_bad_pte(struct vm_area_struct *vma, u= nsigned long addr, if (nr_shown =3D=3D 60) { if (time_before(jiffies, resume)) { nr_unshown++; - return; + return true; } if (nr_unshown) { pr_alert("BUG: Bad page map: %lu messages suppressed\n", @@ -511,15 +497,87 @@ static void print_bad_pte(struct vm_area_struct *vma,= unsigned long addr, } if (nr_shown++ =3D=3D 0) resume =3D jiffies + 60 * HZ; + return false; +} + +static void __dump_bad_page_map_pgtable(struct mm_struct *mm, unsigned lon= g addr) +{ + unsigned long long pgdv, p4dv, pudv, pmdv; + p4d_t p4d, *p4dp; + pud_t pud, *pudp; + pmd_t pmd, *pmdp; + pgd_t *pgdp; + + /* + * This looks like a fully lockless walk, however, the caller is + * expected to hold the leaf page table lock in addition to other + * rmap/mm/vma locks. So this is just a re-walk to dump page table + * content while any concurrent modifications should be completely + * prevented. + */ + pgdp =3D pgd_offset(mm, addr); + pgdv =3D pgd_val(*pgdp); + + if (!pgd_present(*pgdp) || pgd_leaf(*pgdp)) { + pr_alert("pgd:%08llx\n", pgdv); + return; + } + + p4dp =3D p4d_offset(pgdp, addr); + p4d =3D p4dp_get(p4dp); + p4dv =3D p4d_val(p4d); + + if (!p4d_present(p4d) || p4d_leaf(p4d)) { + pr_alert("pgd:%08llx p4d:%08llx\n", pgdv, p4dv); + return; + } + + pudp =3D pud_offset(p4dp, addr); + pud =3D pudp_get(pudp); + pudv =3D pud_val(pud); + + if (!pud_present(pud) || pud_leaf(pud)) { + pr_alert("pgd:%08llx p4d:%08llx pud:%08llx\n", pgdv, p4dv, pudv); + return; + } + + pmdp =3D pmd_offset(pudp, addr); + pmd =3D pmdp_get(pmdp); + pmdv =3D pmd_val(pmd); + + /* + * Dumping the PTE would be nice, but it's tricky with CONFIG_HIGHPTE, + * because the table should already be mapped by the caller and + * doing another map would be bad. print_bad_page_map() should + * already take care of printing the PTE. + */ + pr_alert("pgd:%08llx p4d:%08llx pud:%08llx pmd:%08llx\n", pgdv, + p4dv, pudv, pmdv); +} + +/* + * This function is called to print an error when a bad page table entry (= e.g., + * corrupted page table entry) is found. For example, we might have a + * PFN-mapped pte in a region that doesn't allow it. + * + * The calling function must still handle the error. + */ +static void print_bad_page_map(struct vm_area_struct *vma, + unsigned long addr, unsigned long long entry, struct page *page) +{ + struct address_space *mapping; + pgoff_t index; + + if (is_bad_page_map_ratelimited()) + return; =20 mapping =3D vma->vm_file ? vma->vm_file->f_mapping : NULL; index =3D linear_page_index(vma, addr); =20 - pr_alert("BUG: Bad page map in process %s pte:%08llx pmd:%08llx\n", - current->comm, - (long long)pte_val(pte), (long long)pmd_val(*pmd)); + pr_alert("BUG: Bad page map in process %s entry:%08llx", current->comm, = entry); + __dump_bad_page_map_pgtable(vma->vm_mm, addr); if (page) - dump_page(page, "bad pte"); + dump_page(page, "bad page map"); pr_alert("addr:%px vm_flags:%08lx anon_vma:%px mapping:%px index:%lx\n", (void *)addr, vma->vm_flags, vma->anon_vma, mapping, index); pr_alert("file:%pD fault:%ps mmap:%ps mmap_prepare: %ps read_folio:%ps\n", @@ -597,7 +655,7 @@ struct page *vm_normal_page(struct vm_area_struct *vma,= unsigned long addr, if (is_zero_pfn(pfn)) return NULL; =20 - print_bad_pte(vma, addr, pte, NULL); + print_bad_page_map(vma, addr, pte_val(pte), NULL); return NULL; } =20 @@ -625,7 +683,7 @@ struct page *vm_normal_page(struct vm_area_struct *vma,= unsigned long addr, =20 check_pfn: if (unlikely(pfn > highest_memmap_pfn)) { - print_bad_pte(vma, addr, pte, NULL); + print_bad_page_map(vma, addr, pte_val(pte), NULL); return NULL; } =20 @@ -654,8 +712,15 @@ struct page *vm_normal_page_pmd(struct vm_area_struct = *vma, unsigned long addr, { unsigned long pfn =3D pmd_pfn(pmd); =20 - if (unlikely(pmd_special(pmd))) + if (unlikely(pmd_special(pmd))) { + if (vma->vm_flags & (VM_PFNMAP | VM_MIXEDMAP)) + return NULL; + if (is_huge_zero_pfn(pfn)) + return NULL; + + print_bad_page_map(vma, addr, pmd_val(pmd), NULL); return NULL; + } =20 if (unlikely(vma->vm_flags & (VM_PFNMAP|VM_MIXEDMAP))) { if (vma->vm_flags & VM_MIXEDMAP) { @@ -674,8 +739,10 @@ struct page *vm_normal_page_pmd(struct vm_area_struct = *vma, unsigned long addr, =20 if (is_huge_zero_pfn(pfn)) return NULL; - if (unlikely(pfn > highest_memmap_pfn)) + if (unlikely(pfn > highest_memmap_pfn)) { + print_bad_page_map(vma, addr, pmd_val(pmd), NULL); return NULL; + } =20 /* * NOTE! We still have PageReserved() pages in the page tables. @@ -1509,7 +1576,7 @@ static __always_inline void zap_present_folio_ptes(st= ruct mmu_gather *tlb, folio_remove_rmap_ptes(folio, page, nr, vma); =20 if (unlikely(folio_mapcount(folio) < 0)) - print_bad_pte(vma, addr, ptent, page); + print_bad_page_map(vma, addr, pte_val(ptent), page); } if (unlikely(__tlb_remove_folio_pages(tlb, page, nr, delay_rmap))) { *force_flush =3D true; @@ -4507,7 +4574,8 @@ vm_fault_t do_swap_page(struct vm_fault *vmf) } else if (is_pte_marker_entry(entry)) { ret =3D handle_pte_marker(vmf); } else { - print_bad_pte(vma, vmf->address, vmf->orig_pte, NULL); + print_bad_page_map(vma, vmf->address, + pte_val(vmf->orig_pte), NULL); ret =3D VM_FAULT_SIGBUS; } goto out; --=20 2.50.1 From nobody Mon Oct 6 20:56:44 2025 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass(p=quarantine dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1752753181; cv=none; d=zohomail.com; s=zohoarc; b=ivuJo1bxXqXP+8jcEm7rIjBN58nxbSoSVWbKHMz5nwMdyCq8J/Whhsb06aEcLhq45RtyYu2o7C4fjyKmEORPvza3J/Acj1mMLJ0hWpX8/KmVL083HOYuRnd3KIH7P/9tEL5eZBBjOUvkYzPnJcNtC/WAvrMQgnPId+0zG5c4Vuo= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1752753181; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=S6ocb38IkVcnlHaJeHNVb3Jrmzmc76tJoqI7g7ZBmQA=; b=fv2znbpqehT4XAcJ6h/gFUwmjpsQCDUjVWh8UlF95hGKsijHWKx6dUDAlBjEMgxj3HkLqMaZDLlDwCI5ICgV8SxYSpOxFxUGXftupEHt+UD9ZTT5H13vAHqYpWVHZlwVUrn2NM1lVNSktfKTxKEMuHIKbtjB5mZvVee+sR6aWcg= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1752753181670699.2437073921474; Thu, 17 Jul 2025 04:53:01 -0700 (PDT) Received: from list by lists.xenproject.org with outflank-mailman.1046617.1417022 (Exim 4.92) (envelope-from ) id 1ucNAJ-0003JY-8T; Thu, 17 Jul 2025 11:52:43 +0000 Received: by outflank-mailman (output) from mailman id 1046617.1417022; Thu, 17 Jul 2025 11:52:43 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1ucNAJ-0003JJ-1f; Thu, 17 Jul 2025 11:52:43 +0000 Received: by outflank-mailman (input) for mailman id 1046617; Thu, 17 Jul 2025 11:52:41 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1ucNAH-0001TV-KI for xen-devel@lists.xenproject.org; Thu, 17 Jul 2025 11:52:41 +0000 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id 88c74ac7-6304-11f0-b894-0df219b8e170; Thu, 17 Jul 2025 13:52:38 +0200 (CEST) Received: from mail-wr1-f72.google.com (mail-wr1-f72.google.com [209.85.221.72]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-594-zqNo8uIAM62IBwmAZEEJhQ-1; Thu, 17 Jul 2025 07:52:34 -0400 Received: by mail-wr1-f72.google.com with SMTP id ffacd0b85a97d-3a4f7f1b932so563421f8f.2 for ; Thu, 17 Jul 2025 04:52:33 -0700 (PDT) Received: from localhost (p200300d82f1f36000dc826ee9aa9fdc7.dip0.t-ipconnect.de. [2003:d8:2f1f:3600:dc8:26ee:9aa9:fdc7]) by smtp.gmail.com with UTF8SMTPSA id 5b1f17b1804b1-45634ec9162sm20451645e9.0.2025.07.17.04.52.30 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Thu, 17 Jul 2025 04:52:31 -0700 (PDT) X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 88c74ac7-6304-11f0-b894-0df219b8e170 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1752753157; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=S6ocb38IkVcnlHaJeHNVb3Jrmzmc76tJoqI7g7ZBmQA=; b=LwcilLDLfgeRCmoiCuwYwHVWWlIjWgNT1izgO+mMzfl/vLNUWzS63EIrZAIaoZREUl5mCm IpT6PMW6V/xOXGUkfmwWJU+EJI5iJoG0k3E7+eB/Xfu5SfbMr1JrcQiwTl8RKp0KYz2SFm tTRBzkNTGsizNHmhhtHckbCRiE7/Wlg= X-MC-Unique: zqNo8uIAM62IBwmAZEEJhQ-1 X-Mimecast-MFC-AGG-ID: zqNo8uIAM62IBwmAZEEJhQ_1752753153 X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1752753153; x=1753357953; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=S6ocb38IkVcnlHaJeHNVb3Jrmzmc76tJoqI7g7ZBmQA=; b=uxM+6A3iHS6ZZyyAXBNJ+KMuIBgEmX9brqL8dE1KArz9FTnKz9oE1nidNnDToMIUwX Yx+6e04zyXmU3419kS+onfxN+DSl3ayEyfJ/Y/mTpslTWU22kvPMI7Fh3GYr25+4fNZP dbKCkkCQT9VAdFwYpzkEF9wR4ABzXrgLUPBxrh6M9DC29712r5jyBkWEK99SJcJl5gGK lvuhgsojE8ygehKkApSSLV8XTI5LPqzSIH4+2Mqz28Va60GpuKrwWB+w8bx3LA/c/j09 qYS3ly08aXULsq4dZyy7wXWvVNgxpPA5YK5bn5+TLr047paq4KQ3J8pkKg+/x0dBD6I4 umZA== X-Forwarded-Encrypted: i=1; AJvYcCWqwboSJ9PIyKHMQoK9UAMuew3iFOpaK4+h6qMdePrU4T4HKn4j4slpvVlOrYcC0AeFlo8kRpXkeGU=@lists.xenproject.org X-Gm-Message-State: AOJu0YyuxApq0PsSM/oUYv3wJ/9OZ+CzAKnQSCvAxuVA+YyqjIQLfhVp WZaDF+fb1nBJ7apAAcM9fG4sv7g88Z62jkUCZcGRfZ6ktsq8c0qglCfT7k2jxvfTH2srFFXgX5x 3s9TZZj3RK3KOYt7ktJuc77B+Xm2PpMwDw4F5VErw6eHBRhhiIqSUWOA6agwWiRczeufK X-Gm-Gg: ASbGncusQAH+pTlQr2ODXRZDYmcdCiEsXmxX3YYvqr/eyUrVNycs41+QmiJGrNTaf2T BMM1gXrY5EI3YHq7Ia8Z4g/W+JCoDK4dSQn1MsDCEBLQRglr45B6hUv+9Tdq61g2ftWArUz86hW rmNlgLvi1Y5u3Jwvb6eifc7ZuO1D7QgNu0pdS7dGSL7QnMpBHhSN4k8Ul1A0PtEHQ5TnOPWq/p3 EDOWYlKQphr/VM2kjyEs1SDJKLPZne4qKpjuVuQlRWc5RftWOInSc1FHRrzR9y+8sUUVuIq/3+V zOQ1cpGzjleGLFahiJ2amqNNd9cH52Wtn/TTuBiRVPL7FdICDsD6F+plJWzC/mZXIv/iUFiD4pX +WUpSb8JCbF6B9z5oqOUJhAQ= X-Received: by 2002:a05:6000:43c6:10b0:3a4:f744:e01b with SMTP id ffacd0b85a97d-3b60e50fde7mr3995244f8f.39.1752753152624; Thu, 17 Jul 2025 04:52:32 -0700 (PDT) X-Google-Smtp-Source: AGHT+IGLcgp48Noardv8I0f6W2NZTd1NV9HbsxHjW8GgnbNobVHwjA5VMkKS9pv04KvpTaqXa6tubA== X-Received: by 2002:a05:6000:43c6:10b0:3a4:f744:e01b with SMTP id ffacd0b85a97d-3b60e50fde7mr3995202f8f.39.1752753152069; Thu, 17 Jul 2025 04:52:32 -0700 (PDT) From: David Hildenbrand To: linux-kernel@vger.kernel.org Cc: linux-mm@kvack.org, xen-devel@lists.xenproject.org, linux-fsdevel@vger.kernel.org, nvdimm@lists.linux.dev, David Hildenbrand , Andrew Morton , Juergen Gross , Stefano Stabellini , Oleksandr Tyshchenko , Dan Williams , Matthew Wilcox , Jan Kara , Alexander Viro , Christian Brauner , Lorenzo Stoakes , "Liam R. Howlett" , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Zi Yan , Baolin Wang , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Jann Horn , Pedro Falcato , Hugh Dickins , Oscar Salvador , Lance Yang Subject: [PATCH v2 7/9] mm/memory: factor out common code from vm_normal_page_*() Date: Thu, 17 Jul 2025 13:52:10 +0200 Message-ID: <20250717115212.1825089-8-david@redhat.com> X-Mailer: git-send-email 2.50.1 In-Reply-To: <20250717115212.1825089-1-david@redhat.com> References: <20250717115212.1825089-1-david@redhat.com> MIME-Version: 1.0 X-Mimecast-Spam-Score: 0 X-Mimecast-MFC-PROC-ID: JPw4XTVDYFzyLXlRMBo8KCEGh25hQYBd-JqOrUfmCVk_1752753153 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1752753182694116600 Content-Type: text/plain; charset="utf-8"; x-default="true" Let's reduce the code duplication and factor out the non-pte/pmd related magic into vm_normal_page_pfn(). To keep it simpler, check the pfn against both zero folios. We could optimize this, but as it's only for the !CONFIG_ARCH_HAS_PTE_SPECIAL case, it's not a compelling micro-optimization. With CONFIG_ARCH_HAS_PTE_SPECIAL we don't have to check anything else, really. It's a good question if we can even hit the !CONFIG_ARCH_HAS_PTE_SPECIAL scenario in the PMD case in practice: but doesn't really matter, as it's now all unified in vm_normal_page_pfn(). Add kerneldoc for all involved functions. No functional change intended. Reviewed-by: Oscar Salvador Signed-off-by: David Hildenbrand --- mm/memory.c | 183 +++++++++++++++++++++++++++++++--------------------- 1 file changed, 109 insertions(+), 74 deletions(-) diff --git a/mm/memory.c b/mm/memory.c index 08d16ed7b4cc7..c43ae5e4d7644 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -590,8 +590,13 @@ static void print_bad_page_map(struct vm_area_struct *= vma, add_taint(TAINT_BAD_PAGE, LOCKDEP_NOW_UNRELIABLE); } =20 -/* - * vm_normal_page -- This function gets the "struct page" associated with = a pte. +/** + * vm_normal_page_pfn() - Get the "struct page" associated with a PFN in a + * non-special page table entry. + * @vma: The VMA mapping the @pfn. + * @addr: The address where the @pfn is mapped. + * @pfn: The PFN. + * @entry: The page table entry value for error reporting purposes. * * "Special" mappings do not wish to be associated with a "struct page" (e= ither * it doesn't exist, or it exists but they don't want to touch it). In this @@ -603,10 +608,10 @@ static void print_bad_page_map(struct vm_area_struct = *vma, * (such as GUP) can still identify these mappings and work with the * underlying "struct page". * - * There are 2 broad cases. Firstly, an architecture may define a pte_spec= ial() - * pte bit, in which case this function is trivial. Secondly, an architect= ure - * may not have a spare pte bit, which requires a more complicated scheme, - * described below. + * There are 2 broad cases. Firstly, an architecture may define a "special" + * page table entry bit (e.g., pte_special()), in which case this function= is + * trivial. Secondly, an architecture may not have a spare page table + * entry bit, which requires a more complicated scheme, described below. * * A raw VM_PFNMAP mapping (ie. one that is not COWed) is always considere= d a * special mapping (even if there are underlying and valid "struct pages"). @@ -639,15 +644,72 @@ static void print_bad_page_map(struct vm_area_struct = *vma, * don't have to follow the strict linearity rule of PFNMAP mappings in * order to support COWable mappings. * + * This function is not expected to be called for obviously special mappin= gs: + * when the page table entry has the "special" bit set. + * + * Return: Returns the "struct page" if this is a "normal" mapping. Returns + * NULL if this is a "special" mapping. + */ +static inline struct page *vm_normal_page_pfn(struct vm_area_struct *vma, + unsigned long addr, unsigned long pfn, unsigned long long entry) +{ + /* + * With CONFIG_ARCH_HAS_PTE_SPECIAL, any special page table mappings + * (incl. shared zero folios) are marked accordingly and are handled + * by the caller. + */ + if (!IS_ENABLED(CONFIG_ARCH_HAS_PTE_SPECIAL)) { + if (unlikely(vma->vm_flags & (VM_PFNMAP | VM_MIXEDMAP))) { + if (vma->vm_flags & VM_MIXEDMAP) { + /* If it has a "struct page", it's "normal". */ + if (!pfn_valid(pfn)) + return NULL; + } else { + unsigned long off =3D (addr - vma->vm_start) >> PAGE_SHIFT; + + /* Only CoW'ed anon folios are "normal". */ + if (pfn =3D=3D vma->vm_pgoff + off) + return NULL; + if (!is_cow_mapping(vma->vm_flags)) + return NULL; + } + } + + if (is_zero_pfn(pfn) || is_huge_zero_pfn(pfn)) + return NULL; + } + + /* Cheap check for corrupted page table entries. */ + if (pfn > highest_memmap_pfn) { + print_bad_page_map(vma, addr, entry, NULL); + return NULL; + } + /* + * NOTE! We still have PageReserved() pages in the page tables. + * For example, VDSO mappings can cause them to exist. + */ + VM_WARN_ON_ONCE(is_zero_pfn(pfn) || is_huge_zero_pfn(pfn)); + return pfn_to_page(pfn); +} + +/** + * vm_normal_page() - Get the "struct page" associated with a PTE + * @vma: The VMA mapping the @pte. + * @addr: The address where the @pte is mapped. + * @pte: The PTE. + * + * Get the "struct page" associated with a PTE. See vm_normal_page_pfn() + * for details. + * + * Return: Returns the "struct page" if this is a "normal" mapping. Returns + * NULL if this is a "special" mapping. */ struct page *vm_normal_page(struct vm_area_struct *vma, unsigned long addr, pte_t pte) { unsigned long pfn =3D pte_pfn(pte); =20 - if (IS_ENABLED(CONFIG_ARCH_HAS_PTE_SPECIAL)) { - if (likely(!pte_special(pte))) - goto check_pfn; + if (unlikely(pte_special(pte))) { if (vma->vm_ops && vma->vm_ops->find_special_page) return vma->vm_ops->find_special_page(vma, addr); if (vma->vm_flags & (VM_PFNMAP | VM_MIXEDMAP)) @@ -658,44 +720,21 @@ struct page *vm_normal_page(struct vm_area_struct *vm= a, unsigned long addr, print_bad_page_map(vma, addr, pte_val(pte), NULL); return NULL; } - - /* !CONFIG_ARCH_HAS_PTE_SPECIAL case follows: */ - - if (unlikely(vma->vm_flags & (VM_PFNMAP|VM_MIXEDMAP))) { - if (vma->vm_flags & VM_MIXEDMAP) { - if (!pfn_valid(pfn)) - return NULL; - if (is_zero_pfn(pfn)) - return NULL; - goto out; - } else { - unsigned long off; - off =3D (addr - vma->vm_start) >> PAGE_SHIFT; - if (pfn =3D=3D vma->vm_pgoff + off) - return NULL; - if (!is_cow_mapping(vma->vm_flags)) - return NULL; - } - } - - if (is_zero_pfn(pfn)) - return NULL; - -check_pfn: - if (unlikely(pfn > highest_memmap_pfn)) { - print_bad_page_map(vma, addr, pte_val(pte), NULL); - return NULL; - } - - /* - * NOTE! We still have PageReserved() pages in the page tables. - * eg. VDSO mappings can cause them to exist. - */ -out: - VM_WARN_ON_ONCE(is_zero_pfn(pfn)); - return pfn_to_page(pfn); + return vm_normal_page_pfn(vma, addr, pfn, pte_val(pte)); } =20 +/** + * vm_normal_folio() - Get the "struct folio" associated with a PTE + * @vma: The VMA mapping the @pte. + * @addr: The address where the @pte is mapped. + * @pte: The PTE. + * + * Get the "struct folio" associated with a PTE. See vm_normal_page_pfn() + * for details. + * + * Return: Returns the "struct folio" if this is a "normal" mapping. Retur= ns + * NULL if this is a "special" mapping. + */ struct folio *vm_normal_folio(struct vm_area_struct *vma, unsigned long ad= dr, pte_t pte) { @@ -707,6 +746,18 @@ struct folio *vm_normal_folio(struct vm_area_struct *v= ma, unsigned long addr, } =20 #ifdef CONFIG_PGTABLE_HAS_HUGE_LEAVES +/** + * vm_normal_page_pmd() - Get the "struct page" associated with a PMD + * @vma: The VMA mapping the @pmd. + * @addr: The address where the @pmd is mapped. + * @pmd: The PMD. + * + * Get the "struct page" associated with a PMD. See vm_normal_page_pfn() + * for details. + * + * Return: Returns the "struct page" if this is a "normal" mapping. Returns + * NULL if this is a "special" mapping. + */ struct page *vm_normal_page_pmd(struct vm_area_struct *vma, unsigned long = addr, pmd_t pmd) { @@ -721,37 +772,21 @@ struct page *vm_normal_page_pmd(struct vm_area_struct= *vma, unsigned long addr, print_bad_page_map(vma, addr, pmd_val(pmd), NULL); return NULL; } - - if (unlikely(vma->vm_flags & (VM_PFNMAP|VM_MIXEDMAP))) { - if (vma->vm_flags & VM_MIXEDMAP) { - if (!pfn_valid(pfn)) - return NULL; - goto out; - } else { - unsigned long off; - off =3D (addr - vma->vm_start) >> PAGE_SHIFT; - if (pfn =3D=3D vma->vm_pgoff + off) - return NULL; - if (!is_cow_mapping(vma->vm_flags)) - return NULL; - } - } - - if (is_huge_zero_pfn(pfn)) - return NULL; - if (unlikely(pfn > highest_memmap_pfn)) { - print_bad_page_map(vma, addr, pmd_val(pmd), NULL); - return NULL; - } - - /* - * NOTE! We still have PageReserved() pages in the page tables. - * eg. VDSO mappings can cause them to exist. - */ -out: - return pfn_to_page(pfn); + return vm_normal_page_pfn(vma, addr, pfn, pmd_val(pmd)); } =20 +/** + * vm_normal_folio_pmd() - Get the "struct folio" associated with a PMD + * @vma: The VMA mapping the @pmd. + * @addr: The address where the @pmd is mapped. + * @pmd: The PMD. + * + * Get the "struct folio" associated with a PMD. See vm_normal_page_pfn() + * for details. + * + * Return: Returns the "struct folio" if this is a "normal" mapping. Retur= ns + * NULL if this is a "special" mapping. + */ struct folio *vm_normal_folio_pmd(struct vm_area_struct *vma, unsigned long addr, pmd_t pmd) { --=20 2.50.1 From nobody Mon Oct 6 20:56:44 2025 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 77DA82BE64A for ; Thu, 17 Jul 2025 11:52:38 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1752753161; cv=none; b=GRp1pfBeeKPNP/vwW8oVDF85pOiyOpRA3eHTJ2lGnQVQDzmzYBlxz6RZAVSD3UyJ9w3+Yxn70Bc2NR/ujdL+0gsCf4NGF5gioZCeYeuR/exNurzhG1nQHCltK7Q1UNPx9cvOyIeINNBQJwXIjY8OM9Klz65PL3tkgE41+urvcNA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1752753161; c=relaxed/simple; bh=6/jjLRxyu6Gk21aB2ScFk9YQfi5MLdI3W8BYCgHrJ7Y=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=nlGESsmWWQOR9vTfU3o+6xKggv9XdVR8AcoGJUA99qqEKyxmrafaL6jFMPI/Iu4fvOp6XrJIW1XA/HeRAssD8CwzhDthqrd7rf/X+rTOsceyF8nyegUBGidM9736esOsDpGqIUNTIPSaVjfTUNDS+jBck/0nDCE9wgPvD52KXf8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=Z/3AM0dM; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="Z/3AM0dM" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1752753157; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=y93PTvMkvM029q6ldi/06ftGqYZT07G8pbjSDy6T3hs=; b=Z/3AM0dM5PwCbQza5Oi9+FpboGpcxweZx2NtowW27M5UhR5onZVY8DPTvVx1R9rXCKHCGD cGf6KEQnzIRL2qpu7X+9C2U3XunC6EfTFGo2rxDZZdU0fI+PLoIXrHoqpas6hQg4EGeyQt I4t069yR+tzwPp7pTNag29Pmddd+kQk= Received: from mail-wr1-f72.google.com (mail-wr1-f72.google.com [209.85.221.72]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-542-vJd0IpIGOWifGNhvpVK7qw-1; Thu, 17 Jul 2025 07:52:36 -0400 X-MC-Unique: vJd0IpIGOWifGNhvpVK7qw-1 X-Mimecast-MFC-AGG-ID: vJd0IpIGOWifGNhvpVK7qw_1752753155 Received: by mail-wr1-f72.google.com with SMTP id ffacd0b85a97d-3a4eb6fcd88so525464f8f.1 for ; Thu, 17 Jul 2025 04:52:36 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1752753155; x=1753357955; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=y93PTvMkvM029q6ldi/06ftGqYZT07G8pbjSDy6T3hs=; b=XAc35ZZgQHFu0EvGmdY/982U7/8DHtipil0vCPWnVCJB3ciyWKHtDfnKSjy8M6RcfH N+MbvO0FRmCh4PMTTPasuU/oF5Kxqc5qZdW5pnGsy6LgXUgy+5GfgyqPfoaZ/HY/tOIb Ss4H7jkv7gW9K3pZmjJGVr2O41K7YyYpSDwKtykJ4Hb3jJ0q4N6Ue2b2/0OcCif9bpDG DCT8zT6Cg+yOvDoFAAE5Y30ZDglCcFxS+vlvOEq35VPtpNoVHFYYePe3bJOfUBWsH5Zf yBBS63VUz1C002B+4+5JENkncfwBrA6HCjGGFo3W0t1azV+tVmeqbI5UIAYACD1clKCO kd8w== X-Gm-Message-State: AOJu0YwgAOdgvzbnL96ckWqYxtVa7r+P+L8k57EDY049q9WCo+xMJa4F zAF6HQzlaJs35ZOziri2k9cEHST6SU4riNgsH/U/Xa0TS0ZnQci74OU5gIk2TYhZONdKNEBZU8S CK6gvbJZRbq1ZP+SQpyCA+OiPGc0Q4wIYlBmFdkvLmXlKN21/Ur15ycG8OlhLWD3Dw11AwPYJCI rLfsK7eBGFiNfhTL2g6Vg+Wu8zVDuEvSA+c7HGTND9AofrUpea X-Gm-Gg: ASbGncux1N5Bibhuk1tCRB0a3ub4Hpn6KjkHymqQo4C800pza5ENpJQVGszSrtpU2gT ivXS8hW50mxMjC88vA8ZGRlRk8rY4Nkx79MTxJa4qYOdZZyBNc0hl7UBZWyHztApZNqEPE6Mu6p seJqVeWkqPi11rwNw8POPLuv6oZc1Znen+oOiYBMxS+wlSgp7s2DT0AB+yMF8IixE1DHzGAL9jo EAF1dZG9FJKQrbiwvV/L9k3BkEK+nDKZzHxZqkcUHSsJ9VbdLAEh797YBpbfzCzVkEXJo5MkxVw ugGlFl87G7ZlhsNjlFVHo8Suzq9Kbo0FvxN5iRnlg5n2hgm7BvpFiav7+L5uTyeEzeigFunpsCg ZlhKOcS2jMph7Egb8081aHDI= X-Received: by 2002:a5d:6f14:0:b0:3a4:f661:c3e0 with SMTP id ffacd0b85a97d-3b60e510baamr4180698f8f.45.1752753155028; Thu, 17 Jul 2025 04:52:35 -0700 (PDT) X-Google-Smtp-Source: AGHT+IEAqQmPlaR0MqxERo3NRLrhaU4oEs8iGAGrgiheQNLTojFB3jUgPbGxNACr3plB8i5Bf9h92A== X-Received: by 2002:a5d:6f14:0:b0:3a4:f661:c3e0 with SMTP id ffacd0b85a97d-3b60e510baamr4180643f8f.45.1752753154425; Thu, 17 Jul 2025 04:52:34 -0700 (PDT) Received: from localhost (p200300d82f1f36000dc826ee9aa9fdc7.dip0.t-ipconnect.de. [2003:d8:2f1f:3600:dc8:26ee:9aa9:fdc7]) by smtp.gmail.com with UTF8SMTPSA id ffacd0b85a97d-3b5e8bd18ffsm20281626f8f.9.2025.07.17.04.52.32 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Thu, 17 Jul 2025 04:52:33 -0700 (PDT) From: David Hildenbrand To: linux-kernel@vger.kernel.org Cc: linux-mm@kvack.org, xen-devel@lists.xenproject.org, linux-fsdevel@vger.kernel.org, nvdimm@lists.linux.dev, David Hildenbrand , Andrew Morton , Juergen Gross , Stefano Stabellini , Oleksandr Tyshchenko , Dan Williams , Matthew Wilcox , Jan Kara , Alexander Viro , Christian Brauner , Lorenzo Stoakes , "Liam R. Howlett" , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Zi Yan , Baolin Wang , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Jann Horn , Pedro Falcato , Hugh Dickins , Oscar Salvador , Lance Yang Subject: [PATCH v2 8/9] mm: introduce and use vm_normal_page_pud() Date: Thu, 17 Jul 2025 13:52:11 +0200 Message-ID: <20250717115212.1825089-9-david@redhat.com> X-Mailer: git-send-email 2.50.1 In-Reply-To: <20250717115212.1825089-1-david@redhat.com> References: <20250717115212.1825089-1-david@redhat.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Let's introduce vm_normal_page_pud(), which ends up being fairly simple because of our new common helpers and there not being a PUD-sized zero folio. Use vm_normal_page_pud() in folio_walk_start() to resolve a TODO, structuring the code like the other (pmd/pte) cases. Defer introducing vm_normal_folio_pud() until really used. Reviewed-by: Oscar Salvador Signed-off-by: David Hildenbrand Reviewed-by: Lorenzo Stoakes Reviewed-by: Wei Yang --- include/linux/mm.h | 2 ++ mm/memory.c | 27 +++++++++++++++++++++++++++ mm/pagewalk.c | 20 ++++++++++---------- 3 files changed, 39 insertions(+), 10 deletions(-) diff --git a/include/linux/mm.h b/include/linux/mm.h index abc47f1f307fb..0eb991262fbbf 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -2349,6 +2349,8 @@ struct folio *vm_normal_folio_pmd(struct vm_area_stru= ct *vma, unsigned long addr, pmd_t pmd); struct page *vm_normal_page_pmd(struct vm_area_struct *vma, unsigned long = addr, pmd_t pmd); +struct page *vm_normal_page_pud(struct vm_area_struct *vma, unsigned long = addr, + pud_t pud); =20 void zap_vma_ptes(struct vm_area_struct *vma, unsigned long address, unsigned long size); diff --git a/mm/memory.c b/mm/memory.c index c43ae5e4d7644..00a0d7ae3ba4a 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -796,6 +796,33 @@ struct folio *vm_normal_folio_pmd(struct vm_area_struc= t *vma, return page_folio(page); return NULL; } + +/** + * vm_normal_page_pud() - Get the "struct page" associated with a PUD + * @vma: The VMA mapping the @pud. + * @addr: The address where the @pud is mapped. + * @pud: The PUD. + * + * Get the "struct page" associated with a PUD. See vm_normal_page_pfn() + * for details. + * + * Return: Returns the "struct page" if this is a "normal" mapping. Returns + * NULL if this is a "special" mapping. + */ +struct page *vm_normal_page_pud(struct vm_area_struct *vma, + unsigned long addr, pud_t pud) +{ + unsigned long pfn =3D pud_pfn(pud); + + if (unlikely(pud_special(pud))) { + if (vma->vm_flags & (VM_PFNMAP | VM_MIXEDMAP)) + return NULL; + + print_bad_page_map(vma, addr, pud_val(pud), NULL); + return NULL; + } + return vm_normal_page_pfn(vma, addr, pfn, pud_val(pud)); +} #endif =20 /** diff --git a/mm/pagewalk.c b/mm/pagewalk.c index 648038247a8d2..c6753d370ff4e 100644 --- a/mm/pagewalk.c +++ b/mm/pagewalk.c @@ -902,23 +902,23 @@ struct folio *folio_walk_start(struct folio_walk *fw, fw->pudp =3D pudp; fw->pud =3D pud; =20 - /* - * TODO: FW_MIGRATION support for PUD migration entries - * once there are relevant users. - */ - if (!pud_present(pud) || pud_special(pud)) { + if (pud_none(pud)) { spin_unlock(ptl); goto not_found; - } else if (!pud_leaf(pud)) { + } else if (pud_present(pud) && !pud_leaf(pud)) { spin_unlock(ptl); goto pmd_table; + } else if (pud_present(pud)) { + page =3D vm_normal_page_pud(vma, addr, pud); + if (page) + goto found; } /* - * TODO: vm_normal_page_pud() will be handy once we want to - * support PUD mappings in VM_PFNMAP|VM_MIXEDMAP VMAs. + * TODO: FW_MIGRATION support for PUD migration entries + * once there are relevant users. */ - page =3D pud_page(pud); - goto found; + spin_unlock(ptl); + goto not_found; } =20 pmd_table: --=20 2.50.1 From nobody Mon Oct 6 20:56:44 2025 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass(p=quarantine dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1752753191; cv=none; d=zohomail.com; s=zohoarc; b=IlYn3lWf0mCxejwuWkvTjE58BbTmTQB3uWty8AxkMGKNW/8vnq2nnTWKSV0Hvp4oOlNay5Lt30n51rufY6LRYd07/m9TXtcvbIGnPMmq4NP1H5xXZsw8cOUffARzKkHipIkvkgKTb172VnLnWdA6tBxY3bLlB0yVRzwweZCjTAU= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1752753191; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=+EJnWBjNqAGa4cESjiW0EJbmUIJ/RdvXL+0h/9uaJzw=; b=WqJcXshz0Feqc48AzgurQMXsKam+bci78XKakWzsQD5NfcCRBfW4J5JiN3ERQWCFFnb3+TLmTqWmwUg8yzqCfRkDa+wR2NsmSTv05ep0TigBO9z7EPXYxHGTGj/o70qV5tQ72gdqa9hK63ET/HuNwNvjes7m+NoCWxUyrN4xBVs= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1752753191777206.59850201823906; Thu, 17 Jul 2025 04:53:11 -0700 (PDT) Received: from list by lists.xenproject.org with outflank-mailman.1046620.1417031 (Exim 4.92) (envelope-from ) id 1ucNAM-0003k4-EU; Thu, 17 Jul 2025 11:52:46 +0000 Received: by outflank-mailman (output) from mailman id 1046620.1417031; Thu, 17 Jul 2025 11:52:46 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1ucNAM-0003j6-As; Thu, 17 Jul 2025 11:52:46 +0000 Received: by outflank-mailman (input) for mailman id 1046620; Thu, 17 Jul 2025 11:52:44 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1ucNAK-0001Ft-P7 for xen-devel@lists.xenproject.org; Thu, 17 Jul 2025 11:52:44 +0000 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id 8ba529f6-6304-11f0-a319-13f23c93f187; Thu, 17 Jul 2025 13:52:43 +0200 (CEST) Received: from mail-wm1-f72.google.com (mail-wm1-f72.google.com [209.85.128.72]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-479-AqDfPHwvMgaPFA3ksQyRBA-1; Thu, 17 Jul 2025 07:52:38 -0400 Received: by mail-wm1-f72.google.com with SMTP id 5b1f17b1804b1-451d3f03b74so5321265e9.3 for ; Thu, 17 Jul 2025 04:52:38 -0700 (PDT) Received: from localhost (p200300d82f1f36000dc826ee9aa9fdc7.dip0.t-ipconnect.de. [2003:d8:2f1f:3600:dc8:26ee:9aa9:fdc7]) by smtp.gmail.com with UTF8SMTPSA id ffacd0b85a97d-3b5e8dc22a8sm20945546f8f.34.2025.07.17.04.52.35 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Thu, 17 Jul 2025 04:52:36 -0700 (PDT) X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 8ba529f6-6304-11f0-a319-13f23c93f187 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1752753162; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=+EJnWBjNqAGa4cESjiW0EJbmUIJ/RdvXL+0h/9uaJzw=; b=Q8isTOVz0DV8l5g9EBIqwl0s+9IDeYs86AfRPT6L2/kJZ9d+9DtAaCx0ybrJHpDIR7rbcl p+mN3eZ3vDdWbHqNM0R6+q+FNiUbf329Xj3mnTumXSqqLyYOvXffEAItcHKCMfNlkxiSYu Ag1KHSm8YBf/ow1BisnnhVdOOr4vFbc= X-MC-Unique: AqDfPHwvMgaPFA3ksQyRBA-1 X-Mimecast-MFC-AGG-ID: AqDfPHwvMgaPFA3ksQyRBA_1752753157 X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1752753157; x=1753357957; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=+EJnWBjNqAGa4cESjiW0EJbmUIJ/RdvXL+0h/9uaJzw=; b=dXPsTyA5Mz2tH8fGJZu7NInb2wGpZmqVRKGImzEVAMDUoKaPVDhN6Iz8LZg28hrWXO f0SsvV28l8k+vxcOBnwuFL7CxaVIr506x2+SwfODM9i4Q9ulYCslEVBxJYFtqL1yq3Gw 0VE20dixWbVfSmemJTAiId7Gr4J4RlelSoCVPeNV9zTB0j0KX6ok27VR0z575mQd64cP 3Fw2HADlMfDUB3APkgSVPVgJg1Mfl4mpmeHvZsvIIyYYeWrXah9bM0x2FTR3nNnLDXod WAj3HXvIgUxMkYGv2PTqZgVrsfdmh9uIp/dN3CWvBgl/MENAa4q7DJaBaBc/jlaWdE+H HZtQ== X-Forwarded-Encrypted: i=1; AJvYcCUBTsyUfMRnXUw1K0xpyKlc9LwoLlQwIrioJf/zeqoCEQvwE+qzlsbpvBz9/n1nuATZ6yqY9oJeodg=@lists.xenproject.org X-Gm-Message-State: AOJu0YwoB0r2hdq9FXEarfKZ1bPSTxMgxVh9ePS54NIDShxvDzMVyr8/ jtbPkv2M20j8sTpDL3umfJHZIMkSOc8U45hrBZBHOzRmOOB929Uyc6letTEihHDKD0dQy6UOeeS CilF+oDpnunwuF0yRsCWKDnOnYHl1ZkHCuQkEJMntPZWKtuQdTx9ywukTGULNEm7qdlIG X-Gm-Gg: ASbGncv3hXYKZYc73IufjXaIzecxyl01WJVInI9s3Y5niIIR7XSpoaN8jwkkr9/MlT/ Acvar6xEnI7TqIezucfBGnl1XdAXdRMNtOxbPrLQqKRuVE7NKnLDUUfMmTXq+cNG+8MnC75vdRq TEZdb5F8Odg0HzsKksagV/1APQF1T7r+Safzxnp6DcuEM1SsGVsb6fn0s0rN/e+0+eObuSx2q52 8Xh8jrsqYwHGhocmXFDbcp8o+izfSbpwehnkb8vUJ6TtEM9XrauUdRIhVfuy1wSQ+qMY1de0GUW 6ssY7rELwo+2TPwr73mQyyfG08ZmStWjoMEXJAv8YPhQdGsUJNgcUswtsUog7ACEQTVxR6zfKgl UJZIa87YCqly7qx1zW4E6IAY= X-Received: by 2002:a05:600c:870e:b0:456:285b:db3c with SMTP id 5b1f17b1804b1-456352d2ab1mr20378195e9.3.1752753157372; Thu, 17 Jul 2025 04:52:37 -0700 (PDT) X-Google-Smtp-Source: AGHT+IE3Vszf6T0cq3aQuR+QDmUBqbteOugAuzaA8OTawN8Aj0cx9bUSNlZT+0u55FTlbIHJovWeIQ== X-Received: by 2002:a05:600c:870e:b0:456:285b:db3c with SMTP id 5b1f17b1804b1-456352d2ab1mr20377615e9.3.1752753156790; Thu, 17 Jul 2025 04:52:36 -0700 (PDT) From: David Hildenbrand To: linux-kernel@vger.kernel.org Cc: linux-mm@kvack.org, xen-devel@lists.xenproject.org, linux-fsdevel@vger.kernel.org, nvdimm@lists.linux.dev, David Hildenbrand , Andrew Morton , Juergen Gross , Stefano Stabellini , Oleksandr Tyshchenko , Dan Williams , Matthew Wilcox , Jan Kara , Alexander Viro , Christian Brauner , Lorenzo Stoakes , "Liam R. Howlett" , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Zi Yan , Baolin Wang , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Jann Horn , Pedro Falcato , Hugh Dickins , Oscar Salvador , Lance Yang , David Vrabel Subject: [PATCH v2 9/9] mm: rename vm_ops->find_special_page() to vm_ops->find_normal_page() Date: Thu, 17 Jul 2025 13:52:12 +0200 Message-ID: <20250717115212.1825089-10-david@redhat.com> X-Mailer: git-send-email 2.50.1 In-Reply-To: <20250717115212.1825089-1-david@redhat.com> References: <20250717115212.1825089-1-david@redhat.com> MIME-Version: 1.0 X-Mimecast-Spam-Score: 0 X-Mimecast-MFC-PROC-ID: PG35J5UEZMz_9lE4puyCoRKG43VVxzoOrvdO6VmAyTw_1752753157 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1752753192879116600 Content-Type: text/plain; charset="utf-8"; x-default="true" ... and hide it behind a kconfig option. There is really no need for any !xen code to perform this check. The naming is a bit off: we want to find the "normal" page when a PTE was marked "special". So it's really not "finding a special" page. Improve the documentation, and add a comment in the code where XEN ends up performing the pte_mkspecial() through a hypercall. More details can be found in commit 923b2919e2c3 ("xen/gntdev: mark userspace PTEs as special on x86 PV guests"). Cc: David Vrabel Reviewed-by: Oscar Salvador Signed-off-by: David Hildenbrand Reviewed-by: Lorenzo Stoakes Reviewed-by: Wei Yang --- drivers/xen/Kconfig | 1 + drivers/xen/gntdev.c | 5 +++-- include/linux/mm.h | 18 +++++++++++++----- mm/Kconfig | 2 ++ mm/memory.c | 12 ++++++++++-- tools/testing/vma/vma_internal.h | 18 +++++++++++++----- 6 files changed, 42 insertions(+), 14 deletions(-) diff --git a/drivers/xen/Kconfig b/drivers/xen/Kconfig index 24f485827e039..f9a35ed266ecf 100644 --- a/drivers/xen/Kconfig +++ b/drivers/xen/Kconfig @@ -138,6 +138,7 @@ config XEN_GNTDEV depends on XEN default m select MMU_NOTIFIER + select FIND_NORMAL_PAGE help Allows userspace processes to use grants. =20 diff --git a/drivers/xen/gntdev.c b/drivers/xen/gntdev.c index 61faea1f06630..d1bc0dae2cdf9 100644 --- a/drivers/xen/gntdev.c +++ b/drivers/xen/gntdev.c @@ -309,6 +309,7 @@ static int find_grant_ptes(pte_t *pte, unsigned long ad= dr, void *data) BUG_ON(pgnr >=3D map->count); pte_maddr =3D arbitrary_virt_to_machine(pte).maddr; =20 + /* Note: this will perform a pte_mkspecial() through the hypercall. */ gnttab_set_map_op(&map->map_ops[pgnr], pte_maddr, flags, map->grants[pgnr].ref, map->grants[pgnr].domid); @@ -516,7 +517,7 @@ static void gntdev_vma_close(struct vm_area_struct *vma) gntdev_put_map(priv, map); } =20 -static struct page *gntdev_vma_find_special_page(struct vm_area_struct *vm= a, +static struct page *gntdev_vma_find_normal_page(struct vm_area_struct *vma, unsigned long addr) { struct gntdev_grant_map *map =3D vma->vm_private_data; @@ -527,7 +528,7 @@ static struct page *gntdev_vma_find_special_page(struct= vm_area_struct *vma, static const struct vm_operations_struct gntdev_vmops =3D { .open =3D gntdev_vma_open, .close =3D gntdev_vma_close, - .find_special_page =3D gntdev_vma_find_special_page, + .find_normal_page =3D gntdev_vma_find_normal_page, }; =20 /* ------------------------------------------------------------------ */ diff --git a/include/linux/mm.h b/include/linux/mm.h index 0eb991262fbbf..036800514aa90 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -648,13 +648,21 @@ struct vm_operations_struct { struct mempolicy *(*get_policy)(struct vm_area_struct *vma, unsigned long addr, pgoff_t *ilx); #endif +#ifdef CONFIG_FIND_NORMAL_PAGE /* - * Called by vm_normal_page() for special PTEs to find the - * page for @addr. This is useful if the default behavior - * (using pte_page()) would not find the correct page. + * Called by vm_normal_page() for special PTEs in @vma at @addr. This + * allows for returning a "normal" page from vm_normal_page() even + * though the PTE indicates that the "struct page" either does not exist + * or should not be touched: "special". + * + * Do not add new users: this really only works when a "normal" page + * was mapped, but then the PTE got changed to something weird (+ + * marked special) that would not make pte_pfn() identify the originally + * inserted page. */ - struct page *(*find_special_page)(struct vm_area_struct *vma, - unsigned long addr); + struct page *(*find_normal_page)(struct vm_area_struct *vma, + unsigned long addr); +#endif /* CONFIG_FIND_NORMAL_PAGE */ }; =20 #ifdef CONFIG_NUMA_BALANCING diff --git a/mm/Kconfig b/mm/Kconfig index 0287e8d94aea7..82c281b4f6937 100644 --- a/mm/Kconfig +++ b/mm/Kconfig @@ -1397,6 +1397,8 @@ config PT_RECLAIM =20 Note: now only empty user PTE page table pages will be reclaimed. =20 +config FIND_NORMAL_PAGE + def_bool n =20 source "mm/damon/Kconfig" =20 diff --git a/mm/memory.c b/mm/memory.c index 00a0d7ae3ba4a..52804ca343261 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -613,6 +613,12 @@ static void print_bad_page_map(struct vm_area_struct *= vma, * trivial. Secondly, an architecture may not have a spare page table * entry bit, which requires a more complicated scheme, described below. * + * With CONFIG_FIND_NORMAL_PAGE, we might have the "special" bit set on + * page table entries that actually map "normal" pages: however, that page + * cannot be looked up through the PFN stored in the page table entry, but + * instead will be looked up through vm_ops->find_normal_page(). So far, t= his + * only applies to PTEs. + * * A raw VM_PFNMAP mapping (ie. one that is not COWed) is always considere= d a * special mapping (even if there are underlying and valid "struct pages"). * COWed pages of a VM_PFNMAP are always normal. @@ -710,8 +716,10 @@ struct page *vm_normal_page(struct vm_area_struct *vma= , unsigned long addr, unsigned long pfn =3D pte_pfn(pte); =20 if (unlikely(pte_special(pte))) { - if (vma->vm_ops && vma->vm_ops->find_special_page) - return vma->vm_ops->find_special_page(vma, addr); +#ifdef CONFIG_FIND_NORMAL_PAGE + if (vma->vm_ops && vma->vm_ops->find_normal_page) + return vma->vm_ops->find_normal_page(vma, addr); +#endif /* CONFIG_FIND_NORMAL_PAGE */ if (vma->vm_flags & (VM_PFNMAP | VM_MIXEDMAP)) return NULL; if (is_zero_pfn(pfn)) diff --git a/tools/testing/vma/vma_internal.h b/tools/testing/vma/vma_inter= nal.h index 0fe52fd6782bf..8646af15a5fc0 100644 --- a/tools/testing/vma/vma_internal.h +++ b/tools/testing/vma/vma_internal.h @@ -467,13 +467,21 @@ struct vm_operations_struct { struct mempolicy *(*get_policy)(struct vm_area_struct *vma, unsigned long addr, pgoff_t *ilx); #endif +#ifdef CONFIG_FIND_NORMAL_PAGE /* - * Called by vm_normal_page() for special PTEs to find the - * page for @addr. This is useful if the default behavior - * (using pte_page()) would not find the correct page. + * Called by vm_normal_page() for special PTEs in @vma at @addr. This + * allows for returning a "normal" page from vm_normal_page() even + * though the PTE indicates that the "struct page" either does not exist + * or should not be touched: "special". + * + * Do not add new users: this really only works when a "normal" page + * was mapped, but then the PTE got changed to something weird (+ + * marked special) that would not make pte_pfn() identify the originally + * inserted page. */ - struct page *(*find_special_page)(struct vm_area_struct *vma, - unsigned long addr); + struct page *(*find_normal_page)(struct vm_area_struct *vma, + unsigned long addr); +#endif /* CONFIG_FIND_NORMAL_PAGE */ }; =20 struct vm_unmapped_area_info { --=20 2.50.1