From nobody Tue Jun 30 03:39:48 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 85582C28CF5 for ; Wed, 26 Jan 2022 06:05:32 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234804AbiAZGFb (ORCPT ); Wed, 26 Jan 2022 01:05:31 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53762 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234641AbiAZGFT (ORCPT ); Wed, 26 Jan 2022 01:05:19 -0500 Received: from mail-qv1-xf31.google.com (mail-qv1-xf31.google.com [IPv6:2607:f8b0:4864:20::f31]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E4056C061744 for ; Tue, 25 Jan 2022 22:05:18 -0800 (PST) Received: by mail-qv1-xf31.google.com with SMTP id e20so19854030qvu.7 for ; Tue, 25 Jan 2022 22:05:18 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=aWNRX9QsBPHLZOFVrwNXiWXc+ZQSkTRboCbVzmJISWM=; b=LlMcMekRBawFA/p+rIOXAZAiAa1k3SdlK7Hkpzh6hMhMDHMLaRQKA8ML9fwHV6llAJ Bdxeiz9HYmUpAchr0GfAWwJBjAWkCI7hp2dT6wD+ckNBhnwWq6ACHfH7aUSgQdr2rkI6 mpHA6t/nLQ+gI7EEx382Dl6maFaESiD2TqeWZhSl0Wp5MWF7QFaUj2Mq/Xv0SxWLoibg 4/IQbSGz3QzAafFPlJu6lx2YmPs8ClFWtFFBtMKmssnPMca5EWGQVPAvZhtySQ4pIdN/ 28UuDSoc27Z6S7UpTlIDARr3keJuGWON+9QTLfGn8icD8SJS61b3RMeKZK/dfDT8mLNu 9KFQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=aWNRX9QsBPHLZOFVrwNXiWXc+ZQSkTRboCbVzmJISWM=; b=sZ/qPIujAe/C3wXJ4knTb8z1DZYZpWQR/ozmBAzzBohks3vlHMjj9Nd1AIePFaGgCK GRy9o/PFjti5vpa3aBPg2Zkd7+LI1s1S+g3XQ5w5cgmYU/tnl799E3xdNEyq5dXqwcFN lcYktp+pbWpvsn+lgEv4adfRk44uqFIrbN0dytprSZs3OtDAyWTS/6YYAgTfOclw8jj2 /OE7yGNukrp18cnzylVnpi9M4HR4HwEC5o+Pv0bSjkEL/ab0ymjJVjM/vtj4Ua6Pizqy mX0PsWjcVZZLCwPtSxOfL/bTtFhhvIVZ9Q18kTbsomekG5VrtD0GwY2OkcJm5oHE/Mbh QJAg== X-Gm-Message-State: AOAM530V8+NvGbvojvB31UC3A8xUr6vE8gUz0pt/ty21TEoQ+26MbiZh f113gIyHgdU61R/KxbsEpSFv8A== X-Google-Smtp-Source: ABdhPJwVByTrnm101Bf8YAmr4+nFjtChqbu8bu6/f6QCYcLdWej0+Cltb2+DvquGjJnoBQkLcKwHiA== X-Received: by 2002:a05:6214:1cca:: with SMTP id g10mr22397753qvd.36.1643177118106; Tue, 25 Jan 2022 22:05:18 -0800 (PST) Received: from soleen.c.googlers.com.com (189.216.85.34.bc.googleusercontent.com. [34.85.216.189]) by smtp.gmail.com with ESMTPSA id o19sm9856699qta.40.2022.01.25.22.05.17 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 25 Jan 2022 22:05:17 -0800 (PST) From: Pasha Tatashin To: pasha.tatashin@soleen.com, linux-kernel@vger.kernel.org, linux-mm@kvack.org, akpm@linux-foundation.org, rientjes@google.com, pjt@google.com, weixugc@google.com, gthelen@google.com, mingo@redhat.com, will@kernel.org, rppt@kernel.org, dave.hansen@linux.intel.com, hpa@zytor.com, aneesh.kumar@linux.ibm.com, jirislaby@kernel.org, songmuchun@bytedance.com, qydwhotmail@gmail.com, hughd@google.com, ziy@nvidia.com, anshuman.khandual@arm.com Subject: [PATCH v3 1/4] mm/debug_vm_pgtable: remove pte entry from the page table Date: Wed, 26 Jan 2022 06:05:11 +0000 Message-Id: <20220126060514.1574935-2-pasha.tatashin@soleen.com> X-Mailer: git-send-email 2.35.0.rc0.227.g00780c9af4-goog In-Reply-To: <20220126060514.1574935-1-pasha.tatashin@soleen.com> References: <20220126060514.1574935-1-pasha.tatashin@soleen.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" The pte entry that is used in pte_advanced_tests() is never removed from the page table at the end of the test. The issue is detected by page_table_check, to repro compile kernel with the following configs: CONFIG_DEBUG_VM_PGTABLE=3Dy CONFIG_PAGE_TABLE_CHECK=3Dy CONFIG_PAGE_TABLE_CHECK_ENFORCED=3Dy During the boot the following BUG is printed: [ 2.262821] debug_vm_pgtable: [debug_vm_pgtable ]: Validating architecture page table helpers [ 2.276826] ------------[ cut here ]------------ [ 2.280426] kernel BUG at mm/page_table_check.c:162! [ 2.284118] invalid opcode: 0000 [#1] PREEMPT SMP PTI [ 2.287787] CPU: 0 PID: 1 Comm: swapper/0 Not tainted 5.16.0-11413-g2c271fe77d52 #3 [ 2.293226] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.15.0-0-g2dd4b9b3f840-prebuilt.qemu.org 04/01/2014 ... The entry should be properly removed from the page table before the page is released to the free list. Fixes: a5c3b9ffb0f4 ("mm/debug_vm_pgtable: add tests validating advanced ar= ch page table helpers") Signed-off-by: Pasha Tatashin Reviewed-by: Zi Yan Tested-by: Zi Yan Acked-by: David Rientjes --- mm/debug_vm_pgtable.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/mm/debug_vm_pgtable.c b/mm/debug_vm_pgtable.c index a7ac97c76762..db2abd9e415b 100644 --- a/mm/debug_vm_pgtable.c +++ b/mm/debug_vm_pgtable.c @@ -171,6 +171,8 @@ static void __init pte_advanced_tests(struct pgtable_de= bug_args *args) ptep_test_and_clear_young(args->vma, args->vaddr, args->ptep); pte =3D ptep_get(args->ptep); WARN_ON(pte_young(pte)); + + ptep_get_and_clear_full(args->mm, args->vaddr, args->ptep, 1); } =20 static void __init pte_savedwrite_tests(struct pgtable_debug_args *args) --=20 2.35.0.rc0.227.g00780c9af4-goog From nobody Tue Jun 30 03:39:48 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0F65FC2BA4C for ; Wed, 26 Jan 2022 06:05:34 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234891AbiAZGFd (ORCPT ); Wed, 26 Jan 2022 01:05:33 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53770 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234685AbiAZGFU (ORCPT ); Wed, 26 Jan 2022 01:05:20 -0500 Received: from mail-qt1-x82d.google.com (mail-qt1-x82d.google.com [IPv6:2607:f8b0:4864:20::82d]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D5E80C061747 for ; Tue, 25 Jan 2022 22:05:19 -0800 (PST) Received: by mail-qt1-x82d.google.com with SMTP id c19so26335457qtx.3 for ; Tue, 25 Jan 2022 22:05:19 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=R7V05PwaNjZnBTJ0+d7ZFZvO5dL9q1xRZty8VkA9hmQ=; b=bYJUNPpR1cZ5cpCeIRGCfDxZJpYmpPoh66ZJK77ukWwzsfKXMtKjFR0oyDl2PrVsRW IjZbI251p7s2yxGbyhaCpLtUYBe6eQ/H9o9djosXMfNCEvGwGBN+R0lcfbPMC5Cksznp HzEoAy39YDsLI/M5Ax6D0T9dXeKhcoYrDGg62jHcw+RFE+cRQEohFqYIRggXfbAR09jh c2sbLnBREwPZM/oV0w9qlL4dm7bHOUr+bSxvw/pnn/B3Aa/Q5IwY6gYNswVBk4dgWm/4 tybcJbilgAp6um5hCw1b5mXjM2VS1BA4dZTRuLQV21VwHvpW96HOLQcJFswBr1TlyuL5 O08Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=R7V05PwaNjZnBTJ0+d7ZFZvO5dL9q1xRZty8VkA9hmQ=; b=xWltCoQN9CeLcrx7VTr/vftGbbyCYz1AoqDeEyj33Z1QY2WSQz0GPjog7MHPTV5Jn9 GLLXPQUMg1rC+0fJdqD+ILep55LCY2YR2MwfIP6mg9HQSwcJ4TiKuK+iykHPGluKsUas qMBU0qASJkzaMUGsWP+o02Wv+z9HFE6oeCDgEb6YwIdOgvDpC44J1TAATKpcG2y26Gi0 VMzASSZTW6GQ/vAQ7JwYcKjPCiDos298r6DuMEwjKbOJbpE+RRzPMZpw+JPfoZloQYW7 Y3ZWcqoW2xgBQaEblIlo7pRy6KdPiSwkGAd+ibmoL43eP8YE21yn/b6jf1pcEp8TA0iN 0h1w== X-Gm-Message-State: AOAM5308XwjutAChUAmFKNUtZRZjVOqYvmSiY220YyDYkZRDZrp2Fgwo ipdC6GAxCaOkOK4w3tXSotFdXQ== X-Google-Smtp-Source: ABdhPJyGJu1wLWlwWE4gJcN7imPL+nBoU18Asvh3ix2RWYM+mVME0JOpFCDhcqGNLxudLxhV3f6eqg== X-Received: by 2002:a05:622a:43:: with SMTP id y3mr19448205qtw.347.1643177119022; Tue, 25 Jan 2022 22:05:19 -0800 (PST) Received: from soleen.c.googlers.com.com (189.216.85.34.bc.googleusercontent.com. [34.85.216.189]) by smtp.gmail.com with ESMTPSA id o19sm9856699qta.40.2022.01.25.22.05.18 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 25 Jan 2022 22:05:18 -0800 (PST) From: Pasha Tatashin To: pasha.tatashin@soleen.com, linux-kernel@vger.kernel.org, linux-mm@kvack.org, akpm@linux-foundation.org, rientjes@google.com, pjt@google.com, weixugc@google.com, gthelen@google.com, mingo@redhat.com, will@kernel.org, rppt@kernel.org, dave.hansen@linux.intel.com, hpa@zytor.com, aneesh.kumar@linux.ibm.com, jirislaby@kernel.org, songmuchun@bytedance.com, qydwhotmail@gmail.com, hughd@google.com, ziy@nvidia.com, anshuman.khandual@arm.com Subject: [PATCH v3 2/4] mm/page_table_check: use unsigned long for page counters and cleanup Date: Wed, 26 Jan 2022 06:05:12 +0000 Message-Id: <20220126060514.1574935-3-pasha.tatashin@soleen.com> X-Mailer: git-send-email 2.35.0.rc0.227.g00780c9af4-goog In-Reply-To: <20220126060514.1574935-1-pasha.tatashin@soleen.com> References: <20220126060514.1574935-1-pasha.tatashin@soleen.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" For the consistency, use "unsigned long" for all page counters. Also, reduce code duplication by calling __page_table_check_*_clear() from __page_table_check_*_set() functions. Signed-off-by: Pasha Tatashin Reviewed-by: Wei Xu --- mm/page_table_check.c | 35 +++++++---------------------------- 1 file changed, 7 insertions(+), 28 deletions(-) diff --git a/mm/page_table_check.c b/mm/page_table_check.c index 7504e7caa2a1..c61d7ebe13b1 100644 --- a/mm/page_table_check.c +++ b/mm/page_table_check.c @@ -86,8 +86,8 @@ static void page_table_check_clear(struct mm_struct *mm, = unsigned long addr, { struct page_ext *page_ext; struct page *page; + unsigned long i; bool anon; - int i; =20 if (!pfn_valid(pfn)) return; @@ -121,8 +121,8 @@ static void page_table_check_set(struct mm_struct *mm, = unsigned long addr, { struct page_ext *page_ext; struct page *page; + unsigned long i; bool anon; - int i; =20 if (!pfn_valid(pfn)) return; @@ -152,10 +152,10 @@ static void page_table_check_set(struct mm_struct *mm= , unsigned long addr, void __page_table_check_zero(struct page *page, unsigned int order) { struct page_ext *page_ext =3D lookup_page_ext(page); - int i; + unsigned long i; =20 BUG_ON(!page_ext); - for (i =3D 0; i < (1 << order); i++) { + for (i =3D 0; i < (1ul << order); i++) { struct page_table_check *ptc =3D get_page_table_check(page_ext); =20 BUG_ON(atomic_read(&ptc->anon_map_count)); @@ -206,17 +206,10 @@ EXPORT_SYMBOL(__page_table_check_pud_clear); void __page_table_check_pte_set(struct mm_struct *mm, unsigned long addr, pte_t *ptep, pte_t pte) { - pte_t old_pte; - if (&init_mm =3D=3D mm) return; =20 - old_pte =3D *ptep; - if (pte_user_accessible_page(old_pte)) { - page_table_check_clear(mm, addr, pte_pfn(old_pte), - PAGE_SIZE >> PAGE_SHIFT); - } - + __page_table_check_pte_clear(mm, addr, *ptep); if (pte_user_accessible_page(pte)) { page_table_check_set(mm, addr, pte_pfn(pte), PAGE_SIZE >> PAGE_SHIFT, @@ -228,17 +221,10 @@ EXPORT_SYMBOL(__page_table_check_pte_set); void __page_table_check_pmd_set(struct mm_struct *mm, unsigned long addr, pmd_t *pmdp, pmd_t pmd) { - pmd_t old_pmd; - if (&init_mm =3D=3D mm) return; =20 - old_pmd =3D *pmdp; - if (pmd_user_accessible_page(old_pmd)) { - page_table_check_clear(mm, addr, pmd_pfn(old_pmd), - PMD_PAGE_SIZE >> PAGE_SHIFT); - } - + __page_table_check_pmd_clear(mm, addr, *pmdp); if (pmd_user_accessible_page(pmd)) { page_table_check_set(mm, addr, pmd_pfn(pmd), PMD_PAGE_SIZE >> PAGE_SHIFT, @@ -250,17 +236,10 @@ EXPORT_SYMBOL(__page_table_check_pmd_set); void __page_table_check_pud_set(struct mm_struct *mm, unsigned long addr, pud_t *pudp, pud_t pud) { - pud_t old_pud; - if (&init_mm =3D=3D mm) return; =20 - old_pud =3D *pudp; - if (pud_user_accessible_page(old_pud)) { - page_table_check_clear(mm, addr, pud_pfn(old_pud), - PUD_PAGE_SIZE >> PAGE_SHIFT); - } - + __page_table_check_pud_clear(mm, addr, *pudp); if (pud_user_accessible_page(pud)) { page_table_check_set(mm, addr, pud_pfn(pud), PUD_PAGE_SIZE >> PAGE_SHIFT, --=20 2.35.0.rc0.227.g00780c9af4-goog From nobody Tue Jun 30 03:39:48 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 60028C28CF5 for ; Wed, 26 Jan 2022 06:05:41 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235114AbiAZGFg (ORCPT ); Wed, 26 Jan 2022 01:05:36 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53772 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234569AbiAZGFU (ORCPT ); Wed, 26 Jan 2022 01:05:20 -0500 Received: from mail-qt1-x830.google.com (mail-qt1-x830.google.com [IPv6:2607:f8b0:4864:20::830]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id BBDF6C061748 for ; Tue, 25 Jan 2022 22:05:20 -0800 (PST) Received: by mail-qt1-x830.google.com with SMTP id c19so26335477qtx.3 for ; Tue, 25 Jan 2022 22:05:20 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=HOl6dnjRSlkzFWIwAEZQBaaOxfPRDLrxvpvmNPVil3I=; b=jwFuMnHWqoNk8uXeaYsGmMgbTXTo5euHb6/esSzRi8Hx0i9wGtS6JDG3u8LGq6AmU4 eFdQHw+wFC2IAD7gAkutikIJJdEwF7jOdPz3HURvZDFMbhCd/ezSO9xcpJINsMjiXzM/ ux3ThHF0mwzLaaRsxHG+POu362xUihatB6ZiznsjtQ4fvhmMxxbd8YAbcJLjLQXGr+jW KGJA/EIVl7kOjJRSIaFq3sieVyL0O4O64ErJ4CMN2OkScAJfguQeZMBZ0kmND05wNMNY hqv9KmXfIvyrrebEnHTN4PTse2Py7UlyFStajrNbvPv+9NnRYJab498z0qxxx6Ztv/bY Sltw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=HOl6dnjRSlkzFWIwAEZQBaaOxfPRDLrxvpvmNPVil3I=; b=McniQ7QGRGCreQwOOZ+35kc3DNb5u/vyCm2VUeJsre+R0yr9dorexnkNClRq+YQc/0 KvEdRuwQnq8LF6NpOrQuOjkRQlchIfyeO7dvX2xw3HQE/XPtL4q/80awl8X05bAN9bAP dgQkLM3AL2WFh5MeG07XEW3083DL/0Rot6lJLJ5cQoFbplvNR74yLldgmRC5fJlRYoRo pIod7SPWcysYXUNP7b6ApBoGYN2QJ9J3x9rxD1uPJEmlKm0DcC42rZfUxyP8A8nsxH7p p/XLF4IHUVYRloDutZjxvqLoKafH3xlVodT/iHw7/urPlUFudRTBYpdB56Vi7nLJ7TPn WLYQ== X-Gm-Message-State: AOAM530qfWiaLWWwJyflqYvgyxqejuSmHmRe+/uf9embAcbNnWhL/ep8 jQkEK3W4p/eYEpFW9zxRvKFtHA== X-Google-Smtp-Source: ABdhPJxawlYifreSuisEPDkI9IQt+vr1qpvjrerAOfVLQ1Jx0TsXCwSpoyobPIpCZnLAgMF70g7lVg== X-Received: by 2002:ac8:5b01:: with SMTP id m1mr19121273qtw.360.1643177119921; Tue, 25 Jan 2022 22:05:19 -0800 (PST) Received: from soleen.c.googlers.com.com (189.216.85.34.bc.googleusercontent.com. [34.85.216.189]) by smtp.gmail.com with ESMTPSA id o19sm9856699qta.40.2022.01.25.22.05.19 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 25 Jan 2022 22:05:19 -0800 (PST) From: Pasha Tatashin To: pasha.tatashin@soleen.com, linux-kernel@vger.kernel.org, linux-mm@kvack.org, akpm@linux-foundation.org, rientjes@google.com, pjt@google.com, weixugc@google.com, gthelen@google.com, mingo@redhat.com, will@kernel.org, rppt@kernel.org, dave.hansen@linux.intel.com, hpa@zytor.com, aneesh.kumar@linux.ibm.com, jirislaby@kernel.org, songmuchun@bytedance.com, qydwhotmail@gmail.com, hughd@google.com, ziy@nvidia.com, anshuman.khandual@arm.com Subject: [PATCH v3 3/4] mm/khugepaged: unify collapse pmd clear, flush and free Date: Wed, 26 Jan 2022 06:05:13 +0000 Message-Id: <20220126060514.1574935-4-pasha.tatashin@soleen.com> X-Mailer: git-send-email 2.35.0.rc0.227.g00780c9af4-goog In-Reply-To: <20220126060514.1574935-1-pasha.tatashin@soleen.com> References: <20220126060514.1574935-1-pasha.tatashin@soleen.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Unify the code that flushes, clears pmd entry, and frees the PTE table level into a new function collapse_and_free_pmd(). This clean-up is useful as in the next patch we will add another call to this function to iterate through PTE prior to freeing the level for page table check. Signed-off-by: Pasha Tatashin --- mm/khugepaged.c | 32 ++++++++++++++++---------------- 1 file changed, 16 insertions(+), 16 deletions(-) diff --git a/mm/khugepaged.c b/mm/khugepaged.c index 35f14d0a00a6..440112355ffe 100644 --- a/mm/khugepaged.c +++ b/mm/khugepaged.c @@ -1416,6 +1416,17 @@ static int khugepaged_add_pte_mapped_thp(struct mm_s= truct *mm, return 0; } =20 +static void collapse_and_free_pmd(struct mm_struct *mm, struct vm_area_str= uct *vma, + unsigned long addr, pmd_t *pmdp) +{ + spinlock_t *ptl =3D pmd_lock(vma->vm_mm, pmdp); + pmd_t pmd =3D pmdp_collapse_flush(vma, addr, pmdp); + + spin_unlock(ptl); + mm_dec_nr_ptes(mm); + pte_free(mm, pmd_pgtable(pmd)); +} + /** * collapse_pte_mapped_thp - Try to collapse a pte-mapped THP for mm at * address haddr. @@ -1433,7 +1444,7 @@ void collapse_pte_mapped_thp(struct mm_struct *mm, un= signed long addr) struct vm_area_struct *vma =3D find_vma(mm, haddr); struct page *hpage; pte_t *start_pte, *pte; - pmd_t *pmd, _pmd; + pmd_t *pmd; spinlock_t *ptl; int count =3D 0; int i; @@ -1509,12 +1520,7 @@ void collapse_pte_mapped_thp(struct mm_struct *mm, u= nsigned long addr) } =20 /* step 4: collapse pmd */ - ptl =3D pmd_lock(vma->vm_mm, pmd); - _pmd =3D pmdp_collapse_flush(vma, haddr, pmd); - spin_unlock(ptl); - mm_dec_nr_ptes(mm); - pte_free(mm, pmd_pgtable(_pmd)); - + collapse_and_free_pmd(mm, vma, haddr, pmd); drop_hpage: unlock_page(hpage); put_page(hpage); @@ -1552,7 +1558,7 @@ static void retract_page_tables(struct address_space = *mapping, pgoff_t pgoff) struct vm_area_struct *vma; struct mm_struct *mm; unsigned long addr; - pmd_t *pmd, _pmd; + pmd_t *pmd; =20 i_mmap_lock_write(mapping); vma_interval_tree_foreach(vma, &mapping->i_mmap, pgoff, pgoff) { @@ -1591,14 +1597,8 @@ static void retract_page_tables(struct address_space= *mapping, pgoff_t pgoff) * reverse order. Trylock is a way to avoid deadlock. */ if (mmap_write_trylock(mm)) { - if (!khugepaged_test_exit(mm)) { - spinlock_t *ptl =3D pmd_lock(mm, pmd); - /* assume page table is clear */ - _pmd =3D pmdp_collapse_flush(vma, addr, pmd); - spin_unlock(ptl); - mm_dec_nr_ptes(mm); - pte_free(mm, pmd_pgtable(_pmd)); - } + if (!khugepaged_test_exit(mm)) + collapse_and_free_pmd(mm, vma, addr, pmd); mmap_write_unlock(mm); } else { /* Try again later */ --=20 2.35.0.rc0.227.g00780c9af4-goog From nobody Tue Jun 30 03:39:48 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2FFDEC28CF5 for ; Wed, 26 Jan 2022 06:05:44 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235391AbiAZGFm (ORCPT ); Wed, 26 Jan 2022 01:05:42 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53780 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234593AbiAZGFW (ORCPT ); Wed, 26 Jan 2022 01:05:22 -0500 Received: from mail-qv1-xf29.google.com (mail-qv1-xf29.google.com [IPv6:2607:f8b0:4864:20::f29]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id BDF9BC061749 for ; Tue, 25 Jan 2022 22:05:21 -0800 (PST) Received: by mail-qv1-xf29.google.com with SMTP id e20so19854156qvu.7 for ; Tue, 25 Jan 2022 22:05:21 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=NVid4lwIGU/SbGxET47lcYa9PMp6oZkK6MJSbQCBUP8=; b=VHecgKNFLp6+K8o+yvV6rnlGSNdQlBeJONWAcseLydDnJtbkgoNf0Rw/8Lz/nVAGfV d+hM2tCC86PkP2UUDjdPUnhcUKD1ufG4VzYPbBCK0KNotR7lUwePnhxeYXoFEzh7mcQe w0WJTijjWGUmQZAnUkLRrtcvcEqtQwWd18lPktYpYgcpP67/DcJrLIWRUZUfjeSlMG4Q yx3YUewSwqiwIAjSNilaNWM274WwyWPNRzWFJdLX1tW6rvdv5XrCcv5yq+rBJ44oG2gV VKkBDgU8utXoZDbGBLgYdEGpFCAlDm5eojRZ9+CCDFsXjMdvwtWKsvoy86o9aHx/FLn4 lt+Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=NVid4lwIGU/SbGxET47lcYa9PMp6oZkK6MJSbQCBUP8=; b=bo+ZpiD5WSr1m9t1On7txcgVzVXVZkwm2yfhwF7rNr7H+j6xOFe3PCx36TeeqWsQHz fclQxVWPWozo+BUpqR15HIhjgDaWhFcT1rjLqEgzXCxAK6MSHIcCKGyIMWM5GNd7tYHy 83ZlsDOX42OjAdBT7KT5HTEnu4Ga2tYjIngBYtU6y4K9Sb8Sqq9b/x4DjAdj4xs0fWhC QUNCwmzE5HRf045ZKGWKvZCahAzAR25Rku+ccZiTFK5RDMGmzS70OI/3oGQbq1MITqEa cIcj5CwgIMVUSU0MUgHp0BdUxNFihYMnjezqcph9vJUSvwtnSUfMVteST6TMiGo5bY+L 0MQg== X-Gm-Message-State: AOAM531wU2nrPJv9t2YAEilje29nltgxEFLARyBTc/TuihCbJ0Z/MEQe baVWLjb6Bns6S5Qm8OFnVj2f1A== X-Google-Smtp-Source: ABdhPJzd2K1nQEX6RdRaNxBEBXX9UK+PMvC+jASppza1eRowqKv+fF5BXklIge08ffTBviDpvDBTEg== X-Received: by 2002:a05:6214:2302:: with SMTP id gc2mr22470192qvb.126.1643177120814; Tue, 25 Jan 2022 22:05:20 -0800 (PST) Received: from soleen.c.googlers.com.com (189.216.85.34.bc.googleusercontent.com. [34.85.216.189]) by smtp.gmail.com with ESMTPSA id o19sm9856699qta.40.2022.01.25.22.05.19 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 25 Jan 2022 22:05:20 -0800 (PST) From: Pasha Tatashin To: pasha.tatashin@soleen.com, linux-kernel@vger.kernel.org, linux-mm@kvack.org, akpm@linux-foundation.org, rientjes@google.com, pjt@google.com, weixugc@google.com, gthelen@google.com, mingo@redhat.com, will@kernel.org, rppt@kernel.org, dave.hansen@linux.intel.com, hpa@zytor.com, aneesh.kumar@linux.ibm.com, jirislaby@kernel.org, songmuchun@bytedance.com, qydwhotmail@gmail.com, hughd@google.com, ziy@nvidia.com, anshuman.khandual@arm.com Subject: [PATCH v3 4/4] mm/page_table_check: check entries at pmd levels Date: Wed, 26 Jan 2022 06:05:14 +0000 Message-Id: <20220126060514.1574935-5-pasha.tatashin@soleen.com> X-Mailer: git-send-email 2.35.0.rc0.227.g00780c9af4-goog In-Reply-To: <20220126060514.1574935-1-pasha.tatashin@soleen.com> References: <20220126060514.1574935-1-pasha.tatashin@soleen.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" syzbot detected a case where the page table counters were not properly updated. syzkaller login: ------------[ cut here ]------------ kernel BUG at mm/page_table_check.c:162! invalid opcode: 0000 [#1] PREEMPT SMP KASAN CPU: 0 PID: 3099 Comm: pasha Not tainted 5.16.0+ #48 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIO4 RIP: 0010:__page_table_check_zero+0x159/0x1a0 Code: 7d 3a b2 ff 45 39 f5 74 2a e8 43 38 b2 ff 4d 85 e4 01 RSP: 0018:ffff888010667418 EFLAGS: 00010293 RAX: 0000000000000000 RBX: 0000000000000001 RCX: 0000000000 RDX: ffff88800cea8680 RSI: ffffffff81becaf9 RDI: 0000000003 RBP: ffff888010667450 R08: 0000000000000001 R09: 0000000000 R10: ffffffff81becaab R11: 0000000000000001 R12: ffff888008 R13: 0000000000000001 R14: 0000000000000200 R15: dffffc0000 FS: 0000000000000000(0000) GS:ffff888035e00000(0000) knlG0 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 00007ffd875cad00 CR3: 00000000094ce000 CR4: 0000000000 DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000 DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000 Call Trace: free_pcp_prepare+0x3be/0xaa0 free_unref_page+0x1c/0x650 ? trace_hardirqs_on+0x6a/0x1d0 free_compound_page+0xec/0x130 free_transhuge_page+0x1be/0x260 __put_compound_page+0x90/0xd0 release_pages+0x54c/0x1060 ? filemap_remove_folio+0x161/0x210 ? lock_downgrade+0x720/0x720 ? __put_page+0x150/0x150 ? filemap_free_folio+0x164/0x350 __pagevec_release+0x7c/0x110 shmem_undo_range+0x85e/0x1250 ... The repro involved having a huge page that is split due to uprobe event temporarily replacing one of the pages in the huge page. Later the huge page was combined again, but the counters were off, as the PTE level was not properly updated. Make sure that when PMD is cleared and prior to freeing the level the PTEs are updated. Fixes: df4e817b7108 ("mm: page table check") Signed-off-by: Pasha Tatashin --- include/linux/page_table_check.h | 18 ++++++++++++++++++ mm/khugepaged.c | 2 ++ mm/page_table_check.c | 21 +++++++++++++++++++++ 3 files changed, 41 insertions(+) diff --git a/include/linux/page_table_check.h b/include/linux/page_table_ch= eck.h index 38cace1da7b6..e88bbe37727b 100644 --- a/include/linux/page_table_check.h +++ b/include/linux/page_table_check.h @@ -26,6 +26,8 @@ void __page_table_check_pmd_set(struct mm_struct *mm, uns= igned long addr, pmd_t *pmdp, pmd_t pmd); void __page_table_check_pud_set(struct mm_struct *mm, unsigned long addr, pud_t *pudp, pud_t pud); +void __page_table_check_pmd_clear_full(struct mm_struct *mm, unsigned long= addr, + pmd_t pmd); =20 static inline void page_table_check_alloc(struct page *page, unsigned int = order) { @@ -100,6 +102,16 @@ static inline void page_table_check_pud_set(struct mm_= struct *mm, __page_table_check_pud_set(mm, addr, pudp, pud); } =20 +static inline void page_table_check_pmd_clear_full(struct mm_struct *mm, + unsigned long addr, + pmd_t pmd) +{ + if (static_branch_likely(&page_table_check_disabled)) + return; + + __page_table_check_pmd_clear_full(mm, addr, pmd); +} + #else =20 static inline void page_table_check_alloc(struct page *page, unsigned int = order) @@ -143,5 +155,11 @@ static inline void page_table_check_pud_set(struct mm_= struct *mm, { } =20 +static inline void page_table_check_pmd_clear_full(struct mm_struct *mm, + unsigned long addr, + pmd_t pmd) +{ +} + #endif /* CONFIG_PAGE_TABLE_CHECK */ #endif /* __LINUX_PAGE_TABLE_CHECK_H */ diff --git a/mm/khugepaged.c b/mm/khugepaged.c index 440112355ffe..eefe3706f6c2 100644 --- a/mm/khugepaged.c +++ b/mm/khugepaged.c @@ -16,6 +16,7 @@ #include #include #include +#include #include #include =20 @@ -1424,6 +1425,7 @@ static void collapse_and_free_pmd(struct mm_struct *m= m, struct vm_area_struct *v =20 spin_unlock(ptl); mm_dec_nr_ptes(mm); + page_table_check_pmd_clear_full(mm, addr, pmd); pte_free(mm, pmd_pgtable(pmd)); } =20 diff --git a/mm/page_table_check.c b/mm/page_table_check.c index c61d7ebe13b1..251f95a808b4 100644 --- a/mm/page_table_check.c +++ b/mm/page_table_check.c @@ -247,3 +247,24 @@ void __page_table_check_pud_set(struct mm_struct *mm, = unsigned long addr, } } EXPORT_SYMBOL(__page_table_check_pud_set); + +void __page_table_check_pmd_clear_full(struct mm_struct *mm, unsigned long= addr, + pmd_t pmd) +{ + if (&init_mm =3D=3D mm) + return; + + if (!pmd_bad(pmd) && !pmd_leaf(pmd)) { + pte_t *ptep =3D pte_offset_map(&pmd, addr); + unsigned long i; + + pte_unmap(ptep); + for (i =3D 0; i < PTRS_PER_PTE; i++) { + __page_table_check_pte_clear(mm, addr, *ptep); + addr +=3D PAGE_SIZE; + ptep++; + } + } else { + __page_table_check_pmd_clear(mm, addr, pmd); + } +} --=20 2.35.0.rc0.227.g00780c9af4-goog