From nobody Mon Jun 29 23:27:02 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id CE0C0C433EF for ; Mon, 31 Jan 2022 20:32:59 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1378872AbiAaUc6 (ORCPT ); Mon, 31 Jan 2022 15:32:58 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36222 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229851AbiAaUcz (ORCPT ); Mon, 31 Jan 2022 15:32:55 -0500 Received: from mail-qk1-x733.google.com (mail-qk1-x733.google.com [IPv6:2607:f8b0:4864:20::733]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 486B8C061714 for ; Mon, 31 Jan 2022 12:32:55 -0800 (PST) Received: by mail-qk1-x733.google.com with SMTP id b22so13183715qkk.12 for ; Mon, 31 Jan 2022 12:32:55 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=+b2VDnizluGVh9+la/y+t/dAo3W53LzEqz2Kl5SWq3M=; b=YngBIjjb/nHQSRa+qiaFZjTH9m/x8VBAIf2kGQ32pwNNBHyLxbRoo3ZxdB+I+6Oroe 8As6fNzzko3D/+CepuYqaHnseC9dE4lJZHAKbOeGxwlM3VV2+NEqYRGq7EZKMx/KFChK FWgvvuxmkldI3pOZosmJ7ktVltjev2UMC+qOgyGSEj/dvYjxGHoenQrLZTuXY+MqxtQH G6VqMbiuCYFjJS9Xi9jOPxvLuRc1K4fTP7k8ursBzujYyJxREZFkqCIw5YzvxtROhWa+ AMQF60zVFOrek/a1GrwkxGd6/H7mpiu0H/vGg0l1ymArsRf8Oa0L8c5Ja5h7C2lnZ8RV I4Dw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=+b2VDnizluGVh9+la/y+t/dAo3W53LzEqz2Kl5SWq3M=; b=w7aauq+O6+Q7UdyyewXTnN85LTkpfvpWclm0XejKIRkIZg/SFhgWd8obc0DTrr2XTE n9cowKR6OdTAD5Pxcdz4aI5YL5WVkJFYYL7eBSAPRrsniXnxSpkWvf6QBtH9g1URe56S MHWgLrmat2oy9ZQ8QiAZbX0mKshl1fQxBmvv1HKpb2WjgmN+rLtOIaQy5xkxF2r03Z7v wal21JOl56+V59cRjDnVl8/ymWImb5WvCtaAZkA51pIjW9vIyA28GEVtcSiCD1xjKNy5 B2hCALUH590qT2Zw17+LNhy9goto/hPeZL/Vp0BHWvpX7vd/bExaE9aGud41K702RXUd 1aOg== X-Gm-Message-State: AOAM530ejJar9TGmfcJZAcdsjFpOjCi5c3pxkrH21SOEt0YOqGyBFp0S gmX4SJs/7WtJ2AFJviqzFtn5FQ== X-Google-Smtp-Source: ABdhPJzfANMgqiVqJGB+ggntyJhpzn27PcIgePLyPDDStslF+pnhqRXRQJMP/Ar96PUQKuG6r8ktmQ== X-Received: by 2002:a37:aa08:: with SMTP id t8mr14647255qke.773.1643661174408; Mon, 31 Jan 2022 12:32:54 -0800 (PST) Received: from soleen.c.googlers.com.com (189.216.85.34.bc.googleusercontent.com. [34.85.216.189]) by smtp.gmail.com with ESMTPSA id j14sm5349661qko.10.2022.01.31.12.32.53 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 31 Jan 2022 12:32:53 -0800 (PST) From: Pasha Tatashin To: pasha.tatashin@soleen.com, linux-kernel@vger.kernel.org, linux-mm@kvack.org, akpm@linux-foundation.org, rientjes@google.com, pjt@google.com, weixugc@google.com, gthelen@google.com, mingo@redhat.com, will@kernel.org, rppt@kernel.org, dave.hansen@linux.intel.com, hpa@zytor.com, aneesh.kumar@linux.ibm.com, jirislaby@kernel.org, songmuchun@bytedance.com, qydwhotmail@gmail.com, hughd@google.com, ziy@nvidia.com, anshuman.khandual@arm.com Subject: [PATCH v5 1/4] mm/debug_vm_pgtable: remove pte entry from the page table Date: Mon, 31 Jan 2022 20:32:46 +0000 Message-Id: <20220131203249.2832273-2-pasha.tatashin@soleen.com> X-Mailer: git-send-email 2.35.0.rc2.247.g8bbb082509-goog In-Reply-To: <20220131203249.2832273-1-pasha.tatashin@soleen.com> References: <20220131203249.2832273-1-pasha.tatashin@soleen.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" The pte entry that is used in pte_advanced_tests() is never removed from the page table at the end of the test. The issue is detected by page_table_check, to repro compile kernel with the following configs: CONFIG_DEBUG_VM_PGTABLE=3Dy CONFIG_PAGE_TABLE_CHECK=3Dy CONFIG_PAGE_TABLE_CHECK_ENFORCED=3Dy During the boot the following BUG is printed: [ 2.262821] debug_vm_pgtable: [debug_vm_pgtable ]: Validating architecture page table helpers [ 2.276826] ------------[ cut here ]------------ [ 2.280426] kernel BUG at mm/page_table_check.c:162! [ 2.284118] invalid opcode: 0000 [#1] PREEMPT SMP PTI [ 2.287787] CPU: 0 PID: 1 Comm: swapper/0 Not tainted 5.16.0-11413-g2c271fe77d52 #3 [ 2.293226] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.15.0-0-g2dd4b9b3f840-prebuilt.qemu.org 04/01/2014 ... The entry should be properly removed from the page table before the page is released to the free list. Fixes: a5c3b9ffb0f4 ("mm/debug_vm_pgtable: add tests validating advanced ar= ch page table helpers") Cc: stable@vger.kernel.org # 5.9+ Signed-off-by: Pasha Tatashin Reviewed-by: Zi Yan Tested-by: Zi Yan Acked-by: David Rientjes Reviewed-by: Anshuman Khandual --- mm/debug_vm_pgtable.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/mm/debug_vm_pgtable.c b/mm/debug_vm_pgtable.c index a7ac97c76762..db2abd9e415b 100644 --- a/mm/debug_vm_pgtable.c +++ b/mm/debug_vm_pgtable.c @@ -171,6 +171,8 @@ static void __init pte_advanced_tests(struct pgtable_de= bug_args *args) ptep_test_and_clear_young(args->vma, args->vaddr, args->ptep); pte =3D ptep_get(args->ptep); WARN_ON(pte_young(pte)); + + ptep_get_and_clear_full(args->mm, args->vaddr, args->ptep, 1); } =20 static void __init pte_savedwrite_tests(struct pgtable_debug_args *args) --=20 2.35.0.rc2.247.g8bbb082509-goog From nobody Mon Jun 29 23:27:02 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id C87B9C433EF for ; Mon, 31 Jan 2022 20:33:02 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1379004AbiAaUdB (ORCPT ); Mon, 31 Jan 2022 15:33:01 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36230 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1378817AbiAaUc5 (ORCPT ); Mon, 31 Jan 2022 15:32:57 -0500 Received: from mail-qk1-x72a.google.com (mail-qk1-x72a.google.com [IPv6:2607:f8b0:4864:20::72a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id EE5F0C061714 for ; Mon, 31 Jan 2022 12:32:56 -0800 (PST) Received: by mail-qk1-x72a.google.com with SMTP id w8so13214921qkw.8 for ; Mon, 31 Jan 2022 12:32:56 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=ztMzOXuGTev/L+fnIZswc+dCNyphqzRLyPQWRmdFkGQ=; b=Md6GZ12Jl8JVEpV/uGyYEs0IP3JBRv0zFtP+THkOMK7/Sr8KxQ7O9qiU+8DW/TzpNC Akc5nWbdd35Z3AqErZVBsGtXj+3F+SrZ9IIn+7RS6XqDfLQdq3Dv3BQlUtwGX2qweZSb /bWB+7PT+wjthvCUUbSwK8+lmPKQXhcFXy0Kp1t0Bcjpw0EXmzL1cGfVIMBR0NfbdUUq mIE4DJeLUC7E4uC4wB+mSQbmS5A7bgKJYfLjZd+gYMmrZH8qw3K5rW/ES4bQfgIA3l50 xTxC9ATLUYebxRDdso0G9dbU2k2bni9mX7X3AV6CzbxsVGhBEI2r4qtyLn0+0ALREg9m DU0w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=ztMzOXuGTev/L+fnIZswc+dCNyphqzRLyPQWRmdFkGQ=; b=hZFf2I8mPlSNeHYbfqXW1RL+nzLy1ZDZU0Dtc30VF3miPjiRmkKgFgyadpqBrRzTZn lo3tcBF3sEXYOBtFdY/MQE+XBV8J8fpYx+L9NRZIbdh7ZlM8Fi6dqOtoFEg2/FNMX5ro fYeJ+QUQGMcaTpAmkkX9dV+SIs454lzqsToz0IqxIFfuPa/YVkDwXYZW+j35+5JdjKpw 7s3BTtR5NWAWG8ajw4QvJIkrvWMXxd98GgQLQxDVpclvONdCkmKp4YN2RfbG70bbeY+g whAjepmOlCexZTrDxtBcQVqPkV1359MNwQZ9QC2OU72U8NFFDKgOJAFJzRYXqJq0aoQg HeMg== X-Gm-Message-State: AOAM532/eD2TyRIjihD6Hg5R3hf/CfDBZWrHLrO0/gJNwbJU7GZWpv04 Aire0xDpeNsdaY2AxmYE0Xp2wA== X-Google-Smtp-Source: ABdhPJy7TkmRAP3kG1fRwG+5u3nYXG/B+PDJTVHIT7qNMvRw2rYa1WyDbrqsKB6mPHKWvTMPZpXt/g== X-Received: by 2002:a05:620a:4151:: with SMTP id k17mr14312024qko.630.1643661176090; Mon, 31 Jan 2022 12:32:56 -0800 (PST) Received: from soleen.c.googlers.com.com (189.216.85.34.bc.googleusercontent.com. [34.85.216.189]) by smtp.gmail.com with ESMTPSA id j14sm5349661qko.10.2022.01.31.12.32.54 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 31 Jan 2022 12:32:55 -0800 (PST) From: Pasha Tatashin To: pasha.tatashin@soleen.com, linux-kernel@vger.kernel.org, linux-mm@kvack.org, akpm@linux-foundation.org, rientjes@google.com, pjt@google.com, weixugc@google.com, gthelen@google.com, mingo@redhat.com, will@kernel.org, rppt@kernel.org, dave.hansen@linux.intel.com, hpa@zytor.com, aneesh.kumar@linux.ibm.com, jirislaby@kernel.org, songmuchun@bytedance.com, qydwhotmail@gmail.com, hughd@google.com, ziy@nvidia.com, anshuman.khandual@arm.com Subject: [PATCH v5 2/4] mm/page_table_check: use unsigned long for page counters and cleanup Date: Mon, 31 Jan 2022 20:32:47 +0000 Message-Id: <20220131203249.2832273-3-pasha.tatashin@soleen.com> X-Mailer: git-send-email 2.35.0.rc2.247.g8bbb082509-goog In-Reply-To: <20220131203249.2832273-1-pasha.tatashin@soleen.com> References: <20220131203249.2832273-1-pasha.tatashin@soleen.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" For the consistency, use "unsigned long" for all page counters. Also, reduce code duplication by calling __page_table_check_*_clear() from __page_table_check_*_set() functions. Signed-off-by: Pasha Tatashin Reviewed-by: Wei Xu Acked-by: David Rientjes --- mm/page_table_check.c | 35 +++++++---------------------------- 1 file changed, 7 insertions(+), 28 deletions(-) diff --git a/mm/page_table_check.c b/mm/page_table_check.c index 7504e7caa2a1..c61d7ebe13b1 100644 --- a/mm/page_table_check.c +++ b/mm/page_table_check.c @@ -86,8 +86,8 @@ static void page_table_check_clear(struct mm_struct *mm, = unsigned long addr, { struct page_ext *page_ext; struct page *page; + unsigned long i; bool anon; - int i; =20 if (!pfn_valid(pfn)) return; @@ -121,8 +121,8 @@ static void page_table_check_set(struct mm_struct *mm, = unsigned long addr, { struct page_ext *page_ext; struct page *page; + unsigned long i; bool anon; - int i; =20 if (!pfn_valid(pfn)) return; @@ -152,10 +152,10 @@ static void page_table_check_set(struct mm_struct *mm= , unsigned long addr, void __page_table_check_zero(struct page *page, unsigned int order) { struct page_ext *page_ext =3D lookup_page_ext(page); - int i; + unsigned long i; =20 BUG_ON(!page_ext); - for (i =3D 0; i < (1 << order); i++) { + for (i =3D 0; i < (1ul << order); i++) { struct page_table_check *ptc =3D get_page_table_check(page_ext); =20 BUG_ON(atomic_read(&ptc->anon_map_count)); @@ -206,17 +206,10 @@ EXPORT_SYMBOL(__page_table_check_pud_clear); void __page_table_check_pte_set(struct mm_struct *mm, unsigned long addr, pte_t *ptep, pte_t pte) { - pte_t old_pte; - if (&init_mm =3D=3D mm) return; =20 - old_pte =3D *ptep; - if (pte_user_accessible_page(old_pte)) { - page_table_check_clear(mm, addr, pte_pfn(old_pte), - PAGE_SIZE >> PAGE_SHIFT); - } - + __page_table_check_pte_clear(mm, addr, *ptep); if (pte_user_accessible_page(pte)) { page_table_check_set(mm, addr, pte_pfn(pte), PAGE_SIZE >> PAGE_SHIFT, @@ -228,17 +221,10 @@ EXPORT_SYMBOL(__page_table_check_pte_set); void __page_table_check_pmd_set(struct mm_struct *mm, unsigned long addr, pmd_t *pmdp, pmd_t pmd) { - pmd_t old_pmd; - if (&init_mm =3D=3D mm) return; =20 - old_pmd =3D *pmdp; - if (pmd_user_accessible_page(old_pmd)) { - page_table_check_clear(mm, addr, pmd_pfn(old_pmd), - PMD_PAGE_SIZE >> PAGE_SHIFT); - } - + __page_table_check_pmd_clear(mm, addr, *pmdp); if (pmd_user_accessible_page(pmd)) { page_table_check_set(mm, addr, pmd_pfn(pmd), PMD_PAGE_SIZE >> PAGE_SHIFT, @@ -250,17 +236,10 @@ EXPORT_SYMBOL(__page_table_check_pmd_set); void __page_table_check_pud_set(struct mm_struct *mm, unsigned long addr, pud_t *pudp, pud_t pud) { - pud_t old_pud; - if (&init_mm =3D=3D mm) return; =20 - old_pud =3D *pudp; - if (pud_user_accessible_page(old_pud)) { - page_table_check_clear(mm, addr, pud_pfn(old_pud), - PUD_PAGE_SIZE >> PAGE_SHIFT); - } - + __page_table_check_pud_clear(mm, addr, *pudp); if (pud_user_accessible_page(pud)) { page_table_check_set(mm, addr, pud_pfn(pud), PUD_PAGE_SIZE >> PAGE_SHIFT, --=20 2.35.0.rc2.247.g8bbb082509-goog From nobody Mon Jun 29 23:27:02 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id AC43DC433EF for ; Mon, 31 Jan 2022 20:33:07 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1379194AbiAaUdG (ORCPT ); Mon, 31 Jan 2022 15:33:06 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36244 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229665AbiAaUc7 (ORCPT ); Mon, 31 Jan 2022 15:32:59 -0500 Received: from mail-qv1-xf35.google.com (mail-qv1-xf35.google.com [IPv6:2607:f8b0:4864:20::f35]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 42B10C06173B for ; Mon, 31 Jan 2022 12:32:59 -0800 (PST) Received: by mail-qv1-xf35.google.com with SMTP id e20so13979951qvu.7 for ; Mon, 31 Jan 2022 12:32:59 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=fF5QFjKZhDTZd5B31TRW94GY69vdDJby6v3gBBayCwY=; b=HMakM+xY1YLPhqv6PfRXrYF6lEKAz8ldAQlsUVG77GLVRPT4XgfUdXYBbDqjbHerrt xlAlPDccJTu54jM1IX3IhH9eY7bHfcLScHm8cpZdvEAtBUEe0OwdWTmm3OpsgrHQOxKc p9fJbv6S+USh2jyF9NwYK2PuMR4blz+hZQUSoTJxMiyTUL+opiBq+wcMSSz/d0GmBIHw F8lNPaRz+Uaj2G743xUJPbnHO5FCBhIIRR5PK9Mt7vGZv7uHbvALBbVPi2mD/ESWMX4Q 9JSwCrXHiq/NqgjmRJ945HfsESAs/4CAd03ax9ZYClBPRTsNg5k7euUEIDikB6zCMR8T Za3Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=fF5QFjKZhDTZd5B31TRW94GY69vdDJby6v3gBBayCwY=; b=lC8yYzyc2gaGesqU0z1OtDY1z/sIacFuoWkxXBpIC48rFz5ugXjodDp7NKu4eaEq1C v+5m4JeqT4PdHoX7ZSIws/78+OO77Cz2Wg74eReoHvCSmGeZzBOzOrn/9bwpc3HV5StK gVU6JcmHupoLUI+Cqe8C2ixxOLn98xzqasxIitoBUPGW1MtTK1xwfDsBnMxNnGrNAwE1 GWpIfa7OVqBswDJfbx7DYQ3UkcN0+X8e02UofKRURJjHGQCYEiMWQ9yb3n4phYo0uts7 gWVlg5uL3/GBxartJJxZZj78hJJufVpCXl9Z/8FPgI5nCToTY7jQ/+42zrLJT51+6G2t oc9A== X-Gm-Message-State: AOAM531hDO8qGXn4w0YlFwPNMZzkQuzUn3yyYthsY9dzD4MAL/Efv81D CN/Ab1TMtE4kF/h+Rgz2xzta6w== X-Google-Smtp-Source: ABdhPJx9Y6qc+SG5iROabTkXGy2K2xhjBhgrJadrpqmMWIheHQtwTINxgKfqF3vSMeC0FLwOvq+rfw== X-Received: by 2002:a05:6214:5014:: with SMTP id jo20mr19417458qvb.28.1643661178443; Mon, 31 Jan 2022 12:32:58 -0800 (PST) Received: from soleen.c.googlers.com.com (189.216.85.34.bc.googleusercontent.com. [34.85.216.189]) by smtp.gmail.com with ESMTPSA id j14sm5349661qko.10.2022.01.31.12.32.56 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 31 Jan 2022 12:32:57 -0800 (PST) From: Pasha Tatashin To: pasha.tatashin@soleen.com, linux-kernel@vger.kernel.org, linux-mm@kvack.org, akpm@linux-foundation.org, rientjes@google.com, pjt@google.com, weixugc@google.com, gthelen@google.com, mingo@redhat.com, will@kernel.org, rppt@kernel.org, dave.hansen@linux.intel.com, hpa@zytor.com, aneesh.kumar@linux.ibm.com, jirislaby@kernel.org, songmuchun@bytedance.com, qydwhotmail@gmail.com, hughd@google.com, ziy@nvidia.com, anshuman.khandual@arm.com Subject: [PATCH v5 3/4] mm/khugepaged: unify collapse pmd clear, flush and free Date: Mon, 31 Jan 2022 20:32:48 +0000 Message-Id: <20220131203249.2832273-4-pasha.tatashin@soleen.com> X-Mailer: git-send-email 2.35.0.rc2.247.g8bbb082509-goog In-Reply-To: <20220131203249.2832273-1-pasha.tatashin@soleen.com> References: <20220131203249.2832273-1-pasha.tatashin@soleen.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Unify the code that flushes, clears pmd entry, and frees the PTE table level into a new function collapse_and_free_pmd(). This clean-up is useful as in the next patch we will add another call to this function to iterate through PTE prior to freeing the level for page table check. Signed-off-by: Pasha Tatashin Acked-by: David Rientjes --- mm/khugepaged.c | 34 ++++++++++++++++++---------------- 1 file changed, 18 insertions(+), 16 deletions(-) diff --git a/mm/khugepaged.c b/mm/khugepaged.c index 35f14d0a00a6..30e59e4af272 100644 --- a/mm/khugepaged.c +++ b/mm/khugepaged.c @@ -1416,6 +1416,19 @@ static int khugepaged_add_pte_mapped_thp(struct mm_s= truct *mm, return 0; } =20 +static void collapse_and_free_pmd(struct mm_struct *mm, struct vm_area_str= uct *vma, + unsigned long addr, pmd_t *pmdp) +{ + spinlock_t *ptl; + pmd_t pmd; + + ptl =3D pmd_lock(vma->vm_mm, pmdp); + pmd =3D pmdp_collapse_flush(vma, addr, pmdp); + spin_unlock(ptl); + mm_dec_nr_ptes(mm); + pte_free(mm, pmd_pgtable(pmd)); +} + /** * collapse_pte_mapped_thp - Try to collapse a pte-mapped THP for mm at * address haddr. @@ -1433,7 +1446,7 @@ void collapse_pte_mapped_thp(struct mm_struct *mm, un= signed long addr) struct vm_area_struct *vma =3D find_vma(mm, haddr); struct page *hpage; pte_t *start_pte, *pte; - pmd_t *pmd, _pmd; + pmd_t *pmd; spinlock_t *ptl; int count =3D 0; int i; @@ -1509,12 +1522,7 @@ void collapse_pte_mapped_thp(struct mm_struct *mm, u= nsigned long addr) } =20 /* step 4: collapse pmd */ - ptl =3D pmd_lock(vma->vm_mm, pmd); - _pmd =3D pmdp_collapse_flush(vma, haddr, pmd); - spin_unlock(ptl); - mm_dec_nr_ptes(mm); - pte_free(mm, pmd_pgtable(_pmd)); - + collapse_and_free_pmd(mm, vma, haddr, pmd); drop_hpage: unlock_page(hpage); put_page(hpage); @@ -1552,7 +1560,7 @@ static void retract_page_tables(struct address_space = *mapping, pgoff_t pgoff) struct vm_area_struct *vma; struct mm_struct *mm; unsigned long addr; - pmd_t *pmd, _pmd; + pmd_t *pmd; =20 i_mmap_lock_write(mapping); vma_interval_tree_foreach(vma, &mapping->i_mmap, pgoff, pgoff) { @@ -1591,14 +1599,8 @@ static void retract_page_tables(struct address_space= *mapping, pgoff_t pgoff) * reverse order. Trylock is a way to avoid deadlock. */ if (mmap_write_trylock(mm)) { - if (!khugepaged_test_exit(mm)) { - spinlock_t *ptl =3D pmd_lock(mm, pmd); - /* assume page table is clear */ - _pmd =3D pmdp_collapse_flush(vma, addr, pmd); - spin_unlock(ptl); - mm_dec_nr_ptes(mm); - pte_free(mm, pmd_pgtable(_pmd)); - } + if (!khugepaged_test_exit(mm)) + collapse_and_free_pmd(mm, vma, addr, pmd); mmap_write_unlock(mm); } else { /* Try again later */ --=20 2.35.0.rc2.247.g8bbb082509-goog From nobody Mon Jun 29 23:27:02 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 87008C433EF for ; Mon, 31 Jan 2022 20:33:10 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1378948AbiAaUdJ (ORCPT ); Mon, 31 Jan 2022 15:33:09 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36252 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1378991AbiAaUdB (ORCPT ); Mon, 31 Jan 2022 15:33:01 -0500 Received: from mail-qv1-xf32.google.com (mail-qv1-xf32.google.com [IPv6:2607:f8b0:4864:20::f32]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id DC76DC061714 for ; Mon, 31 Jan 2022 12:33:00 -0800 (PST) Received: by mail-qv1-xf32.google.com with SMTP id a7so14007245qvl.1 for ; Mon, 31 Jan 2022 12:33:00 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=AM3L4Ev7NB1R87m9UVMqXlPBBRGhukRV4JpXV46C9WE=; b=VgxBIxJ0hiHLQyNThvhA4DR5VIF7DaW5tnlYercL6lG4BdPNRcJ4feaFDr49XnPJgj OAuGN9kkbL/k0WS/ESNGQTOxhfQoFghohuY+0yCEAMeQxbUzMydi6tHXfZgMXfbs7Kf4 N4lYqp3WrX/Jf6/domHnPz0jX13A73MaOvIY9txUUBt1/NS77Jes+qhBP4UYT69Y7R6m 1Ki68agcRKkXDE4MXkSN63RF1O9OkCSqwxFJTwX4vxULibefaqLTgi4AStPTCktfFvxT n/sJ3qOOkYjLtNjHR0GU8V8Fn7ozCOvOBMxfmU7OdEpAreN8LFLqdCGdKAYzrMUWNefn 9i5Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=AM3L4Ev7NB1R87m9UVMqXlPBBRGhukRV4JpXV46C9WE=; b=eGhEuy7gDRxvEEVhfwHZWGALDholdFvNqvcDMp3h36DPHJ0MepouIYyu4CpckWsNUr arTtsNK1AnpZvMxuf3Y7+rDjH4+61tYJwEF9kWEP9YBbxLEAlBt+H81Me9xwD7JAd1gn dj0ZjjO52SXvKBxWDwzZQ5esiOPXu1g+40oMSXTSXSJV+sLiY4gneguOYGOkclMStVQa mOlPhuu11gjo+BMGLukXwoc/ybnpn0wAe2rqdxpNz36jdQiXhmTiPVKkrbygcG/I0vT7 nirJcdXtBbVuNpGNTkynjs8mLnBvMSc9PWOYbZlcN0AM+AgvOhKMkDeL9cnbLt78IUwQ owfQ== X-Gm-Message-State: AOAM531lK/8HFmzmModCnU0yD2/lr6EV6Vs2olnqom5ytrnuCwbpit2X cyufsUsUfnhHoVdIotPkk3aIkw== X-Google-Smtp-Source: ABdhPJzoXyE3v1wQg9qr6hkLjh4LSwViEgR3/H8/DbVGMEMW+KOaQDeNNotNwxteh9FE9xUG/VOW+Q== X-Received: by 2002:a05:6214:29ce:: with SMTP id gh14mr18628233qvb.81.1643661180014; Mon, 31 Jan 2022 12:33:00 -0800 (PST) Received: from soleen.c.googlers.com.com (189.216.85.34.bc.googleusercontent.com. [34.85.216.189]) by smtp.gmail.com with ESMTPSA id j14sm5349661qko.10.2022.01.31.12.32.58 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 31 Jan 2022 12:32:59 -0800 (PST) From: Pasha Tatashin To: pasha.tatashin@soleen.com, linux-kernel@vger.kernel.org, linux-mm@kvack.org, akpm@linux-foundation.org, rientjes@google.com, pjt@google.com, weixugc@google.com, gthelen@google.com, mingo@redhat.com, will@kernel.org, rppt@kernel.org, dave.hansen@linux.intel.com, hpa@zytor.com, aneesh.kumar@linux.ibm.com, jirislaby@kernel.org, songmuchun@bytedance.com, qydwhotmail@gmail.com, hughd@google.com, ziy@nvidia.com, anshuman.khandual@arm.com Subject: [PATCH v5 4/4] mm/page_table_check: check entries at pmd levels Date: Mon, 31 Jan 2022 20:32:49 +0000 Message-Id: <20220131203249.2832273-5-pasha.tatashin@soleen.com> X-Mailer: git-send-email 2.35.0.rc2.247.g8bbb082509-goog In-Reply-To: <20220131203249.2832273-1-pasha.tatashin@soleen.com> References: <20220131203249.2832273-1-pasha.tatashin@soleen.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" syzbot detected a case where the page table counters were not properly updated. syzkaller login: ------------[ cut here ]------------ kernel BUG at mm/page_table_check.c:162! invalid opcode: 0000 [#1] PREEMPT SMP KASAN CPU: 0 PID: 3099 Comm: pasha Not tainted 5.16.0+ #48 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIO4 RIP: 0010:__page_table_check_zero+0x159/0x1a0 Code: 7d 3a b2 ff 45 39 f5 74 2a e8 43 38 b2 ff 4d 85 e4 01 RSP: 0018:ffff888010667418 EFLAGS: 00010293 RAX: 0000000000000000 RBX: 0000000000000001 RCX: 0000000000 RDX: ffff88800cea8680 RSI: ffffffff81becaf9 RDI: 0000000003 RBP: ffff888010667450 R08: 0000000000000001 R09: 0000000000 R10: ffffffff81becaab R11: 0000000000000001 R12: ffff888008 R13: 0000000000000001 R14: 0000000000000200 R15: dffffc0000 FS: 0000000000000000(0000) GS:ffff888035e00000(0000) knlG0 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 00007ffd875cad00 CR3: 00000000094ce000 CR4: 0000000000 DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000 DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000 Call Trace: free_pcp_prepare+0x3be/0xaa0 free_unref_page+0x1c/0x650 ? trace_hardirqs_on+0x6a/0x1d0 free_compound_page+0xec/0x130 free_transhuge_page+0x1be/0x260 __put_compound_page+0x90/0xd0 release_pages+0x54c/0x1060 ? filemap_remove_folio+0x161/0x210 ? lock_downgrade+0x720/0x720 ? __put_page+0x150/0x150 ? filemap_free_folio+0x164/0x350 __pagevec_release+0x7c/0x110 shmem_undo_range+0x85e/0x1250 ... The repro involved having a huge page that is split due to uprobe event temporarily replacing one of the pages in the huge page. Later the huge page was combined again, but the counters were off, as the PTE level was not properly updated. Make sure that when PMD is cleared and prior to freeing the level the PTEs are updated. Fixes: df4e817b7108 ("mm: page table check") Signed-off-by: Pasha Tatashin Acked-by: David Rientjes --- include/linux/page_table_check.h | 19 +++++++++++++++++++ mm/khugepaged.c | 3 +++ mm/page_table_check.c | 20 ++++++++++++++++++++ 3 files changed, 42 insertions(+) diff --git a/include/linux/page_table_check.h b/include/linux/page_table_ch= eck.h index 38cace1da7b6..01e16c7696ec 100644 --- a/include/linux/page_table_check.h +++ b/include/linux/page_table_check.h @@ -26,6 +26,9 @@ void __page_table_check_pmd_set(struct mm_struct *mm, uns= igned long addr, pmd_t *pmdp, pmd_t pmd); void __page_table_check_pud_set(struct mm_struct *mm, unsigned long addr, pud_t *pudp, pud_t pud); +void __page_table_check_pte_clear_range(struct mm_struct *mm, + unsigned long addr, + pmd_t pmd); =20 static inline void page_table_check_alloc(struct page *page, unsigned int = order) { @@ -100,6 +103,16 @@ static inline void page_table_check_pud_set(struct mm_= struct *mm, __page_table_check_pud_set(mm, addr, pudp, pud); } =20 +static inline void page_table_check_pte_clear_range(struct mm_struct *mm, + unsigned long addr, + pmd_t pmd) +{ + if (static_branch_likely(&page_table_check_disabled)) + return; + + __page_table_check_pte_clear_range(mm, addr, pmd); +} + #else =20 static inline void page_table_check_alloc(struct page *page, unsigned int = order) @@ -143,5 +156,11 @@ static inline void page_table_check_pud_set(struct mm_= struct *mm, { } =20 +static inline void page_table_check_pte_clear_range(struct mm_struct *mm, + unsigned long addr, + pmd_t pmd) +{ +} + #endif /* CONFIG_PAGE_TABLE_CHECK */ #endif /* __LINUX_PAGE_TABLE_CHECK_H */ diff --git a/mm/khugepaged.c b/mm/khugepaged.c index 30e59e4af272..131492fd1148 100644 --- a/mm/khugepaged.c +++ b/mm/khugepaged.c @@ -16,6 +16,7 @@ #include #include #include +#include #include #include =20 @@ -1422,10 +1423,12 @@ static void collapse_and_free_pmd(struct mm_struct = *mm, struct vm_area_struct *v spinlock_t *ptl; pmd_t pmd; =20 + mmap_assert_write_locked(mm); ptl =3D pmd_lock(vma->vm_mm, pmdp); pmd =3D pmdp_collapse_flush(vma, addr, pmdp); spin_unlock(ptl); mm_dec_nr_ptes(mm); + page_table_check_pte_clear_range(mm, addr, pmd); pte_free(mm, pmd_pgtable(pmd)); } =20 diff --git a/mm/page_table_check.c b/mm/page_table_check.c index c61d7ebe13b1..3763bd077861 100644 --- a/mm/page_table_check.c +++ b/mm/page_table_check.c @@ -247,3 +247,23 @@ void __page_table_check_pud_set(struct mm_struct *mm, = unsigned long addr, } } EXPORT_SYMBOL(__page_table_check_pud_set); + +void __page_table_check_pte_clear_range(struct mm_struct *mm, + unsigned long addr, + pmd_t pmd) +{ + if (&init_mm =3D=3D mm) + return; + + if (!pmd_bad(pmd) && !pmd_leaf(pmd)) { + pte_t *ptep =3D pte_offset_map(&pmd, addr); + unsigned long i; + + pte_unmap(ptep); + for (i =3D 0; i < PTRS_PER_PTE; i++) { + __page_table_check_pte_clear(mm, addr, *ptep); + addr +=3D PAGE_SIZE; + ptep++; + } + } +} --=20 2.35.0.rc2.247.g8bbb082509-goog