From nobody Thu May 7 18:53:26 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9BCB6C433F5 for ; Fri, 20 May 2022 21:16:20 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1353616AbiETVQT (ORCPT ); Fri, 20 May 2022 17:16:19 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44838 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1353606AbiETVQP (ORCPT ); Fri, 20 May 2022 17:16:15 -0400 Received: from mail-pg1-x52c.google.com (mail-pg1-x52c.google.com [IPv6:2607:f8b0:4864:20::52c]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 16C1A186E5 for ; Fri, 20 May 2022 14:16:14 -0700 (PDT) Received: by mail-pg1-x52c.google.com with SMTP id c22so8753253pgu.2 for ; Fri, 20 May 2022 14:16:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=1QLKDkRDVEEGrLLKtQdSS38U6TUL0EcDbwYp3Or4BIw=; b=NYhrj7RyelVXOqRq9pojKSUdPKLFF1GIynVR/ldLf8JQqCX/3ntxNjQlV8L/A0DZMc vZ3c2YpPQjeoSfj1bIIeoFSY9OrsZKsihZwdZVL9hNitrGjSLb+nkpG3pDB8zJalWliK z7pCIOXi+JA7utglSAMGkRg9wmUgLkKmGLZ45dCi0HvrjH9NyUYg12kTdp+RiMT92spe 5kAh9NdQ+7XItGxLIKc7LOvbW3UGvk3GT6Y2UnsuvKQVidxG9rsjG8OvEtzQi7RQ64/R vKvAj2RGL5JUkp3/a7pW0lPJ3ubI9yqBORm5hJxGErWGBNVxb5CSgIczrHxiy0pRnpBs 0HHw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=1QLKDkRDVEEGrLLKtQdSS38U6TUL0EcDbwYp3Or4BIw=; b=aOHY9qlxDkh4I9ixQEMqNyF4/8UDVL/olVvPV5IJJwfAiqFKWbdZuqIL5ynV4u/13E xDt0d522Zi4eN/T72ObfSWKIR/eM53sOqGwn5t9PGywYsKX2Dzc6e6Brjbb4uXo9PcGe 0kzOOTtaJgSiGE0CNeStjs4neABZ7/7YUHWaL7G9A7LpgSkXD6m6zOtfIf/yO5BCP0IW OaExYXTGAgyVOfS13NX0lzFcEToF2miFw2AbdMytvCehv/79bZNiK6FSRur1sAwY1HGK LSxt1gGfpky04rWSKdYU6Pi+sPKaXn40/fPh0RFIXw5T/JRyRlPWC2xMcRlZReQ/oh3+ MQDw== X-Gm-Message-State: AOAM533OcwedY/qMKbl6tpl8yxTiIHMsAa4X5WIBW9URunb5/nZPL7cx O9SG3zYlZzL2fWgXQ8zBL3c= X-Google-Smtp-Source: ABdhPJzjqV91/dbDxBd+KTR/6WHbquHR0Y+DY4HXucFpGAZnlfdhBfKsWiMXYk3Dbuqk5G7NZrTGMQ== X-Received: by 2002:a05:6a00:1251:b0:50d:f2c2:7f4e with SMTP id u17-20020a056a00125100b0050df2c27f4emr11912367pfi.72.1653081373606; Fri, 20 May 2022 14:16:13 -0700 (PDT) Received: from localhost.localdomain (c-67-174-241-145.hsd1.ca.comcast.net. [67.174.241.145]) by smtp.gmail.com with ESMTPSA id h1-20020a170902748100b0015e8d4eb2c2sm167620pll.268.2022.05.20.14.16.12 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 20 May 2022 14:16:12 -0700 (PDT) From: Yang Shi To: vbabka@suse.cz, kirill.shutemov@linux.intel.com, akpm@linux-foundation.org Cc: shy828301@gmail.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH 1/7] mm: khugepaged: check THP flag in hugepage_vma_check() Date: Fri, 20 May 2022 14:15:59 -0700 Message-Id: <20220520211605.51473-2-shy828301@gmail.com> X-Mailer: git-send-email 2.26.3 In-Reply-To: <20220520211605.51473-1-shy828301@gmail.com> References: <20220520211605.51473-1-shy828301@gmail.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Currently the THP flag check in hugepage_vma_check() will fallthrough if the flag is NEVER and VM_HUGEPAGE is set. This is not a problem for now since all the callers have the flag checked before or can't be invoked if the flag is NEVER. However, the following patch will call hugepage_vma_check() in more places, for example, page fault, so this flag must be checked in hugepge_vma_check(). Signed-off-by: Yang Shi --- mm/khugepaged.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/mm/khugepaged.c b/mm/khugepaged.c index 637bfecd6bf5..92e91c08d96a 100644 --- a/mm/khugepaged.c +++ b/mm/khugepaged.c @@ -458,6 +458,9 @@ bool hugepage_vma_check(struct vm_area_struct *vma, if (shmem_file(vma->vm_file)) return shmem_huge_enabled(vma); =20 + if (!khugepaged_enabled()) + return false; + /* THP settings require madvise. */ if (!(vm_flags & VM_HUGEPAGE) && !khugepaged_always()) return false; --=20 2.26.3 From nobody Thu May 7 18:53:26 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 66F8FC433EF for ; Fri, 20 May 2022 21:16:24 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1353628AbiETVQX (ORCPT ); Fri, 20 May 2022 17:16:23 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44920 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1353612AbiETVQQ (ORCPT ); Fri, 20 May 2022 17:16:16 -0400 Received: from mail-pf1-x430.google.com (mail-pf1-x430.google.com [IPv6:2607:f8b0:4864:20::430]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 84B9618378 for ; Fri, 20 May 2022 14:16:15 -0700 (PDT) Received: by mail-pf1-x430.google.com with SMTP id u15so8688616pfi.3 for ; Fri, 20 May 2022 14:16:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=gg6tNe85KQQ/g3LzRoRENfwwXGulqQh5OTFWqrUU8No=; b=PgTK3glRRfd5JkX+9f6IiYFOmD9Z9dNofGK/OipD+mpgQCHsKjBPp/AQFjZRf0O3qR hKge+wGyWYzGvM+XyongdCHlTqQcHcJKsxqSNcz7crYk+saXUcgUuxTUw2w0bZuQj1N9 Ajipf+rSlrmD0+qBqy5vme1F//nPQIIke6CoQSWdFlkX8NfeJlJqmN9bj5lDVLMt/kZx RJAn3mEOXG65s4VtYcaNT6jb+Kl6w42xRHQhZ2EpOng4nvcb026P/FE8v/RncZcKC9xv zV5b4U2kzG890iKByk0lsfUKoZgZ02xxnRFgP+WURkTaZdFSvAuz1kSFbiv7EZUYt79+ auLA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=gg6tNe85KQQ/g3LzRoRENfwwXGulqQh5OTFWqrUU8No=; b=zvUXj/KOTgvYHVpl5nSGSzFG+TFuyz4nOKHTMr3CDSYkMZlENeb0qyBuTKXOh9TQVh WHPvIN6YRnxo/G8QYscqOd7plT9rttIHDXJ+X4I0dgsBlqTC9fjDr6DDeg6JGmG1TdNu HzdNvZoOR7eiblGZAwlU60/PU30krTQ5fZyTGj4L0Zu4V47TwDvE69mwaAJxdBDWsWH/ 1V+ftqlXe0bW+laWhzU752Z4BEOaxwiqBo0rYVbxr710+E99eX6c2TzRVmfpvbA5xIuN 4oXblwi413W8S90aSP36wPYQXRc6t9pbS16rPpSC1yoTWpDjxeiPjqAUIrIO+p+nFzI3 waqQ== X-Gm-Message-State: AOAM531h7P8Q+KMGE5KOXgBkwBF1rmYL/qbtGmFHgnudaTrKUJU55V7s dPBeNugHolk2IiKHDH56oUo= X-Google-Smtp-Source: ABdhPJzSPHAO+qKZMPiyyvZY1u0P5E9VsnEg+dfk2XO7c6Ka5FUtpo8JOyZCJt+tyv57RY71VKVndw== X-Received: by 2002:a63:2002:0:b0:3c6:ae77:1869 with SMTP id g2-20020a632002000000b003c6ae771869mr9888206pgg.71.1653081375058; Fri, 20 May 2022 14:16:15 -0700 (PDT) Received: from localhost.localdomain (c-67-174-241-145.hsd1.ca.comcast.net. [67.174.241.145]) by smtp.gmail.com with ESMTPSA id h1-20020a170902748100b0015e8d4eb2c2sm167620pll.268.2022.05.20.14.16.13 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 20 May 2022 14:16:14 -0700 (PDT) From: Yang Shi To: vbabka@suse.cz, kirill.shutemov@linux.intel.com, akpm@linux-foundation.org Cc: shy828301@gmail.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH 2/7] mm: thp: introduce transhuge_vma_size_ok() helper Date: Fri, 20 May 2022 14:16:00 -0700 Message-Id: <20220520211605.51473-3-shy828301@gmail.com> X-Mailer: git-send-email 2.26.3 In-Reply-To: <20220520211605.51473-1-shy828301@gmail.com> References: <20220520211605.51473-1-shy828301@gmail.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" There are couple of places that check whether the vma size is ok for THP or not, they are open coded and duplicate, introduce transhuge_vma_size_ok() helper to do the job. Signed-off-by: Yang Shi --- include/linux/huge_mm.h | 17 +++++++++++++++++ mm/huge_memory.c | 5 +---- mm/khugepaged.c | 12 ++++++------ 3 files changed, 24 insertions(+), 10 deletions(-) diff --git a/include/linux/huge_mm.h b/include/linux/huge_mm.h index 648cb3ce7099..a8f61db47f2a 100644 --- a/include/linux/huge_mm.h +++ b/include/linux/huge_mm.h @@ -116,6 +116,18 @@ extern struct kobj_attribute shmem_enabled_attr; =20 extern unsigned long transparent_hugepage_flags; =20 +/* + * The vma size has to be large enough to hold an aligned HPAGE_PMD_SIZE a= rea. + */ +static inline bool transhuge_vma_size_ok(struct vm_area_struct *vma) +{ + if (round_up(vma->vm_start, HPAGE_PMD_SIZE) < + (vma->vm_end & HPAGE_PMD_MASK)) + return true; + + return false; +} + static inline bool transhuge_vma_suitable(struct vm_area_struct *vma, unsigned long addr) { @@ -345,6 +357,11 @@ static inline bool transparent_hugepage_active(struct = vm_area_struct *vma) return false; } =20 +static inline bool transhuge_vma_size_ok(struct vm_area_struct *vma) +{ + return false; +} + static inline bool transhuge_vma_suitable(struct vm_area_struct *vma, unsigned long addr) { diff --git a/mm/huge_memory.c b/mm/huge_memory.c index 80e8b58b4f39..d633f97452c1 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -71,10 +71,7 @@ unsigned long huge_zero_pfn __read_mostly =3D ~0UL; =20 bool transparent_hugepage_active(struct vm_area_struct *vma) { - /* The addr is used to check if the vma size fits */ - unsigned long addr =3D (vma->vm_end & HPAGE_PMD_MASK) - HPAGE_PMD_SIZE; - - if (!transhuge_vma_suitable(vma, addr)) + if (!transhuge_vma_size_ok(vma)) return false; if (vma_is_anonymous(vma)) return __transparent_hugepage_enabled(vma); diff --git a/mm/khugepaged.c b/mm/khugepaged.c index 92e91c08d96a..5bdb30a76f05 100644 --- a/mm/khugepaged.c +++ b/mm/khugepaged.c @@ -454,6 +454,9 @@ bool hugepage_vma_check(struct vm_area_struct *vma, vma->vm_pgoff, HPAGE_PMD_NR)) return false; =20 + if (!transhuge_vma_size_ok(vma)) + return false; + /* Enabled via shmem mount options or sysfs settings. */ if (shmem_file(vma->vm_file)) return shmem_huge_enabled(vma); @@ -512,9 +515,7 @@ void khugepaged_enter_vma(struct vm_area_struct *vma, unsigned long vm_flags) { if (!test_bit(MMF_VM_HUGEPAGE, &vma->vm_mm->flags) && - khugepaged_enabled() && - (((vma->vm_start + ~HPAGE_PMD_MASK) & HPAGE_PMD_MASK) < - (vma->vm_end & HPAGE_PMD_MASK))) { + khugepaged_enabled()) { if (hugepage_vma_check(vma, vm_flags)) __khugepaged_enter(vma->vm_mm); } @@ -2144,10 +2145,9 @@ static unsigned int khugepaged_scan_mm_slot(unsigned= int pages, progress++; continue; } - hstart =3D (vma->vm_start + ~HPAGE_PMD_MASK) & HPAGE_PMD_MASK; + + hstart =3D round_up(vma->vm_start, HPAGE_PMD_SIZE); hend =3D vma->vm_end & HPAGE_PMD_MASK; - if (hstart >=3D hend) - goto skip; if (khugepaged_scan.address > hend) goto skip; if (khugepaged_scan.address < hstart) --=20 2.26.3 From nobody Thu May 7 18:53:26 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id A951DC433EF for ; Fri, 20 May 2022 21:16:28 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1353633AbiETVQ0 (ORCPT ); Fri, 20 May 2022 17:16:26 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44982 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1353618AbiETVQT (ORCPT ); Fri, 20 May 2022 17:16:19 -0400 Received: from mail-pf1-x429.google.com (mail-pf1-x429.google.com [IPv6:2607:f8b0:4864:20::429]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 180AA186FC for ; Fri, 20 May 2022 14:16:18 -0700 (PDT) Received: by mail-pf1-x429.google.com with SMTP id x143so8709296pfc.11 for ; Fri, 20 May 2022 14:16:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=CWTSuBzkkT6cGFJIErCii42a5cTy6mj6tu0MEYBpYuw=; b=iGY0yXmtNOFzXqS3vyE8ZWNBhizNBWAbJa2rbH/Vkpzz7T3dDYF9Ol/pbh9S0aT/QO XAqSSTx79FNGM4luPdcTV/Iog4IgdOeHuLsBRaztYmtCMFvj43q1Ue3PA3TBcvLNo5/b vrpVPdbVgaIWH2tZoEFXRgwC8mh2vx6y6OO60N5j6CIvFX1tPqUnOEKRfunPOQr3b0qx FYZjBYPOpKz0LwrsaPFfj7T6567Q+Uik2rhKsrGy8XHJ9+2YtpcJW74Hzu54nURexFHm Bw55LeG5STs7fGwFdmdV3yvrAj0nfoJXzQgEZ/nVABaILAVfFB7HsmDEHYaYhfUATFlZ oyVg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=CWTSuBzkkT6cGFJIErCii42a5cTy6mj6tu0MEYBpYuw=; b=ZFOYBj7dNFeE/tETuNZ33SLQjztWoNrQbiRgaTzqBrqDxPn9uiCXkBieKYgNCoScWV fcnZRcXq2t9mMKK7FrcFS2l+XZzc2unQ5vGvQaBMjij//0XTZjCrJEfbgPLlvWZy4H05 tguX0W8q3jzc/545OQXVbVmM6z89vR0xJ1dtqS+IG6FQKo4Rpv+JhTAcNN9wOB8TsHXc iQ0Lng2SSduDBWpcDgqlZkrMi50dJjDr0OkUSKh6I1phI0RK38VOQw8JlWC2S//louym mq5uN4NroUyu75BknRAjRrTipU2NTN1cIVnQRCX/TuIHAIkQtHDJ8wN/6kAxSaTl0cZK uexg== X-Gm-Message-State: AOAM532Of6l1LbE6zfvJuITQRrRn5bq6lrz3kMlgd9KcERheVJWZK+tx 9hW8GqaAP4dh3LOkritVnkQ= X-Google-Smtp-Source: ABdhPJzR7VPOsIaHO4OZTJQpRymr7nffns8nLDPr1XGwHxgbnuzrVAXgs8g9ePKP8oLouCW6W3jqvA== X-Received: by 2002:a05:6a00:1c76:b0:510:8b76:93b5 with SMTP id s54-20020a056a001c7600b005108b7693b5mr12027301pfw.44.1653081376489; Fri, 20 May 2022 14:16:16 -0700 (PDT) Received: from localhost.localdomain (c-67-174-241-145.hsd1.ca.comcast.net. [67.174.241.145]) by smtp.gmail.com with ESMTPSA id h1-20020a170902748100b0015e8d4eb2c2sm167620pll.268.2022.05.20.14.16.15 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 20 May 2022 14:16:15 -0700 (PDT) From: Yang Shi To: vbabka@suse.cz, kirill.shutemov@linux.intel.com, akpm@linux-foundation.org Cc: shy828301@gmail.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH 3/7] mm: khugepaged: remove the redundant anon vma check Date: Fri, 20 May 2022 14:16:01 -0700 Message-Id: <20220520211605.51473-4-shy828301@gmail.com> X-Mailer: git-send-email 2.26.3 In-Reply-To: <20220520211605.51473-1-shy828301@gmail.com> References: <20220520211605.51473-1-shy828301@gmail.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" The hugepage_vma_check() already checked it, so remove the redundant check. Signed-off-by: Yang Shi --- mm/khugepaged.c | 3 --- 1 file changed, 3 deletions(-) diff --git a/mm/khugepaged.c b/mm/khugepaged.c index 5bdb30a76f05..4fe1dd3d8ef4 100644 --- a/mm/khugepaged.c +++ b/mm/khugepaged.c @@ -966,9 +966,6 @@ static int hugepage_vma_revalidate(struct mm_struct *mm= , unsigned long address, return SCAN_ADDRESS_RANGE; if (!hugepage_vma_check(vma, vma->vm_flags)) return SCAN_VMA_CHECK; - /* Anon VMA expected */ - if (!vma->anon_vma || !vma_is_anonymous(vma)) - return SCAN_VMA_CHECK; return 0; } =20 --=20 2.26.3 From nobody Thu May 7 18:53:26 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5920EC433EF for ; Fri, 20 May 2022 21:16:33 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1353654AbiETVQc (ORCPT ); Fri, 20 May 2022 17:16:32 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44984 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1353620AbiETVQT (ORCPT ); Fri, 20 May 2022 17:16:19 -0400 Received: from mail-pg1-x534.google.com (mail-pg1-x534.google.com [IPv6:2607:f8b0:4864:20::534]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id CC46E186FE for ; Fri, 20 May 2022 14:16:18 -0700 (PDT) Received: by mail-pg1-x534.google.com with SMTP id 137so8739287pgb.5 for ; Fri, 20 May 2022 14:16:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=FWCxIN+M68b8uiRtqF0Q6BeafTmXXsLbaGp3KwE5XtE=; b=MG5vVMB1O1NSDsob7g9pjDp5FCqlGuxDo+ahWdni+xykPUyYHOm+0D8noRAq74+JQ3 ymSgqr4s8HXyKD75hSX82thQ/A9aHCbx1yaH+uEPfkYDwW8qXoJmJOdc3k4PxTJ0l/x/ zem0fbbcKw3CLdBC5NhXLdocPf/BD4U7TApI0nu3S0/wZGBWPblHmC+20JV2qpH86OIq +fNUten4IZo85SyB0S0PDX93GJNkybmS5QUO2lt8NCZryroleEsgnJFw6M0khfq2iGMV 0wvG2C0aQJWNUL0fReMVyC4l1ULdpPVNgBy7a0TGl84OT/JETPlFJ1pjAL87O/I32R5A aHMw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=FWCxIN+M68b8uiRtqF0Q6BeafTmXXsLbaGp3KwE5XtE=; b=H5Eaf9BvXzSRZBheujysjC6mZfxcyOFfBDAO31u82MTL4K99Hw3oO5bDFc16x/27i7 C+HIWZKQPeiXTVee0HlGyz8BEfO8uyVePp+d4Ib/uIUaqjhKUVJlQr9TYoHZWcibZ7PL J0RYgamqlFRGUvMJ3UbPWMWKiCNazxVSfRYXea29Q1kGF3GMD+R5j31/PhW67vHQJKtD VIpL5tHwy8mC0DqnwX3FfqyPM8WuHpg4+D1wE2Aj3rNHXucGdWMo0SPLuciN5szcWMM9 CcPKicBssIq3NZ78TXdQx42yJl4yNfR3T7HnXcdE0okET8FNCcj30G/6JWYMA1inIxqq zxaA== X-Gm-Message-State: AOAM530260QU/Xh46AHoAwtOGJxIQUHIgjCpzUvGekIeFQuAfg+k4BOa ZQ+CUd36HtYdE8utwaUC6Cc= X-Google-Smtp-Source: ABdhPJyMccpJT5QHUtC5TElaI9zRj5Wzpv1ff3zBCJTMwc9y4u0+s7GN6DjhmisDKbpeVNwQvxjWew== X-Received: by 2002:a62:6410:0:b0:4f3:9654:266d with SMTP id y16-20020a626410000000b004f39654266dmr11682191pfb.59.1653081377797; Fri, 20 May 2022 14:16:17 -0700 (PDT) Received: from localhost.localdomain (c-67-174-241-145.hsd1.ca.comcast.net. [67.174.241.145]) by smtp.gmail.com with ESMTPSA id h1-20020a170902748100b0015e8d4eb2c2sm167620pll.268.2022.05.20.14.16.16 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 20 May 2022 14:16:17 -0700 (PDT) From: Yang Shi To: vbabka@suse.cz, kirill.shutemov@linux.intel.com, akpm@linux-foundation.org Cc: shy828301@gmail.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH 4/7] mm: khugepaged: use transhuge_vma_suitable replace open-code Date: Fri, 20 May 2022 14:16:02 -0700 Message-Id: <20220520211605.51473-5-shy828301@gmail.com> X-Mailer: git-send-email 2.26.3 In-Reply-To: <20220520211605.51473-1-shy828301@gmail.com> References: <20220520211605.51473-1-shy828301@gmail.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" The hugepage_vma_revalidate() needs to check if the address is still in the aligned HPAGE_PMD_SIZE area of the vma when reacquiring mmap_lock, but it was open-coded, use transhuge_vma_suitable() to do the job. And add proper comments for transhuge_vma_suitable(). Signed-off-by: Yang Shi --- include/linux/huge_mm.h | 6 ++++++ mm/khugepaged.c | 5 +---- 2 files changed, 7 insertions(+), 4 deletions(-) diff --git a/include/linux/huge_mm.h b/include/linux/huge_mm.h index a8f61db47f2a..79d5919beb83 100644 --- a/include/linux/huge_mm.h +++ b/include/linux/huge_mm.h @@ -128,6 +128,12 @@ static inline bool transhuge_vma_size_ok(struct vm_are= a_struct *vma) return false; } =20 +/* + * Do the below checks: + * - For non-anon vma, check if the vm_pgoff is HPAGE_PMD_NR aligned. + * - For all vmas, check if the haddr is in an aligned HPAGE_PMD_SIZE + * area. + */ static inline bool transhuge_vma_suitable(struct vm_area_struct *vma, unsigned long addr) { diff --git a/mm/khugepaged.c b/mm/khugepaged.c index 4fe1dd3d8ef4..51f0e6ea3977 100644 --- a/mm/khugepaged.c +++ b/mm/khugepaged.c @@ -951,7 +951,6 @@ static int hugepage_vma_revalidate(struct mm_struct *mm= , unsigned long address, struct vm_area_struct **vmap) { struct vm_area_struct *vma; - unsigned long hstart, hend; =20 if (unlikely(khugepaged_test_exit(mm))) return SCAN_ANY_PROCESS; @@ -960,9 +959,7 @@ static int hugepage_vma_revalidate(struct mm_struct *mm= , unsigned long address, if (!vma) return SCAN_VMA_NULL; =20 - hstart =3D (vma->vm_start + ~HPAGE_PMD_MASK) & HPAGE_PMD_MASK; - hend =3D vma->vm_end & HPAGE_PMD_MASK; - if (address < hstart || address + HPAGE_PMD_SIZE > hend) + if (!transhuge_vma_suitable(vma, address)) return SCAN_ADDRESS_RANGE; if (!hugepage_vma_check(vma, vma->vm_flags)) return SCAN_VMA_CHECK; --=20 2.26.3 From nobody Thu May 7 18:53:26 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id F198CC433EF for ; Fri, 20 May 2022 21:16:36 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1353663AbiETVQf (ORCPT ); Fri, 20 May 2022 17:16:35 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:45112 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1353626AbiETVQW (ORCPT ); Fri, 20 May 2022 17:16:22 -0400 Received: from mail-pf1-x429.google.com (mail-pf1-x429.google.com [IPv6:2607:f8b0:4864:20::429]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 3885818B34 for ; Fri, 20 May 2022 14:16:20 -0700 (PDT) Received: by mail-pf1-x429.google.com with SMTP id x143so8709296pfc.11 for ; Fri, 20 May 2022 14:16:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=uZ2MKo3NmwofqEpjSOVwGsJSdmq+ezPANYKqYBOCivQ=; b=GlUHTdvFzA+MiS/hUNdv6efIELq4i8AG5YNwOF3pCD/x8eHQ3mO/nO9rzy2mnR7XH1 ValbE4N4DtXGnshp8xRyTVvJzq2qACTDgi0ajm9hyH7DMsj5u4C2Jg8reA4QlehsVHbR be5ed75Yfa6fp38jcJq+/yfWqPbe2ETtPdPr+IVTBft67Os6XqydSX+tMIGtCeZUUOtp KiuB1bGfYS8zV6EUxtSrDxCoMRTFSmk3fUy67kQv2L/Eti7oN2DIRQTTx8mc0mLSl5Wg 2ppQD4KDMrosfKKb8KfPX1P4BH2GMZ9hcmKAcs2McyocWqVKB4rdSI04zdLEy3XJpuIN 76Cg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=uZ2MKo3NmwofqEpjSOVwGsJSdmq+ezPANYKqYBOCivQ=; b=LSgBCPn+ubTTK8VRE6HAeQfGw6taEF/9FmNh3Ys8hTkD6bCkVdtwDP+QxDDfYQHhpo INsYnAOMBX/uEX4B+7PUa5EJ7zvaqernqriEc0BRG2sIxj1eAdhdbhUG85+jy3JwC0YD Uv2KHFMloY2itVQPptYz0z+3asKtoOzOX1bm6WX8K+afDwKiBuD3zl3OK83svZrSi2d/ 0AoqKMFV8Qq2j8j6V5uGYLl/Mffe1uVfyDR4BHX4/AJsssOKup1CWSbF4y7SV2338mme yNkFwKlOn0OFjJLR7yfXDPqu7s0clgkqFB0esz29bjY1DbmBVCbGREjSjDOXH8Em96p3 dbmg== X-Gm-Message-State: AOAM533XfdN2jkf+NnWZkgHJXbZOfJdMv2IRzgiRdn7rkV8BlS7sIYwr dJBsPPJRpVHKjEMkozgq4ZA= X-Google-Smtp-Source: ABdhPJxHWgmmuDTl0VVnULJFQSbEJPMKkPtl2dOD6vFWj6vfs9p9296+xsaK0tTFIq3fQZmISbfDAQ== X-Received: by 2002:a62:b515:0:b0:50a:3d51:671e with SMTP id y21-20020a62b515000000b0050a3d51671emr11785398pfe.48.1653081379866; Fri, 20 May 2022 14:16:19 -0700 (PDT) Received: from localhost.localdomain (c-67-174-241-145.hsd1.ca.comcast.net. [67.174.241.145]) by smtp.gmail.com with ESMTPSA id h1-20020a170902748100b0015e8d4eb2c2sm167620pll.268.2022.05.20.14.16.17 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 20 May 2022 14:16:19 -0700 (PDT) From: Yang Shi To: vbabka@suse.cz, kirill.shutemov@linux.intel.com, akpm@linux-foundation.org Cc: shy828301@gmail.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH 5/7] mm: thp: kill transparent_hugepage_active() Date: Fri, 20 May 2022 14:16:03 -0700 Message-Id: <20220520211605.51473-6-shy828301@gmail.com> X-Mailer: git-send-email 2.26.3 In-Reply-To: <20220520211605.51473-1-shy828301@gmail.com> References: <20220520211605.51473-1-shy828301@gmail.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" The transparent_hugepage_active() was introduced to show THP eligibility bit in smaps in proc, smaps is the only user. But it actually does the similar check as hugepage_vma_check() which is used by khugepaged. We definitely don't have to maintain two similar checks, so kill transparent_hugepage_active(). Also move hugepage_vma_check() to huge_memory.c and huge_mm.h since it is not only for khugepaged anymore. Signed-off-by: Yang Shi --- fs/proc/task_mmu.c | 2 +- include/linux/huge_mm.h | 14 +++++++------ include/linux/khugepaged.h | 2 -- mm/huge_memory.c | 41 ++++++++++++++++++++++++++++++------ mm/khugepaged.c | 43 -------------------------------------- 5 files changed, 44 insertions(+), 58 deletions(-) diff --git a/fs/proc/task_mmu.c b/fs/proc/task_mmu.c index f9c9abb50bb7..ab4849d07a1f 100644 --- a/fs/proc/task_mmu.c +++ b/fs/proc/task_mmu.c @@ -863,7 +863,7 @@ static int show_smap(struct seq_file *m, void *v) __show_smap(m, &mss, false); =20 seq_printf(m, "THPeligible: %d\n", - transparent_hugepage_active(vma)); + hugepage_vma_check(vma, vma->vm_flags)); =20 if (arch_pkeys_enabled()) seq_printf(m, "ProtectionKey: %8u\n", vma_pkey(vma)); diff --git a/include/linux/huge_mm.h b/include/linux/huge_mm.h index 79d5919beb83..2da52ca8fd52 100644 --- a/include/linux/huge_mm.h +++ b/include/linux/huge_mm.h @@ -209,7 +209,8 @@ static inline bool file_thp_enabled(struct vm_area_stru= ct *vma) !inode_is_open_for_write(inode) && S_ISREG(inode->i_mode); } =20 -bool transparent_hugepage_active(struct vm_area_struct *vma); +extern bool hugepage_vma_check(struct vm_area_struct *vma, + unsigned long vm_flags); =20 #define transparent_hugepage_use_zero_page() \ (transparent_hugepage_flags & \ @@ -358,11 +359,6 @@ static inline bool __transparent_hugepage_enabled(stru= ct vm_area_struct *vma) return false; } =20 -static inline bool transparent_hugepage_active(struct vm_area_struct *vma) -{ - return false; -} - static inline bool transhuge_vma_size_ok(struct vm_area_struct *vma) { return false; @@ -380,6 +376,12 @@ static inline bool transhuge_vma_enabled(struct vm_are= a_struct *vma, return false; } =20 +static inline bool hugepage_vma_check(struct vm_area_struct *vma, + unsigned long vm_flags) +{ + return false; +} + static inline void prep_transhuge_page(struct page *page) {} =20 #define transparent_hugepage_flags 0UL diff --git a/include/linux/khugepaged.h b/include/linux/khugepaged.h index 392d34c3c59a..795c8d2170b0 100644 --- a/include/linux/khugepaged.h +++ b/include/linux/khugepaged.h @@ -10,8 +10,6 @@ extern struct attribute_group khugepaged_attr_group; extern int khugepaged_init(void); extern void khugepaged_destroy(void); extern int start_stop_khugepaged(void); -extern bool hugepage_vma_check(struct vm_area_struct *vma, - unsigned long vm_flags); extern void __khugepaged_enter(struct mm_struct *mm); extern void __khugepaged_exit(struct mm_struct *mm); extern void khugepaged_enter_vma(struct vm_area_struct *vma, diff --git a/mm/huge_memory.c b/mm/huge_memory.c index d633f97452c1..d9ec82892b05 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -69,18 +69,47 @@ static atomic_t huge_zero_refcount; struct page *huge_zero_page __read_mostly; unsigned long huge_zero_pfn __read_mostly =3D ~0UL; =20 -bool transparent_hugepage_active(struct vm_area_struct *vma) +bool hugepage_vma_check(struct vm_area_struct *vma, + unsigned long vm_flags) { + if (!transhuge_vma_enabled(vma, vm_flags)) + return false; + + if (vm_flags & VM_NO_KHUGEPAGED) + return false; + + /* Don't run khugepaged against DAX vma */ + if (vma_is_dax(vma)) + return false; + + if (vma->vm_file && !IS_ALIGNED((vma->vm_start >> PAGE_SHIFT) - + vma->vm_pgoff, HPAGE_PMD_NR)) + return false; + if (!transhuge_vma_size_ok(vma)) return false; - if (vma_is_anonymous(vma)) - return __transparent_hugepage_enabled(vma); - if (vma_is_shmem(vma)) + + /* Enabled via shmem mount options or sysfs settings. */ + if (shmem_file(vma->vm_file)) return shmem_huge_enabled(vma); - if (transhuge_vma_enabled(vma, vma->vm_flags) && file_thp_enabled(vma)) + + if (!khugepaged_enabled()) + return false; + + /* THP settings require madvise. */ + if (!(vm_flags & VM_HUGEPAGE) && !khugepaged_always()) + return false; + + /* Only regular file is valid */ + if (file_thp_enabled(vma)) return true; =20 - return false; + if (!vma->anon_vma || !vma_is_anonymous(vma)) + return false; + if (vma_is_temporary_stack(vma)) + return false; + + return true; } =20 static bool get_huge_zero_page(void) diff --git a/mm/khugepaged.c b/mm/khugepaged.c index 51f0e6ea3977..a7ac1ae418f8 100644 --- a/mm/khugepaged.c +++ b/mm/khugepaged.c @@ -437,49 +437,6 @@ static inline int khugepaged_test_exit(struct mm_struc= t *mm) return atomic_read(&mm->mm_users) =3D=3D 0; } =20 -bool hugepage_vma_check(struct vm_area_struct *vma, - unsigned long vm_flags) -{ - if (!transhuge_vma_enabled(vma, vm_flags)) - return false; - - if (vm_flags & VM_NO_KHUGEPAGED) - return false; - - /* Don't run khugepaged against DAX vma */ - if (vma_is_dax(vma)) - return false; - - if (vma->vm_file && !IS_ALIGNED((vma->vm_start >> PAGE_SHIFT) - - vma->vm_pgoff, HPAGE_PMD_NR)) - return false; - - if (!transhuge_vma_size_ok(vma)) - return false; - - /* Enabled via shmem mount options or sysfs settings. */ - if (shmem_file(vma->vm_file)) - return shmem_huge_enabled(vma); - - if (!khugepaged_enabled()) - return false; - - /* THP settings require madvise. */ - if (!(vm_flags & VM_HUGEPAGE) && !khugepaged_always()) - return false; - - /* Only regular file is valid */ - if (file_thp_enabled(vma)) - return true; - - if (!vma->anon_vma || !vma_is_anonymous(vma)) - return false; - if (vma_is_temporary_stack(vma)) - return false; - - return true; -} - void __khugepaged_enter(struct mm_struct *mm) { struct mm_slot *mm_slot; --=20 2.26.3 From nobody Thu May 7 18:53:26 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id E263BC433EF for ; Fri, 20 May 2022 21:16:41 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1353688AbiETVQj (ORCPT ); Fri, 20 May 2022 17:16:39 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:45376 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1353636AbiETVQY (ORCPT ); Fri, 20 May 2022 17:16:24 -0400 Received: from mail-pj1-x1031.google.com (mail-pj1-x1031.google.com [IPv6:2607:f8b0:4864:20::1031]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 34E0819025 for ; Fri, 20 May 2022 14:16:22 -0700 (PDT) Received: by mail-pj1-x1031.google.com with SMTP id ob14-20020a17090b390e00b001dff2a43f8cso2259599pjb.1 for ; Fri, 20 May 2022 14:16:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=TZGbrX4QM92lpEadFI2/iSQf+FDJdRaQWbUBPCSmje8=; b=UDHxs7Tka/6dGhK1B10RCJNyooHkQLV2mOuI+Tnf9llsxd5DSAeIb0RfTrKjghW7Wu /K1eGBERoI3QiWpcVu0Aj1n5DQvxi6qIoY825v+wpc1gGXTm0U/gZmQzAlTPjbPvdZ89 31sD4OtYTOVJXwEd1rJNyhQxmTqAgYvsL9+JAfsEf0ICkwVrvrBqf0125qAobUEl3f6w //F/ZalYdnOKXF/H8KP5tDveq2wW7joL27gehvWZOoZ4IRGP5kq1/LWqMwR7ZfDsMIqL /vOhfTPWXb9vfWKC5MMIaskT0tyMDMy30YStSBxt9sJaBQKF8bFMPIMAQLCDM3kPy6kG yrEg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=TZGbrX4QM92lpEadFI2/iSQf+FDJdRaQWbUBPCSmje8=; b=PqDCQId+LDU3QgG9rt7U3FAazJ9mMXxkhfC3nxxKPBEMBoMOPZtl8tNSLaDhnZiX64 +WAtH26lNUtIwtlSFMVbqUO8/tnOMVTF1+dWMT8nKExVh3XHYhD06idQCjP0zElvGBm0 zQRtDP6SLuvIwDXzYRmwhloIYaWmq+jOVGtHhjpM+lmES/k7vJ7I3z2mmE6PI50uHPrC XwTUyJreeVhvt6ieRjfqGy19VpCs+/SDQJKJRQRrUtl4ExTg+ncoJ/ta+d/rVwduPc/E OSKkubCM2ENK3Hfl+bE8NswsCNbnDaZFzyjgQDKeEGnbRN5tLaSEu/mmO+nWYNqeZ0K6 9wWA== X-Gm-Message-State: AOAM531HzODDxzGgH6zZ23acMBTu7hLJaXdf3K6QaPgA8paI8s9SKv1o vxdnie5TBp4ACHh9Ofxg8WY= X-Google-Smtp-Source: ABdhPJxu9xr7EW6hkAt6iaVTmS4PakGrCNCaum6wmHT0fFwAs3shwvx4OWMk38pErjAtuqxeDTpfbQ== X-Received: by 2002:a17:90b:4b8a:b0:1df:75f2:d2f4 with SMTP id lr10-20020a17090b4b8a00b001df75f2d2f4mr13096306pjb.109.1653081381644; Fri, 20 May 2022 14:16:21 -0700 (PDT) Received: from localhost.localdomain (c-67-174-241-145.hsd1.ca.comcast.net. [67.174.241.145]) by smtp.gmail.com with ESMTPSA id h1-20020a170902748100b0015e8d4eb2c2sm167620pll.268.2022.05.20.14.16.20 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 20 May 2022 14:16:20 -0700 (PDT) From: Yang Shi To: vbabka@suse.cz, kirill.shutemov@linux.intel.com, akpm@linux-foundation.org Cc: shy828301@gmail.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH 6/7] mm: thp: kill __transhuge_page_enabled() Date: Fri, 20 May 2022 14:16:04 -0700 Message-Id: <20220520211605.51473-7-shy828301@gmail.com> X-Mailer: git-send-email 2.26.3 In-Reply-To: <20220520211605.51473-1-shy828301@gmail.com> References: <20220520211605.51473-1-shy828301@gmail.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" The page fault path checks THP eligibility with __transhuge_page_enabled() which does the similar thing as hugepage_vma_check(), so use hugepage_vma_check() instead. However page fault allows DAX and !anon_vma cases, so added a new flag, in_pf, to hugepage_vma_check() to make page fault work correctly. The in_pf flag is also used to skip shmem and file THP for page fault since shmem handles THP in its own shmem_fault() and file THP allocation on fault is not supported yet. Also remove hugepage_vma_enabled() since hugepage_vma_check() is the only caller now, it is not necessary to have a helper function. Signed-off-by: Yang Shi --- fs/proc/task_mmu.c | 2 +- include/linux/huge_mm.h | 57 ++------------------------------------ include/linux/khugepaged.h | 2 +- mm/huge_memory.c | 24 +++++++++++----- mm/khugepaged.c | 8 +++--- mm/memory.c | 6 ++-- 6 files changed, 29 insertions(+), 70 deletions(-) diff --git a/fs/proc/task_mmu.c b/fs/proc/task_mmu.c index ab4849d07a1f..28f472a2eb51 100644 --- a/fs/proc/task_mmu.c +++ b/fs/proc/task_mmu.c @@ -863,7 +863,7 @@ static int show_smap(struct seq_file *m, void *v) __show_smap(m, &mss, false); =20 seq_printf(m, "THPeligible: %d\n", - hugepage_vma_check(vma, vma->vm_flags)); + hugepage_vma_check(vma, vma->vm_flags, false)); =20 if (arch_pkeys_enabled()) seq_printf(m, "ProtectionKey: %8u\n", vma_pkey(vma)); diff --git a/include/linux/huge_mm.h b/include/linux/huge_mm.h index 2da52ca8fd52..67c913e66e60 100644 --- a/include/linux/huge_mm.h +++ b/include/linux/huge_mm.h @@ -153,48 +153,6 @@ static inline bool transhuge_vma_suitable(struct vm_ar= ea_struct *vma, return true; } =20 -static inline bool transhuge_vma_enabled(struct vm_area_struct *vma, - unsigned long vm_flags) -{ - /* Explicitly disabled through madvise. */ - if ((vm_flags & VM_NOHUGEPAGE) || - test_bit(MMF_DISABLE_THP, &vma->vm_mm->flags)) - return false; - return true; -} - -/* - * to be used on vmas which are known to support THP. - * Use transparent_hugepage_active otherwise - */ -static inline bool __transparent_hugepage_enabled(struct vm_area_struct *v= ma) -{ - - /* - * If the hardware/firmware marked hugepage support disabled. - */ - if (transparent_hugepage_flags & (1 << TRANSPARENT_HUGEPAGE_NEVER_DAX)) - return false; - - if (!transhuge_vma_enabled(vma, vma->vm_flags)) - return false; - - if (vma_is_temporary_stack(vma)) - return false; - - if (transparent_hugepage_flags & (1 << TRANSPARENT_HUGEPAGE_FLAG)) - return true; - - if (vma_is_dax(vma)) - return true; - - if (transparent_hugepage_flags & - (1 << TRANSPARENT_HUGEPAGE_REQ_MADV_FLAG)) - return !!(vma->vm_flags & VM_HUGEPAGE); - - return false; -} - static inline bool file_thp_enabled(struct vm_area_struct *vma) { struct inode *inode; @@ -210,7 +168,7 @@ static inline bool file_thp_enabled(struct vm_area_stru= ct *vma) } =20 extern bool hugepage_vma_check(struct vm_area_struct *vma, - unsigned long vm_flags); + unsigned long vm_flags, bool in_pf); =20 #define transparent_hugepage_use_zero_page() \ (transparent_hugepage_flags & \ @@ -354,11 +312,6 @@ static inline bool folio_test_pmd_mappable(struct foli= o *folio) return false; } =20 -static inline bool __transparent_hugepage_enabled(struct vm_area_struct *v= ma) -{ - return false; -} - static inline bool transhuge_vma_size_ok(struct vm_area_struct *vma) { return false; @@ -370,14 +323,8 @@ static inline bool transhuge_vma_suitable(struct vm_ar= ea_struct *vma, return false; } =20 -static inline bool transhuge_vma_enabled(struct vm_area_struct *vma, - unsigned long vm_flags) -{ - return false; -} - static inline bool hugepage_vma_check(struct vm_area_struct *vma, - unsigned long vm_flags) + unsigned long vm_flags, bool in_pf) { return false; } diff --git a/include/linux/khugepaged.h b/include/linux/khugepaged.h index 795c8d2170b0..8a6452e089ca 100644 --- a/include/linux/khugepaged.h +++ b/include/linux/khugepaged.h @@ -55,7 +55,7 @@ static inline void khugepaged_enter(struct vm_area_struct= *vma, { if (!test_bit(MMF_VM_HUGEPAGE, &vma->vm_mm->flags) && khugepaged_enabled()) { - if (hugepage_vma_check(vma, vm_flags)) + if (hugepage_vma_check(vma, vm_flags, false)) __khugepaged_enter(vma->vm_mm); } } diff --git a/mm/huge_memory.c b/mm/huge_memory.c index d9ec82892b05..b4cab3aa2534 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -70,17 +70,26 @@ struct page *huge_zero_page __read_mostly; unsigned long huge_zero_pfn __read_mostly =3D ~0UL; =20 bool hugepage_vma_check(struct vm_area_struct *vma, - unsigned long vm_flags) + unsigned long vm_flags, bool in_pf) { - if (!transhuge_vma_enabled(vma, vm_flags)) + /* Explicitly disabled through madvise or prctl. */ + if ((vm_flags & VM_NOHUGEPAGE) || + test_bit(MMF_DISABLE_THP, &vma->vm_mm->flags)) return false; =20 + /* + * If the hardware/firmware marked hugepage support disabled. + */ + if (transparent_hugepage_flags & (1 << TRANSPARENT_HUGEPAGE_NEVER_DAX)) + return false; + + /* Special VMA and hugetlb VMA */ if (vm_flags & VM_NO_KHUGEPAGED) return false; =20 - /* Don't run khugepaged against DAX vma */ + /* khugepaged doesn't collapse DAX vma, but page fault is fine. */ if (vma_is_dax(vma)) - return false; + return in_pf; =20 if (vma->vm_file && !IS_ALIGNED((vma->vm_start >> PAGE_SHIFT) - vma->vm_pgoff, HPAGE_PMD_NR)) @@ -90,7 +99,7 @@ bool hugepage_vma_check(struct vm_area_struct *vma, return false; =20 /* Enabled via shmem mount options or sysfs settings. */ - if (shmem_file(vma->vm_file)) + if (!in_pf && shmem_file(vma->vm_file)) return shmem_huge_enabled(vma); =20 if (!khugepaged_enabled()) @@ -101,10 +110,11 @@ bool hugepage_vma_check(struct vm_area_struct *vma, return false; =20 /* Only regular file is valid */ - if (file_thp_enabled(vma)) + if (!in_pf && file_thp_enabled(vma)) return true; =20 - if (!vma->anon_vma || !vma_is_anonymous(vma)) + /* anon_vma may be not prepared before the first page fault */ + if ((!in_pf && !vma->anon_vma) || !vma_is_anonymous(vma)) return false; if (vma_is_temporary_stack(vma)) return false; diff --git a/mm/khugepaged.c b/mm/khugepaged.c index a7ac1ae418f8..292ed9c0db1e 100644 --- a/mm/khugepaged.c +++ b/mm/khugepaged.c @@ -473,7 +473,7 @@ void khugepaged_enter_vma(struct vm_area_struct *vma, { if (!test_bit(MMF_VM_HUGEPAGE, &vma->vm_mm->flags) && khugepaged_enabled()) { - if (hugepage_vma_check(vma, vm_flags)) + if (hugepage_vma_check(vma, vm_flags, false)) __khugepaged_enter(vma->vm_mm); } } @@ -918,7 +918,7 @@ static int hugepage_vma_revalidate(struct mm_struct *mm= , unsigned long address, =20 if (!transhuge_vma_suitable(vma, address)) return SCAN_ADDRESS_RANGE; - if (!hugepage_vma_check(vma, vma->vm_flags)) + if (!hugepage_vma_check(vma, vma->vm_flags, false)) return SCAN_VMA_CHECK; return 0; } @@ -1399,7 +1399,7 @@ void collapse_pte_mapped_thp(struct mm_struct *mm, un= signed long addr) * the valid THP. Add extra VM_HUGEPAGE so hugepage_vma_check() * will not fail the vma for missing VM_HUGEPAGE */ - if (!hugepage_vma_check(vma, vma->vm_flags | VM_HUGEPAGE)) + if (!hugepage_vma_check(vma, vma->vm_flags | VM_HUGEPAGE, false)) return; =20 /* Keep pmd pgtable for uffd-wp; see comment in retract_page_tables() */ @@ -2091,7 +2091,7 @@ static unsigned int khugepaged_scan_mm_slot(unsigned = int pages, progress++; break; } - if (!hugepage_vma_check(vma, vma->vm_flags)) { + if (!hugepage_vma_check(vma, vma->vm_flags, false)) { skip: progress++; continue; diff --git a/mm/memory.c b/mm/memory.c index 95735bee3876..0850b3e38dbf 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -4979,7 +4979,8 @@ static vm_fault_t __handle_mm_fault(struct vm_area_st= ruct *vma, if (!vmf.pud) return VM_FAULT_OOM; retry_pud: - if (pud_none(*vmf.pud) && __transparent_hugepage_enabled(vma)) { + if (pud_none(*vmf.pud) && + hugepage_vma_check(vma, vma->vm_flags, true)) { ret =3D create_huge_pud(&vmf); if (!(ret & VM_FAULT_FALLBACK)) return ret; @@ -5012,7 +5013,8 @@ static vm_fault_t __handle_mm_fault(struct vm_area_st= ruct *vma, if (pud_trans_unstable(vmf.pud)) goto retry_pud; =20 - if (pmd_none(*vmf.pmd) && __transparent_hugepage_enabled(vma)) { + if (pmd_none(*vmf.pmd) && + hugepage_vma_check(vma, vma->vm_flags, true)) { ret =3D create_huge_pmd(&vmf); if (!(ret & VM_FAULT_FALLBACK)) return ret; --=20 2.26.3 From nobody Thu May 7 18:53:26 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id AFA0AC433F5 for ; Fri, 20 May 2022 21:16:54 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1353652AbiETVQp (ORCPT ); Fri, 20 May 2022 17:16:45 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:45110 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1353650AbiETVQ1 (ORCPT ); Fri, 20 May 2022 17:16:27 -0400 Received: from mail-pj1-x1030.google.com (mail-pj1-x1030.google.com [IPv6:2607:f8b0:4864:20::1030]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id BC4931B79D for ; Fri, 20 May 2022 14:16:24 -0700 (PDT) Received: by mail-pj1-x1030.google.com with SMTP id fw21-20020a17090b129500b001df9f62edd6so7111856pjb.0 for ; Fri, 20 May 2022 14:16:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=j4XmJAWPYD2D8xrDs4B9doFxdeAqtS3ydjsiwVqFNQE=; b=Ptl/vJ+4tr0wo8YH9eqwLAvaSJxi7Oycnq92HxfoZk+QdfTfA9ENKktv7+IZjg4dQA Tnji/iUJmJpcFvSWDbCv4VN85SuKnLwbT0huZV6bIyCoRD/7bhs0irQMtF8Lnc7ZY5Q7 wOe9j+hlD+5P7UxV/dOtjIyR381ynQL1SqKIiUsdjoTXtM7kP+Wyw+cZdIglDsx1pDeK 9+HigMz9haP+HCDkKNneVDaB0D2N0tyMbU+HA5dEbgRdmoCqplw1qxf0V8I6+wunGFDl 5lfbF7srBIkNk8zjstYMnHqPez+C1RfCdxiUC3Izu8Owg3I4cjReT4lCRDb/or6V4+zu +/Hw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=j4XmJAWPYD2D8xrDs4B9doFxdeAqtS3ydjsiwVqFNQE=; b=y1kL9eaAId/aNNp1fAKHhd/8ecSDnWt/hDG6gg3PL477CXmEO+eDTQsy6qejxiLR5H EFiQ6FqziMxaLRMUHycwGBktAFG/AfRs690kxiCpjX025QODmKNBYc2Nbmh8ZiuadTe2 HQ3P+fGx6CX9WtAKn0oL84KJ+p/pghEbJZekjYTVtzBkRD6vIhMZ41ONNTdVkC9vxLIL JvhDP22gjLiCMT5o8822x9vlGbY281Vj7YN0l7GEs4FOfD7Qt9cG7tjrdMAvr0W6An7q v/W+2U9nKheYjyeTpsC6di2J3kR5fIkj54SWj7autB1/qMbiiKfZC3cIIb0EluINGnwg hlXg== X-Gm-Message-State: AOAM530Inkys0K4TQQDDOgQL/t7+nMMgx2HJ7yY0V0Rc4aXGHHYaMus5 e3jH8GDSI3sfy15NHKjUZ3U= X-Google-Smtp-Source: ABdhPJy9LubXKnw2CwDr67vvJdDrVydqGO6/z63N55tXZRS8KRReOt/HeiFZIFFyfXfdOhEcnhSYdQ== X-Received: by 2002:a17:902:f551:b0:15e:999a:2425 with SMTP id h17-20020a170902f55100b0015e999a2425mr11172213plf.3.1653081383403; Fri, 20 May 2022 14:16:23 -0700 (PDT) Received: from localhost.localdomain (c-67-174-241-145.hsd1.ca.comcast.net. [67.174.241.145]) by smtp.gmail.com with ESMTPSA id h1-20020a170902748100b0015e8d4eb2c2sm167620pll.268.2022.05.20.14.16.21 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 20 May 2022 14:16:22 -0700 (PDT) From: Yang Shi To: vbabka@suse.cz, kirill.shutemov@linux.intel.com, akpm@linux-foundation.org Cc: shy828301@gmail.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH 7/7] mm: khugepaged: reorg some khugepaged helpers Date: Fri, 20 May 2022 14:16:05 -0700 Message-Id: <20220520211605.51473-8-shy828301@gmail.com> X-Mailer: git-send-email 2.26.3 In-Reply-To: <20220520211605.51473-1-shy828301@gmail.com> References: <20220520211605.51473-1-shy828301@gmail.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" The khugepaged_{enabled|always|req_madv} are not khugepaged only anymore, move them to huge_mm.h and rename to hugepage_flags_xxx, and remove khugepaged_req_madv due to no users. Also move khugepaged_defrag to khugepaged.c since its only caller is in that file, it doesn't have to be in a header file. Signed-off-by: Yang Shi --- include/linux/huge_mm.h | 8 ++++++++ include/linux/khugepaged.h | 17 +---------------- mm/huge_memory.c | 4 ++-- mm/khugepaged.c | 18 +++++++++++------- 4 files changed, 22 insertions(+), 25 deletions(-) diff --git a/include/linux/huge_mm.h b/include/linux/huge_mm.h index 67c913e66e60..e6f89d8ad5f8 100644 --- a/include/linux/huge_mm.h +++ b/include/linux/huge_mm.h @@ -116,6 +116,14 @@ extern struct kobj_attribute shmem_enabled_attr; =20 extern unsigned long transparent_hugepage_flags; =20 +#define hugepage_flags_enabled() \ + (transparent_hugepage_flags & \ + ((1<flags)) @@ -53,8 +39,7 @@ static inline void khugepaged_exit(struct mm_struct *mm) static inline void khugepaged_enter(struct vm_area_struct *vma, unsigned long vm_flags) { - if (!test_bit(MMF_VM_HUGEPAGE, &vma->vm_mm->flags) && - khugepaged_enabled()) { + if (!test_bit(MMF_VM_HUGEPAGE, &vma->vm_mm->flags)) { if (hugepage_vma_check(vma, vm_flags, false)) __khugepaged_enter(vma->vm_mm); } diff --git a/mm/huge_memory.c b/mm/huge_memory.c index b4cab3aa2534..f4d7b6378b83 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -102,11 +102,11 @@ bool hugepage_vma_check(struct vm_area_struct *vma, if (!in_pf && shmem_file(vma->vm_file)) return shmem_huge_enabled(vma); =20 - if (!khugepaged_enabled()) + if (!hugepage_flags_enabled()) return false; =20 /* THP settings require madvise. */ - if (!(vm_flags & VM_HUGEPAGE) && !khugepaged_always()) + if (!(vm_flags & VM_HUGEPAGE) && !hugepage_flags_always()) return false; =20 /* Only regular file is valid */ diff --git a/mm/khugepaged.c b/mm/khugepaged.c index 292ed9c0db1e..37ea5a5396cd 100644 --- a/mm/khugepaged.c +++ b/mm/khugepaged.c @@ -472,7 +472,7 @@ void khugepaged_enter_vma(struct vm_area_struct *vma, unsigned long vm_flags) { if (!test_bit(MMF_VM_HUGEPAGE, &vma->vm_mm->flags) && - khugepaged_enabled()) { + hugepage_flags_enabled()) { if (hugepage_vma_check(vma, vm_flags, false)) __khugepaged_enter(vma->vm_mm); } @@ -763,6 +763,10 @@ static bool khugepaged_scan_abort(int nid) return false; } =20 +#define khugepaged_defrag() \ + (transparent_hugepage_flags & \ + (1<