From nobody Tue Apr 28 03:55:20 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3A2CEC433EF for ; Mon, 6 Jun 2022 21:44:36 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233018AbiFFVoe (ORCPT ); Mon, 6 Jun 2022 17:44:34 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46496 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234010AbiFFVo3 (ORCPT ); Mon, 6 Jun 2022 17:44:29 -0400 Received: from mail-pj1-x1032.google.com (mail-pj1-x1032.google.com [IPv6:2607:f8b0:4864:20::1032]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 37FF88148A for ; Mon, 6 Jun 2022 14:44:21 -0700 (PDT) Received: by mail-pj1-x1032.google.com with SMTP id l20-20020a17090a409400b001dd2a9d555bso13722272pjg.0 for ; Mon, 06 Jun 2022 14:44:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=SWErXY/21cRT+JuPcizRjGDN5I2BQSMNvqCbtDuuEgI=; b=I5APtot2v61wKkN0XCT4995ZIiul394ZNakXAWx3CnTxQvkSfrURRzUJ0YUHHBDmoN n7WES+tRmlehNiQ+3ho3huUDvxWLUQKDPTnFK3bHGNIBaYMyTK43rUlQCWrS8jBRXYqw asEC1jsCThymiE5h5B8duobPkjqKD79mgUEVqb3+uLy+sG21Xr8hCgblpLAIDX9GBdY+ YnoLjc3K3sv/qGe2DmVNbu56MCjX9ZwGKXZc3UA8b6ez1d+JTPISyuuAZIt5r+pAMldH A3XSVAeQxTWsGtKRdSh1OHuMOvFPqiIUvWqLjyZT3XWfA2X2Q+VmkOMkXtoUx/+j9lcl C7dQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=SWErXY/21cRT+JuPcizRjGDN5I2BQSMNvqCbtDuuEgI=; b=f0SRtls/4NDolOUetszSG76QhkOTf4a85+kxMFUGjffxoXL/mzpbWvo70dQYBOoP8r DqFju8Yh6oOJhpn4xYbrvc24S1DPNL00VvbU3unLFLWXvN+qTFWfvVCukG9sYelyAsvH cMGVEsl74uJWA8U6XKmxPXjNdVxZWEy5ZEqJh0VY1f0DviesggN3ooCdqeN/2Euuy7VP yG6cGd9Pj+AcRbRkhTzFf8oOGH2iPKWafUl1aCWKoKjfsVhWNwqFIzEPsvuPPEGUdf6i WcFpUBGzg6gi6t6VNo+RppoeCqPwKpV64CDhKaEfrW8cMse9aHbItDc7lyenAfPGNmR6 Dg7g== X-Gm-Message-State: AOAM533ExJ3P73a3WV+DCiTjSfBqHXuzg3MAcnB0or6DNyiHFHhSQtbU c4oWHdtvAuxslgaEJJyIvC4= X-Google-Smtp-Source: ABdhPJxxmkvlq3kb5ZvctnDvImOgaVYAMOt1gHJ3eNFjJyFulmHv+ybHJWXnTeDs5PmIyxjHCkv45Q== X-Received: by 2002:a17:902:b282:b0:163:ffe6:2cf9 with SMTP id u2-20020a170902b28200b00163ffe62cf9mr26504023plr.76.1654551860779; Mon, 06 Jun 2022 14:44:20 -0700 (PDT) Received: from localhost.biz ([8.25.197.27]) by smtp.gmail.com with ESMTPSA id a4-20020a170903100400b0016397da033csm10881675plb.62.2022.06.06.14.44.19 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 06 Jun 2022 14:44:20 -0700 (PDT) From: Yang Shi To: vbabka@suse.cz, kirill.shutemov@linux.intel.com, willy@infradead.org, akpm@linux-foundation.org Cc: shy828301@gmail.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [v3 PATCH 1/7] mm: khugepaged: check THP flag in hugepage_vma_check() Date: Mon, 6 Jun 2022 14:44:08 -0700 Message-Id: <20220606214414.736109-2-shy828301@gmail.com> X-Mailer: git-send-email 2.26.3 In-Reply-To: <20220606214414.736109-1-shy828301@gmail.com> References: <20220606214414.736109-1-shy828301@gmail.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Currently the THP flag check in hugepage_vma_check() will fallthrough if the flag is NEVER and VM_HUGEPAGE is set. This is not a problem for now since all the callers have the flag checked before or can't be invoked if the flag is NEVER. However, the following patch will call hugepage_vma_check() in more places, for example, page fault, so this flag must be checked in hugepge_vma_check(). Signed-off-by: Yang Shi Reviewed-by: Miaohe Lin Reviewed-by: Zach O'Keefe --- mm/khugepaged.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/mm/khugepaged.c b/mm/khugepaged.c index 671ac7800e53..84b9cf4b9be9 100644 --- a/mm/khugepaged.c +++ b/mm/khugepaged.c @@ -458,6 +458,9 @@ bool hugepage_vma_check(struct vm_area_struct *vma, if (shmem_file(vma->vm_file)) return shmem_huge_enabled(vma); =20 + if (!khugepaged_enabled()) + return false; + /* THP settings require madvise. */ if (!(vm_flags & VM_HUGEPAGE) && !khugepaged_always()) return false; --=20 2.26.3 From nobody Tue Apr 28 03:55:20 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id C3516C433EF for ; Mon, 6 Jun 2022 21:44:39 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233943AbiFFVoh (ORCPT ); Mon, 6 Jun 2022 17:44:37 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46342 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234760AbiFFVoa (ORCPT ); Mon, 6 Jun 2022 17:44:30 -0400 Received: from mail-pl1-x630.google.com (mail-pl1-x630.google.com [IPv6:2607:f8b0:4864:20::630]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C172A91565 for ; Mon, 6 Jun 2022 14:44:22 -0700 (PDT) Received: by mail-pl1-x630.google.com with SMTP id q18so13154041pln.12 for ; Mon, 06 Jun 2022 14:44:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=D/C1dubPl+rj/RbGuMLBKXaMg54FT3nuKqdm5O29Xsg=; b=YRS9DKv1jCdO/JdsfD2Zdx2c6vo3ureR7Qm46tf+wcDQKy1FeN7iCjcyusFTyim11S d0j0ZZpoAaKyN+IaCjtJMSz9RrTX8Q3XcRUCrbEwBT8IazrwYVwuLwUQZAPvljDjYQFO ldHcD5wfIiPmSXhaTPA9LHw9ZUhMTbUSlZ40NzcPhON+Pr3zxvPwDSODHzYSHcjjhK9g Z38Uhow01G4f8nz97ErvS2BY6kUS+gFldseEqN0iI1/pBd1K9x5LG5XvmW3+WTctCrdR mr/60PMYJyB7y+cU2eD3G2ZfqOZ9jzYJM4+mZZ6eRiqmevTRTU534YjwsFgh0MJpVknB Gprg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=D/C1dubPl+rj/RbGuMLBKXaMg54FT3nuKqdm5O29Xsg=; b=NednS/Ov/kzwePIfsn6B3O0oAc5WZ86JcoHl8UlWIwXwtYZ8cRimIhO+x3miz9XvqM fTSHkpYZLU8TOqAGdYZU3rB8ERjQDp1i6GBmYCblD0HOfQ+7hxO9f0BMFge9rVTP9B6G n8BWHUd900C+kB/a9cdSIfQgkX8pg3ANqHTm9hN08b07E7ypkxjmOkRRO2JGvR1Zgg+b YBKnQyauvZyrn+mtLlmAERTDd13RgKq/9dL2qhIZcqoozC6Ah/y3JxePncdzeIKGXC1e 4L7Y5v2HuFsuEh+3Ri8fGR0Dha2dkwo+sOaYi6QCgukiOJ5Z8RmqwGZh2KOF1dEnyBvT M5fg== X-Gm-Message-State: AOAM533S1M18EXizG4+9QVf+uj6TQAhvOUxu8yZZ6DFu55P5krZyCVSG aQLe/TQNXmLZadv0yMkIEzA= X-Google-Smtp-Source: ABdhPJzikW8LtgDjXM0E0yc+iosNsTNVfvkgGIPBuUaO7oCFPC7sJOjOY3z/ay2XBf6L9aDLJBVqBA== X-Received: by 2002:a17:90a:1f4c:b0:1e6:6f77:c573 with SMTP id y12-20020a17090a1f4c00b001e66f77c573mr30435048pjy.17.1654551862332; Mon, 06 Jun 2022 14:44:22 -0700 (PDT) Received: from localhost.biz ([8.25.197.27]) by smtp.gmail.com with ESMTPSA id a4-20020a170903100400b0016397da033csm10881675plb.62.2022.06.06.14.44.21 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 06 Jun 2022 14:44:21 -0700 (PDT) From: Yang Shi To: vbabka@suse.cz, kirill.shutemov@linux.intel.com, willy@infradead.org, akpm@linux-foundation.org Cc: shy828301@gmail.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [v3 PATCH 2/7] mm: thp: introduce transhuge_vma_size_ok() helper Date: Mon, 6 Jun 2022 14:44:09 -0700 Message-Id: <20220606214414.736109-3-shy828301@gmail.com> X-Mailer: git-send-email 2.26.3 In-Reply-To: <20220606214414.736109-1-shy828301@gmail.com> References: <20220606214414.736109-1-shy828301@gmail.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" There are couple of places that check whether the vma size is ok for THP or not, they are open coded and duplicate, introduce transhuge_vma_size_ok() helper to do the job. Signed-off-by: Yang Shi --- include/linux/huge_mm.h | 17 +++++++++++++++++ mm/huge_memory.c | 5 +---- mm/khugepaged.c | 12 ++++++------ 3 files changed, 24 insertions(+), 10 deletions(-) diff --git a/include/linux/huge_mm.h b/include/linux/huge_mm.h index 648cb3ce7099..a8f61db47f2a 100644 --- a/include/linux/huge_mm.h +++ b/include/linux/huge_mm.h @@ -116,6 +116,18 @@ extern struct kobj_attribute shmem_enabled_attr; =20 extern unsigned long transparent_hugepage_flags; =20 +/* + * The vma size has to be large enough to hold an aligned HPAGE_PMD_SIZE a= rea. + */ +static inline bool transhuge_vma_size_ok(struct vm_area_struct *vma) +{ + if (round_up(vma->vm_start, HPAGE_PMD_SIZE) < + (vma->vm_end & HPAGE_PMD_MASK)) + return true; + + return false; +} + static inline bool transhuge_vma_suitable(struct vm_area_struct *vma, unsigned long addr) { @@ -345,6 +357,11 @@ static inline bool transparent_hugepage_active(struct = vm_area_struct *vma) return false; } =20 +static inline bool transhuge_vma_size_ok(struct vm_area_struct *vma) +{ + return false; +} + static inline bool transhuge_vma_suitable(struct vm_area_struct *vma, unsigned long addr) { diff --git a/mm/huge_memory.c b/mm/huge_memory.c index 48182c8fe151..36ada544e494 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -71,10 +71,7 @@ unsigned long huge_zero_pfn __read_mostly =3D ~0UL; =20 bool transparent_hugepage_active(struct vm_area_struct *vma) { - /* The addr is used to check if the vma size fits */ - unsigned long addr =3D (vma->vm_end & HPAGE_PMD_MASK) - HPAGE_PMD_SIZE; - - if (!transhuge_vma_suitable(vma, addr)) + if (!transhuge_vma_size_ok(vma)) return false; if (vma_is_anonymous(vma)) return __transparent_hugepage_enabled(vma); diff --git a/mm/khugepaged.c b/mm/khugepaged.c index 84b9cf4b9be9..d0f8020164fc 100644 --- a/mm/khugepaged.c +++ b/mm/khugepaged.c @@ -454,6 +454,9 @@ bool hugepage_vma_check(struct vm_area_struct *vma, vma->vm_pgoff, HPAGE_PMD_NR)) return false; =20 + if (!transhuge_vma_size_ok(vma)) + return false; + /* Enabled via shmem mount options or sysfs settings. */ if (shmem_file(vma->vm_file)) return shmem_huge_enabled(vma); @@ -512,9 +515,7 @@ void khugepaged_enter_vma(struct vm_area_struct *vma, unsigned long vm_flags) { if (!test_bit(MMF_VM_HUGEPAGE, &vma->vm_mm->flags) && - khugepaged_enabled() && - (((vma->vm_start + ~HPAGE_PMD_MASK) & HPAGE_PMD_MASK) < - (vma->vm_end & HPAGE_PMD_MASK))) { + khugepaged_enabled()) { if (hugepage_vma_check(vma, vm_flags)) __khugepaged_enter(vma->vm_mm); } @@ -2142,10 +2143,9 @@ static unsigned int khugepaged_scan_mm_slot(unsigned= int pages, progress++; continue; } - hstart =3D (vma->vm_start + ~HPAGE_PMD_MASK) & HPAGE_PMD_MASK; + + hstart =3D round_up(vma->vm_start, HPAGE_PMD_SIZE); hend =3D vma->vm_end & HPAGE_PMD_MASK; - if (hstart >=3D hend) - goto skip; if (khugepaged_scan.address > hend) goto skip; if (khugepaged_scan.address < hstart) --=20 2.26.3 From nobody Tue Apr 28 03:55:20 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 38ED8C43334 for ; Mon, 6 Jun 2022 21:45:05 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234974AbiFFVou (ORCPT ); Mon, 6 Jun 2022 17:44:50 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47020 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234802AbiFFVoa (ORCPT ); Mon, 6 Jun 2022 17:44:30 -0400 Received: from mail-pj1-x1030.google.com (mail-pj1-x1030.google.com [IPv6:2607:f8b0:4864:20::1030]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 420E98DDCD for ; Mon, 6 Jun 2022 14:44:24 -0700 (PDT) Received: by mail-pj1-x1030.google.com with SMTP id w13-20020a17090a780d00b001e8961b355dso1823630pjk.5 for ; Mon, 06 Jun 2022 14:44:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=TeR+saO9bGVSVDYLkT6whuJkztc98Y7uqJRw9Zwr6CA=; b=HKhirssn3eLksqebeSYrk+jXcfBxshfBpkTxjBPO9398Kqa18Cm3e71kgcZu26dIvY o6XXpciQWvJwXzV5TU9EluS5IVMmvxncZ5Mi/Vz+lQNfHrPN3rzLvi/l00C5t+Za+YHj YSPmh2kgWJ07k6J+1bq9EH3HdkCP54M7O6Vm/tRIEY34h4UfvDTYtD/vhvyQzncqUZA5 lIVQRmQhGGRj5O6xWtvnzee96vGcj+iwnnmw/OVFSsrJmnCfV4t/Ft+pb31IkLeX0FcM 9gByGWOdP6wH6PFe0kKeAl3/kjT4wP6BDTfJq8QgNMLS5uVGvXEfYFP0xTXUn4EZHlGi 7T5Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=TeR+saO9bGVSVDYLkT6whuJkztc98Y7uqJRw9Zwr6CA=; b=NaZaUoVymyJ0OOmWeRKV2GIKUOLYlsR3I4CDlXSXbfwbCJO2nemnaz/iyWQwPTkkK7 zYcbznfH//EFci4UxWgI84TrT8LADmw+rfU40CIlZ0QfqAKbwcyAp90MVrXkVzCCQ7WQ p1e6asTFwgK0I1ZDbx2Uywp1ed/bKiJZw/m1lHNGl+Mgn+QycreM1o9WhDeB11N0E1Er 0aBaHlRtqKS4VpvykJxb6asiFzb4/0NS1aeJnDNF5nq7SqTMnlR2q0O/EOXASjWmvwS+ i9ADGjqhLNIcN1J1Bpor5tJnEFZiW6yn9eGLMCSgeTxiuK1XJdJzn2DOp775MZD7ZMjx eNEA== X-Gm-Message-State: AOAM532bU3VJONsRwaRWrDPBfWF+W1eNY3wu34z7vzHYlyxF2c2fU2bx PtcEogX1lN1Xy8aeo3mYJ8M= X-Google-Smtp-Source: ABdhPJx+QgadWDwD5L0qXpqeZx6oasMAo4joDNR6AQwapxgpVmCIYIAdm45+392KvIhA5+qYQ6m75A== X-Received: by 2002:a17:90b:4a90:b0:1e8:626c:e1b2 with SMTP id lp16-20020a17090b4a9000b001e8626ce1b2mr13173095pjb.141.1654551863738; Mon, 06 Jun 2022 14:44:23 -0700 (PDT) Received: from localhost.biz ([8.25.197.27]) by smtp.gmail.com with ESMTPSA id a4-20020a170903100400b0016397da033csm10881675plb.62.2022.06.06.14.44.22 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 06 Jun 2022 14:44:23 -0700 (PDT) From: Yang Shi To: vbabka@suse.cz, kirill.shutemov@linux.intel.com, willy@infradead.org, akpm@linux-foundation.org Cc: shy828301@gmail.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [v3 PATCH 3/7] mm: khugepaged: remove the redundant anon vma check Date: Mon, 6 Jun 2022 14:44:10 -0700 Message-Id: <20220606214414.736109-4-shy828301@gmail.com> X-Mailer: git-send-email 2.26.3 In-Reply-To: <20220606214414.736109-1-shy828301@gmail.com> References: <20220606214414.736109-1-shy828301@gmail.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" The hugepage_vma_check() already checked it, so remove the redundant check. Signed-off-by: Yang Shi --- mm/khugepaged.c | 3 --- 1 file changed, 3 deletions(-) diff --git a/mm/khugepaged.c b/mm/khugepaged.c index d0f8020164fc..7a5d1c1a1833 100644 --- a/mm/khugepaged.c +++ b/mm/khugepaged.c @@ -966,9 +966,6 @@ static int hugepage_vma_revalidate(struct mm_struct *mm= , unsigned long address, return SCAN_ADDRESS_RANGE; if (!hugepage_vma_check(vma, vma->vm_flags)) return SCAN_VMA_CHECK; - /* Anon VMA expected */ - if (!vma->anon_vma || !vma_is_anonymous(vma)) - return SCAN_VMA_CHECK; return 0; } =20 --=20 2.26.3 From nobody Tue Apr 28 03:55:20 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8E21BCCA473 for ; Mon, 6 Jun 2022 21:45:06 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235138AbiFFVoy (ORCPT ); Mon, 6 Jun 2022 17:44:54 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47022 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232897AbiFFVoa (ORCPT ); Mon, 6 Jun 2022 17:44:30 -0400 Received: from mail-pg1-x533.google.com (mail-pg1-x533.google.com [IPv6:2607:f8b0:4864:20::533]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 99A1898094 for ; Mon, 6 Jun 2022 14:44:25 -0700 (PDT) Received: by mail-pg1-x533.google.com with SMTP id q123so14021435pgq.6 for ; Mon, 06 Jun 2022 14:44:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=Ij7n7UZO/IK0zKEVoAHKDokBQE/6XWgjwPeS6gPTqcc=; b=mXJ/hahxbI7vbEPKM3HNI31LOwv12MOVMykwZIF/4WcDnhIo7yCNxtwhCyA+9ateh8 /dsaYwTgy+aYi1g7b3kCczzNJ+WEAy6PcjMcJ5J0IiDz9N71++VS+nAfh/2zpRkZHRfa vYvtioaXpeUvymXxTzMCRSVMrRPyXiSIYuL1uN7AZNHMXqfLKTCTvXDqHwZWqahHfM2L WCDWc/QOyZDpHyCDtkPuRhYal0AaWB8/j7KaKcBGmN9/a5lpVSIJnaHwWII2HPckBuW7 rl6/VYaSW5bRJ2ZinxJHxIyok/ClC8cfp/QXquPk6mOoO7jd8h/jlQpaGDZwA+AlTVPK vFDQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Ij7n7UZO/IK0zKEVoAHKDokBQE/6XWgjwPeS6gPTqcc=; b=3Ercl1cuYhfRGc3vubGmBB/N/2c58AYz30wypTaWw9+71Za4Dbbz0zkAsdUe8Zem+q 9BnOMSM+LXoB/MVFDGLFStE6frpLGuwrGhNsHby//UZLyOAsfBqqr26XULFfz+/OD/qP AG8ycwahGsz3sjVcNWiSLBnL+BdrKZpI6H7ITFo+NIejvJ8Il5/RMpZ0CeNm8lxcXi8b ROXrpSPCrf44XAIbE3dtXhiRuG58KiWqcHim5s1AAJUpziqtHdmEurEMXBNbkPBaZ0a2 IMyDjNhV0omp9+2XoyzE2W6eJmH9CXTl47dDCAI0Fy9noewmbsY3v21X9yIeZTkN0TQB COWQ== X-Gm-Message-State: AOAM532KTq+MniNGJVj3Y3b4ogn2n8JooPxFiozbsB2Wir1PZEtW/SNq wNF5Mnb90ABfXLkoJkv+8JpmLUPUbRTm6w== X-Google-Smtp-Source: ABdhPJzMLSIncjP6QHixKNI/DAPcwxVzDs8Kf+dHw2xn4cBNcjVPbSTe/tA+bImKxTtxSNxqvHYUlA== X-Received: by 2002:a65:4907:0:b0:3fd:bc3e:fb0a with SMTP id p7-20020a654907000000b003fdbc3efb0amr6869220pgs.123.1654551865142; Mon, 06 Jun 2022 14:44:25 -0700 (PDT) Received: from localhost.biz ([8.25.197.27]) by smtp.gmail.com with ESMTPSA id a4-20020a170903100400b0016397da033csm10881675plb.62.2022.06.06.14.44.23 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 06 Jun 2022 14:44:24 -0700 (PDT) From: Yang Shi To: vbabka@suse.cz, kirill.shutemov@linux.intel.com, willy@infradead.org, akpm@linux-foundation.org Cc: shy828301@gmail.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [v3 PATCH 4/7] mm: khugepaged: use transhuge_vma_suitable replace open-code Date: Mon, 6 Jun 2022 14:44:11 -0700 Message-Id: <20220606214414.736109-5-shy828301@gmail.com> X-Mailer: git-send-email 2.26.3 In-Reply-To: <20220606214414.736109-1-shy828301@gmail.com> References: <20220606214414.736109-1-shy828301@gmail.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" The hugepage_vma_revalidate() needs to check if the address is still in the aligned HPAGE_PMD_SIZE area of the vma when reacquiring mmap_lock, but it was open-coded, use transhuge_vma_suitable() to do the job. And add proper comments for transhuge_vma_suitable(). Signed-off-by: Yang Shi --- include/linux/huge_mm.h | 6 ++++++ mm/khugepaged.c | 5 +---- 2 files changed, 7 insertions(+), 4 deletions(-) diff --git a/include/linux/huge_mm.h b/include/linux/huge_mm.h index a8f61db47f2a..79d5919beb83 100644 --- a/include/linux/huge_mm.h +++ b/include/linux/huge_mm.h @@ -128,6 +128,12 @@ static inline bool transhuge_vma_size_ok(struct vm_are= a_struct *vma) return false; } =20 +/* + * Do the below checks: + * - For non-anon vma, check if the vm_pgoff is HPAGE_PMD_NR aligned. + * - For all vmas, check if the haddr is in an aligned HPAGE_PMD_SIZE + * area. + */ static inline bool transhuge_vma_suitable(struct vm_area_struct *vma, unsigned long addr) { diff --git a/mm/khugepaged.c b/mm/khugepaged.c index 7a5d1c1a1833..ca1754d3a827 100644 --- a/mm/khugepaged.c +++ b/mm/khugepaged.c @@ -951,7 +951,6 @@ static int hugepage_vma_revalidate(struct mm_struct *mm= , unsigned long address, struct vm_area_struct **vmap) { struct vm_area_struct *vma; - unsigned long hstart, hend; =20 if (unlikely(khugepaged_test_exit(mm))) return SCAN_ANY_PROCESS; @@ -960,9 +959,7 @@ static int hugepage_vma_revalidate(struct mm_struct *mm= , unsigned long address, if (!vma) return SCAN_VMA_NULL; =20 - hstart =3D (vma->vm_start + ~HPAGE_PMD_MASK) & HPAGE_PMD_MASK; - hend =3D vma->vm_end & HPAGE_PMD_MASK; - if (address < hstart || address + HPAGE_PMD_SIZE > hend) + if (!transhuge_vma_suitable(vma, address)) return SCAN_ADDRESS_RANGE; if (!hugepage_vma_check(vma, vma->vm_flags)) return SCAN_VMA_CHECK; --=20 2.26.3 From nobody Tue Apr 28 03:55:20 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 14D8EC43334 for ; Mon, 6 Jun 2022 21:44:48 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234902AbiFFVor (ORCPT ); Mon, 6 Jun 2022 17:44:47 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47010 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233488AbiFFVoa (ORCPT ); Mon, 6 Jun 2022 17:44:30 -0400 Received: from mail-pl1-x636.google.com (mail-pl1-x636.google.com [IPv6:2607:f8b0:4864:20::636]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 3FBC29B189 for ; Mon, 6 Jun 2022 14:44:27 -0700 (PDT) Received: by mail-pl1-x636.google.com with SMTP id u18so13200793plb.3 for ; Mon, 06 Jun 2022 14:44:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=hc3IsUtoDZHlbZz0pKu30HageYNlh4gUZJ0bt5eOoac=; b=ovR/IKgd3FrhNSybEgHRSha6sg3pcMQaSjwDNFQAK9l4/69jfwfcwuFOhND3KawDGB F9I5DSIpK/DYLjEZuokouhEM4T4Amtt3pIH0CZw0Q2a2WhUlbSlv+PAJlR4dmKxKnOSG q53HRkahqgfn267VBDNMdXolYjMgrX9UHO6tUqU4lBCoWACO9n50B3209hpS0n6CcarA nrtdfNFeIlF+1FuQbeU1mcDKx1suwgs212M6Zdl4yhgciSxs2HODGHD5msdQRIbdgBOq J2GumkWaEf52nfTEcQNI9D8h5BfHETV86Ol0dL2u1IX300bbvqB/J76X1Gr09WvSbTeA p12A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=hc3IsUtoDZHlbZz0pKu30HageYNlh4gUZJ0bt5eOoac=; b=dqrV75fFfaIp8pJp7jiDlcdp5hC8DC/KrYgPHQ7WyswpwAlAjr84w9yf+aKcpwhLZy hVFoYgVorOMbjSJ/WLYg2WNZCdkif7FbSmKo0RtZHIVzBe/IFmyULhlC6/j3INSiZf1H +cGg6tuGHqpwFjO8iJvRebFkDED8hdWczTo1SD/PSesaZgYxH2vucecSclEHLBtYVaCl VFgs4mxkNMkffxxQbVkQqnWFnvFx7Xo3a/NsJQDyFrHeFIk8OrcG9o3BmjktkPBjPAlJ nvavNePn4ya91HmfVriXIWIyNV1Fs2RZyAuCkgBetPzsgSpZHORh9Li6q5iGhxMQlFOx HhiA== X-Gm-Message-State: AOAM530f7a6shwyop2oY1BGWK97gTDfwDUhJ4XcEYz2r44QhmaSTgrcQ 5Up3tIYD1rAnUMNmjo4vIc4= X-Google-Smtp-Source: ABdhPJw1vCIJ1tEESyDDFocIeFBC1uMymoEmS7+PYZF4AcUzt+PLNG4KF514jqx5+ae8P8M3ji4Ulw== X-Received: by 2002:a17:90a:ce84:b0:1e4:d803:fad with SMTP id g4-20020a17090ace8400b001e4d8030fadmr34335734pju.99.1654551866768; Mon, 06 Jun 2022 14:44:26 -0700 (PDT) Received: from localhost.biz ([8.25.197.27]) by smtp.gmail.com with ESMTPSA id a4-20020a170903100400b0016397da033csm10881675plb.62.2022.06.06.14.44.25 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 06 Jun 2022 14:44:26 -0700 (PDT) From: Yang Shi To: vbabka@suse.cz, kirill.shutemov@linux.intel.com, willy@infradead.org, akpm@linux-foundation.org Cc: shy828301@gmail.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [v3 PATCH 5/7] mm: thp: kill transparent_hugepage_active() Date: Mon, 6 Jun 2022 14:44:12 -0700 Message-Id: <20220606214414.736109-6-shy828301@gmail.com> X-Mailer: git-send-email 2.26.3 In-Reply-To: <20220606214414.736109-1-shy828301@gmail.com> References: <20220606214414.736109-1-shy828301@gmail.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" The transparent_hugepage_active() was introduced to show THP eligibility bit in smaps in proc, smaps is the only user. But it actually does the similar check as hugepage_vma_check() which is used by khugepaged. We definitely don't have to maintain two similar checks, so kill transparent_hugepage_active(). Also move hugepage_vma_check() to huge_memory.c and huge_mm.h since it is not only for khugepaged anymore. Signed-off-by: Yang Shi --- fs/proc/task_mmu.c | 2 +- include/linux/huge_mm.h | 16 +++++++----- include/linux/khugepaged.h | 4 +-- mm/huge_memory.c | 50 ++++++++++++++++++++++++++++++++----- mm/khugepaged.c | 51 +++----------------------------------- 5 files changed, 60 insertions(+), 63 deletions(-) diff --git a/fs/proc/task_mmu.c b/fs/proc/task_mmu.c index 2dd8c8a66924..fd79566e204c 100644 --- a/fs/proc/task_mmu.c +++ b/fs/proc/task_mmu.c @@ -860,7 +860,7 @@ static int show_smap(struct seq_file *m, void *v) __show_smap(m, &mss, false); =20 seq_printf(m, "THPeligible: %d\n", - transparent_hugepage_active(vma)); + hugepage_vma_check(vma, vma->vm_flags, true)); =20 if (arch_pkeys_enabled()) seq_printf(m, "ProtectionKey: %8u\n", vma_pkey(vma)); diff --git a/include/linux/huge_mm.h b/include/linux/huge_mm.h index 79d5919beb83..f561c3e16def 100644 --- a/include/linux/huge_mm.h +++ b/include/linux/huge_mm.h @@ -209,7 +209,9 @@ static inline bool file_thp_enabled(struct vm_area_stru= ct *vma) !inode_is_open_for_write(inode) && S_ISREG(inode->i_mode); } =20 -bool transparent_hugepage_active(struct vm_area_struct *vma); +bool hugepage_vma_check(struct vm_area_struct *vma, + unsigned long vm_flags, + bool smaps); =20 #define transparent_hugepage_use_zero_page() \ (transparent_hugepage_flags & \ @@ -358,11 +360,6 @@ static inline bool __transparent_hugepage_enabled(stru= ct vm_area_struct *vma) return false; } =20 -static inline bool transparent_hugepage_active(struct vm_area_struct *vma) -{ - return false; -} - static inline bool transhuge_vma_size_ok(struct vm_area_struct *vma) { return false; @@ -380,6 +377,13 @@ static inline bool transhuge_vma_enabled(struct vm_are= a_struct *vma, return false; } =20 +static inline bool hugepage_vma_check(struct vm_area_struct *vma, + unsigned long vm_flags, + bool smaps) +{ + return false; +} + static inline void prep_transhuge_page(struct page *page) {} =20 #define transparent_hugepage_flags 0UL diff --git a/include/linux/khugepaged.h b/include/linux/khugepaged.h index 392d34c3c59a..8a6452e089ca 100644 --- a/include/linux/khugepaged.h +++ b/include/linux/khugepaged.h @@ -10,8 +10,6 @@ extern struct attribute_group khugepaged_attr_group; extern int khugepaged_init(void); extern void khugepaged_destroy(void); extern int start_stop_khugepaged(void); -extern bool hugepage_vma_check(struct vm_area_struct *vma, - unsigned long vm_flags); extern void __khugepaged_enter(struct mm_struct *mm); extern void __khugepaged_exit(struct mm_struct *mm); extern void khugepaged_enter_vma(struct vm_area_struct *vma, @@ -57,7 +55,7 @@ static inline void khugepaged_enter(struct vm_area_struct= *vma, { if (!test_bit(MMF_VM_HUGEPAGE, &vma->vm_mm->flags) && khugepaged_enabled()) { - if (hugepage_vma_check(vma, vm_flags)) + if (hugepage_vma_check(vma, vm_flags, false)) __khugepaged_enter(vma->vm_mm); } } diff --git a/mm/huge_memory.c b/mm/huge_memory.c index 36ada544e494..bc8370856e85 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -69,18 +69,56 @@ static atomic_t huge_zero_refcount; struct page *huge_zero_page __read_mostly; unsigned long huge_zero_pfn __read_mostly =3D ~0UL; =20 -bool transparent_hugepage_active(struct vm_area_struct *vma) +bool hugepage_vma_check(struct vm_area_struct *vma, + unsigned long vm_flags, + bool smaps) { + if (!transhuge_vma_enabled(vma, vm_flags)) + return false; + + if (vm_flags & VM_NO_KHUGEPAGED) + return false; + + /* Don't run khugepaged against DAX vma */ + if (vma_is_dax(vma)) + return false; + + if (vma->vm_file && !IS_ALIGNED((vma->vm_start >> PAGE_SHIFT) - + vma->vm_pgoff, HPAGE_PMD_NR)) + return false; + if (!transhuge_vma_size_ok(vma)) return false; - if (vma_is_anonymous(vma)) - return __transparent_hugepage_enabled(vma); - if (vma_is_shmem(vma)) + + /* Enabled via shmem mount options or sysfs settings. */ + if (shmem_file(vma->vm_file)) return shmem_huge_enabled(vma); - if (transhuge_vma_enabled(vma, vma->vm_flags) && file_thp_enabled(vma)) + + if (!khugepaged_enabled()) + return false; + + /* THP settings require madvise. */ + if (!(vm_flags & VM_HUGEPAGE) && !khugepaged_always()) + return false; + + /* Only regular file is valid */ + if (file_thp_enabled(vma)) return true; =20 - return false; + if (!vma_is_anonymous(vma)) + return false; + + if (vma_is_temporary_stack(vma)) + return false; + + /* + * THPeligible bit of smaps should show 1 for proper VMAs even + * though anon_vma is not initialized yet. + */ + if (!vma->anon_vma) + return smaps; + + return true; } =20 static bool get_huge_zero_page(void) diff --git a/mm/khugepaged.c b/mm/khugepaged.c index ca1754d3a827..aa0769e3b0d9 100644 --- a/mm/khugepaged.c +++ b/mm/khugepaged.c @@ -437,49 +437,6 @@ static inline int khugepaged_test_exit(struct mm_struc= t *mm) return atomic_read(&mm->mm_users) =3D=3D 0; } =20 -bool hugepage_vma_check(struct vm_area_struct *vma, - unsigned long vm_flags) -{ - if (!transhuge_vma_enabled(vma, vm_flags)) - return false; - - if (vm_flags & VM_NO_KHUGEPAGED) - return false; - - /* Don't run khugepaged against DAX vma */ - if (vma_is_dax(vma)) - return false; - - if (vma->vm_file && !IS_ALIGNED((vma->vm_start >> PAGE_SHIFT) - - vma->vm_pgoff, HPAGE_PMD_NR)) - return false; - - if (!transhuge_vma_size_ok(vma)) - return false; - - /* Enabled via shmem mount options or sysfs settings. */ - if (shmem_file(vma->vm_file)) - return shmem_huge_enabled(vma); - - if (!khugepaged_enabled()) - return false; - - /* THP settings require madvise. */ - if (!(vm_flags & VM_HUGEPAGE) && !khugepaged_always()) - return false; - - /* Only regular file is valid */ - if (file_thp_enabled(vma)) - return true; - - if (!vma->anon_vma || !vma_is_anonymous(vma)) - return false; - if (vma_is_temporary_stack(vma)) - return false; - - return true; -} - void __khugepaged_enter(struct mm_struct *mm) { struct mm_slot *mm_slot; @@ -516,7 +473,7 @@ void khugepaged_enter_vma(struct vm_area_struct *vma, { if (!test_bit(MMF_VM_HUGEPAGE, &vma->vm_mm->flags) && khugepaged_enabled()) { - if (hugepage_vma_check(vma, vm_flags)) + if (hugepage_vma_check(vma, vm_flags, false)) __khugepaged_enter(vma->vm_mm); } } @@ -961,7 +918,7 @@ static int hugepage_vma_revalidate(struct mm_struct *mm= , unsigned long address, =20 if (!transhuge_vma_suitable(vma, address)) return SCAN_ADDRESS_RANGE; - if (!hugepage_vma_check(vma, vma->vm_flags)) + if (!hugepage_vma_check(vma, vma->vm_flags, false)) return SCAN_VMA_CHECK; return 0; } @@ -1442,7 +1399,7 @@ void collapse_pte_mapped_thp(struct mm_struct *mm, un= signed long addr) * the valid THP. Add extra VM_HUGEPAGE so hugepage_vma_check() * will not fail the vma for missing VM_HUGEPAGE */ - if (!hugepage_vma_check(vma, vma->vm_flags | VM_HUGEPAGE)) + if (!hugepage_vma_check(vma, vma->vm_flags | VM_HUGEPAGE, false)) return; =20 /* Keep pmd pgtable for uffd-wp; see comment in retract_page_tables() */ @@ -2132,7 +2089,7 @@ static unsigned int khugepaged_scan_mm_slot(unsigned = int pages, progress++; break; } - if (!hugepage_vma_check(vma, vma->vm_flags)) { + if (!hugepage_vma_check(vma, vma->vm_flags, false)) { skip: progress++; continue; --=20 2.26.3 From nobody Tue Apr 28 03:55:20 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id A89BFC433EF for ; Mon, 6 Jun 2022 21:44:43 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234664AbiFFVom (ORCPT ); Mon, 6 Jun 2022 17:44:42 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46692 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233335AbiFFVoa (ORCPT ); Mon, 6 Jun 2022 17:44:30 -0400 Received: from mail-pg1-x52e.google.com (mail-pg1-x52e.google.com [IPv6:2607:f8b0:4864:20::52e]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 281A09B1BC for ; Mon, 6 Jun 2022 14:44:29 -0700 (PDT) Received: by mail-pg1-x52e.google.com with SMTP id h192so7234876pgc.4 for ; Mon, 06 Jun 2022 14:44:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=g/7Ab+lUqbHBKquM7X74tLL7I91M9MvXubqbYkGPOVk=; b=Zl67+u/eyQ1S32sx6Hq1gU3wy6Jj/gEwFrHwfDLfj5kXKbGTga32otHuWaVSHP/BTG 8FV8IFVMzeLwVIZPfohvtCHeQCORqyrMg6C5n5DKyzUkGHywc1SKe9OhZwKnjugA25Ut oQmqgR79kf6dZDTuSd/cGXe1DdZvA87Cy0Z4FGorlcY51sotwt6s2SdxKwzuhX7Wab7h xJLyf6bAEAOAic6jUhaoLF1K6RopUJ03duy3BiMOQL+oMllO4wJU88Rau/eyArAdDaG9 08JtmscsPZLvfVuxDVuX46pUVLpQILO8k86SvuapQ2t6NPh5WDVDdchZrZOa1dSVfyym q/Ew== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=g/7Ab+lUqbHBKquM7X74tLL7I91M9MvXubqbYkGPOVk=; b=JpBn7EkSG3OaAwnEjcFP3Ol0EV+/BaGZ0ZidTSEPbXkgkCB2WaGdRwitWdMUV6H9xZ CJMgLXqq3TcF8MQHN8vsbOfOZDZg3j5yvt955mP6gwjlxfp2NVrFe+PVtD8dclSoFoxi NxzFzQ0XifbDD79xCJpN059ZiWcw4uF4Ouhp8SBbsEYyyvi8iqGZ5JFgNps6IMv8pPDo JOV0hAGNDlyhtX9HJFLmkaMLpEZ7Zh+jNkQH+bxKPb3/vwOq3P56FZugEuji4Jo/waC4 gFA2ILl1kb4JiB40JUmmgJCXgvRk37+ruF4J9PyjE1ij4IqC3nVHhY+noRY6fums/jW+ U3MQ== X-Gm-Message-State: AOAM5301qnvLU9At7o03FH7b7TVGbJTQ8O6lGWxEB+KNIkktf5Ov0FDO hwZxcXqzadametmXeyaQndc= X-Google-Smtp-Source: ABdhPJx6iX/VMjSijaMzPoK9DuQWOtvamU7nFGhMGyi89Cl/lcGM4rONgFuIrgrd2gOAWSyeHWMSnw== X-Received: by 2002:a65:6d0f:0:b0:3fd:8437:c35b with SMTP id bf15-20020a656d0f000000b003fd8437c35bmr9804463pgb.24.1654551868662; Mon, 06 Jun 2022 14:44:28 -0700 (PDT) Received: from localhost.biz ([8.25.197.27]) by smtp.gmail.com with ESMTPSA id a4-20020a170903100400b0016397da033csm10881675plb.62.2022.06.06.14.44.26 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 06 Jun 2022 14:44:27 -0700 (PDT) From: Yang Shi To: vbabka@suse.cz, kirill.shutemov@linux.intel.com, willy@infradead.org, akpm@linux-foundation.org Cc: shy828301@gmail.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [v3 PATCH 6/7] mm: thp: kill __transhuge_page_enabled() Date: Mon, 6 Jun 2022 14:44:13 -0700 Message-Id: <20220606214414.736109-7-shy828301@gmail.com> X-Mailer: git-send-email 2.26.3 In-Reply-To: <20220606214414.736109-1-shy828301@gmail.com> References: <20220606214414.736109-1-shy828301@gmail.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" The page fault path checks THP eligibility with __transhuge_page_enabled() which does the similar thing as hugepage_vma_check(), so use hugepage_vma_check() instead. However page fault allows DAX and !anon_vma cases, so added a new flag, in_pf, to hugepage_vma_check() to make page fault work correctly. The in_pf flag is also used to skip shmem and file THP for page fault since shmem handles THP in its own shmem_fault() and file THP allocation on fault is not supported yet. Also remove hugepage_vma_enabled() since hugepage_vma_check() is the only caller now, it is not necessary to have a helper function. Signed-off-by: Yang Shi --- fs/proc/task_mmu.c | 2 +- include/linux/huge_mm.h | 57 ++------------------------------------ include/linux/khugepaged.h | 2 +- mm/huge_memory.c | 25 ++++++++++++----- mm/khugepaged.c | 8 +++--- mm/memory.c | 7 +++-- 6 files changed, 31 insertions(+), 70 deletions(-) diff --git a/fs/proc/task_mmu.c b/fs/proc/task_mmu.c index fd79566e204c..a0850303baec 100644 --- a/fs/proc/task_mmu.c +++ b/fs/proc/task_mmu.c @@ -860,7 +860,7 @@ static int show_smap(struct seq_file *m, void *v) __show_smap(m, &mss, false); =20 seq_printf(m, "THPeligible: %d\n", - hugepage_vma_check(vma, vma->vm_flags, true)); + hugepage_vma_check(vma, vma->vm_flags, true, false)); =20 if (arch_pkeys_enabled()) seq_printf(m, "ProtectionKey: %8u\n", vma_pkey(vma)); diff --git a/include/linux/huge_mm.h b/include/linux/huge_mm.h index f561c3e16def..d478e8875023 100644 --- a/include/linux/huge_mm.h +++ b/include/linux/huge_mm.h @@ -153,48 +153,6 @@ static inline bool transhuge_vma_suitable(struct vm_ar= ea_struct *vma, return true; } =20 -static inline bool transhuge_vma_enabled(struct vm_area_struct *vma, - unsigned long vm_flags) -{ - /* Explicitly disabled through madvise. */ - if ((vm_flags & VM_NOHUGEPAGE) || - test_bit(MMF_DISABLE_THP, &vma->vm_mm->flags)) - return false; - return true; -} - -/* - * to be used on vmas which are known to support THP. - * Use transparent_hugepage_active otherwise - */ -static inline bool __transparent_hugepage_enabled(struct vm_area_struct *v= ma) -{ - - /* - * If the hardware/firmware marked hugepage support disabled. - */ - if (transparent_hugepage_flags & (1 << TRANSPARENT_HUGEPAGE_NEVER_DAX)) - return false; - - if (!transhuge_vma_enabled(vma, vma->vm_flags)) - return false; - - if (vma_is_temporary_stack(vma)) - return false; - - if (transparent_hugepage_flags & (1 << TRANSPARENT_HUGEPAGE_FLAG)) - return true; - - if (vma_is_dax(vma)) - return true; - - if (transparent_hugepage_flags & - (1 << TRANSPARENT_HUGEPAGE_REQ_MADV_FLAG)) - return !!(vma->vm_flags & VM_HUGEPAGE); - - return false; -} - static inline bool file_thp_enabled(struct vm_area_struct *vma) { struct inode *inode; @@ -211,7 +169,7 @@ static inline bool file_thp_enabled(struct vm_area_stru= ct *vma) =20 bool hugepage_vma_check(struct vm_area_struct *vma, unsigned long vm_flags, - bool smaps); + bool smaps, bool in_pf); =20 #define transparent_hugepage_use_zero_page() \ (transparent_hugepage_flags & \ @@ -355,11 +313,6 @@ static inline bool folio_test_pmd_mappable(struct foli= o *folio) return false; } =20 -static inline bool __transparent_hugepage_enabled(struct vm_area_struct *v= ma) -{ - return false; -} - static inline bool transhuge_vma_size_ok(struct vm_area_struct *vma) { return false; @@ -371,15 +324,9 @@ static inline bool transhuge_vma_suitable(struct vm_ar= ea_struct *vma, return false; } =20 -static inline bool transhuge_vma_enabled(struct vm_area_struct *vma, - unsigned long vm_flags) -{ - return false; -} - static inline bool hugepage_vma_check(struct vm_area_struct *vma, unsigned long vm_flags, - bool smaps) + bool smaps, bool in_pf) { return false; } diff --git a/include/linux/khugepaged.h b/include/linux/khugepaged.h index 8a6452e089ca..e047be601268 100644 --- a/include/linux/khugepaged.h +++ b/include/linux/khugepaged.h @@ -55,7 +55,7 @@ static inline void khugepaged_enter(struct vm_area_struct= *vma, { if (!test_bit(MMF_VM_HUGEPAGE, &vma->vm_mm->flags) && khugepaged_enabled()) { - if (hugepage_vma_check(vma, vm_flags, false)) + if (hugepage_vma_check(vma, vm_flags, false, false)) __khugepaged_enter(vma->vm_mm); } } diff --git a/mm/huge_memory.c b/mm/huge_memory.c index bc8370856e85..b95786ada466 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -71,17 +71,25 @@ unsigned long huge_zero_pfn __read_mostly =3D ~0UL; =20 bool hugepage_vma_check(struct vm_area_struct *vma, unsigned long vm_flags, - bool smaps) + bool smaps, bool in_pf) { - if (!transhuge_vma_enabled(vma, vm_flags)) + /* Explicitly disabled through madvise or prctl. */ + if ((vm_flags & VM_NOHUGEPAGE) || + test_bit(MMF_DISABLE_THP, &vma->vm_mm->flags)) + return false; + /* + * If the hardware/firmware marked hugepage support disabled. + */ + if (transparent_hugepage_flags & (1 << TRANSPARENT_HUGEPAGE_NEVER_DAX)) return false; =20 + /* Special VMA and hugetlb VMA */ if (vm_flags & VM_NO_KHUGEPAGED) return false; =20 - /* Don't run khugepaged against DAX vma */ + /* khugepaged doesn't collapse DAX vma, but page fault is fine. */ if (vma_is_dax(vma)) - return false; + return in_pf; =20 if (vma->vm_file && !IS_ALIGNED((vma->vm_start >> PAGE_SHIFT) - vma->vm_pgoff, HPAGE_PMD_NR)) @@ -91,7 +99,7 @@ bool hugepage_vma_check(struct vm_area_struct *vma, return false; =20 /* Enabled via shmem mount options or sysfs settings. */ - if (shmem_file(vma->vm_file)) + if (!in_pf && shmem_file(vma->vm_file)) return shmem_huge_enabled(vma); =20 if (!khugepaged_enabled()) @@ -102,7 +110,7 @@ bool hugepage_vma_check(struct vm_area_struct *vma, return false; =20 /* Only regular file is valid */ - if (file_thp_enabled(vma)) + if (!in_pf && file_thp_enabled(vma)) return true; =20 if (!vma_is_anonymous(vma)) @@ -114,9 +122,12 @@ bool hugepage_vma_check(struct vm_area_struct *vma, /* * THPeligible bit of smaps should show 1 for proper VMAs even * though anon_vma is not initialized yet. + * + * Allow page fault since anon_vma may be not initialized until + * the first page fault. */ if (!vma->anon_vma) - return smaps; + return (smaps || in_pf); =20 return true; } diff --git a/mm/khugepaged.c b/mm/khugepaged.c index aa0769e3b0d9..ab6183c5489f 100644 --- a/mm/khugepaged.c +++ b/mm/khugepaged.c @@ -473,7 +473,7 @@ void khugepaged_enter_vma(struct vm_area_struct *vma, { if (!test_bit(MMF_VM_HUGEPAGE, &vma->vm_mm->flags) && khugepaged_enabled()) { - if (hugepage_vma_check(vma, vm_flags, false)) + if (hugepage_vma_check(vma, vm_flags, false, false)) __khugepaged_enter(vma->vm_mm); } } @@ -918,7 +918,7 @@ static int hugepage_vma_revalidate(struct mm_struct *mm= , unsigned long address, =20 if (!transhuge_vma_suitable(vma, address)) return SCAN_ADDRESS_RANGE; - if (!hugepage_vma_check(vma, vma->vm_flags, false)) + if (!hugepage_vma_check(vma, vma->vm_flags, false, false)) return SCAN_VMA_CHECK; return 0; } @@ -1399,7 +1399,7 @@ void collapse_pte_mapped_thp(struct mm_struct *mm, un= signed long addr) * the valid THP. Add extra VM_HUGEPAGE so hugepage_vma_check() * will not fail the vma for missing VM_HUGEPAGE */ - if (!hugepage_vma_check(vma, vma->vm_flags | VM_HUGEPAGE, false)) + if (!hugepage_vma_check(vma, vma->vm_flags | VM_HUGEPAGE, false, false)) return; =20 /* Keep pmd pgtable for uffd-wp; see comment in retract_page_tables() */ @@ -2089,7 +2089,7 @@ static unsigned int khugepaged_scan_mm_slot(unsigned = int pages, progress++; break; } - if (!hugepage_vma_check(vma, vma->vm_flags, false)) { + if (!hugepage_vma_check(vma, vma->vm_flags, false, false)) { skip: progress++; continue; diff --git a/mm/memory.c b/mm/memory.c index bc5d40eec5d5..673f7561a30a 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -4962,6 +4962,7 @@ static vm_fault_t __handle_mm_fault(struct vm_area_st= ruct *vma, .gfp_mask =3D __get_fault_gfp_mask(vma), }; struct mm_struct *mm =3D vma->vm_mm; + unsigned long vm_flags =3D vma->vm_flags; pgd_t *pgd; p4d_t *p4d; vm_fault_t ret; @@ -4975,7 +4976,8 @@ static vm_fault_t __handle_mm_fault(struct vm_area_st= ruct *vma, if (!vmf.pud) return VM_FAULT_OOM; retry_pud: - if (pud_none(*vmf.pud) && __transparent_hugepage_enabled(vma)) { + if (pud_none(*vmf.pud) && + hugepage_vma_check(vma, vm_flags, false, true)) { ret =3D create_huge_pud(&vmf); if (!(ret & VM_FAULT_FALLBACK)) return ret; @@ -5008,7 +5010,8 @@ static vm_fault_t __handle_mm_fault(struct vm_area_st= ruct *vma, if (pud_trans_unstable(vmf.pud)) goto retry_pud; =20 - if (pmd_none(*vmf.pmd) && __transparent_hugepage_enabled(vma)) { + if (pmd_none(*vmf.pmd) && + hugepage_vma_check(vma, vm_flags, false, true)) { ret =3D create_huge_pmd(&vmf); if (!(ret & VM_FAULT_FALLBACK)) return ret; --=20 2.26.3 From nobody Tue Apr 28 03:55:20 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9E452C433EF for ; Mon, 6 Jun 2022 21:45:06 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235200AbiFFVo6 (ORCPT ); Mon, 6 Jun 2022 17:44:58 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47124 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231772AbiFFVob (ORCPT ); Mon, 6 Jun 2022 17:44:31 -0400 Received: from mail-pl1-x636.google.com (mail-pl1-x636.google.com [IPv6:2607:f8b0:4864:20::636]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 801E57A817 for ; Mon, 6 Jun 2022 14:44:30 -0700 (PDT) Received: by mail-pl1-x636.google.com with SMTP id u18so13200793plb.3 for ; Mon, 06 Jun 2022 14:44:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=yPXZCZEMUQaqYwKbEzwy4HUgqvcP6p/dfhKRtiPq7Hw=; b=JhXbAthDaDChPe0CIE4lyCWV2sCMXUskpdnFfr0MSsPjrEj6BkRL0mh0lH3+QrxtSK EenH1GV5UxsPW8wRBeB676I26nUwYNyy94grGsy5diUQYVvPnd44oNlg7fZdm7x4uOG6 GIb46aOwj5teb0/tSJ0XYsOqw+loQfIs6+8RldDC9kP/ogs6bRtwqDrtRVN3G2f8jzIL oJbi95NWG0ZA1uTgB5GJxIstWMhDTTzkDHR1Mi7qJ9MVQGLULQyLixJbiyhWH630r56+ zV+o2dt6Pr/AV5Sh+k3eIMVFA4SjiiZros8bZtNrYeNBlcZ/VwZEx/OIXCs/JU61QD7p FzKw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=yPXZCZEMUQaqYwKbEzwy4HUgqvcP6p/dfhKRtiPq7Hw=; b=7iSNjM/uR+bCV+6Cyv08gQPZZShRDMeBFiWHVrrrhZkWDgv4CaN1AgoC9XLV6QpHaX OiV65rK4Jp+gyGGuVSH/6FYYgrRHvaqHnrA4vrIKhBJuxOqXAvF6pjTMFstlB0pJjSKx GNmYvZTt2xAz4mPs7ZAA0HdR5OPhD9zyRNyiw8w28HRB2A+Kavc1Afc7JBvNg2xWII4q HJyCa1Lo5+vPN6yOJO6xdlEaBQsJbUiIep5nDgMQaIZiDtwIEXCvNh25W83N/1UQoAaz QlwE6EClg48lT4YtBqrnSwrWrptJz7u/cfC8aNIwWj6M0bwploAJoxd3LGDSAaS30x8f 7Qww== X-Gm-Message-State: AOAM532xs1nIWGzIGOSf9dw4WjdlWufhEiWPqNp+sZ1XfaX82Rk6B8gc t39PM4K2CUN5UeJcPkNu+eA= X-Google-Smtp-Source: ABdhPJyDZQtvOLvDSpVY2YEfud++1wRn/OaNDsUtVpZmzEHtUgNOTteY+F+x1j6onYQEtMIS01ebLQ== X-Received: by 2002:a17:902:c2c1:b0:161:c6af:278e with SMTP id c1-20020a170902c2c100b00161c6af278emr26163480pla.151.1654551870055; Mon, 06 Jun 2022 14:44:30 -0700 (PDT) Received: from localhost.biz ([8.25.197.27]) by smtp.gmail.com with ESMTPSA id a4-20020a170903100400b0016397da033csm10881675plb.62.2022.06.06.14.44.28 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 06 Jun 2022 14:44:29 -0700 (PDT) From: Yang Shi To: vbabka@suse.cz, kirill.shutemov@linux.intel.com, willy@infradead.org, akpm@linux-foundation.org Cc: shy828301@gmail.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [v3 PATCH 7/7] mm: khugepaged: reorg some khugepaged helpers Date: Mon, 6 Jun 2022 14:44:14 -0700 Message-Id: <20220606214414.736109-8-shy828301@gmail.com> X-Mailer: git-send-email 2.26.3 In-Reply-To: <20220606214414.736109-1-shy828301@gmail.com> References: <20220606214414.736109-1-shy828301@gmail.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" The khugepaged_{enabled|always|req_madv} are not khugepaged only anymore, move them to huge_mm.h and rename to hugepage_flags_xxx, and remove khugepaged_req_madv due to no users. Also move khugepaged_defrag to khugepaged.c since its only caller is in that file, it doesn't have to be in a header file. Signed-off-by: Yang Shi --- include/linux/huge_mm.h | 8 ++++++++ include/linux/khugepaged.h | 17 +---------------- mm/huge_memory.c | 4 ++-- mm/khugepaged.c | 18 +++++++++++------- 4 files changed, 22 insertions(+), 25 deletions(-) diff --git a/include/linux/huge_mm.h b/include/linux/huge_mm.h index d478e8875023..ce2d05ee4816 100644 --- a/include/linux/huge_mm.h +++ b/include/linux/huge_mm.h @@ -116,6 +116,14 @@ extern struct kobj_attribute shmem_enabled_attr; =20 extern unsigned long transparent_hugepage_flags; =20 +#define hugepage_flags_enabled() \ + (transparent_hugepage_flags & \ + ((1<flags)) @@ -53,8 +39,7 @@ static inline void khugepaged_exit(struct mm_struct *mm) static inline void khugepaged_enter(struct vm_area_struct *vma, unsigned long vm_flags) { - if (!test_bit(MMF_VM_HUGEPAGE, &vma->vm_mm->flags) && - khugepaged_enabled()) { + if (!test_bit(MMF_VM_HUGEPAGE, &vma->vm_mm->flags)) { if (hugepage_vma_check(vma, vm_flags, false, false)) __khugepaged_enter(vma->vm_mm); } diff --git a/mm/huge_memory.c b/mm/huge_memory.c index b95786ada466..866b98a39496 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -102,11 +102,11 @@ bool hugepage_vma_check(struct vm_area_struct *vma, if (!in_pf && shmem_file(vma->vm_file)) return shmem_huge_enabled(vma); =20 - if (!khugepaged_enabled()) + if (!hugepage_flags_enabled()) return false; =20 /* THP settings require madvise. */ - if (!(vm_flags & VM_HUGEPAGE) && !khugepaged_always()) + if (!(vm_flags & VM_HUGEPAGE) && !hugepage_flags_always()) return false; =20 /* Only regular file is valid */ diff --git a/mm/khugepaged.c b/mm/khugepaged.c index ab6183c5489f..2523c085625a 100644 --- a/mm/khugepaged.c +++ b/mm/khugepaged.c @@ -472,7 +472,7 @@ void khugepaged_enter_vma(struct vm_area_struct *vma, unsigned long vm_flags) { if (!test_bit(MMF_VM_HUGEPAGE, &vma->vm_mm->flags) && - khugepaged_enabled()) { + hugepage_flags_enabled()) { if (hugepage_vma_check(vma, vm_flags, false, false)) __khugepaged_enter(vma->vm_mm); } @@ -763,6 +763,10 @@ static bool khugepaged_scan_abort(int nid) return false; } =20 +#define khugepaged_defrag() \ + (transparent_hugepage_flags & \ + (1<