From nobody Sat Nov 21 06:18:45 2026 Received: from mail-qv1-f47.google.com (mail-qv1-f47.google.com [209.85.219.47]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id DE5165505C2 for ; Wed, 14 Oct 2026 18:30:29 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.219.47 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1792002631; cv=none; b=QDhMNIB9R3oLC1J6gT7O4obvMRmiQJsRr4b/dmemDRPb4rHeD2wDTwnxP7tf750fGrvFyvQgA8TIiQs97JAku0z+npAxkxE8MK42iz8niHJ1XxrIPfWVHQLUGthPGSUqV9pfUHeq+yXPmF1S/X3LX1syDTsyW1NCJqOmagkBFSI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1792002631; c=relaxed/simple; bh=C7s/WFH5xwOxXEWLhEG2bd5kCY0XbwtAqGvqLqczzYU=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=BbbMl3u4YEPeQ4OltufE96A2lESkQF/ixlMnfUF6KB0jcNVnNFAWX3HpTEr1O/4UbTKWgIUN1lJGRoEHhNSGHBCBvv3OMA+Mm0W03LCB5p/X6ff1gVTlhEblj3o1zEdzRzeKVY3PahTmg0DPCMlN5CNmpQKy9LTVKGZf5gaR4U8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=gourry.net; spf=pass smtp.mailfrom=gourry.net; dkim=pass (2048-bit key) header.d=gourry.net header.i=@gourry.net header.b=O1S1tUg2; arc=none smtp.client-ip=209.85.219.47 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=gourry.net Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gourry.net Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gourry.net header.i=@gourry.net header.b="O1S1tUg2" Received: by mail-qv1-f47.google.com with SMTP id 6a1803df08f44-917a04338c1so47340036d6.0 for ; Wed, 14 Oct 2026 11:30:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gourry.net; s=google; t=1792002629; x=1792607429; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=4zyKedNJpmpuM86GYUmjidd6QzKLk815HOP8Xrauk7M=; b=O1S1tUg2ZKsxqXNNihqTvl/hB1B3lkTY1zb8xzIEtcP5eNapwiOXotygrBXr1nc5Lh pBe6T7BEhN10MyxHe+aGZd3fo+PcOj2zGlT962tsqQo+H12oJ3fhE1JW+N0ElrZOCvRf i+o9SUDt6fJfJm3hg4BcFx0rgvNFEwzaGlsxgxLH4s0NAQnJHkOc3XV+sETczC6J1d24 hSfg65yxZe/XX99uVd62zyJPbk8MPEX57HDHDmnEK5txYRqp0QVJa8hqAdJEL5m+5+NK 8OeyEFRLQ9OUT0SfTdeaI6GdA6yXqX0g06At7JViLH+BZFZkHPCyPYG1hvqsF8al+eme UJrQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20260707; t=1792002629; x=1792607429; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=4zyKedNJpmpuM86GYUmjidd6QzKLk815HOP8Xrauk7M=; b=YPprdhsSMsbeTr0VznDTqEgPw+JFFYiJiobxFw7dAjWrZyzEyZ3RLGzoomI3miZhWW 5gVsJiX4VMZVUlCcDPISGxKjO06OH07816R9L+V9N6ewmiRFTvvqOf2M62VBzf0x3apA IsHdHS1rtuk2qwjYr6QNKlzZpSJp9BWlzQT+5bR5tqI6V6RO0p20aBUJoNq1J8b4EAXk XmNP6WBZtWrJytGEDIResvV73MdW8/lBM63ye07e2/Hkw0tehzqJQzW8FQ+7XRr69TRN RU7I0iSAqKAbYJeIo6wA9zGvxLePB1S1ZGw+PClln5wpsgD8VzK+imOqtB8eLly7xyEF ruEg== X-Gm-Message-State: AFq9FYJ/scJNPRJ1ftMIRMJq4YsxkIsVM7JMM4ZpkpvjKtFk5xWEadRM EMxfRhkt/sbHvvJmMH5Byup0QTMpn2rZ5hPH7VQ/MUSN/H/RO1UdhCZd/OBEnKGFekA= X-Gm-Gg: AYBFou3koM6DHON9UEfiC7O6vrmUYZnNaUiFIQ6I3IilBXuYOZMveNm7gOo6iVBCzjs iI6lc+pODsXLCefuqqffA+0XK+q8h7T3ughyUOp9jd6RO8m05RCIl5it2NEFv0tgDbmy/jOlyLa SuNuAdGjDnEaPBOkCAnsfurC4lstp6VTmzm8xS0+1prbrG2A8c/hEn6jXZ/vVxyD9eqLuRpIU1m ZWtxlH2I9fYgNLkKweyL2GR+OTRKYzaRkaq61fNeX2O0s5M10G/3jTX9UySlpj9a74sF5qrCxTK b1+gs1aRIvpcLephpBHQsWyxypMKyYcMCpAaBfP0mboO/4Uoza5OU/mLaaFswdwSk202GSkKeMI 4T/QpfYkZ+qBLwymOGkKBgHJYrTZN1WLADNilko7YOXaYScBmzkbx4x25LhyQsUj8x3FJe9B3A2 vcd5OHcrwn9vHcOQs63c9kHH2H9EeWPYRlHvNzohlPcY9E2OqTwO97nyIYnhrce2OHaF0NaWdtR 2wwTEpWr43mxT8svfUHhIPlBprNdCVpWFFJmVvagv3Xc2Jbv1Ypl+GB/CKt X-Received: by 2002:a05:620a:6489:b0:93e:9171:b82 with SMTP id af79cd13be357-93ebd208959mr2566072385a.36.1792002628519; Wed, 14 Oct 2026 11:30:28 -0700 (PDT) Received: from gourry-fedora-PF4VCD3F.lan (pool-173-79-60-52.washdc.fios.verizon.net. [173.79.60.52]) by smtp.gmail.com with ESMTPSA id af79cd13be357-93ef508811esm556593085a.36.2026.10.14.11.30.27 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 14 Oct 2026 11:30:28 -0700 (PDT) From: "Gregory Price (Meta)" To: linux-mm@kvack.org Cc: linux-kernel@vger.kernel.org, kernel-team@meta.com, akpm@linux-foundation.org, liam@infradead.org, ljs@kernel.org, david@kernel.org, vbabka@kernel.org, jannh@google.com Subject: [PATCH v2 01/10] mm/madvise: name the shared LRU PMD callback Date: Wed, 14 Oct 2026 14:30:15 -0400 Message-ID: <20261014183024.2075709-2-gourry@gourry.net> X-Mailer: git-send-email 2.56.0 In-Reply-To: <20261014183024.2075709-1-gourry@gourry.net> References: <20261014183024.2075709-1-gourry@gourry.net> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" The MADV_COLD and MADV_PAGEOUT page-walk callback is named after its PTE implementation even though it is registered as a PMD callback and serves both LRU operations. Rename it to madvise_lru_pmd_entry() before separating the PMD and PTE paths. No functional change intended. Reviewed-by: Lorenzo Stoakes (ARM) Signed-off-by: Gregory Price (Meta) --- mm/madvise.c | 7 +++---- 1 file changed, 3 insertions(+), 4 deletions(-) diff --git a/mm/madvise.c b/mm/madvise.c index 75b409e97dbf..f15cac3faf61 100644 --- a/mm/madvise.c +++ b/mm/madvise.c @@ -360,9 +360,8 @@ static inline int madvise_folio_pte_batch(unsigned long= addr, unsigned long end, FPB_MERGE_YOUNG_DIRTY); } =20 -static int madvise_cold_or_pageout_pte_range(pmd_t *pmd, - unsigned long addr, unsigned long end, - struct mm_walk *walk) +static int madvise_lru_pmd_entry(pmd_t *pmd, unsigned long addr, + unsigned long end, struct mm_walk *walk) { struct madvise_walk_private *private =3D walk->private; struct mmu_gather *tlb =3D private->tlb; @@ -578,7 +577,7 @@ static int madvise_cold_or_pageout_pte_range(pmd_t *pmd, } =20 static const struct mm_walk_ops cold_walk_ops =3D { - .pmd_entry =3D madvise_cold_or_pageout_pte_range, + .pmd_entry =3D madvise_lru_pmd_entry, .walk_lock =3D PGWALK_RDLOCK, }; =20 --=20 2.56.0 From nobody Sat Nov 21 06:18:45 2026 Received: from mail-qk1-f175.google.com (mail-qk1-f175.google.com [209.85.222.175]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 32298569F26 for ; Wed, 14 Oct 2026 18:30:33 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.222.175 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1792002634; cv=none; b=OFczyGVBTC4xTNYnTLRqulu+4rXrm4tjEurhS6Smt+Ys9Dw+gwkOgM6BNTTDSMw2jKnVCMS7WhVZ6TDVCRlOCCJa9AnKkGBmbBCsNBAkV8Buyt2Hha70pVpxdoN2kagDVAhNx6e+kADlQG+OWbVRnFg/vMFn+omrY6OEWlHIwVg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1792002634; c=relaxed/simple; bh=Of5GzeDaT49m27wll1J732O9ywIaw/o8HdwLxuaUiF8=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=lb8fb8mGy1pema5iq/GFgZVeKTfCNfISShU6mvQjm2adiwt6vb55LV0SXY7BuQxZSBf8HIaXsq5QwrBPDn6ReDO3PsDaHmNG94pf2oWAM52jPAwcdfxRs8NA75T1aOFHrbTG0eywi7baLj0nO6lc2OO1Vd/5zYa7twyFyMLU3YY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=gourry.net; spf=pass smtp.mailfrom=gourry.net; dkim=pass (2048-bit key) header.d=gourry.net header.i=@gourry.net header.b=nANANuuV; arc=none smtp.client-ip=209.85.222.175 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=gourry.net Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gourry.net Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gourry.net header.i=@gourry.net header.b="nANANuuV" Received: by mail-qk1-f175.google.com with SMTP id af79cd13be357-93e6eb77f0bso60848485a.3 for ; Wed, 14 Oct 2026 11:30:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gourry.net; s=google; t=1792002632; x=1792607432; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=/z7efmS/WkGms6FvbaOcZAe3VwQpzr0tTSUWGVsu8Kk=; b=nANANuuVIxot6ySeClDPAeIvIGfQdy3gAaBkOqbIfpSO1eXtEKZf4BVxIka+v0BTkd 2rNzNdSfFafCNeSr/S2PytwgH+luVo1lpFO7ga66C9a4hgM42G1Ji+CkYzgbWalD02DJ /Z3p4lhCcOPn3prDzlTY6BIcOYSA74Evm+y+IkgAkXD4n9e7/EpQkW3Lwk84OXJarsZe WEcP5j8/+r/j4AqdOmLeqz5d9mnBjl8QnNltNs5WL2I7rmkNb8TuNDs88Eix8K//6fNY S2egqPUSWN0TqP6eHM4vCidSIEEAjpvv2duXTKHUY4FFfnVRcjq8LS0ZNdwAdHU8tPIe bAqg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20260707; t=1792002632; x=1792607432; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=/z7efmS/WkGms6FvbaOcZAe3VwQpzr0tTSUWGVsu8Kk=; b=tkUMW9B1aWX2vvyQGzZUYLC0FSMQV50QvtfVVlLXDR5okLMTa0padKZBLj/OonVXWE K7zSqh/EKogCSdzrM+YUFOZAzA3Q74gJMB7dYONH9AiBPsnRmmH0IvM8jvteuYvS44ND Syb2VIc3FXc4bbkM9rKLIC2CXL6rJyJl9XEWIjA1VzKYXwEdMHIDM0kmEIkd+aZ39fQX 1aQ87ar48pL+4lOqfpi6+IGyiCOmXgaJIK31NEuRX4Jyb+SySdbsu4aA3Lx3/WaJg+5v IBWoyXQ0BMr0AVTGKV1qS9giFcfiIj79FtdyU5RX3Z5Vi6RF9+ploPyLkwdwkQX+9Ruw fFIg== X-Gm-Message-State: AFq9FYJcX1/qg2rPFxuyejSJODpCa8prOmpZCh8VP7sgfZyECW2dUsY0 UVLtlGoCDPzjTGlj8NNeIpawxNAfkaVZPeIDK/4OwIhRzf3f/IiPJIHrD8t1LUzFn9M= X-Gm-Gg: AYBFou2aDnhJ0rqoYqo5xf+4oP8ENHHvkfBVkLekm9j9U+ZvaLVQWLAJNRrqOB6/7CU DJAx745Q9qzihPuXCGtDFUm/aX4cprcICyC/6/aWbo3nhWdeQDp0PMsx/SXcSU7osYPdNsx14GH bob9+sgrohm8PZC8dXfb+3JalpHpXU16tfJN+ogJVDUpsr+t8I91/Is8L7ePpX7PVuBO2byDdns H8PFUfMS0hZCSj2cqmx4Ejj+scmnMHqamC2edTM9NnoDgltjVd+0XJNC+aYLU/4lBzCnYK+i1Qo SLMNi241OgnZrNswbyvwNXhLj+9UkyjPS/gn2XrdI7x33NBgAnoy31NesTO3PdeUBOB1ILPyu3I OoHFSXi4xf0GezqaFDIKimxOFTBVbFvbeohPK6AMdK8J68v4qLcgliifccs3mWEvGD4cTapC5Z0 XhKB/mVBqRiPDgQjXmAudtMLiFSBqHcWqUwANl9I/Lkdj18kw76uXD5Zl+vHJFlQFldjYsbu+c7 KRFXWBm7GQFx28cM52wb5vjJQSyyxf9ByFRUVTr5Rx876+aJe+b80iUx+2E X-Received: by 2002:a05:620a:890d:b0:939:6c9a:8f8d with SMTP id af79cd13be357-93ebd1b7bf1mr2679860385a.28.1792002629963; Wed, 14 Oct 2026 11:30:29 -0700 (PDT) Received: from gourry-fedora-PF4VCD3F.lan (pool-173-79-60-52.washdc.fios.verizon.net. [173.79.60.52]) by smtp.gmail.com with ESMTPSA id af79cd13be357-93ef508811esm556593085a.36.2026.10.14.11.30.29 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 14 Oct 2026 11:30:29 -0700 (PDT) From: "Gregory Price (Meta)" To: linux-mm@kvack.org Cc: linux-kernel@vger.kernel.org, kernel-team@meta.com, akpm@linux-foundation.org, liam@infradead.org, ljs@kernel.org, david@kernel.org, vbabka@kernel.org, jannh@google.com Subject: [PATCH v2 02/10] mm/madvise: factor shared LRU folio handling Date: Wed, 14 Oct 2026 14:30:16 -0400 Message-ID: <20261014183024.2075709-3-gourry@gourry.net> X-Mailer: git-send-email 2.56.0 In-Reply-To: <20261014183024.2075709-1-gourry@gourry.net> References: <20261014183024.2075709-1-gourry@gourry.net> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" The huge-PMD and PTE paths duplicate folio filtering, reference clearing, deactivation and pageout isolation. Keeping both copies synchronized obscures the page-table-specific control flow. Factor the common filtering and LRU operation into helpers. Keep the lock-before-reference sequence visible at each split site because an early reference can prevent split_folio() from succeeding. Also rename pageout_anon_only_filter to pageout_anon_only to match the helper's parameter, and drop redundant parentheses from the file pageout check in madvise_pageout(). No functional change intended. Reviewed-by: Lorenzo Stoakes (ARM) Signed-off-by: Gregory Price (Meta) --- mm/madvise.c | 94 ++++++++++++++++++++++++++-------------------------- 1 file changed, 47 insertions(+), 47 deletions(-) diff --git a/mm/madvise.c b/mm/madvise.c index f15cac3faf61..b0ce4ececba0 100644 --- a/mm/madvise.c +++ b/mm/madvise.c @@ -360,6 +360,43 @@ static inline int madvise_folio_pte_batch(unsigned lon= g addr, unsigned long end, FPB_MERGE_YOUNG_DIRTY); } =20 +static void madvise_lru_folio(struct folio *folio, bool pageout, + struct list_head *folio_list) +{ + /* + * Reclaim keeps a recently referenced folio, so clear its references + * before deactivating or reclaiming it. As a side effect, idle page + * tracking misses those accesses. + */ + folio_clear_referenced(folio); + folio_test_clear_young(folio); + if (folio_test_active(folio)) + folio_set_workingset(folio); + + if (!pageout) { + folio_deactivate(folio); + return; + } + + if (!folio_isolate_lru(folio)) + return; + if (folio_test_unevictable(folio)) + folio_putback_lru(folio); + else + list_add(&folio->lru, folio_list); +} + +/* + * Do not interfere with other mappings of the folio, and skip file folios + * when only anonymous folios may be paged out. + */ +static bool madvise_lru_folio_is_filtered(struct folio *folio, + bool pageout_anon_only) +{ + return folio_maybe_mapped_shared(folio) || + (pageout_anon_only && !folio_test_anon(folio)); +} + static int madvise_lru_pmd_entry(pmd_t *pmd, unsigned long addr, unsigned long end, struct mm_walk *walk) { @@ -372,15 +409,14 @@ static int madvise_lru_pmd_entry(pmd_t *pmd, unsigned= long addr, spinlock_t *ptl; struct folio *folio =3D NULL; LIST_HEAD(folio_list); - bool pageout_anon_only_filter; unsigned int batch_count =3D 0; + bool pageout_anon_only; int nr; =20 if (fatal_signal_pending(current)) return -EINTR; - - pageout_anon_only_filter =3D pageout && !vma_is_anonymous(vma) && - !can_do_file_pageout(vma); + pageout_anon_only =3D pageout && !vma_is_anonymous(vma) && + !can_do_file_pageout(vma); =20 #ifdef CONFIG_TRANSPARENT_HUGEPAGE if (pmd_trans_huge(*pmd)) { @@ -406,11 +442,7 @@ static int madvise_lru_pmd_entry(pmd_t *pmd, unsigned = long addr, if (folio_is_zone_device(folio)) goto huge_unlock; =20 - /* Do not interfere with other mappings of this folio */ - if (folio_maybe_mapped_shared(folio)) - goto huge_unlock; - - if (pageout_anon_only_filter && !folio_test_anon(folio)) + if (madvise_lru_folio_is_filtered(folio, pageout_anon_only)) goto huge_unlock; =20 if (next - addr !=3D HPAGE_PMD_SIZE) { @@ -433,19 +465,7 @@ static int madvise_lru_pmd_entry(pmd_t *pmd, unsigned = long addr, tlb_remove_pmd_tlb_entry(tlb, pmd, addr); } =20 - folio_clear_referenced(folio); - folio_test_clear_young(folio); - if (folio_test_active(folio)) - folio_set_workingset(folio); - if (pageout) { - if (folio_isolate_lru(folio)) { - if (folio_test_unevictable(folio)) - folio_putback_lru(folio); - else - list_add(&folio->lru, &folio_list); - } - } else - folio_deactivate(folio); + madvise_lru_folio(folio, pageout, &folio_list); huge_unlock: spin_unlock(ptl); if (pageout) @@ -499,9 +519,7 @@ static int madvise_lru_pmd_entry(pmd_t *pmd, unsigned l= ong addr, if (nr < folio_nr_pages(folio)) { int err; =20 - if (folio_maybe_mapped_shared(folio)) - continue; - if (pageout_anon_only_filter && !folio_test_anon(folio)) + if (madvise_lru_folio_is_filtered(folio, pageout_anon_only)) continue; if (!folio_trylock(folio)) continue; @@ -534,7 +552,7 @@ static int madvise_lru_pmd_entry(pmd_t *pmd, unsigned l= ong addr, folio_mapcount(folio) !=3D folio_nr_pages(folio)) continue; =20 - if (pageout_anon_only_filter && !folio_test_anon(folio)) + if (pageout_anon_only && !folio_test_anon(folio)) continue; =20 if (!pageout && pte_young(ptent)) { @@ -543,25 +561,7 @@ static int madvise_lru_pmd_entry(pmd_t *pmd, unsigned = long addr, tlb_remove_tlb_entries(tlb, pte, nr, addr); } =20 - /* - * We are deactivating a folio for accelerating reclaiming. - * VM couldn't reclaim the folio unless we clear PG_young. - * As a side effect, it makes confuse idle-page tracking - * because they will miss recent referenced history. - */ - folio_clear_referenced(folio); - folio_test_clear_young(folio); - if (folio_test_active(folio)) - folio_set_workingset(folio); - if (pageout) { - if (folio_isolate_lru(folio)) { - if (folio_test_unevictable(folio)) - folio_putback_lru(folio); - else - list_add(&folio->lru, &folio_list); - } - } else - folio_deactivate(folio); + madvise_lru_folio(folio, pageout, &folio_list); } =20 out: @@ -648,8 +648,8 @@ static long madvise_pageout(struct madvise_behavior *ma= dv_behavior) * owner nor write capable of the file. We allow private file mappings * further to pageout dirty anon pages. */ - if (!vma_is_anonymous(vma) && (!can_do_file_pageout(vma) && - (vma->vm_flags & VM_MAYSHARE))) + if (!vma_is_anonymous(vma) && !can_do_file_pageout(vma) && + (vma->vm_flags & VM_MAYSHARE)) return 0; =20 lru_add_drain(); --=20 2.56.0 From nobody Sat Nov 21 06:18:45 2026 Received: from mail-qk1-f182.google.com (mail-qk1-f182.google.com [209.85.222.182]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id CB171530E0F for ; Wed, 14 Oct 2026 18:30:34 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.222.182 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1792002636; cv=none; b=XqAKe+hUORXN1AbQbJHvhZJS3D8F17YhGYT6PzKI5CGVoJ25k/Vc+hQfJ6R7yPi71WnqE0bmZN1cYw3vOJCGZJMv1/lbXlfsiz8sZFUuKs27SQHS//D7kfWnXusRynvcAKfpS7CFQrcTEX0GSVuT2W54A2XKS7GZMW01cPYGHOk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1792002636; c=relaxed/simple; bh=3W2lDUFdYZ2Hsq57D5LYR7SZI1bhIdjcA4ZjEzjvL7Q=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=pnAcOAYXYbhqEyx3x2IjZmjKBf3VjLaqG+tKbypoKczBG5PHIpd0QEigla+MSdtlUEZpgV1BujC+FLc3v8uG3AyhLtnpleQpClMKy4PJ+o2Xun9j4slqA2pfz28XcDtdwirGcXIMXoHQ++Mtc/rBMrwbYnTH+fr1sd2L/FeHvr8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=gourry.net; spf=pass smtp.mailfrom=gourry.net; dkim=pass (2048-bit key) header.d=gourry.net header.i=@gourry.net header.b=bYQYKzRR; arc=none smtp.client-ip=209.85.222.182 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=gourry.net Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gourry.net Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gourry.net header.i=@gourry.net header.b="bYQYKzRR" Received: by mail-qk1-f182.google.com with SMTP id af79cd13be357-93e76469b93so49890085a.0 for ; Wed, 14 Oct 2026 11:30:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gourry.net; s=google; t=1792002634; x=1792607434; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=vnT5ptqor5qmMMV7Ow2caqqAnK7B10eCBBMNhfxBzB8=; b=bYQYKzRRvgtkocL7sx0ieZXAH4y9W70UpDKACp/DZM13hDb0fujVmc/HgeFRDIY2oK yfdO64AjgDD0nnXziUrB2f072JbkElivz6AI5+AXEpKRltVbW9LFG9AWEBfrcbwU1iv0 AtgZ41GgyVeTh4+Im+3N7KQZlrjG8Yam9cvP6HjMKK2jLaNzjUrVLUqfuTBGtTRxVIvR G4n33Py9h2w1BwRMUxpM669oTZz1406sn3t+7PaPbDHXEdf7/s1gqcIOIWg+1d3ss7WR PzK+N97psKAmZK4ekigJCKRC+JYpkB0poGhSsXwHXnlPhV/HUvRqOQHK0uUgq8+W4j/V jsrA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20260707; t=1792002634; x=1792607434; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=vnT5ptqor5qmMMV7Ow2caqqAnK7B10eCBBMNhfxBzB8=; b=WO67cY4Qa8Hk81+GtDiK6oq5zsaDofaoKgtzg1lkekqKr/CVnRUY3D9D20F7tmolm+ sAWvlUqdiURYMFJcSuNtKFcjxlcr4WN1InAHmidwe9fr3txFi8VPXrMgs59g6Br5nwwf bnIJta8NSnYG4apoArW6pcF+/N7siWK4+wbTTF/vkC0L+GP2zjuQc0rlD6cPAk5L6746 Nv8oCV1SbE5nWCAMbtOwUqF+Joa0LF97Brmwt1FXo4tzvevjv4sxU83PV1r2DQOTsGEB B0AeMV292WSILPosJxfj6LtISH1N2OjjdmCmapYG1uF72K8qJYP59ffFcu1MN8R0yuZK jv3w== X-Gm-Message-State: AFq9FYJ8ydSszmXpupVrHf4Xe4y3VPSrHVffTNiq07vbr2yWutj5uuI6 taLOM6iHMYo8Y7FOqFnDGeCQu3RQaYJBYdrYnhaQvuKjoPfq4vMmxxnkaiyDyXsZK24= X-Gm-Gg: AYBFou25eeIvzz2qXyRhmpJ46uJ6vwLE527rk4qFlqpXlTIczrkfcvYzVuRM8S7xqES gfvlJA+QLQHolJgmC29Sdn8yW5qXfLTo9R9MVcnurvcuRGAGnjbbSFtkJ/5+/2lYXVbmMH372tu 126Qzn7z2fvAgGbH0jyfyQc2RRFRdp+xcqUlIDbj4Ay45ZQNQ4FUn3QCjmt784FqoWr+9k6jSz1 ZzH/5jdFgcJgEyG45jj5YfvUfH5+/LmfB0ZTwgOOT11fuOL77FNsOUYjnovPul5AUxa8l/5V7VU OtC6H34Mi/GzkjpX8DSW97lQXwqleIV10JTlLh/aTM2kOcgoHpiobWwbKte8whTsHXchtZ9M1vj eI7tEBrIrOyW4j+ogAH2tHgFoZQsJTvr+gcDbtgrV8nn3K7sZRclEbyyskeQRchSIzjqtaj3IDT uNY3u/gupv5ygV8ECvi5voGDQ/RDeF3S7swL0CnFrKuIm8D3vIaj/9Hsd/S18JGTHxkvk1nwqQU KwRcIM90NY/9RylCPbMBmRJ85+pjz1lWohyVx5ASi0ZMr7GSYBdLK9GZLt0 X-Received: by 2002:a05:620a:1720:b0:93b:d7a3:45cf with SMTP id af79cd13be357-93ebd29005cmr2604633085a.56.1792002633778; Wed, 14 Oct 2026 11:30:33 -0700 (PDT) Received: from gourry-fedora-PF4VCD3F.lan (pool-173-79-60-52.washdc.fios.verizon.net. [173.79.60.52]) by smtp.gmail.com with ESMTPSA id af79cd13be357-93ef508811esm556593085a.36.2026.10.14.11.30.33 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 14 Oct 2026 11:30:33 -0700 (PDT) From: "Gregory Price (Meta)" To: linux-mm@kvack.org Cc: linux-kernel@vger.kernel.org, kernel-team@meta.com, akpm@linux-foundation.org, liam@infradead.org, ljs@kernel.org, david@kernel.org, vbabka@kernel.org, jannh@google.com Subject: [PATCH v2 03/10] mm/madvise: use the PMD softleaf validity helper Date: Wed, 14 Oct 2026 14:30:17 -0400 Message-ID: <20261014183024.2075709-4-gourry@gourry.net> X-Mailer: git-send-email 2.56.0 In-Reply-To: <20261014183024.2075709-1-gourry@gourry.net> References: <20261014183024.2075709-1-gourry@gourry.net> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" A non-present huge PMD must contain a software leaf type supported at PMD level. The open-coded check names the currently supported migration and device-private entries instead of expressing that invariant. Use pmd_is_valid_softleaf() so the validation follows the central definition of valid PMD softleaf entries. No functional change intended. Reviewed-by: Lorenzo Stoakes (ARM) Signed-off-by: Gregory Price (Meta) --- mm/madvise.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/mm/madvise.c b/mm/madvise.c index b0ce4ececba0..d3edc056ef57 100644 --- a/mm/madvise.c +++ b/mm/madvise.c @@ -430,8 +430,7 @@ static int madvise_lru_pmd_entry(pmd_t *pmd, unsigned l= ong addr, =20 orig_pmd =3D *pmd; if (unlikely(!pmd_present(orig_pmd))) { - VM_WARN_ON_ONCE(!pmd_is_migration_entry(orig_pmd) && - !pmd_is_device_private_entry(orig_pmd)); + VM_WARN_ON_ONCE(!pmd_is_valid_softleaf(orig_pmd)); goto huge_unlock; } =20 --=20 2.56.0 From nobody Sat Nov 21 06:18:45 2026 Received: from mail-qv1-f44.google.com (mail-qv1-f44.google.com [209.85.219.44]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5772C569F0B for ; Wed, 14 Oct 2026 18:30:36 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.219.44 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1792002637; cv=none; b=Bj1jfNUK3+38WcDL1gEYGGD/D703rg/ppPl+FqCGfgI0W8lPvNQqSYVKvrOCxNla2r6O8MQOjidD/2guIkm2QZmsiu9K8ZUrRKp1XZT8sq3se83z/eGgeC5nWFhP08ti3OWeZkC8N91bIoJN6/denwJ5J6BTLzOUFTAKpVEAACU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1792002637; c=relaxed/simple; bh=xPin/lqp+Y3xG4MtsBrRX/6vGgzLDywNumNeEFv9BOU=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=LG6crttPe+l0RMFNDdpU1I26PNoTkU+nRO2we4eWuYbb3GsB/ULXrtJLkOim5M1frXvH0sREJExZBbqIs3mgp/zrbbI560fGGaCdWMFMZXYctKDMyKhPZs/QmLJtZVb3ymr1qMmk/PP7WiScQ4YQBxviv4JxSYgiMa7Qqx+ZJpE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=gourry.net; spf=pass smtp.mailfrom=gourry.net; dkim=pass (2048-bit key) header.d=gourry.net header.i=@gourry.net header.b=i0EagxIP; arc=none smtp.client-ip=209.85.219.44 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=gourry.net Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gourry.net Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gourry.net header.i=@gourry.net header.b="i0EagxIP" Received: by mail-qv1-f44.google.com with SMTP id 6a1803df08f44-91bb5795792so6570896d6.2 for ; Wed, 14 Oct 2026 11:30:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gourry.net; s=google; t=1792002635; x=1792607435; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=NqHIExQRgXi1oeOkHNuWD0OjF5Gt5qjGM1Q50YZlDZM=; b=i0EagxIPkB6D6uK5qGwDqMvCtyQFrhkO6BQ0b+cEGxVkdwu5hRkvCDKL8jFfCWtdLM BCuBHV4jy1w6t0PvR3YK3EZfE1oNeo1G3HO4zcNSO9NzP7egIk+ecQ8D0rbs5P7UAOER qqrRBUKfp+sSmge8ZYxSwVh3LjTJQoHHlcXTjEVQnOydtgcVXRMnxoJUBuZshs56S0zn RO5ORt2myQWYli9pxvwqm/23IgaQbDlRvzF/GXDRYQ9AA1glg69bc5Q4VOSGqzJdi+Ao wTE4al5FgvOc128SXaBe9RZWBdXAgM7ivnCEjo8ccmx5/PMMvs1dO3WeOyZFSfl3CrKH 0ZRw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20260707; t=1792002635; x=1792607435; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=NqHIExQRgXi1oeOkHNuWD0OjF5Gt5qjGM1Q50YZlDZM=; b=ljR8Mhhp4fKOERtcohE3YYp5TldvIE9bs/JGItihWt3Zx2qgBei8rIcSRmEq/INdzc Vy+sYtTONcNgJxZykyggAwiXPt+Y2+8pxkegsz8qeygEBG33HwcXHMcqnFtkS73uFccJ hdQCm5kEJG7BMpi7nWyM+IQVoj2zpswdbBfZbcoLmAZ5lZLzc+33BOpcpRia86ji155X JBOLckk0mujd3vXmBa80ulyLULbuCiORJSLnqy+2KUD0TgY6B++1OpLzRM++coEVgjuB NzRSCfN4SldUpJxasST63VftmXnh5EkMoa0x7bXdjs7iP6RpAbFcutvRHsV6rnldd0k1 EQuQ== X-Gm-Message-State: AFq9FYJpTsXP8nG0MOOaVv86hZtFE0Ml/+U1DRYsEgeZaE7nDUEznE0h 3Z83a0n4Et3S2tqNetVnva6tLVOFhAH9NI3bbMskq8LA7ayN60wGpn8J8ZbjWKP//dY= X-Gm-Gg: AYBFou0u1rtIAObareGZBI1IzS85Ir97jyGczhzhIRbzNgYMaGZpURbt5JclnXeUXQN fb0kuWe13HqjrUD8JOJQCD1TLF5MN6SIutaZyJJCXSfJdESVFGxi2UgsxgDUEavI+NXdNSE9ObP r9Z/wO0aNsgHE56M67sGBgMCcCVt0W3m5oklnMLvjmHAe5nmMdKcdfW24aLCczfbHJrCW09QMOp q44bo16rYHPDvmc5bsWuXK7H9HUDEIzxYdmzA7IMDkqRwSb8qiko99kjgFbT1UmAHwCe5eneChR MuFDArAXKrEFNIOEhFWZ3MklBggj1+7O0/fU766grSuGPpcwQT+H0F1Cs73WgleuebjplFF+Hu4 t6bXzYPfKPqHwWrWVjP/3jZFW/495k4zV2A2wBMGgupl9i7Hq4/Ol9+nNQmL7OV8HDhZP1ApZhB qX7Mm/Clctmopx3ee6PvopPGAPnjYmksxmBbsu6QM8CjSzKSiGNt+MDJmAtTny7H+1DUB/v97lM ZHP5H0tBBh7a97UPEcafSBX5pCuhxPiLRifqUIaaEE9BrNebjR41WGHlJCA X-Received: by 2002:a05:620a:26a4:b0:93c:c73c:91f9 with SMTP id af79cd13be357-93ebd21f2afmr2707599385a.31.1792002635158; Wed, 14 Oct 2026 11:30:35 -0700 (PDT) Received: from gourry-fedora-PF4VCD3F.lan (pool-173-79-60-52.washdc.fios.verizon.net. [173.79.60.52]) by smtp.gmail.com with ESMTPSA id af79cd13be357-93ef508811esm556593085a.36.2026.10.14.11.30.34 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 14 Oct 2026 11:30:34 -0700 (PDT) From: "Gregory Price (Meta)" To: linux-mm@kvack.org Cc: linux-kernel@vger.kernel.org, kernel-team@meta.com, akpm@linux-foundation.org, liam@infradead.org, ljs@kernel.org, david@kernel.org, vbabka@kernel.org, jannh@google.com Subject: [PATCH v2 04/10] mm/madvise: factor the cold and pageout folio split Date: Wed, 14 Oct 2026 14:30:18 -0400 Message-ID: <20261014183024.2075709-5-gourry@gourry.net> X-Mailer: git-send-email 2.56.0 In-Reply-To: <20261014183024.2075709-1-gourry@gourry.net> References: <20261014183024.2075709-1-gourry@gourry.net> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Both the huge-PMD and PTE paths of madvise_lru_pmd_entry() split a folio they hold locked and referenced, then drop the lock and the reference, with the same three calls open-coded at each site. Move that sequence into madvise_lru_split_folio(), which returns the result of split_folio(). No functional change intended. Signed-off-by: Gregory Price (Meta) --- mm/madvise.c | 21 ++++++++++++--------- 1 file changed, 12 insertions(+), 9 deletions(-) diff --git a/mm/madvise.c b/mm/madvise.c index d3edc056ef57..1bb4f9f0e73a 100644 --- a/mm/madvise.c +++ b/mm/madvise.c @@ -397,6 +397,16 @@ static bool madvise_lru_folio_is_filtered(struct folio= *folio, (pageout_anon_only && !folio_test_anon(folio)); } =20 +/* Split a locked and referenced folio, then drop the lock and reference. = */ +static int madvise_lru_split_folio(struct folio *folio) +{ + int err =3D split_folio(folio); + + folio_unlock(folio); + folio_put(folio); + return err; +} + static int madvise_lru_pmd_entry(pmd_t *pmd, unsigned long addr, unsigned long end, struct mm_walk *walk) { @@ -445,16 +455,11 @@ static int madvise_lru_pmd_entry(pmd_t *pmd, unsigned= long addr, goto huge_unlock; =20 if (next - addr !=3D HPAGE_PMD_SIZE) { - int err; - if (!folio_trylock(folio)) goto huge_unlock; folio_get(folio); spin_unlock(ptl); - err =3D split_folio(folio); - folio_unlock(folio); - folio_put(folio); - if (!err) + if (!madvise_lru_split_folio(folio)) goto regular_folio; return 0; } @@ -526,9 +531,7 @@ static int madvise_lru_pmd_entry(pmd_t *pmd, unsigned l= ong addr, lazy_mmu_mode_disable(); pte_unmap_unlock(start_pte, ptl); start_pte =3D NULL; - err =3D split_folio(folio); - folio_unlock(folio); - folio_put(folio); + err =3D madvise_lru_split_folio(folio); start_pte =3D pte =3D pte_offset_map_lock(mm, pmd, addr, &ptl); if (!start_pte) --=20 2.56.0 From nobody Sat Nov 21 06:18:45 2026 Received: from mail-qt1-f176.google.com (mail-qt1-f176.google.com [209.85.160.176]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 1A03A569F35 for ; Wed, 14 Oct 2026 18:30:37 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.160.176 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1792002641; cv=none; b=cyjPIcKrHki1WUmQ4FMYYb6qc34gorA9kHmeQbWPjTrexlAFymNg9aRR8oSWBe5KrLiyaPS474qLiVZ/fB3TfS/ym3an6XLsJrWGEQK23jVsgwwA31qzrcTGcMjE1RhZ88xeZKpOb+jFK16b0wEUZXDIldm08dlMOBOdzbgZZwg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1792002641; c=relaxed/simple; bh=7hUDw895iis/f7DFhEno0C2kwpYd5firVWQqCbVJLxY=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=cyhaQCPtXtQyNVx0nGD8pgSPUaNwdy96DlP+Q9jY34vQuJeI3UpgHorjJSiIfEvfSY/+d4RO3DSBLQ4fwfexdF2rnIS9Kst5XgZge4DQx261iXypmLr0D59/oqyTfFSH+1fZS3fuPynhwPnvLdA9Ma9l6gXd7apEGYjxgG6tvmo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=gourry.net; spf=pass smtp.mailfrom=gourry.net; dkim=pass (2048-bit key) header.d=gourry.net header.i=@gourry.net header.b=Tx+Nitt6; arc=none smtp.client-ip=209.85.160.176 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=gourry.net Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gourry.net Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gourry.net header.i=@gourry.net header.b="Tx+Nitt6" Received: by mail-qt1-f176.google.com with SMTP id d75a77b69052e-535199ac562so43466541cf.3 for ; Wed, 14 Oct 2026 11:30:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gourry.net; s=google; t=1792002637; x=1792607437; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=8ePueSrtTjxyoQqFiqutrxbbN437intRkGhhEYPkovE=; b=Tx+Nitt6zLmA6pWLd+bSjVUtNo6PIicc+agp21Pyb+fXlaxLyXSB/N8WmsyespRzdo Hm5dZhhFbINFfVn1yz3m5Qpjgak5CPUOtEJH/Og7+pxQu/FuP+qrr0r94on6Xi5U7npa AQXUyQV72ICtil7VD9cdbF+SjAMSW7jl2iiBzYJp2DhlrTJbW5VDWtPa/grLeXBToBoY rV3+bdhrhEM3duHFaSlQp/fhqWdCC/vBb9ThEjDZJ2rCHycpaCdrjvVs3e+8ouavKk99 1dFea0lXmQi3RaxMNeuSZ2C9XPqLt3kqSGDTarY9BuPt3jrvv+ZZi6rm11Q0riG+9ayi PnNA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20260707; t=1792002637; x=1792607437; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=8ePueSrtTjxyoQqFiqutrxbbN437intRkGhhEYPkovE=; b=Pn9vTrv6RYD+7KCmoGAsFq7PT5ntHAA+209Su8hE6inRm6WahCPGI0XwmfcqJ+38rW uGCCOvyWYlnAD6U3dSIYImXOFFG6uEaPsQCGptDScbST3Zl4l0YbVmNx7/eCQehQ/cT7 g2n7JpZERPacHMj4Jl+9YVPaiDALLSXP4YNUO+M/vSI/Wun8J5/7cPJcS9cI752aD1JP vKMKHHDbRYFXxGMtMlHg5lzwDAiMoRlG6qSYTy4iBVBdDWj8Xwlzf4I9aSG++6CiqZAX XjGVUEbCkgZkd4ZOntodismpWWyZzSD3CrImx5k8iC7EK2zPD5Ap+IZ2K3FTSGVqJF0S g/8Q== X-Gm-Message-State: AFq9FYIq6PY4FkUnyKWgqcsvx1/cF1kMgKDkqsQtOvuqPMY2rDDReMwU 1nAf3J2BaFjT1s9WS5QmGmLoLmpgBzpkNUqYZncR8Eib5FC26pgN+tifdtgYdCa0iXk= X-Gm-Gg: AYBFou10ALRLr+7LliKlDK6CYYaw4huDZObFOadsiNkZc5voOXznpU7z/PvTs2P1NFg eofJ9ZUq50dH6L2lD1nKLbyd9sWAOOaxuPqSEqUSgOoiOaXj0BRXvTYom6rBypJVuSkyAo7k7fW aFiCTfBkHQTZmR8yEE6BtBKfSGIl/9km/DP3o2xJwbO7H2w2YwFKXfosxatJ790HYyEjc5PF2oz Zhb8N1+TzqF9maS1DGZ7q7kI8pZpqd739wZWXHKZQV38XycOYYtZ4Pnja/spDBE34RN4ooaRogh U6GdlRtcl8Wm2+3nGRHZVcRkqCGsoiZIME2obcUV3/CxrIlpg+TRZZbTfNbGg5B072kMV8ZyPuH tv/h+55LWLzHsZ40pHpGw6O4FOQAHCbGIYCWIbE6YDi6s7hR+0U6LtM5FFZy3KY1wHAAdU8Ogxs Ib+8JZEBcZe/9Oj6iRfuTsSYJ9nav+diaLPV71+6JSQ1rDWtbk4aNlaF6XvgThXcE88jF5IFGDW DxUc8feHADHohWSl2mN+WgoWf5qovFvRIWf6TF4oERXRZnOR9cVgIgr1BAS X-Received: by 2002:a05:620a:46a1:b0:93e:95ce:53b5 with SMTP id af79cd13be357-93ebd28b9b7mr2938174385a.52.1792002636690; Wed, 14 Oct 2026 11:30:36 -0700 (PDT) Received: from gourry-fedora-PF4VCD3F.lan (pool-173-79-60-52.washdc.fios.verizon.net. [173.79.60.52]) by smtp.gmail.com with ESMTPSA id af79cd13be357-93ef508811esm556593085a.36.2026.10.14.11.30.35 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 14 Oct 2026 11:30:36 -0700 (PDT) From: "Gregory Price (Meta)" To: linux-mm@kvack.org Cc: linux-kernel@vger.kernel.org, kernel-team@meta.com, akpm@linux-foundation.org, liam@infradead.org, ljs@kernel.org, david@kernel.org, vbabka@kernel.org, jannh@google.com Subject: [PATCH v2 05/10] mm/madvise: factor huge-PMD folio processing Date: Wed, 14 Oct 2026 14:30:19 -0400 Message-ID: <20261014183024.2075709-6-gourry@gourry.net> X-Mailer: git-send-email 2.56.0 In-Reply-To: <20261014183024.2075709-1-gourry@gourry.net> References: <20261014183024.2075709-1-gourry@gourry.net> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" The huge-PMD branch of madvise_lru_pmd_entry() decides inline whether the PMD maps a folio worth advising: it checks that the PMD is present, looks up the folio and applies the shared filters, all between taking the PMD lock and splitting or advising the folio. Move those checks into madvise_lru_huge_pmd_folio(), which returns NULL when there is nothing to do, and move the MADV_COLD young-bit clearing into madvise_cold_pmd(). The split stays in the caller, which owns the PMD lock that must be dropped before splitting. No functional change intended. Suggested-by: Lorenzo Stoakes (ARM) Signed-off-by: Gregory Price (Meta) --- mm/madvise.c | 55 +++++++++++++++++++++++++++++++++++----------------- 1 file changed, 37 insertions(+), 18 deletions(-) diff --git a/mm/madvise.c b/mm/madvise.c index 1bb4f9f0e73a..f847dddada8a 100644 --- a/mm/madvise.c +++ b/mm/madvise.c @@ -407,6 +407,37 @@ static int madvise_lru_split_folio(struct folio *folio) return err; } =20 +#ifdef CONFIG_TRANSPARENT_HUGEPAGE +static void madvise_cold_pmd(struct mmu_gather *tlb, struct vm_area_struct= *vma, + pmd_t *pmd, unsigned long addr, pmd_t orig_pmd) +{ + if (!pmd_young(orig_pmd)) + return; + + pmdp_test_and_clear_young(vma, addr, pmd); + tlb_remove_pmd_tlb_entry(tlb, pmd, addr); +} + +/* Return the folio mapped by a huge PMD if it should be advised. */ +static struct folio *madvise_lru_huge_pmd_folio(struct vm_area_struct *vma, + unsigned long addr, pmd_t orig_pmd, bool pageout_anon_only) +{ + struct folio *folio; + + if (unlikely(!pmd_present(orig_pmd))) { + VM_WARN_ON_ONCE(!pmd_is_valid_softleaf(orig_pmd)); + return NULL; + } + + folio =3D vm_normal_folio_pmd(vma, addr, orig_pmd); + if (!folio || folio_is_zone_device(folio)) + return NULL; + if (madvise_lru_folio_is_filtered(folio, pageout_anon_only)) + return NULL; + return folio; +} +#endif + static int madvise_lru_pmd_entry(pmd_t *pmd, unsigned long addr, unsigned long end, struct mm_walk *walk) { @@ -432,6 +463,7 @@ static int madvise_lru_pmd_entry(pmd_t *pmd, unsigned l= ong addr, if (pmd_trans_huge(*pmd)) { pmd_t orig_pmd; unsigned long next =3D pmd_addr_end(addr, end); + const bool spans_pmd =3D next - addr =3D=3D HPAGE_PMD_SIZE; =20 tlb_change_page_size(tlb, HPAGE_PMD_SIZE); ptl =3D pmd_trans_huge_lock(pmd, vma); @@ -439,22 +471,12 @@ static int madvise_lru_pmd_entry(pmd_t *pmd, unsigned= long addr, return 0; =20 orig_pmd =3D *pmd; - if (unlikely(!pmd_present(orig_pmd))) { - VM_WARN_ON_ONCE(!pmd_is_valid_softleaf(orig_pmd)); - goto huge_unlock; - } - - folio =3D vm_normal_folio_pmd(vma, addr, orig_pmd); + folio =3D madvise_lru_huge_pmd_folio(vma, addr, orig_pmd, + pageout_anon_only); if (!folio) goto huge_unlock; =20 - if (folio_is_zone_device(folio)) - goto huge_unlock; - - if (madvise_lru_folio_is_filtered(folio, pageout_anon_only)) - goto huge_unlock; - - if (next - addr !=3D HPAGE_PMD_SIZE) { + if (!spans_pmd) { if (!folio_trylock(folio)) goto huge_unlock; folio_get(folio); @@ -464,11 +486,8 @@ static int madvise_lru_pmd_entry(pmd_t *pmd, unsigned = long addr, return 0; } =20 - if (!pageout && pmd_young(orig_pmd)) { - pmdp_test_and_clear_young(vma, addr, pmd); - tlb_remove_pmd_tlb_entry(tlb, pmd, addr); - } - + if (!pageout) + madvise_cold_pmd(tlb, vma, pmd, addr, orig_pmd); madvise_lru_folio(folio, pageout, &folio_list); huge_unlock: spin_unlock(ptl); --=20 2.56.0 From nobody Sat Nov 21 06:18:45 2026 Received: from mail-qk1-f182.google.com (mail-qk1-f182.google.com [209.85.222.182]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5E1A2565112 for ; Wed, 14 Oct 2026 18:30:39 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.222.182 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1792002642; cv=none; b=kfq88mUMKc1A823guCTj/fD2NkemkWV7rms9LQMKKFl21LnoUbT1uduFZHuiFBYhW/bGOIYCBnMrqdNEhqc1iAV/zI8UAVqZNb0ix0OvSFfT8eEfML2C4kC7ZlzUyI2O92LWhcyfmzPnJUBu8yRfjYY4Q/CYjBNyiGIMhpppUcU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1792002642; c=relaxed/simple; bh=JQc3liNbJkS6D9jh3fTfhJUbZklXqQ3ECDvw6JcNln0=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=u1znG3zXetxgEG+plQsL6cIK+kw6K6NALxvFA4EbGkoKlkmnGQ/sccbRojbBUzp0KBJxfwwayOxlvkw4HhCmruCMkR334k/BiT09jYVSzUoOS5ITlnZCdesQRhF6JBhvhG9LDEzcSVwkaSBhv798JxcmlbO8gm81UVjX/xwX6s4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=gourry.net; spf=pass smtp.mailfrom=gourry.net; dkim=pass (2048-bit key) header.d=gourry.net header.i=@gourry.net header.b=D0vj2vQ/; arc=none smtp.client-ip=209.85.222.182 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=gourry.net Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gourry.net Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gourry.net header.i=@gourry.net header.b="D0vj2vQ/" Received: by mail-qk1-f182.google.com with SMTP id af79cd13be357-93e635220b9so314756385a.2 for ; Wed, 14 Oct 2026 11:30:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gourry.net; s=google; t=1792002638; x=1792607438; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=oviJZuz7wJ3+WoQ9W/TGlT7rNjNMCITRK6EOVD5+7G0=; b=D0vj2vQ/ZXSHBan1ouWQRDFOHrOXy2vQxvPm5BO1YPgipFpBoGXdAoItfNKIZZyxaI Ag0QQyZ9jVtijPe3HEP/hrW4pjvpnvDfKKt+dfby1DFwNwWSKTRbUSWQXEH+sTwCdEhw 2cCSdaOmJ1zGmM8OeGWbGQWdDQL0meEyQt+Xb6rw6gt4l6H6o4VSBc2Zibbzv/MoP8MQ ZAM9YaW6rV7WAjUa32sUsvoOUoUhYZtKwsqdd6tA5Q5aau1wQGrxYewhuGoBxvYseivA xJSASRoHDaOO5B6Zs9pQibzx9+YtVdxHU+EdXtoMrVEYZFMI48v9ls6uJNvXmUUt+wdH wYAQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20260707; t=1792002638; x=1792607438; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=oviJZuz7wJ3+WoQ9W/TGlT7rNjNMCITRK6EOVD5+7G0=; b=wd1gLP/m9n5jTaRK1vunUqEnY9b/mNU6favm+wAGe4ULQT5LXV/QQCDAGjpSv5Z823 wMzG+N8HGlMtqSx9CK6ZpkGmlBp3ZqCNBp1foluWDxY4wMAQzEb7xweymrTwXpQQi4mO Py82hvKlGjQYGoEVEnrEfRMjGDi7/D0XfRO6PAqoanHXvw+4PIXtdlM2asAKeYeeUvsj N47h+SKW/Ocibf3qPz6UyA1lP7wZ6vOmRpr2k0liBFJ1xrH1iaWMIJA130gbPh3bFhSD oREb7SMmCqXMZIHak0BJMWn/tZzyWkDzUNST0l9EGm6ZgONt7wJOX7Lk8GFQuuk2HYjE SUlQ== X-Gm-Message-State: AFq9FYLCc+W/vJrDcDhSgaFB28snBmo8tF3ImSlkyJz9d6NPP+MunXzY 3e7CawnArJWb3ZB6aQzOM7QhVs1xes+NnYbeXu6KRS7ayxdbq3TlPbuD+p9N2GjijDc= X-Gm-Gg: AYBFou2LotxXDxjuo4XpQnEnqly/n0TubFVhriA3TbeCzmCx8rUn4Nb9mZfFMbT1nQp St0QVonKne513Un6TPdGFhGNa7UOO1FSS5Q3QsitZsdiI6tNILt6Z3v7g8jtw4K40bI424eYwTc /3zDr2HSFglVsYCj4iCrPnpvJ80y2GbZ1eguMR0dUoX8FFXrMt6PZBmMyP5Eowh703kTIS92HPM Enx9a+NdnsBwfzDQbMtB+w0fTH5kQcc8kT32c8H3uZHDzf/LtGNEZpWMCTIbSqVmYrEv6sI6Otc QRp0GUmH1KFI9uY++fq6Il6faECt2EIMEhBh7ytaDGpvQpBLpl30ybZ9F7vONapaf1GthukPBTk 7JzLVZrM3gWBoJFLMN4lOgZav37OA6fdg80fWZw8qqcTpxlGd6RTconmvtQNs6ke+bJ8WMB/GhA NYh/mjxaO8BnH+mvuha0dydisEMJON2h6ZjBZUq61lnme7vjgrz5mbz/DIFUzSztmL6IghX1ybD KPaU1u2VIOBmVlLOjL+Anamvowai+5BeKIBg0y0B7sUycHGyR60+vldPf/t X-Received: by 2002:a05:620a:7106:b0:93e:c12c:f11b with SMTP id af79cd13be357-93ec12d0658mr2477753285a.71.1792002638192; Wed, 14 Oct 2026 11:30:38 -0700 (PDT) Received: from gourry-fedora-PF4VCD3F.lan (pool-173-79-60-52.washdc.fios.verizon.net. [173.79.60.52]) by smtp.gmail.com with ESMTPSA id af79cd13be357-93ef508811esm556593085a.36.2026.10.14.11.30.37 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 14 Oct 2026 11:30:37 -0700 (PDT) From: "Gregory Price (Meta)" To: linux-mm@kvack.org Cc: linux-kernel@vger.kernel.org, kernel-team@meta.com, akpm@linux-foundation.org, liam@infradead.org, ljs@kernel.org, david@kernel.org, vbabka@kernel.org, jannh@google.com Subject: [PATCH v2 06/10] mm/madvise: separate huge PMDs from the PTE walk Date: Wed, 14 Oct 2026 14:30:20 -0400 Message-ID: <20261014183024.2075709-7-gourry@gourry.net> X-Mailer: git-send-email 2.56.0 In-Reply-To: <20261014183024.2075709-1-gourry@gourry.net> References: <20261014183024.2075709-1-gourry@gourry.net> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" madvise_lru_pmd_entry() still handles a huge PMD inline, and jumps into its PTE walk with a goto when the huge-PMD folio was split. The PMD lock, the split and the reclaim of the advised folio all share a function with the PTE-table loop. Move the huge-PMD path into madvise_lru_huge_pmd(), which takes the PMD lock, advises or splits the folio and reclaims, and returns false only when the caller must walk the new PTEs. It uses the end address passed by walk_pmd_range(), which is already clamped to the PMD boundary, instead of recomputing it. No functional change intended. Signed-off-by: Gregory Price (Meta) --- mm/madvise.c | 97 +++++++++++++++++++++++++++++++--------------------- 1 file changed, 58 insertions(+), 39 deletions(-) diff --git a/mm/madvise.c b/mm/madvise.c index f847dddada8a..a9e5a3e2fab4 100644 --- a/mm/madvise.c +++ b/mm/madvise.c @@ -436,6 +436,62 @@ static struct folio *madvise_lru_huge_pmd_folio(struct= vm_area_struct *vma, return NULL; return folio; } + +/* + * Return false only when the folio was split and the caller must walk the + * new PTEs. + */ +static bool madvise_lru_huge_pmd(pmd_t *pmd, unsigned long addr, + unsigned long next, struct mm_walk *walk, + bool pageout_anon_only) +{ + const struct madvise_walk_private *private =3D walk->private; + const bool spans_pmd =3D next - addr =3D=3D HPAGE_PMD_SIZE; + struct mmu_gather *tlb =3D private->tlb; + struct vm_area_struct *vma =3D walk->vma; + bool pageout =3D private->pageout; + LIST_HEAD(folio_list); + struct folio *folio; + spinlock_t *ptl; + pmd_t orig_pmd; + + tlb_change_page_size(tlb, HPAGE_PMD_SIZE); + ptl =3D pmd_trans_huge_lock(pmd, vma); + if (!ptl) + return true; + + orig_pmd =3D *pmd; + folio =3D madvise_lru_huge_pmd_folio(vma, addr, orig_pmd, + pageout_anon_only); + if (!folio) + goto unlock; + + if (!spans_pmd) { + if (!folio_trylock(folio)) + goto unlock; + folio_get(folio); + spin_unlock(ptl); + return madvise_lru_split_folio(folio) !=3D 0; + } + + if (!pageout) + madvise_cold_pmd(tlb, vma, pmd, addr, orig_pmd); + madvise_lru_folio(folio, pageout, &folio_list); +unlock: + spin_unlock(ptl); + if (pageout) + reclaim_pages(&folio_list); + cond_resched(); + return true; +} +#else +/* pmd_trans_huge() is always false without THP, so this is never called. = */ +static bool madvise_lru_huge_pmd(pmd_t *pmd, unsigned long addr, + unsigned long next, struct mm_walk *walk, + bool pageout_anon_only) +{ + return false; +} #endif =20 static int madvise_lru_pmd_entry(pmd_t *pmd, unsigned long addr, @@ -459,46 +515,9 @@ static int madvise_lru_pmd_entry(pmd_t *pmd, unsigned = long addr, pageout_anon_only =3D pageout && !vma_is_anonymous(vma) && !can_do_file_pageout(vma); =20 -#ifdef CONFIG_TRANSPARENT_HUGEPAGE - if (pmd_trans_huge(*pmd)) { - pmd_t orig_pmd; - unsigned long next =3D pmd_addr_end(addr, end); - const bool spans_pmd =3D next - addr =3D=3D HPAGE_PMD_SIZE; - - tlb_change_page_size(tlb, HPAGE_PMD_SIZE); - ptl =3D pmd_trans_huge_lock(pmd, vma); - if (!ptl) - return 0; - - orig_pmd =3D *pmd; - folio =3D madvise_lru_huge_pmd_folio(vma, addr, orig_pmd, - pageout_anon_only); - if (!folio) - goto huge_unlock; - - if (!spans_pmd) { - if (!folio_trylock(folio)) - goto huge_unlock; - folio_get(folio); - spin_unlock(ptl); - if (!madvise_lru_split_folio(folio)) - goto regular_folio; - return 0; - } - - if (!pageout) - madvise_cold_pmd(tlb, vma, pmd, addr, orig_pmd); - madvise_lru_folio(folio, pageout, &folio_list); -huge_unlock: - spin_unlock(ptl); - if (pageout) - reclaim_pages(&folio_list); - cond_resched(); + if (pmd_trans_huge(*pmd) && + madvise_lru_huge_pmd(pmd, addr, end, walk, pageout_anon_only)) return 0; - } - -regular_folio: -#endif tlb_change_page_size(tlb, PAGE_SIZE); restart: start_pte =3D pte =3D pte_offset_map_lock(vma->vm_mm, pmd, addr, &ptl); --=20 2.56.0 From nobody Sat Nov 21 06:18:45 2026 Received: from mail-qk1-f178.google.com (mail-qk1-f178.google.com [209.85.222.178]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E7EC3569F0F for ; Wed, 14 Oct 2026 18:30:41 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.222.178 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1792002643; cv=none; b=U0mUFVlKS2XNpNMk5wYf+ifZa3hp0x6Uej+j8C5FMHMVmA9HovOJGVUdQCDFAurg5hYrtxe0Cl45VaJ84nbP2Eg972eFR3borAU6eX764ng/eSEod3EczWWEGkCQu7GAVOHtXyAwSx7hRsZy+ep9JJXDFvXNhKSWgtfaQo/QY/M= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1792002643; c=relaxed/simple; bh=UNw2BubWWFmr5yIohRWiLpSz99E6Goio0mZ5j9KXo3g=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=bNjJr4oKnEW+/3LwmvH7UrL+Z5ONgQuEg7v5J2lxWln5gmzeK9Vur75V/UCSc8BI0jywGzcrptRtj2DlKWKuPfwK956+lgjRW5HRpxqDNX74SPOf9k01oLlg/ldzN6FrtJ8zAwDvQpg6d6Z7CBXXZ2dBKxqzJlxKiI4JpG+hNHA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=gourry.net; spf=pass smtp.mailfrom=gourry.net; dkim=pass (2048-bit key) header.d=gourry.net header.i=@gourry.net header.b=gn52DVVG; arc=none smtp.client-ip=209.85.222.178 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=gourry.net Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gourry.net Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gourry.net header.i=@gourry.net header.b="gn52DVVG" Received: by mail-qk1-f178.google.com with SMTP id af79cd13be357-93cb9e9405fso41528785a.2 for ; Wed, 14 Oct 2026 11:30:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gourry.net; s=google; t=1792002641; x=1792607441; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=6rnuGk4ufowJ9LY5euPS4WtjSZLrLvn92pFDqhxj834=; b=gn52DVVG9IhMHLB4MvLynuLwnb/t7eLlxS71I5bezyS7EftiPy9Rwe9kLlB0T38HEe LYCNW7dMaA6gXiOOlVPhvgk6TSvQQmJtO+Iox3jlgCo+0ABS8kIiLdQpeWbZaas0Zlbc cnhgfLUavjjRF/uHAkFgx8x7RSDE8LUbqNbatuwkTU+B8JTINpwfjZhFmqsupKEgIJj3 kdMRcCFaYZPpiu6dgNsbMBxcaJjrnHRIUiN72GQRxGcMutcjAFjg1xxAJSPHK3OoR8m4 h50coUI8mApiBkMdMk73/ybQstNg9Ehq+WyyWbZVg7YflmeeYsn7XOPNgF8KJ/dHz+RS fw+g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20260707; t=1792002641; x=1792607441; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=6rnuGk4ufowJ9LY5euPS4WtjSZLrLvn92pFDqhxj834=; b=OCIxaaDdIo9eV5ioCRNTz2jUQVJiaLlsSC6gjzDY09b1ttOwyww2gctBO2h+3Teju6 4307rJg3oAanBIhVJPXmGkkgpc4Hk9G2nP3FnUag2RLo1RJ1sWDGojXMIxsdtRPhO/uO OrBUC5IHUeeQva7+/J6gzlp+/uoS8jbLqH3EdTie2xizqSEy7e3UiXugKw2/CsSI8T2o 5Sh1+QX4Qb71frdgThMfhAN8DS2Qt0McPUFflDGFA7fcHFVgIoLZZwsyxJAvuxqWpjyn /14Os8FZCWyvXo+FxsTJFxfTDybZOQ+MJXvFySGVVDKRq8fwIuqunp5aelqoEcamziUu YyzA== X-Gm-Message-State: AFq9FYItiPLvXIIPh2nzwhRoNV3AUkH2W5ikdq0XubxUIrOgu0naU0YB PJVQD4bpL1qFgi68c15Bv/hY8tUB4dg7xA6y7LcKrKxdFOrmpcuqNBJb0RRM9/tG0y0= X-Gm-Gg: AYBFou31MwV0aDqvctyCj/r3hiS21/N8FcTazw7pwzQj/sUi1Li7PAbirPhZBhsZBYX Ijc/hBPvhj0ZUzVKrGUGGyoPRKtNaRBZ0A3PshtS1dVmUIEHVNrxL8yVBYWBs8xAW7G5GJyWKG1 GmzdIhFKJyM0AGYXpqf/4o505Rcm2QekKYfL4koIlSSY2V0OI6BZjFY1LLuvaVAJcPBKc4LpBPE CMoQc62RE42KnlNCpznb0f029Y51tFiKi5sgRdyRS+bcelH93WFch4VYUvDtA4T1WHS7IVqx8zS chFoyXjaL+VA4j1dVLj43c6y2ZuqNO+WtbSD83SvL4ne7xrUPJ3sjVX9OreUmMBtWn45m2/gV8D H2+G8e/A5EXEOAQjekd/GK/B+0mnTnob4ui6Q6oGUp0cA7ued+n/MZl6tflfjMOzpwqz3EFE8lX hSjfPbE0qx9iJIzWSC/QhMiSSiU1RUz5mbyTmyDCGvAGFiPr4Ilf3InKndE4zIZkvmOphFvNlzr uWQm4oC88mEzc9u2jcgktyUGnbe+3XdIcI/QIQKhvfKzQE3BHF+PXmH+I7g X-Received: by 2002:a05:620a:2715:b0:93e:6709:bc7f with SMTP id af79cd13be357-93ebd2113a5mr2385591185a.15.1792002639693; Wed, 14 Oct 2026 11:30:39 -0700 (PDT) Received: from gourry-fedora-PF4VCD3F.lan (pool-173-79-60-52.washdc.fios.verizon.net. [173.79.60.52]) by smtp.gmail.com with ESMTPSA id af79cd13be357-93ef508811esm556593085a.36.2026.10.14.11.30.38 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 14 Oct 2026 11:30:39 -0700 (PDT) From: "Gregory Price (Meta)" To: linux-mm@kvack.org Cc: linux-kernel@vger.kernel.org, kernel-team@meta.com, akpm@linux-foundation.org, liam@infradead.org, ljs@kernel.org, david@kernel.org, vbabka@kernel.org, jannh@google.com Subject: [PATCH v2 07/10] mm/madvise: separate PTE-batch folio processing Date: Wed, 14 Oct 2026 14:30:21 -0400 Message-ID: <20261014183024.2075709-8-gourry@gourry.net> X-Mailer: git-send-email 2.56.0 In-Reply-To: <20261014183024.2075709-1-gourry@gourry.net> References: <20261014183024.2075709-1-gourry@gourry.net> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Each pass of the PTE loop validates the entry, batches the PTEs that map one large folio, decides whether a partly mapped large folio must be split, and ages or isolates the folio. The split path also drops and retakes the PTE lock in the middle of that sequence. Move the per-batch work into madvise_lru_pte_batch_locked(), which runs under the PTE lock and returns a folio that needs splitting locked and referenced. The folio lookup goes into madvise_lru_pte_folio() and the MADV_COLD young-bit clearing into madvise_cold_ptes(). The loop drops the lock, splits the folio and maps the PTE table again at the same address. No functional change intended. Signed-off-by: Gregory Price (Meta) --- mm/madvise.c | 151 +++++++++++++++++++++++++++++---------------------- 1 file changed, 86 insertions(+), 65 deletions(-) diff --git a/mm/madvise.c b/mm/madvise.c index a9e5a3e2fab4..0c0a840102eb 100644 --- a/mm/madvise.c +++ b/mm/madvise.c @@ -407,6 +407,77 @@ static int madvise_lru_split_folio(struct folio *folio) return err; } =20 +static void madvise_cold_ptes(struct mmu_gather *tlb, + struct vm_area_struct *vma, pte_t *pte, unsigned long addr, + int nr, pte_t ptent) +{ + if (!pte_young(ptent)) + return; + + clear_young_dirty_ptes(vma, addr, pte, nr, CYDP_CLEAR_YOUNG); + tlb_remove_tlb_entries(tlb, pte, nr, addr); +} + +/* Return the normal folio mapped by a present PTE, or NULL. */ +static struct folio *madvise_lru_pte_folio(struct vm_area_struct *vma, + unsigned long addr, pte_t ptent) +{ + struct folio *folio; + + if (pte_none(ptent) || !pte_present(ptent)) + return NULL; + + folio =3D vm_normal_folio(vma, addr, ptent); + if (!folio || folio_is_zone_device(folio)) + return NULL; + return folio; +} + +/* Return a split candidate locked and referenced for use after the PTL dr= op. */ +static struct folio *madvise_lru_pte_batch_locked(pte_t *pte, + unsigned long addr, unsigned long end, struct mm_walk *walk, + struct list_head *folio_list, bool pageout_anon_only, int *nr) +{ + const struct madvise_walk_private *private =3D walk->private; + struct vm_area_struct *vma =3D walk->vma; + pte_t ptent =3D ptep_get(pte); + struct folio *folio; + + *nr =3D 1; + folio =3D madvise_lru_pte_folio(vma, addr, ptent); + if (!folio) + return NULL; + + /* Split a large folio that the range only partly maps. */ + if (folio_test_large(folio)) { + *nr =3D madvise_folio_pte_batch(addr, end, folio, pte, &ptent); + if (*nr < folio_nr_pages(folio)) { + if (madvise_lru_folio_is_filtered(folio, pageout_anon_only)) + return NULL; + if (!folio_trylock(folio)) + return NULL; + folio_get(folio); + return folio; + } + } + + /* + * Skip non-LRU folios and folios mapped elsewhere. A large folio here + * is fully mapped, so it is exclusive if its mapcount equals its page + * count. + */ + if (!folio_test_lru(folio) || + folio_mapcount(folio) !=3D folio_nr_pages(folio)) + return NULL; + if (pageout_anon_only && !folio_test_anon(folio)) + return NULL; + + if (!private->pageout) + madvise_cold_ptes(private->tlb, vma, pte, addr, *nr, ptent); + madvise_lru_folio(folio, private->pageout, folio_list); + return NULL; +} + #ifdef CONFIG_TRANSPARENT_HUGEPAGE static void madvise_cold_pmd(struct mmu_gather *tlb, struct vm_area_struct= *vma, pmd_t *pmd, unsigned long addr, pmd_t orig_pmd) @@ -502,7 +573,7 @@ static int madvise_lru_pmd_entry(pmd_t *pmd, unsigned l= ong addr, bool pageout =3D private->pageout; struct mm_struct *mm =3D tlb->mm; struct vm_area_struct *vma =3D walk->vma; - pte_t *start_pte, *pte, ptent; + pte_t *start_pte, *pte; spinlock_t *ptl; struct folio *folio =3D NULL; LIST_HEAD(folio_list); @@ -526,9 +597,6 @@ static int madvise_lru_pmd_entry(pmd_t *pmd, unsigned l= ong addr, flush_tlb_batched_pending(mm); lazy_mmu_mode_enable(); for (; addr < end; pte +=3D nr, addr +=3D nr * PAGE_SIZE) { - nr =3D 1; - ptent =3D ptep_get(pte); - if (++batch_count =3D=3D SWAP_CLUSTER_MAX) { batch_count =3D 0; if (need_resched()) { @@ -539,69 +607,22 @@ static int madvise_lru_pmd_entry(pmd_t *pmd, unsigned= long addr, } } =20 - if (pte_none(ptent)) - continue; - - if (!pte_present(ptent)) - continue; - - folio =3D vm_normal_folio(vma, addr, ptent); - if (!folio || folio_is_zone_device(folio)) - continue; - - /* - * If we encounter a large folio, only split it if it is not - * fully mapped within the range we are operating on. Otherwise - * leave it as is so that it can be swapped out whole. If we - * fail to split a folio, leave it in place and advance to the - * next pte in the range. - */ - if (folio_test_large(folio)) { - nr =3D madvise_folio_pte_batch(addr, end, folio, pte, &ptent); - if (nr < folio_nr_pages(folio)) { - int err; - - if (madvise_lru_folio_is_filtered(folio, pageout_anon_only)) - continue; - if (!folio_trylock(folio)) - continue; - folio_get(folio); - lazy_mmu_mode_disable(); - pte_unmap_unlock(start_pte, ptl); - start_pte =3D NULL; - err =3D madvise_lru_split_folio(folio); - start_pte =3D pte =3D - pte_offset_map_lock(mm, pmd, addr, &ptl); - if (!start_pte) - break; - flush_tlb_batched_pending(mm); - lazy_mmu_mode_enable(); - if (!err) - nr =3D 0; - continue; - } - } - - /* - * Do not interfere with other mappings of this folio and - * non-LRU folio. If we have a large folio at this point, we - * know it is fully mapped so if its mapcount is the same as its - * number of pages, it must be exclusive. - */ - if (!folio_test_lru(folio) || - folio_mapcount(folio) !=3D folio_nr_pages(folio)) + folio =3D madvise_lru_pte_batch_locked(pte, addr, end, walk, + &folio_list, pageout_anon_only, &nr); + if (!folio) continue; =20 - if (pageout_anon_only && !folio_test_anon(folio)) - continue; - - if (!pageout && pte_young(ptent)) { - clear_young_dirty_ptes(vma, addr, pte, nr, - CYDP_CLEAR_YOUNG); - tlb_remove_tlb_entries(tlb, pte, nr, addr); - } - - madvise_lru_folio(folio, pageout, &folio_list); + lazy_mmu_mode_disable(); + pte_unmap_unlock(start_pte, ptl); + start_pte =3D NULL; + if (!madvise_lru_split_folio(folio)) + nr =3D 0; + start_pte =3D pte_offset_map_lock(mm, pmd, addr, &ptl); + if (!start_pte) + break; + pte =3D start_pte; + flush_tlb_batched_pending(mm); + lazy_mmu_mode_enable(); } =20 out: --=20 2.56.0 From nobody Sat Nov 21 06:18:45 2026 Received: from mail-qk1-f175.google.com (mail-qk1-f175.google.com [209.85.222.175]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8363256C624 for ; Wed, 14 Oct 2026 18:30:42 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.222.175 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1792002644; cv=none; b=Z1QHQMUSk3GOtrV3aUL/HrndyfoSKx2ucqVI5gGAbbtrMuNu9OLvPfGbJAchquFgyuRKdcyPJ6sZGQfKGU47wANKIkkzjxXaZx/cuyPSs1c9cCYAVoSWz0RClUnt0XqQ1xJNCeAC6NUUaAd9PRfrg1v8swUk+V1/2PO7Ncfb3jQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1792002644; c=relaxed/simple; bh=DJA4oQ46qzUlaXjLajvqMmqhIHbg/eu3yAEpqRirCfw=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=UffTC4R+e8G6bN1az3M/TtDbEbHoErDjQ9ChWlOXGwZ/nqaqGhlhPmnIiZSkUKADg4NGXDV4SMEDKs6GtNN1vZL3Yng7/W6dFJizKw5sVYP9ft0GtpSuE6Og7sn8+FCEq1OaOdJWUIzWHdgQnbOQyJf61zH+v09blAhbD00syic= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=gourry.net; spf=pass smtp.mailfrom=gourry.net; dkim=pass (2048-bit key) header.d=gourry.net header.i=@gourry.net header.b=E7xxucZN; arc=none smtp.client-ip=209.85.222.175 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=gourry.net Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gourry.net Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gourry.net header.i=@gourry.net header.b="E7xxucZN" Received: by mail-qk1-f175.google.com with SMTP id af79cd13be357-930c0f9c1b1so467427885a.1 for ; Wed, 14 Oct 2026 11:30:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gourry.net; s=google; t=1792002641; x=1792607441; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=Ge1qlR5ZIuijZyTa8DlHKLyf7oJA6xVZ0symgBKYKis=; b=E7xxucZNpzum1LJ+YZDYKyXTjzcp/FVh2z2rSjpacERPCB7jwB0l0YylWYLQ0AOEuq wwVMFJDMzacVS6HwObi4qZLG/RdNIIwGzeDmOF+IlLZd+ZKkHN9yeExYJC9Dz2B8NW/e +zmi+vG6x0q/cWUFxndppdUr6zUQy6xTX0gkCRTvRT8amvetpmo4ttgU0Lx96CRZQh/A qw/m9lHg2evn54jnalGxrjIOBIAdUdpnAj+gYDE22XiiSkYwIDIjIbYhQtk3GtXGjZGo occ46aT4WowvMeS2QIeAiSW3vQG5HA6s8V7EQosiqd26HLR8dv8NYobqYSnCb2SmN2FS PZEg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20260707; t=1792002641; x=1792607441; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=Ge1qlR5ZIuijZyTa8DlHKLyf7oJA6xVZ0symgBKYKis=; b=cFIcK6jbsSdyuRv4raejcsu47VMJdnIO+/HSmyJprpoWXSHvbYK5/lLKgQJtaSXca8 BL727JOLF2ZmDRHpdHYVMFjgzAKHmVCfyg0ra4nSjpWVCTQIt2wTDhueU2OpSw5OdVYZ Q9TI6gx2NrYanAPAZEvwFMtQMwQcUTk/7kv7R8Bneuq0B+ZL++Ovp5pM4f3M8yWKtFHb 5CPFWf3GnCvA5I/wQRjXPF5ksPOECuJzA0dfOxe5bRL9eLG+4j4QuDekMTB4u1SOLHwB S7T2cicz7hFNpGKYvuFTQqnJX2EVexNNd1SnAehqdXNMH552PQrB4PHauATW6Ugm98Lj 8bAA== X-Gm-Message-State: AFq9FYIRHzUUwh8tnCLKGOB4Z2fZEh21Z8vxliA5YX6YCSFd3BTom/RA PPNJJiuagEZ8XPWc0fK01LoHq488+Is9hvPdjfEXFw5zX+JkyeEirRBo5CCO5EEVFR4= X-Gm-Gg: AYBFou1jdAZjsYFFxYwYmh1FxShnPk7qo5lcCEsudEniuLTZwV67F7BDHSEfDfkt5y5 zxXvor4firiibe6vSzxErow0JFvnh63Ig+fP/UkgrbS4+dqk5k5f7M7NKqVznB8DyfJmRlfX8OC xP/ikQ9Pr1IKGlcMsShgElg/m6aQkcsqzT95lFUuCEoffwtGPWjvcqsMLhij69apnEXJdIATbkv fQxFKIHIifmT3dCxBJ3baDwPmITErmxcQWku4tjcUD35S6RgdZz0Buw/a5/6glkCYE9h+EaPmYx wNj3iJRvtRnqXWbImjb5Ij4ldWkVZMOtyfXoaSRJlhBg8nLvgPSs3f2DkQd6mOdIEpZ10qOMiLt ReLHmVKvktV/GnB12izaR8ghFYNK75e2mXPpFmV18vjhhugI/Ol2NiSewEf2+ENYBPBSLLKN5a0 kk4e1ujYvvGpmwClY+bu/cDXOLvL1ehmXBykCiy8FUm+jAzijjpNuzm7cxZFmasKVbEfG4CpXOa l/YApqBuxudh9Rhpg48Bm5v4DhNLysvHWzgZewkN0mY620k7HIxjWlAozPt X-Received: by 2002:a05:620a:3942:b0:93e:e531:b78d with SMTP id af79cd13be357-93ee531ba1cmr1506123785a.7.1792002641364; Wed, 14 Oct 2026 11:30:41 -0700 (PDT) Received: from gourry-fedora-PF4VCD3F.lan (pool-173-79-60-52.washdc.fios.verizon.net. [173.79.60.52]) by smtp.gmail.com with ESMTPSA id af79cd13be357-93ef508811esm556593085a.36.2026.10.14.11.30.40 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 14 Oct 2026 11:30:40 -0700 (PDT) From: "Gregory Price (Meta)" To: linux-mm@kvack.org Cc: linux-kernel@vger.kernel.org, kernel-team@meta.com, akpm@linux-foundation.org, liam@infradead.org, ljs@kernel.org, david@kernel.org, vbabka@kernel.org, jannh@google.com Subject: [PATCH v2 08/10] mm/madvise: separate the PTL-held PTE scan Date: Wed, 14 Oct 2026 14:30:22 -0400 Message-ID: <20261014183024.2075709-9-gourry@gourry.net> X-Mailer: git-send-email 2.56.0 In-Reply-To: <20261014183024.2075709-1-gourry@gourry.net> References: <20261014183024.2075709-1-gourry@gourry.net> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" The PTE loop's scan is broken up by the two places that drop the PTE lock: the periodic reschedule and the large-folio split. Move the scan into madvise_lru_pte_range_locked(), which runs entirely under the lock and stops either at a folio to split or at the reschedule point. Both lock drops now happen in madvise_lru_pmd_entry(), which restarts the scan where the helper stopped, or past the folio's PTE batch if splitting it failed. No functional change intended. Signed-off-by: Gregory Price (Meta) --- mm/madvise.c | 67 ++++++++++++++++++++++++++++++++-------------------- 1 file changed, 42 insertions(+), 25 deletions(-) diff --git a/mm/madvise.c b/mm/madvise.c index 0c0a840102eb..3ebc14d30e0b 100644 --- a/mm/madvise.c +++ b/mm/madvise.c @@ -565,6 +565,33 @@ static bool madvise_lru_huge_pmd(pmd_t *pmd, unsigned = long addr, } #endif =20 +/* + * Scan PTEs under the PTE lock until a folio needs splitting, and return = it + * locked and referenced. Return NULL at the end of the range, or earlier + * when the caller should reschedule. *addr is left at the PTE where the + * scan stopped. + */ +static struct folio *madvise_lru_pte_range_locked(pte_t *pte, + unsigned long *addr, unsigned long end, struct mm_walk *walk, + struct list_head *folio_list, bool pageout_anon_only, int *nr, + unsigned int *batch_count) +{ + struct folio *folio; + + for (; *addr < end; pte +=3D *nr, *addr +=3D *nr * PAGE_SIZE) { + if (++(*batch_count) =3D=3D SWAP_CLUSTER_MAX) { + *batch_count =3D 0; + if (need_resched()) + return NULL; + } + folio =3D madvise_lru_pte_batch_locked(pte, *addr, end, walk, + folio_list, pageout_anon_only, nr); + if (folio) + return folio; + } + return NULL; +} + static int madvise_lru_pmd_entry(pmd_t *pmd, unsigned long addr, unsigned long end, struct mm_walk *walk) { @@ -596,34 +623,24 @@ static int madvise_lru_pmd_entry(pmd_t *pmd, unsigned= long addr, goto out; flush_tlb_batched_pending(mm); lazy_mmu_mode_enable(); - for (; addr < end; pte +=3D nr, addr +=3D nr * PAGE_SIZE) { - if (++batch_count =3D=3D SWAP_CLUSTER_MAX) { - batch_count =3D 0; - if (need_resched()) { - lazy_mmu_mode_disable(); - pte_unmap_unlock(start_pte, ptl); - cond_resched(); - goto restart; - } - } - - folio =3D madvise_lru_pte_batch_locked(pte, addr, end, walk, - &folio_list, pageout_anon_only, &nr); - if (!folio) - continue; - + folio =3D madvise_lru_pte_range_locked(pte, &addr, end, walk, + &folio_list, pageout_anon_only, &nr, &batch_count); + if (!folio && addr < end) { lazy_mmu_mode_disable(); pte_unmap_unlock(start_pte, ptl); - start_pte =3D NULL; - if (!madvise_lru_split_folio(folio)) - nr =3D 0; - start_pte =3D pte_offset_map_lock(mm, pmd, addr, &ptl); - if (!start_pte) - break; - pte =3D start_pte; - flush_tlb_batched_pending(mm); - lazy_mmu_mode_enable(); + cond_resched(); + goto restart; } + if (!folio) + goto out; + + lazy_mmu_mode_disable(); + pte_unmap_unlock(start_pte, ptl); + start_pte =3D NULL; + if (!madvise_lru_split_folio(folio)) + nr =3D 0; + addr +=3D nr * PAGE_SIZE; + goto restart; =20 out: if (start_pte) { --=20 2.56.0 From nobody Sat Nov 21 06:18:45 2026 Received: from mail-qv1-f46.google.com (mail-qv1-f46.google.com [209.85.219.46]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0C49F56C64E for ; Wed, 14 Oct 2026 18:30:43 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.219.46 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1792002646; cv=none; b=a45QMrj2oCjxyWIwm5EFs1oznLVSQmvoKf2jN8QxVpngYXDbb+z28snssv/iZmU4rj+lI+bx3ngnNVakKf56bHH2MSzvVb/UvTW7YzeurHZBdysIHIJUXz7vdMdJnhnszq1OCHonQBN5zCeaxHObr2ZFkZaHrWhNsKsJTAxQrnM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1792002646; c=relaxed/simple; bh=+FE8bndYcmaIs/JKxn2Sau5uH/cgn+GtkHp3S1pwk0Q=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=QJioTFyD/+kFybcbUo4HgvQ9/PsDP2erdWSy0SFKmeGFX2rT67f1vrAZ0OyuPAwSQam+3ZC9bK0q+ZmfnsXhO1IzCVquUC12L80yHyZiqpMRDuOsWTkhscVUlqIeCbYKmtx/3+yObOBhdMWCX2r/MShPY9yhuOgL8s5FJzB7ztM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=gourry.net; spf=pass smtp.mailfrom=gourry.net; dkim=pass (2048-bit key) header.d=gourry.net header.i=@gourry.net header.b=UlyO7yIS; arc=none smtp.client-ip=209.85.219.46 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=gourry.net Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gourry.net Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gourry.net header.i=@gourry.net header.b="UlyO7yIS" Received: by mail-qv1-f46.google.com with SMTP id 6a1803df08f44-91ba702cceaso7797306d6.0 for ; Wed, 14 Oct 2026 11:30:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gourry.net; s=google; t=1792002643; x=1792607443; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=z27Ij9K379BsUAXeMo35PAyhHA0HasTqw8AxKpcrJV4=; b=UlyO7yIS+w2tVam5T2O3NlmbvfcyavkVb/uQlnRsDNt+4IsLcD1NbnX92ddLpo1jhf VUfE/tB1GlDe+SjpBDCAD0uvCx+8397XVmKTf4IMAXW37ezsgbvA8/OmWt1YwzHPGQJY s1/2Qv7Qzkj62fbjtAMiP6Z8Wj5phqydFotZOgUXcrsS3RLBfOSoi3jMK2PQfD+ib/7a WJEEy14KvP/Rce/8IQ0nX0O+u7JMoS5jpuFqPxawLAYHRzqL4S4QNjQxRSN14j0KbWQ8 tbVwmgEBbixM52HNfz6wzyHGKWLJefgaX9EGa3n0gu1XbMqZRFHu3SYgym6arCbN8A2R 9wtw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20260707; t=1792002643; x=1792607443; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=z27Ij9K379BsUAXeMo35PAyhHA0HasTqw8AxKpcrJV4=; b=rvVJU+jEud3xqUqS1VuqjbYJhoMqCCNp6YK7o6NDwf0MyulV7Wo6Zsm+Grp3ix5neG YIlyi3NWV0unIN/emVbzIYvvk2SjD9A/vqFbCmbb5d0zLvkve1MKPC+Ivgyi1XC5Ek02 ZWXsAgvMs42JZnEs2W4xk+7+1kaVvK78xdmSCZ/Pvp+IejqjeLiV1sCIFVjA+sR3dh70 8dmvLYk9JlL7WtrBVmxM6SlWBx+IzDPIqnOhYJ2Vy5BrL0N3xSqPf8uFYO/cBcXQcpjf NZJOQ8y/MXnpcOjtZe3UqQFPKT65xTu6c1lAvJ8Ln0BoEPChjInzmQduvqD2YgBD7tYN LN4A== X-Gm-Message-State: AFq9FYKyh6qR2TyDwbrRFisDY9rnQkOg14WfgV1egW8CvbV5ezS9VojO qgV+U+xL1nH6AQWV9X1xNWbMLWxuYUsgsfQBabHo189HoeteQhP41VK//VJbygq9+lcUwwlm3VY b3k0ASw8= X-Gm-Gg: AYBFou0+vEJmNBHFI090ZmBXDpTfcp4mS/7IjM1pqbbc5W2kXmoknQxeazLfsFEjzHc AOPosKVSS7zIf69CdDHOOXkL2tGoJJUEQhZyekbg6Hsa6anugEEkD3n5XnhYVyybAjrfooBYjS+ o05pAntBj2Q7ehFeCn2Nny8ANONJt9cP71cj/z4AsYraWfg3bGofYwoKdxaU2yz5g5vGSiJ+FBE YHymvlGe6Erq3YUEp/JGu9rY2QlaC8oaKtOQAsDDI+8qXSV0/GXMSfXQRgRNyLtC/h+H3UlaVpf TV58f3klMbSU4jgsNDS+JqYax6Y+pZPwaAeWyZ4j1dFj/ON308gCJhdpo33XUbGkXkNB0ifZtNY sWfmrz6JA86E4WCfYUj2osctAGDVPAt8YtLz7EjT0tfnBgIDAl2OxyJ+wXDAK4NRebZ2WiqgJ4X Lz9fjpDIMWSY1LIgcq19clhYeE78elsf9AM30Y2aNi0IpOg+QHAKzxpOVsX4arM+qFkMLCCfdgd fDk952+o1O03SpfYZo4MlT0JpHD9LD/beMqK2wzIFmSS9j5Vq6CigUVNWPt X-Received: by 2002:a05:620a:270d:b0:93e:7939:43f0 with SMTP id af79cd13be357-93ebd247f8dmr2807496785a.32.1792002642732; Wed, 14 Oct 2026 11:30:42 -0700 (PDT) Received: from gourry-fedora-PF4VCD3F.lan (pool-173-79-60-52.washdc.fios.verizon.net. [173.79.60.52]) by smtp.gmail.com with ESMTPSA id af79cd13be357-93ef508811esm556593085a.36.2026.10.14.11.30.42 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 14 Oct 2026 11:30:42 -0700 (PDT) From: "Gregory Price (Meta)" To: linux-mm@kvack.org Cc: linux-kernel@vger.kernel.org, kernel-team@meta.com, akpm@linux-foundation.org, liam@infradead.org, ljs@kernel.org, david@kernel.org, vbabka@kernel.org, jannh@google.com Subject: [PATCH v2 09/10] mm/madvise: make cold and pageout PTE lock ownership explicit Date: Wed, 14 Oct 2026 14:30:23 -0400 Message-ID: <20261014183024.2075709-10-gourry@gourry.net> X-Mailer: git-send-email 2.56.0 In-Reply-To: <20261014183024.2075709-1-gourry@gourry.net> References: <20261014183024.2075709-1-gourry@gourry.net> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" madvise_lru_pmd_entry() maps and unmaps the PTE table through a restart label and an out label, so finding where the PTE lock is held means following every goto. Move the PTE path into madvise_lru_pte_range(), whose loop takes the lock at the top of each pass and drops it before splitting or rescheduling. madvise_lru_pmd_entry() now only checks for a fatal signal, computes the anon-only filter and dispatches to the huge-PMD or PTE path. No functional change intended. Signed-off-by: Gregory Price (Meta) --- mm/madvise.c | 91 +++++++++++++++++++++++++--------------------------- 1 file changed, 44 insertions(+), 47 deletions(-) diff --git a/mm/madvise.c b/mm/madvise.c index 3ebc14d30e0b..acfe5b945eea 100644 --- a/mm/madvise.c +++ b/mm/madvise.c @@ -592,64 +592,61 @@ static struct folio *madvise_lru_pte_range_locked(pte= _t *pte, return NULL; } =20 -static int madvise_lru_pmd_entry(pmd_t *pmd, unsigned long addr, - unsigned long end, struct mm_walk *walk) +static void madvise_lru_pte_range(pmd_t *pmd, unsigned long addr, + unsigned long end, struct mm_walk *walk, + bool pageout_anon_only) { - struct madvise_walk_private *private =3D walk->private; - struct mmu_gather *tlb =3D private->tlb; - bool pageout =3D private->pageout; - struct mm_struct *mm =3D tlb->mm; - struct vm_area_struct *vma =3D walk->vma; - pte_t *start_pte, *pte; - spinlock_t *ptl; - struct folio *folio =3D NULL; + const struct madvise_walk_private *private =3D walk->private; + struct mm_struct *mm =3D private->tlb->mm; LIST_HEAD(folio_list); unsigned int batch_count =3D 0; - bool pageout_anon_only; + struct folio *folio; + pte_t *start_pte; + spinlock_t *ptl; int nr; =20 - if (fatal_signal_pending(current)) - return -EINTR; - pageout_anon_only =3D pageout && !vma_is_anonymous(vma) && - !can_do_file_pageout(vma); - - if (pmd_trans_huge(*pmd) && - madvise_lru_huge_pmd(pmd, addr, end, walk, pageout_anon_only)) - return 0; - tlb_change_page_size(tlb, PAGE_SIZE); -restart: - start_pte =3D pte =3D pte_offset_map_lock(vma->vm_mm, pmd, addr, &ptl); - if (!start_pte) - goto out; - flush_tlb_batched_pending(mm); - lazy_mmu_mode_enable(); - folio =3D madvise_lru_pte_range_locked(pte, &addr, end, walk, - &folio_list, pageout_anon_only, &nr, &batch_count); - if (!folio && addr < end) { + tlb_change_page_size(private->tlb, PAGE_SIZE); + while (addr < end) { + start_pte =3D pte_offset_map_lock(mm, pmd, addr, &ptl); + if (!start_pte) + break; + flush_tlb_batched_pending(mm); + lazy_mmu_mode_enable(); + folio =3D madvise_lru_pte_range_locked(start_pte, &addr, end, + walk, &folio_list, pageout_anon_only, &nr, + &batch_count); lazy_mmu_mode_disable(); pte_unmap_unlock(start_pte, ptl); - cond_resched(); - goto restart; - } - if (!folio) - goto out; =20 - lazy_mmu_mode_disable(); - pte_unmap_unlock(start_pte, ptl); - start_pte =3D NULL; - if (!madvise_lru_split_folio(folio)) - nr =3D 0; - addr +=3D nr * PAGE_SIZE; - goto restart; - -out: - if (start_pte) { - lazy_mmu_mode_disable(); - pte_unmap_unlock(start_pte, ptl); + if (folio) { + if (!madvise_lru_split_folio(folio)) + nr =3D 0; + addr +=3D nr * PAGE_SIZE; + } else if (addr < end) { + cond_resched(); + } } - if (pageout) + if (private->pageout) reclaim_pages(&folio_list); cond_resched(); +} + +static int madvise_lru_pmd_entry(pmd_t *pmd, unsigned long addr, + unsigned long next, struct mm_walk *walk) +{ + const struct madvise_walk_private *private =3D walk->private; + struct vm_area_struct *vma =3D walk->vma; + bool pageout_anon_only; + + if (fatal_signal_pending(current)) + return -EINTR; + pageout_anon_only =3D private->pageout && !vma_is_anonymous(vma) && + !can_do_file_pageout(vma); + + if (pmd_trans_huge(*pmd) && + madvise_lru_huge_pmd(pmd, addr, next, walk, pageout_anon_only)) + return 0; + madvise_lru_pte_range(pmd, addr, next, walk, pageout_anon_only); =20 return 0; } --=20 2.56.0 From nobody Sat Nov 21 06:18:45 2026 Received: from mail-qv1-f47.google.com (mail-qv1-f47.google.com [209.85.219.47]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8C8DB56E051 for ; Wed, 14 Oct 2026 18:30:45 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.219.47 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1792002646; cv=none; b=bVRIKupxJ5dar8tNC7JJOn0vij1Rh6ufnbfVlAhic9LnwP+izBS58dkJhY2nrT9IQOUBmHDqTntLWhOoX1POZ5tqSQaPOK10HSlBxomKFih8s2ZMS1n1AzT+iXA0guM4HLIFtmx7H1v0Rab7X3+SvIIW2HEWmr6TnaT1ollcSgU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1792002646; c=relaxed/simple; bh=Efxa/guM4cW6/ip85xAhRThMMbu2SwQy2HYlRJJVP/c=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=oeTWcS6USsR9hSWE1yXwf2PLvGoH1GsImvrINQuv/5FCy6vNCK0UyYEuCswuVrF7areZeMbV1RUboeGceoxKvGJiZsNAgr+ZtrqkYrjSO1wQEKA5fiu3XJ0ETzeHI5suoUe3TkYwY86/2p7A7Xa6fMz4GA0Ty42/SrnU0nV/VZs= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=gourry.net; spf=pass smtp.mailfrom=gourry.net; dkim=pass (2048-bit key) header.d=gourry.net header.i=@gourry.net header.b=sG6CNwQs; arc=none smtp.client-ip=209.85.219.47 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=gourry.net Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gourry.net Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gourry.net header.i=@gourry.net header.b="sG6CNwQs" Received: by mail-qv1-f47.google.com with SMTP id 6a1803df08f44-91b8306b9f8so50390526d6.3 for ; Wed, 14 Oct 2026 11:30:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gourry.net; s=google; t=1792002644; x=1792607444; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=QJo2gAKsEo0t/BTlHKqqnwXUysT3Hn0OcnvXdcWDu6U=; b=sG6CNwQsKvrWJ67b7V2HkUv9eHBBtCalwa9E01zP8qnspu9YbeCo15NLqcOkBVQQZb QMuQbwz5nYwRrZrp2aMGKrilFCyyM5+3+UJ/U3Q9a/X3vcNkZOGpKz2VCnrm2j8rakRN FmfryyE4jwG1aIsHR9XJaU+kawG35Esy/U32MkMRFuCHVD0bqMNDJqf+ikkElkiWByux UigzFAtT11d8m3Q/b13GouFqi2q6TGKuuwL70Ln0QMk43pzYimO4MIU5UE/m/Q8wJ7fM frSaDwmhG6TfNOWBqeAP5xK48X8DWyqCWrn/cl3sEJ3QkkzLHjxO9yz1ConXv267tog1 xp0w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20260707; t=1792002644; x=1792607444; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=QJo2gAKsEo0t/BTlHKqqnwXUysT3Hn0OcnvXdcWDu6U=; b=V+eokREmCSuSNNCLTnoFYwCLkwgrwIn/HISYJwVlEJHxjqHCaCvjZUQyMy0HH5duJW 01d2l00HKYWbZGnBDopQWPZHZGRk7ZIUY9dpFQzN1dGOUx03afCwyiHfrjKRq0vdZctp NVWpFsrrvK2WIQCdQYq0NT8HbVNjSwg8KGkB/ljrRGTSUzNLGQL+QYnNFk8U/ISvw2Et Q0bPj5uV8sdUm4s1+6TDNeBW8sMUT/I1OoSDpsiJP9FGMaMxsBun4xTuShRWeoebrEgo VPtjLvmWEl9xZnci5gr7YxBLLu0RO9ERX2S2NWoVZ9EAoWG6gu3oMSHrwNzi3+hJIv89 iIQg== X-Gm-Message-State: AFq9FYK68JlENRQzvEBcnpxlj4nHasg5OJtz8ZciYQtCVFV6IzCQsEd/ ChMEK4Q2weG3Ztl/awPWW1ApMI+e0MKFRi7i4DnCvTwO8+c+X84sLJUqw3D6wrxV55I= X-Gm-Gg: AYBFou0bgp7oIuZlhA9Td4XbPPn9E/eCGdyR1uUNXLwEiglwX5KefCd9JVMOAW9KZmk rn0iPvF+Vt7vc6nl0ZEy6+hfcL+cyS3Y6ISmOniWV/dMiVQXWy0Bx976JryUBr1rZqOPtlxD3BS pTmEFoInTMWyjt7M87drcmQ2ESBoD2wdEo8ANRwjeLklpBkKSB0YQQ+KvSximaJWshwZOMrlOLq uxRr5+Qev7Pn5sO6/MM0GevTrcGO4PMujnowaLXaF8OfMuLTQWo14oiBNxbem6SwHRmnb3h3O78 +q7kduuLSxWpoUJXdA6e0U9ePe+GWU7K4jKlPjM75R4cEsDuW85fs6Lktx/1BzVBO5cCb5boD89 nmKSMKCujFZ2X0sMXBcosyPWG3R17B3BUnLxeqV8BwElQ40wzt4AthIbq5sQioFJwKBO8SDeZS+ W1CCy+KfX76pgE/eqBYaGzT3mXx8oKCMlKKssAIC6k8H+8/2Itay24elnO/4y+lMoFkjFMierVX dLQwnU0Vb+ytprmOBV62TixfHY70W8zBEHRXYx+mMRABlRshj80QsJO6syekA== X-Received: by 2002:a05:620a:6493:b0:93e:60e8:c9ad with SMTP id af79cd13be357-93ebd09e6cbmr2693181685a.12.1792002644454; Wed, 14 Oct 2026 11:30:44 -0700 (PDT) Received: from gourry-fedora-PF4VCD3F.lan (pool-173-79-60-52.washdc.fios.verizon.net. [173.79.60.52]) by smtp.gmail.com with ESMTPSA id af79cd13be357-93ef508811esm556593085a.36.2026.10.14.11.30.43 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 14 Oct 2026 11:30:44 -0700 (PDT) From: "Gregory Price (Meta)" To: linux-mm@kvack.org Cc: linux-kernel@vger.kernel.org, kernel-team@meta.com, akpm@linux-foundation.org, liam@infradead.org, ljs@kernel.org, david@kernel.org, vbabka@kernel.org, jannh@google.com Subject: [PATCH v2 10/10] mm/madvise: share cold and pageout walk setup Date: Wed, 14 Oct 2026 14:30:24 -0400 Message-ID: <20261014183024.2075709-11-gourry@gourry.net> X-Mailer: git-send-email 2.56.0 In-Reply-To: <20261014183024.2075709-1-gourry@gourry.net> References: <20261014183024.2075709-1-gourry@gourry.net> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" madvise_cold() and madvise_pageout() each open-code the same mmu_gather and page-walk sequence, differing only in the pageout flag they pass to the walk. Move that sequence into madvise_lru_vma_range(), which takes the flag. The walk now gathers into madv_behavior->tlb instead of an mmu_gather on each caller's stack. madvise_init_tlb() and madvise_finish_tlb() leave that mmu_gather untouched for both behaviors, since neither batches TLB flushes across VMAs. No functional change intended. Signed-off-by: Gregory Price (Meta) --- mm/madvise.c | 37 +++++++++---------------------------- 1 file changed, 9 insertions(+), 28 deletions(-) diff --git a/mm/madvise.c b/mm/madvise.c index acfe5b945eea..db4f498840eb 100644 --- a/mm/madvise.c +++ b/mm/madvise.c @@ -656,21 +656,23 @@ static const struct mm_walk_ops cold_walk_ops =3D { .walk_lock =3D PGWALK_RDLOCK, }; =20 -static void madvise_cold_page_range(struct mmu_gather *tlb, - struct madvise_behavior *madv_behavior) - +static void madvise_lru_vma_range(struct madvise_behavior *madv_behavior, + bool pageout) { struct vm_area_struct *vma =3D madv_behavior->vma; struct madvise_behavior_range *range =3D &madv_behavior->range; + struct mmu_gather *tlb =3D madv_behavior->tlb; struct madvise_walk_private walk_private =3D { - .pageout =3D false, + .pageout =3D pageout, .tlb =3D tlb, }; =20 + tlb_gather_mmu(tlb, madv_behavior->mm); tlb_start_vma(tlb, vma); walk_page_range_vma(vma, range->start, range->end, &cold_walk_ops, - &walk_private); + &walk_private); tlb_end_vma(tlb, vma); + tlb_finish_mmu(tlb); } =20 static inline bool can_madv_lru_vma(struct vm_area_struct *vma) @@ -681,37 +683,18 @@ static inline bool can_madv_lru_vma(struct vm_area_st= ruct *vma) static long madvise_cold(struct madvise_behavior *madv_behavior) { struct vm_area_struct *vma =3D madv_behavior->vma; - struct mmu_gather tlb; =20 if (!can_madv_lru_vma(vma)) return -EINVAL; =20 lru_add_drain(); - tlb_gather_mmu(&tlb, madv_behavior->mm); - madvise_cold_page_range(&tlb, madv_behavior); - tlb_finish_mmu(&tlb); + madvise_lru_vma_range(madv_behavior, false); =20 return 0; } =20 -static void madvise_pageout_page_range(struct mmu_gather *tlb, - struct vm_area_struct *vma, - struct madvise_behavior_range *range) -{ - struct madvise_walk_private walk_private =3D { - .pageout =3D true, - .tlb =3D tlb, - }; - - tlb_start_vma(tlb, vma); - walk_page_range_vma(vma, range->start, range->end, &cold_walk_ops, - &walk_private); - tlb_end_vma(tlb, vma); -} - static long madvise_pageout(struct madvise_behavior *madv_behavior) { - struct mmu_gather tlb; struct vm_area_struct *vma =3D madv_behavior->vma; =20 if (!can_madv_lru_vma(vma)) @@ -728,9 +711,7 @@ static long madvise_pageout(struct madvise_behavior *ma= dv_behavior) return 0; =20 lru_add_drain(); - tlb_gather_mmu(&tlb, madv_behavior->mm); - madvise_pageout_page_range(&tlb, vma, &madv_behavior->range); - tlb_finish_mmu(&tlb); + madvise_lru_vma_range(madv_behavior, true); =20 return 0; } --=20 2.56.0