From nobody Thu Oct 2 07:48:27 2025 Received: from fhigh-a4-smtp.messagingengine.com (fhigh-a4-smtp.messagingengine.com [103.168.172.155]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B4A922066DE for ; Fri, 19 Sep 2025 12:40:46 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=103.168.172.155 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1758285648; cv=none; b=lXkqSGkAGsnFc6yMMvlrlYmOZVCkHBVoDBN7azG3a20c1fM+Wy+6VM61o7Fjc0Mq+Y7lAp7IutP6myWTjOEJuhNxD0ajKyRE6+ejxm+G+nTWV5geRm2SdoEgs6c5jJstcuCoy+9PJhbQHf+rubGXCJrRh0yL2gD8lVo0lH7seuA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1758285648; c=relaxed/simple; bh=k/heMrK8t0hzL7/yyTezvp5EK/48PVIsBkKMmC1F0SA=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=O6EyPzgEw5J5L9RxUNOz71MfKVsCNk/CNJ9jw3Cq6bv8NZPJjdt2Vr5EJgOWIiOgRKpdmXFJ1Y3uuFpik8Es0xCbCmZzJK/kF2hA8ubiHeY3rL550BgvyVT1KYpoJRyJSrcBjhVfoB+/MyJ+kFmQA+kJe78exyirv94pa6UDlH0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=shutemov.name; spf=pass smtp.mailfrom=shutemov.name; dkim=pass (2048-bit key) header.d=shutemov.name header.i=@shutemov.name header.b=obLAWdcH; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b=n8+eomSU; arc=none smtp.client-ip=103.168.172.155 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=shutemov.name Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=shutemov.name Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=shutemov.name header.i=@shutemov.name header.b="obLAWdcH"; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b="n8+eomSU" Received: from phl-compute-09.internal (phl-compute-09.internal [10.202.2.49]) by mailfhigh.phl.internal (Postfix) with ESMTP id B76011400167; Fri, 19 Sep 2025 08:40:45 -0400 (EDT) Received: from phl-mailfrontend-01 ([10.202.2.162]) by phl-compute-09.internal (MEProxy); Fri, 19 Sep 2025 08:40:45 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=shutemov.name; h=cc:cc:content-transfer-encoding:content-type:date:date:from :from:in-reply-to:in-reply-to:message-id:mime-version:references :reply-to:subject:subject:to:to; s=fm3; t=1758285645; x= 1758372045; bh=wuvDNaiAvAvSGeMLUplFIMVyMw23wgCwW9wv1JnwK0k=; b=o bLAWdcHu6MBmN0v7b8k006MBalYYsTHkKHHEyNeUVIWS4zDmTG5RsWD8Eu4st2Ep x72rwYjx8/VZSKfPMntKi/yCB9kcpj2DA8YiSK9DEnlZNX63ZM71YCGeT0/Og8R9 +dBNsiy+urY11QyDWDDGMJHKSjB01/d2dz0mBn8OJCE80Sg8svPjocsh+M2VkpNr JHcup192PXiAwE4DQLbKbqtKWrrg+RedH6NkchsiXZK1O4LeVZxf/ylDlyNsOdy6 yqj3ASnHOfgO6Ifs2HU43VngeJ3qsVJkVxM8LyYQ+CeQIg5Sknwes9/9urg+U7XM CFoGvWUtYJC7I4nDwE07Q== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding :content-type:date:date:feedback-id:feedback-id:from:from :in-reply-to:in-reply-to:message-id:mime-version:references :reply-to:subject:subject:to:to:x-me-proxy:x-me-sender :x-me-sender:x-sasl-enc; s=fm1; t=1758285645; x=1758372045; bh=w uvDNaiAvAvSGeMLUplFIMVyMw23wgCwW9wv1JnwK0k=; b=n8+eomSUAJxKDuHxM 4xsvKPdkstKitVeOM0yyacKzVbSCU0U4yq/NsCsqYSC8VhwTKAT/xnXpA2jt6igB +vPpQE41onWAX3881u4ytmZwOyfNoqQz6drMzZ7bag34xB5BT9vuf2iCknQElzUM GyTA9Up8PEukDhJU7Zp4WfDKgjSp/RRn7TEWReABsMXe0ck5ALgsTvhPti2cI4Wm i3Lg2mXCuiN5EjJO9kpGR9obr6tR0PLEvyw1IwcoPvUhkcPW3XGpQvav7TKui3Jw SO1pqm4NfW9FlkvyaRykShUN8NQYYybdqY+BhDpwm401EWRLrIp38DV5OAwqtEDt CJHvA== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeeffedrtdeggdegledvfecutefuodetggdotefrod ftvfcurfhrohhfihhlvgemucfhrghsthforghilhdpuffrtefokffrpgfnqfghnecuuegr ihhlohhuthemuceftddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmdenucfjug hrpefhvfevufffkffojghfggfgsedtkeertdertddtnecuhfhrohhmpefmihhrhihlucfu hhhuthhsvghmrghuuceokhhirhhilhhlsehshhhuthgvmhhovhdrnhgrmhgvqeenucggtf frrghtthgvrhhnpeegveehtdfgvdfhudegffeuuddvgeevjefhveevgefhvdevieevteei vdehjefhjeenucevlhhushhtvghrufhiiigvpedtnecurfgrrhgrmhepmhgrihhlfhhroh hmpehkihhrihhllhesshhhuhhtvghmohhvrdhnrghmvgdpnhgspghrtghpthhtohepudek pdhmohguvgepshhmthhpohhuthdprhgtphhtthhopegrkhhpmheslhhinhhugidqfhhouh hnuggrthhiohhnrdhorhhgpdhrtghpthhtohepuggrvhhiugesrhgvughhrghtrdgtohhm pdhrtghpthhtohephhhughhhugesghhoohhglhgvrdgtohhmpdhrtghpthhtohepfihilh hlhiesihhnfhhrrgguvggrugdrohhrghdprhgtphhtthhopehlohhrvghniihordhsthho rghkvghssehorhgrtghlvgdrtghomhdprhgtphhtthhopehlihgrmhdrhhhofihlvghtth esohhrrggtlhgvrdgtohhmpdhrtghpthhtohepvhgsrggskhgrsehsuhhsvgdrtgiipdhr tghpthhtoheprhhpphhtsehkvghrnhgvlhdrohhrghdprhgtphhtthhopehsuhhrvghnsg esghhoohhglhgvrdgtohhm X-ME-Proxy: Feedback-ID: ie3994620:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Fri, 19 Sep 2025 08:40:44 -0400 (EDT) From: Kiryl Shutsemau To: Andrew Morton , David Hildenbrand , Hugh Dickins , Matthew Wilcox Cc: Lorenzo Stoakes , "Liam R. Howlett" , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Rik van Riel , Harry Yoo , Johannes Weiner , Shakeel Butt , Baolin Wang , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Kiryl Shutsemau Subject: [PATCHv2 1/5] mm/page_vma_mapped: Track if the page is mapped across page table boundary Date: Fri, 19 Sep 2025 13:40:32 +0100 Message-ID: <20250919124036.455709-2-kirill@shutemov.name> X-Mailer: git-send-email 2.50.1 In-Reply-To: <20250919124036.455709-1-kirill@shutemov.name> References: <20250919124036.455709-1-kirill@shutemov.name> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Kiryl Shutsemau Add a PVMW_PGTABLE_CROSSSED flag that page_vma_mapped_walk() will set if the page is mapped across page table boundary. Unlike other PVMW_* flags, this one is result of page_vma_mapped_walk() and not set by the caller. folio_referenced_one() will use it detect if it safe to mlock the folio. Signed-off-by: Kiryl Shutsemau Reviewed-by: Shakeel Butt --- include/linux/rmap.h | 5 +++++ mm/page_vma_mapped.c | 1 + 2 files changed, 6 insertions(+) diff --git a/include/linux/rmap.h b/include/linux/rmap.h index 6cd020eea37a..04797cea3205 100644 --- a/include/linux/rmap.h +++ b/include/linux/rmap.h @@ -928,6 +928,11 @@ struct page *make_device_exclusive(struct mm_struct *m= m, unsigned long addr, /* Look for migration entries rather than present PTEs */ #define PVMW_MIGRATION (1 << 1) =20 +/* Result flags */ + +/* The page is mapped across page boundary */ +#define PVMW_PGTABLE_CROSSSED (1 << 16) + struct page_vma_mapped_walk { unsigned long pfn; unsigned long nr_pages; diff --git a/mm/page_vma_mapped.c b/mm/page_vma_mapped.c index e981a1a292d2..a184b88743c3 100644 --- a/mm/page_vma_mapped.c +++ b/mm/page_vma_mapped.c @@ -309,6 +309,7 @@ bool page_vma_mapped_walk(struct page_vma_mapped_walk *= pvmw) } pte_unmap(pvmw->pte); pvmw->pte =3D NULL; + pvmw->flags |=3D PVMW_PGTABLE_CROSSSED; goto restart; } pvmw->pte++; --=20 2.50.1 From nobody Thu Oct 2 07:48:27 2025 Received: from fout-a5-smtp.messagingengine.com (fout-a5-smtp.messagingengine.com [103.168.172.148]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 62EE8208961 for ; Fri, 19 Sep 2025 12:40:48 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=103.168.172.148 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1758285650; cv=none; b=a5J93dbBd8XqwDDgSvzznlPOBxtsrzw7R1QjstChsvz7zI5Gti2+nz6eu8zUa7Sxuj6Of1OSRz/29kn/6idv9kqEs5goIy1n4B/1u1oGgMFxEXMI5jPtf3E5B0q175U1IAFsOokXCfQngEs0TCDI8Sa9a6QtEUGZO25s/zehRBQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1758285650; c=relaxed/simple; bh=GSkyuQksWqUgGXpyT/BplWH9AtYMLjAZ8kH9P7wgzKI=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=Pzijil6A7kLS8OzCi5lJ4vnKDfpseoZHC4gqX1jwhceZupwn8Ve3PgTMA4S/Nc04ZW9BMtzmnTkm8LqskAqwn06bOcM3r15SQlOdWyWACaWW6sSQpD+9Gp01ZSDlqSl82BpC6R+mSD14vbbzSSdEOVGOyJps0yVprJDfeWQLQPE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=shutemov.name; spf=pass smtp.mailfrom=shutemov.name; dkim=pass (2048-bit key) header.d=shutemov.name header.i=@shutemov.name header.b=ijmm53xz; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b=i/VJuaJG; arc=none smtp.client-ip=103.168.172.148 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=shutemov.name Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=shutemov.name Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=shutemov.name header.i=@shutemov.name header.b="ijmm53xz"; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b="i/VJuaJG" Received: from phl-compute-04.internal (phl-compute-04.internal [10.202.2.44]) by mailfout.phl.internal (Postfix) with ESMTP id 5ED21EC02E4; Fri, 19 Sep 2025 08:40:47 -0400 (EDT) Received: from phl-mailfrontend-02 ([10.202.2.163]) by phl-compute-04.internal (MEProxy); Fri, 19 Sep 2025 08:40:47 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=shutemov.name; h=cc:cc:content-transfer-encoding:content-type:date:date:from :from:in-reply-to:in-reply-to:message-id:mime-version:references :reply-to:subject:subject:to:to; s=fm3; t=1758285647; x= 1758372047; bh=KaO9m7USZDrpygot74UDSvojuR6ldLnT3ppbJu8X6HA=; b=i jmm53xzPDAUdcPUioFKoDAmGFp4tcdczwXpAyJ9iWew3LsPRnCro82mtzbqx68hz 36Hf1upGlF8e00jOhez2AHxsaykCZXzjNFNq9XkBgWbtROd5WU2fQBl0fVBWA+0C VaZott2TwvoHzBMTQmbMSGDnEMFrxFEZFhHrZU765KTHMUMI0BfZo5S4RnuhQcRm JH0hxVhobBXhnC5jbX9pW4579cx4SeMyba4uAnWIOS3tEHLZifNnDCxLc26mDMvg XXiaDzyR1VRuEm9rHgriSVAp3tUrFRWQCefFUN7D7nWkb6roClcURlK53LoShdwP JzwZnMxZeQ/6/JnYfD99A== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding :content-type:date:date:feedback-id:feedback-id:from:from :in-reply-to:in-reply-to:message-id:mime-version:references :reply-to:subject:subject:to:to:x-me-proxy:x-me-sender :x-me-sender:x-sasl-enc; s=fm1; t=1758285647; x=1758372047; bh=K aO9m7USZDrpygot74UDSvojuR6ldLnT3ppbJu8X6HA=; b=i/VJuaJGPik9isONQ Jrsb97nBw8WpsN2BNFsuFvX9e2k7BD0ZlYRMB/BfN5xScBHtQ6Y5OeZLsDLSyw6D UBzKy6GiVc1swjsVen1/3ztwzZwNqGNf4CLQShjVufqKw6lFX9iepMHurL2EH17k nrym6YJuL4i9cMlVai5+w7WsUBp4pEsngL0tPFCWTPJzXwTXwgB0VnB3ycZzyoSp ehcpM1RRXNLbg3ZNN8Zhmb5BhTOilxGS+5RjQ+Qvtb87URO2qVa+HF3qBPmzL6aM pNoaxj+hk1aZcg/2e+ojbPD37iRKW2Rtb+kx2D8y1g6rqvqr8Lsyy/2ohnfFqyka dVOGA== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeeffedrtdeggdegledvfecutefuodetggdotefrod ftvfcurfhrohhfihhlvgemucfhrghsthforghilhdpuffrtefokffrpgfnqfghnecuuegr ihhlohhuthemuceftddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmdenucfjug hrpefhvfevufffkffojghfggfgsedtkeertdertddtnecuhfhrohhmpefmihhrhihlucfu hhhuthhsvghmrghuuceokhhirhhilhhlsehshhhuthgvmhhovhdrnhgrmhgvqeenucggtf frrghtthgvrhhnpeegveehtdfgvdfhudegffeuuddvgeevjefhveevgefhvdevieevteei vdehjefhjeenucevlhhushhtvghrufhiiigvpedtnecurfgrrhgrmhepmhgrihhlfhhroh hmpehkihhrihhllhesshhhuhhtvghmohhvrdhnrghmvgdpnhgspghrtghpthhtohepudek pdhmohguvgepshhmthhpohhuthdprhgtphhtthhopegrkhhpmheslhhinhhugidqfhhouh hnuggrthhiohhnrdhorhhgpdhrtghpthhtohepuggrvhhiugesrhgvughhrghtrdgtohhm pdhrtghpthhtohephhhughhhugesghhoohhglhgvrdgtohhmpdhrtghpthhtohepfihilh hlhiesihhnfhhrrgguvggrugdrohhrghdprhgtphhtthhopehlohhrvghniihordhsthho rghkvghssehorhgrtghlvgdrtghomhdprhgtphhtthhopehlihgrmhdrhhhofihlvghtth esohhrrggtlhgvrdgtohhmpdhrtghpthhtohepvhgsrggskhgrsehsuhhsvgdrtgiipdhr tghpthhtoheprhhpphhtsehkvghrnhgvlhdrohhrghdprhgtphhtthhopehsuhhrvghnsg esghhoohhglhgvrdgtohhm X-ME-Proxy: Feedback-ID: ie3994620:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Fri, 19 Sep 2025 08:40:46 -0400 (EDT) From: Kiryl Shutsemau To: Andrew Morton , David Hildenbrand , Hugh Dickins , Matthew Wilcox Cc: Lorenzo Stoakes , "Liam R. Howlett" , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Rik van Riel , Harry Yoo , Johannes Weiner , Shakeel Butt , Baolin Wang , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Kiryl Shutsemau Subject: [PATCHv2 2/5] mm/rmap: Fix a mlock race condition in folio_referenced_one() Date: Fri, 19 Sep 2025 13:40:33 +0100 Message-ID: <20250919124036.455709-3-kirill@shutemov.name> X-Mailer: git-send-email 2.50.1 In-Reply-To: <20250919124036.455709-1-kirill@shutemov.name> References: <20250919124036.455709-1-kirill@shutemov.name> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Kiryl Shutsemau The mlock_vma_folio() function requires the page table lock to be held in order to safely mlock the folio. However, folio_referenced_one() mlocks a large folios outside of the page_vma_mapped_walk() loop where the page table lock has already been dropped. Rework the mlock logic to use the same code path inside the loop for both large and small folios. Use PVMW_PGTABLE_CROSSED to detect when the folio is mapped across a page table boundary. Signed-off-by: Kiryl Shutsemau Reviewed-by: Shakeel Butt --- mm/rmap.c | 59 ++++++++++++++++++++----------------------------------- 1 file changed, 21 insertions(+), 38 deletions(-) diff --git a/mm/rmap.c b/mm/rmap.c index 568198e9efc2..3d0235f332de 100644 --- a/mm/rmap.c +++ b/mm/rmap.c @@ -851,34 +851,34 @@ static bool folio_referenced_one(struct folio *folio, { struct folio_referenced_arg *pra =3D arg; DEFINE_FOLIO_VMA_WALK(pvmw, folio, vma, address, 0); - int referenced =3D 0; - unsigned long start =3D address, ptes =3D 0; + int ptes =3D 0, referenced =3D 0; =20 while (page_vma_mapped_walk(&pvmw)) { address =3D pvmw.address; =20 if (vma->vm_flags & VM_LOCKED) { - if (!folio_test_large(folio) || !pvmw.pte) { - /* Restore the mlock which got missed */ - mlock_vma_folio(folio, vma); - page_vma_mapped_walk_done(&pvmw); - pra->vm_flags |=3D VM_LOCKED; - return false; /* To break the loop */ - } - /* - * For large folio fully mapped to VMA, will - * be handled after the pvmw loop. - * - * For large folio cross VMA boundaries, it's - * expected to be picked by page reclaim. But - * should skip reference of pages which are in - * the range of VM_LOCKED vma. As page reclaim - * should just count the reference of pages out - * the range of VM_LOCKED vma. - */ ptes++; pra->mapcount--; - continue; + + /* Only mlock fully mapped pages */ + if (pvmw.pte && ptes !=3D pvmw.nr_pages) + continue; + + /* + * All PTEs must be protected by page table lock in + * order to mlock the page. + * + * If page table boundary has been cross, current ptl + * only protect part of ptes. + */ + if (pvmw.flags & PVMW_PGTABLE_CROSSSED) + continue; + + /* Restore the mlock which got missed */ + mlock_vma_folio(folio, vma); + page_vma_mapped_walk_done(&pvmw); + pra->vm_flags |=3D VM_LOCKED; + return false; /* To break the loop */ } =20 /* @@ -914,23 +914,6 @@ static bool folio_referenced_one(struct folio *folio, pra->mapcount--; } =20 - if ((vma->vm_flags & VM_LOCKED) && - folio_test_large(folio) && - folio_within_vma(folio, vma)) { - unsigned long s_align, e_align; - - s_align =3D ALIGN_DOWN(start, PMD_SIZE); - e_align =3D ALIGN_DOWN(start + folio_size(folio) - 1, PMD_SIZE); - - /* folio doesn't cross page table boundary and fully mapped */ - if ((s_align =3D=3D e_align) && (ptes =3D=3D folio_nr_pages(folio))) { - /* Restore the mlock which got missed */ - mlock_vma_folio(folio, vma); - pra->vm_flags |=3D VM_LOCKED; - return false; /* To break the loop */ - } - } - if (referenced) folio_clear_idle(folio); if (folio_test_clear_young(folio)) --=20 2.50.1 From nobody Thu Oct 2 07:48:27 2025 Received: from fhigh-a4-smtp.messagingengine.com (fhigh-a4-smtp.messagingengine.com [103.168.172.155]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id EC024218596 for ; Fri, 19 Sep 2025 12:40:49 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=103.168.172.155 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1758285651; cv=none; b=dYskwqH7TAiURCLN3mOJqfI6AHfDJZCAgeTNefKtaBTuKYLZCfJtYxSO589cZSXsvzeAf/1XjXR+ZXakblfiP4kxE0mbF/bhCkRBREIy0FL2ddcASfF7f/7OmE0NI7i60i03vDhTzm+JP+V276JcGXZMVU+81AgIkLj3UzdpHCU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1758285651; c=relaxed/simple; bh=z4mJNmjmw5N0EDvGoxq7YE6LN2fEWNHPu/+UWlYFCxo=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=FBFYwD2LojHtBoger+1P9YadE0Xq8ga4ESJB6oh+7vSuJxLR8JpdowMGEBEDumZ+SYho4uLQuTkEcjYYg8ABAbPV5FxmSK8oIkkRwoMO0hWbySnaNxAdDK+fB4rnDjebAIe4FOWAEgMUdY+aGp3ugoO9M3TthiO3Zbe4vKUicXM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=shutemov.name; spf=pass smtp.mailfrom=shutemov.name; dkim=pass (2048-bit key) header.d=shutemov.name header.i=@shutemov.name header.b=NSYViHEW; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b=kDbBi8RO; arc=none smtp.client-ip=103.168.172.155 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=shutemov.name Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=shutemov.name Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=shutemov.name header.i=@shutemov.name header.b="NSYViHEW"; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b="kDbBi8RO" Received: from phl-compute-04.internal (phl-compute-04.internal [10.202.2.44]) by mailfhigh.phl.internal (Postfix) with ESMTP id 1FC241400185; Fri, 19 Sep 2025 08:40:49 -0400 (EDT) Received: from phl-mailfrontend-02 ([10.202.2.163]) by phl-compute-04.internal (MEProxy); Fri, 19 Sep 2025 08:40:49 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=shutemov.name; h=cc:cc:content-transfer-encoding:content-type:date:date:from :from:in-reply-to:in-reply-to:message-id:mime-version:references :reply-to:subject:subject:to:to; s=fm3; t=1758285649; x= 1758372049; bh=zbXQCXRSraPiiKdXSvs7n/9rDBYdGNJLDwAjIbxUIRs=; b=N SYViHEWild38F6WS9INcKPuheF7zhmeVta5B2MH+Fd+EwozMYQm6PWZHXG35G1+m +S52ybxYtD45Hnu1i+BTPyUnqas5Tvf7M8JEkvVL1Ff3SUz2qn/Bahb0dqCpZbE1 sYKDzJlJ2zUUfQCeTpTVOJaY+irQeFVMpbqyVwEfH/MB4BBvzNH+UKmgjIIPDPSH 6lbtRdOJi7B5Q/O44aamoO+/3A4lS4FqtnXMvaR+6lxDdSRC3dKkPtvFlw/7BRYX fSskhHDFupDuGZ+0kkKb+wUnhdonyzT6ktH+dT4juPa17rPxn2z697+ihAg0Y+Bb ZY0e39ykKlHoGZD3n20DA== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding :content-type:date:date:feedback-id:feedback-id:from:from :in-reply-to:in-reply-to:message-id:mime-version:references :reply-to:subject:subject:to:to:x-me-proxy:x-me-sender :x-me-sender:x-sasl-enc; s=fm1; t=1758285649; x=1758372049; bh=z bXQCXRSraPiiKdXSvs7n/9rDBYdGNJLDwAjIbxUIRs=; b=kDbBi8ROkAz0sn539 NhdcLTG+VbQ9pN9Z9Bgf6YgHXcLN8AqtezTZDXrz3R3e94vrfzmJmJm24utk/EDI q51KGt659t2FKcxTy3TVoO8od7xmBQH8IQqiUcdDkbKG8JwYjKClS8ycT7KSxdnu zMLmhcx7nh5BmlXMOl+yrlDzQOTSRwF7B3UynnkjsaNaDgEjemoyV/+RnpuEq1/z P/KErjnLjw1vRNWofg77Afc9LD233PoaUogKpHWLEu+XCxfYN590e7aJyPbOVRfR bT03TDgVRy02KICnyaFMsyZZrSVw0c5iS57ULiQUBhyucEAoJhR/T26aPoy12f+9 VzsvA== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeeffedrtdeggdegledvfecutefuodetggdotefrod ftvfcurfhrohhfihhlvgemucfhrghsthforghilhdpuffrtefokffrpgfnqfghnecuuegr ihhlohhuthemuceftddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmdenucfjug hrpefhvfevufffkffojghfggfgsedtkeertdertddtnecuhfhrohhmpefmihhrhihlucfu hhhuthhsvghmrghuuceokhhirhhilhhlsehshhhuthgvmhhovhdrnhgrmhgvqeenucggtf frrghtthgvrhhnpeegveehtdfgvdfhudegffeuuddvgeevjefhveevgefhvdevieevteei vdehjefhjeenucevlhhushhtvghrufhiiigvpedtnecurfgrrhgrmhepmhgrihhlfhhroh hmpehkihhrihhllhesshhhuhhtvghmohhvrdhnrghmvgdpnhgspghrtghpthhtohepudek pdhmohguvgepshhmthhpohhuthdprhgtphhtthhopegrkhhpmheslhhinhhugidqfhhouh hnuggrthhiohhnrdhorhhgpdhrtghpthhtohepuggrvhhiugesrhgvughhrghtrdgtohhm pdhrtghpthhtohephhhughhhugesghhoohhglhgvrdgtohhmpdhrtghpthhtohepfihilh hlhiesihhnfhhrrgguvggrugdrohhrghdprhgtphhtthhopehlohhrvghniihordhsthho rghkvghssehorhgrtghlvgdrtghomhdprhgtphhtthhopehlihgrmhdrhhhofihlvghtth esohhrrggtlhgvrdgtohhmpdhrtghpthhtohepvhgsrggskhgrsehsuhhsvgdrtgiipdhr tghpthhtoheprhhpphhtsehkvghrnhgvlhdrohhrghdprhgtphhtthhopehsuhhrvghnsg esghhoohhglhgvrdgtohhm X-ME-Proxy: Feedback-ID: ie3994620:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Fri, 19 Sep 2025 08:40:48 -0400 (EDT) From: Kiryl Shutsemau To: Andrew Morton , David Hildenbrand , Hugh Dickins , Matthew Wilcox Cc: Lorenzo Stoakes , "Liam R. Howlett" , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Rik van Riel , Harry Yoo , Johannes Weiner , Shakeel Butt , Baolin Wang , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Kiryl Shutsemau Subject: [PATCHv2 3/5] mm/rmap: mlock large folios in try_to_unmap_one() Date: Fri, 19 Sep 2025 13:40:34 +0100 Message-ID: <20250919124036.455709-4-kirill@shutemov.name> X-Mailer: git-send-email 2.50.1 In-Reply-To: <20250919124036.455709-1-kirill@shutemov.name> References: <20250919124036.455709-1-kirill@shutemov.name> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Kiryl Shutsemau Currently, try_to_unmap_once() only tries to mlock small folios. Use logic similar to folio_referenced_one() to mlock large folios: only do this for fully mapped folios and under page table lock that protects all page table entries. Signed-off-by: Kiryl Shutsemau Reviewed-by: Shakeel Butt --- mm/rmap.c | 23 ++++++++++++++++++++--- 1 file changed, 20 insertions(+), 3 deletions(-) diff --git a/mm/rmap.c b/mm/rmap.c index 3d0235f332de..482e6504fa88 100644 --- a/mm/rmap.c +++ b/mm/rmap.c @@ -1870,6 +1870,7 @@ static bool try_to_unmap_one(struct folio *folio, str= uct vm_area_struct *vma, unsigned long nr_pages =3D 1, end_addr; unsigned long pfn; unsigned long hsz =3D 0; + int ptes =3D 0; =20 /* * When racing against e.g. zap_pte_range() on another cpu, @@ -1910,10 +1911,26 @@ static bool try_to_unmap_one(struct folio *folio, s= truct vm_area_struct *vma, */ if (!(flags & TTU_IGNORE_MLOCK) && (vma->vm_flags & VM_LOCKED)) { + ptes++; + ret =3D false; + + /* Only mlock fully mapped pages */ + if (pvmw.pte && ptes !=3D pvmw.nr_pages) + continue; + + /* + * All PTEs must be protected by page table lock in + * order to mlock the page. + * + * If page table boundary has been cross, current ptl + * only protect part of ptes. + */ + if (pvmw.flags & PVMW_PGTABLE_CROSSSED) + goto walk_done; + /* Restore the mlock which got missed */ - if (!folio_test_large(folio)) - mlock_vma_folio(folio, vma); - goto walk_abort; + mlock_vma_folio(folio, vma); + goto walk_done; } =20 if (!pvmw.pte) { --=20 2.50.1 From nobody Thu Oct 2 07:48:27 2025 Received: from fout-a5-smtp.messagingengine.com (fout-a5-smtp.messagingengine.com [103.168.172.148]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 33D48221FDA for ; Fri, 19 Sep 2025 12:40:52 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=103.168.172.148 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1758285654; cv=none; b=ktMW8///HggXG63AcSEq9vIWWfKdV2Do7xRGF/lafGwsuiZ+0qJ+nZdw1hkgkJXOnyf0KmPEbvpxWRVP3aNI4iCtt0oW/f78RGJNMRJF2/259PzxU9VL2Odb5SAin6efa192QLT4busbNvRwpc61EhFN3L7MYOYQgtMbYnvfhsg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1758285654; c=relaxed/simple; bh=XChvTvuq/7NeECGDeUSWMtBEaCBZNCfi/y1ZkjW4cx4=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=lOWsBCW4eSrEw2SLk4JO/CecGwNHSrcRYtORVhZYkxKOoJORoybUz2fsUctk9n4CAWG6hz1FCi9X2lf5SkDv+gykIXrebAb2vdbmLFzpfsssnoCZFuAJQQvxnAW558dGor3+e5SnpAKmgJH8LLkCbQjRQxuzxeLHkCxhioEgxSM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=shutemov.name; spf=pass smtp.mailfrom=shutemov.name; dkim=pass (2048-bit key) header.d=shutemov.name header.i=@shutemov.name header.b=gYaoqNUW; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b=oBRMyPX6; arc=none smtp.client-ip=103.168.172.148 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=shutemov.name Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=shutemov.name Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=shutemov.name header.i=@shutemov.name header.b="gYaoqNUW"; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b="oBRMyPX6" Received: from phl-compute-03.internal (phl-compute-03.internal [10.202.2.43]) by mailfout.phl.internal (Postfix) with ESMTP id 63487EC02D6; Fri, 19 Sep 2025 08:40:51 -0400 (EDT) Received: from phl-mailfrontend-01 ([10.202.2.162]) by phl-compute-03.internal (MEProxy); Fri, 19 Sep 2025 08:40:51 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=shutemov.name; h=cc:cc:content-transfer-encoding:content-type:date:date:from :from:in-reply-to:in-reply-to:message-id:mime-version:references :reply-to:subject:subject:to:to; s=fm3; t=1758285651; x= 1758372051; bh=TC0qa4dJ8ZfxpD8eOHAvcq6Kh4/gpC+GYroIRc1A+zs=; b=g YaoqNUWaKHhvv3Ed4xHBXRktxSLULfs/iAAuYrd7j55EZwLmi18d89+j4kySwA12 5SyZ2uYXjdwz3FmJ82Kk4p3w2wCOYWDmriHFwCmlw88SQYiMNg73I3OUH5FsYGKo 4qxeC+AqeN71HV3IX9QR7a0wyomCVQhvXau8Gu5fldftzhhA6CetYZelZH8BPMGq EBIYmWRo7FDIkuy8YcbteuqPik/hKtWxRdlsqyYUQ5fNRkfn6uW2wB8K4RWowICL 9WJTQUpVnMwtHUz71tcKCab4m/huYVrtpsMV8gX9HISY15uJj+C+rfv0sCire3yv oiZy9DUWJOjm7E8AwETOQ== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding :content-type:date:date:feedback-id:feedback-id:from:from :in-reply-to:in-reply-to:message-id:mime-version:references :reply-to:subject:subject:to:to:x-me-proxy:x-me-sender :x-me-sender:x-sasl-enc; s=fm1; t=1758285651; x=1758372051; bh=T C0qa4dJ8ZfxpD8eOHAvcq6Kh4/gpC+GYroIRc1A+zs=; b=oBRMyPX6XhIiUnMzG jhA/GKeWruN58gwnj9eQuLD1ahHUAjCToZnr7lsMkryuiYuOpTuc9m1Fq+WwJa66 W2cj6gXLpeu7xZofG855foOYmB0A6U8K8JfoAycT77K2kRM6Go1BJtF3nF7BwbNb VKGPFVhKqORsZS2Q9HVK6JTfeH8ahXLvp5vR3WDi7YDdAKruMYWLUC4M/8EWt/2m /c8bJZkh2BeKd3FwPtK+ox3Vd2Qg6rDhK23e3j29qceVo3pvPArS9yEljfsqnW7F Hmx/b5TA/PrQPyLNmizITnISEmUhEcTrCaodCVpfFnx8oiUscKm8DZ2/ORunpIFc kTCKQ== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeeffedrtdeggdegledvfecutefuodetggdotefrod ftvfcurfhrohhfihhlvgemucfhrghsthforghilhdpuffrtefokffrpgfnqfghnecuuegr ihhlohhuthemuceftddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmdenucfjug hrpefhvfevufffkffojghfggfgsedtkeertdertddtnecuhfhrohhmpefmihhrhihlucfu hhhuthhsvghmrghuuceokhhirhhilhhlsehshhhuthgvmhhovhdrnhgrmhgvqeenucggtf frrghtthgvrhhnpeegveehtdfgvdfhudegffeuuddvgeevjefhveevgefhvdevieevteei vdehjefhjeenucevlhhushhtvghrufhiiigvpedtnecurfgrrhgrmhepmhgrihhlfhhroh hmpehkihhrihhllhesshhhuhhtvghmohhvrdhnrghmvgdpnhgspghrtghpthhtohepudek pdhmohguvgepshhmthhpohhuthdprhgtphhtthhopegrkhhpmheslhhinhhugidqfhhouh hnuggrthhiohhnrdhorhhgpdhrtghpthhtohepuggrvhhiugesrhgvughhrghtrdgtohhm pdhrtghpthhtohephhhughhhugesghhoohhglhgvrdgtohhmpdhrtghpthhtohepfihilh hlhiesihhnfhhrrgguvggrugdrohhrghdprhgtphhtthhopehlohhrvghniihordhsthho rghkvghssehorhgrtghlvgdrtghomhdprhgtphhtthhopehlihgrmhdrhhhofihlvghtth esohhrrggtlhgvrdgtohhmpdhrtghpthhtohepvhgsrggskhgrsehsuhhsvgdrtgiipdhr tghpthhtoheprhhpphhtsehkvghrnhgvlhdrohhrghdprhgtphhtthhopehsuhhrvghnsg esghhoohhglhgvrdgtohhm X-ME-Proxy: Feedback-ID: ie3994620:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Fri, 19 Sep 2025 08:40:50 -0400 (EDT) From: Kiryl Shutsemau To: Andrew Morton , David Hildenbrand , Hugh Dickins , Matthew Wilcox Cc: Lorenzo Stoakes , "Liam R. Howlett" , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Rik van Riel , Harry Yoo , Johannes Weiner , Shakeel Butt , Baolin Wang , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Kiryl Shutsemau Subject: [PATCHv2 4/5] mm/fault: Try to map the entire file folio in finish_fault() Date: Fri, 19 Sep 2025 13:40:35 +0100 Message-ID: <20250919124036.455709-5-kirill@shutemov.name> X-Mailer: git-send-email 2.50.1 In-Reply-To: <20250919124036.455709-1-kirill@shutemov.name> References: <20250919124036.455709-1-kirill@shutemov.name> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Kiryl Shutsemau The finish_fault() function uses per-page fault for file folios. This only occurs for file folios smaller than PMD_SIZE. The comment suggests that this approach prevents RSS inflation. However, it only prevents RSS accounting. The folio is still mapped to the process, and the fact that it is mapped by a single PTE does not affect memory pressure. Additionally, the kernel's ability to map large folios as PMD if they are large enough does not support this argument. When possible, map large folios in one shot. This reduces the number of minor page faults and allows for TLB coalescing. Mapping large folios at once will allow the rmap code to mlock it on add, as it will recognize that it is fully mapped and mlocking is safe. Signed-off-by: Kiryl Shutsemau Reviewed-by: Baolin Wang Reviewed-by: Shakeel Butt --- mm/memory.c | 9 ++------- 1 file changed, 2 insertions(+), 7 deletions(-) diff --git a/mm/memory.c b/mm/memory.c index 0ba4f6b71847..812a7d9f6531 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -5386,13 +5386,8 @@ vm_fault_t finish_fault(struct vm_fault *vmf) =20 nr_pages =3D folio_nr_pages(folio); =20 - /* - * Using per-page fault to maintain the uffd semantics, and same - * approach also applies to non shmem/tmpfs faults to avoid - * inflating the RSS of the process. - */ - if (!vma_is_shmem(vma) || unlikely(userfaultfd_armed(vma)) || - unlikely(needs_fallback)) { + /* Using per-page fault to maintain the uffd semantics */ + if (unlikely(userfaultfd_armed(vma)) || unlikely(needs_fallback)) { nr_pages =3D 1; } else if (nr_pages > 1) { pgoff_t idx =3D folio_page_idx(folio, page); --=20 2.50.1 From nobody Thu Oct 2 07:48:27 2025 Received: from fhigh-a4-smtp.messagingengine.com (fhigh-a4-smtp.messagingengine.com [103.168.172.155]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 029932248B3 for ; Fri, 19 Sep 2025 12:40:53 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=103.168.172.155 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1758285655; cv=none; b=Ec5arCDNWcIUyxFvueeNWOMejgZ+8M46NbySv7Nrxih9jsb1Q6EHohGOnNv8SEr48kcHuqopNPdE5wgnYvzWY51InzsQ4Ew8HcB4fxpfryK0hg7Awo8wpo/zrBKYMaLYTQheRpQfa5LCrDiocpyeP55AkEItFv92XaYI+pBKpNY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1758285655; c=relaxed/simple; bh=zdMoQugj5dZCnEqJberPdOJTUKqP3WcxTTDCP87sqj0=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=nvAGxIwxx5edLE187/aE8jJLOH1WQNHGVj4/bFW+MsiY1KJ5rf0fb4qSKIWDBiivfrs1dlsEg3LwPFAKOWgG715dfAbPizymc7/DU0p3hRoeyUGDgKGkhXAz+IDVhq2Py+G0O53FMqdIxm4VAs9XYU5ZlhyFo6fnplT8Mi2mB4U= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=shutemov.name; spf=pass smtp.mailfrom=shutemov.name; dkim=pass (2048-bit key) header.d=shutemov.name header.i=@shutemov.name header.b=NmsJzCb7; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b=R1+Sr7MG; arc=none smtp.client-ip=103.168.172.155 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=shutemov.name Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=shutemov.name Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=shutemov.name header.i=@shutemov.name header.b="NmsJzCb7"; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b="R1+Sr7MG" Received: from phl-compute-03.internal (phl-compute-03.internal [10.202.2.43]) by mailfhigh.phl.internal (Postfix) with ESMTP id 10F171400189; Fri, 19 Sep 2025 08:40:53 -0400 (EDT) Received: from phl-mailfrontend-02 ([10.202.2.163]) by phl-compute-03.internal (MEProxy); Fri, 19 Sep 2025 08:40:53 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=shutemov.name; h=cc:cc:content-transfer-encoding:content-type:date:date:from :from:in-reply-to:in-reply-to:message-id:mime-version:references :reply-to:subject:subject:to:to; s=fm3; t=1758285653; x= 1758372053; bh=mxh0csfVOqXEIeJ0PDZRZ2dOl352C/UceF+juPeU56o=; b=N msJzCb7eLd2gI0Zfaco0EvDRWDj7CmihiaDFfishQMh/f1pRAkQMAUxmv1ls4dsZ GflJFsnfiRczd++qW8n0M52k5xF9grKIXYfTrf0j2yxEUPgEBpdOSXMVIp1ChtHf fxz7lfWg2UW3pBEs71cnG5hziGfPpxUWojvr+zot2Kg7/Yon95XhXYG9xa7CfNi+ x6Okgvi1Yg0wigzpXuS2cULBwUUpFMEA5by++n8RtpBewT6QEL/vVJ16qofk0n+r ASwx1YG8QCZGKzvAQjOGO27VlkJ8OlLFPePXzqitCAL69ADBs4/o19P5rE837aFi dF3WXnvd87b0/hTeZIMRA== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding :content-type:date:date:feedback-id:feedback-id:from:from :in-reply-to:in-reply-to:message-id:mime-version:references :reply-to:subject:subject:to:to:x-me-proxy:x-me-sender :x-me-sender:x-sasl-enc; s=fm1; t=1758285653; x=1758372053; bh=m xh0csfVOqXEIeJ0PDZRZ2dOl352C/UceF+juPeU56o=; b=R1+Sr7MGq+aSYY4wH kOsTx/2EOwo1JplaQY3c6bXdLkWm352tdLHHJzoBUxpYDTWOW5ryoCBtXWkv9k7O zLgcqiTWtu4Vc3dDcJdeheOWhuQnHXWjFn2JEmxGxPUwg7G4Fea2JWBolCU3Lnld tsHctCtcYdVGie1CJQkNs57ZGtff+u9FcyU8bCGA5OwLzodMNw0nvrv70VXI/svG hU/cse3OmFEgYp1obWTMH0JDGRdvHicpCV0KEuc8HXGpyg7k+d+S3Y/htGnFDpcw fTC7zOqH7oSYtARBG1kpUklkQMuLOL32e6hcN1h6ctEdbATFMWUK+wi17oBMfIFE tPS7g== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeeffedrtdeggdegledvfecutefuodetggdotefrod ftvfcurfhrohhfihhlvgemucfhrghsthforghilhdpuffrtefokffrpgfnqfghnecuuegr ihhlohhuthemuceftddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmdenucfjug hrpefhvfevufffkffojghfggfgsedtkeertdertddtnecuhfhrohhmpefmihhrhihlucfu hhhuthhsvghmrghuuceokhhirhhilhhlsehshhhuthgvmhhovhdrnhgrmhgvqeenucggtf frrghtthgvrhhnpeegveehtdfgvdfhudegffeuuddvgeevjefhveevgefhvdevieevteei vdehjefhjeenucevlhhushhtvghrufhiiigvpedtnecurfgrrhgrmhepmhgrihhlfhhroh hmpehkihhrihhllhesshhhuhhtvghmohhvrdhnrghmvgdpnhgspghrtghpthhtohepudek pdhmohguvgepshhmthhpohhuthdprhgtphhtthhopegrkhhpmheslhhinhhugidqfhhouh hnuggrthhiohhnrdhorhhgpdhrtghpthhtohepuggrvhhiugesrhgvughhrghtrdgtohhm pdhrtghpthhtohephhhughhhugesghhoohhglhgvrdgtohhmpdhrtghpthhtohepfihilh hlhiesihhnfhhrrgguvggrugdrohhrghdprhgtphhtthhopehlohhrvghniihordhsthho rghkvghssehorhgrtghlvgdrtghomhdprhgtphhtthhopehlihgrmhdrhhhofihlvghtth esohhrrggtlhgvrdgtohhmpdhrtghpthhtohepvhgsrggskhgrsehsuhhsvgdrtgiipdhr tghpthhtoheprhhpphhtsehkvghrnhgvlhdrohhrghdprhgtphhtthhopehsuhhrvghnsg esghhoohhglhgvrdgtohhm X-ME-Proxy: Feedback-ID: ie3994620:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Fri, 19 Sep 2025 08:40:52 -0400 (EDT) From: Kiryl Shutsemau To: Andrew Morton , David Hildenbrand , Hugh Dickins , Matthew Wilcox Cc: Lorenzo Stoakes , "Liam R. Howlett" , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Rik van Riel , Harry Yoo , Johannes Weiner , Shakeel Butt , Baolin Wang , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Kiryl Shutsemau Subject: [PATCHv2 5/5] mm/rmap: Improve mlock tracking for large folios Date: Fri, 19 Sep 2025 13:40:36 +0100 Message-ID: <20250919124036.455709-6-kirill@shutemov.name> X-Mailer: git-send-email 2.50.1 In-Reply-To: <20250919124036.455709-1-kirill@shutemov.name> References: <20250919124036.455709-1-kirill@shutemov.name> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Kiryl Shutsemau The kernel currently does not mlock large folios when adding them to rmap, stating that it is difficult to confirm that the folio is fully mapped and safe to mlock it. This leads to a significant undercount of Mlocked in /proc/meminfo, causing problems in production where the stat was used to estimate system utilization and determine if load shedding is required. However, nowadays the caller passes a number of pages of the folio that are getting mapped, making it easy to check if the entire folio is mapped to the VMA. mlock the folio on rmap if it is fully mapped to the VMA. Mlocked in /proc/meminfo can still undercount, but the value is closer the truth and is useful for userspace. Signed-off-by: Kiryl Shutsemau Acked-by: David Hildenbrand Acked-by: Johannes Weiner Acked-by: Shakeel Butt Reviewed-by: Lorenzo Stoakes Reviewed-by: Baolin Wang --- mm/rmap.c | 19 ++++++++++++------- 1 file changed, 12 insertions(+), 7 deletions(-) diff --git a/mm/rmap.c b/mm/rmap.c index 482e6504fa88..6e09956670f4 100644 --- a/mm/rmap.c +++ b/mm/rmap.c @@ -1462,12 +1462,12 @@ static __always_inline void __folio_add_anon_rmap(s= truct folio *folio, } =20 /* - * For large folio, only mlock it if it's fully mapped to VMA. It's - * not easy to check whether the large folio is fully mapped to VMA - * here. Only mlock normal 4K folio and leave page reclaim to handle - * large folio. + * Only mlock it if the folio is fully mapped to the VMA. + * + * Partially mapped folios can be split on reclaim and part outside + * of mlocked VMA can be evicted or freed. */ - if (!folio_test_large(folio)) + if (folio_nr_pages(folio) =3D=3D nr_pages) mlock_vma_folio(folio, vma); } =20 @@ -1603,8 +1603,13 @@ static __always_inline void __folio_add_file_rmap(st= ruct folio *folio, nr =3D __folio_add_rmap(folio, page, nr_pages, vma, level, &nr_pmdmapped); __folio_mod_stat(folio, nr, nr_pmdmapped); =20 - /* See comments in folio_add_anon_rmap_*() */ - if (!folio_test_large(folio)) + /* + * Only mlock it if the folio is fully mapped to the VMA. + * + * Partially mapped folios can be split on reclaim and part outside + * of mlocked VMA can be evicted or freed. + */ + if (folio_nr_pages(folio) =3D=3D nr_pages) mlock_vma_folio(folio, vma); } =20 --=20 2.50.1