Documentation/admin-guide/mm/transhuge.rst | 91 ++- include/linux/huge_mm.h | 5 + include/linux/khugepaged.h | 2 + include/trace/events/huge_memory.h | 34 +- mm/huge_memory.c | 11 + mm/khugepaged.c | 612 ++++++++++++++------- mm/mremap.c | 2 +- 7 files changed, 535 insertions(+), 222 deletions(-)
The following series provides khugepaged with the capability to collapse anonymous memory regions to mTHPs. To achieve this we generalize the khugepaged functions to no longer depend on PMD_ORDER. Then during the PMD scan, we use a bitmap to track individual pages that are occupied (!none/zero). After the PMD scan is done, we do binary recursion on the bitmap to find the optimal mTHP sizes for the PMD range. The restriction on max_ptes_none is removed during the scan, to make sure we account for the whole PMD range. When no mTHP size is enabled, the legacy behavior of khugepaged is maintained. max_ptes_none will be scaled by the attempted collapse order to determine how full a mTHP must be to be eligible for the collapse to occur. If a mTHP collapse is attempted, but contains swapped out, or shared pages, we don't perform the collapse. It is now also possible to collapse to mTHPs without requiring the PMD THP size to be enabled. When enabling (m)THP sizes, if max_ptes_none >= HPAGE_PMD_NR/2 (255 on 4K page size), it will be automatically capped to HPAGE_PMD_NR/2 - 1 for mTHP collapses to prevent collapse "creep" behavior. This prevents constantly promoting mTHPs to the next available size, which would occur because a collapse introduces more non-zero pages that would satisfy the promotion condition on subsequent scans. Patch 1: Refactor/rename hpage_collapse Patch 2: Refactoring to combine madvise_collapse and khugepaged Patch 3-7: Generalize khugepaged functions for arbitrary orders Patch 8: skip collapsing mTHP to smaller orders Patch 9-10: Add per-order mTHP statistics and tracepoints Patch 11: Introduce collapse_allowable_orders Patch 12-14: Introduce bitmap and mTHP collapse support Patch 15: Documentation --------- Testing --------- - Built for x86_64, aarch64, ppc64le, and s390x - selftests mm - I created a test script that I used to push khugepaged to its limits while monitoring a number of stats and tracepoints. The code is available here[1] (Run in legacy mode for these changes and set mthp sizes to inherit) The summary from my testings was that there was no significant regression noticed through this test. In some cases my changes had better collapse latencies, and was able to scan more pages in the same amount of time/work, but for the most part the results were consistent. - redis testing. I tested these changes along with my defer changes (see followup [2] post for more details). We've decided to get the mTHP changes merged first before attempting the defer series. - some basic testing on 64k page size. - lots of general use. V11 Changes: - Minor nits and cleanup/refactoring - commit messages updates - split patches to make them easier to review - added helper functions for checking allowable orders and max_ptes_none - simplified anon_vma locking tracking to improve code readability - Added automatic cap to max_ptes_none for mTHP collapse to prevent "creep" - changed bitmap tracking to use page-level granularity instead of chunk-based this removes inaccuracies/complexity - Improved documentation and comments throughout the series - Refactored collapse_single_pmd() for better readability - Added proper WARN_ON_ONCE() instead of BUG_ON() for better error handling - reordered patches to make them easier to review - merge Baolins patch to run khugepaged for all orders now that collapse_allowable_orders is available V10: https://lore.kernel.org/lkml/20250819134205.622806-1-npache@redhat.com/ V9 : https://lore.kernel.org/lkml/20250714003207.113275-1-npache@redhat.com/ V8 : https://lore.kernel.org/lkml/20250702055742.102808-1-npache@redhat.com/ V7 : https://lore.kernel.org/lkml/20250515032226.128900-1-npache@redhat.com/ V6 : https://lore.kernel.org/lkml/20250515030312.125567-1-npache@redhat.com/ V5 : https://lore.kernel.org/lkml/20250428181218.85925-1-npache@redhat.com/ V4 : https://lore.kernel.org/lkml/20250417000238.74567-1-npache@redhat.com/ V3 : https://lore.kernel.org/lkml/20250414220557.35388-1-npache@redhat.com/ V2 : https://lore.kernel.org/lkml/20250211003028.213461-1-npache@redhat.com/ V1 : https://lore.kernel.org/lkml/20250108233128.14484-1-npache@redhat.com/ A big thanks to everyone that has reviewed, tested, and participated in the development process. Its been a great experience working with all of you on this long endeavour. [1] - https://gitlab.com/npache/khugepaged_mthp_test [2] - https://lore.kernel.org/lkml/20250515033857.132535-1-npache@redhat.com/ Baolin Wang (1): khugepaged: run khugepaged for all orders Dev Jain (1): khugepaged: generalize alloc_charge_folio() Nico Pache (13): khugepaged: rename hpage_collapse_* to collapse_* introduce collapse_single_pmd to unify khugepaged and madvise_collapse khugepaged: generalize hugepage_vma_revalidate for mTHP support khugepaged: generalize __collapse_huge_page_* for mTHP support khugepaged: introduce collapse_max_ptes_none helper function khugepaged: generalize collapse_huge_page for mTHP collapse khugepaged: skip collapsing mTHP to smaller orders khugepaged: add per-order mTHP collapse failure statistics khugepaged: improve tracepoints for mTHP orders khugepaged: introduce collapse_allowable_orders helper function khugepaged: Introduce mTHP collapse support khugepaged: avoid unnecessary mTHP collapse attempts Documentation: mm: update the admin guide for mTHP collapse Documentation/admin-guide/mm/transhuge.rst | 91 ++- include/linux/huge_mm.h | 5 + include/linux/khugepaged.h | 2 + include/trace/events/huge_memory.h | 34 +- mm/huge_memory.c | 11 + mm/khugepaged.c | 612 ++++++++++++++------- mm/mremap.c | 2 +- 7 files changed, 535 insertions(+), 222 deletions(-) -- 2.51.0
Hi Nico, I will take a look at this, but just a brief hint/plea to maybe relax a bit on the respins, there was some ongoing discussion on the v10 yesterday, and then today a new, huge v11 comes along :) THP workload has been crazy this cycle, and this series in particular needs a lot of attention, so it'd be good perhaps to make sure everything is replied to and give it a day or two at least before sending out the next revision. I feel like we probably need to make some changes to how THP works as a whole, perhaps mandating no series in the merge window, or something to help even things out a little as I feel David and I have been somewhat overwhelmed this cycle, and we do need to be sensible with how we handle workload. (Plus I'm going to Kernel Recipes and then on leave for a couple weeks after that soon so I am hoping here that David isn't left with too much workload alone for that time also :) Cheers, Lorenzo
On Thu, Sep 11, 2025 at 09:27:55PM -0600, Nico Pache wrote: > The following series provides khugepaged with the capability to collapse > anonymous memory regions to mTHPs. > > To achieve this we generalize the khugepaged functions to no longer depend > on PMD_ORDER. Then during the PMD scan, we use a bitmap to track individual > pages that are occupied (!none/zero). After the PMD scan is done, we do > binary recursion on the bitmap to find the optimal mTHP sizes for the PMD > range. The restriction on max_ptes_none is removed during the scan, to make > sure we account for the whole PMD range. When no mTHP size is enabled, the > legacy behavior of khugepaged is maintained. max_ptes_none will be scaled > by the attempted collapse order to determine how full a mTHP must be to be > eligible for the collapse to occur. If a mTHP collapse is attempted, but > contains swapped out, or shared pages, we don't perform the collapse. It is > now also possible to collapse to mTHPs without requiring the PMD THP size > to be enabled. > > When enabling (m)THP sizes, if max_ptes_none >= HPAGE_PMD_NR/2 (255 on > 4K page size), it will be automatically capped to HPAGE_PMD_NR/2 - 1 for > mTHP collapses to prevent collapse "creep" behavior. This prevents > constantly promoting mTHPs to the next available size, which would occur > because a collapse introduces more non-zero pages that would satisfy the > promotion condition on subsequent scans. Hm. Maybe instead of capping at HPAGE_PMD_NR/2 - 1 we can count all-zeros 4k as none_or_zero? It mirrors the logic of shrinker. -- Kiryl Shutsemau / Kirill A. Shutemov
On 12.09.25 14:19, Kiryl Shutsemau wrote: > On Thu, Sep 11, 2025 at 09:27:55PM -0600, Nico Pache wrote: >> The following series provides khugepaged with the capability to collapse >> anonymous memory regions to mTHPs. >> >> To achieve this we generalize the khugepaged functions to no longer depend >> on PMD_ORDER. Then during the PMD scan, we use a bitmap to track individual >> pages that are occupied (!none/zero). After the PMD scan is done, we do >> binary recursion on the bitmap to find the optimal mTHP sizes for the PMD >> range. The restriction on max_ptes_none is removed during the scan, to make >> sure we account for the whole PMD range. When no mTHP size is enabled, the >> legacy behavior of khugepaged is maintained. max_ptes_none will be scaled >> by the attempted collapse order to determine how full a mTHP must be to be >> eligible for the collapse to occur. If a mTHP collapse is attempted, but >> contains swapped out, or shared pages, we don't perform the collapse. It is >> now also possible to collapse to mTHPs without requiring the PMD THP size >> to be enabled. >> >> When enabling (m)THP sizes, if max_ptes_none >= HPAGE_PMD_NR/2 (255 on >> 4K page size), it will be automatically capped to HPAGE_PMD_NR/2 - 1 for >> mTHP collapses to prevent collapse "creep" behavior. This prevents >> constantly promoting mTHPs to the next available size, which would occur >> because a collapse introduces more non-zero pages that would satisfy the >> promotion condition on subsequent scans. > > Hm. Maybe instead of capping at HPAGE_PMD_NR/2 - 1 we can count > all-zeros 4k as none_or_zero? It mirrors the logic of shrinker. BTW, I thought further about this and I agree: if we count zero-filled pages towards none_or_zero one we can avoid the "creep" problem. The scanning-for-zero part is rather nasty, though. -- Cheers David / dhildenb
On Fri, Sep 12, 2025 at 7:48 AM David Hildenbrand <david@redhat.com> wrote: > > On 12.09.25 14:19, Kiryl Shutsemau wrote: > > On Thu, Sep 11, 2025 at 09:27:55PM -0600, Nico Pache wrote: > >> The following series provides khugepaged with the capability to collapse > >> anonymous memory regions to mTHPs. > >> > >> To achieve this we generalize the khugepaged functions to no longer depend > >> on PMD_ORDER. Then during the PMD scan, we use a bitmap to track individual > >> pages that are occupied (!none/zero). After the PMD scan is done, we do > >> binary recursion on the bitmap to find the optimal mTHP sizes for the PMD > >> range. The restriction on max_ptes_none is removed during the scan, to make > >> sure we account for the whole PMD range. When no mTHP size is enabled, the > >> legacy behavior of khugepaged is maintained. max_ptes_none will be scaled > >> by the attempted collapse order to determine how full a mTHP must be to be > >> eligible for the collapse to occur. If a mTHP collapse is attempted, but > >> contains swapped out, or shared pages, we don't perform the collapse. It is > >> now also possible to collapse to mTHPs without requiring the PMD THP size > >> to be enabled. > >> > >> When enabling (m)THP sizes, if max_ptes_none >= HPAGE_PMD_NR/2 (255 on > >> 4K page size), it will be automatically capped to HPAGE_PMD_NR/2 - 1 for > >> mTHP collapses to prevent collapse "creep" behavior. This prevents > >> constantly promoting mTHPs to the next available size, which would occur > >> because a collapse introduces more non-zero pages that would satisfy the > >> promotion condition on subsequent scans. > > > > Hm. Maybe instead of capping at HPAGE_PMD_NR/2 - 1 we can count > > all-zeros 4k as none_or_zero? It mirrors the logic of shrinker. > > BTW, I thought further about this and I agree: if we count zero-filled > pages towards none_or_zero one we can avoid the "creep" problem. > > The scanning-for-zero part is rather nasty, though. IIRC me and David have discussed this in the past and decided to avoid this approach at the moment because it would be complicated and "nasty". > > -- > Cheers > > David / dhildenb >
On 12.09.25 15:47, David Hildenbrand wrote: > On 12.09.25 14:19, Kiryl Shutsemau wrote: >> On Thu, Sep 11, 2025 at 09:27:55PM -0600, Nico Pache wrote: >>> The following series provides khugepaged with the capability to collapse >>> anonymous memory regions to mTHPs. >>> >>> To achieve this we generalize the khugepaged functions to no longer depend >>> on PMD_ORDER. Then during the PMD scan, we use a bitmap to track individual >>> pages that are occupied (!none/zero). After the PMD scan is done, we do >>> binary recursion on the bitmap to find the optimal mTHP sizes for the PMD >>> range. The restriction on max_ptes_none is removed during the scan, to make >>> sure we account for the whole PMD range. When no mTHP size is enabled, the >>> legacy behavior of khugepaged is maintained. max_ptes_none will be scaled >>> by the attempted collapse order to determine how full a mTHP must be to be >>> eligible for the collapse to occur. If a mTHP collapse is attempted, but >>> contains swapped out, or shared pages, we don't perform the collapse. It is >>> now also possible to collapse to mTHPs without requiring the PMD THP size >>> to be enabled. >>> >>> When enabling (m)THP sizes, if max_ptes_none >= HPAGE_PMD_NR/2 (255 on >>> 4K page size), it will be automatically capped to HPAGE_PMD_NR/2 - 1 for >>> mTHP collapses to prevent collapse "creep" behavior. This prevents >>> constantly promoting mTHPs to the next available size, which would occur >>> because a collapse introduces more non-zero pages that would satisfy the >>> promotion condition on subsequent scans. >> >> Hm. Maybe instead of capping at HPAGE_PMD_NR/2 - 1 we can count >> all-zeros 4k as none_or_zero? It mirrors the logic of shrinker. > > BTW, I thought further about this and I agree: if we count zero-filled > pages towards none_or_zero one we can avoid the "creep" problem. > > The scanning-for-zero part is rather nasty, though. Aaand, thinking again from the other direction, this would mean that just because pages became zero after some time that we would no longer collapse because none_or_zero would then be higher. Hm .... How I hate all of this so very very much :) -- Cheers David / dhildenb
On Fri, Sep 12, 2025 at 04:28:09PM +0200, David Hildenbrand wrote: > On 12.09.25 15:47, David Hildenbrand wrote: > > On 12.09.25 14:19, Kiryl Shutsemau wrote: > > > On Thu, Sep 11, 2025 at 09:27:55PM -0600, Nico Pache wrote: > > > > The following series provides khugepaged with the capability to collapse > > > > anonymous memory regions to mTHPs. > > > > > > > > To achieve this we generalize the khugepaged functions to no longer depend > > > > on PMD_ORDER. Then during the PMD scan, we use a bitmap to track individual > > > > pages that are occupied (!none/zero). After the PMD scan is done, we do > > > > binary recursion on the bitmap to find the optimal mTHP sizes for the PMD > > > > range. The restriction on max_ptes_none is removed during the scan, to make > > > > sure we account for the whole PMD range. When no mTHP size is enabled, the > > > > legacy behavior of khugepaged is maintained. max_ptes_none will be scaled > > > > by the attempted collapse order to determine how full a mTHP must be to be > > > > eligible for the collapse to occur. If a mTHP collapse is attempted, but > > > > contains swapped out, or shared pages, we don't perform the collapse. It is > > > > now also possible to collapse to mTHPs without requiring the PMD THP size > > > > to be enabled. > > > > > > > > When enabling (m)THP sizes, if max_ptes_none >= HPAGE_PMD_NR/2 (255 on > > > > 4K page size), it will be automatically capped to HPAGE_PMD_NR/2 - 1 for > > > > mTHP collapses to prevent collapse "creep" behavior. This prevents > > > > constantly promoting mTHPs to the next available size, which would occur > > > > because a collapse introduces more non-zero pages that would satisfy the > > > > promotion condition on subsequent scans. > > > > > > Hm. Maybe instead of capping at HPAGE_PMD_NR/2 - 1 we can count > > > all-zeros 4k as none_or_zero? It mirrors the logic of shrinker. > > > > BTW, I thought further about this and I agree: if we count zero-filled > > pages towards none_or_zero one we can avoid the "creep" problem. > > > > The scanning-for-zero part is rather nasty, though. > > Aaand, thinking again from the other direction, this would mean that just > because pages became zero after some time that we would no longer collapse > because none_or_zero would then be higher. Hm .... > > How I hate all of this so very very much :) This is not new. Shrinker has the same problem: it cannot distinguish between hot 4k that happened to be zero from the 4k that is there just because of we faulted in 2M a time. -- Kiryl Shutsemau / Kirill A. Shutemov
On 12.09.25 16:35, Kiryl Shutsemau wrote: > On Fri, Sep 12, 2025 at 04:28:09PM +0200, David Hildenbrand wrote: >> On 12.09.25 15:47, David Hildenbrand wrote: >>> On 12.09.25 14:19, Kiryl Shutsemau wrote: >>>> On Thu, Sep 11, 2025 at 09:27:55PM -0600, Nico Pache wrote: >>>>> The following series provides khugepaged with the capability to collapse >>>>> anonymous memory regions to mTHPs. >>>>> >>>>> To achieve this we generalize the khugepaged functions to no longer depend >>>>> on PMD_ORDER. Then during the PMD scan, we use a bitmap to track individual >>>>> pages that are occupied (!none/zero). After the PMD scan is done, we do >>>>> binary recursion on the bitmap to find the optimal mTHP sizes for the PMD >>>>> range. The restriction on max_ptes_none is removed during the scan, to make >>>>> sure we account for the whole PMD range. When no mTHP size is enabled, the >>>>> legacy behavior of khugepaged is maintained. max_ptes_none will be scaled >>>>> by the attempted collapse order to determine how full a mTHP must be to be >>>>> eligible for the collapse to occur. If a mTHP collapse is attempted, but >>>>> contains swapped out, or shared pages, we don't perform the collapse. It is >>>>> now also possible to collapse to mTHPs without requiring the PMD THP size >>>>> to be enabled. >>>>> >>>>> When enabling (m)THP sizes, if max_ptes_none >= HPAGE_PMD_NR/2 (255 on >>>>> 4K page size), it will be automatically capped to HPAGE_PMD_NR/2 - 1 for >>>>> mTHP collapses to prevent collapse "creep" behavior. This prevents >>>>> constantly promoting mTHPs to the next available size, which would occur >>>>> because a collapse introduces more non-zero pages that would satisfy the >>>>> promotion condition on subsequent scans. >>>> >>>> Hm. Maybe instead of capping at HPAGE_PMD_NR/2 - 1 we can count >>>> all-zeros 4k as none_or_zero? It mirrors the logic of shrinker. >>> >>> BTW, I thought further about this and I agree: if we count zero-filled >>> pages towards none_or_zero one we can avoid the "creep" problem. >>> >>> The scanning-for-zero part is rather nasty, though. >> >> Aaand, thinking again from the other direction, this would mean that just >> because pages became zero after some time that we would no longer collapse >> because none_or_zero would then be higher. Hm .... >> >> How I hate all of this so very very much :) > > This is not new. Shrinker has the same problem: it cannot distinguish > between hot 4k that happened to be zero from the 4k that is there just > because of we faulted in 2M a time. Right. And so far that problem is isolated to the shrinker. To me so far "none_or_zero" really meant "will I consume more memory when collapsing". That's not true for zero-filled pages, obviously. -- Cheers David / dhildenb
On Fri, Sep 12, 2025 at 04:56:47PM +0200, David Hildenbrand wrote: > On 12.09.25 16:35, Kiryl Shutsemau wrote: > > On Fri, Sep 12, 2025 at 04:28:09PM +0200, David Hildenbrand wrote: > > > On 12.09.25 15:47, David Hildenbrand wrote: > > > > On 12.09.25 14:19, Kiryl Shutsemau wrote: > > > > > On Thu, Sep 11, 2025 at 09:27:55PM -0600, Nico Pache wrote: > > > > > > The following series provides khugepaged with the capability to collapse > > > > > > anonymous memory regions to mTHPs. > > > > > > > > > > > > To achieve this we generalize the khugepaged functions to no longer depend > > > > > > on PMD_ORDER. Then during the PMD scan, we use a bitmap to track individual > > > > > > pages that are occupied (!none/zero). After the PMD scan is done, we do > > > > > > binary recursion on the bitmap to find the optimal mTHP sizes for the PMD > > > > > > range. The restriction on max_ptes_none is removed during the scan, to make > > > > > > sure we account for the whole PMD range. When no mTHP size is enabled, the > > > > > > legacy behavior of khugepaged is maintained. max_ptes_none will be scaled > > > > > > by the attempted collapse order to determine how full a mTHP must be to be > > > > > > eligible for the collapse to occur. If a mTHP collapse is attempted, but > > > > > > contains swapped out, or shared pages, we don't perform the collapse. It is > > > > > > now also possible to collapse to mTHPs without requiring the PMD THP size > > > > > > to be enabled. > > > > > > > > > > > > When enabling (m)THP sizes, if max_ptes_none >= HPAGE_PMD_NR/2 (255 on > > > > > > 4K page size), it will be automatically capped to HPAGE_PMD_NR/2 - 1 for > > > > > > mTHP collapses to prevent collapse "creep" behavior. This prevents > > > > > > constantly promoting mTHPs to the next available size, which would occur > > > > > > because a collapse introduces more non-zero pages that would satisfy the > > > > > > promotion condition on subsequent scans. > > > > > > > > > > Hm. Maybe instead of capping at HPAGE_PMD_NR/2 - 1 we can count > > > > > all-zeros 4k as none_or_zero? It mirrors the logic of shrinker. > > > > > > > > BTW, I thought further about this and I agree: if we count zero-filled > > > > pages towards none_or_zero one we can avoid the "creep" problem. > > > > > > > > The scanning-for-zero part is rather nasty, though. > > > > > > Aaand, thinking again from the other direction, this would mean that just > > > because pages became zero after some time that we would no longer collapse > > > because none_or_zero would then be higher. Hm .... > > > > > > How I hate all of this so very very much :) > > > > This is not new. Shrinker has the same problem: it cannot distinguish > > between hot 4k that happened to be zero from the 4k that is there just > > because of we faulted in 2M a time. > > Right. And so far that problem is isolated to the shrinker. > > To me so far "none_or_zero" really meant "will I consume more memory when > collapsing". That's not true for zero-filled pages, obviously. Well, KSM can reclaim these zero-filled memory until we collapse it. -- Kiryl Shutsemau / Kirill A. Shutemov
On 12.09.25 17:41, Kiryl Shutsemau wrote: > On Fri, Sep 12, 2025 at 04:56:47PM +0200, David Hildenbrand wrote: >> On 12.09.25 16:35, Kiryl Shutsemau wrote: >>> On Fri, Sep 12, 2025 at 04:28:09PM +0200, David Hildenbrand wrote: >>>> On 12.09.25 15:47, David Hildenbrand wrote: >>>>> On 12.09.25 14:19, Kiryl Shutsemau wrote: >>>>>> On Thu, Sep 11, 2025 at 09:27:55PM -0600, Nico Pache wrote: >>>>>>> The following series provides khugepaged with the capability to collapse >>>>>>> anonymous memory regions to mTHPs. >>>>>>> >>>>>>> To achieve this we generalize the khugepaged functions to no longer depend >>>>>>> on PMD_ORDER. Then during the PMD scan, we use a bitmap to track individual >>>>>>> pages that are occupied (!none/zero). After the PMD scan is done, we do >>>>>>> binary recursion on the bitmap to find the optimal mTHP sizes for the PMD >>>>>>> range. The restriction on max_ptes_none is removed during the scan, to make >>>>>>> sure we account for the whole PMD range. When no mTHP size is enabled, the >>>>>>> legacy behavior of khugepaged is maintained. max_ptes_none will be scaled >>>>>>> by the attempted collapse order to determine how full a mTHP must be to be >>>>>>> eligible for the collapse to occur. If a mTHP collapse is attempted, but >>>>>>> contains swapped out, or shared pages, we don't perform the collapse. It is >>>>>>> now also possible to collapse to mTHPs without requiring the PMD THP size >>>>>>> to be enabled. >>>>>>> >>>>>>> When enabling (m)THP sizes, if max_ptes_none >= HPAGE_PMD_NR/2 (255 on >>>>>>> 4K page size), it will be automatically capped to HPAGE_PMD_NR/2 - 1 for >>>>>>> mTHP collapses to prevent collapse "creep" behavior. This prevents >>>>>>> constantly promoting mTHPs to the next available size, which would occur >>>>>>> because a collapse introduces more non-zero pages that would satisfy the >>>>>>> promotion condition on subsequent scans. >>>>>> >>>>>> Hm. Maybe instead of capping at HPAGE_PMD_NR/2 - 1 we can count >>>>>> all-zeros 4k as none_or_zero? It mirrors the logic of shrinker. >>>>> >>>>> BTW, I thought further about this and I agree: if we count zero-filled >>>>> pages towards none_or_zero one we can avoid the "creep" problem. >>>>> >>>>> The scanning-for-zero part is rather nasty, though. >>>> >>>> Aaand, thinking again from the other direction, this would mean that just >>>> because pages became zero after some time that we would no longer collapse >>>> because none_or_zero would then be higher. Hm .... >>>> >>>> How I hate all of this so very very much :) >>> >>> This is not new. Shrinker has the same problem: it cannot distinguish >>> between hot 4k that happened to be zero from the 4k that is there just >>> because of we faulted in 2M a time. >> >> Right. And so far that problem is isolated to the shrinker. >> >> To me so far "none_or_zero" really meant "will I consume more memory when >> collapsing". That's not true for zero-filled pages, obviously. > > Well, KSM can reclaim these zero-filled memory until we collapse it. KSM is used so rarely (for good reasons) that I would never ever build an argument based on its existence :P But yes: during the very first shrinker discussion I raised that KSM can do the same thing. Obviously that was not good enough. -- Cheers David / dhildenb
On Fri, Sep 12, 2025 at 05:45:26PM +0200, David Hildenbrand wrote: > On 12.09.25 17:41, Kiryl Shutsemau wrote: > > On Fri, Sep 12, 2025 at 04:56:47PM +0200, David Hildenbrand wrote: > > > On 12.09.25 16:35, Kiryl Shutsemau wrote: > > > > On Fri, Sep 12, 2025 at 04:28:09PM +0200, David Hildenbrand wrote: > > > > > On 12.09.25 15:47, David Hildenbrand wrote: > > > > > > On 12.09.25 14:19, Kiryl Shutsemau wrote: > > > > > > > On Thu, Sep 11, 2025 at 09:27:55PM -0600, Nico Pache wrote: > > > > > > > > The following series provides khugepaged with the capability to collapse > > > > > > > > anonymous memory regions to mTHPs. > > > > > > > > > > > > > > > > To achieve this we generalize the khugepaged functions to no longer depend > > > > > > > > on PMD_ORDER. Then during the PMD scan, we use a bitmap to track individual > > > > > > > > pages that are occupied (!none/zero). After the PMD scan is done, we do > > > > > > > > binary recursion on the bitmap to find the optimal mTHP sizes for the PMD > > > > > > > > range. The restriction on max_ptes_none is removed during the scan, to make > > > > > > > > sure we account for the whole PMD range. When no mTHP size is enabled, the > > > > > > > > legacy behavior of khugepaged is maintained. max_ptes_none will be scaled > > > > > > > > by the attempted collapse order to determine how full a mTHP must be to be > > > > > > > > eligible for the collapse to occur. If a mTHP collapse is attempted, but > > > > > > > > contains swapped out, or shared pages, we don't perform the collapse. It is > > > > > > > > now also possible to collapse to mTHPs without requiring the PMD THP size > > > > > > > > to be enabled. > > > > > > > > > > > > > > > > When enabling (m)THP sizes, if max_ptes_none >= HPAGE_PMD_NR/2 (255 on > > > > > > > > 4K page size), it will be automatically capped to HPAGE_PMD_NR/2 - 1 for > > > > > > > > mTHP collapses to prevent collapse "creep" behavior. This prevents > > > > > > > > constantly promoting mTHPs to the next available size, which would occur > > > > > > > > because a collapse introduces more non-zero pages that would satisfy the > > > > > > > > promotion condition on subsequent scans. > > > > > > > > > > > > > > Hm. Maybe instead of capping at HPAGE_PMD_NR/2 - 1 we can count > > > > > > > all-zeros 4k as none_or_zero? It mirrors the logic of shrinker. > > > > > > > > > > > > BTW, I thought further about this and I agree: if we count zero-filled > > > > > > pages towards none_or_zero one we can avoid the "creep" problem. > > > > > > > > > > > > The scanning-for-zero part is rather nasty, though. > > > > > > > > > > Aaand, thinking again from the other direction, this would mean that just > > > > > because pages became zero after some time that we would no longer collapse > > > > > because none_or_zero would then be higher. Hm .... > > > > > > > > > > How I hate all of this so very very much :) > > > > > > > > This is not new. Shrinker has the same problem: it cannot distinguish > > > > between hot 4k that happened to be zero from the 4k that is there just > > > > because of we faulted in 2M a time. > > > > > > Right. And so far that problem is isolated to the shrinker. > > > > > > To me so far "none_or_zero" really meant "will I consume more memory when > > > collapsing". That's not true for zero-filled pages, obviously. > > > > Well, KSM can reclaim these zero-filled memory until we collapse it. > > KSM is used so rarely (for good reasons) that I would never ever build an > argument based on its existence :P > > But yes: during the very first shrinker discussion I raised that KSM can do > the same thing. Obviously that was not good enough. > > -- > Cheers > > David / dhildenb > With all this stuff said, do we have an actual plan for what we intend to do _now_? As Nico has implemented a basic solution here that we all seem to agree is not what we want. Without needing special new hardware or major reworks, what would this parameter look like? What would the heuristics be? What about the eagerness scales? I'm but a simple kernel developer, and interested in simple pragmatic stuff :) do you have a plan right now David? Maybe we can start with something simple like a rough percentage per eagerness entry that then gets scaled based on utilisation? I'd like us to ideally have something to suggest to Nico before the next respin. Cheers, Lorenzo
On 12.09.25 17:51, Lorenzo Stoakes wrote: > On Fri, Sep 12, 2025 at 05:45:26PM +0200, David Hildenbrand wrote: >> On 12.09.25 17:41, Kiryl Shutsemau wrote: >>> On Fri, Sep 12, 2025 at 04:56:47PM +0200, David Hildenbrand wrote: >>>> On 12.09.25 16:35, Kiryl Shutsemau wrote: >>>>> On Fri, Sep 12, 2025 at 04:28:09PM +0200, David Hildenbrand wrote: >>>>>> On 12.09.25 15:47, David Hildenbrand wrote: >>>>>>> On 12.09.25 14:19, Kiryl Shutsemau wrote: >>>>>>>> On Thu, Sep 11, 2025 at 09:27:55PM -0600, Nico Pache wrote: >>>>>>>>> The following series provides khugepaged with the capability to collapse >>>>>>>>> anonymous memory regions to mTHPs. >>>>>>>>> >>>>>>>>> To achieve this we generalize the khugepaged functions to no longer depend >>>>>>>>> on PMD_ORDER. Then during the PMD scan, we use a bitmap to track individual >>>>>>>>> pages that are occupied (!none/zero). After the PMD scan is done, we do >>>>>>>>> binary recursion on the bitmap to find the optimal mTHP sizes for the PMD >>>>>>>>> range. The restriction on max_ptes_none is removed during the scan, to make >>>>>>>>> sure we account for the whole PMD range. When no mTHP size is enabled, the >>>>>>>>> legacy behavior of khugepaged is maintained. max_ptes_none will be scaled >>>>>>>>> by the attempted collapse order to determine how full a mTHP must be to be >>>>>>>>> eligible for the collapse to occur. If a mTHP collapse is attempted, but >>>>>>>>> contains swapped out, or shared pages, we don't perform the collapse. It is >>>>>>>>> now also possible to collapse to mTHPs without requiring the PMD THP size >>>>>>>>> to be enabled. >>>>>>>>> >>>>>>>>> When enabling (m)THP sizes, if max_ptes_none >= HPAGE_PMD_NR/2 (255 on >>>>>>>>> 4K page size), it will be automatically capped to HPAGE_PMD_NR/2 - 1 for >>>>>>>>> mTHP collapses to prevent collapse "creep" behavior. This prevents >>>>>>>>> constantly promoting mTHPs to the next available size, which would occur >>>>>>>>> because a collapse introduces more non-zero pages that would satisfy the >>>>>>>>> promotion condition on subsequent scans. >>>>>>>> >>>>>>>> Hm. Maybe instead of capping at HPAGE_PMD_NR/2 - 1 we can count >>>>>>>> all-zeros 4k as none_or_zero? It mirrors the logic of shrinker. >>>>>>> >>>>>>> BTW, I thought further about this and I agree: if we count zero-filled >>>>>>> pages towards none_or_zero one we can avoid the "creep" problem. >>>>>>> >>>>>>> The scanning-for-zero part is rather nasty, though. >>>>>> >>>>>> Aaand, thinking again from the other direction, this would mean that just >>>>>> because pages became zero after some time that we would no longer collapse >>>>>> because none_or_zero would then be higher. Hm .... >>>>>> >>>>>> How I hate all of this so very very much :) >>>>> >>>>> This is not new. Shrinker has the same problem: it cannot distinguish >>>>> between hot 4k that happened to be zero from the 4k that is there just >>>>> because of we faulted in 2M a time. >>>> >>>> Right. And so far that problem is isolated to the shrinker. >>>> >>>> To me so far "none_or_zero" really meant "will I consume more memory when >>>> collapsing". That's not true for zero-filled pages, obviously. >>> >>> Well, KSM can reclaim these zero-filled memory until we collapse it. >> >> KSM is used so rarely (for good reasons) that I would never ever build an >> argument based on its existence :P >> >> But yes: during the very first shrinker discussion I raised that KSM can do >> the same thing. Obviously that was not good enough. >> >> -- >> Cheers >> >> David / dhildenb >> > > With all this stuff said, do we have an actual plan for what we intend to do > _now_? Oh no, no I have to use my brain and it's Friday evening. > > As Nico has implemented a basic solution here that we all seem to agree is not > what we want. > > Without needing special new hardware or major reworks, what would this parameter > look like? > > What would the heuristics be? What about the eagerness scales? > > I'm but a simple kernel developer, :) and interested in simple pragmatic stuff :) > do you have a plan right now David? Ehm, if you ask me that way ... > > Maybe we can start with something simple like a rough percentage per eagerness > entry that then gets scaled based on utilisation? ... I think we should probably: 1) Start with something very simple for mTHP that doesn't lock us into any particular direction. 2) Add an "eagerness" parameter with fixed scale and use that for mTHP as well 3) Improve that "eagerness" algorithm using a dynamic scale or #whatever 4) Solve world peace and world hunger 5) Connect it all to memory pressure / reclaim / shrinker / heuristics / hw hotness / #whatever I maintain my initial position that just using max_ptes_none == 511 -> collapse mTHP always max_ptes_none != 511 -> collapse mTHP only if we all PTEs are non-none/zero As a starting point is probably simple and best, and likely leaves room for any changes later. Of course, we could do what Nico is proposing here, as 1) and change it all later. It's just when it comes to documenting all that stuff in patch #15 that I feel like "alright, we shouldn't be doing it longterm like that, so let's not make anybody depend on any weird behavior here by over-domenting it". I mean " +To prevent "creeping" behavior where collapses continuously promote to larger +orders, if max_ptes_none >= HPAGE_PMD_NR/2 (255 on 4K page size), it is +capped to HPAGE_PMD_NR/2 - 1 for mTHP collapses. This is due to the fact +that introducing more than half of the pages to be non-zero it will always +satisfy the eligibility check on the next scan and the region will be collapse. " Is just way, way to detailed. I would just say "The kernel might decide to use a more conservative approach when collapsing smaller THPs" etc. Thoughts? -- Cheers David / dhildenb
On Fri, Sep 12, 2025 at 11:53 AM David Hildenbrand <david@redhat.com> wrote: > > On 12.09.25 17:51, Lorenzo Stoakes wrote: > > On Fri, Sep 12, 2025 at 05:45:26PM +0200, David Hildenbrand wrote: > >> On 12.09.25 17:41, Kiryl Shutsemau wrote: > >>> On Fri, Sep 12, 2025 at 04:56:47PM +0200, David Hildenbrand wrote: > >>>> On 12.09.25 16:35, Kiryl Shutsemau wrote: > >>>>> On Fri, Sep 12, 2025 at 04:28:09PM +0200, David Hildenbrand wrote: > >>>>>> On 12.09.25 15:47, David Hildenbrand wrote: > >>>>>>> On 12.09.25 14:19, Kiryl Shutsemau wrote: > >>>>>>>> On Thu, Sep 11, 2025 at 09:27:55PM -0600, Nico Pache wrote: > >>>>>>>>> The following series provides khugepaged with the capability to collapse > >>>>>>>>> anonymous memory regions to mTHPs. > >>>>>>>>> > >>>>>>>>> To achieve this we generalize the khugepaged functions to no longer depend > >>>>>>>>> on PMD_ORDER. Then during the PMD scan, we use a bitmap to track individual > >>>>>>>>> pages that are occupied (!none/zero). After the PMD scan is done, we do > >>>>>>>>> binary recursion on the bitmap to find the optimal mTHP sizes for the PMD > >>>>>>>>> range. The restriction on max_ptes_none is removed during the scan, to make > >>>>>>>>> sure we account for the whole PMD range. When no mTHP size is enabled, the > >>>>>>>>> legacy behavior of khugepaged is maintained. max_ptes_none will be scaled > >>>>>>>>> by the attempted collapse order to determine how full a mTHP must be to be > >>>>>>>>> eligible for the collapse to occur. If a mTHP collapse is attempted, but > >>>>>>>>> contains swapped out, or shared pages, we don't perform the collapse. It is > >>>>>>>>> now also possible to collapse to mTHPs without requiring the PMD THP size > >>>>>>>>> to be enabled. > >>>>>>>>> > >>>>>>>>> When enabling (m)THP sizes, if max_ptes_none >= HPAGE_PMD_NR/2 (255 on > >>>>>>>>> 4K page size), it will be automatically capped to HPAGE_PMD_NR/2 - 1 for > >>>>>>>>> mTHP collapses to prevent collapse "creep" behavior. This prevents > >>>>>>>>> constantly promoting mTHPs to the next available size, which would occur > >>>>>>>>> because a collapse introduces more non-zero pages that would satisfy the > >>>>>>>>> promotion condition on subsequent scans. > >>>>>>>> > >>>>>>>> Hm. Maybe instead of capping at HPAGE_PMD_NR/2 - 1 we can count > >>>>>>>> all-zeros 4k as none_or_zero? It mirrors the logic of shrinker. > >>>>>>> > >>>>>>> BTW, I thought further about this and I agree: if we count zero-filled > >>>>>>> pages towards none_or_zero one we can avoid the "creep" problem. > >>>>>>> > >>>>>>> The scanning-for-zero part is rather nasty, though. > >>>>>> > >>>>>> Aaand, thinking again from the other direction, this would mean that just > >>>>>> because pages became zero after some time that we would no longer collapse > >>>>>> because none_or_zero would then be higher. Hm .... > >>>>>> > >>>>>> How I hate all of this so very very much :) > >>>>> > >>>>> This is not new. Shrinker has the same problem: it cannot distinguish > >>>>> between hot 4k that happened to be zero from the 4k that is there just > >>>>> because of we faulted in 2M a time. > >>>> > >>>> Right. And so far that problem is isolated to the shrinker. > >>>> > >>>> To me so far "none_or_zero" really meant "will I consume more memory when > >>>> collapsing". That's not true for zero-filled pages, obviously. > >>> > >>> Well, KSM can reclaim these zero-filled memory until we collapse it. > >> > >> KSM is used so rarely (for good reasons) that I would never ever build an > >> argument based on its existence :P > >> > >> But yes: during the very first shrinker discussion I raised that KSM can do > >> the same thing. Obviously that was not good enough. > >> > >> -- > >> Cheers > >> > >> David / dhildenb > >> > > > > With all this stuff said, do we have an actual plan for what we intend to do > > _now_? > > Oh no, no I have to use my brain and it's Friday evening. > > > > > As Nico has implemented a basic solution here that we all seem to agree is not > > what we want. > > > > Without needing special new hardware or major reworks, what would this parameter > > look like? > > > > What would the heuristics be? What about the eagerness scales? > > > > I'm but a simple kernel developer, > > :) > > and interested in simple pragmatic stuff :) > > do you have a plan right now David? > > Ehm, if you ask me that way ... > > > > > Maybe we can start with something simple like a rough percentage per eagerness > > entry that then gets scaled based on utilisation? > > ... I think we should probably: > > 1) Start with something very simple for mTHP that doesn't lock us into any particular direction. > > 2) Add an "eagerness" parameter with fixed scale and use that for mTHP as well I think the best design is to map to different max_ptes_none values, 0-5: 0,32,64,128,255,511 > > 3) Improve that "eagerness" algorithm using a dynamic scale or #whatever > > 4) Solve world peace and world hunger > > 5) Connect it all to memory pressure / reclaim / shrinker / heuristics / hw hotness / #whatever > > > I maintain my initial position that just using > > max_ptes_none == 511 -> collapse mTHP always > max_ptes_none != 511 -> collapse mTHP only if we all PTEs are non-none/zero I think we should implement the eagerness toggle, and map it to different max_pte_none values like I described above. This fits nicely in the current collapse_max_ptes_none() function. If we go with just 0/511 without the eagerness changes, we will be removing configurability, only to reintroduce it again. When we can leave the configurability from the start. > > As a starting point is probably simple and best, and likely leaves room for any > changes later. > > > Of course, we could do what Nico is proposing here, as 1) and change it all later. I dont think this is much different than the eagerness approach; it just compresses the max_ptes_none from 0-512 to 0-5/10. I will wait for your RFC for the next version. Does your implementation/thoughts align with what I describe above? > > It's just when it comes to documenting all that stuff in patch #15 that I feel like > "alright, we shouldn't be doing it longterm like that, so let's not make anybody > depend on any weird behavior here by over-domenting it". > > I mean > > " > +To prevent "creeping" behavior where collapses continuously promote to larger > +orders, if max_ptes_none >= HPAGE_PMD_NR/2 (255 on 4K page size), it is > +capped to HPAGE_PMD_NR/2 - 1 for mTHP collapses. This is due to the fact > +that introducing more than half of the pages to be non-zero it will always > +satisfy the eligibility check on the next scan and the region will be collapse. > " > > Is just way, way to detailed. > > I would just say "The kernel might decide to use a more conservative approach > when collapsing smaller THPs" etc. Sounds good I can make it more ambiguous! Cheers. -- Nico > > > Thoughts? > > -- > Cheers > > David / dhildenb >
On Fri, Sep 12, 2025 at 07:53:22PM +0200, David Hildenbrand wrote: > On 12.09.25 17:51, Lorenzo Stoakes wrote: > > With all this stuff said, do we have an actual plan for what we intend to do > > _now_? > > Oh no, no I have to use my brain and it's Friday evening. I apologise :) > > > > > As Nico has implemented a basic solution here that we all seem to agree is not > > what we want. > > > > Without needing special new hardware or major reworks, what would this parameter > > look like? > > > > What would the heuristics be? What about the eagerness scales? > > > > I'm but a simple kernel developer, > > :) > > and interested in simple pragmatic stuff :) > > do you have a plan right now David? > > Ehm, if you ask me that way ... > > > > > Maybe we can start with something simple like a rough percentage per eagerness > > entry that then gets scaled based on utilisation? > > ... I think we should probably: > > 1) Start with something very simple for mTHP that doesn't lock us into any particular direction. Yes. > > 2) Add an "eagerness" parameter with fixed scale and use that for mTHP as well Yes I think we're all pretty onboard with that it seems! > > 3) Improve that "eagerness" algorithm using a dynamic scale or #whatever Right, I feel like we could start with some very simple linear thing here and later maybe refine it? > > 4) Solve world peace and world hunger Yes! That would be pretty great ;) > > 5) Connect it all to memory pressure / reclaim / shrinker / heuristics / hw hotness / #whatever I think these are TODOs :) > > > I maintain my initial position that just using > > max_ptes_none == 511 -> collapse mTHP always > max_ptes_none != 511 -> collapse mTHP only if we all PTEs are non-none/zero > > As a starting point is probably simple and best, and likely leaves room for any > changes later. Yes. > > > Of course, we could do what Nico is proposing here, as 1) and change it all later. Right. But that does mean for mTHP we're limited to 256 (or 255 was it?) but I guess given the 'creep' issue that's sensible. > > It's just when it comes to documenting all that stuff in patch #15 that I feel like > "alright, we shouldn't be doing it longterm like that, so let's not make anybody > depend on any weird behavior here by over-domenting it". > > I mean > > " > +To prevent "creeping" behavior where collapses continuously promote to larger > +orders, if max_ptes_none >= HPAGE_PMD_NR/2 (255 on 4K page size), it is > +capped to HPAGE_PMD_NR/2 - 1 for mTHP collapses. This is due to the fact > +that introducing more than half of the pages to be non-zero it will always > +satisfy the eligibility check on the next scan and the region will be collapse. > " > > Is just way, way to detailed. > > I would just say "The kernel might decide to use a more conservative approach > when collapsing smaller THPs" etc. > > > Thoughts? Well I've sort of reviewed oppositely there :) well at least that it needs to be a hell of a lot clearer (I find that comment really compressed and I just don't really understand it). I guess I didn't think about people reading that and relying on it, so maybe we could alternatively make that succinct. But I think it'd be better to say something like "mTHP collapse cannot currently correctly function with half or more of the PTE entries empty, so we cap at just below this level" in this case. > > -- > Cheers > > David / dhildenb > Cheers, Lorenzo
>> >> I would just say "The kernel might decide to use a more conservative approach >> when collapsing smaller THPs" etc. >> >> >> Thoughts? > > Well I've sort of reviewed oppositely there :) well at least that it needs to be > a hell of a lot clearer (I find that comment really compressed and I just don't > really understand it). Right. I think these are just details we should hide from the user. And in particular, not over-document it so we can more easily change semantics later. > > I guess I didn't think about people reading that and relying on it, so maybe we > could alternatively make that succinct. > > But I think it'd be better to say something like "mTHP collapse cannot currently > correctly function with half or more of the PTE entries empty, so we cap at just > below this level" in this case. IMHO we should just say that the value might be reduced for internal purposes and that this behavior might change in the future would likely be good enough. -- Cheers David / dhildenb
On Mon, Sep 15, 2025 at 12:25:54PM +0200, David Hildenbrand wrote: > > > > > > I would just say "The kernel might decide to use a more conservative approach > > > when collapsing smaller THPs" etc. > > > > > > > > > Thoughts? > > > > Well I've sort of reviewed oppositely there :) well at least that it needs to be > > a hell of a lot clearer (I find that comment really compressed and I just don't > > really understand it). > > Right. I think these are just details we should hide from the user. And in > particular, not over-document it so we can more easily change semantics > later. And when we change semantics we can't change comments? I mean maybe we're talking across purposes here, I'm talking about code comments, not the documentation. I agree the documentation should not mention any of this. > > > > > I guess I didn't think about people reading that and relying on it, so maybe we > > could alternatively make that succinct. > > > > But I think it'd be better to say something like "mTHP collapse cannot currently > > correctly function with half or more of the PTE entries empty, so we cap at just > > below this level" in this case. > > IMHO we should just say that the value might be reduced for internal > purposes and that this behavior might change in the future would likely be > good enough. Again, I assume you mean documentation rather than comments? > > -- > Cheers > > David / dhildenb > Cheers, Lorenzo
On 15.09.25 12:32, Lorenzo Stoakes wrote:
> On Mon, Sep 15, 2025 at 12:25:54PM +0200, David Hildenbrand wrote:
>>>>
>>>> I would just say "The kernel might decide to use a more conservative approach
>>>> when collapsing smaller THPs" etc.
>>>>
>>>>
>>>> Thoughts?
>>>
>>> Well I've sort of reviewed oppositely there :) well at least that it needs to be
>>> a hell of a lot clearer (I find that comment really compressed and I just don't
>>> really understand it).
>>
>> Right. I think these are just details we should hide from the user. And in
>> particular, not over-document it so we can more easily change semantics
>> later.
>
> And when we change semantics we can't change comments?
>
> I mean maybe we're talking across purposes here, I'm talking about code
> comments, not the documentation.
>
> I agree the documentation should not mention any of this.
Yes, I was talking about patch #15 ("It's just when it comes to
documenting all that stuff in patch #15").
Comments we can adjust as we please of course :)
--
Cheers
David / dhildenb
On Mon, Sep 15, 2025 at 12:37:28PM +0200, David Hildenbrand wrote:
> On 15.09.25 12:32, Lorenzo Stoakes wrote:
> > On Mon, Sep 15, 2025 at 12:25:54PM +0200, David Hildenbrand wrote:
> > > > >
> > > > > I would just say "The kernel might decide to use a more conservative approach
> > > > > when collapsing smaller THPs" etc.
> > > > >
> > > > >
> > > > > Thoughts?
> > > >
> > > > Well I've sort of reviewed oppositely there :) well at least that it needs to be
> > > > a hell of a lot clearer (I find that comment really compressed and I just don't
> > > > really understand it).
> > >
> > > Right. I think these are just details we should hide from the user. And in
> > > particular, not over-document it so we can more easily change semantics
> > > later.
> >
> > And when we change semantics we can't change comments?
> >
> > I mean maybe we're talking across purposes here, I'm talking about code
> > comments, not the documentation.
> >
> > I agree the documentation should not mention any of this.
>
> Yes, I was talking about patch #15 ("It's just when it comes to documenting
> all that stuff in patch #15").
>
> Comments we can adjust as we please of course :)
Yeah, ok this was just a misunderstanding then! :) We are in agreement.
>
> --
> Cheers
>
> David / dhildenb
>
Cheers, Lorenzo
On Fri, Sep 12, 2025 at 12:22 PM Lorenzo Stoakes <lorenzo.stoakes@oracle.com> wrote: > > On Fri, Sep 12, 2025 at 07:53:22PM +0200, David Hildenbrand wrote: > > On 12.09.25 17:51, Lorenzo Stoakes wrote: > > > With all this stuff said, do we have an actual plan for what we intend to do > > > _now_? > > > > Oh no, no I have to use my brain and it's Friday evening. > > I apologise :) > > > > > > > > > As Nico has implemented a basic solution here that we all seem to agree is not > > > what we want. > > > > > > Without needing special new hardware or major reworks, what would this parameter > > > look like? > > > > > > What would the heuristics be? What about the eagerness scales? > > > > > > I'm but a simple kernel developer, > > > > :) > > > > and interested in simple pragmatic stuff :) > > > do you have a plan right now David? > > > > Ehm, if you ask me that way ... > > > > > > > > Maybe we can start with something simple like a rough percentage per eagerness > > > entry that then gets scaled based on utilisation? > > > > ... I think we should probably: > > > > 1) Start with something very simple for mTHP that doesn't lock us into any particular direction. > > Yes. > > > > > 2) Add an "eagerness" parameter with fixed scale and use that for mTHP as well > > Yes I think we're all pretty onboard with that it seems! > > > > > 3) Improve that "eagerness" algorithm using a dynamic scale or #whatever > > Right, I feel like we could start with some very simple linear thing here and > later maybe refine it? I agree, something like 0,32,64,128,255,511 seem to map well, and is not too different from what im doing with the scaling by (HPAGE_PMD_ORDER - order). > > > > > 4) Solve world peace and world hunger > > Yes! That would be pretty great ;) This should probably be a larger priority > > > > > 5) Connect it all to memory pressure / reclaim / shrinker / heuristics / hw hotness / #whatever > > I think these are TODOs :) > > > > > > > I maintain my initial position that just using > > > > max_ptes_none == 511 -> collapse mTHP always > > max_ptes_none != 511 -> collapse mTHP only if we all PTEs are non-none/zero > > > > As a starting point is probably simple and best, and likely leaves room for any > > changes later. > > Yes. > > > > > > > Of course, we could do what Nico is proposing here, as 1) and change it all later. > > Right. > > But that does mean for mTHP we're limited to 256 (or 255 was it?) but I guess > given the 'creep' issue that's sensible. I dont think thats much different to what david is trying to propose, given eagerness=9 would be 50%. at 10 or 511, no matter what, you will only ever collapse to the largest enabled order. The difference in my approach is that technically, with PMD disabled, and 511, you would still need 50% utilization to collapse, which is not ideal if you always want to collapse to some mTHP size even with 1 page occupied. With davids solution this is solved by never allowing anything in between 255-511. > > > > > It's just when it comes to documenting all that stuff in patch #15 that I feel like > > "alright, we shouldn't be doing it longterm like that, so let's not make anybody > > depend on any weird behavior here by over-domenting it". > > > > I mean > > > > " > > +To prevent "creeping" behavior where collapses continuously promote to larger > > +orders, if max_ptes_none >= HPAGE_PMD_NR/2 (255 on 4K page size), it is > > +capped to HPAGE_PMD_NR/2 - 1 for mTHP collapses. This is due to the fact > > +that introducing more than half of the pages to be non-zero it will always > > +satisfy the eligibility check on the next scan and the region will be collapse. > > " > > > > Is just way, way to detailed. > > > > I would just say "The kernel might decide to use a more conservative approach > > when collapsing smaller THPs" etc. > > > > > > Thoughts? > > Well I've sort of reviewed oppositely there :) well at least that it needs to be > a hell of a lot clearer (I find that comment really compressed and I just don't > really understand it). I think your review is still valid to improve the internal code comment. I think David is suggesting to not be so specific in the actual admin-guide docs as we move towards a more opaque tunable. > > I guess I didn't think about people reading that and relying on it, so maybe we > could alternatively make that succinct. > > But I think it'd be better to say something like "mTHP collapse cannot currently > correctly function with half or more of the PTE entries empty, so we cap at just > below this level" in this case. Some middle ground might be the best answer, not too specific, but also allude to the interworking a little. Cheers, -- Nico > > > > > -- > > Cheers > > > > David / dhildenb > > > > Cheers, Lorenzo >
On Fri, Sep 12, 2025 at 06:28:55PM -0600, Nico Pache wrote: > On Fri, Sep 12, 2025 at 12:22 PM Lorenzo Stoakes > <lorenzo.stoakes@oracle.com> wrote: > > > > On Fri, Sep 12, 2025 at 07:53:22PM +0200, David Hildenbrand wrote: > > > On 12.09.25 17:51, Lorenzo Stoakes wrote: > > > > With all this stuff said, do we have an actual plan for what we intend to do > > > > _now_? > > > > > > Oh no, no I have to use my brain and it's Friday evening. > > > > I apologise :) > > > > > > > > > > > > > As Nico has implemented a basic solution here that we all seem to agree is not > > > > what we want. > > > > > > > > Without needing special new hardware or major reworks, what would this parameter > > > > look like? > > > > > > > > What would the heuristics be? What about the eagerness scales? > > > > > > > > I'm but a simple kernel developer, > > > > > > :) > > > > > > and interested in simple pragmatic stuff :) > > > > do you have a plan right now David? > > > > > > Ehm, if you ask me that way ... > > > > > > > > > > > Maybe we can start with something simple like a rough percentage per eagerness > > > > entry that then gets scaled based on utilisation? > > > > > > ... I think we should probably: > > > > > > 1) Start with something very simple for mTHP that doesn't lock us into any particular direction. > > > > Yes. > > > > > > > > 2) Add an "eagerness" parameter with fixed scale and use that for mTHP as well > > > > Yes I think we're all pretty onboard with that it seems! > > > > > > > > 3) Improve that "eagerness" algorithm using a dynamic scale or #whatever > > > > Right, I feel like we could start with some very simple linear thing here and > > later maybe refine it? > > I agree, something like 0,32,64,128,255,511 seem to map well, and is > not too different from what im doing with the scaling by > (HPAGE_PMD_ORDER - order). Actually, I suspect something like what David suggests in [0] is probably the better way, but as I said there I think it should be an internal implementation detail as to what this ultimately ends up being. The idea is we provide an abstract thing a user can set, and the kernel figures out how best to interpret that. [0]:https://lore.kernel.org/linux-mm/cd8e7f1c-a563-4ae9-a0fb-b0d04a4c35b4@redhat.com/ > > > > > > > > > 4) Solve world peace and world hunger > > > > Yes! That would be pretty great ;) > This should probably be a larger priority :))) > > > > > > > > 5) Connect it all to memory pressure / reclaim / shrinker / heuristics / hw hotness / #whatever > > > > I think these are TODOs :) > > > > > > > > > > > I maintain my initial position that just using > > > > > > max_ptes_none == 511 -> collapse mTHP always > > > max_ptes_none != 511 -> collapse mTHP only if we all PTEs are non-none/zero > > > > > > As a starting point is probably simple and best, and likely leaves room for any > > > changes later. > > > > Yes. > > > > > > > > > > > Of course, we could do what Nico is proposing here, as 1) and change it all later. > > > > Right. > > > > But that does mean for mTHP we're limited to 256 (or 255 was it?) but I guess > > given the 'creep' issue that's sensible. > > I dont think thats much different to what david is trying to propose, > given eagerness=9 would be 50%. I think q > at 10 or 511, no matter what, you will only ever collapse to the > largest enabled order. > The difference in my approach is that technically, with PMD disabled, > and 511, you would still need 50% utilization to collapse, which is > not ideal if you always want to collapse to some mTHP size even with 1 > page occupied. With davids solution this is solved by never allowing > anything in between 255-511. Right. Except we default to max eagerness (or min, I asked David about the values there :P) So aren't we, by default, broken on mTHP? Maybe we can change the default though... > > > > > > > > > It's just when it comes to documenting all that stuff in patch #15 that I feel like > > > "alright, we shouldn't be doing it longterm like that, so let's not make anybody > > > depend on any weird behavior here by over-domenting it". > > > > > > I mean > > > > > > " > > > +To prevent "creeping" behavior where collapses continuously promote to larger > > > +orders, if max_ptes_none >= HPAGE_PMD_NR/2 (255 on 4K page size), it is > > > +capped to HPAGE_PMD_NR/2 - 1 for mTHP collapses. This is due to the fact > > > +that introducing more than half of the pages to be non-zero it will always > > > +satisfy the eligibility check on the next scan and the region will be collapse. > > > " > > > > > > Is just way, way to detailed. > > > > > > I would just say "The kernel might decide to use a more conservative approach > > > when collapsing smaller THPs" etc. > > > > > > > > > Thoughts? > > > > Well I've sort of reviewed oppositely there :) well at least that it needs to be > > a hell of a lot clearer (I find that comment really compressed and I just don't > > really understand it). > > I think your review is still valid to improve the internal code > comment. I think David is suggesting to not be so specific in the > actual admin-guide docs as we move towards a more opaque tunable. Yeah thanks for pointing that out! We were talking across purposes. > > > > > I guess I didn't think about people reading that and relying on it, so maybe we > > could alternatively make that succinct. > > > > But I think it'd be better to say something like "mTHP collapse cannot currently > > correctly function with half or more of the PTE entries empty, so we cap at just > > below this level" in this case. > > Some middle ground might be the best answer, not too specific, but > also allude to the interworking a little. Yeah actually I agree with David re: documentation, my comments were wrt err... comments :P only. > > Cheers, > -- Nico Cheers, Lorenzo
On 12.09.25 14:19, Kiryl Shutsemau wrote: > On Thu, Sep 11, 2025 at 09:27:55PM -0600, Nico Pache wrote: >> The following series provides khugepaged with the capability to collapse >> anonymous memory regions to mTHPs. >> >> To achieve this we generalize the khugepaged functions to no longer depend >> on PMD_ORDER. Then during the PMD scan, we use a bitmap to track individual >> pages that are occupied (!none/zero). After the PMD scan is done, we do >> binary recursion on the bitmap to find the optimal mTHP sizes for the PMD >> range. The restriction on max_ptes_none is removed during the scan, to make >> sure we account for the whole PMD range. When no mTHP size is enabled, the >> legacy behavior of khugepaged is maintained. max_ptes_none will be scaled >> by the attempted collapse order to determine how full a mTHP must be to be >> eligible for the collapse to occur. If a mTHP collapse is attempted, but >> contains swapped out, or shared pages, we don't perform the collapse. It is >> now also possible to collapse to mTHPs without requiring the PMD THP size >> to be enabled. >> >> When enabling (m)THP sizes, if max_ptes_none >= HPAGE_PMD_NR/2 (255 on >> 4K page size), it will be automatically capped to HPAGE_PMD_NR/2 - 1 for >> mTHP collapses to prevent collapse "creep" behavior. This prevents >> constantly promoting mTHPs to the next available size, which would occur >> because a collapse introduces more non-zero pages that would satisfy the >> promotion condition on subsequent scans. > > Hm. Maybe instead of capping at HPAGE_PMD_NR/2 - 1 we can count > all-zeros 4k as none_or_zero? It mirrors the logic of shrinker. > I am all for not adding any more ugliness on top of all the ugliness we added in the past. I will soon propose deprecating that parameter in favor of something that makes a bit more sense. In essence, we'll likely have an "eagerness" parameter that ranges from 0 to 10. 10 is essentially "always collapse" and 0 "never collapse if not all is populated". In between we will have more flexibility on how to set these values. Likely 9 will be around 50% to not even motivate the user to set something that does not make sense (creep). Of course, the old parameter will have to stick around in compat mode. -- Cheers David / dhildenb
On Fri, Sep 12, 2025 at 6:25 AM David Hildenbrand <david@redhat.com> wrote: > > On 12.09.25 14:19, Kiryl Shutsemau wrote: > > On Thu, Sep 11, 2025 at 09:27:55PM -0600, Nico Pache wrote: > >> The following series provides khugepaged with the capability to collapse > >> anonymous memory regions to mTHPs. > >> > >> To achieve this we generalize the khugepaged functions to no longer depend > >> on PMD_ORDER. Then during the PMD scan, we use a bitmap to track individual > >> pages that are occupied (!none/zero). After the PMD scan is done, we do > >> binary recursion on the bitmap to find the optimal mTHP sizes for the PMD > >> range. The restriction on max_ptes_none is removed during the scan, to make > >> sure we account for the whole PMD range. When no mTHP size is enabled, the > >> legacy behavior of khugepaged is maintained. max_ptes_none will be scaled > >> by the attempted collapse order to determine how full a mTHP must be to be > >> eligible for the collapse to occur. If a mTHP collapse is attempted, but > >> contains swapped out, or shared pages, we don't perform the collapse. It is > >> now also possible to collapse to mTHPs without requiring the PMD THP size > >> to be enabled. > >> > >> When enabling (m)THP sizes, if max_ptes_none >= HPAGE_PMD_NR/2 (255 on > >> 4K page size), it will be automatically capped to HPAGE_PMD_NR/2 - 1 for > >> mTHP collapses to prevent collapse "creep" behavior. This prevents > >> constantly promoting mTHPs to the next available size, which would occur > >> because a collapse introduces more non-zero pages that would satisfy the > >> promotion condition on subsequent scans. > > > > Hm. Maybe instead of capping at HPAGE_PMD_NR/2 - 1 we can count > > all-zeros 4k as none_or_zero? It mirrors the logic of shrinker. > > > > I am all for not adding any more ugliness on top of all the ugliness we > added in the past. > > I will soon propose deprecating that parameter in favor of something > that makes a bit more sense. > > In essence, we'll likely have an "eagerness" parameter that ranges from > 0 to 10. 10 is essentially "always collapse" and 0 "never collapse if > not all is populated". Hi David, Do you have any reason for 0-10, I'm guessing these will map to different max_ptes_none values. I suggest 0-5, mapping to 0,32,64,128,255,511 You can take my collapse_max_ptes_none() function in this series and rework it for the larger sysctl work you are doing. Cheers, -- Nico > > In between we will have more flexibility on how to set these values. > > Likely 9 will be around 50% to not even motivate the user to set > something that does not make sense (creep). > > Of course, the old parameter will have to stick around in compat mode. > > -- > Cheers > > David / dhildenb >
On Fri, Sep 12, 2025 at 05:31:51PM -0600, Nico Pache wrote: > On Fri, Sep 12, 2025 at 6:25 AM David Hildenbrand <david@redhat.com> wrote: > > > > On 12.09.25 14:19, Kiryl Shutsemau wrote: > > > On Thu, Sep 11, 2025 at 09:27:55PM -0600, Nico Pache wrote: > > >> The following series provides khugepaged with the capability to collapse > > >> anonymous memory regions to mTHPs. > > >> > > >> To achieve this we generalize the khugepaged functions to no longer depend > > >> on PMD_ORDER. Then during the PMD scan, we use a bitmap to track individual > > >> pages that are occupied (!none/zero). After the PMD scan is done, we do > > >> binary recursion on the bitmap to find the optimal mTHP sizes for the PMD > > >> range. The restriction on max_ptes_none is removed during the scan, to make > > >> sure we account for the whole PMD range. When no mTHP size is enabled, the > > >> legacy behavior of khugepaged is maintained. max_ptes_none will be scaled > > >> by the attempted collapse order to determine how full a mTHP must be to be > > >> eligible for the collapse to occur. If a mTHP collapse is attempted, but > > >> contains swapped out, or shared pages, we don't perform the collapse. It is > > >> now also possible to collapse to mTHPs without requiring the PMD THP size > > >> to be enabled. > > >> > > >> When enabling (m)THP sizes, if max_ptes_none >= HPAGE_PMD_NR/2 (255 on > > >> 4K page size), it will be automatically capped to HPAGE_PMD_NR/2 - 1 for > > >> mTHP collapses to prevent collapse "creep" behavior. This prevents > > >> constantly promoting mTHPs to the next available size, which would occur > > >> because a collapse introduces more non-zero pages that would satisfy the > > >> promotion condition on subsequent scans. > > > > > > Hm. Maybe instead of capping at HPAGE_PMD_NR/2 - 1 we can count > > > all-zeros 4k as none_or_zero? It mirrors the logic of shrinker. > > > > > > > I am all for not adding any more ugliness on top of all the ugliness we > > added in the past. > > > > I will soon propose deprecating that parameter in favor of something > > that makes a bit more sense. > > > > In essence, we'll likely have an "eagerness" parameter that ranges from > > 0 to 10. 10 is essentially "always collapse" and 0 "never collapse if > > not all is populated". > Hi David, > > Do you have any reason for 0-10, I'm guessing these will map to > different max_ptes_none values. > I suggest 0-5, mapping to 0,32,64,128,255,511 That's too x86-64 specific. And the whole idea is not to map to directly, but give kernel wiggle room to play. -- Kiryl Shutsemau / Kirill A. Shutemov
On 15.09.25 11:22, Kiryl Shutsemau wrote: > On Fri, Sep 12, 2025 at 05:31:51PM -0600, Nico Pache wrote: >> On Fri, Sep 12, 2025 at 6:25 AM David Hildenbrand <david@redhat.com> wrote: >>> >>> On 12.09.25 14:19, Kiryl Shutsemau wrote: >>>> On Thu, Sep 11, 2025 at 09:27:55PM -0600, Nico Pache wrote: >>>>> The following series provides khugepaged with the capability to collapse >>>>> anonymous memory regions to mTHPs. >>>>> >>>>> To achieve this we generalize the khugepaged functions to no longer depend >>>>> on PMD_ORDER. Then during the PMD scan, we use a bitmap to track individual >>>>> pages that are occupied (!none/zero). After the PMD scan is done, we do >>>>> binary recursion on the bitmap to find the optimal mTHP sizes for the PMD >>>>> range. The restriction on max_ptes_none is removed during the scan, to make >>>>> sure we account for the whole PMD range. When no mTHP size is enabled, the >>>>> legacy behavior of khugepaged is maintained. max_ptes_none will be scaled >>>>> by the attempted collapse order to determine how full a mTHP must be to be >>>>> eligible for the collapse to occur. If a mTHP collapse is attempted, but >>>>> contains swapped out, or shared pages, we don't perform the collapse. It is >>>>> now also possible to collapse to mTHPs without requiring the PMD THP size >>>>> to be enabled. >>>>> >>>>> When enabling (m)THP sizes, if max_ptes_none >= HPAGE_PMD_NR/2 (255 on >>>>> 4K page size), it will be automatically capped to HPAGE_PMD_NR/2 - 1 for >>>>> mTHP collapses to prevent collapse "creep" behavior. This prevents >>>>> constantly promoting mTHPs to the next available size, which would occur >>>>> because a collapse introduces more non-zero pages that would satisfy the >>>>> promotion condition on subsequent scans. >>>> >>>> Hm. Maybe instead of capping at HPAGE_PMD_NR/2 - 1 we can count >>>> all-zeros 4k as none_or_zero? It mirrors the logic of shrinker. >>>> >>> >>> I am all for not adding any more ugliness on top of all the ugliness we >>> added in the past. >>> >>> I will soon propose deprecating that parameter in favor of something >>> that makes a bit more sense. >>> >>> In essence, we'll likely have an "eagerness" parameter that ranges from >>> 0 to 10. 10 is essentially "always collapse" and 0 "never collapse if >>> not all is populated". >> Hi David, >> >> Do you have any reason for 0-10, I'm guessing these will map to >> different max_ptes_none values. >> I suggest 0-5, mapping to 0,32,64,128,255,511 > > That's too x86-64 specific. > > And the whole idea is not to map to directly, but give kernel wiggle > room to play. Initially we will start out simple and map it directly. But yeah, the idea is to give us some more room later. I had something logarithmic in mind which would roughly be (ignoring the the weird -1 for simplicity and expressing it as "used" instead of none-or-zero) 0 -> ~100% used (~0% none) 1 -> ~50% used (~50% none) 2 -> ~25% used (~75% none) 3 -> ~12.5% used (~87.5% none) 4 -> ~11.25% used (~88,75% none) ... 10 -> ~0% used (~100% none) Mapping that to actual THP sizes (#pages in a thp) on an arch will be easy. -- Cheers David / dhildenb
On Mon, Sep 15, 2025 at 4:22 AM David Hildenbrand <david@redhat.com> wrote:
>
> On 15.09.25 11:22, Kiryl Shutsemau wrote:
> > On Fri, Sep 12, 2025 at 05:31:51PM -0600, Nico Pache wrote:
> >> On Fri, Sep 12, 2025 at 6:25 AM David Hildenbrand <david@redhat.com> wrote:
> >>>
> >>> On 12.09.25 14:19, Kiryl Shutsemau wrote:
> >>>> On Thu, Sep 11, 2025 at 09:27:55PM -0600, Nico Pache wrote:
> >>>>> The following series provides khugepaged with the capability to collapse
> >>>>> anonymous memory regions to mTHPs.
> >>>>>
> >>>>> To achieve this we generalize the khugepaged functions to no longer depend
> >>>>> on PMD_ORDER. Then during the PMD scan, we use a bitmap to track individual
> >>>>> pages that are occupied (!none/zero). After the PMD scan is done, we do
> >>>>> binary recursion on the bitmap to find the optimal mTHP sizes for the PMD
> >>>>> range. The restriction on max_ptes_none is removed during the scan, to make
> >>>>> sure we account for the whole PMD range. When no mTHP size is enabled, the
> >>>>> legacy behavior of khugepaged is maintained. max_ptes_none will be scaled
> >>>>> by the attempted collapse order to determine how full a mTHP must be to be
> >>>>> eligible for the collapse to occur. If a mTHP collapse is attempted, but
> >>>>> contains swapped out, or shared pages, we don't perform the collapse. It is
> >>>>> now also possible to collapse to mTHPs without requiring the PMD THP size
> >>>>> to be enabled.
> >>>>>
> >>>>> When enabling (m)THP sizes, if max_ptes_none >= HPAGE_PMD_NR/2 (255 on
> >>>>> 4K page size), it will be automatically capped to HPAGE_PMD_NR/2 - 1 for
> >>>>> mTHP collapses to prevent collapse "creep" behavior. This prevents
> >>>>> constantly promoting mTHPs to the next available size, which would occur
> >>>>> because a collapse introduces more non-zero pages that would satisfy the
> >>>>> promotion condition on subsequent scans.
> >>>>
> >>>> Hm. Maybe instead of capping at HPAGE_PMD_NR/2 - 1 we can count
> >>>> all-zeros 4k as none_or_zero? It mirrors the logic of shrinker.
> >>>>
> >>>
> >>> I am all for not adding any more ugliness on top of all the ugliness we
> >>> added in the past.
> >>>
> >>> I will soon propose deprecating that parameter in favor of something
> >>> that makes a bit more sense.
> >>>
> >>> In essence, we'll likely have an "eagerness" parameter that ranges from
> >>> 0 to 10. 10 is essentially "always collapse" and 0 "never collapse if
> >>> not all is populated".
> >> Hi David,
> >>
> >> Do you have any reason for 0-10, I'm guessing these will map to
> >> different max_ptes_none values.
> >> I suggest 0-5, mapping to 0,32,64,128,255,511
> >
> > That's too x86-64 specific.
Its technically formulated from:
X = ( HPAGE_PMD_NR >> (5 - n) ) - 1
where n is the value of eagerness and X is the number of none_ptes we allow
so 5 == (512 >> 0) - 1 = 511
4 == (512 >> 1) - 1 = 255
3 == 128 - 1 = 127
....
Any scale we use will suffer from inaccuracy
Currently this fits well into the bitmap algorithm because the lower
you go in the bitmap (smaller orders), the more inaccurate the
max_ptes_none (or any scale to that matter) will have on the value.
for example: a 16kB mTHP is 4 pages. you really only have 4 options
for the number of none_ptes you will allow, so any scale will be
rounded heavily towards the lower orders.
128 (max_ptes_none) >> (9 (pmd_order) - 2 (collapse order)) = 1 none pte allowed
255 >> 7 = 1 none_pte allowed
no value inbetween these has any effect
where as
127 >> 7 = 0
So using a consistent scale that is relative to the number of PTEs in
a given mTHP I think is the most straightforward approach.
> >
> > And the whole idea is not to map to directly, but give kernel wiggle
> > room to play.
>
> Initially we will start out simple and map it directly. But yeah, the
> idea is to give us some more room later.
>
> I had something logarithmic in mind which would roughly be (ignoring the
> the weird -1 for simplicity and expressing it as "used" instead of
> none-or-zero)
>
> 0 -> ~100% used (~0% none)
> 1 -> ~50% used (~50% none)
> 2 -> ~25% used (~75% none)
> 3 -> ~12.5% used (~87.5% none)
> 4 -> ~11.25% used (~88,75% none)
> ...
> 10 -> ~0% used (~100% none)
I think this scale is too specific, I think it would be easier to map
to the one above for the reasons stated there. There would be little
to no benefit to having such small adjustments between 4-10
Let me know what you think
-- Nico
>
> Mapping that to actual THP sizes (#pages in a thp) on an arch will be easy.
>
> --
> Cheers
>
> David / dhildenb
>
>> ... >> 10 -> ~0% used (~100% none) > I think this scale is too specific, I think it would be easier to map > to the one above for the reasons stated there. There would be little > to no benefit to having such small adjustments between 4-10 It's probably best to discuss that once I have something more concrete to share :) I yet have to think about some cases I have in mind, and once I figured them out (and had time to do so ...) I'll post something. -- Cheers David / dhildenb
On Mon, Sep 15, 2025 at 12:22:07PM +0200, David Hildenbrand wrote: > > 0 -> ~100% used (~0% none) > 1 -> ~50% used (~50% none) > 2 -> ~25% used (~75% none) > 3 -> ~12.5% used (~87.5% none) > 4 -> ~11.25% used (~88,75% none) > ... > 10 -> ~0% used (~100% none) Oh and shouldn't this be inverted? 0 eagerness = we eat up all none PTE entries? Isn't that pretty eager? :P 10 eagerness = we aren't eager to eat up none PTE entries at all? Or am I being dumb here?
On 15.09.25 12:43, Lorenzo Stoakes wrote: > On Mon, Sep 15, 2025 at 12:22:07PM +0200, David Hildenbrand wrote: >> >> 0 -> ~100% used (~0% none) >> 1 -> ~50% used (~50% none) >> 2 -> ~25% used (~75% none) >> 3 -> ~12.5% used (~87.5% none) >> 4 -> ~11.25% used (~88,75% none) >> ... >> 10 -> ~0% used (~100% none) > > Oh and shouldn't this be inverted? > > 0 eagerness = we eat up all none PTE entries? Isn't that pretty eager? :P > 10 eagerness = we aren't eager to eat up none PTE entries at all? > > Or am I being dumb here? Good question. For swappiness it's: 0 -> no swap (conservative) So intuitively I assumed: 0 -> no pte_none (conservative) You're the native speaker, so you tell me :) -- Cheers David / dhildenb
On Mon, Sep 15, 2025 at 12:52:03PM +0200, David Hildenbrand wrote: > On 15.09.25 12:43, Lorenzo Stoakes wrote: > > On Mon, Sep 15, 2025 at 12:22:07PM +0200, David Hildenbrand wrote: > > > > > > 0 -> ~100% used (~0% none) > > > 1 -> ~50% used (~50% none) > > > 2 -> ~25% used (~75% none) > > > 3 -> ~12.5% used (~87.5% none) > > > 4 -> ~11.25% used (~88,75% none) > > > ... > > > 10 -> ~0% used (~100% none) > > > > Oh and shouldn't this be inverted? > > > > 0 eagerness = we eat up all none PTE entries? Isn't that pretty eager? :P > > 10 eagerness = we aren't eager to eat up none PTE entries at all? > > > > Or am I being dumb here? > > Good question. > > For swappiness it's: 0 -> no swap (conservative) > > So intuitively I assumed: 0 -> no pte_none (conservative) > > You're the native speaker, so you tell me :) To me this is about 'eagerness to consume empty PTE entries' so 10 is more eager, 0 is not eager at all, i.e. inversion of what you suggest :) > > -- > Cheers > > David / dhildenb > Cheers, Lorenzo
On 15.09.25 13:02, Lorenzo Stoakes wrote: > On Mon, Sep 15, 2025 at 12:52:03PM +0200, David Hildenbrand wrote: >> On 15.09.25 12:43, Lorenzo Stoakes wrote: >>> On Mon, Sep 15, 2025 at 12:22:07PM +0200, David Hildenbrand wrote: >>>> >>>> 0 -> ~100% used (~0% none) >>>> 1 -> ~50% used (~50% none) >>>> 2 -> ~25% used (~75% none) >>>> 3 -> ~12.5% used (~87.5% none) >>>> 4 -> ~11.25% used (~88,75% none) >>>> ... >>>> 10 -> ~0% used (~100% none) >>> >>> Oh and shouldn't this be inverted? >>> >>> 0 eagerness = we eat up all none PTE entries? Isn't that pretty eager? :P >>> 10 eagerness = we aren't eager to eat up none PTE entries at all? >>> >>> Or am I being dumb here? >> >> Good question. >> >> For swappiness it's: 0 -> no swap (conservative) >> >> So intuitively I assumed: 0 -> no pte_none (conservative) >> >> You're the native speaker, so you tell me :) > > To me this is about 'eagerness to consume empty PTE entries' so 10 is more > eager, 0 is not eager at all, i.e. inversion of what you suggest :) Just so we are on the same page: it is about "eagerness to collapse", right? Wouldn't a 0 mean "I am not eager, I will not waste any memory, I am very careful and bail out on any pte_none" vs. 10 meaning "I am very eager, I will collapse no matter what I find in the page table, waste as much memory as I want"? -- Cheers David / dhildenb
On Mon, Sep 15, 2025 at 01:14:32PM +0200, David Hildenbrand wrote: > On 15.09.25 13:02, Lorenzo Stoakes wrote: > > On Mon, Sep 15, 2025 at 12:52:03PM +0200, David Hildenbrand wrote: > > > On 15.09.25 12:43, Lorenzo Stoakes wrote: > > > > On Mon, Sep 15, 2025 at 12:22:07PM +0200, David Hildenbrand wrote: > > > > > > > > > > 0 -> ~100% used (~0% none) > > > > > 1 -> ~50% used (~50% none) > > > > > 2 -> ~25% used (~75% none) > > > > > 3 -> ~12.5% used (~87.5% none) > > > > > 4 -> ~11.25% used (~88,75% none) > > > > > ... > > > > > 10 -> ~0% used (~100% none) > > > > > > > > Oh and shouldn't this be inverted? > > > > > > > > 0 eagerness = we eat up all none PTE entries? Isn't that pretty eager? :P > > > > 10 eagerness = we aren't eager to eat up none PTE entries at all? > > > > > > > > Or am I being dumb here? > > > > > > Good question. > > > > > > For swappiness it's: 0 -> no swap (conservative) > > > > > > So intuitively I assumed: 0 -> no pte_none (conservative) > > > > > > You're the native speaker, so you tell me :) > > > > To me this is about 'eagerness to consume empty PTE entries' so 10 is more > > eager, 0 is not eager at all, i.e. inversion of what you suggest :) > > Just so we are on the same page: it is about "eagerness to collapse", right? > > Wouldn't a 0 mean "I am not eager, I will not waste any memory, I am very > careful and bail out on any pte_none" vs. 10 meaning "I am very eager, I > will collapse no matter what I find in the page table, waste as much memory > as I want"? Yeah, this is my understanding of your scale, or is my understanding also inverted? :) Right now it's: eagerness max_ptes_none 0 -> 511 ... 10 -> 0 Right? So we're saying, currently, 0 means 'I will tolerate up to 511 pte_none, and eat them all I am very very eager', and 10 means 'I will not tolerate any pte_none' right? Correct me if I'm wrong here! :>) > > -- > Cheers > > David / dhildenb > Cheers, Lorenzo
On 15.09.25 13:23, Lorenzo Stoakes wrote: > On Mon, Sep 15, 2025 at 01:14:32PM +0200, David Hildenbrand wrote: >> On 15.09.25 13:02, Lorenzo Stoakes wrote: >>> On Mon, Sep 15, 2025 at 12:52:03PM +0200, David Hildenbrand wrote: >>>> On 15.09.25 12:43, Lorenzo Stoakes wrote: >>>>> On Mon, Sep 15, 2025 at 12:22:07PM +0200, David Hildenbrand wrote: >>>>>> >>>>>> 0 -> ~100% used (~0% none) >>>>>> 1 -> ~50% used (~50% none) >>>>>> 2 -> ~25% used (~75% none) >>>>>> 3 -> ~12.5% used (~87.5% none) >>>>>> 4 -> ~11.25% used (~88,75% none) >>>>>> ... >>>>>> 10 -> ~0% used (~100% none) >>>>> >>>>> Oh and shouldn't this be inverted? >>>>> >>>>> 0 eagerness = we eat up all none PTE entries? Isn't that pretty eager? :P >>>>> 10 eagerness = we aren't eager to eat up none PTE entries at all? >>>>> >>>>> Or am I being dumb here? >>>> >>>> Good question. >>>> >>>> For swappiness it's: 0 -> no swap (conservative) >>>> >>>> So intuitively I assumed: 0 -> no pte_none (conservative) >>>> >>>> You're the native speaker, so you tell me :) >>> >>> To me this is about 'eagerness to consume empty PTE entries' so 10 is more >>> eager, 0 is not eager at all, i.e. inversion of what you suggest :) >> >> Just so we are on the same page: it is about "eagerness to collapse", right? >> >> Wouldn't a 0 mean "I am not eager, I will not waste any memory, I am very >> careful and bail out on any pte_none" vs. 10 meaning "I am very eager, I >> will collapse no matter what I find in the page table, waste as much memory >> as I want"? > > Yeah, this is my understanding of your scale, or is my understanding also > inverted? :) > > Right now it's: > > eagerness max_ptes_none > > 0 -> 511 > ... > 10 -> 0 > > Right? Just so we are on the same page, this is what I had: 0 -> ~100% used (~0% none) So "0" -> 0 pte_none or 512 used. (note the used vs. none) -- Cheers David / dhildenb
On Mon, Sep 15, 2025 at 01:29:22PM +0200, David Hildenbrand wrote: > On 15.09.25 13:23, Lorenzo Stoakes wrote: > > On Mon, Sep 15, 2025 at 01:14:32PM +0200, David Hildenbrand wrote: > > > On 15.09.25 13:02, Lorenzo Stoakes wrote: > > > > On Mon, Sep 15, 2025 at 12:52:03PM +0200, David Hildenbrand wrote: > > > > > On 15.09.25 12:43, Lorenzo Stoakes wrote: > > > > > > On Mon, Sep 15, 2025 at 12:22:07PM +0200, David Hildenbrand wrote: > > > > > > > > > > > > > > 0 -> ~100% used (~0% none) > > > > > > > 1 -> ~50% used (~50% none) > > > > > > > 2 -> ~25% used (~75% none) > > > > > > > 3 -> ~12.5% used (~87.5% none) > > > > > > > 4 -> ~11.25% used (~88,75% none) > > > > > > > ... > > > > > > > 10 -> ~0% used (~100% none) > > > > > > > > > > > > Oh and shouldn't this be inverted? > > > > > > > > > > > > 0 eagerness = we eat up all none PTE entries? Isn't that pretty eager? :P > > > > > > 10 eagerness = we aren't eager to eat up none PTE entries at all? > > > > > > > > > > > > Or am I being dumb here? > > > > > > > > > > Good question. > > > > > > > > > > For swappiness it's: 0 -> no swap (conservative) > > > > > > > > > > So intuitively I assumed: 0 -> no pte_none (conservative) > > > > > > > > > > You're the native speaker, so you tell me :) > > > > > > > > To me this is about 'eagerness to consume empty PTE entries' so 10 is more > > > > eager, 0 is not eager at all, i.e. inversion of what you suggest :) > > > > > > Just so we are on the same page: it is about "eagerness to collapse", right? > > > > > > Wouldn't a 0 mean "I am not eager, I will not waste any memory, I am very > > > careful and bail out on any pte_none" vs. 10 meaning "I am very eager, I > > > will collapse no matter what I find in the page table, waste as much memory > > > as I want"? > > > > Yeah, this is my understanding of your scale, or is my understanding also > > inverted? :) > > > > Right now it's: > > > > eagerness max_ptes_none > > > > 0 -> 511 > > ... > > 10 -> 0 > > > > Right? > > Just so we are on the same page, this is what I had: > > 0 -> ~100% used (~0% none) > > So "0" -> 0 pte_none or 512 used. > > (note the used vs. none) OK right so we're talking about the same thing, I guess? I was confused partly becuase of the scale, becuase weren't people setting this parameter to low values in practice? And now we make it so we have equivalent of: 0 -> 0 1 -> 256 2 -> 384 With the logarithmic values more tightly bunched at the 'eager' end? Weren't people setting max_ptes_none to like 20 or 30 or something? So we should 'bunch' at the other end? And also aren't we saying that anything over 256 is broken for mTHP? So weren't we trying to avoid that? I think it should be something like: (eagerness -> max_pte_none) 0 -> 0 1 -> ~small % 2 -> ~slightly larger % etc. 9 -> 50% 10 -> 100% Right? > > -- > Cheers > > David / dhildenb > Cheers, Lorenzo
On 15.09.25 13:35, Lorenzo Stoakes wrote: > On Mon, Sep 15, 2025 at 01:29:22PM +0200, David Hildenbrand wrote: >> On 15.09.25 13:23, Lorenzo Stoakes wrote: >>> On Mon, Sep 15, 2025 at 01:14:32PM +0200, David Hildenbrand wrote: >>>> On 15.09.25 13:02, Lorenzo Stoakes wrote: >>>>> On Mon, Sep 15, 2025 at 12:52:03PM +0200, David Hildenbrand wrote: >>>>>> On 15.09.25 12:43, Lorenzo Stoakes wrote: >>>>>>> On Mon, Sep 15, 2025 at 12:22:07PM +0200, David Hildenbrand wrote: >>>>>>>> >>>>>>>> 0 -> ~100% used (~0% none) >>>>>>>> 1 -> ~50% used (~50% none) >>>>>>>> 2 -> ~25% used (~75% none) >>>>>>>> 3 -> ~12.5% used (~87.5% none) >>>>>>>> 4 -> ~11.25% used (~88,75% none) >>>>>>>> ... >>>>>>>> 10 -> ~0% used (~100% none) >>>>>>> >>>>>>> Oh and shouldn't this be inverted? >>>>>>> >>>>>>> 0 eagerness = we eat up all none PTE entries? Isn't that pretty eager? :P >>>>>>> 10 eagerness = we aren't eager to eat up none PTE entries at all? >>>>>>> >>>>>>> Or am I being dumb here? >>>>>> >>>>>> Good question. >>>>>> >>>>>> For swappiness it's: 0 -> no swap (conservative) >>>>>> >>>>>> So intuitively I assumed: 0 -> no pte_none (conservative) >>>>>> >>>>>> You're the native speaker, so you tell me :) >>>>> >>>>> To me this is about 'eagerness to consume empty PTE entries' so 10 is more >>>>> eager, 0 is not eager at all, i.e. inversion of what you suggest :) >>>> >>>> Just so we are on the same page: it is about "eagerness to collapse", right? >>>> >>>> Wouldn't a 0 mean "I am not eager, I will not waste any memory, I am very >>>> careful and bail out on any pte_none" vs. 10 meaning "I am very eager, I >>>> will collapse no matter what I find in the page table, waste as much memory >>>> as I want"? >>> >>> Yeah, this is my understanding of your scale, or is my understanding also >>> inverted? :) >>> >>> Right now it's: >>> >>> eagerness max_ptes_none >>> >>> 0 -> 511 >>> ... >>> 10 -> 0 >>> >>> Right? >> >> Just so we are on the same page, this is what I had: >> >> 0 -> ~100% used (~0% none) >> >> So "0" -> 0 pte_none or 512 used. >> >> (note the used vs. none) > > OK right so we're talking about the same thing, I guess? > > I was confused partly becuase of the scale, becuase weren't people setting > this parameter to low values in practice? > > And now we make it so we have equivalent of: > > 0 -> 0 > 1 -> 256 > 2 -> 384 Ah, there is the problem, that's not what I had in mind. 0 -> ~100% used (~0% none) ... 8 -> ~87,5% used (~12.5% none) 9 -> ~75% used (~25% none) 9 -> ~50% used (~50% none) 10 -> ~0% used (~100% none) Hopefully I didn't mess it up again. -- Cheers David / dhildenb
On Mon, Sep 15, 2025 at 01:45:39PM +0200, David Hildenbrand wrote: > On 15.09.25 13:35, Lorenzo Stoakes wrote: > > On Mon, Sep 15, 2025 at 01:29:22PM +0200, David Hildenbrand wrote: > > > On 15.09.25 13:23, Lorenzo Stoakes wrote: > > > > On Mon, Sep 15, 2025 at 01:14:32PM +0200, David Hildenbrand wrote: > > > > > On 15.09.25 13:02, Lorenzo Stoakes wrote: > > > > > > On Mon, Sep 15, 2025 at 12:52:03PM +0200, David Hildenbrand wrote: > > > > > > > On 15.09.25 12:43, Lorenzo Stoakes wrote: > > > > > > > > On Mon, Sep 15, 2025 at 12:22:07PM +0200, David Hildenbrand wrote: > > > > > > > > > > > > > > > > > > 0 -> ~100% used (~0% none) > > > > > > > > > 1 -> ~50% used (~50% none) > > > > > > > > > 2 -> ~25% used (~75% none) > > > > > > > > > 3 -> ~12.5% used (~87.5% none) > > > > > > > > > 4 -> ~11.25% used (~88,75% none) > > > > > > > > > ... > > > > > > > > > 10 -> ~0% used (~100% none) > > > > > > > > > > > > > > > > Oh and shouldn't this be inverted? > > > > > > > > > > > > > > > > 0 eagerness = we eat up all none PTE entries? Isn't that pretty eager? :P > > > > > > > > 10 eagerness = we aren't eager to eat up none PTE entries at all? > > > > > > > > > > > > > > > > Or am I being dumb here? > > > > > > > > > > > > > > Good question. > > > > > > > > > > > > > > For swappiness it's: 0 -> no swap (conservative) > > > > > > > > > > > > > > So intuitively I assumed: 0 -> no pte_none (conservative) > > > > > > > > > > > > > > You're the native speaker, so you tell me :) > > > > > > > > > > > > To me this is about 'eagerness to consume empty PTE entries' so 10 is more > > > > > > eager, 0 is not eager at all, i.e. inversion of what you suggest :) > > > > > > > > > > Just so we are on the same page: it is about "eagerness to collapse", right? > > > > > > > > > > Wouldn't a 0 mean "I am not eager, I will not waste any memory, I am very > > > > > careful and bail out on any pte_none" vs. 10 meaning "I am very eager, I > > > > > will collapse no matter what I find in the page table, waste as much memory > > > > > as I want"? > > > > > > > > Yeah, this is my understanding of your scale, or is my understanding also > > > > inverted? :) > > > > > > > > Right now it's: > > > > > > > > eagerness max_ptes_none > > > > > > > > 0 -> 511 > > > > ... > > > > 10 -> 0 > > > > > > > > Right? > > > > > > Just so we are on the same page, this is what I had: > > > > > > 0 -> ~100% used (~0% none) > > > > > > So "0" -> 0 pte_none or 512 used. > > > > > > (note the used vs. none) > > > > OK right so we're talking about the same thing, I guess? > > > > I was confused partly becuase of the scale, becuase weren't people setting > > this parameter to low values in practice? > > > > And now we make it so we have equivalent of: > > > > 0 -> 0 > > 1 -> 256 > > 2 -> 384 > > Ah, there is the problem, that's not what I had in mind. > > 0 -> ~100% used (~0% none) > ... > 8 -> ~87,5% used (~12.5% none) > 9 -> ~75% used (~25% none) > 9 -> ~50% used (~50% none) > 10 -> ~0% used (~100% none) > > Hopefully I didn't mess it up again. I think this kind of table is fine for initial implementation of the knob, but we don't want to document it to userspace like this. I think we want to be strategically ambiguous on what the knob does exactly, so kernel could evolve the meaning of the knob over time. We don't want to repeat the problem we have with max_ptes_none which too prescriptive and got additional meaning with introduction of shrinker. As kernel evolves, we want ability to adjust the meaning and keep the knob useful. -- Kiryl Shutsemau / Kirill A. Shutemov
On Mon, Sep 15, 2025 at 01:01:26PM +0100, Kiryl Shutsemau wrote: > On Mon, Sep 15, 2025 at 01:45:39PM +0200, David Hildenbrand wrote: > > On 15.09.25 13:35, Lorenzo Stoakes wrote: > > > On Mon, Sep 15, 2025 at 01:29:22PM +0200, David Hildenbrand wrote: > > > > On 15.09.25 13:23, Lorenzo Stoakes wrote: > > > > > On Mon, Sep 15, 2025 at 01:14:32PM +0200, David Hildenbrand wrote: > > > > > > On 15.09.25 13:02, Lorenzo Stoakes wrote: > > > > > > > On Mon, Sep 15, 2025 at 12:52:03PM +0200, David Hildenbrand wrote: > > > > > > > > On 15.09.25 12:43, Lorenzo Stoakes wrote: > > > > > > > > > On Mon, Sep 15, 2025 at 12:22:07PM +0200, David Hildenbrand wrote: > > > > > > > > > > > > > > > > > > > > 0 -> ~100% used (~0% none) > > > > > > > > > > 1 -> ~50% used (~50% none) > > > > > > > > > > 2 -> ~25% used (~75% none) > > > > > > > > > > 3 -> ~12.5% used (~87.5% none) > > > > > > > > > > 4 -> ~11.25% used (~88,75% none) > > > > > > > > > > ... > > > > > > > > > > 10 -> ~0% used (~100% none) > > > > > > > > > > > > > > > > > > Oh and shouldn't this be inverted? > > > > > > > > > > > > > > > > > > 0 eagerness = we eat up all none PTE entries? Isn't that pretty eager? :P > > > > > > > > > 10 eagerness = we aren't eager to eat up none PTE entries at all? > > > > > > > > > > > > > > > > > > Or am I being dumb here? > > > > > > > > > > > > > > > > Good question. > > > > > > > > > > > > > > > > For swappiness it's: 0 -> no swap (conservative) > > > > > > > > > > > > > > > > So intuitively I assumed: 0 -> no pte_none (conservative) > > > > > > > > > > > > > > > > You're the native speaker, so you tell me :) > > > > > > > > > > > > > > To me this is about 'eagerness to consume empty PTE entries' so 10 is more > > > > > > > eager, 0 is not eager at all, i.e. inversion of what you suggest :) > > > > > > > > > > > > Just so we are on the same page: it is about "eagerness to collapse", right? > > > > > > > > > > > > Wouldn't a 0 mean "I am not eager, I will not waste any memory, I am very > > > > > > careful and bail out on any pte_none" vs. 10 meaning "I am very eager, I > > > > > > will collapse no matter what I find in the page table, waste as much memory > > > > > > as I want"? > > > > > > > > > > Yeah, this is my understanding of your scale, or is my understanding also > > > > > inverted? :) > > > > > > > > > > Right now it's: > > > > > > > > > > eagerness max_ptes_none > > > > > > > > > > 0 -> 511 > > > > > ... > > > > > 10 -> 0 > > > > > > > > > > Right? > > > > > > > > Just so we are on the same page, this is what I had: > > > > > > > > 0 -> ~100% used (~0% none) > > > > > > > > So "0" -> 0 pte_none or 512 used. > > > > > > > > (note the used vs. none) > > > > > > OK right so we're talking about the same thing, I guess? > > > > > > I was confused partly becuase of the scale, becuase weren't people setting > > > this parameter to low values in practice? > > > > > > And now we make it so we have equivalent of: > > > > > > 0 -> 0 > > > 1 -> 256 > > > 2 -> 384 > > > > Ah, there is the problem, that's not what I had in mind. > > > > 0 -> ~100% used (~0% none) > > ... > > 8 -> ~87,5% used (~12.5% none) > > 9 -> ~75% used (~25% none) > > 9 -> ~50% used (~50% none) > > 10 -> ~0% used (~100% none) > > > > Hopefully I didn't mess it up again. > > I think this kind of table is fine for initial implementation of the > knob, but we don't want to document it to userspace like this. > I think we want to be strategically ambiguous on what the knob does > exactly, so kernel could evolve the meaning of the knob over time. > > We don't want to repeat the problem we have with max_ptes_none which too > prescriptive and got additional meaning with introduction of shrinker. > > As kernel evolves, we want ability to adjust the meaning and keep the > knob useful. I mean, having said this exact thing several times in the thread obviously I agree... FWIW... To repeat, I think it should be an abstraction that we entirely control and whose meaning we can vary over time.
On Mon, Sep 15, 2025 at 12:22:07PM +0200, David Hildenbrand wrote: > On 15.09.25 11:22, Kiryl Shutsemau wrote: > > On Fri, Sep 12, 2025 at 05:31:51PM -0600, Nico Pache wrote: > > > On Fri, Sep 12, 2025 at 6:25 AM David Hildenbrand <david@redhat.com> wrote: > > > > > > > > On 12.09.25 14:19, Kiryl Shutsemau wrote: > > > > > On Thu, Sep 11, 2025 at 09:27:55PM -0600, Nico Pache wrote: > > > > > > The following series provides khugepaged with the capability to collapse > > > > > > anonymous memory regions to mTHPs. > > > > > > > > > > > > To achieve this we generalize the khugepaged functions to no longer depend > > > > > > on PMD_ORDER. Then during the PMD scan, we use a bitmap to track individual > > > > > > pages that are occupied (!none/zero). After the PMD scan is done, we do > > > > > > binary recursion on the bitmap to find the optimal mTHP sizes for the PMD > > > > > > range. The restriction on max_ptes_none is removed during the scan, to make > > > > > > sure we account for the whole PMD range. When no mTHP size is enabled, the > > > > > > legacy behavior of khugepaged is maintained. max_ptes_none will be scaled > > > > > > by the attempted collapse order to determine how full a mTHP must be to be > > > > > > eligible for the collapse to occur. If a mTHP collapse is attempted, but > > > > > > contains swapped out, or shared pages, we don't perform the collapse. It is > > > > > > now also possible to collapse to mTHPs without requiring the PMD THP size > > > > > > to be enabled. > > > > > > > > > > > > When enabling (m)THP sizes, if max_ptes_none >= HPAGE_PMD_NR/2 (255 on > > > > > > 4K page size), it will be automatically capped to HPAGE_PMD_NR/2 - 1 for > > > > > > mTHP collapses to prevent collapse "creep" behavior. This prevents > > > > > > constantly promoting mTHPs to the next available size, which would occur > > > > > > because a collapse introduces more non-zero pages that would satisfy the > > > > > > promotion condition on subsequent scans. > > > > > > > > > > Hm. Maybe instead of capping at HPAGE_PMD_NR/2 - 1 we can count > > > > > all-zeros 4k as none_or_zero? It mirrors the logic of shrinker. > > > > > > > > > > > > > I am all for not adding any more ugliness on top of all the ugliness we > > > > added in the past. > > > > > > > > I will soon propose deprecating that parameter in favor of something > > > > that makes a bit more sense. > > > > > > > > In essence, we'll likely have an "eagerness" parameter that ranges from > > > > 0 to 10. 10 is essentially "always collapse" and 0 "never collapse if > > > > not all is populated". > > > Hi David, > > > > > > Do you have any reason for 0-10, I'm guessing these will map to > > > different max_ptes_none values. > > > I suggest 0-5, mapping to 0,32,64,128,255,511 > > > > That's too x86-64 specific. > > > > And the whole idea is not to map to directly, but give kernel wiggle > > room to play. > > Initially we will start out simple and map it directly. But yeah, the idea > is to give us some more room later. I think it's less 'wiggle room' and more us being able to _abstract_ what this measurement means while reserving the right to adjust this. But maybe we are saying the same thing in different ways. > > I had something logarithmic in mind which would roughly be (ignoring the the > weird -1 for simplicity and expressing it as "used" instead of none-or-zero) > > 0 -> ~100% used (~0% none) So equivalent to 511 today? > 1 -> ~50% used (~50% none) > 2 -> ~25% used (~75% none) > 3 -> ~12.5% used (~87.5% none) > 4 -> ~11.25% used (~88,75% none) > ... > 10 -> ~0% used (~100% none) So equivalent to 0 today? And with a logarithmic weighting towards values closer to "0% used"? This seems sensible given the only reports we've had of non-0/511 uses here are in that range... But ofc this interpretation should be something we determine + treated as an implementation detail that we can modify later. > > Mapping that to actual THP sizes (#pages in a thp) on an arch will be easy. And at different mTHP levels too right? > > -- > Cheers > > David / dhildenb > Cheers, Lorenzo
On Mon, Sep 15, 2025 at 11:35:53AM +0100, Lorenzo Stoakes wrote: > On Mon, Sep 15, 2025 at 12:22:07PM +0200, David Hildenbrand wrote: > > Initially we will start out simple and map it directly. But yeah, the idea > > is to give us some more room later. > > I think it's less 'wiggle room' and more us being able to _abstract_ what this > measurement means while reserving the right to adjust this. > > But maybe we are saying the same thing in different ways. > > > > > I had something logarithmic in mind which would roughly be (ignoring the the > > weird -1 for simplicity and expressing it as "used" instead of none-or-zero) > > > > 0 -> ~100% used (~0% none) > > So equivalent to 511 today? > > > 1 -> ~50% used (~50% none) > > 2 -> ~25% used (~75% none) > > 3 -> ~12.5% used (~87.5% none) > > 4 -> ~11.25% used (~88,75% none) > > ... > > 10 -> ~0% used (~100% none) > > So equivalent to 0 today? > > And with a logarithmic weighting towards values closer to "0% used"? > > This seems sensible given the only reports we've had of non-0/511 uses here are > in that range... > > But ofc this interpretation should be something we determine + treated as an > implementation detail that we can modify later. > > > > > Mapping that to actual THP sizes (#pages in a thp) on an arch will be easy. > > And at different mTHP levels too right? > Another point here, since we have to keep: /sys/kernel/mm/transparent_hugepage/khugepaged/max_ptes_none Around, and users will try to set values there, presumably we will now add: /sys/kernel/mm/transparent_hugepage/khugepaged/eagerness How will we map <-> the two tunables?
>>> Mapping that to actual THP sizes (#pages in a thp) on an arch will be easy. >> >> And at different mTHP levels too right? >> > > Another point here, since we have to keep: > > /sys/kernel/mm/transparent_hugepage/khugepaged/max_ptes_none > > Around, and users will try to set values there, presumably we will now add: > > /sys/kernel/mm/transparent_hugepage/khugepaged/eagerness > > How will we map <-> the two tunables? Well, the easy case if someone updates eagerness, then we simply et it to whatever magic value we compute and document. The other direction is more problematic, likely we'll simply warn and do something reasonable (map it to whatever eagerness scale is closest or simply indicate it as "-1" -- user intervened or sth like that) -- Cheers David / dhildenb
On Mon, Sep 15, 2025 at 12:44:34PM +0200, David Hildenbrand wrote: > > > > Mapping that to actual THP sizes (#pages in a thp) on an arch will be easy. > > > > > > And at different mTHP levels too right? > > > > > > > Another point here, since we have to keep: > > > > /sys/kernel/mm/transparent_hugepage/khugepaged/max_ptes_none > > > > Around, and users will try to set values there, presumably we will now add: > > > > /sys/kernel/mm/transparent_hugepage/khugepaged/eagerness > > > > How will we map <-> the two tunables? > > Well, the easy case if someone updates eagerness, then we simply et it to > whatever magic value we compute and document. > > The other direction is more problematic, likely we'll simply warn and do > something reasonable (map it to whatever eagerness scale is closest or > simply indicate it as "-1" -- user intervened or sth like that) I don't love the idea of a -1 situation, as that's going to create some confusion. I'd really rather we just say out and out 'the kernel decides this based on eagerness'. So either warn or have some method to reverse-engineer what the closest value might be. Or perhaps just accept 0/511 there and map to eagerness min/max + if non-0/511 warn? > > -- > Cheers > > David / dhildenb > Cheers, Lorenzo
On 15.09.25 12:48, Lorenzo Stoakes wrote: > On Mon, Sep 15, 2025 at 12:44:34PM +0200, David Hildenbrand wrote: >>>>> Mapping that to actual THP sizes (#pages in a thp) on an arch will be easy. >>>> >>>> And at different mTHP levels too right? >>>> >>> >>> Another point here, since we have to keep: >>> >>> /sys/kernel/mm/transparent_hugepage/khugepaged/max_ptes_none >>> >>> Around, and users will try to set values there, presumably we will now add: >>> >>> /sys/kernel/mm/transparent_hugepage/khugepaged/eagerness >>> >>> How will we map <-> the two tunables? >> >> Well, the easy case if someone updates eagerness, then we simply et it to >> whatever magic value we compute and document. >> >> The other direction is more problematic, likely we'll simply warn and do >> something reasonable (map it to whatever eagerness scale is closest or >> simply indicate it as "-1" -- user intervened or sth like that) > > I don't love the idea of a -1 situation, as that's going to create some > confusion. swapiness also has a "max" parameter, so we could just say "override" /" disabled" / whatever? -- Cheers David / dhildenb
On Mon, Sep 15, 2025 at 12:52:53PM +0200, David Hildenbrand wrote: > On 15.09.25 12:48, Lorenzo Stoakes wrote: > > On Mon, Sep 15, 2025 at 12:44:34PM +0200, David Hildenbrand wrote: > > > > > > Mapping that to actual THP sizes (#pages in a thp) on an arch will be easy. > > > > > > > > > > And at different mTHP levels too right? > > > > > > > > > > > > > Another point here, since we have to keep: > > > > > > > > /sys/kernel/mm/transparent_hugepage/khugepaged/max_ptes_none > > > > > > > > Around, and users will try to set values there, presumably we will now add: > > > > > > > > /sys/kernel/mm/transparent_hugepage/khugepaged/eagerness > > > > > > > > How will we map <-> the two tunables? > > > > > > Well, the easy case if someone updates eagerness, then we simply et it to > > > whatever magic value we compute and document. > > > > > > The other direction is more problematic, likely we'll simply warn and do > > > something reasonable (map it to whatever eagerness scale is closest or > > > simply indicate it as "-1" -- user intervened or sth like that) > > > > I don't love the idea of a -1 situation, as that's going to create some > > confusion. > > swapiness also has a "max" parameter, so we could just say "override" /" > disabled" / whatever? I don't love the user being able to override this though, let's just nuke their ability to set this pleeeease. Because if they can override it, then we have to do some deeply nasty scaling for mTHP again. Would really prefer us to only accept 0/511 + warn on anything else. We could put the warning in a cycle before we land the change also + just take Nico's current version for now. So that way people are aware it's coming... (Could also put in docs ofc) > > -- > Cheers > > David / dhildenb > Cheers, Lorenzo
On 15.09.25 12:59, Lorenzo Stoakes wrote: > On Mon, Sep 15, 2025 at 12:52:53PM +0200, David Hildenbrand wrote: >> On 15.09.25 12:48, Lorenzo Stoakes wrote: >>> On Mon, Sep 15, 2025 at 12:44:34PM +0200, David Hildenbrand wrote: >>>>>>> Mapping that to actual THP sizes (#pages in a thp) on an arch will be easy. >>>>>> >>>>>> And at different mTHP levels too right? >>>>>> >>>>> >>>>> Another point here, since we have to keep: >>>>> >>>>> /sys/kernel/mm/transparent_hugepage/khugepaged/max_ptes_none >>>>> >>>>> Around, and users will try to set values there, presumably we will now add: >>>>> >>>>> /sys/kernel/mm/transparent_hugepage/khugepaged/eagerness >>>>> >>>>> How will we map <-> the two tunables? >>>> >>>> Well, the easy case if someone updates eagerness, then we simply et it to >>>> whatever magic value we compute and document. >>>> >>>> The other direction is more problematic, likely we'll simply warn and do >>>> something reasonable (map it to whatever eagerness scale is closest or >>>> simply indicate it as "-1" -- user intervened or sth like that) >>> >>> I don't love the idea of a -1 situation, as that's going to create some >>> confusion. >> >> swapiness also has a "max" parameter, so we could just say "override" /" >> disabled" / whatever? > > I don't love the user being able to override this though, let's just nuke their > ability to set this pleeeease. > > Because if they can override it, then we have to do some deeply nasty scaling > for mTHP again. There are ways to have it working internally, just using a different "scale" instead of the 100 -> 50 -> 25 etc. I am afraid we cannot change the parameter to ignore other values because of the interaction with the shrinker that easily .... we might be able to detracted at wait a bunch of kernel releases probably. -- Cheers David / dhildenb
On 15/09/2025 12:10, David Hildenbrand wrote: > On 15.09.25 12:59, Lorenzo Stoakes wrote: >> On Mon, Sep 15, 2025 at 12:52:53PM +0200, David Hildenbrand wrote: >>> On 15.09.25 12:48, Lorenzo Stoakes wrote: >>>> On Mon, Sep 15, 2025 at 12:44:34PM +0200, David Hildenbrand wrote: >>>>>>>> Mapping that to actual THP sizes (#pages in a thp) on an arch will be easy. >>>>>>> >>>>>>> And at different mTHP levels too right? >>>>>>> >>>>>> >>>>>> Another point here, since we have to keep: >>>>>> >>>>>> /sys/kernel/mm/transparent_hugepage/khugepaged/max_ptes_none >>>>>> >>>>>> Around, and users will try to set values there, presumably we will now add: >>>>>> >>>>>> /sys/kernel/mm/transparent_hugepage/khugepaged/eagerness >>>>>> >>>>>> How will we map <-> the two tunables? >>>>> >>>>> Well, the easy case if someone updates eagerness, then we simply et it to >>>>> whatever magic value we compute and document. >>>>> >>>>> The other direction is more problematic, likely we'll simply warn and do >>>>> something reasonable (map it to whatever eagerness scale is closest or >>>>> simply indicate it as "-1" -- user intervened or sth like that) >>>> >>>> I don't love the idea of a -1 situation, as that's going to create some >>>> confusion. >>> >>> swapiness also has a "max" parameter, so we could just say "override" /" >>> disabled" / whatever? >> >> I don't love the user being able to override this though, let's just nuke their >> ability to set this pleeeease. Do you mean stop people from changing max_ptes_none? I am not sure if thats a good idea, Its existed for a very long time and even a few release warnings might not be enough of a warning for sysadmins that might not have a kernel team to notice this. If the eagerness solution is just a logarithmic mapping of max_ptes_none at the start, I do think we need to keep max_ptes_none completely supported. As eagerness isnt really doing something new? Once eagnerness diverges from just setting max_ptes_none, only then we should start thinking about deprecating it? >> >> Because if they can override it, then we have to do some deeply nasty scaling >> for mTHP again. > > There are ways to have it working internally, just using a different "scale" instead of the 100 -> 50 -> 25 etc. > > I am afraid we cannot change the parameter to ignore other values because of the interaction with the shrinker that easily .... we might be able to detracted at wait a bunch of kernel releases probably. >
On Mon, Sep 15, 2025 at 01:10:22PM +0200, David Hildenbrand wrote: > On 15.09.25 12:59, Lorenzo Stoakes wrote: > > On Mon, Sep 15, 2025 at 12:52:53PM +0200, David Hildenbrand wrote: > > > On 15.09.25 12:48, Lorenzo Stoakes wrote: > > > > On Mon, Sep 15, 2025 at 12:44:34PM +0200, David Hildenbrand wrote: > > > > > > > > Mapping that to actual THP sizes (#pages in a thp) on an arch will be easy. > > > > > > > > > > > > > > And at different mTHP levels too right? > > > > > > > > > > > > > > > > > > > Another point here, since we have to keep: > > > > > > > > > > > > /sys/kernel/mm/transparent_hugepage/khugepaged/max_ptes_none > > > > > > > > > > > > Around, and users will try to set values there, presumably we will now add: > > > > > > > > > > > > /sys/kernel/mm/transparent_hugepage/khugepaged/eagerness > > > > > > > > > > > > How will we map <-> the two tunables? > > > > > > > > > > Well, the easy case if someone updates eagerness, then we simply et it to > > > > > whatever magic value we compute and document. > > > > > > > > > > The other direction is more problematic, likely we'll simply warn and do > > > > > something reasonable (map it to whatever eagerness scale is closest or > > > > > simply indicate it as "-1" -- user intervened or sth like that) > > > > > > > > I don't love the idea of a -1 situation, as that's going to create some > > > > confusion. > > > > > > swapiness also has a "max" parameter, so we could just say "override" /" > > > disabled" / whatever? > > > > I don't love the user being able to override this though, let's just nuke their > > ability to set this pleeeease. > > > > Because if they can override it, then we have to do some deeply nasty scaling > > for mTHP again. > > There are ways to have it working internally, just using a different "scale" > instead of the 100 -> 50 -> 25 etc. Right. I mean with the exponential scale we could just algorithimically figure out what the eagerness should be. > > I am afraid we cannot change the parameter to ignore other values because of > the interaction with the shrinker that easily .... we might be able to > detracted at wait a bunch of kernel releases probably. :( BTW 'Detracted at wait'? :P You mean we might be able to remove after a few releases? > > -- > Cheers > > David / dhildenb > Cheers, Lorenzo
>> I am afraid we cannot change the parameter to ignore other values because of >> the interaction with the shrinker that easily .... we might be able to >> detracted at wait a bunch of kernel releases probably. > > :( > > BTW 'Detracted at wait'? :P You mean we might be able to remove after a few > releases? "deprecate and wait" :) -- Cheers David / dhildenb
On 15.09.25 12:35, Lorenzo Stoakes wrote: > On Mon, Sep 15, 2025 at 12:22:07PM +0200, David Hildenbrand wrote: >> On 15.09.25 11:22, Kiryl Shutsemau wrote: >>> On Fri, Sep 12, 2025 at 05:31:51PM -0600, Nico Pache wrote: >>>> On Fri, Sep 12, 2025 at 6:25 AM David Hildenbrand <david@redhat.com> wrote: >>>>> >>>>> On 12.09.25 14:19, Kiryl Shutsemau wrote: >>>>>> On Thu, Sep 11, 2025 at 09:27:55PM -0600, Nico Pache wrote: >>>>>>> The following series provides khugepaged with the capability to collapse >>>>>>> anonymous memory regions to mTHPs. >>>>>>> >>>>>>> To achieve this we generalize the khugepaged functions to no longer depend >>>>>>> on PMD_ORDER. Then during the PMD scan, we use a bitmap to track individual >>>>>>> pages that are occupied (!none/zero). After the PMD scan is done, we do >>>>>>> binary recursion on the bitmap to find the optimal mTHP sizes for the PMD >>>>>>> range. The restriction on max_ptes_none is removed during the scan, to make >>>>>>> sure we account for the whole PMD range. When no mTHP size is enabled, the >>>>>>> legacy behavior of khugepaged is maintained. max_ptes_none will be scaled >>>>>>> by the attempted collapse order to determine how full a mTHP must be to be >>>>>>> eligible for the collapse to occur. If a mTHP collapse is attempted, but >>>>>>> contains swapped out, or shared pages, we don't perform the collapse. It is >>>>>>> now also possible to collapse to mTHPs without requiring the PMD THP size >>>>>>> to be enabled. >>>>>>> >>>>>>> When enabling (m)THP sizes, if max_ptes_none >= HPAGE_PMD_NR/2 (255 on >>>>>>> 4K page size), it will be automatically capped to HPAGE_PMD_NR/2 - 1 for >>>>>>> mTHP collapses to prevent collapse "creep" behavior. This prevents >>>>>>> constantly promoting mTHPs to the next available size, which would occur >>>>>>> because a collapse introduces more non-zero pages that would satisfy the >>>>>>> promotion condition on subsequent scans. >>>>>> >>>>>> Hm. Maybe instead of capping at HPAGE_PMD_NR/2 - 1 we can count >>>>>> all-zeros 4k as none_or_zero? It mirrors the logic of shrinker. >>>>>> >>>>> >>>>> I am all for not adding any more ugliness on top of all the ugliness we >>>>> added in the past. >>>>> >>>>> I will soon propose deprecating that parameter in favor of something >>>>> that makes a bit more sense. >>>>> >>>>> In essence, we'll likely have an "eagerness" parameter that ranges from >>>>> 0 to 10. 10 is essentially "always collapse" and 0 "never collapse if >>>>> not all is populated". >>>> Hi David, >>>> >>>> Do you have any reason for 0-10, I'm guessing these will map to >>>> different max_ptes_none values. >>>> I suggest 0-5, mapping to 0,32,64,128,255,511 >>> >>> That's too x86-64 specific. >>> >>> And the whole idea is not to map to directly, but give kernel wiggle >>> room to play. >> >> Initially we will start out simple and map it directly. But yeah, the idea >> is to give us some more room later. > > I think it's less 'wiggle room' and more us being able to _abstract_ what this > measurement means while reserving the right to adjust this. > > But maybe we are saying the same thing in different ways. > >> >> I had something logarithmic in mind which would roughly be (ignoring the the >> weird -1 for simplicity and expressing it as "used" instead of none-or-zero) >> >> 0 -> ~100% used (~0% none) > > So equivalent to 511 today? > >> 1 -> ~50% used (~50% none) >> 2 -> ~25% used (~75% none) >> 3 -> ~12.5% used (~87.5% none) >> 4 -> ~11.25% used (~88,75% none) >> ... >> 10 -> ~0% used (~100% none) > > So equivalent to 0 today? Yes. > > And with a logarithmic weighting towards values closer to "0% used"? > > This seems sensible given the only reports we've had of non-0/511 uses here are > in that range... > > But ofc this interpretation should be something we determine + treated as an > implementation detail that we can modify later. > >> >> Mapping that to actual THP sizes (#pages in a thp) on an arch will be easy. > > And at different mTHP levels too right? Yes exactly. -- Cheers David / dhildenb
On Fri, Sep 12, 2025 at 02:25:31PM +0200, David Hildenbrand wrote: > On 12.09.25 14:19, Kiryl Shutsemau wrote: > > On Thu, Sep 11, 2025 at 09:27:55PM -0600, Nico Pache wrote: > >> The following series provides khugepaged with the capability to collapse > >> anonymous memory regions to mTHPs. > >> > >> To achieve this we generalize the khugepaged functions to no longer depend > >> on PMD_ORDER. Then during the PMD scan, we use a bitmap to track individual > >> pages that are occupied (!none/zero). After the PMD scan is done, we do > >> binary recursion on the bitmap to find the optimal mTHP sizes for the PMD > >> range. The restriction on max_ptes_none is removed during the scan, to make > >> sure we account for the whole PMD range. When no mTHP size is enabled, the > >> legacy behavior of khugepaged is maintained. max_ptes_none will be scaled > >> by the attempted collapse order to determine how full a mTHP must be to be > >> eligible for the collapse to occur. If a mTHP collapse is attempted, but > >> contains swapped out, or shared pages, we don't perform the collapse. It is > >> now also possible to collapse to mTHPs without requiring the PMD THP size > >> to be enabled. > >> > >> When enabling (m)THP sizes, if max_ptes_none >= HPAGE_PMD_NR/2 (255 on > >> 4K page size), it will be automatically capped to HPAGE_PMD_NR/2 - 1 for > >> mTHP collapses to prevent collapse "creep" behavior. This prevents > >> constantly promoting mTHPs to the next available size, which would occur > >> because a collapse introduces more non-zero pages that would satisfy the > >> promotion condition on subsequent scans. > > > > Hm. Maybe instead of capping at HPAGE_PMD_NR/2 - 1 we can count > > all-zeros 4k as none_or_zero? It mirrors the logic of shrinker. > > > > I am all for not adding any more ugliness on top of all the ugliness we > added in the past. > > I will soon propose deprecating that parameter in favor of something > that makes a bit more sense. > > In essence, we'll likely have an "eagerness" parameter that ranges from > 0 to 10. 10 is essentially "always collapse" and 0 "never collapse if > not all is populated". > > In between we will have more flexibility on how to set these values. > > Likely 9 will be around 50% to not even motivate the user to set > something that does not make sense (creep). One observation we've had from production experiments is that the optimal number here isn't static. If you have plenty of memory, then even very sparse THPs are beneficial. An extreme example: if all your THPs have 2/512 pages populated, that's still cutting TLB pressure in half! So in the absence of memory pressure, allocating and collapsing should optimally be aggressive even on very sparse regions. On the flipside, if there is memory pressure, TLB benefits are very quickly drowned out by faults and paging events. And I mean real memory pressure. If all that's happening is that somebody is streaming through filesystem data, the optimal behavior is still to be greedy. Another consideration is that if we need to break large folios, we should start with colder ones that provide less benefit, and defer the splitting of hotter ones as long as possible. Maybe a good direction would be to move splitting out of the shrinker and tie it to the (refault-aware) anon reclaim. And then instead of a fixed population threshold, collapse on a pressure gradient that starts with "no pressure/thrashing and at least two base pages in THP a region" and ends with "reclaim is splitting everything, back off".
On 12.09.25 15:37, Johannes Weiner wrote: > On Fri, Sep 12, 2025 at 02:25:31PM +0200, David Hildenbrand wrote: >> On 12.09.25 14:19, Kiryl Shutsemau wrote: >>> On Thu, Sep 11, 2025 at 09:27:55PM -0600, Nico Pache wrote: >>>> The following series provides khugepaged with the capability to collapse >>>> anonymous memory regions to mTHPs. >>>> >>>> To achieve this we generalize the khugepaged functions to no longer depend >>>> on PMD_ORDER. Then during the PMD scan, we use a bitmap to track individual >>>> pages that are occupied (!none/zero). After the PMD scan is done, we do >>>> binary recursion on the bitmap to find the optimal mTHP sizes for the PMD >>>> range. The restriction on max_ptes_none is removed during the scan, to make >>>> sure we account for the whole PMD range. When no mTHP size is enabled, the >>>> legacy behavior of khugepaged is maintained. max_ptes_none will be scaled >>>> by the attempted collapse order to determine how full a mTHP must be to be >>>> eligible for the collapse to occur. If a mTHP collapse is attempted, but >>>> contains swapped out, or shared pages, we don't perform the collapse. It is >>>> now also possible to collapse to mTHPs without requiring the PMD THP size >>>> to be enabled. >>>> >>>> When enabling (m)THP sizes, if max_ptes_none >= HPAGE_PMD_NR/2 (255 on >>>> 4K page size), it will be automatically capped to HPAGE_PMD_NR/2 - 1 for >>>> mTHP collapses to prevent collapse "creep" behavior. This prevents >>>> constantly promoting mTHPs to the next available size, which would occur >>>> because a collapse introduces more non-zero pages that would satisfy the >>>> promotion condition on subsequent scans. >>> >>> Hm. Maybe instead of capping at HPAGE_PMD_NR/2 - 1 we can count >>> all-zeros 4k as none_or_zero? It mirrors the logic of shrinker. >>> >> >> I am all for not adding any more ugliness on top of all the ugliness we >> added in the past. >> >> I will soon propose deprecating that parameter in favor of something >> that makes a bit more sense. >> >> In essence, we'll likely have an "eagerness" parameter that ranges from >> 0 to 10. 10 is essentially "always collapse" and 0 "never collapse if >> not all is populated". >> >> In between we will have more flexibility on how to set these values. >> >> Likely 9 will be around 50% to not even motivate the user to set >> something that does not make sense (creep). > > One observation we've had from production experiments is that the > optimal number here isn't static. If you have plenty of memory, then > even very sparse THPs are beneficial. Exactly. And willy suggested something like "eagerness" similar to "swapinness" that gives us more flexibility when implementing it, including dynamically adjusting the values in the future. > > An extreme example: if all your THPs have 2/512 pages populated, > that's still cutting TLB pressure in half! IIRC, you create more pressure on the huge entries, where you might have less TLB entries :) But yes, there can be cases where it is beneficial, if there is absolutely no memory pressure. > > So in the absence of memory pressure, allocating and collapsing should > optimally be aggressive even on very sparse regions. Yes, we discussed that as well in the THP cabal. It's very similar to the max_ptes_swapped: that parameter should not exist. If there is no memory pressure we can just swap it in. If there is memory pressure we probably would not want to swap in much. > > On the flipside, if there is memory pressure, TLB benefits are very > quickly drowned out by faults and paging events. And I mean real > memory pressure. If all that's happening is that somebody is streaming > through filesystem data, the optimal behavior is still to be greedy. > > Another consideration is that if we need to break large folios, we > should start with colder ones that provide less benefit, and defer the > splitting of hotter ones as long as possible. Yes, we discussed that as well: there is no QoS right now, which is rather suboptimal. > > Maybe a good direction would be to move splitting out of the shrinker > and tie it to the (refault-aware) anon reclaim. And then instead of a > fixed population threshold, collapse on a pressure gradient that > starts with "no pressure/thrashing and at least two base pages in THP > a region" and ends with "reclaim is splitting everything, back off". I agree, but have to think further about how that could work in practice. -- Cheers David / dhildenb
On Fri, Sep 12, 2025 at 03:46:36PM +0200, David Hildenbrand wrote: > On 12.09.25 15:37, Johannes Weiner wrote: > > On Fri, Sep 12, 2025 at 02:25:31PM +0200, David Hildenbrand wrote: > >> On 12.09.25 14:19, Kiryl Shutsemau wrote: > >>> On Thu, Sep 11, 2025 at 09:27:55PM -0600, Nico Pache wrote: > >>>> The following series provides khugepaged with the capability to collapse > >>>> anonymous memory regions to mTHPs. > >>>> > >>>> To achieve this we generalize the khugepaged functions to no longer depend > >>>> on PMD_ORDER. Then during the PMD scan, we use a bitmap to track individual > >>>> pages that are occupied (!none/zero). After the PMD scan is done, we do > >>>> binary recursion on the bitmap to find the optimal mTHP sizes for the PMD > >>>> range. The restriction on max_ptes_none is removed during the scan, to make > >>>> sure we account for the whole PMD range. When no mTHP size is enabled, the > >>>> legacy behavior of khugepaged is maintained. max_ptes_none will be scaled > >>>> by the attempted collapse order to determine how full a mTHP must be to be > >>>> eligible for the collapse to occur. If a mTHP collapse is attempted, but > >>>> contains swapped out, or shared pages, we don't perform the collapse. It is > >>>> now also possible to collapse to mTHPs without requiring the PMD THP size > >>>> to be enabled. > >>>> > >>>> When enabling (m)THP sizes, if max_ptes_none >= HPAGE_PMD_NR/2 (255 on > >>>> 4K page size), it will be automatically capped to HPAGE_PMD_NR/2 - 1 for > >>>> mTHP collapses to prevent collapse "creep" behavior. This prevents > >>>> constantly promoting mTHPs to the next available size, which would occur > >>>> because a collapse introduces more non-zero pages that would satisfy the > >>>> promotion condition on subsequent scans. > >>> > >>> Hm. Maybe instead of capping at HPAGE_PMD_NR/2 - 1 we can count > >>> all-zeros 4k as none_or_zero? It mirrors the logic of shrinker. > >>> > >> > >> I am all for not adding any more ugliness on top of all the ugliness we > >> added in the past. > >> > >> I will soon propose deprecating that parameter in favor of something > >> that makes a bit more sense. > >> > >> In essence, we'll likely have an "eagerness" parameter that ranges from > >> 0 to 10. 10 is essentially "always collapse" and 0 "never collapse if > >> not all is populated". > >> > >> In between we will have more flexibility on how to set these values. > >> > >> Likely 9 will be around 50% to not even motivate the user to set > >> something that does not make sense (creep). > > > > One observation we've had from production experiments is that the > > optimal number here isn't static. If you have plenty of memory, then > > even very sparse THPs are beneficial. > > Exactly. > > And willy suggested something like "eagerness" similar to "swapinness" > that gives us more flexibility when implementing it, including > dynamically adjusting the values in the future. I think we talked past each other a bit here. The point I was trying to make is that the optimal behavior depends on the pressure situation inside the kernel; it's fundamentally not something userspace can make informed choices about. So for max_ptes_none, the approach is basically: try a few settings and see which one performs best. Okay, not great. But wouldn't that be the same for an eagerness setting? What would be the mental model for the user when configuring this? If it's the same empirical approach, then the new knob would seem like a lateral move. It would also be difficult to change the implementation without risking regressions once production systems are tuned to the old behavior. > > An extreme example: if all your THPs have 2/512 pages populated, > > that's still cutting TLB pressure in half! > > IIRC, you create more pressure on the huge entries, where you might have > less TLB entries :) But yes, there can be cases where it is beneficial, > if there is absolutely no memory pressure. Ha, the TLB topology is a whole other can of worms. We've tried deploying THP on older systems with separate TLB entries for different page sizes and gave up. It's a nightmare to configure and very easy to do worse than base pages. The kernel itself is using a mix of page sizes for the identity mapping. You basically have to complement the userspace page size distribution in such a way that you don't compete over the wrong entries at runtime. It's just stupid. I'm honestly not sure this is realistically solvable. So we're deploying THP only on newer AMD machines where TLB entries are shared. For split TLBs, we're sticking with hugetlb and trial-and-error. Please don't build CPUs this way.
On 15.09.25 15:43, Johannes Weiner wrote: > On Fri, Sep 12, 2025 at 03:46:36PM +0200, David Hildenbrand wrote: >> On 12.09.25 15:37, Johannes Weiner wrote: >>> On Fri, Sep 12, 2025 at 02:25:31PM +0200, David Hildenbrand wrote: >>>> On 12.09.25 14:19, Kiryl Shutsemau wrote: >>>>> On Thu, Sep 11, 2025 at 09:27:55PM -0600, Nico Pache wrote: >>>>>> The following series provides khugepaged with the capability to collapse >>>>>> anonymous memory regions to mTHPs. >>>>>> >>>>>> To achieve this we generalize the khugepaged functions to no longer depend >>>>>> on PMD_ORDER. Then during the PMD scan, we use a bitmap to track individual >>>>>> pages that are occupied (!none/zero). After the PMD scan is done, we do >>>>>> binary recursion on the bitmap to find the optimal mTHP sizes for the PMD >>>>>> range. The restriction on max_ptes_none is removed during the scan, to make >>>>>> sure we account for the whole PMD range. When no mTHP size is enabled, the >>>>>> legacy behavior of khugepaged is maintained. max_ptes_none will be scaled >>>>>> by the attempted collapse order to determine how full a mTHP must be to be >>>>>> eligible for the collapse to occur. If a mTHP collapse is attempted, but >>>>>> contains swapped out, or shared pages, we don't perform the collapse. It is >>>>>> now also possible to collapse to mTHPs without requiring the PMD THP size >>>>>> to be enabled. >>>>>> >>>>>> When enabling (m)THP sizes, if max_ptes_none >= HPAGE_PMD_NR/2 (255 on >>>>>> 4K page size), it will be automatically capped to HPAGE_PMD_NR/2 - 1 for >>>>>> mTHP collapses to prevent collapse "creep" behavior. This prevents >>>>>> constantly promoting mTHPs to the next available size, which would occur >>>>>> because a collapse introduces more non-zero pages that would satisfy the >>>>>> promotion condition on subsequent scans. >>>>> >>>>> Hm. Maybe instead of capping at HPAGE_PMD_NR/2 - 1 we can count >>>>> all-zeros 4k as none_or_zero? It mirrors the logic of shrinker. >>>>> >>>> >>>> I am all for not adding any more ugliness on top of all the ugliness we >>>> added in the past. >>>> >>>> I will soon propose deprecating that parameter in favor of something >>>> that makes a bit more sense. >>>> >>>> In essence, we'll likely have an "eagerness" parameter that ranges from >>>> 0 to 10. 10 is essentially "always collapse" and 0 "never collapse if >>>> not all is populated". >>>> >>>> In between we will have more flexibility on how to set these values. >>>> >>>> Likely 9 will be around 50% to not even motivate the user to set >>>> something that does not make sense (creep). >>> >>> One observation we've had from production experiments is that the >>> optimal number here isn't static. If you have plenty of memory, then >>> even very sparse THPs are beneficial. >> >> Exactly. >> >> And willy suggested something like "eagerness" similar to "swapinness" >> that gives us more flexibility when implementing it, including >> dynamically adjusting the values in the future. > > I think we talked past each other a bit here. The point I was trying > to make is that the optimal behavior depends on the pressure situation > inside the kernel; it's fundamentally not something userspace can make > informed choices about. I don't think the "no tunable at all" approach solely based on pressure will be workable in the foreseeable future. Collapsing 2 pages to 2 MiB THP all over the system just to split it immediately again is not something particularly helpful. So long term I assume the eagerness will work together with memory pressure and probably some other inputs. > > So for max_ptes_none, the approach is basically: try a few settings > and see which one performs best. Okay, not great. But wouldn't that be > the same for an eagerness setting? What would be the mental model for > the user when configuring this? If it's the same empirical approach, > then the new knob would seem like a lateral move. Consider it a replacement for something that is oddly PMD specific and requires you to punch in magical values (e.g., 511 on x86, 2047 on arm64 64k). Initially I thought about just using a percentage/scale of (m)THP but Willy argued that something more abstract gives us more wiggle room. Yes, for some workloads you will likely still have to fine tune parameters (honestly, I don't think many companies besides Meta are doing that), but the idea is to evolve it over time to something that is smarter than punching in magic values into an obscure interface. > > It would also be difficult to change the implementation without > risking regressions once production systems are tuned to the old > behavior. Companies like Meta that do such a level of fine-tuning probably use the old nasty interface because they know exactly what they are doing. That is a corner case, though. -- Cheers David / dhildenb
On Fri, Sep 12, 2025 at 03:46:36PM +0200, David Hildenbrand wrote: > On 12.09.25 15:37, Johannes Weiner wrote: > > On Fri, Sep 12, 2025 at 02:25:31PM +0200, David Hildenbrand wrote: > > > On 12.09.25 14:19, Kiryl Shutsemau wrote: > > > > On Thu, Sep 11, 2025 at 09:27:55PM -0600, Nico Pache wrote: > > > > > The following series provides khugepaged with the capability to collapse > > > > > anonymous memory regions to mTHPs. > > > > > > > > > > To achieve this we generalize the khugepaged functions to no longer depend > > > > > on PMD_ORDER. Then during the PMD scan, we use a bitmap to track individual > > > > > pages that are occupied (!none/zero). After the PMD scan is done, we do > > > > > binary recursion on the bitmap to find the optimal mTHP sizes for the PMD > > > > > range. The restriction on max_ptes_none is removed during the scan, to make > > > > > sure we account for the whole PMD range. When no mTHP size is enabled, the > > > > > legacy behavior of khugepaged is maintained. max_ptes_none will be scaled > > > > > by the attempted collapse order to determine how full a mTHP must be to be > > > > > eligible for the collapse to occur. If a mTHP collapse is attempted, but > > > > > contains swapped out, or shared pages, we don't perform the collapse. It is > > > > > now also possible to collapse to mTHPs without requiring the PMD THP size > > > > > to be enabled. > > > > > > > > > > When enabling (m)THP sizes, if max_ptes_none >= HPAGE_PMD_NR/2 (255 on > > > > > 4K page size), it will be automatically capped to HPAGE_PMD_NR/2 - 1 for > > > > > mTHP collapses to prevent collapse "creep" behavior. This prevents > > > > > constantly promoting mTHPs to the next available size, which would occur > > > > > because a collapse introduces more non-zero pages that would satisfy the > > > > > promotion condition on subsequent scans. > > > > > > > > Hm. Maybe instead of capping at HPAGE_PMD_NR/2 - 1 we can count > > > > all-zeros 4k as none_or_zero? It mirrors the logic of shrinker. > > > > > > > > > > I am all for not adding any more ugliness on top of all the ugliness we > > > added in the past. > > > > > > I will soon propose deprecating that parameter in favor of something > > > that makes a bit more sense. > > > > > > In essence, we'll likely have an "eagerness" parameter that ranges from > > > 0 to 10. 10 is essentially "always collapse" and 0 "never collapse if > > > not all is populated". > > > > > > In between we will have more flexibility on how to set these values. > > > > > > Likely 9 will be around 50% to not even motivate the user to set > > > something that does not make sense (creep). > > > > One observation we've had from production experiments is that the > > optimal number here isn't static. If you have plenty of memory, then > > even very sparse THPs are beneficial. > > Exactly. > > And willy suggested something like "eagerness" similar to "swapinness" that > gives us more flexibility when implementing it, including dynamically > adjusting the values in the future. > Ideally we would be able to also apply this to the page faulting paths. In many cases, there's no good reason to create a THP on the first fault... > > > > An extreme example: if all your THPs have 2/512 pages populated, > > that's still cutting TLB pressure in half! > > IIRC, you create more pressure on the huge entries, where you might have > less TLB entries :) But yes, there can be cases where it is beneficial, if > there is absolutely no memory pressure. > Correct, but it depends on the microarchitecture. For modern x86_64 AMD, it happens that the L1 TLB entries are shared between 4K/2M/1G. This was not (is not?) the case for Intel, where e.g back on kabylake, you had separate entries for 4K/2MB/1GB. Maybe in the Great Glorious Future (how many of those do we have?!) it would be a good idea to take this kinds of things into account. Just because we can map a THP, doesn't mean we should. Shower thought: it might be in these cases especially where the FreeBSD reservation system comes in handy - best effort allocating a THP, but not actually mapping it as such until you really _know_ it is hot - and until then, memory reclaim can just break your THP down if it really needs to. -- Pedro
On Fri, Sep 12, 2025 at 04:15:23PM +0100, Pedro Falcato wrote: > On Fri, Sep 12, 2025 at 03:46:36PM +0200, David Hildenbrand wrote: > > On 12.09.25 15:37, Johannes Weiner wrote: > > > On Fri, Sep 12, 2025 at 02:25:31PM +0200, David Hildenbrand wrote: > > > > On 12.09.25 14:19, Kiryl Shutsemau wrote: > > > > > On Thu, Sep 11, 2025 at 09:27:55PM -0600, Nico Pache wrote: > > > > > > The following series provides khugepaged with the capability to collapse > > > > > > anonymous memory regions to mTHPs. > > > > > > > > > > > > To achieve this we generalize the khugepaged functions to no longer depend > > > > > > on PMD_ORDER. Then during the PMD scan, we use a bitmap to track individual > > > > > > pages that are occupied (!none/zero). After the PMD scan is done, we do > > > > > > binary recursion on the bitmap to find the optimal mTHP sizes for the PMD > > > > > > range. The restriction on max_ptes_none is removed during the scan, to make > > > > > > sure we account for the whole PMD range. When no mTHP size is enabled, the > > > > > > legacy behavior of khugepaged is maintained. max_ptes_none will be scaled > > > > > > by the attempted collapse order to determine how full a mTHP must be to be > > > > > > eligible for the collapse to occur. If a mTHP collapse is attempted, but > > > > > > contains swapped out, or shared pages, we don't perform the collapse. It is > > > > > > now also possible to collapse to mTHPs without requiring the PMD THP size > > > > > > to be enabled. > > > > > > > > > > > > When enabling (m)THP sizes, if max_ptes_none >= HPAGE_PMD_NR/2 (255 on > > > > > > 4K page size), it will be automatically capped to HPAGE_PMD_NR/2 - 1 for > > > > > > mTHP collapses to prevent collapse "creep" behavior. This prevents > > > > > > constantly promoting mTHPs to the next available size, which would occur > > > > > > because a collapse introduces more non-zero pages that would satisfy the > > > > > > promotion condition on subsequent scans. > > > > > > > > > > Hm. Maybe instead of capping at HPAGE_PMD_NR/2 - 1 we can count > > > > > all-zeros 4k as none_or_zero? It mirrors the logic of shrinker. > > > > > > > > > > > > > I am all for not adding any more ugliness on top of all the ugliness we > > > > added in the past. > > > > > > > > I will soon propose deprecating that parameter in favor of something > > > > that makes a bit more sense. > > > > > > > > In essence, we'll likely have an "eagerness" parameter that ranges from > > > > 0 to 10. 10 is essentially "always collapse" and 0 "never collapse if > > > > not all is populated". > > > > > > > > In between we will have more flexibility on how to set these values. > > > > > > > > Likely 9 will be around 50% to not even motivate the user to set > > > > something that does not make sense (creep). > > > > > > One observation we've had from production experiments is that the > > > optimal number here isn't static. If you have plenty of memory, then > > > even very sparse THPs are beneficial. > > > > Exactly. > > > > And willy suggested something like "eagerness" similar to "swapinness" that > > gives us more flexibility when implementing it, including dynamically > > adjusting the values in the future. > > > > Ideally we would be able to also apply this to the page faulting paths. > In many cases, there's no good reason to create a THP on the first fault... > > > > > > > An extreme example: if all your THPs have 2/512 pages populated, > > > that's still cutting TLB pressure in half! > > > > IIRC, you create more pressure on the huge entries, where you might have > > less TLB entries :) But yes, there can be cases where it is beneficial, if > > there is absolutely no memory pressure. > > > > Correct, but it depends on the microarchitecture. For modern x86_64 AMD, it > happens that the L1 TLB entries are shared between 4K/2M/1G. This was not > (is not?) the case for Intel, where e.g back on kabylake, you had separate > entries for 4K/2MB/1GB. On Intel secondary TLB is shared between 4k and 2M. L2 TLB for 1G is separate. > Maybe in the Great Glorious Future (how many of those do we have?!) it would > be a good idea to take this kinds of things into account. Just because we can > map a THP, doesn't mean we should. > > Shower thought: it might be in these cases especially where the FreeBSD > reservation system comes in handy - best effort allocating a THP, but not > actually mapping it as such until you really _know_ it is hot - and until > then, memory reclaim can just break your THP down if it really needs to. This is just silly. All downsides without benefit until maybe later. And for short-lived processes the "later" never comes. -- Kiryl Shutsemau / Kirill A. Shutemov
On Fri, Sep 12, 2025 at 04:39:02PM +0100, Kiryl Shutsemau wrote: > > Shower thought: it might be in these cases especially where the FreeBSD > > reservation system comes in handy - best effort allocating a THP, but not > > actually mapping it as such until you really _know_ it is hot - and until > > then, memory reclaim can just break your THP down if it really needs to. > > This is just silly. All downsides without benefit until maybe later. And > for short-lived processes the "later" never comes. The right way out is to get better info on access pattern from hardware. For instance, if we move access bit out of page table entry and make it independent of the actually mapping size that would give us much better view on what actually is going on. -- Kiryl Shutsemau / Kirill A. Shutemov
On 12.09.25 17:44, Kiryl Shutsemau wrote: > On Fri, Sep 12, 2025 at 04:39:02PM +0100, Kiryl Shutsemau wrote: >>> Shower thought: it might be in these cases especially where the FreeBSD >>> reservation system comes in handy - best effort allocating a THP, but not >>> actually mapping it as such until you really _know_ it is hot - and until >>> then, memory reclaim can just break your THP down if it really needs to. >> >> This is just silly. All downsides without benefit until maybe later. And >> for short-lived processes the "later" never comes. > > The right way out is to get better info on access pattern from hardware. > For instance, if we move access bit out of page table entry and make it > independent of the actually mapping size that would give us much better > view on what actually is going on. We discussed this a couple of times in the past, the problem is that it does not help anybody really if all but a handful piece of hardware provides such a features. Long long long term I agree, short term we cannot really build core infrastructure around any of that. -- Cheers David / dhildenb
>> Maybe in the Great Glorious Future (how many of those do we have?!) it would >> be a good idea to take this kinds of things into account. Just because we can >> map a THP, doesn't mean we should. >> >> Shower thought: it might be in these cases especially where the FreeBSD >> reservation system comes in handy - best effort allocating a THP, but not >> actually mapping it as such until you really _know_ it is hot - and until >> then, memory reclaim can just break your THP down if it really needs to. > > This is just silly. All downsides without benefit until maybe later. And > for short-lived processes the "later" never comes. Right, that's why I've also been arguing against that (we discussed it recently in the THP cabal). -- Cheers David / dhildenb
On Fri, Sep 12, 2025 at 03:46:36PM +0200, David Hildenbrand wrote: > On 12.09.25 15:37, Johannes Weiner wrote: > > On Fri, Sep 12, 2025 at 02:25:31PM +0200, David Hildenbrand wrote: > > > On 12.09.25 14:19, Kiryl Shutsemau wrote: > > > > On Thu, Sep 11, 2025 at 09:27:55PM -0600, Nico Pache wrote: > > > > > The following series provides khugepaged with the capability to collapse > > > > > anonymous memory regions to mTHPs. > > > > > > > > > > To achieve this we generalize the khugepaged functions to no longer depend > > > > > on PMD_ORDER. Then during the PMD scan, we use a bitmap to track individual > > > > > pages that are occupied (!none/zero). After the PMD scan is done, we do > > > > > binary recursion on the bitmap to find the optimal mTHP sizes for the PMD > > > > > range. The restriction on max_ptes_none is removed during the scan, to make > > > > > sure we account for the whole PMD range. When no mTHP size is enabled, the > > > > > legacy behavior of khugepaged is maintained. max_ptes_none will be scaled > > > > > by the attempted collapse order to determine how full a mTHP must be to be > > > > > eligible for the collapse to occur. If a mTHP collapse is attempted, but > > > > > contains swapped out, or shared pages, we don't perform the collapse. It is > > > > > now also possible to collapse to mTHPs without requiring the PMD THP size > > > > > to be enabled. > > > > > > > > > > When enabling (m)THP sizes, if max_ptes_none >= HPAGE_PMD_NR/2 (255 on > > > > > 4K page size), it will be automatically capped to HPAGE_PMD_NR/2 - 1 for > > > > > mTHP collapses to prevent collapse "creep" behavior. This prevents > > > > > constantly promoting mTHPs to the next available size, which would occur > > > > > because a collapse introduces more non-zero pages that would satisfy the > > > > > promotion condition on subsequent scans. > > > > > > > > Hm. Maybe instead of capping at HPAGE_PMD_NR/2 - 1 we can count > > > > all-zeros 4k as none_or_zero? It mirrors the logic of shrinker. > > > > > > > > > > I am all for not adding any more ugliness on top of all the ugliness we > > > added in the past. > > > > > > I will soon propose deprecating that parameter in favor of something > > > that makes a bit more sense. > > > > > > In essence, we'll likely have an "eagerness" parameter that ranges from > > > 0 to 10. 10 is essentially "always collapse" and 0 "never collapse if > > > not all is populated". > > > > > > In between we will have more flexibility on how to set these values. > > > > > > Likely 9 will be around 50% to not even motivate the user to set > > > something that does not make sense (creep). > > > > One observation we've had from production experiments is that the > > optimal number here isn't static. If you have plenty of memory, then > > even very sparse THPs are beneficial. > > Exactly. > > And willy suggested something like "eagerness" similar to "swapinness" that > gives us more flexibility when implementing it, including dynamically > adjusting the values in the future. I like the idea of abstracting it like this, and - in a rare case of kernel developer agreement (esp. around naming :) - both Matthew, David and I rather loved referring to this as 'eagerness' here :) The great benefit in relation to dynamic state is that we can simply treat this as an _abstract_ thing. I.e. 'how eager are we to establish THPs, trading off against memory pressure and higher order folio resource consumption'. And then we can decide how precisely that is implemented in practice - and a sensible approach would indeed be to differentiate between scenarios where we might be more willing to chomp up memory vs. those we are not. This also aligns nicely with the 'grand glorious future' we all dream off (don't we??) in THP where things are automated as much as possible and the _kernel decides_ what's best as far as is possible. As with swappiness, it is essentially a 'hint' to us in abstract terms rather than simply exposing an internal kernel parameter. (Credit to Matthew for making this abstraction suggestion in the THP cabal meeting by the way!) > > > > > An extreme example: if all your THPs have 2/512 pages populated, > > that's still cutting TLB pressure in half! > > IIRC, you create more pressure on the huge entries, where you might have > less TLB entries :) But yes, there can be cases where it is beneficial, if > there is absolutely no memory pressure. > > > > > So in the absence of memory pressure, allocating and collapsing should > > optimally be aggressive even on very sparse regions. > > Yes, we discussed that as well in the THP cabal. > > It's very similar to the max_ptes_swapped: that parameter should not exist. > If there is no memory pressure we can just swap it in. If there is memory > pressure we probably would not want to swap in much. Yes, but at least an eagerness parameter gets us closer to this ideal. Of course, I agree that max_ptes_none should simply never have been exposed like this. It is emblematic of a 'just shove a parameter into a tunable/sysfs and let the user decide' approach you see in the kernel sometimes. This is problmeatic as users have no earthly idea how to set the parameter (most likely never touch it), and only start fiddling should issues arise and it looks like a viable solution of some kind. The problem is users usually lack a great deal of context the kernel has, and may make incorrect decisions that work in one situation but not another. TL;DR - this kind of interface is just lazy and we have to assess these kinds of tunables based on the actual RoI + understanding from the user's perspective. > > > > > On the flipside, if there is memory pressure, TLB benefits are very > > quickly drowned out by faults and paging events. And I mean real > > memory pressure. If all that's happening is that somebody is streaming > > through filesystem data, the optimal behavior is still to be greedy. > > > > Another consideration is that if we need to break large folios, we > > should start with colder ones that provide less benefit, and defer the > > splitting of hotter ones as long as possible. > > Yes, we discussed that as well: there is no QoS right now, which is rather > suboptimal. It's also kinda funny that the max_pte_none default is 511 right now so pretty damn eager. Which might be part of the reason people often observe THP chomping through resources... > > > > > Maybe a good direction would be to move splitting out of the shrinker > > and tie it to the (refault-aware) anon reclaim. And then instead of a > > fixed population threshold, collapse on a pressure gradient that > > starts with "no pressure/thrashing and at least two base pages in THP > > a region" and ends with "reclaim is splitting everything, back off". > > I agree, but have to think further about how that could work in practice. That'd be lovely actually! > > -- > Cheers > > David / dhildenb > Cheers, Lorenzo
On Fri, Sep 12, 2025 at 03:01:02PM +0100, Lorenzo Stoakes wrote: > On Fri, Sep 12, 2025 at 03:46:36PM +0200, David Hildenbrand wrote: > > <snip> > > Exactly. > > > > And willy suggested something like "eagerness" similar to "swapinness" that > > gives us more flexibility when implementing it, including dynamically > > adjusting the values in the future. > > I like the idea of abstracting it like this, and - in a rare case of kernel > developer agreement (esp. around naming :) - both Matthew, David and I rather > loved referring to this as 'eagerness' here :) > > The great benefit in relation to dynamic state is that we can simply treat this > as an _abstract_ thing. I.e. 'how eager are we to establish THPs, trading off > against memory pressure and higher order folio resource consumption'. > > And then we can decide how precisely that is implemented in practice - and a > sensible approach would indeed be to differentiate between scenarios where we > might be more willing to chomp up memory vs. those we are not. > > This also aligns nicely with the 'grand glorious future' we all dream off (don't > we??) in THP where things are automated as much as possible and the _kernel > decides_ what's best as far as is possible. > > As with swappiness, it is essentially a 'hint' to us in abstract terms rather > than simply exposing an internal kernel parameter. > > (Credit to Matthew for making this abstraction suggestion in the THP cabal > meeting by the way!) > > > > > > > > > An extreme example: if all your THPs have 2/512 pages populated, > > > that's still cutting TLB pressure in half! > > > > IIRC, you create more pressure on the huge entries, where you might have > > less TLB entries :) But yes, there can be cases where it is beneficial, if > > there is absolutely no memory pressure. > > > > > > > > So in the absence of memory pressure, allocating and collapsing should > > > optimally be aggressive even on very sparse regions. > > > > Yes, we discussed that as well in the THP cabal. > > > > It's very similar to the max_ptes_swapped: that parameter should not exist. > > If there is no memory pressure we can just swap it in. If there is memory > > pressure we probably would not want to swap in much. > > Yes, but at least an eagerness parameter gets us closer to this ideal. > > Of course, I agree that max_ptes_none should simply never have been exposed like > this. It is emblematic of a 'just shove a parameter into a tunable/sysfs and let > the user decide' approach you see in the kernel sometimes. > > This is problmeatic as users have no earthly idea how to set the parameter (most > likely never touch it), and only start fiddling should issues arise and it looks > like a viable solution of some kind. > > The problem is users usually lack a great deal of context the kernel has, and > may make incorrect decisions that work in one situation but not another. Note that in this case we really don't have much for context. We can trivially do "check what number of ptes are mapped", but not anything much fancier. You can also attempt to look at A bits (and/or check PG_referenced or PG_active). But currently there's really nothing setup to collect this information in a timely basis, and for anon memory (AFAIK) you only gauge this on reclaim, _if_ you find the page itself. The good news is that there are 3 or 4 separate movements for getting page "temperature" information with their own special infra and daemons, for their own special little features. > > TL;DR - this kind of interface is just lazy and we have to assess these kinds of > tunables based on the actual RoI + understanding from the user's perspective. Fully agreed. -- Pedro
On Fri, Sep 12, 2025 at 04:35:44PM +0100, Pedro Falcato wrote: > On Fri, Sep 12, 2025 at 03:01:02PM +0100, Lorenzo Stoakes wrote: > > Yes, but at least an eagerness parameter gets us closer to this ideal. > > > > Of course, I agree that max_ptes_none should simply never have been exposed like > > this. It is emblematic of a 'just shove a parameter into a tunable/sysfs and let > > the user decide' approach you see in the kernel sometimes. > > > > This is problmeatic as users have no earthly idea how to set the parameter (most > > likely never touch it), and only start fiddling should issues arise and it looks > > like a viable solution of some kind. > > > > The problem is users usually lack a great deal of context the kernel has, and > > may make incorrect decisions that work in one situation but not another. > > Note that in this case we really don't have much for context. We can trivially do > "check what number of ptes are mapped", but not anything much fancier. You can I mean we could in theory change where we determine things, for instance doing things in reclaim as Kiryl alluded to. We _potentially_ have more to work with. > > The good news is that there are 3 or 4 separate movements for getting page > "temperature" information with their own special infra and daemons, for their > own special little features. Right. > > > > > TL;DR - this kind of interface is just lazy and we have to assess these kinds of > > tunables based on the actual RoI + understanding from the user's perspective. > > Fully agreed. > > -- > Pedro My overall point, FWIW, is that a synthetic heuristic tunable works better here than one that maps on to an internal value that we then have no control over. Or 'I agree with David' IOW :) Cheers, Lorenzo
© 2016 - 2025 Red Hat, Inc.