[v6] mm: add per-order mTHP alloc and swpout counters

[PATCH v6 0/4] mm: add per-order mTHP alloc and swpout counters

Posted by Barry Song 1 year, 8 months ago

From: Barry Song <v-songbaohua@oppo.com>

The patchset introduces a framework to facilitate mTHP counters, starting
with the allocation and swap-out counters. Currently, only four new nodes
are appended to the stats directory for each mTHP size.

/sys/kernel/mm/transparent_hugepage/hugepages-<size>/stats
	anon_fault_alloc
	anon_fault_fallback
	anon_fault_fallback_charge
	anon_swpout
	anon_swpout_fallback

These nodes are crucial for us to monitor the fragmentation levels of
both the buddy system and the swap partitions. In the future, we may
consider adding additional nodes for further insights.

-v6:
  * collect reviewed-by tags for patch2/4, 3/4, 4/4, Ryan;
  * move back to static array by using MAX_PTRS_PER_PTE, Ryan;
  * move to for_each_possible_cpu to handle cpu hotplug, Ryan;
  * other minor cleanups according to Ryan;
 
-v5:
  * rename anon_alloc to anon_fault_alloc, Barry/Ryan;
  * add anon_fault_fallback_charge, Ryan;
  * move to dynamic alloc_percpu as powerpc's PMD_ORDER is not const,
    kernel test robot;
  * make anon_fault_alloc and anon_fault_fallback more consistent
    with thp_fault_alloc and thp_fault_fallback, Ryan;
  * handle cpu hotplug properly, Ryan;
  * add docs for new sysfs nodes and ABI, Andrew.
  link:
  https://lore.kernel.org/linux-mm/20240412073740.294272-1-21cnbao@gmail.com/

-v4:
  * Many thanks to David and Ryan for your patience and valuable insights
    throughout the numerous renaming efforts!
  * Guard the case order > PMD_ORDER in count func rather than in callers,
    Ryan;
  * Add swpout counters;
  * Add a helper DEFINE_MTHP_STAT_ATTR to avoid code duplication for various
    counters;
  link:
  https://lore.kernel.org/linux-mm/20240405102704.77559-1-21cnbao@gmail.com/

-v3:
  https://lore.kernel.org/linux-mm/20240403035502.71356-1-21cnbao@gmail.com/

Barry Song (4):
  mm: add per-order mTHP anon_fault_alloc and anon_fault_fallback
    counters
  mm: add per-order mTHP anon_swpout and anon_swpout_fallback counters
  mm: add docs for per-order mTHP counters and transhuge_page ABI
  mm: correct the docs for thp_fault_alloc and thp_fault_fallback

 .../sys-kernel-mm-transparent-hugepage        | 17 ++++++
 Documentation/admin-guide/mm/transhuge.rst    | 32 ++++++++++-
 include/linux/huge_mm.h                       | 23 ++++++++
 mm/huge_memory.c                              | 56 +++++++++++++++++++
 mm/memory.c                                   |  5 ++
 mm/page_io.c                                  |  1 +
 mm/vmscan.c                                   |  3 +
 7 files changed, 135 insertions(+), 2 deletions(-)
 create mode 100644 Documentation/ABI/testing/sys-kernel-mm-transparent-hugepage

-- 
2.34.1

Re: [PATCH v6 0/4] mm: add per-order mTHP alloc and swpout counters

Posted by David Hildenbrand 1 year, 8 months ago

On 12.04.24 13:48, Barry Song wrote:
> From: Barry Song <v-songbaohua@oppo.com>
> 
> The patchset introduces a framework to facilitate mTHP counters, starting
> with the allocation and swap-out counters. Currently, only four new nodes
> are appended to the stats directory for each mTHP size.
> 
> /sys/kernel/mm/transparent_hugepage/hugepages-<size>/stats
> 	anon_fault_alloc
> 	anon_fault_fallback
> 	anon_fault_fallback_charge
> 	anon_swpout
> 	anon_swpout_fallback
> 
> These nodes are crucial for us to monitor the fragmentation levels of
> both the buddy system and the swap partitions. In the future, we may
> consider adding additional nodes for further insights.
> 
> -v6:
>    * collect reviewed-by tags for patch2/4, 3/4, 4/4, Ryan;
>    * move back to static array by using MAX_PTRS_PER_PTE, Ryan;
>    * move to for_each_possible_cpu to handle cpu hotplug, Ryan;
>    * other minor cleanups according to Ryan;

Please *really* not multiple versions of the same patch set on one a 
single day.

-- 
Cheers,

David / dhildenb

Re: [PATCH v6 0/4] mm: add per-order mTHP alloc and swpout counters

Posted by Barry Song 1 year, 8 months ago

On Sat, Apr 13, 2024 at 12:54 AM David Hildenbrand <david@redhat.com> wrote:
>
> On 12.04.24 13:48, Barry Song wrote:
> > From: Barry Song <v-songbaohua@oppo.com>
> >
> > The patchset introduces a framework to facilitate mTHP counters, starting
> > with the allocation and swap-out counters. Currently, only four new nodes
> > are appended to the stats directory for each mTHP size.
> >
> > /sys/kernel/mm/transparent_hugepage/hugepages-<size>/stats
> >       anon_fault_alloc
> >       anon_fault_fallback
> >       anon_fault_fallback_charge
> >       anon_swpout
> >       anon_swpout_fallback
> >
> > These nodes are crucial for us to monitor the fragmentation levels of
> > both the buddy system and the swap partitions. In the future, we may
> > consider adding additional nodes for further insights.
> >
> > -v6:
> >    * collect reviewed-by tags for patch2/4, 3/4, 4/4, Ryan;
> >    * move back to static array by using MAX_PTRS_PER_PTE, Ryan;
> >    * move to for_each_possible_cpu to handle cpu hotplug, Ryan;
> >    * other minor cleanups according to Ryan;
>
> Please *really* not multiple versions of the same patch set on one a
> single day.

Ok. I will leave more time for you to review the older versions before moving
to a new version.

For v5->v6,  it is quite a straightforward re-spin though I can understand
it might be a bit annoying if you got v6 while you were reading v5.

>
> --
> Cheers,
>
> David / dhildenb

Thanks
Barry

Re: [PATCH v6 0/4] mm: add per-order mTHP alloc and swpout counters

Posted by David Hildenbrand 1 year, 8 months ago

On 12.04.24 15:16, Barry Song wrote:
> On Sat, Apr 13, 2024 at 12:54 AM David Hildenbrand <david@redhat.com> wrote:
>>
>> On 12.04.24 13:48, Barry Song wrote:
>>> From: Barry Song <v-songbaohua@oppo.com>
>>>
>>> The patchset introduces a framework to facilitate mTHP counters, starting
>>> with the allocation and swap-out counters. Currently, only four new nodes
>>> are appended to the stats directory for each mTHP size.
>>>
>>> /sys/kernel/mm/transparent_hugepage/hugepages-<size>/stats
>>>        anon_fault_alloc
>>>        anon_fault_fallback
>>>        anon_fault_fallback_charge
>>>        anon_swpout
>>>        anon_swpout_fallback
>>>
>>> These nodes are crucial for us to monitor the fragmentation levels of
>>> both the buddy system and the swap partitions. In the future, we may
>>> consider adding additional nodes for further insights.
>>>
>>> -v6:
>>>     * collect reviewed-by tags for patch2/4, 3/4, 4/4, Ryan;
>>>     * move back to static array by using MAX_PTRS_PER_PTE, Ryan;
>>>     * move to for_each_possible_cpu to handle cpu hotplug, Ryan;
>>>     * other minor cleanups according to Ryan;
>>
>> Please *really* not multiple versions of the same patch set on one a
>> single day.
> 
> Ok. I will leave more time for you to review the older versions before moving
> to a new version.

Yes please. There is not anything gained from sending out stuff to fast, 
besides mixing discussions, same questions/comments ... and effectively 
more work for reviewers.

-- 
Cheers,

David / dhildenb