[PATCH 0/8] mm/damon: introduce hugepage_size probe filter

SJ Park posted 8 patches 1 week, 2 days ago
There is a newer version of this series
.../ABI/testing/sysfs-kernel-mm-damon         | 14 +++++
Documentation/admin-guide/mm/damon/usage.rst  |  4 +-
Documentation/mm/damon/design.rst             |  2 +
include/linux/damon.h                         |  8 +++
mm/damon/core.c                               | 28 ++++++++--
mm/damon/ops-common.c                         | 10 ++++
mm/damon/sysfs.c                              | 55 +++++++++++++++++++
7 files changed, 115 insertions(+), 6 deletions(-)
[PATCH 0/8] mm/damon: introduce hugepage_size probe filter
Posted by SJ Park 1 week, 2 days ago
Knowing whether a given memory is backed by a hugepage of specific size
is useful for efficient utilization of hugepages.  For easy monitoring
of the information, introduce a new data attribute probe filter type,
hugepage_size.  It works similar to the DAMOS filter of the same name.
It works for memory that is backed by a hugepage of a given size range.

Patch 1 introduces the new probe filter type to DAMON API and extends
related data structures.  Patch 2 updates probe filter commit logic to
handle the size range.  Patch 3 Updates the filtering logic to support
the new type.  Patch 4 adds new DAMON sysfs files for the size range.
Patch 5 updates DAMON sysfs interface to fully support the new filter
type.  Patches 6-8 updates design, usage and ABI documents for the new
feature.

Test
====

Set THP always and start a test workload using masim [1].

    $ echo always | sudo tee /sys/kernel/mm/transparent_hugepage/enabled
    $ ./masim.py run --config_file ./configs/stairs-500mb.cfg --repeat 10

Using the DAMON user-space tool, damo [2], start DAMON with two probes.
The primary probe monitors access.  The secondary probe monitors 2 MiB
sized hugepages.  The region adjustment weight is given to only the
primary probe.  That is, this setup is for showing access temperature
and additionally how many 2 MiB hugepages are being used for the region
of the access temperature.

    $ sudo ./damo start $(pidof masim) \
            --probe \
                --probe_prep set_pgidle --probe_filter allow pgidle_unset \
                --probe_weight 1 \
            --probe \
                --probe_filter allow hugepage_size 2M 2M

Show the monitoring results.

    $ sudo ./damo report attrs
    heatmap: 000000000003[...]666666666666666666666666666666666666666666666668888888888888988888888[...]8
    # min/max temperatures: -2,770,000,000, 68,474,934, column size: 69.784 MiB
    probes
    - prep: set_pgidle, filter: allow pgidle_unset (weight: 1)
    - allow hugepage_size [2.000 MiB, 2.000 MiB] (weight: 0)
    intervals: sample 5 ms aggr 100 ms (max probe hits 20)
    #   size         address       age           probe_hits
    0   250.805 MiB  86.185 TiB    27.700 s      0 0
    1   556.285 MiB  86.184 TiB    27.400 s      0 0
    2   1.629 GiB    127.488 TiB   8.500 s       0 4
    3   535.770 MiB  127.490 TiB   8.400 s       0 7
    4   1.066 GiB    127.490 TiB   8.200 s       0 6
    5   501.332 MiB  127.492 TiB   600 ms        0 12
    [...]
    50  146.133 MiB  127.492 TiB   200 ms        2 20
    51  16.000 KiB   127.492 TiB   200 ms        11 20
    52  144.000 KiB  127.492 TiB   200 ms        17 20
    53  48.000 KiB   127.492 TiB   200 ms        18 20
    54  1.930 MiB    127.492 TiB   200 ms        19 20
    [...]
    101 1.996 MiB    127.492 TiB   2.100 s       1 20
    102 1.992 MiB    127.492 TiB   2.100 s       20 20
    103 1.984 MiB    127.492 TiB   2.200 s       1 20
    104 2.000 MiB    127.492 TiB   2.300 s       20 20
    105 2.000 MiB    127.492 TiB   2.600 s       20 20
    memory bw estimate: 0 B per second
    total size: 5.452 GiB
    record DAMON intervals: sample 5 ms, aggr 100 ms

The first probe_hits count and the age show the access temperature of
each region.  The region is sorted by access temperature, which is a
weighted sum of the first probe_hits count and the age.  That is,
regions at the beginning of the list are colder than those at the end of
the list.  The second probe_hits count shows the ratio of 2 MiB
hugepages in the region.

For example, region 5, which has 501.332 MiB size, has not accessed for
the last 600 milliseconds and uses 2 MiB THP for ~60% (12/20) of the
region.

Region 105, which is 2 MiB size, has been accessed at least 20 times per
100 milliseconds for the last 2.6 seconds) and fully backed by 2 MiB
hugepage.

[1] https://github.com/sjp38/masim
[2] https://github.com/damonitor/damo

Changes from RFC v3
- RFC v3: https://lore.kernel.org/20260915020929.102953-1-sj@kernel.org
- Drop RFC tag.
- Rebase to lastes mm-new.
Changes from RFC v2
- RFC v2: https://lore.kernel.org/20260914144448.93732-1-sj@kernel.org
- Free new filter for invalid filter commit case.
Changes from RFC
- RFC: https://lore.kernel.org/20260913171133.102366-1-sj@kernel.org
- Commit size range for newly added hugepage_size filter.
- Rebase to latest mm-new.

SJ Park (8):
  mm/damon/api: introduce DAMON_FILTER_TYPE_HUGEPAGE_SIZE
  mm/damon/core: commit hugepage_size type damon filter
  mm/damon/ops-common: support hugepage_size damon filter matching
  mm/damon/sysfs: add min,max files under probe filter directory
  mm/damon/sysfs: support hugepage_size probe filter
  Docs/mm/damon/design: update for hugepage_size probe filter
  Docs/admin-guide/mm/damon/usage: update for hugepage_size
  Docs/ABI/damon: update for hugepage_size probe filter

 .../ABI/testing/sysfs-kernel-mm-damon         | 14 +++++
 Documentation/admin-guide/mm/damon/usage.rst  |  4 +-
 Documentation/mm/damon/design.rst             |  2 +
 include/linux/damon.h                         |  8 +++
 mm/damon/core.c                               | 28 ++++++++--
 mm/damon/ops-common.c                         | 10 ++++
 mm/damon/sysfs.c                              | 55 +++++++++++++++++++
 7 files changed, 115 insertions(+), 6 deletions(-)


base-commit: c93300483b699bbc752f1000a1309e8fcf2c9fb2
-- 
2.47.3
Re: [PATCH 0/8] mm/damon: introduce hugepage_size probe filter
Posted by Andrew Morton 1 week, 2 days ago
On Tue, 15 Sep 2026 07:33:49 -0700 SJ Park <sj@kernel.org> wrote:

> Knowing whether a given memory is backed by a hugepage of specific size
> is useful for efficient utilization of hugepages.  For easy monitoring
> of the information, introduce a new data attribute probe filter type,
> hugepage_size.  It works similar to the DAMOS filter of the same name.
> It works for memory that is backed by a hugepage of a given size range.
> 
> Patch 1 introduces the new probe filter type to DAMON API and extends
> related data structures.  Patch 2 updates probe filter commit logic to
> handle the size range.  Patch 3 Updates the filtering logic to support
> the new type.  Patch 4 adds new DAMON sysfs files for the size range.
> Patch 5 updates DAMON sysfs interface to fully support the new filter
> type.  Patches 6-8 updates design, usage and ABI documents for the new
> feature.

All queued up, thanks.  With a -fix for [7/8].

> Test

fyi, I'm always reluctant to include testing info in the permanent
record.  Because it's development-time info which becomes obsolete very
quickly.

It isn't useful to someone who is reading the changelog in mainline in
2028 (or even Nov 2026) because the patchset will have had far more testing
since these words were written.

It is of course very useful info but I suggest it be maintained below
the "---", where all short-term development-time info is best
maintained.
Re: [PATCH 0/8] mm/damon: introduce hugepage_size probe filter
Posted by SJ Park 1 week, 2 days ago
Hi Andrew,

On Tue, 15 Sep 2026 16:39:28 -0700 Andrew Morton <akpm@linux-foundation.org> wrote:

> On Tue, 15 Sep 2026 07:33:49 -0700 SJ Park <sj@kernel.org> wrote:
> 
> > Knowing whether a given memory is backed by a hugepage of specific size
> > is useful for efficient utilization of hugepages.  For easy monitoring
> > of the information, introduce a new data attribute probe filter type,
> > hugepage_size.  It works similar to the DAMOS filter of the same name.
> > It works for memory that is backed by a hugepage of a given size range.
> > 
> > Patch 1 introduces the new probe filter type to DAMON API and extends
> > related data structures.  Patch 2 updates probe filter commit logic to
> > handle the size range.  Patch 3 Updates the filtering logic to support
> > the new type.  Patch 4 adds new DAMON sysfs files for the size range.
> > Patch 5 updates DAMON sysfs interface to fully support the new filter
> > type.  Patches 6-8 updates design, usage and ABI documents for the new
> > feature.
> 
> All queued up, thanks.  With a -fix for [7/8].

Thank you!  I found the -fix is in the mm.git as below:

'''
$ git show 81c69fbb4128
[...]
--- a/Documentation/admin-guide/mm/damon/usage.rst
+++ b/Documentation/admin-guide/mm/damon/usage.rst
@@ -309,7 +309,7 @@ named ``0`` to ``N-1``.  Each directory represents each filter and works in a
 way similar to that for :ref:`DAMOS filter <sysfs_filters>`.  When the filter
 ``type`` is ``memcg``, ``path`` file acts as ``memcg_path`` for :ref:`DAMOS
 filter <sysfs_filters>`.  When the filter ``type`` is ``hugepage_size``,
-``min`` and ``max`` files acts as files of the same names for :ref:`DAMOS
+``min`` and ``max`` file acts as files of the same names for :ref:`DAMOS
 filter <sysfs_filters>`.

 .. _sysfs_targets:
'''

But, I think it is slightly wrong.  The fix I requested is
's/files acts/files act/', but the committed fix is doing
's/files acts/file acts/'.  Could you please fix it when you get a chance?  Let
me know if there is anything that I can help, too.


Thanks,
SJ

[...]
Re: [PATCH 0/8] mm/damon: introduce hugepage_size probe filter
Posted by SJ Park 1 week, 2 days ago
On Tue, 15 Sep 2026 16:39:28 -0700 Andrew Morton <akpm@linux-foundation.org> wrote:

> On Tue, 15 Sep 2026 07:33:49 -0700 SJ Park <sj@kernel.org> wrote:
> 
> > Knowing whether a given memory is backed by a hugepage of specific size
> > is useful for efficient utilization of hugepages.  For easy monitoring
> > of the information, introduce a new data attribute probe filter type,
> > hugepage_size.  It works similar to the DAMOS filter of the same name.
> > It works for memory that is backed by a hugepage of a given size range.
> > 
> > Patch 1 introduces the new probe filter type to DAMON API and extends
> > related data structures.  Patch 2 updates probe filter commit logic to
> > handle the size range.  Patch 3 Updates the filtering logic to support
> > the new type.  Patch 4 adds new DAMON sysfs files for the size range.
> > Patch 5 updates DAMON sysfs interface to fully support the new filter
> > type.  Patches 6-8 updates design, usage and ABI documents for the new
> > feature.
> 
> All queued up, thanks.  With a -fix for [7/8].
> 
> > Test
> 
> fyi, I'm always reluctant to include testing info in the permanent
> record.  Because it's development-time info which becomes obsolete very
> quickly.
> 
> It isn't useful to someone who is reading the changelog in mainline in
> 2028 (or even Nov 2026) because the patchset will have had far more testing
> since these words were written.
> 
> It is of course very useful info but I suggest it be maintained below
> the "---", where all short-term development-time info is best
> maintained.

Agreed.  I will do so for individual patches.  For cover letters that don't
have the "---" lines, I will add an informal note saying "please do not add
below to the commit message".  If you prefer, I could add "---" line on the
cover letter instead of the informal note.  Or, I could add the informal note
below the manual "---" line, too.  Let me know if you prefer that.


Thanks,
SJ
Re: [PATCH 0/8] mm/damon: introduce hugepage_size probe filter
Posted by SJ Park 1 week, 2 days ago
On Tue, 15 Sep 2026 07:33:49 -0700 SJ Park <sj@kernel.org> wrote:

> Knowing whether a given memory is backed by a hugepage of specific size
> is useful for efficient utilization of hugepages.  For easy monitoring
> of the information, introduce a new data attribute probe filter type,
> hugepage_size.  It works similar to the DAMOS filter of the same name.
> It works for memory that is backed by a hugepage of a given size range.

Sashiko found no blocker for this series other than a grammartical error in a
document of patch 7.  It is too trivial to respin the series in my opinion.
Andrew, could you please fix it together when you pick this, or let me know if
you have a different opinion?

Sashiko send its findings to damon@ mailing list [1], and I reply those if
anything needs to be clarified.  Please refer to those for details.

[1] https://lore.kernel.org/damon/


Thanks,
SJ

[...]