[RFC PATCH v1 0/4] mm/damon: Introduce a huge page collapsing mechanism using auto tuning

gutierrez.asier@huawei-partners.com posted 4 patches 1 month, 2 weeks ago
There is a newer version of this series
mm/damon/modules-common.c                     |  33 +-
mm/damon/modules-common.h                     |   3 +
9 files changed, 652 insertions(+), 8 deletions(-)
create mode 100644 Documentation/admin-guide/mm/damon/hugepage.rst
create mode 100644 mm/damon/hugepage.c
[RFC PATCH v1 0/4] mm/damon: Introduce a huge page collapsing mechanism using auto tuning
Posted by gutierrez.asier@huawei-partners.com 1 month, 2 weeks ago
From: Asier Gutierrez <gutierrez.asier@huawei-partners.com>

Overview
================

This patch set introduces a new autotuning which allows to collapse
hot regions into hugepages.

Motivation
================

Since TLB is a bottleneck for many systems, a way to optimize TLB
misses (or hits) is to use huge pages. Unfortunately, using "always"
in THP leads to memory fragmentation and memory waste. For this reason,
most application guides and system administrators suggest to disable THP.

Solution
================

A new autotuning quota goal[1], damos_get_used_hugepage_mem_bp, is
introduced, which checks the huge page consumption to total anonymous
memory consumption. This new quota mechanism reuses current autotuning
architecture.

A new module is introduced to demonstrate the use of huge pages
collapse autotuning. The module launches a kdamond thread for a 
certain task provided by the user through monitored_pid module argument.

This module also has a user autotuning knob which allows the user to
adjust the aggressiveness of page collapsing.

Benchmarks
================
In progress, will add them in the next RFC series.

TODO
================

- Since DAMOS_COLLAPSE is not in upstream and this is just a
  RFC, I have used DAMOS_HUGEPAGE instead. This will change
  to DAMOS_COLLAPSE in future series.

Patches Sequence
================
Patch 1 -> damon_modules_new_vaddr_ctx_target
Patch 2 -> Introduce DAMOS_QUOTA_HUGEPAGE and autotuning
Patch 3 -> Module that demonstrates how to use DAMOS_QUOTA_HUGEPAGE
           and the new VADDR ctx creation
Patch 4 -> Documentation

[1] https://lore.kernel.org/e67f05ad-dbb9-45e6-ba30-b167a99ac67d@huawei-partners.com

 .../admin-guide/mm/damon/hugepage.rst (new)   | 258 +++++++++++++
 Documentation/admin-guide/mm/damon/index.rst  |   1 +
 include/linux/damon.h                         |   1 +
 mm/damon/Kconfig                              |   7 +
 mm/damon/Makefile                             |   1 +
 mm/damon/core.c                               |  15 +
 mm/damon/hugepage.c (new)                     | 341 ++++++++++++++++++
 mm/damon/modules-common.c                     |  33 +-
 mm/damon/modules-common.h                     |   3 +
 9 files changed, 652 insertions(+), 8 deletions(-)
 create mode 100644 Documentation/admin-guide/mm/damon/hugepage.rst
 create mode 100644 mm/damon/hugepage.c

-- 
2.43.0
Re: [RFC PATCH v1 0/4] mm/damon: Introduce a huge page collapsing mechanism using auto tuning
Posted by SeongJae Park 1 month, 2 weeks ago
Hello Asier,

On Thu, 30 Apr 2026 13:41:35 +0000 <gutierrez.asier@huawei-partners.com> wrote:

> From: Asier Gutierrez <gutierrez.asier@huawei-partners.com>
> 
> Overview
> ================

Let's align the length of underline with the text.

> 
> This patch set introduces a new autotuning which allows to collapse
> hot regions into hugepages.
> 
> Motivation
> ================
> 
> Since TLB is a bottleneck for many systems, a way to optimize TLB
> misses (or hits) is to use huge pages. Unfortunately, using "always"
> in THP leads to memory fragmentation and memory waste. For this reason,
> most application guides and system administrators suggest to disable THP.

I think the motivation should further explain why existing DAMOS features
(access pattern-based targetting and quota auto-tuning with existing quota
goals) are insufficient.  If that makes the text too long, I think the
explanation for well-known THP benefit and problems can be reduced or skipped.

> 
> Solution
> ================
> 
> A new autotuning quota goal[1], damos_get_used_hugepage_mem_bp, is
> introduced, which checks the huge page consumption to total anonymous
> memory consumption. This new quota mechanism reuses current autotuning
> architecture.

The idea makes sense to me.

The name sounds bit redundant, though.  How about DAMOS_QUOTA_HUGEPAGE_BP for
the enum, and hugeapge_bp for sysfs input?

> 
> A new module is introduced to demonstrate the use of huge pages
> collapse autotuning. The module launches a kdamond thread for a 
> certain task provided by the user through monitored_pid module argument.

If it is only for demonstration, I think it is more fit to be a sample module
(placed under samples/damon/).

> 
> This module also has a user autotuning knob which allows the user to
> adjust the aggressiveness of page collapsing.
> 
> Benchmarks
> ================
> In progress, will add them in the next RFC series.

Looking forward to!

I may request not that much test results for the quota goal.  If the module is
not just a sample but for a real world use case, I may hope somewhat reasonable
test resulsts.

> 
> TODO
> ================
> 
> - Since DAMOS_COLLAPSE is not in upstream and this is just a
>   RFC, I have used DAMOS_HUGEPAGE instead. This will change
>   to DAMOS_COLLAPSE in future series.

It is now in mm-new and it is recommended to use it as a baseline for DAMON
patches.  So please feel free to use DAMOS_COLLAPSE from the next version,
unless there is a reason to worry if DAMOS_COLLAPSE will be dropped from
mm-new.

> 
> Patches Sequence
> ================
> Patch 1 -> damon_modules_new_vaddr_ctx_target
> Patch 2 -> Introduce DAMOS_QUOTA_HUGEPAGE and autotuning
> Patch 3 -> Module that demonstrates how to use DAMOS_QUOTA_HUGEPAGE
>            and the new VADDR ctx creation
> Patch 4 -> Documentation
> 
> [1] https://lore.kernel.org/e67f05ad-dbb9-45e6-ba30-b167a99ac67d@huawei-partners.com

Adding more context about what this link is would be nice.  I can find the link
marker from the above text, but I was unable to expect what this is without
opening the link.


Thanks,
SJ

[...]
Re: [RFC PATCH v1 0/4] mm/damon: Introduce a huge page collapsing mechanism using auto tuning
Posted by Gutierrez Asier 1 month, 2 weeks ago
On 5/1/2026 3:41 AM, SeongJae Park wrote:
> Hello Asier,
Hi, sorry for the delayed answer.
> 
> On Thu, 30 Apr 2026 13:41:35 +0000 <gutierrez.asier@huawei-partners.com> wrote:
> 
>> From: Asier Gutierrez <gutierrez.asier@huawei-partners.com>
>>
>> Overview
>> ================
> 
> Let's align the length of underline with the text.
> 
>>
>> This patch set introduces a new autotuning which allows to collapse
>> hot regions into hugepages.
>>
>> Motivation
>> ================
>>
>> Since TLB is a bottleneck for many systems, a way to optimize TLB
>> misses (or hits) is to use huge pages. Unfortunately, using "always"
>> in THP leads to memory fragmentation and memory waste. For this reason,
>> most application guides and system administrators suggest to disable THP.
> 
> I think the motivation should further explain why existing DAMOS features
> (access pattern-based targetting and quota auto-tuning with existing quota
> goals) are insufficient.  If that makes the text too long, I think the
> explanation for well-known THP benefit and problems can be reduced or skipped.
OK, I will elaborate it in further series.
> 
>>
>> Solution
>> ================
>>
>> A new autotuning quota goal[1], damos_get_used_hugepage_mem_bp, is
>> introduced, which checks the huge page consumption to total anonymous
>> memory consumption. This new quota mechanism reuses current autotuning
>> architecture.
> 
> The idea makes sense to me.
> 
> The name sounds bit redundant, though.  How about DAMOS_QUOTA_HUGEPAGE_BP for
> the enum, and hugeapge_bp for sysfs input?
Agree. In fact, DAMOS_QUOTA_HUGEPAGE_BP would make sense to keep it
consistent with the quota naming.
> 
>>
>> A new module is introduced to demonstrate the use of huge pages
>> collapse autotuning. The module launches a kdamond thread for a 
>> certain task provided by the user through monitored_pid module argument.
> 
> If it is only for demonstration, I think it is more fit to be a sample module
> (placed under samples/damon/).
We are testing this in a bare metal MySQL server. Maybe the description
was not the most accurate.
>>
>> This module also has a user autotuning knob which allows the user to
>> adjust the aggressiveness of page collapsing.
>>
>> Benchmarks
>> ================
>> In progress, will add them in the next RFC series.
> 
> Looking forward to!
> 
> I may request not that much test results for the quota goal.  If the module is
> not just a sample but for a real world use case, I may hope somewhat reasonable
> test resulsts.
OK, will do it.
>>
>> TODO
>> ================
>>
>> - Since DAMOS_COLLAPSE is not in upstream and this is just a
>>   RFC, I have used DAMOS_HUGEPAGE instead. This will change
>>   to DAMOS_COLLAPSE in future series.
> 
> It is now in mm-new and it is recommended to use it as a baseline for DAMON
> patches.  So please feel free to use DAMOS_COLLAPSE from the next version,
> unless there is a reason to worry if DAMOS_COLLAPSE will be dropped from
> mm-new.
Thanks.
>>
>> Patches Sequence
>> ================
>> Patch 1 -> damon_modules_new_vaddr_ctx_target
>> Patch 2 -> Introduce DAMOS_QUOTA_HUGEPAGE and autotuning
>> Patch 3 -> Module that demonstrates how to use DAMOS_QUOTA_HUGEPAGE
>>            and the new VADDR ctx creation
>> Patch 4 -> Documentation
>>
>> [1] https://lore.kernel.org/e67f05ad-dbb9-45e6-ba30-b167a99ac67d@huawei-partners.com
> 
> Adding more context about what this link is would be nice.  I can find the link
> marker from the above text, but I was unable to expect what this is without
> opening the link.
> 
> 
> Thanks,
> SJ
> 
> [...]
> 

-- 
Asier Gutierrez
Huawei
Re: [RFC PATCH v1 0/4] mm/damon: Introduce a huge page collapsing mechanism using auto tuning
Posted by SeongJae Park 1 month, 1 week ago
On Mon, 4 May 2026 16:52:01 +0300 Gutierrez Asier <gutierrez.asier@huawei-partners.com> wrote:

> On 5/1/2026 3:41 AM, SeongJae Park wrote:
> > Hello Asier,
> Hi, sorry for the delayed answer.

No worry, no rush.  Take your time :)

> > 
> > On Thu, 30 Apr 2026 13:41:35 +0000 <gutierrez.asier@huawei-partners.com> wrote:
[...]
> >> A new module is introduced to demonstrate the use of huge pages
> >> collapse autotuning. The module launches a kdamond thread for a 
> >> certain task provided by the user through monitored_pid module argument.
> > 
> > If it is only for demonstration, I think it is more fit to be a sample module
> > (placed under samples/damon/).
> We are testing this in a bare metal MySQL server. Maybe the description
> was not the most accurate.

Do you mean it is not only for demonstration but a real world use case and
therefore better not to be a sample module?

> >>
> >> This module also has a user autotuning knob which allows the user to
> >> adjust the aggressiveness of page collapsing.
> >>
> >> Benchmarks
> >> ================
> >> In progress, will add them in the next RFC series.
> > 
> > Looking forward to!
> > 
> > I may request not that much test results for the quota goal.  If the module is
> > not just a sample but for a real world use case, I may hope somewhat reasonable
> > test resulsts.
> OK, will do it.

Thank you for kindly and flexibly accepting my humble suggestions :)


Thanks,
SJ

[...]