[PATCH v2 0/3] mm/damon: support access monitoring of hugetlb-backed memory

Krishna Iyer posted 3 patches 3 weeks, 3 days ago
There is a newer version of this series
mm/damon/ops-common.c | 122 +++++++++++++++++++++++++++++++++++-------
mm/damon/ops-common.h |  10 ++++
mm/damon/paddr.c      |   4 +-
mm/damon/vaddr.c      |  27 ----------
4 files changed, 116 insertions(+), 47 deletions(-)
[PATCH v2 0/3] mm/damon: support access monitoring of hugetlb-backed memory
Posted by Krishna Iyer 3 weeks, 3 days ago
On virtualization hosts, most system memory is often backed by
hugetlbfs.  On our production hosts, for example, ~95% of RAM is 1 GiB
hugetlb pages backing guest memory.  DAMON's physical address space
monitoring is blind to such memory: every access check starts at
damon_get_folio(), which rejects folios that are not on the LRU lists,
and hugetlb folios are managed outside of the LRU by design.  As a
result, all hugetlb-backed memory is silently reported as never
accessed.  In testing on a 1 TiB host, an hour of 4-thread random
access over 842 GiB inside a guest was statistically indistinguishable
from an idle host.

The first patch moves damon_hugetlb_mkold() from vaddr to ops-common
as a preparation.  The second patch teaches the folio mkold/young rmap
walkers to handle hugetlb folios, aging the huge PTE and notifying
secondary MMUs across the whole huge page size; the secondary MMU
notification is what surfaces guest-side (e.g., KVM/EPT) accessed
bits.  The third patch adds damon_get_monitor_folio() and uses it from
the paddr monitoring primitives only.  DAMOS action appliers such as
DAMON_RECLAIM and DAMON_LRU_SORT keep the LRU-only lookup and are
behaviorally unchanged.

This series is the first half of an earlier six-patch series [1],
split out as SJ suggested [2].  The second half (the 'aging_flush'
TLB-flush-assisted aging) is deferred: we will gather more
quantitative data on the gap it addresses, including the workload-side
impact of the flushes and the working set measurement details SJ asked
about, and post it separately once the data is in hand, aligned with
the ongoing monitoring preparation actions work.

Per Documentation/process/generated-content.rst, this series was
developed with the assistance of an AI coding assistant (Anthropic
Claude, via Claude Code).  The assistant helped draft the code and
changelogs, and applied the v1 review feedback.  All changes were
reviewed by the human submitter, who takes full responsibility for the
contribution.

The series as posted here was regression-tested on its base commit
with a full x86_64 kernel build (no W=1 warnings in mm/damon), the
DAMON kunit suite (41/41 passing) and the DAMON selftests (15/15
passing) on a kernel booted with virtme-ng.

Changes from v1
(https://lore.kernel.org/20260830051407.50008-1-kiyer@crusoe.ai)
- Split the first three patches out as their own series, deferring the
  aging_flush patches (old patches 4-6) until more quantitative data
  is gathered (SJ Park)
- Drop too verbose comments on the hugetlb branches of the rmap
  walkers (SJ Park)
- Factor the non-hugetlb PTE young check out into damon_pte_young()
  to reduce indentation (SJ Park)
- Rename damon_get_folio_incl_hugetlb() to damon_get_monitor_folio()
  and damon_folio_observable(folio, incl_hugetlb) to
  damon_folio_acceptable(folio, monitor) (SJ Park)
- Drop 'kernel' from the host-side monitoring example in the patch 3
  changelog, since kernel memory is not LRU-managed and hence not
  monitorable regardless (SJ Park)
- Add SJ's Reviewed-by to patch 1

[1] https://lore.kernel.org/20260830051407.50008-1-kiyer@crusoe.ai
[2] https://lore.kernel.org/20260830180411.103772-1-sj@kernel.org

Krishna Iyer (3):
  mm/damon: move damon_hugetlb_mkold() from vaddr to ops-common
  mm/damon/ops-common: handle hugetlb folios in folio mkold/young rmap
    walkers
  mm/damon/paddr: support hugetlb folios in access monitoring

 mm/damon/ops-common.c | 122 +++++++++++++++++++++++++++++++++++-------
 mm/damon/ops-common.h |  10 ++++
 mm/damon/paddr.c      |   4 +-
 mm/damon/vaddr.c      |  27 ----------
 4 files changed, 116 insertions(+), 47 deletions(-)


base-commit: d2aad7fdcda7ae8a726926f2d6de7fe9e8ee7563
-- 
2.54.0
Re: [PATCH v2 0/3] mm/damon: support access monitoring of hugetlb-backed memory
Posted by SJ Park 3 weeks, 3 days ago
On Tue,  1 Sep 2026 19:56:57 -0700 Krishna Iyer <kiyer@crusoe.ai> wrote:

> On virtualization hosts, most system memory is often backed by
> hugetlbfs.  On our production hosts, for example, ~95% of RAM is 1 GiB
> hugetlb pages backing guest memory.  DAMON's physical address space
> monitoring is blind to such memory: every access check starts at
> damon_get_folio(), which rejects folios that are not on the LRU lists,
> and hugetlb folios are managed outside of the LRU by design.  As a
> result, all hugetlb-backed memory is silently reported as never
> accessed.  In testing on a 1 TiB host, an hour of 4-thread random
> access over 842 GiB inside a guest was statistically indistinguishable
> from an idle host.
> 
> The first patch moves damon_hugetlb_mkold() from vaddr to ops-common
> as a preparation.  The second patch teaches the folio mkold/young rmap
> walkers to handle hugetlb folios, aging the huge PTE and notifying
> secondary MMUs across the whole huge page size; the secondary MMU
> notification is what surfaces guest-side (e.g., KVM/EPT) accessed
> bits.  The third patch adds damon_get_monitor_folio() and uses it from
> the paddr monitoring primitives only.  DAMOS action appliers such as
> DAMON_RECLAIM and DAMON_LRU_SORT keep the LRU-only lookup and are
> behaviorally unchanged.
> 
> This series is the first half of an earlier six-patch series [1],
> split out as SJ suggested [2].  The second half (the 'aging_flush'
> TLB-flush-assisted aging) is deferred: we will gather more
> quantitative data on the gap it addresses, including the workload-side
> impact of the flushes and the working set measurement details SJ asked
> about, and post it separately once the data is in hand, aligned with
> the ongoing monitoring preparation actions work.
> 
> Per Documentation/process/generated-content.rst, this series was
> developed with the assistance of an AI coding assistant (Anthropic
> Claude, via Claude Code).  The assistant helped draft the code and
> changelogs, and applied the v1 review feedback.  All changes were
> reviewed by the human submitter, who takes full responsibility for the
> contribution.
> 
> The series as posted here was regression-tested on its base commit
> with a full x86_64 kernel build (no W=1 warnings in mm/damon), the
> DAMON kunit suite (41/41 passing) and the DAMON selftests (15/15
> passing) on a kernel booted with virtme-ng.

Looks good to me, thank you for this series Krishna!

With the comment modification I commented to the patch 3, I applied this series
to damon/next [1] tree.  Unless you raise other opinions or Andrew picks this
into mm.git with the comment modification, I will repost the version in my tree
as the next version of this series with the comment modification soon (up to ~1
week later).  If you have a different opinion for the comment modification, it
seems I forgot doing that or you cannot wait for my action, please feel free to
let me know or post the next version on your own.

[1] https://origin.kernel.org/doc/html/latest/mm/damon/maintainer-profile.html#scm-trees


Thanks,
SJ

[...]