From: Hui Zhu <zhuhui@kylinos.cn>

Commit 7404bd37cfbe ("mm: workingset: use lruvec_lru_size() to get the
number of lru pages") broke the workingset shadow node budget under
MGLRU: lruvec_lru_size() reads mz->lru_zone_size, which MGLRU never
maintains, so count_shadow_nodes() sees the evictable LRU lists as
empty and the shadow shrinker reclaims eviction tokens almost as fast
as they are created, losing thrashing protection.

Patch 1 switches count_shadow_nodes() back to lruvec_page_state_local(),
which both classic LRU and MGLRU maintain.

Patch 2 addresses the reparenting race that motivated 7404bd37cfbe:
patch 1 makes cgroup v2 read state_local as well, so extend the
dying-memcg stat redirection (previously cgroup v1 only) to all
hierarchies.

Patch 3 recovers the performance.  Patch 2 added an unconditional
rcu_read_lock() to the stat update fast path; patch 3 moves the dying
check out of the RCU read-side critical section so the lock is only
taken on the rare dying path.

Performance testing
===================

The test script and the raw results are available at [1].

Environment: 10-vCPU QEMU guest, 8 GiB RAM, cgroup v2; 7 runs per
configuration, medians reported.  Workloads:

  w1-anon-churn: single-threaded anon fault/charge loop in a memcg
                 (MADV_DONTNEED + re-fault, no reclaim).  Every touch
                 is a real fault with charge and memcg stat updates,
                 so it stresses exactly the fast path patch 2 changes.
  w2-file-churn: file read loop under memory.high pressure
                 (reclaim-bound, noisier).
  w3-reparent:   reparent accounting sanity check.

w1-anon-churn (pages/s):

                 classic LRU          MGLRU
base             4393028              4377122
patches 1-2      4385996   (-0.2%)    4352887   (-0.6%)
patches 1-3      4381832   (-0.3%)    4377053   (+0.0%)

w2-file-churn (MB/s):

                 classic LRU          MGLRU
base             8277                 8226
patches 1-2      8226      (-0.6%)    8123      (-1.3%)
patches 1-3      8157      (-1.4%)    8294      (+0.8%)

w3-reparent passed on all kernels.

Patch 2 alone shows a small overhead, most visible under MGLRU (-0.6%
on w1); patch 3 brings w1 back to the base level in both LRU
configurations.  The remaining differences are within run-to-run
noise.

[1] https://gist.github.com/teawater/32f373ec41d185d840455eb167321a5a

Hui Zhu (3):
  mm: workingset: use lruvec_page_state_local() to count lru pages
  mm: memcg: redirect stats updates of dying memcgs for all hierarchies
  mm: memcg: skip the RCU lock when the memcg is not dying

 mm/memcontrol.c | 30 ++++++++++++------------------
 mm/workingset.c |  5 ++---
 2 files changed, 14 insertions(+), 21 deletions(-)

-- 
2.53.0

On Mon, Aug 31, 2026 at 05:46:08PM +0800, Hui Zhu wrote:
> From: Hui Zhu <zhuhui@kylinos.cn>
> 
> Commit 7404bd37cfbe ("mm: workingset: use lruvec_lru_size() to get the
> number of lru pages") broke the workingset shadow node budget under
> MGLRU: lruvec_lru_size() reads mz->lru_zone_size, which MGLRU never
> maintains, so count_shadow_nodes() sees the evictable LRU lists as
> empty and the shadow shrinker reclaims eviction tokens almost as fast
> as they are created, losing thrashing protection.
> 
> Patch 1 switches count_shadow_nodes() back to lruvec_page_state_local(),
> which both classic LRU and MGLRU maintain.
> 
> Patch 2 addresses the reparenting race that motivated 7404bd37cfbe:
> patch 1 makes cgroup v2 read state_local as well, so extend the
> dying-memcg stat redirection (previously cgroup v1 only) to all
> hierarchies.
> 
> Patch 3 recovers the performance.  Patch 2 added an unconditional
> rcu_read_lock() to the stat update fast path; patch 3 moves the dying
> check out of the RCU read-side critical section so the lock is only
> taken on the rare dying path.
> 
> Performance testing
> ===================
> 
> The test script and the raw results are available at [1].
> 
> Environment: 10-vCPU QEMU guest, 8 GiB RAM, cgroup v2; 7 runs per
> configuration, medians reported.  Workloads:
> 
>   w1-anon-churn: single-threaded anon fault/charge loop in a memcg
>                  (MADV_DONTNEED + re-fault, no reclaim).  Every touch
>                  is a real fault with charge and memcg stat updates,
>                  so it stresses exactly the fast path patch 2 changes.
>   w2-file-churn: file read loop under memory.high pressure
>                  (reclaim-bound, noisier).
>   w3-reparent:   reparent accounting sanity check.
> 
> w1-anon-churn (pages/s):
> 
>                  classic LRU          MGLRU
> base             4393028              4377122
> patches 1-2      4385996   (-0.2%)    4352887   (-0.6%)
> patches 1-3      4381832   (-0.3%)    4377053   (+0.0%)
> 
> w2-file-churn (MB/s):
> 
>                  classic LRU          MGLRU
> base             8277                 8226
> patches 1-2      8226      (-0.6%)    8123      (-1.3%)
> patches 1-3      8157      (-1.4%)    8294      (+0.8%)
> 
> w3-reparent passed on all kernels.
> 
> Patch 2 alone shows a small overhead, most visible under MGLRU (-0.6%
> on w1); patch 3 brings w1 back to the base level in both LRU
> configurations.  The remaining differences are within run-to-run
> noise.
> 
> [1] https://gist.github.com/teawater/32f373ec41d185d840455eb167321a5a

Thanks for running the benchmarks as well. I think we just need to order the
first patch after the second and add CC:stable to the second. We can ask Andrew
but it might be simpler for Andrew to just resend with the correct ordering.