[PATCH v21 0/6] Hierarchical Percpu Counters for RSS

Mathieu Desnoyers posted 6 patches 3 weeks, 3 days ago
There is a newer version of this series
.../core-api/percpu-counter-tree.rst          |  75 ++
include/linux/mm.h                            |  20 +-
include/linux/mm_types.h                      |  65 +-
include/linux/percpu_counter_tree.h           | 387 ++++++++++
include/trace/events/kmem.h                   |   2 +-
init/main.c                                   |   2 +
kernel/fork.c                                 |  24 +-
lib/Kconfig                                   |  12 +
lib/Makefile                                  |   1 +
lib/percpu_counter_tree.c                     | 694 ++++++++++++++++++
lib/tests/Makefile                            |   2 +
lib/tests/percpu_counter_tree_kunit.c         | 399 ++++++++++
12 files changed, 1651 insertions(+), 32 deletions(-)
create mode 100644 Documentation/core-api/percpu-counter-tree.rst
create mode 100644 include/linux/percpu_counter_tree.h
create mode 100644 lib/percpu_counter_tree.c
create mode 100644 lib/tests/percpu_counter_tree_kunit.c
[PATCH v21 0/6] Hierarchical Percpu Counters for RSS
Posted by Mathieu Desnoyers 3 weeks, 3 days ago
Hi Andrew,

Here is the hierarchical percpu counters for RSS accounting series
rebased on top of v7.3-rc1. It includes small bootup fixes which were
needed to fix bootup sequence on specific architectures, and a rename of
the test config option to include "KUNIT_".

This aims at replacing the prior version of the series you had
in mm.

As a reminder, the goal here is to provide more precise RSS counters
through /proc. A future step would be to also use these counters to
speed up the OOM killer task selection when OOM is triggered by
memcg. In whole-machine OOM situations, latency introduced by the
OOM killer task selection may not be too much of an issue, but for
memcg, where cgroups may routinely depend on the OOM killer, the
acceptable latency tradeoff is quite different.

Thanks,

Mathieu

Mathieu Desnoyers (6):
  lib: introduce hierarchical per-cpu counters
  lib: test hierarchical per-cpu counters
  mm: improve RSS counter approximation accuracy for proc interfaces
  mm: reorder mm_struct flexible array to place mm_cpumask first
  init: move percpu_counter_tree_subsystem_init() earlier in boot
  lib: inline percpu_counter_tree_items_size with boot-safety sentinel

Cc: "Paul E. McKenney" <paulmck@kernel.org>
Cc: Steven Rostedt <rostedt@goodmis.org>
Cc: Masami Hiramatsu <mhiramat@kernel.org>
Cc: Dennis Zhou <dennis@kernel.org>
Cc: Tejun Heo <tj@kernel.org>
Cc: Christoph Lameter <cl@linux.com>
Cc: Martin Liu <liumartin@google.com>
Cc: David Rientjes <rientjes@google.com>
Cc: christian.koenig@amd.com
Cc: Shakeel Butt <shakeel.butt@linux.dev>
Cc: SeongJae Park <sj@kernel.org>
Cc: Michal Hocko <mhocko@suse.com>
Cc: Johannes Weiner <hannes@cmpxchg.org>
Cc: Sweet Tea Dorminy <sweettea-kernel@dorminy.me>
Cc: Lorenzo Stoakes <ljs@kernel.org>
Cc: Liam R. Howlett <liam@infradead.org>
Cc: Mike Rapoport <rppt@kernel.org>
Cc: Suren Baghdasaryan <surenb@google.com>
Cc: Vlastimil Babka <vbabka@kernel.org>
Cc: Christian Brauner <brauner@kernel.org>
Cc: Wei Yang <richard.weiyang@gmail.com>
Cc: David Hildenbrand <david@kernel.org>
Cc: Miaohe Lin <linmiaohe@huawei.com>
Cc: Al Viro <viro@zeniv.linux.org.uk>
Cc: Yu Zhao <yuzhao@google.com>
Cc: Roman Gushchin <roman.gushchin@linux.dev>
Cc: Mateusz Guzik <mjguzik@gmail.com>
Cc: Matthew Wilcox <willy@infradead.org>
Cc: Baolin Wang <baolin.wang@linux.alibaba.com>
Cc: Aboorva Devarajan <aboorvad@linux.ibm.com>
Cc: David Carlier <devnexen@gmail.com>
Cc: Josh Law <objecting@objecting.org>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: linux-mm@kvack.org

 .../core-api/percpu-counter-tree.rst          |  75 ++
 include/linux/mm.h                            |  20 +-
 include/linux/mm_types.h                      |  65 +-
 include/linux/percpu_counter_tree.h           | 387 ++++++++++
 include/trace/events/kmem.h                   |   2 +-
 init/main.c                                   |   2 +
 kernel/fork.c                                 |  24 +-
 lib/Kconfig                                   |  12 +
 lib/Makefile                                  |   1 +
 lib/percpu_counter_tree.c                     | 694 ++++++++++++++++++
 lib/tests/Makefile                            |   2 +
 lib/tests/percpu_counter_tree_kunit.c         | 399 ++++++++++
 12 files changed, 1651 insertions(+), 32 deletions(-)
 create mode 100644 Documentation/core-api/percpu-counter-tree.rst
 create mode 100644 include/linux/percpu_counter_tree.h
 create mode 100644 lib/percpu_counter_tree.c
 create mode 100644 lib/tests/percpu_counter_tree_kunit.c

-- 
2.43.0
Re: [PATCH v21 0/6] Hierarchical Percpu Counters for RSS
Posted by Shakeel Butt 3 weeks, 1 day ago
On Tue, Sep 01, 2026 at 02:28:45PM -0400, Mathieu Desnoyers wrote:
> Hi Andrew,
> 
> Here is the hierarchical percpu counters for RSS accounting series
> rebased on top of v7.3-rc1. It includes small bootup fixes which were
> needed to fix bootup sequence on specific architectures, and a rename of
> the test config option to include "KUNIT_".
> 
> This aims at replacing the prior version of the series you had
> in mm.
> 
> As a reminder, the goal here is to provide more precise RSS counters
> through /proc. A future step would be to also use these counters to
> speed up the OOM killer task selection when OOM is triggered by
> memcg. In whole-machine OOM situations, latency introduced by the
> OOM killer task selection may not be too much of an issue, but for
> memcg, where cgroups may routinely depend on the OOM killer, the
> acceptable latency tradeoff is quite different.
> 

Hi Mathieu, can you please comment on the Sashiko's findings [1]?
Worth addressing or not?

[1] https://sashiko.dev/#/patchset/20260901182857.26690-1-mathieu.desnoyers%40efficios.com
Re: [PATCH v21 0/6] Hierarchical Percpu Counters for RSS
Posted by Mathieu Desnoyers 3 weeks, 1 day ago
On 2026-09-03 13:18, Shakeel Butt wrote:
> On Tue, Sep 01, 2026 at 02:28:45PM -0400, Mathieu Desnoyers wrote:
>> Hi Andrew,
>>
>> Here is the hierarchical percpu counters for RSS accounting series
>> rebased on top of v7.3-rc1. It includes small bootup fixes which were
>> needed to fix bootup sequence on specific architectures, and a rename of
>> the test config option to include "KUNIT_".
>>
>> This aims at replacing the prior version of the series you had
>> in mm.
>>
>> As a reminder, the goal here is to provide more precise RSS counters
>> through /proc. A future step would be to also use these counters to
>> speed up the OOM killer task selection when OOM is triggered by
>> memcg. In whole-machine OOM situations, latency introduced by the
>> OOM killer task selection may not be too much of an issue, but for
>> memcg, where cgroups may routinely depend on the OOM killer, the
>> acceptable latency tradeoff is quite different.
>>
> 
> Hi Mathieu, can you please comment on the Sashiko's findings [1]?
> Worth addressing or not?
> 
> [1] https://sashiko.dev/#/patchset/20260901182857.26690-1-mathieu.desnoyers%40efficios.com
> 

I've looked at the report. There are 4 classes of things happening
there. FWIW I've also used Claude to counter-check the report.

Summary: no code change appears to be needed for classes 1 and 2.
I should do test and module teardown fixes, as well as overflow cleanup
for a v22.

* Class 1: the supposed "over vs under" swapped accuracy bounds. In that
case it appears to be the LLM which is confused. This is because the range
of approximation as a function of the precise sum is expressed in the
opposite direction than the range of precise sum as a function of the
approximation:

+ * The range of the approximation as a function of the precise sum is expressed as:
+ *
+ *   approx_sum >= precise_sum - approx_accuracy_range.under
+ *   approx_sum <= precise_sum + approx_accuracy_range.over
+ *
+ * Therefore, the range of the precise sum as a function of the approximation is expressed as:
+ *
+ *   precise_sum <= approx_sum + approx_accuracy_range.under
+ *   precise_sum >= approx_sum - approx_accuracy_range.over

We could possibly extend this documentation elsewhere to make sure that a
LLM with limited context don't get confused, but the code is OK there.

Sashiko wrote that the positive margin "should be over_a + under_b" and then
claimed the code passes under_a + over_b. The code passes a->over + b->under.
Exactly what Sashiko itself derived. Its reasoning agreed with the code; it
misread which argument was which.

* Class 2:

> +void percpu_counter_tree_add(struct percpu_counter_tree *counter, long inc)
> +{
> +    unsigned long bit_mask = counter->level0_bit_mask, orig, res;
> +
> +    res = this_cpu_add_return(*counter->level0, inc);
> +    orig = res - inc;
> +    inc = percpu_counter_tree_carry(orig, res, inc, bit_mask);
> +    if (likely(!inc))
> +        return;
> +    percpu_counter_tree_add_slowpath(counter, inc);
> +}

Sashiko:
Does this code risk an unbounded approximation error if preemption occurs
after this_cpu_add_return() but before percpu_counter_tree_add_slowpath()?
The level 0 counter reflects the addition immediately, but if the task is
preempted before propagating the carry to the tree, that carry is held back
indefinitely. Multiple preempted tasks could cause the approximate sum to
deviate from the precise sum by an amount far exceeding the topological
accuracy bounds. Could we disable preemption across this entire sequence?

My take on this:

I think we need a clearer _documentation_ statement about what the approximation
upper bounds guarantee. It guarantees upper bounds on the approximation error
_when the counters are quiescent_, never when they are actively being concurrently
updated. So if a thread is preempted _while propagating the carry_, it's not
quiescent. I don't think we need any code change there. And disabling preemption
would not "fix" anything there, it would merely make the race smaller.


* Class 3:

Test cases boilerplate, module teardown and cpu hotplug issues: Good catches,
I'll fix them.

* Class 4: Overflows

In order to be strictly C compliant, I should do the difference in unsigned
(where overflow/underflow is OK), and cast the result as signed to compare
with 0 with a inequality. It won't change things in practice, but it's a good
call.

Thoughts ?

Thanks,

Mathieu



-- 
Mathieu Desnoyers
EfficiOS Inc.
https://www.efficios.com