Documentation/admin-guide/sysctl/vm.rst | 38 +++++ include/linux/mm_types.h | 15 ++ include/linux/sched/mm.h | 6 + include/linux/vm_event_item.h | 5 + include/trace/events/mm_reaper.h | 73 +++++++++ kernel/exit.c | 2 +- kernel/fork.c | 199 ++++++++++++++++++++++++ mm/Kconfig | 12 ++ mm/oom_kill.c | 61 ++++++++ mm/vmstat.c | 5 + 10 files changed, 415 insertions(+), 1 deletion(-) create mode 100644 include/trace/events/mm_reaper.h
Address-space teardown on process exit runs synchronously in the dying
task's context: exit_mm() -> mmput() -> __mmput() -> exit_mmap() walks
page tables, updates rmap, frees the RSS and drops file references, all
on the exiting CPU. For a multi-GB process that is hundreds of
milliseconds of exit-path latency, paid by whoever is waiting on the
death: a supervisor's kill-and-respawn cycle, a shell's waitpid(), an
orchestrator reaping a fleet of workers. On the test box below, killing
a 16GB process costs ~280ms before the parent's waitpid() returns.
This series adds CONFIG_ASYNC_MM_TEARDOWN: an opt-in, default-off path
that defers __mmput() of large exiting processes to a dedicated kernel
thread (mm_reaper), so the exiting CPU is released as soon as the task
is reaped and the teardown runs off to the side.
Design
======
- A single freezable kthread, mm_reaper, modeled on oom_reaper, fed
through a lock-free llist + waitqueue. llist rather than oom_reaper's
spinlock+list so many-core parallel exits do not serialize on a
global lock at enqueue. The kthread runs at nice 19; its affinity is
a preference (kthread_affine_preferred()) for the HK_TYPE_KTHREAD
housekeeping mask, so it keeps off isolcpus= and cpuset-isolated
CPUs without the hard bind of kthread_bind_mask() --
PF_NO_SETAFFINITY would leave admins unable to re-affine it.
- mmput_exit(), called only from exit_mm()'s final reference drop
(patch 7). Every other mmput() caller (get_task_mm() users such as
ptrace and /proc, kthread_unuse_mm(), ...) is unchanged and still
tears down inline if it happens to hold the last reference.
- Deferral only when ALL of the following hold, else inline __mmput():
* vm.async_mm_teardown=1 (static key, default off)
* RSS >= vm.async_mm_teardown_thresh_pages (default 64MB)
* the mm is not an OOM-kill target (MMF_OOM_TARGETED, patch 3) and
not already reaped (MMF_OOM_SKIP)
* charging the RSS to a pending-pages budget stays under
vm.async_mm_teardown_max_pending_pages (add-then-check: can
over-reject under concurrency, never over-admit)
- exit_aio() is pulled forward: mmput_exit() runs it eagerly on the
exiting task before reserving/queuing. exit_aio() can block
indefinitely waiting for in-flight AIO to drain; run eagerly, a
stuck AIO context stalls only that task -- exactly what a
synchronous mmput() would do -- instead of blocking the single
reaper and stranding every teardown queued behind it. It is
idempotent (a no-op once ioctx_table is gone), so __mmput() calling
it again later is harmless.
- OOM exclusion: an OOM victim's memory must come back promptly, not
sit behind a nice-19 kthread working against the oom_reaper.
MMF_OOM_TARGETED is mm-scoped (not per-task) because the final
mm_users drop can come from a CLONE_VM sharer in another thread
group that mark_oom_victim() never sees; it is set before any
SIGKILL for the kill event is dispatched, and the marking and the
eligibility check cannot overlap in time (see patch 3 for the
ordering argument). That argument is belt-and-suspenders, not a
correctness dependency: a missed flag would only defer the
victim's teardown (a latency/priority-inversion regression), never
corrupt anything, and the oom_reaper reaps the victim
independently either way. An LKMM litmus test for the
CLONE_VM-sharer case is available on request.
- No extra mm refcounting: deferring __mmput() keeps the mm_count
reference that mm_users > 0 stands for, exactly as mmput_async()
does; the mm and its embedded llist node stay alive until the
kthread runs.
- Observability: three vmstat counters (queued / sync / rejected) and
two tracepoints. queue carries pid/comm/rss/node of the exiting
task; reap carries the charged RSS vs a fresh RSS read plus the
reaper's node, so the gap measures how much reclaim ate out of the
queue, and queue-vs-reap node mismatches expose NUMA-remote
teardown.
The work is moved, not eliminated. Measured checks (same box
as below):
- Freeing latency: with async on, a killed 16GB region is freed on
the same timescale as inline teardown -- nr_free_pages recovers
within ~300ms of the kill in both configurations (single-rep
watcher readings of 240ms async vs 283ms inline, within run-to-run
variance; not a claim that async frees faster). Deferral does not
delay the return of memory; it makes the exit path stop waiting
for it.
- CPU conservation: 5 consecutive 4GB inline teardowns cost 372ms of
exit-path CPU; mm_reaper spends 440ms of kthread CPU draining the
same five, read from its schedstat over a fully drained window
(ratio 1.18, nice-19 kthread, cache-cold on another CPU).
- Pages queued for teardown are invisible to reclaim until the reaper
frees them. The pending cap bounds that exposure coarsely; it is not
a reclaim mechanism. Reclaim integration (unmap-first draining, a
shrinker) is future work, and is also an answer to the cap
sizing question below.
Scope of the claims: what this series claims today is the common-case
exit-latency win under a modest threshold and a conservative cap. It
deliberately does not claim the multi-GB-teardown-under-memory-pressure
regime: until the queue is visible to reclaim, that regime is exactly
where the cap must stay conservative, and the 16GB+ numbers below are
capability demonstrations, run with the cap deliberately raised on an
otherwise idle machine -- not a recommended production configuration.
Open question 1 is where this either gets fixed (reclaim integration)
or stays scoped.
Numbers
=======
Bare metal, 32 CPUs / 32GB RAM (31 GiB usable), single NUMA node,
performance governor, medians, THP off unless noted. reap = SIGKILL -> waitpid()
returns. For all async-on runs vm.async_mm_teardown_max_pending_pages
was raised to 3/4 of RAM so the measurements exercise the reaper, not
the backpressure fallback: with the RAM/4 default, a single 16GB mm on
this box (8GB cap) -- or the 64GB guest case -- exceeds the cap and
falls back to sync teardown by design. See open question 1.
kill, async off -> on:
1GB 18.01 ms -> 0.14 ms
4GB 73.71 ms -> 0.14 ms
16GB 282.16 ms -> 0.15 ms
16GB 27.68 ms -> 0.13 ms (THP on)
voluntary exit (_exit() -> reap), 16GB:
271.27 ms -> 0.12 ms
M simultaneous 0.5GB exits, tail until ALL reaped:
M=8 40.64 ms -> 0.15 ms
M=16 79.99 ms -> 0.17 ms
M=32 157.42 ms -> 0.32 ms
below-threshold 30MB process, feature ON vs OFF:
0.56 ms -> 0.60 ms (sync path, within noise)
file-backed (tmpfs) 4GB mapping, kill:
35.34 ms -> 0.12 ms
real workload -- redis-server with a 16GB populated dataset
(DEBUG POPULATE, VmRSS ~16.2GB), kill:
292.97 ms -> 0.24 ms
The redis point (5 reps/pass, sync spread 288.5-309.2ms, async
0.12-0.32ms) lands within ~3% of the synthetic 16GB per-GB rate, so
the benchmark's memset region is a fair proxy for a real heap, and
the async side carries over unchanged.
CONFIG=n control: the same driver on a =n build of the same tree
(interface verified fully absent: no sysctls, no vmstat counters, no
kthread, no tracepoints) reproduces the =y async-off column within
noise -- 18.01 vs 18.01 ms at 1GB kill, 282.14 vs 282.16 ms at 16GB
kill -- so with the key off, the config costs nothing measurable.
Single-reaper drain throughput: 16.4GB backlog (32 x 0.5GB) returned in
650ms, ~25GB/s on this box. For scale, a 32-vCPU/88GB QEMU guest,
pinned to one NUMA node of its host (vCPUs and preallocated guest RAM
both bound node-local; guest numbers, same kernel/config): 16GB kill
426.15ms -> 0.13ms, a 64GB backlog (32 x 2GB) drained in ~2.1s
(~31GB/s), a 16GB shmem inode evicted in reaper context, and the
pm_test freezer cycle passed with the full 64GB backlog queued.
Robustness testing (driver script, all sections pass, dmesg clean)
==================================================================
- Backpressure: with the cap forced to 1GB, 8 parallel 0.5GB exits
produce queued += 4, sync fallbacks += 12; the cap is respected and
fallback is graceful.
- OOM killer firing while 8GB of teardowns are queued: 3/3 reps clean,
backlog drains in ~650ms alongside the kill.
- Targeted MMF_OOM_TARGETED probes, verified per-pid against the
tracepoints (a positive control first proves a plain large exit of
the same binary IS seen as queued): a ~512MB memcg OOM victim is
never queued; a CLONE_VM pair sharing one mm, swept by the same
kill, produces no queue event from either thread group. Two claims
are documented as untestable from userspace rather than tested:
task_will_free_mem() fast-path marking, and flag stickiness/fork
non-inheritance (no userspace process can survive owning a targeted
mm: oom_score_adj writes propagate across mm sharers, and the
__oom_kill_process() sweep kills every sharer unconditionally).
- unlink-while-mapped tmpfs file: the VMA's fput at teardown is the
last reference, so the final iput -> shmem_evict_inode of a 4GB
shmem inode runs in reaper context; tmpfs is empty 212ms after the
reap. Note: a kthread's fput cannot use task_work, so the evict
actually executes in the (bound) delayed_fput workqueue -- see open
question 4.
- Toggling: disabling mid-drain still drains the backlog (8.0GB in
670ms) while new exits take the sync path; 50 rapid static-key
flip cycles under concurrent exits, clean.
- pm_test freezer cycle with a 16GB backlog queued: completes, and
the async path works after thaw. Caveat in open question 6.
NOT tested: NUMA. I have no bare-metal access to a multi-node machine;
all bare-metal numbers above are single-node, and the QEMU guests
present a single node too. The driver has a ready-to-run NUMA section
(reaper pinned to node 0, victim on node 0 vs node 1, kthread_tail
compared, tracepoints capture queue-vs-reap node) that skips on
single-node systems. Numbers from a multi-node box would be very
welcome and directly feed open question 2.
Open questions, input wanted
============================
1. The default for vm.async_mm_teardown_max_pending_pages. It is
currently totalram_pages()/4, an explicitly marked placeholder.
A static fraction of RAM is arbitrary; but making it a dynamic
function of available RAM is wrong in a different way: high
occupancy is not load. A box full of page cache or running a
legitimately large in-memory workload is not necessarily under
pressure, and those are exactly the boxes that want async teardown.
Under real pressure the cap is the wrong tool anyway, because the
queued pages are invisible to reclaim regardless of the cap value;
the correct fix there is reclaim integration (above).
The default also cuts against the feature: RAM/4 rejects any
single mm larger than a quarter of RAM from deferral entirely --
the exact processes with the worst exit latency are the first ones
sent back to the sync path. Every async number above needed the
cap raised to 3/4 of RAM to be measurable at all.
And the loose end of the range bites in practice: in a large
guest with the cap at that same 3/4 of RAM, a stress run that
started a new round of large exits before the previous round had
drained ran the box into OOM kills. That is what the design
predicts: queued mms are invisible to reclaim and unreachable by
the OOM killer -- their tasks are already dead, so they appear in
no task dump and the oom_reaper has no task to reap; only
mm_reaper can free them. The kills do resolve the pressure (OOM
victims are MMF_OOM_TARGETED, so their teardown is sync), but
tasks die on a box that is not really out of memory -- the memory
is in the queue. With the RAM/4 default the same pattern
self-limits: reservation fails and exits fall back to sync
teardown, which is its own backpressure.
Note the motivating workload (kill-and-respawn) is exactly the
pattern that re-faults immediately after reap, so transient demand
approaches old+new RSS; the cap bounds the old half. So: is RAM/4
acceptable as a coarse safety bound for now? Should the cap react
to watermarks or PSI instead? Should out_of_memory() drain or
expedite the pending queue before picking a victim, the way it
already coordinates with the oom_reaper? That last one is
attractive because every queued mm is already dead: draining is
guaranteed-progress reclaim with no victim to choose. But an
unbounded synchronous drain inside the allocation path is its own
stall (seconds for a large backlog), so it would need a budget.
Or stay dumb until the shrinker exists?
The shape I have in mind for the real fix: a shrinker whose
count_objects() reports the pending pages and whose scan path
kicks the reaper and switches it to unmap-first draining, so
memory returns ahead of the slow metadata teardown. Running full
__mmput()s inside direct reclaim would trade one latency cliff for
another, so the shrinker's job is to accelerate and prioritize the
drain, not to execute it in the caller's context. If the consensus
is that reclaim visibility must land before exit_mm() is routed
(patch 7), I would make that the centerpiece of v2.
2. One reaper vs per-node reapers vs an unbound workqueue. One thread
sustains ~25GB/s on this box and a 64GB backlog still drains in
~2.1s in the big guest, but all of that is node-local. The tracepoint
node fields exist precisely to evaluate this once multi-node
numbers exist.
3. CPU accounting. Teardown CPU moves from the exiting task (charged
to its cgroup) to a root-cgroup kthread. That is the same class of
escape as kswapd or the oom_reaper, but this extends it to routine
process exit. Patch 5's admin docs state the escape explicitly so
nobody enables this blind; the open question is whether that
suffices for an opt-in feature or whether cgroup-aware charging of
the reaper's CPU is a prerequisite.
4. The delayed_fput hop. Because mm_reaper is a kthread, the final
fput() it drops cannot use task_work and is punted to the bound
delayed_fput workqueue -- so a multi-GB shmem inode evict runs on a
bound system_wq worker (the workqueue watchdog already complains:
"delayed_fput hogged CPU for >13333us, consider switching to
WQ_UNBOUND"). Should the reaper drain its own fput backlog inline,
or is this an argument for making delayed_fput unbound generally?
5. memcg zombies. Deferring teardown also defers the final uncharge,
so a dying memcg can be pinned a little longer by a queued mm
(bounded by the pending cap and measured drain times, i.e.
normally milliseconds). I believe this is acceptable.
6. Freezing. The drain loop calls try_to_freeze() after every entry,
so a freeze request is honoured between mms mid-batch, and the
pm_test freezer cycle passes with a 16GB backlog queued. What
remains is that a single in-flight __mmput() cannot freeze
mid-teardown, so one sufficiently large mm could still exceed the
freeze timeout -- but that is equally true of the same exit_mmap()
running synchronously in the exiting task today. Is per-mm
granularity enough?
Related work
============
- "Two alternatives for mm async teardown" (Claudio Imbrenda, 2021,
https://lore.kernel.org/all/20211111095008.264412-1-imbrenda@linux.ibm.com/
): an earlier RFC with the same
goal, motivated by huge s390 KVM guest address spaces. It offered
either an arch hook in exit_mm() or a process_mmput_async syscall
that runs the teardown in another process's context, plus an OOM
notifier to shield the teardown from the OOM killer. Neither
variant was merged; the s390 protected-guest case was later
addressed with KVM-specific asynchronous destruction. This series
is arch-independent, needs no syscall or userspace agent (kernel
kthread plus an eligibility gate), and integrates with the OOM
killer by exclusion (MMF_OOM_TARGETED keeps victims synchronous)
rather than by blocking it.
- oom_reaper: same kthread pattern, but it strips memory from a
victim that cannot exit; this series is about not making healthy
exits pay the teardown latency.
- process_mrelease(2): userspace-driven reaping of a killed process.
It needs a daemon to issue it per-kill, only helps the kill case
(not voluntary exit), and does not remove the exit-path work from
the dying task -- it races it. This series needs no userspace
agent and covers both exit flavors.
- mmput_async(): existing deferral of __mmput() to a workqueue for
contexts that cannot sleep; this series reuses its lifetime
argument but adds gating, backpressure, OOM exclusion and a
dedicated low-priority thread, none of which mmput_async() has.
Patches
=======
1 mm: add CONFIG_ASYNC_MM_TEARDOWN scaffolding for off-CPU exit
teardown -- config, mm_struct::async_reap_node, mmput_exit()
fallback; no functional change.
2 mm: dispatch __mmput() to an mm_reaper kthread via llist -- the
kthread and queue; built but unreferenced.
3 mm/oom_kill: mark an OOM victim's mm with MMF_OOM_TARGETED -- the
mm-scoped victim flag and its visibility/ordering argument;
nothing reads it yet.
4 mm: gate async teardown on RSS with pending-pages backpressure --
eligibility predicate + budget reservation; charge memoized at
enqueue because RSS keeps shrinking after mm_users hits zero;
exit_aio() pulled forward onto the exiting task.
5 mm: add runtime toggle and sysctls for async teardown -- static
key, three vm.* sysctls, admin-guide docs.
6 mm: add tracepoints and vmstat counters for async teardown.
7 exit: route exit_mm()'s final mmput() through mmput_exit() -- the
switch that makes 1-6 operative.
Patches 1-6 are individually inert (each is either scaffolding or
dead code until patch 7), so the series bisects cleanly with the
feature dark until the last commit, and the runtime default is off
even then.
The benchmark (exit_latency.c) and the driver that produced every
number above (sanity/headline/wait-free/CPU-conservation/reuse/
parallel/backpressure/numa/smallproc/oom/oom-targeted/file-backed/
toggle/freezer sections, ~2k lines total), plus the standalone redis
driver behind the real-workload number, are available on request; if
there is interest I am happy to rework them into
tools/testing/selftests/mm for a later revision.
Based on mm-unstable, commit 890f8c4e827c
("mm/secretmem: don't allow highmem folios").
Aditya Sharma (7):
mm: add CONFIG_ASYNC_MM_TEARDOWN scaffolding for off-CPU exit teardown
mm: dispatch __mmput() to an mm_reaper kthread via llist
mm/oom_kill: mark an OOM victim's mm with MMF_OOM_TARGETED
mm: gate async teardown on RSS with pending-pages backpressure
mm: add runtime toggle and sysctls for async teardown
mm: add tracepoints and vmstat counters for async teardown
exit: route exit_mm()'s final mmput() through mmput_exit()
Documentation/admin-guide/sysctl/vm.rst | 38 +++++
include/linux/mm_types.h | 15 ++
include/linux/sched/mm.h | 6 +
include/linux/vm_event_item.h | 5 +
include/trace/events/mm_reaper.h | 73 +++++++++
kernel/exit.c | 2 +-
kernel/fork.c | 199 ++++++++++++++++++++++++
mm/Kconfig | 12 ++
mm/oom_kill.c | 61 ++++++++
mm/vmstat.c | 5 +
10 files changed, 415 insertions(+), 1 deletion(-)
create mode 100644 include/trace/events/mm_reaper.h
base-commit: 890f8c4e827c918dac668a12eaf63180ba8a9e6d
--
2.34.1
On 7/19/26 19:54, Aditya Sharma wrote: > Address-space teardown on process exit runs synchronously in the dying > task's context: exit_mm() -> mmput() -> __mmput() -> exit_mmap() walks > page tables, updates rmap, frees the RSS and drops file references, all > on the exiting CPU. For a multi-GB process that is hundreds of > milliseconds of exit-path latency, paid by whoever is waiting on the > death: a supervisor's kill-and-respawn cycle, a shell's waitpid(), an > orchestrator reaping a fleet of workers. On the test box below, killing > a 16GB process costs ~280ms before the parent's waitpid() returns. > > This series adds CONFIG_ASYNC_MM_TEARDOWN: an opt-in, default-off path > that defers __mmput() of large exiting processes to a dedicated kernel > thread (mm_reaper), so the exiting CPU is released as soon as the task > is reaped and the teardown runs off to the side. QEMU has an rather short implementation for async teardown using clone(CLONE_VM), which is IIRC essentially the result of Claudios previous kernel work you note below. So nothing got merged because the problem was solvable in userspace. Without something like CONFIG_ASYNC_MM_TEARDOWN in the kernel. [1] https://github.com/qemu/qemu/blob/master/system/async-teardown.c -- Cheers, David
Sent using Zoho Mail From: David Hildenbrand (Arm) <david@kernel.org> To: "Aditya Sharma"<adi.sharma@zohomail.in>, "Andrew Morton"<akpm@linux-foundation.org>, "Lorenzo Stoakes"<ljs@kernel.org>, "Liam R . Howlett"<liam@infradead.org>, "Vlastimil Babka"<vbabka@kernel.org>, "Mike Rapoport"<rppt@kernel.org>, "Suren Baghdasaryan"<surenb@google.com>, "Michal Hocko"<mhocko@suse.com> Cc: "David Rientjes"<rientjes@google.com>, "Shakeel Butt"<shakeel.butt@linux.dev>, "Jonathan Corbet"<corbet@lwn.net>, "Shuah Khan"<skhan@linuxfoundation.org>, "Steven Rostedt"<rostedt@goodmis.org>, "Masami Hiramatsu"<mhiramat@kernel.org>, "Mathieu Desnoyers"<mathieu.desnoyers@efficios.com>, "Kees Cook"<kees@kernel.org>, "Ingo Molnar"<mingo@redhat.com>, "Peter Zijlstra"<peterz@infradead.org>, "Juri Lelli"<juri.lelli@redhat.com>, "Vincent Guittot"<vincent.guittot@linaro.org>, "Dietmar Eggemann"<dietmar.eggemann@arm.com>, "Ben Segall"<bsegall@google.com>, "Mel Gorman"<mgorman@suse.de>, "Valentin Schneider"<vschneid@redhat.com>, "K Prateek Nayak"<kprateek.nayak@amd.com>, <linux-mm@kvack.org>, <linux-doc@vger.kernel.org>, <linux-trace-kernel@vger.kernel.org>, <linux-kernel@vger.kernel.org>, <imbrenda@linux.ibm.com> Date: Mon, 20 Jul 2026 13:43:34 +0530 Subject: Re: [RFC PATCH 0/7] mm: defer address-space teardown of large exiting processes to a kthread > QEMU has an rather short implementation for async teardown using > clone(CLONE_VM), which is IIRC essentially the result of Claudios previous > kernel work you note below. > > So nothing got merged because the problem was solvable in userspace. Thanks for this. My related works paragraph is wrong on this then. It says that the s390 case was addressed with KVM specific async destruction, but what you point to is QEMU's shadow process. I will correct that in v2. Yeah, that CLONE_VM trick works, so I would measure and compare them and get back. I plan to measure: - Memory return latency - N concurrent shadow teardowns on N cpus vs the reaper. One difference between CLONE_VM approach and CONFIG_ASYNC_MM_TEARDOWN is that the shadow process has to be created by the process that would be torn down itself. If that same behaviour is wanted on a workload where we dont own the source, CONFIG_ASYNC_MM_TEARDOWN enables it here as well. Also, in the QEMU case, it also means that the deferred teardown decision was made without knowing how large the process might become. CONFIG_ASYNC_MM_TEARDOWN adds a gate where for the exiting process, RSS is evaluated againsts the current backlog before queuing it for async teardown. I will drop a follow up with numbers Aditya
[...] > says that the s390 case was addressed with KVM specific async destruction, > but what you point to is QEMU's shadow process. I will correct that in v2. CLONE_VM seems to work for the use cases you had in mind. There would have to be a *pretty* convincing story why we would want more complexity in the kernel to handle this. Some smaller performance improvements are not worth it. -- Cheers, David
> CLONE_VM seems to work for the use cases you had in mind. > > There would have to be a *pretty* convincing story why we would want more > complexity in the kernel to handle this. > > Some smaller performance improvements are not worth it. Having thought about it more, I agree. I have a workload of my own, which as you say, I can simply fix using the CLONE_VM way. The only kind of user which the shadow process thing does not cover is someone who does not have control over the source of task over which they want such a behaviour. Here, the only case I can think of is operators who cannot modify or wrap the processes they're managing. I dont have that user. So, instead of further argument or sending benchmarks that would show a performance delta, does anyone on the list have such a use case? If not, I would consider my questions as answered. Thanks Aditya
On 26/07/20 12:24AM, Aditya Sharma wrote: > Address-space teardown on process exit runs synchronously in the dying > task's context: exit_mm() -> mmput() -> __mmput() -> exit_mmap() walks > page tables, updates rmap, frees the RSS and drops file references, all > on the exiting CPU. For a multi-GB process that is hundreds of > milliseconds of exit-path latency, paid by whoever is waiting on the > death: a supervisor's kill-and-respawn cycle, a shell's waitpid(), an > orchestrator reaping a fleet of workers. On the test box below, killing > a 16GB process costs ~280ms before the parent's waitpid() returns. > > This series adds CONFIG_ASYNC_MM_TEARDOWN: an opt-in, default-off path > that defers __mmput() of large exiting processes to a dedicated kernel > thread (mm_reaper), so the exiting CPU is released as soon as the task > is reaped and the teardown runs off to the side. This reads like it was written by an AI, but there is no assisted-by tags. Which AI did you use to generate these patches and/or cover letter? I do not see a benefit to running the task cleanup 'off the side'. In modern computers, that may mean switching core types that run at different speeds. Regardless of speed, the CPU that will be doing the work is remote to the workload by design, (or may be, depending on the scheduling?). That is, you are tasking another CPU to do cleanup of local CPU-aware information.. The ideal CPU to do a task exit is the one that's doing it already. There are hidden costs to your approach and I don't really understand how this could possibly speed things up - even parallelized tasks will take more CPU time in total, even if locking issues are avoided. Sure, the task waiting for cleanup to complete may think it's done and continue to do the Next Thing - but if it really needs the cleanup to be completed then you've just made it impossible to know when it has happened. Also, you've just made this extremely hard to debug if something is missed. The work that is done in teardown is necessary. Your time (and tokens?) would be better spent trying to improve what we have to do instead of shifting the work around. Thanks, Liam
Sent using Zoho Mail From: Liam R . Howlett <liam@infradead.org> To: "Aditya Sharma"<adi.sharma@zohomail.in> Cc: "Andrew Morton"<akpm@linux-foundation.org>, "David Hildenbrand"<david@kernel.org>, "Lorenzo Stoakes"<ljs@kernel.org>, "Vlastimil Babka"<vbabka@kernel.org>, "Mike Rapoport"<rppt@kernel.org>, "Suren Baghdasaryan"<surenb@google.com>, "Michal Hocko"<mhocko@suse.com>, "David Rientjes"<rientjes@google.com>, "Shakeel Butt"<shakeel.butt@linux.dev>, "Jonathan Corbet"<corbet@lwn.net>, "Shuah Khan"<skhan@linuxfoundation.org>, "Steven Rostedt"<rostedt@goodmis.org>, "Masami Hiramatsu"<mhiramat@kernel.org>, "Mathieu Desnoyers"<mathieu.desnoyers@efficios.com>, "Kees Cook"<kees@kernel.org>, "Ingo Molnar"<mingo@redhat.com>, "Peter Zijlstra"<peterz@infradead.org>, "Juri Lelli"<juri.lelli@redhat.com>, "Vincent Guittot"<vincent.guittot@linaro.org>, "Dietmar Eggemann"<dietmar.eggemann@arm.com>, "Ben Segall"<bsegall@google.com>, "Mel Gorman"<mgorman@suse.de>, "Valentin Schneider"<vschneid@redhat.com>, "K Prateek Nayak"<kprateek.nayak@amd.com>, <linux-mm@kvack.org>, <linux-doc@vger.kernel.org>, <linux-trace-kernel@vger.kernel.org>, <linux-kernel@vger.kernel.org>, <imbrenda@linux.ibm.com> Date: Mon, 20 Jul 2026 03:58:47 +0530 Subject: Re: [RFC PATCH 0/7] mm: defer address-space teardown of large exiting processes to a kthread > This reads like it was written by an AI, but there is no assisted-by > tags. Which AI did you use to generate these patches and/or cover > letter? I used claude opus 4.8 to refine commit messages (the design is mine, the model was used to convey the intentions concretely). Further, I used claude fable 5 for drafting the cover letter and reviewing the series, where it flagged issues (I fixed some of those by hand and some were fixed by fable itself, which I then reviewed). It also assisted in writing the benchmark harness behind the numbers and did the litmus test the cover letter mentions. Every numbers was measured by me, on my hardware. I take full responsibily of the entire series. My bad that I did not add an assisted-by tag. Should have added it to v1 v2 will add it to the patches > I do not see a benefit to running the task cleanup 'off the side'. In > modern computers, that may mean switching core types that run at > different speeds. That's why the reaper's affinity is a preference, not a bind (patch 2). It uses kthread_affine_preferred() with housekeeping mask, so it can be re-affined. kthread_bind_mask() would have taken that away. On a hybrid cpu, it can be affined(by the (super)user) to whichever cores are appropriate. I do not have numbers for P core and E core teardown costs. Will measure and send them. > Regardless of speed, the CPU that will be doing the work is remote to > the workload by design, (or may be, depending on the scheduling?). That > is, you are tasking another CPU to do cleanup of local CPU-aware > information.. The ideal CPU to do a task exit is the one that's doing it > already. Yes, it does cost something. Five consecutive 4GB inline teardowns in my testing costed 372ms in exit path, while 440 ms in the case of kthread drain for the same (five teardowns on another cpu). This cost became 1.18x. For multi gb mms, the memory to be reaped would be much larger than the cpu cache can hold. For small mms (< 64 MB, configurable by sysctl), the teardowns happen inline. The oom_reaper already reaps the victim's address space from a remote context, as returning memory matters more than locality of teardown. > There are hidden costs to your approach and I don't really understand > how this could possibly speed things up - even parallelized tasks will > take more CPU time in total, even if locking issues are avoided. This is not meant to speed up teardown. Rather the latency for someone waiting on death is reduced. Killing a 16 GB redis cost 292.97ms before returning to waitpid(). It costs 0.24 with async teardown. The memory itself was returned in about 300ms. "parallelized" - This series does not parallelize the teardown. The design is serial. N procesess exiting together today would tear down concurrently today. The reaper here, uses one thread (nice 19, affined to housekeeping mask). The numbers in cover letter support that this work is memory bound rather than CPU bound. In sync case: teardown is around 56 GB/s on my machine (1GB in 18.01 ms and 16GB in 282.16ms). Thirty two concurrent 0.5GB exits reap in 157.42ms (around 101 GB/s). So, a concurrency of 32 gets about 1.8x. The single reaper thread takes 650ms to clear the same (about 25 GB/s). Thus, this serialization took 4x the drain time, but takes peak concurrency down to one cpu. Memory actully comes back later. Whether that's a good tradeoff, depends on what else we wish to run on the box (open questions #1 in cover letter) > Sure, the task waiting for cleanup to complete may think it's done and > continue to do the Next Thing - but if it really needs the cleanup to be > completed It is off by default and opt in (sysctl), so anything that depends on the current semantics, continues to work exactly the same by doing nothing. Also, mmput_async() already defers the teardown to a workqueue. So, in the current kernel, there already exists a way where continuing to the next thing does not necessarily mean that the memory is reclaimed. > completed then you've just made it impossible to know when it has > happened. There are three vmstat counters and two tracepoint (patch 6). Someone who opts in to the mechanism, and does need to make sure that the cleanup is completed, can use the counters and tracepoints. > Also, you've just made this extremely hard to debug if something is > missed. There are reap tracepoints that carry the pid, so we can get per pid completion status. However, there is no number indicating the current depth of the queue. The series addds queued, sync and rejected counters. Will add a reaped counter in v2 so that current queue depth can be estimated. Please mention what else tracability can be added to this, and what senairo do you have in mind that these counters and tracepoints might not cover. I can add those to v2 as well. > The work that is done in teardown is necessary. Your time (and tokens?) > would be better spent trying to improve what we have to do instead of > shifting the work around. This does not compete with with current teardown path. it goes through the same exit_mm(). Any future improvements there, also improve the reaper's drain times. Thanks Aditya
© 2016 - 2026 Red Hat, Inc.