Jakub Kicinski reported that after the commit d929525c2e30 ("memcg:
net: track network throttling due to memcg memory pressure"), in
Meta fleet, the network intensive workloads started seeing extensive
spinlock contention in the rx path in the function
cgroup_file_notify().
cgroup_file_notify() uses a single global spinlock to protect the
cfile->kn pointer across all cgroups. On systems with many cgroups
under memory pressure, this becomes a bottleneck as multiple CPUs in
reclaim call __memcg_memory_event(), which walks the cgroup hierarchy
calling cgroup_file_notify() at each level -- all serialized on the
same lock.
This series reduces the lock hold time by moving kernfs_notify()
outside the critical section, adds lockless fast-path checks to avoid
the lock entirely in common cases (torn-down files and rate-limited
bursts), and replaces the global lock with a per-cgroup_file lock to
eliminate cross-cgroup contention.
Changes since v1:
- Second patch moves rate-limiting code out of lock
Shakeel Butt (3):
cgroup: reduce cgroup_file_kn_lock hold time in cgroup_file_notify()
cgroup: add lockless fast-path checks to cgroup_file_notify()
cgroup: replace global cgroup_file_kn_lock with per-cgroup_file lock
include/linux/cgroup-defs.h | 1 +
kernel/cgroup/cgroup.c | 53 ++++++++++++++++++++-----------------
2 files changed, 29 insertions(+), 25 deletions(-)
--
2.52.0