This is a followup to the hotfix 038a102535eb ("mm/page_alloc: prevent
pcp corruption with SMP=n"), to simplify the code and deal with the
original issue properly. The previous RFC attempt [1] argued for
changing the UP spinlock implementation, which was discouraged, but
thanks to David's off-list suggestion, we can achieve the goal without
changing the spinlock implementation.
The main change in Patch 1 relies on the fact that on UP we don't need
the pcp lists for scalability, so just make them always bypassed during
alloc/free by making the pcp trylock an unconditional failure.
The various drain paths that use pcp_spin_lock_maybe_irqsave() continue
to exist but will never do any work in practice. In Patch 2 we can again
remove the irq saving from them that commit 038a102535eb added.
Besides simpler code with all the ugly UP_flags removed, we get less
bloat with CONFIG_SMP=n for mm/page_alloc.o as a result:
add/remove: 25/28 grow/shrink: 4/5 up/down: 2105/-6665 (-4560)
Function old new delta
get_page_from_freelist 5689 7248 +1559
free_unref_folios 2006 2324 +318
make_alloc_exact 270 286 +16
__zone_watermark_ok 306 322 +16
drain_pages_zone.isra 119 109 -10
decay_pcp_high 181 149 -32
setup_pcp_cacheinfo 193 147 -46
__free_frozen_pages 1339 1089 -250
alloc_pages_bulk_noprof 1054 419 -635
free_frozen_page_commit 907 - -907
try_to_claim_block 1975 - -1975
__rmqueue_pcplist 2614 - -2614
Total: Before=54624, After=50064, chg -8.35%
[1] https://lore.kernel.org/all/d762c46b-36f0-471a-b5b4-23c8cf5628ae@suse.cz/
Signed-off-by: Vlastimil Babka (SUSE) <vbabka@kernel.org>
---
Vlastimil Babka (3):
mm/page_alloc: effectively disable pcp with CONFIG_SMP=n
mm/page_alloc: remove IRQ saving/restoring from pcp locking
mm/page_alloc: remove pcpu_spin_* wrappers
mm/page_alloc.c | 146 ++++++++++++++++++++------------------------------------
1 file changed, 51 insertions(+), 95 deletions(-)
---
base-commit: 8982358e1c87e3e1dc0aad37f4f93efe9c1cfe03
change-id: 20260227-b4-pcp-locking-cleanup-b7a2d5ff2ead
Best regards,
--
Vlastimil Babka (SUSE) <vbabka@kernel.org>