[PATCH 0/2] x86/mce: Rate-limit storm notices and add a corrected error ceiling

Breno Leitao posted 2 patches 1 month, 1 week ago
Documentation/ABI/testing/sysfs-mce             | 12 ++++++++
Documentation/admin-guide/kernel-parameters.txt | 10 +++++++
arch/x86/kernel/cpu/mce/core.c                  | 14 +++++++--
arch/x86/kernel/cpu/mce/internal.h              |  6 ++++
arch/x86/kernel/cpu/mce/threshold.c             | 39 +++++++++++++++++++++++--
5 files changed, 77 insertions(+), 4 deletions(-)
[PATCH 0/2] x86/mce: Rate-limit storm notices and add a corrected error ceiling
Posted by Breno Leitao 1 month, 1 week ago
Some hosts on the Meta fleet have machine check banks that report
TOO MANY corrected errors, faster than the kernel drains them.

For instance, one host flapped a single bank in and out of storm mode
nearly ten million times over four days. The storm detected and subsided
lines were 82% of everything it wrote to the kernel log, crowding out
whatever would explain the failure.

Patch 1 rate-limits the notices.

Patch 2 adds mce=panic_on_ce_count, off by default. Such a host is not
one to keep in service; a clean panic is the better outcome.

Signed-off-by: Breno Leitao <leitao@debian.org>
---
Breno Leitao (2):
      x86/mce: Rate-limit the CMCI storm transition notices
      x86/mce: Add mce=panic_on_ce_count to panic on a corrected error flood

 Documentation/ABI/testing/sysfs-mce             | 12 ++++++++
 Documentation/admin-guide/kernel-parameters.txt | 10 +++++++
 arch/x86/kernel/cpu/mce/core.c                  | 14 +++++++--
 arch/x86/kernel/cpu/mce/internal.h              |  6 ++++
 arch/x86/kernel/cpu/mce/threshold.c             | 39 +++++++++++++++++++++++--
 5 files changed, 77 insertions(+), 4 deletions(-)
---
base-commit: 7079a12d7506b07fb53b54a664bfad5fa9b16d70
change-id: 20260820-mce-panic-on-storm-78cd09a4dcbe

Best regards,
--  
Breno Leitao <leitao@debian.org>