From nobody Fri Dec 19 14:34:36 2025 Received: from out30-130.freemail.mail.aliyun.com (out30-130.freemail.mail.aliyun.com [115.124.30.130]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id EFBC1242D72 for ; Thu, 6 Nov 2025 02:30:42 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=115.124.30.130 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1762396245; cv=none; b=UfubmrRllYpowGClUiaFDRxWZjoUSoYZ1xyodHom24nkW+t8ALExFxSPaJLR7wVErnrDFJ22BC6g9hynaTz+8WMvo4Alt0lwonw7aNxRqtpp1J/Vk2ArNfvApSBDKGuYLNNNqRmquHzFkw+soo3P/lRvZ4H4aOJnvC8/uY73EE4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1762396245; c=relaxed/simple; bh=88Y4Fzkuyb44cTtdgkZImDMFHK3mk57+XzrGQaNYdeg=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=ZMPJRPsh30Pgr9SPwApaGhuHcJrJf6/L6vgicPCVcHi6JYNHabSFGInoJsb92s9TDycCg+b3rg9C31WN12+3Y0Ru/8r/Xk1AghitENp0mriWNMfY47URn2+y+28Bn/aKmTa+w9nnyCRtPxgOXahtImfGKgFic0gfcemY7Px32J0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.alibaba.com; spf=pass smtp.mailfrom=linux.alibaba.com; dkim=pass (1024-bit key) header.d=linux.alibaba.com header.i=@linux.alibaba.com header.b=s4khbDwW; arc=none smtp.client-ip=115.124.30.130 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.alibaba.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.alibaba.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.alibaba.com header.i=@linux.alibaba.com header.b="s4khbDwW" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.alibaba.com; s=default; t=1762396234; h=From:To:Subject:Date:Message-Id:MIME-Version; bh=A4LB9QeXq6PqZy/d5x1sqzHwaepaEVOdKeBhBHktmu4=; b=s4khbDwWm2BsxaH3DpC7onQQAKjSds2qRV1wLGxL4h6iVkSLUVzxetrWs1s7p44pAhQtDrjZrC7oIeNliVp3jO6FKE/5TZ2abtyUQ0UAfbG6ZaPv3kq5BAjA7fASq+QtIvc16IfdZKg1yUQS2ODXn9KdQWRlEXAc0Ej5iZKaC8g= Received: from localhost(mailfrom:feng.tang@linux.alibaba.com fp:SMTPD_---0Wrne7Zk_1762396233 cluster:ay36) by smtp.aliyun-inc.com; Thu, 06 Nov 2025 10:30:34 +0800 From: Feng Tang To: Andrew Morton , Petr Mladek , Lance Yang , paulmck@kernel.org, Steven Rostedt , linux-kernel@vger.kernel.org Cc: Feng Tang Subject: [PATCH 1/3] docs: panic: correct some sys_ifo names in sysctl doc Date: Thu, 6 Nov 2025 10:30:30 +0800 Message-Id: <20251106023032.25875-2-feng.tang@linux.alibaba.com> X-Mailer: git-send-email 2.39.5 (Apple Git-154) In-Reply-To: <20251106023032.25875-1-feng.tang@linux.alibaba.com> References: <20251106023032.25875-1-feng.tang@linux.alibaba.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Some sys_info names wered forgotten to change in patch iterations, while the right names are defined in kernel/sys_info.c. Fixes: d747755917bf ("panic: add 'panic_sys_info' sysctl to take human read= able string parameter") Signed-off-by: Feng Tang Reviewed-by: Petr Mladek --- Documentation/admin-guide/sysctl/kernel.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/Documentation/admin-guide/sysctl/kernel.rst b/Documentation/ad= min-guide/sysctl/kernel.rst index 0065a55bc09e..a397eeccaea7 100644 --- a/Documentation/admin-guide/sysctl/kernel.rst +++ b/Documentation/admin-guide/sysctl/kernel.rst @@ -911,8 +911,8 @@ to 'panic_print'. Possible values are: =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D tasks print all tasks info mem print system memory info -timer print timers info -lock print locks info if CONFIG_LOCKDEP is on +timers print timers info +locks print locks info if CONFIG_LOCKDEP is on ftrace print ftrace buffer all_bt print all CPUs backtrace (if available in the arch) blocked_tasks print only tasks in uninterruptible (blocked) state --=20 2.43.5 From nobody Fri Dec 19 14:34:36 2025 Received: from out30-101.freemail.mail.aliyun.com (out30-101.freemail.mail.aliyun.com [115.124.30.101]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B4DB1156237 for ; Thu, 6 Nov 2025 02:30:37 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=115.124.30.101 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1762396241; cv=none; b=a55BePLteMpWSpCd7vQDVfOsjrv7c3IiahfiI6Uz814eNK4WUEk8yw5CFWIXEnUztxLLGmWluzwg6U7yIr+kfqyoWD01Yu5Xtvf1PU8/A1LKCT6ypQFAwmWC2YQxUitBfR3TdNss5u4z89Psjw6F+L7Iw0cTGxAcJM/be4e0dlw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1762396241; c=relaxed/simple; bh=rldANu1bM6kh4uMiSWZljVGOAAetSLGM4A/+GRqsFDw=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=WQApCDsgHh1henEoPLnF3tmBkXXDLKMCeP9v9GI8LPzMKSzx2g9y4lbWw7gIb6tIWm+IEERtIOZCvMF/QA54DCeXfxtzAq7x4lAn4i01g+82jFVooA92IxW3jUCBcT6rSH/ooilKAn1RQ8mpfq0VXzfLFA/1W5t1GNKPEjVEuVQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.alibaba.com; spf=pass smtp.mailfrom=linux.alibaba.com; dkim=pass (1024-bit key) header.d=linux.alibaba.com header.i=@linux.alibaba.com header.b=xi7Ig5dk; arc=none smtp.client-ip=115.124.30.101 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.alibaba.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.alibaba.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.alibaba.com header.i=@linux.alibaba.com header.b="xi7Ig5dk" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.alibaba.com; s=default; t=1762396235; h=From:To:Subject:Date:Message-Id:MIME-Version; bh=TvsBwblFDeW5X/bAx8943nICS2fulV+Z3Fcs8DxGh1E=; b=xi7Ig5dkmaF3Xl6iKg7gMdrz4V4F5G+YyJqfmlE1SgWp9wjQr9zHAmfezjJ/5ZSPaNz58+3blfVzctrFNoQhQr2nCb8abHYR5NQWGZwZJCpRS2agr/E0jn2RdiN5L2eI2Tnz5NDSB7FNO14AqZOUlZZJsUDCpvvEwQJTGhoykRk= Received: from localhost(mailfrom:feng.tang@linux.alibaba.com fp:SMTPD_---0Wrne7a-_1762396234 cluster:ay36) by smtp.aliyun-inc.com; Thu, 06 Nov 2025 10:30:34 +0800 From: Feng Tang To: Andrew Morton , Petr Mladek , Lance Yang , paulmck@kernel.org, Steven Rostedt , linux-kernel@vger.kernel.org Cc: Feng Tang Subject: [PATCH 2/3] hung_task: Add hung_task_sys_info sysctl to dump sys info on task-hung Date: Thu, 6 Nov 2025 10:30:31 +0800 Message-Id: <20251106023032.25875-3-feng.tang@linux.alibaba.com> X-Mailer: git-send-email 2.39.5 (Apple Git-154) In-Reply-To: <20251106023032.25875-1-feng.tang@linux.alibaba.com> References: <20251106023032.25875-1-feng.tang@linux.alibaba.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" When task-hung happens, developers may need different kinds of system information (call-stacks, memory info, locks, etc.) to help debugging. Add 'hung_task_sys_info' sysctl knob to take human readable string like "tasks,mem,timers,locks,ftrace,...", and when task-hung happens, all requested information will be dumped. (refer kernel/sys_info.c for more details). Meanwhile, the newly introduced sys_info() call is used to unify some existing info-dumping knobs. Signed-off-by: Feng Tang Reviewed-by: Lance Yang --- Documentation/admin-guide/sysctl/kernel.rst | 5 +++ kernel/hung_task.c | 39 +++++++++++++++------ 2 files changed, 33 insertions(+), 11 deletions(-) diff --git a/Documentation/admin-guide/sysctl/kernel.rst b/Documentation/ad= min-guide/sysctl/kernel.rst index a397eeccaea7..45b4408dad31 100644 --- a/Documentation/admin-guide/sysctl/kernel.rst +++ b/Documentation/admin-guide/sysctl/kernel.rst @@ -422,6 +422,11 @@ the system boot. =20 This file shows up if ``CONFIG_DETECT_HUNG_TASK`` is enabled. =20 +hung_task_sys_info +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D +A comma separated list of extra system information to be dumped when +hung task is detected, for example, "tasks,mem,timers,locks,...". +Refer 'panic_sys_info' section below for more details. =20 hung_task_timeout_secs =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D diff --git a/kernel/hung_task.c b/kernel/hung_task.c index 84b4b049faa5..102be5a8e75a 100644 --- a/kernel/hung_task.c +++ b/kernel/hung_task.c @@ -24,6 +24,7 @@ #include #include #include +#include =20 #include =20 @@ -60,12 +61,23 @@ static unsigned long __read_mostly sysctl_hung_task_che= ck_interval_secs; static int __read_mostly sysctl_hung_task_warnings =3D 10; =20 static int __read_mostly did_panic; -static bool hung_task_show_lock; static bool hung_task_call_panic; -static bool hung_task_show_all_bt; =20 static struct task_struct *watchdog_task; =20 +/* + * A bitmask to control what kinds of system info to be printed when + * a hung task is detected, it could be task, memory, lock etc. Refer + * include/linux/sys_info.h for detailed bit definition. + */ +static unsigned long hung_task_si_mask; + +/* + * There are several sysctl knobs, and this serves as the runtime + * effective sys_info knob + */ +static unsigned long cur_si_mask; + #ifdef CONFIG_SMP /* * Should we dump all CPUs backtraces in a hung task event? @@ -235,9 +247,10 @@ static void check_hung_task(struct task_struct *t, uns= igned long timeout, total_hung_task =3D sysctl_hung_task_detect_count - prev_detect_count; trace_sched_process_hang(t); =20 + cur_si_mask =3D hung_task_si_mask; if (sysctl_hung_task_panic && total_hung_task >=3D sysctl_hung_task_panic= ) { console_verbose(); - hung_task_show_lock =3D true; + cur_si_mask |=3D SYS_INFO_LOCKS; hung_task_call_panic =3D true; } =20 @@ -260,10 +273,10 @@ static void check_hung_task(struct task_struct *t, un= signed long timeout, " disables this message.\n"); sched_show_task(t); debug_show_blocker(t, timeout); - hung_task_show_lock =3D true; + cur_si_mask |=3D SYS_INFO_LOCKS; =20 if (sysctl_hung_task_all_cpu_backtrace) - hung_task_show_all_bt =3D true; + cur_si_mask |=3D SYS_INFO_ALL_BT; if (!sysctl_hung_task_warnings) pr_info("Future hung task reports are suppressed, see sysctl kernel.hun= g_task_warnings\n"); } @@ -313,7 +326,6 @@ static void check_hung_uninterruptible_tasks(unsigned l= ong timeout) if (test_taint(TAINT_DIE) || did_panic) return; =20 - hung_task_show_lock =3D false; rcu_read_lock(); for_each_process_thread(g, t) { =20 @@ -329,12 +341,10 @@ static void check_hung_uninterruptible_tasks(unsigned= long timeout) } unlock: rcu_read_unlock(); - if (hung_task_show_lock) - debug_show_all_locks(); =20 - if (hung_task_show_all_bt) { - hung_task_show_all_bt =3D false; - trigger_all_cpu_backtrace(); + if (unlikely(cur_si_mask)) { + sys_info(cur_si_mask); + cur_si_mask =3D 0; } =20 if (hung_task_call_panic) @@ -435,6 +445,13 @@ static const struct ctl_table hung_task_sysctls[] =3D { .mode =3D 0444, .proc_handler =3D proc_doulongvec_minmax, }, + { + .procname =3D "hung_task_sys_info", + .data =3D &hung_task_si_mask, + .maxlen =3D sizeof(hung_task_si_mask), + .mode =3D 0644, + .proc_handler =3D sysctl_sys_info_handler, + }, }; =20 static void __init hung_task_sysctl_init(void) --=20 2.43.5 From nobody Fri Dec 19 14:34:36 2025 Received: from out30-119.freemail.mail.aliyun.com (out30-119.freemail.mail.aliyun.com [115.124.30.119]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A5471757EA for ; Thu, 6 Nov 2025 02:30:38 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=115.124.30.119 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1762396241; cv=none; b=fDbTto7GapV7gIBbgXibBVQTt34hykx1KsmD06Ek2zVGz7sn3VdKaxfNvHeRoNRTyd70p7rx3dUjiT3ufEe5NaLuCv7jcSOR4CylH9oiprDzGUhQX6Wtyz+nOEUGSALIZzn++dLaTYRAtKhLSlVBmQEIc5FsAVTR/5mQir91Koc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1762396241; c=relaxed/simple; bh=X2S55Lp2RJp/PN0J7+fxeHVpl5M6fOu6HJWEJFiExWM=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=HNgc2/eI4Q65qW65rzHB93mKeYueQrOMY7qEvVcHwbmpChG1IefT4P9jwy8/lVWJi/btJsaZf+tY4ZJLH1xcjKGGYmGMF3IPtASChUq70plTfs2K0g/D9DZCPYfCojUWh6ITEpjQMUNveosnTO8XIzZSQo4aOhKISZ34wBz9NcQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.alibaba.com; spf=pass smtp.mailfrom=linux.alibaba.com; dkim=pass (1024-bit key) header.d=linux.alibaba.com header.i=@linux.alibaba.com header.b=VOCI/jIL; arc=none smtp.client-ip=115.124.30.119 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.alibaba.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.alibaba.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.alibaba.com header.i=@linux.alibaba.com header.b="VOCI/jIL" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.alibaba.com; s=default; t=1762396236; h=From:To:Subject:Date:Message-Id:MIME-Version; bh=pX1HCCwUYPCV9eTsvId92R0ABA4d10Uwvm/lHQmQKuo=; b=VOCI/jILUKw2/AKzNlG3bwXK2B2J8ix59e0TrWl+z1DWEVv6yQxVBQRSoTgnWZqBJGtYxomlGv1TvpIwvNxd9uvhvT8ktg0kzq1mtZIzUfbYve4iAy9IwsmQARBIlwd40pFhC3MuaVTsDQiyfT+xKhnM0uibn2nKHarOYY9Hiu8= Received: from localhost(mailfrom:feng.tang@linux.alibaba.com fp:SMTPD_---0WrnlZnE_1762396235 cluster:ay36) by smtp.aliyun-inc.com; Thu, 06 Nov 2025 10:30:35 +0800 From: Feng Tang To: Andrew Morton , Petr Mladek , Lance Yang , paulmck@kernel.org, Steven Rostedt , linux-kernel@vger.kernel.org Cc: Feng Tang Subject: [PATCH 3/3] watchdog: add lockup_sys_info sysctl to dump sys info on system lockup Date: Thu, 6 Nov 2025 10:30:32 +0800 Message-Id: <20251106023032.25875-4-feng.tang@linux.alibaba.com> X-Mailer: git-send-email 2.39.5 (Apple Git-154) In-Reply-To: <20251106023032.25875-1-feng.tang@linux.alibaba.com> References: <20251106023032.25875-1-feng.tang@linux.alibaba.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" When soft/hard lockup happens, developers may need different kinds of system information (call-stacks, memory info, locks, etc.) to help debuggin= g. Add 'lockup_sys_info' sysctl knob to take human readable string like "tasks,mem,timers,locks,ftrace,...", and when system lockup happens, all requested information will be dumped. (refer kernel/sys_info.c for more details). Signed-off-by: Feng Tang --- Documentation/admin-guide/sysctl/kernel.rst | 5 +++++ kernel/watchdog.c | 21 ++++++++++++++++++++- 2 files changed, 25 insertions(+), 1 deletion(-) diff --git a/Documentation/admin-guide/sysctl/kernel.rst b/Documentation/ad= min-guide/sysctl/kernel.rst index 45b4408dad31..4e39e661d5ab 100644 --- a/Documentation/admin-guide/sysctl/kernel.rst +++ b/Documentation/admin-guide/sysctl/kernel.rst @@ -582,6 +582,11 @@ if leaking kernel pointer values to unprivileged users= is a concern. When ``kptr_restrict`` is set to 2, kernel pointers printed using %pK will be replaced with 0s regardless of privileges. =20 +lockup_sys_info +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D +A comma separated list of extra system information to be dumped when +soft/hard lockup is detected, for example, "tasks,mem,timers,locks,...". +Refer 'panic_sys_info' section below for more details. =20 modprobe =3D=3D=3D=3D=3D=3D=3D=3D diff --git a/kernel/watchdog.c b/kernel/watchdog.c index 659f5844393c..18d8f2a32318 100644 --- a/kernel/watchdog.c +++ b/kernel/watchdog.c @@ -25,6 +25,7 @@ #include #include #include +#include =20 #include #include @@ -53,6 +54,13 @@ static int __read_mostly watchdog_hardlockup_available; struct cpumask watchdog_cpumask __read_mostly; unsigned long *watchdog_cpumask_bits =3D cpumask_bits(&watchdog_cpumask); =20 +/* + * A bitmask to control what kinds of system info to be printed when + * system lockup is detected, it could be task, memory, lock etc. Refer + * include/linux/sys_info.h for detailed bit definition. + */ +static unsigned long lockup_si_mask; + #ifdef CONFIG_HARDLOCKUP_DETECTOR =20 # ifdef CONFIG_SMP @@ -240,6 +248,7 @@ void watchdog_hardlockup_check(unsigned int cpu, struct= pt_regs *regs) clear_bit_unlock(0, &hard_lockup_nmi_warn); } =20 + sys_info(lockup_si_mask); if (hardlockup_panic) nmi_panic(regs, "Hard LOCKUP"); =20 @@ -746,9 +755,11 @@ static enum hrtimer_restart watchdog_timer_fn(struct h= rtimer *hrtimer) unsigned long touch_ts, period_ts, now; struct pt_regs *regs =3D get_irq_regs(); int duration; - int softlockup_all_cpu_backtrace =3D sysctl_softlockup_all_cpu_backtrace; + int softlockup_all_cpu_backtrace; unsigned long flags; =20 + softlockup_all_cpu_backtrace =3D (lockup_si_mask & SYS_INFO_ALL_BT) ? + 1 : sysctl_softlockup_all_cpu_backtrace; if (!watchdog_enabled) return HRTIMER_NORESTART; =20 @@ -846,6 +857,7 @@ static enum hrtimer_restart watchdog_timer_fn(struct hr= timer *hrtimer) } =20 add_taint(TAINT_SOFTLOCKUP, LOCKDEP_STILL_OK); + sys_info(lockup_si_mask & ~SYS_INFO_ALL_BT); if (softlockup_panic) panic("softlockup: hung tasks"); } @@ -1178,6 +1190,13 @@ static const struct ctl_table watchdog_sysctls[] =3D= { .mode =3D 0644, .proc_handler =3D proc_watchdog_cpumask, }, + { + .procname =3D "lockup_sys_info", + .data =3D &lockup_si_mask, + .maxlen =3D sizeof(lockup_si_mask), + .mode =3D 0644, + .proc_handler =3D sysctl_sys_info_handler, + }, #ifdef CONFIG_SOFTLOCKUP_DETECTOR { .procname =3D "soft_watchdog", --=20 2.43.5