From nobody Fri Jun 19 09:50:04 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4210AC3527E for ; Wed, 6 Apr 2022 00:09:50 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1582176AbiDEXmX (ORCPT ); Tue, 5 Apr 2022 19:42:23 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:43516 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1352030AbiDEPLv (ORCPT ); Tue, 5 Apr 2022 11:11:51 -0400 Received: from galois.linutronix.de (Galois.linutronix.de [193.142.43.55]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 3F6A010857C for ; Tue, 5 Apr 2022 06:25:40 -0700 (PDT) From: John Ogness DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1649165138; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=9RxnbEIrXzBC0NMsbr0LXTgI58lfBLedW2KkOWR443M=; b=StmgCpnWJcyWAWVSMuwzQ4qK7xtyNzQ7IponkIaZJlAMzGBB6YToT0O6seLv9qvWKn9Fsv WCW+36ZjElr84fG3Z9GnKAhBXp07h8AtWU7QlPiwli3dnO/SI5DfqujT8Y4byw0Wjq8Epm V4jOwPDLLCdH4FVcttSCDyrQ2w0v/q6QACgAK4Cvx3NFiYgW2IAU0gJGXotvNZAA6FJHq7 fbt/vmNI2nx0DKYWO4/5wbd2uQXimpbf5ixVKH9m+ki6baWE9R25qZ78j2pZ38SIuhqb+U UQTI5A5FHzZSigje1+3TXykAp1AhhZ9/Ekd2eLCCab9AairTNiviFR7cPABABA== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1649165138; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=9RxnbEIrXzBC0NMsbr0LXTgI58lfBLedW2KkOWR443M=; b=+o9H3hvsRb1NxsQrtr1tzPik6Ga8+T69XjZQFtfT+maEhuk2j16bMOllKuNeUfDXDAyTaG esjwVBya0Np3ZeDw== To: Petr Mladek Cc: Sergey Senozhatsky , Steven Rostedt , Thomas Gleixner , linux-kernel@vger.kernel.org, Andrew Morton , Marco Elver , Randy Dunlap , Alexander Potapenko , Stephen Boyd , Nicholas Piggin Subject: [PATCH printk v2 01/12] printk: rename cpulock functions Date: Tue, 5 Apr 2022 15:31:24 +0206 Message-Id: <20220405132535.649171-2-john.ogness@linutronix.de> In-Reply-To: <20220405132535.649171-1-john.ogness@linutronix.de> References: <20220405132535.649171-1-john.ogness@linutronix.de> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Since the printk cpulock is CPU-reentrant and since it is used in all contexts, its usage must be carefully considered and most likely will require programming locklessly. To avoid mistaking the printk cpulock as a typical lock, rename it to cpu_sync. The main functions then become: printk_cpu_sync_get_irqsave(flags); printk_cpu_sync_put_irqrestore(flags); Add extra notes of caution in the function description to help developers understand the requirements for correct usage. Signed-off-by: John Ogness Reviewed-by: Petr Mladek --- include/linux/printk.h | 54 +++++++++++++++++++------------- kernel/printk/printk.c | 71 +++++++++++++++++++++--------------------- lib/dump_stack.c | 4 +-- lib/nmi_backtrace.c | 4 +-- 4 files changed, 73 insertions(+), 60 deletions(-) diff --git a/include/linux/printk.h b/include/linux/printk.h index 1522df223c0f..859323a52985 100644 --- a/include/linux/printk.h +++ b/include/linux/printk.h @@ -277,43 +277,55 @@ static inline void printk_trigger_flush(void) #endif =20 #ifdef CONFIG_SMP -extern int __printk_cpu_trylock(void); -extern void __printk_wait_on_cpu_lock(void); -extern void __printk_cpu_unlock(void); +extern int __printk_cpu_sync_try_get(void); +extern void __printk_cpu_sync_wait(void); +extern void __printk_cpu_sync_put(void); =20 /** - * printk_cpu_lock_irqsave() - Acquire the printk cpu-reentrant spinning - * lock and disable interrupts. + * printk_cpu_sync_get_irqsave() - Acquire the printk cpu-reentrant spinni= ng + * lock and disable interrupts. * @flags: Stack-allocated storage for saving local interrupt state, - * to be passed to printk_cpu_unlock_irqrestore(). + * to be passed to printk_cpu_sync_put_irqrestore(). * * If the lock is owned by another CPU, spin until it becomes available. * Interrupts are restored while spinning. + * + * CAUTION: This function must be used carefully. It does not behave like a + * typical lock. Here are important things to watch out for... + * + * * This function is reentrant on the same CPU. Therefore the calling + * code must not assume exclusive access to data if code accessing t= he + * data can run reentrant or within NMI context on the same CPU. + * + * * If there exists usage of this function from NMI context, it becom= es + * unsafe to perform any type of locking or spinning to wait for oth= er + * CPUs after calling this function from any context. This includes + * using spinlocks or any other busy-waiting synchronization methods. */ -#define printk_cpu_lock_irqsave(flags) \ - for (;;) { \ - local_irq_save(flags); \ - if (__printk_cpu_trylock()) \ - break; \ - local_irq_restore(flags); \ - __printk_wait_on_cpu_lock(); \ +#define printk_cpu_sync_get_irqsave(flags) \ + for (;;) { \ + local_irq_save(flags); \ + if (__printk_cpu_sync_try_get()) \ + break; \ + local_irq_restore(flags); \ + __printk_cpu_sync_wait(); \ } =20 /** - * printk_cpu_unlock_irqrestore() - Release the printk cpu-reentrant spinn= ing - * lock and restore interrupts. - * @flags: Caller's saved interrupt state, from printk_cpu_lock_irqsave(). + * printk_cpu_sync_put_irqrestore() - Release the printk cpu-reentrant spi= nning + * lock and restore interrupts. + * @flags: Caller's saved interrupt state, from printk_cpu_sync_get_irqsav= e(). */ -#define printk_cpu_unlock_irqrestore(flags) \ +#define printk_cpu_sync_put_irqrestore(flags) \ do { \ - __printk_cpu_unlock(); \ + __printk_cpu_sync_put(); \ local_irq_restore(flags); \ - } while (0) \ + } while (0) =20 #else =20 -#define printk_cpu_lock_irqsave(flags) ((void)flags) -#define printk_cpu_unlock_irqrestore(flags) ((void)flags) +#define printk_cpu_sync_get_irqsave(flags) ((void)flags) +#define printk_cpu_sync_put_irqrestore(flags) ((void)flags) =20 #endif /* CONFIG_SMP */ =20 diff --git a/kernel/printk/printk.c b/kernel/printk/printk.c index da03c15ecc89..13a1eebe72af 100644 --- a/kernel/printk/printk.c +++ b/kernel/printk/printk.c @@ -3667,26 +3667,26 @@ EXPORT_SYMBOL_GPL(kmsg_dump_rewind); #endif =20 #ifdef CONFIG_SMP -static atomic_t printk_cpulock_owner =3D ATOMIC_INIT(-1); -static atomic_t printk_cpulock_nested =3D ATOMIC_INIT(0); +static atomic_t printk_cpu_sync_owner =3D ATOMIC_INIT(-1); +static atomic_t printk_cpu_sync_nested =3D ATOMIC_INIT(0); =20 /** - * __printk_wait_on_cpu_lock() - Busy wait until the printk cpu-reentrant - * spinning lock is not owned by any CPU. + * __printk_cpu_sync_wait() - Busy wait until the printk cpu-reentrant + * spinning lock is not owned by any CPU. * * Context: Any context. */ -void __printk_wait_on_cpu_lock(void) +void __printk_cpu_sync_wait(void) { do { cpu_relax(); - } while (atomic_read(&printk_cpulock_owner) !=3D -1); + } while (atomic_read(&printk_cpu_sync_owner) !=3D -1); } -EXPORT_SYMBOL(__printk_wait_on_cpu_lock); +EXPORT_SYMBOL(__printk_cpu_sync_wait); =20 /** - * __printk_cpu_trylock() - Try to acquire the printk cpu-reentrant - * spinning lock. + * __printk_cpu_sync_try_get() - Try to acquire the printk cpu-reentrant + * spinning lock. * * If no processor has the lock, the calling processor takes the lock and * becomes the owner. If the calling processor is already the owner of the @@ -3695,7 +3695,7 @@ EXPORT_SYMBOL(__printk_wait_on_cpu_lock); * Context: Any context. Expects interrupts to be disabled. * Return: 1 on success, otherwise 0. */ -int __printk_cpu_trylock(void) +int __printk_cpu_sync_try_get(void) { int cpu; int old; @@ -3705,79 +3705,80 @@ int __printk_cpu_trylock(void) /* * Guarantee loads and stores from this CPU when it is the lock owner * are _not_ visible to the previous lock owner. This pairs with - * __printk_cpu_unlock:B. + * __printk_cpu_sync_put:B. * * Memory barrier involvement: * - * If __printk_cpu_trylock:A reads from __printk_cpu_unlock:B, then - * __printk_cpu_unlock:A can never read from __printk_cpu_trylock:B. + * If __printk_cpu_sync_try_get:A reads from __printk_cpu_sync_put:B, + * then __printk_cpu_sync_put:A can never read from + * __printk_cpu_sync_try_get:B. * * Relies on: * - * RELEASE from __printk_cpu_unlock:A to __printk_cpu_unlock:B + * RELEASE from __printk_cpu_sync_put:A to __printk_cpu_sync_put:B * of the previous CPU * matching - * ACQUIRE from __printk_cpu_trylock:A to __printk_cpu_trylock:B - * of this CPU + * ACQUIRE from __printk_cpu_sync_try_get:A to + * __printk_cpu_sync_try_get:B of this CPU */ - old =3D atomic_cmpxchg_acquire(&printk_cpulock_owner, -1, - cpu); /* LMM(__printk_cpu_trylock:A) */ + old =3D atomic_cmpxchg_acquire(&printk_cpu_sync_owner, -1, + cpu); /* LMM(__printk_cpu_sync_try_get:A) */ if (old =3D=3D -1) { /* * This CPU is now the owner and begins loading/storing - * data: LMM(__printk_cpu_trylock:B) + * data: LMM(__printk_cpu_sync_try_get:B) */ return 1; =20 } else if (old =3D=3D cpu) { /* This CPU is already the owner. */ - atomic_inc(&printk_cpulock_nested); + atomic_inc(&printk_cpu_sync_nested); return 1; } =20 return 0; } -EXPORT_SYMBOL(__printk_cpu_trylock); +EXPORT_SYMBOL(__printk_cpu_sync_try_get); =20 /** - * __printk_cpu_unlock() - Release the printk cpu-reentrant spinning lock. + * __printk_cpu_sync_put() - Release the printk cpu-reentrant spinning loc= k. * * The calling processor must be the owner of the lock. * * Context: Any context. Expects interrupts to be disabled. */ -void __printk_cpu_unlock(void) +void __printk_cpu_sync_put(void) { - if (atomic_read(&printk_cpulock_nested)) { - atomic_dec(&printk_cpulock_nested); + if (atomic_read(&printk_cpu_sync_nested)) { + atomic_dec(&printk_cpu_sync_nested); return; } =20 /* * This CPU is finished loading/storing data: - * LMM(__printk_cpu_unlock:A) + * LMM(__printk_cpu_sync_put:A) */ =20 /* * Guarantee loads and stores from this CPU when it was the * lock owner are visible to the next lock owner. This pairs - * with __printk_cpu_trylock:A. + * with __printk_cpu_sync_try_get:A. * * Memory barrier involvement: * - * If __printk_cpu_trylock:A reads from __printk_cpu_unlock:B, - * then __printk_cpu_trylock:B reads from __printk_cpu_unlock:A. + * If __printk_cpu_sync_try_get:A reads from __printk_cpu_sync_put:B, + * then __printk_cpu_sync_try_get:B reads from __printk_cpu_sync_put:A. * * Relies on: * - * RELEASE from __printk_cpu_unlock:A to __printk_cpu_unlock:B + * RELEASE from __printk_cpu_sync_put:A to __printk_cpu_sync_put:B * of this CPU * matching - * ACQUIRE from __printk_cpu_trylock:A to __printk_cpu_trylock:B - * of the next CPU + * ACQUIRE from __printk_cpu_sync_try_get:A to + * __printk_cpu_sync_try_get:B of the next CPU */ - atomic_set_release(&printk_cpulock_owner, - -1); /* LMM(__printk_cpu_unlock:B) */ + atomic_set_release(&printk_cpu_sync_owner, + -1); /* LMM(__printk_cpu_sync_put:B) */ } -EXPORT_SYMBOL(__printk_cpu_unlock); +EXPORT_SYMBOL(__printk_cpu_sync_put); #endif /* CONFIG_SMP */ diff --git a/lib/dump_stack.c b/lib/dump_stack.c index 6b7f1bf6715d..83471e81501a 100644 --- a/lib/dump_stack.c +++ b/lib/dump_stack.c @@ -102,9 +102,9 @@ asmlinkage __visible void dump_stack_lvl(const char *lo= g_lvl) * Permit this cpu to perform nested stack dumps while serialising * against other CPUs */ - printk_cpu_lock_irqsave(flags); + printk_cpu_sync_get_irqsave(flags); __dump_stack(log_lvl); - printk_cpu_unlock_irqrestore(flags); + printk_cpu_sync_put_irqrestore(flags); } EXPORT_SYMBOL(dump_stack_lvl); =20 diff --git a/lib/nmi_backtrace.c b/lib/nmi_backtrace.c index 199ab201d501..d01aec6ae15c 100644 --- a/lib/nmi_backtrace.c +++ b/lib/nmi_backtrace.c @@ -99,7 +99,7 @@ bool nmi_cpu_backtrace(struct pt_regs *regs) * Allow nested NMI backtraces while serializing * against other CPUs. */ - printk_cpu_lock_irqsave(flags); + printk_cpu_sync_get_irqsave(flags); if (!READ_ONCE(backtrace_idle) && regs && cpu_in_idle(instruction_pointe= r(regs))) { pr_warn("NMI backtrace for cpu %d skipped: idling at %pS\n", cpu, (void *)instruction_pointer(regs)); @@ -110,7 +110,7 @@ bool nmi_cpu_backtrace(struct pt_regs *regs) else dump_stack(); } - printk_cpu_unlock_irqrestore(flags); + printk_cpu_sync_put_irqrestore(flags); cpumask_clear_cpu(cpu, to_cpumask(backtrace_mask)); return true; } --=20 2.30.2 From nobody Fri Jun 19 09:50:04 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7C90CC433EF for ; Tue, 5 Apr 2022 20:15:57 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1444522AbiDEUIP (ORCPT ); Tue, 5 Apr 2022 16:08:15 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37040 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1352019AbiDEPLv (ORCPT ); Tue, 5 Apr 2022 11:11:51 -0400 Received: from galois.linutronix.de (Galois.linutronix.de [IPv6:2a0a:51c0:0:12e:550::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 4393B108740 for ; Tue, 5 Apr 2022 06:25:40 -0700 (PDT) From: John Ogness DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1649165138; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=w6/o/PjZ0Qpg9L4OKIrmGvdcvR+21LH+ujrVsqfiSt0=; b=vTX9PptTisS4B8CBAJwM/Vt3Innqkp0IOmP3EfSf0o3TJbgWvw4uMWAXno+zj0xKS5gmXm h48mChyV9dtz3Jz+PUjBqAIWm+5xr/bE775UIcJkQjBvty+w9Tk0I5/9sqEH0kwnyODRro 7FTymDUVvdch6af6pTAZCTfsqn8FqEVHa7yrbunD0TvHo/5lOCx0tAFOMR0JJ6BKRrWjWX ruP9k94SLnEvFsGA2hD1gI6Ehm6aTpED92hEW41AbPzpatkFRH6C+RxFbhU16BwYR/tdQv NaoQtQOysaW8oB4AjBIvCCxCwn43dhNyN7pvPSbYR0s54N4JDQA4MPs1YgLdkg== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1649165138; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=w6/o/PjZ0Qpg9L4OKIrmGvdcvR+21LH+ujrVsqfiSt0=; b=VIRHzr2fjG0CztJXCAS8nOaDklkaoRPnvTp89xiTp/mEeqw8+0obiAxx9udA4gnOMwZUk2 ejEvSeO1bAJYNNBw== To: Petr Mladek Cc: Sergey Senozhatsky , Steven Rostedt , Thomas Gleixner , linux-kernel@vger.kernel.org Subject: [PATCH printk v2 02/12] printk: cpu sync always disable interrupts Date: Tue, 5 Apr 2022 15:31:25 +0206 Message-Id: <20220405132535.649171-3-john.ogness@linutronix.de> In-Reply-To: <20220405132535.649171-1-john.ogness@linutronix.de> References: <20220405132535.649171-1-john.ogness@linutronix.de> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" The CPU sync functions are a NOP for !CONFIG_SMP. But for !CONFIG_SMP they still need to disable interrupts in order to preserve context within the CPU sync sections. Signed-off-by: John Ogness Reviewed-by: Petr Mladek Reviewed-by: Sergey Senozhatsky --- include/linux/printk.h | 18 +++++++++--------- 1 file changed, 9 insertions(+), 9 deletions(-) diff --git a/include/linux/printk.h b/include/linux/printk.h index 859323a52985..b70a42f94031 100644 --- a/include/linux/printk.h +++ b/include/linux/printk.h @@ -281,9 +281,16 @@ extern int __printk_cpu_sync_try_get(void); extern void __printk_cpu_sync_wait(void); extern void __printk_cpu_sync_put(void); =20 +#else + +#define __printk_cpu_sync_try_get() true +#define __printk_cpu_sync_wait() +#define __printk_cpu_sync_put() +#endif /* CONFIG_SMP */ + /** - * printk_cpu_sync_get_irqsave() - Acquire the printk cpu-reentrant spinni= ng - * lock and disable interrupts. + * printk_cpu_sync_get_irqsave() - Disable interrupts and acquire the prin= tk + * cpu-reentrant spinning lock. * @flags: Stack-allocated storage for saving local interrupt state, * to be passed to printk_cpu_sync_put_irqrestore(). * @@ -322,13 +329,6 @@ extern void __printk_cpu_sync_put(void); local_irq_restore(flags); \ } while (0) =20 -#else - -#define printk_cpu_sync_get_irqsave(flags) ((void)flags) -#define printk_cpu_sync_put_irqrestore(flags) ((void)flags) - -#endif /* CONFIG_SMP */ - extern int kptr_restrict; =20 /** --=20 2.30.2 From nobody Fri Jun 19 09:50:04 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 36E58C43219 for ; Tue, 5 Apr 2022 22:39:24 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1451813AbiDEWav (ORCPT ); Tue, 5 Apr 2022 18:30:51 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:43672 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1352003AbiDEPLv (ORCPT ); Tue, 5 Apr 2022 11:11:51 -0400 Received: from galois.linutronix.de (Galois.linutronix.de [IPv6:2a0a:51c0:0:12e:550::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 437FD10857E for ; Tue, 5 Apr 2022 06:25:40 -0700 (PDT) From: John Ogness DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1649165138; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=vNeqajyflYseDN9U86copcD82UelhphMBtFer240cuQ=; b=W7/pb8xYxRRmMWdND5mpwMKRennaa0wwlKgGgp6un5Dap/qjWXdJga/yVEx1Lsm8KYQYqi +3c6JE15H0LAtHu33mG3Dep57WnXKRCrw+vyGSipCqT8Ff6HBsfEMkPSzHWhZurAUZZZhZ bLNeJtV0WV7TG/oqVL6padKSmVJsvfZk3DoYS65pdMNxokLQu5HkAQfdTK/7oEYn7T/bKv anAtrztsJMDz/1fUaT2Z8DEYDGwr71G/Uzbr77HUVN+EIrcqW9c2tloVxxxUU2+YIkNA+v JTZPehM+N2Q6U22YJ0T07YG6XCgddWEk/GF7Yyb7piO7iQxhK1BL6VJm3daieA== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1649165138; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=vNeqajyflYseDN9U86copcD82UelhphMBtFer240cuQ=; b=xawPdNh+P1cLeleF+r1Cmfe51m5joVK55Sjm+xx2D1TUS1vEZYcezfancq6p9tzdDvD5Jn GTT7zZcEs5mywaBw== To: Petr Mladek Cc: Sergey Senozhatsky , Steven Rostedt , Thomas Gleixner , linux-kernel@vger.kernel.org Subject: [PATCH printk v2 03/12] printk: get caller_id/timestamp after migration disable Date: Tue, 5 Apr 2022 15:31:26 +0206 Message-Id: <20220405132535.649171-4-john.ogness@linutronix.de> In-Reply-To: <20220405132535.649171-1-john.ogness@linutronix.de> References: <20220405132535.649171-1-john.ogness@linutronix.de> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Currently the local CPU timestamp and caller_id for the record are collected while migration is enabled. Since this information is CPU-specific, it should be collected with migration disabled. Migration is disabled immediately after collecting this information anyway, so just move the information collection to after the migration disabling. Signed-off-by: John Ogness Reviewed-by: Sergey Senozhatsky Reviewed-by: Petr Mladek --- kernel/printk/printk.c | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-) diff --git a/kernel/printk/printk.c b/kernel/printk/printk.c index 13a1eebe72af..9774a3f6bafb 100644 --- a/kernel/printk/printk.c +++ b/kernel/printk/printk.c @@ -2041,7 +2041,7 @@ static inline void printk_delay(void) static inline u32 printk_caller_id(void) { return in_task() ? task_pid_nr(current) : - 0x80000000 + raw_smp_processor_id(); + 0x80000000 + smp_processor_id(); } =20 /** @@ -2123,7 +2123,6 @@ int vprintk_store(int facility, int level, const struct dev_printk_info *dev_info, const char *fmt, va_list args) { - const u32 caller_id =3D printk_caller_id(); struct prb_reserved_entry e; enum printk_info_flags flags =3D 0; struct printk_record r; @@ -2133,10 +2132,14 @@ int vprintk_store(int facility, int level, u8 *recursion_ptr; u16 reserve_size; va_list args2; + u32 caller_id; u16 text_len; int ret =3D 0; u64 ts_nsec; =20 + if (!printk_enter_irqsave(recursion_ptr, irqflags)) + return 0; + /* * Since the duration of printk() can vary depending on the message * and state of the ringbuffer, grab the timestamp now so that it is @@ -2145,8 +2148,7 @@ int vprintk_store(int facility, int level, */ ts_nsec =3D local_clock(); =20 - if (!printk_enter_irqsave(recursion_ptr, irqflags)) - return 0; + caller_id =3D printk_caller_id(); =20 /* * The sprintf needs to come first since the syslog prefix might be --=20 2.30.2 From nobody Fri Jun 19 09:50:04 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id C8EC4C46467 for ; Tue, 5 Apr 2022 21:53:45 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1386971AbiDEVux (ORCPT ); Tue, 5 Apr 2022 17:50:53 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37308 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1352080AbiDEPLv (ORCPT ); Tue, 5 Apr 2022 11:11:51 -0400 Received: from galois.linutronix.de (Galois.linutronix.de [193.142.43.55]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C37EC108752 for ; Tue, 5 Apr 2022 06:25:40 -0700 (PDT) From: John Ogness DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1649165139; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=qCoK+UpOteRpKNTnktOH3CIds5NlB8/teGpvkNbdm/o=; b=nlTj3Ce3uqHvPzkg5VT1YJISIBYJx6jyrBJVZoP/zoMQLOBYgS9/mz/tHZE57dbpsLvNBG t3ogRuHEKYIcxGgFaevsxEf2AYhOY3mUFDRUbwLIGIx8Kd8C/RW5uYx8S6/ElCEM+e+iHc xdmjb2goJeW3HQ1KtK3wdUeHZcVbDA1ikrd81KM/ULK+xqNyVpanpukrYeqdhMEsWsQh/G 98YDrl2rHa862AaLUcBSBUmbdqYnGA2GT8n9OyoVrHwuDawdhy8XzINEFz9E4ttF2FGi0u gGisRV2mVOScC29wyoMvhJEfRfEa1cxDWzH09AG4kJe/MSmj94+sb8ZelUna1g== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1649165139; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=qCoK+UpOteRpKNTnktOH3CIds5NlB8/teGpvkNbdm/o=; b=qey4gXgl+SNWj8Z+K+7T2T0bm3TGMoJicFXj/uRlHpCjXmlqqNNcWYuvLnlVZ3dNii6g4t r6scoEhXCL6Ms8CA== To: Petr Mladek Cc: Sergey Senozhatsky , Steven Rostedt , Thomas Gleixner , linux-kernel@vger.kernel.org Subject: [PATCH printk v2 04/12] printk: call boot_delay_msec() in printk_delay() Date: Tue, 5 Apr 2022 15:31:27 +0206 Message-Id: <20220405132535.649171-5-john.ogness@linutronix.de> In-Reply-To: <20220405132535.649171-1-john.ogness@linutronix.de> References: <20220405132535.649171-1-john.ogness@linutronix.de> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" boot_delay_msec() is always called immediately before printk_delay() so just call it from within printk_delay(). Signed-off-by: John Ogness Reviewed-by: Sergey Senozhatsky Reviewed-by: Petr Mladek --- kernel/printk/printk.c | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/kernel/printk/printk.c b/kernel/printk/printk.c index 9774a3f6bafb..fb9a6eaf54fd 100644 --- a/kernel/printk/printk.c +++ b/kernel/printk/printk.c @@ -2026,8 +2026,10 @@ static u8 *__printk_recursion_counter(void) =20 int printk_delay_msec __read_mostly; =20 -static inline void printk_delay(void) +static inline void printk_delay(int level) { + boot_delay_msec(level); + if (unlikely(printk_delay_msec)) { int m =3D printk_delay_msec; =20 @@ -2252,8 +2254,7 @@ asmlinkage int vprintk_emit(int facility, int level, in_sched =3D true; } =20 - boot_delay_msec(level); - printk_delay(); + printk_delay(level); =20 printed_len =3D vprintk_store(facility, level, dev_info, fmt, args); =20 --=20 2.30.2 From nobody Fri Jun 19 09:50:04 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id C7D74C4167B for ; Wed, 6 Apr 2022 01:33:58 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1842677AbiDFBeD (ORCPT ); Tue, 5 Apr 2022 21:34:03 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38244 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1352955AbiDEPLw (ORCPT ); Tue, 5 Apr 2022 11:11:52 -0400 Received: from galois.linutronix.de (Galois.linutronix.de [IPv6:2a0a:51c0:0:12e:550::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 30FF1108BF8 for ; Tue, 5 Apr 2022 06:25:43 -0700 (PDT) From: John Ogness DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1649165139; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=+FB2Z6hYDK+Vgm7eUKCCLcgZ+HVauZdnVGbPABY+Y1k=; b=icB3EMpOJuf8texvfCSBnBqyOgoXH4jrA/rn3q+E7sG/yby41Mh5RJVHaEJI7dpCiYFw1h u8sMcgI+zzoeJ/2gSxNfpASvWFjAk+zceA9Yjq5Q5aTLMea/YVGUcS3VV7CV34e8IN9UZs u2aETwfQJwYFp6HBLkb6exE9JqEDNJBOVgf3gWG78kmRu2QqJ89FQIWzh7M4uSooYh+eoJ 10F0fFOZdMWVreNSqjvbvq/tkEr0xQiR3hWt8S25/fhLxoLk0IVDORAKlB7IHU1bykGZ1V 3Swttqj/DbI/eff1+04GM9MaFcj4xrBatdY6MjCj5AWClOAEfOKEI3Xlp22Mpg== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1649165139; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=+FB2Z6hYDK+Vgm7eUKCCLcgZ+HVauZdnVGbPABY+Y1k=; b=2M0/eko3SPx6uw1glcr44L6P/SRq4cDR4eI8PzhDrRh6QpnASTNBIx2YwRQp4QzShOodSt 243RTUjuY6GI0rBA== To: Petr Mladek Cc: Sergey Senozhatsky , Steven Rostedt , Thomas Gleixner , linux-kernel@vger.kernel.org Subject: [PATCH printk v2 05/12] printk: add macro for console detail messages Date: Tue, 5 Apr 2022 15:31:28 +0206 Message-Id: <20220405132535.649171-6-john.ogness@linutronix.de> In-Reply-To: <20220405132535.649171-1-john.ogness@linutronix.de> References: <20220405132535.649171-1-john.ogness@linutronix.de> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" It is useful to generate log messages that include details about the related console. Rather than duplicate the logic to assemble the details, put that logic into a macro printk_console_msg(). Once console printers become threaded, this macro will find many users. Signed-off-by: John Ogness --- kernel/printk/printk.c | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/kernel/printk/printk.c b/kernel/printk/printk.c index fb9a6eaf54fd..39ca9d758e52 100644 --- a/kernel/printk/printk.c +++ b/kernel/printk/printk.c @@ -2993,6 +2993,10 @@ static void try_enable_default_console(struct consol= e *newcon) newcon->flags |=3D CON_CONSDEV; } =20 +#define printk_console_msg(c, lvl, msg) \ + printk(lvl pr_fmt("%sconsole [%s%d] " msg "\n"), \ + (c->flags & CON_BOOT) ? "boot" : "", c->name, c->index) + /* * The console driver calls this routine during kernel initialization * to register the console printing procedure with printk() and to @@ -3131,9 +3135,7 @@ void register_console(struct console *newcon) * users know there might be something in the kernel's log buffer that * went to the bootconsole (that they do not see on the real console) */ - pr_info("%sconsole [%s%d] enabled\n", - (newcon->flags & CON_BOOT) ? "boot" : "" , - newcon->name, newcon->index); + printk_console_msg(newcon, KERN_INFO, "enabled"); if (bootcon_enabled && ((newcon->flags & (CON_CONSDEV | CON_BOOT)) =3D=3D CON_CONSDEV) && !keep_bootcon) { @@ -3152,9 +3154,7 @@ int unregister_console(struct console *console) struct console *con; int res; =20 - pr_info("%sconsole [%s%d] disabled\n", - (console->flags & CON_BOOT) ? "boot" : "" , - console->name, console->index); + printk_console_msg(console, KERN_INFO, "disabled"); =20 res =3D _braille_unregister_console(console); if (res < 0) --=20 2.30.2 From nobody Fri Jun 19 09:50:04 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9F5A1C433FE for ; Wed, 6 Apr 2022 01:42:37 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1843098AbiDFBjW (ORCPT ); Tue, 5 Apr 2022 21:39:22 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38684 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1353501AbiDEPLw (ORCPT ); Tue, 5 Apr 2022 11:11:52 -0400 Received: from galois.linutronix.de (Galois.linutronix.de [IPv6:2a0a:51c0:0:12e:550::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 30C0D10854C for ; Tue, 5 Apr 2022 06:25:43 -0700 (PDT) From: John Ogness DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1649165140; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=vrrRJ4Jh7sy6m7VY3IsTkhoyDvtmzs9FV+Ir3K5rujQ=; b=kCZU11I9RK6wHhqwjYD7RLIKNUpK9tploRzv09+7+nn9zcCiWBJYfyFM9HyX+PNd2Bw0Xl I4skEQoHhw42NPbMg05phgNT/Vd+v1VScISIvkBHR0zh/0cYO3VnKP6TLzb+Ui9KrHwvwC cBRXXDhckHDW/DOko81E9J6jdoKkMXBKQBkatATQanJad+r7Q1VMOuJl1rjS77/CVK2DMR oDoeE84Mev8Smglsts4RlvbHZtEKAJPz/NF4DjUidDKXO9B4t4gQjTWuArw6y9/Gp/MbCt NSJk2lcRKtA0sJXia9tgOCqAt4fyvMX1cru6gFflTklLh7oGSwjNb1CsJ/znZg== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1649165140; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=vrrRJ4Jh7sy6m7VY3IsTkhoyDvtmzs9FV+Ir3K5rujQ=; b=hl2vwL5Z8M163dSY0siB9a9YRXB51zfhozycbXvOa1v7MniBOc9qGiPgOFDSZm07c3FvNf k2KuCWa5wUXdQKCw== To: Petr Mladek Cc: Sergey Senozhatsky , Steven Rostedt , Thomas Gleixner , linux-kernel@vger.kernel.org, Greg Kroah-Hartman Subject: [PATCH printk v2 06/12] printk: refactor and rework printing logic Date: Tue, 5 Apr 2022 15:31:29 +0206 Message-Id: <20220405132535.649171-7-john.ogness@linutronix.de> In-Reply-To: <20220405132535.649171-1-john.ogness@linutronix.de> References: <20220405132535.649171-1-john.ogness@linutronix.de> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Refactor/rework printing logic in order to prepare for moving to threaded console printing. - Move @console_seq into struct console so that the current "position" of each console can be tracked individually. - Move @console_dropped into struct console so that the current drop count of each console can be tracked individually. - Modify printing logic so that each console independently loads, prepares, and prints its next record. - Remove exclusive_console logic. Since console positions are handled independently, replaying past records occurs naturally. - Update the comments explaining why preemption is disabled while printing from printk() context. With these changes, there is a change in behavior: the console replaying the log (formerly exclusive console) will no longer block other consoles. New messages appear on the other consoles while the newly added console is still replaying. Signed-off-by: John Ogness --- include/linux/console.h | 2 + kernel/printk/printk.c | 429 ++++++++++++++++++++-------------------- 2 files changed, 220 insertions(+), 211 deletions(-) diff --git a/include/linux/console.h b/include/linux/console.h index 7cd758a4f44e..8c1686e2c233 100644 --- a/include/linux/console.h +++ b/include/linux/console.h @@ -151,6 +151,8 @@ struct console { int cflag; uint ispeed; uint ospeed; + u64 seq; + unsigned long dropped; void *data; struct console *next; }; diff --git a/kernel/printk/printk.c b/kernel/printk/printk.c index 39ca9d758e52..db9278dba775 100644 --- a/kernel/printk/printk.c +++ b/kernel/printk/printk.c @@ -280,11 +280,6 @@ static bool panic_in_progress(void) */ static int console_locked, console_suspended; =20 -/* - * If exclusive_console is non-NULL then only this console is to be printe= d to. - */ -static struct console *exclusive_console; - /* * Array of consoles built from command line options (console=3D) */ @@ -374,12 +369,6 @@ static u64 syslog_seq; static size_t syslog_partial; static bool syslog_time; =20 -/* All 3 protected by @console_sem. */ -/* the next printk record to write to the console */ -static u64 console_seq; -static u64 exclusive_console_stop_seq; -static unsigned long console_dropped; - struct latched_seq { seqcount_latch_t latch; u64 val[2]; @@ -1911,47 +1900,26 @@ static int console_trylock_spinning(void) } =20 /* - * Call the console drivers, asking them to write out - * log_buf[start] to log_buf[end - 1]. - * The console_lock must be held. + * Call the specified console driver, asking it to write out the specified + * text and length. For non-extended consoles, if any records have been + * dropped, a dropped message will be written out first. */ -static void call_console_drivers(const char *ext_text, size_t ext_len, - const char *text, size_t len) +static void call_console_driver(struct console *con, const char *text, siz= e_t len) { static char dropped_text[64]; - size_t dropped_len =3D 0; - struct console *con; + size_t dropped_len; =20 trace_console_rcuidle(text, len); =20 - if (!console_drivers) - return; - - if (console_dropped) { + if (con->dropped && !(con->flags & CON_EXTENDED)) { dropped_len =3D snprintf(dropped_text, sizeof(dropped_text), "** %lu printk messages dropped **\n", - console_dropped); - console_dropped =3D 0; + con->dropped); + con->dropped =3D 0; + con->write(con, dropped_text, dropped_len); } =20 - for_each_console(con) { - if (exclusive_console && con !=3D exclusive_console) - continue; - if (!(con->flags & CON_ENABLED)) - continue; - if (!con->write) - continue; - if (!cpu_online(smp_processor_id()) && - !(con->flags & CON_ANYTIME)) - continue; - if (con->flags & CON_EXTENDED) - con->write(con, ext_text, ext_len); - else { - if (dropped_len) - con->write(con, dropped_text, dropped_len); - con->write(con, text, len); - } - } + con->write(con, text, len); } =20 /* @@ -2261,15 +2229,18 @@ asmlinkage int vprintk_emit(int facility, int level, /* If called from the scheduler, we can not call up(). */ if (!in_sched) { /* - * Disable preemption to avoid being preempted while holding - * console_sem which would prevent anyone from printing to - * console + * The caller may be holding system-critical or + * timing-sensitive locks. Disable preemption during + * printing of all remaining records to all consoles so that + * this context can return as soon as possible. Hopefully + * another printk() caller will take over the printing. */ preempt_disable(); /* * Try to acquire and then immediately release the console - * semaphore. The release will print out buffers and wake up - * /dev/kmsg and syslog() users. + * semaphore. The release will print out buffers. With the + * spinning variant, this context tries to take over the + * printing from another printing context. */ if (console_trylock_spinning()) console_unlock(); @@ -2307,11 +2278,9 @@ EXPORT_SYMBOL(_printk); =20 #define prb_read_valid(rb, seq, r) false #define prb_first_valid_seq(rb) 0 +#define prb_next_seq(rb) 0 =20 static u64 syslog_seq; -static u64 console_seq; -static u64 exclusive_console_stop_seq; -static unsigned long console_dropped; =20 static size_t record_print_text(const struct printk_record *r, bool syslog, bool time) @@ -2328,8 +2297,7 @@ static ssize_t msg_print_ext_body(char *buf, size_t s= ize, struct dev_printk_info *dev_info) { return 0; } static void console_lock_spinning_enable(void) { } static int console_lock_spinning_disable_and_check(void) { return 0; } -static void call_console_drivers(const char *ext_text, size_t ext_len, - const char *text, size_t len) {} +static void call_console_driver(struct console *con, const char *text, siz= e_t len) { } static bool suppress_message_printing(int level) { return false; } =20 #endif /* CONFIG_PRINTK */ @@ -2599,22 +2567,6 @@ int is_console_locked(void) } EXPORT_SYMBOL(is_console_locked); =20 -/* - * Check if we have any console that is capable of printing while cpu is - * booting or shutting down. Requires console_sem. - */ -static int have_callable_console(void) -{ - struct console *con; - - for_each_console(con) - if ((con->flags & CON_ENABLED) && - (con->flags & CON_ANYTIME)) - return 1; - - return 0; -} - /* * Return true when this CPU should unlock console_sem without pushing all * messages to the console. This reduces the chance that the console is @@ -2635,15 +2587,173 @@ static bool abandon_console_lock_in_panic(void) } =20 /* - * Can we actually use the console at this time on this cpu? + * Check if the given console is currently capable and allowed to print + * records. + * + * Requires the console_lock. + */ +static inline bool console_is_usable(struct console *con) +{ + if (!(con->flags & CON_ENABLED)) + return false; + + if (!con->write) + return false; + + /* + * Console drivers may assume that per-cpu resources have been + * allocated. So unless they're explicitly marked as being able to + * cope (CON_ANYTIME) don't call them until this CPU is officially up. + */ + if (!cpu_online(raw_smp_processor_id()) && + !(con->flags & CON_ANYTIME)) + return false; + + return true; +} + +static void __console_unlock(void) +{ + console_locked =3D 0; + up_console_sem(); +} + +/* + * Print one record for the given console. The record printed is whatever + * record is the next available record for the given console. + * + * Requires the console_lock. + * + * Returns false if the given console has no next record to print, otherwi= se + * true. * - * Console drivers may assume that per-cpu resources have been allocated. = So - * unless they're explicitly marked as being able to cope (CON_ANYTIME) do= n't - * call them until this CPU is officially up. + * @handover will be set to true if a printk waiter has taken over the + * console_lock, in which case the caller is no longer holding the + * console_lock. Otherwise it is set to false. */ -static inline int can_use_console(void) +static bool console_emit_next_record(struct console *con, bool *handover) { - return cpu_online(raw_smp_processor_id()) || have_callable_console(); + static char ext_text[CONSOLE_EXT_LOG_MAX]; + static char text[CONSOLE_LOG_MAX]; + static int panic_console_dropped; + struct printk_info info; + struct printk_record r; + unsigned long flags; + char *write_text; + size_t len; + + prb_rec_init_rd(&r, &info, text, sizeof(text)); + + *handover =3D false; + + if (!prb_read_valid(prb, con->seq, &r)) + return false; + + if (con->seq !=3D r.info->seq) { + con->dropped +=3D r.info->seq - con->seq; + con->seq =3D r.info->seq; + if (panic_in_progress() && panic_console_dropped++ > 10) { + suppress_panic_printk =3D 1; + pr_warn_once("Too many dropped messages. Suppress messages on non-panic= CPUs to prevent livelock.\n"); + } + } + + /* Skip record that has level above the console loglevel. */ + if (suppress_message_printing(r.info->level)) { + con->seq++; + goto skip; + } + + if (con->flags & CON_EXTENDED) { + write_text =3D &ext_text[0]; + len =3D info_print_ext_header(ext_text, sizeof(ext_text), r.info); + len +=3D msg_print_ext_body(ext_text + len, sizeof(ext_text) - len, + &r.text_buf[0], r.info->text_len, &r.info->dev_info); + } else { + write_text =3D &text[0]; + len =3D record_print_text(&r, console_msg_format & MSG_FORMAT_SYSLOG, pr= intk_time); + } + + /* + * While actively printing out messages, if another printk() + * were to occur on another CPU, it may wait for this one to + * finish. This task can not be preempted if there is a + * waiter waiting to take over. + * + * Interrupts are disabled because the hand over to a waiter + * must not be interrupted until the hand over is completed + * (@console_waiter is cleared). + */ + printk_safe_enter_irqsave(flags); + console_lock_spinning_enable(); + + stop_critical_timings(); /* don't trace print latency */ + call_console_driver(con, write_text, len); + start_critical_timings(); + + con->seq++; + + *handover =3D console_lock_spinning_disable_and_check(); + printk_safe_exit_irqrestore(flags); +skip: + return true; +} + +/* + * Print out all remaining records to all consoles. + * + * Requires the console_lock. + * + * Returns true if a console was available for flushing, otherwise false. + * + * @next_seq is set to the highest sequence number of all of the consoles = that + * were flushed. + * + * @handover will be set to true if a printk waiter has taken over the + * console_lock, in which case the caller is no longer holding the + * console_lock. Otherwise it is set to false. + */ +static bool console_flush_all(bool do_cond_resched, u64 *next_seq, bool *h= andover) +{ + bool any_usable =3D false; + struct console *con; + bool any_progress; + + *next_seq =3D 0; + *handover =3D false; + + do { + any_progress =3D false; + + for_each_console(con) { + bool progress; + + if (!console_is_usable(con)) + continue; + any_usable =3D true; + + progress =3D console_emit_next_record(con, handover); + if (*handover) + return true; + + /* Track the highest seq flushed. */ + if (con->seq > *next_seq) + *next_seq =3D con->seq; + + if (!progress) + continue; + any_progress =3D true; + + /* Allow panic_cpu to take over the consoles safely */ + if (abandon_console_lock_in_panic()) + return false; + + if (do_cond_resched) + cond_resched(); + } + } while (any_progress); + + return any_usable; } =20 /** @@ -2662,22 +2772,16 @@ static inline int can_use_console(void) */ void console_unlock(void) { - static char ext_text[CONSOLE_EXT_LOG_MAX]; - static char text[CONSOLE_LOG_MAX]; - static int panic_console_dropped; - unsigned long flags; - bool do_cond_resched, retry; - struct printk_info info; - struct printk_record r; - u64 __maybe_unused next_seq; + bool do_cond_resched; + bool handover; + bool flushed; + u64 next_seq; =20 if (console_suspended) { up_console_sem(); return; } =20 - prb_rec_init_rd(&r, &info, text, sizeof(text)); - /* * Console drivers are called with interrupts disabled, so * @console_may_schedule should be cleared before; however, we may @@ -2686,125 +2790,33 @@ void console_unlock(void) * between lines if allowable. Not doing so can cause a very long * scheduling stall on a slow console leading to RCU stall and * softlockup warnings which exacerbate the issue with more - * messages practically incapacitating the system. - * - * console_trylock() is not able to detect the preemptive - * context reliably. Therefore the value must be stored before - * and cleared after the "again" goto label. + * messages practically incapacitating the system. Therefore, create + * a local to use for the printing loop. */ do_cond_resched =3D console_may_schedule; -again: - console_may_schedule =3D 0; - - /* - * We released the console_sem lock, so we need to recheck if - * cpu is online and (if not) is there at least one CON_ANYTIME - * console. - */ - if (!can_use_console()) { - console_locked =3D 0; - up_console_sem(); - return; - } =20 - for (;;) { - size_t ext_len =3D 0; - int handover; - size_t len; + do { + console_may_schedule =3D 0; =20 -skip: - if (!prb_read_valid(prb, console_seq, &r)) + flushed =3D console_flush_all(do_cond_resched, &next_seq, &handover); + if (handover) break; =20 - if (console_seq !=3D r.info->seq) { - console_dropped +=3D r.info->seq - console_seq; - console_seq =3D r.info->seq; - if (panic_in_progress() && panic_console_dropped++ > 10) { - suppress_panic_printk =3D 1; - pr_warn_once("Too many dropped messages. Suppress messages on non-pani= c CPUs to prevent livelock.\n"); - } - } - - if (suppress_message_printing(r.info->level)) { - /* - * Skip record we have buffered and already printed - * directly to the console when we received it, and - * record that has level above the console loglevel. - */ - console_seq++; - goto skip; - } - - /* Output to all consoles once old messages replayed. */ - if (unlikely(exclusive_console && - console_seq >=3D exclusive_console_stop_seq)) { - exclusive_console =3D NULL; - } + __console_unlock(); =20 - /* - * Handle extended console text first because later - * record_print_text() will modify the record buffer in-place. - */ - if (nr_ext_console_drivers) { - ext_len =3D info_print_ext_header(ext_text, - sizeof(ext_text), - r.info); - ext_len +=3D msg_print_ext_body(ext_text + ext_len, - sizeof(ext_text) - ext_len, - &r.text_buf[0], - r.info->text_len, - &r.info->dev_info); - } - len =3D record_print_text(&r, - console_msg_format & MSG_FORMAT_SYSLOG, - printk_time); - console_seq++; + /* Were there any consoles available for flushing? */ + if (!flushed) + break; =20 /* - * While actively printing out messages, if another printk() - * were to occur on another CPU, it may wait for this one to - * finish. This task can not be preempted if there is a - * waiter waiting to take over. - * - * Interrupts are disabled because the hand over to a waiter - * must not be interrupted until the hand over is completed - * (@console_waiter is cleared). + * Some context may have added new records after + * console_flush_all() but before unlocking the console. + * Re-check if there is a new record to flush. If the trylock + * fails, another context is already handling the printing. */ - printk_safe_enter_irqsave(flags); - console_lock_spinning_enable(); - - stop_critical_timings(); /* don't trace print latency */ - call_console_drivers(ext_text, ext_len, text, len); - start_critical_timings(); - - handover =3D console_lock_spinning_disable_and_check(); - printk_safe_exit_irqrestore(flags); - if (handover) - return; - - /* Allow panic_cpu to take over the consoles safely */ - if (abandon_console_lock_in_panic()) - break; - - if (do_cond_resched) - cond_resched(); - } - - /* Get consistent value of the next-to-be-used sequence number. */ - next_seq =3D console_seq; - - console_locked =3D 0; - up_console_sem(); - - /* - * Someone could have filled up the buffer again, so re-check if there's - * something to flush. In case we cannot trylock the console_sem again, - * there's a new owner and the console_unlock() from them will do the - * flush, no worries. - */ - retry =3D prb_read_valid(prb, next_seq, NULL); - if (retry && !abandon_console_lock_in_panic() && console_trylock()) - goto again; + } while (prb_read_valid(prb, next_seq, NULL) && + !abandon_console_lock_in_panic() && + console_trylock()); } EXPORT_SYMBOL(console_unlock); =20 @@ -2864,8 +2876,14 @@ void console_flush_on_panic(enum con_flush_mode mode) console_trylock(); console_may_schedule =3D 0; =20 - if (mode =3D=3D CONSOLE_REPLAY_ALL) - console_seq =3D prb_first_valid_seq(prb); + if (mode =3D=3D CONSOLE_REPLAY_ALL) { + struct console *c; + u64 seq; + + seq =3D prb_first_valid_seq(prb); + for_each_console(c) + c->seq =3D seq; + } console_unlock(); } =20 @@ -3104,26 +3122,15 @@ void register_console(struct console *newcon) if (newcon->flags & CON_EXTENDED) nr_ext_console_drivers++; =20 + newcon->dropped =3D 0; if (newcon->flags & CON_PRINTBUFFER) { - /* - * console_unlock(); will print out the buffered messages - * for us. - * - * We're about to replay the log buffer. Only do this to the - * just-registered console to avoid excessive message spam to - * the already-registered consoles. - * - * Set exclusive_console with disabled interrupts to reduce - * race window with eventual console_flush_on_panic() that - * ignores console_lock. - */ - exclusive_console =3D newcon; - exclusive_console_stop_seq =3D console_seq; - /* Get a consistent copy of @syslog_seq. */ mutex_lock(&syslog_lock); - console_seq =3D syslog_seq; + newcon->seq =3D syslog_seq; mutex_unlock(&syslog_lock); + } else { + /* Begin with next message. */ + newcon->seq =3D prb_next_seq(prb); } console_unlock(); console_sysfs_notify(); --=20 2.30.2 From nobody Fri Jun 19 09:50:04 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id D5754C46467 for ; Wed, 6 Apr 2022 00:35:12 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1835068AbiDFATO (ORCPT ); Tue, 5 Apr 2022 20:19:14 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44228 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1353831AbiDEPL6 (ORCPT ); Tue, 5 Apr 2022 11:11:58 -0400 Received: from galois.linutronix.de (Galois.linutronix.de [193.142.43.55]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 89D79109A4C for ; Tue, 5 Apr 2022 06:25:43 -0700 (PDT) From: John Ogness DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1649165140; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=A/7cUaplOmC+t/jIeiNitOokT+w2JM4UfkfA0L6jRLE=; b=GbLUs+srw/DSjlePOGXQX1kh9BgYOnt1N7m4mkwjE9WMUmakyos4fvIKw2S+CLYqxFI3Yd SybhRVTgX4vcwuh0lLwMFTQZrLbRz05uTg3cXBbAoVmuJYsAdif+kEEIU0mzphBhNQYNUj C+yolb+Fexb6l3TyxZULteztEgTuwEuBpYH5/RDHDelN4CVbQzPsS2aFTq5YlgkA6OEcND AEWOVK1sBJvyYF658hf4V8UE+OuIfIsCX2WQUJsjNKtSF9ZMIq06iZJcX2AQw77y4yxz0P /mCw1UchLh4QIPOxeucmWNUk2MZLWkpPG8SBMS3tdxBLPfLdhPJPvctjX4j/3A== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1649165140; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=A/7cUaplOmC+t/jIeiNitOokT+w2JM4UfkfA0L6jRLE=; b=nr2+XlUJRZe2weCsX7ckkrKqdZezOx1nTpyKylEJrRjEIgjQMEQP0opEskAw4y9A9ZrYgw ZJ0M0YkHryIn5iCA== To: Petr Mladek Cc: Sergey Senozhatsky , Steven Rostedt , Thomas Gleixner , linux-kernel@vger.kernel.org Subject: [PATCH printk v2 07/12] printk: move buffer definitions into console_emit_next_record() caller Date: Tue, 5 Apr 2022 15:31:30 +0206 Message-Id: <20220405132535.649171-8-john.ogness@linutronix.de> In-Reply-To: <20220405132535.649171-1-john.ogness@linutronix.de> References: <20220405132535.649171-1-john.ogness@linutronix.de> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Extended consoles print extended messages and do not print messages about dropped records. Non-extended consoles print "normal" messages as well as extra messages about dropped records. Currently the buffers for these various message types are defined within the functions that might use them and their usage is based upon the CON_EXTENDED flag. This will be a problem when moving to kthread printers because each printer must be able to provide its own buffers. Move all the message buffer definitions outside of console_emit_next_record(). The caller knows if extended or dropped messages should be printed and can specify the appropriate buffers to use. The console_emit_next_record() and call_console_driver() functions can know what to print based on whether specified buffers are non-NULL. With this change, buffer definition/allocation/specification is separated from the code that does the various types of string printing. Signed-off-by: John Ogness Reviewed-by: Petr Mladek --- kernel/printk/printk.c | 60 ++++++++++++++++++++++++++++++------------ 1 file changed, 43 insertions(+), 17 deletions(-) diff --git a/kernel/printk/printk.c b/kernel/printk/printk.c index db9278dba775..06f0b29d909d 100644 --- a/kernel/printk/printk.c +++ b/kernel/printk/printk.c @@ -394,6 +394,9 @@ static struct latched_seq clear_seq =3D { /* the maximum size of a formatted record (i.e. with prefix added per line= ) */ #define CONSOLE_LOG_MAX 1024 =20 +/* the maximum size for a dropped text message */ +#define DROPPED_TEXT_MAX 64 + /* the maximum size allowed to be reserved for a record */ #define LOG_LINE_MAX (CONSOLE_LOG_MAX - PREFIX_MAX) =20 @@ -1901,18 +1904,18 @@ static int console_trylock_spinning(void) =20 /* * Call the specified console driver, asking it to write out the specified - * text and length. For non-extended consoles, if any records have been + * text and length. If @dropped_text is non-NULL and any records have been * dropped, a dropped message will be written out first. */ -static void call_console_driver(struct console *con, const char *text, siz= e_t len) +static void call_console_driver(struct console *con, const char *text, siz= e_t len, + char *dropped_text) { - static char dropped_text[64]; size_t dropped_len; =20 trace_console_rcuidle(text, len); =20 - if (con->dropped && !(con->flags & CON_EXTENDED)) { - dropped_len =3D snprintf(dropped_text, sizeof(dropped_text), + if (con->dropped && dropped_text) { + dropped_len =3D snprintf(dropped_text, DROPPED_TEXT_MAX, "** %lu printk messages dropped **\n", con->dropped); con->dropped =3D 0; @@ -2274,6 +2277,7 @@ EXPORT_SYMBOL(_printk); #else /* CONFIG_PRINTK */ =20 #define CONSOLE_LOG_MAX 0 +#define DROPPED_TEXT_MAX 0 #define printk_time false =20 #define prb_read_valid(rb, seq, r) false @@ -2297,7 +2301,10 @@ static ssize_t msg_print_ext_body(char *buf, size_t = size, struct dev_printk_info *dev_info) { return 0; } static void console_lock_spinning_enable(void) { } static int console_lock_spinning_disable_and_check(void) { return 0; } -static void call_console_driver(struct console *con, const char *text, siz= e_t len) { } +static void call_console_driver(struct console *con, const char *text, siz= e_t len, + char *dropped_text) +{ +} static bool suppress_message_printing(int level) { return false; } =20 #endif /* CONFIG_PRINTK */ @@ -2622,6 +2629,14 @@ static void __console_unlock(void) * Print one record for the given console. The record printed is whatever * record is the next available record for the given console. * + * @text is a buffer of size CONSOLE_LOG_MAX. + * + * If extended messages should be printed, @ext_text is a buffer of size + * CONSOLE_EXT_LOG_MAX. Otherwise @ext_text must be NULL. + * + * If dropped messages should be printed, @dropped_text is a buffer of size + * DROPPED_TEXT_MAX. Otherwise @dropped_text must be NULL. + * * Requires the console_lock. * * Returns false if the given console has no next record to print, otherwi= se @@ -2631,10 +2646,9 @@ static void __console_unlock(void) * console_lock, in which case the caller is no longer holding the * console_lock. Otherwise it is set to false. */ -static bool console_emit_next_record(struct console *con, bool *handover) +static bool console_emit_next_record(struct console *con, char *text, char= *ext_text, + char *dropped_text, bool *handover) { - static char ext_text[CONSOLE_EXT_LOG_MAX]; - static char text[CONSOLE_LOG_MAX]; static int panic_console_dropped; struct printk_info info; struct printk_record r; @@ -2642,7 +2656,7 @@ static bool console_emit_next_record(struct console *= con, bool *handover) char *write_text; size_t len; =20 - prb_rec_init_rd(&r, &info, text, sizeof(text)); + prb_rec_init_rd(&r, &info, text, CONSOLE_LOG_MAX); =20 *handover =3D false; =20 @@ -2664,13 +2678,13 @@ static bool console_emit_next_record(struct console= *con, bool *handover) goto skip; } =20 - if (con->flags & CON_EXTENDED) { - write_text =3D &ext_text[0]; - len =3D info_print_ext_header(ext_text, sizeof(ext_text), r.info); - len +=3D msg_print_ext_body(ext_text + len, sizeof(ext_text) - len, + if (ext_text) { + write_text =3D ext_text; + len =3D info_print_ext_header(ext_text, CONSOLE_EXT_LOG_MAX, r.info); + len +=3D msg_print_ext_body(ext_text + len, CONSOLE_EXT_LOG_MAX - len, &r.text_buf[0], r.info->text_len, &r.info->dev_info); } else { - write_text =3D &text[0]; + write_text =3D text; len =3D record_print_text(&r, console_msg_format & MSG_FORMAT_SYSLOG, pr= intk_time); } =20 @@ -2688,7 +2702,7 @@ static bool console_emit_next_record(struct console *= con, bool *handover) console_lock_spinning_enable(); =20 stop_critical_timings(); /* don't trace print latency */ - call_console_driver(con, write_text, len); + call_console_driver(con, write_text, len, dropped_text); start_critical_timings(); =20 con->seq++; @@ -2715,6 +2729,9 @@ static bool console_emit_next_record(struct console *= con, bool *handover) */ static bool console_flush_all(bool do_cond_resched, u64 *next_seq, bool *h= andover) { + static char dropped_text[DROPPED_TEXT_MAX]; + static char ext_text[CONSOLE_EXT_LOG_MAX]; + static char text[CONSOLE_LOG_MAX]; bool any_usable =3D false; struct console *con; bool any_progress; @@ -2732,7 +2749,16 @@ static bool console_flush_all(bool do_cond_resched, = u64 *next_seq, bool *handove continue; any_usable =3D true; =20 - progress =3D console_emit_next_record(con, handover); + if (con->flags & CON_EXTENDED) { + /* Extended consoles do not print "dropped messages". */ + progress =3D console_emit_next_record(con, &text[0], + &ext_text[0], NULL, + handover); + } else { + progress =3D console_emit_next_record(con, &text[0], + NULL, &dropped_text[0], + handover); + } if (*handover) return true; =20 --=20 2.30.2 From nobody Fri Jun 19 09:50:04 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 06C05C43217 for ; Tue, 5 Apr 2022 20:57:09 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S242642AbiDEU6O (ORCPT ); Tue, 5 Apr 2022 16:58:14 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44226 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1353517AbiDEPLw (ORCPT ); Tue, 5 Apr 2022 11:11:52 -0400 Received: from galois.linutronix.de (Galois.linutronix.de [193.142.43.55]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 7E221109A49 for ; Tue, 5 Apr 2022 06:25:43 -0700 (PDT) From: John Ogness DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1649165140; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=vCqmcIsjQhjlsizLtSjuYMGPoSNu/EUhYwTtgU6s6Kw=; b=1hzmEMqB9EKNoE5JBFRlA4eMbd3igRAH45cAXctFrViaWY0hFB6V86q6tgCcEJToO+DN+7 sxo7S+yfITRySa7SwcGDeZk/X1V5BmRfNLcsHawn68tMynTVnZj9K8EsvrKlN9j5yA84kH VVWTYRE0XBSdopR31SKrZn9TDMHImZqpi6FC4bnT4WN7F2jrFiBL3Edr31ihIt0wrzpx+z WmYe+v9nH0RQmd0wAKETve5e+6Atv24gGp1JnnveM4AM+qqeeE+pgJD6B4Tq9pGzydyVW6 yixlyRzFE1lDKNouGMtQJmjA5oqUd9xAyRZ3h4NTLBdt2E5OFYBTfpAyUXV05g== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1649165140; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=vCqmcIsjQhjlsizLtSjuYMGPoSNu/EUhYwTtgU6s6Kw=; b=T7Y31PyF+fAnDahO6XXYuCQ0QKZg6HgmMFwrcy91vF9YPPLG4HDkYvVNsjBBn/UOogOLi/ MEimoKCOil/FKEAw== To: Petr Mladek Cc: Sergey Senozhatsky , Steven Rostedt , Thomas Gleixner , linux-kernel@vger.kernel.org Subject: [PATCH printk v2 08/12] printk: add pr_flush() Date: Tue, 5 Apr 2022 15:31:31 +0206 Message-Id: <20220405132535.649171-9-john.ogness@linutronix.de> In-Reply-To: <20220405132535.649171-1-john.ogness@linutronix.de> References: <20220405132535.649171-1-john.ogness@linutronix.de> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Provide a might-sleep function to allow waiting for console printers to catch up to the latest logged message. Use pr_flush() whenever it is desirable to get buffered messages printed before continuing: suspend_console(), resume_console(), console_stop(), console_start(), console_unblank(). Signed-off-by: John Ogness --- include/linux/printk.h | 7 ++++ kernel/printk/printk.c | 83 ++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 90 insertions(+) diff --git a/include/linux/printk.h b/include/linux/printk.h index b70a42f94031..091fba7283e1 100644 --- a/include/linux/printk.h +++ b/include/linux/printk.h @@ -170,6 +170,8 @@ extern void __printk_safe_exit(void); #define printk_deferred_enter __printk_safe_enter #define printk_deferred_exit __printk_safe_exit =20 +extern bool pr_flush(int timeout_ms, bool reset_on_progress); + /* * Please don't use printk_ratelimit(), because it shares ratelimiting sta= te * with all other unrelated printk_ratelimit() callsites. Instead use @@ -220,6 +222,11 @@ static inline void printk_deferred_exit(void) { } =20 +static inline bool pr_flush(int timeout_ms, bool reset_on_progress) +{ + return true; +} + static inline int printk_ratelimit(void) { return 0; diff --git a/kernel/printk/printk.c b/kernel/printk/printk.c index 06f0b29d909d..d0369afaf365 100644 --- a/kernel/printk/printk.c +++ b/kernel/printk/printk.c @@ -2274,6 +2274,8 @@ asmlinkage __visible int _printk(const char *fmt, ...) } EXPORT_SYMBOL(_printk); =20 +static bool __pr_flush(struct console *con, int timeout_ms, bool reset_on_= progress); + #else /* CONFIG_PRINTK */ =20 #define CONSOLE_LOG_MAX 0 @@ -2306,6 +2308,7 @@ static void call_console_driver(struct console *con, = const char *text, size_t le { } static bool suppress_message_printing(int level) { return false; } +static bool __pr_flush(struct console *con, int timeout_ms, bool reset_on_= progress) { return true; } =20 #endif /* CONFIG_PRINTK */ =20 @@ -2493,6 +2496,7 @@ void suspend_console(void) if (!console_suspend_enabled) return; pr_info("Suspending console(s) (use no_console_suspend to debug)\n"); + pr_flush(1000, true); console_lock(); console_suspended =3D 1; up_console_sem(); @@ -2505,6 +2509,7 @@ void resume_console(void) down_console_sem(); console_suspended =3D 0; console_unlock(); + pr_flush(1000, true); } =20 /** @@ -2882,6 +2887,9 @@ void console_unblank(void) if ((c->flags & CON_ENABLED) && c->unblank) c->unblank(); console_unlock(); + + if (!oops_in_progress) + pr_flush(1000, true); } =20 /** @@ -2940,6 +2948,7 @@ struct tty_driver *console_device(int *index) */ void console_stop(struct console *console) { + __pr_flush(console, 1000, true); console_lock(); console->flags &=3D ~CON_ENABLED; console_unlock(); @@ -2951,6 +2960,7 @@ void console_start(struct console *console) console_lock(); console->flags |=3D CON_ENABLED; console_unlock(); + __pr_flush(console, 1000, true); } EXPORT_SYMBOL(console_start); =20 @@ -3321,6 +3331,79 @@ static int __init printk_late_init(void) late_initcall(printk_late_init); =20 #if defined CONFIG_PRINTK +/* If @con is specified, only wait for that console. Otherwise wait for al= l. */ +static bool __pr_flush(struct console *con, int timeout_ms, bool reset_on_= progress) +{ + int remaining =3D timeout_ms; + struct console *c; + u64 last_diff =3D 0; + u64 printk_seq; + u64 diff; + u64 seq; + + might_sleep(); + + seq =3D prb_next_seq(prb); + + for (;;) { + diff =3D 0; + + console_lock(); + for_each_console(c) { + if (con && con !=3D c) + continue; + if (!console_is_usable(c)) + continue; + printk_seq =3D c->seq; + if (printk_seq < seq) + diff +=3D seq - printk_seq; + } + console_unlock(); + + if (diff !=3D last_diff && reset_on_progress) + remaining =3D timeout_ms; + + if (diff =3D=3D 0 || remaining =3D=3D 0) + break; + + if (remaining < 0) { + /* no timeout limit */ + msleep(100); + } else if (remaining < 100) { + msleep(remaining); + remaining =3D 0; + } else { + msleep(100); + remaining -=3D 100; + } + + last_diff =3D diff; + } + + return (diff =3D=3D 0); +} + +/** + * pr_flush() - Wait for printing threads to catch up. + * + * @timeout_ms: The maximum time (in ms) to wait. + * @reset_on_progress: Reset the timeout if forward progress is seen. + * + * A value of 0 for @timeout_ms means no waiting will occur. A value of -1 + * represents infinite waiting. + * + * If @reset_on_progress is true, the timeout will be reset whenever any + * printer has been seen to make some forward progress. + * + * Context: Process context. May sleep while acquiring console lock. + * Return: true if all enabled printers are caught up. + */ +bool pr_flush(int timeout_ms, bool reset_on_progress) +{ + return __pr_flush(NULL, timeout_ms, reset_on_progress); +} +EXPORT_SYMBOL(pr_flush); + /* * Delayed printk version, for scheduler-internal messages: */ --=20 2.30.2 From nobody Fri Jun 19 09:50:04 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8D25BC352A1 for ; Tue, 5 Apr 2022 21:37:51 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1382385AbiDEVis (ORCPT ); Tue, 5 Apr 2022 17:38:48 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:56182 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1354062AbiDEPL6 (ORCPT ); Tue, 5 Apr 2022 11:11:58 -0400 Received: from galois.linutronix.de (Galois.linutronix.de [IPv6:2a0a:51c0:0:12e:550::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A8296109A4F; Tue, 5 Apr 2022 06:25:43 -0700 (PDT) From: John Ogness DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1649165141; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=TZOPBfeuilAZ8Xu/MR72GQq/MIyradbJhRuauO+T6Uo=; b=267EUGEryvP3VLlpF8ML/M6EmwsWS9+a5uuw+c4ljg7GyT+cglymNX4ksNhPUvE2ymF3F8 D6T/hbpXQqPnXqQnDngv8CTWEXXuyScGup97jqcUe+VSOa8iXevqad22ZrmHCTmVSUuIb5 VxYrXBgcA/Zdv/1dUHDffwB2Ktulb26qj9zhWFOqT514a9w/CW0yD5wV2BoN6rDqIgE9jQ gPUk/v3IspxobpGBcEp5G+35sQ7byKI950DaO6CCBy/HBjt0xUNwAV2J1jhJhrUp0Rur/9 zjeydRktoPUEhdVII+R0zUnvZdsb2uAXgCgLutpmmaPIs5/qfetbtKf20ptmWg== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1649165141; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=TZOPBfeuilAZ8Xu/MR72GQq/MIyradbJhRuauO+T6Uo=; b=HdHKKDoJ1oPkzPXXy6/c5bAP/oUiRrs0iQ5QKRlPrm7w1qynY+jIibItCzwfGZyH8UNKGY 3QZROXpVuHG9ZBAg== To: Petr Mladek Cc: Sergey Senozhatsky , Steven Rostedt , Thomas Gleixner , linux-kernel@vger.kernel.org, Greg Kroah-Hartman , Jiri Slaby , "Paul E. McKenney" , Frederic Weisbecker , Neeraj Upadhyay , Josh Triplett , Mathieu Desnoyers , Lai Jiangshan , Joel Fernandes , Kees Cook , Andrew Morton , Luis Chamberlain , Wei Liu , Xiaoming Ni , Peter Zijlstra , Andy Shevchenko , Helge Deller , Marco Elver , Sebastian Andrzej Siewior , Daniel Lezcano , Shawn Guo , Dmitry Torokhov , Matti Vaittinen , "Eric W. Biederman" , Phil Auld , Juri Lelli , Wang Qing , rcu@vger.kernel.org Subject: [PATCH printk v2 09/12] printk: add functions to prefer direct printing Date: Tue, 5 Apr 2022 15:31:32 +0206 Message-Id: <20220405132535.649171-10-john.ogness@linutronix.de> In-Reply-To: <20220405132535.649171-1-john.ogness@linutronix.de> References: <20220405132535.649171-1-john.ogness@linutronix.de> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Once kthread printing is available, console printing will no longer occur in the context of the printk caller. However, there are some special contexts where it is desirable for the printk caller to directly print out kernel messages. Using pr_flush() to wait for threaded printers is only possible if the caller is in a sleepable context and the kthreads are active. That is not always the case. Introduce printk_prefer_direct_enter() and printk_prefer_direct_exit() functions to explicitly (and globally) activate/deactivate preferred direct console printing. The term "direct console printing" refers to printing to all enabled consoles from the context of the printk caller. The term "prefer" is used because this type of printing is only best effort. If the console is currently locked or other printers are already actively printing, the printk caller will need to rely on the other contexts to handle the printing. This preferred direct printing is how all printing is currently handled (unless explicitly deferred). When kthread printing is introduced, there may be some unanticipated problems due to kthreads being unable to flush important messages. In order to minimize such risks, preferred direct printing is activated for the primary important messages when the system experiences general types of major errors. These are: - emergency reboot/shutdown - cpu and rcu stalls - hard and soft lockups - hung tasks - warn - sysrq Note that since kthread printing does not yet exist, no behavior changes result from this commit. This is only implementing the counter and marking the various places where preferred direct printing is active. Signed-off-by: John Ogness Acked-by: Paul E. McKenney Reviewed-by: Petr Mladek --- drivers/tty/sysrq.c | 2 ++ include/linux/printk.h | 11 +++++++++++ kernel/hung_task.c | 11 ++++++++++- kernel/panic.c | 4 ++++ kernel/printk/printk.c | 28 ++++++++++++++++++++++++++++ kernel/rcu/tree_stall.h | 2 ++ kernel/reboot.c | 14 +++++++++++++- kernel/watchdog.c | 4 ++++ kernel/watchdog_hld.c | 4 ++++ 9 files changed, 78 insertions(+), 2 deletions(-) diff --git a/drivers/tty/sysrq.c b/drivers/tty/sysrq.c index bbfd004449b5..2884cd638d64 100644 --- a/drivers/tty/sysrq.c +++ b/drivers/tty/sysrq.c @@ -578,6 +578,7 @@ void __handle_sysrq(int key, bool check_mask) =20 rcu_sysrq_start(); rcu_read_lock(); + printk_prefer_direct_enter(); /* * Raise the apparent loglevel to maximum so that the sysrq header * is shown to provide the user with positive feedback. We do not @@ -619,6 +620,7 @@ void __handle_sysrq(int key, bool check_mask) pr_cont("\n"); console_loglevel =3D orig_log_level; } + printk_prefer_direct_exit(); rcu_read_unlock(); rcu_sysrq_end(); =20 diff --git a/include/linux/printk.h b/include/linux/printk.h index 091fba7283e1..cd26aab0ab2a 100644 --- a/include/linux/printk.h +++ b/include/linux/printk.h @@ -170,6 +170,9 @@ extern void __printk_safe_exit(void); #define printk_deferred_enter __printk_safe_enter #define printk_deferred_exit __printk_safe_exit =20 +extern void printk_prefer_direct_enter(void); +extern void printk_prefer_direct_exit(void); + extern bool pr_flush(int timeout_ms, bool reset_on_progress); =20 /* @@ -222,6 +225,14 @@ static inline void printk_deferred_exit(void) { } =20 +static inline void printk_prefer_direct_enter(void) +{ +} + +static inline void printk_prefer_direct_exit(void) +{ +} + static inline bool pr_flush(int timeout_ms, bool reset_on_progress) { return true; diff --git a/kernel/hung_task.c b/kernel/hung_task.c index 52501e5f7655..02a65d554340 100644 --- a/kernel/hung_task.c +++ b/kernel/hung_task.c @@ -127,6 +127,8 @@ static void check_hung_task(struct task_struct *t, unsi= gned long timeout) * complain: */ if (sysctl_hung_task_warnings) { + printk_prefer_direct_enter(); + if (sysctl_hung_task_warnings > 0) sysctl_hung_task_warnings--; pr_err("INFO: task %s:%d blocked for more than %ld seconds.\n", @@ -142,6 +144,8 @@ static void check_hung_task(struct task_struct *t, unsi= gned long timeout) =20 if (sysctl_hung_task_all_cpu_backtrace) hung_task_show_all_bt =3D true; + + printk_prefer_direct_exit(); } =20 touch_nmi_watchdog(); @@ -204,12 +208,17 @@ static void check_hung_uninterruptible_tasks(unsigned= long timeout) } unlock: rcu_read_unlock(); - if (hung_task_show_lock) + if (hung_task_show_lock) { + printk_prefer_direct_enter(); debug_show_all_locks(); + printk_prefer_direct_exit(); + } =20 if (hung_task_show_all_bt) { hung_task_show_all_bt =3D false; + printk_prefer_direct_enter(); trigger_all_cpu_backtrace(); + printk_prefer_direct_exit(); } =20 if (hung_task_call_panic) diff --git a/kernel/panic.c b/kernel/panic.c index 55b50e052ec3..7d422597403f 100644 --- a/kernel/panic.c +++ b/kernel/panic.c @@ -560,6 +560,8 @@ void __warn(const char *file, int line, void *caller, u= nsigned taint, { disable_trace_on_warning(); =20 + printk_prefer_direct_enter(); + if (file) pr_warn("WARNING: CPU: %d PID: %d at %s:%d %pS\n", raw_smp_processor_id(), current->pid, file, line, @@ -597,6 +599,8 @@ void __warn(const char *file, int line, void *caller, u= nsigned taint, =20 /* Just a warning, don't kill lockdep. */ add_taint(taint, LOCKDEP_STILL_OK); + + printk_prefer_direct_exit(); } =20 #ifndef __WARN_FLAGS diff --git a/kernel/printk/printk.c b/kernel/printk/printk.c index d0369afaf365..258d02cff140 100644 --- a/kernel/printk/printk.c +++ b/kernel/printk/printk.c @@ -362,6 +362,34 @@ static int console_msg_format =3D MSG_FORMAT_DEFAULT; static DEFINE_MUTEX(syslog_lock); =20 #ifdef CONFIG_PRINTK +static atomic_t printk_prefer_direct =3D ATOMIC_INIT(0); + +/** + * printk_prefer_direct_enter - cause printk() calls to attempt direct + * printing to all enabled consoles + * + * Since it is not possible to call into the console printing code from any + * context, there is no guarantee that direct printing will occur. + * + * This globally effects all printk() callers. + * + * Context: Any context. + */ +void printk_prefer_direct_enter(void) +{ + atomic_inc(&printk_prefer_direct); +} + +/** + * printk_prefer_direct_exit - restore printk() behavior + * + * Context: Any context. + */ +void printk_prefer_direct_exit(void) +{ + WARN_ON(atomic_dec_if_positive(&printk_prefer_direct) < 0); +} + DECLARE_WAIT_QUEUE_HEAD(log_wait); /* All 3 protected by @syslog_lock. */ /* the next printk record to read by syslog(READ) or /proc/kmsg */ diff --git a/kernel/rcu/tree_stall.h b/kernel/rcu/tree_stall.h index 0c5d8516516a..d612707c2ed0 100644 --- a/kernel/rcu/tree_stall.h +++ b/kernel/rcu/tree_stall.h @@ -619,6 +619,7 @@ static void print_cpu_stall(unsigned long gps) * See Documentation/RCU/stallwarn.rst for info on how to debug * RCU CPU stall warnings. */ + printk_prefer_direct_enter(); trace_rcu_stall_warning(rcu_state.name, TPS("SelfDetected")); pr_err("INFO: %s self-detected stall on CPU\n", rcu_state.name); raw_spin_lock_irqsave_rcu_node(rdp->mynode, flags); @@ -656,6 +657,7 @@ static void print_cpu_stall(unsigned long gps) */ set_tsk_need_resched(current); set_preempt_need_resched(); + printk_prefer_direct_exit(); } =20 static void check_cpu_stall(struct rcu_data *rdp) diff --git a/kernel/reboot.c b/kernel/reboot.c index 6bcc5d6a6572..4177645e74d6 100644 --- a/kernel/reboot.c +++ b/kernel/reboot.c @@ -447,9 +447,11 @@ static int __orderly_reboot(void) ret =3D run_cmd(reboot_cmd); =20 if (ret) { + printk_prefer_direct_enter(); pr_warn("Failed to start orderly reboot: forcing the issue\n"); emergency_sync(); kernel_restart(NULL); + printk_prefer_direct_exit(); } =20 return ret; @@ -462,6 +464,7 @@ static int __orderly_poweroff(bool force) ret =3D run_cmd(poweroff_cmd); =20 if (ret && force) { + printk_prefer_direct_enter(); pr_warn("Failed to start orderly shutdown: forcing the issue\n"); =20 /* @@ -471,6 +474,7 @@ static int __orderly_poweroff(bool force) */ emergency_sync(); kernel_power_off(); + printk_prefer_direct_exit(); } =20 return ret; @@ -528,6 +532,8 @@ EXPORT_SYMBOL_GPL(orderly_reboot); */ static void hw_failure_emergency_poweroff_func(struct work_struct *work) { + printk_prefer_direct_enter(); + /* * We have reached here after the emergency shutdown waiting period has * expired. This means orderly_poweroff has not been able to shut off @@ -544,6 +550,8 @@ static void hw_failure_emergency_poweroff_func(struct w= ork_struct *work) */ pr_emerg("Hardware protection shutdown failed. Trying emergency restart\n= "); emergency_restart(); + + printk_prefer_direct_exit(); } =20 static DECLARE_DELAYED_WORK(hw_failure_emergency_poweroff_work, @@ -582,11 +590,13 @@ void hw_protection_shutdown(const char *reason, int m= s_until_forced) { static atomic_t allow_proceed =3D ATOMIC_INIT(1); =20 + printk_prefer_direct_enter(); + pr_emerg("HARDWARE PROTECTION shutdown (%s)\n", reason); =20 /* Shutdown should be initiated only once. */ if (!atomic_dec_and_test(&allow_proceed)) - return; + goto out; =20 /* * Queue a backup emergency shutdown in the event of @@ -594,6 +604,8 @@ void hw_protection_shutdown(const char *reason, int ms_= until_forced) */ hw_failure_emergency_poweroff(ms_until_forced); orderly_poweroff(true); +out: + printk_prefer_direct_exit(); } EXPORT_SYMBOL_GPL(hw_protection_shutdown); =20 diff --git a/kernel/watchdog.c b/kernel/watchdog.c index 9166220457bc..40024e03d422 100644 --- a/kernel/watchdog.c +++ b/kernel/watchdog.c @@ -424,6 +424,8 @@ static enum hrtimer_restart watchdog_timer_fn(struct hr= timer *hrtimer) /* Start period for the next softlockup warning. */ update_report_ts(); =20 + printk_prefer_direct_enter(); + pr_emerg("BUG: soft lockup - CPU#%d stuck for %us! [%s:%d]\n", smp_processor_id(), duration, current->comm, task_pid_nr(current)); @@ -442,6 +444,8 @@ static enum hrtimer_restart watchdog_timer_fn(struct hr= timer *hrtimer) add_taint(TAINT_SOFTLOCKUP, LOCKDEP_STILL_OK); if (softlockup_panic) panic("softlockup: hung tasks"); + + printk_prefer_direct_exit(); } =20 return HRTIMER_RESTART; diff --git a/kernel/watchdog_hld.c b/kernel/watchdog_hld.c index 247bf0b1582c..701f35f0e2d4 100644 --- a/kernel/watchdog_hld.c +++ b/kernel/watchdog_hld.c @@ -135,6 +135,8 @@ static void watchdog_overflow_callback(struct perf_even= t *event, if (__this_cpu_read(hard_watchdog_warn) =3D=3D true) return; =20 + printk_prefer_direct_enter(); + pr_emerg("Watchdog detected hard LOCKUP on cpu %d\n", this_cpu); print_modules(); @@ -155,6 +157,8 @@ static void watchdog_overflow_callback(struct perf_even= t *event, if (hardlockup_panic) nmi_panic(regs, "Hard LOCKUP"); =20 + printk_prefer_direct_exit(); + __this_cpu_write(hard_watchdog_warn, true); return; } --=20 2.30.2 From nobody Fri Jun 19 09:50:04 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6B5A7C433FE for ; Wed, 6 Apr 2022 00:51:44 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1389856AbiDFAxA (ORCPT ); Tue, 5 Apr 2022 20:53:00 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:45204 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1354338AbiDEPL6 (ORCPT ); Tue, 5 Apr 2022 11:11:58 -0400 Received: from galois.linutronix.de (Galois.linutronix.de [IPv6:2a0a:51c0:0:12e:550::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E4F52109A58 for ; Tue, 5 Apr 2022 06:25:43 -0700 (PDT) From: John Ogness DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1649165142; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=T9nXW5eDdRswHPPVMdRqznswZ+PeANL457bEvSUO9rY=; b=wbUXP3BlFV9pkCs967YQq/HESNZkYjif3wDfX90/TSqxB1i1Bd2zHCilPmBRYTxuYuVZBA Pr/vcptSi9SJjkyC4u/+3traDQr3GobNt5pF0U+u5b195NqJHmzvgqQ90liMzRSKNg2KZ8 6CRUQm8iOLVgmQLcijM8WGt9bmO2poctKB8XRXGWIkGfpZI0VCIV8xdTp7xoTNuu3Ou/On r9g9nVyUYWoaCAfwMwmaM5FOriXMWJFHmlKsH7hjQbKUqKorLKPKAvxPrCGzKG79haSec7 riMZMZp6UXvnaJTpOwodOVr9s0HTjpNQFBPT4VzRaoLBKkRAf+C+LoLU1XK9JA== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1649165142; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=T9nXW5eDdRswHPPVMdRqznswZ+PeANL457bEvSUO9rY=; b=Uvu9ZXlswmTEzarCepu12m1/D6Jy6QGJ+0sfmzmsY7Kb8RePRnW4BHJiFAUdainn6dpt8a VM+zPfqywWgrdFAw== To: Petr Mladek Cc: Sergey Senozhatsky , Steven Rostedt , Thomas Gleixner , linux-kernel@vger.kernel.org, Greg Kroah-Hartman Subject: [PATCH printk v2 10/12] printk: add kthread console printers Date: Tue, 5 Apr 2022 15:31:33 +0206 Message-Id: <20220405132535.649171-11-john.ogness@linutronix.de> In-Reply-To: <20220405132535.649171-1-john.ogness@linutronix.de> References: <20220405132535.649171-1-john.ogness@linutronix.de> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Create a kthread for each console to perform console printing. During normal operation (@system_state =3D=3D SYSTEM_RUNNING), the kthread printers are responsible for all printing on their respective consoles. During non-normal operation, console printing is done as it has been: within the context of the printk caller or within irq work triggered by the printk caller. Console printers synchronize against each other and against console lockers by taking the console lock for each message that is printed. Signed-off-by: John Ogness --- include/linux/console.h | 2 + kernel/printk/printk.c | 262 ++++++++++++++++++++++++++++++++++++++-- 2 files changed, 252 insertions(+), 12 deletions(-) diff --git a/include/linux/console.h b/include/linux/console.h index 8c1686e2c233..9a251e70c090 100644 --- a/include/linux/console.h +++ b/include/linux/console.h @@ -153,6 +153,8 @@ struct console { uint ospeed; u64 seq; unsigned long dropped; + struct task_struct *thread; + void *data; struct console *next; }; diff --git a/kernel/printk/printk.c b/kernel/printk/printk.c index 258d02cff140..a3b8b984c9aa 100644 --- a/kernel/printk/printk.c +++ b/kernel/printk/printk.c @@ -361,6 +361,13 @@ static int console_msg_format =3D MSG_FORMAT_DEFAULT; /* syslog_lock protects syslog_* variables and write access to clear_seq. = */ static DEFINE_MUTEX(syslog_lock); =20 +/* + * A flag to signify if printk_activate_kthreads() has already started the + * kthread printers. If true, any later registered consoles must start the= ir + * own kthread directly. The flag is write protected by the console_lock. + */ +static bool printk_kthreads_available; + #ifdef CONFIG_PRINTK static atomic_t printk_prefer_direct =3D ATOMIC_INIT(0); =20 @@ -388,6 +395,21 @@ void printk_prefer_direct_enter(void) void printk_prefer_direct_exit(void) { WARN_ON(atomic_dec_if_positive(&printk_prefer_direct) < 0); + + /* + * Threaded printers remain asleep while preferred direct printing is + * active. Explicitly wake them in case any new printk records became + * visible after preferred direct printing was deactivated. + */ + wake_up_klogd(); +} + +static inline bool allow_direct_printing(void) +{ + return (!printk_kthreads_available || + system_state > SYSTEM_RUNNING || + oops_in_progress || + atomic_read(&printk_prefer_direct)); } =20 DECLARE_WAIT_QUEUE_HEAD(log_wait); @@ -2258,10 +2280,10 @@ asmlinkage int vprintk_emit(int facility, int level, printed_len =3D vprintk_store(facility, level, dev_info, fmt, args); =20 /* If called from the scheduler, we can not call up(). */ - if (!in_sched) { + if (!in_sched && allow_direct_printing()) { /* * The caller may be holding system-critical or - * timing-sensitive locks. Disable preemption during + * timing-sensitive locks. Disable preemption during direct * printing of all remaining records to all consoles so that * this context can return as soon as possible. Hopefully * another printk() caller will take over the printing. @@ -2304,6 +2326,8 @@ EXPORT_SYMBOL(_printk); =20 static bool __pr_flush(struct console *con, int timeout_ms, bool reset_on_= progress); =20 +static void printk_start_kthread(struct console *con); + #else /* CONFIG_PRINTK */ =20 #define CONSOLE_LOG_MAX 0 @@ -2337,6 +2361,8 @@ static void call_console_driver(struct console *con, = const char *text, size_t le } static bool suppress_message_printing(int level) { return false; } static bool __pr_flush(struct console *con, int timeout_ms, bool reset_on_= progress) { return true; } +static void printk_start_kthread(struct console *con) { } +static bool allow_direct_printing(void) { return true; } =20 #endif /* CONFIG_PRINTK */ =20 @@ -2537,6 +2563,13 @@ void resume_console(void) down_console_sem(); console_suspended =3D 0; console_unlock(); + + /* + * While suspended, new records may have been added to the + * ringbuffer. Wake up the kthread printers to print them. + */ + wake_up_klogd(); + pr_flush(1000, true); } =20 @@ -2555,6 +2588,9 @@ static int console_cpu_notify(unsigned int cpu) /* If trylock fails, someone else is doing the printing */ if (console_trylock()) console_unlock(); + + /* Wake kthread printers. Some may have become usable. */ + wake_up_klogd(); } return 0; } @@ -2773,6 +2809,10 @@ static bool console_flush_all(bool do_cond_resched, = u64 *next_seq, bool *handove *handover =3D false; =20 do { + /* Let the kthread printers do the work if they can. */ + if (!allow_direct_printing()) + break; + any_progress =3D false; =20 for_each_console(con) { @@ -2988,6 +3028,10 @@ void console_start(struct console *console) console_lock(); console->flags |=3D CON_ENABLED; console_unlock(); + + /* Wake the newly enabled kthread printer. */ + wake_up_klogd(); + __pr_flush(console, 1000, true); } EXPORT_SYMBOL(console_start); @@ -3196,6 +3240,8 @@ void register_console(struct console *newcon) /* Begin with next message. */ newcon->seq =3D prb_next_seq(prb); } + if (printk_kthreads_available) + printk_start_kthread(newcon); console_unlock(); console_sysfs_notify(); =20 @@ -3222,6 +3268,7 @@ EXPORT_SYMBOL(register_console); =20 int unregister_console(struct console *console) { + struct task_struct *thd =3D console->thread; struct console *con; int res; =20 @@ -3233,6 +3280,11 @@ int unregister_console(struct console *console) if (res > 0) return 0; =20 + if (thd) { + console->thread =3D NULL; + kthread_stop(thd); + } + res =3D -ENODEV; console_lock(); if (console_drivers =3D=3D console) { @@ -3358,6 +3410,20 @@ static int __init printk_late_init(void) } late_initcall(printk_late_init); =20 +static int __init printk_activate_kthreads(void) +{ + struct console *con; + + console_lock(); + printk_kthreads_available =3D true; + for_each_console(con) + printk_start_kthread(con); + console_unlock(); + + return 0; +} +early_initcall(printk_activate_kthreads); + #if defined CONFIG_PRINTK /* If @con is specified, only wait for that console. Otherwise wait for al= l. */ static bool __pr_flush(struct console *con, int timeout_ms, bool reset_on_= progress) @@ -3432,6 +3498,154 @@ bool pr_flush(int timeout_ms, bool reset_on_progres= s) } EXPORT_SYMBOL(pr_flush); =20 +/* + * Enter preferred direct printing, but never exit. Mark console threads as + * unavailable. The system is then forever in preferred direct printing and + * any printing threads will exit. + */ +static void printk_fallback_preferred_direct(void) +{ + printk_prefer_direct_enter(); + pr_err("falling back to preferred direct printing\n"); + printk_kthreads_available =3D false; +} + +static bool printer_should_wake(struct console *con, u64 seq) +{ + short flags; + + if (kthread_should_stop() || !printk_kthreads_available) + return true; + + if (console_suspended) + return false; + + if (!con->write) + return false; + + /* + * This is an unsafe read to con->flags, but a false positive is not + * a problem. Worst case it would allow the printer to wake up even + * when it is disabled. But the printer will notice that itself when + * attempting to print and instead go back to sleep. + */ + flags =3D data_race(READ_ONCE(con->flags)); + if (!(flags & CON_ENABLED)) + return false; + + if (atomic_read(&printk_prefer_direct)) + return false; + + return prb_read_valid(prb, seq, NULL); +} + +static int printk_kthread_func(void *data) +{ + struct console *con =3D data; + char *dropped_text =3D NULL; + char *ext_text =3D NULL; + bool handover; + u64 seq =3D 0; + char *text; + int error; + + text =3D kmalloc(CONSOLE_LOG_MAX, GFP_KERNEL); + if (!text) { + printk_console_msg(con, KERN_ERR, "failed to allocate text buffer"); + printk_fallback_preferred_direct(); + goto out; + } + + if (con->flags & CON_EXTENDED) { + ext_text =3D kmalloc(CONSOLE_EXT_LOG_MAX, GFP_KERNEL); + if (!ext_text) { + printk_console_msg(con, KERN_ERR, "failed to allocate ext_text buffer"); + printk_fallback_preferred_direct(); + goto out; + } + } else { + dropped_text =3D kmalloc(DROPPED_TEXT_MAX, GFP_KERNEL); + if (!dropped_text) { + printk_console_msg(con, KERN_ERR, + "failed to allocate dropped_text buffer"); + printk_fallback_preferred_direct(); + goto out; + } + } + + printk_console_msg(con, KERN_INFO, "printing thread started"); + + for (;;) { + /* + * Guarantee this task is visible on the waitqueue before + * checking the wake condition. + * + * The full memory barrier within set_current_state() of + * prepare_to_wait_event() pairs with the full memory barrier + * within wq_has_sleeper(). + * + * See __wake_up_klogd:A for the pairing memory barrier. + */ + error =3D wait_event_interruptible(log_wait, + printer_should_wake(con, seq)); /* LMM(printk_kthread_func:A) */ + + if (kthread_should_stop() || !printk_kthreads_available) + break; + + if (error) + continue; + + console_lock(); + + if (console_suspended) { + __console_unlock(); + continue; + } + + if (!console_is_usable(con)) { + __console_unlock(); + continue; + } + + /* + * Even though the printk kthread is always preemptible, it is + * still not allowed to call cond_resched() from within + * console drivers. The task may become non-preemptible in the + * console driver call chain. For example, vt_console_print() + * takes a spinlock and then can call into fbcon_redraw(), + * which can conditionally invoke cond_resched(). + */ + console_may_schedule =3D 0; + console_emit_next_record(con, text, ext_text, dropped_text, &handover); + if (handover) + continue; + + seq =3D con->seq; + + __console_unlock(); + } + + printk_console_msg(con, KERN_INFO, "printing thread stopped"); +out: + kfree(dropped_text); + kfree(ext_text); + kfree(text); + return 0; +} + +/* Must be called within console_lock(). */ +static void printk_start_kthread(struct console *con) +{ + con->thread =3D kthread_run(printk_kthread_func, con, + "pr/%s%d", con->name, con->index); + if (IS_ERR(con->thread)) { + con->thread =3D NULL; + printk_console_msg(con, KERN_ERR, "unable to start printing thread"); + printk_fallback_preferred_direct(); + return; + } +} + /* * Delayed printk version, for scheduler-internal messages: */ @@ -3445,40 +3659,64 @@ static void wake_up_klogd_work_func(struct irq_work= *irq_work) int pending =3D this_cpu_xchg(printk_pending, 0); =20 if (pending & PRINTK_PENDING_OUTPUT) { + printk_prefer_direct_enter(); + /* If trylock fails, someone else is doing the printing */ if (console_trylock()) console_unlock(); + + printk_prefer_direct_exit(); } =20 if (pending & PRINTK_PENDING_WAKEUP) - wake_up_interruptible(&log_wait); + wake_up_interruptible_all(&log_wait); } =20 static DEFINE_PER_CPU(struct irq_work, wake_up_klogd_work) =3D IRQ_WORK_INIT_LAZY(wake_up_klogd_work_func); =20 -void wake_up_klogd(void) +static void __wake_up_klogd(int val) { if (!printk_percpu_data_ready()) return; =20 preempt_disable(); - if (waitqueue_active(&log_wait)) { - this_cpu_or(printk_pending, PRINTK_PENDING_WAKEUP); + /* + * Guarantee any new records can be seen by printing threads before + * checking if the wait queue is empty. + * + * The full memory barrier within wq_has_sleeper() pairs with the full + * memory barrier within set_current_state() of + * prepare_to_wait_event(), which is called after ___wait_event() adds + * the waiter but before it has checked the wait condition. + * + * See printk_kthread_func:A for the pairing memory barrier. + */ + if (wq_has_sleeper(&log_wait) || /* LMM(__wake_up_klogd:A) */ + (val & PRINTK_PENDING_OUTPUT)) { + this_cpu_or(printk_pending, val); irq_work_queue(this_cpu_ptr(&wake_up_klogd_work)); } preempt_enable(); } =20 +void wake_up_klogd(void) +{ + __wake_up_klogd(PRINTK_PENDING_WAKEUP); +} + void defer_console_output(void) { - if (!printk_percpu_data_ready()) - return; + int val =3D PRINTK_PENDING_WAKEUP; =20 - preempt_disable(); - this_cpu_or(printk_pending, PRINTK_PENDING_OUTPUT); - irq_work_queue(this_cpu_ptr(&wake_up_klogd_work)); - preempt_enable(); + /* + * If console deferring was called with preferred direct printing, + * make the irqwork perform the direct printing. + */ + if (atomic_read(&printk_prefer_direct)) + val |=3D PRINTK_PENDING_OUTPUT; + + __wake_up_klogd(val); } =20 void printk_trigger_flush(void) --=20 2.30.2 From nobody Fri Jun 19 09:50:04 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 83D4BC47081 for ; Tue, 5 Apr 2022 23:42:56 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1579265AbiDEX0u (ORCPT ); Tue, 5 Apr 2022 19:26:50 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42350 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1354812AbiDEPML (ORCPT ); Tue, 5 Apr 2022 11:12:11 -0400 Received: from galois.linutronix.de (Galois.linutronix.de [IPv6:2a0a:51c0:0:12e:550::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 55A71109A65 for ; Tue, 5 Apr 2022 06:25:44 -0700 (PDT) From: John Ogness DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1649165142; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=B2yzUBrKob8fBHsOxXzklUpPd/Gj/0aKkS2L92trxYY=; b=z2MktqKTFwZaAVyXXLoLdnZ3wkFkp1YUMl4TluUenm2Dxbj0W0ugExzlx5jKIupy4FwSqm aJoyb8YXAx8X3j9MbXdoZ2rjrB52KRKs8mYDTUPyJ15NEzhWWTh84nw3GpwBu9j83eKmAj B7JGgc013mPyTSgOYOgAqPDJc4MREGkOcuFbbA5cATjYsEXY/taGvwN56OHYel2Q3c5wff PyUpVvNbXqYj0noXrAQtn8vBssyCalQaKHTjg5PiDqCx3I14CyRj9ORFt9ZOWW+n51XddC nC5fPMruyJe/fkNFtCjYZ4JJ0sp/MisZrBBaMLCIsatc0wbrIfCP4VZJnb/FHg== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1649165142; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=B2yzUBrKob8fBHsOxXzklUpPd/Gj/0aKkS2L92trxYY=; b=y2AixWH6nKhuZi9w+bMJdrJGsWb/CJG9MCm5zXWoau6jdJfpqDsuwiUkxleGffCo4OrD/S 8q7A9xecC2zO/HCw== To: Petr Mladek Cc: Sergey Senozhatsky , Steven Rostedt , Thomas Gleixner , linux-kernel@vger.kernel.org, Greg Kroah-Hartman Subject: [PATCH printk v2 11/12] printk: extend console_lock for proper kthread support Date: Tue, 5 Apr 2022 15:31:34 +0206 Message-Id: <20220405132535.649171-12-john.ogness@linutronix.de> In-Reply-To: <20220405132535.649171-1-john.ogness@linutronix.de> References: <20220405132535.649171-1-john.ogness@linutronix.de> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Currently threaded console printers synchronize against each other using console_lock(). However, different console drivers are unrelated and do not require any synchronization between each other. Removing the synchronization between the threaded console printers will allow each console to print at its own speed. But the threaded consoles printers do still need to synchronize against console_lock() callers. Introduce a per-console mutex and a new console flag CON_THD_BLOCKED to provide this synchronization. console_lock() is modified so that it must acquire the mutex of each console in order to set the CON_THD_BLOCKED flag. Console printing threads will acquire their mutex while printing a record. If CON_THD_BLOCKED was set, the thread will go back to sleep instead of printing. The reason for the CON_THD_BLOCKED flag is so that console_lock() callers do not need to acquire multiple console mutexes simultaneously, which would introduce unnecessary complexity due to nested mutex locking. However, threaded console printers also need to synchronize against console_trylock() callers. Since console_trylock() may be called from any context, the per-console mutex cannot be used for this synchronization. (mutex_trylock() cannot be called from atomic contexts.) Introduce a global atomic counter to identify if any threaded printers are active. The threaded printers will also check the atomic counter to identify if the console has been locked by another task via console_trylock(). Note that @console_sem is still used to provide synchronization between console_lock() and console_trylock() callers. A locking overview for console_lock(), console_trylock(), and the threaded printers is as follows (pseudo code): console_lock() { down(&console_sem); for_each_console(con) { mutex_lock(&con->lock); con->flags |=3D CON_THD_BLOCKED; mutex_unlock(&con->lock); } /* console lock acquired */ } console_trylock() { if (down_trylock(&console_sem) =3D=3D 0) { if (atomic_cmpxchg(&console_kthreads_active, 0, -1) =3D=3D = 0) { /* console lock acquired */ } } } threaded_printer() { mutex_lock(&con->lock); if (!(con->flags & CON_THD_BLOCKED)) { /* console_lock() callers blocked */ if (atomic_inc_unless_negative(&console_kthreads_active)) { /* console_trylock() callers blocked */ con->write(); atomic_dec(&console_lock_count); } } mutex_unlock(&con->lock); } The console owner and waiter logic now only applies between contexts that have taken the console lock. Since threaded printers never take the console lock, they do not have a console lock to handover to other contexts. @panic_console_dropped must change to atomic_t since it is no longer protected exclusively by the console lock. Since threaded printers remain asleep if they see that the console is locked, they now must be explicitly woken in __console_unlock(). This means wake_up_klogd() calls following a console_unlock() are no longer necessary and are removed. Signed-off-by: John Ogness --- include/linux/console.h | 15 +++ kernel/printk/printk.c | 243 ++++++++++++++++++++++++++++++---------- 2 files changed, 199 insertions(+), 59 deletions(-) diff --git a/include/linux/console.h b/include/linux/console.h index 9a251e70c090..c1fd4f41c547 100644 --- a/include/linux/console.h +++ b/include/linux/console.h @@ -16,6 +16,7 @@ =20 #include #include +#include =20 struct vc_data; struct console_font_op; @@ -136,6 +137,7 @@ static inline int con_debug_leave(void) #define CON_ANYTIME (16) /* Safe to call when cpu is offline */ #define CON_BRL (32) /* Used for a braille device */ #define CON_EXTENDED (64) /* Use the extended output format a la /dev/kmsg= */ +#define CON_THD_BLOCKED (128) /* Thread blocked because console is locked = */ =20 struct console { char name[16]; @@ -155,6 +157,19 @@ struct console { unsigned long dropped; struct task_struct *thread; =20 + /* + * The per-console lock is used by printing kthreads to synchronize + * this console with callers of console_lock(). This is necessary in + * order to allow printing kthreads to run in parallel to each other, + * while each safely accessing their own @flags and synchronizing + * against direct printing via console_lock/console_unlock. + * + * Note: For synchronizing against direct printing via + * console_trylock/console_unlock, see the static global + * variable @console_kthreads_active. + */ + struct mutex lock; + void *data; struct console *next; }; diff --git a/kernel/printk/printk.c b/kernel/printk/printk.c index a3b8b984c9aa..a5525a7b7bc5 100644 --- a/kernel/printk/printk.c +++ b/kernel/printk/printk.c @@ -223,6 +223,33 @@ int devkmsg_sysctl_set_loglvl(struct ctl_table *table,= int write, /* Number of registered extended console drivers. */ static int nr_ext_console_drivers; =20 +/* + * Used to synchronize printing kthreads against direct printing via + * console_trylock/console_unlock. + * + * Values: + * -1 =3D console kthreads atomically blocked (via global trylock) + * 0 =3D no kthread printing, console not locked (via trylock) + * >0 =3D kthread(s) actively printing + * + * Note: For synchronizing against direct printing via + * console_lock/console_unlock, see the @lock variable in + * struct console. + */ +static atomic_t console_kthreads_active =3D ATOMIC_INIT(0); + +#define console_kthreads_atomic_tryblock() \ + (atomic_cmpxchg(&console_kthreads_active, 0, -1) =3D=3D 0) +#define console_kthreads_atomic_unblock() \ + atomic_cmpxchg(&console_kthreads_active, -1, 0) +#define console_kthreads_atomically_blocked() \ + (atomic_read(&console_kthreads_active) =3D=3D -1) + +#define console_kthread_printing_tryenter() \ + atomic_inc_unless_negative(&console_kthreads_active) +#define console_kthread_printing_exit() \ + atomic_dec(&console_kthreads_active) + /* * Helper macros to handle lockdep when locking/unlocking console_sem. We = use * macros instead of functions so that _RET_IP_ contains useful informatio= n. @@ -270,6 +297,49 @@ static bool panic_in_progress(void) return unlikely(atomic_read(&panic_cpu) !=3D PANIC_CPU_INVALID); } =20 +/* + * Tracks whether kthread printers are all blocked. A value of true implies + * that the console is locked via console_lock() or the console is suspend= ed. + * Reading and writing to this variable requires holding @console_sem. + */ +static bool console_kthreads_blocked; + +/* + * Block all kthread printers from a schedulable context. + * + * Requires the console_lock. + */ +static void console_kthreads_block(void) +{ + struct console *con; + + for_each_console(con) { + mutex_lock(&con->lock); + con->flags |=3D CON_THD_BLOCKED; + mutex_unlock(&con->lock); + } + + console_kthreads_blocked =3D true; +} + +/* + * Unblock all kthread printers from a schedulable context. + * + * Requires the console_lock. + */ +static void console_kthreads_unblock(void) +{ + struct console *con; + + for_each_console(con) { + mutex_lock(&con->lock); + con->flags &=3D ~CON_THD_BLOCKED; + mutex_unlock(&con->lock); + } + + console_kthreads_blocked =3D false; +} + /* * This is used for debugging the mess that is the VT code by * keeping track if we have the console semaphore held. It's @@ -2563,13 +2633,6 @@ void resume_console(void) down_console_sem(); console_suspended =3D 0; console_unlock(); - - /* - * While suspended, new records may have been added to the - * ringbuffer. Wake up the kthread printers to print them. - */ - wake_up_klogd(); - pr_flush(1000, true); } =20 @@ -2588,9 +2651,10 @@ static int console_cpu_notify(unsigned int cpu) /* If trylock fails, someone else is doing the printing */ if (console_trylock()) console_unlock(); - - /* Wake kthread printers. Some may have become usable. */ - wake_up_klogd(); + else { + /* Some kthread printers may have become usable. */ + wake_up_klogd(); + } } return 0; } @@ -2610,6 +2674,7 @@ void console_lock(void) down_console_sem(); if (console_suspended) return; + console_kthreads_block(); console_locked =3D 1; console_may_schedule =3D 1; } @@ -2631,6 +2696,10 @@ int console_trylock(void) up_console_sem(); return 0; } + if (!console_kthreads_atomic_tryblock()) { + up_console_sem(); + return 0; + } console_locked =3D 1; console_may_schedule =3D 0; return 1; @@ -2639,7 +2708,7 @@ EXPORT_SYMBOL(console_trylock); =20 int is_console_locked(void) { - return console_locked; + return (console_locked || atomic_read(&console_kthreads_active)); } EXPORT_SYMBOL(is_console_locked); =20 @@ -2691,6 +2760,22 @@ static inline bool console_is_usable(struct console = *con) static void __console_unlock(void) { console_locked =3D 0; + + /* + * Depending on whether console_lock() or console_trylock() was used, + * appropriately allow the kthread printers to continue. + */ + if (console_kthreads_blocked) + console_kthreads_unblock(); + else + console_kthreads_atomic_unblock(); + + /* + * New records may have arrived while the console was locked. + * Wake the kthread printers to print them. + */ + wake_up_klogd(); + up_console_sem(); } =20 @@ -2706,19 +2791,18 @@ static void __console_unlock(void) * If dropped messages should be printed, @dropped_text is a buffer of size * DROPPED_TEXT_MAX. Otherwise @dropped_text must be NULL. * - * Requires the console_lock. - * * Returns false if the given console has no next record to print, otherwi= se * true. * * @handover will be set to true if a printk waiter has taken over the * console_lock, in which case the caller is no longer holding the - * console_lock. Otherwise it is set to false. + * console_lock. Otherwise it is set to false. A NULL pointer may be provi= ded + * to disable allowing the console_lock to be taken over by a printk waite= r. */ -static bool console_emit_next_record(struct console *con, char *text, char= *ext_text, - char *dropped_text, bool *handover) +static bool __console_emit_next_record(struct console *con, char *text, ch= ar *ext_text, + char *dropped_text, bool *handover) { - static int panic_console_dropped; + static atomic_t panic_console_dropped =3D ATOMIC_INIT(0); struct printk_info info; struct printk_record r; unsigned long flags; @@ -2727,7 +2811,8 @@ static bool console_emit_next_record(struct console *= con, char *text, char *ext_ =20 prb_rec_init_rd(&r, &info, text, CONSOLE_LOG_MAX); =20 - *handover =3D false; + if (handover) + *handover =3D false; =20 if (!prb_read_valid(prb, con->seq, &r)) return false; @@ -2735,7 +2820,8 @@ static bool console_emit_next_record(struct console *= con, char *text, char *ext_ if (con->seq !=3D r.info->seq) { con->dropped +=3D r.info->seq - con->seq; con->seq =3D r.info->seq; - if (panic_in_progress() && panic_console_dropped++ > 10) { + if (panic_in_progress() && + atomic_fetch_inc_relaxed(&panic_console_dropped) > 10) { suppress_panic_printk =3D 1; pr_warn_once("Too many dropped messages. Suppress messages on non-panic= CPUs to prevent livelock.\n"); } @@ -2757,31 +2843,52 @@ static bool console_emit_next_record(struct console= *con, char *text, char *ext_ len =3D record_print_text(&r, console_msg_format & MSG_FORMAT_SYSLOG, pr= intk_time); } =20 - /* - * While actively printing out messages, if another printk() - * were to occur on another CPU, it may wait for this one to - * finish. This task can not be preempted if there is a - * waiter waiting to take over. - * - * Interrupts are disabled because the hand over to a waiter - * must not be interrupted until the hand over is completed - * (@console_waiter is cleared). - */ - printk_safe_enter_irqsave(flags); - console_lock_spinning_enable(); + if (handover) { + /* + * While actively printing out messages, if another printk() + * were to occur on another CPU, it may wait for this one to + * finish. This task can not be preempted if there is a + * waiter waiting to take over. + * + * Interrupts are disabled because the hand over to a waiter + * must not be interrupted until the hand over is completed + * (@console_waiter is cleared). + */ + printk_safe_enter_irqsave(flags); + console_lock_spinning_enable(); + + /* don't trace irqsoff print latency */ + stop_critical_timings(); + } =20 - stop_critical_timings(); /* don't trace print latency */ call_console_driver(con, write_text, len, dropped_text); - start_critical_timings(); =20 con->seq++; =20 - *handover =3D console_lock_spinning_disable_and_check(); - printk_safe_exit_irqrestore(flags); + if (handover) { + start_critical_timings(); + *handover =3D console_lock_spinning_disable_and_check(); + printk_safe_exit_irqrestore(flags); + } skip: return true; } =20 +/* + * Print a record for a given console, but allow another printk() caller to + * take over the console_lock and continue printing. + * + * Requires the console_lock, but depending on @handover after the call, t= he + * caller may no longer have the console_lock. + * + * See __console_emit_next_record() for argument details. + */ +static bool console_emit_next_record_transferable(struct console *con, cha= r *text, char *ext_text, + char *dropped_text, bool *handover) +{ + return __console_emit_next_record(con, text, ext_text, dropped_text, hand= over); +} + /* * Print out all remaining records to all consoles. * @@ -2824,13 +2931,11 @@ static bool console_flush_all(bool do_cond_resched,= u64 *next_seq, bool *handove =20 if (con->flags & CON_EXTENDED) { /* Extended consoles do not print "dropped messages". */ - progress =3D console_emit_next_record(con, &text[0], - &ext_text[0], NULL, - handover); + progress =3D console_emit_next_record_transferable(con, &text[0], + &ext_text[0], NULL, handover); } else { - progress =3D console_emit_next_record(con, &text[0], - NULL, &dropped_text[0], - handover); + progress =3D console_emit_next_record_transferable(con, &text[0], + NULL, &dropped_text[0], handover); } if (*handover) return true; @@ -2946,6 +3051,10 @@ void console_unblank(void) if (oops_in_progress) { if (down_trylock_console_sem() !=3D 0) return; + if (!console_kthreads_atomic_tryblock()) { + up_console_sem(); + return; + } } else console_lock(); =20 @@ -3028,10 +3137,6 @@ void console_start(struct console *console) console_lock(); console->flags |=3D CON_ENABLED; console_unlock(); - - /* Wake the newly enabled kthread printer. */ - wake_up_klogd(); - __pr_flush(console, 1000, true); } EXPORT_SYMBOL(console_start); @@ -3230,7 +3335,11 @@ void register_console(struct console *newcon) if (newcon->flags & CON_EXTENDED) nr_ext_console_drivers++; =20 + if (console_kthreads_blocked) + newcon->flags |=3D CON_THD_BLOCKED; + newcon->dropped =3D 0; + mutex_init(&newcon->lock); if (newcon->flags & CON_PRINTBUFFER) { /* Get a consistent copy of @syslog_seq. */ mutex_lock(&syslog_lock); @@ -3510,6 +3619,19 @@ static void printk_fallback_preferred_direct(void) printk_kthreads_available =3D false; } =20 +/* + * Print a record for a given console, not allowing another printk() caller + * to take over. This is appropriate for contexts that do not have the + * console_lock. + * + * See __console_emit_next_record() for argument details. + */ +static bool console_emit_next_record(struct console *con, char *text, char= *ext_text, + char *dropped_text) +{ + return __console_emit_next_record(con, text, ext_text, dropped_text, NULL= ); +} + static bool printer_should_wake(struct console *con, u64 seq) { short flags; @@ -3517,9 +3639,6 @@ static bool printer_should_wake(struct console *con, = u64 seq) if (kthread_should_stop() || !printk_kthreads_available) return true; =20 - if (console_suspended) - return false; - if (!con->write) return false; =20 @@ -3530,8 +3649,12 @@ static bool printer_should_wake(struct console *con,= u64 seq) * attempting to print and instead go back to sleep. */ flags =3D data_race(READ_ONCE(con->flags)); - if (!(flags & CON_ENABLED)) + + if (!(flags & CON_ENABLED) || + (flags & CON_THD_BLOCKED) || + console_kthreads_atomically_blocked()) { return false; + } =20 if (atomic_read(&printk_prefer_direct)) return false; @@ -3544,7 +3667,6 @@ static int printk_kthread_func(void *data) struct console *con =3D data; char *dropped_text =3D NULL; char *ext_text =3D NULL; - bool handover; u64 seq =3D 0; char *text; int error; @@ -3595,15 +3717,18 @@ static int printk_kthread_func(void *data) if (error) continue; =20 - console_lock(); + error =3D mutex_lock_interruptible(&con->lock); + if (error) + continue; =20 - if (console_suspended) { - __console_unlock(); + if (!console_is_usable(con)) { + mutex_unlock(&con->lock); continue; } =20 - if (!console_is_usable(con)) { - __console_unlock(); + if ((con->flags & CON_THD_BLOCKED) || + !console_kthread_printing_tryenter()) { + mutex_unlock(&con->lock); continue; } =20 @@ -3616,13 +3741,13 @@ static int printk_kthread_func(void *data) * which can conditionally invoke cond_resched(). */ console_may_schedule =3D 0; - console_emit_next_record(con, text, ext_text, dropped_text, &handover); - if (handover) - continue; + console_emit_next_record(con, text, ext_text, dropped_text); =20 seq =3D con->seq; =20 - __console_unlock(); + console_kthread_printing_exit(); + + mutex_unlock(&con->lock); } =20 printk_console_msg(con, KERN_INFO, "printing thread stopped"); --=20 2.30.2 From nobody Fri Jun 19 09:50:04 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0ECDDC4321E for ; Wed, 6 Apr 2022 01:54:04 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1845010AbiDFBxj (ORCPT ); Tue, 5 Apr 2022 21:53:39 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:45854 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1354744AbiDEPMI (ORCPT ); Tue, 5 Apr 2022 11:12:08 -0400 Received: from galois.linutronix.de (Galois.linutronix.de [IPv6:2a0a:51c0:0:12e:550::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 94423109A72 for ; Tue, 5 Apr 2022 06:25:44 -0700 (PDT) From: John Ogness DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1649165143; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=fVfL+n5pd7cP1D7iEgDx1sRuang/QKBVMoT4FjG4/yk=; b=Z4nVgeULPomW2cxkt6UXVHgmZZsRMtR4IhKYUvX3dM5ab8P35/8nOSw2UPU3Z1oXtFdcsa ruMmgDJazjwdxEV9orlxT66kfiVD8ME6ej/Y+L6ugMmgKeVy52hACAmNsxZoA76Kd0Gimj qhgKpK2NHrwmgFJk9GVhLymXgYtxDi2NviRzy4tdcV3vxFUY3Zb5vEq4Z1jOAqfr8CE6DR OCYf2yPixn+L0+8f0H/1Xx3xyYRfYpvPJET58HWbcDlkop0n+K8/jlvov+ssjQ7PEp/kTi K4fW0mqqUNZweAAT4Zw5cPgxWV8+Fgm6NGOmn+xRRzONzYx3ge0d4L1XoqzTRg== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1649165143; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=fVfL+n5pd7cP1D7iEgDx1sRuang/QKBVMoT4FjG4/yk=; b=sTMt6t9kqCm7ejmE2CrJp4LikiVOUb/79aOdzF8qg0ReNoJFzExYHeDQ2pMd6RIuFXjvCG TL6DYm/M7xZ/6eAA== To: Petr Mladek Cc: Sergey Senozhatsky , Steven Rostedt , Thomas Gleixner , linux-kernel@vger.kernel.org Subject: [PATCH printk v2 12/12] printk: remove @console_locked Date: Tue, 5 Apr 2022 15:31:35 +0206 Message-Id: <20220405132535.649171-13-john.ogness@linutronix.de> In-Reply-To: <20220405132535.649171-1-john.ogness@linutronix.de> References: <20220405132535.649171-1-john.ogness@linutronix.de> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" The static global variable @console_locked is used to help debug VT code to make sure that certain code paths are running with the console_lock held. However, this information is also available with the static global variable @console_kthreads_blocked (for locking via console_lock()), and the static global variable @console_kthreads_active (for locking via console_trylock()). Remove @console_locked and update is_console_locked() to use the alternative variables. Signed-off-by: John Ogness Reviewed-by: Petr Mladek --- kernel/printk/printk.c | 29 ++++++++++++++--------------- 1 file changed, 14 insertions(+), 15 deletions(-) diff --git a/kernel/printk/printk.c b/kernel/printk/printk.c index a5525a7b7bc5..2da66adac87e 100644 --- a/kernel/printk/printk.c +++ b/kernel/printk/printk.c @@ -340,15 +340,7 @@ static void console_kthreads_unblock(void) console_kthreads_blocked =3D false; } =20 -/* - * This is used for debugging the mess that is the VT code by - * keeping track if we have the console semaphore held. It's - * definitely not the perfect debug tool (we don't know if _WE_ - * hold it and are racing, but it helps tracking those weird code - * paths in the console code where we end up in places I want - * locked without the console semaphore held). - */ -static int console_locked, console_suspended; +static int console_suspended; =20 /* * Array of consoles built from command line options (console=3D) @@ -2675,7 +2667,6 @@ void console_lock(void) if (console_suspended) return; console_kthreads_block(); - console_locked =3D 1; console_may_schedule =3D 1; } EXPORT_SYMBOL(console_lock); @@ -2700,15 +2691,26 @@ int console_trylock(void) up_console_sem(); return 0; } - console_locked =3D 1; console_may_schedule =3D 0; return 1; } EXPORT_SYMBOL(console_trylock); =20 +/* + * This is used to help to make sure that certain paths within the VT code= are + * running with the console lock held. It is definitely not the perfect de= bug + * tool (it is not known if the VT code is the task holding the console lo= ck), + * but it helps tracking those weird code paths in the console code such as + * when the console is suspended: where the console is not locked but no + * console printing may occur. + * + * Note: This returns true when the console is suspended but is not locked. + * This is intentional because the VT code must consider that situat= ion + * the same as if the console was locked. + */ int is_console_locked(void) { - return (console_locked || atomic_read(&console_kthreads_active)); + return (console_kthreads_blocked || atomic_read(&console_kthreads_active)= ); } EXPORT_SYMBOL(is_console_locked); =20 @@ -2759,8 +2761,6 @@ static inline bool console_is_usable(struct console *= con) =20 static void __console_unlock(void) { - console_locked =3D 0; - /* * Depending on whether console_lock() or console_trylock() was used, * appropriately allow the kthread printers to continue. @@ -3058,7 +3058,6 @@ void console_unblank(void) } else console_lock(); =20 - console_locked =3D 1; console_may_schedule =3D 0; for_each_console(c) if ((c->flags & CON_ENABLED) && c->unblank) --=20 2.30.2