From nobody Mon Dec 1 22:03:46 2025 Received: from mail-pl1-f169.google.com (mail-pl1-f169.google.com [209.85.214.169]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 84AA630E85D for ; Thu, 27 Nov 2025 19:44:32 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.169 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1764272674; cv=none; b=Om9u6rc2RAH2wZ2oUHGZsp8GyG5HH/Dz3fViZdQfW2C9W2WFjKIulw0B8CZs0w+2LRWdvXTmYNO4msdwAkpk4nAO/EDl8vmA+PBr9O0bV+0vC2WcwNSzn9GRUrfB9SnSVXdGsMLSeUmtdvU/XUebN7cUyzVbiqPm4MREUid4lPI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1764272674; c=relaxed/simple; bh=3ZnIa88ksJWv5CPz7EOsMTMd1fbxaIDTlFPQcOYegnc=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=J2QAuWP3NzvTWPVVPmGp9bKU7yapROFIK2cqzXEQwt8TvCN0gstySXH97MAgYtroYCePCtaCI2xauJijoFVLymvqatquIQsUZizlr4xF2CZq8fycjwo9TdLMMQC5HwD/dWor3Dj42NMdqR7yUChW3uZZDaiVmmwpBLui1cJRetk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=chrisdown.name; spf=pass smtp.mailfrom=chrisdown.name; dkim=pass (1024-bit key) header.d=chrisdown.name header.i=@chrisdown.name header.b=PK+Jzbew; arc=none smtp.client-ip=209.85.214.169 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=chrisdown.name Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=chrisdown.name Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=chrisdown.name header.i=@chrisdown.name header.b="PK+Jzbew" Received: by mail-pl1-f169.google.com with SMTP id d9443c01a7336-2955623e6faso12738885ad.1 for ; Thu, 27 Nov 2025 11:44:32 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chrisdown.name; s=google; t=1764272672; x=1764877472; darn=vger.kernel.org; h=user-agent:in-reply-to:content-disposition:mime-version:references :message-id:subject:cc:to:from:date:from:to:cc:subject:date :message-id:reply-to; bh=1NFEt68KAzVxk4+h4IjgxUfH5qTya3AH4ZFnApA2EDQ=; b=PK+Jzbew+0sZBukdDx3pWRhOEbMnbgkZN4uUqgFttKdmTRh6qqOWyzWzlRvZu5rr7f ClOo6G0hXz8oofjDZMhPL73p8tN0ZoiobmEzzMp042wLuJNJ7iBnd04tChJ9zLRYUgeC bGLuvtX8rhpTmWiVrhNZZCNuDzoD6RcEUbyFM= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1764272672; x=1764877472; h=user-agent:in-reply-to:content-disposition:mime-version:references :message-id:subject:cc:to:from:date:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=1NFEt68KAzVxk4+h4IjgxUfH5qTya3AH4ZFnApA2EDQ=; b=RaQSkgx5gLRUEG7MxLFBWsJIH1OLU1IPvEaL1s4DnHxfgxWKo+mVCDiOXd16dbxudN Q8xTWOKT3jzsuvueEiq73/MDYLZEhK/9BWaZ8QOsuAk2eEaGIVkEAaPljeA8q8lvrVSu A9151lhMuqIRgdzfCcbv3g/3YXkLZSio5FLB0aI4NAvs+a6GZFfaoxiJH/xDZYwRXgCN ROSDO6wEawmy/JDIixfOE69kXmSVRLVvJ4Fcbm3OsQrWENGcgRkI7nwpC9cFSoUqvok5 F8caABuQ6rOyrh+rbAT3Xzxv0fgqxHCsU8CBthbYIPipjni5PSxQLwPy0kDSKj3UY+pY QyGw== X-Gm-Message-State: AOJu0Yz8Q9KI5/2bBPhThJh/XhvowsFgBdKy662i4ObQzMR3VKcG6kai yEBTMw7cpsHBxwa2T2iEkdhWjpUbaX+1ZizksALNVAHIGgiJSpNI4AFL3F4vIex50OtXVxSqYTG +kRmocRbC/g== X-Gm-Gg: ASbGnct9gYKPvKQFMVMGtQGTOBVTS+OJ1dZpGxtDoYbn3SNJ3ISPkuYKLaSBsnNgRhH D2/4Ly7DuIZeMIPBxQM50VixkhyMXLUS2dCnQTRiTUPKe+z/5BngpBWeVqxN33+scodauGRMSld gezpXNXB2Hvn/n+E4MHlFcv0Kaoj5AGcMB6rF/IdCTixBtuvu36sh0B00b9MjncAbjd+qt60nAI IRP9DDfvg1nRKoY0SPLillUWy4k9SlRx8SoJSGijOF06oX7Pd10tep+H+9LD51lJKppOKRMV/e/ VMnHpowjX9zpUZEU90k5eqfnJPMMRHWJe8x4k1BOtc6cXzbbsI0S9ukvrpgO5ZIwdTn8D0VZKcd hq7DAW1KePt5CKF7Y3jW2u3TwuHvVWaeNNBn7oYHw3EZDaX5Yu77CYaRs6nllduqlEkhhOSVhKz VyHC+ennNbxYD+aQvfdnI= X-Google-Smtp-Source: AGHT+IGVY9TzQTuiOlJ1CynlC5FFBqbxGbvlQqIfyrvW6Y4Vp+VvHl7f8EjDyviVjMXI3NC/sRVAXA== X-Received: by 2002:a17:90b:3511:b0:340:7b2e:64cc with SMTP id 98e67ed59e1d1-3475ebfa6f7mr12732424a91.15.1764272671703; Thu, 27 Nov 2025 11:44:31 -0800 (PST) Received: from localhost ([116.86.198.140]) by smtp.gmail.com with ESMTPSA id 41be03b00d2f7-be4f5f3416asm2651563a12.0.2025.11.27.11.44.31 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 27 Nov 2025 11:44:31 -0800 (PST) Date: Fri, 28 Nov 2025 03:44:29 +0800 From: Chris Down To: Petr Mladek Cc: linux-kernel@vger.kernel.org, Greg Kroah-Hartman , Sergey Senozhatsky , Steven Rostedt , John Ogness , Geert Uytterhoeven , Tony Lindgren , kernel-team@fb.com Subject: [PATCH v8 19/21] printk: docs: Add comprehensive guidance for per-console loglevels Message-ID: <23043698a820da6e5a50e7ebaef8f7422d8ff471.1764272407.git.chris@chrisdown.name> References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: User-Agent: Mutt/2.2.15 (2b349c5e) (2025-10-02) Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" The per-console loglevel feature documentation could use some practical guidance. This commit adds: - Examples section covering runtime configuration, effective loglevel checking, and boot-time configuration - Common use case demonstrating high-performance netconsole with quiet serial console fallback - Performance impact section explaining how per-console loglevels reduce latency by filtering messages before slow console writes - Edge cases section documenting behavior with concurrent writes, console unregistration, and global loglevel changes The guidance interleaves advice about many parts of this patchset, so let's have it in a distinct commit. This documentation will help users understand how to effectively use and debug per-console loglevels. Signed-off-by: Chris Down --- Documentation/admin-guide/index.rst | 1 + .../admin-guide/per-console-loglevel.rst | 252 ++++++++++++++++++ MAINTAINERS | 1 + 3 files changed, 254 insertions(+) create mode 100644 Documentation/admin-guide/per-console-loglevel.rst diff --git a/Documentation/admin-guide/index.rst b/Documentation/admin-guid= e/index.rst index 259d79fbeb94..5f0ae9c4017b 100644 --- a/Documentation/admin-guide/index.rst +++ b/Documentation/admin-guide/index.rst @@ -155,6 +155,7 @@ How to configure your hardware within your Linux system. media/index nvme-multipath parport + per-console-loglevel pnp rapidio rtc diff --git a/Documentation/admin-guide/per-console-loglevel.rst b/Documenta= tion/admin-guide/per-console-loglevel.rst new file mode 100644 index 000000000000..f2db46c00de3 --- /dev/null +++ b/Documentation/admin-guide/per-console-loglevel.rst @@ -0,0 +1,252 @@ +.. SPDX-License-Identifier: GPL-2.0 + +.. _per_console_loglevel: + +Per-console loglevel support +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D + +Motivation +---------- + +Consoles can have vastly different latencies and throughputs. For example, +writing a message to the serial console can take on the order of tens of +milliseconds to get the UART to successfully write a message. While this m= ight +be fine for a single, one-off message, this can cause significant +application-level stalls in situations where the kernel writes large amoun= ts of +information to the console. + +This means that while you might want to send at least INFO level messages = to +(for example) netconsole, which is relatively fast, you may only want to s= end at +least WARN level messages to the serial console. This permits debugging +using the serial console in cases that netconsole doesn't receive messages +during particularly bad system issues, while still keeping the noise low e= nough +to avoid inducing latency in userspace applications. + +Loglevel +-------- + +Kernel loglevels are defined thus: + ++---+--------------+-----------------------------------+ +| 0 | KERN_EMERG | system is unusable | ++---+--------------+-----------------------------------+ +| 1 | KERN_ALERT | action must be taken immediately | ++---+--------------+-----------------------------------+ +| 2 | KERN_CRIT | critical conditions | ++---+--------------+-----------------------------------+ +| 3 | KERN_ERR | error conditions | ++---+--------------+-----------------------------------+ +| 4 | KERN_WARNING | warning conditions | ++---+--------------+-----------------------------------+ +| 5 | KERN_NOTICE | normal but significant condition | ++---+--------------+-----------------------------------+ +| 6 | KERN_INFO | informational | ++---+--------------+-----------------------------------+ +| 7 | KERN_DEBUG | debug-level messages | ++---+--------------+-----------------------------------+ + +Tunables +-------- + +In order to allow tuning per-console loglevels, the following controls exi= st: + +Global +~~~~~~ + +The global loglevel is set by the ``kernel.console_loglevel`` sysctl, whic= h can +also be set as ``loglevel=3D`` on the kernel command line. + +The printk module also takes two parameters which modify this behaviour +further: + +* ``ignore_loglevel`` on the kernel command line or set in printk paramete= rs: + Emit all messages. All other controls are ignored if this is present. + +* ``ignore_per_console_loglevel`` on the kernel command line or set in pri= ntk + parameters: Ignore all per-console loglevels and use the global loglevel. + +The default value for ``kernel.console_loglevel`` comes from +``CONFIG_CONSOLE_LOGLEVEL_DEFAULT``, or ``CONFIG_CONSOLE_LOGLEVEL_QUIET`` = if +``quiet`` is passed on the kernel command line. + +Console attributes +~~~~~~~~~~~~~~~~~~ + +Registered consoles are exposed at ``/sys/class/console``. For example, if= you +are using ``ttyS0``, the console backing it can be viewed at +``/sys/class/console/ttyS0/``. The following files are available: + +* ``effective_loglevel`` (r): The effective loglevel after considering all + loglevel authorities. For example, it shows the value of the console-spe= cific + loglevel when a console-specific loglevel is defined, and shows the glob= al + console loglevel value when the console-specific one is not defined. + +* ``effective_loglevel_source`` (r): The loglevel authority which resulted= in + the effective loglevel being set. The following values can be present: + + * ``local``: The console-specific loglevel is in effect. + + * ``global``: The global loglevel (``kernel.console_loglevel``) is in + effect. Set a console-specific loglevel to override it. + + * ``ignore_loglevel``: ``ignore_loglevel`` was specified on the kernel + command line or at ``/sys/module/printk/parameters/ignore_loglevel``. + Disable it to use level controls. + +* ``loglevel`` (rw): The local, console-specific loglevel for this console. + This will be in effect if no other global control overrides it. Look at + ``effective_loglevel`` and ``effective_loglevel_source`` to verify that. + +Examples +-------- + +Setting per-console loglevel at runtime +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Set serial console to only show warnings and above (level 4):: + + echo 5 > /sys/class/console/ttyS0/loglevel + +Set netconsole to show info and above (level 6):: + + echo 7 > /sys/class/console/netcon0/loglevel + +Reset a console to use the global loglevel:: + + echo -1 > /sys/class/console/ttyS0/loglevel + +Checking effective loglevel +~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Check what loglevel is actually in effect for a console:: + + $ cat /sys/class/console/ttyS0/effective_loglevel + 4 + $ cat /sys/class/console/ttyS0/effective_loglevel_source + local + +If the source shows ``global``, the console is using the global loglevel. +If it shows ``local``, the console is using its per-console loglevel. +If it shows ``ignore_loglevel``, all loglevel controls are being ignored. + +Boot-time configuration +~~~~~~~~~~~~~~~~~~~~~~~~ + +Set different loglevels for different consoles at boot:: + + console=3DttyS0,115200n8,loglevel:3 console=3Dtty0,loglevel:5 + +This sets the serial console (ttyS0) to level 3 (KERN_ERR) and the VGA +console (tty0) to level 5 (KERN_NOTICE). + +For netconsole:: + + netconsole=3D@/,@192.168.1.1/ console=3Dnetcon0,loglevel:6 + +Common use case - high performance with serial fallback +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +A common configuration is to set netconsole to a verbose level for normal +debugging, while keeping the serial console quiet to avoid performance imp= act, +but still available for emergencies:: + + # Netconsole gets INFO and above (verbose) + echo 7 > /sys/class/console/netcon0/loglevel + + # Serial console gets only WARN and above (quiet, for emergencies) + echo 5 > /sys/class/console/ttyS0/loglevel + +This allows you to see informational messages on the fast netconsole witho= ut +the latency impact of writing them to the slow serial port. + +Performance Impact +------------------ + +Kernel messages used to be flushed to the consoles immediately even from a +context where the scheduling is not possible. It increases a chance to see= the +messages even when the system is in a bad state. But it might cause signif= icant +application-level stalls (e.g., during network debugging or block I/O trac= ing). +Note that serial console writes can take tens of milliseconds per message. + +The console drivers are being converted to nbcon API (the letter 'N' +in /proc/consoles output). These drivers write the messages in a dedicated +kthreads when the system is working properly. It reduces the risk of stall= s. But +the messages are still flushed immediately when the system detects an emer= gency +situation, for example Oops, stall, or a warning. Also the messages can ge= t lost +when the ring buffer is full and the console driver is far behind with flu= shing. + +For example, setting a serial console to WARN level (4) while keeping +netconsole at INFO level (6) prevents INFO and NOTICE messages from being +written to the slow serial port. It reduces the risk of application stalls= or +message loses during verbose logging periods. + +Troubleshooting +--------------- + +Messages not appearing on console despite setting loglevel +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +1. Check effective loglevel source:: + + cat /sys/class/console//effective_loglevel_source + + If it shows ``ignore_loglevel``, you have the ``printk.ignore_loglevel`` + kernel parameter set, which overrides all level controls. Remove it from + your kernel command line or set it to N in sysfs:: + + echo N > /sys/module/printk/parameters/ignore_loglevel + +2. Check if per-console loglevels are being ignored:: + + cat /sys/module/printk/parameters/ignore_per_console_loglevel + + If it shows ``Y``, per-console settings are disabled. Set it to N:: + + echo N > /sys/module/printk/parameters/ignore_per_console_loglevel + +3. Verify the message priority is high enough:: + + cat /sys/class/console//effective_loglevel + + Messages must have priority less than this value to appear. For example, + if effective_loglevel is 4, only messages with priority 0-3 (EMERG, ALE= RT, + CRIT, ERR) will be printed. If you want to see WARN messages (priority = 4), + you need to increase the effective_loglevel to at least 5. + +Cannot set loglevel to 0 +~~~~~~~~~~~~~~~~~~~~~~~~~ + +Per-console loglevels cannot be set to 0 (KERN_EMERG). This is by design, = as +level 0 is reserved for the most critical system messages that should alwa= ys +go to all consoles. To use the global loglevel, set the per-console loglev= el +to -1:: + + echo -1 > /sys/class/console//loglevel + +Edge cases +~~~~~~~~~~ + +**Setting all consoles to high loglevels**: If you set all consoles to +very high loglevels (e.g., 1 or 2), most messages won't appear on any +console. They remain accessible to userspace via ``dmesg`` or ``syslogd``, +but keep at least one console at a reasonable level for monitoring. + +**Console unregistration while sysfs file is open**: If a console is +unregistered (e.g., module unloaded) while you have its sysfs files open, +the files will become stale. Close and reopen them, or they will eventually +return errors. + +**Global loglevel changes**: If you change the global console_loglevel +via sysctl, consoles set to -1 (use global) will immediately reflect the +new level. Consoles with explicit per-console levels are unaffected. + +Deprecated +~~~~~~~~~~ + +* ``kernel.printk`` sysctl: this takes four values, setting + ``kernel.console_loglevel``, ``kernel.default_message_loglevel``, the mi= nimum + console loglevel, and a fourth unused value. The interface is generally + considered to be quite confusing, doesn't perform checks on the values g= iven, + and is unaware of per-console loglevel semantics. + +Chris Down , 18-November-2025 diff --git a/MAINTAINERS b/MAINTAINERS index 2e6faa647b43..9a509a0bc65a 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -20506,6 +20506,7 @@ R: Sergey Senozhatsky S: Maintained T: git git://git.kernel.org/pub/scm/linux/kernel/git/printk/linux.git F: Documentation/ABI/testing/sysfs-class-console +F: Documentation/admin-guide/per-console-loglevel.rst F: Documentation/core-api/printk-basics.rst F: include/linux/printk.h F: kernel/printk/ --=20 2.51.2