From: Adam Simonelli <adamsimonelli@gmail.com>

If CONFIG_NULL_TTY_DEFAULT_CONSOLE is enabled, and CONFIG_VT is disabled,
ttynull will become the default primary console device, based on the link
order.

Many distributions ship with CONFIG_VT enabled. On tested desktop hardware
if CONFIG_VT is disabled, the default console device falls back to
/dev/ttyS0 instead of /dev/tty.

This could cause issues in user space, and hardware problems:

1. The user space issues include the case where  /dev/ttyS0 is
disconnected, and the TCGETS ioctl, which some user space libraries use
as a probe to determine if a file is a tty, is called on /dev/console and
fails. Programs that call isatty() on /dev/console and get an incorrect
false value may skip expected logging to /dev/console

2. The hardware issues include the case if a user has a science instrument
or other device connected to the /dev/ttyS0 port, and they were to upgrade
to a kernel that is disabling the CONFIG_VT option, kernel logs will then be
sent to the device connected to /dev/ttyS0 unless they edit their kernel
command line manually.

The new CONFIG_NULL_TTY_CONSOLE option will give users and distribution
maintainers an option to avoid this. Disabling CONFIG_VT and enabling
CONFIG_NULL_TTY_CONSOLE will ensure the default kernel console behavior
is not dependant on hardware configuration by default, and avoid
unexpected new behavior on devices connected to the /dev/ttyS0 serial
port.

Signed-off-by: Adam Simonelli <adamsimonelli@gmail.com>
---
 drivers/tty/Makefile | 11 +++++++++++
 1 file changed, 11 insertions(+)

diff --git a/drivers/tty/Makefile b/drivers/tty/Makefile
index 07aca5184a55..a4caf32a2014 100644
--- a/drivers/tty/Makefile
+++ b/drivers/tty/Makefile
@@ -11,6 +11,11 @@ obj-$(CONFIG_N_HDLC)		+= n_hdlc.o
 obj-$(CONFIG_N_GSM)		+= n_gsm.o
 
 obj-y				+= vt/
+# If ttynull is configured to be a console by default, ensure that it is linked
+# earlier before a real one is selected.
+obj-$(CONFIG_NULL_TTY_DEFAULT_CONSOLE) \
+				+= ttynull.o
+
 obj-$(CONFIG_HVC_DRIVER)	+= hvc/
 obj-y				+= serial/
 obj-$(CONFIG_SERIAL_DEV_BUS)	+= serdev/
@@ -20,7 +25,13 @@ obj-$(CONFIG_AMIGA_BUILTIN_SERIAL) += amiserial.o
 obj-$(CONFIG_MOXA_INTELLIO)	+= moxa.o
 obj-$(CONFIG_MOXA_SMARTIO)	+= mxser.o
 obj-$(CONFIG_NOZOMI)		+= nozomi.o
+
+# If ttynull is enabled, but not as a boot console, it is linked and used later
+# after the real ones.
+ifneq ($(CONFIG_NULL_TTY_DEFAULT_CONSOLE),y)
 obj-$(CONFIG_NULL_TTY)	        += ttynull.o
+endif
+
 obj-$(CONFIG_SYNCLINK_GT)	+= synclink_gt.o
 obj-$(CONFIG_PPC_EPAPR_HV_BYTECHAN) += ehv_bytechan.o
 obj-$(CONFIG_GOLDFISH_TTY)	+= goldfish.o
-- 
2.45.2

On Tue, Mar 4, 2025 at 5:55 AM <adamsimonelli@gmail.com> wrote:
>
> From: Adam Simonelli <adamsimonelli@gmail.com>
>
> If CONFIG_NULL_TTY_DEFAULT_CONSOLE is enabled, and CONFIG_VT is disabled,
> ttynull will become the default primary console device, based on the link
> order.
>
> Many distributions ship with CONFIG_VT enabled. On tested desktop hardware
> if CONFIG_VT is disabled, the default console device falls back to
> /dev/ttyS0 instead of /dev/tty.
>
> This could cause issues in user space, and hardware problems:
>
> 1. The user space issues include the case where  /dev/ttyS0 is
> disconnected, and the TCGETS ioctl, which some user space libraries use
> as a probe to determine if a file is a tty, is called on /dev/console and
> fails. Programs that call isatty() on /dev/console and get an incorrect
> false value may skip expected logging to /dev/console

Missing period at the end.

> 2. The hardware issues include the case if a user has a science instrument
> or other device connected to the /dev/ttyS0 port, and they were to upgrade
> to a kernel that is disabling the CONFIG_VT option, kernel logs will then be
> sent to the device connected to /dev/ttyS0 unless they edit their kernel
> command line manually.
>
> The new CONFIG_NULL_TTY_CONSOLE option will give users and distribution
> maintainers an option to avoid this. Disabling CONFIG_VT and enabling
> CONFIG_NULL_TTY_CONSOLE will ensure the default kernel console behavior
> is not dependant on hardware configuration by default, and avoid
> unexpected new behavior on devices connected to the /dev/ttyS0 serial
> port.

...

>  obj-y                          += vt/

+ blank line.

> +# If ttynull is configured to be a console by default, ensure that it is linked
> +# earlier before a real one is selected.
> +obj-$(CONFIG_NULL_TTY_DEFAULT_CONSOLE) \
> +                               += ttynull.o

Here is the question: are you sure that all console drivers that exist
in the kernel happen to be here? Have you grepped the source tree for
checking this?

...

> +# If ttynull is enabled, but not as a boot console, it is linked and used later
> +# after the real ones.
> +ifneq ($(CONFIG_NULL_TTY_DEFAULT_CONSOLE),y)

Also can be written as
ifeq ($(...),)
but it might be less explicit. Up to you.

>  obj-$(CONFIG_NULL_TTY)         += ttynull.o
> +endif

-- 
With Best Regards,
Andy Shevchenko