From: Adam Simonelli <adamsimonelli@gmail.com>

When switching to a CONFIG_VT=n world, at least on x86 systems,
/dev/console becomes /dev/ttyS0. This can cause some undesired effects.
/dev/console's behavior is now tied to the physical /dev/ttyS0, which when
disconnected can cause isatty() to fail when /dev/ttyS0 is disconnected,
and users who upgrade to a theoretical vt-less kernel from their
distribution who have a device such as a science instrument connected to
their /dev/ttyS0 port will suddenly see it receive kernel log messages.

When the new CONFIG_NULL_TTY_CONSOLE option is turned on, this will allow
the ttynull device to be leveraged as the default console. Distributions
that had CONFIG_VT turned on before will be able to leverage this option
to where /dev/console is still backed by a psuedo device, avoiding these
issues, without needing to enable the entire VT subsystem.

v2:
    rebase

v3:
    Clarify commit messages.

    Guard the all the register_console()s in ttynull to prevent it from being
    registered twice.

    Only change the link order if CONFIG_NULL_TTY_CONSOLE is enabled, otherwise
    use the existing order for ttynull if only CONFIG_NULL_TTY is enabled.

    Document why the link order changes in the drivers/tty/Makefile file.

    Replace #ifdefs

v4:
    Remember to actually include the changes to v3 in the cover letter.

Adam Simonelli (2):
  ttynull: Add an option to allow ttynull to be used as a console device
  tty: Change order of ttynull to be linked sooner if enabled as a
    console.

 drivers/tty/Kconfig   | 15 ++++++++++++++-
 drivers/tty/Makefile  | 10 ++++++++++
 drivers/tty/ttynull.c | 20 +++++++++++++++++++-
 3 files changed, 43 insertions(+), 2 deletions(-)

-- 
2.45.2

Sun, Feb 23, 2025 at 03:44:54PM -0500, adamsimonelli@gmail.com kirjoitti:
> From: Adam Simonelli <adamsimonelli@gmail.com>
> 
> When switching to a CONFIG_VT=n world, at least on x86 systems,
> /dev/console becomes /dev/ttyS0. This can cause some undesired effects.
> /dev/console's behavior is now tied to the physical /dev/ttyS0, which when
> disconnected can cause isatty() to fail when /dev/ttyS0 is disconnected,
> and users who upgrade to a theoretical vt-less kernel from their
> distribution who have a device such as a science instrument connected to
> their /dev/ttyS0 port will suddenly see it receive kernel log messages.
> 
> When the new CONFIG_NULL_TTY_CONSOLE option is turned on, this will allow
> the ttynull device to be leveraged as the default console. Distributions
> that had CONFIG_VT turned on before will be able to leverage this option
> to where /dev/console is still backed by a psuedo device, avoiding these
> issues, without needing to enable the entire VT subsystem.

This rings a bell of the following

https://lore.kernel.org/all/20201111135450.11214-1-pmladek@suse.com/
https://lore.kernel.org/all/20210107164400.17904-1-pmladek@suse.com/
https://lore.kernel.org/all/20210108114847.23469-1-pmladek@suse.com/

I don't see any mention in the commit message about these, have you studied the
cases? Will your change anyhow affect the described there?

-- 
With Best Regards,
Andy Shevchenko

On Sunday, February 23, 2025 4:10:10 PM EST Andy Shevchenko wrote:
> Sun, Feb 23, 2025 at 03:44:54PM -0500, adamsimonelli@gmail.com kirjoitti:
> > From: Adam Simonelli <adamsimonelli@gmail.com>
> > 
> > When switching to a CONFIG_VT=n world, at least on x86 systems,
> > /dev/console becomes /dev/ttyS0. This can cause some undesired effects.
> > /dev/console's behavior is now tied to the physical /dev/ttyS0, which when
> > disconnected can cause isatty() to fail when /dev/ttyS0 is disconnected,
> > and users who upgrade to a theoretical vt-less kernel from their
> > distribution who have a device such as a science instrument connected to
> > their /dev/ttyS0 port will suddenly see it receive kernel log messages.
> > 
> > When the new CONFIG_NULL_TTY_CONSOLE option is turned on, this will allow
> > the ttynull device to be leveraged as the default console. Distributions
> > that had CONFIG_VT turned on before will be able to leverage this option
> > to where /dev/console is still backed by a psuedo device, avoiding these
> > issues, without needing to enable the entire VT subsystem.
> 
> This rings a bell of the following
> 
> https://lore.kernel.org/all/20201111135450.11214-1-pmladek@suse.com/
> https://lore.kernel.org/all/20210107164400.17904-1-pmladek@suse.com/
> https://lore.kernel.org/all/20210108114847.23469-1-pmladek@suse.com/
> 
> I don't see any mention in the commit message about these, have you studied the
> cases? Will your change anyhow affect the described there?
> 
> 
I did see that sifting through commits, it looks kind of different though, as
that was to make ttynull more always on, and if I am understanding it correctly
it looks more of a last resort? I could be wrong, but I see it was attempted
and reverted because of conflicts on some hardware platforms?

The scope with this new patch set is much different, as it has to be manually
enabled on top of CONFIG_NULL_TTY, rather than assuming CONFIG_NULL_TTY is
enabled, and adding it to the list of preferred consoles. It is more for
Desktop configs that are looking to disable CONFIG_VT, but still want
/dev/console to not be a physical device by default.

On Mon, Feb 24, 2025 at 2:23 AM Adam Simonelli <adamsimonelli@gmail.com> wrote:
> On Sunday, February 23, 2025 4:10:10 PM EST Andy Shevchenko wrote:
> > Sun, Feb 23, 2025 at 03:44:54PM -0500, adamsimonelli@gmail.com kirjoitti:
> > > From: Adam Simonelli <adamsimonelli@gmail.com>
> > >
> > > When switching to a CONFIG_VT=n world, at least on x86 systems,
> > > /dev/console becomes /dev/ttyS0. This can cause some undesired effects.
> > > /dev/console's behavior is now tied to the physical /dev/ttyS0, which when
> > > disconnected can cause isatty() to fail when /dev/ttyS0 is disconnected,
> > > and users who upgrade to a theoretical vt-less kernel from their
> > > distribution who have a device such as a science instrument connected to
> > > their /dev/ttyS0 port will suddenly see it receive kernel log messages.
> > >
> > > When the new CONFIG_NULL_TTY_CONSOLE option is turned on, this will allow
> > > the ttynull device to be leveraged as the default console. Distributions
> > > that had CONFIG_VT turned on before will be able to leverage this option
> > > to where /dev/console is still backed by a psuedo device, avoiding these
> > > issues, without needing to enable the entire VT subsystem.
> >
> > This rings a bell of the following
> >
> > https://lore.kernel.org/all/20201111135450.11214-1-pmladek@suse.com/
> > https://lore.kernel.org/all/20210107164400.17904-1-pmladek@suse.com/
> > https://lore.kernel.org/all/20210108114847.23469-1-pmladek@suse.com/
> >
> > I don't see any mention in the commit message about these, have you studied the
> > cases? Will your change anyhow affect the described there?
> >
> I did see that sifting through commits, it looks kind of different though, as
> that was to make ttynull more always on, and if I am understanding it correctly
> it looks more of a last resort? I could be wrong, but I see it was attempted
> and reverted because of conflicts on some hardware platforms?
>
> The scope with this new patch set is much different, as it has to be manually
> enabled on top of CONFIG_NULL_TTY, rather than assuming CONFIG_NULL_TTY is
> enabled, and adding it to the list of preferred consoles. It is more for
> Desktop configs that are looking to disable CONFIG_VT, but still want
> /dev/console to not be a physical device by default.

Thank you for elaboration. Even if this may be a different case, I
think it would be nice to have the PRINTK maintainers (who are also
involved in the console code) blessing before going in.

In case of a new version, please summarize above in the commit message.

-- 
With Best Regards,
Andy Shevchenko