[PATCH 0/1] Fix to allow more correct isatty()

Gil Pedersen posted 1 patch 2 days, 15 hours ago
drivers/tty/tty_io.c | 29 ++++++++++++++++++++++-------
1 file changed, 22 insertions(+), 7 deletions(-)
[PATCH 0/1] Fix to allow more correct isatty()
Posted by Gil Pedersen 2 days, 15 hours ago
This patch comes from an issue discovered in systemd, where it can fail to
restore a text TTY session after a GUI session is stopped when compiled
with musl libc.

The specific systemd integration issue is currently resolved in the musl
master branch by closer aligning the implementation with glibc, but the
underlying isatty() implementation is still flawed since it can return 0
(false) for something that is definitely a TTY. Essentially both musl
and glibc give up correctly handling this case on Linux due to
inadequate/buggy kernel APIs.

Thus I am proposing this patch as a solution to fix the kernel APIs. An
alternative fix could be to add a new IOCTL to specifically handle this,
but that seems overkill.

Link: https://github.com/systemd/systemd/pull/34039
Link: https://gitlab.alpinelinux.org/alpine/aports/-/merge_requests/70711
Link: https://www.openwall.com/lists/musl/2024/08/20/2
Link: https://git.musl-libc.org/cgit/musl/commit/src/unistd/isatty.c?id=c94a0c16f08894ce3be6dafb0fe80baa77a6ff2a
Link: https://sourceware.org/bugzilla/show_bug.cgi?id=32103

Gil Pedersen (1):
  tty: respond to TIOCGWINSZ when hung

 drivers/tty/tty_io.c | 29 ++++++++++++++++++++++-------
 1 file changed, 22 insertions(+), 7 deletions(-)

-- 
2.45.2
Re: [PATCH 0/1] Fix to allow more correct isatty()
Posted by nerdopolis 1 day, 3 hours ago
On Thursday, November 21, 2024 6:12:53 AM EST Gil Pedersen wrote:
> This patch comes from an issue discovered in systemd, where it can fail to
> restore a text TTY session after a GUI session is stopped when compiled
> with musl libc.
> 
> The specific systemd integration issue is currently resolved in the musl
> master branch by closer aligning the implementation with glibc, but the
> underlying isatty() implementation is still flawed since it can return 0
> (false) for something that is definitely a TTY. Essentially both musl
> and glibc give up correctly handling this case on Linux due to
> inadequate/buggy kernel APIs.
> 
> Thus I am proposing this patch as a solution to fix the kernel APIs. An
> alternative fix could be to add a new IOCTL to specifically handle this,
> but that seems overkill.
> 
This is pretty cool as it fixes the /dev/console-is-a-disconnected-/dev/ttyS0
issue I found this year
https://github.com/systemd/systemd/pull/33690
and

https://lore.kernel.org/all/2669238.7s5MMGUR32@nerdopolis2/
namely on vt-less kernels that now have /dev/console as /dev/ttyS0 instead of
/dev/tty0 or a virtual device.


So yeah, when you have hardware and you've been using vt-enabled kernels
on it for years (where your /dev/ttyS0 doesn't even have an rs232 port, let
alone connected to anything), and then when you upgrade to a vt-less kernel,
sometime in the future, because of systemd's isatty() check failing, and
refusing to write to /dev/console, Plymouth's new log display facility in turn
gets far fewer logs from systemd, because of the ioctl failure in isatty()


MY idea for this solution was that kernels turning off CONFIG_VT_CONSOLE could
then turn on a CONFIG_NULL_TTY_CONSOLE that would get selected at the same time
the VT console would have, so that if distros choose to do so, the virtual
/dev/ttynull device to always work for User Space. (that way kernels that never
had CONFIG_VT_CONSOLE turned on won't have a behavior change as well)


(the one benefit to my idea is that Plymouth goes into text mode logging when
the console device is /dev/ttyS0, so Desktop distros would need to add a new
"plymouth.graphical" to their grub commands to get the splash to work again,
however when the console device is /dev/ttynull it automatically assumes that
the user wants a splash screen.) This is more specific to Plymouth though


However testing this patch and it also works, and this won't need a new
configuration change.
> Link: https://github.com/systemd/systemd/pull/34039
> Link: https://gitlab.alpinelinux.org/alpine/aports/-/merge_requests/70711
> Link: https://www.openwall.com/lists/musl/2024/08/20/2
> Link: https://git.musl-libc.org/cgit/musl/commit/src/unistd/isatty.c?id=c94a0c16f08894ce3be6dafb0fe80baa77a6ff2a
> Link: https://sourceware.org/bugzilla/show_bug.cgi?id=32103
> 
> Gil Pedersen (1):
>   tty: respond to TIOCGWINSZ when hung
> 
>  drivers/tty/tty_io.c | 29 ++++++++++++++++++++++-------
>  1 file changed, 22 insertions(+), 7 deletions(-)
> 
>