According to openpty(3) it is required to include <pty.h> to get the
prototypes for openpty() and login_tty(). But this is not what the
function AX_CHECK_PTYFUNCS actually does. It makes no attempt to include
the required header.

The two source files which call openpty() and login_tty() already contain
the conditionals to include the required header.

Remove the bogus m4 file to fix build with clang, which complains about
calls to undeclared functions.

Signed-off-by: Olaf Hering <olaf@aepfle.de>
---
 m4/ptyfuncs.m4     | 35 -----------------------------------
 tools/configure.ac |  1 -
 2 files changed, 36 deletions(-)
 delete mode 100644 m4/ptyfuncs.m4

diff --git a/m4/ptyfuncs.m4 b/m4/ptyfuncs.m4
deleted file mode 100644
index 3e37b5a23c..0000000000
--- a/m4/ptyfuncs.m4
+++ /dev/null
@@ -1,35 +0,0 @@
-AC_DEFUN([AX_CHECK_PTYFUNCS], [
-    dnl This is a workaround for a bug in Debian package
-    dnl libbsd-dev-0.3.0-1. Once we no longer support that
-    dnl package we can remove the addition of -Werror to
-    dnl CPPFLAGS.
-    AX_SAVEVAR_SAVE(CPPFLAGS)
-    CPPFLAGS="$CPPFLAGS -Werror"
-    AC_CHECK_HEADER([libutil.h],[
-      AC_DEFINE([INCLUDE_LIBUTIL_H],[<libutil.h>],[libutil header file name])
-    ])
-    AX_SAVEVAR_RESTORE(CPPFLAGS)
-    AC_CACHE_CHECK([for openpty et al], [ax_cv_ptyfuncs_libs], [
-        for ax_cv_ptyfuncs_libs in -lutil "" NOT_FOUND; do
-            if test "x$ax_cv_ptyfuncs_libs" = "xNOT_FOUND"; then
-                AC_MSG_FAILURE([Unable to find library for openpty and login_tty])
-            fi
-            AX_SAVEVAR_SAVE(LIBS)
-            LIBS="$LIBS $ax_cv_ptyfuncs_libs"
-            AC_LINK_IFELSE([AC_LANG_SOURCE([
-#ifdef INCLUDE_LIBUTIL_H
-#include INCLUDE_LIBUTIL_H
-#endif
-int main(void) {
-  openpty(0,0,0,0,0);
-  login_tty(0);
-}
-])],[
-                break
-            ],[])
-            AX_SAVEVAR_RESTORE(LIBS)
-        done
-    ])
-    PTYFUNCS_LIBS="$ax_cv_ptyfuncs_libs"
-    AC_SUBST(PTYFUNCS_LIBS)
-])
diff --git a/tools/configure.ac b/tools/configure.ac
index 9bcf42f233..c94257f751 100644
--- a/tools/configure.ac
+++ b/tools/configure.ac
@@ -70,7 +70,6 @@ m4_include([../m4/uuid.m4])
 m4_include([../m4/pkg.m4])
 m4_include([../m4/curses.m4])
 m4_include([../m4/pthread.m4])
-m4_include([../m4/ptyfuncs.m4])
 m4_include([../m4/extfs.m4])
 m4_include([../m4/fetcher.m4])
 m4_include([../m4/ax_compare_version.m4])

On Tue, May 02, 2023 at 08:48:00PM +0000, Olaf Hering wrote:
> According to openpty(3) it is required to include <pty.h> to get the
> prototypes for openpty() and login_tty(). But this is not what the
> function AX_CHECK_PTYFUNCS actually does. It makes no attempt to include
> the required header.
> 
> The two source files which call openpty() and login_tty() already contain
> the conditionals to include the required header.
> 
> Remove the bogus m4 file to fix build with clang, which complains about
> calls to undeclared functions.
> 
> Signed-off-by: Olaf Hering <olaf@aepfle.de>

That change isn't enough. And I'm not convinced that it needs to be
removed.

First, AX_CHECK_PTYFUNCS is still called in "tools/configure.ac".

Then, AX_CHECK_PTYFUNCS define INCLUDE_LIBUTIL_H and PTYFUNCS_LIBS.
Those two are still used in the tree.

Also, that that macro isn't just about the header, but also about the
needed library.

Thanks,

-- 
Anthony PERARD

Tue, 9 May 2023 17:47:33 +0100 Anthony PERARD <anthony.perard@citrix.com>:

> That change isn't enough. And I'm not convinced that it needs to be
> removed.

You are right, the provided functions must be removed as well.
My build scripts do not run autoreconf, perhaps it is about time to
change that.

> First, AX_CHECK_PTYFUNCS is still called in "tools/configure.ac".

This needs to be removed, yes.

> Then, AX_CHECK_PTYFUNCS define INCLUDE_LIBUTIL_H and PTYFUNCS_LIBS.

This is used inconsistently. Some places include <libutil.h> unconditionally.
-lutil is already used unconditionally via UTIL_LIBS

> Also, that that macro isn't just about the header, but also about the
> needed library.

This is already covered.

Olaf