[Xen-devel] [PATCH] tools/configure: generate stubs and long-double 32-bit headers if needed

Christopher Clark posted 1 patch 4 years, 1 month ago
Patches applied successfully (tree, apply log)
git fetch https://github.com/patchew-project/xen tags/patchew/20200210043516.1996-1-christopher.w.clark@gmail.com
tools/configure    | 48 ++++++++++++++++++++++++++++++++++++++++++++++
tools/configure.ac | 24 +++++++++++++++++++++++
2 files changed, 72 insertions(+)
[Xen-devel] [PATCH] tools/configure: generate stubs and long-double 32-bit headers if needed
Posted by Christopher Clark 4 years, 1 month ago
The gnu/stubs-32.h and bits/long-double-32.h headers are required to
build hvmloader but are not always available in 64-bit build
environments. To avoid introducing a build requirement on 32-bit
multilib generate versions of them from the 64-bit equivalent header.

This patch enables the removal of downstream patching that has been
carried in the Yocto/OpenEmbedded meta-virtualization layer since 2012.

Signed-off-by: Christopher Clark <christopher.clark6@baesystems.com>
---
 tools/configure    | 48 ++++++++++++++++++++++++++++++++++++++++++++++
 tools/configure.ac | 24 +++++++++++++++++++++++
 2 files changed, 72 insertions(+)

diff --git a/tools/configure b/tools/configure
index 977a8837c3..29cca6267a 100755
--- a/tools/configure
+++ b/tools/configure
@@ -9769,7 +9769,55 @@ else
   systemd=n
 fi
 
+# If 32-bit stubs header is not already available, check for the 64-bit one
+# and generate a 32-bit stubs header
+ac_fn_c_check_header_mongrel "$LINENO" "gnu/stubs-32.h" "ac_cv_header_gnu_stubs_32_h" "$ac_includes_default"
+if test "x$ac_cv_header_gnu_stubs_32_h" = xyes; then :
 
+else
+
+    ac_fn_c_check_header_mongrel "$LINENO" "gnu/stubs-64.h" "ac_cv_header_gnu_stubs_64_h" "$ac_includes_default"
+if test "x$ac_cv_header_gnu_stubs_64_h" = xyes; then :
+
+        echo '#include <gnu/stubs-64.h>' >conf-stubs.c
+        SIXTY_FOUR_HDR=`$CC -M conf-stubs.c | grep '/stubs-64.h$'`
+        rm conf-stubs.c
+        mkdir -p include/gnu
+        cat "${SIXTY_FOUR_HDR}" | \
+            grep -v 'stub_bdflush\|stub_getmsg\|stub_putmsg' >include/gnu/stubs-32.h
+        echo \#define __stub___kernel_cosl >> include/gnu/stubs-32.h
+        echo \#define __stub___kernel_sinl >> include/gnu/stubs-32.h
+        echo \#define __stub___kernel_tanl >> include/gnu/stubs-32.h
+
+else
+  as_fn_error $? "" "$LINENO" 5No gnu stubs headers found
+fi
+
+
+fi
+
+# If 32-bit long-double header is not already available, check for the 64-bit
+# one and copy it
+ac_fn_c_check_header_mongrel "$LINENO" "bits/long-double-32.h" "ac_cv_header_bits_long_double_32_h" "$ac_includes_default"
+if test "x$ac_cv_header_bits_long_double_32_h" = xyes; then :
+
+else
+
+     ac_fn_c_check_header_mongrel "$LINENO" "bits/long-double-64.h" "ac_cv_header_bits_long_double_64_h" "$ac_includes_default"
+if test "x$ac_cv_header_bits_long_double_64_h" = xyes; then :
+
+        echo '#include <bits/long-double-64.h>' >conf-double.c
+        SIXTY_FOUR_HDR=`$CC -M conf-double.c | grep '/long-double-64.h$'`
+        rm conf-double.c
+        mkdir -p include/bits
+        cat "${SIXTY_FOUR_HDR}" >include/bits/long-double-32.h
+
+else
+  as_fn_error $? "" "$LINENO" 5No long-double headers found
+fi
+
+
+fi
 
 if test "x$systemd" = "xy"; then :
 
diff --git a/tools/configure.ac b/tools/configure.ac
index 8d86c42de8..769406e9ca 100644
--- a/tools/configure.ac
+++ b/tools/configure.ac
@@ -488,4 +488,28 @@ AC_ARG_ENABLE([pvshim],
 ])
 AC_SUBST(pvshim)
 
+# If 32-bit stubs header is not already available, check for the 64-bit one
+# and generate a 32-bit stubs header
+AC_CHECK_HEADER([gnu/stubs-32.h], [], [
+    AC_CHECK_HEADER([gnu/stubs-64.h], [
+        echo '#include <gnu/stubs-64.h>' >conf-stubs.c
+        SIXTY_FOUR_HDR=`$CC -M conf-stubs.c | grep '/stubs-64.h$'`
+        rm conf-stubs.c
+        mkdir -p include/gnu
+        cat "${SIXTY_FOUR_HDR}" | \
+            grep -v 'stub_bdflush\|stub_getmsg\|stub_putmsg' >include/gnu/stubs-32.h
+        echo \#define __stub___kernel_cosl >> include/gnu/stubs-32.h
+        echo \#define __stub___kernel_sinl >> include/gnu/stubs-32.h
+        echo \#define __stub___kernel_tanl >> include/gnu/stubs-32.h
+    ], [AC_MSG_ERROR[No gnu stubs headers found]], []) ], [])
+
+AC_CHECK_HEADER([bits/long-double-32.h], [], [
+     AC_CHECK_HEADER([bits/long-double-64.h], [
+        echo '#include <bits/long-double-64.h>' >conf-double.c
+        SIXTY_FOUR_HDR=`$CC -M conf-double.c | grep '/long-double-64.h$'`
+        rm conf-double.c
+        mkdir -p include/bits
+        cat "${SIXTY_FOUR_HDR}" >include/bits/long-double-32.h
+    ], [AC_MSG_ERROR[No long-double headers found]], []) ], [])
+
 AC_OUTPUT()
-- 
2.17.1


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
[Xen-devel] [PATCH] tools/configure: generate stubs and long-double 32-bit headers if needed
Posted by Ian Jackson 4 years, 1 month ago
Christopher Clark writes ("[PATCH] tools/configure: generate stubs and long-double 32-bit headers if needed"):
> The gnu/stubs-32.h and bits/long-double-32.h headers are required to
> build hvmloader but are not always available in 64-bit build
> environments. To avoid introducing a build requirement on 32-bit
> multilib generate versions of them from the 64-bit equivalent header.
> 
> This patch enables the removal of downstream patching that has been
> carried in the Yocto/OpenEmbedded meta-virtualization layer since 2012.

Thanks for this patch.  However, I'm quite doubtful about the
approach.

> +        echo '#include <gnu/stubs-64.h>' >conf-stubs.c
> +        SIXTY_FOUR_HDR=`$CC -M conf-stubs.c | grep '/stubs-64.h$'`
> +        rm conf-stubs.c
> +        mkdir -p include/gnu
> +        cat "${SIXTY_FOUR_HDR}" | \
> +            grep -v 'stub_bdflush\|stub_getmsg\|stub_putmsg' >include/gnu/stubs-32.h
> +        echo \#define __stub___kernel_cosl >> include/gnu/stubs-32.h
> +        echo \#define __stub___kernel_sinl >> include/gnu/stubs-32.h
> +        echo \#define __stub___kernel_tanl >> include/gnu/stubs-32.h

This code seems to be ad-hoc seddery which depends on the details of
the existing header file.  That seems like it is unprincipled and
fragile, to me.

I don't understand why you wouldn't just make a small package or
tarball or something containing the relevant headers and install
that ?

Also, don't you need a 32-bit libgcc too ?

Thanks,
Ian.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [Xen-devel] [PATCH] tools/configure: generate stubs and long-double 32-bit headers if needed
Posted by Christopher Clark 4 years, 1 month ago
On Mon, Feb 10, 2020 at 8:21 AM Ian Jackson <ian.jackson@citrix.com> wrote:
>
> Christopher Clark writes ("[PATCH] tools/configure: generate stubs and long-double 32-bit headers if needed"):
> > The gnu/stubs-32.h and bits/long-double-32.h headers are required to
> > build hvmloader but are not always available in 64-bit build
> > environments. To avoid introducing a build requirement on 32-bit
> > multilib generate versions of them from the 64-bit equivalent header.
> >
> > This patch enables the removal of downstream patching that has been
> > carried in the Yocto/OpenEmbedded meta-virtualization layer since 2012.
>
> Thanks for this patch.  However, I'm quite doubtful about the
> approach.

Thanks for the review and recommendation for an alternative direction.

> > +        echo '#include <gnu/stubs-64.h>' >conf-stubs.c
> > +        SIXTY_FOUR_HDR=`$CC -M conf-stubs.c | grep '/stubs-64.h$'`
> > +        rm conf-stubs.c
> > +        mkdir -p include/gnu
> > +        cat "${SIXTY_FOUR_HDR}" | \
> > +            grep -v 'stub_bdflush\|stub_getmsg\|stub_putmsg' >include/gnu/stubs-32.h
> > +        echo \#define __stub___kernel_cosl >> include/gnu/stubs-32.h
> > +        echo \#define __stub___kernel_sinl >> include/gnu/stubs-32.h
> > +        echo \#define __stub___kernel_tanl >> include/gnu/stubs-32.h
>
> This code seems to be ad-hoc seddery which depends on the details of
> the existing header file.  That seems like it is unprincipled and
> fragile, to me.
>
> I don't understand why you wouldn't just make a small package or
> tarball or something containing the relevant headers and install
> that ?
>
> Also, don't you need a 32-bit libgcc too ?

OK, I've produced a revised proposal downstream that retires this
header file generation altogether and applies OpenEmbedded's multilib
support  to make a populated 32-bit sysroot with headers and libraries
available to build with.

The remaining challenge is that the OE build populates and makes this
32-bit sysroot available in a different directory to the primary
target architecture sysroot, so: do you have a recommendation for how
to plumb an additional CFLAG into Xen's tools/firmware build?

At the moment, I'm appending this single line:
CFLAGS += "--sysroot=/this/is/my/sysroot32"

to tools/firmware/Rules.mk but I think a dedicated top-level variable
for the purpose, plumbed through, could be more appropriate?

Christopher

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [Xen-devel] [PATCH] tools/configure: generate stubs and long-double 32-bit headers if needed
Posted by Roger Pau Monné 4 years, 1 month ago
On Sun, Feb 09, 2020 at 08:35:14PM -0800, Christopher Clark wrote:
> The gnu/stubs-32.h and bits/long-double-32.h headers are required to
> build hvmloader but are not always available in 64-bit build
> environments. To avoid introducing a build requirement on 32-bit
> multilib generate versions of them from the 64-bit equivalent header.
> 
> This patch enables the removal of downstream patching that has been
> carried in the Yocto/OpenEmbedded meta-virtualization layer since 2012.
> 
> Signed-off-by: Christopher Clark <christopher.clark6@baesystems.com>

I think this would be better done in tools/include as part of
populating tools/include.

Also this looks specific to using gcc to build the tools, which could
be skipped when building with clang?

Thanks, Roger.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
[Xen-devel] [PATCH 1/2] pygrub: fix python3 cross-compile: install with INSTALL_PYTHON_PROG
Posted by Christopher Clark 4 years, 1 month ago
Install pygrub with INSTALL_PYTHON_PROG, as per the other Xen python
executables, to ensure that the hashbang path to the interpreter
is written correctly in cross-compile builds, eg. with OpenEmbedded.

Signed-off-by: Christopher Clark <christopher.clark6@baesystems.com>
---
 tools/pygrub/Makefile | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/tools/pygrub/Makefile b/tools/pygrub/Makefile
index 3063c4998f..b4f6f10ddd 100644
--- a/tools/pygrub/Makefile
+++ b/tools/pygrub/Makefile
@@ -18,6 +18,8 @@ install: all
 	CC="$(CC)" CFLAGS="$(PY_CFLAGS)" LDFLAGS="$(PY_LDFLAGS)" $(PYTHON) \
 		setup.py install --record $(INSTALL_LOG) $(PYTHON_PREFIX_ARG) \
 		 --root="$(DESTDIR)" --install-scripts=$(LIBEXEC_BIN) --force
+	rm -f $(DESTDIR)/$(LIBEXEC_BIN)/pygrub
+	$(INSTALL_PYTHON_PROG) src/pygrub $(DESTDIR)/$(LIBEXEC_BIN)/pygrub
 	set -e; if [ $(bindir) != $(LIBEXEC_BIN) -a \
 	             "`readlink -f $(DESTDIR)/$(bindir)`" != \
 	             "`readlink -f $(LIBEXEC_BIN)`" ]; then \
-- 
2.17.1


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [Xen-devel] [PATCH 1/2] pygrub: fix python3 cross-compile: install with INSTALL_PYTHON_PROG
Posted by Ian Jackson 4 years, 1 month ago
Christopher Clark writes ("[PATCH 1/2] pygrub: fix python3 cross-compile: install with INSTALL_PYTHON_PROG"):
> Install pygrub with INSTALL_PYTHON_PROG, as per the other Xen python
> executables, to ensure that the hashbang path to the interpreter
> is written correctly in cross-compile builds, eg. with OpenEmbedded.

Hrm.  There is definitely a bug here and I think
tools/python/install-wrap needs to be called.

What I don't understand is...

> diff --git a/tools/pygrub/Makefile b/tools/pygrub/Makefile
> index 3063c4998f..b4f6f10ddd 100644
> --- a/tools/pygrub/Makefile
> +++ b/tools/pygrub/Makefile
> @@ -18,6 +18,8 @@ install: all
>  	CC="$(CC)" CFLAGS="$(PY_CFLAGS)" LDFLAGS="$(PY_LDFLAGS)" $(PYTHON) \
>  		setup.py install --record $(INSTALL_LOG) $(PYTHON_PREFIX_ARG) \
>  		 --root="$(DESTDIR)" --install-scripts=$(LIBEXEC_BIN) --force
> +	rm -f $(DESTDIR)/$(LIBEXEC_BIN)/pygrub
> +	$(INSTALL_PYTHON_PROG) src/pygrub $(DESTDIR)/$(LIBEXEC_BIN)/pygrub
>  	set -e; if [ $(bindir) != $(LIBEXEC_BIN) -a \
>  	             "`readlink -f $(DESTDIR)/$(bindir)`" != \
>  	             "`readlink -f $(LIBEXEC_BIN)`" ]; then \

... why this is the right approach in tools/pygrub when it is *not*
the approach used in tools/python, where install-wrap lives, and which
is the other directory which has a setup.py.

tools/python seems to use $(INSTALL_PROG) and not have anything in
`scripts' in setup.py.  Is that wrong, too ?

Perhaps instead of the rm, pygrub/setup.py should lose the line
      scripts = ["src/pygrub"],
?

Or is there maybe a way to get setup.py to use a different `install' ?

Thanks,
Ian.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
[Xen-devel] [PATCH 2/2] python, pygrub: pass DISTUTILS env vars as setup.py args
Posted by Christopher Clark 4 years, 1 month ago
Allow to respect the target install dir (PYTHON_SITEPACKAGES_DIR)
as well as other parameters set by the OpenEmbedded build system.
This is especially useful when the target libdir is not the default one
(/usr/lib), but for example /usr/lib64.

Signed-off-by: Maciej Pijanowski <maciej.pijanowski@3mdeb.com>

This enables the distro build system to pass additional args to the
python setup.py build and install commands.
Signed-off-by: Christopher Clark <christopher.clark6@baesystems.com>
---
Original patch from the Yocto/OpenEmbedded meta-virtualization layer.
Rewrapped and rebased onto the adjacent commit submitted in this series.

 tools/pygrub/Makefile | 6 ++++--
 tools/python/Makefile | 4 ++--
 2 files changed, 6 insertions(+), 4 deletions(-)

diff --git a/tools/pygrub/Makefile b/tools/pygrub/Makefile
index b4f6f10ddd..430ceeed16 100644
--- a/tools/pygrub/Makefile
+++ b/tools/pygrub/Makefile
@@ -10,14 +10,16 @@ INSTALL_LOG = build/installed_files.txt
 all: build
 .PHONY: build
 build:
-	CC="$(CC)" CFLAGS="$(PY_CFLAGS)" LDFLAGS="$(PY_LDFLAGS)" $(PYTHON) setup.py build
+	CC="$(CC)" CFLAGS="$(PY_CFLAGS)" LDFLAGS="$(PY_LDFLAGS)" $(PYTHON) \
+		setup.py build $(DISTUTILS_BUILD_ARGS)
 
 .PHONY: install
 install: all
 	$(INSTALL_DIR) $(DESTDIR)/$(bindir)
 	CC="$(CC)" CFLAGS="$(PY_CFLAGS)" LDFLAGS="$(PY_LDFLAGS)" $(PYTHON) \
 		setup.py install --record $(INSTALL_LOG) $(PYTHON_PREFIX_ARG) \
-		 --root="$(DESTDIR)" --install-scripts=$(LIBEXEC_BIN) --force
+		 --root="$(DESTDIR)" --install-scripts=$(LIBEXEC_BIN) --force \
+		$(DISTUTILS_INSTALL_ARGS)
 	rm -f $(DESTDIR)/$(LIBEXEC_BIN)/pygrub
 	$(INSTALL_PYTHON_PROG) src/pygrub $(DESTDIR)/$(LIBEXEC_BIN)/pygrub
 	set -e; if [ $(bindir) != $(LIBEXEC_BIN) -a \
diff --git a/tools/python/Makefile b/tools/python/Makefile
index e99f78a537..294f8ee4dd 100644
--- a/tools/python/Makefile
+++ b/tools/python/Makefile
@@ -10,7 +10,7 @@ INSTALL_LOG = build/installed_files.txt
 
 .PHONY: build
 build:
-	CC="$(CC)" CFLAGS="$(PY_CFLAGS)" $(PYTHON) setup.py build
+	CC="$(CC)" CFLAGS="$(PY_CFLAGS)" $(PYTHON) setup.py build $(DISTUTILS_BUILD_ARGS)
 
 .PHONY: install
 install:
@@ -18,7 +18,7 @@ install:
 
 	CC="$(CC)" CFLAGS="$(PY_CFLAGS)" LDFLAGS="$(PY_LDFLAGS)" $(PYTHON) \
 		setup.py install --record $(INSTALL_LOG) $(PYTHON_PREFIX_ARG) \
-		--root="$(DESTDIR)" --force
+		--root="$(DESTDIR)" --force $(DISTUTILS_INSTALL_ARGS)
 
 	$(INSTALL_PROG) scripts/convert-legacy-stream $(DESTDIR)$(LIBEXEC_BIN)
 	$(INSTALL_PROG) scripts/verify-stream-v2 $(DESTDIR)$(LIBEXEC_BIN)
-- 
2.17.1


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [Xen-devel] [PATCH 2/2] python, pygrub: pass DISTUTILS env vars as setup.py args
Posted by Ian Jackson 4 years, 1 month ago
Christopher Clark writes ("[PATCH 2/2] python, pygrub: pass DISTUTILS env vars as setup.py args"):
> Allow to respect the target install dir (PYTHON_SITEPACKAGES_DIR)
> as well as other parameters set by the OpenEmbedded build system.
> This is especially useful when the target libdir is not the default one
> (/usr/lib), but for example /usr/lib64.
> 
> Signed-off-by: Maciej Pijanowski <maciej.pijanowski@3mdeb.com>
> 
> This enables the distro build system to pass additional args to the
> python setup.py build and install commands.
> Signed-off-by: Christopher Clark <christopher.clark6@baesystems.com>

Thanks.  The overall idea here is very sound and I would like to take
this patch in some form.  But, and I hope this is not too annoying, I
have a couple of observations/questions:

Firstly, the commit message mentions PYTHON_SITEPACKAGES_DIR which is
a thing which does not appears in this commit.  AIUI the OpenEmbedded
build system honours that and implements it by setting
DISTUTILS_BUILD_ARGS and DISTUTILS_INSTALL_ARGS.  I think this needs
to be explained correctly in the commit message.

Secondly, it bothers me that the env var name does not mention python
at all.  Would it be OK to change it to PYDISTUTILS_... or something ?

Thanks,
Ian.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel