[PATCH 00/16] Cross compilation of embedded firmware

Paolo Bonzini posted 16 patches 1 year, 10 months ago
Patches applied successfully (tree, apply log)
git fetch https://github.com/patchew-project/qemu tags/patchew/20220517092616.1272238-1-pbonzini@redhat.com
Maintainers: Cornelia Huck <cohuck@redhat.com>, Thomas Huth <thuth@redhat.com>, Christian Borntraeger <borntraeger@linux.ibm.com>, Alexey Kardashevskiy <aik@ozlabs.ru>, "Cédric Le Goater" <clg@kaod.org>, Daniel Henrique Barboza <danielhb413@gmail.com>, David Gibson <david@gibson.dropbear.id.au>, Greg Kurz <groug@kaod.org>, John Snow <jsnow@redhat.com>, Cleber Rosa <crosa@redhat.com>
Makefile                     |   9 +-
configure                    | 605 ++++++++++++++++++++++++++++++-----
pc-bios/meson.build          |  18 +-
pc-bios/optionrom/Makefile   |   4 +-
pc-bios/s390-ccw/Makefile    |   9 +-
pc-bios/s390-ccw/netboot.mak |   2 +-
pc-bios/vof/Makefile         |  17 +-
scripts/mtest2make.py        |   8 +-
tests/Makefile.include       |   4 +-
tests/tcg/configure.sh       | 376 ----------------------
10 files changed, 570 insertions(+), 482 deletions(-)
delete mode 100755 tests/tcg/configure.sh
[PATCH 00/16] Cross compilation of embedded firmware
Posted by Paolo Bonzini 1 year, 10 months ago
This is the next part of the firmware cross compilation story.  It only
looks at firmware that's strictly part of QEMU (optionrom, s390-ccw,
vof), and does not do docker cross compilers yet; but it takes the
infrastructure from tests/tcg/configure.sh and moves it in the main
configure script so that others can use it.

I actually expect the compiler tests to move back to tests/tcg, running
at Make time after the docker images are built.  For now, the file is
moved as a whole, including both compiler detection and the tests.

Paolo

RFC->v1:
- new patches 1-4, 6, 16
- patches 9-10 have been merged already

Paolo Bonzini (16):
  configure: do not define or use the CPP variable
  build: clean up ninja invocation
  build: add a more generic way to specify make->ninja dependencies
  build: do a full build before running TCG tests
  configure, meson: move symlinking of ROMs to meson
  tests/tcg: correct target CPU for sparc32
  tests/tcg: merge configure.sh back into main configure script
  configure: add missing cross compiler fallbacks
  configure: handle host compiler in probe_target_compiler
  configure: introduce --cross-prefix-*=
  configure: include more binutils in tests/tcg makefile
  configure: move symlink configuration earlier
  configure: enable cross-compilation of s390-ccw
  configure: enable cross-compilation of optionrom
  configure: enable cross compilation of vof
  configure: remove unused variables from config-host.mak

 Makefile                     |   9 +-
 configure                    | 605 ++++++++++++++++++++++++++++++-----
 pc-bios/meson.build          |  18 +-
 pc-bios/optionrom/Makefile   |   4 +-
 pc-bios/s390-ccw/Makefile    |   9 +-
 pc-bios/s390-ccw/netboot.mak |   2 +-
 pc-bios/vof/Makefile         |  17 +-
 scripts/mtest2make.py        |   8 +-
 tests/Makefile.include       |   4 +-
 tests/tcg/configure.sh       | 376 ----------------------
 10 files changed, 570 insertions(+), 482 deletions(-)
 delete mode 100755 tests/tcg/configure.sh

-- 
2.36.0
Re: [PATCH 00/16] Cross compilation of embedded firmware
Posted by Alex Bennée 1 year, 10 months ago
Paolo Bonzini <pbonzini@redhat.com> writes:

> This is the next part of the firmware cross compilation story.  It only
> looks at firmware that's strictly part of QEMU (optionrom, s390-ccw,
> vof), and does not do docker cross compilers yet; but it takes the
> infrastructure from tests/tcg/configure.sh and moves it in the main
> configure script so that others can use it.
>
> I actually expect the compiler tests to move back to tests/tcg, running
> at Make time after the docker images are built.  For now, the file is
> moved as a whole, including both compiler detection and the tests.

Queued to testing/next, thanks.

-- 
Alex Bennée
Re: [PATCH 00/16] Cross compilation of embedded firmware
Posted by Alex Bennée 1 year, 10 months ago
Paolo Bonzini <pbonzini@redhat.com> writes:

> This is the next part of the firmware cross compilation story.  It only
> looks at firmware that's strictly part of QEMU (optionrom, s390-ccw,
> vof), and does not do docker cross compilers yet; but it takes the
> infrastructure from tests/tcg/configure.sh and moves it in the main
> configure script so that others can use it.
>
> I actually expect the compiler tests to move back to tests/tcg, running
> at Make time after the docker images are built.  For now, the file is
> moved as a whole, including both compiler detection and the tests.

Isn't that just creating a bunch of unnecessary churn? The
tests/tcg/configure.sh sciprt already has a bunch of special casing in
it to detect various compiler features for the purposes of tests. It
would be nice if we could keep it's history of warts intact.

>
> Paolo
>
> RFC->v1:
> - new patches 1-4, 6, 16
> - patches 9-10 have been merged already
>
> Paolo Bonzini (16):
>   configure: do not define or use the CPP variable
>   build: clean up ninja invocation
>   build: add a more generic way to specify make->ninja dependencies
>   build: do a full build before running TCG tests
>   configure, meson: move symlinking of ROMs to meson
>   tests/tcg: correct target CPU for sparc32
>   tests/tcg: merge configure.sh back into main configure script
>   configure: add missing cross compiler fallbacks
>   configure: handle host compiler in probe_target_compiler
>   configure: introduce --cross-prefix-*=
>   configure: include more binutils in tests/tcg makefile
>   configure: move symlink configuration earlier
>   configure: enable cross-compilation of s390-ccw
>   configure: enable cross-compilation of optionrom
>   configure: enable cross compilation of vof
>   configure: remove unused variables from config-host.mak
>
>  Makefile                     |   9 +-
>  configure                    | 605 ++++++++++++++++++++++++++++++-----
>  pc-bios/meson.build          |  18 +-
>  pc-bios/optionrom/Makefile   |   4 +-
>  pc-bios/s390-ccw/Makefile    |   9 +-
>  pc-bios/s390-ccw/netboot.mak |   2 +-
>  pc-bios/vof/Makefile         |  17 +-
>  scripts/mtest2make.py        |   8 +-
>  tests/Makefile.include       |   4 +-
>  tests/tcg/configure.sh       | 376 ----------------------
>  10 files changed, 570 insertions(+), 482 deletions(-)
>  delete mode 100755 tests/tcg/configure.sh


-- 
Alex Bennée
Re: [PATCH 00/16] Cross compilation of embedded firmware
Posted by Paolo Bonzini 1 year, 10 months ago
On 5/24/22 17:53, Alex Bennée wrote:
> 
> Paolo Bonzini <pbonzini@redhat.com> writes:
> 
>> This is the next part of the firmware cross compilation story.  It only
>> looks at firmware that's strictly part of QEMU (optionrom, s390-ccw,
>> vof), and does not do docker cross compilers yet; but it takes the
>> infrastructure from tests/tcg/configure.sh and moves it in the main
>> configure script so that others can use it.
>>
>> I actually expect the compiler tests to move back to tests/tcg, running
>> at Make time after the docker images are built.  For now, the file is
>> moved as a whole, including both compiler detection and the tests.
> 
> Isn't that just creating a bunch of unnecessary churn? The
> tests/tcg/configure.sh sciprt already has a bunch of special casing in
> it to detect various compiler features for the purposes of tests. It
> would be nice if we could keep it's history of warts intact.

Honestly, I don't know what things will look like one year from now.  I 
don't have 100% of the vision of how things will look, just enough to 
make informed guesses at each step.  So far I think it's gone pretty 
well, at the expense of some churn indeed.

What I know If the compiler tests do move back to tests/tcg, they 
probably won't look anything like the code that is there now.

In fact, they might even be done in Makefiles instead of configure, 
similar to the "cc-option" macro in pc-bios/optionrom/Makefile.  My plan 
was to experiment with that in pc-bios/optionrom, in the context of 
supporting docker-based cross-compilers, and then move whatever solution 
I come up with to tests/tcg.

If you prefer that I first try moving them to Makefiles to reduce future 
churn in the main configure script, I can do that, but probably not 
until ~August.

Paolo

Re: [PATCH 00/16] Cross compilation of embedded firmware
Posted by Paolo Bonzini 1 year, 10 months ago
On 5/25/22 15:16, Paolo Bonzini wrote:
> On 5/24/22 17:53, Alex Bennée wrote:
>> Paolo Bonzini <pbonzini@redhat.com> writes:
>>> I actually expect the compiler tests to move back to tests/tcg, running
>>> at Make time after the docker images are built.  For now, the file is
>>> moved as a whole, including both compiler detection and the tests.
>>
>> Isn't that just creating a bunch of unnecessary churn? The
>> tests/tcg/configure.sh sciprt already has a bunch of special casing in
>> it to detect various compiler features for the purposes of tests. It
>> would be nice if we could keep it's history of warts intact.

For clarity the code that we're talking about is just this:

+      case $target in
+          aarch64-*)
+              if do_compiler "$target_cc" $target_cflags \
+                             -march=armv8.1-a+sve -o $TMPE $TMPC; then
+                  echo "CROSS_CC_HAS_SVE=y" >> $config_target_mak
+              fi
+              if do_compiler "$target_cc" $target_cflags \
+                             -march=armv8.1-a+sve2 -o $TMPE $TMPC; then
+                  echo "CROSS_CC_HAS_SVE2=y" >> $config_target_mak
+              fi
+              if do_compiler "$target_cc" $target_cflags \
+                             -march=armv8.3-a -o $TMPE $TMPC; then
+                  echo "CROSS_CC_HAS_ARMV8_3=y" >> $config_target_mak
+              fi
+              if do_compiler "$target_cc" $target_cflags \
+                             -mbranch-protection=standard -o $TMPE $TMPC; then
+                  echo "CROSS_CC_HAS_ARMV8_BTI=y" >> $config_target_mak
+              fi
+              if do_compiler "$target_cc" $target_cflags \
+                             -march=armv8.5-a+memtag -o $TMPE $TMPC; then
+                  echo "CROSS_CC_HAS_ARMV8_MTE=y" >> $config_target_mak
+              fi
+              ;;
+          ppc*)
+              if do_compiler "$target_cc" $target_cflags \
+                             -mpower8-vector -o $TMPE $TMPC; then
+                  echo "CROSS_CC_HAS_POWER8_VECTOR=y" >> $config_target_mak
+              fi
+              if do_compiler "$target_cc" $target_cflags \
+                             -mpower10 -o $TMPE $TMPC; then
+                  echo "CROSS_CC_HAS_POWER10=y" >> $config_target_mak
+              fi
+              ;;
+          i386-linux-user)
+              if do_compiler "$target_cc" $target_cflags \
+                             -Werror -fno-pie -o $TMPE $TMPC; then
+                  echo "CROSS_CC_HAS_I386_NOPIE=y" >> $config_target_mak
+              fi
+              ;;
+      esac

The other ~300 lines of configure.sh belong in the main configure script and
remain there.  So the churn is limited to the above.

Paolo