[PULL 0/8] s390x and misc fixes

Thomas Huth posted 8 patches 2 years, 1 month ago
Test checkpatch failed
Patches applied successfully (tree, apply log)
git fetch https://github.com/patchew-project/qemu tags/patchew/20220315112052.515467-1-thuth@redhat.com
Maintainers: Richard Henderson <richard.henderson@linaro.org>, Paolo Bonzini <pbonzini@redhat.com>, Eduardo Habkost <eduardo@habkost.net>, Marcel Apfelbaum <marcel.apfelbaum@gmail.com>, "Philippe Mathieu-Daudé" <f4bug@amsat.org>, Yanan Wang <wangyanan55@huawei.com>, David Hildenbrand <david@redhat.com>, Cornelia Huck <cohuck@redhat.com>, Thomas Huth <thuth@redhat.com>, Cleber Rosa <crosa@redhat.com>, Wainer dos Santos Moschetta <wainersm@redhat.com>, Beraldo Leal <bleal@redhat.com>
There is a newer version of this series
accel/tcg/cputlb.c                     |  2 +-
configure                              |  2 +-
cpu.c                                  |  9 +++++
include/exec/cpu-common.h              |  2 +
include/sysemu/cpus.h                  |  2 -
meson                                  |  2 +-
softmmu/cpus.c                         |  8 ----
target/s390x/tcg/translate.c           |  4 +-
tests/avocado/avocado_qemu/__init__.py | 13 ++++---
tests/tcg/s390x/Makefile.target        |  3 +-
tests/tcg/s390x/branch-relative-long.c | 68 ++++++++++++++++++++++++++++++++++
11 files changed, 92 insertions(+), 23 deletions(-)
create mode 100644 tests/tcg/s390x/branch-relative-long.c
[PULL 0/8] s390x and misc fixes
Posted by Thomas Huth 2 years, 1 month ago
 Hi Peter!

The following changes since commit 352998df1c53b366413690d95b35f76d0721ebed:

  Merge tag 'i2c-20220314' of https://github.com/philmd/qemu into staging (2022-03-14 14:39:33 +0000)

are available in the Git repository at:

  https://gitlab.com/thuth/qemu.git tags/pull-request-2022-03-15

for you to fetch changes up to 36149534792dcf07a3c59867f967eaee23ab906c:

  meson: Update to version 0.61.3 (2022-03-15 10:32:36 +0100)

----------------------------------------------------------------
* Fixes for s390x branch instruction emulation
* Fixes for the tests/avocado/boot_linux.py:BootLinuxS390X test
* Fix for "-cpu help" output
* Bump meson to 0.61.3 to fix stderr log of the iotests

----------------------------------------------------------------
Alex Bennée (1):
      tests/tcg: drop -cpu max from s390x sha512-mvx invocation

Beraldo Leal (1):
      tests/avocado: start PhoneServer upfront

Ilya Leoshkevich (3):
      s390x/tcg: Fix BRASL with a large negative offset
      s390x/tcg: Fix BRCL with a large negative offset
      tests/tcg/s390x: Test BRASL and BRCL with large negative offsets

Philippe Mathieu-Daudé (1):
      softmmu: List CPU types again

Richard Henderson (1):
      accel/tcg: Fix cpu_ldq_be_mmu typo

Thomas Huth (1):
      meson: Update to version 0.61.3

 accel/tcg/cputlb.c                     |  2 +-
 configure                              |  2 +-
 cpu.c                                  |  9 +++++
 include/exec/cpu-common.h              |  2 +
 include/sysemu/cpus.h                  |  2 -
 meson                                  |  2 +-
 softmmu/cpus.c                         |  8 ----
 target/s390x/tcg/translate.c           |  4 +-
 tests/avocado/avocado_qemu/__init__.py | 13 ++++---
 tests/tcg/s390x/Makefile.target        |  3 +-
 tests/tcg/s390x/branch-relative-long.c | 68 ++++++++++++++++++++++++++++++++++
 11 files changed, 92 insertions(+), 23 deletions(-)
 create mode 100644 tests/tcg/s390x/branch-relative-long.c


Re: [PULL 0/8] s390x and misc fixes
Posted by Peter Maydell 2 years, 1 month ago
On Tue, 15 Mar 2022 at 11:20, Thomas Huth <thuth@redhat.com> wrote:
>
>  Hi Peter!
>
> The following changes since commit 352998df1c53b366413690d95b35f76d0721ebed:
>
>   Merge tag 'i2c-20220314' of https://github.com/philmd/qemu into staging (2022-03-14 14:39:33 +0000)
>
> are available in the Git repository at:
>
>   https://gitlab.com/thuth/qemu.git tags/pull-request-2022-03-15
>
> for you to fetch changes up to 36149534792dcf07a3c59867f967eaee23ab906c:
>
>   meson: Update to version 0.61.3 (2022-03-15 10:32:36 +0100)
>
> ----------------------------------------------------------------
> * Fixes for s390x branch instruction emulation
> * Fixes for the tests/avocado/boot_linux.py:BootLinuxS390X test
> * Fix for "-cpu help" output
> * Bump meson to 0.61.3 to fix stderr log of the iotests
>
> ----------------------------------------------------------------

This results in every "Linking" step on my macos box producing the
warning:

ld: warning: directory not found for option
'-Lns/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/12.0.0'

Obvious suspect here is the new meson version.

thanks
-- PMM
Re: [PULL 0/8] s390x and misc fixes
Posted by Paolo Bonzini 2 years, 1 month ago
On 3/15/22 19:58, Peter Maydell wrote:
> On Tue, 15 Mar 2022 at 11:20, Thomas Huth <thuth@redhat.com> wrote:
>>
>>   Hi Peter!
>>
>> The following changes since commit 352998df1c53b366413690d95b35f76d0721ebed:
>>
>>    Merge tag 'i2c-20220314' of https://github.com/philmd/qemu into staging (2022-03-14 14:39:33 +0000)
>>
>> are available in the Git repository at:
>>
>>    https://gitlab.com/thuth/qemu.git tags/pull-request-2022-03-15
>>
>> for you to fetch changes up to 36149534792dcf07a3c59867f967eaee23ab906c:
>>
>>    meson: Update to version 0.61.3 (2022-03-15 10:32:36 +0100)
>>
>> ----------------------------------------------------------------
>> * Fixes for s390x branch instruction emulation
>> * Fixes for the tests/avocado/boot_linux.py:BootLinuxS390X test
>> * Fix for "-cpu help" output
>> * Bump meson to 0.61.3 to fix stderr log of the iotests
>>
>> ----------------------------------------------------------------
> 
> This results in every "Linking" step on my macos box producing the
> warning:
> 
> ld: warning: directory not found for option
> '-Lns/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/12.0.0'
> 
> Obvious suspect here is the new meson version.
> 

Can you send the meson-logs/meson-log.txt and build.ninja files?  Thanks,

Paolo
Re: [PULL 0/8] s390x and misc fixes
Posted by Peter Maydell 2 years, 1 month ago
On Wed, 16 Mar 2022 at 12:18, Paolo Bonzini <pbonzini@redhat.com> wrote:
>
> On 3/15/22 19:58, Peter Maydell wrote:
> > On Tue, 15 Mar 2022 at 11:20, Thomas Huth <thuth@redhat.com> wrote:
> > This results in every "Linking" step on my macos box producing the
> > warning:
> >
> > ld: warning: directory not found for option
> > '-Lns/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/12.0.0'
> >
> > Obvious suspect here is the new meson version.
> >
>
> Can you send the meson-logs/meson-log.txt and build.ninja files?

https://people.linaro.org/~peter.maydell/meson-log.txt
https://people.linaro.org/~peter.maydell/build.ninja

Something seems to have truncated a path somewhere. You can see
in meson-log.txt that it uses --print-search-dirs and gets

 programs: =/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin
libraries: =/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/12.0.0

but in build.ninja the LINK_ARGS start
-Lns/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/12.0.0

with the "/Applicatio" part removed somehow.

I'm not sure why we're passing this explicitly to the linker anyway,
though.

-- PMM
Re: [PULL 0/8] s390x and misc fixes
Posted by Paolo Bonzini 2 years, 1 month ago
On 3/16/22 14:19, Peter Maydell wrote:
> I'm not sure why we're passing this explicitly to the linker anyway,
> though.

According to the commit message the idea is to add libc++ automatically 
when you have C++ sources, libgfortran when you have Fortran sources 
etc. independent of the language that you're using for the final link.

In other words it removes the need to specify a linker driver for 
mixed-language programs (and allows some other combinations such as C++ 
and Fortran together).

It would be better if it worked though...

Paolo
Re: [PULL 0/8] s390x and misc fixes
Posted by Paolo Bonzini 2 years, 1 month ago
On 3/15/22 19:58, Peter Maydell wrote:
> This results in every "Linking" step on my macos box producing the
> warning:
> 
> ld: warning: directory not found for option
> '-Lns/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/12.0.0'

This was reported already back in January with Meson 0.59:

https://www.mail-archive.com/qemu-devel@nongnu.org/msg861435.html

> 1. Linker has corrupted paths to clang directory (happens on all macOS 
> versions).
> 
> Monterey:
> 
> [732/737] Linking target qemu-system-mips-unsigned
> ld: warning: directory not found for option 
> '-Lns/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/13.0.0'
> [733/737] Linking target qemu-system-mips64-unsigned
> ld: warning: directory not found for option 
> '-Lns/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/13.0.0'
> [737/737] Generating qemu-system-mips64 with a custom command
> 
> Catalina:
> 
> ld: warning: directory not found for option 
> '-Lveloper/CommandLineTools/usr/lib/clang/11.0.0'
> [102/105] Linking target qemu-system-or1k-unsigned
> ld: warning: directory not found for option 
> '-Lveloper/CommandLineTools/usr/lib/clang/11.0.0'
> [104/105] Linking target qemu-system-ppc-unsigned
> ld: warning: directory not found for option 
> '-Lveloper/CommandLineTools/usr/lib/clang/11.0.0'
> [105/105] Generating qemu-system-ppc with a custom command

Roman, was this ever debugged?  Is the broken path to the clang 
directory included in build.ninja or is this a clang bug that somehow 
has been latent so far?

Paolo
Re: [PULL 0/8] s390x and misc fixes
Posted by Peter Maydell 2 years, 1 month ago
On Tue, 15 Mar 2022 at 18:58, Peter Maydell <peter.maydell@linaro.org> wrote:
>
> On Tue, 15 Mar 2022 at 11:20, Thomas Huth <thuth@redhat.com> wrote:
> >
> >  Hi Peter!
> >
> > The following changes since commit 352998df1c53b366413690d95b35f76d0721ebed:
> >
> >   Merge tag 'i2c-20220314' of https://github.com/philmd/qemu into staging (2022-03-14 14:39:33 +0000)
> >
> > are available in the Git repository at:
> >
> >   https://gitlab.com/thuth/qemu.git tags/pull-request-2022-03-15
> >
> > for you to fetch changes up to 36149534792dcf07a3c59867f967eaee23ab906c:
> >
> >   meson: Update to version 0.61.3 (2022-03-15 10:32:36 +0100)
> >
> > ----------------------------------------------------------------
> > * Fixes for s390x branch instruction emulation
> > * Fixes for the tests/avocado/boot_linux.py:BootLinuxS390X test
> > * Fix for "-cpu help" output
> > * Bump meson to 0.61.3 to fix stderr log of the iotests
> >
> > ----------------------------------------------------------------
>
> This results in every "Linking" step on my macos box producing the
> warning:
>
> ld: warning: directory not found for option
> '-Lns/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/12.0.0'
>
> Obvious suspect here is the new meson version.

Also, after rolling this merge attempt back, older meson barfs
on whatever the new one left behind:


[0/1] Regenerating build files.
Traceback (most recent call last):
  File "/Users/pm215/src/qemu-for-merges/meson/mesonbuild/mesonmain.py",
line 228, in run
    return options.run_func(options)
  File "/Users/pm215/src/qemu-for-merges/meson/mesonbuild/msetup.py",
line 281, in run
    app.generate()
  File "/Users/pm215/src/qemu-for-merges/meson/mesonbuild/msetup.py",
line 177, in generate
    env = environment.Environment(self.source_dir, self.build_dir, self.options)
  File "/Users/pm215/src/qemu-for-merges/meson/mesonbuild/environment.py",
line 462, in __init__
    self.coredata = coredata.load(self.get_build_dir())  # type:
coredata.CoreData
  File "/Users/pm215/src/qemu-for-merges/meson/mesonbuild/coredata.py",
line 1003, in load
    obj = pickle.load(f)
  File "/Users/pm215/src/qemu-for-merges/meson/mesonbuild/mesonlib/universal.py",
line 2076, in __setstate__
    self.__init__(**state)  # type: ignore
TypeError: __init__() got an unexpected keyword argument 'module'
FAILED: build.ninja
/usr/local/opt/python@3.9/bin/python3.9
/Users/pm215/src/qemu-for-merges/meson/meson.py --internal regenerate
/Users/pm215/src/qemu-for-merges
/Users/pm215/src/qemu-for-merges/build/all --backend ninja
ninja: error: rebuilding 'build.ninja': subcommand failed
/usr/local/bin/ninja  build.ninja && touch build.ninja.stamp
  GIT     ui/keycodemapdb meson tests/fp/berkeley-testfloat-3
tests/fp/berkeley-softfloat-3 dtc capstone slirp
[0/1] Regenerating build files.
Traceback (most recent call last):
  File "/Users/pm215/src/qemu-for-merges/meson/mesonbuild/mesonmain.py",
line 228, in run
    return options.run_func(options)
  File "/Users/pm215/src/qemu-for-merges/meson/mesonbuild/msetup.py",
line 281, in run
    app.generate()
  File "/Users/pm215/src/qemu-for-merges/meson/mesonbuild/msetup.py",
line 177, in generate
    env = environment.Environment(self.source_dir, self.build_dir, self.options)
  File "/Users/pm215/src/qemu-for-merges/meson/mesonbuild/environment.py",
line 462, in __init__
    self.coredata = coredata.load(self.get_build_dir())  # type:
coredata.CoreData
  File "/Users/pm215/src/qemu-for-merges/meson/mesonbuild/coredata.py",
line 1003, in load
    obj = pickle.load(f)
  File "/Users/pm215/src/qemu-for-merges/meson/mesonbuild/mesonlib/universal.py",
line 2076, in __setstate__
    self.__init__(**state)  # type: ignore
TypeError: __init__() got an unexpected keyword argument 'module'
FAILED: build.ninja


meson ought to be smart enough to spot that it's got data from an
incompatible version and just discard its cache rather than
choking on it.

thanks
-- PMM
Re: [PULL 0/8] s390x and misc fixes
Posted by Paolo Bonzini 2 years, 1 month ago
On 3/15/22 20:30, Peter Maydell wrote:
> coredata.CoreData
>    File "/Users/pm215/src/qemu-for-merges/meson/mesonbuild/coredata.py",
> line 1003, in load
>      obj = pickle.load(f)
>    File "/Users/pm215/src/qemu-for-merges/meson/mesonbuild/mesonlib/universal.py",
> line 2076, in __setstate__
>      self.__init__(**state)  # type: ignore
> TypeError: __init__() got an unexpected keyword argument 'module'
> FAILED: build.ninja
> 
> 
> meson ought to be smart enough to spot that it's got data from an
> incompatible version and just discard its cache rather than
> choking on it.

It tries, but it's playing whack-a-mole with Python:

     except (pickle.UnpicklingError, EOFError):
         raise MesonException(load_fail_msg)
     except (ModuleNotFoundError, AttributeError):
         raise MesonException(
             f"Coredata file {filename!r} references functions or classes that don't "
             "exist. This probably means that it was generated with an old "
             "version of meson.")

I'll submit a patch to add TypeError.

Paolo
Re: [PULL 0/8] s390x and misc fixes
Posted by Thomas Huth 2 years, 1 month ago
On 15/03/2022 20.30, Peter Maydell wrote:
> On Tue, 15 Mar 2022 at 18:58, Peter Maydell <peter.maydell@linaro.org> wrote:
>>
>> On Tue, 15 Mar 2022 at 11:20, Thomas Huth <thuth@redhat.com> wrote:
>>>
>>>   Hi Peter!
>>>
>>> The following changes since commit 352998df1c53b366413690d95b35f76d0721ebed:
>>>
>>>    Merge tag 'i2c-20220314' of https://github.com/philmd/qemu into staging (2022-03-14 14:39:33 +0000)
>>>
>>> are available in the Git repository at:
>>>
>>>    https://gitlab.com/thuth/qemu.git tags/pull-request-2022-03-15
>>>
>>> for you to fetch changes up to 36149534792dcf07a3c59867f967eaee23ab906c:
>>>
>>>    meson: Update to version 0.61.3 (2022-03-15 10:32:36 +0100)
>>>
>>> ----------------------------------------------------------------
>>> * Fixes for s390x branch instruction emulation
>>> * Fixes for the tests/avocado/boot_linux.py:BootLinuxS390X test
>>> * Fix for "-cpu help" output
>>> * Bump meson to 0.61.3 to fix stderr log of the iotests
>>>
>>> ----------------------------------------------------------------
>>
>> This results in every "Linking" step on my macos box producing the
>> warning:
>>
>> ld: warning: directory not found for option
>> '-Lns/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/12.0.0'
>>
>> Obvious suspect here is the new meson version.
> 
> Also, after rolling this merge attempt back, older meson barfs
> on whatever the new one left behind:
> 
> 
> [0/1] Regenerating build files.
> Traceback (most recent call last):
>    File "/Users/pm215/src/qemu-for-merges/meson/mesonbuild/mesonmain.py",
> line 228, in run
>      return options.run_func(options)
>    File "/Users/pm215/src/qemu-for-merges/meson/mesonbuild/msetup.py",
> line 281, in run
>      app.generate()
>    File "/Users/pm215/src/qemu-for-merges/meson/mesonbuild/msetup.py",
> line 177, in generate
>      env = environment.Environment(self.source_dir, self.build_dir, self.options)
>    File "/Users/pm215/src/qemu-for-merges/meson/mesonbuild/environment.py",
> line 462, in __init__
>      self.coredata = coredata.load(self.get_build_dir())  # type:
> coredata.CoreData
>    File "/Users/pm215/src/qemu-for-merges/meson/mesonbuild/coredata.py",
> line 1003, in load
>      obj = pickle.load(f)
>    File "/Users/pm215/src/qemu-for-merges/meson/mesonbuild/mesonlib/universal.py",
> line 2076, in __setstate__
>      self.__init__(**state)  # type: ignore
> TypeError: __init__() got an unexpected keyword argument 'module'
> FAILED: build.ninja
> /usr/local/opt/python@3.9/bin/python3.9
> /Users/pm215/src/qemu-for-merges/meson/meson.py --internal regenerate
> /Users/pm215/src/qemu-for-merges
> /Users/pm215/src/qemu-for-merges/build/all --backend ninja
> ninja: error: rebuilding 'build.ninja': subcommand failed
> /usr/local/bin/ninja  build.ninja && touch build.ninja.stamp
>    GIT     ui/keycodemapdb meson tests/fp/berkeley-testfloat-3
> tests/fp/berkeley-softfloat-3 dtc capstone slirp
> [0/1] Regenerating build files.
> Traceback (most recent call last):
>    File "/Users/pm215/src/qemu-for-merges/meson/mesonbuild/mesonmain.py",
> line 228, in run
>      return options.run_func(options)
>    File "/Users/pm215/src/qemu-for-merges/meson/mesonbuild/msetup.py",
> line 281, in run
>      app.generate()
>    File "/Users/pm215/src/qemu-for-merges/meson/mesonbuild/msetup.py",
> line 177, in generate
>      env = environment.Environment(self.source_dir, self.build_dir, self.options)
>    File "/Users/pm215/src/qemu-for-merges/meson/mesonbuild/environment.py",
> line 462, in __init__
>      self.coredata = coredata.load(self.get_build_dir())  # type:
> coredata.CoreData
>    File "/Users/pm215/src/qemu-for-merges/meson/mesonbuild/coredata.py",
> line 1003, in load
>      obj = pickle.load(f)
>    File "/Users/pm215/src/qemu-for-merges/meson/mesonbuild/mesonlib/universal.py",
> line 2076, in __setstate__
>      self.__init__(**state)  # type: ignore
> TypeError: __init__() got an unexpected keyword argument 'module'
> FAILED: build.ninja
> 
> 
> meson ought to be smart enough to spot that it's got data from an
> incompatible version and just discard its cache rather than
> choking on it.

Ok, I'll respin without the meson update.

Question is: Do we now want to revert the TAPification of the iotests for 
7.0? I guess so, otherwise most people won't see the output of failed tests 
when doing "make check-block" ...

  Thomas