[PULL 00/51] Build system, i386 changes for 2023-09-07

Paolo Bonzini posted 51 patches 8 months, 1 week ago
Failed in applying to current master (apply log)
Maintainers: Paolo Bonzini <pbonzini@redhat.com>, "Alex Bennée" <alex.bennee@linaro.org>, Thomas Huth <thuth@redhat.com>, Richard Henderson <richard.henderson@linaro.org>, "Marc-André Lureau" <marcandre.lureau@redhat.com>, Alexandre Iooss <erdnaxe@crans.org>, Mahmoud Mandour <ma.mandourr@gmail.com>, "Daniel P. Berrangé" <berrange@redhat.com>, Pavel Dovgalyuk <pavel.dovgaluk@ispras.ru>, "Philippe Mathieu-Daudé" <philmd@linaro.org>, "Michael S. Tsirkin" <mst@redhat.com>, Marcel Apfelbaum <marcel.apfelbaum@gmail.com>, Eduardo Habkost <eduardo@habkost.net>, Peter Xu <peterx@redhat.com>, Jason Wang <jasowang@redhat.com>, Stefan Weil <sw@weilnetz.de>, John Snow <jsnow@redhat.com>, Cleber Rosa <crosa@redhat.com>, Beraldo Leal <bleal@redhat.com>, Michael Roth <michael.roth@amd.com>, Konstantin Kostiuk <kkostiuk@redhat.com>, Markus Armbruster <armbru@redhat.com>, Kevin Wolf <kwolf@redhat.com>, Marcelo Tosatti <mtosatti@redhat.com>, Juan Quintela <quintela@redhat.com>, Leonardo Bras <leobras@redhat.com>, Laurent Vivier <lvivier@redhat.com>, Bastian Koppelmann <kbastian@mail.uni-paderborn.de>
Makefile                                  |  29 ++--
accel/tcg/meson.build                     |   4 +-
chardev/meson.build                       |   2 +-
configure                                 | 176 +++++++------------
contrib/plugins/Makefile                  |  22 ++-
contrib/plugins/cache.c                   |  19 ++-
contrib/plugins/drcov.c                   |   2 +-
contrib/plugins/howvec.c                  |   6 +-
contrib/plugins/lockstep.c                |  11 +-
docs/devel/build-system.rst               |   8 +-
docs/devel/kconfig.rst                    |   2 +-
docs/system/replay.rst                    |   2 +-
gdbstub/meson.build                       |   4 +-
hw/i386/fw_cfg.c                          |   1 +
hw/i386/intel_iommu.c                     |   2 +-
hw/i386/kvm/i8254.c                       |   1 +
hw/i386/kvm/ioapic.c                      |   1 +
hw/i386/pc_piix.c                         |   1 +
hw/i386/pc_q35.c                          |   2 +
hw/i386/x86.c                             |  11 +-
include/qemu/qemu-options.h               |  41 -----
include/sysemu/kvm.h                      |  10 --
include/sysemu/os-posix.h                 |  10 +-
include/sysemu/os-win32.h                 |   1 -
meson.build                               | 111 +++++++-----
meson_options.txt                         |   4 +
net/meson.build                           |  18 +-
os-posix.c                                | 157 +++++------------
pc-bios/Makefile                          |  19 ---
pc-bios/meson.build                       |  25 ++-
plugins/meson.build                       |  12 +-
python/Makefile                           |   8 +-
python/scripts/mkvenv.py                  | 272 +++++-------------------------
python/setup.cfg                          |  16 +-
python/tests/minreqs.txt                  |   2 +-
qga/meson.build                           |   4 +-
scripts/meson-buildoptions.sh             |   6 +
scripts/qapi/mypy.ini                     |   2 +-
{util => softmmu}/async-teardown.c        |   0
softmmu/meson.build                       |   1 +
softmmu/vl.c                              |  87 +++++++++-
storage-daemon/meson.build                |   2 +-
subprojects/libblkio.wrap                 |   6 +
target/i386/cpu-sysemu.c                  |   4 +-
target/i386/cpu.c                         |   2 +-
target/i386/cpu.h                         |   2 +
target/i386/helper.c                      |   2 +-
target/i386/kvm/kvm-stub.c                |  51 ------
target/i386/kvm/kvm.c                     |   4 +-
target/i386/kvm/kvm_i386.h                |  36 ++--
target/i386/kvm/meson.build               |   2 -
target/i386/tcg/decode-new.c.inc          |  20 ++-
target/i386/tcg/decode-new.h              |   2 +-
target/i386/tcg/emit.c.inc                |  30 +++-
target/i386/tcg/sysemu/fpu_helper.c       |   6 +
target/i386/tcg/translate.c               |   8 +-
tcg/meson.build                           |   2 +-
tests/Makefile.include                    |   2 +-
tests/meson.build                         |   8 +-
tests/migration/meson.build               |   2 +-
tests/qtest/meson.build                   |  14 +-
tests/tcg/tricore/Makefile.softmmu-target |   2 +-
tests/unit/meson.build                    |   6 +-
util/meson.build                          |   1 -
64 files changed, 551 insertions(+), 775 deletions(-)
delete mode 100644 include/qemu/qemu-options.h
delete mode 100644 pc-bios/Makefile
rename {util => softmmu}/async-teardown.c (100%)
create mode 100644 subprojects/libblkio.wrap
delete mode 100644 target/i386/kvm/kvm-stub.c
[PULL 00/51] Build system, i386 changes for 2023-09-07
Posted by Paolo Bonzini 8 months, 1 week ago
The following changes since commit 17780edd81d27fcfdb7a802efc870a99788bd2fc:

  Merge tag 'quick-fix-pull-request' of https://gitlab.com/bsdimp/qemu into staging (2023-08-31 10:06:29 -0400)

are available in the Git repository at:

  https://gitlab.com/bonzini/qemu.git tags/for-upstream

for you to fetch changes up to 34e8182053c065e5e99017f798fb79259e26f583:

  docs/system/replay: do not show removed command line option (2023-09-07 13:32:37 +0200)

----------------------------------------------------------------
* only build util/async-teardown.c when system build is requested
* target/i386: fix BQL handling of the legacy FERR interrupts
* target/i386: fix memory operand size for CVTPS2PD
* target/i386: Add support for AMX-COMPLEX in CPUID enumeration
* compile plugins on Darwin
* configure and meson cleanups
* drop mkvenv support for Python 3.7 and Debian10
* add wrap file for libblkio
* tweak KVM stubs

----------------------------------------------------------------
Michael Tokarev (8):
      include/sysemu/os-posix.h: move *daemonize* declarations together
      os-posix.c: create and export os_set_runas()
      os-posix.c: create and export os_set_chroot()
      os-posix.c, softmmu/vl.c: move os_parse_cmd_args() into qemu_init()
      os-posix.c: move code around
      os-posix.c: remove unneeded #includes
      softmmu/vl.c: inline include/qemu/qemu-options.h into vl.c
      util/async-teardown.c: move to softmmu/, only build it when system build is requested

Paolo Bonzini (25):
      linux-user, bsd-user: disable on unsupported host architectures
      target/i386: raise FERR interrupt with iothread locked
      target/i386: generalize operand size "ph" for use in CVTPS2PD
      target/i386: fix memory operand size for CVTPS2PD
      contrib/plugins: remove -soname argument
      contrib/plugins: add Darwin support
      meson: do not unnecessarily use cmake for dependencies
      meson: update unsupported host/CPU messages
      configure: remove HOST_CC
      configure: create native file with contents of $host_cc
      meson: compile bundled device trees
      configure: remove boolean variables for targets
      configure: move --enable-debug-tcg to meson
      contrib/plugins: use an independent makefile
      configure: unify recursion into sub-Makefiles
      configure, meson: move --enable-plugins to meson
      configure, meson: remove CONFIG_SOLARIS from config-host.mak
      configure, meson: remove target OS symbols from config-host.mak
      meson: list leftover CONFIG_* symbols
      configure: remove dead code
      Python: Drop support for Python 3.7
      mkvenv: assume presence of importlib.metadata
      Revert "mkvenv: work around broken pip installations on Debian 10"
      subprojects: add wrap file for libblkio
      docs/system/replay: do not show removed command line option

Philippe Mathieu-Daudé (17):
      contrib/plugins/cache: Fix string format
      contrib/plugins/drcov: Fix string format
      contrib/plugins/howvec: Fix string format
      contrib/plugins/lockstep: Fix string format
      hw/i386/pc: Include missing 'sysemu/tcg.h' header
      hw/i386/pc: Include missing 'cpu.h' header
      hw/i386/fw_cfg: Include missing 'cpu.h' header
      target/i386/helper: Restrict KVM declarations to system emulation
      target/i386/cpu-sysemu: Inline kvm_apic_in_kernel()
      target/i386: Remove unused KVM stubs
      target/i386: Allow elision of kvm_enable_x2apic()
      target/i386: Allow elision of kvm_hv_vpindex_settable()
      target/i386: Restrict declarations specific to CONFIG_KVM
      sysemu/kvm: Restrict kvm_arch_get_supported_cpuid/msr() to x86 targets
      sysemu/kvm: Restrict kvm_get_apic_state() to x86 targets
      sysemu/kvm: Restrict kvm_has_pit_state2() to x86 targets
      sysemu/kvm: Restrict kvm_pc_setup_irq_routing() to x86 targets

Tao Su (1):
      target/i386: Add support for AMX-COMPLEX in CPUID enumeration

 Makefile                                  |  29 ++--
 accel/tcg/meson.build                     |   4 +-
 chardev/meson.build                       |   2 +-
 configure                                 | 176 +++++++------------
 contrib/plugins/Makefile                  |  22 ++-
 contrib/plugins/cache.c                   |  19 ++-
 contrib/plugins/drcov.c                   |   2 +-
 contrib/plugins/howvec.c                  |   6 +-
 contrib/plugins/lockstep.c                |  11 +-
 docs/devel/build-system.rst               |   8 +-
 docs/devel/kconfig.rst                    |   2 +-
 docs/system/replay.rst                    |   2 +-
 gdbstub/meson.build                       |   4 +-
 hw/i386/fw_cfg.c                          |   1 +
 hw/i386/intel_iommu.c                     |   2 +-
 hw/i386/kvm/i8254.c                       |   1 +
 hw/i386/kvm/ioapic.c                      |   1 +
 hw/i386/pc_piix.c                         |   1 +
 hw/i386/pc_q35.c                          |   2 +
 hw/i386/x86.c                             |  11 +-
 include/qemu/qemu-options.h               |  41 -----
 include/sysemu/kvm.h                      |  10 --
 include/sysemu/os-posix.h                 |  10 +-
 include/sysemu/os-win32.h                 |   1 -
 meson.build                               | 111 +++++++-----
 meson_options.txt                         |   4 +
 net/meson.build                           |  18 +-
 os-posix.c                                | 157 +++++------------
 pc-bios/Makefile                          |  19 ---
 pc-bios/meson.build                       |  25 ++-
 plugins/meson.build                       |  12 +-
 python/Makefile                           |   8 +-
 python/scripts/mkvenv.py                  | 272 +++++-------------------------
 python/setup.cfg                          |  16 +-
 python/tests/minreqs.txt                  |   2 +-
 qga/meson.build                           |   4 +-
 scripts/meson-buildoptions.sh             |   6 +
 scripts/qapi/mypy.ini                     |   2 +-
 {util => softmmu}/async-teardown.c        |   0
 softmmu/meson.build                       |   1 +
 softmmu/vl.c                              |  87 +++++++++-
 storage-daemon/meson.build                |   2 +-
 subprojects/libblkio.wrap                 |   6 +
 target/i386/cpu-sysemu.c                  |   4 +-
 target/i386/cpu.c                         |   2 +-
 target/i386/cpu.h                         |   2 +
 target/i386/helper.c                      |   2 +-
 target/i386/kvm/kvm-stub.c                |  51 ------
 target/i386/kvm/kvm.c                     |   4 +-
 target/i386/kvm/kvm_i386.h                |  36 ++--
 target/i386/kvm/meson.build               |   2 -
 target/i386/tcg/decode-new.c.inc          |  20 ++-
 target/i386/tcg/decode-new.h              |   2 +-
 target/i386/tcg/emit.c.inc                |  30 +++-
 target/i386/tcg/sysemu/fpu_helper.c       |   6 +
 target/i386/tcg/translate.c               |   8 +-
 tcg/meson.build                           |   2 +-
 tests/Makefile.include                    |   2 +-
 tests/meson.build                         |   8 +-
 tests/migration/meson.build               |   2 +-
 tests/qtest/meson.build                   |  14 +-
 tests/tcg/tricore/Makefile.softmmu-target |   2 +-
 tests/unit/meson.build                    |   6 +-
 util/meson.build                          |   1 -
 64 files changed, 551 insertions(+), 775 deletions(-)
 delete mode 100644 include/qemu/qemu-options.h
 delete mode 100644 pc-bios/Makefile
 rename {util => softmmu}/async-teardown.c (100%)
 create mode 100644 subprojects/libblkio.wrap
 delete mode 100644 target/i386/kvm/kvm-stub.c
-- 
2.41.0


Re: [PULL 00/51] Build system, i386 changes for 2023-09-07
Posted by Stefan Hajnoczi 8 months, 1 week ago
Applied, thanks.

Please update the changelog at https://wiki.qemu.org/ChangeLog/8.2 for any user-visible changes.
Re: [PULL 00/51] Build system, i386 changes for 2023-09-07
Posted by Kevin Wolf 8 months, 1 week ago
Am 07.09.2023 um 17:44 hat Stefan Hajnoczi geschrieben:
> Applied, thanks.
> 
> Please update the changelog at https://wiki.qemu.org/ChangeLog/8.2 for any user-visible changes.

Something in this has broken the build for me, it seems to be the
linux-user binary that doesn't link any more:

  /usr/bin/ld: libqemu-x86_64-linux-user.fa.p/target_i386_cpu.c.o: in function `cpu_x86_cpuid':
  /home/kwolf/source/qemu/build-clang/../target/i386/cpu.c:6180: undefined reference to `kvm_arch_get_supported_cpuid'
  /usr/bin/ld: libqemu-x86_64-linux-user.fa.p/target_i386_cpu.c.o: in function `x86_cpu_filter_features':
  /home/kwolf/source/qemu/build-clang/../target/i386/cpu.c:7158: undefined reference to `kvm_arch_get_supported_cpuid'
  /usr/bin/ld: /home/kwolf/source/qemu/build-clang/../target/i386/cpu.c:7159: undefined reference to `kvm_arch_get_supported_cpuid'
  /usr/bin/ld: /home/kwolf/source/qemu/build-clang/../target/i386/cpu.c:7160: undefined reference to `kvm_arch_get_supported_cpuid'
  /usr/bin/ld: /home/kwolf/source/qemu/build-clang/../target/i386/cpu.c:7161: undefined reference to `kvm_arch_get_supported_cpuid'
  /usr/bin/ld: libqemu-x86_64-linux-user.fa.p/target_i386_cpu.c.o:/home/kwolf/source/qemu/build-clang/../target/i386/cpu.c:7162: more undefined references to `kvm_arch_get_supported_cpuid' follow
  clang-15: error: linker command failed with exit code 1 (use -v to see invocation)

In case it makes a difference, I'm using clang on F37.

Kevin
Re: [PULL 00/51] Build system, i386 changes for 2023-09-07
Posted by Philippe Mathieu-Daudé 8 months, 1 week ago
Hi Kevin,

On 8/9/23 17:01, Kevin Wolf wrote:
> Am 07.09.2023 um 17:44 hat Stefan Hajnoczi geschrieben:
>> Applied, thanks.
>>
>> Please update the changelog at https://wiki.qemu.org/ChangeLog/8.2 for any user-visible changes.
> 
> Something in this has broken the build for me, it seems to be the
> linux-user binary that doesn't link any more:
> 
>    /usr/bin/ld: libqemu-x86_64-linux-user.fa.p/target_i386_cpu.c.o: in function `cpu_x86_cpuid':
>    /home/kwolf/source/qemu/build-clang/../target/i386/cpu.c:6180: undefined reference to `kvm_arch_get_supported_cpuid'

My patches. I can not reproduce so far,
What are your ./configure command options?

>    /usr/bin/ld: libqemu-x86_64-linux-user.fa.p/target_i386_cpu.c.o: in function `x86_cpu_filter_features':
>    /home/kwolf/source/qemu/build-clang/../target/i386/cpu.c:7158: undefined reference to `kvm_arch_get_supported_cpuid'
>    /usr/bin/ld: /home/kwolf/source/qemu/build-clang/../target/i386/cpu.c:7159: undefined reference to `kvm_arch_get_supported_cpuid'
>    /usr/bin/ld: /home/kwolf/source/qemu/build-clang/../target/i386/cpu.c:7160: undefined reference to `kvm_arch_get_supported_cpuid'
>    /usr/bin/ld: /home/kwolf/source/qemu/build-clang/../target/i386/cpu.c:7161: undefined reference to `kvm_arch_get_supported_cpuid'
>    /usr/bin/ld: libqemu-x86_64-linux-user.fa.p/target_i386_cpu.c.o:/home/kwolf/source/qemu/build-clang/../target/i386/cpu.c:7162: more undefined references to `kvm_arch_get_supported_cpuid' follow
>    clang-15: error: linker command failed with exit code 1 (use -v to see invocation)
> 
> In case it makes a difference, I'm using clang on F37.
> 
> Kevin
Re: [PULL 00/51] Build system, i386 changes for 2023-09-07
Posted by Kevin Wolf 8 months, 1 week ago
Am 08.09.2023 um 18:11 hat Philippe Mathieu-Daudé geschrieben:
> Hi Kevin,
> 
> On 8/9/23 17:01, Kevin Wolf wrote:
> > Am 07.09.2023 um 17:44 hat Stefan Hajnoczi geschrieben:
> > > Applied, thanks.
> > > 
> > > Please update the changelog at https://wiki.qemu.org/ChangeLog/8.2 for any user-visible changes.
> > 
> > Something in this has broken the build for me, it seems to be the
> > linux-user binary that doesn't link any more:
> > 
> >    /usr/bin/ld: libqemu-x86_64-linux-user.fa.p/target_i386_cpu.c.o: in function `cpu_x86_cpuid':
> >    /home/kwolf/source/qemu/build-clang/../target/i386/cpu.c:6180: undefined reference to `kvm_arch_get_supported_cpuid'
> 
> My patches. I can not reproduce so far,

Hm, ok. I can try to bisect it further then.

> What are your ./configure command options?

'/home/kwolf/source/qemu/configure' '--cc=clang' '--target-list=x86_64-linux-user,x86_64-softmmu,arm-softmmu,aarch64-softmmu' '--enable-debug' '--cxx=clang++' '--disable-blkio' 

Kevin

> >    /usr/bin/ld: libqemu-x86_64-linux-user.fa.p/target_i386_cpu.c.o: in function `x86_cpu_filter_features':
> >    /home/kwolf/source/qemu/build-clang/../target/i386/cpu.c:7158: undefined reference to `kvm_arch_get_supported_cpuid'
> >    /usr/bin/ld: /home/kwolf/source/qemu/build-clang/../target/i386/cpu.c:7159: undefined reference to `kvm_arch_get_supported_cpuid'
> >    /usr/bin/ld: /home/kwolf/source/qemu/build-clang/../target/i386/cpu.c:7160: undefined reference to `kvm_arch_get_supported_cpuid'
> >    /usr/bin/ld: /home/kwolf/source/qemu/build-clang/../target/i386/cpu.c:7161: undefined reference to `kvm_arch_get_supported_cpuid'
> >    /usr/bin/ld: libqemu-x86_64-linux-user.fa.p/target_i386_cpu.c.o:/home/kwolf/source/qemu/build-clang/../target/i386/cpu.c:7162: more undefined references to `kvm_arch_get_supported_cpuid' follow
> >    clang-15: error: linker command failed with exit code 1 (use -v to see invocation)
> > 
> > In case it makes a difference, I'm using clang on F37.
> > 
> > Kevin
> 
Re: [PULL 00/51] Build system, i386 changes for 2023-09-07
Posted by Michael Tokarev 8 months, 1 week ago
08.09.2023 20:16, Kevin Wolf:
> Am 08.09.2023 um 18:11 hat Philippe Mathieu-Daudé geschrieben:
>> Hi Kevin,
>>
>> On 8/9/23 17:01, Kevin Wolf wrote:
>>> Am 07.09.2023 um 17:44 hat Stefan Hajnoczi geschrieben:
>>>> Applied, thanks.
>>>>
>>>> Please update the changelog at https://wiki.qemu.org/ChangeLog/8.2 for any user-visible changes.
>>>
>>> Something in this has broken the build for me, it seems to be the
>>> linux-user binary that doesn't link any more:
>>>
>>>     /usr/bin/ld: libqemu-x86_64-linux-user.fa.p/target_i386_cpu.c.o: in function `cpu_x86_cpuid':
>>>     /home/kwolf/source/qemu/build-clang/../target/i386/cpu.c:6180: undefined reference to `kvm_arch_get_supported_cpuid'
>>
>> My patches. I can not reproduce so far,
> 
> Hm, ok. I can try to bisect it further then.

     case 7:
         /* Structured Extended Feature Flags Enumeration Leaf */
         if (count == 0) {
...
             if ((*ebx & CPUID_7_0_EBX_SGX) &&
                 (!kvm_enabled() ||
                  !(kvm_arch_get_supported_cpuid(cs->kvm_state, 0x7, 0, R_EBX) &
                     CPUID_7_0_EBX_SGX))) {
                 *ebx &= ~CPUID_7_0_EBX_SGX;
             }

             if ((*ecx & CPUID_7_0_ECX_SGX_LC) &&
                 (!(*ebx & CPUID_7_0_EBX_SGX) || !kvm_enabled() ||
                  !(kvm_arch_get_supported_cpuid(cs->kvm_state, 0x7, 0, R_ECX) &
                     CPUID_7_0_ECX_SGX_LC))) {
                 *ecx &= ~CPUID_7_0_ECX_SGX_LC;
             }

It is most likely patch "target/i386: Remove unused KVM stubs" which removed
kvm_arch_get_supported_cpuid().  Apparently clang isn't smart enough to eliminate
call to this function?

/mjt

Re: [PULL 00/51] Build system, i386 changes for 2023-09-07
Posted by Kevin Wolf 8 months, 1 week ago
Am 08.09.2023 um 19:16 hat Kevin Wolf geschrieben:
> Am 08.09.2023 um 18:11 hat Philippe Mathieu-Daudé geschrieben:
> > Hi Kevin,
> > 
> > On 8/9/23 17:01, Kevin Wolf wrote:
> > > Am 07.09.2023 um 17:44 hat Stefan Hajnoczi geschrieben:
> > > > Applied, thanks.
> > > > 
> > > > Please update the changelog at https://wiki.qemu.org/ChangeLog/8.2 for any user-visible changes.
> > > 
> > > Something in this has broken the build for me, it seems to be the
> > > linux-user binary that doesn't link any more:
> > > 
> > >    /usr/bin/ld: libqemu-x86_64-linux-user.fa.p/target_i386_cpu.c.o: in function `cpu_x86_cpuid':
> > >    /home/kwolf/source/qemu/build-clang/../target/i386/cpu.c:6180: undefined reference to `kvm_arch_get_supported_cpuid'
> > 
> > My patches. I can not reproduce so far,
> 
> Hm, ok. I can try to bisect it further then.

Right, the commit message is obviously related, so you probably knew
this already... :-)

commit 3adce820cf9c454b49cabf52b829001ecb98500e
Author: Philippe Mathieu-Daudé <philmd@linaro.org>
Date:   Mon Sep 4 14:43:17 2023 +0200

    target/i386: Remove unused KVM stubs
    
    All these functions:
    
     - kvm_arch_get_supported_cpuid()
     - kvm_has_smm(()
     - kvm_hyperv_expand_features()
     - kvm_set_max_apic_id()
    
    are called after checking for kvm_enabled(), which is
    false when KVM is not built. Since the compiler elides
    these functions, their stubs are not used and can be
    removed.
    
    Inspired-by: Daniel Henrique Barboza <dbarboza@ventanamicro.com>
    Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
    Message-ID: <20230904124325.79040-7-philmd@linaro.org>
    Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>

 target/i386/kvm/kvm-stub.c | 26 --------------------------
 1 file changed, 26 deletions(-)

Maybe the calls aren't eliminated because --enable-debug implies -O0?

Kevin

> > What are your ./configure command options?
> 
> '/home/kwolf/source/qemu/configure' '--cc=clang' '--target-list=x86_64-linux-user,x86_64-softmmu,arm-softmmu,aarch64-softmmu' '--enable-debug' '--cxx=clang++' '--disable-blkio' 
> 
> Kevin
> 
> > >    /usr/bin/ld: libqemu-x86_64-linux-user.fa.p/target_i386_cpu.c.o: in function `x86_cpu_filter_features':
> > >    /home/kwolf/source/qemu/build-clang/../target/i386/cpu.c:7158: undefined reference to `kvm_arch_get_supported_cpuid'
> > >    /usr/bin/ld: /home/kwolf/source/qemu/build-clang/../target/i386/cpu.c:7159: undefined reference to `kvm_arch_get_supported_cpuid'
> > >    /usr/bin/ld: /home/kwolf/source/qemu/build-clang/../target/i386/cpu.c:7160: undefined reference to `kvm_arch_get_supported_cpuid'
> > >    /usr/bin/ld: /home/kwolf/source/qemu/build-clang/../target/i386/cpu.c:7161: undefined reference to `kvm_arch_get_supported_cpuid'
> > >    /usr/bin/ld: libqemu-x86_64-linux-user.fa.p/target_i386_cpu.c.o:/home/kwolf/source/qemu/build-clang/../target/i386/cpu.c:7162: more undefined references to `kvm_arch_get_supported_cpuid' follow
> > >    clang-15: error: linker command failed with exit code 1 (use -v to see invocation)
> > > 
> > > In case it makes a difference, I'm using clang on F37.
> > > 
> > > Kevin
> > 
Re: [PULL 00/51] Build system, i386 changes for 2023-09-07
Posted by Paolo Bonzini 8 months, 1 week ago
On Fri, Sep 8, 2023 at 7:28 PM Kevin Wolf <kwolf@redhat.com> wrote:
> Maybe the calls aren't eliminated because --enable-debug implies -O0?

My experience is that it will still fold simple dead code like "0 &&
foo()" or even "if (0) { ... }", but maybe it's a GCC vs. clang
difference. Philippe, I take it that you are looking at it?

Paolo

> Kevin
>
> > > What are your ./configure command options?
> >
> > '/home/kwolf/source/qemu/configure' '--cc=clang' '--target-list=x86_64-linux-user,x86_64-softmmu,arm-softmmu,aarch64-softmmu' '--enable-debug' '--cxx=clang++' '--disable-blkio'
> >
> > Kevin
> >
> > > >    /usr/bin/ld: libqemu-x86_64-linux-user.fa.p/target_i386_cpu.c.o: in function `x86_cpu_filter_features':
> > > >    /home/kwolf/source/qemu/build-clang/../target/i386/cpu.c:7158: undefined reference to `kvm_arch_get_supported_cpuid'
> > > >    /usr/bin/ld: /home/kwolf/source/qemu/build-clang/../target/i386/cpu.c:7159: undefined reference to `kvm_arch_get_supported_cpuid'
> > > >    /usr/bin/ld: /home/kwolf/source/qemu/build-clang/../target/i386/cpu.c:7160: undefined reference to `kvm_arch_get_supported_cpuid'
> > > >    /usr/bin/ld: /home/kwolf/source/qemu/build-clang/../target/i386/cpu.c:7161: undefined reference to `kvm_arch_get_supported_cpuid'
> > > >    /usr/bin/ld: libqemu-x86_64-linux-user.fa.p/target_i386_cpu.c.o:/home/kwolf/source/qemu/build-clang/../target/i386/cpu.c:7162: more undefined references to `kvm_arch_get_supported_cpuid' follow
> > > >    clang-15: error: linker command failed with exit code 1 (use -v to see invocation)
> > > >
> > > > In case it makes a difference, I'm using clang on F37.
> > > >
> > > > Kevin
> > >
>
Re: [PULL 00/51] Build system, i386 changes for 2023-09-07
Posted by Michael Tokarev 8 months ago
08.09.2023 22:21, Paolo Bonzini wrote:
> On Fri, Sep 8, 2023 at 7:28 PM Kevin Wolf <kwolf@redhat.com> wrote:
>> Maybe the calls aren't eliminated because --enable-debug implies -O0?
> 
> My experience is that it will still fold simple dead code like "0 &&
> foo()" or even "if (0) { ... }", but maybe it's a GCC vs. clang
> difference. [..]

In my view it's too fragile to rely on this.  And yes, it's the clang
builds which triggers this issue.

We've a bugreport, https://gitlab.com/qemu-project/qemu/-/issues/1848 ,
where obviously-unreachable code hasn't been detected by gcc when
optimizing for certain CPU type.  This smells like a GCC bug but still
it's something which we shouldn't rely on.

What can be done though is, instead of making kvm_arch_get_supported_cpuid
et all a stubs, to replace them in .h file with an empty macro when some
condition(s) are met.  In other words, fold kvm_enabled() "inside" of
kvm_arch_get_supported_cpuid() and use some assert_not_reached() there.

/mjt

Re: [PULL 00/51] Build system, i386 changes for 2023-09-07
Posted by Michael Tokarev 8 months ago
11.09.2023 13:41, Michael Tokarev:

> What can be done though is, instead of making kvm_arch_get_supported_cpuid
> et all a stubs, to replace them in .h file with an empty macro when some
> condition(s) are met.  In other words, fold kvm_enabled() "inside" of
> kvm_arch_get_supported_cpuid() and use some assert_not_reached() there.

Which is exactly what Philippe did in
"Re-introduce few KVM stubs for Clang debug builds".

/mjt


Re: [PULL 00/51] Build system, i386 changes for 2023-09-07
Posted by Philippe Mathieu-Daudé 8 months ago
On 8/9/23 21:21, Paolo Bonzini wrote:
> On Fri, Sep 8, 2023 at 7:28 PM Kevin Wolf <kwolf@redhat.com> wrote:
>> Maybe the calls aren't eliminated because --enable-debug implies -O0?
> 
> My experience is that it will still fold simple dead code like "0 &&
> foo()" or even "if (0) { ... }", but maybe it's a GCC vs. clang
> difference. Philippe, I take it that you are looking at it?

Yes, I'll send a patch soon. Sorry I didn't catch that case.


Re: [PULL 00/51] Build system, i386 changes for 2023-09-07
Posted by Philippe Mathieu-Daudé 8 months ago
On 11/9/23 12:11, Philippe Mathieu-Daudé wrote:
> On 8/9/23 21:21, Paolo Bonzini wrote:
>> On Fri, Sep 8, 2023 at 7:28 PM Kevin Wolf <kwolf@redhat.com> wrote:
>>> Maybe the calls aren't eliminated because --enable-debug implies -O0?
>>
>> My experience is that it will still fold simple dead code like "0 &&
>> foo()" or even "if (0) { ... }", but maybe it's a GCC vs. clang
>> difference. Philippe, I take it that you are looking at it?

FTR preprocessed code is indeed:

# 5682 "../../target/i386/cpu.c"
uint64_t x86_cpu_get_supported_feature_word(FeatureWord w,
                                             _Bool migratable_only)
{
     FeatureWordInfo *wi = &feature_word_info[w];
     uint64_t r = 0;

     if ((0)) {
         switch (wi->type) {
         case CPUID_FEATURE_WORD:
             r = kvm_arch_get_supported_cpuid(kvm_state, wi->cpuid.eax,
                                                         wi->cpuid.ecx,
                                                         wi->cpuid.reg);
             break;
         case MSR_FEATURE_WORD:
             r = kvm_arch_get_supported_msr_feature(kvm_state,
                         wi->msr.index);
             break;
         }
     } else if (0) {
         if (wi->type != CPUID_FEATURE_WORD) {
             return 0;
         }
         r = 0;


     } else if ((tcg_allowed)) {
         r = wi->tcg_features;
     } else {
         return ~0;
     }
# 5725 "../../target/i386/cpu.c"
     if (migratable_only) {
         r &= x86_cpu_get_migratable_flags(w);
     }
     return r;
}

static void x86_cpu_get_supported_cpuid(uint32_t func, uint32_t index,
                                         uint32_t *eax, uint32_t *ebx,
                                         uint32_t *ecx, uint32_t *edx)
{
     if ((0)) {
         *eax = kvm_arch_get_supported_cpuid(kvm_state, func, index, R_EAX);
         *ebx = kvm_arch_get_supported_cpuid(kvm_state, func, index, R_EBX);
         *ecx = kvm_arch_get_supported_cpuid(kvm_state, func, index, R_ECX);
         *edx = kvm_arch_get_supported_cpuid(kvm_state, func, index, R_EDX);
     } else if (0) {
         *eax = 0;
         *ebx = 0;
         *ecx = 0;
         *edx = 0;
     } else {
         *eax = 0;
         *ebx = 0;
         *ecx = 0;
         *edx = 0;
     }
}

Which made me wonder about this sysemu code use from useremu,
in particular because the link failure comes from
libqemu-x86_64-linux-user.fa.p. This pair of #ifdef'ry to
restrict system-specific code seems sufficient to link:

-- >8 --
diff --git a/target/i386/cpu.c b/target/i386/cpu.c
index 24ee67b42d..83914d5d1b 100644
--- a/target/i386/cpu.c
+++ b/target/i386/cpu.c
@@ -6163,6 +6163,7 @@ void cpu_x86_cpuid(CPUX86State *env, uint32_t 
index, uint32_t count,
              }
              *edx = env->features[FEAT_7_0_EDX]; /* Feature flags */

+#ifndef CONFIG_USER_ONLY
              /*
               * SGX cannot be emulated in software.  If hardware does not
               * support enabling SGX and/or SGX flexible launch control,
@@ -6181,6 +6182,7 @@ void cpu_x86_cpuid(CPUX86State *env, uint32_t 
index, uint32_t count,
                      CPUID_7_0_ECX_SGX_LC))) {
                  *ecx &= ~CPUID_7_0_ECX_SGX_LC;
              }
+#endif
          } else if (count == 1) {
              *eax = env->features[FEAT_7_1_EAX];
              *edx = env->features[FEAT_7_1_EDX];
@@ -7152,6 +7154,7 @@ static void x86_cpu_filter_features(X86CPU *cpu, 
bool verbose)
          mark_unavailable_features(cpu, w, unavailable_features, prefix);
      }

+#ifndef CONFIG_USER_ONLY
      if ((env->features[FEAT_7_0_EBX] & CPUID_7_0_EBX_INTEL_PT) &&
          kvm_enabled()) {
          KVMState *s = CPU(cpu)->kvm_state;
@@ -7179,6 +7182,7 @@ static void x86_cpu_filter_features(X86CPU *cpu, 
bool verbose)
              mark_unavailable_features(cpu, FEAT_7_0_EBX, 
CPUID_7_0_EBX_INTEL_PT, prefix);
          }
      }
+#endif
  }
---

So I'll send a v3.


Re: [PULL 00/51] Build system, i386 changes for 2023-09-07
Posted by Daniel P. Berrangé 8 months, 1 week ago
On Fri, Sep 08, 2023 at 07:16:16PM +0200, Kevin Wolf wrote:
> Am 08.09.2023 um 18:11 hat Philippe Mathieu-Daudé geschrieben:
> > Hi Kevin,
> > 
> > On 8/9/23 17:01, Kevin Wolf wrote:
> > > Am 07.09.2023 um 17:44 hat Stefan Hajnoczi geschrieben:
> > > > Applied, thanks.
> > > > 
> > > > Please update the changelog at https://wiki.qemu.org/ChangeLog/8.2 for any user-visible changes.
> > > 
> > > Something in this has broken the build for me, it seems to be the
> > > linux-user binary that doesn't link any more:
> > > 
> > >    /usr/bin/ld: libqemu-x86_64-linux-user.fa.p/target_i386_cpu.c.o: in function `cpu_x86_cpuid':
> > >    /home/kwolf/source/qemu/build-clang/../target/i386/cpu.c:6180: undefined reference to `kvm_arch_get_supported_cpuid'
> > 
> > My patches. I can not reproduce so far,

Given the missing function name, I'm assuming the problem is this commit:

[quote]
Subject: [PULL 42/51] target/i386: Remove unused KVM stubs

From: Philippe Mathieu-Daudé <philmd@linaro.org>

All these functions:

 - kvm_arch_get_supported_cpuid()
 - kvm_has_smm(()
 - kvm_hyperv_expand_features()
 - kvm_set_max_apic_id()

are called after checking for kvm_enabled(), which is
false when KVM is not built. Since the compiler elides
these functions, their stubs are not used and can be
removed.
[/quote]

> Hm, ok. I can try to bisect it further then.
> 
> > What are your ./configure command options?
> 
> '/home/kwolf/source/qemu/configure' '--cc=clang' '--target-list=x86_64-linux-user,x86_64-softmmu,arm-softmmu,aarch64-softmmu' '--enable-debug' '--cxx=clang++' '--disable-blkio' 

--enable-debug results in "-O0" being used for optimization.

I bet this stops the compiler from eliding the functions
as claimed by the commit above. IOW, we do in fact still
need these stubs.


With regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|


Re: [PULL 00/51] Build system, i386 changes for 2023-09-07
Posted by Stefan Hajnoczi 8 months, 1 week ago
I wonder how it passed CI?
https://gitlab.com/qemu-project/qemu/-/pipelines/996175923/

Stefan

On Fri, 8 Sept 2023 at 11:02, Kevin Wolf <kwolf@redhat.com> wrote:
>
> Am 07.09.2023 um 17:44 hat Stefan Hajnoczi geschrieben:
> > Applied, thanks.
> >
> > Please update the changelog at https://wiki.qemu.org/ChangeLog/8.2 for any user-visible changes.
>
> Something in this has broken the build for me, it seems to be the
> linux-user binary that doesn't link any more:
>
>   /usr/bin/ld: libqemu-x86_64-linux-user.fa.p/target_i386_cpu.c.o: in function `cpu_x86_cpuid':
>   /home/kwolf/source/qemu/build-clang/../target/i386/cpu.c:6180: undefined reference to `kvm_arch_get_supported_cpuid'
>   /usr/bin/ld: libqemu-x86_64-linux-user.fa.p/target_i386_cpu.c.o: in function `x86_cpu_filter_features':
>   /home/kwolf/source/qemu/build-clang/../target/i386/cpu.c:7158: undefined reference to `kvm_arch_get_supported_cpuid'
>   /usr/bin/ld: /home/kwolf/source/qemu/build-clang/../target/i386/cpu.c:7159: undefined reference to `kvm_arch_get_supported_cpuid'
>   /usr/bin/ld: /home/kwolf/source/qemu/build-clang/../target/i386/cpu.c:7160: undefined reference to `kvm_arch_get_supported_cpuid'
>   /usr/bin/ld: /home/kwolf/source/qemu/build-clang/../target/i386/cpu.c:7161: undefined reference to `kvm_arch_get_supported_cpuid'
>   /usr/bin/ld: libqemu-x86_64-linux-user.fa.p/target_i386_cpu.c.o:/home/kwolf/source/qemu/build-clang/../target/i386/cpu.c:7162: more undefined references to `kvm_arch_get_supported_cpuid' follow
>   clang-15: error: linker command failed with exit code 1 (use -v to see invocation)
>
> In case it makes a difference, I'm using clang on F37.
>
> Kevin
Re: [PULL 00/51] Build system, i386 changes for 2023-09-07
Posted by Philippe Mathieu-Daudé 8 months ago
On 8/9/23 17:47, Stefan Hajnoczi wrote:
> I wonder how it passed CI?
> https://gitlab.com/qemu-project/qemu/-/pipelines/996175923/

The conditions are:
- x86 host
- both system / user emulation enabled
- KVM disabled
- debug enabled

We have jobs with 3 of the 4, but none with
all the 4.

Is it worth test it?

> 
> Stefan
> 
> On Fri, 8 Sept 2023 at 11:02, Kevin Wolf <kwolf@redhat.com> wrote:
>>
>> Am 07.09.2023 um 17:44 hat Stefan Hajnoczi geschrieben:
>>> Applied, thanks.
>>>
>>> Please update the changelog at https://wiki.qemu.org/ChangeLog/8.2 for any user-visible changes.
>>
>> Something in this has broken the build for me, it seems to be the
>> linux-user binary that doesn't link any more:
>>
>>    /usr/bin/ld: libqemu-x86_64-linux-user.fa.p/target_i386_cpu.c.o: in function `cpu_x86_cpuid':
>>    /home/kwolf/source/qemu/build-clang/../target/i386/cpu.c:6180: undefined reference to `kvm_arch_get_supported_cpuid'
>>    /usr/bin/ld: libqemu-x86_64-linux-user.fa.p/target_i386_cpu.c.o: in function `x86_cpu_filter_features':
>>    /home/kwolf/source/qemu/build-clang/../target/i386/cpu.c:7158: undefined reference to `kvm_arch_get_supported_cpuid'
>>    /usr/bin/ld: /home/kwolf/source/qemu/build-clang/../target/i386/cpu.c:7159: undefined reference to `kvm_arch_get_supported_cpuid'
>>    /usr/bin/ld: /home/kwolf/source/qemu/build-clang/../target/i386/cpu.c:7160: undefined reference to `kvm_arch_get_supported_cpuid'
>>    /usr/bin/ld: /home/kwolf/source/qemu/build-clang/../target/i386/cpu.c:7161: undefined reference to `kvm_arch_get_supported_cpuid'
>>    /usr/bin/ld: libqemu-x86_64-linux-user.fa.p/target_i386_cpu.c.o:/home/kwolf/source/qemu/build-clang/../target/i386/cpu.c:7162: more undefined references to `kvm_arch_get_supported_cpuid' follow
>>    clang-15: error: linker command failed with exit code 1 (use -v to see invocation)
>>
>> In case it makes a difference, I'm using clang on F37.
>>
>> Kevin
>
Re: [PULL 00/51] Build system, i386 changes for 2023-09-07
Posted by Stefan Hajnoczi 8 months ago
On Mon, 11 Sept 2023 at 06:10, Philippe Mathieu-Daudé <philmd@linaro.org> wrote:
>
> On 8/9/23 17:47, Stefan Hajnoczi wrote:
> > I wonder how it passed CI?
> > https://gitlab.com/qemu-project/qemu/-/pipelines/996175923/
>
> The conditions are:
> - x86 host
> - both system / user emulation enabled
> - KVM disabled
> - debug enabled
>
> We have jobs with 3 of the 4, but none with
> all the 4.
>
> Is it worth test it?

I think so.

Kevin: Can you confirm your configuration matches what Philippe described?

Stefan

>
> >
> > Stefan
> >
> > On Fri, 8 Sept 2023 at 11:02, Kevin Wolf <kwolf@redhat.com> wrote:
> >>
> >> Am 07.09.2023 um 17:44 hat Stefan Hajnoczi geschrieben:
> >>> Applied, thanks.
> >>>
> >>> Please update the changelog at https://wiki.qemu.org/ChangeLog/8.2 for any user-visible changes.
> >>
> >> Something in this has broken the build for me, it seems to be the
> >> linux-user binary that doesn't link any more:
> >>
> >>    /usr/bin/ld: libqemu-x86_64-linux-user.fa.p/target_i386_cpu.c.o: in function `cpu_x86_cpuid':
> >>    /home/kwolf/source/qemu/build-clang/../target/i386/cpu.c:6180: undefined reference to `kvm_arch_get_supported_cpuid'
> >>    /usr/bin/ld: libqemu-x86_64-linux-user.fa.p/target_i386_cpu.c.o: in function `x86_cpu_filter_features':
> >>    /home/kwolf/source/qemu/build-clang/../target/i386/cpu.c:7158: undefined reference to `kvm_arch_get_supported_cpuid'
> >>    /usr/bin/ld: /home/kwolf/source/qemu/build-clang/../target/i386/cpu.c:7159: undefined reference to `kvm_arch_get_supported_cpuid'
> >>    /usr/bin/ld: /home/kwolf/source/qemu/build-clang/../target/i386/cpu.c:7160: undefined reference to `kvm_arch_get_supported_cpuid'
> >>    /usr/bin/ld: /home/kwolf/source/qemu/build-clang/../target/i386/cpu.c:7161: undefined reference to `kvm_arch_get_supported_cpuid'
> >>    /usr/bin/ld: libqemu-x86_64-linux-user.fa.p/target_i386_cpu.c.o:/home/kwolf/source/qemu/build-clang/../target/i386/cpu.c:7162: more undefined references to `kvm_arch_get_supported_cpuid' follow
> >>    clang-15: error: linker command failed with exit code 1 (use -v to see invocation)
> >>
> >> In case it makes a difference, I'm using clang on F37.
> >>
> >> Kevin
> >
>
Re: [PULL 00/51] Build system, i386 changes for 2023-09-07
Posted by Philippe Mathieu-Daudé 8 months ago
On 11/9/23 12:10, Philippe Mathieu-Daudé wrote:
> On 8/9/23 17:47, Stefan Hajnoczi wrote:
>> I wonder how it passed CI?
>> https://gitlab.com/qemu-project/qemu/-/pipelines/996175923/
> 
> The conditions are:
> - x86 host
> - both system / user emulation enabled
> - KVM disabled
> - debug enabled

Oh, I forgot:

- clang compiler

(because this isn't an issue with GCC).

> We have jobs with 3 of the 4, but none with
> all the 4.
> 
> Is it worth test it?


Re: [PULL 00/51] Build system, i386 changes for 2023-09-07
Posted by Kevin Wolf 8 months ago
Am 11.09.2023 um 12:22 hat Philippe Mathieu-Daudé geschrieben:
> On 11/9/23 12:10, Philippe Mathieu-Daudé wrote:
> > On 8/9/23 17:47, Stefan Hajnoczi wrote:
> > > I wonder how it passed CI?
> > > https://gitlab.com/qemu-project/qemu/-/pipelines/996175923/
> > 
> > The conditions are:
> > - x86 host

I didn't actually try other hosts, but why do you think it depends on
the host? Is it not x86 target?

> > - both system / user emulation enabled

I did try this one while bisection because I didn't want to build the
system emulator in each step. User emulation only is enough, you don't
need to build the system emulator to get the build failure.

> > - KVM disabled

I didn't explicitly disable KVM, but I suppose it's automatically
disabled for linux-user.

> > - debug enabled
> 
> Oh, I forgot:
> 
> - clang compiler
> 
> (because this isn't an issue with GCC).
> 
> > We have jobs with 3 of the 4, but none with
> > all the 4.
> > 
> > Is it worth test it?

So it's x86, user emulation is built, debug enabled, on clang. This
doesn't feel very exotic, though of course --enable-debug will be more
common for developers while it's not the primary test case for CI.

I don't see any existing test to which it could be added without losing
another important configuration. Not sure if it's worth an additional
test when we're already hitting the CI limits too often.

On the other hand, now that I'm looking at the CI configuration, is it
correct that we have no build for clang + --enable-debug at all? Even if
you ignore the other conditions? Maybe that's basic enough that we
should have it.

Kevin