[PULL 0/9] final misc fixes for 5.1-rc0

Alex Bennée posted 9 patches 3 years, 9 months ago
Only 2 patches received!
There is a newer version of this series
docs/devel/multi-thread-tcg.rst            |  2 +-
configure                                  | 18 +++---
include/hw/core/cpu.h                      | 16 +++++
include/qemu/typedefs.h                    |  1 +
accel/tcg/cputlb.c                         | 38 +++++++++++-
fpu/softfloat-specialize.inc.c             |  4 +-
tests/plugin/bb.c                          | 97 ++++++++++++++++++++++++++----
.gitlab-ci.d/containers.yml                |  1 +
tests/docker/docker.py                     | 16 +++--
tests/docker/dockerfiles/ubuntu2004.docker |  3 -
tests/plugin/Makefile                      | 22 +++++--
11 files changed, 178 insertions(+), 40 deletions(-)
[PULL 0/9] final misc fixes for 5.1-rc0
Posted by Alex Bennée 3 years, 9 months ago
The following changes since commit 20c1df5476e1e9b5d3f5b94f9f3ce01d21f14c46:

  Merge remote-tracking branch 'remotes/kraxel/tags/fixes-20200713-pull-request' into staging (2020-07-13 16:58:44 +0100)

are available in the Git repository at:

  https://github.com/stsquad/qemu.git tags/pull-misc-for-rc0-140720-1

for you to fetch changes up to 4e6400c97024c2660b6db8aab1f0677744449f36:

  plugins: expand the bb plugin to be thread safe and track per-cpu (2020-07-14 09:08:25 +0100)

----------------------------------------------------------------
Final fixes for 5.1-rc0

  - minor documentation nit
  - clean all deps on re(configure)
  - docker.py bootstrap fixes
  - tweak containers.yml wildcards
  - fix float16 nan detection
  - conditional use of -Wpsabi
  - fix missing iotlb data for plugins
  - proper locking for helper based bb count

----------------------------------------------------------------
Alex Bennée (6):
      docs/devel: fix grammar in multi-thread-tcg
      configure: remove all dependencies on a (re)configure
      docker.py: fix fetching of FROM layers
      tests/plugins: don't unconditionally add -Wpsabi
      cputlb: ensure we save the IOTLB data in case of reset
      plugins: expand the bb plugin to be thread safe and track per-cpu

LIU Zhiwei (1):
      fpu/softfloat: fix up float16 nan recognition

Thomas Huth (2):
      tests/docker: Remove the libssh workaround from the ubuntu 20.04 image
      gitlab-ci/containers: Add missing wildcard where we should look for changes

 docs/devel/multi-thread-tcg.rst            |  2 +-
 configure                                  | 18 +++---
 include/hw/core/cpu.h                      | 16 +++++
 include/qemu/typedefs.h                    |  1 +
 accel/tcg/cputlb.c                         | 38 +++++++++++-
 fpu/softfloat-specialize.inc.c             |  4 +-
 tests/plugin/bb.c                          | 97 ++++++++++++++++++++++++++----
 .gitlab-ci.d/containers.yml                |  1 +
 tests/docker/docker.py                     | 16 +++--
 tests/docker/dockerfiles/ubuntu2004.docker |  3 -
 tests/plugin/Makefile                      | 22 +++++--
 11 files changed, 178 insertions(+), 40 deletions(-)

-- 
2.20.1


Re: [PULL 0/9] final misc fixes for 5.1-rc0
Posted by Peter Maydell 3 years, 9 months ago
On Tue, 14 Jul 2020 at 10:52, Alex Bennée <alex.bennee@linaro.org> wrote:
>
> The following changes since commit 20c1df5476e1e9b5d3f5b94f9f3ce01d21f14c46:
>
>   Merge remote-tracking branch 'remotes/kraxel/tags/fixes-20200713-pull-request' into staging (2020-07-13 16:58:44 +0100)
>
> are available in the Git repository at:
>
>   https://github.com/stsquad/qemu.git tags/pull-misc-for-rc0-140720-1
>
> for you to fetch changes up to 4e6400c97024c2660b6db8aab1f0677744449f36:
>
>   plugins: expand the bb plugin to be thread safe and track per-cpu (2020-07-14 09:08:25 +0100)
>
> ----------------------------------------------------------------
> Final fixes for 5.1-rc0
>
>   - minor documentation nit
>   - clean all deps on re(configure)
>   - docker.py bootstrap fixes
>   - tweak containers.yml wildcards
>   - fix float16 nan detection
>   - conditional use of -Wpsabi
>   - fix missing iotlb data for plugins
>   - proper locking for helper based bb count

Fails "make check" on aarch32 and aarch64 hosts, because
eg qemu-system-arm segfaults on startup:

$ gdb --args ./arm-softmmu/qemu-system-arm -M virt -display none
[...]
(gdb) r
Starting program: /home/pm/qemu/build/all/arm-softmmu/qemu-system-arm
-M virt -display none
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/aarch64-linux-gnu/libthread_db.so.1".
[New Thread 0xffffe873e550 (LWP 1666261)]
[New Thread 0xffffe7f3d550 (LWP 1666262)]
[New Thread 0xffffe72e7550 (LWP 1666263)]

Thread 4 "qemu-system-arm" received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0xffffe72e7550 (LWP 1666263)]
0x0000aaaaab1da004 in object_class_dynamic_cast_assert (class=0xaaaaabc209e0,
    typename=typename@entry=0xaaaaab33bfe8 "cpu",
file=file@entry=0xaaaaab3201a0 "/home/pm/qemu/accel/tcg/cputlb.c",
    line=line@entry=1025, func=func@entry=0xaaaaab3202f0
<__func__.35278> "tlb_fill") at /home/pm/qemu/qom/object.c:917
917         trace_object_class_dynamic_cast_assert(class ?
class->type->name : "(null)",
(gdb) bt
#0  0x0000aaaaab1da004 in object_class_dynamic_cast_assert
    (class=0xaaaaabc209e0, typename=typename@entry=0xaaaaab33bfe8
"cpu", file=file@entry=0xaaaaab3201a0
"/home/pm/qemu/accel/tcg/cputlb.c", line=line@entry=1025,
func=func@entry=0xaaaaab3202f0 <__func__.35278> "tlb_fill")
    at /home/pm/qemu/qom/object.c:917
#1  0x0000aaaaaaded754 in tlb_fill
    (cpu=0xaaaaabd301c0, addr=0, size=0, access_type=MMU_INST_FETCH,
mmu_idx=2, retaddr=0)
    at /home/pm/qemu/accel/tcg/cputlb.c:1025
#2  0x0000aaaaaadf0e08 in get_page_addr_code_hostp
(env=0xaaaaabd39870, addr=addr@entry=0, hostp=hostp@entry=0x0)
    at /home/pm/qemu/include/exec/cpu-all.h:451
#3  0x0000aaaaaadf0e70 in get_page_addr_code (env=<optimized out>,
addr=addr@entry=0)
    at /home/pm/qemu/accel/tcg/cputlb.c:1243
#4  0x0000aaaaaae08c30 in tb_htable_lookup
    (cpu=cpu@entry=0xaaaaabd301b0, pc=0, cs_base=<optimized out>,
flags=38928384, cf_mask=4278714368)
    at /home/pm/qemu/accel/tcg/cpu-exec.c:337
#5  0x0000aaaaaae09b14 in tb_lookup__cpu_state
    (cf_mask=<optimized out>, flags=0xffffe72e6b48,
cs_base=0xffffe72e6b40, pc=0xffffe72e6b44, cpu=0xaaaaabd301b0)
    at /home/pm/qemu/include/exec/tb-lookup.h:43
#6  tb_find (cf_mask=<optimized out>, tb_exit=0, last_tb=0x0,
cpu=0xaaaaabd301b0)
    at /home/pm/qemu/accel/tcg/cpu-exec.c:404
#7  cpu_exec (cpu=cpu@entry=0xaaaaabd301b0) at
/home/pm/qemu/accel/tcg/cpu-exec.c:731
#8  0x0000aaaaaaec45c4 in tcg_cpu_exec (cpu=0xaaaaabd301b0) at
/home/pm/qemu/softmmu/cpus.c:1356
#9  0x0000aaaaaaec69ec in qemu_tcg_cpu_thread_fn
(arg=arg@entry=0xaaaaabd301b0) at /home/pm/qemu/softmmu/cpus.c:1664
#10 0x0000aaaaab2e1758 in qemu_thread_start (args=<optimized out>) at
/home/pm/qemu/util/qemu-thread-posix.c:521
#11 0x0000fffff57be4fc in start_thread (arg=0xffffffffe10f) at
pthread_create.c:477
#12 0x0000fffff5716f2c in thread_start () at
../sysdeps/unix/sysv/linux/aarch64/clone.S:78

thanks
-- PMM

Re: [PULL 0/9] final misc fixes for 5.1-rc0
Posted by Alex Bennée 3 years, 9 months ago
Peter Maydell <peter.maydell@linaro.org> writes:

> On Tue, 14 Jul 2020 at 10:52, Alex Bennée <alex.bennee@linaro.org> wrote:
>>
>> The following changes since commit 20c1df5476e1e9b5d3f5b94f9f3ce01d21f14c46:
>>
>>   Merge remote-tracking branch 'remotes/kraxel/tags/fixes-20200713-pull-request' into staging (2020-07-13 16:58:44 +0100)
>>
>> are available in the Git repository at:
>>
>>   https://github.com/stsquad/qemu.git tags/pull-misc-for-rc0-140720-1
>>
>> for you to fetch changes up to 4e6400c97024c2660b6db8aab1f0677744449f36:
>>
>>   plugins: expand the bb plugin to be thread safe and track per-cpu (2020-07-14 09:08:25 +0100)
>>
>> ----------------------------------------------------------------
>> Final fixes for 5.1-rc0
>>
>>   - minor documentation nit
>>   - clean all deps on re(configure)
>>   - docker.py bootstrap fixes
>>   - tweak containers.yml wildcards
>>   - fix float16 nan detection
>>   - conditional use of -Wpsabi
>>   - fix missing iotlb data for plugins
>>   - proper locking for helper based bb count
>
> Fails "make check" on aarch32 and aarch64 hosts, because
> eg qemu-system-arm segfaults on startup:
>
> $ gdb --args ./arm-softmmu/qemu-system-arm -M virt -display none
> [...]
> (gdb) r
> Starting program: /home/pm/qemu/build/all/arm-softmmu/qemu-system-arm
> -M virt -display none
> [Thread debugging using libthread_db enabled]
> Using host libthread_db library "/lib/aarch64-linux-gnu/libthread_db.so.1".
> [New Thread 0xffffe873e550 (LWP 1666261)]
> [New Thread 0xffffe7f3d550 (LWP 1666262)]
> [New Thread 0xffffe72e7550 (LWP 1666263)]
>
> Thread 4 "qemu-system-arm" received signal SIGSEGV, Segmentation fault.
> [Switching to Thread 0xffffe72e7550 (LWP 1666263)]
> 0x0000aaaaab1da004 in object_class_dynamic_cast_assert (class=0xaaaaabc209e0,
>     typename=typename@entry=0xaaaaab33bfe8 "cpu",
> file=file@entry=0xaaaaab3201a0 "/home/pm/qemu/accel/tcg/cputlb.c",
>     line=line@entry=1025, func=func@entry=0xaaaaab3202f0
> <__func__.35278> "tlb_fill") at /home/pm/qemu/qom/object.c:917
> 917         trace_object_class_dynamic_cast_assert(class ?
> class->type->name : "(null)",
> (gdb) bt
> #0  0x0000aaaaab1da004 in object_class_dynamic_cast_assert
>     (class=0xaaaaabc209e0, typename=typename@entry=0xaaaaab33bfe8
> "cpu", file=file@entry=0xaaaaab3201a0
> "/home/pm/qemu/accel/tcg/cputlb.c", line=line@entry=1025,
> func=func@entry=0xaaaaab3202f0 <__func__.35278> "tlb_fill")
>     at /home/pm/qemu/qom/object.c:917
> #1  0x0000aaaaaaded754 in tlb_fill
>     (cpu=0xaaaaabd301c0, addr=0, size=0, access_type=MMU_INST_FETCH,
> mmu_idx=2, retaddr=0)
>     at /home/pm/qemu/accel/tcg/cputlb.c:1025
> #2  0x0000aaaaaadf0e08 in get_page_addr_code_hostp
> (env=0xaaaaabd39870, addr=addr@entry=0, hostp=hostp@entry=0x0)
>     at /home/pm/qemu/include/exec/cpu-all.h:451
> #3  0x0000aaaaaadf0e70 in get_page_addr_code (env=<optimized out>,
> addr=addr@entry=0)
>     at /home/pm/qemu/accel/tcg/cputlb.c:1243
> #4  0x0000aaaaaae08c30 in tb_htable_lookup
>     (cpu=cpu@entry=0xaaaaabd301b0, pc=0, cs_base=<optimized out>,
> flags=38928384, cf_mask=4278714368)
>     at /home/pm/qemu/accel/tcg/cpu-exec.c:337
> #5  0x0000aaaaaae09b14 in tb_lookup__cpu_state
>     (cf_mask=<optimized out>, flags=0xffffe72e6b48,
> cs_base=0xffffe72e6b40, pc=0xffffe72e6b44, cpu=0xaaaaabd301b0)
>     at /home/pm/qemu/include/exec/tb-lookup.h:43
> #6  tb_find (cf_mask=<optimized out>, tb_exit=0, last_tb=0x0,
> cpu=0xaaaaabd301b0)
>     at /home/pm/qemu/accel/tcg/cpu-exec.c:404
> #7  cpu_exec (cpu=cpu@entry=0xaaaaabd301b0) at
> /home/pm/qemu/accel/tcg/cpu-exec.c:731
> #8  0x0000aaaaaaec45c4 in tcg_cpu_exec (cpu=0xaaaaabd301b0) at
> /home/pm/qemu/softmmu/cpus.c:1356
> #9  0x0000aaaaaaec69ec in qemu_tcg_cpu_thread_fn
> (arg=arg@entry=0xaaaaabd301b0) at /home/pm/qemu/softmmu/cpus.c:1664
> #10 0x0000aaaaab2e1758 in qemu_thread_start (args=<optimized out>) at
> /home/pm/qemu/util/qemu-thread-posix.c:521
> #11 0x0000fffff57be4fc in start_thread (arg=0xffffffffe10f) at
> pthread_create.c:477
> #12 0x0000fffff5716f2c in thread_start () at
> ../sysdeps/unix/sysv/linux/aarch64/clone.S:78

This goes away after a make clean and rebuild. I suspect the commit:

  configure: remove all dependencies on a (re)configure

just causes the build to miss more of the bits on a rebuild. I'll drop
it for now and just live with the random syscall_nr.h files.

-- 
Alex Bennée

Re: [PULL 0/9] final misc fixes for 5.1-rc0
Posted by Philippe Mathieu-Daudé 3 years, 9 months ago
Hi Peter,

On 7/15/20 10:06 AM, Peter Maydell wrote:
> On Tue, 14 Jul 2020 at 10:52, Alex Bennée <alex.bennee@linaro.org> wrote:
>>
>> The following changes since commit 20c1df5476e1e9b5d3f5b94f9f3ce01d21f14c46:
>>
>>   Merge remote-tracking branch 'remotes/kraxel/tags/fixes-20200713-pull-request' into staging (2020-07-13 16:58:44 +0100)
>>
>> are available in the Git repository at:
>>
>>   https://github.com/stsquad/qemu.git tags/pull-misc-for-rc0-140720-1
>>
>> for you to fetch changes up to 4e6400c97024c2660b6db8aab1f0677744449f36:
>>
>>   plugins: expand the bb plugin to be thread safe and track per-cpu (2020-07-14 09:08:25 +0100)
>>
>> ----------------------------------------------------------------
>> Final fixes for 5.1-rc0
>>
>>   - minor documentation nit
>>   - clean all deps on re(configure)
>>   - docker.py bootstrap fixes
>>   - tweak containers.yml wildcards
>>   - fix float16 nan detection
>>   - conditional use of -Wpsabi
>>   - fix missing iotlb data for plugins
>>   - proper locking for helper based bb count
> 
> Fails "make check" on aarch32 and aarch64 hosts, because
> eg qemu-system-arm segfaults on startup:

What host/distrib are you using?

I can not reproduce on aarch64 (Ubuntu 20.04 LTS) using:
gcc (Ubuntu 9.3.0-10ubuntu2) 9.3.0

Any config in particular?

> 
> $ gdb --args ./arm-softmmu/qemu-system-arm -M virt -display none
> [...]
> (gdb) r
> Starting program: /home/pm/qemu/build/all/arm-softmmu/qemu-system-arm
> -M virt -display none
> [Thread debugging using libthread_db enabled]
> Using host libthread_db library "/lib/aarch64-linux-gnu/libthread_db.so.1".
> [New Thread 0xffffe873e550 (LWP 1666261)]
> [New Thread 0xffffe7f3d550 (LWP 1666262)]
> [New Thread 0xffffe72e7550 (LWP 1666263)]
> 
> Thread 4 "qemu-system-arm" received signal SIGSEGV, Segmentation fault.
> [Switching to Thread 0xffffe72e7550 (LWP 1666263)]
> 0x0000aaaaab1da004 in object_class_dynamic_cast_assert (class=0xaaaaabc209e0,
>     typename=typename@entry=0xaaaaab33bfe8 "cpu",
> file=file@entry=0xaaaaab3201a0 "/home/pm/qemu/accel/tcg/cputlb.c",
>     line=line@entry=1025, func=func@entry=0xaaaaab3202f0
> <__func__.35278> "tlb_fill") at /home/pm/qemu/qom/object.c:917
> 917         trace_object_class_dynamic_cast_assert(class ?
> class->type->name : "(null)",

Re: [PULL 0/9] final misc fixes for 5.1-rc0
Posted by Peter Maydell 3 years, 9 months ago
On Wed, 15 Jul 2020 at 13:29, Philippe Mathieu-Daudé <f4bug@amsat.org> wrote:
> What host/distrib are you using?
>
> I can not reproduce on aarch64 (Ubuntu 20.04 LTS) using:
> gcc (Ubuntu 9.3.0-10ubuntu2) 9.3.0

Ubuntu 20.04 LTS (this is the aarch64.ci.qemu.org packet.net box
you have access to). Also Ubuntu 18.04.4 LTS aarch32 chroot on
an aarch64 system.

> Any config in particular?

Nope, stock "configure" with no arguments. Incremental build, not
from-clean.

thanks
-- PMM

Re: [PULL 0/9] final misc fixes for 5.1-rc0
Posted by Philippe Mathieu-Daudé 3 years, 9 months ago
On 7/15/20 2:51 PM, Peter Maydell wrote:
> On Wed, 15 Jul 2020 at 13:29, Philippe Mathieu-Daudé <f4bug@amsat.org> wrote:
>> What host/distrib are you using?
>>
>> I can not reproduce on aarch64 (Ubuntu 20.04 LTS) using:
>> gcc (Ubuntu 9.3.0-10ubuntu2) 9.3.0
> 
> Ubuntu 20.04 LTS (this is the aarch64.ci.qemu.org packet.net box
> you have access to).

Yes this is where I tested.

> Also Ubuntu 18.04.4 LTS aarch32 chroot on
> an aarch64 system.
> 
>> Any config in particular?
> 
> Nope, stock "configure" with no arguments. Incremental build, not
> from-clean.

OK, I tested from-clean (in case that helps Alex).

Thanks,

Phil.

[PULL 1/9] docs/devel: fix grammar in multi-thread-tcg
Posted by Alex Bennée 3 years, 9 months ago
Review comment came just too late ;-)

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20200713200415.26214-9-alex.bennee@linaro.org>

diff --git a/docs/devel/multi-thread-tcg.rst b/docs/devel/multi-thread-tcg.rst
index 42158b77c7..21483870db 100644
--- a/docs/devel/multi-thread-tcg.rst
+++ b/docs/devel/multi-thread-tcg.rst
@@ -107,7 +107,7 @@ including:
 
   - debugging operations (breakpoint insertion/removal)
   - some CPU helper functions
-  - linux-user spawning it's first thread
+  - linux-user spawning its first thread
 
 This is done with the async_safe_run_on_cpu() mechanism to ensure all
 vCPUs are quiescent when changes are being made to shared global
-- 
2.20.1


[PULL 2/9] configure: remove all dependencies on a (re)configure
Posted by Alex Bennée 3 years, 9 months ago
The previous code was brittle and missed cases such as the mipn32
variants which for some reason has the 64 bit syscalls. This leads to
a number of binary targets having deps lines like:

  all.clang-sanitizer/mipsn32el-linux-user/linux-user/signal.d
  140:  /home/alex/lsrc/qemu.git/linux-user/mips64/syscall_nr.h \
  455:/home/alex/lsrc/qemu.git/linux-user/mips64/syscall_nr.h:

  all.clang-sanitizer/mipsn32el-linux-user/linux-user/syscall.d
  146:  /home/alex/lsrc/qemu.git/linux-user/mips64/syscall_nr.h \
  485:/home/alex/lsrc/qemu.git/linux-user/mips64/syscall_nr.h:

which in turn would trigger the re-generation of syscall_nr.h in the
source tree (thanks to generic %/syscall_nr.h rules). The previous
code attempts to clean it out but misses edge cases but fails.

After spending a day trying to understand how this was happening I'm
unconvinced that there are not other such breakages possible with this
"caching". As we add more auto-generated code to the build it is likely
to trip up again. Apply a hammer to the problem.

Fixes: 91e5998f18 (which fixes 5f29856b852d and 4d6a835dea47)
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20200713200415.26214-2-alex.bennee@linaro.org>

diff --git a/configure b/configure
index bc3b9ad931..e1de2f5b24 100755
--- a/configure
+++ b/configure
@@ -1955,23 +1955,20 @@ EOF
 exit 0
 fi
 
-# Remove old dependency files to make sure that they get properly regenerated
-rm -f */config-devices.mak.d
-
 # Remove syscall_nr.h to be sure they will be regenerated in the build
 # directory, not in the source directory
 for arch in alpha hppa m68k xtensa sh4 microblaze arm ppc s390x sparc sparc64 \
     i386 x86_64 mips mips64 ; do
     # remove the file if it has been generated in the source directory
     rm -f "${source_path}/linux-user/${arch}/syscall_nr.h"
-    # remove the dependency files
-    for target in ${arch}*-linux-user ; do
-        test -d "${target}" && find "${target}" -type f -name "*.d" \
-             -exec grep -q "${source_path}/linux-user/${arch}/syscall_nr.h" {} \; \
-             -print | while read file ; do rm "${file}" "${file%.d}.o" ; done
-    done
 done
 
+# Clean out all old dependency files. As more files are generated we
+# run the risk of old dependencies triggering generation in the wrong
+# places. Previous brittle attempts to be surgical tend to miss edge
+# cases leading to wasted time and much confusion.
+find -type f -name "*.d" -exec rm -f {} \;
+
 if test -z "$python"
 then
     error_exit "Python not found. Use --python=/path/to/python"
-- 
2.20.1


[PULL 3/9] tests/docker: Remove the libssh workaround from the ubuntu 20.04 image
Posted by Alex Bennée 3 years, 9 months ago
From: Thomas Huth <thuth@redhat.com>

The libssh problem only exists in Ubuntu 18.04 - we can enable it
in 20.04 again.

Signed-off-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Message-Id: <20200713185237.9419-1-thuth@redhat.com>
Message-Id: <20200713200415.26214-3-alex.bennee@linaro.org>

diff --git a/tests/docker/dockerfiles/ubuntu2004.docker b/tests/docker/dockerfiles/ubuntu2004.docker
index f7aac840bf..8d10934a2a 100644
--- a/tests/docker/dockerfiles/ubuntu2004.docker
+++ b/tests/docker/dockerfiles/ubuntu2004.docker
@@ -65,9 +65,6 @@ RUN apt-get update && \
 RUN dpkg -l $PACKAGES | sort > /packages.txt
 ENV FEATURES clang tsan pyyaml sdl2
 
-# https://bugs.launchpad.net/qemu/+bug/1838763
-ENV QEMU_CONFIGURE_OPTS --disable-libssh
-
 # Apply patch https://reviews.llvm.org/D75820
 # This is required for TSan in clang-10 to compile with QEMU.
 RUN sed -i 's/^const/static const/g' /usr/lib/llvm-10/lib/clang/10.0.0/include/sanitizer/tsan_interface.h
-- 
2.20.1


[PULL 4/9] docker.py: fix fetching of FROM layers
Posted by Alex Bennée 3 years, 9 months ago
This worked on a system that was already bootstrapped because the
stage 2 images already existed even if they wouldn't be used. What we
should have pulled down was the FROM line containers first because
building on gitlab doesn't have the advantage of using our build
system to build the pre-requisite bits.

We still pull the image we want to build just in case we can use the
cached data.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Tested-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Message-Id: <20200713200415.26214-4-alex.bennee@linaro.org>

diff --git a/tests/docker/docker.py b/tests/docker/docker.py
index 2d67bbd15a..c9f20d8d09 100755
--- a/tests/docker/docker.py
+++ b/tests/docker/docker.py
@@ -306,14 +306,18 @@ class Docker(object):
         checksum = _text_checksum(_dockerfile_preprocess(dockerfile))
 
         if registry is not None:
-            # see if we can fetch a cache copy, may fail...
-            pull_args = ["pull", "%s/%s" % (registry, tag)]
-            if self._do(pull_args, quiet=quiet) == 0:
+            sources = re.findall("FROM qemu\/(.*)", dockerfile)
+            # Fetch any cache layers we can, may fail
+            for s in sources:
+                pull_args = ["pull", "%s/qemu/%s" % (registry, s)]
+                if self._do(pull_args, quiet=quiet) != 0:
+                    registry = None
+                    break
+            # Make substitutions
+            if registry is not None:
                 dockerfile = dockerfile.replace("FROM qemu/",
                                                 "FROM %s/qemu/" %
                                                 (registry))
-            else:
-                registry = None
 
         tmp_df = tempfile.NamedTemporaryFile(mode="w+t",
                                              encoding='utf-8',
@@ -339,6 +343,8 @@ class Docker(object):
             build_args += ["--build-arg", "BUILDKIT_INLINE_CACHE=1"]
 
         if registry is not None:
+            pull_args = ["pull", "%s/%s" % (registry, tag)]
+            self._do(pull_args, quiet=quiet)
             cache = "%s/%s" % (registry, tag)
             build_args += ["--cache-from", cache]
         build_args += argv
-- 
2.20.1


[PULL 5/9] gitlab-ci/containers: Add missing wildcard where we should look for changes
Posted by Alex Bennée 3 years, 9 months ago
From: Thomas Huth <thuth@redhat.com>

The tests/docker/* wildcard seems to only match the files that are directly
in the tests/docker folder - but changes to the files in the directory
tests/docker/dockerfiles are currently ignored. Seems like we need a
separate entry to match the files in that folder. With this wildcard added,
the stages now get re-run successfully when something in the dockerfiles
has been changed.

Signed-off-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20200713182235.30379-1-thuth@redhat.com>

diff --git a/.gitlab-ci.d/containers.yml b/.gitlab-ci.d/containers.yml
index f3c0ca4d61..8c89efeb6d 100644
--- a/.gitlab-ci.d/containers.yml
+++ b/.gitlab-ci.d/containers.yml
@@ -24,6 +24,7 @@
     - changes:
       - .gitlab-ci.d/containers.yml
       - tests/docker/*
+      - tests/docker/dockerfiles/*
     - if: '$CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH'
     - if: '$CI_COMMIT_REF_NAME == "testing/next"'
 
-- 
2.20.1


[PULL 6/9] fpu/softfloat: fix up float16 nan recognition
Posted by Alex Bennée 3 years, 9 months ago
From: LIU Zhiwei <zhiwei_liu@c-sky.com>

Signed-off-by: LIU Zhiwei <zhiwei_liu@c-sky.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20200712234521.3972-2-zhiwei_liu@c-sky.com>
Message-Id: <20200713200415.26214-5-alex.bennee@linaro.org>

diff --git a/fpu/softfloat-specialize.inc.c b/fpu/softfloat-specialize.inc.c
index 44f5b661f8..034d18199c 100644
--- a/fpu/softfloat-specialize.inc.c
+++ b/fpu/softfloat-specialize.inc.c
@@ -254,7 +254,7 @@ bool float16_is_quiet_nan(float16 a_, float_status *status)
     if (snan_bit_is_one(status)) {
         return (((a >> 9) & 0x3F) == 0x3E) && (a & 0x1FF);
     } else {
-        return ((a & ~0x8000) >= 0x7C80);
+        return ((a >> 9) & 0x3F) == 0x3F;
     }
 #endif
 }
@@ -271,7 +271,7 @@ bool float16_is_signaling_nan(float16 a_, float_status *status)
 #else
     uint16_t a = float16_val(a_);
     if (snan_bit_is_one(status)) {
-        return ((a & ~0x8000) >= 0x7C80);
+        return ((a >> 9) & 0x3F) == 0x3F;
     } else {
         return (((a >> 9) & 0x3F) == 0x3E) && (a & 0x1FF);
     }
-- 
2.20.1


[PULL 9/9] plugins: expand the bb plugin to be thread safe and track per-cpu
Posted by Alex Bennée 3 years, 9 months ago
While there isn't any easy way to make the inline counts thread safe
we can ensure the callback based ones are. While we are at it we can
reduce introduce a new option ("idle") to dump a report of the current
bb and insn count each time a vCPU enters the idle state.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Robert Foley <robert.foley@linaro.org>
Cc: Dave Bort <dbort@dbort.com>

Message-Id: <20200713200415.26214-8-alex.bennee@linaro.org>

diff --git a/tests/plugin/bb.c b/tests/plugin/bb.c
index df19fd359d..e4cc7fdd6e 100644
--- a/tests/plugin/bb.c
+++ b/tests/plugin/bb.c
@@ -16,24 +16,67 @@
 
 QEMU_PLUGIN_EXPORT int qemu_plugin_version = QEMU_PLUGIN_VERSION;
 
-static uint64_t bb_count;
-static uint64_t insn_count;
+typedef struct {
+    GMutex lock;
+    int index;
+    uint64_t bb_count;
+    uint64_t insn_count;
+} CPUCount;
+
+/* Used by the inline & linux-user counts */
 static bool do_inline;
+static CPUCount inline_count;
+
+/* Dump running CPU total on idle? */
+static bool idle_report;
+static GPtrArray *counts;
+static int max_cpus;
+
+static void gen_one_cpu_report(CPUCount *count, GString *report)
+{
+    if (count->bb_count) {
+        g_string_append_printf(report, "CPU%d: "
+                               "bb's: %" PRIu64", insns: %" PRIu64 "\n",
+                               count->index,
+                               count->bb_count, count->insn_count);
+    }
+}
 
 static void plugin_exit(qemu_plugin_id_t id, void *p)
 {
-    g_autofree gchar *out = g_strdup_printf(
-        "bb's: %" PRIu64", insns: %" PRIu64 "\n",
-        bb_count, insn_count);
-    qemu_plugin_outs(out);
+    g_autoptr(GString) report = g_string_new("");
+
+    if (do_inline || !max_cpus) {
+        g_string_printf(report, "bb's: %" PRIu64", insns: %" PRIu64 "\n",
+                        inline_count.bb_count, inline_count.insn_count);
+    } else {
+        g_ptr_array_foreach(counts, (GFunc) gen_one_cpu_report, report);
+    }
+    qemu_plugin_outs(report->str);
+}
+
+static void vcpu_idle(qemu_plugin_id_t id, unsigned int cpu_index)
+{
+    CPUCount *count = g_ptr_array_index(counts, cpu_index);
+    g_autoptr(GString) report = g_string_new("");
+    gen_one_cpu_report(count, report);
+
+    if (report->len > 0) {
+        g_string_prepend(report, "Idling ");
+        qemu_plugin_outs(report->str);
+    }
 }
 
 static void vcpu_tb_exec(unsigned int cpu_index, void *udata)
 {
-    unsigned long n_insns = (unsigned long)udata;
+    CPUCount *count = max_cpus ?
+        g_ptr_array_index(counts, cpu_index) : &inline_count;
 
-    insn_count += n_insns;
-    bb_count++;
+    unsigned long n_insns = (unsigned long)udata;
+    g_mutex_lock(&count->lock);
+    count->insn_count += n_insns;
+    count->bb_count++;
+    g_mutex_unlock(&count->lock);
 }
 
 static void vcpu_tb_trans(qemu_plugin_id_t id, struct qemu_plugin_tb *tb)
@@ -42,9 +85,10 @@ static void vcpu_tb_trans(qemu_plugin_id_t id, struct qemu_plugin_tb *tb)
 
     if (do_inline) {
         qemu_plugin_register_vcpu_tb_exec_inline(tb, QEMU_PLUGIN_INLINE_ADD_U64,
-                                                 &bb_count, 1);
+                                                 &inline_count.bb_count, 1);
         qemu_plugin_register_vcpu_tb_exec_inline(tb, QEMU_PLUGIN_INLINE_ADD_U64,
-                                                 &insn_count, n_insns);
+                                                 &inline_count.insn_count,
+                                                 n_insns);
     } else {
         qemu_plugin_register_vcpu_tb_exec_cb(tb, vcpu_tb_exec,
                                              QEMU_PLUGIN_CB_NO_REGS,
@@ -56,8 +100,35 @@ QEMU_PLUGIN_EXPORT int qemu_plugin_install(qemu_plugin_id_t id,
                                            const qemu_info_t *info,
                                            int argc, char **argv)
 {
-    if (argc && strcmp(argv[0], "inline") == 0) {
-        do_inline = true;
+    int i;
+
+    for (i = 0; i < argc; i++) {
+        char *opt = argv[i];
+        if (g_strcmp0(opt, "inline") == 0) {
+            do_inline = true;
+        } else if (g_strcmp0(opt, "idle") == 0) {
+            idle_report = true;
+        } else {
+            fprintf(stderr, "option parsing failed: %s\n", opt);
+            return -1;
+        }
+    }
+
+    if (info->system_emulation && !do_inline) {
+        max_cpus = info->system.max_vcpus;
+        counts = g_ptr_array_new();
+        for (i = 0; i < max_cpus; i++) {
+            CPUCount *count = g_new0(CPUCount, 1);
+            g_mutex_init(&count->lock);
+            count->index = i;
+            g_ptr_array_add(counts, count);
+        }
+    } else if (!do_inline) {
+        g_mutex_init(&inline_count.lock);
+    }
+
+    if (idle_report) {
+        qemu_plugin_register_vcpu_idle_cb(id, vcpu_idle);
     }
 
     qemu_plugin_register_vcpu_tb_trans_cb(id, vcpu_tb_trans);
-- 
2.20.1