[PATCH v2 00/13] Add Thread Sanitizer support to QEMU

Robert Foley posted 13 patches 3 years, 10 months ago
Test docker-mingw@fedora passed
Test checkpatch passed
Test asan failed
Test docker-quick@centos7 passed
Test FreeBSD passed
Failed in applying to current master (apply log)
There is a newer version of this series
accel/tcg/cputlb.c                         |  15 +++
accel/tcg/translate-all.c                  |  19 +++-
configure                                  |  47 ++++++++-
cpus-common.c                              |  25 ++---
cpus.c                                     |  14 ++-
docs/devel/testing.rst                     | 107 +++++++++++++++++++++
exec.c                                     |   1 +
hw/core/cpu.c                              |   1 +
include/exec/exec-all.h                    |   8 ++
include/hw/core/cpu.h                      |   6 +-
include/qemu/thread.h                      |  38 +++++++-
include/qemu/tsan.h                        |  71 ++++++++++++++
include/tcg/tcg.h                          |   3 +-
tcg/tcg.c                                  |  19 +++-
tests/Makefile.include                     |   9 +-
tests/docker/dockerfiles/ubuntu2004.docker |  65 +++++++++++++
tests/docker/test-tsan                     |  44 +++++++++
tests/qtest/Makefile.include               |   7 +-
tests/tsan/blacklist.tsan                  |  10 ++
tests/tsan/suppressions.tsan               |  14 +++
util/coroutine-ucontext.c                  |  97 +++++++++++++++++--
util/qemu-thread-posix.c                   |   2 +
util/qht.c                                 |   1 +
23 files changed, 581 insertions(+), 42 deletions(-)
create mode 100644 include/qemu/tsan.h
create mode 100644 tests/docker/dockerfiles/ubuntu2004.docker
create mode 100755 tests/docker/test-tsan
create mode 100644 tests/tsan/blacklist.tsan
create mode 100644 tests/tsan/suppressions.tsan
[PATCH v2 00/13] Add Thread Sanitizer support to QEMU
Posted by Robert Foley 3 years, 10 months ago
v1: https://lists.gnu.org/archive/html/qemu-devel/2020-05/msg08302.html

Changes in v2:
- Fixed make check under TSan.  With the below fixes, make check 
  under TSan completes successfully, albeit with TSan warnings.
  - We found that several unit tests and the qtests hit an issue in TSan,
    which results in a hung test.  This is a known issue: 
    https://github.com/google/sanitizers/issues/1116
  - Under TSan, disable the 3 unit tests that hit this above issue.
  - Under TSan, disable the qtests since they hit this issue too.
- Split out the docker testing for tsan into its own test (test-tsan).
- configure:  Error out if tsan and other sanitizers are used together.
- configure: Cleaned up warnings during tsan build caused by tsan libraries.

This patch series continues the work done by Emilio Cota and others to add
Thread Sanitizer (TSan) support to QEMU.

The starting point for this work was Emilio's branch here:
https://github.com/cota/qemu/commits/tsan
specifically this commit: 0be125fc0afd47218b34d2019abdd19b644f3199

The main purpose of this patch is to enable TSan support so that 
QEMU developers can start using the tool.  
We found this tool useful and even ran it on our recent changes in
the cpu-locks series, which fixes many warnings.
Clearly there is work to do here to clean up all the warnings. :)
We have also made an effort to introduce enough of the TSan suppression
mechanisms, so that others can continue this work.

This series adds support for:
- configure option for --enable-tsan.
- testing.rst has the full details on how to use TSan with or without docker,
  including all the suppression mechanisms.
- We added an Ubuntu 20.04 docker that supports TSan builds.
- test-tsan is a new docker test that builds and runs make check under TSan.
- We added an example blacklist file for files or functions TSan should ignore 
  at compile time.  This can now be specified manually.
- Added a suppression file for TSan to suppress certain warnings at run time.
- Added tsan.h with annotations which also can be used to suppress warnings.

Emilio G. Cota (7):
  cpu: convert queued work to a QSIMPLEQ
  thread: add qemu_spin_destroy
  cputlb: destroy CPUTLB with tlb_destroy
  qht: call qemu_spin_destroy for head buckets
  tcg: call qemu_spin_destroy for tb->jmp_lock
  translate-all: call qemu_spin_destroy for PageDesc
  thread: add tsan annotations to QemuSpin

Lingfeng Yang (1):
  configure: add --enable-tsan flag + fiber annotations for
    coroutine-ucontext

Robert Foley (5):
  tests/docker: Added docker build support for TSan.
  include/qemu: Added tsan.h for annotations.
  util: Added tsan annotate for thread name.
  docs: Added details on TSan to testing.rst
  tests:  Disable select tests under TSan, which hit TSan issue.

 accel/tcg/cputlb.c                         |  15 +++
 accel/tcg/translate-all.c                  |  19 +++-
 configure                                  |  47 ++++++++-
 cpus-common.c                              |  25 ++---
 cpus.c                                     |  14 ++-
 docs/devel/testing.rst                     | 107 +++++++++++++++++++++
 exec.c                                     |   1 +
 hw/core/cpu.c                              |   1 +
 include/exec/exec-all.h                    |   8 ++
 include/hw/core/cpu.h                      |   6 +-
 include/qemu/thread.h                      |  38 +++++++-
 include/qemu/tsan.h                        |  71 ++++++++++++++
 include/tcg/tcg.h                          |   3 +-
 tcg/tcg.c                                  |  19 +++-
 tests/Makefile.include                     |   9 +-
 tests/docker/dockerfiles/ubuntu2004.docker |  65 +++++++++++++
 tests/docker/test-tsan                     |  44 +++++++++
 tests/qtest/Makefile.include               |   7 +-
 tests/tsan/blacklist.tsan                  |  10 ++
 tests/tsan/suppressions.tsan               |  14 +++
 util/coroutine-ucontext.c                  |  97 +++++++++++++++++--
 util/qemu-thread-posix.c                   |   2 +
 util/qht.c                                 |   1 +
 23 files changed, 581 insertions(+), 42 deletions(-)
 create mode 100644 include/qemu/tsan.h
 create mode 100644 tests/docker/dockerfiles/ubuntu2004.docker
 create mode 100755 tests/docker/test-tsan
 create mode 100644 tests/tsan/blacklist.tsan
 create mode 100644 tests/tsan/suppressions.tsan

-- 
2.17.1


Re: [PATCH v2 00/13] Add Thread Sanitizer support to QEMU
Posted by no-reply@patchew.org 3 years, 10 months ago
Patchew URL: https://patchew.org/QEMU/20200605173422.1490-1-robert.foley@linaro.org/



Hi,

This series failed the asan build test. Please find the testing commands and
their output below. If you have Docker installed, you can probably reproduce it
locally.

=== TEST SCRIPT BEGIN ===
#!/bin/bash
export ARCH=x86_64
make docker-image-fedora V=1 NETWORK=1
time make docker-test-debug@fedora TARGET_LIST=x86_64-softmmu J=14 NETWORK=1
=== TEST SCRIPT END ===

PASS 1 fdc-test /x86_64/fdc/cmos
PASS 2 fdc-test /x86_64/fdc/no_media_on_start
PASS 3 fdc-test /x86_64/fdc/read_without_media
==9160==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases!
==9160==ERROR: finishing a fiber switch that has not started
PASS 8 test-string-input-visitor /string-visitor/input/fuzz
Broken pipe
/tmp/qemu-test/src/tests/qtest/libqtest.c:166: kill_qemu() tried to terminate QEMU process but encountered exit status 1 (expected 0)
ERROR - too few tests run (expected 13, got 3)
MALLOC_PERTURB_=${MALLOC_PERTURB_:-$(( ${RANDOM:-0} % 255 + 1))}  tests/test-string-output-visitor -m=quick -k --tap < /dev/null | ./scripts/tap-driver.pl --test-name="test-string-output-visitor" 
make: *** [/tmp/qemu-test/src/tests/Makefile.include:642: check-qtest-x86_64] Error 1
make: *** Waiting for unfinished jobs....
PASS 1 test-string-output-visitor /string-visitor/output/int
PASS 2 test-string-output-visitor /string-visitor/output/int-human
---
PASS 32 test-opts-visitor /visitor/opts/range/beyond
PASS 33 test-opts-visitor /visitor/opts/dict/unvisited
MALLOC_PERTURB_=${MALLOC_PERTURB_:-$(( ${RANDOM:-0} % 255 + 1))}  tests/test-coroutine -m=quick -k --tap < /dev/null | ./scripts/tap-driver.pl --test-name="test-coroutine" 
==9190==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases!
==9190==ERROR: finishing a fiber switch that has not started
ERROR - too few tests run (expected 10, got 0)
make: *** [/tmp/qemu-test/src/tests/Makefile.include:647: check-unit] Error 1
Traceback (most recent call last):
  File "./tests/docker/docker.py", line 665, in <module>
    sys.exit(main())
---
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['sudo', '-n', 'docker', 'run', '--label', 'com.qemu.instance.uuid=41c29776cd2c40bf8d43e98f3001772d', '-u', '1003', '--security-opt', 'seccomp=unconfined', '--rm', '-e', 'TARGET_LIST=x86_64-softmmu', '-e', 'EXTRA_CONFIGURE_OPTS=', '-e', 'V=', '-e', 'J=14', '-e', 'DEBUG=', '-e', 'SHOW_ENV=', '-e', 'CCACHE_DIR=/var/tmp/ccache', '-v', '/home/patchew2/.cache/qemu-docker-ccache:/var/tmp/ccache:z', '-v', '/var/tmp/patchew-tester-tmp-s0ok4i_z/src/docker-src.2020-06-05-17.15.46.2022:/var/tmp/qemu:z,ro', 'qemu:fedora', '/var/tmp/qemu/run', 'test-debug']' returned non-zero exit status 2.
filter=--filter=label=com.qemu.instance.uuid=41c29776cd2c40bf8d43e98f3001772d
make[1]: *** [docker-run] Error 1
make[1]: Leaving directory `/var/tmp/patchew-tester-tmp-s0ok4i_z/src'
make: *** [docker-run-test-debug@fedora] Error 2

real    28m54.118s
user    0m8.736s


The full log is available at
http://patchew.org/logs/20200605173422.1490-1-robert.foley@linaro.org/testing.asan/?type=message.
---
Email generated automatically by Patchew [https://patchew.org/].
Please send your feedback to patchew-devel@redhat.com
Re: [PATCH v2 00/13] Add Thread Sanitizer support to QEMU
Posted by Alex Bennée 3 years, 10 months ago
Robert Foley <robert.foley@linaro.org> writes:

> v1: https://lists.gnu.org/archive/html/qemu-devel/2020-05/msg08302.html
>
> Changes in v2:
> - Fixed make check under TSan.  With the below fixes, make check 
>   under TSan completes successfully, albeit with TSan warnings.
>   - We found that several unit tests and the qtests hit an issue in TSan,
>     which results in a hung test.  This is a known issue: 
>     https://github.com/google/sanitizers/issues/1116
>   - Under TSan, disable the 3 unit tests that hit this above issue.
>   - Under TSan, disable the qtests since they hit this issue too.
> - Split out the docker testing for tsan into its own test (test-tsan).
> - configure:  Error out if tsan and other sanitizers are used together.
> - configure: Cleaned up warnings during tsan build caused by tsan libraries.
<snip>

I've complete my pass. I think we are looking pretty good and once the
tracepoints and function pointer stuff is dealt with I think we are
ready to merge.

-- 
Alex Bennée