From nobody Mon Feb 9 07:05:40 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=linaro.org ARC-Seal: i=1; a=rsa-sha256; t=1591733461; cv=none; d=zohomail.com; s=zohoarc; b=RJYS6uAaVjiM4DVGwreZEEUAvLcCvQCSXp/PpMzSYS0wfAWWYVBWW0LEVZy7XkPGPXOCJl/7eJeG1KbQYfjWj5qvOkTL0IbaecjnFULVM0RkM2Tdyjk6JgYHVZ/Dsd1Fy+60b8o2YUIRLlXHVLagFG9w/3z1QLFAPtR9cw/1Spg= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1591733461; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=PEbujPtBaJg3B2IT0hPnYXWImSAP6JwyxZ2Cqb86MEY=; b=MohNNgm7cf6AIf08Fan+2C7G04REQSB2+hII+5dSdqY1dpNGarNFKTnZQ6lZ0TuUMNtYj88U9avvhDDkpEtRRsKF2o5v/Yd0A+GaQNfopm2/TXfuaRjwG5DAA32gHx34Q8elb9x1AO+V4MIYVw4/P+Otkd+Ig1rsCigZt53VgAE= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1591733461621897.5262068736521; Tue, 9 Jun 2020 13:11:01 -0700 (PDT) Received: from localhost ([::1]:60218 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jikaK-0000Wf-5D for importer@patchew.org; Tue, 09 Jun 2020 16:11:00 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:59268) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jikZ1-0007NR-UN for qemu-devel@nongnu.org; Tue, 09 Jun 2020 16:09:39 -0400 Received: from mail-pl1-x643.google.com ([2607:f8b0:4864:20::643]:33148) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jikZ0-0005Yy-AX for qemu-devel@nongnu.org; Tue, 09 Jun 2020 16:09:39 -0400 Received: by mail-pl1-x643.google.com with SMTP id t7so25209plr.0 for ; Tue, 09 Jun 2020 13:09:37 -0700 (PDT) Received: from Rfoley-MA01.hsd1.ma.comcast.net ([2601:199:4480:60c0:1d09:cef2:3b1f:abce]) by smtp.gmail.com with ESMTPSA id n19sm10523374pfu.194.2020.06.09.13.09.34 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 09 Jun 2020 13:09:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=PEbujPtBaJg3B2IT0hPnYXWImSAP6JwyxZ2Cqb86MEY=; b=SvN7s+MjJJ7ADNt5weFq9NU9aiPFWeG9ft6r1PtTQHd+3Gl7ccE4YXSi6/TY6Gxcac D20wS7o7f1naPiFINjoWEKkNr4pVdtSQa0S/aYdAGEVSSl9rpTeE1WFFUnHAVq2WrZBA PFgxacabSIBZ7svHkx90sm50Kia5ly7lZiWHk5d5sXp5ZuP/0FZBdtFlGPqRXLHlwtg9 YP6STyvxaK1o2xW3JeBTkMD+genH0b9J5KssTSRw+TXuJUWuWXYeBiNVKIgUtc9GR0Nh s/0r0LlMZ8y3LVrOLt8Y/0HC7M1TT6DkYv+99MOU9xMp2+ykz47o0OKzLClwjOWwGRje ekNA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=PEbujPtBaJg3B2IT0hPnYXWImSAP6JwyxZ2Cqb86MEY=; b=ouAi0af6BUydq+FGArZCkyP2GK1Pv0V9Z8egfKN/jP0nl+gSHBCYHbz4c7C48KcY4D 9PbwFcAgG05trcVm9rgpX+tGffxerAtAepHS6YMFTBWOtoT3LMg/4y7+4vgjZZ7vvPfE YsWlvAJvfB4+TbdT1Sfx+lcqJChCt9CHKLLRF+npOjmc7xBQLArvYHTyK7CjcWNRQF3z Nr5j5ytqiMyCyqbEjMDM65YuhhDQamKMQEEE9vnvCZ5syzuKF0EzIEyx3rHFEfO2Xi/I L+KuQqgvxIrirp8YDzPmxYt5ppnhU4ictRtuphFo2/84e88SoJSZUy+6sJBPyzfM7D3Z pkXQ== X-Gm-Message-State: AOAM532o3oE9D+Uwqxl4ff9Ixf4ReALGfJ2uestRRlnzu2I/Abvc5g7p t++v28RsF3zafOikxRWysS+UfC1h4hv1Wg== X-Google-Smtp-Source: ABdhPJxxJgmIuAK7Dsrr+C1xb6yzvhfArxJ3Rkx7JmEmqecu1FybFjMEzmK8Zqq482a4R9iRL9os5w== X-Received: by 2002:a17:90a:3608:: with SMTP id s8mr6294130pjb.167.1591733376257; Tue, 09 Jun 2020 13:09:36 -0700 (PDT) From: Robert Foley To: qemu-devel@nongnu.org Subject: [PATCH v3 01/13] configure: add --enable-tsan flag + fiber annotations for coroutine-ucontext Date: Tue, 9 Jun 2020 16:07:26 -0400 Message-Id: <20200609200738.445-2-robert.foley@linaro.org> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20200609200738.445-1-robert.foley@linaro.org> References: <20200609200738.445-1-robert.foley@linaro.org> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2607:f8b0:4864:20::643; envelope-from=robert.foley@linaro.org; helo=mail-pl1-x643.google.com X-detected-operating-system: by eggs.gnu.org: No matching host in p0f cache. That's all we know. X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , robert.foley@linaro.org, alex.bennee@linaro.org, cota@braap.org, Stefan Hajnoczi , Lingfeng Yang , peter.puhov@linaro.org, =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) From: Lingfeng Yang We tried running QEMU under tsan in 2016, but tsan's lack of support for longjmp-based fibers was a blocker: https://groups.google.com/forum/#!topic/thread-sanitizer/se0YuzfWazw Fortunately, thread sanitizer gained fiber support in early 2019: https://reviews.llvm.org/D54889 This patch brings tsan support upstream by importing the patch that annotat= ed QEMU's coroutines as tsan fibers in Android's QEMU fork: https://android-review.googlesource.com/c/platform/external/qemu/+/844675 Tested with '--enable-tsan --cc=3Dclang-9 --cxx=3Dclang++-9 --disable-werro= r' configure flags. Signed-off-by: Lingfeng Yang Signed-off-by: Emilio G. Cota [cota: minor modifications + configure changes] Signed-off-by: Robert Foley [RF: configure changes, coroutine fix + minor modifications] Reviewed-by: Alex Benn=C3=A9e --- configure | 47 +++++++++++++++++++++++++++- util/coroutine-ucontext.c | 66 +++++++++++++++++++++++++++++++++------ 2 files changed, 103 insertions(+), 10 deletions(-) diff --git a/configure b/configure index 597e909b53..da46f9556e 100755 --- a/configure +++ b/configure @@ -395,6 +395,7 @@ gprof=3D"no" debug_tcg=3D"no" debug=3D"no" sanitizers=3D"no" +tsan=3D"no" fortify_source=3D"" strip_opt=3D"yes" tcg_interpreter=3D"no" @@ -1150,6 +1151,10 @@ for opt do ;; --disable-sanitizers) sanitizers=3D"no" ;; + --enable-tsan) tsan=3D"yes" + ;; + --disable-tsan) tsan=3D"no" + ;; --enable-sparse) sparse=3D"yes" ;; --disable-sparse) sparse=3D"no" @@ -1754,6 +1759,7 @@ Advanced options (experts only): --with-pkgversion=3DVERS use specified string as sub-version of the pa= ckage --enable-debug enable common debug build options --enable-sanitizers enable default sanitizers + --enable-tsan enable thread sanitizer --disable-strip disable stripping binaries --disable-werror disable compilation abort on warning --disable-stack-protector disable compiler-provided stack protection @@ -6196,6 +6202,30 @@ if test "$fuzzing" =3D "yes" ; then fi fi =20 +# Thread sanitizer is, for now, much noisier than the other sanitizers; +# keep it separate until that is not the case. +if test "$tsan" =3D "yes" && test "$sanitizers" =3D "yes"; then + error_exit "TSAN is not supported with other sanitiziers." +fi +have_tsan=3Dno +have_tsan_iface_fiber=3Dno +if test "$tsan" =3D "yes" ; then + write_c_skeleton + if compile_prog "$CPU_CFLAGS -Werror -fsanitize=3Dthread" "" ; then + have_tsan=3Dyes + fi + cat > $TMPC << EOF +#include +int main(void) { + __tsan_create_fiber(0); + return 0; +} +EOF + if compile_prog "$CPU_CFLAGS -Werror -fsanitize=3Dthread" "" ; then + have_tsan_iface_fiber=3Dyes + fi +fi + ########################################## # check for libpmem =20 @@ -6297,6 +6327,16 @@ if test "$have_asan" =3D "yes"; then "Without code annotation, the report may be inferior." fi fi +if test "$have_tsan" =3D "yes" ; then + if test "$have_tsan_iface_fiber" =3D "yes" ; then + QEMU_CFLAGS=3D"-fsanitize=3Dthread $QEMU_CFLAGS" + QEMU_LDFLAGS=3D"-fsanitize=3Dthread $QEMU_LDFLAGS" + else + error_exit "Cannot enable TSAN due to missing fiber annotation interfa= ce." + fi +elif test "$tsan" =3D "yes" ; then + error_exit "Cannot enable TSAN due to missing sanitize thread interface." +fi if test "$have_ubsan" =3D "yes"; then QEMU_CFLAGS=3D"-fsanitize=3Dundefined $QEMU_CFLAGS" QEMU_LDFLAGS=3D"-fsanitize=3Dundefined $QEMU_LDFLAGS" @@ -6332,7 +6372,8 @@ if test "$werror" =3D "yes"; then QEMU_CFLAGS=3D"-Werror $QEMU_CFLAGS" fi =20 -if test "$solaris" =3D "no" ; then +# Exclude --warn-common with TSan to suppress warnings from the TSan libra= ries. +if test "$solaris" =3D "no" && test "$tsan" =3D "no"; then if $ld --version 2>/dev/null | grep "GNU ld" >/dev/null 2>/dev/null ; = then QEMU_LDFLAGS=3D"-Wl,--warn-common $QEMU_LDFLAGS" fi @@ -7386,6 +7427,10 @@ if test "$have_asan_iface_fiber" =3D "yes" ; then echo "CONFIG_ASAN_IFACE_FIBER=3Dy" >> $config_host_mak fi =20 +if test "$have_tsan" =3D "yes" && test "$have_tsan_iface_fiber" =3D "yes" = ; then + echo "CONFIG_TSAN=3Dy" >> $config_host_mak +fi + if test "$has_environ" =3D "yes" ; then echo "CONFIG_HAS_ENVIRON=3Dy" >> $config_host_mak fi diff --git a/util/coroutine-ucontext.c b/util/coroutine-ucontext.c index bd593e61bc..613f4c118e 100644 --- a/util/coroutine-ucontext.c +++ b/util/coroutine-ucontext.c @@ -37,12 +37,19 @@ #endif #endif =20 +#ifdef CONFIG_TSAN +#include +#endif + typedef struct { Coroutine base; void *stack; size_t stack_size; sigjmp_buf env; =20 + void *tsan_co_fiber; + void *tsan_caller_fiber; + #ifdef CONFIG_VALGRIND_H unsigned int valgrind_stack_id; #endif @@ -65,7 +72,18 @@ union cc_arg { int i[2]; }; =20 -static void finish_switch_fiber(void *fake_stack_save) +/* QEMU_ALWAYS_INLINE only does so if __OPTIMIZE__, so we cannot use it. */ +static inline __attribute__((always_inline)) +void on_new_fiber(CoroutineUContext *co) +{ +#ifdef CONFIG_TSAN + co->tsan_co_fiber =3D __tsan_create_fiber(0); /* flags: sync on switch= */ + co->tsan_caller_fiber =3D __tsan_get_current_fiber(); +#endif +} + +static inline __attribute__((always_inline)) +void finish_switch_fiber(void *fake_stack_save) { #ifdef CONFIG_ASAN const void *bottom_old; @@ -78,13 +96,30 @@ static void finish_switch_fiber(void *fake_stack_save) leader.stack_size =3D size_old; } #endif +#ifdef CONFIG_TSAN + if (fake_stack_save) { + __tsan_release(fake_stack_save); + __tsan_switch_to_fiber(fake_stack_save, 0); /* 0=3Dsynchronize */ + } +#endif } =20 -static void start_switch_fiber(void **fake_stack_save, - const void *bottom, size_t size) +static inline __attribute__((always_inline)) void start_switch_fiber( + CoroutineAction action, void **fake_stack_save, + const void *bottom, size_t size, void *new_fiber) { #ifdef CONFIG_ASAN - __sanitizer_start_switch_fiber(fake_stack_save, bottom, size); + __sanitizer_start_switch_fiber( + action =3D=3D COROUTINE_TERMINATE ? NULL : fake_stack_save, + bottom, size); +#endif +#ifdef CONFIG_TSAN + void *curr_fiber =3D + __tsan_get_current_fiber(); + __tsan_acquire(curr_fiber); + + *fake_stack_save =3D curr_fiber; + __tsan_switch_to_fiber(new_fiber, 0); /* 0=3Dsynchronize */ #endif } =20 @@ -104,8 +139,12 @@ static void coroutine_trampoline(int i0, int i1) =20 /* Initialize longjmp environment and switch back the caller */ if (!sigsetjmp(self->env, 0)) { - start_switch_fiber(&fake_stack_save, - leader.stack, leader.stack_size); + start_switch_fiber( + COROUTINE_YIELD, + &fake_stack_save, + leader.stack, + leader.stack_size, + self->tsan_caller_fiber); siglongjmp(*(sigjmp_buf *)co->entry_arg, 1); } =20 @@ -154,12 +193,16 @@ Coroutine *qemu_coroutine_new(void) =20 arg.p =3D co; =20 + on_new_fiber(co); makecontext(&uc, (void (*)(void))coroutine_trampoline, 2, arg.i[0], arg.i[1]); =20 /* swapcontext() in, siglongjmp() back out */ if (!sigsetjmp(old_env, 0)) { - start_switch_fiber(&fake_stack_save, co->stack, co->stack_size); + start_switch_fiber( + COROUTINE_YIELD, + &fake_stack_save, + co->stack, co->stack_size, co->tsan_co_fiber); swapcontext(&old_uc, &uc); } =20 @@ -216,8 +259,8 @@ qemu_coroutine_switch(Coroutine *from_, Coroutine *to_, =20 ret =3D sigsetjmp(from->env, 0); if (ret =3D=3D 0) { - start_switch_fiber(action =3D=3D COROUTINE_TERMINATE ? - NULL : &fake_stack_save, to->stack, to->stack_s= ize); + start_switch_fiber(action, &fake_stack_save, + to->stack, to->stack_size, to->tsan_co_fiber); siglongjmp(to->env, action); } =20 @@ -231,6 +274,11 @@ Coroutine *qemu_coroutine_self(void) if (!current) { current =3D &leader.base; } +#ifdef CONFIG_TSAN + if (!leader.tsan_co_fiber) { + leader.tsan_co_fiber =3D __tsan_get_current_fiber(); + } +#endif return current; } =20 --=20 2.17.1