From nobody Sun May 19 12:13:40 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org ARC-Seal: i=1; a=rsa-sha256; t=1673304802; cv=none; d=zohomail.com; s=zohoarc; b=OSPctPK9Xdtvjkzb0l2k4HLk90R0h/vlsOCE8a8dXZwcRc8nZenFc9Jp/suU30eJT4hyTTgBiB9Q2ATUDQfV++80vNzTz2sBnFVXmomYlPuSUBQodM00tHapV1tCuHCBVk7Wglo4rRf2W/3fMBvzh5/SO0mDOhZdkWZMpMjGFCo= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1673304802; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=QwzyW7oZ6debHOnGp+1MstVKgcAlJPD9/UuYM93GsSg=; b=hoYUVTEGiSzYc6MgeTZn2n6H6h8thOLh8pzzFQ1Q/zuMowgGShnHhls3WPTAdpTa0MfzydgqHG5gMMyXvXaCzllhB/Jt1u2ScBoc/E0YLySS1AZ0BQCd3BXlUD707lIxOs441VLoBlx55iez8RQbBJiYZQ8Z+6vtSvgzoWENsB8= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1673304802179226.25456368031246; Mon, 9 Jan 2023 14:53:22 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pF0yS-0005jc-Cp; Mon, 09 Jan 2023 17:50:36 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pF0yE-0005fM-1S for qemu-devel@nongnu.org; Mon, 09 Jan 2023 17:50:22 -0500 Received: from mail-qt1-x82c.google.com ([2607:f8b0:4864:20::82c]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1pF0yB-00076p-SJ for qemu-devel@nongnu.org; Mon, 09 Jan 2023 17:50:21 -0500 Received: by mail-qt1-x82c.google.com with SMTP id x11so9320421qtv.13 for ; Mon, 09 Jan 2023 14:50:19 -0800 (PST) Received: from localhost (pool-100-2-99-22.nycmny.fios.verizon.net. [100.2.99.22]) by smtp.gmail.com with ESMTPSA id l13-20020ac84a8d000000b003a6947863e1sm5167413qtq.11.2023.01.09.14.50.17 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 09 Jan 2023 14:50:17 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=braap-org.20210112.gappssmtp.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=QwzyW7oZ6debHOnGp+1MstVKgcAlJPD9/UuYM93GsSg=; b=hUOjKj8KWtBgSmqnMdYC0Uaxlakn+oVk73gNrdCvnXrv0ADUHzA8PryHjam9OHdWMP yiZ7BC63UFotEBJQMOpWkix6ZxW7wfMiPd70lS7xOAdn+Lwt9j6r3ueJ+BHB23wMovMY y+qTIq5+mX49CgXOQc7vaWS2v/OHSgg+ZpgWMwY/uDXXBe2xyRHq2qWklvukbvCmHgX4 CbphtHHfamWX/wslOlH7s2ys44Ifpd9RljBX4L3AsFIFCyj3y3ex1KbZapTrImlsM7iT cTQZgJOqgAU0tOQTB06YrDo5sBIPWQ0unE0p12bAHuHwaEKezlJ2JyrGvKAycWND3xIB MPEQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=QwzyW7oZ6debHOnGp+1MstVKgcAlJPD9/UuYM93GsSg=; b=5rE5IC8ZYMb20arwGJydMn2ID5RJ4MExq7BDoEIw2334OZSklChCLWOo7+qxQq86Et INPVywjOa8bZvxwsCO9E+hPLyR+qFJtj+f12U3M4Pw25FNC+fle8weL1jHnzvdLT8WfK KeHqc9INnEgsvg/I+qNtcWaK5WKYIb/YglyWeu7xSHkquY4GGlwgQ6Ev/HtcX24OF/J8 fkP4B9ST+Zqcy+nuPcv9WDGgGEN4CKeU0HhCe0bOsgpRZaTbdgyoeFWE6OHID529l8RY 2SZxQ8rKVdkD4/jDA4fGXpPuPQ41Gx70tihtZXCmP+rgkYLgtwd973F7VJT2Fj5ElgCb 7tXw== X-Gm-Message-State: AFqh2kp/nrYttGgGO311SjR3Y7bAY+ee9xYiadPXPoQ4WUlafW7Ge2cb VRSzpxaKocwHbGwC+wGw1lBLntbvGTc8u0tARxs= X-Google-Smtp-Source: AMrXdXtjaW/J6hCYkAV3luj54yWu+MKRGqSo1vdV8NUH7fn99Mlbj3hO8mHMVQTe1/R+Cdc4LJwUkQ== X-Received: by 2002:a05:622a:124e:b0:3ae:3f56:6d26 with SMTP id z14-20020a05622a124e00b003ae3f566d26mr3916166qtx.14.1673304618318; Mon, 09 Jan 2023 14:50:18 -0800 (PST) From: Emilio Cota To: qemu-devel@nongnu.org Cc: =?UTF-8?q?Alex=20Benn=C3=A9e?= , Paolo Bonzini , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , Richard Henderson , Emilio Cota Subject: [PATCH v2 1/5] cpu: free cpu->tb_jmp_cache with RCU Date: Mon, 9 Jan 2023 17:49:50 -0500 Message-Id: <20230109224954.161672-2-cota@braap.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230109224954.161672-1-cota@braap.org> References: <20230109224954.161672-1-cota@braap.org> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: softfail client-ip=2607:f8b0:4864:20::82c; envelope-from=cota@braap.org; helo=mail-qt1-x82c.google.com X-Spam_score_int: -11 X-Spam_score: -1.2 X-Spam_bar: - X-Spam_report: (-1.2 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_SOFTFAIL=0.665 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @braap-org.20210112.gappssmtp.com) X-ZM-MESSAGEID: 1673304803713100011 Content-Type: text/plain; charset="utf-8" Fixes the appended use-after-free. The root cause is that during tb invalidation we use CPU_FOREACH, and therefore to safely free a vCPU we must wait for an RCU grace period to elapse. $ x86_64-linux-user/qemu-x86_64 tests/tcg/x86_64-linux-user/munmap-pthread =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D =3D=3D1800604=3D=3DERROR: AddressSanitizer: heap-use-after-free on address = 0x62d0005f7418 at pc 0x5593da6704eb bp 0x7f4961a7ac70 sp 0x7f4961a7ac60 READ of size 8 at 0x62d0005f7418 thread T2 #0 0x5593da6704ea in tb_jmp_cache_inval_tb ../accel/tcg/tb-maint.c:244 #1 0x5593da6704ea in do_tb_phys_invalidate ../accel/tcg/tb-maint.c:290 #2 0x5593da670631 in tb_phys_invalidate__locked ../accel/tcg/tb-maint.c= :306 #3 0x5593da670631 in tb_invalidate_phys_page_range__locked ../accel/tcg= /tb-maint.c:542 #4 0x5593da67106d in tb_invalidate_phys_range ../accel/tcg/tb-maint.c:6= 14 #5 0x5593da6a64d4 in target_munmap ../linux-user/mmap.c:766 #6 0x5593da6dba05 in do_syscall1 ../linux-user/syscall.c:10105 #7 0x5593da6f564c in do_syscall ../linux-user/syscall.c:13329 #8 0x5593da49e80c in cpu_loop ../linux-user/x86_64/../i386/cpu_loop.c:2= 33 #9 0x5593da6be28c in clone_func ../linux-user/syscall.c:6633 #10 0x7f496231cb42 in start_thread nptl/pthread_create.c:442 #11 0x7f49623ae9ff (/lib/x86_64-linux-gnu/libc.so.6+0x1269ff) 0x62d0005f7418 is located 28696 bytes inside of 32768-byte region [0x62d000= 5f0400,0x62d0005f8400) freed by thread T148 here: #0 0x7f49627b6460 in __interceptor_free ../../../../src/libsanitizer/as= an/asan_malloc_linux.cpp:52 #1 0x5593da5ac057 in cpu_exec_unrealizefn ../cpu.c:180 #2 0x5593da81f851 (/home/cota/src/qemu/build/qemu-x86_64+0x484851) Signed-off-by: Emilio Cota --- accel/tcg/cpu-exec.c | 3 +-- accel/tcg/tb-jmp-cache.h | 1 + cpu.c | 11 ++++++++++- 3 files changed, 12 insertions(+), 3 deletions(-) diff --git a/accel/tcg/cpu-exec.c b/accel/tcg/cpu-exec.c index 356fe348de..6bd29227f3 100644 --- a/accel/tcg/cpu-exec.c +++ b/accel/tcg/cpu-exec.c @@ -1064,13 +1064,12 @@ void tcg_exec_realizefn(CPUState *cpu, Error **errp) /* undo the initializations in reverse order */ void tcg_exec_unrealizefn(CPUState *cpu) { - qemu_plugin_vcpu_exit_hook(cpu); #ifndef CONFIG_USER_ONLY tcg_iommu_free_notifier_list(cpu); #endif /* !CONFIG_USER_ONLY */ =20 tlb_destroy(cpu); - g_free(cpu->tb_jmp_cache); + g_free_rcu(cpu->tb_jmp_cache, rcu); } =20 #ifndef CONFIG_USER_ONLY diff --git a/accel/tcg/tb-jmp-cache.h b/accel/tcg/tb-jmp-cache.h index ff5ffc8fc2..b3f6e78835 100644 --- a/accel/tcg/tb-jmp-cache.h +++ b/accel/tcg/tb-jmp-cache.h @@ -18,6 +18,7 @@ * a load_acquire/store_release to 'tb'. */ struct CPUJumpCache { + struct rcu_head rcu; struct { TranslationBlock *tb; #if TARGET_TB_PCREL diff --git a/cpu.c b/cpu.c index 4a7d865427..21cf809614 100644 --- a/cpu.c +++ b/cpu.c @@ -176,11 +176,20 @@ void cpu_exec_unrealizefn(CPUState *cpu) vmstate_unregister(NULL, &vmstate_cpu_common, cpu); } #endif + + /* Call the plugin hook before clearing cpu->cpu_index in cpu_list_rem= ove */ if (tcg_enabled()) { - tcg_exec_unrealizefn(cpu); + qemu_plugin_vcpu_exit_hook(cpu); } =20 cpu_list_remove(cpu); + /* + * Now that the vCPU has been removed from the RCU list, we can call + * tcg_exec_unrealizefn, which may free fields using call_rcu. + */ + if (tcg_enabled()) { + tcg_exec_unrealizefn(cpu); + } } =20 /* --=20 2.34.1 From nobody Sun May 19 12:13:40 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org ARC-Seal: i=1; a=rsa-sha256; t=1673304800; cv=none; d=zohomail.com; s=zohoarc; b=naYBuDGutwza0gtxQlS1R7fMCPzoGA776U2OuUSMYBujJ3pS/hSEMQ24tFkNM+fJB0aVQWKphsBFbE405ZMBLvITGJhm9VmXE7fYaXHGE3lwByBigMPyRrhw+7jAg5CjOAeqgE48Up7b3KJkPzPBxKuHX3t9Xr6Q++m8DTM0bKE= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1673304800; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=3a1hVR1tfeFDYLy7M6VisZhyTtrl3sivRfIGgYuupak=; b=W2tLfWy+7e0DfjVp5LrIxqYTlHT04SssiPfkNAIock4ZGQ4X0i/iR/oI9obMsmj1RmM2tbvJoAcqZHUqpvRcPXcpaRaQALY1MKgZpBqHGKKYmRgIY/XhJGXYYS195jAJZI2DZW2lPw8tG4c3ZP9Ss8sk4m5ujhFQ46akBKXFEWI= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1673304800308830.1775124197515; Mon, 9 Jan 2023 14:53:20 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pF0yQ-0005jG-6o; Mon, 09 Jan 2023 17:50:34 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pF0yF-0005gJ-PS for qemu-devel@nongnu.org; Mon, 09 Jan 2023 17:50:24 -0500 Received: from mail-qt1-x835.google.com ([2607:f8b0:4864:20::835]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1pF0yE-00077a-DI for qemu-devel@nongnu.org; Mon, 09 Jan 2023 17:50:23 -0500 Received: by mail-qt1-x835.google.com with SMTP id c7so9396339qtw.8 for ; Mon, 09 Jan 2023 14:50:22 -0800 (PST) Received: from localhost (pool-100-2-99-22.nycmny.fios.verizon.net. [100.2.99.22]) by smtp.gmail.com with ESMTPSA id bs25-20020ac86f19000000b0039cb59f00fcsm5237182qtb.30.2023.01.09.14.50.20 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 09 Jan 2023 14:50:20 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=braap-org.20210112.gappssmtp.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=3a1hVR1tfeFDYLy7M6VisZhyTtrl3sivRfIGgYuupak=; b=M9GYJKWmf6W7sVLpYK4fc1KlCvVgw+xaFzt8J059ylUcS5P1hqyf0qes1StNU8RYEs dB/UJxkFw0mCl6hZDudgUKryz4jki7vjTjX3d7yhwsI16tIrHar75F3MO4ZgY7TEInGC LpLyYGgizMLfi6cw4o8sGY49+4RRjFaGhLK0oHedyEd6JyHMfRx4CsgtkASq7kgNQnxm Wb+4zafexMxJql/qhapNP2Lc+zqi9hP/Ga1wj8DZvvKqf+ZsGDcR9sfDxgn2VlUNATb1 e3TspFKw2zDi7a4sFaFT6b4cCU40LZ/gicQwc6UQIptGoLJru/F8+ctih3ET/rL3fd7U sXTQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=3a1hVR1tfeFDYLy7M6VisZhyTtrl3sivRfIGgYuupak=; b=Rpu8OjcM4GvNb8doXNLMlzhodlh9EN6c5RkoZmN/lX7/X7+rjo27R+OXm0nMHaNQ5X EJv1DnP7sVcai1Ew1Q3aqoBBL6ztM0GCcqjTh8YwPOTVgFt0r+WoUlKtUvi+SbVYUgo8 4vQoRI3CKdFSwpPdVb9Gbze+KdlUbS2RqzLn4HWND945mGO1r/0v3NT9hcmUbDcLQZt3 1OXFu5u3kQZelAArwuxKC1kiSFsdZsD5JkojNMGmVmjvuiGydXIV0Z91ZrL1TpeNNZ4w qTXsURAPuQKrtuRnUv923hBGHglA8ADvVGXPK4d1zjiI5n1kZZ/K3yBjz80bvKVRhfu2 jPBA== X-Gm-Message-State: AFqh2kqMQDJ8zIh/Wc/34XEF586saLyyrNAN6JakcXJZIn8tCHwwoaqH NcCLIhwfRXO5EWLkBuP1VRBNcvFVmBd9X4Rhiuw= X-Google-Smtp-Source: AMrXdXuWFa+BukYL+cmPTSJwYwNfKi94Nz0UQToi/pguMiFgroeQGvJNAvAssZ4Bsuc+oSfQd0Jbfg== X-Received: by 2002:ac8:58cc:0:b0:3ab:85d2:5c61 with SMTP id u12-20020ac858cc000000b003ab85d25c61mr79973192qta.61.1673304621246; Mon, 09 Jan 2023 14:50:21 -0800 (PST) From: Emilio Cota To: qemu-devel@nongnu.org Cc: =?UTF-8?q?Alex=20Benn=C3=A9e?= , Paolo Bonzini , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , Richard Henderson , Emilio Cota Subject: [PATCH v2 2/5] util/qht: add missing atomic_set(hashes[i]) Date: Mon, 9 Jan 2023 17:49:51 -0500 Message-Id: <20230109224954.161672-3-cota@braap.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230109224954.161672-1-cota@braap.org> References: <20230109224954.161672-1-cota@braap.org> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: softfail client-ip=2607:f8b0:4864:20::835; envelope-from=cota@braap.org; helo=mail-qt1-x835.google.com X-Spam_score_int: -11 X-Spam_score: -1.2 X-Spam_bar: - X-Spam_report: (-1.2 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_SOFTFAIL=0.665 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @braap-org.20210112.gappssmtp.com) X-ZM-MESSAGEID: 1673304801678100005 We forgot to add this one in "a890643958 util/qht: atomically set b->hashes= ". Detected with tsan. Reviewed-by: Richard Henderson Reviewed-by: Philippe Mathieu-Daud=C3=A9 Signed-off-by: Emilio Cota Reviewed-by: Alex Benn=C3=A9e --- util/qht.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/util/qht.c b/util/qht.c index 065fc501f4..15866299e6 100644 --- a/util/qht.c +++ b/util/qht.c @@ -688,7 +688,7 @@ static inline void qht_bucket_remove_entry(struct qht_b= ucket *orig, int pos) int i; =20 if (qht_entry_is_last(orig, pos)) { - orig->hashes[pos] =3D 0; + qatomic_set(&orig->hashes[pos], 0); qatomic_set(&orig->pointers[pos], NULL); return; } --=20 2.34.1 From nobody Sun May 19 12:13:40 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org ARC-Seal: i=1; a=rsa-sha256; t=1673304753; cv=none; d=zohomail.com; s=zohoarc; b=a0dT2EW6rQeexPj1e7IcmTBCndRnT3OnXLo51GNyqx+Re25XHyizVlZg8IVet4EoO10L5RS629r93db0e9RBLy1jBH663w8xk5MVZumf9144pg10wzO3YxZmFdbVxhDu3Pmx+VlgnzWXk8h1/UJbelEol+y6UfbyFWG/R63nqd8= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1673304753; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=EEeJP6EVgMNhmVau1DTLyaJxdvdOuqLyFWLUvLLVsSo=; b=NHUSeLyjPXdcJD/jKkJYDyrOe1BBSGx8ETFbRkOQfWJhppGb7WF2b/0Z/pddL1EZ1f3oFI6sDWjP94KI7eycb8WlPdJ60bPAtlr7pNUnjQsUrWK8dNvcuHA6W5d+C7bRIfHs06NOdCQKAIOb6g7DUdmH5/TEPMz3beLlXybwNYY= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1673304753298944.8655431236092; Mon, 9 Jan 2023 14:52:33 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pF0yk-0005oZ-LT; Mon, 09 Jan 2023 17:50:55 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pF0yI-0005hA-Aw for qemu-devel@nongnu.org; Mon, 09 Jan 2023 17:50:30 -0500 Received: from mail-qt1-x835.google.com ([2607:f8b0:4864:20::835]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1pF0yH-00078A-2T for qemu-devel@nongnu.org; Mon, 09 Jan 2023 17:50:26 -0500 Received: by mail-qt1-x835.google.com with SMTP id s5so5015829qtx.6 for ; Mon, 09 Jan 2023 14:50:24 -0800 (PST) Received: from localhost (pool-100-2-99-22.nycmny.fios.verizon.net. [100.2.99.22]) by smtp.gmail.com with ESMTPSA id cm8-20020a05622a250800b003a50ef44a77sm5190814qtb.28.2023.01.09.14.50.23 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 09 Jan 2023 14:50:23 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=braap-org.20210112.gappssmtp.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=EEeJP6EVgMNhmVau1DTLyaJxdvdOuqLyFWLUvLLVsSo=; b=RQCNhG6T109zfgAhGh/3LaIjrR67KAu4iPnZ9BCNNLoiKQU6HZYual0UC9XFq02hTC 0RrGyLso88U0tzhKw6t8jyeme/4mKNAwzJ3cC0/Pvw76GXivRjg3mpmRLQvjQqFIQzQx ZwRdeXHD/9efoYEpOXMdKd+aWgonoGr62yzFkPvtiKE1SL2fPfK0r/GdPKB8mb3wcgaZ aob1fLrYJPsgy9gmp99R28fGp4tGym0iamdLedxlum0Y+RQCNGIXo9lW/cz5gFeLdtFt 9jebAZoKQ9y8OFNe+R7HbKoIVRSqyCpl06wGT7Je44YGbo337unoqmsNka98J0j9xRmw b4Xw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=EEeJP6EVgMNhmVau1DTLyaJxdvdOuqLyFWLUvLLVsSo=; b=G2Ex/ZSOExHbA7kAfWm3KaspSQKZai/LmHsxQwo7TTqBKAONScROsLetMNjbxnx/PW agGHVDO1jWxliKeE56PrsTsUgmW0ub0MnfmkkQwNmD/BnF46RD7RoR61KPStEyqdp94w 7KczBVcwjedSFCrD3FqxWyxTnYMoYmdThKN76aISv6Qwn8WgHiMkEK+5jm3iRceF1iF3 ltUfQuxbn0OZ9CzTdZkW/nqfPv9mqeN2cULZ9F1CnRs40UrbS+upIRALTg+Xtn3aDPXB p4xS87JHWt1c0/rp3YeVbn0gRktErnamdEgQw6Pvfr1TcOily2qIO1BDfeFy4kmj7WAR s6EA== X-Gm-Message-State: AFqh2kreflokiXWE9DSbT2QUu+0XYWggF/SaofL4WvAd8CB65kHAB+YU eV5EatJcgFLnsJHme+f5PkOYwOypi6WRygq8lHk= X-Google-Smtp-Source: AMrXdXtnx15OOI9s1zvyu6fhkzBKXiuAER8vCXfdfnIEILlJGUpUWkdN73g1Bbzqw7iqNPcQspaaTg== X-Received: by 2002:a05:622a:5a0d:b0:3ab:cde3:3727 with SMTP id fy13-20020a05622a5a0d00b003abcde33727mr27390364qtb.58.1673304624051; Mon, 09 Jan 2023 14:50:24 -0800 (PST) From: Emilio Cota To: qemu-devel@nongnu.org Cc: =?UTF-8?q?Alex=20Benn=C3=A9e?= , Paolo Bonzini , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , Richard Henderson , Emilio Cota Subject: [PATCH v2 3/5] thread: de-const qemu_spin_destroy Date: Mon, 9 Jan 2023 17:49:52 -0500 Message-Id: <20230109224954.161672-4-cota@braap.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230109224954.161672-1-cota@braap.org> References: <20230109224954.161672-1-cota@braap.org> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: softfail client-ip=2607:f8b0:4864:20::835; envelope-from=cota@braap.org; helo=mail-qt1-x835.google.com X-Spam_score_int: -11 X-Spam_score: -1.2 X-Spam_bar: - X-Spam_report: (-1.2 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_SOFTFAIL=0.665 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @braap-org.20210112.gappssmtp.com) X-ZM-MESSAGEID: 1673304755402100003 Content-Type: text/plain; charset="utf-8" Signed-off-by: Emilio Cota Reviewed-by: Alex Benn=C3=A9e --- include/qemu/thread.h | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/include/qemu/thread.h b/include/qemu/thread.h index 7c6703bce3..7841084199 100644 --- a/include/qemu/thread.h +++ b/include/qemu/thread.h @@ -237,11 +237,10 @@ static inline void qemu_spin_init(QemuSpin *spin) #endif } =20 -/* const parameter because the only purpose here is the TSAN annotation */ -static inline void qemu_spin_destroy(const QemuSpin *spin) +static inline void qemu_spin_destroy(QemuSpin *spin) { #ifdef CONFIG_TSAN - __tsan_mutex_destroy((void *)spin, __tsan_mutex_not_static); + __tsan_mutex_destroy(spin, __tsan_mutex_not_static); #endif } =20 --=20 2.34.1 From nobody Sun May 19 12:13:40 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org ARC-Seal: i=1; a=rsa-sha256; t=1673304818; cv=none; d=zohomail.com; s=zohoarc; b=TUNtW097Fh5y4o3h7iUx+VESUhNec2Cuk/qLW21qFw9YLRkgIbWDlbUWZYF5n9by9aEyq6fBW/Ek+b9f4PmOqcgv6PDbq1X7FUMexUHxt3ST7jntxMlFkxzsbXd2b5wpCV0SOsLsyias/C7cuK5+42WMMcxgVaj6Ukze7xLkruc= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1673304818; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=c1T8P3aFdmYKkV9MiEvlapALeh2kUUtoqOYpyWRERpg=; b=M4cviLt6Q5ICkQ4p6mZfW2/sp7SGaC3qxosDzbeVQoWtsiI7V252Cq7LYfn++i3MEkK4s/CXeMWghyfxd/CFY8r+oBY9rqPbJGZfLIcBEwZ908GKGjpCLG7iWCdaJuuPoZOIK8asKeHcz2hHWszViw7Fp5kPa7p3kPipnP9s1DA= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1673304818006663.326402609795; Mon, 9 Jan 2023 14:53:38 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pF0ym-0005pA-40; Mon, 09 Jan 2023 17:50:56 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pF0yO-0005ig-PD for qemu-devel@nongnu.org; Mon, 09 Jan 2023 17:50:33 -0500 Received: from mail-qt1-x830.google.com ([2607:f8b0:4864:20::830]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1pF0yK-00078W-1l for qemu-devel@nongnu.org; Mon, 09 Jan 2023 17:50:32 -0500 Received: by mail-qt1-x830.google.com with SMTP id j15so3989824qtv.4 for ; Mon, 09 Jan 2023 14:50:27 -0800 (PST) Received: from localhost (pool-100-2-99-22.nycmny.fios.verizon.net. [100.2.99.22]) by smtp.gmail.com with ESMTPSA id bn39-20020a05620a2ae700b006fed2788751sm6067311qkb.76.2023.01.09.14.50.26 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 09 Jan 2023 14:50:26 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=braap-org.20210112.gappssmtp.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=c1T8P3aFdmYKkV9MiEvlapALeh2kUUtoqOYpyWRERpg=; b=C/mUe9X+UqbCW1ICEgI/JSEAptY88sU1qddCVVPlMgeaXV0I6+7Xb+TUXExaUSeDw6 WXXnHqvmtz/XRsqqHQP7w4RDA0s9u5hkxDc+z2X0NKwm6aCXXu5ZagqaMDj746zWqr6W qFLd4gQ8agonhelEbH1Idy67jWNz1fUzUaORS4riScytfwQvm6vl2NlR4hrlgIby+EzN KF0mrF6Eh0k8VWSfvri/bvFMFov2gaUcSdrvwRqzaQ2DXSRVxlNP1Mf5DAdZsu/+TNKH UBgfId1G1zxY7UCjplNS8ectZs1n/rxL65qqHqEoFLzgig8t8jT4+f7XADKq1N5XSypi CbDQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=c1T8P3aFdmYKkV9MiEvlapALeh2kUUtoqOYpyWRERpg=; b=EKn+PjexgdBA3sdYBupVXSIBW5jKtiwthOZ6P401aAF4gz6EIBUbos8+0DZepxrBPo JRTNN0YGnYWvZZBuGOQwR/4ffjl/n3qs7o0853rDXSLSLK5iOgelKitY7dUUbJHp7AQJ MGs2yXe++mlcvPczvM5EVbyyQdX/VJ5vgkoe8B7uooMC6foQMqcwIuAUTVTScptMIcsD ReWNrJfLMrAVajIQNMW/JyDpeV67ZoHlM9reXuRJRr7yVqj9rbUNIibSWDQpdSBiNbTh OsMrO7HjOboI8JOuDQu6TC4SOvC1u5qN6eSOhVgGf6wjHjho3DkbLTQVrK1eJ+Jfgytq aWQg== X-Gm-Message-State: AFqh2koQG7NhSN+HPHfZFIQ1P9TrXS899/PhbcEeZPc9zs8kO7nSzJjM yojBkxBK4jGAEpvAfc6dM1rMneEKSdhFRe9a3cY= X-Google-Smtp-Source: AMrXdXso2acOjUlvH/DgF1MQIRK+DuhlaL+IQ6KgFI4wbbEmkDWjukUTLaSzXIgyLRDtIIyI6h08zQ== X-Received: by 2002:ac8:75c7:0:b0:3a5:24ac:a175 with SMTP id z7-20020ac875c7000000b003a524aca175mr1037167qtq.56.1673304626609; Mon, 09 Jan 2023 14:50:26 -0800 (PST) From: Emilio Cota To: qemu-devel@nongnu.org Cc: =?UTF-8?q?Alex=20Benn=C3=A9e?= , Paolo Bonzini , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , Richard Henderson , Emilio Cota Subject: [PATCH v2 4/5] util/qht: use striped locks under TSAN Date: Mon, 9 Jan 2023 17:49:53 -0500 Message-Id: <20230109224954.161672-5-cota@braap.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230109224954.161672-1-cota@braap.org> References: <20230109224954.161672-1-cota@braap.org> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: softfail client-ip=2607:f8b0:4864:20::830; envelope-from=cota@braap.org; helo=mail-qt1-x830.google.com X-Spam_score_int: -11 X-Spam_score: -1.2 X-Spam_bar: - X-Spam_report: (-1.2 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_SOFTFAIL=0.665 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @braap-org.20210112.gappssmtp.com) X-ZM-MESSAGEID: 1673304819870100007 Content-Type: text/plain; charset="utf-8" Fixes this tsan crash, easy to reproduce with any large enough program: $ tests/unit/test-qht 1..2 ThreadSanitizer: CHECK failed: sanitizer_deadlock_detector.h:67 "((n_all_lo= cks_)) < (((sizeof(all_locks_with_contexts_)/sizeof((all_locks_with_context= s_)[0]))))" (0x40, 0x40) (tid=3D1821568) #0 __tsan::CheckUnwind() ../../../../src/libsanitizer/tsan/tsan_rtl.cpp= :353 (libtsan.so.2+0x90034) #1 __sanitizer::CheckFailed(char const*, int, char const*, unsigned lon= g long, unsigned long long) ../../../../src/libsanitizer/sanitizer_common/s= anitizer_termination.cpp:86 (libtsan.so.2+0xca555) #2 __sanitizer::DeadlockDetectorTLS<__sanitizer::TwoLevelBitVector<1ul,= __sanitizer::BasicBitVector > >::addLock(unsigned long, uns= igned long, unsigned int) ../../../../src/libsanitizer/sanitizer_common/san= itizer_deadlock_detector.h:67 (libtsan.so.2+0xb3616) #3 __sanitizer::DeadlockDetectorTLS<__sanitizer::TwoLevelBitVector<1ul,= __sanitizer::BasicBitVector > >::addLock(unsigned long, uns= igned long, unsigned int) ../../../../src/libsanitizer/sanitizer_common/san= itizer_deadlock_detector.h:59 (libtsan.so.2+0xb3616) #4 __sanitizer::DeadlockDetector<__sanitizer::TwoLevelBitVector<1ul, __= sanitizer::BasicBitVector > >::onLockAfter(__sanitizer::Dead= lockDetectorTLS<__sanitizer::TwoLevelBitVector<1ul, __sanitizer::BasicBitVe= ctor > >*, unsigned long, unsigned int) ../../../../src/libs= anitizer/sanitizer_common/sanitizer_deadlock_detector.h:216 (libtsan.so.2+0= xb3616) #5 __sanitizer::DD::MutexAfterLock(__sanitizer::DDCallback*, __sanitize= r::DDMutex*, bool, bool) ../../../../src/libsanitizer/sanitizer_common/sani= tizer_deadlock_detector1.cpp:169 (libtsan.so.2+0xb3616) #6 __tsan::MutexPostLock(__tsan::ThreadState*, unsigned long, unsigned = long, unsigned int, int) ../../../../src/libsanitizer/tsan/tsan_rtl_mutex.c= pp:200 (libtsan.so.2+0xa3382) #7 __tsan_mutex_post_lock ../../../../src/libsanitizer/tsan/tsan_interf= ace_ann.cpp:384 (libtsan.so.2+0x76bc3) #8 qemu_spin_lock /home/cota/src/qemu/include/qemu/thread.h:259 (test-q= ht+0x44a97) #9 qht_map_lock_buckets ../util/qht.c:253 (test-qht+0x44a97) #10 do_qht_iter ../util/qht.c:809 (test-qht+0x45f33) #11 qht_iter ../util/qht.c:821 (test-qht+0x45f33) #12 iter_check ../tests/unit/test-qht.c:121 (test-qht+0xe473) #13 qht_do_test ../tests/unit/test-qht.c:202 (test-qht+0xe473) #14 qht_test ../tests/unit/test-qht.c:240 (test-qht+0xe7c1) #15 test_default ../tests/unit/test-qht.c:246 (test-qht+0xe828) #16 (libglib-2.0.so.0+0x7daed) #17 (libglib-2.0.so.0+0x7d80a) #18 (libglib-2.0.so.0+0x7d80a) #19 g_test_run_suite (libglib-2.0.so.0+0x7dfe9) #20 g_test_run (libglib-2.0.so.0+0x7e055) #21 main ../tests/unit/test-qht.c:259 (test-qht+0xd2c6) #22 __libc_start_call_main ../sysdeps/nptl/libc_start_call_main.h:58 (l= ibc.so.6+0x29d8f) #23 __libc_start_main_impl ../csu/libc-start.c:392 (libc.so.6+0x29e3f) #24 _start (test-qht+0xdb44) Signed-off-by: Emilio Cota --- util/qht.c | 101 +++++++++++++++++++++++++++++++++++++++++++++-------- 1 file changed, 87 insertions(+), 14 deletions(-) diff --git a/util/qht.c b/util/qht.c index 15866299e6..70cc733d5d 100644 --- a/util/qht.c +++ b/util/qht.c @@ -151,6 +151,22 @@ struct qht_bucket { =20 QEMU_BUILD_BUG_ON(sizeof(struct qht_bucket) > QHT_BUCKET_ALIGN); =20 +/* + * Under TSAN, we use striped locks instead of one lock per bucket chain. + * This avoids crashing under TSAN, since TSAN aborts the program if more = than + * 64 locks are held (this is a hardcoded limit in TSAN). + * When resizing a QHT we grab all the buckets' locks, which can easily + * go over TSAN's limit. By using striped locks, we avoid this problem. + * + * Note: this number must be a power of two for easy index computation. + */ +#define QHT_TSAN_BUCKET_LOCKS_BITS 4 +#define QHT_TSAN_BUCKET_LOCKS (1 << QHT_TSAN_BUCKET_LOCKS_BITS) + +struct qht_tsan_lock { + QemuSpin lock; +} QEMU_ALIGNED(QHT_BUCKET_ALIGN); + /** * struct qht_map - structure to track an array of buckets * @rcu: used by RCU. Keep it as the top field in the struct to help valgr= ind @@ -160,6 +176,7 @@ QEMU_BUILD_BUG_ON(sizeof(struct qht_bucket) > QHT_BUCKE= T_ALIGN); * @n_added_buckets: number of added (i.e. "non-head") buckets * @n_added_buckets_threshold: threshold to trigger an upward resize once = the * number of added buckets surpasses it. + * @tsan_bucket_locks: Array of striped locks to be used only under TSAN. * * Buckets are tracked in what we call a "map", i.e. this structure. */ @@ -169,6 +186,9 @@ struct qht_map { size_t n_buckets; size_t n_added_buckets; size_t n_added_buckets_threshold; +#ifdef CONFIG_TSAN + struct qht_tsan_lock tsan_bucket_locks[QHT_TSAN_BUCKET_LOCKS]; +#endif }; =20 /* trigger a resize when n_added_buckets > n_buckets / div */ @@ -229,10 +249,62 @@ static inline size_t qht_elems_to_buckets(size_t n_el= ems) return pow2ceil(n_elems / QHT_BUCKET_ENTRIES); } =20 -static inline void qht_head_init(struct qht_bucket *b) +/* + * When using striped locks (i.e. under TSAN), we have to be careful not + * to operate on the same lock twice (e.g. when iterating through all buck= ets). + * We achieve this by operating only on each stripe's first matching lock. + */ +static inline void qht_do_if_first_in_stripe(const struct qht_map *map, + struct qht_bucket *b, + void (*func)(QemuSpin *spin)) +{ +#ifdef CONFIG_TSAN + unsigned long bucket_idx =3D b - map->buckets; + bool is_first_in_stripe =3D (bucket_idx >> QHT_TSAN_BUCKET_LOCKS_BITS)= =3D=3D 0; + if (is_first_in_stripe) { + unsigned long lock_idx =3D bucket_idx & (QHT_TSAN_BUCKET_LOCKS - 1= ); + func(&map->tsan_bucket_locks[lock_idx]); + } +#else + func(&b->lock); +#endif +} + +static inline void qht_bucket_lock_destroy(const struct qht_map *map, + struct qht_bucket *b) +{ + qht_do_if_first_in_stripe(map, b, qemu_spin_destroy); +} + +static inline void qht_bucket_lock_do(const struct qht_map *map, + struct qht_bucket *b, + void (*func)(QemuSpin *lock)) +{ +#ifdef CONFIG_TSAN + unsigned long bucket_idx =3D b - map->buckets; + unsigned long lock_idx =3D bucket_idx & (QHT_TSAN_BUCKET_LOCKS - 1); + func(&map->tsan_bucket_locks[lock_idx]); +#else + func(&b->lock); +#endif +} + +static inline void qht_bucket_lock(const struct qht_map *map, + struct qht_bucket *b) +{ + qht_bucket_lock_do(map, b, qemu_spin_lock); +} + +static inline void qht_bucket_unlock(const struct qht_map *map, + struct qht_bucket *b) +{ + qht_bucket_lock_do(map, b, qemu_spin_unlock); +} + +static inline void qht_head_init(struct qht_map *map, struct qht_bucket *b) { memset(b, 0, sizeof(*b)); - qemu_spin_init(&b->lock); + qht_do_if_first_in_stripe(map, b, qemu_spin_init); seqlock_init(&b->sequence); } =20 @@ -250,7 +322,7 @@ static void qht_map_lock_buckets(struct qht_map *map) for (i =3D 0; i < map->n_buckets; i++) { struct qht_bucket *b =3D &map->buckets[i]; =20 - qemu_spin_lock(&b->lock); + qht_do_if_first_in_stripe(map, b, qemu_spin_lock); } } =20 @@ -261,7 +333,7 @@ static void qht_map_unlock_buckets(struct qht_map *map) for (i =3D 0; i < map->n_buckets; i++) { struct qht_bucket *b =3D &map->buckets[i]; =20 - qemu_spin_unlock(&b->lock); + qht_do_if_first_in_stripe(map, b, qemu_spin_unlock); } } =20 @@ -308,7 +380,7 @@ void qht_map_lock_buckets__no_stale(struct qht *ht, str= uct qht_map **pmap) * Get a head bucket and lock it, making sure its parent map is not stale. * @pmap is filled with a pointer to the bucket's parent map. * - * Unlock with qemu_spin_unlock(&b->lock). + * Unlock with qht_bucket_unlock. * * Note: callers cannot have ht->lock held. */ @@ -322,18 +394,18 @@ struct qht_bucket *qht_bucket_lock__no_stale(struct q= ht *ht, uint32_t hash, map =3D qatomic_rcu_read(&ht->map); b =3D qht_map_to_bucket(map, hash); =20 - qemu_spin_lock(&b->lock); + qht_bucket_lock(map, b); if (likely(!qht_map_is_stale__locked(ht, map))) { *pmap =3D map; return b; } - qemu_spin_unlock(&b->lock); + qht_bucket_unlock(map, b); =20 /* we raced with a resize; acquire ht->lock to see the updated ht->map= */ qht_lock(ht); map =3D ht->map; b =3D qht_map_to_bucket(map, hash); - qemu_spin_lock(&b->lock); + qht_bucket_lock(map, b); qht_unlock(ht); *pmap =3D map; return b; @@ -345,12 +417,13 @@ static inline bool qht_map_needs_resize(const struct = qht_map *map) map->n_added_buckets_threshold; } =20 -static inline void qht_chain_destroy(const struct qht_bucket *head) +static inline void qht_chain_destroy(const struct qht_map *map, + struct qht_bucket *head) { struct qht_bucket *curr =3D head->next; struct qht_bucket *prev; =20 - qemu_spin_destroy(&head->lock); + qht_do_if_first_in_stripe(map, head, qemu_spin_destroy); while (curr) { prev =3D curr; curr =3D curr->next; @@ -364,7 +437,7 @@ static void qht_map_destroy(struct qht_map *map) size_t i; =20 for (i =3D 0; i < map->n_buckets; i++) { - qht_chain_destroy(&map->buckets[i]); + qht_chain_destroy(map, &map->buckets[i]); } qemu_vfree(map->buckets); g_free(map); @@ -390,7 +463,7 @@ static struct qht_map *qht_map_create(size_t n_buckets) map->buckets =3D qemu_memalign(QHT_BUCKET_ALIGN, sizeof(*map->buckets) * n_buckets); for (i =3D 0; i < n_buckets; i++) { - qht_head_init(&map->buckets[i]); + qht_head_init(map, &map->buckets[i]); } return map; } @@ -638,7 +711,7 @@ bool qht_insert(struct qht *ht, void *p, uint32_t hash,= void **existing) b =3D qht_bucket_lock__no_stale(ht, hash, &map); prev =3D qht_insert__locked(ht, map, b, p, hash, &needs_resize); qht_bucket_debug__locked(b); - qemu_spin_unlock(&b->lock); + qht_bucket_unlock(map, b); =20 if (unlikely(needs_resize) && ht->mode & QHT_MODE_AUTO_RESIZE) { qht_grow_maybe(ht); @@ -749,7 +822,7 @@ bool qht_remove(struct qht *ht, const void *p, uint32_t= hash) b =3D qht_bucket_lock__no_stale(ht, hash, &map); ret =3D qht_remove__locked(b, p, hash); qht_bucket_debug__locked(b); - qemu_spin_unlock(&b->lock); + qht_bucket_unlock(map, b); return ret; } =20 --=20 2.34.1 From nobody Sun May 19 12:13:40 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org ARC-Seal: i=1; a=rsa-sha256; t=1673304838; cv=none; d=zohomail.com; s=zohoarc; b=L80O3sDG7JzXKZk9egVeLGF0JFwAG5bx0fRx31A4CQWsqMJoU88t+N6cg5xdsy3CMHcaARzPPejx6lwbqjO7vkULttBaRq1CgOKFzEoMmQt/7DQdvZW4k9IHgbJC+IfsmJjyhGV8SoZ4J3b9Uy0DiWgts7LdvziYX1QkSoBoG9c= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1673304838; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=aC4zZ6XSnozWRPIZlr8juBKASRNoGs4qqu58FPIo/+c=; b=BQs4tZA1yAXUYojcRazpFdxyWZLgitxwcjsBHNLLGTR2auFPywlFr3jeoIG82OXCsgL4biMNAItIWalhDfI+zVTk5h1k6xHqIuu5rfwAH0dfL1qzpNlf6B4KAn+w3hWP6DznW6i5qd3nICVvbR7sJGvtMpy/+LrCfWoEqJE5dLs= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1673304838252207.33534382428195; Mon, 9 Jan 2023 14:53:58 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pF0yu-0005x2-Uw; Mon, 09 Jan 2023 17:51:04 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pF0yO-0005ih-PX for qemu-devel@nongnu.org; Mon, 09 Jan 2023 17:50:33 -0500 Received: from mail-qt1-x82c.google.com ([2607:f8b0:4864:20::82c]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1pF0yM-00078b-Lx for qemu-devel@nongnu.org; Mon, 09 Jan 2023 17:50:31 -0500 Received: by mail-qt1-x82c.google.com with SMTP id v14so9465383qtq.3 for ; Mon, 09 Jan 2023 14:50:29 -0800 (PST) Received: from localhost (pool-100-2-99-22.nycmny.fios.verizon.net. [100.2.99.22]) by smtp.gmail.com with ESMTPSA id jr49-20020a05622a803100b003ad373d04b6sm2176671qtb.59.2023.01.09.14.50.28 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 09 Jan 2023 14:50:28 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=braap-org.20210112.gappssmtp.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=aC4zZ6XSnozWRPIZlr8juBKASRNoGs4qqu58FPIo/+c=; b=Yi8Zj38b7clij8AiYrXOcB+aMeJkX3Y7tP51qb9LM6eH64vYWMGFaPQ+mo48t85Zcs DORVnBp/NaFHEn/Y0pF35+kDcb9qNwwh28oHW5y0YqhQnf16huYvlAMw7K2P32pFCcc7 6w1U0q48USTdXI+5RtyOyb7cnFgOrH5bupgXjaB9wIpE+Zizrf3keUBOErrOP/SK8Dho Yhpe2u2vXUhIwNxZ3h5teXwrqMmIA+FYV9WDmp5cv6bmXXNmsuwdYZfo763ZfGXeCVYj AxZfL/lCrF5DKWUCDUQbKl8w78fKCALUDogV0+zbX7lgUTZB64JX84TmikjZhV+rjHTg jF6Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=aC4zZ6XSnozWRPIZlr8juBKASRNoGs4qqu58FPIo/+c=; b=NSlnwKb5P8Bn9j0Crz7A/hOrP8FVadhTkEfkxQkrEb2kWBdHetuWl0dRCK+fpmOxDH 28/Ru73oQxQ9Ai+UZEmtMB1o0tUlqJRpToFyLpmP6GUeQEItiJ19+KF+51FBRV39FNmd pztO5MS0TZjE5wXMDhzTQv+xtRuKR+hlAiEHexbUIRxBW0JapKe3HSxuoJzyUtLe+bgq QjrHEdqPWlbEqQjVf97eCQygszA7Iwdt56HMqxrOQKkGgG3JTwQZ7gJtLTGjQNys0QvM g2LcB9YyHsRSHmgLWn1f7BKECAcsHROrlVqch2TgQ+EwQqqqP4fvbwk6R7rKtwNrBAhX sWdA== X-Gm-Message-State: AFqh2kpHs02Ki65BAJeuU6I+wgiw8pZb09Du/2DT6TdAfEwf61Ar5nhM HbSJAT/EmMnEMmbUctzNaeR2MRRngxYzBloiyIQ= X-Google-Smtp-Source: AMrXdXvQBToluAdF0E+dZT49g86+NKrrVpvdFXJeiNaNQhCkD4onljWdJ+IXZmN1bj6MD7Wp1zYA8g== X-Received: by 2002:a05:622a:4207:b0:3a8:322f:705f with SMTP id cp7-20020a05622a420700b003a8322f705fmr90025883qtb.56.1673304629129; Mon, 09 Jan 2023 14:50:29 -0800 (PST) From: Emilio Cota To: qemu-devel@nongnu.org Cc: =?UTF-8?q?Alex=20Benn=C3=A9e?= , Paolo Bonzini , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , Richard Henderson , Emilio Cota Subject: [PATCH v2 5/5] plugins: make qemu_plugin_user_exit's locking order consistent with fork_start's Date: Mon, 9 Jan 2023 17:49:54 -0500 Message-Id: <20230109224954.161672-6-cota@braap.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230109224954.161672-1-cota@braap.org> References: <20230109224954.161672-1-cota@braap.org> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: softfail client-ip=2607:f8b0:4864:20::82c; envelope-from=cota@braap.org; helo=mail-qt1-x82c.google.com X-Spam_score_int: -11 X-Spam_score: -1.2 X-Spam_bar: - X-Spam_report: (-1.2 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_SOFTFAIL=0.665 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @braap-org.20210112.gappssmtp.com) X-ZM-MESSAGEID: 1673304839914100007 Content-Type: text/plain; charset="utf-8" To fix potential deadlocks as reported by tsan. Reviewed-by: Richard Henderson Signed-off-by: Emilio Cota Reviewed-by: Philippe Mathieu-Daud=C3=A9 --- plugins/core.c | 16 +++++++++++----- 1 file changed, 11 insertions(+), 5 deletions(-) diff --git a/plugins/core.c b/plugins/core.c index ccb770a485..728bacef95 100644 --- a/plugins/core.c +++ b/plugins/core.c @@ -500,10 +500,17 @@ void qemu_plugin_user_exit(void) enum qemu_plugin_event ev; CPUState *cpu; =20 - QEMU_LOCK_GUARD(&plugin.lock); - + /* + * Locking order: we must acquire locks in an order that is consistent + * with the one in fork_start(). That is: + * - start_exclusive(), which acquires qemu_cpu_list_lock, + * must be called before acquiring plugin.lock. + * - tb_flush(), which acquires mmap_lock(), must be called + * while plugin.lock is not held. + */ start_exclusive(); =20 + qemu_rec_mutex_lock(&plugin.lock); /* un-register all callbacks except the final AT_EXIT one */ for (ev =3D 0; ev < QEMU_PLUGIN_EV_MAX; ev++) { if (ev !=3D QEMU_PLUGIN_EV_ATEXIT) { @@ -513,13 +520,12 @@ void qemu_plugin_user_exit(void) } } } - - tb_flush(current_cpu); - CPU_FOREACH(cpu) { qemu_plugin_disable_mem_helpers(cpu); } + qemu_rec_mutex_unlock(&plugin.lock); =20 + tb_flush(current_cpu); end_exclusive(); =20 /* now it's safe to handle the exit case */ --=20 2.34.1