From nobody Sun May 12 17:56:22 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1664885140; cv=none; d=zohomail.com; s=zohoarc; b=WX5eMWUpRm4aKHSGoF5RmoydJ2h6jbDct3oAHJtmvDDZB0x1kgBu6GtsphiKPDI1Oo47sVHrwe3q3i3SG1sNmqBh/aXke+E3yk5jtFekMliUnw30OoaXwt+MM5muOctzL1aU5zKYi03lmzvHIbSdTAryQi5AAgBIcCipuvFeVdc= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1664885140; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:Sender:Subject:To; bh=1OKdKa0BPyNZN/qO95A/qJRyBmoC+8HlBqHXCOjX1Zg=; b=NktelvywkSo6af0xYkXx3rpvcXhh+WvoT//0+VLHYEki5BAHvUhICWnqYc1m4toYQT8xd2DHsnHbkGxI5qdpitjMJZ6y9qdOc0cveYSuf04A9ypDaILUoXOtjBUOyt2GSsJizJOFgZHInqtyC5PykLKklZg7TfTqXVnm4xDhVeo= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1664885140796994.312083510923; Tue, 4 Oct 2022 05:05:40 -0700 (PDT) Received: from localhost ([::1]:50306 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ofgg7-0006FE-6a for importer@patchew.org; Tue, 04 Oct 2022 08:05:39 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:57132) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ofgbc-0004mJ-7f for qemu-devel@nongnu.org; Tue, 04 Oct 2022 08:01:00 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:20063) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ofgbY-0005Qe-7u for qemu-devel@nongnu.org; Tue, 04 Oct 2022 08:00:58 -0400 Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-114-qhmhW5U4PJCqNvUUoNpMuA-1; Tue, 04 Oct 2022 08:00:51 -0400 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.rdu2.redhat.com [10.11.54.2]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id E812285A583; Tue, 4 Oct 2022 12:00:50 +0000 (UTC) Received: from localhost.localdomain.com (unknown [10.33.36.5]) by smtp.corp.redhat.com (Postfix) with ESMTP id 2FF7E40C6EC2; Tue, 4 Oct 2022 12:00:49 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1664884854; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=1OKdKa0BPyNZN/qO95A/qJRyBmoC+8HlBqHXCOjX1Zg=; b=IAVhCrMTanycmqrwGyUQ0ybjMatpQ5FH6qt3lIJ3Ef/+tRQRP9tDNuYOnsXpXERRkbbxDJ +tLLqOUCljHeHv+KZ7gOkYyH7inVtBm8CjgmO3Je6BepSRiIuddrY9RYLAqCS7/loCMy7g RJMbv8Fjs+KZuEL7MIOOYbitWQMLiKE= X-MC-Unique: qhmhW5U4PJCqNvUUoNpMuA-1 From: =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= To: qemu-devel@nongnu.org Cc: =?UTF-8?q?Alex=20Benn=C3=A9e?= , Laurent Vivier , ncopa@alpinelinux.org, Kyle Evans , Warner Losh , Peter Maydell , =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= Subject: [PATCH] linux-user,bsd-user: re-exec with G_SLICE=always-malloc Date: Tue, 4 Oct 2022 13:00:47 +0100 Message-Id: <20221004120047.857591-1-berrange@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.1 on 10.11.54.2 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=berrange@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1664885142166100001 The g_slice custom allocator is not async signal safe with its mutexes. When a multithreaded program running in the qemu user emulator forks, it can end up deadlocking in the g_slice allocator Thread 1: #0 syscall () at ../sysdeps/unix/sysv/linux/x86_64/syscall.S:38 #1 0x00007f54e190c77c in g_mutex_lock_slowpath (mutex=3Dmutex@entry=3D0x7= f54e1dc7600 ) at ../glib/gthread-posix.c:1462 #2 0x00007f54e190d222 in g_mutex_lock (mutex=3Dmutex@entry=3D0x7f54e1dc76= 00 ) at ../glib/gthread-posix.c:1486 #3 0x00007f54e18e39f2 in magazine_cache_pop_magazine (countp=3D0x7f54280e= 6638, ix=3D2) at ../glib/gslice.c:769 #4 thread_memory_magazine1_reload (ix=3D2, tmem=3D0x7f54280e6600) at ../g= lib/gslice.c:845 #5 g_slice_alloc (mem_size=3Dmem_size@entry=3D40) at ../glib/gslice.c:1058 #6 0x00007f54e18f06fa in g_tree_node_new (value=3D0x7f54d4066540 , key=3D0x7f54d4066560 ) at ../glib= /gtree.c:517 #7 g_tree_insert_internal (tree=3D0x555556aed800, key=3D0x7f54d4066560 , value=3D0x7f54d4066540 , re= place=3D0) at ../glib/gtree.c:517 #8 0x00007f54e186b755 in tcg_tb_insert (tb=3D0x7f54d4066540 ) at ../tcg/tcg.c:534 #9 0x00007f54e1820545 in tb_gen_code (cpu=3D0x7f54980b4b60, pc=3D27490640= 7438, cs_base=3D0, flags=3D24832, cflags=3D-16252928) at ../accel/tcg/trans= late-all.c:2118 #10 0x00007f54e18034a5 in tb_find (cpu=3D0x7f54980b4b60, last_tb=3D0x7f54= d4066440 , tb_exit=3D0, cf_mask=3D524288) at ../acc= el/tcg/cpu-exec.c:462 #11 0x00007f54e1803bd9 in cpu_exec (cpu=3D0x7f54980b4b60) at ../accel/tcg= /cpu-exec.c:818 #12 0x00007f54e1735a4c in cpu_loop (env=3D0x7f54980bce40) at ../linux-use= r/riscv/cpu_loop.c:37 #13 0x00007f54e1844b22 in clone_func (arg=3D0x7f5402f3b080) at ../linux-u= ser/syscall.c:6422 #14 0x00007f54e191950a in start_thread (arg=3D) at pthread= _create.c:477 #15 0x00007f54e19a52a3 in clone () at ../sysdeps/unix/sysv/linux/x86_64/c= lone.S:95 The only known workaround for this problem is to disable the g_slice custom allocator, in favor of system malloc which is believed to be async signal safe on all platforms QEMU officially targets. g_slice uses a one-time initializer to check the G_SLICE env variable making it hard for QEMU to set the env before any GLib API call has triggered the initializer. Even attribute((constructor)) is not sufficient as QEMU has many constructors and there is no ordering guarantee between them. This patch attempts to workaround this by re-exec()ing the QEMU user emulators if the G_SLICE env variable is not already set. This means the env variable will be inherited down the process tree spawned from there onwards. There is a possibility this could have unexpected consequences, but this has to be balanced against the real known problem of QEMU user emulators randomly deadlocking. Fixes: https://gitlab.com/qemu-project/qemu/-/issues/285 Signed-off-by: Daniel P. Berrang=C3=A9 --- Can't say I especially like this but I'm out of other ideas for how to guarantee a solution. Users can't set env vars prior to launching QEMU user emulators when using binfmt. NB, I tested the linux-user impl and it stops the hangs in my testing. I've not even compiled tested the bsd-user impl, just blindly copied the linux-user code. bsd-user/main.c | 20 ++++++++++++++++++++ linux-user/main.c | 20 ++++++++++++++++++++ 2 files changed, 40 insertions(+) diff --git a/bsd-user/main.c b/bsd-user/main.c index 6f09180d65..c816ab80b3 100644 --- a/bsd-user/main.c +++ b/bsd-user/main.c @@ -284,6 +284,26 @@ int main(int argc, char **argv) envlist_t *envlist =3D NULL; char *argv0 =3D NULL; =20 + /* + * Historically the g_slice custom allocator was not async + * signal safe with its mutexes, causing deadlocks when a + * threaded program forks. Setting G_SLICE=3Dalways-malloc + * forces use of system malloc which is generally safe. + * + * https://gitlab.com/qemu-project/qemu/-/issues/285 + * + * Remove if we ever bump min GLib to a version that + * drops g_slice's custom allocator: + * + * https://gitlab.gnome.org/GNOME/glib/-/issues/1079 + */ + if (getenv("G_SLICE") =3D=3D NULL) { + setenv("G_SLICE", "always-malloc", 1); + execvp(argv[0], argv); + perror("execvep"); + abort(); + } + adjust_ssize(); =20 if (argc <=3D 1) { diff --git a/linux-user/main.c b/linux-user/main.c index 88fccfe261..62391b9d32 100644 --- a/linux-user/main.c +++ b/linux-user/main.c @@ -654,6 +654,26 @@ int main(int argc, char **argv, char **envp) unsigned long max_reserved_va; bool preserve_argv0; =20 + /* + * Historically the g_slice custom allocator was not async + * signal safe with its mutexes, causing deadlocks when a + * threaded program forks. Setting G_SLICE=3Dalways-malloc + * forces use of system malloc which is generally safe. + * + * https://gitlab.com/qemu-project/qemu/-/issues/285 + * + * Remove if we ever bump min GLib to a version that + * drops g_slice's custom allocator: + * + * https://gitlab.gnome.org/GNOME/glib/-/issues/1079 + */ + if (getenv("G_SLICE") =3D=3D NULL) { + setenv("G_SLICE", "always-malloc", 1); + execvp(argv[0], argv); + perror("execvep"); + abort(); + } + error_init(argv[0]); module_call_init(MODULE_INIT_TRACE); qemu_init_cpu_list(); --=20 2.37.3