From nobody Sun May 19 06:23:14 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1618430757; cv=none; d=zohomail.com; s=zohoarc; b=OUtgkQF12bKgWqjEnTJvMxuJUPZHkXndS+EEbEqNqdNHzoS015+HpgHuKc8XiWHPkiRdvlGXBNbq9ZdNo6gOuBtEIz1YlV62ATHKtRqdC+xIaZ9t2FSDRLzMKSqAawxJy7RC42mPze7rdLV/6ysNWcvnkNlcqBy8Q/QHfIndmyU= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1618430757; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=IxJ2sMfFbnfSo+zmQdD9DcxRFKmm0Qbf35IG+BCrSrI=; b=l0ZtUu3lZViwX9mR2kL7dPGN7CNLuOzpGeIpHh0iD8916t7kxPPXNl/suEMegJV0EoC6SxQANgQ9ge+pLL5hWLEAzSsedUcI0omXgNkVmX5crSh4aEIn5f42JelaHEo8YhelGECsctKtKvizPlVQg5u8N4mJ0BFTUNtaQB5QNu8= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1618430757110105.28348364552937; Wed, 14 Apr 2021 13:05:57 -0700 (PDT) Received: from localhost ([::1]:39852 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lWllp-0006hX-VU for importer@patchew.org; Wed, 14 Apr 2021 16:05:54 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:36310) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lWljH-000524-4Q for qemu-devel@nongnu.org; Wed, 14 Apr 2021 16:03:15 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:24555) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lWljE-0004EA-QV for qemu-devel@nongnu.org; Wed, 14 Apr 2021 16:03:14 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-567-YelpR6JMNW2wV7AwwUGlWw-1; Wed, 14 Apr 2021 16:03:00 -0400 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 10DDB1883521; Wed, 14 Apr 2021 20:02:59 +0000 (UTC) Received: from localhost (ovpn-114-209.ams2.redhat.com [10.36.114.209]) by smtp.corp.redhat.com (Postfix) with ESMTP id 63FD85D9DC; Wed, 14 Apr 2021 20:02:54 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1618430591; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=IxJ2sMfFbnfSo+zmQdD9DcxRFKmm0Qbf35IG+BCrSrI=; b=EtiPe1ssXtxBOi/bGsehXjXi+VD9P/JGEtbaJYaqpwq/3u3wU6AXfYs7AtS8Ww+pXWv8a+ 9LRBYB30Upmdylhh+V58v5AfdTZw9sE6KyKZDLkQtETbWz5gm00fHAeGZdCjk1M3C9ASGa dSs23tYJdMGGoP1DLlU9ji2y/9vxkHs= X-MC-Unique: YelpR6JMNW2wV7AwwUGlWw-1 From: Stefan Hajnoczi To: qemu-devel@nongnu.org Subject: [PATCH 1/2] util/async: add a human-readable name to BHs for debugging Date: Wed, 14 Apr 2021 21:02:46 +0100 Message-Id: <20210414200247.917496-2-stefanha@redhat.com> In-Reply-To: <20210414200247.917496-1-stefanha@redhat.com> References: <20210414200247.917496-1-stefanha@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=stefanha@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=216.205.24.124; envelope-from=stefanha@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -10 X-Spam_score: -1.1 X-Spam_bar: - X-Spam_report: (-1.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, MIME_BASE64_TEXT=1.741, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Fam Zheng , Kevin Wolf , qemu-block@nongnu.org, eric.g.ernst@gmail.com, Stefan Hajnoczi , Paolo Bonzini , Max Reitz Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) Content-Type: text/plain; charset="utf-8" It can be difficult to debug issues with BHs in production environments. Although BHs can usually be identified by looking up their ->cb() function pointer, this requires debug information for the program. It is also not possible to print human-readable diagnostics about BHs because they have no identifier. This patch adds a name to each BH. The name is not unique per instance but differentiates between cb() functions, which is usually enough. It's done by changing aio_bh_new() and friends to macros that stringify cb. The next patch will use the name field when reporting leaked BHs. Signed-off-by: Stefan Hajnoczi Reviewed-by: Fam Zheng Reviewed-by: Philippe Mathieu-Daud=C3=A9 --- include/block/aio.h | 31 ++++++++++++++++++++++++++++--- include/qemu/main-loop.h | 4 +++- tests/unit/ptimer-test-stubs.c | 2 +- util/async.c | 9 +++++++-- util/main-loop.c | 4 ++-- 5 files changed, 41 insertions(+), 9 deletions(-) diff --git a/include/block/aio.h b/include/block/aio.h index 5f342267d5..499668fef5 100644 --- a/include/block/aio.h +++ b/include/block/aio.h @@ -291,20 +291,45 @@ void aio_context_acquire(AioContext *ctx); /* Relinquish ownership of the AioContext. */ void aio_context_release(AioContext *ctx); =20 +/** + * aio_bh_schedule_oneshot_full: Allocate a new bottom half structure that= will + * run only once and as soon as possible. + * + * @name: A human-readable identifier for debugging purposes. + */ +void aio_bh_schedule_oneshot_full(AioContext *ctx, QEMUBHFunc *cb, void *o= paque, + const char *name); + /** * aio_bh_schedule_oneshot: Allocate a new bottom half structure that will= run * only once and as soon as possible. + * + * A convenience wrapper for aio_bh_schedule_oneshot_full() that uses cb a= s the + * name string. */ -void aio_bh_schedule_oneshot(AioContext *ctx, QEMUBHFunc *cb, void *opaque= ); +#define aio_bh_schedule_oneshot(ctx, cb, opaque) \ + aio_bh_schedule_oneshot_full((ctx), (cb), (opaque), (stringify(cb))) =20 /** - * aio_bh_new: Allocate a new bottom half structure. + * aio_bh_new_full: Allocate a new bottom half structure. * * Bottom halves are lightweight callbacks whose invocation is guaranteed * to be wait-free, thread-safe and signal-safe. The #QEMUBH structure * is opaque and must be allocated prior to its use. + * + * @name: A human-readable identifier for debugging purposes. */ -QEMUBH *aio_bh_new(AioContext *ctx, QEMUBHFunc *cb, void *opaque); +QEMUBH *aio_bh_new_full(AioContext *ctx, QEMUBHFunc *cb, void *opaque, + const char *name); + +/** + * aio_bh_new: Allocate a new bottom half structure + * + * A convenience wrapper for aio_bh_new_full() that uses the cb as the name + * string. + */ +#define aio_bh_new(ctx, cb, opaque) \ + aio_bh_new_full((ctx), (cb), (opaque), (stringify(cb))) =20 /** * aio_notify: Force processing of pending events. diff --git a/include/qemu/main-loop.h b/include/qemu/main-loop.h index d6892fd208..c7e8a21b5d 100644 --- a/include/qemu/main-loop.h +++ b/include/qemu/main-loop.h @@ -312,7 +312,9 @@ void qemu_cond_timedwait_iothread(QemuCond *cond, int m= s); =20 void qemu_fd_register(int fd); =20 -QEMUBH *qemu_bh_new(QEMUBHFunc *cb, void *opaque); +#define qemu_bh_new(cb, opaque) \ + qemu_bh_new_full((cb), (opaque), (stringify(cb))) +QEMUBH *qemu_bh_new_full(QEMUBHFunc *cb, void *opaque, const char *name); void qemu_bh_schedule_idle(QEMUBH *bh); =20 enum { diff --git a/tests/unit/ptimer-test-stubs.c b/tests/unit/ptimer-test-stubs.c index 7f801a4d09..2a3ef58799 100644 --- a/tests/unit/ptimer-test-stubs.c +++ b/tests/unit/ptimer-test-stubs.c @@ -108,7 +108,7 @@ int64_t qemu_clock_deadline_ns_all(QEMUClockType type, = int attr_mask) return deadline; } =20 -QEMUBH *qemu_bh_new(QEMUBHFunc *cb, void *opaque) +QEMUBH *qemu_bh_new_full(QEMUBHFunc *cb, void *opaque, const char *name) { QEMUBH *bh =3D g_new(QEMUBH, 1); =20 diff --git a/util/async.c b/util/async.c index 674dbefb7c..b6acb86520 100644 --- a/util/async.c +++ b/util/async.c @@ -57,6 +57,7 @@ enum { =20 struct QEMUBH { AioContext *ctx; + const char *name; QEMUBHFunc *cb; void *opaque; QSLIST_ENTRY(QEMUBH) next; @@ -107,7 +108,8 @@ static QEMUBH *aio_bh_dequeue(BHList *head, unsigned *f= lags) return bh; } =20 -void aio_bh_schedule_oneshot(AioContext *ctx, QEMUBHFunc *cb, void *opaque) +void aio_bh_schedule_oneshot_full(AioContext *ctx, QEMUBHFunc *cb, + void *opaque, const char *name) { QEMUBH *bh; bh =3D g_new(QEMUBH, 1); @@ -115,11 +117,13 @@ void aio_bh_schedule_oneshot(AioContext *ctx, QEMUBHF= unc *cb, void *opaque) .ctx =3D ctx, .cb =3D cb, .opaque =3D opaque, + .name =3D name, }; aio_bh_enqueue(bh, BH_SCHEDULED | BH_ONESHOT); } =20 -QEMUBH *aio_bh_new(AioContext *ctx, QEMUBHFunc *cb, void *opaque) +QEMUBH *aio_bh_new_full(AioContext *ctx, QEMUBHFunc *cb, void *opaque, + const char *name) { QEMUBH *bh; bh =3D g_new(QEMUBH, 1); @@ -127,6 +131,7 @@ QEMUBH *aio_bh_new(AioContext *ctx, QEMUBHFunc *cb, voi= d *opaque) .ctx =3D ctx, .cb =3D cb, .opaque =3D opaque, + .name =3D name, }; return bh; } diff --git a/util/main-loop.c b/util/main-loop.c index 5188ff6540..2c6a9a9d87 100644 --- a/util/main-loop.c +++ b/util/main-loop.c @@ -543,9 +543,9 @@ void main_loop_wait(int nonblocking) =20 /* Functions to operate on the main QEMU AioContext. */ =20 -QEMUBH *qemu_bh_new(QEMUBHFunc *cb, void *opaque) +QEMUBH *qemu_bh_new_full(QEMUBHFunc *cb, void *opaque, const char *name) { - return aio_bh_new(qemu_aio_context, cb, opaque); + return aio_bh_new_full(qemu_aio_context, cb, opaque, name); } =20 /* --=20 2.30.2 From nobody Sun May 19 06:23:14 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1618430756; cv=none; d=zohomail.com; s=zohoarc; b=T79brP0SfRqGAM1hBO+1LnMGB2ll38UDq0zST6PGk4QXW+v+9OEI9Tem69cM+yNaoySe5BlC1tm1ICI9EDEEwmrdRFLBkTclDDga+1Kp9arN67gXH+wsJsUFeAtCfbvgv645Y0HiLxnfKICgb3gtr1mrFIVORfEzVTtpuZidSKM= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1618430756; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=n5c7W/niRMPgoB9EdCgSEVSLkS+BrW9QxtyAsawKKQs=; b=PqCyV/st/tdkeFFcNnWXzpjVv1wNtOrYGoRt+3rXu0qsoAqHy+Z7KYTGChA4Xrn5Rs1kOlpyEQd+bNB5CDBNHs0HnnGhFRohHMs27mtfeLjk6gSRaQlSyKOa6X4pTQ948uJ5JGXNu8aSKKJpVtT9G4uDv9Nkp9kGsds09c0/xBY= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1618430756948758.4790868005066; Wed, 14 Apr 2021 13:05:56 -0700 (PDT) Received: from localhost ([::1]:39986 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lWllq-0006kk-Ta for importer@patchew.org; Wed, 14 Apr 2021 16:05:54 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:36342) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lWljI-00054J-EA for qemu-devel@nongnu.org; Wed, 14 Apr 2021 16:03:16 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:58967) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lWljF-0004En-FK for qemu-devel@nongnu.org; Wed, 14 Apr 2021 16:03:16 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-373-DzKLUTn1PS6OlDzdWZzggw-1; Wed, 14 Apr 2021 16:03:10 -0400 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id B32FD10CE791; Wed, 14 Apr 2021 20:03:04 +0000 (UTC) Received: from localhost (ovpn-114-209.ams2.redhat.com [10.36.114.209]) by smtp.corp.redhat.com (Postfix) with ESMTP id 7125C694CC; Wed, 14 Apr 2021 20:03:00 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1618430592; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=n5c7W/niRMPgoB9EdCgSEVSLkS+BrW9QxtyAsawKKQs=; b=DIR02D1MAJcDbWQ/Xbdf9CHKB9l0eDN1wsVvlJMyhu4hod3veGBWzFImGdFOERVUNps8Ty PzNz2HjFUus9l6TSl3jLLbJ2t0l2WtaeikqMAerjAetqtZZgaoN0CPZtxLDRBjlwktojRo XJWHKQ42GP/JFMa530sCrtUkIaMB1oM= X-MC-Unique: DzKLUTn1PS6OlDzdWZzggw-1 From: Stefan Hajnoczi To: qemu-devel@nongnu.org Subject: [PATCH 2/2] util/async: print leaked BH name when AioContext finalizes Date: Wed, 14 Apr 2021 21:02:47 +0100 Message-Id: <20210414200247.917496-3-stefanha@redhat.com> In-Reply-To: <20210414200247.917496-1-stefanha@redhat.com> References: <20210414200247.917496-1-stefanha@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=stefanha@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=216.205.24.124; envelope-from=stefanha@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -10 X-Spam_score: -1.1 X-Spam_bar: - X-Spam_report: (-1.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, MIME_BASE64_TEXT=1.741, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Fam Zheng , Kevin Wolf , qemu-block@nongnu.org, eric.g.ernst@gmail.com, Stefan Hajnoczi , Paolo Bonzini , Max Reitz Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) Content-Type: text/plain; charset="utf-8" BHs must be deleted before the AioContext is finalized. If not, it's a bug and probably indicates that some part of the program still expects the BH to run in the future. That can lead to memory leaks, inconsistent state, or just hangs. Unfortunately the assert(flags & BH_DELETED) call in aio_ctx_finalize() is difficult to debug because the assertion failure contains no information about the BH! Use the QEMUBH name field added in the previous patch to show a useful error when a leaked BH is detected. Suggested-by: Eric Ernst Signed-off-by: Stefan Hajnoczi Reviewed-by: Fam Zheng --- util/async.c | 16 ++++++++++++++-- 1 file changed, 14 insertions(+), 2 deletions(-) diff --git a/util/async.c b/util/async.c index b6acb86520..2584fca249 100644 --- a/util/async.c +++ b/util/async.c @@ -344,8 +344,20 @@ aio_ctx_finalize(GSource *source) assert(QSIMPLEQ_EMPTY(&ctx->bh_slice_list)); =20 while ((bh =3D aio_bh_dequeue(&ctx->bh_list, &flags))) { - /* qemu_bh_delete() must have been called on BHs in this AioContex= t */ - assert(flags & BH_DELETED); + /* + * qemu_bh_delete() must have been called on BHs in this AioContex= t. In + * many cases memory leaks, hangs, or inconsistent state occur whe= n a + * BH is leaked because something still expects it to run. + * + * If you hit this, fix the lifecycle of the BH so that + * qemu_bh_delete() and any associated cleanup is called before the + * AioContext is finalized. + */ + if (unlikely(!(flags & BH_DELETED))) { + fprintf(stderr, "%s: BH '%s' leaked, aborting...\n", + __func__, bh->name); + abort(); + } =20 g_free(bh); } --=20 2.30.2