From nobody Sat May 18 04:30:04 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1684337379; cv=none; d=zohomail.com; s=zohoarc; b=SX7SKOJlCKIxNATQfAfVIi0H2XX26Lfs71imj4GoToW3U00PcvSKSV7xeatuKqQcIiK1FfBy0qNliCUAjPyoAbIvo2kRBtz0VdijYORFMRkAmNugBnyGTlV380uoRLW7FXl/Gj9osjk9d+T27JR1lEr7w2DDdOAimiA7Z51JGkM= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1684337379; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=307NFVhzkj0Ci6yzvP3eVZ7VIH69hx+a7JKtXTKt04s=; b=YcOQN8eCHXilIsHDgtoC3cF0P6VVzgDTeKQaFJGZ7Rpc40WlxRNhYhOnhc2/6HgL6VH9CG7sAi3qQYWdapIIqY3HDZyZw4PECyQN17I2WdzissNFZfM8RKR/sYcHcjUe9F51oRgfwBLaFbpijNZlsOVoC+uqXBUf2C7MODRHIcE= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1684337379984576.6458600655009; Wed, 17 May 2023 08:29:39 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pzJ5B-0002NZ-5u; Wed, 17 May 2023 11:28:53 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pzJ59-0002Lo-1w for qemu-devel@nongnu.org; Wed, 17 May 2023 11:28:51 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pzJ57-0000tX-D3 for qemu-devel@nongnu.org; Wed, 17 May 2023 11:28:50 -0400 Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-367-gPtw047NO421oFKpKj5ZrA-1; Wed, 17 May 2023 11:28:45 -0400 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.rdu2.redhat.com [10.11.54.3]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id B1FF21C313B5; Wed, 17 May 2023 15:28:39 +0000 (UTC) Received: from merkur.fritz.box (unknown [10.39.194.19]) by smtp.corp.redhat.com (Postfix) with ESMTP id 33FC51121314; Wed, 17 May 2023 15:28:38 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1684337328; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=307NFVhzkj0Ci6yzvP3eVZ7VIH69hx+a7JKtXTKt04s=; b=EP+3nV/eR6MqgUhFhHMAJ2fANcv7uiYbXmO63JAWmAplGYZMSw4bZEdiEuS62BV/kQO577 iyR0qrlzZec2xAdDdQC9LKLWcU8nDfyn+b9zF5pglCSZNGsh+U3E3Kz9FXQv90bOVchSp7 jZL0h8fWro0OTR3AyboUsdjRQNl2tXI= X-MC-Unique: gPtw047NO421oFKpKj5ZrA-1 From: Kevin Wolf To: qemu-block@nongnu.org Cc: kwolf@redhat.com, hreitz@redhat.com, stefanha@redhat.com, mjt@tls.msk.ru, eblake@redhat.com, pbonzini@redhat.com, qemu-devel@nongnu.org, qemu-stable@nongnu.org Subject: [PATCH 1/3] graph-lock: Disable locking for now Date: Wed, 17 May 2023 17:28:32 +0200 Message-Id: <20230517152834.277483-2-kwolf@redhat.com> In-Reply-To: <20230517152834.277483-1-kwolf@redhat.com> References: <20230517152834.277483-1-kwolf@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.1 on 10.11.54.3 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=kwolf@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1684337381062100003 Content-Type: text/plain; charset="utf-8" In QEMU 8.0, we've been seeing deadlocks in bdrv_graph_wrlock(). They come from callers that hold an AioContext lock, which is not allowed during polling. In theory, we could temporarily release the lock, but callers are inconsistent about whether they hold a lock, and if they do, some are also confused about which one they hold. While all of this is fixable, it's not trivial, and the best course of action for 8.0.1 is probably just disabling the graph locking code temporarily. We don't currently rely on graph locking yet. It is supposed to replace the AioContext lock eventually to enable multiqueue support, but as long as we still have the AioContext lock, it is sufficient without the graph lock. Once the AioContext lock goes away, the deadlock doesn't exist any more either and this commit can be reverted. (Of course, it can also be reverted while the AioContext lock still exists if the callers have been fixed.) Cc: qemu-stable@nongnu.org Signed-off-by: Kevin Wolf Reviewed-by: Eric Blake --- block/graph-lock.c | 21 +++++++++++++++++++++ 1 file changed, 21 insertions(+) diff --git a/block/graph-lock.c b/block/graph-lock.c index 9c42bd9799..9806fd4ecb 100644 --- a/block/graph-lock.c +++ b/block/graph-lock.c @@ -30,8 +30,10 @@ BdrvGraphLock graph_lock; /* Protects the list of aiocontext and orphaned_reader_count */ static QemuMutex aio_context_list_lock; =20 +#if 0 /* Written and read with atomic operations. */ static int has_writer; +#endif =20 /* * A reader coroutine could move from an AioContext to another. @@ -88,6 +90,7 @@ void unregister_aiocontext(AioContext *ctx) g_free(ctx->bdrv_graph); } =20 +#if 0 static uint32_t reader_count(void) { BdrvGraphRWlock *brdv_graph; @@ -105,10 +108,17 @@ static uint32_t reader_count(void) assert((int32_t)rd >=3D 0); return rd; } +#endif =20 void bdrv_graph_wrlock(void) { GLOBAL_STATE_CODE(); + /* + * TODO Some callers hold an AioContext lock when this is called, which + * causes deadlocks. Reenable once the AioContext locking is cleaned u= p (or + * AioContext locks are gone). + */ +#if 0 assert(!qatomic_read(&has_writer)); =20 /* Make sure that constantly arriving new I/O doesn't cause starvation= */ @@ -139,11 +149,13 @@ void bdrv_graph_wrlock(void) } while (reader_count() >=3D 1); =20 bdrv_drain_all_end(); +#endif } =20 void bdrv_graph_wrunlock(void) { GLOBAL_STATE_CODE(); +#if 0 QEMU_LOCK_GUARD(&aio_context_list_lock); assert(qatomic_read(&has_writer)); =20 @@ -155,10 +167,13 @@ void bdrv_graph_wrunlock(void) =20 /* Wake up all coroutine that are waiting to read the graph */ qemu_co_enter_all(&reader_queue, &aio_context_list_lock); +#endif } =20 void coroutine_fn bdrv_graph_co_rdlock(void) { + /* TODO Reenable when wrlock is reenabled */ +#if 0 BdrvGraphRWlock *bdrv_graph; bdrv_graph =3D qemu_get_current_aio_context()->bdrv_graph; =20 @@ -218,10 +233,12 @@ void coroutine_fn bdrv_graph_co_rdlock(void) qemu_co_queue_wait(&reader_queue, &aio_context_list_lock); } } +#endif } =20 void coroutine_fn bdrv_graph_co_rdunlock(void) { +#if 0 BdrvGraphRWlock *bdrv_graph; bdrv_graph =3D qemu_get_current_aio_context()->bdrv_graph; =20 @@ -239,6 +256,7 @@ void coroutine_fn bdrv_graph_co_rdunlock(void) if (qatomic_read(&has_writer)) { aio_wait_kick(); } +#endif } =20 void bdrv_graph_rdlock_main_loop(void) @@ -264,5 +282,8 @@ void assert_bdrv_graph_readable(void) void assert_bdrv_graph_writable(void) { assert(qemu_in_main_thread()); + /* TODO Reenable when wrlock is reenabled */ +#if 0 assert(qatomic_read(&has_writer)); +#endif } --=20 2.40.1 From nobody Sat May 18 04:30:04 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1684337358; cv=none; d=zohomail.com; s=zohoarc; b=AhUixzz1ibzEtML23fL4a189x1pi+T0MDYXEPpSLCJd3mhR7/EEDNq8Xn9ZQ4Xakxmdq2XDPG7zHq9ccQgIbUw8Ta5JQj7db/6i2tZpgNNyEfpyQl8kw0wh1OG1b3xslUOJzbbyWiUAoda0b1snpi0rYK+tUrDZlmAhlO8byWRU= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1684337358; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=71BCgfGW3jfYqjQHSMUd2R17uTz6MCoIV8/vS9j2Si0=; b=CzOxz5YPEVsEghs7sRxvmgl3IVMulPayGxqzIse/7qMiec1QMev/SwRbQflXhkYLZhNRTVEc5Uvl+xHMPkvMCddhjiGSxt8EI861uTsc1Ht/EEVIB6qr6M6dgyPwKI7CqD1cECQk5WlGFVn659wavnbGnrAniOiZD11hGWD5nHc= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1684337358400348.3103571861109; Wed, 17 May 2023 08:29:18 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pzJ5C-0002OU-Tn; Wed, 17 May 2023 11:28:54 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pzJ59-0002Md-Sy for qemu-devel@nongnu.org; Wed, 17 May 2023 11:28:51 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pzJ57-0000tn-Rb for qemu-devel@nongnu.org; Wed, 17 May 2023 11:28:51 -0400 Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-259-j0MOZfpPNIaX0vjPHNxrcA-1; Wed, 17 May 2023 11:28:47 -0400 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.rdu2.redhat.com [10.11.54.3]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 43B458EFF1F; Wed, 17 May 2023 15:28:41 +0000 (UTC) Received: from merkur.fritz.box (unknown [10.39.194.19]) by smtp.corp.redhat.com (Postfix) with ESMTP id E92631121315; Wed, 17 May 2023 15:28:39 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1684337329; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=71BCgfGW3jfYqjQHSMUd2R17uTz6MCoIV8/vS9j2Si0=; b=KUWKR9AwdqXOE+EV+YmxjN5N38uQDjrHXuJyonRByNfgsAdJPtUINY8JSS68/MMLPm1v3m m4lzZ4ouwG19VVqRQD/ILfbwe/9AYsSg3/b8/Hc28XEklHp14G2xEI36DgDOK/SoTXL4+K K2hnPFrP0uVrniSvZ8841hUNeapWcKw= X-MC-Unique: j0MOZfpPNIaX0vjPHNxrcA-1 From: Kevin Wolf To: qemu-block@nongnu.org Cc: kwolf@redhat.com, hreitz@redhat.com, stefanha@redhat.com, mjt@tls.msk.ru, eblake@redhat.com, pbonzini@redhat.com, qemu-devel@nongnu.org, qemu-stable@nongnu.org Subject: [PATCH 2/3] nbd/server: Fix drained_poll to wake coroutine in right AioContext Date: Wed, 17 May 2023 17:28:33 +0200 Message-Id: <20230517152834.277483-3-kwolf@redhat.com> In-Reply-To: <20230517152834.277483-1-kwolf@redhat.com> References: <20230517152834.277483-1-kwolf@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.1 on 10.11.54.3 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=kwolf@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1684337360394100003 Content-Type: text/plain; charset="utf-8" nbd_drained_poll() generally runs in the main thread, not whatever iothread the NBD server coroutine is meant to run in, so it can't directly reenter the coroutines to wake them up. The code seems to have the right intention, it specifies the correct AioContext when it calls qemu_aio_coroutine_enter(). However, this functions doesn't schedule the coroutine to run in that AioContext, but it assumes it is already called in the home thread of the AioContext. To fix this, add a new thread-safe qio_channel_wake_read() that can be called in the main thread to wake up the coroutine in its AioContext, and use this in nbd_drained_poll(). Cc: qemu-stable@nongnu.org Signed-off-by: Kevin Wolf Reviewed-by: Eric Blake --- include/io/channel.h | 10 ++++++++++ io/channel.c | 33 +++++++++++++++++++++++++++------ nbd/server.c | 3 +-- 3 files changed, 38 insertions(+), 8 deletions(-) diff --git a/include/io/channel.h b/include/io/channel.h index 446a566e5e..229bf36910 100644 --- a/include/io/channel.h +++ b/include/io/channel.h @@ -757,6 +757,16 @@ void qio_channel_detach_aio_context(QIOChannel *ioc); void coroutine_fn qio_channel_yield(QIOChannel *ioc, GIOCondition condition); =20 +/** + * qio_channel_wake_read: + * @ioc: the channel object + * + * If qio_channel_yield() is currently waiting for the channel to become + * readable, interrupt it and reenter immediately. This function is safe t= o call + * from any thread. + */ +void qio_channel_wake_read(QIOChannel *ioc); + /** * qio_channel_wait: * @ioc: the channel object diff --git a/io/channel.c b/io/channel.c index 375a130a39..72f0066af5 100644 --- a/io/channel.c +++ b/io/channel.c @@ -19,6 +19,7 @@ */ =20 #include "qemu/osdep.h" +#include "block/aio-wait.h" #include "io/channel.h" #include "qapi/error.h" #include "qemu/main-loop.h" @@ -514,7 +515,11 @@ int qio_channel_flush(QIOChannel *ioc, static void qio_channel_restart_read(void *opaque) { QIOChannel *ioc =3D opaque; - Coroutine *co =3D ioc->read_coroutine; + Coroutine *co =3D qatomic_xchg(&ioc->read_coroutine, NULL); + + if (!co) { + return; + } =20 /* Assert that aio_co_wake() reenters the coroutine directly */ assert(qemu_get_current_aio_context() =3D=3D @@ -525,7 +530,11 @@ static void qio_channel_restart_read(void *opaque) static void qio_channel_restart_write(void *opaque) { QIOChannel *ioc =3D opaque; - Coroutine *co =3D ioc->write_coroutine; + Coroutine *co =3D qatomic_xchg(&ioc->write_coroutine, NULL); + + if (!co) { + return; + } =20 /* Assert that aio_co_wake() reenters the coroutine directly */ assert(qemu_get_current_aio_context() =3D=3D @@ -568,7 +577,11 @@ void qio_channel_detach_aio_context(QIOChannel *ioc) void coroutine_fn qio_channel_yield(QIOChannel *ioc, GIOCondition condition) { + AioContext *ioc_ctx =3D ioc->ctx ?: qemu_get_aio_context(); + assert(qemu_in_coroutine()); + assert(in_aio_context_home_thread(ioc_ctx)); + if (condition =3D=3D G_IO_IN) { assert(!ioc->read_coroutine); ioc->read_coroutine =3D qemu_coroutine_self(); @@ -580,18 +593,26 @@ void coroutine_fn qio_channel_yield(QIOChannel *ioc, } qio_channel_set_aio_fd_handlers(ioc); qemu_coroutine_yield(); + assert(in_aio_context_home_thread(ioc_ctx)); =20 /* Allow interrupting the operation by reentering the coroutine other = than * through the aio_fd_handlers. */ - if (condition =3D=3D G_IO_IN && ioc->read_coroutine) { - ioc->read_coroutine =3D NULL; + if (condition =3D=3D G_IO_IN) { + assert(ioc->read_coroutine =3D=3D NULL); qio_channel_set_aio_fd_handlers(ioc); - } else if (condition =3D=3D G_IO_OUT && ioc->write_coroutine) { - ioc->write_coroutine =3D NULL; + } else if (condition =3D=3D G_IO_OUT) { + assert(ioc->write_coroutine =3D=3D NULL); qio_channel_set_aio_fd_handlers(ioc); } } =20 +void qio_channel_wake_read(QIOChannel *ioc) +{ + Coroutine *co =3D qatomic_xchg(&ioc->read_coroutine, NULL); + if (co) { + aio_co_wake(co); + } +} =20 static gboolean qio_channel_wait_complete(QIOChannel *ioc, GIOCondition condition, diff --git a/nbd/server.c b/nbd/server.c index e239c2890f..2664d43bff 100644 --- a/nbd/server.c +++ b/nbd/server.c @@ -1599,8 +1599,7 @@ static bool nbd_drained_poll(void *opaque) * enter it here so we don't depend on the client to wake it u= p. */ if (client->recv_coroutine !=3D NULL && client->read_yielding)= { - qemu_aio_coroutine_enter(exp->common.ctx, - client->recv_coroutine); + qio_channel_wake_read(client->ioc); } =20 return true; --=20 2.40.1 From nobody Sat May 18 04:30:04 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1684337400; cv=none; d=zohomail.com; s=zohoarc; b=G5iEN+mN73x9jtEi6nJQVqN5YISO5LWVCXElLBDPgxfgqnmOPJo1GBSSIcoychz5Kj7uB7ae+LzN2a2G6Bn3tfJ5rzACBS1fNzuivxlafVUj+vIEDxxo3TKi7vdIYHFUG5Y4JYw5K7DkggC1mPW/6dr2ZFUoXyES7qCKCagY5M4= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1684337400; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=lFLcuMrJ88uqg8RrGhan+Qhyt2kDwpqYecyh3DxualQ=; b=lDthlnbkf9qyHwAn/YoNHuqztZk7k9VV6+5XJ3t7p5OX3C0IgOfHkt3gMXoD/v0eDX3VLheqbAXjGGpQJEzA9HLFdfHHrzbw6uBfeyzExyTId848UhuWquMTB1K3o1/y73nn1z5UlHKwkkT6J2TCDFuL8ilLdUTtxnKPTa4N890= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1684337400718387.28365523314005; Wed, 17 May 2023 08:30:00 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pzJ5D-0002QA-I6; Wed, 17 May 2023 11:28:55 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pzJ5C-0002O6-4U for qemu-devel@nongnu.org; Wed, 17 May 2023 11:28:54 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pzJ5A-0000ug-Kq for qemu-devel@nongnu.org; Wed, 17 May 2023 11:28:53 -0400 Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-39-RvarhIJpN2qEgRF8CoXPAw-1; Wed, 17 May 2023 11:28:50 -0400 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.rdu2.redhat.com [10.11.54.3]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id C35AA87DC1E; Wed, 17 May 2023 15:28:42 +0000 (UTC) Received: from merkur.fritz.box (unknown [10.39.194.19]) by smtp.corp.redhat.com (Postfix) with ESMTP id 7BBF81121314; Wed, 17 May 2023 15:28:41 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1684337331; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=lFLcuMrJ88uqg8RrGhan+Qhyt2kDwpqYecyh3DxualQ=; b=XGZZyklmTbseTMlS1WpKICpWtmG8sFIvhCU3GW6JZI+U49Ej1BAt9duXEQIVRWB0V8iJ6S Ny0xm7yqcXmQ7j0g++SoOUBpp0+3gyDteYrYzI5XqV/S7GKKWJsgTg9GLC3ZX82ijjNYL4 vwhXR0rAdwfnkOpwC2ALZNCmBJ/3Kyw= X-MC-Unique: RvarhIJpN2qEgRF8CoXPAw-1 From: Kevin Wolf To: qemu-block@nongnu.org Cc: kwolf@redhat.com, hreitz@redhat.com, stefanha@redhat.com, mjt@tls.msk.ru, eblake@redhat.com, pbonzini@redhat.com, qemu-devel@nongnu.org, qemu-stable@nongnu.org Subject: [PATCH 3/3] iotests: Test commit with iothreads and ongoing I/O Date: Wed, 17 May 2023 17:28:34 +0200 Message-Id: <20230517152834.277483-4-kwolf@redhat.com> In-Reply-To: <20230517152834.277483-1-kwolf@redhat.com> References: <20230517152834.277483-1-kwolf@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.1 on 10.11.54.3 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=kwolf@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1684337402738100003 Content-Type: text/plain; charset="utf-8" This tests exercises graph locking, draining, and graph modifications with AioContext switches a lot. Amongst others, it serves as a regression test for bdrv_graph_wrlock() deadlocking because it is called with a locked AioContext and for AioContext handling in the NBD server. Signed-off-by: Kevin Wolf Reviewed-by: Eric Blake Tested-by: Eric Blake --- tests/qemu-iotests/iotests.py | 4 ++ .../qemu-iotests/tests/graph-changes-while-io | 56 +++++++++++++++++-- .../tests/graph-changes-while-io.out | 4 +- 3 files changed, 58 insertions(+), 6 deletions(-) diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py index 3e82c634cf..7073579a7d 100644 --- a/tests/qemu-iotests/iotests.py +++ b/tests/qemu-iotests/iotests.py @@ -462,6 +462,10 @@ def qmp(self, cmd: str, args: Optional[Dict[str, objec= t]] =3D None) \ assert self._qmp is not None return self._qmp.cmd(cmd, args) =20 + def get_qmp(self) -> QEMUMonitorProtocol: + assert self._qmp is not None + return self._qmp + def stop(self, kill_signal=3D15): self._p.send_signal(kill_signal) self._p.wait() diff --git a/tests/qemu-iotests/tests/graph-changes-while-io b/tests/qemu-i= otests/tests/graph-changes-while-io index 7664f33689..750e7d4d38 100755 --- a/tests/qemu-iotests/tests/graph-changes-while-io +++ b/tests/qemu-iotests/tests/graph-changes-while-io @@ -22,19 +22,19 @@ import os from threading import Thread import iotests -from iotests import imgfmt, qemu_img, qemu_img_create, QMPTestCase, \ - QemuStorageDaemon +from iotests import imgfmt, qemu_img, qemu_img_create, qemu_io, \ + QMPTestCase, QemuStorageDaemon =20 =20 top =3D os.path.join(iotests.test_dir, 'top.img') nbd_sock =3D os.path.join(iotests.sock_dir, 'nbd.sock') =20 =20 -def do_qemu_img_bench() -> None: +def do_qemu_img_bench(count: int =3D 2000000) -> None: """ Do some I/O requests on `nbd_sock`. """ - qemu_img('bench', '-f', 'raw', '-c', '2000000', + qemu_img('bench', '-f', 'raw', '-c', str(count), f'nbd+unix:///node0?socket=3D{nbd_sock}') =20 =20 @@ -84,6 +84,54 @@ class TestGraphChangesWhileIO(QMPTestCase): =20 bench_thr.join() =20 + def test_commit_while_io(self) -> None: + # Run qemu-img bench in the background + bench_thr =3D Thread(target=3Ddo_qemu_img_bench, args=3D(200000, )) + bench_thr.start() + + qemu_io('-c', 'write 0 64k', top) + qemu_io('-c', 'write 128k 64k', top) + + result =3D self.qsd.qmp('blockdev-add', { + 'driver': imgfmt, + 'node-name': 'overlay', + 'backing': None, + 'file': { + 'driver': 'file', + 'filename': top + } + }) + self.assert_qmp(result, 'return', {}) + + result =3D self.qsd.qmp('blockdev-snapshot', { + 'node': 'node0', + 'overlay': 'overlay', + }) + self.assert_qmp(result, 'return', {}) + + # While qemu-img bench is running, repeatedly commit overlay to no= de0 + while bench_thr.is_alive(): + result =3D self.qsd.qmp('block-commit', { + 'job-id': 'job0', + 'device': 'overlay', + }) + self.assert_qmp(result, 'return', {}) + + result =3D self.qsd.qmp('block-job-cancel', { + 'device': 'job0', + }) + self.assert_qmp(result, 'return', {}) + + cancelled =3D False + while not cancelled: + for event in self.qsd.get_qmp().get_events(wait=3D10.0): + if event['event'] !=3D 'JOB_STATUS_CHANGE': + continue + if event['data']['status'] =3D=3D 'null': + cancelled =3D True + + bench_thr.join() + if __name__ =3D=3D '__main__': # Format must support raw backing files iotests.main(supported_fmts=3D['qcow', 'qcow2', 'qed'], diff --git a/tests/qemu-iotests/tests/graph-changes-while-io.out b/tests/qe= mu-iotests/tests/graph-changes-while-io.out index ae1213e6f8..fbc63e62f8 100644 --- a/tests/qemu-iotests/tests/graph-changes-while-io.out +++ b/tests/qemu-iotests/tests/graph-changes-while-io.out @@ -1,5 +1,5 @@ -. +.. ---------------------------------------------------------------------- -Ran 1 tests +Ran 2 tests =20 OK --=20 2.40.1