From nobody Tue Feb 10 01:34:31 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1649959460039956.238962864421; Thu, 14 Apr 2022 11:04:20 -0700 (PDT) Received: from localhost ([::1]:49942 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nf3pL-0004y9-01 for importer@patchew.org; Thu, 14 Apr 2022 14:04:19 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:57924) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nf3jV-0004rU-Mz for qemu-devel@nongnu.org; Thu, 14 Apr 2022 13:58:21 -0400 Received: from mail-wr1-x431.google.com ([2a00:1450:4864:20::431]:42976) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1nf3jT-0005ui-UR for qemu-devel@nongnu.org; Thu, 14 Apr 2022 13:58:17 -0400 Received: by mail-wr1-x431.google.com with SMTP id r13so7946351wrr.9 for ; Thu, 14 Apr 2022 10:58:15 -0700 (PDT) Received: from avogadro.lan ([2001:b07:6468:f312:c8dd:75d4:99ab:290a]) by smtp.gmail.com with ESMTPSA id v13-20020adfe28d000000b0020375f27a5asm2451254wri.4.2022.04.14.10.58.12 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 14 Apr 2022 10:58:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=sender:from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=qPiVlhtFX1uAH/GGISul1eoCjcJOFjzCsMQY+NhZ6Bs=; b=QI9IbrVeGwnT7x3mfvfGYCO8ytk7fa4F1VUplWWuHaCdJ4/d7fuRrFh0Mj7oc6L41g /pTdA4p0Lut3zTc3yqsZK2VtuRkW/czbGoUrZvoPapr+3QA2j389ARLv4MGjchVQGzeK 3eUfVcMVfNiR+pG4+3S+N0ZKYik+ok8vVu49wi0n4aZ9GZXJnvOYcb/wbMXVEYC/uUqT pagjB5kuYiK0Hl7T85pa3iukzoOf3vuS+MBXmgsw70qQ6yzj2rjvMTo4j1dj/6rIUuCG nkukrmrCTfF6HeM4t81nIUaU30A7/vjp9L5sL8M2unJ7nPVE36D4vUN0OmLFW+YUAhW+ oxrQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:sender:from:to:cc:subject:date:message-id :in-reply-to:references:mime-version:content-transfer-encoding; bh=qPiVlhtFX1uAH/GGISul1eoCjcJOFjzCsMQY+NhZ6Bs=; b=iux2hRjqhTsLaHrljXrPRiY6jetxq+Ttur9gCBAqsLzqcCaaJ/C92qfkPU15NRqczK DxAIQGkv6yMHDJUFNcEYhZ5yjkfUIHijSZQcN2zZipInxEQ241tV/ztIS6+G7FNA9mR8 v64Q890TaTnSuclymt6Wz8jAf+08DWwzF7njFk/K5NXH+0225slXy9KP9UvT+QeZsJmi 4jGzhI32mlAByF/eHFsdhCZhKvwb8SFVpqp9JlSaaVFDmVZ88gRH7mGwLSkD3QMHePa3 xloajlBmqe6nYyiuf4HAK5sPWZsGP21IJrUvnaaavgymDO2E0RwzJxWMuXi8g8W6pK0F 0TFg== X-Gm-Message-State: AOAM5326x1LMzPV8DEgo12Gg9BhjChyHlaCfHes9ALW/WVuMPuvUsoVG zE6t6jsdSnxI5pUx2Uxpio49K37BCntzoQ== X-Google-Smtp-Source: ABdhPJyxxLVjNVtqxY9yclQby1ifvA91k07t0fRJ47LlyWvXBK2Wetehl6iVcAbqHb7b2qrPGlc/aw== X-Received: by 2002:a05:6000:18a2:b0:203:d2f5:28a0 with SMTP id b2-20020a05600018a200b00203d2f528a0mr3008909wri.355.1649959094515; Thu, 14 Apr 2022 10:58:14 -0700 (PDT) From: Paolo Bonzini To: qemu-devel@nongnu.org Subject: [PATCH v2 for-7.1 5/9] nbd: use a QemuMutex to synchronize yanking, reconnection and coroutines Date: Thu, 14 Apr 2022 19:57:52 +0200 Message-Id: <20220414175756.671165-6-pbonzini@redhat.com> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220414175756.671165-1-pbonzini@redhat.com> References: <20220414175756.671165-1-pbonzini@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2a00:1450:4864:20::431; envelope-from=paolo.bonzini@gmail.com; helo=mail-wr1-x431.google.com X-Spam_score_int: -14 X-Spam_score: -1.5 X-Spam_bar: - X-Spam_report: (-1.5 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FORGED_FROMDOMAIN=0.249, FREEMAIL_FROM=0.001, HEADER_FROM_DIFFERENT_DOMAINS=0.249, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: v.sementsov-og@mail.ru, eblake@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZM-MESSAGEID: 1649959460407100001 Content-Type: text/plain; charset="utf-8" The condition for waiting on the s->free_sema queue depends on both s->in_flight and s->state. The latter is currently using atomics, but this is quite dubious and probably wrong. Because s->state is written in the main thread too, for example by the yank callback, it cannot be protected by a CoMutex. Introduce a separate lock that can be used by nbd_co_send_request(); later on this lock will also be used for s->state. There will not be any contention on the lock unless there is a yank or reconnect, so this is not performance sensitive. Signed-off-by: Paolo Bonzini Reviewed-by: Eric Blake --- block/nbd.c | 46 +++++++++++++++++++++++++++------------------- 1 file changed, 27 insertions(+), 19 deletions(-) diff --git a/block/nbd.c b/block/nbd.c index 62dd338ef3..a2414566d1 100644 --- a/block/nbd.c +++ b/block/nbd.c @@ -71,17 +71,22 @@ typedef struct BDRVNBDState { QIOChannel *ioc; /* The current I/O channel */ NBDExportInfo info; =20 - CoMutex send_mutex; + /* + * Protects free_sema, in_flight, requests[].coroutine, + * reconnect_delay_timer. + */ + QemuMutex requests_lock; CoQueue free_sema; - - CoMutex receive_mutex; int in_flight; + NBDClientRequest requests[MAX_NBD_REQUESTS]; + QEMUTimer *reconnect_delay_timer; + + CoMutex send_mutex; + CoMutex receive_mutex; NBDClientState state; =20 - QEMUTimer *reconnect_delay_timer; QEMUTimer *open_timer; =20 - NBDClientRequest requests[MAX_NBD_REQUESTS]; NBDReply reply; BlockDriverState *bs; =20 @@ -350,7 +355,7 @@ int coroutine_fn nbd_co_do_establish_connection(BlockDr= iverState *bs, return 0; } =20 -/* called under s->send_mutex */ +/* Called with s->requests_lock taken. */ static coroutine_fn void nbd_reconnect_attempt(BDRVNBDState *s) { bool blocking =3D nbd_client_connecting_wait(s); @@ -382,9 +387,9 @@ static coroutine_fn void nbd_reconnect_attempt(BDRVNBDS= tate *s) s->ioc =3D NULL; } =20 - qemu_co_mutex_unlock(&s->send_mutex); + qemu_mutex_unlock(&s->requests_lock); nbd_co_do_establish_connection(s->bs, blocking, NULL); - qemu_co_mutex_lock(&s->send_mutex); + qemu_mutex_lock(&s->requests_lock); =20 /* * The reconnect attempt is done (maybe successfully, maybe not), so @@ -466,11 +471,10 @@ static int coroutine_fn nbd_co_send_request(BlockDriv= erState *bs, BDRVNBDState *s =3D (BDRVNBDState *)bs->opaque; int rc, i =3D -1; =20 - qemu_co_mutex_lock(&s->send_mutex); - + qemu_mutex_lock(&s->requests_lock); while (s->in_flight =3D=3D MAX_NBD_REQUESTS || (!nbd_client_connected(s) && s->in_flight > 0)) { - qemu_co_queue_wait(&s->free_sema, &s->send_mutex); + qemu_co_queue_wait(&s->free_sema, &s->requests_lock); } =20 s->in_flight++; @@ -491,13 +495,13 @@ static int coroutine_fn nbd_co_send_request(BlockDriv= erState *bs, } } =20 - g_assert(qemu_in_coroutine()); assert(i < MAX_NBD_REQUESTS); - s->requests[i].coroutine =3D qemu_coroutine_self(); s->requests[i].offset =3D request->from; s->requests[i].receiving =3D false; + qemu_mutex_unlock(&s->requests_lock); =20 + qemu_co_mutex_lock(&s->send_mutex); request->handle =3D INDEX_TO_HANDLE(s, i); =20 assert(s->ioc); @@ -517,17 +521,19 @@ static int coroutine_fn nbd_co_send_request(BlockDriv= erState *bs, } else { rc =3D nbd_send_request(s->ioc, request); } + qemu_co_mutex_unlock(&s->send_mutex); =20 -err: if (rc < 0) { + qemu_mutex_lock(&s->requests_lock); +err: nbd_channel_error(s, rc); if (i !=3D -1) { s->requests[i].coroutine =3D NULL; } s->in_flight--; qemu_co_queue_next(&s->free_sema); + qemu_mutex_unlock(&s->requests_lock); } - qemu_co_mutex_unlock(&s->send_mutex); return rc; } =20 @@ -1017,12 +1023,11 @@ static bool nbd_reply_chunk_iter_receive(BDRVNBDSta= te *s, return true; =20 break_loop: + qemu_mutex_lock(&s->requests_lock); s->requests[HANDLE_TO_INDEX(s, handle)].coroutine =3D NULL; - - qemu_co_mutex_lock(&s->send_mutex); s->in_flight--; qemu_co_queue_next(&s->free_sema); - qemu_co_mutex_unlock(&s->send_mutex); + qemu_mutex_unlock(&s->requests_lock); =20 return false; } @@ -1855,8 +1860,9 @@ static int nbd_open(BlockDriverState *bs, QDict *opti= ons, int flags, BDRVNBDState *s =3D (BDRVNBDState *)bs->opaque; =20 s->bs =3D bs; - qemu_co_mutex_init(&s->send_mutex); + qemu_mutex_init(&s->requests_lock); qemu_co_queue_init(&s->free_sema); + qemu_co_mutex_init(&s->send_mutex); qemu_co_mutex_init(&s->receive_mutex); =20 if (!yank_register_instance(BLOCKDEV_YANK_INSTANCE(bs->node_name), err= p)) { @@ -2044,9 +2050,11 @@ static void nbd_cancel_in_flight(BlockDriverState *b= s) =20 reconnect_delay_timer_del(s); =20 + qemu_mutex_lock(&s->requests_lock); if (s->state =3D=3D NBD_CLIENT_CONNECTING_WAIT) { s->state =3D NBD_CLIENT_CONNECTING_NOWAIT; } + qemu_mutex_unlock(&s->requests_lock); =20 nbd_co_establish_connection_cancel(s->conn); } --=20 2.35.1