From nobody Fri May 17 19:12:47 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=yandex-team.ru ARC-Seal: i=1; a=rsa-sha256; t=1615788860; cv=none; d=zohomail.com; s=zohoarc; b=ObiNMP2shAufGTfuFxaxh7JS8qEC8Vf8j36Y4hfKp8BsxkWaUBemz317JC8n0Jw/X3uFR5AB4HYo9tuYXlyEYPnO5ssHLwjKjFQOQ3E7elTUrWyuznoUIvXUSs9/CMNt/61EzPUDRoeJX3Nrl67n0pwQQx0a3WcYggg9kg38V2Q= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1615788860; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=mIy54WIgKT/b/2NM+qYbxSn6vKp+ZnYdfYoMcvdPhbs=; b=AWg2TMp5m/xxqL3qmb+SXP85DMl3q9IifB/LX6lUSvS8Q+fjzlD6ZZ+L9KZg6H9bxf71S3LSmS5BVOuAlUZ6vGS0I41u+vGHH1t9W4lFZEcKA/BhqujD778yVh2FFDtusofwWcujCz8BPz8OClTGlZ/aycHZwjE7BzxAOQ2hHaw= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1615788860254400.5628698673585; Sun, 14 Mar 2021 23:14:20 -0700 (PDT) Received: from localhost ([::1]:51784 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lLgUc-0001bI-Sy for importer@patchew.org; Mon, 15 Mar 2021 02:14:18 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:47060) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lLgMw-000155-0G; Mon, 15 Mar 2021 02:06:22 -0400 Received: from forwardcorp1p.mail.yandex.net ([2a02:6b8:0:1472:2741:0:8b6:217]:50852) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lLgMs-0004bv-C7; Mon, 15 Mar 2021 02:06:20 -0400 Received: from sas1-6b1512233ef6.qloud-c.yandex.net (sas1-6b1512233ef6.qloud-c.yandex.net [IPv6:2a02:6b8:c14:44af:0:640:6b15:1223]) by forwardcorp1p.mail.yandex.net (Yandex) with ESMTP id B932E2E14B4; Mon, 15 Mar 2021 09:06:13 +0300 (MSK) Received: from sas1-24e978739efd.qloud-c.yandex.net (sas1-24e978739efd.qloud-c.yandex.net [2a02:6b8:c14:3088:0:640:24e9:7873]) by sas1-6b1512233ef6.qloud-c.yandex.net (mxbackcorp/Yandex) with ESMTP id 0I2FtcBroh-6DCOrseb; Mon, 15 Mar 2021 09:06:13 +0300 Received: from dynamic-vpn.dhcp.yndx.net (dynamic-vpn.dhcp.yndx.net [2a02:6b8:b081:23::1:15]) by sas1-24e978739efd.qloud-c.yandex.net (smtpcorp/Yandex) with ESMTPSA id FoAWGMWkZt-6Dn0hUl0; Mon, 15 Mar 2021 09:06:13 +0300 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) (Client certificate not present) Precedence: bulk DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yandex-team.ru; s=default; t=1615788373; bh=mIy54WIgKT/b/2NM+qYbxSn6vKp+ZnYdfYoMcvdPhbs=; h=In-Reply-To:Message-Id:References:Date:Subject:To:From:Cc; b=LRdpfr4rj7wK64ZNW3d8T9n+4nztWlEfDrcmXaFO5BImbYLE6gYDyAEG4/tu/KILY 4cS6ojVhMn/Xi+rt95y806E6C+ig8yQzslbMiuzxb/FmatQ89iOvTvxkGE5A/PB7h6 ZDMTWZmxMAk2Ei1nIep7gEtI97VpBBbyUXXCMs2s= Authentication-Results: sas1-6b1512233ef6.qloud-c.yandex.net; dkim=pass header.i=@yandex-team.ru From: Roman Kagan To: qemu-devel@nongnu.org Subject: [PATCH 1/7] block/nbd: avoid touching freed connect_thread Date: Mon, 15 Mar 2021 09:06:05 +0300 Message-Id: <20210315060611.2989049-2-rvkagan@yandex-team.ru> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20210315060611.2989049-1-rvkagan@yandex-team.ru> References: <20210315060611.2989049-1-rvkagan@yandex-team.ru> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2a02:6b8:0:1472:2741:0:8b6:217; envelope-from=rvkagan@yandex-team.ru; helo=forwardcorp1p.mail.yandex.net X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Vladimir Sementsov-Ogievskiy , qemu-block@nongnu.org, Max Reitz , yc-core@yandex-team.ru Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) Content-Type: text/plain; charset="utf-8" When the NBD connection is being torn down, the connection thread gets canceled and "detached", meaning it is about to get freed. If this happens while the connection coroutine yielded waiting for the connection thread to complete, when it resumes it may access the invalidated connection thread data. To prevent this, revalidate the ->connect_thread pointer in nbd_co_establish_connection_cancel before using after the the yield. Signed-off-by: Roman Kagan Reviewed-by: Vladimir Sementsov-Ogievskiy --- block/nbd.c | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/block/nbd.c b/block/nbd.c index c26dc5a54f..447d176b76 100644 --- a/block/nbd.c +++ b/block/nbd.c @@ -486,6 +486,15 @@ nbd_co_establish_connection(BlockDriverState *bs, Erro= r **errp) s->wait_connect =3D true; qemu_coroutine_yield(); =20 + /* + * If nbd_co_establish_connection_cancel had a chance to run it may ha= ve + * invalidated ->connect_thread. + */ + thr =3D s->connect_thread; + if (!thr) { + return -ECONNABORTED; + } + qemu_mutex_lock(&thr->mutex); =20 switch (thr->state) { --=20 2.30.2 From nobody Fri May 17 19:12:47 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=yandex-team.ru ARC-Seal: i=1; a=rsa-sha256; t=1615788667; cv=none; d=zohomail.com; s=zohoarc; b=UHXz07VUSK8BWjf9o4hvHmGw0nTEvaZvmBCEc58+GNkSf5YeJf4vBBwCl4YI7Tf+1clW1Vw3OUyyJDIEhEx0jo+PbeXQMMO8tG1slrt14mVoxgy07Sdapk49Esiv7tZv1N+LDYmthbzzK/f0fXenQrqHLY/tyVZUtGm3CfN15f8= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1615788667; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=xl3BXy0Fo1E+ViyIcbQolDDdibk1njkqm+tk2h+xMyE=; b=N63IwOutLCjSgZCDjhkgoujIT/HUmOAQvMVIFcKBw3r1ed66XlCSni5ghCh7UQVZt31a5LGl4BnoKb3+EGjHVmXHZc4TuUvLgOaIQuCVwlbZcBxuCQTbhQ4VQjcPqD6EDGjUy+e7AEZKj7oFB+pT6kBYxYphYp7cQJkm7jNdOAs= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1615788667286163.59490272422863; Sun, 14 Mar 2021 23:11:07 -0700 (PDT) Received: from localhost ([::1]:43428 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lLgRW-0006XY-3U for importer@patchew.org; Mon, 15 Mar 2021 02:11:06 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:47122) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lLgMx-00017Q-Dt; Mon, 15 Mar 2021 02:06:23 -0400 Received: from forwardcorp1o.mail.yandex.net ([95.108.205.193]:47560) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lLgMs-0004cY-6w; Mon, 15 Mar 2021 02:06:23 -0400 Received: from sas1-ec30c78b6c5b.qloud-c.yandex.net (sas1-ec30c78b6c5b.qloud-c.yandex.net [IPv6:2a02:6b8:c14:2704:0:640:ec30:c78b]) by forwardcorp1o.mail.yandex.net (Yandex) with ESMTP id A8B1F2E14C4; Mon, 15 Mar 2021 09:06:14 +0300 (MSK) Received: from sas1-24e978739efd.qloud-c.yandex.net (sas1-24e978739efd.qloud-c.yandex.net [2a02:6b8:c14:3088:0:640:24e9:7873]) by sas1-ec30c78b6c5b.qloud-c.yandex.net (mxbackcorp/Yandex) with ESMTP id zPBywE1Vnh-6EYOjHpB; Mon, 15 Mar 2021 09:06:14 +0300 Received: from dynamic-vpn.dhcp.yndx.net (dynamic-vpn.dhcp.yndx.net [2a02:6b8:b081:23::1:15]) by sas1-24e978739efd.qloud-c.yandex.net (smtpcorp/Yandex) with ESMTPSA id FoAWGMWkZt-6Dn01PpF; Mon, 15 Mar 2021 09:06:13 +0300 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) (Client certificate not present) Precedence: bulk DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yandex-team.ru; s=default; t=1615788374; bh=xl3BXy0Fo1E+ViyIcbQolDDdibk1njkqm+tk2h+xMyE=; h=In-Reply-To:Message-Id:References:Date:Subject:To:From:Cc; b=xnMLTJAEIsaH8JA8pJqpKp6/1WtodjAVMZLujd7DzRoVtbTgnq7WksAi1UXWTOFpz 7HcTTZajiQ+wkkAYdzTFuSYpNfYSBSnek7sO95E91Dx1tuS+7EmR5k4FYvpI2konu+ AlEDGK98moa8A2q1bUXmfqkYQ5LvLU66FPP5H8KM= Authentication-Results: sas1-ec30c78b6c5b.qloud-c.yandex.net; dkim=pass header.i=@yandex-team.ru From: Roman Kagan To: qemu-devel@nongnu.org Subject: [PATCH 2/7] block/nbd: use uniformly nbd_client_connecting_wait Date: Mon, 15 Mar 2021 09:06:06 +0300 Message-Id: <20210315060611.2989049-3-rvkagan@yandex-team.ru> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20210315060611.2989049-1-rvkagan@yandex-team.ru> References: <20210315060611.2989049-1-rvkagan@yandex-team.ru> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=95.108.205.193; envelope-from=rvkagan@yandex-team.ru; helo=forwardcorp1o.mail.yandex.net X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Vladimir Sementsov-Ogievskiy , qemu-block@nongnu.org, Max Reitz , yc-core@yandex-team.ru Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) Content-Type: text/plain; charset="utf-8" Use nbd_client_connecting_wait uniformly all over the block/nbd.c. While at this, drop the redundant check for nbd_client_connecting_wait in reconnect_delay_timer_init, as all its callsites do this check too. Signed-off-by: Roman Kagan Reviewed-by: Vladimir Sementsov-Ogievskiy --- block/nbd.c | 34 +++++++++++++++------------------- 1 file changed, 15 insertions(+), 19 deletions(-) diff --git a/block/nbd.c b/block/nbd.c index 447d176b76..1d8edb5b21 100644 --- a/block/nbd.c +++ b/block/nbd.c @@ -165,6 +165,18 @@ static void nbd_clear_bdrvstate(BDRVNBDState *s) s->x_dirty_bitmap =3D NULL; } =20 +static bool nbd_client_connecting(BDRVNBDState *s) +{ + NBDClientState state =3D qatomic_load_acquire(&s->state); + return state =3D=3D NBD_CLIENT_CONNECTING_WAIT || + state =3D=3D NBD_CLIENT_CONNECTING_NOWAIT; +} + +static bool nbd_client_connecting_wait(BDRVNBDState *s) +{ + return qatomic_load_acquire(&s->state) =3D=3D NBD_CLIENT_CONNECTING_WA= IT; +} + static void nbd_channel_error(BDRVNBDState *s, int ret) { if (ret =3D=3D -EIO) { @@ -205,7 +217,7 @@ static void reconnect_delay_timer_cb(void *opaque) { BDRVNBDState *s =3D opaque; =20 - if (qatomic_load_acquire(&s->state) =3D=3D NBD_CLIENT_CONNECTING_WAIT)= { + if (nbd_client_connecting_wait(s)) { s->state =3D NBD_CLIENT_CONNECTING_NOWAIT; while (qemu_co_enter_next(&s->free_sema, NULL)) { /* Resume all queued requests */ @@ -217,10 +229,6 @@ static void reconnect_delay_timer_cb(void *opaque) =20 static void reconnect_delay_timer_init(BDRVNBDState *s, uint64_t expire_ti= me_ns) { - if (qatomic_load_acquire(&s->state) !=3D NBD_CLIENT_CONNECTING_WAIT) { - return; - } - assert(!s->reconnect_delay_timer); s->reconnect_delay_timer =3D aio_timer_new(bdrv_get_aio_context(s->bs), QEMU_CLOCK_REALTIME, @@ -346,18 +354,6 @@ static void nbd_teardown_connection(BlockDriverState *= bs) assert(!s->connection_co); } =20 -static bool nbd_client_connecting(BDRVNBDState *s) -{ - NBDClientState state =3D qatomic_load_acquire(&s->state); - return state =3D=3D NBD_CLIENT_CONNECTING_WAIT || - state =3D=3D NBD_CLIENT_CONNECTING_NOWAIT; -} - -static bool nbd_client_connecting_wait(BDRVNBDState *s) -{ - return qatomic_load_acquire(&s->state) =3D=3D NBD_CLIENT_CONNECTING_WA= IT; -} - static void connect_bh(void *opaque) { BDRVNBDState *state =3D opaque; @@ -667,7 +663,7 @@ static coroutine_fn void nbd_co_reconnect_loop(BDRVNBDS= tate *s) uint64_t timeout =3D 1 * NANOSECONDS_PER_SECOND; uint64_t max_timeout =3D 16 * NANOSECONDS_PER_SECOND; =20 - if (qatomic_load_acquire(&s->state) =3D=3D NBD_CLIENT_CONNECTING_WAIT)= { + if (nbd_client_connecting_wait(s)) { reconnect_delay_timer_init(s, qemu_clock_get_ns(QEMU_CLOCK_REALTIM= E) + s->reconnect_delay * NANOSECONDS_PER_SE= COND); } @@ -2473,7 +2469,7 @@ static void nbd_cancel_in_flight(BlockDriverState *bs) =20 reconnect_delay_timer_del(s); =20 - if (s->state =3D=3D NBD_CLIENT_CONNECTING_WAIT) { + if (nbd_client_connecting_wait(s)) { s->state =3D NBD_CLIENT_CONNECTING_NOWAIT; qemu_co_queue_restart_all(&s->free_sema); } --=20 2.30.2 From nobody Fri May 17 19:12:47 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=yandex-team.ru ARC-Seal: i=1; a=rsa-sha256; t=1615788497; cv=none; d=zohomail.com; s=zohoarc; b=eiXW3OJ6WpI3ccyxBX1Eb2V7XvmJ0QbLWvuJw23+g61rKWIzxDT1zuv04ZiiAM27OWm7UNXTNcLNH6+WVotUD3iDYb4Lblnv7P/4JlVH3OY3W9hv5Iz09eCQCCCBeoRAwh1fvYY+42tRV3JB5M0Hs2v4T4VtLdbx9EQR9hMX/6Q= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1615788497; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=ZFOtVVKovF3ArT9MW64RzPKbNlbAihWsxHJHarLjJG8=; b=CdnuTtTPmnRm3T9nb/5fVVldvTWEp1NfOvJ5bP5Zv1eM3fzQX9JbRDMpR8RlOVZ1W+DHN7ywnyQViEtJD4CAn83FApdKc/h7Rv0vCjSeEVKQXEXSDII5r+eYVecI8Pko9E+BhjxRnCLlN+67hNituDugNM/tXy9n3vNPgvitkKA= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1615788497388856.8244384443112; Sun, 14 Mar 2021 23:08:17 -0700 (PDT) Received: from localhost ([::1]:35090 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lLgOm-00035c-79 for importer@patchew.org; Mon, 15 Mar 2021 02:08:16 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:47118) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lLgMw-00015t-Jk; Mon, 15 Mar 2021 02:06:22 -0400 Received: from forwardcorp1o.mail.yandex.net ([2a02:6b8:0:1a2d::193]:34756) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lLgMt-0004d5-6C; Mon, 15 Mar 2021 02:06:22 -0400 Received: from sas1-6b1512233ef6.qloud-c.yandex.net (sas1-6b1512233ef6.qloud-c.yandex.net [IPv6:2a02:6b8:c14:44af:0:640:6b15:1223]) by forwardcorp1o.mail.yandex.net (Yandex) with ESMTP id 6CE722E14CB; Mon, 15 Mar 2021 09:06:15 +0300 (MSK) Received: from sas1-24e978739efd.qloud-c.yandex.net (sas1-24e978739efd.qloud-c.yandex.net [2a02:6b8:c14:3088:0:640:24e9:7873]) by sas1-6b1512233ef6.qloud-c.yandex.net (mxbackcorp/Yandex) with ESMTP id kuNQ7xWcW9-6FC8xRCw; Mon, 15 Mar 2021 09:06:15 +0300 Received: from dynamic-vpn.dhcp.yndx.net (dynamic-vpn.dhcp.yndx.net [2a02:6b8:b081:23::1:15]) by sas1-24e978739efd.qloud-c.yandex.net (smtpcorp/Yandex) with ESMTPSA id FoAWGMWkZt-6En06RxR; Mon, 15 Mar 2021 09:06:15 +0300 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) (Client certificate not present) Precedence: bulk DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yandex-team.ru; s=default; t=1615788375; bh=ZFOtVVKovF3ArT9MW64RzPKbNlbAihWsxHJHarLjJG8=; h=In-Reply-To:Message-Id:References:Date:Subject:To:From:Cc; b=y1f0UBKVQgcU9Bc/W6g5chvl3/inJVUZFajHGa+V31EwSOeEx+2a2kO8HtrlXfs/4 NXqtFIkjM6laxkml9DFcRRh5Pu8J5LmHOBh2iHPu4aaFOYOTSwSsb4wXahN993qT+W EEYA4i21BxOlX+fBOAjLJ4Ru+g+uk9o8SOdG/6io= Authentication-Results: sas1-6b1512233ef6.qloud-c.yandex.net; dkim=pass header.i=@yandex-team.ru From: Roman Kagan To: qemu-devel@nongnu.org Subject: [PATCH 3/7] block/nbd: assert attach/detach runs in the proper context Date: Mon, 15 Mar 2021 09:06:07 +0300 Message-Id: <20210315060611.2989049-4-rvkagan@yandex-team.ru> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20210315060611.2989049-1-rvkagan@yandex-team.ru> References: <20210315060611.2989049-1-rvkagan@yandex-team.ru> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2a02:6b8:0:1a2d::193; envelope-from=rvkagan@yandex-team.ru; helo=forwardcorp1o.mail.yandex.net X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Vladimir Sementsov-Ogievskiy , qemu-block@nongnu.org, Max Reitz , yc-core@yandex-team.ru Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) Content-Type: text/plain; charset="utf-8" Document (via a comment and an assert) that nbd_client_detach_aio_context and nbd_client_attach_aio_context_bh run in the desired aio_context. Signed-off-by: Roman Kagan --- block/nbd.c | 12 ++++++++++++ 1 file changed, 12 insertions(+) diff --git a/block/nbd.c b/block/nbd.c index 1d8edb5b21..658b827d24 100644 --- a/block/nbd.c +++ b/block/nbd.c @@ -241,6 +241,12 @@ static void nbd_client_detach_aio_context(BlockDriverS= tate *bs) { BDRVNBDState *s =3D (BDRVNBDState *)bs->opaque; =20 + /* + * This runs in the (old, about to be detached) aio context of the @bs= so + * accessing the stuff on @s is concurrency-free. + */ + assert(qemu_get_current_aio_context() =3D=3D bdrv_get_aio_context(bs)); + /* Timer is deleted in nbd_client_co_drain_begin() */ assert(!s->reconnect_delay_timer); /* @@ -258,6 +264,12 @@ static void nbd_client_attach_aio_context_bh(void *opa= que) BlockDriverState *bs =3D opaque; BDRVNBDState *s =3D (BDRVNBDState *)bs->opaque; =20 + /* + * This runs in the (new, just attached) aio context of the @bs so + * accessing the stuff on @s is concurrency-free. + */ + assert(qemu_get_current_aio_context() =3D=3D bdrv_get_aio_context(bs)); + if (s->connection_co) { /* * The node is still drained, so we know the coroutine has yielded= in --=20 2.30.2 From nobody Fri May 17 19:12:47 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=yandex-team.ru ARC-Seal: i=1; a=rsa-sha256; t=1615788752; cv=none; d=zohomail.com; s=zohoarc; b=Ew/Y7rkZSRm6WddM5+LCl/Ngp0hM5AWmfu9Xp1huSbGqOrufT7EuRrR9obRVATmuOqQeanFNK7bAfqZ5AKXZr20q5gBjZx+2BpTU2yLQURDofvTmNqFI2vc79KgeiHKdTx6y5B8eSoej3o+ieNi8mlq3XzQFz2wISrLdAgvImKc= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1615788752; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=8puCtjf702zr06rnz2Glwa8au2gwVRWqa0tN3lwDyDc=; b=O7Nk1UTCXwlhGoTZEq0YGJxa+LM17LhLzVXF4iPPleF8raDMWUMfnbh4Ak3N21LZWo5reQfqjJxzUDWrGz+Wb8ysq7uNqIVTt0sqhWBSu4iTmKcPtf8X19ZRqMtkCj/XNa4nt5B4V3nQqbx+oZoAIO2KOzwvzZZovsMyIv+wQAI= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 16157887524351011.1087824647296; Sun, 14 Mar 2021 23:12:32 -0700 (PDT) Received: from localhost ([::1]:49556 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lLgSt-0000ee-FG for importer@patchew.org; Mon, 15 Mar 2021 02:12:31 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:47104) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lLgMv-000157-VZ; Mon, 15 Mar 2021 02:06:22 -0400 Received: from forwardcorp1p.mail.yandex.net ([2a02:6b8:0:1472:2741:0:8b6:217]:50888) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lLgMs-0004dp-CB; Mon, 15 Mar 2021 02:06:21 -0400 Received: from sas1-6b1512233ef6.qloud-c.yandex.net (sas1-6b1512233ef6.qloud-c.yandex.net [IPv6:2a02:6b8:c14:44af:0:640:6b15:1223]) by forwardcorp1p.mail.yandex.net (Yandex) with ESMTP id 3A18F2E1531; Mon, 15 Mar 2021 09:06:16 +0300 (MSK) Received: from sas1-24e978739efd.qloud-c.yandex.net (sas1-24e978739efd.qloud-c.yandex.net [2a02:6b8:c14:3088:0:640:24e9:7873]) by sas1-6b1512233ef6.qloud-c.yandex.net (mxbackcorp/Yandex) with ESMTP id K2ojkeT6qu-6FCaW8gS; Mon, 15 Mar 2021 09:06:16 +0300 Received: from dynamic-vpn.dhcp.yndx.net (dynamic-vpn.dhcp.yndx.net [2a02:6b8:b081:23::1:15]) by sas1-24e978739efd.qloud-c.yandex.net (smtpcorp/Yandex) with ESMTPSA id FoAWGMWkZt-6Fn0HVJ9; Mon, 15 Mar 2021 09:06:15 +0300 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) (Client certificate not present) Precedence: bulk DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yandex-team.ru; s=default; t=1615788376; bh=8puCtjf702zr06rnz2Glwa8au2gwVRWqa0tN3lwDyDc=; h=In-Reply-To:Message-Id:References:Date:Subject:To:From:Cc; b=DbAMWUgKGYFaZombMp8I62TLUmPmEt7IulepZYMoJzr6DivmI+FfEQnKqwHjiNEQj yztuZ6VfZ2NLPLAfBsC5k3Rm9zXS+LF9+EoXb+E7Q2Q0/Z/Er+HeA52JF1sNpaVsTc pTsp1hz1MqvrUyAPbMlKOPaBmt/X5EMuUGew1CKA= Authentication-Results: sas1-6b1512233ef6.qloud-c.yandex.net; dkim=pass header.i=@yandex-team.ru From: Roman Kagan To: qemu-devel@nongnu.org Subject: [PATCH 4/7] block/nbd: transfer reconnection stuff across aio_context switch Date: Mon, 15 Mar 2021 09:06:08 +0300 Message-Id: <20210315060611.2989049-5-rvkagan@yandex-team.ru> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20210315060611.2989049-1-rvkagan@yandex-team.ru> References: <20210315060611.2989049-1-rvkagan@yandex-team.ru> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2a02:6b8:0:1472:2741:0:8b6:217; envelope-from=rvkagan@yandex-team.ru; helo=forwardcorp1p.mail.yandex.net X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Vladimir Sementsov-Ogievskiy , qemu-block@nongnu.org, Max Reitz , yc-core@yandex-team.ru Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) Content-Type: text/plain; charset="utf-8" Make varios pieces of reconnection logic correctly survive the transition of the BDRVNBDState from one aio_context to another. In particular, - cancel the reconnect_delay_timer and rearm it in the new context; - cancel the sleep of the connection_co between reconnect attempt so that it continues in the new context; - prevent the connection thread from delivering its status to the old context, and retartget it to the new context on attach. None of these is needed at the moment because the aio_context switch happens within a drained section and that effectively terminates the reconnection logic on entry and starts it over on exit. However, this patch paves the way to keeping the reconnection process active across the drained section (in a followup patch). Signed-off-by: Roman Kagan --- block/nbd.c | 44 ++++++++++++++++++++++++++++++++++++++++++-- 1 file changed, 42 insertions(+), 2 deletions(-) diff --git a/block/nbd.c b/block/nbd.c index 658b827d24..a6d713ba58 100644 --- a/block/nbd.c +++ b/block/nbd.c @@ -126,6 +126,7 @@ typedef struct BDRVNBDState { bool wait_in_flight; =20 QEMUTimer *reconnect_delay_timer; + int64_t reconnect_expire_time_ns; =20 NBDClientRequest requests[MAX_NBD_REQUESTS]; NBDReply reply; @@ -240,6 +241,7 @@ static void reconnect_delay_timer_init(BDRVNBDState *s,= uint64_t expire_time_ns) static void nbd_client_detach_aio_context(BlockDriverState *bs) { BDRVNBDState *s =3D (BDRVNBDState *)bs->opaque; + NBDConnectThread *thr =3D s->connect_thread; =20 /* * This runs in the (old, about to be detached) aio context of the @bs= so @@ -247,8 +249,31 @@ static void nbd_client_detach_aio_context(BlockDriverS= tate *bs) */ assert(qemu_get_current_aio_context() =3D=3D bdrv_get_aio_context(bs)); =20 - /* Timer is deleted in nbd_client_co_drain_begin() */ - assert(!s->reconnect_delay_timer); + /* + * Make sure the connection thread doesn't try to deliver its status t= o the + * old context. + */ + qemu_mutex_lock(&thr->mutex); + thr->bh_ctx =3D NULL; + qemu_mutex_unlock(&thr->mutex); + + /* + * Preserve the expiration time of the reconnect_delay_timer in order = to + * resume it on the new aio context. + */ + s->reconnect_expire_time_ns =3D s->reconnect_delay_timer ? + timer_expire_time_ns(s->reconnect_delay_timer) : -1; + reconnect_delay_timer_del(s); + + /* + * If the connection coroutine was sleeping between reconnect attempts, + * wake it up now and let it continue the process in the new aio conte= xt. + * This will distort the exponential back-off but that's probably ok. + */ + if (s->connection_co_sleep_ns_state) { + qemu_co_sleep_wake(s->connection_co_sleep_ns_state); + } + /* * If reconnect is in progress we may have no ->ioc. It will be * re-instantiated in the proper aio context once the connection is @@ -263,6 +288,7 @@ static void nbd_client_attach_aio_context_bh(void *opaq= ue) { BlockDriverState *bs =3D opaque; BDRVNBDState *s =3D (BDRVNBDState *)bs->opaque; + NBDConnectThread *thr =3D s->connect_thread; =20 /* * This runs in the (new, just attached) aio context of the @bs so @@ -270,6 +296,20 @@ static void nbd_client_attach_aio_context_bh(void *opa= que) */ assert(qemu_get_current_aio_context() =3D=3D bdrv_get_aio_context(bs)); =20 + if (nbd_client_connecting_wait(s) && s->reconnect_expire_time_ns >=3D = 0) { + reconnect_delay_timer_init(s, s->reconnect_expire_time_ns); + } + + /* + * If the connection thread hasn't completed connecting yet, make sure= it + * can deliver its status in the new context. + */ + qemu_mutex_lock(&thr->mutex); + if (thr->state =3D=3D CONNECT_THREAD_RUNNING) { + thr->bh_ctx =3D qemu_get_current_aio_context(); + } + qemu_mutex_unlock(&thr->mutex); + if (s->connection_co) { /* * The node is still drained, so we know the coroutine has yielded= in --=20 2.30.2 From nobody Fri May 17 19:12:47 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=yandex-team.ru ARC-Seal: i=1; a=rsa-sha256; t=1615788658; cv=none; d=zohomail.com; s=zohoarc; b=W44KVJ3mGBQhRPiWPNT3BoPxZCRKoYS8ukLGC8lDb9lgjo88sYVmwBl2DZ3fHb6s/ZKjO5A669S785ZZGFR8JTF/iixUE896iWyv8ZFShrP4B+65/YK7ccc3XAui4S0tHgu5CZ/wJWHD+cknLSEJSKLBtJSvd4KsFbs7afgJMCo= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1615788658; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=59ZvSWAYCL+QmZmMyzOFun6eZVfFXcIJ0m4xhO4qzKY=; b=TRCZG4vO0AgAE9j5HFjRH1VHMR8JzeI6VxirAQ81ZDltnINDfvXn810Tg96fcGiKD1Zq3m+MmNdH5TCdAzXBLrK8vx0LXDDfmG5dOr2Cn26nhVSx7qTDnhuwGxlg4QDCM37DEEd/U01VJRR9becm3Sk+FHPoEJljAGk+thZ8/0g= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 161578865808113.275243036268876; Sun, 14 Mar 2021 23:10:58 -0700 (PDT) Received: from localhost ([::1]:42446 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lLgRL-000695-QP for importer@patchew.org; Mon, 15 Mar 2021 02:10:55 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:47106) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lLgMw-000158-2Y; Mon, 15 Mar 2021 02:06:22 -0400 Received: from forwardcorp1o.mail.yandex.net ([95.108.205.193]:47644) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lLgMs-0004dx-Pu; Mon, 15 Mar 2021 02:06:21 -0400 Received: from sas1-ec30c78b6c5b.qloud-c.yandex.net (sas1-ec30c78b6c5b.qloud-c.yandex.net [IPv6:2a02:6b8:c14:2704:0:640:ec30:c78b]) by forwardcorp1o.mail.yandex.net (Yandex) with ESMTP id DB6D92E14DF; Mon, 15 Mar 2021 09:06:16 +0300 (MSK) Received: from sas1-24e978739efd.qloud-c.yandex.net (sas1-24e978739efd.qloud-c.yandex.net [2a02:6b8:c14:3088:0:640:24e9:7873]) by sas1-ec30c78b6c5b.qloud-c.yandex.net (mxbackcorp/Yandex) with ESMTP id 1SzG6WpSW2-6GY0jP3q; Mon, 15 Mar 2021 09:06:16 +0300 Received: from dynamic-vpn.dhcp.yndx.net (dynamic-vpn.dhcp.yndx.net [2a02:6b8:b081:23::1:15]) by sas1-24e978739efd.qloud-c.yandex.net (smtpcorp/Yandex) with ESMTPSA id FoAWGMWkZt-6Gn0MMeT; Mon, 15 Mar 2021 09:06:16 +0300 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) (Client certificate not present) Precedence: bulk DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yandex-team.ru; s=default; t=1615788376; bh=59ZvSWAYCL+QmZmMyzOFun6eZVfFXcIJ0m4xhO4qzKY=; h=In-Reply-To:Message-Id:References:Date:Subject:To:From:Cc; b=W/SbjbrCh8M7jZCpBL1JO2RVAXjkjz5mW9Seb5q+vQPZL0djBs599ni8vYp53BsiS WqCm4CYEfjdKbWd4PUEF3EwjIJaetjvme2pdl+93Ez/Ev/s+fhdEmdFnluVXbzkqid zqZID4nQyLvMKJTIpaIGZUCAd8BDk4kdT62nZYYU= Authentication-Results: sas1-ec30c78b6c5b.qloud-c.yandex.net; dkim=pass header.i=@yandex-team.ru From: Roman Kagan To: qemu-devel@nongnu.org Subject: [PATCH 5/7] block/nbd: better document a case in nbd_co_establish_connection Date: Mon, 15 Mar 2021 09:06:09 +0300 Message-Id: <20210315060611.2989049-6-rvkagan@yandex-team.ru> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20210315060611.2989049-1-rvkagan@yandex-team.ru> References: <20210315060611.2989049-1-rvkagan@yandex-team.ru> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=95.108.205.193; envelope-from=rvkagan@yandex-team.ru; helo=forwardcorp1o.mail.yandex.net X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Vladimir Sementsov-Ogievskiy , qemu-block@nongnu.org, Max Reitz , yc-core@yandex-team.ru Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) Content-Type: text/plain; charset="utf-8" Cosmetic: adjust the comment and the return value in nbd_co_establish_connection where it's entered while the connection thread is still running. Signed-off-by: Roman Kagan --- block/nbd.c | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/block/nbd.c b/block/nbd.c index a6d713ba58..a3eb9b9079 100644 --- a/block/nbd.c +++ b/block/nbd.c @@ -562,11 +562,12 @@ nbd_co_establish_connection(BlockDriverState *bs, Err= or **errp) case CONNECT_THREAD_RUNNING: case CONNECT_THREAD_RUNNING_DETACHED: /* - * Obviously, drained section wants to start. Report the attempt as - * failed. Still connect thread is executing in background, and its + * Spurious corotine resume before connection attempt has finished, + * presumably upon attaching a new aio_context. Report the attempt= as + * failed. Still connect thread is executing in background, and i= ts * result may be used for next connection attempt. */ - ret =3D -1; + ret =3D -ECONNABORTED; error_setg(errp, "Connection attempt cancelled by other operation"= ); break; =20 --=20 2.30.2 From nobody Fri May 17 19:12:47 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=yandex-team.ru ARC-Seal: i=1; a=rsa-sha256; t=1615788492; cv=none; d=zohomail.com; s=zohoarc; b=iB1vfkVRWtGcLChBcW8HHoQiRZqlkJi4kpu31FwbGiynYHBMSL0VNIZMTuQl7RsTH1qLqDhHjXdHoVeLw9Rz5B2aP9VZUb78/1cfA2Fh+qLPfp+cDRd0ApL1ulmy6rNyEMnvpzUzxf40YQS23iUWSSlngR+eJ8Xza9cWpCgWXk8= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1615788492; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=lSf6LB0Vf7XGUeUux2+g4MvHc7uITHz4QyRCM6OV388=; b=j46GVLShlY+FTiRoIMWMxlhpUtrW6xnbXgZ7bXl6qBgHDUbc1Ut/NsESbcwR75QK+a3OJDoz0Os7aiSMrOR3+ySXjrJ6OlajDvj/9bHJ546v29N46pBBikE9blhwX09RcEK7jwJOul6w42+AmyKMphE6Jp1nvYeJ2NCK8Ew7UsI= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1615788491764180.62448971847903; Sun, 14 Mar 2021 23:08:11 -0700 (PDT) Received: from localhost ([::1]:34494 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lLgOg-0002r2-GU for importer@patchew.org; Mon, 15 Mar 2021 02:08:10 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:47108) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lLgMw-00015B-1n; Mon, 15 Mar 2021 02:06:22 -0400 Received: from forwardcorp1p.mail.yandex.net ([77.88.29.217]:49200) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lLgMt-0004eP-O7; Mon, 15 Mar 2021 02:06:21 -0400 Received: from sas1-6b1512233ef6.qloud-c.yandex.net (sas1-6b1512233ef6.qloud-c.yandex.net [IPv6:2a02:6b8:c14:44af:0:640:6b15:1223]) by forwardcorp1p.mail.yandex.net (Yandex) with ESMTP id 8050C2E1522; Mon, 15 Mar 2021 09:06:17 +0300 (MSK) Received: from sas1-24e978739efd.qloud-c.yandex.net (sas1-24e978739efd.qloud-c.yandex.net [2a02:6b8:c14:3088:0:640:24e9:7873]) by sas1-6b1512233ef6.qloud-c.yandex.net (mxbackcorp/Yandex) with ESMTP id XNHX0MVvMQ-6HCWuf56; Mon, 15 Mar 2021 09:06:17 +0300 Received: from dynamic-vpn.dhcp.yndx.net (dynamic-vpn.dhcp.yndx.net [2a02:6b8:b081:23::1:15]) by sas1-24e978739efd.qloud-c.yandex.net (smtpcorp/Yandex) with ESMTPSA id FoAWGMWkZt-6Hn0Lu4v; Mon, 15 Mar 2021 09:06:17 +0300 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) (Client certificate not present) Precedence: bulk DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yandex-team.ru; s=default; t=1615788377; bh=lSf6LB0Vf7XGUeUux2+g4MvHc7uITHz4QyRCM6OV388=; h=In-Reply-To:Message-Id:References:Date:Subject:To:From:Cc; b=ieyk/xMiPEqsFgP0oFHKshl3eExZMz0r7KlrqF5N2jAYcTsLr+XYNh5Yv3ZJpVcLO WWDD7+QhDJupXLj1PEr5tpOKjvsisS+TOM+Uwf8XwI1ZhTO291YUYguL3+uf8YtOPM Xtvsu9cKQJgt4r2bb53/cDnKarWk00Tlj6BJPNEs= Authentication-Results: sas1-6b1512233ef6.qloud-c.yandex.net; dkim=pass header.i=@yandex-team.ru From: Roman Kagan To: qemu-devel@nongnu.org Subject: [PATCH 6/7] block/nbd: decouple reconnect from drain Date: Mon, 15 Mar 2021 09:06:10 +0300 Message-Id: <20210315060611.2989049-7-rvkagan@yandex-team.ru> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20210315060611.2989049-1-rvkagan@yandex-team.ru> References: <20210315060611.2989049-1-rvkagan@yandex-team.ru> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=77.88.29.217; envelope-from=rvkagan@yandex-team.ru; helo=forwardcorp1p.mail.yandex.net X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Vladimir Sementsov-Ogievskiy , qemu-block@nongnu.org, Max Reitz , yc-core@yandex-team.ru Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) Content-Type: text/plain; charset="utf-8" The reconnection logic doesn't need to stop while in a drained section. Moreover it has to be active during the drained section, as the requests that were caught in-flight with the connection to the server broken can only usefully get drained if the connection is restored. Otherwise such requests can only either stall resulting in a deadlock (before 8c517de24a), or be aborted defeating the purpose of the reconnection machinery (after 8c517de24a). Since the pieces of the reconnection logic are now properly migrated from one aio_context to another, it appears safe to just stop messing with the drained section in the reconnection code. Fixes: 5ad81b4946 ("nbd: Restrict connection_co reentrance") Fixes: 8c517de24a ("block/nbd: fix drain dead-lock because of nbd reconnect= -delay") Signed-off-by: Roman Kagan --- block/nbd.c | 79 +++-------------------------------------------------- 1 file changed, 4 insertions(+), 75 deletions(-) diff --git a/block/nbd.c b/block/nbd.c index a3eb9b9079..a5a9e4aca5 100644 --- a/block/nbd.c +++ b/block/nbd.c @@ -117,8 +117,6 @@ typedef struct BDRVNBDState { Coroutine *connection_co; Coroutine *teardown_co; QemuCoSleepState *connection_co_sleep_ns_state; - bool drained; - bool wait_drained_end; int in_flight; NBDClientState state; int connect_status; @@ -311,12 +309,6 @@ static void nbd_client_attach_aio_context_bh(void *opa= que) qemu_mutex_unlock(&thr->mutex); =20 if (s->connection_co) { - /* - * The node is still drained, so we know the coroutine has yielded= in - * nbd_read_eof(), the only place where bs->in_flight can reach 0,= or - * it is entered for the first time. Both places are safe for ente= ring - * the coroutine. - */ qemu_aio_coroutine_enter(bs->aio_context, s->connection_co); } bdrv_dec_in_flight(bs); @@ -344,37 +336,6 @@ static void nbd_client_attach_aio_context(BlockDriverS= tate *bs, aio_wait_bh_oneshot(new_context, nbd_client_attach_aio_context_bh, bs); } =20 -static void coroutine_fn nbd_client_co_drain_begin(BlockDriverState *bs) -{ - BDRVNBDState *s =3D (BDRVNBDState *)bs->opaque; - - s->drained =3D true; - if (s->connection_co_sleep_ns_state) { - qemu_co_sleep_wake(s->connection_co_sleep_ns_state); - } - - nbd_co_establish_connection_cancel(bs, false); - - reconnect_delay_timer_del(s); - - if (qatomic_load_acquire(&s->state) =3D=3D NBD_CLIENT_CONNECTING_WAIT)= { - s->state =3D NBD_CLIENT_CONNECTING_NOWAIT; - qemu_co_queue_restart_all(&s->free_sema); - } -} - -static void coroutine_fn nbd_client_co_drain_end(BlockDriverState *bs) -{ - BDRVNBDState *s =3D (BDRVNBDState *)bs->opaque; - - s->drained =3D false; - if (s->wait_drained_end) { - s->wait_drained_end =3D false; - aio_co_wake(s->connection_co); - } -} - - static void nbd_teardown_connection(BlockDriverState *bs) { BDRVNBDState *s =3D (BDRVNBDState *)bs->opaque; @@ -686,16 +647,6 @@ static coroutine_fn void nbd_reconnect_attempt(BDRVNBD= State *s) =20 ret =3D nbd_client_handshake(s->bs, &local_err); =20 - if (s->drained) { - s->wait_drained_end =3D true; - while (s->drained) { - /* - * We may be entered once from nbd_client_attach_aio_context_bh - * and then from nbd_client_co_drain_end. So here is a loop. - */ - qemu_coroutine_yield(); - } - } bdrv_inc_in_flight(s->bs); =20 out: @@ -724,26 +675,10 @@ static coroutine_fn void nbd_co_reconnect_loop(BDRVNB= DState *s) nbd_reconnect_attempt(s); =20 while (nbd_client_connecting(s)) { - if (s->drained) { - bdrv_dec_in_flight(s->bs); - s->wait_drained_end =3D true; - while (s->drained) { - /* - * We may be entered once from nbd_client_attach_aio_conte= xt_bh - * and then from nbd_client_co_drain_end. So here is a loo= p. - */ - qemu_coroutine_yield(); - } - bdrv_inc_in_flight(s->bs); - } else { - qemu_co_sleep_ns_wakeable(QEMU_CLOCK_REALTIME, timeout, - &s->connection_co_sleep_ns_state); - if (s->drained) { - continue; - } - if (timeout < max_timeout) { - timeout *=3D 2; - } + qemu_co_sleep_ns_wakeable(QEMU_CLOCK_REALTIME, timeout, + &s->connection_co_sleep_ns_state); + if (timeout < max_timeout) { + timeout *=3D 2; } =20 nbd_reconnect_attempt(s); @@ -2548,8 +2483,6 @@ static BlockDriver bdrv_nbd =3D { .bdrv_getlength =3D nbd_getlength, .bdrv_detach_aio_context =3D nbd_client_detach_aio_context, .bdrv_attach_aio_context =3D nbd_client_attach_aio_context, - .bdrv_co_drain_begin =3D nbd_client_co_drain_begin, - .bdrv_co_drain_end =3D nbd_client_co_drain_end, .bdrv_refresh_filename =3D nbd_refresh_filename, .bdrv_co_block_status =3D nbd_client_co_block_status, .bdrv_dirname =3D nbd_dirname, @@ -2577,8 +2510,6 @@ static BlockDriver bdrv_nbd_tcp =3D { .bdrv_getlength =3D nbd_getlength, .bdrv_detach_aio_context =3D nbd_client_detach_aio_context, .bdrv_attach_aio_context =3D nbd_client_attach_aio_context, - .bdrv_co_drain_begin =3D nbd_client_co_drain_begin, - .bdrv_co_drain_end =3D nbd_client_co_drain_end, .bdrv_refresh_filename =3D nbd_refresh_filename, .bdrv_co_block_status =3D nbd_client_co_block_status, .bdrv_dirname =3D nbd_dirname, @@ -2606,8 +2537,6 @@ static BlockDriver bdrv_nbd_unix =3D { .bdrv_getlength =3D nbd_getlength, .bdrv_detach_aio_context =3D nbd_client_detach_aio_context, .bdrv_attach_aio_context =3D nbd_client_attach_aio_context, - .bdrv_co_drain_begin =3D nbd_client_co_drain_begin, - .bdrv_co_drain_end =3D nbd_client_co_drain_end, .bdrv_refresh_filename =3D nbd_refresh_filename, .bdrv_co_block_status =3D nbd_client_co_block_status, .bdrv_dirname =3D nbd_dirname, --=20 2.30.2 From nobody Fri May 17 19:12:47 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=yandex-team.ru ARC-Seal: i=1; a=rsa-sha256; t=1615788556; cv=none; d=zohomail.com; s=zohoarc; b=SwY+2eHh7ZVrxlwnstI8rVUiB7Nj931m1jwoT5weRRm4IMGFBbzrxKmVKsfZ66h6Qhub+6FZMoWBmF682vquMa0yQUdfl30UqqSjZFGOLxboQQMjY8kAY38c7u0VZFFh/j4N8lusyL+4tAbivwJSQZCZo6BzRB/Jp1tWGSLex4M= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1615788556; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=c9G60La+2Sf+4noHvdEmbbjc208DGWTp5aj4kVCB4wo=; b=MRdJYKa1DZHCs9c9y6KnsW2nYhkx+8xEZAnlzfpOwh39BWL5aQuaAiLoHb+LotlF04ELL+XqiePxTwGKmACkZTG0kb/U84NC2l5uY3dV9vj2+dgPKn4LK2SLtN8e39TBVAVgPQU+tvTqAnlKAv0Ay3xoKQNcZ8eW9KxIQD84upE= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1615788556308759.5288786167865; Sun, 14 Mar 2021 23:09:16 -0700 (PDT) Received: from localhost ([::1]:36476 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lLgPj-0003eS-9Y for importer@patchew.org; Mon, 15 Mar 2021 02:09:15 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:47120) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lLgMx-00017F-5C; Mon, 15 Mar 2021 02:06:23 -0400 Received: from forwardcorp1o.mail.yandex.net ([95.108.205.193]:47696) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lLgMv-0004gA-C6; Mon, 15 Mar 2021 02:06:22 -0400 Received: from sas1-ec30c78b6c5b.qloud-c.yandex.net (sas1-ec30c78b6c5b.qloud-c.yandex.net [IPv6:2a02:6b8:c14:2704:0:640:ec30:c78b]) by forwardcorp1o.mail.yandex.net (Yandex) with ESMTP id 234BA2E14EE; Mon, 15 Mar 2021 09:06:19 +0300 (MSK) Received: from sas1-24e978739efd.qloud-c.yandex.net (sas1-24e978739efd.qloud-c.yandex.net [2a02:6b8:c14:3088:0:640:24e9:7873]) by sas1-ec30c78b6c5b.qloud-c.yandex.net (mxbackcorp/Yandex) with ESMTP id ZCAZORnlv2-6HYuEtTD; Mon, 15 Mar 2021 09:06:19 +0300 Received: from dynamic-vpn.dhcp.yndx.net (dynamic-vpn.dhcp.yndx.net [2a02:6b8:b081:23::1:15]) by sas1-24e978739efd.qloud-c.yandex.net (smtpcorp/Yandex) with ESMTPSA id FoAWGMWkZt-6Hn0AYWB; Mon, 15 Mar 2021 09:06:17 +0300 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) (Client certificate not present) Precedence: bulk DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yandex-team.ru; s=default; t=1615788379; bh=c9G60La+2Sf+4noHvdEmbbjc208DGWTp5aj4kVCB4wo=; h=In-Reply-To:Message-Id:References:Date:Subject:To:From:Cc; b=uwtMkGCnGnDh/o73vkummk2WXcGrSPZALMjkpMJwKWXNnZzOG/fid9PzhTBlGmqWZ N3XOn0yzk8e5i3zPPSswuQlPLtCXcQRsXrZYX+stqespWl/h1OXDz2pqB+ZXzgzxes axzRSmtlLEiSwrAkFX2NXGev5DD5lshdyV0ZR704= Authentication-Results: sas1-ec30c78b6c5b.qloud-c.yandex.net; dkim=pass header.i=@yandex-team.ru From: Roman Kagan To: qemu-devel@nongnu.org Subject: [PATCH 7/7] block/nbd: stop manipulating in_flight counter Date: Mon, 15 Mar 2021 09:06:11 +0300 Message-Id: <20210315060611.2989049-8-rvkagan@yandex-team.ru> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20210315060611.2989049-1-rvkagan@yandex-team.ru> References: <20210315060611.2989049-1-rvkagan@yandex-team.ru> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=95.108.205.193; envelope-from=rvkagan@yandex-team.ru; helo=forwardcorp1o.mail.yandex.net X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Vladimir Sementsov-Ogievskiy , qemu-block@nongnu.org, Max Reitz , yc-core@yandex-team.ru Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) Content-Type: text/plain; charset="utf-8" As the reconnect logic no longer interferes with drained sections, it appears unnecessary to explicitly manipulate the in_flight counter. Fixes: 5ad81b4946 ("nbd: Restrict connection_co reentrance") Signed-off-by: Roman Kagan --- block/nbd.c | 6 ------ nbd/client.c | 2 -- 2 files changed, 8 deletions(-) diff --git a/block/nbd.c b/block/nbd.c index a5a9e4aca5..3a22f5d897 100644 --- a/block/nbd.c +++ b/block/nbd.c @@ -311,7 +311,6 @@ static void nbd_client_attach_aio_context_bh(void *opaq= ue) if (s->connection_co) { qemu_aio_coroutine_enter(bs->aio_context, s->connection_co); } - bdrv_dec_in_flight(bs); } =20 static void nbd_client_attach_aio_context(BlockDriverState *bs, @@ -327,7 +326,6 @@ static void nbd_client_attach_aio_context(BlockDriverSt= ate *bs, qio_channel_attach_aio_context(QIO_CHANNEL(s->ioc), new_context); } =20 - bdrv_inc_in_flight(bs); =20 /* * Need to wait here for the BH to run because the BH must run while t= he @@ -643,11 +641,9 @@ static coroutine_fn void nbd_reconnect_attempt(BDRVNBD= State *s) goto out; } =20 - bdrv_dec_in_flight(s->bs); =20 ret =3D nbd_client_handshake(s->bs, &local_err); =20 - bdrv_inc_in_flight(s->bs); =20 out: s->connect_status =3D ret; @@ -759,7 +755,6 @@ static coroutine_fn void nbd_connection_entry(void *opa= que) =20 qemu_co_queue_restart_all(&s->free_sema); nbd_recv_coroutines_wake_all(s); - bdrv_dec_in_flight(s->bs); =20 s->connection_co =3D NULL; if (s->ioc) { @@ -2307,7 +2302,6 @@ static int nbd_open(BlockDriverState *bs, QDict *opti= ons, int flags, nbd_init_connect_thread(s); =20 s->connection_co =3D qemu_coroutine_create(nbd_connection_entry, s); - bdrv_inc_in_flight(bs); aio_co_schedule(bdrv_get_aio_context(bs), s->connection_co); =20 return 0; diff --git a/nbd/client.c b/nbd/client.c index 0c2db4bcba..30d5383cb1 100644 --- a/nbd/client.c +++ b/nbd/client.c @@ -1434,9 +1434,7 @@ nbd_read_eof(BlockDriverState *bs, QIOChannel *ioc, v= oid *buffer, size_t size, =20 len =3D qio_channel_readv(ioc, &iov, 1, errp); if (len =3D=3D QIO_CHANNEL_ERR_BLOCK) { - bdrv_dec_in_flight(bs); qio_channel_yield(ioc, G_IO_IN); - bdrv_inc_in_flight(bs); continue; } else if (len < 0) { return -EIO; --=20 2.30.2