From nobody Wed May 8 16:13:14 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1649959451138318.09133399185225; Thu, 14 Apr 2022 11:04:11 -0700 (PDT) Received: from localhost ([::1]:49640 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nf3pB-0004lr-QN for importer@patchew.org; Thu, 14 Apr 2022 14:04:09 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:57824) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nf3jN-0004lJ-Ri for qemu-devel@nongnu.org; Thu, 14 Apr 2022 13:58:09 -0400 Received: from mail-wm1-x333.google.com ([2a00:1450:4864:20::333]:36551) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1nf3jM-0005sA-Ag for qemu-devel@nongnu.org; Thu, 14 Apr 2022 13:58:09 -0400 Received: by mail-wm1-x333.google.com with SMTP id u17-20020a05600c211100b0038eaf4cdaaeso6480640wml.1 for ; Thu, 14 Apr 2022 10:58:07 -0700 (PDT) Received: from avogadro.lan ([2001:b07:6468:f312:c8dd:75d4:99ab:290a]) by smtp.gmail.com with ESMTPSA id v13-20020adfe28d000000b0020375f27a5asm2451254wri.4.2022.04.14.10.58.04 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 14 Apr 2022 10:58:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=sender:from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=3S+twEeplTAQujNKOjHmSIQb0RCwmynjNUyC+p0Vtnk=; b=h5+OXO6imqlhVDI97sPO2ZYvf6DcwQljNcYdxitvX7RR2222ZFabPdRvuAsTjjuNBE WUHSY/KrBVYgdV9LfnkxWsKYJYeSITHBj2cnHjCJAr+br+bgRy/TXpFI0tRrj2UR0uea KQfiqbWFTF3x29poak4PfAWv+JRgA6gFwC3CtU+c1LEs04mMvT66ZnbQzw8RgFOLyi/t 6up9RrwqVQARAOcwwhfID6z6il60A2YAmBHvzE2aZwtUTRZvvW+S6RDx43/2+k/QKP6b hwFmoOqPZ6BP6aD6hBDjilqeYg7V3qM4dYpFiexBazK+6Ahq59AxDUSjfbZbpFIMJs+p RVrw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:sender:from:to:cc:subject:date:message-id :in-reply-to:references:mime-version:content-transfer-encoding; bh=3S+twEeplTAQujNKOjHmSIQb0RCwmynjNUyC+p0Vtnk=; b=4N5lO+5VgP7Bnk+/9bZK/JFA47bDvglD78/ojoRa7aSD9lUdlcneI0axM4qr40RMXU UilJIqokeYZm8nfcB4AGhDzrSUT2hQ3O8sMcVHYj+LeQ5vkrXVKrFvHWEHMt1iKsdccj uMRCYWn/jwHpun5FmNePjHrDP0DKM2t5kVbX4ZvcCkWyX+gbj6KZfTE/IoHgwAVcIVNH HuUtaaFylbqYRmcv6ixYyUxLqYJQ6L/vEIpy9oBzVJUxe+ai+qxUcu8MfPfngQD8+6oi 0oy+h/ceZSicxbI43/RRk55XDJQ6c935xys5jXSTtz6WTVV5/yBUz60XKF0YAo5i9suM 3IjA== X-Gm-Message-State: AOAM531Fz9VqTqpVSlYaJOMdNjqZsweBxRfYmo6/s9h27PV8RbCjLtBY Wmq+6m92ijyiZ7yQSaoVJwP/tRk69VHc1Q== X-Google-Smtp-Source: ABdhPJyYUpXcHDDfXzfMGSeOuHtbeEMOXOTcc8BYpUIM9IDbrnzRAP5sgBwxsUGoC4M9PVwC+BLYrA== X-Received: by 2002:a7b:c841:0:b0:389:864c:e715 with SMTP id c1-20020a7bc841000000b00389864ce715mr4201512wml.72.1649959086860; Thu, 14 Apr 2022 10:58:06 -0700 (PDT) From: Paolo Bonzini To: qemu-devel@nongnu.org Subject: [PATCH v2 for-7.1 1/9] nbd: safeguard against waking up invalid coroutine Date: Thu, 14 Apr 2022 19:57:48 +0200 Message-Id: <20220414175756.671165-2-pbonzini@redhat.com> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220414175756.671165-1-pbonzini@redhat.com> References: <20220414175756.671165-1-pbonzini@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2a00:1450:4864:20::333; envelope-from=paolo.bonzini@gmail.com; helo=mail-wm1-x333.google.com X-Spam_score_int: -14 X-Spam_score: -1.5 X-Spam_bar: - X-Spam_report: (-1.5 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FORGED_FROMDOMAIN=0.249, FREEMAIL_FROM=0.001, HEADER_FROM_DIFFERENT_DOMAINS=0.249, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: v.sementsov-og@mail.ru, eblake@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZM-MESSAGEID: 1649959452587100001 Content-Type: text/plain; charset="utf-8" The .reply_possible field of s->requests is never set to false. This is not a problem as it is only a safeguard to detect protocol errors, but it's sloppy. In fact, the field is actually not necessary at all, because .coroutine is set to NULL in NBD_FOREACH_REPLY_CHUNK after receiving the last chunk. Thus, replace .reply_possible with .coroutine and move the check before deciding the fate of this request. Signed-off-by: Paolo Bonzini Reviewed-by: Eric Blake Reviewed-by: Lukas Straub Reviewed-by: Vladimir Sementsov-Ogievskiy --- block/nbd.c | 12 +++++------- 1 file changed, 5 insertions(+), 7 deletions(-) diff --git a/block/nbd.c b/block/nbd.c index 691d4b05dc..d29bee1122 100644 --- a/block/nbd.c +++ b/block/nbd.c @@ -58,7 +58,6 @@ typedef struct { Coroutine *coroutine; uint64_t offset; /* original offset of the request */ bool receiving; /* sleeping in the yield in nbd_receive_replie= s */ - bool reply_possible; /* reply header not yet received */ } NBDClientRequest; =20 typedef enum NBDClientState { @@ -454,15 +453,15 @@ static coroutine_fn int nbd_receive_replies(BDRVNBDSt= ate *s, uint64_t handle) nbd_channel_error(s, -EINVAL); return -EINVAL; } + ind2 =3D HANDLE_TO_INDEX(s, s->reply.handle); + if (ind2 >=3D MAX_NBD_REQUESTS || !s->requests[ind2].coroutine) { + nbd_channel_error(s, -EINVAL); + return -EINVAL; + } if (s->reply.handle =3D=3D handle) { /* We are done */ return 0; } - ind2 =3D HANDLE_TO_INDEX(s, s->reply.handle); - if (ind2 >=3D MAX_NBD_REQUESTS || !s->requests[ind2].reply_possibl= e) { - nbd_channel_error(s, -EINVAL); - return -EINVAL; - } nbd_recv_coroutine_wake_one(&s->requests[ind2]); } } @@ -505,7 +504,6 @@ static int nbd_co_send_request(BlockDriverState *bs, s->requests[i].coroutine =3D qemu_coroutine_self(); s->requests[i].offset =3D request->from; s->requests[i].receiving =3D false; - s->requests[i].reply_possible =3D true; =20 request->handle =3D INDEX_TO_HANDLE(s, i); =20 --=20 2.35.1 From nobody Wed May 8 16:13:14 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 164995927223793.8244850787845; Thu, 14 Apr 2022 11:01:12 -0700 (PDT) Received: from localhost ([::1]:41226 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nf3mI-0007Sq-9z for importer@patchew.org; Thu, 14 Apr 2022 14:01:10 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:57866) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nf3jR-0004mA-6t for qemu-devel@nongnu.org; Thu, 14 Apr 2022 13:58:13 -0400 Received: from mail-wm1-x333.google.com ([2a00:1450:4864:20::333]:44751) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1nf3jP-0005sm-BC for qemu-devel@nongnu.org; Thu, 14 Apr 2022 13:58:12 -0400 Received: by mail-wm1-x333.google.com with SMTP id m33-20020a05600c3b2100b0038ec0218103so3713645wms.3 for ; Thu, 14 Apr 2022 10:58:10 -0700 (PDT) Received: from avogadro.lan ([2001:b07:6468:f312:c8dd:75d4:99ab:290a]) by smtp.gmail.com with ESMTPSA id v13-20020adfe28d000000b0020375f27a5asm2451254wri.4.2022.04.14.10.58.07 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 14 Apr 2022 10:58:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=sender:from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=31aWf6yWgE2LMOZIgY9XyxQ2VucF93P8DLahNEWZkgE=; b=nJcGmqImpVof1zz39QdESoT25TRUiaJX9UclzLXOXWM11zofV77ihprJNq5eciIh+p Fvy5jfo3H9BGpjoGF+w5lEkcgTixcqgYPpv2d34NIPMoqihCG4OO3oX6clC1rkFVD0k2 2gg3Rztn2j3cVv+A1/wYh+oaEg0OnXmvglKJYSWyXn2KBUGF+z5RFFQGcym8HfgwUdBm pFKvcYp8wYqFOYjtUtjlJMAE4yQW1lrC+rh4Hewp3JMs7/BmenTrfmMOgx4Jw3I6nsav xPaiXBaur/povG6mxODa3Fg4yAPL8Ao8ZhASa8xH7wzcr2Uwq3dF8UhOEomOzjWfwAFS TADA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:sender:from:to:cc:subject:date:message-id :in-reply-to:references:mime-version:content-transfer-encoding; bh=31aWf6yWgE2LMOZIgY9XyxQ2VucF93P8DLahNEWZkgE=; b=BB0j/fTyMAKSwhAjEcbXT27PPWnTKsUOLUzDf/5VvK7kQwBOmtuPUOhF2ixkYui5jO D2PxwQEWpwsORWPPbQRMFMqsBID2DvO7H0ZgXuWlQb3IZWjaKYLpbsT119aSPSpXp1sK mucmbg6g+3qPk5/O+E4XhTgr6ptea14J6AtrSNpiq9tPmoZraGrSnc8GQGTwzmnJxsxI Zo2LcIKrpQySvwRd05r7eSHXc7xC5y9XaeSDu17pLrrgohdqHHQxXpjo1cdWyWc9JEtr Bu1sXYWGjVhBS/qCfWPd8zT9wKiTNFo6F4XB5kAidzfC6fYINOJ4cw0sjwHEZo+MgC8I dNxA== X-Gm-Message-State: AOAM5334cyZvSEVakxBQJxVRGA2qcHilRi/5rJB3wKxlK1lWB0F7Mz+v UUoDpjJP9jZqMdfe0eKBcV+CsOD9mn0+BA== X-Google-Smtp-Source: ABdhPJydklM2cEuOH81dxeS/mluaZ69NLv2hefecCyrp0lxvqvrWYunJli22hmRaTUWpagUp688/6g== X-Received: by 2002:a7b:c341:0:b0:37b:c619:c9f4 with SMTP id l1-20020a7bc341000000b0037bc619c9f4mr4699424wmj.38.1649959090004; Thu, 14 Apr 2022 10:58:10 -0700 (PDT) From: Paolo Bonzini To: qemu-devel@nongnu.org Subject: [PATCH v2 for-7.1 2/9] nbd: mark more coroutine_fns Date: Thu, 14 Apr 2022 19:57:49 +0200 Message-Id: <20220414175756.671165-3-pbonzini@redhat.com> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220414175756.671165-1-pbonzini@redhat.com> References: <20220414175756.671165-1-pbonzini@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2a00:1450:4864:20::333; envelope-from=paolo.bonzini@gmail.com; helo=mail-wm1-x333.google.com X-Spam_score_int: -14 X-Spam_score: -1.5 X-Spam_bar: - X-Spam_report: (-1.5 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FORGED_FROMDOMAIN=0.249, FREEMAIL_FROM=0.001, HEADER_FROM_DIFFERENT_DOMAINS=0.249, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: v.sementsov-og@mail.ru, eblake@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZM-MESSAGEID: 1649959273465100003 Content-Type: text/plain; charset="utf-8" Several coroutine functions in block/nbd.c are not marked as such. This patch adds a few more markers; it is not exhaustive, but it focuses especially on: - places that wake other coroutines, because aio_co_wake() has very different semantics inside a coroutine (queuing after yield vs. entering immediately); - functions with _co_ in their names, to avoid confusion Reviewed-by: Eric Blake Signed-off-by: Paolo Bonzini Reviewed-by: Lukas Straub Reviewed-by: Vladimir Sementsov-Ogievskiy --- block/nbd.c | 64 ++++++++++++++++++++++++++--------------------------- 1 file changed, 32 insertions(+), 32 deletions(-) diff --git a/block/nbd.c b/block/nbd.c index d29bee1122..3cc4ea4430 100644 --- a/block/nbd.c +++ b/block/nbd.c @@ -132,7 +132,7 @@ static bool nbd_client_connected(BDRVNBDState *s) return qatomic_load_acquire(&s->state) =3D=3D NBD_CLIENT_CONNECTED; } =20 -static bool nbd_recv_coroutine_wake_one(NBDClientRequest *req) +static bool coroutine_fn nbd_recv_coroutine_wake_one(NBDClientRequest *req) { if (req->receiving) { req->receiving =3D false; @@ -143,7 +143,7 @@ static bool nbd_recv_coroutine_wake_one(NBDClientReques= t *req) return false; } =20 -static void nbd_recv_coroutines_wake(BDRVNBDState *s, bool all) +static void coroutine_fn nbd_recv_coroutines_wake(BDRVNBDState *s, bool al= l) { int i; =20 @@ -154,7 +154,7 @@ static void nbd_recv_coroutines_wake(BDRVNBDState *s, b= ool all) } } =20 -static void nbd_channel_error(BDRVNBDState *s, int ret) +static void coroutine_fn nbd_channel_error(BDRVNBDState *s, int ret) { if (nbd_client_connected(s)) { qio_channel_shutdown(s->ioc, QIO_CHANNEL_SHUTDOWN_BOTH, NULL); @@ -466,9 +466,9 @@ static coroutine_fn int nbd_receive_replies(BDRVNBDStat= e *s, uint64_t handle) } } =20 -static int nbd_co_send_request(BlockDriverState *bs, - NBDRequest *request, - QEMUIOVector *qiov) +static int coroutine_fn nbd_co_send_request(BlockDriverState *bs, + NBDRequest *request, + QEMUIOVector *qiov) { BDRVNBDState *s =3D (BDRVNBDState *)bs->opaque; int rc, i =3D -1; @@ -721,9 +721,9 @@ static int nbd_parse_error_payload(NBDStructuredReplyCh= unk *chunk, return 0; } =20 -static int nbd_co_receive_offset_data_payload(BDRVNBDState *s, - uint64_t orig_offset, - QEMUIOVector *qiov, Error **= errp) +static int coroutine_fn +nbd_co_receive_offset_data_payload(BDRVNBDState *s, uint64_t orig_offset, + QEMUIOVector *qiov, Error **errp) { QEMUIOVector sub_qiov; uint64_t offset; @@ -1039,8 +1039,8 @@ break_loop: return false; } =20 -static int nbd_co_receive_return_code(BDRVNBDState *s, uint64_t handle, - int *request_ret, Error **errp) +static int coroutine_fn nbd_co_receive_return_code(BDRVNBDState *s, uint64= _t handle, + int *request_ret, Error= **errp) { NBDReplyChunkIter iter; =20 @@ -1053,9 +1053,9 @@ static int nbd_co_receive_return_code(BDRVNBDState *s= , uint64_t handle, return iter.ret; } =20 -static int nbd_co_receive_cmdread_reply(BDRVNBDState *s, uint64_t handle, - uint64_t offset, QEMUIOVector *qio= v, - int *request_ret, Error **errp) +static int coroutine_fn nbd_co_receive_cmdread_reply(BDRVNBDState *s, uint= 64_t handle, + uint64_t offset, QEMU= IOVector *qiov, + int *request_ret, Err= or **errp) { NBDReplyChunkIter iter; NBDReply reply; @@ -1105,10 +1105,10 @@ static int nbd_co_receive_cmdread_reply(BDRVNBDStat= e *s, uint64_t handle, return iter.ret; } =20 -static int nbd_co_receive_blockstatus_reply(BDRVNBDState *s, - uint64_t handle, uint64_t leng= th, - NBDExtent *extent, - int *request_ret, Error **errp) +static int coroutine_fn nbd_co_receive_blockstatus_reply(BDRVNBDState *s, + uint64_t handle, = uint64_t length, + NBDExtent *extent, + int *request_ret,= Error **errp) { NBDReplyChunkIter iter; NBDReply reply; @@ -1165,8 +1165,8 @@ static int nbd_co_receive_blockstatus_reply(BDRVNBDSt= ate *s, return iter.ret; } =20 -static int nbd_co_request(BlockDriverState *bs, NBDRequest *request, - QEMUIOVector *write_qiov) +static int coroutine_fn nbd_co_request(BlockDriverState *bs, NBDRequest *r= equest, + QEMUIOVector *write_qiov) { int ret, request_ret; Error *local_err =3D NULL; @@ -1202,9 +1202,9 @@ static int nbd_co_request(BlockDriverState *bs, NBDRe= quest *request, return ret ? ret : request_ret; } =20 -static int nbd_client_co_preadv(BlockDriverState *bs, int64_t offset, - int64_t bytes, QEMUIOVector *qiov, - BdrvRequestFlags flags) +static int coroutine_fn nbd_client_co_preadv(BlockDriverState *bs, int64_t= offset, + int64_t bytes, QEMUIOVector *= qiov, + BdrvRequestFlags flags) { int ret, request_ret; Error *local_err =3D NULL; @@ -1261,9 +1261,9 @@ static int nbd_client_co_preadv(BlockDriverState *bs,= int64_t offset, return ret ? ret : request_ret; } =20 -static int nbd_client_co_pwritev(BlockDriverState *bs, int64_t offset, - int64_t bytes, QEMUIOVector *qiov, - BdrvRequestFlags flags) +static int coroutine_fn nbd_client_co_pwritev(BlockDriverState *bs, int64_= t offset, + int64_t bytes, QEMUIOVector = *qiov, + BdrvRequestFlags flags) { BDRVNBDState *s =3D (BDRVNBDState *)bs->opaque; NBDRequest request =3D { @@ -1286,8 +1286,8 @@ static int nbd_client_co_pwritev(BlockDriverState *bs= , int64_t offset, return nbd_co_request(bs, &request, qiov); } =20 -static int nbd_client_co_pwrite_zeroes(BlockDriverState *bs, int64_t offse= t, - int64_t bytes, BdrvRequestFlags fla= gs) +static int coroutine_fn nbd_client_co_pwrite_zeroes(BlockDriverState *bs, = int64_t offset, + int64_t bytes, BdrvReq= uestFlags flags) { BDRVNBDState *s =3D (BDRVNBDState *)bs->opaque; NBDRequest request =3D { @@ -1321,7 +1321,7 @@ static int nbd_client_co_pwrite_zeroes(BlockDriverSta= te *bs, int64_t offset, return nbd_co_request(bs, &request, NULL); } =20 -static int nbd_client_co_flush(BlockDriverState *bs) +static int coroutine_fn nbd_client_co_flush(BlockDriverState *bs) { BDRVNBDState *s =3D (BDRVNBDState *)bs->opaque; NBDRequest request =3D { .type =3D NBD_CMD_FLUSH }; @@ -1336,8 +1336,8 @@ static int nbd_client_co_flush(BlockDriverState *bs) return nbd_co_request(bs, &request, NULL); } =20 -static int nbd_client_co_pdiscard(BlockDriverState *bs, int64_t offset, - int64_t bytes) +static int coroutine_fn nbd_client_co_pdiscard(BlockDriverState *bs, int64= _t offset, + int64_t bytes) { BDRVNBDState *s =3D (BDRVNBDState *)bs->opaque; NBDRequest request =3D { @@ -1914,7 +1914,7 @@ fail: return ret; } =20 -static int nbd_co_flush(BlockDriverState *bs) +static int coroutine_fn nbd_co_flush(BlockDriverState *bs) { return nbd_client_co_flush(bs); } --=20 2.35.1 From nobody Wed May 8 16:13:14 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1649959334236327.58673316230477; Thu, 14 Apr 2022 11:02:14 -0700 (PDT) Received: from localhost ([::1]:44222 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nf3nI-0001Ag-RH for importer@patchew.org; Thu, 14 Apr 2022 14:02:12 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:57884) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nf3jS-0004nw-Qt for qemu-devel@nongnu.org; Thu, 14 Apr 2022 13:58:15 -0400 Received: from mail-wm1-x32d.google.com ([2a00:1450:4864:20::32d]:53842) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1nf3jQ-0005t7-Sw for qemu-devel@nongnu.org; Thu, 14 Apr 2022 13:58:14 -0400 Received: by mail-wm1-x32d.google.com with SMTP id p189so3623541wmp.3 for ; Thu, 14 Apr 2022 10:58:12 -0700 (PDT) Received: from avogadro.lan ([2001:b07:6468:f312:c8dd:75d4:99ab:290a]) by smtp.gmail.com with ESMTPSA id v13-20020adfe28d000000b0020375f27a5asm2451254wri.4.2022.04.14.10.58.10 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 14 Apr 2022 10:58:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=sender:from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=gpLSk4k7X7Vp+T/vkrmkU3CU0GY2UodPZOw3JDLE7ec=; b=TTvSXbIn4t1hACT0vTjbHk4ffhsEoHjp7NdeDZg184zVcGHUkCpUofbaxrUU6VYYCR ilqhCG1I0XVPb+z2LbKECxxdAh8dnOkCp+SIUFxOUK0b4MPLCFbcm1Ys0u2W3ZbTKBjL G9O0mzqTR8vrJrgZWN0YvnMwlSEvGX1aeAs7gpZWgm0GBC36dYBhYfnl+trBGRSNUEG9 Ekzm3DGl59onNm8xq4pqp5dHNIMi2k82qrfZmj/C52qj5BJ1ykcqAIUnOsjZxJWTqQYx Oho3cNuWYyjc6rzOrG+2NxeGdHbLbH5uzwIy8qTm7i0V9IFgVqi42tUpVQT+eQMAgV2w 8S0Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:sender:from:to:cc:subject:date:message-id :in-reply-to:references:mime-version:content-transfer-encoding; bh=gpLSk4k7X7Vp+T/vkrmkU3CU0GY2UodPZOw3JDLE7ec=; b=cFtyLq0DcS0bsxJOI6K8PF+p87IS0q8GAivq8rhL/aURy+DGmSJEYZTAZu++lzBceW WKVTdMqdS9XUb8Jv1ZvSViR6Q9Lskee0pXsPffoRiB0NZ+Jynl7rmmIU4Ex71nbDrwT6 uVSbWqq/bbmwiyVf9co/47xcdw74j4J6WVS6gMEBQwWwPJo3XCfe4bqov+WrL9bZpl7t 1Aw8AuISz44Rnkp9GKzRptJyLhEMUP4hIMmEpWHlJis5Me0Ttg9k2NIejBj1enBRx65G tCSpHh4mJlK2uHZgl5HQqlpYLP7Dp26y7Xbn0qlpPrBzjdHr3E1vuijsTrWnqVNBZ2WT 7hEQ== X-Gm-Message-State: AOAM532iHAKBPHL9DRcL9vcb7Hfs3Q4S+uwmPORgQCbSnZWkPOZ9fi+e MfILRGpDRuWC4K+LEjWFPmq2ZdiWQdd9rA== X-Google-Smtp-Source: ABdhPJxodH6Rdi6SWaJLmaPLMa2PncOGtUaNh/wjP83nlJv53JesDs3wag1qjLFjFSPjy3PWu4rJaA== X-Received: by 2002:a7b:c5d5:0:b0:37b:fda9:a5e2 with SMTP id n21-20020a7bc5d5000000b0037bfda9a5e2mr4266242wmk.62.1649959091633; Thu, 14 Apr 2022 10:58:11 -0700 (PDT) From: Paolo Bonzini To: qemu-devel@nongnu.org Subject: [PATCH v2 for-7.1 3/9] nbd: remove peppering of nbd_client_connected Date: Thu, 14 Apr 2022 19:57:50 +0200 Message-Id: <20220414175756.671165-4-pbonzini@redhat.com> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220414175756.671165-1-pbonzini@redhat.com> References: <20220414175756.671165-1-pbonzini@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2a00:1450:4864:20::32d; envelope-from=paolo.bonzini@gmail.com; helo=mail-wm1-x32d.google.com X-Spam_score_int: -14 X-Spam_score: -1.5 X-Spam_bar: - X-Spam_report: (-1.5 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FORGED_FROMDOMAIN=0.249, FREEMAIL_FROM=0.001, HEADER_FROM_DIFFERENT_DOMAINS=0.249, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: v.sementsov-og@mail.ru, eblake@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZM-MESSAGEID: 1649959335653100001 Content-Type: text/plain; charset="utf-8" It is unnecessary to check nbd_client_connected() because every time s->state is moved out of NBD_CLIENT_CONNECTED the socket is shut down and all coroutines are resumed. The only case where it was actually needed is when the NBD server disconnects and there is no reconnect-delay. In that case, nbd_receive_replies() does not set s->reply.handle and nbd_co_do_receive_one_chunk() cannot continue. For that one case, check the return value of nbd_receive_replies(). As to the others: * nbd_receive_replies() can put the current coroutine to sleep if another reply is ongoing; then it will be woken by nbd_channel_error(), called by the ongoing reply. Or it can try itself to read a reply header and fail, thus calling nbd_channel_error() itself. * nbd_co_send_request() will write the body of the request and fail * nbd_reply_chunk_iter_receive() will call nbd_co_receive_one_chunk() and then nbd_co_do_receive_one_chunk(), which will handle the failure as above; or it will just detect a previous call to nbd_iter_channel_error() via iter->ret < 0. Reviewed-by: Eric Blake Signed-off-by: Paolo Bonzini Reviewed-by: Lukas Straub Reviewed-by: Vladimir Sementsov-Ogievskiy --- block/nbd.c | 17 ++++------------- 1 file changed, 4 insertions(+), 13 deletions(-) diff --git a/block/nbd.c b/block/nbd.c index 3cc4ea4430..a30603ce87 100644 --- a/block/nbd.c +++ b/block/nbd.c @@ -409,10 +409,6 @@ static coroutine_fn int nbd_receive_replies(BDRVNBDSta= te *s, uint64_t handle) return 0; } =20 - if (!nbd_client_connected(s)) { - return -EIO; - } - if (s->reply.handle !=3D 0) { /* * Some other request is being handled now. It should already = be @@ -512,7 +508,7 @@ static int coroutine_fn nbd_co_send_request(BlockDriver= State *bs, if (qiov) { qio_channel_set_cork(s->ioc, true); rc =3D nbd_send_request(s->ioc, request); - if (nbd_client_connected(s) && rc >=3D 0) { + if (rc >=3D 0) { if (qio_channel_writev_all(s->ioc, qiov->iov, qiov->niov, NULL) < 0) { rc =3D -EIO; @@ -829,8 +825,8 @@ static coroutine_fn int nbd_co_do_receive_one_chunk( } *request_ret =3D 0; =20 - nbd_receive_replies(s, handle); - if (!nbd_client_connected(s)) { + ret =3D nbd_receive_replies(s, handle); + if (ret < 0) { error_setg(errp, "Connection closed"); return -EIO; } @@ -982,11 +978,6 @@ static bool nbd_reply_chunk_iter_receive(BDRVNBDState = *s, NBDReply local_reply; NBDStructuredReplyChunk *chunk; Error *local_err =3D NULL; - if (!nbd_client_connected(s)) { - error_setg(&local_err, "Connection closed"); - nbd_iter_channel_error(iter, -EIO, &local_err); - goto break_loop; - } =20 if (iter->done) { /* Previous iteration was last. */ @@ -1007,7 +998,7 @@ static bool nbd_reply_chunk_iter_receive(BDRVNBDState = *s, } =20 /* Do not execute the body of NBD_FOREACH_REPLY_CHUNK for simple reply= . */ - if (nbd_reply_is_simple(reply) || !nbd_client_connected(s)) { + if (nbd_reply_is_simple(reply) || iter->ret < 0) { goto break_loop; } =20 --=20 2.35.1 From nobody Wed May 8 16:13:14 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1649959275581165.02073174908116; Thu, 14 Apr 2022 11:01:15 -0700 (PDT) Received: from localhost ([::1]:41478 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nf3mM-0007do-In for importer@patchew.org; Thu, 14 Apr 2022 14:01:14 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:57910) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nf3jU-0004q4-Sy for qemu-devel@nongnu.org; Thu, 14 Apr 2022 13:58:16 -0400 Received: from mail-wr1-x42d.google.com ([2a00:1450:4864:20::42d]:42972) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1nf3jS-0005tI-MJ for qemu-devel@nongnu.org; Thu, 14 Apr 2022 13:58:16 -0400 Received: by mail-wr1-x42d.google.com with SMTP id r13so7946245wrr.9 for ; Thu, 14 Apr 2022 10:58:13 -0700 (PDT) Received: from avogadro.lan ([2001:b07:6468:f312:c8dd:75d4:99ab:290a]) by smtp.gmail.com with ESMTPSA id v13-20020adfe28d000000b0020375f27a5asm2451254wri.4.2022.04.14.10.58.11 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 14 Apr 2022 10:58:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=sender:from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=Jup+XP1pUtySZld8xVq3nH/ymKtZ9nvbp33B8oZgGn0=; b=aweaT99AWr7QkrXRbgxZlPYM2lWTJxgm+ED3cMXTsGlpQcmz8rbv58swGHjAS9h7l+ r0Tpj2YaJTfadeiXyueKm3R4KTE8e5uYD9J1qcBEkEfIqMwZ1LvMlcPuor/I0/QzlJZV 2B5CBU1k/IwFQBK76vMc/p052ggK5xOCBJGaVXg32/YxtlTN1ySs3DUFpcwmQOEAS52c I8SgDyI24p354f1sneD+uw66an7GRI/lPRIXluMICJSN5SNeP4aq405y5WQvtzA/wjwf WD2GIO+yNnPuKgc3+3scnvVAOruNAWlJbs4AqOgBCXN3b0va8mXNRm0WFtCNQx+wivqI Z5Cg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:sender:from:to:cc:subject:date:message-id :in-reply-to:references:mime-version:content-transfer-encoding; bh=Jup+XP1pUtySZld8xVq3nH/ymKtZ9nvbp33B8oZgGn0=; b=Ie0qZvnhDq7N44S347hFoW6dhnYxEzcZm5YUNlKP97tByUMMGZBvWaxYwWqSAvvjxq bg5MI6AHR4eaRYV0UUN582F6fC6m7o+L6s811Yu/4gP/5TR8c31mhZO/XNL3Tc7WA9Ap rt6cuxPrt0nwNGiqABU49eFWzpsPqSnItJDuhp1s8TtTz+olzu5iptdzamJtb51SsGUn F/PtdecKfTY/ehgC6E9g64bqRxuCjI1fC++MjZAaZlqYMAUYYIc8zqglYqGcKiK/HMB8 o1kAf5YTRvBS2o97JpbQBwxVnofAPn/oP4VwcDEV5UjChbn3iNYrGi36gl/6GViJZ6Z2 FXvg== X-Gm-Message-State: AOAM530+Mca5h5IebqBSpgZKanDux8OgQGk63v4NZXtQ9XMP9VvlE3Xu zThIsqvuwYehamNmqFqcLIZdkl7OC5uUXQ== X-Google-Smtp-Source: ABdhPJwpImjl3QVNLPPRb5AEDa8hQrUFveSwmVGJOMFjSez7+7Blox1w3ACG1rQcnFnajaBLNlRvrg== X-Received: by 2002:a5d:4702:0:b0:206:b8d:3dce with SMTP id y2-20020a5d4702000000b002060b8d3dcemr2991619wrq.646.1649959092845; Thu, 14 Apr 2022 10:58:12 -0700 (PDT) From: Paolo Bonzini To: qemu-devel@nongnu.org Subject: [PATCH v2 for-7.1 4/9] nbd: keep send_mutex/free_sema handling outside nbd_co_do_establish_connection Date: Thu, 14 Apr 2022 19:57:51 +0200 Message-Id: <20220414175756.671165-5-pbonzini@redhat.com> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220414175756.671165-1-pbonzini@redhat.com> References: <20220414175756.671165-1-pbonzini@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2a00:1450:4864:20::42d; envelope-from=paolo.bonzini@gmail.com; helo=mail-wr1-x42d.google.com X-Spam_score_int: -14 X-Spam_score: -1.5 X-Spam_bar: - X-Spam_report: (-1.5 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FORGED_FROMDOMAIN=0.249, FREEMAIL_FROM=0.001, HEADER_FROM_DIFFERENT_DOMAINS=0.249, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: v.sementsov-og@mail.ru, eblake@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZM-MESSAGEID: 1649959277467100001 Content-Type: text/plain; charset="utf-8" Elevate s->in_flight early so that other incoming requests will wait on the CoQueue in nbd_co_send_request; restart them after getting back from nbd_reconnect_attempt. This could be after the reconnect timer or nbd_cancel_in_flight have cancelled the attempt, so there is no need anymore to cancel the requests there. nbd_co_send_request now handles both stopping and restarting pending requests after a successful connection, and there is no need to hold send_mutex in nbd_co_do_establish_connection. The current setup is confusing because nbd_co_do_establish_connection is called both with send_mutex taken and without it. Before the patch it uses free_sema which (at least in theory...) is protected by send_mutex, after the patch it does not anymore. Signed-off-by: Paolo Bonzini Reviewed-by: Eric Blake Reviewed-by: Lukas Straub Reviewed-by: Vladimir Sementsov-Ogievskiy --- block/coroutines.h | 4 +-- block/nbd.c | 66 ++++++++++++++++++++++------------------------ 2 files changed, 33 insertions(+), 37 deletions(-) diff --git a/block/coroutines.h b/block/coroutines.h index b293e943c8..8f6e438ef3 100644 --- a/block/coroutines.h +++ b/block/coroutines.h @@ -59,7 +59,7 @@ int coroutine_fn bdrv_co_writev_vmstate(BlockDriverState = *bs, QEMUIOVector *qiov, int64_t pos); =20 int coroutine_fn -nbd_co_do_establish_connection(BlockDriverState *bs, Error **errp); +nbd_co_do_establish_connection(BlockDriverState *bs, bool blocking, Error = **errp); =20 =20 int coroutine_fn @@ -109,7 +109,7 @@ bdrv_common_block_status_above(BlockDriverState *bs, BlockDriverState **file, int *depth); int generated_co_wrapper -nbd_do_establish_connection(BlockDriverState *bs, Error **errp); +nbd_do_establish_connection(BlockDriverState *bs, bool blocking, Error **e= rrp); =20 int generated_co_wrapper blk_do_preadv(BlockBackend *blk, int64_t offset, int64_t bytes, diff --git a/block/nbd.c b/block/nbd.c index a30603ce87..62dd338ef3 100644 --- a/block/nbd.c +++ b/block/nbd.c @@ -187,9 +187,6 @@ static void reconnect_delay_timer_cb(void *opaque) if (qatomic_load_acquire(&s->state) =3D=3D NBD_CLIENT_CONNECTING_WAIT)= { s->state =3D NBD_CLIENT_CONNECTING_NOWAIT; nbd_co_establish_connection_cancel(s->conn); - while (qemu_co_enter_next(&s->free_sema, NULL)) { - /* Resume all queued requests */ - } } =20 reconnect_delay_timer_del(s); @@ -310,11 +307,10 @@ static int nbd_handle_updated_info(BlockDriverState *= bs, Error **errp) } =20 int coroutine_fn nbd_co_do_establish_connection(BlockDriverState *bs, - Error **errp) + bool blocking, Error **err= p) { BDRVNBDState *s =3D (BDRVNBDState *)bs->opaque; int ret; - bool blocking =3D nbd_client_connecting_wait(s); IO_CODE(); =20 assert(!s->ioc); @@ -350,7 +346,6 @@ int coroutine_fn nbd_co_do_establish_connection(BlockDr= iverState *bs, =20 /* successfully connected */ s->state =3D NBD_CLIENT_CONNECTED; - qemu_co_queue_restart_all(&s->free_sema); =20 return 0; } @@ -358,25 +353,25 @@ int coroutine_fn nbd_co_do_establish_connection(Block= DriverState *bs, /* called under s->send_mutex */ static coroutine_fn void nbd_reconnect_attempt(BDRVNBDState *s) { - assert(nbd_client_connecting(s)); - assert(s->in_flight =3D=3D 0); - - if (nbd_client_connecting_wait(s) && s->reconnect_delay && - !s->reconnect_delay_timer) - { - /* - * It's first reconnect attempt after switching to - * NBD_CLIENT_CONNECTING_WAIT - */ - reconnect_delay_timer_init(s, - qemu_clock_get_ns(QEMU_CLOCK_REALTIME) + - s->reconnect_delay * NANOSECONDS_PER_SECOND); - } + bool blocking =3D nbd_client_connecting_wait(s); =20 /* * Now we are sure that nobody is accessing the channel, and no one wi= ll * try until we set the state to CONNECTED. */ + assert(nbd_client_connecting(s)); + assert(s->in_flight =3D=3D 1); + + if (blocking && !s->reconnect_delay_timer) { + /* + * It's the first reconnect attempt after switching to + * NBD_CLIENT_CONNECTING_WAIT + */ + g_assert(s->reconnect_delay); + reconnect_delay_timer_init(s, + qemu_clock_get_ns(QEMU_CLOCK_REALTIME) + + s->reconnect_delay * NANOSECONDS_PER_SECOND); + } =20 /* Finalize previous connection if any */ if (s->ioc) { @@ -387,7 +382,9 @@ static coroutine_fn void nbd_reconnect_attempt(BDRVNBDS= tate *s) s->ioc =3D NULL; } =20 - nbd_co_do_establish_connection(s->bs, NULL); + qemu_co_mutex_unlock(&s->send_mutex); + nbd_co_do_establish_connection(s->bs, blocking, NULL); + qemu_co_mutex_lock(&s->send_mutex); =20 /* * The reconnect attempt is done (maybe successfully, maybe not), so @@ -472,21 +469,21 @@ static int coroutine_fn nbd_co_send_request(BlockDriv= erState *bs, qemu_co_mutex_lock(&s->send_mutex); =20 while (s->in_flight =3D=3D MAX_NBD_REQUESTS || - (!nbd_client_connected(s) && s->in_flight > 0)) - { + (!nbd_client_connected(s) && s->in_flight > 0)) { qemu_co_queue_wait(&s->free_sema, &s->send_mutex); } =20 - if (nbd_client_connecting(s)) { - nbd_reconnect_attempt(s); - } - - if (!nbd_client_connected(s)) { - rc =3D -EIO; - goto err; - } - s->in_flight++; + if (!nbd_client_connected(s)) { + if (nbd_client_connecting(s)) { + nbd_reconnect_attempt(s); + qemu_co_queue_restart_all(&s->free_sema); + } + if (!nbd_client_connected(s)) { + rc =3D -EIO; + goto err; + } + } =20 for (i =3D 0; i < MAX_NBD_REQUESTS; i++) { if (s->requests[i].coroutine =3D=3D NULL) { @@ -526,8 +523,8 @@ err: nbd_channel_error(s, rc); if (i !=3D -1) { s->requests[i].coroutine =3D NULL; - s->in_flight--; } + s->in_flight--; qemu_co_queue_next(&s->free_sema); } qemu_co_mutex_unlock(&s->send_mutex); @@ -1883,7 +1880,7 @@ static int nbd_open(BlockDriverState *bs, QDict *opti= ons, int flags, } =20 s->state =3D NBD_CLIENT_CONNECTING_WAIT; - ret =3D nbd_do_establish_connection(bs, errp); + ret =3D nbd_do_establish_connection(bs, true, errp); if (ret < 0) { goto fail; } @@ -2049,7 +2046,6 @@ static void nbd_cancel_in_flight(BlockDriverState *bs) =20 if (s->state =3D=3D NBD_CLIENT_CONNECTING_WAIT) { s->state =3D NBD_CLIENT_CONNECTING_NOWAIT; - qemu_co_queue_restart_all(&s->free_sema); } =20 nbd_co_establish_connection_cancel(s->conn); --=20 2.35.1 From nobody Wed May 8 16:13:14 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1649959460039956.238962864421; Thu, 14 Apr 2022 11:04:20 -0700 (PDT) Received: from localhost ([::1]:49942 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nf3pL-0004y9-01 for importer@patchew.org; Thu, 14 Apr 2022 14:04:19 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:57924) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nf3jV-0004rU-Mz for qemu-devel@nongnu.org; Thu, 14 Apr 2022 13:58:21 -0400 Received: from mail-wr1-x431.google.com ([2a00:1450:4864:20::431]:42976) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1nf3jT-0005ui-UR for qemu-devel@nongnu.org; Thu, 14 Apr 2022 13:58:17 -0400 Received: by mail-wr1-x431.google.com with SMTP id r13so7946351wrr.9 for ; Thu, 14 Apr 2022 10:58:15 -0700 (PDT) Received: from avogadro.lan ([2001:b07:6468:f312:c8dd:75d4:99ab:290a]) by smtp.gmail.com with ESMTPSA id v13-20020adfe28d000000b0020375f27a5asm2451254wri.4.2022.04.14.10.58.12 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 14 Apr 2022 10:58:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=sender:from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=qPiVlhtFX1uAH/GGISul1eoCjcJOFjzCsMQY+NhZ6Bs=; b=QI9IbrVeGwnT7x3mfvfGYCO8ytk7fa4F1VUplWWuHaCdJ4/d7fuRrFh0Mj7oc6L41g /pTdA4p0Lut3zTc3yqsZK2VtuRkW/czbGoUrZvoPapr+3QA2j389ARLv4MGjchVQGzeK 3eUfVcMVfNiR+pG4+3S+N0ZKYik+ok8vVu49wi0n4aZ9GZXJnvOYcb/wbMXVEYC/uUqT pagjB5kuYiK0Hl7T85pa3iukzoOf3vuS+MBXmgsw70qQ6yzj2rjvMTo4j1dj/6rIUuCG nkukrmrCTfF6HeM4t81nIUaU30A7/vjp9L5sL8M2unJ7nPVE36D4vUN0OmLFW+YUAhW+ oxrQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:sender:from:to:cc:subject:date:message-id :in-reply-to:references:mime-version:content-transfer-encoding; bh=qPiVlhtFX1uAH/GGISul1eoCjcJOFjzCsMQY+NhZ6Bs=; b=iux2hRjqhTsLaHrljXrPRiY6jetxq+Ttur9gCBAqsLzqcCaaJ/C92qfkPU15NRqczK DxAIQGkv6yMHDJUFNcEYhZ5yjkfUIHijSZQcN2zZipInxEQ241tV/ztIS6+G7FNA9mR8 v64Q890TaTnSuclymt6Wz8jAf+08DWwzF7njFk/K5NXH+0225slXy9KP9UvT+QeZsJmi 4jGzhI32mlAByF/eHFsdhCZhKvwb8SFVpqp9JlSaaVFDmVZ88gRH7mGwLSkD3QMHePa3 xloajlBmqe6nYyiuf4HAK5sPWZsGP21IJrUvnaaavgymDO2E0RwzJxWMuXi8g8W6pK0F 0TFg== X-Gm-Message-State: AOAM5326x1LMzPV8DEgo12Gg9BhjChyHlaCfHes9ALW/WVuMPuvUsoVG zE6t6jsdSnxI5pUx2Uxpio49K37BCntzoQ== X-Google-Smtp-Source: ABdhPJyxxLVjNVtqxY9yclQby1ifvA91k07t0fRJ47LlyWvXBK2Wetehl6iVcAbqHb7b2qrPGlc/aw== X-Received: by 2002:a05:6000:18a2:b0:203:d2f5:28a0 with SMTP id b2-20020a05600018a200b00203d2f528a0mr3008909wri.355.1649959094515; Thu, 14 Apr 2022 10:58:14 -0700 (PDT) From: Paolo Bonzini To: qemu-devel@nongnu.org Subject: [PATCH v2 for-7.1 5/9] nbd: use a QemuMutex to synchronize yanking, reconnection and coroutines Date: Thu, 14 Apr 2022 19:57:52 +0200 Message-Id: <20220414175756.671165-6-pbonzini@redhat.com> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220414175756.671165-1-pbonzini@redhat.com> References: <20220414175756.671165-1-pbonzini@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2a00:1450:4864:20::431; envelope-from=paolo.bonzini@gmail.com; helo=mail-wr1-x431.google.com X-Spam_score_int: -14 X-Spam_score: -1.5 X-Spam_bar: - X-Spam_report: (-1.5 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FORGED_FROMDOMAIN=0.249, FREEMAIL_FROM=0.001, HEADER_FROM_DIFFERENT_DOMAINS=0.249, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: v.sementsov-og@mail.ru, eblake@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZM-MESSAGEID: 1649959460407100001 Content-Type: text/plain; charset="utf-8" The condition for waiting on the s->free_sema queue depends on both s->in_flight and s->state. The latter is currently using atomics, but this is quite dubious and probably wrong. Because s->state is written in the main thread too, for example by the yank callback, it cannot be protected by a CoMutex. Introduce a separate lock that can be used by nbd_co_send_request(); later on this lock will also be used for s->state. There will not be any contention on the lock unless there is a yank or reconnect, so this is not performance sensitive. Signed-off-by: Paolo Bonzini Reviewed-by: Eric Blake Reviewed-by: Lukas Straub --- block/nbd.c | 46 +++++++++++++++++++++++++++------------------- 1 file changed, 27 insertions(+), 19 deletions(-) diff --git a/block/nbd.c b/block/nbd.c index 62dd338ef3..a2414566d1 100644 --- a/block/nbd.c +++ b/block/nbd.c @@ -71,17 +71,22 @@ typedef struct BDRVNBDState { QIOChannel *ioc; /* The current I/O channel */ NBDExportInfo info; =20 - CoMutex send_mutex; + /* + * Protects free_sema, in_flight, requests[].coroutine, + * reconnect_delay_timer. + */ + QemuMutex requests_lock; CoQueue free_sema; - - CoMutex receive_mutex; int in_flight; + NBDClientRequest requests[MAX_NBD_REQUESTS]; + QEMUTimer *reconnect_delay_timer; + + CoMutex send_mutex; + CoMutex receive_mutex; NBDClientState state; =20 - QEMUTimer *reconnect_delay_timer; QEMUTimer *open_timer; =20 - NBDClientRequest requests[MAX_NBD_REQUESTS]; NBDReply reply; BlockDriverState *bs; =20 @@ -350,7 +355,7 @@ int coroutine_fn nbd_co_do_establish_connection(BlockDr= iverState *bs, return 0; } =20 -/* called under s->send_mutex */ +/* Called with s->requests_lock taken. */ static coroutine_fn void nbd_reconnect_attempt(BDRVNBDState *s) { bool blocking =3D nbd_client_connecting_wait(s); @@ -382,9 +387,9 @@ static coroutine_fn void nbd_reconnect_attempt(BDRVNBDS= tate *s) s->ioc =3D NULL; } =20 - qemu_co_mutex_unlock(&s->send_mutex); + qemu_mutex_unlock(&s->requests_lock); nbd_co_do_establish_connection(s->bs, blocking, NULL); - qemu_co_mutex_lock(&s->send_mutex); + qemu_mutex_lock(&s->requests_lock); =20 /* * The reconnect attempt is done (maybe successfully, maybe not), so @@ -466,11 +471,10 @@ static int coroutine_fn nbd_co_send_request(BlockDriv= erState *bs, BDRVNBDState *s =3D (BDRVNBDState *)bs->opaque; int rc, i =3D -1; =20 - qemu_co_mutex_lock(&s->send_mutex); - + qemu_mutex_lock(&s->requests_lock); while (s->in_flight =3D=3D MAX_NBD_REQUESTS || (!nbd_client_connected(s) && s->in_flight > 0)) { - qemu_co_queue_wait(&s->free_sema, &s->send_mutex); + qemu_co_queue_wait(&s->free_sema, &s->requests_lock); } =20 s->in_flight++; @@ -491,13 +495,13 @@ static int coroutine_fn nbd_co_send_request(BlockDriv= erState *bs, } } =20 - g_assert(qemu_in_coroutine()); assert(i < MAX_NBD_REQUESTS); - s->requests[i].coroutine =3D qemu_coroutine_self(); s->requests[i].offset =3D request->from; s->requests[i].receiving =3D false; + qemu_mutex_unlock(&s->requests_lock); =20 + qemu_co_mutex_lock(&s->send_mutex); request->handle =3D INDEX_TO_HANDLE(s, i); =20 assert(s->ioc); @@ -517,17 +521,19 @@ static int coroutine_fn nbd_co_send_request(BlockDriv= erState *bs, } else { rc =3D nbd_send_request(s->ioc, request); } + qemu_co_mutex_unlock(&s->send_mutex); =20 -err: if (rc < 0) { + qemu_mutex_lock(&s->requests_lock); +err: nbd_channel_error(s, rc); if (i !=3D -1) { s->requests[i].coroutine =3D NULL; } s->in_flight--; qemu_co_queue_next(&s->free_sema); + qemu_mutex_unlock(&s->requests_lock); } - qemu_co_mutex_unlock(&s->send_mutex); return rc; } =20 @@ -1017,12 +1023,11 @@ static bool nbd_reply_chunk_iter_receive(BDRVNBDSta= te *s, return true; =20 break_loop: + qemu_mutex_lock(&s->requests_lock); s->requests[HANDLE_TO_INDEX(s, handle)].coroutine =3D NULL; - - qemu_co_mutex_lock(&s->send_mutex); s->in_flight--; qemu_co_queue_next(&s->free_sema); - qemu_co_mutex_unlock(&s->send_mutex); + qemu_mutex_unlock(&s->requests_lock); =20 return false; } @@ -1855,8 +1860,9 @@ static int nbd_open(BlockDriverState *bs, QDict *opti= ons, int flags, BDRVNBDState *s =3D (BDRVNBDState *)bs->opaque; =20 s->bs =3D bs; - qemu_co_mutex_init(&s->send_mutex); + qemu_mutex_init(&s->requests_lock); qemu_co_queue_init(&s->free_sema); + qemu_co_mutex_init(&s->send_mutex); qemu_co_mutex_init(&s->receive_mutex); =20 if (!yank_register_instance(BLOCKDEV_YANK_INSTANCE(bs->node_name), err= p)) { @@ -2044,9 +2050,11 @@ static void nbd_cancel_in_flight(BlockDriverState *b= s) =20 reconnect_delay_timer_del(s); =20 + qemu_mutex_lock(&s->requests_lock); if (s->state =3D=3D NBD_CLIENT_CONNECTING_WAIT) { s->state =3D NBD_CLIENT_CONNECTING_NOWAIT; } + qemu_mutex_unlock(&s->requests_lock); =20 nbd_co_establish_connection_cancel(s->conn); } --=20 2.35.1 From nobody Wed May 8 16:13:14 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1649959496286775.2915398559015; Thu, 14 Apr 2022 11:04:56 -0700 (PDT) Received: from localhost ([::1]:52692 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nf3pv-0006md-Ah for importer@patchew.org; Thu, 14 Apr 2022 14:04:55 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:57950) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nf3jX-0004rZ-9p for qemu-devel@nongnu.org; Thu, 14 Apr 2022 13:58:21 -0400 Received: from mail-wr1-x42f.google.com ([2a00:1450:4864:20::42f]:40453) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1nf3jV-0005us-Eu for qemu-devel@nongnu.org; Thu, 14 Apr 2022 13:58:19 -0400 Received: by mail-wr1-x42f.google.com with SMTP id q3so7330665wrj.7 for ; Thu, 14 Apr 2022 10:58:17 -0700 (PDT) Received: from avogadro.lan ([2001:b07:6468:f312:c8dd:75d4:99ab:290a]) by smtp.gmail.com with ESMTPSA id v13-20020adfe28d000000b0020375f27a5asm2451254wri.4.2022.04.14.10.58.14 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 14 Apr 2022 10:58:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=sender:from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=LNVKWTMVIgClVXVxtfwmoVrEMyCS+5+mebl2pSFl+pA=; b=VuToi6ALkczY1gp8QtMdQidWHRvkmbu2SQZonnmfu2wXese6JCpCxUynlKiqLT2mPl cGHvyQVQWGceNLbkvGdZzAlwcR3fnHiFzh1Bz8CofTVAUAKePqozwWqU1fz/AP2bIPBp C7P3YNIiGYfLDywv8Bcx2435d0gItbHo4Q86Sta0z7rUhAqasI/409PBj2gBZG5fxcyO g+cJepqmeSq3jdBhEMXlylsho7/YtYN+GF7OWdHBGEKJmfnzFsJUPA8sQZ0eTDlzODBV 9uvrP8+p9MAmO4qblijGwRD1hKbSKAXc8zstO49UgCalrwXzPg9TevINlSi04EAm/tsD g5rg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:sender:from:to:cc:subject:date:message-id :in-reply-to:references:mime-version:content-transfer-encoding; bh=LNVKWTMVIgClVXVxtfwmoVrEMyCS+5+mebl2pSFl+pA=; b=hIph2FAoMYZ1vy297mafJUpkuFbNxiE1iGZi+eisdeJeawI4xQHyVbPkRDfk1mx/vw Lce7r2UVsZIRJw9OZ03InSWhwEmP4nde+kFGWKgZpf1k4Q67YHZAQb8NYaUZniuXgl3+ jlmnMNakwK4CY4Zm7OgOUVIlkBbTJDTevGDNZTd3KyxtSE1wwaHP+V7BWTsiv9nlfM5T jfdGFikfgeFf5UupkCf2VTmgmTZbJsisks6kcSe3CZJHiXp9o42ZQe5D9SlNBleJZhsT 9nhP9V+l6B79Vx0f/+erxBOdmD7oBoFI9koGVUbeQE4hjxgXT6BgUUhjN1yCZyHsTfkM kP7Q== X-Gm-Message-State: AOAM530sFjvqiqh9Ae6EuYm61Pl39I/LB9nuv8ezzhs3edE609wQKfVQ rJCk4DzIK6LO1Ps/MoErwXG82gPPrAF2hQ== X-Google-Smtp-Source: ABdhPJx9U+SaS8xxIrGGMD+t1zVChHDpDRgyfHrN2GQP87tlE/LuPvQJSYvAhOFAEdgSaVpaZLOX+A== X-Received: by 2002:a05:6000:1866:b0:204:cce:c89e with SMTP id d6-20020a056000186600b002040ccec89emr3012206wri.601.1649959096111; Thu, 14 Apr 2022 10:58:16 -0700 (PDT) From: Paolo Bonzini To: qemu-devel@nongnu.org Subject: [PATCH v2 for-7.1 6/9] nbd: code motion and function renaming Date: Thu, 14 Apr 2022 19:57:53 +0200 Message-Id: <20220414175756.671165-7-pbonzini@redhat.com> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220414175756.671165-1-pbonzini@redhat.com> References: <20220414175756.671165-1-pbonzini@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2a00:1450:4864:20::42f; envelope-from=paolo.bonzini@gmail.com; helo=mail-wr1-x42f.google.com X-Spam_score_int: -14 X-Spam_score: -1.5 X-Spam_bar: - X-Spam_report: (-1.5 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FORGED_FROMDOMAIN=0.249, FREEMAIL_FROM=0.001, HEADER_FROM_DIFFERENT_DOMAINS=0.249, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: v.sementsov-og@mail.ru, eblake@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZM-MESSAGEID: 1649959499056100001 Content-Type: text/plain; charset="utf-8" Prepare for the next patch, so that the diff is less confusing. nbd_client_connecting is moved closer to the definition point. nbd_client_connecting_wait() is kept only for the reconnection logic; when it is used to check if a request has to be reissued, use the renamed function nbd_client_will_reconnect(). In the next patch, the two cases will have different locking requirements. Signed-off-by: Paolo Bonzini Reviewed-by: Eric Blake Reviewed-by: Lukas Straub Reviewed-by: Vladimir Sementsov-Ogievskiy --- block/nbd.c | 24 ++++++++++++++---------- 1 file changed, 14 insertions(+), 10 deletions(-) diff --git a/block/nbd.c b/block/nbd.c index a2414566d1..37d466e435 100644 --- a/block/nbd.c +++ b/block/nbd.c @@ -254,18 +254,15 @@ static void open_timer_init(BDRVNBDState *s, uint64_t= expire_time_ns) timer_mod(s->open_timer, expire_time_ns); } =20 -static bool nbd_client_connecting(BDRVNBDState *s) -{ - NBDClientState state =3D qatomic_load_acquire(&s->state); - return state =3D=3D NBD_CLIENT_CONNECTING_WAIT || - state =3D=3D NBD_CLIENT_CONNECTING_NOWAIT; -} - static bool nbd_client_connecting_wait(BDRVNBDState *s) { return qatomic_load_acquire(&s->state) =3D=3D NBD_CLIENT_CONNECTING_WA= IT; } =20 +static bool nbd_client_will_reconnect(BDRVNBDState *s) +{ + return qatomic_load_acquire(&s->state) =3D=3D NBD_CLIENT_CONNECTING_WA= IT; +} /* * Update @bs with information learned during a completed negotiation proc= ess. * Return failure if the server's advertised options are incompatible with= the @@ -355,6 +352,13 @@ int coroutine_fn nbd_co_do_establish_connection(BlockD= riverState *bs, return 0; } =20 +static bool nbd_client_connecting(BDRVNBDState *s) +{ + NBDClientState state =3D qatomic_load_acquire(&s->state); + return state =3D=3D NBD_CLIENT_CONNECTING_WAIT || + state =3D=3D NBD_CLIENT_CONNECTING_NOWAIT; +} + /* Called with s->requests_lock taken. */ static coroutine_fn void nbd_reconnect_attempt(BDRVNBDState *s) { @@ -1190,7 +1194,7 @@ static int coroutine_fn nbd_co_request(BlockDriverSta= te *bs, NBDRequest *request error_free(local_err); local_err =3D NULL; } - } while (ret < 0 && nbd_client_connecting_wait(s)); + } while (ret < 0 && nbd_client_will_reconnect(s)); =20 return ret ? ret : request_ret; } @@ -1249,7 +1253,7 @@ static int coroutine_fn nbd_client_co_preadv(BlockDri= verState *bs, int64_t offse error_free(local_err); local_err =3D NULL; } - } while (ret < 0 && nbd_client_connecting_wait(s)); + } while (ret < 0 && nbd_client_will_reconnect(s)); =20 return ret ? ret : request_ret; } @@ -1407,7 +1411,7 @@ static int coroutine_fn nbd_client_co_block_status( error_free(local_err); local_err =3D NULL; } - } while (ret < 0 && nbd_client_connecting_wait(s)); + } while (ret < 0 && nbd_client_will_reconnect(s)); =20 if (ret < 0 || request_ret < 0) { return ret ? ret : request_ret; --=20 2.35.1 From nobody Wed May 8 16:13:14 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1649959632217105.85189243341586; Thu, 14 Apr 2022 11:07:12 -0700 (PDT) Received: from localhost ([::1]:57078 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nf3s6-0001R0-Vo for importer@patchew.org; Thu, 14 Apr 2022 14:07:11 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:57968) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nf3jZ-0004sT-KM for qemu-devel@nongnu.org; Thu, 14 Apr 2022 13:58:21 -0400 Received: from mail-wm1-x335.google.com ([2a00:1450:4864:20::335]:37552) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1nf3jW-0005v3-Ao for qemu-devel@nongnu.org; Thu, 14 Apr 2022 13:58:20 -0400 Received: by mail-wm1-x335.google.com with SMTP id m15-20020a7bca4f000000b0038fdc1394b1so2941781wml.2 for ; Thu, 14 Apr 2022 10:58:17 -0700 (PDT) Received: from avogadro.lan ([2001:b07:6468:f312:c8dd:75d4:99ab:290a]) by smtp.gmail.com with ESMTPSA id v13-20020adfe28d000000b0020375f27a5asm2451254wri.4.2022.04.14.10.58.16 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 14 Apr 2022 10:58:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=sender:from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=wZty6AQ7KKf23gAmGW0itmYi2QEVi2hNhiDWTR9fWh0=; b=Y2qf1oH/NhMNQ+3+bP946o1OBUtqcCXEB9rPKeAS6kWov4cJMClmR03h3SWX2ocM2x xRdzU+xIK4jtKQ7gBtgvs+7Mw0jrnco+I7i3ngF3L9f1+2+y3f/kCipCHCZcaiVdUIat EtsdlMKoINKRskNC2lblB4YPBIpBoPpiJ9KGEXUW3bX/HEL0/jEka4Ov0cye0mHoxKnH QmSuR88ZMg2RA2x8W0LX3248GGr3CRV7Ojgpe6BDDic5NIpO3qpnbbOuFqGP/tu+ym5+ hJ93AJNzm9J9uyLpiQMrpE3TvzFW7PGIFj6dWOSnh7DGjKeW1UVAWMlLNgZmOXU3NReH jJ8Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:sender:from:to:cc:subject:date:message-id :in-reply-to:references:mime-version:content-transfer-encoding; bh=wZty6AQ7KKf23gAmGW0itmYi2QEVi2hNhiDWTR9fWh0=; b=NtrQrpTjOUQv+Jc1uq4GlVWviaKXNFX2/rNayLvT1VWsDSXV6h9yr0vrtZfVOctkT8 8M8aqSQ56vJkjlWbqF1Nczyoss9O1BseVmDDyirz8/r6mDjSF1SQG64G3Ylj4afBbrHx zh4cWjyNx7/cbq1JaTCEzWbxhBmimmRcdsBW4mb0sDbsUr1znK7dCJ7VFHrsfL35rirK Wf4BUYa77x9gGXVj+A1WuOq0e6NnG8IuCruxxc2RvFBDbN8bhHzdUtSshNypcf1WMJTl ajgNARZofoW0iGUvWYtR5JDP7aW8DyQy1V6ckVs0pWRcNc+VhmW2LbFr4qXPnIBh/W8J esCQ== X-Gm-Message-State: AOAM532dUkjDED62Q5dty3nZff+9kvs3gY6sYEmycdsdi3ILzfikT8Y7 O1dcoWBZKcV7dDMYgIW4nBqvx1cAcZkCKw== X-Google-Smtp-Source: ABdhPJw5pmXyln55buSyY5u+6SOhuiQXxhlLURcO4XBq5Gj9wpO5Vb/rh5jTjtwDtK8sl7n0ZP8ELg== X-Received: by 2002:a05:600c:3c9b:b0:38e:4c59:68b9 with SMTP id bg27-20020a05600c3c9b00b0038e4c5968b9mr4633174wmb.105.1649959096947; Thu, 14 Apr 2022 10:58:16 -0700 (PDT) From: Paolo Bonzini To: qemu-devel@nongnu.org Subject: [PATCH v2 for-7.1 7/9] nbd: move s->state under requests_lock Date: Thu, 14 Apr 2022 19:57:54 +0200 Message-Id: <20220414175756.671165-8-pbonzini@redhat.com> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220414175756.671165-1-pbonzini@redhat.com> References: <20220414175756.671165-1-pbonzini@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2a00:1450:4864:20::335; envelope-from=paolo.bonzini@gmail.com; helo=mail-wm1-x335.google.com X-Spam_score_int: -14 X-Spam_score: -1.5 X-Spam_bar: - X-Spam_report: (-1.5 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FORGED_FROMDOMAIN=0.249, FREEMAIL_FROM=0.001, HEADER_FROM_DIFFERENT_DOMAINS=0.249, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: v.sementsov-og@mail.ru, eblake@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZM-MESSAGEID: 1649959632786100003 Content-Type: text/plain; charset="utf-8" Remove the confusing, and most likely wrong, atomics. The only function that used to be somewhat in a hot path was nbd_client_connected(), but it is not anymore after the previous patches. The same logic is used both to check if a request had to be reissued and also in nbd_reconnecting_attempt(). The former cases are outside requests_lock, while nbd_reconnecting_attempt() does have the lock, therefore the two have been separated in the previous commit. nbd_client_will_reconnect() can simply take s->requests_lock, while nbd_reconnecting_attempt() can inline the access now that no complicated atomics are involved. Signed-off-by: Paolo Bonzini Reviewed-by: Eric Blake Reviewed-by: Lukas Straub Reviewed-by: Vladimir Sementsov-Ogievskiy --- block/nbd.c | 78 ++++++++++++++++++++++++++++------------------------- 1 file changed, 41 insertions(+), 37 deletions(-) diff --git a/block/nbd.c b/block/nbd.c index 37d466e435..b5aac2548c 100644 --- a/block/nbd.c +++ b/block/nbd.c @@ -35,7 +35,6 @@ #include "qemu/option.h" #include "qemu/cutils.h" #include "qemu/main-loop.h" -#include "qemu/atomic.h" =20 #include "qapi/qapi-visit-sockets.h" #include "qapi/qmp/qstring.h" @@ -72,10 +71,11 @@ typedef struct BDRVNBDState { NBDExportInfo info; =20 /* - * Protects free_sema, in_flight, requests[].coroutine, + * Protects state, free_sema, in_flight, requests[].coroutine, * reconnect_delay_timer. */ QemuMutex requests_lock; + NBDClientState state; CoQueue free_sema; int in_flight; NBDClientRequest requests[MAX_NBD_REQUESTS]; @@ -83,7 +83,6 @@ typedef struct BDRVNBDState { =20 CoMutex send_mutex; CoMutex receive_mutex; - NBDClientState state; =20 QEMUTimer *open_timer; =20 @@ -132,11 +131,6 @@ static void nbd_clear_bdrvstate(BlockDriverState *bs) s->x_dirty_bitmap =3D NULL; } =20 -static bool nbd_client_connected(BDRVNBDState *s) -{ - return qatomic_load_acquire(&s->state) =3D=3D NBD_CLIENT_CONNECTED; -} - static bool coroutine_fn nbd_recv_coroutine_wake_one(NBDClientRequest *req) { if (req->receiving) { @@ -159,14 +153,15 @@ static void coroutine_fn nbd_recv_coroutines_wake(BDR= VNBDState *s, bool all) } } =20 -static void coroutine_fn nbd_channel_error(BDRVNBDState *s, int ret) +/* Called with s->requests_lock held. */ +static void coroutine_fn nbd_channel_error_locked(BDRVNBDState *s, int ret) { - if (nbd_client_connected(s)) { + if (s->state =3D=3D NBD_CLIENT_CONNECTED) { qio_channel_shutdown(s->ioc, QIO_CHANNEL_SHUTDOWN_BOTH, NULL); } =20 if (ret =3D=3D -EIO) { - if (nbd_client_connected(s)) { + if (s->state =3D=3D NBD_CLIENT_CONNECTED) { s->state =3D s->reconnect_delay ? NBD_CLIENT_CONNECTING_WAIT : NBD_CLIENT_CONNECTING_NOWAIT; } @@ -177,6 +172,12 @@ static void coroutine_fn nbd_channel_error(BDRVNBDStat= e *s, int ret) nbd_recv_coroutines_wake(s, true); } =20 +static void coroutine_fn nbd_channel_error(BDRVNBDState *s, int ret) +{ + QEMU_LOCK_GUARD(&s->requests_lock); + nbd_channel_error_locked(s, ret); +} + static void reconnect_delay_timer_del(BDRVNBDState *s) { if (s->reconnect_delay_timer) { @@ -189,20 +190,18 @@ static void reconnect_delay_timer_cb(void *opaque) { BDRVNBDState *s =3D opaque; =20 - if (qatomic_load_acquire(&s->state) =3D=3D NBD_CLIENT_CONNECTING_WAIT)= { - s->state =3D NBD_CLIENT_CONNECTING_NOWAIT; - nbd_co_establish_connection_cancel(s->conn); - } - reconnect_delay_timer_del(s); + WITH_QEMU_LOCK_GUARD(&s->requests_lock) { + if (s->state !=3D NBD_CLIENT_CONNECTING_WAIT) { + return; + } + s->state =3D NBD_CLIENT_CONNECTING_NOWAIT; + } + nbd_co_establish_connection_cancel(s->conn); } =20 static void reconnect_delay_timer_init(BDRVNBDState *s, uint64_t expire_ti= me_ns) { - if (qatomic_load_acquire(&s->state) !=3D NBD_CLIENT_CONNECTING_WAIT) { - return; - } - assert(!s->reconnect_delay_timer); s->reconnect_delay_timer =3D aio_timer_new(bdrv_get_aio_context(s->bs), QEMU_CLOCK_REALTIME, @@ -225,7 +224,9 @@ static void nbd_teardown_connection(BlockDriverState *b= s) s->ioc =3D NULL; } =20 - s->state =3D NBD_CLIENT_QUIT; + WITH_QEMU_LOCK_GUARD(&s->requests_lock) { + s->state =3D NBD_CLIENT_QUIT; + } } =20 static void open_timer_del(BDRVNBDState *s) @@ -254,15 +255,15 @@ static void open_timer_init(BDRVNBDState *s, uint64_t= expire_time_ns) timer_mod(s->open_timer, expire_time_ns); } =20 -static bool nbd_client_connecting_wait(BDRVNBDState *s) -{ - return qatomic_load_acquire(&s->state) =3D=3D NBD_CLIENT_CONNECTING_WA= IT; -} - static bool nbd_client_will_reconnect(BDRVNBDState *s) { - return qatomic_load_acquire(&s->state) =3D=3D NBD_CLIENT_CONNECTING_WA= IT; + /* + * Called only after a socket error, so this is not performance sensit= ive. + */ + QEMU_LOCK_GUARD(&s->requests_lock); + return s->state =3D=3D NBD_CLIENT_CONNECTING_WAIT; } + /* * Update @bs with information learned during a completed negotiation proc= ess. * Return failure if the server's advertised options are incompatible with= the @@ -347,22 +348,24 @@ int coroutine_fn nbd_co_do_establish_connection(Block= DriverState *bs, qio_channel_attach_aio_context(s->ioc, bdrv_get_aio_context(bs)); =20 /* successfully connected */ - s->state =3D NBD_CLIENT_CONNECTED; + WITH_QEMU_LOCK_GUARD(&s->requests_lock) { + s->state =3D NBD_CLIENT_CONNECTED; + } =20 return 0; } =20 +/* Called with s->requests_lock held. */ static bool nbd_client_connecting(BDRVNBDState *s) { - NBDClientState state =3D qatomic_load_acquire(&s->state); - return state =3D=3D NBD_CLIENT_CONNECTING_WAIT || - state =3D=3D NBD_CLIENT_CONNECTING_NOWAIT; + return s->state =3D=3D NBD_CLIENT_CONNECTING_WAIT || + s->state =3D=3D NBD_CLIENT_CONNECTING_NOWAIT; } =20 /* Called with s->requests_lock taken. */ static coroutine_fn void nbd_reconnect_attempt(BDRVNBDState *s) { - bool blocking =3D nbd_client_connecting_wait(s); + bool blocking =3D s->state =3D=3D NBD_CLIENT_CONNECTING_WAIT; =20 /* * Now we are sure that nobody is accessing the channel, and no one wi= ll @@ -477,17 +480,17 @@ static int coroutine_fn nbd_co_send_request(BlockDriv= erState *bs, =20 qemu_mutex_lock(&s->requests_lock); while (s->in_flight =3D=3D MAX_NBD_REQUESTS || - (!nbd_client_connected(s) && s->in_flight > 0)) { + (s->state !=3D NBD_CLIENT_CONNECTED && s->in_flight > 0)) { qemu_co_queue_wait(&s->free_sema, &s->requests_lock); } =20 s->in_flight++; - if (!nbd_client_connected(s)) { + if (s->state !=3D NBD_CLIENT_CONNECTED) { if (nbd_client_connecting(s)) { nbd_reconnect_attempt(s); qemu_co_queue_restart_all(&s->free_sema); } - if (!nbd_client_connected(s)) { + if (s->state !=3D NBD_CLIENT_CONNECTED) { rc =3D -EIO; goto err; } @@ -530,7 +533,7 @@ static int coroutine_fn nbd_co_send_request(BlockDriver= State *bs, if (rc < 0) { qemu_mutex_lock(&s->requests_lock); err: - nbd_channel_error(s, rc); + nbd_channel_error_locked(s, rc); if (i !=3D -1) { s->requests[i].coroutine =3D NULL; } @@ -1443,8 +1446,9 @@ static void nbd_yank(void *opaque) BlockDriverState *bs =3D opaque; BDRVNBDState *s =3D (BDRVNBDState *)bs->opaque; =20 - qatomic_store_release(&s->state, NBD_CLIENT_QUIT); + QEMU_LOCK_GUARD(&s->requests_lock); qio_channel_shutdown(QIO_CHANNEL(s->ioc), QIO_CHANNEL_SHUTDOWN_BOTH, N= ULL); + s->state =3D NBD_CLIENT_QUIT; } =20 static void nbd_client_close(BlockDriverState *bs) --=20 2.35.1 From nobody Wed May 8 16:13:14 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 164995963203242.0054114015777; Thu, 14 Apr 2022 11:07:12 -0700 (PDT) Received: from localhost ([::1]:57066 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nf3s6-0001Qb-EM for importer@patchew.org; Thu, 14 Apr 2022 14:07:10 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:57990) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nf3jb-0004wn-0y for qemu-devel@nongnu.org; Thu, 14 Apr 2022 13:58:23 -0400 Received: from mail-wr1-x434.google.com ([2a00:1450:4864:20::434]:45626) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1nf3jZ-0005vE-4w for qemu-devel@nongnu.org; Thu, 14 Apr 2022 13:58:22 -0400 Received: by mail-wr1-x434.google.com with SMTP id w4so7937811wrg.12 for ; Thu, 14 Apr 2022 10:58:18 -0700 (PDT) Received: from avogadro.lan ([2001:b07:6468:f312:c8dd:75d4:99ab:290a]) by smtp.gmail.com with ESMTPSA id v13-20020adfe28d000000b0020375f27a5asm2451254wri.4.2022.04.14.10.58.17 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 14 Apr 2022 10:58:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=sender:from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=7VWZa+u+p1aPNoE4gcn0kQQDiic91hEbzJAbjLmhctI=; b=EAJ84Wfl2LUOlvzFlzrq//9etArbex02Qech/gXNMsHNRwwsgZ0t6xWehnE8lXsK+7 RSv7XexicBJFbZxHVjLmjnMJroFFtfG0drul8XlbVtM1BtKMBnBplIn/ZwTLHtUVqM5B 9SdKIMOgR7SPAB0JAJGdkLVGeAHhmWgpaxuvs+ks15B9pdRikkSnEMfMUopy07AWeRBD 69XopA38DXQQm6pjnPHHkwsDRV3jJFV/xZwnpSosdS+Mlrs0oIz8rgm/qOcf5Ruu5UFK q35eJFQYCvhRS3Pa1mW9Mxd8PKWxXcxkzy0/h3XN267AtBpOBk+lkYWEqxlvDLf6Vokr 1QdQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:sender:from:to:cc:subject:date:message-id :in-reply-to:references:mime-version:content-transfer-encoding; bh=7VWZa+u+p1aPNoE4gcn0kQQDiic91hEbzJAbjLmhctI=; b=0R8m9pXKFwcQlrHdJBYW1hVfmjSsbi48QzWWnOey4I6F4JwmeMpyoubkrogJWZdErH 0ejxaO3F2O3lUMRbYu/UultldPiUayirUsD9gxdyyT+U4MLRDFEn5rKnxfrh8jwaBmiR OJQVNyeNy+mGUmFH0KBDxej7AVeC9H5in9O3dWqJB61iGql/yxxbWFpNxhBCce3lPtSG sUUl9oC49ayKNJYldbWRvltjJKwZQSHQsoVhcrpvj/ybtrw4qmFpRZmC+jHoDe4GXWcX D2sbYwvEiEYbwQ3DWqFOkxftv8+3XxKCtq2Y6yBzzFMZuafafej2ivuT7jlSJL1vXvUq 0Rzw== X-Gm-Message-State: AOAM532EDBSusrDBO3zvyEwkZo/j3uo2JG3VPsP6B5A/u60sgMTkNZ7D iJMEzppmFZldExSuwkS3dk93HR5eJa0fSQ== X-Google-Smtp-Source: ABdhPJxcTIZrJQZ7SnA6tzq5FmZHCw3qwDHzt39SHfsPK7w69wgU7YNLoRb4Ajw7j/N2HqV8DQ73Dw== X-Received: by 2002:a5d:514c:0:b0:207:bb13:7b23 with SMTP id u12-20020a5d514c000000b00207bb137b23mr2895679wrt.119.1649959098065; Thu, 14 Apr 2022 10:58:18 -0700 (PDT) From: Paolo Bonzini To: qemu-devel@nongnu.org Subject: [PATCH v2 for-7.1 8/9] nbd: take receive_mutex when reading requests[].receiving Date: Thu, 14 Apr 2022 19:57:55 +0200 Message-Id: <20220414175756.671165-9-pbonzini@redhat.com> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220414175756.671165-1-pbonzini@redhat.com> References: <20220414175756.671165-1-pbonzini@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2a00:1450:4864:20::434; envelope-from=paolo.bonzini@gmail.com; helo=mail-wr1-x434.google.com X-Spam_score_int: -14 X-Spam_score: -1.5 X-Spam_bar: - X-Spam_report: (-1.5 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FORGED_FROMDOMAIN=0.249, FREEMAIL_FROM=0.001, HEADER_FROM_DIFFERENT_DOMAINS=0.249, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: v.sementsov-og@mail.ru, eblake@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZM-MESSAGEID: 1649959632636100001 Content-Type: text/plain; charset="utf-8" requests[].receiving is set by nbd_receive_replies() under the receive_mute= x; Read it under the same mutex as well. Waking up receivers on errors happens after each reply finishes processing, in nbd_co_receive_one_chunk(). If there is no currently-active reply, there are two cases: * either there is no active request at all, in which case no element of request[] can have .receiving =3D true * or nbd_receive_replies() must be running and owns receive_mutex; in that case it will get back to nbd_co_receive_one_chunk() because the socket has been shutdown, and all waiting coroutines will wake up in turn. Signed-off-by: Paolo Bonzini Reviewed-by: Eric Blake Reviewed-by: Lukas Straub Reviewed-by: Vladimir Sementsov-Ogievskiy --- block/nbd.c | 15 +++++++-------- 1 file changed, 7 insertions(+), 8 deletions(-) diff --git a/block/nbd.c b/block/nbd.c index b5aac2548c..31c684772e 100644 --- a/block/nbd.c +++ b/block/nbd.c @@ -131,6 +131,7 @@ static void nbd_clear_bdrvstate(BlockDriverState *bs) s->x_dirty_bitmap =3D NULL; } =20 +/* Called with s->receive_mutex taken. */ static bool coroutine_fn nbd_recv_coroutine_wake_one(NBDClientRequest *req) { if (req->receiving) { @@ -142,12 +143,13 @@ static bool coroutine_fn nbd_recv_coroutine_wake_one(= NBDClientRequest *req) return false; } =20 -static void coroutine_fn nbd_recv_coroutines_wake(BDRVNBDState *s, bool al= l) +static void coroutine_fn nbd_recv_coroutines_wake(BDRVNBDState *s) { int i; =20 + QEMU_LOCK_GUARD(&s->receive_mutex); for (i =3D 0; i < MAX_NBD_REQUESTS; i++) { - if (nbd_recv_coroutine_wake_one(&s->requests[i]) && !all) { + if (nbd_recv_coroutine_wake_one(&s->requests[i])) { return; } } @@ -168,8 +170,6 @@ static void coroutine_fn nbd_channel_error_locked(BDRVN= BDState *s, int ret) } else { s->state =3D NBD_CLIENT_QUIT; } - - nbd_recv_coroutines_wake(s, true); } =20 static void coroutine_fn nbd_channel_error(BDRVNBDState *s, int ret) @@ -432,11 +432,10 @@ static coroutine_fn int nbd_receive_replies(BDRVNBDSt= ate *s, uint64_t handle) =20 qemu_coroutine_yield(); /* - * We may be woken for 3 reasons: + * We may be woken for 2 reasons: * 1. From this function, executing in parallel coroutine, whe= n our * handle is received. - * 2. From nbd_channel_error(), when connection is lost. - * 3. From nbd_co_receive_one_chunk(), when previous request is + * 2. From nbd_co_receive_one_chunk(), when previous request is * finished and s->reply.handle set to 0. * Anyway, it's OK to lock the mutex and go to the next iterat= ion. */ @@ -928,7 +927,7 @@ static coroutine_fn int nbd_co_receive_one_chunk( } s->reply.handle =3D 0; =20 - nbd_recv_coroutines_wake(s, false); + nbd_recv_coroutines_wake(s); =20 return ret; } --=20 2.35.1 From nobody Wed May 8 16:13:14 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1649959452381267.9203114931139; Thu, 14 Apr 2022 11:04:12 -0700 (PDT) Received: from localhost ([::1]:49688 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nf3pC-0004nt-U6 for importer@patchew.org; Thu, 14 Apr 2022 14:04:11 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:57986) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nf3ja-0004w5-Ri for qemu-devel@nongnu.org; Thu, 14 Apr 2022 13:58:22 -0400 Received: from mail-wm1-x336.google.com ([2a00:1450:4864:20::336]:38650) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1nf3jZ-0005vO-7V for qemu-devel@nongnu.org; Thu, 14 Apr 2022 13:58:22 -0400 Received: by mail-wm1-x336.google.com with SMTP id i82-20020a1c3b55000000b0038ccb70e239so1833187wma.3 for ; Thu, 14 Apr 2022 10:58:20 -0700 (PDT) Received: from avogadro.lan ([2001:b07:6468:f312:c8dd:75d4:99ab:290a]) by smtp.gmail.com with ESMTPSA id v13-20020adfe28d000000b0020375f27a5asm2451254wri.4.2022.04.14.10.58.18 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 14 Apr 2022 10:58:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=sender:from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=Wvxm2qlPk8qzLFbxsjQC/sIFBmKh+P0XO3nR8MKrs5Y=; b=SlSaXdEMpp7cFkSkQVwL5X3JWWtL8hlsOymvBQBB/+kicq/JVdVITBAKxjkxcwQTJw F8WO5EhRnOqii4zfuWNXbnmGYilbP5e3K07sg+OXNSapQ1l5MQS4eJQpGy08rupqO4x6 2vCIrqujcGiCC+wVXWfjRIl8f9LghKD7iP6bJGyhg+MkL1zk2y+Oj07jWQXbrP2d5Dxc aG2NbIatTlWQOp3xrn3zLEDGk9ooenqlBIvhpgCBHuT+UAr5WBBfV35T0ElelMgtnNfR h7Ux4YHMEr5kHmXpQSg0f1ewlTxqZMAudqnSpSYe4XiDZrxvW8g09vVUf4DbN5jvwL72 5LcA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:sender:from:to:cc:subject:date:message-id :in-reply-to:references:mime-version:content-transfer-encoding; bh=Wvxm2qlPk8qzLFbxsjQC/sIFBmKh+P0XO3nR8MKrs5Y=; b=W35eE4IHeDypdV+AFNeFOJR+u36ayWsVQ67mPF5lysAddpHMgrbJy29IryBFGkAhmS z0IFlokeENZrRzJocDBiN8SivjCiQEqfCQ0kCamg172X9Xub4MxfZPdecUOkn+mY/fHM W7NTCjD/aaDlsRRO320iQVMJKbtK/luWd9ZB5oJKhDmDRA2An0XVGLps2wbDsptPTwH5 dnhf5Dvitm+Wjnd15Vhx4TSeDdu49Pcn4zvOkq4KoN0ThKtxd7cZ7EaRkHCcZ+1jAW+4 ZcMFOuwtXBj1CjF/EKkqa8gdQiqfYlH7aTw3LponUTj8PQFiALRpVfH2tG7/YuvMgX1I QOhw== X-Gm-Message-State: AOAM532MoB727N+i1hBxYEhwjbHdiPT5WldBz3Jdyz+HvK/gJrlKdLjc E/iFtYVRWyFYzOiSlEiMSDZdgV07dVpxCw== X-Google-Smtp-Source: ABdhPJxf9YpR9+VlHWbhBhytsDULzkTwlavtY3/H1F9Y+HAiCqzYz6HTINvxImHBG4zH4lvwtERfDw== X-Received: by 2002:a1c:3584:0:b0:38e:b8ba:181e with SMTP id c126-20020a1c3584000000b0038eb8ba181emr4218362wma.181.1649959099290; Thu, 14 Apr 2022 10:58:19 -0700 (PDT) From: Paolo Bonzini To: qemu-devel@nongnu.org Subject: [PATCH v2 for-7.1 9/9] nbd: document what is protected by the CoMutexes Date: Thu, 14 Apr 2022 19:57:56 +0200 Message-Id: <20220414175756.671165-10-pbonzini@redhat.com> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220414175756.671165-1-pbonzini@redhat.com> References: <20220414175756.671165-1-pbonzini@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2a00:1450:4864:20::336; envelope-from=paolo.bonzini@gmail.com; helo=mail-wm1-x336.google.com X-Spam_score_int: -14 X-Spam_score: -1.5 X-Spam_bar: - X-Spam_report: (-1.5 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FORGED_FROMDOMAIN=0.249, FREEMAIL_FROM=0.001, HEADER_FROM_DIFFERENT_DOMAINS=0.249, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: v.sementsov-og@mail.ru, eblake@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZM-MESSAGEID: 1649959454370100003 Content-Type: text/plain; charset="utf-8" Signed-off-by: Paolo Bonzini Reviewed-by: Eric Blake Reviewed-by: Lukas Straub --- block/nbd.c | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/block/nbd.c b/block/nbd.c index 31c684772e..d0d94b40bd 100644 --- a/block/nbd.c +++ b/block/nbd.c @@ -81,12 +81,18 @@ typedef struct BDRVNBDState { NBDClientRequest requests[MAX_NBD_REQUESTS]; QEMUTimer *reconnect_delay_timer; =20 + /* Protects sending data on the socket. */ CoMutex send_mutex; + + /* + * Protects receiving reply headers from the socket, as well as the + * fields reply and requests[].receiving + */ CoMutex receive_mutex; + NBDReply reply; =20 QEMUTimer *open_timer; =20 - NBDReply reply; BlockDriverState *bs; =20 /* Connection parameters */ --=20 2.35.1