From nobody Mon Nov 10 08:11:35 2025 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=virtuozzo.com ARC-Seal: i=1; a=rsa-sha256; t=1555003761; cv=none; d=zoho.com; s=zohoarc; b=ZndVjf3IvhbMPygMc6FjY4yiiful2RoE7L8xhzwC/tSmeLRCzRz4WYM8xHKbwh3Oe87ywnPshx0WfWbxsxv4eoZSE18iylh9ke6u007Jq0F2Eb3Do+ofyi3EaixAliR8CqU5gCuD52LxcfOJb3MFCTov/5PP9jZpt+rI0Ua/y/0= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1555003761; h=Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results; bh=658Vxf35KUWNVSHeJ5Vk99xVI64/ErWrKn5x3aXAqQs=; b=BheQok0ZBBlVSEG7F8fs8B/XcX8cYPtT/dRjq6BbdbH3y8InYqKzbM+X9hLPnAqanZKCEh5o8Vw40pG/Ceidhyv3RPdvTNOizScO4Qt3XvNo8ZPliCUGmeGDyHv2y1Fd7RbRK0Cz7pijlRIcruHRFskRZD2G67cVi10nImBL4WU= ARC-Authentication-Results: i=1; mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (209.51.188.17 [209.51.188.17]) by mx.zohomail.com with SMTPS id 1555003761213434.71067692274437; Thu, 11 Apr 2019 10:29:21 -0700 (PDT) Received: from localhost ([127.0.0.1]:52547 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hEdVf-0005iw-Kr for importer@patchew.org; Thu, 11 Apr 2019 13:29:11 -0400 Received: from eggs.gnu.org ([209.51.188.92]:50415) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hEdTm-0004ZF-Nu for qemu-devel@nongnu.org; Thu, 11 Apr 2019 13:27:15 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hEdTl-0003bQ-Mk for qemu-devel@nongnu.org; Thu, 11 Apr 2019 13:27:14 -0400 Received: from relay.sw.ru ([185.231.240.75]:59364) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1hEdTl-0003Zz-CA; Thu, 11 Apr 2019 13:27:13 -0400 Received: from [10.28.8.145] (helo=kvm.sw.ru) by relay.sw.ru with esmtp (Exim 4.91) (envelope-from ) id 1hEdTi-00044D-Af; Thu, 11 Apr 2019 20:27:10 +0300 From: Vladimir Sementsov-Ogievskiy To: qemu-devel@nongnu.org, qemu-block@nongnu.org Date: Thu, 11 Apr 2019 20:27:03 +0300 Message-Id: <20190411172709.205032-2-vsementsov@virtuozzo.com> X-Mailer: git-send-email 2.18.0 In-Reply-To: <20190411172709.205032-1-vsementsov@virtuozzo.com> References: <20190411172709.205032-1-vsementsov@virtuozzo.com> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x X-Received-From: 185.231.240.75 Subject: [Qemu-devel] [PATCH v6 1/7] block/nbd-client: split connection_co start out of nbd_client_connect X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, vsementsov@virtuozzo.com, armbru@redhat.com, mreitz@redhat.com, stefanha@redhat.com, den@openvz.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" nbd_client_connect is going to be used from connection_co, so, let's refactor nbd_client_connect in advance, leaving io channel configuration all in nbd_client_connect. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Eric Blake --- block/nbd-client.c | 20 +++++++++++++------- 1 file changed, 13 insertions(+), 7 deletions(-) diff --git a/block/nbd-client.c b/block/nbd-client.c index 790ecc1ee1..facf8f8099 100644 --- a/block/nbd-client.c +++ b/block/nbd-client.c @@ -1180,12 +1180,8 @@ static int nbd_client_connect(BlockDriverState *bs, object_ref(OBJECT(client->ioc)); } =20 - /* Now that we're connected, set the socket to be non-blocking and - * kick the reply mechanism. */ qio_channel_set_blocking(QIO_CHANNEL(sioc), false, NULL); - client->connection_co =3D qemu_coroutine_create(nbd_connection_entry, = client); - bdrv_inc_in_flight(bs); - nbd_client_attach_aio_context(bs, bdrv_get_aio_context(bs)); + qio_channel_attach_aio_context(client->ioc, bdrv_get_aio_context(bs)); =20 logout("Established connection with NBD server\n"); return 0; @@ -1215,12 +1211,22 @@ int nbd_client_init(BlockDriverState *bs, const char *x_dirty_bitmap, Error **errp) { + int ret; NBDClientSession *client =3D nbd_get_client_session(bs); =20 client->bs =3D bs; qemu_co_mutex_init(&client->send_mutex); qemu_co_queue_init(&client->free_sema); =20 - return nbd_client_connect(bs, saddr, export, tlscreds, hostname, - x_dirty_bitmap, errp); + ret =3D nbd_client_connect(bs, saddr, export, tlscreds, hostname, + x_dirty_bitmap, errp); + if (ret < 0) { + return ret; + } + + client->connection_co =3D qemu_coroutine_create(nbd_connection_entry, = client); + bdrv_inc_in_flight(bs); + aio_co_schedule(bdrv_get_aio_context(bs), client->connection_co); + + return 0; } --=20 2.18.0 From nobody Mon Nov 10 08:11:35 2025 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=virtuozzo.com ARC-Seal: i=1; a=rsa-sha256; t=1555004050; cv=none; d=zoho.com; s=zohoarc; b=Wwf3EeWv3ND/4Lqy9/BQQ8AYHXt+9GjzckdnZ3BRVAMtp7OwvY3tjfd49f10vN97k040f8uG+X+WvLBSPQjzu3to1QSX8/Nz1IgHHJhb2/OCDQ9oOjgGhjP4l5BzQsttKIZKY3FBOcJZqs9vuFMxM84/uRj4QGxXX1gpFYLApOU= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1555004050; h=Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results; bh=SDXLQe+/HexX8MykGjYoutTYgYJVe70EfE9DjpZedfU=; b=G7C9TnN2rBaCLbSO1+ZK3vi2M0LtjXZhk6Rd81Vi87xgF7LoGcGiuk3ziaPgQ8Y8mbvA9uHn/LNxx6zf83+4Npd92pjg8dCVKi7129KCir91FFhI3EBlvhyWn1Nwasfvtl5hPI+dGHVxJgrRxAgM/2ilo/TT3LfhM/V2iYyDjTA= ARC-Authentication-Results: i=1; mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1555004050247559.1368032419463; Thu, 11 Apr 2019 10:34:10 -0700 (PDT) Received: from localhost ([127.0.0.1]:52625 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hEdaP-0001PO-8q for importer@patchew.org; Thu, 11 Apr 2019 13:34:05 -0400 Received: from eggs.gnu.org ([209.51.188.92]:50440) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hEdTn-0004ZK-Bq for qemu-devel@nongnu.org; Thu, 11 Apr 2019 13:27:16 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hEdTl-0003bh-Rf for qemu-devel@nongnu.org; Thu, 11 Apr 2019 13:27:15 -0400 Received: from relay.sw.ru ([185.231.240.75]:59354) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1hEdTl-0003Zt-DG; Thu, 11 Apr 2019 13:27:13 -0400 Received: from [10.28.8.145] (helo=kvm.sw.ru) by relay.sw.ru with esmtp (Exim 4.91) (envelope-from ) id 1hEdTi-00044D-ER; Thu, 11 Apr 2019 20:27:10 +0300 From: Vladimir Sementsov-Ogievskiy To: qemu-devel@nongnu.org, qemu-block@nongnu.org Date: Thu, 11 Apr 2019 20:27:04 +0300 Message-Id: <20190411172709.205032-3-vsementsov@virtuozzo.com> X-Mailer: git-send-email 2.18.0 In-Reply-To: <20190411172709.205032-1-vsementsov@virtuozzo.com> References: <20190411172709.205032-1-vsementsov@virtuozzo.com> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x X-Received-From: 185.231.240.75 Subject: [Qemu-devel] [PATCH v6 2/7] block/nbd-client: use non-blocking io channel for nbd negotiation X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, vsementsov@virtuozzo.com, armbru@redhat.com, mreitz@redhat.com, stefanha@redhat.com, den@openvz.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" No reason to use blocking channel for negotiation and we'll benefit in further reconnect feature, as qio_channel reads and writes will do qemu_coroutine_yield while waiting for io completion. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Eric Blake --- include/block/nbd.h | 3 ++- block/nbd-client.c | 16 +++++++--------- nbd/client.c | 16 +++++++++++----- qemu-nbd.c | 2 +- 4 files changed, 21 insertions(+), 16 deletions(-) diff --git a/include/block/nbd.h b/include/block/nbd.h index 6d05983a55..6961835654 100644 --- a/include/block/nbd.h +++ b/include/block/nbd.h @@ -290,7 +290,8 @@ struct NBDExportInfo { }; typedef struct NBDExportInfo NBDExportInfo; =20 -int nbd_receive_negotiate(QIOChannel *ioc, QCryptoTLSCreds *tlscreds, +int nbd_receive_negotiate(AioContext *aio_context, QIOChannel *ioc, + QCryptoTLSCreds *tlscreds, const char *hostname, QIOChannel **outioc, NBDExportInfo *info, Error **errp); void nbd_free_export_list(NBDExportInfo *info, int count); diff --git a/block/nbd-client.c b/block/nbd-client.c index facf8f8099..f9cee25865 100644 --- a/block/nbd-client.c +++ b/block/nbd-client.c @@ -1123,6 +1123,7 @@ static int nbd_client_connect(BlockDriverState *bs, Error **errp) { NBDClientSession *client =3D nbd_get_client_session(bs); + AioContext *aio_context =3D bdrv_get_aio_context(bs); int ret; =20 /* @@ -1137,15 +1138,16 @@ static int nbd_client_connect(BlockDriverState *bs, =20 /* NBD handshake */ logout("session init %s\n", export); - qio_channel_set_blocking(QIO_CHANNEL(sioc), true, NULL); + qio_channel_set_blocking(QIO_CHANNEL(sioc), false, NULL); + qio_channel_attach_aio_context(QIO_CHANNEL(sioc), aio_context); =20 client->info.request_sizes =3D true; client->info.structured_reply =3D true; client->info.base_allocation =3D true; client->info.x_dirty_bitmap =3D g_strdup(x_dirty_bitmap); client->info.name =3D g_strdup(export ?: ""); - ret =3D nbd_receive_negotiate(QIO_CHANNEL(sioc), tlscreds, hostname, - &client->ioc, &client->info, errp); + ret =3D nbd_receive_negotiate(aio_context, QIO_CHANNEL(sioc), tlscreds, + hostname, &client->ioc, &client->info, err= p); g_free(client->info.x_dirty_bitmap); g_free(client->info.name); if (ret < 0) { @@ -1180,17 +1182,13 @@ static int nbd_client_connect(BlockDriverState *bs, object_ref(OBJECT(client->ioc)); } =20 - qio_channel_set_blocking(QIO_CHANNEL(sioc), false, NULL); - qio_channel_attach_aio_context(client->ioc, bdrv_get_aio_context(bs)); - logout("Established connection with NBD server\n"); return 0; =20 fail: /* - * We have connected, but must fail for other reasons. The - * connection is still blocking; send NBD_CMD_DISC as a courtesy - * to the server. + * We have connected, but must fail for other reasons. + * Send NBD_CMD_DISC as a courtesy to the server. */ { NBDRequest request =3D { .type =3D NBD_CMD_DISC }; diff --git a/nbd/client.c b/nbd/client.c index 4de30630c7..8f524c3e35 100644 --- a/nbd/client.c +++ b/nbd/client.c @@ -867,7 +867,8 @@ static int nbd_list_meta_contexts(QIOChannel *ioc, * 2: server is newstyle, but lacks structured replies * 3: server is newstyle and set up for structured replies */ -static int nbd_start_negotiate(QIOChannel *ioc, QCryptoTLSCreds *tlscreds, +static int nbd_start_negotiate(AioContext *aio_context, QIOChannel *ioc, + QCryptoTLSCreds *tlscreds, const char *hostname, QIOChannel **outioc, bool structured_reply, bool *zeroes, Error **errp) @@ -934,6 +935,10 @@ static int nbd_start_negotiate(QIOChannel *ioc, QCrypt= oTLSCreds *tlscreds, return -EINVAL; } ioc =3D *outioc; + if (aio_context) { + qio_channel_set_blocking(ioc, false, NULL); + qio_channel_attach_aio_context(ioc, aio_context); + } } else { error_setg(errp, "Server does not support STARTTLS"); return -EINVAL; @@ -998,7 +1003,8 @@ static int nbd_negotiate_finish_oldstyle(QIOChannel *i= oc, NBDExportInfo *info, * Returns: negative errno: failure talking to server * 0: server is connected */ -int nbd_receive_negotiate(QIOChannel *ioc, QCryptoTLSCreds *tlscreds, +int nbd_receive_negotiate(AioContext *aio_context, QIOChannel *ioc, + QCryptoTLSCreds *tlscreds, const char *hostname, QIOChannel **outioc, NBDExportInfo *info, Error **errp) { @@ -1009,7 +1015,7 @@ int nbd_receive_negotiate(QIOChannel *ioc, QCryptoTLS= Creds *tlscreds, assert(info->name); trace_nbd_receive_negotiate_name(info->name); =20 - result =3D nbd_start_negotiate(ioc, tlscreds, hostname, outioc, + result =3D nbd_start_negotiate(aio_context, ioc, tlscreds, hostname, o= utioc, info->structured_reply, &zeroes, errp); =20 info->structured_reply =3D false; @@ -1129,8 +1135,8 @@ int nbd_receive_export_list(QIOChannel *ioc, QCryptoT= LSCreds *tlscreds, QIOChannel *sioc =3D NULL; =20 *info =3D NULL; - result =3D nbd_start_negotiate(ioc, tlscreds, hostname, &sioc, true, N= ULL, - errp); + result =3D nbd_start_negotiate(NULL, ioc, tlscreds, hostname, &sioc, t= rue, + NULL, errp); if (tlscreds && sioc) { ioc =3D sioc; } diff --git a/qemu-nbd.c b/qemu-nbd.c index 941ba729c2..ed38b55428 100644 --- a/qemu-nbd.c +++ b/qemu-nbd.c @@ -370,7 +370,7 @@ static void *nbd_client_thread(void *arg) goto out; } =20 - ret =3D nbd_receive_negotiate(QIO_CHANNEL(sioc), + ret =3D nbd_receive_negotiate(NULL, QIO_CHANNEL(sioc), NULL, NULL, NULL, &info, &local_error); if (ret < 0) { if (local_error) { --=20 2.18.0 From nobody Mon Nov 10 08:11:35 2025 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=virtuozzo.com ARC-Seal: i=1; a=rsa-sha256; t=1555003940; cv=none; d=zoho.com; s=zohoarc; b=jP6pCAsPkwv+G60bAUnWEpe1Z5QjZrasvRxreM1V+I75DBg0uORf6BLL/nq4WRg1fNw3i/8kP+FiCJbIGSHTOtkhYJlgsK1v4qdcALkVsyEpDRR/k7TLQTAb2/jsZ5Yzs+E8s3jMcE6zgjTgXoEO4F41iMKiH70EiaIA609Itpo= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1555003940; h=Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results; bh=RlxvnXfKIt/O7DdvQbSJajQeUCDkIg3dUAR9pV0Krqo=; b=fc78IMzazdEwKdSXp2B2k/g3F8afD6g0CKZcr+x+yzC3hUoiOuK5II8UckD16CKdxucxEmQRaej8DN8cC0EiK+9NRayDmOzU/MbLBuo0IvEneLW3nxwl3aH6dWyk8Ob4/RCJthqzs7glyT2SYl7adsaqH8Z0bElo9f+6/9TttFE= ARC-Authentication-Results: i=1; mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 155500394096137.53500785752033; Thu, 11 Apr 2019 10:32:20 -0700 (PDT) Received: from localhost ([127.0.0.1]:52607 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hEdYf-0008F1-SC for importer@patchew.org; Thu, 11 Apr 2019 13:32:17 -0400 Received: from eggs.gnu.org ([209.51.188.92]:50438) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hEdTn-0004ZJ-BR for qemu-devel@nongnu.org; Thu, 11 Apr 2019 13:27:16 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hEdTl-0003bX-OH for qemu-devel@nongnu.org; Thu, 11 Apr 2019 13:27:15 -0400 Received: from relay.sw.ru ([185.231.240.75]:59350) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1hEdTl-0003Zu-D5; Thu, 11 Apr 2019 13:27:13 -0400 Received: from [10.28.8.145] (helo=kvm.sw.ru) by relay.sw.ru with esmtp (Exim 4.91) (envelope-from ) id 1hEdTi-00044D-L0; Thu, 11 Apr 2019 20:27:10 +0300 From: Vladimir Sementsov-Ogievskiy To: qemu-devel@nongnu.org, qemu-block@nongnu.org Date: Thu, 11 Apr 2019 20:27:05 +0300 Message-Id: <20190411172709.205032-4-vsementsov@virtuozzo.com> X-Mailer: git-send-email 2.18.0 In-Reply-To: <20190411172709.205032-1-vsementsov@virtuozzo.com> References: <20190411172709.205032-1-vsementsov@virtuozzo.com> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x X-Received-From: 185.231.240.75 Subject: [Qemu-devel] [PATCH v6 3/7] block/nbd-client: move from quit to state X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, vsementsov@virtuozzo.com, armbru@redhat.com, mreitz@redhat.com, stefanha@redhat.com, den@openvz.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" To implement reconnect we need several states for the client: CONNECTED, QUIT and two different CONNECTING states. CONNECTING states will be added in the following patches. This patch implements CONNECTED and QUIT. QUIT means, that we should close the connection and fail all current and further requests (like old quit =3D true). CONNECTED means that connection is ok, we can send requests (like old quit =3D false). For receiving loop we use a comparison of the current state with QUIT, because reconnect will be in the same loop, so it should be looping until the end. Opposite, for requests we use a comparison of the current state with CONNECTED, as we don't want to send requests in future CONNECTING states. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Eric Blake --- block/nbd-client.h | 7 ++++++- block/nbd-client.c | 51 ++++++++++++++++++++++++++++------------------ 2 files changed, 37 insertions(+), 21 deletions(-) diff --git a/block/nbd-client.h b/block/nbd-client.h index 09e03013d2..e45b7b0a14 100644 --- a/block/nbd-client.h +++ b/block/nbd-client.h @@ -23,6 +23,11 @@ typedef struct { bool receiving; /* waiting for connection_co? */ } NBDClientRequest; =20 +typedef enum NBDClientState { + NBD_CLIENT_CONNECTED, + NBD_CLIENT_QUIT +} NBDClientState; + typedef struct NBDClientSession { QIOChannelSocket *sioc; /* The master data channel */ QIOChannel *ioc; /* The current I/O channel which may differ (eg TLS) = */ @@ -32,11 +37,11 @@ typedef struct NBDClientSession { CoQueue free_sema; Coroutine *connection_co; int in_flight; + NBDClientState state; =20 NBDClientRequest requests[MAX_NBD_REQUESTS]; NBDReply reply; BlockDriverState *bs; - bool quit; } NBDClientSession; =20 NBDClientSession *nbd_get_client_session(BlockDriverState *bs); diff --git a/block/nbd-client.c b/block/nbd-client.c index f9cee25865..9f5c86eaa3 100644 --- a/block/nbd-client.c +++ b/block/nbd-client.c @@ -36,6 +36,12 @@ #define HANDLE_TO_INDEX(bs, handle) ((handle) ^ (uint64_t)(intptr_t)(bs)) #define INDEX_TO_HANDLE(bs, index) ((index) ^ (uint64_t)(intptr_t)(bs)) =20 +/* @ret will be used for reconnect in future */ +static void nbd_channel_error(NBDClientSession *s, int ret) +{ + s->state =3D NBD_CLIENT_QUIT; +} + static void nbd_recv_coroutines_wake_all(NBDClientSession *s) { int i; @@ -75,7 +81,7 @@ static coroutine_fn void nbd_connection_entry(void *opaqu= e) int ret =3D 0; Error *local_err =3D NULL; =20 - while (!s->quit) { + while (s->state !=3D NBD_CLIENT_QUIT) { /* * The NBD client can only really be considered idle when it has * yielded from qio_channel_readv_all_eof(), waiting for data. Thi= s is @@ -93,6 +99,7 @@ static coroutine_fn void nbd_connection_entry(void *opaqu= e) error_free(local_err); } if (ret <=3D 0) { + nbd_channel_error(s, ret ? ret : -EIO); break; } =20 @@ -106,6 +113,7 @@ static coroutine_fn void nbd_connection_entry(void *opa= que) !s->requests[i].receiving || (nbd_reply_is_structured(&s->reply) && !s->info.structured_rep= ly)) { + nbd_channel_error(s, -EINVAL); break; } =20 @@ -124,7 +132,6 @@ static coroutine_fn void nbd_connection_entry(void *opa= que) qemu_coroutine_yield(); } =20 - s->quit =3D true; nbd_recv_coroutines_wake_all(s); bdrv_dec_in_flight(s->bs); =20 @@ -137,12 +144,18 @@ static int nbd_co_send_request(BlockDriverState *bs, QEMUIOVector *qiov) { NBDClientSession *s =3D nbd_get_client_session(bs); - int rc, i; + int rc, i =3D -1; =20 qemu_co_mutex_lock(&s->send_mutex); while (s->in_flight =3D=3D MAX_NBD_REQUESTS) { qemu_co_queue_wait(&s->free_sema, &s->send_mutex); } + + if (s->state !=3D NBD_CLIENT_CONNECTED) { + rc =3D -EIO; + goto err; + } + s->in_flight++; =20 for (i =3D 0; i < MAX_NBD_REQUESTS; i++) { @@ -160,16 +173,12 @@ static int nbd_co_send_request(BlockDriverState *bs, =20 request->handle =3D INDEX_TO_HANDLE(s, i); =20 - if (s->quit) { - rc =3D -EIO; - goto err; - } assert(s->ioc); =20 if (qiov) { qio_channel_set_cork(s->ioc, true); rc =3D nbd_send_request(s->ioc, request); - if (rc >=3D 0 && !s->quit) { + if (rc >=3D 0 && s->state =3D=3D NBD_CLIENT_CONNECTED) { if (qio_channel_writev_all(s->ioc, qiov->iov, qiov->niov, NULL) < 0) { rc =3D -EIO; @@ -184,9 +193,11 @@ static int nbd_co_send_request(BlockDriverState *bs, =20 err: if (rc < 0) { - s->quit =3D true; - s->requests[i].coroutine =3D NULL; - s->in_flight--; + nbd_channel_error(s, rc); + if (i !=3D -1) { + s->requests[i].coroutine =3D NULL; + s->in_flight--; + } qemu_co_queue_next(&s->free_sema); } qemu_co_mutex_unlock(&s->send_mutex); @@ -477,7 +488,7 @@ static coroutine_fn int nbd_co_do_receive_one_chunk( s->requests[i].receiving =3D true; qemu_coroutine_yield(); s->requests[i].receiving =3D false; - if (s->quit) { + if (s->state !=3D NBD_CLIENT_CONNECTED) { error_setg(errp, "Connection closed"); return -EIO; } @@ -560,7 +571,7 @@ static coroutine_fn int nbd_co_receive_one_chunk( request_ret, qiov, payload, errp= ); =20 if (ret < 0) { - s->quit =3D true; + nbd_channel_error(s, ret); } else { /* For assert at loop start in nbd_connection_entry */ if (reply) { @@ -626,7 +637,7 @@ static bool nbd_reply_chunk_iter_receive(NBDClientSessi= on *s, NBDReply local_reply; NBDStructuredReplyChunk *chunk; Error *local_err =3D NULL; - if (s->quit) { + if (s->state !=3D NBD_CLIENT_CONNECTED) { error_setg(&local_err, "Connection closed"); nbd_iter_channel_error(iter, -EIO, &local_err); goto break_loop; @@ -651,7 +662,7 @@ static bool nbd_reply_chunk_iter_receive(NBDClientSessi= on *s, } =20 /* Do not execute the body of NBD_FOREACH_REPLY_CHUNK for simple reply= . */ - if (nbd_reply_is_simple(reply) || s->quit) { + if (nbd_reply_is_simple(reply) || s->state !=3D NBD_CLIENT_CONNECTED) { goto break_loop; } =20 @@ -723,14 +734,14 @@ static int nbd_co_receive_cmdread_reply(NBDClientSess= ion *s, uint64_t handle, ret =3D nbd_parse_offset_hole_payload(s, &reply.structured, pa= yload, offset, qiov, &local_err); if (ret < 0) { - s->quit =3D true; + nbd_channel_error(s, ret); nbd_iter_channel_error(&iter, ret, &local_err); } break; default: if (!nbd_reply_type_is_error(chunk->type)) { /* not allowed reply type */ - s->quit =3D true; + nbd_channel_error(s, -EINVAL); error_setg(&local_err, "Unexpected reply type: %d (%s) for CMD_READ", chunk->type, nbd_reply_type_lookup(chunk->type)= ); @@ -768,7 +779,7 @@ static int nbd_co_receive_blockstatus_reply(NBDClientSe= ssion *s, switch (chunk->type) { case NBD_REPLY_TYPE_BLOCK_STATUS: if (received) { - s->quit =3D true; + nbd_channel_error(s, -EINVAL); error_setg(&local_err, "Several BLOCK_STATUS chunks in rep= ly"); nbd_iter_channel_error(&iter, -EINVAL, &local_err); } @@ -778,13 +789,13 @@ static int nbd_co_receive_blockstatus_reply(NBDClient= Session *s, payload, length, extent, &local_err); if (ret < 0) { - s->quit =3D true; + nbd_channel_error(s, ret); nbd_iter_channel_error(&iter, ret, &local_err); } break; default: if (!nbd_reply_type_is_error(chunk->type)) { - s->quit =3D true; + nbd_channel_error(s, -EINVAL); error_setg(&local_err, "Unexpected reply type: %d (%s) " "for CMD_BLOCK_STATUS", --=20 2.18.0 From nobody Mon Nov 10 08:11:35 2025 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=virtuozzo.com ARC-Seal: i=1; a=rsa-sha256; t=1555003940; cv=none; d=zoho.com; s=zohoarc; b=JGgwM9mfMAsaPBaHengxGkKz77XRiyvclnKkEJ88YDwJx7qe/gDHQbCz3okcu0NT/8cyWJC9jsyI7XWyMLSXnypSOwIavuudbglyIwGLaVJou0z83kYppvhqqYAGSZY36dGURZKKfADM8ILWDPVDVzUaicrGUPpARIDJjtbh8rE= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1555003940; h=Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results; bh=dW9dADMXs+S9pcqZfnZEx9tzcBHgsF6w3o67qUq7yk0=; b=LS4ylDETaRd8bQJvzaiwz4Xfe/AarQXK5OeCBkvorDiG632n3XhtISWN3JORAW4igRfvnxsPxZHUltdpNYFAPI1g+mgtIvkyIR8n4Ru11caCC9WPPY3Z7cMrmZyEon8J+OsiKOt7zo2DAo1a2pw8HslhCGRmr+Kf3RVNamieHXY= ARC-Authentication-Results: i=1; mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1555003940428839.413482810743; Thu, 11 Apr 2019 10:32:20 -0700 (PDT) Received: from localhost ([127.0.0.1]:52605 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hEdYe-0008ES-Fh for importer@patchew.org; Thu, 11 Apr 2019 13:32:16 -0400 Received: from eggs.gnu.org ([209.51.188.92]:50420) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hEdTm-0004ZG-Qe for qemu-devel@nongnu.org; Thu, 11 Apr 2019 13:27:16 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hEdTl-0003bH-Kw for qemu-devel@nongnu.org; Thu, 11 Apr 2019 13:27:14 -0400 Received: from relay.sw.ru ([185.231.240.75]:59352) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1hEdTl-0003Zx-9f; Thu, 11 Apr 2019 13:27:13 -0400 Received: from [10.28.8.145] (helo=kvm.sw.ru) by relay.sw.ru with esmtp (Exim 4.91) (envelope-from ) id 1hEdTi-00044D-Tp; Thu, 11 Apr 2019 20:27:10 +0300 From: Vladimir Sementsov-Ogievskiy To: qemu-devel@nongnu.org, qemu-block@nongnu.org Date: Thu, 11 Apr 2019 20:27:06 +0300 Message-Id: <20190411172709.205032-5-vsementsov@virtuozzo.com> X-Mailer: git-send-email 2.18.0 In-Reply-To: <20190411172709.205032-1-vsementsov@virtuozzo.com> References: <20190411172709.205032-1-vsementsov@virtuozzo.com> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x X-Received-From: 185.231.240.75 Subject: [Qemu-devel] [PATCH v6 4/7] block/nbd: add cmdline and qapi parameter reconnect-delay X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, vsementsov@virtuozzo.com, armbru@redhat.com, mreitz@redhat.com, stefanha@redhat.com, den@openvz.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Reconnect will be implemented in the following commit, so for now, in semantics below, disconnect itself is a "serious error". Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Eric Blake --- qapi/block-core.json | 12 +++++++++++- block/nbd-client.h | 1 + block/nbd-client.c | 1 + block/nbd.c | 16 +++++++++++++++- 4 files changed, 28 insertions(+), 2 deletions(-) diff --git a/qapi/block-core.json b/qapi/block-core.json index 7ccbfff9d0..815258bd89 100644 --- a/qapi/block-core.json +++ b/qapi/block-core.json @@ -3798,13 +3798,23 @@ # traditional "base:allocation" block status (see # NBD_OPT_LIST_META_CONTEXT in the NBD protocol) (since 3= .0) # +# @reconnect-delay: Reconnect delay. On unexpected disconnect, nbd client = tries +# to connect again, until success or serious error. Duri= ng +# first @reconnect-delay seconds of reconnecting loop all +# requests are paused and have a chance to rerun, if +# successful connect occurs during this time. After +# @reconnect-delay seconds all delayed requests are fail= ed +# and all following requests will be failed too (until +# successful reconnect). Default 0 (Since 4.1) +# # Since: 2.9 ## { 'struct': 'BlockdevOptionsNbd', 'data': { 'server': 'SocketAddress', '*export': 'str', '*tls-creds': 'str', - '*x-dirty-bitmap': 'str' } } + '*x-dirty-bitmap': 'str', + '*reconnect-delay': 'uint32' } } =20 ## # @BlockdevOptionsRaw: diff --git a/block/nbd-client.h b/block/nbd-client.h index e45b7b0a14..91a6b32bdd 100644 --- a/block/nbd-client.h +++ b/block/nbd-client.h @@ -52,6 +52,7 @@ int nbd_client_init(BlockDriverState *bs, QCryptoTLSCreds *tlscreds, const char *hostname, const char *x_dirty_bitmap, + uint32_t reconnect_delay, Error **errp); void nbd_client_close(BlockDriverState *bs); =20 diff --git a/block/nbd-client.c b/block/nbd-client.c index 9f5c86eaa3..1359aff162 100644 --- a/block/nbd-client.c +++ b/block/nbd-client.c @@ -1218,6 +1218,7 @@ int nbd_client_init(BlockDriverState *bs, QCryptoTLSCreds *tlscreds, const char *hostname, const char *x_dirty_bitmap, + uint32_t reconnect_delay, Error **errp) { int ret; diff --git a/block/nbd.c b/block/nbd.c index 208be59602..cffb2aee89 100644 --- a/block/nbd.c +++ b/block/nbd.c @@ -360,6 +360,18 @@ static QemuOptsList nbd_runtime_opts =3D { .help =3D "experimental: expose named dirty bitmap in place of= " "block status", }, + { + .name =3D "reconnect-delay", + .type =3D QEMU_OPT_NUMBER, + .help =3D "Reconnect delay. On unexpected disconnect, nbd clie= nt " + "tries to connect again, until success or serious erro= r. " + "During first @reconnect-delay seconds of reconnecting= " + "loop all requests are paused and have a chance to rer= un, " + "if successful connect occurs during this time. After" + "@reconnect-delay seconds all delayed requests are fai= led" + "and all following requests will be failed too (until" + "successful reconnect). Default 0", + }, { /* end of list */ } }, }; @@ -411,7 +423,9 @@ static int nbd_open(BlockDriverState *bs, QDict *option= s, int flags, =20 /* NBD handshake */ ret =3D nbd_client_init(bs, s->saddr, s->export, tlscreds, hostname, - qemu_opt_get(opts, "x-dirty-bitmap"), errp); + qemu_opt_get(opts, "x-dirty-bitmap"), + qemu_opt_get_number(opts, "reconnect-delay", 0), + errp); =20 error: if (tlscreds) { --=20 2.18.0 From nobody Mon Nov 10 08:11:35 2025 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=virtuozzo.com ARC-Seal: i=1; a=rsa-sha256; t=1555003761; cv=none; d=zoho.com; s=zohoarc; b=nY9mpJE4RkWGv4gqThPZjrmATNSl1bTYFdGPjHsPitTxLhJ+yGj8JM5FswIZ8Vu04/ornq+9yIou43QAOtcXTOLzUcnTl7HEGGBVUChtTCaszIqCDIpbuozbYiQBOdUjZe/Q/vbm8dpt9vefXu7C+C78RHuiq42e3f+Ra9NQ4gQ= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1555003761; h=Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results; bh=1kqFn0pA2I3tptSK56dtKIQMWMpOFUjWQb3k4AJd1zM=; b=UqrDnsr2UgNokUrxujJbYRK4GWF5KHrMgHC4LO49AjmPl5G1AsE2lxl/gdPXNZ5h8t/gGR3NjaRAwpxz1R7OB2vjvozm3K1ulA0s1QWyeLYoJ2pPP2w+8SO4s8Th93NiKP8IobQyNEo+qb39glreLBA8WdgEwV9HMjIoyxtpBes= ARC-Authentication-Results: i=1; mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (209.51.188.17 [209.51.188.17]) by mx.zohomail.com with SMTPS id 1555003761212597.4916044001446; Thu, 11 Apr 2019 10:29:21 -0700 (PDT) Received: from localhost ([127.0.0.1]:52549 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hEdVf-0005jJ-II for importer@patchew.org; Thu, 11 Apr 2019 13:29:11 -0400 Received: from eggs.gnu.org ([209.51.188.92]:50426) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hEdTm-0004ZH-Vt for qemu-devel@nongnu.org; Thu, 11 Apr 2019 13:27:16 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hEdTl-0003bl-Qs for qemu-devel@nongnu.org; Thu, 11 Apr 2019 13:27:14 -0400 Received: from relay.sw.ru ([185.231.240.75]:59344) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1hEdTl-0003Zv-D7; Thu, 11 Apr 2019 13:27:13 -0400 Received: from [10.28.8.145] (helo=kvm.sw.ru) by relay.sw.ru with esmtp (Exim 4.91) (envelope-from ) id 1hEdTj-00044D-2R; Thu, 11 Apr 2019 20:27:11 +0300 From: Vladimir Sementsov-Ogievskiy To: qemu-devel@nongnu.org, qemu-block@nongnu.org Date: Thu, 11 Apr 2019 20:27:07 +0300 Message-Id: <20190411172709.205032-6-vsementsov@virtuozzo.com> X-Mailer: git-send-email 2.18.0 In-Reply-To: <20190411172709.205032-1-vsementsov@virtuozzo.com> References: <20190411172709.205032-1-vsementsov@virtuozzo.com> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x X-Received-From: 185.231.240.75 Subject: [Qemu-devel] [PATCH v6 5/7] qemu-coroutine-sleep: introduce qemu_co_sleep_wake X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, vsementsov@virtuozzo.com, armbru@redhat.com, mreitz@redhat.com, stefanha@redhat.com, den@openvz.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Introduce a function to gracefully wake-up a coroutine, sleeping in qemu_co_sleep_ns() sleep. Signed-off-by: Vladimir Sementsov-Ogievskiy --- include/qemu/coroutine.h | 6 ++++++ util/qemu-coroutine-sleep.c | 20 ++++++++++++++++---- 2 files changed, 22 insertions(+), 4 deletions(-) diff --git a/include/qemu/coroutine.h b/include/qemu/coroutine.h index 9801e7f5a4..ec765c26f0 100644 --- a/include/qemu/coroutine.h +++ b/include/qemu/coroutine.h @@ -278,6 +278,12 @@ void qemu_co_rwlock_unlock(CoRwlock *lock); */ void coroutine_fn qemu_co_sleep_ns(QEMUClockType type, int64_t ns); =20 +/* + * Wake a coroutine if it is sleeping by qemu_co_sleep_ns. Timer will be + * deleted. + */ +void qemu_co_sleep_wake(Coroutine *co); + /** * Yield until a file descriptor becomes readable * diff --git a/util/qemu-coroutine-sleep.c b/util/qemu-coroutine-sleep.c index 4bfdd30cbf..bcc6afca3e 100644 --- a/util/qemu-coroutine-sleep.c +++ b/util/qemu-coroutine-sleep.c @@ -17,13 +17,24 @@ #include "qemu/timer.h" #include "block/aio.h" =20 +const char *qemu_co_sleep_ns__scheduled =3D "qemu_co_sleep_ns"; + +void qemu_co_sleep_wake(Coroutine *co) +{ + /* Write of schedule protected by barrier write in aio_co_schedule */ + const char *scheduled =3D atomic_cmpxchg(&co->scheduled, + qemu_co_sleep_ns__scheduled, NU= LL); + + if (scheduled =3D=3D qemu_co_sleep_ns__scheduled) { + aio_co_wake(co); + } +} + static void co_sleep_cb(void *opaque) { Coroutine *co =3D opaque; =20 - /* Write of schedule protected by barrier write in aio_co_schedule */ - atomic_set(&co->scheduled, NULL); - aio_co_wake(co); + qemu_co_sleep_wake(co); } =20 void coroutine_fn qemu_co_sleep_ns(QEMUClockType type, int64_t ns) @@ -32,7 +43,8 @@ void coroutine_fn qemu_co_sleep_ns(QEMUClockType type, in= t64_t ns) QEMUTimer *ts; Coroutine *co =3D qemu_coroutine_self(); =20 - const char *scheduled =3D atomic_cmpxchg(&co->scheduled, NULL, __func_= _); + const char *scheduled =3D atomic_cmpxchg(&co->scheduled, NULL, + qemu_co_sleep_ns__scheduled); if (scheduled) { fprintf(stderr, "%s: Co-routine was already scheduled in '%s'\n", --=20 2.18.0 From nobody Mon Nov 10 08:11:35 2025 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=virtuozzo.com ARC-Seal: i=1; a=rsa-sha256; t=1555003801; cv=none; d=zoho.com; s=zohoarc; b=CcInH19Mx8HUfMss5EEKJLSwvnRBNqbzJ8ihfQcCyCp5NFrfwy863b96z0M5nubJrcdE1fP3cIWpHVm/qFj+RMLjG9QhjNlVVUHjsJrceztaI/8WxJRHsurTGeANRs16wur2Zc7KHVMHpIazgprEAzNm+iIwYUkQ7jAGkgULnmk= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1555003801; h=Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results; bh=PzSZwivKRT9xnMRscfRaCxtXRa4v/iNK9R1qPj8EDQ0=; b=DbdYlspHOQiLlNa5PAGT98BUNwdAdf4KzAggUYbvX4tsNUxm9psdZzJ7gCZ4JtTVvJJquQ+uUIxGv3nEKEL5mK9RwlQ6nmUwlr5pOhv3Uinar5Wka8EKRinFstRYCdcutr/IlUB9+C8hpOGbUblKjw3zmpqJ+D7bu13pDf0AFYQ= ARC-Authentication-Results: i=1; mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1555003801087120.98080368983108; Thu, 11 Apr 2019 10:30:01 -0700 (PDT) Received: from localhost ([127.0.0.1]:52553 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hEdWM-0006Ix-ST for importer@patchew.org; Thu, 11 Apr 2019 13:29:54 -0400 Received: from eggs.gnu.org ([209.51.188.92]:50472) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hEdTo-0004ZZ-Co for qemu-devel@nongnu.org; Thu, 11 Apr 2019 13:27:18 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hEdTl-0003c4-Rn for qemu-devel@nongnu.org; Thu, 11 Apr 2019 13:27:16 -0400 Received: from relay.sw.ru ([185.231.240.75]:59366) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1hEdTl-0003Zy-9h; Thu, 11 Apr 2019 13:27:13 -0400 Received: from [10.28.8.145] (helo=kvm.sw.ru) by relay.sw.ru with esmtp (Exim 4.91) (envelope-from ) id 1hEdTj-00044D-8X; Thu, 11 Apr 2019 20:27:11 +0300 From: Vladimir Sementsov-Ogievskiy To: qemu-devel@nongnu.org, qemu-block@nongnu.org Date: Thu, 11 Apr 2019 20:27:08 +0300 Message-Id: <20190411172709.205032-7-vsementsov@virtuozzo.com> X-Mailer: git-send-email 2.18.0 In-Reply-To: <20190411172709.205032-1-vsementsov@virtuozzo.com> References: <20190411172709.205032-1-vsementsov@virtuozzo.com> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x X-Received-From: 185.231.240.75 Subject: [Qemu-devel] [PATCH v6 6/7] block/nbd-client: nbd reconnect X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, vsementsov@virtuozzo.com, armbru@redhat.com, mreitz@redhat.com, stefanha@redhat.com, den@openvz.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Implement reconnect. To achieve this: 1. add new modes: connecting-wait: means, that reconnecting is in progress, and there were small number of reconnect attempts, so all requests are waiting for the connection. connecting-nowait: reconnecting is in progress, there were a lot of attempts of reconnect, all requests will return errors. two old modes are used too: connected: normal state quit: exiting after fatal error or on close Possible transitions are: * -> quit connecting-* -> connected connecting-wait -> connecting-nowait (transition is done after reconnect-delay seconds in connecting-wait mode) connected -> connecting-wait 2. Implement reconnect in connection_co. So, in connecting-* mode, connection_co, tries to reconnect unlimited times. 3. Retry nbd queries on channel error, if we are in connecting-wait state. Signed-off-by: Vladimir Sementsov-Ogievskiy --- block/nbd-client.h | 7 + block/nbd-client.c | 336 +++++++++++++++++++++++++++++++++++---------- 2 files changed, 273 insertions(+), 70 deletions(-) diff --git a/block/nbd-client.h b/block/nbd-client.h index 91a6b32bdd..f366c90e5e 100644 --- a/block/nbd-client.h +++ b/block/nbd-client.h @@ -24,6 +24,8 @@ typedef struct { } NBDClientRequest; =20 typedef enum NBDClientState { + NBD_CLIENT_CONNECTING_WAIT, + NBD_CLIENT_CONNECTING_NOWAIT, NBD_CLIENT_CONNECTED, NBD_CLIENT_QUIT } NBDClientState; @@ -38,10 +40,15 @@ typedef struct NBDClientSession { Coroutine *connection_co; int in_flight; NBDClientState state; + int connect_status; + Error *connect_err; + bool wait_in_flight; =20 NBDClientRequest requests[MAX_NBD_REQUESTS]; NBDReply reply; BlockDriverState *bs; + + uint32_t reconnect_delay; } NBDClientSession; =20 NBDClientSession *nbd_get_client_session(BlockDriverState *bs); diff --git a/block/nbd-client.c b/block/nbd-client.c index 1359aff162..b829a1a1bd 100644 --- a/block/nbd-client.c +++ b/block/nbd-client.c @@ -1,6 +1,7 @@ /* * QEMU Block driver for NBD * + * Copyright (c) 2019 Virtuozzo International GmbH. All rights reserved. * Copyright (C) 2016 Red Hat, Inc. * Copyright (C) 2008 Bull S.A.S. * Author: Laurent Vivier @@ -36,10 +37,27 @@ #define HANDLE_TO_INDEX(bs, handle) ((handle) ^ (uint64_t)(intptr_t)(bs)) #define INDEX_TO_HANDLE(bs, index) ((index) ^ (uint64_t)(intptr_t)(bs)) =20 -/* @ret will be used for reconnect in future */ +static int nbd_client_connect(BlockDriverState *bs, + SocketAddress *saddr, + const char *export, + QCryptoTLSCreds *tlscreds, + const char *hostname, + const char *x_dirty_bitmap, + Error **errp); + static void nbd_channel_error(NBDClientSession *s, int ret) { - s->state =3D NBD_CLIENT_QUIT; + if (ret =3D=3D -EIO) { + if (s->state =3D=3D NBD_CLIENT_CONNECTED) { + s->state =3D s->reconnect_delay ? NBD_CLIENT_CONNECTING_WAIT : + NBD_CLIENT_CONNECTING_NOWAIT; + } + } else { + if (s->state =3D=3D NBD_CLIENT_CONNECTED) { + qio_channel_shutdown(s->ioc, QIO_CHANNEL_SHUTDOWN_BOTH, NULL); + } + s->state =3D NBD_CLIENT_QUIT; + } } =20 static void nbd_recv_coroutines_wake_all(NBDClientSession *s) @@ -59,24 +77,133 @@ static void nbd_teardown_connection(BlockDriverState *= bs) { NBDClientSession *client =3D nbd_get_client_session(bs); =20 - assert(client->ioc); - - /* finish any pending coroutines */ - qio_channel_shutdown(client->ioc, - QIO_CHANNEL_SHUTDOWN_BOTH, - NULL); + if (client->state =3D=3D NBD_CLIENT_CONNECTED) { + /* finish any pending coroutines */ + assert(client->ioc); + qio_channel_shutdown(client->ioc, QIO_CHANNEL_SHUTDOWN_BOTH, NULL); + } + client->state =3D NBD_CLIENT_QUIT; + if (client->connection_co) { + qemu_co_sleep_wake(client->connection_co); + } BDRV_POLL_WHILE(bs, client->connection_co); +} + +typedef struct NBDConnection { + BlockDriverState *bs; + SocketAddress *saddr; + const char *export; + QCryptoTLSCreds *tlscreds; + const char *hostname; + const char *x_dirty_bitmap; +} NBDConnection; + +static bool nbd_client_connecting(NBDClientSession *client) +{ + return client->state =3D=3D NBD_CLIENT_CONNECTING_WAIT || + client->state =3D=3D NBD_CLIENT_CONNECTING_NOWAIT; +} + +static bool nbd_client_connecting_wait(NBDClientSession *client) +{ + return client->state =3D=3D NBD_CLIENT_CONNECTING_WAIT; +} + +static coroutine_fn void nbd_reconnect_attempt(NBDConnection *con) +{ + NBDClientSession *s =3D nbd_get_client_session(con->bs); + Error *local_err =3D NULL; + + if (!nbd_client_connecting(s)) { + return; + } + assert(nbd_client_connecting(s)); + + /* Wait completion of all in-flight requests */ + + qemu_co_mutex_lock(&s->send_mutex); =20 - nbd_client_detach_aio_context(bs); - object_unref(OBJECT(client->sioc)); - client->sioc =3D NULL; - object_unref(OBJECT(client->ioc)); - client->ioc =3D NULL; + while (s->in_flight > 0) { + qemu_co_mutex_unlock(&s->send_mutex); + nbd_recv_coroutines_wake_all(s); + s->wait_in_flight =3D true; + qemu_coroutine_yield(); + s->wait_in_flight =3D false; + qemu_co_mutex_lock(&s->send_mutex); + } + + qemu_co_mutex_unlock(&s->send_mutex); + + if (!nbd_client_connecting(s)) { + return; + } + + /* + * Now we are sure, that nobody accessing the channel now and nobody + * will try to access the channel, until we set state to CONNECTED + */ + + /* Finalize previous connection if any */ + if (s->ioc) { + nbd_client_detach_aio_context(con->bs); + object_unref(OBJECT(s->sioc)); + s->sioc =3D NULL; + object_unref(OBJECT(s->ioc)); + s->ioc =3D NULL; + } + + s->connect_status =3D nbd_client_connect(con->bs, con->saddr, + con->export, con->tlscreds, + con->hostname, con->x_dirty_bit= map, + &local_err); + error_free(s->connect_err); + s->connect_err =3D NULL; + error_propagate(&s->connect_err, local_err); + local_err =3D NULL; + + if (s->connect_status < 0) { + /* failed attempt */ + return; + } + + /* successfully connected */ + s->state =3D NBD_CLIENT_CONNECTED; + qemu_co_queue_restart_all(&s->free_sema); +} + +static coroutine_fn void nbd_reconnect_loop(NBDConnection *con) +{ + NBDClientSession *s =3D nbd_get_client_session(con->bs); + uint64_t start_time_ns =3D qemu_clock_get_ns(QEMU_CLOCK_REALTIME); + uint64_t delay_ns =3D s->reconnect_delay * 1000000000UL; + uint64_t timeout =3D 1000000000UL; /* 1 sec */ + uint64_t max_timeout =3D 16000000000UL; /* 16 sec */ + + nbd_reconnect_attempt(con); + + while (nbd_client_connecting(s)) { + if (s->state =3D=3D NBD_CLIENT_CONNECTING_WAIT && + qemu_clock_get_ns(QEMU_CLOCK_REALTIME) - start_time_ns > delay= _ns) + { + s->state =3D NBD_CLIENT_CONNECTING_NOWAIT; + qemu_co_queue_restart_all(&s->free_sema); + } + + bdrv_dec_in_flight(con->bs); + qemu_co_sleep_ns(QEMU_CLOCK_REALTIME, timeout); + bdrv_inc_in_flight(con->bs); + if (timeout < max_timeout) { + timeout *=3D 2; + } + + nbd_reconnect_attempt(con); + } } =20 static coroutine_fn void nbd_connection_entry(void *opaque) { - NBDClientSession *s =3D opaque; + NBDConnection *con =3D opaque; + NBDClientSession *s =3D nbd_get_client_session(con->bs); uint64_t i; int ret =3D 0; Error *local_err =3D NULL; @@ -91,16 +218,25 @@ static coroutine_fn void nbd_connection_entry(void *op= aque) * Therefore we keep an additional in_flight reference all the tim= e and * only drop it temporarily here. */ + + if (nbd_client_connecting(s)) { + nbd_reconnect_loop(con); + } + + if (s->state !=3D NBD_CLIENT_CONNECTED) { + continue; + } + assert(s->reply.handle =3D=3D 0); ret =3D nbd_receive_reply(s->bs, s->ioc, &s->reply, &local_err); - if (local_err) { trace_nbd_read_reply_entry_fail(ret, error_get_pretty(local_er= r)); error_free(local_err); + local_err =3D NULL; } if (ret <=3D 0) { nbd_channel_error(s, ret ? ret : -EIO); - break; + continue; } =20 /* There's no need for a mutex on the receive side, because the @@ -114,7 +250,7 @@ static coroutine_fn void nbd_connection_entry(void *opa= que) (nbd_reply_is_structured(&s->reply) && !s->info.structured_rep= ly)) { nbd_channel_error(s, -EINVAL); - break; + continue; } =20 /* We're woken up again by the request itself. Note that there @@ -132,10 +268,21 @@ static coroutine_fn void nbd_connection_entry(void *o= paque) qemu_coroutine_yield(); } =20 + qemu_co_queue_restart_all(&s->free_sema); nbd_recv_coroutines_wake_all(s); bdrv_dec_in_flight(s->bs); =20 s->connection_co =3D NULL; + if (s->ioc) { + nbd_client_detach_aio_context(con->bs); + object_unref(OBJECT(s->sioc)); + s->sioc =3D NULL; + object_unref(OBJECT(s->ioc)); + s->ioc =3D NULL; + } + + g_free(con); + aio_wait_kick(); } =20 @@ -147,7 +294,7 @@ static int nbd_co_send_request(BlockDriverState *bs, int rc, i =3D -1; =20 qemu_co_mutex_lock(&s->send_mutex); - while (s->in_flight =3D=3D MAX_NBD_REQUESTS) { + while (s->in_flight =3D=3D MAX_NBD_REQUESTS || nbd_client_connecting_w= ait(s)) { qemu_co_queue_wait(&s->free_sema, &s->send_mutex); } =20 @@ -198,7 +345,11 @@ err: s->requests[i].coroutine =3D NULL; s->in_flight--; } - qemu_co_queue_next(&s->free_sema); + if (s->in_flight =3D=3D 0 && s->wait_in_flight) { + aio_co_wake(s->connection_co); + } else { + qemu_co_queue_next(&s->free_sema); + } } qemu_co_mutex_unlock(&s->send_mutex); return rc; @@ -577,10 +728,15 @@ static coroutine_fn int nbd_co_receive_one_chunk( if (reply) { *reply =3D s->reply; } - s->reply.handle =3D 0; } + s->reply.handle =3D 0; =20 - if (s->connection_co) { + if (s->connection_co && !s->wait_in_flight) { + /* + * We must check s->wait_in_flight, because we may entered by + * nbd_recv_coroutines_wake_all(), in this case we should not + * wake connection_co here, it will woken by last request. + */ aio_co_wake(s->connection_co); } =20 @@ -688,7 +844,11 @@ break_loop: =20 qemu_co_mutex_lock(&s->send_mutex); s->in_flight--; - qemu_co_queue_next(&s->free_sema); + if (s->in_flight =3D=3D 0 && s->wait_in_flight) { + aio_co_wake(s->connection_co); + } else { + qemu_co_queue_next(&s->free_sema); + } qemu_co_mutex_unlock(&s->send_mutex); =20 return false; @@ -832,20 +992,26 @@ static int nbd_co_request(BlockDriverState *bs, NBDRe= quest *request, } else { assert(request->type !=3D NBD_CMD_WRITE); } - ret =3D nbd_co_send_request(bs, request, write_qiov); - if (ret < 0) { - return ret; - } =20 - ret =3D nbd_co_receive_return_code(client, request->handle, - &request_ret, &local_err); - if (local_err) { - trace_nbd_co_request_fail(request->from, request->len, request->ha= ndle, - request->flags, request->type, - nbd_cmd_lookup(request->type), - ret, error_get_pretty(local_err)); - error_free(local_err); - } + do { + ret =3D nbd_co_send_request(bs, request, write_qiov); + if (ret < 0) { + continue; + } + + ret =3D nbd_co_receive_return_code(client, request->handle, + &request_ret, &local_err); + if (local_err) { + trace_nbd_co_request_fail(request->from, request->len, + request->handle, request->flags, + request->type, + nbd_cmd_lookup(request->type), + ret, error_get_pretty(local_err)); + error_free(local_err); + local_err =3D NULL; + } + } while (ret < 0 && nbd_client_connecting_wait(client)); + return ret ? ret : request_ret; } =20 @@ -886,20 +1052,24 @@ int nbd_client_co_preadv(BlockDriverState *bs, uint6= 4_t offset, request.len -=3D slop; } =20 - ret =3D nbd_co_send_request(bs, &request, NULL); - if (ret < 0) { - return ret; - } + do { + ret =3D nbd_co_send_request(bs, &request, NULL); + if (ret < 0) { + continue; + } + + ret =3D nbd_co_receive_cmdread_reply(client, request.handle, offse= t, qiov, + &request_ret, &local_err); + if (local_err) { + trace_nbd_co_request_fail(request.from, request.len, request.h= andle, + request.flags, request.type, + nbd_cmd_lookup(request.type), + ret, error_get_pretty(local_err)); + error_free(local_err); + local_err =3D NULL; + } + } while (ret < 0 && nbd_client_connecting_wait(client)); =20 - ret =3D nbd_co_receive_cmdread_reply(client, request.handle, offset, q= iov, - &request_ret, &local_err); - if (local_err) { - trace_nbd_co_request_fail(request.from, request.len, request.handl= e, - request.flags, request.type, - nbd_cmd_lookup(request.type), - ret, error_get_pretty(local_err)); - error_free(local_err); - } return ret ? ret : request_ret; } =20 @@ -1033,20 +1203,25 @@ int coroutine_fn nbd_client_co_block_status(BlockDr= iverState *bs, if (client->info.min_block) { assert(QEMU_IS_ALIGNED(request.len, client->info.min_block)); } - ret =3D nbd_co_send_request(bs, &request, NULL); - if (ret < 0) { - return ret; - } + do { + ret =3D nbd_co_send_request(bs, &request, NULL); + if (ret < 0) { + continue; + } + + ret =3D nbd_co_receive_blockstatus_reply(client, request.handle, b= ytes, + &extent, &request_ret, + &local_err); + if (local_err) { + trace_nbd_co_request_fail(request.from, request.len, request.h= andle, + request.flags, request.type, + nbd_cmd_lookup(request.type), + ret, error_get_pretty(local_err)); + error_free(local_err); + local_err =3D NULL; + } + } while (ret < 0 && nbd_client_connecting_wait(client)); =20 - ret =3D nbd_co_receive_blockstatus_reply(client, request.handle, bytes, - &extent, &request_ret, &local_e= rr); - if (local_err) { - trace_nbd_co_request_fail(request.from, request.len, request.handl= e, - request.flags, request.type, - nbd_cmd_lookup(request.type), - ret, error_get_pretty(local_err)); - error_free(local_err); - } if (ret < 0 || request_ret < 0) { return ret ? ret : request_ret; } @@ -1083,13 +1258,22 @@ void nbd_client_attach_aio_context(BlockDriverState= *bs, AioContext *new_context) { NBDClientSession *client =3D nbd_get_client_session(bs); - qio_channel_attach_aio_context(QIO_CHANNEL(client->ioc), new_context); =20 - bdrv_inc_in_flight(bs); + /* + * client->connection_co is either yielded from nbd_receive_reply or f= rom + * nbd_reconnect_loop(), in latter case we do nothing + */ + if (client->state =3D=3D NBD_CLIENT_CONNECTED) { + qio_channel_attach_aio_context(QIO_CHANNEL(client->ioc), new_conte= xt); =20 - /* Need to wait here for the BH to run because the BH must run while t= he - * node is still drained. */ - aio_wait_bh_oneshot(new_context, nbd_client_attach_aio_context_bh, bs); + bdrv_inc_in_flight(bs); + + /* + * Need to wait here for the BH to run because the BH must run whi= le the + * node is still drained. + */ + aio_wait_bh_oneshot(new_context, nbd_client_attach_aio_context_bh,= bs); + } } =20 void nbd_client_close(BlockDriverState *bs) @@ -1097,9 +1281,9 @@ void nbd_client_close(BlockDriverState *bs) NBDClientSession *client =3D nbd_get_client_session(bs); NBDRequest request =3D { .type =3D NBD_CMD_DISC }; =20 - assert(client->ioc); - - nbd_send_request(client->ioc, &request); + if (client->ioc) { + nbd_send_request(client->ioc, &request); + } =20 nbd_teardown_connection(bs); } @@ -1223,6 +1407,7 @@ int nbd_client_init(BlockDriverState *bs, { int ret; NBDClientSession *client =3D nbd_get_client_session(bs); + NBDConnection *con; =20 client->bs =3D bs; qemu_co_mutex_init(&client->send_mutex); @@ -1233,8 +1418,19 @@ int nbd_client_init(BlockDriverState *bs, if (ret < 0) { return ret; } + /* successfully connected */ + client->state =3D NBD_CLIENT_CONNECTED; + client->reconnect_delay =3D reconnect_delay; + + con =3D g_new(NBDConnection, 1); + con->bs =3D bs; + con->saddr =3D saddr; + con->export =3D export; + con->tlscreds =3D tlscreds; + con->hostname =3D hostname; + con->x_dirty_bitmap =3D x_dirty_bitmap; =20 - client->connection_co =3D qemu_coroutine_create(nbd_connection_entry, = client); + client->connection_co =3D qemu_coroutine_create(nbd_connection_entry, = con); bdrv_inc_in_flight(bs); aio_co_schedule(bdrv_get_aio_context(bs), client->connection_co); =20 --=20 2.18.0 From nobody Mon Nov 10 08:11:35 2025 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=virtuozzo.com ARC-Seal: i=1; a=rsa-sha256; t=1555004148; cv=none; d=zoho.com; s=zohoarc; b=jmidFvV442U8wHnin1GtJAJoHm6o3sigUjVLIv2kxzjHg+KluoEn6K0spno+lFJ5dqSO+LPry5ipFAAFdp0B9abzmUxS6QV4FGUQMN5W31TroexVnbgpqb4wZPCiNbnQKPUhAjYhApVTC626zB0mMxtyGFbmkpgnezpAXDUzwWs= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1555004148; h=Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results; bh=FnftmpW4IoPbLwxvzw1kdTm1ajWCOPHz3yYtCBr48f4=; b=N1hh5/5vbyYpn7Jfsm7otj2aGGygqkH18mfhQCw4y8k+CWob005X1wbpos45I43C5qPBHF31NOKfP8B7zEREyhVf8bAPdiRFiWA5pIBhC94OgvNB1DG+HSTYvABxLc9NjaW7emlixVLYmLYpPC3xrCqlaFlLUD4IuSPksOV7tzw= ARC-Authentication-Results: i=1; mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1555004148148269.81988806760114; Thu, 11 Apr 2019 10:35:48 -0700 (PDT) Received: from localhost ([127.0.0.1]:52648 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hEdbr-00021z-79 for importer@patchew.org; Thu, 11 Apr 2019 13:35:35 -0400 Received: from eggs.gnu.org ([209.51.188.92]:50445) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hEdTn-0004ZL-Dd for qemu-devel@nongnu.org; Thu, 11 Apr 2019 13:27:17 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hEdTl-0003bz-Ra for qemu-devel@nongnu.org; Thu, 11 Apr 2019 13:27:15 -0400 Received: from relay.sw.ru ([185.231.240.75]:59370) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1hEdTl-0003a0-EQ; Thu, 11 Apr 2019 13:27:13 -0400 Received: from [10.28.8.145] (helo=kvm.sw.ru) by relay.sw.ru with esmtp (Exim 4.91) (envelope-from ) id 1hEdTj-00044D-Fc; Thu, 11 Apr 2019 20:27:11 +0300 From: Vladimir Sementsov-Ogievskiy To: qemu-devel@nongnu.org, qemu-block@nongnu.org Date: Thu, 11 Apr 2019 20:27:09 +0300 Message-Id: <20190411172709.205032-8-vsementsov@virtuozzo.com> X-Mailer: git-send-email 2.18.0 In-Reply-To: <20190411172709.205032-1-vsementsov@virtuozzo.com> References: <20190411172709.205032-1-vsementsov@virtuozzo.com> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x X-Received-From: 185.231.240.75 Subject: [Qemu-devel] [PATCH v6 7/7] iotests: test nbd reconnect X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, vsementsov@virtuozzo.com, armbru@redhat.com, mreitz@redhat.com, stefanha@redhat.com, den@openvz.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Add test, which starts backup to nbd target and restarts nbd server during backup. Signed-off-by: Vladimir Sementsov-Ogievskiy --- tests/qemu-iotests/249 | 63 +++++++++++++++++++++++++++++++++++ tests/qemu-iotests/249.out | 10 ++++++ tests/qemu-iotests/group | 1 + tests/qemu-iotests/iotests.py | 4 +++ 4 files changed, 78 insertions(+) create mode 100755 tests/qemu-iotests/249 create mode 100644 tests/qemu-iotests/249.out diff --git a/tests/qemu-iotests/249 b/tests/qemu-iotests/249 new file mode 100755 index 0000000000..a29a276207 --- /dev/null +++ b/tests/qemu-iotests/249 @@ -0,0 +1,63 @@ +#!/usr/bin/env python +# +# Test nbd reconnect +# +# Copyright (c) 2019 Virtuozzo International GmbH. All rights reserved. +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program. If not, see . +# + +import time + +import iotests +from iotests import qemu_img_create, file_path, qemu_nbd_popen, log + +disk_a, disk_b, nbd_sock =3D file_path('disk_a', 'disk_b', 'nbd-sock') + +qemu_img_create('-f', iotests.imgfmt, disk_a, '5M') +qemu_img_create('-f', iotests.imgfmt, disk_b, '5M') +srv =3D qemu_nbd_popen('-k', nbd_sock, '-f', iotests.imgfmt, disk_b) +time.sleep(1) + +vm =3D iotests.VM().add_drive(disk_a) +vm.launch() +vm.hmp_qemu_io('drive0', 'write 0 5M') + +vm.qmp_log('blockdev-add', filters=3D[iotests.filter_qmp_testfiles], + **{'node_name': 'backup0', + 'driver': 'raw', + 'file': {'driver': 'nbd', + 'server': {'type': 'unix', 'path': nbd_sock}, + 'reconnect-delay': 10}}) +vm.qmp_log('blockdev-backup', device=3D'drive0', sync=3D'full', target=3D'= backup0') + +time.sleep(1) +log('Kill NBD server') +srv.kill() + +jobs =3D vm.qmp('query-block-jobs')['return'] +if jobs and jobs[0]['offset'] < jobs[0]['len']: + log('Backup job is still in progress') + +time.sleep(1) + +log('Start NBD server') +srv =3D qemu_nbd_popen('-k', nbd_sock, '-f', iotests.imgfmt, disk_b) + +e =3D vm.event_wait('BLOCK_JOB_COMPLETED') +log('Backup completed: {}'.format(e['data']['offset'])) + +vm.qmp_log('blockdev-del', node_name=3D'backup0') +srv.kill() +vm.shutdown() diff --git a/tests/qemu-iotests/249.out b/tests/qemu-iotests/249.out new file mode 100644 index 0000000000..15a82ddefb --- /dev/null +++ b/tests/qemu-iotests/249.out @@ -0,0 +1,10 @@ +{"execute": "blockdev-add", "arguments": {"driver": "raw", "file": {"drive= r": "nbd", "reconnect-delay": 10, "server": {"path": "TEST_DIR/PID-nbd-sock= ", "type": "unix"}}, "node-name": "backup0"}} +{"return": {}} +{"execute": "blockdev-backup", "arguments": {"device": "drive0", "sync": "= full", "target": "backup0"}} +{"return": {}} +Kill NBD server +Backup job is still in progress +Start NBD server +Backup completed: 5242880 +{"execute": "blockdev-del", "arguments": {"node-name": "backup0"}} +{"return": {}} diff --git a/tests/qemu-iotests/group b/tests/qemu-iotests/group index bae7718380..7ac9a5ea4a 100644 --- a/tests/qemu-iotests/group +++ b/tests/qemu-iotests/group @@ -248,3 +248,4 @@ 246 rw auto quick 247 rw auto quick 248 rw auto quick +249 rw auto quick diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py index 997dc910cb..3a1f16dbfd 100644 --- a/tests/qemu-iotests/iotests.py +++ b/tests/qemu-iotests/iotests.py @@ -218,6 +218,10 @@ def qemu_nbd_pipe(*args): ' '.join(qemu_nbd_args + ['--fork'] + list(args)= ))) return exitcode, subp.communicate()[0] =20 +def qemu_nbd_popen(*args): + '''Run qemu-nbd in daemon mode and return the parent's exit code''' + return subprocess.Popen(qemu_nbd_args + ['--persistent'] + list(args)) + def compare_images(img1, img2, fmt1=3Dimgfmt, fmt2=3Dimgfmt): '''Return True if two image files are identical''' return qemu_img('compare', '-f', fmt1, --=20 2.18.0