From nobody Thu May 2 15:13:04 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1505300707344903.9427469544638; Wed, 13 Sep 2017 04:05:07 -0700 (PDT) Received: from localhost ([::1]:41565 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ds5Te-0001gj-8a for importer@patchew.org; Wed, 13 Sep 2017 07:05:06 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:41754) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ds5Ok-0005os-5a for qemu-devel@nongnu.org; Wed, 13 Sep 2017 07:00:11 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1ds5Oi-00060g-HP for qemu-devel@nongnu.org; Wed, 13 Sep 2017 07:00:02 -0400 Received: from mx1.redhat.com ([209.132.183.28]:38056) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1ds5Oi-0005zp-9k for qemu-devel@nongnu.org; Wed, 13 Sep 2017 07:00:00 -0400 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 54DF5C0587DD for ; Wed, 13 Sep 2017 10:59:59 +0000 (UTC) Received: from secure.mitica (ovpn-117-188.ams2.redhat.com [10.36.117.188]) by smtp.corp.redhat.com (Postfix) with ESMTP id 04FA45C545; Wed, 13 Sep 2017 10:59:57 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com 54DF5C0587DD Authentication-Results: ext-mx08.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx08.extmail.prod.ext.phx2.redhat.com; spf=fail smtp.mailfrom=quintela@redhat.com From: Juan Quintela To: qemu-devel@nongnu.org Date: Wed, 13 Sep 2017 12:59:34 +0200 Message-Id: <20170913105953.13760-2-quintela@redhat.com> In-Reply-To: <20170913105953.13760-1-quintela@redhat.com> References: <20170913105953.13760-1-quintela@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.32]); Wed, 13 Sep 2017 10:59:59 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH v8 01/20] migration: Create migration_ioc_process_incoming() X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: lvivier@redhat.com, dgilbert@redhat.com, peterx@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" We pass the ioc instead of the fd. This will allow us to have more than one channel open. We also make sure that we set the from_src_file sooner, so we don't need to pass it as a parameter. Signed-off-by: Juan Quintela Reviewed-by: Daniel P. Berrange -- Do not assing mis->from_src_file (peterxu) --- migration/channel.c | 3 +-- migration/migration.c | 22 ++++++++++++++++++---- migration/migration.h | 2 ++ 3 files changed, 21 insertions(+), 6 deletions(-) diff --git a/migration/channel.c b/migration/channel.c index 3b7252f5a2..edceebdb7b 100644 --- a/migration/channel.c +++ b/migration/channel.c @@ -36,8 +36,7 @@ void migration_channel_process_incoming(QIOChannel *ioc) error_report_err(local_err); } } else { - QEMUFile *f =3D qemu_fopen_channel_input(ioc); - migration_fd_process_incoming(f); + migration_ioc_process_incoming(ioc); } } =20 diff --git a/migration/migration.c b/migration/migration.c index 959e8ec88e..2d4c56e612 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -306,17 +306,16 @@ static void process_incoming_migration_bh(void *opaqu= e) =20 static void process_incoming_migration_co(void *opaque) { - QEMUFile *f =3D opaque; MigrationIncomingState *mis =3D migration_incoming_get_current(); PostcopyState ps; int ret; =20 - mis->from_src_file =3D f; + assert(mis->from_src_file); mis->largest_page_size =3D qemu_ram_pagesize_largest(); postcopy_state_set(POSTCOPY_INCOMING_NONE); migrate_set_state(&mis->state, MIGRATION_STATUS_NONE, MIGRATION_STATUS_ACTIVE); - ret =3D qemu_loadvm_state(f); + ret =3D qemu_loadvm_state(mis->from_src_file); =20 ps =3D postcopy_state_get(); trace_process_incoming_migration_co_end(ret, ps); @@ -364,12 +363,27 @@ static void process_incoming_migration_co(void *opaqu= e) =20 void migration_fd_process_incoming(QEMUFile *f) { - Coroutine *co =3D qemu_coroutine_create(process_incoming_migration_co,= f); + Coroutine *co =3D qemu_coroutine_create(process_incoming_migration_co,= NULL); + MigrationIncomingState *mis =3D migration_incoming_get_current(); =20 + if (!mis->from_src_file) { + mis->from_src_file =3D f; + } qemu_file_set_blocking(f, false); qemu_coroutine_enter(co); } =20 +void migration_ioc_process_incoming(QIOChannel *ioc) +{ + MigrationIncomingState *mis =3D migration_incoming_get_current(); + + if (!mis->from_src_file) { + QEMUFile *f =3D qemu_fopen_channel_input(ioc); + migration_fd_process_incoming(f); + } + /* We still only have a single channel. Nothing to do here yet */ +} + /* * Send a 'SHUT' message on the return channel with the given value * to indicate that we've finished with the RP. Non-0 value indicates diff --git a/migration/migration.h b/migration/migration.h index 148c9facbc..99c398d484 100644 --- a/migration/migration.h +++ b/migration/migration.h @@ -20,6 +20,7 @@ #include "exec/cpu-common.h" #include "qemu/coroutine_int.h" #include "hw/qdev.h" +#include "io/channel.h" =20 /* State for the incoming migration */ struct MigrationIncomingState { @@ -152,6 +153,7 @@ struct MigrationState void migrate_set_state(int *state, int old_state, int new_state); =20 void migration_fd_process_incoming(QEMUFile *f); +void migration_ioc_process_incoming(QIOChannel *ioc); =20 uint64_t migrate_max_downtime(void); =20 --=20 2.13.5 From nobody Thu May 2 15:13:04 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1505300871369981.4690446386212; Wed, 13 Sep 2017 04:07:51 -0700 (PDT) Received: from localhost ([::1]:41582 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ds5WH-0004Bg-BG for importer@patchew.org; Wed, 13 Sep 2017 07:07:49 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:41789) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ds5Oo-0005uN-MZ for qemu-devel@nongnu.org; Wed, 13 Sep 2017 07:00:12 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1ds5Ok-00062i-5K for qemu-devel@nongnu.org; Wed, 13 Sep 2017 07:00:06 -0400 Received: from mx1.redhat.com ([209.132.183.28]:51132) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1ds5Ok-000625-0T for qemu-devel@nongnu.org; Wed, 13 Sep 2017 07:00:02 -0400 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 063CD7EA8B for ; Wed, 13 Sep 2017 11:00:01 +0000 (UTC) Received: from secure.mitica (ovpn-117-188.ams2.redhat.com [10.36.117.188]) by smtp.corp.redhat.com (Postfix) with ESMTP id A8FF61717C; Wed, 13 Sep 2017 10:59:59 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com 063CD7EA8B Authentication-Results: ext-mx04.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx04.extmail.prod.ext.phx2.redhat.com; spf=fail smtp.mailfrom=quintela@redhat.com From: Juan Quintela To: qemu-devel@nongnu.org Date: Wed, 13 Sep 2017 12:59:35 +0200 Message-Id: <20170913105953.13760-3-quintela@redhat.com> In-Reply-To: <20170913105953.13760-1-quintela@redhat.com> References: <20170913105953.13760-1-quintela@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.28]); Wed, 13 Sep 2017 11:00:01 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH v8 02/20] migration: Teach it about G_SOURCE_REMOVE X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: lvivier@redhat.com, dgilbert@redhat.com, peterx@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" As this is defined on glib 2.32, add compatibility macros for older glibs. Signed-off-by: Juan Quintela Reviewed-by: Daniel P. Berrange Reviewed-by: Peter Xu --- include/glib-compat.h | 2 ++ migration/exec.c | 2 +- migration/fd.c | 2 +- migration/socket.c | 2 +- 4 files changed, 5 insertions(+), 3 deletions(-) diff --git a/include/glib-compat.h b/include/glib-compat.h index fcffcd3f07..e15aca2d40 100644 --- a/include/glib-compat.h +++ b/include/glib-compat.h @@ -223,6 +223,8 @@ static inline gboolean g_hash_table_contains(GHashTable= *hash_table, { return g_hash_table_lookup_extended(hash_table, key, NULL, NULL); } +#define G_SOURCE_CONTINUE TRUE +#define G_SOURCE_REMOVE FALSE #endif =20 #ifndef g_assert_true diff --git a/migration/exec.c b/migration/exec.c index 08b599e0e2..f3be1baf2e 100644 --- a/migration/exec.c +++ b/migration/exec.c @@ -49,7 +49,7 @@ static gboolean exec_accept_incoming_migration(QIOChannel= *ioc, { migration_channel_process_incoming(ioc); object_unref(OBJECT(ioc)); - return FALSE; /* unregister */ + return G_SOURCE_REMOVE; } =20 void exec_start_incoming_migration(const char *command, Error **errp) diff --git a/migration/fd.c b/migration/fd.c index 30f5258a6a..30de4b9847 100644 --- a/migration/fd.c +++ b/migration/fd.c @@ -49,7 +49,7 @@ static gboolean fd_accept_incoming_migration(QIOChannel *= ioc, { migration_channel_process_incoming(ioc); object_unref(OBJECT(ioc)); - return FALSE; /* unregister */ + return G_SOURCE_REMOVE; } =20 void fd_start_incoming_migration(const char *infd, Error **errp) diff --git a/migration/socket.c b/migration/socket.c index 757d3821a1..b02d37d7a3 100644 --- a/migration/socket.c +++ b/migration/socket.c @@ -154,7 +154,7 @@ static gboolean socket_accept_incoming_migration(QIOCha= nnel *ioc, out: /* Close listening socket as its no longer needed */ qio_channel_close(ioc, NULL); - return FALSE; /* unregister */ + return G_SOURCE_REMOVE; } =20 =20 --=20 2.13.5 From nobody Thu May 2 15:13:04 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1505300756277796.3325294369092; Wed, 13 Sep 2017 04:05:56 -0700 (PDT) Received: from localhost ([::1]:41575 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ds5UR-0002TC-Hs for importer@patchew.org; Wed, 13 Sep 2017 07:05:55 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:41884) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ds5Ov-00067n-Mk for qemu-devel@nongnu.org; Wed, 13 Sep 2017 07:00:14 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1ds5Oq-0006Dy-5e for qemu-devel@nongnu.org; Wed, 13 Sep 2017 07:00:13 -0400 Received: from mx1.redhat.com ([209.132.183.28]:38386) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1ds5Oq-0006Bf-0o for qemu-devel@nongnu.org; Wed, 13 Sep 2017 07:00:08 -0400 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 0629CC058EAF for ; Wed, 13 Sep 2017 11:00:07 +0000 (UTC) Received: from secure.mitica (ovpn-117-188.ams2.redhat.com [10.36.117.188]) by smtp.corp.redhat.com (Postfix) with ESMTP id 5DC605191B; Wed, 13 Sep 2017 11:00:01 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com 0629CC058EAF Authentication-Results: ext-mx08.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx08.extmail.prod.ext.phx2.redhat.com; spf=fail smtp.mailfrom=quintela@redhat.com From: Juan Quintela To: qemu-devel@nongnu.org Date: Wed, 13 Sep 2017 12:59:36 +0200 Message-Id: <20170913105953.13760-4-quintela@redhat.com> In-Reply-To: <20170913105953.13760-1-quintela@redhat.com> References: <20170913105953.13760-1-quintela@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.32]); Wed, 13 Sep 2017 11:00:07 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH v8 03/20] migration: Add comments to channel functions X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: lvivier@redhat.com, dgilbert@redhat.com, peterx@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Signed-off-by: Juan Quintela Reviewed-by: Daniel P. Berrange --- migration/channel.c | 15 +++++++++++++++ 1 file changed, 15 insertions(+) diff --git a/migration/channel.c b/migration/channel.c index edceebdb7b..70ec7ea3b7 100644 --- a/migration/channel.c +++ b/migration/channel.c @@ -19,6 +19,14 @@ #include "qapi/error.h" #include "io/channel-tls.h" =20 +/** + * @migration_channel_process_incoming - Create new incoming migration cha= nnel + * + * Notice that TLS is special. For it we listen in a listener socket, + * and then create a new client socket from the TLS library. + * + * @ioc: Channel to which we are connecting + */ void migration_channel_process_incoming(QIOChannel *ioc) { MigrationState *s =3D migrate_get_current(); @@ -41,6 +49,13 @@ void migration_channel_process_incoming(QIOChannel *ioc) } =20 =20 +/** + * @migration_channel_connect - Create new outgoing migration channel + * + * @s: Current migration state + * @ioc: Channel to which we are connecting + * @hostname: Where we want to connect + */ void migration_channel_connect(MigrationState *s, QIOChannel *ioc, const char *hostname) --=20 2.13.5 From nobody Thu May 2 15:13:04 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1505301029997161.24137532311363; Wed, 13 Sep 2017 04:10:29 -0700 (PDT) Received: from localhost ([::1]:41593 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ds5Yq-0006it-Q3 for importer@patchew.org; Wed, 13 Sep 2017 07:10:28 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:41840) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ds5Ot-00062Y-9U for qemu-devel@nongnu.org; Wed, 13 Sep 2017 07:00:13 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1ds5Os-0006Iv-7i for qemu-devel@nongnu.org; Wed, 13 Sep 2017 07:00:11 -0400 Received: from mx1.redhat.com ([209.132.183.28]:38566) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1ds5Os-0006Gi-2a for qemu-devel@nongnu.org; Wed, 13 Sep 2017 07:00:10 -0400 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 138BFC0587DF for ; Wed, 13 Sep 2017 11:00:09 +0000 (UTC) Received: from secure.mitica (ovpn-117-188.ams2.redhat.com [10.36.117.188]) by smtp.corp.redhat.com (Postfix) with ESMTP id 58D4D51919; Wed, 13 Sep 2017 11:00:07 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com 138BFC0587DF Authentication-Results: ext-mx08.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx08.extmail.prod.ext.phx2.redhat.com; spf=fail smtp.mailfrom=quintela@redhat.com From: Juan Quintela To: qemu-devel@nongnu.org Date: Wed, 13 Sep 2017 12:59:37 +0200 Message-Id: <20170913105953.13760-5-quintela@redhat.com> In-Reply-To: <20170913105953.13760-1-quintela@redhat.com> References: <20170913105953.13760-1-quintela@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.32]); Wed, 13 Sep 2017 11:00:09 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH v8 04/20] migration: Create migration_has_all_channels X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: lvivier@redhat.com, dgilbert@redhat.com, peterx@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" This function allows us to decide when to close the listener socket. For now, we only need one connection. Signed-off-by: Juan Quintela Reviewed-by: Daniel P. Berrange --- migration/migration.c | 11 +++++++++++ migration/migration.h | 2 ++ migration/socket.c | 10 +++++++--- 3 files changed, 20 insertions(+), 3 deletions(-) diff --git a/migration/migration.c b/migration/migration.c index 2d4c56e612..bac4a99277 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -384,6 +384,17 @@ void migration_ioc_process_incoming(QIOChannel *ioc) /* We still only have a single channel. Nothing to do here yet */ } =20 +/** + * @migration_has_all_channels: We have received all channels that we need + * + * Returns true when we have got connections to all the channels that + * we need for migration. + */ +bool migration_has_all_channels(void) +{ + return true; +} + /* * Send a 'SHUT' message on the return channel with the given value * to indicate that we've finished with the RP. Non-0 value indicates diff --git a/migration/migration.h b/migration/migration.h index 99c398d484..1881e4a754 100644 --- a/migration/migration.h +++ b/migration/migration.h @@ -155,6 +155,8 @@ void migrate_set_state(int *state, int old_state, int n= ew_state); void migration_fd_process_incoming(QEMUFile *f); void migration_ioc_process_incoming(QIOChannel *ioc); =20 +bool migration_has_all_channels(void); + uint64_t migrate_max_downtime(void); =20 void migrate_fd_error(MigrationState *s, const Error *error); diff --git a/migration/socket.c b/migration/socket.c index b02d37d7a3..dee869044a 100644 --- a/migration/socket.c +++ b/migration/socket.c @@ -152,9 +152,13 @@ static gboolean socket_accept_incoming_migration(QIOCh= annel *ioc, object_unref(OBJECT(sioc)); =20 out: - /* Close listening socket as its no longer needed */ - qio_channel_close(ioc, NULL); - return G_SOURCE_REMOVE; + if (migration_has_all_channels()) { + /* Close listening socket as its no longer needed */ + qio_channel_close(ioc, NULL); + return G_SOURCE_REMOVE; + } else { + return G_SOURCE_CONTINUE; + } } =20 =20 --=20 2.13.5 From nobody Thu May 2 15:13:04 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1505300918404416.26272224957097; Wed, 13 Sep 2017 04:08:38 -0700 (PDT) Received: from localhost ([::1]:41587 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ds5X3-0004xl-HU for importer@patchew.org; Wed, 13 Sep 2017 07:08:37 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:41872) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ds5Ov-00066n-6d for qemu-devel@nongnu.org; Wed, 13 Sep 2017 07:00:18 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1ds5Ou-0006Pe-7Q for qemu-devel@nongnu.org; Wed, 13 Sep 2017 07:00:13 -0400 Received: from mx1.redhat.com ([209.132.183.28]:45052) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1ds5Ou-0006O9-00 for qemu-devel@nongnu.org; Wed, 13 Sep 2017 07:00:12 -0400 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 04EDA81E14 for ; Wed, 13 Sep 2017 11:00:11 +0000 (UTC) Received: from secure.mitica (ovpn-117-188.ams2.redhat.com [10.36.117.188]) by smtp.corp.redhat.com (Postfix) with ESMTP id 627A461798; Wed, 13 Sep 2017 11:00:09 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com 04EDA81E14 Authentication-Results: ext-mx01.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx01.extmail.prod.ext.phx2.redhat.com; spf=fail smtp.mailfrom=quintela@redhat.com From: Juan Quintela To: qemu-devel@nongnu.org Date: Wed, 13 Sep 2017 12:59:38 +0200 Message-Id: <20170913105953.13760-6-quintela@redhat.com> In-Reply-To: <20170913105953.13760-1-quintela@redhat.com> References: <20170913105953.13760-1-quintela@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.25]); Wed, 13 Sep 2017 11:00:11 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH v8 05/20] migration: Improve migration thread error handling X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: lvivier@redhat.com, dgilbert@redhat.com, peterx@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" We now report errors also when we finish migration, not only on info migrate. We plan to use this error from several places, and we want the first error to happen to win, so we add an mutex to order it. Signed-off-by: Juan Quintela Reviewed-by: Dr. David Alan Gilbert --- migration/migration.c | 19 ++++++++++++++++--- migration/migration.h | 7 ++++++- migration/ram.c | 2 +- migration/tls.c | 1 - 4 files changed, 23 insertions(+), 6 deletions(-) diff --git a/migration/migration.c b/migration/migration.c index bac4a99277..adc07e442a 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -1013,19 +1013,30 @@ static void migrate_fd_cleanup(void *opaque) MIGRATION_STATUS_CANCELLED); } =20 + if (s->error) { + /* It is used on info migrate. We can't free it */ + error_report_err(error_copy(s->error)); + } notifier_list_notify(&migration_state_notifiers, s); block_cleanup_parameters(s); } =20 +void migrate_set_error(MigrationState *s, const Error *error) +{ + qemu_mutex_lock(&s->error_mutex); + if (!s->error) { + s->error =3D error_copy(error); + } + qemu_mutex_unlock(&s->error_mutex); +} + void migrate_fd_error(MigrationState *s, const Error *error) { trace_migrate_fd_error(error_get_pretty(error)); assert(s->to_dst_file =3D=3D NULL); migrate_set_state(&s->state, MIGRATION_STATUS_SETUP, MIGRATION_STATUS_FAILED); - if (!s->error) { - s->error =3D error_copy(error); - } + migrate_set_error(s, error); notifier_list_notify(&migration_state_notifiers, s); block_cleanup_parameters(s); } @@ -2244,6 +2255,7 @@ static void migration_instance_finalize(Object *obj) MigrationState *ms =3D MIGRATION_OBJ(obj); MigrationParameters *params =3D &ms->parameters; =20 + qemu_mutex_destroy(&ms->error_mutex); g_free(params->tls_hostname); g_free(params->tls_creds); } @@ -2256,6 +2268,7 @@ static void migration_instance_init(Object *obj) ms->state =3D MIGRATION_STATUS_NONE; ms->xbzrle_cache_size =3D DEFAULT_MIGRATE_CACHE_SIZE; ms->mbps =3D -1; + qemu_mutex_init(&ms->error_mutex); =20 params->tls_hostname =3D g_strdup(""); params->tls_creds =3D g_strdup(""); diff --git a/migration/migration.h b/migration/migration.h index 1881e4a754..9a81b8a70a 100644 --- a/migration/migration.h +++ b/migration/migration.h @@ -129,8 +129,12 @@ struct MigrationState int64_t colo_checkpoint_time; QEMUTimer *colo_delay_timer; =20 - /* The last error that occurred */ + /* The first error that has occurred. + We used the mutex to be able to return the 1st error message */ Error *error; + /* mutex to protect errp */ + QemuMutex error_mutex; + /* Do we have to clean up -b/-i from old migrate parameters */ /* This feature is deprecated and will be removed */ bool must_remove_block_options; @@ -159,6 +163,7 @@ bool migration_has_all_channels(void); =20 uint64_t migrate_max_downtime(void); =20 +void migrate_set_error(MigrationState *s, const Error *error); void migrate_fd_error(MigrationState *s, const Error *error); =20 void migrate_fd_connect(MigrationState *s); diff --git a/migration/ram.c b/migration/ram.c index e18b3e2d4f..e0179fc838 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -1789,7 +1789,7 @@ int ram_discard_range(const char *rbname, uint64_t st= art, size_t length) RAMBlock *rb =3D qemu_ram_block_by_name(rbname); =20 if (!rb) { - error_report("ram_discard_range: Failed to find block '%s'", rbnam= e); + error_report("ram_discard_rang0e: Failed to find block '%s'", rbna= me); goto err; } =20 diff --git a/migration/tls.c b/migration/tls.c index 596e8790bd..026a008667 100644 --- a/migration/tls.c +++ b/migration/tls.c @@ -119,7 +119,6 @@ static void migration_tls_outgoing_handshake(QIOTask *t= ask, if (qio_task_propagate_error(task, &err)) { trace_migration_tls_outgoing_handshake_error(error_get_pretty(err)= ); migrate_fd_error(s, err); - error_free(err); } else { trace_migration_tls_outgoing_handshake_complete(); migration_channel_connect(s, ioc, NULL); --=20 2.13.5 From nobody Thu May 2 15:13:04 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 150530054968179.27860240261055; Wed, 13 Sep 2017 04:02:29 -0700 (PDT) Received: from localhost ([::1]:41558 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ds5R6-0007jb-RR for importer@patchew.org; Wed, 13 Sep 2017 07:02:28 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:42074) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ds5P8-0006IC-3E for qemu-devel@nongnu.org; Wed, 13 Sep 2017 07:00:30 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1ds5P4-0006bK-Vc for qemu-devel@nongnu.org; Wed, 13 Sep 2017 07:00:26 -0400 Received: from mx1.redhat.com ([209.132.183.28]:39308) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1ds5P4-0006aq-Mu for qemu-devel@nongnu.org; Wed, 13 Sep 2017 07:00:22 -0400 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 9EE1BC058EB1 for ; Wed, 13 Sep 2017 11:00:21 +0000 (UTC) Received: from secure.mitica (ovpn-117-188.ams2.redhat.com [10.36.117.188]) by smtp.corp.redhat.com (Postfix) with ESMTP id 58C165C545; Wed, 13 Sep 2017 11:00:11 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com 9EE1BC058EB1 Authentication-Results: ext-mx08.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx08.extmail.prod.ext.phx2.redhat.com; spf=fail smtp.mailfrom=quintela@redhat.com From: Juan Quintela To: qemu-devel@nongnu.org Date: Wed, 13 Sep 2017 12:59:39 +0200 Message-Id: <20170913105953.13760-7-quintela@redhat.com> In-Reply-To: <20170913105953.13760-1-quintela@redhat.com> References: <20170913105953.13760-1-quintela@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.32]); Wed, 13 Sep 2017 11:00:21 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH v8 06/20] migration: Make migrate_fd_error() the owner of the Error X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: lvivier@redhat.com, dgilbert@redhat.com, peterx@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" So far, we had to free the error after each caller, so just do it here. Once there, tls.c was leaking the error. Signed-off-by: Juan Quintela --- migration/channel.c | 1 - migration/migration.c | 10 ++++------ migration/migration.h | 4 ++-- migration/socket.c | 1 - 4 files changed, 6 insertions(+), 10 deletions(-) diff --git a/migration/channel.c b/migration/channel.c index 70ec7ea3b7..1dd2ae1530 100644 --- a/migration/channel.c +++ b/migration/channel.c @@ -71,7 +71,6 @@ void migration_channel_connect(MigrationState *s, migration_tls_channel_connect(s, ioc, hostname, &local_err); if (local_err) { migrate_fd_error(s, local_err); - error_free(local_err); } } else { QEMUFile *f =3D qemu_fopen_channel_output(ioc); diff --git a/migration/migration.c b/migration/migration.c index adc07e442a..b15f0ba876 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -1021,16 +1021,14 @@ static void migrate_fd_cleanup(void *opaque) block_cleanup_parameters(s); } =20 -void migrate_set_error(MigrationState *s, const Error *error) +void migrate_set_error(MigrationState *s, Error *error) { qemu_mutex_lock(&s->error_mutex); - if (!s->error) { - s->error =3D error_copy(error); - } + error_propagate(&s->error, error); qemu_mutex_unlock(&s->error_mutex); } =20 -void migrate_fd_error(MigrationState *s, const Error *error) +void migrate_fd_error(MigrationState *s, Error *error) { trace_migrate_fd_error(error_get_pretty(error)); assert(s->to_dst_file =3D=3D NULL); @@ -1304,7 +1302,7 @@ void qmp_migrate(const char *uri, bool has_blk, bool = blk, } =20 if (local_err) { - migrate_fd_error(s, local_err); + migrate_fd_error(s, error_copy(local_err)); error_propagate(errp, local_err); return; } diff --git a/migration/migration.h b/migration/migration.h index 9a81b8a70a..7ddb0cdc1a 100644 --- a/migration/migration.h +++ b/migration/migration.h @@ -163,8 +163,8 @@ bool migration_has_all_channels(void); =20 uint64_t migrate_max_downtime(void); =20 -void migrate_set_error(MigrationState *s, const Error *error); -void migrate_fd_error(MigrationState *s, const Error *error); +void migrate_set_error(MigrationState *s, Error *error); +void migrate_fd_error(MigrationState *s, Error *error); =20 void migrate_fd_connect(MigrationState *s); =20 diff --git a/migration/socket.c b/migration/socket.c index dee869044a..2d70747a1a 100644 --- a/migration/socket.c +++ b/migration/socket.c @@ -80,7 +80,6 @@ static void socket_outgoing_migration(QIOTask *task, if (qio_task_propagate_error(task, &err)) { trace_migration_socket_outgoing_error(error_get_pretty(err)); migrate_fd_error(data->s, err); - error_free(err); } else { trace_migration_socket_outgoing_connected(data->hostname); migration_channel_connect(data->s, sioc, data->hostname); --=20 2.13.5 From nobody Thu May 2 15:13:04 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1505300552255713.0844593500644; Wed, 13 Sep 2017 04:02:32 -0700 (PDT) Received: from localhost ([::1]:41559 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ds5R9-0007oR-FQ for importer@patchew.org; Wed, 13 Sep 2017 07:02:31 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:42113) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ds5PA-0006K5-Gq for qemu-devel@nongnu.org; Wed, 13 Sep 2017 07:00:34 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1ds5P6-0006eE-LY for qemu-devel@nongnu.org; Wed, 13 Sep 2017 07:00:28 -0400 Received: from mx1.redhat.com ([209.132.183.28]:56160) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1ds5P6-0006c3-GD for qemu-devel@nongnu.org; Wed, 13 Sep 2017 07:00:24 -0400 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 6DB6F4ACC7 for ; Wed, 13 Sep 2017 11:00:23 +0000 (UTC) Received: from secure.mitica (ovpn-117-188.ams2.redhat.com [10.36.117.188]) by smtp.corp.redhat.com (Postfix) with ESMTP id F38F1177CB; Wed, 13 Sep 2017 11:00:21 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com 6DB6F4ACC7 Authentication-Results: ext-mx09.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx09.extmail.prod.ext.phx2.redhat.com; spf=fail smtp.mailfrom=quintela@redhat.com From: Juan Quintela To: qemu-devel@nongnu.org Date: Wed, 13 Sep 2017 12:59:40 +0200 Message-Id: <20170913105953.13760-8-quintela@redhat.com> In-Reply-To: <20170913105953.13760-1-quintela@redhat.com> References: <20170913105953.13760-1-quintela@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.38]); Wed, 13 Sep 2017 11:00:23 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH v8 07/20] migration: Add multifd capability X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: lvivier@redhat.com, dgilbert@redhat.com, peterx@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Signed-off-by: Juan Quintela Reviewed-by: Dr. David Alan Gilbert Reviewed-by: Peter Xu Reviewed-by: Daniel P. Berrange -- Use new DEFINE_PROP --- migration/migration.c | 10 ++++++++++ migration/migration.h | 1 + qapi/migration.json | 4 +++- 3 files changed, 14 insertions(+), 1 deletion(-) diff --git a/migration/migration.c b/migration/migration.c index b15f0ba876..ef5ce6c55d 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -1458,6 +1458,15 @@ bool migrate_use_events(void) return s->enabled_capabilities[MIGRATION_CAPABILITY_EVENTS]; } =20 +bool migrate_use_multifd(void) +{ + MigrationState *s; + + s =3D migrate_get_current(); + + return s->enabled_capabilities[MIGRATION_CAPABILITY_X_MULTIFD]; +} + int migrate_use_xbzrle(void) { MigrationState *s; @@ -2236,6 +2245,7 @@ static Property migration_properties[] =3D { DEFINE_PROP_MIG_CAP("x-release-ram", MIGRATION_CAPABILITY_RELEASE_RAM), DEFINE_PROP_MIG_CAP("x-block", MIGRATION_CAPABILITY_BLOCK), DEFINE_PROP_MIG_CAP("x-return-path", MIGRATION_CAPABILITY_RETURN_PATH), + DEFINE_PROP_MIG_CAP("x-multifd", MIGRATION_CAPABILITY_X_MULTIFD), =20 DEFINE_PROP_END_OF_LIST(), }; diff --git a/migration/migration.h b/migration/migration.h index 7ddb0cdc1a..023b14aa08 100644 --- a/migration/migration.h +++ b/migration/migration.h @@ -179,6 +179,7 @@ bool migrate_postcopy_ram(void); bool migrate_zero_blocks(void); =20 bool migrate_auto_converge(void); +bool migrate_use_multifd(void); =20 int migrate_use_xbzrle(void); int64_t migrate_xbzrle_cache_size(void); diff --git a/qapi/migration.json b/qapi/migration.json index ee2b3b8733..ec4a88a43a 100644 --- a/qapi/migration.json +++ b/qapi/migration.json @@ -341,12 +341,14 @@ # @return-path: If enabled, migration will use the return path even # for precopy. (since 2.10) # +# @x-multifd: Use more than one fd for migration (since 2.11) +# # Since: 1.2 ## { 'enum': 'MigrationCapability', 'data': ['xbzrle', 'rdma-pin-all', 'auto-converge', 'zero-blocks', 'compress', 'events', 'postcopy-ram', 'x-colo', 'release-ram', - 'block', 'return-path' ] } + 'block', 'return-path', 'x-multifd' ] } =20 ## # @MigrationCapabilityStatus: --=20 2.13.5 From nobody Thu May 2 15:13:04 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 15053007235981009.7285733707846; Wed, 13 Sep 2017 04:05:23 -0700 (PDT) Received: from localhost ([::1]:41568 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ds5Tu-0001vC-Os for importer@patchew.org; Wed, 13 Sep 2017 07:05:22 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:42263) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ds5PI-0006Qd-K3 for qemu-devel@nongnu.org; Wed, 13 Sep 2017 07:00:39 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1ds5P8-0006gz-HF for qemu-devel@nongnu.org; Wed, 13 Sep 2017 07:00:36 -0400 Received: from mx1.redhat.com ([209.132.183.28]:42810) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1ds5P8-0006gO-90 for qemu-devel@nongnu.org; Wed, 13 Sep 2017 07:00:26 -0400 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 4426F461DB for ; Wed, 13 Sep 2017 11:00:25 +0000 (UTC) Received: from secure.mitica (ovpn-117-188.ams2.redhat.com [10.36.117.188]) by smtp.corp.redhat.com (Postfix) with ESMTP id C0E8C5C545; Wed, 13 Sep 2017 11:00:23 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com 4426F461DB Authentication-Results: ext-mx05.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx05.extmail.prod.ext.phx2.redhat.com; spf=fail smtp.mailfrom=quintela@redhat.com From: Juan Quintela To: qemu-devel@nongnu.org Date: Wed, 13 Sep 2017 12:59:41 +0200 Message-Id: <20170913105953.13760-9-quintela@redhat.com> In-Reply-To: <20170913105953.13760-1-quintela@redhat.com> References: <20170913105953.13760-1-quintela@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.29]); Wed, 13 Sep 2017 11:00:25 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH v8 08/20] migration: Create x-multifd-channels parameter X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: lvivier@redhat.com, dgilbert@redhat.com, peterx@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Indicates the number of channels that we will create. By default we create 2 channels. Signed-off-by: Juan Quintela Reviewed-by: Dr. David Alan Gilbert Reviewed-by: Peter Xu -- Catch inconsistent defaults (eric). Improve comment stating that number of threads is the same than number of sockets Use new DEFIN_PROP_* Rename x-multifd-threads to x-multifd-threads --- hmp.c | 7 +++++++ migration/migration.c | 26 ++++++++++++++++++++++++++ migration/migration.h | 1 + qapi/migration.json | 24 +++++++++++++++++++++--- 4 files changed, 55 insertions(+), 3 deletions(-) diff --git a/hmp.c b/hmp.c index cd046c6d71..1f9e39529f 100644 --- a/hmp.c +++ b/hmp.c @@ -336,6 +336,9 @@ void hmp_info_migrate_parameters(Monitor *mon, const QD= ict *qdict) monitor_printf(mon, "%s: %s\n", MigrationParameter_str(MIGRATION_PARAMETER_BLOCK_INCREMENTAL), params->block_incremental ? "on" : "off"); + monitor_printf(mon, "%s: %" PRId64 "\n", + MigrationParameter_str(MIGRATION_PARAMETER_X_MULTIFD_CHANNELS), + params->x_multifd_channels); } =20 qapi_free_MigrationParameters(params); @@ -1621,6 +1624,10 @@ void hmp_migrate_set_parameter(Monitor *mon, const Q= Dict *qdict) p->has_block_incremental =3D true; visit_type_bool(v, param, &p->block_incremental, &err); break; + case MIGRATION_PARAMETER_X_MULTIFD_CHANNELS: + p->has_x_multifd_channels =3D true; + visit_type_int(v, param, &p->x_multifd_channels, &err); + break; default: assert(0); } diff --git a/migration/migration.c b/migration/migration.c index ef5ce6c55d..021586f9f8 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -77,6 +77,7 @@ * Note: Please change this default value to 10000 when we support hybrid = mode. */ #define DEFAULT_MIGRATE_X_CHECKPOINT_DELAY 200 +#define DEFAULT_MIGRATE_MULTIFD_CHANNELS 2 =20 static NotifierList migration_state_notifiers =3D NOTIFIER_LIST_INITIALIZER(migration_state_notifiers); @@ -481,6 +482,8 @@ MigrationParameters *qmp_query_migrate_parameters(Error= **errp) params->x_checkpoint_delay =3D s->parameters.x_checkpoint_delay; params->has_block_incremental =3D true; params->block_incremental =3D s->parameters.block_incremental; + params->has_x_multifd_channels =3D true; + params->x_multifd_channels =3D s->parameters.x_multifd_channels; =20 return params; } @@ -762,6 +765,13 @@ static bool migrate_params_check(MigrationParameters *= params, Error **errp) "is invalid, it should be positive"); return false; } + if (params->has_x_multifd_channels && + (params->x_multifd_channels < 1 || params->x_multifd_channels > 25= 5)) { + error_setg(errp, QERR_INVALID_PARAMETER_VALUE, + "multifd_channels", + "is invalid, it should be in the range of 1 to 255"); + return false; + } =20 return true; } @@ -880,6 +890,9 @@ static void migrate_params_apply(MigrateSetParameters *= params) if (params->has_block_incremental) { s->parameters.block_incremental =3D params->block_incremental; } + if (params->has_x_multifd_channels) { + s->parameters.x_multifd_channels =3D params->x_multifd_channels; + } } =20 void qmp_migrate_set_parameters(MigrateSetParameters *params, Error **errp) @@ -1467,6 +1480,15 @@ bool migrate_use_multifd(void) return s->enabled_capabilities[MIGRATION_CAPABILITY_X_MULTIFD]; } =20 +int migrate_multifd_channels(void) +{ + MigrationState *s; + + s =3D migrate_get_current(); + + return s->parameters.x_multifd_channels; +} + int migrate_use_xbzrle(void) { MigrationState *s; @@ -2232,6 +2254,9 @@ static Property migration_properties[] =3D { DEFINE_PROP_INT64("x-checkpoint-delay", MigrationState, parameters.x_checkpoint_delay, DEFAULT_MIGRATE_X_CHECKPOINT_DELAY), + DEFINE_PROP_INT64("x-multifd-channels", MigrationState, + parameters.x_multifd_channels, + DEFAULT_MIGRATE_MULTIFD_CHANNELS), =20 /* Migration capabilities */ DEFINE_PROP_MIG_CAP("x-xbzrle", MIGRATION_CAPABILITY_XBZRLE), @@ -2291,6 +2316,7 @@ static void migration_instance_init(Object *obj) params->has_downtime_limit =3D true; params->has_x_checkpoint_delay =3D true; params->has_block_incremental =3D true; + params->has_x_multifd_channels =3D true; } =20 /* diff --git a/migration/migration.h b/migration/migration.h index 023b14aa08..a6a7c41823 100644 --- a/migration/migration.h +++ b/migration/migration.h @@ -180,6 +180,7 @@ bool migrate_zero_blocks(void); =20 bool migrate_auto_converge(void); bool migrate_use_multifd(void); +int migrate_multifd_channels(void); =20 int migrate_use_xbzrle(void); int64_t migrate_xbzrle_cache_size(void); diff --git a/qapi/migration.json b/qapi/migration.json index ec4a88a43a..c766fb1e52 100644 --- a/qapi/migration.json +++ b/qapi/migration.json @@ -466,13 +466,19 @@ # migrated and the destination must already have access to the # same backing chain as was used on the source. (since 2.10) # +# @x-multifd-channels: Number of channels used to migrate data in +# parallel. This is the same number that the +# number of sockets used for migration. The +# default value is 2 (since 2.11) +# # Since: 2.4 ## { 'enum': 'MigrationParameter', 'data': ['compress-level', 'compress-threads', 'decompress-threads', 'cpu-throttle-initial', 'cpu-throttle-increment', 'tls-creds', 'tls-hostname', 'max-bandwidth', - 'downtime-limit', 'x-checkpoint-delay', 'block-incremental' ] } + 'downtime-limit', 'x-checkpoint-delay', 'block-incremental', + 'x-multifd-channels'] } =20 ## # @MigrateSetParameters: @@ -528,6 +534,11 @@ # migrated and the destination must already have access to the # same backing chain as was used on the source. (since 2.10) # +# @x-multifd-channels: Number of channels used to migrate data in +# parallel. This is the same number that the +# number of sockets used for migration. The +# default value is 2 (since 2.11) +# # Since: 2.4 ## # TODO either fuse back into MigrationParameters, or make @@ -543,7 +554,8 @@ '*max-bandwidth': 'int', '*downtime-limit': 'int', '*x-checkpoint-delay': 'int', - '*block-incremental': 'bool' } } + '*block-incremental': 'bool', + '*x-multifd-channels': 'int' } } =20 ## # @migrate-set-parameters: @@ -614,6 +626,11 @@ # migrated and the destination must already have access to the # same backing chain as was used on the source. (since 2.10) # +# @x-multifd-channels: Number of channels used to migrate data in +# parallel. This is the same number that the +# number of sockets used for migration. +# The default value is 2 (since 2.11) +# # Since: 2.4 ## { 'struct': 'MigrationParameters', @@ -627,7 +644,8 @@ '*max-bandwidth': 'int', '*downtime-limit': 'int', '*x-checkpoint-delay': 'int', - '*block-incremental': 'bool' } } + '*block-incremental': 'bool' , + '*x-multifd-channels': 'int' } } =20 ## # @query-migrate-parameters: --=20 2.13.5 From nobody Thu May 2 15:13:04 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1505300887940919.0478591979937; Wed, 13 Sep 2017 04:08:07 -0700 (PDT) Received: from localhost ([::1]:41583 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ds5WZ-0004Qp-7X for importer@patchew.org; Wed, 13 Sep 2017 07:08:07 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:42228) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ds5PG-0006Pq-Lz for qemu-devel@nongnu.org; Wed, 13 Sep 2017 07:00:41 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1ds5PA-0006ic-CC for qemu-devel@nongnu.org; Wed, 13 Sep 2017 07:00:34 -0400 Received: from mx1.redhat.com ([209.132.183.28]:55692) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1ds5PA-0006hf-4j for qemu-devel@nongnu.org; Wed, 13 Sep 2017 07:00:28 -0400 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 2C366883AC for ; Wed, 13 Sep 2017 11:00:27 +0000 (UTC) Received: from secure.mitica (ovpn-117-188.ams2.redhat.com [10.36.117.188]) by smtp.corp.redhat.com (Postfix) with ESMTP id 980DE5C545; Wed, 13 Sep 2017 11:00:25 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com 2C366883AC Authentication-Results: ext-mx02.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx02.extmail.prod.ext.phx2.redhat.com; spf=fail smtp.mailfrom=quintela@redhat.com From: Juan Quintela To: qemu-devel@nongnu.org Date: Wed, 13 Sep 2017 12:59:42 +0200 Message-Id: <20170913105953.13760-10-quintela@redhat.com> In-Reply-To: <20170913105953.13760-1-quintela@redhat.com> References: <20170913105953.13760-1-quintela@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.26]); Wed, 13 Sep 2017 11:00:27 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH v8 09/20] migration: Create x-multifd-page-count parameter X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: lvivier@redhat.com, dgilbert@redhat.com, peterx@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Indicates how many pages we are going to send in each batch to a multifd thread. Signed-off-by: Juan Quintela Reviewed-by: Dr. David Alan Gilbert Reviewed-by: Peter Xu -- Be consistent with defaults and documentation Use new DEFINE_PROP_* Rename x-multifd-group to x-multifd-page-count --- hmp.c | 7 +++++++ migration/migration.c | 27 +++++++++++++++++++++++++++ migration/migration.h | 1 + qapi/migration.json | 17 ++++++++++++++--- 4 files changed, 49 insertions(+), 3 deletions(-) diff --git a/hmp.c b/hmp.c index 1f9e39529f..203fe1d50e 100644 --- a/hmp.c +++ b/hmp.c @@ -339,6 +339,9 @@ void hmp_info_migrate_parameters(Monitor *mon, const QD= ict *qdict) monitor_printf(mon, "%s: %" PRId64 "\n", MigrationParameter_str(MIGRATION_PARAMETER_X_MULTIFD_CHANNELS), params->x_multifd_channels); + monitor_printf(mon, "%s: %" PRId64 "\n", + MigrationParameter_str(MIGRATION_PARAMETER_X_MULTIFD_PAGE_COUN= T), + params->x_multifd_page_count); } =20 qapi_free_MigrationParameters(params); @@ -1628,6 +1631,10 @@ void hmp_migrate_set_parameter(Monitor *mon, const Q= Dict *qdict) p->has_x_multifd_channels =3D true; visit_type_int(v, param, &p->x_multifd_channels, &err); break; + case MIGRATION_PARAMETER_X_MULTIFD_PAGE_COUNT: + p->has_x_multifd_page_count =3D true; + visit_type_int(v, param, &p->x_multifd_page_count, &err); + break; default: assert(0); } diff --git a/migration/migration.c b/migration/migration.c index 021586f9f8..5be39fe6e2 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -78,6 +78,7 @@ */ #define DEFAULT_MIGRATE_X_CHECKPOINT_DELAY 200 #define DEFAULT_MIGRATE_MULTIFD_CHANNELS 2 +#define DEFAULT_MIGRATE_MULTIFD_PAGE_COUNT 16 =20 static NotifierList migration_state_notifiers =3D NOTIFIER_LIST_INITIALIZER(migration_state_notifiers); @@ -484,6 +485,8 @@ MigrationParameters *qmp_query_migrate_parameters(Error= **errp) params->block_incremental =3D s->parameters.block_incremental; params->has_x_multifd_channels =3D true; params->x_multifd_channels =3D s->parameters.x_multifd_channels; + params->has_x_multifd_page_count =3D true; + params->x_multifd_page_count =3D s->parameters.x_multifd_page_count; =20 return params; } @@ -772,6 +775,14 @@ static bool migrate_params_check(MigrationParameters *= params, Error **errp) "is invalid, it should be in the range of 1 to 255"); return false; } + if (params->has_x_multifd_page_count && + (params->x_multifd_page_count < 1 || + params->x_multifd_page_count > 10000)) { + error_setg(errp, QERR_INVALID_PARAMETER_VALUE, + "multifd_page_count", + "is invalid, it should be in the range of 1 to 10000"); + return false; + } =20 return true; } @@ -893,6 +904,9 @@ static void migrate_params_apply(MigrateSetParameters *= params) if (params->has_x_multifd_channels) { s->parameters.x_multifd_channels =3D params->x_multifd_channels; } + if (params->has_x_multifd_page_count) { + s->parameters.x_multifd_page_count =3D params->x_multifd_page_coun= t; + } } =20 void qmp_migrate_set_parameters(MigrateSetParameters *params, Error **errp) @@ -1489,6 +1503,15 @@ int migrate_multifd_channels(void) return s->parameters.x_multifd_channels; } =20 +int migrate_multifd_page_count(void) +{ + MigrationState *s; + + s =3D migrate_get_current(); + + return s->parameters.x_multifd_page_count; +} + int migrate_use_xbzrle(void) { MigrationState *s; @@ -2257,6 +2280,9 @@ static Property migration_properties[] =3D { DEFINE_PROP_INT64("x-multifd-channels", MigrationState, parameters.x_multifd_channels, DEFAULT_MIGRATE_MULTIFD_CHANNELS), + DEFINE_PROP_INT64("x-multifd-page-count", MigrationState, + parameters.x_multifd_page_count, + DEFAULT_MIGRATE_MULTIFD_PAGE_COUNT), =20 /* Migration capabilities */ DEFINE_PROP_MIG_CAP("x-xbzrle", MIGRATION_CAPABILITY_XBZRLE), @@ -2317,6 +2343,7 @@ static void migration_instance_init(Object *obj) params->has_x_checkpoint_delay =3D true; params->has_block_incremental =3D true; params->has_x_multifd_channels =3D true; + params->has_x_multifd_page_count =3D true; } =20 /* diff --git a/migration/migration.h b/migration/migration.h index a6a7c41823..da39d4c711 100644 --- a/migration/migration.h +++ b/migration/migration.h @@ -181,6 +181,7 @@ bool migrate_zero_blocks(void); bool migrate_auto_converge(void); bool migrate_use_multifd(void); int migrate_multifd_channels(void); +int migrate_multifd_page_count(void); =20 int migrate_use_xbzrle(void); int64_t migrate_xbzrle_cache_size(void); diff --git a/qapi/migration.json b/qapi/migration.json index c766fb1e52..f8b365e3f5 100644 --- a/qapi/migration.json +++ b/qapi/migration.json @@ -471,6 +471,9 @@ # number of sockets used for migration. The # default value is 2 (since 2.11) # +# @x-multifd-page-count: Number of pages sent together to a thread +# The default value is 16 (since 2.11) +# # Since: 2.4 ## { 'enum': 'MigrationParameter', @@ -478,7 +481,7 @@ 'cpu-throttle-initial', 'cpu-throttle-increment', 'tls-creds', 'tls-hostname', 'max-bandwidth', 'downtime-limit', 'x-checkpoint-delay', 'block-incremental', - 'x-multifd-channels'] } + 'x-multifd-channels', 'x-multifd-page-count' ] } =20 ## # @MigrateSetParameters: @@ -539,6 +542,9 @@ # number of sockets used for migration. The # default value is 2 (since 2.11) # +# @x-multifd-page-count: Number of pages sent together to a thread +# The default value is 16 (since 2.11) +# # Since: 2.4 ## # TODO either fuse back into MigrationParameters, or make @@ -555,7 +561,8 @@ '*downtime-limit': 'int', '*x-checkpoint-delay': 'int', '*block-incremental': 'bool', - '*x-multifd-channels': 'int' } } + '*x-multifd-channels': 'int', + '*x-multifd-page-count': 'int' } } =20 ## # @migrate-set-parameters: @@ -631,6 +638,9 @@ # number of sockets used for migration. # The default value is 2 (since 2.11) # +# @x-multifd-page-count: Number of pages sent together to a thread +# The default value is 16 (since 2.11) +# # Since: 2.4 ## { 'struct': 'MigrationParameters', @@ -645,7 +655,8 @@ '*downtime-limit': 'int', '*x-checkpoint-delay': 'int', '*block-incremental': 'bool' , - '*x-multifd-channels': 'int' } } + '*x-multifd-channels': 'int', + '*x-multifd-page-count': 'int' } } =20 ## # @query-migrate-parameters: --=20 2.13.5 From nobody Thu May 2 15:13:04 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1505300727190134.5451304019989; Wed, 13 Sep 2017 04:05:27 -0700 (PDT) Received: from localhost ([::1]:41569 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ds5Ty-0001xf-5a for importer@patchew.org; Wed, 13 Sep 2017 07:05:26 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:42231) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ds5PG-0006Py-Oy for qemu-devel@nongnu.org; Wed, 13 Sep 2017 07:00:40 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1ds5PC-0006mD-9q for qemu-devel@nongnu.org; Wed, 13 Sep 2017 07:00:34 -0400 Received: from mx1.redhat.com ([209.132.183.28]:42970) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1ds5PC-0006l5-2H for qemu-devel@nongnu.org; Wed, 13 Sep 2017 07:00:30 -0400 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id F12BC2027B for ; Wed, 13 Sep 2017 11:00:28 +0000 (UTC) Received: from secure.mitica (ovpn-117-188.ams2.redhat.com [10.36.117.188]) by smtp.corp.redhat.com (Postfix) with ESMTP id 7FABA61786; Wed, 13 Sep 2017 11:00:27 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com F12BC2027B Authentication-Results: ext-mx05.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx05.extmail.prod.ext.phx2.redhat.com; spf=fail smtp.mailfrom=quintela@redhat.com From: Juan Quintela To: qemu-devel@nongnu.org Date: Wed, 13 Sep 2017 12:59:43 +0200 Message-Id: <20170913105953.13760-11-quintela@redhat.com> In-Reply-To: <20170913105953.13760-1-quintela@redhat.com> References: <20170913105953.13760-1-quintela@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.29]); Wed, 13 Sep 2017 11:00:29 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH v8 10/20] migration: Create multifd migration threads X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: lvivier@redhat.com, dgilbert@redhat.com, peterx@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Creation of the threads, nothing inside yet. Signed-off-by: Juan Quintela Reviewed-by: Dr. David Alan Gilbert -- Use pointers instead of long array names Move to use semaphores instead of conditions as paolo suggestion Put all the state inside one struct. Use a counter for the number of threads created. Needed during cancellatio= n. Add error return to thread creation Add id field Rename functions to multifd_save/load_setup/cleanup Change recv parameters to a pointer to struct Change back to a struct Use Error * for _cleanup --- migration/migration.c | 26 +++++++ migration/ram.c | 202 ++++++++++++++++++++++++++++++++++++++++++++++= ++++ migration/ram.h | 5 ++ 3 files changed, 233 insertions(+) diff --git a/migration/migration.c b/migration/migration.c index 5be39fe6e2..8ab2ae08dd 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -281,6 +281,10 @@ static void process_incoming_migration_bh(void *opaque) */ qemu_announce_self(); =20 + if (multifd_load_cleanup(&local_err) !=3D 0) { + error_report_err(local_err); + autostart =3D false; + } /* If global state section was not received or we are in running state, we need to obey autostart. Any other state is set with runstate_set. */ @@ -353,10 +357,15 @@ static void process_incoming_migration_co(void *opaqu= e) } =20 if (ret < 0) { + Error *local_err =3D NULL; + migrate_set_state(&mis->state, MIGRATION_STATUS_ACTIVE, MIGRATION_STATUS_FAILED); error_report("load of migration failed: %s", strerror(-ret)); qemu_fclose(mis->from_src_file); + if (multifd_load_cleanup(&local_err) !=3D 0) { + error_report_err(local_err); + } exit(EXIT_FAILURE); } mis->bh =3D qemu_bh_new(process_incoming_migration_bh, mis); @@ -368,6 +377,12 @@ void migration_fd_process_incoming(QEMUFile *f) Coroutine *co =3D qemu_coroutine_create(process_incoming_migration_co,= NULL); MigrationIncomingState *mis =3D migration_incoming_get_current(); =20 + if (multifd_load_setup() !=3D 0) { + /* We haven't been able to create multifd threads + nothing better to do */ + exit(EXIT_FAILURE); + } + if (!mis->from_src_file) { mis->from_src_file =3D f; } @@ -1020,6 +1035,8 @@ static void migrate_fd_cleanup(void *opaque) s->cleanup_bh =3D NULL; =20 if (s->to_dst_file) { + Error *local_err =3D NULL; + trace_migrate_fd_cleanup(); qemu_mutex_unlock_iothread(); if (s->migration_thread_running) { @@ -1028,6 +1045,9 @@ static void migrate_fd_cleanup(void *opaque) } qemu_mutex_lock_iothread(); =20 + if (multifd_save_cleanup(&local_err) !=3D 0) { + error_report_err(local_err); + } qemu_fclose(s->to_dst_file); s->to_dst_file =3D NULL; } @@ -2226,6 +2246,12 @@ void migrate_fd_connect(MigrationState *s) } } =20 + if (multifd_save_setup() !=3D 0) { + migrate_set_state(&s->state, MIGRATION_STATUS_SETUP, + MIGRATION_STATUS_FAILED); + migrate_fd_cleanup(s); + return; + } qemu_thread_create(&s->thread, "live_migration", migration_thread, s, QEMU_THREAD_JOINABLE); s->migration_thread_running =3D true; diff --git a/migration/ram.c b/migration/ram.c index e0179fc838..a3e2abb2a5 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -356,6 +356,208 @@ static void compress_threads_save_setup(void) } } =20 +/* Multiple fd's */ + +struct MultiFDSendParams { + uint8_t id; + char *name; + QemuThread thread; + QemuSemaphore sem; + QemuMutex mutex; + bool quit; +}; +typedef struct MultiFDSendParams MultiFDSendParams; + +struct { + MultiFDSendParams *params; + /* number of created threads */ + int count; +} *multifd_send_state; + +static void terminate_multifd_send_threads(Error *errp) +{ + int i; + + for (i =3D 0; i < multifd_send_state->count; i++) { + MultiFDSendParams *p =3D &multifd_send_state->params[i]; + + qemu_mutex_lock(&p->mutex); + p->quit =3D true; + qemu_sem_post(&p->sem); + qemu_mutex_unlock(&p->mutex); + } +} + +int multifd_save_cleanup(Error **errp) +{ + int i; + int ret =3D 0; + + if (!migrate_use_multifd()) { + return 0; + } + terminate_multifd_send_threads(NULL); + for (i =3D 0; i < multifd_send_state->count; i++) { + MultiFDSendParams *p =3D &multifd_send_state->params[i]; + + qemu_thread_join(&p->thread); + qemu_mutex_destroy(&p->mutex); + qemu_sem_destroy(&p->sem); + g_free(p->name); + p->name =3D NULL; + } + g_free(multifd_send_state->params); + multifd_send_state->params =3D NULL; + g_free(multifd_send_state); + multifd_send_state =3D NULL; + return ret; +} + +static void *multifd_send_thread(void *opaque) +{ + MultiFDSendParams *p =3D opaque; + + while (true) { + qemu_mutex_lock(&p->mutex); + if (p->quit) { + qemu_mutex_unlock(&p->mutex); + break; + } + qemu_mutex_unlock(&p->mutex); + qemu_sem_wait(&p->sem); + } + + return NULL; +} + +int multifd_save_setup(void) +{ + int thread_count; + uint8_t i; + + if (!migrate_use_multifd()) { + return 0; + } + thread_count =3D migrate_multifd_channels(); + multifd_send_state =3D g_malloc0(sizeof(*multifd_send_state)); + multifd_send_state->params =3D g_new0(MultiFDSendParams, thread_count); + multifd_send_state->count =3D 0; + for (i =3D 0; i < thread_count; i++) { + MultiFDSendParams *p =3D &multifd_send_state->params[i]; + + qemu_mutex_init(&p->mutex); + qemu_sem_init(&p->sem, 0); + p->quit =3D false; + p->id =3D i; + p->name =3D g_strdup_printf("multifdsend_%d", i); + qemu_thread_create(&p->thread, p->name, multifd_send_thread, p, + QEMU_THREAD_JOINABLE); + + multifd_send_state->count++; + } + return 0; +} + +struct MultiFDRecvParams { + uint8_t id; + char *name; + QemuThread thread; + QemuSemaphore sem; + QemuMutex mutex; + bool quit; +}; +typedef struct MultiFDRecvParams MultiFDRecvParams; + +struct { + MultiFDRecvParams *params; + /* number of created threads */ + int count; +} *multifd_recv_state; + +static void terminate_multifd_recv_threads(Error *errp) +{ + int i; + + for (i =3D 0; i < multifd_recv_state->count; i++) { + MultiFDRecvParams *p =3D &multifd_recv_state->params[i]; + + qemu_mutex_lock(&p->mutex); + p->quit =3D true; + qemu_sem_post(&p->sem); + qemu_mutex_unlock(&p->mutex); + } +} + +int multifd_load_cleanup(Error **errp) +{ + int i; + int ret =3D 0; + + if (!migrate_use_multifd()) { + return 0; + } + terminate_multifd_recv_threads(NULL); + for (i =3D 0; i < multifd_recv_state->count; i++) { + MultiFDRecvParams *p =3D &multifd_recv_state->params[i]; + + qemu_thread_join(&p->thread); + qemu_mutex_destroy(&p->mutex); + qemu_sem_destroy(&p->sem); + g_free(p->name); + p->name =3D NULL; + } + g_free(multifd_recv_state->params); + multifd_recv_state->params =3D NULL; + g_free(multifd_recv_state); + multifd_recv_state =3D NULL; + + return ret; +} + +static void *multifd_recv_thread(void *opaque) +{ + MultiFDRecvParams *p =3D opaque; + + while (true) { + qemu_mutex_lock(&p->mutex); + if (p->quit) { + qemu_mutex_unlock(&p->mutex); + break; + } + qemu_mutex_unlock(&p->mutex); + qemu_sem_wait(&p->sem); + } + + return NULL; +} + +int multifd_load_setup(void) +{ + int thread_count; + uint8_t i; + + if (!migrate_use_multifd()) { + return 0; + } + thread_count =3D migrate_multifd_channels(); + multifd_recv_state =3D g_malloc0(sizeof(*multifd_recv_state)); + multifd_recv_state->params =3D g_new0(MultiFDRecvParams, thread_count); + multifd_recv_state->count =3D 0; + for (i =3D 0; i < thread_count; i++) { + MultiFDRecvParams *p =3D &multifd_recv_state->params[i]; + + qemu_mutex_init(&p->mutex); + qemu_sem_init(&p->sem, 0); + p->quit =3D false; + p->id =3D i; + p->name =3D g_strdup_printf("multifdrecv_%d", i); + qemu_thread_create(&p->thread, p->name, multifd_recv_thread, p, + QEMU_THREAD_JOINABLE); + multifd_recv_state->count++; + } + return 0; +} + /** * save_page_header: write page header to wire * diff --git a/migration/ram.h b/migration/ram.h index c081fde86c..4a72d66503 100644 --- a/migration/ram.h +++ b/migration/ram.h @@ -39,6 +39,11 @@ int64_t xbzrle_cache_resize(int64_t new_size); uint64_t ram_bytes_remaining(void); uint64_t ram_bytes_total(void); =20 +int multifd_save_setup(void); +int multifd_save_cleanup(Error **errp); +int multifd_load_setup(void); +int multifd_load_cleanup(Error **errp); + uint64_t ram_pagesize_summary(void); int ram_save_queue_pages(const char *rbname, ram_addr_t start, ram_addr_t = len); void acct_update_position(QEMUFile *f, size_t size, bool zero); --=20 2.13.5 From nobody Thu May 2 15:13:04 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1505300892380518.7650352096583; Wed, 13 Sep 2017 04:08:12 -0700 (PDT) Received: from localhost ([::1]:41584 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ds5Wb-0004Xn-RL for importer@patchew.org; Wed, 13 Sep 2017 07:08:09 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:42327) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ds5PM-0006Rq-Dk for qemu-devel@nongnu.org; Wed, 13 Sep 2017 07:00:41 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1ds5PD-0006nj-R1 for qemu-devel@nongnu.org; Wed, 13 Sep 2017 07:00:40 -0400 Received: from mx1.redhat.com ([209.132.183.28]:45896) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1ds5PD-0006mz-MC for qemu-devel@nongnu.org; Wed, 13 Sep 2017 07:00:31 -0400 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id A892D81E0D for ; Wed, 13 Sep 2017 11:00:30 +0000 (UTC) Received: from secure.mitica (ovpn-117-188.ams2.redhat.com [10.36.117.188]) by smtp.corp.redhat.com (Postfix) with ESMTP id 53E675C545; Wed, 13 Sep 2017 11:00:29 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com A892D81E0D Authentication-Results: ext-mx01.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx01.extmail.prod.ext.phx2.redhat.com; spf=fail smtp.mailfrom=quintela@redhat.com From: Juan Quintela To: qemu-devel@nongnu.org Date: Wed, 13 Sep 2017 12:59:44 +0200 Message-Id: <20170913105953.13760-12-quintela@redhat.com> In-Reply-To: <20170913105953.13760-1-quintela@redhat.com> References: <20170913105953.13760-1-quintela@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.25]); Wed, 13 Sep 2017 11:00:30 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH v8 11/20] migration: Split migration_fd_process_incoming X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: lvivier@redhat.com, dgilbert@redhat.com, peterx@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" We need that on later patches. Signed-off-by: Juan Quintela Reviewed-by: Dr. David Alan Gilbert Reviewed-by: Peter Xu Reviewed-by: Daniel P. Berrange --- migration/migration.c | 14 ++++++++++++-- 1 file changed, 12 insertions(+), 2 deletions(-) diff --git a/migration/migration.c b/migration/migration.c index 8ab2ae08dd..1401841997 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -372,9 +372,8 @@ static void process_incoming_migration_co(void *opaque) qemu_bh_schedule(mis->bh); } =20 -void migration_fd_process_incoming(QEMUFile *f) +static void migration_incoming_setup(QEMUFile *f) { - Coroutine *co =3D qemu_coroutine_create(process_incoming_migration_co,= NULL); MigrationIncomingState *mis =3D migration_incoming_get_current(); =20 if (multifd_load_setup() !=3D 0) { @@ -387,9 +386,20 @@ void migration_fd_process_incoming(QEMUFile *f) mis->from_src_file =3D f; } qemu_file_set_blocking(f, false); +} + +static void migration_incoming_process(void) +{ + Coroutine *co =3D qemu_coroutine_create(process_incoming_migration_co,= NULL); qemu_coroutine_enter(co); } =20 +void migration_fd_process_incoming(QEMUFile *f) +{ + migration_incoming_setup(f); + migration_incoming_process(); +} + void migration_ioc_process_incoming(QIOChannel *ioc) { MigrationIncomingState *mis =3D migration_incoming_get_current(); --=20 2.13.5 From nobody Thu May 2 15:13:04 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1505301242414913.3645246284092; Wed, 13 Sep 2017 04:14:02 -0700 (PDT) Received: from localhost ([::1]:41609 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ds5cH-0001iY-LO for importer@patchew.org; Wed, 13 Sep 2017 07:14:01 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:42319) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ds5PL-0006RT-VK for qemu-devel@nongnu.org; Wed, 13 Sep 2017 07:00:41 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1ds5PG-0006qt-Fb for qemu-devel@nongnu.org; Wed, 13 Sep 2017 07:00:40 -0400 Received: from mx1.redhat.com ([209.132.183.28]:38800) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1ds5PG-0006qE-7g for qemu-devel@nongnu.org; Wed, 13 Sep 2017 07:00:34 -0400 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 43CEC5F798 for ; Wed, 13 Sep 2017 11:00:33 +0000 (UTC) Received: from secure.mitica (ovpn-117-188.ams2.redhat.com [10.36.117.188]) by smtp.corp.redhat.com (Postfix) with ESMTP id 0556361786; Wed, 13 Sep 2017 11:00:30 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com 43CEC5F798 Authentication-Results: ext-mx10.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx10.extmail.prod.ext.phx2.redhat.com; spf=fail smtp.mailfrom=quintela@redhat.com From: Juan Quintela To: qemu-devel@nongnu.org Date: Wed, 13 Sep 2017 12:59:45 +0200 Message-Id: <20170913105953.13760-13-quintela@redhat.com> In-Reply-To: <20170913105953.13760-1-quintela@redhat.com> References: <20170913105953.13760-1-quintela@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.39]); Wed, 13 Sep 2017 11:00:33 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH v8 12/20] migration: Start of multiple fd work X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: lvivier@redhat.com, dgilbert@redhat.com, peterx@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" We create new channels for each new thread created. We send through them a string containing multifd so we are sure that we connect the right channels in both sides. Signed-off-by: Juan Quintela -- Split SocketArgs into incoming and outgoing args Use UUID's on the initial message, so we are sure we are connecting to the right channel. Remove init semaphore. Now that we use uuids on the init message, we know that this is our channel. Fix recv socket destwroy, we were destroying send channels. This was very interesting, because we were using an unreferred object without problems. Move to struct of pointers init channel sooner. split recv thread creation. listen on main thread We count the number of created threads to know when we need to stop listeni= ng Use g_strdup_printf report channel id on errors Add name parameter Use local_err Add Error * parameter to socket_send_channel_create() Use qio_channel_*_all Use asynchronous connect --- migration/migration.c | 5 ++ migration/ram.c | 138 +++++++++++++++++++++++++++++++++++++++++++---= ---- migration/ram.h | 3 ++ migration/socket.c | 34 ++++++++++++- migration/socket.h | 10 ++++ 5 files changed, 172 insertions(+), 18 deletions(-) diff --git a/migration/migration.c b/migration/migration.c index 1401841997..679be8e8d4 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -419,6 +419,11 @@ void migration_ioc_process_incoming(QIOChannel *ioc) */ bool migration_has_all_channels(void) { + if (migrate_use_multifd()) { + int thread_count =3D migrate_multifd_channels(); + + return thread_count =3D=3D multifd_created_channels(); + } return true; } =20 diff --git a/migration/ram.c b/migration/ram.c index a3e2abb2a5..8577eeb032 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -36,6 +36,7 @@ #include "xbzrle.h" #include "ram.h" #include "migration.h" +#include "socket.h" #include "migration/register.h" #include "migration/misc.h" #include "qemu-file.h" @@ -46,6 +47,8 @@ #include "exec/ram_addr.h" #include "qemu/rcu_queue.h" #include "migration/colo.h" +#include "sysemu/sysemu.h" +#include "qemu/uuid.h" =20 /***********************************************************/ /* ram save/restore */ @@ -362,6 +365,7 @@ struct MultiFDSendParams { uint8_t id; char *name; QemuThread thread; + QIOChannel *c; QemuSemaphore sem; QemuMutex mutex; bool quit; @@ -378,6 +382,12 @@ static void terminate_multifd_send_threads(Error *errp) { int i; =20 + if (errp) { + MigrationState *s =3D migrate_get_current(); + migrate_set_error(s, errp); + migrate_set_state(&s->state, MIGRATION_STATUS_ACTIVE, + MIGRATION_STATUS_FAILED); + } for (i =3D 0; i < multifd_send_state->count; i++) { MultiFDSendParams *p =3D &multifd_send_state->params[i]; =20 @@ -403,6 +413,7 @@ int multifd_save_cleanup(Error **errp) qemu_thread_join(&p->thread); qemu_mutex_destroy(&p->mutex); qemu_sem_destroy(&p->sem); + socket_send_channel_destroy(p->c); g_free(p->name); p->name =3D NULL; } @@ -413,9 +424,32 @@ int multifd_save_cleanup(Error **errp) return ret; } =20 +/* Default uuid for multifd when qemu is not started with uuid */ +static char multifd_uuid[] =3D "5c49fd7e-af88-4a07-b6e8-091fd696ad40"; +/* strlen(multifd) + '-' + + '-' + UUID_FMT + '\0' */ +#define MULTIFD_UUID_MSG (7 + 1 + 3 + 1 + UUID_FMT_LEN + 1) + static void *multifd_send_thread(void *opaque) { MultiFDSendParams *p =3D opaque; + Error *local_err =3D NULL; + char *string; + char *string_uuid; + size_t ret; + + if (qemu_uuid_set) { + string_uuid =3D qemu_uuid_unparse_strdup(&qemu_uuid); + } else { + string_uuid =3D g_strdup(multifd_uuid); + } + string =3D g_strdup_printf("%s multifd %03d", string_uuid, p->id); + g_free(string_uuid); + ret =3D qio_channel_write_all(p->c, string, MULTIFD_UUID_MSG, &local_e= rr); + g_free(string); + if (ret !=3D 0) { + terminate_multifd_send_threads(local_err); + return NULL; + } =20 while (true) { qemu_mutex_lock(&p->mutex); @@ -430,6 +464,27 @@ static void *multifd_send_thread(void *opaque) return NULL; } =20 +static void multifd_new_channel_async(QIOTask *task, gpointer opaque) +{ + MultiFDSendParams *p =3D opaque; + QIOChannel *sioc =3D QIO_CHANNEL(qio_task_get_source(task)); + Error *local_err; + + if (qio_task_propagate_error(task, &local_err)) { + if (multifd_save_cleanup(&local_err) !=3D 0) { + migrate_set_error(migrate_get_current(), local_err); + } + } else { + p->c =3D QIO_CHANNEL(sioc); + qio_channel_set_delay(p->c, false); + + qemu_thread_create(&p->thread, p->name, multifd_send_thread, p, + QEMU_THREAD_JOINABLE); + + multifd_send_state->count++; + } +} + int multifd_save_setup(void) { int thread_count; @@ -450,10 +505,7 @@ int multifd_save_setup(void) p->quit =3D false; p->id =3D i; p->name =3D g_strdup_printf("multifdsend_%d", i); - qemu_thread_create(&p->thread, p->name, multifd_send_thread, p, - QEMU_THREAD_JOINABLE); - - multifd_send_state->count++; + socket_send_channel_create(multifd_new_channel_async, p); } return 0; } @@ -462,6 +514,7 @@ struct MultiFDRecvParams { uint8_t id; char *name; QemuThread thread; + QIOChannel *c; QemuSemaphore sem; QemuMutex mutex; bool quit; @@ -472,12 +525,22 @@ struct { MultiFDRecvParams *params; /* number of created threads */ int count; + /* Should we finish */ + bool quit; } *multifd_recv_state; =20 static void terminate_multifd_recv_threads(Error *errp) { int i; =20 + if (errp) { + MigrationState *s =3D migrate_get_current(); + migrate_set_error(s, errp); + migrate_set_state(&s->state, MIGRATION_STATUS_ACTIVE, + MIGRATION_STATUS_FAILED); + } + multifd_recv_state->quit =3D true; + for (i =3D 0; i < multifd_recv_state->count; i++) { MultiFDRecvParams *p =3D &multifd_recv_state->params[i]; =20 @@ -503,6 +566,7 @@ int multifd_load_cleanup(Error **errp) qemu_thread_join(&p->thread); qemu_mutex_destroy(&p->mutex); qemu_sem_destroy(&p->sem); + socket_recv_channel_destroy(p->c); g_free(p->name); p->name =3D NULL; } @@ -531,10 +595,56 @@ static void *multifd_recv_thread(void *opaque) return NULL; } =20 +void multifd_new_channel(QIOChannel *ioc) +{ + MultiFDRecvParams *p; + char string[MULTIFD_UUID_MSG]; + char string_uuid[UUID_FMT_LEN]; + Error *local_err =3D NULL; + char *uuid; + size_t ret; + int id; + + ret =3D qio_channel_read_all(ioc, string, sizeof(string), &local_err); + if (ret !=3D 0) { + terminate_multifd_recv_threads(local_err); + return; + } + sscanf(string, "%s multifd %03d", string_uuid, &id); + + if (qemu_uuid_set) { + uuid =3D qemu_uuid_unparse_strdup(&qemu_uuid); + } else { + uuid =3D g_strdup(multifd_uuid); + } + if (strcmp(string_uuid, uuid)) { + error_setg(&local_err, "multifd: received uuid '%s' and expected " + "uuid '%s' for channel %d", string_uuid, uuid, id); + terminate_multifd_recv_threads(local_err); + return; + } + g_free(uuid); + + p =3D &multifd_recv_state->params[id]; + if (p->id !=3D 0) { + error_setg(&local_err, "multifd: received id '%d' already setup'",= id); + terminate_multifd_recv_threads(local_err); + return; + } + qemu_mutex_init(&p->mutex); + qemu_sem_init(&p->sem, 0); + p->quit =3D false; + p->id =3D id; + p->c =3D ioc; + multifd_recv_state->count++; + p->name =3D g_strdup_printf("multifdrecv_%d", id); + qemu_thread_create(&p->thread, p->name, multifd_recv_thread, p, + QEMU_THREAD_JOINABLE); +} + int multifd_load_setup(void) { int thread_count; - uint8_t i; =20 if (!migrate_use_multifd()) { return 0; @@ -543,21 +653,15 @@ int multifd_load_setup(void) multifd_recv_state =3D g_malloc0(sizeof(*multifd_recv_state)); multifd_recv_state->params =3D g_new0(MultiFDRecvParams, thread_count); multifd_recv_state->count =3D 0; - for (i =3D 0; i < thread_count; i++) { - MultiFDRecvParams *p =3D &multifd_recv_state->params[i]; - - qemu_mutex_init(&p->mutex); - qemu_sem_init(&p->sem, 0); - p->quit =3D false; - p->id =3D i; - p->name =3D g_strdup_printf("multifdrecv_%d", i); - qemu_thread_create(&p->thread, p->name, multifd_recv_thread, p, - QEMU_THREAD_JOINABLE); - multifd_recv_state->count++; - } + multifd_recv_state->quit =3D false; return 0; } =20 +int multifd_created_channels(void) +{ + return multifd_recv_state->count; +} + /** * save_page_header: write page header to wire * diff --git a/migration/ram.h b/migration/ram.h index 4a72d66503..5221bc9beb 100644 --- a/migration/ram.h +++ b/migration/ram.h @@ -31,6 +31,7 @@ =20 #include "qemu-common.h" #include "exec/cpu-common.h" +#include "io/channel.h" =20 extern MigrationStats ram_counters; extern XBZRLECacheStats xbzrle_counters; @@ -43,6 +44,8 @@ int multifd_save_setup(void); int multifd_save_cleanup(Error **errp); int multifd_load_setup(void); int multifd_load_cleanup(Error **errp); +void multifd_new_channel(QIOChannel *ioc); +int multifd_created_channels(void); =20 uint64_t ram_pagesize_summary(void); int ram_save_queue_pages(const char *rbname, ram_addr_t start, ram_addr_t = len); diff --git a/migration/socket.c b/migration/socket.c index 2d70747a1a..22fb05edc8 100644 --- a/migration/socket.c +++ b/migration/socket.c @@ -26,6 +26,34 @@ #include "io/channel-socket.h" #include "trace.h" =20 +int socket_recv_channel_destroy(QIOChannel *recv) +{ + /* Remove channel */ + object_unref(OBJECT(recv)); + return 0; +} + +struct SocketOutgoingArgs { + SocketAddress *saddr; +} outgoing_args; + +void socket_send_channel_create(void (*f)(QIOTask *, gpointer), void *data) +{ + QIOChannelSocket *sioc =3D qio_channel_socket_new(); + qio_channel_socket_connect_async(sioc, outgoing_args.saddr, + f, data, NULL); +} + +int socket_send_channel_destroy(QIOChannel *send) +{ + /* Remove channel */ + object_unref(OBJECT(send)); + if (outgoing_args.saddr) { + qapi_free_SocketAddress(outgoing_args.saddr); + outgoing_args.saddr =3D NULL; + } + return 0; +} =20 static SocketAddress *tcp_build_address(const char *host_port, Error **err= p) { @@ -95,6 +123,11 @@ static void socket_start_outgoing_migration(MigrationSt= ate *s, struct SocketConnectData *data =3D g_new0(struct SocketConnectData, 1); =20 data->s =3D s; + + /* in case previous migration leaked it */ + qapi_free_SocketAddress(outgoing_args.saddr); + outgoing_args.saddr =3D saddr; + if (saddr->type =3D=3D SOCKET_ADDRESS_TYPE_INET) { data->hostname =3D g_strdup(saddr->u.inet.host); } @@ -105,7 +138,6 @@ static void socket_start_outgoing_migration(MigrationSt= ate *s, socket_outgoing_migration, data, socket_connect_data_free); - qapi_free_SocketAddress(saddr); } =20 void tcp_start_outgoing_migration(MigrationState *s, diff --git a/migration/socket.h b/migration/socket.h index 6b91e9db38..afb0ff0f51 100644 --- a/migration/socket.h +++ b/migration/socket.h @@ -16,6 +16,16 @@ =20 #ifndef QEMU_MIGRATION_SOCKET_H #define QEMU_MIGRATION_SOCKET_H + +#include "io/channel.h" +#include "io/task.h" + +QIOChannel *socket_recv_channel_create(void); +int socket_recv_channel_destroy(QIOChannel *recv); + +void socket_send_channel_create(void (*f)(QIOTask *, gpointer), void *data= ); +int socket_send_channel_destroy(QIOChannel *send); + void tcp_start_incoming_migration(const char *host_port, Error **errp); =20 void tcp_start_outgoing_migration(MigrationState *s, const char *host_port, --=20 2.13.5 From nobody Thu May 2 15:13:04 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1505301084381735.0154545624303; Wed, 13 Sep 2017 04:11:24 -0700 (PDT) Received: from localhost ([::1]:41600 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ds5Zj-0007YZ-GX for importer@patchew.org; Wed, 13 Sep 2017 07:11:23 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:42295) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ds5PJ-0006Qh-TP for qemu-devel@nongnu.org; Wed, 13 Sep 2017 07:00:39 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1ds5PI-0006vA-Ju for qemu-devel@nongnu.org; Wed, 13 Sep 2017 07:00:38 -0400 Received: from mx1.redhat.com ([209.132.183.28]:10316) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1ds5PI-0006tc-CD for qemu-devel@nongnu.org; Wed, 13 Sep 2017 07:00:36 -0400 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 58D88883BC for ; Wed, 13 Sep 2017 11:00:35 +0000 (UTC) Received: from secure.mitica (ovpn-117-188.ams2.redhat.com [10.36.117.188]) by smtp.corp.redhat.com (Postfix) with ESMTP id 0437E5C545; Wed, 13 Sep 2017 11:00:33 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com 58D88883BC Authentication-Results: ext-mx02.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx02.extmail.prod.ext.phx2.redhat.com; spf=fail smtp.mailfrom=quintela@redhat.com From: Juan Quintela To: qemu-devel@nongnu.org Date: Wed, 13 Sep 2017 12:59:46 +0200 Message-Id: <20170913105953.13760-14-quintela@redhat.com> In-Reply-To: <20170913105953.13760-1-quintela@redhat.com> References: <20170913105953.13760-1-quintela@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.26]); Wed, 13 Sep 2017 11:00:35 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH v8 13/20] migration: Create ram_multifd_page X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: lvivier@redhat.com, dgilbert@redhat.com, peterx@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" The function still don't use multifd, but we have simplified ram_save_page, xbzrle and RDMA stuff is gone. We have added a new counter and a new flag for this type of pages. Signed-off-by: Juan Quintela -- Add last_page parameter Add commets for done and address --- hmp.c | 2 ++ migration/migration.c | 1 + migration/ram.c | 94 +++++++++++++++++++++++++++++++++++++++++++++++= +++- qapi/migration.json | 5 ++- 4 files changed, 100 insertions(+), 2 deletions(-) diff --git a/hmp.c b/hmp.c index 203fe1d50e..2e9f9abf1b 100644 --- a/hmp.c +++ b/hmp.c @@ -233,6 +233,8 @@ void hmp_info_migrate(Monitor *mon, const QDict *qdict) monitor_printf(mon, "postcopy request count: %" PRIu64 "\n", info->ram->postcopy_requests); } + monitor_printf(mon, "multifd: %" PRIu64 " pages\n", + info->ram->multifd); } =20 if (info->has_disk) { diff --git a/migration/migration.c b/migration/migration.c index 679be8e8d4..085177ca26 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -555,6 +555,7 @@ static void populate_ram_info(MigrationInfo *info, Migr= ationState *s) info->ram->dirty_sync_count =3D ram_counters.dirty_sync_count; info->ram->postcopy_requests =3D ram_counters.postcopy_requests; info->ram->page_size =3D qemu_target_page_size(); + info->ram->multifd =3D ram_counters.multifd; =20 if (migrate_use_xbzrle()) { info->has_xbzrle_cache =3D true; diff --git a/migration/ram.c b/migration/ram.c index 8577eeb032..1381bfaf8a 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -68,6 +68,7 @@ #define RAM_SAVE_FLAG_XBZRLE 0x40 /* 0x80 is reserved in migration.h start with 0x100 next */ #define RAM_SAVE_FLAG_COMPRESS_PAGE 0x100 +#define RAM_SAVE_FLAG_MULTIFD_PAGE 0x200 =20 static inline bool is_zero_range(uint8_t *p, uint64_t size) { @@ -362,13 +363,23 @@ static void compress_threads_save_setup(void) /* Multiple fd's */ =20 struct MultiFDSendParams { + /* not changed */ uint8_t id; char *name; QemuThread thread; QIOChannel *c; QemuSemaphore sem; QemuMutex mutex; + /* protected by param mutex */ bool quit; + /* This is a temp field. We are using it now to transmit + something the address of the page. Later in the series, we + change it for the real page. + */ + uint8_t *address; + /* protected by multifd mutex */ + /* has the thread finish the last submitted job */ + bool done; }; typedef struct MultiFDSendParams MultiFDSendParams; =20 @@ -376,6 +387,8 @@ struct { MultiFDSendParams *params; /* number of created threads */ int count; + QemuMutex mutex; + QemuSemaphore sem; } *multifd_send_state; =20 static void terminate_multifd_send_threads(Error *errp) @@ -450,6 +463,7 @@ static void *multifd_send_thread(void *opaque) terminate_multifd_send_threads(local_err); return NULL; } + qemu_sem_post(&multifd_send_state->sem); =20 while (true) { qemu_mutex_lock(&p->mutex); @@ -457,6 +471,15 @@ static void *multifd_send_thread(void *opaque) qemu_mutex_unlock(&p->mutex); break; } + if (p->address) { + p->address =3D 0; + qemu_mutex_unlock(&p->mutex); + qemu_mutex_lock(&multifd_send_state->mutex); + p->done =3D true; + qemu_mutex_unlock(&multifd_send_state->mutex); + qemu_sem_post(&multifd_send_state->sem); + continue; + } qemu_mutex_unlock(&p->mutex); qemu_sem_wait(&p->sem); } @@ -497,6 +520,8 @@ int multifd_save_setup(void) multifd_send_state =3D g_malloc0(sizeof(*multifd_send_state)); multifd_send_state->params =3D g_new0(MultiFDSendParams, thread_count); multifd_send_state->count =3D 0; + qemu_mutex_init(&multifd_send_state->mutex); + qemu_sem_init(&multifd_send_state->sem, 0); for (i =3D 0; i < thread_count; i++) { MultiFDSendParams *p =3D &multifd_send_state->params[i]; =20 @@ -504,12 +529,38 @@ int multifd_save_setup(void) qemu_sem_init(&p->sem, 0); p->quit =3D false; p->id =3D i; + p->done =3D true; + p->address =3D 0; p->name =3D g_strdup_printf("multifdsend_%d", i); socket_send_channel_create(multifd_new_channel_async, p); } return 0; } =20 +static uint16_t multifd_send_page(uint8_t *address, bool last_page) +{ + int i; + MultiFDSendParams *p =3D NULL; /* make happy gcc */ + + qemu_sem_wait(&multifd_send_state->sem); + qemu_mutex_lock(&multifd_send_state->mutex); + for (i =3D 0; i < multifd_send_state->count; i++) { + p =3D &multifd_send_state->params[i]; + + if (p->done) { + p->done =3D false; + break; + } + } + qemu_mutex_unlock(&multifd_send_state->mutex); + qemu_mutex_lock(&p->mutex); + p->address =3D address; + qemu_mutex_unlock(&p->mutex); + qemu_sem_post(&p->sem); + + return 0; +} + struct MultiFDRecvParams { uint8_t id; char *name; @@ -1092,6 +1143,32 @@ static int ram_save_page(RAMState *rs, PageSearchSta= tus *pss, bool last_stage) return pages; } =20 +static int ram_multifd_page(RAMState *rs, PageSearchStatus *pss, + bool last_stage) +{ + int pages; + uint8_t *p; + RAMBlock *block =3D pss->block; + ram_addr_t offset =3D pss->page << TARGET_PAGE_BITS; + + p =3D block->host + offset; + + pages =3D save_zero_page(rs, block, offset, p); + if (pages =3D=3D -1) { + ram_counters.transferred +=3D + save_page_header(rs, rs->f, block, + offset | RAM_SAVE_FLAG_MULTIFD_PAGE); + qemu_put_buffer(rs->f, p, TARGET_PAGE_SIZE); + multifd_send_page(p, rs->migration_dirty_pages =3D=3D 1); + ram_counters.transferred +=3D TARGET_PAGE_SIZE; + pages =3D 1; + ram_counters.normal++; + ram_counters.multifd++; + } + + return pages; +} + static int do_compress_ram_page(QEMUFile *f, RAMBlock *block, ram_addr_t offset) { @@ -1520,6 +1597,8 @@ static int ram_save_target_page(RAMState *rs, PageSea= rchStatus *pss, if (migrate_use_compression() && (rs->ram_bulk_stage || !migrate_use_xbzrle())) { res =3D ram_save_compressed_page(rs, pss, last_stage); + } else if (migrate_use_multifd()) { + res =3D ram_multifd_page(rs, pss, last_stage); } else { res =3D ram_save_page(rs, pss, last_stage); } @@ -2812,6 +2891,10 @@ static int ram_load(QEMUFile *f, void *opaque, int v= ersion_id) if (!migrate_use_compression()) { invalid_flags |=3D RAM_SAVE_FLAG_COMPRESS_PAGE; } + + if (!migrate_use_multifd()) { + invalid_flags |=3D RAM_SAVE_FLAG_MULTIFD_PAGE; + } /* This RCU critical section can be very long running. * When RCU reclaims in the code start to become numerous, * it will be necessary to reduce the granularity of this @@ -2836,13 +2919,17 @@ static int ram_load(QEMUFile *f, void *opaque, int = version_id) if (flags & invalid_flags & RAM_SAVE_FLAG_COMPRESS_PAGE) { error_report("Received an unexpected compressed page"); } + if (flags & invalid_flags & RAM_SAVE_FLAG_MULTIFD_PAGE) { + error_report("Received an unexpected multifd page"); + } =20 ret =3D -EINVAL; break; } =20 if (flags & (RAM_SAVE_FLAG_ZERO | RAM_SAVE_FLAG_PAGE | - RAM_SAVE_FLAG_COMPRESS_PAGE | RAM_SAVE_FLAG_XBZRLE)) { + RAM_SAVE_FLAG_COMPRESS_PAGE | RAM_SAVE_FLAG_XBZRLE | + RAM_SAVE_FLAG_MULTIFD_PAGE)) { RAMBlock *block =3D ram_block_from_stream(f, flags); =20 host =3D host_from_ram_block_offset(block, addr); @@ -2930,6 +3017,11 @@ static int ram_load(QEMUFile *f, void *opaque, int v= ersion_id) break; } break; + + case RAM_SAVE_FLAG_MULTIFD_PAGE: + qemu_get_buffer(f, host, TARGET_PAGE_SIZE); + break; + case RAM_SAVE_FLAG_EOS: /* normal exit */ break; diff --git a/qapi/migration.json b/qapi/migration.json index f8b365e3f5..9cb59373fa 100644 --- a/qapi/migration.json +++ b/qapi/migration.json @@ -39,6 +39,8 @@ # @page-size: The number of bytes per page for the various page-based # statistics (since 2.10) # +# @multifd: number of pages sent with multifd (since 2.10) +# # Since: 0.14.0 ## { 'struct': 'MigrationStats', @@ -46,7 +48,8 @@ 'duplicate': 'int', 'skipped': 'int', 'normal': 'int', 'normal-bytes': 'int', 'dirty-pages-rate' : 'int', 'mbps' : 'number', 'dirty-sync-count' : 'int', - 'postcopy-requests' : 'int', 'page-size' : 'int' } } + 'postcopy-requests' : 'int', 'page-size' : 'int', + 'multifd': 'int' } } =20 ## # @XBZRLECacheStats: --=20 2.13.5 From nobody Thu May 2 15:13:04 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1505301050852165.67421699841657; Wed, 13 Sep 2017 04:10:50 -0700 (PDT) Received: from localhost ([::1]:41598 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ds5ZC-00075u-2U for importer@patchew.org; Wed, 13 Sep 2017 07:10:50 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:42350) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ds5PN-0006UK-BO for qemu-devel@nongnu.org; Wed, 13 Sep 2017 07:00:42 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1ds5PM-0006yY-4o for qemu-devel@nongnu.org; Wed, 13 Sep 2017 07:00:41 -0400 Received: from mx1.redhat.com ([209.132.183.28]:46248) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1ds5PL-0006xr-PK for qemu-devel@nongnu.org; Wed, 13 Sep 2017 07:00:40 -0400 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id C56137EBDC for ; Wed, 13 Sep 2017 11:00:38 +0000 (UTC) Received: from secure.mitica (ovpn-117-188.ams2.redhat.com [10.36.117.188]) by smtp.corp.redhat.com (Postfix) with ESMTP id AA68D5191D; Wed, 13 Sep 2017 11:00:35 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com C56137EBDC Authentication-Results: ext-mx03.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx03.extmail.prod.ext.phx2.redhat.com; spf=fail smtp.mailfrom=quintela@redhat.com From: Juan Quintela To: qemu-devel@nongnu.org Date: Wed, 13 Sep 2017 12:59:47 +0200 Message-Id: <20170913105953.13760-15-quintela@redhat.com> In-Reply-To: <20170913105953.13760-1-quintela@redhat.com> References: <20170913105953.13760-1-quintela@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.27]); Wed, 13 Sep 2017 11:00:38 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH v8 14/20] migration: Really use multiple pages at a time X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: lvivier@redhat.com, dgilbert@redhat.com, peterx@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" We now send several pages at a time each time that we wakeup a thread. Signed-off-by: Juan Quintela -- Use iovec's instead of creating the equivalent. Clear memory used by pages (dave) Use g_new0(danp) define MULTIFD_CONTINUE --- migration/ram.c | 57 ++++++++++++++++++++++++++++++++++++++++++++++++-----= ---- 1 file changed, 48 insertions(+), 9 deletions(-) diff --git a/migration/ram.c b/migration/ram.c index 1381bfaf8a..179b45eea7 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -49,6 +49,7 @@ #include "migration/colo.h" #include "sysemu/sysemu.h" #include "qemu/uuid.h" +#include "qemu/iov.h" =20 /***********************************************************/ /* ram save/restore */ @@ -362,6 +363,15 @@ static void compress_threads_save_setup(void) =20 /* Multiple fd's */ =20 +/* used to continue on the same multifd group */ +#define MULTIFD_CONTINUE UINT16_MAX + +typedef struct { + int num; + size_t size; + struct iovec *iov; +} multifd_pages_t; + struct MultiFDSendParams { /* not changed */ uint8_t id; @@ -372,11 +382,7 @@ struct MultiFDSendParams { QemuMutex mutex; /* protected by param mutex */ bool quit; - /* This is a temp field. We are using it now to transmit - something the address of the page. Later in the series, we - change it for the real page. - */ - uint8_t *address; + multifd_pages_t pages; /* protected by multifd mutex */ /* has the thread finish the last submitted job */ bool done; @@ -389,8 +395,24 @@ struct { int count; QemuMutex mutex; QemuSemaphore sem; + multifd_pages_t pages; } *multifd_send_state; =20 +static void multifd_init_pages(multifd_pages_t *pages) +{ + pages->num =3D 0; + pages->size =3D migrate_multifd_page_count(); + pages->iov =3D g_new0(struct iovec, pages->size); +} + +static void multifd_clear_pages(multifd_pages_t *pages) +{ + pages->num =3D 0; + pages->size =3D 0; + g_free(pages->iov); + pages->iov =3D NULL; +} + static void terminate_multifd_send_threads(Error *errp) { int i; @@ -429,9 +451,11 @@ int multifd_save_cleanup(Error **errp) socket_send_channel_destroy(p->c); g_free(p->name); p->name =3D NULL; + multifd_clear_pages(&p->pages); } g_free(multifd_send_state->params); multifd_send_state->params =3D NULL; + multifd_clear_pages(&multifd_send_state->pages); g_free(multifd_send_state); multifd_send_state =3D NULL; return ret; @@ -471,8 +495,8 @@ static void *multifd_send_thread(void *opaque) qemu_mutex_unlock(&p->mutex); break; } - if (p->address) { - p->address =3D 0; + if (p->pages.num) { + p->pages.num =3D 0; qemu_mutex_unlock(&p->mutex); qemu_mutex_lock(&multifd_send_state->mutex); p->done =3D true; @@ -522,6 +546,7 @@ int multifd_save_setup(void) multifd_send_state->count =3D 0; qemu_mutex_init(&multifd_send_state->mutex); qemu_sem_init(&multifd_send_state->sem, 0); + multifd_init_pages(&multifd_send_state->pages); for (i =3D 0; i < thread_count; i++) { MultiFDSendParams *p =3D &multifd_send_state->params[i]; =20 @@ -530,7 +555,7 @@ int multifd_save_setup(void) p->quit =3D false; p->id =3D i; p->done =3D true; - p->address =3D 0; + multifd_init_pages(&p->pages); p->name =3D g_strdup_printf("multifdsend_%d", i); socket_send_channel_create(multifd_new_channel_async, p); } @@ -541,6 +566,17 @@ static uint16_t multifd_send_page(uint8_t *address, bo= ol last_page) { int i; MultiFDSendParams *p =3D NULL; /* make happy gcc */ + multifd_pages_t *pages =3D &multifd_send_state->pages; + + pages->iov[pages->num].iov_base =3D address; + pages->iov[pages->num].iov_len =3D TARGET_PAGE_SIZE; + pages->num++; + + if (!last_page) { + if (pages->num < (pages->size - 1)) { + return MULTIFD_CONTINUE; + } + } =20 qemu_sem_wait(&multifd_send_state->sem); qemu_mutex_lock(&multifd_send_state->mutex); @@ -554,7 +590,10 @@ static uint16_t multifd_send_page(uint8_t *address, bo= ol last_page) } qemu_mutex_unlock(&multifd_send_state->mutex); qemu_mutex_lock(&p->mutex); - p->address =3D address; + p->pages.num =3D pages->num; + iov_copy(p->pages.iov, pages->num, pages->iov, pages->num, 0, + iov_size(pages->iov, pages->num)); + pages->num =3D 0; qemu_mutex_unlock(&p->mutex); qemu_sem_post(&p->sem); =20 --=20 2.13.5 From nobody Thu May 2 15:13:04 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 150530105065243.5525237477442; Wed, 13 Sep 2017 04:10:50 -0700 (PDT) Received: from localhost ([::1]:41599 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ds5ZB-00075x-Ps for importer@patchew.org; Wed, 13 Sep 2017 07:10:49 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:42383) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ds5PP-0006Wr-KQ for qemu-devel@nongnu.org; Wed, 13 Sep 2017 07:00:46 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1ds5PO-00070h-LB for qemu-devel@nongnu.org; Wed, 13 Sep 2017 07:00:43 -0400 Received: from mx1.redhat.com ([209.132.183.28]:56322) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1ds5PO-00070K-Eb for qemu-devel@nongnu.org; Wed, 13 Sep 2017 07:00:42 -0400 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 698A9883A2 for ; Wed, 13 Sep 2017 11:00:41 +0000 (UTC) Received: from secure.mitica (ovpn-117-188.ams2.redhat.com [10.36.117.188]) by smtp.corp.redhat.com (Postfix) with ESMTP id 3A41561786; Wed, 13 Sep 2017 11:00:38 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com 698A9883A2 Authentication-Results: ext-mx02.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx02.extmail.prod.ext.phx2.redhat.com; spf=fail smtp.mailfrom=quintela@redhat.com From: Juan Quintela To: qemu-devel@nongnu.org Date: Wed, 13 Sep 2017 12:59:48 +0200 Message-Id: <20170913105953.13760-16-quintela@redhat.com> In-Reply-To: <20170913105953.13760-1-quintela@redhat.com> References: <20170913105953.13760-1-quintela@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.26]); Wed, 13 Sep 2017 11:00:41 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH v8 15/20] migration: Send the fd number which we are going to use for this page X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: lvivier@redhat.com, dgilbert@redhat.com, peterx@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" We are still sending the page through the main channel, that would change later in the series Signed-off-by: Juan Quintela Reviewed-by: Dr. David Alan Gilbert --- migration/ram.c | 13 +++++++++++-- 1 file changed, 11 insertions(+), 2 deletions(-) diff --git a/migration/ram.c b/migration/ram.c index 179b45eea7..3f60f7e40c 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -597,7 +597,7 @@ static uint16_t multifd_send_page(uint8_t *address, boo= l last_page) qemu_mutex_unlock(&p->mutex); qemu_sem_post(&p->sem); =20 - return 0; + return i; } =20 struct MultiFDRecvParams { @@ -1186,6 +1186,7 @@ static int ram_multifd_page(RAMState *rs, PageSearchS= tatus *pss, bool last_stage) { int pages; + uint16_t fd_num; uint8_t *p; RAMBlock *block =3D pss->block; ram_addr_t offset =3D pss->page << TARGET_PAGE_BITS; @@ -1197,8 +1198,10 @@ static int ram_multifd_page(RAMState *rs, PageSearch= Status *pss, ram_counters.transferred +=3D save_page_header(rs, rs->f, block, offset | RAM_SAVE_FLAG_MULTIFD_PAGE); + fd_num =3D multifd_send_page(p, rs->migration_dirty_pages =3D=3D 1= ); + qemu_put_be16(rs->f, fd_num); + ram_counters.transferred +=3D 2; /* size of fd_num */ qemu_put_buffer(rs->f, p, TARGET_PAGE_SIZE); - multifd_send_page(p, rs->migration_dirty_pages =3D=3D 1); ram_counters.transferred +=3D TARGET_PAGE_SIZE; pages =3D 1; ram_counters.normal++; @@ -2948,6 +2951,7 @@ static int ram_load(QEMUFile *f, void *opaque, int ve= rsion_id) while (!postcopy_running && !ret && !(flags & RAM_SAVE_FLAG_EOS)) { ram_addr_t addr, total_ram_bytes; void *host =3D NULL; + uint16_t fd_num; uint8_t ch; =20 addr =3D qemu_get_be64(f); @@ -3058,6 +3062,11 @@ static int ram_load(QEMUFile *f, void *opaque, int v= ersion_id) break; =20 case RAM_SAVE_FLAG_MULTIFD_PAGE: + fd_num =3D qemu_get_be16(f); + if (fd_num !=3D 0) { + /* this is yet an unused variable, changed later */ + fd_num =3D fd_num; + } qemu_get_buffer(f, host, TARGET_PAGE_SIZE); break; =20 --=20 2.13.5 From nobody Thu May 2 15:13:04 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 15053012078091000.4877913576892; Wed, 13 Sep 2017 04:13:27 -0700 (PDT) Received: from localhost ([::1]:41607 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ds5bi-00019C-Vs for importer@patchew.org; Wed, 13 Sep 2017 07:13:27 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:42452) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ds5PW-0006dh-Pu for qemu-devel@nongnu.org; Wed, 13 Sep 2017 07:00:52 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1ds5PQ-000724-C5 for qemu-devel@nongnu.org; Wed, 13 Sep 2017 07:00:50 -0400 Received: from mx1.redhat.com ([209.132.183.28]:43974) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1ds5PQ-00071F-4E for qemu-devel@nongnu.org; Wed, 13 Sep 2017 07:00:44 -0400 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 1AF96C04B92A for ; Wed, 13 Sep 2017 11:00:43 +0000 (UTC) Received: from secure.mitica (ovpn-117-188.ams2.redhat.com [10.36.117.188]) by smtp.corp.redhat.com (Postfix) with ESMTP id BB44A5C545; Wed, 13 Sep 2017 11:00:41 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com 1AF96C04B92A Authentication-Results: ext-mx07.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx07.extmail.prod.ext.phx2.redhat.com; spf=fail smtp.mailfrom=quintela@redhat.com From: Juan Quintela To: qemu-devel@nongnu.org Date: Wed, 13 Sep 2017 12:59:49 +0200 Message-Id: <20170913105953.13760-17-quintela@redhat.com> In-Reply-To: <20170913105953.13760-1-quintela@redhat.com> References: <20170913105953.13760-1-quintela@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.31]); Wed, 13 Sep 2017 11:00:43 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH v8 16/20] migration: Create thread infrastructure for multifd recv side X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: lvivier@redhat.com, dgilbert@redhat.com, peterx@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" We make the locking and the transfer of information specific, even if we are still receiving things through the main thread. Signed-off-by: Juan Quintela -- We split when we create the main channel and where we start the main migration thread, so we wait for the creation of the other threads. Use multifd_clear_pages(). Don't remove object_unref() --- migration/migration.c | 7 ++++--- migration/migration.h | 1 + migration/ram.c | 55 +++++++++++++++++++++++++++++++++++++++++++++++= ---- migration/socket.c | 2 +- 4 files changed, 57 insertions(+), 8 deletions(-) diff --git a/migration/migration.c b/migration/migration.c index 085177ca26..674c254c3f 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -388,7 +388,7 @@ static void migration_incoming_setup(QEMUFile *f) qemu_file_set_blocking(f, false); } =20 -static void migration_incoming_process(void) +void migration_incoming_process(void) { Coroutine *co =3D qemu_coroutine_create(process_incoming_migration_co,= NULL); qemu_coroutine_enter(co); @@ -406,9 +406,10 @@ void migration_ioc_process_incoming(QIOChannel *ioc) =20 if (!mis->from_src_file) { QEMUFile *f =3D qemu_fopen_channel_input(ioc); - migration_fd_process_incoming(f); + migration_incoming_setup(f); + return; } - /* We still only have a single channel. Nothing to do here yet */ + multifd_new_channel(ioc); } =20 /** diff --git a/migration/migration.h b/migration/migration.h index da39d4c711..382a0449d5 100644 --- a/migration/migration.h +++ b/migration/migration.h @@ -158,6 +158,7 @@ void migrate_set_state(int *state, int old_state, int n= ew_state); =20 void migration_fd_process_incoming(QEMUFile *f); void migration_ioc_process_incoming(QIOChannel *ioc); +void migration_incoming_process(void); =20 bool migration_has_all_channels(void); =20 diff --git a/migration/ram.c b/migration/ram.c index 3f60f7e40c..d6aa516f6d 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -601,13 +601,18 @@ static uint16_t multifd_send_page(uint8_t *address, b= ool last_page) } =20 struct MultiFDRecvParams { + /* not changed */ uint8_t id; char *name; QemuThread thread; QIOChannel *c; + QemuSemaphore ready; QemuSemaphore sem; QemuMutex mutex; + /* proteced by param mutex */ bool quit; + multifd_pages_t pages; + bool done; }; typedef struct MultiFDRecvParams MultiFDRecvParams; =20 @@ -617,6 +622,7 @@ struct { int count; /* Should we finish */ bool quit; + multifd_pages_t pages; } *multifd_recv_state; =20 static void terminate_multifd_recv_threads(Error *errp) @@ -638,6 +644,7 @@ static void terminate_multifd_recv_threads(Error *errp) p->quit =3D true; qemu_sem_post(&p->sem); qemu_mutex_unlock(&p->mutex); + multifd_clear_pages(&p->pages); } } =20 @@ -662,6 +669,7 @@ int multifd_load_cleanup(Error **errp) } g_free(multifd_recv_state->params); multifd_recv_state->params =3D NULL; + multifd_clear_pages(&multifd_recv_state->pages); g_free(multifd_recv_state); multifd_recv_state =3D NULL; =20 @@ -672,12 +680,20 @@ static void *multifd_recv_thread(void *opaque) { MultiFDRecvParams *p =3D opaque; =20 + qemu_sem_post(&p->ready); while (true) { qemu_mutex_lock(&p->mutex); if (p->quit) { qemu_mutex_unlock(&p->mutex); break; } + if (p->pages.num) { + p->pages.num =3D 0; + p->done =3D true; + qemu_mutex_unlock(&p->mutex); + qemu_sem_post(&p->ready); + continue; + } qemu_mutex_unlock(&p->mutex); qemu_sem_wait(&p->sem); } @@ -723,8 +739,11 @@ void multifd_new_channel(QIOChannel *ioc) } qemu_mutex_init(&p->mutex); qemu_sem_init(&p->sem, 0); + qemu_sem_init(&p->ready, 0); p->quit =3D false; p->id =3D id; + p->done =3D false; + multifd_init_pages(&p->pages); p->c =3D ioc; multifd_recv_state->count++; p->name =3D g_strdup_printf("multifdrecv_%d", id); @@ -744,6 +763,7 @@ int multifd_load_setup(void) multifd_recv_state->params =3D g_new0(MultiFDRecvParams, thread_count); multifd_recv_state->count =3D 0; multifd_recv_state->quit =3D false; + multifd_init_pages(&multifd_recv_state->pages); return 0; } =20 @@ -752,6 +772,36 @@ int multifd_created_channels(void) return multifd_recv_state->count; } =20 +static void multifd_recv_page(uint8_t *address, uint16_t fd_num) +{ + int thread_count; + MultiFDRecvParams *p; + multifd_pages_t *pages =3D &multifd_recv_state->pages; + + pages->iov[pages->num].iov_base =3D address; + pages->iov[pages->num].iov_len =3D TARGET_PAGE_SIZE; + pages->num++; + + if (fd_num =3D=3D MULTIFD_CONTINUE) { + return; + } + + thread_count =3D migrate_multifd_channels(); + assert(fd_num < thread_count); + p =3D &multifd_recv_state->params[fd_num]; + + qemu_sem_wait(&p->ready); + + qemu_mutex_lock(&p->mutex); + p->done =3D false; + iov_copy(p->pages.iov, pages->num, pages->iov, pages->num, 0, + iov_size(pages->iov, pages->num)); + p->pages.num =3D pages->num; + pages->num =3D 0; + qemu_mutex_unlock(&p->mutex); + qemu_sem_post(&p->sem); +} + /** * save_page_header: write page header to wire * @@ -3063,10 +3113,7 @@ static int ram_load(QEMUFile *f, void *opaque, int v= ersion_id) =20 case RAM_SAVE_FLAG_MULTIFD_PAGE: fd_num =3D qemu_get_be16(f); - if (fd_num !=3D 0) { - /* this is yet an unused variable, changed later */ - fd_num =3D fd_num; - } + multifd_recv_page(host, fd_num); qemu_get_buffer(f, host, TARGET_PAGE_SIZE); break; =20 diff --git a/migration/socket.c b/migration/socket.c index 22fb05edc8..63dedb22e8 100644 --- a/migration/socket.c +++ b/migration/socket.c @@ -180,12 +180,12 @@ static gboolean socket_accept_incoming_migration(QIOC= hannel *ioc, =20 qio_channel_set_name(QIO_CHANNEL(sioc), "migration-socket-incoming"); migration_channel_process_incoming(QIO_CHANNEL(sioc)); - object_unref(OBJECT(sioc)); =20 out: if (migration_has_all_channels()) { /* Close listening socket as its no longer needed */ qio_channel_close(ioc, NULL); + migration_incoming_process(); return G_SOURCE_REMOVE; } else { return G_SOURCE_CONTINUE; --=20 2.13.5 From nobody Thu May 2 15:13:04 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1505301189222418.7778362194829; Wed, 13 Sep 2017 04:13:09 -0700 (PDT) Received: from localhost ([::1]:41606 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ds5bQ-0000pH-8r for importer@patchew.org; Wed, 13 Sep 2017 07:13:08 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:42458) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ds5PW-0006dp-UM for qemu-devel@nongnu.org; Wed, 13 Sep 2017 07:00:51 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1ds5PT-00074R-08 for qemu-devel@nongnu.org; Wed, 13 Sep 2017 07:00:51 -0400 Received: from mx1.redhat.com ([209.132.183.28]:43596) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1ds5PS-00073e-Eg for qemu-devel@nongnu.org; Wed, 13 Sep 2017 07:00:46 -0400 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 69EB7155DF for ; Wed, 13 Sep 2017 11:00:45 +0000 (UTC) Received: from secure.mitica (ovpn-117-188.ams2.redhat.com [10.36.117.188]) by smtp.corp.redhat.com (Postfix) with ESMTP id 71CC75C545; Wed, 13 Sep 2017 11:00:43 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com 69EB7155DF Authentication-Results: ext-mx05.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx05.extmail.prod.ext.phx2.redhat.com; spf=fail smtp.mailfrom=quintela@redhat.com From: Juan Quintela To: qemu-devel@nongnu.org Date: Wed, 13 Sep 2017 12:59:50 +0200 Message-Id: <20170913105953.13760-18-quintela@redhat.com> In-Reply-To: <20170913105953.13760-1-quintela@redhat.com> References: <20170913105953.13760-1-quintela@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.29]); Wed, 13 Sep 2017 11:00:45 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH v8 17/20] migration: Test new fd infrastructure X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: lvivier@redhat.com, dgilbert@redhat.com, peterx@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" We just send the address through the alternate channels and test that it is ok. Signed-off-by: Juan Quintela -- Use qio_channel_*all functions --- migration/ram.c | 39 +++++++++++++++++++++++++++++++++++++++ 1 file changed, 39 insertions(+) diff --git a/migration/ram.c b/migration/ram.c index d6aa516f6d..1b1e346ae0 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -496,8 +496,24 @@ static void *multifd_send_thread(void *opaque) break; } if (p->pages.num) { + Error *local_err =3D NULL; + size_t ret; + int i; + int num; + + num =3D p->pages.num; p->pages.num =3D 0; qemu_mutex_unlock(&p->mutex); + + for (i =3D 0; i < num; i++) { + ret =3D qio_channel_write_all(p->c, + (const char *)&p->pages.iov[i].iov_base, + sizeof(uint8_t *), &local_err); + if (ret !=3D 0) { + terminate_multifd_send_threads(local_err); + return NULL; + } + } qemu_mutex_lock(&multifd_send_state->mutex); p->done =3D true; qemu_mutex_unlock(&multifd_send_state->mutex); @@ -679,6 +695,7 @@ int multifd_load_cleanup(Error **errp) static void *multifd_recv_thread(void *opaque) { MultiFDRecvParams *p =3D opaque; + uint8_t *recv_address; =20 qemu_sem_post(&p->ready); while (true) { @@ -688,7 +705,29 @@ static void *multifd_recv_thread(void *opaque) break; } if (p->pages.num) { + Error *local_err =3D NULL; + size_t ret; + int i; + int num; + + num =3D p->pages.num; p->pages.num =3D 0; + + for (i =3D 0; i < num; i++) { + ret =3D qio_channel_read_all(p->c, (char *)&recv_address, + sizeof(uint8_t *), &local_err); + if (ret !=3D 0) { + terminate_multifd_recv_threads(local_err); + return NULL; + } + if (recv_address !=3D p->pages.iov[i].iov_base) { + error_setg(&local_err, "received %p and expecting %p (= %d)", + recv_address, p->pages.iov[i].iov_base, i); + terminate_multifd_recv_threads(local_err); + return NULL; + } + } + p->done =3D true; qemu_mutex_unlock(&p->mutex); qemu_sem_post(&p->ready); --=20 2.13.5 From nobody Thu May 2 15:13:04 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 150530151466218.67402692691269; Wed, 13 Sep 2017 04:18:34 -0700 (PDT) Received: from localhost ([::1]:41638 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ds5gf-00064n-B3 for importer@patchew.org; Wed, 13 Sep 2017 07:18:33 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:42512) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ds5Pa-0006gt-1Y for qemu-devel@nongnu.org; Wed, 13 Sep 2017 07:00:59 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1ds5PU-00075e-Iy for qemu-devel@nongnu.org; Wed, 13 Sep 2017 07:00:54 -0400 Received: from mx1.redhat.com ([209.132.183.28]:44160) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1ds5PU-000757-Bb for qemu-devel@nongnu.org; Wed, 13 Sep 2017 07:00:48 -0400 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 57E42C04B944 for ; Wed, 13 Sep 2017 11:00:47 +0000 (UTC) Received: from secure.mitica (ovpn-117-188.ams2.redhat.com [10.36.117.188]) by smtp.corp.redhat.com (Postfix) with ESMTP id BDE9E61786; Wed, 13 Sep 2017 11:00:45 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com 57E42C04B944 Authentication-Results: ext-mx07.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx07.extmail.prod.ext.phx2.redhat.com; spf=fail smtp.mailfrom=quintela@redhat.com From: Juan Quintela To: qemu-devel@nongnu.org Date: Wed, 13 Sep 2017 12:59:51 +0200 Message-Id: <20170913105953.13760-19-quintela@redhat.com> In-Reply-To: <20170913105953.13760-1-quintela@redhat.com> References: <20170913105953.13760-1-quintela@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.31]); Wed, 13 Sep 2017 11:00:47 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH v8 18/20] migration: Rename initial_bytes X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: lvivier@redhat.com, dgilbert@redhat.com, peterx@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Now it is called qemu_file_bytes that reflects better what it does, and we create qemu_file_bytes_now to not have to call qemu_ftell() twice. Signed-off-by: Juan Quintela Reviewed-by: Dr. David Alan Gilbert --- migration/migration.c | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/migration/migration.c b/migration/migration.c index 674c254c3f..67f3375ddc 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -2065,13 +2065,13 @@ static void *migration_thread(void *opaque) /* Used by the bandwidth calcs, updated later */ int64_t initial_time =3D qemu_clock_get_ms(QEMU_CLOCK_REALTIME); int64_t setup_start =3D qemu_clock_get_ms(QEMU_CLOCK_HOST); - int64_t initial_bytes =3D 0; /* * The final stage happens when the remaining data is smaller than * this threshold; it's calculated from the requested downtime and * measured bandwidth */ int64_t threshold_size =3D 0; + int64_t qemu_file_bytes =3D 0; int64_t start_time =3D initial_time; int64_t end_time; bool old_vm_running =3D false; @@ -2159,8 +2159,9 @@ static void *migration_thread(void *opaque) } current_time =3D qemu_clock_get_ms(QEMU_CLOCK_REALTIME); if (current_time >=3D initial_time + BUFFER_DELAY) { - uint64_t transferred_bytes =3D qemu_ftell(s->to_dst_file) - - initial_bytes; + uint64_t qemu_file_bytes_now =3D qemu_ftell(s->to_dst_file); + uint64_t transferred_bytes =3D + qemu_file_bytes_now - qemu_file_bytes; uint64_t time_spent =3D current_time - initial_time; double bandwidth =3D (double)transferred_bytes / time_spent; threshold_size =3D bandwidth * s->parameters.downtime_limit; @@ -2179,7 +2180,7 @@ static void *migration_thread(void *opaque) =20 qemu_file_reset_rate_limit(s->to_dst_file); initial_time =3D current_time; - initial_bytes =3D qemu_ftell(s->to_dst_file); + qemu_file_bytes =3D qemu_file_bytes_now; } if (qemu_file_rate_limit(s->to_dst_file)) { /* usleep expects microseconds */ --=20 2.13.5 From nobody Thu May 2 15:13:04 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1505301367822934.0199918001316; Wed, 13 Sep 2017 04:16:07 -0700 (PDT) Received: from localhost ([::1]:41629 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ds5eI-0003qc-RI for importer@patchew.org; Wed, 13 Sep 2017 07:16:06 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:42493) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ds5PZ-0006fj-0G for qemu-devel@nongnu.org; Wed, 13 Sep 2017 07:00:56 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1ds5PX-00079o-Te for qemu-devel@nongnu.org; Wed, 13 Sep 2017 07:00:53 -0400 Received: from mx1.redhat.com ([209.132.183.28]:43798) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1ds5PX-00078V-Ke for qemu-devel@nongnu.org; Wed, 13 Sep 2017 07:00:51 -0400 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 94DAD13A56 for ; Wed, 13 Sep 2017 11:00:50 +0000 (UTC) Received: from secure.mitica (ovpn-117-188.ams2.redhat.com [10.36.117.188]) by smtp.corp.redhat.com (Postfix) with ESMTP id ACCE161786; Wed, 13 Sep 2017 11:00:47 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com 94DAD13A56 Authentication-Results: ext-mx05.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx05.extmail.prod.ext.phx2.redhat.com; spf=fail smtp.mailfrom=quintela@redhat.com From: Juan Quintela To: qemu-devel@nongnu.org Date: Wed, 13 Sep 2017 12:59:52 +0200 Message-Id: <20170913105953.13760-20-quintela@redhat.com> In-Reply-To: <20170913105953.13760-1-quintela@redhat.com> References: <20170913105953.13760-1-quintela@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.29]); Wed, 13 Sep 2017 11:00:50 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH v8 19/20] migration: Transfer pages over new channels X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: lvivier@redhat.com, dgilbert@redhat.com, peterx@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" We switch for sending the page number to send real pages. Signed-off-by: Juan Quintela -- Remove the HACK bit, now we have the function that calculates the size of a page exported. --- migration/migration.c | 7 ++++++- migration/ram.c | 39 +++++++++++---------------------------- 2 files changed, 17 insertions(+), 29 deletions(-) diff --git a/migration/migration.c b/migration/migration.c index 67f3375ddc..b26b987fe9 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -2072,6 +2072,7 @@ static void *migration_thread(void *opaque) */ int64_t threshold_size =3D 0; int64_t qemu_file_bytes =3D 0; + int64_t multifd_pages =3D 0; int64_t start_time =3D initial_time; int64_t end_time; bool old_vm_running =3D false; @@ -2160,8 +2161,11 @@ static void *migration_thread(void *opaque) current_time =3D qemu_clock_get_ms(QEMU_CLOCK_REALTIME); if (current_time >=3D initial_time + BUFFER_DELAY) { uint64_t qemu_file_bytes_now =3D qemu_ftell(s->to_dst_file); + uint64_t multifd_pages_now =3D ram_counters.multifd; uint64_t transferred_bytes =3D - qemu_file_bytes_now - qemu_file_bytes; + (qemu_file_bytes_now - qemu_file_bytes) + + (multifd_pages_now - multifd_pages) * + qemu_target_page_size(); uint64_t time_spent =3D current_time - initial_time; double bandwidth =3D (double)transferred_bytes / time_spent; threshold_size =3D bandwidth * s->parameters.downtime_limit; @@ -2181,6 +2185,7 @@ static void *migration_thread(void *opaque) qemu_file_reset_rate_limit(s->to_dst_file); initial_time =3D current_time; qemu_file_bytes =3D qemu_file_bytes_now; + multifd_pages =3D multifd_pages_now; } if (qemu_file_rate_limit(s->to_dst_file)) { /* usleep expects microseconds */ diff --git a/migration/ram.c b/migration/ram.c index 1b1e346ae0..2d89d4e39f 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -498,21 +498,15 @@ static void *multifd_send_thread(void *opaque) if (p->pages.num) { Error *local_err =3D NULL; size_t ret; - int i; int num; =20 num =3D p->pages.num; p->pages.num =3D 0; qemu_mutex_unlock(&p->mutex); - - for (i =3D 0; i < num; i++) { - ret =3D qio_channel_write_all(p->c, - (const char *)&p->pages.iov[i].iov_base, - sizeof(uint8_t *), &local_err); - if (ret !=3D 0) { - terminate_multifd_send_threads(local_err); - return NULL; - } + ret =3D qio_channel_writev_all(p->c, p->pages.iov, num, &local= _err); + if (ret !=3D 0) { + terminate_multifd_send_threads(local_err); + return NULL; } qemu_mutex_lock(&multifd_send_state->mutex); p->done =3D true; @@ -695,7 +689,6 @@ int multifd_load_cleanup(Error **errp) static void *multifd_recv_thread(void *opaque) { MultiFDRecvParams *p =3D opaque; - uint8_t *recv_address; =20 qemu_sem_post(&p->ready); while (true) { @@ -707,27 +700,16 @@ static void *multifd_recv_thread(void *opaque) if (p->pages.num) { Error *local_err =3D NULL; size_t ret; - int i; int num; =20 num =3D p->pages.num; p->pages.num =3D 0; =20 - for (i =3D 0; i < num; i++) { - ret =3D qio_channel_read_all(p->c, (char *)&recv_address, - sizeof(uint8_t *), &local_err); - if (ret !=3D 0) { - terminate_multifd_recv_threads(local_err); - return NULL; - } - if (recv_address !=3D p->pages.iov[i].iov_base) { - error_setg(&local_err, "received %p and expecting %p (= %d)", - recv_address, p->pages.iov[i].iov_base, i); - terminate_multifd_recv_threads(local_err); - return NULL; - } + ret =3D qio_channel_readv_all(p->c, p->pages.iov, num, &local_= err); + if (ret !=3D 0) { + terminate_multifd_recv_threads(local_err); + return NULL; } - p->done =3D true; qemu_mutex_unlock(&p->mutex); qemu_sem_post(&p->ready); @@ -1289,8 +1271,10 @@ static int ram_multifd_page(RAMState *rs, PageSearch= Status *pss, offset | RAM_SAVE_FLAG_MULTIFD_PAGE); fd_num =3D multifd_send_page(p, rs->migration_dirty_pages =3D=3D 1= ); qemu_put_be16(rs->f, fd_num); + if (fd_num !=3D MULTIFD_CONTINUE) { + qemu_fflush(rs->f); + } ram_counters.transferred +=3D 2; /* size of fd_num */ - qemu_put_buffer(rs->f, p, TARGET_PAGE_SIZE); ram_counters.transferred +=3D TARGET_PAGE_SIZE; pages =3D 1; ram_counters.normal++; @@ -3153,7 +3137,6 @@ static int ram_load(QEMUFile *f, void *opaque, int ve= rsion_id) case RAM_SAVE_FLAG_MULTIFD_PAGE: fd_num =3D qemu_get_be16(f); multifd_recv_page(host, fd_num); - qemu_get_buffer(f, host, TARGET_PAGE_SIZE); break; =20 case RAM_SAVE_FLAG_EOS: --=20 2.13.5 From nobody Thu May 2 15:13:04 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1505301207984311.25568207286494; Wed, 13 Sep 2017 04:13:27 -0700 (PDT) Received: from localhost ([::1]:41608 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ds5bj-0001Ae-1s for importer@patchew.org; Wed, 13 Sep 2017 07:13:27 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:42542) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ds5Pc-0006j0-Cn for qemu-devel@nongnu.org; Wed, 13 Sep 2017 07:00:57 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1ds5PZ-0007Bm-Ri for qemu-devel@nongnu.org; Wed, 13 Sep 2017 07:00:56 -0400 Received: from mx1.redhat.com ([209.132.183.28]:46818) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1ds5PZ-0007Aw-Jb for qemu-devel@nongnu.org; Wed, 13 Sep 2017 07:00:53 -0400 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 91CCC80F99 for ; Wed, 13 Sep 2017 11:00:52 +0000 (UTC) Received: from secure.mitica (ovpn-117-188.ams2.redhat.com [10.36.117.188]) by smtp.corp.redhat.com (Postfix) with ESMTP id E424561786; Wed, 13 Sep 2017 11:00:50 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com 91CCC80F99 Authentication-Results: ext-mx03.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx03.extmail.prod.ext.phx2.redhat.com; spf=fail smtp.mailfrom=quintela@redhat.com From: Juan Quintela To: qemu-devel@nongnu.org Date: Wed, 13 Sep 2017 12:59:53 +0200 Message-Id: <20170913105953.13760-21-quintela@redhat.com> In-Reply-To: <20170913105953.13760-1-quintela@redhat.com> References: <20170913105953.13760-1-quintela@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.27]); Wed, 13 Sep 2017 11:00:52 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH v8 20/20] migration: Flush receive queue X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: lvivier@redhat.com, dgilbert@redhat.com, peterx@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Each time that we sync the bitmap, it is a possiblity that we receive a page that is being processed by a different thread. We fix this problem just making sure that we wait for all receiving threads to finish its work before we procedeed with the next stage. We are low on page flags, so we use a combination that is not valid to emit that message: MULTIFD_PAGE and COMPRESSED. I tried to make a migration command for it, but it don't work because we sync the bitmap sometimes when we have already sent the beggining of the section, so I just added a new page flag. Signed-off-by: Juan Quintela -- Create RAM_SAVE_FLAG_MULTIFD_SYNC (dave suggestion) Move the set of need_flush to inside the bitmap_sync code (peter suggestion) --- migration/ram.c | 55 +++++++++++++++++++++++++++++++++++++++++++++++++++++= ++ 1 file changed, 55 insertions(+) diff --git a/migration/ram.c b/migration/ram.c index 2d89d4e39f..2ab4ddb65e 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -71,6 +71,14 @@ #define RAM_SAVE_FLAG_COMPRESS_PAGE 0x100 #define RAM_SAVE_FLAG_MULTIFD_PAGE 0x200 =20 +/* We are getting low on pages flags, so we start using combinations + When we need to flush a page, we sent it as + RAM_SAVE_FLAG_MULTIFD_PAGE | RAM_SAVE_FLAG_COMPRESS_PAGE + We don't allow that combination +*/ +#define RAM_SAVE_FLAG_MULTIFD_SYNC \ + (RAM_SAVE_FLAG_MULTIFD_PAGE | RAM_SAVE_FLAG_ZERO) + static inline bool is_zero_range(uint8_t *p, uint64_t size) { return buffer_is_zero(p, size); @@ -193,6 +201,9 @@ struct RAMState { uint64_t iterations_prev; /* Iterations since start */ uint64_t iterations; + /* Indicates if we have synced the bitmap and we need to assure that + target has processeed all previous pages */ + bool multifd_needs_flush; /* number of dirty bits in the bitmap */ uint64_t migration_dirty_pages; /* protects modification of the bitmap */ @@ -618,9 +629,11 @@ struct MultiFDRecvParams { QIOChannel *c; QemuSemaphore ready; QemuSemaphore sem; + QemuCond cond_sync; QemuMutex mutex; /* proteced by param mutex */ bool quit; + bool sync; multifd_pages_t pages; bool done; }; @@ -673,6 +686,7 @@ int multifd_load_cleanup(Error **errp) qemu_thread_join(&p->thread); qemu_mutex_destroy(&p->mutex); qemu_sem_destroy(&p->sem); + qemu_cond_destroy(&p->cond_sync); socket_recv_channel_destroy(p->c); g_free(p->name); p->name =3D NULL; @@ -711,6 +725,10 @@ static void *multifd_recv_thread(void *opaque) return NULL; } p->done =3D true; + if (p->sync) { + qemu_cond_signal(&p->cond_sync); + p->sync =3D false; + } qemu_mutex_unlock(&p->mutex); qemu_sem_post(&p->ready); continue; @@ -761,9 +779,11 @@ void multifd_new_channel(QIOChannel *ioc) qemu_mutex_init(&p->mutex); qemu_sem_init(&p->sem, 0); qemu_sem_init(&p->ready, 0); + qemu_cond_init(&p->cond_sync); p->quit =3D false; p->id =3D id; p->done =3D false; + p->sync =3D false; multifd_init_pages(&p->pages); p->c =3D ioc; multifd_recv_state->count++; @@ -823,6 +843,27 @@ static void multifd_recv_page(uint8_t *address, uint16= _t fd_num) qemu_sem_post(&p->sem); } =20 +static int multifd_flush(void) +{ + int i, thread_count; + + if (!migrate_use_multifd()) { + return 0; + } + thread_count =3D migrate_multifd_channels(); + for (i =3D 0; i < thread_count; i++) { + MultiFDRecvParams *p =3D &multifd_recv_state->params[i]; + + qemu_mutex_lock(&p->mutex); + while (!p->done) { + p->sync =3D true; + qemu_cond_wait(&p->cond_sync, &p->mutex); + } + qemu_mutex_unlock(&p->mutex); + } + return 0; +} + /** * save_page_header: write page header to wire * @@ -840,6 +881,12 @@ static size_t save_page_header(RAMState *rs, QEMUFile = *f, RAMBlock *block, { size_t size, len; =20 + if (rs->multifd_needs_flush && + (offset & RAM_SAVE_FLAG_MULTIFD_PAGE)) { + offset |=3D RAM_SAVE_FLAG_ZERO; + rs->multifd_needs_flush =3D false; + } + if (block =3D=3D rs->last_sent_block) { offset |=3D RAM_SAVE_FLAG_CONTINUE; } @@ -1125,6 +1172,9 @@ static void migration_bitmap_sync(RAMState *rs) if (migrate_use_events()) { qapi_event_send_migration_pass(ram_counters.dirty_sync_count, NULL= ); } + if (!rs->ram_bulk_stage && migrate_use_multifd()) { + rs->multifd_needs_flush =3D true; + } } =20 /** @@ -3043,6 +3093,11 @@ static int ram_load(QEMUFile *f, void *opaque, int v= ersion_id) break; } =20 + if ((flags & RAM_SAVE_FLAG_MULTIFD_SYNC) + =3D=3D RAM_SAVE_FLAG_MULTIFD_SYNC) { + multifd_flush(); + flags =3D flags & ~RAM_SAVE_FLAG_ZERO; + } if (flags & (RAM_SAVE_FLAG_ZERO | RAM_SAVE_FLAG_PAGE | RAM_SAVE_FLAG_COMPRESS_PAGE | RAM_SAVE_FLAG_XBZRLE | RAM_SAVE_FLAG_MULTIFD_PAGE)) { --=20 2.13.5