From nobody Wed Oct 29 06:44:06 2025 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 152465577233918.883491882460476; Wed, 25 Apr 2018 04:29:32 -0700 (PDT) Received: from localhost ([::1]:35846 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fBIc7-00043B-H3 for importer@patchew.org; Wed, 25 Apr 2018 07:29:31 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:34004) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fBIaC-0002gF-3T for qemu-devel@nongnu.org; Wed, 25 Apr 2018 07:27:32 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fBIaB-0004lt-4O for qemu-devel@nongnu.org; Wed, 25 Apr 2018 07:27:32 -0400 Received: from mx3-rdu2.redhat.com ([66.187.233.73]:42382 helo=mx1.redhat.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1fBIaB-0004lB-0f for qemu-devel@nongnu.org; Wed, 25 Apr 2018 07:27:31 -0400 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.rdu2.redhat.com [10.11.54.6]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id CBF1481A88AD for ; Wed, 25 Apr 2018 11:27:27 +0000 (UTC) Received: from secure.mitica (ovpn-117-169.ams2.redhat.com [10.36.117.169]) by smtp.corp.redhat.com (Postfix) with ESMTP id C35B7215CDC8; Wed, 25 Apr 2018 11:27:26 +0000 (UTC) From: Juan Quintela To: qemu-devel@nongnu.org Date: Wed, 25 Apr 2018 13:27:03 +0200 Message-Id: <20180425112723.1111-2-quintela@redhat.com> In-Reply-To: <20180425112723.1111-1-quintela@redhat.com> References: <20180425112723.1111-1-quintela@redhat.com> X-Scanned-By: MIMEDefang 2.78 on 10.11.54.6 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.8]); Wed, 25 Apr 2018 11:27:27 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.8]); Wed, 25 Apr 2018 11:27:27 +0000 (UTC) for IP:'10.11.54.6' DOMAIN:'int-mx06.intmail.prod.int.rdu2.redhat.com' HELO:'smtp.corp.redhat.com' FROM:'quintela@redhat.com' RCPT:'' X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 66.187.233.73 Subject: [Qemu-devel] [PATCH v12 01/21] migration: Set error state in case of error X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: lvivier@redhat.com, dgilbert@redhat.com, peterx@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Signed-off-by: Juan Quintela --- migration/ram.c | 24 ++++++++++++++++++++++-- 1 file changed, 22 insertions(+), 2 deletions(-) diff --git a/migration/ram.c b/migration/ram.c index 0e90efa092..2ae560ea80 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -415,10 +415,20 @@ struct { int count; } *multifd_send_state; =20 -static void terminate_multifd_send_threads(Error *errp) +static void terminate_multifd_send_threads(Error *err) { int i; =20 + if (err) { + MigrationState *s =3D migrate_get_current(); + migrate_set_error(s, err); + if (s->state =3D=3D MIGRATION_STATUS_SETUP || + s->state =3D=3D MIGRATION_STATUS_ACTIVE) { + migrate_set_state(&s->state, s->state, + MIGRATION_STATUS_FAILED); + } + } + for (i =3D 0; i < multifd_send_state->count; i++) { MultiFDSendParams *p =3D &multifd_send_state->params[i]; =20 @@ -515,10 +525,20 @@ struct { int count; } *multifd_recv_state; =20 -static void terminate_multifd_recv_threads(Error *errp) +static void terminate_multifd_recv_threads(Error *err) { int i; =20 + if (err) { + MigrationState *s =3D migrate_get_current(); + migrate_set_error(s, err); + if (s->state =3D=3D MIGRATION_STATUS_SETUP || + s->state =3D=3D MIGRATION_STATUS_ACTIVE) { + migrate_set_state(&s->state, s->state, + MIGRATION_STATUS_FAILED); + } + } + for (i =3D 0; i < multifd_recv_state->count; i++) { MultiFDRecvParams *p =3D &multifd_recv_state->params[i]; =20 --=20 2.17.0 From nobody Wed Oct 29 06:44:06 2025 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1524656139818326.48576554910505; Wed, 25 Apr 2018 04:35:39 -0700 (PDT) Received: from localhost ([::1]:35881 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fBIi2-0000Xh-Va for importer@patchew.org; Wed, 25 Apr 2018 07:35:39 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:34001) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fBIaC-0002gB-2T for qemu-devel@nongnu.org; Wed, 25 Apr 2018 07:27:32 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fBIaB-0004lo-4L for qemu-devel@nongnu.org; Wed, 25 Apr 2018 07:27:32 -0400 Received: from mx3-rdu2.redhat.com ([66.187.233.73]:42384 helo=mx1.redhat.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1fBIaB-0004lA-0s for qemu-devel@nongnu.org; Wed, 25 Apr 2018 07:27:31 -0400 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.rdu2.redhat.com [10.11.54.6]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 200D0814F0D2 for ; Wed, 25 Apr 2018 11:27:29 +0000 (UTC) Received: from secure.mitica (ovpn-117-169.ams2.redhat.com [10.36.117.169]) by smtp.corp.redhat.com (Postfix) with ESMTP id 192B92166BC6; Wed, 25 Apr 2018 11:27:27 +0000 (UTC) From: Juan Quintela To: qemu-devel@nongnu.org Date: Wed, 25 Apr 2018 13:27:04 +0200 Message-Id: <20180425112723.1111-3-quintela@redhat.com> In-Reply-To: <20180425112723.1111-1-quintela@redhat.com> References: <20180425112723.1111-1-quintela@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.78 on 10.11.54.6 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.8]); Wed, 25 Apr 2018 11:27:29 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.8]); Wed, 25 Apr 2018 11:27:29 +0000 (UTC) for IP:'10.11.54.6' DOMAIN:'int-mx06.intmail.prod.int.rdu2.redhat.com' HELO:'smtp.corp.redhat.com' FROM:'quintela@redhat.com' RCPT:'' Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 66.187.233.73 Subject: [Qemu-devel] [PATCH v12 02/21] migration: Introduce multifd_recv_new_channel() X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: lvivier@redhat.com, dgilbert@redhat.com, peterx@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Type: text/plain; charset="utf-8" Signed-off-by: Juan Quintela Reviewed-by: Daniel P. Berrang=C3=A9 --- migration/migration.c | 3 ++- migration/ram.c | 6 ++++++ migration/ram.h | 2 ++ 3 files changed, 10 insertions(+), 1 deletion(-) diff --git a/migration/migration.c b/migration/migration.c index 7eca65d1f0..604722cec9 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -466,8 +466,9 @@ void migration_ioc_process_incoming(QIOChannel *ioc) if (!mis->from_src_file) { QEMUFile *f =3D qemu_fopen_channel_input(ioc); migration_fd_process_incoming(f); + return; } - /* We still only have a single channel. Nothing to do here yet */ + multifd_recv_new_channel(ioc); } =20 /** diff --git a/migration/ram.c b/migration/ram.c index 2ae560ea80..c3c330b0e0 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -36,6 +36,7 @@ #include "xbzrle.h" #include "ram.h" #include "migration.h" +#include "socket.h" #include "migration/register.h" #include "migration/misc.h" #include "qemu-file.h" @@ -619,6 +620,11 @@ int multifd_load_setup(void) return 0; } =20 +void multifd_recv_new_channel(QIOChannel *ioc) +{ + /* nothing to do yet */ +} + /** * save_page_header: write page header to wire * diff --git a/migration/ram.h b/migration/ram.h index 5030be110a..06dbddc2a2 100644 --- a/migration/ram.h +++ b/migration/ram.h @@ -32,6 +32,7 @@ #include "qemu-common.h" #include "qapi/qapi-types-migration.h" #include "exec/cpu-common.h" +#include "io/channel.h" =20 extern MigrationStats ram_counters; extern XBZRLECacheStats xbzrle_counters; @@ -44,6 +45,7 @@ int multifd_save_setup(void); int multifd_save_cleanup(Error **errp); int multifd_load_setup(void); int multifd_load_cleanup(Error **errp); +void multifd_recv_new_channel(QIOChannel *ioc); =20 uint64_t ram_pagesize_summary(void); int ram_save_queue_pages(const char *rbname, ram_addr_t start, ram_addr_t = len); --=20 2.17.0 From nobody Wed Oct 29 06:44:06 2025 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1524655968483763.2926146363488; Wed, 25 Apr 2018 04:32:48 -0700 (PDT) Received: from localhost ([::1]:35866 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fBIfH-0006hX-IT for importer@patchew.org; Wed, 25 Apr 2018 07:32:47 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:34026) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fBIaC-0002gi-LQ for qemu-devel@nongnu.org; Wed, 25 Apr 2018 07:27:34 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fBIaB-0004m0-5E for qemu-devel@nongnu.org; Wed, 25 Apr 2018 07:27:32 -0400 Received: from mx3-rdu2.redhat.com ([66.187.233.73]:46184 helo=mx1.redhat.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1fBIaB-0004l7-1J for qemu-devel@nongnu.org; Wed, 25 Apr 2018 07:27:31 -0400 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.rdu2.redhat.com [10.11.54.6]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 6B755406C740 for ; Wed, 25 Apr 2018 11:27:30 +0000 (UTC) Received: from secure.mitica (ovpn-117-169.ams2.redhat.com [10.36.117.169]) by smtp.corp.redhat.com (Postfix) with ESMTP id 6239B215CDCF; Wed, 25 Apr 2018 11:27:29 +0000 (UTC) From: Juan Quintela To: qemu-devel@nongnu.org Date: Wed, 25 Apr 2018 13:27:05 +0200 Message-Id: <20180425112723.1111-4-quintela@redhat.com> In-Reply-To: <20180425112723.1111-1-quintela@redhat.com> References: <20180425112723.1111-1-quintela@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.78 on 10.11.54.6 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.5]); Wed, 25 Apr 2018 11:27:30 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.5]); Wed, 25 Apr 2018 11:27:30 +0000 (UTC) for IP:'10.11.54.6' DOMAIN:'int-mx06.intmail.prod.int.rdu2.redhat.com' HELO:'smtp.corp.redhat.com' FROM:'quintela@redhat.com' RCPT:'' Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 66.187.233.73 Subject: [Qemu-devel] [PATCH v12 03/21] migration: terminate_* can be called for other threads X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: lvivier@redhat.com, dgilbert@redhat.com, peterx@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Type: text/plain; charset="utf-8" Once there, make count field to always be accessed with atomic operations. To make blocking operations, we need to know that the thread is running, so create a bool to indicate that. Signed-off-by: Juan Quintela Reviewed-by: Daniel P. Berrang=C3=A9 -- Once here, s/terminate_multifd_*-threads/multifd_*_terminate_threads/ This is consistente with every other function --- migration/ram.c | 44 ++++++++++++++++++++++++++++++-------------- 1 file changed, 30 insertions(+), 14 deletions(-) diff --git a/migration/ram.c b/migration/ram.c index c3c330b0e0..c0ad6d8e9f 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -406,6 +406,7 @@ struct MultiFDSendParams { QemuThread thread; QemuSemaphore sem; QemuMutex mutex; + bool running; bool quit; }; typedef struct MultiFDSendParams MultiFDSendParams; @@ -416,7 +417,7 @@ struct { int count; } *multifd_send_state; =20 -static void terminate_multifd_send_threads(Error *err) +static void multifd_send_terminate_threads(Error *err) { int i; =20 @@ -430,7 +431,7 @@ static void terminate_multifd_send_threads(Error *err) } } =20 - for (i =3D 0; i < multifd_send_state->count; i++) { + for (i =3D 0; i < migrate_multifd_channels(); i++) { MultiFDSendParams *p =3D &multifd_send_state->params[i]; =20 qemu_mutex_lock(&p->mutex); @@ -448,11 +449,13 @@ int multifd_save_cleanup(Error **errp) if (!migrate_use_multifd()) { return 0; } - terminate_multifd_send_threads(NULL); - for (i =3D 0; i < multifd_send_state->count; i++) { + multifd_send_terminate_threads(NULL); + for (i =3D 0; i < migrate_multifd_channels(); i++) { MultiFDSendParams *p =3D &multifd_send_state->params[i]; =20 - qemu_thread_join(&p->thread); + if (p->running) { + qemu_thread_join(&p->thread); + } qemu_mutex_destroy(&p->mutex); qemu_sem_destroy(&p->sem); g_free(p->name); @@ -479,6 +482,10 @@ static void *multifd_send_thread(void *opaque) qemu_sem_wait(&p->sem); } =20 + qemu_mutex_lock(&p->mutex); + p->running =3D false; + qemu_mutex_unlock(&p->mutex); + return NULL; } =20 @@ -493,7 +500,7 @@ int multifd_save_setup(void) thread_count =3D migrate_multifd_channels(); multifd_send_state =3D g_malloc0(sizeof(*multifd_send_state)); multifd_send_state->params =3D g_new0(MultiFDSendParams, thread_count); - multifd_send_state->count =3D 0; + atomic_set(&multifd_send_state->count, 0); for (i =3D 0; i < thread_count; i++) { MultiFDSendParams *p =3D &multifd_send_state->params[i]; =20 @@ -502,10 +509,11 @@ int multifd_save_setup(void) p->quit =3D false; p->id =3D i; p->name =3D g_strdup_printf("multifdsend_%d", i); + p->running =3D true; qemu_thread_create(&p->thread, p->name, multifd_send_thread, p, QEMU_THREAD_JOINABLE); =20 - multifd_send_state->count++; + atomic_inc(&multifd_send_state->count); } return 0; } @@ -516,6 +524,7 @@ struct MultiFDRecvParams { QemuThread thread; QemuSemaphore sem; QemuMutex mutex; + bool running; bool quit; }; typedef struct MultiFDRecvParams MultiFDRecvParams; @@ -526,7 +535,7 @@ struct { int count; } *multifd_recv_state; =20 -static void terminate_multifd_recv_threads(Error *err) +static void multifd_recv_terminate_threads(Error *err) { int i; =20 @@ -540,7 +549,7 @@ static void terminate_multifd_recv_threads(Error *err) } } =20 - for (i =3D 0; i < multifd_recv_state->count; i++) { + for (i =3D 0; i < migrate_multifd_channels(); i++) { MultiFDRecvParams *p =3D &multifd_recv_state->params[i]; =20 qemu_mutex_lock(&p->mutex); @@ -558,11 +567,13 @@ int multifd_load_cleanup(Error **errp) if (!migrate_use_multifd()) { return 0; } - terminate_multifd_recv_threads(NULL); - for (i =3D 0; i < multifd_recv_state->count; i++) { + multifd_recv_terminate_threads(NULL); + for (i =3D 0; i < migrate_multifd_channels(); i++) { MultiFDRecvParams *p =3D &multifd_recv_state->params[i]; =20 - qemu_thread_join(&p->thread); + if (p->running) { + qemu_thread_join(&p->thread); + } qemu_mutex_destroy(&p->mutex); qemu_sem_destroy(&p->sem); g_free(p->name); @@ -590,6 +601,10 @@ static void *multifd_recv_thread(void *opaque) qemu_sem_wait(&p->sem); } =20 + qemu_mutex_lock(&p->mutex); + p->running =3D false; + qemu_mutex_unlock(&p->mutex); + return NULL; } =20 @@ -604,7 +619,7 @@ int multifd_load_setup(void) thread_count =3D migrate_multifd_channels(); multifd_recv_state =3D g_malloc0(sizeof(*multifd_recv_state)); multifd_recv_state->params =3D g_new0(MultiFDRecvParams, thread_count); - multifd_recv_state->count =3D 0; + atomic_set(&multifd_recv_state->count, 0); for (i =3D 0; i < thread_count; i++) { MultiFDRecvParams *p =3D &multifd_recv_state->params[i]; =20 @@ -613,9 +628,10 @@ int multifd_load_setup(void) p->quit =3D false; p->id =3D i; p->name =3D g_strdup_printf("multifdrecv_%d", i); + p->running =3D true; qemu_thread_create(&p->thread, p->name, multifd_recv_thread, p, QEMU_THREAD_JOINABLE); - multifd_recv_state->count++; + atomic_inc(&multifd_recv_state->count); } return 0; } --=20 2.17.0 From nobody Wed Oct 29 06:44:06 2025 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1524656322154593.1007474932331; Wed, 25 Apr 2018 04:38:42 -0700 (PDT) Received: from localhost ([::1]:35898 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fBIky-00030C-PF for importer@patchew.org; Wed, 25 Apr 2018 07:38:40 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:34035) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fBIaD-0002h4-1Y for qemu-devel@nongnu.org; Wed, 25 Apr 2018 07:27:34 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fBIaC-0004nw-86 for qemu-devel@nongnu.org; Wed, 25 Apr 2018 07:27:33 -0400 Received: from mx3-rdu2.redhat.com ([66.187.233.73]:45560 helo=mx1.redhat.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1fBIaC-0004n9-4V for qemu-devel@nongnu.org; Wed, 25 Apr 2018 07:27:32 -0400 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.rdu2.redhat.com [10.11.54.6]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id B51C0402290A for ; Wed, 25 Apr 2018 11:27:31 +0000 (UTC) Received: from secure.mitica (ovpn-117-169.ams2.redhat.com [10.36.117.169]) by smtp.corp.redhat.com (Postfix) with ESMTP id ACF8A215CDCF; Wed, 25 Apr 2018 11:27:30 +0000 (UTC) From: Juan Quintela To: qemu-devel@nongnu.org Date: Wed, 25 Apr 2018 13:27:06 +0200 Message-Id: <20180425112723.1111-5-quintela@redhat.com> In-Reply-To: <20180425112723.1111-1-quintela@redhat.com> References: <20180425112723.1111-1-quintela@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.78 on 10.11.54.6 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.6]); Wed, 25 Apr 2018 11:27:31 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.6]); Wed, 25 Apr 2018 11:27:31 +0000 (UTC) for IP:'10.11.54.6' DOMAIN:'int-mx06.intmail.prod.int.rdu2.redhat.com' HELO:'smtp.corp.redhat.com' FROM:'quintela@redhat.com' RCPT:'' Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 66.187.233.73 Subject: [Qemu-devel] [PATCH v12 04/21] migration: Be sure all recv channels are created X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: lvivier@redhat.com, dgilbert@redhat.com, peterx@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Type: text/plain; charset="utf-8" We need them before we start migration. Signed-off-by: Juan Quintela Reviewed-by: Daniel P. Berrang=C3=A9 --- migration/migration.c | 6 +++++- migration/ram.c | 11 +++++++++++ migration/ram.h | 1 + 3 files changed, 17 insertions(+), 1 deletion(-) diff --git a/migration/migration.c b/migration/migration.c index 604722cec9..98f85e982c 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -479,7 +479,11 @@ void migration_ioc_process_incoming(QIOChannel *ioc) */ bool migration_has_all_channels(void) { - return true; + bool all_channels; + + all_channels =3D multifd_recv_all_channels_created(); + + return all_channels; } =20 /* diff --git a/migration/ram.c b/migration/ram.c index c0ad6d8e9f..3a01472835 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -636,6 +636,17 @@ int multifd_load_setup(void) return 0; } =20 +bool multifd_recv_all_channels_created(void) +{ + int thread_count =3D migrate_multifd_channels(); + + if (!migrate_use_multifd()) { + return true; + } + + return thread_count =3D=3D atomic_read(&multifd_recv_state->count); +} + void multifd_recv_new_channel(QIOChannel *ioc) { /* nothing to do yet */ diff --git a/migration/ram.h b/migration/ram.h index 06dbddc2a2..3f4b7daee8 100644 --- a/migration/ram.h +++ b/migration/ram.h @@ -45,6 +45,7 @@ int multifd_save_setup(void); int multifd_save_cleanup(Error **errp); int multifd_load_setup(void); int multifd_load_cleanup(Error **errp); +bool multifd_recv_all_channels_created(void); void multifd_recv_new_channel(QIOChannel *ioc); =20 uint64_t ram_pagesize_summary(void); --=20 2.17.0 From nobody Wed Oct 29 06:44:06 2025 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1524655920997887.661468164503; Wed, 25 Apr 2018 04:32:00 -0700 (PDT) Received: from localhost ([::1]:35863 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fBIeR-00065C-Hs for importer@patchew.org; Wed, 25 Apr 2018 07:31:55 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:34076) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fBIaE-0002iO-Cc for qemu-devel@nongnu.org; Wed, 25 Apr 2018 07:27:35 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fBIaD-0004q3-Gs for qemu-devel@nongnu.org; Wed, 25 Apr 2018 07:27:34 -0400 Received: from mx3-rdu2.redhat.com ([66.187.233.73]:45562 helo=mx1.redhat.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1fBIaD-0004pN-Cn for qemu-devel@nongnu.org; Wed, 25 Apr 2018 07:27:33 -0400 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.rdu2.redhat.com [10.11.54.6]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 0B3F94022414 for ; Wed, 25 Apr 2018 11:27:33 +0000 (UTC) Received: from secure.mitica (ovpn-117-169.ams2.redhat.com [10.36.117.169]) by smtp.corp.redhat.com (Postfix) with ESMTP id 040AA215CDCF; Wed, 25 Apr 2018 11:27:31 +0000 (UTC) From: Juan Quintela To: qemu-devel@nongnu.org Date: Wed, 25 Apr 2018 13:27:07 +0200 Message-Id: <20180425112723.1111-6-quintela@redhat.com> In-Reply-To: <20180425112723.1111-1-quintela@redhat.com> References: <20180425112723.1111-1-quintela@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.78 on 10.11.54.6 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.6]); Wed, 25 Apr 2018 11:27:33 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.6]); Wed, 25 Apr 2018 11:27:33 +0000 (UTC) for IP:'10.11.54.6' DOMAIN:'int-mx06.intmail.prod.int.rdu2.redhat.com' HELO:'smtp.corp.redhat.com' FROM:'quintela@redhat.com' RCPT:'' Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 66.187.233.73 Subject: [Qemu-devel] [PATCH v12 05/21] migration: Export functions to create send channels X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: lvivier@redhat.com, dgilbert@redhat.com, peterx@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Type: text/plain; charset="utf-8" Signed-off-by: Juan Quintela Reviewed-by: Daniel P. Berrang=C3=A9 --- migration/socket.c | 28 +++++++++++++++++++++++++++- migration/socket.h | 7 +++++++ 2 files changed, 34 insertions(+), 1 deletion(-) diff --git a/migration/socket.c b/migration/socket.c index edf33c70cf..893a04f4cc 100644 --- a/migration/socket.c +++ b/migration/socket.c @@ -29,6 +29,28 @@ #include "trace.h" =20 =20 +struct SocketOutgoingArgs { + SocketAddress *saddr; +} outgoing_args; + +void socket_send_channel_create(QIOTaskFunc f, void *data) +{ + QIOChannelSocket *sioc =3D qio_channel_socket_new(); + qio_channel_socket_connect_async(sioc, outgoing_args.saddr, + f, data, NULL, NULL); +} + +int socket_send_channel_destroy(QIOChannel *send) +{ + /* Remove channel */ + object_unref(OBJECT(send)); + if (outgoing_args.saddr) { + qapi_free_SocketAddress(outgoing_args.saddr); + outgoing_args.saddr =3D NULL; + } + return 0; +} + static SocketAddress *tcp_build_address(const char *host_port, Error **err= p) { SocketAddress *saddr; @@ -96,6 +118,11 @@ static void socket_start_outgoing_migration(MigrationSt= ate *s, struct SocketConnectData *data =3D g_new0(struct SocketConnectData, 1); =20 data->s =3D s; + + /* in case previous migration leaked it */ + qapi_free_SocketAddress(outgoing_args.saddr); + outgoing_args.saddr =3D saddr; + if (saddr->type =3D=3D SOCKET_ADDRESS_TYPE_INET) { data->hostname =3D g_strdup(saddr->u.inet.host); } @@ -107,7 +134,6 @@ static void socket_start_outgoing_migration(MigrationSt= ate *s, data, socket_connect_data_free, NULL); - qapi_free_SocketAddress(saddr); } =20 void tcp_start_outgoing_migration(MigrationState *s, diff --git a/migration/socket.h b/migration/socket.h index 6b91e9db38..528c3b0202 100644 --- a/migration/socket.h +++ b/migration/socket.h @@ -16,6 +16,13 @@ =20 #ifndef QEMU_MIGRATION_SOCKET_H #define QEMU_MIGRATION_SOCKET_H + +#include "io/channel.h" +#include "io/task.h" + +void socket_send_channel_create(QIOTaskFunc f, void *data); +int socket_send_channel_destroy(QIOChannel *send); + void tcp_start_incoming_migration(const char *host_port, Error **errp); =20 void tcp_start_outgoing_migration(MigrationState *s, const char *host_port, --=20 2.17.0 From nobody Wed Oct 29 06:44:06 2025 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1524656159634877.2490027057904; Wed, 25 Apr 2018 04:35:59 -0700 (PDT) Received: from localhost ([::1]:35887 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fBIiM-0000tH-Ot for importer@patchew.org; Wed, 25 Apr 2018 07:35:58 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:34121) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fBIaF-0002jp-SE for qemu-devel@nongnu.org; Wed, 25 Apr 2018 07:27:37 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fBIaE-0004sx-Rn for qemu-devel@nongnu.org; Wed, 25 Apr 2018 07:27:35 -0400 Received: from mx3-rdu2.redhat.com ([66.187.233.73]:33566 helo=mx1.redhat.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1fBIaE-0004sD-NC for qemu-devel@nongnu.org; Wed, 25 Apr 2018 07:27:34 -0400 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.rdu2.redhat.com [10.11.54.6]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 55E40EC016 for ; Wed, 25 Apr 2018 11:27:34 +0000 (UTC) Received: from secure.mitica (ovpn-117-169.ams2.redhat.com [10.36.117.169]) by smtp.corp.redhat.com (Postfix) with ESMTP id 4D9AC2166BC6; Wed, 25 Apr 2018 11:27:33 +0000 (UTC) From: Juan Quintela To: qemu-devel@nongnu.org Date: Wed, 25 Apr 2018 13:27:08 +0200 Message-Id: <20180425112723.1111-7-quintela@redhat.com> In-Reply-To: <20180425112723.1111-1-quintela@redhat.com> References: <20180425112723.1111-1-quintela@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.78 on 10.11.54.6 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.1]); Wed, 25 Apr 2018 11:27:34 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.1]); Wed, 25 Apr 2018 11:27:34 +0000 (UTC) for IP:'10.11.54.6' DOMAIN:'int-mx06.intmail.prod.int.rdu2.redhat.com' HELO:'smtp.corp.redhat.com' FROM:'quintela@redhat.com' RCPT:'' Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 66.187.233.73 Subject: [Qemu-devel] [PATCH v12 06/21] migration: Create multifd channels X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: lvivier@redhat.com, dgilbert@redhat.com, peterx@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Type: text/plain; charset="utf-8" In both sides. We still don't transmit anything through them. Signed-off-by: Juan Quintela Reviewed-by: Daniel P. Berrang=C3=A9 --- migration/ram.c | 52 +++++++++++++++++++++++++++++++++++++++---------- 1 file changed, 42 insertions(+), 10 deletions(-) diff --git a/migration/ram.c b/migration/ram.c index 3a01472835..54b1f8e836 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -404,6 +404,7 @@ struct MultiFDSendParams { uint8_t id; char *name; QemuThread thread; + QIOChannel *c; QemuSemaphore sem; QemuMutex mutex; bool running; @@ -456,6 +457,8 @@ int multifd_save_cleanup(Error **errp) if (p->running) { qemu_thread_join(&p->thread); } + socket_send_channel_destroy(p->c); + p->c =3D NULL; qemu_mutex_destroy(&p->mutex); qemu_sem_destroy(&p->sem); g_free(p->name); @@ -489,6 +492,27 @@ static void *multifd_send_thread(void *opaque) return NULL; } =20 +static void multifd_new_send_channel_async(QIOTask *task, gpointer opaque) +{ + MultiFDSendParams *p =3D opaque; + QIOChannel *sioc =3D QIO_CHANNEL(qio_task_get_source(task)); + Error *local_err =3D NULL; + + if (qio_task_propagate_error(task, &local_err)) { + if (multifd_save_cleanup(&local_err) !=3D 0) { + migrate_set_error(migrate_get_current(), local_err); + } + } else { + p->c =3D QIO_CHANNEL(sioc); + qio_channel_set_delay(p->c, false); + p->running =3D true; + qemu_thread_create(&p->thread, p->name, multifd_send_thread, p, + QEMU_THREAD_JOINABLE); + + atomic_inc(&multifd_send_state->count); + } +} + int multifd_save_setup(void) { int thread_count; @@ -509,11 +533,7 @@ int multifd_save_setup(void) p->quit =3D false; p->id =3D i; p->name =3D g_strdup_printf("multifdsend_%d", i); - p->running =3D true; - qemu_thread_create(&p->thread, p->name, multifd_send_thread, p, - QEMU_THREAD_JOINABLE); - - atomic_inc(&multifd_send_state->count); + socket_send_channel_create(multifd_new_send_channel_async, p); } return 0; } @@ -522,6 +542,7 @@ struct MultiFDRecvParams { uint8_t id; char *name; QemuThread thread; + QIOChannel *c; QemuSemaphore sem; QemuMutex mutex; bool running; @@ -574,6 +595,8 @@ int multifd_load_cleanup(Error **errp) if (p->running) { qemu_thread_join(&p->thread); } + object_unref(OBJECT(p->c)); + p->c =3D NULL; qemu_mutex_destroy(&p->mutex); qemu_sem_destroy(&p->sem); g_free(p->name); @@ -628,10 +651,6 @@ int multifd_load_setup(void) p->quit =3D false; p->id =3D i; p->name =3D g_strdup_printf("multifdrecv_%d", i); - p->running =3D true; - qemu_thread_create(&p->thread, p->name, multifd_recv_thread, p, - QEMU_THREAD_JOINABLE); - atomic_inc(&multifd_recv_state->count); } return 0; } @@ -649,7 +668,20 @@ bool multifd_recv_all_channels_created(void) =20 void multifd_recv_new_channel(QIOChannel *ioc) { - /* nothing to do yet */ + MultiFDRecvParams *p; + /* we need to invent channels id's until we transmit */ + /* we will remove this on a later patch */ + static int i; + + p =3D &multifd_recv_state->params[i]; + i++; + p->c =3D ioc; + object_ref(OBJECT(ioc)); + + p->running =3D true; + qemu_thread_create(&p->thread, p->name, multifd_recv_thread, p, + QEMU_THREAD_JOINABLE); + atomic_inc(&multifd_recv_state->count); } =20 /** --=20 2.17.0 From nobody Wed Oct 29 06:44:06 2025 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1524656338973760.5603249940378; Wed, 25 Apr 2018 04:38:58 -0700 (PDT) Received: from localhost ([::1]:35899 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fBIlG-0003Fi-5T for importer@patchew.org; Wed, 25 Apr 2018 07:38:58 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:34141) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fBIaH-0002l9-2h for qemu-devel@nongnu.org; Wed, 25 Apr 2018 07:27:38 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fBIaG-0004vH-3z for qemu-devel@nongnu.org; Wed, 25 Apr 2018 07:27:37 -0400 Received: from mx3-rdu2.redhat.com ([66.187.233.73]:33576 helo=mx1.redhat.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1fBIaG-0004uW-0I for qemu-devel@nongnu.org; Wed, 25 Apr 2018 07:27:36 -0400 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.rdu2.redhat.com [10.11.54.6]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 9F74D738E0 for ; Wed, 25 Apr 2018 11:27:35 +0000 (UTC) Received: from secure.mitica (ovpn-117-169.ams2.redhat.com [10.36.117.169]) by smtp.corp.redhat.com (Postfix) with ESMTP id 98088215CDD1; Wed, 25 Apr 2018 11:27:34 +0000 (UTC) From: Juan Quintela To: qemu-devel@nongnu.org Date: Wed, 25 Apr 2018 13:27:09 +0200 Message-Id: <20180425112723.1111-8-quintela@redhat.com> In-Reply-To: <20180425112723.1111-1-quintela@redhat.com> References: <20180425112723.1111-1-quintela@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.78 on 10.11.54.6 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.1]); Wed, 25 Apr 2018 11:27:35 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.1]); Wed, 25 Apr 2018 11:27:35 +0000 (UTC) for IP:'10.11.54.6' DOMAIN:'int-mx06.intmail.prod.int.rdu2.redhat.com' HELO:'smtp.corp.redhat.com' FROM:'quintela@redhat.com' RCPT:'' Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 66.187.233.73 Subject: [Qemu-devel] [PATCH v12 07/21] migration: Delay start of migration main routines X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: lvivier@redhat.com, dgilbert@redhat.com, peterx@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Type: text/plain; charset="utf-8" We need to make sure that we have started all the multifd threads. Signed-off-by: Juan Quintela Reviewed-by: Daniel P. Berrang=C3=A9 --- migration/migration.c | 4 ++-- migration/migration.h | 1 + migration/ram.c | 3 +++ migration/socket.c | 4 ++++ 4 files changed, 10 insertions(+), 2 deletions(-) diff --git a/migration/migration.c b/migration/migration.c index 98f85e982c..9b510a809a 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -447,7 +447,7 @@ static void migration_incoming_setup(QEMUFile *f) qemu_file_set_blocking(f, false); } =20 -static void migration_incoming_process(void) +void migration_incoming_process(void) { Coroutine *co =3D qemu_coroutine_create(process_incoming_migration_co,= NULL); qemu_coroutine_enter(co); @@ -465,7 +465,7 @@ void migration_ioc_process_incoming(QIOChannel *ioc) =20 if (!mis->from_src_file) { QEMUFile *f =3D qemu_fopen_channel_input(ioc); - migration_fd_process_incoming(f); + migration_incoming_setup(f); return; } multifd_recv_new_channel(ioc); diff --git a/migration/migration.h b/migration/migration.h index 4774ee305f..e20f680bac 100644 --- a/migration/migration.h +++ b/migration/migration.h @@ -188,6 +188,7 @@ void migrate_set_state(int *state, int old_state, int n= ew_state); =20 void migration_fd_process_incoming(QEMUFile *f); void migration_ioc_process_incoming(QIOChannel *ioc); +void migration_incoming_process(void); =20 bool migration_has_all_channels(void); =20 diff --git a/migration/ram.c b/migration/ram.c index 54b1f8e836..5a87d74862 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -682,6 +682,9 @@ void multifd_recv_new_channel(QIOChannel *ioc) qemu_thread_create(&p->thread, p->name, multifd_recv_thread, p, QEMU_THREAD_JOINABLE); atomic_inc(&multifd_recv_state->count); + if (multifd_recv_state->count =3D=3D migrate_multifd_channels()) { + migration_incoming_process(); + } } =20 /** diff --git a/migration/socket.c b/migration/socket.c index 893a04f4cc..d4a2c1e916 100644 --- a/migration/socket.c +++ b/migration/socket.c @@ -171,6 +171,10 @@ static void socket_accept_incoming_migration(QIONetLis= tener *listener, qio_net_listener_disconnect(listener); =20 object_unref(OBJECT(listener)); + + if (!migrate_use_multifd()) { + migration_incoming_process(); + } } } =20 --=20 2.17.0 From nobody Wed Oct 29 06:44:06 2025 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1524656492754377.85422410454737; Wed, 25 Apr 2018 04:41:32 -0700 (PDT) Received: from localhost ([::1]:35915 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fBInj-0005Ub-UU for importer@patchew.org; Wed, 25 Apr 2018 07:41:31 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:34164) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fBIaI-0002nB-NH for qemu-devel@nongnu.org; Wed, 25 Apr 2018 07:27:42 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fBIaH-0004xW-Fy for qemu-devel@nongnu.org; Wed, 25 Apr 2018 07:27:38 -0400 Received: from mx3-rdu2.redhat.com ([66.187.233.73]:33580 helo=mx1.redhat.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1fBIaH-0004wc-At for qemu-devel@nongnu.org; Wed, 25 Apr 2018 07:27:37 -0400 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.rdu2.redhat.com [10.11.54.6]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id E8983EC013 for ; Wed, 25 Apr 2018 11:27:36 +0000 (UTC) Received: from secure.mitica (ovpn-117-169.ams2.redhat.com [10.36.117.169]) by smtp.corp.redhat.com (Postfix) with ESMTP id E2847215CDCE; Wed, 25 Apr 2018 11:27:35 +0000 (UTC) From: Juan Quintela To: qemu-devel@nongnu.org Date: Wed, 25 Apr 2018 13:27:10 +0200 Message-Id: <20180425112723.1111-9-quintela@redhat.com> In-Reply-To: <20180425112723.1111-1-quintela@redhat.com> References: <20180425112723.1111-1-quintela@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.78 on 10.11.54.6 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.1]); Wed, 25 Apr 2018 11:27:36 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.1]); Wed, 25 Apr 2018 11:27:36 +0000 (UTC) for IP:'10.11.54.6' DOMAIN:'int-mx06.intmail.prod.int.rdu2.redhat.com' HELO:'smtp.corp.redhat.com' FROM:'quintela@redhat.com' RCPT:'' Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 66.187.233.73 Subject: [Qemu-devel] [PATCH v12 08/21] migration: Transmit initial package through the multifd channels X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: lvivier@redhat.com, dgilbert@redhat.com, peterx@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Type: text/plain; charset="utf-8" Signed-off-by: Juan Quintela Reviewed-by: Daniel P. Berrang=C3=A9 -- Be network agnostic. Add error checking for all values. --- migration/ram.c | 101 +++++++++++++++++++++++++++++++++++++++++++++--- 1 file changed, 96 insertions(+), 5 deletions(-) diff --git a/migration/ram.c b/migration/ram.c index 5a87d74862..1aab392d8f 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -52,6 +52,8 @@ #include "qemu/rcu_queue.h" #include "migration/colo.h" #include "migration/block.h" +#include "sysemu/sysemu.h" +#include "qemu/uuid.h" =20 /***********************************************************/ /* ram save/restore */ @@ -400,6 +402,16 @@ static void compress_threads_save_setup(void) =20 /* Multiple fd's */ =20 +#define MULTIFD_MAGIC 0x11223344U +#define MULTIFD_VERSION 1 + +typedef struct { + uint32_t magic; + uint32_t version; + unsigned char uuid[16]; /* QemuUUID */ + uint8_t id; +} __attribute__((packed)) MultiFDInit_t; + struct MultiFDSendParams { uint8_t id; char *name; @@ -412,6 +424,65 @@ struct MultiFDSendParams { }; typedef struct MultiFDSendParams MultiFDSendParams; =20 +static int multifd_send_initial_packet(MultiFDSendParams *p, Error **errp) +{ + MultiFDInit_t msg; + int ret; + + msg.magic =3D cpu_to_be32(MULTIFD_MAGIC); + msg.version =3D cpu_to_be32(MULTIFD_VERSION); + msg.id =3D p->id; + memcpy(msg.uuid, &qemu_uuid.data, sizeof(msg.uuid)); + + ret =3D qio_channel_write_all(p->c, (char *)&msg, sizeof(msg), errp); + if (ret !=3D 0) { + return -1; + } + return 0; +} + +static int multifd_recv_initial_packet(QIOChannel *c, Error **errp) +{ + MultiFDInit_t msg; + int ret; + + ret =3D qio_channel_read_all(c, (char *)&msg, sizeof(msg), errp); + if (ret !=3D 0) { + return -1; + } + + be32_to_cpus(&msg.magic); + be32_to_cpus(&msg.version); + + if (msg.magic !=3D MULTIFD_MAGIC) { + error_setg(errp, "multifd: received packet magic %d " + "expected %d", msg.magic, MULTIFD_MAGIC); + return -1; + } + + if (msg.version !=3D MULTIFD_VERSION) { + error_setg(errp, "multifd: received packet version %d " + "expected %d", msg.version, MULTIFD_VERSION); + return -1; + } + + if (memcmp(msg.uuid, &qemu_uuid, sizeof(qemu_uuid))) { + char *uuid =3D qemu_uuid_unparse_strdup(&qemu_uuid); + error_setg(errp, "multifd: received uuid '%s' and expected " + "uuid '%s' for channel %hhd", msg.uuid, uuid, msg.id); + g_free(uuid); + return -1; + } + + if (msg.id > migrate_multifd_channels()) { + error_setg(errp, "multifd: received channel version %d " + "expected %d", msg.version, MULTIFD_VERSION); + return -1; + } + + return msg.id; +} + struct { MultiFDSendParams *params; /* number of created threads */ @@ -474,6 +545,11 @@ int multifd_save_cleanup(Error **errp) static void *multifd_send_thread(void *opaque) { MultiFDSendParams *p =3D opaque; + Error *local_err =3D NULL; + + if (multifd_send_initial_packet(p, &local_err) < 0) { + goto out; + } =20 while (true) { qemu_mutex_lock(&p->mutex); @@ -485,6 +561,11 @@ static void *multifd_send_thread(void *opaque) qemu_sem_wait(&p->sem); } =20 +out: + if (local_err) { + multifd_send_terminate_threads(local_err); + } + qemu_mutex_lock(&p->mutex); p->running =3D false; qemu_mutex_unlock(&p->mutex); @@ -669,12 +750,22 @@ bool multifd_recv_all_channels_created(void) void multifd_recv_new_channel(QIOChannel *ioc) { MultiFDRecvParams *p; - /* we need to invent channels id's until we transmit */ - /* we will remove this on a later patch */ - static int i; + Error *local_err =3D NULL; + int id; =20 - p =3D &multifd_recv_state->params[i]; - i++; + id =3D multifd_recv_initial_packet(ioc, &local_err); + if (id < 0) { + multifd_recv_terminate_threads(local_err); + return; + } + + p =3D &multifd_recv_state->params[id]; + if (p->c !=3D NULL) { + error_setg(&local_err, "multifd: received id '%d' already setup'", + id); + multifd_recv_terminate_threads(local_err); + return; + } p->c =3D ioc; object_ref(OBJECT(ioc)); =20 --=20 2.17.0 From nobody Wed Oct 29 06:44:06 2025 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1524656103820221.22293258132618; Wed, 25 Apr 2018 04:35:03 -0700 (PDT) Received: from localhost ([::1]:35880 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fBIhS-00005J-Ma for importer@patchew.org; Wed, 25 Apr 2018 07:35:02 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:34245) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fBIaM-0002rg-RU for qemu-devel@nongnu.org; Wed, 25 Apr 2018 07:27:43 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fBIaI-0004zJ-OS for qemu-devel@nongnu.org; Wed, 25 Apr 2018 07:27:42 -0400 Received: from mx3-rdu2.redhat.com ([66.187.233.73]:33582 helo=mx1.redhat.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1fBIaI-0004yg-JP for qemu-devel@nongnu.org; Wed, 25 Apr 2018 07:27:38 -0400 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.rdu2.redhat.com [10.11.54.6]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 3DAE1EC013 for ; Wed, 25 Apr 2018 11:27:38 +0000 (UTC) Received: from secure.mitica (ovpn-117-169.ams2.redhat.com [10.36.117.169]) by smtp.corp.redhat.com (Postfix) with ESMTP id 3775A215CDCE; Wed, 25 Apr 2018 11:27:37 +0000 (UTC) From: Juan Quintela To: qemu-devel@nongnu.org Date: Wed, 25 Apr 2018 13:27:11 +0200 Message-Id: <20180425112723.1111-10-quintela@redhat.com> In-Reply-To: <20180425112723.1111-1-quintela@redhat.com> References: <20180425112723.1111-1-quintela@redhat.com> X-Scanned-By: MIMEDefang 2.78 on 10.11.54.6 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.1]); Wed, 25 Apr 2018 11:27:38 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.1]); Wed, 25 Apr 2018 11:27:38 +0000 (UTC) for IP:'10.11.54.6' DOMAIN:'int-mx06.intmail.prod.int.rdu2.redhat.com' HELO:'smtp.corp.redhat.com' FROM:'quintela@redhat.com' RCPT:'' X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 66.187.233.73 Subject: [Qemu-devel] [PATCH v12 09/21] migration: Define MultifdRecvParams sooner X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: lvivier@redhat.com, dgilbert@redhat.com, peterx@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Once there, we don't need the struct names anywhere, just the typedefs. And now also document all fields. Signed-off-by: Juan Quintela Reviewed-by: Dr. David Alan Gilbert --- migration/ram.c | 46 +++++++++++++++++++++++++++++++--------------- 1 file changed, 31 insertions(+), 15 deletions(-) diff --git a/migration/ram.c b/migration/ram.c index 1aab392d8f..ffefa73099 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -412,17 +412,45 @@ typedef struct { uint8_t id; } __attribute__((packed)) MultiFDInit_t; =20 -struct MultiFDSendParams { +typedef struct { + /* this fields are not changed once the thread is created */ + /* channel number */ uint8_t id; + /* channel thread name */ char *name; + /* channel thread id */ QemuThread thread; + /* communication channel */ QIOChannel *c; + /* sem where to wait for more work */ QemuSemaphore sem; + /* this mutex protects the following parameters */ QemuMutex mutex; + /* is this channel thread running */ bool running; + /* should this thread finish */ bool quit; -}; -typedef struct MultiFDSendParams MultiFDSendParams; +} MultiFDSendParams; + +typedef struct { + /* this fields are not changed once the thread is created */ + /* channel number */ + uint8_t id; + /* channel thread name */ + char *name; + /* channel thread id */ + QemuThread thread; + /* communication channel */ + QIOChannel *c; + /* sem where to wait for more work */ + QemuSemaphore sem; + /* this mutex protects the following parameters */ + QemuMutex mutex; + /* is this channel thread running */ + bool running; + /* should this thread finish */ + bool quit; +} MultiFDRecvParams; =20 static int multifd_send_initial_packet(MultiFDSendParams *p, Error **errp) { @@ -619,18 +647,6 @@ int multifd_save_setup(void) return 0; } =20 -struct MultiFDRecvParams { - uint8_t id; - char *name; - QemuThread thread; - QIOChannel *c; - QemuSemaphore sem; - QemuMutex mutex; - bool running; - bool quit; -}; -typedef struct MultiFDRecvParams MultiFDRecvParams; - struct { MultiFDRecvParams *params; /* number of created threads */ --=20 2.17.0 From nobody Wed Oct 29 06:44:06 2025 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1524656177128815.4297588087542; Wed, 25 Apr 2018 04:36:17 -0700 (PDT) Received: from localhost ([::1]:35888 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fBIiZ-00011D-LR for importer@patchew.org; Wed, 25 Apr 2018 07:36:11 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:34242) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fBIaM-0002rX-Q9 for qemu-devel@nongnu.org; Wed, 25 Apr 2018 07:27:43 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fBIaK-00051h-2q for qemu-devel@nongnu.org; Wed, 25 Apr 2018 07:27:42 -0400 Received: from mx3-rdu2.redhat.com ([66.187.233.73]:42390 helo=mx1.redhat.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1fBIaJ-000512-Uk for qemu-devel@nongnu.org; Wed, 25 Apr 2018 07:27:40 -0400 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.rdu2.redhat.com [10.11.54.6]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 8B9C081A88D2 for ; Wed, 25 Apr 2018 11:27:39 +0000 (UTC) Received: from secure.mitica (ovpn-117-169.ams2.redhat.com [10.36.117.169]) by smtp.corp.redhat.com (Postfix) with ESMTP id 7FEA02166BC6; Wed, 25 Apr 2018 11:27:38 +0000 (UTC) From: Juan Quintela To: qemu-devel@nongnu.org Date: Wed, 25 Apr 2018 13:27:12 +0200 Message-Id: <20180425112723.1111-11-quintela@redhat.com> In-Reply-To: <20180425112723.1111-1-quintela@redhat.com> References: <20180425112723.1111-1-quintela@redhat.com> X-Scanned-By: MIMEDefang 2.78 on 10.11.54.6 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.8]); Wed, 25 Apr 2018 11:27:39 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.8]); Wed, 25 Apr 2018 11:27:39 +0000 (UTC) for IP:'10.11.54.6' DOMAIN:'int-mx06.intmail.prod.int.rdu2.redhat.com' HELO:'smtp.corp.redhat.com' FROM:'quintela@redhat.com' RCPT:'' X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 66.187.233.73 Subject: [Qemu-devel] [PATCH v12 10/21] migration: Create multipage support X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: lvivier@redhat.com, dgilbert@redhat.com, peterx@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" We only create/destry the page list here. We will use it later. Signed-off-by: Juan Quintela Reviewed-by: Peter Xu --- migration/ram.c | 56 +++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 56 insertions(+) diff --git a/migration/ram.c b/migration/ram.c index ffefa73099..b19300992e 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -412,6 +412,20 @@ typedef struct { uint8_t id; } __attribute__((packed)) MultiFDInit_t; =20 +typedef struct { + /* number of used pages */ + uint32_t used; + /* number of allocated pages */ + uint32_t allocated; + /* global number of generated multifd packets */ + uint32_t seq; + /* offset of each page */ + ram_addr_t *offset; + /* pointer to each page */ + struct iovec *iov; + RAMBlock *block; +} MultiFDPages_t; + typedef struct { /* this fields are not changed once the thread is created */ /* channel number */ @@ -430,6 +444,8 @@ typedef struct { bool running; /* should this thread finish */ bool quit; + /* array of pages to sent */ + MultiFDPages_t *pages; } MultiFDSendParams; =20 typedef struct { @@ -450,6 +466,8 @@ typedef struct { bool running; /* should this thread finish */ bool quit; + /* array of pages to receive */ + MultiFDPages_t *pages; } MultiFDRecvParams; =20 static int multifd_send_initial_packet(MultiFDSendParams *p, Error **errp) @@ -511,10 +529,35 @@ static int multifd_recv_initial_packet(QIOChannel *c,= Error **errp) return msg.id; } =20 +static void multifd_pages_init(MultiFDPages_t **ppages, size_t size) +{ + MultiFDPages_t *pages =3D g_new0(MultiFDPages_t, 1); + + pages->allocated =3D size; + pages->iov =3D g_new0(struct iovec, size); + pages->offset =3D g_new0(ram_addr_t, size); + *ppages =3D pages; +} + +static void multifd_pages_clear(MultiFDPages_t *pages) +{ + pages->used =3D 0; + pages->allocated =3D 0; + pages->seq =3D 0; + pages->block =3D NULL; + g_free(pages->iov); + pages->iov =3D NULL; + g_free(pages->offset); + pages->offset =3D NULL; + g_free(pages); +} + struct { MultiFDSendParams *params; /* number of created threads */ int count; + /* array of pages to sent */ + MultiFDPages_t *pages; } *multifd_send_state; =20 static void multifd_send_terminate_threads(Error *err) @@ -562,9 +605,13 @@ int multifd_save_cleanup(Error **errp) qemu_sem_destroy(&p->sem); g_free(p->name); p->name =3D NULL; + multifd_pages_clear(p->pages); + p->pages =3D NULL; } g_free(multifd_send_state->params); multifd_send_state->params =3D NULL; + multifd_pages_clear(multifd_send_state->pages); + multifd_send_state->pages =3D NULL; g_free(multifd_send_state); multifd_send_state =3D NULL; return ret; @@ -625,6 +672,7 @@ static void multifd_new_send_channel_async(QIOTask *tas= k, gpointer opaque) int multifd_save_setup(void) { int thread_count; + uint32_t page_count =3D migrate_multifd_page_count(); uint8_t i; =20 if (!migrate_use_multifd()) { @@ -634,6 +682,8 @@ int multifd_save_setup(void) multifd_send_state =3D g_malloc0(sizeof(*multifd_send_state)); multifd_send_state->params =3D g_new0(MultiFDSendParams, thread_count); atomic_set(&multifd_send_state->count, 0); + multifd_pages_init(&multifd_send_state->pages, page_count); + for (i =3D 0; i < thread_count; i++) { MultiFDSendParams *p =3D &multifd_send_state->params[i]; =20 @@ -641,6 +691,7 @@ int multifd_save_setup(void) qemu_sem_init(&p->sem, 0); p->quit =3D false; p->id =3D i; + multifd_pages_init(&p->pages, page_count); p->name =3D g_strdup_printf("multifdsend_%d", i); socket_send_channel_create(multifd_new_send_channel_async, p); } @@ -698,6 +749,8 @@ int multifd_load_cleanup(Error **errp) qemu_sem_destroy(&p->sem); g_free(p->name); p->name =3D NULL; + multifd_pages_clear(p->pages); + p->pages =3D NULL; } g_free(multifd_recv_state->params); multifd_recv_state->params =3D NULL; @@ -731,6 +784,7 @@ static void *multifd_recv_thread(void *opaque) int multifd_load_setup(void) { int thread_count; + uint32_t page_count =3D migrate_multifd_page_count(); uint8_t i; =20 if (!migrate_use_multifd()) { @@ -740,6 +794,7 @@ int multifd_load_setup(void) multifd_recv_state =3D g_malloc0(sizeof(*multifd_recv_state)); multifd_recv_state->params =3D g_new0(MultiFDRecvParams, thread_count); atomic_set(&multifd_recv_state->count, 0); + for (i =3D 0; i < thread_count; i++) { MultiFDRecvParams *p =3D &multifd_recv_state->params[i]; =20 @@ -747,6 +802,7 @@ int multifd_load_setup(void) qemu_sem_init(&p->sem, 0); p->quit =3D false; p->id =3D i; + multifd_pages_init(&p->pages, page_count); p->name =3D g_strdup_printf("multifdrecv_%d", i); } return 0; --=20 2.17.0 From nobody Wed Oct 29 06:44:06 2025 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1524656350811294.3047338377164; Wed, 25 Apr 2018 04:39:10 -0700 (PDT) Received: from localhost ([::1]:35900 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fBIlR-0003PN-Si for importer@patchew.org; Wed, 25 Apr 2018 07:39:09 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:34243) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fBIaM-0002ra-Qw for qemu-devel@nongnu.org; Wed, 25 Apr 2018 07:27:44 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fBIaL-00053y-Cb for qemu-devel@nongnu.org; Wed, 25 Apr 2018 07:27:42 -0400 Received: from mx3-rdu2.redhat.com ([66.187.233.73]:46190 helo=mx1.redhat.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1fBIaL-000534-7P for qemu-devel@nongnu.org; Wed, 25 Apr 2018 07:27:41 -0400 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.rdu2.redhat.com [10.11.54.6]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id D6D8E406C740 for ; Wed, 25 Apr 2018 11:27:40 +0000 (UTC) Received: from secure.mitica (ovpn-117-169.ams2.redhat.com [10.36.117.169]) by smtp.corp.redhat.com (Postfix) with ESMTP id CD6FE215CDD1; Wed, 25 Apr 2018 11:27:39 +0000 (UTC) From: Juan Quintela To: qemu-devel@nongnu.org Date: Wed, 25 Apr 2018 13:27:13 +0200 Message-Id: <20180425112723.1111-12-quintela@redhat.com> In-Reply-To: <20180425112723.1111-1-quintela@redhat.com> References: <20180425112723.1111-1-quintela@redhat.com> X-Scanned-By: MIMEDefang 2.78 on 10.11.54.6 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.5]); Wed, 25 Apr 2018 11:27:40 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.5]); Wed, 25 Apr 2018 11:27:40 +0000 (UTC) for IP:'10.11.54.6' DOMAIN:'int-mx06.intmail.prod.int.rdu2.redhat.com' HELO:'smtp.corp.redhat.com' FROM:'quintela@redhat.com' RCPT:'' X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 66.187.233.73 Subject: [Qemu-devel] [PATCH v12 11/21] migration: Create multifd packet X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: lvivier@redhat.com, dgilbert@redhat.com, peterx@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" We still don't put anything there. Signed-off-by: Juan Quintela --- migration/ram.c | 137 +++++++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 136 insertions(+), 1 deletion(-) diff --git a/migration/ram.c b/migration/ram.c index b19300992e..804c83ed89 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -412,6 +412,17 @@ typedef struct { uint8_t id; } __attribute__((packed)) MultiFDInit_t; =20 +typedef struct { + uint32_t magic; + uint32_t version; + uint32_t flags; + uint32_t size; + uint32_t used; + uint32_t seq; + char ramblock[256]; + uint64_t offset[]; +} __attribute__((packed)) MultiFDPacket_t; + typedef struct { /* number of used pages */ uint32_t used; @@ -446,6 +457,14 @@ typedef struct { bool quit; /* array of pages to sent */ MultiFDPages_t *pages; + /* packet allocated len */ + uint32_t packet_len; + /* pointer to the packet */ + MultiFDPacket_t *packet; + /* multifd flags for each packet */ + uint32_t flags; + /* global number of generated multifd packets */ + uint32_t seq; } MultiFDSendParams; =20 typedef struct { @@ -468,6 +487,14 @@ typedef struct { bool quit; /* array of pages to receive */ MultiFDPages_t *pages; + /* packet allocated len */ + uint32_t packet_len; + /* pointer to the packet */ + MultiFDPacket_t *packet; + /* multifd flags for each packet */ + uint32_t flags; + /* global number of generated multifd packets */ + uint32_t seq; } MultiFDRecvParams; =20 static int multifd_send_initial_packet(MultiFDSendParams *p, Error **errp) @@ -552,6 +579,91 @@ static void multifd_pages_clear(MultiFDPages_t *pages) g_free(pages); } =20 +static void multifd_send_fill_packet(MultiFDSendParams *p) +{ + MultiFDPacket_t *packet =3D p->packet; + int i; + + packet->magic =3D cpu_to_be32(MULTIFD_MAGIC); + packet->version =3D cpu_to_be32(MULTIFD_VERSION); + packet->flags =3D cpu_to_be32(p->flags); + packet->size =3D cpu_to_be32(migrate_multifd_page_count()); + packet->used =3D cpu_to_be32(p->pages->used); + packet->seq =3D cpu_to_be32(p->seq); + + if (p->pages->block) { + strncpy(packet->ramblock, p->pages->block->idstr, 256); + } + + for (i =3D 0; i < p->pages->used; i++) { + packet->offset[i] =3D cpu_to_be64(p->pages->offset[i]); + } +} + +static int multifd_recv_unfill_packet(MultiFDRecvParams *p, Error **errp) +{ + MultiFDPacket_t *packet =3D p->packet; + RAMBlock *block; + int i; + + /* ToDo: We can't use it until we haven't received a message */ + return 0; + + be32_to_cpus(&packet->magic); + if (packet->magic !=3D MULTIFD_MAGIC) { + error_setg(errp, "multifd: received packet " + "version %d and expected version %d", + packet->magic, MULTIFD_VERSION); + return -1; + } + + be32_to_cpus(&packet->version); + if (packet->version !=3D MULTIFD_VERSION) { + error_setg(errp, "multifd: received packet " + "version %d and expected version %d", + packet->version, MULTIFD_VERSION); + return -1; + } + + p->flags =3D be32_to_cpu(packet->flags); + + be32_to_cpus(&packet->size); + if (packet->size > migrate_multifd_page_count()) { + error_setg(errp, "multifd: received packet " + "with size %d and expected maximum size %d", + packet->size, migrate_multifd_page_count()) ; + return -1; + } + + p->pages->used =3D be32_to_cpu(packet->used); + if (p->pages->used > packet->size) { + error_setg(errp, "multifd: received packet " + "with size %d and expected maximum size %d", + p->pages->used, packet->size) ; + return -1; + } + + p->seq =3D be32_to_cpu(packet->seq); + + if (p->pages->used) { + block =3D qemu_ram_block_by_name(packet->ramblock); + if (!block) { + error_setg(errp, "multifd: unknown ram block %s", + packet->ramblock); + return -1; + } + } + + for (i =3D 0; i < p->pages->used; i++) { + ram_addr_t offset =3D be64_to_cpu(packet->offset[i]); + + p->pages->iov[i].iov_base =3D block->host + offset; + p->pages->iov[i].iov_len =3D TARGET_PAGE_SIZE; + } + + return 0; +} + struct { MultiFDSendParams *params; /* number of created threads */ @@ -607,6 +719,9 @@ int multifd_save_cleanup(Error **errp) p->name =3D NULL; multifd_pages_clear(p->pages); p->pages =3D NULL; + p->packet_len =3D 0; + g_free(p->packet); + p->packet =3D NULL; } g_free(multifd_send_state->params); multifd_send_state->params =3D NULL; @@ -628,6 +743,7 @@ static void *multifd_send_thread(void *opaque) =20 while (true) { qemu_mutex_lock(&p->mutex); + multifd_send_fill_packet(p); if (p->quit) { qemu_mutex_unlock(&p->mutex); break; @@ -692,6 +808,9 @@ int multifd_save_setup(void) p->quit =3D false; p->id =3D i; multifd_pages_init(&p->pages, page_count); + p->packet_len =3D sizeof(MultiFDPacket_t) + + sizeof(ram_addr_t) * page_count; + p->packet =3D g_malloc0(p->packet_len); p->name =3D g_strdup_printf("multifdsend_%d", i); socket_send_channel_create(multifd_new_send_channel_async, p); } @@ -751,6 +870,9 @@ int multifd_load_cleanup(Error **errp) p->name =3D NULL; multifd_pages_clear(p->pages); p->pages =3D NULL; + p->packet_len =3D 0; + g_free(p->packet); + p->packet =3D NULL; } g_free(multifd_recv_state->params); multifd_recv_state->params =3D NULL; @@ -763,10 +885,20 @@ int multifd_load_cleanup(Error **errp) static void *multifd_recv_thread(void *opaque) { MultiFDRecvParams *p =3D opaque; + Error *local_err =3D NULL; + int ret; =20 while (true) { qemu_mutex_lock(&p->mutex); - if (p->quit) { + if (false) { + /* ToDo: Packet reception goes here */ + + ret =3D multifd_recv_unfill_packet(p, &local_err); + qemu_mutex_unlock(&p->mutex); + if (ret) { + break; + } + } else if (p->quit) { qemu_mutex_unlock(&p->mutex); break; } @@ -803,6 +935,9 @@ int multifd_load_setup(void) p->quit =3D false; p->id =3D i; multifd_pages_init(&p->pages, page_count); + p->packet_len =3D sizeof(MultiFDPacket_t) + + sizeof(ram_addr_t) * page_count; + p->packet =3D g_malloc0(p->packet_len); p->name =3D g_strdup_printf("multifdrecv_%d", i); } return 0; --=20 2.17.0 From nobody Wed Oct 29 06:44:06 2025 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1524656519702898.1041824104196; Wed, 25 Apr 2018 04:41:59 -0700 (PDT) Received: from localhost ([::1]:35917 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fBIoA-0005tI-RB for importer@patchew.org; Wed, 25 Apr 2018 07:41:58 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:34278) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fBIaN-0002sb-Qn for qemu-devel@nongnu.org; Wed, 25 Apr 2018 07:27:44 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fBIaM-00056J-Ni for qemu-devel@nongnu.org; Wed, 25 Apr 2018 07:27:43 -0400 Received: from mx3-rdu2.redhat.com ([66.187.233.73]:46192 helo=mx1.redhat.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1fBIaM-00055Y-I7 for qemu-devel@nongnu.org; Wed, 25 Apr 2018 07:27:42 -0400 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.rdu2.redhat.com [10.11.54.6]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 2E027406C740 for ; Wed, 25 Apr 2018 11:27:42 +0000 (UTC) Received: from secure.mitica (ovpn-117-169.ams2.redhat.com [10.36.117.169]) by smtp.corp.redhat.com (Postfix) with ESMTP id 24F26215CDD1; Wed, 25 Apr 2018 11:27:41 +0000 (UTC) From: Juan Quintela To: qemu-devel@nongnu.org Date: Wed, 25 Apr 2018 13:27:14 +0200 Message-Id: <20180425112723.1111-13-quintela@redhat.com> In-Reply-To: <20180425112723.1111-1-quintela@redhat.com> References: <20180425112723.1111-1-quintela@redhat.com> X-Scanned-By: MIMEDefang 2.78 on 10.11.54.6 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.5]); Wed, 25 Apr 2018 11:27:42 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.5]); Wed, 25 Apr 2018 11:27:42 +0000 (UTC) for IP:'10.11.54.6' DOMAIN:'int-mx06.intmail.prod.int.rdu2.redhat.com' HELO:'smtp.corp.redhat.com' FROM:'quintela@redhat.com' RCPT:'' X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 66.187.233.73 Subject: [Qemu-devel] [PATCH v12 12/21] migration: Add multifd traces for start/end thread X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: lvivier@redhat.com, dgilbert@redhat.com, peterx@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" We want to know how many pages/packets each channel has sent. Add counters for those. Signed-off-by: Juan Quintela --- migration/ram.c | 20 ++++++++++++++++++++ migration/trace-events | 4 ++++ 2 files changed, 24 insertions(+) diff --git a/migration/ram.c b/migration/ram.c index 804c83ed89..0f1340b4e3 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -465,6 +465,10 @@ typedef struct { uint32_t flags; /* global number of generated multifd packets */ uint32_t seq; + /* packets sent through this channel */ + uint32_t num_packets; + /* pages sent through this channel */ + uint32_t num_pages; } MultiFDSendParams; =20 typedef struct { @@ -495,6 +499,10 @@ typedef struct { uint32_t flags; /* global number of generated multifd packets */ uint32_t seq; + /* packets sent through this channel */ + uint32_t num_packets; + /* pages sent through this channel */ + uint32_t num_pages; } MultiFDRecvParams; =20 static int multifd_send_initial_packet(MultiFDSendParams *p, Error **errp) @@ -737,9 +745,13 @@ static void *multifd_send_thread(void *opaque) MultiFDSendParams *p =3D opaque; Error *local_err =3D NULL; =20 + trace_multifd_send_thread_start(p->id); + if (multifd_send_initial_packet(p, &local_err) < 0) { goto out; } + /* initial packet */ + p->num_packets =3D 1; =20 while (true) { qemu_mutex_lock(&p->mutex); @@ -761,6 +773,8 @@ out: p->running =3D false; qemu_mutex_unlock(&p->mutex); =20 + trace_multifd_send_thread_end(p->id, p->num_packets, p->num_pages); + return NULL; } =20 @@ -888,6 +902,8 @@ static void *multifd_recv_thread(void *opaque) Error *local_err =3D NULL; int ret; =20 + trace_multifd_recv_thread_start(p->id); + while (true) { qemu_mutex_lock(&p->mutex); if (false) { @@ -910,6 +926,8 @@ static void *multifd_recv_thread(void *opaque) p->running =3D false; qemu_mutex_unlock(&p->mutex); =20 + trace_multifd_recv_thread_end(p->id, p->num_packets, p->num_pages); + return NULL; } =20 @@ -975,6 +993,8 @@ void multifd_recv_new_channel(QIOChannel *ioc) } p->c =3D ioc; object_ref(OBJECT(ioc)); + /* initial packet */ + p->num_packets =3D 1; =20 p->running =3D true; qemu_thread_create(&p->thread, p->name, multifd_recv_thread, p, diff --git a/migration/trace-events b/migration/trace-events index a180d7b008..e480eb050e 100644 --- a/migration/trace-events +++ b/migration/trace-events @@ -77,6 +77,10 @@ ram_load_postcopy_loop(uint64_t addr, int flags) "@%" PR= Ix64 " %x" ram_postcopy_send_discard_bitmap(void) "" ram_save_page(const char *rbname, uint64_t offset, void *host) "%s: offset= : 0x%" PRIx64 " host: %p" ram_save_queue_pages(const char *rbname, size_t start, size_t len) "%s: st= art: 0x%zx len: 0x%zx" +multifd_send_thread_start(uint8_t id) "%d" +multifd_send_thread_end(uint8_t id, uint32_t packets, uint32_t pages) "cha= nnel %d packets %d pages %d" +multifd_recv_thread_start(uint8_t id) "%d" +multifd_recv_thread_end(uint8_t id, uint32_t packets, uint32_t pages) "cha= nnel %d packets %d pages %d" =20 # migration/migration.c await_return_path_close_on_source_close(void) "" --=20 2.17.0 From nobody Wed Oct 29 06:44:06 2025 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1524656660637766.9535018857515; Wed, 25 Apr 2018 04:44:20 -0700 (PDT) Received: from localhost ([::1]:35928 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fBIqN-0007sU-BZ for importer@patchew.org; Wed, 25 Apr 2018 07:44:15 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:34294) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fBIaO-0002tg-Td for qemu-devel@nongnu.org; Wed, 25 Apr 2018 07:27:45 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fBIaO-00058i-0P for qemu-devel@nongnu.org; Wed, 25 Apr 2018 07:27:44 -0400 Received: from mx3-rdu2.redhat.com ([66.187.233.73]:45566 helo=mx1.redhat.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1fBIaN-000585-SY for qemu-devel@nongnu.org; Wed, 25 Apr 2018 07:27:43 -0400 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.rdu2.redhat.com [10.11.54.6]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 781D84023155 for ; Wed, 25 Apr 2018 11:27:43 +0000 (UTC) Received: from secure.mitica (ovpn-117-169.ams2.redhat.com [10.36.117.169]) by smtp.corp.redhat.com (Postfix) with ESMTP id 7068A215CDD1; Wed, 25 Apr 2018 11:27:42 +0000 (UTC) From: Juan Quintela To: qemu-devel@nongnu.org Date: Wed, 25 Apr 2018 13:27:15 +0200 Message-Id: <20180425112723.1111-14-quintela@redhat.com> In-Reply-To: <20180425112723.1111-1-quintela@redhat.com> References: <20180425112723.1111-1-quintela@redhat.com> X-Scanned-By: MIMEDefang 2.78 on 10.11.54.6 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.6]); Wed, 25 Apr 2018 11:27:43 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.6]); Wed, 25 Apr 2018 11:27:43 +0000 (UTC) for IP:'10.11.54.6' DOMAIN:'int-mx06.intmail.prod.int.rdu2.redhat.com' HELO:'smtp.corp.redhat.com' FROM:'quintela@redhat.com' RCPT:'' X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 66.187.233.73 Subject: [Qemu-devel] [PATCH v12 13/21] migration: Calculate transferred ram correctly X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: lvivier@redhat.com, dgilbert@redhat.com, peterx@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" On multifd we send data from more places that main channel. Signed-off-by: Juan Quintela --- migration/migration.c | 11 +++++++++-- 1 file changed, 9 insertions(+), 2 deletions(-) diff --git a/migration/migration.c b/migration/migration.c index 9b510a809a..75d30661e9 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -2246,12 +2246,19 @@ static void migration_update_counters(MigrationStat= e *s, { uint64_t transferred, time_spent; double bandwidth; + uint64_t now; =20 if (current_time < s->iteration_start_time + BUFFER_DELAY) { return; } =20 - transferred =3D qemu_ftell(s->to_dst_file) - s->iteration_initial_byte= s; + if (migrate_use_multifd()) { + now =3D ram_counters.normal * qemu_target_page_size() + + qemu_ftell(s->to_dst_file); + } else { + now =3D qemu_ftell(s->to_dst_file); + } + transferred =3D now - s->iteration_initial_bytes; time_spent =3D current_time - s->iteration_start_time; bandwidth =3D (double)transferred / time_spent; s->threshold_size =3D bandwidth * s->parameters.downtime_limit; @@ -2271,7 +2278,7 @@ static void migration_update_counters(MigrationState = *s, qemu_file_reset_rate_limit(s->to_dst_file); =20 s->iteration_start_time =3D current_time; - s->iteration_initial_bytes =3D qemu_ftell(s->to_dst_file); + s->iteration_initial_bytes =3D now; =20 trace_migrate_transferred(transferred, time_spent, bandwidth, s->threshold_size); --=20 2.17.0 From nobody Wed Oct 29 06:44:06 2025 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 15246565062924.042753495521765; Wed, 25 Apr 2018 04:41:46 -0700 (PDT) Received: from localhost ([::1]:35916 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fBInx-0005hw-E9 for importer@patchew.org; Wed, 25 Apr 2018 07:41:45 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:34308) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fBIaQ-0002v3-7C for qemu-devel@nongnu.org; Wed, 25 Apr 2018 07:27:47 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fBIaP-0005An-9b for qemu-devel@nongnu.org; Wed, 25 Apr 2018 07:27:46 -0400 Received: from mx3-rdu2.redhat.com ([66.187.233.73]:42398 helo=mx1.redhat.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1fBIaP-0005A1-5h for qemu-devel@nongnu.org; Wed, 25 Apr 2018 07:27:45 -0400 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.rdu2.redhat.com [10.11.54.6]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id C07CF818AAE8 for ; Wed, 25 Apr 2018 11:27:44 +0000 (UTC) Received: from secure.mitica (ovpn-117-169.ams2.redhat.com [10.36.117.169]) by smtp.corp.redhat.com (Postfix) with ESMTP id B98562166BC6; Wed, 25 Apr 2018 11:27:43 +0000 (UTC) From: Juan Quintela To: qemu-devel@nongnu.org Date: Wed, 25 Apr 2018 13:27:16 +0200 Message-Id: <20180425112723.1111-15-quintela@redhat.com> In-Reply-To: <20180425112723.1111-1-quintela@redhat.com> References: <20180425112723.1111-1-quintela@redhat.com> X-Scanned-By: MIMEDefang 2.78 on 10.11.54.6 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.8]); Wed, 25 Apr 2018 11:27:44 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.8]); Wed, 25 Apr 2018 11:27:44 +0000 (UTC) for IP:'10.11.54.6' DOMAIN:'int-mx06.intmail.prod.int.rdu2.redhat.com' HELO:'smtp.corp.redhat.com' FROM:'quintela@redhat.com' RCPT:'' X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 66.187.233.73 Subject: [Qemu-devel] [PATCH v12 14/21] migration: Multifd channels always wait on the sem X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: lvivier@redhat.com, dgilbert@redhat.com, peterx@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Either for quit, sync or packet, we first wake them. Signed-off-by: Juan Quintela --- migration/ram.c | 13 +++++++++++-- 1 file changed, 11 insertions(+), 2 deletions(-) diff --git a/migration/ram.c b/migration/ram.c index 0f1340b4e3..21b448c4ed 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -754,6 +754,7 @@ static void *multifd_send_thread(void *opaque) p->num_packets =3D 1; =20 while (true) { + qemu_sem_wait(&p->sem); qemu_mutex_lock(&p->mutex); multifd_send_fill_packet(p); if (p->quit) { @@ -761,7 +762,9 @@ static void *multifd_send_thread(void *opaque) break; } qemu_mutex_unlock(&p->mutex); - qemu_sem_wait(&p->sem); + /* this is impossible */ + error_setg(&local_err, "multifd_send_thread: Unknown command"); + break; } =20 out: @@ -905,6 +908,7 @@ static void *multifd_recv_thread(void *opaque) trace_multifd_recv_thread_start(p->id); =20 while (true) { + qemu_sem_wait(&p->sem); qemu_mutex_lock(&p->mutex); if (false) { /* ToDo: Packet reception goes here */ @@ -919,9 +923,14 @@ static void *multifd_recv_thread(void *opaque) break; } qemu_mutex_unlock(&p->mutex); - qemu_sem_wait(&p->sem); + /* this is impossible */ + error_setg(&local_err, "multifd_recv_thread: Unknown command"); + break; } =20 + if (local_err) { + multifd_recv_terminate_threads(local_err); + } qemu_mutex_lock(&p->mutex); p->running =3D false; qemu_mutex_unlock(&p->mutex); --=20 2.17.0 From nobody Wed Oct 29 06:44:06 2025 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1524656669944296.2602252045448; Wed, 25 Apr 2018 04:44:29 -0700 (PDT) Received: from localhost ([::1]:35929 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fBIqb-00085h-68 for importer@patchew.org; Wed, 25 Apr 2018 07:44:29 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:34319) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fBIaR-0002wk-OJ for qemu-devel@nongnu.org; Wed, 25 Apr 2018 07:27:48 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fBIaQ-0005CS-Jc for qemu-devel@nongnu.org; Wed, 25 Apr 2018 07:27:47 -0400 Received: from mx3-rdu2.redhat.com ([66.187.233.73]:45568 helo=mx1.redhat.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1fBIaQ-0005Bo-E6 for qemu-devel@nongnu.org; Wed, 25 Apr 2018 07:27:46 -0400 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.rdu2.redhat.com [10.11.54.6]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 1628A402314E for ; Wed, 25 Apr 2018 11:27:46 +0000 (UTC) Received: from secure.mitica (ovpn-117-169.ams2.redhat.com [10.36.117.169]) by smtp.corp.redhat.com (Postfix) with ESMTP id 0F483215CDC8; Wed, 25 Apr 2018 11:27:44 +0000 (UTC) From: Juan Quintela To: qemu-devel@nongnu.org Date: Wed, 25 Apr 2018 13:27:17 +0200 Message-Id: <20180425112723.1111-16-quintela@redhat.com> In-Reply-To: <20180425112723.1111-1-quintela@redhat.com> References: <20180425112723.1111-1-quintela@redhat.com> X-Scanned-By: MIMEDefang 2.78 on 10.11.54.6 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.6]); Wed, 25 Apr 2018 11:27:46 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.6]); Wed, 25 Apr 2018 11:27:46 +0000 (UTC) for IP:'10.11.54.6' DOMAIN:'int-mx06.intmail.prod.int.rdu2.redhat.com' HELO:'smtp.corp.redhat.com' FROM:'quintela@redhat.com' RCPT:'' X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 66.187.233.73 Subject: [Qemu-devel] [PATCH v12 15/21] migration: Add block where to send/receive packets X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: lvivier@redhat.com, dgilbert@redhat.com, peterx@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Once there add tracepoints. Signed-off-by: Juan Quintela Reviewed-by: Dr. David Alan Gilbert --- migration/ram.c | 49 +++++++++++++++++++++++++++++++++++++----- migration/trace-events | 2 ++ 2 files changed, 46 insertions(+), 5 deletions(-) diff --git a/migration/ram.c b/migration/ram.c index 21b448c4ed..c4c185cc4c 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -455,6 +455,8 @@ typedef struct { bool running; /* should this thread finish */ bool quit; + /* thread has work to do */ + int pending_job; /* array of pages to sent */ MultiFDPages_t *pages; /* packet allocated len */ @@ -489,6 +491,8 @@ typedef struct { bool running; /* should this thread finish */ bool quit; + /* thread has work to do */ + bool pending_job; /* array of pages to receive */ MultiFDPages_t *pages; /* packet allocated len */ @@ -756,8 +760,28 @@ static void *multifd_send_thread(void *opaque) while (true) { qemu_sem_wait(&p->sem); qemu_mutex_lock(&p->mutex); - multifd_send_fill_packet(p); - if (p->quit) { + + if (p->pending_job) { + uint32_t used =3D p->pages->used; + uint32_t seq =3D p->seq; + uint32_t flags =3D p->flags; + + multifd_send_fill_packet(p); + p->flags =3D 0; + p->num_packets++; + p->num_pages +=3D used; + p->pages->used =3D 0; + qemu_mutex_unlock(&p->mutex); + + trace_multifd_send(p->id, seq, used, flags); + + /* ToDo: send packet here */ + + qemu_mutex_lock(&p->mutex); + p->pending_job--; + qemu_mutex_unlock(&p->mutex); + continue; + } else if (p->quit) { qemu_mutex_unlock(&p->mutex); break; } @@ -823,6 +847,7 @@ int multifd_save_setup(void) qemu_mutex_init(&p->mutex); qemu_sem_init(&p->sem, 0); p->quit =3D false; + p->pending_job =3D 0; p->id =3D i; multifd_pages_init(&p->pages, page_count); p->packet_len =3D sizeof(MultiFDPacket_t) @@ -910,14 +935,27 @@ static void *multifd_recv_thread(void *opaque) while (true) { qemu_sem_wait(&p->sem); qemu_mutex_lock(&p->mutex); - if (false) { - /* ToDo: Packet reception goes here */ + if (p->pending_job) { + uint32_t used; + uint32_t flags; + qemu_mutex_unlock(&p->mutex); =20 + /* ToDo: recv packet here */ + + qemu_mutex_lock(&p->mutex); ret =3D multifd_recv_unfill_packet(p, &local_err); - qemu_mutex_unlock(&p->mutex); if (ret) { + qemu_mutex_unlock(&p->mutex); break; } + + used =3D p->pages->used; + flags =3D p->flags; + trace_multifd_recv(p->id, p->seq, used, flags); + p->pending_job =3D false; + p->num_packets++; + p->num_pages +=3D used; + qemu_mutex_unlock(&p->mutex); } else if (p->quit) { qemu_mutex_unlock(&p->mutex); break; @@ -960,6 +998,7 @@ int multifd_load_setup(void) qemu_mutex_init(&p->mutex); qemu_sem_init(&p->sem, 0); p->quit =3D false; + p->pending_job =3D false; p->id =3D i; multifd_pages_init(&p->pages, page_count); p->packet_len =3D sizeof(MultiFDPacket_t) diff --git a/migration/trace-events b/migration/trace-events index e480eb050e..9eee048287 100644 --- a/migration/trace-events +++ b/migration/trace-events @@ -81,6 +81,8 @@ multifd_send_thread_start(uint8_t id) "%d" multifd_send_thread_end(uint8_t id, uint32_t packets, uint32_t pages) "cha= nnel %d packets %d pages %d" multifd_recv_thread_start(uint8_t id) "%d" multifd_recv_thread_end(uint8_t id, uint32_t packets, uint32_t pages) "cha= nnel %d packets %d pages %d" +multifd_send(uint8_t id, uint32_t seq, uint32_t used, uint32_t flags) "cha= nnel %d seq number %d pages %d flags 0x%x" +multifd_recv(uint8_t id, uint32_t seq, uint32_t used, uint32_t flags) "cha= nnel %d seq number %d pages %d flags 0x%x" =20 # migration/migration.c await_return_path_close_on_source_close(void) "" --=20 2.17.0 From nobody Wed Oct 29 06:44:06 2025 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1524656295326877.8833847295258; Wed, 25 Apr 2018 04:38:15 -0700 (PDT) Received: from localhost ([::1]:35897 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fBIkS-0002aP-Hp for importer@patchew.org; Wed, 25 Apr 2018 07:38:08 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:34356) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fBIaW-00032q-0F for qemu-devel@nongnu.org; Wed, 25 Apr 2018 07:27:53 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fBIaR-0005EP-Uh for qemu-devel@nongnu.org; Wed, 25 Apr 2018 07:27:52 -0400 Received: from mx3-rdu2.redhat.com ([66.187.233.73]:45570 helo=mx1.redhat.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1fBIaR-0005Dm-Oc for qemu-devel@nongnu.org; Wed, 25 Apr 2018 07:27:47 -0400 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.rdu2.redhat.com [10.11.54.6]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 61BAA402314E for ; Wed, 25 Apr 2018 11:27:47 +0000 (UTC) Received: from secure.mitica (ovpn-117-169.ams2.redhat.com [10.36.117.169]) by smtp.corp.redhat.com (Postfix) with ESMTP id 57B08215CDC8; Wed, 25 Apr 2018 11:27:46 +0000 (UTC) From: Juan Quintela To: qemu-devel@nongnu.org Date: Wed, 25 Apr 2018 13:27:18 +0200 Message-Id: <20180425112723.1111-17-quintela@redhat.com> In-Reply-To: <20180425112723.1111-1-quintela@redhat.com> References: <20180425112723.1111-1-quintela@redhat.com> X-Scanned-By: MIMEDefang 2.78 on 10.11.54.6 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.6]); Wed, 25 Apr 2018 11:27:47 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.6]); Wed, 25 Apr 2018 11:27:47 +0000 (UTC) for IP:'10.11.54.6' DOMAIN:'int-mx06.intmail.prod.int.rdu2.redhat.com' HELO:'smtp.corp.redhat.com' FROM:'quintela@redhat.com' RCPT:'' X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 66.187.233.73 Subject: [Qemu-devel] [PATCH v12 16/21] migration: Synchronize multifd threads with main thread X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: lvivier@redhat.com, dgilbert@redhat.com, peterx@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" We synchronize all threads each RAM_SAVE_FLAG_EOS. Bitmap synchronizations don't happen inside a ram section, so we are safe about two channels trying to overwrite the same memory. Signed-off-by: Juan Quintela --- migration/ram.c | 118 +++++++++++++++++++++++++++++++++++++---- migration/trace-events | 6 +++ 2 files changed, 113 insertions(+), 11 deletions(-) diff --git a/migration/ram.c b/migration/ram.c index c4c185cc4c..398cb0af3b 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -405,6 +405,8 @@ static void compress_threads_save_setup(void) #define MULTIFD_MAGIC 0x11223344U #define MULTIFD_VERSION 1 =20 +#define MULTIFD_FLAG_SYNC (1 << 0) + typedef struct { uint32_t magic; uint32_t version; @@ -471,6 +473,8 @@ typedef struct { uint32_t num_packets; /* pages sent through this channel */ uint32_t num_pages; + /* syncs main thread and channels */ + QemuSemaphore sem_sync; } MultiFDSendParams; =20 typedef struct { @@ -507,6 +511,8 @@ typedef struct { uint32_t num_packets; /* pages sent through this channel */ uint32_t num_pages; + /* syncs main thread and channels */ + QemuSemaphore sem_sync; } MultiFDRecvParams; =20 static int multifd_send_initial_packet(MultiFDSendParams *p, Error **errp) @@ -682,6 +688,10 @@ struct { int count; /* array of pages to sent */ MultiFDPages_t *pages; + /* syncs main thread and channels */ + QemuSemaphore sem_sync; + /* global number of generated multifd packets */ + uint32_t seq; } *multifd_send_state; =20 static void multifd_send_terminate_threads(Error *err) @@ -727,6 +737,7 @@ int multifd_save_cleanup(Error **errp) p->c =3D NULL; qemu_mutex_destroy(&p->mutex); qemu_sem_destroy(&p->sem); + qemu_sem_destroy(&p->sem_sync); g_free(p->name); p->name =3D NULL; multifd_pages_clear(p->pages); @@ -735,6 +746,7 @@ int multifd_save_cleanup(Error **errp) g_free(p->packet); p->packet =3D NULL; } + qemu_sem_destroy(&multifd_send_state->sem_sync); g_free(multifd_send_state->params); multifd_send_state->params =3D NULL; multifd_pages_clear(multifd_send_state->pages); @@ -744,6 +756,33 @@ int multifd_save_cleanup(Error **errp) return ret; } =20 +static void multifd_send_sync_main(void) +{ + int i; + + if (!migrate_use_multifd()) { + return; + } + for (i =3D 0; i < migrate_multifd_channels(); i++) { + MultiFDSendParams *p =3D &multifd_send_state->params[i]; + + trace_multifd_send_sync_main_signal(p->id); + + qemu_mutex_lock(&p->mutex); + p->flags |=3D MULTIFD_FLAG_SYNC; + p->pending_job++; + qemu_mutex_unlock(&p->mutex); + qemu_sem_post(&p->sem); + } + for (i =3D 0; i < migrate_multifd_channels(); i++) { + MultiFDSendParams *p =3D &multifd_send_state->params[i]; + + trace_multifd_send_sync_main_wait(p->id); + qemu_sem_wait(&multifd_send_state->sem_sync); + } + trace_multifd_send_sync_main(multifd_send_state->seq); +} + static void *multifd_send_thread(void *opaque) { MultiFDSendParams *p =3D opaque; @@ -778,17 +817,20 @@ static void *multifd_send_thread(void *opaque) /* ToDo: send packet here */ =20 qemu_mutex_lock(&p->mutex); + p->flags =3D 0; p->pending_job--; qemu_mutex_unlock(&p->mutex); - continue; + + if (flags & MULTIFD_FLAG_SYNC) { + qemu_sem_post(&multifd_send_state->sem_sync); + } } else if (p->quit) { qemu_mutex_unlock(&p->mutex); break; + } else { + qemu_mutex_unlock(&p->mutex); + /* sometimes there are spurious wakeups */ } - qemu_mutex_unlock(&p->mutex); - /* this is impossible */ - error_setg(&local_err, "multifd_send_thread: Unknown command"); - break; } =20 out: @@ -840,12 +882,14 @@ int multifd_save_setup(void) multifd_send_state->params =3D g_new0(MultiFDSendParams, thread_count); atomic_set(&multifd_send_state->count, 0); multifd_pages_init(&multifd_send_state->pages, page_count); + qemu_sem_init(&multifd_send_state->sem_sync, 0); =20 for (i =3D 0; i < thread_count; i++) { MultiFDSendParams *p =3D &multifd_send_state->params[i]; =20 qemu_mutex_init(&p->mutex); qemu_sem_init(&p->sem, 0); + qemu_sem_init(&p->sem_sync, 0); p->quit =3D false; p->pending_job =3D 0; p->id =3D i; @@ -863,6 +907,10 @@ struct { MultiFDRecvParams *params; /* number of created threads */ int count; + /* syncs main thread and channels */ + QemuSemaphore sem_sync; + /* global number of generated multifd packets */ + uint32_t seq; } *multifd_recv_state; =20 static void multifd_recv_terminate_threads(Error *err) @@ -908,6 +956,7 @@ int multifd_load_cleanup(Error **errp) p->c =3D NULL; qemu_mutex_destroy(&p->mutex); qemu_sem_destroy(&p->sem); + qemu_sem_destroy(&p->sem_sync); g_free(p->name); p->name =3D NULL; multifd_pages_clear(p->pages); @@ -916,6 +965,7 @@ int multifd_load_cleanup(Error **errp) g_free(p->packet); p->packet =3D NULL; } + qemu_sem_destroy(&multifd_recv_state->sem_sync); g_free(multifd_recv_state->params); multifd_recv_state->params =3D NULL; g_free(multifd_recv_state); @@ -924,6 +974,42 @@ int multifd_load_cleanup(Error **errp) return ret; } =20 +static void multifd_recv_sync_main(void) +{ + int i; + + if (!migrate_use_multifd()) { + return; + } + for (i =3D 0; i < migrate_multifd_channels(); i++) { + MultiFDRecvParams *p =3D &multifd_recv_state->params[i]; + + trace_multifd_recv_sync_main_signal(p->id); + qemu_mutex_lock(&p->mutex); + p->pending_job =3D true; + qemu_mutex_unlock(&p->mutex); + } + for (i =3D 0; i < migrate_multifd_channels(); i++) { + MultiFDRecvParams *p =3D &multifd_recv_state->params[i]; + + trace_multifd_recv_sync_main_wait(p->id); + qemu_sem_wait(&multifd_recv_state->sem_sync); + qemu_mutex_lock(&p->mutex); + if (multifd_recv_state->seq < p->seq) { + multifd_recv_state->seq =3D p->seq; + } + qemu_mutex_unlock(&p->mutex); + } + for (i =3D 0; i < migrate_multifd_channels(); i++) { + MultiFDRecvParams *p =3D &multifd_recv_state->params[i]; + + trace_multifd_recv_sync_main_signal(p->id); + + qemu_sem_post(&p->sem_sync); + } + trace_multifd_recv_sync_main(multifd_recv_state->seq); +} + static void *multifd_recv_thread(void *opaque) { MultiFDRecvParams *p =3D opaque; @@ -933,9 +1019,8 @@ static void *multifd_recv_thread(void *opaque) trace_multifd_recv_thread_start(p->id); =20 while (true) { - qemu_sem_wait(&p->sem); qemu_mutex_lock(&p->mutex); - if (p->pending_job) { + if (true || p->pending_job) { uint32_t used; uint32_t flags; qemu_mutex_unlock(&p->mutex); @@ -956,14 +1041,18 @@ static void *multifd_recv_thread(void *opaque) p->num_packets++; p->num_pages +=3D used; qemu_mutex_unlock(&p->mutex); + + if (flags & MULTIFD_FLAG_SYNC) { + qemu_sem_post(&multifd_recv_state->sem_sync); + qemu_sem_wait(&p->sem_sync); + } } else if (p->quit) { qemu_mutex_unlock(&p->mutex); break; + } else { + qemu_mutex_unlock(&p->mutex); + /* sometimes there are spurious wakeups */ } - qemu_mutex_unlock(&p->mutex); - /* this is impossible */ - error_setg(&local_err, "multifd_recv_thread: Unknown command"); - break; } =20 if (local_err) { @@ -991,12 +1080,14 @@ int multifd_load_setup(void) multifd_recv_state =3D g_malloc0(sizeof(*multifd_recv_state)); multifd_recv_state->params =3D g_new0(MultiFDRecvParams, thread_count); atomic_set(&multifd_recv_state->count, 0); + qemu_sem_init(&multifd_recv_state->sem_sync, 0); =20 for (i =3D 0; i < thread_count; i++) { MultiFDRecvParams *p =3D &multifd_recv_state->params[i]; =20 qemu_mutex_init(&p->mutex); qemu_sem_init(&p->sem, 0); + qemu_sem_init(&p->sem_sync, 0); p->quit =3D false; p->pending_job =3D false; p->id =3D i; @@ -2695,6 +2786,7 @@ static int ram_save_setup(QEMUFile *f, void *opaque) ram_control_before_iterate(f, RAM_CONTROL_SETUP); ram_control_after_iterate(f, RAM_CONTROL_SETUP); =20 + multifd_send_sync_main(); qemu_put_be64(f, RAM_SAVE_FLAG_EOS); =20 return 0; @@ -2770,6 +2862,7 @@ static int ram_save_iterate(QEMUFile *f, void *opaque) */ ram_control_after_iterate(f, RAM_CONTROL_ROUND); =20 + multifd_send_sync_main(); out: qemu_put_be64(f, RAM_SAVE_FLAG_EOS); ram_counters.transferred +=3D 8; @@ -2823,6 +2916,7 @@ static int ram_save_complete(QEMUFile *f, void *opaqu= e) =20 rcu_read_unlock(); =20 + multifd_send_sync_main(); qemu_put_be64(f, RAM_SAVE_FLAG_EOS); =20 return 0; @@ -3253,6 +3347,7 @@ static int ram_load_postcopy(QEMUFile *f) break; case RAM_SAVE_FLAG_EOS: /* normal exit */ + multifd_recv_sync_main(); break; default: error_report("Unknown combination of migration flags: %#x" @@ -3438,6 +3533,7 @@ static int ram_load(QEMUFile *f, void *opaque, int ve= rsion_id) break; case RAM_SAVE_FLAG_EOS: /* normal exit */ + multifd_recv_sync_main(); break; default: if (flags & RAM_SAVE_FLAG_HOOK) { diff --git a/migration/trace-events b/migration/trace-events index 9eee048287..b0ab8e2d03 100644 --- a/migration/trace-events +++ b/migration/trace-events @@ -83,6 +83,12 @@ multifd_recv_thread_start(uint8_t id) "%d" multifd_recv_thread_end(uint8_t id, uint32_t packets, uint32_t pages) "cha= nnel %d packets %d pages %d" multifd_send(uint8_t id, uint32_t seq, uint32_t used, uint32_t flags) "cha= nnel %d seq number %d pages %d flags 0x%x" multifd_recv(uint8_t id, uint32_t seq, uint32_t used, uint32_t flags) "cha= nnel %d seq number %d pages %d flags 0x%x" +multifd_send_sync_main(uint32_t seq) "seq %d" +multifd_send_sync_main_signal(uint8_t id) "channel %d" +multifd_send_sync_main_wait(uint8_t id) "channel %d" +multifd_recv_sync_main(uint32_t seq) "seq %d" +multifd_recv_sync_main_signal(uint8_t id) "channel %d" +multifd_recv_sync_main_wait(uint8_t id) "channel %d" =20 # migration/migration.c await_return_path_close_on_source_close(void) "" --=20 2.17.0 From nobody Wed Oct 29 06:44:06 2025 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1524656816238314.1881596181636; Wed, 25 Apr 2018 04:46:56 -0700 (PDT) Received: from localhost ([::1]:35950 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fBIsx-0001uO-3S for importer@patchew.org; Wed, 25 Apr 2018 07:46:55 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:34355) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fBIaW-00032p-0J for qemu-devel@nongnu.org; Wed, 25 Apr 2018 07:27:53 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fBIaT-0005GJ-5S for qemu-devel@nongnu.org; Wed, 25 Apr 2018 07:27:52 -0400 Received: from mx3-rdu2.redhat.com ([66.187.233.73]:51558 helo=mx1.redhat.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1fBIaT-0005Fw-1Y for qemu-devel@nongnu.org; Wed, 25 Apr 2018 07:27:49 -0400 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.rdu2.redhat.com [10.11.54.6]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id ACF028D76C for ; Wed, 25 Apr 2018 11:27:48 +0000 (UTC) Received: from secure.mitica (ovpn-117-169.ams2.redhat.com [10.36.117.169]) by smtp.corp.redhat.com (Postfix) with ESMTP id A3C6C215CDC8; Wed, 25 Apr 2018 11:27:47 +0000 (UTC) From: Juan Quintela To: qemu-devel@nongnu.org Date: Wed, 25 Apr 2018 13:27:19 +0200 Message-Id: <20180425112723.1111-18-quintela@redhat.com> In-Reply-To: <20180425112723.1111-1-quintela@redhat.com> References: <20180425112723.1111-1-quintela@redhat.com> X-Scanned-By: MIMEDefang 2.78 on 10.11.54.6 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.2]); Wed, 25 Apr 2018 11:27:48 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.2]); Wed, 25 Apr 2018 11:27:48 +0000 (UTC) for IP:'10.11.54.6' DOMAIN:'int-mx06.intmail.prod.int.rdu2.redhat.com' HELO:'smtp.corp.redhat.com' FROM:'quintela@redhat.com' RCPT:'' X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 66.187.233.73 Subject: [Qemu-devel] [PATCH v12 17/21] migration: Create ram_multifd_page X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: lvivier@redhat.com, dgilbert@redhat.com, peterx@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" The function still don't use multifd, but we have simplified ram_save_page, xbzrle and RDMA stuff is gone. We have added a new counter. Signed-off-by: Juan Quintela -- Add last_page parameter Add commets for done and address Remove multifd field, it is the same than normal pages Merge next patch, now we send multiple pages at a time Remove counter for multifd pages, it is identical to normal pages Use iovec's instead of creating the equivalent. Clear memory used by pages (dave) Use g_new0(danp) define MULTIFD_CONTINUE now pages member is a pointer Fix off-by-one in number of pages in one packet Remove RAM_SAVE_FLAG_MULTIFD_PAGE s/multifd_pages_t/MultiFDPages_t/ --- migration/ram.c | 93 +++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 93 insertions(+) diff --git a/migration/ram.c b/migration/ram.c index 398cb0af3b..862ec53d32 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -54,6 +54,7 @@ #include "migration/block.h" #include "sysemu/sysemu.h" #include "qemu/uuid.h" +#include "qemu/iov.h" =20 /***********************************************************/ /* ram save/restore */ @@ -692,8 +693,65 @@ struct { QemuSemaphore sem_sync; /* global number of generated multifd packets */ uint32_t seq; + /* send channels ready */ + QemuSemaphore channels_ready; } *multifd_send_state; =20 +static void multifd_send_pages(void) +{ + int i; + static int next_channel; + MultiFDSendParams *p =3D NULL; /* make happy gcc */ + MultiFDPages_t *pages =3D multifd_send_state->pages; + + qemu_sem_wait(&multifd_send_state->channels_ready); + for (i =3D next_channel;; i =3D (i + 1) % migrate_multifd_channels()) { + p =3D &multifd_send_state->params[i]; + + qemu_mutex_lock(&p->mutex); + if (!p->pending_job) { + p->pending_job++; + next_channel =3D (i + 1) % migrate_multifd_channels(); + break; + } + qemu_mutex_unlock(&p->mutex); + } + p->pages->used =3D 0; + multifd_send_state->seq++; + p->seq =3D multifd_send_state->seq; + p->pages->block =3D NULL; + multifd_send_state->pages =3D p->pages; + p->pages =3D pages; + qemu_mutex_unlock(&p->mutex); + qemu_sem_post(&p->sem); +} + +static void multifd_queue_page(RAMBlock *block, ram_addr_t offset) +{ + MultiFDPages_t *pages =3D multifd_send_state->pages; + + if (!pages->block) { + pages->block =3D block; + } + + if (pages->block =3D=3D block) { + pages->offset[pages->used] =3D offset; + pages->iov[pages->used].iov_base =3D block->host + offset; + pages->iov[pages->used].iov_len =3D TARGET_PAGE_SIZE; + pages->used++; + + if (pages->used < pages->allocated) { + return; + } + } + + multifd_send_pages(); + + if (pages->block !=3D block) { + multifd_queue_page(block, offset); + } +} + static void multifd_send_terminate_threads(Error *err) { int i; @@ -746,6 +804,7 @@ int multifd_save_cleanup(Error **errp) g_free(p->packet); p->packet =3D NULL; } + qemu_sem_destroy(&multifd_send_state->channels_ready); qemu_sem_destroy(&multifd_send_state->sem_sync); g_free(multifd_send_state->params); multifd_send_state->params =3D NULL; @@ -763,12 +822,17 @@ static void multifd_send_sync_main(void) if (!migrate_use_multifd()) { return; } + if (multifd_send_state->pages->used) { + multifd_send_pages(); + } for (i =3D 0; i < migrate_multifd_channels(); i++) { MultiFDSendParams *p =3D &multifd_send_state->params[i]; =20 trace_multifd_send_sync_main_signal(p->id); =20 qemu_mutex_lock(&p->mutex); + multifd_send_state->seq++; + p->seq =3D multifd_send_state->seq; p->flags |=3D MULTIFD_FLAG_SYNC; p->pending_job++; qemu_mutex_unlock(&p->mutex); @@ -824,6 +888,7 @@ static void *multifd_send_thread(void *opaque) if (flags & MULTIFD_FLAG_SYNC) { qemu_sem_post(&multifd_send_state->sem_sync); } + qemu_sem_post(&multifd_send_state->channels_ready); } else if (p->quit) { qemu_mutex_unlock(&p->mutex); break; @@ -883,6 +948,7 @@ int multifd_save_setup(void) atomic_set(&multifd_send_state->count, 0); multifd_pages_init(&multifd_send_state->pages, page_count); qemu_sem_init(&multifd_send_state->sem_sync, 0); + qemu_sem_init(&multifd_send_state->channels_ready, 0); =20 for (i =3D 0; i < thread_count; i++) { MultiFDSendParams *p =3D &multifd_send_state->params[i]; @@ -1576,6 +1642,31 @@ static int ram_save_page(RAMState *rs, PageSearchSta= tus *pss, bool last_stage) return pages; } =20 +static int ram_multifd_page(RAMState *rs, PageSearchStatus *pss, + bool last_stage) +{ + int pages; + uint8_t *p; + RAMBlock *block =3D pss->block; + ram_addr_t offset =3D pss->page << TARGET_PAGE_BITS; + + p =3D block->host + offset; + + pages =3D save_zero_page(rs, block, offset); + if (pages =3D=3D -1) { + ram_counters.transferred +=3D + save_page_header(rs, rs->f, block, + offset | RAM_SAVE_FLAG_PAGE); + multifd_queue_page(block, offset); + qemu_put_buffer(rs->f, p, TARGET_PAGE_SIZE); + ram_counters.transferred +=3D TARGET_PAGE_SIZE; + pages =3D 1; + ram_counters.normal++; + } + + return pages; +} + static int do_compress_ram_page(QEMUFile *f, RAMBlock *block, ram_addr_t offset) { @@ -2004,6 +2095,8 @@ static int ram_save_target_page(RAMState *rs, PageSea= rchStatus *pss, if (migrate_use_compression() && (rs->ram_bulk_stage || !migrate_use_xbzrle())) { res =3D ram_save_compressed_page(rs, pss, last_stage); + } else if (migrate_use_multifd()) { + res =3D ram_multifd_page(rs, pss, last_stage); } else { res =3D ram_save_page(rs, pss, last_stage); } --=20 2.17.0 From nobody Wed Oct 29 06:44:06 2025 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1524656680250853.5303541835033; Wed, 25 Apr 2018 04:44:40 -0700 (PDT) Received: from localhost ([::1]:35930 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fBIql-0008Ih-CB for importer@patchew.org; Wed, 25 Apr 2018 07:44:39 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:34354) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fBIaW-00032o-02 for qemu-devel@nongnu.org; Wed, 25 Apr 2018 07:27:52 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fBIaU-0005IH-GG for qemu-devel@nongnu.org; Wed, 25 Apr 2018 07:27:52 -0400 Received: from mx3-rdu2.redhat.com ([66.187.233.73]:51560 helo=mx1.redhat.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1fBIaU-0005HX-CR for qemu-devel@nongnu.org; Wed, 25 Apr 2018 07:27:50 -0400 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.rdu2.redhat.com [10.11.54.6]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 01BD78DC4D for ; Wed, 25 Apr 2018 11:27:50 +0000 (UTC) Received: from secure.mitica (ovpn-117-169.ams2.redhat.com [10.36.117.169]) by smtp.corp.redhat.com (Postfix) with ESMTP id EF5AF2166BC6; Wed, 25 Apr 2018 11:27:48 +0000 (UTC) From: Juan Quintela To: qemu-devel@nongnu.org Date: Wed, 25 Apr 2018 13:27:20 +0200 Message-Id: <20180425112723.1111-19-quintela@redhat.com> In-Reply-To: <20180425112723.1111-1-quintela@redhat.com> References: <20180425112723.1111-1-quintela@redhat.com> X-Scanned-By: MIMEDefang 2.78 on 10.11.54.6 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.2]); Wed, 25 Apr 2018 11:27:50 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.2]); Wed, 25 Apr 2018 11:27:50 +0000 (UTC) for IP:'10.11.54.6' DOMAIN:'int-mx06.intmail.prod.int.rdu2.redhat.com' HELO:'smtp.corp.redhat.com' FROM:'quintela@redhat.com' RCPT:'' X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 66.187.233.73 Subject: [Qemu-devel] [PATCH v12 18/21] migration: Start sending messages X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: lvivier@redhat.com, dgilbert@redhat.com, peterx@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Signed-off-by: Juan Quintela --- migration/ram.c | 30 ++++++++++++++++++++++++------ 1 file changed, 24 insertions(+), 6 deletions(-) diff --git a/migration/ram.c b/migration/ram.c index 862ec53d32..9adbaa81f9 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -625,9 +625,6 @@ static int multifd_recv_unfill_packet(MultiFDRecvParams= *p, Error **errp) RAMBlock *block; int i; =20 - /* ToDo: We can't use it until we haven't received a message */ - return 0; - be32_to_cpus(&packet->magic); if (packet->magic !=3D MULTIFD_MAGIC) { error_setg(errp, "multifd: received packet " @@ -851,6 +848,7 @@ static void *multifd_send_thread(void *opaque) { MultiFDSendParams *p =3D opaque; Error *local_err =3D NULL; + int ret; =20 trace_multifd_send_thread_start(p->id); =20 @@ -878,10 +876,18 @@ static void *multifd_send_thread(void *opaque) =20 trace_multifd_send(p->id, seq, used, flags); =20 - /* ToDo: send packet here */ + ret =3D qio_channel_write_all(p->c, (void *)p->packet, + p->packet_len, &local_err); + if (ret !=3D 0) { + break; + } + + ret =3D qio_channel_writev_all(p->c, p->pages->iov, used, &loc= al_err); + if (ret !=3D 0) { + break; + } =20 qemu_mutex_lock(&p->mutex); - p->flags =3D 0; p->pending_job--; qemu_mutex_unlock(&p->mutex); =20 @@ -1091,7 +1097,14 @@ static void *multifd_recv_thread(void *opaque) uint32_t flags; qemu_mutex_unlock(&p->mutex); =20 - /* ToDo: recv packet here */ + ret =3D qio_channel_read_all_eof(p->c, (void *)p->packet, + p->packet_len, &local_err); + if (ret =3D=3D 0) { /* EOF */ + break; + } + if (ret =3D=3D -1) { /* Error */ + break; + } =20 qemu_mutex_lock(&p->mutex); ret =3D multifd_recv_unfill_packet(p, &local_err); @@ -1108,6 +1121,11 @@ static void *multifd_recv_thread(void *opaque) p->num_pages +=3D used; qemu_mutex_unlock(&p->mutex); =20 + ret =3D qio_channel_readv_all(p->c, p->pages->iov, used, &loca= l_err); + if (ret !=3D 0) { + break; + } + if (flags & MULTIFD_FLAG_SYNC) { qemu_sem_post(&multifd_recv_state->sem_sync); qemu_sem_wait(&p->sem_sync); --=20 2.17.0 From nobody Wed Oct 29 06:44:06 2025 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1524656463096506.66916515656794; Wed, 25 Apr 2018 04:41:03 -0700 (PDT) Received: from localhost ([::1]:35914 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fBInG-00054M-8D for importer@patchew.org; Wed, 25 Apr 2018 07:41:02 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:34376) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fBIaW-00033i-V5 for qemu-devel@nongnu.org; Wed, 25 Apr 2018 07:27:54 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fBIaV-0005K8-Rc for qemu-devel@nongnu.org; Wed, 25 Apr 2018 07:27:52 -0400 Received: from mx3-rdu2.redhat.com ([66.187.233.73]:33588 helo=mx1.redhat.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1fBIaV-0005JZ-M2 for qemu-devel@nongnu.org; Wed, 25 Apr 2018 07:27:51 -0400 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.rdu2.redhat.com [10.11.54.6]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 4E886EB6E5 for ; Wed, 25 Apr 2018 11:27:51 +0000 (UTC) Received: from secure.mitica (ovpn-117-169.ams2.redhat.com [10.36.117.169]) by smtp.corp.redhat.com (Postfix) with ESMTP id 44FAD215CDD1; Wed, 25 Apr 2018 11:27:50 +0000 (UTC) From: Juan Quintela To: qemu-devel@nongnu.org Date: Wed, 25 Apr 2018 13:27:21 +0200 Message-Id: <20180425112723.1111-20-quintela@redhat.com> In-Reply-To: <20180425112723.1111-1-quintela@redhat.com> References: <20180425112723.1111-1-quintela@redhat.com> X-Scanned-By: MIMEDefang 2.78 on 10.11.54.6 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.1]); Wed, 25 Apr 2018 11:27:51 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.1]); Wed, 25 Apr 2018 11:27:51 +0000 (UTC) for IP:'10.11.54.6' DOMAIN:'int-mx06.intmail.prod.int.rdu2.redhat.com' HELO:'smtp.corp.redhat.com' FROM:'quintela@redhat.com' RCPT:'' X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 66.187.233.73 Subject: [Qemu-devel] [PATCH v12 19/21] migration: Wait for blocking IO X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: lvivier@redhat.com, dgilbert@redhat.com, peterx@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" We have three conditions here: - channel fails -> error - we have to quit: we close the channel and reads fails - normal read that success, we are in bussiness So forget the complications of waiting in a semaphore. Signed-off-by: Juan Quintela --- migration/ram.c | 81 ++++++++++++++++++------------------------------- 1 file changed, 29 insertions(+), 52 deletions(-) diff --git a/migration/ram.c b/migration/ram.c index 9adbaa81f9..2734f91ded 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -496,8 +496,6 @@ typedef struct { bool running; /* should this thread finish */ bool quit; - /* thread has work to do */ - bool pending_job; /* array of pages to receive */ MultiFDPages_t *pages; /* packet allocated len */ @@ -1056,14 +1054,6 @@ static void multifd_recv_sync_main(void) for (i =3D 0; i < migrate_multifd_channels(); i++) { MultiFDRecvParams *p =3D &multifd_recv_state->params[i]; =20 - trace_multifd_recv_sync_main_signal(p->id); - qemu_mutex_lock(&p->mutex); - p->pending_job =3D true; - qemu_mutex_unlock(&p->mutex); - } - for (i =3D 0; i < migrate_multifd_channels(); i++) { - MultiFDRecvParams *p =3D &multifd_recv_state->params[i]; - trace_multifd_recv_sync_main_wait(p->id); qemu_sem_wait(&multifd_recv_state->sem_sync); qemu_mutex_lock(&p->mutex); @@ -1076,7 +1066,6 @@ static void multifd_recv_sync_main(void) MultiFDRecvParams *p =3D &multifd_recv_state->params[i]; =20 trace_multifd_recv_sync_main_signal(p->id); - qemu_sem_post(&p->sem_sync); } trace_multifd_recv_sync_main(multifd_recv_state->seq); @@ -1091,51 +1080,40 @@ static void *multifd_recv_thread(void *opaque) trace_multifd_recv_thread_start(p->id); =20 while (true) { + uint32_t used; + uint32_t flags; + + ret =3D qio_channel_read_all_eof(p->c, (void *)p->packet, + p->packet_len, &local_err); + if (ret =3D=3D 0) { /* EOF */ + break; + } + if (ret =3D=3D -1) { /* Error */ + break; + } + qemu_mutex_lock(&p->mutex); - if (true || p->pending_job) { - uint32_t used; - uint32_t flags; - qemu_mutex_unlock(&p->mutex); - - ret =3D qio_channel_read_all_eof(p->c, (void *)p->packet, - p->packet_len, &local_err); - if (ret =3D=3D 0) { /* EOF */ - break; - } - if (ret =3D=3D -1) { /* Error */ - break; - } - - qemu_mutex_lock(&p->mutex); - ret =3D multifd_recv_unfill_packet(p, &local_err); - if (ret) { - qemu_mutex_unlock(&p->mutex); - break; - } - - used =3D p->pages->used; - flags =3D p->flags; - trace_multifd_recv(p->id, p->seq, used, flags); - p->pending_job =3D false; - p->num_packets++; - p->num_pages +=3D used; + ret =3D multifd_recv_unfill_packet(p, &local_err); + if (ret) { qemu_mutex_unlock(&p->mutex); + break; + } =20 - ret =3D qio_channel_readv_all(p->c, p->pages->iov, used, &loca= l_err); - if (ret !=3D 0) { - break; - } + used =3D p->pages->used; + flags =3D p->flags; + trace_multifd_recv(p->id, p->seq, used, flags); + p->num_packets++; + p->num_pages +=3D used; + qemu_mutex_unlock(&p->mutex); =20 - if (flags & MULTIFD_FLAG_SYNC) { - qemu_sem_post(&multifd_recv_state->sem_sync); - qemu_sem_wait(&p->sem_sync); - } - } else if (p->quit) { - qemu_mutex_unlock(&p->mutex); + ret =3D qio_channel_readv_all(p->c, p->pages->iov, used, &local_er= r); + if (ret !=3D 0) { break; - } else { - qemu_mutex_unlock(&p->mutex); - /* sometimes there are spurious wakeups */ + } + + if (flags & MULTIFD_FLAG_SYNC) { + qemu_sem_post(&multifd_recv_state->sem_sync); + qemu_sem_wait(&p->sem_sync); } } =20 @@ -1173,7 +1151,6 @@ int multifd_load_setup(void) qemu_sem_init(&p->sem, 0); qemu_sem_init(&p->sem_sync, 0); p->quit =3D false; - p->pending_job =3D false; p->id =3D i; multifd_pages_init(&p->pages, page_count); p->packet_len =3D sizeof(MultiFDPacket_t) --=20 2.17.0 From nobody Wed Oct 29 06:44:06 2025 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (208.118.235.17 [208.118.235.17]) by mx.zohomail.com with SMTPS id 1524656636003236.94526686253982; Wed, 25 Apr 2018 04:43:56 -0700 (PDT) Received: from localhost ([::1]:35927 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fBIps-0007QY-Em for importer@patchew.org; Wed, 25 Apr 2018 07:43:44 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:34396) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fBIaY-00034m-2I for qemu-devel@nongnu.org; Wed, 25 Apr 2018 07:27:55 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fBIaX-0005MQ-4l for qemu-devel@nongnu.org; Wed, 25 Apr 2018 07:27:54 -0400 Received: from mx3-rdu2.redhat.com ([66.187.233.73]:45572 helo=mx1.redhat.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1fBIaW-0005Lh-WC for qemu-devel@nongnu.org; Wed, 25 Apr 2018 07:27:53 -0400 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.rdu2.redhat.com [10.11.54.6]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 98B144022414 for ; Wed, 25 Apr 2018 11:27:52 +0000 (UTC) Received: from secure.mitica (ovpn-117-169.ams2.redhat.com [10.36.117.169]) by smtp.corp.redhat.com (Postfix) with ESMTP id 91D37215CDD1; Wed, 25 Apr 2018 11:27:51 +0000 (UTC) From: Juan Quintela To: qemu-devel@nongnu.org Date: Wed, 25 Apr 2018 13:27:22 +0200 Message-Id: <20180425112723.1111-21-quintela@redhat.com> In-Reply-To: <20180425112723.1111-1-quintela@redhat.com> References: <20180425112723.1111-1-quintela@redhat.com> X-Scanned-By: MIMEDefang 2.78 on 10.11.54.6 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.6]); Wed, 25 Apr 2018 11:27:52 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.6]); Wed, 25 Apr 2018 11:27:52 +0000 (UTC) for IP:'10.11.54.6' DOMAIN:'int-mx06.intmail.prod.int.rdu2.redhat.com' HELO:'smtp.corp.redhat.com' FROM:'quintela@redhat.com' RCPT:'' X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 66.187.233.73 Subject: [Qemu-devel] [PATCH v12 20/21] migration: Remove not needed semaphore and quit X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: lvivier@redhat.com, dgilbert@redhat.com, peterx@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" We know quit closing the QIO. Signed-off-by: Juan Quintela --- migration/ram.c | 11 ++--------- 1 file changed, 2 insertions(+), 9 deletions(-) diff --git a/migration/ram.c b/migration/ram.c index 2734f91ded..23203756b7 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -488,14 +488,10 @@ typedef struct { QemuThread thread; /* communication channel */ QIOChannel *c; - /* sem where to wait for more work */ - QemuSemaphore sem; /* this mutex protects the following parameters */ QemuMutex mutex; /* is this channel thread running */ bool running; - /* should this thread finish */ - bool quit; /* array of pages to receive */ MultiFDPages_t *pages; /* packet allocated len */ @@ -1001,8 +997,8 @@ static void multifd_recv_terminate_threads(Error *err) MultiFDRecvParams *p =3D &multifd_recv_state->params[i]; =20 qemu_mutex_lock(&p->mutex); - p->quit =3D true; - qemu_sem_post(&p->sem); + object_unref(OBJECT(p->c)); + p->c =3D NULL; qemu_mutex_unlock(&p->mutex); } } @@ -1025,7 +1021,6 @@ int multifd_load_cleanup(Error **errp) object_unref(OBJECT(p->c)); p->c =3D NULL; qemu_mutex_destroy(&p->mutex); - qemu_sem_destroy(&p->sem); qemu_sem_destroy(&p->sem_sync); g_free(p->name); p->name =3D NULL; @@ -1148,9 +1143,7 @@ int multifd_load_setup(void) MultiFDRecvParams *p =3D &multifd_recv_state->params[i]; =20 qemu_mutex_init(&p->mutex); - qemu_sem_init(&p->sem, 0); qemu_sem_init(&p->sem_sync, 0); - p->quit =3D false; p->id =3D i; multifd_pages_init(&p->pages, page_count); p->packet_len =3D sizeof(MultiFDPacket_t) --=20 2.17.0 From nobody Wed Oct 29 06:44:06 2025 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1524656824302946.1588365585789; Wed, 25 Apr 2018 04:47:04 -0700 (PDT) Received: from localhost ([::1]:35951 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fBIt5-00021J-Ek for importer@patchew.org; Wed, 25 Apr 2018 07:47:03 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:34414) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fBIaZ-00036C-8j for qemu-devel@nongnu.org; Wed, 25 Apr 2018 07:27:56 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fBIaY-0005OR-DZ for qemu-devel@nongnu.org; Wed, 25 Apr 2018 07:27:55 -0400 Received: from mx3-rdu2.redhat.com ([66.187.233.73]:59520 helo=mx1.redhat.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1fBIaY-0005Np-8y for qemu-devel@nongnu.org; Wed, 25 Apr 2018 07:27:54 -0400 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.rdu2.redhat.com [10.11.54.6]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id E20E7406E973 for ; Wed, 25 Apr 2018 11:27:53 +0000 (UTC) Received: from secure.mitica (ovpn-117-169.ams2.redhat.com [10.36.117.169]) by smtp.corp.redhat.com (Postfix) with ESMTP id D9FFC215CDCF; Wed, 25 Apr 2018 11:27:52 +0000 (UTC) From: Juan Quintela To: qemu-devel@nongnu.org Date: Wed, 25 Apr 2018 13:27:23 +0200 Message-Id: <20180425112723.1111-22-quintela@redhat.com> In-Reply-To: <20180425112723.1111-1-quintela@redhat.com> References: <20180425112723.1111-1-quintela@redhat.com> X-Scanned-By: MIMEDefang 2.78 on 10.11.54.6 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.7]); Wed, 25 Apr 2018 11:27:53 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.7]); Wed, 25 Apr 2018 11:27:53 +0000 (UTC) for IP:'10.11.54.6' DOMAIN:'int-mx06.intmail.prod.int.rdu2.redhat.com' HELO:'smtp.corp.redhat.com' FROM:'quintela@redhat.com' RCPT:'' X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 66.187.233.73 Subject: [Qemu-devel] [PATCH v12 21/21] migration: Stop sending whole pages through main channel X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: lvivier@redhat.com, dgilbert@redhat.com, peterx@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" We have to flush() the QEMUFile because now we sent really few data through that channel. Signed-off-by: Juan Quintela Reviewed-by: Dr. David Alan Gilbert --- migration/ram.c | 11 +++-------- 1 file changed, 3 insertions(+), 8 deletions(-) diff --git a/migration/ram.c b/migration/ram.c index 23203756b7..f5cff2eb59 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -1634,20 +1634,12 @@ static int ram_multifd_page(RAMState *rs, PageSearc= hStatus *pss, bool last_stage) { int pages; - uint8_t *p; RAMBlock *block =3D pss->block; ram_addr_t offset =3D pss->page << TARGET_PAGE_BITS; =20 - p =3D block->host + offset; - pages =3D save_zero_page(rs, block, offset); if (pages =3D=3D -1) { - ram_counters.transferred +=3D - save_page_header(rs, rs->f, block, - offset | RAM_SAVE_FLAG_PAGE); multifd_queue_page(block, offset); - qemu_put_buffer(rs->f, p, TARGET_PAGE_SIZE); - ram_counters.transferred +=3D TARGET_PAGE_SIZE; pages =3D 1; ram_counters.normal++; } @@ -2869,6 +2861,7 @@ static int ram_save_setup(QEMUFile *f, void *opaque) =20 multifd_send_sync_main(); qemu_put_be64(f, RAM_SAVE_FLAG_EOS); + qemu_fflush(f); =20 return 0; } @@ -2946,6 +2939,7 @@ static int ram_save_iterate(QEMUFile *f, void *opaque) multifd_send_sync_main(); out: qemu_put_be64(f, RAM_SAVE_FLAG_EOS); + qemu_fflush(f); ram_counters.transferred +=3D 8; =20 ret =3D qemu_file_get_error(f); @@ -2999,6 +2993,7 @@ static int ram_save_complete(QEMUFile *f, void *opaqu= e) =20 multifd_send_sync_main(); qemu_put_be64(f, RAM_SAVE_FLAG_EOS); + qemu_fflush(f); =20 return 0; } --=20 2.17.0