From nobody Sat Apr 27 21:21:01 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1529537452173677.4694488065027; Wed, 20 Jun 2018 16:30:52 -0700 (PDT) Received: from localhost ([::1]:52165 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fVmYt-0002e6-3N for importer@patchew.org; Wed, 20 Jun 2018 19:30:51 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:48358) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fVmX8-0001a1-KN for qemu-devel@nongnu.org; Wed, 20 Jun 2018 19:29:04 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fVmX6-0007BC-0F for qemu-devel@nongnu.org; Wed, 20 Jun 2018 19:29:02 -0400 Received: from mx3-rdu2.redhat.com ([66.187.233.73]:46174 helo=mx1.redhat.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1fVmX5-0007Ay-Q2 for qemu-devel@nongnu.org; Wed, 20 Jun 2018 19:28:59 -0400 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.rdu2.redhat.com [10.11.54.6]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 650E74022909 for ; Wed, 20 Jun 2018 23:28:59 +0000 (UTC) Received: from secure.mitica (ovpn-116-118.ams2.redhat.com [10.36.116.118]) by smtp.corp.redhat.com (Postfix) with ESMTP id 597592156880; Wed, 20 Jun 2018 23:28:58 +0000 (UTC) From: Juan Quintela To: qemu-devel@nongnu.org Date: Thu, 21 Jun 2018 01:28:40 +0200 Message-Id: <20180620232851.7152-2-quintela@redhat.com> In-Reply-To: <20180620232851.7152-1-quintela@redhat.com> References: <20180620232851.7152-1-quintela@redhat.com> X-Scanned-By: MIMEDefang 2.78 on 10.11.54.6 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.6]); Wed, 20 Jun 2018 23:28:59 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.6]); Wed, 20 Jun 2018 23:28:59 +0000 (UTC) for IP:'10.11.54.6' DOMAIN:'int-mx06.intmail.prod.int.rdu2.redhat.com' HELO:'smtp.corp.redhat.com' FROM:'quintela@redhat.com' RCPT:'' X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 66.187.233.73 Subject: [Qemu-devel] [PATCH v15 01/12] migration: Create multipage support X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: lvivier@redhat.com, dgilbert@redhat.com, peterx@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" We only create/destry the page list here. We will use it later. Signed-off-by: Juan Quintela Reviewed-by: Dr. David Alan Gilbert --- migration/ram.c | 57 +++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 57 insertions(+) diff --git a/migration/ram.c b/migration/ram.c index cd5f55117d..ed4401ee46 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -517,6 +517,20 @@ typedef struct { uint8_t id; } __attribute__((packed)) MultiFDInit_t; =20 +typedef struct { + /* number of used pages */ + uint32_t used; + /* number of allocated pages */ + uint32_t allocated; + /* global number of generated multifd packets */ + uint64_t packet_num; + /* offset of each page */ + ram_addr_t *offset; + /* pointer to each page */ + struct iovec *iov; + RAMBlock *block; +} MultiFDPages_t; + typedef struct { /* this fields are not changed once the thread is created */ /* channel number */ @@ -535,6 +549,8 @@ typedef struct { bool running; /* should this thread finish */ bool quit; + /* array of pages to sent */ + MultiFDPages_t *pages; } MultiFDSendParams; =20 typedef struct { @@ -555,6 +571,8 @@ typedef struct { bool running; /* should this thread finish */ bool quit; + /* array of pages to receive */ + MultiFDPages_t *pages; } MultiFDRecvParams; =20 static int multifd_send_initial_packet(MultiFDSendParams *p, Error **errp) @@ -619,10 +637,36 @@ static int multifd_recv_initial_packet(QIOChannel *c,= Error **errp) return msg.id; } =20 +static MultiFDPages_t *multifd_pages_init(size_t size) +{ + MultiFDPages_t *pages =3D g_new0(MultiFDPages_t, 1); + + pages->allocated =3D size; + pages->iov =3D g_new0(struct iovec, size); + pages->offset =3D g_new0(ram_addr_t, size); + + return pages; +} + +static void multifd_pages_clear(MultiFDPages_t *pages) +{ + pages->used =3D 0; + pages->allocated =3D 0; + pages->packet_num =3D 0; + pages->block =3D NULL; + g_free(pages->iov); + pages->iov =3D NULL; + g_free(pages->offset); + pages->offset =3D NULL; + g_free(pages); +} + struct { MultiFDSendParams *params; /* number of created threads */ int count; + /* array of pages to sent */ + MultiFDPages_t *pages; } *multifd_send_state; =20 static void multifd_send_terminate_threads(Error *err) @@ -672,9 +716,13 @@ int multifd_save_cleanup(Error **errp) qemu_sem_destroy(&p->sem); g_free(p->name); p->name =3D NULL; + multifd_pages_clear(p->pages); + p->pages =3D NULL; } g_free(multifd_send_state->params); multifd_send_state->params =3D NULL; + multifd_pages_clear(multifd_send_state->pages); + multifd_send_state->pages =3D NULL; g_free(multifd_send_state); multifd_send_state =3D NULL; return ret; @@ -735,6 +783,7 @@ static void multifd_new_send_channel_async(QIOTask *tas= k, gpointer opaque) int multifd_save_setup(void) { int thread_count; + uint32_t page_count =3D migrate_multifd_page_count(); uint8_t i; =20 if (!migrate_use_multifd()) { @@ -744,6 +793,8 @@ int multifd_save_setup(void) multifd_send_state =3D g_malloc0(sizeof(*multifd_send_state)); multifd_send_state->params =3D g_new0(MultiFDSendParams, thread_count); atomic_set(&multifd_send_state->count, 0); + multifd_send_state->pages =3D multifd_pages_init(page_count); + for (i =3D 0; i < thread_count; i++) { MultiFDSendParams *p =3D &multifd_send_state->params[i]; =20 @@ -751,6 +802,7 @@ int multifd_save_setup(void) qemu_sem_init(&p->sem, 0); p->quit =3D false; p->id =3D i; + p->pages =3D multifd_pages_init(page_count); p->name =3D g_strdup_printf("multifdsend_%d", i); socket_send_channel_create(multifd_new_send_channel_async, p); } @@ -808,6 +860,8 @@ int multifd_load_cleanup(Error **errp) qemu_sem_destroy(&p->sem); g_free(p->name); p->name =3D NULL; + multifd_pages_clear(p->pages); + p->pages =3D NULL; } g_free(multifd_recv_state->params); multifd_recv_state->params =3D NULL; @@ -841,6 +895,7 @@ static void *multifd_recv_thread(void *opaque) int multifd_load_setup(void) { int thread_count; + uint32_t page_count =3D migrate_multifd_page_count(); uint8_t i; =20 if (!migrate_use_multifd()) { @@ -850,6 +905,7 @@ int multifd_load_setup(void) multifd_recv_state =3D g_malloc0(sizeof(*multifd_recv_state)); multifd_recv_state->params =3D g_new0(MultiFDRecvParams, thread_count); atomic_set(&multifd_recv_state->count, 0); + for (i =3D 0; i < thread_count; i++) { MultiFDRecvParams *p =3D &multifd_recv_state->params[i]; =20 @@ -857,6 +913,7 @@ int multifd_load_setup(void) qemu_sem_init(&p->sem, 0); p->quit =3D false; p->id =3D i; + p->pages =3D multifd_pages_init(page_count); p->name =3D g_strdup_printf("multifdrecv_%d", i); } return 0; --=20 2.17.1 From nobody Sat Apr 27 21:21:01 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (208.118.235.17 [208.118.235.17]) by mx.zohomail.com with SMTPS id 1529537634640884.0180609796208; Wed, 20 Jun 2018 16:33:54 -0700 (PDT) Received: from localhost ([::1]:52176 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fVmbf-0004ey-Le for importer@patchew.org; Wed, 20 Jun 2018 19:33:43 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:48362) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fVmX8-0001a2-Nt for qemu-devel@nongnu.org; Wed, 20 Jun 2018 19:29:04 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fVmX7-0007Bs-95 for qemu-devel@nongnu.org; Wed, 20 Jun 2018 19:29:02 -0400 Received: from mx3-rdu2.redhat.com ([66.187.233.73]:53754 helo=mx1.redhat.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1fVmX7-0007BZ-3B for qemu-devel@nongnu.org; Wed, 20 Jun 2018 19:29:01 -0400 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.rdu2.redhat.com [10.11.54.6]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id B3C38859C5 for ; Wed, 20 Jun 2018 23:29:00 +0000 (UTC) Received: from secure.mitica (ovpn-116-118.ams2.redhat.com [10.36.116.118]) by smtp.corp.redhat.com (Postfix) with ESMTP id A89A52156880; Wed, 20 Jun 2018 23:28:59 +0000 (UTC) From: Juan Quintela To: qemu-devel@nongnu.org Date: Thu, 21 Jun 2018 01:28:41 +0200 Message-Id: <20180620232851.7152-3-quintela@redhat.com> In-Reply-To: <20180620232851.7152-1-quintela@redhat.com> References: <20180620232851.7152-1-quintela@redhat.com> X-Scanned-By: MIMEDefang 2.78 on 10.11.54.6 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.2]); Wed, 20 Jun 2018 23:29:00 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.2]); Wed, 20 Jun 2018 23:29:00 +0000 (UTC) for IP:'10.11.54.6' DOMAIN:'int-mx06.intmail.prod.int.rdu2.redhat.com' HELO:'smtp.corp.redhat.com' FROM:'quintela@redhat.com' RCPT:'' X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 66.187.233.73 Subject: [Qemu-devel] [PATCH v15 02/12] migration: Create multifd packet X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: lvivier@redhat.com, dgilbert@redhat.com, peterx@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" We still don't put anything there. Signed-off-by: Juan Quintela Reviewed-by: Juan Quintela -- fix magic (dave) check offset/ramblock (dave) s/seq/packet_num/ and make it 64bit --- migration/ram.c | 145 +++++++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 144 insertions(+), 1 deletion(-) diff --git a/migration/ram.c b/migration/ram.c index ed4401ee46..6504b492da 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -517,6 +517,17 @@ typedef struct { uint8_t id; } __attribute__((packed)) MultiFDInit_t; =20 +typedef struct { + uint32_t magic; + uint32_t version; + uint32_t flags; + uint32_t size; + uint32_t used; + uint64_t packet_num; + char ramblock[256]; + uint64_t offset[]; +} __attribute__((packed)) MultiFDPacket_t; + typedef struct { /* number of used pages */ uint32_t used; @@ -551,6 +562,14 @@ typedef struct { bool quit; /* array of pages to sent */ MultiFDPages_t *pages; + /* packet allocated len */ + uint32_t packet_len; + /* pointer to the packet */ + MultiFDPacket_t *packet; + /* multifd flags for each packet */ + uint32_t flags; + /* global number of generated multifd packets */ + uint64_t packet_num; } MultiFDSendParams; =20 typedef struct { @@ -573,6 +592,14 @@ typedef struct { bool quit; /* array of pages to receive */ MultiFDPages_t *pages; + /* packet allocated len */ + uint32_t packet_len; + /* pointer to the packet */ + MultiFDPacket_t *packet; + /* multifd flags for each packet */ + uint32_t flags; + /* global number of generated multifd packets */ + uint64_t packet_num; } MultiFDRecvParams; =20 static int multifd_send_initial_packet(MultiFDSendParams *p, Error **errp) @@ -661,6 +688,99 @@ static void multifd_pages_clear(MultiFDPages_t *pages) g_free(pages); } =20 +static void multifd_send_fill_packet(MultiFDSendParams *p) +{ + MultiFDPacket_t *packet =3D p->packet; + int i; + + packet->magic =3D cpu_to_be32(MULTIFD_MAGIC); + packet->version =3D cpu_to_be32(MULTIFD_VERSION); + packet->flags =3D cpu_to_be32(p->flags); + packet->size =3D cpu_to_be32(migrate_multifd_page_count()); + packet->used =3D cpu_to_be32(p->pages->used); + packet->packet_num =3D cpu_to_be64(p->packet_num); + + if (p->pages->block) { + strncpy(packet->ramblock, p->pages->block->idstr, 256); + } + + for (i =3D 0; i < p->pages->used; i++) { + packet->offset[i] =3D cpu_to_be64(p->pages->offset[i]); + } +} + +static int multifd_recv_unfill_packet(MultiFDRecvParams *p, Error **errp) +{ + MultiFDPacket_t *packet =3D p->packet; + RAMBlock *block; + int i; + + /* ToDo: We can't use it until we haven't received a message */ + return 0; + + be32_to_cpus(&packet->magic); + if (packet->magic !=3D MULTIFD_MAGIC) { + error_setg(errp, "multifd: received packet " + "magic %x and expected magic %x", + packet->magic, MULTIFD_MAGIC); + return -1; + } + + be32_to_cpus(&packet->version); + if (packet->version !=3D MULTIFD_VERSION) { + error_setg(errp, "multifd: received packet " + "version %d and expected version %d", + packet->version, MULTIFD_VERSION); + return -1; + } + + p->flags =3D be32_to_cpu(packet->flags); + + be32_to_cpus(&packet->size); + if (packet->size > migrate_multifd_page_count()) { + error_setg(errp, "multifd: received packet " + "with size %d and expected maximum size %d", + packet->size, migrate_multifd_page_count()) ; + return -1; + } + + p->pages->used =3D be32_to_cpu(packet->used); + if (p->pages->used > packet->size) { + error_setg(errp, "multifd: received packet " + "with size %d and expected maximum size %d", + p->pages->used, packet->size) ; + return -1; + } + + p->packet_num =3D be64_to_cpu(packet->packet_num); + + if (p->pages->used) { + /* make sure that ramblock is 0 terminated */ + packet->ramblock[255] =3D 0; + block =3D qemu_ram_block_by_name(packet->ramblock); + if (!block) { + error_setg(errp, "multifd: unknown ram block %s", + packet->ramblock); + return -1; + } + } + + for (i =3D 0; i < p->pages->used; i++) { + ram_addr_t offset =3D be64_to_cpu(packet->offset[i]); + + if (offset > (block->used_length - TARGET_PAGE_SIZE)) { + error_setg(errp, "multifd: offset too long %" PRId64 + " (max %" PRId64 ")", + offset, block->max_length); + return -1; + } + p->pages->iov[i].iov_base =3D block->host + offset; + p->pages->iov[i].iov_len =3D TARGET_PAGE_SIZE; + } + + return 0; +} + struct { MultiFDSendParams *params; /* number of created threads */ @@ -718,6 +838,9 @@ int multifd_save_cleanup(Error **errp) p->name =3D NULL; multifd_pages_clear(p->pages); p->pages =3D NULL; + p->packet_len =3D 0; + g_free(p->packet); + p->packet =3D NULL; } g_free(multifd_send_state->params); multifd_send_state->params =3D NULL; @@ -739,6 +862,7 @@ static void *multifd_send_thread(void *opaque) =20 while (true) { qemu_mutex_lock(&p->mutex); + multifd_send_fill_packet(p); if (p->quit) { qemu_mutex_unlock(&p->mutex); break; @@ -803,6 +927,9 @@ int multifd_save_setup(void) p->quit =3D false; p->id =3D i; p->pages =3D multifd_pages_init(page_count); + p->packet_len =3D sizeof(MultiFDPacket_t) + + sizeof(ram_addr_t) * page_count; + p->packet =3D g_malloc0(p->packet_len); p->name =3D g_strdup_printf("multifdsend_%d", i); socket_send_channel_create(multifd_new_send_channel_async, p); } @@ -862,6 +989,9 @@ int multifd_load_cleanup(Error **errp) p->name =3D NULL; multifd_pages_clear(p->pages); p->pages =3D NULL; + p->packet_len =3D 0; + g_free(p->packet); + p->packet =3D NULL; } g_free(multifd_recv_state->params); multifd_recv_state->params =3D NULL; @@ -874,10 +1004,20 @@ int multifd_load_cleanup(Error **errp) static void *multifd_recv_thread(void *opaque) { MultiFDRecvParams *p =3D opaque; + Error *local_err =3D NULL; + int ret; =20 while (true) { qemu_mutex_lock(&p->mutex); - if (p->quit) { + if (false) { + /* ToDo: Packet reception goes here */ + + ret =3D multifd_recv_unfill_packet(p, &local_err); + qemu_mutex_unlock(&p->mutex); + if (ret) { + break; + } + } else if (p->quit) { qemu_mutex_unlock(&p->mutex); break; } @@ -914,6 +1054,9 @@ int multifd_load_setup(void) p->quit =3D false; p->id =3D i; p->pages =3D multifd_pages_init(page_count); + p->packet_len =3D sizeof(MultiFDPacket_t) + + sizeof(ram_addr_t) * page_count; + p->packet =3D g_malloc0(p->packet_len); p->name =3D g_strdup_printf("multifdrecv_%d", i); } return 0; --=20 2.17.1 From nobody Sat Apr 27 21:21:01 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (208.118.235.17 [208.118.235.17]) by mx.zohomail.com with SMTPS id 1529537632210285.90567109129427; Wed, 20 Jun 2018 16:33:52 -0700 (PDT) Received: from localhost ([::1]:52177 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fVmbi-0004gl-0o for importer@patchew.org; Wed, 20 Jun 2018 19:33:46 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:48379) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fVmX9-0001a4-KT for qemu-devel@nongnu.org; Wed, 20 Jun 2018 19:29:04 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fVmX8-0007Cb-Kv for qemu-devel@nongnu.org; Wed, 20 Jun 2018 19:29:03 -0400 Received: from mx3-rdu2.redhat.com ([66.187.233.73]:59388 helo=mx1.redhat.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1fVmX8-0007CL-G5 for qemu-devel@nongnu.org; Wed, 20 Jun 2018 19:29:02 -0400 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.rdu2.redhat.com [10.11.54.6]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 0C2B1401EF03 for ; Wed, 20 Jun 2018 23:29:02 +0000 (UTC) Received: from secure.mitica (ovpn-116-118.ams2.redhat.com [10.36.116.118]) by smtp.corp.redhat.com (Postfix) with ESMTP id F377E2156880; Wed, 20 Jun 2018 23:29:00 +0000 (UTC) From: Juan Quintela To: qemu-devel@nongnu.org Date: Thu, 21 Jun 2018 01:28:42 +0200 Message-Id: <20180620232851.7152-4-quintela@redhat.com> In-Reply-To: <20180620232851.7152-1-quintela@redhat.com> References: <20180620232851.7152-1-quintela@redhat.com> X-Scanned-By: MIMEDefang 2.78 on 10.11.54.6 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.5]); Wed, 20 Jun 2018 23:29:02 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.5]); Wed, 20 Jun 2018 23:29:02 +0000 (UTC) for IP:'10.11.54.6' DOMAIN:'int-mx06.intmail.prod.int.rdu2.redhat.com' HELO:'smtp.corp.redhat.com' FROM:'quintela@redhat.com' RCPT:'' X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 66.187.233.73 Subject: [Qemu-devel] [PATCH v15 03/12] migration: Add multifd traces for start/end thread X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: lvivier@redhat.com, dgilbert@redhat.com, peterx@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" We want to know how many pages/packets each channel has sent. Add counters for those. Signed-off-by: Juan Quintela Reviewed-by: Juan Quintela -- sort trace-events (dave) --- migration/ram.c | 22 ++++++++++++++++++++++ migration/trace-events | 4 ++++ 2 files changed, 26 insertions(+) diff --git a/migration/ram.c b/migration/ram.c index 6504b492da..d146689d3a 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -570,6 +570,11 @@ typedef struct { uint32_t flags; /* global number of generated multifd packets */ uint64_t packet_num; + /* thread local variables */ + /* packets sent through this channel */ + uint32_t num_packets; + /* pages sent through this channel */ + uint32_t num_pages; } MultiFDSendParams; =20 typedef struct { @@ -600,6 +605,11 @@ typedef struct { uint32_t flags; /* global number of generated multifd packets */ uint64_t packet_num; + /* thread local variables */ + /* packets sent through this channel */ + uint32_t num_packets; + /* pages sent through this channel */ + uint32_t num_pages; } MultiFDRecvParams; =20 static int multifd_send_initial_packet(MultiFDSendParams *p, Error **errp) @@ -856,9 +866,13 @@ static void *multifd_send_thread(void *opaque) MultiFDSendParams *p =3D opaque; Error *local_err =3D NULL; =20 + trace_multifd_send_thread_start(p->id); + if (multifd_send_initial_packet(p, &local_err) < 0) { goto out; } + /* initial packet */ + p->num_packets =3D 1; =20 while (true) { qemu_mutex_lock(&p->mutex); @@ -880,6 +894,8 @@ out: p->running =3D false; qemu_mutex_unlock(&p->mutex); =20 + trace_multifd_send_thread_end(p->id, p->num_packets, p->num_pages); + return NULL; } =20 @@ -1007,6 +1023,8 @@ static void *multifd_recv_thread(void *opaque) Error *local_err =3D NULL; int ret; =20 + trace_multifd_recv_thread_start(p->id); + while (true) { qemu_mutex_lock(&p->mutex); if (false) { @@ -1029,6 +1047,8 @@ static void *multifd_recv_thread(void *opaque) p->running =3D false; qemu_mutex_unlock(&p->mutex); =20 + trace_multifd_recv_thread_end(p->id, p->num_packets, p->num_pages); + return NULL; } =20 @@ -1094,6 +1114,8 @@ void multifd_recv_new_channel(QIOChannel *ioc) } p->c =3D ioc; object_ref(OBJECT(ioc)); + /* initial packet */ + p->num_packets =3D 1; =20 p->running =3D true; qemu_thread_create(&p->thread, p->name, multifd_recv_thread, p, diff --git a/migration/trace-events b/migration/trace-events index 3f67758893..6d499448b3 100644 --- a/migration/trace-events +++ b/migration/trace-events @@ -76,6 +76,10 @@ get_queued_page_not_dirty(const char *block_name, uint64= _t tmp_offset, unsigned migration_bitmap_sync_start(void) "" migration_bitmap_sync_end(uint64_t dirty_pages) "dirty_pages %" PRIu64 migration_throttle(void) "" +multifd_recv_thread_end(uint8_t id, uint32_t packets, uint32_t pages) "cha= nnel %d packets %d pages %d" +multifd_recv_thread_start(uint8_t id) "%d" +multifd_send_thread_end(uint8_t id, uint32_t packets, uint32_t pages) "cha= nnel %d packets %d pages %d" +multifd_send_thread_start(uint8_t id) "%d" ram_discard_range(const char *rbname, uint64_t start, size_t len) "%s: sta= rt: %" PRIx64 " %zx" ram_load_loop(const char *rbname, uint64_t addr, int flags, void *host) "%= s: addr: 0x%" PRIx64 " flags: 0x%x host: %p" ram_load_postcopy_loop(uint64_t addr, int flags) "@%" PRIx64 " %x" --=20 2.17.1 From nobody Sat Apr 27 21:21:01 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 152953745848269.96960870635985; Wed, 20 Jun 2018 16:30:58 -0700 (PDT) Received: from localhost ([::1]:52167 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fVmYz-0002pN-Nu for importer@patchew.org; Wed, 20 Jun 2018 19:30:57 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:48394) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fVmXA-0001aL-SF for qemu-devel@nongnu.org; Wed, 20 Jun 2018 19:29:05 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fVmX9-0007DZ-S3 for qemu-devel@nongnu.org; Wed, 20 Jun 2018 19:29:04 -0400 Received: from mx3-rdu2.redhat.com ([66.187.233.73]:53756 helo=mx1.redhat.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1fVmX9-0007DE-O6 for qemu-devel@nongnu.org; Wed, 20 Jun 2018 19:29:03 -0400 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.rdu2.redhat.com [10.11.54.6]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 57333859C5 for ; Wed, 20 Jun 2018 23:29:03 +0000 (UTC) Received: from secure.mitica (ovpn-116-118.ams2.redhat.com [10.36.116.118]) by smtp.corp.redhat.com (Postfix) with ESMTP id 4CB5F2156880; Wed, 20 Jun 2018 23:29:02 +0000 (UTC) From: Juan Quintela To: qemu-devel@nongnu.org Date: Thu, 21 Jun 2018 01:28:43 +0200 Message-Id: <20180620232851.7152-5-quintela@redhat.com> In-Reply-To: <20180620232851.7152-1-quintela@redhat.com> References: <20180620232851.7152-1-quintela@redhat.com> X-Scanned-By: MIMEDefang 2.78 on 10.11.54.6 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.2]); Wed, 20 Jun 2018 23:29:03 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.2]); Wed, 20 Jun 2018 23:29:03 +0000 (UTC) for IP:'10.11.54.6' DOMAIN:'int-mx06.intmail.prod.int.rdu2.redhat.com' HELO:'smtp.corp.redhat.com' FROM:'quintela@redhat.com' RCPT:'' X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 66.187.233.73 Subject: [Qemu-devel] [PATCH v15 04/12] migration: Calculate transferred ram correctly X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: lvivier@redhat.com, dgilbert@redhat.com, peterx@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" On multifd we send data from more places that main channel. Signed-off-by: Juan Quintela -- Add placeholder for packets size --- migration/migration.c | 12 ++++++++++-- migration/ram.c | 7 +++++++ migration/ram.h | 1 + 3 files changed, 18 insertions(+), 2 deletions(-) diff --git a/migration/migration.c b/migration/migration.c index e1eaa97df4..224629533b 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -2728,12 +2728,20 @@ static void migration_update_counters(MigrationStat= e *s, { uint64_t transferred, time_spent; double bandwidth; + uint64_t now; =20 if (current_time < s->iteration_start_time + BUFFER_DELAY) { return; } =20 - transferred =3D qemu_ftell(s->to_dst_file) - s->iteration_initial_byte= s; + if (migrate_use_multifd()) { + now =3D ram_counters.normal * qemu_target_page_size() + + multifd_packets_size() + + qemu_ftell(s->to_dst_file); + } else { + now =3D qemu_ftell(s->to_dst_file); + } + transferred =3D now - s->iteration_initial_bytes; time_spent =3D current_time - s->iteration_start_time; bandwidth =3D (double)transferred / time_spent; s->threshold_size =3D bandwidth * s->parameters.downtime_limit; @@ -2752,7 +2760,7 @@ static void migration_update_counters(MigrationState = *s, qemu_file_reset_rate_limit(s->to_dst_file); =20 s->iteration_start_time =3D current_time; - s->iteration_initial_bytes =3D qemu_ftell(s->to_dst_file); + s->iteration_initial_bytes =3D now; =20 trace_migrate_transferred(transferred, time_spent, bandwidth, s->threshold_size); diff --git a/migration/ram.c b/migration/ram.c index d146689d3a..d7f8b0d989 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -952,6 +952,13 @@ int multifd_save_setup(void) return 0; } =20 +/* Size in bytes of the page headers */ +int multifd_packets_size(void) +{ + /* We are not yet sending any data through channels */ + return 0; +} + struct { MultiFDRecvParams *params; /* number of created threads */ diff --git a/migration/ram.h b/migration/ram.h index d386f4d641..e1decb7418 100644 --- a/migration/ram.h +++ b/migration/ram.h @@ -47,6 +47,7 @@ int multifd_load_setup(void); int multifd_load_cleanup(Error **errp); bool multifd_recv_all_channels_created(void); void multifd_recv_new_channel(QIOChannel *ioc); +int multifd_packets_size(void); =20 uint64_t ram_pagesize_summary(void); int ram_save_queue_pages(const char *rbname, ram_addr_t start, ram_addr_t = len); --=20 2.17.1 From nobody Sat Apr 27 21:21:01 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1529537791730139.4734622082026; Wed, 20 Jun 2018 16:36:31 -0700 (PDT) Received: from localhost ([::1]:52191 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fVmeM-0006o8-TN for importer@patchew.org; Wed, 20 Jun 2018 19:36:30 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:48411) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fVmXB-0001ao-W5 for qemu-devel@nongnu.org; Wed, 20 Jun 2018 19:29:06 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fVmXB-0007ED-5n for qemu-devel@nongnu.org; Wed, 20 Jun 2018 19:29:06 -0400 Received: from mx3-rdu2.redhat.com ([66.187.233.73]:46176 helo=mx1.redhat.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1fVmXB-0007Dx-1a for qemu-devel@nongnu.org; Wed, 20 Jun 2018 19:29:05 -0400 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.rdu2.redhat.com [10.11.54.6]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id A30374022909 for ; Wed, 20 Jun 2018 23:29:04 +0000 (UTC) Received: from secure.mitica (ovpn-116-118.ams2.redhat.com [10.36.116.118]) by smtp.corp.redhat.com (Postfix) with ESMTP id 98B2A2156880; Wed, 20 Jun 2018 23:29:03 +0000 (UTC) From: Juan Quintela To: qemu-devel@nongnu.org Date: Thu, 21 Jun 2018 01:28:44 +0200 Message-Id: <20180620232851.7152-6-quintela@redhat.com> In-Reply-To: <20180620232851.7152-1-quintela@redhat.com> References: <20180620232851.7152-1-quintela@redhat.com> X-Scanned-By: MIMEDefang 2.78 on 10.11.54.6 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.6]); Wed, 20 Jun 2018 23:29:04 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.6]); Wed, 20 Jun 2018 23:29:04 +0000 (UTC) for IP:'10.11.54.6' DOMAIN:'int-mx06.intmail.prod.int.rdu2.redhat.com' HELO:'smtp.corp.redhat.com' FROM:'quintela@redhat.com' RCPT:'' X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 66.187.233.73 Subject: [Qemu-devel] [PATCH v15 05/12] migration: Multifd channels always wait on the sem X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: lvivier@redhat.com, dgilbert@redhat.com, peterx@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Either for quit, sync or packet, we first wake them. Signed-off-by: Juan Quintela Reviewed-by: Dr. David Alan Gilbert --- migration/ram.c | 13 +++++++++++-- 1 file changed, 11 insertions(+), 2 deletions(-) diff --git a/migration/ram.c b/migration/ram.c index d7f8b0d989..617da76a2e 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -875,6 +875,7 @@ static void *multifd_send_thread(void *opaque) p->num_packets =3D 1; =20 while (true) { + qemu_sem_wait(&p->sem); qemu_mutex_lock(&p->mutex); multifd_send_fill_packet(p); if (p->quit) { @@ -882,7 +883,9 @@ static void *multifd_send_thread(void *opaque) break; } qemu_mutex_unlock(&p->mutex); - qemu_sem_wait(&p->sem); + /* this is impossible */ + error_setg(&local_err, "multifd_send_thread: Unknown command"); + break; } =20 out: @@ -1033,6 +1036,7 @@ static void *multifd_recv_thread(void *opaque) trace_multifd_recv_thread_start(p->id); =20 while (true) { + qemu_sem_wait(&p->sem); qemu_mutex_lock(&p->mutex); if (false) { /* ToDo: Packet reception goes here */ @@ -1047,9 +1051,14 @@ static void *multifd_recv_thread(void *opaque) break; } qemu_mutex_unlock(&p->mutex); - qemu_sem_wait(&p->sem); + /* this is impossible */ + error_setg(&local_err, "multifd_recv_thread: Unknown command"); + break; } =20 + if (local_err) { + multifd_recv_terminate_threads(local_err); + } qemu_mutex_lock(&p->mutex); p->running =3D false; qemu_mutex_unlock(&p->mutex); --=20 2.17.1 From nobody Sat Apr 27 21:21:01 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1529537634634709.3502147516077; Wed, 20 Jun 2018 16:33:54 -0700 (PDT) Received: from localhost ([::1]:52178 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fVmbp-0004kJ-Gb for importer@patchew.org; Wed, 20 Jun 2018 19:33:53 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:48422) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fVmXD-0001cI-Hg for qemu-devel@nongnu.org; Wed, 20 Jun 2018 19:29:08 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fVmXC-0007Ex-Gb for qemu-devel@nongnu.org; Wed, 20 Jun 2018 19:29:07 -0400 Received: from mx3-rdu2.redhat.com ([66.187.233.73]:53758 helo=mx1.redhat.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1fVmXC-0007Em-BO for qemu-devel@nongnu.org; Wed, 20 Jun 2018 19:29:06 -0400 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.rdu2.redhat.com [10.11.54.6]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id EC38B859C5 for ; Wed, 20 Jun 2018 23:29:05 +0000 (UTC) Received: from secure.mitica (ovpn-116-118.ams2.redhat.com [10.36.116.118]) by smtp.corp.redhat.com (Postfix) with ESMTP id E5B962156880; Wed, 20 Jun 2018 23:29:04 +0000 (UTC) From: Juan Quintela To: qemu-devel@nongnu.org Date: Thu, 21 Jun 2018 01:28:45 +0200 Message-Id: <20180620232851.7152-7-quintela@redhat.com> In-Reply-To: <20180620232851.7152-1-quintela@redhat.com> References: <20180620232851.7152-1-quintela@redhat.com> X-Scanned-By: MIMEDefang 2.78 on 10.11.54.6 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.2]); Wed, 20 Jun 2018 23:29:05 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.2]); Wed, 20 Jun 2018 23:29:05 +0000 (UTC) for IP:'10.11.54.6' DOMAIN:'int-mx06.intmail.prod.int.rdu2.redhat.com' HELO:'smtp.corp.redhat.com' FROM:'quintela@redhat.com' RCPT:'' X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 66.187.233.73 Subject: [Qemu-devel] [PATCH v15 06/12] migration: Add block where to send/receive packets X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: lvivier@redhat.com, dgilbert@redhat.com, peterx@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Once there add tracepoints. Signed-off-by: Juan Quintela Reviewed-by: Dr. David Alan Gilbert --- migration/ram.c | 49 +++++++++++++++++++++++++++++++++++++----- migration/trace-events | 2 ++ 2 files changed, 46 insertions(+), 5 deletions(-) diff --git a/migration/ram.c b/migration/ram.c index 617da76a2e..793f0dc5d3 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -560,6 +560,8 @@ typedef struct { bool running; /* should this thread finish */ bool quit; + /* thread has work to do */ + int pending_job; /* array of pages to sent */ MultiFDPages_t *pages; /* packet allocated len */ @@ -595,6 +597,8 @@ typedef struct { bool running; /* should this thread finish */ bool quit; + /* thread has work to do */ + bool pending_job; /* array of pages to receive */ MultiFDPages_t *pages; /* packet allocated len */ @@ -877,8 +881,28 @@ static void *multifd_send_thread(void *opaque) while (true) { qemu_sem_wait(&p->sem); qemu_mutex_lock(&p->mutex); - multifd_send_fill_packet(p); - if (p->quit) { + + if (p->pending_job) { + uint32_t used =3D p->pages->used; + uint32_t packet_num =3D p->packet_num; + uint32_t flags =3D p->flags; + + multifd_send_fill_packet(p); + p->flags =3D 0; + p->num_packets++; + p->num_pages +=3D used; + p->pages->used =3D 0; + qemu_mutex_unlock(&p->mutex); + + trace_multifd_send(p->id, packet_num, used, flags); + + /* ToDo: send packet here */ + + qemu_mutex_lock(&p->mutex); + p->pending_job--; + qemu_mutex_unlock(&p->mutex); + continue; + } else if (p->quit) { qemu_mutex_unlock(&p->mutex); break; } @@ -944,6 +968,7 @@ int multifd_save_setup(void) qemu_mutex_init(&p->mutex); qemu_sem_init(&p->sem, 0); p->quit =3D false; + p->pending_job =3D 0; p->id =3D i; p->pages =3D multifd_pages_init(page_count); p->packet_len =3D sizeof(MultiFDPacket_t) @@ -1038,14 +1063,27 @@ static void *multifd_recv_thread(void *opaque) while (true) { qemu_sem_wait(&p->sem); qemu_mutex_lock(&p->mutex); - if (false) { - /* ToDo: Packet reception goes here */ + if (p->pending_job) { + uint32_t used; + uint32_t flags; + qemu_mutex_unlock(&p->mutex); =20 + /* ToDo: recv packet here */ + + qemu_mutex_lock(&p->mutex); ret =3D multifd_recv_unfill_packet(p, &local_err); - qemu_mutex_unlock(&p->mutex); if (ret) { + qemu_mutex_unlock(&p->mutex); break; } + + used =3D p->pages->used; + flags =3D p->flags; + trace_multifd_recv(p->id, p->packet_num, used, flags); + p->pending_job =3D false; + p->num_packets++; + p->num_pages +=3D used; + qemu_mutex_unlock(&p->mutex); } else if (p->quit) { qemu_mutex_unlock(&p->mutex); break; @@ -1088,6 +1126,7 @@ int multifd_load_setup(void) qemu_mutex_init(&p->mutex); qemu_sem_init(&p->sem, 0); p->quit =3D false; + p->pending_job =3D false; p->id =3D i; p->pages =3D multifd_pages_init(page_count); p->packet_len =3D sizeof(MultiFDPacket_t) diff --git a/migration/trace-events b/migration/trace-events index 6d499448b3..c667d98529 100644 --- a/migration/trace-events +++ b/migration/trace-events @@ -76,8 +76,10 @@ get_queued_page_not_dirty(const char *block_name, uint64= _t tmp_offset, unsigned migration_bitmap_sync_start(void) "" migration_bitmap_sync_end(uint64_t dirty_pages) "dirty_pages %" PRIu64 migration_throttle(void) "" +multifd_recv(uint8_t id, uint64_t packet_num, uint32_t used, uint32_t flag= s) "channel %d packet number %ld pages %d flags 0x%x" multifd_recv_thread_end(uint8_t id, uint32_t packets, uint32_t pages) "cha= nnel %d packets %d pages %d" multifd_recv_thread_start(uint8_t id) "%d" +multifd_send(uint8_t id, uint64_t packet_num, uint32_t used, uint32_t flag= s) "channel %d packet_num %ld pages %d flags 0x%x" multifd_send_thread_end(uint8_t id, uint32_t packets, uint32_t pages) "cha= nnel %d packets %d pages %d" multifd_send_thread_start(uint8_t id) "%d" ram_discard_range(const char *rbname, uint64_t start, size_t len) "%s: sta= rt: %" PRIx64 " %zx" --=20 2.17.1 From nobody Sat Apr 27 21:21:01 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (208.118.235.17 [208.118.235.17]) by mx.zohomail.com with SMTPS id 1529537956143649.6733897470895; Wed, 20 Jun 2018 16:39:16 -0700 (PDT) Received: from localhost ([::1]:52199 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fVmgr-00009O-5W for importer@patchew.org; Wed, 20 Jun 2018 19:39:05 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:48435) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fVmXF-0001ds-EA for qemu-devel@nongnu.org; Wed, 20 Jun 2018 19:29:12 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fVmXD-0007FW-RS for qemu-devel@nongnu.org; Wed, 20 Jun 2018 19:29:09 -0400 Received: from mx3-rdu2.redhat.com ([66.187.233.73]:53760 helo=mx1.redhat.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1fVmXD-0007FI-LZ for qemu-devel@nongnu.org; Wed, 20 Jun 2018 19:29:07 -0400 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.rdu2.redhat.com [10.11.54.6]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 477A6859C5 for ; Wed, 20 Jun 2018 23:29:07 +0000 (UTC) Received: from secure.mitica (ovpn-116-118.ams2.redhat.com [10.36.116.118]) by smtp.corp.redhat.com (Postfix) with ESMTP id 3D7702156880; Wed, 20 Jun 2018 23:29:06 +0000 (UTC) From: Juan Quintela To: qemu-devel@nongnu.org Date: Thu, 21 Jun 2018 01:28:46 +0200 Message-Id: <20180620232851.7152-8-quintela@redhat.com> In-Reply-To: <20180620232851.7152-1-quintela@redhat.com> References: <20180620232851.7152-1-quintela@redhat.com> X-Scanned-By: MIMEDefang 2.78 on 10.11.54.6 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.2]); Wed, 20 Jun 2018 23:29:07 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.2]); Wed, 20 Jun 2018 23:29:07 +0000 (UTC) for IP:'10.11.54.6' DOMAIN:'int-mx06.intmail.prod.int.rdu2.redhat.com' HELO:'smtp.corp.redhat.com' FROM:'quintela@redhat.com' RCPT:'' X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 66.187.233.73 Subject: [Qemu-devel] [PATCH v15 07/12] migration: Synchronize multifd threads with main thread X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: lvivier@redhat.com, dgilbert@redhat.com, peterx@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" We synchronize all threads each RAM_SAVE_FLAG_EOS. Bitmap synchronizations don't happen inside a ram section, so we are safe about two channels trying to overwrite the same memory. Signed-off-by: Juan Quintela -- seq needs to be atomic now, will also be accessed from main thread. Fix the if (true || ...) leftover --- migration/ram.c | 147 ++++++++++++++++++++++++++++++++--------- migration/trace-events | 6 ++ 2 files changed, 122 insertions(+), 31 deletions(-) diff --git a/migration/ram.c b/migration/ram.c index 793f0dc5d3..516f347d24 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -510,6 +510,8 @@ exit: #define MULTIFD_MAGIC 0x11223344U #define MULTIFD_VERSION 1 =20 +#define MULTIFD_FLAG_SYNC (1 << 0) + typedef struct { uint32_t magic; uint32_t version; @@ -577,6 +579,8 @@ typedef struct { uint32_t num_packets; /* pages sent through this channel */ uint32_t num_pages; + /* syncs main thread and channels */ + QemuSemaphore sem_sync; } MultiFDSendParams; =20 typedef struct { @@ -614,6 +618,8 @@ typedef struct { uint32_t num_packets; /* pages sent through this channel */ uint32_t num_pages; + /* syncs main thread and channels */ + QemuSemaphore sem_sync; } MultiFDRecvParams; =20 static int multifd_send_initial_packet(MultiFDSendParams *p, Error **errp) @@ -801,6 +807,10 @@ struct { int count; /* array of pages to sent */ MultiFDPages_t *pages; + /* syncs main thread and channels */ + QemuSemaphore sem_sync; + /* global number of generated multifd packets */ + uint64_t packet_num; } *multifd_send_state; =20 static void multifd_send_terminate_threads(Error *err) @@ -848,6 +858,7 @@ int multifd_save_cleanup(Error **errp) p->c =3D NULL; qemu_mutex_destroy(&p->mutex); qemu_sem_destroy(&p->sem); + qemu_sem_destroy(&p->sem_sync); g_free(p->name); p->name =3D NULL; multifd_pages_clear(p->pages); @@ -856,6 +867,7 @@ int multifd_save_cleanup(Error **errp) g_free(p->packet); p->packet =3D NULL; } + qemu_sem_destroy(&multifd_send_state->sem_sync); g_free(multifd_send_state->params); multifd_send_state->params =3D NULL; multifd_pages_clear(multifd_send_state->pages); @@ -865,6 +877,33 @@ int multifd_save_cleanup(Error **errp) return ret; } =20 +static void multifd_send_sync_main(void) +{ + int i; + + if (!migrate_use_multifd()) { + return; + } + for (i =3D 0; i < migrate_multifd_channels(); i++) { + MultiFDSendParams *p =3D &multifd_send_state->params[i]; + + trace_multifd_send_sync_main_signal(p->id); + + qemu_mutex_lock(&p->mutex); + p->flags |=3D MULTIFD_FLAG_SYNC; + p->pending_job++; + qemu_mutex_unlock(&p->mutex); + qemu_sem_post(&p->sem); + } + for (i =3D 0; i < migrate_multifd_channels(); i++) { + MultiFDSendParams *p =3D &multifd_send_state->params[i]; + + trace_multifd_send_sync_main_wait(p->id); + qemu_sem_wait(&multifd_send_state->sem_sync); + } + trace_multifd_send_sync_main(atomic_read(&multifd_send_state->packet_n= um)); +} + static void *multifd_send_thread(void *opaque) { MultiFDSendParams *p =3D opaque; @@ -901,15 +940,17 @@ static void *multifd_send_thread(void *opaque) qemu_mutex_lock(&p->mutex); p->pending_job--; qemu_mutex_unlock(&p->mutex); - continue; + + if (flags & MULTIFD_FLAG_SYNC) { + qemu_sem_post(&multifd_send_state->sem_sync); + } } else if (p->quit) { qemu_mutex_unlock(&p->mutex); break; + } else { + qemu_mutex_unlock(&p->mutex); + /* sometimes there are spurious wakeups */ } - qemu_mutex_unlock(&p->mutex); - /* this is impossible */ - error_setg(&local_err, "multifd_send_thread: Unknown command"); - break; } =20 out: @@ -961,12 +1002,14 @@ int multifd_save_setup(void) multifd_send_state->params =3D g_new0(MultiFDSendParams, thread_count); atomic_set(&multifd_send_state->count, 0); multifd_send_state->pages =3D multifd_pages_init(page_count); + qemu_sem_init(&multifd_send_state->sem_sync, 0); =20 for (i =3D 0; i < thread_count; i++) { MultiFDSendParams *p =3D &multifd_send_state->params[i]; =20 qemu_mutex_init(&p->mutex); qemu_sem_init(&p->sem, 0); + qemu_sem_init(&p->sem_sync, 0); p->quit =3D false; p->pending_job =3D 0; p->id =3D i; @@ -991,6 +1034,10 @@ struct { MultiFDRecvParams *params; /* number of created threads */ int count; + /* syncs main thread and channels */ + QemuSemaphore sem_sync; + /* global number of generated multifd packets */ + uint64_t packet_num; } *multifd_recv_state; =20 static void multifd_recv_terminate_threads(Error *err) @@ -1036,6 +1083,7 @@ int multifd_load_cleanup(Error **errp) p->c =3D NULL; qemu_mutex_destroy(&p->mutex); qemu_sem_destroy(&p->sem); + qemu_sem_destroy(&p->sem_sync); g_free(p->name); p->name =3D NULL; multifd_pages_clear(p->pages); @@ -1044,6 +1092,7 @@ int multifd_load_cleanup(Error **errp) g_free(p->packet); p->packet =3D NULL; } + qemu_sem_destroy(&multifd_recv_state->sem_sync); g_free(multifd_recv_state->params); multifd_recv_state->params =3D NULL; g_free(multifd_recv_state); @@ -1052,6 +1101,42 @@ int multifd_load_cleanup(Error **errp) return ret; } =20 +static void multifd_recv_sync_main(void) +{ + int i; + + if (!migrate_use_multifd()) { + return; + } + for (i =3D 0; i < migrate_multifd_channels(); i++) { + MultiFDRecvParams *p =3D &multifd_recv_state->params[i]; + + trace_multifd_recv_sync_main_signal(p->id); + qemu_mutex_lock(&p->mutex); + p->pending_job =3D true; + qemu_mutex_unlock(&p->mutex); + } + for (i =3D 0; i < migrate_multifd_channels(); i++) { + MultiFDRecvParams *p =3D &multifd_recv_state->params[i]; + + trace_multifd_recv_sync_main_wait(p->id); + qemu_sem_wait(&multifd_recv_state->sem_sync); + qemu_mutex_lock(&p->mutex); + if (atomic_read(&multifd_recv_state->packet_num) < p->packet_num) { + atomic_set(&multifd_recv_state->packet_num, p->packet_num); + } + qemu_mutex_unlock(&p->mutex); + } + for (i =3D 0; i < migrate_multifd_channels(); i++) { + MultiFDRecvParams *p =3D &multifd_recv_state->params[i]; + + trace_multifd_recv_sync_main_signal(p->id); + + qemu_sem_post(&p->sem_sync); + } + trace_multifd_recv_sync_main(atomic_read(&multifd_recv_state->packet_n= um)); +} + static void *multifd_recv_thread(void *opaque) { MultiFDRecvParams *p =3D opaque; @@ -1061,37 +1146,30 @@ static void *multifd_recv_thread(void *opaque) trace_multifd_recv_thread_start(p->id); =20 while (true) { - qemu_sem_wait(&p->sem); + uint32_t used; + uint32_t flags; + + /* ToDo: recv packet here */ + qemu_mutex_lock(&p->mutex); - if (p->pending_job) { - uint32_t used; - uint32_t flags; - qemu_mutex_unlock(&p->mutex); - - /* ToDo: recv packet here */ - - qemu_mutex_lock(&p->mutex); - ret =3D multifd_recv_unfill_packet(p, &local_err); - if (ret) { - qemu_mutex_unlock(&p->mutex); - break; - } - - used =3D p->pages->used; - flags =3D p->flags; - trace_multifd_recv(p->id, p->packet_num, used, flags); - p->pending_job =3D false; - p->num_packets++; - p->num_pages +=3D used; - qemu_mutex_unlock(&p->mutex); - } else if (p->quit) { + ret =3D multifd_recv_unfill_packet(p, &local_err); + if (ret) { qemu_mutex_unlock(&p->mutex); break; } + + used =3D p->pages->used; + flags =3D p->flags; + trace_multifd_recv(p->id, p->packet_num, used, flags); + p->pending_job =3D false; + p->num_packets++; + p->num_pages +=3D used; qemu_mutex_unlock(&p->mutex); - /* this is impossible */ - error_setg(&local_err, "multifd_recv_thread: Unknown command"); - break; + + if (flags & MULTIFD_FLAG_SYNC) { + qemu_sem_post(&multifd_recv_state->sem_sync); + qemu_sem_wait(&p->sem_sync); + } } =20 if (local_err) { @@ -1119,12 +1197,14 @@ int multifd_load_setup(void) multifd_recv_state =3D g_malloc0(sizeof(*multifd_recv_state)); multifd_recv_state->params =3D g_new0(MultiFDRecvParams, thread_count); atomic_set(&multifd_recv_state->count, 0); + qemu_sem_init(&multifd_recv_state->sem_sync, 0); =20 for (i =3D 0; i < thread_count; i++) { MultiFDRecvParams *p =3D &multifd_recv_state->params[i]; =20 qemu_mutex_init(&p->mutex); qemu_sem_init(&p->sem, 0); + qemu_sem_init(&p->sem_sync, 0); p->quit =3D false; p->pending_job =3D false; p->id =3D i; @@ -2882,6 +2962,7 @@ static int ram_save_setup(QEMUFile *f, void *opaque) ram_control_before_iterate(f, RAM_CONTROL_SETUP); ram_control_after_iterate(f, RAM_CONTROL_SETUP); =20 + multifd_send_sync_main(); qemu_put_be64(f, RAM_SAVE_FLAG_EOS); =20 return 0; @@ -2962,6 +3043,7 @@ static int ram_save_iterate(QEMUFile *f, void *opaque) */ ram_control_after_iterate(f, RAM_CONTROL_ROUND); =20 + multifd_send_sync_main(); out: qemu_put_be64(f, RAM_SAVE_FLAG_EOS); ram_counters.transferred +=3D 8; @@ -3015,6 +3097,7 @@ static int ram_save_complete(QEMUFile *f, void *opaqu= e) =20 rcu_read_unlock(); =20 + multifd_send_sync_main(); qemu_put_be64(f, RAM_SAVE_FLAG_EOS); =20 return 0; @@ -3504,6 +3587,7 @@ static int ram_load_postcopy(QEMUFile *f) break; case RAM_SAVE_FLAG_EOS: /* normal exit */ + multifd_recv_sync_main(); break; default: error_report("Unknown combination of migration flags: %#x" @@ -3692,6 +3776,7 @@ static int ram_load(QEMUFile *f, void *opaque, int ve= rsion_id) break; case RAM_SAVE_FLAG_EOS: /* normal exit */ + multifd_recv_sync_main(); break; default: if (flags & RAM_SAVE_FLAG_HOOK) { diff --git a/migration/trace-events b/migration/trace-events index c667d98529..ea39bb2bc5 100644 --- a/migration/trace-events +++ b/migration/trace-events @@ -77,9 +77,15 @@ migration_bitmap_sync_start(void) "" migration_bitmap_sync_end(uint64_t dirty_pages) "dirty_pages %" PRIu64 migration_throttle(void) "" multifd_recv(uint8_t id, uint64_t packet_num, uint32_t used, uint32_t flag= s) "channel %d packet number %ld pages %d flags 0x%x" +multifd_recv_sync_main(uint32_t seq) "seq %d" +multifd_recv_sync_main_signal(uint8_t id) "channel %d" +multifd_recv_sync_main_wait(uint8_t id) "channel %d" multifd_recv_thread_end(uint8_t id, uint32_t packets, uint32_t pages) "cha= nnel %d packets %d pages %d" multifd_recv_thread_start(uint8_t id) "%d" multifd_send(uint8_t id, uint64_t packet_num, uint32_t used, uint32_t flag= s) "channel %d packet_num %ld pages %d flags 0x%x" +multifd_send_sync_main(uint32_t seq) "seq %d" +multifd_send_sync_main_signal(uint8_t id) "channel %d" +multifd_send_sync_main_wait(uint8_t id) "channel %d" multifd_send_thread_end(uint8_t id, uint32_t packets, uint32_t pages) "cha= nnel %d packets %d pages %d" multifd_send_thread_start(uint8_t id) "%d" ram_discard_range(const char *rbname, uint64_t start, size_t len) "%s: sta= rt: %" PRIx64 " %zx" --=20 2.17.1 From nobody Sat Apr 27 21:21:01 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1529537799831711.796146259839; Wed, 20 Jun 2018 16:36:39 -0700 (PDT) Received: from localhost ([::1]:52192 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fVmeU-0006st-Uo for importer@patchew.org; Wed, 20 Jun 2018 19:36:38 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:48457) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fVmXI-0001em-B0 for qemu-devel@nongnu.org; Wed, 20 Jun 2018 19:29:13 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fVmXF-0007Ft-5l for qemu-devel@nongnu.org; Wed, 20 Jun 2018 19:29:12 -0400 Received: from mx3-rdu2.redhat.com ([66.187.233.73]:46178 helo=mx1.redhat.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1fVmXE-0007Fm-W8 for qemu-devel@nongnu.org; Wed, 20 Jun 2018 19:29:09 -0400 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.rdu2.redhat.com [10.11.54.6]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 978664022909 for ; Wed, 20 Jun 2018 23:29:08 +0000 (UTC) Received: from secure.mitica (ovpn-116-118.ams2.redhat.com [10.36.116.118]) by smtp.corp.redhat.com (Postfix) with ESMTP id 8C5732156880; Wed, 20 Jun 2018 23:29:07 +0000 (UTC) From: Juan Quintela To: qemu-devel@nongnu.org Date: Thu, 21 Jun 2018 01:28:47 +0200 Message-Id: <20180620232851.7152-9-quintela@redhat.com> In-Reply-To: <20180620232851.7152-1-quintela@redhat.com> References: <20180620232851.7152-1-quintela@redhat.com> X-Scanned-By: MIMEDefang 2.78 on 10.11.54.6 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.6]); Wed, 20 Jun 2018 23:29:08 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.6]); Wed, 20 Jun 2018 23:29:08 +0000 (UTC) for IP:'10.11.54.6' DOMAIN:'int-mx06.intmail.prod.int.rdu2.redhat.com' HELO:'smtp.corp.redhat.com' FROM:'quintela@redhat.com' RCPT:'' X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 66.187.233.73 Subject: [Qemu-devel] [PATCH v15 08/12] migration: Create ram_save_multifd_page X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: lvivier@redhat.com, dgilbert@redhat.com, peterx@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" The function still don't use multifd, but we have simplified ram_save_page, xbzrle and RDMA stuff is gone. We have added a new counter. Signed-off-by: Juan Quintela Reviewed-by: Dr. David Alan Gilbert -- Add last_page parameter Add commets for done and address Remove multifd field, it is the same than normal pages Merge next patch, now we send multiple pages at a time Remove counter for multifd pages, it is identical to normal pages Use iovec's instead of creating the equivalent. Clear memory used by pages (dave) Use g_new0(danp) define MULTIFD_CONTINUE now pages member is a pointer Fix off-by-one in number of pages in one packet Remove RAM_SAVE_FLAG_MULTIFD_PAGE s/multifd_pages_t/MultiFDPages_t/ add comment explaining what it means --- migration/ram.c | 112 +++++++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 110 insertions(+), 2 deletions(-) diff --git a/migration/ram.c b/migration/ram.c index 516f347d24..71a33b73e7 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -55,6 +55,7 @@ #include "sysemu/sysemu.h" #include "qemu/uuid.h" #include "savevm.h" +#include "qemu/iov.h" =20 /***********************************************************/ /* ram save/restore */ @@ -811,8 +812,83 @@ struct { QemuSemaphore sem_sync; /* global number of generated multifd packets */ uint64_t packet_num; + /* send channels ready */ + QemuSemaphore channels_ready; } *multifd_send_state; =20 +/* + * How we use multifd_send_state->pages and channel->pages? + * + * We create a pages for each channel, and a main one. Each time that + * we need to send a batch of pages we interchange the ones between + * multifd_send_state and the channel that is sending it. There are + * two reasons for that: + * - to not have to do so many mallocs during migration + * - to make easier to know what to free at the end of migration + * + * This way we always know who is the owner of each "pages" struct, + * and we don't need any loocking. It belongs to the migration thread + * or to the channel thread. Switching is safe because the migration + * thread is using the channel mutex when changing it, and the channel + * have to had finish with its own, otherwise pending_job can't be + * false. + */ + +static void multifd_send_pages(void) +{ + int i; + static int next_channel; + MultiFDSendParams *p =3D NULL; /* make happy gcc */ + MultiFDPages_t *pages =3D multifd_send_state->pages; + + qemu_sem_wait(&multifd_send_state->channels_ready); + for (i =3D next_channel;; i =3D (i + 1) % migrate_multifd_channels()) { + p =3D &multifd_send_state->params[i]; + + qemu_mutex_lock(&p->mutex); + if (!p->pending_job) { + p->pending_job++; + next_channel =3D (i + 1) % migrate_multifd_channels(); + break; + } + qemu_mutex_unlock(&p->mutex); + } + p->pages->used =3D 0; + + p->packet_num =3D atomic_inc_fetch(&multifd_send_state->packet_num); + p->pages->block =3D NULL; + multifd_send_state->pages =3D p->pages; + p->pages =3D pages; + qemu_mutex_unlock(&p->mutex); + qemu_sem_post(&p->sem); +} + +static void multifd_queue_page(RAMBlock *block, ram_addr_t offset) +{ + MultiFDPages_t *pages =3D multifd_send_state->pages; + + if (!pages->block) { + pages->block =3D block; + } + + if (pages->block =3D=3D block) { + pages->offset[pages->used] =3D offset; + pages->iov[pages->used].iov_base =3D block->host + offset; + pages->iov[pages->used].iov_len =3D TARGET_PAGE_SIZE; + pages->used++; + + if (pages->used < pages->allocated) { + return; + } + } + + multifd_send_pages(); + + if (pages->block !=3D block) { + multifd_queue_page(block, offset); + } +} + static void multifd_send_terminate_threads(Error *err) { int i; @@ -867,6 +943,7 @@ int multifd_save_cleanup(Error **errp) g_free(p->packet); p->packet =3D NULL; } + qemu_sem_destroy(&multifd_send_state->channels_ready); qemu_sem_destroy(&multifd_send_state->sem_sync); g_free(multifd_send_state->params); multifd_send_state->params =3D NULL; @@ -884,12 +961,17 @@ static void multifd_send_sync_main(void) if (!migrate_use_multifd()) { return; } + if (multifd_send_state->pages->used) { + multifd_send_pages(); + } for (i =3D 0; i < migrate_multifd_channels(); i++) { MultiFDSendParams *p =3D &multifd_send_state->params[i]; =20 trace_multifd_send_sync_main_signal(p->id); =20 qemu_mutex_lock(&p->mutex); + + p->packet_num =3D atomic_inc_fetch(&multifd_send_state->packet_num= ); p->flags |=3D MULTIFD_FLAG_SYNC; p->pending_job++; qemu_mutex_unlock(&p->mutex); @@ -944,6 +1026,7 @@ static void *multifd_send_thread(void *opaque) if (flags & MULTIFD_FLAG_SYNC) { qemu_sem_post(&multifd_send_state->sem_sync); } + qemu_sem_post(&multifd_send_state->channels_ready); } else if (p->quit) { qemu_mutex_unlock(&p->mutex); break; @@ -1003,6 +1086,7 @@ int multifd_save_setup(void) atomic_set(&multifd_send_state->count, 0); multifd_send_state->pages =3D multifd_pages_init(page_count); qemu_sem_init(&multifd_send_state->sem_sync, 0); + qemu_sem_init(&multifd_send_state->channels_ready, 0); =20 for (i =3D 0; i < thread_count; i++) { MultiFDSendParams *p =3D &multifd_send_state->params[i]; @@ -1026,8 +1110,13 @@ int multifd_save_setup(void) /* Size in bytes of the page headers */ int multifd_packets_size(void) { - /* We are not yet sending any data through channels */ - return 0; + int packet_num =3D atomic_read(&multifd_send_state->packet_num); + int packet_size =3D sizeof(MultiFDPacket_t); + + /* Add sincronization packets */ + packet_num +=3D ram_counters.dirty_sync_count * migrate_multifd_channe= ls(); + + return packet_num * packet_size; } =20 struct { @@ -1731,6 +1820,23 @@ static int ram_save_page(RAMState *rs, PageSearchSta= tus *pss, bool last_stage) return pages; } =20 +static int ram_save_multifd_page(RAMState *rs, RAMBlock *block, + ram_addr_t offset) +{ + uint8_t *p; + + p =3D block->host + offset; + + ram_counters.transferred +=3D save_page_header(rs, rs->f, block, + offset | RAM_SAVE_FLAG_PA= GE); + multifd_queue_page(block, offset); + qemu_put_buffer(rs->f, p, TARGET_PAGE_SIZE); + ram_counters.transferred +=3D TARGET_PAGE_SIZE; + ram_counters.normal++; + + return 1; +} + static int do_compress_ram_page(QEMUFile *f, z_stream *stream, RAMBlock *b= lock, ram_addr_t offset, uint8_t *source_buf) { @@ -2136,6 +2242,8 @@ static int ram_save_target_page(RAMState *rs, PageSea= rchStatus *pss, */ if (block =3D=3D rs->last_sent_block && save_page_use_compression(rs))= { return compress_page_with_multi_thread(rs, block, offset); + } else if (migrate_use_multifd()) { + return ram_save_multifd_page(rs, block, offset); } =20 return ram_save_page(rs, pss, last_stage); --=20 2.17.1 From nobody Sat Apr 27 21:21:01 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1529537888355553.9314768467544; Wed, 20 Jun 2018 16:38:08 -0700 (PDT) Received: from localhost ([::1]:52198 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fVmfv-0007v1-JP for importer@patchew.org; Wed, 20 Jun 2018 19:38:07 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:48456) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fVmXI-0001el-Ad for qemu-devel@nongnu.org; Wed, 20 Jun 2018 19:29:13 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fVmXG-0007GL-FN for qemu-devel@nongnu.org; Wed, 20 Jun 2018 19:29:12 -0400 Received: from mx3-rdu2.redhat.com ([66.187.233.73]:42502 helo=mx1.redhat.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1fVmXG-0007GA-B3 for qemu-devel@nongnu.org; Wed, 20 Jun 2018 19:29:10 -0400 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.rdu2.redhat.com [10.11.54.6]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id E6A9E40250FF for ; Wed, 20 Jun 2018 23:29:09 +0000 (UTC) Received: from secure.mitica (ovpn-116-118.ams2.redhat.com [10.36.116.118]) by smtp.corp.redhat.com (Postfix) with ESMTP id DE3C72156880; Wed, 20 Jun 2018 23:29:08 +0000 (UTC) From: Juan Quintela To: qemu-devel@nongnu.org Date: Thu, 21 Jun 2018 01:28:48 +0200 Message-Id: <20180620232851.7152-10-quintela@redhat.com> In-Reply-To: <20180620232851.7152-1-quintela@redhat.com> References: <20180620232851.7152-1-quintela@redhat.com> X-Scanned-By: MIMEDefang 2.78 on 10.11.54.6 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.7]); Wed, 20 Jun 2018 23:29:09 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.7]); Wed, 20 Jun 2018 23:29:09 +0000 (UTC) for IP:'10.11.54.6' DOMAIN:'int-mx06.intmail.prod.int.rdu2.redhat.com' HELO:'smtp.corp.redhat.com' FROM:'quintela@redhat.com' RCPT:'' X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 66.187.233.73 Subject: [Qemu-devel] [PATCH v15 09/12] migration: Start sending messages X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: lvivier@redhat.com, dgilbert@redhat.com, peterx@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Signed-off-by: Juan Quintela Reviewed-by: Dr. David Alan Gilbert --- migration/ram.c | 29 ++++++++++++++++++++++++----- 1 file changed, 24 insertions(+), 5 deletions(-) diff --git a/migration/ram.c b/migration/ram.c index 71a33b73e7..09df573441 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -736,9 +736,6 @@ static int multifd_recv_unfill_packet(MultiFDRecvParams= *p, Error **errp) RAMBlock *block; int i; =20 - /* ToDo: We can't use it until we haven't received a message */ - return 0; - be32_to_cpus(&packet->magic); if (packet->magic !=3D MULTIFD_MAGIC) { error_setg(errp, "multifd: received packet " @@ -990,6 +987,7 @@ static void *multifd_send_thread(void *opaque) { MultiFDSendParams *p =3D opaque; Error *local_err =3D NULL; + int ret; =20 trace_multifd_send_thread_start(p->id); =20 @@ -1017,7 +1015,16 @@ static void *multifd_send_thread(void *opaque) =20 trace_multifd_send(p->id, packet_num, used, flags); =20 - /* ToDo: send packet here */ + ret =3D qio_channel_write_all(p->c, (void *)p->packet, + p->packet_len, &local_err); + if (ret !=3D 0) { + break; + } + + ret =3D qio_channel_writev_all(p->c, p->pages->iov, used, &loc= al_err); + if (ret !=3D 0) { + break; + } =20 qemu_mutex_lock(&p->mutex); p->pending_job--; @@ -1238,7 +1245,14 @@ static void *multifd_recv_thread(void *opaque) uint32_t used; uint32_t flags; =20 - /* ToDo: recv packet here */ + ret =3D qio_channel_read_all_eof(p->c, (void *)p->packet, + p->packet_len, &local_err); + if (ret =3D=3D 0) { /* EOF */ + break; + } + if (ret =3D=3D -1) { /* Error */ + break; + } =20 qemu_mutex_lock(&p->mutex); ret =3D multifd_recv_unfill_packet(p, &local_err); @@ -1255,6 +1269,11 @@ static void *multifd_recv_thread(void *opaque) p->num_pages +=3D used; qemu_mutex_unlock(&p->mutex); =20 + ret =3D qio_channel_readv_all(p->c, p->pages->iov, used, &local_er= r); + if (ret !=3D 0) { + break; + } + if (flags & MULTIFD_FLAG_SYNC) { qemu_sem_post(&multifd_recv_state->sem_sync); qemu_sem_wait(&p->sem_sync); --=20 2.17.1 From nobody Sat Apr 27 21:21:01 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (208.118.235.17 [208.118.235.17]) by mx.zohomail.com with SMTPS id 1529537956057843.1802569535909; Wed, 20 Jun 2018 16:39:16 -0700 (PDT) Received: from localhost ([::1]:52200 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fVmgu-0000Df-90 for importer@patchew.org; Wed, 20 Jun 2018 19:39:08 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:48460) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fVmXI-0001en-La for qemu-devel@nongnu.org; Wed, 20 Jun 2018 19:29:13 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fVmXH-0007H5-P6 for qemu-devel@nongnu.org; Wed, 20 Jun 2018 19:29:12 -0400 Received: from mx3-rdu2.redhat.com ([66.187.233.73]:46182 helo=mx1.redhat.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1fVmXH-0007Gq-K9 for qemu-devel@nongnu.org; Wed, 20 Jun 2018 19:29:11 -0400 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.rdu2.redhat.com [10.11.54.6]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 3D1674022905 for ; Wed, 20 Jun 2018 23:29:11 +0000 (UTC) Received: from secure.mitica (ovpn-116-118.ams2.redhat.com [10.36.116.118]) by smtp.corp.redhat.com (Postfix) with ESMTP id 3271E2156880; Wed, 20 Jun 2018 23:29:10 +0000 (UTC) From: Juan Quintela To: qemu-devel@nongnu.org Date: Thu, 21 Jun 2018 01:28:49 +0200 Message-Id: <20180620232851.7152-11-quintela@redhat.com> In-Reply-To: <20180620232851.7152-1-quintela@redhat.com> References: <20180620232851.7152-1-quintela@redhat.com> X-Scanned-By: MIMEDefang 2.78 on 10.11.54.6 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.6]); Wed, 20 Jun 2018 23:29:11 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.6]); Wed, 20 Jun 2018 23:29:11 +0000 (UTC) for IP:'10.11.54.6' DOMAIN:'int-mx06.intmail.prod.int.rdu2.redhat.com' HELO:'smtp.corp.redhat.com' FROM:'quintela@redhat.com' RCPT:'' X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 66.187.233.73 Subject: [Qemu-devel] [PATCH v15 10/12] migration: Wait for blocking IO X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: lvivier@redhat.com, dgilbert@redhat.com, peterx@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" We have three conditions here: - channel fails -> error - we have to quit: we close the channel and reads fails - normal read that success, we are in bussiness So forget the complications of waiting in a semaphore. Signed-off-by: Juan Quintela Reviewed-by: Dr. David Alan Gilbert --- migration/ram.c | 13 ------------- 1 file changed, 13 deletions(-) diff --git a/migration/ram.c b/migration/ram.c index 09df573441..2c3a452a7d 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -602,8 +602,6 @@ typedef struct { bool running; /* should this thread finish */ bool quit; - /* thread has work to do */ - bool pending_job; /* array of pages to receive */ MultiFDPages_t *pages; /* packet allocated len */ @@ -1207,14 +1205,6 @@ static void multifd_recv_sync_main(void) for (i =3D 0; i < migrate_multifd_channels(); i++) { MultiFDRecvParams *p =3D &multifd_recv_state->params[i]; =20 - trace_multifd_recv_sync_main_signal(p->id); - qemu_mutex_lock(&p->mutex); - p->pending_job =3D true; - qemu_mutex_unlock(&p->mutex); - } - for (i =3D 0; i < migrate_multifd_channels(); i++) { - MultiFDRecvParams *p =3D &multifd_recv_state->params[i]; - trace_multifd_recv_sync_main_wait(p->id); qemu_sem_wait(&multifd_recv_state->sem_sync); qemu_mutex_lock(&p->mutex); @@ -1227,7 +1217,6 @@ static void multifd_recv_sync_main(void) MultiFDRecvParams *p =3D &multifd_recv_state->params[i]; =20 trace_multifd_recv_sync_main_signal(p->id); - qemu_sem_post(&p->sem_sync); } trace_multifd_recv_sync_main(atomic_read(&multifd_recv_state->packet_n= um)); @@ -1264,7 +1253,6 @@ static void *multifd_recv_thread(void *opaque) used =3D p->pages->used; flags =3D p->flags; trace_multifd_recv(p->id, p->packet_num, used, flags); - p->pending_job =3D false; p->num_packets++; p->num_pages +=3D used; qemu_mutex_unlock(&p->mutex); @@ -1314,7 +1302,6 @@ int multifd_load_setup(void) qemu_sem_init(&p->sem, 0); qemu_sem_init(&p->sem_sync, 0); p->quit =3D false; - p->pending_job =3D false; p->id =3D i; p->pages =3D multifd_pages_init(page_count); p->packet_len =3D sizeof(MultiFDPacket_t) --=20 2.17.1 From nobody Sat Apr 27 21:21:01 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1529538032399625.2638346378205; Wed, 20 Jun 2018 16:40:32 -0700 (PDT) Received: from localhost ([::1]:52210 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fVmiF-0001dI-OY for importer@patchew.org; Wed, 20 Jun 2018 19:40:31 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:48482) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fVmXJ-0001gK-VF for qemu-devel@nongnu.org; Wed, 20 Jun 2018 19:29:14 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fVmXJ-0007Hj-3G for qemu-devel@nongnu.org; Wed, 20 Jun 2018 19:29:14 -0400 Received: from mx3-rdu2.redhat.com ([66.187.233.73]:46184 helo=mx1.redhat.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1fVmXI-0007HX-Uz for qemu-devel@nongnu.org; Wed, 20 Jun 2018 19:29:13 -0400 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.rdu2.redhat.com [10.11.54.6]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 879F64022905 for ; Wed, 20 Jun 2018 23:29:12 +0000 (UTC) Received: from secure.mitica (ovpn-116-118.ams2.redhat.com [10.36.116.118]) by smtp.corp.redhat.com (Postfix) with ESMTP id 7F14F2156880; Wed, 20 Jun 2018 23:29:11 +0000 (UTC) From: Juan Quintela To: qemu-devel@nongnu.org Date: Thu, 21 Jun 2018 01:28:50 +0200 Message-Id: <20180620232851.7152-12-quintela@redhat.com> In-Reply-To: <20180620232851.7152-1-quintela@redhat.com> References: <20180620232851.7152-1-quintela@redhat.com> X-Scanned-By: MIMEDefang 2.78 on 10.11.54.6 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.6]); Wed, 20 Jun 2018 23:29:12 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.6]); Wed, 20 Jun 2018 23:29:12 +0000 (UTC) for IP:'10.11.54.6' DOMAIN:'int-mx06.intmail.prod.int.rdu2.redhat.com' HELO:'smtp.corp.redhat.com' FROM:'quintela@redhat.com' RCPT:'' X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 66.187.233.73 Subject: [Qemu-devel] [PATCH v15 11/12] migration: Remove not needed semaphore and quit X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: lvivier@redhat.com, dgilbert@redhat.com, peterx@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" We know quit closing the QIO. Signed-off-by: Juan Quintela -- Add comment don't object_ref() twice. --- migration/ram.c | 17 ++++++----------- 1 file changed, 6 insertions(+), 11 deletions(-) diff --git a/migration/ram.c b/migration/ram.c index 2c3a452a7d..57d3ad1c45 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -594,14 +594,10 @@ typedef struct { QemuThread thread; /* communication channel */ QIOChannel *c; - /* sem where to wait for more work */ - QemuSemaphore sem; /* this mutex protects the following parameters */ QemuMutex mutex; /* is this channel thread running */ bool running; - /* should this thread finish */ - bool quit; /* array of pages to receive */ MultiFDPages_t *pages; /* packet allocated len */ @@ -1152,8 +1148,12 @@ static void multifd_recv_terminate_threads(Error *er= r) MultiFDRecvParams *p =3D &multifd_recv_state->params[i]; =20 qemu_mutex_lock(&p->mutex); - p->quit =3D true; - qemu_sem_post(&p->sem); + /* We could arrive here for two reasons: + - normal quit, i.e. everything went fine, just finished + - error quit: We close the channels so the channel threads + finish the qio_channel_read_all_eof() */ + object_unref(OBJECT(p->c)); + p->c =3D NULL; qemu_mutex_unlock(&p->mutex); } } @@ -1173,10 +1173,7 @@ int multifd_load_cleanup(Error **errp) if (p->running) { qemu_thread_join(&p->thread); } - object_unref(OBJECT(p->c)); - p->c =3D NULL; qemu_mutex_destroy(&p->mutex); - qemu_sem_destroy(&p->sem); qemu_sem_destroy(&p->sem_sync); g_free(p->name); p->name =3D NULL; @@ -1299,9 +1296,7 @@ int multifd_load_setup(void) MultiFDRecvParams *p =3D &multifd_recv_state->params[i]; =20 qemu_mutex_init(&p->mutex); - qemu_sem_init(&p->sem, 0); qemu_sem_init(&p->sem_sync, 0); - p->quit =3D false; p->id =3D i; p->pages =3D multifd_pages_init(page_count); p->packet_len =3D sizeof(MultiFDPacket_t) --=20 2.17.1 From nobody Sat Apr 27 21:21:01 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1529538105795182.71655782525738; Wed, 20 Jun 2018 16:41:45 -0700 (PDT) Received: from localhost ([::1]:52213 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fVmjR-0002Dv-4U for importer@patchew.org; Wed, 20 Jun 2018 19:41:45 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:48498) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fVmXL-0001hY-5z for qemu-devel@nongnu.org; Wed, 20 Jun 2018 19:29:16 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fVmXK-0007IW-Bd for qemu-devel@nongnu.org; Wed, 20 Jun 2018 19:29:15 -0400 Received: from mx3-rdu2.redhat.com ([66.187.233.73]:53768 helo=mx1.redhat.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1fVmXK-0007IE-7U for qemu-devel@nongnu.org; Wed, 20 Jun 2018 19:29:14 -0400 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.rdu2.redhat.com [10.11.54.6]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id D6F911698B for ; Wed, 20 Jun 2018 23:29:13 +0000 (UTC) Received: from secure.mitica (ovpn-116-118.ams2.redhat.com [10.36.116.118]) by smtp.corp.redhat.com (Postfix) with ESMTP id CD37B2156880; Wed, 20 Jun 2018 23:29:12 +0000 (UTC) From: Juan Quintela To: qemu-devel@nongnu.org Date: Thu, 21 Jun 2018 01:28:51 +0200 Message-Id: <20180620232851.7152-13-quintela@redhat.com> In-Reply-To: <20180620232851.7152-1-quintela@redhat.com> References: <20180620232851.7152-1-quintela@redhat.com> X-Scanned-By: MIMEDefang 2.78 on 10.11.54.6 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.2]); Wed, 20 Jun 2018 23:29:13 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.2]); Wed, 20 Jun 2018 23:29:13 +0000 (UTC) for IP:'10.11.54.6' DOMAIN:'int-mx06.intmail.prod.int.rdu2.redhat.com' HELO:'smtp.corp.redhat.com' FROM:'quintela@redhat.com' RCPT:'' X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 66.187.233.73 Subject: [Qemu-devel] [PATCH v15 12/12] migration: Stop sending whole pages through main channel X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: lvivier@redhat.com, dgilbert@redhat.com, peterx@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" We have to flush() the QEMUFile because now we sent really few data through that channel. Signed-off-by: Juan Quintela Reviewed-by: Dr. David Alan Gilbert --- migration/ram.c | 11 +++-------- 1 file changed, 3 insertions(+), 8 deletions(-) diff --git a/migration/ram.c b/migration/ram.c index 57d3ad1c45..6d0782623c 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -1824,15 +1824,7 @@ static int ram_save_page(RAMState *rs, PageSearchSta= tus *pss, bool last_stage) static int ram_save_multifd_page(RAMState *rs, RAMBlock *block, ram_addr_t offset) { - uint8_t *p; - - p =3D block->host + offset; - - ram_counters.transferred +=3D save_page_header(rs, rs->f, block, - offset | RAM_SAVE_FLAG_PA= GE); multifd_queue_page(block, offset); - qemu_put_buffer(rs->f, p, TARGET_PAGE_SIZE); - ram_counters.transferred +=3D TARGET_PAGE_SIZE; ram_counters.normal++; =20 return 1; @@ -3073,6 +3065,7 @@ static int ram_save_setup(QEMUFile *f, void *opaque) =20 multifd_send_sync_main(); qemu_put_be64(f, RAM_SAVE_FLAG_EOS); + qemu_fflush(f); =20 return 0; } @@ -3155,6 +3148,7 @@ static int ram_save_iterate(QEMUFile *f, void *opaque) multifd_send_sync_main(); out: qemu_put_be64(f, RAM_SAVE_FLAG_EOS); + qemu_fflush(f); ram_counters.transferred +=3D 8; =20 ret =3D qemu_file_get_error(f); @@ -3208,6 +3202,7 @@ static int ram_save_complete(QEMUFile *f, void *opaqu= e) =20 multifd_send_sync_main(); qemu_put_be64(f, RAM_SAVE_FLAG_EOS); + qemu_fflush(f); =20 return 0; } --=20 2.17.1