From nobody Wed May 15 06:08:33 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1652222902; cv=none; d=zohomail.com; s=zohoarc; b=NMWZW+blchgMUtEBtKUTv5+Oj/PBGr1g5AxTW8qwpbYoJ3ldNFSG9l2eiLA7Dk3OZj6QFYFvOsyIhgRREaVnw8TXuW5CImYwfAd4XH5+H4UAIb1sGN57Abh+519D9HgAyo/imYynGpmUqGz4xUVcJlOQABswY2lzgGlmZIuMYHg= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1652222902; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=stlpYdjoY1E0s2U959zM4zGk2+1jhMuLun8jHEno+aI=; b=n/sjsU4O30Zfzy4+OwlvsYjtRx2zIYJHmNN1no/6yYca0GdvOQsDA+yOP3uROS1u1oNbfwyhprChagrsRd/GFAg7kZ+pCitdptQh02i8Vfhnv/RqBM6qAJlnYDumGjo7CWLXNxCVO9IUFOd2EViWZHG4gSFvAQwLtFQALRLuNCU= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1652222902335969.7667286659824; Tue, 10 May 2022 15:48:22 -0700 (PDT) Received: from localhost ([::1]:48666 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1noYeT-0003lW-8o for importer@patchew.org; Tue, 10 May 2022 18:48:21 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:47238) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1noYZu-0004Mj-Eo for qemu-devel@nongnu.org; Tue, 10 May 2022 18:43:38 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]:26729) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1noYZr-0006IE-JR for qemu-devel@nongnu.org; Tue, 10 May 2022 18:43:37 -0400 Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-490-AN1rdLkHPbyesVzwAK4s9w-1; Tue, 10 May 2022 18:42:26 -0400 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.rdu2.redhat.com [10.11.54.1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 254C485A5BC; Tue, 10 May 2022 22:42:26 +0000 (UTC) Received: from secure.mitica (unknown [10.39.195.247]) by smtp.corp.redhat.com (Postfix) with ESMTP id 969FD40CFD06; Tue, 10 May 2022 22:42:24 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1652222613; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=stlpYdjoY1E0s2U959zM4zGk2+1jhMuLun8jHEno+aI=; b=H4GIea4ew8LX5UM2t/M7PVD0V7UXZ2ig/7aFOYzfkYtKTg9fUoEQ858LoHCz78DCK+WQS/ Ov8MviUouhFvwjSg1Gqtbm8lfLl7uTWkI2UjnP4JqQZyhet//CTRX08b7qQBCN3MP0F0ja 3jvJVnE4xfrKjih8UaW6oSp9lOW8poc= X-MC-Unique: AN1rdLkHPbyesVzwAK4s9w-1 From: Juan Quintela To: qemu-devel@nongnu.org Cc: Juan Quintela , "Dr. David Alan Gilbert" , Eduardo Habkost , Peter Xu , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , Yanan Wang , Leonardo Bras , Marcel Apfelbaum Subject: [PATCH v6 01/13] multifd: Document the locking of MultiFD{Send/Recv}Params Date: Wed, 11 May 2022 00:42:08 +0200 Message-Id: <20220510224220.5912-2-quintela@redhat.com> In-Reply-To: <20220510224220.5912-1-quintela@redhat.com> References: <20220510224220.5912-1-quintela@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 2.84 on 10.11.54.1 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=quintela@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -28 X-Spam_score: -2.9 X-Spam_bar: -- X-Spam_report: (-2.9 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.082, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1652222903026100001 Content-Type: text/plain; charset="utf-8" Reorder the structures so we can know if the fields are: - Read only - Their own locking (i.e. sems) - Protected by 'mutex' - Only for the multifd channel Signed-off-by: Juan Quintela --- migration/multifd.h | 86 +++++++++++++++++++++++++++------------------ 1 file changed, 51 insertions(+), 35 deletions(-) diff --git a/migration/multifd.h b/migration/multifd.h index 7d0effcb03..f1f88c6737 100644 --- a/migration/multifd.h +++ b/migration/multifd.h @@ -65,7 +65,9 @@ typedef struct { } MultiFDPages_t; =20 typedef struct { - /* this fields are not changed once the thread is created */ + /* Fiields are only written at creating/deletion time */ + /* No lock required for them, they are read only */ + /* channel number */ uint8_t id; /* channel thread name */ @@ -74,37 +76,45 @@ typedef struct { QemuThread thread; /* communication channel */ QIOChannel *c; - /* sem where to wait for more work */ - QemuSemaphore sem; - /* this mutex protects the following parameters */ - QemuMutex mutex; - /* is this channel thread running */ - bool running; - /* should this thread finish */ - bool quit; /* is the yank function registered */ bool registered_yank; + /* packet allocated len */ + uint32_t packet_len; + + /* sem where to wait for more work */ + QemuSemaphore sem; + /* syncs main thread and channels */ + QemuSemaphore sem_sync; + + /* this mutex protects the following parameters */ + QemuMutex mutex; + /* is this channel thread running */ + bool running; + /* should this thread finish */ + bool quit; + /* multifd flags for each packet */ + uint32_t flags; + /* global number of generated multifd packets */ + uint64_t packet_num; /* thread has work to do */ int pending_job; - /* array of pages to sent */ + /* array of pages to sent. + * The owner of 'pages' depends of 'pending_job' value: + * pending_job =3D=3D 0 -> migration_thread can use it. + * pending_job !=3D 0 -> multifd_channel can use it. + */ MultiFDPages_t *pages; - /* packet allocated len */ - uint32_t packet_len; + + /* thread local variables. No locking required */ + /* pointer to the packet */ MultiFDPacket_t *packet; - /* multifd flags for each packet */ - uint32_t flags; /* size of the next packet that contains pages */ uint32_t next_packet_size; - /* global number of generated multifd packets */ - uint64_t packet_num; - /* thread local variables */ /* packets sent through this channel */ uint64_t num_packets; /* non zero pages sent through this channel */ uint64_t total_normal_pages; - /* syncs main thread and channels */ - QemuSemaphore sem_sync; /* buffers to send */ struct iovec *iov; /* number of iovs used */ @@ -118,7 +128,9 @@ typedef struct { } MultiFDSendParams; =20 typedef struct { - /* this fields are not changed once the thread is created */ + /* Fiields are only written at creating/deletion time */ + /* No lock required for them, they are read only */ + /* channel number */ uint8_t id; /* channel thread name */ @@ -127,31 +139,35 @@ typedef struct { QemuThread thread; /* communication channel */ QIOChannel *c; + /* packet allocated len */ + uint32_t packet_len; + + /* syncs main thread and channels */ + QemuSemaphore sem_sync; + /* this mutex protects the following parameters */ QemuMutex mutex; /* is this channel thread running */ bool running; /* should this thread finish */ bool quit; + /* multifd flags for each packet */ + uint32_t flags; + /* global number of generated multifd packets */ + uint64_t packet_num; + + /* thread local variables. No locking required */ + + /* pointer to the packet */ + MultiFDPacket_t *packet; + /* size of the next packet that contains pages */ + uint32_t next_packet_size; + /* packets sent through this channel */ + uint64_t num_packets; /* ramblock host address */ uint8_t *host; - /* packet allocated len */ - uint32_t packet_len; - /* pointer to the packet */ - MultiFDPacket_t *packet; - /* multifd flags for each packet */ - uint32_t flags; - /* global number of generated multifd packets */ - uint64_t packet_num; - /* thread local variables */ - /* size of the next packet that contains pages */ - uint32_t next_packet_size; - /* packets sent through this channel */ - uint64_t num_packets; /* non zero pages recv through this channel */ uint64_t total_normal_pages; - /* syncs main thread and channels */ - QemuSemaphore sem_sync; /* buffers to recv */ struct iovec *iov; /* Pages that are not zero */ --=20 2.35.1 From nobody Wed May 15 06:08:33 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1652223403566523.1399607215619; Tue, 10 May 2022 15:56:43 -0700 (PDT) Received: from localhost ([::1]:39300 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1noYmX-00006t-Ru for importer@patchew.org; Tue, 10 May 2022 18:56:41 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:47424) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1noYaM-0004dd-NX for qemu-devel@nongnu.org; Tue, 10 May 2022 18:44:08 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:27322) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1noYaH-0006JP-Un for qemu-devel@nongnu.org; Tue, 10 May 2022 18:44:06 -0400 Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-656-BcGZ2wRsMX21gmDDLkJEww-1; Tue, 10 May 2022 18:42:28 -0400 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.rdu2.redhat.com [10.11.54.1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 1BEDE101AA44; Tue, 10 May 2022 22:42:28 +0000 (UTC) Received: from secure.mitica (unknown [10.39.195.247]) by smtp.corp.redhat.com (Postfix) with ESMTP id 6C0A940CF8E7; Tue, 10 May 2022 22:42:26 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1652222641; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=FoKoBbw1eaYF5iZSMP7bLau+7FbeZUUKkDEVJILBG3U=; b=a4K2JylAP31f4rCuaZrrgYzbzcHjBfxggsSRNjWNftn/25oq/q2O+BEDolUd9FgzZrcxot s4Un4BOdXlqy9GWP6E4LpeV7D9+ByQ2wrMNC7Y29m1xWlbNKy/nVHolmMjwLR0gviZafaA 4JzuoQw27yZ95jz3MQSippA7DMqIPwg= X-MC-Unique: BcGZ2wRsMX21gmDDLkJEww-1 From: Juan Quintela To: qemu-devel@nongnu.org Cc: Juan Quintela , "Dr. David Alan Gilbert" , Eduardo Habkost , Peter Xu , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , Yanan Wang , Leonardo Bras , Marcel Apfelbaum Subject: [PATCH v6 02/13] multifd: Create page_size fields into both MultiFD{Recv, Send}Params Date: Wed, 11 May 2022 00:42:09 +0200 Message-Id: <20220510224220.5912-3-quintela@redhat.com> In-Reply-To: <20220510224220.5912-1-quintela@redhat.com> References: <20220510224220.5912-1-quintela@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 2.84 on 10.11.54.1 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=quintela@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -28 X-Spam_score: -2.9 X-Spam_bar: -- X-Spam_report: (-2.9 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.082, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZM-MESSAGEID: 1652223406181100001 Content-Type: text/plain; charset="utf-8" We were calling qemu_target_page_size() left and right. Signed-off-by: Juan Quintela --- migration/multifd.h | 4 ++++ migration/multifd-zlib.c | 12 +++++------- migration/multifd-zstd.c | 12 +++++------- migration/multifd.c | 18 ++++++++---------- 4 files changed, 22 insertions(+), 24 deletions(-) diff --git a/migration/multifd.h b/migration/multifd.h index f1f88c6737..4de80d9e53 100644 --- a/migration/multifd.h +++ b/migration/multifd.h @@ -80,6 +80,8 @@ typedef struct { bool registered_yank; /* packet allocated len */ uint32_t packet_len; + /* guest page size */ + uint32_t page_size; =20 /* sem where to wait for more work */ QemuSemaphore sem; @@ -141,6 +143,8 @@ typedef struct { QIOChannel *c; /* packet allocated len */ uint32_t packet_len; + /* guest page size */ + uint32_t page_size; =20 /* syncs main thread and channels */ QemuSemaphore sem_sync; diff --git a/migration/multifd-zlib.c b/migration/multifd-zlib.c index 3a7ae44485..28349ff2e0 100644 --- a/migration/multifd-zlib.c +++ b/migration/multifd-zlib.c @@ -100,7 +100,6 @@ static void zlib_send_cleanup(MultiFDSendParams *p, Err= or **errp) static int zlib_send_prepare(MultiFDSendParams *p, Error **errp) { struct zlib_data *z =3D p->data; - size_t page_size =3D qemu_target_page_size(); z_stream *zs =3D &z->zs; uint32_t out_size =3D 0; int ret; @@ -114,7 +113,7 @@ static int zlib_send_prepare(MultiFDSendParams *p, Erro= r **errp) flush =3D Z_SYNC_FLUSH; } =20 - zs->avail_in =3D page_size; + zs->avail_in =3D p->page_size; zs->next_in =3D p->pages->block->host + p->normal[i]; =20 zs->avail_out =3D available; @@ -220,12 +219,11 @@ static void zlib_recv_cleanup(MultiFDRecvParams *p) static int zlib_recv_pages(MultiFDRecvParams *p, Error **errp) { struct zlib_data *z =3D p->data; - size_t page_size =3D qemu_target_page_size(); z_stream *zs =3D &z->zs; uint32_t in_size =3D p->next_packet_size; /* we measure the change of total_out */ uint32_t out_size =3D zs->total_out; - uint32_t expected_size =3D p->normal_num * page_size; + uint32_t expected_size =3D p->normal_num * p->page_size; uint32_t flags =3D p->flags & MULTIFD_FLAG_COMPRESSION_MASK; int ret; int i; @@ -252,7 +250,7 @@ static int zlib_recv_pages(MultiFDRecvParams *p, Error = **errp) flush =3D Z_SYNC_FLUSH; } =20 - zs->avail_out =3D page_size; + zs->avail_out =3D p->page_size; zs->next_out =3D p->host + p->normal[i]; =20 /* @@ -266,8 +264,8 @@ static int zlib_recv_pages(MultiFDRecvParams *p, Error = **errp) do { ret =3D inflate(zs, flush); } while (ret =3D=3D Z_OK && zs->avail_in - && (zs->total_out - start) < page_size); - if (ret =3D=3D Z_OK && (zs->total_out - start) < page_size) { + && (zs->total_out - start) < p->page_size); + if (ret =3D=3D Z_OK && (zs->total_out - start) < p->page_size) { error_setg(errp, "multifd %u: inflate generated too few output= ", p->id); return -1; diff --git a/migration/multifd-zstd.c b/migration/multifd-zstd.c index d788d309f2..f4a8e1ed1f 100644 --- a/migration/multifd-zstd.c +++ b/migration/multifd-zstd.c @@ -113,7 +113,6 @@ static void zstd_send_cleanup(MultiFDSendParams *p, Err= or **errp) static int zstd_send_prepare(MultiFDSendParams *p, Error **errp) { struct zstd_data *z =3D p->data; - size_t page_size =3D qemu_target_page_size(); int ret; uint32_t i; =20 @@ -128,7 +127,7 @@ static int zstd_send_prepare(MultiFDSendParams *p, Erro= r **errp) flush =3D ZSTD_e_flush; } z->in.src =3D p->pages->block->host + p->normal[i]; - z->in.size =3D page_size; + z->in.size =3D p->page_size; z->in.pos =3D 0; =20 /* @@ -241,8 +240,7 @@ static int zstd_recv_pages(MultiFDRecvParams *p, Error = **errp) { uint32_t in_size =3D p->next_packet_size; uint32_t out_size =3D 0; - size_t page_size =3D qemu_target_page_size(); - uint32_t expected_size =3D p->normal_num * page_size; + uint32_t expected_size =3D p->normal_num * p->page_size; uint32_t flags =3D p->flags & MULTIFD_FLAG_COMPRESSION_MASK; struct zstd_data *z =3D p->data; int ret; @@ -265,7 +263,7 @@ static int zstd_recv_pages(MultiFDRecvParams *p, Error = **errp) =20 for (i =3D 0; i < p->normal_num; i++) { z->out.dst =3D p->host + p->normal[i]; - z->out.size =3D page_size; + z->out.size =3D p->page_size; z->out.pos =3D 0; =20 /* @@ -279,8 +277,8 @@ static int zstd_recv_pages(MultiFDRecvParams *p, Error = **errp) do { ret =3D ZSTD_decompressStream(z->zds, &z->out, &z->in); } while (ret > 0 && (z->in.size - z->in.pos > 0) - && (z->out.pos < page_size)); - if (ret > 0 && (z->out.pos < page_size)) { + && (z->out.pos < p->page_size)); + if (ret > 0 && (z->out.pos < p->page_size)) { error_setg(errp, "multifd %u: decompressStream buffer too smal= l", p->id); return -1; diff --git a/migration/multifd.c b/migration/multifd.c index 9ea4f581e2..f15fed5f1f 100644 --- a/migration/multifd.c +++ b/migration/multifd.c @@ -87,15 +87,14 @@ static void nocomp_send_cleanup(MultiFDSendParams *p, E= rror **errp) static int nocomp_send_prepare(MultiFDSendParams *p, Error **errp) { MultiFDPages_t *pages =3D p->pages; - size_t page_size =3D qemu_target_page_size(); =20 for (int i =3D 0; i < p->normal_num; i++) { p->iov[p->iovs_num].iov_base =3D pages->block->host + p->normal[i]; - p->iov[p->iovs_num].iov_len =3D page_size; + p->iov[p->iovs_num].iov_len =3D p->page_size; p->iovs_num++; } =20 - p->next_packet_size =3D p->normal_num * page_size; + p->next_packet_size =3D p->normal_num * p->page_size; p->flags |=3D MULTIFD_FLAG_NOCOMP; return 0; } @@ -139,7 +138,6 @@ static void nocomp_recv_cleanup(MultiFDRecvParams *p) static int nocomp_recv_pages(MultiFDRecvParams *p, Error **errp) { uint32_t flags =3D p->flags & MULTIFD_FLAG_COMPRESSION_MASK; - size_t page_size =3D qemu_target_page_size(); =20 if (flags !=3D MULTIFD_FLAG_NOCOMP) { error_setg(errp, "multifd %u: flags received %x flags expected %x", @@ -148,7 +146,7 @@ static int nocomp_recv_pages(MultiFDRecvParams *p, Erro= r **errp) } for (int i =3D 0; i < p->normal_num; i++) { p->iov[i].iov_base =3D p->host + p->normal[i]; - p->iov[i].iov_len =3D page_size; + p->iov[i].iov_len =3D p->page_size; } return qio_channel_readv_all(p->c, p->iov, p->normal_num, errp); } @@ -281,8 +279,7 @@ static void multifd_send_fill_packet(MultiFDSendParams = *p) static int multifd_recv_unfill_packet(MultiFDRecvParams *p, Error **errp) { MultiFDPacket_t *packet =3D p->packet; - size_t page_size =3D qemu_target_page_size(); - uint32_t page_count =3D MULTIFD_PACKET_SIZE / page_size; + uint32_t page_count =3D MULTIFD_PACKET_SIZE / p->page_size; RAMBlock *block; int i; =20 @@ -344,7 +341,7 @@ static int multifd_recv_unfill_packet(MultiFDRecvParams= *p, Error **errp) for (i =3D 0; i < p->normal_num; i++) { uint64_t offset =3D be64_to_cpu(packet->offset[i]); =20 - if (offset > (block->used_length - page_size)) { + if (offset > (block->used_length - p->page_size)) { error_setg(errp, "multifd: offset too long %" PRIu64 " (max " RAM_ADDR_FMT ")", offset, block->used_length); @@ -433,8 +430,7 @@ static int multifd_send_pages(QEMUFile *f) p->packet_num =3D multifd_send_state->packet_num++; multifd_send_state->pages =3D p->pages; p->pages =3D pages; - transferred =3D ((uint64_t) pages->num) * qemu_target_page_size() - + p->packet_len; + transferred =3D ((uint64_t) pages->num) * p->page_size + p->packet_len; qemu_file_update_transfer(f, transferred); ram_counters.multifd_bytes +=3D transferred; ram_counters.transferred +=3D transferred; @@ -898,6 +894,7 @@ int multifd_save_setup(Error **errp) /* We need one extra place for the packet header */ p->iov =3D g_new0(struct iovec, page_count + 1); p->normal =3D g_new0(ram_addr_t, page_count); + p->page_size =3D qemu_target_page_size(); socket_send_channel_create(multifd_new_send_channel_async, p); } =20 @@ -1138,6 +1135,7 @@ int multifd_load_setup(Error **errp) p->name =3D g_strdup_printf("multifdrecv_%d", i); p->iov =3D g_new0(struct iovec, page_count); p->normal =3D g_new0(ram_addr_t, page_count); + p->page_size =3D qemu_target_page_size(); } =20 for (i =3D 0; i < thread_count; i++) { --=20 2.35.1 From nobody Wed May 15 06:08:33 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1652222763228637.8037992273436; Tue, 10 May 2022 15:46:03 -0700 (PDT) Received: from localhost ([::1]:40202 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1noYcC-0006Nm-Vd for importer@patchew.org; Tue, 10 May 2022 18:46:01 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:47252) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1noYZu-0004Mr-TT for qemu-devel@nongnu.org; Tue, 10 May 2022 18:43:38 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]:20712) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1noYZt-0006II-8K for qemu-devel@nongnu.org; Tue, 10 May 2022 18:43:38 -0400 Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-101-4nkwDxyZOpWQOzgCsQcAZA-1; Tue, 10 May 2022 18:42:30 -0400 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.rdu2.redhat.com [10.11.54.1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id E53FD185A7B2; Tue, 10 May 2022 22:42:29 +0000 (UTC) Received: from secure.mitica (unknown [10.39.195.247]) by smtp.corp.redhat.com (Postfix) with ESMTP id 6347640CFD06; Tue, 10 May 2022 22:42:28 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1652222616; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=5lW4wy8pzw4gzn9OhK1jLGcZYO+lKbnNhX/fIXtY35Q=; b=VfKGpt5y3q4tO3nVmy4GNCWUjIA7ZATQQ2f2aMKrWgRm7LUXS7BaHR0aFlInXpDL1kf0SX SdC35/gXjUXlSH9iSN6p3X0wcbBQA51fkzcuje/Gp6fT5iZTzja5+ec1jefIKqa7Nd4FqY gGnmrtWs+ejldud4/HvCpE0vbbYzkR4= X-MC-Unique: 4nkwDxyZOpWQOzgCsQcAZA-1 From: Juan Quintela To: qemu-devel@nongnu.org Cc: Juan Quintela , "Dr. David Alan Gilbert" , Eduardo Habkost , Peter Xu , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , Yanan Wang , Leonardo Bras , Marcel Apfelbaum Subject: [PATCH v6 03/13] multifd: Create page_count fields into both MultiFD{Recv, Send}Params Date: Wed, 11 May 2022 00:42:10 +0200 Message-Id: <20220510224220.5912-4-quintela@redhat.com> In-Reply-To: <20220510224220.5912-1-quintela@redhat.com> References: <20220510224220.5912-1-quintela@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 2.84 on 10.11.54.1 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=quintela@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -28 X-Spam_score: -2.9 X-Spam_bar: -- X-Spam_report: (-2.9 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.082, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZM-MESSAGEID: 1652222765017100001 Content-Type: text/plain; charset="utf-8" We were recalculating it left and right. We plan to change that values on next patches. Signed-off-by: Juan Quintela --- migration/multifd.h | 4 ++++ migration/multifd.c | 7 ++++--- 2 files changed, 8 insertions(+), 3 deletions(-) diff --git a/migration/multifd.h b/migration/multifd.h index 4de80d9e53..f707e2a8b8 100644 --- a/migration/multifd.h +++ b/migration/multifd.h @@ -82,6 +82,8 @@ typedef struct { uint32_t packet_len; /* guest page size */ uint32_t page_size; + /* number of pages in a full packet */ + uint32_t page_count; =20 /* sem where to wait for more work */ QemuSemaphore sem; @@ -145,6 +147,8 @@ typedef struct { uint32_t packet_len; /* guest page size */ uint32_t page_size; + /* number of pages in a full packet */ + uint32_t page_count; =20 /* syncs main thread and channels */ QemuSemaphore sem_sync; diff --git a/migration/multifd.c b/migration/multifd.c index f15fed5f1f..893b90072d 100644 --- a/migration/multifd.c +++ b/migration/multifd.c @@ -279,7 +279,6 @@ static void multifd_send_fill_packet(MultiFDSendParams = *p) static int multifd_recv_unfill_packet(MultiFDRecvParams *p, Error **errp) { MultiFDPacket_t *packet =3D p->packet; - uint32_t page_count =3D MULTIFD_PACKET_SIZE / p->page_size; RAMBlock *block; int i; =20 @@ -306,10 +305,10 @@ static int multifd_recv_unfill_packet(MultiFDRecvPara= ms *p, Error **errp) * If we received a packet that is 100 times bigger than expected * just stop migration. It is a magic number. */ - if (packet->pages_alloc > page_count) { + if (packet->pages_alloc > p->page_count) { error_setg(errp, "multifd: received packet " "with size %u and expected a size of %u", - packet->pages_alloc, page_count) ; + packet->pages_alloc, p->page_count) ; return -1; } =20 @@ -895,6 +894,7 @@ int multifd_save_setup(Error **errp) p->iov =3D g_new0(struct iovec, page_count + 1); p->normal =3D g_new0(ram_addr_t, page_count); p->page_size =3D qemu_target_page_size(); + p->page_count =3D page_count; socket_send_channel_create(multifd_new_send_channel_async, p); } =20 @@ -1135,6 +1135,7 @@ int multifd_load_setup(Error **errp) p->name =3D g_strdup_printf("multifdrecv_%d", i); p->iov =3D g_new0(struct iovec, page_count); p->normal =3D g_new0(ram_addr_t, page_count); + p->page_count =3D page_count; p->page_size =3D qemu_target_page_size(); } =20 --=20 2.35.1 From nobody Wed May 15 06:08:33 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1652222805; cv=none; d=zohomail.com; s=zohoarc; b=QEGJJK7cSVXPfafFASO3rPiitJeLMIcBSpfqtRW7EdLxkCDT86WqrlXvodLp5eR5Qpu1GSo+KqnvpuWZrqVgZ7BoZ387XUOVGe8ILGpW/xUOSdg2cMbekOEyRiAUikKBPz9tV2DQN0HZ+ucKxtxRG8VYkx6hs/vDfE+yo1k7Ikg= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1652222805; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=XWmJF/8QkPo8ktolodObHTFwS8S/eqaP/NswxZP6mLM=; b=GJFmbkm6o1KfGK4opZRWK7d6akG2rA5DzzC18g9Gq03iO1cewR6prCGvFH/bggmcHG1hbLBzxrL/lw3s7Pal3tUBw7BsJvxwgQUrz2Fsz7MJCMTq9kSk7WtoBAjDXM9hQ1UXcKrNA01/N6s6/tas1rixiS3nalskyfqkJRrdqYc= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1652222805647584.226640926925; Tue, 10 May 2022 15:46:45 -0700 (PDT) Received: from localhost ([::1]:41632 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1noYcu-0007RR-Kq for importer@patchew.org; Tue, 10 May 2022 18:46:44 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:47322) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1noYaJ-0004af-9v for qemu-devel@nongnu.org; Tue, 10 May 2022 18:44:03 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:44197) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1noYaH-0006JE-Ov for qemu-devel@nongnu.org; Tue, 10 May 2022 18:44:02 -0400 Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-527-MJFcF7dWOgKyIbu7zi8nAg-1; Tue, 10 May 2022 18:42:32 -0400 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.rdu2.redhat.com [10.11.54.1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id BBA3A185A7BA; Tue, 10 May 2022 22:42:31 +0000 (UTC) Received: from secure.mitica (unknown [10.39.195.247]) by smtp.corp.redhat.com (Postfix) with ESMTP id 3ACB140CFD06; Tue, 10 May 2022 22:42:30 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1652222641; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=XWmJF/8QkPo8ktolodObHTFwS8S/eqaP/NswxZP6mLM=; b=KnnTPOzyD2Vh0FEzrEZJbfBzarSAvCU10oVHTMkfRc87AvcTk4D6a2bpaxxVRQooMBx4xw eVMCtQnA1MZjqEDc/ofQ6UFGDWZsTpx/QdLWYfgUKbirYKfhSn2DUAWd+LXxyAtXqiv53g L0O6NmI3/QYQvr624RaQvuO4dQXZ6TU= X-MC-Unique: MJFcF7dWOgKyIbu7zi8nAg-1 From: Juan Quintela To: qemu-devel@nongnu.org Cc: Juan Quintela , "Dr. David Alan Gilbert" , Eduardo Habkost , Peter Xu , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , Yanan Wang , Leonardo Bras , Marcel Apfelbaum Subject: [PATCH v6 04/13] migration: Export ram_transferred_ram() Date: Wed, 11 May 2022 00:42:11 +0200 Message-Id: <20220510224220.5912-5-quintela@redhat.com> In-Reply-To: <20220510224220.5912-1-quintela@redhat.com> References: <20220510224220.5912-1-quintela@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 2.84 on 10.11.54.1 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=quintela@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -28 X-Spam_score: -2.9 X-Spam_bar: -- X-Spam_report: (-2.9 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.082, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1652222806714100002 Content-Type: text/plain; charset="utf-8" Signed-off-by: Juan Quintela --- migration/ram.h | 2 ++ migration/ram.c | 2 +- 2 files changed, 3 insertions(+), 1 deletion(-) diff --git a/migration/ram.h b/migration/ram.h index ded0a3a086..7b641adc55 100644 --- a/migration/ram.h +++ b/migration/ram.h @@ -65,6 +65,8 @@ int ram_load_postcopy(QEMUFile *f); =20 void ram_handle_compressed(void *host, uint8_t ch, uint64_t size); =20 +void ram_transferred_add(uint64_t bytes); + int ramblock_recv_bitmap_test(RAMBlock *rb, void *host_addr); bool ramblock_recv_bitmap_test_byte_offset(RAMBlock *rb, uint64_t byte_off= set); void ramblock_recv_bitmap_set(RAMBlock *rb, void *host_addr); diff --git a/migration/ram.c b/migration/ram.c index a2489a2699..738769ba15 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -394,7 +394,7 @@ uint64_t ram_bytes_remaining(void) =20 MigrationStats ram_counters; =20 -static void ram_transferred_add(uint64_t bytes) +void ram_transferred_add(uint64_t bytes) { if (runstate_is_running()) { ram_counters.precopy_bytes +=3D bytes; --=20 2.35.1 From nobody Wed May 15 06:08:33 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1652223201; cv=none; d=zohomail.com; s=zohoarc; b=Oq2k2k/5BDhOOaf2zXthtf7WYZassZGNzOxAXU3s1k8/Jr9/8Avdgy6cZblLq95b65MjYcbdKAPi2IxKOfn+FSkzJm41tTkjERqcElvUXq5bSEA4gziGPCrlKZh/g4KjoibCPc4bXLMHOgq+UbM/Nrkl010f52fg8q1mX3kM51Y= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1652223201; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=hi/qoHoZ2fBQsuMOS4nAvKLfTubjT+5SOaaavsjdjJ8=; b=QJ2mkELppZdAxj32wp9oIHZi0laeBa+SSBSag6z1tbauMQRc7jlF7vGwKOS9rtT5a7Jvkb2ajxMJU5Tff7eTHYgFxx8COfU+PKBLHrHvpcuW/meCKNrA1QJ+7ENiYsFR5retd09SBooZiIWkmB+T+cfFH2ZZHWwU0IRSOU6F2iU= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1652223201851370.8403205874464; Tue, 10 May 2022 15:53:21 -0700 (PDT) Received: from localhost ([::1]:34114 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1noYjI-0004pt-QW for importer@patchew.org; Tue, 10 May 2022 18:53:20 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:47276) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1noYa2-0004Rn-NT for qemu-devel@nongnu.org; Tue, 10 May 2022 18:43:47 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]:22262) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1noYZz-0006Ii-6s for qemu-devel@nongnu.org; Tue, 10 May 2022 18:43:45 -0400 Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-583-kZaHiY2CNR6b3Jf6crLunw-1; Tue, 10 May 2022 18:42:35 -0400 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.rdu2.redhat.com [10.11.54.1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 9F98C101AA42; Tue, 10 May 2022 22:42:33 +0000 (UTC) Received: from secure.mitica (unknown [10.39.195.247]) by smtp.corp.redhat.com (Postfix) with ESMTP id 114E040CFD06; Tue, 10 May 2022 22:42:31 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1652222622; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=hi/qoHoZ2fBQsuMOS4nAvKLfTubjT+5SOaaavsjdjJ8=; b=EH1DkzF4eUe/5U78hP7gb7aVLoqI6fvQekJtct7BCt8CRG0+KqbL8CntSYjhKgIRJ6Z/uM TDmWy87VRZ85SybYMTqi1sVDJtcyfswSaRL0it9rJ7kME2Jv+V0VZOwQRKcHlksbEUDEF+ IFxPjegxSA6yPVKykLrUJ3mDInMUswI= X-MC-Unique: kZaHiY2CNR6b3Jf6crLunw-1 From: Juan Quintela To: qemu-devel@nongnu.org Cc: Juan Quintela , "Dr. David Alan Gilbert" , Eduardo Habkost , Peter Xu , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , Yanan Wang , Leonardo Bras , Marcel Apfelbaum Subject: [PATCH v6 05/13] multifd: Count the number of bytes sent correctly Date: Wed, 11 May 2022 00:42:12 +0200 Message-Id: <20220510224220.5912-6-quintela@redhat.com> In-Reply-To: <20220510224220.5912-1-quintela@redhat.com> References: <20220510224220.5912-1-quintela@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 2.84 on 10.11.54.1 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=quintela@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -28 X-Spam_score: -2.9 X-Spam_bar: -- X-Spam_report: (-2.9 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.082, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1652223202623100001 Content-Type: text/plain; charset="utf-8" Current code asumes that all pages are whole. That is not true for example for compression already. Fix it for creating a new field ->sent_bytes that includes it. All ram_counters are used only from the migration thread, so we have two options: - put a mutex and fill everything when we sent it (not only ram_counters, also qemu_file->xfer_bytes). - Create a local variable that implements how much has been sent through each channel. And when we push another packet, we "add" the previous stats. I choose two due to less changes overall. On the previous code we increase transferred and then we sent. Current code goes the other way around. It sents the data, and after the fact, it updates the counters. Notice that each channel can have a maximum of half a megabyte of data without counting, so it is not very important. Signed-off-by: Juan Quintela --- migration/multifd.h | 2 ++ migration/multifd.c | 14 ++++++-------- 2 files changed, 8 insertions(+), 8 deletions(-) diff --git a/migration/multifd.h b/migration/multifd.h index f707e2a8b8..b29be5de06 100644 --- a/migration/multifd.h +++ b/migration/multifd.h @@ -100,6 +100,8 @@ typedef struct { uint32_t flags; /* global number of generated multifd packets */ uint64_t packet_num; + /* How many bytes have we sent on the last packet */ + uint64_t sent_bytes; /* thread has work to do */ int pending_job; /* array of pages to sent. diff --git a/migration/multifd.c b/migration/multifd.c index 893b90072d..427cbe2ceb 100644 --- a/migration/multifd.c +++ b/migration/multifd.c @@ -394,7 +394,6 @@ static int multifd_send_pages(QEMUFile *f) static int next_channel; MultiFDSendParams *p =3D NULL; /* make happy gcc */ MultiFDPages_t *pages =3D multifd_send_state->pages; - uint64_t transferred; =20 if (qatomic_read(&multifd_send_state->exiting)) { return -1; @@ -429,10 +428,10 @@ static int multifd_send_pages(QEMUFile *f) p->packet_num =3D multifd_send_state->packet_num++; multifd_send_state->pages =3D p->pages; p->pages =3D pages; - transferred =3D ((uint64_t) pages->num) * p->page_size + p->packet_len; - qemu_file_update_transfer(f, transferred); - ram_counters.multifd_bytes +=3D transferred; - ram_counters.transferred +=3D transferred; + ram_transferred_add(p->sent_bytes); + ram_counters.multifd_bytes +=3D p->sent_bytes; + qemu_file_update_transfer(f, p->sent_bytes); + p->sent_bytes =3D 0; qemu_mutex_unlock(&p->mutex); qemu_sem_post(&p->sem); =20 @@ -590,9 +589,6 @@ void multifd_send_sync_main(QEMUFile *f) p->packet_num =3D multifd_send_state->packet_num++; p->flags |=3D MULTIFD_FLAG_SYNC; p->pending_job++; - qemu_file_update_transfer(f, p->packet_len); - ram_counters.multifd_bytes +=3D p->packet_len; - ram_counters.transferred +=3D p->packet_len; qemu_mutex_unlock(&p->mutex); qemu_sem_post(&p->sem); } @@ -668,6 +664,8 @@ static void *multifd_send_thread(void *opaque) } =20 qemu_mutex_lock(&p->mutex); + p->sent_bytes +=3D p->packet_len;; + p->sent_bytes +=3D p->next_packet_size; p->pending_job--; qemu_mutex_unlock(&p->mutex); =20 --=20 2.35.1 From nobody Wed May 15 06:08:33 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1652223161; cv=none; d=zohomail.com; s=zohoarc; b=J0/iS6PaIhKQ7jFu2cycKP6t0hv+eQktXuUgZhixSSuxpY/9i/ZzyelBRt2rEgHCe5vUFjP0R0khtuyY45SdCjsG+2QBmlK+U10yQCrsiw0PUAlTKm37WAwct1IsK18+OXUTQhAcYvMSHUxYCu7NYD0O7OX9zejBmXK8N7JGVAU= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1652223161; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=kgMQyf+6cAYMyPOfa3g6WhIfSHURWnjP0FicBX58YGo=; b=Y1xgj9CbftUIOZF9ZqO1QeGZUDoxsdHzRwJCVXWZ4dTU+s32l+DxeJ8u4iJuZNOLfFLVgqwOuEqZo6KBEyb07FiEPCdRH9RWx30CDQFiH7eXTRQc9eI5SkTLmZNH+C6G8f9BzOHk2UeOb2mJR1BTymCX3VWvpzNj+DRLQTs/NW0= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1652223161081920.1736879555402; Tue, 10 May 2022 15:52:41 -0700 (PDT) Received: from localhost ([::1]:59632 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1noYie-0002xR-2z for importer@patchew.org; Tue, 10 May 2022 18:52:40 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:47432) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1noYaN-0004du-EP for qemu-devel@nongnu.org; Tue, 10 May 2022 18:44:08 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:60595) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1noYaI-0006Jj-OQ for qemu-devel@nongnu.org; Tue, 10 May 2022 18:44:07 -0400 Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-154-xfV491WaOZehLVHKxKCUkA-1; Tue, 10 May 2022 18:42:35 -0400 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.rdu2.redhat.com [10.11.54.1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 758A838349A7; Tue, 10 May 2022 22:42:35 +0000 (UTC) Received: from secure.mitica (unknown [10.39.195.247]) by smtp.corp.redhat.com (Postfix) with ESMTP id E663D40CFD25; Tue, 10 May 2022 22:42:33 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1652222642; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=kgMQyf+6cAYMyPOfa3g6WhIfSHURWnjP0FicBX58YGo=; b=U5MJwzcTzhjaqdiOkH+dm5osTkbuBCpNTuamTTf1A57Ooimwyd12EcIiPRm6difcX9a/32 HJY29fDKxAl538j5P93a6HIQR+9UeLlra/H+5qb8lOJXi8oDixb7bJaFgC8Po+a05mJQD8 urw5qYt1qvFV5v4S6ZqS8hdZC2BN1oc= X-MC-Unique: xfV491WaOZehLVHKxKCUkA-1 From: Juan Quintela To: qemu-devel@nongnu.org Cc: Juan Quintela , "Dr. David Alan Gilbert" , Eduardo Habkost , Peter Xu , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , Yanan Wang , Leonardo Bras , Marcel Apfelbaum Subject: [PATCH v6 06/13] migration: Make ram_save_target_page() a pointer Date: Wed, 11 May 2022 00:42:13 +0200 Message-Id: <20220510224220.5912-7-quintela@redhat.com> In-Reply-To: <20220510224220.5912-1-quintela@redhat.com> References: <20220510224220.5912-1-quintela@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 2.84 on 10.11.54.1 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=quintela@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -28 X-Spam_score: -2.9 X-Spam_bar: -- X-Spam_report: (-2.9 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.082, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1652223162378100001 Content-Type: text/plain; charset="utf-8" We are going to create a new function for multifd latest in the series. Signed-off-by: Juan Quintela Reviewed-by: Dr. David Alan Gilbert --- migration/ram.c | 12 ++++++++---- 1 file changed, 8 insertions(+), 4 deletions(-) diff --git a/migration/ram.c b/migration/ram.c index 738769ba15..14269a2671 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -295,6 +295,9 @@ struct RAMSrcPageRequest { QSIMPLEQ_ENTRY(RAMSrcPageRequest) next_req; }; =20 +typedef struct RAMState RAMState; +typedef struct PageSearchStatus PageSearchStatus; + /* State of RAM for migration */ struct RAMState { /* QEMUFile used for this migration */ @@ -349,8 +352,8 @@ struct RAMState { /* Queue of outstanding page requests from the destination */ QemuMutex src_page_req_mutex; QSIMPLEQ_HEAD(, RAMSrcPageRequest) src_page_requests; + int (*ram_save_target_page)(RAMState *rs, PageSearchStatus *pss); }; -typedef struct RAMState RAMState; =20 static RAMState *ram_state; =20 @@ -2132,14 +2135,14 @@ static bool save_compress_page(RAMState *rs, RAMBlo= ck *block, ram_addr_t offset) } =20 /** - * ram_save_target_page: save one target page + * ram_save_target_page_legacy: save one target page * * Returns the number of pages written * * @rs: current RAM state * @pss: data about the page we want to send */ -static int ram_save_target_page(RAMState *rs, PageSearchStatus *pss) +static int ram_save_target_page_legacy(RAMState *rs, PageSearchStatus *pss) { RAMBlock *block =3D pss->block; ram_addr_t offset =3D ((ram_addr_t)pss->page) << TARGET_PAGE_BITS; @@ -2214,7 +2217,7 @@ static int ram_save_host_page(RAMState *rs, PageSearc= hStatus *pss) do { /* Check the pages is dirty and if it is send it */ if (migration_bitmap_clear_dirty(rs, pss->block, pss->page)) { - tmppages =3D ram_save_target_page(rs, pss); + tmppages =3D rs->ram_save_target_page(rs, pss); if (tmppages < 0) { return tmppages; } @@ -2943,6 +2946,7 @@ static int ram_save_setup(QEMUFile *f, void *opaque) ram_control_before_iterate(f, RAM_CONTROL_SETUP); ram_control_after_iterate(f, RAM_CONTROL_SETUP); =20 + (*rsp)->ram_save_target_page =3D ram_save_target_page_legacy; multifd_send_sync_main(f); qemu_put_be64(f, RAM_SAVE_FLAG_EOS); qemu_fflush(f); --=20 2.35.1 From nobody Wed May 15 06:08:33 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1652222920; cv=none; d=zohomail.com; s=zohoarc; b=YxNYz4dwmokQiodv+CzzqWZDpjrCRQxbbkNAvNHiqJM9L66sF5Bqf4SxqOEyp5eBjF4vH7VgJskh1ejAfhqEaUY/SdZR1+ilZtHepOYuBeow4iuOhJ+PMdCHN0OzmoI6IFZ/vbssdcX/BB0kGi1G/Gi3D9sI5EDSnW9suY8NfW4= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1652222920; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=r1siDZNQAtdLDhjEaMo34hoB9UrL4m3nzoFR5wopWm4=; b=auANyQ8kiGjVaF+LvN+cmINuc+JXO0cAUxv/DXjv8R/QF3iKCeHIa5kBIeDez5g8KBtdRQPvF7CCOWQ02di2jKn9DxwW8kkYAUkuwRYNCARTMIaFQ9JI/dr7fp0sB2VS5mvlnixyN8sdbqJ8Ih6sif7gJIYMUVMe7RzFF0C9ZiE= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1652222920171545.5205731975395; Tue, 10 May 2022 15:48:40 -0700 (PDT) Received: from localhost ([::1]:50018 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1noYel-0004ft-56 for importer@patchew.org; Tue, 10 May 2022 18:48:39 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:47376) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1noYaL-0004dF-1v for qemu-devel@nongnu.org; Tue, 10 May 2022 18:44:06 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]:50857) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1noYaH-0006JC-Nd for qemu-devel@nongnu.org; Tue, 10 May 2022 18:44:04 -0400 Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-513-vDK2J832P0mqmU86VYhY4w-1; Tue, 10 May 2022 18:42:37 -0400 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.rdu2.redhat.com [10.11.54.1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 4BAC98015BA; Tue, 10 May 2022 22:42:37 +0000 (UTC) Received: from secure.mitica (unknown [10.39.195.247]) by smtp.corp.redhat.com (Postfix) with ESMTP id BEE9040CF8E7; Tue, 10 May 2022 22:42:35 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1652222640; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=r1siDZNQAtdLDhjEaMo34hoB9UrL4m3nzoFR5wopWm4=; b=cvceDAOLNJYdv/SGYBlnk/LaKesge3FmL3U89AeA5MX2Yjf4j6HKmMYvoqSImCVgO2A15w cUJoSj2D/uTWS0shEyEfgLhBV0W/PludRoQ2wZWyj5NKTnnjGLfzknnGncNxeB19mqUkb0 upkK9E+4LCjgCP5yOF2VPIiraLYRtDQ= X-MC-Unique: vDK2J832P0mqmU86VYhY4w-1 From: Juan Quintela To: qemu-devel@nongnu.org Cc: Juan Quintela , "Dr. David Alan Gilbert" , Eduardo Habkost , Peter Xu , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , Yanan Wang , Leonardo Bras , Marcel Apfelbaum Subject: [PATCH v6 07/13] multifd: Make flags field thread local Date: Wed, 11 May 2022 00:42:14 +0200 Message-Id: <20220510224220.5912-8-quintela@redhat.com> In-Reply-To: <20220510224220.5912-1-quintela@redhat.com> References: <20220510224220.5912-1-quintela@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 2.84 on 10.11.54.1 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=quintela@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -28 X-Spam_score: -2.9 X-Spam_bar: -- X-Spam_report: (-2.9 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.082, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1652222921167100002 Content-Type: text/plain; charset="utf-8" Use of flags with respect to locking was incensistant. For the sending side: - it was set to 0 with mutex held on the multifd channel. - MULTIFD_FLAG_SYNC was set with mutex held on the migration thread. - Everything else was done without the mutex held on the multifd channel. On the reception side, it is not used on the migration thread, only on the multifd channels threads. So we move it to the multifd channels thread only variables, and we introduce a new bool sync_needed on the send side to pass that information. Signed-off-by: Juan Quintela --- migration/multifd.h | 10 ++++++---- migration/multifd.c | 23 +++++++++++++---------- 2 files changed, 19 insertions(+), 14 deletions(-) diff --git a/migration/multifd.h b/migration/multifd.h index b29be5de06..ecf5a8e868 100644 --- a/migration/multifd.h +++ b/migration/multifd.h @@ -96,12 +96,12 @@ typedef struct { bool running; /* should this thread finish */ bool quit; - /* multifd flags for each packet */ - uint32_t flags; /* global number of generated multifd packets */ uint64_t packet_num; /* How many bytes have we sent on the last packet */ uint64_t sent_bytes; + /* Do we need to do an iteration sync */ + bool sync_needed; /* thread has work to do */ int pending_job; /* array of pages to sent. @@ -115,6 +115,8 @@ typedef struct { =20 /* pointer to the packet */ MultiFDPacket_t *packet; + /* multifd flags for each packet */ + uint32_t flags; /* size of the next packet that contains pages */ uint32_t next_packet_size; /* packets sent through this channel */ @@ -161,8 +163,6 @@ typedef struct { bool running; /* should this thread finish */ bool quit; - /* multifd flags for each packet */ - uint32_t flags; /* global number of generated multifd packets */ uint64_t packet_num; =20 @@ -170,6 +170,8 @@ typedef struct { =20 /* pointer to the packet */ MultiFDPacket_t *packet; + /* multifd flags for each packet */ + uint32_t flags; /* size of the next packet that contains pages */ uint32_t next_packet_size; /* packets sent through this channel */ diff --git a/migration/multifd.c b/migration/multifd.c index 427cbe2ceb..3f9d9e3a56 100644 --- a/migration/multifd.c +++ b/migration/multifd.c @@ -587,7 +587,7 @@ void multifd_send_sync_main(QEMUFile *f) } =20 p->packet_num =3D multifd_send_state->packet_num++; - p->flags |=3D MULTIFD_FLAG_SYNC; + p->sync_needed =3D true; p->pending_job++; qemu_mutex_unlock(&p->mutex); qemu_sem_post(&p->sem); @@ -627,7 +627,11 @@ static void *multifd_send_thread(void *opaque) =20 if (p->pending_job) { uint64_t packet_num =3D p->packet_num; - uint32_t flags =3D p->flags; + p->flags =3D 0; + if (p->sync_needed) { + p->flags |=3D MULTIFD_FLAG_SYNC; + p->sync_needed =3D false; + } p->iovs_num =3D 1; p->normal_num =3D 0; =20 @@ -644,14 +648,13 @@ static void *multifd_send_thread(void *opaque) } } multifd_send_fill_packet(p); - p->flags =3D 0; p->num_packets++; p->total_normal_pages +=3D p->normal_num; p->pages->num =3D 0; p->pages->block =3D NULL; qemu_mutex_unlock(&p->mutex); =20 - trace_multifd_send(p->id, packet_num, p->normal_num, flags, + trace_multifd_send(p->id, packet_num, p->normal_num, p->flags, p->next_packet_size); =20 p->iov[0].iov_len =3D p->packet_len; @@ -669,7 +672,7 @@ static void *multifd_send_thread(void *opaque) p->pending_job--; qemu_mutex_unlock(&p->mutex); =20 - if (flags & MULTIFD_FLAG_SYNC) { + if (p->flags & MULTIFD_FLAG_SYNC) { qemu_sem_post(&p->sem_sync); } qemu_sem_post(&multifd_send_state->channels_ready); @@ -1042,7 +1045,7 @@ static void *multifd_recv_thread(void *opaque) rcu_register_thread(); =20 while (true) { - uint32_t flags; + bool sync_needed =3D false; =20 if (p->quit) { break; @@ -1064,11 +1067,11 @@ static void *multifd_recv_thread(void *opaque) break; } =20 - flags =3D p->flags; + trace_multifd_recv(p->id, p->packet_num, p->normal_num, p->flags, + p->next_packet_size); + sync_needed =3D p->flags & MULTIFD_FLAG_SYNC; /* recv methods don't know how to handle the SYNC flag */ p->flags &=3D ~MULTIFD_FLAG_SYNC; - trace_multifd_recv(p->id, p->packet_num, p->normal_num, flags, - p->next_packet_size); p->num_packets++; p->total_normal_pages +=3D p->normal_num; qemu_mutex_unlock(&p->mutex); @@ -1080,7 +1083,7 @@ static void *multifd_recv_thread(void *opaque) } } =20 - if (flags & MULTIFD_FLAG_SYNC) { + if (sync_needed) { qemu_sem_post(&multifd_recv_state->sem_sync); qemu_sem_wait(&p->sem_sync); } --=20 2.35.1 From nobody Wed May 15 06:08:33 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1652222805; cv=none; d=zohomail.com; s=zohoarc; b=YYvEwguYq3KRGHFK8xLxC0tzR6CVW4JbzLfqOV6BdPjhr5mMPUpuDlMDKLgoJCJMF5zdBPtQynKcZbazkQTERwr2kair/z6k68pPJusF5xkES1SJBReN52190BQ3rMlz0h+MOVdWyyNFrIaWv8PMNoO2XcW1Qa+PnrRjuIccCIY= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1652222805; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=vs8n8rfHPzW28VW8OO5SR80yoUi9AvYtSnpc5snK8dk=; b=GQHZpNT7euH4NGBu3BQplF7ume5oCl0RmPyG7FgzILXuUBRtDu6tabpQNV4hebvhO4kNFkAS5kFUac3Z+w2ocZUHjFlm1GTxwcMysWezchnu4VvTzcxzGzMGXQpN2BYcyCYv3i65GGDHzIMp6nryyBPM53kfE/frXUFxUFtSVYE= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1652222805905656.4499663560323; Tue, 10 May 2022 15:46:45 -0700 (PDT) Received: from localhost ([::1]:41668 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1noYcu-0007Sa-R5 for importer@patchew.org; Tue, 10 May 2022 18:46:44 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:47358) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1noYaK-0004cW-I0 for qemu-devel@nongnu.org; Tue, 10 May 2022 18:44:04 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]:24349) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1noYaH-0006JG-P4 for qemu-devel@nongnu.org; Tue, 10 May 2022 18:44:04 -0400 Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-611-mDlWBoM4PUuyVG5Wo6598w-1; Tue, 10 May 2022 18:42:39 -0400 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.rdu2.redhat.com [10.11.54.1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 249BA1C05AA8; Tue, 10 May 2022 22:42:39 +0000 (UTC) Received: from secure.mitica (unknown [10.39.195.247]) by smtp.corp.redhat.com (Postfix) with ESMTP id 93EAB40CFD25; Tue, 10 May 2022 22:42:37 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1652222641; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=vs8n8rfHPzW28VW8OO5SR80yoUi9AvYtSnpc5snK8dk=; b=YuQqtJoDUZnMl8gAQXrGNaFMhlIVc3D4a528x/6g2Ukz9gLjREsWYjvXOHMd+Q9OQKU2V2 6QRzVgo9fNQneJ05TFeD6ycsU3rh+vjV4w6RFHLuqj+ku8Y1OQWtzQTWLM7UsA2hxzeCZl 2CXMxcubb3SeJ7lnJSFsjK990jviXJs= X-MC-Unique: mDlWBoM4PUuyVG5Wo6598w-1 From: Juan Quintela To: qemu-devel@nongnu.org Cc: Juan Quintela , "Dr. David Alan Gilbert" , Eduardo Habkost , Peter Xu , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , Yanan Wang , Leonardo Bras , Marcel Apfelbaum Subject: [PATCH v6 08/13] multifd: Prepare to send a packet without the mutex held Date: Wed, 11 May 2022 00:42:15 +0200 Message-Id: <20220510224220.5912-9-quintela@redhat.com> In-Reply-To: <20220510224220.5912-1-quintela@redhat.com> References: <20220510224220.5912-1-quintela@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 2.84 on 10.11.54.1 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=quintela@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -28 X-Spam_score: -2.9 X-Spam_bar: -- X-Spam_report: (-2.9 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.082, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1652222806707100001 Content-Type: text/plain; charset="utf-8" We do the send_prepare() and the fill of the head packet without the mutex held. It will help a lot for compression and later in the series for zero pages. Notice that we can use p->pages without holding p->mutex because p->pending_job =3D=3D 1. Signed-off-by: Juan Quintela --- migration/multifd.h | 2 ++ migration/multifd.c | 11 ++++++----- 2 files changed, 8 insertions(+), 5 deletions(-) diff --git a/migration/multifd.h b/migration/multifd.h index ecf5a8e868..ed81f249d7 100644 --- a/migration/multifd.h +++ b/migration/multifd.h @@ -107,7 +107,9 @@ typedef struct { /* array of pages to sent. * The owner of 'pages' depends of 'pending_job' value: * pending_job =3D=3D 0 -> migration_thread can use it. + * No need for mutex lock. * pending_job !=3D 0 -> multifd_channel can use it. + * No need for mutex lock. */ MultiFDPages_t *pages; =20 diff --git a/migration/multifd.c b/migration/multifd.c index 3f9d9e3a56..c8f2caa2ae 100644 --- a/migration/multifd.c +++ b/migration/multifd.c @@ -632,6 +632,8 @@ static void *multifd_send_thread(void *opaque) p->flags |=3D MULTIFD_FLAG_SYNC; p->sync_needed =3D false; } + qemu_mutex_unlock(&p->mutex); + p->iovs_num =3D 1; p->normal_num =3D 0; =20 @@ -648,11 +650,6 @@ static void *multifd_send_thread(void *opaque) } } multifd_send_fill_packet(p); - p->num_packets++; - p->total_normal_pages +=3D p->normal_num; - p->pages->num =3D 0; - p->pages->block =3D NULL; - qemu_mutex_unlock(&p->mutex); =20 trace_multifd_send(p->id, packet_num, p->normal_num, p->flags, p->next_packet_size); @@ -667,6 +664,10 @@ static void *multifd_send_thread(void *opaque) } =20 qemu_mutex_lock(&p->mutex); + p->num_packets++; + p->total_normal_pages +=3D p->normal_num; + p->pages->num =3D 0; + p->pages->block =3D NULL; p->sent_bytes +=3D p->packet_len;; p->sent_bytes +=3D p->next_packet_size; p->pending_job--; --=20 2.35.1 From nobody Wed May 15 06:08:33 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1652222920; cv=none; d=zohomail.com; s=zohoarc; b=guegltxfmQJm1m/Xcar9SXSP7Xt7x8zxs+3d9aIw5LVJhgPAKKUY4pq+8ZWLcg6O2g8oTxbuql+v4VdIkXvYPIGizfhU8onk/BL29CB1Tt0D7HTVrMp4ei3bTfVi94G3PH+sYC8QnXlX3BGufIhiuWZckBdQxqP3K7luhma/FmQ= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1652222920; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=297XkggsqFJrCt8Cdj+dg0kFfmrKBouMvt5oU76YnFY=; b=de1rBRqbI4k78LH1QOUoqlMq9Fa2W1jPcH2Tf5RwdJAPLHU0AAsUVyp49RUcfi/6ybh1nePcjpMpkt8Ro24OZpugfU//9GJsF6j/CfdcSKjDj1TKGnPvJP75VazCYjxMoFyukks1VTXNqL701HaRCO3R18HY3M/gR5cXrTbv/Kw= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1652222920519854.2176516903346; Tue, 10 May 2022 15:48:40 -0700 (PDT) Received: from localhost ([::1]:50112 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1noYel-0004jV-FF for importer@patchew.org; Tue, 10 May 2022 18:48:39 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:47426) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1noYaM-0004de-PL for qemu-devel@nongnu.org; Tue, 10 May 2022 18:44:08 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:43199) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1noYaI-0006Jb-8A for qemu-devel@nongnu.org; Tue, 10 May 2022 18:44:06 -0400 Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-557-n9z07mGfNROdBdgzrkME-w-1; Tue, 10 May 2022 18:42:41 -0400 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.rdu2.redhat.com [10.11.54.1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id F01A480A0B9; Tue, 10 May 2022 22:42:40 +0000 (UTC) Received: from secure.mitica (unknown [10.39.195.247]) by smtp.corp.redhat.com (Postfix) with ESMTP id 6C2DC400E89E; Tue, 10 May 2022 22:42:39 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1652222641; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=297XkggsqFJrCt8Cdj+dg0kFfmrKBouMvt5oU76YnFY=; b=fDDm0YGRWPC41nI0e4KMf8gv78npTMGVsRnvcFuKCMuo+e1/dKUPMQSqw7wAjcoc7BCzI4 s6OSBPdVxX1fygTMIRLdPBO0NxTAQFp6oa+k4ih3iFvUkBwkhruCE+yjCEZW8sUYh6gzjB xXLngUIBaoXghYh12Ec+6nyVKTviOW8= X-MC-Unique: n9z07mGfNROdBdgzrkME-w-1 From: Juan Quintela To: qemu-devel@nongnu.org Cc: Juan Quintela , "Dr. David Alan Gilbert" , Eduardo Habkost , Peter Xu , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , Yanan Wang , Leonardo Bras , Marcel Apfelbaum Subject: [PATCH v6 09/13] multifd: Add property to enable/disable zero_page Date: Wed, 11 May 2022 00:42:16 +0200 Message-Id: <20220510224220.5912-10-quintela@redhat.com> In-Reply-To: <20220510224220.5912-1-quintela@redhat.com> References: <20220510224220.5912-1-quintela@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 2.84 on 10.11.54.1 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=quintela@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -28 X-Spam_score: -2.9 X-Spam_bar: -- X-Spam_report: (-2.9 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.082, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1652222921183100003 Content-Type: text/plain; charset="utf-8" Signed-off-by: Juan Quintela Reviewed-by: Dr. David Alan Gilbert --- migration/migration.h | 3 +++ hw/core/machine.c | 4 +++- migration/migration.c | 11 +++++++++++ 3 files changed, 17 insertions(+), 1 deletion(-) diff --git a/migration/migration.h b/migration/migration.h index a863032b71..068e66ca9a 100644 --- a/migration/migration.h +++ b/migration/migration.h @@ -332,6 +332,8 @@ struct MigrationState { * This save hostname when out-going migration starts */ char *hostname; + /* Use multifd channel to send zero pages */ + bool multifd_zero_pages; }; =20 void migrate_set_state(int *state, int old_state, int new_state); @@ -374,6 +376,7 @@ int migrate_multifd_channels(void); MultiFDCompression migrate_multifd_compression(void); int migrate_multifd_zlib_level(void); int migrate_multifd_zstd_level(void); +bool migrate_use_multifd_zero_page(void); =20 int migrate_use_xbzrle(void); uint64_t migrate_xbzrle_cache_size(void); diff --git a/hw/core/machine.c b/hw/core/machine.c index 700c1e76b8..d02977d5df 100644 --- a/hw/core/machine.c +++ b/hw/core/machine.c @@ -37,7 +37,9 @@ #include "hw/virtio/virtio.h" #include "hw/virtio/virtio-pci.h" =20 -GlobalProperty hw_compat_7_0[] =3D {}; +GlobalProperty hw_compat_7_0[] =3D { + { "migration", "multifd-zero-pages", "false" }, +}; const size_t hw_compat_7_0_len =3D G_N_ELEMENTS(hw_compat_7_0); =20 GlobalProperty hw_compat_6_2[] =3D { diff --git a/migration/migration.c b/migration/migration.c index 5a31b23bd6..7e591990ef 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -2517,6 +2517,15 @@ bool migrate_use_multifd(void) return s->enabled_capabilities[MIGRATION_CAPABILITY_MULTIFD]; } =20 +bool migrate_use_multifd_zero_page(void) +{ + MigrationState *s; + + s =3D migrate_get_current(); + + return s->multifd_zero_pages; +} + bool migrate_pause_before_switchover(void) { MigrationState *s; @@ -4164,6 +4173,8 @@ static Property migration_properties[] =3D { clear_bitmap_shift, CLEAR_BITMAP_SHIFT_DEFAULT), =20 /* Migration parameters */ + DEFINE_PROP_BOOL("multifd-zero-pages", MigrationState, + multifd_zero_pages, true), DEFINE_PROP_UINT8("x-compress-level", MigrationState, parameters.compress_level, DEFAULT_MIGRATE_COMPRESS_LEVEL), --=20 2.35.1 From nobody Wed May 15 06:08:33 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1652222919; cv=none; d=zohomail.com; s=zohoarc; b=g6shXKWVej5DeSb84UW/0PmMSInGppkxoUZM6x5e4lRN8jF4r8+1XW3nlaPLov7BT7dO2/ywRoOHGNb++6mAbwZeZv0i5ZvzPet9Z1038mSpwdvQu5UqPGS+ZGknjUnGken15s3HZjafdJG7EkFaxPXZY4QCMmK4d+wBKV5GxjU= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1652222919; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=C4oF1RhtlZ0gyI5ZLp1fidQR9q4Lxwn6XCwpOR2V/f0=; b=XYJnhYNNVpo0nllJoc4kIdrQMPV5/zNFsHgXQhOGucJzJH5yrImiWCS1v7YjHITkdnznib/q6kum3Fz4u/dQ1VAuCuMPU+OOtyIF3p6K0668hm9MpCGKAhjeXQCQ8zxxgLoPf08C1ZEgKuqcvO7hPCA1fdEiNcUCjtJjDHx5/8c= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1652222919774706.8803949418677; Tue, 10 May 2022 15:48:39 -0700 (PDT) Received: from localhost ([::1]:50010 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1noYek-0004fc-8E for importer@patchew.org; Tue, 10 May 2022 18:48:38 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:47366) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1noYaK-0004cw-To for qemu-devel@nongnu.org; Tue, 10 May 2022 18:44:04 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:29661) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1noYaH-0006J8-Ow for qemu-devel@nongnu.org; Tue, 10 May 2022 18:44:04 -0400 Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-468-fickj7YqMouTIUF_t6xVhQ-1; Tue, 10 May 2022 18:42:43 -0400 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.rdu2.redhat.com [10.11.54.1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 7374F29ABA31; Tue, 10 May 2022 22:42:43 +0000 (UTC) Received: from secure.mitica (unknown [10.39.195.247]) by smtp.corp.redhat.com (Postfix) with ESMTP id 4480540CFD25; Tue, 10 May 2022 22:42:41 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1652222640; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=C4oF1RhtlZ0gyI5ZLp1fidQR9q4Lxwn6XCwpOR2V/f0=; b=FMJGO+teZkt7phM/XoM27e/sdE1cK8+9j36vbJX+0+LYax3o6IcePm3Xsn6SkNmJ6FXDyZ CvCNG5xlitZ+jSesC3kh5OB7obOlPH+TQrFJ56nW4bMuq9C04G57x0fjL4+351Ldcyh095 75hAeajCoqxwdt0zqamEx3RxaVsFe0A= X-MC-Unique: fickj7YqMouTIUF_t6xVhQ-1 From: Juan Quintela To: qemu-devel@nongnu.org Cc: Juan Quintela , "Dr. David Alan Gilbert" , Eduardo Habkost , Peter Xu , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , Yanan Wang , Leonardo Bras , Marcel Apfelbaum Subject: [PATCH v6 10/13] migration: Export ram_release_page() Date: Wed, 11 May 2022 00:42:17 +0200 Message-Id: <20220510224220.5912-11-quintela@redhat.com> In-Reply-To: <20220510224220.5912-1-quintela@redhat.com> References: <20220510224220.5912-1-quintela@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 2.84 on 10.11.54.1 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=quintela@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -28 X-Spam_score: -2.9 X-Spam_bar: -- X-Spam_report: (-2.9 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.082, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1652222921152100001 Content-Type: text/plain; charset="utf-8" Signed-off-by: Juan Quintela --- migration/ram.h | 1 + migration/ram.c | 2 +- 2 files changed, 2 insertions(+), 1 deletion(-) diff --git a/migration/ram.h b/migration/ram.h index 7b641adc55..aee08de2a5 100644 --- a/migration/ram.h +++ b/migration/ram.h @@ -66,6 +66,7 @@ int ram_load_postcopy(QEMUFile *f); void ram_handle_compressed(void *host, uint8_t ch, uint64_t size); =20 void ram_transferred_add(uint64_t bytes); +void ram_release_page(const char *rbname, uint64_t offset); =20 int ramblock_recv_bitmap_test(RAMBlock *rb, void *host_addr); bool ramblock_recv_bitmap_test_byte_offset(RAMBlock *rb, uint64_t byte_off= set); diff --git a/migration/ram.c b/migration/ram.c index 14269a2671..34da0d71df 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -1182,7 +1182,7 @@ static void migration_bitmap_sync_precopy(RAMState *r= s) } } =20 -static void ram_release_page(const char *rbname, uint64_t offset) +void ram_release_page(const char *rbname, uint64_t offset) { if (!migrate_release_ram() || !migration_in_postcopy()) { return; --=20 2.35.1 From nobody Wed May 15 06:08:33 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1652222808; cv=none; d=zohomail.com; s=zohoarc; b=RUqb0r8El8iVmznzUkLVqR4SXkCHl1vtEvwfKkG787SzqfiS1QgXakTkwOMF12Qi3s19zwr/HOBs77RIlWp5IfaN7WqAhlfgbRDNhIx2v4eoyn/oSAgDJRhaHd1pq010nHzFn286jgPMBJlGE3kz0drJr2bqAIaGhR1txJsgbSw= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1652222808; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=mWS/5FeV6dsloVMcQQ2NAu5WOn/IjzqL1/D0GIxLup8=; b=JTVnMxcMN2pt+RviyWnk/HhZbzuqnlvzCFWZd46QLPLB92jzwXXwgrFMQEw3hS3Bc0BERD5Se2wlSn3Haihb56/5H0Z7Z09DqdNA4b3R+N/3lcWkrMcfqiHo/igtDNXEHST/gi5OeLiN4KRWTYFGQkcU2fnaxov162KUj53ZlOY= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1652222808539898.1006869340265; Tue, 10 May 2022 15:46:48 -0700 (PDT) Received: from localhost ([::1]:41692 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1noYcx-0007Tg-EF for importer@patchew.org; Tue, 10 May 2022 18:46:47 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:47430) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1noYaM-0004dg-T0 for qemu-devel@nongnu.org; Tue, 10 May 2022 18:44:08 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]:55530) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1noYaI-0006Jh-JF for qemu-devel@nongnu.org; Tue, 10 May 2022 18:44:06 -0400 Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-591-VggH-SgGN0W4dzLDyTqRng-1; Tue, 10 May 2022 18:42:45 -0400 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.rdu2.redhat.com [10.11.54.1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 6F48185A5A8; Tue, 10 May 2022 22:42:45 +0000 (UTC) Received: from secure.mitica (unknown [10.39.195.247]) by smtp.corp.redhat.com (Postfix) with ESMTP id BDCFD40CFD06; Tue, 10 May 2022 22:42:43 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1652222642; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=mWS/5FeV6dsloVMcQQ2NAu5WOn/IjzqL1/D0GIxLup8=; b=GUiJb+Hg/lVPZ+nKa8nRdiLzN52whxhLTAnirt8zO8rSXHmPvVwmsyUPujbJcz3GfkI6lp mKJ113zxrW/dTqbzxC78YhA40xyhjzU/ZWDipbc17d9B+bm86ZIous2K3U5zeC8sCtTFPW wqikZpS4ie7RvjysY6y2Lika5QT7uB0= X-MC-Unique: VggH-SgGN0W4dzLDyTqRng-1 From: Juan Quintela To: qemu-devel@nongnu.org Cc: Juan Quintela , "Dr. David Alan Gilbert" , Eduardo Habkost , Peter Xu , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , Yanan Wang , Leonardo Bras , Marcel Apfelbaum Subject: [PATCH v6 11/13] multifd: Support for zero pages transmission Date: Wed, 11 May 2022 00:42:18 +0200 Message-Id: <20220510224220.5912-12-quintela@redhat.com> In-Reply-To: <20220510224220.5912-1-quintela@redhat.com> References: <20220510224220.5912-1-quintela@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 2.84 on 10.11.54.1 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=quintela@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -28 X-Spam_score: -2.9 X-Spam_bar: -- X-Spam_report: (-2.9 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.082, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1652222810878100001 Content-Type: text/plain; charset="utf-8" This patch adds counters and similar. Logic will be added on the following patch. Signed-off-by: Juan Quintela --- Added counters for duplicated/non duplicated pages. Removed reviewed by from David. Add total_zero_pages --- migration/multifd.h | 17 ++++++++++++++++- migration/multifd.c | 36 +++++++++++++++++++++++++++++------- migration/ram.c | 2 -- migration/trace-events | 8 ++++---- 4 files changed, 49 insertions(+), 14 deletions(-) diff --git a/migration/multifd.h b/migration/multifd.h index ed81f249d7..3bb33eb4af 100644 --- a/migration/multifd.h +++ b/migration/multifd.h @@ -47,7 +47,10 @@ typedef struct { /* size of the next packet that contains pages */ uint32_t next_packet_size; uint64_t packet_num; - uint64_t unused[4]; /* Reserved for future use */ + /* zero pages */ + uint32_t zero_pages; + uint32_t unused32[1]; /* Reserved for future use */ + uint64_t unused64[3]; /* Reserved for future use */ char ramblock[256]; uint64_t offset[]; } __attribute__((packed)) MultiFDPacket_t; @@ -125,6 +128,8 @@ typedef struct { uint64_t num_packets; /* non zero pages sent through this channel */ uint64_t total_normal_pages; + /* zero pages sent through this channel */ + uint64_t total_zero_pages; /* buffers to send */ struct iovec *iov; /* number of iovs used */ @@ -133,6 +138,10 @@ typedef struct { ram_addr_t *normal; /* num of non zero pages */ uint32_t normal_num; + /* Pages that are zero */ + ram_addr_t *zero; + /* num of zero pages */ + uint32_t zero_num; /* used for compression methods */ void *data; } MultiFDSendParams; @@ -182,12 +191,18 @@ typedef struct { uint8_t *host; /* non zero pages recv through this channel */ uint64_t total_normal_pages; + /* zero pages recv through this channel */ + uint64_t total_zero_pages; /* buffers to recv */ struct iovec *iov; /* Pages that are not zero */ ram_addr_t *normal; /* num of non zero pages */ uint32_t normal_num; + /* Pages that are zero */ + ram_addr_t *zero; + /* num of zero pages */ + uint32_t zero_num; /* used for de-compression methods */ void *data; } MultiFDRecvParams; diff --git a/migration/multifd.c b/migration/multifd.c index c8f2caa2ae..15a16686db 100644 --- a/migration/multifd.c +++ b/migration/multifd.c @@ -263,6 +263,7 @@ static void multifd_send_fill_packet(MultiFDSendParams = *p) packet->normal_pages =3D cpu_to_be32(p->normal_num); packet->next_packet_size =3D cpu_to_be32(p->next_packet_size); packet->packet_num =3D cpu_to_be64(p->packet_num); + packet->zero_pages =3D cpu_to_be32(p->zero_num); =20 if (p->pages->block) { strncpy(packet->ramblock, p->pages->block->idstr, 256); @@ -323,7 +324,15 @@ static int multifd_recv_unfill_packet(MultiFDRecvParam= s *p, Error **errp) p->next_packet_size =3D be32_to_cpu(packet->next_packet_size); p->packet_num =3D be64_to_cpu(packet->packet_num); =20 - if (p->normal_num =3D=3D 0) { + p->zero_num =3D be32_to_cpu(packet->zero_pages); + if (p->zero_num > packet->pages_alloc - p->normal_num) { + error_setg(errp, "multifd: received packet " + "with %u zero pages and expected maximum pages are %u", + p->zero_num, packet->pages_alloc - p->normal_num) ; + return -1; + } + + if (p->normal_num =3D=3D 0 && p->zero_num =3D=3D 0) { return 0; } =20 @@ -432,6 +441,8 @@ static int multifd_send_pages(QEMUFile *f) ram_counters.multifd_bytes +=3D p->sent_bytes; qemu_file_update_transfer(f, p->sent_bytes); p->sent_bytes =3D 0; + ram_counters.normal +=3D p->normal_num; + ram_counters.duplicate +=3D p->zero_num; qemu_mutex_unlock(&p->mutex); qemu_sem_post(&p->sem); =20 @@ -545,6 +556,8 @@ void multifd_save_cleanup(void) p->iov =3D NULL; g_free(p->normal); p->normal =3D NULL; + g_free(p->zero); + p->zero =3D NULL; multifd_send_state->ops->send_cleanup(p, &local_err); if (local_err) { migrate_set_error(migrate_get_current(), local_err); @@ -636,6 +649,7 @@ static void *multifd_send_thread(void *opaque) =20 p->iovs_num =3D 1; p->normal_num =3D 0; + p->zero_num =3D 0; =20 for (int i =3D 0; i < p->pages->num; i++) { p->normal[p->normal_num] =3D p->pages->offset[i]; @@ -651,8 +665,8 @@ static void *multifd_send_thread(void *opaque) } multifd_send_fill_packet(p); =20 - trace_multifd_send(p->id, packet_num, p->normal_num, p->flags, - p->next_packet_size); + trace_multifd_send(p->id, packet_num, p->normal_num, p->zero_n= um, + p->flags, p->next_packet_size); =20 p->iov[0].iov_len =3D p->packet_len; p->iov[0].iov_base =3D p->packet; @@ -666,6 +680,7 @@ static void *multifd_send_thread(void *opaque) qemu_mutex_lock(&p->mutex); p->num_packets++; p->total_normal_pages +=3D p->normal_num; + p->total_zero_pages +=3D p->zero_num; p->pages->num =3D 0; p->pages->block =3D NULL; p->sent_bytes +=3D p->packet_len;; @@ -707,7 +722,8 @@ out: qemu_mutex_unlock(&p->mutex); =20 rcu_unregister_thread(); - trace_multifd_send_thread_end(p->id, p->num_packets, p->total_normal_p= ages); + trace_multifd_send_thread_end(p->id, p->num_packets, p->total_normal_p= ages, + p->total_zero_pages); =20 return NULL; } @@ -897,6 +913,7 @@ int multifd_save_setup(Error **errp) p->normal =3D g_new0(ram_addr_t, page_count); p->page_size =3D qemu_target_page_size(); p->page_count =3D page_count; + p->zero =3D g_new0(ram_addr_t, page_count); socket_send_channel_create(multifd_new_send_channel_async, p); } =20 @@ -998,6 +1015,8 @@ int multifd_load_cleanup(Error **errp) p->iov =3D NULL; g_free(p->normal); p->normal =3D NULL; + g_free(p->zero); + p->zero =3D NULL; multifd_recv_state->ops->recv_cleanup(p); } qemu_sem_destroy(&multifd_recv_state->sem_sync); @@ -1068,13 +1087,14 @@ static void *multifd_recv_thread(void *opaque) break; } =20 - trace_multifd_recv(p->id, p->packet_num, p->normal_num, p->flags, - p->next_packet_size); + trace_multifd_recv(p->id, p->packet_num, p->normal_num, p->zero_nu= m, + p->flags, p->next_packet_size); sync_needed =3D p->flags & MULTIFD_FLAG_SYNC; /* recv methods don't know how to handle the SYNC flag */ p->flags &=3D ~MULTIFD_FLAG_SYNC; p->num_packets++; p->total_normal_pages +=3D p->normal_num; + p->total_normal_pages +=3D p->zero_num; qemu_mutex_unlock(&p->mutex); =20 if (p->normal_num) { @@ -1099,7 +1119,8 @@ static void *multifd_recv_thread(void *opaque) qemu_mutex_unlock(&p->mutex); =20 rcu_unregister_thread(); - trace_multifd_recv_thread_end(p->id, p->num_packets, p->total_normal_p= ages); + trace_multifd_recv_thread_end(p->id, p->num_packets, p->total_normal_p= ages, + p->total_zero_pages); =20 return NULL; } @@ -1139,6 +1160,7 @@ int multifd_load_setup(Error **errp) p->normal =3D g_new0(ram_addr_t, page_count); p->page_count =3D page_count; p->page_size =3D qemu_target_page_size(); + p->zero =3D g_new0(ram_addr_t, page_count); } =20 for (i =3D 0; i < thread_count; i++) { diff --git a/migration/ram.c b/migration/ram.c index 34da0d71df..0a91b87bd2 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -1356,8 +1356,6 @@ static int ram_save_multifd_page(RAMState *rs, RAMBlo= ck *block, if (multifd_queue_page(rs->f, block, offset) < 0) { return -1; } - ram_counters.normal++; - return 1; } =20 diff --git a/migration/trace-events b/migration/trace-events index 1aec580e92..d70e89dbb9 100644 --- a/migration/trace-events +++ b/migration/trace-events @@ -114,21 +114,21 @@ unqueue_page(char *block, uint64_t offset, bool dirty= ) "ramblock '%s' offset 0x% =20 # multifd.c multifd_new_send_channel_async(uint8_t id) "channel %u" -multifd_recv(uint8_t id, uint64_t packet_num, uint32_t used, uint32_t flag= s, uint32_t next_packet_size) "channel %u packet_num %" PRIu64 " pages %u f= lags 0x%x next packet size %u" +multifd_recv(uint8_t id, uint64_t packet_num, uint32_t normal, uint32_t ze= ro, uint32_t flags, uint32_t next_packet_size) "channel %u packet_num %" PR= Iu64 " normal pages %u zero pages %u flags 0x%x next packet size %u" multifd_recv_new_channel(uint8_t id) "channel %u" multifd_recv_sync_main(long packet_num) "packet num %ld" multifd_recv_sync_main_signal(uint8_t id) "channel %u" multifd_recv_sync_main_wait(uint8_t id) "channel %u" multifd_recv_terminate_threads(bool error) "error %d" -multifd_recv_thread_end(uint8_t id, uint64_t packets, uint64_t pages) "cha= nnel %u packets %" PRIu64 " pages %" PRIu64 +multifd_recv_thread_end(uint8_t id, uint64_t packets, uint64_t normal_page= s, uint64_t zero_pages) "channel %u packets %" PRIu64 " normal pages %" PRI= u64 " zero pages %" PRIu64 multifd_recv_thread_start(uint8_t id) "%u" -multifd_send(uint8_t id, uint64_t packet_num, uint32_t normal, uint32_t fl= ags, uint32_t next_packet_size) "channel %u packet_num %" PRIu64 " normal p= ages %u flags 0x%x next packet size %u" +multifd_send(uint8_t id, uint64_t packet_num, uint32_t normalpages, uint32= _t zero_pages, uint32_t flags, uint32_t next_packet_size) "channel %u packe= t_num %" PRIu64 " normal pages %u zero pages %u flags 0x%x next packet size= %u" multifd_send_error(uint8_t id) "channel %u" multifd_send_sync_main(long packet_num) "packet num %ld" multifd_send_sync_main_signal(uint8_t id) "channel %u" multifd_send_sync_main_wait(uint8_t id) "channel %u" multifd_send_terminate_threads(bool error) "error %d" -multifd_send_thread_end(uint8_t id, uint64_t packets, uint64_t normal_page= s) "channel %u packets %" PRIu64 " normal pages %" PRIu64 +multifd_send_thread_end(uint8_t id, uint64_t packets, uint64_t normal_page= s, uint64_t zero_pages) "channel %u packets %" PRIu64 " normal pages %" PR= Iu64 " zero pages %" PRIu64 multifd_send_thread_start(uint8_t id) "%u" multifd_tls_outgoing_handshake_start(void *ioc, void *tioc, const char *ho= stname) "ioc=3D%p tioc=3D%p hostname=3D%s" multifd_tls_outgoing_handshake_error(void *ioc, const char *err) "ioc=3D%p= err=3D%s" --=20 2.35.1 From nobody Wed May 15 06:08:33 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1652223269; cv=none; d=zohomail.com; s=zohoarc; b=JVT/AKOeXEzIm0CdPly4NqO2x/KR1po9BlHvIL+nOvVPbqxRhR4+Qza3jxvKeGAWFTtkVU2WYOGl+rp3y3Ekm9bUnCCyc7kdgwmJTtitKkZ2otmx7jQ4eBxGlGd+k1Bv+lfFkAcIqjGYiVfx3LsMorMofFzX9ey5C4ik/tMDxrQ= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1652223269; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=/+4pbh5rbtXNZZhoy76swj5qaTqi9OaLHyKHpLqF118=; b=nZN4ZYB2skpiDeaIXpVMr7ni2c4CIuXTfitAVqFWNpwugkMeW9D+TMwoeSS1gufd8WINMAXd6s7RpVUNq+bD+j5JMZUNwg6eecs511mnzKHAASSA5/dPNUZ2AhOhgtzSe/+cyvAb0b8bYL4jWKnkJsGABbqH0KKrHviXcufbqG0= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1652223269783286.72498722465366; Tue, 10 May 2022 15:54:29 -0700 (PDT) Received: from localhost ([::1]:37044 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1noYkO-0006qF-D4 for importer@patchew.org; Tue, 10 May 2022 18:54:28 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:47412) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1noYaM-0004db-6r for qemu-devel@nongnu.org; Tue, 10 May 2022 18:44:08 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:52169) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1noYaH-0006JO-V6 for qemu-devel@nongnu.org; Tue, 10 May 2022 18:44:05 -0400 Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-661-FBpbveAxMn21Gnj1eZn25A-1; Tue, 10 May 2022 18:42:47 -0400 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.rdu2.redhat.com [10.11.54.1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 480FF29ABA36; Tue, 10 May 2022 22:42:47 +0000 (UTC) Received: from secure.mitica (unknown [10.39.195.247]) by smtp.corp.redhat.com (Postfix) with ESMTP id B8DD940CFD25; Tue, 10 May 2022 22:42:45 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1652222641; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=/+4pbh5rbtXNZZhoy76swj5qaTqi9OaLHyKHpLqF118=; b=TV43L71EiCBDjdPwiE2BaQrDlgfJQEzKjseR4vJMWukBICqedVX7E6GOG30itskc9kxZcP VpHG1qh+lhHbfEM97l8LkoVIPGNb9p91WYk2qlfitZKRvbuPKQxYAs7pW25oM5rF39Bc8Y SJXtF33YTH61uOExRzbUfVhgjpdFbaY= X-MC-Unique: FBpbveAxMn21Gnj1eZn25A-1 From: Juan Quintela To: qemu-devel@nongnu.org Cc: Juan Quintela , "Dr. David Alan Gilbert" , Eduardo Habkost , Peter Xu , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , Yanan Wang , Leonardo Bras , Marcel Apfelbaum Subject: [PATCH v6 12/13] multifd: Zero pages transmission Date: Wed, 11 May 2022 00:42:19 +0200 Message-Id: <20220510224220.5912-13-quintela@redhat.com> In-Reply-To: <20220510224220.5912-1-quintela@redhat.com> References: <20220510224220.5912-1-quintela@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 2.84 on 10.11.54.1 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=quintela@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -28 X-Spam_score: -2.9 X-Spam_bar: -- X-Spam_report: (-2.9 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.082, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1652223271106100001 Content-Type: text/plain; charset="utf-8" This implements the zero page dection and handling. Signed-off-by: Juan Quintela --- Add comment for offset (dave) Use local variables for offset/block to have shorter lines --- migration/multifd.h | 5 +++++ migration/multifd.c | 41 +++++++++++++++++++++++++++++++++++++++-- 2 files changed, 44 insertions(+), 2 deletions(-) diff --git a/migration/multifd.h b/migration/multifd.h index 3bb33eb4af..b885390116 100644 --- a/migration/multifd.h +++ b/migration/multifd.h @@ -52,6 +52,11 @@ typedef struct { uint32_t unused32[1]; /* Reserved for future use */ uint64_t unused64[3]; /* Reserved for future use */ char ramblock[256]; + /* + * This array contains the pointers to: + * - normal pages (initial normal_pages entries) + * - zero pages (following zero_pages entries) + */ uint64_t offset[]; } __attribute__((packed)) MultiFDPacket_t; =20 diff --git a/migration/multifd.c b/migration/multifd.c index 15a16686db..aa012634cf 100644 --- a/migration/multifd.c +++ b/migration/multifd.c @@ -11,6 +11,7 @@ */ =20 #include "qemu/osdep.h" +#include "qemu/cutils.h" #include "qemu/rcu.h" #include "exec/target_page.h" #include "sysemu/sysemu.h" @@ -275,6 +276,12 @@ static void multifd_send_fill_packet(MultiFDSendParams= *p) =20 packet->offset[i] =3D cpu_to_be64(temp); } + for (i =3D 0; i < p->zero_num; i++) { + /* there are architectures where ram_addr_t is 32 bit */ + uint64_t temp =3D p->zero[i]; + + packet->offset[p->normal_num + i] =3D cpu_to_be64(temp); + } } =20 static int multifd_recv_unfill_packet(MultiFDRecvParams *p, Error **errp) @@ -358,6 +365,18 @@ static int multifd_recv_unfill_packet(MultiFDRecvParam= s *p, Error **errp) p->normal[i] =3D offset; } =20 + for (i =3D 0; i < p->zero_num; i++) { + uint64_t offset =3D be64_to_cpu(packet->offset[p->normal_num + i]); + + if (offset > (block->used_length - p->page_size)) { + error_setg(errp, "multifd: offset too long %" PRIu64 + " (max " RAM_ADDR_FMT ")", + offset, block->used_length); + return -1; + } + p->zero[i] =3D offset; + } + return 0; } =20 @@ -618,6 +637,8 @@ static void *multifd_send_thread(void *opaque) { MultiFDSendParams *p =3D opaque; Error *local_err =3D NULL; + /* qemu older than 7.0 don't understand zero page on multifd channel */ + bool use_zero_page =3D migrate_use_multifd_zero_page(); int ret =3D 0; =20 trace_multifd_send_thread_start(p->id); @@ -639,6 +660,7 @@ static void *multifd_send_thread(void *opaque) qemu_mutex_lock(&p->mutex); =20 if (p->pending_job) { + RAMBlock *rb =3D p->pages->block; uint64_t packet_num =3D p->packet_num; p->flags =3D 0; if (p->sync_needed) { @@ -652,8 +674,16 @@ static void *multifd_send_thread(void *opaque) p->zero_num =3D 0; =20 for (int i =3D 0; i < p->pages->num; i++) { - p->normal[p->normal_num] =3D p->pages->offset[i]; - p->normal_num++; + uint64_t offset =3D p->pages->offset[i]; + if (use_zero_page && + buffer_is_zero(rb->host + offset, p->page_size)) { + p->zero[p->zero_num] =3D offset; + p->zero_num++; + ram_release_page(rb->idstr, offset); + } else { + p->normal[p->normal_num] =3D offset; + p->normal_num++; + } } =20 if (p->normal_num) { @@ -1104,6 +1134,13 @@ static void *multifd_recv_thread(void *opaque) } } =20 + for (int i =3D 0; i < p->zero_num; i++) { + void *page =3D p->host + p->zero[i]; + if (!buffer_is_zero(page, p->page_size)) { + memset(page, 0, p->page_size); + } + } + if (sync_needed) { qemu_sem_post(&multifd_recv_state->sem_sync); qemu_sem_wait(&p->sem_sync); --=20 2.35.1 From nobody Wed May 15 06:08:33 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1652223116; cv=none; d=zohomail.com; s=zohoarc; b=crq1U5itoKeyO5h1GtGaG6H5Kawy90EnpKTPqfyrLAl8JWyXdokbTsEWJP/3gi8dQb5XNgLbDzfs9lH/VIFV4klHpz+5PedtMb2xFlD/hMNp7N6KlQc8qF2gMIni4Sg76whybvtS3pX3EaGRXPbhRR/2ixQWGq25yIHo1xsDZew= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1652223116; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=fJx+e2UF8mwpACIt6H6aR95/QNq8iH9YSpFDergikic=; b=WouQc5b09KNjHpJmxtp3p0WcjRqa3fPdn5EDM3pUS03DKJPPnI4oj4LP3weTCow+89pTXVqft8Ek13jxTt/+iJAej4mrxVjwTY0bcIaPIZ7ldo82HBlUxjp9ANCvHFwWtiGBiooDPJEVdhFlKZJaK0Ok5on+TjHgcfZ37lx7bZ8= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1652223116397751.7324301425685; Tue, 10 May 2022 15:51:56 -0700 (PDT) Received: from localhost ([::1]:57276 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1noYhu-0001N1-TS for importer@patchew.org; Tue, 10 May 2022 18:51:54 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:47380) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1noYaL-0004dM-3B for qemu-devel@nongnu.org; Tue, 10 May 2022 18:44:06 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]:57135) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1noYaH-0006J9-Of for qemu-devel@nongnu.org; Tue, 10 May 2022 18:44:04 -0400 Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-424-3Y_TEpVCOj-vfsEIJAcRuQ-1; Tue, 10 May 2022 18:42:49 -0400 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.rdu2.redhat.com [10.11.54.1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 1F1633C021B2; Tue, 10 May 2022 22:42:49 +0000 (UTC) Received: from secure.mitica (unknown [10.39.195.247]) by smtp.corp.redhat.com (Postfix) with ESMTP id 914B140CFD06; Tue, 10 May 2022 22:42:47 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1652222640; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=fJx+e2UF8mwpACIt6H6aR95/QNq8iH9YSpFDergikic=; b=HKL6BAnlcMqDFddYS1m2DSW/VcKOId9eu4M9kRCfcCt/svK4CkhOcLoZo14z4VGGGazj+E AY1OaMhIigDi0qdLeM+jHEF6RHOJQGKszM1nIS6Hn9XuK0yUnJW78QGD/Mhg/6bZNH9A97 accipxvhgAI6tSvaSMQYFWZYChTTC+w= X-MC-Unique: 3Y_TEpVCOj-vfsEIJAcRuQ-1 From: Juan Quintela To: qemu-devel@nongnu.org Cc: Juan Quintela , "Dr. David Alan Gilbert" , Eduardo Habkost , Peter Xu , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , Yanan Wang , Leonardo Bras , Marcel Apfelbaum Subject: [PATCH v6 13/13] migration: Use multifd before we check for the zero page Date: Wed, 11 May 2022 00:42:20 +0200 Message-Id: <20220510224220.5912-14-quintela@redhat.com> In-Reply-To: <20220510224220.5912-1-quintela@redhat.com> References: <20220510224220.5912-1-quintela@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 2.84 on 10.11.54.1 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=quintela@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -28 X-Spam_score: -2.9 X-Spam_bar: -- X-Spam_report: (-2.9 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.082, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1652223118236100001 Content-Type: text/plain; charset="utf-8" So we use multifd to transmit zero pages. Signed-off-by: Juan Quintela --- - Check zero_page property before using new code (Dave) --- migration/ram.c | 32 +++++++++++++++++++++++++++++++- 1 file changed, 31 insertions(+), 1 deletion(-) diff --git a/migration/ram.c b/migration/ram.c index 0a91b87bd2..64e45ba915 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -2181,6 +2181,32 @@ static int ram_save_target_page_legacy(RAMState *rs,= PageSearchStatus *pss) return ram_save_page(rs, pss); } =20 +/** + * ram_save_target_page_multifd: save one target page + * + * Returns the number of pages written + * + * @rs: current RAM state + * @pss: data about the page we want to send + */ +static int ram_save_target_page_multifd(RAMState *rs, PageSearchStatus *ps= s) +{ + RAMBlock *block =3D pss->block; + ram_addr_t offset =3D ((ram_addr_t)pss->page) << TARGET_PAGE_BITS; + int res; + + if (!migration_in_postcopy()) { + return ram_save_multifd_page(rs, block, offset); + } + + res =3D save_zero_page(rs, block, offset); + if (res > 0) { + return res; + } + + return ram_save_page(rs, pss); +} + /** * ram_save_host_page: save a whole host page * @@ -2944,7 +2970,11 @@ static int ram_save_setup(QEMUFile *f, void *opaque) ram_control_before_iterate(f, RAM_CONTROL_SETUP); ram_control_after_iterate(f, RAM_CONTROL_SETUP); =20 - (*rsp)->ram_save_target_page =3D ram_save_target_page_legacy; + if (migrate_use_multifd() && migrate_use_multifd_zero_page()) { + (*rsp)->ram_save_target_page =3D ram_save_target_page_multifd; + } else { + (*rsp)->ram_save_target_page =3D ram_save_target_page_legacy; + } multifd_send_sync_main(f); qemu_put_be64(f, RAM_SAVE_FLAG_EOS); qemu_fflush(f); --=20 2.35.1