From nobody Fri Nov 1 01:22:51 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1653994021; cv=none; d=zohomail.com; s=zohoarc; b=A++w5X2KkGJQbpVRrB2iyRsfcfAkoyBE9Q4yf8z8E5Esa5ibBfeVt9VXa7FmcFsMynopAytMh32Sd/t3pISkeM8kU6R5uYbiYrWIvswAI266ZLcRIAsR705hT7/KIxiZiWbzKWOVj4sR91OhPf4C6Z5nwsRZ2haRjA8MtE+dajs= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1653994021; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=4j2rLNjevgLhfGWu285qSycgTkHK7H5IzCueHbV+jGY=; b=Z1PzmrcEENXmx93XBD12LxOnX/VADk06DzFvLrinVruuxV6HoYMz4IdACi6NWWJdtNwbMaa6WJrsEpFcry+wgCAU8Uw2UqLM1putl3z1ozCyq7IEKebDGp714Gca4xLkloPPL602UyzZF+ldJ7kwtDFBc+6wE49uS4gOuudRG1s= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1653994021356389.54458991023705; Tue, 31 May 2022 03:47:01 -0700 (PDT) Received: from localhost ([::1]:54544 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nvzOs-0005id-60 for importer@patchew.org; Tue, 31 May 2022 06:46:59 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:46416) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nvzLZ-0002wE-Qn for qemu-devel@nongnu.org; Tue, 31 May 2022 06:43:33 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]:60268) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nvzLV-0004dB-NE for qemu-devel@nongnu.org; Tue, 31 May 2022 06:43:31 -0400 Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-135-DhQ7eZrEPYyhcXGOjMv-cA-1; Tue, 31 May 2022 06:43:25 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.rdu2.redhat.com [10.11.54.5]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 57D923C1105A; Tue, 31 May 2022 10:43:25 +0000 (UTC) Received: from secure.mitica (unknown [10.39.192.225]) by smtp.corp.redhat.com (Postfix) with ESMTP id 9ACA91131A; Tue, 31 May 2022 10:43:23 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1653993808; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=4j2rLNjevgLhfGWu285qSycgTkHK7H5IzCueHbV+jGY=; b=RjrMUP9gJF4GXY/waa2lGRoS7DvcYyVgkFsrzDya5qEwUw5xyGs7yP5UOBYcrwKOnA3FCl WhcgRhGsH60TKoX44H+suD4pl03oaCVND8PjhQ7riXL9FvHvgqy7IiZjb/kRzl6dHeMrRm +vsrvmI6pRLEZGPCNB7f3RQ92GrMxS8= X-MC-Unique: DhQ7eZrEPYyhcXGOjMv-cA-1 From: Juan Quintela To: qemu-devel@nongnu.org Cc: Marcel Apfelbaum , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , Yanan Wang , "Dr. David Alan Gilbert" , Juan Quintela , Eduardo Habkost , Peter Xu , Leonardo Bras Subject: [PATCH v7 01/13] multifd: Document the locking of MultiFD{Send/Recv}Params Date: Tue, 31 May 2022 12:43:06 +0200 Message-Id: <20220531104318.7494-2-quintela@redhat.com> In-Reply-To: <20220531104318.7494-1-quintela@redhat.com> References: <20220531104318.7494-1-quintela@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 2.79 on 10.11.54.5 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=quintela@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -21 X-Spam_score: -2.2 X-Spam_bar: -- X-Spam_report: (-2.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.082, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1653994023670100001 Content-Type: text/plain; charset="utf-8" Reorder the structures so we can know if the fields are: - Read only - Their own locking (i.e. sems) - Protected by 'mutex' - Only for the multifd channel Signed-off-by: Juan Quintela Reviewed-by: Dr. David Alan Gilbert --- migration/multifd.h | 90 ++++++++++++++++++++++++++------------------- 1 file changed, 53 insertions(+), 37 deletions(-) diff --git a/migration/multifd.h b/migration/multifd.h index 4d8d89e5e5..345cfdb50c 100644 --- a/migration/multifd.h +++ b/migration/multifd.h @@ -65,7 +65,9 @@ typedef struct { } MultiFDPages_t; =20 typedef struct { - /* this fields are not changed once the thread is created */ + /* Fiields are only written at creating/deletion time */ + /* No lock required for them, they are read only */ + /* channel number */ uint8_t id; /* channel thread name */ @@ -74,39 +76,47 @@ typedef struct { QemuThread thread; /* communication channel */ QIOChannel *c; - /* sem where to wait for more work */ - QemuSemaphore sem; - /* this mutex protects the following parameters */ - QemuMutex mutex; - /* is this channel thread running */ - bool running; - /* should this thread finish */ - bool quit; /* is the yank function registered */ bool registered_yank; + /* packet allocated len */ + uint32_t packet_len; + /* multifd flags for sending ram */ + int write_flags; + + /* sem where to wait for more work */ + QemuSemaphore sem; + /* syncs main thread and channels */ + QemuSemaphore sem_sync; + + /* this mutex protects the following parameters */ + QemuMutex mutex; + /* is this channel thread running */ + bool running; + /* should this thread finish */ + bool quit; + /* multifd flags for each packet */ + uint32_t flags; + /* global number of generated multifd packets */ + uint64_t packet_num; /* thread has work to do */ int pending_job; - /* array of pages to sent */ + /* array of pages to sent. + * The owner of 'pages' depends of 'pending_job' value: + * pending_job =3D=3D 0 -> migration_thread can use it. + * pending_job !=3D 0 -> multifd_channel can use it. + */ MultiFDPages_t *pages; - /* packet allocated len */ - uint32_t packet_len; + + /* thread local variables. No locking required */ + /* pointer to the packet */ MultiFDPacket_t *packet; - /* multifd flags for sending ram */ - int write_flags; - /* multifd flags for each packet */ - uint32_t flags; /* size of the next packet that contains pages */ uint32_t next_packet_size; - /* global number of generated multifd packets */ - uint64_t packet_num; - /* thread local variables */ /* packets sent through this channel */ uint64_t num_packets; /* non zero pages sent through this channel */ uint64_t total_normal_pages; - /* syncs main thread and channels */ - QemuSemaphore sem_sync; /* buffers to send */ struct iovec *iov; /* number of iovs used */ @@ -120,7 +130,9 @@ typedef struct { } MultiFDSendParams; =20 typedef struct { - /* this fields are not changed once the thread is created */ + /* Fiields are only written at creating/deletion time */ + /* No lock required for them, they are read only */ + /* channel number */ uint8_t id; /* channel thread name */ @@ -129,31 +141,35 @@ typedef struct { QemuThread thread; /* communication channel */ QIOChannel *c; + /* packet allocated len */ + uint32_t packet_len; + + /* syncs main thread and channels */ + QemuSemaphore sem_sync; + /* this mutex protects the following parameters */ QemuMutex mutex; /* is this channel thread running */ bool running; /* should this thread finish */ bool quit; + /* multifd flags for each packet */ + uint32_t flags; + /* global number of generated multifd packets */ + uint64_t packet_num; + + /* thread local variables. No locking required */ + + /* pointer to the packet */ + MultiFDPacket_t *packet; + /* size of the next packet that contains pages */ + uint32_t next_packet_size; + /* packets sent through this channel */ + uint64_t num_packets; /* ramblock host address */ uint8_t *host; - /* packet allocated len */ - uint32_t packet_len; - /* pointer to the packet */ - MultiFDPacket_t *packet; - /* multifd flags for each packet */ - uint32_t flags; - /* global number of generated multifd packets */ - uint64_t packet_num; - /* thread local variables */ - /* size of the next packet that contains pages */ - uint32_t next_packet_size; - /* packets sent through this channel */ - uint64_t num_packets; /* non zero pages recv through this channel */ uint64_t total_normal_pages; - /* syncs main thread and channels */ - QemuSemaphore sem_sync; /* buffers to recv */ struct iovec *iov; /* Pages that are not zero */ --=20 2.35.3 From nobody Fri Nov 1 01:22:51 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1653993976492210.967007595135; Tue, 31 May 2022 03:46:16 -0700 (PDT) Received: from localhost ([::1]:54182 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nvzOA-0005U4-QL for importer@patchew.org; Tue, 31 May 2022 06:46:14 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:46428) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nvzLZ-0002wQ-TR for qemu-devel@nongnu.org; Tue, 31 May 2022 06:43:34 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:60194) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nvzLX-0004dW-La for qemu-devel@nongnu.org; Tue, 31 May 2022 06:43:33 -0400 Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-404-gf44T6kmPFyVMbJtLliglg-1; Tue, 31 May 2022 06:43:28 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.rdu2.redhat.com [10.11.54.5]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id A00243C11058; Tue, 31 May 2022 10:43:27 +0000 (UTC) Received: from secure.mitica (unknown [10.39.192.225]) by smtp.corp.redhat.com (Postfix) with ESMTP id A66C282872; Tue, 31 May 2022 10:43:25 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1653993811; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=MQCihQbql/hzpKzebKrBmunbZjGKMbodHwrcDkCZH+M=; b=MMw15N+oVoHdNvKqCtSZU4yRNq1yhVHNKFQL6raImqos2YTJPwaoV6aYuO5QJiT+i+twaK +Cl6gfJoHXmygvi6pJz3S/CnMIqzFAhcyF3bqPAKQ9K0Pp8+FMe6zdc3jgE0505CCQm8GQ tftz18ryiQECUwGana1+r+1mHCef5pI= X-MC-Unique: gf44T6kmPFyVMbJtLliglg-1 From: Juan Quintela To: qemu-devel@nongnu.org Cc: Marcel Apfelbaum , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , Yanan Wang , "Dr. David Alan Gilbert" , Juan Quintela , Eduardo Habkost , Peter Xu , Leonardo Bras Subject: [PATCH v7 02/13] multifd: Create page_size fields into both MultiFD{Recv, Send}Params Date: Tue, 31 May 2022 12:43:07 +0200 Message-Id: <20220531104318.7494-3-quintela@redhat.com> In-Reply-To: <20220531104318.7494-1-quintela@redhat.com> References: <20220531104318.7494-1-quintela@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 2.79 on 10.11.54.5 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=quintela@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -21 X-Spam_score: -2.2 X-Spam_bar: -- X-Spam_report: (-2.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.082, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZM-MESSAGEID: 1653993977605100001 Content-Type: text/plain; charset="utf-8" We were calling qemu_target_page_size() left and right. Signed-off-by: Juan Quintela Reviewed-by: Dr. David Alan Gilbert --- migration/multifd.h | 4 ++++ migration/multifd-zlib.c | 12 +++++------- migration/multifd-zstd.c | 12 +++++------- migration/multifd.c | 18 ++++++++---------- 4 files changed, 22 insertions(+), 24 deletions(-) diff --git a/migration/multifd.h b/migration/multifd.h index 345cfdb50c..9e07dd00f4 100644 --- a/migration/multifd.h +++ b/migration/multifd.h @@ -80,6 +80,8 @@ typedef struct { bool registered_yank; /* packet allocated len */ uint32_t packet_len; + /* guest page size */ + uint32_t page_size; /* multifd flags for sending ram */ int write_flags; =20 @@ -143,6 +145,8 @@ typedef struct { QIOChannel *c; /* packet allocated len */ uint32_t packet_len; + /* guest page size */ + uint32_t page_size; =20 /* syncs main thread and channels */ QemuSemaphore sem_sync; diff --git a/migration/multifd-zlib.c b/migration/multifd-zlib.c index 3a7ae44485..28349ff2e0 100644 --- a/migration/multifd-zlib.c +++ b/migration/multifd-zlib.c @@ -100,7 +100,6 @@ static void zlib_send_cleanup(MultiFDSendParams *p, Err= or **errp) static int zlib_send_prepare(MultiFDSendParams *p, Error **errp) { struct zlib_data *z =3D p->data; - size_t page_size =3D qemu_target_page_size(); z_stream *zs =3D &z->zs; uint32_t out_size =3D 0; int ret; @@ -114,7 +113,7 @@ static int zlib_send_prepare(MultiFDSendParams *p, Erro= r **errp) flush =3D Z_SYNC_FLUSH; } =20 - zs->avail_in =3D page_size; + zs->avail_in =3D p->page_size; zs->next_in =3D p->pages->block->host + p->normal[i]; =20 zs->avail_out =3D available; @@ -220,12 +219,11 @@ static void zlib_recv_cleanup(MultiFDRecvParams *p) static int zlib_recv_pages(MultiFDRecvParams *p, Error **errp) { struct zlib_data *z =3D p->data; - size_t page_size =3D qemu_target_page_size(); z_stream *zs =3D &z->zs; uint32_t in_size =3D p->next_packet_size; /* we measure the change of total_out */ uint32_t out_size =3D zs->total_out; - uint32_t expected_size =3D p->normal_num * page_size; + uint32_t expected_size =3D p->normal_num * p->page_size; uint32_t flags =3D p->flags & MULTIFD_FLAG_COMPRESSION_MASK; int ret; int i; @@ -252,7 +250,7 @@ static int zlib_recv_pages(MultiFDRecvParams *p, Error = **errp) flush =3D Z_SYNC_FLUSH; } =20 - zs->avail_out =3D page_size; + zs->avail_out =3D p->page_size; zs->next_out =3D p->host + p->normal[i]; =20 /* @@ -266,8 +264,8 @@ static int zlib_recv_pages(MultiFDRecvParams *p, Error = **errp) do { ret =3D inflate(zs, flush); } while (ret =3D=3D Z_OK && zs->avail_in - && (zs->total_out - start) < page_size); - if (ret =3D=3D Z_OK && (zs->total_out - start) < page_size) { + && (zs->total_out - start) < p->page_size); + if (ret =3D=3D Z_OK && (zs->total_out - start) < p->page_size) { error_setg(errp, "multifd %u: inflate generated too few output= ", p->id); return -1; diff --git a/migration/multifd-zstd.c b/migration/multifd-zstd.c index d788d309f2..f4a8e1ed1f 100644 --- a/migration/multifd-zstd.c +++ b/migration/multifd-zstd.c @@ -113,7 +113,6 @@ static void zstd_send_cleanup(MultiFDSendParams *p, Err= or **errp) static int zstd_send_prepare(MultiFDSendParams *p, Error **errp) { struct zstd_data *z =3D p->data; - size_t page_size =3D qemu_target_page_size(); int ret; uint32_t i; =20 @@ -128,7 +127,7 @@ static int zstd_send_prepare(MultiFDSendParams *p, Erro= r **errp) flush =3D ZSTD_e_flush; } z->in.src =3D p->pages->block->host + p->normal[i]; - z->in.size =3D page_size; + z->in.size =3D p->page_size; z->in.pos =3D 0; =20 /* @@ -241,8 +240,7 @@ static int zstd_recv_pages(MultiFDRecvParams *p, Error = **errp) { uint32_t in_size =3D p->next_packet_size; uint32_t out_size =3D 0; - size_t page_size =3D qemu_target_page_size(); - uint32_t expected_size =3D p->normal_num * page_size; + uint32_t expected_size =3D p->normal_num * p->page_size; uint32_t flags =3D p->flags & MULTIFD_FLAG_COMPRESSION_MASK; struct zstd_data *z =3D p->data; int ret; @@ -265,7 +263,7 @@ static int zstd_recv_pages(MultiFDRecvParams *p, Error = **errp) =20 for (i =3D 0; i < p->normal_num; i++) { z->out.dst =3D p->host + p->normal[i]; - z->out.size =3D page_size; + z->out.size =3D p->page_size; z->out.pos =3D 0; =20 /* @@ -279,8 +277,8 @@ static int zstd_recv_pages(MultiFDRecvParams *p, Error = **errp) do { ret =3D ZSTD_decompressStream(z->zds, &z->out, &z->in); } while (ret > 0 && (z->in.size - z->in.pos > 0) - && (z->out.pos < page_size)); - if (ret > 0 && (z->out.pos < page_size)) { + && (z->out.pos < p->page_size)); + if (ret > 0 && (z->out.pos < p->page_size)) { error_setg(errp, "multifd %u: decompressStream buffer too smal= l", p->id); return -1; diff --git a/migration/multifd.c b/migration/multifd.c index 9282ab6aa4..7505aa3412 100644 --- a/migration/multifd.c +++ b/migration/multifd.c @@ -87,15 +87,14 @@ static void nocomp_send_cleanup(MultiFDSendParams *p, E= rror **errp) static int nocomp_send_prepare(MultiFDSendParams *p, Error **errp) { MultiFDPages_t *pages =3D p->pages; - size_t page_size =3D qemu_target_page_size(); =20 for (int i =3D 0; i < p->normal_num; i++) { p->iov[p->iovs_num].iov_base =3D pages->block->host + p->normal[i]; - p->iov[p->iovs_num].iov_len =3D page_size; + p->iov[p->iovs_num].iov_len =3D p->page_size; p->iovs_num++; } =20 - p->next_packet_size =3D p->normal_num * page_size; + p->next_packet_size =3D p->normal_num * p->page_size; p->flags |=3D MULTIFD_FLAG_NOCOMP; return 0; } @@ -139,7 +138,6 @@ static void nocomp_recv_cleanup(MultiFDRecvParams *p) static int nocomp_recv_pages(MultiFDRecvParams *p, Error **errp) { uint32_t flags =3D p->flags & MULTIFD_FLAG_COMPRESSION_MASK; - size_t page_size =3D qemu_target_page_size(); =20 if (flags !=3D MULTIFD_FLAG_NOCOMP) { error_setg(errp, "multifd %u: flags received %x flags expected %x", @@ -148,7 +146,7 @@ static int nocomp_recv_pages(MultiFDRecvParams *p, Erro= r **errp) } for (int i =3D 0; i < p->normal_num; i++) { p->iov[i].iov_base =3D p->host + p->normal[i]; - p->iov[i].iov_len =3D page_size; + p->iov[i].iov_len =3D p->page_size; } return qio_channel_readv_all(p->c, p->iov, p->normal_num, errp); } @@ -281,8 +279,7 @@ static void multifd_send_fill_packet(MultiFDSendParams = *p) static int multifd_recv_unfill_packet(MultiFDRecvParams *p, Error **errp) { MultiFDPacket_t *packet =3D p->packet; - size_t page_size =3D qemu_target_page_size(); - uint32_t page_count =3D MULTIFD_PACKET_SIZE / page_size; + uint32_t page_count =3D MULTIFD_PACKET_SIZE / p->page_size; RAMBlock *block; int i; =20 @@ -344,7 +341,7 @@ static int multifd_recv_unfill_packet(MultiFDRecvParams= *p, Error **errp) for (i =3D 0; i < p->normal_num; i++) { uint64_t offset =3D be64_to_cpu(packet->offset[i]); =20 - if (offset > (block->used_length - page_size)) { + if (offset > (block->used_length - p->page_size)) { error_setg(errp, "multifd: offset too long %" PRIu64 " (max " RAM_ADDR_FMT ")", offset, block->used_length); @@ -433,8 +430,7 @@ static int multifd_send_pages(QEMUFile *f) p->packet_num =3D multifd_send_state->packet_num++; multifd_send_state->pages =3D p->pages; p->pages =3D pages; - transferred =3D ((uint64_t) pages->num) * qemu_target_page_size() - + p->packet_len; + transferred =3D ((uint64_t) pages->num) * p->page_size + p->packet_len; qemu_file_update_transfer(f, transferred); ram_counters.multifd_bytes +=3D transferred; ram_counters.transferred +=3D transferred; @@ -939,6 +935,7 @@ int multifd_save_setup(Error **errp) /* We need one extra place for the packet header */ p->iov =3D g_new0(struct iovec, page_count + 1); p->normal =3D g_new0(ram_addr_t, page_count); + p->page_size =3D qemu_target_page_size(); =20 if (migrate_use_zero_copy_send()) { p->write_flags =3D QIO_CHANNEL_WRITE_FLAG_ZERO_COPY; @@ -1186,6 +1183,7 @@ int multifd_load_setup(Error **errp) p->name =3D g_strdup_printf("multifdrecv_%d", i); p->iov =3D g_new0(struct iovec, page_count); p->normal =3D g_new0(ram_addr_t, page_count); + p->page_size =3D qemu_target_page_size(); } =20 for (i =3D 0; i < thread_count; i++) { --=20 2.35.3 From nobody Fri Nov 1 01:22:51 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1653994430587810.1591891682253; Tue, 31 May 2022 03:53:50 -0700 (PDT) Received: from localhost ([::1]:33936 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nvzVV-0002jM-Fe for importer@patchew.org; Tue, 31 May 2022 06:53:49 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:46456) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nvzLd-00030N-Gr for qemu-devel@nongnu.org; Tue, 31 May 2022 06:43:37 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:22160) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nvzLb-0004e0-UE for qemu-devel@nongnu.org; Tue, 31 May 2022 06:43:37 -0400 Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-470-il_GdgQtM6y92Ufjr3AZcg-1; Tue, 31 May 2022 06:43:30 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.rdu2.redhat.com [10.11.54.5]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id BBF3B101A56C; Tue, 31 May 2022 10:43:29 +0000 (UTC) Received: from secure.mitica (unknown [10.39.192.225]) by smtp.corp.redhat.com (Postfix) with ESMTP id 0635C10725; Tue, 31 May 2022 10:43:27 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1653993815; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=1jaimjsfu5x74+m9j1TFzu9/oLD4/Ct1XsRdfgJ7OEo=; b=aCTkM6dQmAU9uJ4y9oRumWX0gkQ8CZFkEOZHRBVFYKPlRL6h3PK574/EqQo61QdLO6UaC0 1KfgdFqd7p1kHV145HxORChBZnFz83gGjjoaLUBuDE5OxALWV0L+4eIWClH7ArbpusMNGZ gLM+Lwj+71qvndo8Gnj5M6/e5kB2dRA= X-MC-Unique: il_GdgQtM6y92Ufjr3AZcg-1 From: Juan Quintela To: qemu-devel@nongnu.org Cc: Marcel Apfelbaum , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , Yanan Wang , "Dr. David Alan Gilbert" , Juan Quintela , Eduardo Habkost , Peter Xu , Leonardo Bras Subject: [PATCH v7 03/13] multifd: Create page_count fields into both MultiFD{Recv, Send}Params Date: Tue, 31 May 2022 12:43:08 +0200 Message-Id: <20220531104318.7494-4-quintela@redhat.com> In-Reply-To: <20220531104318.7494-1-quintela@redhat.com> References: <20220531104318.7494-1-quintela@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 2.79 on 10.11.54.5 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=quintela@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -21 X-Spam_score: -2.2 X-Spam_bar: -- X-Spam_report: (-2.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.082, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZM-MESSAGEID: 1653994431606100001 Content-Type: text/plain; charset="utf-8" We were recalculating it left and right. We plan to change that values on next patches. Signed-off-by: Juan Quintela Reviewed-by: Dr. David Alan Gilbert --- migration/multifd.h | 4 ++++ migration/multifd.c | 7 ++++--- 2 files changed, 8 insertions(+), 3 deletions(-) diff --git a/migration/multifd.h b/migration/multifd.h index 9e07dd00f4..71f49b4063 100644 --- a/migration/multifd.h +++ b/migration/multifd.h @@ -82,6 +82,8 @@ typedef struct { uint32_t packet_len; /* guest page size */ uint32_t page_size; + /* number of pages in a full packet */ + uint32_t page_count; /* multifd flags for sending ram */ int write_flags; =20 @@ -147,6 +149,8 @@ typedef struct { uint32_t packet_len; /* guest page size */ uint32_t page_size; + /* number of pages in a full packet */ + uint32_t page_count; =20 /* syncs main thread and channels */ QemuSemaphore sem_sync; diff --git a/migration/multifd.c b/migration/multifd.c index 7505aa3412..166246b9b7 100644 --- a/migration/multifd.c +++ b/migration/multifd.c @@ -279,7 +279,6 @@ static void multifd_send_fill_packet(MultiFDSendParams = *p) static int multifd_recv_unfill_packet(MultiFDRecvParams *p, Error **errp) { MultiFDPacket_t *packet =3D p->packet; - uint32_t page_count =3D MULTIFD_PACKET_SIZE / p->page_size; RAMBlock *block; int i; =20 @@ -306,10 +305,10 @@ static int multifd_recv_unfill_packet(MultiFDRecvPara= ms *p, Error **errp) * If we received a packet that is 100 times bigger than expected * just stop migration. It is a magic number. */ - if (packet->pages_alloc > page_count) { + if (packet->pages_alloc > p->page_count) { error_setg(errp, "multifd: received packet " "with size %u and expected a size of %u", - packet->pages_alloc, page_count) ; + packet->pages_alloc, p->page_count) ; return -1; } =20 @@ -936,6 +935,7 @@ int multifd_save_setup(Error **errp) p->iov =3D g_new0(struct iovec, page_count + 1); p->normal =3D g_new0(ram_addr_t, page_count); p->page_size =3D qemu_target_page_size(); + p->page_count =3D page_count; =20 if (migrate_use_zero_copy_send()) { p->write_flags =3D QIO_CHANNEL_WRITE_FLAG_ZERO_COPY; @@ -1183,6 +1183,7 @@ int multifd_load_setup(Error **errp) p->name =3D g_strdup_printf("multifdrecv_%d", i); p->iov =3D g_new0(struct iovec, page_count); p->normal =3D g_new0(ram_addr_t, page_count); + p->page_count =3D page_count; p->page_size =3D qemu_target_page_size(); } =20 --=20 2.35.3 From nobody Fri Nov 1 01:22:51 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1653994746; cv=none; d=zohomail.com; s=zohoarc; b=DW3ttg1u+BFORRi/7sP76J3Zg/axAED9Hx+WBLL/00l9tupxCbMF353JjOTcYdEGHgWcLhzhXynuNjO+rWs0pMkMfLUiIbc7L2sIoMgzdCgqlcb7WhDX7BEZ5x7XNjqWVDue7DkyoNO4M2fGV7QXJaPk0YmpV7b0lcFRrTxftlU= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1653994746; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=y6TPDPWNSLvB1rRBWx3h74hQFgsR0CXV8yULLTaUiIY=; b=Pj0OLlIpsuFpcBPejA8SAVirR+WZa4jx+0fA0LkJUWGjdXYN21MTI+xBUmxtteYeA0BdKA+xuXEmI/n4Jc3miGszmFGhfYKgU3MuerOuUWWCU7hF15ko/UHJ8bGTdFkePipwBK1fdrHb0dWNsIrHicZc8F094+DbSpIksQeNITA= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 165399474623610.586041232878529; Tue, 31 May 2022 03:59:06 -0700 (PDT) Received: from localhost ([::1]:40324 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nvzab-0007A4-44 for importer@patchew.org; Tue, 31 May 2022 06:59:05 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:46454) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nvzLd-00030E-Ia for qemu-devel@nongnu.org; Tue, 31 May 2022 06:43:37 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:54340) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nvzLb-0004e1-U6 for qemu-devel@nongnu.org; Tue, 31 May 2022 06:43:37 -0400 Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-232-s3OcjtI5MRKGSdsCOxKYFQ-1; Tue, 31 May 2022 06:43:32 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.rdu2.redhat.com [10.11.54.5]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id CB33B858F11; Tue, 31 May 2022 10:43:31 +0000 (UTC) Received: from secure.mitica (unknown [10.39.192.225]) by smtp.corp.redhat.com (Postfix) with ESMTP id 15A118287E; Tue, 31 May 2022 10:43:29 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1653993815; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=y6TPDPWNSLvB1rRBWx3h74hQFgsR0CXV8yULLTaUiIY=; b=ZxVaBpu6htAcbkfEUeT/G/vMYGZPBJmzpAsf2j4ypsG/EIk+gRELpn0wua/VlPMwpPTBzd uk6/4uueZnPgVOyA3w9wlwWc9wd7nKzsCy3l4S5T5rm2nJaXXdiitf8Vfry2oVvwvuC8Rq VzcSAXPnu8EmQlkt7UlLE1pEon0siyg= X-MC-Unique: s3OcjtI5MRKGSdsCOxKYFQ-1 From: Juan Quintela To: qemu-devel@nongnu.org Cc: Marcel Apfelbaum , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , Yanan Wang , "Dr. David Alan Gilbert" , Juan Quintela , Eduardo Habkost , Peter Xu , Leonardo Bras Subject: [PATCH v7 04/13] migration: Export ram_transferred_ram() Date: Tue, 31 May 2022 12:43:09 +0200 Message-Id: <20220531104318.7494-5-quintela@redhat.com> In-Reply-To: <20220531104318.7494-1-quintela@redhat.com> References: <20220531104318.7494-1-quintela@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 2.79 on 10.11.54.5 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=quintela@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -21 X-Spam_score: -2.2 X-Spam_bar: -- X-Spam_report: (-2.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.082, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1653994748212100001 Content-Type: text/plain; charset="utf-8" Signed-off-by: Juan Quintela Reviewed-by: Dr. David Alan Gilbert --- migration/ram.h | 2 ++ migration/ram.c | 2 +- 2 files changed, 3 insertions(+), 1 deletion(-) diff --git a/migration/ram.h b/migration/ram.h index ded0a3a086..7b641adc55 100644 --- a/migration/ram.h +++ b/migration/ram.h @@ -65,6 +65,8 @@ int ram_load_postcopy(QEMUFile *f); =20 void ram_handle_compressed(void *host, uint8_t ch, uint64_t size); =20 +void ram_transferred_add(uint64_t bytes); + int ramblock_recv_bitmap_test(RAMBlock *rb, void *host_addr); bool ramblock_recv_bitmap_test_byte_offset(RAMBlock *rb, uint64_t byte_off= set); void ramblock_recv_bitmap_set(RAMBlock *rb, void *host_addr); diff --git a/migration/ram.c b/migration/ram.c index 5f5e37f64d..30b0680942 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -394,7 +394,7 @@ uint64_t ram_bytes_remaining(void) =20 MigrationStats ram_counters; =20 -static void ram_transferred_add(uint64_t bytes) +void ram_transferred_add(uint64_t bytes) { if (runstate_is_running()) { ram_counters.precopy_bytes +=3D bytes; --=20 2.35.3 From nobody Fri Nov 1 01:22:51 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1653994442; cv=none; d=zohomail.com; s=zohoarc; b=DX5X6mlE9B9IIvF+AItvSAphCQVFm9H6TvgkAirYuQDYmqoohTLrA3mTa0aBHqwWwLGflWOvHsBRVKhM/6DaQFi/qmimjVzs19Z8Gosv4nEWUWCAm+li9Nn4W6ULGGRuTlACdSwayeeJfp3dRCdVJ5kJWFSGo8HyHthxnq+i4xg= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1653994442; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=AlEVbx27k4NJKxFP5FmrzMb1eR3akXHSJ5ErDvKdWBU=; b=YXFsaYW39WkWft70+vx/xLsNCPmm/Hz7SEz3KrxI9DzYd0LbYOCMyfv2iWb0U01urFzs6VxatF2v9J0sWRqOafx7AHjVaxdsEjPVvGPMbDvmanrF3AuXOmGqvmqWHKgA6wz7QoQdlcZqPrPaOaYb7nkdQ2ABZAtwF/kzkErxu9M= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 16539944428101006.5433491040737; Tue, 31 May 2022 03:54:02 -0700 (PDT) Received: from localhost ([::1]:34388 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nvzVh-00033S-El for importer@patchew.org; Tue, 31 May 2022 06:54:01 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:46470) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nvzLg-000377-2u for qemu-devel@nongnu.org; Tue, 31 May 2022 06:43:40 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]:55981) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nvzLe-0004eK-CM for qemu-devel@nongnu.org; Tue, 31 May 2022 06:43:39 -0400 Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-669-_vrRBAMMMGGlhG8rkHgYJw-1; Tue, 31 May 2022 06:43:34 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.rdu2.redhat.com [10.11.54.5]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id DF73680A0B9; Tue, 31 May 2022 10:43:33 +0000 (UTC) Received: from secure.mitica (unknown [10.39.192.225]) by smtp.corp.redhat.com (Postfix) with ESMTP id 2DF811131A; Tue, 31 May 2022 10:43:32 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1653993817; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=AlEVbx27k4NJKxFP5FmrzMb1eR3akXHSJ5ErDvKdWBU=; b=deKw+Nat13FDMwXFIf1kdTkjh5iAeKlgDfJjCHqWhQchPcxyHoEO8QqEnu1p7XLpHMnlN+ MA7c4u1GDwf3Wv4sVBzyZlLceJ2mgYvewqjAMfPPWS2V39wtFF48aw2snsJEq6Sr41v061 wEoP3LeiL7yfPpF+9yhDBRpjk1PBfG8= X-MC-Unique: _vrRBAMMMGGlhG8rkHgYJw-1 From: Juan Quintela To: qemu-devel@nongnu.org Cc: Marcel Apfelbaum , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , Yanan Wang , "Dr. David Alan Gilbert" , Juan Quintela , Eduardo Habkost , Peter Xu , Leonardo Bras Subject: [PATCH v7 05/13] multifd: Count the number of bytes sent correctly Date: Tue, 31 May 2022 12:43:10 +0200 Message-Id: <20220531104318.7494-6-quintela@redhat.com> In-Reply-To: <20220531104318.7494-1-quintela@redhat.com> References: <20220531104318.7494-1-quintela@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 2.79 on 10.11.54.5 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=quintela@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -21 X-Spam_score: -2.2 X-Spam_bar: -- X-Spam_report: (-2.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.082, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1653994443492100002 Content-Type: text/plain; charset="utf-8" Current code asumes that all pages are whole. That is not true for example for compression already. Fix it for creating a new field ->sent_bytes that includes it. All ram_counters are used only from the migration thread, so we have two options: - put a mutex and fill everything when we sent it (not only ram_counters, also qemu_file->xfer_bytes). - Create a local variable that implements how much has been sent through each channel. And when we push another packet, we "add" the previous stats. I choose two due to less changes overall. On the previous code we increase transferred and then we sent. Current code goes the other way around. It sents the data, and after the fact, it updates the counters. Notice that each channel can have a maximum of half a megabyte of data without counting, so it is not very important. Signed-off-by: Juan Quintela Reviewed-by: Dr. David Alan Gilbert --- migration/multifd.h | 2 ++ migration/multifd.c | 14 ++++++-------- 2 files changed, 8 insertions(+), 8 deletions(-) diff --git a/migration/multifd.h b/migration/multifd.h index 71f49b4063..8a45dda58c 100644 --- a/migration/multifd.h +++ b/migration/multifd.h @@ -102,6 +102,8 @@ typedef struct { uint32_t flags; /* global number of generated multifd packets */ uint64_t packet_num; + /* How many bytes have we sent on the last packet */ + uint64_t sent_bytes; /* thread has work to do */ int pending_job; /* array of pages to sent. diff --git a/migration/multifd.c b/migration/multifd.c index 166246b9b7..eef47c274f 100644 --- a/migration/multifd.c +++ b/migration/multifd.c @@ -394,7 +394,6 @@ static int multifd_send_pages(QEMUFile *f) static int next_channel; MultiFDSendParams *p =3D NULL; /* make happy gcc */ MultiFDPages_t *pages =3D multifd_send_state->pages; - uint64_t transferred; =20 if (qatomic_read(&multifd_send_state->exiting)) { return -1; @@ -429,10 +428,10 @@ static int multifd_send_pages(QEMUFile *f) p->packet_num =3D multifd_send_state->packet_num++; multifd_send_state->pages =3D p->pages; p->pages =3D pages; - transferred =3D ((uint64_t) pages->num) * p->page_size + p->packet_len; - qemu_file_update_transfer(f, transferred); - ram_counters.multifd_bytes +=3D transferred; - ram_counters.transferred +=3D transferred; + ram_transferred_add(p->sent_bytes); + ram_counters.multifd_bytes +=3D p->sent_bytes; + qemu_file_update_transfer(f, p->sent_bytes); + p->sent_bytes =3D 0; qemu_mutex_unlock(&p->mutex); qemu_sem_post(&p->sem); =20 @@ -605,9 +604,6 @@ int multifd_send_sync_main(QEMUFile *f) p->packet_num =3D multifd_send_state->packet_num++; p->flags |=3D MULTIFD_FLAG_SYNC; p->pending_job++; - qemu_file_update_transfer(f, p->packet_len); - ram_counters.multifd_bytes +=3D p->packet_len; - ram_counters.transferred +=3D p->packet_len; qemu_mutex_unlock(&p->mutex); qemu_sem_post(&p->sem); =20 @@ -712,6 +708,8 @@ static void *multifd_send_thread(void *opaque) } =20 qemu_mutex_lock(&p->mutex); + p->sent_bytes +=3D p->packet_len;; + p->sent_bytes +=3D p->next_packet_size; p->pending_job--; qemu_mutex_unlock(&p->mutex); =20 --=20 2.35.3 From nobody Fri Nov 1 01:22:51 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1653994754; cv=none; d=zohomail.com; s=zohoarc; b=jrkb8LmuGFA3aA4Fp+Hg3BN87nQI5wyxkAhDXCzZHJ0zsQ/mJ5ghm7tuvJaupVwfNML+F1rTgnlVdUq/IfdoVjFsqr2P8m8l+TMXv/ShAcGGPGmdQx6+ulip8rG5Um+TGP8/iDfDfFwQkSbnUYKAfgnQ+nlXZ8CjBVL0+DJJJZE= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1653994754; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=Pe3jHr3FUhCx5bKb1p681cF2NkGCgqVuIWV/2hR/CGI=; b=eiJHEL7XxbxU5VOXp+aZTra8UkJUNfv4mFkcWnn9Sr6h1GriFQ0X6YbdqoYT5QMIVVStEzc6ORqKmMZ7i0TWxkRe+g/lNR1V2Ilm4oucd4XnobHVmZW04nMV/1G4F2/Dmssj3AuuUsb0a7jGAbT7NP6C2uOmD2oWRel2IhnPjqc= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1653994754161566.1457361831348; Tue, 31 May 2022 03:59:14 -0700 (PDT) Received: from localhost ([::1]:40768 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nvzaj-0007Tp-2Y for importer@patchew.org; Tue, 31 May 2022 06:59:13 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:46484) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nvzLj-0003Dp-5E for qemu-devel@nongnu.org; Tue, 31 May 2022 06:43:43 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]:30785) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nvzLg-0004eV-66 for qemu-devel@nongnu.org; Tue, 31 May 2022 06:43:41 -0400 Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-641-LgyiDXyLNNq_d7eZPGclrg-1; Tue, 31 May 2022 06:43:36 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.rdu2.redhat.com [10.11.54.5]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id D96AD2809CAA; Tue, 31 May 2022 10:43:35 +0000 (UTC) Received: from secure.mitica (unknown [10.39.192.225]) by smtp.corp.redhat.com (Postfix) with ESMTP id 3D03A10725; Tue, 31 May 2022 10:43:34 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1653993819; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Pe3jHr3FUhCx5bKb1p681cF2NkGCgqVuIWV/2hR/CGI=; b=JjKts+II91XwhurNrwLkgVQGv4oZ8YO77u3UtVUOqZbjpj7x2NO8XSvJwU178qIiTfj5y0 KXrgY7aamWyTNNTTkilTreqCR9wE0S4wT6VMZ2bpjNnKgk+ix01+/w8Eq/ACzUWO47alha Ox2+5KTS+gazrA7MsTIUTqn0lA5QLEw= X-MC-Unique: LgyiDXyLNNq_d7eZPGclrg-1 From: Juan Quintela To: qemu-devel@nongnu.org Cc: Marcel Apfelbaum , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , Yanan Wang , "Dr. David Alan Gilbert" , Juan Quintela , Eduardo Habkost , Peter Xu , Leonardo Bras Subject: [PATCH v7 06/13] migration: Make ram_save_target_page() a pointer Date: Tue, 31 May 2022 12:43:11 +0200 Message-Id: <20220531104318.7494-7-quintela@redhat.com> In-Reply-To: <20220531104318.7494-1-quintela@redhat.com> References: <20220531104318.7494-1-quintela@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 2.79 on 10.11.54.5 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=quintela@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -21 X-Spam_score: -2.2 X-Spam_bar: -- X-Spam_report: (-2.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.082, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1653994755882100001 Content-Type: text/plain; charset="utf-8" We are going to create a new function for multifd latest in the series. Signed-off-by: Juan Quintela Reviewed-by: Dr. David Alan Gilbert --- migration/ram.c | 12 ++++++++---- 1 file changed, 8 insertions(+), 4 deletions(-) diff --git a/migration/ram.c b/migration/ram.c index 30b0680942..71506b1b20 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -295,6 +295,9 @@ struct RAMSrcPageRequest { QSIMPLEQ_ENTRY(RAMSrcPageRequest) next_req; }; =20 +typedef struct RAMState RAMState; +typedef struct PageSearchStatus PageSearchStatus; + /* State of RAM for migration */ struct RAMState { /* QEMUFile used for this migration */ @@ -349,8 +352,8 @@ struct RAMState { /* Queue of outstanding page requests from the destination */ QemuMutex src_page_req_mutex; QSIMPLEQ_HEAD(, RAMSrcPageRequest) src_page_requests; + int (*ram_save_target_page)(RAMState *rs, PageSearchStatus *pss); }; -typedef struct RAMState RAMState; =20 static RAMState *ram_state; =20 @@ -2132,14 +2135,14 @@ static bool save_compress_page(RAMState *rs, RAMBlo= ck *block, ram_addr_t offset) } =20 /** - * ram_save_target_page: save one target page + * ram_save_target_page_legacy: save one target page * * Returns the number of pages written * * @rs: current RAM state * @pss: data about the page we want to send */ -static int ram_save_target_page(RAMState *rs, PageSearchStatus *pss) +static int ram_save_target_page_legacy(RAMState *rs, PageSearchStatus *pss) { RAMBlock *block =3D pss->block; ram_addr_t offset =3D ((ram_addr_t)pss->page) << TARGET_PAGE_BITS; @@ -2214,7 +2217,7 @@ static int ram_save_host_page(RAMState *rs, PageSearc= hStatus *pss) do { /* Check the pages is dirty and if it is send it */ if (migration_bitmap_clear_dirty(rs, pss->block, pss->page)) { - tmppages =3D ram_save_target_page(rs, pss); + tmppages =3D rs->ram_save_target_page(rs, pss); if (tmppages < 0) { return tmppages; } @@ -2944,6 +2947,7 @@ static int ram_save_setup(QEMUFile *f, void *opaque) ram_control_before_iterate(f, RAM_CONTROL_SETUP); ram_control_after_iterate(f, RAM_CONTROL_SETUP); =20 + (*rsp)->ram_save_target_page =3D ram_save_target_page_legacy; ret =3D multifd_send_sync_main(f); if (ret < 0) { return ret; --=20 2.35.3 From nobody Fri Nov 1 01:22:51 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1653995296; cv=none; d=zohomail.com; s=zohoarc; b=GHrff0BvuS1Zl54QS4N5q4+/cHey0w+CppwJ2ApdUlyOO3H7mxS9onKBUCu9HlafxuwrNgpF+IrqnEBLf6G6NLXA8u7rBhahakSKDATtqq/7B4H0vAbD3CC2UhzROMz/n2dGKoJjxWXDflsGr3uyIrgI7vvggrjhdabKCvYK5pI= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1653995296; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=IS8/A1+2LN3G+NxGJwHePXx4Wgtz98BoC2Tycoao7+8=; b=C6brbVGUQ0hBmfFLtzOtyIi3w9cCxNBMdgZVVjNYESW+KWijLuR4gvo/d8nZ95RcET4ZWhROfOsLquHq1e2kjd6FVJ8ywzKzPYcxKRw7vAO7EmIXZWsY65CfVCfsNJyrt6N9jO+2uo/V6wPUeS/eq/IoqBGElaspj2MqkD70akQ= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1653995296811315.14988774424455; Tue, 31 May 2022 04:08:16 -0700 (PDT) Received: from localhost ([::1]:46114 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nvzjS-0003Dw-BM for importer@patchew.org; Tue, 31 May 2022 07:08:14 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:46506) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nvzLl-0003K8-6q for qemu-devel@nongnu.org; Tue, 31 May 2022 06:43:45 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:49780) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nvzLj-0004ei-D1 for qemu-devel@nongnu.org; Tue, 31 May 2022 06:43:44 -0400 Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-480-YIvHgRnONh63QV013ZJgtA-1; Tue, 31 May 2022 06:43:38 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.rdu2.redhat.com [10.11.54.5]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 120301D32362; Tue, 31 May 2022 10:43:38 +0000 (UTC) Received: from secure.mitica (unknown [10.39.192.225]) by smtp.corp.redhat.com (Postfix) with ESMTP id 35B811131A; Tue, 31 May 2022 10:43:36 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1653993821; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=IS8/A1+2LN3G+NxGJwHePXx4Wgtz98BoC2Tycoao7+8=; b=HmPdkAxLW1m0yezJ4UiTuhU4U7E8ozlBO0QrAmMxzrQKqYDhO2Unh+3fcdGgUNMPNXHB7O B5V0NZhDX7NtileQpni+BEBn2TlIa2PiS2aaGCuyG4OVvCASkYmKlJMhpYfIu6mVgrTOWt UZC/mDApKnfZO/ZmM08ITX8N5iFQyIE= X-MC-Unique: YIvHgRnONh63QV013ZJgtA-1 From: Juan Quintela To: qemu-devel@nongnu.org Cc: Marcel Apfelbaum , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , Yanan Wang , "Dr. David Alan Gilbert" , Juan Quintela , Eduardo Habkost , Peter Xu , Leonardo Bras Subject: [PATCH v7 07/13] multifd: Make flags field thread local Date: Tue, 31 May 2022 12:43:12 +0200 Message-Id: <20220531104318.7494-8-quintela@redhat.com> In-Reply-To: <20220531104318.7494-1-quintela@redhat.com> References: <20220531104318.7494-1-quintela@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 2.79 on 10.11.54.5 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=quintela@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -21 X-Spam_score: -2.2 X-Spam_bar: -- X-Spam_report: (-2.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.082, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1653995297652100001 Content-Type: text/plain; charset="utf-8" Use of flags with respect to locking was incensistant. For the sending side: - it was set to 0 with mutex held on the multifd channel. - MULTIFD_FLAG_SYNC was set with mutex held on the migration thread. - Everything else was done without the mutex held on the multifd channel. On the reception side, it is not used on the migration thread, only on the multifd channels threads. So we move it to the multifd channels thread only variables, and we introduce a new bool sync_needed on the send side to pass that information. Signed-off-by: Juan Quintela --- migration/multifd.h | 10 ++++++---- migration/multifd.c | 23 +++++++++++++---------- 2 files changed, 19 insertions(+), 14 deletions(-) diff --git a/migration/multifd.h b/migration/multifd.h index 8a45dda58c..af8ce8921d 100644 --- a/migration/multifd.h +++ b/migration/multifd.h @@ -98,12 +98,12 @@ typedef struct { bool running; /* should this thread finish */ bool quit; - /* multifd flags for each packet */ - uint32_t flags; /* global number of generated multifd packets */ uint64_t packet_num; /* How many bytes have we sent on the last packet */ uint64_t sent_bytes; + /* Do we need to do an iteration sync */ + bool sync_needed; /* thread has work to do */ int pending_job; /* array of pages to sent. @@ -117,6 +117,8 @@ typedef struct { =20 /* pointer to the packet */ MultiFDPacket_t *packet; + /* multifd flags for each packet */ + uint32_t flags; /* size of the next packet that contains pages */ uint32_t next_packet_size; /* packets sent through this channel */ @@ -163,8 +165,6 @@ typedef struct { bool running; /* should this thread finish */ bool quit; - /* multifd flags for each packet */ - uint32_t flags; /* global number of generated multifd packets */ uint64_t packet_num; =20 @@ -172,6 +172,8 @@ typedef struct { =20 /* pointer to the packet */ MultiFDPacket_t *packet; + /* multifd flags for each packet */ + uint32_t flags; /* size of the next packet that contains pages */ uint32_t next_packet_size; /* packets sent through this channel */ diff --git a/migration/multifd.c b/migration/multifd.c index eef47c274f..69b9d7cf98 100644 --- a/migration/multifd.c +++ b/migration/multifd.c @@ -602,7 +602,7 @@ int multifd_send_sync_main(QEMUFile *f) } =20 p->packet_num =3D multifd_send_state->packet_num++; - p->flags |=3D MULTIFD_FLAG_SYNC; + p->sync_needed =3D true; p->pending_job++; qemu_mutex_unlock(&p->mutex); qemu_sem_post(&p->sem); @@ -656,7 +656,11 @@ static void *multifd_send_thread(void *opaque) =20 if (p->pending_job) { uint64_t packet_num =3D p->packet_num; - uint32_t flags =3D p->flags; + p->flags =3D 0; + if (p->sync_needed) { + p->flags |=3D MULTIFD_FLAG_SYNC; + p->sync_needed =3D false; + } p->normal_num =3D 0; =20 if (use_zero_copy_send) { @@ -678,14 +682,13 @@ static void *multifd_send_thread(void *opaque) } } multifd_send_fill_packet(p); - p->flags =3D 0; p->num_packets++; p->total_normal_pages +=3D p->normal_num; p->pages->num =3D 0; p->pages->block =3D NULL; qemu_mutex_unlock(&p->mutex); =20 - trace_multifd_send(p->id, packet_num, p->normal_num, flags, + trace_multifd_send(p->id, packet_num, p->normal_num, p->flags, p->next_packet_size); =20 if (use_zero_copy_send) { @@ -713,7 +716,7 @@ static void *multifd_send_thread(void *opaque) p->pending_job--; qemu_mutex_unlock(&p->mutex); =20 - if (flags & MULTIFD_FLAG_SYNC) { + if (p->flags & MULTIFD_FLAG_SYNC) { qemu_sem_post(&p->sem_sync); } qemu_sem_post(&multifd_send_state->channels_ready); @@ -1090,7 +1093,7 @@ static void *multifd_recv_thread(void *opaque) rcu_register_thread(); =20 while (true) { - uint32_t flags; + bool sync_needed =3D false; =20 if (p->quit) { break; @@ -1112,11 +1115,11 @@ static void *multifd_recv_thread(void *opaque) break; } =20 - flags =3D p->flags; + trace_multifd_recv(p->id, p->packet_num, p->normal_num, p->flags, + p->next_packet_size); + sync_needed =3D p->flags & MULTIFD_FLAG_SYNC; /* recv methods don't know how to handle the SYNC flag */ p->flags &=3D ~MULTIFD_FLAG_SYNC; - trace_multifd_recv(p->id, p->packet_num, p->normal_num, flags, - p->next_packet_size); p->num_packets++; p->total_normal_pages +=3D p->normal_num; qemu_mutex_unlock(&p->mutex); @@ -1128,7 +1131,7 @@ static void *multifd_recv_thread(void *opaque) } } =20 - if (flags & MULTIFD_FLAG_SYNC) { + if (sync_needed) { qemu_sem_post(&multifd_recv_state->sem_sync); qemu_sem_wait(&p->sem_sync); } --=20 2.35.3 From nobody Fri Nov 1 01:22:51 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1653995605; cv=none; d=zohomail.com; s=zohoarc; b=ZBmSMLw6sf6RWR0BDM0qS5dARqmPrqzhhft7BzKIXxKIQ8B2P+98m2TyLXj665WoOpXhBPU8CPLwwR+FJpPoRioWFlQ69+0/35YP4imSS2o/2oNQUmbHnZ3oe2gbsYDi78bcMyI2Q+FRCEs5leCXWsPlhjizDo95EirWYKcmgbo= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1653995605; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=KaYL2VfLUjkgTupNU10ufAv96hul6CJ8wTO/z1f1Edg=; b=N8IBbgfhScmljmpV0vFM5JeWj/8snptOBE2xIXu7e92omAgxKCER1+9xYgKQCtiM8E92APmAGF1eUWpLzrmgbOMAKc9CCTyad5nLfT1eq3eZIMsthE4MEgkpC0Yj7VyOMiD8HeCBl10bIXBDc2TmswW7oet63PPv9LN/qA28mCE= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1653995605660835.8049902316073; Tue, 31 May 2022 04:13:25 -0700 (PDT) Received: from localhost ([::1]:51180 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nvzoS-0007cW-Fr for importer@patchew.org; Tue, 31 May 2022 07:13:24 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:46512) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nvzLm-0003MI-1X for qemu-devel@nongnu.org; Tue, 31 May 2022 06:43:46 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]:31329) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nvzLk-0004eu-DQ for qemu-devel@nongnu.org; Tue, 31 May 2022 06:43:45 -0400 Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-271-ncFR8uYhNDKBhlyIu1JCRw-1; Tue, 31 May 2022 06:43:40 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.rdu2.redhat.com [10.11.54.5]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 1B5DF185A7A4; Tue, 31 May 2022 10:43:40 +0000 (UTC) Received: from secure.mitica (unknown [10.39.192.225]) by smtp.corp.redhat.com (Postfix) with ESMTP id 677108287E; Tue, 31 May 2022 10:43:38 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1653993823; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=KaYL2VfLUjkgTupNU10ufAv96hul6CJ8wTO/z1f1Edg=; b=NtjVVQAyp4IwjkWwhFqI4wwXKvhoEN7rTyCVhnExSLXTk88hBUEu5RiJ/RybEBif/XFjlZ Pb9Lbh7NTmcqa5KQkMuG2679x3/1T1xLeaGHGWCP5h1Ze3/oPEoBodejXh54KQ4iN2/nyx 54TjB3URwi9Uk7WGxErQeW4U+1TRle8= X-MC-Unique: ncFR8uYhNDKBhlyIu1JCRw-1 From: Juan Quintela To: qemu-devel@nongnu.org Cc: Marcel Apfelbaum , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , Yanan Wang , "Dr. David Alan Gilbert" , Juan Quintela , Eduardo Habkost , Peter Xu , Leonardo Bras Subject: [PATCH v7 08/13] multifd: Prepare to send a packet without the mutex held Date: Tue, 31 May 2022 12:43:13 +0200 Message-Id: <20220531104318.7494-9-quintela@redhat.com> In-Reply-To: <20220531104318.7494-1-quintela@redhat.com> References: <20220531104318.7494-1-quintela@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 2.79 on 10.11.54.5 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=quintela@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -21 X-Spam_score: -2.2 X-Spam_bar: -- X-Spam_report: (-2.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.082, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1653995607905100001 Content-Type: text/plain; charset="utf-8" We do the send_prepare() and the fill of the head packet without the mutex held. It will help a lot for compression and later in the series for zero pages. Notice that we can use p->pages without holding p->mutex because p->pending_job =3D=3D 1. Signed-off-by: Juan Quintela --- migration/multifd.h | 2 ++ migration/multifd.c | 11 ++++++----- 2 files changed, 8 insertions(+), 5 deletions(-) diff --git a/migration/multifd.h b/migration/multifd.h index af8ce8921d..d48597a1ea 100644 --- a/migration/multifd.h +++ b/migration/multifd.h @@ -109,7 +109,9 @@ typedef struct { /* array of pages to sent. * The owner of 'pages' depends of 'pending_job' value: * pending_job =3D=3D 0 -> migration_thread can use it. + * No need for mutex lock. * pending_job !=3D 0 -> multifd_channel can use it. + * No need for mutex lock. */ MultiFDPages_t *pages; =20 diff --git a/migration/multifd.c b/migration/multifd.c index 69b9d7cf98..056599cbaf 100644 --- a/migration/multifd.c +++ b/migration/multifd.c @@ -661,6 +661,8 @@ static void *multifd_send_thread(void *opaque) p->flags |=3D MULTIFD_FLAG_SYNC; p->sync_needed =3D false; } + qemu_mutex_unlock(&p->mutex); + p->normal_num =3D 0; =20 if (use_zero_copy_send) { @@ -682,11 +684,6 @@ static void *multifd_send_thread(void *opaque) } } multifd_send_fill_packet(p); - p->num_packets++; - p->total_normal_pages +=3D p->normal_num; - p->pages->num =3D 0; - p->pages->block =3D NULL; - qemu_mutex_unlock(&p->mutex); =20 trace_multifd_send(p->id, packet_num, p->normal_num, p->flags, p->next_packet_size); @@ -711,6 +708,10 @@ static void *multifd_send_thread(void *opaque) } =20 qemu_mutex_lock(&p->mutex); + p->num_packets++; + p->total_normal_pages +=3D p->normal_num; + p->pages->num =3D 0; + p->pages->block =3D NULL; p->sent_bytes +=3D p->packet_len;; p->sent_bytes +=3D p->next_packet_size; p->pending_job--; --=20 2.35.3 From nobody Fri Nov 1 01:22:51 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1653995307; cv=none; d=zohomail.com; s=zohoarc; b=SpsxLnpQyirGLECHAzYf6LZNNx9/CMN/dT5ZR0tthfjFn34jQ73DBv+0+C/DVBxWL8xrqCogl6VBdLRDEZyj/wzsO/6dQod/LL1CSjDWl/hgslv5RwFBwO4//jTLEGWthIM8pc7n2uIuxNYy8IWQseGNtmV2R/iaOvYWb0OBJPI= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1653995307; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=oxTASCIcBA5iquuLNS4rUrcwGQ05et4haViEzl1TLb8=; b=Jz0eVSJy8DzJZG60J5nirqcBQkFfdhYrIMvg8y5q4EiViLijcHB62ofFqCTYRC57kP+dZ7gm8EFYsk49EgIHEt9jeVs2zUbAYBlgmbJ6Vw26HMZZTgn4X5zOOZ5c340xjQ9/Lxs711K+HL14utr4FgSYlwmY3YvhhSh3VHD/6VY= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1653995307912262.9925014319259; Tue, 31 May 2022 04:08:27 -0700 (PDT) Received: from localhost ([::1]:46582 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nvzje-0003bq-Sj for importer@patchew.org; Tue, 31 May 2022 07:08:26 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:46538) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nvzLq-0003YI-Dj for qemu-devel@nongnu.org; Tue, 31 May 2022 06:43:50 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:47948) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nvzLo-0004fE-LM for qemu-devel@nongnu.org; Tue, 31 May 2022 06:43:49 -0400 Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-464-VNIRHoQsN7yQ08XVcn900Q-1; Tue, 31 May 2022 06:43:42 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.rdu2.redhat.com [10.11.54.5]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 537B680A0B5; Tue, 31 May 2022 10:43:42 +0000 (UTC) Received: from secure.mitica (unknown [10.39.192.225]) by smtp.corp.redhat.com (Postfix) with ESMTP id 70CA01131A; Tue, 31 May 2022 10:43:40 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1653993828; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=oxTASCIcBA5iquuLNS4rUrcwGQ05et4haViEzl1TLb8=; b=iYajH1Ds4QLoj9vj9NwTCQzyvW5F6g6Tu38pX77/MBQhIVFgZxQ9ahKCZ6Dd4n397fUo6d dscFkvQSLofUCnaKAF9bn5g4NnWH8gVJeIJ34/gmYVJJB8aAwVBD9djElOsdNn8onpNrrF j6WTPGOyOozWDKovdlVbujXf4sZUByo= X-MC-Unique: VNIRHoQsN7yQ08XVcn900Q-1 From: Juan Quintela To: qemu-devel@nongnu.org Cc: Marcel Apfelbaum , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , Yanan Wang , "Dr. David Alan Gilbert" , Juan Quintela , Eduardo Habkost , Peter Xu , Leonardo Bras Subject: [PATCH v7 09/13] multifd: Add property to enable/disable zero_page Date: Tue, 31 May 2022 12:43:14 +0200 Message-Id: <20220531104318.7494-10-quintela@redhat.com> In-Reply-To: <20220531104318.7494-1-quintela@redhat.com> References: <20220531104318.7494-1-quintela@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 2.79 on 10.11.54.5 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=quintela@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -21 X-Spam_score: -2.2 X-Spam_bar: -- X-Spam_report: (-2.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.082, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1653995309395100001 Content-Type: text/plain; charset="utf-8" Signed-off-by: Juan Quintela Reviewed-by: Dr. David Alan Gilbert --- migration/migration.h | 3 +++ hw/core/machine.c | 1 + migration/migration.c | 11 +++++++++++ 3 files changed, 15 insertions(+) diff --git a/migration/migration.h b/migration/migration.h index 485d58b95f..1017c1bf4a 100644 --- a/migration/migration.h +++ b/migration/migration.h @@ -332,6 +332,8 @@ struct MigrationState { * This save hostname when out-going migration starts */ char *hostname; + /* Use multifd channel to send zero pages */ + bool multifd_zero_pages; }; =20 void migrate_set_state(int *state, int old_state, int new_state); @@ -374,6 +376,7 @@ int migrate_multifd_channels(void); MultiFDCompression migrate_multifd_compression(void); int migrate_multifd_zlib_level(void); int migrate_multifd_zstd_level(void); +bool migrate_use_multifd_zero_page(void); =20 #ifdef CONFIG_LINUX bool migrate_use_zero_copy_send(void); diff --git a/hw/core/machine.c b/hw/core/machine.c index bb0dc8f6a9..a5b5f87329 100644 --- a/hw/core/machine.c +++ b/hw/core/machine.c @@ -43,6 +43,7 @@ =20 GlobalProperty hw_compat_7_0[] =3D { { "arm-gicv3-common", "force-8-bit-prio", "on" }, + { "migration", "multifd-zero-pages", "false" }, }; const size_t hw_compat_7_0_len =3D G_N_ELEMENTS(hw_compat_7_0); =20 diff --git a/migration/migration.c b/migration/migration.c index 31739b2af9..224abe2555 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -2540,6 +2540,15 @@ bool migrate_use_multifd(void) return s->enabled_capabilities[MIGRATION_CAPABILITY_MULTIFD]; } =20 +bool migrate_use_multifd_zero_page(void) +{ + MigrationState *s; + + s =3D migrate_get_current(); + + return s->multifd_zero_pages; +} + bool migrate_pause_before_switchover(void) { MigrationState *s; @@ -4207,6 +4216,8 @@ static Property migration_properties[] =3D { clear_bitmap_shift, CLEAR_BITMAP_SHIFT_DEFAULT), =20 /* Migration parameters */ + DEFINE_PROP_BOOL("multifd-zero-pages", MigrationState, + multifd_zero_pages, true), DEFINE_PROP_UINT8("x-compress-level", MigrationState, parameters.compress_level, DEFAULT_MIGRATE_COMPRESS_LEVEL), --=20 2.35.3 From nobody Fri Nov 1 01:22:51 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1653994441; cv=none; d=zohomail.com; s=zohoarc; b=LDU6Cf2aF3AjlOy/t5ImNWRPhNsYNziwkUCHWecHVluEByTSKKo8zi7C9pqwPQGoBDZuFCY8OoFJ0VfGr0BOk1GluAk53UI/PknIUJGERMUlXrVGYMn1S8aA6PWYGW/Q3QtD9JoSrfczTuUd6f/aEc3LYFkhes613+3jvOxfDU4= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1653994441; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=SxF195sGwkezALJ6dYZJZALZ/ukTIeSsQu0q5lm1GGE=; b=Z1yI849BStu/NWLOsXkZceJY3Pex+Pt2q8gQP+ipP/RkGP8FLfzfiDQJCJzvYynRvImA5tH1ZlrZ31DgidJfU8OnUPM/o0PhuRpj2M302QOv0kNi/KZCvqpfCxneYH9c4S+yZXCg+GkKS4qwvtpVF+Ai3NHNzjV1+CH9IGIacj8= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1653994441272838.9654840253871; Tue, 31 May 2022 03:54:01 -0700 (PDT) Received: from localhost ([::1]:34296 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nvzVg-00030D-3e for importer@patchew.org; Tue, 31 May 2022 06:54:00 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:46540) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nvzLq-0003YO-De for qemu-devel@nongnu.org; Tue, 31 May 2022 06:43:50 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:46527) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nvzLo-0004fP-RO for qemu-devel@nongnu.org; Tue, 31 May 2022 06:43:50 -0400 Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-524-SomwDmoDPvumo8mAFiahSA-1; Tue, 31 May 2022 06:43:44 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.rdu2.redhat.com [10.11.54.5]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 5F5273C11058; Tue, 31 May 2022 10:43:44 +0000 (UTC) Received: from secure.mitica (unknown [10.39.192.225]) by smtp.corp.redhat.com (Postfix) with ESMTP id A4F7B5F245; Tue, 31 May 2022 10:43:42 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1653993828; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=SxF195sGwkezALJ6dYZJZALZ/ukTIeSsQu0q5lm1GGE=; b=EvFgaGAPm7AR0dCXNJkrQuemKUL7FgV+aebcwmH9ioz4pkb6QAChIECHwOouPaBE56KycH 1C6Z0QJU31AcRGdRiPfP1DG37dipI5tSkpj7zxZWsFCizQMLyPFgfPe7MRYNiK0QH5gI/p jOzzI6LLysm6ovIveOK9MEDvJ+7IT9o= X-MC-Unique: SomwDmoDPvumo8mAFiahSA-1 From: Juan Quintela To: qemu-devel@nongnu.org Cc: Marcel Apfelbaum , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , Yanan Wang , "Dr. David Alan Gilbert" , Juan Quintela , Eduardo Habkost , Peter Xu , Leonardo Bras Subject: [PATCH v7 10/13] migration: Export ram_release_page() Date: Tue, 31 May 2022 12:43:15 +0200 Message-Id: <20220531104318.7494-11-quintela@redhat.com> In-Reply-To: <20220531104318.7494-1-quintela@redhat.com> References: <20220531104318.7494-1-quintela@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 2.79 on 10.11.54.5 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=quintela@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -21 X-Spam_score: -2.2 X-Spam_bar: -- X-Spam_report: (-2.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.082, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1653994443481100001 Content-Type: text/plain; charset="utf-8" Signed-off-by: Juan Quintela Reviewed-by: Dr. David Alan Gilbert --- migration/ram.h | 1 + migration/ram.c | 2 +- 2 files changed, 2 insertions(+), 1 deletion(-) diff --git a/migration/ram.h b/migration/ram.h index 7b641adc55..aee08de2a5 100644 --- a/migration/ram.h +++ b/migration/ram.h @@ -66,6 +66,7 @@ int ram_load_postcopy(QEMUFile *f); void ram_handle_compressed(void *host, uint8_t ch, uint64_t size); =20 void ram_transferred_add(uint64_t bytes); +void ram_release_page(const char *rbname, uint64_t offset); =20 int ramblock_recv_bitmap_test(RAMBlock *rb, void *host_addr); bool ramblock_recv_bitmap_test_byte_offset(RAMBlock *rb, uint64_t byte_off= set); diff --git a/migration/ram.c b/migration/ram.c index 71506b1b20..3b2af07341 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -1182,7 +1182,7 @@ static void migration_bitmap_sync_precopy(RAMState *r= s) } } =20 -static void ram_release_page(const char *rbname, uint64_t offset) +void ram_release_page(const char *rbname, uint64_t offset) { if (!migrate_release_ram() || !migration_in_postcopy()) { return; --=20 2.35.3 From nobody Fri Nov 1 01:22:51 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1653995868; cv=none; d=zohomail.com; s=zohoarc; b=kP0/kRpi+mWLCbUW35Jd4DjaFu8vfQVhn+Qn5OCr1UJ9qEjBZixZAUEt2tPUahPbgWddQe7JD1/r8EEPWCfsqaysfCAHafJQpo0WGzSFJRb+HePdRAYMppEHdS1uKvi0aMV9tmJpv5WT7RhynFNCbkjjJvqRaPhuIpF/fef9r2k= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1653995868; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=LlsSpXZ9SpS3c6FgLBtME5gxQzJYSB+4KtzUw9H7/uU=; b=c+u/Y94K3P/M2h6k9cvS+vnRvqOwQj+GBgj0+09BxuBdqmI72NI81VO/A1gZW1wbEDksd8nCHtit+jP0aXaNftAfd19DBBNSMFvmWBNdEXYhU8h11nCjdxp1sdDytxKZKKnm/tpS9YjmuVwo4Ixj3k+0U+ojZ1ka2Ql11CJUXaM= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1653995868748667.6530811808516; Tue, 31 May 2022 04:17:48 -0700 (PDT) Received: from localhost ([::1]:56270 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nvzsh-0002zM-MQ for importer@patchew.org; Tue, 31 May 2022 07:17:47 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:46606) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nvzLv-0003oX-Cj for qemu-devel@nongnu.org; Tue, 31 May 2022 06:43:55 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]:47301) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nvzLs-0004gy-Mb for qemu-devel@nongnu.org; Tue, 31 May 2022 06:43:55 -0400 Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-571-dRoNH9bPN9-X82-c1IAxEA-1; Tue, 31 May 2022 06:43:47 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.rdu2.redhat.com [10.11.54.5]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 9913F811E75; Tue, 31 May 2022 10:43:46 +0000 (UTC) Received: from secure.mitica (unknown [10.39.192.225]) by smtp.corp.redhat.com (Postfix) with ESMTP id B07F28287E; Tue, 31 May 2022 10:43:44 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1653993832; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=LlsSpXZ9SpS3c6FgLBtME5gxQzJYSB+4KtzUw9H7/uU=; b=QoDwSXpE8U1/ksMTJg0x/vB7H5hHEEY5b5kXu+Tbuig5RmpXfVyIlaPe1N/fZKn9yDn6vi ngdWByA3qcKC/WNfjp9IEbnicHJ+L2FYTWriSUrCiO2AV2J6E/nWunSEO1NEo64mMEKBnj j22v4b+eVsCFYeaRql5Ex7xlG+Gue2M= X-MC-Unique: dRoNH9bPN9-X82-c1IAxEA-1 From: Juan Quintela To: qemu-devel@nongnu.org Cc: Marcel Apfelbaum , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , Yanan Wang , "Dr. David Alan Gilbert" , Juan Quintela , Eduardo Habkost , Peter Xu , Leonardo Bras Subject: [PATCH v7 11/13] multifd: Support for zero pages transmission Date: Tue, 31 May 2022 12:43:16 +0200 Message-Id: <20220531104318.7494-12-quintela@redhat.com> In-Reply-To: <20220531104318.7494-1-quintela@redhat.com> References: <20220531104318.7494-1-quintela@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 2.79 on 10.11.54.5 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=quintela@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -21 X-Spam_score: -2.2 X-Spam_bar: -- X-Spam_report: (-2.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.082, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1653995869485100001 Content-Type: text/plain; charset="utf-8" This patch adds counters and similar. Logic will be added on the following patch. Signed-off-by: Juan Quintela Reviewed-by: Dr. David Alan Gilbert --- Added counters for duplicated/non duplicated pages. Removed reviewed by from David. Add total_zero_pages --- migration/multifd.h | 17 ++++++++++++++++- migration/multifd.c | 36 +++++++++++++++++++++++++++++------- migration/ram.c | 2 -- migration/trace-events | 8 ++++---- 4 files changed, 49 insertions(+), 14 deletions(-) diff --git a/migration/multifd.h b/migration/multifd.h index d48597a1ea..c36d7ff876 100644 --- a/migration/multifd.h +++ b/migration/multifd.h @@ -47,7 +47,10 @@ typedef struct { /* size of the next packet that contains pages */ uint32_t next_packet_size; uint64_t packet_num; - uint64_t unused[4]; /* Reserved for future use */ + /* zero pages */ + uint32_t zero_pages; + uint32_t unused32[1]; /* Reserved for future use */ + uint64_t unused64[3]; /* Reserved for future use */ char ramblock[256]; uint64_t offset[]; } __attribute__((packed)) MultiFDPacket_t; @@ -127,6 +130,8 @@ typedef struct { uint64_t num_packets; /* non zero pages sent through this channel */ uint64_t total_normal_pages; + /* zero pages sent through this channel */ + uint64_t total_zero_pages; /* buffers to send */ struct iovec *iov; /* number of iovs used */ @@ -135,6 +140,10 @@ typedef struct { ram_addr_t *normal; /* num of non zero pages */ uint32_t normal_num; + /* Pages that are zero */ + ram_addr_t *zero; + /* num of zero pages */ + uint32_t zero_num; /* used for compression methods */ void *data; } MultiFDSendParams; @@ -184,12 +193,18 @@ typedef struct { uint8_t *host; /* non zero pages recv through this channel */ uint64_t total_normal_pages; + /* zero pages recv through this channel */ + uint64_t total_zero_pages; /* buffers to recv */ struct iovec *iov; /* Pages that are not zero */ ram_addr_t *normal; /* num of non zero pages */ uint32_t normal_num; + /* Pages that are zero */ + ram_addr_t *zero; + /* num of zero pages */ + uint32_t zero_num; /* used for de-compression methods */ void *data; } MultiFDRecvParams; diff --git a/migration/multifd.c b/migration/multifd.c index 056599cbaf..0f7c27c08e 100644 --- a/migration/multifd.c +++ b/migration/multifd.c @@ -263,6 +263,7 @@ static void multifd_send_fill_packet(MultiFDSendParams = *p) packet->normal_pages =3D cpu_to_be32(p->normal_num); packet->next_packet_size =3D cpu_to_be32(p->next_packet_size); packet->packet_num =3D cpu_to_be64(p->packet_num); + packet->zero_pages =3D cpu_to_be32(p->zero_num); =20 if (p->pages->block) { strncpy(packet->ramblock, p->pages->block->idstr, 256); @@ -323,7 +324,15 @@ static int multifd_recv_unfill_packet(MultiFDRecvParam= s *p, Error **errp) p->next_packet_size =3D be32_to_cpu(packet->next_packet_size); p->packet_num =3D be64_to_cpu(packet->packet_num); =20 - if (p->normal_num =3D=3D 0) { + p->zero_num =3D be32_to_cpu(packet->zero_pages); + if (p->zero_num > packet->pages_alloc - p->normal_num) { + error_setg(errp, "multifd: received packet " + "with %u zero pages and expected maximum pages are %u", + p->zero_num, packet->pages_alloc - p->normal_num) ; + return -1; + } + + if (p->normal_num =3D=3D 0 && p->zero_num =3D=3D 0) { return 0; } =20 @@ -432,6 +441,8 @@ static int multifd_send_pages(QEMUFile *f) ram_counters.multifd_bytes +=3D p->sent_bytes; qemu_file_update_transfer(f, p->sent_bytes); p->sent_bytes =3D 0; + ram_counters.normal +=3D p->normal_num; + ram_counters.duplicate +=3D p->zero_num; qemu_mutex_unlock(&p->mutex); qemu_sem_post(&p->sem); =20 @@ -545,6 +556,8 @@ void multifd_save_cleanup(void) p->iov =3D NULL; g_free(p->normal); p->normal =3D NULL; + g_free(p->zero); + p->zero =3D NULL; multifd_send_state->ops->send_cleanup(p, &local_err); if (local_err) { migrate_set_error(migrate_get_current(), local_err); @@ -664,6 +677,7 @@ static void *multifd_send_thread(void *opaque) qemu_mutex_unlock(&p->mutex); =20 p->normal_num =3D 0; + p->zero_num =3D 0; =20 if (use_zero_copy_send) { p->iovs_num =3D 0; @@ -685,8 +699,8 @@ static void *multifd_send_thread(void *opaque) } multifd_send_fill_packet(p); =20 - trace_multifd_send(p->id, packet_num, p->normal_num, p->flags, - p->next_packet_size); + trace_multifd_send(p->id, packet_num, p->normal_num, p->zero_n= um, + p->flags, p->next_packet_size); =20 if (use_zero_copy_send) { /* Send header first, without zerocopy */ @@ -710,6 +724,7 @@ static void *multifd_send_thread(void *opaque) qemu_mutex_lock(&p->mutex); p->num_packets++; p->total_normal_pages +=3D p->normal_num; + p->total_zero_pages +=3D p->zero_num; p->pages->num =3D 0; p->pages->block =3D NULL; p->sent_bytes +=3D p->packet_len;; @@ -751,7 +766,8 @@ out: qemu_mutex_unlock(&p->mutex); =20 rcu_unregister_thread(); - trace_multifd_send_thread_end(p->id, p->num_packets, p->total_normal_p= ages); + trace_multifd_send_thread_end(p->id, p->num_packets, p->total_normal_p= ages, + p->total_zero_pages); =20 return NULL; } @@ -938,6 +954,7 @@ int multifd_save_setup(Error **errp) p->normal =3D g_new0(ram_addr_t, page_count); p->page_size =3D qemu_target_page_size(); p->page_count =3D page_count; + p->zero =3D g_new0(ram_addr_t, page_count); =20 if (migrate_use_zero_copy_send()) { p->write_flags =3D QIO_CHANNEL_WRITE_FLAG_ZERO_COPY; @@ -1046,6 +1063,8 @@ int multifd_load_cleanup(Error **errp) p->iov =3D NULL; g_free(p->normal); p->normal =3D NULL; + g_free(p->zero); + p->zero =3D NULL; multifd_recv_state->ops->recv_cleanup(p); } qemu_sem_destroy(&multifd_recv_state->sem_sync); @@ -1116,13 +1135,14 @@ static void *multifd_recv_thread(void *opaque) break; } =20 - trace_multifd_recv(p->id, p->packet_num, p->normal_num, p->flags, - p->next_packet_size); + trace_multifd_recv(p->id, p->packet_num, p->normal_num, p->zero_nu= m, + p->flags, p->next_packet_size); sync_needed =3D p->flags & MULTIFD_FLAG_SYNC; /* recv methods don't know how to handle the SYNC flag */ p->flags &=3D ~MULTIFD_FLAG_SYNC; p->num_packets++; p->total_normal_pages +=3D p->normal_num; + p->total_normal_pages +=3D p->zero_num; qemu_mutex_unlock(&p->mutex); =20 if (p->normal_num) { @@ -1147,7 +1167,8 @@ static void *multifd_recv_thread(void *opaque) qemu_mutex_unlock(&p->mutex); =20 rcu_unregister_thread(); - trace_multifd_recv_thread_end(p->id, p->num_packets, p->total_normal_p= ages); + trace_multifd_recv_thread_end(p->id, p->num_packets, p->total_normal_p= ages, + p->total_zero_pages); =20 return NULL; } @@ -1187,6 +1208,7 @@ int multifd_load_setup(Error **errp) p->normal =3D g_new0(ram_addr_t, page_count); p->page_count =3D page_count; p->page_size =3D qemu_target_page_size(); + p->zero =3D g_new0(ram_addr_t, page_count); } =20 for (i =3D 0; i < thread_count; i++) { diff --git a/migration/ram.c b/migration/ram.c index 3b2af07341..7ceef7976b 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -1356,8 +1356,6 @@ static int ram_save_multifd_page(RAMState *rs, RAMBlo= ck *block, if (multifd_queue_page(rs->f, block, offset) < 0) { return -1; } - ram_counters.normal++; - return 1; } =20 diff --git a/migration/trace-events b/migration/trace-events index 1aec580e92..d70e89dbb9 100644 --- a/migration/trace-events +++ b/migration/trace-events @@ -114,21 +114,21 @@ unqueue_page(char *block, uint64_t offset, bool dirty= ) "ramblock '%s' offset 0x% =20 # multifd.c multifd_new_send_channel_async(uint8_t id) "channel %u" -multifd_recv(uint8_t id, uint64_t packet_num, uint32_t used, uint32_t flag= s, uint32_t next_packet_size) "channel %u packet_num %" PRIu64 " pages %u f= lags 0x%x next packet size %u" +multifd_recv(uint8_t id, uint64_t packet_num, uint32_t normal, uint32_t ze= ro, uint32_t flags, uint32_t next_packet_size) "channel %u packet_num %" PR= Iu64 " normal pages %u zero pages %u flags 0x%x next packet size %u" multifd_recv_new_channel(uint8_t id) "channel %u" multifd_recv_sync_main(long packet_num) "packet num %ld" multifd_recv_sync_main_signal(uint8_t id) "channel %u" multifd_recv_sync_main_wait(uint8_t id) "channel %u" multifd_recv_terminate_threads(bool error) "error %d" -multifd_recv_thread_end(uint8_t id, uint64_t packets, uint64_t pages) "cha= nnel %u packets %" PRIu64 " pages %" PRIu64 +multifd_recv_thread_end(uint8_t id, uint64_t packets, uint64_t normal_page= s, uint64_t zero_pages) "channel %u packets %" PRIu64 " normal pages %" PRI= u64 " zero pages %" PRIu64 multifd_recv_thread_start(uint8_t id) "%u" -multifd_send(uint8_t id, uint64_t packet_num, uint32_t normal, uint32_t fl= ags, uint32_t next_packet_size) "channel %u packet_num %" PRIu64 " normal p= ages %u flags 0x%x next packet size %u" +multifd_send(uint8_t id, uint64_t packet_num, uint32_t normalpages, uint32= _t zero_pages, uint32_t flags, uint32_t next_packet_size) "channel %u packe= t_num %" PRIu64 " normal pages %u zero pages %u flags 0x%x next packet size= %u" multifd_send_error(uint8_t id) "channel %u" multifd_send_sync_main(long packet_num) "packet num %ld" multifd_send_sync_main_signal(uint8_t id) "channel %u" multifd_send_sync_main_wait(uint8_t id) "channel %u" multifd_send_terminate_threads(bool error) "error %d" -multifd_send_thread_end(uint8_t id, uint64_t packets, uint64_t normal_page= s) "channel %u packets %" PRIu64 " normal pages %" PRIu64 +multifd_send_thread_end(uint8_t id, uint64_t packets, uint64_t normal_page= s, uint64_t zero_pages) "channel %u packets %" PRIu64 " normal pages %" PR= Iu64 " zero pages %" PRIu64 multifd_send_thread_start(uint8_t id) "%u" multifd_tls_outgoing_handshake_start(void *ioc, void *tioc, const char *ho= stname) "ioc=3D%p tioc=3D%p hostname=3D%s" multifd_tls_outgoing_handshake_error(void *ioc, const char *err) "ioc=3D%p= err=3D%s" --=20 2.35.3 From nobody Fri Nov 1 01:22:51 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1653994753; cv=none; d=zohomail.com; s=zohoarc; b=BX0DZNRkNDbeKT9vC2U/TrluRO2zWDYiX1m88chDNujLg6Hwr5s5hqpiFkVBVtztXhPOVsJf9zPslvlNEpIDss7CFKrJPeHXb0OjkAKWKZGjlISDtnwrZoDoq37IZu53zdmrUKXVSNS7+Y/C+AZZP+5ov/fK1afXeXoEHifWg0c= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1653994753; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=CyG/QV+4nMiaGsMNPoSSCmF//51XCpJ1FC4gsqGv66A=; b=f/dqTW20djHL34AJVzPaHu4OcJ/JOPHj5x9uY4nA/8jGX4gxsZ/JFe9IBBQJWSim7/7RnOjn4oM3OYXSlX8hlGLsxqWOGDivUduD0DrixlDaUCuJMccMHmLvBQXu5CwSfOxoOiBMPdx3DheDHLrBrxAZjnaPjQpuEAwfUXLg1Tg= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1653994753590953.6580066280899; Tue, 31 May 2022 03:59:13 -0700 (PDT) Received: from localhost ([::1]:40694 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nvzai-0007Qb-Cx for importer@patchew.org; Tue, 31 May 2022 06:59:12 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:46584) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nvzLu-0003l0-BN for qemu-devel@nongnu.org; Tue, 31 May 2022 06:43:54 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]:43178) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nvzLs-0004gw-KH for qemu-devel@nongnu.org; Tue, 31 May 2022 06:43:54 -0400 Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-593-XeKT1NbFP76_VZ-XRyxzQQ-1; Tue, 31 May 2022 06:43:49 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.rdu2.redhat.com [10.11.54.5]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 9DDB93C11733; Tue, 31 May 2022 10:43:48 +0000 (UTC) Received: from secure.mitica (unknown [10.39.192.225]) by smtp.corp.redhat.com (Postfix) with ESMTP id E8DCB1131A; Tue, 31 May 2022 10:43:46 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1653993832; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=CyG/QV+4nMiaGsMNPoSSCmF//51XCpJ1FC4gsqGv66A=; b=D62OiC1Jau//leNEEB1XX5Tl4ruLsyaF4ZplL3/qMPz5tkuC1q2JykhebwNyH/NhyQcmU7 Wy5OGpdWXefLZmcpjTN51j0zqOzTh+1TP91XER4AsBvhDhUyqHxXYfrbElzzQ25AfISNum Ylo2r1DHWoVERloPA5eE6dGOyoQM0oE= X-MC-Unique: XeKT1NbFP76_VZ-XRyxzQQ-1 From: Juan Quintela To: qemu-devel@nongnu.org Cc: Marcel Apfelbaum , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , Yanan Wang , "Dr. David Alan Gilbert" , Juan Quintela , Eduardo Habkost , Peter Xu , Leonardo Bras Subject: [PATCH v7 12/13] multifd: Zero pages transmission Date: Tue, 31 May 2022 12:43:17 +0200 Message-Id: <20220531104318.7494-13-quintela@redhat.com> In-Reply-To: <20220531104318.7494-1-quintela@redhat.com> References: <20220531104318.7494-1-quintela@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 2.79 on 10.11.54.5 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=quintela@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -21 X-Spam_score: -2.2 X-Spam_bar: -- X-Spam_report: (-2.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.082, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1653994755886100002 Content-Type: text/plain; charset="utf-8" This implements the zero page dection and handling. Signed-off-by: Juan Quintela Reviewed-by: Dr. David Alan Gilbert --- Add comment for offset (dave) Use local variables for offset/block to have shorter lines --- migration/multifd.h | 5 +++++ migration/multifd.c | 41 +++++++++++++++++++++++++++++++++++++++-- 2 files changed, 44 insertions(+), 2 deletions(-) diff --git a/migration/multifd.h b/migration/multifd.h index c36d7ff876..209e3a9756 100644 --- a/migration/multifd.h +++ b/migration/multifd.h @@ -52,6 +52,11 @@ typedef struct { uint32_t unused32[1]; /* Reserved for future use */ uint64_t unused64[3]; /* Reserved for future use */ char ramblock[256]; + /* + * This array contains the pointers to: + * - normal pages (initial normal_pages entries) + * - zero pages (following zero_pages entries) + */ uint64_t offset[]; } __attribute__((packed)) MultiFDPacket_t; =20 diff --git a/migration/multifd.c b/migration/multifd.c index 0f7c27c08e..fcdd241c5c 100644 --- a/migration/multifd.c +++ b/migration/multifd.c @@ -11,6 +11,7 @@ */ =20 #include "qemu/osdep.h" +#include "qemu/cutils.h" #include "qemu/rcu.h" #include "exec/target_page.h" #include "sysemu/sysemu.h" @@ -275,6 +276,12 @@ static void multifd_send_fill_packet(MultiFDSendParams= *p) =20 packet->offset[i] =3D cpu_to_be64(temp); } + for (i =3D 0; i < p->zero_num; i++) { + /* there are architectures where ram_addr_t is 32 bit */ + uint64_t temp =3D p->zero[i]; + + packet->offset[p->normal_num + i] =3D cpu_to_be64(temp); + } } =20 static int multifd_recv_unfill_packet(MultiFDRecvParams *p, Error **errp) @@ -358,6 +365,18 @@ static int multifd_recv_unfill_packet(MultiFDRecvParam= s *p, Error **errp) p->normal[i] =3D offset; } =20 + for (i =3D 0; i < p->zero_num; i++) { + uint64_t offset =3D be64_to_cpu(packet->offset[p->normal_num + i]); + + if (offset > (block->used_length - p->page_size)) { + error_setg(errp, "multifd: offset too long %" PRIu64 + " (max " RAM_ADDR_FMT ")", + offset, block->used_length); + return -1; + } + p->zero[i] =3D offset; + } + return 0; } =20 @@ -646,6 +665,8 @@ static void *multifd_send_thread(void *opaque) { MultiFDSendParams *p =3D opaque; Error *local_err =3D NULL; + /* qemu older than 7.0 don't understand zero page on multifd channel */ + bool use_zero_page =3D migrate_use_multifd_zero_page(); int ret =3D 0; bool use_zero_copy_send =3D migrate_use_zero_copy_send(); =20 @@ -668,6 +689,7 @@ static void *multifd_send_thread(void *opaque) qemu_mutex_lock(&p->mutex); =20 if (p->pending_job) { + RAMBlock *rb =3D p->pages->block; uint64_t packet_num =3D p->packet_num; p->flags =3D 0; if (p->sync_needed) { @@ -686,8 +708,16 @@ static void *multifd_send_thread(void *opaque) } =20 for (int i =3D 0; i < p->pages->num; i++) { - p->normal[p->normal_num] =3D p->pages->offset[i]; - p->normal_num++; + uint64_t offset =3D p->pages->offset[i]; + if (use_zero_page && + buffer_is_zero(rb->host + offset, p->page_size)) { + p->zero[p->zero_num] =3D offset; + p->zero_num++; + ram_release_page(rb->idstr, offset); + } else { + p->normal[p->normal_num] =3D offset; + p->normal_num++; + } } =20 if (p->normal_num) { @@ -1152,6 +1182,13 @@ static void *multifd_recv_thread(void *opaque) } } =20 + for (int i =3D 0; i < p->zero_num; i++) { + void *page =3D p->host + p->zero[i]; + if (!buffer_is_zero(page, p->page_size)) { + memset(page, 0, p->page_size); + } + } + if (sync_needed) { qemu_sem_post(&multifd_recv_state->sem_sync); qemu_sem_wait(&p->sem_sync); --=20 2.35.3 From nobody Fri Nov 1 01:22:51 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1653995621; cv=none; d=zohomail.com; s=zohoarc; b=PAfiGarvUa6IV8XT0s7f35SU4tplGLQ0y6nJ9SXwBtKRF7ZOyUOe35gis6jBJIlJFkVdBFjCArWU44SlcmEQnwgyJMfiAqQmyu38O0ZLg/ML4H0XAW06oPuWVqi3ESd0TVNg54iSL8mw6DZte5qGnlEDG4LqeRtkEclGn6gJnws= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1653995621; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=IzwhjLoqScc49CkenZZj/4FIwnFGKi7Ccd80ETn04LQ=; b=Q269zVOgAjjtUxGnZwYXXvKiPmWRXLpPbfcruxtYVq8leNnP88OoK7AGHHooZSmeiK8wu4vgvH71I/nVUorKVkCEoZzedVV+IS7D9fGQizXpdV6uBauQJ2EypzPaJpa+x41Dal+OHv1m6ILlvmicVgXkzWRtlzMm/nBUsk+m/1E= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1653995621071817.9310024108022; Tue, 31 May 2022 04:13:41 -0700 (PDT) Received: from localhost ([::1]:51496 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nvzoh-0007s5-RI for importer@patchew.org; Tue, 31 May 2022 07:13:39 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:46622) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nvzM3-0004DT-W8 for qemu-devel@nongnu.org; Tue, 31 May 2022 06:44:04 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]:45078) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nvzM2-0004hs-Cw for qemu-devel@nongnu.org; Tue, 31 May 2022 06:44:03 -0400 Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-213-fuj6YzJFNFmeFjPwNHGQAQ-1; Tue, 31 May 2022 06:43:51 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.rdu2.redhat.com [10.11.54.5]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 9506185A5BC; Tue, 31 May 2022 10:43:50 +0000 (UTC) Received: from secure.mitica (unknown [10.39.192.225]) by smtp.corp.redhat.com (Postfix) with ESMTP id EA10C1131A; Tue, 31 May 2022 10:43:48 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1653993841; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=IzwhjLoqScc49CkenZZj/4FIwnFGKi7Ccd80ETn04LQ=; b=fRYrLgli8f7iakozXgLoEDvorQQp7f5L6n3bq7XtepTD0xnf3k7UC7I8f1+2Qp02zMmNWi UbYpbDzrIEKjVAg1hLSsoK8lWeWTS8WIVBCTrCLzW9n1cSrDPbEcsJP+YS899VkmMrtGV3 X1ig72qmM376E1SsH7uxxvJj1I4E3ZU= X-MC-Unique: fuj6YzJFNFmeFjPwNHGQAQ-1 From: Juan Quintela To: qemu-devel@nongnu.org Cc: Marcel Apfelbaum , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , Yanan Wang , "Dr. David Alan Gilbert" , Juan Quintela , Eduardo Habkost , Peter Xu , Leonardo Bras Subject: [PATCH v7 13/13] migration: Use multifd before we check for the zero page Date: Tue, 31 May 2022 12:43:18 +0200 Message-Id: <20220531104318.7494-14-quintela@redhat.com> In-Reply-To: <20220531104318.7494-1-quintela@redhat.com> References: <20220531104318.7494-1-quintela@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 2.79 on 10.11.54.5 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=quintela@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -21 X-Spam_score: -2.2 X-Spam_bar: -- X-Spam_report: (-2.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.082, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1653995622147100001 Content-Type: text/plain; charset="utf-8" So we use multifd to transmit zero pages. Signed-off-by: Juan Quintela Reviewed-by: Dr. David Alan Gilbert --- - Check zero_page property before using new code (Dave) --- migration/ram.c | 33 ++++++++++++++++++++++++++++++++- 1 file changed, 32 insertions(+), 1 deletion(-) diff --git a/migration/ram.c b/migration/ram.c index 7ceef7976b..f9959262fa 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -2181,6 +2181,32 @@ static int ram_save_target_page_legacy(RAMState *rs,= PageSearchStatus *pss) return ram_save_page(rs, pss); } =20 +/** + * ram_save_target_page_multifd: save one target page + * + * Returns the number of pages written + * + * @rs: current RAM state + * @pss: data about the page we want to send + */ +static int ram_save_target_page_multifd(RAMState *rs, PageSearchStatus *ps= s) +{ + RAMBlock *block =3D pss->block; + ram_addr_t offset =3D ((ram_addr_t)pss->page) << TARGET_PAGE_BITS; + int res; + + if (!migration_in_postcopy()) { + return ram_save_multifd_page(rs, block, offset); + } + + res =3D save_zero_page(rs, block, offset); + if (res > 0) { + return res; + } + + return ram_save_page(rs, pss); +} + /** * ram_save_host_page: save a whole host page * @@ -2945,7 +2971,12 @@ static int ram_save_setup(QEMUFile *f, void *opaque) ram_control_before_iterate(f, RAM_CONTROL_SETUP); ram_control_after_iterate(f, RAM_CONTROL_SETUP); =20 - (*rsp)->ram_save_target_page =3D ram_save_target_page_legacy; + if (migrate_use_multifd() && migrate_use_multifd_zero_page()) { + (*rsp)->ram_save_target_page =3D ram_save_target_page_multifd; + } else { + (*rsp)->ram_save_target_page =3D ram_save_target_page_legacy; + } + ret =3D multifd_send_sync_main(f); if (ret < 0) { return ret; --=20 2.35.3