From nobody Mon May 13 18:32:02 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1684180654; cv=none; d=zohomail.com; s=zohoarc; b=e7P7fDr0tvFLgjyWbZ8VadGQHFvQaMxmAgILob2ImwJfxUoHNiIeEqB/jheLPR4dpcg02bawLjndyF3JAo236Btz/EAvY+OovMiu/1kYcdgMXjjuLoWhBD+HObRid/E89Beef6yf7kIu2oYkY3AZgqASMpBSbqTkq71XxrFucEw= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1684180654; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=FeHm5dXsEgwrKffR59GDRBqQ+8zkfqjjN8hVgbkHBPE=; b=i+04MybKC8qiDdm3mZaAxZr838vwu6NNv9UJ0tx1fPaZd11VbEpniXPhuFv7u1ep4DPfMKTLu5CyDlV3jeK78oTbrIe4AqwrvZ5KpZNdDB4GwDHl7BGhH7OjX2ZutcddBo15H+7FNEHrmtDMK3WmLNWywPgdvDkscaQhwaEiofg= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 168418065438961.40706679892071; Mon, 15 May 2023 12:57:34 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pyeK4-0007NR-03; Mon, 15 May 2023 15:57:32 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pyeK2-0007Mq-1g for qemu-devel@nongnu.org; Mon, 15 May 2023 15:57:30 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pyeJx-0005Gi-Dq for qemu-devel@nongnu.org; Mon, 15 May 2023 15:57:26 -0400 Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-175-vaAekEdaPG-1H9IUn4HgJg-1; Mon, 15 May 2023 15:57:19 -0400 Received: from smtp.corp.redhat.com (int-mx09.intmail.prod.int.rdu2.redhat.com [10.11.54.9]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 82520101A557; Mon, 15 May 2023 19:57:18 +0000 (UTC) Received: from secure.mitica (unknown [10.39.194.171]) by smtp.corp.redhat.com (Postfix) with ESMTP id ABD40492B00; Mon, 15 May 2023 19:57:14 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1684180644; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=FeHm5dXsEgwrKffR59GDRBqQ+8zkfqjjN8hVgbkHBPE=; b=KUyt9OcLFODihTCcl9JuMT2apCJjiiedRgnXfw1/4bqavZoirV/brRlG79ChjXCinrxfEO nG6nM6kjALzUAf7rb/VyMKuede2DgErpfwf4HUejC/uEyh0xYbMOa8OeAfE/Ty/TDtP/Jj LHLji+EEkUPso4dVDALH7iVjhDi5fGs= X-MC-Unique: vaAekEdaPG-1H9IUn4HgJg-1 From: Juan Quintela To: qemu-devel@nongnu.org Cc: Peter Xu , Greg Kurz , qemu-s390x@nongnu.org, Fam Zheng , Ilya Leoshkevich , Halil Pasic , John Snow , qemu-ppc@nongnu.org, Daniel Henrique Barboza , Harsh Prateek Bora , Christian Borntraeger , Richard Henderson , David Gibson , David Hildenbrand , Stefan Hajnoczi , Eric Farman , qemu-block@nongnu.org, =?UTF-8?q?C=C3=A9dric=20Le=20Goater?= , Eric Blake , Leonardo Bras , Thomas Huth , Juan Quintela , Vladimir Sementsov-Ogievskiy Subject: [PATCH v2 01/16] migration: Don't use INT64_MAX for unlimited rate Date: Mon, 15 May 2023 21:56:54 +0200 Message-Id: <20230515195709.63843-2-quintela@redhat.com> In-Reply-To: <20230515195709.63843-1-quintela@redhat.com> References: <20230515195709.63843-1-quintela@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.1 on 10.11.54.9 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=quintela@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1684180655962100003 Content-Type: text/plain; charset="utf-8" Define and use RATE_LIMIT_MAX instead. Signed-off-by: Juan Quintela Reviewed-by: Harsh Prateek Bora Reviewed-by: Leonardo Bras --- migration/migration-stats.h | 6 ++++++ migration/migration.c | 4 ++-- migration/qemu-file.c | 6 +++++- 3 files changed, 13 insertions(+), 3 deletions(-) diff --git a/migration/migration-stats.h b/migration/migration-stats.h index cf8a4f0410..e782f1b0df 100644 --- a/migration/migration-stats.h +++ b/migration/migration-stats.h @@ -15,6 +15,12 @@ =20 #include "qemu/stats64.h" =20 +/* + * If rate_limit_max is 0, there is special code to remove the rate + * limit. + */ +#define RATE_LIMIT_MAX 0 + /* * These are the ram migration statistic counters. It is loosely * based on MigrationStats. We change to Stat64 any counter that diff --git a/migration/migration.c b/migration/migration.c index 039bba4804..c41c7491bb 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -2304,7 +2304,7 @@ static void migration_completion(MigrationState *s) * them if migration fails or is cancelled. */ s->block_inactive =3D !migrate_colo(); - qemu_file_set_rate_limit(s->to_dst_file, INT64_MAX); + qemu_file_set_rate_limit(s->to_dst_file, RATE_LIMIT_MAX); ret =3D qemu_savevm_state_complete_precopy(s->to_dst_file,= false, s->block_inactive= ); } @@ -3048,7 +3048,7 @@ static void *bg_migration_thread(void *opaque) rcu_register_thread(); object_ref(OBJECT(s)); =20 - qemu_file_set_rate_limit(s->to_dst_file, INT64_MAX); + qemu_file_set_rate_limit(s->to_dst_file, RATE_LIMIT_MAX); =20 setup_start =3D qemu_clock_get_ms(QEMU_CLOCK_HOST); /* diff --git a/migration/qemu-file.c b/migration/qemu-file.c index 597054759d..4bc875b452 100644 --- a/migration/qemu-file.c +++ b/migration/qemu-file.c @@ -27,6 +27,7 @@ #include "qemu/error-report.h" #include "qemu/iov.h" #include "migration.h" +#include "migration-stats.h" #include "qemu-file.h" #include "trace.h" #include "options.h" @@ -732,7 +733,10 @@ int qemu_file_rate_limit(QEMUFile *f) if (qemu_file_get_error(f)) { return 1; } - if (f->rate_limit_max > 0 && f->rate_limit_used > f->rate_limit_max) { + if (f->rate_limit_max =3D=3D RATE_LIMIT_MAX) { + return 0; + } + if (f->rate_limit_used > f->rate_limit_max) { return 1; } return 0; --=20 2.40.1 From nobody Mon May 13 18:32:02 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1684180684; cv=none; d=zohomail.com; s=zohoarc; b=BTvyg2eT9BS7Ot6tCRLNpcFFi4DVCoaFfBfeJrz+2tsY26jmD1/13LmCwDEvvsX0BeQkbZ9LiiAMTm5n3QekUpHpvcrFD8ZWtpPcI1JWQ9nTKP2wV8fuemJwfoJkfU5PgFXbrVhShBYjgIfe03F8mcPyWacHW48EORdbmSteiZs= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1684180684; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=6mze1g03PTIY1a/hojYwaKt+/bNVU2PR11sxbqIxHE0=; b=LkoakmrUtCWFJLuJpRXnD+G0NHPqJyiK76WVXwURCSdDc2Cyn/KhR0RzDRcQWIzWZpro4JeIFPVavs4VLjuRvKPBUQqEn8MALqBJ9kno4WEtpSjUiwQk0iosiYMLPW/WdkKWi5B2wLYva69pkw2+FS3RUsHqPJsDagWBQnCIyzo= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1684180684832832.3717593969623; Mon, 15 May 2023 12:58:04 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pyeKN-0007UE-UD; Mon, 15 May 2023 15:57:53 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pyeK4-0007OH-SE for qemu-devel@nongnu.org; Mon, 15 May 2023 15:57:36 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pyeK1-0005HK-Ro for qemu-devel@nongnu.org; Mon, 15 May 2023 15:57:32 -0400 Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-467-YYay1RDZPYu710-0bbY4yQ-1; Mon, 15 May 2023 15:57:23 -0400 Received: from smtp.corp.redhat.com (int-mx09.intmail.prod.int.rdu2.redhat.com [10.11.54.9]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 13245867955; Mon, 15 May 2023 19:57:23 +0000 (UTC) Received: from secure.mitica (unknown [10.39.194.171]) by smtp.corp.redhat.com (Postfix) with ESMTP id 1AEF3492B00; Mon, 15 May 2023 19:57:18 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1684180648; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=6mze1g03PTIY1a/hojYwaKt+/bNVU2PR11sxbqIxHE0=; b=bKzXPy9NQeooNpzv/RxSvGIpF+HG5JTJX/aB1H8y8z9eOjtAfPap6yvoZQhQOrwklSFMwF 0RM9Yiik7uasFaDlJhm4ISlw/bNlLxTm7YMhj7VvbsQ0jolqVQG8jvCXxLweh+qRAcL2V9 MP9pW/kmGof+G8mAKeachkKXsowerVY= X-MC-Unique: YYay1RDZPYu710-0bbY4yQ-1 From: Juan Quintela To: qemu-devel@nongnu.org Cc: Peter Xu , Greg Kurz , qemu-s390x@nongnu.org, Fam Zheng , Ilya Leoshkevich , Halil Pasic , John Snow , qemu-ppc@nongnu.org, Daniel Henrique Barboza , Harsh Prateek Bora , Christian Borntraeger , Richard Henderson , David Gibson , David Hildenbrand , Stefan Hajnoczi , Eric Farman , qemu-block@nongnu.org, =?UTF-8?q?C=C3=A9dric=20Le=20Goater?= , Eric Blake , Leonardo Bras , Thomas Huth , Juan Quintela , Vladimir Sementsov-Ogievskiy Subject: [PATCH v2 02/16] migration: Correct transferred bytes value Date: Mon, 15 May 2023 21:56:55 +0200 Message-Id: <20230515195709.63843-3-quintela@redhat.com> In-Reply-To: <20230515195709.63843-1-quintela@redhat.com> References: <20230515195709.63843-1-quintela@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.1 on 10.11.54.9 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=quintela@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1684180685991100003 Content-Type: text/plain; charset="utf-8" We forget several places to add to trasferred amount of data. With this fixes I get: qemu_file_transferred() + multifd_bytes =3D=3D transferred The only place whrer this is not true is during devices sending. But going all through the full tree searching for devices that use QEMUFile directly is a bit too much. Multifd, precopy and xbzrle work as expected. Postocpy still misses 35 bytes, but searching for them is getting complicated, so I stop here. Signed-off-by: Juan Quintela Reviewed-by: Leonardo Bras --- migration/ram.c | 14 ++++++++++++++ migration/savevm.c | 19 +++++++++++++++++-- migration/vmstate.c | 3 +++ migration/meson.build | 2 +- 4 files changed, 35 insertions(+), 3 deletions(-) diff --git a/migration/ram.c b/migration/ram.c index f69d8d42b0..fd5a8db0f8 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -337,6 +337,7 @@ int64_t ramblock_recv_bitmap_send(QEMUFile *file, =20 g_free(le_bitmap); =20 + stat64_add(&mig_stats.transferred, 8 + size + 8); if (qemu_file_get_error(file)) { return qemu_file_get_error(file); } @@ -1392,6 +1393,7 @@ static int find_dirty_block(RAMState *rs, PageSearchS= tatus *pss) return ret; } qemu_put_be64(f, RAM_SAVE_FLAG_MULTIFD_FLUSH); + stat64_add(&mig_stats.transferred, 8); qemu_fflush(f); } /* @@ -3020,6 +3022,7 @@ static int ram_save_setup(QEMUFile *f, void *opaque) RAMState **rsp =3D opaque; RAMBlock *block; int ret; + size_t size =3D 0; =20 if (compress_threads_save_setup()) { return -1; @@ -3038,16 +3041,20 @@ static int ram_save_setup(QEMUFile *f, void *opaque) qemu_put_be64(f, ram_bytes_total_with_ignored() | RAM_SAVE_FLAG_MEM_SIZE); =20 + size +=3D 8; RAMBLOCK_FOREACH_MIGRATABLE(block) { qemu_put_byte(f, strlen(block->idstr)); qemu_put_buffer(f, (uint8_t *)block->idstr, strlen(block->idst= r)); qemu_put_be64(f, block->used_length); + size +=3D 1 + strlen(block->idstr) + 8; if (migrate_postcopy_ram() && block->page_size !=3D qemu_host_page_size) { qemu_put_be64(f, block->page_size); + size +=3D 8; } if (migrate_ignore_shared()) { qemu_put_be64(f, block->mr->addr); + size +=3D 8; } } } @@ -3064,11 +3071,14 @@ static int ram_save_setup(QEMUFile *f, void *opaque) =20 if (!migrate_multifd_flush_after_each_section()) { qemu_put_be64(f, RAM_SAVE_FLAG_MULTIFD_FLUSH); + size +=3D 8; } =20 qemu_put_be64(f, RAM_SAVE_FLAG_EOS); + size +=3D 8; qemu_fflush(f); =20 + stat64_add(&mig_stats.transferred, size); return 0; } =20 @@ -3209,6 +3219,7 @@ static int ram_save_complete(QEMUFile *f, void *opaqu= e) RAMState **temp =3D opaque; RAMState *rs =3D *temp; int ret =3D 0; + size_t size =3D 0; =20 rs->last_stage =3D !migration_in_colo_state(); =20 @@ -3253,8 +3264,11 @@ static int ram_save_complete(QEMUFile *f, void *opaq= ue) =20 if (!migrate_multifd_flush_after_each_section()) { qemu_put_be64(f, RAM_SAVE_FLAG_MULTIFD_FLUSH); + size +=3D 8; } qemu_put_be64(f, RAM_SAVE_FLAG_EOS); + size +=3D 8; + stat64_add(&mig_stats.transferred, size); qemu_fflush(f); =20 return 0; diff --git a/migration/savevm.c b/migration/savevm.c index e33788343a..c7af9050c2 100644 --- a/migration/savevm.c +++ b/migration/savevm.c @@ -952,6 +952,7 @@ static void save_section_header(QEMUFile *f, SaveStateE= ntry *se, qemu_put_byte(f, section_type); qemu_put_be32(f, se->section_id); =20 + size_t size =3D 1 + 4; if (section_type =3D=3D QEMU_VM_SECTION_FULL || section_type =3D=3D QEMU_VM_SECTION_START) { /* ID string */ @@ -961,7 +962,9 @@ static void save_section_header(QEMUFile *f, SaveStateE= ntry *se, =20 qemu_put_be32(f, se->instance_id); qemu_put_be32(f, se->version_id); + size +=3D 1 + len + 4 + 4; } + stat64_add(&mig_stats.transferred, size); } =20 /* @@ -973,6 +976,7 @@ static void save_section_footer(QEMUFile *f, SaveStateE= ntry *se) if (migrate_get_current()->send_section_footer) { qemu_put_byte(f, QEMU_VM_SECTION_FOOTER); qemu_put_be32(f, se->section_id); + stat64_add(&mig_stats.transferred, 1 + 4); } } =20 @@ -1032,6 +1036,7 @@ static void qemu_savevm_command_send(QEMUFile *f, qemu_put_be16(f, (uint16_t)command); qemu_put_be16(f, len); qemu_put_buffer(f, data, len); + stat64_add(&mig_stats.transferred, 1 + 2 + 2 + len); qemu_fflush(f); } =20 @@ -1212,11 +1217,13 @@ void qemu_savevm_state_header(QEMUFile *f) trace_savevm_state_header(); qemu_put_be32(f, QEMU_VM_FILE_MAGIC); qemu_put_be32(f, QEMU_VM_FILE_VERSION); - + size_t size =3D 4 + 4; if (migrate_get_current()->send_configuration) { qemu_put_byte(f, QEMU_VM_CONFIGURATION); + size +=3D 1; vmstate_save_state(f, &vmstate_configuration, &savevm_state, 0); } + stat64_add(&mig_stats.transferred, size); } =20 bool qemu_savevm_state_guest_unplug_pending(void) @@ -1384,6 +1391,7 @@ void qemu_savevm_state_complete_postcopy(QEMUFile *f) { SaveStateEntry *se; int ret; + size_t size =3D 0; =20 QTAILQ_FOREACH(se, &savevm_state.handlers, entry) { if (!se->ops || !se->ops->save_live_complete_postcopy) { @@ -1398,7 +1406,7 @@ void qemu_savevm_state_complete_postcopy(QEMUFile *f) /* Section type */ qemu_put_byte(f, QEMU_VM_SECTION_END); qemu_put_be32(f, se->section_id); - + size +=3D 1 + 4; ret =3D se->ops->save_live_complete_postcopy(f, se->opaque); trace_savevm_section_end(se->idstr, se->section_id, ret); save_section_footer(f, se); @@ -1409,6 +1417,8 @@ void qemu_savevm_state_complete_postcopy(QEMUFile *f) } =20 qemu_put_byte(f, QEMU_VM_EOF); + size +=3D 1; + stat64_add(&mig_stats.transferred, size); qemu_fflush(f); } =20 @@ -1484,6 +1494,7 @@ int qemu_savevm_state_complete_precopy_non_iterable(Q= EMUFile *f, if (!in_postcopy) { /* Postcopy stream will still be going */ qemu_put_byte(f, QEMU_VM_EOF); + stat64_add(&mig_stats.transferred, 1); } =20 json_writer_end_array(vmdesc); @@ -1664,15 +1675,18 @@ void qemu_savevm_live_state(QEMUFile *f) /* save QEMU_VM_SECTION_END section */ qemu_savevm_state_complete_precopy(f, true, false); qemu_put_byte(f, QEMU_VM_EOF); + stat64_add(&mig_stats.transferred, 1); } =20 int qemu_save_device_state(QEMUFile *f) { SaveStateEntry *se; + size_t size =3D 0; =20 if (!migration_in_colo_state()) { qemu_put_be32(f, QEMU_VM_FILE_MAGIC); qemu_put_be32(f, QEMU_VM_FILE_VERSION); + size =3D 4 + 4; } cpu_synchronize_all_states(); =20 @@ -1690,6 +1704,7 @@ int qemu_save_device_state(QEMUFile *f) =20 qemu_put_byte(f, QEMU_VM_EOF); =20 + stat64_add(&mig_stats.transferred, size + 1); return qemu_file_get_error(f); } =20 diff --git a/migration/vmstate.c b/migration/vmstate.c index af01d54b6f..ee3b70a6a8 100644 --- a/migration/vmstate.c +++ b/migration/vmstate.c @@ -12,6 +12,7 @@ =20 #include "qemu/osdep.h" #include "migration.h" +#include "migration-stats.h" #include "migration/vmstate.h" #include "savevm.h" #include "qapi/qmp/json-writer.h" @@ -394,6 +395,7 @@ int vmstate_save_state_v(QEMUFile *f, const VMStateDesc= ription *vmsd, written_bytes =3D qemu_file_transferred_fast(f) - old_offs= et; vmsd_desc_field_end(vmsd, vmdesc_loop, field, written_byte= s, i); =20 + stat64_add(&mig_stats.transferred, written_bytes); /* Compressed arrays only care about the first element */ if (vmdesc_loop && vmsd_can_compress(field)) { vmdesc_loop =3D NULL; @@ -517,6 +519,7 @@ static int vmstate_subsection_save(QEMUFile *f, const V= MStateDescription *vmsd, qemu_put_byte(f, len); qemu_put_buffer(f, (uint8_t *)vmsdsub->name, len); qemu_put_be32(f, vmsdsub->version_id); + stat64_add(&mig_stats.transferred, 1 + 1 + len + 4); ret =3D vmstate_save_state(f, vmsdsub, opaque, vmdesc); if (ret) { return ret; diff --git a/migration/meson.build b/migration/meson.build index dc8b1daef5..b3d0c537c8 100644 --- a/migration/meson.build +++ b/migration/meson.build @@ -1,5 +1,6 @@ # Files needed by unit tests migration_files =3D files( + 'migration-stats.c', 'page_cache.c', 'xbzrle.c', 'vmstate-types.c', @@ -18,7 +19,6 @@ softmmu_ss.add(files( 'fd.c', 'global_state.c', 'migration-hmp-cmds.c', - 'migration-stats.c', 'migration.c', 'multifd.c', 'multifd-zlib.c', --=20 2.40.1 From nobody Mon May 13 18:32:02 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1684180772; cv=none; d=zohomail.com; s=zohoarc; b=k62EUbzROxZaTrpuFAM909b9+dOxIX49Coc9W6+bcM4EzfTJ2XIz8N9+N4CIEHTCyz2+n5lDUI7jO72l2lqcmlXXLLT6mWobqKIrriLWsW+i94ozsnZHK7qF2TuMI4SzoPQS3wr4EPEUHkRFV/VGFqjCRkjrgCRdEuvsd/E2ovU= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1684180772; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=Lb+aY3qgzi/sGzNZ53b4636ejNtUkewYSRIjO76Ct1I=; b=h4zUlNFyRyw8njkZ/RufboZrr94G6xqfl3mr1y7y0P4mif/rT9Cz0eiW55ZNhEoJ9/DdMu/DeCYytJp4Vo4n3WtaJWVAgQHfAojMgFUzO+NIXeJkWHpBVdXt/mvTg9HLaojHxvW+Rzp3EhdczBFfdl3fpCp3rCsB71FNK23v1JY= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1684180772225782.5930728611693; Mon, 15 May 2023 12:59:32 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pyeKR-0007mP-Kh; Mon, 15 May 2023 15:57:55 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pyeK9-0007RM-QI for qemu-devel@nongnu.org; Mon, 15 May 2023 15:57:40 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pyeK3-0005I2-Sc for qemu-devel@nongnu.org; Mon, 15 May 2023 15:57:37 -0400 Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-313-_Sm7NKe3NCm8R68sj51zSg-1; Mon, 15 May 2023 15:57:28 -0400 Received: from smtp.corp.redhat.com (int-mx09.intmail.prod.int.rdu2.redhat.com [10.11.54.9]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 50EA2101A553; Mon, 15 May 2023 19:57:27 +0000 (UTC) Received: from secure.mitica (unknown [10.39.194.171]) by smtp.corp.redhat.com (Postfix) with ESMTP id 5AC4E492B00; Mon, 15 May 2023 19:57:23 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1684180651; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Lb+aY3qgzi/sGzNZ53b4636ejNtUkewYSRIjO76Ct1I=; b=GWCpd0V9/dvPGatxlwN9/AjfNnzMWrRs3K3GTDeRhVx+hpi7858seVRjAgGuz7r3Qz0Hoq UMWkhXk0z+EDRv1k+sMPSIIi9CFZJx7iY739kQcFseMTHahTl2TrWVJjnki9VM+IsrNWSV NXKossh/jfzg7zY4pJnNd84+LBPlBCw= X-MC-Unique: _Sm7NKe3NCm8R68sj51zSg-1 From: Juan Quintela To: qemu-devel@nongnu.org Cc: Peter Xu , Greg Kurz , qemu-s390x@nongnu.org, Fam Zheng , Ilya Leoshkevich , Halil Pasic , John Snow , qemu-ppc@nongnu.org, Daniel Henrique Barboza , Harsh Prateek Bora , Christian Borntraeger , Richard Henderson , David Gibson , David Hildenbrand , Stefan Hajnoczi , Eric Farman , qemu-block@nongnu.org, =?UTF-8?q?C=C3=A9dric=20Le=20Goater?= , Eric Blake , Leonardo Bras , Thomas Huth , Juan Quintela , Vladimir Sementsov-Ogievskiy Subject: [PATCH v2 03/16] migration: Move setup_time to mig_stats Date: Mon, 15 May 2023 21:56:56 +0200 Message-Id: <20230515195709.63843-4-quintela@redhat.com> In-Reply-To: <20230515195709.63843-1-quintela@redhat.com> References: <20230515195709.63843-1-quintela@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.1 on 10.11.54.9 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=quintela@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1684180774111100001 It is a time that needs to be cleaned each time cancel migration. Once there create migration_time_since() to calculate how time since a time in the past. Signed-off-by: Juan Quintela Reviewed-by: Leonardo Bras --- Rename to migration_time_since (c=C3=A9dric) --- migration/migration-stats.h | 13 +++++++++++++ migration/migration.h | 1 - migration/migration-stats.c | 7 +++++++ migration/migration.c | 9 ++++----- 4 files changed, 24 insertions(+), 6 deletions(-) diff --git a/migration/migration-stats.h b/migration/migration-stats.h index e782f1b0df..21402af9e4 100644 --- a/migration/migration-stats.h +++ b/migration/migration-stats.h @@ -75,6 +75,10 @@ typedef struct { * Number of bytes sent during precopy stage. */ Stat64 precopy_bytes; + /* + * How long has the setup stage took. + */ + Stat64 setup_time; /* * Total number of bytes transferred. */ @@ -87,4 +91,13 @@ typedef struct { =20 extern MigrationAtomicStats mig_stats; =20 +/** + * migration_time_since: Calculate how much time has passed + * + * @stats: migration stats + * @since: reference time since we want to calculate + * + * Returns: Nothing. The time is stored in val. + */ +void migration_time_since(MigrationAtomicStats *stats, int64_t since); #endif diff --git a/migration/migration.h b/migration/migration.h index 48a46123a0..27aa3b1035 100644 --- a/migration/migration.h +++ b/migration/migration.h @@ -316,7 +316,6 @@ struct MigrationState { int64_t downtime; int64_t expected_downtime; bool capabilities[MIGRATION_CAPABILITY__MAX]; - int64_t setup_time; /* * Whether guest was running when we enter the completion stage. * If migration is interrupted by any reason, we need to continue diff --git a/migration/migration-stats.c b/migration/migration-stats.c index 2f2cea965c..3431453c90 100644 --- a/migration/migration-stats.c +++ b/migration/migration-stats.c @@ -12,6 +12,13 @@ =20 #include "qemu/osdep.h" #include "qemu/stats64.h" +#include "qemu/timer.h" #include "migration-stats.h" =20 MigrationAtomicStats mig_stats; + +void migration_time_since(MigrationAtomicStats *stats, int64_t since) +{ + int64_t now =3D qemu_clock_get_ms(QEMU_CLOCK_HOST); + stat64_set(&stats->setup_time, now - since); +} diff --git a/migration/migration.c b/migration/migration.c index c41c7491bb..e9466273bb 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -887,7 +887,7 @@ static void populate_time_info(MigrationInfo *info, Mig= rationState *s) { info->has_status =3D true; info->has_setup_time =3D true; - info->setup_time =3D s->setup_time; + info->setup_time =3D stat64_get(&mig_stats.setup_time); =20 if (s->state =3D=3D MIGRATION_STATUS_COMPLETED) { info->has_total_time =3D true; @@ -1390,7 +1390,6 @@ void migrate_init(MigrationState *s) s->pages_per_second =3D 0.0; s->downtime =3D 0; s->expected_downtime =3D 0; - s->setup_time =3D 0; s->start_postcopy =3D false; s->postcopy_after_devices =3D false; s->migration_thread_running =3D false; @@ -2647,7 +2646,7 @@ static void migration_calculate_complete(MigrationSta= te *s) s->downtime =3D end_time - s->downtime_start; } =20 - transfer_time =3D s->total_time - s->setup_time; + transfer_time =3D s->total_time - stat64_get(&mig_stats.setup_time); if (transfer_time) { s->mbps =3D ((double) bytes * 8.0) / transfer_time / 1000; } @@ -2969,7 +2968,7 @@ static void *migration_thread(void *opaque) qemu_savevm_wait_unplug(s, MIGRATION_STATUS_SETUP, MIGRATION_STATUS_ACTIVE); =20 - s->setup_time =3D qemu_clock_get_ms(QEMU_CLOCK_HOST) - setup_start; + migration_time_since(&mig_stats, setup_start); =20 trace_migration_thread_setup_complete(); =20 @@ -3081,7 +3080,7 @@ static void *bg_migration_thread(void *opaque) qemu_savevm_wait_unplug(s, MIGRATION_STATUS_SETUP, MIGRATION_STATUS_ACTIVE); =20 - s->setup_time =3D qemu_clock_get_ms(QEMU_CLOCK_HOST) - setup_start; + migration_time_since(&mig_stats, setup_start); =20 trace_migration_thread_setup_complete(); s->downtime_start =3D qemu_clock_get_ms(QEMU_CLOCK_REALTIME); --=20 2.40.1 From nobody Mon May 13 18:32:02 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1684180735; cv=none; d=zohomail.com; s=zohoarc; b=Fs4RNl6A2DCOwiXPMKe+tmj9gljRNIZokuDJ8RNe6+0hfF4qbHOC9rSewkp1BOUCPTFHT5PT2DqKoNjRcJzQujPA4LseIbcC2f1lHd7zrmdKPzpRgONG0hxw357mBd+cw5jaTgggmkWjFYfL9h11TlNWxg3jOGQyHVXbzM326AY= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1684180735; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=xlYe2Rxwb69cnr45U0hOL4Ulmyl99p21DIV2iWrX2EA=; b=USt/nKEB88FlERyJxH8mPd313V3Ts0Ry41xILY0t5VZyErbNLlYMU4zOssiL2q5+IXWNHyXOOh/OfMtGGF+56MKiyTu4Z7Gs3fWas/LlZ5FlHI8dsAr28OtGJ5wp9GJdTfZmR0KSMaifWWxjXOapcgdBPGc0pB9JcPxeRS+Ct5o= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 168418073519242.420155802202885; Mon, 15 May 2023 12:58:55 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pyeKR-0007nM-Pf; Mon, 15 May 2023 15:57:55 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pyeKE-0007S5-SS for qemu-devel@nongnu.org; Mon, 15 May 2023 15:57:45 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pyeKD-0005Lm-Ii for qemu-devel@nongnu.org; Mon, 15 May 2023 15:57:42 -0400 Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-443-E8pRzP5MMpap5hb0VvPL6A-1; Mon, 15 May 2023 15:57:38 -0400 Received: from smtp.corp.redhat.com (int-mx09.intmail.prod.int.rdu2.redhat.com [10.11.54.9]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id DEB69380671E; Mon, 15 May 2023 19:57:37 +0000 (UTC) Received: from secure.mitica (unknown [10.39.194.171]) by smtp.corp.redhat.com (Postfix) with ESMTP id 9655E492B00; Mon, 15 May 2023 19:57:27 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1684180661; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=xlYe2Rxwb69cnr45U0hOL4Ulmyl99p21DIV2iWrX2EA=; b=DJ7IG9u28cfPa3KXRjX5gvZsTSb8AlW3BIS21kId4T6uTeaQhSbtSqZN7mpJXykPK+d+nt 8C6hFwNaoOITlqrASRdMWiF/RyOABZLcc1HRSpAguzUUY1CyF5WBUliizfLRC+VxRD9obp Nt3n0jD2lqETrRBywcceMOn8eZPFF8w= X-MC-Unique: E8pRzP5MMpap5hb0VvPL6A-1 From: Juan Quintela To: qemu-devel@nongnu.org Cc: Peter Xu , Greg Kurz , qemu-s390x@nongnu.org, Fam Zheng , Ilya Leoshkevich , Halil Pasic , John Snow , qemu-ppc@nongnu.org, Daniel Henrique Barboza , Harsh Prateek Bora , Christian Borntraeger , Richard Henderson , David Gibson , David Hildenbrand , Stefan Hajnoczi , Eric Farman , qemu-block@nongnu.org, =?UTF-8?q?C=C3=A9dric=20Le=20Goater?= , Eric Blake , Leonardo Bras , Thomas Huth , Juan Quintela , Vladimir Sementsov-Ogievskiy Subject: [PATCH v2 04/16] qemu-file: Account for rate_limit usage on qemu_fflush() Date: Mon, 15 May 2023 21:56:57 +0200 Message-Id: <20230515195709.63843-5-quintela@redhat.com> In-Reply-To: <20230515195709.63843-1-quintela@redhat.com> References: <20230515195709.63843-1-quintela@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.1 on 10.11.54.9 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=quintela@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1684180735882100001 That is the moment we know we have transferred something. Signed-off-by: Juan Quintela Reviewed-by: C=C3=A9dric Le Goater Reviewed-by: Leonardo Bras --- migration/qemu-file.c | 7 +++---- 1 file changed, 3 insertions(+), 4 deletions(-) diff --git a/migration/qemu-file.c b/migration/qemu-file.c index 4bc875b452..956bd2a580 100644 --- a/migration/qemu-file.c +++ b/migration/qemu-file.c @@ -302,7 +302,9 @@ void qemu_fflush(QEMUFile *f) &local_error) < 0) { qemu_file_set_error_obj(f, -EIO, local_error); } else { - f->total_transferred +=3D iov_size(f->iov, f->iovcnt); + uint64_t size =3D iov_size(f->iov, f->iovcnt); + qemu_file_acct_rate_limit(f, size); + f->total_transferred +=3D size; } =20 qemu_iovec_release_ram(f); @@ -519,7 +521,6 @@ void qemu_put_buffer_async(QEMUFile *f, const uint8_t *= buf, size_t size, return; } =20 - f->rate_limit_used +=3D size; add_to_iovec(f, buf, size, may_free); } =20 @@ -537,7 +538,6 @@ void qemu_put_buffer(QEMUFile *f, const uint8_t *buf, s= ize_t size) l =3D size; } memcpy(f->buf + f->buf_index, buf, l); - f->rate_limit_used +=3D l; add_buf_to_iovec(f, l); if (qemu_file_get_error(f)) { break; @@ -554,7 +554,6 @@ void qemu_put_byte(QEMUFile *f, int v) } =20 f->buf[f->buf_index] =3D v; - f->rate_limit_used++; add_buf_to_iovec(f, 1); } =20 --=20 2.40.1 From nobody Mon May 13 18:32:02 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1684180710; cv=none; d=zohomail.com; s=zohoarc; b=f+O9aFEyuMfDKOwRa9oPmJu5xR6F7nyv7MIrhA9HSDFiKM3QaPHzVq2mCL9vKB6HqgOnAiHR2byvqltbyM0hRlJCgiCeqj8BeH6/BWV9k4NPGMPmDZCilUon4ZLw2957l1AdsNsvRYLSaKW2XW1PfPqjAGIAcd+tRgYSlx32DvU= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1684180710; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=EN6XYOkkV312Bdr5yyuCmAMqH33G8DerrB7QE8xvnZs=; b=LEjKgBr2VVRs8ZeluvIayV3Sqfbhg+ACIpskQ1rexyNOah1BeLks/xFxSJJWtYdQc1Yo4KU7Rm924OmlKtKfRVM9/E80hIRupyOagZVPB2FbOLo3sOwBwhMLEqeIR1KEBpQ1N/JMKwB2d7uCzI1B4r8VSRd4A4hUtNve55GWcA0= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1684180710053789.2544143873486; Mon, 15 May 2023 12:58:30 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pyeKT-0007yH-Ff; Mon, 15 May 2023 15:57:57 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pyeKO-0007a1-U8 for qemu-devel@nongnu.org; Mon, 15 May 2023 15:57:53 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pyeKK-0005Mp-Vb for qemu-devel@nongnu.org; Mon, 15 May 2023 15:57:52 -0400 Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-468-UGKygdn5NCm0d_UCwFJDiQ-1; Mon, 15 May 2023 15:57:43 -0400 Received: from smtp.corp.redhat.com (int-mx09.intmail.prod.int.rdu2.redhat.com [10.11.54.9]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 28ED71C0A581; Mon, 15 May 2023 19:57:42 +0000 (UTC) Received: from secure.mitica (unknown [10.39.194.171]) by smtp.corp.redhat.com (Postfix) with ESMTP id 30814492B00; Mon, 15 May 2023 19:57:38 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1684180665; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=EN6XYOkkV312Bdr5yyuCmAMqH33G8DerrB7QE8xvnZs=; b=UBPDrKURZrpRP03N3lXXFrx12XL1qDvjy7opifMLlIzJu2q5iarReU2LaNcxxmIcCE5wGI GozAcXt4jsI4B6dx3y+972MBliGU3yTUJs4t51TrCF4odv94Xigy8i6usUyKVILsEYvjQt N/0mB/tIpkRu5C50mBVdSO4BArTenLg= X-MC-Unique: UGKygdn5NCm0d_UCwFJDiQ-1 From: Juan Quintela To: qemu-devel@nongnu.org Cc: Peter Xu , Greg Kurz , qemu-s390x@nongnu.org, Fam Zheng , Ilya Leoshkevich , Halil Pasic , John Snow , qemu-ppc@nongnu.org, Daniel Henrique Barboza , Harsh Prateek Bora , Christian Borntraeger , Richard Henderson , David Gibson , David Hildenbrand , Stefan Hajnoczi , Eric Farman , qemu-block@nongnu.org, =?UTF-8?q?C=C3=A9dric=20Le=20Goater?= , Eric Blake , Leonardo Bras , Thomas Huth , Juan Quintela , Vladimir Sementsov-Ogievskiy Subject: [PATCH v2 05/16] migration: Move rate_limit_max and rate_limit_used to migration_stats Date: Mon, 15 May 2023 21:56:58 +0200 Message-Id: <20230515195709.63843-6-quintela@redhat.com> In-Reply-To: <20230515195709.63843-1-quintela@redhat.com> References: <20230515195709.63843-1-quintela@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.1 on 10.11.54.9 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=quintela@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1684180712231100003 Content-Type: text/plain; charset="utf-8" These way we can make them atomic and use this functions from any place. I also moved all functions that use rate_limit to migration-stats. Functions got renamed, they are not qemu_file anymore. qemu_file_rate_limit -> migration_rate_exceeded qemu_file_set_rate_limit -> migration_rate_set qemu_file_get_rate_limit -> migration_rate_get qemu_file_reset_rate_limit -> migration_rate_reset qemu_file_acct_rate_limit -> migration_rate_account. Signed-off-by: Juan Quintela Reviewed-by: Harsh Prateek Bora Reviewed-by: C=C3=A9dric Le Goater --- s/this/these/ (harsh) If you have any good suggestion for better names, I am all ears. Fix missing / XFER_LIMIT_RATIO in migration_rate_set(quintela) --- include/migration/qemu-file-types.h | 12 ++++++- migration/migration-stats.h | 47 ++++++++++++++++++++++++++ migration/options.h | 7 ---- migration/qemu-file.h | 11 ------ hw/ppc/spapr.c | 4 +-- hw/s390x/s390-stattrib.c | 2 +- migration/block-dirty-bitmap.c | 2 +- migration/block.c | 5 +-- migration/migration-stats.c | 44 ++++++++++++++++++++++++ migration/migration.c | 14 ++++---- migration/multifd.c | 2 +- migration/options.c | 7 ++-- migration/qemu-file.c | 52 ++--------------------------- migration/ram.c | 2 +- migration/savevm.c | 2 +- 15 files changed, 124 insertions(+), 89 deletions(-) diff --git a/include/migration/qemu-file-types.h b/include/migration/qemu-f= ile-types.h index 1436f9ce92..9ba163f333 100644 --- a/include/migration/qemu-file-types.h +++ b/include/migration/qemu-file-types.h @@ -165,6 +165,16 @@ size_t coroutine_mixed_fn qemu_get_counted_string(QEMU= File *f, char buf[256]); =20 void qemu_put_counted_string(QEMUFile *f, const char *name); =20 -int qemu_file_rate_limit(QEMUFile *f); +/** + * migration_rate_exceeded: Check if we have exceeded rate for this interv= al + * + * Checks if we have already transferred more data that we are allowed + * in the current interval. + * + * @f: QEMUFile used for main migration channel + * + * Returns if we should stop sending data for this interval. + */ +bool migration_rate_exceeded(QEMUFile *f); =20 #endif diff --git a/migration/migration-stats.h b/migration/migration-stats.h index 21402af9e4..e39c083245 100644 --- a/migration/migration-stats.h +++ b/migration/migration-stats.h @@ -15,6 +15,12 @@ =20 #include "qemu/stats64.h" =20 +/* + * Amount of time to allocate to each "chunk" of bandwidth-throttled + * data. + */ +#define BUFFER_DELAY 100 + /* * If rate_limit_max is 0, there is special code to remove the rate * limit. @@ -75,6 +81,14 @@ typedef struct { * Number of bytes sent during precopy stage. */ Stat64 precopy_bytes; + /* + * Maximum amount of data we can send in a cycle. + */ + Stat64 rate_limit_max; + /* + * Amount of data we have sent in the current cycle. + */ + Stat64 rate_limit_used; /* * How long has the setup stage took. */ @@ -100,4 +114,37 @@ extern MigrationAtomicStats mig_stats; * Returns: Nothing. The time is stored in val. */ void migration_time_since(MigrationAtomicStats *stats, int64_t since); + +/** + * migration_rate_account: Increase the number of bytes transferred. + * + * Report on a number of bytes the have been transferred that need to + * be applied to the rate limiting calcuations. + * + * @len: amount of bytes transferred + */ +void migration_rate_account(uint64_t len); + +/** + * migration_rate_get: Get the maximum amount that can be transferred. + * + * Returns the maximum number of bytes that can be transferred in a cycle. + */ +uint64_t migration_rate_get(void); + +/** + * migration_rate_reset: Reset the rate limit counter. + * + * This is called when we know we start a new transfer cycle. + */ +void migration_rate_reset(void); + +/** + * migration_rate_set: Set the maximum amount that can be transferred. + * + * Sets the maximum amount of bytes that can be transferred in one cycle. + * + * @new_rate: new maximum amount + */ +void migration_rate_set(uint64_t new_rate); #endif diff --git a/migration/options.h b/migration/options.h index 5cca3326d6..45991af3c2 100644 --- a/migration/options.h +++ b/migration/options.h @@ -17,13 +17,6 @@ #include "hw/qdev-properties.h" #include "hw/qdev-properties-system.h" =20 -/* constants */ - -/* Amount of time to allocate to each "chunk" of bandwidth-throttled - * data. */ -#define BUFFER_DELAY 100 -#define XFER_LIMIT_RATIO (1000 / BUFFER_DELAY) - /* migration properties */ =20 extern Property migration_properties[]; diff --git a/migration/qemu-file.h b/migration/qemu-file.h index bcc39081f2..e649718492 100644 --- a/migration/qemu-file.h +++ b/migration/qemu-file.h @@ -130,17 +130,6 @@ void qemu_file_skip(QEMUFile *f, int size); * accounting information tracks the total migration traffic. */ void qemu_file_credit_transfer(QEMUFile *f, size_t size); -void qemu_file_reset_rate_limit(QEMUFile *f); -/* - * qemu_file_acct_rate_limit: - * - * Report on a number of bytes the have been transferred - * out of band from the main file object I/O methods, and - * need to be applied to the rate limiting calcuations - */ -void qemu_file_acct_rate_limit(QEMUFile *f, uint64_t len); -void qemu_file_set_rate_limit(QEMUFile *f, uint64_t new_rate); -uint64_t qemu_file_get_rate_limit(QEMUFile *f); int qemu_file_get_error_obj(QEMUFile *f, Error **errp); int qemu_file_get_error_obj_any(QEMUFile *f1, QEMUFile *f2, Error **errp); void qemu_file_set_error_obj(QEMUFile *f, int ret, Error *err); diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c index ddc9c7b1a1..1baea16c96 100644 --- a/hw/ppc/spapr.c +++ b/hw/ppc/spapr.c @@ -2166,7 +2166,7 @@ static void htab_save_first_pass(QEMUFile *f, SpaprMa= chineState *spapr, break; } } - } while ((index < htabslots) && !qemu_file_rate_limit(f)); + } while ((index < htabslots) && !migration_rate_exceeded(f)); =20 if (index >=3D htabslots) { assert(index =3D=3D htabslots); @@ -2237,7 +2237,7 @@ static int htab_save_later_pass(QEMUFile *f, SpaprMac= hineState *spapr, assert(index =3D=3D htabslots); index =3D 0; } - } while ((examined < htabslots) && (!qemu_file_rate_limit(f) || final)= ); + } while ((examined < htabslots) && (!migration_rate_exceeded(f) || fin= al)); =20 if (index >=3D htabslots) { assert(index =3D=3D htabslots); diff --git a/hw/s390x/s390-stattrib.c b/hw/s390x/s390-stattrib.c index aed919ad7d..220e845d12 100644 --- a/hw/s390x/s390-stattrib.c +++ b/hw/s390x/s390-stattrib.c @@ -209,7 +209,7 @@ static int cmma_save(QEMUFile *f, void *opaque, int fin= al) return -ENOMEM; } =20 - while (final ? 1 : qemu_file_rate_limit(f) =3D=3D 0) { + while (final ? 1 : migration_rate_exceeded(f) =3D=3D 0) { reallen =3D sac->get_stattr(sas, &start_gfn, buflen, buf); if (reallen < 0) { g_free(buf); diff --git a/migration/block-dirty-bitmap.c b/migration/block-dirty-bitmap.c index 20f36e6bd8..032fc5f405 100644 --- a/migration/block-dirty-bitmap.c +++ b/migration/block-dirty-bitmap.c @@ -706,7 +706,7 @@ static void bulk_phase(QEMUFile *f, DBMSaveState *s, bo= ol limit) QSIMPLEQ_FOREACH(dbms, &s->dbms_list, entry) { while (!dbms->bulk_completed) { bulk_phase_send_chunk(f, s, dbms); - if (limit && qemu_file_rate_limit(f)) { + if (limit && migration_rate_exceeded(f)) { return; } } diff --git a/migration/block.c b/migration/block.c index 12617b4152..b9580a6c7e 100644 --- a/migration/block.c +++ b/migration/block.c @@ -23,6 +23,7 @@ #include "block/dirty-bitmap.h" #include "migration/misc.h" #include "migration.h" +#include "migration-stats.h" #include "migration/register.h" #include "qemu-file.h" #include "migration/vmstate.h" @@ -625,7 +626,7 @@ static int flush_blks(QEMUFile *f) =20 blk_mig_lock(); while ((blk =3D QSIMPLEQ_FIRST(&block_mig_state.blk_list)) !=3D NULL) { - if (qemu_file_rate_limit(f)) { + if (migration_rate_exceeded(f)) { break; } if (blk->ret < 0) { @@ -762,7 +763,7 @@ static int block_save_iterate(QEMUFile *f, void *opaque) /* control the rate of transfer */ blk_mig_lock(); while (block_mig_state.read_done * BLK_MIG_BLOCK_SIZE < - qemu_file_get_rate_limit(f) && + migration_rate_get() && block_mig_state.submitted < MAX_PARALLEL_IO && (block_mig_state.submitted + block_mig_state.read_done) < MAX_IO_BUFFERS) { diff --git a/migration/migration-stats.c b/migration/migration-stats.c index 3431453c90..1b16edae7d 100644 --- a/migration/migration-stats.c +++ b/migration/migration-stats.c @@ -13,6 +13,7 @@ #include "qemu/osdep.h" #include "qemu/stats64.h" #include "qemu/timer.h" +#include "qemu-file.h" #include "migration-stats.h" =20 MigrationAtomicStats mig_stats; @@ -22,3 +23,46 @@ void migration_time_since(MigrationAtomicStats *stats, i= nt64_t since) int64_t now =3D qemu_clock_get_ms(QEMU_CLOCK_HOST); stat64_set(&stats->setup_time, now - since); } + +bool migration_rate_exceeded(QEMUFile *f) +{ + if (qemu_file_get_error(f)) { + return true; + } + + uint64_t rate_limit_used =3D stat64_get(&mig_stats.rate_limit_used); + uint64_t rate_limit_max =3D stat64_get(&mig_stats.rate_limit_max); + + if (rate_limit_max =3D=3D RATE_LIMIT_MAX) { + return false; + } + if (rate_limit_max > 0 && rate_limit_used > rate_limit_max) { + return true; + } + return false; +} + +uint64_t migration_rate_get(void) +{ + return stat64_get(&mig_stats.rate_limit_max); +} + +#define XFER_LIMIT_RATIO (1000 / BUFFER_DELAY) + +void migration_rate_set(uint64_t limit) +{ + /* + * 'limit' is per second. But we check it each BUFER_DELAY milisecond= s. + */ + stat64_set(&mig_stats.rate_limit_max, limit / XFER_LIMIT_RATIO); +} + +void migration_rate_reset(void) +{ + stat64_set(&mig_stats.rate_limit_used, 0); +} + +void migration_rate_account(uint64_t len) +{ + stat64_add(&mig_stats.rate_limit_used, len); +} diff --git a/migration/migration.c b/migration/migration.c index e9466273bb..594709dbbc 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -2119,7 +2119,7 @@ static int postcopy_start(MigrationState *ms) * will notice we're in POSTCOPY_ACTIVE and not actually * wrap their state up here */ - qemu_file_set_rate_limit(ms->to_dst_file, bandwidth); + migration_rate_set(bandwidth); if (migrate_postcopy_ram()) { /* Ping just for debugging, helps line traces up */ qemu_savevm_send_ping(ms->to_dst_file, 2); @@ -2303,7 +2303,7 @@ static void migration_completion(MigrationState *s) * them if migration fails or is cancelled. */ s->block_inactive =3D !migrate_colo(); - qemu_file_set_rate_limit(s->to_dst_file, RATE_LIMIT_MAX); + migration_rate_set(RATE_LIMIT_MAX); ret =3D qemu_savevm_state_complete_precopy(s->to_dst_file,= false, s->block_inactive= ); } @@ -2698,7 +2698,7 @@ static void migration_update_counters(MigrationState = *s, stat64_get(&mig_stats.dirty_bytes_last_sync) / bandwidth; } =20 - qemu_file_reset_rate_limit(s->to_dst_file); + migration_rate_reset(); =20 update_iteration_initial_status(s); =20 @@ -2851,7 +2851,7 @@ bool migration_rate_limit(void) =20 bool urgent =3D false; migration_update_counters(s, now); - if (qemu_file_rate_limit(s->to_dst_file)) { + if (migration_rate_exceeded(s->to_dst_file)) { =20 if (qemu_file_get_error(s->to_dst_file)) { return false; @@ -2973,7 +2973,7 @@ static void *migration_thread(void *opaque) trace_migration_thread_setup_complete(); =20 while (migration_is_active(s)) { - if (urgent || !qemu_file_rate_limit(s->to_dst_file)) { + if (urgent || !migration_rate_exceeded(s->to_dst_file)) { MigIterateState iter_state =3D migration_iteration_run(s); if (iter_state =3D=3D MIG_ITERATE_SKIP) { continue; @@ -3047,7 +3047,7 @@ static void *bg_migration_thread(void *opaque) rcu_register_thread(); object_ref(OBJECT(s)); =20 - qemu_file_set_rate_limit(s->to_dst_file, RATE_LIMIT_MAX); + migration_rate_set(RATE_LIMIT_MAX); =20 setup_start =3D qemu_clock_get_ms(QEMU_CLOCK_HOST); /* @@ -3219,7 +3219,7 @@ void migrate_fd_connect(MigrationState *s, Error *err= or_in) notifier_list_notify(&migration_state_notifiers, s); } =20 - qemu_file_set_rate_limit(s->to_dst_file, rate_limit); + migration_rate_set(rate_limit); qemu_file_set_blocking(s->to_dst_file, true); =20 /* diff --git a/migration/multifd.c b/migration/multifd.c index 5c4298eadf..5052091ce2 100644 --- a/migration/multifd.c +++ b/migration/multifd.c @@ -431,7 +431,7 @@ static int multifd_send_pages(QEMUFile *f) multifd_send_state->pages =3D p->pages; p->pages =3D pages; transferred =3D ((uint64_t) pages->num) * p->page_size + p->packet_len; - qemu_file_acct_rate_limit(f, transferred); + migration_rate_account(transferred); qemu_mutex_unlock(&p->mutex); stat64_add(&mig_stats.transferred, transferred); stat64_add(&mig_stats.multifd_bytes, transferred); diff --git a/migration/options.c b/migration/options.c index c2a278ee2d..b62ab30cd5 100644 --- a/migration/options.c +++ b/migration/options.c @@ -23,6 +23,7 @@ #include "migration/colo.h" #include "migration/misc.h" #include "migration.h" +#include "migration-stats.h" #include "qemu-file.h" #include "ram.h" #include "options.h" @@ -1242,8 +1243,7 @@ static void migrate_params_apply(MigrateSetParameters= *params, Error **errp) if (params->has_max_bandwidth) { s->parameters.max_bandwidth =3D params->max_bandwidth; if (s->to_dst_file && !migration_in_postcopy()) { - qemu_file_set_rate_limit(s->to_dst_file, - s->parameters.max_bandwidth); + migration_rate_set(s->parameters.max_bandwidth); } } =20 @@ -1272,8 +1272,7 @@ static void migrate_params_apply(MigrateSetParameters= *params, Error **errp) if (params->has_max_postcopy_bandwidth) { s->parameters.max_postcopy_bandwidth =3D params->max_postcopy_band= width; if (s->to_dst_file && migration_in_postcopy()) { - qemu_file_set_rate_limit(s->to_dst_file, - s->parameters.max_postcopy_bandwidth); + migration_rate_set(s->parameters.max_postcopy_bandwidth); } } if (params->has_max_cpu_throttle) { diff --git a/migration/qemu-file.c b/migration/qemu-file.c index 956bd2a580..9c67b52fe0 100644 --- a/migration/qemu-file.c +++ b/migration/qemu-file.c @@ -41,17 +41,6 @@ struct QEMUFile { QIOChannel *ioc; bool is_writable; =20 - /* - * Maximum amount of data in bytes to transfer during one - * rate limiting time window - */ - uint64_t rate_limit_max; - /* - * Total amount of data in bytes queued for transfer - * during this rate limiting time window - */ - uint64_t rate_limit_used; - /* The sum of bytes transferred on the wire */ uint64_t total_transferred; =20 @@ -303,7 +292,7 @@ void qemu_fflush(QEMUFile *f) qemu_file_set_error_obj(f, -EIO, local_error); } else { uint64_t size =3D iov_size(f->iov, f->iovcnt); - qemu_file_acct_rate_limit(f, size); + migration_rate_account(size); f->total_transferred +=3D size; } =20 @@ -356,7 +345,7 @@ size_t ram_control_save_page(QEMUFile *f, ram_addr_t bl= ock_offset, int ret =3D f->hooks->save_page(f, block_offset, offset, size, bytes_sent); if (ret !=3D RAM_SAVE_CONTROL_NOT_SUPP) { - qemu_file_acct_rate_limit(f, size); + migration_rate_account(size); } =20 if (ret !=3D RAM_SAVE_CONTROL_DELAYED && @@ -727,43 +716,6 @@ uint64_t qemu_file_transferred(QEMUFile *f) return f->total_transferred; } =20 -int qemu_file_rate_limit(QEMUFile *f) -{ - if (qemu_file_get_error(f)) { - return 1; - } - if (f->rate_limit_max =3D=3D RATE_LIMIT_MAX) { - return 0; - } - if (f->rate_limit_used > f->rate_limit_max) { - return 1; - } - return 0; -} - -uint64_t qemu_file_get_rate_limit(QEMUFile *f) -{ - return f->rate_limit_max; -} - -void qemu_file_set_rate_limit(QEMUFile *f, uint64_t limit) -{ - /* - * 'limit' is per second. But we check it each 100 miliseconds. - */ - f->rate_limit_max =3D limit / XFER_LIMIT_RATIO; -} - -void qemu_file_reset_rate_limit(QEMUFile *f) -{ - f->rate_limit_used =3D 0; -} - -void qemu_file_acct_rate_limit(QEMUFile *f, uint64_t len) -{ - f->rate_limit_used +=3D len; -} - void qemu_put_be16(QEMUFile *f, unsigned int v) { qemu_put_byte(f, v >> 8); diff --git a/migration/ram.c b/migration/ram.c index fd5a8db0f8..a706edecc0 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -3126,7 +3126,7 @@ static int ram_save_iterate(QEMUFile *f, void *opaque) =20 t0 =3D qemu_clock_get_ns(QEMU_CLOCK_REALTIME); i =3D 0; - while ((ret =3D qemu_file_rate_limit(f)) =3D=3D 0 || + while ((ret =3D migration_rate_exceeded(f)) =3D=3D 0 || postcopy_has_request(rs)) { int pages; =20 diff --git a/migration/savevm.c b/migration/savevm.c index c7af9050c2..6b783d9c39 100644 --- a/migration/savevm.c +++ b/migration/savevm.c @@ -1345,7 +1345,7 @@ int qemu_savevm_state_iterate(QEMUFile *f, bool postc= opy) !(se->ops->has_postcopy && se->ops->has_postcopy(se->opaque)))= { continue; } - if (qemu_file_rate_limit(f)) { + if (migration_rate_exceeded(f)) { return 0; } trace_savevm_section_start(se->idstr, se->section_id); --=20 2.40.1 From nobody Mon May 13 18:32:02 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1684180794; cv=none; d=zohomail.com; s=zohoarc; b=b9FBMuLnNRiimRrc1Kc3u9f2j72H5OLXja3qFhO52JUfzZzU7+wzbF3CXI/cAlPhJHlEcEI+5JXGcgt92Vthgf9dwy5nsGaxqHYA6JFwN+IRBbngJ0mSfnWin4kcXxL0nfc46jVqfmQaTVLXGk3+Io7J0D4rzPMmlDfFo3PjkVE= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1684180794; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=ghmx+kI9dt2gIdE/AzzkaNMrKaLWXjfyrkUNIMra0so=; b=QLE1kNoMf+4SWDs2xOk2SHQh/E5v2WKIH30h8X5bg1CTBBtrP2uo/RzHQnrmCbW4HKrg6pbME266y5/6SBLaASrHgyOHU4etjV/fwi7nGnfgAK+2ND9CorDSzytCp5LIeIZlLkerprgstuVG3fQ+eln/5erbIvb6wuQCAxA4l9I= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1684180794821160.36297617464663; Mon, 15 May 2023 12:59:54 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pyeKi-0000cw-53; Mon, 15 May 2023 15:58:12 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pyeKe-0000O0-HB for qemu-devel@nongnu.org; Mon, 15 May 2023 15:58:08 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pyeKN-0005No-Gx for qemu-devel@nongnu.org; Mon, 15 May 2023 15:58:08 -0400 Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-5-80JIUr_kM5-KmoZyNMLqWA-1; Mon, 15 May 2023 15:57:47 -0400 Received: from smtp.corp.redhat.com (int-mx09.intmail.prod.int.rdu2.redhat.com [10.11.54.9]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 37BC185A588; Mon, 15 May 2023 19:57:46 +0000 (UTC) Received: from secure.mitica (unknown [10.39.194.171]) by smtp.corp.redhat.com (Postfix) with ESMTP id 6E61F492B00; Mon, 15 May 2023 19:57:42 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1684180670; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ghmx+kI9dt2gIdE/AzzkaNMrKaLWXjfyrkUNIMra0so=; b=AOAx2txNxOSKMmc5cF7Aio9M2CoyFGGeGtZGmn7xj5PJhNs7YqP6YmGCQ+EcmFmIAkc+Sb 7q77k9TpOkOeWD22pwh3gDcqpnfaAA+d3/uBnZXV7DlBtivBMfTDhESMfZb3OysmNx5G7j tJvkWbQRX3r+vIzxMTHVueQYGFmlzCU= X-MC-Unique: 80JIUr_kM5-KmoZyNMLqWA-1 From: Juan Quintela To: qemu-devel@nongnu.org Cc: Peter Xu , Greg Kurz , qemu-s390x@nongnu.org, Fam Zheng , Ilya Leoshkevich , Halil Pasic , John Snow , qemu-ppc@nongnu.org, Daniel Henrique Barboza , Harsh Prateek Bora , Christian Borntraeger , Richard Henderson , David Gibson , David Hildenbrand , Stefan Hajnoczi , Eric Farman , qemu-block@nongnu.org, =?UTF-8?q?C=C3=A9dric=20Le=20Goater?= , Eric Blake , Leonardo Bras , Thomas Huth , Juan Quintela , Vladimir Sementsov-Ogievskiy Subject: [PATCH v2 06/16] migration: Move migration_total_bytes() to migration-stats.c Date: Mon, 15 May 2023 21:56:59 +0200 Message-Id: <20230515195709.63843-7-quintela@redhat.com> In-Reply-To: <20230515195709.63843-1-quintela@redhat.com> References: <20230515195709.63843-1-quintela@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.1 on 10.11.54.9 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=quintela@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1684180796188100011 Once there rename it to migration_transferred_bytes() and pass a QEMUFile instead of a migration object. Signed-off-by: Juan Quintela Reviewed-by: C=C3=A9dric Le Goater Reviewed-by: Leonardo Bras --- migration/migration-stats.h | 11 +++++++++++ migration/migration-stats.c | 6 ++++++ migration/migration.c | 13 +++---------- 3 files changed, 20 insertions(+), 10 deletions(-) diff --git a/migration/migration-stats.h b/migration/migration-stats.h index e39c083245..91fda378d3 100644 --- a/migration/migration-stats.h +++ b/migration/migration-stats.h @@ -147,4 +147,15 @@ void migration_rate_reset(void); * @new_rate: new maximum amount */ void migration_rate_set(uint64_t new_rate); + +/** + * migration_transferred_bytes: Return number of bytes transferred + * + * @f: QEMUFile used for main migration channel + * + * Returns how many bytes have we transferred since the beginning of + * the migration. It accounts for bytes sent through any migration + * channel, multifd, qemu_file, rdma, .... + */ +uint64_t migration_transferred_bytes(QEMUFile *f); #endif diff --git a/migration/migration-stats.c b/migration/migration-stats.c index 1b16edae7d..9bd97caa23 100644 --- a/migration/migration-stats.c +++ b/migration/migration-stats.c @@ -66,3 +66,9 @@ void migration_rate_account(uint64_t len) { stat64_add(&mig_stats.rate_limit_used, len); } + +uint64_t migration_transferred_bytes(QEMUFile *f) +{ + return qemu_file_transferred(f) + stat64_get(&mig_stats.multifd_bytes); +} + diff --git a/migration/migration.c b/migration/migration.c index 594709dbbc..39ff538046 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -2624,16 +2624,9 @@ static MigThrError migration_detect_error(MigrationS= tate *s) } } =20 -/* How many bytes have we transferred since the beginning of the migration= */ -static uint64_t migration_total_bytes(MigrationState *s) -{ - return qemu_file_transferred(s->to_dst_file) + - stat64_get(&mig_stats.multifd_bytes); -} - static void migration_calculate_complete(MigrationState *s) { - uint64_t bytes =3D migration_total_bytes(s); + uint64_t bytes =3D migration_transferred_bytes(s->to_dst_file); int64_t end_time =3D qemu_clock_get_ms(QEMU_CLOCK_REALTIME); int64_t transfer_time; =20 @@ -2659,7 +2652,7 @@ static void update_iteration_initial_status(Migration= State *s) * wrong speed calculation. */ s->iteration_start_time =3D qemu_clock_get_ms(QEMU_CLOCK_REALTIME); - s->iteration_initial_bytes =3D migration_total_bytes(s); + s->iteration_initial_bytes =3D migration_transferred_bytes(s->to_dst_f= ile); s->iteration_initial_pages =3D ram_get_total_transferred_pages(); } =20 @@ -2674,7 +2667,7 @@ static void migration_update_counters(MigrationState = *s, return; } =20 - current_bytes =3D migration_total_bytes(s); + current_bytes =3D migration_transferred_bytes(s->to_dst_file); transferred =3D current_bytes - s->iteration_initial_bytes; time_spent =3D current_time - s->iteration_start_time; bandwidth =3D (double)transferred / time_spent; --=20 2.40.1 From nobody Mon May 13 18:32:02 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1684180727; cv=none; d=zohomail.com; s=zohoarc; b=jl1UjiICi3zVy8Qp0Zl/uHSs5nSIW6LiRTrjUM4+D3m2QSEnr6rHlA3RqEoj3rv5fjRJP1nAQAjn2LK/aPJ51eLZZ9Xi8efehZ6eSCb+p3BdziCYPCHFrPtqxgPdsSUUdski/sg+Zv0RHQhqemwiiRHVdnkRXclutt7Rsr3Zy5c= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1684180727; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=XwOo5l6s6g5v2W4LKjokXrwRyxz7Hp3IhysEG1zOzao=; b=RjklIMnyYzSlsQBOSatiKu7dq6J3upvWGL/zw60RmkPkWo/P6ydCJhiuydiOHUHKztMDx6YP7W/S4Ss8Ye61oLDXHk8JQsM/0L+RMzXHQ7iax3rBowL4f4o/ArbYdmmfLLdwW/3yi4cNShz7iOevUGg59KeqLAljrCln4pevKTM= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1684180727061439.2005448910933; Mon, 15 May 2023 12:58:47 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pyeKV-0008E9-Rb; Mon, 15 May 2023 15:57:59 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pyeKU-00084P-8s for qemu-devel@nongnu.org; Mon, 15 May 2023 15:57:58 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pyeKS-0005PF-53 for qemu-devel@nongnu.org; Mon, 15 May 2023 15:57:58 -0400 Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-196-pwLtuQWPMwqq6XIFOsObmw-1; Mon, 15 May 2023 15:57:51 -0400 Received: from smtp.corp.redhat.com (int-mx09.intmail.prod.int.rdu2.redhat.com [10.11.54.9]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 55C0885C074; Mon, 15 May 2023 19:57:50 +0000 (UTC) Received: from secure.mitica (unknown [10.39.194.171]) by smtp.corp.redhat.com (Postfix) with ESMTP id 7E306492B00; Mon, 15 May 2023 19:57:46 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1684180675; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=XwOo5l6s6g5v2W4LKjokXrwRyxz7Hp3IhysEG1zOzao=; b=UU0qzYLicHHTQYC+lQnexbjJfzsq2+lzXaIplXupaNfM67U0ggaoQm98Y6j3eQrQi3cmvB ctAPoX4PiqKs3nIubvcdEMhCt2pLELUJDhOwvfCXAAOZLzpj0y1IxfXyuAPfXq+fhyU6kO BLaXOIwtSbTitCcf6Hb1GUyfOcaVCWY= X-MC-Unique: pwLtuQWPMwqq6XIFOsObmw-1 From: Juan Quintela To: qemu-devel@nongnu.org Cc: Peter Xu , Greg Kurz , qemu-s390x@nongnu.org, Fam Zheng , Ilya Leoshkevich , Halil Pasic , John Snow , qemu-ppc@nongnu.org, Daniel Henrique Barboza , Harsh Prateek Bora , Christian Borntraeger , Richard Henderson , David Gibson , David Hildenbrand , Stefan Hajnoczi , Eric Farman , qemu-block@nongnu.org, =?UTF-8?q?C=C3=A9dric=20Le=20Goater?= , Eric Blake , Leonardo Bras , Thomas Huth , Juan Quintela , Vladimir Sementsov-Ogievskiy Subject: [PATCH v2 07/16] migration: Add a trace for migration_transferred_bytes Date: Mon, 15 May 2023 21:57:00 +0200 Message-Id: <20230515195709.63843-8-quintela@redhat.com> In-Reply-To: <20230515195709.63843-1-quintela@redhat.com> References: <20230515195709.63843-1-quintela@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.1 on 10.11.54.9 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=quintela@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1684180728433100005 Signed-off-by: Juan Quintela Reviewed-by: C=C3=A9dric Le Goater Reviewed-by: Leonardo Bras --- migration/migration-stats.c | 8 ++++++-- migration/trace-events | 3 +++ 2 files changed, 9 insertions(+), 2 deletions(-) diff --git a/migration/migration-stats.c b/migration/migration-stats.c index 9bd97caa23..301392d208 100644 --- a/migration/migration-stats.c +++ b/migration/migration-stats.c @@ -14,6 +14,7 @@ #include "qemu/stats64.h" #include "qemu/timer.h" #include "qemu-file.h" +#include "trace.h" #include "migration-stats.h" =20 MigrationAtomicStats mig_stats; @@ -69,6 +70,9 @@ void migration_rate_account(uint64_t len) =20 uint64_t migration_transferred_bytes(QEMUFile *f) { - return qemu_file_transferred(f) + stat64_get(&mig_stats.multifd_bytes); -} + uint64_t multifd =3D stat64_get(&mig_stats.multifd_bytes); + uint64_t qemu_file =3D qemu_file_transferred(f); =20 + trace_migration_transferred_bytes(qemu_file, multifd); + return qemu_file + multifd; +} diff --git a/migration/trace-events b/migration/trace-events index f39818c329..cdaef7a1ea 100644 --- a/migration/trace-events +++ b/migration/trace-events @@ -186,6 +186,9 @@ process_incoming_migration_co_end(int ret, int ps) "ret= =3D%d postcopy-state=3D%d" process_incoming_migration_co_postcopy_end_main(void) "" postcopy_preempt_enabled(bool value) "%d" =20 +# migration-stats +migration_transferred_bytes(uint64_t qemu_file, uint64_t multifd) "qemu_fi= le %" PRIu64 " multifd %" PRIu64 + # channel.c migration_set_incoming_channel(void *ioc, const char *ioctype) "ioc=3D%p i= octype=3D%s" migration_set_outgoing_channel(void *ioc, const char *ioctype, const char = *hostname, void *err) "ioc=3D%p ioctype=3D%s hostname=3D%s err=3D%p" --=20 2.40.1 From nobody Mon May 13 18:32:02 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1684180846; cv=none; d=zohomail.com; s=zohoarc; b=gHng0LOJdBxiV9wQkRviJ4/DY4R9biwTAr1JvCYPi3y8hVZsFv7UjwccwOQd3+rVGn3/Gpn2U0RRisASXaDLw/Z5VJgJ1f+6xqzZMqEpk2vz1SU0KX/axpdt4LWjNQoluwEL7p0haGxxSVcwMbVH918VUvdD44uJTU6LZv6S098= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1684180846; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=ETJF60hxjmKnJX+sqEumgfInbHjCRkAKUrzHn6nnpNQ=; b=IBDuXwiAaZSbsqD+1TDQ7bJp5BaWNvXStafmb+PJupTWesW5x5t5DzKyYaeDuubCuuU9luiL5OnWANgRxJp2AZC2d4hs2NuBa3clm5cUn6uNTQGzyNgeiE8sDQjYlH7ccCDyYGL1UnMq8NQbK5xwebdyvbGg4ZvLCb0gB3AIGEE= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1684180846747667.1323463156618; Mon, 15 May 2023 13:00:46 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pyeKZ-0008TY-RC; Mon, 15 May 2023 15:58:03 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pyeKX-0008NO-M0 for qemu-devel@nongnu.org; Mon, 15 May 2023 15:58:01 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pyeKW-0005QE-4l for qemu-devel@nongnu.org; Mon, 15 May 2023 15:58:01 -0400 Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-588-IRhQcHelPXCaaKmtw0perg-1; Mon, 15 May 2023 15:57:55 -0400 Received: from smtp.corp.redhat.com (int-mx09.intmail.prod.int.rdu2.redhat.com [10.11.54.9]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 65C352999B28; Mon, 15 May 2023 19:57:54 +0000 (UTC) Received: from secure.mitica (unknown [10.39.194.171]) by smtp.corp.redhat.com (Postfix) with ESMTP id 9B923492B00; Mon, 15 May 2023 19:57:50 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1684180679; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ETJF60hxjmKnJX+sqEumgfInbHjCRkAKUrzHn6nnpNQ=; b=EdLabVCM4A+9s5qsSlBtkaAM8779alfaDEnbWQ3LFdWjkTchIizToFNF1jGi2OUn5ldOz+ TBab6IT1sdalclFEM8PvodqyvkLlc94RYV/QRxp+GcjkcO6baFRTAdRMNu7IpbZAhrNWww uMeUWLx8/QqC6w/4/NVYB2vHCHcRcco= X-MC-Unique: IRhQcHelPXCaaKmtw0perg-1 From: Juan Quintela To: qemu-devel@nongnu.org Cc: Peter Xu , Greg Kurz , qemu-s390x@nongnu.org, Fam Zheng , Ilya Leoshkevich , Halil Pasic , John Snow , qemu-ppc@nongnu.org, Daniel Henrique Barboza , Harsh Prateek Bora , Christian Borntraeger , Richard Henderson , David Gibson , David Hildenbrand , Stefan Hajnoczi , Eric Farman , qemu-block@nongnu.org, =?UTF-8?q?C=C3=A9dric=20Le=20Goater?= , Eric Blake , Leonardo Bras , Thomas Huth , Juan Quintela , Vladimir Sementsov-Ogievskiy Subject: [PATCH v2 08/16] migration: Use migration_transferred_bytes() to calculate rate_limit Date: Mon, 15 May 2023 21:57:01 +0200 Message-Id: <20230515195709.63843-9-quintela@redhat.com> In-Reply-To: <20230515195709.63843-1-quintela@redhat.com> References: <20230515195709.63843-1-quintela@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.1 on 10.11.54.9 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=quintela@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1684180848520100007 Signed-off-by: Juan Quintela Reviewed-by: C=C3=A9dric Le Goater Reviewed-by: Leonardo Bras --- migration/migration-stats.h | 8 +++++++- migration/migration-stats.c | 7 +++++-- migration/migration.c | 2 +- 3 files changed, 13 insertions(+), 4 deletions(-) diff --git a/migration/migration-stats.h b/migration/migration-stats.h index 91fda378d3..f1465c2ebe 100644 --- a/migration/migration-stats.h +++ b/migration/migration-stats.h @@ -81,6 +81,10 @@ typedef struct { * Number of bytes sent during precopy stage. */ Stat64 precopy_bytes; + /* + * Amount of transferred data at the start of current cycle. + */ + Stat64 rate_limit_start; /* * Maximum amount of data we can send in a cycle. */ @@ -136,8 +140,10 @@ uint64_t migration_rate_get(void); * migration_rate_reset: Reset the rate limit counter. * * This is called when we know we start a new transfer cycle. + * + * @f: QEMUFile used for main migration channel */ -void migration_rate_reset(void); +void migration_rate_reset(QEMUFile *f); =20 /** * migration_rate_set: Set the maximum amount that can be transferred. diff --git a/migration/migration-stats.c b/migration/migration-stats.c index 301392d208..da2bb69a15 100644 --- a/migration/migration-stats.c +++ b/migration/migration-stats.c @@ -31,7 +31,9 @@ bool migration_rate_exceeded(QEMUFile *f) return true; } =20 - uint64_t rate_limit_used =3D stat64_get(&mig_stats.rate_limit_used); + uint64_t rate_limit_start =3D stat64_get(&mig_stats.rate_limit_start); + uint64_t rate_limit_current =3D migration_transferred_bytes(f); + uint64_t rate_limit_used =3D rate_limit_current - rate_limit_start; uint64_t rate_limit_max =3D stat64_get(&mig_stats.rate_limit_max); =20 if (rate_limit_max =3D=3D RATE_LIMIT_MAX) { @@ -58,9 +60,10 @@ void migration_rate_set(uint64_t limit) stat64_set(&mig_stats.rate_limit_max, limit / XFER_LIMIT_RATIO); } =20 -void migration_rate_reset(void) +void migration_rate_reset(QEMUFile *f) { stat64_set(&mig_stats.rate_limit_used, 0); + stat64_set(&mig_stats.rate_limit_start, migration_transferred_bytes(f)= ); } =20 void migration_rate_account(uint64_t len) diff --git a/migration/migration.c b/migration/migration.c index 39ff538046..e48dd593ed 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -2691,7 +2691,7 @@ static void migration_update_counters(MigrationState = *s, stat64_get(&mig_stats.dirty_bytes_last_sync) / bandwidth; } =20 - migration_rate_reset(); + migration_rate_reset(s->to_dst_file); =20 update_iteration_initial_status(s); =20 --=20 2.40.1 From nobody Mon May 13 18:32:02 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1684180727; cv=none; d=zohomail.com; s=zohoarc; b=OmCtA9/bGfdNeuJcv/eJFzj2vlrdhEKTqt6LsIuHlBkRmMy8AzOYzVUuO63fASDve/ME7d42ypZ/PbBK4pllrF+kOhmiwBk6cNPGftK/6qMlGSBvCD9ErDD/FKzHo5vF6p4FfDBfwh7SleWD26Fqg+L4eoH+55X0SidGs/IoL9M= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1684180727; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=G8IHujxuK+JumnUX8kPzIabXCwUMnOMo9sIpXhKBr0Q=; b=kVpxAgjdR36ViIC6iC3kTtjVT6Hqn6lq2hI26+rfTIHbKDeRf4SaeBbC9DbxYKXr+vN9vDNjqXWS3fQeoVhkX/q+QhJTcvRO4CyEbew6jL9uW8lWYwFn1DwX4cK/3kuICs94tw9uxWSW2ZymcZLJir4Pn4BPJhZocgrcZN0D+hY= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 16841807273261.456479618158255; Mon, 15 May 2023 12:58:47 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pyeKe-0000MB-4Z; Mon, 15 May 2023 15:58:08 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pyeKc-0000FJ-K1 for qemu-devel@nongnu.org; Mon, 15 May 2023 15:58:06 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pyeKb-0005RZ-05 for qemu-devel@nongnu.org; Mon, 15 May 2023 15:58:06 -0400 Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-656-23N1y86DMuGYye-uWnCWbA-1; Mon, 15 May 2023 15:57:59 -0400 Received: from smtp.corp.redhat.com (int-mx09.intmail.prod.int.rdu2.redhat.com [10.11.54.9]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 755D8101A54F; Mon, 15 May 2023 19:57:58 +0000 (UTC) Received: from secure.mitica (unknown [10.39.194.171]) by smtp.corp.redhat.com (Postfix) with ESMTP id AEF38492B00; Mon, 15 May 2023 19:57:54 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1684180684; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=G8IHujxuK+JumnUX8kPzIabXCwUMnOMo9sIpXhKBr0Q=; b=HkupFu9qNemCdMrf8gzgdJpa/D1lk4xCX0reHHSm7OHTZCXbTbDI/tffPpuuLKm8++LDIu /vYuRnNvJhiyHdpyse8ZeVD6voVEG5BUJbuux86YP/BXTmXjjjZFEVPfLHXLTFCKFEBSWz 1/EIVygVhHd9N0aTNx6IsrZ2GQj6tnI= X-MC-Unique: 23N1y86DMuGYye-uWnCWbA-1 From: Juan Quintela To: qemu-devel@nongnu.org Cc: Peter Xu , Greg Kurz , qemu-s390x@nongnu.org, Fam Zheng , Ilya Leoshkevich , Halil Pasic , John Snow , qemu-ppc@nongnu.org, Daniel Henrique Barboza , Harsh Prateek Bora , Christian Borntraeger , Richard Henderson , David Gibson , David Hildenbrand , Stefan Hajnoczi , Eric Farman , qemu-block@nongnu.org, =?UTF-8?q?C=C3=A9dric=20Le=20Goater?= , Eric Blake , Leonardo Bras , Thomas Huth , Juan Quintela , Vladimir Sementsov-Ogievskiy Subject: [PATCH v2 09/16] migration: We don't need the field rate_limit_used anymore Date: Mon, 15 May 2023 21:57:02 +0200 Message-Id: <20230515195709.63843-10-quintela@redhat.com> In-Reply-To: <20230515195709.63843-1-quintela@redhat.com> References: <20230515195709.63843-1-quintela@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.1 on 10.11.54.9 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=quintela@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1684180728445100006 Since previous commit, we calculate how much data we have send with migration_transferred_bytes() so no need to maintain this counter and remember to always update it. Signed-off-by: Juan Quintela Reviewed-by: C=C3=A9dric Le Goater Reviewed-by: Leonardo Bras --- migration/migration-stats.h | 14 -------------- migration/migration-stats.c | 6 ------ migration/multifd.c | 1 - migration/qemu-file.c | 4 ---- 4 files changed, 25 deletions(-) diff --git a/migration/migration-stats.h b/migration/migration-stats.h index f1465c2ebe..9568b5b473 100644 --- a/migration/migration-stats.h +++ b/migration/migration-stats.h @@ -89,10 +89,6 @@ typedef struct { * Maximum amount of data we can send in a cycle. */ Stat64 rate_limit_max; - /* - * Amount of data we have sent in the current cycle. - */ - Stat64 rate_limit_used; /* * How long has the setup stage took. */ @@ -119,16 +115,6 @@ extern MigrationAtomicStats mig_stats; */ void migration_time_since(MigrationAtomicStats *stats, int64_t since); =20 -/** - * migration_rate_account: Increase the number of bytes transferred. - * - * Report on a number of bytes the have been transferred that need to - * be applied to the rate limiting calcuations. - * - * @len: amount of bytes transferred - */ -void migration_rate_account(uint64_t len); - /** * migration_rate_get: Get the maximum amount that can be transferred. * diff --git a/migration/migration-stats.c b/migration/migration-stats.c index da2bb69a15..abf2d38b18 100644 --- a/migration/migration-stats.c +++ b/migration/migration-stats.c @@ -62,15 +62,9 @@ void migration_rate_set(uint64_t limit) =20 void migration_rate_reset(QEMUFile *f) { - stat64_set(&mig_stats.rate_limit_used, 0); stat64_set(&mig_stats.rate_limit_start, migration_transferred_bytes(f)= ); } =20 -void migration_rate_account(uint64_t len) -{ - stat64_add(&mig_stats.rate_limit_used, len); -} - uint64_t migration_transferred_bytes(QEMUFile *f) { uint64_t multifd =3D stat64_get(&mig_stats.multifd_bytes); diff --git a/migration/multifd.c b/migration/multifd.c index 5052091ce2..aabf9b6d98 100644 --- a/migration/multifd.c +++ b/migration/multifd.c @@ -431,7 +431,6 @@ static int multifd_send_pages(QEMUFile *f) multifd_send_state->pages =3D p->pages; p->pages =3D pages; transferred =3D ((uint64_t) pages->num) * p->page_size + p->packet_len; - migration_rate_account(transferred); qemu_mutex_unlock(&p->mutex); stat64_add(&mig_stats.transferred, transferred); stat64_add(&mig_stats.multifd_bytes, transferred); diff --git a/migration/qemu-file.c b/migration/qemu-file.c index 9c67b52fe0..acc282654a 100644 --- a/migration/qemu-file.c +++ b/migration/qemu-file.c @@ -292,7 +292,6 @@ void qemu_fflush(QEMUFile *f) qemu_file_set_error_obj(f, -EIO, local_error); } else { uint64_t size =3D iov_size(f->iov, f->iovcnt); - migration_rate_account(size); f->total_transferred +=3D size; } =20 @@ -344,9 +343,6 @@ size_t ram_control_save_page(QEMUFile *f, ram_addr_t bl= ock_offset, if (f->hooks && f->hooks->save_page) { int ret =3D f->hooks->save_page(f, block_offset, offset, size, bytes_sent); - if (ret !=3D RAM_SAVE_CONTROL_NOT_SUPP) { - migration_rate_account(size); - } =20 if (ret !=3D RAM_SAVE_CONTROL_DELAYED && ret !=3D RAM_SAVE_CONTROL_NOT_SUPP) { --=20 2.40.1 From nobody Mon May 13 18:32:02 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1684180795; cv=none; d=zohomail.com; s=zohoarc; b=Pojontx7OMcs66VNi2fC6fum6QEiHsrefW0L9CZz05p3yWrwI/9E3VhCZwmUtGZY4UptE07ccSqbOKN4h3Bw0EGVByCGu6LR66d8NoVL3sVRLcszbCXgWhbhYAjIsQSxP5JjXDvNzSO9omI0UMVxtYhAb9JB+OlK90BFvcMfLS8= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1684180795; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=gobYIRvaKfvCoZK4wzFc/Wuvw0+LhHU4CtsHZJUa6sI=; b=lunPsggObqLQPmlQWauoHrR9WTI4uwI2dKXcB7hrdKkyApbVypinuf6p/Gm6uFYKnPursE2Ep49KwR75oLMG5+Q123YYBtcwUC7H3E2rnp679vCPOLZA2bigXCXYgEcyKv1IjJRHgFhiHDawVPmU/9UnPnt61EaI5CrOW/+K8sE= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1684180795036683.4957425385486; Mon, 15 May 2023 12:59:55 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pyeKg-0000XY-G6; Mon, 15 May 2023 15:58:10 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pyeKe-0000Mh-79 for qemu-devel@nongnu.org; Mon, 15 May 2023 15:58:08 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pyeKc-0005S8-Ht for qemu-devel@nongnu.org; Mon, 15 May 2023 15:58:07 -0400 Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-116-4T1nbLSaP4OeV6_SzcQttA-1; Mon, 15 May 2023 15:58:03 -0400 Received: from smtp.corp.redhat.com (int-mx09.intmail.prod.int.rdu2.redhat.com [10.11.54.9]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id A63423806701; Mon, 15 May 2023 19:58:02 +0000 (UTC) Received: from secure.mitica (unknown [10.39.194.171]) by smtp.corp.redhat.com (Postfix) with ESMTP id BBAFB492B00; Mon, 15 May 2023 19:57:58 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1684180685; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=gobYIRvaKfvCoZK4wzFc/Wuvw0+LhHU4CtsHZJUa6sI=; b=VUe1pzxP93jKJRKEyMZ+DJ1+FF2l4SfTmng1KYf+fWFD/k/uAsHZ7S6NbizDZ5xV1mro5R 72OpzUwt2IUaVf0yGyxoIX3XY+ISn5ibShH/d0zcePP7BTqjGKKoDXINq+U+26eyH6ghKK t+84ME9X4cRQqz+aEa0AR63Dut6K/CU= X-MC-Unique: 4T1nbLSaP4OeV6_SzcQttA-1 From: Juan Quintela To: qemu-devel@nongnu.org Cc: Peter Xu , Greg Kurz , qemu-s390x@nongnu.org, Fam Zheng , Ilya Leoshkevich , Halil Pasic , John Snow , qemu-ppc@nongnu.org, Daniel Henrique Barboza , Harsh Prateek Bora , Christian Borntraeger , Richard Henderson , David Gibson , David Hildenbrand , Stefan Hajnoczi , Eric Farman , qemu-block@nongnu.org, =?UTF-8?q?C=C3=A9dric=20Le=20Goater?= , Eric Blake , Leonardo Bras , Thomas Huth , Juan Quintela , Vladimir Sementsov-Ogievskiy Subject: [PATCH v2 10/16] migration: Don't abuse qemu_file transferred for RDMA Date: Mon, 15 May 2023 21:57:03 +0200 Message-Id: <20230515195709.63843-11-quintela@redhat.com> In-Reply-To: <20230515195709.63843-1-quintela@redhat.com> References: <20230515195709.63843-1-quintela@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.1 on 10.11.54.9 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=quintela@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1684180795345100009 Content-Type: text/plain; charset="utf-8" Just create a variable for it, the same way that multifd does. This way it is safe to use for other thread, etc, etc. Signed-off-by: Juan Quintela Reviewed-by: Leonardo Bras --- migration/migration-stats.h | 4 ++++ migration/migration-stats.c | 5 +++-- migration/rdma.c | 22 ++++++++++++++++++++-- migration/trace-events | 2 +- 4 files changed, 28 insertions(+), 5 deletions(-) diff --git a/migration/migration-stats.h b/migration/migration-stats.h index 9568b5b473..2e3e894307 100644 --- a/migration/migration-stats.h +++ b/migration/migration-stats.h @@ -89,6 +89,10 @@ typedef struct { * Maximum amount of data we can send in a cycle. */ Stat64 rate_limit_max; + /* + * Number of bytes sent through RDMA. + */ + Stat64 rdma_bytes; /* * How long has the setup stage took. */ diff --git a/migration/migration-stats.c b/migration/migration-stats.c index abf2d38b18..4d8e9f93b7 100644 --- a/migration/migration-stats.c +++ b/migration/migration-stats.c @@ -68,8 +68,9 @@ void migration_rate_reset(QEMUFile *f) uint64_t migration_transferred_bytes(QEMUFile *f) { uint64_t multifd =3D stat64_get(&mig_stats.multifd_bytes); + uint64_t rdma =3D stat64_get(&mig_stats.rdma_bytes); uint64_t qemu_file =3D qemu_file_transferred(f); =20 - trace_migration_transferred_bytes(qemu_file, multifd); - return qemu_file + multifd; + trace_migration_transferred_bytes(qemu_file, multifd, rdma); + return qemu_file + multifd + rdma; } diff --git a/migration/rdma.c b/migration/rdma.c index 2e4dcff1c9..074456f9df 100644 --- a/migration/rdma.c +++ b/migration/rdma.c @@ -2122,9 +2122,18 @@ retry: return -EIO; } =20 + /* + * TODO: Here we are sending something, but we are not + * accounting for anything transferred. The following is = wrong: + * + * stat64_add(&mig_stats.rdma_bytes, sge.length); + * + * because we are using some kind of compression. I + * would think that head.len would be the more similar + * thing to a correct value. + */ stat64_add(&mig_stats.zero_pages, sge.length / qemu_target_page_size()); - return 1; } =20 @@ -2232,8 +2241,17 @@ retry: =20 set_bit(chunk, block->transit_bitmap); stat64_add(&mig_stats.normal_pages, sge.length / qemu_target_page_size= ()); + /* + * We are adding to transferred the amount of data written, but no + * overhead at all. I will asume that RDMA is magicaly and don't + * need to transfer (at least) the addresses where it wants to + * write the pages. Here it looks like it should be something + * like: + * sizeof(send_wr) + sge.length + * but this being RDMA, who knows. + */ + stat64_add(&mig_stats.rdma_bytes, sge.length); ram_transferred_add(sge.length); - qemu_file_credit_transfer(f, sge.length); rdma->total_writes++; =20 return 0; diff --git a/migration/trace-events b/migration/trace-events index cdaef7a1ea..54ae5653fd 100644 --- a/migration/trace-events +++ b/migration/trace-events @@ -187,7 +187,7 @@ process_incoming_migration_co_postcopy_end_main(void) "" postcopy_preempt_enabled(bool value) "%d" =20 # migration-stats -migration_transferred_bytes(uint64_t qemu_file, uint64_t multifd) "qemu_fi= le %" PRIu64 " multifd %" PRIu64 +migration_transferred_bytes(uint64_t qemu_file, uint64_t multifd, uint64_t= rdma) "qemu_file %" PRIu64 " multifd %" PRIu64 " RDMA %" PRIu64 =20 # channel.c migration_set_incoming_channel(void *ioc, const char *ioctype) "ioc=3D%p i= octype=3D%s" --=20 2.40.1 From nobody Mon May 13 18:32:02 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1684180792; cv=none; d=zohomail.com; s=zohoarc; b=gqLcR01i2PKzWHjqkZdnRSRfGhlmdWWziY4nhsoBsEHAyqKI79/0CCpARGdEV5S55t1v2psFMq2F+4aOZ6BWwxVsBDdDnXDPmC2sx8cuiWe4bTo5cNHGbjg1TVQq261tJsPkGlWN7TH07Iy48Km0Stc2LvYnXOEfAaUt71QU/Us= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1684180792; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=FGzgKVTv1njdcPyGc4+egV7aCVT2a1VZQnBJw3rYfKI=; b=RuJd+q8r62hXylIGk2zxBbCOK3OfzCyCp7y5slmL8D1wD28YvgNWhooE/tkH9nWujFKnZ6QJhBiSKTYLfHkE26xPDB25WMMuKSyoyo33ld4xPg9x+vcaaqXd/0PuEiwzWqPZlVPuK+ZkS9C1Q0pwOtSb7PrVvAlgyY9dFYYNIU8= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1684180791996607.7539621357839; Mon, 15 May 2023 12:59:51 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pyeKm-00010J-Fq; Mon, 15 May 2023 15:58:16 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pyeKl-0000vH-KP for qemu-devel@nongnu.org; Mon, 15 May 2023 15:58:15 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pyeKj-0005Um-3C for qemu-devel@nongnu.org; Mon, 15 May 2023 15:58:14 -0400 Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-341-ZOAX-zNnMf6F3CoxWvn1Sw-1; Mon, 15 May 2023 15:58:07 -0400 Received: from smtp.corp.redhat.com (int-mx09.intmail.prod.int.rdu2.redhat.com [10.11.54.9]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id B1F733C02184; Mon, 15 May 2023 19:58:06 +0000 (UTC) Received: from secure.mitica (unknown [10.39.194.171]) by smtp.corp.redhat.com (Postfix) with ESMTP id EC382492B00; Mon, 15 May 2023 19:58:02 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1684180692; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=FGzgKVTv1njdcPyGc4+egV7aCVT2a1VZQnBJw3rYfKI=; b=Yo2utjV5L3c08IEHNfUbfDeSHizSce7M+rLJWZo4InxoLk7302dN5ANMdHtCO37VXbC2FZ 5GLf5kA+fGiLvUICpaaTRCZxQ1KmNb/3Cmb5edseoHoFGZG74qm/710ru1eGyeEnzuOfD+ L3h285WEnKVBhZa9z1tz0TbKE/cdjj4= X-MC-Unique: ZOAX-zNnMf6F3CoxWvn1Sw-1 From: Juan Quintela To: qemu-devel@nongnu.org Cc: Peter Xu , Greg Kurz , qemu-s390x@nongnu.org, Fam Zheng , Ilya Leoshkevich , Halil Pasic , John Snow , qemu-ppc@nongnu.org, Daniel Henrique Barboza , Harsh Prateek Bora , Christian Borntraeger , Richard Henderson , David Gibson , David Hildenbrand , Stefan Hajnoczi , Eric Farman , qemu-block@nongnu.org, =?UTF-8?q?C=C3=A9dric=20Le=20Goater?= , Eric Blake , Leonardo Bras , Thomas Huth , Juan Quintela , Vladimir Sementsov-Ogievskiy Subject: [PATCH v2 11/16] migration/RDMA: It is accounting for zero/normal pages in two places Date: Mon, 15 May 2023 21:57:04 +0200 Message-Id: <20230515195709.63843-12-quintela@redhat.com> In-Reply-To: <20230515195709.63843-1-quintela@redhat.com> References: <20230515195709.63843-1-quintela@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.1 on 10.11.54.9 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=quintela@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1684180793341100001 Content-Type: text/plain; charset="utf-8" Remove the one in control_save_page(). Signed-off-by: Juan Quintela Reviewed-by: Leonardo Bras --- migration/ram.c | 7 ------- 1 file changed, 7 deletions(-) diff --git a/migration/ram.c b/migration/ram.c index a706edecc0..67ed49b387 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -1191,13 +1191,6 @@ static bool control_save_page(PageSearchStatus *pss,= RAMBlock *block, if (ret =3D=3D RAM_SAVE_CONTROL_DELAYED) { return true; } - - if (bytes_xmit > 0) { - stat64_add(&mig_stats.normal_pages, 1); - } else if (bytes_xmit =3D=3D 0) { - stat64_add(&mig_stats.zero_pages, 1); - } - return true; } =20 --=20 2.40.1 From nobody Mon May 13 18:32:02 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1684180831; cv=none; d=zohomail.com; s=zohoarc; b=gh4+e5DaPGZCpywpTC1RGerq3O5NYB+GKLS55AUDDiPeR2GcCpiK6Cskv9sfz1K4cL3uU5GVzbTvYqOIDbEEyC4n7IPCPbEiXVLLiNtDh/3x/EkaBV30gTnJjgUxPCOzjq9DbnEzEZABE+K3ytcVuNVp9ACgX4FEej8zHZweQ/g= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1684180831; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=Vhk6rop+cTT+GAlqjeKDryhAk2ZhYNKypyyyjbsJ6QQ=; b=GGaMX3HiIcrb8D4J/45nRM6SiDYrXj6RmvGGdrUn9rtYc0EXW8YHHfzZ3WaPVLqv78okl0pNrS3QRri+pNB6bCaVReRyP0JfNZkDlbdPyPuMjbAVstpT5qHy3XWB/3KBEzlfZR22Lc0t7Qd3TOb0uZ5zDaHBUrUYO08NuFIxBQo= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1684180831454434.44333619297765; Mon, 15 May 2023 13:00:31 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pyeLE-00021C-4r; Mon, 15 May 2023 15:58:44 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pyeKu-0001Qh-TO for qemu-devel@nongnu.org; Mon, 15 May 2023 15:58:26 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pyeKs-0005Ww-LO for qemu-devel@nongnu.org; Mon, 15 May 2023 15:58:24 -0400 Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-10-WxE2cb3uN0y0jLb3LYYmRQ-1; Mon, 15 May 2023 15:58:11 -0400 Received: from smtp.corp.redhat.com (int-mx09.intmail.prod.int.rdu2.redhat.com [10.11.54.9]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id C3559101A555; Mon, 15 May 2023 19:58:10 +0000 (UTC) Received: from secure.mitica (unknown [10.39.194.171]) by smtp.corp.redhat.com (Postfix) with ESMTP id 03F9D492B00; Mon, 15 May 2023 19:58:06 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1684180700; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Vhk6rop+cTT+GAlqjeKDryhAk2ZhYNKypyyyjbsJ6QQ=; b=bhd0mzL4uuzUz4GzYETUScNCsPfEfgY1PrQJcSbQq9NJdEMWgZbYCCzAmBO5ut1bUzaANk RsJcv2Zy0fjUnyLxt2pLD1J/bQtoxlSJNYQxIfeLxTZQ91BazT4uApkB4Bal8GTlP0o52L TeaAYnXY0oRIE6c39ss4VHKqMy6peQQ= X-MC-Unique: WxE2cb3uN0y0jLb3LYYmRQ-1 From: Juan Quintela To: qemu-devel@nongnu.org Cc: Peter Xu , Greg Kurz , qemu-s390x@nongnu.org, Fam Zheng , Ilya Leoshkevich , Halil Pasic , John Snow , qemu-ppc@nongnu.org, Daniel Henrique Barboza , Harsh Prateek Bora , Christian Borntraeger , Richard Henderson , David Gibson , David Hildenbrand , Stefan Hajnoczi , Eric Farman , qemu-block@nongnu.org, =?UTF-8?q?C=C3=A9dric=20Le=20Goater?= , Eric Blake , Leonardo Bras , Thomas Huth , Juan Quintela , Vladimir Sementsov-Ogievskiy Subject: [PATCH v2 12/16] migration/rdma: Remove QEMUFile parameter when not used Date: Mon, 15 May 2023 21:57:05 +0200 Message-Id: <20230515195709.63843-13-quintela@redhat.com> In-Reply-To: <20230515195709.63843-1-quintela@redhat.com> References: <20230515195709.63843-1-quintela@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.1 on 10.11.54.9 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=quintela@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1684180833767100003 Content-Type: text/plain; charset="utf-8" Signed-off-by: Juan Quintela Reviewed-by: Leonardo Bras --- migration/rdma.c | 23 +++++++++++------------ 1 file changed, 11 insertions(+), 12 deletions(-) diff --git a/migration/rdma.c b/migration/rdma.c index 074456f9df..416dec00a2 100644 --- a/migration/rdma.c +++ b/migration/rdma.c @@ -2027,7 +2027,7 @@ static int qemu_rdma_exchange_recv(RDMAContext *rdma,= RDMAControlHeader *head, * If we're using dynamic registration on the dest-side, we have to * send a registration command first. */ -static int qemu_rdma_write_one(QEMUFile *f, RDMAContext *rdma, +static int qemu_rdma_write_one(RDMAContext *rdma, int current_index, uint64_t current_addr, uint64_t length) { @@ -2263,7 +2263,7 @@ retry: * We support sending out multiple chunks at the same time. * Not all of them need to get signaled in the completion queue. */ -static int qemu_rdma_write_flush(QEMUFile *f, RDMAContext *rdma) +static int qemu_rdma_write_flush(RDMAContext *rdma) { int ret; =20 @@ -2271,7 +2271,7 @@ static int qemu_rdma_write_flush(QEMUFile *f, RDMACon= text *rdma) return 0; } =20 - ret =3D qemu_rdma_write_one(f, rdma, + ret =3D qemu_rdma_write_one(rdma, rdma->current_index, rdma->current_addr, rdma->current_length); =20 if (ret < 0) { @@ -2344,7 +2344,7 @@ static inline int qemu_rdma_buffer_mergable(RDMAConte= xt *rdma, * and only require that a batch gets acknowledged in the completion * queue instead of each individual chunk. */ -static int qemu_rdma_write(QEMUFile *f, RDMAContext *rdma, +static int qemu_rdma_write(RDMAContext *rdma, uint64_t block_offset, uint64_t offset, uint64_t len) { @@ -2355,7 +2355,7 @@ static int qemu_rdma_write(QEMUFile *f, RDMAContext *= rdma, =20 /* If we cannot merge it, we flush the current buffer first. */ if (!qemu_rdma_buffer_mergable(rdma, current_addr, len)) { - ret =3D qemu_rdma_write_flush(f, rdma); + ret =3D qemu_rdma_write_flush(rdma); if (ret) { return ret; } @@ -2377,7 +2377,7 @@ static int qemu_rdma_write(QEMUFile *f, RDMAContext *= rdma, =20 /* flush it if buffer is too large */ if (rdma->current_length >=3D RDMA_MERGE_MAX) { - return qemu_rdma_write_flush(f, rdma); + return qemu_rdma_write_flush(rdma); } =20 return 0; @@ -2798,7 +2798,6 @@ static ssize_t qio_channel_rdma_writev(QIOChannel *io= c, Error **errp) { QIOChannelRDMA *rioc =3D QIO_CHANNEL_RDMA(ioc); - QEMUFile *f =3D rioc->file; RDMAContext *rdma; int ret; ssize_t done =3D 0; @@ -2819,7 +2818,7 @@ static ssize_t qio_channel_rdma_writev(QIOChannel *io= c, * Push out any writes that * we're queued up for VM's ram. */ - ret =3D qemu_rdma_write_flush(f, rdma); + ret =3D qemu_rdma_write_flush(rdma); if (ret < 0) { rdma->error_state =3D ret; error_setg(errp, "qemu_rdma_write_flush returned %d", ret); @@ -2958,11 +2957,11 @@ static ssize_t qio_channel_rdma_readv(QIOChannel *i= oc, /* * Block until all the outstanding chunks have been delivered by the hardw= are. */ -static int qemu_rdma_drain_cq(QEMUFile *f, RDMAContext *rdma) +static int qemu_rdma_drain_cq(RDMAContext *rdma) { int ret; =20 - if (qemu_rdma_write_flush(f, rdma) < 0) { + if (qemu_rdma_write_flush(rdma) < 0) { return -EIO; } =20 @@ -3272,7 +3271,7 @@ static size_t qemu_rdma_save_page(QEMUFile *f, * is full, or the page doesn't belong to the current chunk, * an actual RDMA write will occur and a new chunk will be formed. */ - ret =3D qemu_rdma_write(f, rdma, block_offset, offset, size); + ret =3D qemu_rdma_write(rdma, block_offset, offset, size); if (ret < 0) { error_report("rdma migration: write error! %d", ret); goto err; @@ -3927,7 +3926,7 @@ static int qemu_rdma_registration_stop(QEMUFile *f, CHECK_ERROR_STATE(); =20 qemu_fflush(f); - ret =3D qemu_rdma_drain_cq(f, rdma); + ret =3D qemu_rdma_drain_cq(rdma); =20 if (ret < 0) { goto err; --=20 2.40.1 From nobody Mon May 13 18:32:02 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1684180765; cv=none; d=zohomail.com; s=zohoarc; b=LkQ7JRRpegF+BfbhfQAFCLcB1isUYKGP8wp6W2O8n9aKaEc/C1PpEfaGIVEHeUpR6C5/0CALSQ7vOhOsWuOkt973lFOevMXsAeQBsR7+JGmNA7A4UNl394lm3ND3FBAfUPkxaRKsseGkAJ0PlP6pUlCVgoRkQ98d+rJCgNDuclQ= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1684180765; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=eGdqKidErSNvzt22AsUy8QBiTCA6ghBZKDOw4L5rBtQ=; b=hsjKlBUe65KcpFZQxKVQs7EVqH3c9XMBwmN/h5zqzSHa8XJZJeBjAS86Lm046LfM/qQ3u1fhl9mfIR1C+4lcJFRRklz9v0IKyCzEiP0nAJSesBlbE+9iJ2vC5CrLTFxC/nueLiHthSlSBFyn4t1ef+6qlfqjXFq8ZdDXnFvGXFI= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1684180765213831.5373120089049; Mon, 15 May 2023 12:59:25 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pyeKw-0001Qi-Gm; Mon, 15 May 2023 15:58:26 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pyeKs-0001GS-El for qemu-devel@nongnu.org; Mon, 15 May 2023 15:58:22 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pyeKq-0005Wm-Oz for qemu-devel@nongnu.org; Mon, 15 May 2023 15:58:22 -0400 Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-636-LzIJM2p_PJS0NYKvTjTZFA-1; Mon, 15 May 2023 15:58:15 -0400 Received: from smtp.corp.redhat.com (int-mx09.intmail.prod.int.rdu2.redhat.com [10.11.54.9]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id D2E8D185A7A2; Mon, 15 May 2023 19:58:14 +0000 (UTC) Received: from secure.mitica (unknown [10.39.194.171]) by smtp.corp.redhat.com (Postfix) with ESMTP id 1429E492B00; Mon, 15 May 2023 19:58:10 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1684180700; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=eGdqKidErSNvzt22AsUy8QBiTCA6ghBZKDOw4L5rBtQ=; b=QOQzsKi3zddTdRRs6LMlKyyR3K+1RwIm64xPNFhEKxht1enlNmuePMcH8NJHSvaAMKafUT kEIyXXcuI6vHTahS2XkWMVENx3N8iuQacyoXNLJNJT7talOfdP9iYyIKOoT5BvoxVb+1Bb q1z3CJH2Ob6wQova8qqbT5X31gCBimM= X-MC-Unique: LzIJM2p_PJS0NYKvTjTZFA-1 From: Juan Quintela To: qemu-devel@nongnu.org Cc: Peter Xu , Greg Kurz , qemu-s390x@nongnu.org, Fam Zheng , Ilya Leoshkevich , Halil Pasic , John Snow , qemu-ppc@nongnu.org, Daniel Henrique Barboza , Harsh Prateek Bora , Christian Borntraeger , Richard Henderson , David Gibson , David Hildenbrand , Stefan Hajnoczi , Eric Farman , qemu-block@nongnu.org, =?UTF-8?q?C=C3=A9dric=20Le=20Goater?= , Eric Blake , Leonardo Bras , Thomas Huth , Juan Quintela , Vladimir Sementsov-Ogievskiy Subject: [PATCH v2 13/16] migration/rdma: Don't use imaginary transfers Date: Mon, 15 May 2023 21:57:06 +0200 Message-Id: <20230515195709.63843-14-quintela@redhat.com> In-Reply-To: <20230515195709.63843-1-quintela@redhat.com> References: <20230515195709.63843-1-quintela@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.1 on 10.11.54.9 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=quintela@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1684180767168100001 Content-Type: text/plain; charset="utf-8" RDMA protocol is completely asynchronous, so in qemu_rdma_save_page() they "invent" that a byte has been transferred. And then they call qemu_file_credit_transfer() and ram_transferred_add() with that byte. Just remove that calls as nothing has been sent. Signed-off-by: Juan Quintela Reviewed-by: Leonardo Bras --- migration/qemu-file.c | 5 +---- migration/ram.c | 1 - 2 files changed, 1 insertion(+), 5 deletions(-) diff --git a/migration/qemu-file.c b/migration/qemu-file.c index acc282654a..23a21e2331 100644 --- a/migration/qemu-file.c +++ b/migration/qemu-file.c @@ -346,13 +346,10 @@ size_t ram_control_save_page(QEMUFile *f, ram_addr_t = block_offset, =20 if (ret !=3D RAM_SAVE_CONTROL_DELAYED && ret !=3D RAM_SAVE_CONTROL_NOT_SUPP) { - if (bytes_sent && *bytes_sent > 0) { - qemu_file_credit_transfer(f, *bytes_sent); - } else if (ret < 0) { + if (ret < 0) { qemu_file_set_error(f, ret); } } - return ret; } =20 diff --git a/migration/ram.c b/migration/ram.c index 67ed49b387..2d3927a15f 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -1184,7 +1184,6 @@ static bool control_save_page(PageSearchStatus *pss, = RAMBlock *block, } =20 if (bytes_xmit) { - ram_transferred_add(bytes_xmit); *pages =3D 1; } =20 --=20 2.40.1 From nobody Mon May 13 18:32:02 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1684180788; cv=none; d=zohomail.com; s=zohoarc; b=S6ngbrdZUx0c4sfKyEs1k3SkfikTO543mzSqihxaUgDfWcC3u6EgsOWSQbGOcU9ZENHNBG6qq3u99kwHQ6whztKEHtWvG4di9bM2D/ueDVH3AZ6RWaqYF94OVeU5vuUt87cywuxC44E/ZFVWZI45Q+brwpyW8GaMrl8ILW5hUZY= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1684180788; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=Wpd/wbmeDwX8T87oWYgGSnRA9O+pyRhF35RG1ynYdxk=; b=NVyR0iZwXo5GqjeKt/ZcCSmWn2KwVf9nZ/2PIHKkjxE8pJ28Mpil2aTYxgdbKR4Yyl5lln2avRZfo2XzeVYs7igXNmu3Ehg8qFUWwJDHYwE4PZBSCxafLbiAnnxTzLtfIrJjdVJ1EyRCPl1d5pfR9ZFaXZg3Ta6xbTPZf/CqnAI= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1684180788322935.3127823512777; Mon, 15 May 2023 12:59:48 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pyeLE-00028T-Mu; Mon, 15 May 2023 15:58:44 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pyeKw-0001Ql-6c for qemu-devel@nongnu.org; Mon, 15 May 2023 15:58:26 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pyeKu-0005YH-Nr for qemu-devel@nongnu.org; Mon, 15 May 2023 15:58:25 -0400 Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-645-e0FTZLZdMTyp9czNajJEMw-1; Mon, 15 May 2023 15:58:19 -0400 Received: from smtp.corp.redhat.com (int-mx09.intmail.prod.int.rdu2.redhat.com [10.11.54.9]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id E479B3C0CF0A; Mon, 15 May 2023 19:58:18 +0000 (UTC) Received: from secure.mitica (unknown [10.39.194.171]) by smtp.corp.redhat.com (Postfix) with ESMTP id 25087492B00; Mon, 15 May 2023 19:58:15 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1684180704; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Wpd/wbmeDwX8T87oWYgGSnRA9O+pyRhF35RG1ynYdxk=; b=NTobFA7Pe0qu16vqRXd0lqZ30g/qSJV1yxwsRPqaXsM7zT5fMG4AIfpAZCCDPzbzcfRRq4 uTD3Mgqh2axmmigt2U9nSU5BS60QN95/9DQ2IV9J1DvrrKKUd9QwSZo4OpIyszDg4ymawC C6QODyV+kqj2AH1zkCoNYPEECpWLpgs= X-MC-Unique: e0FTZLZdMTyp9czNajJEMw-1 From: Juan Quintela To: qemu-devel@nongnu.org Cc: Peter Xu , Greg Kurz , qemu-s390x@nongnu.org, Fam Zheng , Ilya Leoshkevich , Halil Pasic , John Snow , qemu-ppc@nongnu.org, Daniel Henrique Barboza , Harsh Prateek Bora , Christian Borntraeger , Richard Henderson , David Gibson , David Hildenbrand , Stefan Hajnoczi , Eric Farman , qemu-block@nongnu.org, =?UTF-8?q?C=C3=A9dric=20Le=20Goater?= , Eric Blake , Leonardo Bras , Thomas Huth , Juan Quintela , Vladimir Sementsov-Ogievskiy Subject: [PATCH v2 14/16] migration: Remove unused qemu_file_credit_transfer() Date: Mon, 15 May 2023 21:57:07 +0200 Message-Id: <20230515195709.63843-15-quintela@redhat.com> In-Reply-To: <20230515195709.63843-1-quintela@redhat.com> References: <20230515195709.63843-1-quintela@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.1 on 10.11.54.9 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=quintela@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1684180790112100003 Content-Type: text/plain; charset="utf-8" After this change, nothing abuses QEMUFile to account for data transferrefd during migration. Signed-off-by: Juan Quintela Reviewed-by: Leonardo Bras --- migration/qemu-file.h | 8 -------- migration/qemu-file.c | 5 ----- 2 files changed, 13 deletions(-) diff --git a/migration/qemu-file.h b/migration/qemu-file.h index e649718492..37f42315c7 100644 --- a/migration/qemu-file.h +++ b/migration/qemu-file.h @@ -122,14 +122,6 @@ bool qemu_file_buffer_empty(QEMUFile *file); */ int coroutine_mixed_fn qemu_peek_byte(QEMUFile *f, int offset); void qemu_file_skip(QEMUFile *f, int size); -/* - * qemu_file_credit_transfer: - * - * Report on a number of bytes that have been transferred - * out of band from the main file object I/O methods. This - * accounting information tracks the total migration traffic. - */ -void qemu_file_credit_transfer(QEMUFile *f, size_t size); int qemu_file_get_error_obj(QEMUFile *f, Error **errp); int qemu_file_get_error_obj_any(QEMUFile *f1, QEMUFile *f2, Error **errp); void qemu_file_set_error_obj(QEMUFile *f, int ret, Error *err); diff --git a/migration/qemu-file.c b/migration/qemu-file.c index 23a21e2331..72e130631d 100644 --- a/migration/qemu-file.c +++ b/migration/qemu-file.c @@ -411,11 +411,6 @@ static ssize_t coroutine_mixed_fn qemu_fill_buffer(QEM= UFile *f) return len; } =20 -void qemu_file_credit_transfer(QEMUFile *f, size_t size) -{ - f->total_transferred +=3D size; -} - /** Closes the file * * Returns negative error value if any error happened on previous operatio= ns or --=20 2.40.1 From nobody Mon May 13 18:32:02 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1684180835; cv=none; d=zohomail.com; s=zohoarc; b=bM+oS10lIY9gBb9YD+bgfoIa4ud2+vhUCccwHNC+hYSST4PnH13G39QQFiZtIftleK+4HPAcstJuPBilTpVVFsEvG3gc8s6yok7UvociM5BB2ZPEZtEEYmI7FB8LX/i5grq5H4MnmVjceVQ5KxXZnw4A7oe9GNJy5VE2DlYPLCU= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1684180835; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=U0xWEIKpsTavsJwcCEf3C5p/wD4gGvVFo6Epfs0gtWE=; b=E739gVXEEkMBOBir4mH3HtqXXVKIYelikmmHno8GKJiw/xV7maQJtOSOUCleABuz/sWNYiP4FozEWzmZm63DxMqi7v1uuVBDOYL1U+odALUETI21gd33Hg2srMxzV8bHXisjI4czkRHAwxVOX97DOlQxfWMR1ai5fkle/haoKH4= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1684180835599287.5324739386748; Mon, 15 May 2023 13:00:35 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pyeLE-00025B-DJ; Mon, 15 May 2023 15:58:44 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pyeKz-0001ci-Jv for qemu-devel@nongnu.org; Mon, 15 May 2023 15:58:30 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pyeKx-0005ZV-U6 for qemu-devel@nongnu.org; Mon, 15 May 2023 15:58:29 -0400 Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-189-5GcHYF1xN-mPvOKZ2K3nSg-1; Mon, 15 May 2023 15:58:24 -0400 Received: from smtp.corp.redhat.com (int-mx09.intmail.prod.int.rdu2.redhat.com [10.11.54.9]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 142A5868900; Mon, 15 May 2023 19:58:23 +0000 (UTC) Received: from secure.mitica (unknown [10.39.194.171]) by smtp.corp.redhat.com (Postfix) with ESMTP id 3AE2D492B02; Mon, 15 May 2023 19:58:19 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1684180707; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=U0xWEIKpsTavsJwcCEf3C5p/wD4gGvVFo6Epfs0gtWE=; b=iUPgC9lSQQah4tlzVMr6wns+LwTCiNM7KSxS4Y3O6Z7BhySQTcTFEIURjuEwgb+/8yaL2Y teBp8lH/brZhPH6SIqwkVJluDf7GeSy/zqNG8c7G39mlblprpH1p9YVaF2CAyyVMQ+xi+w RT2E9joemnS6kz4BkByN2s4DOGF/wTc= X-MC-Unique: 5GcHYF1xN-mPvOKZ2K3nSg-1 From: Juan Quintela To: qemu-devel@nongnu.org Cc: Peter Xu , Greg Kurz , qemu-s390x@nongnu.org, Fam Zheng , Ilya Leoshkevich , Halil Pasic , John Snow , qemu-ppc@nongnu.org, Daniel Henrique Barboza , Harsh Prateek Bora , Christian Borntraeger , Richard Henderson , David Gibson , David Hildenbrand , Stefan Hajnoczi , Eric Farman , qemu-block@nongnu.org, =?UTF-8?q?C=C3=A9dric=20Le=20Goater?= , Eric Blake , Leonardo Bras , Thomas Huth , Juan Quintela , Vladimir Sementsov-Ogievskiy Subject: [PATCH v2 15/16] migration/rdma: Simplify the function that saves a page Date: Mon, 15 May 2023 21:57:08 +0200 Message-Id: <20230515195709.63843-16-quintela@redhat.com> In-Reply-To: <20230515195709.63843-1-quintela@redhat.com> References: <20230515195709.63843-1-quintela@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.1 on 10.11.54.9 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=quintela@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1684180836459100001 Content-Type: text/plain; charset="utf-8" When we sent a page through QEMUFile hooks (RDMA) there are three posiblities: - We are not using RDMA. return RAM_SAVE_CONTROL_DELAYED and control_save_page() returns false to let anything else to proceed. - There is one error but we are using RDMA. Then we return a negative value, control_save_page() needs to return true. - Everything goes well and RDMA start the sent of the page asynchronously. It returns RAM_SAVE_CONTROL_DELAYED and we need to return 1 for ram_save_page_legacy. Clear? I know, I know, the interfaz is as bad as it gets. I think that now it is a bit clearer, but this needs to be done some other way. Signed-off-by: Juan Quintela Reviewed-by: Leonardo Bras --- migration/qemu-file.h | 14 ++++++-------- migration/qemu-file.c | 12 ++++++------ migration/ram.c | 10 +++------- migration/rdma.c | 19 +++---------------- 4 files changed, 18 insertions(+), 37 deletions(-) diff --git a/migration/qemu-file.h b/migration/qemu-file.h index 37f42315c7..ed77996201 100644 --- a/migration/qemu-file.h +++ b/migration/qemu-file.h @@ -49,11 +49,10 @@ typedef int (QEMURamHookFunc)(QEMUFile *f, uint64_t fla= gs, void *data); * This function allows override of where the RAM page * is saved (such as RDMA, for example.) */ -typedef size_t (QEMURamSaveFunc)(QEMUFile *f, - ram_addr_t block_offset, - ram_addr_t offset, - size_t size, - uint64_t *bytes_sent); +typedef int (QEMURamSaveFunc)(QEMUFile *f, + ram_addr_t block_offset, + ram_addr_t offset, + size_t size); =20 typedef struct QEMUFileHooks { QEMURamHookFunc *before_ram_iterate; @@ -146,9 +145,8 @@ void ram_control_load_hook(QEMUFile *f, uint64_t flags,= void *data); #define RAM_SAVE_CONTROL_NOT_SUPP -1000 #define RAM_SAVE_CONTROL_DELAYED -2000 =20 -size_t ram_control_save_page(QEMUFile *f, ram_addr_t block_offset, - ram_addr_t offset, size_t size, - uint64_t *bytes_sent); +int ram_control_save_page(QEMUFile *f, ram_addr_t block_offset, + ram_addr_t offset, size_t size); QIOChannel *qemu_file_get_ioc(QEMUFile *file); =20 #endif diff --git a/migration/qemu-file.c b/migration/qemu-file.c index 72e130631d..32ef5e9651 100644 --- a/migration/qemu-file.c +++ b/migration/qemu-file.c @@ -336,14 +336,14 @@ void ram_control_load_hook(QEMUFile *f, uint64_t flag= s, void *data) } } =20 -size_t ram_control_save_page(QEMUFile *f, ram_addr_t block_offset, - ram_addr_t offset, size_t size, - uint64_t *bytes_sent) +int ram_control_save_page(QEMUFile *f, ram_addr_t block_offset, + ram_addr_t offset, size_t size) { if (f->hooks && f->hooks->save_page) { - int ret =3D f->hooks->save_page(f, block_offset, - offset, size, bytes_sent); - + int ret =3D f->hooks->save_page(f, block_offset, offset, size); + /* + * RAM_SAVE_CONTROL_* are negative values + */ if (ret !=3D RAM_SAVE_CONTROL_DELAYED && ret !=3D RAM_SAVE_CONTROL_NOT_SUPP) { if (ret < 0) { diff --git a/migration/ram.c b/migration/ram.c index 2d3927a15f..f9fcbb3bb8 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -1173,23 +1173,19 @@ static int save_zero_page(PageSearchStatus *pss, QE= MUFile *f, RAMBlock *block, static bool control_save_page(PageSearchStatus *pss, RAMBlock *block, ram_addr_t offset, int *pages) { - uint64_t bytes_xmit =3D 0; int ret; =20 - *pages =3D -1; ret =3D ram_control_save_page(pss->pss_channel, block->offset, offset, - TARGET_PAGE_SIZE, &bytes_xmit); + TARGET_PAGE_SIZE); if (ret =3D=3D RAM_SAVE_CONTROL_NOT_SUPP) { return false; } =20 - if (bytes_xmit) { - *pages =3D 1; - } - if (ret =3D=3D RAM_SAVE_CONTROL_DELAYED) { + *pages =3D 1; return true; } + *pages =3D ret; return true; } =20 diff --git a/migration/rdma.c b/migration/rdma.c index 416dec00a2..12d3c23fdc 100644 --- a/migration/rdma.c +++ b/migration/rdma.c @@ -3239,13 +3239,12 @@ qio_channel_rdma_shutdown(QIOChannel *ioc, * * @size : Number of bytes to transfer * - * @bytes_sent : User-specificed pointer to indicate how many bytes were + * @pages_sent : User-specificed pointer to indicate how many pages were * sent. Usually, this will not be more than a few bytes = of * the protocol because most transfers are sent asynchron= ously. */ -static size_t qemu_rdma_save_page(QEMUFile *f, - ram_addr_t block_offset, ram_addr_t offs= et, - size_t size, uint64_t *bytes_sent) +static int qemu_rdma_save_page(QEMUFile *f, ram_addr_t block_offset, + ram_addr_t offset, size_t size) { QIOChannelRDMA *rioc =3D QIO_CHANNEL_RDMA(qemu_file_get_ioc(f)); RDMAContext *rdma; @@ -3277,18 +3276,6 @@ static size_t qemu_rdma_save_page(QEMUFile *f, goto err; } =20 - /* - * We always return 1 bytes because the RDMA - * protocol is completely asynchronous. We do not yet know - * whether an identified chunk is zero or not because we're - * waiting for other pages to potentially be merged with - * the current chunk. So, we have to call qemu_update_position() - * later on when the actual write occurs. - */ - if (bytes_sent) { - *bytes_sent =3D 1; - } - /* * Drain the Completion Queue if possible, but do not block, * just poll. --=20 2.40.1 From nobody Mon May 13 18:32:02 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1684180845; cv=none; d=zohomail.com; s=zohoarc; b=dl5VwHoLeAFUkieUlnmeqyaM3dFZMWhT5FXi/w9g5WSLXwWcYlRozhUju0hVtRm4Uh65+ESMScG5a96MiFyNG8ml9h6Fn64Wc4IhTEMH2RftHvF+YDV1qhPbTIK+Q11hwsgdP5gECQOR2z/VSBToxNtyZXz8cdVvvEUOa1qhle4= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1684180845; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=rv0OW6U2taWTBZp3ks0AuejA0Q/nSmNZ/Pvn3eecT5c=; b=UrGgtlTPSYUjeJHHOdWs9hhj+M9YMAWBuWnLI9HLjsTD2Xt3081m9+rpzay2RY/REa+NioTIvbzRMBa7jnwpqWkVy5YiHIgA5fBYoOziprNWOlDelKT/weSQWaN5X4ChJuwvu8n9PAGyUNZwZAzNKD8GV52Q0U9tQNiJUlnsNO0= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1684180845162148.38116767963982; Mon, 15 May 2023 13:00:45 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pyeLH-0002bs-LB; Mon, 15 May 2023 15:58:47 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pyeL4-0001oM-4h for qemu-devel@nongnu.org; Mon, 15 May 2023 15:58:34 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pyeL2-0005aY-06 for qemu-devel@nongnu.org; Mon, 15 May 2023 15:58:33 -0400 Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-501-Tu5HA8yyPMy551Q7mUAx1g-1; Mon, 15 May 2023 15:58:28 -0400 Received: from smtp.corp.redhat.com (int-mx09.intmail.prod.int.rdu2.redhat.com [10.11.54.9]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 2B4FA2999B29; Mon, 15 May 2023 19:58:27 +0000 (UTC) Received: from secure.mitica (unknown [10.39.194.171]) by smtp.corp.redhat.com (Postfix) with ESMTP id 5B80E492B00; Mon, 15 May 2023 19:58:23 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1684180711; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=rv0OW6U2taWTBZp3ks0AuejA0Q/nSmNZ/Pvn3eecT5c=; b=S9JMN6LxiW/EuhRSwfzUhrd6YpdpaFBQ526U8Hm4iShuYLGXhnxIolOXYK/1Sz2Ef5yVvM 5QQWq/3J7AS9H0FBzH0scNL7OUB8KV4C58YxoDWpcbQLQK0gyE5h3ZugEWTxAgc2egPl1B dSNHukkEf73g0IAsRmPu39lVT6zRidE= X-MC-Unique: Tu5HA8yyPMy551Q7mUAx1g-1 From: Juan Quintela To: qemu-devel@nongnu.org Cc: Peter Xu , Greg Kurz , qemu-s390x@nongnu.org, Fam Zheng , Ilya Leoshkevich , Halil Pasic , John Snow , qemu-ppc@nongnu.org, Daniel Henrique Barboza , Harsh Prateek Bora , Christian Borntraeger , Richard Henderson , David Gibson , David Hildenbrand , Stefan Hajnoczi , Eric Farman , qemu-block@nongnu.org, =?UTF-8?q?C=C3=A9dric=20Le=20Goater?= , Eric Blake , Leonardo Bras , Thomas Huth , Juan Quintela , Vladimir Sementsov-Ogievskiy Subject: [PATCH v2 16/16] migration/multifd: Compute transferred bytes correctly Date: Mon, 15 May 2023 21:57:09 +0200 Message-Id: <20230515195709.63843-17-quintela@redhat.com> In-Reply-To: <20230515195709.63843-1-quintela@redhat.com> References: <20230515195709.63843-1-quintela@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.1 on 10.11.54.9 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=quintela@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1684180845618100001 Content-Type: text/plain; charset="utf-8" In the past, we had to put the in the main thread all the operations related with sizes due to qemu_file not beeing thread safe. As now all counters are atomic, we can update the counters just after the do the write. As an aditional bonus, we are able to use the right value for the compression methods. Right now we were assuming that there were no compression at all. Signed-off-by: Juan Quintela Reviewed-by: Leonardo Bras --- migration/multifd.c | 13 ++++++++----- 1 file changed, 8 insertions(+), 5 deletions(-) diff --git a/migration/multifd.c b/migration/multifd.c index aabf9b6d98..0bf5958a9c 100644 --- a/migration/multifd.c +++ b/migration/multifd.c @@ -175,6 +175,7 @@ void multifd_register_ops(int method, MultiFDMethods *o= ps) static int multifd_send_initial_packet(MultiFDSendParams *p, Error **errp) { MultiFDInit_t msg =3D {}; + size_t size =3D sizeof(msg); int ret; =20 msg.magic =3D cpu_to_be32(MULTIFD_MAGIC); @@ -182,10 +183,12 @@ static int multifd_send_initial_packet(MultiFDSendPar= ams *p, Error **errp) msg.id =3D p->id; memcpy(msg.uuid, &qemu_uuid.data, sizeof(msg.uuid)); =20 - ret =3D qio_channel_write_all(p->c, (char *)&msg, sizeof(msg), errp); + ret =3D qio_channel_write_all(p->c, (char *)&msg, size, errp); if (ret !=3D 0) { return -1; } + stat64_add(&mig_stats.multifd_bytes, size); + stat64_add(&mig_stats.transferred, size); return 0; } =20 @@ -395,7 +398,6 @@ static int multifd_send_pages(QEMUFile *f) static int next_channel; MultiFDSendParams *p =3D NULL; /* make happy gcc */ MultiFDPages_t *pages =3D multifd_send_state->pages; - uint64_t transferred; =20 if (qatomic_read(&multifd_send_state->exiting)) { return -1; @@ -430,10 +432,7 @@ static int multifd_send_pages(QEMUFile *f) p->packet_num =3D multifd_send_state->packet_num++; multifd_send_state->pages =3D p->pages; p->pages =3D pages; - transferred =3D ((uint64_t) pages->num) * p->page_size + p->packet_len; qemu_mutex_unlock(&p->mutex); - stat64_add(&mig_stats.transferred, transferred); - stat64_add(&mig_stats.multifd_bytes, transferred); qemu_sem_post(&p->sem); =20 return 1; @@ -715,6 +714,8 @@ static void *multifd_send_thread(void *opaque) if (ret !=3D 0) { break; } + stat64_add(&mig_stats.multifd_bytes, p->packet_len); + stat64_add(&mig_stats.transferred, p->packet_len); } else { /* Send header using the same writev call */ p->iov[0].iov_len =3D p->packet_len; @@ -727,6 +728,8 @@ static void *multifd_send_thread(void *opaque) break; } =20 + stat64_add(&mig_stats.multifd_bytes, p->next_packet_size); + stat64_add(&mig_stats.transferred, p->next_packet_size); qemu_mutex_lock(&p->mutex); p->pending_job--; qemu_mutex_unlock(&p->mutex); --=20 2.40.1