From nobody Sun May 5 09:42:17 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1652770111361618.6642007023611; Mon, 16 May 2022 23:48:31 -0700 (PDT) Received: from localhost ([::1]:59710 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nqr0Q-0003x9-6V for importer@patchew.org; Tue, 17 May 2022 02:48:30 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:58684) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nqqoK-0004l4-Ae for qemu-devel@nongnu.org; Tue, 17 May 2022 02:36:03 -0400 Received: from prt-mail.chinatelecom.cn ([42.123.76.219]:46484 helo=chinatelecom.cn) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nqqoG-0007fF-EX for qemu-devel@nongnu.org; Tue, 17 May 2022 02:36:00 -0400 Received: from clientip-36.111.64.84 (unknown [172.18.0.48]) by chinatelecom.cn (HERMES) with SMTP id 165162800C4; Tue, 17 May 2022 14:35:28 +0800 (CST) Received: from ([172.18.0.48]) by app0024 with ESMTP id 0f37b2a2d829483b9507aa9146dd6eda for qemu-devel@nongnu.org; Tue, 17 May 2022 14:35:33 CST HMM_SOURCE_IP: 172.18.0.48:35216.1855245433 HMM_ATTACHE_NUM: 0000 HMM_SOURCE_TYPE: SMTP X-189-SAVE-TO-SEND: +huangy81@chinatelecom.cn X-Transaction-ID: 0f37b2a2d829483b9507aa9146dd6eda X-Real-From: huangy81@chinatelecom.cn X-Receive-IP: 172.18.0.48 X-MEDUSA-Status: 0 From: huangy81@chinatelecom.cn To: qemu-devel Cc: "Dr. David Alan Gilbert" , Juan Quintela , Eric Blake , Markus Armbruster , Thomas Huth , Laurent Vivier , Paolo Bonzini , Hyman Huang Subject: [RFC 1/6] qapi/migration: Introduce vcpu-dirtylimit-period parameters Date: Tue, 17 May 2022 14:35:01 +0800 Message-Id: <23b507e7d9d230f1ea46bfe907acc09315505174.1652762652.git.huangy81@chinatelecom.cn> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: References: In-Reply-To: References: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=42.123.76.219; envelope-from=huangy81@chinatelecom.cn; helo=chinatelecom.cn X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZM-MESSAGEID: 1652770112235100001 From: Hyman Huang(=E9=BB=84=E5=8B=87) Introduce "vcpu-dirtylimit-period" migration parameters, which is used to makes dirtyrate calculation period configurable. To implement that, refactor vcpu_dirty_rate_stat_collect so that period can be configured instead of hardcode. Meanwhile, introduce migrate_dirtylimit function to help check if dirtylimit enabled during live migration, set it false by default. Signed-off-by: Hyman Huang(=E9=BB=84=E5=8B=87) --- migration/migration.c | 20 ++++++++++++++++++++ migration/migration.h | 1 + monitor/hmp-cmds.c | 8 ++++++++ qapi/migration.json | 19 +++++++++++++++---- softmmu/dirtylimit.c | 11 ++++++++++- 5 files changed, 54 insertions(+), 5 deletions(-) diff --git a/migration/migration.c b/migration/migration.c index 695f0f2..5e20b2a 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -116,6 +116,8 @@ #define DEFAULT_MIGRATE_ANNOUNCE_ROUNDS 5 #define DEFAULT_MIGRATE_ANNOUNCE_STEP 100 =20 +#define DEFAULT_MIGRATE_VCPU_DIRTYLIMIT_PERIOD 500 /* ms */ + static NotifierList migration_state_notifiers =3D NOTIFIER_LIST_INITIALIZER(migration_state_notifiers); =20 @@ -921,6 +923,9 @@ MigrationParameters *qmp_query_migrate_parameters(Error= **errp) s->parameters.block_bitmap_mapping); } =20 + params->has_vcpu_dirtylimit_period =3D true; + params->vcpu_dirtylimit_period =3D s->parameters.vcpu_dirtylimit_perio= d; + return params; } =20 @@ -1581,6 +1586,10 @@ static void migrate_params_test_apply(MigrateSetPara= meters *params, dest->has_block_bitmap_mapping =3D true; dest->block_bitmap_mapping =3D params->block_bitmap_mapping; } + + if (params->has_vcpu_dirtylimit_period) { + dest->vcpu_dirtylimit_period =3D params->vcpu_dirtylimit_period; + } } =20 static void migrate_params_apply(MigrateSetParameters *params, Error **err= p) @@ -1703,6 +1712,9 @@ static void migrate_params_apply(MigrateSetParameters= *params, Error **errp) QAPI_CLONE(BitmapMigrationNodeAliasList, params->block_bitmap_mapping); } + if (params->has_vcpu_dirtylimit_period) { + s->parameters.vcpu_dirtylimit_period =3D params->vcpu_dirtylimit_p= eriod; + } } =20 void qmp_migrate_set_parameters(MigrateSetParameters *params, Error **errp) @@ -2403,6 +2415,11 @@ bool migrate_auto_converge(void) return s->enabled_capabilities[MIGRATION_CAPABILITY_AUTO_CONVERGE]; } =20 +bool migrate_dirtylimit(void) +{ + return false; +} + bool migrate_zero_blocks(void) { MigrationState *s; @@ -4221,6 +4238,9 @@ static Property migration_properties[] =3D { DEFINE_PROP_SIZE("announce-step", MigrationState, parameters.announce_step, DEFAULT_MIGRATE_ANNOUNCE_STEP), + DEFINE_PROP_UINT64("vcpu-dirtylimit-period", MigrationState, + parameters.vcpu_dirtylimit_period, + DEFAULT_MIGRATE_VCPU_DIRTYLIMIT_PERIOD), =20 /* Migration capabilities */ DEFINE_PROP_MIG_CAP("x-xbzrle", MIGRATION_CAPABILITY_XBZRLE), diff --git a/migration/migration.h b/migration/migration.h index 2de861d..43541cf 100644 --- a/migration/migration.h +++ b/migration/migration.h @@ -369,6 +369,7 @@ bool migrate_ignore_shared(void); bool migrate_validate_uuid(void); =20 bool migrate_auto_converge(void); +bool migrate_dirtylimit(void); bool migrate_use_multifd(void); bool migrate_pause_before_switchover(void); int migrate_multifd_channels(void); diff --git a/monitor/hmp-cmds.c b/monitor/hmp-cmds.c index 6349684..fa33a35 100644 --- a/monitor/hmp-cmds.c +++ b/monitor/hmp-cmds.c @@ -525,6 +525,10 @@ void hmp_info_migrate_parameters(Monitor *mon, const Q= Dict *qdict) } } } + + monitor_printf(mon, "%s: %" PRIu64 " MB/s\n", + MigrationParameter_str(MIGRATION_PARAMETER_VCPU_DIRTYLIMIT_PER= IOD), + params->vcpu_dirtylimit_period); } =20 qapi_free_MigrationParameters(params); @@ -1344,6 +1348,10 @@ void hmp_migrate_set_parameter(Monitor *mon, const Q= Dict *qdict) error_setg(&err, "The block-bitmap-mapping parameter can only be s= et " "through QMP"); break; + case MIGRATION_PARAMETER_VCPU_DIRTYLIMIT_PERIOD: + p->has_vcpu_dirtylimit_period =3D true; + visit_type_size(v, param, &p->vcpu_dirtylimit_period, &err); + break; default: assert(0); } diff --git a/qapi/migration.json b/qapi/migration.json index 6a21027..260fb66 100644 --- a/qapi/migration.json +++ b/qapi/migration.json @@ -760,6 +760,9 @@ # block device name if there is one, and to their n= ode name # otherwise. (Since 5.2) # +# @vcpu-dirtylimit-period: Periodic time (ms) of dirtylimit during live mi= gration. +# Defaults to 500ms. (Since 7.0) +# # Features: # @unstable: Member @x-checkpoint-delay is experimental. # @@ -779,8 +782,8 @@ 'multifd-channels', 'xbzrle-cache-size', 'max-postcopy-bandwidth', 'max-cpu-throttle', 'multifd-compression', - 'multifd-zlib-level' ,'multifd-zstd-level', - 'block-bitmap-mapping' ] } + 'multifd-zlib-level', 'multifd-zstd-level', + 'block-bitmap-mapping', 'vcpu-dirtylimit-period'] } =20 ## # @MigrateSetParameters: @@ -925,6 +928,9 @@ # block device name if there is one, and to their n= ode name # otherwise. (Since 5.2) # +# @vcpu-dirtylimit-period: Periodic time (ms) of dirtylimit during live mi= gration. +# Defaults to 500ms. (Since 7.0) +# # Features: # @unstable: Member @x-checkpoint-delay is experimental. # @@ -960,7 +966,8 @@ '*multifd-compression': 'MultiFDCompression', '*multifd-zlib-level': 'uint8', '*multifd-zstd-level': 'uint8', - '*block-bitmap-mapping': [ 'BitmapMigrationNodeAlias' ] } } + '*block-bitmap-mapping': [ 'BitmapMigrationNodeAlias' ], + '*vcpu-dirtylimit-period': 'uint64'} } =20 ## # @migrate-set-parameters: @@ -1125,6 +1132,9 @@ # block device name if there is one, and to their n= ode name # otherwise. (Since 5.2) # +# @vcpu-dirtylimit-period: Periodic time (ms) of dirtylimit during live mi= gration. +# Defaults to 500ms. (Since 7.0) +# # Features: # @unstable: Member @x-checkpoint-delay is experimental. # @@ -1158,7 +1168,8 @@ '*multifd-compression': 'MultiFDCompression', '*multifd-zlib-level': 'uint8', '*multifd-zstd-level': 'uint8', - '*block-bitmap-mapping': [ 'BitmapMigrationNodeAlias' ] } } + '*block-bitmap-mapping': [ 'BitmapMigrationNodeAlias' ], + '*vcpu-dirtylimit-period': 'uint64'} } =20 ## # @query-migrate-parameters: diff --git a/softmmu/dirtylimit.c b/softmmu/dirtylimit.c index 365bd43..affe993 100644 --- a/softmmu/dirtylimit.c +++ b/softmmu/dirtylimit.c @@ -23,6 +23,8 @@ #include "exec/memory.h" #include "hw/boards.h" #include "sysemu/kvm.h" +#include "migration/misc.h" +#include "migration/migration.h" #include "trace.h" =20 /* @@ -75,11 +77,18 @@ static bool dirtylimit_quit; =20 static void vcpu_dirty_rate_stat_collect(void) { + MigrationState *s =3D migrate_get_current(); VcpuStat stat; int i =3D 0; + int64_t period =3D DIRTYLIMIT_CALC_TIME_MS; + + if (migrate_dirtylimit() && + migration_is_active(s)) { + period =3D s->parameters.vcpu_dirtylimit_period; + } =20 /* calculate vcpu dirtyrate */ - vcpu_calculate_dirtyrate(DIRTYLIMIT_CALC_TIME_MS, + vcpu_calculate_dirtyrate(period, &stat, GLOBAL_DIRTY_LIMIT, false); --=20 1.8.3.1 From nobody Sun May 5 09:42:17 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1652770106493495.711723231415; Mon, 16 May 2022 23:48:26 -0700 (PDT) Received: from localhost ([::1]:59454 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nqr0L-0003mV-2L for importer@patchew.org; Tue, 17 May 2022 02:48:25 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:58678) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nqqoK-0004kx-5d for qemu-devel@nongnu.org; Tue, 17 May 2022 02:36:03 -0400 Received: from prt-mail.chinatelecom.cn ([42.123.76.219]:46489 helo=chinatelecom.cn) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nqqoG-0007fb-Dv for qemu-devel@nongnu.org; Tue, 17 May 2022 02:35:59 -0400 Received: from clientip-36.111.64.84 (unknown [172.18.0.48]) by chinatelecom.cn (HERMES) with SMTP id D872B2800CB; Tue, 17 May 2022 14:35:33 +0800 (CST) Received: from ([172.18.0.48]) by app0024 with ESMTP id 036f46472a594c689f4d67785179dbc2 for qemu-devel@nongnu.org; Tue, 17 May 2022 14:35:35 CST HMM_SOURCE_IP: 172.18.0.48:35216.1855245433 HMM_ATTACHE_NUM: 0000 HMM_SOURCE_TYPE: SMTP X-189-SAVE-TO-SEND: +huangy81@chinatelecom.cn X-Transaction-ID: 036f46472a594c689f4d67785179dbc2 X-Real-From: huangy81@chinatelecom.cn X-Receive-IP: 172.18.0.48 X-MEDUSA-Status: 0 From: huangy81@chinatelecom.cn To: qemu-devel Cc: "Dr. David Alan Gilbert" , Juan Quintela , Eric Blake , Markus Armbruster , Thomas Huth , Laurent Vivier , Paolo Bonzini , Hyman Huang Subject: [RFC 2/6] qapi/migration: Introduce vcpu-dirtylimit parameters Date: Tue, 17 May 2022 14:35:02 +0800 Message-Id: <2fbc49b77115b5f8fbebbee00476f6f34dad4770.1652762652.git.huangy81@chinatelecom.cn> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: References: In-Reply-To: References: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=42.123.76.219; envelope-from=huangy81@chinatelecom.cn; helo=chinatelecom.cn X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZM-MESSAGEID: 1652770108442100001 From: Hyman Huang(=E9=BB=84=E5=8B=87) Introduce "vcpu-dirtylimit" migration parameter used to limit dirty page rate during live migration. "vcpu-dirtylimit" and "vcpu-dirtylimit-period" are two dirtylimit-related migration parameters, which can be set before and during live migration by qmp migrate-set-parameters. This two parameters are used to help implement the dirty page rate limit algo of migration. Signed-off-by: Hyman Huang(=E9=BB=84=E5=8B=87) --- migration/migration.c | 13 +++++++++++++ monitor/hmp-cmds.c | 7 +++++++ qapi/migration.json | 18 +++++++++++++++--- 3 files changed, 35 insertions(+), 3 deletions(-) diff --git a/migration/migration.c b/migration/migration.c index 5e20b2a..9e4ce01 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -117,6 +117,7 @@ #define DEFAULT_MIGRATE_ANNOUNCE_STEP 100 =20 #define DEFAULT_MIGRATE_VCPU_DIRTYLIMIT_PERIOD 500 /* ms */ +#define DEFAULT_MIGRATE_VCPU_DIRTYLIMIT 1 /* MB/s */ =20 static NotifierList migration_state_notifiers =3D NOTIFIER_LIST_INITIALIZER(migration_state_notifiers); @@ -925,6 +926,8 @@ MigrationParameters *qmp_query_migrate_parameters(Error= **errp) =20 params->has_vcpu_dirtylimit_period =3D true; params->vcpu_dirtylimit_period =3D s->parameters.vcpu_dirtylimit_perio= d; + params->has_vcpu_dirtylimit =3D true; + params->vcpu_dirtylimit =3D s->parameters.vcpu_dirtylimit; =20 return params; } @@ -1590,6 +1593,10 @@ static void migrate_params_test_apply(MigrateSetPara= meters *params, if (params->has_vcpu_dirtylimit_period) { dest->vcpu_dirtylimit_period =3D params->vcpu_dirtylimit_period; } + + if (params->has_vcpu_dirtylimit) { + dest->vcpu_dirtylimit =3D params->vcpu_dirtylimit; + } } =20 static void migrate_params_apply(MigrateSetParameters *params, Error **err= p) @@ -1715,6 +1722,9 @@ static void migrate_params_apply(MigrateSetParameters= *params, Error **errp) if (params->has_vcpu_dirtylimit_period) { s->parameters.vcpu_dirtylimit_period =3D params->vcpu_dirtylimit_p= eriod; } + if (params->has_vcpu_dirtylimit) { + s->parameters.vcpu_dirtylimit =3D params->vcpu_dirtylimit; + } } =20 void qmp_migrate_set_parameters(MigrateSetParameters *params, Error **errp) @@ -4241,6 +4251,9 @@ static Property migration_properties[] =3D { DEFINE_PROP_UINT64("vcpu-dirtylimit-period", MigrationState, parameters.vcpu_dirtylimit_period, DEFAULT_MIGRATE_VCPU_DIRTYLIMIT_PERIOD), + DEFINE_PROP_UINT64("vcpu-dirtylimit", MigrationState, + parameters.vcpu_dirtylimit, + DEFAULT_MIGRATE_VCPU_DIRTYLIMIT), =20 /* Migration capabilities */ DEFINE_PROP_MIG_CAP("x-xbzrle", MIGRATION_CAPABILITY_XBZRLE), diff --git a/monitor/hmp-cmds.c b/monitor/hmp-cmds.c index fa33a35..31a0564 100644 --- a/monitor/hmp-cmds.c +++ b/monitor/hmp-cmds.c @@ -529,6 +529,9 @@ void hmp_info_migrate_parameters(Monitor *mon, const QD= ict *qdict) monitor_printf(mon, "%s: %" PRIu64 " MB/s\n", MigrationParameter_str(MIGRATION_PARAMETER_VCPU_DIRTYLIMIT_PER= IOD), params->vcpu_dirtylimit_period); + monitor_printf(mon, "%s: %" PRIu64 " MB/s\n", + MigrationParameter_str(MIGRATION_PARAMETER_VCPU_DIRTYLIMIT), + params->vcpu_dirtylimit); } =20 qapi_free_MigrationParameters(params); @@ -1352,6 +1355,10 @@ void hmp_migrate_set_parameter(Monitor *mon, const Q= Dict *qdict) p->has_vcpu_dirtylimit_period =3D true; visit_type_size(v, param, &p->vcpu_dirtylimit_period, &err); break; + case MIGRATION_PARAMETER_VCPU_DIRTYLIMIT: + p->has_vcpu_dirtylimit =3D true; + visit_type_size(v, param, &p->vcpu_dirtylimit, &err); + break; default: assert(0); } diff --git a/qapi/migration.json b/qapi/migration.json index 260fb66..68c1fe0 100644 --- a/qapi/migration.json +++ b/qapi/migration.json @@ -763,6 +763,9 @@ # @vcpu-dirtylimit-period: Periodic time (ms) of dirtylimit during live mi= gration. # Defaults to 500ms. (Since 7.0) # +# @vcpu-dirtylimit: Dirtyrate limit (MB/s) during live migration. +# Defaults to 1. (Since 7.0) +# # Features: # @unstable: Member @x-checkpoint-delay is experimental. # @@ -783,7 +786,8 @@ 'xbzrle-cache-size', 'max-postcopy-bandwidth', 'max-cpu-throttle', 'multifd-compression', 'multifd-zlib-level', 'multifd-zstd-level', - 'block-bitmap-mapping', 'vcpu-dirtylimit-period'] } + 'block-bitmap-mapping', 'vcpu-dirtylimit-period', + 'vcpu-dirtylimit'] } =20 ## # @MigrateSetParameters: @@ -931,6 +935,9 @@ # @vcpu-dirtylimit-period: Periodic time (ms) of dirtylimit during live mi= gration. # Defaults to 500ms. (Since 7.0) # +# @vcpu-dirtylimit: Dirtyrate limit (MB/s) during live migration. +# Defaults to 1. (Since 7.0) +# # Features: # @unstable: Member @x-checkpoint-delay is experimental. # @@ -967,7 +974,8 @@ '*multifd-zlib-level': 'uint8', '*multifd-zstd-level': 'uint8', '*block-bitmap-mapping': [ 'BitmapMigrationNodeAlias' ], - '*vcpu-dirtylimit-period': 'uint64'} } + '*vcpu-dirtylimit-period': 'uint64', + '*vcpu-dirtylimit': 'uint64'} } =20 ## # @migrate-set-parameters: @@ -1135,6 +1143,9 @@ # @vcpu-dirtylimit-period: Periodic time (ms) of dirtylimit during live mi= gration. # Defaults to 500ms. (Since 7.0) # +# @vcpu-dirtylimit: Dirtyrate limit (MB/s) during live migration. +# Defaults to 1. (Since 7.0) +# # Features: # @unstable: Member @x-checkpoint-delay is experimental. # @@ -1169,7 +1180,8 @@ '*multifd-zlib-level': 'uint8', '*multifd-zstd-level': 'uint8', '*block-bitmap-mapping': [ 'BitmapMigrationNodeAlias' ], - '*vcpu-dirtylimit-period': 'uint64'} } + '*vcpu-dirtylimit-period': 'uint64', + '*vcpu-dirtylimit': 'uint64'} } =20 ## # @query-migrate-parameters: --=20 1.8.3.1 From nobody Sun May 5 09:42:17 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1652769760847664.7045621400621; Mon, 16 May 2022 23:42:40 -0700 (PDT) Received: from localhost ([::1]:53294 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nqqul-0007rn-4U for importer@patchew.org; Tue, 17 May 2022 02:42:39 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:58674) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nqqoK-0004kq-4d for qemu-devel@nongnu.org; Tue, 17 May 2022 02:36:03 -0400 Received: from prt-mail.chinatelecom.cn ([42.123.76.219]:46494 helo=chinatelecom.cn) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nqqoG-0007fd-Dv for qemu-devel@nongnu.org; Tue, 17 May 2022 02:35:59 -0400 Received: from clientip-36.111.64.84 (unknown [172.18.0.48]) by chinatelecom.cn (HERMES) with SMTP id 10A9D2800CE; Tue, 17 May 2022 14:35:36 +0800 (CST) Received: from ([172.18.0.48]) by app0024 with ESMTP id 91bf7af147a143d1b4d585b131797977 for qemu-devel@nongnu.org; Tue, 17 May 2022 14:35:39 CST HMM_SOURCE_IP: 172.18.0.48:35216.1855245433 HMM_ATTACHE_NUM: 0000 HMM_SOURCE_TYPE: SMTP X-189-SAVE-TO-SEND: +huangy81@chinatelecom.cn X-Transaction-ID: 91bf7af147a143d1b4d585b131797977 X-Real-From: huangy81@chinatelecom.cn X-Receive-IP: 172.18.0.48 X-MEDUSA-Status: 0 From: huangy81@chinatelecom.cn To: qemu-devel Cc: "Dr. David Alan Gilbert" , Juan Quintela , Eric Blake , Markus Armbruster , Thomas Huth , Laurent Vivier , Paolo Bonzini , Hyman Huang Subject: [RFC 3/6] migration: Implement dirtylimit convergence algo Date: Tue, 17 May 2022 14:35:03 +0800 Message-Id: <8d66c3055495d81c81a645885ce23bbdb40bcdc9.1652762652.git.huangy81@chinatelecom.cn> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: References: In-Reply-To: References: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=42.123.76.219; envelope-from=huangy81@chinatelecom.cn; helo=chinatelecom.cn X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZM-MESSAGEID: 1652769764478100003 From: Hyman Huang(=E9=BB=84=E5=8B=87) Implement dirtylimit convergence algo for live migration, which is kind of like auto-converge algo but using dirtylimit instead of cpu throttle to make migration convergent. Signed-off-by: Hyman Huang(=E9=BB=84=E5=8B=87) --- migration/ram.c | 53 +++++++++++++++++++++++++++++++++++++---------= ---- migration/trace-events | 1 + 2 files changed, 41 insertions(+), 13 deletions(-) diff --git a/migration/ram.c b/migration/ram.c index 3532f64..5dd3e69 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -44,6 +44,7 @@ #include "qapi/error.h" #include "qapi/qapi-types-migration.h" #include "qapi/qapi-events-migration.h" +#include "qapi/qapi-commands-migration.h" #include "qapi/qmp/qerror.h" #include "trace.h" #include "exec/ram_addr.h" @@ -56,6 +57,8 @@ #include "qemu/iov.h" #include "multifd.h" #include "sysemu/runstate.h" +#include "sysemu/dirtylimit.h" +#include "sysemu/kvm.h" =20 #include "hw/boards.h" /* for machine_dump_guest_core() */ =20 @@ -1082,6 +1085,21 @@ static void migration_update_rates(RAMState *rs, int= 64_t end_time) } } =20 +/* + * Enable dirtylimit to throttle down the guest + */ +static void migration_dirtylimit_guest(void) +{ + if (!dirtylimit_in_service()) { + MigrationState *s =3D migrate_get_current(); + int64_t quota_dirtyrate =3D s->parameters.vcpu_dirtylimit; + + /* Set quota dirtyrate if dirty limit not in service */ + qmp_set_vcpu_dirty_limit(false, -1, quota_dirtyrate, NULL); + trace_migration_dirtylimit_guest(quota_dirtyrate); + } +} + static void migration_trigger_throttle(RAMState *rs) { MigrationState *s =3D migrate_get_current(); @@ -1091,22 +1109,31 @@ static void migration_trigger_throttle(RAMState *rs) uint64_t bytes_dirty_period =3D rs->num_dirty_pages_period * TARGET_PA= GE_SIZE; uint64_t bytes_dirty_threshold =3D bytes_xfer_period * threshold / 100; =20 - /* During block migration the auto-converge logic incorrectly detects - * that ram migration makes no progress. Avoid this by disabling the - * throttling logic during the bulk phase of block migration. */ - if (migrate_auto_converge() && !blk_mig_bulk_active()) { - /* The following detection logic can be refined later. For now: - Check to see if the ratio between dirtied bytes and the approx. - amount of bytes that just got transferred since the last time - we were in this routine reaches the threshold. If that happens - twice, start or increase throttling. */ - - if ((bytes_dirty_period > bytes_dirty_threshold) && - (++rs->dirty_rate_high_cnt >=3D 2)) { + /* + * The following detection logic can be refined later. For now: + * Check to see if the ratio between dirtied bytes and the approx. + * amount of bytes that just got transferred since the last time + * we were in this routine reaches the threshold. If that happens + * twice, start or increase throttling. + */ + + if ((bytes_dirty_period > bytes_dirty_threshold) && + (++rs->dirty_rate_high_cnt >=3D 2)) { + rs->dirty_rate_high_cnt =3D 0; + /* + * During block migration the auto-converge logic incorrectly dete= cts + * that ram migration makes no progress. Avoid this by disabling t= he + * throttling logic during the bulk phase of block migration. + */ + + if (migrate_auto_converge() && !blk_mig_bulk_active()) { trace_migration_throttle(); - rs->dirty_rate_high_cnt =3D 0; mig_throttle_guest_down(bytes_dirty_period, bytes_dirty_threshold); + } else if (migrate_dirtylimit() && + kvm_dirty_ring_enabled() && + migration_is_active(s)) { + migration_dirtylimit_guest(); } } } diff --git a/migration/trace-events b/migration/trace-events index 1aec580..2c341fc 100644 --- a/migration/trace-events +++ b/migration/trace-events @@ -89,6 +89,7 @@ migration_bitmap_sync_start(void) "" migration_bitmap_sync_end(uint64_t dirty_pages) "dirty_pages %" PRIu64 migration_bitmap_clear_dirty(char *str, uint64_t start, uint64_t size, uns= igned long page) "rb %s start 0x%"PRIx64" size 0x%"PRIx64" page 0x%lx" migration_throttle(void) "" +migration_dirtylimit_guest(int64_t dirtyrate) "guest dirty page rate limit= %" PRIi64 " MB/s" ram_discard_range(const char *rbname, uint64_t start, size_t len) "%s: sta= rt: %" PRIx64 " %zx" ram_load_loop(const char *rbname, uint64_t addr, int flags, void *host) "%= s: addr: 0x%" PRIx64 " flags: 0x%x host: %p" ram_load_postcopy_loop(uint64_t addr, int flags) "@%" PRIx64 " %x" --=20 1.8.3.1 From nobody Sun May 5 09:42:17 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1652769758642353.1537799917986; Mon, 16 May 2022 23:42:38 -0700 (PDT) Received: from localhost ([::1]:53004 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nqquh-0007Yt-7P for importer@patchew.org; Tue, 17 May 2022 02:42:35 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:58676) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nqqoK-0004ks-D4 for qemu-devel@nongnu.org; Tue, 17 May 2022 02:36:03 -0400 Received: from prt-mail.chinatelecom.cn ([42.123.76.219]:46516 helo=chinatelecom.cn) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nqqoG-0007fk-EW for qemu-devel@nongnu.org; Tue, 17 May 2022 02:35:59 -0400 Received: from clientip-36.111.64.84 (unknown [172.18.0.48]) by chinatelecom.cn (HERMES) with SMTP id B9D232800D4; Tue, 17 May 2022 14:35:39 +0800 (CST) Received: from ([172.18.0.48]) by app0024 with ESMTP id 3d1468d696774e69bc9b1d05cec76146 for qemu-devel@nongnu.org; Tue, 17 May 2022 14:35:41 CST HMM_SOURCE_IP: 172.18.0.48:35216.1855245433 HMM_ATTACHE_NUM: 0000 HMM_SOURCE_TYPE: SMTP X-189-SAVE-TO-SEND: +huangy81@chinatelecom.cn X-Transaction-ID: 3d1468d696774e69bc9b1d05cec76146 X-Real-From: huangy81@chinatelecom.cn X-Receive-IP: 172.18.0.48 X-MEDUSA-Status: 0 From: huangy81@chinatelecom.cn To: qemu-devel Cc: "Dr. David Alan Gilbert" , Juan Quintela , Eric Blake , Markus Armbruster , Thomas Huth , Laurent Vivier , Paolo Bonzini , Hyman Huang Subject: [RFC 4/6] migration: Introduce dirtylimit capability Date: Tue, 17 May 2022 14:35:04 +0800 Message-Id: <2ef63cf661750cd848492c4f917e46e3700c1409.1652762652.git.huangy81@chinatelecom.cn> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: References: In-Reply-To: References: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=42.123.76.219; envelope-from=huangy81@chinatelecom.cn; helo=chinatelecom.cn X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZM-MESSAGEID: 1652769762853100001 From: Hyman Huang(=E9=BB=84=E5=8B=87) Introduce migration dirtylimit capability, which can be turned on before live migration and limit dirty page rate durty live migration. Dirtylimit dirtylimit capability is kind of like auto-converge but using dirtylimit instead of traditional cpu-throttle to throttle guest down. To enable this feature, turn on the dirtylimit capability before live migration using migratioin-set-capabilities, and set dirtylimit-related parameters "vcpu-dirtylimit", "vcpu-dirtylimit-period" suitably to speed up convergence. Signed-off-by: Hyman Huang(=E9=BB=84=E5=8B=87) --- migration/migration.c | 7 ++++++- qapi/migration.json | 5 ++++- 2 files changed, 10 insertions(+), 2 deletions(-) diff --git a/migration/migration.c b/migration/migration.c index 9e4ce01..4a659b6 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -2427,7 +2427,11 @@ bool migrate_auto_converge(void) =20 bool migrate_dirtylimit(void) { - return false; + MigrationState *s; + + s =3D migrate_get_current(); + + return s->enabled_capabilities[MIGRATION_CAPABILITY_DIRTYLIMIT]; } =20 bool migrate_zero_blocks(void) @@ -4270,6 +4274,7 @@ static Property migration_properties[] =3D { DEFINE_PROP_MIG_CAP("x-multifd", MIGRATION_CAPABILITY_MULTIFD), DEFINE_PROP_MIG_CAP("x-background-snapshot", MIGRATION_CAPABILITY_BACKGROUND_SNAPSHOT), + DEFINE_PROP_MIG_CAP("x-dirtylimit", MIGRATION_CAPABILITY_DIRTYLIMIT), =20 DEFINE_PROP_END_OF_LIST(), }; diff --git a/qapi/migration.json b/qapi/migration.json index 68c1fe0..30ad413 100644 --- a/qapi/migration.json +++ b/qapi/migration.json @@ -463,6 +463,9 @@ # procedure starts. The VM RAM is saved with running= VM. # (since 6.0) # +# @dirtylimit: Use dirtylimit to throttle down guest if enabled. +# (since 7.0) +# # Features: # @unstable: Members @x-colo and @x-ignore-shared are experimental. # @@ -476,7 +479,7 @@ 'block', 'return-path', 'pause-before-switchover', 'multifd', 'dirty-bitmaps', 'postcopy-blocktime', 'late-block-activate', { 'name': 'x-ignore-shared', 'features': [ 'unstable' ] }, - 'validate-uuid', 'background-snapshot'] } + 'validate-uuid', 'background-snapshot', 'dirtylimit'] } =20 ## # @MigrationCapabilityStatus: --=20 1.8.3.1 From nobody Sun May 5 09:42:17 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1652769826267176.00562229068487; Mon, 16 May 2022 23:43:46 -0700 (PDT) Received: from localhost ([::1]:55898 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nqqvp-0001Dk-7Y for importer@patchew.org; Tue, 17 May 2022 02:43:45 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:58714) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nqqoN-0004mC-KF for qemu-devel@nongnu.org; Tue, 17 May 2022 02:36:04 -0400 Received: from prt-mail.chinatelecom.cn ([42.123.76.219]:46625 helo=chinatelecom.cn) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nqqoL-0007mR-FN for qemu-devel@nongnu.org; Tue, 17 May 2022 02:36:03 -0400 Received: from clientip-36.111.64.84 (unknown [172.18.0.48]) by chinatelecom.cn (HERMES) with SMTP id 73F9E2800D6; Tue, 17 May 2022 14:35:41 +0800 (CST) Received: from ([172.18.0.48]) by app0024 with ESMTP id b65c6fbde9e2463098aabac31856ca78 for qemu-devel@nongnu.org; Tue, 17 May 2022 14:35:45 CST HMM_SOURCE_IP: 172.18.0.48:35216.1855245433 HMM_ATTACHE_NUM: 0000 HMM_SOURCE_TYPE: SMTP X-189-SAVE-TO-SEND: +huangy81@chinatelecom.cn X-Transaction-ID: b65c6fbde9e2463098aabac31856ca78 X-Real-From: huangy81@chinatelecom.cn X-Receive-IP: 172.18.0.48 X-MEDUSA-Status: 0 From: huangy81@chinatelecom.cn To: qemu-devel Cc: "Dr. David Alan Gilbert" , Juan Quintela , Eric Blake , Markus Armbruster , Thomas Huth , Laurent Vivier , Paolo Bonzini , Hyman Huang Subject: [RFC 5/6] migration: Add dirtylimit data into migration info Date: Tue, 17 May 2022 14:35:05 +0800 Message-Id: X-Mailer: git-send-email 1.8.3.1 In-Reply-To: References: In-Reply-To: References: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=42.123.76.219; envelope-from=huangy81@chinatelecom.cn; helo=chinatelecom.cn X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZM-MESSAGEID: 1652769826881100001 From: Hyman Huang(=E9=BB=84=E5=8B=87) Add dirtylimit throttle data into migration info, through which we can observe the process of dirtylimit during live migration. Signed-off-by: Hyman Huang(=E9=BB=84=E5=8B=87) --- include/sysemu/dirtylimit.h | 2 ++ migration/migration.c | 10 ++++++++++ qapi/migration.json | 11 ++++++++++- softmmu/dirtylimit.c | 22 ++++++++++++++++++++++ 4 files changed, 44 insertions(+), 1 deletion(-) diff --git a/include/sysemu/dirtylimit.h b/include/sysemu/dirtylimit.h index 8d2c1f3..0b8dd76 100644 --- a/include/sysemu/dirtylimit.h +++ b/include/sysemu/dirtylimit.h @@ -34,4 +34,6 @@ void dirtylimit_set_vcpu(int cpu_index, void dirtylimit_set_all(uint64_t quota, bool enable); void dirtylimit_vcpu_execute(CPUState *cpu); +int64_t dirtylimit_throttle_us_per_full(void); +int64_t dirtylimit_us_per_full(void); #endif diff --git a/migration/migration.c b/migration/migration.c index 4a659b6..935179e 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -61,6 +61,7 @@ #include "sysemu/cpus.h" #include "yank_functions.h" #include "sysemu/qtest.h" +#include "sysemu/dirtylimit.h" =20 #define MAX_THROTTLE (128 << 20) /* Migration transfer speed throttl= ing */ =20 @@ -1065,6 +1066,15 @@ static void populate_ram_info(MigrationInfo *info, M= igrationState *s) info->ram->remaining =3D ram_bytes_remaining(); info->ram->dirty_pages_rate =3D ram_counters.dirty_pages_rate; } + + if (migrate_dirtylimit() && dirtylimit_in_service()) { + info->has_dirtylimit_throttle_us_per_full =3D true; + info->dirtylimit_throttle_us_per_full =3D + dirtylimit_throttle_us_per_full(); + + info->has_dirtylimit_us_per_full =3D true; + info->dirtylimit_us_per_full =3D dirtylimit_us_per_full(); + } } =20 static void populate_disk_info(MigrationInfo *info) diff --git a/qapi/migration.json b/qapi/migration.json index 30ad413..cac4c8d 100644 --- a/qapi/migration.json +++ b/qapi/migration.json @@ -239,6 +239,13 @@ # Present and non-empty when migration is blocked. # (since 6.0) # +# @dirtylimit-throttle-us-per-full: Throttle time (us) during the period of +# dirty ring full. +# (since 7.0) +# +# @dirtylimit-us-per-full: Estimated the periodic time (us) of dirty ring = full. +# (since 7.0) +# # Since: 0.14 ## { 'struct': 'MigrationInfo', @@ -256,7 +263,9 @@ '*postcopy-blocktime' : 'uint32', '*postcopy-vcpu-blocktime': ['uint32'], '*compression': 'CompressionStats', - '*socket-address': ['SocketAddress'] } } + '*socket-address': ['SocketAddress'], + '*dirtylimit-throttle-us-per-full': 'int64', + '*dirtylimit-us-per-full': 'int64'} } =20 ## # @query-migrate: diff --git a/softmmu/dirtylimit.c b/softmmu/dirtylimit.c index affe993..33440c0 100644 --- a/softmmu/dirtylimit.c +++ b/softmmu/dirtylimit.c @@ -546,6 +546,28 @@ static struct DirtyLimitInfo *dirtylimit_query_vcpu(in= t cpu_index) return info; } =20 +/* Pick up first vcpu throttle time by default */ +int64_t dirtylimit_throttle_us_per_full(void) +{ + CPUState *cpu =3D first_cpu; + return cpu->throttle_us_per_full; +} + +/* + * Estimate dirty ring full time under current dirty page rate. + * Return -1 if guest doesn't dirty memory. + */ +int64_t dirtylimit_us_per_full(void) +{ + uint64_t curr_rate =3D vcpu_dirty_rate_get(0); + + if (!curr_rate) { + return -1; + } + + return dirtylimit_dirty_ring_full_time(curr_rate); +} + static struct DirtyLimitInfoList *dirtylimit_query_all(void) { int i, index; --=20 1.8.3.1 From nobody Sun May 5 09:42:17 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1652770405398192.89784742073914; Mon, 16 May 2022 23:53:25 -0700 (PDT) Received: from localhost ([::1]:36128 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nqr5A-0007Tu-62 for importer@patchew.org; Tue, 17 May 2022 02:53:24 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:58712) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nqqoN-0004mB-Kc for qemu-devel@nongnu.org; Tue, 17 May 2022 02:36:04 -0400 Received: from prt-mail.chinatelecom.cn ([42.123.76.219]:46624 helo=chinatelecom.cn) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nqqoL-0007mT-Fa for qemu-devel@nongnu.org; Tue, 17 May 2022 02:36:03 -0400 Received: from clientip-36.111.64.84 (unknown [172.18.0.48]) by chinatelecom.cn (HERMES) with SMTP id 35DA62800B4; Tue, 17 May 2022 14:35:45 +0800 (CST) Received: from ([172.18.0.48]) by app0024 with ESMTP id 9482ce7fa186442aa22549714a1cd696 for qemu-devel@nongnu.org; Tue, 17 May 2022 14:35:48 CST HMM_SOURCE_IP: 172.18.0.48:35216.1855245433 HMM_ATTACHE_NUM: 0000 HMM_SOURCE_TYPE: SMTP X-189-SAVE-TO-SEND: +huangy81@chinatelecom.cn X-Transaction-ID: 9482ce7fa186442aa22549714a1cd696 X-Real-From: huangy81@chinatelecom.cn X-Receive-IP: 172.18.0.48 X-MEDUSA-Status: 0 From: huangy81@chinatelecom.cn To: qemu-devel Cc: "Dr. David Alan Gilbert" , Juan Quintela , Eric Blake , Markus Armbruster , Thomas Huth , Laurent Vivier , Paolo Bonzini , Hyman Huang Subject: [RFC 6/6] tests: Add migration dirtylimit capability test Date: Tue, 17 May 2022 14:35:06 +0800 Message-Id: X-Mailer: git-send-email 1.8.3.1 In-Reply-To: References: In-Reply-To: References: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=42.123.76.219; envelope-from=huangy81@chinatelecom.cn; helo=chinatelecom.cn X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZM-MESSAGEID: 1652770406618100001 From: Hyman Huang(=E9=BB=84=E5=8B=87) Add migration dirtylimit capability test if kernel support dirty ring. Migration dirtylimit capability introduce dirtylimit capability , two parameters: vcpu-dirtylimit-period and vcpu-dirtylimit to implement the live migration with dirtylimit. The test case enable the capability and set the corresponding parameters to test migration. When migration switch to pre-switchover phase, like the auto-converge, checking if migration satisfy the convergence condition. Signed-off-by: Hyman Huang(=E9=BB=84=E5=8B=87) --- tests/qtest/migration-test.c | 89 ++++++++++++++++++++++++++++++++++++++++= ++++ 1 file changed, 89 insertions(+) diff --git a/tests/qtest/migration-test.c b/tests/qtest/migration-test.c index 40f2873..b0a53d7 100644 --- a/tests/qtest/migration-test.c +++ b/tests/qtest/migration-test.c @@ -1650,6 +1650,93 @@ static void test_vcpu_dirty_limit(void) dirtylimit_stop_vm(vm); } =20 +static void test_migrate_dirty_limit(void) +{ + g_autofree char *uri =3D g_strdup_printf("unix:%s/migsocket", tmpfs); + MigrateStart *args =3D migrate_start_new(); + QTestState *from, *to; + int64_t remaining, throttle_us_per_full; + /* + * We want the test to be stable and as fast as possible. + * E.g., with 1Gb/s bandwith migration may pass without dirtylimit, + * so we need to decrease a bandwidth. + */ + const int64_t dirtylimit_period =3D 1000, dirtylimit_value =3D 50; + const int64_t max_bandwidth =3D 400000000; /* ~400Mb/s */ + const int64_t downtime_limit =3D 250; /* 250ms */ + /* + * We migrate through unix-socket (> 500Mb/s). + * Thus, expected migration speed ~=3D bandwidth limit (< 500Mb/s). + * So, we can predict expected_threshold + */ + const int64_t expected_threshold =3D max_bandwidth * downtime_limit / = 1000; + + /* Enable dirty ring logging */ + args->use_dirty_ring =3D true; + + if (test_migrate_start(&from, &to, uri, &args)) { + return; + } + + migrate_set_capability(from, "dirtylimit", true); + migrate_set_parameter_int(from, "vcpu-dirtylimit-period", + dirtylimit_period); + migrate_set_parameter_int(from, "vcpu-dirtylimit", dirtylimit_value); + + /* + * Set the initial parameters so that the migration could not converge + * without dirtylimit. + */ + migrate_set_parameter_int(from, "downtime-limit", 1); + migrate_set_parameter_int(from, "max-bandwidth", 100000000); /* ~100Mb= /s */ + + /* To check limit rate after precopy */ + migrate_set_capability(from, "pause-before-switchover", true); + + /* Wait for the first serial output from the source */ + wait_for_serial("src_serial"); + + migrate_qmp(from, uri, "{}"); + + /* Wait for dirtylimit begins */ + throttle_us_per_full =3D 0; + while (throttle_us_per_full =3D=3D 0) { + throttle_us_per_full =3D + read_migrate_property_int(from, "dirtylimit-throttle-us-per-fu= ll"); + usleep(100); + g_assert_false(got_stop); + } + + /* + * The dirtylimit rate should equals the return value of + * query-vcpu-dirty-limit if dirtylimit cap set + */ + g_assert_cmpint(dirtylimit_value, =3D=3D, get_limit_rate(from)); + + /* Now, when we tested if dirtylimit works, let it converge */ + migrate_set_parameter_int(from, "downtime-limit", downtime_limit); + migrate_set_parameter_int(from, "max-bandwidth", max_bandwidth); + + /* + * Wait for pre-switchover status to check if migration + * satisfy the convergence condition + */ + wait_for_migration_status(from, "pre-switchover", NULL); + + remaining =3D read_ram_property_int(from, "remaining"); + g_assert_cmpint(remaining, <, + (expected_threshold + expected_threshold / 100)); + + migrate_continue(from, "pre-switchover"); + + qtest_qmp_eventwait(to, "RESUME"); + + wait_for_serial("dest_serial"); + wait_for_migration_complete(from); + + test_migrate_end(from, to, true); +} + static bool kvm_dirty_ring_supported(void) { #if defined(__linux__) && defined(HOST_X86_64) @@ -1741,6 +1828,8 @@ int main(int argc, char **argv) test_precopy_unix_dirty_ring); qtest_add_func("/migration/vcpu_dirty_limit", test_vcpu_dirty_limit); + qtest_add_func("/migration/dirty_limit", + test_migrate_dirty_limit); } =20 ret =3D g_test_run(); --=20 1.8.3.1