From nobody Sun May 5 00:58:04 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org ARC-Seal: i=1; a=rsa-sha256; t=1584333575; cv=none; d=zohomail.com; s=zohoarc; b=lxbOgMMz6JvuYNpDSHzpEGf3PiG73Wf0IcCDz2sFFmSKXbNv7TiJcJUKUWiczZQt7m174VO2Ef9yfQgS4DGH5PxQ/Y3AK1R6KDbGdDVOIhIt6N6xwbuSbZLYehLfVKff8hjmohuugaMH/RY/HEqoZ7AVXBYbN351tqMjBJdBVpg= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1584333575; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:Sender:Subject:To; bh=KwBtUf8l573M0A+Fq4NFvQe56rI/MwhASa5vSduimCw=; b=ZW3CLuUALbG2qelGsA2R4bQMh3mlBZEq9T2iKHqH27oHoaPEfuJjfjXs/BAeihUy4X4uJdH7uQA2E57OELz6VGLaHeFfpKs26y7D2i4hfjmWjty6njlygE/T6o+IKrvbS9roFnkf3vv5ZLMCzsA+V6dhqdc6xe1GtiG4g6GzWJA= ARC-Authentication-Results: i=1; mx.zohomail.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1584333575536824.7429203012118; Sun, 15 Mar 2020 21:39:35 -0700 (PDT) Received: from localhost ([::1]:34106 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jDhXJ-0005dK-AZ for importer@patchew.org; Mon, 16 Mar 2020 00:39:33 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:44312) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jDhQH-0004X5-FV for qemu-devel@nongnu.org; Mon, 16 Mar 2020 00:32:20 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1jDhQE-0004Ei-3R for qemu-devel@nongnu.org; Mon, 16 Mar 2020 00:32:17 -0400 Received: from szxga05-in.huawei.com ([45.249.212.191]:3723 helo=huawei.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1jDhQ8-0001vx-U2; Mon, 16 Mar 2020 00:32:09 -0400 Received: from DGGEMS404-HUB.china.huawei.com (unknown [172.30.72.58]) by Forcepoint Email with ESMTP id 7960E42CF3B8E82A115F; Mon, 16 Mar 2020 12:31:55 +0800 (CST) Received: from linux-kDCJWP.huawei.com (10.175.104.212) by DGGEMS404-HUB.china.huawei.com (10.3.19.204) with Microsoft SMTP Server id 14.3.487.0; Mon, 16 Mar 2020 12:31:46 +0800 From: Keqian Zhu To: Subject: [PATCH v2] migration/throttle: Add cpu-throttle-tailslow migration parameter Date: Mon, 16 Mar 2020 12:29:35 +0800 Message-ID: <20200316042935.28306-1-zhukeqian1@huawei.com> X-Mailer: git-send-email 2.19.1 MIME-Version: 1.0 X-Originating-IP: [10.175.104.212] X-CFilter-Loop: Reflected Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 45.249.212.191 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Juan Quintela , "Dr. David Alan Gilbert" , Markus Armbruster , qemu-arm@nongnu.org, wanghaibin.wang@huawei.com, Keqian Zhu Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Type: text/plain; charset="utf-8" At the tail stage of throttling, the Guest is very sensitive to CPU percentage while the @cpu-throttle-increment is excessive usually at tail stage. If this parameter is true, we will compute the ideal CPU percentage used by the Guest, which may exactly makes dirty rate to be dirty rate threshold. Then we will choose a smaller throttle increment between the one specified by @cpu-throttle-increment and the one generated by ideal CPU percentage. Therefore, it is compatible to traditional throttling, meanwhile the throttle increment won't be excessive at tail stage. This may make migration time longer, and is disabled by default. Signed-off-by: Keqian Zhu --- Cc: Juan Quintela Cc: "Dr. David Alan Gilbert" Cc: Eric Blake Cc: Markus Armbruster --- migration/migration.c | 13 ++++++++++++ migration/ram.c | 25 +++++++++++++++++----- monitor/hmp-cmds.c | 8 ++++++++ qapi/migration.json | 48 +++++++++++++++++++++++++++++++++++++++++++ 4 files changed, 89 insertions(+), 5 deletions(-) diff --git a/migration/migration.c b/migration/migration.c index c1d88ace7f..cc157cbf90 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -785,6 +785,8 @@ MigrationParameters *qmp_query_migrate_parameters(Error= **errp) params->cpu_throttle_initial =3D s->parameters.cpu_throttle_initial; params->has_cpu_throttle_increment =3D true; params->cpu_throttle_increment =3D s->parameters.cpu_throttle_incremen= t; + params->has_cpu_throttle_tailslow =3D true; + params->cpu_throttle_tailslow =3D s->parameters.cpu_throttle_tailslow; params->has_tls_creds =3D true; params->tls_creds =3D g_strdup(s->parameters.tls_creds); params->has_tls_hostname =3D true; @@ -1323,6 +1325,10 @@ static void migrate_params_test_apply(MigrateSetPara= meters *params, dest->cpu_throttle_increment =3D params->cpu_throttle_increment; } =20 + if (params->has_cpu_throttle_tailslow) { + dest->cpu_throttle_tailslow =3D params->cpu_throttle_tailslow; + } + if (params->has_tls_creds) { assert(params->tls_creds->type =3D=3D QTYPE_QSTRING); dest->tls_creds =3D g_strdup(params->tls_creds->u.s); @@ -1411,6 +1417,10 @@ static void migrate_params_apply(MigrateSetParameter= s *params, Error **errp) s->parameters.cpu_throttle_increment =3D params->cpu_throttle_incr= ement; } =20 + if (params->has_cpu_throttle_tailslow) { + s->parameters.cpu_throttle_tailslow =3D params->cpu_throttle_tails= low; + } + if (params->has_tls_creds) { g_free(s->parameters.tls_creds); assert(params->tls_creds->type =3D=3D QTYPE_QSTRING); @@ -3588,6 +3598,8 @@ static Property migration_properties[] =3D { DEFINE_PROP_UINT8("x-cpu-throttle-increment", MigrationState, parameters.cpu_throttle_increment, DEFAULT_MIGRATE_CPU_THROTTLE_INCREMENT), + DEFINE_PROP_BOOL("x-cpu-throttle-tailslow", MigrationState, + parameters.cpu_throttle_tailslow, false), DEFINE_PROP_SIZE("x-max-bandwidth", MigrationState, parameters.max_bandwidth, MAX_THROTTLE), DEFINE_PROP_UINT64("x-downtime-limit", MigrationState, @@ -3694,6 +3706,7 @@ static void migration_instance_init(Object *obj) params->has_throttle_trigger_threshold =3D true; params->has_cpu_throttle_initial =3D true; params->has_cpu_throttle_increment =3D true; + params->has_cpu_throttle_tailslow =3D true; params->has_max_bandwidth =3D true; params->has_downtime_limit =3D true; params->has_x_checkpoint_delay =3D true; diff --git a/migration/ram.c b/migration/ram.c index c12cfdbe26..4b74461306 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -616,20 +616,34 @@ static size_t save_page_header(RAMState *rs, QEMUFile= *f, RAMBlock *block, * able to complete migration. Some workloads dirty memory way too * fast and will not effectively converge, even with auto-converge. */ -static void mig_throttle_guest_down(void) +static void mig_throttle_guest_down(uint64_t bytes_dirty_period, + uint64_t bytes_dirty_threshold) { MigrationState *s =3D migrate_get_current(); uint64_t pct_initial =3D s->parameters.cpu_throttle_initial; - uint64_t pct_icrement =3D s->parameters.cpu_throttle_increment; + uint64_t pct_increment =3D s->parameters.cpu_throttle_increment; + bool pct_tailslow =3D s->parameters.cpu_throttle_tailslow; int pct_max =3D s->parameters.max_cpu_throttle; =20 + uint64_t throttle_now =3D cpu_throttle_get_percentage(); + uint64_t cpu_now, cpu_ideal, throttle_inc; + /* We have not started throttling yet. Let's start it. */ if (!cpu_throttle_active()) { cpu_throttle_set(pct_initial); } else { /* Throttling already on, just increase the rate */ - cpu_throttle_set(MIN(cpu_throttle_get_percentage() + pct_icrement, - pct_max)); + if (!pct_tailslow) { + throttle_inc =3D pct_increment; + } else { + /* Compute the ideal CPU percentage used by Guest, which + * may makes dirty rate to be dirty rate threshold. */ + cpu_now =3D 100 - throttle_now; + cpu_ideal =3D cpu_now * (bytes_dirty_threshold * 1.0 / + bytes_dirty_period); + throttle_inc =3D MIN(cpu_now - cpu_ideal, pct_increment); + } + cpu_throttle_set(MIN(throttle_now + throttle_inc, pct_max)); } } =20 @@ -919,7 +933,8 @@ static void migration_trigger_throttle(RAMState *rs) (++rs->dirty_rate_high_cnt >=3D 2)) { trace_migration_throttle(); rs->dirty_rate_high_cnt =3D 0; - mig_throttle_guest_down(); + mig_throttle_guest_down(bytes_dirty_period, + bytes_dirty_threshold); } } } diff --git a/monitor/hmp-cmds.c b/monitor/hmp-cmds.c index 58724031ea..29878632c8 100644 --- a/monitor/hmp-cmds.c +++ b/monitor/hmp-cmds.c @@ -419,6 +419,10 @@ void hmp_info_migrate_parameters(Monitor *mon, const Q= Dict *qdict) monitor_printf(mon, "%s: %u\n", MigrationParameter_str(MIGRATION_PARAMETER_CPU_THROTTLE_INCREM= ENT), params->cpu_throttle_increment); + assert(params->has_cpu_throttle_tailslow); + monitor_printf(mon, "%s: %s\n", + MigrationParameter_str(MIGRATION_PARAMETER_CPU_THROTTLE_TAILSL= OW), + params->cpu_throttle_tailslow ? "on" : "off"); assert(params->has_max_cpu_throttle); monitor_printf(mon, "%s: %u\n", MigrationParameter_str(MIGRATION_PARAMETER_MAX_CPU_THROTTLE), @@ -1269,6 +1273,10 @@ void hmp_migrate_set_parameter(Monitor *mon, const Q= Dict *qdict) p->has_cpu_throttle_increment =3D true; visit_type_int(v, param, &p->cpu_throttle_increment, &err); break; + case MIGRATION_PARAMETER_CPU_THROTTLE_TAILSLOW: + p->has_cpu_throttle_tailslow =3D true; + visit_type_bool(v, param, &p->cpu_throttle_tailslow, &err); + break; case MIGRATION_PARAMETER_MAX_CPU_THROTTLE: p->has_max_cpu_throttle =3D true; visit_type_int(v, param, &p->max_cpu_throttle, &err); diff --git a/qapi/migration.json b/qapi/migration.json index 0d1c0712ca..4bf82b74d1 100644 --- a/qapi/migration.json +++ b/qapi/migration.json @@ -552,6 +552,21 @@ # auto-converge detects that migration is not mak= ing # progress. The default value is 10. (Since 2.7) # +# @cpu-throttle-tailslow: Make CPU throttling slower at tail stage +# At the tail stage of throttling, the Guest is ve= ry +# sensitive to CPU percentage while the @cpu-throt= tle +# -increment is excessive usually at tail stage. +# If this parameter is true, we will compute the i= deal +# CPU percentage used by the Guest, which may exac= tly +# makes dirty rate to be dirty rate threshold. The= n we +# will choose a smaller throttle increment between= the +# one specified by @cpu-throttle-increment and the= one +# generated by ideal CPU percentage. +# Therefore, it is compatible to traditional throt= tling, +# meanwhile the throttle increment won't be excess= ive +# at tail stage. +# The default value is false. (Since 5.0) +# # @tls-creds: ID of the 'tls-creds' object that provides credentials for # establishing a TLS connection over the migration data channe= l. # On the outgoing side of the migration, the credentials must @@ -631,6 +646,7 @@ 'compress-level', 'compress-threads', 'decompress-threads', 'compress-wait-thread', 'throttle-trigger-threshold', 'cpu-throttle-initial', 'cpu-throttle-increment', + 'cpu-throttle-tailslow', 'tls-creds', 'tls-hostname', 'tls-authz', 'max-bandwidth', 'downtime-limit', 'x-checkpoint-delay', 'block-incremental', 'multifd-channels', @@ -676,6 +692,21 @@ # auto-converge detects that migration is not mak= ing # progress. The default value is 10. (Since 2.7) # +# @cpu-throttle-tailslow: Make CPU throttling slower at tail stage +# At the tail stage of throttling, the Guest is ve= ry +# sensitive to CPU percentage while the @cpu-throt= tle +# -increment is excessive usually at tail stage. +# If this parameter is true, we will compute the i= deal +# CPU percentage used by the Guest, which may exac= tly +# makes dirty rate to be dirty rate threshold. The= n we +# will choose a smaller throttle increment between= the +# one specified by @cpu-throttle-increment and the= one +# generated by ideal CPU percentage. +# Therefore, it is compatible to traditional throt= tling, +# meanwhile the throttle increment won't be excess= ive +# at tail stage. +# The default value is false. (Since 5.0) +# # @tls-creds: ID of the 'tls-creds' object that provides credentials # for establishing a TLS connection over the migration data # channel. On the outgoing side of the migration, the credenti= als @@ -763,6 +794,7 @@ '*throttle-trigger-threshold': 'int', '*cpu-throttle-initial': 'int', '*cpu-throttle-increment': 'int', + '*cpu-throttle-tailslow': 'bool', '*tls-creds': 'StrOrNull', '*tls-hostname': 'StrOrNull', '*tls-authz': 'StrOrNull', @@ -834,6 +866,21 @@ # auto-converge detects that migration is not mak= ing # progress. (Since 2.7) # +# @cpu-throttle-tailslow: Make CPU throttling slower at tail stage +# At the tail stage of throttling, the Guest is ve= ry +# sensitive to CPU percentage while the @cpu-throt= tle +# -increment is excessive usually at tail stage. +# If this parameter is true, we will compute the i= deal +# CPU percentage used by the Guest, which may exac= tly +# makes dirty rate to be dirty rate threshold. The= n we +# will choose a smaller throttle increment between= the +# one specified by @cpu-throttle-increment and the= one +# generated by ideal CPU percentage. +# Therefore, it is compatible to traditional throt= tling, +# meanwhile the throttle increment won't be excess= ive +# at tail stage. +# The default value is false. (Since 5.0) +# # @tls-creds: ID of the 'tls-creds' object that provides credentials # for establishing a TLS connection over the migration data # channel. On the outgoing side of the migration, the credenti= als @@ -921,6 +968,7 @@ '*throttle-trigger-threshold': 'uint8', '*cpu-throttle-initial': 'uint8', '*cpu-throttle-increment': 'uint8', + '*cpu-throttle-tailslow': 'bool', '*tls-creds': 'str', '*tls-hostname': 'str', '*tls-authz': 'str', --=20 2.19.1