From nobody Tue Feb 10 04:17:48 2026 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1534940085032662.8255726122436; Wed, 22 Aug 2018 05:14:45 -0700 (PDT) Received: from localhost ([::1]:58710 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fsS27-0002pm-Ka for importer@patchew.org; Wed, 22 Aug 2018 08:14:43 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:39048) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fsRpL-0006qu-9n for qemu-devel@nongnu.org; Wed, 22 Aug 2018 08:01:37 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fsRpJ-0000Ar-Mv for qemu-devel@nongnu.org; Wed, 22 Aug 2018 08:01:31 -0400 Received: from mx3-rdu2.redhat.com ([66.187.233.73]:41932 helo=mx1.redhat.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1fsRpJ-0000AK-8z for qemu-devel@nongnu.org; Wed, 22 Aug 2018 08:01:29 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.rdu2.redhat.com [10.11.54.5]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id E4B9B4021FC2; Wed, 22 Aug 2018 12:01:28 +0000 (UTC) Received: from secure.mitica (ovpn-116-35.ams2.redhat.com [10.36.116.35]) by smtp.corp.redhat.com (Postfix) with ESMTP id A70BFA9E6F; Wed, 22 Aug 2018 12:01:27 +0000 (UTC) From: Juan Quintela To: qemu-devel@nongnu.org Date: Wed, 22 Aug 2018 14:00:45 +0200 Message-Id: <20180822120050.12694-16-quintela@redhat.com> In-Reply-To: <20180822120050.12694-1-quintela@redhat.com> References: <20180822120050.12694-1-quintela@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.11.54.5 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.7]); Wed, 22 Aug 2018 12:01:28 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.7]); Wed, 22 Aug 2018 12:01:28 +0000 (UTC) for IP:'10.11.54.5' DOMAIN:'int-mx05.intmail.prod.int.rdu2.redhat.com' HELO:'smtp.corp.redhat.com' FROM:'quintela@redhat.com' RCPT:'' X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 66.187.233.73 Subject: [Qemu-devel] [PULL 15/20] migration: do not wait for free thread X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: lvivier@redhat.com, Xiao Guangrong , dgilbert@redhat.com, peterx@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RDMRC_1 RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: Xiao Guangrong Instead of putting the main thread to sleep state to wait for free compression thread, we can directly post it out as normal page that reduces the latency and uses CPUs more efficiently A parameter, compress-wait-thread, is introduced, it can be enabled if the user really wants the old behavior Reviewed-by: Peter Xu Signed-off-by: Xiao Guangrong Reviewed-by: Juan Quintela Signed-off-by: Juan Quintela --- hmp.c | 8 ++++++++ migration/migration.c | 21 ++++++++++++++++++++ migration/migration.h | 1 + migration/ram.c | 45 +++++++++++++++++++++++++------------------ qapi/migration.json | 18 +++++++++++++++++ 5 files changed, 74 insertions(+), 19 deletions(-) diff --git a/hmp.c b/hmp.c index c38e8b1f78..9147037951 100644 --- a/hmp.c +++ b/hmp.c @@ -327,6 +327,10 @@ void hmp_info_migrate_parameters(Monitor *mon, const Q= Dict *qdict) monitor_printf(mon, "%s: %u\n", MigrationParameter_str(MIGRATION_PARAMETER_COMPRESS_THREADS), params->compress_threads); + assert(params->has_compress_wait_thread); + monitor_printf(mon, "%s: %s\n", + MigrationParameter_str(MIGRATION_PARAMETER_COMPRESS_WAIT_THREA= D), + params->compress_wait_thread ? "on" : "off"); assert(params->has_decompress_threads); monitor_printf(mon, "%s: %u\n", MigrationParameter_str(MIGRATION_PARAMETER_DECOMPRESS_THREADS), @@ -1627,6 +1631,10 @@ void hmp_migrate_set_parameter(Monitor *mon, const Q= Dict *qdict) p->has_compress_threads =3D true; visit_type_int(v, param, &p->compress_threads, &err); break; + case MIGRATION_PARAMETER_COMPRESS_WAIT_THREAD: + p->has_compress_wait_thread =3D true; + visit_type_bool(v, param, &p->compress_wait_thread, &err); + break; case MIGRATION_PARAMETER_DECOMPRESS_THREADS: p->has_decompress_threads =3D true; visit_type_int(v, param, &p->decompress_threads, &err); diff --git a/migration/migration.c b/migration/migration.c index 4d76bee0da..4b316ec343 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -673,6 +673,8 @@ MigrationParameters *qmp_query_migrate_parameters(Error= **errp) params->compress_level =3D s->parameters.compress_level; params->has_compress_threads =3D true; params->compress_threads =3D s->parameters.compress_threads; + params->has_compress_wait_thread =3D true; + params->compress_wait_thread =3D s->parameters.compress_wait_thread; params->has_decompress_threads =3D true; params->decompress_threads =3D s->parameters.decompress_threads; params->has_cpu_throttle_initial =3D true; @@ -1074,6 +1076,10 @@ static void migrate_params_test_apply(MigrateSetPara= meters *params, dest->compress_threads =3D params->compress_threads; } =20 + if (params->has_compress_wait_thread) { + dest->compress_wait_thread =3D params->compress_wait_thread; + } + if (params->has_decompress_threads) { dest->decompress_threads =3D params->decompress_threads; } @@ -1142,6 +1148,10 @@ static void migrate_params_apply(MigrateSetParameter= s *params, Error **errp) s->parameters.compress_threads =3D params->compress_threads; } =20 + if (params->has_compress_wait_thread) { + s->parameters.compress_wait_thread =3D params->compress_wait_threa= d; + } + if (params->has_decompress_threads) { s->parameters.decompress_threads =3D params->decompress_threads; } @@ -1890,6 +1900,15 @@ int migrate_compress_threads(void) return s->parameters.compress_threads; } =20 +int migrate_compress_wait_thread(void) +{ + MigrationState *s; + + s =3D migrate_get_current(); + + return s->parameters.compress_wait_thread; +} + int migrate_decompress_threads(void) { MigrationState *s; @@ -3151,6 +3170,8 @@ static Property migration_properties[] =3D { DEFINE_PROP_UINT8("x-compress-threads", MigrationState, parameters.compress_threads, DEFAULT_MIGRATE_COMPRESS_THREAD_COUNT), + DEFINE_PROP_BOOL("x-compress-wait-thread", MigrationState, + parameters.compress_wait_thread, true), DEFINE_PROP_UINT8("x-decompress-threads", MigrationState, parameters.decompress_threads, DEFAULT_MIGRATE_DECOMPRESS_THREAD_COUNT), diff --git a/migration/migration.h b/migration/migration.h index eb0c04a7b4..f7813f8261 100644 --- a/migration/migration.h +++ b/migration/migration.h @@ -272,6 +272,7 @@ bool migrate_use_return_path(void); bool migrate_use_compression(void); int migrate_compress_level(void); int migrate_compress_threads(void); +int migrate_compress_wait_thread(void); int migrate_decompress_threads(void); bool migrate_use_events(void); bool migrate_postcopy_blocktime(void); diff --git a/migration/ram.c b/migration/ram.c index 2f9e8bd7e0..498834bdbe 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -1896,30 +1896,34 @@ static int compress_page_with_multi_thread(RAMState= *rs, RAMBlock *block, ram_addr_t offset) { int idx, thread_count, bytes_xmit =3D -1, pages =3D -1; + bool wait =3D migrate_compress_wait_thread(); =20 thread_count =3D migrate_compress_threads(); qemu_mutex_lock(&comp_done_lock); - while (true) { - for (idx =3D 0; idx < thread_count; idx++) { - if (comp_param[idx].done) { - comp_param[idx].done =3D false; - bytes_xmit =3D qemu_put_qemu_file(rs->f, comp_param[idx].f= ile); - qemu_mutex_lock(&comp_param[idx].mutex); - set_compress_params(&comp_param[idx], block, offset); - qemu_cond_signal(&comp_param[idx].cond); - qemu_mutex_unlock(&comp_param[idx].mutex); - pages =3D 1; - ram_counters.normal++; - ram_counters.transferred +=3D bytes_xmit; - break; - } - } - if (pages > 0) { +retry: + for (idx =3D 0; idx < thread_count; idx++) { + if (comp_param[idx].done) { + comp_param[idx].done =3D false; + bytes_xmit =3D qemu_put_qemu_file(rs->f, comp_param[idx].file); + qemu_mutex_lock(&comp_param[idx].mutex); + set_compress_params(&comp_param[idx], block, offset); + qemu_cond_signal(&comp_param[idx].cond); + qemu_mutex_unlock(&comp_param[idx].mutex); + pages =3D 1; + ram_counters.normal++; + ram_counters.transferred +=3D bytes_xmit; break; - } else { - qemu_cond_wait(&comp_done_cond, &comp_done_lock); } } + + /* + * wait for the free thread if the user specifies 'compress-wait-threa= d', + * otherwise we will post the page out in the main thread as normal pa= ge. + */ + if (pages < 0 && wait) { + qemu_cond_wait(&comp_done_cond, &comp_done_lock); + goto retry; + } qemu_mutex_unlock(&comp_done_lock); =20 return pages; @@ -2233,7 +2237,10 @@ static int ram_save_target_page(RAMState *rs, PageSe= archStatus *pss, * CPU resource. */ if (block =3D=3D rs->last_sent_block && save_page_use_compression(rs))= { - return compress_page_with_multi_thread(rs, block, offset); + res =3D compress_page_with_multi_thread(rs, block, offset); + if (res > 0) { + return res; + } } else if (migrate_use_multifd()) { return ram_save_multifd_page(rs, block, offset); } diff --git a/qapi/migration.json b/qapi/migration.json index cabe234c36..f62d3f9a4b 100644 --- a/qapi/migration.json +++ b/qapi/migration.json @@ -461,6 +461,11 @@ # @compress-threads: Set compression thread count to be used in live migra= tion, # the compression thread count is an integer between 1 and 255. # +# @compress-wait-thread: Controls behavior when all compression threads are +# currently busy. If true (default), wait for a free +# compression thread to become available; otherwise, +# send the page uncompressed. (Since 3.1) +# # @decompress-threads: Set decompression thread count to be used in live # migration, the decompression thread count is an integer between= 1 # and 255. Usually, decompression is at least 4 times as fast as @@ -529,6 +534,7 @@ ## { 'enum': 'MigrationParameter', 'data': ['compress-level', 'compress-threads', 'decompress-threads', + 'compress-wait-thread', 'cpu-throttle-initial', 'cpu-throttle-increment', 'tls-creds', 'tls-hostname', 'max-bandwidth', 'downtime-limit', 'x-checkpoint-delay', 'block-incremental', @@ -543,6 +549,11 @@ # # @compress-threads: compression thread count # +# @compress-wait-thread: Controls behavior when all compression threads are +# currently busy. If true (default), wait for a free +# compression thread to become available; otherwise, +# send the page uncompressed. (Since 3.1) +# # @decompress-threads: decompression thread count # # @cpu-throttle-initial: Initial percentage of time guest cpus are @@ -617,6 +628,7 @@ { 'struct': 'MigrateSetParameters', 'data': { '*compress-level': 'int', '*compress-threads': 'int', + '*compress-wait-thread': 'bool', '*decompress-threads': 'int', '*cpu-throttle-initial': 'int', '*cpu-throttle-increment': 'int', @@ -657,6 +669,11 @@ # # @compress-threads: compression thread count # +# @compress-wait-thread: Controls behavior when all compression threads are +# currently busy. If true (default), wait for a free +# compression thread to become available; otherwise, +# send the page uncompressed. (Since 3.1) +# # @decompress-threads: decompression thread count # # @cpu-throttle-initial: Initial percentage of time guest cpus are @@ -727,6 +744,7 @@ { 'struct': 'MigrationParameters', 'data': { '*compress-level': 'uint8', '*compress-threads': 'uint8', + '*compress-wait-thread': 'bool', '*decompress-threads': 'uint8', '*cpu-throttle-initial': 'uint8', '*cpu-throttle-increment': 'uint8', --=20 2.17.1