From nobody Thu May 2 23:10:44 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=gmail.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1532002859227461.3669435883412; Thu, 19 Jul 2018 05:20:59 -0700 (PDT) Received: from localhost ([::1]:42088 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fg7vM-0005cu-NA for importer@patchew.org; Thu, 19 Jul 2018 08:20:48 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:34931) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fg7qV-00021t-8k for qemu-devel@nongnu.org; Thu, 19 Jul 2018 08:15:48 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fg7qT-0005MQ-K2 for qemu-devel@nongnu.org; Thu, 19 Jul 2018 08:15:47 -0400 Received: from mail-pg1-x533.google.com ([2607:f8b0:4864:20::533]:45094) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1fg7qT-0005Ly-B6 for qemu-devel@nongnu.org; Thu, 19 Jul 2018 08:15:45 -0400 Received: by mail-pg1-x533.google.com with SMTP id f1-v6so3576531pgq.12 for ; Thu, 19 Jul 2018 05:15:45 -0700 (PDT) Received: from localhost.localdomain ([203.205.141.37]) by smtp.gmail.com with ESMTPSA id h190-v6sm21270456pge.85.2018.07.19.05.15.41 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Thu, 19 Jul 2018 05:15:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=wIkMXGa4+XsJUNU1At1Pyq6kscGlTfiG6YmWlcF9meI=; b=CC7Z4aOz0fuJQf2gu3H7BL5L8N/cM1e5suY+FinHOWb5LzYAhpDK+vlQ+piSFRxGbd T7BHnn2sj3J22NFTcI0IdZTJM9nN+EKAiX160jDEvJLvJve3IQsR/ZiXYDtuwi0hpYBE RTiDgg1N+cC15AikusLxwJEwUQ56ncTDS1zSNwBAO5Ae2omddbDIuaRzdBcZ4L2RFmKL lclzztRrR959CrOspgHhkDHQ0H5nbwY665sJ3vMN7scriG6HR84KJHSmyoDGK7M05Rul s3OGEVGMvSyBiLNl5IPPM+PEXfH78oAoI9RXXKYDEUsDjkZSdXxAXPeaa4/6lriZiWbp x/Qw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=wIkMXGa4+XsJUNU1At1Pyq6kscGlTfiG6YmWlcF9meI=; b=NDfextHSTE/vPlqhKwuLHTNklvz9vYVeAuGqesy7Loz3H5CklAZGnoxopTLnV5jD2K 4FPY8+x2e/mtMw55grbfCzxD3tutdZAFUCnbcPasu0io+LrD86Zy0a78j0laVJ2V9Kda FpFxapNFADDcpNEK+bL3fKgT3epdXU80Lt74UFVPZ4nYaAeXQE8+PJ+VqOWvb0Sl06rJ 1CH9O2LhRGBx5Ap+XxPWikXu1GGD+O/uOkkSWwnBQZrPie9yGP+n2+dTfjBwjXE/tYFt gTzu2JEKXuSr6eKn1IogYadfI6CKISs8k65G1hvRhUJEUVV8hxHcx/Nu760QyaFJIeOB mkGQ== X-Gm-Message-State: AOUpUlG1cCzmdxYmdTG03hTdc0642mrh8Jnry9RbYOkxbo21b5NgjbvV +PACV9r7zxXXz+FWePrjYl0= X-Google-Smtp-Source: AAOMgpd0+qavtrd2GJ6sJHWzmU145so4kN1JABA7pxwBVHClRqa+UlvO/6zMj/HM1BOMlIbwyroe9g== X-Received: by 2002:a62:e30c:: with SMTP id g12-v6mr9393923pfh.25.1532002544474; Thu, 19 Jul 2018 05:15:44 -0700 (PDT) From: guangrong.xiao@gmail.com X-Google-Original-From: xiaoguangrong@tencent.com To: pbonzini@redhat.com, mst@redhat.com, mtosatti@redhat.com Date: Thu, 19 Jul 2018 20:15:13 +0800 Message-Id: <20180719121520.30026-2-xiaoguangrong@tencent.com> X-Mailer: git-send-email 2.14.4 In-Reply-To: <20180719121520.30026-1-xiaoguangrong@tencent.com> References: <20180719121520.30026-1-xiaoguangrong@tencent.com> X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2607:f8b0:4864:20::533 Subject: [Qemu-devel] [PATCH v2 1/8] migration: do not wait for free thread X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kvm@vger.kernel.org, Xiao Guangrong , qemu-devel@nongnu.org, peterx@redhat.com, dgilbert@redhat.com, wei.w.wang@intel.com, jiang.biao2@zte.com.cn Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZohoMail: RDKM_2 RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: Xiao Guangrong Instead of putting the main thread to sleep state to wait for free compression thread, we can directly post it out as normal page that reduces the latency and uses CPUs more efficiently A parameter, compress-wait-thread, is introduced, it can be enabled if the user really wants the old behavior Signed-off-by: Xiao Guangrong --- hmp.c | 8 ++++++++ migration/migration.c | 21 +++++++++++++++++++++ migration/migration.h | 1 + migration/ram.c | 45 ++++++++++++++++++++++++++------------------- qapi/migration.json | 23 ++++++++++++++++++----- 5 files changed, 74 insertions(+), 24 deletions(-) diff --git a/hmp.c b/hmp.c index 2aafb50e8e..47d36e3ccf 100644 --- a/hmp.c +++ b/hmp.c @@ -327,6 +327,10 @@ void hmp_info_migrate_parameters(Monitor *mon, const Q= Dict *qdict) monitor_printf(mon, "%s: %u\n", MigrationParameter_str(MIGRATION_PARAMETER_COMPRESS_THREADS), params->compress_threads); + assert(params->has_compress_wait_thread); + monitor_printf(mon, "%s: %s\n", + MigrationParameter_str(MIGRATION_PARAMETER_COMPRESS_WAIT_THREA= D), + params->compress_wait_thread ? "on" : "off"); assert(params->has_decompress_threads); monitor_printf(mon, "%s: %u\n", MigrationParameter_str(MIGRATION_PARAMETER_DECOMPRESS_THREADS), @@ -1623,6 +1627,10 @@ void hmp_migrate_set_parameter(Monitor *mon, const Q= Dict *qdict) p->has_compress_threads =3D true; visit_type_int(v, param, &p->compress_threads, &err); break; + case MIGRATION_PARAMETER_COMPRESS_WAIT_THREAD: + p->has_compress_wait_thread =3D true; + visit_type_bool(v, param, &p->compress_wait_thread, &err); + break; case MIGRATION_PARAMETER_DECOMPRESS_THREADS: p->has_decompress_threads =3D true; visit_type_int(v, param, &p->decompress_threads, &err); diff --git a/migration/migration.c b/migration/migration.c index 8d56d56930..0af75465b3 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -671,6 +671,8 @@ MigrationParameters *qmp_query_migrate_parameters(Error= **errp) params->compress_level =3D s->parameters.compress_level; params->has_compress_threads =3D true; params->compress_threads =3D s->parameters.compress_threads; + params->has_compress_wait_thread =3D true; + params->compress_wait_thread =3D s->parameters.compress_wait_thread; params->has_decompress_threads =3D true; params->decompress_threads =3D s->parameters.decompress_threads; params->has_cpu_throttle_initial =3D true; @@ -1061,6 +1063,10 @@ static void migrate_params_test_apply(MigrateSetPara= meters *params, dest->compress_threads =3D params->compress_threads; } =20 + if (params->has_compress_wait_thread) { + dest->compress_wait_thread =3D params->compress_wait_thread; + } + if (params->has_decompress_threads) { dest->decompress_threads =3D params->decompress_threads; } @@ -1126,6 +1132,10 @@ static void migrate_params_apply(MigrateSetParameter= s *params, Error **errp) s->parameters.compress_threads =3D params->compress_threads; } =20 + if (params->has_compress_wait_thread) { + s->parameters.compress_wait_thread =3D params->compress_wait_threa= d; + } + if (params->has_decompress_threads) { s->parameters.decompress_threads =3D params->decompress_threads; } @@ -1852,6 +1862,15 @@ int migrate_compress_threads(void) return s->parameters.compress_threads; } =20 +int migrate_compress_wait_thread(void) +{ + MigrationState *s; + + s =3D migrate_get_current(); + + return s->parameters.compress_wait_thread; +} + int migrate_decompress_threads(void) { MigrationState *s; @@ -3113,6 +3132,8 @@ static Property migration_properties[] =3D { DEFINE_PROP_UINT8("x-compress-threads", MigrationState, parameters.compress_threads, DEFAULT_MIGRATE_COMPRESS_THREAD_COUNT), + DEFINE_PROP_BOOL("x-compress-wait-thread", MigrationState, + parameters.compress_wait_thread, false), DEFINE_PROP_UINT8("x-decompress-threads", MigrationState, parameters.decompress_threads, DEFAULT_MIGRATE_DECOMPRESS_THREAD_COUNT), diff --git a/migration/migration.h b/migration/migration.h index 64a7b33735..a46b9e6c8d 100644 --- a/migration/migration.h +++ b/migration/migration.h @@ -271,6 +271,7 @@ bool migrate_use_return_path(void); bool migrate_use_compression(void); int migrate_compress_level(void); int migrate_compress_threads(void); +int migrate_compress_wait_thread(void); int migrate_decompress_threads(void); bool migrate_use_events(void); bool migrate_postcopy_blocktime(void); diff --git a/migration/ram.c b/migration/ram.c index 52dd678092..0ad234c692 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -1889,30 +1889,34 @@ static int compress_page_with_multi_thread(RAMState= *rs, RAMBlock *block, ram_addr_t offset) { int idx, thread_count, bytes_xmit =3D -1, pages =3D -1; + bool wait =3D migrate_compress_wait_thread(); =20 thread_count =3D migrate_compress_threads(); qemu_mutex_lock(&comp_done_lock); - while (true) { - for (idx =3D 0; idx < thread_count; idx++) { - if (comp_param[idx].done) { - comp_param[idx].done =3D false; - bytes_xmit =3D qemu_put_qemu_file(rs->f, comp_param[idx].f= ile); - qemu_mutex_lock(&comp_param[idx].mutex); - set_compress_params(&comp_param[idx], block, offset); - qemu_cond_signal(&comp_param[idx].cond); - qemu_mutex_unlock(&comp_param[idx].mutex); - pages =3D 1; - ram_counters.normal++; - ram_counters.transferred +=3D bytes_xmit; - break; - } - } - if (pages > 0) { +retry: + for (idx =3D 0; idx < thread_count; idx++) { + if (comp_param[idx].done) { + comp_param[idx].done =3D false; + bytes_xmit =3D qemu_put_qemu_file(rs->f, comp_param[idx].file); + qemu_mutex_lock(&comp_param[idx].mutex); + set_compress_params(&comp_param[idx], block, offset); + qemu_cond_signal(&comp_param[idx].cond); + qemu_mutex_unlock(&comp_param[idx].mutex); + pages =3D 1; + ram_counters.normal++; + ram_counters.transferred +=3D bytes_xmit; break; - } else { - qemu_cond_wait(&comp_done_cond, &comp_done_lock); } } + + /* + * if there is no thread is free to compress the data and the user + * really expects the slowdown, wait it. + */ + if (pages < 0 && wait) { + qemu_cond_wait(&comp_done_cond, &comp_done_lock); + goto retry; + } qemu_mutex_unlock(&comp_done_lock); =20 return pages; @@ -2226,7 +2230,10 @@ static int ram_save_target_page(RAMState *rs, PageSe= archStatus *pss, * CPU resource. */ if (block =3D=3D rs->last_sent_block && save_page_use_compression(rs))= { - return compress_page_with_multi_thread(rs, block, offset); + res =3D compress_page_with_multi_thread(rs, block, offset); + if (res > 0) { + return res; + } } else if (migrate_use_multifd()) { return ram_save_multifd_page(rs, block, offset); } diff --git a/qapi/migration.json b/qapi/migration.json index 186e8a7303..b4f394844b 100644 --- a/qapi/migration.json +++ b/qapi/migration.json @@ -462,6 +462,11 @@ # @compress-threads: Set compression thread count to be used in live migra= tion, # the compression thread count is an integer between 1 and 255. # +# @compress-wait-thread: Wait if no thread is free to compress the memory = page +# if it's enabled, otherwise, the page will be posted out immedia= tely +# in the main thread without compression. It's off on default. +# (Since: 3.0) +# # @decompress-threads: Set decompression thread count to be used in live # migration, the decompression thread count is an integer between= 1 # and 255. Usually, decompression is at least 4 times as fast as @@ -526,11 +531,11 @@ # Since: 2.4 ## { 'enum': 'MigrationParameter', - 'data': ['compress-level', 'compress-threads', 'decompress-threads', - 'cpu-throttle-initial', 'cpu-throttle-increment', - 'tls-creds', 'tls-hostname', 'max-bandwidth', - 'downtime-limit', 'x-checkpoint-delay', 'block-incremental', - 'x-multifd-channels', 'x-multifd-page-count', + 'data': ['compress-level', 'compress-threads', 'compress-wait-thread', + 'decompress-threads', 'cpu-throttle-initial', + 'cpu-throttle-increment', 'tls-creds', 'tls-hostname', + 'max-bandwidth', 'downtime-limit', 'x-checkpoint-delay', + 'block-incremental', 'x-multifd-channels', 'x-multifd-page-coun= t', 'xbzrle-cache-size', 'max-postcopy-bandwidth' ] } =20 ## @@ -540,6 +545,9 @@ # # @compress-threads: compression thread count # +# @compress-wait-thread: Wait if no thread is free to compress the memory = page +# (Since: 3.0) +# # @decompress-threads: decompression thread count # # @cpu-throttle-initial: Initial percentage of time guest cpus are @@ -610,6 +618,7 @@ { 'struct': 'MigrateSetParameters', 'data': { '*compress-level': 'int', '*compress-threads': 'int', + '*compress-wait-thread': 'bool', '*decompress-threads': 'int', '*cpu-throttle-initial': 'int', '*cpu-throttle-increment': 'int', @@ -649,6 +658,9 @@ # # @compress-threads: compression thread count # +# @compress-wait-thread: Wait if no thread is free to compress the memory = page +# (Since: 3.0) +# # @decompress-threads: decompression thread count # # @cpu-throttle-initial: Initial percentage of time guest cpus are @@ -714,6 +726,7 @@ { 'struct': 'MigrationParameters', 'data': { '*compress-level': 'uint8', '*compress-threads': 'uint8', + '*compress-wait-thread': 'bool', '*decompress-threads': 'uint8', '*cpu-throttle-initial': 'uint8', '*cpu-throttle-increment': 'uint8', --=20 2.14.4 From nobody Thu May 2 23:10:44 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=gmail.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1532002690194766.7620890288248; Thu, 19 Jul 2018 05:18:10 -0700 (PDT) Received: from localhost ([::1]:42071 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fg7sn-0003Ps-64 for importer@patchew.org; Thu, 19 Jul 2018 08:18:09 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:34952) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fg7qX-00023o-KI for qemu-devel@nongnu.org; Thu, 19 Jul 2018 08:15:50 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fg7qW-0005Nx-OX for qemu-devel@nongnu.org; Thu, 19 Jul 2018 08:15:49 -0400 Received: from mail-pg1-x52c.google.com ([2607:f8b0:4864:20::52c]:41514) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1fg7qW-0005Nj-IJ for qemu-devel@nongnu.org; Thu, 19 Jul 2018 08:15:48 -0400 Received: by mail-pg1-x52c.google.com with SMTP id z8-v6so3578706pgu.8 for ; Thu, 19 Jul 2018 05:15:48 -0700 (PDT) Received: from localhost.localdomain ([203.205.141.37]) by smtp.gmail.com with ESMTPSA id h190-v6sm21270456pge.85.2018.07.19.05.15.44 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Thu, 19 Jul 2018 05:15:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=6EwPktPxAlxHbR8/kguT2vBMzx8nbLUekWgCRsZWX3A=; b=uqk2pbm7R6PsXbzR8gtS6S+5HyYPNR2+6/BeAAlt7QGv8r/qcBhZYPmMVcDomraC5l PTZPKcBfNt5FveMjumCQXd49mL/O1auFP8OyKIoTIkNSKNJEbL0IGzi23dGkbGGkI+l4 5GlDvqexyoHoG1//vWmYuizVt8UKNE6Prf8u94rRbG9EI/B/K24YrKBMrpuq0ov4mNmZ YhkLKh1NsNhBk01UJpSqkXkwGVsMIVmFYyDFwAWdB7X/T0cw+TEjjp0VkVOubgcR3Vj1 0mMJhvxQSkCbySyiE+uA20ItKD0n8SPk1dd3HLX/XxB3CAkE6diCIypwjjYKjWnuI29E 4fgg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=6EwPktPxAlxHbR8/kguT2vBMzx8nbLUekWgCRsZWX3A=; b=b4JMA3sp7qP0rvkjyGdMCrM/gE5x46+yaUjFfLWOYNN+rYjAZ9q4/8LSQT1et4FeBR nNn/LVE3NP1yfSDGEQ6CCRVK9NNGtNLfWiJB2QrMW8viL1QmssugvVnYUQebMBvR45Dl F2GWrhCk+QFHeM6KUMZzdRR8U5EgpbVC0su8bsc/PEzaeOuwzxyQhI2U5YHlrceinCrb s+1uvzHppPX92guhIzeSciye9BfTQ6tRcTi7WWe18Cs6NtQ1RwwZ1VvY66USWV1FLr9o yjFnBnHmbxuLSrEuvqnkCJE2FY4Rl9ZfjxGvszWdwZFjefnMwZoAJr22VaFBDWYLPKLY m8vQ== X-Gm-Message-State: AOUpUlEmCpX0G+mbyA+SJEO8QchZov7Iq8q4UnOSLXP88gmFz0u0avet FamfVODpsJ9iG3eqyaCpyg8= X-Google-Smtp-Source: AAOMgpfDpeBizBaP98BTrF73kXXo74k+p+FdFSoVedHoLARPznptBjV7xg6wTdIibS8NvXlhM20WTQ== X-Received: by 2002:a65:460e:: with SMTP id v14-v6mr9565596pgq.177.1532002547864; Thu, 19 Jul 2018 05:15:47 -0700 (PDT) From: guangrong.xiao@gmail.com X-Google-Original-From: xiaoguangrong@tencent.com To: pbonzini@redhat.com, mst@redhat.com, mtosatti@redhat.com Date: Thu, 19 Jul 2018 20:15:14 +0800 Message-Id: <20180719121520.30026-3-xiaoguangrong@tencent.com> X-Mailer: git-send-email 2.14.4 In-Reply-To: <20180719121520.30026-1-xiaoguangrong@tencent.com> References: <20180719121520.30026-1-xiaoguangrong@tencent.com> X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2607:f8b0:4864:20::52c Subject: [Qemu-devel] [PATCH v2 2/8] migration: fix counting normal page for compression X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kvm@vger.kernel.org, Xiao Guangrong , qemu-devel@nongnu.org, peterx@redhat.com, dgilbert@redhat.com, wei.w.wang@intel.com, jiang.biao2@zte.com.cn Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZohoMail: RDKM_2 RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: Xiao Guangrong The compressed page is not normal page Signed-off-by: Xiao Guangrong Reviewed-by: Peter Xu --- migration/ram.c | 1 - 1 file changed, 1 deletion(-) diff --git a/migration/ram.c b/migration/ram.c index 0ad234c692..1b016e048d 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -1903,7 +1903,6 @@ retry: qemu_cond_signal(&comp_param[idx].cond); qemu_mutex_unlock(&comp_param[idx].mutex); pages =3D 1; - ram_counters.normal++; ram_counters.transferred +=3D bytes_xmit; break; } --=20 2.14.4 From nobody Thu May 2 23:10:44 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=gmail.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1532002792909263.280925834261; Thu, 19 Jul 2018 05:19:52 -0700 (PDT) Received: from localhost ([::1]:42082 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fg7uR-0004Xc-OJ for importer@patchew.org; Thu, 19 Jul 2018 08:19:51 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:34999) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fg7qd-00028j-KK for qemu-devel@nongnu.org; Thu, 19 Jul 2018 08:15:57 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fg7qa-0005Po-EN for qemu-devel@nongnu.org; Thu, 19 Jul 2018 08:15:55 -0400 Received: from mail-pg1-x544.google.com ([2607:f8b0:4864:20::544]:36632) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1fg7qa-0005PI-5k for qemu-devel@nongnu.org; Thu, 19 Jul 2018 08:15:52 -0400 Received: by mail-pg1-x544.google.com with SMTP id s7-v6so151993pgv.3 for ; Thu, 19 Jul 2018 05:15:52 -0700 (PDT) Received: from localhost.localdomain ([203.205.141.37]) by smtp.gmail.com with ESMTPSA id h190-v6sm21270456pge.85.2018.07.19.05.15.48 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Thu, 19 Jul 2018 05:15:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=mReR75DCk+TpnJZj+H9n61iCvECX4EIvqFABg0+KinE=; b=XSmoFszewfAkYZywSny2RwUb8fDAxtwANQWCawHJ7ohzlO7xgp60hJnxyLaB63dCjb Du8PJQwgf1ICPnQ7Mr0tmzWoqT7rqiGY4RfFVhT7wH9h1Qhw/3S5gJCae7a1r5r9oaEx R+atUHGIG8KS+fGeg8YpT6UNpb/LdavlnXlbHd/ZcfVr/VSUdmrVmC2asSCRo6CIbB6N gtUpLd1OenvkGizurHLZn4sKMAa8bqkTR/aIb4WgNtVVfF5Mw96Emnd4s2Fjzo9ygkrY 6bL9MWQQ2g09vP8TqmUSvjI8dFhmg97jiWvp8bK4E/inrI75uaMAHckQhgcFJiTay+ad o+Sg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=mReR75DCk+TpnJZj+H9n61iCvECX4EIvqFABg0+KinE=; b=JGpt5l7V7KOrAPGGIvArbimY/jufb980ywl42971rI1E5JvoJ5NrsYg7fhaUVRQp2F FYci3ATuWW4PPGz/9f7+Gzmf6dU3jp44/vOYnlVNpzORNc2/qfMQjHkTvZefvSKmlVKD +bNL67m/q5AnRVHGDqFdoXVPL1lTi4yIlPsaZ7N3L9jWHiwWNL8jvdBVOURErl0zSEQO KiqgrM45BK6UiTu+BxFkgrW7sOduq8dSTTfomGe14GZaqkhfDN/G8omcnLUB7u3Uumgx fSca7ginZk87UGPy/UNfSegNv5kG7U7CB046WNQemJw5CPSwX/AWqEZOfljIVdqkKQl6 21dQ== X-Gm-Message-State: AOUpUlF2daZNawamRa9MjA/jYWv+0B/Bk4Yuw9YGzygafoe3UAnQYz/0 l2wEn8jFHeJSw6ztmakj/ag= X-Google-Smtp-Source: AAOMgpfaLcuRrkn+YaNf9iL/fTYYvt3TW78eJ6T/SAqDSiQ5RI9fLY0wZY7JUQ/xxFvthPA2jSwqyg== X-Received: by 2002:a63:c114:: with SMTP id w20-v6mr9723873pgf.234.1532002551301; Thu, 19 Jul 2018 05:15:51 -0700 (PDT) From: guangrong.xiao@gmail.com X-Google-Original-From: xiaoguangrong@tencent.com To: pbonzini@redhat.com, mst@redhat.com, mtosatti@redhat.com Date: Thu, 19 Jul 2018 20:15:15 +0800 Message-Id: <20180719121520.30026-4-xiaoguangrong@tencent.com> X-Mailer: git-send-email 2.14.4 In-Reply-To: <20180719121520.30026-1-xiaoguangrong@tencent.com> References: <20180719121520.30026-1-xiaoguangrong@tencent.com> X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2607:f8b0:4864:20::544 Subject: [Qemu-devel] [PATCH v2 3/8] migration: show the statistics of compression X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kvm@vger.kernel.org, Xiao Guangrong , qemu-devel@nongnu.org, peterx@redhat.com, dgilbert@redhat.com, wei.w.wang@intel.com, jiang.biao2@zte.com.cn Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZohoMail: RDKM_2 RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: Xiao Guangrong Currently, it includes: pages: amount of pages compressed and transferred to the target VM busy: amount of count that no free thread to compress data busy-rate: rate of thread busy reduced-size: amount of bytes reduced by compression compression-rate: rate of compressed size Signed-off-by: Xiao Guangrong --- hmp.c | 13 +++++++++++++ migration/migration.c | 11 +++++++++++ migration/ram.c | 37 +++++++++++++++++++++++++++++++++++++ migration/ram.h | 1 + qapi/migration.json | 26 +++++++++++++++++++++++++- 5 files changed, 87 insertions(+), 1 deletion(-) diff --git a/hmp.c b/hmp.c index 47d36e3ccf..a9e2776d60 100644 --- a/hmp.c +++ b/hmp.c @@ -271,6 +271,19 @@ void hmp_info_migrate(Monitor *mon, const QDict *qdict) info->xbzrle_cache->overflow); } =20 + if (info->has_compression) { + monitor_printf(mon, "compression pages: %" PRIu64 " pages\n", + info->compression->pages); + monitor_printf(mon, "compression busy: %" PRIu64 "\n", + info->compression->busy); + monitor_printf(mon, "compression busy rate: %0.2f\n", + info->compression->busy_rate); + monitor_printf(mon, "compression reduced size: %" PRIu64 "\n", + info->compression->reduced_size); + monitor_printf(mon, "compression rate: %0.2f\n", + info->compression->compression_rate); + } + if (info->has_cpu_throttle_percentage) { monitor_printf(mon, "cpu throttle percentage: %" PRIu64 "\n", info->cpu_throttle_percentage); diff --git a/migration/migration.c b/migration/migration.c index 0af75465b3..8fedf13889 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -754,6 +754,17 @@ static void populate_ram_info(MigrationInfo *info, Mig= rationState *s) info->xbzrle_cache->overflow =3D xbzrle_counters.overflow; } =20 + if (migrate_use_compression()) { + info->has_compression =3D true; + info->compression =3D g_malloc0(sizeof(*info->compression)); + info->compression->pages =3D compression_counters.pages; + info->compression->busy =3D compression_counters.busy; + info->compression->busy_rate =3D compression_counters.busy_rate; + info->compression->reduced_size =3D compression_counters.reduced_s= ize; + info->compression->compression_rate =3D + compression_counters.compression_rate; + } + if (cpu_throttle_active()) { info->has_cpu_throttle_percentage =3D true; info->cpu_throttle_percentage =3D cpu_throttle_get_percentage(); diff --git a/migration/ram.c b/migration/ram.c index 1b016e048d..e68b0e6dec 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -300,6 +300,15 @@ struct RAMState { uint64_t num_dirty_pages_period; /* xbzrle misses since the beginning of the period */ uint64_t xbzrle_cache_miss_prev; + + /* compression statistics since the beginning of the period */ + /* amount of count that no free thread to compress data */ + uint64_t compress_thread_busy_prev; + /* amount bytes reduced by compression */ + uint64_t compress_reduced_size_prev; + /* amount of compressed pages */ + uint64_t compress_pages_prev; + /* number of iterations at the beginning of period */ uint64_t iterations_prev; /* Iterations since start */ @@ -337,6 +346,8 @@ struct PageSearchStatus { }; typedef struct PageSearchStatus PageSearchStatus; =20 +CompressionStats compression_counters; + struct CompressParam { bool done; bool quit; @@ -1597,6 +1608,24 @@ static void migration_update_rates(RAMState *rs, int= 64_t end_time) rs->xbzrle_cache_miss_prev) / iter_count; rs->xbzrle_cache_miss_prev =3D xbzrle_counters.cache_miss; } + + if (migrate_use_compression()) { + uint64_t comp_pages; + + compression_counters.busy_rate =3D (double)(compression_counters.b= usy - + rs->compress_thread_busy_prev) / iter_count; + rs->compress_thread_busy_prev =3D compression_counters.busy; + + comp_pages =3D compression_counters.pages - rs->compress_pages_pre= v; + if (comp_pages) { + compression_counters.compression_rate =3D + (double)(compression_counters.reduced_size - + rs->compress_reduced_size_prev) / + (comp_pages * TARGET_PAGE_SIZE); + rs->compress_pages_prev =3D compression_counters.pages; + rs->compress_reduced_size_prev =3D compression_counters.reduce= d_size; + } + } } =20 static void migration_bitmap_sync(RAMState *rs) @@ -1872,6 +1901,9 @@ static void flush_compressed_data(RAMState *rs) qemu_mutex_lock(&comp_param[idx].mutex); if (!comp_param[idx].quit) { len =3D qemu_put_qemu_file(rs->f, comp_param[idx].file); + /* 8 means a header with RAM_SAVE_FLAG_CONTINUE. */ + compression_counters.reduced_size +=3D TARGET_PAGE_SIZE - len = + 8; + compression_counters.pages++; ram_counters.transferred +=3D len; } qemu_mutex_unlock(&comp_param[idx].mutex); @@ -1903,6 +1935,10 @@ retry: qemu_cond_signal(&comp_param[idx].cond); qemu_mutex_unlock(&comp_param[idx].mutex); pages =3D 1; + /* 8 means a header with RAM_SAVE_FLAG_CONTINUE. */ + compression_counters.reduced_size +=3D TARGET_PAGE_SIZE - + bytes_xmit + 8; + compression_counters.pages++; ram_counters.transferred +=3D bytes_xmit; break; } @@ -2233,6 +2269,7 @@ static int ram_save_target_page(RAMState *rs, PageSea= rchStatus *pss, if (res > 0) { return res; } + compression_counters.busy++; } else if (migrate_use_multifd()) { return ram_save_multifd_page(rs, block, offset); } diff --git a/migration/ram.h b/migration/ram.h index 457bf54b8c..a139066846 100644 --- a/migration/ram.h +++ b/migration/ram.h @@ -36,6 +36,7 @@ =20 extern MigrationStats ram_counters; extern XBZRLECacheStats xbzrle_counters; +extern CompressionStats compression_counters; =20 int xbzrle_cache_resize(int64_t new_size, Error **errp); uint64_t ram_bytes_remaining(void); diff --git a/qapi/migration.json b/qapi/migration.json index b4f394844b..5a34aa1754 100644 --- a/qapi/migration.json +++ b/qapi/migration.json @@ -75,6 +75,27 @@ 'cache-miss': 'int', 'cache-miss-rate': 'number', 'overflow': 'int' } } =20 +## +# @CompressionStats: +# +# Detailed migration compression statistics +# +# @pages: amount of pages compressed and transferred to the target VM +# +# @busy: count of times that no free thread was available to compress data +# +# @busy-rate: rate of thread busy +# +# @reduced-size: amount of bytes reduced by compression +# +# @compression-rate: rate of compressed size +# +# Since: 3.0 +## +{ 'struct': 'CompressionStats', + 'data': {'pages': 'int', 'busy': 'int', 'busy-rate': 'number', + 'reduced-size': 'int', 'compression-rate': 'number' } } + ## # @MigrationStatus: # @@ -172,6 +193,8 @@ # only present when the postcopy-blocktime migration capability # is enabled. (Since 3.0) # +# @compression: migration compression statistics, only returned if compres= sion +# feature is on and status is 'active' or 'completed' (Since 3.0) # # Since: 0.14.0 ## @@ -186,7 +209,8 @@ '*cpu-throttle-percentage': 'int', '*error-desc': 'str', '*postcopy-blocktime' : 'uint32', - '*postcopy-vcpu-blocktime': ['uint32']} } + '*postcopy-vcpu-blocktime': ['uint32'], + '*compression': 'CompressionStats'} } =20 ## # @query-migrate: --=20 2.14.4 From nobody Thu May 2 23:10:44 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=gmail.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1532002859157886.745829886686; Thu, 19 Jul 2018 05:20:59 -0700 (PDT) Received: from localhost ([::1]:42089 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fg7vS-0005iZ-JG for importer@patchew.org; Thu, 19 Jul 2018 08:20:54 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:35026) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fg7qh-0002Bx-Kq for qemu-devel@nongnu.org; Thu, 19 Jul 2018 08:16:00 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fg7qd-0005Rb-JY for qemu-devel@nongnu.org; Thu, 19 Jul 2018 08:15:59 -0400 Received: from mail-pg1-x541.google.com ([2607:f8b0:4864:20::541]:43532) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1fg7qd-0005R7-D2 for qemu-devel@nongnu.org; Thu, 19 Jul 2018 08:15:55 -0400 Received: by mail-pg1-x541.google.com with SMTP id v13-v6so3582142pgr.10 for ; Thu, 19 Jul 2018 05:15:55 -0700 (PDT) Received: from localhost.localdomain ([203.205.141.37]) by smtp.gmail.com with ESMTPSA id h190-v6sm21270456pge.85.2018.07.19.05.15.51 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Thu, 19 Jul 2018 05:15:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=wEcNUFI0wdozoTLrgZXruOP9i+eqNPRyhLre85OGP5w=; b=jJbPGKmPQKaTXPoqM2PKwAi4jRndVDlwoEZ1obj03HpB6rH7YwSPU+cZlZnE2QtEuI Gq7bKhTtk8c3YPTKqnsH2my3re4wVe4ARo14nuueI1vScwGXOxv30U/q0jom/ggmxQ3N kR65XNcLC8otFmQnY5osLZZSjYl2iUQZV+deXK9erDTh/VDk5P3WBluAIaCGkHCxY+Ew jrn1irBs/zSd76G38PbdgW5KEw8gwWse5DWGvQh7wJzIe+MBaQXuT2I14XvigLkpjKc+ 1pF+gdjC6qiMkg4BOOczaI7z9oUcSdmhraYXyj+LMdXnE7dKnSBqv/Map8YgAsRsC7qN Yw4Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=wEcNUFI0wdozoTLrgZXruOP9i+eqNPRyhLre85OGP5w=; b=bRnr/wcFA7lh0HtE67lCaOFAiysCiW61aG3XEbaQo9T3a1kgbMG6422aVMe/4Xd2FP tzXMua8OUYEgtaEEtijmS+s/rnGUij5ZnIPBOtptcerHr9PEePd3kgbsE1wG16FTg4vu A4A+GlIYpdA+Uy9eiXg5dhPnUmlx4BVp82nfGpzPqxw1+KIUPiLtPn4jMImI1mqgxXsc 7raVxvXBKPfV03XOEwuVsXjsHrGTwNulTeGYDhsJfRoEHMKzuysZmszwzZ4iYlPoKD6n +cRvZXd8Xwt3qEaF8d0Eko9CJXG4xbvs4ruDcintK5LFEqhUBs7/qVcXhkfNme51lFGz bjIg== X-Gm-Message-State: AOUpUlHhekZWqWwFozm8Oe0TL2mflEj/1qu+M7klLLAEkrWbLQtstay/ WS3ykBFvlMggIJJcHWSsEz+56v3y X-Google-Smtp-Source: AAOMgpd5ovkSHqsxe043T0iz+OmS2kj92HWQLTksnZxJs5DnKiizTH8Vm7Iqm1sTpgn3fO5PfTYCZQ== X-Received: by 2002:a63:bf43:: with SMTP id i3-v6mr9871707pgo.342.1532002554658; Thu, 19 Jul 2018 05:15:54 -0700 (PDT) From: guangrong.xiao@gmail.com X-Google-Original-From: xiaoguangrong@tencent.com To: pbonzini@redhat.com, mst@redhat.com, mtosatti@redhat.com Date: Thu, 19 Jul 2018 20:15:16 +0800 Message-Id: <20180719121520.30026-5-xiaoguangrong@tencent.com> X-Mailer: git-send-email 2.14.4 In-Reply-To: <20180719121520.30026-1-xiaoguangrong@tencent.com> References: <20180719121520.30026-1-xiaoguangrong@tencent.com> X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2607:f8b0:4864:20::541 Subject: [Qemu-devel] [PATCH v2 4/8] migration: introduce save_zero_page_to_file X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kvm@vger.kernel.org, Xiao Guangrong , qemu-devel@nongnu.org, peterx@redhat.com, dgilbert@redhat.com, wei.w.wang@intel.com, jiang.biao2@zte.com.cn Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZohoMail: RDKM_2 RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: Xiao Guangrong It will be used by the compression threads Signed-off-by: Xiao Guangrong Reviewed-by: Peter Xu --- migration/ram.c | 40 ++++++++++++++++++++++++++++++---------- 1 file changed, 30 insertions(+), 10 deletions(-) diff --git a/migration/ram.c b/migration/ram.c index e68b0e6dec..ce6e69b649 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -1696,27 +1696,47 @@ static void migration_bitmap_sync(RAMState *rs) /** * save_zero_page: send the zero page to the stream * - * Returns the number of pages written. + * Returns the size of data written to the file, 0 means the page is not + * a zero page * * @rs: current RAM state + * @file: the file where the data is saved * @block: block that contains the page we want to send * @offset: offset inside the block for the page */ -static int save_zero_page(RAMState *rs, RAMBlock *block, ram_addr_t offset) +static int save_zero_page_to_file(RAMState *rs, QEMUFile *file, + RAMBlock *block, ram_addr_t offset) { uint8_t *p =3D block->host + offset; - int pages =3D -1; + int len =3D 0; =20 if (is_zero_range(p, TARGET_PAGE_SIZE)) { - ram_counters.duplicate++; - ram_counters.transferred +=3D - save_page_header(rs, rs->f, block, offset | RAM_SAVE_FLAG_ZERO= ); - qemu_put_byte(rs->f, 0); - ram_counters.transferred +=3D 1; - pages =3D 1; + len +=3D save_page_header(rs, file, block, offset | RAM_SAVE_FLAG_= ZERO); + qemu_put_byte(file, 0); + len +=3D 1; } + return len; +} =20 - return pages; +/** + * save_zero_page: send the zero page to the stream + * + * Returns the number of pages written. + * + * @rs: current RAM state + * @block: block that contains the page we want to send + * @offset: offset inside the block for the page + */ +static int save_zero_page(RAMState *rs, RAMBlock *block, ram_addr_t offset) +{ + int len =3D save_zero_page_to_file(rs, rs->f, block, offset); + + if (len) { + ram_counters.duplicate++; + ram_counters.transferred +=3D len; + return 1; + } + return -1; } =20 static void ram_release_pages(const char *rbname, uint64_t offset, int pag= es) --=20 2.14.4 From nobody Thu May 2 23:10:44 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=gmail.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1532004876696768.847907334175; Thu, 19 Jul 2018 05:54:36 -0700 (PDT) Received: from localhost ([::1]:42100 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fg7wy-0006kR-38 for importer@patchew.org; Thu, 19 Jul 2018 08:22:28 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:35038) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fg7qi-0002Cz-9e for qemu-devel@nongnu.org; Thu, 19 Jul 2018 08:16:03 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fg7qh-0005Th-7G for qemu-devel@nongnu.org; Thu, 19 Jul 2018 08:16:00 -0400 Received: from mail-pg1-x52e.google.com ([2607:f8b0:4864:20::52e]:40342) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1fg7qh-0005TE-0u for qemu-devel@nongnu.org; Thu, 19 Jul 2018 08:15:59 -0400 Received: by mail-pg1-x52e.google.com with SMTP id x5-v6so3581247pgp.7 for ; Thu, 19 Jul 2018 05:15:58 -0700 (PDT) Received: from localhost.localdomain ([203.205.141.37]) by smtp.gmail.com with ESMTPSA id h190-v6sm21270456pge.85.2018.07.19.05.15.54 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Thu, 19 Jul 2018 05:15:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=O4pS8gEbytTb3R6OrpbGMegiK9SBuvi/9NY49ZO68JM=; b=FrMvyd7Q7U39M2UMpk1/Z9FnL+O3cXYT2o2RdDJm+iBuO/adoewJtRNafJtAosugma 8dRONdwV2zzWpr2Y9BjNMHd9JpaqCd4vsKaq/hE2Ud4q0RvWmoiK8n1sqpuUJgcE9Z9C 369gIBwfIvK/ybHuHWytUnF4IailT4D8y+lcimqXnfJQ7TAcV/DAqes4eBh5EPIDwWww tC553UE8k+EZy23L5n6QzCk+N0jINZpuEF5VpYtEefZ3rdzXYhIPBwVBSulOJgdeY8Id PwU2GxSv3ibob8plLeQpXakLY65dvpLSrfek8uJ/K2k7jXJ8AICKBTQY143kotf+UJK+ WFHw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=O4pS8gEbytTb3R6OrpbGMegiK9SBuvi/9NY49ZO68JM=; b=XVG+OQO4HN4rOODkzMWmqgAhW8mJwz+qY/NsAAg9yej4jglS1vscvyxVPBd04N3aMl 8D8IX1LeCnAH06uBhPgkB1xaQuuE512A0KaDsxd2C8Rtz1RQ6yBD23KTHe9rJbO3M+rF D88tR7yumq9o8LFWfsNU62PTMcuk7PEW9scCK7/bK9Pqz0nzl1OT/g7bdgqEZejkvVIJ GbuBl8M2ylo/cW948BbbSnhyyfdZxUmAtDlJSsBtimxmOBL055h1ZaF9hFmi84dxsomp SJ4/Hl4HfamU1NQ5vLM7AAoO3atAWLw60R9Y9JvJli5sCLz2WfSqjd8qBK60VNBwPH1B 3dqA== X-Gm-Message-State: AOUpUlHEPFdDt2mcQoNdlb58XRjUfXmiFt1sc29eNBWW8l7e37UOCQ/n tuiOqakZt20xpN6QjhGeLv5eDBdd X-Google-Smtp-Source: AAOMgpc6VOOU7c+cwupZ30jiyiXRDsQuG/g3rdB69VSLgHD/zzmILV5+EPYetm/S9aGt523YBCQq3Q== X-Received: by 2002:a63:ba10:: with SMTP id k16-v6mr9933305pgf.145.1532002558249; Thu, 19 Jul 2018 05:15:58 -0700 (PDT) From: guangrong.xiao@gmail.com X-Google-Original-From: xiaoguangrong@tencent.com To: pbonzini@redhat.com, mst@redhat.com, mtosatti@redhat.com Date: Thu, 19 Jul 2018 20:15:17 +0800 Message-Id: <20180719121520.30026-6-xiaoguangrong@tencent.com> X-Mailer: git-send-email 2.14.4 In-Reply-To: <20180719121520.30026-1-xiaoguangrong@tencent.com> References: <20180719121520.30026-1-xiaoguangrong@tencent.com> X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2607:f8b0:4864:20::52e Subject: [Qemu-devel] [PATCH v2 5/8] migration: drop the return value of do_compress_ram_page X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kvm@vger.kernel.org, Xiao Guangrong , qemu-devel@nongnu.org, peterx@redhat.com, dgilbert@redhat.com, wei.w.wang@intel.com, jiang.biao2@zte.com.cn Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZohoMail: RDKM_2 RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: Xiao Guangrong It is not used and cleans the code up a little Signed-off-by: Xiao Guangrong Reviewed-by: Peter Xu --- migration/ram.c | 26 +++++++++++--------------- 1 file changed, 11 insertions(+), 15 deletions(-) diff --git a/migration/ram.c b/migration/ram.c index ce6e69b649..5aa624b3b9 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -392,8 +392,8 @@ static QemuThread *decompress_threads; static QemuMutex decomp_done_lock; static QemuCond decomp_done_cond; =20 -static int do_compress_ram_page(QEMUFile *f, z_stream *stream, RAMBlock *b= lock, - ram_addr_t offset, uint8_t *source_buf); +static void do_compress_ram_page(QEMUFile *f, z_stream *stream, RAMBlock *= block, + ram_addr_t offset, uint8_t *source_buf); =20 static void *do_data_compress(void *opaque) { @@ -1871,15 +1871,14 @@ static int ram_save_multifd_page(RAMState *rs, RAMB= lock *block, return 1; } =20 -static int do_compress_ram_page(QEMUFile *f, z_stream *stream, RAMBlock *b= lock, - ram_addr_t offset, uint8_t *source_buf) +static void do_compress_ram_page(QEMUFile *f, z_stream *stream, RAMBlock *= block, + ram_addr_t offset, uint8_t *source_buf) { RAMState *rs =3D ram_state; - int bytes_sent, blen; uint8_t *p =3D block->host + (offset & TARGET_PAGE_MASK); + int ret; =20 - bytes_sent =3D save_page_header(rs, f, block, offset | - RAM_SAVE_FLAG_COMPRESS_PAGE); + save_page_header(rs, f, block, offset | RAM_SAVE_FLAG_COMPRESS_PAGE); =20 /* * copy it to a internal buffer to avoid it being modified by VM @@ -1887,17 +1886,14 @@ static int do_compress_ram_page(QEMUFile *f, z_stre= am *stream, RAMBlock *block, * decompression */ memcpy(source_buf, p, TARGET_PAGE_SIZE); - blen =3D qemu_put_compression_data(f, stream, source_buf, TARGET_PAGE_= SIZE); - if (blen < 0) { - bytes_sent =3D 0; - qemu_file_set_error(migrate_get_current()->to_dst_file, blen); + ret =3D qemu_put_compression_data(f, stream, source_buf, TARGET_PAGE_S= IZE); + if (ret < 0) { + qemu_file_set_error(migrate_get_current()->to_dst_file, ret); error_report("compressed data failed!"); - } else { - bytes_sent +=3D blen; - ram_release_pages(block->idstr, offset & TARGET_PAGE_MASK, 1); + return; } =20 - return bytes_sent; + ram_release_pages(block->idstr, offset & TARGET_PAGE_MASK, 1); } =20 static void flush_compressed_data(RAMState *rs) --=20 2.14.4 From nobody Thu May 2 23:10:44 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=gmail.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1532004866380957.4119205900823; Thu, 19 Jul 2018 05:54:26 -0700 (PDT) Received: from localhost ([::1]:42113 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fg7zM-0008Qm-Dd for importer@patchew.org; Thu, 19 Jul 2018 08:24:56 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:35091) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fg7qr-0002FR-40 for qemu-devel@nongnu.org; Thu, 19 Jul 2018 08:16:10 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fg7qk-0005WT-RM for qemu-devel@nongnu.org; Thu, 19 Jul 2018 08:16:09 -0400 Received: from mail-pg1-x529.google.com ([2607:f8b0:4864:20::529]:33069) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1fg7qk-0005VL-JL for qemu-devel@nongnu.org; Thu, 19 Jul 2018 08:16:02 -0400 Received: by mail-pg1-x529.google.com with SMTP id r5-v6so3591486pgv.0 for ; Thu, 19 Jul 2018 05:16:02 -0700 (PDT) Received: from localhost.localdomain ([203.205.141.37]) by smtp.gmail.com with ESMTPSA id h190-v6sm21270456pge.85.2018.07.19.05.15.58 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Thu, 19 Jul 2018 05:16:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=Xn13UKBIIS7bezDpJYirzTV6oEJqpxCKsm/KDJZKtr4=; b=a3ec1DqBcXOLodLGOLrlwanwsrL/W5Ry9Ze5HWOuuVEmvsd1Lz6XZ5+ROE251Yr1iB GwJGfWbVDlVmVMu9XrWd7N1WTzFWjvV/VdPRXSq6ee6jP+FgdevhGPJO25Fr8Z3afo+N IWXVv/WB7XSgxVHhtNL0hz++eOoWftl/f9NaEDi86M8BvyL7H/flliD1590k2PB2rngk GYuRcWzmI7wojHXEaBxDCL6N/7FAzvPQwa98aEf+p9vVO0xoWJ7Y5C9OtcRLbczIlRvn +7D8PfDKWGyJVa/JfTDZJcGevYKekDSWC3xYe8zk1idXCWzKQWuwYWuHYM9VlSfwlRty kZSA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=Xn13UKBIIS7bezDpJYirzTV6oEJqpxCKsm/KDJZKtr4=; b=MTYazKNLnMOILxQHGX/RPNOjA7wJzyyFkHZU5L6mRLu9O7KPuKB8DTIkhK0FX9Vv/Y O7Wf+mf9qnFcbeCSEpEvjp2/YmNv6BfnZ2uQdINyJNvuJcE4kNOAsD7Q/ZD0yiWSUFY5 sbdPisycnd4sD8vlRzwZq0HclSPyuqFYjDYZruAhj1HlNsMKXJ2ZUhfmhRLTgvClmskN +gE1gvkcKWjQOerFjITyDYZEr6W8NvMpsnHaKbpuTeN6gTi4VE08sDkLfPBmnLzqXF7m E7p7HWW/lPExRGpetNeJ1fsnNLerukcqZaF/sKTqK1S0aFFsQ3z08i8iat81U79Sj9hN 3JIg== X-Gm-Message-State: AOUpUlFEwb1eIFpwRaixbgvZHUXxyiX3kp11QKKYUTg3mpXjVZQ6su+Q RaXhi/UUOATRhekF1+rUI2k= X-Google-Smtp-Source: AAOMgpdNwJcu86r/6zGmiopVdzAv861/PeucLOPD0a8RidMxgO3kt2ZbndgdIAIKKppgSZJUuEawlw== X-Received: by 2002:a62:d39b:: with SMTP id z27-v6mr9442738pfk.182.1532002561719; Thu, 19 Jul 2018 05:16:01 -0700 (PDT) From: guangrong.xiao@gmail.com X-Google-Original-From: xiaoguangrong@tencent.com To: pbonzini@redhat.com, mst@redhat.com, mtosatti@redhat.com Date: Thu, 19 Jul 2018 20:15:18 +0800 Message-Id: <20180719121520.30026-7-xiaoguangrong@tencent.com> X-Mailer: git-send-email 2.14.4 In-Reply-To: <20180719121520.30026-1-xiaoguangrong@tencent.com> References: <20180719121520.30026-1-xiaoguangrong@tencent.com> X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2607:f8b0:4864:20::529 Subject: [Qemu-devel] [PATCH v2 6/8] migration: move handle of zero page to the thread X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kvm@vger.kernel.org, Xiao Guangrong , qemu-devel@nongnu.org, peterx@redhat.com, dgilbert@redhat.com, wei.w.wang@intel.com, jiang.biao2@zte.com.cn Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZohoMail: RDKM_2 RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: Xiao Guangrong Detecting zero page is not a light work, moving it to the thread to speed the main thread up Signed-off-by: Xiao Guangrong --- migration/ram.c | 112 +++++++++++++++++++++++++++++++++++++++-------------= ---- 1 file changed, 78 insertions(+), 34 deletions(-) diff --git a/migration/ram.c b/migration/ram.c index 5aa624b3b9..e1909502da 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -351,6 +351,7 @@ CompressionStats compression_counters; struct CompressParam { bool done; bool quit; + bool zero_page; QEMUFile *file; QemuMutex mutex; QemuCond cond; @@ -392,7 +393,7 @@ static QemuThread *decompress_threads; static QemuMutex decomp_done_lock; static QemuCond decomp_done_cond; =20 -static void do_compress_ram_page(QEMUFile *f, z_stream *stream, RAMBlock *= block, +static bool do_compress_ram_page(QEMUFile *f, z_stream *stream, RAMBlock *= block, ram_addr_t offset, uint8_t *source_buf); =20 static void *do_data_compress(void *opaque) @@ -400,6 +401,7 @@ static void *do_data_compress(void *opaque) CompressParam *param =3D opaque; RAMBlock *block; ram_addr_t offset; + bool zero_page; =20 qemu_mutex_lock(¶m->mutex); while (!param->quit) { @@ -409,11 +411,12 @@ static void *do_data_compress(void *opaque) param->block =3D NULL; qemu_mutex_unlock(¶m->mutex); =20 - do_compress_ram_page(param->file, ¶m->stream, block, offse= t, - param->originbuf); + zero_page =3D do_compress_ram_page(param->file, ¶m->stream, + block, offset, param->originb= uf); =20 qemu_mutex_lock(&comp_done_lock); param->done =3D true; + param->zero_page =3D zero_page; qemu_cond_signal(&comp_done_cond); qemu_mutex_unlock(&comp_done_lock); =20 @@ -1871,13 +1874,19 @@ static int ram_save_multifd_page(RAMState *rs, RAMB= lock *block, return 1; } =20 -static void do_compress_ram_page(QEMUFile *f, z_stream *stream, RAMBlock *= block, +static bool do_compress_ram_page(QEMUFile *f, z_stream *stream, RAMBlock *= block, ram_addr_t offset, uint8_t *source_buf) { RAMState *rs =3D ram_state; uint8_t *p =3D block->host + (offset & TARGET_PAGE_MASK); + bool zero_page =3D false; int ret; =20 + if (save_zero_page_to_file(rs, f, block, offset)) { + zero_page =3D true; + goto exit; + } + save_page_header(rs, f, block, offset | RAM_SAVE_FLAG_COMPRESS_PAGE); =20 /* @@ -1890,10 +1899,12 @@ static void do_compress_ram_page(QEMUFile *f, z_str= eam *stream, RAMBlock *block, if (ret < 0) { qemu_file_set_error(migrate_get_current()->to_dst_file, ret); error_report("compressed data failed!"); - return; + return false; } =20 +exit: ram_release_pages(block->idstr, offset & TARGET_PAGE_MASK, 1); + return zero_page; } =20 static void flush_compressed_data(RAMState *rs) @@ -1917,10 +1928,20 @@ static void flush_compressed_data(RAMState *rs) qemu_mutex_lock(&comp_param[idx].mutex); if (!comp_param[idx].quit) { len =3D qemu_put_qemu_file(rs->f, comp_param[idx].file); - /* 8 means a header with RAM_SAVE_FLAG_CONTINUE. */ - compression_counters.reduced_size +=3D TARGET_PAGE_SIZE - len = + 8; - compression_counters.pages++; ram_counters.transferred +=3D len; + + /* + * it's safe to fetch zero_page without holding comp_done_lock + * as there is no further request submitted to the thread, + * i.e, the thread should be waiting for a request at this poi= nt. + */ + if (comp_param[idx].zero_page) { + ram_counters.duplicate++; + } else { + /* 8 means a header with RAM_SAVE_FLAG_CONTINUE. */ + compression_counters.reduced_size +=3D TARGET_PAGE_SIZE - = len + 8; + compression_counters.pages++; + } } qemu_mutex_unlock(&comp_param[idx].mutex); } @@ -1950,12 +1971,16 @@ retry: set_compress_params(&comp_param[idx], block, offset); qemu_cond_signal(&comp_param[idx].cond); qemu_mutex_unlock(&comp_param[idx].mutex); - pages =3D 1; - /* 8 means a header with RAM_SAVE_FLAG_CONTINUE. */ - compression_counters.reduced_size +=3D TARGET_PAGE_SIZE - - bytes_xmit + 8; - compression_counters.pages++; ram_counters.transferred +=3D bytes_xmit; + pages =3D 1; + if (comp_param[idx].zero_page) { + ram_counters.duplicate++; + } else { + /* 8 means a header with RAM_SAVE_FLAG_CONTINUE. */ + compression_counters.reduced_size +=3D TARGET_PAGE_SIZE - + bytes_xmit + 8; + compression_counters.pages++; + } break; } } @@ -2229,6 +2254,40 @@ static bool save_page_use_compression(RAMState *rs) return false; } =20 +/* + * try to compress the page before post it out, return true if the page + * has been properly handled by compression, otherwise needs other + * paths to handle it + */ +static bool save_compress_page(RAMState *rs, RAMBlock *block, ram_addr_t o= ffset) +{ + if (!save_page_use_compression(rs)) { + return false; + } + + /* + * When starting the process of a new block, the first page of + * the block should be sent out before other pages in the same + * block, and all the pages in last block should have been sent + * out, keeping this order is important, because the 'cont' flag + * is used to avoid resending the block name. + * + * We post the fist page as normal page as compression will take + * much CPU resource. + */ + if (block !=3D rs->last_sent_block) { + flush_compressed_data(rs); + return false; + } + + if (compress_page_with_multi_thread(rs, block, offset) > 0) { + return true; + } + + compression_counters.busy++; + return false; +} + /** * ram_save_target_page: save one target page * @@ -2249,15 +2308,8 @@ static int ram_save_target_page(RAMState *rs, PageSe= archStatus *pss, return res; } =20 - /* - * When starting the process of a new block, the first page of - * the block should be sent out before other pages in the same - * block, and all the pages in last block should have been sent - * out, keeping this order is important, because the 'cont' flag - * is used to avoid resending the block name. - */ - if (block !=3D rs->last_sent_block && save_page_use_compression(rs)) { - flush_compressed_data(rs); + if (save_compress_page(rs, block, offset)) { + return 1; } =20 res =3D save_zero_page(rs, block, offset); @@ -2275,18 +2327,10 @@ static int ram_save_target_page(RAMState *rs, PageS= earchStatus *pss, } =20 /* - * Make sure the first page is sent out before other pages. - * - * we post it as normal page as compression will take much - * CPU resource. - */ - if (block =3D=3D rs->last_sent_block && save_page_use_compression(rs))= { - res =3D compress_page_with_multi_thread(rs, block, offset); - if (res > 0) { - return res; - } - compression_counters.busy++; - } else if (migrate_use_multifd()) { + * do not use multifd for compression as the first page in the new + * block should be posted out before sending the compressed page + */ + if (!save_page_use_compression(rs) && migrate_use_multifd()) { return ram_save_multifd_page(rs, block, offset); } =20 --=20 2.14.4 From nobody Thu May 2 23:10:44 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=gmail.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1532004855170492.19360986471634; Thu, 19 Jul 2018 05:54:15 -0700 (PDT) Received: from localhost ([::1]:42121 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fg802-00012r-TS for importer@patchew.org; Thu, 19 Jul 2018 08:25:38 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:35113) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fg7qt-0002HV-UK for qemu-devel@nongnu.org; Thu, 19 Jul 2018 08:16:15 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fg7qo-0005Yr-38 for qemu-devel@nongnu.org; Thu, 19 Jul 2018 08:16:11 -0400 Received: from mail-pg1-x544.google.com ([2607:f8b0:4864:20::544]:39245) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1fg7qn-0005YM-RH for qemu-devel@nongnu.org; Thu, 19 Jul 2018 08:16:05 -0400 Received: by mail-pg1-x544.google.com with SMTP id g2-v6so3589063pgs.6 for ; Thu, 19 Jul 2018 05:16:05 -0700 (PDT) Received: from localhost.localdomain ([203.205.141.37]) by smtp.gmail.com with ESMTPSA id h190-v6sm21270456pge.85.2018.07.19.05.16.01 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Thu, 19 Jul 2018 05:16:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=wu/nz6fAWAIdK88AMQKiO9Vg+H9xHdRSbNE5knvlfxw=; b=fGm5D8VWrWiY4jf+i9Y7ttG1ZqYNfCUDlwbBpK+y4nAdSyMyVK/2O45qT5UR6pxi+F eTqjKV8KM1QERSTk0cW+vzlc7WQIplEOTGnQ+NsTmydMcr0EvjcI66Yktnu3AClFeEd3 4lFYjnbp3B127dx8PLhihg8bodczZx5eYW8MxZ7JFdeSC5C7f9DnoPlZbqztHm+UrI2b afTLxqCpkoW0vOv2LjhYuuWwvVTlORXNyR214VowCRyEb4dSWxgk76LR784l1/Y72uYv d8LzxsvlorU077Uy8oYsG/KeZ6gLSDsmwRPfpEjWp3Uf3q9hy3SXLzlpXYD6Kyzp5vzy QmXw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=wu/nz6fAWAIdK88AMQKiO9Vg+H9xHdRSbNE5knvlfxw=; b=r4NL7awyaojXs4VxUITmDbFCb1w1hyDnM0gMd8aYd4P0IAJ3SOwfjsV4MhfiIaNDJa C28tyZZFUVU98DK5MEt7gjIoNWp6s+f6xdz0wYBaH9QKtgZwGAruMlTj+xi5ulku/wSR 3pDJFSs27OkDXkHV1Zc7zmIJMPOt+HRuGwVM6b1rDgUawRereJa8V4aZdX1Ui8k2DHEd N3yKAABgC11h8dBtu5OckKoP1mAe0Fv0G+LfGVaQH5smi3rjVzcnxNzs9razossdRtN2 dHl0JxYvCQkn5W7LZcn2I4pYm6/DejT6QYz8BP8be5A+pAKAPmSqBWjzbbBmt0ydIygw Vutg== X-Gm-Message-State: AOUpUlFrrbT1x1uMjSoCsoAg77ozCbhVsbmp17QhDF98TjA4/RhUeUTU h5hWac1wR2udTOQvL4ngL6Q= X-Google-Smtp-Source: AAOMgpekGQoYhSNEF1w+YESeZ33Ab1TQ3nomKIYrZpUHxfDAVivj2JyBtVSieunVFGCtWi7u3FweNg== X-Received: by 2002:a63:ba10:: with SMTP id k16-v6mr9933706pgf.145.1532002565175; Thu, 19 Jul 2018 05:16:05 -0700 (PDT) From: guangrong.xiao@gmail.com X-Google-Original-From: xiaoguangrong@tencent.com To: pbonzini@redhat.com, mst@redhat.com, mtosatti@redhat.com Date: Thu, 19 Jul 2018 20:15:19 +0800 Message-Id: <20180719121520.30026-8-xiaoguangrong@tencent.com> X-Mailer: git-send-email 2.14.4 In-Reply-To: <20180719121520.30026-1-xiaoguangrong@tencent.com> References: <20180719121520.30026-1-xiaoguangrong@tencent.com> X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2607:f8b0:4864:20::544 Subject: [Qemu-devel] [PATCH v2 7/8] migration: hold the lock only if it is really needed X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kvm@vger.kernel.org, Xiao Guangrong , qemu-devel@nongnu.org, peterx@redhat.com, dgilbert@redhat.com, wei.w.wang@intel.com, jiang.biao2@zte.com.cn Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZohoMail: RDKM_2 RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: Xiao Guangrong Try to hold src_page_req_mutex only if the queue is not empty Reviewed-by: Dr. David Alan Gilbert Signed-off-by: Xiao Guangrong Reviewed-by: Peter Xu --- include/qemu/queue.h | 1 + migration/ram.c | 4 ++++ 2 files changed, 5 insertions(+) diff --git a/include/qemu/queue.h b/include/qemu/queue.h index 59fd1203a1..ac418efc43 100644 --- a/include/qemu/queue.h +++ b/include/qemu/queue.h @@ -341,6 +341,7 @@ struct { = \ /* * Simple queue access methods. */ +#define QSIMPLEQ_EMPTY_ATOMIC(head) (atomic_read(&((head)->sqh_first)) =3D= =3D NULL) #define QSIMPLEQ_EMPTY(head) ((head)->sqh_first =3D=3D NULL) #define QSIMPLEQ_FIRST(head) ((head)->sqh_first) #define QSIMPLEQ_NEXT(elm, field) ((elm)->field.sqe_next) diff --git a/migration/ram.c b/migration/ram.c index e1909502da..89305c7af5 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -2062,6 +2062,10 @@ static RAMBlock *unqueue_page(RAMState *rs, ram_addr= _t *offset) { RAMBlock *block =3D NULL; =20 + if (QSIMPLEQ_EMPTY_ATOMIC(&rs->src_page_requests)) { + return NULL; + } + qemu_mutex_lock(&rs->src_page_req_mutex); if (!QSIMPLEQ_EMPTY(&rs->src_page_requests)) { struct RAMSrcPageRequest *entry =3D --=20 2.14.4 From nobody Thu May 2 23:10:44 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=gmail.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1532004893610788.0271373310055; Thu, 19 Jul 2018 05:54:53 -0700 (PDT) Received: from localhost ([::1]:42105 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fg7y0-0007Os-S4 for importer@patchew.org; Thu, 19 Jul 2018 08:23:32 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:35106) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fg7qs-0002Gi-Nm for qemu-devel@nongnu.org; Thu, 19 Jul 2018 08:16:13 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fg7qr-0005bm-Nn for qemu-devel@nongnu.org; Thu, 19 Jul 2018 08:16:10 -0400 Received: from mail-pl0-x243.google.com ([2607:f8b0:400e:c01::243]:33306) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1fg7qr-0005bB-HP for qemu-devel@nongnu.org; Thu, 19 Jul 2018 08:16:09 -0400 Received: by mail-pl0-x243.google.com with SMTP id 6-v6so3585895plb.0 for ; Thu, 19 Jul 2018 05:16:09 -0700 (PDT) Received: from localhost.localdomain ([203.205.141.37]) by smtp.gmail.com with ESMTPSA id h190-v6sm21270456pge.85.2018.07.19.05.16.05 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Thu, 19 Jul 2018 05:16:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=jL/6YSOrNyL/s7T74f7jW6HHfWELAyr7TPcJtMb86yk=; b=CavB5tmOx/YDEIys4RvgT2p6mAK95q4AZjp4IThQhZsu5rt5LwoRSxU2PV1fH6LojU SSS8BEL3ZmVIMn9aJKTwTxRlZlXsysFI8ELcegq2E/AZt91XQvWaBenLU2nJd/gqwlHB Spt+m2fIGmFl7P+wM4fPVvudtZFvql+W+PUw6POxciuVaEbze73vvHoL1YDwKtVQ8FNz Vn3iF6kvsuNCYtC/072s6VCfRz2Bh9ornjgiBwpblKVVHC/bj9C5WSUjllf8uZt7XOgH sV3kbIydNLrpPw8zRj2ptoWaAY/wq9k1BRLuWHTH4O39oYMbhXRb5F1qmQhDCOP5cobq GonA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=jL/6YSOrNyL/s7T74f7jW6HHfWELAyr7TPcJtMb86yk=; b=I2CCiT9zHEC+BQ/+plGD+UgcgkrJZn88qpbLpTfd2Tw65C1fPkJUVxIozjhibhIOJO IHOBoBqynFoMb9pNoktMTNPw92ktMTGyQJWfuGVV8IDvNlCuoN4rQoQTWkZ4NQBYJ05Q 0Cp+MHntT3Fanb8zr7nsuy3A1gpQyauk33ugfGvD3SobHxKzyUzkFbFfRd5v/QJGeVkr ek1R5PEGsWe7AdsUhdjaKEx1qtxhK72a8udZ0p5FG6XeCH0aKv8Yg6OwOQXTJ4L2b/Rs 1+5G4exDt+ky1i8ki2kPu0FzWDCa3sDdtP5Gh1ZEdnL3eqiQO4NiOYzU97jPscHhj0E+ vJPA== X-Gm-Message-State: AOUpUlHxVQHolpI3aXmPdjEnODgW3384RqSghhQyUqybvFglANQACOa/ 6gNI1VB1RqIg8p/rR/GsGkM= X-Google-Smtp-Source: AAOMgpdRFhLQGFOuDoLK8+53PHsuaQb2i3Fi4BFpHQW5AiSPV6kt5CLvtO8kvxDG1v1wa3iWFOdIEg== X-Received: by 2002:a17:902:8207:: with SMTP id x7-v6mr9787553pln.57.1532002568733; Thu, 19 Jul 2018 05:16:08 -0700 (PDT) From: guangrong.xiao@gmail.com X-Google-Original-From: xiaoguangrong@tencent.com To: pbonzini@redhat.com, mst@redhat.com, mtosatti@redhat.com Date: Thu, 19 Jul 2018 20:15:20 +0800 Message-Id: <20180719121520.30026-9-xiaoguangrong@tencent.com> X-Mailer: git-send-email 2.14.4 In-Reply-To: <20180719121520.30026-1-xiaoguangrong@tencent.com> References: <20180719121520.30026-1-xiaoguangrong@tencent.com> X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2607:f8b0:400e:c01::243 Subject: [Qemu-devel] [PATCH v2 8/8] migration: do not flush_compressed_data at the end of each iteration X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kvm@vger.kernel.org, Xiao Guangrong , qemu-devel@nongnu.org, peterx@redhat.com, dgilbert@redhat.com, wei.w.wang@intel.com, jiang.biao2@zte.com.cn Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZohoMail: RDKM_2 RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: Xiao Guangrong flush_compressed_data() needs to wait all compression threads to finish their work, after that all threads are free until the migration feeds new request to them, reducing its call can improve the throughput and use CPU resource more effectively We do not need to flush all threads at the end of iteration, the data can be kept locally until the memory block is changed or memory migration starts over in that case we will meet a dirtied page which may still exists in compression threads's ring Signed-off-by: Xiao Guangrong --- migration/ram.c | 15 ++++++++++++++- 1 file changed, 14 insertions(+), 1 deletion(-) diff --git a/migration/ram.c b/migration/ram.c index 89305c7af5..fdab13821d 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -315,6 +315,8 @@ struct RAMState { uint64_t iterations; /* number of dirty bits in the bitmap */ uint64_t migration_dirty_pages; + /* last dirty_sync_count we have seen */ + uint64_t dirty_sync_count; /* protects modification of the bitmap */ QemuMutex bitmap_mutex; /* The RAMBlock used in the last src_page_requests */ @@ -2532,6 +2534,7 @@ static void ram_save_cleanup(void *opaque) } =20 xbzrle_cleanup(); + flush_compressed_data(*rsp); compress_threads_save_cleanup(); ram_state_cleanup(rsp); } @@ -3203,6 +3206,17 @@ static int ram_save_iterate(QEMUFile *f, void *opaqu= e) =20 ram_control_before_iterate(f, RAM_CONTROL_ROUND); =20 + /* + * if memory migration starts over, we will meet a dirtied page which + * may still exists in compression threads's ring, so we should flush + * the compressed data to make sure the new page is not overwritten by + * the old one in the destination. + */ + if (ram_counters.dirty_sync_count !=3D rs->dirty_sync_count) { + rs->dirty_sync_count =3D ram_counters.dirty_sync_count; + flush_compressed_data(rs); + } + t0 =3D qemu_clock_get_ns(QEMU_CLOCK_REALTIME); i =3D 0; while ((ret =3D qemu_file_rate_limit(f)) =3D=3D 0 || @@ -3235,7 +3249,6 @@ static int ram_save_iterate(QEMUFile *f, void *opaque) } i++; } - flush_compressed_data(rs); rcu_read_unlock(); =20 /* --=20 2.14.4