From nobody Sun Feb 8 14:35:19 2026 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1537982578124990.0362213653576; Wed, 26 Sep 2018 10:22:58 -0700 (PDT) Received: from localhost ([::1]:59908 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1g5DWb-0008JZ-0e for importer@patchew.org; Wed, 26 Sep 2018 13:22:57 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:52690) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1g5DNC-0000qu-HT for qemu-devel@nongnu.org; Wed, 26 Sep 2018 13:13:15 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1g5DN9-0007Bs-9R for qemu-devel@nongnu.org; Wed, 26 Sep 2018 13:13:14 -0400 Received: from mx1.redhat.com ([209.132.183.28]:51442) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1g5DN8-00070f-Tc for qemu-devel@nongnu.org; Wed, 26 Sep 2018 13:13:11 -0400 Received: from smtp.corp.redhat.com (int-mx12.intmail.prod.int.phx2.redhat.com [10.5.11.27]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 4526330001EF; Wed, 26 Sep 2018 17:13:09 +0000 (UTC) Received: from dgilbert-t530.redhat.com (unknown [10.36.118.26]) by smtp.corp.redhat.com (Postfix) with ESMTP id B4986DB570; Wed, 26 Sep 2018 17:12:57 +0000 (UTC) From: "Dr. David Alan Gilbert (git)" To: qemu-devel@nongnu.org, xiaoguangrong@tencent.com, joserz@linux.ibm.com, wei@redhat.com, thuth@redhat.com, marcandre.lureau@redhat.com, fli@suse.com Date: Wed, 26 Sep 2018 18:12:26 +0100 Message-Id: <20180926171236.45203-7-dgilbert@redhat.com> In-Reply-To: <20180926171236.45203-1-dgilbert@redhat.com> References: <20180926171236.45203-1-dgilbert@redhat.com> X-Scanned-By: MIMEDefang 2.84 on 10.5.11.27 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.46]); Wed, 26 Sep 2018 17:13:09 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PULL 06/16] migration: show the statistics of compression X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: quintela@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RDMRC_1 RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: Xiao Guangrong Currently, it includes: pages: amount of pages compressed and transferred to the target VM busy: amount of count that no free thread to compress data busy-rate: rate of thread busy compressed-size: amount of bytes after compression compression-rate: rate of compressed size Reviewed-by: Juan Quintela Reviewed-by: Peter Xu Signed-off-by: Xiao Guangrong Message-Id: <20180906070101.27280-3-xiaoguangrong@tencent.com> Signed-off-by: Juan Quintela Signed-off-by: Dr. David Alan Gilbert --- hmp.c | 13 +++++++++++++ migration/migration.c | 12 ++++++++++++ migration/ram.c | 41 ++++++++++++++++++++++++++++++++++++++++- migration/ram.h | 1 + qapi/migration.json | 26 +++++++++++++++++++++++++- 5 files changed, 91 insertions(+), 2 deletions(-) diff --git a/hmp.c b/hmp.c index 3a9f797677..61ef120423 100644 --- a/hmp.c +++ b/hmp.c @@ -271,6 +271,19 @@ void hmp_info_migrate(Monitor *mon, const QDict *qdict) info->xbzrle_cache->overflow); } =20 + if (info->has_compression) { + monitor_printf(mon, "compression pages: %" PRIu64 " pages\n", + info->compression->pages); + monitor_printf(mon, "compression busy: %" PRIu64 "\n", + info->compression->busy); + monitor_printf(mon, "compression busy rate: %0.2f\n", + info->compression->busy_rate); + monitor_printf(mon, "compressed size: %" PRIu64 "\n", + info->compression->compressed_size); + monitor_printf(mon, "compression rate: %0.2f\n", + info->compression->compression_rate); + } + if (info->has_cpu_throttle_percentage) { monitor_printf(mon, "cpu throttle percentage: %" PRIu64 "\n", info->cpu_throttle_percentage); diff --git a/migration/migration.c b/migration/migration.c index 05d0a7296a..6a7731de50 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -758,6 +758,18 @@ static void populate_ram_info(MigrationInfo *info, Mig= rationState *s) info->xbzrle_cache->overflow =3D xbzrle_counters.overflow; } =20 + if (migrate_use_compression()) { + info->has_compression =3D true; + info->compression =3D g_malloc0(sizeof(*info->compression)); + info->compression->pages =3D compression_counters.pages; + info->compression->busy =3D compression_counters.busy; + info->compression->busy_rate =3D compression_counters.busy_rate; + info->compression->compressed_size =3D + compression_counters.compressed_size; + info->compression->compression_rate =3D + compression_counters.compression_rate; + } + if (cpu_throttle_active()) { info->has_cpu_throttle_percentage =3D true; info->cpu_throttle_percentage =3D cpu_throttle_get_percentage(); diff --git a/migration/ram.c b/migration/ram.c index 2c039892d3..7c12f2792c 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -301,6 +301,15 @@ struct RAMState { uint64_t num_dirty_pages_period; /* xbzrle misses since the beginning of the period */ uint64_t xbzrle_cache_miss_prev; + + /* compression statistics since the beginning of the period */ + /* amount of count that no free thread to compress data */ + uint64_t compress_thread_busy_prev; + /* amount bytes after compression */ + uint64_t compressed_size_prev; + /* amount of compressed pages */ + uint64_t compress_pages_prev; + /* total handled target pages at the beginning of period */ uint64_t target_page_count_prev; /* total handled target pages since start */ @@ -338,6 +347,8 @@ struct PageSearchStatus { }; typedef struct PageSearchStatus PageSearchStatus; =20 +CompressionStats compression_counters; + struct CompressParam { bool done; bool quit; @@ -1593,6 +1604,7 @@ uint64_t ram_pagesize_summary(void) static void migration_update_rates(RAMState *rs, int64_t end_time) { uint64_t page_count =3D rs->target_page_count - rs->target_page_count_= prev; + double compressed_size; =20 /* calculate period counters */ ram_counters.dirty_pages_rate =3D rs->num_dirty_pages_period * 1000 @@ -1607,6 +1619,26 @@ static void migration_update_rates(RAMState *rs, int= 64_t end_time) rs->xbzrle_cache_miss_prev) / page_count; rs->xbzrle_cache_miss_prev =3D xbzrle_counters.cache_miss; } + + if (migrate_use_compression()) { + compression_counters.busy_rate =3D (double)(compression_counters.b= usy - + rs->compress_thread_busy_prev) / page_count; + rs->compress_thread_busy_prev =3D compression_counters.busy; + + compressed_size =3D compression_counters.compressed_size - + rs->compressed_size_prev; + if (compressed_size) { + double uncompressed_size =3D (compression_counters.pages - + rs->compress_pages_prev) * TARGET_PAGE= _SIZE; + + /* Compression-Ratio =3D Uncompressed-size / Compressed-size */ + compression_counters.compression_rate =3D + uncompressed_size / compressed_siz= e; + + rs->compress_pages_prev =3D compression_counters.pages; + rs->compressed_size_prev =3D compression_counters.compressed_s= ize; + } + } } =20 static void migration_bitmap_sync(RAMState *rs) @@ -1888,10 +1920,16 @@ exit: static void update_compress_thread_counts(const CompressParam *param, int bytes_xmit) { + ram_counters.transferred +=3D bytes_xmit; + if (param->zero_page) { ram_counters.duplicate++; + return; } - ram_counters.transferred +=3D bytes_xmit; + + /* 8 means a header with RAM_SAVE_FLAG_CONTINUE. */ + compression_counters.compressed_size +=3D bytes_xmit - 8; + compression_counters.pages++; } =20 static void flush_compressed_data(RAMState *rs) @@ -2264,6 +2302,7 @@ static bool save_compress_page(RAMState *rs, RAMBlock= *block, ram_addr_t offset) return true; } =20 + compression_counters.busy++; return false; } =20 diff --git a/migration/ram.h b/migration/ram.h index 457bf54b8c..a139066846 100644 --- a/migration/ram.h +++ b/migration/ram.h @@ -36,6 +36,7 @@ =20 extern MigrationStats ram_counters; extern XBZRLECacheStats xbzrle_counters; +extern CompressionStats compression_counters; =20 int xbzrle_cache_resize(int64_t new_size, Error **errp); uint64_t ram_bytes_remaining(void); diff --git a/qapi/migration.json b/qapi/migration.json index f62d3f9a4b..6e8c21258a 100644 --- a/qapi/migration.json +++ b/qapi/migration.json @@ -75,6 +75,27 @@ 'cache-miss': 'int', 'cache-miss-rate': 'number', 'overflow': 'int' } } =20 +## +# @CompressionStats: +# +# Detailed migration compression statistics +# +# @pages: amount of pages compressed and transferred to the target VM +# +# @busy: count of times that no free thread was available to compress data +# +# @busy-rate: rate of thread busy +# +# @compressed-size: amount of bytes after compression +# +# @compression-rate: rate of compressed size +# +# Since: 3.1 +## +{ 'struct': 'CompressionStats', + 'data': {'pages': 'int', 'busy': 'int', 'busy-rate': 'number', + 'compressed-size': 'int', 'compression-rate': 'number' } } + ## # @MigrationStatus: # @@ -172,6 +193,8 @@ # only present when the postcopy-blocktime migration capability # is enabled. (Since 3.0) # +# @compression: migration compression statistics, only returned if compres= sion +# feature is on and status is 'active' or 'completed' (Since 3.1) # # Since: 0.14.0 ## @@ -186,7 +209,8 @@ '*cpu-throttle-percentage': 'int', '*error-desc': 'str', '*postcopy-blocktime' : 'uint32', - '*postcopy-vcpu-blocktime': ['uint32']} } + '*postcopy-vcpu-blocktime': ['uint32'], + '*compression': 'CompressionStats'} } =20 ## # @query-migrate: --=20 2.17.1