From nobody Tue May 7 18:44:44 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1537982100462222.728201372228; Wed, 26 Sep 2018 10:15:00 -0700 (PDT) Received: from localhost ([::1]:59869 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1g5DOs-0001gr-LL for importer@patchew.org; Wed, 26 Sep 2018 13:14:58 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:52585) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1g5DMp-0000Q2-08 for qemu-devel@nongnu.org; Wed, 26 Sep 2018 13:12:51 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1g5DMn-0006dh-DR for qemu-devel@nongnu.org; Wed, 26 Sep 2018 13:12:50 -0400 Received: from mx1.redhat.com ([209.132.183.28]:45084) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1g5DMn-0006cr-6v for qemu-devel@nongnu.org; Wed, 26 Sep 2018 13:12:49 -0400 Received: from smtp.corp.redhat.com (int-mx12.intmail.prod.int.phx2.redhat.com [10.5.11.27]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 692CB307B978; Wed, 26 Sep 2018 17:12:48 +0000 (UTC) Received: from dgilbert-t530.redhat.com (unknown [10.36.118.26]) by smtp.corp.redhat.com (Postfix) with ESMTP id A4BEEDB561; Wed, 26 Sep 2018 17:12:46 +0000 (UTC) From: "Dr. David Alan Gilbert (git)" To: qemu-devel@nongnu.org, xiaoguangrong@tencent.com, joserz@linux.ibm.com, wei@redhat.com, thuth@redhat.com, marcandre.lureau@redhat.com, fli@suse.com Date: Wed, 26 Sep 2018 18:12:21 +0100 Message-Id: <20180926171236.45203-2-dgilbert@redhat.com> In-Reply-To: <20180926171236.45203-1-dgilbert@redhat.com> References: <20180926171236.45203-1-dgilbert@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.27 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.47]); Wed, 26 Sep 2018 17:12:48 +0000 (UTC) Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PULL 01/16] migration/rdma: Fix uninitialised rdma_return_path X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: quintela@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RDMRC_1 RSF_0 Z_629925259 SPT_0 Content-Type: text/plain; charset="utf-8" From: "Dr. David Alan Gilbert" Clang correctly errors out moaning that rdma_return_path is used uninitialised in the earlier error paths. Make it NULL so that the error path ignores it. Fixes: 55cc1b5937a8e709e4c102e74b206281073aab82 Signed-off-by: Dr. David Alan Gilbert Reported-by: Cornelia Huck Message-Id: <20180830173657.22939-1-dgilbert@redhat.com> Reviewed-by: Philippe Mathieu-Daud=C3=A9 Reviewed-by: Juan Quintela Signed-off-by: Juan Quintela Signed-off-by: Dr. David Alan Gilbert --- migration/rdma.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/migration/rdma.c b/migration/rdma.c index ae07515e83..9b2e7e10aa 100644 --- a/migration/rdma.c +++ b/migration/rdma.c @@ -4012,7 +4012,7 @@ static void rdma_accept_incoming_migration(void *opaq= ue) void rdma_start_incoming_migration(const char *host_port, Error **errp) { int ret; - RDMAContext *rdma, *rdma_return_path; + RDMAContext *rdma, *rdma_return_path =3D NULL; Error *local_err =3D NULL; =20 trace_rdma_start_incoming_migration(); --=20 2.17.1 From nobody Tue May 7 18:44:44 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1537982181756859.86113148054; Wed, 26 Sep 2018 10:16:21 -0700 (PDT) Received: from localhost ([::1]:59877 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1g5DQC-0002eJ-En for importer@patchew.org; Wed, 26 Sep 2018 13:16:20 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:52606) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1g5DMq-0000Qy-JW for qemu-devel@nongnu.org; Wed, 26 Sep 2018 13:12:53 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1g5DMp-0006gB-HW for qemu-devel@nongnu.org; Wed, 26 Sep 2018 13:12:52 -0400 Received: from mx1.redhat.com ([209.132.183.28]:49268) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1g5DMp-0006fC-8r for qemu-devel@nongnu.org; Wed, 26 Sep 2018 13:12:51 -0400 Received: from smtp.corp.redhat.com (int-mx12.intmail.prod.int.phx2.redhat.com [10.5.11.27]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 76949308212A; Wed, 26 Sep 2018 17:12:50 +0000 (UTC) Received: from dgilbert-t530.redhat.com (unknown [10.36.118.26]) by smtp.corp.redhat.com (Postfix) with ESMTP id AD6CDDB561; Wed, 26 Sep 2018 17:12:48 +0000 (UTC) From: "Dr. David Alan Gilbert (git)" To: qemu-devel@nongnu.org, xiaoguangrong@tencent.com, joserz@linux.ibm.com, wei@redhat.com, thuth@redhat.com, marcandre.lureau@redhat.com, fli@suse.com Date: Wed, 26 Sep 2018 18:12:22 +0100 Message-Id: <20180926171236.45203-3-dgilbert@redhat.com> In-Reply-To: <20180926171236.45203-1-dgilbert@redhat.com> References: <20180926171236.45203-1-dgilbert@redhat.com> X-Scanned-By: MIMEDefang 2.84 on 10.5.11.27 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.42]); Wed, 26 Sep 2018 17:12:50 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PULL 02/16] migration: fix calculating xbzrle_counters.cache_miss_rate X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: quintela@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RDMRC_1 RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: Xiao Guangrong As Peter pointed out: | - xbzrle_counters.cache_miss is done in save_xbzrle_page(), so it's | per-guest-page granularity | | - RAMState.iterations is done for each ram_find_and_save_block(), so | it's per-host-page granularity | | An example is that when we migrate a 2M huge page in the guest, we | will only increase the RAMState.iterations by 1 (since | ram_find_and_save_block() will be called once), but we might increase | xbzrle_counters.cache_miss for 2M/4K=3D512 times (we'll call | save_xbzrle_page() that many times) if all the pages got cache miss. | Then IMHO the cache miss rate will be 512/1=3D51200% (while it should | actually be just 100% cache miss). And he also suggested as xbzrle_counters.cache_miss_rate is the only user of rs->iterations we can adapt it to count target guest page numbers After that, rename 'iterations' to 'target_page_count' to better reflect its meaning Suggested-by: Peter Xu Reviewed-by: Peter Xu Reviewed-by: Juan Quintela Signed-off-by: Xiao Guangrong Message-Id: <20180903092644.25812-3-xiaoguangrong@tencent.com> Signed-off-by: Juan Quintela Signed-off-by: Dr. David Alan Gilbert --- migration/ram.c | 18 +++++++++--------- 1 file changed, 9 insertions(+), 9 deletions(-) diff --git a/migration/ram.c b/migration/ram.c index f6fd8e5e09..0423ef0f08 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -301,10 +301,10 @@ struct RAMState { uint64_t num_dirty_pages_period; /* xbzrle misses since the beginning of the period */ uint64_t xbzrle_cache_miss_prev; - /* number of iterations at the beginning of period */ - uint64_t iterations_prev; - /* Iterations since start */ - uint64_t iterations; + /* total handled target pages at the beginning of period */ + uint64_t target_page_count_prev; + /* total handled target pages since start */ + uint64_t target_page_count; /* number of dirty bits in the bitmap */ uint64_t migration_dirty_pages; /* protects modification of the bitmap */ @@ -1592,19 +1592,19 @@ uint64_t ram_pagesize_summary(void) =20 static void migration_update_rates(RAMState *rs, int64_t end_time) { - uint64_t iter_count =3D rs->iterations - rs->iterations_prev; + uint64_t page_count =3D rs->target_page_count - rs->target_page_count_= prev; =20 /* calculate period counters */ ram_counters.dirty_pages_rate =3D rs->num_dirty_pages_period * 1000 / (end_time - rs->time_last_bitmap_sync); =20 - if (!iter_count) { + if (!page_count) { return; } =20 if (migrate_use_xbzrle()) { xbzrle_counters.cache_miss_rate =3D (double)(xbzrle_counters.cache= _miss - - rs->xbzrle_cache_miss_prev) / iter_count; + rs->xbzrle_cache_miss_prev) / page_count; rs->xbzrle_cache_miss_prev =3D xbzrle_counters.cache_miss; } } @@ -1662,7 +1662,7 @@ static void migration_bitmap_sync(RAMState *rs) =20 migration_update_rates(rs, end_time); =20 - rs->iterations_prev =3D rs->iterations; + rs->target_page_count_prev =3D rs->target_page_count; =20 /* reset period counters */ rs->time_last_bitmap_sync =3D end_time; @@ -3196,7 +3196,7 @@ static int ram_save_iterate(QEMUFile *f, void *opaque) done =3D 1; break; } - rs->iterations++; + rs->target_page_count +=3D pages; =20 /* we want to check in the 1st loop, just in case it was the 1st t= ime and we had to sync the dirty bitmap. --=20 2.17.1 From nobody Tue May 7 18:44:44 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1537982390084913.5495596507582; Wed, 26 Sep 2018 10:19:50 -0700 (PDT) Received: from localhost ([::1]:59889 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1g5DTZ-0005YL-3Q for importer@patchew.org; Wed, 26 Sep 2018 13:19:49 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:52617) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1g5DMs-0000SS-5Q for qemu-devel@nongnu.org; Wed, 26 Sep 2018 13:12:59 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1g5DMr-0006iF-De for qemu-devel@nongnu.org; Wed, 26 Sep 2018 13:12:54 -0400 Received: from mx1.redhat.com ([209.132.183.28]:35222) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1g5DMr-0006hX-7i for qemu-devel@nongnu.org; Wed, 26 Sep 2018 13:12:53 -0400 Received: from smtp.corp.redhat.com (int-mx12.intmail.prod.int.phx2.redhat.com [10.5.11.27]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 7A87B80464; Wed, 26 Sep 2018 17:12:52 +0000 (UTC) Received: from dgilbert-t530.redhat.com (unknown [10.36.118.26]) by smtp.corp.redhat.com (Postfix) with ESMTP id B93C3DB561; Wed, 26 Sep 2018 17:12:50 +0000 (UTC) From: "Dr. David Alan Gilbert (git)" To: qemu-devel@nongnu.org, xiaoguangrong@tencent.com, joserz@linux.ibm.com, wei@redhat.com, thuth@redhat.com, marcandre.lureau@redhat.com, fli@suse.com Date: Wed, 26 Sep 2018 18:12:23 +0100 Message-Id: <20180926171236.45203-4-dgilbert@redhat.com> In-Reply-To: <20180926171236.45203-1-dgilbert@redhat.com> References: <20180926171236.45203-1-dgilbert@redhat.com> X-Scanned-By: MIMEDefang 2.84 on 10.5.11.27 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.28]); Wed, 26 Sep 2018 17:12:52 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PULL 03/16] migration: handle the error condition properly X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: quintela@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RDMRC_1 RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: Xiao Guangrong ram_find_and_save_block() can return negative if any error hanppens, however, it is completely ignored in current code Signed-off-by: Xiao Guangrong Reviewed-by: Juan Quintela Message-Id: <20180903092644.25812-5-xiaoguangrong@tencent.com> Signed-off-by: Juan Quintela Signed-off-by: Dr. David Alan Gilbert --- migration/ram.c | 18 +++++++++++++++--- 1 file changed, 15 insertions(+), 3 deletions(-) diff --git a/migration/ram.c b/migration/ram.c index 0423ef0f08..43360f6483 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -2372,7 +2372,8 @@ static int ram_save_host_page(RAMState *rs, PageSearc= hStatus *pss, * * Called within an RCU critical section. * - * Returns the number of pages written where zero means no dirty pages + * Returns the number of pages written where zero means no dirty pages, + * or negative on error * * @rs: current RAM state * @last_stage: if we are at the completion stage @@ -3196,6 +3197,12 @@ static int ram_save_iterate(QEMUFile *f, void *opaqu= e) done =3D 1; break; } + + if (pages < 0) { + qemu_file_set_error(f, pages); + break; + } + rs->target_page_count +=3D pages; =20 /* we want to check in the 1st loop, just in case it was the 1st t= ime @@ -3238,7 +3245,7 @@ out: /** * ram_save_complete: function called to send the remaining amount of ram * - * Returns zero to indicate success + * Returns zero to indicate success or negative on error * * Called with iothread lock * @@ -3249,6 +3256,7 @@ static int ram_save_complete(QEMUFile *f, void *opaqu= e) { RAMState **temp =3D opaque; RAMState *rs =3D *temp; + int ret =3D 0; =20 rcu_read_lock(); =20 @@ -3269,6 +3277,10 @@ static int ram_save_complete(QEMUFile *f, void *opaq= ue) if (pages =3D=3D 0) { break; } + if (pages < 0) { + ret =3D pages; + break; + } } =20 flush_compressed_data(rs); @@ -3280,7 +3292,7 @@ static int ram_save_complete(QEMUFile *f, void *opaqu= e) qemu_put_be64(f, RAM_SAVE_FLAG_EOS); qemu_fflush(f); =20 - return 0; + return ret; } =20 static void ram_save_pending(QEMUFile *f, void *opaque, uint64_t max_size, --=20 2.17.1 From nobody Tue May 7 18:44:44 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1537982118807666.7221239600593; Wed, 26 Sep 2018 10:15:18 -0700 (PDT) Received: from localhost ([::1]:59870 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1g5DP8-0001ti-52 for importer@patchew.org; Wed, 26 Sep 2018 13:15:14 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:52629) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1g5DMu-0000V8-21 for qemu-devel@nongnu.org; Wed, 26 Sep 2018 13:12:59 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1g5DMt-0006kN-DX for qemu-devel@nongnu.org; Wed, 26 Sep 2018 13:12:56 -0400 Received: from mx1.redhat.com ([209.132.183.28]:37952) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1g5DMt-0006jh-76 for qemu-devel@nongnu.org; Wed, 26 Sep 2018 13:12:55 -0400 Received: from smtp.corp.redhat.com (int-mx12.intmail.prod.int.phx2.redhat.com [10.5.11.27]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 7FE03285CE; Wed, 26 Sep 2018 17:12:54 +0000 (UTC) Received: from dgilbert-t530.redhat.com (unknown [10.36.118.26]) by smtp.corp.redhat.com (Postfix) with ESMTP id C1022DB56F; Wed, 26 Sep 2018 17:12:52 +0000 (UTC) From: "Dr. David Alan Gilbert (git)" To: qemu-devel@nongnu.org, xiaoguangrong@tencent.com, joserz@linux.ibm.com, wei@redhat.com, thuth@redhat.com, marcandre.lureau@redhat.com, fli@suse.com Date: Wed, 26 Sep 2018 18:12:24 +0100 Message-Id: <20180926171236.45203-5-dgilbert@redhat.com> In-Reply-To: <20180926171236.45203-1-dgilbert@redhat.com> References: <20180926171236.45203-1-dgilbert@redhat.com> X-Scanned-By: MIMEDefang 2.84 on 10.5.11.27 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.30]); Wed, 26 Sep 2018 17:12:54 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PULL 04/16] Add a hint message to loadvm and exits on failure X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: quintela@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RDMRC_1 RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: Jose Ricardo Ziviani This patch adds a small hint for the failure case of the load snapshot process. It may be useful for users to remember that the VM configuration has changed between the save and load processes. (qemu) loadvm vm-20180903083641 Unknown savevm section or instance 'cpu_common' 4. Make sure that your current VM setup matches your saved VM setup, including= any hotplugged devices Error -22 while loading VM state (qemu) device_add host-spapr-cpu-core,core-id=3D4 (qemu) loadvm vm-20180903083641 (qemu) c (qemu) info status VM status: running It also exits Qemu if the snapshot cannot be loaded before reaching the main loop (-loadvm in the command line). $ qemu-system-ppc64 ... -loadvm vm-20180903083641 qemu-system-ppc64: Unknown savevm section or instance 'cpu_common' 4. Make sure that your current VM setup matches your saved VM setup, including= any hotplugged devices qemu-system-ppc64: Error -22 while loading VM state $ Signed-off-by: Jose Ricardo Ziviani Message-Id: <20180903162613.15877-1-joserz@linux.ibm.com> Reviewed-by: Juan Quintela Signed-off-by: Juan Quintela Signed-off-by: Dr. David Alan Gilbert --- migration/savevm.c | 4 +++- vl.c | 1 + 2 files changed, 4 insertions(+), 1 deletion(-) diff --git a/migration/savevm.c b/migration/savevm.c index 13e51f0e34..9692577318 100644 --- a/migration/savevm.c +++ b/migration/savevm.c @@ -2078,7 +2078,9 @@ qemu_loadvm_section_start_full(QEMUFile *f, Migration= IncomingState *mis) /* Find savevm section */ se =3D find_se(idstr, instance_id); if (se =3D=3D NULL) { - error_report("Unknown savevm section or instance '%s' %d", + error_report("Unknown savevm section or instance '%s' %d. " + "Make sure that your current VM setup matches your " + "saved VM setup, including any hotplugged devices", idstr, instance_id); return -EINVAL; } diff --git a/vl.c b/vl.c index 694bb67890..afbbec2383 100644 --- a/vl.c +++ b/vl.c @@ -4530,6 +4530,7 @@ int main(int argc, char **argv, char **envp) if (load_snapshot(loadvm, &local_err) < 0) { error_report_err(local_err); autostart =3D 0; + exit(1); } } =20 --=20 2.17.1 From nobody Tue May 7 18:44:44 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1537982120330331.7923821437174; Wed, 26 Sep 2018 10:15:20 -0700 (PDT) Received: from localhost ([::1]:59871 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1g5DP8-0001u4-5I for importer@patchew.org; Wed, 26 Sep 2018 13:15:14 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:52640) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1g5DMx-0000gY-7p for qemu-devel@nongnu.org; Wed, 26 Sep 2018 13:13:00 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1g5DMw-0006nV-EC for qemu-devel@nongnu.org; Wed, 26 Sep 2018 13:12:59 -0400 Received: from mx1.redhat.com ([209.132.183.28]:34372) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1g5DMw-0006mf-70 for qemu-devel@nongnu.org; Wed, 26 Sep 2018 13:12:58 -0400 Received: from smtp.corp.redhat.com (int-mx12.intmail.prod.int.phx2.redhat.com [10.5.11.27]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 6DDCD307D865; Wed, 26 Sep 2018 17:12:57 +0000 (UTC) Received: from dgilbert-t530.redhat.com (unknown [10.36.118.26]) by smtp.corp.redhat.com (Postfix) with ESMTP id C8735DB561; Wed, 26 Sep 2018 17:12:54 +0000 (UTC) From: "Dr. David Alan Gilbert (git)" To: qemu-devel@nongnu.org, xiaoguangrong@tencent.com, joserz@linux.ibm.com, wei@redhat.com, thuth@redhat.com, marcandre.lureau@redhat.com, fli@suse.com Date: Wed, 26 Sep 2018 18:12:25 +0100 Message-Id: <20180926171236.45203-6-dgilbert@redhat.com> In-Reply-To: <20180926171236.45203-1-dgilbert@redhat.com> References: <20180926171236.45203-1-dgilbert@redhat.com> X-Scanned-By: MIMEDefang 2.84 on 10.5.11.27 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.48]); Wed, 26 Sep 2018 17:12:57 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PULL 05/16] migration: do not flush_compressed_data at the end of iteration X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: quintela@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RDMRC_1 RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: Xiao Guangrong flush_compressed_data() needs to wait all compression threads to finish their work, after that all threads are free until the migration feeds new request to them, reducing its call can improve the throughput and use CPU resource more effectively We do not need to flush all threads at the end of iteration, the data can be kept locally until the memory block is changed or memory migration starts over in that case we will meet a dirtied page which may still exists in compression threads's ring Signed-off-by: Xiao Guangrong Reviewed-by: Juan Quintela Message-Id: <20180906070101.27280-2-xiaoguangrong@tencent.com> Signed-off-by: Juan Quintela Signed-off-by: Dr. David Alan Gilbert --- migration/ram.c | 18 +++++++++++------- 1 file changed, 11 insertions(+), 7 deletions(-) diff --git a/migration/ram.c b/migration/ram.c index 43360f6483..2c039892d3 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -1996,17 +1996,22 @@ static bool find_dirty_block(RAMState *rs, PageSear= chStatus *pss, bool *again) pss->page =3D 0; pss->block =3D QLIST_NEXT_RCU(pss->block, next); if (!pss->block) { + /* + * If memory migration starts over, we will meet a dirtied page + * which may still exists in compression threads's ring, so we + * should flush the compressed data to make sure the new page + * is not overwritten by the old one in the destination. + * + * Also If xbzrle is on, stop using the data compression at th= is + * point. In theory, xbzrle can do better than compression. + */ + flush_compressed_data(rs); + /* Hit the end of the list */ pss->block =3D QLIST_FIRST_RCU(&ram_list.blocks); /* Flag that we've looped */ pss->complete_round =3D true; rs->ram_bulk_stage =3D false; - if (migrate_use_xbzrle()) { - /* If xbzrle is on, stop using the data compression at this - * point. In theory, xbzrle can do better than compression. - */ - flush_compressed_data(rs); - } } /* Didn't find anything this time, but try again on the new block = */ *again =3D true; @@ -3219,7 +3224,6 @@ static int ram_save_iterate(QEMUFile *f, void *opaque) } i++; } - flush_compressed_data(rs); rcu_read_unlock(); =20 /* --=20 2.17.1 From nobody Tue May 7 18:44:44 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1537982578124990.0362213653576; Wed, 26 Sep 2018 10:22:58 -0700 (PDT) Received: from localhost ([::1]:59908 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1g5DWb-0008JZ-0e for importer@patchew.org; Wed, 26 Sep 2018 13:22:57 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:52690) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1g5DNC-0000qu-HT for qemu-devel@nongnu.org; Wed, 26 Sep 2018 13:13:15 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1g5DN9-0007Bs-9R for qemu-devel@nongnu.org; Wed, 26 Sep 2018 13:13:14 -0400 Received: from mx1.redhat.com ([209.132.183.28]:51442) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1g5DN8-00070f-Tc for qemu-devel@nongnu.org; Wed, 26 Sep 2018 13:13:11 -0400 Received: from smtp.corp.redhat.com (int-mx12.intmail.prod.int.phx2.redhat.com [10.5.11.27]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 4526330001EF; Wed, 26 Sep 2018 17:13:09 +0000 (UTC) Received: from dgilbert-t530.redhat.com (unknown [10.36.118.26]) by smtp.corp.redhat.com (Postfix) with ESMTP id B4986DB570; Wed, 26 Sep 2018 17:12:57 +0000 (UTC) From: "Dr. David Alan Gilbert (git)" To: qemu-devel@nongnu.org, xiaoguangrong@tencent.com, joserz@linux.ibm.com, wei@redhat.com, thuth@redhat.com, marcandre.lureau@redhat.com, fli@suse.com Date: Wed, 26 Sep 2018 18:12:26 +0100 Message-Id: <20180926171236.45203-7-dgilbert@redhat.com> In-Reply-To: <20180926171236.45203-1-dgilbert@redhat.com> References: <20180926171236.45203-1-dgilbert@redhat.com> X-Scanned-By: MIMEDefang 2.84 on 10.5.11.27 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.46]); Wed, 26 Sep 2018 17:13:09 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PULL 06/16] migration: show the statistics of compression X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: quintela@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RDMRC_1 RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: Xiao Guangrong Currently, it includes: pages: amount of pages compressed and transferred to the target VM busy: amount of count that no free thread to compress data busy-rate: rate of thread busy compressed-size: amount of bytes after compression compression-rate: rate of compressed size Reviewed-by: Juan Quintela Reviewed-by: Peter Xu Signed-off-by: Xiao Guangrong Message-Id: <20180906070101.27280-3-xiaoguangrong@tencent.com> Signed-off-by: Juan Quintela Signed-off-by: Dr. David Alan Gilbert --- hmp.c | 13 +++++++++++++ migration/migration.c | 12 ++++++++++++ migration/ram.c | 41 ++++++++++++++++++++++++++++++++++++++++- migration/ram.h | 1 + qapi/migration.json | 26 +++++++++++++++++++++++++- 5 files changed, 91 insertions(+), 2 deletions(-) diff --git a/hmp.c b/hmp.c index 3a9f797677..61ef120423 100644 --- a/hmp.c +++ b/hmp.c @@ -271,6 +271,19 @@ void hmp_info_migrate(Monitor *mon, const QDict *qdict) info->xbzrle_cache->overflow); } =20 + if (info->has_compression) { + monitor_printf(mon, "compression pages: %" PRIu64 " pages\n", + info->compression->pages); + monitor_printf(mon, "compression busy: %" PRIu64 "\n", + info->compression->busy); + monitor_printf(mon, "compression busy rate: %0.2f\n", + info->compression->busy_rate); + monitor_printf(mon, "compressed size: %" PRIu64 "\n", + info->compression->compressed_size); + monitor_printf(mon, "compression rate: %0.2f\n", + info->compression->compression_rate); + } + if (info->has_cpu_throttle_percentage) { monitor_printf(mon, "cpu throttle percentage: %" PRIu64 "\n", info->cpu_throttle_percentage); diff --git a/migration/migration.c b/migration/migration.c index 05d0a7296a..6a7731de50 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -758,6 +758,18 @@ static void populate_ram_info(MigrationInfo *info, Mig= rationState *s) info->xbzrle_cache->overflow =3D xbzrle_counters.overflow; } =20 + if (migrate_use_compression()) { + info->has_compression =3D true; + info->compression =3D g_malloc0(sizeof(*info->compression)); + info->compression->pages =3D compression_counters.pages; + info->compression->busy =3D compression_counters.busy; + info->compression->busy_rate =3D compression_counters.busy_rate; + info->compression->compressed_size =3D + compression_counters.compressed_size; + info->compression->compression_rate =3D + compression_counters.compression_rate; + } + if (cpu_throttle_active()) { info->has_cpu_throttle_percentage =3D true; info->cpu_throttle_percentage =3D cpu_throttle_get_percentage(); diff --git a/migration/ram.c b/migration/ram.c index 2c039892d3..7c12f2792c 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -301,6 +301,15 @@ struct RAMState { uint64_t num_dirty_pages_period; /* xbzrle misses since the beginning of the period */ uint64_t xbzrle_cache_miss_prev; + + /* compression statistics since the beginning of the period */ + /* amount of count that no free thread to compress data */ + uint64_t compress_thread_busy_prev; + /* amount bytes after compression */ + uint64_t compressed_size_prev; + /* amount of compressed pages */ + uint64_t compress_pages_prev; + /* total handled target pages at the beginning of period */ uint64_t target_page_count_prev; /* total handled target pages since start */ @@ -338,6 +347,8 @@ struct PageSearchStatus { }; typedef struct PageSearchStatus PageSearchStatus; =20 +CompressionStats compression_counters; + struct CompressParam { bool done; bool quit; @@ -1593,6 +1604,7 @@ uint64_t ram_pagesize_summary(void) static void migration_update_rates(RAMState *rs, int64_t end_time) { uint64_t page_count =3D rs->target_page_count - rs->target_page_count_= prev; + double compressed_size; =20 /* calculate period counters */ ram_counters.dirty_pages_rate =3D rs->num_dirty_pages_period * 1000 @@ -1607,6 +1619,26 @@ static void migration_update_rates(RAMState *rs, int= 64_t end_time) rs->xbzrle_cache_miss_prev) / page_count; rs->xbzrle_cache_miss_prev =3D xbzrle_counters.cache_miss; } + + if (migrate_use_compression()) { + compression_counters.busy_rate =3D (double)(compression_counters.b= usy - + rs->compress_thread_busy_prev) / page_count; + rs->compress_thread_busy_prev =3D compression_counters.busy; + + compressed_size =3D compression_counters.compressed_size - + rs->compressed_size_prev; + if (compressed_size) { + double uncompressed_size =3D (compression_counters.pages - + rs->compress_pages_prev) * TARGET_PAGE= _SIZE; + + /* Compression-Ratio =3D Uncompressed-size / Compressed-size */ + compression_counters.compression_rate =3D + uncompressed_size / compressed_siz= e; + + rs->compress_pages_prev =3D compression_counters.pages; + rs->compressed_size_prev =3D compression_counters.compressed_s= ize; + } + } } =20 static void migration_bitmap_sync(RAMState *rs) @@ -1888,10 +1920,16 @@ exit: static void update_compress_thread_counts(const CompressParam *param, int bytes_xmit) { + ram_counters.transferred +=3D bytes_xmit; + if (param->zero_page) { ram_counters.duplicate++; + return; } - ram_counters.transferred +=3D bytes_xmit; + + /* 8 means a header with RAM_SAVE_FLAG_CONTINUE. */ + compression_counters.compressed_size +=3D bytes_xmit - 8; + compression_counters.pages++; } =20 static void flush_compressed_data(RAMState *rs) @@ -2264,6 +2302,7 @@ static bool save_compress_page(RAMState *rs, RAMBlock= *block, ram_addr_t offset) return true; } =20 + compression_counters.busy++; return false; } =20 diff --git a/migration/ram.h b/migration/ram.h index 457bf54b8c..a139066846 100644 --- a/migration/ram.h +++ b/migration/ram.h @@ -36,6 +36,7 @@ =20 extern MigrationStats ram_counters; extern XBZRLECacheStats xbzrle_counters; +extern CompressionStats compression_counters; =20 int xbzrle_cache_resize(int64_t new_size, Error **errp); uint64_t ram_bytes_remaining(void); diff --git a/qapi/migration.json b/qapi/migration.json index f62d3f9a4b..6e8c21258a 100644 --- a/qapi/migration.json +++ b/qapi/migration.json @@ -75,6 +75,27 @@ 'cache-miss': 'int', 'cache-miss-rate': 'number', 'overflow': 'int' } } =20 +## +# @CompressionStats: +# +# Detailed migration compression statistics +# +# @pages: amount of pages compressed and transferred to the target VM +# +# @busy: count of times that no free thread was available to compress data +# +# @busy-rate: rate of thread busy +# +# @compressed-size: amount of bytes after compression +# +# @compression-rate: rate of compressed size +# +# Since: 3.1 +## +{ 'struct': 'CompressionStats', + 'data': {'pages': 'int', 'busy': 'int', 'busy-rate': 'number', + 'compressed-size': 'int', 'compression-rate': 'number' } } + ## # @MigrationStatus: # @@ -172,6 +193,8 @@ # only present when the postcopy-blocktime migration capability # is enabled. (Since 3.0) # +# @compression: migration compression statistics, only returned if compres= sion +# feature is on and status is 'active' or 'completed' (Since 3.1) # # Since: 0.14.0 ## @@ -186,7 +209,8 @@ '*cpu-throttle-percentage': 'int', '*error-desc': 'str', '*postcopy-blocktime' : 'uint32', - '*postcopy-vcpu-blocktime': ['uint32']} } + '*postcopy-vcpu-blocktime': ['uint32'], + '*compression': 'CompressionStats'} } =20 ## # @query-migrate: --=20 2.17.1 From nobody Tue May 7 18:44:44 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1537982334915546.405438414468; Wed, 26 Sep 2018 10:18:54 -0700 (PDT) Received: from localhost ([::1]:59887 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1g5DSc-0004Xf-3i for importer@patchew.org; Wed, 26 Sep 2018 13:18:50 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:52691) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1g5DNC-0000qv-HV for qemu-devel@nongnu.org; Wed, 26 Sep 2018 13:13:15 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1g5DNB-0007SK-61 for qemu-devel@nongnu.org; Wed, 26 Sep 2018 13:13:14 -0400 Received: from mx1.redhat.com ([209.132.183.28]:49578) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1g5DNA-0007MN-W7 for qemu-devel@nongnu.org; Wed, 26 Sep 2018 13:13:13 -0400 Received: from smtp.corp.redhat.com (int-mx12.intmail.prod.int.phx2.redhat.com [10.5.11.27]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 47C60C05681F; Wed, 26 Sep 2018 17:13:12 +0000 (UTC) Received: from dgilbert-t530.redhat.com (unknown [10.36.118.26]) by smtp.corp.redhat.com (Postfix) with ESMTP id 8A554DB561; Wed, 26 Sep 2018 17:13:09 +0000 (UTC) From: "Dr. David Alan Gilbert (git)" To: qemu-devel@nongnu.org, xiaoguangrong@tencent.com, joserz@linux.ibm.com, wei@redhat.com, thuth@redhat.com, marcandre.lureau@redhat.com, fli@suse.com Date: Wed, 26 Sep 2018 18:12:27 +0100 Message-Id: <20180926171236.45203-8-dgilbert@redhat.com> In-Reply-To: <20180926171236.45203-1-dgilbert@redhat.com> References: <20180926171236.45203-1-dgilbert@redhat.com> X-Scanned-By: MIMEDefang 2.84 on 10.5.11.27 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.32]); Wed, 26 Sep 2018 17:13:12 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PULL 07/16] migration: use save_page_use_compression in flush_compressed_data X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: quintela@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RDMRC_1 RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: Xiao Guangrong It avoids to touch compression locks if xbzrle and compression are both enabled Signed-off-by: Xiao Guangrong Reviewed-by: Juan Quintela Message-Id: <20180906070101.27280-4-xiaoguangrong@tencent.com> Signed-off-by: Juan Quintela Signed-off-by: Dr. David Alan Gilbert --- migration/ram.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/migration/ram.c b/migration/ram.c index 7c12f2792c..0fdaa8efa3 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -1932,11 +1932,13 @@ update_compress_thread_counts(const CompressParam *= param, int bytes_xmit) compression_counters.pages++; } =20 +static bool save_page_use_compression(RAMState *rs); + static void flush_compressed_data(RAMState *rs) { int idx, len, thread_count; =20 - if (!migrate_use_compression()) { + if (!save_page_use_compression(rs)) { return; } thread_count =3D migrate_compress_threads(); --=20 2.17.1 From nobody Tue May 7 18:44:44 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1537982520374443.6728473244972; Wed, 26 Sep 2018 10:22:00 -0700 (PDT) Received: from localhost ([::1]:59905 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1g5DVa-0007Iu-KG for importer@patchew.org; Wed, 26 Sep 2018 13:21:54 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:52724) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1g5DNE-0000sq-DK for qemu-devel@nongnu.org; Wed, 26 Sep 2018 13:13:17 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1g5DND-0007j1-Cd for qemu-devel@nongnu.org; Wed, 26 Sep 2018 13:13:16 -0400 Received: from mx1.redhat.com ([209.132.183.28]:45490) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1g5DND-0007c7-2k for qemu-devel@nongnu.org; Wed, 26 Sep 2018 13:13:15 -0400 Received: from smtp.corp.redhat.com (int-mx12.intmail.prod.int.phx2.redhat.com [10.5.11.27]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 54AD7C04BD3B; Wed, 26 Sep 2018 17:13:14 +0000 (UTC) Received: from dgilbert-t530.redhat.com (unknown [10.36.118.26]) by smtp.corp.redhat.com (Postfix) with ESMTP id 8E1C0DB561; Wed, 26 Sep 2018 17:13:12 +0000 (UTC) From: "Dr. David Alan Gilbert (git)" To: qemu-devel@nongnu.org, xiaoguangrong@tencent.com, joserz@linux.ibm.com, wei@redhat.com, thuth@redhat.com, marcandre.lureau@redhat.com, fli@suse.com Date: Wed, 26 Sep 2018 18:12:28 +0100 Message-Id: <20180926171236.45203-9-dgilbert@redhat.com> In-Reply-To: <20180926171236.45203-1-dgilbert@redhat.com> References: <20180926171236.45203-1-dgilbert@redhat.com> X-Scanned-By: MIMEDefang 2.84 on 10.5.11.27 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.31]); Wed, 26 Sep 2018 17:13:14 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PULL 08/16] tests/migration: Convert x86 boot block compilation script into Makefile X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: quintela@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RDMRC_1 RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: Wei Huang The x86 boot block header currently is generated with a shell script. To better support other CPUs (e.g. aarch64), we convert the script into Makefile. This allows us to 1) support cross-compilation easily, and 2) avoid creating a script file for every architecture. Note that, in the new design, the cross compiler prefix can be specified by setting the CROSS_PREFIX in "make" command. Also to allow gcc pre-processor to include the C-style file correctly, it also renames the x86-a-b-bootblock.s file extension from .s to .S. Reviewed-by: Juan Quintela Reviewed-by: Andrew Jones Signed-off-by: Wei Huang Message-Id: <1536174934-26022-2-git-send-email-wei@redhat.com> Signed-off-by: Juan Quintela Signed-off-by: Dr. David Alan Gilbert --- tests/migration/Makefile | 31 +++++++++++++++++ tests/migration/rebuild-x86-bootblock.sh | 33 ------------------- ...86-a-b-bootblock.s =3D> x86-a-b-bootblock.S} | 0 3 files changed, 31 insertions(+), 33 deletions(-) create mode 100644 tests/migration/Makefile delete mode 100755 tests/migration/rebuild-x86-bootblock.sh rename tests/migration/{x86-a-b-bootblock.s =3D> x86-a-b-bootblock.S} (100= %) diff --git a/tests/migration/Makefile b/tests/migration/Makefile new file mode 100644 index 0000000000..c0824b4fbe --- /dev/null +++ b/tests/migration/Makefile @@ -0,0 +1,31 @@ +# To specify cross compiler prefix, use CROSS_PREFIX=3D +# $ make CROSS_PREFIX=3Dx86_64-linux-gnu- + +override define __note +/* This file is automatically generated from + * tests/migration/x86-a-b-bootblock.S, edit that and then run + * tests/migration/rebuild-x86-bootblock.sh to update, + * and then remember to send both in your patch submission. + */ +endef +export __note + +.PHONY: all clean +all: x86-a-b-bootblock.h + +x86-a-b-bootblock.h: x86.bootsect + echo "$$__note" > header.tmp + xxd -i $< | sed -e 's/.*int.*//' >> header.tmp + mv header.tmp $@ + +x86.bootsect: x86.boot + dd if=3D$< of=3D$@ bs=3D256 count=3D2 skip=3D124 + +x86.boot: x86.o + $(CROSS_PREFIX)objcopy -O binary $< $@ + +x86.o: x86-a-b-bootblock.S + $(CROSS_PREFIX)gcc -m32 -march=3Di486 -c $< -o $@ + +clean: + @rm -rf *.boot *.o *.bootsect diff --git a/tests/migration/rebuild-x86-bootblock.sh b/tests/migration/reb= uild-x86-bootblock.sh deleted file mode 100755 index 86cec5d284..0000000000 --- a/tests/migration/rebuild-x86-bootblock.sh +++ /dev/null @@ -1,33 +0,0 @@ -#!/bin/sh -# Copyright (c) 2016-2018 Red Hat, Inc. and/or its affiliates -# This work is licensed under the terms of the GNU GPL, version 2 or later. -# See the COPYING file in the top-level directory. -# -# Author: dgilbert@redhat.com - -ASMFILE=3D$PWD/tests/migration/x86-a-b-bootblock.s -HEADER=3D$PWD/tests/migration/x86-a-b-bootblock.h - -if [ ! -e "$ASMFILE" ] -then - echo "Couldn't find $ASMFILE" >&2 - exit 1 -fi - -ASM_WORK_DIR=3D$(mktemp -d --tmpdir X86BB.XXXXXX) -cd "$ASM_WORK_DIR" && -as --32 -march=3Di486 "$ASMFILE" -o x86.o && -objcopy -O binary x86.o x86.boot && -dd if=3Dx86.boot of=3Dx86.bootsect bs=3D256 count=3D2 skip=3D124 && -xxd -i x86.bootsect | -sed -e 's/.*int.*//' > x86.hex && -cat - x86.hex < "$HEADER" -/* This file is automatically generated from - * tests/migration/x86-a-b-bootblock.s, edit that and then run - * tests/migration/rebuild-x86-bootblock.sh to update, - * and then remember to send both in your patch submission. - */ -HERE - -rm x86.hex x86.bootsect x86.boot x86.o -cd .. && rmdir "$ASM_WORK_DIR" diff --git a/tests/migration/x86-a-b-bootblock.s b/tests/migration/x86-a-b-= bootblock.S similarity index 100% rename from tests/migration/x86-a-b-bootblock.s rename to tests/migration/x86-a-b-bootblock.S --=20 2.17.1 From nobody Tue May 7 18:44:44 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1537982526225761.9089340696097; Wed, 26 Sep 2018 10:22:06 -0700 (PDT) Received: from localhost ([::1]:59906 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1g5DVk-0007SM-KK for importer@patchew.org; Wed, 26 Sep 2018 13:22:04 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:52743) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1g5DNG-0000vD-Fh for qemu-devel@nongnu.org; Wed, 26 Sep 2018 13:13:19 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1g5DNF-00080s-BA for qemu-devel@nongnu.org; Wed, 26 Sep 2018 13:13:18 -0400 Received: from mx1.redhat.com ([209.132.183.28]:50294) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1g5DNF-0007v0-2k for qemu-devel@nongnu.org; Wed, 26 Sep 2018 13:13:17 -0400 Received: from smtp.corp.redhat.com (int-mx12.intmail.prod.int.phx2.redhat.com [10.5.11.27]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 5BEA33084049; Wed, 26 Sep 2018 17:13:16 +0000 (UTC) Received: from dgilbert-t530.redhat.com (unknown [10.36.118.26]) by smtp.corp.redhat.com (Postfix) with ESMTP id 9A061DB561; Wed, 26 Sep 2018 17:13:14 +0000 (UTC) From: "Dr. David Alan Gilbert (git)" To: qemu-devel@nongnu.org, xiaoguangrong@tencent.com, joserz@linux.ibm.com, wei@redhat.com, thuth@redhat.com, marcandre.lureau@redhat.com, fli@suse.com Date: Wed, 26 Sep 2018 18:12:29 +0100 Message-Id: <20180926171236.45203-10-dgilbert@redhat.com> In-Reply-To: <20180926171236.45203-1-dgilbert@redhat.com> References: <20180926171236.45203-1-dgilbert@redhat.com> X-Scanned-By: MIMEDefang 2.84 on 10.5.11.27 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.40]); Wed, 26 Sep 2018 17:13:16 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PULL 09/16] tests/migration: Support cross compilation in generating boot header file X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: quintela@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RDMRC_1 RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: Wei Huang Recently a new configure option, CROSS_CC_GUEST, was added to $(TARGET)-softmmu/config-target.mak to support TCG-related tests. This patch tries to leverage this option to support cross compilation when the migration boot block file is being re-generated: * The x86 related files are moved to a new sub-dir (named ./i386). * A new top-layer Makefile is created in tests/migration/ directory. This Makefile searches and parses CROSS_CC_GUEST to generate CROSS_PREFI= X. The CROSS_PREFIX, if available, is then passed to migration/$ARCH/Makefi= le. Reviewed-by: Juan Quintela Signed-off-by: Wei Huang Message-Id: <1536174934-26022-3-git-send-email-wei@redhat.com> Signed-off-by: Juan Quintela Signed-off-by: Dr. David Alan Gilbert --- tests/migration-test.c | 2 +- tests/migration/Makefile | 44 ++++++++++--------- tests/migration/i386/Makefile | 22 ++++++++++ .../a-b-bootblock.S} | 4 -- .../a-b-bootblock.h} | 8 ++-- 5 files changed, 51 insertions(+), 29 deletions(-) create mode 100644 tests/migration/i386/Makefile rename tests/migration/{x86-a-b-bootblock.S =3D> i386/a-b-bootblock.S} (93= %) rename tests/migration/{x86-a-b-bootblock.h =3D> i386/a-b-bootblock.h} (92= %) diff --git a/tests/migration-test.c b/tests/migration-test.c index 0e687b7512..fe6b41a38d 100644 --- a/tests/migration-test.c +++ b/tests/migration-test.c @@ -83,7 +83,7 @@ static const char *tmpfs; /* A simple PC boot sector that modifies memory (1-100MB) quickly * outputting a 'B' every so often if it's still running. */ -#include "tests/migration/x86-a-b-bootblock.h" +#include "tests/migration/i386/a-b-bootblock.h" =20 static void init_bootfile_x86(const char *bootpath) { diff --git a/tests/migration/Makefile b/tests/migration/Makefile index c0824b4fbe..dc3b551976 100644 --- a/tests/migration/Makefile +++ b/tests/migration/Makefile @@ -1,31 +1,35 @@ -# To specify cross compiler prefix, use CROSS_PREFIX=3D -# $ make CROSS_PREFIX=3Dx86_64-linux-gnu- +# +# Copyright (c) 2018 Red Hat, Inc. and/or its affiliates +# +# This work is licensed under the terms of the GNU GPL, version 2 or later. +# See the COPYING file in the top-level directory. +# + +TARGET_LIST =3D i386 + +SRC_PATH =3D ../.. =20 override define __note -/* This file is automatically generated from - * tests/migration/x86-a-b-bootblock.S, edit that and then run - * tests/migration/rebuild-x86-bootblock.sh to update, - * and then remember to send both in your patch submission. +/* This file is automatically generated from the assembly file in + * tests/migration/$@. Edit that file and then run "make all" + * inside tests/migration to update, and then remember to send both + * the header and the assembler differences in your patch submission. */ endef export __note =20 -.PHONY: all clean -all: x86-a-b-bootblock.h - -x86-a-b-bootblock.h: x86.bootsect - echo "$$__note" > header.tmp - xxd -i $< | sed -e 's/.*int.*//' >> header.tmp - mv header.tmp $@ +find-arch-cross-cc =3D $(lastword $(shell grep -h "CROSS_CC_GUEST=3D" $(wi= ldcard $(SRC_PATH)/$(patsubst i386,*86*,$(1))-softmmu/config-target.mak) /d= ev/null)) +parse-cross-prefix =3D $(subst gcc,,$(patsubst cc,gcc,$(patsubst CROSS_CC_= GUEST=3D"%",%,$(call find-arch-cross-cc,$(1))))) +gen-cross-prefix =3D $(patsubst %-,CROSS_PREFIX=3D%-,$(call parse-cross-pr= efix,$(1))) =20 -x86.bootsect: x86.boot - dd if=3D$< of=3D$@ bs=3D256 count=3D2 skip=3D124 +.PHONY: all $(TARGET_LIST) =20 -x86.boot: x86.o - $(CROSS_PREFIX)objcopy -O binary $< $@ +all: $(TARGET_LIST) =20 -x86.o: x86-a-b-bootblock.S - $(CROSS_PREFIX)gcc -m32 -march=3Di486 -c $< -o $@ +$(TARGET_LIST): + $(MAKE) -C $@ $(call gen-cross-prefix,$@) =20 clean: - @rm -rf *.boot *.o *.bootsect + for target in $(TARGET_LIST); do \ + $(MAKE) -C $$target clean; \ + done diff --git a/tests/migration/i386/Makefile b/tests/migration/i386/Makefile new file mode 100644 index 0000000000..5c0324134a --- /dev/null +++ b/tests/migration/i386/Makefile @@ -0,0 +1,22 @@ +# To specify cross compiler prefix, use CROSS_PREFIX=3D +# $ make CROSS_PREFIX=3Dx86_64-linux-gnu- + +.PHONY: all clean +all: a-b-bootblock.h + +a-b-bootblock.h: x86.bootsect + echo "$$__note" > header.tmp + xxd -i $< | sed -e 's/.*int.*//' >> header.tmp + mv header.tmp $@ + +x86.bootsect: x86.boot + dd if=3D$< of=3D$@ bs=3D256 count=3D2 skip=3D124 + +x86.boot: x86.o + $(CROSS_PREFIX)objcopy -O binary $< $@ + +x86.o: a-b-bootblock.S + $(CROSS_PREFIX)gcc -m32 -march=3Di486 -c $< -o $@ + +clean: + @rm -rf *.boot *.o *.bootsect diff --git a/tests/migration/x86-a-b-bootblock.S b/tests/migration/i386/a-b= -bootblock.S similarity index 93% rename from tests/migration/x86-a-b-bootblock.S rename to tests/migration/i386/a-b-bootblock.S index b1642641a7..3f97f28023 100644 --- a/tests/migration/x86-a-b-bootblock.S +++ b/tests/migration/i386/a-b-bootblock.S @@ -3,10 +3,6 @@ # range. # Outputs an initial 'A' on serial followed by repeated 'B's # -# run tests/migration/rebuild-x86-bootblock.sh -# to regenerate the hex, and remember to include both the .h and .s -# in any patches. -# # Copyright (c) 2016 Red Hat, Inc. and/or its affiliates # This work is licensed under the terms of the GNU GPL, version 2 or later. # See the COPYING file in the top-level directory. diff --git a/tests/migration/x86-a-b-bootblock.h b/tests/migration/i386/a-b= -bootblock.h similarity index 92% rename from tests/migration/x86-a-b-bootblock.h rename to tests/migration/i386/a-b-bootblock.h index 78a151fe2a..7d459d4fde 100644 --- a/tests/migration/x86-a-b-bootblock.h +++ b/tests/migration/i386/a-b-bootblock.h @@ -1,7 +1,7 @@ -/* This file is automatically generated from - * tests/migration/x86-a-b-bootblock.s, edit that and then run - * tests/migration/rebuild-x86-bootblock.sh to update, - * and then remember to send both in your patch submission. +/* This file is automatically generated from the assembly file in + * tests/migration/i386. Edit that file and then run "make all" + * inside tests/migration to update, and then remember to send both + * the header and the assembler differences in your patch submission. */ unsigned char x86_bootsect[] =3D { 0xfa, 0x0f, 0x01, 0x16, 0x74, 0x7c, 0x66, 0xb8, 0x01, 0x00, 0x00, 0x00, --=20 2.17.1 From nobody Tue May 7 18:44:44 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1537982705806268.7470676052891; Wed, 26 Sep 2018 10:25:05 -0700 (PDT) Received: from localhost ([::1]:59919 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1g5DYb-0002hV-9u for importer@patchew.org; Wed, 26 Sep 2018 13:25:01 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:52754) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1g5DNI-0000wv-9R for qemu-devel@nongnu.org; Wed, 26 Sep 2018 13:13:23 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1g5DNH-00083C-Bq for qemu-devel@nongnu.org; Wed, 26 Sep 2018 13:13:20 -0400 Received: from mx1.redhat.com ([209.132.183.28]:43062) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1g5DNH-00082Q-3N for qemu-devel@nongnu.org; Wed, 26 Sep 2018 13:13:19 -0400 Received: from smtp.corp.redhat.com (int-mx12.intmail.prod.int.phx2.redhat.com [10.5.11.27]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 656B53082B23; Wed, 26 Sep 2018 17:13:18 +0000 (UTC) Received: from dgilbert-t530.redhat.com (unknown [10.36.118.26]) by smtp.corp.redhat.com (Postfix) with ESMTP id A3676DB56F; Wed, 26 Sep 2018 17:13:16 +0000 (UTC) From: "Dr. David Alan Gilbert (git)" To: qemu-devel@nongnu.org, xiaoguangrong@tencent.com, joserz@linux.ibm.com, wei@redhat.com, thuth@redhat.com, marcandre.lureau@redhat.com, fli@suse.com Date: Wed, 26 Sep 2018 18:12:30 +0100 Message-Id: <20180926171236.45203-11-dgilbert@redhat.com> In-Reply-To: <20180926171236.45203-1-dgilbert@redhat.com> References: <20180926171236.45203-1-dgilbert@redhat.com> X-Scanned-By: MIMEDefang 2.84 on 10.5.11.27 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.45]); Wed, 26 Sep 2018 17:13:18 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PULL 10/16] tests/migration: Add migration-test header file X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: quintela@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RDMRC_1 RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: Wei Huang This patch moves the settings related migration-test from the migration-test.c file to a new header file. Reviewed-by: Juan Quintela Reviewed-by: Andrew Jones Signed-off-by: Wei Huang Message-Id: <1536174934-26022-4-git-send-email-wei@redhat.com> Signed-off-by: Juan Quintela Signed-off-by: Dr. David Alan Gilbert --- tests/migration-test.c | 28 ++++++++++++++++++---------- tests/migration/migration-test.h | 21 +++++++++++++++++++++ 2 files changed, 39 insertions(+), 10 deletions(-) create mode 100644 tests/migration/migration-test.h diff --git a/tests/migration-test.c b/tests/migration-test.c index fe6b41a38d..17c689601f 100644 --- a/tests/migration-test.c +++ b/tests/migration-test.c @@ -21,11 +21,13 @@ #include "chardev/char.h" #include "sysemu/sysemu.h" =20 +#include "migration/migration-test.h" + /* TODO actually test the results and get rid of this */ #define qtest_qmp_discard_response(...) qobject_unref(qtest_qmp(__VA_ARGS_= _)) =20 -const unsigned start_address =3D 1024 * 1024; -const unsigned end_address =3D 100 * 1024 * 1024; +unsigned start_address; +unsigned end_address; bool got_stop; static bool uffd_feature_thread_id; =20 @@ -80,8 +82,8 @@ static bool ufd_version_check(void) =20 static const char *tmpfs; =20 -/* A simple PC boot sector that modifies memory (1-100MB) quickly - * outputting a 'B' every so often if it's still running. +/* The boot file modifies memory area in [start_address, end_address) + * repeatedly. It outputs a 'B' at a fixed rate while it's still running. */ #include "tests/migration/i386/a-b-bootblock.h" =20 @@ -270,11 +272,11 @@ static void wait_for_migration_pass(QTestState *who) static void check_guests_ram(QTestState *who) { /* Our ASM test will have been incrementing one byte from each page fr= om - * 1MB to <100MB in order. - * This gives us a constraint that any page's byte should be equal or = less - * than the previous pages byte (mod 256); and they should all be equal - * except for one transition at the point where we meet the incremente= r. - * (We're running this with the guest stopped). + * start_address to < end_address in order. This gives us a constraint + * that any page's byte should be equal or less than the previous pages + * byte (mod 256); and they should all be equal except for one transit= ion + * at the point where we meet the incrementer. (We're running this with + * the guest stopped). */ unsigned address; uint8_t first_byte; @@ -285,7 +287,8 @@ static void check_guests_ram(QTestState *who) qtest_memread(who, start_address, &first_byte, 1); last_byte =3D first_byte; =20 - for (address =3D start_address + 4096; address < end_address; address = +=3D 4096) + for (address =3D start_address + TEST_MEM_PAGE_SIZE; address < end_add= ress; + address +=3D TEST_MEM_PAGE_SIZE) { uint8_t b; qtest_memread(who, address, &b, 1); @@ -437,6 +440,8 @@ static int test_migrate_start(QTestState **from, QTestS= tate **to, " -drive file=3D%s,format=3Draw" " -incoming %s", accel, tmpfs, bootpath, uri); + start_address =3D X86_TEST_MEM_START; + end_address =3D X86_TEST_MEM_END; } else if (strcmp(arch, "ppc64") =3D=3D 0) { cmd_src =3D g_strdup_printf("-machine accel=3D%s -m 256M" " -name source,debug-threads=3Don" @@ -451,6 +456,9 @@ static int test_migrate_start(QTestState **from, QTestS= tate **to, " -serial file:%s/dest_serial" " -incoming %s", accel, tmpfs, uri); + + start_address =3D PPC_TEST_MEM_START; + end_address =3D PPC_TEST_MEM_END; } else { g_assert_not_reached(); } diff --git a/tests/migration/migration-test.h b/tests/migration/migration-t= est.h new file mode 100644 index 0000000000..c4c0c526b6 --- /dev/null +++ b/tests/migration/migration-test.h @@ -0,0 +1,21 @@ +/* + * Copyright (c) 2018 Red Hat, Inc. and/or its affiliates + * + * This work is licensed under the terms of the GNU GPL, version 2 or late= r. + * See the COPYING file in the top-level directory. + */ +#ifndef _TEST_MIGRATION_H_ +#define _TEST_MIGRATION_H_ + +/* Common */ +#define TEST_MEM_PAGE_SIZE 4096 + +/* x86 */ +#define X86_TEST_MEM_START (1 * 1024 * 1024) +#define X86_TEST_MEM_END (100 * 1024 * 1024) + +/* PPC */ +#define PPC_TEST_MEM_START (1 * 1024 * 1024) +#define PPC_TEST_MEM_END (100 * 1024 * 1024) + +#endif /* _TEST_MIGRATION_H_ */ --=20 2.17.1 From nobody Tue May 7 18:44:44 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 153798287457443.13841665575433; Wed, 26 Sep 2018 10:27:54 -0700 (PDT) Received: from localhost ([::1]:59935 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1g5DbN-00059B-At for importer@patchew.org; Wed, 26 Sep 2018 13:27:53 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:52774) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1g5DNL-0000yP-Iu for qemu-devel@nongnu.org; Wed, 26 Sep 2018 13:13:24 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1g5DNJ-00085E-Bv for qemu-devel@nongnu.org; Wed, 26 Sep 2018 13:13:23 -0400 Received: from mx1.redhat.com ([209.132.183.28]:52128) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1g5DNJ-00084k-5B for qemu-devel@nongnu.org; Wed, 26 Sep 2018 13:13:21 -0400 Received: from smtp.corp.redhat.com (int-mx12.intmail.prod.int.phx2.redhat.com [10.5.11.27]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 6E4E965871; Wed, 26 Sep 2018 17:13:20 +0000 (UTC) Received: from dgilbert-t530.redhat.com (unknown [10.36.118.26]) by smtp.corp.redhat.com (Postfix) with ESMTP id AEC86DB561; Wed, 26 Sep 2018 17:13:18 +0000 (UTC) From: "Dr. David Alan Gilbert (git)" To: qemu-devel@nongnu.org, xiaoguangrong@tencent.com, joserz@linux.ibm.com, wei@redhat.com, thuth@redhat.com, marcandre.lureau@redhat.com, fli@suse.com Date: Wed, 26 Sep 2018 18:12:31 +0100 Message-Id: <20180926171236.45203-12-dgilbert@redhat.com> In-Reply-To: <20180926171236.45203-1-dgilbert@redhat.com> References: <20180926171236.45203-1-dgilbert@redhat.com> X-Scanned-By: MIMEDefang 2.84 on 10.5.11.27 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.38]); Wed, 26 Sep 2018 17:13:20 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PULL 11/16] migration/postcopy: Clear have_listen_thread X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: quintela@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RDMRC_1 RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: "Dr. David Alan Gilbert" Clear have_listen_thread when we exit the thread. The fallout from this was that various things thought there was an ongoing postcopy after the postcopy had finished. The case that failed was postcopy->savevm->loadvm. This corresponds to RH bug https://bugzilla.redhat.com/show_bug.cgi?id=3D16= 08765 Signed-off-by: Dr. David Alan Gilbert Message-Id: <20180914170430.54271-2-dgilbert@redhat.com> Reviewed-by: Peter Xu Signed-off-by: Dr. David Alan Gilbert --- migration/savevm.c | 1 + 1 file changed, 1 insertion(+) diff --git a/migration/savevm.c b/migration/savevm.c index 9692577318..d35e87b88c 100644 --- a/migration/savevm.c +++ b/migration/savevm.c @@ -1679,6 +1679,7 @@ static void *postcopy_ram_listen_thread(void *opaque) qemu_loadvm_state_cleanup(); =20 rcu_unregister_thread(); + mis->have_listen_thread =3D false; return NULL; } =20 --=20 2.17.1 From nobody Tue May 7 18:44:44 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1537982755878892.3396034076014; Wed, 26 Sep 2018 10:25:55 -0700 (PDT) Received: from localhost ([::1]:59923 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1g5DZS-0003Os-Nr for importer@patchew.org; Wed, 26 Sep 2018 13:25:54 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:52826) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1g5DNW-00015o-It for qemu-devel@nongnu.org; Wed, 26 Sep 2018 13:13:35 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1g5DNS-0008H6-2z for qemu-devel@nongnu.org; Wed, 26 Sep 2018 13:13:34 -0400 Received: from mx1.redhat.com ([209.132.183.28]:34294) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1g5DNR-0008GO-TC for qemu-devel@nongnu.org; Wed, 26 Sep 2018 13:13:30 -0400 Received: from smtp.corp.redhat.com (int-mx12.intmail.prod.int.phx2.redhat.com [10.5.11.27]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 25A07811B7; Wed, 26 Sep 2018 17:13:29 +0000 (UTC) Received: from dgilbert-t530.redhat.com (unknown [10.36.118.26]) by smtp.corp.redhat.com (Postfix) with ESMTP id B9986DB56F; Wed, 26 Sep 2018 17:13:20 +0000 (UTC) From: "Dr. David Alan Gilbert (git)" To: qemu-devel@nongnu.org, xiaoguangrong@tencent.com, joserz@linux.ibm.com, wei@redhat.com, thuth@redhat.com, marcandre.lureau@redhat.com, fli@suse.com Date: Wed, 26 Sep 2018 18:12:32 +0100 Message-Id: <20180926171236.45203-13-dgilbert@redhat.com> In-Reply-To: <20180926171236.45203-1-dgilbert@redhat.com> References: <20180926171236.45203-1-dgilbert@redhat.com> X-Scanned-By: MIMEDefang 2.84 on 10.5.11.27 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.27]); Wed, 26 Sep 2018 17:13:29 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PULL 12/16] migration: cleanup in error paths in loadvm X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: quintela@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RDMRC_1 RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: "Dr. David Alan Gilbert" There's a couple of error paths in qemu_loadvm_state which happen early on but after we've initialised the load state; that needs to be cleaned up otherwise we can hit asserts if the state gets reinitialised later. Signed-off-by: Dr. David Alan Gilbert Message-Id: <20180914170430.54271-3-dgilbert@redhat.com> Reviewed-by: Peter Xu Signed-off-by: Dr. David Alan Gilbert --- migration/savevm.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/migration/savevm.c b/migration/savevm.c index d35e87b88c..2d10e45582 100644 --- a/migration/savevm.c +++ b/migration/savevm.c @@ -2333,11 +2333,13 @@ int qemu_loadvm_state(QEMUFile *f) if (migrate_get_current()->send_configuration) { if (qemu_get_byte(f) !=3D QEMU_VM_CONFIGURATION) { error_report("Configuration section missing"); + qemu_loadvm_state_cleanup(); return -EINVAL; } ret =3D vmstate_load_state(f, &vmstate_configuration, &savevm_stat= e, 0); =20 if (ret) { + qemu_loadvm_state_cleanup(); return ret; } } --=20 2.17.1 From nobody Tue May 7 18:44:44 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1537982696056535.3665630736742; Wed, 26 Sep 2018 10:24:56 -0700 (PDT) Received: from localhost ([::1]:59918 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1g5DYQ-0002YJ-IY for importer@patchew.org; Wed, 26 Sep 2018 13:24:50 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:52824) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1g5DNW-00015k-Hm for qemu-devel@nongnu.org; Wed, 26 Sep 2018 13:13:35 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1g5DNT-0008Ig-Jw for qemu-devel@nongnu.org; Wed, 26 Sep 2018 13:13:34 -0400 Received: from mx1.redhat.com ([209.132.183.28]:43128) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1g5DNT-0008Hx-DX for qemu-devel@nongnu.org; Wed, 26 Sep 2018 13:13:31 -0400 Received: from smtp.corp.redhat.com (int-mx12.intmail.prod.int.phx2.redhat.com [10.5.11.27]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id ABF9F3082A2C; Wed, 26 Sep 2018 17:13:30 +0000 (UTC) Received: from dgilbert-t530.redhat.com (unknown [10.36.118.26]) by smtp.corp.redhat.com (Postfix) with ESMTP id D4A22DB561; Wed, 26 Sep 2018 17:13:28 +0000 (UTC) From: "Dr. David Alan Gilbert (git)" To: qemu-devel@nongnu.org, xiaoguangrong@tencent.com, joserz@linux.ibm.com, wei@redhat.com, thuth@redhat.com, marcandre.lureau@redhat.com, fli@suse.com Date: Wed, 26 Sep 2018 18:12:33 +0100 Message-Id: <20180926171236.45203-14-dgilbert@redhat.com> In-Reply-To: <20180926171236.45203-1-dgilbert@redhat.com> References: <20180926171236.45203-1-dgilbert@redhat.com> X-Scanned-By: MIMEDefang 2.84 on 10.5.11.27 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.45]); Wed, 26 Sep 2018 17:13:30 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PULL 13/16] tests/migration: Speed up the test on ppc64 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: quintela@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RDMRC_1 RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: Thomas Huth The SLOF boot process is always quite slow ... but we can speed it up a little bit by specifying "-nodefaults" and by using the "nvramrc" variable instead of "boot-command" (since "nvramrc" is evaluated earlier in the SLOF boot process than "boot-command"). Signed-off-by: Thomas Huth Message-Id: <1537204330-16076-1-git-send-email-thuth@redhat.com> Reviewed-by: Dr. David Alan Gilbert Reviewed-by: Laurent Vivier Signed-off-by: Dr. David Alan Gilbert --- tests/migration-test.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/tests/migration-test.c b/tests/migration-test.c index 17c689601f..20f38f1930 100644 --- a/tests/migration-test.c +++ b/tests/migration-test.c @@ -443,11 +443,11 @@ static int test_migrate_start(QTestState **from, QTes= tState **to, start_address =3D X86_TEST_MEM_START; end_address =3D X86_TEST_MEM_END; } else if (strcmp(arch, "ppc64") =3D=3D 0) { - cmd_src =3D g_strdup_printf("-machine accel=3D%s -m 256M" + cmd_src =3D g_strdup_printf("-machine accel=3D%s -m 256M -nodefaul= ts" " -name source,debug-threads=3Don" " -serial file:%s/src_serial" - " -prom-env '" - "boot-command=3Dhex .\" _\" begin %x %x " + " -prom-env 'use-nvramrc?=3Dtrue' -prom-= env " + "'nvramrc=3Dhex .\" _\" begin %x %x " "do i c@ 1 + i c! 1000 +loop .\" B\" 0 " "until'", accel, tmpfs, end_address, start_address); --=20 2.17.1 From nobody Tue May 7 18:44:44 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1537982335110302.5607869728974; Wed, 26 Sep 2018 10:18:55 -0700 (PDT) Received: from localhost ([::1]:59888 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1g5DSd-0004a8-PL for importer@patchew.org; Wed, 26 Sep 2018 13:18:51 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:52846) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1g5DNX-00017q-Lr for qemu-devel@nongnu.org; Wed, 26 Sep 2018 13:13:36 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1g5DNW-0008M5-Pb for qemu-devel@nongnu.org; Wed, 26 Sep 2018 13:13:35 -0400 Received: from mx1.redhat.com ([209.132.183.28]:49784) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1g5DNW-0008KC-I4 for qemu-devel@nongnu.org; Wed, 26 Sep 2018 13:13:34 -0400 Received: from smtp.corp.redhat.com (int-mx12.intmail.prod.int.phx2.redhat.com [10.5.11.27]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id AA6537F6A7; Wed, 26 Sep 2018 17:13:32 +0000 (UTC) Received: from dgilbert-t530.redhat.com (unknown [10.36.118.26]) by smtp.corp.redhat.com (Postfix) with ESMTP id E8DBEDB561; Wed, 26 Sep 2018 17:13:30 +0000 (UTC) From: "Dr. David Alan Gilbert (git)" To: qemu-devel@nongnu.org, xiaoguangrong@tencent.com, joserz@linux.ibm.com, wei@redhat.com, thuth@redhat.com, marcandre.lureau@redhat.com, fli@suse.com Date: Wed, 26 Sep 2018 18:12:34 +0100 Message-Id: <20180926171236.45203-15-dgilbert@redhat.com> In-Reply-To: <20180926171236.45203-1-dgilbert@redhat.com> References: <20180926171236.45203-1-dgilbert@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.27 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.25]); Wed, 26 Sep 2018 17:13:32 +0000 (UTC) Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PULL 14/16] migration: fix QEMUFile leak X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: quintela@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RDMRC_1 RSF_0 Z_629925259 SPT_0 Content-Type: text/plain; charset="utf-8" From: Marc-Andr=C3=A9 Lureau Spotted by ASAN while running: $ tests/migration-test -p /x86_64/migration/postcopy/recovery =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D =3D=3D18034=3D=3DERROR: LeakSanitizer: detected memory leaks Direct leak of 33864 byte(s) in 1 object(s) allocated from: #0 0x7f3da7f31e50 in calloc (/lib64/libasan.so.5+0xeee50) #1 0x7f3da644441d in g_malloc0 (/lib64/libglib-2.0.so.0+0x5241d) #2 0x55af9db15440 in qemu_fopen_channel_input /home/elmarco/src/qemu/mi= gration/qemu-file-channel.c:183 #3 0x55af9db15413 in channel_get_output_return_path /home/elmarco/src/q= emu/migration/qemu-file-channel.c:159 #4 0x55af9db0d4ac in qemu_file_get_return_path /home/elmarco/src/qemu/m= igration/qemu-file.c:78 #5 0x55af9dad5e4f in open_return_path_on_source /home/elmarco/src/qemu/= migration/migration.c:2295 #6 0x55af9dadb3bf in migrate_fd_connect /home/elmarco/src/qemu/migratio= n/migration.c:3111 #7 0x55af9dae1bf3 in migration_channel_connect /home/elmarco/src/qemu/m= igration/channel.c:91 #8 0x55af9daddeca in socket_outgoing_migration /home/elmarco/src/qemu/m= igration/socket.c:108 #9 0x55af9e13d3db in qio_task_complete /home/elmarco/src/qemu/io/task.c= :158 #10 0x55af9e13ca03 in qio_task_thread_result /home/elmarco/src/qemu/io/= task.c:89 #11 0x7f3da643b1ca in g_idle_dispatch gmain.c:5535 Signed-off-by: Marc-Andr=C3=A9 Lureau Message-Id: <20180925092245.29565-1-marcandre.lureau@redhat.com> Reviewed-by: Peter Xu Reviewed-by: Dr. David Alan Gilbert Signed-off-by: Dr. David Alan Gilbert --- migration/migration.c | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/migration/migration.c b/migration/migration.c index 6a7731de50..d6ae879dc8 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -2280,7 +2280,10 @@ out: */ if (postcopy_pause_return_path_thread(ms)) { /* Reload rp, reset the rest */ - rp =3D ms->rp_state.from_dst_file; + if (rp !=3D ms->rp_state.from_dst_file) { + qemu_fclose(rp); + rp =3D ms->rp_state.from_dst_file; + } ms->rp_state.error =3D false; goto retry; } --=20 2.17.1 From nobody Tue May 7 18:44:44 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1537982864591398.5984025633079; Wed, 26 Sep 2018 10:27:44 -0700 (PDT) Received: from localhost ([::1]:59934 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1g5DbD-0004tf-KE for importer@patchew.org; Wed, 26 Sep 2018 13:27:43 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:52863) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1g5DNY-00018o-Ns for qemu-devel@nongnu.org; Wed, 26 Sep 2018 13:13:37 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1g5DNX-0008NR-T7 for qemu-devel@nongnu.org; Wed, 26 Sep 2018 13:13:36 -0400 Received: from mx1.redhat.com ([209.132.183.28]:47590) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1g5DNX-0008MR-MQ for qemu-devel@nongnu.org; Wed, 26 Sep 2018 13:13:35 -0400 Received: from smtp.corp.redhat.com (int-mx12.intmail.prod.int.phx2.redhat.com [10.5.11.27]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id B78A830832E6; Wed, 26 Sep 2018 17:13:34 +0000 (UTC) Received: from dgilbert-t530.redhat.com (unknown [10.36.118.26]) by smtp.corp.redhat.com (Postfix) with ESMTP id 03264DB564; Wed, 26 Sep 2018 17:13:32 +0000 (UTC) From: "Dr. David Alan Gilbert (git)" To: qemu-devel@nongnu.org, xiaoguangrong@tencent.com, joserz@linux.ibm.com, wei@redhat.com, thuth@redhat.com, marcandre.lureau@redhat.com, fli@suse.com Date: Wed, 26 Sep 2018 18:12:35 +0100 Message-Id: <20180926171236.45203-16-dgilbert@redhat.com> In-Reply-To: <20180926171236.45203-1-dgilbert@redhat.com> References: <20180926171236.45203-1-dgilbert@redhat.com> X-Scanned-By: MIMEDefang 2.84 on 10.5.11.27 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.44]); Wed, 26 Sep 2018 17:13:34 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PULL 15/16] migration: fix the compression code X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: quintela@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RDMRC_1 RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: Fei Li Add judgement in compress_threads_save_cleanup() to check whether the static CompressParam *comp_param has been allocated. If not, just return; or else segmentation fault will occur when using the NULL comp_param's parameters. One test case can reproduce this is: set the compression on and migrate to a wrong nonexistent host IP address. Our current code does not judge before handling comp_param[idx]'s quit and cond that whether they have been initialized. If not initialized, "qemu_mutex_lock_impl: Assertion `mutex->initialized' failed." will occur. Fix this by squashing the terminate_compression_threads() into compress_threads_save_cleanup() and employing the existing judgement condition. One test case can reproduce this error is: set the compression on and fail to fully setup the default eight compression thread in compress_threads_save_setup(). Signed-off-by: Fei Li Message-Id: <20180925091440.18910-1-fli@suse.com> Reviewed-by: Peter Xu Signed-off-by: Dr. David Alan Gilbert --- migration/ram.c | 24 ++++++++---------------- 1 file changed, 8 insertions(+), 16 deletions(-) diff --git a/migration/ram.c b/migration/ram.c index 0fdaa8efa3..408d237700 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -431,28 +431,14 @@ static void *do_data_compress(void *opaque) return NULL; } =20 -static inline void terminate_compression_threads(void) -{ - int idx, thread_count; - - thread_count =3D migrate_compress_threads(); - - for (idx =3D 0; idx < thread_count; idx++) { - qemu_mutex_lock(&comp_param[idx].mutex); - comp_param[idx].quit =3D true; - qemu_cond_signal(&comp_param[idx].cond); - qemu_mutex_unlock(&comp_param[idx].mutex); - } -} - static void compress_threads_save_cleanup(void) { int i, thread_count; =20 - if (!migrate_use_compression()) { + if (!migrate_use_compression() || !comp_param) { return; } - terminate_compression_threads(); + thread_count =3D migrate_compress_threads(); for (i =3D 0; i < thread_count; i++) { /* @@ -462,6 +448,12 @@ static void compress_threads_save_cleanup(void) if (!comp_param[i].file) { break; } + + qemu_mutex_lock(&comp_param[i].mutex); + comp_param[i].quit =3D true; + qemu_cond_signal(&comp_param[i].cond); + qemu_mutex_unlock(&comp_param[i].mutex); + qemu_thread_join(compress_threads + i); qemu_mutex_destroy(&comp_param[i].mutex); qemu_cond_destroy(&comp_param[i].cond); --=20 2.17.1 From nobody Tue May 7 18:44:44 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1537982958851751.0662333099949; Wed, 26 Sep 2018 10:29:18 -0700 (PDT) Received: from localhost ([::1]:59947 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1g5Dcj-00079l-Sf for importer@patchew.org; Wed, 26 Sep 2018 13:29:17 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:52875) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1g5DNa-0001Ap-Jz for qemu-devel@nongnu.org; Wed, 26 Sep 2018 13:13:39 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1g5DNZ-0008PV-R1 for qemu-devel@nongnu.org; Wed, 26 Sep 2018 13:13:38 -0400 Received: from mx1.redhat.com ([209.132.183.28]:41486) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1g5DNZ-0008Oq-HC for qemu-devel@nongnu.org; Wed, 26 Sep 2018 13:13:37 -0400 Received: from smtp.corp.redhat.com (int-mx12.intmail.prod.int.phx2.redhat.com [10.5.11.27]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id C65F93097031; Wed, 26 Sep 2018 17:13:36 +0000 (UTC) Received: from dgilbert-t530.redhat.com (unknown [10.36.118.26]) by smtp.corp.redhat.com (Postfix) with ESMTP id 0C5CCDB561; Wed, 26 Sep 2018 17:13:34 +0000 (UTC) From: "Dr. David Alan Gilbert (git)" To: qemu-devel@nongnu.org, xiaoguangrong@tencent.com, joserz@linux.ibm.com, wei@redhat.com, thuth@redhat.com, marcandre.lureau@redhat.com, fli@suse.com Date: Wed, 26 Sep 2018 18:12:36 +0100 Message-Id: <20180926171236.45203-17-dgilbert@redhat.com> In-Reply-To: <20180926171236.45203-1-dgilbert@redhat.com> References: <20180926171236.45203-1-dgilbert@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.27 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.43]); Wed, 26 Sep 2018 17:13:36 +0000 (UTC) Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PULL 16/16] migration/ram.c: Avoid taking address of fields in packed MultiFDInit_t struct X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: quintela@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RDMRC_1 RSF_0 Z_629925259 SPT_0 Content-Type: text/plain; charset="utf-8" From: Peter Maydell Taking the address of a field in a packed struct is a bad idea, because it might not be actually aligned enough for that pointer type (and thus cause a crash on dereference on some host architectures). Newer versions of clang warn about this: migration/ram.c:651:19: warning: taking address of packed member 'magic' of= class or structure 'MultiFDInit_t' may result in an unaligned pointer valu= e [-Waddress-of-packed-member] migration/ram.c:652:19: warning: taking address of packed member 'version' = of class or structure 'MultiFDInit_t' may result in an unaligned pointer va= lue [-Waddress-of-packed-member] migration/ram.c:737:19: warning: taking address of packed member 'magic' of= class or structure 'MultiFDPacket_t' may result in an unaligned pointer va= lue [-Waddress-of-packed-member] migration/ram.c:745:19: warning: taking address of packed member 'version' = of class or structure 'MultiFDPacket_t' may result in an unaligned pointer = value [-Waddress-of-packed-member] migration/ram.c:755:19: warning: taking address of packed member 'size' of = class or structure 'MultiFDPacket_t' may result in an unaligned pointer val= ue [-Waddress-of-packed-member] Avoid the bug by not using the "modify in place" byteswapping functions. Signed-off-by: Peter Maydell Message-Id: <20180925161924.7832-1-peter.maydell@linaro.org> Reviewed-by: Marc-Andr=C3=A9 Lureau Signed-off-by: Dr. David Alan Gilbert --- migration/ram.c | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/migration/ram.c b/migration/ram.c index 408d237700..bc38d98cc3 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -651,8 +651,8 @@ static int multifd_recv_initial_packet(QIOChannel *c, E= rror **errp) return -1; } =20 - be32_to_cpus(&msg.magic); - be32_to_cpus(&msg.version); + msg.magic =3D be32_to_cpu(msg.magic); + msg.version =3D be32_to_cpu(msg.version); =20 if (msg.magic !=3D MULTIFD_MAGIC) { error_setg(errp, "multifd: received packet magic %x " @@ -737,7 +737,7 @@ static int multifd_recv_unfill_packet(MultiFDRecvParams= *p, Error **errp) RAMBlock *block; int i; =20 - be32_to_cpus(&packet->magic); + packet->magic =3D be32_to_cpu(packet->magic); if (packet->magic !=3D MULTIFD_MAGIC) { error_setg(errp, "multifd: received packet " "magic %x and expected magic %x", @@ -745,7 +745,7 @@ static int multifd_recv_unfill_packet(MultiFDRecvParams= *p, Error **errp) return -1; } =20 - be32_to_cpus(&packet->version); + packet->version =3D be32_to_cpu(packet->version); if (packet->version !=3D MULTIFD_VERSION) { error_setg(errp, "multifd: received packet " "version %d and expected version %d", @@ -755,7 +755,7 @@ static int multifd_recv_unfill_packet(MultiFDRecvParams= *p, Error **errp) =20 p->flags =3D be32_to_cpu(packet->flags); =20 - be32_to_cpus(&packet->size); + packet->size =3D be32_to_cpu(packet->size); if (packet->size > migrate_multifd_page_count()) { error_setg(errp, "multifd: received packet " "with size %d and expected maximum size %d", --=20 2.17.1