From nobody Sun Apr 28 01:03:39 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1486480091367925.2447393525198; Tue, 7 Feb 2017 07:08:11 -0800 (PST) Received: from localhost ([::1]:54653 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cb7NJ-0001xP-0I for importer@patchew.org; Tue, 07 Feb 2017 10:08:09 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:57797) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cb7L9-0000Vx-Ed for qemu-devel@nongnu.org; Tue, 07 Feb 2017 10:05:57 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1cb7L3-00077v-Dk for qemu-devel@nongnu.org; Tue, 07 Feb 2017 10:05:55 -0500 Received: from mailhub.sw.ru ([195.214.232.25]:12785 helo=relay.sw.ru) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1cb7L3-00072f-1P; Tue, 07 Feb 2017 10:05:49 -0500 Received: from kvm.sw.ru (msk-vpn.virtuozzo.com [195.214.232.6]) by relay.sw.ru (8.13.4/8.13.4) with ESMTP id v17F5ekw013037; Tue, 7 Feb 2017 18:05:42 +0300 (MSK) From: Vladimir Sementsov-Ogievskiy To: qemu-block@nongnu.org, qemu-devel@nongnu.org Date: Tue, 7 Feb 2017 18:05:24 +0300 Message-Id: <1486479940-163258-2-git-send-email-vsementsov@virtuozzo.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1486479940-163258-1-git-send-email-vsementsov@virtuozzo.com> References: <1486479940-163258-1-git-send-email-vsementsov@virtuozzo.com> X-detected-operating-system: by eggs.gnu.org: OpenBSD 3.x [fuzzy] X-Received-From: 195.214.232.25 Subject: [Qemu-devel] [PATCH 01/17] migration: add has_postcopy savevm handler X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, peter.maydell@linaro.org, vsementsov@virtuozzo.com, famz@redhat.com, lirans@il.ibm.com, quintela@redhat.com, jsnow@redhat.com, armbru@redhat.com, mreitz@redhat.com, stefanha@redhat.com, den@openvz.org, amit.shah@redhat.com, pbonzini@redhat.com, dgilbert@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Now postcopy-able states are recognized by not NULL save_live_complete_postcopy handler. But when we have several different postcopy-able states, it is not convenient. Ram postcopy may be disabled, while some other postcopy enabled, in this case Ram state should behave as it is not postcopy-able. This patch add separate has_postcopy handler to specify behaviour of savevm state. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Juan Quintela --- include/migration/vmstate.h | 1 + migration/ram.c | 6 ++++++ migration/savevm.c | 6 ++++-- 3 files changed, 11 insertions(+), 2 deletions(-) diff --git a/include/migration/vmstate.h b/include/migration/vmstate.h index 3bbe3ed..1eadcf0 100644 --- a/include/migration/vmstate.h +++ b/include/migration/vmstate.h @@ -46,6 +46,7 @@ typedef struct SaveVMHandlers { =20 /* This runs both outside and inside the iothread lock. */ bool (*is_active)(void *opaque); + bool (*has_postcopy)(void *opaque); =20 /* This runs outside the iothread lock in the migration case, and * within the lock in the savevm case. The callback had better only diff --git a/migration/ram.c b/migration/ram.c index ef8fadf..c688866 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -2589,10 +2589,16 @@ static int ram_load(QEMUFile *f, void *opaque, int = version_id) return ret; } =20 +static bool ram_has_postcopy(void *opaque) +{ + return migrate_postcopy_ram(); +} + static SaveVMHandlers savevm_ram_handlers =3D { .save_live_setup =3D ram_save_setup, .save_live_iterate =3D ram_save_iterate, .save_live_complete_postcopy =3D ram_save_complete, + .has_postcopy =3D ram_has_postcopy, .save_live_complete_precopy =3D ram_save_complete, .save_live_pending =3D ram_save_pending, .load_state =3D ram_load, diff --git a/migration/savevm.c b/migration/savevm.c index 204012e..5235833 100644 --- a/migration/savevm.c +++ b/migration/savevm.c @@ -1018,7 +1018,8 @@ int qemu_savevm_state_iterate(QEMUFile *f, bool postc= opy) * call that's already run, it might get confused if we call * iterate afterwards. */ - if (postcopy && !se->ops->save_live_complete_postcopy) { + if (postcopy && + !(se->ops->has_postcopy && se->ops->has_postcopy(se->opaque)))= { continue; } if (qemu_file_rate_limit(f)) { @@ -1106,7 +1107,8 @@ void qemu_savevm_state_complete_precopy(QEMUFile *f, = bool iterable_only) =20 QTAILQ_FOREACH(se, &savevm_state.handlers, entry) { if (!se->ops || - (in_postcopy && se->ops->save_live_complete_postcopy) || + (in_postcopy && se->ops->has_postcopy && + se->ops->has_postcopy(se->opaque)) || (in_postcopy && !iterable_only) || !se->ops->save_live_complete_precopy) { continue; --=20 1.8.3.1 From nobody Sun Apr 28 01:03:39 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1486481206453400.66230303862153; Tue, 7 Feb 2017 07:26:46 -0800 (PST) Received: from localhost ([::1]:54811 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cb7fJ-0001dj-3A for importer@patchew.org; Tue, 07 Feb 2017 10:26:45 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:57887) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cb7LC-0000YR-Jk for qemu-devel@nongnu.org; Tue, 07 Feb 2017 10:06:01 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1cb7L5-0007A3-1f for qemu-devel@nongnu.org; Tue, 07 Feb 2017 10:05:58 -0500 Received: from mailhub.sw.ru ([195.214.232.25]:42139 helo=relay.sw.ru) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1cb7L4-00077J-Bp; Tue, 07 Feb 2017 10:05:50 -0500 Received: from kvm.sw.ru (msk-vpn.virtuozzo.com [195.214.232.6]) by relay.sw.ru (8.13.4/8.13.4) with ESMTP id v17F5ekx013037; Tue, 7 Feb 2017 18:05:42 +0300 (MSK) From: Vladimir Sementsov-Ogievskiy To: qemu-block@nongnu.org, qemu-devel@nongnu.org Date: Tue, 7 Feb 2017 18:05:25 +0300 Message-Id: <1486479940-163258-3-git-send-email-vsementsov@virtuozzo.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1486479940-163258-1-git-send-email-vsementsov@virtuozzo.com> References: <1486479940-163258-1-git-send-email-vsementsov@virtuozzo.com> X-detected-operating-system: by eggs.gnu.org: OpenBSD 3.x [fuzzy] X-Received-From: 195.214.232.25 Subject: [Qemu-devel] [PATCH 02/17] migration: fix ram_save_pending X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, peter.maydell@linaro.org, vsementsov@virtuozzo.com, famz@redhat.com, lirans@il.ibm.com, quintela@redhat.com, jsnow@redhat.com, armbru@redhat.com, mreitz@redhat.com, stefanha@redhat.com, den@openvz.org, amit.shah@redhat.com, pbonzini@redhat.com, dgilbert@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Fill postcopy-able pending only if ram postcopy is enabled. It is necessary because of there will be other postcopy-able states and when ram postcopy is disabled, it should not spoil common postcopy related pending. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Dr. David Alan Gilbert Reviewed-by: Juan Quintela --- migration/ram.c | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-) diff --git a/migration/ram.c b/migration/ram.c index c688866..5d0bbfd 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -2091,8 +2091,12 @@ static void ram_save_pending(QEMUFile *f, void *opaq= ue, uint64_t max_size, remaining_size =3D ram_save_remaining() * TARGET_PAGE_SIZE; } =20 - /* We can do postcopy, and all the data is postcopiable */ - *postcopiable_pending +=3D remaining_size; + if (migrate_postcopy_ram()) { + /* We can do postcopy, and all the data is postcopiable */ + *postcopiable_pending +=3D remaining_size; + } else { + *non_postcopiable_pending +=3D remaining_size; + } } =20 static int load_xbzrle(QEMUFile *f, ram_addr_t addr, void *host) --=20 1.8.3.1 From nobody Sun Apr 28 01:03:39 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1486480948369334.1604355228832; Tue, 7 Feb 2017 07:22:28 -0800 (PST) Received: from localhost ([::1]:54769 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cb7b8-0005rE-5x for importer@patchew.org; Tue, 07 Feb 2017 10:22:26 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:57812) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cb7L9-0000WE-Nu for qemu-devel@nongnu.org; Tue, 07 Feb 2017 10:06:00 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1cb7L5-00079L-08 for qemu-devel@nongnu.org; Tue, 07 Feb 2017 10:05:55 -0500 Received: from mailhub.sw.ru ([195.214.232.25]:5819 helo=relay.sw.ru) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1cb7L3-00072j-2L; Tue, 07 Feb 2017 10:05:49 -0500 Received: from kvm.sw.ru (msk-vpn.virtuozzo.com [195.214.232.6]) by relay.sw.ru (8.13.4/8.13.4) with ESMTP id v17F5el0013037; Tue, 7 Feb 2017 18:05:42 +0300 (MSK) From: Vladimir Sementsov-Ogievskiy To: qemu-block@nongnu.org, qemu-devel@nongnu.org Date: Tue, 7 Feb 2017 18:05:26 +0300 Message-Id: <1486479940-163258-4-git-send-email-vsementsov@virtuozzo.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1486479940-163258-1-git-send-email-vsementsov@virtuozzo.com> References: <1486479940-163258-1-git-send-email-vsementsov@virtuozzo.com> X-detected-operating-system: by eggs.gnu.org: OpenBSD 3.x [fuzzy] X-Received-From: 195.214.232.25 Subject: [Qemu-devel] [PATCH 03/17] migration: split common postcopy out of ram postcopy X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, peter.maydell@linaro.org, vsementsov@virtuozzo.com, famz@redhat.com, lirans@il.ibm.com, quintela@redhat.com, jsnow@redhat.com, armbru@redhat.com, mreitz@redhat.com, stefanha@redhat.com, den@openvz.org, amit.shah@redhat.com, pbonzini@redhat.com, dgilbert@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Split common postcopy staff from ram postcopy staff. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Juan Quintela --- include/migration/migration.h | 1 + migration/migration.c | 39 ++++++++++++++++++++++++----------- migration/postcopy-ram.c | 4 +++- migration/savevm.c | 48 +++++++++++++++++++++++++++++++++++----= ---- 4 files changed, 70 insertions(+), 22 deletions(-) diff --git a/include/migration/migration.h b/include/migration/migration.h index af9135f..3aa228c 100644 --- a/include/migration/migration.h +++ b/include/migration/migration.h @@ -305,6 +305,7 @@ int migrate_add_blocker(Error *reason, Error **errp); */ void migrate_del_blocker(Error *reason); =20 +bool migrate_postcopy(void); bool migrate_postcopy_ram(void); bool migrate_zero_blocks(void); =20 diff --git a/migration/migration.c b/migration/migration.c index 2766d2f..0a5fd38 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -1310,6 +1310,11 @@ bool migrate_postcopy_ram(void) return s->enabled_capabilities[MIGRATION_CAPABILITY_POSTCOPY_RAM]; } =20 +bool migrate_postcopy(void) +{ + return migrate_postcopy_ram(); +} + bool migrate_auto_converge(void) { MigrationState *s; @@ -1637,9 +1642,11 @@ static int postcopy_start(MigrationState *ms, bool *= old_vm_running) * need to tell the destination to throw any pages it's already receiv= ed * that are dirty */ - if (ram_postcopy_send_discard_bitmap(ms)) { - error_report("postcopy send discard bitmap failed"); - goto fail; + if (migrate_postcopy_ram()) { + if (ram_postcopy_send_discard_bitmap(ms)) { + error_report("postcopy send discard bitmap failed"); + goto fail; + } } =20 /* @@ -1648,8 +1655,10 @@ static int postcopy_start(MigrationState *ms, bool *= old_vm_running) * wrap their state up here */ qemu_file_set_rate_limit(ms->to_dst_file, INT64_MAX); - /* Ping just for debugging, helps line traces up */ - qemu_savevm_send_ping(ms->to_dst_file, 2); + if (migrate_postcopy_ram()) { + /* Ping just for debugging, helps line traces up */ + qemu_savevm_send_ping(ms->to_dst_file, 2); + } =20 /* * While loading the device state we may trigger page transfer @@ -1674,7 +1683,9 @@ static int postcopy_start(MigrationState *ms, bool *o= ld_vm_running) qemu_savevm_send_postcopy_listen(fb); =20 qemu_savevm_state_complete_precopy(fb, false); - qemu_savevm_send_ping(fb, 3); + if (migrate_postcopy_ram()) { + qemu_savevm_send_ping(fb, 3); + } =20 qemu_savevm_send_postcopy_run(fb); =20 @@ -1697,11 +1708,13 @@ static int postcopy_start(MigrationState *ms, bool = *old_vm_running) =20 qemu_mutex_unlock_iothread(); =20 - /* - * Although this ping is just for debug, it could potentially be - * used for getting a better measurement of downtime at the source. - */ - qemu_savevm_send_ping(ms->to_dst_file, 4); + if (migrate_postcopy_ram()) { + /* + * Although this ping is just for debug, it could potentially be + * used for getting a better measurement of downtime at the source. + */ + qemu_savevm_send_ping(ms->to_dst_file, 4); + } =20 ret =3D qemu_file_get_error(ms->to_dst_file); if (ret) { @@ -1857,7 +1870,9 @@ static void *migration_thread(void *opaque) =20 /* And do a ping that will make stuff easier to debug */ qemu_savevm_send_ping(s->to_dst_file, 1); + } =20 + if (migrate_postcopy()) { /* * Tell the destination that we *might* want to do postcopy later; * if the other end can't do postcopy it should fail now, nice and @@ -1891,7 +1906,7 @@ static void *migration_thread(void *opaque) if (pending_size && pending_size >=3D max_size) { /* Still a significant amount to transfer */ =20 - if (migrate_postcopy_ram() && + if (migrate_postcopy() && s->state !=3D MIGRATION_STATUS_POSTCOPY_ACTIVE && pend_nonpost <=3D max_size && atomic_read(&s->start_postcopy)) { diff --git a/migration/postcopy-ram.c b/migration/postcopy-ram.c index a40dddb..aef5690 100644 --- a/migration/postcopy-ram.c +++ b/migration/postcopy-ram.c @@ -339,7 +339,9 @@ int postcopy_ram_incoming_cleanup(MigrationIncomingStat= e *mis) } =20 postcopy_state_set(POSTCOPY_INCOMING_END); - migrate_send_rp_shut(mis, qemu_file_get_error(mis->from_src_file) !=3D= 0); + if (migrate_postcopy_ram()) { + migrate_send_rp_shut(mis, qemu_file_get_error(mis->from_src_file) = !=3D 0); + } =20 if (mis->postcopy_tmp_page) { munmap(mis->postcopy_tmp_page, getpagesize()); diff --git a/migration/savevm.c b/migration/savevm.c index 5235833..965a58c 100644 --- a/migration/savevm.c +++ b/migration/savevm.c @@ -73,7 +73,7 @@ static struct mig_cmd_args { [MIG_CMD_INVALID] =3D { .len =3D -1, .name =3D "INVALID" }, [MIG_CMD_OPEN_RETURN_PATH] =3D { .len =3D 0, .name =3D "OPEN_RETURN_P= ATH" }, [MIG_CMD_PING] =3D { .len =3D sizeof(uint32_t), .name =3D = "PING" }, - [MIG_CMD_POSTCOPY_ADVISE] =3D { .len =3D 16, .name =3D "POSTCOPY_ADVI= SE" }, + [MIG_CMD_POSTCOPY_ADVISE] =3D { .len =3D -1, .name =3D "POSTCOPY_ADVI= SE" }, [MIG_CMD_POSTCOPY_LISTEN] =3D { .len =3D 0, .name =3D "POSTCOPY_LIST= EN" }, [MIG_CMD_POSTCOPY_RUN] =3D { .len =3D 0, .name =3D "POSTCOPY_RUN"= }, [MIG_CMD_POSTCOPY_RAM_DISCARD] =3D { @@ -82,6 +82,23 @@ static struct mig_cmd_args { [MIG_CMD_MAX] =3D { .len =3D -1, .name =3D "MAX" }, }; =20 +/* Note for MIG_CMD_POSTCOPY_ADVISE: + * The format of arguments is depending on postcopy mode: + * - postcopy RAM only + * uint64_t host page size + * uint64_t taget page size + * + * - postcopy RAM and postcopy dirty bitmaps + * format is the same as for postcopy RAM only + * + * - postcopy dirty bitmaps only + * Nothing. Command length field is 0. + * + * Be careful: adding a new postcopy entity with some other parameters sho= uld + * not break format self-description ability. Good way is to introduce some + * generic extendable format with an exception for two old entities. + */ + static int announce_self_create(uint8_t *buf, uint8_t *mac_addr) { @@ -856,12 +873,17 @@ int qemu_savevm_send_packaged(QEMUFile *f, const uint= 8_t *buf, size_t len) /* Send prior to any postcopy transfer */ void qemu_savevm_send_postcopy_advise(QEMUFile *f) { - uint64_t tmp[2]; - tmp[0] =3D cpu_to_be64(getpagesize()); - tmp[1] =3D cpu_to_be64(1ul << qemu_target_page_bits()); + if (migrate_postcopy_ram()) { + uint64_t tmp[2]; + tmp[0] =3D cpu_to_be64(getpagesize()); + tmp[1] =3D cpu_to_be64(1ul << qemu_target_page_bits()); =20 - trace_qemu_savevm_send_postcopy_advise(); - qemu_savevm_command_send(f, MIG_CMD_POSTCOPY_ADVISE, 16, (uint8_t *)tm= p); + trace_qemu_savevm_send_postcopy_advise(); + qemu_savevm_command_send(f, MIG_CMD_POSTCOPY_ADVISE, + 16, (uint8_t *)tmp); + } else { + qemu_savevm_command_send(f, MIG_CMD_POSTCOPY_ADVISE, 0, NULL); + } } =20 /* Sent prior to starting the destination running in postcopy, discard pag= es @@ -1344,6 +1366,10 @@ static int loadvm_postcopy_handle_advise(MigrationIn= comingState *mis) return -1; } =20 + if (!migrate_postcopy_ram()) { + return 0; + } + if (!postcopy_ram_supported_by_host()) { return -1; } @@ -1544,7 +1570,9 @@ static int loadvm_postcopy_handle_listen(MigrationInc= omingState *mis) * A rare case, we entered listen without having to do any discard= s, * so do the setup that's normally done at the time of the 1st dis= card. */ - postcopy_ram_prepare_discard(mis); + if (migrate_postcopy_ram()) { + postcopy_ram_prepare_discard(mis); + } } =20 /* @@ -1552,8 +1580,10 @@ static int loadvm_postcopy_handle_listen(MigrationIn= comingState *mis) * However, at this point the CPU shouldn't be running, and the IO * shouldn't be doing anything yet so don't actually expect requests */ - if (postcopy_ram_enable_notify(mis)) { - return -1; + if (migrate_postcopy_ram()) { + if (postcopy_ram_enable_notify(mis)) { + return -1; + } } =20 if (mis->have_listen_thread) { --=20 1.8.3.1 From nobody Sun Apr 28 01:03:39 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 148648203262670.65902345007078; Tue, 7 Feb 2017 07:40:32 -0800 (PST) Received: from localhost ([::1]:54903 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cb7sd-00042x-4Z for importer@patchew.org; Tue, 07 Feb 2017 10:40:31 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:58008) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cb7LG-0000c2-2S for qemu-devel@nongnu.org; Tue, 07 Feb 2017 10:06:06 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1cb7L5-0007AD-1x for qemu-devel@nongnu.org; Tue, 07 Feb 2017 10:06:02 -0500 Received: from mailhub.sw.ru ([195.214.232.25]:23098 helo=relay.sw.ru) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1cb7L4-00077G-BM; Tue, 07 Feb 2017 10:05:50 -0500 Received: from kvm.sw.ru (msk-vpn.virtuozzo.com [195.214.232.6]) by relay.sw.ru (8.13.4/8.13.4) with ESMTP id v17F5el1013037; Tue, 7 Feb 2017 18:05:42 +0300 (MSK) From: Vladimir Sementsov-Ogievskiy To: qemu-block@nongnu.org, qemu-devel@nongnu.org Date: Tue, 7 Feb 2017 18:05:27 +0300 Message-Id: <1486479940-163258-5-git-send-email-vsementsov@virtuozzo.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1486479940-163258-1-git-send-email-vsementsov@virtuozzo.com> References: <1486479940-163258-1-git-send-email-vsementsov@virtuozzo.com> X-detected-operating-system: by eggs.gnu.org: OpenBSD 3.x [fuzzy] X-Received-From: 195.214.232.25 Subject: [Qemu-devel] [PATCH 04/17] migration: introduce postcopy-only pending X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, peter.maydell@linaro.org, vsementsov@virtuozzo.com, famz@redhat.com, lirans@il.ibm.com, quintela@redhat.com, jsnow@redhat.com, armbru@redhat.com, mreitz@redhat.com, stefanha@redhat.com, den@openvz.org, amit.shah@redhat.com, pbonzini@redhat.com, dgilbert@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" There would be savevm states (dirty-bitmap) which can migrate only in postcopy stage. The corresponding pending is introduced here. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Juan Quintela --- include/migration/vmstate.h | 17 +++++++++++++++-- include/sysemu/sysemu.h | 5 +++-- migration/block.c | 7 ++++--- migration/migration.c | 15 ++++++++------- migration/ram.c | 9 +++++---- migration/savevm.c | 14 ++++++++------ migration/trace-events | 2 +- 7 files changed, 44 insertions(+), 25 deletions(-) diff --git a/include/migration/vmstate.h b/include/migration/vmstate.h index 1eadcf0..da62a73 100644 --- a/include/migration/vmstate.h +++ b/include/migration/vmstate.h @@ -58,8 +58,21 @@ typedef struct SaveVMHandlers { /* This runs outside the iothread lock! */ int (*save_live_setup)(QEMUFile *f, void *opaque); void (*save_live_pending)(QEMUFile *f, void *opaque, uint64_t max_size, - uint64_t *non_postcopiable_pending, - uint64_t *postcopiable_pending); + uint64_t *res_precopy_only, + uint64_t *res_compatible, + uint64_t *res_postcopy_only); + /* Note for save_live_pending: + * - res_precopy_only is for data which must be migrated in precopy ph= ase + * or in stopped state, in other words - before target vm start + * - res_compatible is for data which may be migrated in any phase + * - res_postcopy_only is for data which must be migrated in postcopy = phase + * or in stopped state, in other words - after source vm stop + * + * Sum of res_postcopy_only, res_compatible and res_postcopy_only is t= he + * whole amount of pending data. + */ + + LoadStateHandler *load_state; } SaveVMHandlers; =20 diff --git a/include/sysemu/sysemu.h b/include/sysemu/sysemu.h index 4d50694..5d59d35 100644 --- a/include/sysemu/sysemu.h +++ b/include/sysemu/sysemu.h @@ -111,8 +111,9 @@ void qemu_savevm_state_cleanup(void); void qemu_savevm_state_complete_postcopy(QEMUFile *f); void qemu_savevm_state_complete_precopy(QEMUFile *f, bool iterable_only); void qemu_savevm_state_pending(QEMUFile *f, uint64_t max_size, - uint64_t *res_non_postcopiable, - uint64_t *res_postcopiable); + uint64_t *res_precopy_only, + uint64_t *res_compatible, + uint64_t *res_postcopy_only); void qemu_savevm_command_send(QEMUFile *f, enum qemu_vm_cmd command, uint16_t len, uint8_t *data); void qemu_savevm_send_ping(QEMUFile *f, uint32_t value); diff --git a/migration/block.c b/migration/block.c index ebc10e6..83b0785 100644 --- a/migration/block.c +++ b/migration/block.c @@ -830,8 +830,9 @@ static int block_save_complete(QEMUFile *f, void *opaqu= e) } =20 static void block_save_pending(QEMUFile *f, void *opaque, uint64_t max_siz= e, - uint64_t *non_postcopiable_pending, - uint64_t *postcopiable_pending) + uint64_t *res_precopy_only, + uint64_t *res_compatible, + uint64_t *res_postcopy_only) { /* Estimate pending number of bytes to send */ uint64_t pending; @@ -852,7 +853,7 @@ static void block_save_pending(QEMUFile *f, void *opaqu= e, uint64_t max_size, =20 DPRINTF("Enter save live pending %" PRIu64 "\n", pending); /* We don't do postcopy */ - *non_postcopiable_pending +=3D pending; + *res_precopy_only +=3D pending; } =20 static int block_load(QEMUFile *f, void *opaque, int version_id) diff --git a/migration/migration.c b/migration/migration.c index 0a5fd38..2fb734e 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -1896,20 +1896,21 @@ static void *migration_thread(void *opaque) uint64_t pending_size; =20 if (!qemu_file_rate_limit(s->to_dst_file)) { - uint64_t pend_post, pend_nonpost; + uint64_t pend_pre, pend_compat, pend_post; =20 - qemu_savevm_state_pending(s->to_dst_file, max_size, &pend_nonp= ost, - &pend_post); - pending_size =3D pend_nonpost + pend_post; + qemu_savevm_state_pending(s->to_dst_file, max_size, &pend_pre, + &pend_compat, &pend_post); + pending_size =3D pend_pre + pend_compat + pend_post; trace_migrate_pending(pending_size, max_size, - pend_post, pend_nonpost); + pend_pre, pend_compat, pend_post); if (pending_size && pending_size >=3D max_size) { /* Still a significant amount to transfer */ =20 if (migrate_postcopy() && s->state !=3D MIGRATION_STATUS_POSTCOPY_ACTIVE && - pend_nonpost <=3D max_size && - atomic_read(&s->start_postcopy)) { + pend_pre <=3D max_size && + (atomic_read(&s->start_postcopy) || + (pend_pre + pend_compat <=3D max_size))) { =20 if (!postcopy_start(s, &old_vm_running)) { current_active_state =3D MIGRATION_STATUS_POSTCOPY= _ACTIVE; diff --git a/migration/ram.c b/migration/ram.c index 5d0bbfd..3b60010 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -2074,8 +2074,9 @@ static int ram_save_complete(QEMUFile *f, void *opaqu= e) } =20 static void ram_save_pending(QEMUFile *f, void *opaque, uint64_t max_size, - uint64_t *non_postcopiable_pending, - uint64_t *postcopiable_pending) + uint64_t *res_precopy_only, + uint64_t *res_compatible, + uint64_t *res_postcopy_only) { uint64_t remaining_size; =20 @@ -2093,9 +2094,9 @@ static void ram_save_pending(QEMUFile *f, void *opaqu= e, uint64_t max_size, =20 if (migrate_postcopy_ram()) { /* We can do postcopy, and all the data is postcopiable */ - *postcopiable_pending +=3D remaining_size; + *res_compatible +=3D remaining_size; } else { - *non_postcopiable_pending +=3D remaining_size; + *res_precopy_only +=3D remaining_size; } } =20 diff --git a/migration/savevm.c b/migration/savevm.c index 965a58c..d1bab59 100644 --- a/migration/savevm.c +++ b/migration/savevm.c @@ -1209,13 +1209,15 @@ void qemu_savevm_state_complete_precopy(QEMUFile *f= , bool iterable_only) * for units that can't do postcopy. */ void qemu_savevm_state_pending(QEMUFile *f, uint64_t max_size, - uint64_t *res_non_postcopiable, - uint64_t *res_postcopiable) + uint64_t *res_precopy_only, + uint64_t *res_compatible, + uint64_t *res_postcopy_only) { SaveStateEntry *se; =20 - *res_non_postcopiable =3D 0; - *res_postcopiable =3D 0; + *res_precopy_only =3D 0; + *res_compatible =3D 0; + *res_postcopy_only =3D 0; =20 =20 QTAILQ_FOREACH(se, &savevm_state.handlers, entry) { @@ -1227,8 +1229,8 @@ void qemu_savevm_state_pending(QEMUFile *f, uint64_t = max_size, continue; } } - se->ops->save_live_pending(f, se->opaque, max_size, - res_non_postcopiable, res_postcopiable); + se->ops->save_live_pending(f, se->opaque, max_size, res_precopy_on= ly, + res_compatible, res_postcopy_only); } } =20 diff --git a/migration/trace-events b/migration/trace-events index fa660e3..0212929 100644 --- a/migration/trace-events +++ b/migration/trace-events @@ -80,7 +80,7 @@ migrate_fd_cleanup(void) "" migrate_fd_error(const char *error_desc) "error=3D%s" migrate_fd_cancel(void) "" migrate_handle_rp_req_pages(const char *rbname, size_t start, size_t len) = "in %s at %zx len %zx" -migrate_pending(uint64_t size, uint64_t max, uint64_t post, uint64_t nonpo= st) "pending size %" PRIu64 " max %" PRIu64 " (post=3D%" PRIu64 " nonpost= =3D%" PRIu64 ")" +migrate_pending(uint64_t size, uint64_t max, uint64_t pre, uint64_t compat= , uint64_t post) "pending size %" PRIu64 " max %" PRIu64 " (pre =3D %" PRIu= 64 " compat=3D%" PRIu64 " post=3D%" PRIu64 ")" migrate_send_rp_message(int msg_type, uint16_t len) "%d: len %d" migration_completion_file_err(void) "" migration_completion_postcopy_end(void) "" --=20 1.8.3.1 From nobody Sun Apr 28 01:03:39 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 14864803742966.085340321808076; Tue, 7 Feb 2017 07:12:54 -0800 (PST) Received: from localhost ([::1]:54685 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cb7Rq-0006Gx-Vu for importer@patchew.org; Tue, 07 Feb 2017 10:12:51 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:57800) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cb7L9-0000W5-LC for qemu-devel@nongnu.org; Tue, 07 Feb 2017 10:05:58 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1cb7L4-00078h-Dx for qemu-devel@nongnu.org; Tue, 07 Feb 2017 10:05:55 -0500 Received: from mailhub.sw.ru ([195.214.232.25]:26931 helo=relay.sw.ru) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1cb7L3-00072V-6G; Tue, 07 Feb 2017 10:05:49 -0500 Received: from kvm.sw.ru (msk-vpn.virtuozzo.com [195.214.232.6]) by relay.sw.ru (8.13.4/8.13.4) with ESMTP id v17F5el2013037; Tue, 7 Feb 2017 18:05:42 +0300 (MSK) From: Vladimir Sementsov-Ogievskiy To: qemu-block@nongnu.org, qemu-devel@nongnu.org Date: Tue, 7 Feb 2017 18:05:28 +0300 Message-Id: <1486479940-163258-6-git-send-email-vsementsov@virtuozzo.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1486479940-163258-1-git-send-email-vsementsov@virtuozzo.com> References: <1486479940-163258-1-git-send-email-vsementsov@virtuozzo.com> X-detected-operating-system: by eggs.gnu.org: OpenBSD 3.x [fuzzy] X-Received-From: 195.214.232.25 Subject: [Qemu-devel] [PATCH 05/17] block: add bdrv_next_dirty_bitmap() X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, peter.maydell@linaro.org, vsementsov@virtuozzo.com, famz@redhat.com, lirans@il.ibm.com, quintela@redhat.com, jsnow@redhat.com, armbru@redhat.com, mreitz@redhat.com, stefanha@redhat.com, den@openvz.org, amit.shah@redhat.com, pbonzini@redhat.com, dgilbert@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Like bdrv_next() - bdrv_next_dirty_bitmap() is a function to provide access to private dirty bitmaps list. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: John Snow --- block/dirty-bitmap.c | 10 ++++++++++ include/block/dirty-bitmap.h | 3 +++ 2 files changed, 13 insertions(+) diff --git a/block/dirty-bitmap.c b/block/dirty-bitmap.c index 519737c..7547c78 100644 --- a/block/dirty-bitmap.c +++ b/block/dirty-bitmap.c @@ -533,3 +533,13 @@ int64_t bdrv_get_meta_dirty_count(BdrvDirtyBitmap *bit= map) { return hbitmap_count(bitmap->meta); } + +BdrvDirtyBitmap *bdrv_next_dirty_bitmap(BlockDriverState *bs, + BdrvDirtyBitmap *bitmap) +{ + if (bitmap =3D=3D NULL) { + return QLIST_FIRST(&bs->dirty_bitmaps); + } + + return QLIST_NEXT(bitmap, list); +} diff --git a/include/block/dirty-bitmap.h b/include/block/dirty-bitmap.h index 9dea14b..1a457d5 100644 --- a/include/block/dirty-bitmap.h +++ b/include/block/dirty-bitmap.h @@ -72,4 +72,7 @@ void bdrv_dirty_bitmap_deserialize_zeroes(BdrvDirtyBitmap= *bitmap, bool finish); void bdrv_dirty_bitmap_deserialize_finish(BdrvDirtyBitmap *bitmap); =20 +BdrvDirtyBitmap *bdrv_next_dirty_bitmap(BlockDriverState *bs, + BdrvDirtyBitmap *bitmap); + #endif --=20 1.8.3.1 From nobody Sun Apr 28 01:03:39 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1486480622803626.281821575369; Tue, 7 Feb 2017 07:17:02 -0800 (PST) Received: from localhost ([::1]:54718 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cb7Vt-0001Rt-Ix for importer@patchew.org; Tue, 07 Feb 2017 10:17:01 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:57802) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cb7L9-0000W8-M8 for qemu-devel@nongnu.org; Tue, 07 Feb 2017 10:05:59 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1cb7L3-00078V-VX for qemu-devel@nongnu.org; Tue, 07 Feb 2017 10:05:55 -0500 Received: from mailhub.sw.ru ([195.214.232.25]:34240 helo=relay.sw.ru) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1cb7L3-00072X-3z; Tue, 07 Feb 2017 10:05:49 -0500 Received: from kvm.sw.ru (msk-vpn.virtuozzo.com [195.214.232.6]) by relay.sw.ru (8.13.4/8.13.4) with ESMTP id v17F5el3013037; Tue, 7 Feb 2017 18:05:42 +0300 (MSK) From: Vladimir Sementsov-Ogievskiy To: qemu-block@nongnu.org, qemu-devel@nongnu.org Date: Tue, 7 Feb 2017 18:05:29 +0300 Message-Id: <1486479940-163258-7-git-send-email-vsementsov@virtuozzo.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1486479940-163258-1-git-send-email-vsementsov@virtuozzo.com> References: <1486479940-163258-1-git-send-email-vsementsov@virtuozzo.com> X-detected-operating-system: by eggs.gnu.org: OpenBSD 3.x [fuzzy] X-Received-From: 195.214.232.25 Subject: [Qemu-devel] [PATCH 06/17] block: add bdrv_dirty_bitmap_enable_successor() X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, peter.maydell@linaro.org, vsementsov@virtuozzo.com, famz@redhat.com, lirans@il.ibm.com, quintela@redhat.com, jsnow@redhat.com, armbru@redhat.com, mreitz@redhat.com, stefanha@redhat.com, den@openvz.org, amit.shah@redhat.com, pbonzini@redhat.com, dgilbert@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Enabling bitmap successor is necessary to enable successors of bitmaps being migrated before target vm start. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Max Reitz --- block/dirty-bitmap.c | 5 +++++ include/block/dirty-bitmap.h | 1 + 2 files changed, 6 insertions(+) diff --git a/block/dirty-bitmap.c b/block/dirty-bitmap.c index 7547c78..ab2c7f5 100644 --- a/block/dirty-bitmap.c +++ b/block/dirty-bitmap.c @@ -217,6 +217,11 @@ int bdrv_dirty_bitmap_create_successor(BlockDriverStat= e *bs, return 0; } =20 +void bdrv_dirty_bitmap_enable_successor(BdrvDirtyBitmap *bitmap) +{ + bdrv_enable_dirty_bitmap(bitmap->successor); +} + /** * For a bitmap with a successor, yield our name to the successor, * delete the old bitmap, and return a handle to the new bitmap. diff --git a/include/block/dirty-bitmap.h b/include/block/dirty-bitmap.h index 1a457d5..57b476c 100644 --- a/include/block/dirty-bitmap.h +++ b/include/block/dirty-bitmap.h @@ -20,6 +20,7 @@ BdrvDirtyBitmap *bdrv_dirty_bitmap_abdicate(BlockDriverSt= ate *bs, BdrvDirtyBitmap *bdrv_reclaim_dirty_bitmap(BlockDriverState *bs, BdrvDirtyBitmap *bitmap, Error **errp); +void bdrv_dirty_bitmap_enable_successor(BdrvDirtyBitmap *bitmap); BdrvDirtyBitmap *bdrv_find_dirty_bitmap(BlockDriverState *bs, const char *name); void bdrv_dirty_bitmap_make_anon(BdrvDirtyBitmap *bitmap); --=20 1.8.3.1 From nobody Sun Apr 28 01:03:39 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1486480599737332.3258825843428; Tue, 7 Feb 2017 07:16:39 -0800 (PST) Received: from localhost ([::1]:54715 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cb7VV-0001A0-RZ for importer@patchew.org; Tue, 07 Feb 2017 10:16:37 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:57801) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cb7L9-0000W6-M0 for qemu-devel@nongnu.org; Tue, 07 Feb 2017 10:05:58 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1cb7L5-00079n-1A for qemu-devel@nongnu.org; Tue, 07 Feb 2017 10:05:55 -0500 Received: from mailhub.sw.ru ([195.214.232.25]:4835 helo=relay.sw.ru) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1cb7L3-00072S-Bv; Tue, 07 Feb 2017 10:05:49 -0500 Received: from kvm.sw.ru (msk-vpn.virtuozzo.com [195.214.232.6]) by relay.sw.ru (8.13.4/8.13.4) with ESMTP id v17F5el4013037; Tue, 7 Feb 2017 18:05:42 +0300 (MSK) From: Vladimir Sementsov-Ogievskiy To: qemu-block@nongnu.org, qemu-devel@nongnu.org Date: Tue, 7 Feb 2017 18:05:30 +0300 Message-Id: <1486479940-163258-8-git-send-email-vsementsov@virtuozzo.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1486479940-163258-1-git-send-email-vsementsov@virtuozzo.com> References: <1486479940-163258-1-git-send-email-vsementsov@virtuozzo.com> X-detected-operating-system: by eggs.gnu.org: OpenBSD 3.x [fuzzy] X-Received-From: 195.214.232.25 Subject: [Qemu-devel] [PATCH 07/17] qapi: add dirty-bitmaps migration capability X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, peter.maydell@linaro.org, vsementsov@virtuozzo.com, famz@redhat.com, lirans@il.ibm.com, quintela@redhat.com, jsnow@redhat.com, armbru@redhat.com, mreitz@redhat.com, stefanha@redhat.com, den@openvz.org, amit.shah@redhat.com, pbonzini@redhat.com, dgilbert@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: John Snow Reviewed-by: Eric Blake Reviewed-by: Juan Quintela --- include/migration/migration.h | 1 + migration/migration.c | 9 +++++++++ qapi-schema.json | 4 +++- 3 files changed, 13 insertions(+), 1 deletion(-) diff --git a/include/migration/migration.h b/include/migration/migration.h index 3aa228c..38eba2e 100644 --- a/include/migration/migration.h +++ b/include/migration/migration.h @@ -308,6 +308,7 @@ void migrate_del_blocker(Error *reason); bool migrate_postcopy(void); bool migrate_postcopy_ram(void); bool migrate_zero_blocks(void); +bool migrate_dirty_bitmaps(void); =20 bool migrate_auto_converge(void); =20 diff --git a/migration/migration.c b/migration/migration.c index 2fb734e..64f88ac 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -1369,6 +1369,15 @@ int migrate_decompress_threads(void) return s->parameters.decompress_threads; } =20 +bool migrate_dirty_bitmaps(void) +{ + MigrationState *s; + + s =3D migrate_get_current(); + + return s->enabled_capabilities[MIGRATION_CAPABILITY_DIRTY_BITMAPS]; +} + bool migrate_use_events(void) { MigrationState *s; diff --git a/qapi-schema.json b/qapi-schema.json index cbdffdd..5ae1d79 100644 --- a/qapi-schema.json +++ b/qapi-schema.json @@ -865,11 +865,13 @@ # side, this process is called COarse-Grain LOck Stepping (COLO) for # Non-stop Service. (since 2.8) # +# @dirty-bitmaps: If enabled, QEMU will migrate named dirty bitmaps. (sinc= e 2.9) +# # Since: 1.2 ## { 'enum': 'MigrationCapability', 'data': ['xbzrle', 'rdma-pin-all', 'auto-converge', 'zero-blocks', - 'compress', 'events', 'postcopy-ram', 'x-colo'] } + 'compress', 'events', 'postcopy-ram', 'x-colo', 'dirty-bitmaps'= ] } =20 ## # @MigrationCapabilityStatus: --=20 1.8.3.1 From nobody Sun Apr 28 01:03:39 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1486481646278806.9466536443745; Tue, 7 Feb 2017 07:34:06 -0800 (PST) Received: from localhost ([::1]:54864 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cb7mN-0007sZ-Tc for importer@patchew.org; Tue, 07 Feb 2017 10:34:03 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:57929) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cb7LE-0000Zu-2i for qemu-devel@nongnu.org; Tue, 07 Feb 2017 10:06:04 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1cb7L4-00079B-Uy for qemu-devel@nongnu.org; Tue, 07 Feb 2017 10:05:59 -0500 Received: from mailhub.sw.ru ([195.214.232.25]:45014 helo=relay.sw.ru) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1cb7L3-00072Z-88; Tue, 07 Feb 2017 10:05:49 -0500 Received: from kvm.sw.ru (msk-vpn.virtuozzo.com [195.214.232.6]) by relay.sw.ru (8.13.4/8.13.4) with ESMTP id v17F5el5013037; Tue, 7 Feb 2017 18:05:42 +0300 (MSK) From: Vladimir Sementsov-Ogievskiy To: qemu-block@nongnu.org, qemu-devel@nongnu.org Date: Tue, 7 Feb 2017 18:05:31 +0300 Message-Id: <1486479940-163258-9-git-send-email-vsementsov@virtuozzo.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1486479940-163258-1-git-send-email-vsementsov@virtuozzo.com> References: <1486479940-163258-1-git-send-email-vsementsov@virtuozzo.com> X-detected-operating-system: by eggs.gnu.org: OpenBSD 3.x [fuzzy] X-Received-From: 195.214.232.25 Subject: [Qemu-devel] [PATCH 08/17] block/dirty-bitmap: add bdrv_dirty_bitmap_release_successor X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, peter.maydell@linaro.org, vsementsov@virtuozzo.com, famz@redhat.com, lirans@il.ibm.com, quintela@redhat.com, jsnow@redhat.com, armbru@redhat.com, mreitz@redhat.com, stefanha@redhat.com, den@openvz.org, amit.shah@redhat.com, pbonzini@redhat.com, dgilbert@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" To just release successor and unfreeze bitmap without any additional work. Signed-off-by: Vladimir Sementsov-Ogievskiy Signed-off-by: Denis V. Lunev Reviewed-by: Max Reitz --- block/dirty-bitmap.c | 10 ++++++++++ include/block/dirty-bitmap.h | 2 ++ 2 files changed, 12 insertions(+) diff --git a/block/dirty-bitmap.c b/block/dirty-bitmap.c index ab2c7f5..32aa6eb 100644 --- a/block/dirty-bitmap.c +++ b/block/dirty-bitmap.c @@ -274,6 +274,16 @@ BdrvDirtyBitmap *bdrv_reclaim_dirty_bitmap(BlockDriver= State *bs, return parent; } =20 +void bdrv_dirty_bitmap_release_successor(BlockDriverState *bs, + BdrvDirtyBitmap *parent) +{ + BdrvDirtyBitmap *successor =3D parent->successor; + + if (successor) { + bdrv_release_dirty_bitmap(bs, successor); + parent->successor =3D NULL; + } +} /** * Truncates _all_ bitmaps attached to a BDS. */ diff --git a/include/block/dirty-bitmap.h b/include/block/dirty-bitmap.h index 57b476c..20b3ec7 100644 --- a/include/block/dirty-bitmap.h +++ b/include/block/dirty-bitmap.h @@ -20,6 +20,8 @@ BdrvDirtyBitmap *bdrv_dirty_bitmap_abdicate(BlockDriverSt= ate *bs, BdrvDirtyBitmap *bdrv_reclaim_dirty_bitmap(BlockDriverState *bs, BdrvDirtyBitmap *bitmap, Error **errp); +void bdrv_dirty_bitmap_release_successor(BlockDriverState *bs, + BdrvDirtyBitmap *bitmap); void bdrv_dirty_bitmap_enable_successor(BdrvDirtyBitmap *bitmap); BdrvDirtyBitmap *bdrv_find_dirty_bitmap(BlockDriverState *bs, const char *name); --=20 1.8.3.1 From nobody Sun Apr 28 01:03:39 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1486481179018938.7404018195956; Tue, 7 Feb 2017 07:26:19 -0800 (PST) Received: from localhost ([::1]:54803 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cb7eq-00018P-FI for importer@patchew.org; Tue, 07 Feb 2017 10:26:16 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:57805) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cb7L9-0000W9-MB for qemu-devel@nongnu.org; Tue, 07 Feb 2017 10:06:01 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1cb7L4-00079U-Vb for qemu-devel@nongnu.org; Tue, 07 Feb 2017 10:05:55 -0500 Received: from mailhub.sw.ru ([195.214.232.25]:9442 helo=relay.sw.ru) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1cb7L3-00072Q-87; Tue, 07 Feb 2017 10:05:49 -0500 Received: from kvm.sw.ru (msk-vpn.virtuozzo.com [195.214.232.6]) by relay.sw.ru (8.13.4/8.13.4) with ESMTP id v17F5el6013037; Tue, 7 Feb 2017 18:05:42 +0300 (MSK) From: Vladimir Sementsov-Ogievskiy To: qemu-block@nongnu.org, qemu-devel@nongnu.org Date: Tue, 7 Feb 2017 18:05:32 +0300 Message-Id: <1486479940-163258-10-git-send-email-vsementsov@virtuozzo.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1486479940-163258-1-git-send-email-vsementsov@virtuozzo.com> References: <1486479940-163258-1-git-send-email-vsementsov@virtuozzo.com> X-detected-operating-system: by eggs.gnu.org: OpenBSD 3.x [fuzzy] X-Received-From: 195.214.232.25 Subject: [Qemu-devel] [PATCH 09/17] migration: include migrate_dirty_bitmaps in migrate_postcopy X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, peter.maydell@linaro.org, vsementsov@virtuozzo.com, famz@redhat.com, lirans@il.ibm.com, quintela@redhat.com, jsnow@redhat.com, armbru@redhat.com, mreitz@redhat.com, stefanha@redhat.com, den@openvz.org, amit.shah@redhat.com, pbonzini@redhat.com, dgilbert@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Enable postcopy if dirty bitmap migration is endabled. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Juan Quintela --- migration/migration.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/migration/migration.c b/migration/migration.c index 64f88ac..4716efd 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -1312,7 +1312,7 @@ bool migrate_postcopy_ram(void) =20 bool migrate_postcopy(void) { - return migrate_postcopy_ram(); + return migrate_postcopy_ram() || migrate_dirty_bitmaps(); } =20 bool migrate_auto_converge(void) --=20 1.8.3.1 From nobody Sun Apr 28 01:03:39 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 148648035524457.34446417901279; Tue, 7 Feb 2017 07:12:35 -0800 (PST) Received: from localhost ([::1]:54680 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cb7RY-000617-H9 for importer@patchew.org; Tue, 07 Feb 2017 10:12:32 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:57794) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cb7L9-0000Vu-Co for qemu-devel@nongnu.org; Tue, 07 Feb 2017 10:05:57 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1cb7L3-00077p-Ck for qemu-devel@nongnu.org; Tue, 07 Feb 2017 10:05:55 -0500 Received: from mailhub.sw.ru ([195.214.232.25]:1681 helo=relay.sw.ru) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1cb7L2-00072i-VG; Tue, 07 Feb 2017 10:05:49 -0500 Received: from kvm.sw.ru (msk-vpn.virtuozzo.com [195.214.232.6]) by relay.sw.ru (8.13.4/8.13.4) with ESMTP id v17F5el7013037; Tue, 7 Feb 2017 18:05:42 +0300 (MSK) From: Vladimir Sementsov-Ogievskiy To: qemu-block@nongnu.org, qemu-devel@nongnu.org Date: Tue, 7 Feb 2017 18:05:33 +0300 Message-Id: <1486479940-163258-11-git-send-email-vsementsov@virtuozzo.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1486479940-163258-1-git-send-email-vsementsov@virtuozzo.com> References: <1486479940-163258-1-git-send-email-vsementsov@virtuozzo.com> X-detected-operating-system: by eggs.gnu.org: OpenBSD 3.x [fuzzy] X-Received-From: 195.214.232.25 Subject: [Qemu-devel] [PATCH 10/17] migration/qemu-file: add qemu_put_counted_string() X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, peter.maydell@linaro.org, vsementsov@virtuozzo.com, famz@redhat.com, lirans@il.ibm.com, quintela@redhat.com, jsnow@redhat.com, armbru@redhat.com, mreitz@redhat.com, stefanha@redhat.com, den@openvz.org, amit.shah@redhat.com, pbonzini@redhat.com, dgilbert@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Add function opposite to qemu_get_counted_string. qemu_put_counted_string puts one-byte length of the string (string should not be longer than 255 characters), and then it puts the string, without last zero byte. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: John Snow Reviewed-by: Juan Quintela --- include/migration/qemu-file.h | 2 ++ migration/qemu-file.c | 13 +++++++++++++ 2 files changed, 15 insertions(+) diff --git a/include/migration/qemu-file.h b/include/migration/qemu-file.h index abedd46..d860c92 100644 --- a/include/migration/qemu-file.h +++ b/include/migration/qemu-file.h @@ -309,4 +309,6 @@ static inline void qemu_get_sbe64s(QEMUFile *f, int64_t= *pv) =20 size_t qemu_get_counted_string(QEMUFile *f, char buf[256]); =20 +void qemu_put_counted_string(QEMUFile *f, const char *name); + #endif diff --git a/migration/qemu-file.c b/migration/qemu-file.c index e9fae31..8e928b2 100644 --- a/migration/qemu-file.c +++ b/migration/qemu-file.c @@ -691,6 +691,19 @@ size_t qemu_get_counted_string(QEMUFile *f, char buf[2= 56]) } =20 /* + * Put a string with one preceding byte containing its length. The length = of + * the string should be less than 256. + */ +void qemu_put_counted_string(QEMUFile *f, const char *str) +{ + size_t len =3D strlen(str); + + assert(len < 256); + qemu_put_byte(f, len); + qemu_put_buffer(f, (const uint8_t *)str, len); +} + +/* * Set the blocking state of the QEMUFile. * Note: On some transports the OS only keeps a single blocking state for * both directions, and thus changing the blocking on the main --=20 1.8.3.1 From nobody Sun Apr 28 01:03:39 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1486481829955530.2070675217988; Tue, 7 Feb 2017 07:37:09 -0800 (PST) Received: from localhost ([::1]:54890 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cb7pM-0001PR-Lq for importer@patchew.org; Tue, 07 Feb 2017 10:37:08 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:57884) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cb7LC-0000YJ-GY for qemu-devel@nongnu.org; Tue, 07 Feb 2017 10:06:05 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1cb7L5-00079a-0N for qemu-devel@nongnu.org; Tue, 07 Feb 2017 10:05:58 -0500 Received: from mailhub.sw.ru ([195.214.232.25]:4559 helo=relay.sw.ru) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1cb7L3-00072U-Ch; Tue, 07 Feb 2017 10:05:49 -0500 Received: from kvm.sw.ru (msk-vpn.virtuozzo.com [195.214.232.6]) by relay.sw.ru (8.13.4/8.13.4) with ESMTP id v17F5el8013037; Tue, 7 Feb 2017 18:05:42 +0300 (MSK) From: Vladimir Sementsov-Ogievskiy To: qemu-block@nongnu.org, qemu-devel@nongnu.org Date: Tue, 7 Feb 2017 18:05:34 +0300 Message-Id: <1486479940-163258-12-git-send-email-vsementsov@virtuozzo.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1486479940-163258-1-git-send-email-vsementsov@virtuozzo.com> References: <1486479940-163258-1-git-send-email-vsementsov@virtuozzo.com> X-detected-operating-system: by eggs.gnu.org: OpenBSD 3.x [fuzzy] X-Received-From: 195.214.232.25 Subject: [Qemu-devel] [PATCH 11/17] migration: add is_active_iterate handler X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, peter.maydell@linaro.org, vsementsov@virtuozzo.com, famz@redhat.com, lirans@il.ibm.com, quintela@redhat.com, jsnow@redhat.com, armbru@redhat.com, mreitz@redhat.com, stefanha@redhat.com, den@openvz.org, amit.shah@redhat.com, pbonzini@redhat.com, dgilbert@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Only-postcopy savevm states (dirty-bitmap) don't need live iteration, so to disable them and stop transporting empty sections there is a new savevm handler. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Juan Quintela --- include/migration/vmstate.h | 1 + migration/savevm.c | 5 +++++ 2 files changed, 6 insertions(+) diff --git a/include/migration/vmstate.h b/include/migration/vmstate.h index da62a73..6049deb 100644 --- a/include/migration/vmstate.h +++ b/include/migration/vmstate.h @@ -47,6 +47,7 @@ typedef struct SaveVMHandlers { /* This runs both outside and inside the iothread lock. */ bool (*is_active)(void *opaque); bool (*has_postcopy)(void *opaque); + bool (*is_active_iterate)(void *opaque); =20 /* This runs outside the iothread lock in the migration case, and * within the lock in the savevm case. The callback had better only diff --git a/migration/savevm.c b/migration/savevm.c index d1bab59..94783e5 100644 --- a/migration/savevm.c +++ b/migration/savevm.c @@ -1034,6 +1034,11 @@ int qemu_savevm_state_iterate(QEMUFile *f, bool post= copy) continue; } } + if (se->ops && se->ops->is_active_iterate) { + if (!se->ops->is_active_iterate(se->opaque)) { + continue; + } + } /* * In the postcopy phase, any device that doesn't know how to * do postcopy should have saved it's state in the _complete --=20 1.8.3.1 From nobody Sun Apr 28 01:03:39 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1486481394980456.87291854950513; Tue, 7 Feb 2017 07:29:54 -0800 (PST) Received: from localhost ([::1]:54835 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cb7iL-0004Uo-58 for importer@patchew.org; Tue, 07 Feb 2017 10:29:53 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:57869) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cb7LB-0000Xo-Qt for qemu-devel@nongnu.org; Tue, 07 Feb 2017 10:06:03 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1cb7L5-00079s-1V for qemu-devel@nongnu.org; Tue, 07 Feb 2017 10:05:57 -0500 Received: from mailhub.sw.ru ([195.214.232.25]:49082 helo=relay.sw.ru) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1cb7L3-00072c-B2; Tue, 07 Feb 2017 10:05:50 -0500 Received: from kvm.sw.ru (msk-vpn.virtuozzo.com [195.214.232.6]) by relay.sw.ru (8.13.4/8.13.4) with ESMTP id v17F5el9013037; Tue, 7 Feb 2017 18:05:42 +0300 (MSK) From: Vladimir Sementsov-Ogievskiy To: qemu-block@nongnu.org, qemu-devel@nongnu.org Date: Tue, 7 Feb 2017 18:05:35 +0300 Message-Id: <1486479940-163258-13-git-send-email-vsementsov@virtuozzo.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1486479940-163258-1-git-send-email-vsementsov@virtuozzo.com> References: <1486479940-163258-1-git-send-email-vsementsov@virtuozzo.com> X-detected-operating-system: by eggs.gnu.org: OpenBSD 3.x [fuzzy] X-Received-From: 195.214.232.25 Subject: [Qemu-devel] [PATCH 12/17] migration: add postcopy migration of dirty bitmaps X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, peter.maydell@linaro.org, vsementsov@virtuozzo.com, famz@redhat.com, lirans@il.ibm.com, quintela@redhat.com, jsnow@redhat.com, armbru@redhat.com, mreitz@redhat.com, stefanha@redhat.com, den@openvz.org, amit.shah@redhat.com, pbonzini@redhat.com, dgilbert@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Postcopy migration of dirty bitmaps. Only named dirty bitmaps, associated with root nodes and non-root named nodes are migrated. If destination qemu is already containing a dirty bitmap with the same name as a migrated bitmap (for the same node), than, if their granularities are the same the migration will be done, otherwise the error will be generated. If destination qemu doesn't contain such bitmap it will be created. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Juan Quintela Reviewed-by: Max Reitz --- include/migration/block.h | 1 + include/migration/migration.h | 4 + migration/Makefile.objs | 2 +- migration/block-dirty-bitmap.c | 679 +++++++++++++++++++++++++++++++++++++= ++++ migration/migration.c | 3 + migration/savevm.c | 2 + migration/trace-events | 14 + vl.c | 1 + 8 files changed, 705 insertions(+), 1 deletion(-) create mode 100644 migration/block-dirty-bitmap.c diff --git a/include/migration/block.h b/include/migration/block.h index 41a1ac8..8333c43 100644 --- a/include/migration/block.h +++ b/include/migration/block.h @@ -14,6 +14,7 @@ #ifndef MIGRATION_BLOCK_H #define MIGRATION_BLOCK_H =20 +void dirty_bitmap_mig_init(void); void blk_mig_init(void); int blk_mig_active(void); uint64_t blk_mig_bytes_transferred(void); diff --git a/include/migration/migration.h b/include/migration/migration.h index 38eba2e..9915e2d 100644 --- a/include/migration/migration.h +++ b/include/migration/migration.h @@ -372,4 +372,8 @@ int ram_save_queue_pages(MigrationState *ms, const char= *rbname, PostcopyState postcopy_state_get(void); /* Set the state and return the old state */ PostcopyState postcopy_state_set(PostcopyState new_state); + +void dirty_bitmap_mig_before_vm_start(void); +void init_dirty_bitmap_incoming_migration(void); + #endif diff --git a/migration/Makefile.objs b/migration/Makefile.objs index 480dd49..fa3bf6a 100644 --- a/migration/Makefile.objs +++ b/migration/Makefile.objs @@ -9,5 +9,5 @@ common-obj-y +=3D qjson.o =20 common-obj-$(CONFIG_RDMA) +=3D rdma.o =20 -common-obj-y +=3D block.o +common-obj-y +=3D block.o block-dirty-bitmap.o =20 diff --git a/migration/block-dirty-bitmap.c b/migration/block-dirty-bitmap.c new file mode 100644 index 0000000..28e3732 --- /dev/null +++ b/migration/block-dirty-bitmap.c @@ -0,0 +1,679 @@ +/* + * Block dirty bitmap postcopy migration + * + * Copyright IBM, Corp. 2009 + * Copyright (C) 2016 Parallels IP Holdings GmbH. All rights reserved. + * + * Authors: + * Liran Schour + * Vladimir Sementsov-Ogievskiy + * + * This work is licensed under the terms of the GNU GPL, version 2. See + * the COPYING file in the top-level directory. + * This file is derived from migration/block.c, so it's author and IBM cop= yright + * are here, although content is quite different. + * + * Contributions after 2012-01-13 are licensed under the terms of the + * GNU GPL, version 2 or (at your option) any later version. + * + * *** + * + * Here postcopy migration of dirty bitmaps is realized. Only named dirty + * bitmaps, associated with root nodes and non-root named nodes are migrat= ed. + * + * If destination qemu is already containing a dirty bitmap with the same = name + * as a migrated bitmap (for the same node), then, if their granularities = are + * the same the migration will be done, otherwise the error will be genera= ted. + * + * If destination qemu doesn't contain such bitmap it will be created. + * + * format of migration: + * + * # Header (shared for different chunk types) + * 1, 2 or 4 bytes: flags (see qemu_{put,put}_flags) + * [ 1 byte: node name size ] \ flags & DEVICE_NAME + * [ n bytes: node name ] / + * [ 1 byte: bitmap name size ] \ flags & BITMAP_NAME + * [ n bytes: bitmap name ] / + * + * # Start of bitmap migration (flags & START) + * header + * be64: granularity + * 1 byte: bitmap enabled flag + * + * # Complete of bitmap migration (flags & COMPLETE) + * header + * + * # Data chunk of bitmap migration + * header + * be64: start sector + * be32: number of sectors + * [ be64: buffer size ] \ ! (flags & ZEROES) + * [ n bytes: buffer ] / + * + * The last chunk in stream should contain flags & EOS. The chunk may skip + * device and/or bitmap names, assuming them to be the same with the previ= ous + * chunk. + */ + +#include "qemu/osdep.h" +#include "block/block.h" +#include "block/block_int.h" +#include "sysemu/block-backend.h" +#include "qemu/main-loop.h" +#include "qemu/error-report.h" +#include "migration/block.h" +#include "migration/migration.h" +#include "qemu/hbitmap.h" +#include "sysemu/sysemu.h" +#include "qemu/cutils.h" +#include "qapi/error.h" +#include "trace.h" +#include + +#define CHUNK_SIZE (1 << 10) + +/* Flags occupy from one to four bytes. In all but one the 7-th (EXTRA_FLA= GS) + * bit should be set. */ +#define DIRTY_BITMAP_MIG_FLAG_EOS 0x01 +#define DIRTY_BITMAP_MIG_FLAG_ZEROES 0x02 +#define DIRTY_BITMAP_MIG_FLAG_BITMAP_NAME 0x04 +#define DIRTY_BITMAP_MIG_FLAG_DEVICE_NAME 0x08 +#define DIRTY_BITMAP_MIG_FLAG_START 0x10 +#define DIRTY_BITMAP_MIG_FLAG_COMPLETE 0x20 +#define DIRTY_BITMAP_MIG_FLAG_BITS 0x40 + +#define DIRTY_BITMAP_MIG_EXTRA_FLAGS 0x80 +#define DIRTY_BITMAP_MIG_FLAGS_SIZE_16 0x8000 +#define DIRTY_BITMAP_MIG_FLAGS_SIZE_32 0x8080 + +#define DEBUG_DIRTY_BITMAP_MIGRATION 0 + +typedef struct DirtyBitmapMigBitmapState { + /* Written during setup phase. */ + BlockDriverState *bs; + const char *node_name; + BdrvDirtyBitmap *bitmap; + uint64_t total_sectors; + uint64_t sectors_per_chunk; + QSIMPLEQ_ENTRY(DirtyBitmapMigBitmapState) entry; + + /* For bulk phase. */ + bool bulk_completed; + uint64_t cur_sector; +} DirtyBitmapMigBitmapState; + +typedef struct DirtyBitmapMigState { + QSIMPLEQ_HEAD(dbms_list, DirtyBitmapMigBitmapState) dbms_list; + + bool bulk_completed; + + /* for send_bitmap_bits() */ + BlockDriverState *prev_bs; + BdrvDirtyBitmap *prev_bitmap; +} DirtyBitmapMigState; + +typedef struct DirtyBitmapLoadState { + uint32_t flags; + char node_name[256]; + char bitmap_name[256]; + BlockDriverState *bs; + BdrvDirtyBitmap *bitmap; +} DirtyBitmapLoadState; + +static DirtyBitmapMigState dirty_bitmap_mig_state; + +typedef struct DirtyBitmapLoadBitmapState { + BlockDriverState *bs; + BdrvDirtyBitmap *bitmap; + bool migrated; +} DirtyBitmapLoadBitmapState; +static GSList *enabled_bitmaps; +QemuMutex finish_lock; + +void init_dirty_bitmap_incoming_migration(void) +{ + qemu_mutex_init(&finish_lock); +} + +static uint32_t qemu_get_bitmap_flags(QEMUFile *f) +{ + uint8_t flags =3D qemu_get_byte(f); + if (flags & DIRTY_BITMAP_MIG_EXTRA_FLAGS) { + flags =3D flags << 8 | qemu_get_byte(f); + if (flags & DIRTY_BITMAP_MIG_EXTRA_FLAGS) { + flags =3D flags << 16 | qemu_get_be16(f); + } + } + + return flags; +} + +static void qemu_put_bitmap_flags(QEMUFile *f, uint32_t flags) +{ + if (!(flags & 0xffffff00)) { + qemu_put_byte(f, flags); + return; + } + + if (!(flags & 0xffff0000)) { + qemu_put_be16(f, flags | DIRTY_BITMAP_MIG_FLAGS_SIZE_16); + return; + } + + qemu_put_be32(f, flags | DIRTY_BITMAP_MIG_FLAGS_SIZE_32); +} + +static void send_bitmap_header(QEMUFile *f, DirtyBitmapMigBitmapState *dbm= s, + uint32_t additional_flags) +{ + BlockDriverState *bs =3D dbms->bs; + BdrvDirtyBitmap *bitmap =3D dbms->bitmap; + uint32_t flags =3D additional_flags; + trace_send_bitmap_header_enter(); + + if (bs !=3D dirty_bitmap_mig_state.prev_bs) { + dirty_bitmap_mig_state.prev_bs =3D bs; + flags |=3D DIRTY_BITMAP_MIG_FLAG_DEVICE_NAME; + } + + if (bitmap !=3D dirty_bitmap_mig_state.prev_bitmap) { + dirty_bitmap_mig_state.prev_bitmap =3D bitmap; + flags |=3D DIRTY_BITMAP_MIG_FLAG_BITMAP_NAME; + } + + qemu_put_bitmap_flags(f, flags); + + if (flags & DIRTY_BITMAP_MIG_FLAG_DEVICE_NAME) { + qemu_put_counted_string(f, dbms->node_name); + } + + if (flags & DIRTY_BITMAP_MIG_FLAG_BITMAP_NAME) { + qemu_put_counted_string(f, bdrv_dirty_bitmap_name(bitmap)); + } +} + +static void send_bitmap_start(QEMUFile *f, DirtyBitmapMigBitmapState *dbms) +{ + send_bitmap_header(f, dbms, DIRTY_BITMAP_MIG_FLAG_START); + qemu_put_be32(f, bdrv_dirty_bitmap_granularity(dbms->bitmap)); + qemu_put_byte(f, bdrv_dirty_bitmap_enabled(dbms->bitmap)); +} + +static void send_bitmap_complete(QEMUFile *f, DirtyBitmapMigBitmapState *d= bms) +{ + send_bitmap_header(f, dbms, DIRTY_BITMAP_MIG_FLAG_COMPLETE); +} + +static void send_bitmap_bits(QEMUFile *f, DirtyBitmapMigBitmapState *dbms, + uint64_t start_sector, uint32_t nr_sectors) +{ + /* align for buffer_is_zero() */ + uint64_t align =3D 4 * sizeof(long); + uint64_t unaligned_size =3D + bdrv_dirty_bitmap_serialization_size(dbms->bitmap, + start_sector, nr_sectors); + uint64_t buf_size =3D (unaligned_size + align - 1) & ~(align - 1); + uint8_t *buf =3D g_malloc0(buf_size); + uint32_t flags =3D DIRTY_BITMAP_MIG_FLAG_BITS; + + bdrv_dirty_bitmap_serialize_part(dbms->bitmap, buf, + start_sector, nr_sectors); + + if (buffer_is_zero(buf, buf_size)) { + g_free(buf); + buf =3D NULL; + flags |=3D DIRTY_BITMAP_MIG_FLAG_ZEROES; + } + + trace_send_bitmap_bits(flags, start_sector, nr_sectors, buf_size); + + send_bitmap_header(f, dbms, flags); + + qemu_put_be64(f, start_sector); + qemu_put_be32(f, nr_sectors); + + /* if a block is zero we need to flush here since the network + * bandwidth is now a lot higher than the storage device bandwidth. + * thus if we queue zero blocks we slow down the migration. */ + if (flags & DIRTY_BITMAP_MIG_FLAG_ZEROES) { + qemu_fflush(f); + } else { + qemu_put_be64(f, buf_size); + qemu_put_buffer(f, buf, buf_size); + } + + g_free(buf); +} + + +/* Called with iothread lock taken. */ + +static void init_dirty_bitmap_migration(void) +{ + BlockDriverState *bs; + BdrvDirtyBitmap *bitmap; + DirtyBitmapMigBitmapState *dbms; + BdrvNextIterator it; + uint64_t total_bytes =3D 0; + + dirty_bitmap_mig_state.bulk_completed =3D false; + dirty_bitmap_mig_state.prev_bs =3D NULL; + dirty_bitmap_mig_state.prev_bitmap =3D NULL; + + for (bs =3D bdrv_first(&it); bs; bs =3D bdrv_next(&it)) { + for (bitmap =3D bdrv_next_dirty_bitmap(bs, NULL); bitmap; + bitmap =3D bdrv_next_dirty_bitmap(bs, bitmap)) { + if (!bdrv_dirty_bitmap_name(bitmap)) { + continue; + } + + if (!bdrv_get_device_or_node_name(bs)) { + /* not named non-root node */ + continue; + } + + dbms =3D g_new0(DirtyBitmapMigBitmapState, 1); + dbms->bs =3D bs; + dbms->node_name =3D bdrv_get_node_name(bs); + if (!dbms->node_name || dbms->node_name[0] =3D=3D '\0') { + dbms->node_name =3D bdrv_get_device_name(bs); + } + dbms->bitmap =3D bitmap; + dbms->total_sectors =3D bdrv_nb_sectors(bs); + dbms->sectors_per_chunk =3D CHUNK_SIZE * 8 * + bdrv_dirty_bitmap_granularity(bitmap) >> BDRV_SECTOR_BITS; + + total_bytes +=3D + bdrv_dirty_bitmap_serialization_size(bitmap, + 0, dbms->total_sector= s); + + QSIMPLEQ_INSERT_TAIL(&dirty_bitmap_mig_state.dbms_list, + dbms, entry); + } + } +} + +/* Called with no lock taken. */ +static void bulk_phase_send_chunk(QEMUFile *f, DirtyBitmapMigBitmapState *= dbms) +{ + uint32_t nr_sectors =3D MIN(dbms->total_sectors - dbms->cur_sector, + dbms->sectors_per_chunk); + + send_bitmap_bits(f, dbms, dbms->cur_sector, nr_sectors); + + dbms->cur_sector +=3D nr_sectors; + if (dbms->cur_sector >=3D dbms->total_sectors) { + dbms->bulk_completed =3D true; + } +} + +/* Called with no lock taken. */ +static void bulk_phase(QEMUFile *f, bool limit) +{ + DirtyBitmapMigBitmapState *dbms; + + QSIMPLEQ_FOREACH(dbms, &dirty_bitmap_mig_state.dbms_list, entry) { + while (!dbms->bulk_completed) { + bulk_phase_send_chunk(f, dbms); + if (limit && qemu_file_rate_limit(f)) { + return; + } + } + } + + dirty_bitmap_mig_state.bulk_completed =3D true; +} + +/* Called with iothread lock taken. */ +static void dirty_bitmap_mig_cleanup(void) +{ + DirtyBitmapMigBitmapState *dbms; + + while ((dbms =3D QSIMPLEQ_FIRST(&dirty_bitmap_mig_state.dbms_list)) != =3D NULL) { + QSIMPLEQ_REMOVE_HEAD(&dirty_bitmap_mig_state.dbms_list, entry); + g_free(dbms); + } +} + +/* for SaveVMHandlers */ +static void dirty_bitmap_migration_cleanup(void *opaque) +{ + dirty_bitmap_mig_cleanup(); +} + +static int dirty_bitmap_save_iterate(QEMUFile *f, void *opaque) +{ + trace_dirty_bitmap_save_iterate( + migration_in_postcopy(migrate_get_current())); + + if (migration_in_postcopy(migrate_get_current()) && + !dirty_bitmap_mig_state.bulk_completed) { + bulk_phase(f, true); + } + + qemu_put_bitmap_flags(f, DIRTY_BITMAP_MIG_FLAG_EOS); + + return dirty_bitmap_mig_state.bulk_completed; +} + +/* Called with iothread lock taken. */ + +static int dirty_bitmap_save_complete(QEMUFile *f, void *opaque) +{ + DirtyBitmapMigBitmapState *dbms; + trace_dirty_bitmap_save_complete_enter(); + + if (!dirty_bitmap_mig_state.bulk_completed) { + bulk_phase(f, false); + } + + QSIMPLEQ_FOREACH(dbms, &dirty_bitmap_mig_state.dbms_list, entry) { + send_bitmap_complete(f, dbms); + } + + qemu_put_bitmap_flags(f, DIRTY_BITMAP_MIG_FLAG_EOS); + + trace_dirty_bitmap_save_complete_finish(); + + dirty_bitmap_mig_cleanup(); + return 0; +} + +static void dirty_bitmap_save_pending(QEMUFile *f, void *opaque, + uint64_t max_size, + uint64_t *res_precopy_only, + uint64_t *res_compatible, + uint64_t *res_postcopy_only) +{ + DirtyBitmapMigBitmapState *dbms; + uint64_t pending =3D 0; + + qemu_mutex_lock_iothread(); + + QSIMPLEQ_FOREACH(dbms, &dirty_bitmap_mig_state.dbms_list, entry) { + uint64_t gran =3D bdrv_dirty_bitmap_granularity(dbms->bitmap); + uint64_t sectors =3D dbms->bulk_completed ? 0 : + dbms->total_sectors - dbms->cur_sector; + + pending +=3D (sectors * BDRV_SECTOR_SIZE + gran - 1) / gran; + } + + qemu_mutex_unlock_iothread(); + + trace_dirty_bitmap_save_pending(pending, max_size); + + *res_postcopy_only +=3D pending; +} + +/* First occurrence of this bitmap. It should be created if doesn't exist = */ +static int dirty_bitmap_load_start(QEMUFile *f, DirtyBitmapLoadState *s) +{ + Error *local_err =3D NULL; + uint32_t granularity =3D qemu_get_be32(f); + bool enabled =3D qemu_get_byte(f); + + if (!s->bitmap) { + s->bitmap =3D bdrv_create_dirty_bitmap(s->bs, granularity, + s->bitmap_name, &local_err); + if (!s->bitmap) { + error_report_err(local_err); + return -EINVAL; + } + } else { + uint32_t dest_granularity =3D + bdrv_dirty_bitmap_granularity(s->bitmap); + if (dest_granularity !=3D granularity) { + fprintf(stderr, + "Error: " + "Migrated bitmap granularity (%" PRIu32 ") " + "doesn't match the destination bitmap '%s' " + "granularity (%" PRIu32 ")\n", + granularity, + bdrv_dirty_bitmap_name(s->bitmap), + dest_granularity); + return -EINVAL; + } + } + + bdrv_disable_dirty_bitmap(s->bitmap); + if (enabled) { + DirtyBitmapLoadBitmapState *b; + + bdrv_dirty_bitmap_create_successor(s->bs, s->bitmap, &local_err); + if (local_err) { + error_report_err(local_err); + return -EINVAL; + } + + b =3D g_new(DirtyBitmapLoadBitmapState, 1); + b->bs =3D s->bs; + b->bitmap =3D s->bitmap; + b->migrated =3D false; + enabled_bitmaps =3D g_slist_prepend(enabled_bitmaps, b); + } + + return 0; +} + +void dirty_bitmap_mig_before_vm_start(void) +{ + GSList *item; + + qemu_mutex_lock(&finish_lock); + + for (item =3D enabled_bitmaps; item; item =3D g_slist_next(item)) { + DirtyBitmapLoadBitmapState *b =3D item->data; + + if (b->migrated) { + bdrv_enable_dirty_bitmap(b->bitmap); + } else { + bdrv_dirty_bitmap_enable_successor(b->bitmap); + } + + g_free(b); + } + + g_slist_free(enabled_bitmaps); + enabled_bitmaps =3D NULL; + + qemu_mutex_unlock(&finish_lock); +} + +static void dirty_bitmap_load_complete(QEMUFile *f, DirtyBitmapLoadState *= s) +{ + GSList *item; + trace_dirty_bitmap_load_complete(); + bdrv_dirty_bitmap_deserialize_finish(s->bitmap); + + qemu_mutex_lock(&finish_lock); + + for (item =3D enabled_bitmaps; item; item =3D g_slist_next(item)) { + DirtyBitmapLoadBitmapState *b =3D item->data; + + if (b->bitmap =3D=3D s->bitmap) { + b->migrated =3D true; + } + } + + if (bdrv_dirty_bitmap_frozen(s->bitmap)) { + if (enabled_bitmaps =3D=3D NULL) { + /* in postcopy */ + AioContext *aio_context =3D bdrv_get_aio_context(s->bs); + aio_context_acquire(aio_context); + + bdrv_reclaim_dirty_bitmap(s->bs, s->bitmap, &error_abort); + bdrv_enable_dirty_bitmap(s->bitmap); + + aio_context_release(aio_context); + } else { + /* target not started, successor is empty */ + bdrv_dirty_bitmap_release_successor(s->bs, s->bitmap); + } + } + + qemu_mutex_unlock(&finish_lock); +} + +static int dirty_bitmap_load_bits(QEMUFile *f, DirtyBitmapLoadState *s) +{ + uint64_t first_sector =3D qemu_get_be64(f); + uint32_t nr_sectors =3D qemu_get_be32(f); + trace_dirty_bitmap_load_bits_enter(first_sector, nr_sectors); + + if (s->flags & DIRTY_BITMAP_MIG_FLAG_ZEROES) { + trace_dirty_bitmap_load_bits_zeroes(); + bdrv_dirty_bitmap_deserialize_zeroes(s->bitmap, first_sector, + nr_sectors, false); + } else { + uint8_t *buf; + uint64_t buf_size =3D qemu_get_be64(f); + uint64_t needed_size =3D + bdrv_dirty_bitmap_serialization_size(s->bitmap, + first_sector, nr_sectors); + + if (needed_size > buf_size) { + fprintf(stderr, + "Error: Migrated bitmap granularity doesn't " + "match the destination bitmap '%s' granularity\n", + bdrv_dirty_bitmap_name(s->bitmap)); + return -EINVAL; + } + + buf =3D g_malloc(buf_size); + qemu_get_buffer(f, buf, buf_size); + bdrv_dirty_bitmap_deserialize_part(s->bitmap, buf, + first_sector, + nr_sectors, false); + g_free(buf); + } + + return 0; +} + +static int dirty_bitmap_load_header(QEMUFile *f, DirtyBitmapLoadState *s) +{ + Error *local_err =3D NULL; + s->flags =3D qemu_get_bitmap_flags(f); + trace_dirty_bitmap_load_header(s->flags); + + if (s->flags & DIRTY_BITMAP_MIG_FLAG_DEVICE_NAME) { + if (!qemu_get_counted_string(f, s->node_name)) { + fprintf(stderr, "Unable to read node name string\n"); + return -EINVAL; + } + s->bs =3D bdrv_lookup_bs(s->node_name, s->node_name, &local_err); + if (!s->bs) { + error_report("%s", error_get_pretty(local_err)); + error_free(local_err); + return -EINVAL; + } + } else if (!s->bs) { + fprintf(stderr, "Error: block device name is not set\n"); + return -EINVAL; + } + + if (s->flags & DIRTY_BITMAP_MIG_FLAG_BITMAP_NAME) { + if (!qemu_get_counted_string(f, s->bitmap_name)) { + fprintf(stderr, "Unable to read node name string\n"); + return -EINVAL; + } + s->bitmap =3D bdrv_find_dirty_bitmap(s->bs, s->bitmap_name); + + /* bitmap may be NULL here, it wouldn't be an error if it is the + * first occurrence of the bitmap */ + if (!s->bitmap && !(s->flags & DIRTY_BITMAP_MIG_FLAG_START)) { + fprintf(stderr, "Error: unknown dirty bitmap " + "'%s' for block device '%s'\n", + s->bitmap_name, s->node_name); + return -EINVAL; + } + } else if (!s->bitmap) { + fprintf(stderr, "Error: block device name is not set\n"); + return -EINVAL; + } + + return 0; +} + +static int dirty_bitmap_load(QEMUFile *f, void *opaque, int version_id) +{ + static DirtyBitmapLoadState s; + + int ret =3D 0; + + trace_dirty_bitmap_load_enter(); + + do { + dirty_bitmap_load_header(f, &s); + + if (s.flags & DIRTY_BITMAP_MIG_FLAG_START) { + ret =3D dirty_bitmap_load_start(f, &s); + } else if (s.flags & DIRTY_BITMAP_MIG_FLAG_COMPLETE) { + dirty_bitmap_load_complete(f, &s); + } else if (s.flags & DIRTY_BITMAP_MIG_FLAG_BITS) { + ret =3D dirty_bitmap_load_bits(f, &s); + } + + if (!ret) { + ret =3D qemu_file_get_error(f); + } + + if (ret) { + return ret; + } + } while (!(s.flags & DIRTY_BITMAP_MIG_FLAG_EOS)); + + trace_dirty_bitmap_load_success(); + return 0; +} + +static int dirty_bitmap_save_setup(QEMUFile *f, void *opaque) +{ + DirtyBitmapMigBitmapState *dbms =3D NULL; + init_dirty_bitmap_migration(); + + QSIMPLEQ_FOREACH(dbms, &dirty_bitmap_mig_state.dbms_list, entry) { + send_bitmap_start(f, dbms); + } + qemu_put_bitmap_flags(f, DIRTY_BITMAP_MIG_FLAG_EOS); + + return 0; +} + +static bool dirty_bitmap_is_active(void *opaque) +{ + return migrate_dirty_bitmaps(); +} + +static bool dirty_bitmap_is_active_iterate(void *opaque) +{ + return dirty_bitmap_is_active(opaque) && !runstate_is_running(); +} + +static bool dirty_bitmap_has_postcopy(void *opaque) +{ + return true; +} + +static SaveVMHandlers savevm_dirty_bitmap_handlers =3D { + .save_live_setup =3D dirty_bitmap_save_setup, + .save_live_complete_postcopy =3D dirty_bitmap_save_complete, + .save_live_complete_precopy =3D dirty_bitmap_save_complete, + .has_postcopy =3D dirty_bitmap_has_postcopy, + .save_live_pending =3D dirty_bitmap_save_pending, + .save_live_iterate =3D dirty_bitmap_save_iterate, + .is_active_iterate =3D dirty_bitmap_is_active_iterate, + .load_state =3D dirty_bitmap_load, + .cleanup =3D dirty_bitmap_migration_cleanup, + .is_active =3D dirty_bitmap_is_active, +}; + +void dirty_bitmap_mig_init(void) +{ + QSIMPLEQ_INIT(&dirty_bitmap_mig_state.dbms_list); + + register_savevm_live(NULL, "dirty-bitmap", 0, 1, + &savevm_dirty_bitmap_handlers, + &dirty_bitmap_mig_state); +} diff --git a/migration/migration.c b/migration/migration.c index 4716efd..221be74 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -387,6 +387,9 @@ static void process_incoming_migration_co(void *opaque) int ret; =20 mis =3D migration_incoming_state_new(f); + + init_dirty_bitmap_incoming_migration(); + postcopy_state_set(POSTCOPY_INCOMING_NONE); migrate_set_state(&mis->state, MIGRATION_STATUS_NONE, MIGRATION_STATUS_ACTIVE); diff --git a/migration/savevm.c b/migration/savevm.c index 94783e5..cc44e5e 100644 --- a/migration/savevm.c +++ b/migration/savevm.c @@ -1638,6 +1638,8 @@ static void loadvm_postcopy_handle_run_bh(void *opaqu= e) =20 trace_loadvm_postcopy_handle_run_vmstart(); =20 + dirty_bitmap_mig_before_vm_start(); + if (autostart) { /* Hold onto your hats, starting the CPU */ vm_start(); diff --git a/migration/trace-events b/migration/trace-events index 0212929..38fca41 100644 --- a/migration/trace-events +++ b/migration/trace-events @@ -222,3 +222,17 @@ colo_vm_state_change(const char *old, const char *new)= "Change '%s' =3D> '%s'" colo_send_message(const char *msg) "Send '%s' message" colo_receive_message(const char *msg) "Receive '%s' message" colo_failover_set_state(const char *new_state) "new state %s" + +# migration/block-dirty-bitmap.c +send_bitmap_header_enter(void) "" +send_bitmap_bits(uint32_t flags, uint64_t start_sector, uint32_t nr_sector= s, uint64_t data_size) "\n flags: %x\n start_sector: %" PRIu64 "= \n nr_sectors: %" PRIu32 "\n data_size: %" PRIu64 "\n" +dirty_bitmap_save_iterate(int in_postcopy) "in postcopy: %d" +dirty_bitmap_save_complete_enter(void) "" +dirty_bitmap_save_complete_finish(void) "" +dirty_bitmap_save_pending(uint64_t pending, uint64_t max_size) "pending %"= PRIu64 " max: %" PRIu64 +dirty_bitmap_load_complete(void) "" +dirty_bitmap_load_bits_enter(uint64_t first_sector, uint32_t nr_sectors) "= chunk: %" PRIu64 " %" PRIu32 +dirty_bitmap_load_bits_zeroes(void) "" +dirty_bitmap_load_header(uint32_t flags) "flags %x" +dirty_bitmap_load_enter(void) "" +dirty_bitmap_load_success(void) "" diff --git a/vl.c b/vl.c index b4eaf03..f1ee9ff 100644 --- a/vl.c +++ b/vl.c @@ -4405,6 +4405,7 @@ int main(int argc, char **argv, char **envp) =20 blk_mig_init(); ram_mig_init(); + dirty_bitmap_mig_init(); =20 /* If the currently selected machine wishes to override the units-per-= bus * property of its default HBA interface type, do so now. */ --=20 1.8.3.1 From nobody Sun Apr 28 01:03:39 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1486480965739279.3605910468724; Tue, 7 Feb 2017 07:22:45 -0800 (PST) Received: from localhost ([::1]:54770 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cb7bQ-000692-AS for importer@patchew.org; Tue, 07 Feb 2017 10:22:44 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:57779) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cb7L9-0000VD-2A for qemu-devel@nongnu.org; Tue, 07 Feb 2017 10:06:00 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1cb7L3-000781-Fj for qemu-devel@nongnu.org; Tue, 07 Feb 2017 10:05:55 -0500 Received: from mailhub.sw.ru ([195.214.232.25]:34853 helo=relay.sw.ru) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1cb7L3-00072a-3R; Tue, 07 Feb 2017 10:05:49 -0500 Received: from kvm.sw.ru (msk-vpn.virtuozzo.com [195.214.232.6]) by relay.sw.ru (8.13.4/8.13.4) with ESMTP id v17F5elA013037; Tue, 7 Feb 2017 18:05:42 +0300 (MSK) From: Vladimir Sementsov-Ogievskiy To: qemu-block@nongnu.org, qemu-devel@nongnu.org Date: Tue, 7 Feb 2017 18:05:36 +0300 Message-Id: <1486479940-163258-14-git-send-email-vsementsov@virtuozzo.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1486479940-163258-1-git-send-email-vsementsov@virtuozzo.com> References: <1486479940-163258-1-git-send-email-vsementsov@virtuozzo.com> X-detected-operating-system: by eggs.gnu.org: OpenBSD 3.x [fuzzy] X-Received-From: 195.214.232.25 Subject: [Qemu-devel] [PATCH 13/17] iotests: add add_incoming_migration to VM class X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, peter.maydell@linaro.org, vsementsov@virtuozzo.com, famz@redhat.com, lirans@il.ibm.com, quintela@redhat.com, jsnow@redhat.com, armbru@redhat.com, mreitz@redhat.com, stefanha@redhat.com, den@openvz.org, amit.shah@redhat.com, pbonzini@redhat.com, dgilbert@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: John Snow --- tests/qemu-iotests/iotests.py | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py index bec8eb4..f5ca4b8 100644 --- a/tests/qemu-iotests/iotests.py +++ b/tests/qemu-iotests/iotests.py @@ -177,6 +177,12 @@ class VM(qtest.QEMUQtestMachine): self._num_drives +=3D 1 return self =20 + def add_incoming_migration(self, desc): + '''Add an incoming migration to the VM''' + self._args.append('-incoming') + self._args.append(desc) + return self + def pause_drive(self, drive, event=3DNone): '''Pause drive r/w operations''' if not event: --=20 1.8.3.1 From nobody Sun Apr 28 01:03:39 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1486481422433251.70645949454274; Tue, 7 Feb 2017 07:30:22 -0800 (PST) Received: from localhost ([::1]:54836 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cb7il-0004sA-S9 for importer@patchew.org; Tue, 07 Feb 2017 10:30:19 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:57786) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cb7L9-0000Vn-7U for qemu-devel@nongnu.org; Tue, 07 Feb 2017 10:06:01 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1cb7L3-00077d-7d for qemu-devel@nongnu.org; Tue, 07 Feb 2017 10:05:55 -0500 Received: from mailhub.sw.ru ([195.214.232.25]:1308 helo=relay.sw.ru) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1cb7L2-00072b-FQ; Tue, 07 Feb 2017 10:05:49 -0500 Received: from kvm.sw.ru (msk-vpn.virtuozzo.com [195.214.232.6]) by relay.sw.ru (8.13.4/8.13.4) with ESMTP id v17F5elB013037; Tue, 7 Feb 2017 18:05:42 +0300 (MSK) From: Vladimir Sementsov-Ogievskiy To: qemu-block@nongnu.org, qemu-devel@nongnu.org Date: Tue, 7 Feb 2017 18:05:37 +0300 Message-Id: <1486479940-163258-15-git-send-email-vsementsov@virtuozzo.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1486479940-163258-1-git-send-email-vsementsov@virtuozzo.com> References: <1486479940-163258-1-git-send-email-vsementsov@virtuozzo.com> X-detected-operating-system: by eggs.gnu.org: OpenBSD 3.x [fuzzy] X-Received-From: 195.214.232.25 Subject: [Qemu-devel] [PATCH 14/17] qmp: add x-debug-block-dirty-bitmap-sha256 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, peter.maydell@linaro.org, vsementsov@virtuozzo.com, famz@redhat.com, lirans@il.ibm.com, quintela@redhat.com, jsnow@redhat.com, armbru@redhat.com, mreitz@redhat.com, stefanha@redhat.com, den@openvz.org, amit.shah@redhat.com, pbonzini@redhat.com, dgilbert@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Max Reitz --- block/dirty-bitmap.c | 5 +++++ blockdev.c | 29 +++++++++++++++++++++++++++++ include/block/dirty-bitmap.h | 2 ++ include/qemu/hbitmap.h | 8 ++++++++ qapi/block-core.json | 27 +++++++++++++++++++++++++++ tests/Makefile.include | 2 +- util/hbitmap.c | 11 +++++++++++ 7 files changed, 83 insertions(+), 1 deletion(-) diff --git a/block/dirty-bitmap.c b/block/dirty-bitmap.c index 32aa6eb..5bec99b 100644 --- a/block/dirty-bitmap.c +++ b/block/dirty-bitmap.c @@ -558,3 +558,8 @@ BdrvDirtyBitmap *bdrv_next_dirty_bitmap(BlockDriverStat= e *bs, =20 return QLIST_NEXT(bitmap, list); } + +char *bdrv_dirty_bitmap_sha256(const BdrvDirtyBitmap *bitmap, Error **errp) +{ + return hbitmap_sha256(bitmap->bitmap, errp); +} diff --git a/blockdev.c b/blockdev.c index db82ac9..4d06885 100644 --- a/blockdev.c +++ b/blockdev.c @@ -2790,6 +2790,35 @@ void qmp_block_dirty_bitmap_clear(const char *node, = const char *name, aio_context_release(aio_context); } =20 +BlockDirtyBitmapSha256 *qmp_x_debug_block_dirty_bitmap_sha256(const char *= node, + const char *= name, + Error **errp) +{ + AioContext *aio_context; + BdrvDirtyBitmap *bitmap; + BlockDriverState *bs; + BlockDirtyBitmapSha256 *ret =3D NULL; + char *sha256; + + bitmap =3D block_dirty_bitmap_lookup(node, name, &bs, &aio_context, er= rp); + if (!bitmap || !bs) { + return NULL; + } + + sha256 =3D bdrv_dirty_bitmap_sha256(bitmap, errp); + if (sha256 =3D=3D NULL) { + goto out; + } + + ret =3D g_new(BlockDirtyBitmapSha256, 1); + ret->sha256 =3D sha256; + +out: + aio_context_release(aio_context); + + return ret; +} + void hmp_drive_del(Monitor *mon, const QDict *qdict) { const char *id =3D qdict_get_str(qdict, "id"); diff --git a/include/block/dirty-bitmap.h b/include/block/dirty-bitmap.h index 20b3ec7..ded872a 100644 --- a/include/block/dirty-bitmap.h +++ b/include/block/dirty-bitmap.h @@ -78,4 +78,6 @@ void bdrv_dirty_bitmap_deserialize_finish(BdrvDirtyBitmap= *bitmap); BdrvDirtyBitmap *bdrv_next_dirty_bitmap(BlockDriverState *bs, BdrvDirtyBitmap *bitmap); =20 +char *bdrv_dirty_bitmap_sha256(const BdrvDirtyBitmap *bitmap, Error **errp= ); + #endif diff --git a/include/qemu/hbitmap.h b/include/qemu/hbitmap.h index 9239fe5..f353e56 100644 --- a/include/qemu/hbitmap.h +++ b/include/qemu/hbitmap.h @@ -238,6 +238,14 @@ void hbitmap_deserialize_zeroes(HBitmap *hb, uint64_t = start, uint64_t count, void hbitmap_deserialize_finish(HBitmap *hb); =20 /** + * hbitmap_sha256: + * @bitmap: HBitmap to operate on. + * + * Returns SHA256 hash of the last level. + */ +char *hbitmap_sha256(const HBitmap *bitmap, Error **errp); + +/** * hbitmap_free: * @hb: HBitmap to operate on. * diff --git a/qapi/block-core.json b/qapi/block-core.json index 932f5bb..8646054 100644 --- a/qapi/block-core.json +++ b/qapi/block-core.json @@ -1632,6 +1632,33 @@ 'data': 'BlockDirtyBitmap' } =20 ## +# @BlockDirtyBitmapSha256: +# +# SHA256 hash of dirty bitmap data +# +# @sha256: ASCII representation of SHA256 bitmap hash +# +# Since: 2.9 +## + { 'struct': 'BlockDirtyBitmapSha256', + 'data': {'sha256': 'str'} } + +## +# @x-debug-block-dirty-bitmap-sha256: +# +# Get bitmap SHA256 +# +# Returns: BlockDirtyBitmapSha256 on success +# If @node is not a valid block device, DeviceNotFound +# If @name is not found or if hashing has failed, GenericError wi= th an +# explanation +# +# Since: 2.9 +## + { 'command': 'x-debug-block-dirty-bitmap-sha256', + 'data': 'BlockDirtyBitmap', 'returns': 'BlockDirtyBitmapSha256' } + +## # @blockdev-mirror: # # Start mirroring a block device's writes to a new destination. diff --git a/tests/Makefile.include b/tests/Makefile.include index 634394a..7a71b4d 100644 --- a/tests/Makefile.include +++ b/tests/Makefile.include @@ -526,7 +526,7 @@ tests/test-blockjob$(EXESUF): tests/test-blockjob.o $(t= est-block-obj-y) $(test-u tests/test-blockjob-txn$(EXESUF): tests/test-blockjob-txn.o $(test-block-o= bj-y) $(test-util-obj-y) tests/test-thread-pool$(EXESUF): tests/test-thread-pool.o $(test-block-obj= -y) tests/test-iov$(EXESUF): tests/test-iov.o $(test-util-obj-y) -tests/test-hbitmap$(EXESUF): tests/test-hbitmap.o $(test-util-obj-y) +tests/test-hbitmap$(EXESUF): tests/test-hbitmap.o $(test-util-obj-y) $(tes= t-crypto-obj-y) tests/test-x86-cpuid$(EXESUF): tests/test-x86-cpuid.o tests/test-xbzrle$(EXESUF): tests/test-xbzrle.o migration/xbzrle.o page_ca= che.o $(test-util-obj-y) tests/test-cutils$(EXESUF): tests/test-cutils.o util/cutils.o diff --git a/util/hbitmap.c b/util/hbitmap.c index 35088e1..412b0fa 100644 --- a/util/hbitmap.c +++ b/util/hbitmap.c @@ -13,6 +13,7 @@ #include "qemu/hbitmap.h" #include "qemu/host-utils.h" #include "trace.h" +#include "crypto/hash.h" =20 /* HBitmaps provides an array of bits. The bits are stored as usual in an * array of unsigned longs, but HBitmap is also optimized to provide fast @@ -689,3 +690,13 @@ void hbitmap_free_meta(HBitmap *hb) hbitmap_free(hb->meta); hb->meta =3D NULL; } + +char *hbitmap_sha256(const HBitmap *bitmap, Error **errp) +{ + size_t size =3D bitmap->sizes[HBITMAP_LEVELS - 1] * sizeof(unsigned lo= ng); + char *data =3D (char *)bitmap->levels[HBITMAP_LEVELS - 1]; + char *hash =3D NULL; + qcrypto_hash_digest(QCRYPTO_HASH_ALG_SHA256, data, size, &hash, errp); + + return hash; +} --=20 1.8.3.1 From nobody Sun Apr 28 01:03:39 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1486480099255923.3721432557204; Tue, 7 Feb 2017 07:08:19 -0800 (PST) Received: from localhost ([::1]:54654 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cb7NQ-0002An-Gk for importer@patchew.org; Tue, 07 Feb 2017 10:08:16 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:57806) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cb7L9-0000WA-MO for qemu-devel@nongnu.org; Tue, 07 Feb 2017 10:06:03 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1cb7L4-00078i-Ds for qemu-devel@nongnu.org; Tue, 07 Feb 2017 10:05:55 -0500 Received: from mailhub.sw.ru ([195.214.232.25]:15242 helo=relay.sw.ru) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1cb7L3-00072R-6A; Tue, 07 Feb 2017 10:05:49 -0500 Received: from kvm.sw.ru (msk-vpn.virtuozzo.com [195.214.232.6]) by relay.sw.ru (8.13.4/8.13.4) with ESMTP id v17F5elC013037; Tue, 7 Feb 2017 18:05:42 +0300 (MSK) From: Vladimir Sementsov-Ogievskiy To: qemu-block@nongnu.org, qemu-devel@nongnu.org Date: Tue, 7 Feb 2017 18:05:38 +0300 Message-Id: <1486479940-163258-16-git-send-email-vsementsov@virtuozzo.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1486479940-163258-1-git-send-email-vsementsov@virtuozzo.com> References: <1486479940-163258-1-git-send-email-vsementsov@virtuozzo.com> X-detected-operating-system: by eggs.gnu.org: OpenBSD 3.x [fuzzy] X-Received-From: 195.214.232.25 Subject: [Qemu-devel] [PATCH 15/17] iotests: add default node-name X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, peter.maydell@linaro.org, vsementsov@virtuozzo.com, famz@redhat.com, lirans@il.ibm.com, quintela@redhat.com, jsnow@redhat.com, armbru@redhat.com, mreitz@redhat.com, stefanha@redhat.com, den@openvz.org, amit.shah@redhat.com, pbonzini@redhat.com, dgilbert@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" When testing migration, auto-generated by qemu node-names differs in source and destination qemu and migration fails. After this patch, auto-generated by iotest nodenames will be the same. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Max Reitz --- tests/qemu-iotests/iotests.py | 2 ++ 1 file changed, 2 insertions(+) diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py index f5ca4b8..e110c90 100644 --- a/tests/qemu-iotests/iotests.py +++ b/tests/qemu-iotests/iotests.py @@ -168,6 +168,8 @@ class VM(qtest.QEMUQtestMachine): options.append('file=3D%s' % path) options.append('format=3D%s' % format) options.append('cache=3D%s' % cachemode) + if 'node-name' not in opts: + options.append('node-name=3Ddrivenode%d' % self._num_drive= s) =20 if opts: options.append(opts) --=20 1.8.3.1 From nobody Sun Apr 28 01:03:39 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1486480911768234.55798857231628; Tue, 7 Feb 2017 07:21:51 -0800 (PST) Received: from localhost ([::1]:54767 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cb7aW-0005It-C4 for importer@patchew.org; Tue, 07 Feb 2017 10:21:48 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:57815) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cb7L9-0000WI-RI for qemu-devel@nongnu.org; Tue, 07 Feb 2017 10:06:00 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1cb7L4-000793-Rn for qemu-devel@nongnu.org; Tue, 07 Feb 2017 10:05:55 -0500 Received: from mailhub.sw.ru ([195.214.232.25]:2272 helo=relay.sw.ru) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1cb7L3-00072Y-6i; Tue, 07 Feb 2017 10:05:49 -0500 Received: from kvm.sw.ru (msk-vpn.virtuozzo.com [195.214.232.6]) by relay.sw.ru (8.13.4/8.13.4) with ESMTP id v17F5elD013037; Tue, 7 Feb 2017 18:05:42 +0300 (MSK) From: Vladimir Sementsov-Ogievskiy To: qemu-block@nongnu.org, qemu-devel@nongnu.org Date: Tue, 7 Feb 2017 18:05:39 +0300 Message-Id: <1486479940-163258-17-git-send-email-vsementsov@virtuozzo.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1486479940-163258-1-git-send-email-vsementsov@virtuozzo.com> References: <1486479940-163258-1-git-send-email-vsementsov@virtuozzo.com> X-detected-operating-system: by eggs.gnu.org: OpenBSD 3.x [fuzzy] X-Received-From: 195.214.232.25 Subject: [Qemu-devel] [PATCH 16/17] iotests: add dirty bitmap migration test X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, peter.maydell@linaro.org, vsementsov@virtuozzo.com, famz@redhat.com, lirans@il.ibm.com, quintela@redhat.com, jsnow@redhat.com, armbru@redhat.com, mreitz@redhat.com, stefanha@redhat.com, den@openvz.org, amit.shah@redhat.com, pbonzini@redhat.com, dgilbert@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" The test starts two vms (vm_a, vm_b), create dirty bitmap in the first one, do several writes to corresponding device and then migrate vm_a to vm_b with dirty bitmaps. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Max Reitz --- tests/qemu-iotests/169 | 86 ++++++++++++++++++++++++++++++++++++++++++= ++++ tests/qemu-iotests/169.out | 5 +++ tests/qemu-iotests/group | 1 + 3 files changed, 92 insertions(+) create mode 100755 tests/qemu-iotests/169 create mode 100644 tests/qemu-iotests/169.out diff --git a/tests/qemu-iotests/169 b/tests/qemu-iotests/169 new file mode 100755 index 0000000..ec6cd46 --- /dev/null +++ b/tests/qemu-iotests/169 @@ -0,0 +1,86 @@ +#!/usr/bin/env python +# +# Tests for dirty bitmaps migration. +# +# Copyright (C) Vladimir Sementsov-Ogievskiy 2015-2016 +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program. If not, see . +# + +import os +import iotests +import time +from iotests import qemu_img + +disk_a =3D os.path.join(iotests.test_dir, 'disk_a') +disk_b =3D os.path.join(iotests.test_dir, 'disk_b') +fifo =3D os.path.join(iotests.test_dir, 'mig_fifo') + +class TestDirtyBitmapMigration(iotests.QMPTestCase): + + def setUp(self): + size =3D 0x400000000 # 1G + os.mkfifo(fifo) + qemu_img('create', '-f', iotests.imgfmt, disk_a, str(size)) + qemu_img('create', '-f', iotests.imgfmt, disk_b, str(size)) + self.vm_a =3D iotests.VM(path_suffix=3D'a').add_drive(disk_a) + self.vm_b =3D iotests.VM(path_suffix=3D'b').add_drive(disk_b) + self.vm_b.add_incoming_migration("exec: cat '" + fifo + "'") + self.vm_a.launch() + self.vm_b.launch() + + def tearDown(self): + self.vm_a.shutdown() + self.vm_b.shutdown() + os.remove(disk_a) + os.remove(disk_b) + os.remove(fifo) + + def test_migration(self): + granularity =3D 512 + regions =3D [ + { 'start': 0, 'count': 0x100000 }, + { 'start': 0x100000000, 'count': 0x200000 }, + { 'start': 0x399900000, 'count': 0x100000 } + ] + + result =3D self.vm_a.qmp('block-dirty-bitmap-add', node=3D'drive0', + name=3D'bitmap', granularity=3Dgranularity) + self.assert_qmp(result, 'return', {}); + + for r in regions: + self.vm_a.hmp_qemu_io('drive0', + 'write %d %d' % (r['start'], r['count'])) + + result =3D self.vm_a.qmp('x-debug-block-dirty-bitmap-sha256', + node=3D'drive0', name=3D'bitmap') + sha256 =3D result['return']['sha256'] + + result =3D self.vm_a.qmp('migrate-set-capabilities', + capabilities=3D[{'capability': 'dirty-bitma= ps', + 'state': True}]) + self.assert_qmp(result, 'return', {}) + + result =3D self.vm_a.qmp('migrate', uri=3D'exec:cat>' + fifo) + self.assertNotEqual(self.vm_a.event_wait("STOP"), None) + self.assertNotEqual(self.vm_b.event_wait("RESUME"), None) + time.sleep(2) + + result =3D self.vm_b.qmp('x-debug-block-dirty-bitmap-sha256', + node=3D'drive0', name=3D'bitmap') + self.assert_qmp(result, 'return/sha256', sha256); + + +if __name__ =3D=3D '__main__': + iotests.main() diff --git a/tests/qemu-iotests/169.out b/tests/qemu-iotests/169.out new file mode 100644 index 0000000..ae1213e --- /dev/null +++ b/tests/qemu-iotests/169.out @@ -0,0 +1,5 @@ +. +---------------------------------------------------------------------- +Ran 1 tests + +OK diff --git a/tests/qemu-iotests/group b/tests/qemu-iotests/group index 866c1a0..5dc75ef 100644 --- a/tests/qemu-iotests/group +++ b/tests/qemu-iotests/group @@ -162,6 +162,7 @@ 159 rw auto quick 160 rw auto quick 162 auto quick +169 rw auto quick 170 rw auto quick 171 rw auto quick 172 auto --=20 1.8.3.1 From nobody Sun Apr 28 01:03:39 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1486481626281549.6143782939198; Tue, 7 Feb 2017 07:33:46 -0800 (PST) Received: from localhost ([::1]:54861 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cb7m3-0007gw-LI for importer@patchew.org; Tue, 07 Feb 2017 10:33:43 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:57947) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cb7LE-0000aU-Fd for qemu-devel@nongnu.org; Tue, 07 Feb 2017 10:06:06 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1cb7L4-00079A-UZ for qemu-devel@nongnu.org; Tue, 07 Feb 2017 10:06:00 -0500 Received: from mailhub.sw.ru ([195.214.232.25]:18397 helo=relay.sw.ru) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1cb7L3-00072W-3T; Tue, 07 Feb 2017 10:05:49 -0500 Received: from kvm.sw.ru (msk-vpn.virtuozzo.com [195.214.232.6]) by relay.sw.ru (8.13.4/8.13.4) with ESMTP id v17F5elE013037; Tue, 7 Feb 2017 18:05:42 +0300 (MSK) From: Vladimir Sementsov-Ogievskiy To: qemu-block@nongnu.org, qemu-devel@nongnu.org Date: Tue, 7 Feb 2017 18:05:40 +0300 Message-Id: <1486479940-163258-18-git-send-email-vsementsov@virtuozzo.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1486479940-163258-1-git-send-email-vsementsov@virtuozzo.com> References: <1486479940-163258-1-git-send-email-vsementsov@virtuozzo.com> X-detected-operating-system: by eggs.gnu.org: OpenBSD 3.x [fuzzy] X-Received-From: 195.214.232.25 Subject: [Qemu-devel] [PATCH 17/17] iotests: add dirty bitmap postcopy test X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, peter.maydell@linaro.org, vsementsov@virtuozzo.com, famz@redhat.com, lirans@il.ibm.com, quintela@redhat.com, jsnow@redhat.com, armbru@redhat.com, mreitz@redhat.com, stefanha@redhat.com, den@openvz.org, amit.shah@redhat.com, pbonzini@redhat.com, dgilbert@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Test - start two vms (vm_a, vm_b) - in a - do writes from set A - do writes from set B - fix bitmap sha256 - clear bitmap - do writes from set A - start migration - than, in b - wait vm start (postcopy should start) - do writes from set B - check bitmap sha256 The test should verify postcopy migration and then merging with delta (changes in target, during postcopy process). Reduce supported cache modes to only 'none', because with cache on time from source.STOP to target.RESUME is unpredictable and we can fail with timout while waiting for target.RESUME. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Max Reitz --- tests/qemu-iotests/169 | 74 +++++++++++++++++++++++++++++++++++++--= ---- tests/qemu-iotests/169.out | 4 +-- tests/qemu-iotests/iotests.py | 7 +++- 3 files changed, 72 insertions(+), 13 deletions(-) diff --git a/tests/qemu-iotests/169 b/tests/qemu-iotests/169 index ec6cd46..333df58 100755 --- a/tests/qemu-iotests/169 +++ b/tests/qemu-iotests/169 @@ -29,8 +29,14 @@ fifo =3D os.path.join(iotests.test_dir, 'mig_fifo') =20 class TestDirtyBitmapMigration(iotests.QMPTestCase): =20 - def setUp(self): - size =3D 0x400000000 # 1G + def tearDown(self): + self.vm_a.shutdown() + self.vm_b.shutdown() + os.remove(disk_a) + os.remove(disk_b) + os.remove(fifo) + + def init(self, size): os.mkfifo(fifo) qemu_img('create', '-f', iotests.imgfmt, disk_a, str(size)) qemu_img('create', '-f', iotests.imgfmt, disk_b, str(size)) @@ -40,14 +46,8 @@ class TestDirtyBitmapMigration(iotests.QMPTestCase): self.vm_a.launch() self.vm_b.launch() =20 - def tearDown(self): - self.vm_a.shutdown() - self.vm_b.shutdown() - os.remove(disk_a) - os.remove(disk_b) - os.remove(fifo) - def test_migration(self): + self.init(0x400000000) # 1G granularity =3D 512 regions =3D [ { 'start': 0, 'count': 0x100000 }, @@ -81,6 +81,60 @@ class TestDirtyBitmapMigration(iotests.QMPTestCase): node=3D'drive0', name=3D'bitmap') self.assert_qmp(result, 'return/sha256', sha256); =20 + def test_postcopy(self): + self.init(0x4000000000) # 256G + write_size =3D 0x40000000 + granularity =3D 512 + chunk =3D 4096 + + result =3D self.vm_a.qmp('block-dirty-bitmap-add', node=3D'drive0', + name=3D'bitmap', granularity=3Dgranularity) + self.assert_qmp(result, 'return', {}); + + s =3D 0 + while s < write_size: + self.vm_a.hmp_qemu_io('drive0', 'write %d %d' % (s, chunk)) + s +=3D 0x10000 + s =3D 0x8000 + while s < write_size: + self.vm_a.hmp_qemu_io('drive0', 'write %d %d' % (s, chunk)) + s +=3D 0x10000 + + result =3D self.vm_a.qmp('x-debug-block-dirty-bitmap-sha256', + node=3D'drive0', name=3D'bitmap') + sha256 =3D result['return']['sha256'] + + result =3D self.vm_a.qmp('block-dirty-bitmap-clear', node=3D'drive= 0', + name=3D'bitmap') + self.assert_qmp(result, 'return', {}); + s =3D 0 + while s < write_size: + self.vm_a.hmp_qemu_io('drive0', 'write %d %d' % (s, chunk)) + s +=3D 0x10000 + + result =3D self.vm_a.qmp('migrate-set-capabilities', + capabilities=3D[{'capability': 'dirty-bitma= ps', + 'state': True}]) + self.assert_qmp(result, 'return', {}) + + result =3D self.vm_a.qmp('migrate', uri=3D'exec:cat>' + fifo) + self.assertNotEqual(self.vm_a.event_wait("STOP"), None) + self.assertNotEqual(self.vm_b.event_wait("RESUME"), None) + + s =3D 0x8000 + while s < write_size: + self.vm_b.hmp_qemu_io('drive0', 'write %d %d' % (s, chunk)) + s +=3D 0x10000 + + result =3D self.vm_b.qmp('query-block'); + while len(result['return'][0]['dirty-bitmaps']) > 1: + time.sleep(2) + result =3D self.vm_b.qmp('query-block'); + + result =3D self.vm_b.qmp('x-debug-block-dirty-bitmap-sha256', + node=3D'drive0', name=3D'bitmap') + + self.assert_qmp(result, 'return/sha256', sha256); =20 if __name__ =3D=3D '__main__': - iotests.main() + iotests.main(supported_fmts=3D['qcow2'], supported_cache_modes=3D['non= e']) diff --git a/tests/qemu-iotests/169.out b/tests/qemu-iotests/169.out index ae1213e..fbc63e6 100644 --- a/tests/qemu-iotests/169.out +++ b/tests/qemu-iotests/169.out @@ -1,5 +1,5 @@ -. +.. ---------------------------------------------------------------------- -Ran 1 tests +Ran 2 tests =20 OK diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py index e110c90..7b64411 100644 --- a/tests/qemu-iotests/iotests.py +++ b/tests/qemu-iotests/iotests.py @@ -356,6 +356,10 @@ def verify_platform(supported_oses=3D['linux']): if True not in [sys.platform.startswith(x) for x in supported_oses]: notrun('not suitable for this OS: %s' % sys.platform) =20 +def verify_cache_mode(supported_cache_modes=3D[]): + if supported_cache_modes and (cachemode not in supported_cache_modes): + notrun('not suitable for this cache mode: %s' % cachemode) + def supports_quorum(): return 'quorum' in qemu_img_pipe('--help') =20 @@ -364,7 +368,7 @@ def verify_quorum(): if not supports_quorum(): notrun('quorum support missing') =20 -def main(supported_fmts=3D[], supported_oses=3D['linux']): +def main(supported_fmts=3D[], supported_oses=3D['linux'], supported_cache_= modes=3D[]): '''Run tests''' =20 global debug @@ -381,6 +385,7 @@ def main(supported_fmts=3D[], supported_oses=3D['linux'= ]): verbosity =3D 1 verify_image_format(supported_fmts) verify_platform(supported_oses) + verify_cache_mode(supported_cache_modes) =20 # We need to filter out the time taken from the output so that qemu-io= test # can reliably diff the results against master output. --=20 1.8.3.1