From nobody Sun Apr 28 04:06:08 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (208.118.235.17 [208.118.235.17]) by mx.zohomail.com with SMTPS id 1508137036427509.9379614780347; Sun, 15 Oct 2017 23:57:16 -0700 (PDT) Received: from localhost ([::1]:59620 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1e3zKe-0003hm-HL for importer@patchew.org; Mon, 16 Oct 2017 02:57:00 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:42228) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1e3zGK-0000Lh-58 for qemu-devel@nongnu.org; Mon, 16 Oct 2017 02:52:33 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1e3zGI-00013l-R6 for qemu-devel@nongnu.org; Mon, 16 Oct 2017 02:52:32 -0400 Received: from mx1.redhat.com ([209.132.183.28]:36722) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1e3zGI-00012c-Il for qemu-devel@nongnu.org; Mon, 16 Oct 2017 02:52:30 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 75D3381DEB; Mon, 16 Oct 2017 06:52:29 +0000 (UTC) Received: from pxdev.xzpeter.org.com (dhcp-15-225.nay.redhat.com [10.66.15.225]) by smtp.corp.redhat.com (Postfix) with ESMTP id 0840A62930; Mon, 16 Oct 2017 06:52:26 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com 75D3381DEB Authentication-Results: ext-mx01.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx01.extmail.prod.ext.phx2.redhat.com; spf=fail smtp.mailfrom=peterx@redhat.com From: Peter Xu To: qemu-devel@nongnu.org Date: Mon, 16 Oct 2017 14:51:45 +0800 Message-Id: <20171016065216.18162-2-peterx@redhat.com> In-Reply-To: <20171016065216.18162-1-peterx@redhat.com> References: <20171016065216.18162-1-peterx@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.25]); Mon, 16 Oct 2017 06:52:29 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH v3 01/32] migration: better error handling with QEMUFile X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Andrea Arcangeli , Juan Quintela , Alexey Perevalov , peterx@redhat.com, "Dr . David Alan Gilbert" Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" If the postcopy down due to some reason, we can always see this on dst: qemu-system-x86_64: RP: Received invalid message 0x0000 length 0x0000 However in most cases that's not the real issue. The problem is that qemu_get_be16() has no way to show whether the returned data is valid or not, and we are _always_ assuming it is valid. That's possibly not wise. The best approach to solve this would be: refactoring QEMUFile interface to allow the APIs to return error if there is. However it needs quite a bit of work and testing. For now, let's explicitly check the validity first before using the data in all places for qemu_get_*(). This patch tries to fix most of the cases I can see. Only if we are with this, can we make sure we are processing the valid data, and also can we make sure we can capture the channel down events correctly. Signed-off-by: Peter Xu --- migration/migration.c | 5 +++++ migration/ram.c | 26 ++++++++++++++++++++++---- migration/savevm.c | 40 ++++++++++++++++++++++++++++++++++++++-- 3 files changed, 65 insertions(+), 6 deletions(-) diff --git a/migration/migration.c b/migration/migration.c index 157b4e4c7d..3366bdddcc 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -1657,6 +1657,11 @@ static void *source_return_path_thread(void *opaque) header_type =3D qemu_get_be16(rp); header_len =3D qemu_get_be16(rp); =20 + if (qemu_file_get_error(rp)) { + mark_source_rp_bad(ms); + goto out; + } + if (header_type >=3D MIG_RP_MSG_MAX || header_type =3D=3D MIG_RP_MSG_INVALID) { error_report("RP: Received invalid message 0x%04x length 0x%04= x", diff --git a/migration/ram.c b/migration/ram.c index c30db15d4c..85cb965a14 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -2622,7 +2622,7 @@ static int ram_load_postcopy(QEMUFile *f) void *last_host =3D NULL; bool all_zero =3D false; =20 - while (!ret && !(flags & RAM_SAVE_FLAG_EOS)) { + while (!(flags & RAM_SAVE_FLAG_EOS)) { ram_addr_t addr; void *host =3D NULL; void *page_buffer =3D NULL; @@ -2631,6 +2631,16 @@ static int ram_load_postcopy(QEMUFile *f) uint8_t ch; =20 addr =3D qemu_get_be64(f); + + /* + * If qemu file error, we should stop here, and then "addr" + * may be invalid + */ + ret =3D qemu_file_get_error(f); + if (ret) { + break; + } + flags =3D addr & ~TARGET_PAGE_MASK; addr &=3D TARGET_PAGE_MASK; =20 @@ -2711,6 +2721,13 @@ static int ram_load_postcopy(QEMUFile *f) error_report("Unknown combination of migration flags: %#x" " (postcopy mode)", flags); ret =3D -EINVAL; + break; + } + + /* Detect for any possible file errors */ + if (qemu_file_get_error(f)) { + ret =3D qemu_file_get_error(f); + break; } =20 if (place_needed) { @@ -2724,9 +2741,10 @@ static int ram_load_postcopy(QEMUFile *f) ret =3D postcopy_place_page(mis, place_dest, place_source, block); } - } - if (!ret) { - ret =3D qemu_file_get_error(f); + + if (ret) { + break; + } } } =20 diff --git a/migration/savevm.c b/migration/savevm.c index 4a88228614..1da0255cd7 100644 --- a/migration/savevm.c +++ b/migration/savevm.c @@ -1765,6 +1765,11 @@ static int loadvm_process_command(QEMUFile *f) cmd =3D qemu_get_be16(f); len =3D qemu_get_be16(f); =20 + /* Check validity before continue processing of cmds */ + if (qemu_file_get_error(f)) { + return qemu_file_get_error(f); + } + trace_loadvm_process_command(cmd, len); if (cmd >=3D MIG_CMD_MAX || cmd =3D=3D MIG_CMD_INVALID) { error_report("MIG_CMD 0x%x unknown (len 0x%x)", cmd, len); @@ -1830,6 +1835,7 @@ static int loadvm_process_command(QEMUFile *f) */ static bool check_section_footer(QEMUFile *f, SaveStateEntry *se) { + int ret; uint8_t read_mark; uint32_t read_section_id; =20 @@ -1840,6 +1846,13 @@ static bool check_section_footer(QEMUFile *f, SaveSt= ateEntry *se) =20 read_mark =3D qemu_get_byte(f); =20 + ret =3D qemu_file_get_error(f); + if (ret) { + error_report("%s: Read section footer failed: %d", + __func__, ret); + return false; + } + if (read_mark !=3D QEMU_VM_SECTION_FOOTER) { error_report("Missing section footer for %s", se->idstr); return false; @@ -1875,6 +1888,13 @@ qemu_loadvm_section_start_full(QEMUFile *f, Migratio= nIncomingState *mis) instance_id =3D qemu_get_be32(f); version_id =3D qemu_get_be32(f); =20 + ret =3D qemu_file_get_error(f); + if (ret) { + error_report("%s: Failed to read instance/version ID: %d", + __func__, ret); + return ret; + } + trace_qemu_loadvm_state_section_startfull(section_id, idstr, instance_id, version_id); /* Find savevm section */ @@ -1922,6 +1942,13 @@ qemu_loadvm_section_part_end(QEMUFile *f, MigrationI= ncomingState *mis) =20 section_id =3D qemu_get_be32(f); =20 + ret =3D qemu_file_get_error(f); + if (ret) { + error_report("%s: Failed to read section ID: %d", + __func__, ret); + return ret; + } + trace_qemu_loadvm_state_section_partend(section_id); QTAILQ_FOREACH(se, &savevm_state.handlers, entry) { if (se->load_section_id =3D=3D section_id) { @@ -1989,8 +2016,14 @@ static int qemu_loadvm_state_main(QEMUFile *f, Migra= tionIncomingState *mis) uint8_t section_type; int ret =3D 0; =20 - while ((section_type =3D qemu_get_byte(f)) !=3D QEMU_VM_EOF) { - ret =3D 0; + while (true) { + section_type =3D qemu_get_byte(f); + + if (qemu_file_get_error(f)) { + ret =3D qemu_file_get_error(f); + break; + } + trace_qemu_loadvm_state_section(section_type); switch (section_type) { case QEMU_VM_SECTION_START: @@ -2014,6 +2047,9 @@ static int qemu_loadvm_state_main(QEMUFile *f, Migrat= ionIncomingState *mis) goto out; } break; + case QEMU_VM_EOF: + /* This is the end of migration */ + goto out; default: error_report("Unknown savevm section type %d", section_type); ret =3D -EINVAL; --=20 2.13.5 From nobody Sun Apr 28 04:06:08 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1508136870275667.3353466287874; Sun, 15 Oct 2017 23:54:30 -0700 (PDT) Received: from localhost ([::1]:59607 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1e3zIC-0001Wb-PM for importer@patchew.org; Mon, 16 Oct 2017 02:54:28 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:42245) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1e3zGM-0000Lt-EH for qemu-devel@nongnu.org; Mon, 16 Oct 2017 02:52:35 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1e3zGL-00016L-Eo for qemu-devel@nongnu.org; Mon, 16 Oct 2017 02:52:34 -0400 Received: from mx1.redhat.com ([209.132.183.28]:59388) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1e3zGL-00015Y-5m for qemu-devel@nongnu.org; Mon, 16 Oct 2017 02:52:33 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 2620680461; Mon, 16 Oct 2017 06:52:32 +0000 (UTC) Received: from pxdev.xzpeter.org.com (dhcp-15-225.nay.redhat.com [10.66.15.225]) by smtp.corp.redhat.com (Postfix) with ESMTP id E73985EE07; Mon, 16 Oct 2017 06:52:29 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com 2620680461 Authentication-Results: ext-mx04.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx04.extmail.prod.ext.phx2.redhat.com; spf=fail smtp.mailfrom=peterx@redhat.com From: Peter Xu To: qemu-devel@nongnu.org Date: Mon, 16 Oct 2017 14:51:46 +0800 Message-Id: <20171016065216.18162-3-peterx@redhat.com> In-Reply-To: <20171016065216.18162-1-peterx@redhat.com> References: <20171016065216.18162-1-peterx@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.28]); Mon, 16 Oct 2017 06:52:32 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH v3 02/32] migration: reuse mis->userfault_quit_fd X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Andrea Arcangeli , Juan Quintela , Alexey Perevalov , peterx@redhat.com, "Dr . David Alan Gilbert" Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" It was only used for quitting the page fault thread before. Let it be something more useful - now we can use it to notify a "wake" for the page fault thread (for any reason), and it only means "quit" if the fault_thread_quit is set. Since we changed what it does, renaming it to userfault_event_fd. Reviewed-by: Dr. David Alan Gilbert Signed-off-by: Peter Xu --- migration/migration.h | 6 ++++-- migration/postcopy-ram.c | 26 +++++++++++++++++--------- 2 files changed, 21 insertions(+), 11 deletions(-) diff --git a/migration/migration.h b/migration/migration.h index b83cceadc4..4642469d27 100644 --- a/migration/migration.h +++ b/migration/migration.h @@ -36,6 +36,8 @@ struct MigrationIncomingState { bool have_fault_thread; QemuThread fault_thread; QemuSemaphore fault_thread_sem; + /* Set this when we want the fault thread to quit */ + bool fault_thread_quit; =20 bool have_listen_thread; QemuThread listen_thread; @@ -43,8 +45,8 @@ struct MigrationIncomingState { =20 /* For the kernel to send us notifications */ int userfault_fd; - /* To tell the fault_thread to quit */ - int userfault_quit_fd; + /* To notify the fault_thread to wake, e.g., when need to quit */ + int userfault_event_fd; QEMUFile *to_src_file; QemuMutex rp_mutex; /* We send replies from multiple threads */ void *postcopy_tmp_page; diff --git a/migration/postcopy-ram.c b/migration/postcopy-ram.c index bec6c2c66b..07dd29c9d7 100644 --- a/migration/postcopy-ram.c +++ b/migration/postcopy-ram.c @@ -387,17 +387,18 @@ int postcopy_ram_incoming_cleanup(MigrationIncomingSt= ate *mis) * currently be at 0, we're going to increment it to 1 */ tmp64 =3D 1; - if (write(mis->userfault_quit_fd, &tmp64, 8) =3D=3D 8) { + atomic_set(&mis->fault_thread_quit, 1); + if (write(mis->userfault_event_fd, &tmp64, 8) =3D=3D 8) { trace_postcopy_ram_incoming_cleanup_join(); qemu_thread_join(&mis->fault_thread); } else { /* Not much we can do here, but may as well report it */ - error_report("%s: incrementing userfault_quit_fd: %s", __func_= _, + error_report("%s: incrementing userfault_event_fd: %s", __func= __, strerror(errno)); } trace_postcopy_ram_incoming_cleanup_closeuf(); close(mis->userfault_fd); - close(mis->userfault_quit_fd); + close(mis->userfault_event_fd); mis->have_fault_thread =3D false; } =20 @@ -520,7 +521,7 @@ static void *postcopy_ram_fault_thread(void *opaque) pfd[0].fd =3D mis->userfault_fd; pfd[0].events =3D POLLIN; pfd[0].revents =3D 0; - pfd[1].fd =3D mis->userfault_quit_fd; + pfd[1].fd =3D mis->userfault_event_fd; pfd[1].events =3D POLLIN; /* Waiting for eventfd to go positive */ pfd[1].revents =3D 0; =20 @@ -530,8 +531,15 @@ static void *postcopy_ram_fault_thread(void *opaque) } =20 if (pfd[1].revents) { - trace_postcopy_ram_fault_thread_quit(); - break; + uint64_t tmp64 =3D 0; + + /* Consume the signal */ + read(mis->userfault_event_fd, &tmp64, 8); + + if (atomic_read(&mis->fault_thread_quit)) { + trace_postcopy_ram_fault_thread_quit(); + break; + } } =20 ret =3D read(mis->userfault_fd, &msg, sizeof(msg)); @@ -610,9 +618,9 @@ int postcopy_ram_enable_notify(MigrationIncomingState *= mis) } =20 /* Now an eventfd we use to tell the fault-thread to quit */ - mis->userfault_quit_fd =3D eventfd(0, EFD_CLOEXEC); - if (mis->userfault_quit_fd =3D=3D -1) { - error_report("%s: Opening userfault_quit_fd: %s", __func__, + mis->userfault_event_fd =3D eventfd(0, EFD_CLOEXEC); + if (mis->userfault_event_fd =3D=3D -1) { + error_report("%s: Opening userfault_event_fd: %s", __func__, strerror(errno)); close(mis->userfault_fd); return -1; --=20 2.13.5 From nobody Sun Apr 28 04:06:08 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1508136876367825.568322489762; Sun, 15 Oct 2017 23:54:36 -0700 (PDT) Received: from localhost ([::1]:59608 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1e3zIC-0001XL-JE for importer@patchew.org; Mon, 16 Oct 2017 02:54:28 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:42256) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1e3zGP-0000Ns-4x for qemu-devel@nongnu.org; Mon, 16 Oct 2017 02:52:37 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1e3zGO-00019F-5I for qemu-devel@nongnu.org; Mon, 16 Oct 2017 02:52:37 -0400 Received: from mx1.redhat.com ([209.132.183.28]:36932) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1e3zGN-00018e-SP for qemu-devel@nongnu.org; Mon, 16 Oct 2017 02:52:36 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id D686181DEB; Mon, 16 Oct 2017 06:52:34 +0000 (UTC) Received: from pxdev.xzpeter.org.com (dhcp-15-225.nay.redhat.com [10.66.15.225]) by smtp.corp.redhat.com (Postfix) with ESMTP id 94FA962923; Mon, 16 Oct 2017 06:52:32 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com D686181DEB Authentication-Results: ext-mx01.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx01.extmail.prod.ext.phx2.redhat.com; spf=fail smtp.mailfrom=peterx@redhat.com From: Peter Xu To: qemu-devel@nongnu.org Date: Mon, 16 Oct 2017 14:51:47 +0800 Message-Id: <20171016065216.18162-4-peterx@redhat.com> In-Reply-To: <20171016065216.18162-1-peterx@redhat.com> References: <20171016065216.18162-1-peterx@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.25]); Mon, 16 Oct 2017 06:52:35 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH v3 03/32] migration: provide postcopy_fault_thread_notify() X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Andrea Arcangeli , Juan Quintela , Alexey Perevalov , peterx@redhat.com, "Dr . David Alan Gilbert" Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" A general helper to notify the fault thread. Reviewed-by: Dr. David Alan Gilbert Signed-off-by: Peter Xu --- migration/postcopy-ram.c | 35 ++++++++++++++++++++--------------- migration/postcopy-ram.h | 2 ++ 2 files changed, 22 insertions(+), 15 deletions(-) diff --git a/migration/postcopy-ram.c b/migration/postcopy-ram.c index 07dd29c9d7..e770077a81 100644 --- a/migration/postcopy-ram.c +++ b/migration/postcopy-ram.c @@ -369,6 +369,21 @@ int postcopy_ram_incoming_init(MigrationIncomingState = *mis, size_t ram_pages) return 0; } =20 +void postcopy_fault_thread_notify(MigrationIncomingState *mis) +{ + uint64_t tmp64 =3D 1; + + /* + * Wakeup the fault_thread. It's an eventfd that should currently + * be at 0, we're going to increment it to 1 + */ + if (write(mis->userfault_event_fd, &tmp64, 8) !=3D 8) { + /* Not much we can do here, but may as well report it */ + error_report("%s: incrementing failed: %s", __func__, + strerror(errno)); + } +} + /* * At the end of a migration where postcopy_ram_incoming_init was called. */ @@ -377,25 +392,15 @@ int postcopy_ram_incoming_cleanup(MigrationIncomingSt= ate *mis) trace_postcopy_ram_incoming_cleanup_entry(); =20 if (mis->have_fault_thread) { - uint64_t tmp64; - if (qemu_ram_foreach_block(cleanup_range, mis)) { return -1; } - /* - * Tell the fault_thread to exit, it's an eventfd that should - * currently be at 0, we're going to increment it to 1 - */ - tmp64 =3D 1; + /* Let the fault thread quit */ atomic_set(&mis->fault_thread_quit, 1); - if (write(mis->userfault_event_fd, &tmp64, 8) =3D=3D 8) { - trace_postcopy_ram_incoming_cleanup_join(); - qemu_thread_join(&mis->fault_thread); - } else { - /* Not much we can do here, but may as well report it */ - error_report("%s: incrementing userfault_event_fd: %s", __func= __, - strerror(errno)); - } + postcopy_fault_thread_notify(mis); + trace_postcopy_ram_incoming_cleanup_join(); + qemu_thread_join(&mis->fault_thread); + trace_postcopy_ram_incoming_cleanup_closeuf(); close(mis->userfault_fd); close(mis->userfault_event_fd); diff --git a/migration/postcopy-ram.h b/migration/postcopy-ram.h index 77ea0fd264..14f6cadcbd 100644 --- a/migration/postcopy-ram.h +++ b/migration/postcopy-ram.h @@ -114,4 +114,6 @@ PostcopyState postcopy_state_get(void); /* Set the state and return the old state */ PostcopyState postcopy_state_set(PostcopyState new_state); =20 +void postcopy_fault_thread_notify(MigrationIncomingState *mis); + #endif --=20 2.13.5 From nobody Sun Apr 28 04:06:08 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1508136882500230.86956108691084; Sun, 15 Oct 2017 23:54:42 -0700 (PDT) Received: from localhost ([::1]:59609 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1e3zIP-0001iI-8L for importer@patchew.org; Mon, 16 Oct 2017 02:54:41 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:42267) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1e3zGW-0000U3-Fp for qemu-devel@nongnu.org; Mon, 16 Oct 2017 02:52:45 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1e3zGT-0001Dz-EC for qemu-devel@nongnu.org; Mon, 16 Oct 2017 02:52:44 -0400 Received: from mx1.redhat.com ([209.132.183.28]:40396) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1e3zGT-0001D9-8S for qemu-devel@nongnu.org; Mon, 16 Oct 2017 02:52:41 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 249CC2DA; Mon, 16 Oct 2017 06:52:40 +0000 (UTC) Received: from pxdev.xzpeter.org.com (dhcp-15-225.nay.redhat.com [10.66.15.225]) by smtp.corp.redhat.com (Postfix) with ESMTP id 4C34B5EE02; Mon, 16 Oct 2017 06:52:35 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com 249CC2DA Authentication-Results: ext-mx05.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx05.extmail.prod.ext.phx2.redhat.com; spf=fail smtp.mailfrom=peterx@redhat.com From: Peter Xu To: qemu-devel@nongnu.org Date: Mon, 16 Oct 2017 14:51:48 +0800 Message-Id: <20171016065216.18162-5-peterx@redhat.com> In-Reply-To: <20171016065216.18162-1-peterx@redhat.com> References: <20171016065216.18162-1-peterx@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.29]); Mon, 16 Oct 2017 06:52:40 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH v3 04/32] migration: new postcopy-pause state X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Andrea Arcangeli , Juan Quintela , Alexey Perevalov , peterx@redhat.com, "Dr . David Alan Gilbert" Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Introducing a new state "postcopy-paused", which can be used when the postcopy migration is paused. It is targeted for postcopy network failure recovery. Reviewed-by: Dr. David Alan Gilbert Signed-off-by: Peter Xu --- migration/migration.c | 2 ++ qapi/migration.json | 5 ++++- 2 files changed, 6 insertions(+), 1 deletion(-) diff --git a/migration/migration.c b/migration/migration.c index 3366bdddcc..22142c8194 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -525,6 +525,7 @@ static bool migration_is_setup_or_active(int state) switch (state) { case MIGRATION_STATUS_ACTIVE: case MIGRATION_STATUS_POSTCOPY_ACTIVE: + case MIGRATION_STATUS_POSTCOPY_PAUSED: case MIGRATION_STATUS_SETUP: return true; =20 @@ -600,6 +601,7 @@ MigrationInfo *qmp_query_migrate(Error **errp) case MIGRATION_STATUS_ACTIVE: case MIGRATION_STATUS_CANCELLING: case MIGRATION_STATUS_POSTCOPY_ACTIVE: + case MIGRATION_STATUS_POSTCOPY_PAUSED: /* TODO add some postcopy stats */ info->has_status =3D true; info->has_total_time =3D true; diff --git a/qapi/migration.json b/qapi/migration.json index f8b365e3f5..d5fd937c26 100644 --- a/qapi/migration.json +++ b/qapi/migration.json @@ -89,6 +89,8 @@ # # @postcopy-active: like active, but now in postcopy mode. (since 2.5) # +# @postcopy-paused: during postcopy but paused. (since 2.11) +# # @completed: migration is finished. # # @failed: some error occurred during migration process. @@ -101,7 +103,8 @@ ## { 'enum': 'MigrationStatus', 'data': [ 'none', 'setup', 'cancelling', 'cancelled', - 'active', 'postcopy-active', 'completed', 'failed', 'colo' ] } + 'active', 'postcopy-active', 'postcopy-paused', + 'completed', 'failed', 'colo' ] } =20 ## # @MigrationInfo: --=20 2.13.5 From nobody Sun Apr 28 04:06:08 2024 Delivered-To: importer@patchew.org Received-SPF: temperror (zoho.com: Error in retrieving data from DNS) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=temperror (zoho.com: Error in retrieving data from DNS) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1508137040022498.3225099239803; Sun, 15 Oct 2017 23:57:20 -0700 (PDT) Received: from localhost ([::1]:59623 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1e3zKw-00042E-Md for importer@patchew.org; Mon, 16 Oct 2017 02:57:18 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:42303) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1e3zGd-0000a2-2T for qemu-devel@nongnu.org; Mon, 16 Oct 2017 02:52:55 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1e3zGb-0001Ly-Sw for qemu-devel@nongnu.org; Mon, 16 Oct 2017 02:52:51 -0400 Received: from mx1.redhat.com ([209.132.183.28]:51678) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1e3zGb-0001LE-K3 for qemu-devel@nongnu.org; Mon, 16 Oct 2017 02:52:49 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 91523859FA; Mon, 16 Oct 2017 06:52:48 +0000 (UTC) Received: from pxdev.xzpeter.org.com (dhcp-15-225.nay.redhat.com [10.66.15.225]) by smtp.corp.redhat.com (Postfix) with ESMTP id 929045EE02; Mon, 16 Oct 2017 06:52:40 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com 91523859FA Authentication-Results: ext-mx02.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx02.extmail.prod.ext.phx2.redhat.com; spf=fail smtp.mailfrom=peterx@redhat.com From: Peter Xu To: qemu-devel@nongnu.org Date: Mon, 16 Oct 2017 14:51:49 +0800 Message-Id: <20171016065216.18162-6-peterx@redhat.com> In-Reply-To: <20171016065216.18162-1-peterx@redhat.com> References: <20171016065216.18162-1-peterx@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.26]); Mon, 16 Oct 2017 06:52:48 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH v3 05/32] migration: implement "postcopy-pause" src logic X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Andrea Arcangeli , Juan Quintela , Alexey Perevalov , peterx@redhat.com, "Dr . David Alan Gilbert" Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_6 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Now when network down for postcopy, the source side will not fail the migration. Instead we convert the status into this new paused state, and we will try to wait for a rescue in the future. If a recovery is detected, migration_thread() will reset its local variables to prepare for that. Reviewed-by: Dr. David Alan Gilbert Signed-off-by: Peter Xu --- migration/migration.c | 98 ++++++++++++++++++++++++++++++++++++++++++++++= +--- migration/migration.h | 3 ++ migration/trace-events | 1 + 3 files changed, 98 insertions(+), 4 deletions(-) diff --git a/migration/migration.c b/migration/migration.c index 22142c8194..88c5188988 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -1075,6 +1075,8 @@ static void migrate_fd_cleanup(void *opaque) =20 notifier_list_notify(&migration_state_notifiers, s); block_cleanup_parameters(s); + + qemu_sem_destroy(&s->postcopy_pause_sem); } =20 void migrate_fd_error(MigrationState *s, const Error *error) @@ -1218,6 +1220,7 @@ MigrationState *migrate_init(void) s->migration_thread_running =3D false; error_free(s->error); s->error =3D NULL; + qemu_sem_init(&s->postcopy_pause_sem, 0); =20 migrate_set_state(&s->state, MIGRATION_STATUS_NONE, MIGRATION_STATUS_S= ETUP); =20 @@ -2060,6 +2063,80 @@ bool migrate_colo_enabled(void) return s->enabled_capabilities[MIGRATION_CAPABILITY_X_COLO]; } =20 +typedef enum MigThrError { + /* No error detected */ + MIG_THR_ERR_NONE =3D 0, + /* Detected error, but resumed successfully */ + MIG_THR_ERR_RECOVERED =3D 1, + /* Detected fatal error, need to exit */ + MIG_THR_ERR_FATAL =3D 2, +} MigThrError; + +/* + * We don't return until we are in a safe state to continue current + * postcopy migration. Returns MIG_THR_ERR_RECOVERED if recovered, or + * MIG_THR_ERR_FATAL if unrecovery failure happened. + */ +static MigThrError postcopy_pause(MigrationState *s) +{ + assert(s->state =3D=3D MIGRATION_STATUS_POSTCOPY_ACTIVE); + migrate_set_state(&s->state, MIGRATION_STATUS_POSTCOPY_ACTIVE, + MIGRATION_STATUS_POSTCOPY_PAUSED); + + /* Current channel is possibly broken. Release it. */ + assert(s->to_dst_file); + qemu_file_shutdown(s->to_dst_file); + qemu_fclose(s->to_dst_file); + s->to_dst_file =3D NULL; + + error_report("Detected IO failure for postcopy. " + "Migration paused."); + + /* + * We wait until things fixed up. Then someone will setup the + * status back for us. + */ + while (s->state =3D=3D MIGRATION_STATUS_POSTCOPY_PAUSED) { + qemu_sem_wait(&s->postcopy_pause_sem); + } + + trace_postcopy_pause_continued(); + + return MIG_THR_ERR_RECOVERED; +} + +static MigThrError migration_detect_error(MigrationState *s) +{ + int ret; + + /* Try to detect any file errors */ + ret =3D qemu_file_get_error(s->to_dst_file); + + if (!ret) { + /* Everything is fine */ + return MIG_THR_ERR_NONE; + } + + if (s->state =3D=3D MIGRATION_STATUS_POSTCOPY_ACTIVE && ret =3D=3D -EI= O) { + /* + * For postcopy, we allow the network to be down for a + * while. After that, it can be continued by a + * recovery phase. + */ + return postcopy_pause(s); + } else { + /* + * For precopy (or postcopy with error outside IO), we fail + * with no time. + */ + migrate_set_state(&s->state, s->state, MIGRATION_STATUS_FAILED); + trace_migration_thread_file_err(); + + /* Time to stop the migration, now. */ + return MIG_THR_ERR_FATAL; + } +} + /* * Master migration thread on the source VM. * It drives the migration and pumps the data down the outgoing channel. @@ -2084,6 +2161,7 @@ static void *migration_thread(void *opaque) /* The active state we expect to be in; ACTIVE or POSTCOPY_ACTIVE */ enum MigrationStatus current_active_state =3D MIGRATION_STATUS_ACTIVE; bool enable_colo =3D migrate_colo_enabled(); + MigThrError thr_error; =20 rcu_register_thread(); =20 @@ -2156,12 +2234,24 @@ static void *migration_thread(void *opaque) } } =20 - if (qemu_file_get_error(s->to_dst_file)) { - migrate_set_state(&s->state, current_active_state, - MIGRATION_STATUS_FAILED); - trace_migration_thread_file_err(); + /* + * Try to detect any kind of failures, and see whether we + * should stop the migration now. + */ + thr_error =3D migration_detect_error(s); + if (thr_error =3D=3D MIG_THR_ERR_FATAL) { + /* Stop migration */ break; + } else if (thr_error =3D=3D MIG_THR_ERR_RECOVERED) { + /* + * Just recovered from a e.g. network failure, reset all + * the local variables. This is important to avoid + * breaking transferred_bytes and bandwidth calculation + */ + initial_time =3D qemu_clock_get_ms(QEMU_CLOCK_REALTIME); + initial_bytes =3D 0; } + current_time =3D qemu_clock_get_ms(QEMU_CLOCK_REALTIME); if (current_time >=3D initial_time + BUFFER_DELAY) { uint64_t transferred_bytes =3D qemu_ftell(s->to_dst_file) - diff --git a/migration/migration.h b/migration/migration.h index 4642469d27..065268432d 100644 --- a/migration/migration.h +++ b/migration/migration.h @@ -150,6 +150,9 @@ struct MigrationState bool send_configuration; /* Whether we send section footer during migration */ bool send_section_footer; + + /* Needed by postcopy-pause state */ + QemuSemaphore postcopy_pause_sem; }; =20 void migrate_set_state(int *state, int old_state, int new_state); diff --git a/migration/trace-events b/migration/trace-events index 6f29fcc686..da1c63a933 100644 --- a/migration/trace-events +++ b/migration/trace-events @@ -99,6 +99,7 @@ migration_thread_setup_complete(void) "" open_return_path_on_source(void) "" open_return_path_on_source_continue(void) "" postcopy_start(void) "" +postcopy_pause_continued(void) "" postcopy_start_set_run(void) "" source_return_path_thread_bad_end(void) "" source_return_path_thread_end(void) "" --=20 2.13.5 From nobody Sun Apr 28 04:06:08 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 150813703569039.31936487672522; Sun, 15 Oct 2017 23:57:15 -0700 (PDT) Received: from localhost ([::1]:59622 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1e3zKl-0003sK-Kc for importer@patchew.org; Mon, 16 Oct 2017 02:57:07 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:42314) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1e3zGh-0000dU-0O for qemu-devel@nongnu.org; Mon, 16 Oct 2017 02:52:56 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1e3zGf-0001Oz-Qe for qemu-devel@nongnu.org; Mon, 16 Oct 2017 02:52:55 -0400 Received: from mx1.redhat.com ([209.132.183.28]:37222) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1e3zGf-0001OJ-Hy for qemu-devel@nongnu.org; Mon, 16 Oct 2017 02:52:53 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 8716681DE1; Mon, 16 Oct 2017 06:52:52 +0000 (UTC) Received: from pxdev.xzpeter.org.com (dhcp-15-225.nay.redhat.com [10.66.15.225]) by smtp.corp.redhat.com (Postfix) with ESMTP id 0B77862923; Mon, 16 Oct 2017 06:52:48 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com 8716681DE1 Authentication-Results: ext-mx01.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx01.extmail.prod.ext.phx2.redhat.com; spf=fail smtp.mailfrom=peterx@redhat.com From: Peter Xu To: qemu-devel@nongnu.org Date: Mon, 16 Oct 2017 14:51:50 +0800 Message-Id: <20171016065216.18162-7-peterx@redhat.com> In-Reply-To: <20171016065216.18162-1-peterx@redhat.com> References: <20171016065216.18162-1-peterx@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.25]); Mon, 16 Oct 2017 06:52:52 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH v3 06/32] migration: allow dst vm pause on postcopy X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Andrea Arcangeli , Juan Quintela , Alexey Perevalov , peterx@redhat.com, "Dr . David Alan Gilbert" Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" When there is IO error on the incoming channel (e.g., network down), instead of bailing out immediately, we allow the dst vm to switch to the new POSTCOPY_PAUSE state. Currently it is still simple - it waits the new semaphore, until someone poke it for another attempt. Signed-off-by: Peter Xu --- migration/migration.c | 1 + migration/migration.h | 3 +++ migration/savevm.c | 60 ++++++++++++++++++++++++++++++++++++++++++++++= ++-- migration/trace-events | 2 ++ 4 files changed, 64 insertions(+), 2 deletions(-) diff --git a/migration/migration.c b/migration/migration.c index 88c5188988..3cc6d4933d 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -147,6 +147,7 @@ MigrationIncomingState *migration_incoming_get_current(= void) memset(&mis_current, 0, sizeof(MigrationIncomingState)); qemu_mutex_init(&mis_current.rp_mutex); qemu_event_init(&mis_current.main_thread_load_event, false); + qemu_sem_init(&mis_current.postcopy_pause_sem_dst, 0); once =3D true; } return &mis_current; diff --git a/migration/migration.h b/migration/migration.h index 065268432d..1280060d54 100644 --- a/migration/migration.h +++ b/migration/migration.h @@ -61,6 +61,9 @@ struct MigrationIncomingState { /* The coroutine we should enter (back) after failover */ Coroutine *migration_incoming_co; QemuSemaphore colo_incoming_sem; + + /* notify PAUSED postcopy incoming migrations to try to continue */ + QemuSemaphore postcopy_pause_sem_dst; }; =20 MigrationIncomingState *migration_incoming_get_current(void); diff --git a/migration/savevm.c b/migration/savevm.c index 1da0255cd7..93e308ebf0 100644 --- a/migration/savevm.c +++ b/migration/savevm.c @@ -1529,8 +1529,8 @@ static int loadvm_postcopy_ram_handle_discard(Migrati= onIncomingState *mis, */ static void *postcopy_ram_listen_thread(void *opaque) { - QEMUFile *f =3D opaque; MigrationIncomingState *mis =3D migration_incoming_get_current(); + QEMUFile *f =3D mis->from_src_file; int load_res; =20 migrate_set_state(&mis->state, MIGRATION_STATUS_ACTIVE, @@ -1544,6 +1544,14 @@ static void *postcopy_ram_listen_thread(void *opaque) */ qemu_file_set_blocking(f, true); load_res =3D qemu_loadvm_state_main(f, mis); + + /* + * This is tricky, but, mis->from_src_file can change after it + * returns, when postcopy recovery happened. In the future, we may + * want a wrapper for the QEMUFile handle. + */ + f =3D mis->from_src_file; + /* And non-blocking again so we don't block in any cleanup */ qemu_file_set_blocking(f, false); =20 @@ -1626,7 +1634,7 @@ static int loadvm_postcopy_handle_listen(MigrationInc= omingState *mis) /* Start up the listening thread and wait for it to signal ready */ qemu_sem_init(&mis->listen_thread_sem, 0); qemu_thread_create(&mis->listen_thread, "postcopy/listen", - postcopy_ram_listen_thread, mis->from_src_file, + postcopy_ram_listen_thread, NULL, QEMU_THREAD_DETACHED); qemu_sem_wait(&mis->listen_thread_sem); qemu_sem_destroy(&mis->listen_thread_sem); @@ -2011,11 +2019,44 @@ void qemu_loadvm_state_cleanup(void) } } =20 +/* Return true if we should continue the migration, or false. */ +static bool postcopy_pause_incoming(MigrationIncomingState *mis) +{ + trace_postcopy_pause_incoming(); + + migrate_set_state(&mis->state, MIGRATION_STATUS_POSTCOPY_ACTIVE, + MIGRATION_STATUS_POSTCOPY_PAUSED); + + assert(mis->from_src_file); + qemu_file_shutdown(mis->from_src_file); + qemu_fclose(mis->from_src_file); + mis->from_src_file =3D NULL; + + assert(mis->to_src_file); + qemu_file_shutdown(mis->to_src_file); + qemu_mutex_lock(&mis->rp_mutex); + qemu_fclose(mis->to_src_file); + mis->to_src_file =3D NULL; + qemu_mutex_unlock(&mis->rp_mutex); + + error_report("Detected IO failure for postcopy. " + "Migration paused."); + + while (mis->state =3D=3D MIGRATION_STATUS_POSTCOPY_PAUSED) { + qemu_sem_wait(&mis->postcopy_pause_sem_dst); + } + + trace_postcopy_pause_incoming_continued(); + + return true; +} + static int qemu_loadvm_state_main(QEMUFile *f, MigrationIncomingState *mis) { uint8_t section_type; int ret =3D 0; =20 +retry: while (true) { section_type =3D qemu_get_byte(f); =20 @@ -2060,6 +2101,21 @@ static int qemu_loadvm_state_main(QEMUFile *f, Migra= tionIncomingState *mis) out: if (ret < 0) { qemu_file_set_error(f, ret); + + /* + * Detect whether it is: + * + * 1. postcopy running + * 2. network failure (-EIO) + * + * If so, we try to wait for a recovery. + */ + if (mis->state =3D=3D MIGRATION_STATUS_POSTCOPY_ACTIVE && + ret =3D=3D -EIO && postcopy_pause_incoming(mis)) { + /* Reset f to point to the newly created channel */ + f =3D mis->from_src_file; + goto retry; + } } return ret; } diff --git a/migration/trace-events b/migration/trace-events index da1c63a933..bed1646cd6 100644 --- a/migration/trace-events +++ b/migration/trace-events @@ -100,6 +100,8 @@ open_return_path_on_source(void) "" open_return_path_on_source_continue(void) "" postcopy_start(void) "" postcopy_pause_continued(void) "" +postcopy_pause_incoming(void) "" +postcopy_pause_incoming_continued(void) "" postcopy_start_set_run(void) "" source_return_path_thread_bad_end(void) "" source_return_path_thread_end(void) "" --=20 2.13.5 From nobody Sun Apr 28 04:06:08 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1508137183370805.5301927955614; Sun, 15 Oct 2017 23:59:43 -0700 (PDT) Received: from localhost ([::1]:59630 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1e3zNC-0006BC-Py for importer@patchew.org; Mon, 16 Oct 2017 02:59:38 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:42330) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1e3zGj-0000fn-Fz for qemu-devel@nongnu.org; Mon, 16 Oct 2017 02:52:58 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1e3zGi-0001RJ-Ib for qemu-devel@nongnu.org; Mon, 16 Oct 2017 02:52:57 -0400 Received: from mx1.redhat.com ([209.132.183.28]:37290) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1e3zGi-0001QM-A7 for qemu-devel@nongnu.org; Mon, 16 Oct 2017 02:52:56 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 5A78381DE1; Mon, 16 Oct 2017 06:52:55 +0000 (UTC) Received: from pxdev.xzpeter.org.com (dhcp-15-225.nay.redhat.com [10.66.15.225]) by smtp.corp.redhat.com (Postfix) with ESMTP id 024E75EE07; Mon, 16 Oct 2017 06:52:52 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com 5A78381DE1 Authentication-Results: ext-mx01.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx01.extmail.prod.ext.phx2.redhat.com; spf=fail smtp.mailfrom=peterx@redhat.com From: Peter Xu To: qemu-devel@nongnu.org Date: Mon, 16 Oct 2017 14:51:51 +0800 Message-Id: <20171016065216.18162-8-peterx@redhat.com> In-Reply-To: <20171016065216.18162-1-peterx@redhat.com> References: <20171016065216.18162-1-peterx@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.25]); Mon, 16 Oct 2017 06:52:55 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH v3 07/32] migration: allow src return path to pause X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Andrea Arcangeli , Juan Quintela , Alexey Perevalov , peterx@redhat.com, "Dr . David Alan Gilbert" Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Let the thread pause for network issues. Reviewed-by: Dr. David Alan Gilbert Signed-off-by: Peter Xu --- migration/migration.c | 35 +++++++++++++++++++++++++++++++++-- migration/migration.h | 1 + migration/trace-events | 2 ++ 3 files changed, 36 insertions(+), 2 deletions(-) diff --git a/migration/migration.c b/migration/migration.c index 3cc6d4933d..17a8ec39ef 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -1078,6 +1078,7 @@ static void migrate_fd_cleanup(void *opaque) block_cleanup_parameters(s); =20 qemu_sem_destroy(&s->postcopy_pause_sem); + qemu_sem_destroy(&s->postcopy_pause_rp_sem); } =20 void migrate_fd_error(MigrationState *s, const Error *error) @@ -1222,6 +1223,7 @@ MigrationState *migrate_init(void) error_free(s->error); s->error =3D NULL; qemu_sem_init(&s->postcopy_pause_sem, 0); + qemu_sem_init(&s->postcopy_pause_rp_sem, 0); =20 migrate_set_state(&s->state, MIGRATION_STATUS_NONE, MIGRATION_STATUS_S= ETUP); =20 @@ -1641,6 +1643,18 @@ static void migrate_handle_rp_req_pages(MigrationSta= te *ms, const char* rbname, } } =20 +/* Return true to retry, false to quit */ +static bool postcopy_pause_return_path_thread(MigrationState *s) +{ + trace_postcopy_pause_return_path(); + + qemu_sem_wait(&s->postcopy_pause_rp_sem); + + trace_postcopy_pause_return_path_continued(); + + return true; +} + /* * Handles messages sent on the return path towards the source VM * @@ -1657,6 +1671,8 @@ static void *source_return_path_thread(void *opaque) int res; =20 trace_source_return_path_thread_entry(); + +retry: while (!ms->rp_state.error && !qemu_file_get_error(rp) && migration_is_setup_or_active(ms->state)) { trace_source_return_path_thread_loop_top(); @@ -1748,13 +1764,28 @@ static void *source_return_path_thread(void *opaque) break; } } - if (qemu_file_get_error(rp)) { + +out: + res =3D qemu_file_get_error(rp); + if (res) { + if (res =3D=3D -EIO) { + /* + * Maybe there is something we can do: it looks like a + * network down issue, and we pause for a recovery. + */ + if (postcopy_pause_return_path_thread(ms)) { + /* Reload rp, reset the rest */ + rp =3D ms->rp_state.from_dst_file; + ms->rp_state.error =3D false; + goto retry; + } + } + trace_source_return_path_thread_bad_end(); mark_source_rp_bad(ms); } =20 trace_source_return_path_thread_end(); -out: ms->rp_state.from_dst_file =3D NULL; qemu_fclose(rp); return NULL; diff --git a/migration/migration.h b/migration/migration.h index 1280060d54..6776405edb 100644 --- a/migration/migration.h +++ b/migration/migration.h @@ -156,6 +156,7 @@ struct MigrationState =20 /* Needed by postcopy-pause state */ QemuSemaphore postcopy_pause_sem; + QemuSemaphore postcopy_pause_rp_sem; }; =20 void migrate_set_state(int *state, int old_state, int new_state); diff --git a/migration/trace-events b/migration/trace-events index bed1646cd6..a4031cfe00 100644 --- a/migration/trace-events +++ b/migration/trace-events @@ -99,6 +99,8 @@ migration_thread_setup_complete(void) "" open_return_path_on_source(void) "" open_return_path_on_source_continue(void) "" postcopy_start(void) "" +postcopy_pause_return_path(void) "" +postcopy_pause_return_path_continued(void) "" postcopy_pause_continued(void) "" postcopy_pause_incoming(void) "" postcopy_pause_incoming_continued(void) "" --=20 2.13.5 From nobody Sun Apr 28 04:06:08 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 150813718124124.47620262330247; Sun, 15 Oct 2017 23:59:41 -0700 (PDT) Received: from localhost ([::1]:59628 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1e3zN6-00065G-7d for importer@patchew.org; Mon, 16 Oct 2017 02:59:32 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:42370) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1e3zGq-0000mT-Hl for qemu-devel@nongnu.org; Mon, 16 Oct 2017 02:53:05 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1e3zGn-0001Yh-HB for qemu-devel@nongnu.org; Mon, 16 Oct 2017 02:53:04 -0400 Received: from mx1.redhat.com ([209.132.183.28]:8993) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1e3zGn-0001XB-8D for qemu-devel@nongnu.org; Mon, 16 Oct 2017 02:53:01 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 419037E426; Mon, 16 Oct 2017 06:53:00 +0000 (UTC) Received: from pxdev.xzpeter.org.com (dhcp-15-225.nay.redhat.com [10.66.15.225]) by smtp.corp.redhat.com (Postfix) with ESMTP id C9B6E5EE02; Mon, 16 Oct 2017 06:52:55 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com 419037E426 Authentication-Results: ext-mx03.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx03.extmail.prod.ext.phx2.redhat.com; spf=fail smtp.mailfrom=peterx@redhat.com From: Peter Xu To: qemu-devel@nongnu.org Date: Mon, 16 Oct 2017 14:51:52 +0800 Message-Id: <20171016065216.18162-9-peterx@redhat.com> In-Reply-To: <20171016065216.18162-1-peterx@redhat.com> References: <20171016065216.18162-1-peterx@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.27]); Mon, 16 Oct 2017 06:53:00 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH v3 08/32] migration: allow send_rq to fail X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Andrea Arcangeli , Juan Quintela , Alexey Perevalov , peterx@redhat.com, "Dr . David Alan Gilbert" Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" We will not allow failures to happen when sending data from destination to source via the return path. However it is possible that there can be errors along the way. This patch allows the migrate_send_rp_message() to return error when it happens, and further extended it to migrate_send_rp_req_pages(). Reviewed-by: Dr. David Alan Gilbert Signed-off-by: Peter Xu --- migration/migration.c | 38 ++++++++++++++++++++++++++++++-------- migration/migration.h | 2 +- 2 files changed, 31 insertions(+), 9 deletions(-) diff --git a/migration/migration.c b/migration/migration.c index 17a8ec39ef..033f9e3809 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -196,17 +196,35 @@ static void deferred_incoming_migration(Error **errp) * Send a message on the return channel back to the source * of the migration. */ -static void migrate_send_rp_message(MigrationIncomingState *mis, - enum mig_rp_message_type message_type, - uint16_t len, void *data) +static int migrate_send_rp_message(MigrationIncomingState *mis, + enum mig_rp_message_type message_type, + uint16_t len, void *data) { + int ret =3D 0; + trace_migrate_send_rp_message((int)message_type, len); qemu_mutex_lock(&mis->rp_mutex); + + /* + * It's possible that the file handle got lost due to network + * failures. + */ + if (!mis->to_src_file) { + ret =3D -EIO; + goto error; + } + qemu_put_be16(mis->to_src_file, (unsigned int)message_type); qemu_put_be16(mis->to_src_file, len); qemu_put_buffer(mis->to_src_file, data, len); qemu_fflush(mis->to_src_file); + + /* It's possible that qemu file got error during sending */ + ret =3D qemu_file_get_error(mis->to_src_file); + +error: qemu_mutex_unlock(&mis->rp_mutex); + return ret; } =20 /* Request a range of pages from the source VM at the given @@ -216,26 +234,30 @@ static void migrate_send_rp_message(MigrationIncoming= State *mis, * Start: Address offset within the RB * Len: Length in bytes required - must be a multiple of pagesize */ -void migrate_send_rp_req_pages(MigrationIncomingState *mis, const char *rb= name, - ram_addr_t start, size_t len) +int migrate_send_rp_req_pages(MigrationIncomingState *mis, const char *rbn= ame, + ram_addr_t start, size_t len) { uint8_t bufc[12 + 1 + 255]; /* start (8), len (4), rbname up to 256 */ size_t msglen =3D 12; /* start + len */ + int rbname_len; + enum mig_rp_message_type msg_type; =20 *(uint64_t *)bufc =3D cpu_to_be64((uint64_t)start); *(uint32_t *)(bufc + 8) =3D cpu_to_be32((uint32_t)len); =20 if (rbname) { - int rbname_len =3D strlen(rbname); + rbname_len =3D strlen(rbname); assert(rbname_len < 256); =20 bufc[msglen++] =3D rbname_len; memcpy(bufc + msglen, rbname, rbname_len); msglen +=3D rbname_len; - migrate_send_rp_message(mis, MIG_RP_MSG_REQ_PAGES_ID, msglen, bufc= ); + msg_type =3D MIG_RP_MSG_REQ_PAGES_ID; } else { - migrate_send_rp_message(mis, MIG_RP_MSG_REQ_PAGES, msglen, bufc); + msg_type =3D MIG_RP_MSG_REQ_PAGES; } + + return migrate_send_rp_message(mis, msg_type, msglen, bufc); } =20 void qemu_start_incoming_migration(const char *uri, Error **errp) diff --git a/migration/migration.h b/migration/migration.h index 6776405edb..2710627357 100644 --- a/migration/migration.h +++ b/migration/migration.h @@ -208,7 +208,7 @@ void migrate_send_rp_shut(MigrationIncomingState *mis, uint32_t value); void migrate_send_rp_pong(MigrationIncomingState *mis, uint32_t value); -void migrate_send_rp_req_pages(MigrationIncomingState *mis, const char* rb= name, +int migrate_send_rp_req_pages(MigrationIncomingState *mis, const char* rbn= ame, ram_addr_t start, size_t len); =20 #endif --=20 2.13.5 From nobody Sun Apr 28 04:06:08 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1508137189904657.9241462609386; Sun, 15 Oct 2017 23:59:49 -0700 (PDT) Received: from localhost ([::1]:59631 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1e3zNN-0006Jc-4I for importer@patchew.org; Mon, 16 Oct 2017 02:59:49 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:42383) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1e3zGw-0000rU-DS for qemu-devel@nongnu.org; Mon, 16 Oct 2017 02:53:13 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1e3zGv-0001l0-9y for qemu-devel@nongnu.org; Mon, 16 Oct 2017 02:53:10 -0400 Received: from mx1.redhat.com ([209.132.183.28]:52090) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1e3zGv-0001jl-0u for qemu-devel@nongnu.org; Mon, 16 Oct 2017 02:53:09 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 11EBF883BB; Mon, 16 Oct 2017 06:53:08 +0000 (UTC) Received: from pxdev.xzpeter.org.com (dhcp-15-225.nay.redhat.com [10.66.15.225]) by smtp.corp.redhat.com (Postfix) with ESMTP id B10B65EE02; Mon, 16 Oct 2017 06:53:00 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com 11EBF883BB Authentication-Results: ext-mx02.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx02.extmail.prod.ext.phx2.redhat.com; spf=fail smtp.mailfrom=peterx@redhat.com From: Peter Xu To: qemu-devel@nongnu.org Date: Mon, 16 Oct 2017 14:51:53 +0800 Message-Id: <20171016065216.18162-10-peterx@redhat.com> In-Reply-To: <20171016065216.18162-1-peterx@redhat.com> References: <20171016065216.18162-1-peterx@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.26]); Mon, 16 Oct 2017 06:53:08 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH v3 09/32] migration: allow fault thread to pause X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Andrea Arcangeli , Juan Quintela , Alexey Perevalov , peterx@redhat.com, "Dr . David Alan Gilbert" Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Allows the fault thread to stop handling page faults temporarily. When network failure happened (and if we expect a recovery afterwards), we should not allow the fault thread to continue sending things to source, instead, it should halt for a while until the connection is rebuilt. When the dest main thread noticed the failure, it kicks the fault thread to switch to pause state. Reviewed-by: Dr. David Alan Gilbert Signed-off-by: Peter Xu --- migration/migration.c | 1 + migration/migration.h | 1 + migration/postcopy-ram.c | 50 ++++++++++++++++++++++++++++++++++++++++++++= ---- migration/savevm.c | 3 +++ migration/trace-events | 2 ++ 5 files changed, 53 insertions(+), 4 deletions(-) diff --git a/migration/migration.c b/migration/migration.c index 033f9e3809..cebd75c5ce 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -148,6 +148,7 @@ MigrationIncomingState *migration_incoming_get_current(= void) qemu_mutex_init(&mis_current.rp_mutex); qemu_event_init(&mis_current.main_thread_load_event, false); qemu_sem_init(&mis_current.postcopy_pause_sem_dst, 0); + qemu_sem_init(&mis_current.postcopy_pause_sem_fault, 0); once =3D true; } return &mis_current; diff --git a/migration/migration.h b/migration/migration.h index 2710627357..b9bc617175 100644 --- a/migration/migration.h +++ b/migration/migration.h @@ -64,6 +64,7 @@ struct MigrationIncomingState { =20 /* notify PAUSED postcopy incoming migrations to try to continue */ QemuSemaphore postcopy_pause_sem_dst; + QemuSemaphore postcopy_pause_sem_fault; }; =20 MigrationIncomingState *migration_incoming_get_current(void); diff --git a/migration/postcopy-ram.c b/migration/postcopy-ram.c index e770077a81..b09adebc1d 100644 --- a/migration/postcopy-ram.c +++ b/migration/postcopy-ram.c @@ -500,6 +500,17 @@ static int ram_block_enable_notify(const char *block_n= ame, void *host_addr, return 0; } =20 +static bool postcopy_pause_fault_thread(MigrationIncomingState *mis) +{ + trace_postcopy_pause_fault_thread(); + + qemu_sem_wait(&mis->postcopy_pause_sem_fault); + + trace_postcopy_pause_fault_thread_continued(); + + return true; +} + /* * Handle faults detected by the USERFAULT markings */ @@ -547,6 +558,22 @@ static void *postcopy_ram_fault_thread(void *opaque) } } =20 + if (!mis->to_src_file) { + /* + * Possibly someone tells us that the return path is + * broken already using the event. We should hold until + * the channel is rebuilt. + */ + if (postcopy_pause_fault_thread(mis)) { + last_rb =3D NULL; + /* Continue to read the userfaultfd */ + } else { + error_report("%s: paused but don't allow to continue", + __func__); + break; + } + } + ret =3D read(mis->userfault_fd, &msg, sizeof(msg)); if (ret !=3D sizeof(msg)) { if (errno =3D=3D EAGAIN) { @@ -586,18 +613,33 @@ static void *postcopy_ram_fault_thread(void *opaque) qemu_ram_get_idstr(rb), rb_offset); =20 +retry: /* * Send the request to the source - we want to request one * of our host page sizes (which is >=3D TPS) */ if (rb !=3D last_rb) { last_rb =3D rb; - migrate_send_rp_req_pages(mis, qemu_ram_get_idstr(rb), - rb_offset, qemu_ram_pagesize(rb)); + ret =3D migrate_send_rp_req_pages(mis, qemu_ram_get_idstr(rb), + rb_offset, qemu_ram_pagesize(r= b)); } else { /* Save some space */ - migrate_send_rp_req_pages(mis, NULL, - rb_offset, qemu_ram_pagesize(rb)); + ret =3D migrate_send_rp_req_pages(mis, NULL, + rb_offset, qemu_ram_pagesize(r= b)); + } + + if (ret) { + /* May be network failure, try to wait for recovery */ + if (ret =3D=3D -EIO && postcopy_pause_fault_thread(mis)) { + /* We got reconnected somehow, try to continue */ + last_rb =3D NULL; + goto retry; + } else { + /* This is a unavoidable fault */ + error_report("%s: migrate_send_rp_req_pages() get %d", + __func__, ret); + break; + } } } trace_postcopy_ram_fault_thread_exit(); diff --git a/migration/savevm.c b/migration/savevm.c index 93e308ebf0..86ada6d0e7 100644 --- a/migration/savevm.c +++ b/migration/savevm.c @@ -2039,6 +2039,9 @@ static bool postcopy_pause_incoming(MigrationIncoming= State *mis) mis->to_src_file =3D NULL; qemu_mutex_unlock(&mis->rp_mutex); =20 + /* Notify the fault thread for the invalidated file handle */ + postcopy_fault_thread_notify(mis); + error_report("Detected IO failure for postcopy. " "Migration paused."); =20 diff --git a/migration/trace-events b/migration/trace-events index a4031cfe00..32f02cbdcc 100644 --- a/migration/trace-events +++ b/migration/trace-events @@ -101,6 +101,8 @@ open_return_path_on_source_continue(void) "" postcopy_start(void) "" postcopy_pause_return_path(void) "" postcopy_pause_return_path_continued(void) "" +postcopy_pause_fault_thread(void) "" +postcopy_pause_fault_thread_continued(void) "" postcopy_pause_continued(void) "" postcopy_pause_incoming(void) "" postcopy_pause_incoming_continued(void) "" --=20 2.13.5 From nobody Sun Apr 28 04:06:08 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 15081370356331016.7059925088877; Sun, 15 Oct 2017 23:57:15 -0700 (PDT) Received: from localhost ([::1]:59621 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1e3zKj-0003qf-3I for importer@patchew.org; Mon, 16 Oct 2017 02:57:05 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:42405) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1e3zGz-0000u3-Jq for qemu-devel@nongnu.org; Mon, 16 Oct 2017 02:53:14 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1e3zGy-0001o4-09 for qemu-devel@nongnu.org; Mon, 16 Oct 2017 02:53:13 -0400 Received: from mx1.redhat.com ([209.132.183.28]:60426) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1e3zGx-0001n9-OV for qemu-devel@nongnu.org; Mon, 16 Oct 2017 02:53:11 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id C62D14E909; Mon, 16 Oct 2017 06:53:10 +0000 (UTC) Received: from pxdev.xzpeter.org.com (dhcp-15-225.nay.redhat.com [10.66.15.225]) by smtp.corp.redhat.com (Postfix) with ESMTP id 817AE6269A; Mon, 16 Oct 2017 06:53:08 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com C62D14E909 Authentication-Results: ext-mx09.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx09.extmail.prod.ext.phx2.redhat.com; spf=fail smtp.mailfrom=peterx@redhat.com From: Peter Xu To: qemu-devel@nongnu.org Date: Mon, 16 Oct 2017 14:51:54 +0800 Message-Id: <20171016065216.18162-11-peterx@redhat.com> In-Reply-To: <20171016065216.18162-1-peterx@redhat.com> References: <20171016065216.18162-1-peterx@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.38]); Mon, 16 Oct 2017 06:53:10 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH v3 10/32] qmp: hmp: add migrate "resume" option X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Andrea Arcangeli , Juan Quintela , Alexey Perevalov , peterx@redhat.com, "Dr . David Alan Gilbert" Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" It will be used when we want to resume one paused migration. Reviewed-by: Dr. David Alan Gilbert Signed-off-by: Peter Xu --- hmp-commands.hx | 7 ++++--- hmp.c | 4 +++- migration/migration.c | 2 +- qapi/migration.json | 5 ++++- 4 files changed, 12 insertions(+), 6 deletions(-) diff --git a/hmp-commands.hx b/hmp-commands.hx index 1941e19932..7adb029b34 100644 --- a/hmp-commands.hx +++ b/hmp-commands.hx @@ -928,13 +928,14 @@ ETEXI =20 { .name =3D "migrate", - .args_type =3D "detach:-d,blk:-b,inc:-i,uri:s", - .params =3D "[-d] [-b] [-i] uri", + .args_type =3D "detach:-d,blk:-b,inc:-i,resume:-r,uri:s", + .params =3D "[-d] [-b] [-i] [-r] uri", .help =3D "migrate to URI (using -d to not wait for completi= on)" "\n\t\t\t -b for migration without shared storage with" " full copy of disk\n\t\t\t -i for migration without " "shared storage with incremental copy of disk " - "(base image shared between src and destination)", + "(base image shared between src and destination)" + "\n\t\t\t -r to resume a paused migration", .cmd =3D hmp_migrate, }, =20 diff --git a/hmp.c b/hmp.c index 739d330f4e..7b1abcd535 100644 --- a/hmp.c +++ b/hmp.c @@ -1899,10 +1899,12 @@ void hmp_migrate(Monitor *mon, const QDict *qdict) bool detach =3D qdict_get_try_bool(qdict, "detach", false); bool blk =3D qdict_get_try_bool(qdict, "blk", false); bool inc =3D qdict_get_try_bool(qdict, "inc", false); + bool resume =3D qdict_get_try_bool(qdict, "resume", false); const char *uri =3D qdict_get_str(qdict, "uri"); Error *err =3D NULL; =20 - qmp_migrate(uri, !!blk, blk, !!inc, inc, false, false, &err); + qmp_migrate(uri, !!blk, blk, !!inc, inc, + false, false, true, resume, &err); if (err) { error_report_err(err); return; diff --git a/migration/migration.c b/migration/migration.c index cebd75c5ce..059c9cc334 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -1320,7 +1320,7 @@ bool migration_is_blocked(Error **errp) =20 void qmp_migrate(const char *uri, bool has_blk, bool blk, bool has_inc, bool inc, bool has_detach, bool detach, - Error **errp) + bool has_resume, bool resume, Error **errp) { Error *local_err =3D NULL; MigrationState *s =3D migrate_get_current(); diff --git a/qapi/migration.json b/qapi/migration.json index d5fd937c26..26df532f29 100644 --- a/qapi/migration.json +++ b/qapi/migration.json @@ -966,6 +966,8 @@ # @detach: this argument exists only for compatibility reasons and # is ignored by QEMU # +# @resume: resume one paused migration, default "off". (since 2.11) +# # Returns: nothing on success # # Since: 0.14.0 @@ -987,7 +989,8 @@ # ## { 'command': 'migrate', - 'data': {'uri': 'str', '*blk': 'bool', '*inc': 'bool', '*detach': 'bool'= } } + 'data': {'uri': 'str', '*blk': 'bool', '*inc': 'bool', + '*detach': 'bool', '*resume': 'bool' } } =20 ## # @migrate-incoming: --=20 2.13.5 From nobody Sun Apr 28 04:06:08 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 150813735229562.594806164781744; Mon, 16 Oct 2017 00:02:32 -0700 (PDT) Received: from localhost ([::1]:59648 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1e3zPp-0000Ho-DF for importer@patchew.org; Mon, 16 Oct 2017 03:02:21 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:42422) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1e3zH1-0000vY-Fs for qemu-devel@nongnu.org; Mon, 16 Oct 2017 02:53:16 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1e3zH0-0001vp-Mb for qemu-devel@nongnu.org; Mon, 16 Oct 2017 02:53:15 -0400 Received: from mx1.redhat.com ([209.132.183.28]:50982) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1e3zH0-0001tD-H0 for qemu-devel@nongnu.org; Mon, 16 Oct 2017 02:53:14 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 79888C04AC58; Mon, 16 Oct 2017 06:53:13 +0000 (UTC) Received: from pxdev.xzpeter.org.com (dhcp-15-225.nay.redhat.com [10.66.15.225]) by smtp.corp.redhat.com (Postfix) with ESMTP id 40E0262923; Mon, 16 Oct 2017 06:53:11 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com 79888C04AC58 Authentication-Results: ext-mx07.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx07.extmail.prod.ext.phx2.redhat.com; spf=fail smtp.mailfrom=peterx@redhat.com From: Peter Xu To: qemu-devel@nongnu.org Date: Mon, 16 Oct 2017 14:51:55 +0800 Message-Id: <20171016065216.18162-12-peterx@redhat.com> In-Reply-To: <20171016065216.18162-1-peterx@redhat.com> References: <20171016065216.18162-1-peterx@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.31]); Mon, 16 Oct 2017 06:53:13 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH v3 11/32] migration: pass MigrationState to migrate_init() X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Andrea Arcangeli , Juan Quintela , Alexey Perevalov , peterx@redhat.com, "Dr . David Alan Gilbert" Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Let the callers take the object, then pass it to migrate_init(). Reviewed-by: Dr. David Alan Gilbert Signed-off-by: Peter Xu --- migration/migration.c | 7 ++----- migration/migration.h | 2 +- migration/savevm.c | 5 ++++- 3 files changed, 7 insertions(+), 7 deletions(-) diff --git a/migration/migration.c b/migration/migration.c index 059c9cc334..3a734e9dda 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -1220,10 +1220,8 @@ bool migration_is_idle(void) return false; } =20 -MigrationState *migrate_init(void) +void migrate_init(MigrationState *s) { - MigrationState *s =3D migrate_get_current(); - /* * Reinitialise all migration state, except * parameters/capabilities that the user set, and @@ -1251,7 +1249,6 @@ MigrationState *migrate_init(void) migrate_set_state(&s->state, MIGRATION_STATUS_NONE, MIGRATION_STATUS_S= ETUP); =20 s->total_time =3D qemu_clock_get_ms(QEMU_CLOCK_REALTIME); - return s; } =20 static GSList *migration_blockers; @@ -1359,7 +1356,7 @@ void qmp_migrate(const char *uri, bool has_blk, bool = blk, migrate_set_block_incremental(s, true); } =20 - s =3D migrate_init(); + migrate_init(s); =20 if (strstart(uri, "tcp:", &p)) { tcp_start_outgoing_migration(s, p, &local_err); diff --git a/migration/migration.h b/migration/migration.h index b9bc617175..9502b2aad6 100644 --- a/migration/migration.h +++ b/migration/migration.h @@ -173,7 +173,7 @@ void migrate_fd_error(MigrationState *s, const Error *e= rror); =20 void migrate_fd_connect(MigrationState *s); =20 -MigrationState *migrate_init(void); +void migrate_init(MigrationState *s); bool migration_is_blocked(Error **errp); /* True if outgoing migration has entered postcopy phase */ bool migration_in_postcopy(void); diff --git a/migration/savevm.c b/migration/savevm.c index 86ada6d0e7..6d6f8ee3e4 100644 --- a/migration/savevm.c +++ b/migration/savevm.c @@ -1256,8 +1256,11 @@ void qemu_savevm_state_cleanup(void) static int qemu_savevm_state(QEMUFile *f, Error **errp) { int ret; - MigrationState *ms =3D migrate_init(); + MigrationState *ms =3D migrate_get_current(); MigrationStatus status; + + migrate_init(ms); + ms->to_dst_file =3D f; =20 if (migration_is_blocked(errp)) { --=20 2.13.5 From nobody Sun Apr 28 04:06:08 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1508137183539578.8983840410665; Sun, 15 Oct 2017 23:59:43 -0700 (PDT) Received: from localhost ([::1]:59629 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1e3zNA-000698-J4 for importer@patchew.org; Mon, 16 Oct 2017 02:59:36 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:42493) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1e3zH9-00011q-2v for qemu-devel@nongnu.org; Mon, 16 Oct 2017 02:53:24 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1e3zH6-000270-04 for qemu-devel@nongnu.org; Mon, 16 Oct 2017 02:53:23 -0400 Received: from mx1.redhat.com ([209.132.183.28]:52792) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1e3zH5-00025u-Nt for qemu-devel@nongnu.org; Mon, 16 Oct 2017 02:53:19 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id B1ED77E428; Mon, 16 Oct 2017 06:53:18 +0000 (UTC) Received: from pxdev.xzpeter.org.com (dhcp-15-225.nay.redhat.com [10.66.15.225]) by smtp.corp.redhat.com (Postfix) with ESMTP id ED30D627DB; Mon, 16 Oct 2017 06:53:13 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com B1ED77E428 Authentication-Results: ext-mx03.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx03.extmail.prod.ext.phx2.redhat.com; spf=fail smtp.mailfrom=peterx@redhat.com From: Peter Xu To: qemu-devel@nongnu.org Date: Mon, 16 Oct 2017 14:51:56 +0800 Message-Id: <20171016065216.18162-13-peterx@redhat.com> In-Reply-To: <20171016065216.18162-1-peterx@redhat.com> References: <20171016065216.18162-1-peterx@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.27]); Mon, 16 Oct 2017 06:53:18 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH v3 12/32] migration: rebuild channel on source X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Andrea Arcangeli , Juan Quintela , Alexey Perevalov , peterx@redhat.com, "Dr . David Alan Gilbert" Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" This patch detects the "resume" flag of migration command, rebuild the channels only if the flag is set. Reviewed-by: Dr. David Alan Gilbert Signed-off-by: Peter Xu --- migration/migration.c | 92 ++++++++++++++++++++++++++++++++++++++---------= ---- 1 file changed, 69 insertions(+), 23 deletions(-) diff --git a/migration/migration.c b/migration/migration.c index 3a734e9dda..56ba011007 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -1315,49 +1315,75 @@ bool migration_is_blocked(Error **errp) return false; } =20 -void qmp_migrate(const char *uri, bool has_blk, bool blk, - bool has_inc, bool inc, bool has_detach, bool detach, - bool has_resume, bool resume, Error **errp) +/* Returns true if continue to migrate, or false if error detected */ +static bool migrate_prepare(MigrationState *s, bool blk, bool blk_inc, + bool resume, Error **errp) { Error *local_err =3D NULL; - MigrationState *s =3D migrate_get_current(); - const char *p; + + if (resume) { + if (s->state !=3D MIGRATION_STATUS_POSTCOPY_PAUSED) { + error_setg(errp, "Cannot resume if there is no " + "paused migration"); + return false; + } + /* This is a resume, skip init status */ + return true; + } =20 if (migration_is_setup_or_active(s->state) || s->state =3D=3D MIGRATION_STATUS_CANCELLING || s->state =3D=3D MIGRATION_STATUS_COLO) { error_setg(errp, QERR_MIGRATION_ACTIVE); - return; + return false; } + if (runstate_check(RUN_STATE_INMIGRATE)) { error_setg(errp, "Guest is waiting for an incoming migration"); - return; + return false; } =20 if (migration_is_blocked(errp)) { - return; + return false; } =20 - if ((has_blk && blk) || (has_inc && inc)) { + if (blk || blk_inc) { if (migrate_use_block() || migrate_use_block_incremental()) { error_setg(errp, "Command options are incompatible with " "current migration capabilities"); - return; + return false; } migrate_set_block_enabled(true, &local_err); if (local_err) { error_propagate(errp, local_err); - return; + return false; } s->must_remove_block_options =3D true; } =20 - if (has_inc && inc) { + if (blk_inc) { migrate_set_block_incremental(s, true); } =20 migrate_init(s); =20 + return true; +} + +void qmp_migrate(const char *uri, bool has_blk, bool blk, + bool has_inc, bool inc, bool has_detach, bool detach, + bool has_resume, bool resume, Error **errp) +{ + Error *local_err =3D NULL; + MigrationState *s =3D migrate_get_current(); + const char *p; + + if (!migrate_prepare(s, has_blk && blk, has_inc && inc, + has_resume && resume, errp)) { + /* Error detected, put into errp */ + return; + } + if (strstart(uri, "tcp:", &p)) { tcp_start_outgoing_migration(s, p, &local_err); #ifdef CONFIG_RDMA @@ -1811,7 +1837,8 @@ out: return NULL; } =20 -static int open_return_path_on_source(MigrationState *ms) +static int open_return_path_on_source(MigrationState *ms, + bool create_thread) { =20 ms->rp_state.from_dst_file =3D qemu_file_get_return_path(ms->to_dst_fi= le); @@ -1820,6 +1847,12 @@ static int open_return_path_on_source(MigrationState= *ms) } =20 trace_open_return_path_on_source(); + + if (!create_thread) { + /* We're done */ + return 0; + } + qemu_thread_create(&ms->rp_state.rp_thread, "return path", source_return_path_thread, ms, QEMU_THREAD_JOINABLE= ); =20 @@ -2385,15 +2418,24 @@ static void *migration_thread(void *opaque) =20 void migrate_fd_connect(MigrationState *s) { - s->expected_downtime =3D s->parameters.downtime_limit; - s->cleanup_bh =3D qemu_bh_new(migrate_fd_cleanup, s); + int64_t rate_limit; + bool resume =3D s->state =3D=3D MIGRATION_STATUS_POSTCOPY_PAUSED; =20 - qemu_file_set_blocking(s->to_dst_file, true); - qemu_file_set_rate_limit(s->to_dst_file, - s->parameters.max_bandwidth / XFER_LIMIT_RATI= O); + if (resume) { + /* This is a resumed migration */ + rate_limit =3D INT64_MAX; + } else { + /* This is a fresh new migration */ + rate_limit =3D s->parameters.max_bandwidth / XFER_LIMIT_RATIO; + s->expected_downtime =3D s->parameters.downtime_limit; + s->cleanup_bh =3D qemu_bh_new(migrate_fd_cleanup, s); =20 - /* Notify before starting migration thread */ - notifier_list_notify(&migration_state_notifiers, s); + /* Notify before starting migration thread */ + notifier_list_notify(&migration_state_notifiers, s); + } + + qemu_file_set_rate_limit(s->to_dst_file, rate_limit); + qemu_file_set_blocking(s->to_dst_file, true); =20 /* * Open the return path. For postcopy, it is used exclusively. For @@ -2401,15 +2443,19 @@ void migrate_fd_connect(MigrationState *s) * QEMU uses the return path. */ if (migrate_postcopy_ram() || migrate_use_return_path()) { - if (open_return_path_on_source(s)) { + if (open_return_path_on_source(s, !resume)) { error_report("Unable to open return-path for postcopy"); - migrate_set_state(&s->state, MIGRATION_STATUS_SETUP, - MIGRATION_STATUS_FAILED); + migrate_set_state(&s->state, s->state, MIGRATION_STATUS_FAILED= ); migrate_fd_cleanup(s); return; } } =20 + if (resume) { + /* TODO: do the resume logic */ + return; + } + if (multifd_save_setup() !=3D 0) { migrate_set_state(&s->state, MIGRATION_STATUS_SETUP, MIGRATION_STATUS_FAILED); --=20 2.13.5 From nobody Sun Apr 28 04:06:08 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1508137399731990.6977740322529; Mon, 16 Oct 2017 00:03:19 -0700 (PDT) Received: from localhost ([::1]:59650 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1e3zQi-0000zP-UA for importer@patchew.org; Mon, 16 Oct 2017 03:03:16 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:42514) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1e3zHB-00013X-H6 for qemu-devel@nongnu.org; Mon, 16 Oct 2017 02:53:26 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1e3zHA-0002Bx-AB for qemu-devel@nongnu.org; Mon, 16 Oct 2017 02:53:25 -0400 Received: from mx1.redhat.com ([209.132.183.28]:38010) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1e3zHA-0002B8-2G for qemu-devel@nongnu.org; Mon, 16 Oct 2017 02:53:24 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 191D881DED; Mon, 16 Oct 2017 06:53:23 +0000 (UTC) Received: from pxdev.xzpeter.org.com (dhcp-15-225.nay.redhat.com [10.66.15.225]) by smtp.corp.redhat.com (Postfix) with ESMTP id 2CD2E5EE02; Mon, 16 Oct 2017 06:53:18 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com 191D881DED Authentication-Results: ext-mx01.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx01.extmail.prod.ext.phx2.redhat.com; spf=fail smtp.mailfrom=peterx@redhat.com From: Peter Xu To: qemu-devel@nongnu.org Date: Mon, 16 Oct 2017 14:51:57 +0800 Message-Id: <20171016065216.18162-14-peterx@redhat.com> In-Reply-To: <20171016065216.18162-1-peterx@redhat.com> References: <20171016065216.18162-1-peterx@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.25]); Mon, 16 Oct 2017 06:53:23 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH v3 13/32] migration: new state "postcopy-recover" X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Andrea Arcangeli , Juan Quintela , Alexey Perevalov , peterx@redhat.com, "Dr . David Alan Gilbert" Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Introducing new migration state "postcopy-recover". If a migration procedure is paused and the connection is rebuilt afterward successfully, we'll switch the source VM state from "postcopy-paused" to the new state "postcopy-recover", then we'll do the resume logic in the migration thread (along with the return path thread). This patch only do the state switch on source side. Another following up patch will handle the state switching on destination side using the same status bit. Reviewed-by: Dr. David Alan Gilbert Signed-off-by: Peter Xu --- migration/migration.c | 76 ++++++++++++++++++++++++++++++++++++++---------= ---- qapi/migration.json | 4 ++- 2 files changed, 60 insertions(+), 20 deletions(-) diff --git a/migration/migration.c b/migration/migration.c index 56ba011007..76fdad9a72 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -550,6 +550,7 @@ static bool migration_is_setup_or_active(int state) case MIGRATION_STATUS_ACTIVE: case MIGRATION_STATUS_POSTCOPY_ACTIVE: case MIGRATION_STATUS_POSTCOPY_PAUSED: + case MIGRATION_STATUS_POSTCOPY_RECOVER: case MIGRATION_STATUS_SETUP: return true; =20 @@ -626,6 +627,7 @@ MigrationInfo *qmp_query_migrate(Error **errp) case MIGRATION_STATUS_CANCELLING: case MIGRATION_STATUS_POSTCOPY_ACTIVE: case MIGRATION_STATUS_POSTCOPY_PAUSED: + case MIGRATION_STATUS_POSTCOPY_RECOVER: /* TODO add some postcopy stats */ info->has_status =3D true; info->has_total_time =3D true; @@ -2157,6 +2159,13 @@ typedef enum MigThrError { MIG_THR_ERR_FATAL =3D 2, } MigThrError; =20 +/* Return zero if success, or <0 for error */ +static int postcopy_do_resume(MigrationState *s) +{ + /* TODO: do the resume logic */ + return 0; +} + /* * We don't return until we are in a safe state to continue current * postcopy migration. Returns MIG_THR_ERR_RECOVERED if recovered, or @@ -2165,29 +2174,55 @@ typedef enum MigThrError { static MigThrError postcopy_pause(MigrationState *s) { assert(s->state =3D=3D MIGRATION_STATUS_POSTCOPY_ACTIVE); - migrate_set_state(&s->state, MIGRATION_STATUS_POSTCOPY_ACTIVE, - MIGRATION_STATUS_POSTCOPY_PAUSED); =20 - /* Current channel is possibly broken. Release it. */ - assert(s->to_dst_file); - qemu_file_shutdown(s->to_dst_file); - qemu_fclose(s->to_dst_file); - s->to_dst_file =3D NULL; + while (true) { + migrate_set_state(&s->state, s->state, + MIGRATION_STATUS_POSTCOPY_PAUSED); =20 - error_report("Detected IO failure for postcopy. " - "Migration paused."); + /* Current channel is possibly broken. Release it. */ + assert(s->to_dst_file); + qemu_file_shutdown(s->to_dst_file); + qemu_fclose(s->to_dst_file); + s->to_dst_file =3D NULL; =20 - /* - * We wait until things fixed up. Then someone will setup the - * status back for us. - */ - while (s->state =3D=3D MIGRATION_STATUS_POSTCOPY_PAUSED) { - qemu_sem_wait(&s->postcopy_pause_sem); - } + error_report("Detected IO failure for postcopy. " + "Migration paused."); + + /* + * We wait until things fixed up. Then someone will setup the + * status back for us. + */ + while (s->state =3D=3D MIGRATION_STATUS_POSTCOPY_PAUSED) { + qemu_sem_wait(&s->postcopy_pause_sem); + } =20 - trace_postcopy_pause_continued(); + if (s->state =3D=3D MIGRATION_STATUS_POSTCOPY_RECOVER) { + /* Woken up by a recover procedure. Give it a shot */ + + /* + * Firstly, let's wake up the return path now, with a new + * return path channel. + */ + qemu_sem_post(&s->postcopy_pause_rp_sem); =20 - return MIG_THR_ERR_RECOVERED; + /* Do the resume logic */ + if (postcopy_do_resume(s) =3D=3D 0) { + /* Let's continue! */ + trace_postcopy_pause_continued(); + return MIG_THR_ERR_RECOVERED; + } else { + /* + * Something wrong happened during the recovery, let's + * pause again. Pause is always better than throwing + * data away. + */ + continue; + } + } else { + /* This is not right... Time to quit. */ + return MIG_THR_ERR_FATAL; + } + } } =20 static MigThrError migration_detect_error(MigrationState *s) @@ -2452,7 +2487,10 @@ void migrate_fd_connect(MigrationState *s) } =20 if (resume) { - /* TODO: do the resume logic */ + /* Wakeup the main migration thread to do the recovery */ + migrate_set_state(&s->state, MIGRATION_STATUS_POSTCOPY_PAUSED, + MIGRATION_STATUS_POSTCOPY_RECOVER); + qemu_sem_post(&s->postcopy_pause_sem); return; } =20 diff --git a/qapi/migration.json b/qapi/migration.json index 26df532f29..f8132e683a 100644 --- a/qapi/migration.json +++ b/qapi/migration.json @@ -91,6 +91,8 @@ # # @postcopy-paused: during postcopy but paused. (since 2.11) # +# @postcopy-recover: trying to recover from a paused postcopy. (since 2.11) +# # @completed: migration is finished. # # @failed: some error occurred during migration process. @@ -104,7 +106,7 @@ { 'enum': 'MigrationStatus', 'data': [ 'none', 'setup', 'cancelling', 'cancelled', 'active', 'postcopy-active', 'postcopy-paused', - 'completed', 'failed', 'colo' ] } + 'postcopy-recover', 'completed', 'failed', 'colo' ] } =20 ## # @MigrationInfo: --=20 2.13.5 From nobody Sun Apr 28 04:06:08 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1508137346633859.6659559440359; Mon, 16 Oct 2017 00:02:26 -0700 (PDT) Received: from localhost ([::1]:59647 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1e3zPi-0000BY-5h for importer@patchew.org; Mon, 16 Oct 2017 03:02:14 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:42568) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1e3zHI-0001A7-T0 for qemu-devel@nongnu.org; Mon, 16 Oct 2017 02:53:34 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1e3zHF-0002HA-Qi for qemu-devel@nongnu.org; Mon, 16 Oct 2017 02:53:32 -0400 Received: from mx1.redhat.com ([209.132.183.28]:52398) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1e3zHF-0002GB-KR for qemu-devel@nongnu.org; Mon, 16 Oct 2017 02:53:29 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 8E5FA859FA; Mon, 16 Oct 2017 06:53:28 +0000 (UTC) Received: from pxdev.xzpeter.org.com (dhcp-15-225.nay.redhat.com [10.66.15.225]) by smtp.corp.redhat.com (Postfix) with ESMTP id 8993162923; Mon, 16 Oct 2017 06:53:23 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com 8E5FA859FA Authentication-Results: ext-mx02.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx02.extmail.prod.ext.phx2.redhat.com; spf=fail smtp.mailfrom=peterx@redhat.com From: Peter Xu To: qemu-devel@nongnu.org Date: Mon, 16 Oct 2017 14:51:58 +0800 Message-Id: <20171016065216.18162-15-peterx@redhat.com> In-Reply-To: <20171016065216.18162-1-peterx@redhat.com> References: <20171016065216.18162-1-peterx@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.26]); Mon, 16 Oct 2017 06:53:28 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH v3 14/32] migration: wakeup dst ram-load-thread for recover X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Andrea Arcangeli , Juan Quintela , Alexey Perevalov , peterx@redhat.com, "Dr . David Alan Gilbert" Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" On the destination side, we cannot wake up all the threads when we got reconnected. The first thing to do is to wake up the main load thread, so that we can continue to receive valid messages from source again and reply when needed. At this point, we switch the destination VM state from postcopy-paused back to postcopy-recover. Now we are finally ready to do the resume logic. Reviewed-by: Dr. David Alan Gilbert Signed-off-by: Peter Xu --- migration/migration.c | 30 ++++++++++++++++++++++++++++-- 1 file changed, 28 insertions(+), 2 deletions(-) diff --git a/migration/migration.c b/migration/migration.c index 76fdad9a72..c869d6ceb5 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -420,8 +420,34 @@ static void migration_incoming_process(void) =20 void migration_fd_process_incoming(QEMUFile *f) { - migration_incoming_setup(f); - migration_incoming_process(); + MigrationIncomingState *mis =3D migration_incoming_get_current(); + + if (mis->state =3D=3D MIGRATION_STATUS_POSTCOPY_PAUSED) { + /* Resumed from a paused postcopy migration */ + + mis->from_src_file =3D f; + /* Postcopy has standalone thread to do vm load */ + qemu_file_set_blocking(f, true); + + /* Re-configure the return path */ + mis->to_src_file =3D qemu_file_get_return_path(f); + + migrate_set_state(&mis->state, MIGRATION_STATUS_POSTCOPY_PAUSED, + MIGRATION_STATUS_POSTCOPY_RECOVER); + + /* + * Here, we only wake up the main loading thread (while the + * fault thread will still be waiting), so that we can receive + * commands from source now, and answer it if needed. The + * fault thread will be woken up afterwards until we are sure + * that source is ready to reply to page requests. + */ + qemu_sem_post(&mis->postcopy_pause_sem_dst); + } else { + /* New incoming migration */ + migration_incoming_setup(f); + migration_incoming_process(); + } } =20 void migration_ioc_process_incoming(QIOChannel *ioc) --=20 2.13.5 From nobody Sun Apr 28 04:06:08 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1508137517645108.89650227524135; Mon, 16 Oct 2017 00:05:17 -0700 (PDT) Received: from localhost ([::1]:59656 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1e3zSZ-0002SE-Qz for importer@patchew.org; Mon, 16 Oct 2017 03:05:11 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:42580) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1e3zHJ-0001A8-Ib for qemu-devel@nongnu.org; Mon, 16 Oct 2017 02:53:34 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1e3zHI-0002Kr-Gl for qemu-devel@nongnu.org; Mon, 16 Oct 2017 02:53:33 -0400 Received: from mx1.redhat.com ([209.132.183.28]:60256) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1e3zHI-0002Jd-7u for qemu-devel@nongnu.org; Mon, 16 Oct 2017 02:53:32 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 3C9FA80B22; Mon, 16 Oct 2017 06:53:31 +0000 (UTC) Received: from pxdev.xzpeter.org.com (dhcp-15-225.nay.redhat.com [10.66.15.225]) by smtp.corp.redhat.com (Postfix) with ESMTP id 084EE62685; Mon, 16 Oct 2017 06:53:28 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com 3C9FA80B22 Authentication-Results: ext-mx04.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx04.extmail.prod.ext.phx2.redhat.com; spf=fail smtp.mailfrom=peterx@redhat.com From: Peter Xu To: qemu-devel@nongnu.org Date: Mon, 16 Oct 2017 14:51:59 +0800 Message-Id: <20171016065216.18162-16-peterx@redhat.com> In-Reply-To: <20171016065216.18162-1-peterx@redhat.com> References: <20171016065216.18162-1-peterx@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.28]); Mon, 16 Oct 2017 06:53:31 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH v3 15/32] migration: new cmd MIG_CMD_RECV_BITMAP X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Andrea Arcangeli , Juan Quintela , Alexey Perevalov , peterx@redhat.com, "Dr . David Alan Gilbert" Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Add a new vm command MIG_CMD_RECV_BITMAP to request received bitmap for one ramblock. Reviewed-by: Dr. David Alan Gilbert Signed-off-by: Peter Xu --- migration/savevm.c | 61 ++++++++++++++++++++++++++++++++++++++++++++++= ++++ migration/savevm.h | 1 + migration/trace-events | 2 ++ 3 files changed, 64 insertions(+) diff --git a/migration/savevm.c b/migration/savevm.c index 6d6f8ee3e4..0f61da3ebb 100644 --- a/migration/savevm.c +++ b/migration/savevm.c @@ -78,6 +78,7 @@ enum qemu_vm_cmd { were previously sent during precopy but are dirty. */ MIG_CMD_PACKAGED, /* Send a wrapped stream within this stream= */ + MIG_CMD_RECV_BITMAP, /* Request for recved bitmap on dst */ MIG_CMD_MAX }; =20 @@ -95,6 +96,7 @@ static struct mig_cmd_args { [MIG_CMD_POSTCOPY_RAM_DISCARD] =3D { .len =3D -1, .name =3D "POSTCOPY_RAM_DI= SCARD" }, [MIG_CMD_PACKAGED] =3D { .len =3D 4, .name =3D "PACKAGED" }, + [MIG_CMD_RECV_BITMAP] =3D { .len =3D -1, .name =3D "RECV_BITMAP" = }, [MIG_CMD_MAX] =3D { .len =3D -1, .name =3D "MAX" }, }; =20 @@ -953,6 +955,19 @@ void qemu_savevm_send_postcopy_run(QEMUFile *f) qemu_savevm_command_send(f, MIG_CMD_POSTCOPY_RUN, 0, NULL); } =20 +void qemu_savevm_send_recv_bitmap(QEMUFile *f, char *block_name) +{ + size_t len; + char buf[256]; + + trace_savevm_send_recv_bitmap(block_name); + + buf[0] =3D len =3D strlen(block_name); + memcpy(buf + 1, block_name, len); + + qemu_savevm_command_send(f, MIG_CMD_RECV_BITMAP, len + 1, (uint8_t *)b= uf); +} + bool qemu_savevm_state_blocked(Error **errp) { SaveStateEntry *se; @@ -1761,6 +1776,49 @@ static int loadvm_handle_cmd_packaged(MigrationIncom= ingState *mis) } =20 /* + * Handle request that source requests for recved_bitmap on + * destination. Payload format: + * + * len (1 byte) + ramblock_name (<255 bytes) + */ +static int loadvm_handle_recv_bitmap(MigrationIncomingState *mis, + uint16_t len) +{ + QEMUFile *file =3D mis->from_src_file; + RAMBlock *rb; + char block_name[256]; + size_t cnt; + + cnt =3D qemu_get_counted_string(file, block_name); + if (!cnt) { + error_report("%s: failed to read block name", __func__); + return -EINVAL; + } + + /* Validate before using the data */ + if (qemu_file_get_error(file)) { + return qemu_file_get_error(file); + } + + if (len !=3D cnt + 1) { + error_report("%s: invalid payload length (%d)", __func__, len); + return -EINVAL; + } + + rb =3D qemu_ram_block_by_name(block_name); + if (!rb) { + error_report("%s: block '%s' not found", __func__, block_name); + return -EINVAL; + } + + /* TODO: send the bitmap back to source */ + + trace_loadvm_handle_recv_bitmap(block_name); + + return 0; +} + +/* * Process an incoming 'QEMU_VM_COMMAND' * 0 just a normal return * LOADVM_QUIT All good, but exit the loop @@ -1833,6 +1891,9 @@ static int loadvm_process_command(QEMUFile *f) =20 case MIG_CMD_POSTCOPY_RAM_DISCARD: return loadvm_postcopy_ram_handle_discard(mis, len); + + case MIG_CMD_RECV_BITMAP: + return loadvm_handle_recv_bitmap(mis, len); } =20 return 0; diff --git a/migration/savevm.h b/migration/savevm.h index 295c4a1f2c..8126b1cc14 100644 --- a/migration/savevm.h +++ b/migration/savevm.h @@ -46,6 +46,7 @@ int qemu_savevm_send_packaged(QEMUFile *f, const uint8_t = *buf, size_t len); void qemu_savevm_send_postcopy_advise(QEMUFile *f); void qemu_savevm_send_postcopy_listen(QEMUFile *f); void qemu_savevm_send_postcopy_run(QEMUFile *f); +void qemu_savevm_send_recv_bitmap(QEMUFile *f, char *block_name); =20 void qemu_savevm_send_postcopy_ram_discard(QEMUFile *f, const char *name, uint16_t len, diff --git a/migration/trace-events b/migration/trace-events index 32f02cbdcc..55c0412aaa 100644 --- a/migration/trace-events +++ b/migration/trace-events @@ -12,6 +12,7 @@ loadvm_state_cleanup(void) "" loadvm_handle_cmd_packaged(unsigned int length) "%u" loadvm_handle_cmd_packaged_main(int ret) "%d" loadvm_handle_cmd_packaged_received(int ret) "%d" +loadvm_handle_recv_bitmap(char *s) "%s" loadvm_postcopy_handle_advise(void) "" loadvm_postcopy_handle_listen(void) "" loadvm_postcopy_handle_run(void) "" @@ -34,6 +35,7 @@ savevm_send_open_return_path(void) "" savevm_send_ping(uint32_t val) "0x%x" savevm_send_postcopy_listen(void) "" savevm_send_postcopy_run(void) "" +savevm_send_recv_bitmap(char *name) "%s" savevm_state_setup(void) "" savevm_state_header(void) "" savevm_state_iterate(void) "" --=20 2.13.5 From nobody Sun Apr 28 04:06:08 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1508137569771413.72622103494405; Mon, 16 Oct 2017 00:06:09 -0700 (PDT) Received: from localhost ([::1]:59662 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1e3zTR-0003FJ-Fs for importer@patchew.org; Mon, 16 Oct 2017 03:06:05 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:42634) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1e3zHM-0001DA-VV for qemu-devel@nongnu.org; Mon, 16 Oct 2017 02:53:39 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1e3zHL-0002P6-6K for qemu-devel@nongnu.org; Mon, 16 Oct 2017 02:53:37 -0400 Received: from mx1.redhat.com ([209.132.183.28]:49114) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1e3zHK-0002No-Tx for qemu-devel@nongnu.org; Mon, 16 Oct 2017 02:53:35 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id E416C5D687; Mon, 16 Oct 2017 06:53:33 +0000 (UTC) Received: from pxdev.xzpeter.org.com (dhcp-15-225.nay.redhat.com [10.66.15.225]) by smtp.corp.redhat.com (Postfix) with ESMTP id AE54062923; Mon, 16 Oct 2017 06:53:31 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com E416C5D687 Authentication-Results: ext-mx10.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx10.extmail.prod.ext.phx2.redhat.com; spf=fail smtp.mailfrom=peterx@redhat.com From: Peter Xu To: qemu-devel@nongnu.org Date: Mon, 16 Oct 2017 14:52:00 +0800 Message-Id: <20171016065216.18162-17-peterx@redhat.com> In-Reply-To: <20171016065216.18162-1-peterx@redhat.com> References: <20171016065216.18162-1-peterx@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.39]); Mon, 16 Oct 2017 06:53:34 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH v3 16/32] migration: new message MIG_RP_MSG_RECV_BITMAP X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Andrea Arcangeli , Juan Quintela , Alexey Perevalov , peterx@redhat.com, "Dr . David Alan Gilbert" Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Introducing new return path message MIG_RP_MSG_RECV_BITMAP to send received bitmap of ramblock back to source. This is the reply message of MIG_CMD_RECV_BITMAP, it contains not only the header (including the ramblock name), and it was appended with the whole ramblock received bitmap on the destination side. When the source receives such a reply message (MIG_RP_MSG_RECV_BITMAP), it parses it, convert it to the dirty bitmap by inverting the bits. One thing to mention is that, when we send the recv bitmap, we are doing these things in extra: - converting the bitmap to little endian, to support when hosts are using different endianess on src/dst. - do proper alignment for 8 bytes, to support when hosts are using different word size (32/64 bits) on src/dst. Signed-off-by: Peter Xu --- migration/migration.c | 68 +++++++++++++++++++++++ migration/migration.h | 2 + migration/ram.c | 144 +++++++++++++++++++++++++++++++++++++++++++++= ++++ migration/ram.h | 3 ++ migration/savevm.c | 2 +- migration/trace-events | 3 ++ 6 files changed, 221 insertions(+), 1 deletion(-) diff --git a/migration/migration.c b/migration/migration.c index c869d6ceb5..2e992e891e 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -93,6 +93,7 @@ enum mig_rp_message_type { =20 MIG_RP_MSG_REQ_PAGES_ID, /* data (start: be64, len: be32, id: string) = */ MIG_RP_MSG_REQ_PAGES, /* data (start: be64, len: be32) */ + MIG_RP_MSG_RECV_BITMAP, /* send recved_bitmap back to source */ =20 MIG_RP_MSG_MAX }; @@ -499,6 +500,45 @@ void migrate_send_rp_pong(MigrationIncomingState *mis, migrate_send_rp_message(mis, MIG_RP_MSG_PONG, sizeof(buf), &buf); } =20 +void migrate_send_rp_recv_bitmap(MigrationIncomingState *mis, + char *block_name) +{ + char buf[512]; + int len; + int64_t res; + + /* + * First, we send the header part. It contains only the len of + * idstr, and the idstr itself. + */ + len =3D strlen(block_name); + buf[0] =3D len; + memcpy(buf + 1, block_name, len); + + if (mis->state !=3D MIGRATION_STATUS_POSTCOPY_RECOVER) { + error_report("%s: MSG_RP_RECV_BITMAP only used for recovery", + __func__); + return; + } + + migrate_send_rp_message(mis, MIG_RP_MSG_RECV_BITMAP, len + 1, buf); + + /* + * Next, we dump the received bitmap to the stream. + * + * TODO: currently we are safe since we are the only one that is + * using the to_src_file handle (fault thread is still paused), + * and it's ok even not taking the mutex. However the best way is + * to take the lock before sending the message header, and release + * the lock after sending the bitmap. + */ + qemu_mutex_lock(&mis->rp_mutex); + res =3D ramblock_recv_bitmap_send(mis->to_src_file, block_name); + qemu_mutex_unlock(&mis->rp_mutex); + + trace_migrate_send_rp_recv_bitmap(block_name, res); +} + MigrationCapabilityStatusList *qmp_query_migrate_capabilities(Error **errp) { MigrationCapabilityStatusList *head =3D NULL; @@ -1685,6 +1725,7 @@ static struct rp_cmd_args { [MIG_RP_MSG_PONG] =3D { .len =3D 4, .name =3D "PONG" }, [MIG_RP_MSG_REQ_PAGES] =3D { .len =3D 12, .name =3D "REQ_PAGES" }, [MIG_RP_MSG_REQ_PAGES_ID] =3D { .len =3D -1, .name =3D "REQ_PAGES_ID= " }, + [MIG_RP_MSG_RECV_BITMAP] =3D { .len =3D -1, .name =3D "RECV_BITMAP"= }, [MIG_RP_MSG_MAX] =3D { .len =3D -1, .name =3D "MAX" }, }; =20 @@ -1729,6 +1770,19 @@ static bool postcopy_pause_return_path_thread(Migrat= ionState *s) return true; } =20 +static int migrate_handle_rp_recv_bitmap(MigrationState *s, char *block_na= me) +{ + RAMBlock *block =3D qemu_ram_block_by_name(block_name); + + if (!block) { + error_report("%s: invalid block name '%s'", __func__, block_name); + return -EINVAL; + } + + /* Fetch the received bitmap and refresh the dirty bitmap */ + return ram_dirty_bitmap_reload(s, block); +} + /* * Handles messages sent on the return path towards the source VM * @@ -1834,6 +1888,20 @@ retry: migrate_handle_rp_req_pages(ms, (char *)&buf[13], start, len); break; =20 + case MIG_RP_MSG_RECV_BITMAP: + if (header_len < 1) { + error_report("%s: missing block name", __func__); + mark_source_rp_bad(ms); + goto out; + } + /* Format: len (1B) + idstr (<255B). This ends the idstr. */ + buf[buf[0] + 1] =3D '\0'; + if (migrate_handle_rp_recv_bitmap(ms, (char *)(buf + 1))) { + mark_source_rp_bad(ms); + goto out; + } + break; + default: break; } diff --git a/migration/migration.h b/migration/migration.h index 9502b2aad6..56ac50839e 100644 --- a/migration/migration.h +++ b/migration/migration.h @@ -211,5 +211,7 @@ void migrate_send_rp_pong(MigrationIncomingState *mis, uint32_t value); int migrate_send_rp_req_pages(MigrationIncomingState *mis, const char* rbn= ame, ram_addr_t start, size_t len); +void migrate_send_rp_recv_bitmap(MigrationIncomingState *mis, + char *block_name); =20 #endif diff --git a/migration/ram.c b/migration/ram.c index 85cb965a14..fa2b4ed207 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -178,6 +178,70 @@ void ramblock_recv_bitmap_set_range(RAMBlock *rb, void= *host_addr, nr); } =20 +#define RAMBLOCK_RECV_BITMAP_ENDING (0x0123456789abcdefULL) + +/* + * Format: bitmap_size (8 bytes) + whole_bitmap (N bytes). + * + * Returns >0 if success with sent bytes, or <0 if error. + */ +int64_t ramblock_recv_bitmap_send(QEMUFile *file, + const char *block_name) +{ + RAMBlock *block =3D qemu_ram_block_by_name(block_name); + unsigned long *le_bitmap, nbits; + uint64_t size; + + if (!block) { + error_report("%s: invalid block name: %s", __func__, block_name); + return -1; + } + + nbits =3D block->used_length >> TARGET_PAGE_BITS; + + /* + * Make sure the tmp bitmap buffer is big enough, e.g., on 32bit + * machines we may need 4 more bytes for padding (see below + * comment). So extend it a bit before hand. + */ + le_bitmap =3D bitmap_new(nbits + BITS_PER_LONG); + + /* + * Always use little endian when sending the bitmap. This is + * required that when source and destination VMs are not using the + * same endianess. (Note: big endian won't work.) + */ + bitmap_to_le(le_bitmap, block->receivedmap, nbits); + + /* Size of the bitmap, in bytes */ + size =3D nbits / 8; + + /* + * size is always aligned to 8 bytes for 64bit machines, but it + * may not be true for 32bit machines. We need this padding to + * make sure the migration can survive even between 32bit and + * 64bit machines. + */ + size =3D ROUND_UP(size, 8); + + qemu_put_be64(file, size); + qemu_put_buffer(file, (const uint8_t *)le_bitmap, size); + /* + * Mark as an end, in case the middle part is screwed up due to + * some "misterious" reason. + */ + qemu_put_be64(file, RAMBLOCK_RECV_BITMAP_ENDING); + qemu_fflush(file); + + free(le_bitmap); + + if (qemu_file_get_error(file)) { + return qemu_file_get_error(file); + } + + return size + sizeof(size); +} + /* * An outstanding page request, on the source, having been received * and queued @@ -2920,6 +2984,86 @@ static bool ram_has_postcopy(void *opaque) return migrate_postcopy_ram(); } =20 +/* + * Read the received bitmap, revert it as the initial dirty bitmap. + * This is only used when the postcopy migration is paused but wants + * to resume from a middle point. + */ +int ram_dirty_bitmap_reload(MigrationState *s, RAMBlock *block) +{ + int ret =3D -EINVAL; + QEMUFile *file =3D s->rp_state.from_dst_file; + unsigned long *le_bitmap, nbits =3D block->used_length >> TARGET_PAGE_= BITS; + uint64_t local_size =3D nbits / 8; + uint64_t size, end_mark; + + trace_ram_dirty_bitmap_reload_begin(block->idstr); + + if (s->state !=3D MIGRATION_STATUS_POSTCOPY_RECOVER) { + error_report("%s: incorrect state %s", __func__, + MigrationStatus_str(s->state)); + return -EINVAL; + } + + /* + * Note: see comments in ramblock_recv_bitmap_send() on why we + * need the endianess convertion, and the paddings. + */ + local_size =3D ROUND_UP(local_size, 8); + + /* Add addings */ + le_bitmap =3D bitmap_new(nbits + BITS_PER_LONG); + + size =3D qemu_get_be64(file); + + /* The size of the bitmap should match with our ramblock */ + if (size !=3D local_size) { + error_report("%s: ramblock '%s' bitmap size mismatch " + "(0x%"PRIx64" !=3D 0x%"PRIx64")", __func__, + block->idstr, size, local_size); + ret =3D -EINVAL; + goto out; + } + + size =3D qemu_get_buffer(file, (uint8_t *)le_bitmap, local_size); + end_mark =3D qemu_get_be64(file); + + ret =3D qemu_file_get_error(file); + if (ret || size !=3D local_size) { + error_report("%s: read bitmap failed for ramblock '%s': %d" + " (size 0x%"PRIx64", got: 0x%"PRIx64")", + __func__, block->idstr, ret, local_size, size); + ret =3D -EIO; + goto out; + } + + if (end_mark !=3D RAMBLOCK_RECV_BITMAP_ENDING) { + error_report("%s: ramblock '%s' end mark incorrect: 0x%"PRIu64, + __func__, block->idstr, end_mark); + ret =3D -EINVAL; + goto out; + } + + /* + * Endianess convertion. We are during postcopy (though paused). + * The dirty bitmap won't change. We can directly modify it. + */ + bitmap_from_le(block->bmap, le_bitmap, nbits); + + /* + * What we received is "received bitmap". Revert it as the initial + * dirty bitmap for this ramblock. + */ + bitmap_complement(block->bmap, block->bmap, nbits); + + trace_ram_dirty_bitmap_reload_complete(block->idstr); + + ret =3D 0; +out: + free(le_bitmap); + return ret; +} + static SaveVMHandlers savevm_ram_handlers =3D { .save_setup =3D ram_save_setup, .save_live_iterate =3D ram_save_iterate, diff --git a/migration/ram.h b/migration/ram.h index 3f5dd232d6..b4195e08ae 100644 --- a/migration/ram.h +++ b/migration/ram.h @@ -61,5 +61,8 @@ void ram_handle_compressed(void *host, uint8_t ch, uint64= _t size); int ramblock_recv_bitmap_test(RAMBlock *rb, void *host_addr); void ramblock_recv_bitmap_set(RAMBlock *rb, void *host_addr); void ramblock_recv_bitmap_set_range(RAMBlock *rb, void *host_addr, size_t = nr); +int64_t ramblock_recv_bitmap_send(QEMUFile *file, + const char *block_name); +int ram_dirty_bitmap_reload(MigrationState *s, RAMBlock *rb); =20 #endif diff --git a/migration/savevm.c b/migration/savevm.c index 0f61da3ebb..2148b198c7 100644 --- a/migration/savevm.c +++ b/migration/savevm.c @@ -1811,7 +1811,7 @@ static int loadvm_handle_recv_bitmap(MigrationIncomin= gState *mis, return -EINVAL; } =20 - /* TODO: send the bitmap back to source */ + migrate_send_rp_recv_bitmap(mis, block_name); =20 trace_loadvm_handle_recv_bitmap(block_name); =20 diff --git a/migration/trace-events b/migration/trace-events index 55c0412aaa..3dcf8a93d9 100644 --- a/migration/trace-events +++ b/migration/trace-events @@ -79,6 +79,8 @@ ram_load_postcopy_loop(uint64_t addr, int flags) "@%" PRI= x64 " %x" ram_postcopy_send_discard_bitmap(void) "" ram_save_page(const char *rbname, uint64_t offset, void *host) "%s: offset= : 0x%" PRIx64 " host: %p" ram_save_queue_pages(const char *rbname, size_t start, size_t len) "%s: st= art: 0x%zx len: 0x%zx" +ram_dirty_bitmap_reload_begin(char *str) "%s" +ram_dirty_bitmap_reload_complete(char *str) "%s" =20 # migration/migration.c await_return_path_close_on_source_close(void) "" @@ -90,6 +92,7 @@ migrate_fd_cancel(void) "" migrate_handle_rp_req_pages(const char *rbname, size_t start, size_t len) = "in %s at 0x%zx len 0x%zx" migrate_pending(uint64_t size, uint64_t max, uint64_t post, uint64_t nonpo= st) "pending size %" PRIu64 " max %" PRIu64 " (post=3D%" PRIu64 " nonpost= =3D%" PRIu64 ")" migrate_send_rp_message(int msg_type, uint16_t len) "%d: len %d" +migrate_send_rp_recv_bitmap(char *name, int64_t size) "block '%s' size 0x%= "PRIi64 migration_completion_file_err(void) "" migration_completion_postcopy_end(void) "" migration_completion_postcopy_end_after_complete(void) "" --=20 2.13.5 From nobody Sun Apr 28 04:06:08 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1508137515159853.5204892341699; Mon, 16 Oct 2017 00:05:15 -0700 (PDT) Received: from localhost ([::1]:59655 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1e3zSU-0002Nc-0Y for importer@patchew.org; Mon, 16 Oct 2017 03:05:06 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:42687) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1e3zHR-0001H4-6L for qemu-devel@nongnu.org; Mon, 16 Oct 2017 02:53:42 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1e3zHO-0002TU-0b for qemu-devel@nongnu.org; Mon, 16 Oct 2017 02:53:41 -0400 Received: from mx1.redhat.com ([209.132.183.28]:60344) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1e3zHN-0002SQ-OV for qemu-devel@nongnu.org; Mon, 16 Oct 2017 02:53:37 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id C224E80467; Mon, 16 Oct 2017 06:53:36 +0000 (UTC) Received: from pxdev.xzpeter.org.com (dhcp-15-225.nay.redhat.com [10.66.15.225]) by smtp.corp.redhat.com (Postfix) with ESMTP id 5EEF85EE02; Mon, 16 Oct 2017 06:53:34 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com C224E80467 Authentication-Results: ext-mx04.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx04.extmail.prod.ext.phx2.redhat.com; spf=fail smtp.mailfrom=peterx@redhat.com From: Peter Xu To: qemu-devel@nongnu.org Date: Mon, 16 Oct 2017 14:52:01 +0800 Message-Id: <20171016065216.18162-18-peterx@redhat.com> In-Reply-To: <20171016065216.18162-1-peterx@redhat.com> References: <20171016065216.18162-1-peterx@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.28]); Mon, 16 Oct 2017 06:53:36 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH v3 17/32] migration: new cmd MIG_CMD_POSTCOPY_RESUME X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Andrea Arcangeli , Juan Quintela , Alexey Perevalov , peterx@redhat.com, "Dr . David Alan Gilbert" Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Introducing this new command to be sent when the source VM is ready to resume the paused migration. What the destination does here is basically release the fault thread to continue service page faults. Reviewed-by: Dr. David Alan Gilbert Signed-off-by: Peter Xu --- migration/savevm.c | 35 +++++++++++++++++++++++++++++++++++ migration/savevm.h | 1 + migration/trace-events | 2 ++ 3 files changed, 38 insertions(+) diff --git a/migration/savevm.c b/migration/savevm.c index 2148b198c7..bb6639812b 100644 --- a/migration/savevm.c +++ b/migration/savevm.c @@ -77,6 +77,7 @@ enum qemu_vm_cmd { MIG_CMD_POSTCOPY_RAM_DISCARD, /* A list of pages to discard that were previously sent during precopy but are dirty. */ + MIG_CMD_POSTCOPY_RESUME, /* resume postcopy on dest */ MIG_CMD_PACKAGED, /* Send a wrapped stream within this stream= */ MIG_CMD_RECV_BITMAP, /* Request for recved bitmap on dst */ MIG_CMD_MAX @@ -95,6 +96,7 @@ static struct mig_cmd_args { [MIG_CMD_POSTCOPY_RUN] =3D { .len =3D 0, .name =3D "POSTCOPY_RUN"= }, [MIG_CMD_POSTCOPY_RAM_DISCARD] =3D { .len =3D -1, .name =3D "POSTCOPY_RAM_DI= SCARD" }, + [MIG_CMD_POSTCOPY_RESUME] =3D { .len =3D 0, .name =3D "POSTCOPY_RESU= ME" }, [MIG_CMD_PACKAGED] =3D { .len =3D 4, .name =3D "PACKAGED" }, [MIG_CMD_RECV_BITMAP] =3D { .len =3D -1, .name =3D "RECV_BITMAP" = }, [MIG_CMD_MAX] =3D { .len =3D -1, .name =3D "MAX" }, @@ -955,6 +957,12 @@ void qemu_savevm_send_postcopy_run(QEMUFile *f) qemu_savevm_command_send(f, MIG_CMD_POSTCOPY_RUN, 0, NULL); } =20 +void qemu_savevm_send_postcopy_resume(QEMUFile *f) +{ + trace_savevm_send_postcopy_resume(); + qemu_savevm_command_send(f, MIG_CMD_POSTCOPY_RESUME, 0, NULL); +} + void qemu_savevm_send_recv_bitmap(QEMUFile *f, char *block_name) { size_t len; @@ -1727,6 +1735,30 @@ static int loadvm_postcopy_handle_run(MigrationIncom= ingState *mis) return LOADVM_QUIT; } =20 +static int loadvm_postcopy_handle_resume(MigrationIncomingState *mis) +{ + if (mis->state !=3D MIGRATION_STATUS_POSTCOPY_RECOVER) { + error_report("%s: illegal resume received", __func__); + /* Don't fail the load, only for this. */ + return 0; + } + + /* + * This means source VM is ready to resume the postcopy migration. + * It's time to switch state and release the fault thread to + * continue service page faults. + */ + migrate_set_state(&mis->state, MIGRATION_STATUS_POSTCOPY_RECOVER, + MIGRATION_STATUS_POSTCOPY_ACTIVE); + qemu_sem_post(&mis->postcopy_pause_sem_fault); + + trace_loadvm_postcopy_handle_resume(); + + /* TODO: Tell source that "we are ready" */ + + return 0; +} + /** * Immediately following this command is a blob of data containing an embe= dded * chunk of migration stream; read it and load it. @@ -1892,6 +1924,9 @@ static int loadvm_process_command(QEMUFile *f) case MIG_CMD_POSTCOPY_RAM_DISCARD: return loadvm_postcopy_ram_handle_discard(mis, len); =20 + case MIG_CMD_POSTCOPY_RESUME: + return loadvm_postcopy_handle_resume(mis); + case MIG_CMD_RECV_BITMAP: return loadvm_handle_recv_bitmap(mis, len); } diff --git a/migration/savevm.h b/migration/savevm.h index 8126b1cc14..a5f3879191 100644 --- a/migration/savevm.h +++ b/migration/savevm.h @@ -46,6 +46,7 @@ int qemu_savevm_send_packaged(QEMUFile *f, const uint8_t = *buf, size_t len); void qemu_savevm_send_postcopy_advise(QEMUFile *f); void qemu_savevm_send_postcopy_listen(QEMUFile *f); void qemu_savevm_send_postcopy_run(QEMUFile *f); +void qemu_savevm_send_postcopy_resume(QEMUFile *f); void qemu_savevm_send_recv_bitmap(QEMUFile *f, char *block_name); =20 void qemu_savevm_send_postcopy_ram_discard(QEMUFile *f, const char *name, diff --git a/migration/trace-events b/migration/trace-events index 3dcf8a93d9..4b60865194 100644 --- a/migration/trace-events +++ b/migration/trace-events @@ -18,6 +18,7 @@ loadvm_postcopy_handle_listen(void) "" loadvm_postcopy_handle_run(void) "" loadvm_postcopy_handle_run_cpu_sync(void) "" loadvm_postcopy_handle_run_vmstart(void) "" +loadvm_postcopy_handle_resume(void) "" loadvm_postcopy_ram_handle_discard(void) "" loadvm_postcopy_ram_handle_discard_end(void) "" loadvm_postcopy_ram_handle_discard_header(const char *ramid, uint16_t len)= "%s: %ud" @@ -35,6 +36,7 @@ savevm_send_open_return_path(void) "" savevm_send_ping(uint32_t val) "0x%x" savevm_send_postcopy_listen(void) "" savevm_send_postcopy_run(void) "" +savevm_send_postcopy_resume(void) "" savevm_send_recv_bitmap(char *name) "%s" savevm_state_setup(void) "" savevm_state_header(void) "" --=20 2.13.5 From nobody Sun Apr 28 04:06:08 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1508137678393554.4477025296644; Mon, 16 Oct 2017 00:07:58 -0700 (PDT) Received: from localhost ([::1]:59669 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1e3zV2-0004hC-RJ for importer@patchew.org; Mon, 16 Oct 2017 03:07:44 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:42700) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1e3zHR-0001Hj-UX for qemu-devel@nongnu.org; Mon, 16 Oct 2017 02:53:43 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1e3zHQ-0002XT-Mv for qemu-devel@nongnu.org; Mon, 16 Oct 2017 02:53:42 -0400 Received: from mx1.redhat.com ([209.132.183.28]:49252) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1e3zHQ-0002W7-FL for qemu-devel@nongnu.org; Mon, 16 Oct 2017 02:53:40 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 77C3B5AFED; Mon, 16 Oct 2017 06:53:39 +0000 (UTC) Received: from pxdev.xzpeter.org.com (dhcp-15-225.nay.redhat.com [10.66.15.225]) by smtp.corp.redhat.com (Postfix) with ESMTP id 3BE525EE02; Mon, 16 Oct 2017 06:53:37 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com 77C3B5AFED Authentication-Results: ext-mx10.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx10.extmail.prod.ext.phx2.redhat.com; spf=fail smtp.mailfrom=peterx@redhat.com From: Peter Xu To: qemu-devel@nongnu.org Date: Mon, 16 Oct 2017 14:52:02 +0800 Message-Id: <20171016065216.18162-19-peterx@redhat.com> In-Reply-To: <20171016065216.18162-1-peterx@redhat.com> References: <20171016065216.18162-1-peterx@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.39]); Mon, 16 Oct 2017 06:53:39 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH v3 18/32] migration: new message MIG_RP_MSG_RESUME_ACK X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Andrea Arcangeli , Juan Quintela , Alexey Perevalov , peterx@redhat.com, "Dr . David Alan Gilbert" Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Creating new message to reply for MIG_CMD_POSTCOPY_RESUME. One uint32_t is used as payload to let the source know whether destination is ready to continue the migration. Reviewed-by: Dr. David Alan Gilbert Signed-off-by: Peter Xu --- migration/migration.c | 37 +++++++++++++++++++++++++++++++++++++ migration/migration.h | 3 +++ migration/savevm.c | 3 ++- migration/trace-events | 1 + 4 files changed, 43 insertions(+), 1 deletion(-) diff --git a/migration/migration.c b/migration/migration.c index 2e992e891e..49117111ae 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -94,6 +94,7 @@ enum mig_rp_message_type { MIG_RP_MSG_REQ_PAGES_ID, /* data (start: be64, len: be32, id: string) = */ MIG_RP_MSG_REQ_PAGES, /* data (start: be64, len: be32) */ MIG_RP_MSG_RECV_BITMAP, /* send recved_bitmap back to source */ + MIG_RP_MSG_RESUME_ACK, /* tell source that we are ready to resume */ =20 MIG_RP_MSG_MAX }; @@ -539,6 +540,14 @@ void migrate_send_rp_recv_bitmap(MigrationIncomingStat= e *mis, trace_migrate_send_rp_recv_bitmap(block_name, res); } =20 +void migrate_send_rp_resume_ack(MigrationIncomingState *mis, uint32_t valu= e) +{ + uint32_t buf; + + buf =3D cpu_to_be32(value); + migrate_send_rp_message(mis, MIG_RP_MSG_RESUME_ACK, sizeof(buf), &buf); +} + MigrationCapabilityStatusList *qmp_query_migrate_capabilities(Error **errp) { MigrationCapabilityStatusList *head =3D NULL; @@ -1726,6 +1735,7 @@ static struct rp_cmd_args { [MIG_RP_MSG_REQ_PAGES] =3D { .len =3D 12, .name =3D "REQ_PAGES" }, [MIG_RP_MSG_REQ_PAGES_ID] =3D { .len =3D -1, .name =3D "REQ_PAGES_ID= " }, [MIG_RP_MSG_RECV_BITMAP] =3D { .len =3D -1, .name =3D "RECV_BITMAP"= }, + [MIG_RP_MSG_RESUME_ACK] =3D { .len =3D 4, .name =3D "RESUME_ACK" = }, [MIG_RP_MSG_MAX] =3D { .len =3D -1, .name =3D "MAX" }, }; =20 @@ -1783,6 +1793,25 @@ static int migrate_handle_rp_recv_bitmap(MigrationSt= ate *s, char *block_name) return ram_dirty_bitmap_reload(s, block); } =20 +static int migrate_handle_rp_resume_ack(MigrationState *s, uint32_t value) +{ + trace_source_return_path_thread_resume_ack(value); + + if (value !=3D MIGRATION_RESUME_ACK_VALUE) { + error_report("%s: illegal resume_ack value %"PRIu32, + __func__, value); + return -1; + } + + /* Now both sides are active. */ + migrate_set_state(&s->state, MIGRATION_STATUS_POSTCOPY_RECOVER, + MIGRATION_STATUS_POSTCOPY_ACTIVE); + + /* TODO: notify send thread that time to continue send pages */ + + return 0; +} + /* * Handles messages sent on the return path towards the source VM * @@ -1902,6 +1931,14 @@ retry: } break; =20 + case MIG_RP_MSG_RESUME_ACK: + tmp32 =3D ldl_be_p(buf); + if (migrate_handle_rp_resume_ack(ms, tmp32)) { + mark_source_rp_bad(ms); + goto out; + } + break; + default: break; } diff --git a/migration/migration.h b/migration/migration.h index 56ac50839e..feeb1997a2 100644 --- a/migration/migration.h +++ b/migration/migration.h @@ -22,6 +22,8 @@ #include "hw/qdev.h" #include "io/channel.h" =20 +#define MIGRATION_RESUME_ACK_VALUE (1) + /* State for the incoming migration */ struct MigrationIncomingState { QEMUFile *from_src_file; @@ -213,5 +215,6 @@ int migrate_send_rp_req_pages(MigrationIncomingState *m= is, const char* rbname, ram_addr_t start, size_t len); void migrate_send_rp_recv_bitmap(MigrationIncomingState *mis, char *block_name); +void migrate_send_rp_resume_ack(MigrationIncomingState *mis, uint32_t valu= e); =20 #endif diff --git a/migration/savevm.c b/migration/savevm.c index bb6639812b..611b3f1a09 100644 --- a/migration/savevm.c +++ b/migration/savevm.c @@ -1754,7 +1754,8 @@ static int loadvm_postcopy_handle_resume(MigrationInc= omingState *mis) =20 trace_loadvm_postcopy_handle_resume(); =20 - /* TODO: Tell source that "we are ready" */ + /* Tell source that "we are ready" */ + migrate_send_rp_resume_ack(mis, MIGRATION_RESUME_ACK_VALUE); =20 return 0; } diff --git a/migration/trace-events b/migration/trace-events index 4b60865194..2bf8301293 100644 --- a/migration/trace-events +++ b/migration/trace-events @@ -120,6 +120,7 @@ source_return_path_thread_entry(void) "" source_return_path_thread_loop_top(void) "" source_return_path_thread_pong(uint32_t val) "0x%x" source_return_path_thread_shut(uint32_t val) "0x%x" +source_return_path_thread_resume_ack(uint32_t v) "%"PRIu32 migrate_global_state_post_load(const char *state) "loaded state: %s" migrate_global_state_pre_save(const char *state) "saved state: %s" migration_thread_low_pending(uint64_t pending) "%" PRIu64 --=20 2.13.5 From nobody Sun Apr 28 04:06:08 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1508137679108478.3296423815142; Mon, 16 Oct 2017 00:07:59 -0700 (PDT) Received: from localhost ([::1]:59670 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1e3zVD-0004np-BX for importer@patchew.org; Mon, 16 Oct 2017 03:07:55 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:42735) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1e3zHU-0001KJ-R7 for qemu-devel@nongnu.org; Mon, 16 Oct 2017 02:53:45 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1e3zHT-0002b0-Kg for qemu-devel@nongnu.org; Mon, 16 Oct 2017 02:53:44 -0400 Received: from mx1.redhat.com ([209.132.183.28]:38512) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1e3zHT-0002Zm-CD for qemu-devel@nongnu.org; Mon, 16 Oct 2017 02:53:43 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 5854B81DEA; Mon, 16 Oct 2017 06:53:42 +0000 (UTC) Received: from pxdev.xzpeter.org.com (dhcp-15-225.nay.redhat.com [10.66.15.225]) by smtp.corp.redhat.com (Postfix) with ESMTP id EA1B462923; Mon, 16 Oct 2017 06:53:39 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com 5854B81DEA Authentication-Results: ext-mx01.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx01.extmail.prod.ext.phx2.redhat.com; spf=fail smtp.mailfrom=peterx@redhat.com From: Peter Xu To: qemu-devel@nongnu.org Date: Mon, 16 Oct 2017 14:52:03 +0800 Message-Id: <20171016065216.18162-20-peterx@redhat.com> In-Reply-To: <20171016065216.18162-1-peterx@redhat.com> References: <20171016065216.18162-1-peterx@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.25]); Mon, 16 Oct 2017 06:53:42 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH v3 19/32] migration: introduce SaveVMHandlers.resume_prepare X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Andrea Arcangeli , Juan Quintela , Alexey Perevalov , peterx@redhat.com, "Dr . David Alan Gilbert" Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" This is hook function to be called when a postcopy migration wants to resume from a failure. For each module, it should provide its own recovery logic before we switch to the postcopy-active state. Reviewed-by: Dr. David Alan Gilbert Signed-off-by: Peter Xu --- include/migration/register.h | 2 ++ migration/migration.c | 20 +++++++++++++++++++- migration/savevm.c | 25 +++++++++++++++++++++++++ migration/savevm.h | 1 + migration/trace-events | 1 + 5 files changed, 48 insertions(+), 1 deletion(-) diff --git a/include/migration/register.h b/include/migration/register.h index f4f7bdc177..128124f008 100644 --- a/include/migration/register.h +++ b/include/migration/register.h @@ -42,6 +42,8 @@ typedef struct SaveVMHandlers { LoadStateHandler *load_state; int (*load_setup)(QEMUFile *f, void *opaque); int (*load_cleanup)(void *opaque); + /* Called when postcopy migration wants to resume from failure */ + int (*resume_prepare)(MigrationState *s, void *opaque); } SaveVMHandlers; =20 int register_savevm_live(DeviceState *dev, diff --git a/migration/migration.c b/migration/migration.c index 49117111ae..3762e69eb3 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -2293,7 +2293,25 @@ typedef enum MigThrError { /* Return zero if success, or <0 for error */ static int postcopy_do_resume(MigrationState *s) { - /* TODO: do the resume logic */ + int ret; + + /* + * Call all the resume_prepare() hooks, so that modules can be + * ready for the migration resume. + */ + ret =3D qemu_savevm_state_resume_prepare(s); + if (ret) { + error_report("%s: resume_prepare() failure detected: %d", + __func__, ret); + return ret; + } + + /* + * TODO: handshake with dest using MIG_CMD_RESUME, + * MIG_RP_MSG_RESUME_ACK, then switch source state to + * "postcopy-active" + */ + return 0; } =20 diff --git a/migration/savevm.c b/migration/savevm.c index 611b3f1a09..bc87b0e5b1 100644 --- a/migration/savevm.c +++ b/migration/savevm.c @@ -1028,6 +1028,31 @@ void qemu_savevm_state_setup(QEMUFile *f) } } =20 +int qemu_savevm_state_resume_prepare(MigrationState *s) +{ + SaveStateEntry *se; + int ret; + + trace_savevm_state_resume_prepare(); + + QTAILQ_FOREACH(se, &savevm_state.handlers, entry) { + if (!se->ops || !se->ops->resume_prepare) { + continue; + } + if (se->ops && se->ops->is_active) { + if (!se->ops->is_active(se->opaque)) { + continue; + } + } + ret =3D se->ops->resume_prepare(s, se->opaque); + if (ret < 0) { + return ret; + } + } + + return 0; +} + /* * this function has three return values: * negative: there was one error, and we have -errno. diff --git a/migration/savevm.h b/migration/savevm.h index a5f3879191..3193f04cca 100644 --- a/migration/savevm.h +++ b/migration/savevm.h @@ -31,6 +31,7 @@ =20 bool qemu_savevm_state_blocked(Error **errp); void qemu_savevm_state_setup(QEMUFile *f); +int qemu_savevm_state_resume_prepare(MigrationState *s); void qemu_savevm_state_header(QEMUFile *f); int qemu_savevm_state_iterate(QEMUFile *f, bool postcopy); void qemu_savevm_state_cleanup(void); diff --git a/migration/trace-events b/migration/trace-events index 2bf8301293..eadabf03e8 100644 --- a/migration/trace-events +++ b/migration/trace-events @@ -39,6 +39,7 @@ savevm_send_postcopy_run(void) "" savevm_send_postcopy_resume(void) "" savevm_send_recv_bitmap(char *name) "%s" savevm_state_setup(void) "" +savevm_state_resume_prepare(void) "" savevm_state_header(void) "" savevm_state_iterate(void) "" savevm_state_cleanup(void) "" --=20 2.13.5 From nobody Sun Apr 28 04:06:08 2024 Delivered-To: importer@patchew.org Received-SPF: temperror (zoho.com: Error in retrieving data from DNS) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=temperror (zoho.com: Error in retrieving data from DNS) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (208.118.235.17 [208.118.235.17]) by mx.zohomail.com with SMTPS id 1508137840446141.9436635627045; Mon, 16 Oct 2017 00:10:40 -0700 (PDT) Received: from localhost ([::1]:59678 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1e3zXa-0006wq-Je for importer@patchew.org; Mon, 16 Oct 2017 03:10:22 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:42785) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1e3zHc-0001Pa-BA for qemu-devel@nongnu.org; Mon, 16 Oct 2017 02:53:53 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1e3zHZ-0002hy-EI for qemu-devel@nongnu.org; Mon, 16 Oct 2017 02:53:52 -0400 Received: from mx1.redhat.com ([209.132.183.28]:49408) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1e3zHZ-0002gc-61 for qemu-devel@nongnu.org; Mon, 16 Oct 2017 02:53:49 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 1D955356D9; Mon, 16 Oct 2017 06:53:48 +0000 (UTC) Received: from pxdev.xzpeter.org.com (dhcp-15-225.nay.redhat.com [10.66.15.225]) by smtp.corp.redhat.com (Postfix) with ESMTP id C83936269A; Mon, 16 Oct 2017 06:53:42 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com 1D955356D9 Authentication-Results: ext-mx06.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx06.extmail.prod.ext.phx2.redhat.com; spf=fail smtp.mailfrom=peterx@redhat.com From: Peter Xu To: qemu-devel@nongnu.org Date: Mon, 16 Oct 2017 14:52:04 +0800 Message-Id: <20171016065216.18162-21-peterx@redhat.com> In-Reply-To: <20171016065216.18162-1-peterx@redhat.com> References: <20171016065216.18162-1-peterx@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.30]); Mon, 16 Oct 2017 06:53:48 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH v3 20/32] migration: synchronize dirty bitmap for resume X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Andrea Arcangeli , Juan Quintela , Alexey Perevalov , peterx@redhat.com, "Dr . David Alan Gilbert" Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_6 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" This patch implements the first part of core RAM resume logic for postcopy. ram_resume_prepare() is provided for the work. When the migration is interrupted by network failure, the dirty bitmap on the source side will be meaningless, because even the dirty bit is cleared, it is still possible that the sent page was lost along the way to destination. Here instead of continue the migration with the old dirty bitmap on source, we ask the destination side to send back its received bitmap, then invert it to be our initial dirty bitmap. The source side send thread will issue the MIG_CMD_RECV_BITMAP requests, once per ramblock, to ask for the received bitmap. On destination side, MIG_RP_MSG_RECV_BITMAP will be issued, along with the requested bitmap. Data will be received on the return-path thread of source, and the main migration thread will be notified when all the ramblock bitmaps are synchronized. Signed-off-by: Peter Xu --- migration/migration.c | 4 ++++ migration/migration.h | 1 + migration/ram.c | 47 +++++++++++++++++++++++++++++++++++++++++++++++ migration/trace-events | 4 ++++ 4 files changed, 56 insertions(+) diff --git a/migration/migration.c b/migration/migration.c index 3762e69eb3..52f81c3add 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -2739,6 +2739,8 @@ static void migration_instance_finalize(Object *obj) =20 g_free(params->tls_hostname); g_free(params->tls_creds); + + qemu_sem_destroy(&ms->rp_state.rp_sem); } =20 static void migration_instance_init(Object *obj) @@ -2765,6 +2767,8 @@ static void migration_instance_init(Object *obj) params->has_block_incremental =3D true; params->has_x_multifd_channels =3D true; params->has_x_multifd_page_count =3D true; + + qemu_sem_init(&ms->rp_state.rp_sem, 0); } =20 /* diff --git a/migration/migration.h b/migration/migration.h index feeb1997a2..c8d5939d43 100644 --- a/migration/migration.h +++ b/migration/migration.h @@ -108,6 +108,7 @@ struct MigrationState QEMUFile *from_dst_file; QemuThread rp_thread; bool error; + QemuSemaphore rp_sem; } rp_state; =20 double mbps; diff --git a/migration/ram.c b/migration/ram.c index fa2b4ed207..a7431a802e 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -48,6 +48,7 @@ #include "qemu/rcu_queue.h" #include "migration/colo.h" #include "migration/block.h" +#include "savevm.h" =20 /***********************************************************/ /* ram save/restore */ @@ -2984,6 +2985,38 @@ static bool ram_has_postcopy(void *opaque) return migrate_postcopy_ram(); } =20 +/* Sync all the dirty bitmap with destination VM. */ +static int ram_dirty_bitmap_sync_all(MigrationState *s, RAMState *rs) +{ + RAMBlock *block; + QEMUFile *file =3D s->to_dst_file; + int ramblock_count =3D 0; + + trace_ram_dirty_bitmap_sync_start(); + + RAMBLOCK_FOREACH(block) { + qemu_savevm_send_recv_bitmap(file, block->idstr); + trace_ram_dirty_bitmap_request(block->idstr); + ramblock_count++; + } + + trace_ram_dirty_bitmap_sync_wait(); + + /* Wait until all the ramblocks' dirty bitmap synced */ + while (ramblock_count--) { + qemu_sem_wait(&s->rp_state.rp_sem); + } + + trace_ram_dirty_bitmap_sync_complete(); + + return 0; +} + +static void ram_dirty_bitmap_reload_notify(MigrationState *s) +{ + qemu_sem_post(&s->rp_state.rp_sem); +} + /* * Read the received bitmap, revert it as the initial dirty bitmap. * This is only used when the postcopy migration is paused but wants @@ -3058,12 +3091,25 @@ int ram_dirty_bitmap_reload(MigrationState *s, RAMB= lock *block) =20 trace_ram_dirty_bitmap_reload_complete(block->idstr); =20 + /* + * We succeeded to sync bitmap for current ramblock. If this is + * the last one to sync, we need to notify the main send thread. + */ + ram_dirty_bitmap_reload_notify(s); + ret =3D 0; out: free(le_bitmap); return ret; } =20 +static int ram_resume_prepare(MigrationState *s, void *opaque) +{ + RAMState *rs =3D *(RAMState **)opaque; + + return ram_dirty_bitmap_sync_all(s, rs); +} + static SaveVMHandlers savevm_ram_handlers =3D { .save_setup =3D ram_save_setup, .save_live_iterate =3D ram_save_iterate, @@ -3075,6 +3121,7 @@ static SaveVMHandlers savevm_ram_handlers =3D { .save_cleanup =3D ram_save_cleanup, .load_setup =3D ram_load_setup, .load_cleanup =3D ram_load_cleanup, + .resume_prepare =3D ram_resume_prepare, }; =20 void ram_mig_init(void) diff --git a/migration/trace-events b/migration/trace-events index eadabf03e8..804f18d492 100644 --- a/migration/trace-events +++ b/migration/trace-events @@ -82,8 +82,12 @@ ram_load_postcopy_loop(uint64_t addr, int flags) "@%" PR= Ix64 " %x" ram_postcopy_send_discard_bitmap(void) "" ram_save_page(const char *rbname, uint64_t offset, void *host) "%s: offset= : 0x%" PRIx64 " host: %p" ram_save_queue_pages(const char *rbname, size_t start, size_t len) "%s: st= art: 0x%zx len: 0x%zx" +ram_dirty_bitmap_request(char *str) "%s" ram_dirty_bitmap_reload_begin(char *str) "%s" ram_dirty_bitmap_reload_complete(char *str) "%s" +ram_dirty_bitmap_sync_start(void) "" +ram_dirty_bitmap_sync_wait(void) "" +ram_dirty_bitmap_sync_complete(void) "" =20 # migration/migration.c await_return_path_close_on_source_close(void) "" --=20 2.13.5 From nobody Sun Apr 28 04:06:08 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1508137978234521.5828674464724; Mon, 16 Oct 2017 00:12:58 -0700 (PDT) Received: from localhost ([::1]:59692 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1e3za5-000111-8t for importer@patchew.org; Mon, 16 Oct 2017 03:12:57 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:42791) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1e3zHc-0001Q6-St for qemu-devel@nongnu.org; Mon, 16 Oct 2017 02:53:53 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1e3zHb-0002kR-W0 for qemu-devel@nongnu.org; Mon, 16 Oct 2017 02:53:52 -0400 Received: from mx1.redhat.com ([209.132.183.28]:38692) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1e3zHb-0002ji-PA for qemu-devel@nongnu.org; Mon, 16 Oct 2017 02:53:51 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id C2C7781F01; Mon, 16 Oct 2017 06:53:50 +0000 (UTC) Received: from pxdev.xzpeter.org.com (dhcp-15-225.nay.redhat.com [10.66.15.225]) by smtp.corp.redhat.com (Postfix) with ESMTP id 8D6E55EE02; Mon, 16 Oct 2017 06:53:48 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com C2C7781F01 Authentication-Results: ext-mx01.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx01.extmail.prod.ext.phx2.redhat.com; spf=fail smtp.mailfrom=peterx@redhat.com From: Peter Xu To: qemu-devel@nongnu.org Date: Mon, 16 Oct 2017 14:52:05 +0800 Message-Id: <20171016065216.18162-22-peterx@redhat.com> In-Reply-To: <20171016065216.18162-1-peterx@redhat.com> References: <20171016065216.18162-1-peterx@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.25]); Mon, 16 Oct 2017 06:53:50 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH v3 21/32] migration: setup ramstate for resume X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Andrea Arcangeli , Juan Quintela , Alexey Perevalov , peterx@redhat.com, "Dr . David Alan Gilbert" Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" After we updated the dirty bitmaps of ramblocks, we also need to update the critical fields in RAMState to make sure it is ready for a resume. Reviewed-by: Dr. David Alan Gilbert Signed-off-by: Peter Xu --- migration/ram.c | 37 ++++++++++++++++++++++++++++++++++++- migration/trace-events | 1 + 2 files changed, 37 insertions(+), 1 deletion(-) diff --git a/migration/ram.c b/migration/ram.c index a7431a802e..79c52631b9 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -2185,6 +2185,33 @@ static int ram_state_init(RAMState **rsp) return 0; } =20 +static void ram_state_resume_prepare(RAMState *rs) +{ + RAMBlock *block; + long pages =3D 0; + + /* + * Postcopy is not using xbzrle/compression, so no need for that. + * Also, since source are already halted, we don't need to care + * about dirty page logging as well. + */ + + RAMBLOCK_FOREACH(block) { + pages +=3D bitmap_count_one(block->bmap, + block->used_length >> TARGET_PAGE_BITS); + } + + /* This may not be aligned with current bitmaps. Recalculate. */ + rs->migration_dirty_pages =3D pages; + + rs->last_seen_block =3D NULL; + rs->last_sent_block =3D NULL; + rs->last_page =3D 0; + rs->last_version =3D ram_list.version; + + trace_ram_state_resume_prepare(pages); +} + /* * Each of ram_save_setup, ram_save_iterate and ram_save_complete has * long-running RCU critical section. When rcu-reclaims in the code @@ -3106,8 +3133,16 @@ out: static int ram_resume_prepare(MigrationState *s, void *opaque) { RAMState *rs =3D *(RAMState **)opaque; + int ret; =20 - return ram_dirty_bitmap_sync_all(s, rs); + ret =3D ram_dirty_bitmap_sync_all(s, rs); + if (ret) { + return ret; + } + + ram_state_resume_prepare(rs); + + return 0; } =20 static SaveVMHandlers savevm_ram_handlers =3D { diff --git a/migration/trace-events b/migration/trace-events index 804f18d492..98c2e4de58 100644 --- a/migration/trace-events +++ b/migration/trace-events @@ -88,6 +88,7 @@ ram_dirty_bitmap_reload_complete(char *str) "%s" ram_dirty_bitmap_sync_start(void) "" ram_dirty_bitmap_sync_wait(void) "" ram_dirty_bitmap_sync_complete(void) "" +ram_state_resume_prepare(long v) "%ld" =20 # migration/migration.c await_return_path_close_on_source_close(void) "" --=20 2.13.5 From nobody Sun Apr 28 04:06:08 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1508137734070912.3641383406709; Mon, 16 Oct 2017 00:08:54 -0700 (PDT) Received: from localhost ([::1]:59672 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1e3zW0-0005bP-4F for importer@patchew.org; Mon, 16 Oct 2017 03:08:44 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:42813) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1e3zHf-0001Tz-MN for qemu-devel@nongnu.org; Mon, 16 Oct 2017 02:53:56 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1e3zHe-0002p5-Vd for qemu-devel@nongnu.org; Mon, 16 Oct 2017 02:53:55 -0400 Received: from mx1.redhat.com ([209.132.183.28]:32948) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1e3zHe-0002o4-PT for qemu-devel@nongnu.org; Mon, 16 Oct 2017 02:53:54 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 98CA34E909; Mon, 16 Oct 2017 06:53:53 +0000 (UTC) Received: from pxdev.xzpeter.org.com (dhcp-15-225.nay.redhat.com [10.66.15.225]) by smtp.corp.redhat.com (Postfix) with ESMTP id 3F22A5EE02; Mon, 16 Oct 2017 06:53:51 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com 98CA34E909 Authentication-Results: ext-mx09.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx09.extmail.prod.ext.phx2.redhat.com; spf=fail smtp.mailfrom=peterx@redhat.com From: Peter Xu To: qemu-devel@nongnu.org Date: Mon, 16 Oct 2017 14:52:06 +0800 Message-Id: <20171016065216.18162-23-peterx@redhat.com> In-Reply-To: <20171016065216.18162-1-peterx@redhat.com> References: <20171016065216.18162-1-peterx@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.38]); Mon, 16 Oct 2017 06:53:53 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH v3 22/32] migration: final handshake for the resume X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Andrea Arcangeli , Juan Quintela , Alexey Perevalov , peterx@redhat.com, "Dr . David Alan Gilbert" Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Finish the last step to do the final handshake for the recovery. First source sends one MIG_CMD_RESUME to dst, telling that source is ready to resume. Then, dest replies with MIG_RP_MSG_RESUME_ACK to source, telling that dest is ready to resume (after switch to postcopy-active state). When source received the RESUME_ACK, it switches its state to postcopy-active, and finally the recovery is completed. Reviewed-by: Dr. David Alan Gilbert Signed-off-by: Peter Xu --- migration/migration.c | 28 ++++++++++++++++++++++++---- 1 file changed, 24 insertions(+), 4 deletions(-) diff --git a/migration/migration.c b/migration/migration.c index 52f81c3add..edf7365b69 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -1807,7 +1807,8 @@ static int migrate_handle_rp_resume_ack(MigrationStat= e *s, uint32_t value) migrate_set_state(&s->state, MIGRATION_STATUS_POSTCOPY_RECOVER, MIGRATION_STATUS_POSTCOPY_ACTIVE); =20 - /* TODO: notify send thread that time to continue send pages */ + /* Notify send thread that time to continue send pages */ + qemu_sem_post(&s->rp_state.rp_sem); =20 return 0; } @@ -2290,6 +2291,21 @@ typedef enum MigThrError { MIG_THR_ERR_FATAL =3D 2, } MigThrError; =20 +static int postcopy_resume_handshake(MigrationState *s) +{ + qemu_savevm_send_postcopy_resume(s->to_dst_file); + + while (s->state =3D=3D MIGRATION_STATUS_POSTCOPY_RECOVER) { + qemu_sem_wait(&s->rp_state.rp_sem); + } + + if (s->state =3D=3D MIGRATION_STATUS_POSTCOPY_ACTIVE) { + return 0; + } + + return -1; +} + /* Return zero if success, or <0 for error */ static int postcopy_do_resume(MigrationState *s) { @@ -2307,10 +2323,14 @@ static int postcopy_do_resume(MigrationState *s) } =20 /* - * TODO: handshake with dest using MIG_CMD_RESUME, - * MIG_RP_MSG_RESUME_ACK, then switch source state to - * "postcopy-active" + * Last handshake with destination on the resume (destination will + * switch to postcopy-active afterwards) */ + ret =3D postcopy_resume_handshake(s); + if (ret) { + error_report("%s: handshake failed: %d", __func__, ret); + return ret; + } =20 return 0; } --=20 2.13.5 From nobody Sun Apr 28 04:06:08 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1508137887485795.0372851491204; Mon, 16 Oct 2017 00:11:27 -0700 (PDT) Received: from localhost ([::1]:59686 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1e3zYY-0007qh-HE for importer@patchew.org; Mon, 16 Oct 2017 03:11:22 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:42865) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1e3zHl-0001ZD-3D for qemu-devel@nongnu.org; Mon, 16 Oct 2017 02:54:02 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1e3zHi-0002s8-0i for qemu-devel@nongnu.org; Mon, 16 Oct 2017 02:54:01 -0400 Received: from mx1.redhat.com ([209.132.183.28]:50978) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1e3zHh-0002rJ-Qw for qemu-devel@nongnu.org; Mon, 16 Oct 2017 02:53:57 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id D1CCDC057F91; Mon, 16 Oct 2017 06:53:56 +0000 (UTC) Received: from pxdev.xzpeter.org.com (dhcp-15-225.nay.redhat.com [10.66.15.225]) by smtp.corp.redhat.com (Postfix) with ESMTP id 153575EE02; Mon, 16 Oct 2017 06:53:53 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com D1CCDC057F91 Authentication-Results: ext-mx08.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx08.extmail.prod.ext.phx2.redhat.com; spf=fail smtp.mailfrom=peterx@redhat.com From: Peter Xu To: qemu-devel@nongnu.org Date: Mon, 16 Oct 2017 14:52:07 +0800 Message-Id: <20171016065216.18162-24-peterx@redhat.com> In-Reply-To: <20171016065216.18162-1-peterx@redhat.com> References: <20171016065216.18162-1-peterx@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.32]); Mon, 16 Oct 2017 06:53:57 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH v3 23/32] migration: free SocketAddress where allocated X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Andrea Arcangeli , Juan Quintela , Alexey Perevalov , peterx@redhat.com, "Dr . David Alan Gilbert" Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Freeing the SocketAddress struct in socket_start_incoming_migration is slightly confusing. Let's free the address in the same context where we allocated it. Reviewed-by: Dr. David Alan Gilbert Signed-off-by: Peter Xu --- migration/socket.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/migration/socket.c b/migration/socket.c index dee869044a..4879f11e0f 100644 --- a/migration/socket.c +++ b/migration/socket.c @@ -172,7 +172,6 @@ static void socket_start_incoming_migration(SocketAddre= ss *saddr, =20 if (qio_channel_socket_listen_sync(listen_ioc, saddr, errp) < 0) { object_unref(OBJECT(listen_ioc)); - qapi_free_SocketAddress(saddr); return; } =20 @@ -181,7 +180,6 @@ static void socket_start_incoming_migration(SocketAddre= ss *saddr, socket_accept_incoming_migration, listen_ioc, (GDestroyNotify)object_unref); - qapi_free_SocketAddress(saddr); } =20 void tcp_start_incoming_migration(const char *host_port, Error **errp) @@ -192,10 +190,12 @@ void tcp_start_incoming_migration(const char *host_po= rt, Error **errp) socket_start_incoming_migration(saddr, &err); } error_propagate(errp, err); + qapi_free_SocketAddress(saddr); } =20 void unix_start_incoming_migration(const char *path, Error **errp) { SocketAddress *saddr =3D unix_build_address(path); socket_start_incoming_migration(saddr, errp); + qapi_free_SocketAddress(saddr); } --=20 2.13.5 From nobody Sun Apr 28 04:06:08 2024 Delivered-To: importer@patchew.org Received-SPF: temperror (zoho.com: Error in retrieving data from DNS) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=temperror (zoho.com: Error in retrieving data from DNS) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1508137362095345.1465380377035; Mon, 16 Oct 2017 00:02:42 -0700 (PDT) Received: from localhost ([::1]:59649 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1e3zPx-0000PB-8G for importer@patchew.org; Mon, 16 Oct 2017 03:02:29 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:42886) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1e3zHm-0001aN-D9 for qemu-devel@nongnu.org; Mon, 16 Oct 2017 02:54:03 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1e3zHl-0002ua-DK for qemu-devel@nongnu.org; Mon, 16 Oct 2017 02:54:02 -0400 Received: from mx1.redhat.com ([209.132.183.28]:53574) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1e3zHl-0002tm-5C for qemu-devel@nongnu.org; Mon, 16 Oct 2017 02:54:01 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 141057E423; Mon, 16 Oct 2017 06:54:00 +0000 (UTC) Received: from pxdev.xzpeter.org.com (dhcp-15-225.nay.redhat.com [10.66.15.225]) by smtp.corp.redhat.com (Postfix) with ESMTP id 4DAE662923; Mon, 16 Oct 2017 06:53:57 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com 141057E423 Authentication-Results: ext-mx03.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx03.extmail.prod.ext.phx2.redhat.com; spf=fail smtp.mailfrom=peterx@redhat.com From: Peter Xu To: qemu-devel@nongnu.org Date: Mon, 16 Oct 2017 14:52:08 +0800 Message-Id: <20171016065216.18162-25-peterx@redhat.com> In-Reply-To: <20171016065216.18162-1-peterx@redhat.com> References: <20171016065216.18162-1-peterx@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.27]); Mon, 16 Oct 2017 06:54:00 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH v3 24/32] migration: return incoming task tag for sockets X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Andrea Arcangeli , Juan Quintela , Alexey Perevalov , peterx@redhat.com, "Dr . David Alan Gilbert" Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_6 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" For socket based incoming migration, we attached a background task onto main loop to handle the acception of connections. We never had a way to destroy it before, only if we finished the migration. Let's allow socket_start_incoming_migration() to return the source tag of the listening async work, so that we may be able to clean it up in the future. Reviewed-by: Dr. David Alan Gilbert Signed-off-by: Peter Xu --- migration/socket.c | 36 ++++++++++++++++++++++++------------ migration/socket.h | 4 ++-- 2 files changed, 26 insertions(+), 14 deletions(-) diff --git a/migration/socket.c b/migration/socket.c index 4879f11e0f..e8f3325155 100644 --- a/migration/socket.c +++ b/migration/socket.c @@ -162,8 +162,12 @@ out: } =20 =20 -static void socket_start_incoming_migration(SocketAddress *saddr, - Error **errp) +/* + * Returns the tag ID of the watch that is attached to global main + * loop (>0), or zero if failure detected. + */ +static guint socket_start_incoming_migration(SocketAddress *saddr, + Error **errp) { QIOChannelSocket *listen_ioc =3D qio_channel_socket_new(); =20 @@ -172,30 +176,38 @@ static void socket_start_incoming_migration(SocketAdd= ress *saddr, =20 if (qio_channel_socket_listen_sync(listen_ioc, saddr, errp) < 0) { object_unref(OBJECT(listen_ioc)); - return; + return 0; } =20 - qio_channel_add_watch(QIO_CHANNEL(listen_ioc), - G_IO_IN, - socket_accept_incoming_migration, - listen_ioc, - (GDestroyNotify)object_unref); + return qio_channel_add_watch(QIO_CHANNEL(listen_ioc), + G_IO_IN, + socket_accept_incoming_migration, + listen_ioc, + (GDestroyNotify)object_unref); } =20 -void tcp_start_incoming_migration(const char *host_port, Error **errp) +guint tcp_start_incoming_migration(const char *host_port, Error **errp) { Error *err =3D NULL; SocketAddress *saddr =3D tcp_build_address(host_port, &err); + guint tag =3D 0; + if (!err) { - socket_start_incoming_migration(saddr, &err); + tag =3D socket_start_incoming_migration(saddr, &err); } error_propagate(errp, err); qapi_free_SocketAddress(saddr); + + return tag; } =20 -void unix_start_incoming_migration(const char *path, Error **errp) +guint unix_start_incoming_migration(const char *path, Error **errp) { SocketAddress *saddr =3D unix_build_address(path); - socket_start_incoming_migration(saddr, errp); + guint tag; + + tag =3D socket_start_incoming_migration(saddr, errp); qapi_free_SocketAddress(saddr); + + return tag; } diff --git a/migration/socket.h b/migration/socket.h index 6b91e9db38..bc8a59aee4 100644 --- a/migration/socket.h +++ b/migration/socket.h @@ -16,12 +16,12 @@ =20 #ifndef QEMU_MIGRATION_SOCKET_H #define QEMU_MIGRATION_SOCKET_H -void tcp_start_incoming_migration(const char *host_port, Error **errp); +guint tcp_start_incoming_migration(const char *host_port, Error **errp); =20 void tcp_start_outgoing_migration(MigrationState *s, const char *host_port, Error **errp); =20 -void unix_start_incoming_migration(const char *path, Error **errp); +guint unix_start_incoming_migration(const char *path, Error **errp); =20 void unix_start_outgoing_migration(MigrationState *s, const char *path, Error **errp); --=20 2.13.5 From nobody Sun Apr 28 04:06:08 2024 Delivered-To: importer@patchew.org Received-SPF: temperror (zoho.com: Error in retrieving data from DNS) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=temperror (zoho.com: Error in retrieving data from DNS) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1508137840391316.3235087376197; Mon, 16 Oct 2017 00:10:40 -0700 (PDT) Received: from localhost ([::1]:59679 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1e3zXh-000750-Kb for importer@patchew.org; Mon, 16 Oct 2017 03:10:29 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:42905) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1e3zHo-0001c8-S7 for qemu-devel@nongnu.org; Mon, 16 Oct 2017 02:54:05 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1e3zHo-0002xS-4T for qemu-devel@nongnu.org; Mon, 16 Oct 2017 02:54:04 -0400 Received: from mx1.redhat.com ([209.132.183.28]:33150) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1e3zHn-0002vt-Us for qemu-devel@nongnu.org; Mon, 16 Oct 2017 02:54:04 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id DF33D4E334; Mon, 16 Oct 2017 06:54:02 +0000 (UTC) Received: from pxdev.xzpeter.org.com (dhcp-15-225.nay.redhat.com [10.66.15.225]) by smtp.corp.redhat.com (Postfix) with ESMTP id 834485EE02; Mon, 16 Oct 2017 06:54:00 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com DF33D4E334 Authentication-Results: ext-mx09.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx09.extmail.prod.ext.phx2.redhat.com; spf=fail smtp.mailfrom=peterx@redhat.com From: Peter Xu To: qemu-devel@nongnu.org Date: Mon, 16 Oct 2017 14:52:09 +0800 Message-Id: <20171016065216.18162-26-peterx@redhat.com> In-Reply-To: <20171016065216.18162-1-peterx@redhat.com> References: <20171016065216.18162-1-peterx@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.38]); Mon, 16 Oct 2017 06:54:03 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH v3 25/32] migration: return incoming task tag for exec X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Andrea Arcangeli , Juan Quintela , Alexey Perevalov , peterx@redhat.com, "Dr . David Alan Gilbert" Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_6 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Return the async task tag for exec typed incoming migration in exec_start_incoming_migration(). Reviewed-by: Dr. David Alan Gilbert Signed-off-by: Peter Xu --- migration/exec.c | 18 +++++++++++------- migration/exec.h | 2 +- 2 files changed, 12 insertions(+), 8 deletions(-) diff --git a/migration/exec.c b/migration/exec.c index f3be1baf2e..a0796c2c70 100644 --- a/migration/exec.c +++ b/migration/exec.c @@ -52,7 +52,11 @@ static gboolean exec_accept_incoming_migration(QIOChanne= l *ioc, return G_SOURCE_REMOVE; } =20 -void exec_start_incoming_migration(const char *command, Error **errp) +/* + * Returns the tag ID of the watch that is attached to global main + * loop (>0), or zero if failure detected. + */ +guint exec_start_incoming_migration(const char *command, Error **errp) { QIOChannel *ioc; const char *argv[] =3D { "/bin/sh", "-c", command, NULL }; @@ -62,13 +66,13 @@ void exec_start_incoming_migration(const char *command,= Error **errp) O_RDWR, errp)); if (!ioc) { - return; + return 0; } =20 qio_channel_set_name(ioc, "migration-exec-incoming"); - qio_channel_add_watch(ioc, - G_IO_IN, - exec_accept_incoming_migration, - NULL, - NULL); + return qio_channel_add_watch(ioc, + G_IO_IN, + exec_accept_incoming_migration, + NULL, + NULL); } diff --git a/migration/exec.h b/migration/exec.h index b210ffde7a..0a7aadacd3 100644 --- a/migration/exec.h +++ b/migration/exec.h @@ -19,7 +19,7 @@ =20 #ifndef QEMU_MIGRATION_EXEC_H #define QEMU_MIGRATION_EXEC_H -void exec_start_incoming_migration(const char *host_port, Error **errp); +guint exec_start_incoming_migration(const char *host_port, Error **errp); =20 void exec_start_outgoing_migration(MigrationState *s, const char *host_por= t, Error **errp); --=20 2.13.5 From nobody Sun Apr 28 04:06:08 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1508137984032811.7948515907058; Mon, 16 Oct 2017 00:13:04 -0700 (PDT) Received: from localhost ([::1]:59693 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1e3zaB-000174-9u for importer@patchew.org; Mon, 16 Oct 2017 03:13:03 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:42940) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1e3zHx-0001i2-4x for qemu-devel@nongnu.org; Mon, 16 Oct 2017 02:54:14 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1e3zHt-00037A-UD for qemu-devel@nongnu.org; Mon, 16 Oct 2017 02:54:13 -0400 Received: from mx1.redhat.com ([209.132.183.28]:52296) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1e3zHt-000363-OE for qemu-devel@nongnu.org; Mon, 16 Oct 2017 02:54:09 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id BBA61C04AC58; Mon, 16 Oct 2017 06:54:08 +0000 (UTC) Received: from pxdev.xzpeter.org.com (dhcp-15-225.nay.redhat.com [10.66.15.225]) by smtp.corp.redhat.com (Postfix) with ESMTP id 59CFB5EE02; Mon, 16 Oct 2017 06:54:03 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com BBA61C04AC58 Authentication-Results: ext-mx07.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx07.extmail.prod.ext.phx2.redhat.com; spf=fail smtp.mailfrom=peterx@redhat.com From: Peter Xu To: qemu-devel@nongnu.org Date: Mon, 16 Oct 2017 14:52:10 +0800 Message-Id: <20171016065216.18162-27-peterx@redhat.com> In-Reply-To: <20171016065216.18162-1-peterx@redhat.com> References: <20171016065216.18162-1-peterx@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.31]); Mon, 16 Oct 2017 06:54:08 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH v3 26/32] migration: return incoming task tag for fd X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Andrea Arcangeli , Juan Quintela , Alexey Perevalov , peterx@redhat.com, "Dr . David Alan Gilbert" Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Allow to return the task tag in fd_start_incoming_migration(). Reviewed-by: Dr. David Alan Gilbert Signed-off-by: Peter Xu --- migration/fd.c | 18 +++++++++++------- migration/fd.h | 2 +- 2 files changed, 12 insertions(+), 8 deletions(-) diff --git a/migration/fd.c b/migration/fd.c index 30de4b9847..7ead2f26cc 100644 --- a/migration/fd.c +++ b/migration/fd.c @@ -52,7 +52,11 @@ static gboolean fd_accept_incoming_migration(QIOChannel = *ioc, return G_SOURCE_REMOVE; } =20 -void fd_start_incoming_migration(const char *infd, Error **errp) +/* + * Returns the tag ID of the watch that is attached to global main + * loop (>0), or zero if failure detected. + */ +guint fd_start_incoming_migration(const char *infd, Error **errp) { QIOChannel *ioc; int fd; @@ -63,13 +67,13 @@ void fd_start_incoming_migration(const char *infd, Erro= r **errp) ioc =3D qio_channel_new_fd(fd, errp); if (!ioc) { close(fd); - return; + return 0; } =20 qio_channel_set_name(QIO_CHANNEL(ioc), "migration-fd-incoming"); - qio_channel_add_watch(ioc, - G_IO_IN, - fd_accept_incoming_migration, - NULL, - NULL); + return qio_channel_add_watch(ioc, + G_IO_IN, + fd_accept_incoming_migration, + NULL, + NULL); } diff --git a/migration/fd.h b/migration/fd.h index a14a63ce2e..94cdea87d8 100644 --- a/migration/fd.h +++ b/migration/fd.h @@ -16,7 +16,7 @@ =20 #ifndef QEMU_MIGRATION_FD_H #define QEMU_MIGRATION_FD_H -void fd_start_incoming_migration(const char *path, Error **errp); +guint fd_start_incoming_migration(const char *path, Error **errp); =20 void fd_start_outgoing_migration(MigrationState *s, const char *fdname, Error **errp); --=20 2.13.5 From nobody Sun Apr 28 04:06:08 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1508138116460208.7538541165377; Mon, 16 Oct 2017 00:15:16 -0700 (PDT) Received: from localhost ([::1]:59706 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1e3zcH-0002xD-Nu for importer@patchew.org; Mon, 16 Oct 2017 03:15:13 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:42952) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1e3zHy-0001jM-RJ for qemu-devel@nongnu.org; Mon, 16 Oct 2017 02:54:15 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1e3zHx-0003C2-Qj for qemu-devel@nongnu.org; Mon, 16 Oct 2017 02:54:14 -0400 Received: from mx1.redhat.com ([209.132.183.28]:33330) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1e3zHx-0003Ap-IM for qemu-devel@nongnu.org; Mon, 16 Oct 2017 02:54:13 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 8F8264E909; Mon, 16 Oct 2017 06:54:12 +0000 (UTC) Received: from pxdev.xzpeter.org.com (dhcp-15-225.nay.redhat.com [10.66.15.225]) by smtp.corp.redhat.com (Postfix) with ESMTP id 384CF62923; Mon, 16 Oct 2017 06:54:08 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com 8F8264E909 Authentication-Results: ext-mx09.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx09.extmail.prod.ext.phx2.redhat.com; spf=fail smtp.mailfrom=peterx@redhat.com From: Peter Xu To: qemu-devel@nongnu.org Date: Mon, 16 Oct 2017 14:52:11 +0800 Message-Id: <20171016065216.18162-28-peterx@redhat.com> In-Reply-To: <20171016065216.18162-1-peterx@redhat.com> References: <20171016065216.18162-1-peterx@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.38]); Mon, 16 Oct 2017 06:54:12 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH v3 27/32] migration: store listen task tag X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Andrea Arcangeli , Juan Quintela , Alexey Perevalov , peterx@redhat.com, "Dr . David Alan Gilbert" Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Store the task tag for migration types: tcp/unix/fd/exec in current MigrationIncomingState struct. For defered migration, no need to store task tag since there is no task running in the main loop at all. For RDMA, let's mark it as todo. Reviewed-by: Dr. David Alan Gilbert Signed-off-by: Peter Xu --- migration/migration.c | 22 ++++++++++++++++++---- migration/migration.h | 2 ++ 2 files changed, 20 insertions(+), 4 deletions(-) diff --git a/migration/migration.c b/migration/migration.c index edf7365b69..0baa09ada3 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -172,6 +172,7 @@ void migration_incoming_state_destroy(void) mis->from_src_file =3D NULL; } =20 + mis->listen_task_tag =3D 0; qemu_event_reset(&mis->main_thread_load_event); } =20 @@ -266,25 +267,31 @@ int migrate_send_rp_req_pages(MigrationIncomingState = *mis, const char *rbname, void qemu_start_incoming_migration(const char *uri, Error **errp) { const char *p; + guint task_tag =3D 0; + MigrationIncomingState *mis =3D migration_incoming_get_current(); =20 qapi_event_send_migration(MIGRATION_STATUS_SETUP, &error_abort); if (!strcmp(uri, "defer")) { deferred_incoming_migration(errp); } else if (strstart(uri, "tcp:", &p)) { - tcp_start_incoming_migration(p, errp); + task_tag =3D tcp_start_incoming_migration(p, errp); #ifdef CONFIG_RDMA } else if (strstart(uri, "rdma:", &p)) { + /* TODO: store task tag for RDMA migrations */ rdma_start_incoming_migration(p, errp); #endif } else if (strstart(uri, "exec:", &p)) { - exec_start_incoming_migration(p, errp); + task_tag =3D exec_start_incoming_migration(p, errp); } else if (strstart(uri, "unix:", &p)) { - unix_start_incoming_migration(p, errp); + task_tag =3D unix_start_incoming_migration(p, errp); } else if (strstart(uri, "fd:", &p)) { - fd_start_incoming_migration(p, errp); + task_tag =3D fd_start_incoming_migration(p, errp); } else { error_setg(errp, "unknown migration protocol: %s", uri); + return; } + + mis->listen_task_tag =3D task_tag; } =20 static void process_incoming_migration_bh(void *opaque) @@ -450,6 +457,13 @@ void migration_fd_process_incoming(QEMUFile *f) migration_incoming_setup(f); migration_incoming_process(); } + + /* + * When reach here, we should not need the listening port any + * more. We'll detach the listening task soon, let's reset the + * listen task tag. + */ + mis->listen_task_tag =3D 0; } =20 void migration_ioc_process_incoming(QIOChannel *ioc) diff --git a/migration/migration.h b/migration/migration.h index c8d5939d43..d89a0387cf 100644 --- a/migration/migration.h +++ b/migration/migration.h @@ -27,6 +27,8 @@ /* State for the incoming migration */ struct MigrationIncomingState { QEMUFile *from_src_file; + /* Task tag for incoming listen port. Valid when >0. */ + guint listen_task_tag; =20 /* * Free at the start of the main state load, set as the main thread fi= nishes --=20 2.13.5 From nobody Sun Apr 28 04:06:08 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1508137521703687.7880255251915; Mon, 16 Oct 2017 00:05:21 -0700 (PDT) Received: from localhost ([::1]:59657 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1e3zSf-0002X3-Uc for importer@patchew.org; Mon, 16 Oct 2017 03:05:17 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:42964) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1e3zI1-0001lz-LD for qemu-devel@nongnu.org; Mon, 16 Oct 2017 02:54:19 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1e3zI0-0003G1-HP for qemu-devel@nongnu.org; Mon, 16 Oct 2017 02:54:17 -0400 Received: from mx1.redhat.com ([209.132.183.28]:54042) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1e3zI0-0003F7-8k for qemu-devel@nongnu.org; Mon, 16 Oct 2017 02:54:16 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 494F57E423; Mon, 16 Oct 2017 06:54:15 +0000 (UTC) Received: from pxdev.xzpeter.org.com (dhcp-15-225.nay.redhat.com [10.66.15.225]) by smtp.corp.redhat.com (Postfix) with ESMTP id 0BBFC6292A; Mon, 16 Oct 2017 06:54:12 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com 494F57E423 Authentication-Results: ext-mx03.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx03.extmail.prod.ext.phx2.redhat.com; spf=fail smtp.mailfrom=peterx@redhat.com From: Peter Xu To: qemu-devel@nongnu.org Date: Mon, 16 Oct 2017 14:52:12 +0800 Message-Id: <20171016065216.18162-29-peterx@redhat.com> In-Reply-To: <20171016065216.18162-1-peterx@redhat.com> References: <20171016065216.18162-1-peterx@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.27]); Mon, 16 Oct 2017 06:54:15 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH v3 28/32] migration: allow migrate_incoming for paused VM X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Andrea Arcangeli , Juan Quintela , Alexey Perevalov , peterx@redhat.com, "Dr . David Alan Gilbert" Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" migrate_incoming command is previously only used when we were providing "-incoming defer" in the command line, to defer the incoming migration channel creation. However there is similar requirement when we are paused during postcopy migration. The old incoming channel might have been destroyed already. We may need another new channel for the recovery to happen. This patch leveraged the same interface, but allows the user to specify incoming migration channel even for paused postcopy. Meanwhile, now migration listening ports are always detached manually using the tag, rather than using return values of dispatchers. Signed-off-by: Peter Xu --- migration/exec.c | 2 +- migration/fd.c | 2 +- migration/migration.c | 50 +++++++++++++++++++++++++++++++++++++++++-----= ---- migration/socket.c | 4 +--- migration/trace-events | 2 ++ 5 files changed, 46 insertions(+), 14 deletions(-) diff --git a/migration/exec.c b/migration/exec.c index a0796c2c70..9d20d10899 100644 --- a/migration/exec.c +++ b/migration/exec.c @@ -49,7 +49,7 @@ static gboolean exec_accept_incoming_migration(QIOChannel= *ioc, { migration_channel_process_incoming(ioc); object_unref(OBJECT(ioc)); - return G_SOURCE_REMOVE; + return G_SOURCE_CONTINUE; } =20 /* diff --git a/migration/fd.c b/migration/fd.c index 7ead2f26cc..54b36888e2 100644 --- a/migration/fd.c +++ b/migration/fd.c @@ -49,7 +49,7 @@ static gboolean fd_accept_incoming_migration(QIOChannel *= ioc, { migration_channel_process_incoming(ioc); object_unref(OBJECT(ioc)); - return G_SOURCE_REMOVE; + return G_SOURCE_CONTINUE; } =20 /* diff --git a/migration/migration.c b/migration/migration.c index 0baa09ada3..387fbefad4 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -176,6 +176,17 @@ void migration_incoming_state_destroy(void) qemu_event_reset(&mis->main_thread_load_event); } =20 +static bool migrate_incoming_detach_listen(MigrationIncomingState *mis) +{ + if (mis->listen_task_tag) { + /* Never fail */ + g_source_remove(mis->listen_task_tag); + mis->listen_task_tag =3D 0; + return true; + } + return false; +} + static void migrate_generate_event(int new_state) { if (migrate_use_events()) { @@ -460,10 +471,9 @@ void migration_fd_process_incoming(QEMUFile *f) =20 /* * When reach here, we should not need the listening port any - * more. We'll detach the listening task soon, let's reset the - * listen task tag. + * more. Detach the listening port explicitly. */ - mis->listen_task_tag =3D 0; + migrate_incoming_detach_listen(mis); } =20 void migration_ioc_process_incoming(QIOChannel *ioc) @@ -1373,13 +1383,35 @@ void qmp_migrate_incoming(const char *uri, Error **= errp) { Error *local_err =3D NULL; static bool once =3D true; + MigrationIncomingState *mis =3D migration_incoming_get_current(); =20 - if (!deferred_incoming) { - error_setg(errp, "For use with '-incoming defer'"); - return; - } - if (!once) { - error_setg(errp, "The incoming migration has already been started"= ); + if (mis->state =3D=3D MIGRATION_STATUS_POSTCOPY_PAUSED && + mis->listen_task_tag =3D=3D 0) { + /* + * We are in postcopy-paused state, and we don't have + * listening port. It's very possible that the old listening + * port is already gone, so we allow to create a new one. + * + * NOTE: RDMA migration currently does not really use + * listen_task_tag for now, so even if listen_task_tag is + * zero, RDMA can still have its accept port listening. + * However, RDMA is not supported by postcopy at all (yet), so + * we are safe here. + */ + trace_migrate_incoming_recover(); + } else if (deferred_incoming) { + /* + * We don't need recovery, but we possibly has a deferred + * incoming parameter, this allows us to manually specify + * incoming port once. + */ + if (!once) { + error_setg(errp, "The incoming migration has already been star= ted"); + return; + } else { + /* PASS */ + trace_migrate_incoming_deferred(); + } } =20 qemu_start_incoming_migration(uri, &local_err); diff --git a/migration/socket.c b/migration/socket.c index e8f3325155..54095a80a0 100644 --- a/migration/socket.c +++ b/migration/socket.c @@ -155,10 +155,8 @@ out: if (migration_has_all_channels()) { /* Close listening socket as its no longer needed */ qio_channel_close(ioc, NULL); - return G_SOURCE_REMOVE; - } else { - return G_SOURCE_CONTINUE; } + return G_SOURCE_CONTINUE; } =20 =20 diff --git a/migration/trace-events b/migration/trace-events index 98c2e4de58..65b1c7e459 100644 --- a/migration/trace-events +++ b/migration/trace-events @@ -136,6 +136,8 @@ process_incoming_migration_co_end(int ret, int ps) "ret= =3D%d postcopy-state=3D%d" process_incoming_migration_co_postcopy_end_main(void) "" migration_set_incoming_channel(void *ioc, const char *ioctype) "ioc=3D%p i= octype=3D%s" migration_set_outgoing_channel(void *ioc, const char *ioctype, const char = *hostname) "ioc=3D%p ioctype=3D%s hostname=3D%s" +migrate_incoming_deferred(void) "" +migrate_incoming_recover(void) "" =20 # migration/rdma.c qemu_rdma_accept_incoming_migration(void) "" --=20 2.13.5 From nobody Sun Apr 28 04:06:08 2024 Delivered-To: importer@patchew.org Received-SPF: temperror (zoho.com: Error in retrieving data from DNS) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=temperror (zoho.com: Error in retrieving data from DNS) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1508138241716834.4084490438313; Mon, 16 Oct 2017 00:17:21 -0700 (PDT) Received: from localhost ([::1]:59720 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1e3ze8-0004Xw-WE for importer@patchew.org; Mon, 16 Oct 2017 03:17:09 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:42987) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1e3zI6-0001qR-BT for qemu-devel@nongnu.org; Mon, 16 Oct 2017 02:54:23 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1e3zI3-0003Kf-A9 for qemu-devel@nongnu.org; Mon, 16 Oct 2017 02:54:22 -0400 Received: from mx1.redhat.com ([209.132.183.28]:32858) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1e3zI3-0003Ih-4F for qemu-devel@nongnu.org; Mon, 16 Oct 2017 02:54:19 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id EE0E980465; Mon, 16 Oct 2017 06:54:17 +0000 (UTC) Received: from pxdev.xzpeter.org.com (dhcp-15-225.nay.redhat.com [10.66.15.225]) by smtp.corp.redhat.com (Postfix) with ESMTP id B83885EE02; Mon, 16 Oct 2017 06:54:15 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com EE0E980465 Authentication-Results: ext-mx04.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx04.extmail.prod.ext.phx2.redhat.com; spf=fail smtp.mailfrom=peterx@redhat.com From: Peter Xu To: qemu-devel@nongnu.org Date: Mon, 16 Oct 2017 14:52:13 +0800 Message-Id: <20171016065216.18162-30-peterx@redhat.com> In-Reply-To: <20171016065216.18162-1-peterx@redhat.com> References: <20171016065216.18162-1-peterx@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.28]); Mon, 16 Oct 2017 06:54:18 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH v3 29/32] migration: init dst in migration_object_init too X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Andrea Arcangeli , Juan Quintela , Alexey Perevalov , peterx@redhat.com, "Dr . David Alan Gilbert" Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_6 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Though we may not need it, now we init both the src/dst migration objects in migration_object_init() so that even incoming migration object would be thread safe (it was not). Reviewed-by: Dr. David Alan Gilbert Signed-off-by: Peter Xu --- migration/migration.c | 28 +++++++++++++++------------- 1 file changed, 15 insertions(+), 13 deletions(-) diff --git a/migration/migration.c b/migration/migration.c index 387fbefad4..51e771685c 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -104,6 +104,7 @@ enum mig_rp_message_type { dynamic creation of migration */ =20 static MigrationState *current_migration; +static MigrationIncomingState *current_incoming; =20 static bool migration_object_check(MigrationState *ms, Error **errp); =20 @@ -116,6 +117,18 @@ void migration_object_init(void) assert(!current_migration); current_migration =3D MIGRATION_OBJ(object_new(TYPE_MIGRATION)); =20 + /* + * Init the migrate incoming object as well no matter whether + * we'll use it or not. + */ + assert(!current_incoming); + current_incoming =3D g_new0(MigrationIncomingState, 1); + current_incoming->state =3D MIGRATION_STATUS_NONE; + qemu_mutex_init(¤t_incoming->rp_mutex); + qemu_event_init(¤t_incoming->main_thread_load_event, false); + qemu_sem_init(¤t_incoming->postcopy_pause_sem_dst, 0); + qemu_sem_init(¤t_incoming->postcopy_pause_sem_fault, 0); + if (!migration_object_check(current_migration, &err)) { error_report_err(err); exit(1); @@ -141,19 +154,8 @@ MigrationState *migrate_get_current(void) =20 MigrationIncomingState *migration_incoming_get_current(void) { - static bool once; - static MigrationIncomingState mis_current; - - if (!once) { - mis_current.state =3D MIGRATION_STATUS_NONE; - memset(&mis_current, 0, sizeof(MigrationIncomingState)); - qemu_mutex_init(&mis_current.rp_mutex); - qemu_event_init(&mis_current.main_thread_load_event, false); - qemu_sem_init(&mis_current.postcopy_pause_sem_dst, 0); - qemu_sem_init(&mis_current.postcopy_pause_sem_fault, 0); - once =3D true; - } - return &mis_current; + assert(current_incoming); + return current_incoming; } =20 void migration_incoming_state_destroy(void) --=20 2.13.5 From nobody Sun Apr 28 04:06:08 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (208.118.235.17 [208.118.235.17]) by mx.zohomail.com with SMTPS id 1508138323419386.6606087302865; Mon, 16 Oct 2017 00:18:43 -0700 (PDT) Received: from localhost ([::1]:59724 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1e3zfP-0005eP-MX for importer@patchew.org; Mon, 16 Oct 2017 03:18:27 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:42990) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1e3zI6-0001qk-NL for qemu-devel@nongnu.org; Mon, 16 Oct 2017 02:54:23 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1e3zI5-0003P9-Uu for qemu-devel@nongnu.org; Mon, 16 Oct 2017 02:54:22 -0400 Received: from mx1.redhat.com ([209.132.183.28]:33626) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1e3zI5-0003OA-OR for qemu-devel@nongnu.org; Mon, 16 Oct 2017 02:54:21 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id B891E4E33D; Mon, 16 Oct 2017 06:54:20 +0000 (UTC) Received: from pxdev.xzpeter.org.com (dhcp-15-225.nay.redhat.com [10.66.15.225]) by smtp.corp.redhat.com (Postfix) with ESMTP id 69BE66269A; Mon, 16 Oct 2017 06:54:18 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com B891E4E33D Authentication-Results: ext-mx09.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx09.extmail.prod.ext.phx2.redhat.com; spf=fail smtp.mailfrom=peterx@redhat.com From: Peter Xu To: qemu-devel@nongnu.org Date: Mon, 16 Oct 2017 14:52:14 +0800 Message-Id: <20171016065216.18162-31-peterx@redhat.com> In-Reply-To: <20171016065216.18162-1-peterx@redhat.com> References: <20171016065216.18162-1-peterx@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.38]); Mon, 16 Oct 2017 06:54:20 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH v3 30/32] migration: delay the postcopy-active state switch X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Andrea Arcangeli , Juan Quintela , Alexey Perevalov , peterx@redhat.com, "Dr . David Alan Gilbert" Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Switch the state until we try to start the VM on destination side. The problem is that without doing this we may have a very small window that we'll be in such a state: - dst VM is in postcopy-active state, - main thread is handling MIG_CMD_PACKAGED message, which loads all the device states, - ram load thread is reading memory data from source. Then if we failed at this point when reading the migration stream we'll also switch to postcopy-paused state, but that is not what we want. If device states failed to load, we should fail the migration directly instead of pause. Postponing the state switch to the point when we have already loaded the devices' states and been ready to start running destination VM. Signed-off-by: Peter Xu --- migration/savevm.c | 10 ++++++++-- 1 file changed, 8 insertions(+), 2 deletions(-) diff --git a/migration/savevm.c b/migration/savevm.c index bc87b0e5b1..3bc792e320 100644 --- a/migration/savevm.c +++ b/migration/savevm.c @@ -1584,8 +1584,6 @@ static void *postcopy_ram_listen_thread(void *opaque) QEMUFile *f =3D mis->from_src_file; int load_res; =20 - migrate_set_state(&mis->state, MIGRATION_STATUS_ACTIVE, - MIGRATION_STATUS_POSTCOPY_ACTIVE); qemu_sem_post(&mis->listen_thread_sem); trace_postcopy_ram_listen_thread_start(); =20 @@ -1748,6 +1746,14 @@ static int loadvm_postcopy_handle_run(MigrationIncom= ingState *mis) return -1; } =20 + /* + * Declare that we are in postcopy now. We should already have + * all the device states loaded ready when reach here, and also + * the ram load thread running. + */ + migrate_set_state(&mis->state, MIGRATION_STATUS_ACTIVE, + MIGRATION_STATUS_POSTCOPY_ACTIVE); + data =3D g_new(HandleRunBhData, 1); data->bh =3D qemu_bh_new(loadvm_postcopy_handle_run_bh, data); qemu_bh_schedule(data->bh); --=20 2.13.5 From nobody Sun Apr 28 04:06:08 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1508138385828400.0954641849348; Mon, 16 Oct 2017 00:19:45 -0700 (PDT) Received: from localhost ([::1]:59727 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1e3zgZ-0006Fg-1U for importer@patchew.org; Mon, 16 Oct 2017 03:19:39 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:43053) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1e3zIF-0001yI-BF for qemu-devel@nongnu.org; Mon, 16 Oct 2017 02:54:32 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1e3zIC-0003bb-4O for qemu-devel@nongnu.org; Mon, 16 Oct 2017 02:54:31 -0400 Received: from mx1.redhat.com ([209.132.183.28]:53734) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1e3zIB-0003Zp-To for qemu-devel@nongnu.org; Mon, 16 Oct 2017 02:54:28 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id E6981883C4; Mon, 16 Oct 2017 06:54:26 +0000 (UTC) Received: from pxdev.xzpeter.org.com (dhcp-15-225.nay.redhat.com [10.66.15.225]) by smtp.corp.redhat.com (Postfix) with ESMTP id 3446262923; Mon, 16 Oct 2017 06:54:20 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com E6981883C4 Authentication-Results: ext-mx02.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx02.extmail.prod.ext.phx2.redhat.com; spf=fail smtp.mailfrom=peterx@redhat.com From: Peter Xu To: qemu-devel@nongnu.org Date: Mon, 16 Oct 2017 14:52:15 +0800 Message-Id: <20171016065216.18162-32-peterx@redhat.com> In-Reply-To: <20171016065216.18162-1-peterx@redhat.com> References: <20171016065216.18162-1-peterx@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.26]); Mon, 16 Oct 2017 06:54:27 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH v3 31/32] migration, qmp: new command "migrate-pause" X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Andrea Arcangeli , Juan Quintela , Alexey Perevalov , peterx@redhat.com, "Dr . David Alan Gilbert" Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" It is used to manually trigger the postcopy pause state. It works just like when we found the migration stream failed during postcopy, but provide an explicit way for user in case of misterious socket hangs. Signed-off-by: Peter Xu --- migration/migration.c | 18 ++++++++++++++++++ qapi/migration.json | 22 ++++++++++++++++++++++ 2 files changed, 40 insertions(+) diff --git a/migration/migration.c b/migration/migration.c index 51e771685c..3d852d233f 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -1426,6 +1426,24 @@ void qmp_migrate_incoming(const char *uri, Error **e= rrp) once =3D false; } =20 +void qmp_migrate_pause(Error **errp) +{ + int ret; + MigrationState *ms =3D migrate_get_current(); + + if (ms->state !=3D MIGRATION_STATUS_POSTCOPY_ACTIVE) { + error_setg(errp, "Migration pause is currently only allowed during" + " an active postcopy phase."); + return; + } + + ret =3D qemu_file_shutdown(ms->to_dst_file); + + if (ret) { + error_setg(errp, "Failed to pause migration stream."); + } +} + bool migration_is_blocked(Error **errp) { if (qemu_savevm_state_blocked(errp)) { diff --git a/qapi/migration.json b/qapi/migration.json index f8132e683a..095aeb4dd2 100644 --- a/qapi/migration.json +++ b/qapi/migration.json @@ -1028,6 +1028,28 @@ { 'command': 'migrate-incoming', 'data': {'uri': 'str' } } =20 ## +# @migrate-pause: +# +# Pause an migration. Currently it can only pause a postcopy +# migration. Pausing a precopy migration is not supported yet. +# +# It is mostly used as a manual way to trigger the postcopy paused +# state when the network sockets hang due to some reason, so that we +# can try a recovery afterward. +# +# Returns: nothing on success +# +# Since: 2.10 +# +# Example: +# +# -> { "execute": "migrate-pause" } +# <- { "return": {} } +# +## +{ 'command': 'migrate-pause' } + +## # @xen-save-devices-state: # # Save the state of all devices to file. The RAM and the block devices --=20 2.13.5 From nobody Sun Apr 28 04:06:08 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1508137683394461.8904029515845; Mon, 16 Oct 2017 00:08:03 -0700 (PDT) Received: from localhost ([::1]:59671 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1e3zVI-0004sS-JP for importer@patchew.org; Mon, 16 Oct 2017 03:08:00 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:43067) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1e3zIH-0001zy-5P for qemu-devel@nongnu.org; Mon, 16 Oct 2017 02:54:34 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1e3zIG-0003iU-9G for qemu-devel@nongnu.org; Mon, 16 Oct 2017 02:54:33 -0400 Received: from mx1.redhat.com ([209.132.183.28]:33478) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1e3zIG-0003hS-38 for qemu-devel@nongnu.org; Mon, 16 Oct 2017 02:54:32 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id ED53A80461; Mon, 16 Oct 2017 06:54:30 +0000 (UTC) Received: from pxdev.xzpeter.org.com (dhcp-15-225.nay.redhat.com [10.66.15.225]) by smtp.corp.redhat.com (Postfix) with ESMTP id 47D265EE02; Mon, 16 Oct 2017 06:54:27 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com ED53A80461 Authentication-Results: ext-mx04.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx04.extmail.prod.ext.phx2.redhat.com; spf=fail smtp.mailfrom=peterx@redhat.com From: Peter Xu To: qemu-devel@nongnu.org Date: Mon, 16 Oct 2017 14:52:16 +0800 Message-Id: <20171016065216.18162-33-peterx@redhat.com> In-Reply-To: <20171016065216.18162-1-peterx@redhat.com> References: <20171016065216.18162-1-peterx@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.28]); Mon, 16 Oct 2017 06:54:31 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH v3 32/32] migration, hmp: new command "migrate_pause" X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Andrea Arcangeli , Juan Quintela , Alexey Perevalov , peterx@redhat.com, "Dr . David Alan Gilbert" Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" HMP version of QMP "migrate-pause". Signed-off-by: Peter Xu --- hmp-commands.hx | 14 ++++++++++++++ hmp.c | 9 +++++++++ hmp.h | 1 + 3 files changed, 24 insertions(+) diff --git a/hmp-commands.hx b/hmp-commands.hx index 7adb029b34..bfba3ea977 100644 --- a/hmp-commands.hx +++ b/hmp-commands.hx @@ -980,6 +980,20 @@ as the -incoming option). ETEXI =20 { + .name =3D "migrate_pause", + .args_type =3D "", + .params =3D "", + .help =3D "Pause a migration stream (only supported by postc= opy)", + .cmd =3D hmp_migrate_pause, + }, + +STEXI +@item migrate_pause +@findex migrate_pause +Pause an existing migration manually. Currently it only support postcopy. +ETEXI + + { .name =3D "migrate_set_cache_size", .args_type =3D "value:o", .params =3D "value", diff --git a/hmp.c b/hmp.c index 7b1abcd535..689092f534 100644 --- a/hmp.c +++ b/hmp.c @@ -1500,6 +1500,15 @@ void hmp_migrate_incoming(Monitor *mon, const QDict = *qdict) hmp_handle_error(mon, &err); } =20 +void hmp_migrate_pause(Monitor *mon, const QDict *qdict) +{ + Error *err =3D NULL; + + qmp_migrate_pause(&err); + + hmp_handle_error(mon, &err); +} + /* Kept for backwards compatibility */ void hmp_migrate_set_downtime(Monitor *mon, const QDict *qdict) { diff --git a/hmp.h b/hmp.h index 3605003e4c..2c1dab6b8f 100644 --- a/hmp.h +++ b/hmp.h @@ -69,6 +69,7 @@ void hmp_delvm(Monitor *mon, const QDict *qdict); void hmp_info_snapshots(Monitor *mon, const QDict *qdict); void hmp_migrate_cancel(Monitor *mon, const QDict *qdict); void hmp_migrate_incoming(Monitor *mon, const QDict *qdict); +void hmp_migrate_pause(Monitor *mon, const QDict *qdict); void hmp_migrate_set_downtime(Monitor *mon, const QDict *qdict); void hmp_migrate_set_speed(Monitor *mon, const QDict *qdict); void hmp_migrate_set_capability(Monitor *mon, const QDict *qdict); --=20 2.13.5