From nobody Tue Nov 4 16:02:40 2025 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1504083183050681.418385824348; Wed, 30 Aug 2017 01:53:03 -0700 (PDT) Received: from localhost ([::1]:49102 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dmyk8-0007Vs-Ck for importer@patchew.org; Wed, 30 Aug 2017 04:53:00 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:34061) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dmyR5-00078n-2y for qemu-devel@nongnu.org; Wed, 30 Aug 2017 04:33:20 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1dmyR3-0003ID-Ng for qemu-devel@nongnu.org; Wed, 30 Aug 2017 04:33:19 -0400 Received: from mx1.redhat.com ([209.132.183.28]:47414) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1dmyR3-0003Hs-FH for qemu-devel@nongnu.org; Wed, 30 Aug 2017 04:33:17 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 7FF1E80F6C; Wed, 30 Aug 2017 08:33:16 +0000 (UTC) Received: from pxdev.xzpeter.org.com (dhcp-14-103.nay.redhat.com [10.66.14.103]) by smtp.corp.redhat.com (Postfix) with ESMTP id 7C022871DD; Wed, 30 Aug 2017 08:33:13 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com 7FF1E80F6C Authentication-Results: ext-mx03.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx03.extmail.prod.ext.phx2.redhat.com; spf=fail smtp.mailfrom=peterx@redhat.com From: Peter Xu To: qemu-devel@nongnu.org Date: Wed, 30 Aug 2017 16:32:06 +0800 Message-Id: <1504081950-2528-10-git-send-email-peterx@redhat.com> In-Reply-To: <1504081950-2528-1-git-send-email-peterx@redhat.com> References: <1504081950-2528-1-git-send-email-peterx@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.27]); Wed, 30 Aug 2017 08:33:16 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [RFC v2 09/33] migration: implement "postcopy-pause" src logic X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Laurent Vivier , Andrea Arcangeli , Juan Quintela , Alexey Perevalov , peterx@redhat.com, "Dr . David Alan Gilbert" Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Now when network down for postcopy, the source side will not fail the migration. Instead we convert the status into this new paused state, and we will try to wait for a rescue in the future. If a recovery is detected, migration_thread() will reset its local variables to prepare for that. Signed-off-by: Peter Xu Reviewed-by: Dr. David Alan Gilbert --- migration/migration.c | 98 ++++++++++++++++++++++++++++++++++++++++++++++= +--- migration/migration.h | 3 ++ migration/trace-events | 1 + 3 files changed, 98 insertions(+), 4 deletions(-) diff --git a/migration/migration.c b/migration/migration.c index f6130db..8d26ea8 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -993,6 +993,8 @@ static void migrate_fd_cleanup(void *opaque) =20 notifier_list_notify(&migration_state_notifiers, s); block_cleanup_parameters(s); + + qemu_sem_destroy(&s->postcopy_pause_sem); } =20 void migrate_fd_error(MigrationState *s, const Error *error) @@ -1136,6 +1138,7 @@ MigrationState *migrate_init(void) s->migration_thread_running =3D false; error_free(s->error); s->error =3D NULL; + qemu_sem_init(&s->postcopy_pause_sem, 0); =20 migrate_set_state(&s->state, MIGRATION_STATUS_NONE, MIGRATION_STATUS_S= ETUP); =20 @@ -1938,6 +1941,80 @@ bool migrate_colo_enabled(void) return s->enabled_capabilities[MIGRATION_CAPABILITY_X_COLO]; } =20 +typedef enum MigThrError { + /* No error detected */ + MIG_THR_ERR_NONE =3D 0, + /* Detected error, but resumed successfully */ + MIG_THR_ERR_RECOVERED =3D 1, + /* Detected fatal error, need to exit */ + MIG_THR_ERR_FATAL =3D 2, +} MigThrError; + +/* + * We don't return until we are in a safe state to continue current + * postcopy migration. Returns MIG_THR_ERR_RECOVERED if recovered, or + * MIG_THR_ERR_FATAL if unrecovery failure happened. + */ +static MigThrError postcopy_pause(MigrationState *s) +{ + assert(s->state =3D=3D MIGRATION_STATUS_POSTCOPY_ACTIVE); + migrate_set_state(&s->state, MIGRATION_STATUS_POSTCOPY_ACTIVE, + MIGRATION_STATUS_POSTCOPY_PAUSED); + + /* Current channel is possibly broken. Release it. */ + assert(s->to_dst_file); + qemu_file_shutdown(s->to_dst_file); + qemu_fclose(s->to_dst_file); + s->to_dst_file =3D NULL; + + error_report("Detected IO failure for postcopy. " + "Migration paused."); + + /* + * We wait until things fixed up. Then someone will setup the + * status back for us. + */ + while (s->state =3D=3D MIGRATION_STATUS_POSTCOPY_PAUSED) { + qemu_sem_wait(&s->postcopy_pause_sem); + } + + trace_postcopy_pause_continued(); + + return MIG_THR_ERR_RECOVERED; +} + +static MigThrError migration_detect_error(MigrationState *s) +{ + int ret; + + /* Try to detect any file errors */ + ret =3D qemu_file_get_error(s->to_dst_file); + + if (!ret) { + /* Everything is fine */ + return MIG_THR_ERR_NONE; + } + + if (s->state =3D=3D MIGRATION_STATUS_POSTCOPY_ACTIVE && ret =3D=3D -EI= O) { + /* + * For postcopy, we allow the network to be down for a + * while. After that, it can be continued by a + * recovery phase. + */ + return postcopy_pause(s); + } else { + /* + * For precopy (or postcopy with error outside IO), we fail + * with no time. + */ + migrate_set_state(&s->state, s->state, MIGRATION_STATUS_FAILED); + trace_migration_thread_file_err(); + + /* Time to stop the migration, now. */ + return MIG_THR_ERR_FATAL; + } +} + /* * Master migration thread on the source VM. * It drives the migration and pumps the data down the outgoing channel. @@ -1962,6 +2039,7 @@ static void *migration_thread(void *opaque) /* The active state we expect to be in; ACTIVE or POSTCOPY_ACTIVE */ enum MigrationStatus current_active_state =3D MIGRATION_STATUS_ACTIVE; bool enable_colo =3D migrate_colo_enabled(); + MigThrError thr_error; =20 rcu_register_thread(); =20 @@ -2034,12 +2112,24 @@ static void *migration_thread(void *opaque) } } =20 - if (qemu_file_get_error(s->to_dst_file)) { - migrate_set_state(&s->state, current_active_state, - MIGRATION_STATUS_FAILED); - trace_migration_thread_file_err(); + /* + * Try to detect any kind of failures, and see whether we + * should stop the migration now. + */ + thr_error =3D migration_detect_error(s); + if (thr_error =3D=3D MIG_THR_ERR_FATAL) { + /* Stop migration */ break; + } else if (thr_error =3D=3D MIG_THR_ERR_RECOVERED) { + /* + * Just recovered from a e.g. network failure, reset all + * the local variables. This is important to avoid + * breaking transferred_bytes and bandwidth calculation + */ + initial_time =3D qemu_clock_get_ms(QEMU_CLOCK_REALTIME); + initial_bytes =3D 0; } + current_time =3D qemu_clock_get_ms(QEMU_CLOCK_REALTIME); if (current_time >=3D initial_time + BUFFER_DELAY) { uint64_t transferred_bytes =3D qemu_ftell(s->to_dst_file) - diff --git a/migration/migration.h b/migration/migration.h index 70e3094..0c957c9 100644 --- a/migration/migration.h +++ b/migration/migration.h @@ -149,6 +149,9 @@ struct MigrationState bool send_configuration; /* Whether we send section footer during migration */ bool send_section_footer; + + /* Needed by postcopy-pause state */ + QemuSemaphore postcopy_pause_sem; }; =20 void migrate_set_state(int *state, int old_state, int new_state); diff --git a/migration/trace-events b/migration/trace-events index d2910a6..907564b 100644 --- a/migration/trace-events +++ b/migration/trace-events @@ -98,6 +98,7 @@ migration_thread_setup_complete(void) "" open_return_path_on_source(void) "" open_return_path_on_source_continue(void) "" postcopy_start(void) "" +postcopy_pause_continued(void) "" postcopy_start_set_run(void) "" source_return_path_thread_bad_end(void) "" source_return_path_thread_end(void) "" --=20 2.7.4