From nobody Fri Nov 14 16:47:11 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1761861109; cv=none; d=zohomail.com; s=zohoarc; b=nI2sZA74FNcGWGJn+7kGjvydSWtnw35sZu/trHgbbaNBLLeo0Xx810Bo9kUh/4y1EWi4jGDQUdtck5KfyxHP7/OXmzHbqqexQfo8PUzF6zr5vbH7quzRSZ4FqGAYOCA2Ugb0KvcaLSS54wA1e/Qpg4PitswirPCOyHpt2BNp1qo= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1761861109; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=/EGZjmbie7WvN5P/atFBVzVk3p2GtY/pSoH3cMkB8LE=; b=ae+eMZM0N/Hi9QppEZJ/ry98Rbk3yTpS8vQkdpGy809zDgasdRDC1F+it112mYCRL/ftLYPofo35MwVuRQkLj/nqR0ArzBqjRt7HA9qJ7xxoWP6EtAKrstKoF6ypcnpyxXKAkm+QdglHX7p2BHVcNzfakOXytePVHwu7ECJfj48= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1761861109395643.2585802836552; Thu, 30 Oct 2025 14:51:49 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1vEaWu-0006nH-0I; Thu, 30 Oct 2025 17:50:00 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1vEaWo-0006lC-Dg for qemu-devel@nongnu.org; Thu, 30 Oct 2025 17:49:55 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1vEaWY-0000YZ-9m for qemu-devel@nongnu.org; Thu, 30 Oct 2025 17:49:54 -0400 Received: from mx-prod-mc-03.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-475--ycmEvJiMDGRYDNjhqaOzg-1; Thu, 30 Oct 2025 17:49:28 -0400 Received: from mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.93]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-03.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 9A1791955DC1; Thu, 30 Oct 2025 21:49:27 +0000 (UTC) Received: from fedora.redhat.com (unknown [10.45.224.58]) by mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 79DB0180057E; Thu, 30 Oct 2025 21:49:25 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1761860972; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=/EGZjmbie7WvN5P/atFBVzVk3p2GtY/pSoH3cMkB8LE=; b=GZhCMiAAz+s55du8n+lHcqY9knCB819ZeaVPnLmOTx39BDyEvPjZh8sg+C24XyBnQjSPGh 4atr+GsNaFRomdcfPN3MwFtH5/IFzErnNrWHnVULrz60T30i1tr6Ad08yYUrzry1gZQiod s2WTkEdR34dTt0kR0TOfRAlV28r22WI= X-MC-Unique: -ycmEvJiMDGRYDNjhqaOzg-1 X-Mimecast-MFC-AGG-ID: -ycmEvJiMDGRYDNjhqaOzg_1761860968 From: Juraj Marcin To: qemu-devel@nongnu.org Cc: Juraj Marcin , Peter Xu , "Dr. David Alan Gilbert" , Jiri Denemark , Fabiano Rosas Subject: [PATCH v3 1/7] migration: Do not try to start VM if disk activation fails Date: Thu, 30 Oct 2025 22:49:05 +0100 Message-ID: <20251030214915.1411860-2-jmarcin@redhat.com> In-Reply-To: <20251030214915.1411860-1-jmarcin@redhat.com> References: <20251030214915.1411860-1-jmarcin@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.93 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=jmarcin@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1761861122192158500 Content-Type: text/plain; charset="utf-8" From: Peter Xu If a rare split brain happens (e.g. dest QEMU started running somehow, taking shared drive locks), src QEMU may not be able to activate the drives anymore. In this case, src QEMU shouldn't start the VM or it might crash the block layer later with something like: Meanwhile, src QEMU cannot try to continue either even if dest QEMU can release the drive locks (e.g. by QMP "stop"). Because as long as dest QEMU started running, it means dest QEMU's RAM is the only version that is consistent with current status of the shared storage. Signed-off-by: Peter Xu --- migration/migration.c | 29 ++++++++++++++++++++++++----- 1 file changed, 24 insertions(+), 5 deletions(-) diff --git a/migration/migration.c b/migration/migration.c index 5e74993b46..6e647c7c4a 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -3526,6 +3526,8 @@ static MigIterateState migration_iteration_run(Migrat= ionState *s) =20 static void migration_iteration_finish(MigrationState *s) { + Error *local_err =3D NULL; + bql_lock(); =20 /* @@ -3549,11 +3551,28 @@ static void migration_iteration_finish(MigrationSta= te *s) case MIGRATION_STATUS_FAILED: case MIGRATION_STATUS_CANCELLED: case MIGRATION_STATUS_CANCELLING: - /* - * Re-activate the block drives if they're inactivated. Note, COLO - * shouldn't use block_active at all, so it should be no-op there. - */ - migration_block_activate(NULL); + if (!migration_block_activate(&local_err)) { + /* + * Re-activate the block drives if they're inactivated. + * + * If it fails (e.g. in case of a split brain, where dest QEMU + * might have taken some of the drive locks and running!), do + * not start VM, instead wait for mgmt to decide the next step. + * + * If dest already started, it means dest QEMU should contain + * all the data it needs and it properly owns all the drive + * locks. Then even if src QEMU got a FAILED in migration, it + * normally should mean we should treat the migration as + * COMPLETED. + * + * NOTE: it's not safe anymore to start VM on src now even if + * dest would release the drive locks. It's because as long as + * dest started running then only dest QEMU's RAM is consistent + * with the shared storage. + */ + error_free(local_err); + break; + } if (runstate_is_live(s->vm_old_state)) { if (!runstate_check(RUN_STATE_SHUTDOWN)) { vm_start(); --=20 2.51.0 From nobody Fri Nov 14 16:47:11 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1761861085; cv=none; d=zohomail.com; s=zohoarc; b=NiTREPsm1q4lxr4hKWfRba4eL1cnepprJZMYlvIBeqcFnsz5+d6R+oere68cMz8oyt9x4apGekZlyZKiE/iQvZfeLShXR9+IxbilsmGehFiJ8eZaDgQJ4ZvzQyzc/30JpCp22+dNmzbZx3Npqf/E6hypQRdLMYp2pIf7VFB9z+w= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1761861085; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=Qn2mexWiAvmWha/R1AUWfcqQhrERs1W29mYUgY0+v1M=; b=LHwKws4IS70FntoTFtT3FjNGkb1AKo0/K9Qxosh6ELHYE9BHwuEJfzcdqZzYb3ZYNPp97n5E2k64qeo/YAYIpy64emM4BzNGmW4KJVkGuMtLS9Uf+3gP9fPT/QFyAxpb4s10jhD7TOL373mSLxas1RlBPP3KvtoZYXcjuZ7qrQQ= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1761861085366186.4311762860881; Thu, 30 Oct 2025 14:51:25 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1vEaWp-0006l1-HI; Thu, 30 Oct 2025 17:49:57 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1vEaWk-0006jV-Bs for qemu-devel@nongnu.org; Thu, 30 Oct 2025 17:49:50 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1vEaWZ-0000Yn-AU for qemu-devel@nongnu.org; Thu, 30 Oct 2025 17:49:50 -0400 Received: from mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (ec2-35-165-154-97.us-west-2.compute.amazonaws.com [35.165.154.97]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-517-ldiViGs2NQajSMWMm03fYA-1; Thu, 30 Oct 2025 17:49:32 -0400 Received: from mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.93]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id C469D180AE16; Thu, 30 Oct 2025 21:49:31 +0000 (UTC) Received: from fedora.redhat.com (unknown [10.45.224.58]) by mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id D4D451800581; Thu, 30 Oct 2025 21:49:29 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1761860976; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Qn2mexWiAvmWha/R1AUWfcqQhrERs1W29mYUgY0+v1M=; b=X0He86Vh9J64T1PkZ4CuMUA4AxJ1ZblEw1/mYMscXtQ3tbsvdkzNjQwB2qbCC9SkBLQId8 pXoCcT7NRCwxj/Xo82QaAT58FJs2SUU6CZClTuo6Y5EekodkIJUozfOJAM3KEEM1qgpZYW d+sSQb7NrdItOxkZYzUVM8oZ4yJbI+o= X-MC-Unique: ldiViGs2NQajSMWMm03fYA-1 X-Mimecast-MFC-AGG-ID: ldiViGs2NQajSMWMm03fYA_1761860971 From: Juraj Marcin To: qemu-devel@nongnu.org Cc: Juraj Marcin , Peter Xu , "Dr. David Alan Gilbert" , Jiri Denemark , Fabiano Rosas Subject: [PATCH v3 2/7] migration: Move postcopy_ram_listen_thread() to postcopy-ram.c Date: Thu, 30 Oct 2025 22:49:06 +0100 Message-ID: <20251030214915.1411860-3-jmarcin@redhat.com> In-Reply-To: <20251030214915.1411860-1-jmarcin@redhat.com> References: <20251030214915.1411860-1-jmarcin@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.93 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=jmarcin@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1761861090121158500 Content-Type: text/plain; charset="utf-8" From: Juraj Marcin This patch addresses a TODO about moving postcopy_ram_listen_thread() to postcopy file. Signed-off-by: Juraj Marcin Reviewed-by: Peter Xu --- migration/postcopy-ram.c | 107 +++++++++++++++++++++++++++++++++++++++ migration/postcopy-ram.h | 2 + migration/savevm.c | 107 --------------------------------------- 3 files changed, 109 insertions(+), 107 deletions(-) diff --git a/migration/postcopy-ram.c b/migration/postcopy-ram.c index 5471efb4f0..36d5415554 100644 --- a/migration/postcopy-ram.c +++ b/migration/postcopy-ram.c @@ -2077,3 +2077,110 @@ bool postcopy_is_paused(MigrationStatus status) return status =3D=3D MIGRATION_STATUS_POSTCOPY_PAUSED || status =3D=3D MIGRATION_STATUS_POSTCOPY_RECOVER_SETUP; } + +/* + * Triggered by a postcopy_listen command; this thread takes over reading + * the input stream, leaving the main thread free to carry on loading the = rest + * of the device state (from RAM). + * (TODO:This could do with being in a postcopy file - but there again it's + * just another input loop, not that postcopy specific) + */ +void *postcopy_ram_listen_thread(void *opaque) +{ + MigrationIncomingState *mis =3D migration_incoming_get_current(); + QEMUFile *f =3D mis->from_src_file; + int load_res; + MigrationState *migr =3D migrate_get_current(); + Error *local_err =3D NULL; + + object_ref(OBJECT(migr)); + + migrate_set_state(&mis->state, MIGRATION_STATUS_ACTIVE, + MIGRATION_STATUS_POSTCOPY_ACTIVE); + qemu_event_set(&mis->thread_sync_event); + trace_postcopy_ram_listen_thread_start(); + + rcu_register_thread(); + /* + * Because we're a thread and not a coroutine we can't yield + * in qemu_file, and thus we must be blocking now. + */ + qemu_file_set_blocking(f, true, &error_fatal); + + /* TODO: sanity check that only postcopiable data will be loaded here = */ + load_res =3D qemu_loadvm_state_main(f, mis, &local_err); + + /* + * This is tricky, but, mis->from_src_file can change after it + * returns, when postcopy recovery happened. In the future, we may + * want a wrapper for the QEMUFile handle. + */ + f =3D mis->from_src_file; + + /* And non-blocking again so we don't block in any cleanup */ + qemu_file_set_blocking(f, false, &error_fatal); + + trace_postcopy_ram_listen_thread_exit(); + if (load_res < 0) { + qemu_file_set_error(f, load_res); + dirty_bitmap_mig_cancel_incoming(); + if (postcopy_state_get() =3D=3D POSTCOPY_INCOMING_RUNNING && + !migrate_postcopy_ram() && migrate_dirty_bitmaps()) + { + error_report("%s: loadvm failed during postcopy: %d: %s. All s= tates " + "are migrated except dirty bitmaps. Some dirty " + "bitmaps may be lost, and present migrated dirty " + "bitmaps are correctly migrated and valid.", + __func__, load_res, error_get_pretty(local_err)); + g_clear_pointer(&local_err, error_free); + load_res =3D 0; /* prevent further exit() */ + } else { + error_prepend(&local_err, + "loadvm failed during postcopy: %d: ", load_res); + migrate_set_error(migr, local_err); + g_clear_pointer(&local_err, error_report_err); + migrate_set_state(&mis->state, MIGRATION_STATUS_POSTCOPY_ACTIV= E, + MIGRATION_STATUS_FAILED); + } + } + if (load_res >=3D 0) { + /* + * This looks good, but it's possible that the device loading in t= he + * main thread hasn't finished yet, and so we might not be in 'RUN' + * state yet; wait for the end of the main thread. + */ + qemu_event_wait(&mis->main_thread_load_event); + } + postcopy_ram_incoming_cleanup(mis); + + if (load_res < 0) { + /* + * If something went wrong then we have a bad state so exit; + * depending how far we got it might be possible at this point + * to leave the guest running and fire MCEs for pages that never + * arrived as a desperate recovery step. + */ + rcu_unregister_thread(); + exit(EXIT_FAILURE); + } + + migrate_set_state(&mis->state, MIGRATION_STATUS_POSTCOPY_ACTIVE, + MIGRATION_STATUS_COMPLETED); + /* + * If everything has worked fine, then the main thread has waited + * for us to start, and we're the last use of the mis. + * (If something broke then qemu will have to exit anyway since it's + * got a bad migration state). + */ + bql_lock(); + migration_incoming_state_destroy(); + bql_unlock(); + + rcu_unregister_thread(); + mis->have_listen_thread =3D false; + postcopy_state_set(POSTCOPY_INCOMING_END); + + object_unref(OBJECT(migr)); + + return NULL; +} diff --git a/migration/postcopy-ram.h b/migration/postcopy-ram.h index ca19433b24..3e26db3e6b 100644 --- a/migration/postcopy-ram.h +++ b/migration/postcopy-ram.h @@ -199,4 +199,6 @@ bool postcopy_is_paused(MigrationStatus status); void mark_postcopy_blocktime_begin(uintptr_t addr, uint32_t ptid, RAMBlock *rb); =20 +void *postcopy_ram_listen_thread(void *opaque); + #endif diff --git a/migration/savevm.c b/migration/savevm.c index 232cae090b..97fdd08c08 100644 --- a/migration/savevm.c +++ b/migration/savevm.c @@ -2087,113 +2087,6 @@ static int loadvm_postcopy_ram_handle_discard(Migra= tionIncomingState *mis, return 0; } =20 -/* - * Triggered by a postcopy_listen command; this thread takes over reading - * the input stream, leaving the main thread free to carry on loading the = rest - * of the device state (from RAM). - * (TODO:This could do with being in a postcopy file - but there again it's - * just another input loop, not that postcopy specific) - */ -static void *postcopy_ram_listen_thread(void *opaque) -{ - MigrationIncomingState *mis =3D migration_incoming_get_current(); - QEMUFile *f =3D mis->from_src_file; - int load_res; - MigrationState *migr =3D migrate_get_current(); - Error *local_err =3D NULL; - - object_ref(OBJECT(migr)); - - migrate_set_state(&mis->state, MIGRATION_STATUS_ACTIVE, - MIGRATION_STATUS_POSTCOPY_ACTIVE); - qemu_event_set(&mis->thread_sync_event); - trace_postcopy_ram_listen_thread_start(); - - rcu_register_thread(); - /* - * Because we're a thread and not a coroutine we can't yield - * in qemu_file, and thus we must be blocking now. - */ - qemu_file_set_blocking(f, true, &error_fatal); - - /* TODO: sanity check that only postcopiable data will be loaded here = */ - load_res =3D qemu_loadvm_state_main(f, mis, &local_err); - - /* - * This is tricky, but, mis->from_src_file can change after it - * returns, when postcopy recovery happened. In the future, we may - * want a wrapper for the QEMUFile handle. - */ - f =3D mis->from_src_file; - - /* And non-blocking again so we don't block in any cleanup */ - qemu_file_set_blocking(f, false, &error_fatal); - - trace_postcopy_ram_listen_thread_exit(); - if (load_res < 0) { - qemu_file_set_error(f, load_res); - dirty_bitmap_mig_cancel_incoming(); - if (postcopy_state_get() =3D=3D POSTCOPY_INCOMING_RUNNING && - !migrate_postcopy_ram() && migrate_dirty_bitmaps()) - { - error_report("%s: loadvm failed during postcopy: %d: %s. All s= tates " - "are migrated except dirty bitmaps. Some dirty " - "bitmaps may be lost, and present migrated dirty " - "bitmaps are correctly migrated and valid.", - __func__, load_res, error_get_pretty(local_err)); - g_clear_pointer(&local_err, error_free); - load_res =3D 0; /* prevent further exit() */ - } else { - error_prepend(&local_err, - "loadvm failed during postcopy: %d: ", load_res); - migrate_set_error(migr, local_err); - g_clear_pointer(&local_err, error_report_err); - migrate_set_state(&mis->state, MIGRATION_STATUS_POSTCOPY_ACTIV= E, - MIGRATION_STATUS_FAILED); - } - } - if (load_res >=3D 0) { - /* - * This looks good, but it's possible that the device loading in t= he - * main thread hasn't finished yet, and so we might not be in 'RUN' - * state yet; wait for the end of the main thread. - */ - qemu_event_wait(&mis->main_thread_load_event); - } - postcopy_ram_incoming_cleanup(mis); - - if (load_res < 0) { - /* - * If something went wrong then we have a bad state so exit; - * depending how far we got it might be possible at this point - * to leave the guest running and fire MCEs for pages that never - * arrived as a desperate recovery step. - */ - rcu_unregister_thread(); - exit(EXIT_FAILURE); - } - - migrate_set_state(&mis->state, MIGRATION_STATUS_POSTCOPY_ACTIVE, - MIGRATION_STATUS_COMPLETED); - /* - * If everything has worked fine, then the main thread has waited - * for us to start, and we're the last use of the mis. - * (If something broke then qemu will have to exit anyway since it's - * got a bad migration state). - */ - bql_lock(); - migration_incoming_state_destroy(); - bql_unlock(); - - rcu_unregister_thread(); - mis->have_listen_thread =3D false; - postcopy_state_set(POSTCOPY_INCOMING_END); - - object_unref(OBJECT(migr)); - - return NULL; -} - /* After this message we must be able to immediately receive postcopy data= */ static int loadvm_postcopy_handle_listen(MigrationIncomingState *mis, Error **errp) --=20 2.51.0 From nobody Fri Nov 14 16:47:11 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1761861077; cv=none; d=zohomail.com; s=zohoarc; b=leL4mqfXwOmao8UkpjsYjgwgS3EK4rT2FSlW6LrmS/kpSaVqFElpxkLn89Nl/924xPsM8zWkipEbwufRx1LIJRlW89Q850vFsqxvmycqTWMAgrjgA2Dc3TUEdgxcjrP6h0oVQVK2AgpbmlI5riVtT+UdfGD5+nLZTb31szeFDTE= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1761861077; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=IuZ7wefnnYr02Da6Q6UYgXoyQkAcXJIfUpdX+fPhoSk=; b=QSooSt0/y2CxhmOYqkg9lWZ1j0+hFr96uMshM6VghfwStMZW7GBs30VRUUa0tqnR/N+PZEU5zJSInQln/o6d2tqIe4GnCY33vzPyDuLf2WJ4xcAwImfqRARS6Gua3wxE2a4lBV1YnzlkL5X4eaiOctjUR588E+RzUAPkySUAvFY= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1761861077688790.4172696331963; Thu, 30 Oct 2025 14:51:17 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1vEaWv-0006nY-KI; Thu, 30 Oct 2025 17:50:01 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1vEaWq-0006mW-98 for qemu-devel@nongnu.org; Thu, 30 Oct 2025 17:49:57 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1vEaWd-0000au-3k for qemu-devel@nongnu.org; Thu, 30 Oct 2025 17:49:56 -0400 Received: from mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (ec2-35-165-154-97.us-west-2.compute.amazonaws.com [35.165.154.97]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-662-Q9UmRYkSMsuLePEp0JFvAw-1; Thu, 30 Oct 2025 17:49:37 -0400 Received: from mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.93]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id D320E1809A02; Thu, 30 Oct 2025 21:49:35 +0000 (UTC) Received: from fedora.redhat.com (unknown [10.45.224.58]) by mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id ACA3F1800587; Thu, 30 Oct 2025 21:49:33 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1761860980; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=IuZ7wefnnYr02Da6Q6UYgXoyQkAcXJIfUpdX+fPhoSk=; b=TA448DjyH5D54NhH271/OifnPZDWvBCY2KcyD6o3TxzIuJBiKb5yeO17BvX4ZdPW+YkiS2 ma6an/X9NmaGXxrhPfdo3N0M9I08rA8jeog6l35TJk1o/A1jUwOS3f1wXz9YiTlTnPJeFx S+7xqFVJ/ArAJavEkhezQ7k8FxM7rIw= X-MC-Unique: Q9UmRYkSMsuLePEp0JFvAw-1 X-Mimecast-MFC-AGG-ID: Q9UmRYkSMsuLePEp0JFvAw_1761860976 From: Juraj Marcin To: qemu-devel@nongnu.org Cc: Juraj Marcin , Peter Xu , "Dr. David Alan Gilbert" , Jiri Denemark , Fabiano Rosas Subject: [PATCH v3 3/7] migration: Introduce postcopy incoming setup and cleanup functions Date: Thu, 30 Oct 2025 22:49:07 +0100 Message-ID: <20251030214915.1411860-4-jmarcin@redhat.com> In-Reply-To: <20251030214915.1411860-1-jmarcin@redhat.com> References: <20251030214915.1411860-1-jmarcin@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.93 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=jmarcin@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1761861078982154100 Content-Type: text/plain; charset="utf-8" From: Juraj Marcin After moving postcopy_ram_listen_thread() to postcopy file, this patch introduces a pair of functions, postcopy_incoming_setup() and postcopy_incoming_cleanup(). These functions encapsulate setup and cleanup of all incoming postcopy resources, postcopy-ram and postcopy listen thread. Furthermore, this patch also renames the postcopy_ram_listen_thread to postcopy_listen_thread, as this thread handles not only postcopy-ram, but also dirty-bitmaps and in the future it could handle other postcopiable devices. Signed-off-by: Juraj Marcin Reviewed-by: Peter Xu --- migration/migration.c | 2 +- migration/postcopy-ram.c | 46 ++++++++++++++++++++++++++++++++++++---- migration/postcopy-ram.h | 3 ++- migration/savevm.c | 25 ++-------------------- 4 files changed, 47 insertions(+), 29 deletions(-) diff --git a/migration/migration.c b/migration/migration.c index 6e647c7c4a..9a367f717e 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -892,7 +892,7 @@ process_incoming_migration_co(void *opaque) * but managed to complete within the precopy period, we can u= se * the normal exit. */ - postcopy_ram_incoming_cleanup(mis); + postcopy_incoming_cleanup(mis); } else if (ret >=3D 0) { /* * Postcopy was started, cleanup should happen at the end of t= he diff --git a/migration/postcopy-ram.c b/migration/postcopy-ram.c index 36d5415554..b47c955763 100644 --- a/migration/postcopy-ram.c +++ b/migration/postcopy-ram.c @@ -2082,10 +2082,8 @@ bool postcopy_is_paused(MigrationStatus status) * Triggered by a postcopy_listen command; this thread takes over reading * the input stream, leaving the main thread free to carry on loading the = rest * of the device state (from RAM). - * (TODO:This could do with being in a postcopy file - but there again it's - * just another input loop, not that postcopy specific) */ -void *postcopy_ram_listen_thread(void *opaque) +static void *postcopy_listen_thread(void *opaque) { MigrationIncomingState *mis =3D migration_incoming_get_current(); QEMUFile *f =3D mis->from_src_file; @@ -2151,7 +2149,7 @@ void *postcopy_ram_listen_thread(void *opaque) */ qemu_event_wait(&mis->main_thread_load_event); } - postcopy_ram_incoming_cleanup(mis); + postcopy_incoming_cleanup(mis); =20 if (load_res < 0) { /* @@ -2184,3 +2182,43 @@ void *postcopy_ram_listen_thread(void *opaque) =20 return NULL; } + +int postcopy_incoming_setup(MigrationIncomingState *mis, Error **errp) +{ + /* + * Sensitise RAM - can now generate requests for blocks that don't exi= st + * However, at this point the CPU shouldn't be running, and the IO + * shouldn't be doing anything yet so don't actually expect requests + */ + if (migrate_postcopy_ram()) { + if (postcopy_ram_incoming_setup(mis)) { + postcopy_ram_incoming_cleanup(mis); + error_setg(errp, "Failed to setup incoming postcopy RAM blocks= "); + return -1; + } + } + + trace_loadvm_postcopy_handle_listen("after uffd"); + + if (postcopy_notify(POSTCOPY_NOTIFY_INBOUND_LISTEN, errp)) { + return -1; + } + + mis->have_listen_thread =3D true; + postcopy_thread_create(mis, &mis->listen_thread, + MIGRATION_THREAD_DST_LISTEN, + postcopy_listen_thread, QEMU_THREAD_DETACHED); + + return 0; +} + +int postcopy_incoming_cleanup(MigrationIncomingState *mis) +{ + int rc =3D 0; + + if (migrate_postcopy_ram()) { + rc =3D postcopy_ram_incoming_cleanup(mis); + } + + return rc; +} diff --git a/migration/postcopy-ram.h b/migration/postcopy-ram.h index 3e26db3e6b..a080dd65a7 100644 --- a/migration/postcopy-ram.h +++ b/migration/postcopy-ram.h @@ -199,6 +199,7 @@ bool postcopy_is_paused(MigrationStatus status); void mark_postcopy_blocktime_begin(uintptr_t addr, uint32_t ptid, RAMBlock *rb); =20 -void *postcopy_ram_listen_thread(void *opaque); +int postcopy_incoming_setup(MigrationIncomingState *mis, Error **errp); +int postcopy_incoming_cleanup(MigrationIncomingState *mis); =20 #endif diff --git a/migration/savevm.c b/migration/savevm.c index 97fdd08c08..6ae3f740b5 100644 --- a/migration/savevm.c +++ b/migration/savevm.c @@ -2112,32 +2112,11 @@ static int loadvm_postcopy_handle_listen(MigrationI= ncomingState *mis, =20 trace_loadvm_postcopy_handle_listen("after discard"); =20 - /* - * Sensitise RAM - can now generate requests for blocks that don't exi= st - * However, at this point the CPU shouldn't be running, and the IO - * shouldn't be doing anything yet so don't actually expect requests - */ - if (migrate_postcopy_ram()) { - if (postcopy_ram_incoming_setup(mis)) { - postcopy_ram_incoming_cleanup(mis); - error_setg(errp, "Failed to setup incoming postcopy RAM blocks= "); - return -1; - } - } + int rc =3D postcopy_incoming_setup(mis, errp); =20 - trace_loadvm_postcopy_handle_listen("after uffd"); - - if (postcopy_notify(POSTCOPY_NOTIFY_INBOUND_LISTEN, errp)) { - return -1; - } - - mis->have_listen_thread =3D true; - postcopy_thread_create(mis, &mis->listen_thread, - MIGRATION_THREAD_DST_LISTEN, - postcopy_ram_listen_thread, QEMU_THREAD_DETACHE= D); trace_loadvm_postcopy_handle_listen("return"); =20 - return 0; + return rc; } =20 static void loadvm_postcopy_handle_run_bh(void *opaque) --=20 2.51.0 From nobody Fri Nov 14 16:47:11 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1761861167; cv=none; d=zohomail.com; s=zohoarc; b=BDAqlLvlWBzBlt373Q1P+7gwX/AIXtQdWnx66DmBPYkCk/es+yPeMaTMPT2azm379UUslF3iAEbwEJCudwGx1GBMmOcgPNff13tl++ZaooS2N3SOX508pw+9+N8mnLeTliSfZIs3vMwehg26ywWYi4yySIzKGUD3KMtiu+cWdak= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1761861167; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=kpQ1d9E8ywS7tsGQjPzaihaMRcKLRk5yK7Fv7M8iwJI=; b=eNoK9PQd1sLXTTlAlu06ibRXVf2cIwnqRuguLwEHNJd4q1sULOJIPk0r7kihU5tTFIUyK3dz0QIIXMpiwNYOuzuQBzhl/jwgulTS6I+ZKyruHf+RFOswet3THUUUTXnldEtk4TwOv/zpFHjOHUr3D5phaanR/LpPWTnsBbbN5Bg= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1761861167035514.8551253736006; Thu, 30 Oct 2025 14:52:47 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1vEaWz-0006qk-Qu; Thu, 30 Oct 2025 17:50:05 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1vEaWx-0006ok-7n for qemu-devel@nongnu.org; Thu, 30 Oct 2025 17:50:03 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1vEaWg-0000cg-N8 for qemu-devel@nongnu.org; Thu, 30 Oct 2025 17:50:02 -0400 Received: from mx-prod-mc-03.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-678-KU6RiZMQMb-3kVZ643P1Jg-1; Thu, 30 Oct 2025 17:49:40 -0400 Received: from mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.93]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-03.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 95C261955F2A; Thu, 30 Oct 2025 21:49:39 +0000 (UTC) Received: from fedora.redhat.com (unknown [10.45.224.58]) by mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 84F8E1800579; Thu, 30 Oct 2025 21:49:37 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1761860983; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=kpQ1d9E8ywS7tsGQjPzaihaMRcKLRk5yK7Fv7M8iwJI=; b=Uz+HQOTfbExsf30ZEyfIal7mEP6hSewiOsLox0ZZRnR/17JN6xAV/JpZ0uAGEH68kNYKOM ken+79waWoS26VpXiKyuLCpRzddOQW4pk0jskmvq4nT6RBMLdl33cm9Z7hF825Ke6lS37j UPjKjrOAwFzOSOue69DO78x/ItLexbY= X-MC-Unique: KU6RiZMQMb-3kVZ643P1Jg-1 X-Mimecast-MFC-AGG-ID: KU6RiZMQMb-3kVZ643P1Jg_1761860979 From: Juraj Marcin To: qemu-devel@nongnu.org Cc: Juraj Marcin , Peter Xu , "Dr. David Alan Gilbert" , Jiri Denemark , Fabiano Rosas Subject: [PATCH v3 4/7] migration: Refactor all incoming cleanup info migration_incoming_destroy() Date: Thu, 30 Oct 2025 22:49:08 +0100 Message-ID: <20251030214915.1411860-5-jmarcin@redhat.com> In-Reply-To: <20251030214915.1411860-1-jmarcin@redhat.com> References: <20251030214915.1411860-1-jmarcin@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.93 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=jmarcin@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1761861169702154100 Content-Type: text/plain; charset="utf-8" From: Juraj Marcin Currently, there are two functions that are responsible for calling the cleanup of the incoming migration state. With successful precopy, it's the incoming migration coroutine, and with successful postcopy it's the postcopy listen thread. However, if postcopy fails during in the device load, both functions will try to do the cleanup. This patch refactors all cleanup that needs to be done on the incoming side into a common function and defines a clear boundary, who is responsible for the cleanup. The incoming migration coroutine is responsible for calling the cleanup function, unless the listen thread has been started, in which case the postcopy listen thread runs the incoming migration cleanup in its BH. Signed-off-by: Juraj Marcin Reviewed-by: Peter Xu --- migration/migration.c | 44 +++++++++------------------- migration/migration.h | 1 + migration/postcopy-ram.c | 63 +++++++++++++++++++++------------------- migration/trace-events | 2 +- 4 files changed, 49 insertions(+), 61 deletions(-) diff --git a/migration/migration.c b/migration/migration.c index 9a367f717e..637be71bfe 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -438,10 +438,15 @@ void migration_incoming_transport_cleanup(MigrationIn= comingState *mis) =20 void migration_incoming_state_destroy(void) { - struct MigrationIncomingState *mis =3D migration_incoming_get_current(= ); + MigrationIncomingState *mis =3D migration_incoming_get_current(); + PostcopyState ps =3D postcopy_state_get(); =20 multifd_recv_cleanup(); =20 + if (ps !=3D POSTCOPY_INCOMING_NONE) { + postcopy_incoming_cleanup(mis); + } + /* * RAM state cleanup needs to happen after multifd cleanup, because * multifd threads can use some of its states (receivedmap). @@ -866,7 +871,6 @@ process_incoming_migration_co(void *opaque) { MigrationState *s =3D migrate_get_current(); MigrationIncomingState *mis =3D migration_incoming_get_current(); - PostcopyState ps; int ret; Error *local_err =3D NULL; =20 @@ -883,25 +887,14 @@ process_incoming_migration_co(void *opaque) =20 trace_vmstate_downtime_checkpoint("dst-precopy-loadvm-completed"); =20 - ps =3D postcopy_state_get(); - trace_process_incoming_migration_co_end(ret, ps); - if (ps !=3D POSTCOPY_INCOMING_NONE) { - if (ps =3D=3D POSTCOPY_INCOMING_ADVISE) { - /* - * Where a migration had postcopy enabled (and thus went to ad= vise) - * but managed to complete within the precopy period, we can u= se - * the normal exit. - */ - postcopy_incoming_cleanup(mis); - } else if (ret >=3D 0) { - /* - * Postcopy was started, cleanup should happen at the end of t= he - * postcopy thread. - */ - trace_process_incoming_migration_co_postcopy_end_main(); - goto out; - } - /* Else if something went wrong then just fall out of the normal e= xit */ + trace_process_incoming_migration_co_end(ret); + if (mis->have_listen_thread) { + /* + * Postcopy was started, cleanup should happen at the end of the + * postcopy listen thread. + */ + trace_process_incoming_migration_co_postcopy_end_main(); + goto out; } =20 if (ret < 0) { @@ -933,15 +926,6 @@ fail: } =20 exit(EXIT_FAILURE); - } else { - /* - * Report the error here in case that QEMU abruptly exits - * when postcopy is enabled. - */ - WITH_QEMU_LOCK_GUARD(&s->error_mutex) { - error_report_err(s->error); - s->error =3D NULL; - } } out: /* Pairs with the refcount taken in qmp_migrate_incoming() */ diff --git a/migration/migration.h b/migration/migration.h index 01329bf824..4a37f7202c 100644 --- a/migration/migration.h +++ b/migration/migration.h @@ -254,6 +254,7 @@ struct MigrationIncomingState { MigrationIncomingState *migration_incoming_get_current(void); void migration_incoming_state_destroy(void); void migration_incoming_transport_cleanup(MigrationIncomingState *mis); +void migration_incoming_qemu_exit(void); /* * Functions to work with blocktime context */ diff --git a/migration/postcopy-ram.c b/migration/postcopy-ram.c index b47c955763..48cbb46c27 100644 --- a/migration/postcopy-ram.c +++ b/migration/postcopy-ram.c @@ -2078,6 +2078,24 @@ bool postcopy_is_paused(MigrationStatus status) status =3D=3D MIGRATION_STATUS_POSTCOPY_RECOVER_SETUP; } =20 +static void postcopy_listen_thread_bh(void *opaque) +{ + MigrationIncomingState *mis =3D migration_incoming_get_current(); + + migration_incoming_state_destroy(); + + if (mis->state =3D=3D MIGRATION_STATUS_FAILED) { + /* + * If something went wrong then we have a bad state so exit; + * we only could have gotten here if something failed before + * POSTCOPY_INCOMING_RUNNING (for example device load), otherwise + * postcopy migration would pause inside qemu_loadvm_state_main(). + * Failing dirty-bitmaps won't fail the whole migration. + */ + exit(1); + } +} + /* * Triggered by a postcopy_listen command; this thread takes over reading * the input stream, leaving the main thread free to carry on loading the = rest @@ -2131,53 +2149,38 @@ static void *postcopy_listen_thread(void *opaque) "bitmaps are correctly migrated and valid.", __func__, load_res, error_get_pretty(local_err)); g_clear_pointer(&local_err, error_free); - load_res =3D 0; /* prevent further exit() */ } else { + /* + * Something went fatally wrong and we have a bad state, QEMU = will + * exit depending on if postcopy-exit-on-error is true, but the + * migration cannot be recovered. + */ error_prepend(&local_err, "loadvm failed during postcopy: %d: ", load_res); migrate_set_error(migr, local_err); g_clear_pointer(&local_err, error_report_err); migrate_set_state(&mis->state, MIGRATION_STATUS_POSTCOPY_ACTIV= E, MIGRATION_STATUS_FAILED); + goto out; } } - if (load_res >=3D 0) { - /* - * This looks good, but it's possible that the device loading in t= he - * main thread hasn't finished yet, and so we might not be in 'RUN' - * state yet; wait for the end of the main thread. - */ - qemu_event_wait(&mis->main_thread_load_event); - } - postcopy_incoming_cleanup(mis); - - if (load_res < 0) { - /* - * If something went wrong then we have a bad state so exit; - * depending how far we got it might be possible at this point - * to leave the guest running and fire MCEs for pages that never - * arrived as a desperate recovery step. - */ - rcu_unregister_thread(); - exit(EXIT_FAILURE); - } + /* + * This looks good, but it's possible that the device loading in the + * main thread hasn't finished yet, and so we might not be in 'RUN' + * state yet; wait for the end of the main thread. + */ + qemu_event_wait(&mis->main_thread_load_event); =20 migrate_set_state(&mis->state, MIGRATION_STATUS_POSTCOPY_ACTIVE, MIGRATION_STATUS_COMPLETED); - /* - * If everything has worked fine, then the main thread has waited - * for us to start, and we're the last use of the mis. - * (If something broke then qemu will have to exit anyway since it's - * got a bad migration state). - */ - bql_lock(); - migration_incoming_state_destroy(); - bql_unlock(); =20 +out: rcu_unregister_thread(); mis->have_listen_thread =3D false; postcopy_state_set(POSTCOPY_INCOMING_END); =20 + migration_bh_schedule(postcopy_listen_thread_bh, NULL); + object_unref(OBJECT(migr)); =20 return NULL; diff --git a/migration/trace-events b/migration/trace-events index e8edd1fbba..772636f3ac 100644 --- a/migration/trace-events +++ b/migration/trace-events @@ -193,7 +193,7 @@ source_return_path_thread_resume_ack(uint32_t v) "%"PRI= u32 source_return_path_thread_switchover_acked(void) "" migration_thread_low_pending(uint64_t pending) "%" PRIu64 migrate_transferred(uint64_t transferred, uint64_t time_spent, uint64_t ba= ndwidth, uint64_t avail_bw, uint64_t size) "transferred %" PRIu64 " time_sp= ent %" PRIu64 " bandwidth %" PRIu64 " switchover_bw %" PRIu64 " max_size %"= PRId64 -process_incoming_migration_co_end(int ret, int ps) "ret=3D%d postcopy-stat= e=3D%d" +process_incoming_migration_co_end(int ret) "ret=3D%d" process_incoming_migration_co_postcopy_end_main(void) "" postcopy_preempt_enabled(bool value) "%d" migration_precopy_complete(void) "" --=20 2.51.0 From nobody Fri Nov 14 16:47:11 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1761861079; cv=none; d=zohomail.com; s=zohoarc; b=RDDYXqP5OZPezn8D5hkHB3E2OE8UhaXJTl8ncZdLiHtzQzRd40RXxhXXmvtJpx9dJXohuLWwOaV7cjnS85h1vMrfjvC/9whhFcvpnzAuVaO2kTLhwbRDZu+rjvI1fh57LCTxaeiWqy+Lfo5NxgJ1Nh4h+KZ0C8Aty4LM05bmMp8= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1761861079; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=iFLNI3Xt+VhG5ZpFCctys9d2h8+theDhwcHMrg2Y7jg=; b=DbGS2sL8kfQ1QC4kooxWqLuOQ6RWrs1k54zfLkM29mTidvZXTc8yBwu05flM9cUwc8t9RvbewTb2SANPeHuPwu+9oVZNq+RlLl/oxe7npWmwv2V3mauGiL3zdAJ8C++z71blG34lpegOady9EkwRZem87Dwq276eav1bETV2fAw= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1761861079766967.8533837206817; Thu, 30 Oct 2025 14:51:19 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1vEaX1-0006uE-PO; Thu, 30 Oct 2025 17:50:07 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1vEaWy-0006pn-Lt for qemu-devel@nongnu.org; Thu, 30 Oct 2025 17:50:05 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1vEaWm-0000fK-0m for qemu-devel@nongnu.org; Thu, 30 Oct 2025 17:50:02 -0400 Received: from mx-prod-mc-03.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-679-0Ka8PTcNMO-0qHqQ-wbv7A-1; Thu, 30 Oct 2025 17:49:44 -0400 Received: from mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.93]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-03.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 344AE1955F2D; Thu, 30 Oct 2025 21:49:43 +0000 (UTC) Received: from fedora.redhat.com (unknown [10.45.224.58]) by mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 446DC1800579; Thu, 30 Oct 2025 21:49:41 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1761860987; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=iFLNI3Xt+VhG5ZpFCctys9d2h8+theDhwcHMrg2Y7jg=; b=gDNOjl9nAKPJWAF/Xp/eLYnky2t0ilbzdwBk+yGj22kmVw0jY81p1PdS/aAcdsVIoK2J6b /ezS7gWI1C+y5zqBe2Ya9KVvyC53gr20kcy9+zpu8QAYtYJR/Gy2SwTUNs2TiFUENDADjp rFTlFq8AleFYOU6y5yGHEUOTGRAcc/M= X-MC-Unique: 0Ka8PTcNMO-0qHqQ-wbv7A-1 X-Mimecast-MFC-AGG-ID: 0Ka8PTcNMO-0qHqQ-wbv7A_1761860983 From: Juraj Marcin To: qemu-devel@nongnu.org Cc: Juraj Marcin , Peter Xu , "Dr. David Alan Gilbert" , Jiri Denemark , Fabiano Rosas Subject: [PATCH v3 5/7] migration: Respect exit-on-error when migration fails before resuming Date: Thu, 30 Oct 2025 22:49:09 +0100 Message-ID: <20251030214915.1411860-6-jmarcin@redhat.com> In-Reply-To: <20251030214915.1411860-1-jmarcin@redhat.com> References: <20251030214915.1411860-1-jmarcin@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.93 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=jmarcin@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1761861090020158500 Content-Type: text/plain; charset="utf-8" From: Juraj Marcin When exit-on-error was added to migration, it wasn't added to postcopy. Even though postcopy migration will usually pause and not fail, in cases it does unrecoverably fail before destination side has been started, exit-on-error will allow management to query the error. Signed-off-by: Juraj Marcin Reviewed-by: Peter Xu --- migration/postcopy-ram.c | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/migration/postcopy-ram.c b/migration/postcopy-ram.c index 48cbb46c27..91431f02a4 100644 --- a/migration/postcopy-ram.c +++ b/migration/postcopy-ram.c @@ -2080,11 +2080,16 @@ bool postcopy_is_paused(MigrationStatus status) =20 static void postcopy_listen_thread_bh(void *opaque) { + MigrationState *s =3D migrate_get_current(); MigrationIncomingState *mis =3D migration_incoming_get_current(); =20 migration_incoming_state_destroy(); =20 - if (mis->state =3D=3D MIGRATION_STATUS_FAILED) { + if (mis->state =3D=3D MIGRATION_STATUS_FAILED && mis->exit_on_error) { + WITH_QEMU_LOCK_GUARD(&s->error_mutex) { + error_report_err(s->error); + s->error =3D NULL; + } /* * If something went wrong then we have a bad state so exit; * we only could have gotten here if something failed before --=20 2.51.0 From nobody Fri Nov 14 16:47:11 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1761861131; cv=none; d=zohomail.com; s=zohoarc; b=ETPrPGEyPQMWHDZ0w8Yi4fSVArCt1iR7Lv7BXlJSXV7kjBjpaTqqAPfWMwz6l5L/Yji1qtX8SANoL+oBB6V+2wQzkSiKu5hFqeiqWZb9XASNykRr5xb9TZDOSZ43u1ly9pHzzS9DPbwfaFNe9gQ/x8rUkClzad1QKAVhf9tgbdE= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1761861131; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=y6wqd5lciJsAo9IQM8zPf6C+QCoE2/o8d67nBJyAwDY=; b=k7oYQsYf5Sqbpjqy5CqqsUnVrM9wFlA7urTFxu+OviaeFlRBVGwXh9JeVeQm/6YzIHN7VdG/e3bQJeE9bm98bGvc7ncwoDSypSEz4RiM4wgeW1NGUGBvg37QfUENwZAz/GdaBXoC+6tzBcBGm/+jtPerq+zR9r81JPst9sbCArI= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 176186113138471.24176714260705; Thu, 30 Oct 2025 14:52:11 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1vEaX8-0006vS-Ov; Thu, 30 Oct 2025 17:50:14 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1vEaX4-0006vF-Ky for qemu-devel@nongnu.org; Thu, 30 Oct 2025 17:50:10 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1vEaWt-0000gR-Ou for qemu-devel@nongnu.org; Thu, 30 Oct 2025 17:50:08 -0400 Received: from mx-prod-mc-03.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-681-x27j4jd0O9ahP7qU_g8lRQ-1; Thu, 30 Oct 2025 17:49:47 -0400 Received: from mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.93]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-03.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id B0F7A1955F3B; Thu, 30 Oct 2025 21:49:46 +0000 (UTC) Received: from fedora.redhat.com (unknown [10.45.224.58]) by mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id C0CA71800586; Thu, 30 Oct 2025 21:49:44 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1761860990; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=y6wqd5lciJsAo9IQM8zPf6C+QCoE2/o8d67nBJyAwDY=; b=jMNge5ZZvrMDu+fqJX5BxABsm7mwuVFLk7ue82E8gNycZ18WM+DRoC/Wr023+8bO2hD5bc Uw7KFiwW8SfVCua72qUKo4vxSwn9YXjW/FW/iZv8ayYDvMFJPqeOY5nuPGwpArj4Ij58Yz JUH09VT+v5rQUBOn03YO7c/Ip45otVA= X-MC-Unique: x27j4jd0O9ahP7qU_g8lRQ-1 X-Mimecast-MFC-AGG-ID: x27j4jd0O9ahP7qU_g8lRQ_1761860986 From: Juraj Marcin To: qemu-devel@nongnu.org Cc: Juraj Marcin , Peter Xu , "Dr. David Alan Gilbert" , Jiri Denemark , Fabiano Rosas Subject: [PATCH v3 6/7] migration: Make postcopy listen thread joinable Date: Thu, 30 Oct 2025 22:49:10 +0100 Message-ID: <20251030214915.1411860-7-jmarcin@redhat.com> In-Reply-To: <20251030214915.1411860-1-jmarcin@redhat.com> References: <20251030214915.1411860-1-jmarcin@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.93 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=jmarcin@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.001, SPF_HELO_PASS=-0.001, T_SPF_TEMPERROR=0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1761861133063154100 Content-Type: text/plain; charset="utf-8" From: Juraj Marcin This patch makes the listen thread joinable instead detached, and joins it alongside other postcopy threads. Signed-off-by: Juraj Marcin Reviewed-by: Peter Xu --- migration/postcopy-ram.c | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-) diff --git a/migration/postcopy-ram.c b/migration/postcopy-ram.c index 91431f02a4..8405cce7b4 100644 --- a/migration/postcopy-ram.c +++ b/migration/postcopy-ram.c @@ -2181,7 +2181,6 @@ static void *postcopy_listen_thread(void *opaque) =20 out: rcu_unregister_thread(); - mis->have_listen_thread =3D false; postcopy_state_set(POSTCOPY_INCOMING_END); =20 migration_bh_schedule(postcopy_listen_thread_bh, NULL); @@ -2215,7 +2214,7 @@ int postcopy_incoming_setup(MigrationIncomingState *m= is, Error **errp) mis->have_listen_thread =3D true; postcopy_thread_create(mis, &mis->listen_thread, MIGRATION_THREAD_DST_LISTEN, - postcopy_listen_thread, QEMU_THREAD_DETACHED); + postcopy_listen_thread, QEMU_THREAD_JOINABLE); =20 return 0; } @@ -2224,6 +2223,11 @@ int postcopy_incoming_cleanup(MigrationIncomingState= *mis) { int rc =3D 0; =20 + if (mis->have_listen_thread) { + qemu_thread_join(&mis->listen_thread); + mis->have_listen_thread =3D false; + } + if (migrate_postcopy_ram()) { rc =3D postcopy_ram_incoming_cleanup(mis); } --=20 2.51.0 From nobody Fri Nov 14 16:47:11 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1761861080; cv=none; d=zohomail.com; s=zohoarc; b=FNS4/RHH0E0Pltu+Ujiw7BYH+7qRTzoq5PvAnoHl/u2yT8kBb+qci5SemNZ3+wSrZrqjnVj/6LSlmqmbIgVeapLfbhb9ey0cb1WjUI3MCoBok8UOywyBTvNSLGqoOz+0RR+u8RnVv99Au7Tp0hnC8eJVRAJqKQbWWReewfPAWUM= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1761861080; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=FS8GflE7nNCmyxJ4v84LIBXO8gCcCi937lmuHGnHuHo=; b=Hfc89DXX56OK8z3YmTjRrf8W7wJsbqDeyou6ktBHBKFu2CUrQ3TDH8sEdsFcf9Urrclw1L0fcvUYg3krupf1ynPgiA3SfgchzRFRYcfm7vfZxyPlamBwm4usQWXTQ2jojf6p7O24WRTDvT2z1Z+oc8XVHTd0eq/8I3wGyZNjS7o= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 17618610800841021.1796146326709; Thu, 30 Oct 2025 14:51:20 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1vEaXE-0006wM-E7; Thu, 30 Oct 2025 17:50:22 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1vEaX8-0006vN-Kg for qemu-devel@nongnu.org; Thu, 30 Oct 2025 17:50:14 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1vEaWt-0000hg-Pj for qemu-devel@nongnu.org; Thu, 30 Oct 2025 17:50:11 -0400 Received: from mx-prod-mc-03.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-171-ZLpXnQl-NnC3gGJkYeqpWA-1; Thu, 30 Oct 2025 17:49:52 -0400 Received: from mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.93]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-03.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 65E98195606F; Thu, 30 Oct 2025 21:49:50 +0000 (UTC) Received: from fedora.redhat.com (unknown [10.45.224.58]) by mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 752ED1800583; Thu, 30 Oct 2025 21:49:48 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1761860994; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=FS8GflE7nNCmyxJ4v84LIBXO8gCcCi937lmuHGnHuHo=; b=PIWMR7Bb0RlRD75v+EFLrYMbmsJZp4kk+MRqyi5bZ80xx9dpiQI49heoMHZPncLfoIIdwe L0fz0+o0ziUR4U/3j8xNU4oOWmzrL9odllXFxMaqQyPktZS5d9STMngzlb+VyG1E0SOiTU i4RTZWJNJepyWU5F6dyi9fKr8U0dLoo= X-MC-Unique: ZLpXnQl-NnC3gGJkYeqpWA-1 X-Mimecast-MFC-AGG-ID: ZLpXnQl-NnC3gGJkYeqpWA_1761860990 From: Juraj Marcin To: qemu-devel@nongnu.org Cc: Juraj Marcin , Peter Xu , "Dr. David Alan Gilbert" , Jiri Denemark , Fabiano Rosas Subject: [PATCH v3 7/7] migration: Introduce POSTCOPY_DEVICE state Date: Thu, 30 Oct 2025 22:49:11 +0100 Message-ID: <20251030214915.1411860-8-jmarcin@redhat.com> In-Reply-To: <20251030214915.1411860-1-jmarcin@redhat.com> References: <20251030214915.1411860-1-jmarcin@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.93 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=jmarcin@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.001, SPF_PASS=-0.001, T_SPF_HELO_TEMPERROR=0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1761861082904154100 Content-Type: text/plain; charset="utf-8" From: Juraj Marcin Currently, when postcopy starts, the source VM starts switchover and sends a package containing the state of all non-postcopiable devices. When the destination loads this package, the switchover is complete and the destination VM starts. However, if the device state load fails or the destination side crashes, the source side is already in POSTCOPY_ACTIVE state and cannot be recovered, even when it has the most up-to-date machine state as the destination has not yet started. This patch introduces a new POSTCOPY_DEVICE state which is active while the destination machine is loading the device state, is not yet running, and the source side can be resumed in case of a migration failure. To transition from POSTCOPY_DEVICE to POSTCOPY_ACTIVE, the source side uses a PONG message that is a response to a PING message processed just before the POSTCOPY_RUN command that starts the destination VM. Thus, this feature is effective even if the destination side does not yet support this new state. Signed-off-by: Juraj Marcin Reviewed-by: Peter Xu --- migration/migration.c | 43 ++++++++++++++++++++++++--- migration/migration.h | 3 ++ migration/postcopy-ram.c | 9 ++++-- migration/savevm.c | 2 ++ migration/savevm.h | 2 ++ migration/trace-events | 1 + qapi/migration.json | 8 +++-- tests/qtest/migration/precopy-tests.c | 3 +- 8 files changed, 61 insertions(+), 10 deletions(-) diff --git a/migration/migration.c b/migration/migration.c index 637be71bfe..76223cb940 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -1206,6 +1206,7 @@ bool migration_is_running(void) =20 switch (s->state) { case MIGRATION_STATUS_ACTIVE: + case MIGRATION_STATUS_POSTCOPY_DEVICE: case MIGRATION_STATUS_POSTCOPY_ACTIVE: case MIGRATION_STATUS_POSTCOPY_PAUSED: case MIGRATION_STATUS_POSTCOPY_RECOVER_SETUP: @@ -1227,6 +1228,7 @@ static bool migration_is_active(void) MigrationState *s =3D current_migration; =20 return (s->state =3D=3D MIGRATION_STATUS_ACTIVE || + s->state =3D=3D MIGRATION_STATUS_POSTCOPY_DEVICE || s->state =3D=3D MIGRATION_STATUS_POSTCOPY_ACTIVE); } =20 @@ -1349,6 +1351,7 @@ static void fill_source_migration_info(MigrationInfo = *info) break; case MIGRATION_STATUS_ACTIVE: case MIGRATION_STATUS_CANCELLING: + case MIGRATION_STATUS_POSTCOPY_DEVICE: case MIGRATION_STATUS_POSTCOPY_ACTIVE: case MIGRATION_STATUS_PRE_SWITCHOVER: case MIGRATION_STATUS_DEVICE: @@ -1402,6 +1405,7 @@ static void fill_destination_migration_info(Migration= Info *info) case MIGRATION_STATUS_CANCELLING: case MIGRATION_STATUS_CANCELLED: case MIGRATION_STATUS_ACTIVE: + case MIGRATION_STATUS_POSTCOPY_DEVICE: case MIGRATION_STATUS_POSTCOPY_ACTIVE: case MIGRATION_STATUS_POSTCOPY_PAUSED: case MIGRATION_STATUS_POSTCOPY_RECOVER: @@ -1732,6 +1736,7 @@ bool migration_in_postcopy(void) MigrationState *s =3D migrate_get_current(); =20 switch (s->state) { + case MIGRATION_STATUS_POSTCOPY_DEVICE: case MIGRATION_STATUS_POSTCOPY_ACTIVE: case MIGRATION_STATUS_POSTCOPY_PAUSED: case MIGRATION_STATUS_POSTCOPY_RECOVER_SETUP: @@ -1833,6 +1838,9 @@ int migrate_init(MigrationState *s, Error **errp) memset(&mig_stats, 0, sizeof(mig_stats)); migration_reset_vfio_bytes_transferred(); =20 + s->postcopy_package_loaded =3D false; + qemu_event_reset(&s->postcopy_package_loaded_event); + return 0; } =20 @@ -2568,6 +2576,11 @@ static void *source_return_path_thread(void *opaque) tmp32 =3D ldl_be_p(buf); trace_source_return_path_thread_pong(tmp32); qemu_sem_post(&ms->rp_state.rp_pong_acks); + if (tmp32 =3D=3D QEMU_VM_PING_PACKAGED_LOADED) { + trace_source_return_path_thread_postcopy_package_loaded(); + ms->postcopy_package_loaded =3D true; + qemu_event_set(&ms->postcopy_package_loaded_event); + } break; =20 case MIG_RP_MSG_REQ_PAGES: @@ -2813,6 +2826,13 @@ static int postcopy_start(MigrationState *ms, Error = **errp) if (migrate_postcopy_ram()) { qemu_savevm_send_ping(fb, 3); } + /* + * This ping will tell us that all non-postcopiable device state has b= een + * successfully loaded and the destination is about to start. When res= ponse + * is received, it will trigger transition from POSTCOPY_DEVICE to + * POSTCOPY_ACTIVE state. + */ + qemu_savevm_send_ping(fb, QEMU_VM_PING_PACKAGED_LOADED); =20 qemu_savevm_send_postcopy_run(fb); =20 @@ -2870,7 +2890,7 @@ static int postcopy_start(MigrationState *ms, Error *= *errp) =20 /* Now, switchover looks all fine, switching to postcopy-active */ migrate_set_state(&ms->state, MIGRATION_STATUS_DEVICE, - MIGRATION_STATUS_POSTCOPY_ACTIVE); + MIGRATION_STATUS_POSTCOPY_DEVICE); =20 bql_unlock(); =20 @@ -3311,8 +3331,8 @@ static MigThrError migration_detect_error(MigrationSt= ate *s) return postcopy_pause(s); } else { /* - * For precopy (or postcopy with error outside IO), we fail - * with no time. + * For precopy (or postcopy with error outside IO, or before dest + * starts), we fail with no time. */ migrate_set_state(&s->state, state, MIGRATION_STATUS_FAILED); trace_migration_thread_file_err(); @@ -3447,7 +3467,8 @@ static MigIterateState migration_iteration_run(Migrat= ionState *s) { uint64_t must_precopy, can_postcopy, pending_size; Error *local_err =3D NULL; - bool in_postcopy =3D s->state =3D=3D MIGRATION_STATUS_POSTCOPY_ACTIVE; + bool in_postcopy =3D (s->state =3D=3D MIGRATION_STATUS_POSTCOPY_DEVICE= || + s->state =3D=3D MIGRATION_STATUS_POSTCOPY_ACTIVE); bool can_switchover =3D migration_can_switchover(s); bool complete_ready; =20 @@ -3463,6 +3484,18 @@ static MigIterateState migration_iteration_run(Migra= tionState *s) * POSTCOPY_ACTIVE it means switchover already happened. */ complete_ready =3D !pending_size; + if (s->state =3D=3D MIGRATION_STATUS_POSTCOPY_DEVICE && + (s->postcopy_package_loaded || complete_ready)) { + /* + * If package has been loaded, the event is set and we will + * immediatelly transition to POSTCOPY_ACTIVE. If we are ready= for + * completion, we need to wait for destination to load the pos= tcopy + * package before actually completing. + */ + qemu_event_wait(&s->postcopy_package_loaded_event); + migrate_set_state(&s->state, MIGRATION_STATUS_POSTCOPY_DEVICE, + MIGRATION_STATUS_POSTCOPY_ACTIVE); + } } else { /* * Exact pending reporting is only needed for precopy. Taking RAM @@ -4117,6 +4150,7 @@ static void migration_instance_finalize(Object *obj) qemu_sem_destroy(&ms->rp_state.rp_pong_acks); qemu_sem_destroy(&ms->postcopy_qemufile_src_sem); error_free(ms->error); + qemu_event_destroy(&ms->postcopy_package_loaded_event); } =20 static void migration_instance_init(Object *obj) @@ -4138,6 +4172,7 @@ static void migration_instance_init(Object *obj) qemu_sem_init(&ms->wait_unplug_sem, 0); qemu_sem_init(&ms->postcopy_qemufile_src_sem, 0); qemu_mutex_init(&ms->qemu_file_lock); + qemu_event_init(&ms->postcopy_package_loaded_event, 0); } =20 /* diff --git a/migration/migration.h b/migration/migration.h index 4a37f7202c..213b33fe6e 100644 --- a/migration/migration.h +++ b/migration/migration.h @@ -510,6 +510,9 @@ struct MigrationState { /* Is this a rdma migration */ bool rdma_migration; =20 + bool postcopy_package_loaded; + QemuEvent postcopy_package_loaded_event; + GSource *hup_source; }; =20 diff --git a/migration/postcopy-ram.c b/migration/postcopy-ram.c index 8405cce7b4..c1e3b110b6 100644 --- a/migration/postcopy-ram.c +++ b/migration/postcopy-ram.c @@ -2117,7 +2117,7 @@ static void *postcopy_listen_thread(void *opaque) object_ref(OBJECT(migr)); =20 migrate_set_state(&mis->state, MIGRATION_STATUS_ACTIVE, - MIGRATION_STATUS_POSTCOPY_ACTIVE); + MIGRATION_STATUS_POSTCOPY_DEVICE); qemu_event_set(&mis->thread_sync_event); trace_postcopy_ram_listen_thread_start(); =20 @@ -2164,8 +2164,7 @@ static void *postcopy_listen_thread(void *opaque) "loadvm failed during postcopy: %d: ", load_res); migrate_set_error(migr, local_err); g_clear_pointer(&local_err, error_report_err); - migrate_set_state(&mis->state, MIGRATION_STATUS_POSTCOPY_ACTIV= E, - MIGRATION_STATUS_FAILED); + migrate_set_state(&mis->state, mis->state, MIGRATION_STATUS_FA= ILED); goto out; } } @@ -2176,6 +2175,10 @@ static void *postcopy_listen_thread(void *opaque) */ qemu_event_wait(&mis->main_thread_load_event); =20 + /* + * Device load in the main thread has finished, we should be in + * POSTCOPY_ACTIVE now. + */ migrate_set_state(&mis->state, MIGRATION_STATUS_POSTCOPY_ACTIVE, MIGRATION_STATUS_COMPLETED); =20 diff --git a/migration/savevm.c b/migration/savevm.c index 6ae3f740b5..ff01eb1a56 100644 --- a/migration/savevm.c +++ b/migration/savevm.c @@ -2169,6 +2169,8 @@ static int loadvm_postcopy_handle_run(MigrationIncomi= ngState *mis, Error **errp) return -1; } =20 + migrate_set_state(&mis->state, MIGRATION_STATUS_POSTCOPY_DEVICE, + MIGRATION_STATUS_POSTCOPY_ACTIVE); postcopy_state_set(POSTCOPY_INCOMING_RUNNING); migration_bh_schedule(loadvm_postcopy_handle_run_bh, mis); =20 diff --git a/migration/savevm.h b/migration/savevm.h index c337e3e3d1..125a2507b7 100644 --- a/migration/savevm.h +++ b/migration/savevm.h @@ -29,6 +29,8 @@ #define QEMU_VM_COMMAND 0x08 #define QEMU_VM_SECTION_FOOTER 0x7e =20 +#define QEMU_VM_PING_PACKAGED_LOADED 0x42 + bool qemu_savevm_state_blocked(Error **errp); void qemu_savevm_non_migratable_list(strList **reasons); int qemu_savevm_state_prepare(Error **errp); diff --git a/migration/trace-events b/migration/trace-events index 772636f3ac..bf11b62b17 100644 --- a/migration/trace-events +++ b/migration/trace-events @@ -191,6 +191,7 @@ source_return_path_thread_pong(uint32_t val) "0x%x" source_return_path_thread_shut(uint32_t val) "0x%x" source_return_path_thread_resume_ack(uint32_t v) "%"PRIu32 source_return_path_thread_switchover_acked(void) "" +source_return_path_thread_postcopy_package_loaded(void) "" migration_thread_low_pending(uint64_t pending) "%" PRIu64 migrate_transferred(uint64_t transferred, uint64_t time_spent, uint64_t ba= ndwidth, uint64_t avail_bw, uint64_t size) "transferred %" PRIu64 " time_sp= ent %" PRIu64 " bandwidth %" PRIu64 " switchover_bw %" PRIu64 " max_size %"= PRId64 process_incoming_migration_co_end(int ret) "ret=3D%d" diff --git a/qapi/migration.json b/qapi/migration.json index c7a6737cc1..b63de1bd36 100644 --- a/qapi/migration.json +++ b/qapi/migration.json @@ -142,6 +142,10 @@ # @postcopy-active: like active, but now in postcopy mode. # (since 2.5) # +# @postcopy-device: like postcopy-active, but the destination is still +# loading device state and is not running yet. If migration fails +# during this state, the source side will resume. (since 10.2) +# # @postcopy-paused: during postcopy but paused. (since 3.0) # # @postcopy-recover-setup: setup phase for a postcopy recovery @@ -173,8 +177,8 @@ ## { 'enum': 'MigrationStatus', 'data': [ 'none', 'setup', 'cancelling', 'cancelled', - 'active', 'postcopy-active', 'postcopy-paused', - 'postcopy-recover-setup', + 'active', 'postcopy-device', 'postcopy-active', + 'postcopy-paused', 'postcopy-recover-setup', 'postcopy-recover', 'completed', 'failed', 'colo', 'pre-switchover', 'device', 'wait-unplug' ] } ## diff --git a/tests/qtest/migration/precopy-tests.c b/tests/qtest/migration/= precopy-tests.c index bb38292550..57ca623de5 100644 --- a/tests/qtest/migration/precopy-tests.c +++ b/tests/qtest/migration/precopy-tests.c @@ -1316,13 +1316,14 @@ void migration_test_add_precopy(MigrationTestEnv *e= nv) } =20 /* ensure new status don't go unnoticed */ - assert(MIGRATION_STATUS__MAX =3D=3D 15); + assert(MIGRATION_STATUS__MAX =3D=3D 16); =20 for (int i =3D MIGRATION_STATUS_NONE; i < MIGRATION_STATUS__MAX; i++) { switch (i) { case MIGRATION_STATUS_DEVICE: /* happens too fast */ case MIGRATION_STATUS_WAIT_UNPLUG: /* no support in tests */ case MIGRATION_STATUS_COLO: /* no support in tests */ + case MIGRATION_STATUS_POSTCOPY_DEVICE: /* postcopy can't be cancel= led */ case MIGRATION_STATUS_POSTCOPY_ACTIVE: /* postcopy can't be cancel= led */ case MIGRATION_STATUS_POSTCOPY_PAUSED: case MIGRATION_STATUS_POSTCOPY_RECOVER_SETUP: --=20 2.51.0