From nobody Mon Aug 24 19:53:20 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=reject dis=none) header.from=rsg.ci.i.u-tokyo.ac.jp Return-Path: Received: from lists1p.gnu.org (lists1p.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1783056264482856.7553397329998; Thu, 2 Jul 2026 22:24:24 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists1p.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1wfWMb-0005XE-T8; Fri, 03 Jul 2026 01:22:57 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists1p.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1wfWMW-0005TL-La; Fri, 03 Jul 2026 01:22:53 -0400 Received: from www3579.sakura.ne.jp ([49.212.243.89]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1wfWMS-0005rz-4H; Fri, 03 Jul 2026 01:22:52 -0400 Received: from h183.csg.ci.i.u-tokyo.ac.jp (h183.csg.ci.i.u-tokyo.ac.jp [133.11.54.183]) (authenticated bits=0) by www3579.sakura.ne.jp (8.16.1/8.16.1) with ESMTPSA id 6635LT8U018373 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Fri, 3 Jul 2026 14:21:46 +0900 (JST) (envelope-from odaki@rsg.ci.i.u-tokyo.ac.jp) DKIM-Signature: a=rsa-sha256; bh=5vN8LLN1cwVkJSeHOFH42uOAevgNgauoNhfEjaG1Qa4=; c=relaxed/relaxed; d=rsg.ci.i.u-tokyo.ac.jp; h=From:Message-Id:To:Subject:Date; s=rs20250326; t=1783056106; v=1; b=BsX3PEyPKdeLkzVfmBzo06LGKElOOo4+ah4cn6ElKvQv6pS3I84i72LCh2Ygd+Q+ KDaWQ2m+tZvx46uoqZBXBA4+Vty4ktd2u9uauB1cfyhLfLoaBEnOYKKv9+yWx+p6 3i0Eas/wc80ZCw251oDLMWVlgt/Vx1XwCf/TOiXxicyTTNmn2pAT8fE1RMmaLcWX 3p8uy0kElZYeIjYLDbBDv0L+dDdmMW/+7Omnq9KGsCivqBGdFhH0wzbn5P+3pBFL RS7JDgp5oZl6wxwExWswamp7PU2OUWFfa3kxg/WDgcGq16bt+dUCBvBCdocCqspz XyyALYQIlcPWHOF0OvmrhQ== From: Akihiko Odaki Date: Fri, 03 Jul 2026 14:19:23 +0900 Subject: [PATCH v2 7/7] migration/ram: Assert RAMBlock stability while streaming RAM MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260703-ram-v2-7-7f07fc06fba9@rsg.ci.i.u-tokyo.ac.jp> References: <20260703-ram-v2-0-7f07fc06fba9@rsg.ci.i.u-tokyo.ac.jp> In-Reply-To: <20260703-ram-v2-0-7f07fc06fba9@rsg.ci.i.u-tokyo.ac.jp> To: qemu-devel@nongnu.org Cc: Kevin Wolf , Hanna Reitz , =?utf-8?q?Philippe_Mathieu-Daud=C3=A9?= , Zhao Liu , Stefano Stabellini , Anthony PERARD , "Edgar E. Iglesias" , Peter Xu , Fabiano Rosas , Paolo Bonzini , Reinoud Zandijk , Marcelo Tosatti , Alex Williamson , =?utf-8?q?C=C3=A9dric_Le_Goater?= , qemu-block@nongnu.org, xen-devel@lists.xenproject.org, kvm@vger.kernel.org, Gerd Hoffmann , Mauro Carvalho Chehab , "Michael S. Tsirkin" , Igor Mammedov , Ani Sinha , Peter Maydell , Richard Henderson , Song Gao , Bibo Mao , Jiaxun Yang , =?utf-8?q?Marc-Andr=C3=A9_Lureau?= , Sunil V L , Palmer Dabbelt , Alistair Francis , Weiwei Li , Daniel Henrique Barboza , Liu Zhiwei , Chao Liu , linux-edac@vger.kernel.org, qemu-arm@nongnu.org, qemu-riscv@nongnu.org, Akihiko Odaki X-Mailer: b4 0.16-dev-925f5 X-Developer-Signature: v=1; a=openpgp-sha256; l=9631; i=odaki@rsg.ci.i.u-tokyo.ac.jp; h=from:subject:message-id; bh=6rxiBJMURMDjjavfjxxSoOmIDh11hItlYvDbOT13ApM=; b=owGbwMvMwCWmMbc20y1CyJDxtFoSQ5a725OdNgt3n132cuXK3IPcvbM95i8M/lib9V949hfxi 2+e9s7o6yhlYRDjYpAVU2RJKdrNrRFd+6kwIb4FZg4rE8gQBi5OAZiImCsjw8KbEy5yn/oeJTav meUWYw37tlkyRrvuyUSd/21sWsMbI8Dw35fBlYORNSrL/VeSf4bcXpujDyZKxTxJe8zvt3/93Qk neAE= X-Developer-Key: i=odaki@rsg.ci.i.u-tokyo.ac.jp; a=openpgp; fpr=AEDC03C9AF734F2EC26A7BFFA4BAEAA73536753C Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists1p.gnu.org; Received-SPF: pass client-ip=49.212.243.89; envelope-from=odaki@rsg.ci.i.u-tokyo.ac.jp; helo=www3579.sakura.ne.jp X-Spam_score_int: -16 X-Spam_score: -1.7 X-Spam_bar: - X-Spam_report: (-1.7 / 5.0 requ) BAYES_00=-1.9, DKIM_INVALID=0.1, DKIM_SIGNED=0.1, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: qemu development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZM-MESSAGEID: 1783056267420158500 RAM migration sends the RAMBlock list and sizes before transferring page contents. The RAM code cannot cope with migratable RAMBlocks being added, removed, renamed, made migratable/non-migratable, or resized while that stream is active, because the destination has already parsed the stream layout. Add RAMBlock notifier checks so such changes are caught immediately. Ignored RAMBlocks remain exempt because they are not represented in the RAM migration stream. Signed-off-by: Akihiko Odaki --- include/migration/misc.h | 2 +- migration/ram.c | 94 ++++++++++++++++++++++++++++++++++----------= ---- 2 files changed, 68 insertions(+), 28 deletions(-) diff --git a/include/migration/misc.h b/include/migration/misc.h index 3159a5e53c3c..91015524a83d 100644 --- a/include/migration/misc.h +++ b/include/migration/misc.h @@ -40,7 +40,7 @@ void precopy_remove_notifier(NotifierWithReturn *n); int precopy_notify(PrecopyNotifyReason reason, Error **errp); =20 void qemu_guest_free_page_hint(void *addr, size_t len); -bool migrate_ram_is_ignored(RAMBlock *block); +bool migrate_ram_is_ignored(const RAMBlock *block); =20 /* migration/block.c */ =20 diff --git a/migration/ram.c b/migration/ram.c index e1c191ec4383..3cfc1461f13e 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -224,7 +224,7 @@ static bool postcopy_preempt_active(void) return migrate_postcopy_preempt() && migration_in_postcopy(); } =20 -bool migrate_ram_is_ignored(RAMBlock *block) +bool migrate_ram_is_ignored(const RAMBlock *block) { MigMode mode =3D migrate_mode(); return !qemu_ram_is_migratable(block) || @@ -364,6 +364,12 @@ struct RAMSrcPageRequest { QSIMPLEQ_ENTRY(RAMSrcPageRequest) next_req; }; =20 +typedef enum RAMStateStage { + RAM_STATE_STAGE_ITERATING, + RAM_STATE_STAGE_COMPLETING, + RAM_STATE_STAGE_COMPLETED, +} RAMStateStage; + /* State of RAM for migration */ struct RAMState { /* @@ -398,8 +404,8 @@ struct RAMState { uint64_t xbzrle_bytes_prev; /* Are we really using XBZRLE (e.g., after the first round). */ bool xbzrle_started; - /* Are we on the last stage of migration */ - bool last_stage; + /* Migration stage */ + RAMStateStage stage; =20 /* total handled target pages at the beginning of period */ uint64_t target_page_count_prev; @@ -635,7 +641,7 @@ static int save_xbzrle_page(RAMState *rs, PageSearchSta= tus *pss, =20 if (!cache_is_cached(XBZRLE.cache, current_addr, generation)) { xbzrle_counters.cache_miss++; - if (!rs->last_stage) { + if (rs->stage =3D=3D RAM_STATE_STAGE_ITERATING) { if (cache_insert(XBZRLE.cache, current_addr, *current_data, generation) =3D=3D -1) { return -1; @@ -674,7 +680,7 @@ static int save_xbzrle_page(RAMState *rs, PageSearchSta= tus *pss, * Update the cache contents, so that it corresponds to the data * sent, in all cases except where we skip the page. */ - if (!rs->last_stage && encoded_len !=3D 0) { + if (rs->stage =3D=3D RAM_STATE_STAGE_ITERATING && encoded_len !=3D 0) { memcpy(prev_cached_page, XBZRLE.current_buf, TARGET_PAGE_SIZE); /* * In the case where we couldn't compress, ensure that the caller @@ -840,7 +846,8 @@ static inline bool migration_bitmap_clear_dirty(RAMStat= e *rs, * * Do the same for postcopy due to the same reason. */ - if (!rs->last_stage && !migration_in_postcopy()) { + if (rs->stage =3D=3D RAM_STATE_STAGE_ITERATING && + !migration_in_postcopy()) { /* * Clear dirty bitmap if needed. This _must_ be called before we * send any of the page in the chunk because we need to make sure @@ -1316,7 +1323,7 @@ static int ram_save_page(RAMState *rs, PageSearchStat= us *pss) if (rs->xbzrle_started && !migration_in_postcopy()) { pages =3D save_xbzrle_page(rs, pss, &p, current_addr, block, offset); - if (!rs->last_stage) { + if (rs->stage =3D=3D RAM_STATE_STAGE_ITERATING) { /* Can't send this cached data async, since the cache page * might get updated before it gets to the wire */ @@ -2920,6 +2927,8 @@ static void ram_state_resume_prepare(RAMState *rs, QE= MUFile *out) RAMBlock *block; uint64_t pages =3D 0; =20 + rs->stage =3D RAM_STATE_STAGE_ITERATING; + /* * Postcopy is not using xbzrle/compression, so no need for that. * Also, since source are already halted, we don't need to care @@ -3373,7 +3382,9 @@ static int ram_save_complete(QEMUFile *f, void *opaqu= e) =20 trace_ram_save_complete(rs->migration_dirty_pages, 0); =20 - rs->last_stage =3D !migration_in_colo_state(); + if (!migration_in_colo_state()) { + rs->stage =3D RAM_STATE_STAGE_COMPLETING; + } =20 WITH_RCU_READ_LOCK_GUARD() { if (!migration_in_postcopy()) { @@ -3383,7 +3394,7 @@ static int ram_save_complete(QEMUFile *f, void *opaqu= e) ret =3D rdma_registration_start(f, RAM_CONTROL_FINISH); if (ret < 0) { qemu_file_set_error(f, ret); - return ret; + goto err; } =20 /* try transferring iterative blocks of memory */ @@ -3400,7 +3411,8 @@ static int ram_save_complete(QEMUFile *f, void *opaqu= e) } if (pages < 0) { qemu_mutex_unlock(&rs->bitmap_mutex); - return pages; + ret =3D pages; + goto err; } } qemu_mutex_unlock(&rs->bitmap_mutex); @@ -3408,7 +3420,7 @@ static int ram_save_complete(QEMUFile *f, void *opaqu= e) ret =3D rdma_registration_stop(f, RAM_CONTROL_FINISH); if (ret < 0) { qemu_file_set_error(f, ret); - return ret; + goto err; } } =20 @@ -3419,7 +3431,7 @@ static int ram_save_complete(QEMUFile *f, void *opaqu= e) */ ret =3D multifd_ram_flush_and_sync(f); if (ret < 0) { - return ret; + goto err; } } =20 @@ -3428,10 +3440,10 @@ static int ram_save_complete(QEMUFile *f, void *opa= que) =20 if (qemu_file_get_error(f)) { Error *local_err =3D NULL; - int err =3D qemu_file_get_error_obj(f, &local_err); + ret =3D qemu_file_get_error_obj(f, &local_err); =20 error_reportf_err(local_err, "Failed to write bitmap to file: = "); - return err; + goto err; } } =20 @@ -3439,7 +3451,14 @@ static int ram_save_complete(QEMUFile *f, void *opaq= ue) =20 trace_ram_save_complete(rs->migration_dirty_pages, 1); =20 - return qemu_fflush(f); + ret =3D qemu_fflush(f); + +err: + if (!migration_in_colo_state()) { + rs->stage =3D RAM_STATE_STAGE_COMPLETED; + } + + return ret; } =20 static void ram_state_pending(void *opaque, MigPendingData *pending, @@ -4699,28 +4718,45 @@ static SaveVMHandlers savevm_ram_handlers =3D { .save_postcopy_prepare =3D ram_save_postcopy_prepare, }; =20 +static bool ram_migration_is_running(void) +{ + return ram_state && ram_state->stage !=3D RAM_STATE_STAGE_COMPLETED; +} + +static void ram_mig_ram_block_set_migratable(RAMBlockNotifier *n, + const RAMBlock *rb) +{ + assert(!ram_migration_is_running()); +} + +static void ram_mig_ram_block_unset_migratable(RAMBlockNotifier *n, + const RAMBlock *rb) +{ + assert(!ram_migration_is_running()); +} + +static void ram_mig_ram_block_set_idstr(RAMBlockNotifier *n, const RAMBloc= k *rb) +{ + assert(!ram_migration_is_running() || migrate_ram_is_ignored(rb)); +} + +static void ram_mig_ram_block_removed(RAMBlockNotifier *n, const RAMBlock = *rb, + void *host, size_t size, size_t max_= size) +{ + assert(!rb || !ram_migration_is_running() || migrate_ram_is_ignored(rb= )); +} + static void ram_mig_ram_block_resized(RAMBlockNotifier *n, RAMBlock *rb, size_t new_size) { PostcopyState ps =3D postcopy_state_get(); - Error *err =3D NULL; ram_addr_t old_size =3D qemu_ram_get_used_length(rb); =20 if (migrate_ram_is_ignored(rb)) { return; } =20 - if (migration_is_running()) { - /* - * Precopy code on the source cannot deal with the size of RAM blo= cks - * changing at random points in time - especially after sending the - * RAM block sizes in the migration stream, they must no longer ch= ange. - * Abort and indicate a proper reason. - */ - error_setg(&err, "RAM block '%s' resized during precopy.", rb->ids= tr); - migrate_error_propagate(migrate_get_current(), err); - migration_cancel(); - } + assert(!ram_migration_is_running()); =20 switch (ps) { case POSTCOPY_INCOMING_ADVISE: @@ -4754,6 +4790,10 @@ static void ram_mig_ram_block_resized(RAMBlockNotifi= er *n, RAMBlock *rb, } =20 static RAMBlockNotifier ram_mig_ram_notifier =3D { + .ram_block_removed =3D ram_mig_ram_block_removed, + .ram_block_set_migratable =3D ram_mig_ram_block_set_migratable, + .ram_block_unset_migratable =3D ram_mig_ram_block_unset_migratable, + .ram_block_set_idstr =3D ram_mig_ram_block_set_idstr, .ram_block_resized =3D ram_mig_ram_block_resized, }; =20 --=20 2.54.0