From nobody Tue Feb 10 03:39:13 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1675731556; cv=none; d=zohomail.com; s=zohoarc; b=CswW9vWS8UkqwdbuVEsdB9wSDHrIwoq89BqMHVohWIx/DnCIUxFhLpFOg7ljvZyWInM/hxPbixZz1Bb4YkcAijkkibc7bjsepWCQdzlQAapfEbL3GCSAi8gQ1yjOdHWH5nVptFfcO5XM3TGaB/ltqAsV9zK0eO8zpffXOET6wCU= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1675731556; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=Lw58clu7Y1bzBf2f7fvddUJNRU5/Wbva2ZRRL1vM7CM=; b=KfQBJbmo2ihE1AX+CFd4YSoJOxJhlmIPjUkbVcjy1Goo29vP7RAvGT4iEICRVapqwrfeZ5Nj/rAwgBfpr83QhgMv8YdNgnrHzobOK45CFRjO2y/+vLa3DQ9lIMLZlLGAzonrglbSDF2vt9PUhKD71U2i3/TdKohtYDOtOlDC+2s= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1675731556542258.8031522894347; Mon, 6 Feb 2023 16:59:16 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pPCJQ-0004Py-AR; Mon, 06 Feb 2023 19:58:20 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pPCJO-0004DH-Ha for qemu-devel@nongnu.org; Mon, 06 Feb 2023 19:58:18 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pPCJM-0003mD-J6 for qemu-devel@nongnu.org; Mon, 06 Feb 2023 19:58:18 -0500 Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-625-5gYAzHyFMp-7ya_0fhxQiA-1; Mon, 06 Feb 2023 19:58:09 -0500 Received: from smtp.corp.redhat.com (int-mx09.intmail.prod.int.rdu2.redhat.com [10.11.54.9]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 5FF601871D97; Tue, 7 Feb 2023 00:58:08 +0000 (UTC) Received: from secure.mitica (unknown [10.39.192.29]) by smtp.corp.redhat.com (Postfix) with ESMTP id 33CDE492C3C; Tue, 7 Feb 2023 00:58:03 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1675731496; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Lw58clu7Y1bzBf2f7fvddUJNRU5/Wbva2ZRRL1vM7CM=; b=hAdkXvjy6FK3BAPASg74KJ8bH8POTA3U/zpUm4MfnijNCcDD+QpqWOOh+ay9VV98vJmi73 qESqEg4rpxiwZ0khB/2smJgSn3E2vl1q2d8a1E1gtFja6PLDI/t+B1oi3Wulm24NFjT3ov h6G8+Cds74l1cNtiaAHC9WWCga9maNM= X-MC-Unique: 5gYAzHyFMp-7ya_0fhxQiA-1 From: Juan Quintela To: qemu-devel@nongnu.org Cc: qemu-block@nongnu.org, Stefan Berger , Stefan Hajnoczi , Halil Pasic , John Snow , David Hildenbrand , Fam Zheng , Thomas Huth , =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= , Laurent Vivier , Vladimir Sementsov-Ogievskiy , qemu-s390x@nongnu.org, Christian Borntraeger , =?UTF-8?q?Marc-Andr=C3=A9=20Lureau?= , "Michael S. Tsirkin" , Juan Quintela , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , "Dr. David Alan Gilbert" , Marcel Apfelbaum , Coiby Xu , Ilya Leoshkevich , Eduardo Habkost , Yanan Wang , Richard Henderson , Markus Armbruster , Paolo Bonzini , Alex Williamson , Eric Blake , Eric Farman , Peter Xu Subject: [PULL 14/30] migration/savevm: Allow immutable device state to be migrated early (i.e., before RAM) Date: Tue, 7 Feb 2023 01:56:34 +0100 Message-Id: <20230207005650.1810-15-quintela@redhat.com> In-Reply-To: <20230207005650.1810-1-quintela@redhat.com> References: <20230207005650.1810-1-quintela@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.1 on 10.11.54.9 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=quintela@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1675731557132100003 Content-Type: text/plain; charset="utf-8" From: David Hildenbrand For virtio-mem, we want to have the plugged/unplugged state of memory blocks available before migrating any actual RAM content, and perform sanity checks before touching anything on the destination. This information is immutable on the migration source while migration is active, We want to use this information for proper preallocation support with migration: currently, we don't preallocate memory on the migration target, and especially with hugetlb, we can easily run out of hugetlb pages during RAM migration and will crash (SIGBUS) instead of catching this gracefully via preallocation. Migrating device state via a VMSD before we start iterating is currently impossible: the only approach that would be possible is avoiding a VMSD and migrating state manually during save_setup(), to be restored during load_state(). Let's allow for migrating device state via a VMSD early, during the setup phase in qemu_savevm_state_setup(). To keep it simple, we indicate applicable VMSD's using an "early_setup" flag. Note that only very selected devices (i.e., ones seriously messing with RAM setup) are supposed to make use of such early state migration. While at it, also use a bool for the "unmigratable" member. Reviewed-by: Peter Xu Reviewed-by: Michael S. Tsirkin Reviewed-by: Juan Quintela S Signed-off-by: David Hildenbrand Signed-off-by: Juan Quintela --- include/migration/vmstate.h | 16 +++++++++++++++- migration/savevm.c | 14 ++++++++++++++ 2 files changed, 29 insertions(+), 1 deletion(-) diff --git a/include/migration/vmstate.h b/include/migration/vmstate.h index ad24aa1934..64680d824e 100644 --- a/include/migration/vmstate.h +++ b/include/migration/vmstate.h @@ -178,7 +178,21 @@ struct VMStateField { =20 struct VMStateDescription { const char *name; - int unmigratable; + bool unmigratable; + /* + * This VMSD describes something that should be sent during setup phase + * of migration. It plays similar role as save_setup() for explicitly + * registered vmstate entries, so it can be seen as a way to describe + * save_setup() in VMSD structures. + * + * Note that for now, a SaveStateEntry cannot have a VMSD and + * operations (e.g., save_setup()) set at the same time. Consequently, + * save_setup() and a VMSD with early_setup set to true are mutually + * exclusive. For this reason, also early_setup VMSDs are migrated in a + * QEMU_VM_SECTION_FULL section, while save_setup() data is migrated in + * a QEMU_VM_SECTION_START section. + */ + bool early_setup; int version_id; int minimum_version_id; MigrationPriority priority; diff --git a/migration/savevm.c b/migration/savevm.c index 28f88b5521..6d985ad4af 100644 --- a/migration/savevm.c +++ b/migration/savevm.c @@ -1201,6 +1201,15 @@ void qemu_savevm_state_setup(QEMUFile *f) =20 trace_savevm_state_setup(); QTAILQ_FOREACH(se, &savevm_state.handlers, entry) { + if (se->vmsd && se->vmsd->early_setup) { + ret =3D vmstate_save(f, se, ms->vmdesc); + if (ret) { + qemu_file_set_error(f, ret); + break; + } + continue; + } + if (!se->ops || !se->ops->save_setup) { continue; } @@ -1403,6 +1412,11 @@ int qemu_savevm_state_complete_precopy_non_iterable(= QEMUFile *f, int ret; =20 QTAILQ_FOREACH(se, &savevm_state.handlers, entry) { + if (se->vmsd && se->vmsd->early_setup) { + /* Already saved during qemu_savevm_state_setup(). */ + continue; + } + ret =3D vmstate_save(f, se, vmdesc); if (ret) { qemu_file_set_error(f, ret); --=20 2.39.1