From nobody Mon Feb 9 11:47:10 2026 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 170.10.129.124 as permitted sender) client-ip=170.10.129.124; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-delivery-124.mimecast.com; Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=fail(p=none dis=none) header.from=suse.de Return-Path: Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by mx.zohomail.com with SMTPS id 1654593630252135.28239190373245; Tue, 7 Jun 2022 02:20:30 -0700 (PDT) Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-619-2IrKNUDTMnyt7OEUs5SMbw-1; Tue, 07 Jun 2022 05:20:23 -0400 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.rdu2.redhat.com [10.11.54.1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 8CB471C0E345; Tue, 7 Jun 2022 09:20:03 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (unknown [10.30.29.100]) by smtp.corp.redhat.com (Postfix) with ESMTP id 73D3F406AD4A; Tue, 7 Jun 2022 09:20:03 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (localhost [IPv6:::1]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id 38CF019452D2; Tue, 7 Jun 2022 09:20:03 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx10.intmail.prod.int.rdu2.redhat.com [10.11.54.10]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id 1E0A419452D2 for ; Tue, 7 Jun 2022 09:20:01 +0000 (UTC) Received: by smtp.corp.redhat.com (Postfix) id F2E4440336E; Tue, 7 Jun 2022 09:20:00 +0000 (UTC) Received: from mimecast-mx02.redhat.com (mimecast04.extmail.prod.ext.rdu2.redhat.com [10.11.55.20]) by smtp.corp.redhat.com (Postfix) with ESMTPS id EEB09492C3B for ; Tue, 7 Jun 2022 09:20:00 +0000 (UTC) Received: from us-smtp-1.mimecast.com (us-smtp-2.mimecast.com [205.139.110.61]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id D3FA610665A4 for ; Tue, 7 Jun 2022 09:20:00 +0000 (UTC) Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.220.28]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-414-vorV-vQ_MM2YbPF2jpDnsA-1; Tue, 07 Jun 2022 05:19:56 -0400 Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id D980121B3F; Tue, 7 Jun 2022 09:19:47 +0000 (UTC) Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 9F23013638; Tue, 7 Jun 2022 09:19:47 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id mG1SJTMYn2IoLwAAMHmgww (envelope-from ); Tue, 07 Jun 2022 09:19:47 +0000 X-MC-Unique: 2IrKNUDTMnyt7OEUs5SMbw-1 X-Original-To: libvir-list@listman.corp.redhat.com X-MC-Unique: vorV-vQ_MM2YbPF2jpDnsA-1 From: Claudio Fontana To: =?UTF-8?q?Daniel=20P=20=2E=20Berrang=C3=A9?= Subject: [libvirt RFCv11 23/33] qemu: implement qemuSaveImageLoadMultiFd for restore Date: Tue, 7 Jun 2022 11:19:26 +0200 Message-Id: <20220607091936.7948-24-cfontana@suse.de> In-Reply-To: <20220607091936.7948-1-cfontana@suse.de> References: <20220607091936.7948-1-cfontana@suse.de> MIME-Version: 1.0 X-Mimecast-Impersonation-Protect: Policy=CLT - Impersonation Protection Definition; Similar Internal Domain=false; Similar Monitored External Domain=false; Custom External Domain=false; Mimecast External Domain=false; Newly Observed Domain=false; Internal User Name=false; Custom Display Name List=false; Reply-to Address Mismatch=false; Targeted Threat Dictionary=false; Mimecast Threat Dictionary=false; Custom Threat Dictionary=false X-Scanned-By: MIMEDefang 2.85 on 10.11.54.10 X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: libvir-list@redhat.com, Claudio Fontana , "Dr . David Alan Gilbert" , Anthony Iliopoulos Errors-To: libvir-list-bounces@redhat.com Sender: "libvir-list" X-Scanned-By: MIMEDefang 2.84 on 10.11.54.1 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=libvir-list-bounces@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable X-ZM-MESSAGEID: 1654593631110100010 Content-Type: text/plain; charset="utf-8"; x-default="true" use multifd to restore parallel images, if VIR_DOMAIN_SAVE_PARALLEL is enabled. Signed-off-by: Claudio Fontana --- src/qemu/qemu_driver.c | 16 +++++- src/qemu/qemu_migration.c | 2 +- src/qemu/qemu_migration.h | 6 ++ src/qemu/qemu_saveimage.c | 114 +++++++++++++++++++++++++++++++++++++- src/qemu/qemu_saveimage.h | 8 ++- 5 files changed, 139 insertions(+), 7 deletions(-) diff --git a/src/qemu/qemu_driver.c b/src/qemu/qemu_driver.c index 3d2b5d78e6..c3b2afe0d8 100644 --- a/src/qemu/qemu_driver.c +++ b/src/qemu/qemu_driver.c @@ -5883,7 +5883,7 @@ qemuDomainRestoreInternal(virConnectPtr conn, } oflags |=3D O_DIRECT; } - if (virQEMUSaveFdInit(&saveFd, path, oflags, cfg, false) < 0) + if (virQEMUSaveFdInit(&saveFd, path, oflags, cfg, flags & VIR_DOMAIN_S= AVE_PARALLEL) < 0) return -1; if (qemuSaveImageOpen(driver, NULL, &def, &data, false, &saveFd) < 0) goto cleanup; @@ -5939,8 +5939,18 @@ qemuDomainRestoreInternal(virConnectPtr conn, flags) < 0) goto cleanup; =20 - ret =3D qemuSaveImageStartVM(conn, driver, vm, &saveFd.fd, data, path, - false, reset_nvram, true, VIR_ASYNC_JOB_STA= RT); + if (flags & VIR_DOMAIN_SAVE_PARALLEL) { + ret =3D qemuSaveImageLoadMultiFd(conn, vm, data, reset_nvram, + &saveFd, VIR_ASYNC_JOB_START); + + } else if (data->header.multifd_channels !=3D 0) { + virReportError(VIR_ERR_OPERATION_FAILED, "%s", + _("save file seems to contain multifd channels info= rmation, and restore is not flagged as 'parallel'")); + ret =3D -1; + } else { + ret =3D qemuSaveImageStartVM(conn, driver, vm, &saveFd.fd, data, p= ath, + false, reset_nvram, true, VIR_ASYNC_JOB= _START); + } =20 qemuProcessEndJob(vm); =20 diff --git a/src/qemu/qemu_migration.c b/src/qemu/qemu_migration.c index 439dd7478d..c214f88dd1 100644 --- a/src/qemu/qemu_migration.c +++ b/src/qemu/qemu_migration.c @@ -1920,7 +1920,7 @@ qemuMigrationSrcWaitForCompletion(virQEMUDriver *driv= er, } =20 =20 -static int +int qemuMigrationDstWaitForCompletion(virQEMUDriver *driver, virDomainObj *vm, virDomainAsyncJob asyncJob, diff --git a/src/qemu/qemu_migration.h b/src/qemu/qemu_migration.h index c3c48c19c0..38f4877cf0 100644 --- a/src/qemu/qemu_migration.h +++ b/src/qemu/qemu_migration.h @@ -191,6 +191,12 @@ qemuMigrationDstFinish(virQEMUDriver *driver, int retcode, bool v3proto); =20 +int +qemuMigrationDstWaitForCompletion(virQEMUDriver *driver, + virDomainObj *vm, + virDomainAsyncJob asyncJob, + bool postcopy); + int qemuMigrationSrcConfirm(virQEMUDriver *driver, virDomainObj *vm, diff --git a/src/qemu/qemu_saveimage.c b/src/qemu/qemu_saveimage.c index 9c86e20f30..2ed02d31de 100644 --- a/src/qemu/qemu_saveimage.c +++ b/src/qemu/qemu_saveimage.c @@ -643,6 +643,105 @@ qemuSaveImageCreate(virQEMUDriver *driver, } =20 =20 +int qemuSaveImageLoadMultiFd(virConnectPtr conn, virDomainObj *vm, + virQEMUSaveData *data, bool reset_nvram, + virQEMUSaveFd *saveFd, virDomainAsyncJob asyn= cJob) +{ + virQEMUDriver *driver =3D conn->privateData; + qemuDomainObjPrivate *priv =3D vm->privateData; + g_autoptr(virQEMUDriverConfig) cfg =3D virQEMUDriverGetConfig(driver); + g_autoptr(virCommand) cmd =3D NULL; + g_autofree char *helper_path =3D NULL; + g_autofree char *sun_path =3D g_strdup_printf("%s/restore-multifd.sock= ", cfg->saveDir); + bool qemu_started =3D false; + int ret =3D -1; + + if (!(helper_path =3D virFileFindResource("libvirt_multifd_helper", + abs_top_builddir "/src", + LIBEXECDIR))) + goto cleanup; + cmd =3D virCommandNewArgList(helper_path, sun_path, saveFd->path, NULL= ); + virCommandAddArgFormat(cmd, "%d", saveFd->nchannels); + virCommandAddArgFormat(cmd, "%d", saveFd->fd); + virCommandPassFD(cmd, saveFd->fd, 0); + + if (qemuSaveImageStartVM(conn, driver, vm, NULL, data, sun_path, + false, reset_nvram, false, asyncJob) < 0) + goto cleanup; + + qemu_started =3D true; + + if (!virQEMUCapsGet(priv->qemuCaps, QEMU_CAPS_MIGRATE_MULTIFD)) { + virReportError(VIR_ERR_OPERATION_UNSUPPORTED, "%s", + _("QEMU does not seem to support multifd migration"= )); + goto cleanup; + } else { + g_autoptr(qemuMigrationParams) migParams =3D qemuMigrationParamsNe= w(); + bool bwParam =3D virQEMUCapsGet(priv->qemuCaps, QEMU_CAPS_MIGRATIO= N_PARAM_BANDWIDTH); + + if (bwParam) { + if (qemuMigrationParamsSetULL(migParams, + QEMU_MIGRATION_PARAM_MAX_BANDWID= TH, + QEMU_DOMAIN_MIG_BANDWIDTH_MAX * = 1024 * 1024) < 0) + goto cleanup; + priv->migMaxBandwidth =3D QEMU_DOMAIN_MIG_BANDWIDTH_MAX; + } else { + if (qemuDomainObjEnterMonitorAsync(driver, vm, asyncJob) =3D= =3D 0) { + qemuMonitorSetMigrationSpeed(priv->mon, + QEMU_DOMAIN_MIG_BANDWIDTH_MAX= ); + priv->migMaxBandwidth =3D QEMU_DOMAIN_MIG_BANDWIDTH_MAX; + qemuDomainObjExitMonitor(vm); + } + } + qemuMigrationParamsSetCap(migParams, QEMU_MIGRATION_CAP_MULTIFD); + if (qemuMigrationParamsSetInt(migParams, + QEMU_MIGRATION_PARAM_MULTIFD_CHANNEL= S, + saveFd->nchannels) < 0) + goto cleanup; + if (qemuMigrationParamsApply(driver, vm, asyncJob, migParams) < 0) + goto cleanup; + + if (!virDomainObjIsActive(vm)) { + virReportError(VIR_ERR_INTERNAL_ERROR, "%s", + _("guest unexpectedly quit")); + goto cleanup; + } + /* multifd helper can now connect, then wait for migration to comp= lete */ + if (virCommandRunAsync(cmd, NULL) < 0) + goto cleanup; + + if (qemuMigrationDstWaitForCompletion(driver, vm, asyncJob, false)= < 0) + goto cleanup; + + if (qemuProcessRefreshState(driver, vm, asyncJob) < 0) + goto cleanup; + + /* run 'cont' on the destination */ + if (qemuProcessStartCPUs(driver, vm, + VIR_DOMAIN_RUNNING_RESTORED, + asyncJob) < 0) { + if (virGetLastErrorCode() =3D=3D VIR_ERR_OK) + virReportError(VIR_ERR_OPERATION_FAILED, + "%s", _("failed to resume domain")); + goto cleanup; + } + if (virDomainObjSave(vm, driver->xmlopt, cfg->stateDir) < 0) { + VIR_WARN("Failed to save status on vm %s", vm->def->name); + goto cleanup; + } + } + qemuDomainEventEmitJobCompleted(driver, vm); + ret =3D 0; + + cleanup: + if (ret < 0 && qemu_started) { + qemuProcessStop(driver, vm, VIR_DOMAIN_SHUTOFF_FAILED, + asyncJob, VIR_QEMU_PROCESS_STOP_MIGRATED); + } + return ret; +} + + /* qemuSaveImageGetCompressionProgram: * @imageFormat: String representation from qemu.conf for the compression * image format being used (dump, save, or snapshot). @@ -758,6 +857,10 @@ qemuSaveImageOpen(virQEMUDriver *driver, } return ret; } + if (data->header.multifd_channels > 0) { + if (virQEMUSaveFdAddChannels(saveFd, data->header.multifd_channels= ) < 0) + return -1; + } =20 /* Create a domain from this XML */ if (!(def =3D virDomainDefParseString(data->xml, driver->xmlopt, qemuC= aps, @@ -788,6 +891,7 @@ qemuSaveImageStartVM(virConnectPtr conn, bool started =3D false; virObjectEvent *event; VIR_AUTOCLOSE intermediatefd =3D -1; + g_autofree char *migrate_from =3D NULL; g_autoptr(virCommand) cmd =3D NULL; g_autofree char *errbuf =3D NULL; g_autoptr(virQEMUDriverConfig) cfg =3D virQEMUDriverGetConfig(driver); @@ -834,8 +938,14 @@ qemuSaveImageStartVM(virConnectPtr conn, if (cookie && !cookie->slirpHelper) priv->disableSlirp =3D true; =20 + if (fd) { + migrate_from =3D g_strdup("stdio"); + } else { + migrate_from =3D g_strdup_printf("unix://%s", path); + } + if (qemuProcessStart(conn, driver, vm, cookie ? cookie->cpu : NULL, - asyncJob, "stdio", *fd, path, wait_incoming, + asyncJob, migrate_from, fd ? *fd : -1, path, wait= _incoming, NULL, VIR_NETDEV_VPORT_PROFILE_OP_RESTORE, start_flags) =3D=3D 0) @@ -859,7 +969,7 @@ qemuSaveImageStartVM(virConnectPtr conn, VIR_DEBUG("Decompression binary stderr: %s", NULLSTR(errbuf)); virErrorRestore(&orig_err); } - if (VIR_CLOSE(*fd) < 0) { + if (fd && VIR_CLOSE(*fd) < 0) { virReportSystemError(errno, _("cannot close file: %s"), path); rc =3D -1; } diff --git a/src/qemu/qemu_saveimage.h b/src/qemu/qemu_saveimage.h index 91ea8dcbf2..158bcaff50 100644 --- a/src/qemu/qemu_saveimage.h +++ b/src/qemu/qemu_saveimage.h @@ -99,7 +99,7 @@ qemuSaveImageStartVM(virConnectPtr conn, bool reset_nvram, bool wait_incoming, virDomainAsyncJob asyncJob) - ATTRIBUTE_NONNULL(4) ATTRIBUTE_NONNULL(5) ATTRIBUTE_NONNULL(6); + ATTRIBUTE_NONNULL(5) ATTRIBUTE_NONNULL(6); =20 int qemuSaveImageOpen(virQEMUDriver *driver, @@ -117,6 +117,12 @@ qemuSaveImageGetCompressionProgram(const char *imageFo= rmat, bool use_raw_on_fail) ATTRIBUTE_NONNULL(2); =20 +int qemuSaveImageLoadMultiFd(virConnectPtr conn, virDomainObj *vm, + virQEMUSaveData *data, bool reset_nvram, + virQEMUSaveFd *saveFd, virDomainAsyncJob asyn= cJob) + ATTRIBUTE_NONNULL(1) ATTRIBUTE_NONNULL(2) ATTRIBUTE_NONNULL(3) + ATTRIBUTE_NONNULL(5) G_GNUC_WARN_UNUSED_RESULT; + int qemuSaveImageCreate(virQEMUDriver *driver, virDomainObj *vm, --=20 2.26.2