From nobody Thu Sep 19 01:14:31 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.libvirt.org designates 8.43.85.245 as permitted sender) client-ip=8.43.85.245; envelope-from=devel-bounces@lists.libvirt.org; helo=lists.libvirt.org; Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of lists.libvirt.org designates 8.43.85.245 as permitted sender) smtp.mailfrom=devel-bounces@lists.libvirt.org; dmarc=pass(p=reject dis=none) header.from=lists.libvirt.org ARC-Seal: i=1; a=rsa-sha256; t=1718319435; cv=none; d=zohomail.com; s=zohoarc; b=XCzr10u91dbHSDaxun27zcDTu0X+8PoPpSzk5FiKaKiQR9yWQc/vbczrLn+zbo2jGMnl9/Y+k5VbXMzzLZe2CiXrkcAtbutX8Z+zNyuX8r6Aufv3NHSgUYaXqVN9r291brMUme2rDhwWUnvFd7Tq8BiMsEwYgD6OfSWRmwa51E8= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1718319435; h=Content-Transfer-Encoding:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:Reply-To:Reply-To:References:Subject:Subject:To:To:Message-Id:Cc; bh=qLHhFwtvA+42q5TuT2eH4+lMo12AbQg7W/P/q1dZc+s=; b=l+LdrbIsawxR0kbWAYrY+zZiPKelWDZAgJPcsM5SmCY9SeFqZxKSEjMUQrd7srHgWJvvpwu9b1lnF13F9BdJOGe6t8waBXPCLQPZ8durZm6V5+XFjrD6wlZyvqPi/FlhWQBfW7M0K6IxogDxPpfGaIo+adA1obtydujMkYD8ty4= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of lists.libvirt.org designates 8.43.85.245 as permitted sender) smtp.mailfrom=devel-bounces@lists.libvirt.org; dmarc=pass header.from= (p=reject dis=none) Return-Path: Received: from lists.libvirt.org (lists.libvirt.org [8.43.85.245]) by mx.zohomail.com with SMTPS id 1718319435046329.9360603904307; Thu, 13 Jun 2024 15:57:15 -0700 (PDT) Received: by lists.libvirt.org (Postfix, from userid 996) id C7F951275; Thu, 13 Jun 2024 18:57:13 -0400 (EDT) Received: from lists.libvirt.org (localhost [IPv6:::1]) by lists.libvirt.org (Postfix) with ESMTP id EBFE112B0; Thu, 13 Jun 2024 18:53:43 -0400 (EDT) Received: by lists.libvirt.org (Postfix, from userid 996) id 5C63A12AA; Thu, 13 Jun 2024 18:53:41 -0400 (EDT) Received: from mail-lj1-f174.google.com (mail-lj1-f174.google.com [209.85.208.174]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by lists.libvirt.org (Postfix) with ESMTPS id 51AB711EC for ; Thu, 13 Jun 2024 18:52:44 -0400 (EDT) Received: by mail-lj1-f174.google.com with SMTP id 38308e7fff4ca-2eaafda3b5cso17982481fa.3 for ; Thu, 13 Jun 2024 15:52:44 -0700 (PDT) Received: from localhost ([69.51.97.134]) by smtp.gmail.com with ESMTPSA id 8926c6da1cb9f-4b9569e8f94sm612884173.89.2024.06.13.15.52.42 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 13 Jun 2024 15:52:42 -0700 (PDT) X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on lists.libvirt.org X-Spam-Level: X-Spam-Status: No, score=-0.8 required=5.0 tests=DKIM_INVALID,DKIM_SIGNED, MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE,RCVD_IN_MSPIKE_H2,SPF_HELO_NONE, T_SCC_BODY_TEXT_LINE autolearn=unavailable autolearn_force=no version=3.4.4 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=google; t=1718319163; x=1718923963; darn=lists.libvirt.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:from:to:cc:subject:date:message-id :reply-to; bh=HuMqZhiI8mcxfHbacNPVnCAMh/8vHkU7mdGHgJIgOTk=; b=Rb/fcKcsa5I3v7SOc8t3Ydpy5QXuxjwM9qPxLQtxPNojfrqPHqUDLN5e6eAn3pZhVz 8yNXNV9gx6MR/vK1ffDtd5K211GFTkOztqJmJsz7FyOTdyfIX8hQE895f3+ct0Ea1CHT MKKoCLBiPFjbMQ3EY1XnNsadSqyYyrAiBPkX6RKdlwzcPyxhBk9g8ECkWp2CHf2Kb61N Ru6Zg/3jzHoMrpQzFSdg/s8I57aa0whGFpKWLd2UNDGNg1ckYqwTsmUbgFMfJz23ZJc0 kdjwYO09Nr4ezUMqKW1fw79OO0leCziyn5I3Bt8ERP0J7pyljYnoEWvEbNTNX7QkKbXI WQEQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1718319163; x=1718923963; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=HuMqZhiI8mcxfHbacNPVnCAMh/8vHkU7mdGHgJIgOTk=; b=sRzyFOmZvV1L+8D/MdlpOOF7h72QAFfHS+mtDLtUXI4DgO+veDUlcmkfOu6PJZlEV0 h1nH0hzGdCmGTMQk4acE0onzHn4k5Ntcc5GkJEpRjTGWfzOsMEzljnAb+AjUAqaCGf5r o5NxQMSK35P7VUH50nTRocK0AZXoyoOO0uxrJo597/scAaFZduVz6BXDkRPBGkHGjlFj 3vuGTGhuLVBhj96yb7X9qk2owavw7sXbMHfdak5QmyTfGC9x0lbhBE0fqB9dOhiTRPGV R47QIOP+SBei1nQeHxi7mNyKPshyAO6MpzL8nJoSYUbo8rWcdidZG1zAhiDpWSr3inml ewDg== X-Gm-Message-State: AOJu0YxbbwjHmZkMXbWyKJ/A/3t7fFxtxK05ctZYNKPGVVz4r08cou11 V1nmRvT7ThFEpL7hXgkTISgSFzg363HzfvOecbQTJa64GKW8mnQYcSz5qmzs3ECxMrJf3QX7a6A b X-Google-Smtp-Source: AGHT+IEh31X74A+7bFfTwCvDk2/Q4noa+LCp44x5LMdXHHb+8AVR2/KFbQC5Yrt+Z8+YlrV6mSzOUw== X-Received: by 2002:a2e:8046:0:b0:2eb:dd8e:fbf5 with SMTP id 38308e7fff4ca-2ec0e46ed3bmr6687301fa.22.1718319162970; Thu, 13 Jun 2024 15:52:42 -0700 (PDT) To: devel@lists.libvirt.org Subject: [PATCH RFC 6/9] qemu: Add support for mapped-ram on save Date: Thu, 13 Jun 2024 16:43:20 -0600 Message-ID: <20240613225228.10665-7-jfehlig@suse.com> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240613225228.10665-1-jfehlig@suse.com> References: <20240613225228.10665-1-jfehlig@suse.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Message-ID-Hash: FTNIAFGHIN2CMDFBFY2J6YQILT2DIMR3 X-Message-ID-Hash: FTNIAFGHIN2CMDFBFY2J6YQILT2DIMR3 X-MailFrom: jfehlig@suse.com X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation; header-match-config-1; header-match-config-2; header-match-config-3; header-match-devel.lists.libvirt.org-0; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; suspicious-header X-Mailman-Version: 3.2.2 Precedence: list List-Id: Development discussions about the libvirt library & tools Archived-At: List-Archive: List-Help: List-Post: List-Subscribe: List-Unsubscribe: From: Jim Fehlig via Devel Reply-To: Jim Fehlig X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZM-MESSAGEID: 1718319436436100001 Content-Type: text/plain; charset="utf-8" Signed-off-by: Jim Fehlig --- src/qemu/qemu_driver.c | 2 +- src/qemu/qemu_migration.c | 79 ++++++++++++++++++++++++++++ src/qemu/qemu_migration.h | 7 +++ src/qemu/qemu_monitor.c | 32 ++++++++++++ src/qemu/qemu_monitor.h | 4 ++ src/qemu/qemu_saveimage.c | 105 ++++++++++++++++++++++++++++++-------- src/qemu/qemu_saveimage.h | 1 + src/qemu/qemu_snapshot.c | 2 +- 8 files changed, 208 insertions(+), 24 deletions(-) diff --git a/src/qemu/qemu_driver.c b/src/qemu/qemu_driver.c index f9761242d2..34f37210d9 100644 --- a/src/qemu/qemu_driver.c +++ b/src/qemu/qemu_driver.c @@ -2696,7 +2696,7 @@ qemuDomainSaveInternal(virQEMUDriver *driver, if (!(cookie =3D qemuDomainSaveCookieNew(vm))) goto endjob; =20 - if (!(data =3D virQEMUSaveDataNew(driver, xml, cookie, was_running, co= mpressed))) + if (!(data =3D virQEMUSaveDataNew(driver, vm, xml, cookie, was_running= , compressed))) goto endjob; xml =3D NULL; =20 diff --git a/src/qemu/qemu_migration.c b/src/qemu/qemu_migration.c index 1faab5dd23..3110ef2621 100644 --- a/src/qemu/qemu_migration.c +++ b/src/qemu/qemu_migration.c @@ -7072,6 +7072,85 @@ qemuMigrationSrcToFile(virQEMUDriver *driver, virDom= ainObj *vm, } =20 =20 +int +qemuMigrationSrcToMappedFile(virQEMUDriver *driver, virDomainObj *vm, + int fd, + virDomainAsyncJob asyncJob) +{ + qemuDomainObjPrivate *priv =3D vm->privateData; + g_autoptr(qemuMigrationParams) saveParams =3D NULL; + unsigned long saveMigBandwidth =3D priv->migMaxBandwidth; + int rc; + int ret =3D -1; + virErrorPtr orig_err =3D NULL; + + if (qemuMigrationSetDBusVMState(driver, vm) < 0) + return -1; + + if (!(saveParams =3D qemuMigrationParamsForMappedSave())) + return -1; + + /* Increase migration bandwidth to unlimited since target is a file. + * Failure to change migration speed is not fatal. */ + if (qemuMigrationParamsSetULL(saveParams, + QEMU_MIGRATION_PARAM_MAX_BANDWIDTH, + QEMU_DOMAIN_MIG_BANDWIDTH_MAX * 1024 * 1= 024) < 0) + return -1; + + if (qemuMigrationParamsApply(vm, asyncJob, saveParams, 0) < 0) + return -1; + + priv->migMaxBandwidth =3D QEMU_DOMAIN_MIG_BANDWIDTH_MAX; + + if (!virDomainObjIsActive(vm)) { + virReportError(VIR_ERR_INTERNAL_ERROR, "%s", + _("guest unexpectedly quit")); + /* nothing to tear down */ + return -1; + } + + if (qemuDomainObjEnterMonitorAsync(vm, asyncJob) < 0) + goto cleanup; + + rc =3D qemuMonitorMigrateToFdSet(vm, 0, fd); + qemuDomainObjExitMonitor(vm); + if (rc < 0) + goto cleanup; + + rc =3D qemuMigrationSrcWaitForCompletion(vm, asyncJob, NULL, 0); + + if (rc < 0) { + if (rc =3D=3D -2) { + virErrorPreserveLast(&orig_err); + if (virDomainObjIsActive(vm)) + qemuMigrationSrcCancel(vm, asyncJob, true); + } + goto cleanup; + } + + qemuDomainEventEmitJobCompleted(driver, vm); + ret =3D 0; + + cleanup: + if (ret < 0 && !orig_err) + virErrorPreserveLast(&orig_err); + + /* Restore max migration bandwidth */ + if (virDomainObjIsActive(vm)) { + if (qemuMigrationParamsSetULL(saveParams, + QEMU_MIGRATION_PARAM_MAX_BANDWIDTH, + saveMigBandwidth * 1024 * 1024) =3D= =3D 0) + ignore_value(qemuMigrationParamsApply(vm, asyncJob, + saveParams, 0)); + priv->migMaxBandwidth =3D saveMigBandwidth; + } + + virErrorRestore(&orig_err); + + return ret; +} + + /** * This function is supposed to be used only to while reconnecting to a do= main * with an active migration job. diff --git a/src/qemu/qemu_migration.h b/src/qemu/qemu_migration.h index ed62fd4a91..f845a0198b 100644 --- a/src/qemu/qemu_migration.h +++ b/src/qemu/qemu_migration.h @@ -241,6 +241,13 @@ qemuMigrationSrcToFile(virQEMUDriver *driver, virDomainAsyncJob asyncJob) ATTRIBUTE_NONNULL(1) ATTRIBUTE_NONNULL(2) G_GNUC_WARN_UNUSED_RESULT; =20 +int +qemuMigrationSrcToMappedFile(virQEMUDriver *driver, + virDomainObj *vm, + int fd, + virDomainAsyncJob asyncJob) + ATTRIBUTE_NONNULL(1) ATTRIBUTE_NONNULL(2) G_GNUC_WARN_UNUSED_RESULT; + int qemuMigrationSrcCancelUnattended(virDomainObj *vm, virDomainJobObj *oldJob); diff --git a/src/qemu/qemu_monitor.c b/src/qemu/qemu_monitor.c index 34e2ccab97..4c92bd740a 100644 --- a/src/qemu/qemu_monitor.c +++ b/src/qemu/qemu_monitor.c @@ -2237,6 +2237,38 @@ qemuMonitorMigrateToFd(qemuMonitor *mon, } =20 =20 +int +qemuMonitorMigrateToFdSet(virDomainObj *vm, + unsigned int flags, + int fd) +{ + qemuDomainObjPrivate *priv =3D vm->privateData; + qemuMonitor *mon =3D priv->mon; + off_t offset; + g_autoptr(qemuFDPass) fdPassMigrate =3D NULL; + unsigned int setId; + g_autofree char *uri =3D NULL; + + VIR_DEBUG("fd=3D%d flags=3D0x%x", fd, flags); + + QEMU_CHECK_MONITOR(mon); + + if ((offset =3D lseek(fd, 0, SEEK_CUR)) =3D=3D -1) + return -1; + + fdPassMigrate =3D qemuFDPassNew("migrate", priv); + qemuFDPassAddFD(fdPassMigrate, &fd, "-fd"); + qemuFDPassTransferMonitor(fdPassMigrate, mon); + + if (qemuFDPassGetId(fdPassMigrate, &setId) < 0) + return -1; + + uri =3D g_strdup_printf("file:/dev/fdset/%u,offset=3D%#lx", setId, off= set); + + return qemuMonitorJSONMigrate(mon, flags, uri); +} + + int qemuMonitorMigrateToHost(qemuMonitor *mon, unsigned int flags, diff --git a/src/qemu/qemu_monitor.h b/src/qemu/qemu_monitor.h index 6e81945201..c477def138 100644 --- a/src/qemu/qemu_monitor.h +++ b/src/qemu/qemu_monitor.h @@ -843,6 +843,10 @@ int qemuMonitorMigrateToFd(qemuMonitor *mon, unsigned int flags, int fd); =20 +int qemuMonitorMigrateToFdSet(virDomainObj *vm, + unsigned int flags, + int fd); + int qemuMonitorMigrateToHost(qemuMonitor *mon, unsigned int flags, const char *protocol, diff --git a/src/qemu/qemu_saveimage.c b/src/qemu/qemu_saveimage.c index 30085dc7bc..8f28770086 100644 --- a/src/qemu/qemu_saveimage.c +++ b/src/qemu/qemu_saveimage.c @@ -96,6 +96,7 @@ G_DEFINE_AUTOPTR_CLEANUP_FUNC(virQEMUSaveData, virQEMUSav= eDataFree); */ virQEMUSaveData * virQEMUSaveDataNew(virQEMUDriver *driver, + virDomainObj *vm, char *domXML, qemuDomainSaveCookie *cookieObj, bool running, @@ -122,6 +123,10 @@ virQEMUSaveDataNew(virQEMUDriver *driver, goto error; } header->version =3D cfg->saveImageVersion; + /* Enable mapped-ram feature if available and save version >=3D 3 */ + if (header->version >=3D QEMU_SAVE_VERSION && + qemuMigrationCapsGet(vm, QEMU_MIGRATION_CAP_MAPPED_RAM)) + header->features |=3D QEMU_SAVE_FEATURE_MAPPED_RAM; =20 header->was_running =3D running ? 1 : 0; header->compressed =3D compressed; @@ -369,6 +374,79 @@ qemuSaveImageDecompressionStop(virCommand *cmd, } =20 =20 +static int +qemuSaveImageCreateSequential(virQEMUDriver *driver, + virDomainObj *vm, + const char *path, + int fd, + virQEMUSaveData *data, + virCommand *compressor, + unsigned int flags, + virDomainAsyncJob asyncJob) +{ + int ret =3D -1; + virFileWrapperFd *wrapperFd =3D NULL; + unsigned int wrapperFlags =3D VIR_FILE_WRAPPER_NON_BLOCKING; + + if ((flags & VIR_DOMAIN_SAVE_BYPASS_CACHE)) + wrapperFlags |=3D VIR_FILE_WRAPPER_BYPASS_CACHE; + + if (!(wrapperFd =3D virFileWrapperFdNew(&fd, path, wrapperFlags))) + goto cleanup; + + if (virQEMUSaveDataWrite(data, fd, path) < 0) + goto cleanup; + + /* Perform the migration */ + if (qemuMigrationSrcToFile(driver, vm, fd, compressor, asyncJob) < 0) + goto cleanup; + + if (VIR_CLOSE(fd) < 0) { + virReportSystemError(errno, _("unable to close %1$s"), path); + goto cleanup; + } + + if (qemuDomainFileWrapperFDClose(vm, wrapperFd) < 0) + goto cleanup; + + ret =3D 0; + + cleanup: + VIR_FORCE_CLOSE(fd); + if (qemuDomainFileWrapperFDClose(vm, wrapperFd) < 0) + ret =3D -1; + virFileWrapperFdFree(wrapperFd); + + return ret; +} + + +static int +qemuSaveImageCreateMapped(virQEMUDriver *driver, + virDomainObj *vm, + const char *path, + int fd, + virQEMUSaveData *data, + unsigned int flags, + virDomainAsyncJob asyncJob) +{ + g_autoptr(virQEMUDriverConfig) cfg =3D virQEMUDriverGetConfig(driver); + + /* mapped-ram does not support directIO */ + if ((flags & VIR_DOMAIN_SAVE_BYPASS_CACHE)) { + virReportError(VIR_ERR_OPERATION_FAILED, "%s", + _("bypass cache unsupported by this system")); + return -1; + } + + if (virQEMUSaveDataWrite(data, fd, path) < 0) + return -1; + + /* Perform the migration */ + return qemuMigrationSrcToMappedFile(driver, vm, fd, asyncJob); +} + + /* Helper function to execute a migration to file with a correct save head= er * the caller needs to make sure that the processors are stopped and do al= l other * actions besides saving memory */ @@ -386,12 +464,9 @@ qemuSaveImageCreate(virQEMUDriver *driver, int ret =3D -1; int fd =3D -1; int directFlag =3D 0; - virFileWrapperFd *wrapperFd =3D NULL; - unsigned int wrapperFlags =3D VIR_FILE_WRAPPER_NON_BLOCKING; =20 /* Obtain the file handle. */ if ((flags & VIR_DOMAIN_SAVE_BYPASS_CACHE)) { - wrapperFlags |=3D VIR_FILE_WRAPPER_BYPASS_CACHE; directFlag =3D virFileDirectFdFlag(); if (directFlag < 0) { virReportError(VIR_ERR_OPERATION_FAILED, "%s", @@ -409,14 +484,12 @@ qemuSaveImageCreate(virQEMUDriver *driver, if (qemuSecuritySetImageFDLabel(driver->securityManager, vm->def, fd) = < 0) goto cleanup; =20 - if (!(wrapperFd =3D virFileWrapperFdNew(&fd, path, wrapperFlags))) - goto cleanup; + if (data->header.features & QEMU_SAVE_FEATURE_MAPPED_RAM) + ret =3D qemuSaveImageCreateMapped(driver, vm, path, fd, data, flag= s, asyncJob); + else + ret =3D qemuSaveImageCreateSequential(driver, vm, path, fd, data, = compressor, flags, asyncJob); =20 - if (virQEMUSaveDataWrite(data, fd, path) < 0) - goto cleanup; - - /* Perform the migration */ - if (qemuMigrationSrcToFile(driver, vm, fd, compressor, asyncJob) < 0) + if (ret < 0) goto cleanup; =20 /* Touch up file header to mark image complete. */ @@ -425,14 +498,6 @@ qemuSaveImageCreate(virQEMUDriver *driver, * up to seek backwards on wrapperFd. The reopened fd will * trigger a single page of file system cache pollution, but * that's acceptable. */ - if (VIR_CLOSE(fd) < 0) { - virReportSystemError(errno, _("unable to close %1$s"), path); - goto cleanup; - } - - if (qemuDomainFileWrapperFDClose(vm, wrapperFd) < 0) - goto cleanup; - if ((fd =3D qemuDomainOpenFile(cfg, vm->def, path, O_WRONLY, NULL)) < = 0 || virQEMUSaveDataFinish(data, &fd, path) < 0) goto cleanup; @@ -441,10 +506,6 @@ qemuSaveImageCreate(virQEMUDriver *driver, =20 cleanup: VIR_FORCE_CLOSE(fd); - if (qemuDomainFileWrapperFDClose(vm, wrapperFd) < 0) - ret =3D -1; - virFileWrapperFdFree(wrapperFd); - if (ret < 0 && needUnlink) unlink(path); =20 diff --git a/src/qemu/qemu_saveimage.h b/src/qemu/qemu_saveimage.h index 63ad5508ed..81d93bf33c 100644 --- a/src/qemu/qemu_saveimage.h +++ b/src/qemu/qemu_saveimage.h @@ -124,6 +124,7 @@ virQEMUSaveDataWrite(virQEMUSaveData *data, =20 virQEMUSaveData * virQEMUSaveDataNew(virQEMUDriver *driver, + virDomainObj *vm, char *domXML, qemuDomainSaveCookie *cookieObj, bool running, diff --git a/src/qemu/qemu_snapshot.c b/src/qemu/qemu_snapshot.c index 1d75208814..1e9e0e31d7 100644 --- a/src/qemu/qemu_snapshot.c +++ b/src/qemu/qemu_snapshot.c @@ -1390,7 +1390,7 @@ qemuSnapshotCreateActiveExternal(virQEMUDriver *drive= r, !(snapdef->cookie =3D (virObject *) qemuDomainSaveCookieNew(vm= ))) goto cleanup; =20 - if (!(data =3D virQEMUSaveDataNew(driver, xml, + if (!(data =3D virQEMUSaveDataNew(driver, vm, xml, (qemuDomainSaveCookie *) snapdef->= cookie, resume, compressed))) goto cleanup; --=20 2.44.0