From nobody Wed Mar 12 16:58:03 2025 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.libvirt.org designates 8.43.85.245 as permitted sender) client-ip=8.43.85.245; envelope-from=devel-bounces@lists.libvirt.org; helo=lists.libvirt.org; Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of lists.libvirt.org designates 8.43.85.245 as permitted sender) smtp.mailfrom=devel-bounces@lists.libvirt.org; dmarc=pass(p=reject dis=none) header.from=lists.libvirt.org ARC-Seal: i=1; a=rsa-sha256; t=1741215384; cv=none; d=zohomail.com; s=zohoarc; b=ctt4w73QXxso8Y3E8JMMLAZzjWy3r2+6AoXHigYZdZTp32GMbbYr0dYeX0mxenLqi2h11fCnVtLTDSuk3R/N4kZ6Z7aO7wOI/IEnisotoE0UiPaetbeVPV7jT6k1Us8sXRNEv4eEWbLzPkoToEIYkk0hMsASiG3Ja1kffdbDbKM= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1741215384; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:Reply-To:Reply-To:References:Subject:Subject:To:To:Message-Id; bh=P5FZ0LeU5ZO1gL8jbOOoZlf1WMB+XDJMOd05Z5fAkhg=; b=Rzn/Ck/l754HaGSZlnnkdZKjo9ZatWEo+/OqRTcdc5XG2TmB3pEdFrLf785X71i7kJFsvqct9ceAdB054ASGtH2QF2JE6QskFoCsd0FlDhJspXN45ymy4/LoQnFTuM9USBbMLH3vAQRpUAcZMZABNsIwQVYLGq/IS8sSendR8i4= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of lists.libvirt.org designates 8.43.85.245 as permitted sender) smtp.mailfrom=devel-bounces@lists.libvirt.org; dmarc=pass header.from= (p=reject dis=none) Return-Path: Received: from lists.libvirt.org (lists.libvirt.org [8.43.85.245]) by mx.zohomail.com with SMTPS id 1741215384249288.55626430834786; Wed, 5 Mar 2025 14:56:24 -0800 (PST) Received: by lists.libvirt.org (Postfix, from userid 996) id 8263C15D0; Wed, 5 Mar 2025 17:56:23 -0500 (EST) Received: from lists.libvirt.org (localhost [IPv6:::1]) by lists.libvirt.org (Postfix) with ESMTP id 827571648; Wed, 5 Mar 2025 17:50:53 -0500 (EST) Received: by lists.libvirt.org (Postfix, from userid 996) id 665B915D2; Wed, 5 Mar 2025 17:50:43 -0500 (EST) Received: from mail-wr1-f49.google.com (mail-wr1-f49.google.com [209.85.221.49]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by lists.libvirt.org (Postfix) with ESMTPS id 4173016E8 for ; Wed, 5 Mar 2025 17:50:24 -0500 (EST) Received: by mail-wr1-f49.google.com with SMTP id ffacd0b85a97d-390e3b3d3bcso1518f8f.3 for ; Wed, 05 Mar 2025 14:50:24 -0800 (PST) Received: from localhost (205-201-32-8.scinternet.net. [205.201.32.8]) by smtp.gmail.com with UTF8SMTPSA id d2e1a72fcca58-734a003ec9csm13981988b3a.146.2025.03.05.14.50.22 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 05 Mar 2025 14:50:22 -0800 (PST) X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on lists.libvirt.org X-Spam-Level: X-Spam-Status: No, score=-0.8 required=5.0 tests=DKIM_INVALID,DKIM_SIGNED, MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE,RCVD_IN_MSPIKE_H2, RCVD_IN_VALIDITY_RPBL_BLOCKED,RCVD_IN_VALIDITY_SAFE_BLOCKED, SPF_HELO_NONE autolearn=unavailable autolearn_force=no version=3.4.4 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=google; t=1741215023; x=1741819823; darn=lists.libvirt.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=oMX2NWWASV24LCXTcVgFNm0/xfawCDIUCNmwPVEmWgk=; b=LyzNCuipw9Yrx6N9rpXu16yGDO1Qw5CYb70L/WCp7apkOlGNmiFpSvaWqyltrDN11l w3Hg7xESxzf6i/ot01uttY1Ik9AE2vKlzOukdkQTETlCsviwuQN9pR6bQGuIsjASU5h2 X731Hi0oakL2NbCY46IY/SmuMeoabUeOq9VpNpHhfIvfDcuBM3xBS4oTZo71V9OytNvR LpuTxs1muhU9r6JjY5JDU4Kdr8AVShZ+878lg8Euxc0mfViR6CaCSbn4UWrD97ayfFkm PE2lwx4BJKVITwm7k9b2tWjTzduQ8nC6UeXSQwKSPJZMMfd3M6WUsNm1VhpZAhWFGcut RgtQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1741215023; x=1741819823; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=oMX2NWWASV24LCXTcVgFNm0/xfawCDIUCNmwPVEmWgk=; b=mvbyzNeVvX2uNRs2YlJvLFw3rXmiHNP9K9j/SengBUsu8mP00jzUdi40RvVOQAnl2y GCMbK9wuTE12iej0HgR+fGvTbkBPZruFu/emoKZA/6W5FPi83tOAe1iOQPhYnhbunQnL Q7GEeLqEb+bTSgAjjAlfApc5KvAsn8bhuuagqoQai82RhY/gek9gIdOOZviv+8O8hg3c HRmsZloUMcYaxAV7tUvDCh/B5Z75hJ7+vcGJMu33gqEJ00g7up98BWR8LIp5DLA8zg/F e44fdipPiFamJuplny+16RaSAfjkzcIt1Awmt9r3bKKqBFjVBmK1/wsQ8xIe2O6c0PYC haJQ== X-Gm-Message-State: AOJu0Yxtql/4XhSSa6r1gm/4F2LyGopNwY5fQCvlza9ho6Se1Oi42JHf KjeEvA+566yuoCHTzQSpYk0LmnR3aC6bPkezcyPjwAdzr2fYjh3XfybwpPYu5oSBRk52RQGi9cU 2 X-Gm-Gg: ASbGnctWhbJWRmPbmiZnEQ0uVqYxttclILW9Jy9J+Vc2wbJmjXSKtCw5z2dn3eqlw5B l7HAy4emAA6mv/8CZKTuAduh7zPx8ZFxaacEzPoOUZirl6b3rop8YUUhCdD2G0hf/GppLV0ij+c HdX89pJjdwnGRGcDzBDT4dUwHWi4tvTS6KEUPQXBvrPrS5oJfwZ1tQNDeMrtn4gogG1vkyx8YVx 4da3kuk/cTQEjAIlu022BWMjJO/JHpcccmgIASRkQbCt33OWZ2UyLpbnVf/uv4n1ZmD2O8Wyc9O jxHXtI9jq6med4cWvuKE24pCuS/sNoASAq7As/qXJNglJerCz/xF/pdML0HXMYY= X-Google-Smtp-Source: AGHT+IGFVmhlMhK7AHVSPl+pwWVEX2FhRqpMYRWWnFORx+AOccDrx6u5XelJEqTqVo633T8EDtER/g== X-Received: by 2002:a05:6000:4189:b0:390:d82d:6d4f with SMTP id ffacd0b85a97d-3911f7bf98dmr3387511f8f.52.1741215022944; Wed, 05 Mar 2025 14:50:22 -0800 (PST) To: devel@lists.libvirt.org Subject: [PATCH V4 13/18] qemu: Support O_DIRECT with mapped-ram on save Date: Wed, 5 Mar 2025 15:48:22 -0700 Message-ID: <20250305224957.5369-14-jfehlig@suse.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250305224957.5369-1-jfehlig@suse.com> References: <20250305224957.5369-1-jfehlig@suse.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Message-ID-Hash: YOTPHE6OZCMK3UQQWW3Q7JBK26DA4JXX X-Message-ID-Hash: YOTPHE6OZCMK3UQQWW3Q7JBK26DA4JXX X-MailFrom: jfehlig@suse.com X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation; header-match-config-1; header-match-config-2; header-match-config-3; header-match-devel.lists.libvirt.org-0; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; suspicious-header CC: farosas@suse.de X-Mailman-Version: 3.2.2 Precedence: list List-Id: Development discussions about the libvirt library & tools Archived-At: List-Archive: List-Help: List-Post: List-Subscribe: List-Unsubscribe: From: Jim Fehlig via Devel Reply-To: Jim Fehlig X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZM-MESSAGEID: 1741215385989019000 Content-Type: text/plain; charset="utf-8" When using the mapped-ram migration capability, direct IO is enabled by setting the "direct-io" migration parameter to "true" and passing QEMU an additional fd with O_DIRECT set. Signed-off-by: Jim Fehlig --- src/qemu/qemu_driver.c | 10 ++++++---- src/qemu/qemu_migration.c | 32 ++++++++++++++++++++++++++------ src/qemu/qemu_migration.h | 1 + src/qemu/qemu_migration_params.c | 11 ++++++++++- src/qemu/qemu_migration_params.h | 3 ++- src/qemu/qemu_monitor.c | 7 +++++-- src/qemu/qemu_monitor.h | 3 ++- src/qemu/qemu_saveimage.c | 2 +- 8 files changed, 53 insertions(+), 16 deletions(-) diff --git a/src/qemu/qemu_driver.c b/src/qemu/qemu_driver.c index 11a67fab57..e098a96a4e 100644 --- a/src/qemu/qemu_driver.c +++ b/src/qemu/qemu_driver.c @@ -2679,7 +2679,8 @@ qemuDomainSaveInternal(virQEMUDriver *driver, goto endjob; xml =3D NULL; =20 - if (!(saveParams =3D qemuMigrationParamsForSave(format =3D=3D QEMU_SAV= E_FORMAT_SPARSE))) + if (!(saveParams =3D qemuMigrationParamsForSave(format =3D=3D QEMU_SAV= E_FORMAT_SPARSE, + flags))) goto endjob; =20 ret =3D qemuSaveImageCreate(driver, vm, path, data, compressor, @@ -3129,7 +3130,7 @@ doCoreDump(virQEMUDriver *driver, if (!(dump_params =3D qemuMigrationParamsNew())) goto cleanup; =20 - if (qemuMigrationSrcToFile(driver, vm, &fd, compressor, + if (qemuMigrationSrcToFile(driver, vm, path, &fd, compressor, dump_params, dump_flags, VIR_ASYNC_JOB_= DUMP) < 0) goto cleanup; } @@ -5773,7 +5774,7 @@ qemuDomainRestoreInternal(virConnectPtr conn, goto cleanup; =20 sparse =3D data->header.format =3D=3D QEMU_SAVE_FORMAT_SPARSE; - if (!(restoreParams =3D qemuMigrationParamsForSave(sparse))) + if (!(restoreParams =3D qemuMigrationParamsForSave(sparse, flags))) goto cleanup; =20 fd =3D qemuSaveImageOpen(driver, path, @@ -6107,7 +6108,8 @@ qemuDomainObjRestore(virConnectPtr conn, } =20 sparse =3D data->header.format =3D=3D QEMU_SAVE_FORMAT_SPARSE; - if (!(restoreParams =3D qemuMigrationParamsForSave(sparse))) + if (!(restoreParams =3D qemuMigrationParamsForSave(sparse, + bypass_cache ? VIR_DO= MAIN_SAVE_BYPASS_CACHE : 0))) return -1; =20 fd =3D qemuSaveImageOpen(driver, path, bypass_cache, sparse, &wrapperF= d, false); diff --git a/src/qemu/qemu_migration.c b/src/qemu/qemu_migration.c index 1a85ec1c51..4554a52165 100644 --- a/src/qemu/qemu_migration.c +++ b/src/qemu/qemu_migration.c @@ -7188,17 +7188,36 @@ qemuMigrationSrcToLegacyFile(virQEMUDriver *driver, static int qemuMigrationSrcToSparseFile(virQEMUDriver *driver, virDomainObj *vm, + const char *path, int *fd, unsigned int flags, virDomainAsyncJob asyncJob) { + g_autoptr(virQEMUDriverConfig) cfg =3D virQEMUDriverGetConfig(driver); + VIR_AUTOCLOSE directFd =3D -1; + int directFlag =3D 0; + bool needUnlink =3D false; int ret; =20 - /* mapped-ram does not support directIO */ + /* When using directio with mapped-ram, qemu needs two fds. One with + * O_DIRECT set writing the memory, and another without it set for + * writing small bits of unaligned state. */ if ((flags & VIR_DOMAIN_SAVE_BYPASS_CACHE)) { - virReportError(VIR_ERR_OPERATION_FAILED, "%s", - _("bypass cache unsupported by this system")); - return -1; + directFlag =3D virFileDirectFdFlag(); + if (directFlag < 0) { + virReportError(VIR_ERR_OPERATION_FAILED, "%s", + _("bypass cache unsupported by this system")); + return -1; + } + directFd =3D virQEMUFileOpenAs(cfg->user, cfg->group, false, path, + O_WRONLY | directFlag, &needUnlink); + + if (directFd < 0) + return -1; + + if (qemuSecuritySetImageFDLabel(driver->securityManager, vm->def, = directFd) < 0) + return -1; + } =20 if (qemuSecuritySetImageFDLabel(driver->securityManager, vm->def, *fd)= < 0) @@ -7207,7 +7226,7 @@ qemuMigrationSrcToSparseFile(virQEMUDriver *driver, if (qemuDomainObjEnterMonitorAsync(vm, asyncJob) < 0) return -1; =20 - ret =3D qemuMonitorMigrateToFdSet(vm, 0, fd); + ret =3D qemuMonitorMigrateToFdSet(vm, 0, fd, &directFd); qemuDomainObjExitMonitor(vm); return ret; } @@ -7216,6 +7235,7 @@ qemuMigrationSrcToSparseFile(virQEMUDriver *driver, /* Helper function called while vm is active. */ int qemuMigrationSrcToFile(virQEMUDriver *driver, virDomainObj *vm, + const char *path, int *fd, virCommand *compressor, qemuMigrationParams *migParams, @@ -7253,7 +7273,7 @@ qemuMigrationSrcToFile(virQEMUDriver *driver, virDoma= inObj *vm, =20 if (migParams && qemuMigrationParamsCapEnabled(migParams, QEMU_MIGRATION_CAP_MAPPED= _RAM)) - rc =3D qemuMigrationSrcToSparseFile(driver, vm, fd, flags, asyncJo= b); + rc =3D qemuMigrationSrcToSparseFile(driver, vm, path, fd, flags, a= syncJob); else rc =3D qemuMigrationSrcToLegacyFile(driver, vm, *fd, compressor, a= syncJob); =20 diff --git a/src/qemu/qemu_migration.h b/src/qemu/qemu_migration.h index 71a9974753..beb888160a 100644 --- a/src/qemu/qemu_migration.h +++ b/src/qemu/qemu_migration.h @@ -238,6 +238,7 @@ qemuMigrationSrcIsAllowed(virDomainObj *vm, int qemuMigrationSrcToFile(virQEMUDriver *driver, virDomainObj *vm, + const char *path, int *fd, virCommand *compressor, qemuMigrationParams *migParams, diff --git a/src/qemu/qemu_migration_params.c b/src/qemu/qemu_migration_par= ams.c index d48cdd5506..16e93f27d6 100644 --- a/src/qemu/qemu_migration_params.c +++ b/src/qemu/qemu_migration_params.c @@ -130,6 +130,7 @@ VIR_ENUM_IMPL(qemuMigrationParam, "multifd-zlib-level", "multifd-zstd-level", "avail-switchover-bandwidth", + "direct-io", ); =20 typedef struct _qemuMigrationParamsAlwaysOnItem qemuMigrationParamsAlwaysO= nItem; @@ -328,6 +329,9 @@ static const qemuMigrationParamInfoItem qemuMigrationPa= ramInfo[] =3D { [QEMU_MIGRATION_PARAM_AVAIL_SWITCHOVER_BANDWIDTH] =3D { .type =3D QEMU_MIGRATION_PARAM_TYPE_ULL, }, + [QEMU_MIGRATION_PARAM_DIRECT_IO] =3D { + .type =3D QEMU_MIGRATION_PARAM_TYPE_BOOL, + }, }; G_STATIC_ASSERT(G_N_ELEMENTS(qemuMigrationParamInfo) =3D=3D QEMU_MIGRATION= _PARAM_LAST); =20 @@ -793,7 +797,7 @@ qemuMigrationParamsFromFlags(virTypedParameterPtr param= s, =20 =20 qemuMigrationParams * -qemuMigrationParamsForSave(bool sparse) +qemuMigrationParamsForSave(bool sparse, unsigned int flags) { g_autoptr(qemuMigrationParams) saveParams =3D NULL; =20 @@ -807,6 +811,11 @@ qemuMigrationParamsForSave(bool sparse) return NULL; saveParams->params[QEMU_MIGRATION_PARAM_MULTIFD_CHANNELS].value.i = =3D 1; saveParams->params[QEMU_MIGRATION_PARAM_MULTIFD_CHANNELS].set =3D = true; + + if (flags & VIR_DOMAIN_SAVE_BYPASS_CACHE) { + saveParams->params[QEMU_MIGRATION_PARAM_DIRECT_IO].value.b =3D= true; + saveParams->params[QEMU_MIGRATION_PARAM_DIRECT_IO].set =3D tru= e; + } } =20 return g_steal_pointer(&saveParams); diff --git a/src/qemu/qemu_migration_params.h b/src/qemu/qemu_migration_par= ams.h index 88a1bc1a66..87822332d3 100644 --- a/src/qemu/qemu_migration_params.h +++ b/src/qemu/qemu_migration_params.h @@ -66,6 +66,7 @@ typedef enum { QEMU_MIGRATION_PARAM_MULTIFD_ZLIB_LEVEL, QEMU_MIGRATION_PARAM_MULTIFD_ZSTD_LEVEL, QEMU_MIGRATION_PARAM_AVAIL_SWITCHOVER_BANDWIDTH, + QEMU_MIGRATION_PARAM_DIRECT_IO, =20 QEMU_MIGRATION_PARAM_LAST } qemuMigrationParam; @@ -88,7 +89,7 @@ qemuMigrationParamsFromFlags(virTypedParameterPtr params, qemuMigrationParty party); =20 qemuMigrationParams * -qemuMigrationParamsForSave(bool sparse); +qemuMigrationParamsForSave(bool sparse, unsigned int flags); =20 int qemuMigrationParamsDump(qemuMigrationParams *migParams, diff --git a/src/qemu/qemu_monitor.c b/src/qemu/qemu_monitor.c index 8d244c6f4a..3b26fab90d 100644 --- a/src/qemu/qemu_monitor.c +++ b/src/qemu/qemu_monitor.c @@ -2235,7 +2235,8 @@ qemuMonitorMigrateToFd(qemuMonitor *mon, int qemuMonitorMigrateToFdSet(virDomainObj *vm, unsigned int flags, - int *fd) + int *fd, + int *directFd) { qemuDomainObjPrivate *priv =3D vm->privateData; qemuMonitor *mon =3D priv->mon; @@ -2244,7 +2245,7 @@ qemuMonitorMigrateToFdSet(virDomainObj *vm, g_autofree char *uri =3D NULL; int ret; =20 - VIR_DEBUG("fd=3D%d flags=3D0x%x", *fd, flags); + VIR_DEBUG("fd=3D%d directFd=3D%d flags=3D0x%x", *fd, *directFd, flags); =20 QEMU_CHECK_MONITOR(mon); =20 @@ -2256,6 +2257,8 @@ qemuMonitorMigrateToFdSet(virDomainObj *vm, =20 fdPassMigrate =3D qemuFDPassNew("libvirt-outgoing-migrate", priv); qemuFDPassAddFD(fdPassMigrate, fd, "-fd"); + if (*directFd !=3D -1) + qemuFDPassAddFD(fdPassMigrate, directFd, "-directio-fd"); qemuFDPassTransferMonitor(fdPassMigrate, mon); =20 uri =3D g_strdup_printf("file:%s,offset=3D%#lx", diff --git a/src/qemu/qemu_monitor.h b/src/qemu/qemu_monitor.h index 4022bc3c7b..d5fa157696 100644 --- a/src/qemu/qemu_monitor.h +++ b/src/qemu/qemu_monitor.h @@ -861,7 +861,8 @@ int qemuMonitorMigrateToFd(qemuMonitor *mon, =20 int qemuMonitorMigrateToFdSet(virDomainObj *vm, unsigned int flags, - int *fd); + int *fd, + int *directFd); =20 int qemuMonitorMigrateToHost(qemuMonitor *mon, unsigned int flags, diff --git a/src/qemu/qemu_saveimage.c b/src/qemu/qemu_saveimage.c index f00078bfcf..4c91c1be67 100644 --- a/src/qemu/qemu_saveimage.c +++ b/src/qemu/qemu_saveimage.c @@ -501,7 +501,7 @@ qemuSaveImageCreate(virQEMUDriver *driver, goto cleanup; =20 /* Perform the migration */ - if (qemuMigrationSrcToFile(driver, vm, &fd, compressor, saveParams, fl= ags, asyncJob) < 0) + if (qemuMigrationSrcToFile(driver, vm, path, &fd, compressor, savePara= ms, flags, asyncJob) < 0) goto cleanup; =20 /* Touch up file header to mark image complete. */ --=20 2.43.0