From nobody Fri Dec 19 20:18:31 2025 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.libvirt.org designates 8.43.85.245 as permitted sender) client-ip=8.43.85.245; envelope-from=devel-bounces@lists.libvirt.org; helo=lists.libvirt.org; Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of lists.libvirt.org designates 8.43.85.245 as permitted sender) smtp.mailfrom=devel-bounces@lists.libvirt.org; dmarc=pass(p=reject dis=none) header.from=lists.libvirt.org ARC-Seal: i=1; a=rsa-sha256; t=1738953890; cv=none; d=zohomail.com; s=zohoarc; b=C4ShwHcUyHy8sXWJXugi0RPU6vWBAAHiHQFPxK1o6nKYnbEWKpgG2oxsgV/2EbTSz8N78nr8KK19DPf72/JYBzwj7Cld804aVb7YBL/d2Ffr0eZYgefcULnAYx4ZL8tIS8cO0QQGfGoGNJunsVXdD1ZwaMaZOqNajv890hxbniQ= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1738953890; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:Reply-To:Reply-To:References:Subject:Subject:To:To:Message-Id; bh=uFgXsFhLQbLt1FR1N6R+Dqdi6At3OfvVGuZd63JXBNE=; b=fFDsiXnchu8czIVHeGa79RsgvnVBVAQBbrQBBbhJTUdCiRdaXAeMhuqQ4C77LSaosf3o/hFhr6H6YlI8utHzm1+K5pX9TgmqWssZtdzHFa/BqCv+BAHztqkiIM/oANAANGczXh0N9pKp/674ziZpV6DulZXmcI8s/PduCyFJLaM= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of lists.libvirt.org designates 8.43.85.245 as permitted sender) smtp.mailfrom=devel-bounces@lists.libvirt.org; dmarc=pass header.from= (p=reject dis=none) Return-Path: Received: from lists.libvirt.org (lists.libvirt.org [8.43.85.245]) by mx.zohomail.com with SMTPS id 1738953889950702.167262763239; Fri, 7 Feb 2025 10:44:49 -0800 (PST) Received: by lists.libvirt.org (Postfix, from userid 996) id 504461939; Fri, 7 Feb 2025 13:44:49 -0500 (EST) Received: from lists.libvirt.org (localhost [IPv6:::1]) by lists.libvirt.org (Postfix) with ESMTP id E10BF1613; Fri, 7 Feb 2025 13:38:31 -0500 (EST) Received: by lists.libvirt.org (Postfix, from userid 996) id 0E4FC1531; Fri, 7 Feb 2025 13:38:24 -0500 (EST) Received: from mail-wm1-f53.google.com (mail-wm1-f53.google.com [209.85.128.53]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by lists.libvirt.org (Postfix) with ESMTPS id 3AE771516 for ; Fri, 7 Feb 2025 13:38:05 -0500 (EST) Received: by mail-wm1-f53.google.com with SMTP id 5b1f17b1804b1-43625c4a50dso16424325e9.0 for ; Fri, 07 Feb 2025 10:38:05 -0800 (PST) Received: from localhost (205-201-32-8.scinternet.net. [205.201.32.8]) by smtp.gmail.com with UTF8SMTPSA id d2e1a72fcca58-73048c1634asm3444503b3a.147.2025.02.07.10.38.03 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 07 Feb 2025 10:38:03 -0800 (PST) X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on lists.libvirt.org X-Spam-Level: X-Spam-Status: No, score=-0.8 required=5.0 tests=DKIM_INVALID,DKIM_SIGNED, MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE,RCVD_IN_MSPIKE_H2, RCVD_IN_VALIDITY_RPBL_BLOCKED,RCVD_IN_VALIDITY_SAFE_BLOCKED, SPF_HELO_NONE autolearn=unavailable autolearn_force=no version=3.4.4 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=google; t=1738953484; x=1739558284; darn=lists.libvirt.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=YQSC1QcYlkGHVhDInXl9Hs65Nlm4+sBFCtKoPnFB4dA=; b=dht5cuxYnfZn0lwr/XhCpjMPUd8yKSWj4t2dzWQfd2RWPvkbqpt0xdTtSWlbJt4UCy BfjyrY/TYOhMEZQZ0reOWxnGrzIhPyt/9Y2kaLMgHTQX+o2IDwB8s+92yw5s0TUVBdRV bjDY1PhWDHE97qcH0Oj97W/spwpYZmz8f9d6NLWOdgF5yNsX/OccG2+3oI2RQ/mQM/TQ FOft33S9h9rARiIA76pabxBTKn/jEPebgZ4OWdGs2qZLoM1oalsDvnB3uFDxlJoFByKR 1Q/Ad4sznCrJg8soIW9XoPVLQEkDUr/MPew1A4f0LD3Dcd1Iga6dPMFNf4f33f8BRnNo 3ZFw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1738953484; x=1739558284; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=YQSC1QcYlkGHVhDInXl9Hs65Nlm4+sBFCtKoPnFB4dA=; b=vq2IDmSMgu3yrhLTJJMxXF2z2J0vUWZprjixMaat4W3bGO6H/mR68iOZkOyCbRGt4T qgVeuGpwcU31qj/afvoziv6HoFC79BiLG7hW2sGaocrc/yzoh5p/9ILDiMJ7k/ietTcc A5Tz8/0hV8XMRJf0v3WCUWrPpufjlQOl8Ka/mwUGj2h7lFY/X+vPEnPl4t/lAn72o42d rkrThyE3rLGC/WmsUXTTEtFVU0BpNvX2J+BXnfjReHHxK4pQjBgHJU4z+JJrR3siG3/Q blNBAhnKJn6jtlSzJUYkaqHVGYrgwx4aQNtW+wNXDXnLhwYm6B1wWZWNocm52Iy+L8DJ B7ug== X-Gm-Message-State: AOJu0YwEJSI4LYPUoTsBEZH9h6DBrHNKr6FHsdfYPL2l/bIKKMxIEzGn axRnotzKuoFD2/12Tuoj8r9zqSdB64OauDJIwqdXcyp14QC678RzRJOuWO0Fds6oCMDZ2GkTuHo R X-Gm-Gg: ASbGncvlTiSLoVQicyjq1kol2IxFVCaSQKj0oE8jUN7nZPIx+ugkrlUqOLzXMjQFJq6 cdtgopYCMIrkKaK4LJfv70XNuX+Bq+IKajzdSfZqrA/HlM9Kz2XJRTTlOjE/yAuyBlkURj6XUky 9DJqOc2Nt2EL7AMCrn0vmRamePE3xjkS/YM+3arz9Prki7KRM8cfgNR6JQHeEk8Sids7WQNyNi2 BVQQusXO8NEMWZfc5K/xnDD1sTs0FYweZ1OYioc4I4Bq7aAr6sWNpQeLBmG/84QWhFl8RfEM4Ak SDXCruRT83M3k6ht7pFA2JVB9zsNCEtsOg== X-Google-Smtp-Source: AGHT+IFu1SV9guJxTXigzqyG30uqt66XXIEQCyhI2XzLvVdfQH0p1n7la18+cbCb0ALeE6lPeWEzMA== X-Received: by 2002:a5d:6d01:0:b0:38d:b998:a600 with SMTP id ffacd0b85a97d-38dc8dc3071mr2972888f8f.7.1738953483988; Fri, 07 Feb 2025 10:38:03 -0800 (PST) To: devel@lists.libvirt.org Subject: [PATCH V3 14/19] qemu: Support O_DIRECT with mapped-ram on save Date: Fri, 7 Feb 2025 11:27:25 -0700 Message-ID: <20250207183730.21686-15-jfehlig@suse.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250207183730.21686-1-jfehlig@suse.com> References: <20250207183730.21686-1-jfehlig@suse.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Message-ID-Hash: POZNZI4FVBIUSTO7ZJYHFU5W2NJHKSMK X-Message-ID-Hash: POZNZI4FVBIUSTO7ZJYHFU5W2NJHKSMK X-MailFrom: jfehlig@suse.com X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation; header-match-config-1; header-match-config-2; header-match-config-3; header-match-devel.lists.libvirt.org-0; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; suspicious-header CC: farosas@suse.de X-Mailman-Version: 3.2.2 Precedence: list List-Id: Development discussions about the libvirt library & tools Archived-At: List-Archive: List-Help: List-Post: List-Subscribe: List-Unsubscribe: From: Jim Fehlig via Devel Reply-To: Jim Fehlig X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZM-MESSAGEID: 1738953891229019000 Content-Type: text/plain; charset="utf-8" When using the mapped-ram migration capability, direct IO is enabled by setting the "direct-io" migration parameter to "true" and passing QEMU an additional fd with O_DIRECT set. Signed-off-by: Jim Fehlig --- src/qemu/qemu_driver.c | 10 ++++++---- src/qemu/qemu_migration.c | 32 ++++++++++++++++++++++++++------ src/qemu/qemu_migration.h | 1 + src/qemu/qemu_migration_params.c | 11 ++++++++++- src/qemu/qemu_migration_params.h | 3 ++- src/qemu/qemu_monitor.c | 7 +++++-- src/qemu/qemu_monitor.h | 3 ++- src/qemu/qemu_saveimage.c | 2 +- 8 files changed, 53 insertions(+), 16 deletions(-) diff --git a/src/qemu/qemu_driver.c b/src/qemu/qemu_driver.c index 0f363849ba..ad15689eb8 100644 --- a/src/qemu/qemu_driver.c +++ b/src/qemu/qemu_driver.c @@ -2700,7 +2700,8 @@ qemuDomainSaveInternal(virQEMUDriver *driver, goto endjob; xml =3D NULL; =20 - if (!(saveParams =3D qemuMigrationParamsForSave(format =3D=3D QEMU_SAV= E_FORMAT_SPARSE))) + if (!(saveParams =3D qemuMigrationParamsForSave(format =3D=3D QEMU_SAV= E_FORMAT_SPARSE, + flags))) goto endjob; =20 ret =3D qemuSaveImageCreate(driver, vm, path, data, compressor, @@ -3152,7 +3153,7 @@ doCoreDump(virQEMUDriver *driver, if (!(dump_params =3D qemuMigrationParamsNew())) goto cleanup; =20 - if (qemuMigrationSrcToFile(driver, vm, &fd, compressor, + if (qemuMigrationSrcToFile(driver, vm, path, &fd, compressor, dump_params, dump_flags, VIR_ASYNC_JOB_= DUMP) < 0) goto cleanup; } @@ -5798,7 +5799,7 @@ qemuDomainRestoreInternal(virConnectPtr conn, goto cleanup; =20 sparse =3D data->header.format =3D=3D QEMU_SAVE_FORMAT_SPARSE; - if (!(restoreParams =3D qemuMigrationParamsForSave(sparse))) + if (!(restoreParams =3D qemuMigrationParamsForSave(sparse, flags))) goto cleanup; =20 fd =3D qemuSaveImageOpen(driver, path, @@ -6132,7 +6133,8 @@ qemuDomainObjRestore(virConnectPtr conn, } =20 sparse =3D data->header.format =3D=3D QEMU_SAVE_FORMAT_SPARSE; - if (!(restoreParams =3D qemuMigrationParamsForSave(sparse))) + if (!(restoreParams =3D qemuMigrationParamsForSave(sparse, + bypass_cache ? VIR_DO= MAIN_SAVE_BYPASS_CACHE : 0))) return -1; =20 fd =3D qemuSaveImageOpen(driver, path, bypass_cache, sparse, &wrapperF= d, false); diff --git a/src/qemu/qemu_migration.c b/src/qemu/qemu_migration.c index 9665b523af..a5a80230b2 100644 --- a/src/qemu/qemu_migration.c +++ b/src/qemu/qemu_migration.c @@ -7139,17 +7139,36 @@ qemuMigrationSrcToLegacyFile(virQEMUDriver *driver, static int qemuMigrationSrcToSparseFile(virQEMUDriver *driver, virDomainObj *vm, + const char *path, int *fd, unsigned int flags, virDomainAsyncJob asyncJob) { + g_autoptr(virQEMUDriverConfig) cfg =3D virQEMUDriverGetConfig(driver); + VIR_AUTOCLOSE directFd =3D -1; + int directFlag =3D 0; + bool needUnlink =3D false; int ret; =20 - /* mapped-ram does not support directIO */ + /* When using directio with mapped-ram, qemu needs two fds. One with + * O_DIRECT set writing the memory, and another without it set for + * writing small bits of unaligned state. */ if ((flags & VIR_DOMAIN_SAVE_BYPASS_CACHE)) { - virReportError(VIR_ERR_OPERATION_FAILED, "%s", - _("bypass cache unsupported by this system")); - return -1; + directFlag =3D virFileDirectFdFlag(); + if (directFlag < 0) { + virReportError(VIR_ERR_OPERATION_FAILED, "%s", + _("bypass cache unsupported by this system")); + return -1; + } + directFd =3D virQEMUFileOpenAs(cfg->user, cfg->group, false, path, + O_WRONLY | directFlag, &needUnlink); + + if (directFd < 0) + return -1; + + if (qemuSecuritySetImageFDLabel(driver->securityManager, vm->def, = directFd) < 0) + return -1; + } =20 if (qemuSecuritySetImageFDLabel(driver->securityManager, vm->def, *fd)= < 0) @@ -7158,7 +7177,7 @@ qemuMigrationSrcToSparseFile(virQEMUDriver *driver, if (qemuDomainObjEnterMonitorAsync(vm, asyncJob) < 0) return -1; =20 - ret =3D qemuMonitorMigrateToFdSet(vm, 0, fd); + ret =3D qemuMonitorMigrateToFdSet(vm, 0, fd, &directFd); qemuDomainObjExitMonitor(vm); return ret; } @@ -7167,6 +7186,7 @@ qemuMigrationSrcToSparseFile(virQEMUDriver *driver, /* Helper function called while vm is active. */ int qemuMigrationSrcToFile(virQEMUDriver *driver, virDomainObj *vm, + const char *path, int *fd, virCommand *compressor, qemuMigrationParams *migParams, @@ -7204,7 +7224,7 @@ qemuMigrationSrcToFile(virQEMUDriver *driver, virDoma= inObj *vm, =20 if (migParams && qemuMigrationParamsCapEnabled(migParams, QEMU_MIGRATION_CAP_MAPPED= _RAM)) - rc =3D qemuMigrationSrcToSparseFile(driver, vm, fd, flags, asyncJo= b); + rc =3D qemuMigrationSrcToSparseFile(driver, vm, path, fd, flags, a= syncJob); else rc =3D qemuMigrationSrcToLegacyFile(driver, vm, *fd, compressor, a= syncJob); =20 diff --git a/src/qemu/qemu_migration.h b/src/qemu/qemu_migration.h index 71a9974753..beb888160a 100644 --- a/src/qemu/qemu_migration.h +++ b/src/qemu/qemu_migration.h @@ -238,6 +238,7 @@ qemuMigrationSrcIsAllowed(virDomainObj *vm, int qemuMigrationSrcToFile(virQEMUDriver *driver, virDomainObj *vm, + const char *path, int *fd, virCommand *compressor, qemuMigrationParams *migParams, diff --git a/src/qemu/qemu_migration_params.c b/src/qemu/qemu_migration_par= ams.c index d48cdd5506..16e93f27d6 100644 --- a/src/qemu/qemu_migration_params.c +++ b/src/qemu/qemu_migration_params.c @@ -130,6 +130,7 @@ VIR_ENUM_IMPL(qemuMigrationParam, "multifd-zlib-level", "multifd-zstd-level", "avail-switchover-bandwidth", + "direct-io", ); =20 typedef struct _qemuMigrationParamsAlwaysOnItem qemuMigrationParamsAlwaysO= nItem; @@ -328,6 +329,9 @@ static const qemuMigrationParamInfoItem qemuMigrationPa= ramInfo[] =3D { [QEMU_MIGRATION_PARAM_AVAIL_SWITCHOVER_BANDWIDTH] =3D { .type =3D QEMU_MIGRATION_PARAM_TYPE_ULL, }, + [QEMU_MIGRATION_PARAM_DIRECT_IO] =3D { + .type =3D QEMU_MIGRATION_PARAM_TYPE_BOOL, + }, }; G_STATIC_ASSERT(G_N_ELEMENTS(qemuMigrationParamInfo) =3D=3D QEMU_MIGRATION= _PARAM_LAST); =20 @@ -793,7 +797,7 @@ qemuMigrationParamsFromFlags(virTypedParameterPtr param= s, =20 =20 qemuMigrationParams * -qemuMigrationParamsForSave(bool sparse) +qemuMigrationParamsForSave(bool sparse, unsigned int flags) { g_autoptr(qemuMigrationParams) saveParams =3D NULL; =20 @@ -807,6 +811,11 @@ qemuMigrationParamsForSave(bool sparse) return NULL; saveParams->params[QEMU_MIGRATION_PARAM_MULTIFD_CHANNELS].value.i = =3D 1; saveParams->params[QEMU_MIGRATION_PARAM_MULTIFD_CHANNELS].set =3D = true; + + if (flags & VIR_DOMAIN_SAVE_BYPASS_CACHE) { + saveParams->params[QEMU_MIGRATION_PARAM_DIRECT_IO].value.b =3D= true; + saveParams->params[QEMU_MIGRATION_PARAM_DIRECT_IO].set =3D tru= e; + } } =20 return g_steal_pointer(&saveParams); diff --git a/src/qemu/qemu_migration_params.h b/src/qemu/qemu_migration_par= ams.h index 88a1bc1a66..87822332d3 100644 --- a/src/qemu/qemu_migration_params.h +++ b/src/qemu/qemu_migration_params.h @@ -66,6 +66,7 @@ typedef enum { QEMU_MIGRATION_PARAM_MULTIFD_ZLIB_LEVEL, QEMU_MIGRATION_PARAM_MULTIFD_ZSTD_LEVEL, QEMU_MIGRATION_PARAM_AVAIL_SWITCHOVER_BANDWIDTH, + QEMU_MIGRATION_PARAM_DIRECT_IO, =20 QEMU_MIGRATION_PARAM_LAST } qemuMigrationParam; @@ -88,7 +89,7 @@ qemuMigrationParamsFromFlags(virTypedParameterPtr params, qemuMigrationParty party); =20 qemuMigrationParams * -qemuMigrationParamsForSave(bool sparse); +qemuMigrationParamsForSave(bool sparse, unsigned int flags); =20 int qemuMigrationParamsDump(qemuMigrationParams *migParams, diff --git a/src/qemu/qemu_monitor.c b/src/qemu/qemu_monitor.c index e2043c0120..1e80af8a58 100644 --- a/src/qemu/qemu_monitor.c +++ b/src/qemu/qemu_monitor.c @@ -2235,7 +2235,8 @@ qemuMonitorMigrateToFd(qemuMonitor *mon, int qemuMonitorMigrateToFdSet(virDomainObj *vm, unsigned int flags, - int *fd) + int *fd, + int *directFd) { qemuDomainObjPrivate *priv =3D vm->privateData; qemuMonitor *mon =3D priv->mon; @@ -2244,7 +2245,7 @@ qemuMonitorMigrateToFdSet(virDomainObj *vm, g_autofree char *uri =3D NULL; int ret; =20 - VIR_DEBUG("fd=3D%d flags=3D0x%x", *fd, flags); + VIR_DEBUG("fd=3D%d directFd=3D%d flags=3D0x%x", *fd, *directFd, flags); =20 QEMU_CHECK_MONITOR(mon); =20 @@ -2256,6 +2257,8 @@ qemuMonitorMigrateToFdSet(virDomainObj *vm, =20 fdPassMigrate =3D qemuFDPassNew("libvirt-outgoing-migrate", priv); qemuFDPassAddFD(fdPassMigrate, fd, "-fd"); + if (*directFd !=3D -1) + qemuFDPassAddFD(fdPassMigrate, directFd, "-directio-fd"); qemuFDPassTransferMonitor(fdPassMigrate, mon); =20 uri =3D g_strdup_printf("file:%s,offset=3D%#lx", diff --git a/src/qemu/qemu_monitor.h b/src/qemu/qemu_monitor.h index 6da380aa65..dfb0e8abb6 100644 --- a/src/qemu/qemu_monitor.h +++ b/src/qemu/qemu_monitor.h @@ -861,7 +861,8 @@ int qemuMonitorMigrateToFd(qemuMonitor *mon, =20 int qemuMonitorMigrateToFdSet(virDomainObj *vm, unsigned int flags, - int *fd); + int *fd, + int *directFd); =20 int qemuMonitorMigrateToHost(qemuMonitor *mon, unsigned int flags, diff --git a/src/qemu/qemu_saveimage.c b/src/qemu/qemu_saveimage.c index 3f20ffd58c..28d668231e 100644 --- a/src/qemu/qemu_saveimage.c +++ b/src/qemu/qemu_saveimage.c @@ -501,7 +501,7 @@ qemuSaveImageCreate(virQEMUDriver *driver, goto cleanup; =20 /* Perform the migration */ - if (qemuMigrationSrcToFile(driver, vm, &fd, compressor, saveParams, fl= ags, asyncJob) < 0) + if (qemuMigrationSrcToFile(driver, vm, path, &fd, compressor, savePara= ms, flags, asyncJob) < 0) goto cleanup; =20 /* Touch up file header to mark image complete. */ --=20 2.43.0