From nobody Tue Sep 9 23:57:24 2025 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.libvirt.org designates 8.43.85.245 as permitted sender) client-ip=8.43.85.245; envelope-from=devel-bounces@lists.libvirt.org; helo=lists.libvirt.org; Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of lists.libvirt.org designates 8.43.85.245 as permitted sender) smtp.mailfrom=devel-bounces@lists.libvirt.org; dmarc=pass(p=reject dis=none) header.from=lists.libvirt.org ARC-Seal: i=1; a=rsa-sha256; t=1737588588; cv=none; d=zohomail.com; s=zohoarc; b=j308rP6ExBgZJshEVvQvvgWon/BgVLUC9VRagT60k2veEOxuGb12pLCBjw4h8S9URuULd8vkn4YoclpqmURqmr19NR3R/NIRuUeV/fFvLN9l9AkUu6JoopDhl+BfX5YS5LZ3FThRgk3NKk6Gpz7FRXHO+iLjfWVaIRUQ2TY5rDE= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1737588588; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:Reply-To:Reply-To:References:Subject:Subject:To:To:Message-Id; bh=GOx3qds9aMDz1LEWF2zyw/F6BRdDgq3FqDAWdgCWKYk=; b=ccu1K5sWEkKroOLJXi6IkOb0apimLxKZhCmbnualRPXpewevl/XyLNCQ8dW8zNye3x3Tm1Z0wAi17FDwYVePVFR3X+h7p/vttOkIwEd8XAGQpHYvC/ByHAvaDj1MxAvzLRGXgid4JR6tXlIJj8v9ZzCwhoNeMcuVRJQUdw7awqk= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of lists.libvirt.org designates 8.43.85.245 as permitted sender) smtp.mailfrom=devel-bounces@lists.libvirt.org; dmarc=pass header.from= (p=reject dis=none) Return-Path: Received: from lists.libvirt.org (lists.libvirt.org [8.43.85.245]) by mx.zohomail.com with SMTPS id 1737588588889305.1868199124033; Wed, 22 Jan 2025 15:29:48 -0800 (PST) Received: by lists.libvirt.org (Postfix, from userid 996) id 68637145A; Wed, 22 Jan 2025 18:29:48 -0500 (EST) Received: from lists.libvirt.org (localhost [IPv6:::1]) by lists.libvirt.org (Postfix) with ESMTP id EDD5413F4; Wed, 22 Jan 2025 18:23:26 -0500 (EST) Received: by lists.libvirt.org (Postfix, from userid 996) id B5E2213B8; Wed, 22 Jan 2025 18:23:18 -0500 (EST) Received: from mail-ej1-f51.google.com (mail-ej1-f51.google.com [209.85.218.51]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by lists.libvirt.org (Postfix) with ESMTPS id 61947130E for ; Wed, 22 Jan 2025 18:22:59 -0500 (EST) Received: by mail-ej1-f51.google.com with SMTP id a640c23a62f3a-ab651f1dd36so71800666b.0 for ; Wed, 22 Jan 2025 15:22:59 -0800 (PST) Received: from localhost (75-169-8-111.slkc.qwest.net. [75.169.8.111]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-72daba5752esm11891368b3a.146.2025.01.22.15.22.57 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 22 Jan 2025 15:22:57 -0800 (PST) X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on lists.libvirt.org X-Spam-Level: X-Spam-Status: No, score=-0.8 required=5.0 tests=DKIM_INVALID,DKIM_SIGNED, MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE,RCVD_IN_MSPIKE_H3, RCVD_IN_MSPIKE_WL,RCVD_IN_VALIDITY_RPBL_BLOCKED, RCVD_IN_VALIDITY_SAFE_BLOCKED,SPF_HELO_NONE autolearn=unavailable autolearn_force=no version=3.4.4 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=google; t=1737588178; x=1738192978; darn=lists.libvirt.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=ROzJAlJ0GbgBrcoqcAhd8J7r8lDIlxwZOqxUop8NDa8=; b=B4p0UMo4T8iT5gPQSDi6YCUg5r3T25e6bsXsdE+NAGRtReENeJhUz0iUa6B9LTY6JH Pmt0o8WiALCZ6kFfwQzM88tbm59JPKBXZ3SoiWHtNscCZtN6+D0moH/3QQY8uR50Qa2d IoWPCYrCYm1zqMrnhAd+vYzD884vPA3zfDyRM0epdoG0fFsLYGkq1emGAy+OCdcNUfyf 1q0I1kPPtYa9GqrMWH/uOwi+wv+lknUY6tfboiMK74LqSQZfvzItaFxHCQkT1CPBGvF8 Re8AhPe5D8b/pEkxCLriN/yYZ76UnMc6SFEBUREuaobyUgVq2lOECf4eCTxRWveZ66Do U3NQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1737588178; x=1738192978; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=ROzJAlJ0GbgBrcoqcAhd8J7r8lDIlxwZOqxUop8NDa8=; b=B1SXIfNlWKqhqxq02VLcRAT7GFzIaUraqWxxfFNLt5DIJ50a1toeE+X/tasMuOJl/W TCcz3sE3Yacz4AuAfYcGwIoFBh+DnJqFNcO/54sZZvjIIa2Ml8CTpU7zzEKEEf+JmyQE HOhf8ZsPjFSIvwA3KaCCl5TWrj+JMghfSlXA03O0AqXUy5EOIUmuB/BoLdrWtW1S4hM3 wUtABYHZtpO3J5i1AIhMyZaYzpEgSE0Cx5QnvMPMXjG+nEOniRqrUoXAjiN412XmSbcp 8S97j2AQ8/5jMTr+fBY1xYg0Acgq8Ob78E0oP6o8stufP+Pj5anQx01Y2AByEV37Fqh4 cJDQ== X-Gm-Message-State: AOJu0YwyK8LEwCBKk/exaJZ8WumQkgpHVrGFMGl/wbnhieXObmXW+AJW RWFd/hI5x6E8pOHFclg8reHV1FCTzOIndu9uKwTK8UNuivHoV5gCeAhPr2ffi1WWy87yl80dBn6 O X-Gm-Gg: ASbGncuEzNIA3o1BaQ19bmEHrnVPCvbJZUZLhk4UJT5CjU6O6WBMZ2YYu8prpRJTSBv eKWUAH99/B9pSja9vtylm2dBvtST1WK6Hq6qJJR6zmGaAEkmcZ3g/wd3rlZ/d8qdvRxK91LQ2kr FOQeV9qWh3P89sqpX+fTXypuV/fGpZnLM+ZmWJQKiFt9CTs2wgG28x0syZ+M5utDPMmekSHuZGn SBg9SOgRbKGW35sVS2nHDzu2dKGJDXg94H7zrMMGfCKIU4lsPyWIb/qSgbGIeBpo4NDmccl+fQp SVb6JTV20qKh X-Google-Smtp-Source: AGHT+IFLmcg/Gl4wK5e47NT7yTuQa1H+ZYgd1yGVx1y6wazd7/bS2VsJJGlhxx3QVmkCtvdL8taxcA== X-Received: by 2002:a17:906:f58e:b0:aa6:8e9e:1b5 with SMTP id a640c23a62f3a-ab38b1b4598mr2021917066b.3.1737588178087; Wed, 22 Jan 2025 15:22:58 -0800 (PST) To: devel@lists.libvirt.org Subject: [PATCH V2 15/20] qemu: Support O_DIRECT with mapped-ram on save Date: Wed, 22 Jan 2025 16:16:49 -0700 Message-ID: <20250122232228.19306-16-jfehlig@suse.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250122232228.19306-1-jfehlig@suse.com> References: <20250122232228.19306-1-jfehlig@suse.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Message-ID-Hash: 3VRY5XWWXXFD5OBGGJTWJ2P5I632F2GI X-Message-ID-Hash: 3VRY5XWWXXFD5OBGGJTWJ2P5I632F2GI X-MailFrom: jfehlig@suse.com X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation; header-match-config-1; header-match-config-2; header-match-config-3; header-match-devel.lists.libvirt.org-0; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; suspicious-header CC: farosas@suse.de X-Mailman-Version: 3.2.2 Precedence: list List-Id: Development discussions about the libvirt library & tools Archived-At: List-Archive: List-Help: List-Post: List-Subscribe: List-Unsubscribe: From: Jim Fehlig via Devel Reply-To: Jim Fehlig X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZM-MESSAGEID: 1737588589863019000 Content-Type: text/plain; charset="utf-8" When using the mapped-ram migration capability, direct IO is enabled by setting the "direct-io" migration parameter to "true" and passing QEMU an additional fd with O_DIRECT set. Signed-off-by: Jim Fehlig --- src/qemu/qemu_driver.c | 10 ++++++---- src/qemu/qemu_migration.c | 32 ++++++++++++++++++++++++++------ src/qemu/qemu_migration.h | 1 + src/qemu/qemu_migration_params.c | 11 ++++++++++- src/qemu/qemu_migration_params.h | 3 ++- src/qemu/qemu_monitor.c | 7 +++++-- src/qemu/qemu_monitor.h | 3 ++- src/qemu/qemu_saveimage.c | 2 +- 8 files changed, 53 insertions(+), 16 deletions(-) diff --git a/src/qemu/qemu_driver.c b/src/qemu/qemu_driver.c index ad58ec92f1..803c7be3f7 100644 --- a/src/qemu/qemu_driver.c +++ b/src/qemu/qemu_driver.c @@ -2700,7 +2700,8 @@ qemuDomainSaveInternal(virQEMUDriver *driver, goto endjob; xml =3D NULL; =20 - if (!(saveParams =3D qemuMigrationParamsForSave(format =3D=3D QEMU_SAV= E_FORMAT_SPARSE))) + if (!(saveParams =3D qemuMigrationParamsForSave(format =3D=3D QEMU_SAV= E_FORMAT_SPARSE, + flags))) goto endjob; =20 ret =3D qemuSaveImageCreate(driver, vm, path, data, compressor, @@ -3152,7 +3153,7 @@ doCoreDump(virQEMUDriver *driver, if (!(dump_params =3D qemuMigrationParamsNew())) goto cleanup; =20 - if (qemuMigrationSrcToFile(driver, vm, &fd, compressor, + if (qemuMigrationSrcToFile(driver, vm, path, &fd, compressor, dump_params, dump_flags, VIR_ASYNC_JOB_= DUMP) < 0) goto cleanup; } @@ -5798,7 +5799,7 @@ qemuDomainRestoreInternal(virConnectPtr conn, goto cleanup; =20 sparse =3D data->header.format =3D=3D QEMU_SAVE_FORMAT_SPARSE; - if (!(restoreParams =3D qemuMigrationParamsForSave(sparse))) + if (!(restoreParams =3D qemuMigrationParamsForSave(sparse, flags))) goto cleanup; =20 fd =3D qemuSaveImageOpen(driver, path, @@ -6124,7 +6125,8 @@ qemuDomainObjRestore(virConnectPtr conn, } =20 sparse =3D data->header.format =3D=3D QEMU_SAVE_FORMAT_SPARSE; - if (!(restoreParams =3D qemuMigrationParamsForSave(sparse))) + if (!(restoreParams =3D qemuMigrationParamsForSave(sparse, + bypass_cache ? VIR_DO= MAIN_SAVE_BYPASS_CACHE : 0))) return -1; =20 fd =3D qemuSaveImageOpen(driver, path, bypass_cache, sparse, &wrapperF= d, false); diff --git a/src/qemu/qemu_migration.c b/src/qemu/qemu_migration.c index 8af6d1d3e2..61a14dc9d6 100644 --- a/src/qemu/qemu_migration.c +++ b/src/qemu/qemu_migration.c @@ -7135,17 +7135,36 @@ qemuMigrationSrcToLegacyFile(virQEMUDriver *driver, static int qemuMigrationSrcToMappedFile(virQEMUDriver *driver, virDomainObj *vm, + const char *path, int *fd, unsigned int flags, virDomainAsyncJob asyncJob) { + g_autoptr(virQEMUDriverConfig) cfg =3D virQEMUDriverGetConfig(driver); + VIR_AUTOCLOSE directFd =3D -1; + int directFlag =3D 0; + bool needUnlink =3D false; int ret; =20 - /* mapped-ram does not support directIO */ + /* When using directio with mapped-ram, qemu needs two fds. One with + * O_DIRECT set writing the memory, and another without it set for + * writing small bits of unaligned state. */ if ((flags & VIR_DOMAIN_SAVE_BYPASS_CACHE)) { - virReportError(VIR_ERR_OPERATION_FAILED, "%s", - _("bypass cache unsupported by this system")); - return -1; + directFlag =3D virFileDirectFdFlag(); + if (directFlag < 0) { + virReportError(VIR_ERR_OPERATION_FAILED, "%s", + _("bypass cache unsupported by this system")); + return -1; + } + directFd =3D virQEMUFileOpenAs(cfg->user, cfg->group, false, path, + O_WRONLY | directFlag, &needUnlink); + + if (directFd < 0) + return -1; + + if (qemuSecuritySetImageFDLabel(driver->securityManager, vm->def, = directFd) < 0) + return -1; + } =20 if (qemuSecuritySetImageFDLabel(driver->securityManager, vm->def, *fd)= < 0) @@ -7154,7 +7173,7 @@ qemuMigrationSrcToMappedFile(virQEMUDriver *driver, if (qemuDomainObjEnterMonitorAsync(vm, asyncJob) < 0) return -1; =20 - ret =3D qemuMonitorMigrateToFdSet(vm, 0, fd); + ret =3D qemuMonitorMigrateToFdSet(vm, 0, fd, &directFd); qemuDomainObjExitMonitor(vm); return ret; } @@ -7163,6 +7182,7 @@ qemuMigrationSrcToMappedFile(virQEMUDriver *driver, /* Helper function called while vm is active. */ int qemuMigrationSrcToFile(virQEMUDriver *driver, virDomainObj *vm, + const char *path, int *fd, virCommand *compressor, qemuMigrationParams *migParams, @@ -7200,7 +7220,7 @@ qemuMigrationSrcToFile(virQEMUDriver *driver, virDoma= inObj *vm, =20 if (migParams && qemuMigrationParamsCapEnabled(migParams, QEMU_MIGRATION_CAP_MAPPED= _RAM)) { - rc =3D qemuMigrationSrcToMappedFile(driver, vm, fd, flags, asyncJo= b); + rc =3D qemuMigrationSrcToMappedFile(driver, vm, path, fd, flags, a= syncJob); } else { rc =3D qemuMigrationSrcToLegacyFile(driver, vm, *fd, compressor, a= syncJob); } diff --git a/src/qemu/qemu_migration.h b/src/qemu/qemu_migration.h index b1f894e501..13ce9d33f2 100644 --- a/src/qemu/qemu_migration.h +++ b/src/qemu/qemu_migration.h @@ -237,6 +237,7 @@ qemuMigrationSrcIsAllowed(virDomainObj *vm, int qemuMigrationSrcToFile(virQEMUDriver *driver, virDomainObj *vm, + const char *path, int *fd, virCommand *compressor, qemuMigrationParams *migParams, diff --git a/src/qemu/qemu_migration_params.c b/src/qemu/qemu_migration_par= ams.c index 0faeb6b0c1..79b47333f9 100644 --- a/src/qemu/qemu_migration_params.c +++ b/src/qemu/qemu_migration_params.c @@ -129,6 +129,7 @@ VIR_ENUM_IMPL(qemuMigrationParam, "multifd-compression", "multifd-zlib-level", "multifd-zstd-level", + "direct-io", ); =20 typedef struct _qemuMigrationParamsAlwaysOnItem qemuMigrationParamsAlwaysO= nItem; @@ -319,6 +320,9 @@ static const qemuMigrationParamInfoItem qemuMigrationPa= ramInfo[] =3D { [QEMU_MIGRATION_PARAM_MULTIFD_ZSTD_LEVEL] =3D { .type =3D QEMU_MIGRATION_PARAM_TYPE_INT, }, + [QEMU_MIGRATION_PARAM_DIRECT_IO] =3D { + .type =3D QEMU_MIGRATION_PARAM_TYPE_BOOL, + }, }; G_STATIC_ASSERT(G_N_ELEMENTS(qemuMigrationParamInfo) =3D=3D QEMU_MIGRATION= _PARAM_LAST); =20 @@ -784,7 +788,7 @@ qemuMigrationParamsFromFlags(virTypedParameterPtr param= s, =20 =20 qemuMigrationParams * -qemuMigrationParamsForSave(bool sparse) +qemuMigrationParamsForSave(bool sparse, unsigned int flags) { g_autoptr(qemuMigrationParams) saveParams =3D NULL; =20 @@ -798,6 +802,11 @@ qemuMigrationParamsForSave(bool sparse) return NULL; saveParams->params[QEMU_MIGRATION_PARAM_MULTIFD_CHANNELS].value.i = =3D 1; saveParams->params[QEMU_MIGRATION_PARAM_MULTIFD_CHANNELS].set =3D = true; + + if (flags & VIR_DOMAIN_SAVE_BYPASS_CACHE) { + saveParams->params[QEMU_MIGRATION_PARAM_DIRECT_IO].value.b =3D= true; + saveParams->params[QEMU_MIGRATION_PARAM_DIRECT_IO].set =3D tru= e; + } } =20 return g_steal_pointer(&saveParams); diff --git a/src/qemu/qemu_migration_params.h b/src/qemu/qemu_migration_par= ams.h index bb24af4ef9..b326aa5bc1 100644 --- a/src/qemu/qemu_migration_params.h +++ b/src/qemu/qemu_migration_params.h @@ -65,6 +65,7 @@ typedef enum { QEMU_MIGRATION_PARAM_MULTIFD_COMPRESSION, QEMU_MIGRATION_PARAM_MULTIFD_ZLIB_LEVEL, QEMU_MIGRATION_PARAM_MULTIFD_ZSTD_LEVEL, + QEMU_MIGRATION_PARAM_DIRECT_IO, =20 QEMU_MIGRATION_PARAM_LAST } qemuMigrationParam; @@ -87,7 +88,7 @@ qemuMigrationParamsFromFlags(virTypedParameterPtr params, qemuMigrationParty party); =20 qemuMigrationParams * -qemuMigrationParamsForSave(bool sparse); +qemuMigrationParamsForSave(bool sparse, unsigned int flags); =20 int qemuMigrationParamsDump(qemuMigrationParams *migParams, diff --git a/src/qemu/qemu_monitor.c b/src/qemu/qemu_monitor.c index d88e1b1ecd..385d103ad3 100644 --- a/src/qemu/qemu_monitor.c +++ b/src/qemu/qemu_monitor.c @@ -2233,7 +2233,8 @@ qemuMonitorMigrateToFd(qemuMonitor *mon, int qemuMonitorMigrateToFdSet(virDomainObj *vm, unsigned int flags, - int *fd) + int *fd, + int *directFd) { qemuDomainObjPrivate *priv =3D vm->privateData; qemuMonitor *mon =3D priv->mon; @@ -2243,7 +2244,7 @@ qemuMonitorMigrateToFdSet(virDomainObj *vm, g_autofree char *uri =3D NULL; int ret; =20 - VIR_DEBUG("fd=3D%d flags=3D0x%x", *fd, flags); + VIR_DEBUG("fd=3D%d directFd=3D%d flags=3D0x%x", *fd, *directFd, flags); =20 QEMU_CHECK_MONITOR(mon); =20 @@ -2255,6 +2256,8 @@ qemuMonitorMigrateToFdSet(virDomainObj *vm, =20 fdPassMigrate =3D qemuFDPassNew("migrate", priv); qemuFDPassAddFD(fdPassMigrate, fd, "-fd"); + if (*directFd !=3D -1) + qemuFDPassAddFD(fdPassMigrate, directFd, "-directio-fd"); qemuFDPassTransferMonitor(fdPassMigrate, mon); =20 if (qemuFDPassGetId(fdPassMigrate, &setId) < 0) diff --git a/src/qemu/qemu_monitor.h b/src/qemu/qemu_monitor.h index 7f54203fe3..79dbe5fcd7 100644 --- a/src/qemu/qemu_monitor.h +++ b/src/qemu/qemu_monitor.h @@ -857,7 +857,8 @@ int qemuMonitorMigrateToFd(qemuMonitor *mon, =20 int qemuMonitorMigrateToFdSet(virDomainObj *vm, unsigned int flags, - int *fd); + int *fd, + int *directFd); =20 int qemuMonitorMigrateToHost(qemuMonitor *mon, unsigned int flags, diff --git a/src/qemu/qemu_saveimage.c b/src/qemu/qemu_saveimage.c index 3e89f88301..d417762749 100644 --- a/src/qemu/qemu_saveimage.c +++ b/src/qemu/qemu_saveimage.c @@ -417,7 +417,7 @@ qemuSaveImageCreate(virQEMUDriver *driver, goto cleanup; =20 /* Perform the migration */ - if (qemuMigrationSrcToFile(driver, vm, &fd, compressor, saveParams, fl= ags, asyncJob) < 0) + if (qemuMigrationSrcToFile(driver, vm, path, &fd, compressor, savePara= ms, flags, asyncJob) < 0) goto cleanup; =20 /* Touch up file header to mark image complete. */ --=20 2.43.0