From nobody Sat Nov 23 16:50:13 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.libvirt.org designates 8.43.85.245 as permitted sender) client-ip=8.43.85.245; envelope-from=devel-bounces@lists.libvirt.org; helo=lists.libvirt.org; Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of lists.libvirt.org designates 8.43.85.245 as permitted sender) smtp.mailfrom=devel-bounces@lists.libvirt.org; dmarc=pass(p=reject dis=none) header.from=lists.libvirt.org ARC-Seal: i=1; a=rsa-sha256; t=1723160819; cv=none; d=zohomail.com; s=zohoarc; b=WAbP87KXpZ5ewz+TX3n85+8lAPyLHVitrzgqahQLAnP7gzZS/RxJR/LcRmYk/F/A3wYNjWVP2qUdN6uZv/tEhT0iOfwX2Sgv/PLiPUngmc8ZKrlemsFTLqeW/JRKO9FlMalF5q7s11Sr4z97QjrQmMY4A7ypZ+AnCiGFxd9vbcc= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1723160819; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:Reply-To:Reply-To:References:Subject:Subject:To:To:Message-Id; bh=GeEzbKA6On+WYVAmq0Poun0z2eBGHEr8alFmS83bmdc=; b=HI1PByS6XlL7AGEQ48XBiuwfQW/P0gzPn6sCzZlimt0j9YywBrip6XQ8C4CmdIKuqEoheNObLCD3wnJO09LWZsNXH12OqYNsFoyH4ejWn13iME0FAUbq0RHmwX3lH0As9nMaki5BNpxh3hob5PLesU4cLmdZWXrZ/O1j9csqFME= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of lists.libvirt.org designates 8.43.85.245 as permitted sender) smtp.mailfrom=devel-bounces@lists.libvirt.org; dmarc=pass header.from= (p=reject dis=none) Return-Path: Received: from lists.libvirt.org (lists.libvirt.org [8.43.85.245]) by mx.zohomail.com with SMTPS id 1723160819480275.7243510871916; Thu, 8 Aug 2024 16:46:59 -0700 (PDT) Received: by lists.libvirt.org (Postfix, from userid 996) id 72724B65; Thu, 8 Aug 2024 19:46:58 -0400 (EDT) Received: from lists.libvirt.org (localhost [IPv6:::1]) by lists.libvirt.org (Postfix) with ESMTP id 630E21493; Thu, 8 Aug 2024 19:39:20 -0400 (EDT) Received: by lists.libvirt.org (Postfix, from userid 996) id 875201528; Thu, 8 Aug 2024 19:39:13 -0400 (EDT) Received: from mail-lj1-f181.google.com (mail-lj1-f181.google.com [209.85.208.181]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by lists.libvirt.org (Postfix) with ESMTPS id E346914C8 for ; Thu, 8 Aug 2024 19:38:49 -0400 (EDT) Received: by mail-lj1-f181.google.com with SMTP id 38308e7fff4ca-2eeb1ba040aso20130811fa.1 for ; Thu, 08 Aug 2024 16:38:49 -0700 (PDT) Received: from localhost ([69.51.98.127]) by smtp.gmail.com with ESMTPSA id 8926c6da1cb9f-4c8d69a6bbdsm3769049173.38.2024.08.08.16.38.47 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 08 Aug 2024 16:38:47 -0700 (PDT) X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on lists.libvirt.org X-Spam-Level: X-Spam-Status: No, score=-0.8 required=5.0 tests=DKIM_INVALID,DKIM_SIGNED, MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE,RCVD_IN_MSPIKE_H2,SPF_HELO_NONE autolearn=unavailable autolearn_force=no version=3.4.4 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=google; t=1723160328; x=1723765128; darn=lists.libvirt.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=BHxZ9mbDeg3mWrmAkW80BpmFagGfEpKGjmqxXbuJItg=; b=Wgt6TGAn/a9iXyOwLiRYMBRzZ7djcDSXwEx0NI+UKmHM7u3uAc/bRdy9bjiPCBQa0K Hf5UvrErZ6yIH9n9le3MalNSg0EebELWHt/oBdiNBuOtRaN8rj9fwUApoqr5+A7vC8Ii CItRr6GNdbbgyfOmxSpL2r5utasogscOl9ByIoJ7118QNd+2YRPFLvGfmPE58r9kpbjz qDdoat4Bak3KgmwZM0IEiI7iIN0EtnTW3AfV+F1O5KxlRvtT81hlxU5G5D7L87z+qgV4 W4awtKjROoK9FQfAWrfxjBBrKITHLM5u93LPEhfl9pT3vpCxoYkSTsdoRhS0X6uWLzAS BRAA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1723160328; x=1723765128; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=BHxZ9mbDeg3mWrmAkW80BpmFagGfEpKGjmqxXbuJItg=; b=HTex/e1bswiJ9WlXC5wDhW3t6pTZvJ/aHxC19Boo5rzbAkpZqGULqrlKoA3HCUzYBL Bm014HEZeb/fIy8HiL3yANp5YpNlmURNQ0dWZY2rvqmkiaP0wdN/dt85gqRbWWMZBLY0 WvKsAJvYFIsKkKJtWapeXLZSSwwqy8incNqGbqy2J/CzYdy7qPpLmEB5iM6DwVDjkaav G2eNhCNKHl8z4YqV0kkTEt0/KAX2Qjvvd5KaKqinGtQ3J2pX8zSZXMRRNM0MESF8fB2K rgjL7xwYDT+8zbfbNB1bvOLavtcX09WhxDhFK9Oiu7IuwhFxyp6CuVuaDisr0+5FxBcw 3imw== X-Gm-Message-State: AOJu0YzTgJtv3Z4TTF+eR6JBXILmoWEFALuqStyqW6zdgR/t9FGei8WV 8XDp3vQKzFySf3xaT7s/QQHR6cYCCFT6uoxvicLXCeVHvceZ+NgoinM3AyDSyXKEz21IfQfLBQ2 V X-Google-Smtp-Source: AGHT+IEUFf44e2sHxI18oZyeWPWHOg7pkx3HJ4Pxgpwne4mTriHeyQ3iSw+oPTf59Vluti3Hpu9UTA== X-Received: by 2002:a05:651c:1993:b0:2ef:259f:a569 with SMTP id 38308e7fff4ca-2f19ee80b06mr24839791fa.15.1723160328317; Thu, 08 Aug 2024 16:38:48 -0700 (PDT) To: devel@lists.libvirt.org Subject: [PATCH 15/20] qemu: Support O_DIRECT with mapped-ram on save Date: Thu, 8 Aug 2024 17:38:08 -0600 Message-Id: <20240808233813.22905-16-jfehlig@suse.com> X-Mailer: git-send-email 2.35.3 In-Reply-To: <20240808233813.22905-1-jfehlig@suse.com> References: <20240808233813.22905-1-jfehlig@suse.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Message-ID-Hash: NZXES33XMN2TGCI7KMEQ5MV3277CNINN X-Message-ID-Hash: NZXES33XMN2TGCI7KMEQ5MV3277CNINN X-MailFrom: jfehlig@suse.com X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation; header-match-config-1; header-match-config-2; header-match-config-3; header-match-devel.lists.libvirt.org-0; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; suspicious-header CC: farosas@suse.de X-Mailman-Version: 3.2.2 Precedence: list List-Id: Development discussions about the libvirt library & tools Archived-At: List-Archive: List-Help: List-Post: List-Subscribe: List-Unsubscribe: From: Jim Fehlig via Devel Reply-To: Jim Fehlig X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZM-MESSAGEID: 1723160820272116600 Content-Type: text/plain; charset="utf-8" When using the mapped-ram migration capability, direct IO is enabled by setting the "direct-io" migration parameter to "true" and passing QEMU an additional fd with O_DIRECT set. Signed-off-by: Jim Fehlig --- src/qemu/qemu_driver.c | 9 +++++---- src/qemu/qemu_migration.c | 32 ++++++++++++++++++++++++++------ src/qemu/qemu_migration.h | 1 + src/qemu/qemu_migration_params.c | 11 ++++++++++- src/qemu/qemu_migration_params.h | 3 ++- src/qemu/qemu_monitor.c | 7 +++++-- src/qemu/qemu_monitor.h | 3 ++- src/qemu/qemu_saveimage.c | 2 +- 8 files changed, 52 insertions(+), 16 deletions(-) diff --git a/src/qemu/qemu_driver.c b/src/qemu/qemu_driver.c index 6d0f52951c..0025bad6e7 100644 --- a/src/qemu/qemu_driver.c +++ b/src/qemu/qemu_driver.c @@ -2691,7 +2691,7 @@ qemuDomainSaveInternal(virQEMUDriver *driver, xml =3D NULL; =20 mappedRam =3D data->header.features & QEMU_SAVE_FEATURE_MAPPED_RAM; - if (!(saveParams =3D qemuMigrationParamsForSave(mappedRam))) + if (!(saveParams =3D qemuMigrationParamsForSave(mappedRam, flags))) goto endjob; =20 ret =3D qemuSaveImageCreate(driver, vm, path, data, compressor, @@ -3143,7 +3143,7 @@ doCoreDump(virQEMUDriver *driver, if (!(dump_params =3D qemuMigrationParamsNew())) goto cleanup; =20 - if (qemuMigrationSrcToFile(driver, vm, &fd, compressor, + if (qemuMigrationSrcToFile(driver, vm, path, &fd, compressor, dump_params, dump_flags, VIR_ASYNC_JOB_= DUMP) < 0) goto cleanup; } @@ -5789,7 +5789,7 @@ qemuDomainRestoreInternal(virConnectPtr conn, goto cleanup; =20 mapped_ram =3D data->header.features & QEMU_SAVE_FEATURE_MAPPED_RAM; - if (!(restoreParams =3D qemuMigrationParamsForSave(mapped_ram))) + if (!(restoreParams =3D qemuMigrationParamsForSave(mapped_ram, flags))) return -1; =20 fd =3D qemuSaveImageOpen(driver, path, @@ -6115,7 +6115,8 @@ qemuDomainObjRestore(virConnectPtr conn, } =20 mapped_ram =3D data->header.features & QEMU_SAVE_FEATURE_MAPPED_RAM; - if (!(restoreParams =3D qemuMigrationParamsForSave(mapped_ram))) + if (!(restoreParams =3D qemuMigrationParamsForSave(mapped_ram, + bypass_cache ? VIR_DO= MAIN_SAVE_BYPASS_CACHE : 0))) return -1; =20 fd =3D qemuSaveImageOpen(driver, path, bypass_cache, mapped_ram, &wrap= perFd, false); diff --git a/src/qemu/qemu_migration.c b/src/qemu/qemu_migration.c index 35d3e26908..daa42bcfe4 100644 --- a/src/qemu/qemu_migration.c +++ b/src/qemu/qemu_migration.c @@ -7012,17 +7012,36 @@ qemuMigrationSrcToLegacyFile(virQEMUDriver *driver, static int qemuMigrationSrcToMappedFile(virQEMUDriver *driver, virDomainObj *vm, + const char *path, int *fd, unsigned int flags, virDomainAsyncJob asyncJob) { + g_autoptr(virQEMUDriverConfig) cfg =3D virQEMUDriverGetConfig(driver); + VIR_AUTOCLOSE directFd =3D -1; + int directFlag =3D 0; + bool needUnlink =3D false; int ret; =20 - /* mapped-ram does not support directIO */ + /* When using directio with mapped-ram, qemu needs two fds. One with + * O_DIRECT set writing the memory, and another without it set for + * writing small bits of unaligned state. */ if ((flags & VIR_DOMAIN_SAVE_BYPASS_CACHE)) { - virReportError(VIR_ERR_OPERATION_FAILED, "%s", - _("bypass cache unsupported by this system")); - return -1; + directFlag =3D virFileDirectFdFlag(); + if (directFlag < 0) { + virReportError(VIR_ERR_OPERATION_FAILED, "%s", + _("bypass cache unsupported by this system")); + return -1; + } + directFd =3D virQEMUFileOpenAs(cfg->user, cfg->group, false, path, + O_WRONLY | directFlag, &needUnlink); + + if (directFd < 0) + return -1; + + if (qemuSecuritySetImageFDLabel(driver->securityManager, vm->def, = directFd) < 0) + return -1; + } =20 if (qemuSecuritySetImageFDLabel(driver->securityManager, vm->def, *fd)= < 0) @@ -7031,7 +7050,7 @@ qemuMigrationSrcToMappedFile(virQEMUDriver *driver, if (qemuDomainObjEnterMonitorAsync(vm, asyncJob) < 0) return -1; =20 - ret =3D qemuMonitorMigrateToFdSet(vm, 0, fd); + ret =3D qemuMonitorMigrateToFdSet(vm, 0, fd, &directFd); qemuDomainObjExitMonitor(vm); return ret; } @@ -7040,6 +7059,7 @@ qemuMigrationSrcToMappedFile(virQEMUDriver *driver, /* Helper function called while vm is active. */ int qemuMigrationSrcToFile(virQEMUDriver *driver, virDomainObj *vm, + const char *path, int *fd, virCommand *compressor, qemuMigrationParams *migParams, @@ -7077,7 +7097,7 @@ qemuMigrationSrcToFile(virQEMUDriver *driver, virDoma= inObj *vm, =20 if (migParams && qemuMigrationParamsCapEnabled(migParams, QEMU_MIGRATION_CAP_MAPPED= _RAM)) { - rc =3D qemuMigrationSrcToMappedFile(driver, vm, fd, flags, asyncJo= b); + rc =3D qemuMigrationSrcToMappedFile(driver, vm, path, fd, flags, a= syncJob); } else { rc =3D qemuMigrationSrcToLegacyFile(driver, vm, *fd, compressor, a= syncJob); } diff --git a/src/qemu/qemu_migration.h b/src/qemu/qemu_migration.h index 5529f2ee21..51d5b680bc 100644 --- a/src/qemu/qemu_migration.h +++ b/src/qemu/qemu_migration.h @@ -236,6 +236,7 @@ qemuMigrationSrcIsAllowed(virDomainObj *vm, int qemuMigrationSrcToFile(virQEMUDriver *driver, virDomainObj *vm, + const char *path, int *fd, virCommand *compressor, qemuMigrationParams *migParams, diff --git a/src/qemu/qemu_migration_params.c b/src/qemu/qemu_migration_par= ams.c index 503d6165b0..8f6003005c 100644 --- a/src/qemu/qemu_migration_params.c +++ b/src/qemu/qemu_migration_params.c @@ -129,6 +129,7 @@ VIR_ENUM_IMPL(qemuMigrationParam, "multifd-compression", "multifd-zlib-level", "multifd-zstd-level", + "direct-io", ); =20 typedef struct _qemuMigrationParamsAlwaysOnItem qemuMigrationParamsAlwaysO= nItem; @@ -319,6 +320,9 @@ static const qemuMigrationParamInfoItem qemuMigrationPa= ramInfo[] =3D { [QEMU_MIGRATION_PARAM_MULTIFD_ZSTD_LEVEL] =3D { .type =3D QEMU_MIGRATION_PARAM_TYPE_INT, }, + [QEMU_MIGRATION_PARAM_DIRECT_IO] =3D { + .type =3D QEMU_MIGRATION_PARAM_TYPE_BOOL, + }, }; G_STATIC_ASSERT(G_N_ELEMENTS(qemuMigrationParamInfo) =3D=3D QEMU_MIGRATION= _PARAM_LAST); =20 @@ -784,7 +788,7 @@ qemuMigrationParamsFromFlags(virTypedParameterPtr param= s, =20 =20 qemuMigrationParams * -qemuMigrationParamsForSave(bool mappedRam) +qemuMigrationParamsForSave(bool mappedRam, unsigned int flags) { g_autoptr(qemuMigrationParams) saveParams =3D NULL; =20 @@ -798,6 +802,11 @@ qemuMigrationParamsForSave(bool mappedRam) return NULL; saveParams->params[QEMU_MIGRATION_PARAM_MULTIFD_CHANNELS].value.i = =3D 1; saveParams->params[QEMU_MIGRATION_PARAM_MULTIFD_CHANNELS].set =3D = true; + + if (flags & VIR_DOMAIN_SAVE_BYPASS_CACHE) { + saveParams->params[QEMU_MIGRATION_PARAM_DIRECT_IO].value.b =3D= true; + saveParams->params[QEMU_MIGRATION_PARAM_DIRECT_IO].set =3D tru= e; + } } =20 return g_steal_pointer(&saveParams); diff --git a/src/qemu/qemu_migration_params.h b/src/qemu/qemu_migration_par= ams.h index fe239d9a8f..9700469b5e 100644 --- a/src/qemu/qemu_migration_params.h +++ b/src/qemu/qemu_migration_params.h @@ -65,6 +65,7 @@ typedef enum { QEMU_MIGRATION_PARAM_MULTIFD_COMPRESSION, QEMU_MIGRATION_PARAM_MULTIFD_ZLIB_LEVEL, QEMU_MIGRATION_PARAM_MULTIFD_ZSTD_LEVEL, + QEMU_MIGRATION_PARAM_DIRECT_IO, =20 QEMU_MIGRATION_PARAM_LAST } qemuMigrationParam; @@ -87,7 +88,7 @@ qemuMigrationParamsFromFlags(virTypedParameterPtr params, qemuMigrationParty party); =20 qemuMigrationParams * -qemuMigrationParamsForSave(bool mappedRam); +qemuMigrationParamsForSave(bool mappedRam, unsigned int flags); =20 int qemuMigrationParamsDump(qemuMigrationParams *migParams, diff --git a/src/qemu/qemu_monitor.c b/src/qemu/qemu_monitor.c index 9a454a1d08..832ede639e 100644 --- a/src/qemu/qemu_monitor.c +++ b/src/qemu/qemu_monitor.c @@ -2240,7 +2240,8 @@ qemuMonitorMigrateToFd(qemuMonitor *mon, int qemuMonitorMigrateToFdSet(virDomainObj *vm, unsigned int flags, - int *fd) + int *fd, + int *directFd) { qemuDomainObjPrivate *priv =3D vm->privateData; qemuMonitor *mon =3D priv->mon; @@ -2250,7 +2251,7 @@ qemuMonitorMigrateToFdSet(virDomainObj *vm, g_autofree char *uri =3D NULL; int ret; =20 - VIR_DEBUG("fd=3D%d flags=3D0x%x", *fd, flags); + VIR_DEBUG("fd=3D%d directFd=3D%d flags=3D0x%x", *fd, *directFd, flags); =20 QEMU_CHECK_MONITOR(mon); =20 @@ -2262,6 +2263,8 @@ qemuMonitorMigrateToFdSet(virDomainObj *vm, =20 fdPassMigrate =3D qemuFDPassNew("migrate", priv); qemuFDPassAddFD(fdPassMigrate, fd, "-buffered-fd"); + if (*directFd !=3D -1) + qemuFDPassAddFD(fdPassMigrate, directFd, "-directio-fd"); if (qemuFDPassTransferMonitor(fdPassMigrate, mon) < 0) return -1; =20 diff --git a/src/qemu/qemu_monitor.h b/src/qemu/qemu_monitor.h index ebacdf110e..63385e93f3 100644 --- a/src/qemu/qemu_monitor.h +++ b/src/qemu/qemu_monitor.h @@ -846,7 +846,8 @@ int qemuMonitorMigrateToFd(qemuMonitor *mon, =20 int qemuMonitorMigrateToFdSet(virDomainObj *vm, unsigned int flags, - int *fd); + int *fd, + int *directFd); =20 int qemuMonitorMigrateToHost(qemuMonitor *mon, unsigned int flags, diff --git a/src/qemu/qemu_saveimage.c b/src/qemu/qemu_saveimage.c index b99e0de1ff..8ffd26d57a 100644 --- a/src/qemu/qemu_saveimage.c +++ b/src/qemu/qemu_saveimage.c @@ -450,7 +450,7 @@ qemuSaveImageCreate(virQEMUDriver *driver, goto cleanup; =20 /* Perform the migration */ - if (qemuMigrationSrcToFile(driver, vm, &fd, compressor, saveParams, fl= ags, asyncJob) < 0) + if (qemuMigrationSrcToFile(driver, vm, path, &fd, compressor, savePara= ms, flags, asyncJob) < 0) goto cleanup; =20 /* Touch up file header to mark image complete. */ --=20 2.35.3