From nobody Tue Feb 10 00:58:56 2026 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 170.10.129.124 as permitted sender) client-ip=170.10.129.124; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-delivery-124.mimecast.com; Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=fail(p=none dis=none) header.from=suse.de Return-Path: Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by mx.zohomail.com with SMTPS id 1652608205032612.7374523204644; Sun, 15 May 2022 02:50:05 -0700 (PDT) Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-83-L6I32W9dN1mQBfwLimWqRg-1; Sun, 15 May 2022 05:49:53 -0400 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.rdu2.redhat.com [10.11.54.1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 6A635382ECC3; Sun, 15 May 2022 09:49:49 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (unknown [10.30.29.100]) by smtp.corp.redhat.com (Postfix) with ESMTP id 54C4A40CF910; Sun, 15 May 2022 09:49:49 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (localhost [IPv6:::1]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id 2B178193223E; Sun, 15 May 2022 09:49:49 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx10.intmail.prod.int.rdu2.redhat.com [10.11.54.10]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id 4DE5E1932118 for ; Sun, 15 May 2022 09:49:46 +0000 (UTC) Received: by smtp.corp.redhat.com (Postfix) id 2E081415F5E; Sun, 15 May 2022 09:49:46 +0000 (UTC) Received: from mimecast-mx02.redhat.com (mimecast09.extmail.prod.ext.rdu2.redhat.com [10.11.55.25]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 29D2B415F30 for ; Sun, 15 May 2022 09:49:46 +0000 (UTC) Received: from us-smtp-1.mimecast.com (us-smtp-2.mimecast.com [205.139.110.61]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 0E5C829AB400 for ; Sun, 15 May 2022 09:49:46 +0000 (UTC) Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.220.28]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-44-89CJIhR5OOWDNAbuDS_Xew-1; Sun, 15 May 2022 05:49:44 -0400 Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id DBF9621D94; Sun, 15 May 2022 09:49:42 +0000 (UTC) Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id A9C7A13AFD; Sun, 15 May 2022 09:49:42 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id MDvqJ7bMgGIHbQAAMHmgww (envelope-from ); Sun, 15 May 2022 09:49:42 +0000 X-MC-Unique: L6I32W9dN1mQBfwLimWqRg-1 X-Original-To: libvir-list@listman.corp.redhat.com X-MC-Unique: 89CJIhR5OOWDNAbuDS_Xew-1 From: Claudio Fontana To: =?UTF-8?q?Daniel=20P=20=2E=20Berrang=C3=A9?= Subject: [libvirt RFCv10 15/32] qemu: saveimage: add virQEMUSaveFd APIs for multifd Date: Sun, 15 May 2022 11:49:11 +0200 Message-Id: <20220515094928.2783-16-cfontana@suse.de> In-Reply-To: <20220515094928.2783-1-cfontana@suse.de> References: <20220515094928.2783-1-cfontana@suse.de> MIME-Version: 1.0 X-Mimecast-Impersonation-Protect: Policy=CLT - Impersonation Protection Definition; Similar Internal Domain=false; Similar Monitored External Domain=false; Custom External Domain=false; Mimecast External Domain=false; Newly Observed Domain=false; Internal User Name=false; Custom Display Name List=false; Reply-to Address Mismatch=false; Targeted Threat Dictionary=false; Mimecast Threat Dictionary=false; Custom Threat Dictionary=false X-Scanned-By: MIMEDefang 2.85 on 10.11.54.10 X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: libvir-list@redhat.com, "Dr . David Alan Gilbert" , Claudio Fontana Errors-To: libvir-list-bounces@redhat.com Sender: "libvir-list" X-Scanned-By: MIMEDefang 2.84 on 10.11.54.1 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=libvir-list-bounces@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable X-ZM-MESSAGEID: 1652608206048100010 Content-Type: text/plain; charset="utf-8"; x-default="true" add APIs to Create, Close and Free MultiFD files to associate with multifd channels. Adapt virQEMUSaveFdInit to consider multifd. Signed-off-by: Claudio Fontana --- src/qemu/qemu_driver.c | 10 ++-- src/qemu/qemu_saveimage.c | 117 +++++++++++++++++++++++++++++++++++--- src/qemu/qemu_saveimage.h | 17 +++++- 3 files changed, 129 insertions(+), 15 deletions(-) diff --git a/src/qemu/qemu_driver.c b/src/qemu/qemu_driver.c index d071df1c81..a03ead960b 100644 --- a/src/qemu/qemu_driver.c +++ b/src/qemu/qemu_driver.c @@ -5869,7 +5869,7 @@ qemuDomainRestoreInternal(virConnectPtr conn, } oflags |=3D O_DIRECT; } - if (virQEMUSaveFdInit(&saveFd, path, oflags, cfg) < 0) + if (virQEMUSaveFdInit(&saveFd, path, 0, oflags, cfg, false) < 0) return -1; if (qemuSaveImageOpen(driver, NULL, &def, &data, false, &saveFd) < 0) goto cleanup; @@ -6003,7 +6003,7 @@ qemuDomainSaveImageGetXMLDesc(virConnectPtr conn, con= st char *path, =20 virCheckFlags(VIR_DOMAIN_SAVE_IMAGE_XML_SECURE, NULL); =20 - if (virQEMUSaveFdInit(&saveFd, path, O_RDONLY, cfg) < 0) + if (virQEMUSaveFdInit(&saveFd, path, 0, O_RDONLY, cfg, false) < 0) return NULL; if (qemuSaveImageOpen(driver, NULL, &def, &data, false, &saveFd) < 0) goto cleanup; @@ -6041,7 +6041,7 @@ qemuDomainSaveImageDefineXML(virConnectPtr conn, cons= t char *path, else if (flags & VIR_DOMAIN_SAVE_PAUSED) state =3D 0; =20 - if (virQEMUSaveFdInit(&saveFd, path, O_RDWR, cfg) < 0) + if (virQEMUSaveFdInit(&saveFd, path, 0, O_RDWR, cfg, false) < 0) return -1; if (qemuSaveImageOpen(driver, NULL, &def, &data, false, &saveFd) < 0) goto cleanup; @@ -6122,7 +6122,7 @@ qemuDomainManagedSaveGetXMLDesc(virDomainPtr dom, uns= igned int flags) goto cleanup; } =20 - if (virQEMUSaveFdInit(&saveFd, path, O_RDONLY, cfg) < 0) + if (virQEMUSaveFdInit(&saveFd, path, 0, O_RDONLY, cfg, false) < 0) goto cleanup; if (qemuSaveImageOpen(driver, priv->qemuCaps, &def, &data, false, &saveFd) < 0) @@ -6198,7 +6198,7 @@ qemuDomainObjRestore(virConnectPtr conn, } oflags |=3D O_DIRECT; } - if (virQEMUSaveFdInit(&saveFd, path, oflags, cfg) < 0) + if (virQEMUSaveFdInit(&saveFd, path, 0, oflags, cfg, false) < 0) goto cleanup; ret =3D qemuSaveImageOpen(driver, NULL, &def, &data, true, &saveFd); if (ret < 0) { diff --git a/src/qemu/qemu_saveimage.c b/src/qemu/qemu_saveimage.c index ca6eccd7b5..5c35c39d83 100644 --- a/src/qemu/qemu_saveimage.c +++ b/src/qemu/qemu_saveimage.c @@ -400,15 +400,17 @@ qemuSaveImageGetCompressionCommand(virQEMUSaveFormat = compression) * virQEMUSaveFdInit: initialize a virQEMUSaveFd * * @saveFd: the structure to initialize - * @base: the file name + * @base: the main file name + * @idx: 0 for the main file, > 0 for the multifd channels. * @oflags the file descriptor open flags * @cfg: the driver config + * @parallel: whether parallel save is enabled * * Returns -1 on error, 0 on success, * and in both cases virQEMUSaveFdFini must be called to free resources. */ -int virQEMUSaveFdInit(virQEMUSaveFd *saveFd, const char *base, - int oflags, virQEMUDriverConfig *cfg) +int virQEMUSaveFdInit(virQEMUSaveFd *saveFd, const char *base, int idx, + int oflags, virQEMUDriverConfig *cfg, bool parallel) { unsigned int wrapperFlags =3D VIR_FILE_WRAPPER_NON_BLOCKING; bool isCreat =3D oflags & O_CREAT; @@ -416,8 +418,11 @@ int virQEMUSaveFdInit(virQEMUSaveFd *saveFd, const cha= r *base, =20 if (isDirect) wrapperFlags |=3D VIR_FILE_WRAPPER_BYPASS_CACHE; - - saveFd->path =3D g_strdup(base); + if (idx > 0) { + saveFd->path =3D g_strdup_printf("%s.%d", base, idx); + } else { + saveFd->path =3D g_strdup(base); + } saveFd->wrapper =3D NULL; if (isCreat) { saveFd->fd =3D virQEMUFileOpenAs(cfg->user, cfg->group, false, sav= eFd->path, @@ -428,10 +433,11 @@ int virQEMUSaveFdInit(virQEMUSaveFd *saveFd, const ch= ar *base, if (saveFd->fd < 0) return -1; /* + * iohelper Wrapper is never required for multifd parallel save. * For O_CREAT, we always add the wrapper, * and for !O_CREAT, we only add the wrapper if using O_DIRECT. */ - if (isDirect || isCreat) { + if (!parallel && (isDirect || isCreat)) { saveFd->wrapper =3D virFileWrapperFdNew(&saveFd->fd, saveFd->path,= wrapperFlags); if (!saveFd->wrapper) return -1; @@ -508,6 +514,103 @@ int virQEMUSaveFdFini(virQEMUSaveFd *saveFd, virDomai= nObj *vm, int ret) return ret; } =20 +/* + * qemuSaveImageFreeMultiFd: free all multifd virQEMUSaveFds. + * @multiFd: the array of saveFds + * @vm: the virDomainObj, to release lock + * @nconn: number of multifd channels + * @ret: the current operation result (< 0 is failure) + * + * If multiFd is NULL, the return value will be unchanged. + * + * Returns ret, or -1 if an error is detected. + */ +int qemuSaveImageFreeMultiFd(virQEMUSaveFd *multiFd, virDomainObj *vm, int= nconn, int ret) +{ + int idx; + + if (!multiFd) + return ret; + + for (idx =3D 0; idx < nconn; idx++) { + ret =3D virQEMUSaveFdFini(&multiFd[idx], vm, ret); + } + /* + * do it again to unlink all in the error case, + * if error happened in the middle of previous loop. + */ + for (idx =3D 0; idx < nconn; idx++) { + ret =3D virQEMUSaveFdFini(&multiFd[idx], vm, ret); + } + g_free(multiFd); + return ret; +} + +/* + * qemuSaveImageCloseMultiFd: perform normal close on all multifd virQEMUS= aveFds. + * + * @multiFd: the array of saveFds + * @nconn: number of multifd channels + * @vm: the virDomainObj, to release lock + * + * If multiFd is NULL, the function will return success. + * Returns -1 on error, 0 on success. + */ +int qemuSaveImageCloseMultiFd(virQEMUSaveFd *multiFd, int nconn, virDomain= Obj *vm) +{ + int idx; + + if (!multiFd) + return 0; + + for (idx =3D 0; idx < nconn; idx++) { + if (virQEMUSaveFdClose(&multiFd[idx], vm) < 0) { + return -1; + } + } + return 0; +} + +/* + * qemuSaveImageCreateMultiFd: allocate and initialize all multifd virQEMU= SaveFds. + * + * @driver: qemu driver data + * @vm: the virDomainObj + * @cmd: the existing multifd helper command, to pass each fd as argum= ent. + * @path: pathname of the main file. + * @oflags: the open flags desired, to be passed to virQEMUSaveFdInit. + * @cfg: the driver config + * @nconn: number of channel files to create or open, depending on oflag= s. + * + * Returns the new array of virQEMUSaveFds, or NULL on error. + */ +virQEMUSaveFd * +qemuSaveImageCreateMultiFd(virQEMUDriver *driver, virDomainObj *vm, + virCommand *cmd, const char *path, + int oflags, virQEMUDriverConfig *cfg, + int nconn) +{ + virQEMUSaveFd *multiFd =3D g_new0(virQEMUSaveFd, nconn); + int idx; + + for (idx =3D 0; idx < nconn; idx++) { + virQEMUSaveFd *m =3D &multiFd[idx]; + if (virQEMUSaveFdInit(m, path, idx + 1, oflags, cfg, true) < 0 || + qemuSecuritySetImageFDLabel(driver->securityManager, vm->def, = m->fd) < 0) { + + virQEMUSaveFdFini(m, vm, -1); + goto error; + } + virCommandAddArgFormat(cmd, "%d", m->fd); + virCommandPassFD(cmd, m->fd, 0); + } + return multiFd; + + error: + qemuSaveImageFreeMultiFd(multiFd, vm, nconn, -1); + return NULL; +} + =20 /* Helper function to execute a migration to file with a correct save head= er * the caller needs to make sure that the processors are stopped and do al= l other @@ -536,7 +639,7 @@ qemuSaveImageCreate(virQEMUDriver *driver, oflags |=3D O_DIRECT; } =20 - if (virQEMUSaveFdInit(&saveFd, path, oflags, cfg) < 0) + if (virQEMUSaveFdInit(&saveFd, path, 0, oflags, cfg, false) < 0) goto cleanup; if (qemuSecuritySetImageFDLabel(driver->securityManager, vm->def, save= Fd.fd) < 0) goto cleanup; diff --git a/src/qemu/qemu_saveimage.h b/src/qemu/qemu_saveimage.h index e5a51955da..e85e5690b2 100644 --- a/src/qemu/qemu_saveimage.h +++ b/src/qemu/qemu_saveimage.h @@ -64,14 +64,25 @@ struct _virQEMUSaveFd { =20 #define QEMU_SAVEFD_INVALID (virQEMUSaveFd) { .path =3D NULL, .fd =3D -1, = .need_unlink =3D false, .wrapper =3D NULL } =20 -int virQEMUSaveFdInit(virQEMUSaveFd *saveFd, const char *base, - int oflags, virQEMUDriverConfig *cfg) - ATTRIBUTE_NONNULL(1) ATTRIBUTE_NONNULL(4); +int virQEMUSaveFdInit(virQEMUSaveFd *saveFd, const char *base, int idx, + int oflags, virQEMUDriverConfig *cfg, bool parallel) + ATTRIBUTE_NONNULL(1) ATTRIBUTE_NONNULL(5); =20 int virQEMUSaveFdClose(virQEMUSaveFd *saveFd, virDomainObj *vm); =20 int virQEMUSaveFdFini(virQEMUSaveFd *saveFd, virDomainObj *vm, int ret); =20 +virQEMUSaveFd * +qemuSaveImageCreateMultiFd(virQEMUDriver *driver, virDomainObj *vm, + virCommand *cmd, const char *path, + int oflags, virQEMUDriverConfig *cfg, + int nconn) + ATTRIBUTE_NONNULL(1) ATTRIBUTE_NONNULL(2) ATTRIBUTE_NONNULL(3) ATTRIBU= TE_NONNULL(4) ATTRIBUTE_NONNULL(6); + +int qemuSaveImageCloseMultiFd(virQEMUSaveFd *multiFd, int nconn, virDomain= Obj *vm); + +int qemuSaveImageFreeMultiFd(virQEMUSaveFd *multiFd, virDomainObj *vm, int= nconn, int ret); + virDomainDef * qemuSaveImageUpdateDef(virQEMUDriver *driver, virDomainDef *def, --=20 2.35.3