From nobody Fri May 17 11:58:59 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 170.10.133.124 as permitted sender) client-ip=170.10.133.124; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-delivery-124.mimecast.com; Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=fail(p=none dis=none) header.from=suse.de Return-Path: Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by mx.zohomail.com with SMTPS id 1654593619769720.4007815294814; Tue, 7 Jun 2022 02:20:19 -0700 (PDT) Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-518-3de7etXdNDa4QUtRVHhMOg-1; Tue, 07 Jun 2022 05:20:11 -0400 Received: from smtp.corp.redhat.com (int-mx09.intmail.prod.int.rdu2.redhat.com [10.11.54.9]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 8EBB2101A56D; Tue, 7 Jun 2022 09:19:53 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (unknown [10.30.29.100]) by smtp.corp.redhat.com (Postfix) with ESMTP id 792AB492C3B; Tue, 7 Jun 2022 09:19:53 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (localhost [IPv6:::1]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id 47DE519452D8; Tue, 7 Jun 2022 09:19:53 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.rdu2.redhat.com [10.11.54.3]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id 6704319452D8 for ; Tue, 7 Jun 2022 09:19:51 +0000 (UTC) Received: by smtp.corp.redhat.com (Postfix) id 39AE01121319; Tue, 7 Jun 2022 09:19:51 +0000 (UTC) Received: from mimecast-mx02.redhat.com (mimecast10.extmail.prod.ext.rdu2.redhat.com [10.11.55.26]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 3516F1121315 for ; Tue, 7 Jun 2022 09:19:51 +0000 (UTC) Received: from us-smtp-1.mimecast.com (us-smtp-delivery-1.mimecast.com [207.211.31.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 10D281C01709 for ; Tue, 7 Jun 2022 09:19:51 +0000 (UTC) Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.220.28]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-561-0tveJGznO3mZqdIxVFYjkA-1; Tue, 07 Jun 2022 05:19:43 -0400 Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id B01EB21B2F; Tue, 7 Jun 2022 09:19:41 +0000 (UTC) Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 743F313638; Tue, 7 Jun 2022 09:19:41 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id aNOuGi0Yn2IoLwAAMHmgww (envelope-from ); Tue, 07 Jun 2022 09:19:41 +0000 X-MC-Unique: 3de7etXdNDa4QUtRVHhMOg-1 X-Original-To: libvir-list@listman.corp.redhat.com X-MC-Unique: 0tveJGznO3mZqdIxVFYjkA-1 From: Claudio Fontana To: =?UTF-8?q?Daniel=20P=20=2E=20Berrang=C3=A9?= Subject: [libvirt RFCv11 01/33] virfile: introduce virFileDirect APIs Date: Tue, 7 Jun 2022 11:19:04 +0200 Message-Id: <20220607091936.7948-2-cfontana@suse.de> In-Reply-To: <20220607091936.7948-1-cfontana@suse.de> References: <20220607091936.7948-1-cfontana@suse.de> MIME-Version: 1.0 X-Mimecast-Impersonation-Protect: Policy=CLT - Impersonation Protection Definition; Similar Internal Domain=false; Similar Monitored External Domain=false; Custom External Domain=false; Mimecast External Domain=false; Newly Observed Domain=false; Internal User Name=false; Custom Display Name List=false; Reply-to Address Mismatch=false; Targeted Threat Dictionary=false; Mimecast Threat Dictionary=false; Custom Threat Dictionary=false X-Scanned-By: MIMEDefang 2.78 on 10.11.54.3 X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: libvir-list@redhat.com, Claudio Fontana , "Dr . David Alan Gilbert" , Anthony Iliopoulos Errors-To: libvir-list-bounces@redhat.com Sender: "libvir-list" X-Scanned-By: MIMEDefang 2.85 on 10.11.54.9 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=libvir-list-bounces@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable X-ZM-MESSAGEID: 1654593620433100008 Content-Type: text/plain; charset="utf-8"; x-default="true" these functions help with allocating buffers, aligning, reading and writing files opened with O_DIRECT. Signed-off-by: Claudio Fontana --- src/libvirt_private.syms | 6 ++ src/util/virfile.c | 174 +++++++++++++++++++++++++++++++++++++++ src/util/virfile.h | 9 ++ 3 files changed, 189 insertions(+) diff --git a/src/libvirt_private.syms b/src/libvirt_private.syms index bfedd85326..702f640b5d 100644 --- a/src/libvirt_private.syms +++ b/src/libvirt_private.syms @@ -2240,7 +2240,13 @@ virFileComparePaths; virFileCopyACLs; virFileDataSync; virFileDeleteTree; +virFileDirectAlign; +virFileDirectBufferNew; +virFileDirectCopyBuf; virFileDirectFdFlag; +virFileDirectRead; +virFileDirectReadCopy; +virFileDirectWrite; virFileExists; virFileFclose; virFileFdopen; diff --git a/src/util/virfile.c b/src/util/virfile.c index 99da058db3..e8500704e5 100644 --- a/src/util/virfile.c +++ b/src/util/virfile.c @@ -192,6 +192,135 @@ virFileDirectFdFlag(void) return O_DIRECT ? O_DIRECT : -1; } =20 +/** + * virFileDirectAlign: align a value, which may include a pointer. + * + * @value: the value that will be incremented to reach alignment + * + * Returns the aligned value. + */ +uintptr_t +virFileDirectAlign(uintptr_t value) +{ + return (value + VIR_FILE_DIRECT_ALIGN_MASK) & ~VIR_FILE_DIRECT_ALIGN_M= ASK; +} + +/** + * virFileDirectRead: perform a single aligned read. + * @fd: O_DIRECT file descriptor + * @buf: aligned buffer to read into + * @count: the desired read size. Note that if unaligned, + * extra bytes will be read based on alignment. + * + * Note that buf should be able to contain count plus the alignment! + * + * Returns < 0 and errno set on error, or the number of bytes read, + * which may be smaller or even greater than count. + */ +ssize_t +virFileDirectRead(int fd, void *buf, size_t count) +{ + size_t aligned_count =3D virFileDirectAlign(count); + while (count > 0) { + ssize_t r =3D read(fd, buf, aligned_count); + if (r < 0 && errno =3D=3D EINTR) + continue; + return r; + } + return 0; +} + +/** + * virFileDirectCopyBuf: copy a buffer contents to destination in memory + * @buf: aligned buffer to copy from + * @count: amount of data to copy + * @dst: the destination in memory + * @dst_len: the maximum the destination can hold. + * + * copies up to count bytes from buf into dst, but not more than dst_len. + * increments buf pointer and dst pointer, as well as decrementing the + * maximum the destination can hold (dst_len). + * + * Returns the amount copied. + */ +size_t +virFileDirectCopyBuf(void **buf, size_t count, void **dst, size_t *dst_len) +{ + size_t to_copy; + + to_copy =3D count > *dst_len ? *dst_len : count; + memcpy(*dst, *buf, to_copy); + *dst_len -=3D to_copy; + *(char **)dst +=3D to_copy; + *(char **)buf +=3D to_copy; + return to_copy; +} + +/** + * virFileDirectReadCopy: read an fd and copy contents to memory + * @fd: O_DIRECT file descriptor + * @buf: aligned buffer to read the fd into, and then copy from + * @buflen: size of the buffer + * @dst: the destination in memory + * @dst_len: the maximum the destination can hold. + * + * reads data from the fd file descriptor into the buffer, + * and then copy to the destination, filling it up to dst_len. + * + * Returns < 0 and errno set on error, + * or the number of bytes read, which may be past the requested dst_len, + * or may be smaller if the fd does not contain enough data. + * + * The buf pointer is updated to point to eventual exccess data in the buf= fer. + */ +ssize_t +virFileDirectReadCopy(int fd, void **buf, size_t buflen, void *dst, size_t= dst_len) +{ + ssize_t nread =3D 0; + void *d =3D dst; + char *s =3D *buf; + + while (dst_len > 0) { + ssize_t rv; + *buf =3D s; + rv =3D virFileDirectRead(fd, s, dst_len < buflen ? dst_len : bufle= n); + if (rv < 0) + return rv; + if (rv =3D=3D 0) + return nread; /* not enough data to fulfill request */ + + nread +=3D rv; /* note, we might read past the requested len = */ + virFileDirectCopyBuf(buf, rv, &d, &dst_len); + } + return nread; +} + +/** + * virFileDirectWrite: perform a single aligned write. + * @fd: O_DIRECT file descriptor to write to + * @buf: aligned buffer to write from + * @count: the desired write size. Note that if unaligned, + * extra 0 bytes will be written based on alignment. + * + * Returns < 0 and errno set on error, or the number of bytes written, + * which may be smaller or even greater than count. + */ +ssize_t +virFileDirectWrite(int fd, void *buf, size_t count) +{ + size_t aligned_count =3D virFileDirectAlign(count); + if (aligned_count > count) { + memset((char *)buf + count, 0, aligned_count - count); + } + while (count > 0) { + ssize_t r =3D write(fd, buf, aligned_count); /* sc_avoid_write */ + if (r < 0 && errno =3D=3D EINTR) + continue; + return r; + } + return 0; +} + /* Opaque type for managing a wrapper around a fd. For now, * read-write is not supported, just a single direction. */ struct _virFileWrapperFd { @@ -202,6 +331,41 @@ struct _virFileWrapperFd { =20 #ifndef WIN32 =20 +/** + * virFileDirectBufferNew: allocate a buffer and return the first + * block-aligned address in it. + * + * @alloc_base: pointer to the to-be-allocated memory buffer. + * @buflen: desired length, which should be greater than alignment. + * + * Allocate a memory area large enough to accommodate an aligned + * buffer of size buflen. + * + * On success, *alloc_base is set to the newly allocated memory, + * and the aligned buffer within it is returned. + * + * On failure, *alloc_base is set to NULL and the function + * returns NULL. + */ +void * +virFileDirectBufferNew(void **alloc_base, size_t buflen) +{ + void *buf; + buflen =3D virFileDirectAlign(buflen); + +# if WITH_POSIX_MEMALIGN + if (posix_memalign(alloc_base, VIR_FILE_DIRECT_ALIGN_MASK + 1, buflen)= ) { + *alloc_base =3D NULL; + return NULL; + } + buf =3D *alloc_base; +# else + *alloc_base =3D g_malloc(buflen + VIR_FILE_DIRECT_ALIGN_MASK); + buf =3D virFileDirectAlign((uintptr_t)*alloc_base); +# endif + return buf; +} + # ifdef __linux__ =20 /** @@ -372,6 +536,16 @@ virFileWrapperFdNew(int *fd, const char *name, unsigne= d int flags) return NULL; } #else /* WIN32 */ + +void * +virFileDirectBufferNew(void **alloc_base G_GNUC_UNUSED, + size_t buflen G_GNUC_UNUSED) +{ + virReportError(VIR_ERR_INTERNAL_ERROR, "%s", + _("virFileDirectBufferNew unsupported on this platform"= )); + return NULL; +} + virFileWrapperFd * virFileWrapperFdNew(int *fd G_GNUC_UNUSED, const char *name G_GNUC_UNUSED, diff --git a/src/util/virfile.h b/src/util/virfile.h index 8e378efe30..844261e0a4 100644 --- a/src/util/virfile.h +++ b/src/util/virfile.h @@ -104,6 +104,15 @@ typedef struct _virFileWrapperFd virFileWrapperFd; =20 int virFileDirectFdFlag(void); =20 +#define VIR_FILE_DIRECT_ALIGN_MASK ((64 * 1024) - 1) + +void *virFileDirectBufferNew(void **alloc_base, size_t buflen); +uintptr_t virFileDirectAlign(uintptr_t value); +ssize_t virFileDirectRead(int fd, void *buf, size_t count); +ssize_t virFileDirectWrite(int fd, void *buf, size_t count); +size_t virFileDirectCopyBuf(void **buf, size_t count, void **dst, size_t *= dst_len); +ssize_t virFileDirectReadCopy(int fd, void **buf, size_t buflen, void *dst= , size_t dst_len); + typedef enum { VIR_FILE_WRAPPER_BYPASS_CACHE =3D (1 << 0), VIR_FILE_WRAPPER_NON_BLOCKING =3D (1 << 1), --=20 2.26.2 From nobody Fri May 17 11:58:59 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 170.10.129.124 as permitted sender) client-ip=170.10.129.124; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-delivery-124.mimecast.com; Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=fail(p=none dis=none) header.from=suse.de Return-Path: Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by mx.zohomail.com with SMTPS id 1654593603271149.83264779206468; Tue, 7 Jun 2022 02:20:03 -0700 (PDT) Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-375-GsuZMQfAOWeaenn9Xc_mlQ-1; Tue, 07 Jun 2022 05:19:48 -0400 Received: from smtp.corp.redhat.com (int-mx10.intmail.prod.int.rdu2.redhat.com [10.11.54.10]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 4826C858EED; Tue, 7 Jun 2022 09:19:46 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (unknown [10.30.29.100]) by smtp.corp.redhat.com (Postfix) with ESMTP id 31C3D40336E; Tue, 7 Jun 2022 09:19:46 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (localhost [IPv6:::1]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id EE74019451EF; Tue, 7 Jun 2022 09:19:45 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.rdu2.redhat.com [10.11.54.1]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id 6B24619452D8 for ; Tue, 7 Jun 2022 09:19:45 +0000 (UTC) Received: by smtp.corp.redhat.com (Postfix) id 43AF9406AD4B; Tue, 7 Jun 2022 09:19:45 +0000 (UTC) Received: from mimecast-mx02.redhat.com (mimecast07.extmail.prod.ext.rdu2.redhat.com [10.11.55.23]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 3F7D5400E861 for ; Tue, 7 Jun 2022 09:19:45 +0000 (UTC) Received: from us-smtp-1.mimecast.com (us-smtp-1.mimecast.com [207.211.31.81]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 26CC53C8ACE1 for ; Tue, 7 Jun 2022 09:19:45 +0000 (UTC) Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.220.28]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-589-7R7LWloNOYOR2sDVMRT23g-1; Tue, 07 Jun 2022 05:19:43 -0400 Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id F2CDF21B35; Tue, 7 Jun 2022 09:19:41 +0000 (UTC) Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id B78FE13638; Tue, 7 Jun 2022 09:19:41 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id mFsxKy0Yn2IoLwAAMHmgww (envelope-from ); Tue, 07 Jun 2022 09:19:41 +0000 X-MC-Unique: GsuZMQfAOWeaenn9Xc_mlQ-1 X-Original-To: libvir-list@listman.corp.redhat.com X-MC-Unique: 7R7LWloNOYOR2sDVMRT23g-1 From: Claudio Fontana To: =?UTF-8?q?Daniel=20P=20=2E=20Berrang=C3=A9?= Subject: [libvirt RFCv11 02/33] virfile: use virFileDirect API in runIOCopy Date: Tue, 7 Jun 2022 11:19:05 +0200 Message-Id: <20220607091936.7948-3-cfontana@suse.de> In-Reply-To: <20220607091936.7948-1-cfontana@suse.de> References: <20220607091936.7948-1-cfontana@suse.de> MIME-Version: 1.0 X-Mimecast-Impersonation-Protect: Policy=CLT - Impersonation Protection Definition; Similar Internal Domain=false; Similar Monitored External Domain=false; Custom External Domain=false; Mimecast External Domain=false; Newly Observed Domain=false; Internal User Name=false; Custom Display Name List=false; Reply-to Address Mismatch=false; Targeted Threat Dictionary=false; Mimecast Threat Dictionary=false; Custom Threat Dictionary=false X-Scanned-By: MIMEDefang 2.84 on 10.11.54.1 X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: libvir-list@redhat.com, Claudio Fontana , "Dr . David Alan Gilbert" , Anthony Iliopoulos Errors-To: libvir-list-bounces@redhat.com Sender: "libvir-list" X-Scanned-By: MIMEDefang 2.85 on 10.11.54.10 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=libvir-list-bounces@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable X-ZM-MESSAGEID: 1654593603826100001 Content-Type: text/plain; charset="utf-8"; x-default="true" Signed-off-by: Claudio Fontana --- src/util/virfile.c | 28 ++++++++-------------------- 1 file changed, 8 insertions(+), 20 deletions(-) diff --git a/src/util/virfile.c b/src/util/virfile.c index e8500704e5..770649108f 100644 --- a/src/util/virfile.c +++ b/src/util/virfile.c @@ -4776,20 +4776,14 @@ static off_t runIOCopy(const struct runIOParams p) { g_autofree void *base =3D NULL; /* Location to be freed */ - char *buf =3D NULL; /* Aligned location within base */ - size_t buflen =3D 1024*1024; - intptr_t alignMask =3D 64*1024 - 1; off_t total =3D 0; + size_t buflen =3D 1024*1024; + char *buf =3D virFileDirectBufferNew(&base, buflen); =20 -# if WITH_POSIX_MEMALIGN - if (posix_memalign(&base, alignMask + 1, buflen)) - abort(); - buf =3D base; -# else - buf =3D g_new0(char, buflen + alignMask); - base =3D buf; - buf =3D (char *) (((intptr_t) base + alignMask) & ~alignMask); -# endif + if (!buf) { + virReportSystemError(errno, _("Failed to allocate aligned memory i= n function %s"), __FUNCTION__); + return -5; + } =20 while (1) { ssize_t got; @@ -4801,9 +4795,7 @@ runIOCopy(const struct runIOParams p) * In other cases using saferead reduces number of syscalls. */ if (!p.isWrite && p.isDirect) { - if ((got =3D read(p.fdin, buf, buflen)) < 0 && - errno =3D=3D EINTR) - continue; + got =3D virFileDirectRead(p.fdin, buf, buflen); } else { got =3D saferead(p.fdin, buf, buflen); } @@ -4819,11 +4811,7 @@ runIOCopy(const struct runIOParams p) =20 /* handle last write size align in direct case */ if (got < buflen && p.isDirect && p.isWrite) { - ssize_t aligned_got =3D (got + alignMask) & ~alignMask; - - memset(buf + got, 0, aligned_got - got); - - if (safewrite(p.fdout, buf, aligned_got) < 0) { + if (virFileDirectWrite(p.fdout, buf, got) < 0) { virReportSystemError(errno, _("Unable to write %s"), p.fdo= utname); return -3; } --=20 2.26.2 From nobody Fri May 17 11:58:59 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 170.10.129.124 as permitted sender) client-ip=170.10.129.124; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-delivery-124.mimecast.com; Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=fail(p=none dis=none) header.from=suse.de Return-Path: Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by mx.zohomail.com with SMTPS id 1654593619827555.8402750004028; Tue, 7 Jun 2022 02:20:19 -0700 (PDT) Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-369-JwpkT_kWPhun4xsV-dxvvQ-1; Tue, 07 Jun 2022 05:20:05 -0400 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.rdu2.redhat.com [10.11.54.3]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id F368B804199; Tue, 7 Jun 2022 09:19:50 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (unknown [10.30.29.100]) by smtp.corp.redhat.com (Postfix) with ESMTP id D81581121314; Tue, 7 Jun 2022 09:19:50 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (localhost [IPv6:::1]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id 3B64B1947B9E; Tue, 7 Jun 2022 09:19:50 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.rdu2.redhat.com [10.11.54.8]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id E530419451F0 for ; Tue, 7 Jun 2022 09:19:46 +0000 (UTC) Received: by smtp.corp.redhat.com (Postfix) id B8046C23DBF; Tue, 7 Jun 2022 09:19:46 +0000 (UTC) Received: from mimecast-mx02.redhat.com (mimecast04.extmail.prod.ext.rdu2.redhat.com [10.11.55.20]) by smtp.corp.redhat.com (Postfix) with ESMTPS id B3FF2C15E72 for ; Tue, 7 Jun 2022 09:19:46 +0000 (UTC) Received: from us-smtp-1.mimecast.com (us-smtp-2.mimecast.com [207.211.31.81]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 9769E10726A2 for ; Tue, 7 Jun 2022 09:19:46 +0000 (UTC) Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.220.28]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-53-6jbiIn8TPDS4BzgMZZDzkw-1; Tue, 07 Jun 2022 05:19:43 -0400 Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id 4289821B38; Tue, 7 Jun 2022 09:19:42 +0000 (UTC) Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 07B9E13638; Tue, 7 Jun 2022 09:19:42 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id oOhEAC4Yn2IoLwAAMHmgww (envelope-from ); Tue, 07 Jun 2022 09:19:42 +0000 X-MC-Unique: JwpkT_kWPhun4xsV-dxvvQ-1 X-Original-To: libvir-list@listman.corp.redhat.com X-MC-Unique: 6jbiIn8TPDS4BzgMZZDzkw-1 From: Claudio Fontana To: =?UTF-8?q?Daniel=20P=20=2E=20Berrang=C3=A9?= Subject: [libvirt RFCv11 03/33] qemu: saveimage: rework image read/write to be O_DIRECT friendly Date: Tue, 7 Jun 2022 11:19:06 +0200 Message-Id: <20220607091936.7948-4-cfontana@suse.de> In-Reply-To: <20220607091936.7948-1-cfontana@suse.de> References: <20220607091936.7948-1-cfontana@suse.de> MIME-Version: 1.0 X-Mimecast-Impersonation-Protect: Policy=CLT - Impersonation Protection Definition; Similar Internal Domain=false; Similar Monitored External Domain=false; Custom External Domain=false; Mimecast External Domain=false; Newly Observed Domain=false; Internal User Name=false; Custom Display Name List=false; Reply-to Address Mismatch=false; Targeted Threat Dictionary=false; Mimecast Threat Dictionary=false; Custom Threat Dictionary=false X-Scanned-By: MIMEDefang 2.85 on 10.11.54.8 X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: libvir-list@redhat.com, Claudio Fontana , "Dr . David Alan Gilbert" , Anthony Iliopoulos Errors-To: libvir-list-bounces@redhat.com Sender: "libvir-list" X-Scanned-By: MIMEDefang 2.78 on 10.11.54.3 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=libvir-list-bounces@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable X-ZM-MESSAGEID: 1654593620421100007 Content-Type: text/plain; charset="utf-8"; x-default="true" maintain a compatible image format (still QEMU_SAVE_VERSION 2), but change the on-disk representation to be more O_DIRECT friendly: 1) ensure that the header struct fields are packed, so we can be sure no padding will ever ruin the day in the future 2) finish the libvirt header (header + xml + cookie) with zero padding, in order to ensure that the QEMU VM (QEVM Magic) is aligned. Adapt the read and write of the libvirt header accordingly. The old images should be still loadable after this change, and the new images should be still loadable in old libvirt as well. Signed-off-by: Claudio Fontana --- src/qemu/qemu_saveimage.c | 267 ++++++++++++++++++++++++-------------- src/qemu/qemu_saveimage.h | 20 +-- 2 files changed, 182 insertions(+), 105 deletions(-) diff --git a/src/qemu/qemu_saveimage.c b/src/qemu/qemu_saveimage.c index 4fd4c5cfcd..276033ff6a 100644 --- a/src/qemu/qemu_saveimage.c +++ b/src/qemu/qemu_saveimage.c @@ -139,12 +139,12 @@ virQEMUSaveDataWrite(virQEMUSaveData *data, int fd, const char *path) { + g_autofree void *base =3D NULL; + char *buf, *cur; virQEMUSaveHeader *header =3D &data->header; size_t len; size_t xml_len; size_t cookie_len =3D 0; - size_t zerosLen =3D 0; - g_autofree char *zeros =3D NULL; =20 xml_len =3D strlen(data->xml) + 1; if (data->cookie) @@ -165,42 +165,190 @@ virQEMUSaveDataWrite(virQEMUSaveData *data, return -1; } } + /* align data_len */ + len =3D sizeof(*header) + header->data_len; + header->data_len +=3D virFileDirectAlign(len) - len; =20 - zerosLen =3D header->data_len - len; - zeros =3D g_new0(char, zerosLen); + buf =3D virFileDirectBufferNew(&base, sizeof(*header) + header->data_l= en); + cur =3D buf; =20 if (data->cookie) header->cookieOffset =3D xml_len; =20 - if (safewrite(fd, header, sizeof(*header)) !=3D sizeof(*header)) { - virReportSystemError(errno, - _("failed to write header to domain save file= '%s'"), - path); - return -1; + memcpy(cur, header, sizeof(*header)); + cur +=3D sizeof(*header); + memcpy(cur, data->xml, xml_len); + cur +=3D xml_len; + if (data->cookie) { + memcpy(cur, data->cookie, cookie_len); + cur +=3D cookie_len; } =20 - if (safewrite(fd, data->xml, xml_len) !=3D xml_len) { + if (virFileDirectWrite(fd, buf, sizeof(*header) + header->data_len) < = 0) { virReportSystemError(errno, - _("failed to write domain xml to '%s'"), + _("failed to write libvirt header of domain s= ave file '%s'"), path); return -1; } =20 - if (data->cookie && - safewrite(fd, data->cookie, cookie_len) !=3D cookie_len) { + return 0; +} + +/* virQEMUSaveDataRead: + * + * Reads libvirt's header (including domain XML) from a saved image. + * + * Returns -1 on generic failure, -3 on a corrupted image, or 0 on success. + */ +int +virQEMUSaveDataRead(virQEMUSaveData *data, + int fd, + const char *path) +{ + g_autofree void *base =3D NULL; + virQEMUSaveHeader *header =3D &data->header; + size_t xml_len; + size_t cookie_len; + ssize_t rv; + void *dst; + int oflags =3D fcntl(fd, F_GETFL); + bool isDirect =3D O_DIRECT && (oflags & O_DIRECT); + size_t buflen =3D 1024 * 1024; + char *buf =3D NULL; + void *src =3D NULL; + + header =3D &data->header; + if (isDirect) { + buf =3D virFileDirectBufferNew(&base, buflen); + src =3D buf; + rv =3D virFileDirectReadCopy(fd, &src, buflen, header, sizeof(*hea= der)); + } else { + rv =3D saferead(fd, header, sizeof(*header)); + } + if (rv < 0) { virReportSystemError(errno, - _("failed to write cookie to '%s'"), + _("failed to read libvirt header of domain sa= ve file '%s'"), path); return -1; } + if (rv < sizeof(*header)) { + virReportError(VIR_ERR_OPERATION_FAILED, + _("domain save file '%s' libvirt header appears tru= ncated"), + path); + return -3; + } + rv -=3D sizeof(*header); =20 - if (safewrite(fd, zeros, zerosLen) !=3D zerosLen) { - virReportSystemError(errno, - _("failed to write padding to '%s'"), - path); + if (memcmp(header->magic, QEMU_SAVE_MAGIC, sizeof(header->magic)) !=3D= 0) { + if (memcmp(header->magic, QEMU_SAVE_PARTIAL, sizeof(header->magic)= ) =3D=3D 0) { + virReportError(VIR_ERR_OPERATION_FAILED, + _("domain save file '%s' seems incomplete"), + path); + return -3; + } + virReportError(VIR_ERR_OPERATION_FAILED, "%s", + _("image magic is incorrect")); + return -1; + } + if (header->version > QEMU_SAVE_VERSION) { + /* convert endianness and try again */ + qemuSaveImageBswapHeader(header); + } + if (header->version > QEMU_SAVE_VERSION) { + virReportError(VIR_ERR_OPERATION_FAILED, + _("image version is not supported (%d > %d)"), + header->version, QEMU_SAVE_VERSION); return -1; } + if (header->cookieOffset) + xml_len =3D header->cookieOffset; + else + xml_len =3D header->data_len; =20 + if (xml_len <=3D 0) { + virReportError(VIR_ERR_OPERATION_FAILED, + _("invalid xml length: %lu"), (unsigned long)xml_le= n); + return -1; + } + if (header->data_len < xml_len) { + virReportError(VIR_ERR_OPERATION_FAILED, + _("invalid cookieOffset: %u"), header->cookieOffset= ); + return -1; + } + cookie_len =3D header->data_len - xml_len; + data->xml =3D g_new0(char, xml_len); + dst =3D data->xml; + if (rv > 0) { + if (isDirect) { + rv -=3D virFileDirectCopyBuf(&src, rv, &dst, &xml_len); + } else { + virReportError(VIR_ERR_INTERNAL_ERROR, "%s", _("unexpected lef= tover data in non-direct mode")); + return -1; + } + } + if (xml_len > 0) { + if (isDirect) { + rv =3D virFileDirectReadCopy(fd, &src, buflen, dst, xml_len); + } else { + rv =3D saferead(fd, dst, xml_len); + } + if (rv < 0) { + virReportSystemError(errno, + _("failed to read libvirt xml in domain s= ave file '%s'"), + path); + return -1; + } + if (rv < xml_len) { + virReportError(VIR_ERR_OPERATION_FAILED, + _("domain save file '%s' xml seems incomplete"), + path); + return -3; + } + rv -=3D xml_len; + } + if (cookie_len > 0) { + data->cookie =3D g_new0(char, cookie_len); + dst =3D data->cookie; + if (rv > 0) { + if (isDirect) { + rv -=3D virFileDirectCopyBuf(&src, rv, &dst, &cookie_len); + } else { + virReportError(VIR_ERR_INTERNAL_ERROR, "%s", _("unexpected= leftover data in non-direct mode")); + return -1; + } + } + if (cookie_len > 0) { + if (isDirect) { + rv =3D virFileDirectReadCopy(fd, &src, buflen, dst, cookie= _len); + } else { + rv =3D saferead(fd, dst, cookie_len); + } + if (rv < 0) { + virReportSystemError(errno, + _("failed to read libvirt cookie in d= omain save file '%s'"), + path); + return -1; + } + if (rv < cookie_len) { + virReportError(VIR_ERR_OPERATION_FAILED, + _("domain save file '%s' cookie seems incom= plete"), + path); + return -3; + } + rv -=3D cookie_len; + } + } + /* + * we should now be aligned and ready to read the QEVM. + */ + if (rv > 0) { + if (isDirect) { + virReportError(VIR_ERR_INTERNAL_ERROR, "%s", _("unexpected una= ligned data in direct mode")); + } else { + virReportError(VIR_ERR_INTERNAL_ERROR, "%s", _("unexpected lef= tover data in non-direct mode")); + return -1; + } + } return 0; } =20 @@ -444,11 +592,8 @@ qemuSaveImageOpen(virQEMUDriver *driver, VIR_AUTOCLOSE fd =3D -1; int ret =3D -1; g_autoptr(virQEMUSaveData) data =3D NULL; - virQEMUSaveHeader *header; g_autoptr(virDomainDef) def =3D NULL; int oflags =3D open_write ? O_RDWR : O_RDONLY; - size_t xml_len; - size_t cookie_len; =20 if (bypass_cache) { int directFlag =3D virFileDirectFdFlag(); @@ -469,89 +614,17 @@ qemuSaveImageOpen(virQEMUDriver *driver, return -1; =20 data =3D g_new0(virQEMUSaveData, 1); - - header =3D &data->header; - if (saferead(fd, header, sizeof(*header)) !=3D sizeof(*header)) { - if (unlink_corrupt) { + ret =3D virQEMUSaveDataRead(data, fd, path); + if (ret < 0) { + if (unlink_corrupt && ret =3D=3D -3) { if (unlink(path) < 0) { virReportSystemError(errno, _("cannot remove corrupt file: %s"), path); return -1; - } else { - return -3; } } - - virReportError(VIR_ERR_OPERATION_FAILED, - "%s", _("failed to read qemu header")); - return -1; - } - - if (memcmp(header->magic, QEMU_SAVE_MAGIC, sizeof(header->magic)) !=3D= 0) { - if (memcmp(header->magic, QEMU_SAVE_PARTIAL, sizeof(header->magic)= ) =3D=3D 0) { - if (unlink_corrupt) { - if (unlink(path) < 0) { - virReportSystemError(errno, - _("cannot remove corrupt file: %s= "), - path); - return -1; - } else { - return -3; - } - } - - virReportError(VIR_ERR_OPERATION_FAILED, "%s", - _("save image is incomplete")); - return -1; - } - - virReportError(VIR_ERR_OPERATION_FAILED, "%s", - _("image magic is incorrect")); - return -1; - } - - if (header->version > QEMU_SAVE_VERSION) { - /* convert endianness and try again */ - qemuSaveImageBswapHeader(header); - } - - if (header->version > QEMU_SAVE_VERSION) { - virReportError(VIR_ERR_OPERATION_FAILED, - _("image version is not supported (%d > %d)"), - header->version, QEMU_SAVE_VERSION); - return -1; - } - - if (header->data_len <=3D 0) { - virReportError(VIR_ERR_OPERATION_FAILED, - _("invalid header data length: %d"), header->data_l= en); - return -1; - } - - if (header->cookieOffset) - xml_len =3D header->cookieOffset; - else - xml_len =3D header->data_len; - - cookie_len =3D header->data_len - xml_len; - - data->xml =3D g_new0(char, xml_len); - - if (saferead(fd, data->xml, xml_len) !=3D xml_len) { - virReportError(VIR_ERR_OPERATION_FAILED, - "%s", _("failed to read domain XML")); - return -1; - } - - if (cookie_len > 0) { - data->cookie =3D g_new0(char, cookie_len); - - if (saferead(fd, data->cookie, cookie_len) !=3D cookie_len) { - virReportError(VIR_ERR_OPERATION_FAILED, "%s", - _("failed to read cookie")); - return -1; - } + return ret; } =20 /* Create a domain from this XML */ diff --git a/src/qemu/qemu_saveimage.h b/src/qemu/qemu_saveimage.h index 391cd55ed0..f12374b3d7 100644 --- a/src/qemu/qemu_saveimage.h +++ b/src/qemu/qemu_saveimage.h @@ -36,14 +36,14 @@ G_STATIC_ASSERT(sizeof(QEMU_SAVE_MAGIC) =3D=3D sizeof(Q= EMU_SAVE_PARTIAL)); =20 typedef struct _virQEMUSaveHeader virQEMUSaveHeader; struct _virQEMUSaveHeader { - char magic[sizeof(QEMU_SAVE_MAGIC)-1]; - uint32_t version; - uint32_t data_len; - uint32_t was_running; - uint32_t compressed; - uint32_t cookieOffset; - uint32_t unused[14]; -}; + char magic[sizeof(QEMU_SAVE_MAGIC)-1]; /* 16 bytes */ + uint32_t version; /* 4 bytes */ + uint32_t data_len; /* 4 bytes */ + uint32_t was_running; /* 4 bytes */ + uint32_t compressed; /* 4 bytes */ + uint32_t cookieOffset; /* 4 bytes */ + uint32_t unused[14]; /* 56 bytes */ +} ATTRIBUTE_PACKED; /* =3D 92 bytes */ =20 =20 typedef struct _virQEMUSaveData virQEMUSaveData; @@ -103,6 +103,10 @@ int virQEMUSaveDataWrite(virQEMUSaveData *data, int fd, const char *path); +int +virQEMUSaveDataRead(virQEMUSaveData *data, + int fd, + const char *path); =20 virQEMUSaveData * virQEMUSaveDataNew(char *domXML, --=20 2.26.2 From nobody Fri May 17 11:58:59 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 170.10.133.124 as permitted sender) client-ip=170.10.133.124; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-delivery-124.mimecast.com; Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=fail(p=none dis=none) header.from=suse.de Return-Path: Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by mx.zohomail.com with SMTPS id 1654593610341392.37839381953654; Tue, 7 Jun 2022 02:20:10 -0700 (PDT) Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-562-6gj3ECNxPviZ6Je2uUZRRQ-1; Tue, 07 Jun 2022 05:19:57 -0400 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 5A3C910726CC; Tue, 7 Jun 2022 09:19:50 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (unknown [10.30.29.100]) by smtp.corp.redhat.com (Postfix) with ESMTP id 3BDDE2026987; Tue, 7 Jun 2022 09:19:50 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (localhost [IPv6:::1]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id BFF5C1947BA0; Tue, 7 Jun 2022 09:19:49 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx10.intmail.prod.int.rdu2.redhat.com [10.11.54.10]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id 502771947B93 for ; Tue, 7 Jun 2022 09:19:46 +0000 (UTC) Received: by smtp.corp.redhat.com (Postfix) id 1C6A2492C3B; Tue, 7 Jun 2022 09:19:46 +0000 (UTC) Received: from mimecast-mx02.redhat.com (mimecast04.extmail.prod.ext.rdu2.redhat.com [10.11.55.20]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 17CB240336A for ; Tue, 7 Jun 2022 09:19:46 +0000 (UTC) Received: from us-smtp-1.mimecast.com (us-smtp-2.mimecast.com [205.139.110.61]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 01A0F10726A5 for ; Tue, 7 Jun 2022 09:19:46 +0000 (UTC) Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.220.28]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-556-aJsUDyq9Mk2ptPgqrsagFg-1; Tue, 07 Jun 2022 05:19:44 -0400 Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id 8C00D21B3B; Tue, 7 Jun 2022 09:19:42 +0000 (UTC) Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 4B66E13638; Tue, 7 Jun 2022 09:19:42 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id 2BvOEC4Yn2IoLwAAMHmgww (envelope-from ); Tue, 07 Jun 2022 09:19:42 +0000 X-MC-Unique: 6gj3ECNxPviZ6Je2uUZRRQ-1 X-Original-To: libvir-list@listman.corp.redhat.com X-MC-Unique: aJsUDyq9Mk2ptPgqrsagFg-1 From: Claudio Fontana To: =?UTF-8?q?Daniel=20P=20=2E=20Berrang=C3=A9?= Subject: [libvirt RFCv11 04/33] qemu: saveimage: assume future formats will also support compression Date: Tue, 7 Jun 2022 11:19:07 +0200 Message-Id: <20220607091936.7948-5-cfontana@suse.de> In-Reply-To: <20220607091936.7948-1-cfontana@suse.de> References: <20220607091936.7948-1-cfontana@suse.de> MIME-Version: 1.0 X-Mimecast-Impersonation-Protect: Policy=CLT - Impersonation Protection Definition; Similar Internal Domain=false; Similar Monitored External Domain=false; Custom External Domain=false; Mimecast External Domain=false; Newly Observed Domain=false; Internal User Name=false; Custom Display Name List=false; Reply-to Address Mismatch=false; Targeted Threat Dictionary=false; Mimecast Threat Dictionary=false; Custom Threat Dictionary=false X-Scanned-By: MIMEDefang 2.85 on 10.11.54.10 X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: libvir-list@redhat.com, Claudio Fontana , "Dr . David Alan Gilbert" , Anthony Iliopoulos Errors-To: libvir-list-bounces@redhat.com Sender: "libvir-list" X-Scanned-By: MIMEDefang 2.78 on 10.11.54.4 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=libvir-list-bounces@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable X-ZM-MESSAGEID: 1654593611848100011 Content-Type: text/plain; charset="utf-8"; x-default="true" change the version check to assume even future versions of the format will continue to support compression. Signed-off-by: Claudio Fontana --- src/qemu/qemu_saveimage.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/qemu/qemu_saveimage.c b/src/qemu/qemu_saveimage.c index 276033ff6a..af97f45114 100644 --- a/src/qemu/qemu_saveimage.c +++ b/src/qemu/qemu_saveimage.c @@ -674,7 +674,7 @@ qemuSaveImageStartVM(virConnectPtr conn, virDomainXMLOptionGetSaveCookie(driver->x= mlopt)) < 0) goto cleanup; =20 - if ((header->version =3D=3D 2) && + if ((header->version >=3D 2) && (header->compressed !=3D QEMU_SAVE_FORMAT_RAW)) { if (!(cmd =3D qemuSaveImageGetCompressionCommand(header->compresse= d))) goto cleanup; --=20 2.26.2 From nobody Fri May 17 11:58:59 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 170.10.129.124 as permitted sender) client-ip=170.10.129.124; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-delivery-124.mimecast.com; Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=fail(p=none dis=none) header.from=suse.de Return-Path: Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by mx.zohomail.com with SMTPS id 1654593609452320.94469553279373; Tue, 7 Jun 2022 02:20:09 -0700 (PDT) Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-219-JeUnh-RRNYGfnWEvkNlRMw-1; Tue, 07 Jun 2022 05:20:04 -0400 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.rdu2.redhat.com [10.11.54.6]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 5A3363C8ACF9; Tue, 7 Jun 2022 09:19:49 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (unknown [10.30.29.100]) by smtp.corp.redhat.com (Postfix) with ESMTP id 712352166B26; Tue, 7 Jun 2022 09:19:48 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (localhost [IPv6:::1]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id 4F27F19452D8; Tue, 7 Jun 2022 09:19:48 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id 698B519452D8 for ; Tue, 7 Jun 2022 09:19:46 +0000 (UTC) Received: by smtp.corp.redhat.com (Postfix) id 398C62026985; Tue, 7 Jun 2022 09:19:46 +0000 (UTC) Received: from mimecast-mx02.redhat.com (mimecast09.extmail.prod.ext.rdu2.redhat.com [10.11.55.25]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 351BC2026D64 for ; Tue, 7 Jun 2022 09:19:46 +0000 (UTC) Received: from us-smtp-1.mimecast.com (us-smtp-2.mimecast.com [205.139.110.61]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 19FE529DD984 for ; Tue, 7 Jun 2022 09:19:46 +0000 (UTC) Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.220.29]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-518-kt8Hq6f_M2e1zHagmzGscQ-1; Tue, 07 Jun 2022 05:19:44 -0400 Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id CD3281F941; Tue, 7 Jun 2022 09:19:42 +0000 (UTC) Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 9443613638; Tue, 7 Jun 2022 09:19:42 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id QBOtIi4Yn2IoLwAAMHmgww (envelope-from ); Tue, 07 Jun 2022 09:19:42 +0000 X-MC-Unique: JeUnh-RRNYGfnWEvkNlRMw-1 X-Original-To: libvir-list@listman.corp.redhat.com X-MC-Unique: kt8Hq6f_M2e1zHagmzGscQ-1 From: Claudio Fontana To: =?UTF-8?q?Daniel=20P=20=2E=20Berrang=C3=A9?= Subject: [libvirt RFCv11 05/33] virfile: virFileDiskCopy: prepare for O_DIRECT files without wrapper Date: Tue, 7 Jun 2022 11:19:08 +0200 Message-Id: <20220607091936.7948-6-cfontana@suse.de> In-Reply-To: <20220607091936.7948-1-cfontana@suse.de> References: <20220607091936.7948-1-cfontana@suse.de> MIME-Version: 1.0 X-Mimecast-Impersonation-Protect: Policy=CLT - Impersonation Protection Definition; Similar Internal Domain=false; Similar Monitored External Domain=false; Custom External Domain=false; Mimecast External Domain=false; Newly Observed Domain=false; Internal User Name=false; Custom Display Name List=false; Reply-to Address Mismatch=false; Targeted Threat Dictionary=false; Mimecast Threat Dictionary=false; Custom Threat Dictionary=false X-Scanned-By: MIMEDefang 2.78 on 10.11.54.4 X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: libvir-list@redhat.com, Claudio Fontana , "Dr . David Alan Gilbert" , Anthony Iliopoulos Errors-To: libvir-list-bounces@redhat.com Sender: "libvir-list" X-Scanned-By: MIMEDefang 2.78 on 10.11.54.6 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=libvir-list-bounces@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable X-ZM-MESSAGEID: 1654593609871100008 Content-Type: text/plain; charset="utf-8"; x-default="true" we will allow to use already open fds that are not empty for both read and write, as long as they are properly aligned. Adapt the truncation to take into account the initial offset. Signed-off-by: Claudio Fontana --- src/util/virfile.c | 44 +++++++++++++++++++++++--------------------- 1 file changed, 23 insertions(+), 21 deletions(-) diff --git a/src/util/virfile.c b/src/util/virfile.c index 770649108f..201d7f4e64 100644 --- a/src/util/virfile.c +++ b/src/util/virfile.c @@ -4811,14 +4811,24 @@ runIOCopy(const struct runIOParams p) =20 /* handle last write size align in direct case */ if (got < buflen && p.isDirect && p.isWrite) { - if (virFileDirectWrite(p.fdout, buf, got) < 0) { + ssize_t nwritten =3D virFileDirectWrite(p.fdout, buf, got); + if (nwritten < 0) { virReportSystemError(errno, _("Unable to write %s"), p.fdo= utname); return -3; } - - if (!p.isBlockDev && ftruncate(p.fdout, total) < 0) { - virReportSystemError(errno, _("Unable to truncate %s"), p.= fdoutname); - return -4; + if (!p.isBlockDev) { + off_t off =3D lseek(p.fdout, (off_t)0, SEEK_CUR); + if (off < 0) { + virReportSystemError(errno, "%s", _("Failed to lseek t= o get current file offset")); + return -6; + } + if (nwritten > got) { + off -=3D nwritten - got; + } + if (ftruncate(p.fdout, off) < 0) { + virReportSystemError(errno, _("Unable to truncate %s")= , p.fdoutname); + return -4; + } } =20 break; @@ -4898,23 +4908,15 @@ virFileDiskCopy(int disk_fd, const char *disk_path,= int remote_fd, const char *r (oflags & O_ACCMODE)); goto cleanup; } - /* To make the implementation simpler, we give up on any - * attempt to use O_DIRECT in a non-trivial manner. */ if (!p.isBlockDev && p.isDirect) { - off_t off; - if (p.isWrite) { - /* - * note: for write we do not only check that disk_fd is seekab= le, - * we also want to know that the file is empty, so we need SEE= K_END. - */ - if ((off =3D lseek(disk_fd, 0, SEEK_END)) !=3D 0) { - virReportSystemError(off < 0 ? errno : EINVAL, "%s", - _("O_DIRECT write needs empty seekabl= e file")); - goto cleanup; - } - } else if ((off =3D lseek(disk_fd, 0, SEEK_CUR)) !=3D 0) { - virReportSystemError(off < 0 ? errno : EINVAL, "%s", - _("O_DIRECT read needs entire seekable fi= le")); + off_t off =3D lseek(disk_fd, 0, SEEK_CUR); + if (off < 0) { + virReportSystemError(errno, "%s", _("O_DIRECT needs a seekable= file")); + goto cleanup; + } + if (virFileDirectAlign(off) !=3D off) { + /* we could write some zeroes, but maybe it is safer to just f= ail */ + virReportSystemError(EINVAL, "%s", _("O_DIRECT attempted with = unaligned file pointer")); goto cleanup; } } --=20 2.26.2 From nobody Fri May 17 11:58:59 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 170.10.129.124 as permitted sender) client-ip=170.10.129.124; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-delivery-124.mimecast.com; Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=fail(p=none dis=none) header.from=suse.de Return-Path: Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by mx.zohomail.com with SMTPS id 1654593605960684.6668662531481; Tue, 7 Jun 2022 02:20:05 -0700 (PDT) Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-461-LGqCZ5AIPiuGrzw2hUEKVA-1; Tue, 07 Jun 2022 05:19:55 -0400 Received: from smtp.corp.redhat.com (int-mx09.intmail.prod.int.rdu2.redhat.com [10.11.54.9]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 35DFA8041B2; Tue, 7 Jun 2022 09:19:49 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (unknown [10.30.29.100]) by smtp.corp.redhat.com (Postfix) with ESMTP id 1F445492C3B; Tue, 7 Jun 2022 09:19:49 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (localhost [IPv6:::1]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id E6DD71947B8B; Tue, 7 Jun 2022 09:19:48 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.rdu2.redhat.com [10.11.54.6]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id 6295E1947B97 for ; Tue, 7 Jun 2022 09:19:46 +0000 (UTC) Received: by smtp.corp.redhat.com (Postfix) id 3A4EA2166B29; Tue, 7 Jun 2022 09:19:46 +0000 (UTC) Received: from mimecast-mx02.redhat.com (mimecast10.extmail.prod.ext.rdu2.redhat.com [10.11.55.26]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 35B1D2166B26 for ; Tue, 7 Jun 2022 09:19:46 +0000 (UTC) Received: from us-smtp-1.mimecast.com (us-smtp-delivery-1.mimecast.com [205.139.110.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 19FC11C08968 for ; Tue, 7 Jun 2022 09:19:46 +0000 (UTC) Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.220.28]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-580-JMAkR0JJOvmp5WoBw2L3sA-1; Tue, 07 Jun 2022 05:19:44 -0400 Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id 1B52A21B3F; Tue, 7 Jun 2022 09:19:43 +0000 (UTC) Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id D62E713638; Tue, 7 Jun 2022 09:19:42 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id mPqoMi4Yn2IoLwAAMHmgww (envelope-from ); Tue, 07 Jun 2022 09:19:42 +0000 X-MC-Unique: LGqCZ5AIPiuGrzw2hUEKVA-1 X-Original-To: libvir-list@listman.corp.redhat.com X-MC-Unique: JMAkR0JJOvmp5WoBw2L3sA-1 From: Claudio Fontana To: =?UTF-8?q?Daniel=20P=20=2E=20Berrang=C3=A9?= Subject: [libvirt RFCv11 06/33] qemu: saveimage: introduce virQEMUSaveFd Date: Tue, 7 Jun 2022 11:19:09 +0200 Message-Id: <20220607091936.7948-7-cfontana@suse.de> In-Reply-To: <20220607091936.7948-1-cfontana@suse.de> References: <20220607091936.7948-1-cfontana@suse.de> MIME-Version: 1.0 X-Mimecast-Impersonation-Protect: Policy=CLT - Impersonation Protection Definition; Similar Internal Domain=false; Similar Monitored External Domain=false; Custom External Domain=false; Mimecast External Domain=false; Newly Observed Domain=false; Internal User Name=false; Custom Display Name List=false; Reply-to Address Mismatch=false; Targeted Threat Dictionary=false; Mimecast Threat Dictionary=false; Custom Threat Dictionary=false X-Scanned-By: MIMEDefang 2.78 on 10.11.54.6 X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: libvir-list@redhat.com, Claudio Fontana , "Dr . David Alan Gilbert" , Anthony Iliopoulos Errors-To: libvir-list-bounces@redhat.com Sender: "libvir-list" X-Scanned-By: MIMEDefang 2.85 on 10.11.54.9 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=libvir-list-bounces@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable X-ZM-MESSAGEID: 1654593607854100003 Content-Type: text/plain; charset="utf-8"; x-default="true" use this data type to encapsulate the pathname, file descriptor, wrapper, and need to unlink. We also carry along the oflags used to open the file, and the driver configuration. This will make management of the resources associated with an FD used for QEMU save/restore much easier, reducing the amount of explicit cleanup required. Signed-off-by: Claudio Fontana --- src/qemu/qemu_saveimage.c | 115 ++++++++++++++++++++++++++++++++++++++ src/qemu/qemu_saveimage.h | 23 ++++++++ 2 files changed, 138 insertions(+) diff --git a/src/qemu/qemu_saveimage.c b/src/qemu/qemu_saveimage.c index af97f45114..4c440098fa 100644 --- a/src/qemu/qemu_saveimage.c +++ b/src/qemu/qemu_saveimage.c @@ -396,6 +396,121 @@ qemuSaveImageGetCompressionCommand(virQEMUSaveFormat = compression) return ret; } =20 +/* + * virQEMUSaveFdInit: initialize a virQEMUSaveFd + * + * @saveFd: the structure to initialize + * @base: the file name + * @oflags the file descriptor open flags + * @cfg: the driver config + * + * Returns -1 on error, 0 on success, + * and in both cases virQEMUSaveFdFini must be called to free resources. + */ +int virQEMUSaveFdInit(virQEMUSaveFd *saveFd, const char *base, + int oflags, virQEMUDriverConfig *cfg) +{ + unsigned int wrapperFlags =3D VIR_FILE_WRAPPER_NON_BLOCKING; + bool isCreat =3D oflags & O_CREAT; + bool isDirect =3D O_DIRECT && (oflags & O_DIRECT); + + saveFd->oflags =3D oflags; + saveFd->cfg =3D cfg; + + if (isDirect) + wrapperFlags |=3D VIR_FILE_WRAPPER_BYPASS_CACHE; + + saveFd->path =3D g_strdup(base); + saveFd->wrapper =3D NULL; + if (isCreat) { + saveFd->fd =3D virQEMUFileOpenAs(cfg->user, cfg->group, false, sav= eFd->path, + oflags, &saveFd->need_unlink); + } else { + saveFd->fd =3D qemuDomainOpenFile(cfg, NULL, saveFd->path, oflags,= NULL); + } + if (saveFd->fd < 0) + return -1; + /* + * For O_CREAT, we always add the wrapper, + * and for !O_CREAT, we only add the wrapper if using O_DIRECT. + */ + if (isDirect || isCreat) { + saveFd->wrapper =3D virFileWrapperFdNew(&saveFd->fd, saveFd->path,= wrapperFlags); + if (!saveFd->wrapper) + return -1; + } + return 0; +} + +/* + * virQEMUSaveFdClose: close a virQEMUSaveFd descriptor with normal close. + * + * @saveFd: the saveFd structure with the file descriptors to close. + * @vm: the virDomainObj (necessary to release lock), or NULL. + * + * If saveFd is NULL, the function will return success. + * + * Returns -1 on error, 0 on success. + */ +int virQEMUSaveFdClose(virQEMUSaveFd *saveFd, virDomainObj *vm) +{ + if (!saveFd) + return 0; + + if (VIR_CLOSE(saveFd->fd) < 0) { + virReportSystemError(errno, _("unable to close %s"), saveFd->path); + return -1; + } + if (vm) { + if (qemuDomainFileWrapperFDClose(vm, saveFd->wrapper) < 0) + return -1; + } else { + if (virFileWrapperFdClose(saveFd->wrapper) < 0) + return -1; + } + return 0; +} + +/* + * virQEMUSaveFdFini: finalize a virQEMUSaveFd + * + * @saveFd: the saveFd structure containing the resources to free. + * @vm: the virDomainObj (necessary to release lock for long close ops= ), or NULL. + * @ret: the current operation result (< 0 is failure) + * + * If saveFd is NULL, the return value will be unchanged. + * + * Returns ret, or -1 if an error is detected. + */ +int virQEMUSaveFdFini(virQEMUSaveFd *saveFd, virDomainObj *vm, int ret) +{ + if (!saveFd) + return ret; + VIR_FORCE_CLOSE(saveFd->fd); + if (vm) { + if (qemuDomainFileWrapperFDClose(vm, saveFd->wrapper) < 0) + ret =3D -1; + } else { + if (virFileWrapperFdClose(saveFd->wrapper) < 0) + ret =3D -1; + } + + if (ret < 0 && saveFd->need_unlink && saveFd->path) { + if (unlink(saveFd->path) < 0) { + virReportSystemError(errno, _("cannot remove file: %s"), + saveFd->path); + } + } + if (saveFd->wrapper) { + virFileWrapperFdFree(saveFd->wrapper); + saveFd->wrapper =3D NULL; + } + + g_free(saveFd->path); + saveFd->path =3D NULL; + return ret; +} + =20 /* Helper function to execute a migration to file with a correct save head= er * the caller needs to make sure that the processors are stopped and do al= l other diff --git a/src/qemu/qemu_saveimage.h b/src/qemu/qemu_saveimage.h index f12374b3d7..5768e31fa6 100644 --- a/src/qemu/qemu_saveimage.h +++ b/src/qemu/qemu_saveimage.h @@ -54,6 +54,29 @@ struct _virQEMUSaveData { }; =20 =20 +typedef struct _virQEMUSaveFd virQEMUSaveFd; +struct _virQEMUSaveFd { + char *path; + int fd; + int oflags; + bool need_unlink; + virFileWrapperFd *wrapper; + virQEMUDriverConfig *cfg; +}; + +#define QEMU_SAVEFD_INVALID (virQEMUSaveFd) { \ + .path =3D NULL, .fd =3D -1, .oflags =3D 0, \ + .need_unlink =3D false, .wrapper =3D NULL, .cfg =3D NULL, \ +} + +int virQEMUSaveFdInit(virQEMUSaveFd *saveFd, const char *base, + int oflags, virQEMUDriverConfig *cfg) + ATTRIBUTE_NONNULL(1) ATTRIBUTE_NONNULL(4); + +int virQEMUSaveFdClose(virQEMUSaveFd *saveFd, virDomainObj *vm); + +int virQEMUSaveFdFini(virQEMUSaveFd *saveFd, virDomainObj *vm, int ret); + virDomainDef * qemuSaveImageUpdateDef(virQEMUDriver *driver, virDomainDef *def, --=20 2.26.2 From nobody Fri May 17 11:58:59 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 170.10.129.124 as permitted sender) client-ip=170.10.129.124; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-delivery-124.mimecast.com; Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=fail(p=none dis=none) header.from=suse.de Return-Path: Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by mx.zohomail.com with SMTPS id 165459361489514.756857784391627; Tue, 7 Jun 2022 02:20:14 -0700 (PDT) Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-561-WOetrE5iMXeyPF1OazZ_pw-1; Tue, 07 Jun 2022 05:20:09 -0400 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id D8FA41C08964; Tue, 7 Jun 2022 09:19:49 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (unknown [10.30.29.100]) by smtp.corp.redhat.com (Postfix) with ESMTP id C2F012026D64; Tue, 7 Jun 2022 09:19:49 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (localhost [IPv6:::1]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id 8F9B51947B99; Tue, 7 Jun 2022 09:19:49 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx10.intmail.prod.int.rdu2.redhat.com [10.11.54.10]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id 7647319451F1 for ; Tue, 7 Jun 2022 09:19:46 +0000 (UTC) Received: by smtp.corp.redhat.com (Postfix) id 6753F40336E; Tue, 7 Jun 2022 09:19:46 +0000 (UTC) Received: from mimecast-mx02.redhat.com (mimecast10.extmail.prod.ext.rdu2.redhat.com [10.11.55.26]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 63218492C3B for ; Tue, 7 Jun 2022 09:19:46 +0000 (UTC) Received: from us-smtp-1.mimecast.com (us-smtp-delivery-1.mimecast.com [207.211.31.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 4C2F11C08968 for ; Tue, 7 Jun 2022 09:19:46 +0000 (UTC) Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.220.29]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-583-S-V9g4piPIK3dgU7RFcD-Q-1; Tue, 07 Jun 2022 05:19:44 -0400 Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id 5DC5C1F97E; Tue, 7 Jun 2022 09:19:43 +0000 (UTC) Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 2431A13638; Tue, 7 Jun 2022 09:19:43 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id EM83By8Yn2IoLwAAMHmgww (envelope-from ); Tue, 07 Jun 2022 09:19:43 +0000 X-MC-Unique: WOetrE5iMXeyPF1OazZ_pw-1 X-Original-To: libvir-list@listman.corp.redhat.com X-MC-Unique: S-V9g4piPIK3dgU7RFcD-Q-1 From: Claudio Fontana To: =?UTF-8?q?Daniel=20P=20=2E=20Berrang=C3=A9?= Subject: [libvirt RFCv11 07/33] qemu: saveimage: convert qemuSaveImageCreate to use virQEMUSaveFd Date: Tue, 7 Jun 2022 11:19:10 +0200 Message-Id: <20220607091936.7948-8-cfontana@suse.de> In-Reply-To: <20220607091936.7948-1-cfontana@suse.de> References: <20220607091936.7948-1-cfontana@suse.de> MIME-Version: 1.0 X-Mimecast-Impersonation-Protect: Policy=CLT - Impersonation Protection Definition; Similar Internal Domain=false; Similar Monitored External Domain=false; Custom External Domain=false; Mimecast External Domain=false; Newly Observed Domain=false; Internal User Name=false; Custom Display Name List=false; Reply-to Address Mismatch=false; Targeted Threat Dictionary=false; Mimecast Threat Dictionary=false; Custom Threat Dictionary=false X-Scanned-By: MIMEDefang 2.85 on 10.11.54.10 X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: libvir-list@redhat.com, Claudio Fontana , "Dr . David Alan Gilbert" , Anthony Iliopoulos Errors-To: libvir-list-bounces@redhat.com Sender: "libvir-list" X-Scanned-By: MIMEDefang 2.78 on 10.11.54.4 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=libvir-list-bounces@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable X-ZM-MESSAGEID: 1654593615971100021 Content-Type: text/plain; charset="utf-8"; x-default="true" now that we introduced virQEMUSaveFd, use it in the creation of a new save image. Signed-off-by: Claudio Fontana --- src/qemu/qemu_saveimage.c | 54 +++++++++++---------------------------- 1 file changed, 15 insertions(+), 39 deletions(-) diff --git a/src/qemu/qemu_saveimage.c b/src/qemu/qemu_saveimage.c index 4c440098fa..2a58bd23b8 100644 --- a/src/qemu/qemu_saveimage.c +++ b/src/qemu/qemu_saveimage.c @@ -525,41 +525,31 @@ qemuSaveImageCreate(virQEMUDriver *driver, virDomainAsyncJob asyncJob) { g_autoptr(virQEMUDriverConfig) cfg =3D virQEMUDriverGetConfig(driver); - bool needUnlink =3D false; + virQEMUSaveFd saveFd =3D QEMU_SAVEFD_INVALID; + unsigned int oflags =3D O_WRONLY | O_TRUNC | O_CREAT; int ret =3D -1; - int fd =3D -1; - int directFlag =3D 0; - virFileWrapperFd *wrapperFd =3D NULL; - unsigned int wrapperFlags =3D VIR_FILE_WRAPPER_NON_BLOCKING; =20 - /* Obtain the file handle. */ - if ((flags & VIR_DOMAIN_SAVE_BYPASS_CACHE)) { - wrapperFlags |=3D VIR_FILE_WRAPPER_BYPASS_CACHE; - directFlag =3D virFileDirectFdFlag(); - if (directFlag < 0) { + if (flags & VIR_DOMAIN_SAVE_BYPASS_CACHE) { + if (virFileDirectFdFlag() < 0) { virReportError(VIR_ERR_OPERATION_FAILED, "%s", _("bypass cache unsupported by this system")); - goto cleanup; + return -1; } + oflags |=3D O_DIRECT; } =20 - fd =3D virQEMUFileOpenAs(cfg->user, cfg->group, false, path, - O_WRONLY | O_TRUNC | O_CREAT | directFlag, - &needUnlink); - if (fd < 0) + if (virQEMUSaveFdInit(&saveFd, path, oflags, cfg) < 0) goto cleanup; - - if (qemuSecuritySetImageFDLabel(driver->securityManager, vm->def, fd) = < 0) + if (qemuSecuritySetImageFDLabel(driver->securityManager, vm->def, save= Fd.fd) < 0) goto cleanup; - - if (!(wrapperFd =3D virFileWrapperFdNew(&fd, path, wrapperFlags))) + if (virQEMUSaveDataWrite(data, saveFd.fd, saveFd.path) < 0) goto cleanup; =20 - if (virQEMUSaveDataWrite(data, fd, path) < 0) + /* Perform the migration */ + if (qemuMigrationSrcToFile(driver, vm, saveFd.fd, compressor, asyncJob= ) < 0) goto cleanup; =20 - /* Perform the migration */ - if (qemuMigrationSrcToFile(driver, vm, fd, compressor, asyncJob) < 0) + if (virQEMUSaveFdClose(&saveFd, vm) < 0) goto cleanup; =20 /* Touch up file header to mark image complete. */ @@ -568,29 +558,15 @@ qemuSaveImageCreate(virQEMUDriver *driver, * up to seek backwards on wrapperFd. The reopened fd will * trigger a single page of file system cache pollution, but * that's acceptable. */ - if (VIR_CLOSE(fd) < 0) { - virReportSystemError(errno, _("unable to close %s"), path); - goto cleanup; - } =20 - if (qemuDomainFileWrapperFDClose(vm, wrapperFd) < 0) - goto cleanup; - - if ((fd =3D qemuDomainOpenFile(cfg, vm->def, path, O_WRONLY, NULL)) < = 0 || - virQEMUSaveDataFinish(data, &fd, path) < 0) + if ((saveFd.fd =3D qemuDomainOpenFile(cfg, vm->def, saveFd.path, O_WRO= NLY, NULL)) < 0 || + virQEMUSaveDataFinish(data, &saveFd.fd, saveFd.path) < 0) goto cleanup; =20 ret =3D 0; =20 cleanup: - VIR_FORCE_CLOSE(fd); - if (qemuDomainFileWrapperFDClose(vm, wrapperFd) < 0) - ret =3D -1; - virFileWrapperFdFree(wrapperFd); - - if (ret < 0 && needUnlink) - unlink(path); - + ret =3D virQEMUSaveFdFini(&saveFd, vm, ret); return ret; } =20 --=20 2.26.2 From nobody Fri May 17 11:58:59 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 170.10.129.124 as permitted sender) client-ip=170.10.129.124; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-delivery-124.mimecast.com; Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=fail(p=none dis=none) header.from=suse.de Return-Path: Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by mx.zohomail.com with SMTPS id 1654593604409701.451378088777; Tue, 7 Jun 2022 02:20:04 -0700 (PDT) Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-393-ipq3h_IkOVaunE-p0WoFLA-1; Tue, 07 Jun 2022 05:19:54 -0400 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.rdu2.redhat.com [10.11.54.6]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id F3025858F11; Tue, 7 Jun 2022 09:19:48 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (unknown [10.30.29.100]) by smtp.corp.redhat.com (Postfix) with ESMTP id D97292166B34; Tue, 7 Jun 2022 09:19:48 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (localhost [IPv6:::1]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id A98A11947B92; Tue, 7 Jun 2022 09:19:48 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx10.intmail.prod.int.rdu2.redhat.com [10.11.54.10]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id 89F2A19452D2 for ; Tue, 7 Jun 2022 09:19:47 +0000 (UTC) Received: by smtp.corp.redhat.com (Postfix) id 68655492C3B; Tue, 7 Jun 2022 09:19:47 +0000 (UTC) Received: from mimecast-mx02.redhat.com (mimecast02.extmail.prod.ext.rdu2.redhat.com [10.11.55.18]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 5F141416384 for ; Tue, 7 Jun 2022 09:19:47 +0000 (UTC) Received: from us-smtp-1.mimecast.com (us-smtp-2.mimecast.com [207.211.31.81]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 408F180419E for ; Tue, 7 Jun 2022 09:19:47 +0000 (UTC) Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.220.28]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-125-gk_e8OZON96XBCYbOn7rGA-1; Tue, 07 Jun 2022 05:19:45 -0400 Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id A103021B46; Tue, 7 Jun 2022 09:19:43 +0000 (UTC) Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 66BD113638; Tue, 7 Jun 2022 09:19:43 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id MF10Fy8Yn2IoLwAAMHmgww (envelope-from ); Tue, 07 Jun 2022 09:19:43 +0000 X-MC-Unique: ipq3h_IkOVaunE-p0WoFLA-1 X-Original-To: libvir-list@listman.corp.redhat.com X-MC-Unique: gk_e8OZON96XBCYbOn7rGA-1 From: Claudio Fontana To: =?UTF-8?q?Daniel=20P=20=2E=20Berrang=C3=A9?= Subject: [libvirt RFCv11 08/33] qemu: saveimage: convert qemuSaveImageOpen to use virQEMUSaveFd Date: Tue, 7 Jun 2022 11:19:11 +0200 Message-Id: <20220607091936.7948-9-cfontana@suse.de> In-Reply-To: <20220607091936.7948-1-cfontana@suse.de> References: <20220607091936.7948-1-cfontana@suse.de> MIME-Version: 1.0 X-Mimecast-Impersonation-Protect: Policy=CLT - Impersonation Protection Definition; Similar Internal Domain=false; Similar Monitored External Domain=false; Custom External Domain=false; Mimecast External Domain=false; Newly Observed Domain=false; Internal User Name=false; Custom Display Name List=false; Reply-to Address Mismatch=false; Targeted Threat Dictionary=false; Mimecast Threat Dictionary=false; Custom Threat Dictionary=false X-Scanned-By: MIMEDefang 2.85 on 10.11.54.10 X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: libvir-list@redhat.com, Claudio Fontana , "Dr . David Alan Gilbert" , Anthony Iliopoulos Errors-To: libvir-list-bounces@redhat.com Sender: "libvir-list" X-Scanned-By: MIMEDefang 2.78 on 10.11.54.6 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=libvir-list-bounces@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable X-ZM-MESSAGEID: 1654593605909100001 Content-Type: text/plain; charset="utf-8"; x-default="true" all the logic to open a fd, create a wrapper etc, is boilerplate code that is best reused, so change the Open function to take an existing already initialized virQEMUSaveFd. Adapt all callers of qemuSaveImageOpen. Signed-off-by: Claudio Fontana --- src/qemu/qemu_driver.c | 101 ++++++++++++++++++++++---------------- src/qemu/qemu_saveimage.c | 56 +++++---------------- src/qemu/qemu_saveimage.h | 9 ++-- 3 files changed, 73 insertions(+), 93 deletions(-) diff --git a/src/qemu/qemu_driver.c b/src/qemu/qemu_driver.c index 0c6645ed89..9ad4945928 100644 --- a/src/qemu/qemu_driver.c +++ b/src/qemu/qemu_driver.c @@ -5852,12 +5852,13 @@ qemuDomainRestoreInternal(virConnectPtr conn, virDomainObj *vm =3D NULL; g_autofree char *xmlout =3D NULL; const char *newxml =3D dxml; - int fd =3D -1; int ret =3D -1; virQEMUSaveData *data =3D NULL; - virFileWrapperFd *wrapperFd =3D NULL; + virQEMUSaveFd saveFd =3D QEMU_SAVEFD_INVALID; bool hook_taint =3D false; bool reset_nvram =3D false; + g_autoptr(virQEMUDriverConfig) cfg =3D virQEMUDriverGetConfig(driver); + int oflags =3D O_RDONLY; =20 virCheckFlags(VIR_DOMAIN_SAVE_BYPASS_CACHE | VIR_DOMAIN_SAVE_RUNNING | @@ -5867,10 +5868,17 @@ qemuDomainRestoreInternal(virConnectPtr conn, if (flags & VIR_DOMAIN_SAVE_RESET_NVRAM) reset_nvram =3D true; =20 - fd =3D qemuSaveImageOpen(driver, NULL, path, &def, &data, - (flags & VIR_DOMAIN_SAVE_BYPASS_CACHE) !=3D 0, - &wrapperFd, false, false); - if (fd < 0) + if (flags & VIR_DOMAIN_SAVE_BYPASS_CACHE) { + if (virFileDirectFdFlag() < 0) { + virReportError(VIR_ERR_OPERATION_FAILED, "%s", + _("bypass cache unsupported by this system")); + return -1; + } + oflags |=3D O_DIRECT; + } + if (virQEMUSaveFdInit(&saveFd, path, oflags, cfg) < 0) + return -1; + if (qemuSaveImageOpen(driver, NULL, &def, &data, false, &saveFd) < 0) goto cleanup; =20 if (ensureACL(conn, def) < 0) @@ -5924,16 +5932,13 @@ qemuDomainRestoreInternal(virConnectPtr conn, flags) < 0) goto cleanup; =20 - ret =3D qemuSaveImageStartVM(conn, driver, vm, &fd, data, path, + ret =3D qemuSaveImageStartVM(conn, driver, vm, &saveFd.fd, data, path, false, reset_nvram, VIR_ASYNC_JOB_START); =20 qemuProcessEndJob(vm); =20 cleanup: - VIR_FORCE_CLOSE(fd); - if (virFileWrapperFdClose(wrapperFd) < 0) - ret =3D -1; - virFileWrapperFdFree(wrapperFd); + ret =3D virQEMUSaveFdFini(&saveFd, vm, ret); virQEMUSaveDataFree(data); if (vm && ret < 0) qemuDomainRemoveInactive(driver, vm); @@ -5999,15 +6004,15 @@ qemuDomainSaveImageGetXMLDesc(virConnectPtr conn, c= onst char *path, virQEMUDriver *driver =3D conn->privateData; char *ret =3D NULL; g_autoptr(virDomainDef) def =3D NULL; - int fd =3D -1; virQEMUSaveData *data =3D NULL; + g_autoptr(virQEMUDriverConfig) cfg =3D virQEMUDriverGetConfig(driver); + virQEMUSaveFd saveFd =3D QEMU_SAVEFD_INVALID; =20 virCheckFlags(VIR_DOMAIN_SAVE_IMAGE_XML_SECURE, NULL); =20 - fd =3D qemuSaveImageOpen(driver, NULL, path, &def, &data, - false, NULL, false, false); - - if (fd < 0) + if (virQEMUSaveFdInit(&saveFd, path, O_RDONLY, cfg) < 0) + return NULL; + if (qemuSaveImageOpen(driver, NULL, &def, &data, false, &saveFd) < 0) goto cleanup; =20 if (virDomainSaveImageGetXMLDescEnsureACL(conn, def) < 0) @@ -6017,7 +6022,8 @@ qemuDomainSaveImageGetXMLDesc(virConnectPtr conn, con= st char *path, =20 cleanup: virQEMUSaveDataFree(data); - VIR_FORCE_CLOSE(fd); + if (virQEMUSaveFdFini(&saveFd, NULL, ret ? 0 : -1) < 0) + ret =3D NULL; return ret; } =20 @@ -6029,8 +6035,9 @@ qemuDomainSaveImageDefineXML(virConnectPtr conn, cons= t char *path, int ret =3D -1; g_autoptr(virDomainDef) def =3D NULL; g_autoptr(virDomainDef) newdef =3D NULL; - int fd =3D -1; virQEMUSaveData *data =3D NULL; + virQEMUSaveFd saveFd =3D QEMU_SAVEFD_INVALID; + g_autoptr(virQEMUDriverConfig) cfg =3D virQEMUDriverGetConfig(driver); int state =3D -1; =20 virCheckFlags(VIR_DOMAIN_SAVE_RUNNING | @@ -6041,10 +6048,9 @@ qemuDomainSaveImageDefineXML(virConnectPtr conn, con= st char *path, else if (flags & VIR_DOMAIN_SAVE_PAUSED) state =3D 0; =20 - fd =3D qemuSaveImageOpen(driver, NULL, path, &def, &data, - false, NULL, true, false); - - if (fd < 0) + if (virQEMUSaveFdInit(&saveFd, path, O_RDWR, cfg) < 0) + return -1; + if (qemuSaveImageOpen(driver, NULL, &def, &data, false, &saveFd) < 0) goto cleanup; =20 if (virDomainSaveImageDefineXMLEnsureACL(conn, def) < 0) @@ -6071,15 +6077,15 @@ qemuDomainSaveImageDefineXML(virConnectPtr conn, co= nst char *path, VIR_DOMAIN_XML_MIGRATABLE))) goto cleanup; =20 - if (lseek(fd, 0, SEEK_SET) !=3D 0) { + if (lseek(saveFd.fd, 0, SEEK_SET) !=3D 0) { virReportSystemError(errno, _("cannot seek in '%s'"), path); goto cleanup; } =20 - if (virQEMUSaveDataWrite(data, fd, path) < 0) + if (virQEMUSaveDataWrite(data, saveFd.fd, path) < 0) goto cleanup; =20 - if (VIR_CLOSE(fd) < 0) { + if (virQEMUSaveFdClose(&saveFd, NULL) < 0) { virReportSystemError(errno, _("failed to write header data to '%s'= "), path); goto cleanup; } @@ -6087,8 +6093,8 @@ qemuDomainSaveImageDefineXML(virConnectPtr conn, cons= t char *path, ret =3D 0; =20 cleanup: - VIR_FORCE_CLOSE(fd); virQEMUSaveDataFree(data); + ret =3D virQEMUSaveFdFini(&saveFd, NULL, ret); return ret; } =20 @@ -6100,8 +6106,9 @@ qemuDomainManagedSaveGetXMLDesc(virDomainPtr dom, uns= igned int flags) g_autofree char *path =3D NULL; char *ret =3D NULL; g_autoptr(virDomainDef) def =3D NULL; - int fd =3D -1; virQEMUSaveData *data =3D NULL; + virQEMUSaveFd saveFd =3D QEMU_SAVEFD_INVALID; + g_autoptr(virQEMUDriverConfig) cfg =3D virQEMUDriverGetConfig(driver); qemuDomainObjPrivate *priv; =20 virCheckFlags(VIR_DOMAIN_SAVE_IMAGE_XML_SECURE, NULL); @@ -6122,15 +6129,18 @@ qemuDomainManagedSaveGetXMLDesc(virDomainPtr dom, u= nsigned int flags) goto cleanup; } =20 - if ((fd =3D qemuSaveImageOpen(driver, priv->qemuCaps, path, &def, &dat= a, - false, NULL, false, false)) < 0) + if (virQEMUSaveFdInit(&saveFd, path, O_RDONLY, cfg) < 0) + goto cleanup; + if (qemuSaveImageOpen(driver, priv->qemuCaps, &def, &data, false, + &saveFd) < 0) goto cleanup; =20 ret =3D qemuDomainDefFormatXML(driver, priv->qemuCaps, def, flags); =20 cleanup: virQEMUSaveDataFree(data); - VIR_FORCE_CLOSE(fd); + if (virQEMUSaveFdFini(&saveFd, vm, ret ? 0 : -1) < 0) + ret =3D NULL; virDomainObjEndAPI(&vm); return ret; } @@ -6180,20 +6190,30 @@ qemuDomainObjRestore(virConnectPtr conn, { g_autoptr(virDomainDef) def =3D NULL; qemuDomainObjPrivate *priv =3D vm->privateData; - int fd =3D -1; int ret =3D -1; g_autofree char *xmlout =3D NULL; virQEMUSaveData *data =3D NULL; - virFileWrapperFd *wrapperFd =3D NULL; + virQEMUSaveFd saveFd =3D QEMU_SAVEFD_INVALID; + int oflags =3D O_RDONLY; + g_autoptr(virQEMUDriverConfig) cfg =3D virQEMUDriverGetConfig(driver); =20 - fd =3D qemuSaveImageOpen(driver, NULL, path, &def, &data, - bypass_cache, &wrapperFd, false, true); - if (fd < 0) { - if (fd =3D=3D -3) + if (bypass_cache) { + if (virFileDirectFdFlag() < 0) { + virReportError(VIR_ERR_OPERATION_FAILED, "%s", + _("bypass cache unsupported by this system")); + return -1; + } + oflags |=3D O_DIRECT; + } + if (virQEMUSaveFdInit(&saveFd, path, oflags, cfg) < 0) + goto cleanup; + ret =3D qemuSaveImageOpen(driver, NULL, &def, &data, true, &saveFd); + if (ret < 0) { + if (ret =3D=3D -3) ret =3D 1; goto cleanup; } - + ret =3D -1; if (virHookPresent(VIR_HOOK_DRIVER_QEMU)) { int hookret; =20 @@ -6233,15 +6253,12 @@ qemuDomainObjRestore(virConnectPtr conn, =20 virDomainObjAssignDef(vm, &def, true, NULL); =20 - ret =3D qemuSaveImageStartVM(conn, driver, vm, &fd, data, path, + ret =3D qemuSaveImageStartVM(conn, driver, vm, &saveFd.fd, data, path, start_paused, reset_nvram, asyncJob); =20 cleanup: virQEMUSaveDataFree(data); - VIR_FORCE_CLOSE(fd); - if (virFileWrapperFdClose(wrapperFd) < 0) - ret =3D -1; - virFileWrapperFdFree(wrapperFd); + ret =3D virQEMUSaveFdFini(&saveFd, vm, ret); return ret; } =20 diff --git a/src/qemu/qemu_saveimage.c b/src/qemu/qemu_saveimage.c index 2a58bd23b8..dee87ecfad 100644 --- a/src/qemu/qemu_saveimage.c +++ b/src/qemu/qemu_saveimage.c @@ -659,61 +659,30 @@ qemuSaveImageGetCompressionProgram(const char *imageF= ormat, * @path: path of the save image * @ret_def: returns domain definition created from the XML stored in the = image * @ret_data: returns structure filled with data from the image header - * @bypass_cache: bypass cache when opening the file - * @wrapperFd: returns the file wrapper structure - * @open_write: open the file for writing (for updates) - * @unlink_corrupt: remove the image file if it is corrupted + * @unlink_corrupt: mark the image file for removal if it is corrupted + * @saveFd: the save file * - * Returns the opened fd of the save image file and fills the appropriate = fields - * on success. On error returns -1 on most failures, -3 if corrupt image w= as - * unlinked (no error raised). + * Returns 0 on success or -1 on failure. + * -3 is a special failure in which the saveFd has been marked for unlinki= ng. + * On success, the appropriate fields are filled. */ int qemuSaveImageOpen(virQEMUDriver *driver, virQEMUCaps *qemuCaps, - const char *path, virDomainDef **ret_def, virQEMUSaveData **ret_data, - bool bypass_cache, - virFileWrapperFd **wrapperFd, - bool open_write, - bool unlink_corrupt) + bool unlink_corrupt, + virQEMUSaveFd *saveFd) { - g_autoptr(virQEMUDriverConfig) cfg =3D virQEMUDriverGetConfig(driver); - VIR_AUTOCLOSE fd =3D -1; - int ret =3D -1; + int ret; g_autoptr(virQEMUSaveData) data =3D NULL; g_autoptr(virDomainDef) def =3D NULL; - int oflags =3D open_write ? O_RDWR : O_RDONLY; - - if (bypass_cache) { - int directFlag =3D virFileDirectFdFlag(); - if (directFlag < 0) { - virReportError(VIR_ERR_OPERATION_FAILED, "%s", - _("bypass cache unsupported by this system")); - return -1; - } - oflags |=3D directFlag; - } - - if ((fd =3D qemuDomainOpenFile(cfg, NULL, path, oflags, NULL)) < 0) - return -1; - - if (bypass_cache && - !(*wrapperFd =3D virFileWrapperFdNew(&fd, path, - VIR_FILE_WRAPPER_BYPASS_CACHE))) - return -1; =20 data =3D g_new0(virQEMUSaveData, 1); - ret =3D virQEMUSaveDataRead(data, fd, path); + ret =3D virQEMUSaveDataRead(data, saveFd->fd, saveFd->path); if (ret < 0) { if (unlink_corrupt && ret =3D=3D -3) { - if (unlink(path) < 0) { - virReportSystemError(errno, - _("cannot remove corrupt file: %s"), - path); - return -1; - } + saveFd->need_unlink =3D true; } return ret; } @@ -727,10 +696,7 @@ qemuSaveImageOpen(virQEMUDriver *driver, *ret_def =3D g_steal_pointer(&def); *ret_data =3D g_steal_pointer(&data); =20 - ret =3D fd; - fd =3D -1; - - return ret; + return 0; } =20 int diff --git a/src/qemu/qemu_saveimage.h b/src/qemu/qemu_saveimage.h index 5768e31fa6..6db4e120cb 100644 --- a/src/qemu/qemu_saveimage.h +++ b/src/qemu/qemu_saveimage.h @@ -97,14 +97,11 @@ qemuSaveImageStartVM(virConnectPtr conn, int qemuSaveImageOpen(virQEMUDriver *driver, virQEMUCaps *qemuCaps, - const char *path, virDomainDef **ret_def, virQEMUSaveData **ret_data, - bool bypass_cache, - virFileWrapperFd **wrapperFd, - bool open_write, - bool unlink_corrupt) - ATTRIBUTE_NONNULL(3) ATTRIBUTE_NONNULL(4); + bool unlink_corrupt, + virQEMUSaveFd *saveFd) + ATTRIBUTE_NONNULL(1) ATTRIBUTE_NONNULL(3) ATTRIBUTE_NONNULL(6); =20 int qemuSaveImageGetCompressionProgram(const char *imageFormat, --=20 2.26.2 From nobody Fri May 17 11:58:59 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 170.10.133.124 as permitted sender) client-ip=170.10.133.124; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-delivery-124.mimecast.com; Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=fail(p=none dis=none) header.from=suse.de Return-Path: Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by mx.zohomail.com with SMTPS id 1654593668017493.10993255267954; Tue, 7 Jun 2022 02:21:08 -0700 (PDT) Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-630-9-aiQrLcNValN0MNXlIUNg-1; Tue, 07 Jun 2022 05:20:03 -0400 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.rdu2.redhat.com [10.11.54.2]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 4F0771C08DA4; Tue, 7 Jun 2022 09:19:51 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (unknown [10.30.29.100]) by smtp.corp.redhat.com (Postfix) with ESMTP id 3184D40EC008; Tue, 7 Jun 2022 09:19:51 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (localhost [IPv6:::1]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id 957B81947BA1; Tue, 7 Jun 2022 09:19:50 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx10.intmail.prod.int.rdu2.redhat.com [10.11.54.10]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id 88CBC1947B92 for ; Tue, 7 Jun 2022 09:19:48 +0000 (UTC) Received: by smtp.corp.redhat.com (Postfix) id 590A141637E; Tue, 7 Jun 2022 09:19:48 +0000 (UTC) Received: from mimecast-mx02.redhat.com (mimecast03.extmail.prod.ext.rdu2.redhat.com [10.11.55.19]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 551BC40336A for ; Tue, 7 Jun 2022 09:19:48 +0000 (UTC) Received: from us-smtp-1.mimecast.com (us-smtp-2.mimecast.com [207.211.31.81]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 3EC07811E83 for ; Tue, 7 Jun 2022 09:19:48 +0000 (UTC) Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.220.28]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-647-zszrR4HZPFqC9Q3vTU4Jbg-1; Tue, 07 Jun 2022 05:19:45 -0400 Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id E25AD21B57; Tue, 7 Jun 2022 09:19:43 +0000 (UTC) Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id A904213638; Tue, 7 Jun 2022 09:19:43 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id eJzDJy8Yn2IoLwAAMHmgww (envelope-from ); Tue, 07 Jun 2022 09:19:43 +0000 X-MC-Unique: 9-aiQrLcNValN0MNXlIUNg-1 X-Original-To: libvir-list@listman.corp.redhat.com X-MC-Unique: zszrR4HZPFqC9Q3vTU4Jbg-1 From: Claudio Fontana To: =?UTF-8?q?Daniel=20P=20=2E=20Berrang=C3=A9?= Subject: [libvirt RFCv11 09/33] tools: prepare doSave to use parameters Date: Tue, 7 Jun 2022 11:19:12 +0200 Message-Id: <20220607091936.7948-10-cfontana@suse.de> In-Reply-To: <20220607091936.7948-1-cfontana@suse.de> References: <20220607091936.7948-1-cfontana@suse.de> MIME-Version: 1.0 X-Mimecast-Impersonation-Protect: Policy=CLT - Impersonation Protection Definition; Similar Internal Domain=false; Similar Monitored External Domain=false; Custom External Domain=false; Mimecast External Domain=false; Newly Observed Domain=false; Internal User Name=false; Custom Display Name List=false; Reply-to Address Mismatch=false; Targeted Threat Dictionary=false; Mimecast Threat Dictionary=false; Custom Threat Dictionary=false X-Scanned-By: MIMEDefang 2.85 on 10.11.54.10 X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: libvir-list@redhat.com, Claudio Fontana , "Dr . David Alan Gilbert" , Anthony Iliopoulos Errors-To: libvir-list-bounces@redhat.com Sender: "libvir-list" X-Scanned-By: MIMEDefang 2.84 on 10.11.54.2 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=libvir-list-bounces@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable X-ZM-MESSAGEID: 1654593669813100003 Content-Type: text/plain; charset="utf-8"; x-default="true" Signed-off-by: Claudio Fontana --- tools/virsh-domain.c | 23 +++++++++++++++++------ 1 file changed, 17 insertions(+), 6 deletions(-) diff --git a/tools/virsh-domain.c b/tools/virsh-domain.c index bac4cc0fb6..120d0ef5f0 100644 --- a/tools/virsh-domain.c +++ b/tools/virsh-domain.c @@ -4203,6 +4203,9 @@ doSave(void *opaque) g_autoptr(virshDomain) dom =3D NULL; const char *name =3D NULL; const char *to =3D NULL; + virTypedParameterPtr params =3D NULL; + int nparams =3D 0; + int maxparams =3D 0; unsigned int flags =3D 0; const char *xmlfile =3D NULL; g_autofree char *xml =3D NULL; @@ -4216,9 +4219,12 @@ doSave(void *opaque) goto out_sig; #endif /* !WIN32 */ =20 - if (vshCommandOptStringReq(ctl, cmd, "file", &to) < 0) + if (vshCommandOptStringReq(ctl, cmd, "file", &to) < 0) { goto out; - + } else if (virTypedParamsAddString(¶ms, &nparams, &maxparams, + VIR_DOMAIN_SAVE_PARAM_FILE, to) < 0= ) { + goto out; + } if (vshCommandOptBool(cmd, "bypass-cache")) flags |=3D VIR_DOMAIN_SAVE_BYPASS_CACHE; if (vshCommandOptBool(cmd, "running")) @@ -4232,10 +4238,14 @@ doSave(void *opaque) if (!(dom =3D virshCommandOptDomain(ctl, cmd, &name))) goto out; =20 - if (xmlfile && - virFileReadAll(xmlfile, VSH_MAX_XML_FILE, &xml) < 0) { - vshReportError(ctl); - goto out; + if (xmlfile) { + if (virFileReadAll(xmlfile, VSH_MAX_XML_FILE, &xml) < 0) { + vshReportError(ctl); + goto out; + } else if (virTypedParamsAddString(¶ms, &nparams, &maxparams, + VIR_DOMAIN_SAVE_PARAM_DXML, xml= ) < 0) { + goto out; + } } =20 if (flags || xml) { @@ -4252,6 +4262,7 @@ doSave(void *opaque) data->ret =3D 0; =20 out: + virTypedParamsFree(params, nparams); #ifndef WIN32 pthread_sigmask(SIG_SETMASK, &oldsigmask, NULL); out_sig: --=20 2.26.2 From nobody Fri May 17 11:58:59 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 170.10.129.124 as permitted sender) client-ip=170.10.129.124; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-delivery-124.mimecast.com; Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=fail(p=none dis=none) header.from=suse.de Return-Path: Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by mx.zohomail.com with SMTPS id 1654593613465295.41597892680386; Tue, 7 Jun 2022 02:20:13 -0700 (PDT) Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-513-PNN1J02NPGCRUg7zbQudCA-1; Tue, 07 Jun 2022 05:20:07 -0400 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.rdu2.redhat.com [10.11.54.1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id A59AE294EDEA; Tue, 7 Jun 2022 09:19:50 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (unknown [10.30.29.100]) by smtp.corp.redhat.com (Postfix) with ESMTP id 8A70340CFD10; Tue, 7 Jun 2022 09:19:50 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (localhost [IPv6:::1]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id 089B41947BA5; Tue, 7 Jun 2022 09:19:50 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id 07B251947B82 for ; Tue, 7 Jun 2022 09:19:48 +0000 (UTC) Received: by smtp.corp.redhat.com (Postfix) id E11492026D07; Tue, 7 Jun 2022 09:19:47 +0000 (UTC) Received: from mimecast-mx02.redhat.com (mimecast07.extmail.prod.ext.rdu2.redhat.com [10.11.55.23]) by smtp.corp.redhat.com (Postfix) with ESMTPS id DC4C02026D64 for ; Tue, 7 Jun 2022 09:19:47 +0000 (UTC) Received: from us-smtp-1.mimecast.com (us-smtp-2.mimecast.com [207.211.31.81]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id C5A3B3C8ACF2 for ; Tue, 7 Jun 2022 09:19:47 +0000 (UTC) Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.220.28]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-225-4Q_KasWQPZGDoaRRp96fRQ-1; Tue, 07 Jun 2022 05:19:45 -0400 Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id 317B421B2F; Tue, 7 Jun 2022 09:19:44 +0000 (UTC) Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id EB22C13638; Tue, 7 Jun 2022 09:19:43 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id qELENy8Yn2IoLwAAMHmgww (envelope-from ); Tue, 07 Jun 2022 09:19:43 +0000 X-MC-Unique: PNN1J02NPGCRUg7zbQudCA-1 X-Original-To: libvir-list@listman.corp.redhat.com X-MC-Unique: 4Q_KasWQPZGDoaRRp96fRQ-1 From: Claudio Fontana To: =?UTF-8?q?Daniel=20P=20=2E=20Berrang=C3=A9?= Subject: [libvirt RFCv11 10/33] tools: prepare cmdRestore to use parameters Date: Tue, 7 Jun 2022 11:19:13 +0200 Message-Id: <20220607091936.7948-11-cfontana@suse.de> In-Reply-To: <20220607091936.7948-1-cfontana@suse.de> References: <20220607091936.7948-1-cfontana@suse.de> MIME-Version: 1.0 X-Mimecast-Impersonation-Protect: Policy=CLT - Impersonation Protection Definition; Similar Internal Domain=false; Similar Monitored External Domain=false; Custom External Domain=false; Mimecast External Domain=false; Newly Observed Domain=false; Internal User Name=false; Custom Display Name List=false; Reply-to Address Mismatch=false; Targeted Threat Dictionary=false; Mimecast Threat Dictionary=false; Custom Threat Dictionary=false X-Scanned-By: MIMEDefang 2.78 on 10.11.54.4 X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: libvir-list@redhat.com, Claudio Fontana , "Dr . David Alan Gilbert" , Anthony Iliopoulos Errors-To: libvir-list-bounces@redhat.com Sender: "libvir-list" X-Scanned-By: MIMEDefang 2.84 on 10.11.54.1 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=libvir-list-bounces@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable X-ZM-MESSAGEID: 1654593613865100015 Content-Type: text/plain; charset="utf-8"; x-default="true" Signed-off-by: Claudio Fontana --- tools/virsh-domain.c | 32 ++++++++++++++++++++++---------- 1 file changed, 22 insertions(+), 10 deletions(-) diff --git a/tools/virsh-domain.c b/tools/virsh-domain.c index 120d0ef5f0..a3f007a1d2 100644 --- a/tools/virsh-domain.c +++ b/tools/virsh-domain.c @@ -5316,15 +5316,21 @@ static bool cmdRestore(vshControl *ctl, const vshCmd *cmd) { const char *from =3D NULL; + virTypedParameterPtr params =3D NULL; + int nparams =3D 0; + int maxparams =3D 0; unsigned int flags =3D 0; const char *xmlfile =3D NULL; g_autofree char *xml =3D NULL; virshControl *priv =3D ctl->privData; - int rc; - - if (vshCommandOptStringReq(ctl, cmd, "file", &from) < 0) - return false; + int rc =3D -1; =20 + if (vshCommandOptStringReq(ctl, cmd, "file", &from) < 0) { + goto out; + } else if (virTypedParamsAddString(¶ms, &nparams, &maxparams, + VIR_DOMAIN_SAVE_PARAM_FILE, from) <= 0) { + goto out; + } if (vshCommandOptBool(cmd, "bypass-cache")) flags |=3D VIR_DOMAIN_SAVE_BYPASS_CACHE; if (vshCommandOptBool(cmd, "running")) @@ -5335,11 +5341,15 @@ cmdRestore(vshControl *ctl, const vshCmd *cmd) flags |=3D VIR_DOMAIN_SAVE_RESET_NVRAM; =20 if (vshCommandOptStringReq(ctl, cmd, "xml", &xmlfile) < 0) - return false; + goto out; =20 - if (xmlfile && - virFileReadAll(xmlfile, VSH_MAX_XML_FILE, &xml) < 0) - return false; + if (xmlfile) { + if (virFileReadAll(xmlfile, VSH_MAX_XML_FILE, &xml) < 0) + goto out; + else if (virTypedParamsAddString(¶ms, &nparams, &maxparams, + VIR_DOMAIN_SAVE_PARAM_DXML, xml) = < 0) + goto out; + } =20 if (flags || xml) { rc =3D virDomainRestoreFlags(priv->conn, from, xml, flags); @@ -5349,11 +5359,13 @@ cmdRestore(vshControl *ctl, const vshCmd *cmd) =20 if (rc < 0) { vshError(ctl, _("Failed to restore domain from %s"), from); - return false; + goto out; } =20 vshPrintExtra(ctl, _("Domain restored from %s\n"), from); - return true; + out: + virTypedParamsFree(params, nparams); + return rc >=3D 0; } =20 /* --=20 2.26.2 From nobody Fri May 17 11:58:59 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 170.10.129.124 as permitted sender) client-ip=170.10.129.124; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-delivery-124.mimecast.com; Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=fail(p=none dis=none) header.from=suse.de Return-Path: Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by mx.zohomail.com with SMTPS id 1654593610945154.4389293779136; Tue, 7 Jun 2022 02:20:10 -0700 (PDT) Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-461-rkotPY-8NQ2W1nFIRZqDZA-1; Tue, 07 Jun 2022 05:19:55 -0400 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 9B55510726B7; Tue, 7 Jun 2022 09:19:49 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (unknown [10.30.29.100]) by smtp.corp.redhat.com (Postfix) with ESMTP id 847892026D2D; Tue, 7 Jun 2022 09:19:49 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (localhost [IPv6:::1]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id 3FCE01947B83; Tue, 7 Jun 2022 09:19:49 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.rdu2.redhat.com [10.11.54.8]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id D41811947B82 for ; Tue, 7 Jun 2022 09:19:47 +0000 (UTC) Received: by smtp.corp.redhat.com (Postfix) id B4D06C27E92; Tue, 7 Jun 2022 09:19:47 +0000 (UTC) Received: from mimecast-mx02.redhat.com (mimecast03.extmail.prod.ext.rdu2.redhat.com [10.11.55.19]) by smtp.corp.redhat.com (Postfix) with ESMTPS id B12D6C27E8F for ; Tue, 7 Jun 2022 09:19:47 +0000 (UTC) Received: from us-smtp-1.mimecast.com (us-smtp-2.mimecast.com [207.211.31.81]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 993F08117B0 for ; Tue, 7 Jun 2022 09:19:47 +0000 (UTC) Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.220.29]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-319-NeDO6YC-PmOStzffAfoCfQ-1; Tue, 07 Jun 2022 05:19:45 -0400 Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id 811991F980; Tue, 7 Jun 2022 09:19:44 +0000 (UTC) Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 3A55E13638; Tue, 7 Jun 2022 09:19:44 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id yMKaDDAYn2IoLwAAMHmgww (envelope-from ); Tue, 07 Jun 2022 09:19:44 +0000 X-MC-Unique: rkotPY-8NQ2W1nFIRZqDZA-1 X-Original-To: libvir-list@listman.corp.redhat.com X-MC-Unique: NeDO6YC-PmOStzffAfoCfQ-1 From: Claudio Fontana To: =?UTF-8?q?Daniel=20P=20=2E=20Berrang=C3=A9?= Subject: [libvirt RFCv11 11/33] libvirt: add new VIR_DOMAIN_SAVE_PARALLEL flag and parameter Date: Tue, 7 Jun 2022 11:19:14 +0200 Message-Id: <20220607091936.7948-12-cfontana@suse.de> In-Reply-To: <20220607091936.7948-1-cfontana@suse.de> References: <20220607091936.7948-1-cfontana@suse.de> MIME-Version: 1.0 X-Mimecast-Impersonation-Protect: Policy=CLT - Impersonation Protection Definition; Similar Internal Domain=false; Similar Monitored External Domain=false; Custom External Domain=false; Mimecast External Domain=false; Newly Observed Domain=false; Internal User Name=false; Custom Display Name List=false; Reply-to Address Mismatch=false; Targeted Threat Dictionary=false; Mimecast Threat Dictionary=false; Custom Threat Dictionary=false X-Scanned-By: MIMEDefang 2.85 on 10.11.54.8 X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: libvir-list@redhat.com, Claudio Fontana , "Dr . David Alan Gilbert" , Anthony Iliopoulos Errors-To: libvir-list-bounces@redhat.com Sender: "libvir-list" X-Scanned-By: MIMEDefang 2.78 on 10.11.54.4 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=libvir-list-bounces@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable X-ZM-MESSAGEID: 1654593611852100012 Content-Type: text/plain; charset="utf-8"; x-default="true" in order to enable parallel save functionality, we will need an opportune new flag and a parameter to specify the number of extra connections to use. Signed-off-by: Claudio Fontana --- include/libvirt/libvirt-domain.h | 13 +++++++++++++ 1 file changed, 13 insertions(+) diff --git a/include/libvirt/libvirt-domain.h b/include/libvirt/libvirt-dom= ain.h index 24846046aa..0b73ee27b6 100644 --- a/include/libvirt/libvirt-domain.h +++ b/include/libvirt/libvirt-domain.h @@ -1555,6 +1555,7 @@ typedef enum { VIR_DOMAIN_SAVE_RUNNING =3D 1 << 1, /* Favor running over paused = (Since: 0.9.5) */ VIR_DOMAIN_SAVE_PAUSED =3D 1 << 2, /* Favor paused over running = (Since: 0.9.5) */ VIR_DOMAIN_SAVE_RESET_NVRAM =3D 1 << 3, /* Re-initialize NVRAM from t= emplate (Since: 8.1.0) */ + VIR_DOMAIN_SAVE_PARALLEL =3D 1 << 4, /* Parallel Save/Restore (Sin= ce: 8.4.0) */ } virDomainSaveRestoreFlags; =20 int virDomainSave (virDomainPtr domain, @@ -1600,6 +1601,18 @@ int virDomainRestoreParams (vir= ConnectPtr conn, */ # define VIR_DOMAIN_SAVE_PARAM_DXML "dxml" =20 +/** + * VIR_DOMAIN_SAVE_PARAM_PARALLEL_CONNECTIONS: + * + * this optional parameter mirrors the migration parameter + * VIR_MIGRATE_PARAM_PARALLEL_CONNECTIONS. + * + * It specifies the number of extra connections to use when transfering st= ate. + * + * Since: 8.4.0 + */ +# define VIR_DOMAIN_SAVE_PARAM_PARALLEL_CONNECTIONS "parallel.connecti= ons" + /* See below for virDomainSaveImageXMLFlags */ char * virDomainSaveImageGetXMLDesc (virConnectPtr conn, const char *file, --=20 2.26.2 From nobody Fri May 17 11:58:59 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 170.10.133.124 as permitted sender) client-ip=170.10.133.124; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-delivery-124.mimecast.com; Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=fail(p=none dis=none) header.from=suse.de Return-Path: Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by mx.zohomail.com with SMTPS id 1654593618562712.2513887556639; Tue, 7 Jun 2022 02:20:18 -0700 (PDT) Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-656-by-cqFd4OfiXAoMDjKn-3Q-1; Tue, 07 Jun 2022 05:20:08 -0400 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 14E9F8339A0; Tue, 7 Jun 2022 09:19:57 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (unknown [10.30.29.100]) by smtp.corp.redhat.com (Postfix) with ESMTP id EF8932026D64; Tue, 7 Jun 2022 09:19:56 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (localhost [IPv6:::1]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id BD8F31947B83; Tue, 7 Jun 2022 09:19:56 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx10.intmail.prod.int.rdu2.redhat.com [10.11.54.10]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id 281DB19452D8 for ; Tue, 7 Jun 2022 09:19:53 +0000 (UTC) Received: by smtp.corp.redhat.com (Postfix) id 18248416144; Tue, 7 Jun 2022 09:19:53 +0000 (UTC) Received: from mimecast-mx02.redhat.com (mimecast10.extmail.prod.ext.rdu2.redhat.com [10.11.55.26]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 14102492C3B for ; Tue, 7 Jun 2022 09:19:53 +0000 (UTC) Received: from us-smtp-1.mimecast.com (us-smtp-1.mimecast.com [205.139.110.61]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id F08631C00AD4 for ; Tue, 7 Jun 2022 09:19:52 +0000 (UTC) Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.220.29]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-622-QTIgRbr_Mi6YAk4l3HvFvg-1; Tue, 07 Jun 2022 05:19:47 -0400 Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id BA0AB1F989; Tue, 7 Jun 2022 09:19:44 +0000 (UTC) Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 7EF3313B40; Tue, 7 Jun 2022 09:19:44 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id 2J9qHTAYn2IoLwAAMHmgww (envelope-from ); Tue, 07 Jun 2022 09:19:44 +0000 X-MC-Unique: by-cqFd4OfiXAoMDjKn-3Q-1 X-Original-To: libvir-list@listman.corp.redhat.com X-MC-Unique: QTIgRbr_Mi6YAk4l3HvFvg-1 From: Claudio Fontana To: =?UTF-8?q?Daniel=20P=20=2E=20Berrang=C3=A9?= Subject: [libvirt RFCv11 12/33] qemu: add stub support for VIR_DOMAIN_SAVE_PARALLEL in save Date: Tue, 7 Jun 2022 11:19:15 +0200 Message-Id: <20220607091936.7948-13-cfontana@suse.de> In-Reply-To: <20220607091936.7948-1-cfontana@suse.de> References: <20220607091936.7948-1-cfontana@suse.de> MIME-Version: 1.0 X-Mimecast-Impersonation-Protect: Policy=CLT - Impersonation Protection Definition; Similar Internal Domain=false; Similar Monitored External Domain=false; Custom External Domain=false; Mimecast External Domain=false; Newly Observed Domain=false; Internal User Name=false; Custom Display Name List=false; Reply-to Address Mismatch=false; Targeted Threat Dictionary=false; Mimecast Threat Dictionary=false; Custom Threat Dictionary=false X-Scanned-By: MIMEDefang 2.85 on 10.11.54.10 X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: libvir-list@redhat.com, Claudio Fontana , "Dr . David Alan Gilbert" , Anthony Iliopoulos Errors-To: libvir-list-bounces@redhat.com Sender: "libvir-list" X-Scanned-By: MIMEDefang 2.78 on 10.11.54.4 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=libvir-list-bounces@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable X-ZM-MESSAGEID: 1654593620390100006 Content-Type: text/plain; charset="utf-8"; x-default="true" and its companion param VIR_SAVE_PARAM_PARALLEL_CONNECTIONS Signed-off-by: Claudio Fontana --- src/qemu/qemu_driver.c | 18 ++++++++++++------ src/qemu/qemu_saveimage.c | 1 + src/qemu/qemu_saveimage.h | 1 + src/qemu/qemu_snapshot.c | 2 +- 4 files changed, 15 insertions(+), 7 deletions(-) diff --git a/src/qemu/qemu_driver.c b/src/qemu/qemu_driver.c index 9ad4945928..5a8d3d1309 100644 --- a/src/qemu/qemu_driver.c +++ b/src/qemu/qemu_driver.c @@ -2650,7 +2650,7 @@ static int qemuDomainSaveInternal(virQEMUDriver *driver, virDomainObj *vm, const char *path, int compressed, virCommand *compressor, - const char *xmlin, unsigned int flags) + const char *xmlin, int nconn, unsigned int flags) { g_autofree char *xml =3D NULL; bool was_running =3D false; @@ -2731,7 +2731,7 @@ qemuDomainSaveInternal(virQEMUDriver *driver, xml =3D NULL; =20 ret =3D qemuSaveImageCreate(driver, vm, path, data, compressor, - flags, VIR_ASYNC_JOB_SAVE); + nconn, flags, VIR_ASYNC_JOB_SAVE); if (ret < 0) goto endjob; =20 @@ -2809,7 +2809,7 @@ qemuDomainManagedSaveHelper(virQEMUDriver *driver, VIR_INFO("Saving state of domain '%s' to '%s'", vm->def->name, path); =20 if (qemuDomainSaveInternal(driver, vm, path, compressed, - compressor, dxml, flags) < 0) + compressor, dxml, -1, flags) < 0) return -1; =20 vm->hasManagedSave =3D true; @@ -2848,7 +2848,7 @@ qemuDomainSaveFlags(virDomainPtr dom, const char *pat= h, const char *dxml, goto cleanup; =20 ret =3D qemuDomainSaveInternal(driver, vm, path, compressed, - compressor, dxml, flags); + compressor, dxml, -1, flags); =20 cleanup: virDomainObjEndAPI(&vm); @@ -2875,16 +2875,20 @@ qemuDomainSaveParams(virDomainPtr dom, const char *dxml =3D NULL; int compressed; int ret =3D -1; + int nconn =3D 2; =20 virCheckFlags(VIR_DOMAIN_SAVE_BYPASS_CACHE | VIR_DOMAIN_SAVE_RUNNING | - VIR_DOMAIN_SAVE_PAUSED, -1); + VIR_DOMAIN_SAVE_PAUSED | + VIR_DOMAIN_SAVE_PARALLEL, -1); =20 if (virTypedParamsValidate(params, nparams, VIR_DOMAIN_SAVE_PARAM_FILE, VIR_TYPED_PARAM_STRING, VIR_DOMAIN_SAVE_PARAM_DXML, VIR_TYPED_PARAM_STRING, + VIR_DOMAIN_SAVE_PARAM_PARALLEL_CONNECTIONS, + VIR_TYPED_PARAM_INT, NULL) < 0) return -1; =20 @@ -2894,6 +2898,8 @@ qemuDomainSaveParams(virDomainPtr dom, if (virTypedParamsGetString(params, nparams, VIR_DOMAIN_SAVE_PARAM_DXML, &dxml) < 0) return -1; + if (virTypedParamsGetInt(params, nparams, VIR_DOMAIN_SAVE_PARAM_PARALL= EL_CONNECTIONS, &nconn) < 0) + return -1; =20 if (!(vm =3D qemuDomainObjFromDomain(dom))) goto cleanup; @@ -2916,7 +2922,7 @@ qemuDomainSaveParams(virDomainPtr dom, goto cleanup; =20 ret =3D qemuDomainSaveInternal(driver, vm, to, compressed, - compressor, dxml, flags); + compressor, dxml, nconn, flags); =20 cleanup: virDomainObjEndAPI(&vm); diff --git a/src/qemu/qemu_saveimage.c b/src/qemu/qemu_saveimage.c index dee87ecfad..70ded3c439 100644 --- a/src/qemu/qemu_saveimage.c +++ b/src/qemu/qemu_saveimage.c @@ -521,6 +521,7 @@ qemuSaveImageCreate(virQEMUDriver *driver, const char *path, virQEMUSaveData *data, virCommand *compressor, + int nconn G_GNUC_UNUSED, unsigned int flags, virDomainAsyncJob asyncJob) { diff --git a/src/qemu/qemu_saveimage.h b/src/qemu/qemu_saveimage.h index 6db4e120cb..9b04af2fb9 100644 --- a/src/qemu/qemu_saveimage.h +++ b/src/qemu/qemu_saveimage.h @@ -116,6 +116,7 @@ qemuSaveImageCreate(virQEMUDriver *driver, const char *path, virQEMUSaveData *data, virCommand *compressor, + int nconn, unsigned int flags, virDomainAsyncJob asyncJob); =20 diff --git a/src/qemu/qemu_snapshot.c b/src/qemu/qemu_snapshot.c index c6beeda1d0..3437c3e0cb 100644 --- a/src/qemu/qemu_snapshot.c +++ b/src/qemu/qemu_snapshot.c @@ -1457,7 +1457,7 @@ qemuSnapshotCreateActiveExternal(virQEMUDriver *drive= r, memory_existing =3D virFileExists(snapdef->memorysnapshotfile); =20 if ((ret =3D qemuSaveImageCreate(driver, vm, snapdef->memorysnapsh= otfile, - data, compressor, 0, + data, compressor, -1, 0, VIR_ASYNC_JOB_SNAPSHOT)) < 0) goto cleanup; =20 --=20 2.26.2 From nobody Fri May 17 11:58:59 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 170.10.133.124 as permitted sender) client-ip=170.10.133.124; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-delivery-124.mimecast.com; Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=fail(p=none dis=none) header.from=suse.de Return-Path: Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by mx.zohomail.com with SMTPS id 165459361965477.2038682493627; Tue, 7 Jun 2022 02:20:19 -0700 (PDT) Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-56-bfFdmVxpO86S5mU--WYW6w-1; Tue, 07 Jun 2022 05:20:11 -0400 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.rdu2.redhat.com [10.11.54.1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 86CA98027EE; Tue, 7 Jun 2022 09:19:54 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (unknown [10.30.29.100]) by smtp.corp.redhat.com (Postfix) with ESMTP id 72AD4406AD48; Tue, 7 Jun 2022 09:19:54 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (localhost [IPv6:::1]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id BB9EE1947B9C; Tue, 7 Jun 2022 09:19:53 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx10.intmail.prod.int.rdu2.redhat.com [10.11.54.10]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id F067319452D2 for ; Tue, 7 Jun 2022 09:19:51 +0000 (UTC) Received: by smtp.corp.redhat.com (Postfix) id C516E492C3B; Tue, 7 Jun 2022 09:19:51 +0000 (UTC) Received: from mimecast-mx02.redhat.com (mimecast08.extmail.prod.ext.rdu2.redhat.com [10.11.55.24]) by smtp.corp.redhat.com (Postfix) with ESMTPS id C052640336E for ; Tue, 7 Jun 2022 09:19:51 +0000 (UTC) Received: from us-smtp-1.mimecast.com (us-smtp-2.mimecast.com [205.139.110.61]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id AA6233804538 for ; Tue, 7 Jun 2022 09:19:51 +0000 (UTC) Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.220.29]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-202-NKfjH4tPOLSr43SYtHEH4Q-1; Tue, 07 Jun 2022 05:19:48 -0400 Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id 142EF1F98B; Tue, 7 Jun 2022 09:19:45 +0000 (UTC) Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id C4AB213638; Tue, 7 Jun 2022 09:19:44 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id sINrLjAYn2IoLwAAMHmgww (envelope-from ); Tue, 07 Jun 2022 09:19:44 +0000 X-MC-Unique: bfFdmVxpO86S5mU--WYW6w-1 X-Original-To: libvir-list@listman.corp.redhat.com X-MC-Unique: NKfjH4tPOLSr43SYtHEH4Q-1 From: Claudio Fontana To: =?UTF-8?q?Daniel=20P=20=2E=20Berrang=C3=A9?= Subject: [libvirt RFCv11 13/33] qemu: add stub support for VIR_DOMAIN_SAVE_PARALLEL in restore Date: Tue, 7 Jun 2022 11:19:16 +0200 Message-Id: <20220607091936.7948-14-cfontana@suse.de> In-Reply-To: <20220607091936.7948-1-cfontana@suse.de> References: <20220607091936.7948-1-cfontana@suse.de> MIME-Version: 1.0 X-Mimecast-Impersonation-Protect: Policy=CLT - Impersonation Protection Definition; Similar Internal Domain=false; Similar Monitored External Domain=false; Custom External Domain=false; Mimecast External Domain=false; Newly Observed Domain=false; Internal User Name=false; Custom Display Name List=false; Reply-to Address Mismatch=false; Targeted Threat Dictionary=false; Mimecast Threat Dictionary=false; Custom Threat Dictionary=false X-Scanned-By: MIMEDefang 2.85 on 10.11.54.10 X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: libvir-list@redhat.com, Claudio Fontana , "Dr . David Alan Gilbert" , Anthony Iliopoulos Errors-To: libvir-list-bounces@redhat.com Sender: "libvir-list" X-Scanned-By: MIMEDefang 2.84 on 10.11.54.1 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=libvir-list-bounces@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable X-ZM-MESSAGEID: 1654593620390100005 Content-Type: text/plain; charset="utf-8"; x-default="true" Signed-off-by: Claudio Fontana --- src/qemu/qemu_driver.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/src/qemu/qemu_driver.c b/src/qemu/qemu_driver.c index 5a8d3d1309..986bbf9e58 100644 --- a/src/qemu/qemu_driver.c +++ b/src/qemu/qemu_driver.c @@ -5869,7 +5869,8 @@ qemuDomainRestoreInternal(virConnectPtr conn, virCheckFlags(VIR_DOMAIN_SAVE_BYPASS_CACHE | VIR_DOMAIN_SAVE_RUNNING | VIR_DOMAIN_SAVE_PAUSED | - VIR_DOMAIN_SAVE_RESET_NVRAM, -1); + VIR_DOMAIN_SAVE_RESET_NVRAM | + VIR_DOMAIN_SAVE_PARALLEL, -1); =20 if (flags & VIR_DOMAIN_SAVE_RESET_NVRAM) reset_nvram =3D true; --=20 2.26.2 From nobody Fri May 17 11:58:59 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 170.10.129.124 as permitted sender) client-ip=170.10.129.124; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-delivery-124.mimecast.com; Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=fail(p=none dis=none) header.from=suse.de Return-Path: Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by mx.zohomail.com with SMTPS id 1654593666167343.9430320041952; Tue, 7 Jun 2022 02:21:06 -0700 (PDT) Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-515-h3lrXgQEOSuuUNEeP-KaNw-1; Tue, 07 Jun 2022 05:21:00 -0400 Received: from smtp.corp.redhat.com (int-mx10.intmail.prod.int.rdu2.redhat.com [10.11.54.10]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 60C8580B71A; Tue, 7 Jun 2022 09:20:58 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (unknown [10.30.29.100]) by smtp.corp.redhat.com (Postfix) with ESMTP id 4667F492C3B; Tue, 7 Jun 2022 09:20:58 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (localhost [IPv6:::1]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id C436C19451EF; Tue, 7 Jun 2022 09:20:57 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.rdu2.redhat.com [10.11.54.8]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id 2917119452D2 for ; Tue, 7 Jun 2022 09:20:56 +0000 (UTC) Received: by smtp.corp.redhat.com (Postfix) id 1B396C23DBF; Tue, 7 Jun 2022 09:20:56 +0000 (UTC) Received: from mimecast-mx02.redhat.com (mimecast06.extmail.prod.ext.rdu2.redhat.com [10.11.55.22]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 174E5C15E72 for ; Tue, 7 Jun 2022 09:20:56 +0000 (UTC) Received: from us-smtp-1.mimecast.com (us-smtp-delivery-1.mimecast.com [205.139.110.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id EFEB5185A7A4 for ; Tue, 7 Jun 2022 09:20:55 +0000 (UTC) Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.220.29]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-588-IzYDd_lWO1CCyoIBLabqgA-1; Tue, 07 Jun 2022 05:20:54 -0400 Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id 52C171F8C3; Tue, 7 Jun 2022 09:19:45 +0000 (UTC) Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 1CE1413638; Tue, 7 Jun 2022 09:19:45 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id 0L9qBTEYn2IoLwAAMHmgww (envelope-from ); Tue, 07 Jun 2022 09:19:45 +0000 X-MC-Unique: h3lrXgQEOSuuUNEeP-KaNw-1 X-Original-To: libvir-list@listman.corp.redhat.com X-MC-Unique: IzYDd_lWO1CCyoIBLabqgA-1 From: Claudio Fontana To: =?UTF-8?q?Daniel=20P=20=2E=20Berrang=C3=A9?= Subject: [libvirt RFCv11 14/33] virfile: add new API virFileDiskCopyChannel Date: Tue, 7 Jun 2022 11:19:17 +0200 Message-Id: <20220607091936.7948-15-cfontana@suse.de> In-Reply-To: <20220607091936.7948-1-cfontana@suse.de> References: <20220607091936.7948-1-cfontana@suse.de> MIME-Version: 1.0 X-Mimecast-Impersonation-Protect: Policy=CLT - Impersonation Protection Definition; Similar Internal Domain=false; Similar Monitored External Domain=false; Custom External Domain=false; Mimecast External Domain=false; Newly Observed Domain=false; Internal User Name=false; Custom Display Name List=false; Reply-to Address Mismatch=false; Targeted Threat Dictionary=false; Mimecast Threat Dictionary=false; Custom Threat Dictionary=false X-Scanned-By: MIMEDefang 2.85 on 10.11.54.8 X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: libvir-list@redhat.com, Claudio Fontana , "Dr . David Alan Gilbert" , Anthony Iliopoulos Errors-To: libvir-list-bounces@redhat.com Sender: "libvir-list" X-Scanned-By: MIMEDefang 2.85 on 10.11.54.10 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=libvir-list-bounces@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable X-ZM-MESSAGEID: 1654593667855100001 Content-Type: text/plain; charset="utf-8"; x-default="true" allow interleaved parallel write to a single file, using a record size equal to the io buffer size (1MB). Signed-off-by: Claudio Fontana --- src/util/iohelper.c | 3 + src/util/virfile.c | 151 +++++++++++++++++++++++++++++--------------- src/util/virfile.h | 2 + 3 files changed, 106 insertions(+), 50 deletions(-) diff --git a/src/util/iohelper.c b/src/util/iohelper.c index 055540c8c4..dcbdda366f 100644 --- a/src/util/iohelper.c +++ b/src/util/iohelper.c @@ -85,6 +85,9 @@ main(int argc, char **argv) if (fd < 0 || virFileDiskCopy(fd, path, -1, "stdio") < 0) goto error; =20 + if (VIR_CLOSE(fd) < 0) + goto error; + return 0; =20 error: diff --git a/src/util/virfile.c b/src/util/virfile.c index 201d7f4e64..f9ae7d94c4 100644 --- a/src/util/virfile.c +++ b/src/util/virfile.c @@ -4761,6 +4761,9 @@ struct runIOParams { const char *fdinname; int fdout; const char *fdoutname; + int idx; + int nchannels; + off_t total; }; =20 /** @@ -4779,12 +4782,18 @@ runIOCopy(const struct runIOParams p) off_t total =3D 0; size_t buflen =3D 1024*1024; char *buf =3D virFileDirectBufferNew(&base, buflen); + int diskfd =3D p.isWrite ? p.fdout : p.fdin; =20 if (!buf) { virReportSystemError(errno, _("Failed to allocate aligned memory i= n function %s"), __FUNCTION__); return -5; } - + if (p.idx >=3D 0) { + if (lseek(diskfd, p.idx * buflen, SEEK_CUR) < 0) { + virReportSystemError(errno, "%s", _("Failed to lseek to file c= hannel offset")); + return -6; + } + } while (1) { ssize_t got; =20 @@ -4808,7 +4817,12 @@ runIOCopy(const struct runIOParams p) break; =20 total +=3D got; - + if (p.idx >=3D 0 && !p.isWrite && total > p.total) { + /* do not write to socket too much for this channel, according= to CLIA */ + off_t difference =3D total - p.total; + got -=3D difference; + total -=3D difference; + } /* handle last write size align in direct case */ if (got < buflen && p.isDirect && p.isWrite) { ssize_t nwritten =3D virFileDirectWrite(p.fdout, buf, got); @@ -4816,7 +4830,7 @@ runIOCopy(const struct runIOParams p) virReportSystemError(errno, _("Unable to write %s"), p.fdo= utname); return -3; } - if (!p.isBlockDev) { + if (!p.isBlockDev && p.idx < 0) { off_t off =3D lseek(p.fdout, (off_t)0, SEEK_CUR); if (off < 0) { virReportSystemError(errno, "%s", _("Failed to lseek t= o get current file offset")); @@ -4824,6 +4838,7 @@ runIOCopy(const struct runIOParams p) } if (nwritten > got) { off -=3D nwritten - got; + total -=3D nwritten - got; } if (ftruncate(p.fdout, off) < 0) { virReportSystemError(errno, _("Unable to truncate %s")= , p.fdoutname); @@ -4838,51 +4853,61 @@ runIOCopy(const struct runIOParams p) virReportSystemError(errno, _("Unable to write %s"), p.fdoutna= me); return -3; } + if (p.idx >=3D 0) { + if (!p.isWrite && total >=3D p.total) { + /* done for this channel */ + break; + } + /* move channel cursor to the next record */ + if (lseek(diskfd, buflen * (p.nchannels - 1), SEEK_CUR) < 0) { + virReportSystemError(errno, "%s", _("Failed to lseek to ne= xt channel record")); + return -7; + } + } } return total; } =20 /** - * virFileDiskCopy: run IO to copy data between storage and a pipe or sock= et. - * - * @disk_fd: the already open regular file or block device - * @disk_path: the pathname corresponding to disk_fd (for error reportin= g) - * @remote_fd: the pipe or socket - * Use -1 to auto-choose between STDIN or STDOUT. - * @remote_path: the pathname corresponding to remote_fd (for error report= ing) - * - * Note that the direction of the transfer is detected based on the @disk_= fd - * file access mode (man 2 open). Therefore @disk_fd must be opened with - * O_RDONLY or O_WRONLY. O_RDWR is not supported. - * - * virFileDiskCopy always closes the file descriptor disk_fd, - * and any error during close(2) is reported and considered a failure. - * - * Returns: bytes transferred or < 0 on failure. + * virFileDiskCopyChannel: like virFileDiskCopy, channel interleaved read/= write + * ... + * @idx: channel index + * @nchannels: total number of channels */ =20 off_t -virFileDiskCopy(int disk_fd, const char *disk_path, int remote_fd, const c= har *remote_path) +virFileDiskCopyChannel(int disk_fd, const char *disk_path, int remote_fd, = const char *remote_path, + int idx, int nchannels, off_t total) { - int ret =3D -1; - off_t total =3D 0; + off_t new_total =3D -1; struct stat sb; struct runIOParams p; int oflags =3D -1; =20 + if ((nchannels =3D=3D 0) || + (nchannels > 0 && idx >=3D nchannels) || + (nchannels > 0 && idx < 0) || + (nchannels < 0 && idx >=3D 0)) { + virReportSystemError(EINVAL, "%s", _("Invalid channel arguments")); + goto out; + } + p.idx =3D idx; + p.nchannels =3D nchannels; + p.total =3D total; + oflags =3D fcntl(disk_fd, F_GETFL); =20 if (oflags < 0) { virReportSystemError(errno, _("unable to determine access mode of %s"), disk_path); - goto cleanup; + goto out; } if (fstat(disk_fd, &sb) < 0) { virReportSystemError(errno, _("unable to stat file descriptor %d path %s"= ), disk_fd, disk_path); - goto cleanup; + goto out; } p.isBlockDev =3D S_ISBLK(sb.st_mode); p.isDirect =3D O_DIRECT && (oflags & O_DIRECT); @@ -4906,53 +4931,79 @@ virFileDiskCopy(int disk_fd, const char *disk_path,= int remote_fd, const char *r default: virReportSystemError(EINVAL, _("Unable to process file with flags = %d"), (oflags & O_ACCMODE)); - goto cleanup; + goto out; } if (!p.isBlockDev && p.isDirect) { off_t off =3D lseek(disk_fd, 0, SEEK_CUR); if (off < 0) { virReportSystemError(errno, "%s", _("O_DIRECT needs a seekable= file")); - goto cleanup; + goto out; } if (virFileDirectAlign(off) !=3D off) { /* we could write some zeroes, but maybe it is safer to just f= ail */ virReportSystemError(EINVAL, "%s", _("O_DIRECT attempted with = unaligned file pointer")); - goto cleanup; + goto out; } } - total =3D runIOCopy(p); - if (total < 0) - goto cleanup; - - /* Ensure all data is written */ - if (virFileDataSync(p.fdout) < 0) { - if (errno !=3D EINVAL && errno !=3D EROFS) { - /* fdatasync() may fail on some special FDs, e.g. pipes */ - virReportSystemError(errno, _("unable to fsync %s"), p.fdoutna= me); - goto cleanup; + new_total =3D runIOCopy(p); + if (new_total < 0) + goto out; + + if (p.idx < 0 && p.isWrite) { + /* without channels we can run the fdatasync here */ + if (virFileDataSync(disk_fd) < 0) { + if (errno !=3D EINVAL && errno !=3D EROFS) { + virReportSystemError(errno, _("unable to fsyncdata %s"), p= .fdoutname); + new_total =3D -1; + goto out; + } } } =20 - ret =3D 0; - - cleanup: - if (VIR_CLOSE(disk_fd) < 0 && ret =3D=3D 0) { - virReportSystemError(errno, _("Unable to close %s"), disk_path); - ret =3D -1; - } - return ret; + out: + return new_total; } =20 #else /* WIN32 */ =20 off_t -virFileDiskCopy(int disk_fd G_GNUC_UNUSED, - const char *disk_path G_GNUC_UNUSED, - int remote_fd G_GNUC_UNUSED, - const char *remote_path G_GNUC_UNUSED) +virFileDiskCopyChannel(int disk_fd G_GNUC_UNUSED, + const char *disk_path G_GNUC_UNUSED, + int remote_fd G_GNUC_UNUSED, + const char *remote_path G_GNUC_UNUSED, + int idx G_GNUC_UNUSED, + int nchannels G_GNUC_UNUSED, + off_t total G_GNUC_UNUSED) { virReportError(VIR_ERR_INTERNAL_ERROR, "%s", - _("virFileDiskCopy unsupported on this platform")); + _("virFileDiskCopyChannel unsupported on this platform"= )); return -1; } #endif /* WIN32 */ + +/** + * virFileDiskCopy: run IO to copy data between storage and a pipe or sock= et. + * + * @disk_fd: the already open regular file or block device + * @disk_path: the pathname corresponding to disk_fd (for error reportin= g) + * @remote_fd: the pipe or socket + * Use -1 to auto-choose between STDIN or STDOUT. + * @remote_path: the pathname corresponding to remote_fd (for error report= ing) + * + * Note that the direction of the transfer is detected based on the @disk_= fd + * file access mode (man 2 open). Therefore @disk_fd must be opened with + * O_RDONLY or O_WRONLY. O_RDWR is not supported. + * + * virFileDiskCopy always closes the file descriptor disk_fd, + * and any error during close(2) is reported and considered a failure. + * + * Returns: bytes transferred or < 0 on failure. + */ + +off_t +virFileDiskCopy(int disk_fd, const char *disk_path, + int remote_fd, const char *remote_path) +{ + return virFileDiskCopyChannel(disk_fd, disk_path, remote_fd, remote_pa= th, + -1, -1, 0); +} diff --git a/src/util/virfile.h b/src/util/virfile.h index 844261e0a4..4d75389c84 100644 --- a/src/util/virfile.h +++ b/src/util/virfile.h @@ -394,3 +394,5 @@ int virFileSetCOW(const char *path, virTristateBool state); =20 off_t virFileDiskCopy(int disk_fd, const char *disk_path, int remote_fd, c= onst char *remote_path); +off_t virFileDiskCopyChannel(int disk_fd, const char *disk_path, int remot= e_fd, const char *remote_path, + int idx, int nchannels, off_t total); --=20 2.26.2 From nobody Fri May 17 11:58:59 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 170.10.133.124 as permitted sender) client-ip=170.10.133.124; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-delivery-124.mimecast.com; Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=fail(p=none dis=none) header.from=suse.de Return-Path: Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by mx.zohomail.com with SMTPS id 1654593620225391.80703272047003; Tue, 7 Jun 2022 02:20:20 -0700 (PDT) Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-300-rVwECxG4OqywWyGKj2jIrw-1; Tue, 07 Jun 2022 05:20:13 -0400 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.rdu2.redhat.com [10.11.54.7]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 17D58802809; Tue, 7 Jun 2022 09:19:55 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (unknown [10.30.29.100]) by smtp.corp.redhat.com (Postfix) with ESMTP id 02C0F1410DDB; Tue, 7 Jun 2022 09:19:55 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (localhost [IPv6:::1]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id C4A9019452D8; Tue, 7 Jun 2022 09:19:54 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx10.intmail.prod.int.rdu2.redhat.com [10.11.54.10]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id 7403519452D2 for ; Tue, 7 Jun 2022 09:19:51 +0000 (UTC) Received: by smtp.corp.redhat.com (Postfix) id 41E8740336E; Tue, 7 Jun 2022 09:19:51 +0000 (UTC) Received: from mimecast-mx02.redhat.com (mimecast04.extmail.prod.ext.rdu2.redhat.com [10.11.55.20]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 3C62A492C3B for ; Tue, 7 Jun 2022 09:19:51 +0000 (UTC) Received: from us-smtp-1.mimecast.com (us-smtp-1.mimecast.com [207.211.31.81]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 20B02108C1CA for ; Tue, 7 Jun 2022 09:19:51 +0000 (UTC) Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.220.28]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-167-y7kRW5ygPDSxYrTN7b23Ug-1; Tue, 07 Jun 2022 05:19:47 -0400 Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id 952D721B38; Tue, 7 Jun 2022 09:19:45 +0000 (UTC) Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 5C15A13638; Tue, 7 Jun 2022 09:19:45 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id oO3aFDEYn2IoLwAAMHmgww (envelope-from ); Tue, 07 Jun 2022 09:19:45 +0000 X-MC-Unique: rVwECxG4OqywWyGKj2jIrw-1 X-Original-To: libvir-list@listman.corp.redhat.com X-MC-Unique: y7kRW5ygPDSxYrTN7b23Ug-1 From: Claudio Fontana To: =?UTF-8?q?Daniel=20P=20=2E=20Berrang=C3=A9?= Subject: [libvirt RFCv11 15/33] multifd-helper: new helper for parallel save/restore Date: Tue, 7 Jun 2022 11:19:18 +0200 Message-Id: <20220607091936.7948-16-cfontana@suse.de> In-Reply-To: <20220607091936.7948-1-cfontana@suse.de> References: <20220607091936.7948-1-cfontana@suse.de> MIME-Version: 1.0 X-Mimecast-Impersonation-Protect: Policy=CLT - Impersonation Protection Definition; Similar Internal Domain=false; Similar Monitored External Domain=false; Custom External Domain=false; Mimecast External Domain=false; Newly Observed Domain=false; Internal User Name=false; Custom Display Name List=false; Reply-to Address Mismatch=false; Targeted Threat Dictionary=false; Mimecast Threat Dictionary=false; Custom Threat Dictionary=false X-Scanned-By: MIMEDefang 2.85 on 10.11.54.10 X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: libvir-list@redhat.com, Claudio Fontana , "Dr . David Alan Gilbert" , Anthony Iliopoulos Errors-To: libvir-list-bounces@redhat.com Sender: "libvir-list" X-Scanned-By: MIMEDefang 2.85 on 10.11.54.7 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=libvir-list-bounces@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable X-ZM-MESSAGEID: 1654593622507100016 Content-Type: text/plain; charset="utf-8"; x-default="true" For the save direction, this helper listens on a unix socket which QEMU connects to for multifd migration to a file. For the restore direction, this helper connects to a unix socket QEMU listens at for multifd migration from a file. The file descriptor is passed as a command line parameter, and interleaved channels are used to allow reading/writing to different parts of the file depending on the channel used. Signed-off-by: Claudio Fontana --- po/POTFILES | 1 + src/util/meson.build | 16 ++ src/util/multifd-helper.c | 359 ++++++++++++++++++++++++++++++++++++++ 3 files changed, 376 insertions(+) create mode 100644 src/util/multifd-helper.c diff --git a/po/POTFILES b/po/POTFILES index faaba53c8f..97ecbb0ead 100644 --- a/po/POTFILES +++ b/po/POTFILES @@ -241,6 +241,7 @@ src/storage_file/storage_source.c src/storage_file/storage_source_backingstore.c src/test/test_driver.c src/util/iohelper.c +src/util/multifd-helper.c src/util/viralloc.c src/util/virarptable.c src/util/viraudit.c diff --git a/src/util/meson.build b/src/util/meson.build index 07ae94631c..2e08ed8745 100644 --- a/src/util/meson.build +++ b/src/util/meson.build @@ -179,6 +179,11 @@ io_helper_sources =3D [ 'virfile.c', ] =20 +multifd_helper_sources =3D [ + 'multifd-helper.c', + 'virfile.c', +] + virt_util_lib =3D static_library( 'virt_util', [ @@ -220,6 +225,17 @@ if conf.has('WITH_LIBVIRTD') libutil_dep, ], } + virt_helpers +=3D { + 'name': 'libvirt_multifd_helper', + 'sources': [ + files(multifd_helper_sources), + dtrace_gen_headers, + ], + 'deps': [ + acl_dep, + libutil_dep, + ], + } endif =20 util_inc_dir =3D include_directories('.') diff --git a/src/util/multifd-helper.c b/src/util/multifd-helper.c new file mode 100644 index 0000000000..6d26e2210a --- /dev/null +++ b/src/util/multifd-helper.c @@ -0,0 +1,359 @@ +/* + * multifd-helper.c: listens on Unix socket to perform I/O to multiple fil= es + * + * Copyright (C) 2022 SUSE LLC + * + * This library is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * This library is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with this library. If not, see + * . + * + * This has been written to support QEMU multifd migration to file, + * allowing better use of cpu resources to speed up the save/restore. + */ + +#include + +#include +#include +#include +#include +#include +#include +#include + +#include "virfile.h" +#include "virerror.h" +#include "virstring.h" +#include "virgettext.h" + +#define VIR_FROM_THIS VIR_FROM_STORAGE + +typedef struct _multiFdConnData multiFdConnData; +struct _multiFdConnData { + int idx; + int nchannels; + int clientfd; + int filefd; + int oflags; + const char *sun_path; + const char *disk_path; + off_t total; + + GThread *tid; +}; + +typedef struct _multiFdThreadArgs multiFdThreadArgs; +struct _multiFdThreadArgs { + int nchannels; + multiFdConnData *conn; /* contains main fd + nchannels */ + const char *sun_path; /* unix socket name to use for the server */ + const char *disk_path; /* disk pathname */ + struct sockaddr_un serv_addr; + + off_t total; +}; + +static gpointer clientThreadFunc(void *a) +{ + multiFdConnData *c =3D a; + c->total =3D virFileDiskCopyChannel(c->filefd, c->disk_path, c->client= fd, c->sun_path, + c->idx, c->nchannels, c->total); + return &c->total; +} + +static off_t waitClientThreads(multiFdConnData *conn, int n) +{ + int idx; + off_t total =3D 0; + + for (idx =3D 0; idx < n; idx++) { + multiFdConnData *c =3D &conn[idx]; + off_t *ctotal; + + ctotal =3D g_thread_join(c->tid); + if (*ctotal < (off_t)0) { + total =3D -1; + } else if (total >=3D 0) { + total +=3D *ctotal; + } + if (VIR_CLOSE(c->clientfd) < 0) { + total =3D -1; + } + } + return total; +} + +static gpointer loadThreadFunc(void *a) +{ + multiFdThreadArgs *args =3D a; + int idx; + args->total =3D -1; + + for (idx =3D 0; idx < args->nchannels; idx++) { + /* Perform outgoing connections */ + multiFdConnData *c =3D &args->conn[idx]; + c->clientfd =3D socket(AF_UNIX, SOCK_STREAM, 0); + if (c->clientfd < 0) { + virReportSystemError(errno, "%s", _("loadThread: socket() fail= ed")); + goto cleanup; + } + if (connect(c->clientfd, (const struct sockaddr *)&args->serv_addr, + sizeof(struct sockaddr_un)) < 0) { + virReportSystemError(errno, "%s", _("loadThread: connect() fai= led")); + goto cleanup; + } + c->tid =3D g_thread_new("libvirt_multifd_load", &clientThreadFunc,= c); + } + args->total =3D waitClientThreads(args->conn, args->nchannels); + + cleanup: + for (idx =3D 0; idx < args->nchannels; idx++) { + multiFdConnData *c =3D &args->conn[idx]; + VIR_FORCE_CLOSE(c->clientfd); + } + return &args->total; +} + +static gpointer saveThreadFunc(void *a) +{ + multiFdThreadArgs *args =3D a; + int idx; + const char buf[1] =3D {'R'}; + int sockfd; + + args->total =3D -1; + + if ((sockfd =3D socket(AF_UNIX, SOCK_STREAM, 0)) < 0) { + virReportSystemError(errno, "%s", _("saveThread: socket() failed")= ); + return &args->total; + } + unlink(args->sun_path); + if (bind(sockfd, (struct sockaddr *)&args->serv_addr, sizeof(args->ser= v_addr)) < 0) { + virReportSystemError(errno, "%s", _("saveThread: bind() failed")); + goto cleanup; + } + if (listen(sockfd, args->nchannels) < 0) { + virReportSystemError(errno, "%s", _("saveThread: listen() failed")= ); + goto cleanup; + } + /* signal that the server is ready */ + if (safewrite(STDOUT_FILENO, &buf, 1) !=3D 1) { + virReportSystemError(errno, "%s", _("saveThread: safewrite failed"= )); + goto cleanup; + } + for (idx =3D 0; idx < args->nchannels; idx++) { + /* Wait for incoming connection. */ + multiFdConnData *c =3D &args->conn[idx]; + if ((c->clientfd =3D accept(sockfd, NULL, NULL)) < 0) { + virReportSystemError(errno, "%s", _("saveThread: accept() fail= ed")); + goto cleanup; + } + c->tid =3D g_thread_new("libvirt_multifd_save", &clientThreadFunc,= c); + } + args->total =3D waitClientThreads(args->conn, args->nchannels); + + cleanup: + for (idx =3D 0; idx < args->nchannels; idx++) { + multiFdConnData *c =3D &args->conn[idx]; + VIR_FORCE_CLOSE(c->clientfd); + } + if (VIR_CLOSE(sockfd) < 0) + args->total =3D -1; + return &args->total; +} + +static int readCLIA(int disk_fd, int nchannels, multiFdConnData *conn) +{ + int idx; + g_autofree void *base =3D NULL; /* Location to be freed */ + size_t buflen =3D virFileDirectAlign(nchannels * 8); + int64_t *buf =3D virFileDirectBufferNew(&base, buflen); + ssize_t got =3D saferead(disk_fd, buf, buflen); + + if (got < buflen) + return -1; + + for (idx =3D 0; idx < nchannels; idx++) { + multiFdConnData *c =3D &conn[idx]; + c->total =3D buf[idx]; + } + return 0; +} + +static int writeCLIA(int disk_fd, int nchannels, multiFdConnData *conn) +{ + int idx; + g_autofree void *base =3D NULL; /* Location to be freed */ + size_t buflen =3D virFileDirectAlign(nchannels * 8); + int64_t *buf =3D virFileDirectBufferNew(&base, buflen); + + for (idx =3D 0; idx < nchannels; idx++) { + multiFdConnData *c =3D &conn[idx]; + buf[idx] =3D c->total; + } + if (safewrite(disk_fd, buf, buflen) < buflen) + return -1; + return 0; +} + +static const char *program_name; + +G_GNUC_NORETURN static void +usage(int status) +{ + if (status) { + fprintf(stderr, _("%s: try --help for more details"), program_name= ); + } else { + fprintf(stderr, _("Usage: %s UNIX_SOCNAME DISK_PATHNAME N MAINFD")= , program_name); + } + exit(status); +} + +int +main(int argc, char **argv) +{ + GThread *tid; + GThreadFunc func; + multiFdThreadArgs args =3D { 0 }; + multiFdConnData *mc; + int idx; + off_t clia_off, data_off, *total; + + program_name =3D argv[0]; + if (virGettextInitialize() < 0 || + virErrorInitialize() < 0) { + fprintf(stderr, _("%s: initialization failed\n"), program_name); + exit(EXIT_FAILURE); + } + + if (argc > 1 && STREQ(argv[1], "--help")) + usage(EXIT_SUCCESS); + if (argc < 5) + usage(EXIT_FAILURE); + + args.sun_path =3D argv[1]; + args.disk_path =3D argv[2]; + if (virStrToLong_i(argv[3], NULL, 10, &args.nchannels) < 0) { + fprintf(stderr, _("%s: malformed number of channels N %s\n"), prog= ram_name, argv[3]); + usage(EXIT_FAILURE); + } + /* consider the main channel as just another channel */ + args.nchannels +=3D 1; + args.conn =3D g_new0(multiFdConnData, args.nchannels); + + /* set main channel connection data */ + mc =3D &args.conn[0]; + mc->idx =3D 0; + mc->nchannels =3D args.nchannels; + if (virStrToLong_i(argv[4], NULL, 10, &mc->filefd) < 0) { + fprintf(stderr, _("%s: malformed MAINFD %s\n"), program_name, argv= [4]); + usage(EXIT_FAILURE); + } + +#ifndef F_GETFL +# error "multifd-helper requires F_GETFL parameter of fcntl" +#endif + + mc->oflags =3D fcntl(mc->filefd, F_GETFL); + mc->sun_path =3D args.sun_path; + mc->disk_path =3D args.disk_path; + clia_off =3D lseek(mc->filefd, 0, SEEK_CUR); + if (clia_off < 0) { + fprintf(stderr, _("%s: failed to seek %s\n"), program_name, args.d= isk_path); + exit(EXIT_FAILURE); + } + if ((mc->oflags & O_ACCMODE) =3D=3D O_RDONLY) { + func =3D loadThreadFunc; + /* set totals from the Channel Length Indicators Area */ + if (readCLIA(mc->filefd, args.nchannels, args.conn) < 0) { + fprintf(stderr, _("%s: failed to read CLIA\n"), program_name); + exit(EXIT_FAILURE); + } + } else { + func =3D saveThreadFunc; + /* skip Channel Length Indicators Area */ + if (lseek(mc->filefd, virFileDirectAlign(args.nchannels * 8), SEEK= _CUR) < 0) { + fprintf(stderr, _("%s: failed to seek %s\n"), program_name, ar= gs.disk_path); + exit(EXIT_FAILURE); + } + mc->total =3D 0; + } + if ((data_off =3D lseek(mc->filefd, 0, SEEK_CUR)) < 0) { + fprintf(stderr, _("%s: failed to seek %s\n"), program_name, args.d= isk_path); + exit(EXIT_FAILURE); + } + + /* initialize channels */ + for (idx =3D 1; idx < args.nchannels; idx++) { + multiFdConnData *c =3D &args.conn[idx]; + c->idx =3D idx; + c->nchannels =3D args.nchannels; + c->oflags =3D mc->oflags & ~(O_TRUNC | O_CREAT); + c->filefd =3D open(args.disk_path, c->oflags); + if (c->filefd < 0) { + fprintf(stderr, _("%s: failed to open %s\n"), program_name, ar= gs.disk_path); + exit(EXIT_FAILURE); + } + c->sun_path =3D args.sun_path; + c->disk_path =3D args.disk_path; + if (mc->total =3D=3D 0) + c->total =3D 0; + if (lseek(c->filefd, data_off, SEEK_SET) < 0) { + fprintf(stderr, _("%s: failed to seek %s\n"), program_name, ar= gs.disk_path); + exit(EXIT_FAILURE); + } + } + + /* initialize server address structure */ + memset(&args.serv_addr, 0, sizeof(args.serv_addr)); + args.serv_addr.sun_family =3D AF_UNIX; + virStrcpyStatic(args.serv_addr.sun_path, args.sun_path); + + tid =3D g_thread_new("libvirt_multifd_func", func, &args); + + total =3D g_thread_join(tid); + if (*total < 0) { + exit(EXIT_FAILURE); + } + if (func =3D=3D saveThreadFunc) { + /* write CLIA */ + if (lseek(mc->filefd, clia_off, SEEK_SET) < 0) { + fprintf(stderr, _("%s: failed to seek %s\n"), program_name, ar= gs.disk_path); + exit(EXIT_FAILURE); + } + /* set totals into the Channel Length Indicators Area */ + if (writeCLIA(mc->filefd, args.nchannels, args.conn) < 0) { + fprintf(stderr, _("%s: failed to write CLIA\n"), program_name); + exit(EXIT_FAILURE); + } + if (lseek(mc->filefd, 0, SEEK_END) < 0) { + fprintf(stderr, _("%s: failed to seek %s\n"), program_name, ar= gs.disk_path); + exit(EXIT_FAILURE); + } + if (virFileDataSync(mc->filefd) < 0) { + if (errno !=3D EINVAL && errno !=3D EROFS) { + fprintf(stderr, _("%s: failed to fsyncdata %s\n"), program= _name, args.disk_path); + exit(EXIT_FAILURE); + } + } + } + /* close up */ + for (idx =3D 0; idx < args.nchannels; idx++) { + multiFdConnData *c =3D &args.conn[idx]; + if (VIR_CLOSE(c->filefd) < 0) { + fprintf(stderr, _("%s: failed to close %s\n"), program_name, a= rgs.disk_path); + exit(EXIT_FAILURE); + } + } + exit(EXIT_SUCCESS); +} --=20 2.26.2 From nobody Fri May 17 11:58:59 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 170.10.129.124 as permitted sender) client-ip=170.10.129.124; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-delivery-124.mimecast.com; Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=fail(p=none dis=none) header.from=suse.de Return-Path: Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by mx.zohomail.com with SMTPS id 1654593645506426.71689182601267; Tue, 7 Jun 2022 02:20:45 -0700 (PDT) Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-643-KySZuqTnNE-LzzpnMC2lgA-1; Tue, 07 Jun 2022 05:20:40 -0400 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.rdu2.redhat.com [10.11.54.3]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 3C4A73810787; Tue, 7 Jun 2022 09:20:38 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (unknown [10.30.29.100]) by smtp.corp.redhat.com (Postfix) with ESMTP id 24EE21121315; Tue, 7 Jun 2022 09:20:38 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (localhost [IPv6:::1]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id B67A41947B91; Tue, 7 Jun 2022 09:20:37 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx09.intmail.prod.int.rdu2.redhat.com [10.11.54.9]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id 89E3A1947B8B for ; Tue, 7 Jun 2022 09:20:36 +0000 (UTC) Received: by smtp.corp.redhat.com (Postfix) id 6D444492C3B; Tue, 7 Jun 2022 09:20:36 +0000 (UTC) Received: from mimecast-mx02.redhat.com (mimecast07.extmail.prod.ext.rdu2.redhat.com [10.11.55.23]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 694BF492CA3 for ; Tue, 7 Jun 2022 09:20:36 +0000 (UTC) Received: from us-smtp-1.mimecast.com (us-smtp-2.mimecast.com [205.139.110.61]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 51A273C0D848 for ; Tue, 7 Jun 2022 09:20:36 +0000 (UTC) Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.220.29]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-304-mN82Bwg7NWqAa2yjUY-uLw-1; Tue, 07 Jun 2022 05:20:28 -0400 Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id D94F71F941; Tue, 7 Jun 2022 09:19:45 +0000 (UTC) Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 9E27113638; Tue, 7 Jun 2022 09:19:45 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id iOv4JDEYn2IoLwAAMHmgww (envelope-from ); Tue, 07 Jun 2022 09:19:45 +0000 X-MC-Unique: KySZuqTnNE-LzzpnMC2lgA-1 X-Original-To: libvir-list@listman.corp.redhat.com X-MC-Unique: mN82Bwg7NWqAa2yjUY-uLw-1 From: Claudio Fontana To: =?UTF-8?q?Daniel=20P=20=2E=20Berrang=C3=A9?= Subject: [libvirt RFCv11 16/33] qemu: saveimage: update virQEMUSaveFd struct for parallel save Date: Tue, 7 Jun 2022 11:19:19 +0200 Message-Id: <20220607091936.7948-17-cfontana@suse.de> In-Reply-To: <20220607091936.7948-1-cfontana@suse.de> References: <20220607091936.7948-1-cfontana@suse.de> MIME-Version: 1.0 X-Mimecast-Impersonation-Protect: Policy=CLT - Impersonation Protection Definition; Similar Internal Domain=false; Similar Monitored External Domain=false; Custom External Domain=false; Mimecast External Domain=false; Newly Observed Domain=false; Internal User Name=false; Custom Display Name List=false; Reply-to Address Mismatch=false; Targeted Threat Dictionary=false; Mimecast Threat Dictionary=false; Custom Threat Dictionary=false X-Scanned-By: MIMEDefang 2.85 on 10.11.54.9 X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: libvir-list@redhat.com, Claudio Fontana , "Dr . David Alan Gilbert" , Anthony Iliopoulos Errors-To: libvir-list-bounces@redhat.com Sender: "libvir-list" X-Scanned-By: MIMEDefang 2.78 on 10.11.54.3 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=libvir-list-bounces@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable X-ZM-MESSAGEID: 1654593647324100001 Content-Type: text/plain; charset="utf-8"; x-default="true" add nchannels to the struct and adapt the APIs slightly. Add a new API virQEMUSaveFdAddChannels to set the number of channels, which might not be known at Init time. Signed-off-by: Claudio Fontana --- src/qemu/qemu_driver.c | 10 +++++----- src/qemu/qemu_saveimage.c | 27 ++++++++++++++++++++++++--- src/qemu/qemu_saveimage.h | 6 +++++- 3 files changed, 34 insertions(+), 9 deletions(-) diff --git a/src/qemu/qemu_driver.c b/src/qemu/qemu_driver.c index 986bbf9e58..d403ff6bf2 100644 --- a/src/qemu/qemu_driver.c +++ b/src/qemu/qemu_driver.c @@ -5883,7 +5883,7 @@ qemuDomainRestoreInternal(virConnectPtr conn, } oflags |=3D O_DIRECT; } - if (virQEMUSaveFdInit(&saveFd, path, oflags, cfg) < 0) + if (virQEMUSaveFdInit(&saveFd, path, oflags, cfg, false) < 0) return -1; if (qemuSaveImageOpen(driver, NULL, &def, &data, false, &saveFd) < 0) goto cleanup; @@ -6017,7 +6017,7 @@ qemuDomainSaveImageGetXMLDesc(virConnectPtr conn, con= st char *path, =20 virCheckFlags(VIR_DOMAIN_SAVE_IMAGE_XML_SECURE, NULL); =20 - if (virQEMUSaveFdInit(&saveFd, path, O_RDONLY, cfg) < 0) + if (virQEMUSaveFdInit(&saveFd, path, O_RDONLY, cfg, false) < 0) return NULL; if (qemuSaveImageOpen(driver, NULL, &def, &data, false, &saveFd) < 0) goto cleanup; @@ -6055,7 +6055,7 @@ qemuDomainSaveImageDefineXML(virConnectPtr conn, cons= t char *path, else if (flags & VIR_DOMAIN_SAVE_PAUSED) state =3D 0; =20 - if (virQEMUSaveFdInit(&saveFd, path, O_RDWR, cfg) < 0) + if (virQEMUSaveFdInit(&saveFd, path, O_RDWR, cfg, false) < 0) return -1; if (qemuSaveImageOpen(driver, NULL, &def, &data, false, &saveFd) < 0) goto cleanup; @@ -6136,7 +6136,7 @@ qemuDomainManagedSaveGetXMLDesc(virDomainPtr dom, uns= igned int flags) goto cleanup; } =20 - if (virQEMUSaveFdInit(&saveFd, path, O_RDONLY, cfg) < 0) + if (virQEMUSaveFdInit(&saveFd, path, O_RDONLY, cfg, false) < 0) goto cleanup; if (qemuSaveImageOpen(driver, priv->qemuCaps, &def, &data, false, &saveFd) < 0) @@ -6212,7 +6212,7 @@ qemuDomainObjRestore(virConnectPtr conn, } oflags |=3D O_DIRECT; } - if (virQEMUSaveFdInit(&saveFd, path, oflags, cfg) < 0) + if (virQEMUSaveFdInit(&saveFd, path, oflags, cfg, false) < 0) goto cleanup; ret =3D qemuSaveImageOpen(driver, NULL, &def, &data, true, &saveFd); if (ret < 0) { diff --git a/src/qemu/qemu_saveimage.c b/src/qemu/qemu_saveimage.c index 70ded3c439..6418114d8a 100644 --- a/src/qemu/qemu_saveimage.c +++ b/src/qemu/qemu_saveimage.c @@ -403,12 +403,13 @@ qemuSaveImageGetCompressionCommand(virQEMUSaveFormat = compression) * @base: the file name * @oflags the file descriptor open flags * @cfg: the driver config + * @parallel: whether parallel save/restore is enabled * * Returns -1 on error, 0 on success, * and in both cases virQEMUSaveFdFini must be called to free resources. */ int virQEMUSaveFdInit(virQEMUSaveFd *saveFd, const char *base, - int oflags, virQEMUDriverConfig *cfg) + int oflags, virQEMUDriverConfig *cfg, bool parallel) { unsigned int wrapperFlags =3D VIR_FILE_WRAPPER_NON_BLOCKING; bool isCreat =3D oflags & O_CREAT; @@ -430,11 +431,14 @@ int virQEMUSaveFdInit(virQEMUSaveFd *saveFd, const ch= ar *base, } if (saveFd->fd < 0) return -1; + + saveFd->nchannels =3D 0; /* + * iohelper Wrapper is never required for multifd parallel save. * For O_CREAT, we always add the wrapper, * and for !O_CREAT, we only add the wrapper if using O_DIRECT. */ - if (isDirect || isCreat) { + if (!parallel && (isDirect || isCreat)) { saveFd->wrapper =3D virFileWrapperFdNew(&saveFd->fd, saveFd->path,= wrapperFlags); if (!saveFd->wrapper) return -1; @@ -442,6 +446,23 @@ int virQEMUSaveFdInit(virQEMUSaveFd *saveFd, const cha= r *base, return 0; } =20 +/* + * virQEMUSaveFdAddChannels: add multifd channels to a virQEMUSaveFd + * + * @saveFd: the structure to add channels to. + * @nchannels:number of multifd channels, must be > 0. + * + * Returns -1 on error, 0 on success. + */ +int virQEMUSaveFdAddChannels(virQEMUSaveFd *saveFd, int nchannels) +{ + if (nchannels > 0) { + saveFd->nchannels =3D nchannels; + return 0; + } + return -1; +} + /* * virQEMUSaveFdClose: close a virQEMUSaveFd descriptor with normal close. * @@ -539,7 +560,7 @@ qemuSaveImageCreate(virQEMUDriver *driver, oflags |=3D O_DIRECT; } =20 - if (virQEMUSaveFdInit(&saveFd, path, oflags, cfg) < 0) + if (virQEMUSaveFdInit(&saveFd, path, oflags, cfg, false) < 0) goto cleanup; if (qemuSecuritySetImageFDLabel(driver->securityManager, vm->def, save= Fd.fd) < 0) goto cleanup; diff --git a/src/qemu/qemu_saveimage.h b/src/qemu/qemu_saveimage.h index 9b04af2fb9..dcc9aa7d63 100644 --- a/src/qemu/qemu_saveimage.h +++ b/src/qemu/qemu_saveimage.h @@ -62,17 +62,21 @@ struct _virQEMUSaveFd { bool need_unlink; virFileWrapperFd *wrapper; virQEMUDriverConfig *cfg; + int nchannels; }; =20 #define QEMU_SAVEFD_INVALID (virQEMUSaveFd) { \ .path =3D NULL, .fd =3D -1, .oflags =3D 0, \ .need_unlink =3D false, .wrapper =3D NULL, .cfg =3D NULL, \ + .nchannels =3D 0, \ } =20 int virQEMUSaveFdInit(virQEMUSaveFd *saveFd, const char *base, - int oflags, virQEMUDriverConfig *cfg) + int oflags, virQEMUDriverConfig *cfg, bool parallel) ATTRIBUTE_NONNULL(1) ATTRIBUTE_NONNULL(4); =20 +int virQEMUSaveFdAddChannels(virQEMUSaveFd *saveFd, int nchannels); + int virQEMUSaveFdClose(virQEMUSaveFd *saveFd, virDomainObj *vm); =20 int virQEMUSaveFdFini(virQEMUSaveFd *saveFd, virDomainObj *vm, int ret); --=20 2.26.2 From nobody Fri May 17 11:58:59 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 170.10.133.124 as permitted sender) client-ip=170.10.133.124; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-delivery-124.mimecast.com; Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=fail(p=none dis=none) header.from=suse.de Return-Path: Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by mx.zohomail.com with SMTPS id 1654593628397319.12227519003284; Tue, 7 Jun 2022 02:20:28 -0700 (PDT) Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-400-s9PrjjRzPJC0Is5CSf1EiA-1; Tue, 07 Jun 2022 05:20:19 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.rdu2.redhat.com [10.11.54.5]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id DD996380114F; Tue, 7 Jun 2022 09:20:01 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (unknown [10.30.29.100]) by smtp.corp.redhat.com (Postfix) with ESMTP id C693C7AD8; Tue, 7 Jun 2022 09:20:01 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (localhost [IPv6:::1]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id 97FF51947B87; Tue, 7 Jun 2022 09:20:01 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx09.intmail.prod.int.rdu2.redhat.com [10.11.54.9]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id D184719452D2 for ; Tue, 7 Jun 2022 09:20:00 +0000 (UTC) Received: by smtp.corp.redhat.com (Postfix) id 950D9492CA3; Tue, 7 Jun 2022 09:20:00 +0000 (UTC) Received: from mimecast-mx02.redhat.com (mimecast07.extmail.prod.ext.rdu2.redhat.com [10.11.55.23]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 90DB8492C3B for ; Tue, 7 Jun 2022 09:20:00 +0000 (UTC) Received: from us-smtp-1.mimecast.com (us-smtp-1.mimecast.com [205.139.110.61]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 7AB453C0D191 for ; Tue, 7 Jun 2022 09:20:00 +0000 (UTC) Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.220.28]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-368-6UVrUoimPSC9MlEO8KnqCQ-1; Tue, 07 Jun 2022 05:19:56 -0400 Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id 2889121B3B; Tue, 7 Jun 2022 09:19:46 +0000 (UTC) Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id E130C13638; Tue, 7 Jun 2022 09:19:45 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id CH9wNTEYn2IoLwAAMHmgww (envelope-from ); Tue, 07 Jun 2022 09:19:45 +0000 X-MC-Unique: s9PrjjRzPJC0Is5CSf1EiA-1 X-Original-To: libvir-list@listman.corp.redhat.com X-MC-Unique: 6UVrUoimPSC9MlEO8KnqCQ-1 From: Claudio Fontana To: =?UTF-8?q?Daniel=20P=20=2E=20Berrang=C3=A9?= Subject: [libvirt RFCv11 17/33] qemu: saveimage: wire up saveimage code with the multifd helper Date: Tue, 7 Jun 2022 11:19:20 +0200 Message-Id: <20220607091936.7948-18-cfontana@suse.de> In-Reply-To: <20220607091936.7948-1-cfontana@suse.de> References: <20220607091936.7948-1-cfontana@suse.de> MIME-Version: 1.0 X-Mimecast-Impersonation-Protect: Policy=CLT - Impersonation Protection Definition; Similar Internal Domain=false; Similar Monitored External Domain=false; Custom External Domain=false; Mimecast External Domain=false; Newly Observed Domain=false; Internal User Name=false; Custom Display Name List=false; Reply-to Address Mismatch=false; Targeted Threat Dictionary=false; Mimecast Threat Dictionary=false; Custom Threat Dictionary=false X-Scanned-By: MIMEDefang 2.85 on 10.11.54.9 X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: libvir-list@redhat.com, Claudio Fontana , "Dr . David Alan Gilbert" , Anthony Iliopoulos Errors-To: libvir-list-bounces@redhat.com Sender: "libvir-list" X-Scanned-By: MIMEDefang 2.79 on 10.11.54.5 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=libvir-list-bounces@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable X-ZM-MESSAGEID: 1654593629171100005 Content-Type: text/plain; charset="utf-8"; x-default="true" prepare to use the multifd helper. Signed-off-by: Claudio Fontana --- src/qemu/qemu_saveimage.c | 41 ++++++++++++++++++++++++++++++++++----- 1 file changed, 36 insertions(+), 5 deletions(-) diff --git a/src/qemu/qemu_saveimage.c b/src/qemu/qemu_saveimage.c index 6418114d8a..8af9b92b35 100644 --- a/src/qemu/qemu_saveimage.c +++ b/src/qemu/qemu_saveimage.c @@ -17,6 +17,7 @@ */ =20 #include +#include =20 #include "qemu_saveimage.h" #include "qemu_domain.h" @@ -542,7 +543,7 @@ qemuSaveImageCreate(virQEMUDriver *driver, const char *path, virQEMUSaveData *data, virCommand *compressor, - int nconn G_GNUC_UNUSED, + int nconn, unsigned int flags, virDomainAsyncJob asyncJob) { @@ -566,10 +567,40 @@ qemuSaveImageCreate(virQEMUDriver *driver, goto cleanup; if (virQEMUSaveDataWrite(data, saveFd.fd, saveFd.path) < 0) goto cleanup; - - /* Perform the migration */ - if (qemuMigrationSrcToFile(driver, vm, saveFd.fd, compressor, asyncJob= ) < 0) - goto cleanup; + if (nconn > 0) { + if (virQEMUSaveFdAddChannels(&saveFd, nconn) < 0) + goto cleanup; + } + if (flags & VIR_DOMAIN_SAVE_PARALLEL) { + g_autoptr(virCommand) cmd =3D NULL; + g_autofree char *helper_path =3D NULL; + qemuDomainObjPrivate *priv =3D vm->privateData; + g_autofree char *sun_path =3D g_strdup_printf("%s/save-multifd.soc= k", priv->libDir); + char buf[1]; + int helper_out =3D -1; + if (!(helper_path =3D virFileFindResource("libvirt_multifd_helper", + abs_top_builddir "/src", + LIBEXECDIR))) + goto cleanup; + cmd =3D virCommandNewArgList(helper_path, sun_path, saveFd.path, N= ULL); + virCommandAddArgFormat(cmd, "%d", nconn); + virCommandAddArgFormat(cmd, "%d", saveFd.fd); + virCommandPassFD(cmd, saveFd.fd, 0); + virCommandSetOutputFD(cmd, &helper_out); /* should create pipe aut= omagically */ + if (virCommandRunAsync(cmd, NULL) < 0) + goto cleanup; + if (saferead(helper_out, &buf, 1) !=3D 1 || buf[0] !=3D 'R') + goto cleanup; + if (chown(sun_path, cfg->user, cfg->group) < 0) + goto cleanup; + /* still using single fd migration for now */ + if (qemuMigrationSrcToFile(driver, vm, saveFd.fd, compressor, asyn= cJob) < 0) + goto cleanup; + } else { + /* Perform non-parallel migration to file */ + if (qemuMigrationSrcToFile(driver, vm, saveFd.fd, compressor, asyn= cJob) < 0) + goto cleanup; + } =20 if (virQEMUSaveFdClose(&saveFd, vm) < 0) goto cleanup; --=20 2.26.2 From nobody Fri May 17 11:58:59 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 170.10.129.124 as permitted sender) client-ip=170.10.129.124; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-delivery-124.mimecast.com; Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=fail(p=none dis=none) header.from=suse.de Return-Path: Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by mx.zohomail.com with SMTPS id 1654593621992954.5455281845693; Tue, 7 Jun 2022 02:20:21 -0700 (PDT) Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-225-NKmBOvMJMc2hlxNtrJGlVw-1; Tue, 07 Jun 2022 05:20:15 -0400 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.rdu2.redhat.com [10.11.54.1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id C01E5905E8E; Tue, 7 Jun 2022 09:20:04 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (unknown [10.30.29.100]) by smtp.corp.redhat.com (Postfix) with ESMTP id 9FD24406AD48; Tue, 7 Jun 2022 09:20:04 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (localhost [IPv6:::1]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id 78B781947BA7; Tue, 7 Jun 2022 09:20:04 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.rdu2.redhat.com [10.11.54.2]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id 3F77619451F1 for ; Tue, 7 Jun 2022 09:20:01 +0000 (UTC) Received: by smtp.corp.redhat.com (Postfix) id 1755B40EC003; Tue, 7 Jun 2022 09:20:01 +0000 (UTC) Received: from mimecast-mx02.redhat.com (mimecast01.extmail.prod.ext.rdu2.redhat.com [10.11.55.17]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 1244F40EC002 for ; Tue, 7 Jun 2022 09:20:01 +0000 (UTC) Received: from us-smtp-1.mimecast.com (us-smtp-2.mimecast.com [205.139.110.61]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id E960F85828B for ; Tue, 7 Jun 2022 09:20:00 +0000 (UTC) Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.220.28]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-412-62_ntvbsOvqa6rP6CglZdg-1; Tue, 07 Jun 2022 05:19:55 -0400 Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id 77D4821B35; Tue, 7 Jun 2022 09:19:46 +0000 (UTC) Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 3061013638; Tue, 7 Jun 2022 09:19:46 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id GHhWCjIYn2IoLwAAMHmgww (envelope-from ); Tue, 07 Jun 2022 09:19:46 +0000 X-MC-Unique: NKmBOvMJMc2hlxNtrJGlVw-1 X-Original-To: libvir-list@listman.corp.redhat.com X-MC-Unique: 62_ntvbsOvqa6rP6CglZdg-1 From: Claudio Fontana To: =?UTF-8?q?Daniel=20P=20=2E=20Berrang=C3=A9?= Subject: [libvirt RFCv11 18/33] qemu: capabilities: add multifd to the probed migration capabilities Date: Tue, 7 Jun 2022 11:19:21 +0200 Message-Id: <20220607091936.7948-19-cfontana@suse.de> In-Reply-To: <20220607091936.7948-1-cfontana@suse.de> References: <20220607091936.7948-1-cfontana@suse.de> MIME-Version: 1.0 X-Mimecast-Impersonation-Protect: Policy=CLT - Impersonation Protection Definition; Similar Internal Domain=false; Similar Monitored External Domain=false; Custom External Domain=false; Mimecast External Domain=false; Newly Observed Domain=false; Internal User Name=false; Custom Display Name List=false; Reply-to Address Mismatch=false; Targeted Threat Dictionary=false; Mimecast Threat Dictionary=false; Custom Threat Dictionary=false X-Scanned-By: MIMEDefang 2.84 on 10.11.54.2 X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Anthony Iliopoulos , libvir-list@redhat.com, "Dr . David Alan Gilbert" , Claudio Fontana , Ani Sinha Errors-To: libvir-list-bounces@redhat.com Sender: "libvir-list" X-Scanned-By: MIMEDefang 2.84 on 10.11.54.1 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=libvir-list-bounces@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable X-ZM-MESSAGEID: 1654593622487100015 Content-Type: text/plain; charset="utf-8"; x-default="true" Signed-off-by: Claudio Fontana Reviewed-by: Ani Sinha --- src/qemu/qemu_capabilities.c | 2 ++ src/qemu/qemu_capabilities.h | 1 + tests/qemucapabilitiesdata/caps_4.0.0.aarch64.xml | 1 + tests/qemucapabilitiesdata/caps_4.0.0.ppc64.xml | 1 + tests/qemucapabilitiesdata/caps_4.0.0.riscv32.xml | 1 + tests/qemucapabilitiesdata/caps_4.0.0.riscv64.xml | 1 + tests/qemucapabilitiesdata/caps_4.0.0.s390x.xml | 1 + tests/qemucapabilitiesdata/caps_4.0.0.x86_64.xml | 1 + tests/qemucapabilitiesdata/caps_4.1.0.x86_64.xml | 1 + tests/qemucapabilitiesdata/caps_4.2.0.aarch64.xml | 1 + tests/qemucapabilitiesdata/caps_4.2.0.ppc64.xml | 1 + tests/qemucapabilitiesdata/caps_4.2.0.s390x.xml | 1 + tests/qemucapabilitiesdata/caps_4.2.0.x86_64.xml | 1 + tests/qemucapabilitiesdata/caps_5.0.0.aarch64.xml | 1 + tests/qemucapabilitiesdata/caps_5.0.0.ppc64.xml | 1 + tests/qemucapabilitiesdata/caps_5.0.0.riscv64.xml | 1 + tests/qemucapabilitiesdata/caps_5.0.0.x86_64.xml | 1 + tests/qemucapabilitiesdata/caps_5.1.0.sparc.xml | 1 + tests/qemucapabilitiesdata/caps_5.1.0.x86_64.xml | 1 + tests/qemucapabilitiesdata/caps_5.2.0.aarch64.xml | 1 + tests/qemucapabilitiesdata/caps_5.2.0.ppc64.xml | 1 + tests/qemucapabilitiesdata/caps_5.2.0.riscv64.xml | 1 + tests/qemucapabilitiesdata/caps_5.2.0.s390x.xml | 1 + tests/qemucapabilitiesdata/caps_5.2.0.x86_64.xml | 1 + tests/qemucapabilitiesdata/caps_6.0.0.aarch64.xml | 1 + tests/qemucapabilitiesdata/caps_6.0.0.s390x.xml | 1 + tests/qemucapabilitiesdata/caps_6.0.0.x86_64.xml | 1 + tests/qemucapabilitiesdata/caps_6.1.0.x86_64.xml | 1 + tests/qemucapabilitiesdata/caps_6.2.0.aarch64.xml | 1 + tests/qemucapabilitiesdata/caps_6.2.0.ppc64.xml | 1 + tests/qemucapabilitiesdata/caps_6.2.0.x86_64.xml | 1 + tests/qemucapabilitiesdata/caps_7.0.0.aarch64.xml | 1 + tests/qemucapabilitiesdata/caps_7.0.0.ppc64.xml | 1 + tests/qemucapabilitiesdata/caps_7.0.0.x86_64.xml | 1 + tests/qemucapabilitiesdata/caps_7.1.0.x86_64.xml | 1 + 35 files changed, 36 insertions(+) diff --git a/src/qemu/qemu_capabilities.c b/src/qemu/qemu_capabilities.c index d0c8217825..44e3f9b81f 100644 --- a/src/qemu/qemu_capabilities.c +++ b/src/qemu/qemu_capabilities.c @@ -676,6 +676,7 @@ VIR_ENUM_IMPL(virQEMUCaps, /* 430 */ "chardev.qemu-vdagent", /* QEMU_CAPS_CHARDEV_QEMU_VDAGENT */ "display-dbus", /* QEMU_CAPS_DISPLAY_DBUS */ + "migrate-multifd", /* QEMU_CAPS_MIGRATE_MULTIFD */ ); =20 =20 @@ -1234,6 +1235,7 @@ struct virQEMUCapsStringFlags virQEMUCapsCommands[] = =3D { =20 struct virQEMUCapsStringFlags virQEMUCapsMigration[] =3D { { "rdma-pin-all", QEMU_CAPS_MIGRATE_RDMA }, + { "multifd", QEMU_CAPS_MIGRATE_MULTIFD }, }; =20 /* Use virQEMUCapsQMPSchemaQueries for querying parameters of events */ diff --git a/src/qemu/qemu_capabilities.h b/src/qemu/qemu_capabilities.h index 26cd655db3..4c7e031e28 100644 --- a/src/qemu/qemu_capabilities.h +++ b/src/qemu/qemu_capabilities.h @@ -651,6 +651,7 @@ typedef enum { /* virQEMUCapsFlags grouping marker for = syntax-check */ /* 430 */ QEMU_CAPS_CHARDEV_QEMU_VDAGENT, /* -chardev qemu-vdagent */ QEMU_CAPS_DISPLAY_DBUS, /* -display dbus */ + QEMU_CAPS_MIGRATE_MULTIFD, /* migrate can set multifd parameter */ =20 QEMU_CAPS_LAST /* this must always be the last item */ } virQEMUCapsFlags; diff --git a/tests/qemucapabilitiesdata/caps_4.0.0.aarch64.xml b/tests/qemu= capabilitiesdata/caps_4.0.0.aarch64.xml index 7e0b8fbddf..c3df1b403d 100644 --- a/tests/qemucapabilitiesdata/caps_4.0.0.aarch64.xml +++ b/tests/qemucapabilitiesdata/caps_4.0.0.aarch64.xml @@ -147,6 +147,7 @@ + 4000000 0 61700240 diff --git a/tests/qemucapabilitiesdata/caps_4.0.0.ppc64.xml b/tests/qemuca= pabilitiesdata/caps_4.0.0.ppc64.xml index 19bbbd1de3..17b1c6dc29 100644 --- a/tests/qemucapabilitiesdata/caps_4.0.0.ppc64.xml +++ b/tests/qemucapabilitiesdata/caps_4.0.0.ppc64.xml @@ -152,6 +152,7 @@ + 4000000 0 42900240 diff --git a/tests/qemucapabilitiesdata/caps_4.0.0.riscv32.xml b/tests/qemu= capabilitiesdata/caps_4.0.0.riscv32.xml index ef6f04f54b..8c0ca3cd92 100644 --- a/tests/qemucapabilitiesdata/caps_4.0.0.riscv32.xml +++ b/tests/qemucapabilitiesdata/caps_4.0.0.riscv32.xml @@ -144,6 +144,7 @@ + 4000000 0 0 diff --git a/tests/qemucapabilitiesdata/caps_4.0.0.riscv64.xml b/tests/qemu= capabilitiesdata/caps_4.0.0.riscv64.xml index 7c65aff290..08b1f87f08 100644 --- a/tests/qemucapabilitiesdata/caps_4.0.0.riscv64.xml +++ b/tests/qemucapabilitiesdata/caps_4.0.0.riscv64.xml @@ -144,6 +144,7 @@ + 4000000 0 0 diff --git a/tests/qemucapabilitiesdata/caps_4.0.0.s390x.xml b/tests/qemuca= pabilitiesdata/caps_4.0.0.s390x.xml index 9b5ed96ba3..a024383b72 100644 --- a/tests/qemucapabilitiesdata/caps_4.0.0.s390x.xml +++ b/tests/qemucapabilitiesdata/caps_4.0.0.s390x.xml @@ -114,6 +114,7 @@ + 4000000 0 39100240 diff --git a/tests/qemucapabilitiesdata/caps_4.0.0.x86_64.xml b/tests/qemuc= apabilitiesdata/caps_4.0.0.x86_64.xml index 3cf6a66389..d110bd0eaf 100644 --- a/tests/qemucapabilitiesdata/caps_4.0.0.x86_64.xml +++ b/tests/qemucapabilitiesdata/caps_4.0.0.x86_64.xml @@ -187,6 +187,7 @@ + 4000000 0 43100240 diff --git a/tests/qemucapabilitiesdata/caps_4.1.0.x86_64.xml b/tests/qemuc= apabilitiesdata/caps_4.1.0.x86_64.xml index 5daa7bda75..50e80aab9f 100644 --- a/tests/qemucapabilitiesdata/caps_4.1.0.x86_64.xml +++ b/tests/qemucapabilitiesdata/caps_4.1.0.x86_64.xml @@ -194,6 +194,7 @@ + 4001000 0 43100241 diff --git a/tests/qemucapabilitiesdata/caps_4.2.0.aarch64.xml b/tests/qemu= capabilitiesdata/caps_4.2.0.aarch64.xml index 833bf955f9..250bf8a433 100644 --- a/tests/qemucapabilitiesdata/caps_4.2.0.aarch64.xml +++ b/tests/qemucapabilitiesdata/caps_4.2.0.aarch64.xml @@ -162,6 +162,7 @@ + 4001050 0 61700242 diff --git a/tests/qemucapabilitiesdata/caps_4.2.0.ppc64.xml b/tests/qemuca= pabilitiesdata/caps_4.2.0.ppc64.xml index 1586f28ca5..06432c25bd 100644 --- a/tests/qemucapabilitiesdata/caps_4.2.0.ppc64.xml +++ b/tests/qemucapabilitiesdata/caps_4.2.0.ppc64.xml @@ -159,6 +159,7 @@ + 4001050 0 42900242 diff --git a/tests/qemucapabilitiesdata/caps_4.2.0.s390x.xml b/tests/qemuca= pabilitiesdata/caps_4.2.0.s390x.xml index ce5b782afd..1bd6f44f16 100644 --- a/tests/qemucapabilitiesdata/caps_4.2.0.s390x.xml +++ b/tests/qemucapabilitiesdata/caps_4.2.0.s390x.xml @@ -127,6 +127,7 @@ + 4002000 0 39100242 diff --git a/tests/qemucapabilitiesdata/caps_4.2.0.x86_64.xml b/tests/qemuc= apabilitiesdata/caps_4.2.0.x86_64.xml index 9ee4c0534d..d96f13b650 100644 --- a/tests/qemucapabilitiesdata/caps_4.2.0.x86_64.xml +++ b/tests/qemucapabilitiesdata/caps_4.2.0.x86_64.xml @@ -205,6 +205,7 @@ + 4002000 0 43100242 diff --git a/tests/qemucapabilitiesdata/caps_5.0.0.aarch64.xml b/tests/qemu= capabilitiesdata/caps_5.0.0.aarch64.xml index 29ee31473f..2a7f5074bc 100644 --- a/tests/qemucapabilitiesdata/caps_5.0.0.aarch64.xml +++ b/tests/qemucapabilitiesdata/caps_5.0.0.aarch64.xml @@ -174,6 +174,7 @@ + 5000000 0 61700241 diff --git a/tests/qemucapabilitiesdata/caps_5.0.0.ppc64.xml b/tests/qemuca= pabilitiesdata/caps_5.0.0.ppc64.xml index 1fdec901a6..d2c083e8f7 100644 --- a/tests/qemucapabilitiesdata/caps_5.0.0.ppc64.xml +++ b/tests/qemucapabilitiesdata/caps_5.0.0.ppc64.xml @@ -180,6 +180,7 @@ + 5000000 0 42900241 diff --git a/tests/qemucapabilitiesdata/caps_5.0.0.riscv64.xml b/tests/qemu= capabilitiesdata/caps_5.0.0.riscv64.xml index 3b58e7fece..933b4870cd 100644 --- a/tests/qemucapabilitiesdata/caps_5.0.0.riscv64.xml +++ b/tests/qemucapabilitiesdata/caps_5.0.0.riscv64.xml @@ -166,6 +166,7 @@ + 5000000 0 0 diff --git a/tests/qemucapabilitiesdata/caps_5.0.0.x86_64.xml b/tests/qemuc= apabilitiesdata/caps_5.0.0.x86_64.xml index bee5a84cf9..9bb48e1982 100644 --- a/tests/qemucapabilitiesdata/caps_5.0.0.x86_64.xml +++ b/tests/qemucapabilitiesdata/caps_5.0.0.x86_64.xml @@ -214,6 +214,7 @@ + 5000000 0 43100241 diff --git a/tests/qemucapabilitiesdata/caps_5.1.0.sparc.xml b/tests/qemuca= pabilitiesdata/caps_5.1.0.sparc.xml index e6a3ed5ec0..f01cd25278 100644 --- a/tests/qemucapabilitiesdata/caps_5.1.0.sparc.xml +++ b/tests/qemucapabilitiesdata/caps_5.1.0.sparc.xml @@ -86,6 +86,7 @@ + 5001000 0 0 diff --git a/tests/qemucapabilitiesdata/caps_5.1.0.x86_64.xml b/tests/qemuc= apabilitiesdata/caps_5.1.0.x86_64.xml index 070f64cb1c..f4446674a4 100644 --- a/tests/qemucapabilitiesdata/caps_5.1.0.x86_64.xml +++ b/tests/qemucapabilitiesdata/caps_5.1.0.x86_64.xml @@ -218,6 +218,7 @@ + 5001000 0 43100242 diff --git a/tests/qemucapabilitiesdata/caps_5.2.0.aarch64.xml b/tests/qemu= capabilitiesdata/caps_5.2.0.aarch64.xml index 8e17532f3a..f3e3640657 100644 --- a/tests/qemucapabilitiesdata/caps_5.2.0.aarch64.xml +++ b/tests/qemucapabilitiesdata/caps_5.2.0.aarch64.xml @@ -181,6 +181,7 @@ + 5002000 0 61700243 diff --git a/tests/qemucapabilitiesdata/caps_5.2.0.ppc64.xml b/tests/qemuca= pabilitiesdata/caps_5.2.0.ppc64.xml index df02e264d7..86d021c2fc 100644 --- a/tests/qemucapabilitiesdata/caps_5.2.0.ppc64.xml +++ b/tests/qemucapabilitiesdata/caps_5.2.0.ppc64.xml @@ -185,6 +185,7 @@ + 5002000 0 42900243 diff --git a/tests/qemucapabilitiesdata/caps_5.2.0.riscv64.xml b/tests/qemu= capabilitiesdata/caps_5.2.0.riscv64.xml index 7cb4383693..2079257759 100644 --- a/tests/qemucapabilitiesdata/caps_5.2.0.riscv64.xml +++ b/tests/qemucapabilitiesdata/caps_5.2.0.riscv64.xml @@ -171,6 +171,7 @@ + 5002000 0 0 diff --git a/tests/qemucapabilitiesdata/caps_5.2.0.s390x.xml b/tests/qemuca= pabilitiesdata/caps_5.2.0.s390x.xml index 30d10236e9..d800e35096 100644 --- a/tests/qemucapabilitiesdata/caps_5.2.0.s390x.xml +++ b/tests/qemucapabilitiesdata/caps_5.2.0.s390x.xml @@ -139,6 +139,7 @@ + 5002000 0 39100243 diff --git a/tests/qemucapabilitiesdata/caps_5.2.0.x86_64.xml b/tests/qemuc= apabilitiesdata/caps_5.2.0.x86_64.xml index cad4ed40e6..f189e299c3 100644 --- a/tests/qemucapabilitiesdata/caps_5.2.0.x86_64.xml +++ b/tests/qemucapabilitiesdata/caps_5.2.0.x86_64.xml @@ -222,6 +222,7 @@ + 5002000 0 43100243 diff --git a/tests/qemucapabilitiesdata/caps_6.0.0.aarch64.xml b/tests/qemu= capabilitiesdata/caps_6.0.0.aarch64.xml index 4b4cc2d3aa..e86a2cf5d6 100644 --- a/tests/qemucapabilitiesdata/caps_6.0.0.aarch64.xml +++ b/tests/qemucapabilitiesdata/caps_6.0.0.aarch64.xml @@ -189,6 +189,7 @@ + 6000000 0 61700242 diff --git a/tests/qemucapabilitiesdata/caps_6.0.0.s390x.xml b/tests/qemuca= pabilitiesdata/caps_6.0.0.s390x.xml index 06543071aa..bbc5825def 100644 --- a/tests/qemucapabilitiesdata/caps_6.0.0.s390x.xml +++ b/tests/qemucapabilitiesdata/caps_6.0.0.s390x.xml @@ -147,6 +147,7 @@ + 6000000 0 39100242 diff --git a/tests/qemucapabilitiesdata/caps_6.0.0.x86_64.xml b/tests/qemuc= apabilitiesdata/caps_6.0.0.x86_64.xml index 8c61bf8a84..496dd5564c 100644 --- a/tests/qemucapabilitiesdata/caps_6.0.0.x86_64.xml +++ b/tests/qemucapabilitiesdata/caps_6.0.0.x86_64.xml @@ -231,6 +231,7 @@ + 6000000 0 43100242 diff --git a/tests/qemucapabilitiesdata/caps_6.1.0.x86_64.xml b/tests/qemuc= apabilitiesdata/caps_6.1.0.x86_64.xml index afd8f606eb..fab5e40e35 100644 --- a/tests/qemucapabilitiesdata/caps_6.1.0.x86_64.xml +++ b/tests/qemucapabilitiesdata/caps_6.1.0.x86_64.xml @@ -236,6 +236,7 @@ + 6001000 0 43100243 diff --git a/tests/qemucapabilitiesdata/caps_6.2.0.aarch64.xml b/tests/qemu= capabilitiesdata/caps_6.2.0.aarch64.xml index 86fc46918f..a83b8c8d77 100644 --- a/tests/qemucapabilitiesdata/caps_6.2.0.aarch64.xml +++ b/tests/qemucapabilitiesdata/caps_6.2.0.aarch64.xml @@ -201,6 +201,7 @@ + 6001050 0 61700244 diff --git a/tests/qemucapabilitiesdata/caps_6.2.0.ppc64.xml b/tests/qemuca= pabilitiesdata/caps_6.2.0.ppc64.xml index 983b54430d..bfacae9b3b 100644 --- a/tests/qemucapabilitiesdata/caps_6.2.0.ppc64.xml +++ b/tests/qemucapabilitiesdata/caps_6.2.0.ppc64.xml @@ -196,6 +196,7 @@ + 6002000 0 42900244 diff --git a/tests/qemucapabilitiesdata/caps_6.2.0.x86_64.xml b/tests/qemuc= apabilitiesdata/caps_6.2.0.x86_64.xml index 19605d93ae..1293cd1bf1 100644 --- a/tests/qemucapabilitiesdata/caps_6.2.0.x86_64.xml +++ b/tests/qemucapabilitiesdata/caps_6.2.0.x86_64.xml @@ -238,6 +238,7 @@ + 6002000 0 43100244 diff --git a/tests/qemucapabilitiesdata/caps_7.0.0.aarch64.xml b/tests/qemu= capabilitiesdata/caps_7.0.0.aarch64.xml index e24e2235fb..16626a1342 100644 --- a/tests/qemucapabilitiesdata/caps_7.0.0.aarch64.xml +++ b/tests/qemucapabilitiesdata/caps_7.0.0.aarch64.xml @@ -209,6 +209,7 @@ + 6002092 0 61700243 diff --git a/tests/qemucapabilitiesdata/caps_7.0.0.ppc64.xml b/tests/qemuca= pabilitiesdata/caps_7.0.0.ppc64.xml index 83e0f50e3a..9f4242c5c0 100644 --- a/tests/qemucapabilitiesdata/caps_7.0.0.ppc64.xml +++ b/tests/qemucapabilitiesdata/caps_7.0.0.ppc64.xml @@ -213,6 +213,7 @@ + 7000000 0 42900243 diff --git a/tests/qemucapabilitiesdata/caps_7.0.0.x86_64.xml b/tests/qemuc= apabilitiesdata/caps_7.0.0.x86_64.xml index 05f844fd5b..1d003a6fb7 100644 --- a/tests/qemucapabilitiesdata/caps_7.0.0.x86_64.xml +++ b/tests/qemucapabilitiesdata/caps_7.0.0.x86_64.xml @@ -243,6 +243,7 @@ + 7000000 0 43100243 diff --git a/tests/qemucapabilitiesdata/caps_7.1.0.x86_64.xml b/tests/qemuc= apabilitiesdata/caps_7.1.0.x86_64.xml index bdc613a54a..80762d9fe9 100644 --- a/tests/qemucapabilitiesdata/caps_7.1.0.x86_64.xml +++ b/tests/qemucapabilitiesdata/caps_7.1.0.x86_64.xml @@ -243,6 +243,7 @@ + 7000050 0 43100244 --=20 2.26.2 From nobody Fri May 17 11:58:59 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 170.10.133.124 as permitted sender) client-ip=170.10.133.124; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-delivery-124.mimecast.com; Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=fail(p=none dis=none) header.from=suse.de Return-Path: Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by mx.zohomail.com with SMTPS id 1654593615277341.86407585010136; Tue, 7 Jun 2022 02:20:15 -0700 (PDT) Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-668-q3-hR3MYP_yYo5EdPIRwHQ-1; Tue, 07 Jun 2022 05:20:10 -0400 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.rdu2.redhat.com [10.11.54.6]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id AF81A29324A4; Tue, 7 Jun 2022 09:19:55 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (unknown [10.30.29.100]) by smtp.corp.redhat.com (Postfix) with ESMTP id 984BC2166B26; Tue, 7 Jun 2022 09:19:55 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (localhost [IPv6:::1]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id 302A51947B8F; Tue, 7 Jun 2022 09:19:55 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.rdu2.redhat.com [10.11.54.6]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id 91EB81947B9A for ; Tue, 7 Jun 2022 09:19:50 +0000 (UTC) Received: by smtp.corp.redhat.com (Postfix) id 729452166B2B; Tue, 7 Jun 2022 09:19:50 +0000 (UTC) Received: from mimecast-mx02.redhat.com (mimecast01.extmail.prod.ext.rdu2.redhat.com [10.11.55.17]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 6E3A62166B26 for ; Tue, 7 Jun 2022 09:19:50 +0000 (UTC) Received: from us-smtp-1.mimecast.com (us-smtp-delivery-1.mimecast.com [207.211.31.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 56C70905E81 for ; Tue, 7 Jun 2022 09:19:50 +0000 (UTC) Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.220.29]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-295-hbHCNMC1OPem43YyD94A-Q-1; Tue, 07 Jun 2022 05:19:48 -0400 Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id BEEEE1F956; Tue, 7 Jun 2022 09:19:46 +0000 (UTC) Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 7FE8C13638; Tue, 7 Jun 2022 09:19:46 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id 6C6tHTIYn2IoLwAAMHmgww (envelope-from ); Tue, 07 Jun 2022 09:19:46 +0000 X-MC-Unique: q3-hR3MYP_yYo5EdPIRwHQ-1 X-Original-To: libvir-list@listman.corp.redhat.com X-MC-Unique: hbHCNMC1OPem43YyD94A-Q-1 From: Claudio Fontana To: =?UTF-8?q?Daniel=20P=20=2E=20Berrang=C3=A9?= Subject: [libvirt RFCv11 19/33] qemu: saveimage: add multifd related fields to save format Date: Tue, 7 Jun 2022 11:19:22 +0200 Message-Id: <20220607091936.7948-20-cfontana@suse.de> In-Reply-To: <20220607091936.7948-1-cfontana@suse.de> References: <20220607091936.7948-1-cfontana@suse.de> MIME-Version: 1.0 X-Mimecast-Impersonation-Protect: Policy=CLT - Impersonation Protection Definition; Similar Internal Domain=false; Similar Monitored External Domain=false; Custom External Domain=false; Mimecast External Domain=false; Newly Observed Domain=false; Internal User Name=false; Custom Display Name List=false; Reply-to Address Mismatch=false; Targeted Threat Dictionary=false; Mimecast Threat Dictionary=false; Custom Threat Dictionary=false X-Scanned-By: MIMEDefang 2.78 on 10.11.54.6 X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: libvir-list@redhat.com, Claudio Fontana , "Dr . David Alan Gilbert" , Anthony Iliopoulos Errors-To: libvir-list-bounces@redhat.com Sender: "libvir-list" X-Scanned-By: MIMEDefang 2.78 on 10.11.54.6 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=libvir-list-bounces@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable X-ZM-MESSAGEID: 1654593615882100017 Content-Type: text/plain; charset="utf-8"; x-default="true" add both multifd compression and number of multifd channels Signed-off-by: Claudio Fontana --- src/qemu/qemu_saveimage.c | 17 +++++++++++++++++ src/qemu/qemu_saveimage.h | 4 +++- 2 files changed, 20 insertions(+), 1 deletion(-) diff --git a/src/qemu/qemu_saveimage.c b/src/qemu/qemu_saveimage.c index 8af9b92b35..7d8527596a 100644 --- a/src/qemu/qemu_saveimage.c +++ b/src/qemu/qemu_saveimage.c @@ -67,6 +67,23 @@ VIR_ENUM_IMPL(qemuSaveCompression, "lzop", ); =20 +typedef enum { + QEMU_SAVE_MULTIFD_COMP_NONE =3D 0, + QEMU_SAVE_MULTIFD_COMP_ZLIB =3D 1, + QEMU_SAVE_MULTIFD_COMP_ZSTD =3D 2, + + /* used for the on-disk format, do not change/re-use numbers */ + QEMU_SAVE_MULTIFD_COMP_LAST +} virQEMUSaveMultiFdComp; + +VIR_ENUM_DECL(qemuSaveMultiFdComp); +VIR_ENUM_IMPL(qemuSaveMultiFdComp, + QEMU_SAVE_MULTIFD_COMP_LAST, + "none", + "zlib", + "zstd", +); + static inline void qemuSaveImageBswapHeader(virQEMUSaveHeader *hdr) { diff --git a/src/qemu/qemu_saveimage.h b/src/qemu/qemu_saveimage.h index dcc9aa7d63..2af873dea9 100644 --- a/src/qemu/qemu_saveimage.h +++ b/src/qemu/qemu_saveimage.h @@ -42,7 +42,9 @@ struct _virQEMUSaveHeader { uint32_t was_running; /* 4 bytes */ uint32_t compressed; /* 4 bytes */ uint32_t cookieOffset; /* 4 bytes */ - uint32_t unused[14]; /* 56 bytes */ + uint16_t multifd_channels; /* 2 bytes */ + uint16_t multifd_comp; /* 2 bytes */ + uint32_t unused[13]; /* 52 bytes */ } ATTRIBUTE_PACKED; /* =3D 92 bytes */ =20 =20 --=20 2.26.2 From nobody Fri May 17 11:58:59 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 170.10.129.124 as permitted sender) client-ip=170.10.129.124; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-delivery-124.mimecast.com; Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=fail(p=none dis=none) header.from=suse.de Return-Path: Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by mx.zohomail.com with SMTPS id 1654593617445727.9334387762037; Tue, 7 Jun 2022 02:20:17 -0700 (PDT) Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-47-UsRO_JoLObq9o86DemwENA-1; Tue, 07 Jun 2022 05:20:12 -0400 Received: from smtp.corp.redhat.com (int-mx10.intmail.prod.int.rdu2.redhat.com [10.11.54.10]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 2A69C801228; Tue, 7 Jun 2022 09:20:06 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (unknown [10.30.29.100]) by smtp.corp.redhat.com (Postfix) with ESMTP id 12E7C416147; Tue, 7 Jun 2022 09:20:06 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (localhost [IPv6:::1]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id D68CA19452D2; Tue, 7 Jun 2022 09:20:05 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.rdu2.redhat.com [10.11.54.2]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id 65B281947B8F for ; Tue, 7 Jun 2022 09:20:03 +0000 (UTC) Received: by smtp.corp.redhat.com (Postfix) id 31A4040EC004; Tue, 7 Jun 2022 09:20:03 +0000 (UTC) Received: from mimecast-mx02.redhat.com (mimecast04.extmail.prod.ext.rdu2.redhat.com [10.11.55.20]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 2CFB540EC002 for ; Tue, 7 Jun 2022 09:20:03 +0000 (UTC) Received: from us-smtp-1.mimecast.com (us-smtp-delivery-1.mimecast.com [205.139.110.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 16D131066683 for ; Tue, 7 Jun 2022 09:20:03 +0000 (UTC) Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.220.29]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-232-EIp2rCjAMrKebcs1-dkhZQ-1; Tue, 07 Jun 2022 05:19:51 -0400 Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id 0DB4E1F97E; Tue, 7 Jun 2022 09:19:47 +0000 (UTC) Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id C5E3113638; Tue, 7 Jun 2022 09:19:46 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id AHm3LjIYn2IoLwAAMHmgww (envelope-from ); Tue, 07 Jun 2022 09:19:46 +0000 X-MC-Unique: UsRO_JoLObq9o86DemwENA-1 X-Original-To: libvir-list@listman.corp.redhat.com X-MC-Unique: EIp2rCjAMrKebcs1-dkhZQ-1 From: Claudio Fontana To: =?UTF-8?q?Daniel=20P=20=2E=20Berrang=C3=A9?= Subject: [libvirt RFCv11 20/33] qemu: migration_params: add APIs to set Int and Cap Date: Tue, 7 Jun 2022 11:19:23 +0200 Message-Id: <20220607091936.7948-21-cfontana@suse.de> In-Reply-To: <20220607091936.7948-1-cfontana@suse.de> References: <20220607091936.7948-1-cfontana@suse.de> MIME-Version: 1.0 X-Mimecast-Impersonation-Protect: Policy=CLT - Impersonation Protection Definition; Similar Internal Domain=false; Similar Monitored External Domain=false; Custom External Domain=false; Mimecast External Domain=false; Newly Observed Domain=false; Internal User Name=false; Custom Display Name List=false; Reply-to Address Mismatch=false; Targeted Threat Dictionary=false; Mimecast Threat Dictionary=false; Custom Threat Dictionary=false X-Scanned-By: MIMEDefang 2.84 on 10.11.54.2 X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: libvir-list@redhat.com, Claudio Fontana , "Dr . David Alan Gilbert" , Anthony Iliopoulos Errors-To: libvir-list-bounces@redhat.com Sender: "libvir-list" X-Scanned-By: MIMEDefang 2.85 on 10.11.54.10 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=libvir-list-bounces@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable X-ZM-MESSAGEID: 1654593618650100002 Content-Type: text/plain; charset="utf-8"; x-default="true" similarly to qemuMigrationParamsSetULL, we need to be able to set fields from qemu_saveimage. Signed-off-by: Claudio Fontana --- src/qemu/qemu_migration_params.c | 22 ++++++++++++++++++++++ src/qemu/qemu_migration_params.h | 9 +++++++++ 2 files changed, 31 insertions(+) diff --git a/src/qemu/qemu_migration_params.c b/src/qemu/qemu_migration_par= ams.c index 95fd773645..fad61631fa 100644 --- a/src/qemu/qemu_migration_params.c +++ b/src/qemu/qemu_migration_params.c @@ -1109,6 +1109,28 @@ qemuMigrationParamsFetch(virQEMUDriver *driver, } =20 =20 +void +qemuMigrationParamsSetCap(qemuMigrationParams *migParams, + qemuMigrationCapability mcap) +{ + ignore_value(virBitmapSetBit(migParams->caps, mcap)); +} + + +int +qemuMigrationParamsSetInt(qemuMigrationParams *migParams, + qemuMigrationParam param, + int value) +{ + if (qemuMigrationParamsCheckType(param, QEMU_MIGRATION_PARAM_TYPE_INT)= < 0) + return -1; + + migParams->params[param].value.i =3D value; + migParams->params[param].set =3D true; + return 0; +} + + int qemuMigrationParamsSetULL(qemuMigrationParams *migParams, qemuMigrationParam param, diff --git a/src/qemu/qemu_migration_params.h b/src/qemu/qemu_migration_par= ams.h index a0909b9f3d..036c4e7582 100644 --- a/src/qemu/qemu_migration_params.h +++ b/src/qemu/qemu_migration_params.h @@ -123,6 +123,15 @@ qemuMigrationParamsFetch(virQEMUDriver *driver, int asyncJob, qemuMigrationParams **migParams); =20 +void +qemuMigrationParamsSetCap(qemuMigrationParams *migParams, + qemuMigrationCapability mcap); + +int +qemuMigrationParamsSetInt(qemuMigrationParams *migParams, + qemuMigrationParam param, + int value); + int qemuMigrationParamsSetULL(qemuMigrationParams *migParams, qemuMigrationParam param, --=20 2.26.2 From nobody Fri May 17 11:58:59 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 170.10.133.124 as permitted sender) client-ip=170.10.133.124; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-delivery-124.mimecast.com; Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=fail(p=none dis=none) header.from=suse.de Return-Path: Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by mx.zohomail.com with SMTPS id 1654593630349281.8690397416583; Tue, 7 Jun 2022 02:20:30 -0700 (PDT) Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-114-ejGbg37jPKGqCsmEV1V4sQ-1; Tue, 07 Jun 2022 05:20:25 -0400 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.rdu2.redhat.com [10.11.54.6]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id E98C9811E80; Tue, 7 Jun 2022 09:20:22 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (unknown [10.30.29.100]) by smtp.corp.redhat.com (Postfix) with ESMTP id D29C22166B26; Tue, 7 Jun 2022 09:20:22 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (localhost [IPv6:::1]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id 9616119378D4; Tue, 7 Jun 2022 09:20:22 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx10.intmail.prod.int.rdu2.redhat.com [10.11.54.10]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id 170831947B9C for ; Tue, 7 Jun 2022 09:20:21 +0000 (UTC) Received: by smtp.corp.redhat.com (Postfix) id DBC9940336E; Tue, 7 Jun 2022 09:20:20 +0000 (UTC) Received: from mimecast-mx02.redhat.com (mimecast03.extmail.prod.ext.rdu2.redhat.com [10.11.55.19]) by smtp.corp.redhat.com (Postfix) with ESMTPS id D76D9492C3B for ; Tue, 7 Jun 2022 09:20:20 +0000 (UTC) Received: from us-smtp-1.mimecast.com (us-smtp-delivery-1.mimecast.com [205.139.110.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id BA2F2811E90 for ; Tue, 7 Jun 2022 09:20:20 +0000 (UTC) Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.220.29]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-44-lf0vBzoyNC6jyjvlEP527A-1; Tue, 07 Jun 2022 05:20:17 -0400 Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id 515EC1F980; Tue, 7 Jun 2022 09:19:47 +0000 (UTC) Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 1551D13638; Tue, 7 Jun 2022 09:19:47 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id IEWrAzMYn2IoLwAAMHmgww (envelope-from ); Tue, 07 Jun 2022 09:19:47 +0000 X-MC-Unique: ejGbg37jPKGqCsmEV1V4sQ-1 X-Original-To: libvir-list@listman.corp.redhat.com X-MC-Unique: lf0vBzoyNC6jyjvlEP527A-1 From: Claudio Fontana To: =?UTF-8?q?Daniel=20P=20=2E=20Berrang=C3=A9?= Subject: [libvirt RFCv11 21/33] qemu: migration: implement qemuMigrationSrcToFilesMultiFd for save Date: Tue, 7 Jun 2022 11:19:24 +0200 Message-Id: <20220607091936.7948-22-cfontana@suse.de> In-Reply-To: <20220607091936.7948-1-cfontana@suse.de> References: <20220607091936.7948-1-cfontana@suse.de> MIME-Version: 1.0 X-Mimecast-Impersonation-Protect: Policy=CLT - Impersonation Protection Definition; Similar Internal Domain=false; Similar Monitored External Domain=false; Custom External Domain=false; Mimecast External Domain=false; Newly Observed Domain=false; Internal User Name=false; Custom Display Name List=false; Reply-to Address Mismatch=false; Targeted Threat Dictionary=false; Mimecast Threat Dictionary=false; Custom Threat Dictionary=false X-Scanned-By: MIMEDefang 2.85 on 10.11.54.10 X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: libvir-list@redhat.com, Claudio Fontana , "Dr . David Alan Gilbert" , Anthony Iliopoulos Errors-To: libvir-list-bounces@redhat.com Sender: "libvir-list" X-Scanned-By: MIMEDefang 2.78 on 10.11.54.6 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=libvir-list-bounces@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable X-ZM-MESSAGEID: 1654593631186100012 Content-Type: text/plain; charset="utf-8"; x-default="true" implement a function similar to qemuMigrationSrcToFile that migrates to multiple files using QEMU multifd, and use it for VIR_DOMAIN_SAVE_PARALLEL saves. Signed-off-by: Claudio Fontana --- src/qemu/qemu_migration.c | 131 +++++++++++++++++++++++++------------- src/qemu/qemu_migration.h | 7 ++ src/qemu/qemu_saveimage.c | 8 ++- 3 files changed, 98 insertions(+), 48 deletions(-) diff --git a/src/qemu/qemu_migration.c b/src/qemu/qemu_migration.c index f130ccbe26..fa6cddcc59 100644 --- a/src/qemu/qemu_migration.c +++ b/src/qemu/qemu_migration.c @@ -5875,13 +5875,14 @@ qemuMigrationDstFinish(virQEMUDriver *driver, return dom; } =20 - /* Helper function called while vm is active. */ -int -qemuMigrationSrcToFile(virQEMUDriver *driver, virDomainObj *vm, - int fd, - virCommand *compressor, - virDomainAsyncJob asyncJob) +static int +qemuMigrationSrcToFileAux(virQEMUDriver *driver, virDomainObj *vm, + int fd, + virCommand *compressor, + virDomainAsyncJob asyncJob, + const char *sun_path, + int nchannels) { qemuDomainObjPrivate *priv =3D vm->privateData; bool bwParam =3D virQEMUCapsGet(priv->qemuCaps, QEMU_CAPS_MIGRATION_PA= RAM_BANDWIDTH); @@ -5892,24 +5893,26 @@ qemuMigrationSrcToFile(virQEMUDriver *driver, virDo= mainObj *vm, char *errbuf =3D NULL; virErrorPtr orig_err =3D NULL; g_autoptr(qemuMigrationParams) migParams =3D NULL; + bool needParams =3D (bwParam || sun_path); =20 + if (sun_path && !virQEMUCapsGet(priv->qemuCaps, QEMU_CAPS_MIGRATE_MULT= IFD)) { + virReportError(VIR_ERR_OPERATION_UNSUPPORTED, "%s", + _("QEMU does not seem to support multifd migration,= required for parallel migration to files")); + return -1; + } if (qemuMigrationSetDBusVMState(driver, vm) < 0) return -1; =20 /* Increase migration bandwidth to unlimited since target is a file. * Failure to change migration speed is not fatal. */ - if (bwParam) { - if (!(migParams =3D qemuMigrationParamsNew())) - return -1; + if (needParams && !((migParams =3D qemuMigrationParamsNew()))) + return -1; =20 + if (bwParam) { if (qemuMigrationParamsSetULL(migParams, QEMU_MIGRATION_PARAM_MAX_BANDWIDTH, QEMU_DOMAIN_MIG_BANDWIDTH_MAX * 1024= * 1024) < 0) return -1; - - if (qemuMigrationParamsApply(driver, vm, asyncJob, migParams) < 0) - return -1; - priv->migMaxBandwidth =3D QEMU_DOMAIN_MIG_BANDWIDTH_MAX; } else { if (qemuDomainObjEnterMonitorAsync(driver, vm, asyncJob) =3D=3D 0)= { @@ -5920,6 +5923,17 @@ qemuMigrationSrcToFile(virQEMUDriver *driver, virDom= ainObj *vm, } } =20 + if (sun_path) { + qemuMigrationParamsSetCap(migParams, QEMU_MIGRATION_CAP_MULTIFD); + if (qemuMigrationParamsSetInt(migParams, + QEMU_MIGRATION_PARAM_MULTIFD_CHANNEL= S, + nchannels) < 0) + return -1; + } + + if (needParams && qemuMigrationParamsApply(driver, vm, asyncJob, migPa= rams) < 0) + return -1; + if (!virDomainObjIsActive(vm)) { virReportError(VIR_ERR_INTERNAL_ERROR, "%s", _("guest unexpectedly quit")); @@ -5927,45 +5941,53 @@ qemuMigrationSrcToFile(virQEMUDriver *driver, virDo= mainObj *vm, return -1; } =20 - if (compressor && virPipe(pipeFD) < 0) + if (!sun_path && compressor && virPipe(pipeFD) < 0) return -1; =20 - /* All right! We can use fd migration, which means that qemu - * doesn't have to open() the file, so while we still have to - * grant SELinux access, we can do it on fd and avoid cleanup - * later, as well as skip futzing with cgroup. */ - if (qemuSecuritySetImageFDLabel(driver->securityManager, vm->def, - compressor ? pipeFD[1] : fd) < 0) - goto cleanup; - if (qemuDomainObjEnterMonitorAsync(driver, vm, asyncJob) < 0) goto cleanup; =20 - if (!compressor) { - rc =3D qemuMonitorMigrateToFd(priv->mon, - QEMU_MONITOR_MIGRATE_BACKGROUND, - fd); + if (sun_path) { + rc =3D qemuMonitorMigrateToSocket(priv->mon, + QEMU_MONITOR_MIGRATE_BACKGROUND, + sun_path); } else { - virCommandSetInputFD(compressor, pipeFD[0]); - virCommandSetOutputFD(compressor, &fd); - virCommandSetErrorBuffer(compressor, &errbuf); - virCommandDoAsyncIO(compressor); - if (virSetCloseExec(pipeFD[1]) < 0) { - virReportSystemError(errno, "%s", - _("Unable to set cloexec flag")); - qemuDomainObjExitMonitor(vm); - goto cleanup; - } - if (virCommandRunAsync(compressor, NULL) < 0) { - qemuDomainObjExitMonitor(vm); + /* + * All right! We can use fd migration, which means that qemu + * doesn't have to open() the file, so while we still have to + * grant SELinux access, we can do it on fd and avoid cleanup + * later, as well as skip futzing with cgroup. + */ + if (qemuSecuritySetImageFDLabel(driver->securityManager, vm->def, + compressor ? pipeFD[1] : fd) < 0) goto cleanup; + + if (!compressor) { + rc =3D qemuMonitorMigrateToFd(priv->mon, + QEMU_MONITOR_MIGRATE_BACKGROUND, + fd); + } else { + virCommandSetInputFD(compressor, pipeFD[0]); + virCommandSetOutputFD(compressor, &fd); + virCommandSetErrorBuffer(compressor, &errbuf); + virCommandDoAsyncIO(compressor); + if (virSetCloseExec(pipeFD[1]) < 0) { + virReportSystemError(errno, "%s", + _("Unable to set cloexec flag")); + qemuDomainObjExitMonitor(vm); + goto cleanup; + } + if (virCommandRunAsync(compressor, NULL) < 0) { + qemuDomainObjExitMonitor(vm); + goto cleanup; + } + rc =3D qemuMonitorMigrateToFd(priv->mon, + QEMU_MONITOR_MIGRATE_BACKGROUND, + pipeFD[1]); + if (VIR_CLOSE(pipeFD[0]) < 0 || + VIR_CLOSE(pipeFD[1]) < 0) + VIR_WARN("failed to close intermediate pipe"); } - rc =3D qemuMonitorMigrateToFd(priv->mon, - QEMU_MONITOR_MIGRATE_BACKGROUND, - pipeFD[1]); - if (VIR_CLOSE(pipeFD[0]) < 0 || - VIR_CLOSE(pipeFD[1]) < 0) - VIR_WARN("failed to close intermediate pipe"); } qemuDomainObjExitMonitor(vm); if (rc < 0) @@ -5986,7 +6008,7 @@ qemuMigrationSrcToFile(virQEMUDriver *driver, virDoma= inObj *vm, goto cleanup; } =20 - if (compressor && virCommandWait(compressor, NULL) < 0) + if (!sun_path && compressor && virCommandWait(compressor, NULL) < 0) goto cleanup; =20 qemuDomainEventEmitJobCompleted(driver, vm); @@ -6025,6 +6047,25 @@ qemuMigrationSrcToFile(virQEMUDriver *driver, virDom= ainObj *vm, return ret; } =20 +int +qemuMigrationSrcToFile(virQEMUDriver *driver, virDomainObj *vm, + int fd, + virCommand *compressor, + virDomainAsyncJob asyncJob) +{ + return qemuMigrationSrcToFileAux(driver, vm, fd, compressor, + asyncJob, NULL, -1); +} + +int +qemuMigrationSrcToFilesMultiFd(virQEMUDriver *driver, virDomainObj *vm, + virDomainAsyncJob asyncJob, + const char *sun_path, + int nchannels) +{ + return qemuMigrationSrcToFileAux(driver, vm, -1, NULL, + asyncJob, sun_path, nchannels); +} =20 int qemuMigrationSrcCancel(virQEMUDriver *driver, diff --git a/src/qemu/qemu_migration.h b/src/qemu/qemu_migration.h index a8afa66119..ddc8e65489 100644 --- a/src/qemu/qemu_migration.h +++ b/src/qemu/qemu_migration.h @@ -213,6 +213,13 @@ qemuMigrationSrcToFile(virQEMUDriver *driver, virDomainAsyncJob asyncJob) ATTRIBUTE_NONNULL(1) ATTRIBUTE_NONNULL(2) G_GNUC_WARN_UNUSED_RESULT; =20 +int +qemuMigrationSrcToFilesMultiFd(virQEMUDriver *driver, virDomainObj *vm, + virDomainAsyncJob asyncJob, + const char *sun_path, + int nchannels) + ATTRIBUTE_NONNULL(1) ATTRIBUTE_NONNULL(2) G_GNUC_WARN_UNUSED_RESULT; + int qemuMigrationSrcCancel(virQEMUDriver *driver, virDomainObj *vm); diff --git a/src/qemu/qemu_saveimage.c b/src/qemu/qemu_saveimage.c index 7d8527596a..8abb919459 100644 --- a/src/qemu/qemu_saveimage.c +++ b/src/qemu/qemu_saveimage.c @@ -578,8 +578,11 @@ qemuSaveImageCreate(virQEMUDriver *driver, oflags |=3D O_DIRECT; } =20 - if (virQEMUSaveFdInit(&saveFd, path, oflags, cfg, false) < 0) + if (virQEMUSaveFdInit(&saveFd, path, oflags, cfg, flags & VIR_DOMAIN_S= AVE_PARALLEL) < 0) goto cleanup; + if (nconn > 0) { + data->header.multifd_channels =3D nconn; + } if (qemuSecuritySetImageFDLabel(driver->securityManager, vm->def, save= Fd.fd) < 0) goto cleanup; if (virQEMUSaveDataWrite(data, saveFd.fd, saveFd.path) < 0) @@ -610,8 +613,7 @@ qemuSaveImageCreate(virQEMUDriver *driver, goto cleanup; if (chown(sun_path, cfg->user, cfg->group) < 0) goto cleanup; - /* still using single fd migration for now */ - if (qemuMigrationSrcToFile(driver, vm, saveFd.fd, compressor, asyn= cJob) < 0) + if (qemuMigrationSrcToFilesMultiFd(driver, vm, asyncJob, sun_path,= nconn) < 0) goto cleanup; } else { /* Perform non-parallel migration to file */ --=20 2.26.2 From nobody Fri May 17 11:58:59 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 170.10.129.124 as permitted sender) client-ip=170.10.129.124; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-delivery-124.mimecast.com; Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=fail(p=none dis=none) header.from=suse.de Return-Path: Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by mx.zohomail.com with SMTPS id 1654593617467660.3824291998017; Tue, 7 Jun 2022 02:20:17 -0700 (PDT) Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-323--8dhOQQwNTytgHiOlF1vAQ-1; Tue, 07 Jun 2022 05:20:04 -0400 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.rdu2.redhat.com [10.11.54.2]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 4AD2E833968; Tue, 7 Jun 2022 09:19:52 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (unknown [10.30.29.100]) by smtp.corp.redhat.com (Postfix) with ESMTP id 35B1440EC002; Tue, 7 Jun 2022 09:19:52 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (localhost [IPv6:::1]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id 0905619451F0; Tue, 7 Jun 2022 09:19:52 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.rdu2.redhat.com [10.11.54.8]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id 5866C19452D2 for ; Tue, 7 Jun 2022 09:19:51 +0000 (UTC) Received: by smtp.corp.redhat.com (Postfix) id 35E5BC27E8F; Tue, 7 Jun 2022 09:19:51 +0000 (UTC) Received: from mimecast-mx02.redhat.com (mimecast05.extmail.prod.ext.rdu2.redhat.com [10.11.55.21]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 317A2C27E92 for ; Tue, 7 Jun 2022 09:19:51 +0000 (UTC) Received: from us-smtp-1.mimecast.com (us-smtp-1.mimecast.com [207.211.31.81]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 154978339A0 for ; Tue, 7 Jun 2022 09:19:51 +0000 (UTC) Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.220.28]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-9-sB_76tQjNbWORJtFBWjcqg-1; Tue, 07 Jun 2022 05:19:49 -0400 Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id 9690121B2F; Tue, 7 Jun 2022 09:19:47 +0000 (UTC) Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 59E5B13638; Tue, 7 Jun 2022 09:19:47 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id 8BBPFDMYn2IoLwAAMHmgww (envelope-from ); Tue, 07 Jun 2022 09:19:47 +0000 X-MC-Unique: -8dhOQQwNTytgHiOlF1vAQ-1 X-Original-To: libvir-list@listman.corp.redhat.com X-MC-Unique: sB_76tQjNbWORJtFBWjcqg-1 From: Claudio Fontana To: =?UTF-8?q?Daniel=20P=20=2E=20Berrang=C3=A9?= Subject: [libvirt RFCv11 22/33] qemu: add parameter to qemuMigrationDstRun to skip waiting Date: Tue, 7 Jun 2022 11:19:25 +0200 Message-Id: <20220607091936.7948-23-cfontana@suse.de> In-Reply-To: <20220607091936.7948-1-cfontana@suse.de> References: <20220607091936.7948-1-cfontana@suse.de> MIME-Version: 1.0 X-Mimecast-Impersonation-Protect: Policy=CLT - Impersonation Protection Definition; Similar Internal Domain=false; Similar Monitored External Domain=false; Custom External Domain=false; Mimecast External Domain=false; Newly Observed Domain=false; Internal User Name=false; Custom Display Name List=false; Reply-to Address Mismatch=false; Targeted Threat Dictionary=false; Mimecast Threat Dictionary=false; Custom Threat Dictionary=false X-Scanned-By: MIMEDefang 2.85 on 10.11.54.8 X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: libvir-list@redhat.com, Claudio Fontana , "Dr . David Alan Gilbert" , Anthony Iliopoulos Errors-To: libvir-list-bounces@redhat.com Sender: "libvir-list" X-Scanned-By: MIMEDefang 2.84 on 10.11.54.2 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=libvir-list-bounces@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable X-ZM-MESSAGEID: 1654593618646100001 Content-Type: text/plain; charset="utf-8"; x-default="true" The distinction on whether to wait for the migration completion or not was made on the async job type, but with the future addition of multifd migration from files, we need a way to avoid waiting, so we can prepare multifd migration parameters before starting the transfers. Adapt all callers. Signed-off-by: Claudio Fontana --- src/qemu/qemu_driver.c | 8 ++++---- src/qemu/qemu_migration.c | 18 ++++++++++-------- src/qemu/qemu_migration.h | 3 ++- src/qemu/qemu_process.c | 3 ++- src/qemu/qemu_process.h | 5 +++-- src/qemu/qemu_saveimage.c | 4 +++- src/qemu/qemu_saveimage.h | 1 + src/qemu/qemu_snapshot.c | 4 ++-- 8 files changed, 27 insertions(+), 19 deletions(-) diff --git a/src/qemu/qemu_driver.c b/src/qemu/qemu_driver.c index d403ff6bf2..3d2b5d78e6 100644 --- a/src/qemu/qemu_driver.c +++ b/src/qemu/qemu_driver.c @@ -1639,7 +1639,7 @@ static virDomainPtr qemuDomainCreateXML(virConnectPtr= conn, } =20 if (qemuProcessStart(conn, driver, vm, NULL, VIR_ASYNC_JOB_START, - NULL, -1, NULL, NULL, + NULL, -1, NULL, false, NULL, VIR_NETDEV_VPORT_PROFILE_OP_CREATE, start_flags) < 0) { virDomainAuditStart(vm, "booted", false); @@ -5940,7 +5940,7 @@ qemuDomainRestoreInternal(virConnectPtr conn, goto cleanup; =20 ret =3D qemuSaveImageStartVM(conn, driver, vm, &saveFd.fd, data, path, - false, reset_nvram, VIR_ASYNC_JOB_START); + false, reset_nvram, true, VIR_ASYNC_JOB_STA= RT); =20 qemuProcessEndJob(vm); =20 @@ -6261,7 +6261,7 @@ qemuDomainObjRestore(virConnectPtr conn, virDomainObjAssignDef(vm, &def, true, NULL); =20 ret =3D qemuSaveImageStartVM(conn, driver, vm, &saveFd.fd, data, path, - start_paused, reset_nvram, asyncJob); + start_paused, reset_nvram, true, asyncJob); =20 cleanup: virQEMUSaveDataFree(data); @@ -6515,7 +6515,7 @@ qemuDomainObjStart(virConnectPtr conn, } =20 ret =3D qemuProcessStart(conn, driver, vm, NULL, asyncJob, - NULL, -1, NULL, NULL, + NULL, -1, NULL, false, NULL, VIR_NETDEV_VPORT_PROFILE_OP_CREATE, start_flags= ); virDomainAuditStart(vm, "booted", ret >=3D 0); if (ret >=3D 0) { diff --git a/src/qemu/qemu_migration.c b/src/qemu/qemu_migration.c index fa6cddcc59..439dd7478d 100644 --- a/src/qemu/qemu_migration.c +++ b/src/qemu/qemu_migration.c @@ -2122,7 +2122,8 @@ int qemuMigrationDstRun(virQEMUDriver *driver, virDomainObj *vm, const char *uri, - virDomainAsyncJob asyncJob) + virDomainAsyncJob asyncJob, + bool wait) { qemuDomainObjPrivate *priv =3D vm->privateData; int rv; @@ -2143,14 +2144,15 @@ qemuMigrationDstRun(virQEMUDriver *driver, if (rv < 0) return -1; =20 - if (asyncJob =3D=3D VIR_ASYNC_JOB_MIGRATION_IN) { - /* qemuMigrationDstWaitForCompletion is called from the Finish pha= se */ - return 0; + if (wait) { + /* + * the Migration Finish phase, as well as the multifd load from fi= les, + * need to call qemuMigrationDstWaitForCompletion separately, not = here. + */ + if (qemuMigrationDstWaitForCompletion(driver, vm, asyncJob, false)= < 0) + return -1; } =20 - if (qemuMigrationDstWaitForCompletion(driver, vm, asyncJob, false) < 0) - return -1; - return 0; } =20 @@ -3024,7 +3026,7 @@ qemuMigrationDstPrepareAny(virQEMUDriver *driver, } =20 if (qemuMigrationDstRun(driver, vm, incoming->uri, - VIR_ASYNC_JOB_MIGRATION_IN) < 0) + VIR_ASYNC_JOB_MIGRATION_IN, false) < 0) goto stopjob; =20 if (qemuProcessFinishStartup(driver, vm, VIR_ASYNC_JOB_MIGRATION_IN, diff --git a/src/qemu/qemu_migration.h b/src/qemu/qemu_migration.h index ddc8e65489..c3c48c19c0 100644 --- a/src/qemu/qemu_migration.h +++ b/src/qemu/qemu_migration.h @@ -255,7 +255,8 @@ int qemuMigrationDstRun(virQEMUDriver *driver, virDomainObj *vm, const char *uri, - virDomainAsyncJob asyncJob); + virDomainAsyncJob asyncJob, + bool wait); =20 void qemuMigrationAnyPostcopyFailed(virQEMUDriver *driver, diff --git a/src/qemu/qemu_process.c b/src/qemu/qemu_process.c index 1593ca7933..40870b8f7a 100644 --- a/src/qemu/qemu_process.c +++ b/src/qemu/qemu_process.c @@ -7742,6 +7742,7 @@ qemuProcessStart(virConnectPtr conn, const char *migrateFrom, int migrateFd, const char *migratePath, + bool wait_incoming, virDomainMomentObj *snapshot, virNetDevVPortProfileOp vmop, unsigned int flags) @@ -7804,7 +7805,7 @@ qemuProcessStart(virConnectPtr conn, relabel =3D true; =20 if (incoming) { - if (qemuMigrationDstRun(driver, vm, incoming->uri, asyncJob) < 0) + if (qemuMigrationDstRun(driver, vm, incoming->uri, asyncJob, wait_= incoming) < 0) goto stop; } else { /* Refresh state of devices from QEMU. During migration this happe= ns diff --git a/src/qemu/qemu_process.h b/src/qemu/qemu_process.h index 9866d2bf35..aad34ae0d8 100644 --- a/src/qemu/qemu_process.h +++ b/src/qemu/qemu_process.h @@ -86,8 +86,9 @@ int qemuProcessStart(virConnectPtr conn, virCPUDef *updatedCPU, virDomainAsyncJob asyncJob, const char *migrateFrom, - int stdin_fd, - const char *stdin_path, + int fd, + const char *migratePath, + bool wait_incoming, virDomainMomentObj *snapshot, virNetDevVPortProfileOp vmop, unsigned int flags); diff --git a/src/qemu/qemu_saveimage.c b/src/qemu/qemu_saveimage.c index 8abb919459..9c86e20f30 100644 --- a/src/qemu/qemu_saveimage.c +++ b/src/qemu/qemu_saveimage.c @@ -780,6 +780,7 @@ qemuSaveImageStartVM(virConnectPtr conn, const char *path, bool start_paused, bool reset_nvram, + bool wait_incoming, virDomainAsyncJob asyncJob) { qemuDomainObjPrivate *priv =3D vm->privateData; @@ -834,7 +835,8 @@ qemuSaveImageStartVM(virConnectPtr conn, priv->disableSlirp =3D true; =20 if (qemuProcessStart(conn, driver, vm, cookie ? cookie->cpu : NULL, - asyncJob, "stdio", *fd, path, NULL, + asyncJob, "stdio", *fd, path, wait_incoming, + NULL, VIR_NETDEV_VPORT_PROFILE_OP_RESTORE, start_flags) =3D=3D 0) started =3D true; diff --git a/src/qemu/qemu_saveimage.h b/src/qemu/qemu_saveimage.h index 2af873dea9..91ea8dcbf2 100644 --- a/src/qemu/qemu_saveimage.h +++ b/src/qemu/qemu_saveimage.h @@ -97,6 +97,7 @@ qemuSaveImageStartVM(virConnectPtr conn, const char *path, bool start_paused, bool reset_nvram, + bool wait_incoming, virDomainAsyncJob asyncJob) ATTRIBUTE_NONNULL(4) ATTRIBUTE_NONNULL(5) ATTRIBUTE_NONNULL(6); =20 diff --git a/src/qemu/qemu_snapshot.c b/src/qemu/qemu_snapshot.c index 3437c3e0cb..dec653c6d3 100644 --- a/src/qemu/qemu_snapshot.c +++ b/src/qemu/qemu_snapshot.c @@ -2092,7 +2092,7 @@ qemuSnapshotRevertActive(virDomainObj *vm, =20 rc =3D qemuProcessStart(snapshot->domain->conn, driver, vm, cookie ? cookie->cpu : NULL, - VIR_ASYNC_JOB_START, NULL, -1, NULL, snap, + VIR_ASYNC_JOB_START, NULL, -1, NULL, false, snap, VIR_NETDEV_VPORT_PROFILE_OP_CREATE, start_flags); virDomainAuditStart(vm, "from-snapshot", rc >=3D 0); @@ -2215,7 +2215,7 @@ qemuSnapshotRevertInactive(virDomainObj *vm, start_flags |=3D paused ? VIR_QEMU_PROCESS_START_PAUSED : 0; =20 rc =3D qemuProcessStart(snapshot->domain->conn, driver, vm, NULL, - VIR_ASYNC_JOB_START, NULL, -1, NULL, NULL, + VIR_ASYNC_JOB_START, NULL, -1, NULL, false, = NULL, VIR_NETDEV_VPORT_PROFILE_OP_CREATE, start_flags); virDomainAuditStart(vm, "from-snapshot", rc >=3D 0); --=20 2.26.2 From nobody Fri May 17 11:58:59 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 170.10.129.124 as permitted sender) client-ip=170.10.129.124; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-delivery-124.mimecast.com; Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=fail(p=none dis=none) header.from=suse.de Return-Path: Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by mx.zohomail.com with SMTPS id 1654593630252135.28239190373245; Tue, 7 Jun 2022 02:20:30 -0700 (PDT) Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-619-2IrKNUDTMnyt7OEUs5SMbw-1; Tue, 07 Jun 2022 05:20:23 -0400 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.rdu2.redhat.com [10.11.54.1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 8CB471C0E345; Tue, 7 Jun 2022 09:20:03 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (unknown [10.30.29.100]) by smtp.corp.redhat.com (Postfix) with ESMTP id 73D3F406AD4A; Tue, 7 Jun 2022 09:20:03 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (localhost [IPv6:::1]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id 38CF019452D2; Tue, 7 Jun 2022 09:20:03 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx10.intmail.prod.int.rdu2.redhat.com [10.11.54.10]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id 1E0A419452D2 for ; Tue, 7 Jun 2022 09:20:01 +0000 (UTC) Received: by smtp.corp.redhat.com (Postfix) id F2E4440336E; Tue, 7 Jun 2022 09:20:00 +0000 (UTC) Received: from mimecast-mx02.redhat.com (mimecast04.extmail.prod.ext.rdu2.redhat.com [10.11.55.20]) by smtp.corp.redhat.com (Postfix) with ESMTPS id EEB09492C3B for ; Tue, 7 Jun 2022 09:20:00 +0000 (UTC) Received: from us-smtp-1.mimecast.com (us-smtp-2.mimecast.com [205.139.110.61]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id D3FA610665A4 for ; Tue, 7 Jun 2022 09:20:00 +0000 (UTC) Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.220.28]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-414-vorV-vQ_MM2YbPF2jpDnsA-1; Tue, 07 Jun 2022 05:19:56 -0400 Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id D980121B3F; Tue, 7 Jun 2022 09:19:47 +0000 (UTC) Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 9F23013638; Tue, 7 Jun 2022 09:19:47 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id mG1SJTMYn2IoLwAAMHmgww (envelope-from ); Tue, 07 Jun 2022 09:19:47 +0000 X-MC-Unique: 2IrKNUDTMnyt7OEUs5SMbw-1 X-Original-To: libvir-list@listman.corp.redhat.com X-MC-Unique: vorV-vQ_MM2YbPF2jpDnsA-1 From: Claudio Fontana To: =?UTF-8?q?Daniel=20P=20=2E=20Berrang=C3=A9?= Subject: [libvirt RFCv11 23/33] qemu: implement qemuSaveImageLoadMultiFd for restore Date: Tue, 7 Jun 2022 11:19:26 +0200 Message-Id: <20220607091936.7948-24-cfontana@suse.de> In-Reply-To: <20220607091936.7948-1-cfontana@suse.de> References: <20220607091936.7948-1-cfontana@suse.de> MIME-Version: 1.0 X-Mimecast-Impersonation-Protect: Policy=CLT - Impersonation Protection Definition; Similar Internal Domain=false; Similar Monitored External Domain=false; Custom External Domain=false; Mimecast External Domain=false; Newly Observed Domain=false; Internal User Name=false; Custom Display Name List=false; Reply-to Address Mismatch=false; Targeted Threat Dictionary=false; Mimecast Threat Dictionary=false; Custom Threat Dictionary=false X-Scanned-By: MIMEDefang 2.85 on 10.11.54.10 X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: libvir-list@redhat.com, Claudio Fontana , "Dr . David Alan Gilbert" , Anthony Iliopoulos Errors-To: libvir-list-bounces@redhat.com Sender: "libvir-list" X-Scanned-By: MIMEDefang 2.84 on 10.11.54.1 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=libvir-list-bounces@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable X-ZM-MESSAGEID: 1654593631110100010 Content-Type: text/plain; charset="utf-8"; x-default="true" use multifd to restore parallel images, if VIR_DOMAIN_SAVE_PARALLEL is enabled. Signed-off-by: Claudio Fontana --- src/qemu/qemu_driver.c | 16 +++++- src/qemu/qemu_migration.c | 2 +- src/qemu/qemu_migration.h | 6 ++ src/qemu/qemu_saveimage.c | 114 +++++++++++++++++++++++++++++++++++++- src/qemu/qemu_saveimage.h | 8 ++- 5 files changed, 139 insertions(+), 7 deletions(-) diff --git a/src/qemu/qemu_driver.c b/src/qemu/qemu_driver.c index 3d2b5d78e6..c3b2afe0d8 100644 --- a/src/qemu/qemu_driver.c +++ b/src/qemu/qemu_driver.c @@ -5883,7 +5883,7 @@ qemuDomainRestoreInternal(virConnectPtr conn, } oflags |=3D O_DIRECT; } - if (virQEMUSaveFdInit(&saveFd, path, oflags, cfg, false) < 0) + if (virQEMUSaveFdInit(&saveFd, path, oflags, cfg, flags & VIR_DOMAIN_S= AVE_PARALLEL) < 0) return -1; if (qemuSaveImageOpen(driver, NULL, &def, &data, false, &saveFd) < 0) goto cleanup; @@ -5939,8 +5939,18 @@ qemuDomainRestoreInternal(virConnectPtr conn, flags) < 0) goto cleanup; =20 - ret =3D qemuSaveImageStartVM(conn, driver, vm, &saveFd.fd, data, path, - false, reset_nvram, true, VIR_ASYNC_JOB_STA= RT); + if (flags & VIR_DOMAIN_SAVE_PARALLEL) { + ret =3D qemuSaveImageLoadMultiFd(conn, vm, data, reset_nvram, + &saveFd, VIR_ASYNC_JOB_START); + + } else if (data->header.multifd_channels !=3D 0) { + virReportError(VIR_ERR_OPERATION_FAILED, "%s", + _("save file seems to contain multifd channels info= rmation, and restore is not flagged as 'parallel'")); + ret =3D -1; + } else { + ret =3D qemuSaveImageStartVM(conn, driver, vm, &saveFd.fd, data, p= ath, + false, reset_nvram, true, VIR_ASYNC_JOB= _START); + } =20 qemuProcessEndJob(vm); =20 diff --git a/src/qemu/qemu_migration.c b/src/qemu/qemu_migration.c index 439dd7478d..c214f88dd1 100644 --- a/src/qemu/qemu_migration.c +++ b/src/qemu/qemu_migration.c @@ -1920,7 +1920,7 @@ qemuMigrationSrcWaitForCompletion(virQEMUDriver *driv= er, } =20 =20 -static int +int qemuMigrationDstWaitForCompletion(virQEMUDriver *driver, virDomainObj *vm, virDomainAsyncJob asyncJob, diff --git a/src/qemu/qemu_migration.h b/src/qemu/qemu_migration.h index c3c48c19c0..38f4877cf0 100644 --- a/src/qemu/qemu_migration.h +++ b/src/qemu/qemu_migration.h @@ -191,6 +191,12 @@ qemuMigrationDstFinish(virQEMUDriver *driver, int retcode, bool v3proto); =20 +int +qemuMigrationDstWaitForCompletion(virQEMUDriver *driver, + virDomainObj *vm, + virDomainAsyncJob asyncJob, + bool postcopy); + int qemuMigrationSrcConfirm(virQEMUDriver *driver, virDomainObj *vm, diff --git a/src/qemu/qemu_saveimage.c b/src/qemu/qemu_saveimage.c index 9c86e20f30..2ed02d31de 100644 --- a/src/qemu/qemu_saveimage.c +++ b/src/qemu/qemu_saveimage.c @@ -643,6 +643,105 @@ qemuSaveImageCreate(virQEMUDriver *driver, } =20 =20 +int qemuSaveImageLoadMultiFd(virConnectPtr conn, virDomainObj *vm, + virQEMUSaveData *data, bool reset_nvram, + virQEMUSaveFd *saveFd, virDomainAsyncJob asyn= cJob) +{ + virQEMUDriver *driver =3D conn->privateData; + qemuDomainObjPrivate *priv =3D vm->privateData; + g_autoptr(virQEMUDriverConfig) cfg =3D virQEMUDriverGetConfig(driver); + g_autoptr(virCommand) cmd =3D NULL; + g_autofree char *helper_path =3D NULL; + g_autofree char *sun_path =3D g_strdup_printf("%s/restore-multifd.sock= ", cfg->saveDir); + bool qemu_started =3D false; + int ret =3D -1; + + if (!(helper_path =3D virFileFindResource("libvirt_multifd_helper", + abs_top_builddir "/src", + LIBEXECDIR))) + goto cleanup; + cmd =3D virCommandNewArgList(helper_path, sun_path, saveFd->path, NULL= ); + virCommandAddArgFormat(cmd, "%d", saveFd->nchannels); + virCommandAddArgFormat(cmd, "%d", saveFd->fd); + virCommandPassFD(cmd, saveFd->fd, 0); + + if (qemuSaveImageStartVM(conn, driver, vm, NULL, data, sun_path, + false, reset_nvram, false, asyncJob) < 0) + goto cleanup; + + qemu_started =3D true; + + if (!virQEMUCapsGet(priv->qemuCaps, QEMU_CAPS_MIGRATE_MULTIFD)) { + virReportError(VIR_ERR_OPERATION_UNSUPPORTED, "%s", + _("QEMU does not seem to support multifd migration"= )); + goto cleanup; + } else { + g_autoptr(qemuMigrationParams) migParams =3D qemuMigrationParamsNe= w(); + bool bwParam =3D virQEMUCapsGet(priv->qemuCaps, QEMU_CAPS_MIGRATIO= N_PARAM_BANDWIDTH); + + if (bwParam) { + if (qemuMigrationParamsSetULL(migParams, + QEMU_MIGRATION_PARAM_MAX_BANDWID= TH, + QEMU_DOMAIN_MIG_BANDWIDTH_MAX * = 1024 * 1024) < 0) + goto cleanup; + priv->migMaxBandwidth =3D QEMU_DOMAIN_MIG_BANDWIDTH_MAX; + } else { + if (qemuDomainObjEnterMonitorAsync(driver, vm, asyncJob) =3D= =3D 0) { + qemuMonitorSetMigrationSpeed(priv->mon, + QEMU_DOMAIN_MIG_BANDWIDTH_MAX= ); + priv->migMaxBandwidth =3D QEMU_DOMAIN_MIG_BANDWIDTH_MAX; + qemuDomainObjExitMonitor(vm); + } + } + qemuMigrationParamsSetCap(migParams, QEMU_MIGRATION_CAP_MULTIFD); + if (qemuMigrationParamsSetInt(migParams, + QEMU_MIGRATION_PARAM_MULTIFD_CHANNEL= S, + saveFd->nchannels) < 0) + goto cleanup; + if (qemuMigrationParamsApply(driver, vm, asyncJob, migParams) < 0) + goto cleanup; + + if (!virDomainObjIsActive(vm)) { + virReportError(VIR_ERR_INTERNAL_ERROR, "%s", + _("guest unexpectedly quit")); + goto cleanup; + } + /* multifd helper can now connect, then wait for migration to comp= lete */ + if (virCommandRunAsync(cmd, NULL) < 0) + goto cleanup; + + if (qemuMigrationDstWaitForCompletion(driver, vm, asyncJob, false)= < 0) + goto cleanup; + + if (qemuProcessRefreshState(driver, vm, asyncJob) < 0) + goto cleanup; + + /* run 'cont' on the destination */ + if (qemuProcessStartCPUs(driver, vm, + VIR_DOMAIN_RUNNING_RESTORED, + asyncJob) < 0) { + if (virGetLastErrorCode() =3D=3D VIR_ERR_OK) + virReportError(VIR_ERR_OPERATION_FAILED, + "%s", _("failed to resume domain")); + goto cleanup; + } + if (virDomainObjSave(vm, driver->xmlopt, cfg->stateDir) < 0) { + VIR_WARN("Failed to save status on vm %s", vm->def->name); + goto cleanup; + } + } + qemuDomainEventEmitJobCompleted(driver, vm); + ret =3D 0; + + cleanup: + if (ret < 0 && qemu_started) { + qemuProcessStop(driver, vm, VIR_DOMAIN_SHUTOFF_FAILED, + asyncJob, VIR_QEMU_PROCESS_STOP_MIGRATED); + } + return ret; +} + + /* qemuSaveImageGetCompressionProgram: * @imageFormat: String representation from qemu.conf for the compression * image format being used (dump, save, or snapshot). @@ -758,6 +857,10 @@ qemuSaveImageOpen(virQEMUDriver *driver, } return ret; } + if (data->header.multifd_channels > 0) { + if (virQEMUSaveFdAddChannels(saveFd, data->header.multifd_channels= ) < 0) + return -1; + } =20 /* Create a domain from this XML */ if (!(def =3D virDomainDefParseString(data->xml, driver->xmlopt, qemuC= aps, @@ -788,6 +891,7 @@ qemuSaveImageStartVM(virConnectPtr conn, bool started =3D false; virObjectEvent *event; VIR_AUTOCLOSE intermediatefd =3D -1; + g_autofree char *migrate_from =3D NULL; g_autoptr(virCommand) cmd =3D NULL; g_autofree char *errbuf =3D NULL; g_autoptr(virQEMUDriverConfig) cfg =3D virQEMUDriverGetConfig(driver); @@ -834,8 +938,14 @@ qemuSaveImageStartVM(virConnectPtr conn, if (cookie && !cookie->slirpHelper) priv->disableSlirp =3D true; =20 + if (fd) { + migrate_from =3D g_strdup("stdio"); + } else { + migrate_from =3D g_strdup_printf("unix://%s", path); + } + if (qemuProcessStart(conn, driver, vm, cookie ? cookie->cpu : NULL, - asyncJob, "stdio", *fd, path, wait_incoming, + asyncJob, migrate_from, fd ? *fd : -1, path, wait= _incoming, NULL, VIR_NETDEV_VPORT_PROFILE_OP_RESTORE, start_flags) =3D=3D 0) @@ -859,7 +969,7 @@ qemuSaveImageStartVM(virConnectPtr conn, VIR_DEBUG("Decompression binary stderr: %s", NULLSTR(errbuf)); virErrorRestore(&orig_err); } - if (VIR_CLOSE(*fd) < 0) { + if (fd && VIR_CLOSE(*fd) < 0) { virReportSystemError(errno, _("cannot close file: %s"), path); rc =3D -1; } diff --git a/src/qemu/qemu_saveimage.h b/src/qemu/qemu_saveimage.h index 91ea8dcbf2..158bcaff50 100644 --- a/src/qemu/qemu_saveimage.h +++ b/src/qemu/qemu_saveimage.h @@ -99,7 +99,7 @@ qemuSaveImageStartVM(virConnectPtr conn, bool reset_nvram, bool wait_incoming, virDomainAsyncJob asyncJob) - ATTRIBUTE_NONNULL(4) ATTRIBUTE_NONNULL(5) ATTRIBUTE_NONNULL(6); + ATTRIBUTE_NONNULL(5) ATTRIBUTE_NONNULL(6); =20 int qemuSaveImageOpen(virQEMUDriver *driver, @@ -117,6 +117,12 @@ qemuSaveImageGetCompressionProgram(const char *imageFo= rmat, bool use_raw_on_fail) ATTRIBUTE_NONNULL(2); =20 +int qemuSaveImageLoadMultiFd(virConnectPtr conn, virDomainObj *vm, + virQEMUSaveData *data, bool reset_nvram, + virQEMUSaveFd *saveFd, virDomainAsyncJob asyn= cJob) + ATTRIBUTE_NONNULL(1) ATTRIBUTE_NONNULL(2) ATTRIBUTE_NONNULL(3) + ATTRIBUTE_NONNULL(5) G_GNUC_WARN_UNUSED_RESULT; + int qemuSaveImageCreate(virQEMUDriver *driver, virDomainObj *vm, --=20 2.26.2 From nobody Fri May 17 11:58:59 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 170.10.133.124 as permitted sender) client-ip=170.10.133.124; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-delivery-124.mimecast.com; Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=fail(p=none dis=none) header.from=suse.de Return-Path: Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by mx.zohomail.com with SMTPS id 1654593620609182.8403188690237; Tue, 7 Jun 2022 02:20:20 -0700 (PDT) Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-638-9f9ardkIM-ik7i-_F8GOhg-1; Tue, 07 Jun 2022 05:20:17 -0400 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id B01641C006CF; Tue, 7 Jun 2022 09:19:55 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (unknown [10.30.29.100]) by smtp.corp.redhat.com (Postfix) with ESMTP id 98DF72026D2D; Tue, 7 Jun 2022 09:19:55 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (localhost [IPv6:::1]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id 2FE951947B8D; Tue, 7 Jun 2022 09:19:55 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx10.intmail.prod.int.rdu2.redhat.com [10.11.54.10]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id BA1F51947B9B for ; Tue, 7 Jun 2022 09:19:53 +0000 (UTC) Received: by smtp.corp.redhat.com (Postfix) id AB6DF416147; Tue, 7 Jun 2022 09:19:53 +0000 (UTC) Received: from mimecast-mx02.redhat.com (mimecast04.extmail.prod.ext.rdu2.redhat.com [10.11.55.20]) by smtp.corp.redhat.com (Postfix) with ESMTPS id A7810416144 for ; Tue, 7 Jun 2022 09:19:53 +0000 (UTC) Received: from us-smtp-1.mimecast.com (us-smtp-1.mimecast.com [207.211.31.81]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 9113E101AA45 for ; Tue, 7 Jun 2022 09:19:53 +0000 (UTC) Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.220.29]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-563-kFfKtHGuMDCihwK8SxVktA-1; Tue, 07 Jun 2022 05:19:49 -0400 Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id 33E991F98D; Tue, 7 Jun 2022 09:19:48 +0000 (UTC) Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id E1DE113638; Tue, 7 Jun 2022 09:19:47 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id YBCHNTMYn2IoLwAAMHmgww (envelope-from ); Tue, 07 Jun 2022 09:19:47 +0000 X-MC-Unique: 9f9ardkIM-ik7i-_F8GOhg-1 X-Original-To: libvir-list@listman.corp.redhat.com X-MC-Unique: kFfKtHGuMDCihwK8SxVktA-1 From: Claudio Fontana To: =?UTF-8?q?Daniel=20P=20=2E=20Berrang=C3=A9?= Subject: [libvirt RFCv11 24/33] tools: add parallel parameter to virsh save command Date: Tue, 7 Jun 2022 11:19:27 +0200 Message-Id: <20220607091936.7948-25-cfontana@suse.de> In-Reply-To: <20220607091936.7948-1-cfontana@suse.de> References: <20220607091936.7948-1-cfontana@suse.de> MIME-Version: 1.0 X-Mimecast-Impersonation-Protect: Policy=CLT - Impersonation Protection Definition; Similar Internal Domain=false; Similar Monitored External Domain=false; Custom External Domain=false; Mimecast External Domain=false; Newly Observed Domain=false; Internal User Name=false; Custom Display Name List=false; Reply-to Address Mismatch=false; Targeted Threat Dictionary=false; Mimecast Threat Dictionary=false; Custom Threat Dictionary=false X-Scanned-By: MIMEDefang 2.85 on 10.11.54.10 X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: libvir-list@redhat.com, Claudio Fontana , "Dr . David Alan Gilbert" , Anthony Iliopoulos Errors-To: libvir-list-bounces@redhat.com Sender: "libvir-list" X-Scanned-By: MIMEDefang 2.78 on 10.11.54.4 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=libvir-list-bounces@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable X-ZM-MESSAGEID: 1654593622485100014 Content-Type: text/plain; charset="utf-8"; x-default="true" Signed-off-by: Claudio Fontana --- docs/manpages/virsh.rst | 11 ++++++++++- tools/virsh-domain.c | 24 ++++++++++++++++++++++-- 2 files changed, 32 insertions(+), 3 deletions(-) diff --git a/docs/manpages/virsh.rst b/docs/manpages/virsh.rst index 965b89c99d..9f33b0aaef 100644 --- a/docs/manpages/virsh.rst +++ b/docs/manpages/virsh.rst @@ -3803,6 +3803,7 @@ save :: =20 save domain state-file [--bypass-cache] [--xml file] + [--parallel] [--parallel-connections connections] [{--running | --paused}] [--verbose] =20 Saves a running domain (RAM, but not disk state) to a state file so that @@ -3810,8 +3811,10 @@ it can be restored later. Once saved, the domain will no longer be running on the system, thus the memory allocated for the domain will be free for other domains to use. ``virsh restore`` restores from this state file. + If *--bypass-cache* is specified, the save will avoid the file system -cache, although this may slow down the operation. +cache; depending on the specific scenario this may slow down or speed up +the operation. =20 The progress may be monitored using ``domjobinfo`` virsh command and cance= led with ``domjobabort`` command (sent by another virsh instance). Another opt= ion @@ -3833,6 +3836,12 @@ based on the state the domain was in when the save w= as done; passing either the *--running* or *--paused* flag will allow overriding which state the ``restore`` should use. =20 +*--parallel* option will cause the save data to be sent over multiple +parallel connections to the file. The number of extra connections +can be set using *--parallel-connections*. + +Parallel connections help in speeding up large save operations. + Domain saved state files assume that disk images will be unchanged between the creation and restore point. For a more complete system restore point, where the disk state is saved alongside the memory diff --git a/tools/virsh-domain.c b/tools/virsh-domain.c index a3f007a1d2..7a313cb29e 100644 --- a/tools/virsh-domain.c +++ b/tools/virsh-domain.c @@ -4174,6 +4174,14 @@ static const vshCmdOptDef opts_save[] =3D { .type =3D VSH_OT_BOOL, .help =3D N_("avoid file system cache when saving") }, + {.name =3D "parallel", + .type =3D VSH_OT_BOOL, + .help =3D N_("enable parallel save") + }, + {.name =3D "parallel-connections", + .type =3D VSH_OT_INT, + .help =3D N_("number of extra connections to use for parallel save") + }, {.name =3D "xml", .type =3D VSH_OT_STRING, .completer =3D virshCompletePathLocalExisting, @@ -4206,6 +4214,8 @@ doSave(void *opaque) virTypedParameterPtr params =3D NULL; int nparams =3D 0; int maxparams =3D 0; + int intOpt =3D 0; + int rv =3D -1; unsigned int flags =3D 0; const char *xmlfile =3D NULL; g_autofree char *xml =3D NULL; @@ -4227,6 +4237,15 @@ doSave(void *opaque) } if (vshCommandOptBool(cmd, "bypass-cache")) flags |=3D VIR_DOMAIN_SAVE_BYPASS_CACHE; + if (vshCommandOptBool(cmd, "parallel")) + flags |=3D VIR_DOMAIN_SAVE_PARALLEL; + if ((rv =3D vshCommandOptInt(ctl, cmd, "parallel-connections", &intOpt= )) < 0) { + goto out; + } else if (rv > 0) { + if (virTypedParamsAddInt(¶ms, &nparams, &maxparams, + VIR_DOMAIN_SAVE_PARAM_PARALLEL_CONNECTION= S, intOpt) < 0) + goto out; + } if (vshCommandOptBool(cmd, "running")) flags |=3D VIR_DOMAIN_SAVE_RUNNING; if (vshCommandOptBool(cmd, "paused")) @@ -4247,8 +4266,9 @@ doSave(void *opaque) goto out; } } - - if (flags || xml) { + if (flags & VIR_DOMAIN_SAVE_PARALLEL) { + rc =3D virDomainSaveParams(dom, params, nparams, flags); + } else if (flags || xml) { rc =3D virDomainSaveFlags(dom, to, xml, flags); } else { rc =3D virDomainSave(dom, to); --=20 2.26.2 From nobody Fri May 17 11:58:59 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 170.10.129.124 as permitted sender) client-ip=170.10.129.124; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-delivery-124.mimecast.com; Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=fail(p=none dis=none) header.from=suse.de Return-Path: Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by mx.zohomail.com with SMTPS id 1654593624954499.6125558392828; Tue, 7 Jun 2022 02:20:24 -0700 (PDT) Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-345-gYOMoa-8MoeM_8AU5L5Oog-1; Tue, 07 Jun 2022 05:20:19 -0400 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.rdu2.redhat.com [10.11.54.7]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 17BB22A59541; Tue, 7 Jun 2022 09:19:59 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (unknown [10.30.29.100]) by smtp.corp.redhat.com (Postfix) with ESMTP id 005671415100; Tue, 7 Jun 2022 09:19:58 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (localhost [IPv6:::1]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id BBB0719452D8; Tue, 7 Jun 2022 09:19:58 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx10.intmail.prod.int.rdu2.redhat.com [10.11.54.10]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id CAE8C19451EF for ; Tue, 7 Jun 2022 09:19:54 +0000 (UTC) Received: by smtp.corp.redhat.com (Postfix) id AB50840336A; Tue, 7 Jun 2022 09:19:54 +0000 (UTC) Received: from mimecast-mx02.redhat.com (mimecast05.extmail.prod.ext.rdu2.redhat.com [10.11.55.21]) by smtp.corp.redhat.com (Postfix) with ESMTPS id A60B2492C3B for ; Tue, 7 Jun 2022 09:19:54 +0000 (UTC) Received: from us-smtp-1.mimecast.com (us-smtp-delivery-1.mimecast.com [207.211.31.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 8D6988339C5 for ; Tue, 7 Jun 2022 09:19:54 +0000 (UTC) Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.220.28]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-589-amx6YFB2OFiyxGqO2UklkQ-1; Tue, 07 Jun 2022 05:19:50 -0400 Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id 6A31121B38; Tue, 7 Jun 2022 09:19:48 +0000 (UTC) Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 30A9C13B40; Tue, 7 Jun 2022 09:19:48 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id oJMrCjQYn2IoLwAAMHmgww (envelope-from ); Tue, 07 Jun 2022 09:19:48 +0000 X-MC-Unique: gYOMoa-8MoeM_8AU5L5Oog-1 X-Original-To: libvir-list@listman.corp.redhat.com X-MC-Unique: amx6YFB2OFiyxGqO2UklkQ-1 From: Claudio Fontana To: =?UTF-8?q?Daniel=20P=20=2E=20Berrang=C3=A9?= Subject: [libvirt RFCv11 25/33] tools: add parallel parameter to virsh restore command Date: Tue, 7 Jun 2022 11:19:28 +0200 Message-Id: <20220607091936.7948-26-cfontana@suse.de> In-Reply-To: <20220607091936.7948-1-cfontana@suse.de> References: <20220607091936.7948-1-cfontana@suse.de> MIME-Version: 1.0 X-Mimecast-Impersonation-Protect: Policy=CLT - Impersonation Protection Definition; Similar Internal Domain=false; Similar Monitored External Domain=false; Custom External Domain=false; Mimecast External Domain=false; Newly Observed Domain=false; Internal User Name=false; Custom Display Name List=false; Reply-to Address Mismatch=false; Targeted Threat Dictionary=false; Mimecast Threat Dictionary=false; Custom Threat Dictionary=false X-Scanned-By: MIMEDefang 2.85 on 10.11.54.10 X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: libvir-list@redhat.com, Claudio Fontana , "Dr . David Alan Gilbert" , Anthony Iliopoulos Errors-To: libvir-list-bounces@redhat.com Sender: "libvir-list" X-Scanned-By: MIMEDefang 2.85 on 10.11.54.7 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=libvir-list-bounces@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable X-ZM-MESSAGEID: 1654593627121100001 Content-Type: text/plain; charset="utf-8"; x-default="true" Signed-off-by: Claudio Fontana --- docs/manpages/virsh.rst | 11 +++++++++-- tools/virsh-domain.c | 10 +++++++++- 2 files changed, 18 insertions(+), 3 deletions(-) diff --git a/docs/manpages/virsh.rst b/docs/manpages/virsh.rst index 9f33b0aaef..5abb03be9a 100644 --- a/docs/manpages/virsh.rst +++ b/docs/manpages/virsh.rst @@ -3754,12 +3754,13 @@ restore :: =20 restore state-file [--bypass-cache] [--xml file] - [{--running | --paused}] [--reset-nvram] + [{--running | --paused}] [--reset-nvram] [--parallel] =20 Restores a domain from a ``virsh save`` state file. See *save* for more in= fo. =20 If *--bypass-cache* is specified, the restore will avoid the file system -cache, although this may slow down the operation. +cache; depending on the specific scenario this may slow down or speed up +the operation. =20 *--xml* ``file`` is usually omitted, but can be used to supply an alternative XML file for use on the restored guest with changes only @@ -3775,6 +3776,12 @@ domain should be started in. If *--reset-nvram* is specified, any existing NVRAM file will be deleted and re-initialized from its pristine template. =20 +*--parallel* option will cause the save data to be loaded using parallel +connections. The state file stores the number of channels in the file, +so there is no need to specify the number of parallel connections. + +Parallel connections may help in speeding up large restore operations. + ``Note``: To avoid corrupting file system contents within the domain, you should not reuse the saved state file for a second ``restore`` unless you have also reverted all storage volumes back to the same contents as when diff --git a/tools/virsh-domain.c b/tools/virsh-domain.c index 7a313cb29e..98a1825c98 100644 --- a/tools/virsh-domain.c +++ b/tools/virsh-domain.c @@ -5312,6 +5312,10 @@ static const vshCmdOptDef opts_restore[] =3D { .type =3D VSH_OT_BOOL, .help =3D N_("avoid file system cache when restoring") }, + {.name =3D "parallel", + .type =3D VSH_OT_BOOL, + .help =3D N_("enable parallel restore") + }, {.name =3D "xml", .type =3D VSH_OT_STRING, .completer =3D virshCompletePathLocalExisting, @@ -5353,6 +5357,8 @@ cmdRestore(vshControl *ctl, const vshCmd *cmd) } if (vshCommandOptBool(cmd, "bypass-cache")) flags |=3D VIR_DOMAIN_SAVE_BYPASS_CACHE; + if (vshCommandOptBool(cmd, "parallel")) + flags |=3D VIR_DOMAIN_SAVE_PARALLEL; if (vshCommandOptBool(cmd, "running")) flags |=3D VIR_DOMAIN_SAVE_RUNNING; if (vshCommandOptBool(cmd, "paused")) @@ -5371,7 +5377,9 @@ cmdRestore(vshControl *ctl, const vshCmd *cmd) goto out; } =20 - if (flags || xml) { + if (flags & VIR_DOMAIN_SAVE_PARALLEL) { + rc =3D virDomainRestoreParams(priv->conn, params, nparams, flags); + } else if (flags || xml) { rc =3D virDomainRestoreFlags(priv->conn, from, xml, flags); } else { rc =3D virDomainRestore(priv->conn, from); --=20 2.26.2 From nobody Fri May 17 11:58:59 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 170.10.129.124 as permitted sender) client-ip=170.10.129.124; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-delivery-124.mimecast.com; Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=fail(p=none dis=none) header.from=suse.de Return-Path: Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by mx.zohomail.com with SMTPS id 1654593668308744.7231030740029; Tue, 7 Jun 2022 02:21:08 -0700 (PDT) Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-272-sl3q6OkzMZCPoKBuU9Uk3A-1; Tue, 07 Jun 2022 05:20:17 -0400 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.rdu2.redhat.com [10.11.54.3]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id A46EC824079; Tue, 7 Jun 2022 09:19:56 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (unknown [10.30.29.100]) by smtp.corp.redhat.com (Postfix) with ESMTP id 899ED1121315; Tue, 7 Jun 2022 09:19:56 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (localhost [IPv6:::1]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id 46C6419451EF; Tue, 7 Jun 2022 09:19:56 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx10.intmail.prod.int.rdu2.redhat.com [10.11.54.10]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id 6F62E1947B8C for ; Tue, 7 Jun 2022 09:19:53 +0000 (UTC) Received: by smtp.corp.redhat.com (Postfix) id 4F01F40336A; Tue, 7 Jun 2022 09:19:53 +0000 (UTC) Received: from mimecast-mx02.redhat.com (mimecast01.extmail.prod.ext.rdu2.redhat.com [10.11.55.17]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 49BAE492C3B for ; Tue, 7 Jun 2022 09:19:53 +0000 (UTC) Received: from us-smtp-1.mimecast.com (us-smtp-2.mimecast.com [207.211.31.81]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 2ED83858284 for ; Tue, 7 Jun 2022 09:19:53 +0000 (UTC) Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.220.29]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-387-86yrqEPKO4C8mVxKoWiVZg-1; Tue, 07 Jun 2022 05:19:50 -0400 Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id AE0861F999; Tue, 7 Jun 2022 09:19:48 +0000 (UTC) Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 73C5A13638; Tue, 7 Jun 2022 09:19:48 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id sLuwGjQYn2IoLwAAMHmgww (envelope-from ); Tue, 07 Jun 2022 09:19:48 +0000 X-MC-Unique: sl3q6OkzMZCPoKBuU9Uk3A-1 X-Original-To: libvir-list@listman.corp.redhat.com X-MC-Unique: 86yrqEPKO4C8mVxKoWiVZg-1 From: Claudio Fontana To: =?UTF-8?q?Daniel=20P=20=2E=20Berrang=C3=A9?= Subject: [libvirt RFCv11 26/33] qemu: add migration parameter multifd-compression Date: Tue, 7 Jun 2022 11:19:29 +0200 Message-Id: <20220607091936.7948-27-cfontana@suse.de> In-Reply-To: <20220607091936.7948-1-cfontana@suse.de> References: <20220607091936.7948-1-cfontana@suse.de> MIME-Version: 1.0 X-Mimecast-Impersonation-Protect: Policy=CLT - Impersonation Protection Definition; Similar Internal Domain=false; Similar Monitored External Domain=false; Custom External Domain=false; Mimecast External Domain=false; Newly Observed Domain=false; Internal User Name=false; Custom Display Name List=false; Reply-to Address Mismatch=false; Targeted Threat Dictionary=false; Mimecast Threat Dictionary=false; Custom Threat Dictionary=false X-Scanned-By: MIMEDefang 2.85 on 10.11.54.10 X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: libvir-list@redhat.com, Claudio Fontana , "Dr . David Alan Gilbert" , Anthony Iliopoulos Errors-To: libvir-list-bounces@redhat.com Sender: "libvir-list" X-Scanned-By: MIMEDefang 2.78 on 10.11.54.3 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=libvir-list-bounces@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable X-ZM-MESSAGEID: 1654593669823100004 Content-Type: text/plain; charset="utf-8"; x-default="true" add it to both capabilities and migration parameters Signed-off-by: Claudio Fontana --- src/qemu/qemu_capabilities.c | 2 ++ src/qemu/qemu_capabilities.h | 1 + src/qemu/qemu_migration_params.c | 2 ++ src/qemu/qemu_migration_params.h | 1 + tests/qemucapabilitiesdata/caps_5.0.0.aarch64.xml | 1 + tests/qemucapabilitiesdata/caps_5.0.0.ppc64.xml | 1 + tests/qemucapabilitiesdata/caps_5.0.0.riscv64.xml | 1 + tests/qemucapabilitiesdata/caps_5.0.0.x86_64.xml | 1 + tests/qemucapabilitiesdata/caps_5.1.0.sparc.xml | 1 + tests/qemucapabilitiesdata/caps_5.1.0.x86_64.xml | 1 + tests/qemucapabilitiesdata/caps_5.2.0.aarch64.xml | 1 + tests/qemucapabilitiesdata/caps_5.2.0.ppc64.xml | 1 + tests/qemucapabilitiesdata/caps_5.2.0.riscv64.xml | 1 + tests/qemucapabilitiesdata/caps_5.2.0.s390x.xml | 1 + tests/qemucapabilitiesdata/caps_5.2.0.x86_64.xml | 1 + tests/qemucapabilitiesdata/caps_6.0.0.aarch64.xml | 1 + tests/qemucapabilitiesdata/caps_6.0.0.s390x.xml | 1 + tests/qemucapabilitiesdata/caps_6.0.0.x86_64.xml | 1 + tests/qemucapabilitiesdata/caps_6.1.0.x86_64.xml | 1 + tests/qemucapabilitiesdata/caps_6.2.0.aarch64.xml | 1 + tests/qemucapabilitiesdata/caps_6.2.0.ppc64.xml | 1 + tests/qemucapabilitiesdata/caps_6.2.0.x86_64.xml | 1 + tests/qemucapabilitiesdata/caps_7.0.0.aarch64.xml | 1 + tests/qemucapabilitiesdata/caps_7.0.0.ppc64.xml | 1 + tests/qemucapabilitiesdata/caps_7.0.0.x86_64.xml | 1 + tests/qemucapabilitiesdata/caps_7.1.0.x86_64.xml | 1 + 26 files changed, 28 insertions(+) diff --git a/src/qemu/qemu_capabilities.c b/src/qemu/qemu_capabilities.c index 44e3f9b81f..065d45e1f9 100644 --- a/src/qemu/qemu_capabilities.c +++ b/src/qemu/qemu_capabilities.c @@ -677,6 +677,7 @@ VIR_ENUM_IMPL(virQEMUCaps, "chardev.qemu-vdagent", /* QEMU_CAPS_CHARDEV_QEMU_VDAGENT */ "display-dbus", /* QEMU_CAPS_DISPLAY_DBUS */ "migrate-multifd", /* QEMU_CAPS_MIGRATE_MULTIFD */ + "migration-param.multifd-compression", /* QEMU_CAPS_MIGRATIO= N_PARAM_MULTIFD_COMPRESSION */ ); =20 =20 @@ -1613,6 +1614,7 @@ static struct virQEMUCapsStringFlags virQEMUCapsQMPSc= hemaQueries[] =3D { { "migrate-set-parameters/arg-type/downtime-limit", QEMU_CAPS_MIGRATIO= N_PARAM_DOWNTIME }, { "migrate-set-parameters/arg-type/xbzrle-cache-size", QEMU_CAPS_MIGRA= TION_PARAM_XBZRLE_CACHE_SIZE }, { "migrate-set-parameters/arg-type/block-bitmap-mapping/bitmaps/transf= orm", QEMU_CAPS_MIGRATION_PARAM_BLOCK_BITMAP_MAPPING }, + { "migrate-set-parameters/arg-type/multifd-compression", QEMU_CAPS_MIG= RATION_PARAM_MULTIFD_COMPRESSION }, { "nbd-server-start/arg-type/tls-creds", QEMU_CAPS_NBD_TLS }, { "nbd-server-add/arg-type/bitmap", QEMU_CAPS_NBD_BITMAP }, { "netdev_add/arg-type/+vhost-vdpa", QEMU_CAPS_NETDEV_VHOST_VDPA }, diff --git a/src/qemu/qemu_capabilities.h b/src/qemu/qemu_capabilities.h index 4c7e031e28..ceac26a320 100644 --- a/src/qemu/qemu_capabilities.h +++ b/src/qemu/qemu_capabilities.h @@ -652,6 +652,7 @@ typedef enum { /* virQEMUCapsFlags grouping marker for = syntax-check */ QEMU_CAPS_CHARDEV_QEMU_VDAGENT, /* -chardev qemu-vdagent */ QEMU_CAPS_DISPLAY_DBUS, /* -display dbus */ QEMU_CAPS_MIGRATE_MULTIFD, /* migrate can set multifd parameter */ + QEMU_CAPS_MIGRATION_PARAM_MULTIFD_COMPRESSION, /* multifd-compression = in migrate-set-parameters */ =20 QEMU_CAPS_LAST /* this must always be the last item */ } virQEMUCapsFlags; diff --git a/src/qemu/qemu_migration_params.c b/src/qemu/qemu_migration_par= ams.c index fad61631fa..43975640d9 100644 --- a/src/qemu/qemu_migration_params.c +++ b/src/qemu/qemu_migration_params.c @@ -115,6 +115,7 @@ VIR_ENUM_IMPL(qemuMigrationParam, "xbzrle-cache-size", "max-postcopy-bandwidth", "multifd-channels", + "multifd-compression", ); =20 typedef struct _qemuMigrationParamsAlwaysOnItem qemuMigrationParamsAlwaysO= nItem; @@ -234,6 +235,7 @@ static const qemuMigrationParamType qemuMigrationParamT= ypes[] =3D { [QEMU_MIGRATION_PARAM_XBZRLE_CACHE_SIZE] =3D QEMU_MIGRATION_PARAM_TYPE= _ULL, [QEMU_MIGRATION_PARAM_MAX_POSTCOPY_BANDWIDTH] =3D QEMU_MIGRATION_PARAM= _TYPE_ULL, [QEMU_MIGRATION_PARAM_MULTIFD_CHANNELS] =3D QEMU_MIGRATION_PARAM_TYPE_= INT, + [QEMU_MIGRATION_PARAM_MULTIFD_COMPRESSION] =3D QEMU_MIGRATION_PARAM_TY= PE_STRING, }; G_STATIC_ASSERT(G_N_ELEMENTS(qemuMigrationParamTypes) =3D=3D QEMU_MIGRATIO= N_PARAM_LAST); =20 diff --git a/src/qemu/qemu_migration_params.h b/src/qemu/qemu_migration_par= ams.h index 036c4e7582..900ad7cfc8 100644 --- a/src/qemu/qemu_migration_params.h +++ b/src/qemu/qemu_migration_params.h @@ -60,6 +60,7 @@ typedef enum { QEMU_MIGRATION_PARAM_XBZRLE_CACHE_SIZE, QEMU_MIGRATION_PARAM_MAX_POSTCOPY_BANDWIDTH, QEMU_MIGRATION_PARAM_MULTIFD_CHANNELS, + QEMU_MIGRATION_PARAM_MULTIFD_COMPRESSION, =20 QEMU_MIGRATION_PARAM_LAST } qemuMigrationParam; diff --git a/tests/qemucapabilitiesdata/caps_5.0.0.aarch64.xml b/tests/qemu= capabilitiesdata/caps_5.0.0.aarch64.xml index 2a7f5074bc..de995c4434 100644 --- a/tests/qemucapabilitiesdata/caps_5.0.0.aarch64.xml +++ b/tests/qemucapabilitiesdata/caps_5.0.0.aarch64.xml @@ -175,6 +175,7 @@ + 5000000 0 61700241 diff --git a/tests/qemucapabilitiesdata/caps_5.0.0.ppc64.xml b/tests/qemuca= pabilitiesdata/caps_5.0.0.ppc64.xml index d2c083e8f7..a366282d55 100644 --- a/tests/qemucapabilitiesdata/caps_5.0.0.ppc64.xml +++ b/tests/qemucapabilitiesdata/caps_5.0.0.ppc64.xml @@ -181,6 +181,7 @@ + 5000000 0 42900241 diff --git a/tests/qemucapabilitiesdata/caps_5.0.0.riscv64.xml b/tests/qemu= capabilitiesdata/caps_5.0.0.riscv64.xml index 933b4870cd..c350c1b9a9 100644 --- a/tests/qemucapabilitiesdata/caps_5.0.0.riscv64.xml +++ b/tests/qemucapabilitiesdata/caps_5.0.0.riscv64.xml @@ -167,6 +167,7 @@ + 5000000 0 0 diff --git a/tests/qemucapabilitiesdata/caps_5.0.0.x86_64.xml b/tests/qemuc= apabilitiesdata/caps_5.0.0.x86_64.xml index 9bb48e1982..938775e7bb 100644 --- a/tests/qemucapabilitiesdata/caps_5.0.0.x86_64.xml +++ b/tests/qemucapabilitiesdata/caps_5.0.0.x86_64.xml @@ -215,6 +215,7 @@ + 5000000 0 43100241 diff --git a/tests/qemucapabilitiesdata/caps_5.1.0.sparc.xml b/tests/qemuca= pabilitiesdata/caps_5.1.0.sparc.xml index f01cd25278..cef1e418ba 100644 --- a/tests/qemucapabilitiesdata/caps_5.1.0.sparc.xml +++ b/tests/qemucapabilitiesdata/caps_5.1.0.sparc.xml @@ -87,6 +87,7 @@ + 5001000 0 0 diff --git a/tests/qemucapabilitiesdata/caps_5.1.0.x86_64.xml b/tests/qemuc= apabilitiesdata/caps_5.1.0.x86_64.xml index f4446674a4..c78c078b24 100644 --- a/tests/qemucapabilitiesdata/caps_5.1.0.x86_64.xml +++ b/tests/qemucapabilitiesdata/caps_5.1.0.x86_64.xml @@ -219,6 +219,7 @@ + 5001000 0 43100242 diff --git a/tests/qemucapabilitiesdata/caps_5.2.0.aarch64.xml b/tests/qemu= capabilitiesdata/caps_5.2.0.aarch64.xml index f3e3640657..9426ef386c 100644 --- a/tests/qemucapabilitiesdata/caps_5.2.0.aarch64.xml +++ b/tests/qemucapabilitiesdata/caps_5.2.0.aarch64.xml @@ -182,6 +182,7 @@ + 5002000 0 61700243 diff --git a/tests/qemucapabilitiesdata/caps_5.2.0.ppc64.xml b/tests/qemuca= pabilitiesdata/caps_5.2.0.ppc64.xml index 86d021c2fc..c39e51d595 100644 --- a/tests/qemucapabilitiesdata/caps_5.2.0.ppc64.xml +++ b/tests/qemucapabilitiesdata/caps_5.2.0.ppc64.xml @@ -186,6 +186,7 @@ + 5002000 0 42900243 diff --git a/tests/qemucapabilitiesdata/caps_5.2.0.riscv64.xml b/tests/qemu= capabilitiesdata/caps_5.2.0.riscv64.xml index 2079257759..59742d607f 100644 --- a/tests/qemucapabilitiesdata/caps_5.2.0.riscv64.xml +++ b/tests/qemucapabilitiesdata/caps_5.2.0.riscv64.xml @@ -172,6 +172,7 @@ + 5002000 0 0 diff --git a/tests/qemucapabilitiesdata/caps_5.2.0.s390x.xml b/tests/qemuca= pabilitiesdata/caps_5.2.0.s390x.xml index d800e35096..bc7e701f1a 100644 --- a/tests/qemucapabilitiesdata/caps_5.2.0.s390x.xml +++ b/tests/qemucapabilitiesdata/caps_5.2.0.s390x.xml @@ -140,6 +140,7 @@ + 5002000 0 39100243 diff --git a/tests/qemucapabilitiesdata/caps_5.2.0.x86_64.xml b/tests/qemuc= apabilitiesdata/caps_5.2.0.x86_64.xml index f189e299c3..b1aff5f7b9 100644 --- a/tests/qemucapabilitiesdata/caps_5.2.0.x86_64.xml +++ b/tests/qemucapabilitiesdata/caps_5.2.0.x86_64.xml @@ -223,6 +223,7 @@ + 5002000 0 43100243 diff --git a/tests/qemucapabilitiesdata/caps_6.0.0.aarch64.xml b/tests/qemu= capabilitiesdata/caps_6.0.0.aarch64.xml index e86a2cf5d6..c3dae8c8db 100644 --- a/tests/qemucapabilitiesdata/caps_6.0.0.aarch64.xml +++ b/tests/qemucapabilitiesdata/caps_6.0.0.aarch64.xml @@ -190,6 +190,7 @@ + 6000000 0 61700242 diff --git a/tests/qemucapabilitiesdata/caps_6.0.0.s390x.xml b/tests/qemuca= pabilitiesdata/caps_6.0.0.s390x.xml index bbc5825def..654eb92d4f 100644 --- a/tests/qemucapabilitiesdata/caps_6.0.0.s390x.xml +++ b/tests/qemucapabilitiesdata/caps_6.0.0.s390x.xml @@ -148,6 +148,7 @@ + 6000000 0 39100242 diff --git a/tests/qemucapabilitiesdata/caps_6.0.0.x86_64.xml b/tests/qemuc= apabilitiesdata/caps_6.0.0.x86_64.xml index 496dd5564c..2350d61f93 100644 --- a/tests/qemucapabilitiesdata/caps_6.0.0.x86_64.xml +++ b/tests/qemucapabilitiesdata/caps_6.0.0.x86_64.xml @@ -232,6 +232,7 @@ + 6000000 0 43100242 diff --git a/tests/qemucapabilitiesdata/caps_6.1.0.x86_64.xml b/tests/qemuc= apabilitiesdata/caps_6.1.0.x86_64.xml index fab5e40e35..2e682ae4ba 100644 --- a/tests/qemucapabilitiesdata/caps_6.1.0.x86_64.xml +++ b/tests/qemucapabilitiesdata/caps_6.1.0.x86_64.xml @@ -237,6 +237,7 @@ + 6001000 0 43100243 diff --git a/tests/qemucapabilitiesdata/caps_6.2.0.aarch64.xml b/tests/qemu= capabilitiesdata/caps_6.2.0.aarch64.xml index a83b8c8d77..ee90c76c30 100644 --- a/tests/qemucapabilitiesdata/caps_6.2.0.aarch64.xml +++ b/tests/qemucapabilitiesdata/caps_6.2.0.aarch64.xml @@ -202,6 +202,7 @@ + 6001050 0 61700244 diff --git a/tests/qemucapabilitiesdata/caps_6.2.0.ppc64.xml b/tests/qemuca= pabilitiesdata/caps_6.2.0.ppc64.xml index bfacae9b3b..07b06a36f6 100644 --- a/tests/qemucapabilitiesdata/caps_6.2.0.ppc64.xml +++ b/tests/qemucapabilitiesdata/caps_6.2.0.ppc64.xml @@ -197,6 +197,7 @@ + 6002000 0 42900244 diff --git a/tests/qemucapabilitiesdata/caps_6.2.0.x86_64.xml b/tests/qemuc= apabilitiesdata/caps_6.2.0.x86_64.xml index 1293cd1bf1..8597367f20 100644 --- a/tests/qemucapabilitiesdata/caps_6.2.0.x86_64.xml +++ b/tests/qemucapabilitiesdata/caps_6.2.0.x86_64.xml @@ -239,6 +239,7 @@ + 6002000 0 43100244 diff --git a/tests/qemucapabilitiesdata/caps_7.0.0.aarch64.xml b/tests/qemu= capabilitiesdata/caps_7.0.0.aarch64.xml index 16626a1342..b15510d7d1 100644 --- a/tests/qemucapabilitiesdata/caps_7.0.0.aarch64.xml +++ b/tests/qemucapabilitiesdata/caps_7.0.0.aarch64.xml @@ -210,6 +210,7 @@ + 6002092 0 61700243 diff --git a/tests/qemucapabilitiesdata/caps_7.0.0.ppc64.xml b/tests/qemuca= pabilitiesdata/caps_7.0.0.ppc64.xml index 9f4242c5c0..4017023486 100644 --- a/tests/qemucapabilitiesdata/caps_7.0.0.ppc64.xml +++ b/tests/qemucapabilitiesdata/caps_7.0.0.ppc64.xml @@ -214,6 +214,7 @@ + 7000000 0 42900243 diff --git a/tests/qemucapabilitiesdata/caps_7.0.0.x86_64.xml b/tests/qemuc= apabilitiesdata/caps_7.0.0.x86_64.xml index 1d003a6fb7..74f2e28a06 100644 --- a/tests/qemucapabilitiesdata/caps_7.0.0.x86_64.xml +++ b/tests/qemucapabilitiesdata/caps_7.0.0.x86_64.xml @@ -244,6 +244,7 @@ + 7000000 0 43100243 diff --git a/tests/qemucapabilitiesdata/caps_7.1.0.x86_64.xml b/tests/qemuc= apabilitiesdata/caps_7.1.0.x86_64.xml index 80762d9fe9..efbfc94dfe 100644 --- a/tests/qemucapabilitiesdata/caps_7.1.0.x86_64.xml +++ b/tests/qemucapabilitiesdata/caps_7.1.0.x86_64.xml @@ -244,6 +244,7 @@ + 7000050 0 43100244 --=20 2.26.2 From nobody Fri May 17 11:58:59 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 170.10.129.124 as permitted sender) client-ip=170.10.129.124; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-delivery-124.mimecast.com; Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=fail(p=none dis=none) header.from=suse.de Return-Path: Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by mx.zohomail.com with SMTPS id 1654593629913684.1890438060872; Tue, 7 Jun 2022 02:20:29 -0700 (PDT) Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-453-klYe0b_FMDq0lkL0KtXbYw-1; Tue, 07 Jun 2022 05:20:24 -0400 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.rdu2.redhat.com [10.11.54.6]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 734E68027EE; Tue, 7 Jun 2022 09:20:14 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (unknown [10.30.29.100]) by smtp.corp.redhat.com (Postfix) with ESMTP id 5CB0C2166B26; Tue, 7 Jun 2022 09:20:14 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (localhost [IPv6:::1]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id 0EBFC1937429; Tue, 7 Jun 2022 09:20:14 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.rdu2.redhat.com [10.11.54.2]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id 9B9E919451F1 for ; Tue, 7 Jun 2022 09:20:12 +0000 (UTC) Received: by smtp.corp.redhat.com (Postfix) id 9105840EC003; Tue, 7 Jun 2022 09:20:12 +0000 (UTC) Received: from mimecast-mx02.redhat.com (mimecast10.extmail.prod.ext.rdu2.redhat.com [10.11.55.26]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 8CFBB40EC002 for ; Tue, 7 Jun 2022 09:20:12 +0000 (UTC) Received: from us-smtp-1.mimecast.com (us-smtp-delivery-1.mimecast.com [205.139.110.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 75C351C13953 for ; Tue, 7 Jun 2022 09:20:12 +0000 (UTC) Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.220.28]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-618-jjBkyF6EPaKgB45KYH0Xkw-1; Tue, 07 Jun 2022 05:20:04 -0400 Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id F035021B46; Tue, 7 Jun 2022 09:19:48 +0000 (UTC) Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id B6B3F13638; Tue, 7 Jun 2022 09:19:48 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id oPwPKzQYn2IoLwAAMHmgww (envelope-from ); Tue, 07 Jun 2022 09:19:48 +0000 X-MC-Unique: klYe0b_FMDq0lkL0KtXbYw-1 X-Original-To: libvir-list@listman.corp.redhat.com X-MC-Unique: jjBkyF6EPaKgB45KYH0Xkw-1 From: Claudio Fontana To: =?UTF-8?q?Daniel=20P=20=2E=20Berrang=C3=A9?= Subject: [libvirt RFCv11 27/33] libvirt: add new VIR_DOMAIN_SAVE_PARAM_PARALLEL_COMPRESSION Date: Tue, 7 Jun 2022 11:19:30 +0200 Message-Id: <20220607091936.7948-28-cfontana@suse.de> In-Reply-To: <20220607091936.7948-1-cfontana@suse.de> References: <20220607091936.7948-1-cfontana@suse.de> MIME-Version: 1.0 X-Mimecast-Impersonation-Protect: Policy=CLT - Impersonation Protection Definition; Similar Internal Domain=false; Similar Monitored External Domain=false; Custom External Domain=false; Mimecast External Domain=false; Newly Observed Domain=false; Internal User Name=false; Custom Display Name List=false; Reply-to Address Mismatch=false; Targeted Threat Dictionary=false; Mimecast Threat Dictionary=false; Custom Threat Dictionary=false X-Scanned-By: MIMEDefang 2.84 on 10.11.54.2 X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: libvir-list@redhat.com, Claudio Fontana , "Dr . David Alan Gilbert" , Anthony Iliopoulos Errors-To: libvir-list-bounces@redhat.com Sender: "libvir-list" X-Scanned-By: MIMEDefang 2.78 on 10.11.54.6 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=libvir-list-bounces@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable X-ZM-MESSAGEID: 1654593631099100009 Content-Type: text/plain; charset="utf-8"; x-default="true" Signed-off-by: Claudio Fontana --- include/libvirt/libvirt-domain.h | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/include/libvirt/libvirt-domain.h b/include/libvirt/libvirt-dom= ain.h index 0b73ee27b6..0858d02e48 100644 --- a/include/libvirt/libvirt-domain.h +++ b/include/libvirt/libvirt-domain.h @@ -1613,6 +1613,17 @@ int virDomainRestoreParams (vir= ConnectPtr conn, */ # define VIR_DOMAIN_SAVE_PARAM_PARALLEL_CONNECTIONS "parallel.connecti= ons" =20 +/** + * VIR_DOMAIN_SAVE_PARAM_PARALLEL_COMPRESSION: + * + * this optional parameter is used in conjunction with the flag + * VIR_DOMAIN_SAVE_PARALLEL during save to ask the hypervisor for + * compressed channels to be used using this algorithm. + * + * Since: 8.4.0 + */ +# define VIR_DOMAIN_SAVE_PARAM_PARALLEL_COMPRESSION "parallel.compress= ion" + /* See below for virDomainSaveImageXMLFlags */ char * virDomainSaveImageGetXMLDesc (virConnectPtr conn, const char *file, --=20 2.26.2 From nobody Fri May 17 11:58:59 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 170.10.133.124 as permitted sender) client-ip=170.10.133.124; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-delivery-124.mimecast.com; Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=fail(p=none dis=none) header.from=suse.de Return-Path: Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by mx.zohomail.com with SMTPS id 1654593614584485.1282540319994; Tue, 7 Jun 2022 02:20:14 -0700 (PDT) Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-142-aM4mIr_wPdirX_IOWKg5GQ-1; Tue, 07 Jun 2022 05:20:10 -0400 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.rdu2.redhat.com [10.11.54.2]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id E5BEC857FAD; Tue, 7 Jun 2022 09:19:53 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (unknown [10.30.29.100]) by smtp.corp.redhat.com (Postfix) with ESMTP id D02E240EC002; Tue, 7 Jun 2022 09:19:53 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (localhost [IPv6:::1]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id 56FBE1947B82; Tue, 7 Jun 2022 09:19:53 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx09.intmail.prod.int.rdu2.redhat.com [10.11.54.9]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id F0D4D19452D2 for ; Tue, 7 Jun 2022 09:19:52 +0000 (UTC) Received: by smtp.corp.redhat.com (Postfix) id D2C98492CA3; Tue, 7 Jun 2022 09:19:52 +0000 (UTC) Received: from mimecast-mx02.redhat.com (mimecast09.extmail.prod.ext.rdu2.redhat.com [10.11.55.25]) by smtp.corp.redhat.com (Postfix) with ESMTPS id CF14F492C3B for ; Tue, 7 Jun 2022 09:19:52 +0000 (UTC) Received: from us-smtp-1.mimecast.com (us-smtp-delivery-1.mimecast.com [207.211.31.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id B5F8B2949BA6 for ; Tue, 7 Jun 2022 09:19:52 +0000 (UTC) Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.220.29]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-97-RnQdRpI3PsKTPgadcAxvcA-1; Tue, 07 Jun 2022 05:19:50 -0400 Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id 3DD391F9A6; Tue, 7 Jun 2022 09:19:49 +0000 (UTC) Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 0482013638; Tue, 7 Jun 2022 09:19:48 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id UEUqOzQYn2IoLwAAMHmgww (envelope-from ); Tue, 07 Jun 2022 09:19:48 +0000 X-MC-Unique: aM4mIr_wPdirX_IOWKg5GQ-1 X-Original-To: libvir-list@listman.corp.redhat.com X-MC-Unique: RnQdRpI3PsKTPgadcAxvcA-1 From: Claudio Fontana To: =?UTF-8?q?Daniel=20P=20=2E=20Berrang=C3=A9?= Subject: [libvirt RFCv11 28/33] qemu: saveimage: add parallel compression argument to ImageCreate Date: Tue, 7 Jun 2022 11:19:31 +0200 Message-Id: <20220607091936.7948-29-cfontana@suse.de> In-Reply-To: <20220607091936.7948-1-cfontana@suse.de> References: <20220607091936.7948-1-cfontana@suse.de> MIME-Version: 1.0 X-Mimecast-Impersonation-Protect: Policy=CLT - Impersonation Protection Definition; Similar Internal Domain=false; Similar Monitored External Domain=false; Custom External Domain=false; Mimecast External Domain=false; Newly Observed Domain=false; Internal User Name=false; Custom Display Name List=false; Reply-to Address Mismatch=false; Targeted Threat Dictionary=false; Mimecast Threat Dictionary=false; Custom Threat Dictionary=false X-Scanned-By: MIMEDefang 2.85 on 10.11.54.9 X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: libvir-list@redhat.com, Claudio Fontana , "Dr . David Alan Gilbert" , Anthony Iliopoulos Errors-To: libvir-list-bounces@redhat.com Sender: "libvir-list" X-Scanned-By: MIMEDefang 2.84 on 10.11.54.2 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=libvir-list-bounces@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable X-ZM-MESSAGEID: 1654593615889100018 Content-Type: text/plain; charset="utf-8"; x-default="true" Signed-off-by: Claudio Fontana --- src/qemu/qemu_driver.c | 11 ++++++----- src/qemu/qemu_saveimage.c | 1 + src/qemu/qemu_saveimage.h | 1 + src/qemu/qemu_snapshot.c | 2 +- 4 files changed, 9 insertions(+), 6 deletions(-) diff --git a/src/qemu/qemu_driver.c b/src/qemu/qemu_driver.c index c3b2afe0d8..8839e0031d 100644 --- a/src/qemu/qemu_driver.c +++ b/src/qemu/qemu_driver.c @@ -2650,7 +2650,8 @@ static int qemuDomainSaveInternal(virQEMUDriver *driver, virDomainObj *vm, const char *path, int compressed, virCommand *compressor, - const char *xmlin, int nconn, unsigned int flags) + const char *xmlin, int nconn, const char *pcomp, + unsigned int flags) { g_autofree char *xml =3D NULL; bool was_running =3D false; @@ -2731,7 +2732,7 @@ qemuDomainSaveInternal(virQEMUDriver *driver, xml =3D NULL; =20 ret =3D qemuSaveImageCreate(driver, vm, path, data, compressor, - nconn, flags, VIR_ASYNC_JOB_SAVE); + nconn, pcomp, flags, VIR_ASYNC_JOB_SAVE); if (ret < 0) goto endjob; =20 @@ -2809,7 +2810,7 @@ qemuDomainManagedSaveHelper(virQEMUDriver *driver, VIR_INFO("Saving state of domain '%s' to '%s'", vm->def->name, path); =20 if (qemuDomainSaveInternal(driver, vm, path, compressed, - compressor, dxml, -1, flags) < 0) + compressor, dxml, -1, NULL, flags) < 0) return -1; =20 vm->hasManagedSave =3D true; @@ -2848,7 +2849,7 @@ qemuDomainSaveFlags(virDomainPtr dom, const char *pat= h, const char *dxml, goto cleanup; =20 ret =3D qemuDomainSaveInternal(driver, vm, path, compressed, - compressor, dxml, -1, flags); + compressor, dxml, -1, NULL, flags); =20 cleanup: virDomainObjEndAPI(&vm); @@ -2922,7 +2923,7 @@ qemuDomainSaveParams(virDomainPtr dom, goto cleanup; =20 ret =3D qemuDomainSaveInternal(driver, vm, to, compressed, - compressor, dxml, nconn, flags); + compressor, dxml, nconn, NULL, flags); =20 cleanup: virDomainObjEndAPI(&vm); diff --git a/src/qemu/qemu_saveimage.c b/src/qemu/qemu_saveimage.c index 2ed02d31de..505d9b615f 100644 --- a/src/qemu/qemu_saveimage.c +++ b/src/qemu/qemu_saveimage.c @@ -561,6 +561,7 @@ qemuSaveImageCreate(virQEMUDriver *driver, virQEMUSaveData *data, virCommand *compressor, int nconn, + const char *pcomp G_GNUC_UNUSED, unsigned int flags, virDomainAsyncJob asyncJob) { diff --git a/src/qemu/qemu_saveimage.h b/src/qemu/qemu_saveimage.h index 158bcaff50..8542cd9081 100644 --- a/src/qemu/qemu_saveimage.h +++ b/src/qemu/qemu_saveimage.h @@ -130,6 +130,7 @@ qemuSaveImageCreate(virQEMUDriver *driver, virQEMUSaveData *data, virCommand *compressor, int nconn, + const char *pcomp, unsigned int flags, virDomainAsyncJob asyncJob); =20 diff --git a/src/qemu/qemu_snapshot.c b/src/qemu/qemu_snapshot.c index dec653c6d3..e1c18bbaff 100644 --- a/src/qemu/qemu_snapshot.c +++ b/src/qemu/qemu_snapshot.c @@ -1457,7 +1457,7 @@ qemuSnapshotCreateActiveExternal(virQEMUDriver *drive= r, memory_existing =3D virFileExists(snapdef->memorysnapshotfile); =20 if ((ret =3D qemuSaveImageCreate(driver, vm, snapdef->memorysnapsh= otfile, - data, compressor, -1, 0, + data, compressor, -1, NULL, 0, VIR_ASYNC_JOB_SNAPSHOT)) < 0) goto cleanup; =20 --=20 2.26.2 From nobody Fri May 17 11:58:59 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 170.10.129.124 as permitted sender) client-ip=170.10.129.124; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-delivery-124.mimecast.com; Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=fail(p=none dis=none) header.from=suse.de Return-Path: Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by mx.zohomail.com with SMTPS id 1654593627488571.558480420746; Tue, 7 Jun 2022 02:20:27 -0700 (PDT) Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-671-jlNlry4TOXahUkEJjVCgYA-1; Tue, 07 Jun 2022 05:20:18 -0400 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.rdu2.redhat.com [10.11.54.3]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 65AD7293248D; Tue, 7 Jun 2022 09:19:56 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (unknown [10.30.29.100]) by smtp.corp.redhat.com (Postfix) with ESMTP id 4DDF91121314; Tue, 7 Jun 2022 09:19:56 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (localhost [IPv6:::1]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id B9B3A1947B8D; Tue, 7 Jun 2022 09:19:55 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx10.intmail.prod.int.rdu2.redhat.com [10.11.54.10]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id 1B5E219452D2 for ; Tue, 7 Jun 2022 09:19:53 +0000 (UTC) Received: by smtp.corp.redhat.com (Postfix) id F397640336A; Tue, 7 Jun 2022 09:19:52 +0000 (UTC) Received: from mimecast-mx02.redhat.com (mimecast03.extmail.prod.ext.rdu2.redhat.com [10.11.55.19]) by smtp.corp.redhat.com (Postfix) with ESMTPS id EFAEF492C3B for ; Tue, 7 Jun 2022 09:19:52 +0000 (UTC) Received: from us-smtp-1.mimecast.com (us-smtp-delivery-1.mimecast.com [207.211.31.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id D9B9680C8FE for ; Tue, 7 Jun 2022 09:19:52 +0000 (UTC) Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.220.28]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-609-tNOEZ3GhPqeE0lKlc_t6Kg-1; Tue, 07 Jun 2022 05:19:51 -0400 Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id 8027021B57; Tue, 7 Jun 2022 09:19:49 +0000 (UTC) Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 46AEE13638; Tue, 7 Jun 2022 09:19:49 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id eFa4DzUYn2IoLwAAMHmgww (envelope-from ); Tue, 07 Jun 2022 09:19:49 +0000 X-MC-Unique: jlNlry4TOXahUkEJjVCgYA-1 X-Original-To: libvir-list@listman.corp.redhat.com X-MC-Unique: tNOEZ3GhPqeE0lKlc_t6Kg-1 From: Claudio Fontana To: =?UTF-8?q?Daniel=20P=20=2E=20Berrang=C3=A9?= Subject: [libvirt RFCv11 29/33] qemu: saveimage: add stub support for multifd compression parameter Date: Tue, 7 Jun 2022 11:19:32 +0200 Message-Id: <20220607091936.7948-30-cfontana@suse.de> In-Reply-To: <20220607091936.7948-1-cfontana@suse.de> References: <20220607091936.7948-1-cfontana@suse.de> MIME-Version: 1.0 X-Mimecast-Impersonation-Protect: Policy=CLT - Impersonation Protection Definition; Similar Internal Domain=false; Similar Monitored External Domain=false; Custom External Domain=false; Mimecast External Domain=false; Newly Observed Domain=false; Internal User Name=false; Custom Display Name List=false; Reply-to Address Mismatch=false; Targeted Threat Dictionary=false; Mimecast Threat Dictionary=false; Custom Threat Dictionary=false X-Scanned-By: MIMEDefang 2.85 on 10.11.54.10 X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: libvir-list@redhat.com, Claudio Fontana , "Dr . David Alan Gilbert" , Anthony Iliopoulos Errors-To: libvir-list-bounces@redhat.com Sender: "libvir-list" X-Scanned-By: MIMEDefang 2.78 on 10.11.54.3 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=libvir-list-bounces@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable X-ZM-MESSAGEID: 1654593629153100003 Content-Type: text/plain; charset="utf-8"; x-default="true" Signed-off-by: Claudio Fontana --- src/qemu/qemu_driver.c | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/src/qemu/qemu_driver.c b/src/qemu/qemu_driver.c index 8839e0031d..52f3300c55 100644 --- a/src/qemu/qemu_driver.c +++ b/src/qemu/qemu_driver.c @@ -2874,6 +2874,7 @@ qemuDomainSaveParams(virDomainPtr dom, g_autoptr(virCommand) compressor =3D NULL; const char *to =3D NULL; const char *dxml =3D NULL; + const char *pcomp =3D NULL; int compressed; int ret =3D -1; int nconn =3D 2; @@ -2890,6 +2891,8 @@ qemuDomainSaveParams(virDomainPtr dom, VIR_TYPED_PARAM_STRING, VIR_DOMAIN_SAVE_PARAM_PARALLEL_CONNECTIONS, VIR_TYPED_PARAM_INT, + VIR_DOMAIN_SAVE_PARAM_PARALLEL_COMPRESSION, + VIR_TYPED_PARAM_STRING, NULL) < 0) return -1; =20 @@ -2901,6 +2904,8 @@ qemuDomainSaveParams(virDomainPtr dom, return -1; if (virTypedParamsGetInt(params, nparams, VIR_DOMAIN_SAVE_PARAM_PARALL= EL_CONNECTIONS, &nconn) < 0) return -1; + if (virTypedParamsGetString(params, nparams, VIR_DOMAIN_SAVE_PARAM_PAR= ALLEL_COMPRESSION, &pcomp) < 0) + return -1; =20 if (!(vm =3D qemuDomainObjFromDomain(dom))) goto cleanup; @@ -2923,7 +2928,7 @@ qemuDomainSaveParams(virDomainPtr dom, goto cleanup; =20 ret =3D qemuDomainSaveInternal(driver, vm, to, compressed, - compressor, dxml, nconn, NULL, flags); + compressor, dxml, nconn, pcomp, flags); =20 cleanup: virDomainObjEndAPI(&vm); --=20 2.26.2 From nobody Fri May 17 11:58:59 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 170.10.133.124 as permitted sender) client-ip=170.10.133.124; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-delivery-124.mimecast.com; Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=fail(p=none dis=none) header.from=suse.de Return-Path: Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by mx.zohomail.com with SMTPS id 1654593664717345.5699468805327; Tue, 7 Jun 2022 02:21:04 -0700 (PDT) Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-422-jaqKQ7lWOuOBo-wBZtm2BA-1; Tue, 07 Jun 2022 05:20:18 -0400 Received: from smtp.corp.redhat.com (int-mx09.intmail.prod.int.rdu2.redhat.com [10.11.54.9]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 7A0BC804197; Tue, 7 Jun 2022 09:20:05 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (unknown [10.30.29.100]) by smtp.corp.redhat.com (Postfix) with ESMTP id 63CFA492CA3; Tue, 7 Jun 2022 09:20:05 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (localhost [IPv6:::1]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id D4C281947BA1; Tue, 7 Jun 2022 09:20:04 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.rdu2.redhat.com [10.11.54.8]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id 956091947B9E for ; Tue, 7 Jun 2022 09:20:03 +0000 (UTC) Received: by smtp.corp.redhat.com (Postfix) id 7F99DC27E92; Tue, 7 Jun 2022 09:20:03 +0000 (UTC) Received: from mimecast-mx02.redhat.com (mimecast10.extmail.prod.ext.rdu2.redhat.com [10.11.55.26]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 7B860C27E8F for ; Tue, 7 Jun 2022 09:20:03 +0000 (UTC) Received: from us-smtp-1.mimecast.com (us-smtp-2.mimecast.com [205.139.110.61]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 6488F1C0E345 for ; Tue, 7 Jun 2022 09:20:03 +0000 (UTC) Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.220.28]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-654-FViMiDDJOZuqJprDyfkogg-1; Tue, 07 Jun 2022 05:19:56 -0400 Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id C168B21B2F; Tue, 7 Jun 2022 09:19:49 +0000 (UTC) Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 89AB013638; Tue, 7 Jun 2022 09:19:49 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id 2Hr2HzUYn2IoLwAAMHmgww (envelope-from ); Tue, 07 Jun 2022 09:19:49 +0000 X-MC-Unique: jaqKQ7lWOuOBo-wBZtm2BA-1 X-Original-To: libvir-list@listman.corp.redhat.com X-MC-Unique: FViMiDDJOZuqJprDyfkogg-1 From: Claudio Fontana To: =?UTF-8?q?Daniel=20P=20=2E=20Berrang=C3=A9?= Subject: [libvirt RFCv11 30/33] qemu: migration: expose qemuMigrationParamsSetString Date: Tue, 7 Jun 2022 11:19:33 +0200 Message-Id: <20220607091936.7948-31-cfontana@suse.de> In-Reply-To: <20220607091936.7948-1-cfontana@suse.de> References: <20220607091936.7948-1-cfontana@suse.de> MIME-Version: 1.0 X-Mimecast-Impersonation-Protect: Policy=CLT - Impersonation Protection Definition; Similar Internal Domain=false; Similar Monitored External Domain=false; Custom External Domain=false; Mimecast External Domain=false; Newly Observed Domain=false; Internal User Name=false; Custom Display Name List=false; Reply-to Address Mismatch=false; Targeted Threat Dictionary=false; Mimecast Threat Dictionary=false; Custom Threat Dictionary=false X-Scanned-By: MIMEDefang 2.85 on 10.11.54.8 X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: libvir-list@redhat.com, Claudio Fontana , "Dr . David Alan Gilbert" , Anthony Iliopoulos Errors-To: libvir-list-bounces@redhat.com Sender: "libvir-list" X-Scanned-By: MIMEDefang 2.85 on 10.11.54.9 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=libvir-list-bounces@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable X-ZM-MESSAGEID: 1654593665526100001 Content-Type: text/plain; charset="utf-8"; x-default="true" change from static to external linkage, and move the function close to the other similar ones, near qemuMigrationParamsSetULL. Signed-off-by: Claudio Fontana --- src/qemu/qemu_migration_params.c | 47 +++++++++++++++----------------- src/qemu/qemu_migration_params.h | 5 ++++ 2 files changed, 27 insertions(+), 25 deletions(-) diff --git a/src/qemu/qemu_migration_params.c b/src/qemu/qemu_migration_par= ams.c index 43975640d9..3efe40fc66 100644 --- a/src/qemu/qemu_migration_params.c +++ b/src/qemu/qemu_migration_params.c @@ -900,31 +900,6 @@ qemuMigrationParamsApply(virQEMUDriver *driver, } =20 =20 -/** - * qemuMigrationParamsSetString: - * @migrParams: migration parameter object - * @param: parameter to set - * @value: new value - * - * Enables and sets the migration parameter @param in @migrParams. Returns= 0 on - * success and -1 on error. Libvirt error is reported. - */ -static int -qemuMigrationParamsSetString(qemuMigrationParams *migParams, - qemuMigrationParam param, - const char *value) -{ - if (qemuMigrationParamsCheckType(param, QEMU_MIGRATION_PARAM_TYPE_STRI= NG) < 0) - return -1; - - migParams->params[param].value.s =3D g_strdup(value); - - migParams->params[param].set =3D true; - - return 0; -} - - /* qemuMigrationParamsEnableTLS * @driver: pointer to qemu driver * @vm: domain object @@ -1146,6 +1121,28 @@ qemuMigrationParamsSetULL(qemuMigrationParams *migPa= rams, return 0; } =20 +/** + * qemuMigrationParamsSetString: + * @migrParams: migration parameter object + * @param: parameter to set + * @value: new value + * + * Enables and sets the migration parameter @param in @migrParams. Returns= 0 on + * success and -1 on error. Libvirt error is reported. + */ +int +qemuMigrationParamsSetString(qemuMigrationParams *migParams, + qemuMigrationParam param, + const char *value) +{ + if (qemuMigrationParamsCheckType(param, QEMU_MIGRATION_PARAM_TYPE_STRI= NG) < 0) + return -1; + + migParams->params[param].value.s =3D g_strdup(value); + migParams->params[param].set =3D true; + return 0; +} + =20 /** * Returns -1 on error, diff --git a/src/qemu/qemu_migration_params.h b/src/qemu/qemu_migration_par= ams.h index 900ad7cfc8..f69bb9b302 100644 --- a/src/qemu/qemu_migration_params.h +++ b/src/qemu/qemu_migration_params.h @@ -138,6 +138,11 @@ qemuMigrationParamsSetULL(qemuMigrationParams *migPara= ms, qemuMigrationParam param, unsigned long long value); =20 +int +qemuMigrationParamsSetString(qemuMigrationParams *migParams, + qemuMigrationParam param, + const char *value); + int qemuMigrationParamsGetULL(qemuMigrationParams *migParams, qemuMigrationParam param, --=20 2.26.2 From nobody Fri May 17 11:58:59 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 170.10.133.124 as permitted sender) client-ip=170.10.133.124; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-delivery-124.mimecast.com; Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=fail(p=none dis=none) header.from=suse.de Return-Path: Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by mx.zohomail.com with SMTPS id 1654593627943567.9490330312657; Tue, 7 Jun 2022 02:20:27 -0700 (PDT) Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-232-7kL3dUnJOAGLZJNEn3Hmkw-1; Tue, 07 Jun 2022 05:20:19 -0400 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.rdu2.redhat.com [10.11.54.1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 1C9151066683; Tue, 7 Jun 2022 09:20:08 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (unknown [10.30.29.100]) by smtp.corp.redhat.com (Postfix) with ESMTP id 05CAF40CFD0A; Tue, 7 Jun 2022 09:20:08 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (localhost [IPv6:::1]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id D816D1947B8C; Tue, 7 Jun 2022 09:20:07 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx09.intmail.prod.int.rdu2.redhat.com [10.11.54.9]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id BF3A01947B92 for ; Tue, 7 Jun 2022 09:20:04 +0000 (UTC) Received: by smtp.corp.redhat.com (Postfix) id A36AF492CA3; Tue, 7 Jun 2022 09:20:04 +0000 (UTC) Received: from mimecast-mx02.redhat.com (mimecast03.extmail.prod.ext.rdu2.redhat.com [10.11.55.19]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 9E52C492CA5 for ; Tue, 7 Jun 2022 09:20:04 +0000 (UTC) Received: from us-smtp-1.mimecast.com (us-smtp-1.mimecast.com [205.139.110.61]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 8842680C8EE for ; Tue, 7 Jun 2022 09:20:04 +0000 (UTC) Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.220.28]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-642-xQ-niNenOYuuFnu9goHR8g-1; Tue, 07 Jun 2022 05:20:01 -0400 Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id 0E09E21B59; Tue, 7 Jun 2022 09:19:50 +0000 (UTC) Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id CA43713638; Tue, 7 Jun 2022 09:19:49 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id QDfWLzUYn2IoLwAAMHmgww (envelope-from ); Tue, 07 Jun 2022 09:19:49 +0000 X-MC-Unique: 7kL3dUnJOAGLZJNEn3Hmkw-1 X-Original-To: libvir-list@listman.corp.redhat.com X-MC-Unique: xQ-niNenOYuuFnu9goHR8g-1 From: Claudio Fontana To: =?UTF-8?q?Daniel=20P=20=2E=20Berrang=C3=A9?= Subject: [libvirt RFCv11 31/33] qemu: saveimage: implement multifd-compression in parallel save Date: Tue, 7 Jun 2022 11:19:34 +0200 Message-Id: <20220607091936.7948-32-cfontana@suse.de> In-Reply-To: <20220607091936.7948-1-cfontana@suse.de> References: <20220607091936.7948-1-cfontana@suse.de> MIME-Version: 1.0 X-Mimecast-Impersonation-Protect: Policy=CLT - Impersonation Protection Definition; Similar Internal Domain=false; Similar Monitored External Domain=false; Custom External Domain=false; Mimecast External Domain=false; Newly Observed Domain=false; Internal User Name=false; Custom Display Name List=false; Reply-to Address Mismatch=false; Targeted Threat Dictionary=false; Mimecast Threat Dictionary=false; Custom Threat Dictionary=false X-Scanned-By: MIMEDefang 2.85 on 10.11.54.9 X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: libvir-list@redhat.com, Claudio Fontana , "Dr . David Alan Gilbert" , Anthony Iliopoulos Errors-To: libvir-list-bounces@redhat.com Sender: "libvir-list" X-Scanned-By: MIMEDefang 2.84 on 10.11.54.1 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=libvir-list-bounces@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable X-ZM-MESSAGEID: 1654593629158100004 Content-Type: text/plain; charset="utf-8"; x-default="true" Signed-off-by: Claudio Fontana --- src/qemu/qemu_migration.c | 17 +++++++++++++---- src/qemu/qemu_migration.h | 2 +- src/qemu/qemu_saveimage.c | 15 ++++++++++++--- 3 files changed, 26 insertions(+), 8 deletions(-) diff --git a/src/qemu/qemu_migration.c b/src/qemu/qemu_migration.c index c214f88dd1..5127dc07fe 100644 --- a/src/qemu/qemu_migration.c +++ b/src/qemu/qemu_migration.c @@ -5884,7 +5884,7 @@ qemuMigrationSrcToFileAux(virQEMUDriver *driver, virD= omainObj *vm, virCommand *compressor, virDomainAsyncJob asyncJob, const char *sun_path, - int nchannels) + int nchannels, const char *pcomp) { qemuDomainObjPrivate *priv =3D vm->privateData; bool bwParam =3D virQEMUCapsGet(priv->qemuCaps, QEMU_CAPS_MIGRATION_PA= RAM_BANDWIDTH); @@ -5931,6 +5931,15 @@ qemuMigrationSrcToFileAux(virQEMUDriver *driver, vir= DomainObj *vm, QEMU_MIGRATION_PARAM_MULTIFD_CHANNEL= S, nchannels) < 0) return -1; + if (virQEMUCapsGet(priv->qemuCaps, QEMU_CAPS_MIGRATION_PARAM_MULTI= FD_COMPRESSION)) { + if (qemuMigrationParamsSetString(migParams, + QEMU_MIGRATION_PARAM_MULTIFD_= COMPRESSION, pcomp) < 0) + return -1; + } else { + virReportError(VIR_ERR_OPERATION_UNSUPPORTED, "%s", + _("QEMU does not seem to support multifd compre= ssion")); + return -1; + } } =20 if (needParams && qemuMigrationParamsApply(driver, vm, asyncJob, migPa= rams) < 0) @@ -6056,17 +6065,17 @@ qemuMigrationSrcToFile(virQEMUDriver *driver, virDo= mainObj *vm, virDomainAsyncJob asyncJob) { return qemuMigrationSrcToFileAux(driver, vm, fd, compressor, - asyncJob, NULL, -1); + asyncJob, NULL, -1, NULL); } =20 int qemuMigrationSrcToFilesMultiFd(virQEMUDriver *driver, virDomainObj *vm, virDomainAsyncJob asyncJob, const char *sun_path, - int nchannels) + int nchannels, const char *pcomp) { return qemuMigrationSrcToFileAux(driver, vm, -1, NULL, - asyncJob, sun_path, nchannels); + asyncJob, sun_path, nchannels, pcomp); } =20 int diff --git a/src/qemu/qemu_migration.h b/src/qemu/qemu_migration.h index 38f4877cf0..d6185770b2 100644 --- a/src/qemu/qemu_migration.h +++ b/src/qemu/qemu_migration.h @@ -223,7 +223,7 @@ int qemuMigrationSrcToFilesMultiFd(virQEMUDriver *driver, virDomainObj *vm, virDomainAsyncJob asyncJob, const char *sun_path, - int nchannels) + int nchannels, const char *pcomp) ATTRIBUTE_NONNULL(1) ATTRIBUTE_NONNULL(2) G_GNUC_WARN_UNUSED_RESULT; =20 int diff --git a/src/qemu/qemu_saveimage.c b/src/qemu/qemu_saveimage.c index 505d9b615f..f60040ef25 100644 --- a/src/qemu/qemu_saveimage.c +++ b/src/qemu/qemu_saveimage.c @@ -561,12 +561,13 @@ qemuSaveImageCreate(virQEMUDriver *driver, virQEMUSaveData *data, virCommand *compressor, int nconn, - const char *pcomp G_GNUC_UNUSED, + const char *pcomp, unsigned int flags, virDomainAsyncJob asyncJob) { g_autoptr(virQEMUDriverConfig) cfg =3D virQEMUDriverGetConfig(driver); virQEMUSaveFd saveFd =3D QEMU_SAVEFD_INVALID; + virQEMUSaveMultiFdComp multiComp =3D QEMU_SAVE_MULTIFD_COMP_NONE; unsigned int oflags =3D O_WRONLY | O_TRUNC | O_CREAT; int ret =3D -1; =20 @@ -578,11 +579,19 @@ qemuSaveImageCreate(virQEMUDriver *driver, } oflags |=3D O_DIRECT; } - + if (!pcomp || !pcomp[0]) { + pcomp =3D qemuSaveMultiFdCompTypeToString(QEMU_SAVE_MULTIFD_COMP_N= ONE); + } if (virQEMUSaveFdInit(&saveFd, path, oflags, cfg, flags & VIR_DOMAIN_S= AVE_PARALLEL) < 0) goto cleanup; if (nconn > 0) { data->header.multifd_channels =3D nconn; + if ((multiComp =3D qemuSaveMultiFdCompTypeFromString(pcomp)) < 0) { + virReportError(VIR_ERR_OPERATION_FAILED, + _("Invalid %s multifd compression format specif= ied"), pcomp); + goto cleanup; + } + data->header.multifd_comp =3D multiComp; } if (qemuSecuritySetImageFDLabel(driver->securityManager, vm->def, save= Fd.fd) < 0) goto cleanup; @@ -614,7 +623,7 @@ qemuSaveImageCreate(virQEMUDriver *driver, goto cleanup; if (chown(sun_path, cfg->user, cfg->group) < 0) goto cleanup; - if (qemuMigrationSrcToFilesMultiFd(driver, vm, asyncJob, sun_path,= nconn) < 0) + if (qemuMigrationSrcToFilesMultiFd(driver, vm, asyncJob, sun_path,= nconn, pcomp) < 0) goto cleanup; } else { /* Perform non-parallel migration to file */ --=20 2.26.2 From nobody Fri May 17 11:58:59 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 170.10.133.124 as permitted sender) client-ip=170.10.133.124; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-delivery-124.mimecast.com; Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=fail(p=none dis=none) header.from=suse.de Return-Path: Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by mx.zohomail.com with SMTPS id 165459362195556.79217310346826; Tue, 7 Jun 2022 02:20:21 -0700 (PDT) Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-80-FxlQGHj2MxWhfUkqYZDMZg-1; Tue, 07 Jun 2022 05:20:15 -0400 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.rdu2.redhat.com [10.11.54.6]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 5D4F48517C6; Tue, 7 Jun 2022 09:19:57 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (unknown [10.30.29.100]) by smtp.corp.redhat.com (Postfix) with ESMTP id 43A782166B26; Tue, 7 Jun 2022 09:19:57 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (localhost [IPv6:::1]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id 208F51947B86; Tue, 7 Jun 2022 09:19:57 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx10.intmail.prod.int.rdu2.redhat.com [10.11.54.10]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id 802D719452D2 for ; Tue, 7 Jun 2022 09:19:54 +0000 (UTC) Received: by smtp.corp.redhat.com (Postfix) id 6544440336E; Tue, 7 Jun 2022 09:19:54 +0000 (UTC) Received: from mimecast-mx02.redhat.com (mimecast04.extmail.prod.ext.rdu2.redhat.com [10.11.55.20]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 610DE492C3B for ; Tue, 7 Jun 2022 09:19:54 +0000 (UTC) Received: from us-smtp-1.mimecast.com (us-smtp-delivery-1.mimecast.com [207.211.31.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 4A93C100BAB8 for ; Tue, 7 Jun 2022 09:19:54 +0000 (UTC) Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.220.29]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-463-og3Td2qLOJGnl-rZMYAvUA-1; Tue, 07 Jun 2022 05:19:52 -0400 Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id 5C08E1F9A8; Tue, 7 Jun 2022 09:19:50 +0000 (UTC) Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 17DD313638; Tue, 7 Jun 2022 09:19:50 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id sP0vBDYYn2IoLwAAMHmgww (envelope-from ); Tue, 07 Jun 2022 09:19:50 +0000 X-MC-Unique: FxlQGHj2MxWhfUkqYZDMZg-1 X-Original-To: libvir-list@listman.corp.redhat.com X-MC-Unique: og3Td2qLOJGnl-rZMYAvUA-1 From: Claudio Fontana To: =?UTF-8?q?Daniel=20P=20=2E=20Berrang=C3=A9?= Subject: [libvirt RFCv11 32/33] qemu: saveimage: restore compressed parallel images Date: Tue, 7 Jun 2022 11:19:35 +0200 Message-Id: <20220607091936.7948-33-cfontana@suse.de> In-Reply-To: <20220607091936.7948-1-cfontana@suse.de> References: <20220607091936.7948-1-cfontana@suse.de> MIME-Version: 1.0 X-Mimecast-Impersonation-Protect: Policy=CLT - Impersonation Protection Definition; Similar Internal Domain=false; Similar Monitored External Domain=false; Custom External Domain=false; Mimecast External Domain=false; Newly Observed Domain=false; Internal User Name=false; Custom Display Name List=false; Reply-to Address Mismatch=false; Targeted Threat Dictionary=false; Mimecast Threat Dictionary=false; Custom Threat Dictionary=false X-Scanned-By: MIMEDefang 2.85 on 10.11.54.10 X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: libvir-list@redhat.com, Claudio Fontana , "Dr . David Alan Gilbert" , Anthony Iliopoulos Errors-To: libvir-list-bounces@redhat.com Sender: "libvir-list" X-Scanned-By: MIMEDefang 2.78 on 10.11.54.6 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=libvir-list-bounces@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable X-ZM-MESSAGEID: 1654593622396100013 Content-Type: text/plain; charset="utf-8"; x-default="true" Signed-off-by: Claudio Fontana --- src/qemu/qemu_saveimage.c | 17 +++++++++++++++++ 1 file changed, 17 insertions(+) diff --git a/src/qemu/qemu_saveimage.c b/src/qemu/qemu_saveimage.c index f60040ef25..d3757254a9 100644 --- a/src/qemu/qemu_saveimage.c +++ b/src/qemu/qemu_saveimage.c @@ -708,6 +708,23 @@ int qemuSaveImageLoadMultiFd(virConnectPtr conn, virDo= mainObj *vm, QEMU_MIGRATION_PARAM_MULTIFD_CHANNEL= S, saveFd->nchannels) < 0) goto cleanup; + if (virQEMUCapsGet(priv->qemuCaps, QEMU_CAPS_MIGRATION_PARAM_MULTI= FD_COMPRESSION)) { + const char *pcomp =3D qemuSaveMultiFdCompTypeToString(data->he= ader.multifd_comp); + if (!pcomp) { + virReportError(VIR_ERR_OPERATION_UNSUPPORTED, + _("libvirt does not support parallel compre= ssion type %u"), + data->header.multifd_comp); + goto cleanup; + } + if (qemuMigrationParamsSetString(migParams, + QEMU_MIGRATION_PARAM_MULTIFD_= COMPRESSION, + pcomp) < 0) + goto cleanup; + } else if (data->header.multifd_comp !=3D QEMU_SAVE_MULTIFD_COMP_N= ONE) { + virReportError(VIR_ERR_OPERATION_UNSUPPORTED, "%s", + _("QEMU does not seem to support multifd compre= ssion")); + goto cleanup; + } if (qemuMigrationParamsApply(driver, vm, asyncJob, migParams) < 0) goto cleanup; =20 --=20 2.26.2 From nobody Fri May 17 11:58:59 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 170.10.133.124 as permitted sender) client-ip=170.10.133.124; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-delivery-124.mimecast.com; Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=fail(p=none dis=none) header.from=suse.de Return-Path: Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by mx.zohomail.com with SMTPS id 1654593628918976.3390442063948; Tue, 7 Jun 2022 02:20:28 -0700 (PDT) Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-468-s4t3o9MrN3qXV0_uiLb5VA-1; Tue, 07 Jun 2022 05:20:19 -0400 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id B541480C914; Tue, 7 Jun 2022 09:20:06 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (unknown [10.30.29.100]) by smtp.corp.redhat.com (Postfix) with ESMTP id 9C0992026D64; Tue, 7 Jun 2022 09:20:06 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (localhost [IPv6:::1]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id 6D6041947BA4; Tue, 7 Jun 2022 09:20:06 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id 9AE0A1947BAD for ; Tue, 7 Jun 2022 09:20:04 +0000 (UTC) Received: by smtp.corp.redhat.com (Postfix) id 816022026D07; Tue, 7 Jun 2022 09:20:04 +0000 (UTC) Received: from mimecast-mx02.redhat.com (mimecast09.extmail.prod.ext.rdu2.redhat.com [10.11.55.25]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 7D46F2026D64 for ; Tue, 7 Jun 2022 09:20:04 +0000 (UTC) Received: from us-smtp-1.mimecast.com (us-smtp-delivery-1.mimecast.com [205.139.110.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 62887280BE2E for ; Tue, 7 Jun 2022 09:20:04 +0000 (UTC) Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.220.29]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-308-3gQdqaWJPc-V3kPLOu5VIQ-1; Tue, 07 Jun 2022 05:20:02 -0400 Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id 94F821F9B2; Tue, 7 Jun 2022 09:19:50 +0000 (UTC) Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 5A2B813B40; Tue, 7 Jun 2022 09:19:50 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id IMZuFDYYn2IoLwAAMHmgww (envelope-from ); Tue, 07 Jun 2022 09:19:50 +0000 X-MC-Unique: s4t3o9MrN3qXV0_uiLb5VA-1 X-Original-To: libvir-list@listman.corp.redhat.com X-MC-Unique: 3gQdqaWJPc-V3kPLOu5VIQ-1 From: Claudio Fontana To: =?UTF-8?q?Daniel=20P=20=2E=20Berrang=C3=A9?= Subject: [libvirt RFCv11 33/33] tools: add parallel-compression parameter to virsh save command Date: Tue, 7 Jun 2022 11:19:36 +0200 Message-Id: <20220607091936.7948-34-cfontana@suse.de> In-Reply-To: <20220607091936.7948-1-cfontana@suse.de> References: <20220607091936.7948-1-cfontana@suse.de> MIME-Version: 1.0 X-Mimecast-Impersonation-Protect: Policy=CLT - Impersonation Protection Definition; Similar Internal Domain=false; Similar Monitored External Domain=false; Custom External Domain=false; Mimecast External Domain=false; Newly Observed Domain=false; Internal User Name=false; Custom Display Name List=false; Reply-to Address Mismatch=false; Targeted Threat Dictionary=false; Mimecast Threat Dictionary=false; Custom Threat Dictionary=false X-Scanned-By: MIMEDefang 2.78 on 10.11.54.4 X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: libvir-list@redhat.com, Claudio Fontana , "Dr . David Alan Gilbert" , Anthony Iliopoulos Errors-To: libvir-list-bounces@redhat.com Sender: "libvir-list" X-Scanned-By: MIMEDefang 2.78 on 10.11.54.4 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=libvir-list-bounces@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable X-ZM-MESSAGEID: 1654593631112100011 Content-Type: text/plain; charset="utf-8"; x-default="true" this completes the save side of the parallel compression support. Signed-off-by: Claudio Fontana --- docs/manpages/virsh.rst | 4 ++++ tools/virsh-domain.c | 12 ++++++++++++ 2 files changed, 16 insertions(+) diff --git a/docs/manpages/virsh.rst b/docs/manpages/virsh.rst index 5abb03be9a..19fe94ed73 100644 --- a/docs/manpages/virsh.rst +++ b/docs/manpages/virsh.rst @@ -3811,6 +3811,7 @@ save =20 save domain state-file [--bypass-cache] [--xml file] [--parallel] [--parallel-connections connections] + [--parallel-compression algo] [{--running | --paused}] [--verbose] =20 Saves a running domain (RAM, but not disk state) to a state file so that @@ -3849,6 +3850,9 @@ can be set using *--parallel-connections*. =20 Parallel connections help in speeding up large save operations. =20 +*--parallel-compression* can be used to ask the hypervisor to provide +compressed channels in the save stream using algorithm ``algo``. + Domain saved state files assume that disk images will be unchanged between the creation and restore point. For a more complete system restore point, where the disk state is saved alongside the memory diff --git a/tools/virsh-domain.c b/tools/virsh-domain.c index 98a1825c98..dfb62b592e 100644 --- a/tools/virsh-domain.c +++ b/tools/virsh-domain.c @@ -4182,6 +4182,10 @@ static const vshCmdOptDef opts_save[] =3D { .type =3D VSH_OT_INT, .help =3D N_("number of extra connections to use for parallel save") }, + {.name =3D "parallel-compression", + .type =3D VSH_OT_STRING, + .help =3D N_("compression algorithm and format for parallel save") + }, {.name =3D "xml", .type =3D VSH_OT_STRING, .completer =3D virshCompletePathLocalExisting, @@ -4211,6 +4215,7 @@ doSave(void *opaque) g_autoptr(virshDomain) dom =3D NULL; const char *name =3D NULL; const char *to =3D NULL; + const char *pcomp =3D NULL; virTypedParameterPtr params =3D NULL; int nparams =3D 0; int maxparams =3D 0; @@ -4246,6 +4251,13 @@ doSave(void *opaque) VIR_DOMAIN_SAVE_PARAM_PARALLEL_CONNECTION= S, intOpt) < 0) goto out; } + if ((rv =3D vshCommandOptStringReq(ctl, cmd, "parallel-compression", &= pcomp)) < 0) { + goto out; + } else { + if (virTypedParamsAddString(¶ms, &nparams, &maxparams, + VIR_DOMAIN_SAVE_PARAM_PARALLEL_COMPRES= SION, pcomp) < 0) + goto out; + } if (vshCommandOptBool(cmd, "running")) flags |=3D VIR_DOMAIN_SAVE_RUNNING; if (vshCommandOptBool(cmd, "paused")) --=20 2.26.2