From nobody Mon Feb 9 06:34:02 2026 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 170.10.129.124 as permitted sender) client-ip=170.10.129.124; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-delivery-124.mimecast.com; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1638543858; cv=none; d=zohomail.com; s=zohoarc; b=MNYGLKpUIfpVy2nH+MjnZL0X+3/c8VsF1vfitDow5A6l59LHxx6wgjneya12xay4RSxJOnRu7OXzwTwCbdY4620pxUuXPXrsXroYGG8JSgI7DqdkBrYm7vfSoRGVP3M0zYk3cvTxo8WQbnDrYBo+4v43i+eEy9aaMG3c/fC2P50= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1638543858; h=Content-Type:Content-Transfer-Encoding:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=R86AHGcsUTcq3Nw8varfGJ4PcyuzD077JibZTf2UqPA=; b=kq025Vkl2O8HJ3i4YQSuICnP37lNOyNFmTWwxK3kZG1s0+JX4AYUYfbjrLCBCEEDY/I7xaHor1wE353KJ35SSv2rVknHOjEv39sySMKT8ZHF6gejQ0SRWknkU7lBWunEikEQA0l0t55DBfZLOt/SQFdNf0gYnMUxlYcccwNOL5A= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by mx.zohomail.com with SMTPS id 1638543858723888.2139129384227; Fri, 3 Dec 2021 07:04:18 -0800 (PST) Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-37-R2Y_Sx3lNcu-IZhMNIuHDQ-1; Fri, 03 Dec 2021 10:04:13 -0500 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id AD09683DBF5; Fri, 3 Dec 2021 15:04:06 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.20]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 82D705E26C; Fri, 3 Dec 2021 15:04:06 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id 4E14B1802E38; Fri, 3 Dec 2021 15:04:06 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id 1B3F44gr031209 for ; Fri, 3 Dec 2021 10:04:04 -0500 Received: by smtp.corp.redhat.com (Postfix) id 588BA5DAA5; Fri, 3 Dec 2021 15:04:04 +0000 (UTC) Received: from speedmetal.lan (unknown [10.40.208.10]) by smtp.corp.redhat.com (Postfix) with ESMTP id A47CF5D9D5 for ; Fri, 3 Dec 2021 15:04:03 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1638543857; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=R86AHGcsUTcq3Nw8varfGJ4PcyuzD077JibZTf2UqPA=; b=N7O0/qoa8zhVcey8xdNf9XCvQzNZweo8NVLeI1wmdezYPBBXEX7U5UGyP6twSLPzJFeRWh 9xJdCqu6WOavOrV/XCOxURqDcRtifGcwGJHzYUa2s2zrRH5CZ7Zw5xvBroF59iBa+/CVCt GDLXF08BW3v8CDQ9WgcIL82uHoSfZeI= X-MC-Unique: R2Y_Sx3lNcu-IZhMNIuHDQ-1 From: Peter Krempa To: libvir-list@redhat.com Subject: [PATCH 13/17] virDomainMigrate: Introduce VIR_MIGRATE_NON_SHARED_SYNCHRONOUS_WRITES flag Date: Fri, 3 Dec 2021 16:01:52 +0100 Message-Id: In-Reply-To: References: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 X-loop: libvir-list@redhat.com X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=libvir-list-bounces@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1638543859160100006 Content-Type: text/plain; charset="utf-8" Non-shared storage migration of guests which are disk I/O intensive and have fast local storage may actually never converge if the guest happens to dirty the disk faster than it can be copied. This patch introduces a new flag 'VIR_MIGRATE_NON_SHARED_SYNCHRONOUS_WRITES' which will instruct hypervisors to synchronize local I/O writes with the writes to remote storage used for migration so that the guest can't overwhelm the migration. This comes at a cost of decreased local I/O performance for guests which behave well on average. Signed-off-by: Peter Krempa --- docs/manpages/virsh.rst | 6 +++++- include/libvirt/libvirt-domain.h | 10 ++++++++++ src/libvirt-domain.c | 20 ++++++++++++++++++++ tools/virsh-domain.c | 13 +++++++++++++ 4 files changed, 48 insertions(+), 1 deletion(-) diff --git a/docs/manpages/virsh.rst b/docs/manpages/virsh.rst index 7c50388216..1ce3e77c9f 100644 --- a/docs/manpages/virsh.rst +++ b/docs/manpages/virsh.rst @@ -3180,7 +3180,7 @@ migrate [--postcopy-bandwidth bandwidth] [--parallel [--parallel-connections connections]] [--bandwidth bandwidth] [--tls-destination hostname] - [--disks-uri URI] + [--disks-uri URI] [--copy-storage-synchronous-writes] Migrate domain to another host. Add *--live* for live migration; <--p2p> for peer-2-peer migration; *--direct* for direct migration; or *--tunnelle= d* @@ -3202,6 +3202,10 @@ images on source host to the images found at the sam= e place on the destination host. By default only non-shared non-readonly images are transferred. Use *--migrate-disks* to explicitly specify a list of disk targets to transfer via the comma separated ``disk-list`` argument. +With *--copy-storage-synchronous-writes* flag used the disk data migration= will +synchronous handle guest disk writes to both the original soure and the +destination to ensure that the disk migration coverges at the price of pos= sibly +decreased burst performance. *--change-protection* enforces that no incompatible configuration changes = will be made to the domain while the migration is underway; this flag is implic= itly diff --git a/include/libvirt/libvirt-domain.h b/include/libvirt/libvirt-dom= ain.h index f81e96d374..d0dd11ab01 100644 --- a/include/libvirt/libvirt-domain.h +++ b/include/libvirt/libvirt-domain.h @@ -850,6 +850,16 @@ typedef enum { */ VIR_MIGRATE_PARALLEL =3D (1 << 17), + /* Force the guest writes which happen when copying disk images for + * non-shared storage migration to be synchronously written to the + * destination. This ensures the storage migration converges for VMs + * doing heavy I/O on fast local storage and slow mirror. + * + * Requires one of VIR_MIGRATE_NON_SHARED_DISK, VIR_MIGRATE_NON_SHARE= D_INC + * to be present as well. + */ + VIR_MIGRATE_NON_SHARED_SYNCHRONOUS_WRITES =3D (1 << 18), + } virDomainMigrateFlags; diff --git a/src/libvirt-domain.c b/src/libvirt-domain.c index 8ee2490867..5708ff839b 100644 --- a/src/libvirt-domain.c +++ b/src/libvirt-domain.c @@ -3567,6 +3567,10 @@ virDomainMigrate(virDomainPtr domain, VIR_MIGRATE_PARALLEL, error); + VIR_REQUIRE_FLAG_GOTO(VIR_MIGRATE_NON_SHARED_SYNCHRONOUS_WRITES, + VIR_MIGRATE_NON_SHARED_DISK | VIR_MIGRATE_NON_SH= ARED_INC, + error); + if (flags & VIR_MIGRATE_OFFLINE) { rc =3D VIR_DRV_SUPPORTS_FEATURE(domain->conn->driver, domain->conn, VIR_DRV_FEATURE_MIGRATION_OFFLINE); @@ -3760,6 +3764,10 @@ virDomainMigrate2(virDomainPtr domain, VIR_MIGRATE_PARALLEL, error); + VIR_REQUIRE_FLAG_GOTO(VIR_MIGRATE_NON_SHARED_SYNCHRONOUS_WRITES, + VIR_MIGRATE_NON_SHARED_DISK | VIR_MIGRATE_NON_SH= ARED_INC, + error); + if (flags & VIR_MIGRATE_OFFLINE) { rc =3D VIR_DRV_SUPPORTS_FEATURE(domain->conn->driver, domain->conn, VIR_DRV_FEATURE_MIGRATION_OFFLINE); @@ -3966,6 +3974,14 @@ virDomainMigrate3(virDomainPtr domain, VIR_MIGRATE_NON_SHARED_INC, error); + VIR_REQUIRE_FLAG_GOTO(VIR_MIGRATE_NON_SHARED_SYNCHRONOUS_WRITES, + VIR_MIGRATE_NON_SHARED_DISK | VIR_MIGRATE_NON_SH= ARED_INC, + error); + + VIR_REQUIRE_FLAG_GOTO(VIR_MIGRATE_NON_SHARED_SYNCHRONOUS_WRITES, + VIR_MIGRATE_NON_SHARED_DISK | VIR_MIGRATE_NON_SH= ARED_INC, + error); + if (flags & VIR_MIGRATE_PEER2PEER) { virReportInvalidArg(flags, "%s", _("use virDomainMigrateToURI3 for peer-to-peer= " @@ -4137,6 +4153,10 @@ int virDomainMigrateUnmanagedCheckCompat(virDomainPt= r domain, VIR_MIGRATE_NON_SHARED_INC, -1); + VIR_REQUIRE_FLAG_RET(VIR_MIGRATE_NON_SHARED_SYNCHRONOUS_WRITES, + VIR_MIGRATE_NON_SHARED_DISK | VIR_MIGRATE_NON_SHA= RED_INC, + -1); + if (flags & VIR_MIGRATE_OFFLINE) { rc =3D VIR_DRV_SUPPORTS_FEATURE(domain->conn->driver, domain->conn, VIR_DRV_FEATURE_MIGRATION_OFFLINE); diff --git a/tools/virsh-domain.c b/tools/virsh-domain.c index a00b4cc243..8379f9f135 100644 --- a/tools/virsh-domain.c +++ b/tools/virsh-domain.c @@ -10543,6 +10543,10 @@ static const vshCmdOptDef opts_migrate[] =3D { .type =3D VSH_OT_BOOL, .help =3D N_("migration with non-shared storage with incremental copy= (same base image shared between source and destination)") }, + {.name =3D "copy-storage-synchronous-writes", + .type =3D VSH_OT_BOOL, + .help =3D N_("force guest disk writes to be synchronously written to = the destination to improve storage migration convergence") + }, {.name =3D "change-protection", .type =3D VSH_OT_BOOL, .help =3D N_("prevent any configuration changes to domain until migra= tion ends") @@ -10949,6 +10953,15 @@ doMigrate(void *opaque) if (vshCommandOptBool(cmd, "copy-storage-inc")) flags |=3D VIR_MIGRATE_NON_SHARED_INC; + if (vshCommandOptBool(cmd, "copy-storage-synchronous-writes")) { + if (!(flags & VIR_MIGRATE_NON_SHARED_DISK) && + !(flags & VIR_MIGRATE_NON_SHARED_INC)) { + vshError(ctl, "'--copy-storage-synchronous-writes' requires on= e of '--copy-storage-all', 'copy-storage-inc'"); + goto out; + } + flags |=3D VIR_MIGRATE_NON_SHARED_SYNCHRONOUS_WRITES; + } + if (vshCommandOptBool(cmd, "change-protection")) flags |=3D VIR_MIGRATE_CHANGE_PROTECTION; --=20 2.31.1