From nobody Tue Apr 16 23:00:06 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) client-ip=209.132.183.28; envelope-from=libvir-list-bounces@redhat.com; helo=mx1.redhat.com; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com Return-Path: Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by mx.zohomail.com with SMTPS id 1549638515689637.4110597640234; Fri, 8 Feb 2019 07:08:35 -0800 (PST) Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 1FAF588E4F; Fri, 8 Feb 2019 15:08:32 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.20]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 919C45DE6E; Fri, 8 Feb 2019 15:08:31 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id 78E301803390; Fri, 8 Feb 2019 15:08:30 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.phx2.redhat.com [10.5.11.22]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id x18F8StG003444 for ; Fri, 8 Feb 2019 10:08:28 -0500 Received: by smtp.corp.redhat.com (Postfix) id ACC781042A62; Fri, 8 Feb 2019 15:08:28 +0000 (UTC) Received: from virval.usersys.redhat.com (unknown [10.43.2.20]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 7FB4A1048109 for ; Fri, 8 Feb 2019 15:08:26 +0000 (UTC) Received: by virval.usersys.redhat.com (Postfix, from userid 500) id 027621002A8; Fri, 8 Feb 2019 16:08:25 +0100 (CET) From: Jiri Denemark To: libvir-list@redhat.com Date: Fri, 8 Feb 2019 16:08:21 +0100 Message-Id: <4e3e117ba2bcb333b3cbbd7f0802332816d094a1.1549638450.git.jdenemar@redhat.com> In-Reply-To: References: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.22 X-loop: libvir-list@redhat.com Subject: [libvirt] [PATCH v2 1/3] Public API for parallel migration X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Transfer-Encoding: quoted-printable Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.25]); Fri, 08 Feb 2019 15:08:34 +0000 (UTC) Content-Type: text/plain; charset="utf-8" This patch adds a new VIR_MIGRATE_PARALLEL flag for migration APIs which will ask the hypervisor to use multiple parallel connections for migrating a domain. The number of parallel connections can be set using VIR_MIGRATE_PARAM_PARALLEL_CONNECTIONS typed parameter. Signed-off-by: Jiri Denemark Reviewed-by: John Ferlan --- include/libvirt/libvirt-domain.h | 14 ++++++++++++++ 1 file changed, 14 insertions(+) diff --git a/include/libvirt/libvirt-domain.h b/include/libvirt/libvirt-dom= ain.h index d82c75a9d9..b2d82d8152 100644 --- a/include/libvirt/libvirt-domain.h +++ b/include/libvirt/libvirt-domain.h @@ -831,6 +831,12 @@ typedef enum { */ VIR_MIGRATE_TLS =3D (1 << 16), =20 + /* Send memory pages to the destination host through several network + * connections. See VIR_MIGRATE_PARAM_PARALLEL_* parameters for + * configuring the parallel migration. + */ + VIR_MIGRATE_PARALLEL =3D (1 << 17), + } virDomainMigrateFlags; =20 =20 @@ -1025,6 +1031,14 @@ typedef enum { */ # define VIR_MIGRATE_PARAM_AUTO_CONVERGE_INCREMENT "auto_converge.increme= nt" =20 +/** + * VIR_MIGRATE_PARAM_PARALLEL_CONNECTIONS: + * + * virDomainMigrate* params field: number of connections used during paral= lel + * migration. As VIR_TYPED_PARAM_INT. + */ +# define VIR_MIGRATE_PARAM_PARALLEL_CONNECTIONS "parallel.connections" + /* Domain migration. */ virDomainPtr virDomainMigrate (virDomainPtr domain, virConnectPtr dconn, unsigned long flags, const char *dname, --=20 2.20.1 -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list From nobody Tue Apr 16 23:00:06 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) client-ip=209.132.183.28; envelope-from=libvir-list-bounces@redhat.com; helo=mx1.redhat.com; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com Return-Path: Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by mx.zohomail.com with SMTPS id 1549638517777245.08015938504445; Fri, 8 Feb 2019 07:08:37 -0800 (PST) Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 0AA59C4EDF; Fri, 8 Feb 2019 15:08:34 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.21]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 879A6608C2; Fri, 8 Feb 2019 15:08:33 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id EB4453F7CD; Fri, 8 Feb 2019 15:08:32 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id x18F8Tbk003454 for ; Fri, 8 Feb 2019 10:08:29 -0500 Received: by smtp.corp.redhat.com (Postfix) id 6243D608C2; Fri, 8 Feb 2019 15:08:29 +0000 (UTC) Received: from virval.usersys.redhat.com (unknown [10.43.2.20]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 072AF18233 for ; Fri, 8 Feb 2019 15:08:26 +0000 (UTC) Received: by virval.usersys.redhat.com (Postfix, from userid 500) id 05596102EBB; Fri, 8 Feb 2019 16:08:25 +0100 (CET) From: Jiri Denemark To: libvir-list@redhat.com Date: Fri, 8 Feb 2019 16:08:22 +0100 Message-Id: <4131db38e0b75ed52ccd5e31d049dbf7cd9545b1.1549638450.git.jdenemar@redhat.com> In-Reply-To: References: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 X-loop: libvir-list@redhat.com Subject: [libvirt] [PATCH v2 2/3] qemu: Add support for parallel migration X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Transfer-Encoding: quoted-printable Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.38]); Fri, 08 Feb 2019 15:08:35 +0000 (UTC) Content-Type: text/plain; charset="utf-8" The VIR_MIGRATE_PARALLEL flag is implemented using QEMU's multifd migration capability and the corresponding multifd-channels migration parameter. Signed-off-by: Jiri Denemark Reviewed-by: John Ferlan --- Notes: QEMU still uses the x- prefix for multifd capability and multifd-channe= ls, but Juan already sent a series of patches to make multifd migration ful= ly supported by dropping the experimental prefix. src/qemu/qemu_migration.c | 9 ++++++--- src/qemu/qemu_migration.h | 5 ++++- src/qemu/qemu_migration_params.c | 18 ++++++++++++++++++ src/qemu/qemu_migration_params.h | 2 ++ 4 files changed, 30 insertions(+), 4 deletions(-) diff --git a/src/qemu/qemu_migration.c b/src/qemu/qemu_migration.c index 3107a279dd..22f31680c9 100644 --- a/src/qemu/qemu_migration.c +++ b/src/qemu/qemu_migration.c @@ -3822,10 +3822,13 @@ qemuMigrationSrcPerformNative(virQEMUDriverPtr driv= er, } } =20 - if (STRNEQ(uribits->scheme, "rdma")) - spec.destType =3D MIGRATION_DEST_CONNECT_HOST; - else + /* RDMA and multi-fd migration requires QEMU to connect to the destina= tion + * itself. + */ + if (STREQ(uribits->scheme, "rdma") || (flags & VIR_MIGRATE_PARALLEL)) spec.destType =3D MIGRATION_DEST_HOST; + else + spec.destType =3D MIGRATION_DEST_CONNECT_HOST; spec.dest.host.protocol =3D uribits->scheme; spec.dest.host.name =3D uribits->server; spec.dest.host.port =3D uribits->port; diff --git a/src/qemu/qemu_migration.h b/src/qemu/qemu_migration.h index ca73d3e467..fea5000a4d 100644 --- a/src/qemu/qemu_migration.h +++ b/src/qemu/qemu_migration.h @@ -57,7 +57,9 @@ VIR_MIGRATE_AUTO_CONVERGE | \ VIR_MIGRATE_RDMA_PIN_ALL | \ VIR_MIGRATE_POSTCOPY | \ - VIR_MIGRATE_TLS) + VIR_MIGRATE_TLS | \ + VIR_MIGRATE_PARALLEL | \ + 0) =20 /* All supported migration parameters and their types. */ # define QEMU_MIGRATION_PARAMETERS \ @@ -80,6 +82,7 @@ VIR_MIGRATE_PARAM_AUTO_CONVERGE_INITIAL, VIR_TYPED_PARAM_INT, \ VIR_MIGRATE_PARAM_AUTO_CONVERGE_INCREMENT, VIR_TYPED_PARAM_INT, \ VIR_MIGRATE_PARAM_BANDWIDTH_POSTCOPY, VIR_TYPED_PARAM_ULLONG, \ + VIR_MIGRATE_PARAM_PARALLEL_CONNECTIONS, VIR_TYPED_PARAM_INT, \ NULL =20 =20 diff --git a/src/qemu/qemu_migration_params.c b/src/qemu/qemu_migration_par= ams.c index 28f2c75ea1..82416667a6 100644 --- a/src/qemu/qemu_migration_params.c +++ b/src/qemu/qemu_migration_params.c @@ -86,6 +86,7 @@ VIR_ENUM_IMPL(qemuMigrationCapability, QEMU_MIGRATION_CAP= _LAST, "compress", "pause-before-switchover", "late-block-activate", + "multifd", ); =20 =20 @@ -103,6 +104,7 @@ VIR_ENUM_IMPL(qemuMigrationParam, QEMU_MIGRATION_PARAM_= LAST, "block-incremental", "xbzrle-cache-size", "max-postcopy-bandwidth", + "multifd-channels", ); =20 typedef struct _qemuMigrationParamsAlwaysOnItem qemuMigrationParamsAlwaysO= nItem; @@ -151,6 +153,10 @@ static const qemuMigrationParamsFlagMapItem qemuMigrat= ionParamsFlagMap[] =3D { {VIR_MIGRATE_POSTCOPY, QEMU_MIGRATION_CAP_POSTCOPY, QEMU_MIGRATION_SOURCE | QEMU_MIGRATION_DESTINATION}, + + {VIR_MIGRATE_PARALLEL, + QEMU_MIGRATION_CAP_MULTIFD, + QEMU_MIGRATION_SOURCE | QEMU_MIGRATION_DESTINATION}, }; =20 /* Translation from VIR_MIGRATE_PARAM_* typed parameters to @@ -184,6 +190,10 @@ static const qemuMigrationParamsTPMapItem qemuMigratio= nParamsTPMap[] =3D { .unit =3D 1024 * 1024, /* MiB/s */ .param =3D QEMU_MIGRATION_PARAM_MAX_POSTCOPY_BANDWIDTH, .party =3D QEMU_MIGRATION_SOURCE | QEMU_MIGRATION_DESTINATION}, + + {.typedParam =3D VIR_MIGRATE_PARAM_PARALLEL_CONNECTIONS, + .param =3D QEMU_MIGRATION_PARAM_MULTIFD_CHANNELS, + .party =3D QEMU_MIGRATION_SOURCE | QEMU_MIGRATION_DESTINATION}, }; =20 static const qemuMigrationParamType qemuMigrationParamTypes[] =3D { @@ -199,6 +209,7 @@ static const qemuMigrationParamType qemuMigrationParamT= ypes[] =3D { [QEMU_MIGRATION_PARAM_BLOCK_INCREMENTAL] =3D QEMU_MIGRATION_PARAM_TYPE= _BOOL, [QEMU_MIGRATION_PARAM_XBZRLE_CACHE_SIZE] =3D QEMU_MIGRATION_PARAM_TYPE= _ULL, [QEMU_MIGRATION_PARAM_MAX_POSTCOPY_BANDWIDTH] =3D QEMU_MIGRATION_PARAM= _TYPE_ULL, + [QEMU_MIGRATION_PARAM_MULTIFD_CHANNELS] =3D QEMU_MIGRATION_PARAM_TYPE_= INT, }; verify(ARRAY_CARDINALITY(qemuMigrationParamTypes) =3D=3D QEMU_MIGRATION_PA= RAM_LAST); =20 @@ -535,6 +546,13 @@ qemuMigrationParamsFromFlags(virTypedParameterPtr para= ms, goto error; } =20 + if (migParams->params[QEMU_MIGRATION_PARAM_MULTIFD_CHANNELS].set && + !(flags & VIR_MIGRATE_PARALLEL)) { + virReportError(VIR_ERR_INVALID_ARG, "%s", + _("Turn parallel migration on to tune it")); + goto error; + } + if (qemuMigrationParamsSetCompression(params, nparams, flags, migParam= s) < 0) goto error; =20 diff --git a/src/qemu/qemu_migration_params.h b/src/qemu/qemu_migration_par= ams.h index 233ebe772b..0515339d91 100644 --- a/src/qemu/qemu_migration_params.h +++ b/src/qemu/qemu_migration_params.h @@ -38,6 +38,7 @@ typedef enum { QEMU_MIGRATION_CAP_COMPRESS, QEMU_MIGRATION_CAP_PAUSE_BEFORE_SWITCHOVER, QEMU_MIGRATION_CAP_LATE_BLOCK_ACTIVATE, + QEMU_MIGRATION_CAP_MULTIFD, =20 QEMU_MIGRATION_CAP_LAST } qemuMigrationCapability; @@ -56,6 +57,7 @@ typedef enum { QEMU_MIGRATION_PARAM_BLOCK_INCREMENTAL, QEMU_MIGRATION_PARAM_XBZRLE_CACHE_SIZE, QEMU_MIGRATION_PARAM_MAX_POSTCOPY_BANDWIDTH, + QEMU_MIGRATION_PARAM_MULTIFD_CHANNELS, =20 QEMU_MIGRATION_PARAM_LAST } qemuMigrationParam; --=20 2.20.1 -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list From nobody Tue Apr 16 23:00:06 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) client-ip=209.132.183.28; envelope-from=libvir-list-bounces@redhat.com; helo=mx1.redhat.com; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com Return-Path: Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by mx.zohomail.com with SMTPS id 1549638528921228.6757057976946; Fri, 8 Feb 2019 07:08:48 -0800 (PST) Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 15F4FC062D16; Fri, 8 Feb 2019 15:08:46 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.21]) by smtp.corp.redhat.com (Postfix) with ESMTPS id A8797173D3; Fri, 8 Feb 2019 15:08:45 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id 5A9523F7D5; Fri, 8 Feb 2019 15:08:45 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id x18F8T0S003459 for ; Fri, 8 Feb 2019 10:08:29 -0500 Received: by smtp.corp.redhat.com (Postfix) id CAF235C269; Fri, 8 Feb 2019 15:08:29 +0000 (UTC) Received: from virval.usersys.redhat.com (unknown [10.43.2.20]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 9DE035C6A6 for ; Fri, 8 Feb 2019 15:08:26 +0000 (UTC) Received: by virval.usersys.redhat.com (Postfix, from userid 500) id 08ABC102EBC; Fri, 8 Feb 2019 16:08:25 +0100 (CET) From: Jiri Denemark To: libvir-list@redhat.com Date: Fri, 8 Feb 2019 16:08:23 +0100 Message-Id: In-Reply-To: References: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 X-loop: libvir-list@redhat.com Subject: [libvirt] [PATCH v2 3/3] virsh: Add options for parallel migration X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Transfer-Encoding: quoted-printable Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.32]); Fri, 08 Feb 2019 15:08:47 +0000 (UTC) Content-Type: text/plain; charset="utf-8" Signed-off-by: Jiri Denemark Reviewed-by: John Ferlan --- tools/virsh-domain.c | 19 +++++++++++++++++++ tools/virsh.pod | 7 +++++++ 2 files changed, 26 insertions(+) diff --git a/tools/virsh-domain.c b/tools/virsh-domain.c index 8b20059335..c704faf7e1 100644 --- a/tools/virsh-domain.c +++ b/tools/virsh-domain.c @@ -10561,6 +10561,14 @@ static const vshCmdOptDef opts_migrate[] =3D { .type =3D VSH_OT_INT, .help =3D N_("post-copy migration bandwidth limit in MiB/s") }, + {.name =3D "parallel", + .type =3D VSH_OT_BOOL, + .help =3D N_("enable parallel migration") + }, + {.name =3D "parallel-connections", + .type =3D VSH_OT_INT, + .help =3D N_("number of connections for parallel migration") + }, {.name =3D NULL} }; =20 @@ -10766,6 +10774,14 @@ doMigrate(void *opaque) goto save_error; } =20 + if (vshCommandOptInt(ctl, cmd, "parallel-connections", &intOpt) < 0) + goto out; + if (intOpt && + virTypedParamsAddInt(¶ms, &nparams, &maxparams, + VIR_MIGRATE_PARAM_PARALLEL_CONNECTIONS, + intOpt) < 0) + goto save_error; + if (vshCommandOptBool(cmd, "live")) flags |=3D VIR_MIGRATE_LIVE; if (vshCommandOptBool(cmd, "p2p")) @@ -10814,6 +10830,9 @@ doMigrate(void *opaque) if (vshCommandOptBool(cmd, "tls")) flags |=3D VIR_MIGRATE_TLS; =20 + if (vshCommandOptBool(cmd, "parallel")) + flags |=3D VIR_MIGRATE_PARALLEL; + if (flags & VIR_MIGRATE_PEER2PEER || vshCommandOptBool(cmd, "direct"))= { if (virDomainMigrateToURI3(dom, desturi, params, nparams, flags) = =3D=3D 0) ret =3D '0'; diff --git a/tools/virsh.pod b/tools/virsh.pod index 67edb57b14..7604c7536e 100644 --- a/tools/virsh.pod +++ b/tools/virsh.pod @@ -1904,6 +1904,7 @@ I I [I] [I]= [I] [I] [I<--auto-converge>] [I] [I] [I<--persistent-xml> B] [I<--tls>] [I<--postcopy-bandwidth> B] +[I<--parallel> [I B]] =20 Migrate domain to another host. Add I<--live> for live migration; <--p2p> for peer-2-peer migration; I<--direct> for direct migration; or I<--tunnel= led> @@ -1997,6 +1998,12 @@ Providing I<--tls> causes the migration to use the h= ost configured TLS setup the migration of the domain. Usage requires proper TLS setup for both sour= ce and target. =20 +I<--parallel> option will cause migration data to be sent over multiple +parallel connections. The number of such connections can be set using +I. Parallel connections may help with saturating the +network link between the source and the target and thus speeding up the +migration. + Running migration can be canceled by interrupting virsh (usually using C) or by B command sent from another virsh instance. =20 --=20 2.20.1 -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list