From nobody Mon May 13 12:02:45 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 166904817544352.6637011897476; Mon, 21 Nov 2022 08:29:35 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ox9dc-0000bw-Lz; Mon, 21 Nov 2022 11:27:16 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ox9db-0000bV-4t for qemu-devel@nongnu.org; Mon, 21 Nov 2022 11:27:15 -0500 Received: from prt-mail.chinatelecom.cn ([42.123.76.226] helo=chinatelecom.cn) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ox9dZ-0001st-26 for qemu-devel@nongnu.org; Mon, 21 Nov 2022 11:27:14 -0500 Received: from clientip-125.69.40.159 (unknown [172.18.0.218]) by chinatelecom.cn (HERMES) with SMTP id 1DE2E28008F; Tue, 22 Nov 2022 00:27:01 +0800 (CST) Received: from ([125.69.40.159]) by app0025 with ESMTP id ac7fda2a64f54d5d972721063256527b for qemu-devel@nongnu.org; Tue, 22 Nov 2022 00:27:04 CST HMM_SOURCE_IP: 172.18.0.218:59746.263116816 HMM_ATTACHE_NUM: 0000 HMM_SOURCE_TYPE: SMTP X-189-SAVE-TO-SEND: +huangy81@chinatelecom.cn X-Transaction-ID: ac7fda2a64f54d5d972721063256527b X-Real-From: huangy81@chinatelecom.cn X-Receive-IP: 125.69.40.159 X-MEDUSA-Status: 0 From: huangy81@chinatelecom.cn To: qemu-devel Cc: Peter Xu , Markus Armbruster , "Dr. David Alan Gilbert" , Paolo Bonzini , Laurent Vivier , Eric Blake , Juan Quintela , Thomas Huth , Peter Maydell , Richard Henderson , =?UTF-8?q?Hyman=20Huang=28=E9=BB=84=E5=8B=87=29?= Subject: [PATCH v2 01/11] dirtylimit: Fix overflow when computing MB Date: Mon, 21 Nov 2022 11:26:33 -0500 Message-Id: <73f83b1c55119851e5ac96ebe1d6b6c318e25771.1669047366.git.huangy81@chinatelecom.cn> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: References: In-Reply-To: References: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=42.123.76.226; envelope-from=huangy81@chinatelecom.cn; helo=chinatelecom.cn X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZM-MESSAGEID: 1669048176337100001 From: Hyman Huang(=E9=BB=84=E5=8B=87) overity points out a overflow problem when computing MB, dirty_ring_size and TARGET_PAGE_SIZE are both 32 bits, multiplication will be done as a 32-bit operation, which could overflow. Simplify the formula. Meanwhile, fix spelling mistake of variable name. Reported-by: Peter Maydell Signed-off-by: Peter Maydell Signed-off-by: Richard Henderson Signed-off-by: Hyman Huang(=E9=BB=84=E5=8B=87) Reviewed-by: Peter Xu --- softmmu/dirtylimit.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/softmmu/dirtylimit.c b/softmmu/dirtylimit.c index 1266855..940d238 100644 --- a/softmmu/dirtylimit.c +++ b/softmmu/dirtylimit.c @@ -236,14 +236,14 @@ static inline int64_t dirtylimit_dirty_ring_full_time= (uint64_t dirtyrate) { static uint64_t max_dirtyrate; uint32_t dirty_ring_size =3D kvm_dirty_ring_size(); - uint64_t dirty_ring_size_meory_MB =3D - dirty_ring_size * TARGET_PAGE_SIZE >> 20; + uint32_t dirty_ring_size_memory_MB =3D + dirty_ring_size >> (20 - TARGET_PAGE_BITS); =20 if (max_dirtyrate < dirtyrate) { max_dirtyrate =3D dirtyrate; } =20 - return dirty_ring_size_meory_MB * 1000000 / max_dirtyrate; + return dirty_ring_size_memory_MB * 1000000ULL / max_dirtyrate; } =20 static inline bool dirtylimit_done(uint64_t quota, --=20 1.8.3.1 From nobody Mon May 13 12:02:45 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1669048106911741.4352323871634; Mon, 21 Nov 2022 08:28:26 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ox9dc-0000bp-Ti; Mon, 21 Nov 2022 11:27:16 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ox9db-0000bT-0w for qemu-devel@nongnu.org; Mon, 21 Nov 2022 11:27:15 -0500 Received: from prt-mail.chinatelecom.cn ([42.123.76.226] helo=chinatelecom.cn) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ox9dY-0001sw-Tp for qemu-devel@nongnu.org; Mon, 21 Nov 2022 11:27:14 -0500 Received: from clientip-125.69.40.159 (unknown [172.18.0.218]) by chinatelecom.cn (HERMES) with SMTP id 11BB22800A3; Tue, 22 Nov 2022 00:27:04 +0800 (CST) Received: from ([125.69.40.159]) by app0025 with ESMTP id f1adc6e37b564d6992ecced2f1a6a755 for qemu-devel@nongnu.org; Tue, 22 Nov 2022 00:27:07 CST HMM_SOURCE_IP: 172.18.0.218:59746.263116816 HMM_ATTACHE_NUM: 0000 HMM_SOURCE_TYPE: SMTP X-189-SAVE-TO-SEND: +huangy81@chinatelecom.cn X-Transaction-ID: f1adc6e37b564d6992ecced2f1a6a755 X-Real-From: huangy81@chinatelecom.cn X-Receive-IP: 125.69.40.159 X-MEDUSA-Status: 0 From: huangy81@chinatelecom.cn To: qemu-devel Cc: Peter Xu , Markus Armbruster , "Dr. David Alan Gilbert" , Paolo Bonzini , Laurent Vivier , Eric Blake , Juan Quintela , Thomas Huth , Peter Maydell , Richard Henderson , =?UTF-8?q?Hyman=20Huang=28=E9=BB=84=E5=8B=87=29?= Subject: [PATCH v2 02/11] softmmu/dirtylimit: Add parameter check for hmp "set_vcpu_dirty_limit" Date: Mon, 21 Nov 2022 11:26:34 -0500 Message-Id: <56f207f3f962da7d90772cce1e724d50ba415d79.1669047366.git.huangy81@chinatelecom.cn> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: References: In-Reply-To: References: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=42.123.76.226; envelope-from=huangy81@chinatelecom.cn; helo=chinatelecom.cn X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZM-MESSAGEID: 1669048109702100007 From: Hyman Huang(=E9=BB=84=E5=8B=87) dirty_rate paraemter of hmp command "set_vcpu_dirty_limit" is invalid if less than 0, so add parameter check for it. Signed-off-by: Hyman Huang(=E9=BB=84=E5=8B=87) Reviewed-by: Peter Xu --- softmmu/dirtylimit.c | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/softmmu/dirtylimit.c b/softmmu/dirtylimit.c index 940d238..c42eddd 100644 --- a/softmmu/dirtylimit.c +++ b/softmmu/dirtylimit.c @@ -515,6 +515,11 @@ void hmp_set_vcpu_dirty_limit(Monitor *mon, const QDic= t *qdict) int64_t cpu_index =3D qdict_get_try_int(qdict, "cpu_index", -1); Error *err =3D NULL; =20 + if (dirty_rate < 0) { + monitor_printf(mon, "invalid dirty page limit %ld\n", dirty_rate); + return; + } + qmp_set_vcpu_dirty_limit(!!(cpu_index !=3D -1), cpu_index, dirty_rate,= &err); if (err) { hmp_handle_error(mon, err); --=20 1.8.3.1 From nobody Mon May 13 12:02:45 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 166904819472994.33869610736997; Mon, 21 Nov 2022 08:29:54 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ox9dd-0000cn-Hh; Mon, 21 Nov 2022 11:27:17 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ox9dc-0000bx-6O for qemu-devel@nongnu.org; Mon, 21 Nov 2022 11:27:16 -0500 Received: from prt-mail.chinatelecom.cn ([42.123.76.226] helo=chinatelecom.cn) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ox9dZ-0001tf-Ho for qemu-devel@nongnu.org; Mon, 21 Nov 2022 11:27:15 -0500 Received: from clientip-125.69.40.159 (unknown [172.18.0.218]) by chinatelecom.cn (HERMES) with SMTP id 11EF72800AB; Tue, 22 Nov 2022 00:27:07 +0800 (CST) Received: from ([125.69.40.159]) by app0025 with ESMTP id 665c72253cf24e9495f5832f9780cb89 for qemu-devel@nongnu.org; Tue, 22 Nov 2022 00:27:10 CST HMM_SOURCE_IP: 172.18.0.218:59746.263116816 HMM_ATTACHE_NUM: 0000 HMM_SOURCE_TYPE: SMTP X-189-SAVE-TO-SEND: +huangy81@chinatelecom.cn X-Transaction-ID: 665c72253cf24e9495f5832f9780cb89 X-Real-From: huangy81@chinatelecom.cn X-Receive-IP: 125.69.40.159 X-MEDUSA-Status: 0 From: huangy81@chinatelecom.cn To: qemu-devel Cc: Peter Xu , Markus Armbruster , "Dr. David Alan Gilbert" , Paolo Bonzini , Laurent Vivier , Eric Blake , Juan Quintela , Thomas Huth , Peter Maydell , Richard Henderson , =?UTF-8?q?Hyman=20Huang=28=E9=BB=84=E5=8B=87=29?= Subject: [PATCH v2 03/11] kvm-all: Do not allow reap vcpu dirty ring buffer if not ready Date: Mon, 21 Nov 2022 11:26:35 -0500 Message-Id: X-Mailer: git-send-email 1.8.3.1 In-Reply-To: References: In-Reply-To: References: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=42.123.76.226; envelope-from=huangy81@chinatelecom.cn; helo=chinatelecom.cn X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZM-MESSAGEID: 1669048196473100001 From: Hyman Huang(=E9=BB=84=E5=8B=87) When tested large vcpu size vm with dirtylimit feature, Qemu crashed due to the assertion in kvm_dirty_ring_reap_one, which assert that vcpu's kvm_dirty_gfns has been allocated and not NULL. Because dirty ring reaper thread races with Qemu main thread, reaper may reap vcpu's dirty ring buffer when main thread doesn't complete vcpu instantiation. So add the waiting logic in reaper thread and start to reap until vcpu instantiation is completed. Signed-off-by: Hyman Huang(=E9=BB=84=E5=8B=87) --- accel/kvm/kvm-all.c | 36 ++++++++++++++++++++++++++++++++++++ 1 file changed, 36 insertions(+) diff --git a/accel/kvm/kvm-all.c b/accel/kvm/kvm-all.c index f99b0be..9457715 100644 --- a/accel/kvm/kvm-all.c +++ b/accel/kvm/kvm-all.c @@ -1401,6 +1401,35 @@ out: kvm_slots_unlock(); } =20 +/* + * test if dirty ring has been initialized by checking if vcpu + * has been initialized and gfns was allocated correspondlingly. + * return true if dirty ring has been initialized, false otherwise. + */ +static bool kvm_vcpu_dirty_ring_initialized(void) +{ + CPUState *cpu; + MachineState *ms =3D MACHINE(qdev_get_machine()); + int ncpus =3D ms->smp.cpus; + + /* + * assume vcpu has not been initilaized if generation + * id less than number of vcpu + */ + if (ncpus > cpu_list_generation_id_get()) { + return false; + } + + CPU_FOREACH(cpu) { + if (!cpu->kvm_dirty_gfns) { + return false; + } + } + + return true; +} + + static void *kvm_dirty_ring_reaper_thread(void *data) { KVMState *s =3D data; @@ -1410,6 +1439,13 @@ static void *kvm_dirty_ring_reaper_thread(void *data) =20 trace_kvm_dirty_ring_reaper("init"); =20 +retry: + /* don't allow reaping dirty ring if ring buffer hasn't been mapped */ + if (!kvm_vcpu_dirty_ring_initialized()) { + sleep(1); + goto retry; + } + while (true) { r->reaper_state =3D KVM_DIRTY_RING_REAPER_WAIT; trace_kvm_dirty_ring_reaper("wait"); --=20 1.8.3.1 From nobody Mon May 13 12:02:45 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1669048162694182.71924487825424; Mon, 21 Nov 2022 08:29:22 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ox9dr-0000fl-6n; Mon, 21 Nov 2022 11:27:31 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ox9do-0000fY-Kz for qemu-devel@nongnu.org; Mon, 21 Nov 2022 11:27:28 -0500 Received: from prt-mail.chinatelecom.cn ([42.123.76.226] helo=chinatelecom.cn) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ox9da-0001ts-NZ for qemu-devel@nongnu.org; Mon, 21 Nov 2022 11:27:28 -0500 Received: from clientip-125.69.40.159 (unknown [172.18.0.218]) by chinatelecom.cn (HERMES) with SMTP id AD2672800AF; Tue, 22 Nov 2022 00:27:10 +0800 (CST) Received: from ([125.69.40.159]) by app0025 with ESMTP id 11eb1200f1104633836d904588ffb041 for qemu-devel@nongnu.org; Tue, 22 Nov 2022 00:27:12 CST HMM_SOURCE_IP: 172.18.0.218:59746.263116816 HMM_ATTACHE_NUM: 0000 HMM_SOURCE_TYPE: SMTP X-189-SAVE-TO-SEND: +huangy81@chinatelecom.cn X-Transaction-ID: 11eb1200f1104633836d904588ffb041 X-Real-From: huangy81@chinatelecom.cn X-Receive-IP: 125.69.40.159 X-MEDUSA-Status: 0 From: huangy81@chinatelecom.cn To: qemu-devel Cc: Peter Xu , Markus Armbruster , "Dr. David Alan Gilbert" , Paolo Bonzini , Laurent Vivier , Eric Blake , Juan Quintela , Thomas Huth , Peter Maydell , Richard Henderson , =?UTF-8?q?Hyman=20Huang=28=E9=BB=84=E5=8B=87=29?= Subject: [PATCH v2 04/11] qapi/migration: Introduce x-vcpu-dirty-limit-period parameter Date: Mon, 21 Nov 2022 11:26:36 -0500 Message-Id: <9e6d5e0ab54c82c8b42a3b318249320fb20e3a44.1669047366.git.huangy81@chinatelecom.cn> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: References: In-Reply-To: References: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=42.123.76.226; envelope-from=huangy81@chinatelecom.cn; helo=chinatelecom.cn X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZM-MESSAGEID: 1669048164409100001 From: Hyman Huang(=E9=BB=84=E5=8B=87) Introduce "x-vcpu-dirty-limit-period" migration experimental parameter, which is in the range of 1 to 1000ms and used to make dirtyrate calculation period configurable. Signed-off-by: Hyman Huang(=E9=BB=84=E5=8B=87) --- migration/migration.c | 26 ++++++++++++++++++++++++++ monitor/hmp-cmds.c | 8 ++++++++ qapi/migration.json | 34 +++++++++++++++++++++++++++------- 3 files changed, 61 insertions(+), 7 deletions(-) diff --git a/migration/migration.c b/migration/migration.c index 739bb68..701267c 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -116,6 +116,8 @@ #define DEFAULT_MIGRATE_ANNOUNCE_ROUNDS 5 #define DEFAULT_MIGRATE_ANNOUNCE_STEP 100 =20 +#define DEFAULT_MIGRATE_VCPU_DIRTY_LIMIT_PERIOD 500 /* ms */ + static NotifierList migration_state_notifiers =3D NOTIFIER_LIST_INITIALIZER(migration_state_notifiers); =20 @@ -963,6 +965,9 @@ MigrationParameters *qmp_query_migrate_parameters(Error= **errp) s->parameters.block_bitmap_mapping); } =20 + params->has_x_vcpu_dirty_limit_period =3D true; + params->x_vcpu_dirty_limit_period =3D s->parameters.x_vcpu_dirty_limit= _period; + return params; } =20 @@ -1564,6 +1569,15 @@ static bool migrate_params_check(MigrationParameters= *params, Error **errp) } #endif =20 + if (params->has_x_vcpu_dirty_limit_period && + (params->x_vcpu_dirty_limit_period < 1 || + params->x_vcpu_dirty_limit_period > 1000)) { + error_setg(errp, QERR_INVALID_PARAMETER_VALUE, + "x_vcpu_dirty_limit_period", + "is invalid, it must be in the range of 1 to 1000 ms"); + return false; + } + return true; } =20 @@ -1663,6 +1677,10 @@ static void migrate_params_test_apply(MigrateSetPara= meters *params, dest->has_block_bitmap_mapping =3D true; dest->block_bitmap_mapping =3D params->block_bitmap_mapping; } + + if (params->has_x_vcpu_dirty_limit_period) { + dest->x_vcpu_dirty_limit_period =3D params->x_vcpu_dirty_limit_per= iod; + } } =20 static void migrate_params_apply(MigrateSetParameters *params, Error **err= p) @@ -1785,6 +1803,10 @@ static void migrate_params_apply(MigrateSetParameter= s *params, Error **errp) QAPI_CLONE(BitmapMigrationNodeAliasList, params->block_bitmap_mapping); } + if (params->has_x_vcpu_dirty_limit_period) { + s->parameters.x_vcpu_dirty_limit_period =3D + params->x_vcpu_dirty_limit_period; + } } =20 void qmp_migrate_set_parameters(MigrateSetParameters *params, Error **errp) @@ -4386,6 +4408,9 @@ static Property migration_properties[] =3D { DEFINE_PROP_STRING("tls-creds", MigrationState, parameters.tls_creds), DEFINE_PROP_STRING("tls-hostname", MigrationState, parameters.tls_host= name), DEFINE_PROP_STRING("tls-authz", MigrationState, parameters.tls_authz), + DEFINE_PROP_UINT64("x-vcpu-dirty-limit-period", MigrationState, + parameters.x_vcpu_dirty_limit_period, + DEFAULT_MIGRATE_VCPU_DIRTY_LIMIT_PERIOD), =20 /* Migration capabilities */ DEFINE_PROP_MIG_CAP("x-xbzrle", MIGRATION_CAPABILITY_XBZRLE), @@ -4477,6 +4502,7 @@ static void migration_instance_init(Object *obj) params->has_tls_creds =3D true; params->has_tls_hostname =3D true; params->has_tls_authz =3D true; + params->has_x_vcpu_dirty_limit_period =3D true; =20 qemu_sem_init(&ms->postcopy_pause_sem, 0); qemu_sem_init(&ms->postcopy_pause_rp_sem, 0); diff --git a/monitor/hmp-cmds.c b/monitor/hmp-cmds.c index 01b789a..a3170ca 100644 --- a/monitor/hmp-cmds.c +++ b/monitor/hmp-cmds.c @@ -513,6 +513,10 @@ void hmp_info_migrate_parameters(Monitor *mon, const Q= Dict *qdict) } } } + + monitor_printf(mon, "%s: %" PRIu64 " ms\n", + MigrationParameter_str(MIGRATION_PARAMETER_X_VCPU_DIRTY_LIMIT_PERI= OD), + params->x_vcpu_dirty_limit_period); } =20 qapi_free_MigrationParameters(params); @@ -1332,6 +1336,10 @@ void hmp_migrate_set_parameter(Monitor *mon, const Q= Dict *qdict) error_setg(&err, "The block-bitmap-mapping parameter can only be s= et " "through QMP"); break; + case MIGRATION_PARAMETER_X_VCPU_DIRTY_LIMIT_PERIOD: + p->has_x_vcpu_dirty_limit_period =3D true; + visit_type_size(v, param, &p->x_vcpu_dirty_limit_period, &err); + break; default: assert(0); } diff --git a/qapi/migration.json b/qapi/migration.json index 88ecf86..5175779 100644 --- a/qapi/migration.json +++ b/qapi/migration.json @@ -776,8 +776,13 @@ # block device name if there is one, and to their n= ode name # otherwise. (Since 5.2) # +# @x-vcpu-dirty-limit-period: Periodic time (ms) of dirty limit during liv= e migration. +# Should be in the range 1 to 1000ms, defaults= to 500ms. +# (Since 7.1) +# # Features: -# @unstable: Member @x-checkpoint-delay is experimental. +# @unstable: Member @x-checkpoint-delay and @x-vcpu-dirty-limit-period +# are experimental. # # Since: 2.4 ## @@ -795,8 +800,9 @@ 'multifd-channels', 'xbzrle-cache-size', 'max-postcopy-bandwidth', 'max-cpu-throttle', 'multifd-compression', - 'multifd-zlib-level' ,'multifd-zstd-level', - 'block-bitmap-mapping' ] } + 'multifd-zlib-level', 'multifd-zstd-level', + 'block-bitmap-mapping', + { 'name': 'x-vcpu-dirty-limit-period', 'features': ['unstable']= } ] } =20 ## # @MigrateSetParameters: @@ -941,8 +947,13 @@ # block device name if there is one, and to their n= ode name # otherwise. (Since 5.2) # +# @x-vcpu-dirty-limit-period: Periodic time (ms) of dirty limit during liv= e migration. +# Should be in the range 1 to 1000ms, defaults= to 500ms. +# (Since 7.1) +# # Features: -# @unstable: Member @x-checkpoint-delay is experimental. +# @unstable: Member @x-checkpoint-delay and @x-vcpu-dirty-limit-period +# are experimental. # # Since: 2.4 ## @@ -976,7 +987,9 @@ '*multifd-compression': 'MultiFDCompression', '*multifd-zlib-level': 'uint8', '*multifd-zstd-level': 'uint8', - '*block-bitmap-mapping': [ 'BitmapMigrationNodeAlias' ] } } + '*block-bitmap-mapping': [ 'BitmapMigrationNodeAlias' ], + '*x-vcpu-dirty-limit-period': { 'type': 'uint64', + 'features': [ 'unstable' ] } }= } =20 ## # @migrate-set-parameters: @@ -1141,8 +1154,13 @@ # block device name if there is one, and to their n= ode name # otherwise. (Since 5.2) # +# @x-vcpu-dirty-limit-period: Periodic time (ms) of dirty limit during liv= e migration. +# Should be in the range 1 to 1000ms, defaults= to 500ms. +# (Since 7.1) +# # Features: -# @unstable: Member @x-checkpoint-delay is experimental. +# @unstable: Member @x-checkpoint-delay and @x-vcpu-dirty-limit-period +# are experimental. # # Since: 2.4 ## @@ -1174,7 +1192,9 @@ '*multifd-compression': 'MultiFDCompression', '*multifd-zlib-level': 'uint8', '*multifd-zstd-level': 'uint8', - '*block-bitmap-mapping': [ 'BitmapMigrationNodeAlias' ] } } + '*block-bitmap-mapping': [ 'BitmapMigrationNodeAlias' ], + '*x-vcpu-dirty-limit-period': { 'type': 'uint64', + 'features': [ 'unstable' ] } }= } =20 ## # @query-migrate-parameters: --=20 1.8.3.1 From nobody Mon May 13 12:02:45 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1669048126873333.0586130929918; Mon, 21 Nov 2022 08:28:46 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ox9dj-0000dl-4q; Mon, 21 Nov 2022 11:27:23 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ox9dh-0000dS-MC for qemu-devel@nongnu.org; Mon, 21 Nov 2022 11:27:21 -0500 Received: from prt-mail.chinatelecom.cn ([42.123.76.226] helo=chinatelecom.cn) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ox9dd-0001uI-AC for qemu-devel@nongnu.org; Mon, 21 Nov 2022 11:27:19 -0500 Received: from clientip-125.69.40.159 (unknown [172.18.0.218]) by chinatelecom.cn (HERMES) with SMTP id 545652800C1; Tue, 22 Nov 2022 00:27:12 +0800 (CST) Received: from ([125.69.40.159]) by app0025 with ESMTP id fa910eb28b474418b8a989063fac653b for qemu-devel@nongnu.org; Tue, 22 Nov 2022 00:27:15 CST HMM_SOURCE_IP: 172.18.0.218:59746.263116816 HMM_ATTACHE_NUM: 0000 HMM_SOURCE_TYPE: SMTP X-189-SAVE-TO-SEND: +huangy81@chinatelecom.cn X-Transaction-ID: fa910eb28b474418b8a989063fac653b X-Real-From: huangy81@chinatelecom.cn X-Receive-IP: 125.69.40.159 X-MEDUSA-Status: 0 From: huangy81@chinatelecom.cn To: qemu-devel Cc: Peter Xu , Markus Armbruster , "Dr. David Alan Gilbert" , Paolo Bonzini , Laurent Vivier , Eric Blake , Juan Quintela , Thomas Huth , Peter Maydell , Richard Henderson , =?UTF-8?q?Hyman=20Huang=28=E9=BB=84=E5=8B=87=29?= Subject: [PATCH v2 05/11] qapi/migration: Introduce vcpu-dirty-limit parameters Date: Mon, 21 Nov 2022 11:26:37 -0500 Message-Id: <3fe7b2324c350b65f2a6fe6e4c585e9de5402e0c.1669047366.git.huangy81@chinatelecom.cn> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: References: In-Reply-To: References: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=42.123.76.226; envelope-from=huangy81@chinatelecom.cn; helo=chinatelecom.cn X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZM-MESSAGEID: 1669048128051100001 From: Hyman Huang(=E9=BB=84=E5=8B=87) Introduce "vcpu-dirty-limit" migration parameter used to limit dirty page rate during live migration. "vcpu-dirty-limit" and "x-vcpu-dirty-limit-period" are two dirty-limit-related migration parameters, which can be set before and during live migration by qmp migrate-set-parameters. This two parameters are used to help implement the dirty page rate limit algo of migration. Signed-off-by: Hyman Huang(=E9=BB=84=E5=8B=87) Acked-by: Peter Xu --- migration/migration.c | 23 +++++++++++++++++++++++ monitor/hmp-cmds.c | 8 ++++++++ qapi/migration.json | 18 +++++++++++++++--- 3 files changed, 46 insertions(+), 3 deletions(-) diff --git a/migration/migration.c b/migration/migration.c index 701267c..e2aada2 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -117,6 +117,7 @@ #define DEFAULT_MIGRATE_ANNOUNCE_STEP 100 =20 #define DEFAULT_MIGRATE_VCPU_DIRTY_LIMIT_PERIOD 500 /* ms */ +#define DEFAULT_MIGRATE_VCPU_DIRTY_LIMIT 1 /* MB/s */ =20 static NotifierList migration_state_notifiers =3D NOTIFIER_LIST_INITIALIZER(migration_state_notifiers); @@ -968,6 +969,9 @@ MigrationParameters *qmp_query_migrate_parameters(Error= **errp) params->has_x_vcpu_dirty_limit_period =3D true; params->x_vcpu_dirty_limit_period =3D s->parameters.x_vcpu_dirty_limit= _period; =20 + params->has_vcpu_dirty_limit =3D true; + params->vcpu_dirty_limit =3D s->parameters.vcpu_dirty_limit; + return params; } =20 @@ -1578,6 +1582,14 @@ static bool migrate_params_check(MigrationParameters= *params, Error **errp) return false; } =20 + if (params->has_vcpu_dirty_limit && + (params->vcpu_dirty_limit < 1)) { + error_setg(errp, QERR_INVALID_PARAMETER_VALUE, + "vcpu_dirty_limit", + "is invalid, it must greater then 1 MB/s"); + return false; + } + return true; } =20 @@ -1681,6 +1693,10 @@ static void migrate_params_test_apply(MigrateSetPara= meters *params, if (params->has_x_vcpu_dirty_limit_period) { dest->x_vcpu_dirty_limit_period =3D params->x_vcpu_dirty_limit_per= iod; } + + if (params->has_vcpu_dirty_limit) { + dest->vcpu_dirty_limit =3D params->vcpu_dirty_limit; + } } =20 static void migrate_params_apply(MigrateSetParameters *params, Error **err= p) @@ -1807,6 +1823,9 @@ static void migrate_params_apply(MigrateSetParameters= *params, Error **errp) s->parameters.x_vcpu_dirty_limit_period =3D params->x_vcpu_dirty_limit_period; } + if (params->has_vcpu_dirty_limit) { + s->parameters.vcpu_dirty_limit =3D params->vcpu_dirty_limit; + } } =20 void qmp_migrate_set_parameters(MigrateSetParameters *params, Error **errp) @@ -4411,6 +4430,9 @@ static Property migration_properties[] =3D { DEFINE_PROP_UINT64("x-vcpu-dirty-limit-period", MigrationState, parameters.x_vcpu_dirty_limit_period, DEFAULT_MIGRATE_VCPU_DIRTY_LIMIT_PERIOD), + DEFINE_PROP_UINT64("vcpu-dirty-limit", MigrationState, + parameters.vcpu_dirty_limit, + DEFAULT_MIGRATE_VCPU_DIRTY_LIMIT), =20 /* Migration capabilities */ DEFINE_PROP_MIG_CAP("x-xbzrle", MIGRATION_CAPABILITY_XBZRLE), @@ -4503,6 +4525,7 @@ static void migration_instance_init(Object *obj) params->has_tls_hostname =3D true; params->has_tls_authz =3D true; params->has_x_vcpu_dirty_limit_period =3D true; + params->has_vcpu_dirty_limit =3D true; =20 qemu_sem_init(&ms->postcopy_pause_sem, 0); qemu_sem_init(&ms->postcopy_pause_rp_sem, 0); diff --git a/monitor/hmp-cmds.c b/monitor/hmp-cmds.c index a3170ca..9ad6ee5 100644 --- a/monitor/hmp-cmds.c +++ b/monitor/hmp-cmds.c @@ -517,6 +517,10 @@ void hmp_info_migrate_parameters(Monitor *mon, const Q= Dict *qdict) monitor_printf(mon, "%s: %" PRIu64 " ms\n", MigrationParameter_str(MIGRATION_PARAMETER_X_VCPU_DIRTY_LIMIT_PERI= OD), params->x_vcpu_dirty_limit_period); + + monitor_printf(mon, "%s: %" PRIu64 " MB/s\n", + MigrationParameter_str(MIGRATION_PARAMETER_VCPU_DIRTY_LIMIT), + params->vcpu_dirty_limit); } =20 qapi_free_MigrationParameters(params); @@ -1340,6 +1344,10 @@ void hmp_migrate_set_parameter(Monitor *mon, const Q= Dict *qdict) p->has_x_vcpu_dirty_limit_period =3D true; visit_type_size(v, param, &p->x_vcpu_dirty_limit_period, &err); break; + case MIGRATION_PARAMETER_VCPU_DIRTY_LIMIT: + p->has_vcpu_dirty_limit =3D true; + visit_type_size(v, param, &p->vcpu_dirty_limit, &err); + break; default: assert(0); } diff --git a/qapi/migration.json b/qapi/migration.json index 5175779..dd667dd 100644 --- a/qapi/migration.json +++ b/qapi/migration.json @@ -780,6 +780,9 @@ # Should be in the range 1 to 1000ms, defaults= to 500ms. # (Since 7.1) # +# @vcpu-dirty-limit: Dirtyrate limit (MB/s) during live migration. +# Defaults to 1. (Since 7.1) +# # Features: # @unstable: Member @x-checkpoint-delay and @x-vcpu-dirty-limit-period # are experimental. @@ -802,7 +805,8 @@ 'max-cpu-throttle', 'multifd-compression', 'multifd-zlib-level', 'multifd-zstd-level', 'block-bitmap-mapping', - { 'name': 'x-vcpu-dirty-limit-period', 'features': ['unstable']= } ] } + { 'name': 'x-vcpu-dirty-limit-period', 'features': ['unstable']= }, + 'vcpu-dirty-limit'] } =20 ## # @MigrateSetParameters: @@ -951,6 +955,9 @@ # Should be in the range 1 to 1000ms, defaults= to 500ms. # (Since 7.1) # +# @vcpu-dirty-limit: Dirtyrate limit (MB/s) during live migration. +# Defaults to 1. (Since 7.1) +# # Features: # @unstable: Member @x-checkpoint-delay and @x-vcpu-dirty-limit-period # are experimental. @@ -989,7 +996,8 @@ '*multifd-zstd-level': 'uint8', '*block-bitmap-mapping': [ 'BitmapMigrationNodeAlias' ], '*x-vcpu-dirty-limit-period': { 'type': 'uint64', - 'features': [ 'unstable' ] } }= } + 'features': [ 'unstable' ] }, + '*vcpu-dirty-limit': 'uint64'} } =20 ## # @migrate-set-parameters: @@ -1158,6 +1166,9 @@ # Should be in the range 1 to 1000ms, defaults= to 500ms. # (Since 7.1) # +# @vcpu-dirty-limit: Dirtyrate limit (MB/s) during live migration. +# Defaults to 1. (Since 7.1) +# # Features: # @unstable: Member @x-checkpoint-delay and @x-vcpu-dirty-limit-period # are experimental. @@ -1194,7 +1205,8 @@ '*multifd-zstd-level': 'uint8', '*block-bitmap-mapping': [ 'BitmapMigrationNodeAlias' ], '*x-vcpu-dirty-limit-period': { 'type': 'uint64', - 'features': [ 'unstable' ] } }= } + 'features': [ 'unstable' ] }, + '*vcpu-dirty-limit': 'uint64'} } =20 ## # @query-migrate-parameters: --=20 1.8.3.1 From nobody Mon May 13 12:02:45 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1669048121640629.8985321555851; Mon, 21 Nov 2022 08:28:41 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ox9dk-0000e0-QW; Mon, 21 Nov 2022 11:27:24 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ox9dj-0000do-IA for qemu-devel@nongnu.org; Mon, 21 Nov 2022 11:27:23 -0500 Received: from prt-mail.chinatelecom.cn ([42.123.76.226] helo=chinatelecom.cn) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ox9dh-0001ub-FV for qemu-devel@nongnu.org; Mon, 21 Nov 2022 11:27:23 -0500 Received: from clientip-125.69.40.159 (unknown [172.18.0.218]) by chinatelecom.cn (HERMES) with SMTP id 06AB62800C9; Tue, 22 Nov 2022 00:27:15 +0800 (CST) Received: from ([125.69.40.159]) by app0025 with ESMTP id 2049753d87ac4a87910f1fb92b20b649 for qemu-devel@nongnu.org; Tue, 22 Nov 2022 00:27:17 CST HMM_SOURCE_IP: 172.18.0.218:59746.263116816 HMM_ATTACHE_NUM: 0000 HMM_SOURCE_TYPE: SMTP X-189-SAVE-TO-SEND: +huangy81@chinatelecom.cn X-Transaction-ID: 2049753d87ac4a87910f1fb92b20b649 X-Real-From: huangy81@chinatelecom.cn X-Receive-IP: 125.69.40.159 X-MEDUSA-Status: 0 From: huangy81@chinatelecom.cn To: qemu-devel Cc: Peter Xu , Markus Armbruster , "Dr. David Alan Gilbert" , Paolo Bonzini , Laurent Vivier , Eric Blake , Juan Quintela , Thomas Huth , Peter Maydell , Richard Henderson , =?UTF-8?q?Hyman=20Huang=28=E9=BB=84=E5=8B=87=29?= Subject: [PATCH v2 06/11] migration: Introduce dirty-limit capability Date: Mon, 21 Nov 2022 11:26:38 -0500 Message-Id: X-Mailer: git-send-email 1.8.3.1 In-Reply-To: References: In-Reply-To: References: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=42.123.76.226; envelope-from=huangy81@chinatelecom.cn; helo=chinatelecom.cn X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZM-MESSAGEID: 1669048122123100001 From: Hyman Huang(=E9=BB=84=E5=8B=87) Introduce migration dirty-limit capability, which can be turned on before live migration and limit dirty page rate durty live migration. Introduce migrate_dirty_limit function to help check if dirty-limit capability enabled during live migration. Meanwhile, refactor vcpu_dirty_rate_stat_collect so that period can be configured instead of hardcoded. dirty-limit capability is kind of like auto-converge but using dirty limit instead of traditional cpu-throttle to throttle guest down. To enable this feature, turn on the dirty-limit capability before live migration using migratioin-set-capabilities, and set the parameters "x-vcpu-dirty-limit-period", "vcpu-dirty-limit" suitably to speed up convergence. Signed-off-by: Hyman Huang(=E9=BB=84=E5=8B=87) Acked-by: Peter Xu --- migration/migration.c | 25 +++++++++++++++++++++++++ migration/migration.h | 1 + qapi/migration.json | 4 +++- softmmu/dirtylimit.c | 11 ++++++++++- 4 files changed, 39 insertions(+), 2 deletions(-) diff --git a/migration/migration.c b/migration/migration.c index e2aada2..86950a1 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -61,6 +61,7 @@ #include "sysemu/cpus.h" #include "yank_functions.h" #include "sysemu/qtest.h" +#include "sysemu/kvm.h" =20 #define MAX_THROTTLE (128 << 20) /* Migration transfer speed throttl= ing */ =20 @@ -1348,6 +1349,20 @@ static bool migrate_caps_check(bool *cap_list, } } =20 + if (cap_list[MIGRATION_CAPABILITY_DIRTY_LIMIT]) { + if (cap_list[MIGRATION_CAPABILITY_AUTO_CONVERGE]) { + error_setg(errp, "dirty-limit conflicts with auto-converge" + " either of then available currently"); + return false; + } + + if (!kvm_enabled() || !kvm_dirty_ring_enabled()) { + error_setg(errp, "dirty-limit requires KVM with accelerator" + " property 'dirty-ring-size' set"); + return false; + } + } + return true; } =20 @@ -2526,6 +2541,15 @@ bool migrate_auto_converge(void) return s->enabled_capabilities[MIGRATION_CAPABILITY_AUTO_CONVERGE]; } =20 +bool migrate_dirty_limit(void) +{ + MigrationState *s; + + s =3D migrate_get_current(); + + return s->enabled_capabilities[MIGRATION_CAPABILITY_DIRTY_LIMIT]; +} + bool migrate_zero_blocks(void) { MigrationState *s; @@ -4455,6 +4479,7 @@ static Property migration_properties[] =3D { DEFINE_PROP_MIG_CAP("x-zero-copy-send", MIGRATION_CAPABILITY_ZERO_COPY_SEND), #endif + DEFINE_PROP_MIG_CAP("x-dirty-limit", MIGRATION_CAPABILITY_DIRTY_LIMIT), =20 DEFINE_PROP_END_OF_LIST(), }; diff --git a/migration/migration.h b/migration/migration.h index cdad8ac..7fbb9f8 100644 --- a/migration/migration.h +++ b/migration/migration.h @@ -409,6 +409,7 @@ bool migrate_ignore_shared(void); bool migrate_validate_uuid(void); =20 bool migrate_auto_converge(void); +bool migrate_dirty_limit(void); bool migrate_use_multifd(void); bool migrate_pause_before_switchover(void); int migrate_multifd_channels(void); diff --git a/qapi/migration.json b/qapi/migration.json index dd667dd..af6b2da 100644 --- a/qapi/migration.json +++ b/qapi/migration.json @@ -477,6 +477,8 @@ # will be handled faster. This is a performance featur= e and # should not affect the correctness of postcopy migrati= on. # (since 7.1) +# @dirty-limit: Use dirty-limit to throttle down guest if enabled. +# (since 7.1) # # Features: # @unstable: Members @x-colo and @x-ignore-shared are experimental. @@ -492,7 +494,7 @@ 'dirty-bitmaps', 'postcopy-blocktime', 'late-block-activate', { 'name': 'x-ignore-shared', 'features': [ 'unstable' ] }, 'validate-uuid', 'background-snapshot', - 'zero-copy-send', 'postcopy-preempt'] } + 'zero-copy-send', 'postcopy-preempt', 'dirty-limit'] } =20 ## # @MigrationCapabilityStatus: diff --git a/softmmu/dirtylimit.c b/softmmu/dirtylimit.c index c42eddd..4537c51 100644 --- a/softmmu/dirtylimit.c +++ b/softmmu/dirtylimit.c @@ -23,6 +23,8 @@ #include "exec/memory.h" #include "hw/boards.h" #include "sysemu/kvm.h" +#include "migration/misc.h" +#include "migration/migration.h" #include "trace.h" =20 /* @@ -75,11 +77,18 @@ static bool dirtylimit_quit; =20 static void vcpu_dirty_rate_stat_collect(void) { + MigrationState *s =3D migrate_get_current(); VcpuStat stat; int i =3D 0; + int64_t period =3D DIRTYLIMIT_CALC_TIME_MS; + + if (migrate_dirty_limit() && + migration_is_active(s)) { + period =3D s->parameters.x_vcpu_dirty_limit_period; + } =20 /* calculate vcpu dirtyrate */ - vcpu_calculate_dirtyrate(DIRTYLIMIT_CALC_TIME_MS, + vcpu_calculate_dirtyrate(period, &stat, GLOBAL_DIRTY_LIMIT, false); --=20 1.8.3.1 From nobody Mon May 13 12:02:45 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1669048214491959.4108789600635; Mon, 21 Nov 2022 08:30:14 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ox9dn-0000f3-Ol; Mon, 21 Nov 2022 11:27:27 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ox9dl-0000eW-SU for qemu-devel@nongnu.org; Mon, 21 Nov 2022 11:27:25 -0500 Received: from prt-mail.chinatelecom.cn ([42.123.76.226] helo=chinatelecom.cn) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ox9dj-0001uI-1h for qemu-devel@nongnu.org; Mon, 21 Nov 2022 11:27:25 -0500 Received: from clientip-125.69.40.159 (unknown [172.18.0.218]) by chinatelecom.cn (HERMES) with SMTP id A60052800CA; Tue, 22 Nov 2022 00:27:18 +0800 (CST) Received: from ([125.69.40.159]) by app0025 with ESMTP id cff7564a3b5e484ba6b4d280f53ca994 for qemu-devel@nongnu.org; Tue, 22 Nov 2022 00:27:20 CST HMM_SOURCE_IP: 172.18.0.218:59746.263116816 HMM_ATTACHE_NUM: 0000 HMM_SOURCE_TYPE: SMTP X-189-SAVE-TO-SEND: +huangy81@chinatelecom.cn X-Transaction-ID: cff7564a3b5e484ba6b4d280f53ca994 X-Real-From: huangy81@chinatelecom.cn X-Receive-IP: 125.69.40.159 X-MEDUSA-Status: 0 From: huangy81@chinatelecom.cn To: qemu-devel Cc: Peter Xu , Markus Armbruster , "Dr. David Alan Gilbert" , Paolo Bonzini , Laurent Vivier , Eric Blake , Juan Quintela , Thomas Huth , Peter Maydell , Richard Henderson , =?UTF-8?q?Hyman=20Huang=28=E9=BB=84=E5=8B=87=29?= Subject: [PATCH v2 07/11] migration: Implement dirty-limit convergence algo Date: Mon, 21 Nov 2022 11:26:39 -0500 Message-Id: <5727cd0876e1a8d3db451feb46afb3281875cca3.1669047366.git.huangy81@chinatelecom.cn> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: References: In-Reply-To: References: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=42.123.76.226; envelope-from=huangy81@chinatelecom.cn; helo=chinatelecom.cn X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZM-MESSAGEID: 1669048216727100001 From: Hyman Huang(=E9=BB=84=E5=8B=87) Implement dirty-limit convergence algo for live migration, which is kind of like auto-converge algo but using dirty-limit instead of cpu throttle to make migration convergent. Enable dirty page limit if dirty_rate_high_cnt greater than 2 when dirty-limit capability enabled, Disable dirty-limit if migration be cancled. Note that "set_vcpu_dirty_limit", "cancel_vcpu_dirty_limit" commands are not allowed during dirty-limit live migration. Signed-off-by: Hyman Huang(=E9=BB=84=E5=8B=87) --- migration/migration.c | 1 + migration/ram.c | 62 +++++++++++++++++++++++++++++++++++++++-------= ---- migration/trace-events | 1 + softmmu/dirtylimit.c | 22 ++++++++++++++++++ 4 files changed, 73 insertions(+), 13 deletions(-) diff --git a/migration/migration.c b/migration/migration.c index 86950a1..096b61a 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -240,6 +240,7 @@ void migration_cancel(const Error *error) if (error) { migrate_set_error(current_migration, error); } + qmp_cancel_vcpu_dirty_limit(false, -1, NULL); migrate_fd_cancel(current_migration); } =20 diff --git a/migration/ram.c b/migration/ram.c index dc1de9d..94516b7 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -45,6 +45,7 @@ #include "qapi/error.h" #include "qapi/qapi-types-migration.h" #include "qapi/qapi-events-migration.h" +#include "qapi/qapi-commands-migration.h" #include "qapi/qmp/qerror.h" #include "trace.h" #include "exec/ram_addr.h" @@ -57,6 +58,8 @@ #include "qemu/iov.h" #include "multifd.h" #include "sysemu/runstate.h" +#include "sysemu/dirtylimit.h" +#include "sysemu/kvm.h" =20 #include "hw/boards.h" /* for machine_dump_guest_core() */ =20 @@ -1139,6 +1142,30 @@ static void migration_update_rates(RAMState *rs, int= 64_t end_time) } } =20 +/* + * Enable dirty-limit to throttle down the guest + */ +static void migration_dirty_limit_guest(void) +{ + static int64_t quota_dirtyrate; + MigrationState *s =3D migrate_get_current(); + + /* + * If dirty limit already enabled and migration parameter + * vcpu-dirty-limit untouched. + */ + if (dirtylimit_in_service() && + quota_dirtyrate =3D=3D s->parameters.vcpu_dirty_limit) { + return; + } + + quota_dirtyrate =3D s->parameters.vcpu_dirty_limit; + + /* Set or update quota dirty limit */ + qmp_set_vcpu_dirty_limit(false, -1, quota_dirtyrate, NULL); + trace_migration_dirty_limit_guest(quota_dirtyrate); +} + static void migration_trigger_throttle(RAMState *rs) { MigrationState *s =3D migrate_get_current(); @@ -1148,22 +1175,31 @@ static void migration_trigger_throttle(RAMState *rs) uint64_t bytes_dirty_period =3D rs->num_dirty_pages_period * TARGET_PA= GE_SIZE; uint64_t bytes_dirty_threshold =3D bytes_xfer_period * threshold / 100; =20 - /* During block migration the auto-converge logic incorrectly detects - * that ram migration makes no progress. Avoid this by disabling the - * throttling logic during the bulk phase of block migration. */ - if (migrate_auto_converge() && !blk_mig_bulk_active()) { - /* The following detection logic can be refined later. For now: - Check to see if the ratio between dirtied bytes and the approx. - amount of bytes that just got transferred since the last time - we were in this routine reaches the threshold. If that happens - twice, start or increase throttling. */ - - if ((bytes_dirty_period > bytes_dirty_threshold) && - (++rs->dirty_rate_high_cnt >=3D 2)) { + /* + * The following detection logic can be refined later. For now: + * Check to see if the ratio between dirtied bytes and the approx. + * amount of bytes that just got transferred since the last time + * we were in this routine reaches the threshold. If that happens + * twice, start or increase throttling. + */ + + if ((bytes_dirty_period > bytes_dirty_threshold) && + (++rs->dirty_rate_high_cnt >=3D 2)) { + rs->dirty_rate_high_cnt =3D 0; + /* + * During block migration the auto-converge logic incorrectly dete= cts + * that ram migration makes no progress. Avoid this by disabling t= he + * throttling logic during the bulk phase of block migration + */ + + if (migrate_auto_converge() && !blk_mig_bulk_active()) { trace_migration_throttle(); - rs->dirty_rate_high_cnt =3D 0; mig_throttle_guest_down(bytes_dirty_period, bytes_dirty_threshold); + } else if (migrate_dirty_limit() && + kvm_dirty_ring_enabled() && + migration_is_active(s)) { + migration_dirty_limit_guest(); } } } diff --git a/migration/trace-events b/migration/trace-events index 57003ed..33a2666 100644 --- a/migration/trace-events +++ b/migration/trace-events @@ -91,6 +91,7 @@ migration_bitmap_sync_start(void) "" migration_bitmap_sync_end(uint64_t dirty_pages) "dirty_pages %" PRIu64 migration_bitmap_clear_dirty(char *str, uint64_t start, uint64_t size, uns= igned long page) "rb %s start 0x%"PRIx64" size 0x%"PRIx64" page 0x%lx" migration_throttle(void) "" +migration_dirty_limit_guest(int64_t dirtyrate) "guest dirty page rate limi= t %" PRIi64 " MB/s" ram_discard_range(const char *rbname, uint64_t start, size_t len) "%s: sta= rt: %" PRIx64 " %zx" ram_load_loop(const char *rbname, uint64_t addr, int flags, void *host) "%= s: addr: 0x%" PRIx64 " flags: 0x%x host: %p" ram_load_postcopy_loop(int channel, uint64_t addr, int flags) "chan=3D%d a= ddr=3D0x%" PRIx64 " flags=3D0x%x" diff --git a/softmmu/dirtylimit.c b/softmmu/dirtylimit.c index 4537c51..3f3c405 100644 --- a/softmmu/dirtylimit.c +++ b/softmmu/dirtylimit.c @@ -439,6 +439,8 @@ void qmp_cancel_vcpu_dirty_limit(bool has_cpu_index, int64_t cpu_index, Error **errp) { + MigrationState *ms =3D migrate_get_current(); + if (!kvm_enabled() || !kvm_dirty_ring_enabled()) { return; } @@ -452,6 +454,15 @@ void qmp_cancel_vcpu_dirty_limit(bool has_cpu_index, return; } =20 + if (migration_is_running(ms->state) && + (!qemu_thread_is_self(&ms->thread)) && + migrate_dirty_limit() && + dirtylimit_in_service()) { + error_setg(errp, "dirty-limit live migration is running, not" + " allowing dirty page limit being canceled manually"); + return; + } + dirtylimit_state_lock(); =20 if (has_cpu_index) { @@ -487,6 +498,8 @@ void qmp_set_vcpu_dirty_limit(bool has_cpu_index, uint64_t dirty_rate, Error **errp) { + MigrationState *ms =3D migrate_get_current(); + if (!kvm_enabled() || !kvm_dirty_ring_enabled()) { error_setg(errp, "dirty page limit feature requires KVM with" " accelerator property 'dirty-ring-size' set'"); @@ -503,6 +516,15 @@ void qmp_set_vcpu_dirty_limit(bool has_cpu_index, return; } =20 + if (migration_is_running(ms->state) && + (!qemu_thread_is_self(&ms->thread)) && + migrate_dirty_limit() && + dirtylimit_in_service()) { + error_setg(errp, "dirty-limit live migration is running, not" + " allowing dirty page limit being configured manually"); + return; + } + dirtylimit_state_lock(); =20 if (!dirtylimit_in_service()) { --=20 1.8.3.1 From nobody Mon May 13 12:02:45 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1669048458309662.3491365509307; Mon, 21 Nov 2022 08:34:18 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ox9do-0000fU-Aj; Mon, 21 Nov 2022 11:27:28 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ox9dm-0000ed-6N for qemu-devel@nongnu.org; Mon, 21 Nov 2022 11:27:26 -0500 Received: from prt-mail.chinatelecom.cn ([42.123.76.226] helo=chinatelecom.cn) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ox9dk-0001ub-4m for qemu-devel@nongnu.org; Mon, 21 Nov 2022 11:27:25 -0500 Received: from clientip-125.69.40.159 (unknown [172.18.0.218]) by chinatelecom.cn (HERMES) with SMTP id 6D61F2800D1; Tue, 22 Nov 2022 00:27:20 +0800 (CST) Received: from ([125.69.40.159]) by app0025 with ESMTP id 3c79e9f4c6624b28a9b74f6e63b5ceb0 for qemu-devel@nongnu.org; Tue, 22 Nov 2022 00:27:23 CST HMM_SOURCE_IP: 172.18.0.218:59746.263116816 HMM_ATTACHE_NUM: 0000 HMM_SOURCE_TYPE: SMTP X-189-SAVE-TO-SEND: +huangy81@chinatelecom.cn X-Transaction-ID: 3c79e9f4c6624b28a9b74f6e63b5ceb0 X-Real-From: huangy81@chinatelecom.cn X-Receive-IP: 125.69.40.159 X-MEDUSA-Status: 0 From: huangy81@chinatelecom.cn To: qemu-devel Cc: Peter Xu , Markus Armbruster , "Dr. David Alan Gilbert" , Paolo Bonzini , Laurent Vivier , Eric Blake , Juan Quintela , Thomas Huth , Peter Maydell , Richard Henderson , =?UTF-8?q?Hyman=20Huang=28=E9=BB=84=E5=8B=87=29?= Subject: [PATCH v2 08/11] migration: Export dirty-limit time info Date: Mon, 21 Nov 2022 11:26:40 -0500 Message-Id: <513421b79099d7f73b3db227b5eb347fe9a3c241.1669047366.git.huangy81@chinatelecom.cn> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: References: In-Reply-To: References: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=42.123.76.226; envelope-from=huangy81@chinatelecom.cn; helo=chinatelecom.cn X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZM-MESSAGEID: 1669048459045100001 From: Hyman Huang(=E9=BB=84=E5=8B=87) Export dirty limit throttle time and estimated ring full time, through which we can observe the process of dirty limit during live migration. Signed-off-by: Hyman Huang(=E9=BB=84=E5=8B=87) --- include/sysemu/dirtylimit.h | 2 ++ migration/migration.c | 10 ++++++++++ monitor/hmp-cmds.c | 10 ++++++++++ qapi/migration.json | 10 +++++++++- softmmu/dirtylimit.c | 31 +++++++++++++++++++++++++++++++ 5 files changed, 62 insertions(+), 1 deletion(-) diff --git a/include/sysemu/dirtylimit.h b/include/sysemu/dirtylimit.h index 8d2c1f3..98cc4a6 100644 --- a/include/sysemu/dirtylimit.h +++ b/include/sysemu/dirtylimit.h @@ -34,4 +34,6 @@ void dirtylimit_set_vcpu(int cpu_index, void dirtylimit_set_all(uint64_t quota, bool enable); void dirtylimit_vcpu_execute(CPUState *cpu); +int64_t dirtylimit_throttle_us_per_full(void); +int64_t dirtylimit_us_ring_full(void); #endif diff --git a/migration/migration.c b/migration/migration.c index 096b61a..886c25d 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -62,6 +62,7 @@ #include "yank_functions.h" #include "sysemu/qtest.h" #include "sysemu/kvm.h" +#include "sysemu/dirtylimit.h" =20 #define MAX_THROTTLE (128 << 20) /* Migration transfer speed throttl= ing */ =20 @@ -1112,6 +1113,15 @@ static void populate_ram_info(MigrationInfo *info, M= igrationState *s) info->ram->remaining =3D ram_bytes_remaining(); info->ram->dirty_pages_rate =3D ram_counters.dirty_pages_rate; } + + if (migrate_dirty_limit() && dirtylimit_in_service()) { + info->has_dirty_limit_throttle_us_per_full =3D true; + info->dirty_limit_throttle_us_per_full =3D + dirtylimit_throttle_us_per_full(); + + info->has_dirty_limit_us_ring_full =3D true; + info->dirty_limit_us_ring_full =3D dirtylimit_us_ring_full(); + } } =20 static void populate_disk_info(MigrationInfo *info) diff --git a/monitor/hmp-cmds.c b/monitor/hmp-cmds.c index 9ad6ee5..9d02baf 100644 --- a/monitor/hmp-cmds.c +++ b/monitor/hmp-cmds.c @@ -339,6 +339,16 @@ void hmp_info_migrate(Monitor *mon, const QDict *qdict) info->cpu_throttle_percentage); } =20 + if (info->has_dirty_limit_throttle_us_per_full) { + monitor_printf(mon, "dirty-limit throttle time: %" PRIi64 " us\n", + info->dirty_limit_throttle_us_per_full); + } + + if (info->has_dirty_limit_us_ring_full) { + monitor_printf(mon, "dirty-limit ring full time: %" PRIi64 " us\n", + info->dirty_limit_us_ring_full); + } + if (info->has_postcopy_blocktime) { monitor_printf(mon, "postcopy blocktime: %u\n", info->postcopy_blocktime); diff --git a/qapi/migration.json b/qapi/migration.json index af6b2da..62db5cb 100644 --- a/qapi/migration.json +++ b/qapi/migration.json @@ -242,6 +242,12 @@ # Present and non-empty when migration is blocked. # (since 6.0) # +# @dirty-limit-throttle-us-per-full: Throttle time (us) during the period = of +# dirty ring full (since 7.1) +# +# @dirty-limit-us-ring-full: Estimated periodic time (us) of dirty ring fu= ll. +# (since 7.1) +# # Since: 0.14 ## { 'struct': 'MigrationInfo', @@ -259,7 +265,9 @@ '*postcopy-blocktime' : 'uint32', '*postcopy-vcpu-blocktime': ['uint32'], '*compression': 'CompressionStats', - '*socket-address': ['SocketAddress'] } } + '*socket-address': ['SocketAddress'], + '*dirty-limit-throttle-us-per-full': 'int64', + '*dirty-limit-us-ring-full': 'int64'} } =20 ## # @query-migrate: diff --git a/softmmu/dirtylimit.c b/softmmu/dirtylimit.c index 3f3c405..9d1df9b 100644 --- a/softmmu/dirtylimit.c +++ b/softmmu/dirtylimit.c @@ -573,6 +573,37 @@ static struct DirtyLimitInfo *dirtylimit_query_vcpu(in= t cpu_index) return info; } =20 +/* Pick up first vcpu throttle time by default */ +int64_t dirtylimit_throttle_us_per_full(void) +{ + CPUState *cpu =3D first_cpu; + return cpu->throttle_us_per_full; +} + +/* + * Estimate dirty ring full time under current dirty page rate. + * Return -1 if guest doesn't dirty memory. + */ +int64_t dirtylimit_us_ring_full(void) +{ + CPUState *cpu; + uint64_t curr_rate =3D 0; + int nvcpus =3D 0; + + CPU_FOREACH(cpu) { + if (cpu->running) { + nvcpus++; + curr_rate +=3D vcpu_dirty_rate_get(cpu->cpu_index); + } + } + + if (!curr_rate || !nvcpus) { + return -1; + } + + return dirtylimit_dirty_ring_full_time(curr_rate / nvcpus); +} + static struct DirtyLimitInfoList *dirtylimit_query_all(void) { int i, index; --=20 1.8.3.1 From nobody Mon May 13 12:02:45 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 166904842102585.02440074090202; Mon, 21 Nov 2022 08:33:41 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ox9dv-0000jH-2Q; Mon, 21 Nov 2022 11:27:35 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ox9du-0000gA-5k for qemu-devel@nongnu.org; Mon, 21 Nov 2022 11:27:34 -0500 Received: from prt-mail.chinatelecom.cn ([42.123.76.226] helo=chinatelecom.cn) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ox9do-0001uI-18 for qemu-devel@nongnu.org; Mon, 21 Nov 2022 11:27:31 -0500 Received: from clientip-125.69.40.159 (unknown [172.18.0.218]) by chinatelecom.cn (HERMES) with SMTP id 33C562800DB; Tue, 22 Nov 2022 00:27:23 +0800 (CST) Received: from ([125.69.40.159]) by app0025 with ESMTP id 177ba1192ea849d89813edb200e107a8 for qemu-devel@nongnu.org; Tue, 22 Nov 2022 00:27:26 CST HMM_SOURCE_IP: 172.18.0.218:59746.263116816 HMM_ATTACHE_NUM: 0000 HMM_SOURCE_TYPE: SMTP X-189-SAVE-TO-SEND: +huangy81@chinatelecom.cn X-Transaction-ID: 177ba1192ea849d89813edb200e107a8 X-Real-From: huangy81@chinatelecom.cn X-Receive-IP: 125.69.40.159 X-MEDUSA-Status: 0 From: huangy81@chinatelecom.cn To: qemu-devel Cc: Peter Xu , Markus Armbruster , "Dr. David Alan Gilbert" , Paolo Bonzini , Laurent Vivier , Eric Blake , Juan Quintela , Thomas Huth , Peter Maydell , Richard Henderson , =?UTF-8?q?Hyman=20Huang=28=E9=BB=84=E5=8B=87=29?= Subject: [PATCH v2 09/11] tests: Add migration dirty-limit capability test Date: Mon, 21 Nov 2022 11:26:41 -0500 Message-Id: <1b4c8f506b0b3018f2dad61a9e3485f26ce4fcba.1669047366.git.huangy81@chinatelecom.cn> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: References: In-Reply-To: References: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=42.123.76.226; envelope-from=huangy81@chinatelecom.cn; helo=chinatelecom.cn X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZM-MESSAGEID: 1669048423027100003 From: Hyman Huang(=E9=BB=84=E5=8B=87) Add migration dirty-limit capability test if kernel support dirty ring. Migration dirty-limit capability introduce dirty limit capability, two parameters: x-vcpu-dirty-limit-period and x-vcpu-dirty-limit are introduced to implement the live migration with dirty limit. The test case does the following things: 1. start src, dst vm and enable dirty-limit capability 2. start migrate and set cancel it to check if dirty limit stop working. 3. restart dst vm 4. start migrate and enable dirty-limit capability 5. check if migration satisfy the convergence condition during pre-switchover phase. Signed-off-by: Hyman Huang(=E9=BB=84=E5=8B=87) --- tests/qtest/migration-test.c | 154 +++++++++++++++++++++++++++++++++++++++= ++++ 1 file changed, 154 insertions(+) diff --git a/tests/qtest/migration-test.c b/tests/qtest/migration-test.c index 442998d..baa614c 100644 --- a/tests/qtest/migration-test.c +++ b/tests/qtest/migration-test.c @@ -2422,6 +2422,158 @@ static void test_vcpu_dirty_limit(void) dirtylimit_stop_vm(vm); } =20 +static void migrate_dirty_limit_wait_showup(QTestState *from, + const int64_t period, + const int64_t value) +{ + /* Enable dirty limit capability */ + migrate_set_capability(from, "dirty-limit", true); + + /* Set dirty limit parameters */ + migrate_set_parameter_int(from, "x-vcpu-dirty-limit-period", period); + migrate_set_parameter_int(from, "vcpu-dirty-limit", value); + + /* Make sure migrate can't converge */ + migrate_ensure_non_converge(from); + + /* To check limit rate after precopy */ + migrate_set_capability(from, "pause-before-switchover", true); + + /* Wait for the serial output from the source */ + wait_for_serial("src_serial"); +} + +/* + * This test does: + * source target + * migrate_incoming + * migrate + * migrate_cancel + * restart target + * migrate + * + * And see that if dirty limit works correctly + */ +static void test_migrate_dirty_limit(void) +{ + g_autofree char *uri =3D g_strdup_printf("unix:%s/migsocket", tmpfs); + QTestState *from, *to; + int64_t remaining, throttle_us_per_full; + /* + * We want the test to be stable and as fast as possible. + * E.g., with 1Gb/s bandwith migration may pass without dirty limit, + * so we need to decrease a bandwidth. + */ + const int64_t dirtylimit_period =3D 1000, dirtylimit_value =3D 50; + const int64_t max_bandwidth =3D 400000000; /* ~400Mb/s */ + const int64_t downtime_limit =3D 250; /* 250ms */ + /* + * We migrate through unix-socket (> 500Mb/s). + * Thus, expected migration speed ~=3D bandwidth limit (< 500Mb/s). + * So, we can predict expected_threshold + */ + const int64_t expected_threshold =3D max_bandwidth * downtime_limit / = 1000; + int max_try_count =3D 10; + MigrateCommon args =3D { + .start =3D { + .hide_stderr =3D true, + .use_dirty_ring =3D true, + }, + .listen_uri =3D uri, + .connect_uri =3D uri, + }; + + /* Start src, dst vm */ + if (test_migrate_start(&from, &to, args.listen_uri, &args.start)) { + return; + } + + /* Prepare for dirty limit migration and wait src vm show up */ + migrate_dirty_limit_wait_showup(from, dirtylimit_period, dirtylimit_va= lue); + + /* Start migrate */ + migrate_qmp(from, uri, "{}"); + + /* Wait for dirty limit throttle begin */ + throttle_us_per_full =3D 0; + while (throttle_us_per_full =3D=3D 0) { + throttle_us_per_full =3D + read_migrate_property_int(from, "dirty-limit-throttle-us-per-f= ull"); + usleep(100); + g_assert_false(got_stop); + } + + /* Now cancel migrate and wait for dirty limit throttle switch off */ + migrate_cancel(from); + wait_for_migration_status(from, "cancelled", NULL); + + /* Check if dirty limit throttle switched off, set timeout 1ms */ + do { + throttle_us_per_full =3D + read_migrate_property_int(from, "dirty-limit-throttle-us-per-f= ull"); + usleep(100); + g_assert_false(got_stop); + } while (throttle_us_per_full !=3D 0 && --max_try_count); + + /* Assert dirty limit is not in service */ + g_assert_cmpint(throttle_us_per_full, =3D=3D, 0); + + args =3D (MigrateCommon) { + .start =3D { + .only_target =3D true, + .use_dirty_ring =3D true, + }, + .listen_uri =3D uri, + .connect_uri =3D uri, + }; + + /* Restart dst vm, src vm already show up so we needn't wait anymore */ + if (test_migrate_start(&from, &to, args.listen_uri, &args.start)) { + return; + } + + /* Start migrate */ + migrate_qmp(from, uri, "{}"); + + /* Wait for dirty limit throttle begin */ + throttle_us_per_full =3D 0; + while (throttle_us_per_full =3D=3D 0) { + throttle_us_per_full =3D + read_migrate_property_int(from, "dirty-limit-throttle-us-per-f= ull"); + usleep(100); + g_assert_false(got_stop); + } + + /* + * The dirty limit rate should equals the return value of + * query-vcpu-dirty-limit if dirty limit cap set + */ + g_assert_cmpint(dirtylimit_value, =3D=3D, get_limit_rate(from)); + + /* Now, we have tested if dirty limit works, let it converge */ + migrate_set_parameter_int(from, "downtime-limit", downtime_limit); + migrate_set_parameter_int(from, "max-bandwidth", max_bandwidth); + + /* + * Wait for pre-switchover status to check if migration + * satisfy the convergence condition + */ + wait_for_migration_status(from, "pre-switchover", NULL); + + remaining =3D read_ram_property_int(from, "remaining"); + g_assert_cmpint(remaining, <, + (expected_threshold + expected_threshold / 100)); + + migrate_continue(from, "pre-switchover"); + + qtest_qmp_eventwait(to, "RESUME"); + + wait_for_serial("dest_serial"); + wait_for_migration_complete(from); + + test_migrate_end(from, to, true); +} + static bool kvm_dirty_ring_supported(void) { #if defined(__linux__) && defined(HOST_X86_64) @@ -2592,6 +2744,8 @@ int main(int argc, char **argv) test_precopy_unix_dirty_ring); qtest_add_func("/migration/vcpu_dirty_limit", test_vcpu_dirty_limit); + qtest_add_func("/migration/dirty_limit", + test_migrate_dirty_limit); } =20 ret =3D g_test_run(); --=20 1.8.3.1 From nobody Mon May 13 12:02:45 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1669048183089358.58732984193284; Mon, 21 Nov 2022 08:29:43 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ox9e0-0000xl-V6; Mon, 21 Nov 2022 11:27:41 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ox9dw-0000lQ-1T for qemu-devel@nongnu.org; Mon, 21 Nov 2022 11:27:36 -0500 Received: from prt-mail.chinatelecom.cn ([42.123.76.226] helo=chinatelecom.cn) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ox9dt-0001vu-UE for qemu-devel@nongnu.org; Mon, 21 Nov 2022 11:27:35 -0500 Received: from clientip-125.69.40.159 (unknown [172.18.0.218]) by chinatelecom.cn (HERMES) with SMTP id 7F0CE2800DE; Tue, 22 Nov 2022 00:27:26 +0800 (CST) Received: from ([125.69.40.159]) by app0025 with ESMTP id 24a37f7b918f499b88889b0262b303bc for qemu-devel@nongnu.org; Tue, 22 Nov 2022 00:27:29 CST HMM_SOURCE_IP: 172.18.0.218:59746.263116816 HMM_ATTACHE_NUM: 0000 HMM_SOURCE_TYPE: SMTP X-189-SAVE-TO-SEND: +huangy81@chinatelecom.cn X-Transaction-ID: 24a37f7b918f499b88889b0262b303bc X-Real-From: huangy81@chinatelecom.cn X-Receive-IP: 125.69.40.159 X-MEDUSA-Status: 0 From: huangy81@chinatelecom.cn To: qemu-devel Cc: Peter Xu , Markus Armbruster , "Dr. David Alan Gilbert" , Paolo Bonzini , Laurent Vivier , Eric Blake , Juan Quintela , Thomas Huth , Peter Maydell , Richard Henderson , =?UTF-8?q?Hyman=20Huang=28=E9=BB=84=E5=8B=87=29?= Subject: [PATCH v2 10/11] tests/migration: Introduce dirty-ring-size option into guestperf Date: Mon, 21 Nov 2022 11:26:42 -0500 Message-Id: <72595118953e8c2b5c548052788a22e9e3b1fa6e.1669047366.git.huangy81@chinatelecom.cn> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: References: In-Reply-To: References: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=42.123.76.226; envelope-from=huangy81@chinatelecom.cn; helo=chinatelecom.cn X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZM-MESSAGEID: 1669048184399100003 From: Hyman Huang(=E9=BB=84=E5=8B=87) Guestperf tool does not enable diry ring feature when test migration by default. To support dirty ring migration performance test, introduce dirty-ring-size option into guestperf tools, which ranges in [1024, 65536]. To set dirty ring size with 4096 during migration test: $ ./tests/migration/guestperf.py --dirty-ring-size 4096 xxx Signed-off-by: Hyman Huang(=E9=BB=84=E5=8B=87) --- tests/migration/guestperf/engine.py | 7 ++++++- tests/migration/guestperf/hardware.py | 8 ++++++-- tests/migration/guestperf/shell.py | 7 ++++++- 3 files changed, 18 insertions(+), 4 deletions(-) diff --git a/tests/migration/guestperf/engine.py b/tests/migration/guestper= f/engine.py index 59fca2c..d7b75b9 100644 --- a/tests/migration/guestperf/engine.py +++ b/tests/migration/guestperf/engine.py @@ -303,7 +303,6 @@ def _get_common_args(self, hardware, tunnelled=3DFalse): cmdline =3D "'" + cmdline + "'" =20 argv =3D [ - "-accel", "kvm", "-cpu", "host", "-kernel", self._kernel, "-initrd", self._initrd, @@ -314,6 +313,12 @@ def _get_common_args(self, hardware, tunnelled=3DFalse= ): "-smp", str(hardware._cpus), ] =20 + if hardware._dirty_ring_size: + argv.extend(["-accel", "kvm,dirty-ring-size=3D%s" % + hardware._dirty_ring_size]) + else: + argv.extend(["-accel", "kvm"]) + if self._debug: argv.extend(["-device", "sga"]) =20 diff --git a/tests/migration/guestperf/hardware.py b/tests/migration/guestp= erf/hardware.py index 3145785..f779cc0 100644 --- a/tests/migration/guestperf/hardware.py +++ b/tests/migration/guestperf/hardware.py @@ -23,7 +23,8 @@ def __init__(self, cpus=3D1, mem=3D1, src_cpu_bind=3DNone, src_mem_bind=3DNone, dst_cpu_bind=3DNone, dst_mem_bind=3DNone, prealloc_pages =3D False, - huge_pages=3DFalse, locked_pages=3DFalse): + huge_pages=3DFalse, locked_pages=3DFalse, + dirty_ring_size=3D0): self._cpus =3D cpus self._mem =3D mem # GiB self._src_mem_bind =3D src_mem_bind # List of NUMA nodes @@ -33,6 +34,7 @@ def __init__(self, cpus=3D1, mem=3D1, self._prealloc_pages =3D prealloc_pages self._huge_pages =3D huge_pages self._locked_pages =3D locked_pages + self._dirty_ring_size =3D dirty_ring_size =20 =20 def serialize(self): @@ -46,6 +48,7 @@ def serialize(self): "prealloc_pages": self._prealloc_pages, "huge_pages": self._huge_pages, "locked_pages": self._locked_pages, + "dirty_ring_size": self._dirty_ring_size, } =20 @classmethod @@ -59,4 +62,5 @@ def deserialize(cls, data): data["dst_mem_bind"], data["prealloc_pages"], data["huge_pages"], - data["locked_pages"]) + data["locked_pages"], + data["dirty_ring_size"]) diff --git a/tests/migration/guestperf/shell.py b/tests/migration/guestperf= /shell.py index 8a809e3..559616f 100644 --- a/tests/migration/guestperf/shell.py +++ b/tests/migration/guestperf/shell.py @@ -60,6 +60,8 @@ def __init__(self): parser.add_argument("--prealloc-pages", dest=3D"prealloc_pages", d= efault=3DFalse) parser.add_argument("--huge-pages", dest=3D"huge_pages", default= =3DFalse) parser.add_argument("--locked-pages", dest=3D"locked_pages", defau= lt=3DFalse) + parser.add_argument("--dirty-ring-size", dest=3D"dirty_ring_size", + default=3D0, type=3Dint) =20 self._parser =3D parser =20 @@ -89,7 +91,10 @@ def split_map(value): =20 locked_pages=3Dargs.locked_pages, huge_pages=3Dargs.huge_pages, - prealloc_pages=3Dargs.prealloc_pages) + prealloc_pages=3Dargs.prealloc_pages, + + dirty_ring_size=3Dargs.dirty_ring_size) + =20 =20 class Shell(BaseShell): --=20 1.8.3.1 From nobody Mon May 13 12:02:45 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1669048116466852.772850362028; Mon, 21 Nov 2022 08:28:36 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ox9e1-0000yy-Pj; Mon, 21 Nov 2022 11:27:41 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ox9dx-0000pu-0r for qemu-devel@nongnu.org; Mon, 21 Nov 2022 11:27:40 -0500 Received: from prt-mail.chinatelecom.cn ([42.123.76.226] helo=chinatelecom.cn) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ox9dt-0001wA-Us for qemu-devel@nongnu.org; Mon, 21 Nov 2022 11:27:36 -0500 Received: from clientip-125.69.40.159 (unknown [172.18.0.218]) by chinatelecom.cn (HERMES) with SMTP id 48BCA2800E1; Tue, 22 Nov 2022 00:27:29 +0800 (CST) Received: from ([125.69.40.159]) by app0025 with ESMTP id 771c87054c20464fb2742dae09d48645 for qemu-devel@nongnu.org; Tue, 22 Nov 2022 00:27:32 CST HMM_SOURCE_IP: 172.18.0.218:59746.263116816 HMM_ATTACHE_NUM: 0000 HMM_SOURCE_TYPE: SMTP X-189-SAVE-TO-SEND: +huangy81@chinatelecom.cn X-Transaction-ID: 771c87054c20464fb2742dae09d48645 X-Real-From: huangy81@chinatelecom.cn X-Receive-IP: 125.69.40.159 X-MEDUSA-Status: 0 From: huangy81@chinatelecom.cn To: qemu-devel Cc: Peter Xu , Markus Armbruster , "Dr. David Alan Gilbert" , Paolo Bonzini , Laurent Vivier , Eric Blake , Juan Quintela , Thomas Huth , Peter Maydell , Richard Henderson , =?UTF-8?q?Hyman=20Huang=28=E9=BB=84=E5=8B=87=29?= Subject: [PATCH v2 11/11] tests/migration: Introduce dirty-limit into guestperf Date: Mon, 21 Nov 2022 11:26:43 -0500 Message-Id: X-Mailer: git-send-email 1.8.3.1 In-Reply-To: References: In-Reply-To: References: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=42.123.76.226; envelope-from=huangy81@chinatelecom.cn; helo=chinatelecom.cn X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZM-MESSAGEID: 1669048117866100001 From: Hyman Huang(=E9=BB=84=E5=8B=87) Guestperf tool does not cover the dirty-limit migration currently, support this feature. To enable dirty-limit, setting x-vcpu-dirty-limit-period as 500ms and x-vcpu-dirty-limit as 10MB/s: $ ./tests/migration/guestperf.py \ --dirty-limit --x-vcpu-dirty-limit-period 500 \ --vcpu-dirty-limit 10 --output output.json \ To run the entire standardized set of dirty-limit-enabled comparisons, with unix migration: $ ./tests/migration/guestperf-batch.py \ --dst-host localhost --transport unix \ --filter compr-dirty-limit* --output outputdir Signed-off-by: Hyman Huang(=E9=BB=84=E5=8B=87) --- tests/migration/guestperf/comparison.py | 24 ++++++++++++++++++++++++ tests/migration/guestperf/engine.py | 17 +++++++++++++++++ tests/migration/guestperf/progress.py | 17 +++++++++++++++-- tests/migration/guestperf/scenario.py | 11 ++++++++++- tests/migration/guestperf/shell.py | 18 +++++++++++++++++- 5 files changed, 83 insertions(+), 4 deletions(-) diff --git a/tests/migration/guestperf/comparison.py b/tests/migration/gues= tperf/comparison.py index c03b3f6..ad403f9 100644 --- a/tests/migration/guestperf/comparison.py +++ b/tests/migration/guestperf/comparison.py @@ -135,4 +135,28 @@ def __init__(self, name, scenarios): Scenario("compr-multifd-channels-64", multifd=3DTrue, multifd_channels=3D64), ]), + + + # Looking at effect of dirty-limit with + # varying x_vcpu_dirty_limit_period + Comparison("compr-dirty-limit-period", scenarios =3D [ + Scenario("compr-dirty-limit-period-100", + dirty_limit=3DTrue, x_vcpu_dirty_limit_period=3D100), + Scenario("compr-dirty-limit-period-500", + dirty_limit=3DTrue, x_vcpu_dirty_limit_period=3D500), + Scenario("compr-dirty-limit-period-1000", + dirty_limit=3DTrue, x_vcpu_dirty_limit_period=3D1000), + ]), + + + # Looking at effect of dirty-limit with + # varying vcpu_dirty_limit + Comparison("compr-dirty-limit", scenarios =3D [ + Scenario("compr-dirty-limit-10MB", + dirty_limit=3DTrue, vcpu_dirty_limit=3D10), + Scenario("compr-dirty-limit-20MB", + dirty_limit=3DTrue, vcpu_dirty_limit=3D20), + Scenario("compr-dirty-limit-50MB", + dirty_limit=3DTrue, vcpu_dirty_limit=3D50), + ]), ] diff --git a/tests/migration/guestperf/engine.py b/tests/migration/guestper= f/engine.py index d7b75b9..e3940bf 100644 --- a/tests/migration/guestperf/engine.py +++ b/tests/migration/guestperf/engine.py @@ -102,6 +102,8 @@ def _migrate_progress(self, vm): info.get("expected-downtime", 0), info.get("setup-time", 0), info.get("cpu-throttle-percentage", 0), + info.get("dirty-limit-throttle-us-per-full", 0), + info.get("dirty-limit-us-ring-full", 0), ) =20 def _migrate(self, hardware, scenario, src, dst, connect_uri): @@ -203,6 +205,21 @@ def _migrate(self, hardware, scenario, src, dst, conne= ct_uri): resp =3D dst.command("migrate-set-parameters", multifd_channels=3Dscenario._multifd_channe= ls) =20 + if scenario._dirty_limit: + if not hardware._dirty_ring_size: + raise Exception("dirty ring size must be configured when " + "testing dirty limit migration") + + resp =3D src.command("migrate-set-capabilities", + capabilities =3D [ + { "capability": "dirty-limit", + "state": True } + ]) + resp =3D src.command("migrate-set-parameters", + x_vcpu_dirty_limit_period=3Dscenario._x_vcpu_dirty_limit_p= eriod) + resp =3D src.command("migrate-set-parameters", + vcpu_dirty_limit=3Dscenario._vcpu_dirty_lim= it) + resp =3D src.command("migrate", uri=3Dconnect_uri) =20 post_copy =3D False diff --git a/tests/migration/guestperf/progress.py b/tests/migration/guestp= erf/progress.py index ab1ee57..dd5d86b 100644 --- a/tests/migration/guestperf/progress.py +++ b/tests/migration/guestperf/progress.py @@ -81,7 +81,9 @@ def __init__(self, downtime, downtime_expected, setup_time, - throttle_pcent): + throttle_pcent, + dirty_limit_throttle_us_per_full, + dirty_limit_us_ring_full): =20 self._status =3D status self._ram =3D ram @@ -91,6 +93,11 @@ def __init__(self, self._downtime_expected =3D downtime_expected self._setup_time =3D setup_time self._throttle_pcent =3D throttle_pcent + self._dirty_limit_throttle_us_per_full =3D + dirty_limit_throttle_us_per_full + self._dirty_limit_us_ring_full =3D + dirty_limit_us_ring_full + =20 def serialize(self): return { @@ -102,6 +109,10 @@ def serialize(self): "downtime_expected": self._downtime_expected, "setup_time": self._setup_time, "throttle_pcent": self._throttle_pcent, + "dirty_limit_throttle_time_per_full": + self._dirty_limit_throttle_us_per_full, + "dirty_limit_ring_full_time": + self._dirty_limit_us_ring_full, } =20 @classmethod @@ -114,4 +125,6 @@ def deserialize(cls, data): data["downtime"], data["downtime_expected"], data["setup_time"], - data["throttle_pcent"]) + data["throttle_pcent"], + data["dirty_limit_throttle_time_per_full"], + data["dirty_limit_ring_full_time"]) diff --git a/tests/migration/guestperf/scenario.py b/tests/migration/guestp= erf/scenario.py index de70d9b..154c4f5 100644 --- a/tests/migration/guestperf/scenario.py +++ b/tests/migration/guestperf/scenario.py @@ -30,7 +30,9 @@ def __init__(self, name, auto_converge=3DFalse, auto_converge_step=3D10, compression_mt=3DFalse, compression_mt_threads=3D1, compression_xbzrle=3DFalse, compression_xbzrle_cache=3D10, - multifd=3DFalse, multifd_channels=3D2): + multifd=3DFalse, multifd_channels=3D2, + dirty_limit=3DFalse, x_vcpu_dirty_limit_period=3D500, + vcpu_dirty_limit=3D1): =20 self._name =3D name =20 @@ -60,6 +62,10 @@ def __init__(self, name, self._multifd =3D multifd self._multifd_channels =3D multifd_channels =20 + self._dirty_limit =3D dirty_limit + self._x_vcpu_dirty_limit_period =3D x_vcpu_dirty_limit_period + self._vcpu_dirty_limit =3D vcpu_dirty_limit + def serialize(self): return { "name": self._name, @@ -79,6 +85,9 @@ def serialize(self): "compression_xbzrle_cache": self._compression_xbzrle_cache, "multifd": self._multifd, "multifd_channels": self._multifd_channels, + "dirty_limit": self._dirty_limit, + "x_vcpu_dirty_limit_period": self._x_vcpu_dirty_limit_period, + "vcpu_dirty_limit": self._vcpu_dirty_limit, } =20 @classmethod diff --git a/tests/migration/guestperf/shell.py b/tests/migration/guestperf= /shell.py index 559616f..23fe895 100644 --- a/tests/migration/guestperf/shell.py +++ b/tests/migration/guestperf/shell.py @@ -132,6 +132,17 @@ def __init__(self): parser.add_argument("--multifd-channels", dest=3D"multifd_channels= ", default=3D2, type=3Dint) =20 + parser.add_argument("--dirty-limit", dest=3D"dirty_limit", default= =3DFalse, + action=3D"store_true") + + parser.add_argument("--x-vcpu-dirty-limit-period", + dest=3D"x_vcpu_dirty_limit_period", + default=3D500, type=3Dint) + + parser.add_argument("--vcpu-dirty-limit", + dest=3D"vcpu_dirty_limit", + default=3D1, type=3Dint) + def get_scenario(self, args): return Scenario(name=3D"perfreport", downtime=3Dargs.downtime, @@ -155,7 +166,12 @@ def get_scenario(self, args): compression_xbzrle_cache=3Dargs.compression_xbzrle= _cache, =20 multifd=3Dargs.multifd, - multifd_channels=3Dargs.multifd_channels) + multifd_channels=3Dargs.multifd_channels, + + dirty_limit=3Dargs.dirty_limit, + x_vcpu_dirty_limit_period=3D + args.x_vcpu_dirty_limit_period, + vcpu_dirty_limit=3Dargs.vcpu_dirty_limit) =20 def run(self, argv): args =3D self._parser.parse_args(argv) --=20 1.8.3.1