From nobody Fri Mar 29 06:05:52 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1495642355251739.7780805329438; Wed, 24 May 2017 09:12:35 -0700 (PDT) Received: from localhost ([::1]:55835 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dDYth-0005Xy-Ok for importer@patchew.org; Wed, 24 May 2017 12:12:29 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:54836) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dDYs1-0004Ni-BS for qemu-devel@nongnu.org; Wed, 24 May 2017 12:10:46 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1dDYrx-0001ki-AU for qemu-devel@nongnu.org; Wed, 24 May 2017 12:10:45 -0400 Received: from [62.254.189.133] (port=10573 helo=centos.localdomain) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dDYrw-0001e2-Uz for qemu-devel@nongnu.org; Wed, 24 May 2017 12:10:41 -0400 Received: by centos.localdomain (Postfix, from userid 500) id 1815A9FCD3; Wed, 24 May 2017 17:10:16 +0100 (BST) From: Felipe Franciosi To: Peter Xu , Juan Quintela , "Jason J. Herne" , amit Shah , "Dr. David Alan Gilbert" Date: Wed, 24 May 2017 17:10:00 +0100 Message-Id: <1495642203-12702-2-git-send-email-felipe@nutanix.com> X-Mailer: git-send-email 1.9.5 In-Reply-To: <1495642203-12702-1-git-send-email-felipe@nutanix.com> References: <1495642203-12702-1-git-send-email-felipe@nutanix.com> X-detected-operating-system: by eggs.gnu.org: Mac OS X [generic] [fuzzy] X-Received-From: 62.254.189.133 Subject: [Qemu-devel] [PATCH 1/4] migration: keep bytes_xfer_prev init'd to zero X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Malcolm Crossley , "qemu-devel@nongnu.org" , Felipe Franciosi Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" The first time migration_bitmap_sync() is called, bytes_xfer_prev is set to ram_state.bytes_transferred which is, at this point, zero. The next time migration_bitmap_sync() is called, an iteration has happened and bytes_xfer_prev is set to 'x' bytes. Most likely, more than one second has passed, so the auto converge logic will be triggered and bytes_xfer_now will also be set to 'x' bytes. This condition is currently masked by dirty_rate_high_cnt, which will wait for a few iterations before throttling. It would otherwise always assume zero bytes have been copied and therefore throttle the guest (possibly) prematurely. Given bytes_xfer_prev is only used by the auto convergence logic, it makes sense to only set its value after a check has been made against bytes_xfer_now. Signed-off-by: Felipe Franciosi Reviewed-by: Juan Quintela Reviewed-by: Peter Xu --- migration/ram.c | 4 ---- 1 file changed, 4 deletions(-) diff --git a/migration/ram.c b/migration/ram.c index c07a9c0..36bf720 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -673,10 +673,6 @@ static void migration_bitmap_sync(RAMState *rs) =20 rs->bitmap_sync_count++; =20 - if (!rs->bytes_xfer_prev) { - rs->bytes_xfer_prev =3D ram_bytes_transferred(); - } - if (!rs->time_last_bitmap_sync) { rs->time_last_bitmap_sync =3D qemu_clock_get_ms(QEMU_CLOCK_REALTIM= E); } --=20 1.9.5 From nobody Fri Mar 29 06:05:52 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1495642351425715.7963630992666; Wed, 24 May 2017 09:12:31 -0700 (PDT) Received: from localhost ([::1]:55834 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dDYth-0005XN-HE for importer@patchew.org; Wed, 24 May 2017 12:12:29 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:54837) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dDYs1-0004Nk-Bu for qemu-devel@nongnu.org; Wed, 24 May 2017 12:10:46 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1dDYrx-0001kS-A0 for qemu-devel@nongnu.org; Wed, 24 May 2017 12:10:45 -0400 Received: from [62.254.189.133] (port=26545 helo=centos.localdomain) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dDYrw-0001eC-Uy for qemu-devel@nongnu.org; Wed, 24 May 2017 12:10:41 -0400 Received: by centos.localdomain (Postfix, from userid 500) id BF48B9FCD4; Wed, 24 May 2017 17:10:16 +0100 (BST) From: Felipe Franciosi To: Peter Xu , Juan Quintela , "Jason J. Herne" , amit Shah , "Dr. David Alan Gilbert" Date: Wed, 24 May 2017 17:10:01 +0100 Message-Id: <1495642203-12702-3-git-send-email-felipe@nutanix.com> X-Mailer: git-send-email 1.9.5 In-Reply-To: <1495642203-12702-1-git-send-email-felipe@nutanix.com> References: <1495642203-12702-1-git-send-email-felipe@nutanix.com> X-detected-operating-system: by eggs.gnu.org: Mac OS X [generic] [fuzzy] X-Received-From: 62.254.189.133 Subject: [Qemu-devel] [PATCH 2/4] migration: set dirty_pages_rate before autoconverge logic X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Malcolm Crossley , "qemu-devel@nongnu.org" , Felipe Franciosi Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Currently, a "period" in the RAM migration logic is at least a second long and accounts for what happened since the last period (or the beginning of the migration). The dirty_pages_rate counter is calculated at the end this logic. If the auto convergence capability is enabled from the start of the migration, it won't be able to use this counter the first time around. This calculates dirty_pages_rate as soon as a period is deemed over, which allows for it to be used immediately. Signed-off-by: Felipe Franciosi Reviewed-by: Juan Quintela Reviewed-by: Peter Xu --- migration/ram.c | 17 ++++++++++------- 1 file changed, 10 insertions(+), 7 deletions(-) diff --git a/migration/ram.c b/migration/ram.c index 36bf720..495ecbe 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -694,6 +694,10 @@ static void migration_bitmap_sync(RAMState *rs) =20 /* more than 1 second =3D 1000 millisecons */ if (end_time > rs->time_last_bitmap_sync + 1000) { + /* calculate period counters */ + rs->dirty_pages_rate =3D rs->num_dirty_pages_period * 1000 + / (end_time - rs->time_last_bitmap_sync); + if (migrate_auto_converge()) { /* The following detection logic can be refined later. For now: Check to see if the dirtied bytes is 50% more than the appr= ox. @@ -702,15 +706,14 @@ static void migration_bitmap_sync(RAMState *rs) throttling */ bytes_xfer_now =3D ram_bytes_transferred(); =20 - if (rs->dirty_pages_rate && - (rs->num_dirty_pages_period * TARGET_PAGE_SIZE > + if ((rs->num_dirty_pages_period * TARGET_PAGE_SIZE > (bytes_xfer_now - rs->bytes_xfer_prev) / 2) && - (rs->dirty_rate_high_cnt++ >=3D 2)) { + (rs->dirty_rate_high_cnt++ >=3D 2)) { trace_migration_throttle(); rs->dirty_rate_high_cnt =3D 0; mig_throttle_guest_down(); - } - rs->bytes_xfer_prev =3D bytes_xfer_now; + } + rs->bytes_xfer_prev =3D bytes_xfer_now; } =20 if (migrate_use_xbzrle()) { @@ -723,8 +726,8 @@ static void migration_bitmap_sync(RAMState *rs) rs->iterations_prev =3D rs->iterations; rs->xbzrle_cache_miss_prev =3D rs->xbzrle_cache_miss; } - rs->dirty_pages_rate =3D rs->num_dirty_pages_period * 1000 - / (end_time - rs->time_last_bitmap_sync); + + /* reset period counters */ rs->time_last_bitmap_sync =3D end_time; rs->num_dirty_pages_period =3D 0; } --=20 1.9.5 From nobody Fri Mar 29 06:05:52 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1495642497936367.79062363014555; Wed, 24 May 2017 09:14:57 -0700 (PDT) Received: from localhost ([::1]:55845 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dDYw4-00080U-K6 for importer@patchew.org; Wed, 24 May 2017 12:14:56 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:54842) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dDYs1-0004Nn-Cd for qemu-devel@nongnu.org; Wed, 24 May 2017 12:10:46 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1dDYrx-0001kX-AY for qemu-devel@nongnu.org; Wed, 24 May 2017 12:10:45 -0400 Received: from [62.254.189.133] (port=14873 helo=centos.localdomain) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dDYrw-0001eK-V4 for qemu-devel@nongnu.org; Wed, 24 May 2017 12:10:41 -0400 Received: by centos.localdomain (Postfix, from userid 500) id 61E8D9FDA8; Wed, 24 May 2017 17:10:17 +0100 (BST) From: Felipe Franciosi To: Peter Xu , Juan Quintela , "Jason J. Herne" , amit Shah , "Dr. David Alan Gilbert" Date: Wed, 24 May 2017 17:10:02 +0100 Message-Id: <1495642203-12702-4-git-send-email-felipe@nutanix.com> X-Mailer: git-send-email 1.9.5 In-Reply-To: <1495642203-12702-1-git-send-email-felipe@nutanix.com> References: <1495642203-12702-1-git-send-email-felipe@nutanix.com> X-detected-operating-system: by eggs.gnu.org: Mac OS X [generic] [fuzzy] X-Received-From: 62.254.189.133 Subject: [Qemu-devel] [PATCH 3/4] migration: set bytes_xfer_* outside of autoconverge logic X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Malcolm Crossley , "qemu-devel@nongnu.org" , Felipe Franciosi Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" The bytes_xfer_now/prev counters are only used by the auto convergence logic. However, they are used alongside the dirty_pages_rate counter, which is calculated (and required) outside of this logic. The problem with this approach is that if the auto convergence capability is changed while a migration is ongoing, the relationship of the counters will be broken. This moves the management of bytes_xfer_now/prev counters outside of the auto convergence logic to address this issue. Signed-off-by: Felipe Franciosi Reviewed-by: Juan Quintela Reviewed-by: Peter Xu --- migration/ram.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/migration/ram.c b/migration/ram.c index 495ecbe..1a3d9e6 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -697,6 +697,7 @@ static void migration_bitmap_sync(RAMState *rs) /* calculate period counters */ rs->dirty_pages_rate =3D rs->num_dirty_pages_period * 1000 / (end_time - rs->time_last_bitmap_sync); + bytes_xfer_now =3D ram_bytes_transferred(); =20 if (migrate_auto_converge()) { /* The following detection logic can be refined later. For now: @@ -704,7 +705,6 @@ static void migration_bitmap_sync(RAMState *rs) amount of bytes that just got transferred since the last ti= me we were in this routine. If that happens twice, start or incre= ase throttling */ - bytes_xfer_now =3D ram_bytes_transferred(); =20 if ((rs->num_dirty_pages_period * TARGET_PAGE_SIZE > (bytes_xfer_now - rs->bytes_xfer_prev) / 2) && @@ -713,7 +713,6 @@ static void migration_bitmap_sync(RAMState *rs) rs->dirty_rate_high_cnt =3D 0; mig_throttle_guest_down(); } - rs->bytes_xfer_prev =3D bytes_xfer_now; } =20 if (migrate_use_xbzrle()) { @@ -730,6 +729,7 @@ static void migration_bitmap_sync(RAMState *rs) /* reset period counters */ rs->time_last_bitmap_sync =3D end_time; rs->num_dirty_pages_period =3D 0; + rs->bytes_xfer_prev =3D bytes_xfer_now; } if (migrate_use_events()) { qapi_event_send_migration_pass(rs->bitmap_sync_count, NULL); --=20 1.9.5 From nobody Fri Mar 29 06:05:52 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1495642439120233.47533587942928; Wed, 24 May 2017 09:13:59 -0700 (PDT) Received: from localhost ([::1]:55842 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dDYv7-0006rN-Fn for importer@patchew.org; Wed, 24 May 2017 12:13:57 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:54839) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dDYs1-0004Nl-CH for qemu-devel@nongnu.org; Wed, 24 May 2017 12:10:46 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1dDYrx-0001kW-9m for qemu-devel@nongnu.org; Wed, 24 May 2017 12:10:45 -0400 Received: from [62.254.189.133] (port=33355 helo=centos.localdomain) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dDYrw-0001eb-Uq for qemu-devel@nongnu.org; Wed, 24 May 2017 12:10:41 -0400 Received: by centos.localdomain (Postfix, from userid 500) id 12DCE9FDAC; Wed, 24 May 2017 17:10:18 +0100 (BST) From: Felipe Franciosi To: Peter Xu , Juan Quintela , "Jason J. Herne" , amit Shah , "Dr. David Alan Gilbert" Date: Wed, 24 May 2017 17:10:03 +0100 Message-Id: <1495642203-12702-5-git-send-email-felipe@nutanix.com> X-Mailer: git-send-email 1.9.5 In-Reply-To: <1495642203-12702-1-git-send-email-felipe@nutanix.com> References: <1495642203-12702-1-git-send-email-felipe@nutanix.com> X-detected-operating-system: by eggs.gnu.org: Mac OS X [generic] [fuzzy] X-Received-From: 62.254.189.133 Subject: [Qemu-devel] [PATCH 4/4] migration: use dirty_rate_high_cnt more aggressively X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Malcolm Crossley , "qemu-devel@nongnu.org" , Felipe Franciosi Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" The commit message from 070afca25 suggests that dirty_rate_high_cnt should be used more aggressively to start throttling after two iterations instead of four. The code, however, only changes the auto convergence behaviour to throttle after three iterations. This makes the behaviour more aggressive by kicking off throttling after two iterations as originally intended. Signed-off-by: Felipe Franciosi Reviewed-by: Juan Quintela --- migration/ram.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/migration/ram.c b/migration/ram.c index 1a3d9e6..26e03a5 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -708,7 +708,7 @@ static void migration_bitmap_sync(RAMState *rs) =20 if ((rs->num_dirty_pages_period * TARGET_PAGE_SIZE > (bytes_xfer_now - rs->bytes_xfer_prev) / 2) && - (rs->dirty_rate_high_cnt++ >=3D 2)) { + (++rs->dirty_rate_high_cnt >=3D 2)) { trace_migration_throttle(); rs->dirty_rate_high_cnt =3D 0; mig_throttle_guest_down(); --=20 1.9.5