From nobody Mon Feb 9 08:57:04 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1595883466; cv=none; d=zohomail.com; s=zohoarc; b=VhmlF9EVPqaEsjAbuq+Du9KuHJ6f7KjYtGScS/4ZrSKuJ15RK1Y+Wfl6Lqs+R5Ni9CrB19oMmCJesR/bG7C6q3EyCZZBIKniUIMXG+64SoHs8+GIdCHNupHkA23QzBW0yu2u6pSeKBlKa5WihbFuvvw4cDf/RhTNIUMgAexFHDI= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1595883466; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=hOJpc8RV8qvMz8aH/tH1tmr2hdSTKc/5P7btzVhxFp0=; b=iu9qX2hdS/xeIW3BXGw5rKMMj/tyap09CnKGmX+eUMKcNLOB+uI7XIPgxWweyRAP8BqmTHiJppcflNLXWHhxzDYfvHjF9tnKPKm4o+tjTKynMynkggxwuwvfPTO1HXGIWTP5wKyiFzTmzl8JyI97d9M2gmPOOijD7mUEO1eTlc4= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1595883466120599.9281703039179; Mon, 27 Jul 2020 13:57:46 -0700 (PDT) Received: from localhost ([::1]:43202 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1k0ABs-0006pn-Pa for importer@patchew.org; Mon, 27 Jul 2020 16:57:44 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:34792) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1k0AAC-0004v4-C9 for qemu-devel@nongnu.org; Mon, 27 Jul 2020 16:56:00 -0400 Received: from us-smtp-1.mimecast.com ([207.211.31.81]:47810 helo=us-smtp-delivery-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1k0AAA-0004rn-FU for qemu-devel@nongnu.org; Mon, 27 Jul 2020 16:56:00 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-429-dMQax1MiM5uFCizsQ1mqMQ-1; Mon, 27 Jul 2020 16:55:55 -0400 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id D2DC38E1361; Mon, 27 Jul 2020 20:55:53 +0000 (UTC) Received: from blue.redhat.com (ovpn-118-248.rdu2.redhat.com [10.10.118.248]) by smtp.corp.redhat.com (Postfix) with ESMTP id EB39D19724; Mon, 27 Jul 2020 20:55:52 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1595883357; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=hOJpc8RV8qvMz8aH/tH1tmr2hdSTKc/5P7btzVhxFp0=; b=OpnSgIfBjvnb2/VkzC0lC7w9ArxMFI4BBJeFhy2pNgAy+zn+bsXoi1hpp4HtvhXSlwSzGq yLlle2h9BEBKJhJX7xb5nIZxul6ALClJGZlMy6uDCQTiq7BlCOuIH852qmJgCUTuMQtyQT erSLi3aHCZIsKEERk3TUgwt9KrF6Ow4= X-MC-Unique: dMQax1MiM5uFCizsQ1mqMQ-1 From: Eric Blake To: qemu-devel@nongnu.org Subject: [PULL 04/24] qemu-iotests/199: better catch postcopy time Date: Mon, 27 Jul 2020 15:55:23 -0500 Message-Id: <20200727205543.206624-5-eblake@redhat.com> In-Reply-To: <20200727205543.206624-1-eblake@redhat.com> References: <20200727205543.206624-1-eblake@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.23 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=207.211.31.81; envelope-from=eblake@redhat.com; helo=us-smtp-delivery-1.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/07/27 03:37:14 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Spam_score_int: -30 X-Spam_score: -3.1 X-Spam_bar: --- X-Spam_report: (-3.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-1, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Andrey Shinkevich , Vladimir Sementsov-Ogievskiy , "open list:Block layer core" , Max Reitz Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) Content-Type: text/plain; charset="utf-8" From: Vladimir Sementsov-Ogievskiy The test aims to test _postcopy_ migration, and wants to do some write operations during postcopy time. Test considers migrate status=3Dcomplete event on source as start of postcopy. This is completely wrong, completion is completion of the whole migration process. Let's instead consider destination start as start of postcopy, and use RESUME event for it. Next, as migration finish, let's use migration status=3Dcomplete event on target, as such method is closer to what libvirt or another user will do, than tracking number of dirty-bitmaps. Finally, add a possibility to dump events for debug. And if set debug to True, we see, that actual postcopy period is very small relatively to the whole test duration time (~0.2 seconds to >40 seconds for me). This means, that test is very inefficient in what it supposed to do. Let's improve it in following commits. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Andrey Shinkevich Tested-by: Eric Blake Message-Id: <20200727194236.19551-4-vsementsov@virtuozzo.com> Signed-off-by: Eric Blake --- tests/qemu-iotests/199 | 72 +++++++++++++++++++++++++++++++++--------- 1 file changed, 57 insertions(+), 15 deletions(-) diff --git a/tests/qemu-iotests/199 b/tests/qemu-iotests/199 index dda918450a8b..dd6044768c76 100755 --- a/tests/qemu-iotests/199 +++ b/tests/qemu-iotests/199 @@ -20,17 +20,43 @@ import os import iotests -import time from iotests import qemu_img +debug =3D False + disk_a =3D os.path.join(iotests.test_dir, 'disk_a') disk_b =3D os.path.join(iotests.test_dir, 'disk_b') size =3D '256G' fifo =3D os.path.join(iotests.test_dir, 'mig_fifo') +def event_seconds(event): + return event['timestamp']['seconds'] + \ + event['timestamp']['microseconds'] / 1000000.0 + + +def event_dist(e1, e2): + return event_seconds(e2) - event_seconds(e1) + + class TestDirtyBitmapPostcopyMigration(iotests.QMPTestCase): def tearDown(self): + if debug: + self.vm_a_events +=3D self.vm_a.get_qmp_events() + self.vm_b_events +=3D self.vm_b.get_qmp_events() + for e in self.vm_a_events: + e['vm'] =3D 'SRC' + for e in self.vm_b_events: + e['vm'] =3D 'DST' + events =3D (self.vm_a_events + self.vm_b_events) + events =3D [(e['timestamp']['seconds'], + e['timestamp']['microseconds'], + e['vm'], + e['event'], + e.get('data', '')) for e in events] + for e in sorted(events): + print('{}.{:06} {} {} {}'.format(*e)) + self.vm_a.shutdown() self.vm_b.shutdown() os.remove(disk_a) @@ -47,6 +73,10 @@ class TestDirtyBitmapPostcopyMigration(iotests.QMPTestCa= se): self.vm_a.launch() self.vm_b.launch() + # collect received events for debug + self.vm_a_events =3D [] + self.vm_b_events =3D [] + def test_postcopy(self): write_size =3D 0x40000000 granularity =3D 512 @@ -77,15 +107,13 @@ class TestDirtyBitmapPostcopyMigration(iotests.QMPTest= Case): self.vm_a.hmp_qemu_io('drive0', 'write %d %d' % (s, chunk)) s +=3D 0x10000 - bitmaps_cap =3D {'capability': 'dirty-bitmaps', 'state': True} - events_cap =3D {'capability': 'events', 'state': True} + caps =3D [{'capability': 'dirty-bitmaps', 'state': True}, + {'capability': 'events', 'state': True}] - result =3D self.vm_a.qmp('migrate-set-capabilities', - capabilities=3D[bitmaps_cap, events_cap]) + result =3D self.vm_a.qmp('migrate-set-capabilities', capabilities= =3Dcaps) self.assert_qmp(result, 'return', {}) - result =3D self.vm_b.qmp('migrate-set-capabilities', - capabilities=3D[bitmaps_cap]) + result =3D self.vm_b.qmp('migrate-set-capabilities', capabilities= =3Dcaps) self.assert_qmp(result, 'return', {}) result =3D self.vm_a.qmp('migrate', uri=3D'exec:cat>' + fifo) @@ -94,24 +122,38 @@ class TestDirtyBitmapPostcopyMigration(iotests.QMPTest= Case): result =3D self.vm_a.qmp('migrate-start-postcopy') self.assert_qmp(result, 'return', {}) - while True: - event =3D self.vm_a.event_wait('MIGRATION') - if event['data']['status'] =3D=3D 'completed': - break + event_resume =3D self.vm_b.event_wait('RESUME') + self.vm_b_events.append(event_resume) s =3D 0x8000 while s < write_size: self.vm_b.hmp_qemu_io('drive0', 'write %d %d' % (s, chunk)) s +=3D 0x10000 + match =3D {'data': {'status': 'completed'}} + event_complete =3D self.vm_b.event_wait('MIGRATION', match=3Dmatch) + self.vm_b_events.append(event_complete) + + # take queued event, should already been happened + event_stop =3D self.vm_a.event_wait('STOP') + self.vm_a_events.append(event_stop) + + downtime =3D event_dist(event_stop, event_resume) + postcopy_time =3D event_dist(event_resume, event_complete) + + # TODO: assert downtime * 10 < postcopy_time + if debug: + print('downtime:', downtime) + print('postcopy_time:', postcopy_time) + + # Assert that bitmap migration is finished (check that successor b= itmap + # is removed) result =3D self.vm_b.qmp('query-block') - while len(result['return'][0]['dirty-bitmaps']) > 1: - time.sleep(2) - result =3D self.vm_b.qmp('query-block') + assert len(result['return'][0]['dirty-bitmaps']) =3D=3D 1 + # Check content of migrated (and updated by new writes) bitmap result =3D self.vm_b.qmp('x-debug-block-dirty-bitmap-sha256', node=3D'drive0', name=3D'bitmap') - self.assert_qmp(result, 'return/sha256', sha256) --=20 2.27.0