From nobody Mon Feb 9 01:37:05 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1595883450; cv=none; d=zohomail.com; s=zohoarc; b=YF6JR2ol4uGP+WcbLWM30KLhIEY6Lm2dBojBvwBmg1dGimaQHC4k6U9XRiTp1uM73UwJy/vGztmVpDxPrIAmEZ4Hq3ir9YeLM8iqokVALLagbMRcH/FmCTatJs5FTDYXosTMFyhdl+ldZ2c95wMejuX+YrKjUDFYmaAKkctQVZY= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1595883450; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=O5NbTZAmm+WYBkXgq6rCd6kSo+mW8LZZy66ctqa9KBo=; b=JB+N3bIMKbyUAHFjEIhs+Bluqy1g6UypHUQ5lTgln2Q9u/aATlJKA6TxhYRFUoOzYZsei/5xaAPRwln5TCj2ZNBLTzydI0YWi7UctHOBJjmtP1zGtV7azrEYlUMeljY2XJVpUZWaqO4nCnNSGVeEfVEsq6qMv9BjwZRHOgFCxAU= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 159588345007117.986905789426032; Mon, 27 Jul 2020 13:57:30 -0700 (PDT) Received: from localhost ([::1]:42412 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1k0ABc-0006VG-4x for importer@patchew.org; Mon, 27 Jul 2020 16:57:28 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:34706) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1k0AA6-0004m7-Bs for qemu-devel@nongnu.org; Mon, 27 Jul 2020 16:55:54 -0400 Received: from us-smtp-delivery-1.mimecast.com ([205.139.110.120]:31103 helo=us-smtp-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1k0AA4-0004r6-5i for qemu-devel@nongnu.org; Mon, 27 Jul 2020 16:55:53 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-54-M3irSXQVMxyAj0FKcyUwdQ-1; Mon, 27 Jul 2020 16:55:48 -0400 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 7B81E186A82A; Mon, 27 Jul 2020 20:55:47 +0000 (UTC) Received: from blue.redhat.com (ovpn-118-248.rdu2.redhat.com [10.10.118.248]) by smtp.corp.redhat.com (Postfix) with ESMTP id 1CFAA19D82; Mon, 27 Jul 2020 20:55:46 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1595883351; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=O5NbTZAmm+WYBkXgq6rCd6kSo+mW8LZZy66ctqa9KBo=; b=SWFAU9Qle9DWaSxSxhW4Rw0+xf3buXnwtHvLyV6SCG+BBxG2HpA/TEsmdsdLO97fCcxtIP CfIPzkAuQZVLJlbVua5MMdnYQ7UxRtaey/EjSONf2j9jlz7m94EpS2RuDOMoqpA3LrkaFM 5NVag2/wCqOvLUBWOPWgpOcMfRZKamw= X-MC-Unique: M3irSXQVMxyAj0FKcyUwdQ-1 From: Eric Blake To: qemu-devel@nongnu.org Subject: [PULL 01/24] qcow2: Fix capitalization of header extension constant. Date: Mon, 27 Jul 2020 15:55:20 -0500 Message-Id: <20200727205543.206624-2-eblake@redhat.com> In-Reply-To: <20200727205543.206624-1-eblake@redhat.com> References: <20200727205543.206624-1-eblake@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.23 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=eblake@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=205.139.110.120; envelope-from=eblake@redhat.com; helo=us-smtp-1.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/07/27 01:44:14 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Spam_score_int: -40 X-Spam_score: -4.1 X-Spam_bar: ---- X-Spam_report: (-4.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-1, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-1, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Andrey Shinkevich , Vladimir Sementsov-Ogievskiy , "open list:qcow2" , Max Reitz Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) Content-Type: text/plain; charset="utf-8" From: Andrey Shinkevich Make the capitalization of the hexadecimal numbers consistent for the QCOW2 header extension constants in docs/interop/qcow2.txt. Suggested-by: Eric Blake Signed-off-by: Andrey Shinkevich Reviewed-by: Vladimir Sementsov-Ogievskiy Message-Id: <1594973699-781898-2-git-send-email-andrey.shinkevich@virtuozzo= .com> Reviewed-by: Eric Blake Signed-off-by: Eric Blake --- docs/interop/qcow2.txt | 2 +- block/qcow2.c | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/interop/qcow2.txt b/docs/interop/qcow2.txt index cb723463f241..f072e27900e6 100644 --- a/docs/interop/qcow2.txt +++ b/docs/interop/qcow2.txt @@ -231,7 +231,7 @@ be stored. Each extension has a structure like the foll= owing: Byte 0 - 3: Header extension type: 0x00000000 - End of the header extension area - 0xE2792ACA - Backing file format name string + 0xe2792aca - Backing file format name string 0x6803f857 - Feature name table 0x23852875 - Bitmaps extension 0x0537be77 - Full disk encryption header pointer diff --git a/block/qcow2.c b/block/qcow2.c index fadf3422f8c5..6ad6bdc166ea 100644 --- a/block/qcow2.c +++ b/block/qcow2.c @@ -66,7 +66,7 @@ typedef struct { } QEMU_PACKED QCowExtension; #define QCOW2_EXT_MAGIC_END 0 -#define QCOW2_EXT_MAGIC_BACKING_FORMAT 0xE2792ACA +#define QCOW2_EXT_MAGIC_BACKING_FORMAT 0xe2792aca #define QCOW2_EXT_MAGIC_FEATURE_TABLE 0x6803f857 #define QCOW2_EXT_MAGIC_CRYPTO_HEADER 0x0537be77 #define QCOW2_EXT_MAGIC_BITMAPS 0x23852875 --=20 2.27.0 From nobody Mon Feb 9 01:37:05 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1595883565; cv=none; d=zohomail.com; s=zohoarc; b=PSRrOjaR0LSPg5Ygh0obdh9eKFPKT6cdvUAOIeujT6y3hLUqKIzBZi54xRVq7LJBuz5j9UUw24773dEZ3u1wEA4/Ipcq2LgSkdLbDE6OO85fsbGQHbd24915kvs5XVvwNu4B311/LZmKpuozOeVJS+etfcZzYQVqy+aVDswBsdw= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1595883565; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=8aZEQrwMTXCA2sgozS27dagsM0wNVpw9yMUeiPgdBQY=; b=hVZ0t8Dn4mYuntLCL10+tNvxxHoFNzKROFcPc4ruI5bnc9maqilIfa35IXmQ20Hinh8FbSrm8Fpp1vZE9YJF03/KQZpcXaheXn2UY9FJcjJ0dlXP/p/z4QRkg3H+LDDC0oIeYtVd85PdBigDyuqlOZaRbj4nL/6Ye9+DBwklN9M= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1595883565889380.59943199685324; Mon, 27 Jul 2020 13:59:25 -0700 (PDT) Received: from localhost ([::1]:51028 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1k0ADU-0001ZX-IG for importer@patchew.org; Mon, 27 Jul 2020 16:59:24 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:34760) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1k0AAA-0004sG-VT for qemu-devel@nongnu.org; Mon, 27 Jul 2020 16:55:58 -0400 Received: from us-smtp-delivery-1.mimecast.com ([207.211.31.120]:35022 helo=us-smtp-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1k0AA9-0004rh-G0 for qemu-devel@nongnu.org; Mon, 27 Jul 2020 16:55:58 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-407-NPq5r6PYNhOPOzUbWEwtfA-1; Mon, 27 Jul 2020 16:55:52 -0400 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 8502E79ECB; Mon, 27 Jul 2020 20:55:51 +0000 (UTC) Received: from blue.redhat.com (ovpn-118-248.rdu2.redhat.com [10.10.118.248]) by smtp.corp.redhat.com (Postfix) with ESMTP id C048C19724; Mon, 27 Jul 2020 20:55:47 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1595883356; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=8aZEQrwMTXCA2sgozS27dagsM0wNVpw9yMUeiPgdBQY=; b=gZxXJsgHnNlBe2xI3bw5Dtda1YGoEJuvJeuWb8KLtck3HAXSiodsVp3aloiWGbM5VD5f0k 5KEuaTTfLMGc7gel9qd5jWJqoGjEusDClc/zKvEss5RbTJPFyOIcHSGBf46UEfkMHezRI0 VlqAVXbSC3WGskhOdFlMy/qMr4XL/No= X-MC-Unique: NPq5r6PYNhOPOzUbWEwtfA-1 From: Eric Blake To: qemu-devel@nongnu.org Subject: [PULL 02/24] qemu-iotests/199: fix style Date: Mon, 27 Jul 2020 15:55:21 -0500 Message-Id: <20200727205543.206624-3-eblake@redhat.com> In-Reply-To: <20200727205543.206624-1-eblake@redhat.com> References: <20200727205543.206624-1-eblake@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.23 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=207.211.31.120; envelope-from=eblake@redhat.com; helo=us-smtp-1.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/07/27 15:02:33 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Spam_score_int: -30 X-Spam_score: -3.1 X-Spam_bar: --- X-Spam_report: (-3.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-1, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Andrey Shinkevich , Vladimir Sementsov-Ogievskiy , "open list:Block layer core" , Max Reitz Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) Content-Type: text/plain; charset="utf-8" From: Vladimir Sementsov-Ogievskiy Mostly, satisfy pep8 complaints. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Andrey Shinkevich Tested-by: Eric Blake Message-Id: <20200727194236.19551-2-vsementsov@virtuozzo.com> Signed-off-by: Eric Blake --- tests/qemu-iotests/199 | 13 +++++++------ 1 file changed, 7 insertions(+), 6 deletions(-) diff --git a/tests/qemu-iotests/199 b/tests/qemu-iotests/199 index 40774eed74c2..de9ba8d94c23 100755 --- a/tests/qemu-iotests/199 +++ b/tests/qemu-iotests/199 @@ -28,8 +28,8 @@ disk_b =3D os.path.join(iotests.test_dir, 'disk_b') size =3D '256G' fifo =3D os.path.join(iotests.test_dir, 'mig_fifo') + class TestDirtyBitmapPostcopyMigration(iotests.QMPTestCase): - def tearDown(self): self.vm_a.shutdown() self.vm_b.shutdown() @@ -54,7 +54,7 @@ class TestDirtyBitmapPostcopyMigration(iotests.QMPTestCas= e): result =3D self.vm_a.qmp('block-dirty-bitmap-add', node=3D'drive0', name=3D'bitmap', granularity=3Dgranularity) - self.assert_qmp(result, 'return', {}); + self.assert_qmp(result, 'return', {}) s =3D 0 while s < write_size: @@ -71,7 +71,7 @@ class TestDirtyBitmapPostcopyMigration(iotests.QMPTestCas= e): result =3D self.vm_a.qmp('block-dirty-bitmap-clear', node=3D'drive= 0', name=3D'bitmap') - self.assert_qmp(result, 'return', {}); + self.assert_qmp(result, 'return', {}) s =3D 0 while s < write_size: self.vm_a.hmp_qemu_io('drive0', 'write %d %d' % (s, chunk)) @@ -104,15 +104,16 @@ class TestDirtyBitmapPostcopyMigration(iotests.QMPTes= tCase): self.vm_b.hmp_qemu_io('drive0', 'write %d %d' % (s, chunk)) s +=3D 0x10000 - result =3D self.vm_b.qmp('query-block'); + result =3D self.vm_b.qmp('query-block') while len(result['return'][0]['dirty-bitmaps']) > 1: time.sleep(2) - result =3D self.vm_b.qmp('query-block'); + result =3D self.vm_b.qmp('query-block') result =3D self.vm_b.qmp('x-debug-block-dirty-bitmap-sha256', node=3D'drive0', name=3D'bitmap') - self.assert_qmp(result, 'return/sha256', sha256); + self.assert_qmp(result, 'return/sha256', sha256) + if __name__ =3D=3D '__main__': iotests.main(supported_fmts=3D['qcow2'], supported_cache_modes=3D['non= e'], --=20 2.27.0 From nobody Mon Feb 9 01:37:05 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1595883473; cv=none; d=zohomail.com; s=zohoarc; b=HbhZKAmjo+N2DzYuWgFKX76Mp2sH0BxOr/lZoxyWxpKghOVHmFzNntrq3Sojq5qZYdnEG7ZTyvQzPWthpdG8c3BrPrZvw+W9FwBQ8klwv4WTEIo1AmEWFaNYtB9kcACKuOsPUfGrfxZAXHAmNlL/k6NeTmPZEHlGKUwxNDx0aVE= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1595883473; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=AgJoIRjnTNwCL5kqvJULBm/tZVF2+MW/dtpEorcGLAQ=; b=hoTGRESrw07uq5M/N0tl1frpjM+ZWYByPnmX7R5vYJwMflJFwbeLp3qkLfEww7y2gtST5nMpyBV9eIuGxkfXdbXlc93iMi07oUdGWTnZbk18nWLz00HsOPIqYdRqPmyWet9FEW7GFLxL/MwbusQMTuroZU31xqZqNgp2wK5QL5k= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1595883473256857.2959258516751; Mon, 27 Jul 2020 13:57:53 -0700 (PDT) Received: from localhost ([::1]:43520 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1k0ABz-0006xi-Uc for importer@patchew.org; Mon, 27 Jul 2020 16:57:51 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:34756) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1k0AAA-0004ry-PU for qemu-devel@nongnu.org; Mon, 27 Jul 2020 16:55:58 -0400 Received: from us-smtp-delivery-1.mimecast.com ([207.211.31.120]:56425 helo=us-smtp-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1k0AA9-0004rb-3t for qemu-devel@nongnu.org; Mon, 27 Jul 2020 16:55:58 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-447-QRd5DpwJOBOv9ILsjvUCjw-1; Mon, 27 Jul 2020 16:55:53 -0400 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id A90E4186A82A; Mon, 27 Jul 2020 20:55:52 +0000 (UTC) Received: from blue.redhat.com (ovpn-118-248.rdu2.redhat.com [10.10.118.248]) by smtp.corp.redhat.com (Postfix) with ESMTP id C698119D82; Mon, 27 Jul 2020 20:55:51 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1595883356; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=AgJoIRjnTNwCL5kqvJULBm/tZVF2+MW/dtpEorcGLAQ=; b=IUXqpYKeECTiv4wyIXe6AyIksuqyBFSM/WjBhJYKFL4DhnWzHiWQquBzcTZyIWEoOQStrN fk/5ppxuhTTgSR2LBvke2ELt0s+6D7GfsnfAYDp3oZgqdG3H7wUNCRbU55Z4/IJOiws3f/ Bld4Uov9ZblEV2NSBCnzYbgIDr8+70E= X-MC-Unique: QRd5DpwJOBOv9ILsjvUCjw-1 From: Eric Blake To: qemu-devel@nongnu.org Subject: [PULL 03/24] qemu-iotests/199: drop extra constraints Date: Mon, 27 Jul 2020 15:55:22 -0500 Message-Id: <20200727205543.206624-4-eblake@redhat.com> In-Reply-To: <20200727205543.206624-1-eblake@redhat.com> References: <20200727205543.206624-1-eblake@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.23 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=207.211.31.120; envelope-from=eblake@redhat.com; helo=us-smtp-1.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/07/27 15:02:33 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Spam_score_int: -30 X-Spam_score: -3.1 X-Spam_bar: --- X-Spam_report: (-3.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-1, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Andrey Shinkevich , Vladimir Sementsov-Ogievskiy , "open list:Block layer core" , Max Reitz Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) Content-Type: text/plain; charset="utf-8" From: Vladimir Sementsov-Ogievskiy We don't need any specific format constraints here. Still keep qcow2 for two reasons: 1. No extra calls of format-unrelated test 2. Add some check around persistent bitmap in future (require qcow2) Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Andrey Shinkevich Tested-by: Eric Blake Message-Id: <20200727194236.19551-3-vsementsov@virtuozzo.com> Signed-off-by: Eric Blake --- tests/qemu-iotests/199 | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/tests/qemu-iotests/199 b/tests/qemu-iotests/199 index de9ba8d94c23..dda918450a8b 100755 --- a/tests/qemu-iotests/199 +++ b/tests/qemu-iotests/199 @@ -116,5 +116,4 @@ class TestDirtyBitmapPostcopyMigration(iotests.QMPTestC= ase): if __name__ =3D=3D '__main__': - iotests.main(supported_fmts=3D['qcow2'], supported_cache_modes=3D['non= e'], - supported_protocols=3D['file']) + iotests.main(supported_fmts=3D['qcow2']) --=20 2.27.0 From nobody Mon Feb 9 01:37:05 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1595883466; cv=none; d=zohomail.com; s=zohoarc; b=VhmlF9EVPqaEsjAbuq+Du9KuHJ6f7KjYtGScS/4ZrSKuJ15RK1Y+Wfl6Lqs+R5Ni9CrB19oMmCJesR/bG7C6q3EyCZZBIKniUIMXG+64SoHs8+GIdCHNupHkA23QzBW0yu2u6pSeKBlKa5WihbFuvvw4cDf/RhTNIUMgAexFHDI= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1595883466; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=hOJpc8RV8qvMz8aH/tH1tmr2hdSTKc/5P7btzVhxFp0=; b=iu9qX2hdS/xeIW3BXGw5rKMMj/tyap09CnKGmX+eUMKcNLOB+uI7XIPgxWweyRAP8BqmTHiJppcflNLXWHhxzDYfvHjF9tnKPKm4o+tjTKynMynkggxwuwvfPTO1HXGIWTP5wKyiFzTmzl8JyI97d9M2gmPOOijD7mUEO1eTlc4= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1595883466120599.9281703039179; Mon, 27 Jul 2020 13:57:46 -0700 (PDT) Received: from localhost ([::1]:43202 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1k0ABs-0006pn-Pa for importer@patchew.org; Mon, 27 Jul 2020 16:57:44 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:34792) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1k0AAC-0004v4-C9 for qemu-devel@nongnu.org; Mon, 27 Jul 2020 16:56:00 -0400 Received: from us-smtp-1.mimecast.com ([207.211.31.81]:47810 helo=us-smtp-delivery-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1k0AAA-0004rn-FU for qemu-devel@nongnu.org; Mon, 27 Jul 2020 16:56:00 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-429-dMQax1MiM5uFCizsQ1mqMQ-1; Mon, 27 Jul 2020 16:55:55 -0400 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id D2DC38E1361; Mon, 27 Jul 2020 20:55:53 +0000 (UTC) Received: from blue.redhat.com (ovpn-118-248.rdu2.redhat.com [10.10.118.248]) by smtp.corp.redhat.com (Postfix) with ESMTP id EB39D19724; Mon, 27 Jul 2020 20:55:52 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1595883357; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=hOJpc8RV8qvMz8aH/tH1tmr2hdSTKc/5P7btzVhxFp0=; b=OpnSgIfBjvnb2/VkzC0lC7w9ArxMFI4BBJeFhy2pNgAy+zn+bsXoi1hpp4HtvhXSlwSzGq yLlle2h9BEBKJhJX7xb5nIZxul6ALClJGZlMy6uDCQTiq7BlCOuIH852qmJgCUTuMQtyQT erSLi3aHCZIsKEERk3TUgwt9KrF6Ow4= X-MC-Unique: dMQax1MiM5uFCizsQ1mqMQ-1 From: Eric Blake To: qemu-devel@nongnu.org Subject: [PULL 04/24] qemu-iotests/199: better catch postcopy time Date: Mon, 27 Jul 2020 15:55:23 -0500 Message-Id: <20200727205543.206624-5-eblake@redhat.com> In-Reply-To: <20200727205543.206624-1-eblake@redhat.com> References: <20200727205543.206624-1-eblake@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.23 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=207.211.31.81; envelope-from=eblake@redhat.com; helo=us-smtp-delivery-1.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/07/27 03:37:14 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Spam_score_int: -30 X-Spam_score: -3.1 X-Spam_bar: --- X-Spam_report: (-3.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-1, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Andrey Shinkevich , Vladimir Sementsov-Ogievskiy , "open list:Block layer core" , Max Reitz Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) Content-Type: text/plain; charset="utf-8" From: Vladimir Sementsov-Ogievskiy The test aims to test _postcopy_ migration, and wants to do some write operations during postcopy time. Test considers migrate status=3Dcomplete event on source as start of postcopy. This is completely wrong, completion is completion of the whole migration process. Let's instead consider destination start as start of postcopy, and use RESUME event for it. Next, as migration finish, let's use migration status=3Dcomplete event on target, as such method is closer to what libvirt or another user will do, than tracking number of dirty-bitmaps. Finally, add a possibility to dump events for debug. And if set debug to True, we see, that actual postcopy period is very small relatively to the whole test duration time (~0.2 seconds to >40 seconds for me). This means, that test is very inefficient in what it supposed to do. Let's improve it in following commits. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Andrey Shinkevich Tested-by: Eric Blake Message-Id: <20200727194236.19551-4-vsementsov@virtuozzo.com> Signed-off-by: Eric Blake --- tests/qemu-iotests/199 | 72 +++++++++++++++++++++++++++++++++--------- 1 file changed, 57 insertions(+), 15 deletions(-) diff --git a/tests/qemu-iotests/199 b/tests/qemu-iotests/199 index dda918450a8b..dd6044768c76 100755 --- a/tests/qemu-iotests/199 +++ b/tests/qemu-iotests/199 @@ -20,17 +20,43 @@ import os import iotests -import time from iotests import qemu_img +debug =3D False + disk_a =3D os.path.join(iotests.test_dir, 'disk_a') disk_b =3D os.path.join(iotests.test_dir, 'disk_b') size =3D '256G' fifo =3D os.path.join(iotests.test_dir, 'mig_fifo') +def event_seconds(event): + return event['timestamp']['seconds'] + \ + event['timestamp']['microseconds'] / 1000000.0 + + +def event_dist(e1, e2): + return event_seconds(e2) - event_seconds(e1) + + class TestDirtyBitmapPostcopyMigration(iotests.QMPTestCase): def tearDown(self): + if debug: + self.vm_a_events +=3D self.vm_a.get_qmp_events() + self.vm_b_events +=3D self.vm_b.get_qmp_events() + for e in self.vm_a_events: + e['vm'] =3D 'SRC' + for e in self.vm_b_events: + e['vm'] =3D 'DST' + events =3D (self.vm_a_events + self.vm_b_events) + events =3D [(e['timestamp']['seconds'], + e['timestamp']['microseconds'], + e['vm'], + e['event'], + e.get('data', '')) for e in events] + for e in sorted(events): + print('{}.{:06} {} {} {}'.format(*e)) + self.vm_a.shutdown() self.vm_b.shutdown() os.remove(disk_a) @@ -47,6 +73,10 @@ class TestDirtyBitmapPostcopyMigration(iotests.QMPTestCa= se): self.vm_a.launch() self.vm_b.launch() + # collect received events for debug + self.vm_a_events =3D [] + self.vm_b_events =3D [] + def test_postcopy(self): write_size =3D 0x40000000 granularity =3D 512 @@ -77,15 +107,13 @@ class TestDirtyBitmapPostcopyMigration(iotests.QMPTest= Case): self.vm_a.hmp_qemu_io('drive0', 'write %d %d' % (s, chunk)) s +=3D 0x10000 - bitmaps_cap =3D {'capability': 'dirty-bitmaps', 'state': True} - events_cap =3D {'capability': 'events', 'state': True} + caps =3D [{'capability': 'dirty-bitmaps', 'state': True}, + {'capability': 'events', 'state': True}] - result =3D self.vm_a.qmp('migrate-set-capabilities', - capabilities=3D[bitmaps_cap, events_cap]) + result =3D self.vm_a.qmp('migrate-set-capabilities', capabilities= =3Dcaps) self.assert_qmp(result, 'return', {}) - result =3D self.vm_b.qmp('migrate-set-capabilities', - capabilities=3D[bitmaps_cap]) + result =3D self.vm_b.qmp('migrate-set-capabilities', capabilities= =3Dcaps) self.assert_qmp(result, 'return', {}) result =3D self.vm_a.qmp('migrate', uri=3D'exec:cat>' + fifo) @@ -94,24 +122,38 @@ class TestDirtyBitmapPostcopyMigration(iotests.QMPTest= Case): result =3D self.vm_a.qmp('migrate-start-postcopy') self.assert_qmp(result, 'return', {}) - while True: - event =3D self.vm_a.event_wait('MIGRATION') - if event['data']['status'] =3D=3D 'completed': - break + event_resume =3D self.vm_b.event_wait('RESUME') + self.vm_b_events.append(event_resume) s =3D 0x8000 while s < write_size: self.vm_b.hmp_qemu_io('drive0', 'write %d %d' % (s, chunk)) s +=3D 0x10000 + match =3D {'data': {'status': 'completed'}} + event_complete =3D self.vm_b.event_wait('MIGRATION', match=3Dmatch) + self.vm_b_events.append(event_complete) + + # take queued event, should already been happened + event_stop =3D self.vm_a.event_wait('STOP') + self.vm_a_events.append(event_stop) + + downtime =3D event_dist(event_stop, event_resume) + postcopy_time =3D event_dist(event_resume, event_complete) + + # TODO: assert downtime * 10 < postcopy_time + if debug: + print('downtime:', downtime) + print('postcopy_time:', postcopy_time) + + # Assert that bitmap migration is finished (check that successor b= itmap + # is removed) result =3D self.vm_b.qmp('query-block') - while len(result['return'][0]['dirty-bitmaps']) > 1: - time.sleep(2) - result =3D self.vm_b.qmp('query-block') + assert len(result['return'][0]['dirty-bitmaps']) =3D=3D 1 + # Check content of migrated (and updated by new writes) bitmap result =3D self.vm_b.qmp('x-debug-block-dirty-bitmap-sha256', node=3D'drive0', name=3D'bitmap') - self.assert_qmp(result, 'return/sha256', sha256) --=20 2.27.0 From nobody Mon Feb 9 01:37:05 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1595883579; cv=none; d=zohomail.com; s=zohoarc; b=Ri7WLu+jYx7iER0Izzzfct5NCqVSlOnLUSz3GulwHnergh1LpKx9Ru8rYUIbMagZx9Y8deBaV8OgHzyGWWxnzKYJnxJcVCe7kQUZCWqi/syEo5Aw7o5dPlvG908xNs+aWaiiXR1KlDfMABokpNJC2Vb1Rj2HWWyhf+UegYyy5Nc= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1595883579; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=c88SJ1ynOV3k49UvCPQbS7iGEmD+ITDXGccJB9WpNSQ=; b=jvXKDtoHDIF47H8egjYklo5PdB9fhehfvV9LZ3KLeExim6iTHMHJVZNi4xvA/FclPInKc7LrPVrAtkboWwq7fb8Yl+Tw5AsM5WU7sZxNVpjBrU1G5+ESfiwkxL5NxxN+7c0fzxQh0GhQ+bIj7cL9xBf0OeU0cN9hrjaafoCV+W8= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1595883579736687.8785708951758; Mon, 27 Jul 2020 13:59:39 -0700 (PDT) Received: from localhost ([::1]:52088 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1k0ADi-00020j-Jm for importer@patchew.org; Mon, 27 Jul 2020 16:59:38 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:34818) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1k0AAE-00050s-QQ for qemu-devel@nongnu.org; Mon, 27 Jul 2020 16:56:02 -0400 Received: from us-smtp-1.mimecast.com ([205.139.110.61]:27087 helo=us-smtp-delivery-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1k0AAD-0004sd-5P for qemu-devel@nongnu.org; Mon, 27 Jul 2020 16:56:02 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-264-crgLTz4DNI-hhyJ0GgrmVA-1; Mon, 27 Jul 2020 16:55:56 -0400 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 0236F8015F7; Mon, 27 Jul 2020 20:55:55 +0000 (UTC) Received: from blue.redhat.com (ovpn-118-248.rdu2.redhat.com [10.10.118.248]) by smtp.corp.redhat.com (Postfix) with ESMTP id 1EFC719945; Mon, 27 Jul 2020 20:55:54 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1595883360; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=c88SJ1ynOV3k49UvCPQbS7iGEmD+ITDXGccJB9WpNSQ=; b=Tig4AZJ1kvZClzoZfdRCg7kTgsRf4a6Cf725/OIpetg9C1tWv+U4z3yFV0buWd9lMUaaqx qFx4AcCTZ+KP9MOioUM2//Q3LKNafCzv+BrAinitXtt+zDF1un0TW76iVbEATlhpcnuScB hiiC7A0FXm35wK0GQaI5niq3qOqwAuQ= X-MC-Unique: crgLTz4DNI-hhyJ0GgrmVA-1 From: Eric Blake To: qemu-devel@nongnu.org Subject: [PULL 05/24] qemu-iotests/199: improve performance: set bitmap by discard Date: Mon, 27 Jul 2020 15:55:24 -0500 Message-Id: <20200727205543.206624-6-eblake@redhat.com> In-Reply-To: <20200727205543.206624-1-eblake@redhat.com> References: <20200727205543.206624-1-eblake@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.23 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=205.139.110.61; envelope-from=eblake@redhat.com; helo=us-smtp-delivery-1.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/07/27 01:46:13 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Spam_score_int: -40 X-Spam_score: -4.1 X-Spam_bar: ---- X-Spam_report: (-4.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-1, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-1, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Andrey Shinkevich , Vladimir Sementsov-Ogievskiy , "open list:Block layer core" , Max Reitz Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) Content-Type: text/plain; charset="utf-8" From: Vladimir Sementsov-Ogievskiy Discard dirties dirty-bitmap as well as write, but works faster. Let's use it instead. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Andrey Shinkevich Tested-by: Eric Blake Message-Id: <20200727194236.19551-5-vsementsov@virtuozzo.com> Signed-off-by: Eric Blake --- tests/qemu-iotests/199 | 31 ++++++++++++++++++++----------- 1 file changed, 20 insertions(+), 11 deletions(-) diff --git a/tests/qemu-iotests/199 b/tests/qemu-iotests/199 index dd6044768c76..190e820b8408 100755 --- a/tests/qemu-iotests/199 +++ b/tests/qemu-iotests/199 @@ -67,8 +67,10 @@ class TestDirtyBitmapPostcopyMigration(iotests.QMPTestCa= se): os.mkfifo(fifo) qemu_img('create', '-f', iotests.imgfmt, disk_a, size) qemu_img('create', '-f', iotests.imgfmt, disk_b, size) - self.vm_a =3D iotests.VM(path_suffix=3D'a').add_drive(disk_a) - self.vm_b =3D iotests.VM(path_suffix=3D'b').add_drive(disk_b) + self.vm_a =3D iotests.VM(path_suffix=3D'a').add_drive(disk_a, + 'discard=3Dunmap= ') + self.vm_b =3D iotests.VM(path_suffix=3D'b').add_drive(disk_b, + 'discard=3Dunmap= ') self.vm_b.add_incoming("exec: cat '" + fifo + "'") self.vm_a.launch() self.vm_b.launch() @@ -78,7 +80,7 @@ class TestDirtyBitmapPostcopyMigration(iotests.QMPTestCas= e): self.vm_b_events =3D [] def test_postcopy(self): - write_size =3D 0x40000000 + discard_size =3D 0x40000000 granularity =3D 512 chunk =3D 4096 @@ -86,25 +88,32 @@ class TestDirtyBitmapPostcopyMigration(iotests.QMPTestC= ase): name=3D'bitmap', granularity=3Dgranularity) self.assert_qmp(result, 'return', {}) + result =3D self.vm_a.qmp('x-debug-block-dirty-bitmap-sha256', + node=3D'drive0', name=3D'bitmap') + empty_sha256 =3D result['return']['sha256'] + s =3D 0 - while s < write_size: - self.vm_a.hmp_qemu_io('drive0', 'write %d %d' % (s, chunk)) + while s < discard_size: + self.vm_a.hmp_qemu_io('drive0', 'discard %d %d' % (s, chunk)) s +=3D 0x10000 s =3D 0x8000 - while s < write_size: - self.vm_a.hmp_qemu_io('drive0', 'write %d %d' % (s, chunk)) + while s < discard_size: + self.vm_a.hmp_qemu_io('drive0', 'discard %d %d' % (s, chunk)) s +=3D 0x10000 result =3D self.vm_a.qmp('x-debug-block-dirty-bitmap-sha256', node=3D'drive0', name=3D'bitmap') sha256 =3D result['return']['sha256'] + # Check, that updating the bitmap by discards works + assert sha256 !=3D empty_sha256 + result =3D self.vm_a.qmp('block-dirty-bitmap-clear', node=3D'drive= 0', name=3D'bitmap') self.assert_qmp(result, 'return', {}) s =3D 0 - while s < write_size: - self.vm_a.hmp_qemu_io('drive0', 'write %d %d' % (s, chunk)) + while s < discard_size: + self.vm_a.hmp_qemu_io('drive0', 'discard %d %d' % (s, chunk)) s +=3D 0x10000 caps =3D [{'capability': 'dirty-bitmaps', 'state': True}, @@ -126,8 +135,8 @@ class TestDirtyBitmapPostcopyMigration(iotests.QMPTestC= ase): self.vm_b_events.append(event_resume) s =3D 0x8000 - while s < write_size: - self.vm_b.hmp_qemu_io('drive0', 'write %d %d' % (s, chunk)) + while s < discard_size: + self.vm_b.hmp_qemu_io('drive0', 'discard %d %d' % (s, chunk)) s +=3D 0x10000 match =3D {'data': {'status': 'completed'}} --=20 2.27.0 From nobody Mon Feb 9 01:37:05 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1595883730; cv=none; d=zohomail.com; s=zohoarc; b=iOIUBOwSVrg7BnSIlscgfg4CNTBe0zx+8Bdb5ITPub8BLfXxuhIP2azGpdcW1LP+M9WL7wafzHs+GO+B+mDVKmz/kzgzDp9ceNWprRv6x+VuxcTwMCCTj5NMuiEECr3P9ykaqbHt4ENwCPHls9aHO0/psFwi9CSOiA9DG8ZxJ1Q= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1595883730; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=PD0Ql2SFu37Jc2pUAqddf4cWEviu9Iqud0o+66za5yI=; b=FlHmeSrnBPOLD1HrlopVXQv4xG3z3rUQ8/wZCJ1djadRw/5x1BJRbb4iIjQ/BGqhLbFw+pCgSD58WvCVj+9YLoWJW6h2IO/tFE47+bO6rnjbPBi2EZExxK88z6ff7mgMjkC6JJ+H4BOj4Btv/XFJ9rr5JcZ94slK4kHpSf1FCpo= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1595883730372695.9893040263872; Mon, 27 Jul 2020 14:02:10 -0700 (PDT) Received: from localhost ([::1]:60534 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1k0AG8-0005ab-VJ for importer@patchew.org; Mon, 27 Jul 2020 17:02:09 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:34850) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1k0AAH-00057h-IA for qemu-devel@nongnu.org; Mon, 27 Jul 2020 16:56:05 -0400 Received: from us-smtp-2.mimecast.com ([205.139.110.61]:44244 helo=us-smtp-delivery-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1k0AAF-0004u7-75 for qemu-devel@nongnu.org; Mon, 27 Jul 2020 16:56:05 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-388-ZtPzh8rcNJmqRZy6kXoLAw-1; Mon, 27 Jul 2020 16:56:00 -0400 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id CD17558; Mon, 27 Jul 2020 20:55:58 +0000 (UTC) Received: from blue.redhat.com (ovpn-118-248.rdu2.redhat.com [10.10.118.248]) by smtp.corp.redhat.com (Postfix) with ESMTP id 4832319724; Mon, 27 Jul 2020 20:55:55 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1595883362; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=PD0Ql2SFu37Jc2pUAqddf4cWEviu9Iqud0o+66za5yI=; b=VhUpvir99RNFAIcd0V/anQ0K4vwB8h2qmHzcAmtSwSDlP9f8HrTuijRSdOz1sL0FcgqV6i l30zbZpLIA5KDdG5ddPdyAtGQ+hgd4LAwlxEQB6cdqDdRz5KnXnEfavvZ+WEks9dIKwYgT DOWyfnrsJ1FdD82iLaqYYRhHj/PX5mk= X-MC-Unique: ZtPzh8rcNJmqRZy6kXoLAw-1 From: Eric Blake To: qemu-devel@nongnu.org Subject: [PULL 06/24] qemu-iotests/199: change discard patterns Date: Mon, 27 Jul 2020 15:55:25 -0500 Message-Id: <20200727205543.206624-7-eblake@redhat.com> In-Reply-To: <20200727205543.206624-1-eblake@redhat.com> References: <20200727205543.206624-1-eblake@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.23 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=205.139.110.61; envelope-from=eblake@redhat.com; helo=us-smtp-delivery-1.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/07/27 01:46:13 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Spam_score_int: -40 X-Spam_score: -4.1 X-Spam_bar: ---- X-Spam_report: (-4.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-1, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-1, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Andrey Shinkevich , Vladimir Sementsov-Ogievskiy , "open list:Block layer core" , Max Reitz Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) Content-Type: text/plain; charset="utf-8" From: Vladimir Sementsov-Ogievskiy iotest 199 works too long because of many discard operations. At the same time, postcopy period is very short, in spite of all these efforts. So, let's use less discards (and with more interesting patterns) to reduce test timing. In the next commit we'll increase postcopy period. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Andrey Shinkevich Tested-by: Eric Blake Message-Id: <20200727194236.19551-6-vsementsov@virtuozzo.com> Signed-off-by: Eric Blake --- tests/qemu-iotests/199 | 44 +++++++++++++++++++++++++----------------- 1 file changed, 26 insertions(+), 18 deletions(-) diff --git a/tests/qemu-iotests/199 b/tests/qemu-iotests/199 index 190e820b8408..da4dae01fb5d 100755 --- a/tests/qemu-iotests/199 +++ b/tests/qemu-iotests/199 @@ -30,6 +30,28 @@ size =3D '256G' fifo =3D os.path.join(iotests.test_dir, 'mig_fifo') +GiB =3D 1024 * 1024 * 1024 + +discards1 =3D ( + (0, GiB), + (2 * GiB + 512 * 5, 512), + (3 * GiB + 512 * 5, 512), + (100 * GiB, GiB) +) + +discards2 =3D ( + (3 * GiB + 512 * 8, 512), + (4 * GiB + 512 * 8, 512), + (50 * GiB, GiB), + (100 * GiB + GiB // 2, GiB) +) + + +def apply_discards(vm, discards): + for d in discards: + vm.hmp_qemu_io('drive0', 'discard {} {}'.format(*d)) + + def event_seconds(event): return event['timestamp']['seconds'] + \ event['timestamp']['microseconds'] / 1000000.0 @@ -80,9 +102,7 @@ class TestDirtyBitmapPostcopyMigration(iotests.QMPTestCa= se): self.vm_b_events =3D [] def test_postcopy(self): - discard_size =3D 0x40000000 granularity =3D 512 - chunk =3D 4096 result =3D self.vm_a.qmp('block-dirty-bitmap-add', node=3D'drive0', name=3D'bitmap', granularity=3Dgranularity) @@ -92,14 +112,7 @@ class TestDirtyBitmapPostcopyMigration(iotests.QMPTestC= ase): node=3D'drive0', name=3D'bitmap') empty_sha256 =3D result['return']['sha256'] - s =3D 0 - while s < discard_size: - self.vm_a.hmp_qemu_io('drive0', 'discard %d %d' % (s, chunk)) - s +=3D 0x10000 - s =3D 0x8000 - while s < discard_size: - self.vm_a.hmp_qemu_io('drive0', 'discard %d %d' % (s, chunk)) - s +=3D 0x10000 + apply_discards(self.vm_a, discards1 + discards2) result =3D self.vm_a.qmp('x-debug-block-dirty-bitmap-sha256', node=3D'drive0', name=3D'bitmap') @@ -111,10 +124,8 @@ class TestDirtyBitmapPostcopyMigration(iotests.QMPTest= Case): result =3D self.vm_a.qmp('block-dirty-bitmap-clear', node=3D'drive= 0', name=3D'bitmap') self.assert_qmp(result, 'return', {}) - s =3D 0 - while s < discard_size: - self.vm_a.hmp_qemu_io('drive0', 'discard %d %d' % (s, chunk)) - s +=3D 0x10000 + + apply_discards(self.vm_a, discards1) caps =3D [{'capability': 'dirty-bitmaps', 'state': True}, {'capability': 'events', 'state': True}] @@ -134,10 +145,7 @@ class TestDirtyBitmapPostcopyMigration(iotests.QMPTest= Case): event_resume =3D self.vm_b.event_wait('RESUME') self.vm_b_events.append(event_resume) - s =3D 0x8000 - while s < discard_size: - self.vm_b.hmp_qemu_io('drive0', 'discard %d %d' % (s, chunk)) - s +=3D 0x10000 + apply_discards(self.vm_b, discards2) match =3D {'data': {'status': 'completed'}} event_complete =3D self.vm_b.event_wait('MIGRATION', match=3Dmatch) --=20 2.27.0 From nobody Mon Feb 9 01:37:05 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1595883595; cv=none; d=zohomail.com; s=zohoarc; b=aarNrEbsziE8fBG/qQ5OLcuIurFPQDKczz7h9DLbt4Z9G1U53U/XxkQNf5AnylGjIkX2WHBGyvE+9jKzRr9M+B1G4J3dCbrKa6P34OlgZISN1DE3EGiDIMKhtXFQ7ROO48NQ+T/2dj9x5Bgf94hrWfEHwYctDIKIgj86CMUR944= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1595883595; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=cA9n+9vu4RgRBMfOB0BWmNw9KgmpftfRQAnntZebbp8=; b=MPK9OTPGp6XWjb8nmS8uJ/xre3my/ZdL9WkG/AM8FyEDxkeBQKzsZs+wmXLbfOIG/Yy0WOHva8/YSr84Q0sCGfNaFewEP95AOohM3LGy39VNz5++ACXsM9/OsM0S6V1kWi/EsnF1Lbj9bV6ej4rnF98X3WT1WTciQ6EIEEd0Jp0= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1595883595538735.1575523901103; Mon, 27 Jul 2020 13:59:55 -0700 (PDT) Received: from localhost ([::1]:52864 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1k0ADy-0002L9-6q for importer@patchew.org; Mon, 27 Jul 2020 16:59:54 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:34898) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1k0AAK-0005DT-3x for qemu-devel@nongnu.org; Mon, 27 Jul 2020 16:56:08 -0400 Received: from us-smtp-1.mimecast.com ([207.211.31.81]:34776 helo=us-smtp-delivery-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1k0AAI-0004uq-1q for qemu-devel@nongnu.org; Mon, 27 Jul 2020 16:56:07 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-461-sT8M28lkPWyJpMFtz0h3Qg-1; Mon, 27 Jul 2020 16:56:01 -0400 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id F2934101C8A0; Mon, 27 Jul 2020 20:55:59 +0000 (UTC) Received: from blue.redhat.com (ovpn-118-248.rdu2.redhat.com [10.10.118.248]) by smtp.corp.redhat.com (Postfix) with ESMTP id 1D6ED19945; Mon, 27 Jul 2020 20:55:59 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1595883365; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=cA9n+9vu4RgRBMfOB0BWmNw9KgmpftfRQAnntZebbp8=; b=O3bChZA04s3hqNhXKcpzDWE/VhCXfBUP5fQYs2bVAJeP863pychiY+lMLRdQWGNt4uK7Sp ugHFW+/fm1SV6W1fjtFbk4cGN77fjKmQYHVlqoheeJByVza063XRawpXXSU2DzTNWSkKRL r0b6N3FEMjHG31Ex0TbdtDh6Zv1Ryc8= X-MC-Unique: sT8M28lkPWyJpMFtz0h3Qg-1 From: Eric Blake To: qemu-devel@nongnu.org Subject: [PULL 07/24] qemu-iotests/199: increase postcopy period Date: Mon, 27 Jul 2020 15:55:26 -0500 Message-Id: <20200727205543.206624-8-eblake@redhat.com> In-Reply-To: <20200727205543.206624-1-eblake@redhat.com> References: <20200727205543.206624-1-eblake@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.23 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=207.211.31.81; envelope-from=eblake@redhat.com; helo=us-smtp-delivery-1.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/07/27 03:37:14 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Spam_score_int: -30 X-Spam_score: -3.1 X-Spam_bar: --- X-Spam_report: (-3.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-1, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Andrey Shinkevich , Vladimir Sementsov-Ogievskiy , "open list:Block layer core" , Max Reitz Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) Content-Type: text/plain; charset="utf-8" From: Vladimir Sementsov-Ogievskiy The test wants to force a bitmap postcopy. Still, the resulting postcopy period is very small. Let's increase it by adding more bitmaps to migrate. Also, test disabled bitmaps migration. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Andrey Shinkevich Tested-by: Eric Blake Message-Id: <20200727194236.19551-7-vsementsov@virtuozzo.com> Signed-off-by: Eric Blake --- tests/qemu-iotests/199 | 66 +++++++++++++++++++++++++++--------------- 1 file changed, 43 insertions(+), 23 deletions(-) diff --git a/tests/qemu-iotests/199 b/tests/qemu-iotests/199 index da4dae01fb5d..d8532e49da00 100755 --- a/tests/qemu-iotests/199 +++ b/tests/qemu-iotests/199 @@ -103,30 +103,46 @@ class TestDirtyBitmapPostcopyMigration(iotests.QMPTes= tCase): def test_postcopy(self): granularity =3D 512 + nb_bitmaps =3D 15 - result =3D self.vm_a.qmp('block-dirty-bitmap-add', node=3D'drive0', - name=3D'bitmap', granularity=3Dgranularity) - self.assert_qmp(result, 'return', {}) + for i in range(nb_bitmaps): + result =3D self.vm_a.qmp('block-dirty-bitmap-add', node=3D'dri= ve0', + name=3D'bitmap{}'.format(i), + granularity=3Dgranularity) + self.assert_qmp(result, 'return', {}) result =3D self.vm_a.qmp('x-debug-block-dirty-bitmap-sha256', - node=3D'drive0', name=3D'bitmap') + node=3D'drive0', name=3D'bitmap0') empty_sha256 =3D result['return']['sha256'] - apply_discards(self.vm_a, discards1 + discards2) - - result =3D self.vm_a.qmp('x-debug-block-dirty-bitmap-sha256', - node=3D'drive0', name=3D'bitmap') - sha256 =3D result['return']['sha256'] - - # Check, that updating the bitmap by discards works - assert sha256 !=3D empty_sha256 - - result =3D self.vm_a.qmp('block-dirty-bitmap-clear', node=3D'drive= 0', - name=3D'bitmap') - self.assert_qmp(result, 'return', {}) - apply_discards(self.vm_a, discards1) + result =3D self.vm_a.qmp('x-debug-block-dirty-bitmap-sha256', + node=3D'drive0', name=3D'bitmap0') + discards1_sha256 =3D result['return']['sha256'] + + # Check, that updating the bitmap by discards works + assert discards1_sha256 !=3D empty_sha256 + + # We want to calculate resulting sha256. Do it in bitmap0, so, dis= able + # other bitmaps + for i in range(1, nb_bitmaps): + result =3D self.vm_a.qmp('block-dirty-bitmap-disable', node=3D= 'drive0', + name=3D'bitmap{}'.format(i)) + self.assert_qmp(result, 'return', {}) + + apply_discards(self.vm_a, discards2) + + result =3D self.vm_a.qmp('x-debug-block-dirty-bitmap-sha256', + node=3D'drive0', name=3D'bitmap0') + all_discards_sha256 =3D result['return']['sha256'] + + # Now, enable some bitmaps, to be updated during migration + for i in range(2, nb_bitmaps, 2): + result =3D self.vm_a.qmp('block-dirty-bitmap-enable', node=3D'= drive0', + name=3D'bitmap{}'.format(i)) + self.assert_qmp(result, 'return', {}) + caps =3D [{'capability': 'dirty-bitmaps', 'state': True}, {'capability': 'events', 'state': True}] @@ -145,6 +161,7 @@ class TestDirtyBitmapPostcopyMigration(iotests.QMPTestC= ase): event_resume =3D self.vm_b.event_wait('RESUME') self.vm_b_events.append(event_resume) + # enabled bitmaps should be updated apply_discards(self.vm_b, discards2) match =3D {'data': {'status': 'completed'}} @@ -158,7 +175,7 @@ class TestDirtyBitmapPostcopyMigration(iotests.QMPTestC= ase): downtime =3D event_dist(event_stop, event_resume) postcopy_time =3D event_dist(event_resume, event_complete) - # TODO: assert downtime * 10 < postcopy_time + assert downtime * 10 < postcopy_time if debug: print('downtime:', downtime) print('postcopy_time:', postcopy_time) @@ -166,12 +183,15 @@ class TestDirtyBitmapPostcopyMigration(iotests.QMPTes= tCase): # Assert that bitmap migration is finished (check that successor b= itmap # is removed) result =3D self.vm_b.qmp('query-block') - assert len(result['return'][0]['dirty-bitmaps']) =3D=3D 1 + assert len(result['return'][0]['dirty-bitmaps']) =3D=3D nb_bitmaps - # Check content of migrated (and updated by new writes) bitmap - result =3D self.vm_b.qmp('x-debug-block-dirty-bitmap-sha256', - node=3D'drive0', name=3D'bitmap') - self.assert_qmp(result, 'return/sha256', sha256) + # Check content of migrated bitmaps. Still, don't waste time check= ing + # every bitmap + for i in range(0, nb_bitmaps, 5): + result =3D self.vm_b.qmp('x-debug-block-dirty-bitmap-sha256', + node=3D'drive0', name=3D'bitmap{}'.form= at(i)) + sha256 =3D discards1_sha256 if i % 2 else all_discards_sha256 + self.assert_qmp(result, 'return/sha256', sha256) if __name__ =3D=3D '__main__': --=20 2.27.0 From nobody Mon Feb 9 01:37:05 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1595883880; cv=none; d=zohomail.com; s=zohoarc; b=cpJa78QLxn8zhdWvhn/8OMdEjpawQymKXX+1QOTn62GwQV2XXZcLC/RE37yRkUc17mSnNHdRmp3yFR0+/U44wSQmBm1TZ4bdJJVMDwge5bPmVEfk0w7MM496KSmWhnw/AvDo6Fpzn9RyNFAtVA3Pf2P/mLKVdE7mHCo2gPHBRVI= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1595883880; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=z40Q5HYVAdYWq9rhFSLOMYa+2be9RGrWon7GvmEWR/k=; b=foPlFYeqEI/UistPnGt+YM0iKFuclVk5LidA/gWQXivMz4+ZtbKYcvdUkJoLnz2twmDVs9pDuSMNdrj0h6OHFOIhAZr2kFaBXzLJl/SgLALfSTDgOVdZ+A6C2yqTlDYKxy2D2M0JyFSszGEwtFCl3gxvAxqAAO/V3CNgITa4vRs= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1595883880330293.43005720178246; Mon, 27 Jul 2020 14:04:40 -0700 (PDT) Received: from localhost ([::1]:41328 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1k0AIZ-0000st-1K for importer@patchew.org; Mon, 27 Jul 2020 17:04:39 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:34998) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1k0AAP-0005Iz-SV for qemu-devel@nongnu.org; Mon, 27 Jul 2020 16:56:15 -0400 Received: from us-smtp-delivery-1.mimecast.com ([207.211.31.120]:57842 helo=us-smtp-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1k0AAO-0004w9-Dl for qemu-devel@nongnu.org; Mon, 27 Jul 2020 16:56:13 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-125-wF9uBFxzNKitcD4MDOxDSg-1; Mon, 27 Jul 2020 16:56:06 -0400 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 9BA9C10059A2; Mon, 27 Jul 2020 20:56:04 +0000 (UTC) Received: from blue.redhat.com (ovpn-118-248.rdu2.redhat.com [10.10.118.248]) by smtp.corp.redhat.com (Postfix) with ESMTP id 4210B19724; Mon, 27 Jul 2020 20:56:00 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1595883370; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=z40Q5HYVAdYWq9rhFSLOMYa+2be9RGrWon7GvmEWR/k=; b=DEMNfz2zR/vTXK/QsUyPf+dOi8GOC/pKaAdZu8NWfCQQsQdlz90+b482r0IKpfdfMzKOil 04ZLaiZdmx68aSZkz5SH0dqdMPVKo23Cx5RboUE5CfjKHrlKp1hmrs9EUE2OrkPSLEdiD1 z+x3SrRZr/WxflVxBJCBqDOZch5CjSk= X-MC-Unique: wF9uBFxzNKitcD4MDOxDSg-1 From: Eric Blake To: qemu-devel@nongnu.org Subject: [PULL 08/24] migration/block-dirty-bitmap: fix dirty_bitmap_mig_before_vm_start Date: Mon, 27 Jul 2020 15:55:27 -0500 Message-Id: <20200727205543.206624-9-eblake@redhat.com> In-Reply-To: <20200727205543.206624-1-eblake@redhat.com> References: <20200727205543.206624-1-eblake@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.23 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=eblake@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=207.211.31.120; envelope-from=eblake@redhat.com; helo=us-smtp-1.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/07/27 15:02:33 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Spam_score_int: -30 X-Spam_score: -3.1 X-Spam_bar: --- X-Spam_report: (-3.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-1, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Fam Zheng , Vladimir Sementsov-Ogievskiy , "open list:Dirty Bitmaps" , Juan Quintela , qemu-stable@nongnu.org, "Dr. David Alan Gilbert" , Stefan Hajnoczi , Andrey Shinkevich , John Snow Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) Content-Type: text/plain; charset="utf-8" From: Vladimir Sementsov-Ogievskiy Using the _locked version of bdrv_enable_dirty_bitmap to bypass locking is wrong as we do not already own the mutex. Moreover, the adjacent call to bdrv_dirty_bitmap_enable_successor grabs the mutex. Fixes: 58f72b965e9e1q Cc: qemu-stable@nongnu.org # v3.0 Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Andrey Shinkevich Reviewed-by: Eric Blake Message-Id: <20200727194236.19551-8-vsementsov@virtuozzo.com> Signed-off-by: Eric Blake --- migration/block-dirty-bitmap.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/migration/block-dirty-bitmap.c b/migration/block-dirty-bitmap.c index b0dbf9eeed43..0739f1259e05 100644 --- a/migration/block-dirty-bitmap.c +++ b/migration/block-dirty-bitmap.c @@ -566,7 +566,7 @@ void dirty_bitmap_mig_before_vm_start(void) DirtyBitmapLoadBitmapState *b =3D item->data; if (b->migrated) { - bdrv_enable_dirty_bitmap_locked(b->bitmap); + bdrv_enable_dirty_bitmap(b->bitmap); } else { bdrv_dirty_bitmap_enable_successor(b->bitmap); } --=20 2.27.0 From nobody Mon Feb 9 01:37:05 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1595883743; cv=none; d=zohomail.com; s=zohoarc; b=Wz5Wx2EksuqWe0HDVqKrXFWimepZi9dvtX6AKSBnl2Ws6ZN6WQ8pakKddS1GkG7fPWH/J5h9MkJSlGfSwb4UlIN56sTGCXLIii5kP9jUNhOCGaw8Ef3TVkgudWnr//aH0oaV07q7FjUev3iZ7YT66OGD7DCFiyphgnuaEy9e3m4= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1595883743; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=sg6tJDbO72LwAKARdg9zDX9TDfIxVD+Q8BzNg0L4F0I=; b=N+YNS357SOw9T9+WKXlF1H8nimz4nbnqEtRLXHuPTdyexq6OTkoziKef8hb4IY+MXdfuxDItIUAmuNeuWmuv0nHE/TobA0TaQcXBTSUwPioKiq36WhYnkYGh1/K9VyFriYuH2fuFR/NErb0l0bJ1xsCUPdYXakOzDM6noiVLZeU= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1595883743266405.1248855986604; Mon, 27 Jul 2020 14:02:23 -0700 (PDT) Received: from localhost ([::1]:33072 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1k0AGL-0005u1-Nb for importer@patchew.org; Mon, 27 Jul 2020 17:02:21 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:35024) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1k0AAS-0005Jv-BH for qemu-devel@nongnu.org; Mon, 27 Jul 2020 16:56:16 -0400 Received: from us-smtp-delivery-1.mimecast.com ([205.139.110.120]:50809 helo=us-smtp-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1k0AAO-0004wE-EC for qemu-devel@nongnu.org; Mon, 27 Jul 2020 16:56:16 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-402-8RAXLN6-OsOIv5zbIspMCg-1; Mon, 27 Jul 2020 16:56:08 -0400 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 243B458; Mon, 27 Jul 2020 20:56:07 +0000 (UTC) Received: from blue.redhat.com (ovpn-118-248.rdu2.redhat.com [10.10.118.248]) by smtp.corp.redhat.com (Postfix) with ESMTP id E0DA419D82; Mon, 27 Jul 2020 20:56:04 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1595883371; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=sg6tJDbO72LwAKARdg9zDX9TDfIxVD+Q8BzNg0L4F0I=; b=To74MJQ9t0KPu6gTUYSn0iZsxVkNQ4U9VpKFEZJ2wCI2UBsSRKNVCMjeGUkJ8m2r4dATab I1993hfOSb3/6ZAg0X77mXkB12lJnxYKoqQB02dRTwCk9M3nSRTEaBmTaIIss5/lwGeCr3 Y6dfoIY3NLn+VaRaOt7TQviUkecK9Yc= X-MC-Unique: 8RAXLN6-OsOIv5zbIspMCg-1 From: Eric Blake To: qemu-devel@nongnu.org Subject: [PULL 09/24] migration/block-dirty-bitmap: rename state structure types Date: Mon, 27 Jul 2020 15:55:28 -0500 Message-Id: <20200727205543.206624-10-eblake@redhat.com> In-Reply-To: <20200727205543.206624-1-eblake@redhat.com> References: <20200727205543.206624-1-eblake@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.23 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=205.139.110.120; envelope-from=eblake@redhat.com; helo=us-smtp-1.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/07/27 01:44:14 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Spam_score_int: -40 X-Spam_score: -4.1 X-Spam_bar: ---- X-Spam_report: (-4.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-1, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-1, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Fam Zheng , Vladimir Sementsov-Ogievskiy , "open list:Dirty Bitmaps" , Juan Quintela , "Dr. David Alan Gilbert" , Stefan Hajnoczi , Andrey Shinkevich , John Snow Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) Content-Type: text/plain; charset="utf-8" From: Vladimir Sementsov-Ogievskiy Rename types to be symmetrical for load/save part and shorter. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Andrey Shinkevich Reviewed-by: Eric Blake Message-Id: <20200727194236.19551-9-vsementsov@virtuozzo.com> Signed-off-by: Eric Blake --- migration/block-dirty-bitmap.c | 70 ++++++++++++++++++---------------- 1 file changed, 37 insertions(+), 33 deletions(-) diff --git a/migration/block-dirty-bitmap.c b/migration/block-dirty-bitmap.c index 0739f1259e05..1d57bff4f6c7 100644 --- a/migration/block-dirty-bitmap.c +++ b/migration/block-dirty-bitmap.c @@ -100,23 +100,25 @@ /* 0x04 was "AUTOLOAD" flags on elder versions, no it is ignored */ #define DIRTY_BITMAP_MIG_START_FLAG_RESERVED_MASK 0xf8 -typedef struct DirtyBitmapMigBitmapState { +/* State of one bitmap during save process */ +typedef struct SaveBitmapState { /* Written during setup phase. */ BlockDriverState *bs; const char *node_name; BdrvDirtyBitmap *bitmap; uint64_t total_sectors; uint64_t sectors_per_chunk; - QSIMPLEQ_ENTRY(DirtyBitmapMigBitmapState) entry; + QSIMPLEQ_ENTRY(SaveBitmapState) entry; uint8_t flags; /* For bulk phase. */ bool bulk_completed; uint64_t cur_sector; -} DirtyBitmapMigBitmapState; +} SaveBitmapState; -typedef struct DirtyBitmapMigState { - QSIMPLEQ_HEAD(, DirtyBitmapMigBitmapState) dbms_list; +/* State of the dirty bitmap migration (DBM) during save process */ +typedef struct DBMSaveState { + QSIMPLEQ_HEAD(, SaveBitmapState) dbms_list; bool bulk_completed; bool no_bitmaps; @@ -124,23 +126,25 @@ typedef struct DirtyBitmapMigState { /* for send_bitmap_bits() */ BlockDriverState *prev_bs; BdrvDirtyBitmap *prev_bitmap; -} DirtyBitmapMigState; +} DBMSaveState; -typedef struct DirtyBitmapLoadState { +/* State of the dirty bitmap migration (DBM) during load process */ +typedef struct DBMLoadState { uint32_t flags; char node_name[256]; char bitmap_name[256]; BlockDriverState *bs; BdrvDirtyBitmap *bitmap; -} DirtyBitmapLoadState; +} DBMLoadState; -static DirtyBitmapMigState dirty_bitmap_mig_state; +static DBMSaveState dirty_bitmap_mig_state; -typedef struct DirtyBitmapLoadBitmapState { +/* State of one bitmap during load process */ +typedef struct LoadBitmapState { BlockDriverState *bs; BdrvDirtyBitmap *bitmap; bool migrated; -} DirtyBitmapLoadBitmapState; +} LoadBitmapState; static GSList *enabled_bitmaps; QemuMutex finish_lock; @@ -170,7 +174,7 @@ static void qemu_put_bitmap_flags(QEMUFile *f, uint32_t= flags) qemu_put_byte(f, flags); } -static void send_bitmap_header(QEMUFile *f, DirtyBitmapMigBitmapState *dbm= s, +static void send_bitmap_header(QEMUFile *f, SaveBitmapState *dbms, uint32_t additional_flags) { BlockDriverState *bs =3D dbms->bs; @@ -199,19 +203,19 @@ static void send_bitmap_header(QEMUFile *f, DirtyBitm= apMigBitmapState *dbms, } } -static void send_bitmap_start(QEMUFile *f, DirtyBitmapMigBitmapState *dbms) +static void send_bitmap_start(QEMUFile *f, SaveBitmapState *dbms) { send_bitmap_header(f, dbms, DIRTY_BITMAP_MIG_FLAG_START); qemu_put_be32(f, bdrv_dirty_bitmap_granularity(dbms->bitmap)); qemu_put_byte(f, dbms->flags); } -static void send_bitmap_complete(QEMUFile *f, DirtyBitmapMigBitmapState *d= bms) +static void send_bitmap_complete(QEMUFile *f, SaveBitmapState *dbms) { send_bitmap_header(f, dbms, DIRTY_BITMAP_MIG_FLAG_COMPLETE); } -static void send_bitmap_bits(QEMUFile *f, DirtyBitmapMigBitmapState *dbms, +static void send_bitmap_bits(QEMUFile *f, SaveBitmapState *dbms, uint64_t start_sector, uint32_t nr_sectors) { /* align for buffer_is_zero() */ @@ -257,7 +261,7 @@ static void send_bitmap_bits(QEMUFile *f, DirtyBitmapMi= gBitmapState *dbms, /* Called with iothread lock taken. */ static void dirty_bitmap_mig_cleanup(void) { - DirtyBitmapMigBitmapState *dbms; + SaveBitmapState *dbms; while ((dbms =3D QSIMPLEQ_FIRST(&dirty_bitmap_mig_state.dbms_list)) != =3D NULL) { QSIMPLEQ_REMOVE_HEAD(&dirty_bitmap_mig_state.dbms_list, entry); @@ -271,7 +275,7 @@ static void dirty_bitmap_mig_cleanup(void) static int add_bitmaps_to_list(BlockDriverState *bs, const char *bs_name) { BdrvDirtyBitmap *bitmap; - DirtyBitmapMigBitmapState *dbms; + SaveBitmapState *dbms; Error *local_err =3D NULL; FOR_EACH_DIRTY_BITMAP(bs, bitmap) { @@ -309,7 +313,7 @@ static int add_bitmaps_to_list(BlockDriverState *bs, co= nst char *bs_name) bdrv_ref(bs); bdrv_dirty_bitmap_set_busy(bitmap, true); - dbms =3D g_new0(DirtyBitmapMigBitmapState, 1); + dbms =3D g_new0(SaveBitmapState, 1); dbms->bs =3D bs; dbms->node_name =3D bs_name; dbms->bitmap =3D bitmap; @@ -334,7 +338,7 @@ static int add_bitmaps_to_list(BlockDriverState *bs, co= nst char *bs_name) static int init_dirty_bitmap_migration(void) { BlockDriverState *bs; - DirtyBitmapMigBitmapState *dbms; + SaveBitmapState *dbms; GHashTable *handled_by_blk =3D g_hash_table_new(NULL, NULL); BlockBackend *blk; @@ -408,7 +412,7 @@ fail: } /* Called with no lock taken. */ -static void bulk_phase_send_chunk(QEMUFile *f, DirtyBitmapMigBitmapState *= dbms) +static void bulk_phase_send_chunk(QEMUFile *f, SaveBitmapState *dbms) { uint32_t nr_sectors =3D MIN(dbms->total_sectors - dbms->cur_sector, dbms->sectors_per_chunk); @@ -424,7 +428,7 @@ static void bulk_phase_send_chunk(QEMUFile *f, DirtyBit= mapMigBitmapState *dbms) /* Called with no lock taken. */ static void bulk_phase(QEMUFile *f, bool limit) { - DirtyBitmapMigBitmapState *dbms; + SaveBitmapState *dbms; QSIMPLEQ_FOREACH(dbms, &dirty_bitmap_mig_state.dbms_list, entry) { while (!dbms->bulk_completed) { @@ -461,7 +465,7 @@ static int dirty_bitmap_save_iterate(QEMUFile *f, void = *opaque) static int dirty_bitmap_save_complete(QEMUFile *f, void *opaque) { - DirtyBitmapMigBitmapState *dbms; + SaveBitmapState *dbms; trace_dirty_bitmap_save_complete_enter(); if (!dirty_bitmap_mig_state.bulk_completed) { @@ -486,7 +490,7 @@ static void dirty_bitmap_save_pending(QEMUFile *f, void= *opaque, uint64_t *res_compatible, uint64_t *res_postcopy_only) { - DirtyBitmapMigBitmapState *dbms; + SaveBitmapState *dbms; uint64_t pending =3D 0; qemu_mutex_lock_iothread(); @@ -507,7 +511,7 @@ static void dirty_bitmap_save_pending(QEMUFile *f, void= *opaque, } /* First occurrence of this bitmap. It should be created if doesn't exist = */ -static int dirty_bitmap_load_start(QEMUFile *f, DirtyBitmapLoadState *s) +static int dirty_bitmap_load_start(QEMUFile *f, DBMLoadState *s) { Error *local_err =3D NULL; uint32_t granularity =3D qemu_get_be32(f); @@ -538,7 +542,7 @@ static int dirty_bitmap_load_start(QEMUFile *f, DirtyBi= tmapLoadState *s) bdrv_disable_dirty_bitmap(s->bitmap); if (flags & DIRTY_BITMAP_MIG_START_FLAG_ENABLED) { - DirtyBitmapLoadBitmapState *b; + LoadBitmapState *b; bdrv_dirty_bitmap_create_successor(s->bitmap, &local_err); if (local_err) { @@ -546,7 +550,7 @@ static int dirty_bitmap_load_start(QEMUFile *f, DirtyBi= tmapLoadState *s) return -EINVAL; } - b =3D g_new(DirtyBitmapLoadBitmapState, 1); + b =3D g_new(LoadBitmapState, 1); b->bs =3D s->bs; b->bitmap =3D s->bitmap; b->migrated =3D false; @@ -563,7 +567,7 @@ void dirty_bitmap_mig_before_vm_start(void) qemu_mutex_lock(&finish_lock); for (item =3D enabled_bitmaps; item; item =3D g_slist_next(item)) { - DirtyBitmapLoadBitmapState *b =3D item->data; + LoadBitmapState *b =3D item->data; if (b->migrated) { bdrv_enable_dirty_bitmap(b->bitmap); @@ -580,7 +584,7 @@ void dirty_bitmap_mig_before_vm_start(void) qemu_mutex_unlock(&finish_lock); } -static void dirty_bitmap_load_complete(QEMUFile *f, DirtyBitmapLoadState *= s) +static void dirty_bitmap_load_complete(QEMUFile *f, DBMLoadState *s) { GSList *item; trace_dirty_bitmap_load_complete(); @@ -589,7 +593,7 @@ static void dirty_bitmap_load_complete(QEMUFile *f, Dir= tyBitmapLoadState *s) qemu_mutex_lock(&finish_lock); for (item =3D enabled_bitmaps; item; item =3D g_slist_next(item)) { - DirtyBitmapLoadBitmapState *b =3D item->data; + LoadBitmapState *b =3D item->data; if (b->bitmap =3D=3D s->bitmap) { b->migrated =3D true; @@ -621,7 +625,7 @@ static void dirty_bitmap_load_complete(QEMUFile *f, Dir= tyBitmapLoadState *s) qemu_mutex_unlock(&finish_lock); } -static int dirty_bitmap_load_bits(QEMUFile *f, DirtyBitmapLoadState *s) +static int dirty_bitmap_load_bits(QEMUFile *f, DBMLoadState *s) { uint64_t first_byte =3D qemu_get_be64(f) << BDRV_SECTOR_BITS; uint64_t nr_bytes =3D (uint64_t)qemu_get_be32(f) << BDRV_SECTOR_BITS; @@ -666,7 +670,7 @@ static int dirty_bitmap_load_bits(QEMUFile *f, DirtyBit= mapLoadState *s) return 0; } -static int dirty_bitmap_load_header(QEMUFile *f, DirtyBitmapLoadState *s) +static int dirty_bitmap_load_header(QEMUFile *f, DBMLoadState *s) { Error *local_err =3D NULL; bool nothing; @@ -715,7 +719,7 @@ static int dirty_bitmap_load_header(QEMUFile *f, DirtyB= itmapLoadState *s) static int dirty_bitmap_load(QEMUFile *f, void *opaque, int version_id) { - static DirtyBitmapLoadState s; + static DBMLoadState s; int ret =3D 0; trace_dirty_bitmap_load_enter(); @@ -753,7 +757,7 @@ static int dirty_bitmap_load(QEMUFile *f, void *opaque,= int version_id) static int dirty_bitmap_save_setup(QEMUFile *f, void *opaque) { - DirtyBitmapMigBitmapState *dbms =3D NULL; + SaveBitmapState *dbms =3D NULL; if (init_dirty_bitmap_migration() < 0) { return -1; } --=20 2.27.0 From nobody Mon Feb 9 01:37:05 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1595883576; cv=none; d=zohomail.com; s=zohoarc; b=P01keFZZkmiVjFG1SsXghb4z/4obOOPFVzV1c11GMK8AOaAAV/CwLO9v1/FvovG9CXr6nuviKjhIwhH946+lU3O8WREJU2ENNlFNCNZtXSWmdY9PuGYokD1Oigeti4IExuXCl5eQiIplPuBG2gQ3nSqOAg8PZAxjLnhqumB/HAA= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1595883576; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=fVx0VAHepZVGUOdQt72FjQmlLnKaNCNllgZ9kWYIfkg=; b=BrkhsUwWGiC6jAOmtW74BQH+URcHgQfBJ2n2pqXIZXm/uM4mHSR/aojqlHWFyRBPQR7uQqeo8LQwohHc7VGRCZRM+QkhKMJJ32U30ejEykRFPuGXd+xXdZ1j1aNC24FPMwEXT87J2TWv9ArtqGEUFxdGgJBEqWEYvQm5Ocn/IjQ= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1595883576627561.9530865203064; Mon, 27 Jul 2020 13:59:36 -0700 (PDT) Received: from localhost ([::1]:51716 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1k0ADf-0001rh-Bz for importer@patchew.org; Mon, 27 Jul 2020 16:59:35 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:35084) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1k0AAa-0005Lx-1L for qemu-devel@nongnu.org; Mon, 27 Jul 2020 16:56:24 -0400 Received: from us-smtp-1.mimecast.com ([205.139.110.61]:29231 helo=us-smtp-delivery-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1k0AAT-0004xw-I2 for qemu-devel@nongnu.org; Mon, 27 Jul 2020 16:56:23 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-494-DjFWZ96TPdeaV_Xv6jOI9Q-1; Mon, 27 Jul 2020 16:56:09 -0400 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 9995B101C8A0; Mon, 27 Jul 2020 20:56:08 +0000 (UTC) Received: from blue.redhat.com (ovpn-118-248.rdu2.redhat.com [10.10.118.248]) by smtp.corp.redhat.com (Postfix) with ESMTP id 65EF419D82; Mon, 27 Jul 2020 20:56:07 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1595883376; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=fVx0VAHepZVGUOdQt72FjQmlLnKaNCNllgZ9kWYIfkg=; b=GjEe2R+zGSqrQnFFWXLPGCcDSN4m7GHdKO9LmoHuRpwnSl5/+VTdObUtRMdC8jdy6QqZ3A d4/+MvccV1ZeWyEB2Gj7+nWLtLslvFQUMFD1HA4s1N1d60Z12RIQ65xkCHFDg1EH0soxLI f9ywAvcguMgYrMcg4L0jfJPmrC7Dg6A= X-MC-Unique: DjFWZ96TPdeaV_Xv6jOI9Q-1 From: Eric Blake To: qemu-devel@nongnu.org Subject: [PULL 10/24] migration/block-dirty-bitmap: rename dirty_bitmap_mig_cleanup Date: Mon, 27 Jul 2020 15:55:29 -0500 Message-Id: <20200727205543.206624-11-eblake@redhat.com> In-Reply-To: <20200727205543.206624-1-eblake@redhat.com> References: <20200727205543.206624-1-eblake@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.23 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=205.139.110.61; envelope-from=eblake@redhat.com; helo=us-smtp-delivery-1.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/07/27 01:46:13 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Spam_score_int: -40 X-Spam_score: -4.1 X-Spam_bar: ---- X-Spam_report: (-4.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-1, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-1, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Fam Zheng , Vladimir Sementsov-Ogievskiy , "open list:Block I/O path" , Juan Quintela , "Dr. David Alan Gilbert" , Stefan Hajnoczi , Andrey Shinkevich , John Snow Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) Content-Type: text/plain; charset="utf-8" From: Vladimir Sementsov-Ogievskiy Rename dirty_bitmap_mig_cleanup to dirty_bitmap_do_save_cleanup, to stress that it is on save part. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Andrey Shinkevich Reviewed-by: Eric Blake Message-Id: <20200727194236.19551-10-vsementsov@virtuozzo.com> Signed-off-by: Eric Blake --- migration/block-dirty-bitmap.c | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/migration/block-dirty-bitmap.c b/migration/block-dirty-bitmap.c index 1d57bff4f6c7..01a536d7d3d3 100644 --- a/migration/block-dirty-bitmap.c +++ b/migration/block-dirty-bitmap.c @@ -259,7 +259,7 @@ static void send_bitmap_bits(QEMUFile *f, SaveBitmapSta= te *dbms, } /* Called with iothread lock taken. */ -static void dirty_bitmap_mig_cleanup(void) +static void dirty_bitmap_do_save_cleanup(void) { SaveBitmapState *dbms; @@ -406,7 +406,7 @@ static int init_dirty_bitmap_migration(void) fail: g_hash_table_destroy(handled_by_blk); - dirty_bitmap_mig_cleanup(); + dirty_bitmap_do_save_cleanup(); return -1; } @@ -445,7 +445,7 @@ static void bulk_phase(QEMUFile *f, bool limit) /* for SaveVMHandlers */ static void dirty_bitmap_save_cleanup(void *opaque) { - dirty_bitmap_mig_cleanup(); + dirty_bitmap_do_save_cleanup(); } static int dirty_bitmap_save_iterate(QEMUFile *f, void *opaque) @@ -480,7 +480,7 @@ static int dirty_bitmap_save_complete(QEMUFile *f, void= *opaque) trace_dirty_bitmap_save_complete_finish(); - dirty_bitmap_mig_cleanup(); + dirty_bitmap_do_save_cleanup(); return 0; } --=20 2.27.0 From nobody Mon Feb 9 01:37:05 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1595883977; cv=none; d=zohomail.com; s=zohoarc; b=AQNPFDrnY+yO4Ogt6QvB8P7XhfJk66gXXWrw8+b8ITY0jvGeV9SoJqKsj/f7QMQosXnYkc55mJI9BtzSUndyKI3JoDBslZlslPv5wADFgEC4926e4kIAVxYcg5KPjWyOyxhxgPJH98xEXnDO1jlDLYgetT0hwsKjeHyUL+8LxPM= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1595883977; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=TWfFQWHl1Ne31lryN1BAiMSTvBo6CJMM+z/0evN+18Y=; b=lkTx5gkE6Y29gjTqa+LNuzEDbE5hCMa0EVkEeP3XlO5F8loyIASLHrIJOONhmlkbtdQAkfN+FAEZIXGhLTCm2z2QvgGqpF6SgZ+ycVmQlBmJy/1yPuyOoAFBnJDSokb9UTaeextBKBTUjnb2E5i6l9drfti61OHWOv8PMGFk74w= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 159588397709114.972720930698529; Mon, 27 Jul 2020 14:06:17 -0700 (PDT) Received: from localhost ([::1]:50856 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1k0AK7-0004rF-PB for importer@patchew.org; Mon, 27 Jul 2020 17:06:15 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:35066) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1k0AAZ-0005Ls-SY for qemu-devel@nongnu.org; Mon, 27 Jul 2020 16:56:23 -0400 Received: from us-smtp-1.mimecast.com ([207.211.31.81]:44703 helo=us-smtp-delivery-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1k0AAS-0004xn-N9 for qemu-devel@nongnu.org; Mon, 27 Jul 2020 16:56:20 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-99-AU1-ourUPyGfSzU0zBtKIA-1; Mon, 27 Jul 2020 16:56:11 -0400 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 281338015F7; Mon, 27 Jul 2020 20:56:10 +0000 (UTC) Received: from blue.redhat.com (ovpn-118-248.rdu2.redhat.com [10.10.118.248]) by smtp.corp.redhat.com (Postfix) with ESMTP id E36C719724; Mon, 27 Jul 2020 20:56:08 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1595883375; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=TWfFQWHl1Ne31lryN1BAiMSTvBo6CJMM+z/0evN+18Y=; b=FQWc5DpEhS8T3CFZkOkc4BlP0Zo6Y0JsYUUVyYjw4Djm4vvIkA0dCkzOkf2as7EGYZeVpb 0cuX77YPQdcITNMGPQvaJceIIvogYxHynv7pMZX0xiIsMg32qcoVl0cFnxH+l9nEX6DK7O VyFWmWQ7TAP7AxsPCIhBSNe6WL+39Ro= X-MC-Unique: AU1-ourUPyGfSzU0zBtKIA-1 From: Eric Blake To: qemu-devel@nongnu.org Subject: [PULL 11/24] migration/block-dirty-bitmap: move mutex init to dirty_bitmap_mig_init Date: Mon, 27 Jul 2020 15:55:30 -0500 Message-Id: <20200727205543.206624-12-eblake@redhat.com> In-Reply-To: <20200727205543.206624-1-eblake@redhat.com> References: <20200727205543.206624-1-eblake@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.23 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=eblake@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=207.211.31.81; envelope-from=eblake@redhat.com; helo=us-smtp-delivery-1.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/07/27 03:37:14 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Spam_score_int: -30 X-Spam_score: -3.1 X-Spam_bar: --- X-Spam_report: (-3.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-1, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Fam Zheng , Vladimir Sementsov-Ogievskiy , "open list:Dirty Bitmaps" , Juan Quintela , "Dr . David Alan Gilbert" , Stefan Hajnoczi , Andrey Shinkevich , John Snow Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) Content-Type: text/plain; charset="utf-8" From: Vladimir Sementsov-Ogievskiy No reasons to keep two public init functions. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Andrey Shinkevich Reviewed-by: Dr. David Alan Gilbert Message-Id: <20200727194236.19551-11-vsementsov@virtuozzo.com> Signed-off-by: Eric Blake --- migration/migration.h | 1 - migration/block-dirty-bitmap.c | 6 +----- migration/migration.c | 2 -- 3 files changed, 1 insertion(+), 8 deletions(-) diff --git a/migration/migration.h b/migration/migration.h index f617960522aa..ab20c756f549 100644 --- a/migration/migration.h +++ b/migration/migration.h @@ -335,7 +335,6 @@ void migrate_send_rp_recv_bitmap(MigrationIncomingState= *mis, void migrate_send_rp_resume_ack(MigrationIncomingState *mis, uint32_t valu= e); void dirty_bitmap_mig_before_vm_start(void); -void init_dirty_bitmap_incoming_migration(void); void migrate_add_address(SocketAddress *address); int foreach_not_ignored_block(RAMBlockIterFunc func, void *opaque); diff --git a/migration/block-dirty-bitmap.c b/migration/block-dirty-bitmap.c index 01a536d7d3d3..4b67e4f4fbcd 100644 --- a/migration/block-dirty-bitmap.c +++ b/migration/block-dirty-bitmap.c @@ -148,11 +148,6 @@ typedef struct LoadBitmapState { static GSList *enabled_bitmaps; QemuMutex finish_lock; -void init_dirty_bitmap_incoming_migration(void) -{ - qemu_mutex_init(&finish_lock); -} - static uint32_t qemu_get_bitmap_flags(QEMUFile *f) { uint8_t flags =3D qemu_get_byte(f); @@ -801,6 +796,7 @@ static SaveVMHandlers savevm_dirty_bitmap_handlers =3D { void dirty_bitmap_mig_init(void) { QSIMPLEQ_INIT(&dirty_bitmap_mig_state.dbms_list); + qemu_mutex_init(&finish_lock); register_savevm_live("dirty-bitmap", 0, 1, &savevm_dirty_bitmap_handlers, diff --git a/migration/migration.c b/migration/migration.c index 2ed99232272e..1c61428988e9 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -165,8 +165,6 @@ void migration_object_init(void) qemu_sem_init(¤t_incoming->postcopy_pause_sem_dst, 0); qemu_sem_init(¤t_incoming->postcopy_pause_sem_fault, 0); - init_dirty_bitmap_incoming_migration(); - if (!migration_object_check(current_migration, &err)) { error_report_err(err); exit(1); --=20 2.27.0 From nobody Mon Feb 9 01:37:05 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1595883745; cv=none; d=zohomail.com; s=zohoarc; b=DtwOtJyl/EhiAwmPheq+/xTapg1RXU6oDoP+3Hmc+DUXQxDC2WuVkmS7KEvxKuK/Ge/zDe1N26auSWDVyJNejCs1i1zWdYhQv7pOVfzsGcG1NlUhxq3nb42bCsYVi/N6i/WMBNGHjG++nhdnIPOaihKU+jZ+nZGwdmZJ1bwgXT4= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1595883745; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=etZ4YFR5z2HMLe84KKA4NnmGz8eOF+r3IjzereHUr6Q=; b=DvEeWaY1q2P9mtbo0jzHUFYbOi/HPKilX7qhGyXzUxwupUGSzZe2wbgXdT8MRzGUbySxbyKL4RbKwxClLIYn8phd/pmFhjFONr3qn5YlMk6G67EJmMwlyssb/qFBHmoK8V7b3fvunrTvuqZ1GdJnE5oKPWeWOyTitjBTVM8dTDI= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1595883745376396.69475036209474; Mon, 27 Jul 2020 14:02:25 -0700 (PDT) Received: from localhost ([::1]:33262 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1k0AGN-0005zQ-UN for importer@patchew.org; Mon, 27 Jul 2020 17:02:23 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:35062) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1k0AAZ-0005Lp-SR for qemu-devel@nongnu.org; Mon, 27 Jul 2020 16:56:23 -0400 Received: from us-smtp-delivery-1.mimecast.com ([207.211.31.120]:29944 helo=us-smtp-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1k0AAS-0004xe-2H for qemu-devel@nongnu.org; Mon, 27 Jul 2020 16:56:20 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-331-XASRwDDXPy6J5eF-m4_UVQ-1; Mon, 27 Jul 2020 16:56:12 -0400 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id A222A59; Mon, 27 Jul 2020 20:56:11 +0000 (UTC) Received: from blue.redhat.com (ovpn-118-248.rdu2.redhat.com [10.10.118.248]) by smtp.corp.redhat.com (Postfix) with ESMTP id 6B2F519724; Mon, 27 Jul 2020 20:56:10 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1595883375; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=etZ4YFR5z2HMLe84KKA4NnmGz8eOF+r3IjzereHUr6Q=; b=LLp8/YongIP4D0r3h2ZX38XIWwXOrI7XDvEGG9Yc58ngEPQazEY7BvlgDUEwQwLgHl8/f+ G1fTHmoIxepPMpCuLBFZzWo+uInwgc/9wkp5khS963vdqiGqhJwhyJH56B77ZDR5PgPMn7 pPoBAkwHNKslXFiOCJ2H2G9IAZN7648= X-MC-Unique: XASRwDDXPy6J5eF-m4_UVQ-1 From: Eric Blake To: qemu-devel@nongnu.org Subject: [PULL 12/24] migration/block-dirty-bitmap: refactor state global variables Date: Mon, 27 Jul 2020 15:55:31 -0500 Message-Id: <20200727205543.206624-13-eblake@redhat.com> In-Reply-To: <20200727205543.206624-1-eblake@redhat.com> References: <20200727205543.206624-1-eblake@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.23 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=207.211.31.120; envelope-from=eblake@redhat.com; helo=us-smtp-1.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/07/27 15:02:33 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Spam_score_int: -30 X-Spam_score: -3.1 X-Spam_bar: --- X-Spam_report: (-3.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-1, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Fam Zheng , Vladimir Sementsov-Ogievskiy , "open list:Block I/O path" , Juan Quintela , "Dr. David Alan Gilbert" , Stefan Hajnoczi , Andrey Shinkevich , John Snow Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) Content-Type: text/plain; charset="utf-8" From: Vladimir Sementsov-Ogievskiy Move all state variables into one global struct. Reduce global variable usage, utilizing opaque pointer where possible. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Andrey Shinkevich Message-Id: <20200727194236.19551-12-vsementsov@virtuozzo.com> Signed-off-by: Eric Blake --- migration/block-dirty-bitmap.c | 179 ++++++++++++++++++--------------- 1 file changed, 99 insertions(+), 80 deletions(-) diff --git a/migration/block-dirty-bitmap.c b/migration/block-dirty-bitmap.c index 4b67e4f4fbcd..9b39e7aa2b4f 100644 --- a/migration/block-dirty-bitmap.c +++ b/migration/block-dirty-bitmap.c @@ -128,6 +128,12 @@ typedef struct DBMSaveState { BdrvDirtyBitmap *prev_bitmap; } DBMSaveState; +typedef struct LoadBitmapState { + BlockDriverState *bs; + BdrvDirtyBitmap *bitmap; + bool migrated; +} LoadBitmapState; + /* State of the dirty bitmap migration (DBM) during load process */ typedef struct DBMLoadState { uint32_t flags; @@ -135,18 +141,17 @@ typedef struct DBMLoadState { char bitmap_name[256]; BlockDriverState *bs; BdrvDirtyBitmap *bitmap; + + GSList *enabled_bitmaps; + QemuMutex finish_lock; } DBMLoadState; -static DBMSaveState dirty_bitmap_mig_state; +typedef struct DBMState { + DBMSaveState save; + DBMLoadState load; +} DBMState; -/* State of one bitmap during load process */ -typedef struct LoadBitmapState { - BlockDriverState *bs; - BdrvDirtyBitmap *bitmap; - bool migrated; -} LoadBitmapState; -static GSList *enabled_bitmaps; -QemuMutex finish_lock; +static DBMState dbm_state; static uint32_t qemu_get_bitmap_flags(QEMUFile *f) { @@ -169,21 +174,21 @@ static void qemu_put_bitmap_flags(QEMUFile *f, uint32= _t flags) qemu_put_byte(f, flags); } -static void send_bitmap_header(QEMUFile *f, SaveBitmapState *dbms, - uint32_t additional_flags) +static void send_bitmap_header(QEMUFile *f, DBMSaveState *s, + SaveBitmapState *dbms, uint32_t additional_= flags) { BlockDriverState *bs =3D dbms->bs; BdrvDirtyBitmap *bitmap =3D dbms->bitmap; uint32_t flags =3D additional_flags; trace_send_bitmap_header_enter(); - if (bs !=3D dirty_bitmap_mig_state.prev_bs) { - dirty_bitmap_mig_state.prev_bs =3D bs; + if (bs !=3D s->prev_bs) { + s->prev_bs =3D bs; flags |=3D DIRTY_BITMAP_MIG_FLAG_DEVICE_NAME; } - if (bitmap !=3D dirty_bitmap_mig_state.prev_bitmap) { - dirty_bitmap_mig_state.prev_bitmap =3D bitmap; + if (bitmap !=3D s->prev_bitmap) { + s->prev_bitmap =3D bitmap; flags |=3D DIRTY_BITMAP_MIG_FLAG_BITMAP_NAME; } @@ -198,19 +203,22 @@ static void send_bitmap_header(QEMUFile *f, SaveBitma= pState *dbms, } } -static void send_bitmap_start(QEMUFile *f, SaveBitmapState *dbms) +static void send_bitmap_start(QEMUFile *f, DBMSaveState *s, + SaveBitmapState *dbms) { - send_bitmap_header(f, dbms, DIRTY_BITMAP_MIG_FLAG_START); + send_bitmap_header(f, s, dbms, DIRTY_BITMAP_MIG_FLAG_START); qemu_put_be32(f, bdrv_dirty_bitmap_granularity(dbms->bitmap)); qemu_put_byte(f, dbms->flags); } -static void send_bitmap_complete(QEMUFile *f, SaveBitmapState *dbms) +static void send_bitmap_complete(QEMUFile *f, DBMSaveState *s, + SaveBitmapState *dbms) { - send_bitmap_header(f, dbms, DIRTY_BITMAP_MIG_FLAG_COMPLETE); + send_bitmap_header(f, s, dbms, DIRTY_BITMAP_MIG_FLAG_COMPLETE); } -static void send_bitmap_bits(QEMUFile *f, SaveBitmapState *dbms, +static void send_bitmap_bits(QEMUFile *f, DBMSaveState *s, + SaveBitmapState *dbms, uint64_t start_sector, uint32_t nr_sectors) { /* align for buffer_is_zero() */ @@ -235,7 +243,7 @@ static void send_bitmap_bits(QEMUFile *f, SaveBitmapSta= te *dbms, trace_send_bitmap_bits(flags, start_sector, nr_sectors, buf_size); - send_bitmap_header(f, dbms, flags); + send_bitmap_header(f, s, dbms, flags); qemu_put_be64(f, start_sector); qemu_put_be32(f, nr_sectors); @@ -254,12 +262,12 @@ static void send_bitmap_bits(QEMUFile *f, SaveBitmapS= tate *dbms, } /* Called with iothread lock taken. */ -static void dirty_bitmap_do_save_cleanup(void) +static void dirty_bitmap_do_save_cleanup(DBMSaveState *s) { SaveBitmapState *dbms; - while ((dbms =3D QSIMPLEQ_FIRST(&dirty_bitmap_mig_state.dbms_list)) != =3D NULL) { - QSIMPLEQ_REMOVE_HEAD(&dirty_bitmap_mig_state.dbms_list, entry); + while ((dbms =3D QSIMPLEQ_FIRST(&s->dbms_list)) !=3D NULL) { + QSIMPLEQ_REMOVE_HEAD(&s->dbms_list, entry); bdrv_dirty_bitmap_set_busy(dbms->bitmap, false); bdrv_unref(dbms->bs); g_free(dbms); @@ -267,7 +275,8 @@ static void dirty_bitmap_do_save_cleanup(void) } /* Called with iothread lock taken. */ -static int add_bitmaps_to_list(BlockDriverState *bs, const char *bs_name) +static int add_bitmaps_to_list(DBMSaveState *s, BlockDriverState *bs, + const char *bs_name) { BdrvDirtyBitmap *bitmap; SaveBitmapState *dbms; @@ -322,25 +331,24 @@ static int add_bitmaps_to_list(BlockDriverState *bs, = const char *bs_name) dbms->flags |=3D DIRTY_BITMAP_MIG_START_FLAG_PERSISTENT; } - QSIMPLEQ_INSERT_TAIL(&dirty_bitmap_mig_state.dbms_list, - dbms, entry); + QSIMPLEQ_INSERT_TAIL(&s->dbms_list, dbms, entry); } return 0; } /* Called with iothread lock taken. */ -static int init_dirty_bitmap_migration(void) +static int init_dirty_bitmap_migration(DBMSaveState *s) { BlockDriverState *bs; SaveBitmapState *dbms; GHashTable *handled_by_blk =3D g_hash_table_new(NULL, NULL); BlockBackend *blk; - dirty_bitmap_mig_state.bulk_completed =3D false; - dirty_bitmap_mig_state.prev_bs =3D NULL; - dirty_bitmap_mig_state.prev_bitmap =3D NULL; - dirty_bitmap_mig_state.no_bitmaps =3D false; + s->bulk_completed =3D false; + s->prev_bs =3D NULL; + s->prev_bitmap =3D NULL; + s->no_bitmaps =3D false; /* * Use blockdevice name for direct (or filtered) children of named blo= ck @@ -369,7 +377,7 @@ static int init_dirty_bitmap_migration(void) } if (bs && bs->drv && !bs->drv->is_filter) { - if (add_bitmaps_to_list(bs, name)) { + if (add_bitmaps_to_list(s, bs, name)) { goto fail; } g_hash_table_add(handled_by_blk, bs); @@ -381,18 +389,18 @@ static int init_dirty_bitmap_migration(void) continue; } - if (add_bitmaps_to_list(bs, bdrv_get_node_name(bs))) { + if (add_bitmaps_to_list(s, bs, bdrv_get_node_name(bs))) { goto fail; } } /* unset migration flags here, to not roll back it */ - QSIMPLEQ_FOREACH(dbms, &dirty_bitmap_mig_state.dbms_list, entry) { + QSIMPLEQ_FOREACH(dbms, &s->dbms_list, entry) { bdrv_dirty_bitmap_skip_store(dbms->bitmap, true); } - if (QSIMPLEQ_EMPTY(&dirty_bitmap_mig_state.dbms_list)) { - dirty_bitmap_mig_state.no_bitmaps =3D true; + if (QSIMPLEQ_EMPTY(&s->dbms_list)) { + s->no_bitmaps =3D true; } g_hash_table_destroy(handled_by_blk); @@ -401,18 +409,19 @@ static int init_dirty_bitmap_migration(void) fail: g_hash_table_destroy(handled_by_blk); - dirty_bitmap_do_save_cleanup(); + dirty_bitmap_do_save_cleanup(s); return -1; } /* Called with no lock taken. */ -static void bulk_phase_send_chunk(QEMUFile *f, SaveBitmapState *dbms) +static void bulk_phase_send_chunk(QEMUFile *f, DBMSaveState *s, + SaveBitmapState *dbms) { uint32_t nr_sectors =3D MIN(dbms->total_sectors - dbms->cur_sector, dbms->sectors_per_chunk); - send_bitmap_bits(f, dbms, dbms->cur_sector, nr_sectors); + send_bitmap_bits(f, s, dbms, dbms->cur_sector, nr_sectors); dbms->cur_sector +=3D nr_sectors; if (dbms->cur_sector >=3D dbms->total_sectors) { @@ -421,61 +430,66 @@ static void bulk_phase_send_chunk(QEMUFile *f, SaveBi= tmapState *dbms) } /* Called with no lock taken. */ -static void bulk_phase(QEMUFile *f, bool limit) +static void bulk_phase(QEMUFile *f, DBMSaveState *s, bool limit) { SaveBitmapState *dbms; - QSIMPLEQ_FOREACH(dbms, &dirty_bitmap_mig_state.dbms_list, entry) { + QSIMPLEQ_FOREACH(dbms, &s->dbms_list, entry) { while (!dbms->bulk_completed) { - bulk_phase_send_chunk(f, dbms); + bulk_phase_send_chunk(f, s, dbms); if (limit && qemu_file_rate_limit(f)) { return; } } } - dirty_bitmap_mig_state.bulk_completed =3D true; + s->bulk_completed =3D true; } /* for SaveVMHandlers */ static void dirty_bitmap_save_cleanup(void *opaque) { - dirty_bitmap_do_save_cleanup(); + DBMSaveState *s =3D &((DBMState *)opaque)->save; + + dirty_bitmap_do_save_cleanup(s); } static int dirty_bitmap_save_iterate(QEMUFile *f, void *opaque) { + DBMSaveState *s =3D &((DBMState *)opaque)->save; + trace_dirty_bitmap_save_iterate(migration_in_postcopy()); - if (migration_in_postcopy() && !dirty_bitmap_mig_state.bulk_completed)= { - bulk_phase(f, true); + if (migration_in_postcopy() && !s->bulk_completed) { + bulk_phase(f, s, true); } qemu_put_bitmap_flags(f, DIRTY_BITMAP_MIG_FLAG_EOS); - return dirty_bitmap_mig_state.bulk_completed; + return s->bulk_completed; } /* Called with iothread lock taken. */ static int dirty_bitmap_save_complete(QEMUFile *f, void *opaque) { + DBMSaveState *s =3D &((DBMState *)opaque)->save; SaveBitmapState *dbms; trace_dirty_bitmap_save_complete_enter(); - if (!dirty_bitmap_mig_state.bulk_completed) { - bulk_phase(f, false); + if (!s->bulk_completed) { + bulk_phase(f, s, false); } - QSIMPLEQ_FOREACH(dbms, &dirty_bitmap_mig_state.dbms_list, entry) { - send_bitmap_complete(f, dbms); + QSIMPLEQ_FOREACH(dbms, &s->dbms_list, entry) { + send_bitmap_complete(f, s, dbms); } qemu_put_bitmap_flags(f, DIRTY_BITMAP_MIG_FLAG_EOS); trace_dirty_bitmap_save_complete_finish(); - dirty_bitmap_do_save_cleanup(); + dirty_bitmap_save_cleanup(opaque); return 0; } @@ -485,12 +499,13 @@ static void dirty_bitmap_save_pending(QEMUFile *f, vo= id *opaque, uint64_t *res_compatible, uint64_t *res_postcopy_only) { + DBMSaveState *s =3D &((DBMState *)opaque)->save; SaveBitmapState *dbms; uint64_t pending =3D 0; qemu_mutex_lock_iothread(); - QSIMPLEQ_FOREACH(dbms, &dirty_bitmap_mig_state.dbms_list, entry) { + QSIMPLEQ_FOREACH(dbms, &s->dbms_list, entry) { uint64_t gran =3D bdrv_dirty_bitmap_granularity(dbms->bitmap); uint64_t sectors =3D dbms->bulk_completed ? 0 : dbms->total_sectors - dbms->cur_sector; @@ -549,7 +564,7 @@ static int dirty_bitmap_load_start(QEMUFile *f, DBMLoad= State *s) b->bs =3D s->bs; b->bitmap =3D s->bitmap; b->migrated =3D false; - enabled_bitmaps =3D g_slist_prepend(enabled_bitmaps, b); + s->enabled_bitmaps =3D g_slist_prepend(s->enabled_bitmaps, b); } return 0; @@ -557,11 +572,12 @@ static int dirty_bitmap_load_start(QEMUFile *f, DBMLo= adState *s) void dirty_bitmap_mig_before_vm_start(void) { + DBMLoadState *s =3D &dbm_state.load; GSList *item; - qemu_mutex_lock(&finish_lock); + qemu_mutex_lock(&s->finish_lock); - for (item =3D enabled_bitmaps; item; item =3D g_slist_next(item)) { + for (item =3D s->enabled_bitmaps; item; item =3D g_slist_next(item)) { LoadBitmapState *b =3D item->data; if (b->migrated) { @@ -573,10 +589,10 @@ void dirty_bitmap_mig_before_vm_start(void) g_free(b); } - g_slist_free(enabled_bitmaps); - enabled_bitmaps =3D NULL; + g_slist_free(s->enabled_bitmaps); + s->enabled_bitmaps =3D NULL; - qemu_mutex_unlock(&finish_lock); + qemu_mutex_unlock(&s->finish_lock); } static void dirty_bitmap_load_complete(QEMUFile *f, DBMLoadState *s) @@ -585,9 +601,9 @@ static void dirty_bitmap_load_complete(QEMUFile *f, DBM= LoadState *s) trace_dirty_bitmap_load_complete(); bdrv_dirty_bitmap_deserialize_finish(s->bitmap); - qemu_mutex_lock(&finish_lock); + qemu_mutex_lock(&s->finish_lock); - for (item =3D enabled_bitmaps; item; item =3D g_slist_next(item)) { + for (item =3D s->enabled_bitmaps; item; item =3D g_slist_next(item)) { LoadBitmapState *b =3D item->data; if (b->bitmap =3D=3D s->bitmap) { @@ -598,7 +614,7 @@ static void dirty_bitmap_load_complete(QEMUFile *f, DBM= LoadState *s) if (bdrv_dirty_bitmap_has_successor(s->bitmap)) { bdrv_dirty_bitmap_lock(s->bitmap); - if (enabled_bitmaps =3D=3D NULL) { + if (s->enabled_bitmaps =3D=3D NULL) { /* in postcopy */ bdrv_reclaim_dirty_bitmap_locked(s->bitmap, &error_abort); bdrv_enable_dirty_bitmap_locked(s->bitmap); @@ -617,7 +633,7 @@ static void dirty_bitmap_load_complete(QEMUFile *f, DBM= LoadState *s) bdrv_dirty_bitmap_unlock(s->bitmap); } - qemu_mutex_unlock(&finish_lock); + qemu_mutex_unlock(&s->finish_lock); } static int dirty_bitmap_load_bits(QEMUFile *f, DBMLoadState *s) @@ -714,7 +730,7 @@ static int dirty_bitmap_load_header(QEMUFile *f, DBMLoa= dState *s) static int dirty_bitmap_load(QEMUFile *f, void *opaque, int version_id) { - static DBMLoadState s; + DBMLoadState *s =3D &((DBMState *)opaque)->load; int ret =3D 0; trace_dirty_bitmap_load_enter(); @@ -724,17 +740,17 @@ static int dirty_bitmap_load(QEMUFile *f, void *opaqu= e, int version_id) } do { - ret =3D dirty_bitmap_load_header(f, &s); + ret =3D dirty_bitmap_load_header(f, s); if (ret < 0) { return ret; } - if (s.flags & DIRTY_BITMAP_MIG_FLAG_START) { - ret =3D dirty_bitmap_load_start(f, &s); - } else if (s.flags & DIRTY_BITMAP_MIG_FLAG_COMPLETE) { - dirty_bitmap_load_complete(f, &s); - } else if (s.flags & DIRTY_BITMAP_MIG_FLAG_BITS) { - ret =3D dirty_bitmap_load_bits(f, &s); + if (s->flags & DIRTY_BITMAP_MIG_FLAG_START) { + ret =3D dirty_bitmap_load_start(f, s); + } else if (s->flags & DIRTY_BITMAP_MIG_FLAG_COMPLETE) { + dirty_bitmap_load_complete(f, s); + } else if (s->flags & DIRTY_BITMAP_MIG_FLAG_BITS) { + ret =3D dirty_bitmap_load_bits(f, s); } if (!ret) { @@ -744,7 +760,7 @@ static int dirty_bitmap_load(QEMUFile *f, void *opaque,= int version_id) if (ret) { return ret; } - } while (!(s.flags & DIRTY_BITMAP_MIG_FLAG_EOS)); + } while (!(s->flags & DIRTY_BITMAP_MIG_FLAG_EOS)); trace_dirty_bitmap_load_success(); return 0; @@ -752,13 +768,14 @@ static int dirty_bitmap_load(QEMUFile *f, void *opaqu= e, int version_id) static int dirty_bitmap_save_setup(QEMUFile *f, void *opaque) { + DBMSaveState *s =3D &((DBMState *)opaque)->save; SaveBitmapState *dbms =3D NULL; - if (init_dirty_bitmap_migration() < 0) { + if (init_dirty_bitmap_migration(s) < 0) { return -1; } - QSIMPLEQ_FOREACH(dbms, &dirty_bitmap_mig_state.dbms_list, entry) { - send_bitmap_start(f, dbms); + QSIMPLEQ_FOREACH(dbms, &s->dbms_list, entry) { + send_bitmap_start(f, s, dbms); } qemu_put_bitmap_flags(f, DIRTY_BITMAP_MIG_FLAG_EOS); @@ -767,7 +784,9 @@ static int dirty_bitmap_save_setup(QEMUFile *f, void *o= paque) static bool dirty_bitmap_is_active(void *opaque) { - return migrate_dirty_bitmaps() && !dirty_bitmap_mig_state.no_bitmaps; + DBMSaveState *s =3D &((DBMState *)opaque)->save; + + return migrate_dirty_bitmaps() && !s->no_bitmaps; } static bool dirty_bitmap_is_active_iterate(void *opaque) @@ -795,10 +814,10 @@ static SaveVMHandlers savevm_dirty_bitmap_handlers = =3D { void dirty_bitmap_mig_init(void) { - QSIMPLEQ_INIT(&dirty_bitmap_mig_state.dbms_list); - qemu_mutex_init(&finish_lock); + QSIMPLEQ_INIT(&dbm_state.save.dbms_list); + qemu_mutex_init(&dbm_state.load.finish_lock); register_savevm_live("dirty-bitmap", 0, 1, &savevm_dirty_bitmap_handlers, - &dirty_bitmap_mig_state); + &dbm_state); } --=20 2.27.0 From nobody Mon Feb 9 01:37:05 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1595883898; cv=none; d=zohomail.com; s=zohoarc; b=TJiD97g9gXtwfFrzxbM3DOf9DBp2m87JBThB1TO/RjJ5fD8X7aSENzqKy++ItQtUoojnUHlGse6ch44ACy53lcHQYj2F0shjhTlKv5fuFnTIRPGuWb8mPS0d3fmMvIVOq4D/EuVrxt8L67oJeQUJhYXnpfLcVqibQ58AdmkhojI= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1595883898; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=RhcK+hLL7eIFd4Jr8vV11mkyvJ6l8+i1wKkgojeo9Ag=; b=bqXRtHOx+pvw27hIi246O2ANdrOBYoFLzb1ZkNCvn5wSmNsm/BiLAgsHfm89uGxPXQ7dJlhe9aW4jYqY6PGdwrgktfhJIwczC6W5TS25aKwXaAiOBzTOtY8odn7KDQyim/FtxwQt6iKfhxKkTQYHzT6k/9+2ZmS8jEh/goAAPls= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1595883898907659.3041745071768; Mon, 27 Jul 2020 14:04:58 -0700 (PDT) Received: from localhost ([::1]:43190 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1k0AIr-0001eO-F8 for importer@patchew.org; Mon, 27 Jul 2020 17:04:57 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:35164) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1k0AAc-0005Su-Th for qemu-devel@nongnu.org; Mon, 27 Jul 2020 16:56:26 -0400 Received: from us-smtp-delivery-1.mimecast.com ([207.211.31.120]:52592 helo=us-smtp-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1k0AAZ-0004yM-RB for qemu-devel@nongnu.org; Mon, 27 Jul 2020 16:56:26 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-413-n6NUVXdVP_GGV0Vy_mI54g-1; Mon, 27 Jul 2020 16:56:17 -0400 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id E8CA1101C8A0; Mon, 27 Jul 2020 20:56:15 +0000 (UTC) Received: from blue.redhat.com (ovpn-118-248.rdu2.redhat.com [10.10.118.248]) by smtp.corp.redhat.com (Postfix) with ESMTP id E5AE719724; Mon, 27 Jul 2020 20:56:11 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1595883379; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=RhcK+hLL7eIFd4Jr8vV11mkyvJ6l8+i1wKkgojeo9Ag=; b=abUpdsDx3eaNFmMRMBX6pa3n/zYKZQs+Xzyjc3bsekWrE2orn4czz93JPvN8yNvOUoZIj8 ZBr9QMb24RK7rlgj0xovR65JtZAcYVji4W2fpA/M7KnzMAs/P84b/PggWPX+B9sk/BHJpT YI8C9BoZTVceGc+9qIh6A1J3AG2VShI= X-MC-Unique: n6NUVXdVP_GGV0Vy_mI54g-1 From: Eric Blake To: qemu-devel@nongnu.org Subject: [PULL 13/24] migration/block-dirty-bitmap: rename finish_lock to just lock Date: Mon, 27 Jul 2020 15:55:32 -0500 Message-Id: <20200727205543.206624-14-eblake@redhat.com> In-Reply-To: <20200727205543.206624-1-eblake@redhat.com> References: <20200727205543.206624-1-eblake@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.23 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=207.211.31.120; envelope-from=eblake@redhat.com; helo=us-smtp-1.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/07/27 15:02:33 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Spam_score_int: -30 X-Spam_score: -3.1 X-Spam_bar: --- X-Spam_report: (-3.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-1, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Fam Zheng , Vladimir Sementsov-Ogievskiy , "open list:Dirty Bitmaps" , Juan Quintela , "Dr. David Alan Gilbert" , Stefan Hajnoczi , Andrey Shinkevich , John Snow Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) Content-Type: text/plain; charset="utf-8" From: Vladimir Sementsov-Ogievskiy finish_lock is bad name, as lock used not only on process end. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Andrey Shinkevich Message-Id: <20200727194236.19551-13-vsementsov@virtuozzo.com> Signed-off-by: Eric Blake --- migration/block-dirty-bitmap.c | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/migration/block-dirty-bitmap.c b/migration/block-dirty-bitmap.c index 9b39e7aa2b4f..9194807b54f1 100644 --- a/migration/block-dirty-bitmap.c +++ b/migration/block-dirty-bitmap.c @@ -143,7 +143,7 @@ typedef struct DBMLoadState { BdrvDirtyBitmap *bitmap; GSList *enabled_bitmaps; - QemuMutex finish_lock; + QemuMutex lock; /* protect enabled_bitmaps */ } DBMLoadState; typedef struct DBMState { @@ -575,7 +575,7 @@ void dirty_bitmap_mig_before_vm_start(void) DBMLoadState *s =3D &dbm_state.load; GSList *item; - qemu_mutex_lock(&s->finish_lock); + qemu_mutex_lock(&s->lock); for (item =3D s->enabled_bitmaps; item; item =3D g_slist_next(item)) { LoadBitmapState *b =3D item->data; @@ -592,7 +592,7 @@ void dirty_bitmap_mig_before_vm_start(void) g_slist_free(s->enabled_bitmaps); s->enabled_bitmaps =3D NULL; - qemu_mutex_unlock(&s->finish_lock); + qemu_mutex_unlock(&s->lock); } static void dirty_bitmap_load_complete(QEMUFile *f, DBMLoadState *s) @@ -601,7 +601,7 @@ static void dirty_bitmap_load_complete(QEMUFile *f, DBM= LoadState *s) trace_dirty_bitmap_load_complete(); bdrv_dirty_bitmap_deserialize_finish(s->bitmap); - qemu_mutex_lock(&s->finish_lock); + qemu_mutex_lock(&s->lock); for (item =3D s->enabled_bitmaps; item; item =3D g_slist_next(item)) { LoadBitmapState *b =3D item->data; @@ -633,7 +633,7 @@ static void dirty_bitmap_load_complete(QEMUFile *f, DBM= LoadState *s) bdrv_dirty_bitmap_unlock(s->bitmap); } - qemu_mutex_unlock(&s->finish_lock); + qemu_mutex_unlock(&s->lock); } static int dirty_bitmap_load_bits(QEMUFile *f, DBMLoadState *s) @@ -815,7 +815,7 @@ static SaveVMHandlers savevm_dirty_bitmap_handlers =3D { void dirty_bitmap_mig_init(void) { QSIMPLEQ_INIT(&dbm_state.save.dbms_list); - qemu_mutex_init(&dbm_state.load.finish_lock); + qemu_mutex_init(&dbm_state.load.lock); register_savevm_live("dirty-bitmap", 0, 1, &savevm_dirty_bitmap_handlers, --=20 2.27.0 From nobody Mon Feb 9 01:37:05 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1595883982; cv=none; d=zohomail.com; s=zohoarc; b=KShjxPKPLtZgPWo4q+54gctcRhVDBXRFuHh8egt8/rsBTHV4KJQzfJnKgdA6a4iOg433nhtNErMyKLKW4o2XWtpKSDCKttwHHBcYsuYzyiFCtbO7VnW9dRaZ5zTP2bPm5Bh7591Pt/aC5rH3flU4iqgHj6zQDvbzbR+NO2aHfkI= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1595883982; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=uLYCusIIYVBIf0xiCXWTmILkEuREcQ5pTTnFopi18ZM=; b=OmwRok+VELOgRKc0lj2SgZEdPE33vpArIJGuUA6qnsnle3VgFfBCTN7DxoWqAan1ugRVCtXEw8fRV5UjMMMdpuAwaqmLAgm8ekpDzoTkIJN6rPmi42IBDxHOFL2EsI3pGJyGPaPApz4cTgnAHpXBPI4vxiGNgy8TKUqYmxh2aJE= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1595883982942222.7758932620012; Mon, 27 Jul 2020 14:06:22 -0700 (PDT) Received: from localhost ([::1]:51350 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1k0AKD-00053L-LI for importer@patchew.org; Mon, 27 Jul 2020 17:06:21 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:35192) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1k0AAd-0005V3-L5 for qemu-devel@nongnu.org; Mon, 27 Jul 2020 16:56:27 -0400 Received: from us-smtp-1.mimecast.com ([207.211.31.81]:24659 helo=us-smtp-delivery-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1k0AAZ-0004yW-RH for qemu-devel@nongnu.org; Mon, 27 Jul 2020 16:56:27 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-341-ypknWNVTPRO0pN_9js9nuA-1; Mon, 27 Jul 2020 16:56:18 -0400 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 683DB10059A9; Mon, 27 Jul 2020 20:56:17 +0000 (UTC) Received: from blue.redhat.com (ovpn-118-248.rdu2.redhat.com [10.10.118.248]) by smtp.corp.redhat.com (Postfix) with ESMTP id 3554D19724; Mon, 27 Jul 2020 20:56:16 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1595883380; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=uLYCusIIYVBIf0xiCXWTmILkEuREcQ5pTTnFopi18ZM=; b=XQrFrU8OU/SeCGTIJoZT56A2C96VBy4U+FCLsqUql+H4eOEl/Je8GwdUDZ8/Tnv5CANsc5 6jLJqH3pyvxuklHtq35bNFqg0pJZEU5SfyJp2jTb9j7EmAm2IIFxirPDJvytiJw6fGq0+4 9jVlsbQivkRHDk901WEKEaTq4icoSV4= X-MC-Unique: ypknWNVTPRO0pN_9js9nuA-1 From: Eric Blake To: qemu-devel@nongnu.org Subject: [PULL 14/24] migration/block-dirty-bitmap: simplify dirty_bitmap_load_complete Date: Mon, 27 Jul 2020 15:55:33 -0500 Message-Id: <20200727205543.206624-15-eblake@redhat.com> In-Reply-To: <20200727205543.206624-1-eblake@redhat.com> References: <20200727205543.206624-1-eblake@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.23 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=207.211.31.81; envelope-from=eblake@redhat.com; helo=us-smtp-delivery-1.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/07/27 03:37:14 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Spam_score_int: -30 X-Spam_score: -3.1 X-Spam_bar: --- X-Spam_report: (-3.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-1, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Fam Zheng , Vladimir Sementsov-Ogievskiy , "open list:Block I/O path" , Juan Quintela , "Dr. David Alan Gilbert" , Stefan Hajnoczi , Andrey Shinkevich , John Snow Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) Content-Type: text/plain; charset="utf-8" From: Vladimir Sementsov-Ogievskiy bdrv_enable_dirty_bitmap_locked() call does nothing, as if we are in postcopy, bitmap successor must be enabled, and reclaim operation will enable the bitmap. So, actually we need just call _reclaim_ in both if branches, and making differences only to add an assertion seems not really good. The logic becomes simple: on load complete we do reclaim and that's all. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Andrey Shinkevich Message-Id: <20200727194236.19551-14-vsementsov@virtuozzo.com> Signed-off-by: Eric Blake --- migration/block-dirty-bitmap.c | 25 ++++--------------------- 1 file changed, 4 insertions(+), 21 deletions(-) diff --git a/migration/block-dirty-bitmap.c b/migration/block-dirty-bitmap.c index 9194807b54f1..405a259296d9 100644 --- a/migration/block-dirty-bitmap.c +++ b/migration/block-dirty-bitmap.c @@ -603,6 +603,10 @@ static void dirty_bitmap_load_complete(QEMUFile *f, DB= MLoadState *s) qemu_mutex_lock(&s->lock); + if (bdrv_dirty_bitmap_has_successor(s->bitmap)) { + bdrv_reclaim_dirty_bitmap(s->bitmap, &error_abort); + } + for (item =3D s->enabled_bitmaps; item; item =3D g_slist_next(item)) { LoadBitmapState *b =3D item->data; @@ -612,27 +616,6 @@ static void dirty_bitmap_load_complete(QEMUFile *f, DB= MLoadState *s) } } - if (bdrv_dirty_bitmap_has_successor(s->bitmap)) { - bdrv_dirty_bitmap_lock(s->bitmap); - if (s->enabled_bitmaps =3D=3D NULL) { - /* in postcopy */ - bdrv_reclaim_dirty_bitmap_locked(s->bitmap, &error_abort); - bdrv_enable_dirty_bitmap_locked(s->bitmap); - } else { - /* target not started, successor must be empty */ - int64_t count =3D bdrv_get_dirty_count(s->bitmap); - BdrvDirtyBitmap *ret =3D bdrv_reclaim_dirty_bitmap_locked(s->b= itmap, - NULL); - /* bdrv_reclaim_dirty_bitmap can fail only on no successor (it - * must be) or on merge fail, but merge can't fail when second - * bitmap is empty - */ - assert(ret =3D=3D s->bitmap && - count =3D=3D bdrv_get_dirty_count(s->bitmap)); - } - bdrv_dirty_bitmap_unlock(s->bitmap); - } - qemu_mutex_unlock(&s->lock); } --=20 2.27.0 From nobody Mon Feb 9 01:37:05 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1595883773; cv=none; d=zohomail.com; s=zohoarc; b=RRazS8aoYHOuEnbkqnEZDs1tgDGnfQYHx6JMqU6HAKHUnxvr8g2OOwDFiNUODgcKoIeFkQSxUaBBktfqekDKZxm0BHKDlmDdFy/7w6jDelAk6+EtQVRFdTzfD0j0ZWi9ZbUbeGULKjSK6v0AUmHf3L2caxpKKi9QhD/WjnbVde4= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1595883773; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=kQPGEaurRWylMezQbdlsxycojMuLC7uNHt6+OKArsZ0=; b=klldkY8747MRb4GhBLM8ZX4+1S2IVAUcic6zFjDoBOk4uIjXM6O2eO5ah3chE//NdQMUv+jq6OIGk1uLZLEsy/iVX351Mvi6+65ek2g/jWWO/kZP14qSPyHfALqk1+POiei9nzbfbo5Z47IkTZQQM1BmATCKjc+lVBrGtiJBanw= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1595883773746820.7587249810208; Mon, 27 Jul 2020 14:02:53 -0700 (PDT) Received: from localhost ([::1]:35654 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1k0AGq-0006ym-De for importer@patchew.org; Mon, 27 Jul 2020 17:02:52 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:35198) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1k0AAd-0005Vn-Vz for qemu-devel@nongnu.org; Mon, 27 Jul 2020 16:56:28 -0400 Received: from us-smtp-delivery-1.mimecast.com ([205.139.110.120]:59247 helo=us-smtp-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1k0AAZ-0004yh-Qb for qemu-devel@nongnu.org; Mon, 27 Jul 2020 16:56:27 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-410-D64mk4X_MI6TZFIcNH0NIg-1; Mon, 27 Jul 2020 16:56:20 -0400 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 40CD51005510; Mon, 27 Jul 2020 20:56:19 +0000 (UTC) Received: from blue.redhat.com (ovpn-118-248.rdu2.redhat.com [10.10.118.248]) by smtp.corp.redhat.com (Postfix) with ESMTP id AD34919724; Mon, 27 Jul 2020 20:56:17 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1595883382; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=kQPGEaurRWylMezQbdlsxycojMuLC7uNHt6+OKArsZ0=; b=dAqYMS6d+N7R26qUmrgiUj8mMiVei1H6hbkVJUzjHy7ZuJgufE2h1zvxPUqeKF4n3kAKnZ giBOGDeXZuwcuxDJdmEIqXKTxVyjLbGx+WX+JKJCf7vfw3MBR0VKPZuQ+zPYhgBEgZxsnU E7SzhyDA8zyU54aIIIAH9KQt85Hm+pY= X-MC-Unique: D64mk4X_MI6TZFIcNH0NIg-1 From: Eric Blake To: qemu-devel@nongnu.org Subject: [PULL 15/24] migration/block-dirty-bitmap: keep bitmap state for all bitmaps Date: Mon, 27 Jul 2020 15:55:34 -0500 Message-Id: <20200727205543.206624-16-eblake@redhat.com> In-Reply-To: <20200727205543.206624-1-eblake@redhat.com> References: <20200727205543.206624-1-eblake@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.23 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=205.139.110.120; envelope-from=eblake@redhat.com; helo=us-smtp-1.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/07/27 01:44:14 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Spam_score_int: -40 X-Spam_score: -4.1 X-Spam_bar: ---- X-Spam_report: (-4.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-1, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-1, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Fam Zheng , Vladimir Sementsov-Ogievskiy , "open list:Dirty Bitmaps" , Juan Quintela , "Dr. David Alan Gilbert" , Stefan Hajnoczi , Andrey Shinkevich , John Snow Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) Content-Type: text/plain; charset="utf-8" From: Vladimir Sementsov-Ogievskiy Keep bitmap state for disabled bitmaps too. Keep the state until the end of the process. It's needed for the following commit to implement bitmap postcopy canceling. To clean-up the new list the following logic is used: We need two events to consider bitmap migration finished: 1. chunk with DIRTY_BITMAP_MIG_FLAG_COMPLETE flag should be received 2. dirty_bitmap_mig_before_vm_start should be called These two events may come in any order, so we understand which one is last, and on the last of them we remove bitmap migration state from the list. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Andrey Shinkevich Message-Id: <20200727194236.19551-15-vsementsov@virtuozzo.com> Signed-off-by: Eric Blake --- migration/block-dirty-bitmap.c | 64 +++++++++++++++++++++++----------- 1 file changed, 43 insertions(+), 21 deletions(-) diff --git a/migration/block-dirty-bitmap.c b/migration/block-dirty-bitmap.c index 405a259296d9..eb4ffeac4d1b 100644 --- a/migration/block-dirty-bitmap.c +++ b/migration/block-dirty-bitmap.c @@ -132,6 +132,7 @@ typedef struct LoadBitmapState { BlockDriverState *bs; BdrvDirtyBitmap *bitmap; bool migrated; + bool enabled; } LoadBitmapState; /* State of the dirty bitmap migration (DBM) during load process */ @@ -142,8 +143,10 @@ typedef struct DBMLoadState { BlockDriverState *bs; BdrvDirtyBitmap *bitmap; - GSList *enabled_bitmaps; - QemuMutex lock; /* protect enabled_bitmaps */ + bool before_vm_start_handled; /* set in dirty_bitmap_mig_before_vm_sta= rt */ + + GSList *bitmaps; + QemuMutex lock; /* protect bitmaps */ } DBMLoadState; typedef struct DBMState { @@ -526,6 +529,7 @@ static int dirty_bitmap_load_start(QEMUFile *f, DBMLoad= State *s) Error *local_err =3D NULL; uint32_t granularity =3D qemu_get_be32(f); uint8_t flags =3D qemu_get_byte(f); + LoadBitmapState *b; if (s->bitmap) { error_report("Bitmap with the same name ('%s') already exists on " @@ -552,45 +556,59 @@ static int dirty_bitmap_load_start(QEMUFile *f, DBMLo= adState *s) bdrv_disable_dirty_bitmap(s->bitmap); if (flags & DIRTY_BITMAP_MIG_START_FLAG_ENABLED) { - LoadBitmapState *b; - bdrv_dirty_bitmap_create_successor(s->bitmap, &local_err); if (local_err) { error_report_err(local_err); return -EINVAL; } - - b =3D g_new(LoadBitmapState, 1); - b->bs =3D s->bs; - b->bitmap =3D s->bitmap; - b->migrated =3D false; - s->enabled_bitmaps =3D g_slist_prepend(s->enabled_bitmaps, b); } + b =3D g_new(LoadBitmapState, 1); + b->bs =3D s->bs; + b->bitmap =3D s->bitmap; + b->migrated =3D false; + b->enabled =3D flags & DIRTY_BITMAP_MIG_START_FLAG_ENABLED; + + s->bitmaps =3D g_slist_prepend(s->bitmaps, b); + return 0; } -void dirty_bitmap_mig_before_vm_start(void) +/* + * before_vm_start_handle_item + * + * g_slist_foreach helper + * + * item is LoadBitmapState* + * opaque is DBMLoadState* + */ +static void before_vm_start_handle_item(void *item, void *opaque) { - DBMLoadState *s =3D &dbm_state.load; - GSList *item; - - qemu_mutex_lock(&s->lock); - - for (item =3D s->enabled_bitmaps; item; item =3D g_slist_next(item)) { - LoadBitmapState *b =3D item->data; + DBMLoadState *s =3D opaque; + LoadBitmapState *b =3D item; + if (b->enabled) { if (b->migrated) { bdrv_enable_dirty_bitmap(b->bitmap); } else { bdrv_dirty_bitmap_enable_successor(b->bitmap); } + } + if (b->migrated) { + s->bitmaps =3D g_slist_remove(s->bitmaps, b); g_free(b); } +} - g_slist_free(s->enabled_bitmaps); - s->enabled_bitmaps =3D NULL; +void dirty_bitmap_mig_before_vm_start(void) +{ + DBMLoadState *s =3D &dbm_state.load; + qemu_mutex_lock(&s->lock); + + assert(!s->before_vm_start_handled); + g_slist_foreach(s->bitmaps, before_vm_start_handle_item, s); + s->before_vm_start_handled =3D true; qemu_mutex_unlock(&s->lock); } @@ -607,11 +625,15 @@ static void dirty_bitmap_load_complete(QEMUFile *f, D= BMLoadState *s) bdrv_reclaim_dirty_bitmap(s->bitmap, &error_abort); } - for (item =3D s->enabled_bitmaps; item; item =3D g_slist_next(item)) { + for (item =3D s->bitmaps; item; item =3D g_slist_next(item)) { LoadBitmapState *b =3D item->data; if (b->bitmap =3D=3D s->bitmap) { b->migrated =3D true; + if (s->before_vm_start_handled) { + s->bitmaps =3D g_slist_remove(s->bitmaps, b); + g_free(b); + } break; } } --=20 2.27.0 From nobody Mon Feb 9 01:37:05 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1595883899; cv=none; d=zohomail.com; s=zohoarc; b=fODgyzJjmmTQUES3cF13SvZNXIKwLzKlWfiauCiuw5N9jEHSESn6x6h0t8mRPxVgifFzShCNMNh0m/0i4ttdfYAb6gihtpDSjvPOiXnHTGCy6T7Qa9VWtURF8lhFsia7F/loF74P3M5HktvCNpwSpsnYSuIDmidFy0LyldOKlmw= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1595883899; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=KyHjnLKpLvqYoE+oroXl2vtS0ava36K8D5QFtoAHegI=; b=Itmzbam7Zb0T/vajfx+uAC9fXG2NGhpFyuH8MxOaG5DcaMS6J6kzMB+wIJLnT7NJdiQp9N29m6C62uayaQozAO4UIc0yoxAu8sFaAckVb7gNjgtZSL/SLmFwqbsRc742W35OY9482QCI/0qqWJUxnAZaoIU53mCFitxY6qH8zN8= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1595883899585696.2929037037262; Mon, 27 Jul 2020 14:04:59 -0700 (PDT) Received: from localhost ([::1]:43210 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1k0AIs-0001eo-1p for importer@patchew.org; Mon, 27 Jul 2020 17:04:58 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:35230) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1k0AAg-0005ab-9k for qemu-devel@nongnu.org; Mon, 27 Jul 2020 16:56:30 -0400 Received: from us-smtp-1.mimecast.com ([207.211.31.81]:34388 helo=us-smtp-delivery-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1k0AAd-0004ze-NV for qemu-devel@nongnu.org; Mon, 27 Jul 2020 16:56:29 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-189-6fWG8R4GPqCKd4Fwc_fAZg-1; Mon, 27 Jul 2020 16:56:24 -0400 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 99B66101C8A0; Mon, 27 Jul 2020 20:56:23 +0000 (UTC) Received: from blue.redhat.com (ovpn-118-248.rdu2.redhat.com [10.10.118.248]) by smtp.corp.redhat.com (Postfix) with ESMTP id 867B319724; Mon, 27 Jul 2020 20:56:19 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1595883387; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=KyHjnLKpLvqYoE+oroXl2vtS0ava36K8D5QFtoAHegI=; b=OQjSm7Gfd3eHAQhU/AW3W20qnqVSOq2ThfUX1vl6/u3gAbmntRLZ1FiQyzPhANMl4ygd1T mqkrjrttHutLXob6ggNANVwqqXcRysF3AYsCojf4UB4/H82DL35B7bS99KNZQU3jYHil+c GsAQ2d9lyCsz3XjtiIKEfTy2V7FP/CQ= X-MC-Unique: 6fWG8R4GPqCKd4Fwc_fAZg-1 From: Eric Blake To: qemu-devel@nongnu.org Subject: [PULL 16/24] migration/block-dirty-bitmap: relax error handling in incoming part Date: Mon, 27 Jul 2020 15:55:35 -0500 Message-Id: <20200727205543.206624-17-eblake@redhat.com> In-Reply-To: <20200727205543.206624-1-eblake@redhat.com> References: <20200727205543.206624-1-eblake@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.23 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=207.211.31.81; envelope-from=eblake@redhat.com; helo=us-smtp-delivery-1.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/07/27 03:37:14 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Spam_score_int: -30 X-Spam_score: -3.1 X-Spam_bar: --- X-Spam_report: (-3.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-1, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Fam Zheng , Vladimir Sementsov-Ogievskiy , "open list:Block I/O path" , Juan Quintela , "Dr. David Alan Gilbert" , Stefan Hajnoczi , John Snow Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) Content-Type: text/plain; charset="utf-8" From: Vladimir Sementsov-Ogievskiy Bitmaps data is not critical, and we should not fail the migration (or use postcopy recovering) because of dirty-bitmaps migration failure. Instead we should just lose unfinished bitmaps. Still we have to report io stream violation errors, as they affect the whole migration stream. While touching this, tighten code that was previously blindly calling malloc on a size read from the migration stream, as a corrupted stream (perhaps from a malicious user) should not be able to convince us to allocate an inordinate amount of memory. Signed-off-by: Vladimir Sementsov-Ogievskiy Message-Id: <20200727194236.19551-16-vsementsov@virtuozzo.com> Reviewed-by: Eric Blake [eblake: typo fixes, enhance commit message] Signed-off-by: Eric Blake --- migration/block-dirty-bitmap.c | 162 +++++++++++++++++++++++++-------- 1 file changed, 126 insertions(+), 36 deletions(-) diff --git a/migration/block-dirty-bitmap.c b/migration/block-dirty-bitmap.c index eb4ffeac4d1b..f91015a4f88f 100644 --- a/migration/block-dirty-bitmap.c +++ b/migration/block-dirty-bitmap.c @@ -145,6 +145,15 @@ typedef struct DBMLoadState { bool before_vm_start_handled; /* set in dirty_bitmap_mig_before_vm_sta= rt */ + /* + * cancelled + * Incoming migration is cancelled for some reason. That means that we + * still should read our chunks from migration stream, to not affect o= ther + * migration objects (like RAM), but just ignore them and do not touch= any + * bitmaps or nodes. + */ + bool cancelled; + GSList *bitmaps; QemuMutex lock; /* protect bitmaps */ } DBMLoadState; @@ -531,6 +540,10 @@ static int dirty_bitmap_load_start(QEMUFile *f, DBMLoa= dState *s) uint8_t flags =3D qemu_get_byte(f); LoadBitmapState *b; + if (s->cancelled) { + return 0; + } + if (s->bitmap) { error_report("Bitmap with the same name ('%s') already exists on " "destination", bdrv_dirty_bitmap_name(s->bitmap)); @@ -613,14 +626,48 @@ void dirty_bitmap_mig_before_vm_start(void) qemu_mutex_unlock(&s->lock); } +static void cancel_incoming_locked(DBMLoadState *s) +{ + GSList *item; + + if (s->cancelled) { + return; + } + + s->cancelled =3D true; + s->bs =3D NULL; + s->bitmap =3D NULL; + + /* Drop all unfinished bitmaps */ + for (item =3D s->bitmaps; item; item =3D g_slist_next(item)) { + LoadBitmapState *b =3D item->data; + + /* + * Bitmap must be unfinished, as finished bitmaps should already be + * removed from the list. + */ + assert(!s->before_vm_start_handled || !b->migrated); + if (bdrv_dirty_bitmap_has_successor(b->bitmap)) { + bdrv_reclaim_dirty_bitmap(b->bitmap, &error_abort); + } + bdrv_release_dirty_bitmap(b->bitmap); + } + + g_slist_free_full(s->bitmaps, g_free); + s->bitmaps =3D NULL; +} + static void dirty_bitmap_load_complete(QEMUFile *f, DBMLoadState *s) { GSList *item; trace_dirty_bitmap_load_complete(); + + if (s->cancelled) { + return; + } + bdrv_dirty_bitmap_deserialize_finish(s->bitmap); - qemu_mutex_lock(&s->lock); - if (bdrv_dirty_bitmap_has_successor(s->bitmap)) { bdrv_reclaim_dirty_bitmap(s->bitmap, &error_abort); } @@ -637,8 +684,6 @@ static void dirty_bitmap_load_complete(QEMUFile *f, DBM= LoadState *s) break; } } - - qemu_mutex_unlock(&s->lock); } static int dirty_bitmap_load_bits(QEMUFile *f, DBMLoadState *s) @@ -650,15 +695,46 @@ static int dirty_bitmap_load_bits(QEMUFile *f, DBMLoa= dState *s) if (s->flags & DIRTY_BITMAP_MIG_FLAG_ZEROES) { trace_dirty_bitmap_load_bits_zeroes(); - bdrv_dirty_bitmap_deserialize_zeroes(s->bitmap, first_byte, nr_byt= es, - false); + if (!s->cancelled) { + bdrv_dirty_bitmap_deserialize_zeroes(s->bitmap, first_byte, + nr_bytes, false); + } } else { size_t ret; - uint8_t *buf; + g_autofree uint8_t *buf =3D NULL; uint64_t buf_size =3D qemu_get_be64(f); - uint64_t needed_size =3D - bdrv_dirty_bitmap_serialization_size(s->bitmap, - first_byte, nr_bytes); + uint64_t needed_size; + + /* + * The actual check for buf_size is done a bit later. We can't do = it in + * cancelled mode as we don't have the bitmap to check the constra= ints + * (so, we allocate a buffer and read prior to the check). On the = other + * hand, we shouldn't blindly g_malloc the number from the stream. + * Actually one chunk should not be larger than CHUNK_SIZE. Let's = allow + * a bit larger (which means that bitmap migration will fail anywa= y and + * the whole migration will most probably fail soon due to broken + * stream). + */ + if (buf_size > 10 * CHUNK_SIZE) { + error_report("Bitmap migration stream buffer allocation reques= t " + "is too large"); + return -EIO; + } + + buf =3D g_malloc(buf_size); + ret =3D qemu_get_buffer(f, buf, buf_size); + if (ret !=3D buf_size) { + error_report("Failed to read bitmap bits"); + return -EIO; + } + + if (s->cancelled) { + return 0; + } + + needed_size =3D bdrv_dirty_bitmap_serialization_size(s->bitmap, + first_byte, + nr_bytes); if (needed_size > buf_size || buf_size > QEMU_ALIGN_UP(needed_size, 4 * sizeof(long)) @@ -667,20 +743,12 @@ static int dirty_bitmap_load_bits(QEMUFile *f, DBMLoa= dState *s) error_report("Migrated bitmap granularity doesn't " "match the destination bitmap '%s' granularity", bdrv_dirty_bitmap_name(s->bitmap)); - return -EINVAL; - } - - buf =3D g_malloc(buf_size); - ret =3D qemu_get_buffer(f, buf, buf_size); - if (ret !=3D buf_size) { - error_report("Failed to read bitmap bits"); - g_free(buf); - return -EIO; + cancel_incoming_locked(s); + return 0; } bdrv_dirty_bitmap_deserialize_part(s->bitmap, buf, first_byte, nr_= bytes, false); - g_free(buf); } return 0; @@ -700,14 +768,16 @@ static int dirty_bitmap_load_header(QEMUFile *f, DBML= oadState *s) error_report("Unable to read node name string"); return -EINVAL; } - s->bs =3D bdrv_lookup_bs(s->node_name, s->node_name, &local_err); - if (!s->bs) { - error_report_err(local_err); - return -EINVAL; + if (!s->cancelled) { + s->bs =3D bdrv_lookup_bs(s->node_name, s->node_name, &local_er= r); + if (!s->bs) { + error_report_err(local_err); + cancel_incoming_locked(s); + } } - } else if (!s->bs && !nothing) { + } else if (!s->bs && !nothing && !s->cancelled) { error_report("Error: block device name is not set"); - return -EINVAL; + cancel_incoming_locked(s); } if (s->flags & DIRTY_BITMAP_MIG_FLAG_BITMAP_NAME) { @@ -715,24 +785,38 @@ static int dirty_bitmap_load_header(QEMUFile *f, DBML= oadState *s) error_report("Unable to read bitmap name string"); return -EINVAL; } - s->bitmap =3D bdrv_find_dirty_bitmap(s->bs, s->bitmap_name); + if (!s->cancelled) { + s->bitmap =3D bdrv_find_dirty_bitmap(s->bs, s->bitmap_name); - /* bitmap may be NULL here, it wouldn't be an error if it is the - * first occurrence of the bitmap */ - if (!s->bitmap && !(s->flags & DIRTY_BITMAP_MIG_FLAG_START)) { - error_report("Error: unknown dirty bitmap " - "'%s' for block device '%s'", - s->bitmap_name, s->node_name); - return -EINVAL; + /* + * bitmap may be NULL here, it wouldn't be an error if it is t= he + * first occurrence of the bitmap + */ + if (!s->bitmap && !(s->flags & DIRTY_BITMAP_MIG_FLAG_START)) { + error_report("Error: unknown dirty bitmap " + "'%s' for block device '%s'", + s->bitmap_name, s->node_name); + cancel_incoming_locked(s); + } } - } else if (!s->bitmap && !nothing) { + } else if (!s->bitmap && !nothing && !s->cancelled) { error_report("Error: block device name is not set"); - return -EINVAL; + cancel_incoming_locked(s); } return 0; } +/* + * dirty_bitmap_load + * + * Load sequence of dirty bitmap chunks. Return error only on fatal io str= eam + * violations. On other errors just cancel bitmaps incoming migration and = return + * 0. + * + * Note, than when incoming bitmap migration is canceled, we still must re= ad all + * our chunks (and just ignore them), to not affect other migration object= s. + */ static int dirty_bitmap_load(QEMUFile *f, void *opaque, int version_id) { DBMLoadState *s =3D &((DBMState *)opaque)->load; @@ -741,12 +825,17 @@ static int dirty_bitmap_load(QEMUFile *f, void *opaqu= e, int version_id) trace_dirty_bitmap_load_enter(); if (version_id !=3D 1) { + QEMU_LOCK_GUARD(&s->lock); + cancel_incoming_locked(s); return -EINVAL; } do { + QEMU_LOCK_GUARD(&s->lock); + ret =3D dirty_bitmap_load_header(f, s); if (ret < 0) { + cancel_incoming_locked(s); return ret; } @@ -763,6 +852,7 @@ static int dirty_bitmap_load(QEMUFile *f, void *opaque,= int version_id) } if (ret) { + cancel_incoming_locked(s); return ret; } } while (!(s->flags & DIRTY_BITMAP_MIG_FLAG_EOS)); --=20 2.27.0 From nobody Mon Feb 9 01:37:05 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1595884088; cv=none; d=zohomail.com; s=zohoarc; b=KpWyH//IcP6TRyYi27U9s9Nv70/+6v9GAmHpPIbcBkzL3rN4xqtwHTJAedMWROlagKmQoB48LRx7fF4nxbu2+Lz5k8u5ujkbM7XF5MmpB72gwVG+x7GggVrIhKQsVn5MIMU1DIWsdXkwozk1K2uWbm5B/iAbrWDO5jDZ4TnJ1jQ= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1595884088; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=zogdDvU2bj1UI7GHi+Qx4u1q8JzcQ4QUoc8I84kZS+g=; b=U4msyGazSFzfwZnzZGJn6F/RuxoE6NrMa8gmYdnCjTmfICoJ/K8bQFWvCSj73DaJ0hHw8J1SEZ11zhBBIAsLgZYcw+yuGqkeLmLD/xWmTJlkwA0VpIqBHZQu7sfqGvQmR4MxcfdYHkxH+FZgqz9fUQbbQl6tYfjb596BNQtREh0= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1595884088228392.5709451094634; Mon, 27 Jul 2020 14:08:08 -0700 (PDT) Received: from localhost ([::1]:58250 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1k0ALu-0007sk-Ol for importer@patchew.org; Mon, 27 Jul 2020 17:08:06 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:35282) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1k0AAj-0005gV-5B for qemu-devel@nongnu.org; Mon, 27 Jul 2020 16:56:33 -0400 Received: from us-smtp-delivery-1.mimecast.com ([207.211.31.120]:44230 helo=us-smtp-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1k0AAh-00050C-8u for qemu-devel@nongnu.org; Mon, 27 Jul 2020 16:56:32 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-55-6qgb5BfxNWmiRDZskimuTQ-1; Mon, 27 Jul 2020 16:56:26 -0400 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 23954101C8A7; Mon, 27 Jul 2020 20:56:25 +0000 (UTC) Received: from blue.redhat.com (ovpn-118-248.rdu2.redhat.com [10.10.118.248]) by smtp.corp.redhat.com (Postfix) with ESMTP id E11A019724; Mon, 27 Jul 2020 20:56:23 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1595883390; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=zogdDvU2bj1UI7GHi+Qx4u1q8JzcQ4QUoc8I84kZS+g=; b=YjmriHEvVN9UzJM2BZoNjMttIKHu3N+BkoEfKdfKZIOeiIGUxU+X4i/rGAIvijnSVBRecM wdkvep1VdHLMZHAOR4H864m0ztOLhXfWR+fS4iWo33BVQoTdqMW0kDBNNM2U6pszeLchsC 0NiSYt+XFNfae7yDf+rxuc+3RqXrAyw= X-MC-Unique: 6qgb5BfxNWmiRDZskimuTQ-1 From: Eric Blake To: qemu-devel@nongnu.org Subject: [PULL 17/24] migration/block-dirty-bitmap: cancel migration on shutdown Date: Mon, 27 Jul 2020 15:55:36 -0500 Message-Id: <20200727205543.206624-18-eblake@redhat.com> In-Reply-To: <20200727205543.206624-1-eblake@redhat.com> References: <20200727205543.206624-1-eblake@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.23 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=eblake@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=207.211.31.120; envelope-from=eblake@redhat.com; helo=us-smtp-1.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/07/27 15:02:33 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Spam_score_int: -30 X-Spam_score: -3.1 X-Spam_bar: --- X-Spam_report: (-3.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-1, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Fam Zheng , Vladimir Sementsov-Ogievskiy , "open list:Dirty Bitmaps" , Juan Quintela , "Dr. David Alan Gilbert" , Stefan Hajnoczi , Andrey Shinkevich , John Snow Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) Content-Type: text/plain; charset="utf-8" From: Vladimir Sementsov-Ogievskiy If target is turned off prior to postcopy finished, target crashes because busy bitmaps are found at shutdown. Canceling incoming migration helps, as it removes all unfinished (and therefore busy) bitmaps. Similarly on source we crash in bdrv_close_all which asserts that all bdrv states are removed, because bdrv states involved into dirty bitmap migration are referenced by it. So, we need to cancel outgoing migration as well. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Andrey Shinkevich Message-Id: <20200727194236.19551-17-vsementsov@virtuozzo.com> Signed-off-by: Eric Blake --- migration/migration.h | 2 ++ migration/block-dirty-bitmap.c | 16 ++++++++++++++++ migration/migration.c | 13 +++++++++++++ 3 files changed, 31 insertions(+) diff --git a/migration/migration.h b/migration/migration.h index ab20c756f549..6c6a931d0dc2 100644 --- a/migration/migration.h +++ b/migration/migration.h @@ -335,6 +335,8 @@ void migrate_send_rp_recv_bitmap(MigrationIncomingState= *mis, void migrate_send_rp_resume_ack(MigrationIncomingState *mis, uint32_t valu= e); void dirty_bitmap_mig_before_vm_start(void); +void dirty_bitmap_mig_cancel_outgoing(void); +void dirty_bitmap_mig_cancel_incoming(void); void migrate_add_address(SocketAddress *address); int foreach_not_ignored_block(RAMBlockIterFunc func, void *opaque); diff --git a/migration/block-dirty-bitmap.c b/migration/block-dirty-bitmap.c index f91015a4f88f..1f675b792fc9 100644 --- a/migration/block-dirty-bitmap.c +++ b/migration/block-dirty-bitmap.c @@ -657,6 +657,22 @@ static void cancel_incoming_locked(DBMLoadState *s) s->bitmaps =3D NULL; } +void dirty_bitmap_mig_cancel_outgoing(void) +{ + dirty_bitmap_do_save_cleanup(&dbm_state.save); +} + +void dirty_bitmap_mig_cancel_incoming(void) +{ + DBMLoadState *s =3D &dbm_state.load; + + qemu_mutex_lock(&s->lock); + + cancel_incoming_locked(s); + + qemu_mutex_unlock(&s->lock); +} + static void dirty_bitmap_load_complete(QEMUFile *f, DBMLoadState *s) { GSList *item; diff --git a/migration/migration.c b/migration/migration.c index 1c61428988e9..8fe36339dbe8 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -188,6 +188,19 @@ void migration_shutdown(void) */ migrate_fd_cancel(current_migration); object_unref(OBJECT(current_migration)); + + /* + * Cancel outgoing migration of dirty bitmaps. It should + * at least unref used block nodes. + */ + dirty_bitmap_mig_cancel_outgoing(); + + /* + * Cancel incoming migration of dirty bitmaps. Dirty bitmaps + * are non-critical data, and their loss never considered as + * something serious. + */ + dirty_bitmap_mig_cancel_incoming(); } /* For outgoing */ --=20 2.27.0 From nobody Mon Feb 9 01:37:05 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1595883932; cv=none; d=zohomail.com; s=zohoarc; b=TfI/oA0G3l4EGj+6/AZwyIEnI5haBEl3Hj2N+92kXZIo+QPSTyxH5LcGB7IbzKGcYRyBW2jrWKfxVuIzpRZQ2Z6yjIfBo5fdsyIduUpA6o3pH6kPmfwHP1/JHbNkauYAcFFOQFFgMUTPCsse7wezvFHDD2vrNL2tdJxUYYpdhYw= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1595883932; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=tHc0nQj+QkzuTabuTPWO7D1hcUTptMOxXx4vvq4SA+E=; b=EB/PqQrQn7LEeXeGE3KyxLmi7hVbP0FNdB8uiC3v1aI/TX9TKUhIzPk9v5EuHqFUnvqxNaCP/RSiab2DgtE1QbW4K1ZtBiraZ1pC1eIVjmabNJuTDjkHQMNh6VZZ7RUz5GsY0J8Tl1CL3+8IIxtp6bMs8L3Luy/Qf6tV92x5l9I= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1595883932190745.4070930536761; Mon, 27 Jul 2020 14:05:32 -0700 (PDT) Received: from localhost ([::1]:46196 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1k0AJO-0002t8-OL for importer@patchew.org; Mon, 27 Jul 2020 17:05:30 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:35332) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1k0AAk-0005jG-Af for qemu-devel@nongnu.org; Mon, 27 Jul 2020 16:56:34 -0400 Received: from us-smtp-delivery-1.mimecast.com ([205.139.110.120]:58505 helo=us-smtp-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1k0AAi-00050P-3Z for qemu-devel@nongnu.org; Mon, 27 Jul 2020 16:56:33 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-94-FEK2kFEsMJ6pBu0VvjupSg-1; Mon, 27 Jul 2020 16:56:27 -0400 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 31B42101C8A5; Mon, 27 Jul 2020 20:56:26 +0000 (UTC) Received: from blue.redhat.com (ovpn-118-248.rdu2.redhat.com [10.10.118.248]) by smtp.corp.redhat.com (Postfix) with ESMTP id 6865119C66; Mon, 27 Jul 2020 20:56:25 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1595883391; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=tHc0nQj+QkzuTabuTPWO7D1hcUTptMOxXx4vvq4SA+E=; b=OT8E3EYTTwrNRRSrKHUT9iISiuDEFivugy7VLXUgbENwFF300c4mO3gJfrGK6+U83HQ9pH ZtdPSCYDRyTENKWIa5YCzw/eke7pEx+pEVsfRTqy8zmZ7jg1lNDE3z4wocRlN7hOg06w0c 8PnMDrnDC2qJhlm5cDM/XAXOFP25n1A= X-MC-Unique: FEK2kFEsMJ6pBu0VvjupSg-1 From: Eric Blake To: qemu-devel@nongnu.org Subject: [PULL 18/24] migration/savevm: don't worry if bitmap migration postcopy failed Date: Mon, 27 Jul 2020 15:55:37 -0500 Message-Id: <20200727205543.206624-19-eblake@redhat.com> In-Reply-To: <20200727205543.206624-1-eblake@redhat.com> References: <20200727205543.206624-1-eblake@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.23 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=eblake@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=205.139.110.120; envelope-from=eblake@redhat.com; helo=us-smtp-1.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/07/27 01:44:14 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Spam_score_int: -40 X-Spam_score: -4.1 X-Spam_bar: ---- X-Spam_report: (-4.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-1, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-1, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Andrey Shinkevich , Vladimir Sementsov-Ogievskiy , "Dr . David Alan Gilbert" , Juan Quintela Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) Content-Type: text/plain; charset="utf-8" From: Vladimir Sementsov-Ogievskiy First, if only bitmaps postcopy is enabled (and not ram postcopy) postcopy_pause_incoming crashes on an assertion assert(mis->to_src_file). And anyway, bitmaps postcopy is not prepared to be somehow recovered. The original idea instead is that if bitmaps postcopy failed, we just lose some bitmaps, which is not critical. So, on failure we just need to remove unfinished bitmaps and guest should continue execution on destination. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Dr. David Alan Gilbert Reviewed-by: Andrey Shinkevich Reviewed-by: Eric Blake Message-Id: <20200727194236.19551-18-vsementsov@virtuozzo.com> Signed-off-by: Eric Blake --- migration/savevm.c | 37 ++++++++++++++++++++++++++++++++----- 1 file changed, 32 insertions(+), 5 deletions(-) diff --git a/migration/savevm.c b/migration/savevm.c index 45c9dd9d8a6d..a843d202b5b4 100644 --- a/migration/savevm.c +++ b/migration/savevm.c @@ -1813,6 +1813,9 @@ static void *postcopy_ram_listen_thread(void *opaque) MigrationIncomingState *mis =3D migration_incoming_get_current(); QEMUFile *f =3D mis->from_src_file; int load_res; + MigrationState *migr =3D migrate_get_current(); + + object_ref(OBJECT(migr)); migrate_set_state(&mis->state, MIGRATION_STATUS_ACTIVE, MIGRATION_STATUS_POSTCOPY_ACTIVE); @@ -1839,11 +1842,24 @@ static void *postcopy_ram_listen_thread(void *opaqu= e) trace_postcopy_ram_listen_thread_exit(); if (load_res < 0) { - error_report("%s: loadvm failed: %d", __func__, load_res); qemu_file_set_error(f, load_res); - migrate_set_state(&mis->state, MIGRATION_STATUS_POSTCOPY_ACTIVE, - MIGRATION_STATUS_FAILED); - } else { + dirty_bitmap_mig_cancel_incoming(); + if (postcopy_state_get() =3D=3D POSTCOPY_INCOMING_RUNNING && + !migrate_postcopy_ram() && migrate_dirty_bitmaps()) + { + error_report("%s: loadvm failed during postcopy: %d. All state= s " + "are migrated except dirty bitmaps. Some dirty " + "bitmaps may be lost, and present migrated dirty " + "bitmaps are correctly migrated and valid.", + __func__, load_res); + load_res =3D 0; /* prevent further exit() */ + } else { + error_report("%s: loadvm failed: %d", __func__, load_res); + migrate_set_state(&mis->state, MIGRATION_STATUS_POSTCOPY_ACTIV= E, + MIGRATION_STATUS_FAILED); + } + } + if (load_res >=3D 0) { /* * This looks good, but it's possible that the device loading in t= he * main thread hasn't finished yet, and so we might not be in 'RUN' @@ -1879,6 +1895,8 @@ static void *postcopy_ram_listen_thread(void *opaque) mis->have_listen_thread =3D false; postcopy_state_set(POSTCOPY_INCOMING_END); + object_unref(OBJECT(migr)); + return NULL; } @@ -2437,6 +2455,8 @@ static bool postcopy_pause_incoming(MigrationIncoming= State *mis) { trace_postcopy_pause_incoming(); + assert(migrate_postcopy_ram()); + /* Clear the triggered bit to allow one recovery */ mis->postcopy_recover_triggered =3D false; @@ -2521,15 +2541,22 @@ out: if (ret < 0) { qemu_file_set_error(f, ret); + /* Cancel bitmaps incoming regardless of recovery */ + dirty_bitmap_mig_cancel_incoming(); + /* * If we are during an active postcopy, then we pause instead * of bail out to at least keep the VM's dirty data. Note * that POSTCOPY_INCOMING_LISTENING stage is still not enough, * during which we're still receiving device states and we * still haven't yet started the VM on destination. + * + * Only RAM postcopy supports recovery. Still, if RAM postcopy is + * enabled, canceled bitmaps postcopy will not affect RAM postcopy + * recovering. */ if (postcopy_state_get() =3D=3D POSTCOPY_INCOMING_RUNNING && - postcopy_pause_incoming(mis)) { + migrate_postcopy_ram() && postcopy_pause_incoming(mis)) { /* Reset f to point to the newly created channel */ f =3D mis->from_src_file; goto retry; --=20 2.27.0 From nobody Mon Feb 9 01:37:05 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1595884167; cv=none; d=zohomail.com; s=zohoarc; b=j21Nw8D628yb6nY2J6fMbOOXvSAGOfy2cnADLJQux+8vg9nM1+Jr9vyG1N5/UpTcKbCYFeY/woiH4Cna/xyVypYKceSup8ZWgAopO3osNzcrh4sFclPNICpX5dvlqPlUmVE595T77qz3HyNwBD0YtyeQfIomwJIJaPKM/z6mtJ0= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1595884167; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=4YU+2DllB+WRMBKkKdzwQniGyf8W+ZI6W9hF8rTdQIM=; b=YuxzD/y+kB2asplzE8H3d/XBQjAqeRLVriTrYrBHHKD6sfsR/M+5jUgsu5lgwZNpsMn1d9T/qHdusuJP75msbQw9T1NY/0P/6nWPKwcI8ke0sMy9Fr+GAqtAFHc/PN0P7AtUrrJw7fwcWjv3Ov6V5rMHPwhnni5fw8Yw/LRZ6UY= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1595884167253550.6709588419179; Mon, 27 Jul 2020 14:09:27 -0700 (PDT) Received: from localhost ([::1]:35826 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1k0ANC-0001q2-2w for importer@patchew.org; Mon, 27 Jul 2020 17:09:26 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:35374) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1k0AAl-0005lk-Jc for qemu-devel@nongnu.org; Mon, 27 Jul 2020 16:56:35 -0400 Received: from us-smtp-delivery-1.mimecast.com ([207.211.31.120]:31642 helo=us-smtp-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1k0AAj-00050l-Bp for qemu-devel@nongnu.org; Mon, 27 Jul 2020 16:56:35 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-181-z3Y2Q7ZUMDWmS8rYQyh41g-1; Mon, 27 Jul 2020 16:56:28 -0400 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 56B4E80183C; Mon, 27 Jul 2020 20:56:27 +0000 (UTC) Received: from blue.redhat.com (ovpn-118-248.rdu2.redhat.com [10.10.118.248]) by smtp.corp.redhat.com (Postfix) with ESMTP id 7299619C66; Mon, 27 Jul 2020 20:56:26 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1595883392; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=4YU+2DllB+WRMBKkKdzwQniGyf8W+ZI6W9hF8rTdQIM=; b=HhuIT7JpBbgoF0SV0TFbxDL790Wj62ycCVB0L4JP5UH9+MctCOnrl5pGc0voSG7nRtdHj3 WpmVrpYR5zkaSc12Em49TtPLI5PY003v6g7bPpPl4Fbzj+GFY9wM17W8pkzsfxfRAlcUJU b03yvsOBZMOl3lxBsrpFtzGFzIeGm1Q= X-MC-Unique: z3Y2Q7ZUMDWmS8rYQyh41g-1 From: Eric Blake To: qemu-devel@nongnu.org Subject: [PULL 19/24] qemu-iotests/199: prepare for new test-cases addition Date: Mon, 27 Jul 2020 15:55:38 -0500 Message-Id: <20200727205543.206624-20-eblake@redhat.com> In-Reply-To: <20200727205543.206624-1-eblake@redhat.com> References: <20200727205543.206624-1-eblake@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.23 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=207.211.31.120; envelope-from=eblake@redhat.com; helo=us-smtp-1.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/07/27 15:02:33 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Spam_score_int: -30 X-Spam_score: -3.1 X-Spam_bar: --- X-Spam_report: (-3.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-1, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Andrey Shinkevich , Vladimir Sementsov-Ogievskiy , "open list:Block layer core" , Max Reitz Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) Content-Type: text/plain; charset="utf-8" From: Vladimir Sementsov-Ogievskiy Move future common part to start_postcopy() method. Move checking number of bitmaps to check_bitmap(). Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Andrey Shinkevich Message-Id: <20200727194236.19551-19-vsementsov@virtuozzo.com> Signed-off-by: Eric Blake --- tests/qemu-iotests/199 | 36 +++++++++++++++++++++++------------- 1 file changed, 23 insertions(+), 13 deletions(-) diff --git a/tests/qemu-iotests/199 b/tests/qemu-iotests/199 index d8532e49da00..355c0b288592 100755 --- a/tests/qemu-iotests/199 +++ b/tests/qemu-iotests/199 @@ -29,6 +29,8 @@ disk_b =3D os.path.join(iotests.test_dir, 'disk_b') size =3D '256G' fifo =3D os.path.join(iotests.test_dir, 'mig_fifo') +granularity =3D 512 +nb_bitmaps =3D 15 GiB =3D 1024 * 1024 * 1024 @@ -61,6 +63,15 @@ def event_dist(e1, e2): return event_seconds(e2) - event_seconds(e1) +def check_bitmaps(vm, count): + result =3D vm.qmp('query-block') + + if count =3D=3D 0: + assert 'dirty-bitmaps' not in result['return'][0] + else: + assert len(result['return'][0]['dirty-bitmaps']) =3D=3D count + + class TestDirtyBitmapPostcopyMigration(iotests.QMPTestCase): def tearDown(self): if debug: @@ -101,10 +112,8 @@ class TestDirtyBitmapPostcopyMigration(iotests.QMPTest= Case): self.vm_a_events =3D [] self.vm_b_events =3D [] - def test_postcopy(self): - granularity =3D 512 - nb_bitmaps =3D 15 - + def start_postcopy(self): + """ Run migration until RESUME event on target. Return this event.= """ for i in range(nb_bitmaps): result =3D self.vm_a.qmp('block-dirty-bitmap-add', node=3D'dri= ve0', name=3D'bitmap{}'.format(i), @@ -119,10 +128,10 @@ class TestDirtyBitmapPostcopyMigration(iotests.QMPTes= tCase): result =3D self.vm_a.qmp('x-debug-block-dirty-bitmap-sha256', node=3D'drive0', name=3D'bitmap0') - discards1_sha256 =3D result['return']['sha256'] + self.discards1_sha256 =3D result['return']['sha256'] # Check, that updating the bitmap by discards works - assert discards1_sha256 !=3D empty_sha256 + assert self.discards1_sha256 !=3D empty_sha256 # We want to calculate resulting sha256. Do it in bitmap0, so, dis= able # other bitmaps @@ -135,7 +144,7 @@ class TestDirtyBitmapPostcopyMigration(iotests.QMPTestC= ase): result =3D self.vm_a.qmp('x-debug-block-dirty-bitmap-sha256', node=3D'drive0', name=3D'bitmap0') - all_discards_sha256 =3D result['return']['sha256'] + self.all_discards_sha256 =3D result['return']['sha256'] # Now, enable some bitmaps, to be updated during migration for i in range(2, nb_bitmaps, 2): @@ -160,6 +169,10 @@ class TestDirtyBitmapPostcopyMigration(iotests.QMPTest= Case): event_resume =3D self.vm_b.event_wait('RESUME') self.vm_b_events.append(event_resume) + return event_resume + + def test_postcopy_success(self): + event_resume =3D self.start_postcopy() # enabled bitmaps should be updated apply_discards(self.vm_b, discards2) @@ -180,18 +193,15 @@ class TestDirtyBitmapPostcopyMigration(iotests.QMPTes= tCase): print('downtime:', downtime) print('postcopy_time:', postcopy_time) - # Assert that bitmap migration is finished (check that successor b= itmap - # is removed) - result =3D self.vm_b.qmp('query-block') - assert len(result['return'][0]['dirty-bitmaps']) =3D=3D nb_bitmaps + check_bitmaps(self.vm_b, nb_bitmaps) # Check content of migrated bitmaps. Still, don't waste time check= ing # every bitmap for i in range(0, nb_bitmaps, 5): result =3D self.vm_b.qmp('x-debug-block-dirty-bitmap-sha256', node=3D'drive0', name=3D'bitmap{}'.form= at(i)) - sha256 =3D discards1_sha256 if i % 2 else all_discards_sha256 - self.assert_qmp(result, 'return/sha256', sha256) + sha =3D self.discards1_sha256 if i % 2 else self.all_discards_= sha256 + self.assert_qmp(result, 'return/sha256', sha) if __name__ =3D=3D '__main__': --=20 2.27.0 From nobody Mon Feb 9 01:37:05 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1595883984; cv=none; d=zohomail.com; s=zohoarc; b=g8uak9w0LMlxDgggDzYhMFkcbM0D5gHKEoWproqkSK5juq9VgJJC1lGhaJNl33WlPP+1FXIUhb9Jq93cXLT1oO0u6QjFwIXA1WNFIluyRMqpzW9boJmOsblExBG0qlg1mn2yiWZlVmEOxcvp518JpY7g6OVRBVSV/s641CM5iIU= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1595883984; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=E0SsZalrTNJQpz+pmftCyRtrAIMx1wRHI8kl7yGvixY=; b=I0LQeVBVruElIcRPRPiUAjZqjrW6wdDvy7xlCOe7swNSFo9dEfdWcITewhy4isC1NP4RmfkHOHYvKEDMlBoe2fOfG0rj3D1P6qmWuUrXe+VLv+U6wzh+OVlVN1nDJ1yDH7zc7gt5TJWEznQCJrTAceASvV1OQ9G+YfHQrH6Wu4U= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1595883984633404.03318013127625; Mon, 27 Jul 2020 14:06:24 -0700 (PDT) Received: from localhost ([::1]:51520 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1k0AKF-00057t-EX for importer@patchew.org; Mon, 27 Jul 2020 17:06:23 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:35354) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1k0AAk-0005kM-U8 for qemu-devel@nongnu.org; Mon, 27 Jul 2020 16:56:35 -0400 Received: from us-smtp-delivery-1.mimecast.com ([207.211.31.120]:38909 helo=us-smtp-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1k0AAi-00050W-PO for qemu-devel@nongnu.org; Mon, 27 Jul 2020 16:56:34 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-67-uOGbgQnnOkCJ2khiIWhinw-1; Mon, 27 Jul 2020 16:56:29 -0400 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 7D764186A822; Mon, 27 Jul 2020 20:56:28 +0000 (UTC) Received: from blue.redhat.com (ovpn-118-248.rdu2.redhat.com [10.10.118.248]) by smtp.corp.redhat.com (Postfix) with ESMTP id 99CC319C66; Mon, 27 Jul 2020 20:56:27 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1595883391; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=E0SsZalrTNJQpz+pmftCyRtrAIMx1wRHI8kl7yGvixY=; b=R1iVpES4/dA4iaVFHamkTWPcs0rjsvMhXOI6YSGTr5n5GDdu6bKWnN8npe0nxsIk0VgXft a/8gJYYPrsSKaHQK27lUwz+34z1gneQZkO2eiYCa9mwVrUIDnQCW8c+jW1hWWT8TFZYrf1 CqM6v/P55iZYnrlwqt7Qt4JVDluwB2Y= X-MC-Unique: uOGbgQnnOkCJ2khiIWhinw-1 From: Eric Blake To: qemu-devel@nongnu.org Subject: [PULL 20/24] qemu-iotests/199: check persistent bitmaps Date: Mon, 27 Jul 2020 15:55:39 -0500 Message-Id: <20200727205543.206624-21-eblake@redhat.com> In-Reply-To: <20200727205543.206624-1-eblake@redhat.com> References: <20200727205543.206624-1-eblake@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.23 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=eblake@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=207.211.31.120; envelope-from=eblake@redhat.com; helo=us-smtp-1.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/07/27 15:02:33 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Spam_score_int: -30 X-Spam_score: -3.1 X-Spam_bar: --- X-Spam_report: (-3.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-1, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Andrey Shinkevich , Vladimir Sementsov-Ogievskiy , "open list:Block layer core" , Max Reitz Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) Content-Type: text/plain; charset="utf-8" From: Vladimir Sementsov-Ogievskiy Check that persistent bitmaps are not stored on source and that bitmaps are persistent on destination. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Andrey Shinkevich Message-Id: <20200727194236.19551-20-vsementsov@virtuozzo.com> Signed-off-by: Eric Blake --- tests/qemu-iotests/199 | 16 +++++++++++++++- 1 file changed, 15 insertions(+), 1 deletion(-) diff --git a/tests/qemu-iotests/199 b/tests/qemu-iotests/199 index 355c0b288592..5fd34f0fcdfa 100755 --- a/tests/qemu-iotests/199 +++ b/tests/qemu-iotests/199 @@ -117,7 +117,8 @@ class TestDirtyBitmapPostcopyMigration(iotests.QMPTestC= ase): for i in range(nb_bitmaps): result =3D self.vm_a.qmp('block-dirty-bitmap-add', node=3D'dri= ve0', name=3D'bitmap{}'.format(i), - granularity=3Dgranularity) + granularity=3Dgranularity, + persistent=3DTrue) self.assert_qmp(result, 'return', {}) result =3D self.vm_a.qmp('x-debug-block-dirty-bitmap-sha256', @@ -193,6 +194,19 @@ class TestDirtyBitmapPostcopyMigration(iotests.QMPTest= Case): print('downtime:', downtime) print('postcopy_time:', postcopy_time) + # check that there are no bitmaps stored on source + self.vm_a_events +=3D self.vm_a.get_qmp_events() + self.vm_a.shutdown() + self.vm_a.launch() + check_bitmaps(self.vm_a, 0) + + # check that bitmaps are migrated and persistence works + check_bitmaps(self.vm_b, nb_bitmaps) + self.vm_b.shutdown() + # recreate vm_b, so there is no incoming option, which prevents + # loading bitmaps from disk + self.vm_b =3D iotests.VM(path_suffix=3D'b').add_drive(disk_b) + self.vm_b.launch() check_bitmaps(self.vm_b, nb_bitmaps) # Check content of migrated bitmaps. Still, don't waste time check= ing --=20 2.27.0 From nobody Mon Feb 9 01:37:05 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1595884089; cv=none; d=zohomail.com; s=zohoarc; b=mcDnx6QWtqUtLDfJIbnlYjbeaGY9iHgaNQ2XzYmlQzCj8XZ7YcUpw7UyD8ftjUlkQpQrDZ+d/9CYFf8DmCdcxtkwbEzFtsbCQNN6FyBPp5wnVrh1txneyzsDd4VQvineZfMHYz6u3lzj/fJqw+HWflIAIhCmwss3mVE5TlAM9sk= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1595884089; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=RaLpXHOjSOyF8h+j3s+lrxRvzgSS2ZL0ZYdygl711Ys=; b=UkLBfFy690VamU0wWxxibWtKCecBDrHqU6bsag1mjcnWjJjmexyOGLg8KNjlhMHW3nqShnZPslrwDW+RYTzZgOrP4G2Q9q4RQDsUD3OImfrI0eg31xn+JCb6n3ZJmNFBCiqbIoEoTjhhxqwvOK+aFibKh5dy4RloGctpg4HIVbs= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1595884089863524.5324912063888; Mon, 27 Jul 2020 14:08:09 -0700 (PDT) Received: from localhost ([::1]:58368 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1k0ALw-0007vg-97 for importer@patchew.org; Mon, 27 Jul 2020 17:08:08 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:35378) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1k0AAl-0005mE-RF for qemu-devel@nongnu.org; Mon, 27 Jul 2020 16:56:35 -0400 Received: from us-smtp-delivery-1.mimecast.com ([205.139.110.120]:39331 helo=us-smtp-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1k0AAj-000525-Ta for qemu-devel@nongnu.org; Mon, 27 Jul 2020 16:56:35 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-436-kH-Ra7i4MQSfxsnWo7Jy4A-1; Mon, 27 Jul 2020 16:56:30 -0400 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id A2AD9186A838; Mon, 27 Jul 2020 20:56:29 +0000 (UTC) Received: from blue.redhat.com (ovpn-118-248.rdu2.redhat.com [10.10.118.248]) by smtp.corp.redhat.com (Postfix) with ESMTP id BE55619C66; Mon, 27 Jul 2020 20:56:28 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1595883393; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=RaLpXHOjSOyF8h+j3s+lrxRvzgSS2ZL0ZYdygl711Ys=; b=W5FIRXSZ1SwzvGO3K7pCM3Mht8Ud+VpI46fbXsUC2Ywv6+1Hv9cKfcTJ3oHFRoY+QJLgMH YaMKPQ01o5r034jG19SVDqPuXCw7s8tR0sJyTPSujj6NLyAWrvefMczBCNOOEFLYZ59Z2J lJvFyYX+RSRZFh8VnUymF0aiTo9ycRg= X-MC-Unique: kH-Ra7i4MQSfxsnWo7Jy4A-1 From: Eric Blake To: qemu-devel@nongnu.org Subject: [PULL 21/24] qemu-iotests/199: add early shutdown case to bitmaps postcopy Date: Mon, 27 Jul 2020 15:55:40 -0500 Message-Id: <20200727205543.206624-22-eblake@redhat.com> In-Reply-To: <20200727205543.206624-1-eblake@redhat.com> References: <20200727205543.206624-1-eblake@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.23 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=205.139.110.120; envelope-from=eblake@redhat.com; helo=us-smtp-1.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/07/27 01:44:14 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Spam_score_int: -40 X-Spam_score: -4.1 X-Spam_bar: ---- X-Spam_report: (-4.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-1, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-1, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Andrey Shinkevich , Vladimir Sementsov-Ogievskiy , "open list:Block layer core" , Max Reitz Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) Content-Type: text/plain; charset="utf-8" From: Vladimir Sementsov-Ogievskiy Previous patches fixed two crashes which may occur on shutdown prior to bitmaps postcopy finished. Check that it works now. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Andrey Shinkevich Tested-by: Eric Blake Message-Id: <20200727194236.19551-21-vsementsov@virtuozzo.com> Signed-off-by: Eric Blake --- tests/qemu-iotests/199 | 24 ++++++++++++++++++++++++ tests/qemu-iotests/199.out | 4 ++-- 2 files changed, 26 insertions(+), 2 deletions(-) diff --git a/tests/qemu-iotests/199 b/tests/qemu-iotests/199 index 5fd34f0fcdfa..140930b2b12e 100755 --- a/tests/qemu-iotests/199 +++ b/tests/qemu-iotests/199 @@ -217,6 +217,30 @@ class TestDirtyBitmapPostcopyMigration(iotests.QMPTest= Case): sha =3D self.discards1_sha256 if i % 2 else self.all_discards_= sha256 self.assert_qmp(result, 'return/sha256', sha) + def test_early_shutdown_destination(self): + self.start_postcopy() + + self.vm_b_events +=3D self.vm_b.get_qmp_events() + self.vm_b.shutdown() + # recreate vm_b, so there is no incoming option, which prevents + # loading bitmaps from disk + self.vm_b =3D iotests.VM(path_suffix=3D'b').add_drive(disk_b) + self.vm_b.launch() + check_bitmaps(self.vm_b, 0) + + # Bitmaps will be lost if we just shutdown the vm, as they are mar= ked + # to skip storing to disk when prepared for migration. And that's + # correct, as actual data may be modified in target vm, so we play + # safe. + # Still, this mark would be taken away if we do 'cont', and bitmaps + # become persistent again. (see iotest 169 for such behavior case) + result =3D self.vm_a.qmp('query-status') + assert not result['return']['running'] + self.vm_a_events +=3D self.vm_a.get_qmp_events() + self.vm_a.shutdown() + self.vm_a.launch() + check_bitmaps(self.vm_a, 0) + if __name__ =3D=3D '__main__': iotests.main(supported_fmts=3D['qcow2']) diff --git a/tests/qemu-iotests/199.out b/tests/qemu-iotests/199.out index ae1213e6f863..fbc63e62f885 100644 --- a/tests/qemu-iotests/199.out +++ b/tests/qemu-iotests/199.out @@ -1,5 +1,5 @@ -. +.. ---------------------------------------------------------------------- -Ran 1 tests +Ran 2 tests OK --=20 2.27.0 From nobody Mon Feb 9 01:37:05 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1595884263; cv=none; d=zohomail.com; s=zohoarc; b=cEECZstESt0vHHGD6VjO09TEZJR5XCzkbvIbhJTV6zkrC0cWaez2qwAK/oKtsEgPkuERP+5c/+w7idex8ZzQiNQAT37t+8slZAS9Rd8+ElOtU1r7rYTOzPEsXYOZRBd79gn15QNv9CZ2sjjX41TncbDSQrOBW9OUcUQ6LQvVk/g= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1595884263; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=41H6xYQ80fnDfWINvOexeCmGtB1H8JoFrDrMWEmGXkY=; b=LV1rsoc4hNYviKd9IsMqJgfeY14Fqj/6Mwj+K+hYiRlvjb9Ben4+BENYzqKflmT86c/8LqUIlgFRE7P9xrWXqwyrMwMMpSsxMJ4ccm+dw+G2Jjl2b6R+Kh9Q/wGQHpb8MTE1YSHRgGD46PkEQJ82Z3+DOTep1AxOLoKQlSWTEHk= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1595884263698836.0414131847381; Mon, 27 Jul 2020 14:11:03 -0700 (PDT) Received: from localhost ([::1]:39242 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1k0AOk-0003NZ-Bm for importer@patchew.org; Mon, 27 Jul 2020 17:11:02 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:35414) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1k0AAn-0005ou-2E for qemu-devel@nongnu.org; Mon, 27 Jul 2020 16:56:37 -0400 Received: from us-smtp-1.mimecast.com ([205.139.110.61]:26362 helo=us-smtp-delivery-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1k0AAl-00052S-6L for qemu-devel@nongnu.org; Mon, 27 Jul 2020 16:56:36 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-366-dlpDkqI-PHm4D1VgDc7KnQ-1; Mon, 27 Jul 2020 16:56:31 -0400 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id CD25C79EC6; Mon, 27 Jul 2020 20:56:30 +0000 (UTC) Received: from blue.redhat.com (ovpn-118-248.rdu2.redhat.com [10.10.118.248]) by smtp.corp.redhat.com (Postfix) with ESMTP id E7AD719C66; Mon, 27 Jul 2020 20:56:29 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1595883394; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=41H6xYQ80fnDfWINvOexeCmGtB1H8JoFrDrMWEmGXkY=; b=F9RUKk1YMLahSRNw8QXXVf2EKvJ9IpgvF5Zd/DWn+Lbb/2TRA5ZN/epjOlJiEoz1Vrd3tl 4SBV7xi5Xf7LqqGPzpgqM5crAJ/Xi/CHSBvT1Oig2unYyoWg3qXWOY+3FKSRtzJzhCZmlH Lh0p3zos9w3zdf3C3lKSUAX0Vsa7y9k= X-MC-Unique: dlpDkqI-PHm4D1VgDc7KnQ-1 From: Eric Blake To: qemu-devel@nongnu.org Subject: [PULL 22/24] qemu-iotests/199: add source-killed case to bitmaps postcopy Date: Mon, 27 Jul 2020 15:55:41 -0500 Message-Id: <20200727205543.206624-23-eblake@redhat.com> In-Reply-To: <20200727205543.206624-1-eblake@redhat.com> References: <20200727205543.206624-1-eblake@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.23 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=205.139.110.61; envelope-from=eblake@redhat.com; helo=us-smtp-delivery-1.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/07/27 01:46:13 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Spam_score_int: -40 X-Spam_score: -4.1 X-Spam_bar: ---- X-Spam_report: (-4.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-1, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-1, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Andrey Shinkevich , Vladimir Sementsov-Ogievskiy , "open list:Block layer core" , Max Reitz Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) Content-Type: text/plain; charset="utf-8" From: Vladimir Sementsov-Ogievskiy Previous patches fixes behavior of bitmaps migration, so that errors are handled by just removing unfinished bitmaps, and not fail or try to recover postcopy migration. Add corresponding test. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Andrey Shinkevich Tested-by: Eric Blake Message-Id: <20200727194236.19551-22-vsementsov@virtuozzo.com> Signed-off-by: Eric Blake --- tests/qemu-iotests/199 | 15 +++++++++++++++ tests/qemu-iotests/199.out | 4 ++-- 2 files changed, 17 insertions(+), 2 deletions(-) diff --git a/tests/qemu-iotests/199 b/tests/qemu-iotests/199 index 140930b2b12e..58fad872a12c 100755 --- a/tests/qemu-iotests/199 +++ b/tests/qemu-iotests/199 @@ -241,6 +241,21 @@ class TestDirtyBitmapPostcopyMigration(iotests.QMPTest= Case): self.vm_a.launch() check_bitmaps(self.vm_a, 0) + def test_early_kill_source(self): + self.start_postcopy() + + self.vm_a_events =3D self.vm_a.get_qmp_events() + self.vm_a.kill() + + self.vm_a.launch() + + match =3D {'data': {'status': 'completed'}} + e_complete =3D self.vm_b.event_wait('MIGRATION', match=3Dmatch) + self.vm_b_events.append(e_complete) + + check_bitmaps(self.vm_a, 0) + check_bitmaps(self.vm_b, 0) + if __name__ =3D=3D '__main__': iotests.main(supported_fmts=3D['qcow2']) diff --git a/tests/qemu-iotests/199.out b/tests/qemu-iotests/199.out index fbc63e62f885..8d7e99670093 100644 --- a/tests/qemu-iotests/199.out +++ b/tests/qemu-iotests/199.out @@ -1,5 +1,5 @@ -.. +... ---------------------------------------------------------------------- -Ran 2 tests +Ran 3 tests OK --=20 2.27.0 From nobody Mon Feb 9 01:37:05 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1595884082; cv=none; d=zohomail.com; s=zohoarc; b=iwqJABNrPgqVRBIt7md23qiwTQTVXiHm4nyLYKM6KFTGKtJA+YB8fLIjyI1AvBwHYTclrUmoV7Uyw9cs1CYjS9FFw49y1Wxnzjj8KPlims1o/jrKugJXkLDsiYfoFhxv+5X35aFm/DrjjTBRMxrCdaJbs8AwLLsktO268dQIERw= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1595884082; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=kRI3td16T7P/hRiIt2yR87EFEmDjt2EHrQN/juIKf6A=; b=IP0gdYoClZcANf54+lyNTU40IDKYp5K7w8bO8tT+096EA4HrlBemteSkL/MAvKeH/CV0WQKQo9oHOuC2NhLB9qjH9H+Dli+5m4jK7HVaUwnxfFb3WDBfLgLEJYhYlX1YcohQnTnqYFZsjoSUP2JQVoTNgB/hp0l3SwfgGp7JgwA= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1595884082669271.3531859637868; Mon, 27 Jul 2020 14:08:02 -0700 (PDT) Received: from localhost ([::1]:57768 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1k0ALp-0007gu-Fc for importer@patchew.org; Mon, 27 Jul 2020 17:08:01 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:35420) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1k0AAn-0005qA-Jj for qemu-devel@nongnu.org; Mon, 27 Jul 2020 16:56:37 -0400 Received: from us-smtp-1.mimecast.com ([207.211.31.81]:56229) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1k0AAl-00052h-QD for qemu-devel@nongnu.org; Mon, 27 Jul 2020 16:56:37 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-13-WFZVNi3jP1S77bdu2aOTnQ-1; Mon, 27 Jul 2020 16:56:32 -0400 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id DB3B679EC8; Mon, 27 Jul 2020 20:56:31 +0000 (UTC) Received: from blue.redhat.com (ovpn-118-248.rdu2.redhat.com [10.10.118.248]) by smtp.corp.redhat.com (Postfix) with ESMTP id 1D7DD19C66; Mon, 27 Jul 2020 20:56:31 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1595883395; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=kRI3td16T7P/hRiIt2yR87EFEmDjt2EHrQN/juIKf6A=; b=guAxAkKP2Ogxf6QiLGO5Q02YYNr3XworlfNL4GqyCBZsOa/gJYyOhnod0zFIPnoNYq8nvm apGBAfB6plVQXJQDpEDosB1iRab9BhcjjJfxuMtNNkwz9fyITZPOFcYNvwPuGcpYaW2up/ BgeCGDY2dXRmYZwC4oRqs2JcXUkzvho= X-MC-Unique: WFZVNi3jP1S77bdu2aOTnQ-1 From: Eric Blake To: qemu-devel@nongnu.org Subject: [PULL 23/24] iotests: Adjust which migration tests are quick Date: Mon, 27 Jul 2020 15:55:42 -0500 Message-Id: <20200727205543.206624-24-eblake@redhat.com> In-Reply-To: <20200727205543.206624-1-eblake@redhat.com> References: <20200727205543.206624-1-eblake@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.23 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=eblake@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=207.211.31.81; envelope-from=eblake@redhat.com; helo=us-smtp-1.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/07/27 03:37:14 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Spam_score_int: -30 X-Spam_score: -3.1 X-Spam_bar: --- X-Spam_report: (-3.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-1, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Vladimir Sementsov-Ogievskiy , "open list:Block layer core" , Max Reitz Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) Content-Type: text/plain; charset="utf-8" A quick run of './check -qcow2 -g migration' shows that test 169 is NOT quick, but meanwhile several other tests ARE quick. Let's adjust the test designations accordingly. Signed-off-by: Eric Blake Message-Id: <20200727195117.132151-1-eblake@redhat.com> Reviewed-by: Vladimir Sementsov-Ogievskiy --- tests/qemu-iotests/group | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/tests/qemu-iotests/group b/tests/qemu-iotests/group index 1d0252e1f051..806044642c69 100644 --- a/tests/qemu-iotests/group +++ b/tests/qemu-iotests/group @@ -112,7 +112,7 @@ 088 rw quick 089 rw auto quick 090 rw auto quick -091 rw migration +091 rw migration quick 092 rw quick 093 throttle 094 rw quick @@ -186,7 +186,7 @@ 162 quick 163 rw 165 rw quick -169 rw quick migration +169 rw migration 170 rw auto quick 171 rw quick 172 auto @@ -197,9 +197,9 @@ 177 rw auto quick 178 img 179 rw auto quick -181 rw auto migration +181 rw auto migration quick 182 rw quick -183 rw migration +183 rw migration quick 184 rw auto quick 185 rw 186 rw auto @@ -216,9 +216,9 @@ 198 rw 199 rw migration 200 rw -201 rw migration +201 rw migration quick 202 rw quick -203 rw auto migration +203 rw auto migration quick 204 rw quick 205 rw quick 206 rw --=20 2.27.0 From nobody Mon Feb 9 01:37:05 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1595884321; cv=none; d=zohomail.com; s=zohoarc; b=GPIlLO1IEFdoXaQLlVqOg0nmFuHUoICfmLaU1Y2JTp6MGPvF+F94Qc2/cqThBwlTlqNsZ2W9Gzfz4MxU896h+SpuAv37qWkMYoDJE5JPH/j5/kVmk5RZjj/IbidmCRkJ/PbWYGK13asTZ075/9qJ7TllXAdp56iqJ50gUAmdt3I= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1595884321; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=3HGKl7Xu7uAqUHOkXtf9D5rZqcQtEe5Y64LPYPFuQz4=; b=ZWOdj6/ppJpSKU98em+27ldY53lRaYocqkT0kccp7gkDWXXl/ziUPLHsz45FRbUvT9Tku6jwS+HCzAsAHxpEkJVvT/K4e87zRQDvapXufayuUXiMxUgZFtqjmN1DExgsY+bmM4W7ewNcmASywz1EEjy90mp37hYwLp2jhHVSzhw= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 15958843216921017.2268404622304; Mon, 27 Jul 2020 14:12:01 -0700 (PDT) Received: from localhost ([::1]:41372 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1k0APg-0004Hq-DW for importer@patchew.org; Mon, 27 Jul 2020 17:12:00 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:35466) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1k0AAp-0005um-Jz for qemu-devel@nongnu.org; Mon, 27 Jul 2020 16:56:39 -0400 Received: from us-smtp-1.mimecast.com ([207.211.31.81]:49422 helo=us-smtp-delivery-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1k0AAo-00053M-0A for qemu-devel@nongnu.org; Mon, 27 Jul 2020 16:56:39 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-361-Q9y4uWpcMdCpDgIQ3MDE0g-1; Mon, 27 Jul 2020 16:56:34 -0400 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 4312258; Mon, 27 Jul 2020 20:56:33 +0000 (UTC) Received: from blue.redhat.com (ovpn-118-248.rdu2.redhat.com [10.10.118.248]) by smtp.corp.redhat.com (Postfix) with ESMTP id 277F019C66; Mon, 27 Jul 2020 20:56:32 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1595883397; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=3HGKl7Xu7uAqUHOkXtf9D5rZqcQtEe5Y64LPYPFuQz4=; b=fbJIZOB2EHKZgxid66e9h7YdoCvhZuRM7SCrwTFqO+vxOcxUnGJejCuaLBJ6WzNvJyjC9k DxuMLKdMCwa7aaiuSwsM1NCbsobfI7fKgu3+nuHjkUzcfA74myLD1D1US+vKa6zjssf0c9 16EgkF6KvFKFVaPik2imwHB03bqqAPI= X-MC-Unique: Q9y4uWpcMdCpDgIQ3MDE0g-1 From: Eric Blake To: qemu-devel@nongnu.org Subject: [PULL 24/24] migration: Fix typos in bitmap migration comments Date: Mon, 27 Jul 2020 15:55:43 -0500 Message-Id: <20200727205543.206624-25-eblake@redhat.com> In-Reply-To: <20200727205543.206624-1-eblake@redhat.com> References: <20200727205543.206624-1-eblake@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.23 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=207.211.31.81; envelope-from=eblake@redhat.com; helo=us-smtp-delivery-1.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/07/27 03:37:14 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Spam_score_int: -30 X-Spam_score: -3.1 X-Spam_bar: --- X-Spam_report: (-3.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-1, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Fam Zheng , Vladimir Sementsov-Ogievskiy , "open list:Block I/O path" , Juan Quintela , "Dr. David Alan Gilbert" , Stefan Hajnoczi , John Snow Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) Content-Type: text/plain; charset="utf-8" Noticed while reviewing the file for newer patches. Fixes: b35ebdf076 Signed-off-by: Eric Blake Message-Id: <20200727203206.134996-1-eblake@redhat.com> --- migration/block-dirty-bitmap.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/migration/block-dirty-bitmap.c b/migration/block-dirty-bitmap.c index 1f675b792fc9..784330ebe130 100644 --- a/migration/block-dirty-bitmap.c +++ b/migration/block-dirty-bitmap.c @@ -97,7 +97,7 @@ #define DIRTY_BITMAP_MIG_START_FLAG_ENABLED 0x01 #define DIRTY_BITMAP_MIG_START_FLAG_PERSISTENT 0x02 -/* 0x04 was "AUTOLOAD" flags on elder versions, no it is ignored */ +/* 0x04 was "AUTOLOAD" flags on older versions, now it is ignored */ #define DIRTY_BITMAP_MIG_START_FLAG_RESERVED_MASK 0xf8 /* State of one bitmap during save process */ @@ -180,7 +180,7 @@ static uint32_t qemu_get_bitmap_flags(QEMUFile *f) static void qemu_put_bitmap_flags(QEMUFile *f, uint32_t flags) { - /* The code currently do not send flags more than one byte */ + /* The code currently does not send flags as more than one byte */ assert(!(flags & (0xffffff00 | DIRTY_BITMAP_MIG_EXTRA_FLAGS))); qemu_put_byte(f, flags); --=20 2.27.0