From nobody Sat May 4 15:42:58 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1498652111404329.3724409542607; Wed, 28 Jun 2017 05:15:11 -0700 (PDT) Received: from localhost ([::1]:32867 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dQBsE-0006RY-6J for importer@patchew.org; Wed, 28 Jun 2017 08:15:10 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:60301) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dQBj5-0006QH-46 for qemu-devel@nongnu.org; Wed, 28 Jun 2017 08:05:48 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1dQBj1-00030M-Rp for qemu-devel@nongnu.org; Wed, 28 Jun 2017 08:05:42 -0400 Received: from mailhub.sw.ru ([195.214.232.25]:21521 helo=relay.sw.ru) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1dQBj1-0002v3-HY for qemu-devel@nongnu.org; Wed, 28 Jun 2017 08:05:39 -0400 Received: from kvm.sw.ru (msk-vpn.virtuozzo.com [195.214.232.6]) by relay.sw.ru (8.13.4/8.13.4) with ESMTP id v5SC5UU3025017; Wed, 28 Jun 2017 15:05:31 +0300 (MSK) From: Vladimir Sementsov-Ogievskiy To: qemu-block@nongnu.org, qemu-devel@nongnu.org Date: Wed, 28 Jun 2017 15:05:01 +0300 Message-Id: <20170628120530.31251-2-vsementsov@virtuozzo.com> X-Mailer: git-send-email 2.11.1 In-Reply-To: <20170628120530.31251-1-vsementsov@virtuozzo.com> References: <20170628120530.31251-1-vsementsov@virtuozzo.com> X-detected-operating-system: by eggs.gnu.org: OpenBSD 3.x [fuzzy] X-Received-From: 195.214.232.25 Subject: [Qemu-devel] [PATCH v22 01/30] specs/qcow2: fix bitmap granularity qemu-specific note X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, vsementsov@virtuozzo.com, famz@redhat.com, armbru@redhat.com, mreitz@redhat.com, stefanha@redhat.com, pbonzini@redhat.com, den@openvz.org, jsnow@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: John Snow --- docs/interop/qcow2.txt | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/docs/interop/qcow2.txt b/docs/interop/qcow2.txt index 80cdfd0e91..dda53dd2a3 100644 --- a/docs/interop/qcow2.txt +++ b/docs/interop/qcow2.txt @@ -472,8 +472,7 @@ Structure of a bitmap directory entry: 17: granularity_bits Granularity bits. Valid values: 0 - 63. =20 - Note: Qemu currently doesn't support granularity_bits - greater than 31. + Note: Qemu currently supports only values 9 - 31. =20 Granularity is calculated as granularity =3D 1 << granularity_bits --=20 2.11.1 From nobody Sat May 4 15:42:58 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1498651667011323.6614609658923; Wed, 28 Jun 2017 05:07:47 -0700 (PDT) Received: from localhost ([::1]:32835 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dQBkz-0007nV-MG for importer@patchew.org; Wed, 28 Jun 2017 08:07:41 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:60408) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dQBj7-0006Qc-Cf for qemu-devel@nongnu.org; Wed, 28 Jun 2017 08:05:48 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1dQBj2-00030u-7O for qemu-devel@nongnu.org; Wed, 28 Jun 2017 08:05:45 -0400 Received: from mailhub.sw.ru ([195.214.232.25]:48962 helo=relay.sw.ru) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1dQBj1-0002vA-SJ for qemu-devel@nongnu.org; Wed, 28 Jun 2017 08:05:40 -0400 Received: from kvm.sw.ru (msk-vpn.virtuozzo.com [195.214.232.6]) by relay.sw.ru (8.13.4/8.13.4) with ESMTP id v5SC5UU4025017; Wed, 28 Jun 2017 15:05:31 +0300 (MSK) From: Vladimir Sementsov-Ogievskiy To: qemu-block@nongnu.org, qemu-devel@nongnu.org Date: Wed, 28 Jun 2017 15:05:02 +0300 Message-Id: <20170628120530.31251-3-vsementsov@virtuozzo.com> X-Mailer: git-send-email 2.11.1 In-Reply-To: <20170628120530.31251-1-vsementsov@virtuozzo.com> References: <20170628120530.31251-1-vsementsov@virtuozzo.com> X-detected-operating-system: by eggs.gnu.org: OpenBSD 3.x [fuzzy] X-Received-From: 195.214.232.25 Subject: [Qemu-devel] [PATCH v22 02/30] specs/qcow2: do not use wording 'bitmap header' X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, vsementsov@virtuozzo.com, famz@redhat.com, armbru@redhat.com, mreitz@redhat.com, stefanha@redhat.com, pbonzini@redhat.com, den@openvz.org, jsnow@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" A bitmap directory entry is sometimes called a 'bitmap header'. This patch leaves only one name - 'bitmap directory entry'. The name 'bitmap header' creates misunderstandings with 'qcow2 header' and 'qcow2 bitmap header extension' (which is extension of qcow2 header) Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Eric Blake Reviewed-by: John Snow --- docs/interop/qcow2.txt | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/docs/interop/qcow2.txt b/docs/interop/qcow2.txt index dda53dd2a3..8874e8c774 100644 --- a/docs/interop/qcow2.txt +++ b/docs/interop/qcow2.txt @@ -201,7 +201,7 @@ The fields of the bitmaps extension are: =20 8 - 15: bitmap_directory_size Size of the bitmap directory in bytes. It is the cumula= tive - size of all (nb_bitmaps) bitmap headers. + size of all (nb_bitmaps) bitmap directory entries. =20 16 - 23: bitmap_directory_offset Offset into the image file at which the bitmap directory @@ -426,8 +426,7 @@ Each bitmap saved in the image is described in a bitmap= directory entry. The bitmap directory is a contiguous area in the image file, whose starting of= fset and length are given by the header extension fields bitmap_directory_offse= t and bitmap_directory_size. The entries of the bitmap directory have variable -length, depending on the lengths of the bitmap name and extra data. These -entries are also called bitmap headers. +length, depending on the lengths of the bitmap name and extra data. =20 Structure of a bitmap directory entry: =20 --=20 2.11.1 From nobody Sat May 4 15:42:58 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1498651965252108.85919011434908; Wed, 28 Jun 2017 05:12:45 -0700 (PDT) Received: from localhost ([::1]:32855 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dQBpr-00048S-Ua for importer@patchew.org; Wed, 28 Jun 2017 08:12:43 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:60286) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dQBj5-0006Q9-0w for qemu-devel@nongnu.org; Wed, 28 Jun 2017 08:05:45 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1dQBj1-00030G-Qo for qemu-devel@nongnu.org; Wed, 28 Jun 2017 08:05:42 -0400 Received: from mailhub.sw.ru ([195.214.232.25]:27042 helo=relay.sw.ru) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1dQBj1-0002wn-FR for qemu-devel@nongnu.org; Wed, 28 Jun 2017 08:05:39 -0400 Received: from kvm.sw.ru (msk-vpn.virtuozzo.com [195.214.232.6]) by relay.sw.ru (8.13.4/8.13.4) with ESMTP id v5SC5UU5025017; Wed, 28 Jun 2017 15:05:31 +0300 (MSK) From: Vladimir Sementsov-Ogievskiy To: qemu-block@nongnu.org, qemu-devel@nongnu.org Date: Wed, 28 Jun 2017 15:05:03 +0300 Message-Id: <20170628120530.31251-4-vsementsov@virtuozzo.com> X-Mailer: git-send-email 2.11.1 In-Reply-To: <20170628120530.31251-1-vsementsov@virtuozzo.com> References: <20170628120530.31251-1-vsementsov@virtuozzo.com> X-detected-operating-system: by eggs.gnu.org: OpenBSD 3.x [fuzzy] X-Received-From: 195.214.232.25 Subject: [Qemu-devel] [PATCH v22 03/30] hbitmap: improve dirty iter X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, vsementsov@virtuozzo.com, famz@redhat.com, armbru@redhat.com, mreitz@redhat.com, stefanha@redhat.com, pbonzini@redhat.com, den@openvz.org, jsnow@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Make dirty iter resistant to resetting bits in corresponding HBitmap. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Max Reitz Reviewed-by: John Snow --- include/qemu/hbitmap.h | 26 ++++---------------------- util/hbitmap.c | 23 ++++++++++++++++++++++- 2 files changed, 26 insertions(+), 23 deletions(-) diff --git a/include/qemu/hbitmap.h b/include/qemu/hbitmap.h index 9239fe515e..6b04391266 100644 --- a/include/qemu/hbitmap.h +++ b/include/qemu/hbitmap.h @@ -256,10 +256,9 @@ void hbitmap_free(HBitmap *hb); * the lowest-numbered bit that is set in @hb, starting at @first. * * Concurrent setting of bits is acceptable, and will at worst cause the - * iteration to miss some of those bits. Resetting bits before the current - * position of the iterator is also okay. However, concurrent resetting of - * bits can lead to unexpected behavior if the iterator has not yet reached - * those bits. + * iteration to miss some of those bits. + * + * The concurrent resetting of bits is OK. */ void hbitmap_iter_init(HBitmapIter *hbi, const HBitmap *hb, uint64_t first= ); =20 @@ -298,24 +297,7 @@ void hbitmap_free_meta(HBitmap *hb); * Return the next bit that is set in @hbi's associated HBitmap, * or -1 if all remaining bits are zero. */ -static inline int64_t hbitmap_iter_next(HBitmapIter *hbi) -{ - unsigned long cur =3D hbi->cur[HBITMAP_LEVELS - 1]; - int64_t item; - - if (cur =3D=3D 0) { - cur =3D hbitmap_iter_skip_words(hbi); - if (cur =3D=3D 0) { - return -1; - } - } - - /* The next call will resume work from the next bit. */ - hbi->cur[HBITMAP_LEVELS - 1] =3D cur & (cur - 1); - item =3D ((uint64_t)hbi->pos << BITS_PER_LEVEL) + ctzl(cur); - - return item << hbi->granularity; -} +int64_t hbitmap_iter_next(HBitmapIter *hbi); =20 /** * hbitmap_iter_next_word: diff --git a/util/hbitmap.c b/util/hbitmap.c index 35088e19c4..0b38817505 100644 --- a/util/hbitmap.c +++ b/util/hbitmap.c @@ -106,8 +106,9 @@ unsigned long hbitmap_iter_skip_words(HBitmapIter *hbi) =20 unsigned long cur; do { - cur =3D hbi->cur[--i]; + i--; pos >>=3D BITS_PER_LEVEL; + cur =3D hbi->cur[i] & hb->levels[i][pos]; } while (cur =3D=3D 0); =20 /* Check for end of iteration. We always use fewer than BITS_PER_LONG @@ -139,6 +140,26 @@ unsigned long hbitmap_iter_skip_words(HBitmapIter *hbi) return cur; } =20 +int64_t hbitmap_iter_next(HBitmapIter *hbi) +{ + unsigned long cur =3D hbi->cur[HBITMAP_LEVELS - 1] & + hbi->hb->levels[HBITMAP_LEVELS - 1][hbi->pos]; + int64_t item; + + if (cur =3D=3D 0) { + cur =3D hbitmap_iter_skip_words(hbi); + if (cur =3D=3D 0) { + return -1; + } + } + + /* The next call will resume work from the next bit. */ + hbi->cur[HBITMAP_LEVELS - 1] =3D cur & (cur - 1); + item =3D ((uint64_t)hbi->pos << BITS_PER_LEVEL) + ctzl(cur); + + return item << hbi->granularity; +} + void hbitmap_iter_init(HBitmapIter *hbi, const HBitmap *hb, uint64_t first) { unsigned i, bit; --=20 2.11.1 From nobody Sat May 4 15:42:58 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1498652422054521.681767714656; Wed, 28 Jun 2017 05:20:22 -0700 (PDT) Received: from localhost ([::1]:32886 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dQBxC-0003Ek-Mh for importer@patchew.org; Wed, 28 Jun 2017 08:20:18 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:60287) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dQBj5-0006QA-17 for qemu-devel@nongnu.org; Wed, 28 Jun 2017 08:05:48 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1dQBj1-00030B-Q2 for qemu-devel@nongnu.org; Wed, 28 Jun 2017 08:05:42 -0400 Received: from mailhub.sw.ru ([195.214.232.25]:16036 helo=relay.sw.ru) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1dQBj1-0002v2-Ex for qemu-devel@nongnu.org; Wed, 28 Jun 2017 08:05:39 -0400 Received: from kvm.sw.ru (msk-vpn.virtuozzo.com [195.214.232.6]) by relay.sw.ru (8.13.4/8.13.4) with ESMTP id v5SC5UU6025017; Wed, 28 Jun 2017 15:05:31 +0300 (MSK) From: Vladimir Sementsov-Ogievskiy To: qemu-block@nongnu.org, qemu-devel@nongnu.org Date: Wed, 28 Jun 2017 15:05:04 +0300 Message-Id: <20170628120530.31251-5-vsementsov@virtuozzo.com> X-Mailer: git-send-email 2.11.1 In-Reply-To: <20170628120530.31251-1-vsementsov@virtuozzo.com> References: <20170628120530.31251-1-vsementsov@virtuozzo.com> X-detected-operating-system: by eggs.gnu.org: OpenBSD 3.x [fuzzy] X-Received-From: 195.214.232.25 Subject: [Qemu-devel] [PATCH v22 04/30] tests: add hbitmap iter test X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, vsementsov@virtuozzo.com, famz@redhat.com, armbru@redhat.com, mreitz@redhat.com, stefanha@redhat.com, pbonzini@redhat.com, den@openvz.org, jsnow@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Test that hbitmap iter is resistant to bitmap resetting. Signed-off-by: Vladimir Sementsov-Ogievskiy Signed-off-by: Denis V. Lunev Reviewed-by: Max Reitz Reviewed-by: John Snow --- tests/test-hbitmap.c | 19 +++++++++++++++++++ 1 file changed, 19 insertions(+) diff --git a/tests/test-hbitmap.c b/tests/test-hbitmap.c index 23773d2051..1acb353889 100644 --- a/tests/test-hbitmap.c +++ b/tests/test-hbitmap.c @@ -909,6 +909,22 @@ static void hbitmap_test_add(const char *testpath, hbitmap_test_teardown); } =20 +static void test_hbitmap_iter_and_reset(TestHBitmapData *data, + const void *unused) +{ + HBitmapIter hbi; + + hbitmap_test_init(data, L1 * 2, 0); + hbitmap_set(data->hb, 0, data->size); + + hbitmap_iter_init(&hbi, data->hb, BITS_PER_LONG - 1); + + hbitmap_iter_next(&hbi); + + hbitmap_reset_all(data->hb); + hbitmap_iter_next(&hbi); +} + int main(int argc, char **argv) { g_test_init(&argc, &argv, NULL); @@ -966,6 +982,9 @@ int main(int argc, char **argv) test_hbitmap_serialize_part); hbitmap_test_add("/hbitmap/serialize/zeroes", test_hbitmap_serialize_zeroes); + + hbitmap_test_add("/hbitmap/iter/iter_and_reset", + test_hbitmap_iter_and_reset); g_test_run(); =20 return 0; --=20 2.11.1 From nobody Sat May 4 15:42:58 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1498651819567233.22058457392723; Wed, 28 Jun 2017 05:10:19 -0700 (PDT) Received: from localhost ([::1]:32844 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dQBnT-00020t-A1 for importer@patchew.org; Wed, 28 Jun 2017 08:10:15 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:60289) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dQBj5-0006QC-1c for qemu-devel@nongnu.org; Wed, 28 Jun 2017 08:05:45 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1dQBj2-00030V-2X for qemu-devel@nongnu.org; Wed, 28 Jun 2017 08:05:42 -0400 Received: from mailhub.sw.ru ([195.214.232.25]:13034 helo=relay.sw.ru) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1dQBj1-0002uz-MJ for qemu-devel@nongnu.org; Wed, 28 Jun 2017 08:05:40 -0400 Received: from kvm.sw.ru (msk-vpn.virtuozzo.com [195.214.232.6]) by relay.sw.ru (8.13.4/8.13.4) with ESMTP id v5SC5UU7025017; Wed, 28 Jun 2017 15:05:31 +0300 (MSK) From: Vladimir Sementsov-Ogievskiy To: qemu-block@nongnu.org, qemu-devel@nongnu.org Date: Wed, 28 Jun 2017 15:05:05 +0300 Message-Id: <20170628120530.31251-6-vsementsov@virtuozzo.com> X-Mailer: git-send-email 2.11.1 In-Reply-To: <20170628120530.31251-1-vsementsov@virtuozzo.com> References: <20170628120530.31251-1-vsementsov@virtuozzo.com> X-detected-operating-system: by eggs.gnu.org: OpenBSD 3.x [fuzzy] X-Received-From: 195.214.232.25 Subject: [Qemu-devel] [PATCH v22 05/30] block: fix bdrv_dirty_bitmap_granularity signature X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, vsementsov@virtuozzo.com, famz@redhat.com, armbru@redhat.com, mreitz@redhat.com, stefanha@redhat.com, pbonzini@redhat.com, den@openvz.org, jsnow@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Make getter signature const-correct. This allows other functions with const dirty bitmap parameter use bdrv_dirty_bitmap_granularity(). Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Eric Blake Reviewed-by: John Snow Reviewed-by: Kevin Wolf --- block/dirty-bitmap.c | 2 +- include/block/dirty-bitmap.h | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/block/dirty-bitmap.c b/block/dirty-bitmap.c index a04c6e4154..df0110cf9f 100644 --- a/block/dirty-bitmap.c +++ b/block/dirty-bitmap.c @@ -455,7 +455,7 @@ uint32_t bdrv_get_default_bitmap_granularity(BlockDrive= rState *bs) return granularity; } =20 -uint32_t bdrv_dirty_bitmap_granularity(BdrvDirtyBitmap *bitmap) +uint32_t bdrv_dirty_bitmap_granularity(const BdrvDirtyBitmap *bitmap) { return BDRV_SECTOR_SIZE << hbitmap_granularity(bitmap->bitmap); } diff --git a/include/block/dirty-bitmap.h b/include/block/dirty-bitmap.h index ad6558af56..ab89f08e3d 100644 --- a/include/block/dirty-bitmap.h +++ b/include/block/dirty-bitmap.h @@ -29,7 +29,7 @@ void bdrv_disable_dirty_bitmap(BdrvDirtyBitmap *bitmap); void bdrv_enable_dirty_bitmap(BdrvDirtyBitmap *bitmap); BlockDirtyInfoList *bdrv_query_dirty_bitmaps(BlockDriverState *bs); uint32_t bdrv_get_default_bitmap_granularity(BlockDriverState *bs); -uint32_t bdrv_dirty_bitmap_granularity(BdrvDirtyBitmap *bitmap); +uint32_t bdrv_dirty_bitmap_granularity(const BdrvDirtyBitmap *bitmap); uint32_t bdrv_dirty_bitmap_meta_granularity(BdrvDirtyBitmap *bitmap); bool bdrv_dirty_bitmap_enabled(BdrvDirtyBitmap *bitmap); bool bdrv_dirty_bitmap_frozen(BdrvDirtyBitmap *bitmap); --=20 2.11.1 From nobody Sat May 4 15:42:58 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1498652421826217.99792958832734; Wed, 28 Jun 2017 05:20:21 -0700 (PDT) Received: from localhost ([::1]:32890 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dQBxE-0003Gc-JZ for importer@patchew.org; Wed, 28 Jun 2017 08:20:20 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:60300) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dQBj5-0006QG-3y for qemu-devel@nongnu.org; Wed, 28 Jun 2017 08:05:51 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1dQBj2-00030b-41 for qemu-devel@nongnu.org; Wed, 28 Jun 2017 08:05:42 -0400 Received: from mailhub.sw.ru ([195.214.232.25]:31383 helo=relay.sw.ru) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1dQBj1-0002vD-No for qemu-devel@nongnu.org; Wed, 28 Jun 2017 08:05:40 -0400 Received: from kvm.sw.ru (msk-vpn.virtuozzo.com [195.214.232.6]) by relay.sw.ru (8.13.4/8.13.4) with ESMTP id v5SC5UU8025017; Wed, 28 Jun 2017 15:05:31 +0300 (MSK) From: Vladimir Sementsov-Ogievskiy To: qemu-block@nongnu.org, qemu-devel@nongnu.org Date: Wed, 28 Jun 2017 15:05:06 +0300 Message-Id: <20170628120530.31251-7-vsementsov@virtuozzo.com> X-Mailer: git-send-email 2.11.1 In-Reply-To: <20170628120530.31251-1-vsementsov@virtuozzo.com> References: <20170628120530.31251-1-vsementsov@virtuozzo.com> X-detected-operating-system: by eggs.gnu.org: OpenBSD 3.x [fuzzy] X-Received-From: 195.214.232.25 Subject: [Qemu-devel] [PATCH v22 06/30] block/dirty-bitmap: add deserialize_ones func X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, vsementsov@virtuozzo.com, famz@redhat.com, armbru@redhat.com, mreitz@redhat.com, stefanha@redhat.com, pbonzini@redhat.com, den@openvz.org, jsnow@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Add bdrv_dirty_bitmap_deserialize_ones() function, which is needed for qcow2 bitmap loading, to handle unallocated bitmap parts, marked as all-ones. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Kevin Wolf Reviewed-by: John Snow --- block/dirty-bitmap.c | 7 +++++++ include/block/dirty-bitmap.h | 3 +++ include/qemu/hbitmap.h | 15 +++++++++++++++ util/hbitmap.c | 17 +++++++++++++++++ 4 files changed, 42 insertions(+) diff --git a/block/dirty-bitmap.c b/block/dirty-bitmap.c index df0110cf9f..f502c45a70 100644 --- a/block/dirty-bitmap.c +++ b/block/dirty-bitmap.c @@ -586,6 +586,13 @@ void bdrv_dirty_bitmap_deserialize_zeroes(BdrvDirtyBit= map *bitmap, hbitmap_deserialize_zeroes(bitmap->bitmap, start, count, finish); } =20 +void bdrv_dirty_bitmap_deserialize_ones(BdrvDirtyBitmap *bitmap, + uint64_t start, uint64_t count, + bool finish) +{ + hbitmap_deserialize_ones(bitmap->bitmap, start, count, finish); +} + void bdrv_dirty_bitmap_deserialize_finish(BdrvDirtyBitmap *bitmap) { hbitmap_deserialize_finish(bitmap->bitmap); diff --git a/include/block/dirty-bitmap.h b/include/block/dirty-bitmap.h index ab89f08e3d..05451c727d 100644 --- a/include/block/dirty-bitmap.h +++ b/include/block/dirty-bitmap.h @@ -66,6 +66,9 @@ void bdrv_dirty_bitmap_deserialize_part(BdrvDirtyBitmap *= bitmap, void bdrv_dirty_bitmap_deserialize_zeroes(BdrvDirtyBitmap *bitmap, uint64_t start, uint64_t count, bool finish); +void bdrv_dirty_bitmap_deserialize_ones(BdrvDirtyBitmap *bitmap, + uint64_t start, uint64_t count, + bool finish); void bdrv_dirty_bitmap_deserialize_finish(BdrvDirtyBitmap *bitmap); =20 /* Functions that require manual locking. */ diff --git a/include/qemu/hbitmap.h b/include/qemu/hbitmap.h index 6b04391266..b52304ac29 100644 --- a/include/qemu/hbitmap.h +++ b/include/qemu/hbitmap.h @@ -229,6 +229,21 @@ void hbitmap_deserialize_zeroes(HBitmap *hb, uint64_t = start, uint64_t count, bool finish); =20 /** + * hbitmap_deserialize_ones + * @hb: HBitmap to operate on. + * @start: First bit to restore. + * @count: Number of bits to restore. + * @finish: Whether to call hbitmap_deserialize_finish automatically. + * + * Fills the bitmap with ones. + * + * If @finish is false, caller must call hbitmap_serialize_finish before u= sing + * the bitmap. + */ +void hbitmap_deserialize_ones(HBitmap *hb, uint64_t start, uint64_t count, + bool finish); + +/** * hbitmap_deserialize_finish * @hb: HBitmap to operate on. * diff --git a/util/hbitmap.c b/util/hbitmap.c index 0b38817505..0c1591a594 100644 --- a/util/hbitmap.c +++ b/util/hbitmap.c @@ -551,6 +551,23 @@ void hbitmap_deserialize_zeroes(HBitmap *hb, uint64_t = start, uint64_t count, } } =20 +void hbitmap_deserialize_ones(HBitmap *hb, uint64_t start, uint64_t count, + bool finish) +{ + uint64_t el_count; + unsigned long *first; + + if (!count) { + return; + } + serialization_chunk(hb, start, count, &first, &el_count); + + memset(first, 0xff, el_count * sizeof(unsigned long)); + if (finish) { + hbitmap_deserialize_finish(hb); + } +} + void hbitmap_deserialize_finish(HBitmap *bitmap) { int64_t i, size, prev_size; --=20 2.11.1 From nobody Sat May 4 15:42:58 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1498652268409458.99360564474773; Wed, 28 Jun 2017 05:17:48 -0700 (PDT) Received: from localhost ([::1]:32879 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dQBui-0000Xd-3A for importer@patchew.org; Wed, 28 Jun 2017 08:17:44 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:60307) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dQBj5-0006QJ-90 for qemu-devel@nongnu.org; Wed, 28 Jun 2017 08:05:48 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1dQBj2-00032J-SD for qemu-devel@nongnu.org; Wed, 28 Jun 2017 08:05:43 -0400 Received: from mailhub.sw.ru ([195.214.232.25]:13868 helo=relay.sw.ru) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1dQBj2-0002vL-CJ for qemu-devel@nongnu.org; Wed, 28 Jun 2017 08:05:40 -0400 Received: from kvm.sw.ru (msk-vpn.virtuozzo.com [195.214.232.6]) by relay.sw.ru (8.13.4/8.13.4) with ESMTP id v5SC5UU9025017; Wed, 28 Jun 2017 15:05:32 +0300 (MSK) From: Vladimir Sementsov-Ogievskiy To: qemu-block@nongnu.org, qemu-devel@nongnu.org Date: Wed, 28 Jun 2017 15:05:07 +0300 Message-Id: <20170628120530.31251-8-vsementsov@virtuozzo.com> X-Mailer: git-send-email 2.11.1 In-Reply-To: <20170628120530.31251-1-vsementsov@virtuozzo.com> References: <20170628120530.31251-1-vsementsov@virtuozzo.com> X-detected-operating-system: by eggs.gnu.org: OpenBSD 3.x [fuzzy] X-Received-From: 195.214.232.25 Subject: [Qemu-devel] [PATCH v22 07/30] qcow2-refcount: rename inc_refcounts() and make it public X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, vsementsov@virtuozzo.com, famz@redhat.com, armbru@redhat.com, mreitz@redhat.com, stefanha@redhat.com, pbonzini@redhat.com, den@openvz.org, jsnow@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" This is needed for the following patch, which will introduce refcounts checking for qcow2 bitmaps. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Max Reitz Reviewed-by: John Snow --- block/qcow2-refcount.c | 53 ++++++++++++++++++++++++++--------------------= ---- block/qcow2.h | 4 ++++ 2 files changed, 32 insertions(+), 25 deletions(-) diff --git a/block/qcow2-refcount.c b/block/qcow2-refcount.c index 7c06061aae..d7066c875b 100644 --- a/block/qcow2-refcount.c +++ b/block/qcow2-refcount.c @@ -1323,11 +1323,10 @@ static int realloc_refcount_array(BDRVQcow2State *s= , void **array, * * Modifies the number of errors in res. */ -static int inc_refcounts(BlockDriverState *bs, - BdrvCheckResult *res, - void **refcount_table, - int64_t *refcount_table_size, - int64_t offset, int64_t size) +int qcow2_inc_refcounts_imrt(BlockDriverState *bs, BdrvCheckResult *res, + void **refcount_table, + int64_t *refcount_table_size, + int64_t offset, int64_t size) { BDRVQcow2State *s =3D bs->opaque; uint64_t start, last, cluster_offset, k, refcount; @@ -1420,8 +1419,9 @@ static int check_refcounts_l2(BlockDriverState *bs, B= drvCheckResult *res, nb_csectors =3D ((l2_entry >> s->csize_shift) & s->csize_mask) + 1; l2_entry &=3D s->cluster_offset_mask; - ret =3D inc_refcounts(bs, res, refcount_table, refcount_table_= size, - l2_entry & ~511, nb_csectors * 512); + ret =3D qcow2_inc_refcounts_imrt(bs, res, + refcount_table, refcount_table_= size, + l2_entry & ~511, nb_csectors * = 512); if (ret < 0) { goto fail; } @@ -1454,8 +1454,9 @@ static int check_refcounts_l2(BlockDriverState *bs, B= drvCheckResult *res, } =20 /* Mark cluster as used */ - ret =3D inc_refcounts(bs, res, refcount_table, refcount_table_= size, - offset, s->cluster_size); + ret =3D qcow2_inc_refcounts_imrt(bs, res, + refcount_table, refcount_table_= size, + offset, s->cluster_size); if (ret < 0) { goto fail; } @@ -1508,8 +1509,8 @@ static int check_refcounts_l1(BlockDriverState *bs, l1_size2 =3D l1_size * sizeof(uint64_t); =20 /* Mark L1 table as used */ - ret =3D inc_refcounts(bs, res, refcount_table, refcount_table_size, - l1_table_offset, l1_size2); + ret =3D qcow2_inc_refcounts_imrt(bs, res, refcount_table, refcount_tab= le_size, + l1_table_offset, l1_size2); if (ret < 0) { goto fail; } @@ -1538,8 +1539,9 @@ static int check_refcounts_l1(BlockDriverState *bs, if (l2_offset) { /* Mark L2 table as used */ l2_offset &=3D L1E_OFFSET_MASK; - ret =3D inc_refcounts(bs, res, refcount_table, refcount_table_= size, - l2_offset, s->cluster_size); + ret =3D qcow2_inc_refcounts_imrt(bs, res, + refcount_table, refcount_table_= size, + l2_offset, s->cluster_size); if (ret < 0) { goto fail; } @@ -1757,14 +1759,15 @@ static int check_refblocks(BlockDriverState *bs, Bd= rvCheckResult *res, } =20 res->corruptions_fixed++; - ret =3D inc_refcounts(bs, res, refcount_table, nb_clusters, - offset, s->cluster_size); + ret =3D qcow2_inc_refcounts_imrt(bs, res, + refcount_table, nb_clusters, + offset, s->cluster_size); if (ret < 0) { return ret; } /* No need to check whether the refcount is now greater th= an 1: * This area was just allocated and zeroed, so it can only= be - * exactly 1 after inc_refcounts() */ + * exactly 1 after qcow2_inc_refcounts_imrt() */ continue; =20 resize_fail: @@ -1779,8 +1782,8 @@ resize_fail: } =20 if (offset !=3D 0) { - ret =3D inc_refcounts(bs, res, refcount_table, nb_clusters, - offset, s->cluster_size); + ret =3D qcow2_inc_refcounts_imrt(bs, res, refcount_table, nb_c= lusters, + offset, s->cluster_size); if (ret < 0) { return ret; } @@ -1820,8 +1823,8 @@ static int calculate_refcounts(BlockDriverState *bs, = BdrvCheckResult *res, } =20 /* header */ - ret =3D inc_refcounts(bs, res, refcount_table, nb_clusters, - 0, s->cluster_size); + ret =3D qcow2_inc_refcounts_imrt(bs, res, refcount_table, nb_clusters, + 0, s->cluster_size); if (ret < 0) { return ret; } @@ -1842,16 +1845,16 @@ static int calculate_refcounts(BlockDriverState *bs= , BdrvCheckResult *res, return ret; } } - ret =3D inc_refcounts(bs, res, refcount_table, nb_clusters, - s->snapshots_offset, s->snapshots_size); + ret =3D qcow2_inc_refcounts_imrt(bs, res, refcount_table, nb_clusters, + s->snapshots_offset, s->snapshots_size); if (ret < 0) { return ret; } =20 /* refcount data */ - ret =3D inc_refcounts(bs, res, refcount_table, nb_clusters, - s->refcount_table_offset, - s->refcount_table_size * sizeof(uint64_t)); + ret =3D qcow2_inc_refcounts_imrt(bs, res, refcount_table, nb_clusters, + s->refcount_table_offset, + s->refcount_table_size * sizeof(uint64_= t)); if (ret < 0) { return ret; } diff --git a/block/qcow2.h b/block/qcow2.h index 1801dc30dc..41f8160629 100644 --- a/block/qcow2.h +++ b/block/qcow2.h @@ -527,6 +527,10 @@ int qcow2_check_metadata_overlap(BlockDriverState *bs,= int ign, int64_t offset, int64_t size); int qcow2_pre_write_overlap_check(BlockDriverState *bs, int ign, int64_t o= ffset, int64_t size); +int qcow2_inc_refcounts_imrt(BlockDriverState *bs, BdrvCheckResult *res, + void **refcount_table, + int64_t *refcount_table_size, + int64_t offset, int64_t size); =20 int qcow2_change_refcount_order(BlockDriverState *bs, int refcount_order, BlockDriverAmendStatusCB *status_cb, --=20 2.11.1 From nobody Sat May 4 15:42:58 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1498653023033836.9753427643526; Wed, 28 Jun 2017 05:30:23 -0700 (PDT) Received: from localhost ([::1]:32966 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dQC6t-0004aR-PV for importer@patchew.org; Wed, 28 Jun 2017 08:30:19 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:60444) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dQBj8-0006Qn-7z for qemu-devel@nongnu.org; Wed, 28 Jun 2017 08:05:50 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1dQBj2-000329-NP for qemu-devel@nongnu.org; Wed, 28 Jun 2017 08:05:46 -0400 Received: from mailhub.sw.ru ([195.214.232.25]:24299 helo=relay.sw.ru) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1dQBj1-0002vO-UX for qemu-devel@nongnu.org; Wed, 28 Jun 2017 08:05:40 -0400 Received: from kvm.sw.ru (msk-vpn.virtuozzo.com [195.214.232.6]) by relay.sw.ru (8.13.4/8.13.4) with ESMTP id v5SC5UUA025017; Wed, 28 Jun 2017 15:05:32 +0300 (MSK) From: Vladimir Sementsov-Ogievskiy To: qemu-block@nongnu.org, qemu-devel@nongnu.org Date: Wed, 28 Jun 2017 15:05:08 +0300 Message-Id: <20170628120530.31251-9-vsementsov@virtuozzo.com> X-Mailer: git-send-email 2.11.1 In-Reply-To: <20170628120530.31251-1-vsementsov@virtuozzo.com> References: <20170628120530.31251-1-vsementsov@virtuozzo.com> X-detected-operating-system: by eggs.gnu.org: OpenBSD 3.x [fuzzy] X-Received-From: 195.214.232.25 Subject: [Qemu-devel] [PATCH v22 08/30] qcow2: add bitmaps extension X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, vsementsov@virtuozzo.com, famz@redhat.com, armbru@redhat.com, mreitz@redhat.com, stefanha@redhat.com, pbonzini@redhat.com, den@openvz.org, jsnow@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Add bitmap extension as specified in docs/specs/qcow2.txt. For now, just mirror extension header into Qcow2 state and check constraints. Also, calculate refcounts for qcow2 bitmaps, to not break qemu-img check. For now, disable image resize if it has bitmaps. It will be fixed later. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Max Reitz Reviewed-by: John Snow --- block/Makefile.objs | 2 +- block/qcow2-bitmap.c | 439 +++++++++++++++++++++++++++++++++++++++++++++= ++++ block/qcow2-refcount.c | 6 + block/qcow2.c | 124 +++++++++++++- block/qcow2.h | 27 +++ 5 files changed, 592 insertions(+), 6 deletions(-) create mode 100644 block/qcow2-bitmap.c diff --git a/block/Makefile.objs b/block/Makefile.objs index ea955302c8..9efc6c49ea 100644 --- a/block/Makefile.objs +++ b/block/Makefile.objs @@ -1,5 +1,5 @@ block-obj-y +=3D raw-format.o qcow.o vdi.o vmdk.o cloop.o bochs.o vpc.o vv= fat.o dmg.o -block-obj-y +=3D qcow2.o qcow2-refcount.o qcow2-cluster.o qcow2-snapshot.o= qcow2-cache.o +block-obj-y +=3D qcow2.o qcow2-refcount.o qcow2-cluster.o qcow2-snapshot.o= qcow2-cache.o qcow2-bitmap.o block-obj-y +=3D qed.o qed-gencb.o qed-l2-cache.o qed-table.o qed-cluster.o block-obj-y +=3D qed-check.o block-obj-y +=3D vhdx.o vhdx-endian.o vhdx-log.o diff --git a/block/qcow2-bitmap.c b/block/qcow2-bitmap.c new file mode 100644 index 0000000000..b8e472b3e8 --- /dev/null +++ b/block/qcow2-bitmap.c @@ -0,0 +1,439 @@ +/* + * Bitmaps for the QCOW version 2 format + * + * Copyright (c) 2014-2017 Vladimir Sementsov-Ogievskiy + * + * This file is derived from qcow2-snapshot.c, original copyright: + * Copyright (c) 2004-2006 Fabrice Bellard + * + * Permission is hereby granted, free of charge, to any person obtaining a= copy + * of this software and associated documentation files (the "Software"), t= o deal + * in the Software without restriction, including without limitation the r= ights + * to use, copy, modify, merge, publish, distribute, sublicense, and/or se= ll + * copies of the Software, and to permit persons to whom the Software is + * furnished to do so, subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included= in + * all copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS= OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL + * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OT= HER + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING= FROM, + * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS = IN + * THE SOFTWARE. + */ + +#include "qemu/osdep.h" +#include "qapi/error.h" + +#include "block/block_int.h" +#include "block/qcow2.h" + +/* NOTICE: BME here means Bitmaps Extension and used as a namespace for + * _internal_ constants. Please do not use this _internal_ abbreviation for + * other needs and/or outside of this file. */ + +/* Bitmap directory entry constraints */ +#define BME_MAX_TABLE_SIZE 0x8000000 +#define BME_MAX_PHYS_SIZE 0x20000000 /* restrict BdrvDirtyBitmap size in R= AM */ +#define BME_MAX_GRANULARITY_BITS 31 +#define BME_MIN_GRANULARITY_BITS 9 +#define BME_MAX_NAME_SIZE 1023 + +/* Bitmap directory entry flags */ +#define BME_RESERVED_FLAGS 0xfffffffcU + +/* bits [1, 8] U [56, 63] are reserved */ +#define BME_TABLE_ENTRY_RESERVED_MASK 0xff000000000001feULL +#define BME_TABLE_ENTRY_OFFSET_MASK 0x00fffffffffffe00ULL +#define BME_TABLE_ENTRY_FLAG_ALL_ONES (1ULL << 0) + +typedef struct QEMU_PACKED Qcow2BitmapDirEntry { + /* header is 8 byte aligned */ + uint64_t bitmap_table_offset; + + uint32_t bitmap_table_size; + uint32_t flags; + + uint8_t type; + uint8_t granularity_bits; + uint16_t name_size; + uint32_t extra_data_size; + /* extra data follows */ + /* name follows */ +} Qcow2BitmapDirEntry; + +typedef struct Qcow2BitmapTable { + uint64_t offset; + uint32_t size; /* number of 64bit entries */ + QSIMPLEQ_ENTRY(Qcow2BitmapTable) entry; +} Qcow2BitmapTable; + +typedef struct Qcow2Bitmap { + Qcow2BitmapTable table; + uint32_t flags; + uint8_t granularity_bits; + char *name; + + QSIMPLEQ_ENTRY(Qcow2Bitmap) entry; +} Qcow2Bitmap; +typedef QSIMPLEQ_HEAD(Qcow2BitmapList, Qcow2Bitmap) Qcow2BitmapList; + +typedef enum BitmapType { + BT_DIRTY_TRACKING_BITMAP =3D 1 +} BitmapType; + +static int check_table_entry(uint64_t entry, int cluster_size) +{ + uint64_t offset; + + if (entry & BME_TABLE_ENTRY_RESERVED_MASK) { + return -EINVAL; + } + + offset =3D entry & BME_TABLE_ENTRY_OFFSET_MASK; + if (offset !=3D 0) { + /* if offset specified, bit 0 is reserved */ + if (entry & BME_TABLE_ENTRY_FLAG_ALL_ONES) { + return -EINVAL; + } + + if (offset % cluster_size !=3D 0) { + return -EINVAL; + } + } + + return 0; +} + +static int bitmap_table_load(BlockDriverState *bs, Qcow2BitmapTable *tb, + uint64_t **bitmap_table) +{ + int ret; + BDRVQcow2State *s =3D bs->opaque; + uint32_t i; + uint64_t *table; + + assert(tb->size !=3D 0); + table =3D g_try_new(uint64_t, tb->size); + if (table =3D=3D NULL) { + return -ENOMEM; + } + + assert(tb->size <=3D BME_MAX_TABLE_SIZE); + ret =3D bdrv_pread(bs->file, tb->offset, + table, tb->size * sizeof(uint64_t)); + if (ret < 0) { + goto fail; + } + + for (i =3D 0; i < tb->size; ++i) { + be64_to_cpus(&table[i]); + ret =3D check_table_entry(table[i], s->cluster_size); + if (ret < 0) { + goto fail; + } + } + + *bitmap_table =3D table; + return 0; + +fail: + g_free(table); + + return ret; +} + +/* + * Bitmap List + */ + +/* + * Bitmap List private functions + * Only Bitmap List knows about bitmap directory structure in Qcow2. + */ + +static inline void bitmap_dir_entry_to_cpu(Qcow2BitmapDirEntry *entry) +{ + be64_to_cpus(&entry->bitmap_table_offset); + be32_to_cpus(&entry->bitmap_table_size); + be32_to_cpus(&entry->flags); + be16_to_cpus(&entry->name_size); + be32_to_cpus(&entry->extra_data_size); +} + +static inline int calc_dir_entry_size(size_t name_size, size_t extra_data_= size) +{ + return align_offset(sizeof(Qcow2BitmapDirEntry) + + name_size + extra_data_size, 8); +} + +static inline int dir_entry_size(Qcow2BitmapDirEntry *entry) +{ + return calc_dir_entry_size(entry->name_size, entry->extra_data_size); +} + +static inline const char *dir_entry_name_field(Qcow2BitmapDirEntry *entry) +{ + return (const char *)(entry + 1) + entry->extra_data_size; +} + +static inline char *dir_entry_copy_name(Qcow2BitmapDirEntry *entry) +{ + const char *name_field =3D dir_entry_name_field(entry); + return g_strndup(name_field, entry->name_size); +} + +static inline Qcow2BitmapDirEntry *next_dir_entry(Qcow2BitmapDirEntry *ent= ry) +{ + return (Qcow2BitmapDirEntry *)((uint8_t *)entry + dir_entry_size(entry= )); +} + +static int check_dir_entry(BlockDriverState *bs, Qcow2BitmapDirEntry *entr= y) +{ + BDRVQcow2State *s =3D bs->opaque; + uint64_t phys_bitmap_bytes; + int64_t len; + + bool fail =3D (entry->bitmap_table_size =3D=3D 0) || + (entry->bitmap_table_offset =3D=3D 0) || + (entry->bitmap_table_offset % s->cluster_size) || + (entry->bitmap_table_size > BME_MAX_TABLE_SIZE) || + (entry->granularity_bits > BME_MAX_GRANULARITY_BITS) || + (entry->granularity_bits < BME_MIN_GRANULARITY_BITS) || + (entry->flags & BME_RESERVED_FLAGS) || + (entry->name_size > BME_MAX_NAME_SIZE) || + (entry->type !=3D BT_DIRTY_TRACKING_BITMAP); + + if (fail) { + return -EINVAL; + } + + phys_bitmap_bytes =3D (uint64_t)entry->bitmap_table_size * s->cluster_= size; + len =3D bdrv_getlength(bs); + + if (len < 0) { + return len; + } + + fail =3D (phys_bitmap_bytes > BME_MAX_PHYS_SIZE) || + (len > ((phys_bitmap_bytes * 8) << entry->granularity_bits)); + + return fail ? -EINVAL : 0; +} + +/* + * Bitmap List public functions + */ + +static void bitmap_free(Qcow2Bitmap *bm) +{ + g_free(bm->name); + g_free(bm); +} + +static void bitmap_list_free(Qcow2BitmapList *bm_list) +{ + Qcow2Bitmap *bm; + + if (bm_list =3D=3D NULL) { + return; + } + + while ((bm =3D QSIMPLEQ_FIRST(bm_list)) !=3D NULL) { + QSIMPLEQ_REMOVE_HEAD(bm_list, entry); + bitmap_free(bm); + } + + g_free(bm_list); +} + +static Qcow2BitmapList *bitmap_list_new(void) +{ + Qcow2BitmapList *bm_list =3D g_new(Qcow2BitmapList, 1); + QSIMPLEQ_INIT(bm_list); + + return bm_list; +} + +/* bitmap_list_load + * Get bitmap list from qcow2 image. Actually reads bitmap directory, + * checks it and convert to bitmap list. + */ +static Qcow2BitmapList *bitmap_list_load(BlockDriverState *bs, uint64_t of= fset, + uint64_t size, Error **errp) +{ + int ret; + BDRVQcow2State *s =3D bs->opaque; + uint8_t *dir, *dir_end; + Qcow2BitmapDirEntry *e; + uint32_t nb_dir_entries =3D 0; + Qcow2BitmapList *bm_list =3D NULL; + + if (size =3D=3D 0) { + error_setg(errp, "Requested bitmap directory size is zero"); + return NULL; + } + + if (size > QCOW2_MAX_BITMAP_DIRECTORY_SIZE) { + error_setg(errp, "Requested bitmap directory size is too big"); + return NULL; + } + + dir =3D g_try_malloc(size); + if (dir =3D=3D NULL) { + error_setg(errp, "Failed to allocate space for bitmap directory"); + return NULL; + } + dir_end =3D dir + size; + + ret =3D bdrv_pread(bs->file, offset, dir, size); + if (ret < 0) { + error_setg_errno(errp, -ret, "Failed to read bitmap directory"); + goto fail; + } + + bm_list =3D bitmap_list_new(); + for (e =3D (Qcow2BitmapDirEntry *)dir; + e < (Qcow2BitmapDirEntry *)dir_end; + e =3D next_dir_entry(e)) + { + Qcow2Bitmap *bm; + + if ((uint8_t *)(e + 1) > dir_end) { + goto broken_dir; + } + + if (++nb_dir_entries > s->nb_bitmaps) { + error_setg(errp, "More bitmaps found than specified in header" + " extension"); + goto fail; + } + bitmap_dir_entry_to_cpu(e); + + if ((uint8_t *)next_dir_entry(e) > dir_end) { + goto broken_dir; + } + + if (e->extra_data_size !=3D 0) { + error_setg(errp, "Bitmap extra data is not supported"); + goto fail; + } + + ret =3D check_dir_entry(bs, e); + if (ret < 0) { + error_setg(errp, "Bitmap '%.*s' doesn't satisfy the constraint= s", + e->name_size, dir_entry_name_field(e)); + goto fail; + } + + bm =3D g_new(Qcow2Bitmap, 1); + bm->table.offset =3D e->bitmap_table_offset; + bm->table.size =3D e->bitmap_table_size; + bm->flags =3D e->flags; + bm->granularity_bits =3D e->granularity_bits; + bm->name =3D dir_entry_copy_name(e); + QSIMPLEQ_INSERT_TAIL(bm_list, bm, entry); + } + + if (nb_dir_entries !=3D s->nb_bitmaps) { + error_setg(errp, "Less bitmaps found than specified in header" + " extension"); + goto fail; + } + + if ((uint8_t *)e !=3D dir_end) { + goto broken_dir; + } + + g_free(dir); + return bm_list; + +broken_dir: + ret =3D -EINVAL; + error_setg(errp, "Broken bitmap directory"); + +fail: + g_free(dir); + bitmap_list_free(bm_list); + + return NULL; +} + +int qcow2_check_bitmaps_refcounts(BlockDriverState *bs, BdrvCheckResult *r= es, + void **refcount_table, + int64_t *refcount_table_size) +{ + int ret; + BDRVQcow2State *s =3D bs->opaque; + Qcow2BitmapList *bm_list; + Qcow2Bitmap *bm; + + if (s->nb_bitmaps =3D=3D 0) { + return 0; + } + + ret =3D qcow2_inc_refcounts_imrt(bs, res, refcount_table, refcount_tab= le_size, + s->bitmap_directory_offset, + s->bitmap_directory_size); + if (ret < 0) { + return ret; + } + + bm_list =3D bitmap_list_load(bs, s->bitmap_directory_offset, + s->bitmap_directory_size, NULL); + if (bm_list =3D=3D NULL) { + res->corruptions++; + return -EINVAL; + } + + QSIMPLEQ_FOREACH(bm, bm_list, entry) { + uint64_t *bitmap_table =3D NULL; + int i; + + ret =3D qcow2_inc_refcounts_imrt(bs, res, + refcount_table, refcount_table_size, + bm->table.offset, + bm->table.size * sizeof(uint64_t)); + if (ret < 0) { + goto out; + } + + ret =3D bitmap_table_load(bs, &bm->table, &bitmap_table); + if (ret < 0) { + res->corruptions++; + goto out; + } + + for (i =3D 0; i < bm->table.size; ++i) { + uint64_t entry =3D bitmap_table[i]; + uint64_t offset =3D entry & BME_TABLE_ENTRY_OFFSET_MASK; + + if (check_table_entry(entry, s->cluster_size) < 0) { + res->corruptions++; + continue; + } + + if (offset =3D=3D 0) { + continue; + } + + ret =3D qcow2_inc_refcounts_imrt(bs, res, + refcount_table, refcount_table_= size, + offset, s->cluster_size); + if (ret < 0) { + g_free(bitmap_table); + goto out; + } + } + + g_free(bitmap_table); + } + +out: + bitmap_list_free(bm_list); + + return ret; +} diff --git a/block/qcow2-refcount.c b/block/qcow2-refcount.c index d7066c875b..bdb4cb3a6d 100644 --- a/block/qcow2-refcount.c +++ b/block/qcow2-refcount.c @@ -1859,6 +1859,12 @@ static int calculate_refcounts(BlockDriverState *bs,= BdrvCheckResult *res, return ret; } =20 + /* bitmaps */ + ret =3D qcow2_check_bitmaps_refcounts(bs, res, refcount_table, nb_clus= ters); + if (ret < 0) { + return ret; + } + return check_refblocks(bs, res, fix, rebuild, refcount_table, nb_clust= ers); } =20 diff --git a/block/qcow2.c b/block/qcow2.c index b3ba5daa93..b9657d8099 100644 --- a/block/qcow2.c +++ b/block/qcow2.c @@ -63,6 +63,7 @@ typedef struct { #define QCOW2_EXT_MAGIC_END 0 #define QCOW2_EXT_MAGIC_BACKING_FORMAT 0xE2792ACA #define QCOW2_EXT_MAGIC_FEATURE_TABLE 0x6803f857 +#define QCOW2_EXT_MAGIC_BITMAPS 0x23852875 =20 static int qcow2_probe(const uint8_t *buf, int buf_size, const char *filen= ame) { @@ -86,12 +87,17 @@ static int qcow2_probe(const uint8_t *buf, int buf_size= , const char *filename) */ static int qcow2_read_extensions(BlockDriverState *bs, uint64_t start_offs= et, uint64_t end_offset, void **p_feature_tab= le, - Error **errp) + bool *need_update_header, Error **errp) { BDRVQcow2State *s =3D bs->opaque; QCowExtension ext; uint64_t offset; int ret; + Qcow2BitmapHeaderExt bitmaps_ext; + + if (need_update_header !=3D NULL) { + *need_update_header =3D false; + } =20 #ifdef DEBUG_EXT printf("qcow2_read_extensions: start=3D%ld end=3D%ld\n", start_offset,= end_offset); @@ -162,6 +168,85 @@ static int qcow2_read_extensions(BlockDriverState *bs,= uint64_t start_offset, } break; =20 + case QCOW2_EXT_MAGIC_BITMAPS: + if (ext.len !=3D sizeof(bitmaps_ext)) { + error_setg_errno(errp, -ret, "bitmaps_ext: " + "Invalid extension length"); + return -EINVAL; + } + + if (!(s->autoclear_features & QCOW2_AUTOCLEAR_BITMAPS)) { + error_report("WARNING: a program lacking bitmap support " + "modified this file, so all bitmaps are now " + "considered inconsistent. Some clusters may b= e " + "leaked, run 'qemu-img check -r' on the image= " + "file to fix."); + if (need_update_header !=3D NULL) { + /* Updating is needed to drop invalid bitmap extension= . */ + *need_update_header =3D true; + } + break; + } + + ret =3D bdrv_pread(bs->file, offset, &bitmaps_ext, ext.len); + if (ret < 0) { + error_setg_errno(errp, -ret, "bitmaps_ext: " + "Could not read ext header"); + return ret; + } + + if (bitmaps_ext.reserved32 !=3D 0) { + error_setg_errno(errp, -ret, "bitmaps_ext: " + "Reserved field is not zero"); + return -EINVAL; + } + + be32_to_cpus(&bitmaps_ext.nb_bitmaps); + be64_to_cpus(&bitmaps_ext.bitmap_directory_size); + be64_to_cpus(&bitmaps_ext.bitmap_directory_offset); + + if (bitmaps_ext.nb_bitmaps > QCOW2_MAX_BITMAPS) { + error_setg(errp, + "bitmaps_ext: Image has %" PRIu32 " bitmaps, " + "exceeding the QEMU supported maximum of %d", + bitmaps_ext.nb_bitmaps, QCOW2_MAX_BITMAPS); + return -EINVAL; + } + + if (bitmaps_ext.nb_bitmaps =3D=3D 0) { + error_setg(errp, "found bitmaps extension with zero bitmap= s"); + return -EINVAL; + } + + if (bitmaps_ext.bitmap_directory_offset & (s->cluster_size - 1= )) { + error_setg(errp, "bitmaps_ext: " + "invalid bitmap directory offset"); + return -EINVAL; + } + + if (bitmaps_ext.bitmap_directory_size > + QCOW2_MAX_BITMAP_DIRECTORY_SIZE) { + error_setg(errp, "bitmaps_ext: " + "bitmap directory size (%" PRIu64 ") exce= eds " + "the maximum supported size (%d)", + bitmaps_ext.bitmap_directory_size, + QCOW2_MAX_BITMAP_DIRECTORY_SIZE); + return -EINVAL; + } + + s->nb_bitmaps =3D bitmaps_ext.nb_bitmaps; + s->bitmap_directory_offset =3D + bitmaps_ext.bitmap_directory_offset; + s->bitmap_directory_size =3D + bitmaps_ext.bitmap_directory_size; + +#ifdef DEBUG_EXT + printf("Qcow2: Got bitmaps extension: " + "offset=3D%" PRIu64 " nb_bitmaps=3D%" PRIu32 "\n", + s->bitmap_directory_offset, s->nb_bitmaps); +#endif + break; + default: /* unknown magic - save it in case we need to rewrite the head= er */ { @@ -824,6 +909,7 @@ static int qcow2_do_open(BlockDriverState *bs, QDict *o= ptions, int flags, Error *local_err =3D NULL; uint64_t ext_end; uint64_t l1_vm_state_index; + bool update_header =3D false; =20 ret =3D bdrv_pread(bs->file, 0, &header, sizeof(header)); if (ret < 0) { @@ -929,7 +1015,7 @@ static int qcow2_do_open(BlockDriverState *bs, QDict *= options, int flags, if (s->incompatible_features & ~QCOW2_INCOMPAT_MASK) { void *feature_table =3D NULL; qcow2_read_extensions(bs, header.header_length, ext_end, - &feature_table, NULL); + &feature_table, NULL, NULL); report_unsupported_feature(errp, feature_table, s->incompatible_features & ~QCOW2_INCOMPAT_MASK); @@ -1116,7 +1202,7 @@ static int qcow2_do_open(BlockDriverState *bs, QDict = *options, int flags, =20 /* read qcow2 extensions */ if (qcow2_read_extensions(bs, header.header_length, ext_end, NULL, - &local_err)) { + &update_header, &local_err)) { error_propagate(errp, local_err); ret =3D -EINVAL; goto fail; @@ -1152,8 +1238,10 @@ static int qcow2_do_open(BlockDriverState *bs, QDict= *options, int flags, } =20 /* Clear unknown autoclear feature bits */ - if (!bs->read_only && !(flags & BDRV_O_INACTIVE) && s->autoclear_featu= res) { - s->autoclear_features =3D 0; + update_header |=3D s->autoclear_features & ~QCOW2_AUTOCLEAR_MASK; + + if (update_header && !bs->read_only && !(flags & BDRV_O_INACTIVE)) { + s->autoclear_features &=3D QCOW2_AUTOCLEAR_MASK; ret =3D qcow2_update_header(bs); if (ret < 0) { error_setg_errno(errp, -ret, "Could not update qcow2 header"); @@ -1966,6 +2054,25 @@ int qcow2_update_header(BlockDriverState *bs) buflen -=3D ret; } =20 + /* Bitmap extension */ + if (s->nb_bitmaps > 0) { + Qcow2BitmapHeaderExt bitmaps_header =3D { + .nb_bitmaps =3D cpu_to_be32(s->nb_bitmaps), + .bitmap_directory_size =3D + cpu_to_be64(s->bitmap_directory_size), + .bitmap_directory_offset =3D + cpu_to_be64(s->bitmap_directory_offset) + }; + ret =3D header_ext_add(buf, QCOW2_EXT_MAGIC_BITMAPS, + &bitmaps_header, sizeof(bitmaps_header), + buflen); + if (ret < 0) { + goto fail; + } + buf +=3D ret; + buflen -=3D ret; + } + /* Keep unknown header extensions */ QLIST_FOREACH(uext, &s->unknown_header_ext, next) { ret =3D header_ext_add(buf, uext->magic, uext->data, uext->len, bu= flen); @@ -2558,6 +2665,13 @@ static int qcow2_truncate(BlockDriverState *bs, int6= 4_t offset, Error **errp) return -ENOTSUP; } =20 + /* cannot proceed if image has bitmaps */ + if (s->nb_bitmaps) { + /* TODO: resize bitmaps in the image */ + error_setg(errp, "Can't resize an image which has bitmaps"); + return -ENOTSUP; + } + /* shrinking is currently not supported */ if (offset < bs->total_sectors * 512) { error_setg(errp, "qcow2 doesn't support shrinking images yet"); diff --git a/block/qcow2.h b/block/qcow2.h index 41f8160629..f4a1f4cd78 100644 --- a/block/qcow2.h +++ b/block/qcow2.h @@ -52,6 +52,10 @@ * space for snapshot names and IDs */ #define QCOW_MAX_SNAPSHOTS_SIZE (1024 * QCOW_MAX_SNAPSHOTS) =20 +/* Bitmap header extension constraints */ +#define QCOW2_MAX_BITMAPS 65535 +#define QCOW2_MAX_BITMAP_DIRECTORY_SIZE (1024 * QCOW2_MAX_BITMAPS) + /* indicate that the refcount of the referenced cluster is exactly one. */ #define QCOW_OFLAG_COPIED (1ULL << 63) /* indicate that the cluster is compressed (they never have the copied fla= g) */ @@ -195,6 +199,14 @@ enum { QCOW2_COMPAT_FEAT_MASK =3D QCOW2_COMPAT_LAZY_REFCOUNTS, }; =20 +/* Autoclear feature bits */ +enum { + QCOW2_AUTOCLEAR_BITMAPS_BITNR =3D 0, + QCOW2_AUTOCLEAR_BITMAPS =3D 1 << QCOW2_AUTOCLEAR_BITMAPS_BITNR, + + QCOW2_AUTOCLEAR_MASK =3D QCOW2_AUTOCLEAR_BITMAPS, +}; + enum qcow2_discard_type { QCOW2_DISCARD_NEVER =3D 0, QCOW2_DISCARD_ALWAYS, @@ -222,6 +234,13 @@ typedef uint64_t Qcow2GetRefcountFunc(const void *refc= ount_array, typedef void Qcow2SetRefcountFunc(void *refcount_array, uint64_t index, uint64_t value); =20 +typedef struct Qcow2BitmapHeaderExt { + uint32_t nb_bitmaps; + uint32_t reserved32; + uint64_t bitmap_directory_size; + uint64_t bitmap_directory_offset; +} QEMU_PACKED Qcow2BitmapHeaderExt; + typedef struct BDRVQcow2State { int cluster_bits; int cluster_size; @@ -264,6 +283,10 @@ typedef struct BDRVQcow2State { unsigned int nb_snapshots; QCowSnapshot *snapshots; =20 + uint32_t nb_bitmaps; + uint64_t bitmap_directory_size; + uint64_t bitmap_directory_offset; + int flags; int qcow_version; bool use_lazy_refcounts; @@ -602,4 +625,8 @@ int qcow2_cache_get_empty(BlockDriverState *bs, Qcow2Ca= che *c, uint64_t offset, void **table); void qcow2_cache_put(BlockDriverState *bs, Qcow2Cache *c, void **table); =20 +/* qcow2-bitmap.c functions */ +int qcow2_check_bitmaps_refcounts(BlockDriverState *bs, BdrvCheckResult *r= es, + void **refcount_table, + int64_t *refcount_table_size); #endif --=20 2.11.1 From nobody Sat May 4 15:42:58 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1498652267253577.5787452105019; Wed, 28 Jun 2017 05:17:47 -0700 (PDT) Received: from localhost ([::1]:32877 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dQBuh-0000Vz-6b for importer@patchew.org; Wed, 28 Jun 2017 08:17:43 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:60288) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dQBj5-0006QB-1E for qemu-devel@nongnu.org; Wed, 28 Jun 2017 08:05:48 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1dQBj2-00031p-LV for qemu-devel@nongnu.org; Wed, 28 Jun 2017 08:05:42 -0400 Received: from mailhub.sw.ru ([195.214.232.25]:36592 helo=relay.sw.ru) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1dQBj2-0002vl-9L for qemu-devel@nongnu.org; Wed, 28 Jun 2017 08:05:40 -0400 Received: from kvm.sw.ru (msk-vpn.virtuozzo.com [195.214.232.6]) by relay.sw.ru (8.13.4/8.13.4) with ESMTP id v5SC5UUB025017; Wed, 28 Jun 2017 15:05:32 +0300 (MSK) From: Vladimir Sementsov-Ogievskiy To: qemu-block@nongnu.org, qemu-devel@nongnu.org Date: Wed, 28 Jun 2017 15:05:09 +0300 Message-Id: <20170628120530.31251-10-vsementsov@virtuozzo.com> X-Mailer: git-send-email 2.11.1 In-Reply-To: <20170628120530.31251-1-vsementsov@virtuozzo.com> References: <20170628120530.31251-1-vsementsov@virtuozzo.com> X-detected-operating-system: by eggs.gnu.org: OpenBSD 3.x [fuzzy] X-Received-From: 195.214.232.25 Subject: [Qemu-devel] [PATCH v22 09/30] block/dirty-bitmap: fix comment for BlockDirtyBitmap.disabled field X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, vsementsov@virtuozzo.com, famz@redhat.com, armbru@redhat.com, mreitz@redhat.com, stefanha@redhat.com, pbonzini@redhat.com, den@openvz.org, jsnow@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: John Snow Reviewed-by: Max Reitz --- block/dirty-bitmap.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/block/dirty-bitmap.c b/block/dirty-bitmap.c index f502c45a70..a8fe149c4a 100644 --- a/block/dirty-bitmap.c +++ b/block/dirty-bitmap.c @@ -43,7 +43,8 @@ struct BdrvDirtyBitmap { BdrvDirtyBitmap *successor; /* Anonymous child; implies frozen status = */ char *name; /* Optional non-empty unique ID */ int64_t size; /* Size of the bitmap (Number of sectors) = */ - bool disabled; /* Bitmap is read-only */ + bool disabled; /* Bitmap is disabled. It ignores all writ= es to + the device */ int active_iterators; /* How many iterators are active */ QLIST_ENTRY(BdrvDirtyBitmap) list; }; --=20 2.11.1 From nobody Sat May 4 15:42:58 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1498652573996461.1710234603855; Wed, 28 Jun 2017 05:22:53 -0700 (PDT) Received: from localhost ([::1]:32906 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dQBzg-0005NQ-4R for importer@patchew.org; Wed, 28 Jun 2017 08:22:52 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:60414) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dQBj7-0006Qh-GN for qemu-devel@nongnu.org; Wed, 28 Jun 2017 08:05:52 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1dQBj2-00032Z-VD for qemu-devel@nongnu.org; Wed, 28 Jun 2017 08:05:45 -0400 Received: from mailhub.sw.ru ([195.214.232.25]:35222 helo=relay.sw.ru) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1dQBj2-0002vK-F9 for qemu-devel@nongnu.org; Wed, 28 Jun 2017 08:05:40 -0400 Received: from kvm.sw.ru (msk-vpn.virtuozzo.com [195.214.232.6]) by relay.sw.ru (8.13.4/8.13.4) with ESMTP id v5SC5UUC025017; Wed, 28 Jun 2017 15:05:32 +0300 (MSK) From: Vladimir Sementsov-Ogievskiy To: qemu-block@nongnu.org, qemu-devel@nongnu.org Date: Wed, 28 Jun 2017 15:05:10 +0300 Message-Id: <20170628120530.31251-11-vsementsov@virtuozzo.com> X-Mailer: git-send-email 2.11.1 In-Reply-To: <20170628120530.31251-1-vsementsov@virtuozzo.com> References: <20170628120530.31251-1-vsementsov@virtuozzo.com> X-detected-operating-system: by eggs.gnu.org: OpenBSD 3.x [fuzzy] X-Received-From: 195.214.232.25 Subject: [Qemu-devel] [PATCH v22 10/30] block/dirty-bitmap: add readonly field to BdrvDirtyBitmap X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, vsementsov@virtuozzo.com, famz@redhat.com, armbru@redhat.com, mreitz@redhat.com, stefanha@redhat.com, pbonzini@redhat.com, den@openvz.org, jsnow@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" It will be needed in following commits for persistent bitmaps. If bitmap is loaded from read-only storage (and we can't mark it "in use" in this storage) corresponding BdrvDirtyBitmap should be read-only. Signed-off-by: Vladimir Sementsov-Ogievskiy --- block/dirty-bitmap.c | 36 ++++++++++++++++++++++++++++++++++++ block/io.c | 8 ++++++++ blockdev.c | 6 ++++++ include/block/dirty-bitmap.h | 4 ++++ 4 files changed, 54 insertions(+) diff --git a/block/dirty-bitmap.c b/block/dirty-bitmap.c index a8fe149c4a..17d3068336 100644 --- a/block/dirty-bitmap.c +++ b/block/dirty-bitmap.c @@ -46,6 +46,12 @@ struct BdrvDirtyBitmap { bool disabled; /* Bitmap is disabled. It ignores all writ= es to the device */ int active_iterators; /* How many iterators are active */ + bool readonly; /* Bitmap is read-only. This field also + prevents the respective image from being + modified (i.e. blocks writes and discar= ds). + Such operations must fail and both the = image + and this bitmap must remain unchanged w= hile + this flag is set. */ QLIST_ENTRY(BdrvDirtyBitmap) list; }; =20 @@ -505,6 +511,7 @@ void bdrv_set_dirty_bitmap_locked(BdrvDirtyBitmap *bitm= ap, int64_t cur_sector, int64_t nr_sectors) { assert(bdrv_dirty_bitmap_enabled(bitmap)); + assert(!bdrv_dirty_bitmap_readonly(bitmap)); hbitmap_set(bitmap->bitmap, cur_sector, nr_sectors); } =20 @@ -521,6 +528,7 @@ void bdrv_reset_dirty_bitmap_locked(BdrvDirtyBitmap *bi= tmap, int64_t cur_sector, int64_t nr_sectors) { assert(bdrv_dirty_bitmap_enabled(bitmap)); + assert(!bdrv_dirty_bitmap_readonly(bitmap)); hbitmap_reset(bitmap->bitmap, cur_sector, nr_sectors); } =20 @@ -535,6 +543,7 @@ void bdrv_reset_dirty_bitmap(BdrvDirtyBitmap *bitmap, void bdrv_clear_dirty_bitmap(BdrvDirtyBitmap *bitmap, HBitmap **out) { assert(bdrv_dirty_bitmap_enabled(bitmap)); + assert(!bdrv_dirty_bitmap_readonly(bitmap)); bdrv_dirty_bitmap_lock(bitmap); if (!out) { hbitmap_reset_all(bitmap->bitmap); @@ -551,6 +560,7 @@ void bdrv_undo_clear_dirty_bitmap(BdrvDirtyBitmap *bitm= ap, HBitmap *in) { HBitmap *tmp =3D bitmap->bitmap; assert(bdrv_dirty_bitmap_enabled(bitmap)); + assert(!bdrv_dirty_bitmap_readonly(bitmap)); bitmap->bitmap =3D in; hbitmap_free(tmp); } @@ -613,6 +623,7 @@ void bdrv_set_dirty(BlockDriverState *bs, int64_t cur_s= ector, if (!bdrv_dirty_bitmap_enabled(bitmap)) { continue; } + assert(!bdrv_dirty_bitmap_readonly(bitmap)); hbitmap_set(bitmap->bitmap, cur_sector, nr_sectors); } bdrv_dirty_bitmaps_unlock(bs); @@ -635,3 +646,28 @@ int64_t bdrv_get_meta_dirty_count(BdrvDirtyBitmap *bit= map) { return hbitmap_count(bitmap->meta); } + +bool bdrv_dirty_bitmap_readonly(const BdrvDirtyBitmap *bitmap) +{ + return bitmap->readonly; +} + +/* Called with BQL taken. */ +void bdrv_dirty_bitmap_set_readonly(BdrvDirtyBitmap *bitmap, bool value) +{ + qemu_mutex_lock(bitmap->mutex); + bitmap->readonly =3D value; + qemu_mutex_unlock(bitmap->mutex); +} + +bool bdrv_has_readonly_bitmaps(BlockDriverState *bs) +{ + BdrvDirtyBitmap *bm; + QLIST_FOREACH(bm, &bs->dirty_bitmaps, list) { + if (bm->readonly) { + return true; + } + } + + return false; +} diff --git a/block/io.c b/block/io.c index 91611ffb2a..49057f19af 100644 --- a/block/io.c +++ b/block/io.c @@ -1343,6 +1343,10 @@ static int coroutine_fn bdrv_aligned_pwritev(BdrvChi= ld *child, uint64_t bytes_remaining =3D bytes; int max_transfer; =20 + if (bdrv_has_readonly_bitmaps(bs)) { + return -EPERM; + } + assert(is_power_of_2(align)); assert((offset & (align - 1)) =3D=3D 0); assert((bytes & (align - 1)) =3D=3D 0); @@ -2435,6 +2439,10 @@ int coroutine_fn bdrv_co_pdiscard(BlockDriverState *= bs, int64_t offset, return -ENOMEDIUM; } =20 + if (bdrv_has_readonly_bitmaps(bs)) { + return -EPERM; + } + ret =3D bdrv_check_byte_request(bs, offset, count); if (ret < 0) { return ret; diff --git a/blockdev.c b/blockdev.c index f92dcf24bf..64e03c0caf 100644 --- a/blockdev.c +++ b/blockdev.c @@ -2023,6 +2023,9 @@ static void block_dirty_bitmap_clear_prepare(BlkActio= nState *common, } else if (!bdrv_dirty_bitmap_enabled(state->bitmap)) { error_setg(errp, "Cannot clear a disabled bitmap"); return; + } else if (bdrv_dirty_bitmap_readonly(state->bitmap)) { + error_setg(errp, "Cannot clear a readonly bitmap"); + return; } =20 bdrv_clear_dirty_bitmap(state->bitmap, &state->backup); @@ -2791,6 +2794,9 @@ void qmp_block_dirty_bitmap_clear(const char *node, c= onst char *name, "Bitmap '%s' is currently disabled and cannot be cleare= d", name); return; + } else if (bdrv_dirty_bitmap_readonly(bitmap)) { + error_setg(errp, "Bitmap '%s' is readonly and cannot be cleared", = name); + return; } =20 bdrv_clear_dirty_bitmap(bitmap, NULL); diff --git a/include/block/dirty-bitmap.h b/include/block/dirty-bitmap.h index 05451c727d..cb43fa37e2 100644 --- a/include/block/dirty-bitmap.h +++ b/include/block/dirty-bitmap.h @@ -71,6 +71,8 @@ void bdrv_dirty_bitmap_deserialize_ones(BdrvDirtyBitmap *= bitmap, bool finish); void bdrv_dirty_bitmap_deserialize_finish(BdrvDirtyBitmap *bitmap); =20 +void bdrv_dirty_bitmap_set_readonly(BdrvDirtyBitmap *bitmap, bool value); + /* Functions that require manual locking. */ void bdrv_dirty_bitmap_lock(BdrvDirtyBitmap *bitmap); void bdrv_dirty_bitmap_unlock(BdrvDirtyBitmap *bitmap); @@ -85,5 +87,7 @@ void bdrv_set_dirty_iter(BdrvDirtyBitmapIter *hbi, int64_= t sector_num); int64_t bdrv_get_dirty_count(BdrvDirtyBitmap *bitmap); int64_t bdrv_get_meta_dirty_count(BdrvDirtyBitmap *bitmap); void bdrv_dirty_bitmap_truncate(BlockDriverState *bs); +bool bdrv_dirty_bitmap_readonly(const BdrvDirtyBitmap *bitmap); +bool bdrv_has_readonly_bitmaps(BlockDriverState *bs); =20 #endif --=20 2.11.1 From nobody Sat May 4 15:42:58 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 149865211252519.123425790124998; Wed, 28 Jun 2017 05:15:12 -0700 (PDT) Received: from localhost ([::1]:32866 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dQBsE-0006RK-6y for importer@patchew.org; Wed, 28 Jun 2017 08:15:10 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:60376) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dQBj6-0006QN-OP for qemu-devel@nongnu.org; Wed, 28 Jun 2017 08:05:49 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1dQBj2-00031O-Gh for qemu-devel@nongnu.org; Wed, 28 Jun 2017 08:05:44 -0400 Received: from mailhub.sw.ru ([195.214.232.25]:2871 helo=relay.sw.ru) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1dQBj1-0002ws-UQ for qemu-devel@nongnu.org; Wed, 28 Jun 2017 08:05:40 -0400 Received: from kvm.sw.ru (msk-vpn.virtuozzo.com [195.214.232.6]) by relay.sw.ru (8.13.4/8.13.4) with ESMTP id v5SC5UUD025017; Wed, 28 Jun 2017 15:05:32 +0300 (MSK) From: Vladimir Sementsov-Ogievskiy To: qemu-block@nongnu.org, qemu-devel@nongnu.org Date: Wed, 28 Jun 2017 15:05:11 +0300 Message-Id: <20170628120530.31251-12-vsementsov@virtuozzo.com> X-Mailer: git-send-email 2.11.1 In-Reply-To: <20170628120530.31251-1-vsementsov@virtuozzo.com> References: <20170628120530.31251-1-vsementsov@virtuozzo.com> X-detected-operating-system: by eggs.gnu.org: OpenBSD 3.x [fuzzy] X-Received-From: 195.214.232.25 Subject: [Qemu-devel] [PATCH v22 11/30] qcow2: autoloading dirty bitmaps X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, vsementsov@virtuozzo.com, famz@redhat.com, armbru@redhat.com, mreitz@redhat.com, stefanha@redhat.com, pbonzini@redhat.com, den@openvz.org, jsnow@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Auto loading bitmaps are bitmaps in Qcow2, with the AUTO flag set. They are loaded when the image is opened and become BdrvDirtyBitmaps for the corresponding drive. Extra data in bitmaps is not supported for now. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Max Reitz Reviewed-by: John Snow --- block/qcow2-bitmap.c | 389 +++++++++++++++++++++++++++++++++++++++++++++++= ++++ block/qcow2.c | 17 ++- block/qcow2.h | 2 + 3 files changed, 406 insertions(+), 2 deletions(-) diff --git a/block/qcow2-bitmap.c b/block/qcow2-bitmap.c index b8e472b3e8..2c7b057e21 100644 --- a/block/qcow2-bitmap.c +++ b/block/qcow2-bitmap.c @@ -44,6 +44,8 @@ =20 /* Bitmap directory entry flags */ #define BME_RESERVED_FLAGS 0xfffffffcU +#define BME_FLAG_IN_USE (1U << 0) +#define BME_FLAG_AUTO (1U << 1) =20 /* bits [1, 8] U [56, 63] are reserved */ #define BME_TABLE_ENTRY_RESERVED_MASK 0xff000000000001feULL @@ -85,6 +87,23 @@ typedef enum BitmapType { BT_DIRTY_TRACKING_BITMAP =3D 1 } BitmapType; =20 +static inline bool can_write(BlockDriverState *bs) +{ + return !bdrv_is_read_only(bs) && !(bdrv_get_flags(bs) & BDRV_O_INACTIV= E); +} + +static int update_header_sync(BlockDriverState *bs) +{ + int ret; + + ret =3D qcow2_update_header(bs); + if (ret < 0) { + return ret; + } + + return bdrv_flush(bs); +} + static int check_table_entry(uint64_t entry, int cluster_size) { uint64_t offset; @@ -146,6 +165,120 @@ fail: return ret; } =20 +/* This function returns the number of disk sectors covered by a single qc= ow2 + * cluster of bitmap data. */ +static uint64_t sectors_covered_by_bitmap_cluster(const BDRVQcow2State *s, + const BdrvDirtyBitmap *b= itmap) +{ + uint32_t sector_granularity =3D + bdrv_dirty_bitmap_granularity(bitmap) >> BDRV_SECTOR_BITS; + + return (uint64_t)sector_granularity * (s->cluster_size << 3); +} + +/* load_bitmap_data + * @bitmap_table entries must satisfy specification constraints. + * @bitmap must be cleared */ +static int load_bitmap_data(BlockDriverState *bs, + const uint64_t *bitmap_table, + uint32_t bitmap_table_size, + BdrvDirtyBitmap *bitmap) +{ + int ret =3D 0; + BDRVQcow2State *s =3D bs->opaque; + uint64_t sector, sbc; + uint64_t bm_size =3D bdrv_dirty_bitmap_size(bitmap); + uint8_t *buf =3D NULL; + uint64_t i, tab_size =3D + size_to_clusters(s, + bdrv_dirty_bitmap_serialization_size(bitmap, 0, bm_size)); + + if (tab_size !=3D bitmap_table_size || tab_size > BME_MAX_TABLE_SIZE) { + return -EINVAL; + } + + buf =3D g_malloc(s->cluster_size); + sbc =3D sectors_covered_by_bitmap_cluster(s, bitmap); + for (i =3D 0, sector =3D 0; i < tab_size; ++i, sector +=3D sbc) { + uint64_t count =3D MIN(bm_size - sector, sbc); + uint64_t entry =3D bitmap_table[i]; + uint64_t offset =3D entry & BME_TABLE_ENTRY_OFFSET_MASK; + + assert(check_table_entry(entry, s->cluster_size) =3D=3D 0); + + if (offset =3D=3D 0) { + if (entry & BME_TABLE_ENTRY_FLAG_ALL_ONES) { + bdrv_dirty_bitmap_deserialize_ones(bitmap, sector, count, + false); + } else { + /* No need to deserialize zeros because the dirty bitmap is + * already cleared */ + } + } else { + ret =3D bdrv_pread(bs->file, offset, buf, s->cluster_size); + if (ret < 0) { + goto finish; + } + bdrv_dirty_bitmap_deserialize_part(bitmap, buf, sector, count, + false); + } + } + ret =3D 0; + + bdrv_dirty_bitmap_deserialize_finish(bitmap); + +finish: + g_free(buf); + + return ret; +} + +static BdrvDirtyBitmap *load_bitmap(BlockDriverState *bs, + Qcow2Bitmap *bm, Error **errp) +{ + int ret; + uint64_t *bitmap_table =3D NULL; + uint32_t granularity; + BdrvDirtyBitmap *bitmap =3D NULL; + + if (bm->flags & BME_FLAG_IN_USE) { + error_setg(errp, "Bitmap '%s' is in use", bm->name); + goto fail; + } + + ret =3D bitmap_table_load(bs, &bm->table, &bitmap_table); + if (ret < 0) { + error_setg_errno(errp, -ret, + "Could not read bitmap_table table from image for= " + "bitmap '%s'", bm->name); + goto fail; + } + + granularity =3D 1U << bm->granularity_bits; + bitmap =3D bdrv_create_dirty_bitmap(bs, granularity, bm->name, errp); + if (bitmap =3D=3D NULL) { + goto fail; + } + + ret =3D load_bitmap_data(bs, bitmap_table, bm->table.size, bitmap); + if (ret < 0) { + error_setg_errno(errp, -ret, "Could not read bitmap '%s' from imag= e", + bm->name); + goto fail; + } + + g_free(bitmap_table); + return bitmap; + +fail: + g_free(bitmap_table); + if (bitmap !=3D NULL) { + bdrv_release_dirty_bitmap(bs, bitmap); + } + + return NULL; +} + /* * Bitmap List */ @@ -164,6 +297,15 @@ static inline void bitmap_dir_entry_to_cpu(Qcow2Bitmap= DirEntry *entry) be32_to_cpus(&entry->extra_data_size); } =20 +static inline void bitmap_dir_entry_to_be(Qcow2BitmapDirEntry *entry) +{ + cpu_to_be64s(&entry->bitmap_table_offset); + cpu_to_be32s(&entry->bitmap_table_size); + cpu_to_be32s(&entry->flags); + cpu_to_be16s(&entry->name_size); + cpu_to_be32s(&entry->extra_data_size); +} + static inline int calc_dir_entry_size(size_t name_size, size_t extra_data_= size) { return align_offset(sizeof(Qcow2BitmapDirEntry) + @@ -224,6 +366,17 @@ static int check_dir_entry(BlockDriverState *bs, Qcow2= BitmapDirEntry *entry) return fail ? -EINVAL : 0; } =20 +static inline void bitmap_directory_to_be(uint8_t *dir, size_t size) +{ + uint8_t *end =3D dir + size; + while (dir < end) { + Qcow2BitmapDirEntry *e =3D (Qcow2BitmapDirEntry *)dir; + dir +=3D dir_entry_size(e); + + bitmap_dir_entry_to_be(e); + } +} + /* * Bitmap List public functions */ @@ -258,6 +411,18 @@ static Qcow2BitmapList *bitmap_list_new(void) return bm_list; } =20 +static uint32_t bitmap_list_count(Qcow2BitmapList *bm_list) +{ + Qcow2Bitmap *bm; + uint32_t nb_bitmaps =3D 0; + + QSIMPLEQ_FOREACH(bm, bm_list, entry) { + nb_bitmaps++; + } + + return nb_bitmaps; +} + /* bitmap_list_load * Get bitmap list from qcow2 image. Actually reads bitmap directory, * checks it and convert to bitmap list. @@ -437,3 +602,227 @@ out: =20 return ret; } + +/* bitmap_list_store + * Store bitmap list to qcow2 image as a bitmap directory. + * Everything is checked. + */ +static int bitmap_list_store(BlockDriverState *bs, Qcow2BitmapList *bm_lis= t, + uint64_t *offset, uint64_t *size, bool in_pla= ce) +{ + int ret; + uint8_t *dir; + int64_t dir_offset =3D 0; + uint64_t dir_size =3D 0; + Qcow2Bitmap *bm; + Qcow2BitmapDirEntry *e; + + QSIMPLEQ_FOREACH(bm, bm_list, entry) { + dir_size +=3D calc_dir_entry_size(strlen(bm->name), 0); + } + + if (dir_size =3D=3D 0 || dir_size > QCOW2_MAX_BITMAP_DIRECTORY_SIZE) { + return -EINVAL; + } + + if (in_place) { + if (*size !=3D dir_size || *offset =3D=3D 0) { + return -EINVAL; + } + + dir_offset =3D *offset; + } + + dir =3D g_try_malloc(dir_size); + if (dir =3D=3D NULL) { + return -ENOMEM; + } + + e =3D (Qcow2BitmapDirEntry *)dir; + QSIMPLEQ_FOREACH(bm, bm_list, entry) { + e->bitmap_table_offset =3D bm->table.offset; + e->bitmap_table_size =3D bm->table.size; + e->flags =3D bm->flags; + e->type =3D BT_DIRTY_TRACKING_BITMAP; + e->granularity_bits =3D bm->granularity_bits; + e->name_size =3D strlen(bm->name); + e->extra_data_size =3D 0; + memcpy(e + 1, bm->name, e->name_size); + + if (check_dir_entry(bs, e) < 0) { + ret =3D -EINVAL; + goto fail; + } + + e =3D next_dir_entry(e); + } + + bitmap_directory_to_be(dir, dir_size); + + if (!in_place) { + dir_offset =3D qcow2_alloc_clusters(bs, dir_size); + if (dir_offset < 0) { + ret =3D dir_offset; + goto fail; + } + } + + ret =3D qcow2_pre_write_overlap_check(bs, 0, dir_offset, dir_size); + if (ret < 0) { + goto fail; + } + + ret =3D bdrv_pwrite(bs->file, dir_offset, dir, dir_size); + if (ret < 0) { + goto fail; + } + + g_free(dir); + + if (!in_place) { + *size =3D dir_size; + *offset =3D dir_offset; + } + + return 0; + +fail: + g_free(dir); + + if (!in_place && dir_offset > 0) { + qcow2_free_clusters(bs, dir_offset, dir_size, QCOW2_DISCARD_OTHER); + } + + return ret; +} + +/* + * Bitmap List end + */ + +static int update_ext_header_and_dir_in_place(BlockDriverState *bs, + Qcow2BitmapList *bm_list) +{ + BDRVQcow2State *s =3D bs->opaque; + int ret; + + if (!(s->autoclear_features & QCOW2_AUTOCLEAR_BITMAPS) || + bm_list =3D=3D NULL || QSIMPLEQ_EMPTY(bm_list) || + bitmap_list_count(bm_list) !=3D s->nb_bitmaps) + { + return -EINVAL; + } + + s->autoclear_features &=3D ~(uint64_t)QCOW2_AUTOCLEAR_BITMAPS; + ret =3D update_header_sync(bs); + if (ret < 0) { + /* Two variants are possible here: + * 1. Autoclear flag is dropped, all bitmaps will be lost. + * 2. Autoclear flag is not dropped, old state is left. + */ + return ret; + } + + /* autoclear bit is not set, so we can safely update bitmap directory = */ + + ret =3D bitmap_list_store(bs, bm_list, &s->bitmap_directory_offset, + &s->bitmap_directory_size, true); + if (ret < 0) { + /* autoclear bit is cleared, so all leaked clusters would be remov= ed on + * qemu-img check */ + return ret; + } + + ret =3D update_header_sync(bs); + if (ret < 0) { + /* autoclear bit is cleared, so all leaked clusters would be remov= ed on + * qemu-img check */ + return ret; + } + + s->autoclear_features |=3D QCOW2_AUTOCLEAR_BITMAPS; + return update_header_sync(bs); + /* If final update_header_sync() fails, two variants are possible: + * 1. Autoclear flag is not set, all bitmaps will be lost. + * 2. Autoclear flag is set, header and directory are successfully upd= ated. + */ +} + +/* for g_slist_foreach for GSList of BdrvDirtyBitmap* elements */ +static void release_dirty_bitmap_helper(gpointer bitmap, + gpointer bs) +{ + bdrv_release_dirty_bitmap(bs, bitmap); +} + +/* for g_slist_foreach for GSList of BdrvDirtyBitmap* elements */ +static void set_readonly_helper(gpointer bitmap, gpointer value) +{ + bdrv_dirty_bitmap_set_readonly(bitmap, (bool)value); +} + +/* qcow2_load_autoloading_dirty_bitmaps() + * Return value is a hint for caller: true means that the Qcow2 header was + * updated. (false doesn't mean that the header should be updated by the + * caller, it just means that updating was not needed or the image cannot = be + * written to). + * On failure the function returns false. + */ +bool qcow2_load_autoloading_dirty_bitmaps(BlockDriverState *bs, Error **er= rp) +{ + BDRVQcow2State *s =3D bs->opaque; + Qcow2BitmapList *bm_list; + Qcow2Bitmap *bm; + GSList *created_dirty_bitmaps =3D NULL; + bool header_updated =3D false; + + if (s->nb_bitmaps =3D=3D 0) { + /* No bitmaps - nothing to do */ + return false; + } + + bm_list =3D bitmap_list_load(bs, s->bitmap_directory_offset, + s->bitmap_directory_size, errp); + if (bm_list =3D=3D NULL) { + return false; + } + + QSIMPLEQ_FOREACH(bm, bm_list, entry) { + if ((bm->flags & BME_FLAG_AUTO) && !(bm->flags & BME_FLAG_IN_USE))= { + BdrvDirtyBitmap *bitmap =3D load_bitmap(bs, bm, errp); + if (bitmap =3D=3D NULL) { + goto fail; + } + bm->flags |=3D BME_FLAG_IN_USE; + created_dirty_bitmaps =3D + g_slist_append(created_dirty_bitmaps, bitmap); + } + } + + if (created_dirty_bitmaps !=3D NULL) { + if (can_write(bs)) { + /* in_use flags must be updated */ + int ret =3D update_ext_header_and_dir_in_place(bs, bm_list); + if (ret < 0) { + error_setg_errno(errp, -ret, "Can't update bitmap director= y"); + goto fail; + } + header_updated =3D true; + } else { + g_slist_foreach(created_dirty_bitmaps, set_readonly_helper, + (gpointer)true); + } + } + + g_slist_free(created_dirty_bitmaps); + bitmap_list_free(bm_list); + + return header_updated; + +fail: + g_slist_foreach(created_dirty_bitmaps, release_dirty_bitmap_helper, bs= ); + g_slist_free(created_dirty_bitmaps); + bitmap_list_free(bm_list); + + return false; +} diff --git a/block/qcow2.c b/block/qcow2.c index b9657d8099..92e8ff064d 100644 --- a/block/qcow2.c +++ b/block/qcow2.c @@ -1239,9 +1239,22 @@ static int qcow2_do_open(BlockDriverState *bs, QDict= *options, int flags, =20 /* Clear unknown autoclear feature bits */ update_header |=3D s->autoclear_features & ~QCOW2_AUTOCLEAR_MASK; - - if (update_header && !bs->read_only && !(flags & BDRV_O_INACTIVE)) { + update_header =3D + update_header && !bs->read_only && !(flags & BDRV_O_INACTIVE); + if (update_header) { s->autoclear_features &=3D QCOW2_AUTOCLEAR_MASK; + } + + if (qcow2_load_autoloading_dirty_bitmaps(bs, &local_err)) { + update_header =3D false; + } + if (local_err !=3D NULL) { + error_propagate(errp, local_err); + ret =3D -EINVAL; + goto fail; + } + + if (update_header) { ret =3D qcow2_update_header(bs); if (ret < 0) { error_setg_errno(errp, -ret, "Could not update qcow2 header"); diff --git a/block/qcow2.h b/block/qcow2.h index f4a1f4cd78..67c61de008 100644 --- a/block/qcow2.h +++ b/block/qcow2.h @@ -629,4 +629,6 @@ void qcow2_cache_put(BlockDriverState *bs, Qcow2Cache *= c, void **table); int qcow2_check_bitmaps_refcounts(BlockDriverState *bs, BdrvCheckResult *r= es, void **refcount_table, int64_t *refcount_table_size); +bool qcow2_load_autoloading_dirty_bitmaps(BlockDriverState *bs, Error **er= rp); + #endif --=20 2.11.1 From nobody Sat May 4 15:42:58 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1498652575337475.5952139379324; Wed, 28 Jun 2017 05:22:55 -0700 (PDT) Received: from localhost ([::1]:32907 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dQBzg-0005Nl-VQ for importer@patchew.org; Wed, 28 Jun 2017 08:22:53 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:60299) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dQBj5-0006QF-3z for qemu-devel@nongnu.org; Wed, 28 Jun 2017 08:05:48 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1dQBj2-00031U-Gs for qemu-devel@nongnu.org; Wed, 28 Jun 2017 08:05:42 -0400 Received: from mailhub.sw.ru ([195.214.232.25]:13803 helo=relay.sw.ru) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1dQBj2-0002vJ-4w for qemu-devel@nongnu.org; Wed, 28 Jun 2017 08:05:40 -0400 Received: from kvm.sw.ru (msk-vpn.virtuozzo.com [195.214.232.6]) by relay.sw.ru (8.13.4/8.13.4) with ESMTP id v5SC5UUE025017; Wed, 28 Jun 2017 15:05:32 +0300 (MSK) From: Vladimir Sementsov-Ogievskiy To: qemu-block@nongnu.org, qemu-devel@nongnu.org Date: Wed, 28 Jun 2017 15:05:12 +0300 Message-Id: <20170628120530.31251-13-vsementsov@virtuozzo.com> X-Mailer: git-send-email 2.11.1 In-Reply-To: <20170628120530.31251-1-vsementsov@virtuozzo.com> References: <20170628120530.31251-1-vsementsov@virtuozzo.com> X-detected-operating-system: by eggs.gnu.org: OpenBSD 3.x [fuzzy] X-Received-From: 195.214.232.25 Subject: [Qemu-devel] [PATCH v22 12/30] block: refactor bdrv_reopen_commit X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, vsementsov@virtuozzo.com, famz@redhat.com, armbru@redhat.com, mreitz@redhat.com, stefanha@redhat.com, pbonzini@redhat.com, den@openvz.org, jsnow@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Add bs local variable to simplify code. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: John Snow --- block.c | 14 ++++++++------ 1 file changed, 8 insertions(+), 6 deletions(-) diff --git a/block.c b/block.c index 694396281b..37d68e3276 100644 --- a/block.c +++ b/block.c @@ -2989,9 +2989,11 @@ error: void bdrv_reopen_commit(BDRVReopenState *reopen_state) { BlockDriver *drv; + BlockDriverState *bs; =20 assert(reopen_state !=3D NULL); - drv =3D reopen_state->bs->drv; + bs =3D reopen_state->bs; + drv =3D bs->drv; assert(drv !=3D NULL); =20 /* If there are any driver level actions to take */ @@ -3000,13 +3002,13 @@ void bdrv_reopen_commit(BDRVReopenState *reopen_sta= te) } =20 /* set BDS specific flags now */ - QDECREF(reopen_state->bs->explicit_options); + QDECREF(bs->explicit_options); =20 - reopen_state->bs->explicit_options =3D reopen_state->explicit_option= s; - reopen_state->bs->open_flags =3D reopen_state->flags; - reopen_state->bs->read_only =3D !(reopen_state->flags & BDRV_O_RDWR); + bs->explicit_options =3D reopen_state->explicit_options; + bs->open_flags =3D reopen_state->flags; + bs->read_only =3D !(reopen_state->flags & BDRV_O_RDWR); =20 - bdrv_refresh_limits(reopen_state->bs, NULL); + bdrv_refresh_limits(bs, NULL); } =20 /* --=20 2.11.1 From nobody Sat May 4 15:42:58 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1498651819800927.1288640907575; Wed, 28 Jun 2017 05:10:19 -0700 (PDT) Received: from localhost ([::1]:32842 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dQBnS-0001zv-Hq for importer@patchew.org; Wed, 28 Jun 2017 08:10:14 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:60386) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dQBj6-0006QQ-Sm for qemu-devel@nongnu.org; Wed, 28 Jun 2017 08:05:48 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1dQBj3-00032g-1K for qemu-devel@nongnu.org; Wed, 28 Jun 2017 08:05:44 -0400 Received: from mailhub.sw.ru ([195.214.232.25]:42486 helo=relay.sw.ru) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1dQBj2-0002wZ-L4 for qemu-devel@nongnu.org; Wed, 28 Jun 2017 08:05:40 -0400 Received: from kvm.sw.ru (msk-vpn.virtuozzo.com [195.214.232.6]) by relay.sw.ru (8.13.4/8.13.4) with ESMTP id v5SC5UUF025017; Wed, 28 Jun 2017 15:05:32 +0300 (MSK) From: Vladimir Sementsov-Ogievskiy To: qemu-block@nongnu.org, qemu-devel@nongnu.org Date: Wed, 28 Jun 2017 15:05:13 +0300 Message-Id: <20170628120530.31251-14-vsementsov@virtuozzo.com> X-Mailer: git-send-email 2.11.1 In-Reply-To: <20170628120530.31251-1-vsementsov@virtuozzo.com> References: <20170628120530.31251-1-vsementsov@virtuozzo.com> X-detected-operating-system: by eggs.gnu.org: OpenBSD 3.x [fuzzy] X-Received-From: 195.214.232.25 Subject: [Qemu-devel] [PATCH v22 13/30] block: new bdrv_reopen_bitmaps_rw interface X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, vsementsov@virtuozzo.com, famz@redhat.com, armbru@redhat.com, mreitz@redhat.com, stefanha@redhat.com, pbonzini@redhat.com, den@openvz.org, jsnow@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Add format driver handler, which should mark loaded read-only bitmaps as 'IN_USE' in the image and unset read_only field in corresponding BdrvDirtyBitmap's. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: John Snow Reviewed-by: Max Reitz --- block.c | 19 +++++++++++++++++++ include/block/block_int.h | 7 +++++++ 2 files changed, 26 insertions(+) diff --git a/block.c b/block.c index 37d68e3276..3f83da178d 100644 --- a/block.c +++ b/block.c @@ -2990,12 +2990,16 @@ void bdrv_reopen_commit(BDRVReopenState *reopen_sta= te) { BlockDriver *drv; BlockDriverState *bs; + bool old_can_write, new_can_write; =20 assert(reopen_state !=3D NULL); bs =3D reopen_state->bs; drv =3D bs->drv; assert(drv !=3D NULL); =20 + old_can_write =3D + !bdrv_is_read_only(bs) && !(bdrv_get_flags(bs) & BDRV_O_INACTIVE); + /* If there are any driver level actions to take */ if (drv->bdrv_reopen_commit) { drv->bdrv_reopen_commit(reopen_state); @@ -3009,6 +3013,21 @@ void bdrv_reopen_commit(BDRVReopenState *reopen_stat= e) bs->read_only =3D !(reopen_state->flags & BDRV_O_RDWR); =20 bdrv_refresh_limits(bs, NULL); + + new_can_write =3D + !bdrv_is_read_only(bs) && !(bdrv_get_flags(bs) & BDRV_O_INACTIVE); + if (!old_can_write && new_can_write && drv->bdrv_reopen_bitmaps_rw) { + Error *local_err =3D NULL; + if (drv->bdrv_reopen_bitmaps_rw(bs, &local_err) < 0) { + /* This is not fatal, bitmaps just left read-only, so all foll= owing + * writes will fail. User can remove read-only bitmaps to unbl= ock + * writes. + */ + error_reportf_err(local_err, + "%s: Failed to make dirty bitmaps writable: = ", + bdrv_get_node_name(bs)); + } + } } =20 /* diff --git a/include/block/block_int.h b/include/block/block_int.h index 748970055e..4ad8eec2dd 100644 --- a/include/block/block_int.h +++ b/include/block/block_int.h @@ -381,6 +381,13 @@ struct BlockDriver { uint64_t parent_perm, uint64_t parent_shared, uint64_t *nperm, uint64_t *nshared); =20 + /** + * Bitmaps should be marked as 'IN_USE' in the image on reopening image + * as rw. This handler should realize it. It also should unset readonly + * field of BlockDirtyBitmap's in case of success. + */ + int (*bdrv_reopen_bitmaps_rw)(BlockDriverState *bs, Error **errp); + QLIST_ENTRY(BlockDriver) list; }; =20 --=20 2.11.1 From nobody Sat May 4 15:42:58 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 149865242285520.364966807983592; Wed, 28 Jun 2017 05:20:22 -0700 (PDT) Received: from localhost ([::1]:32887 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dQBxE-0003FA-8V for importer@patchew.org; Wed, 28 Jun 2017 08:20:20 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:60381) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dQBj6-0006QP-Q6 for qemu-devel@nongnu.org; Wed, 28 Jun 2017 08:05:51 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1dQBj3-000332-5p for qemu-devel@nongnu.org; Wed, 28 Jun 2017 08:05:44 -0400 Received: from mailhub.sw.ru ([195.214.232.25]:14838 helo=relay.sw.ru) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1dQBj2-0002v7-Pt for qemu-devel@nongnu.org; Wed, 28 Jun 2017 08:05:41 -0400 Received: from kvm.sw.ru (msk-vpn.virtuozzo.com [195.214.232.6]) by relay.sw.ru (8.13.4/8.13.4) with ESMTP id v5SC5UUG025017; Wed, 28 Jun 2017 15:05:32 +0300 (MSK) From: Vladimir Sementsov-Ogievskiy To: qemu-block@nongnu.org, qemu-devel@nongnu.org Date: Wed, 28 Jun 2017 15:05:14 +0300 Message-Id: <20170628120530.31251-15-vsementsov@virtuozzo.com> X-Mailer: git-send-email 2.11.1 In-Reply-To: <20170628120530.31251-1-vsementsov@virtuozzo.com> References: <20170628120530.31251-1-vsementsov@virtuozzo.com> X-detected-operating-system: by eggs.gnu.org: OpenBSD 3.x [fuzzy] X-Received-From: 195.214.232.25 Subject: [Qemu-devel] [PATCH v22 14/30] qcow2: support .bdrv_reopen_bitmaps_rw X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, vsementsov@virtuozzo.com, famz@redhat.com, armbru@redhat.com, mreitz@redhat.com, stefanha@redhat.com, pbonzini@redhat.com, den@openvz.org, jsnow@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Realize bdrv_reopen_bitmaps_rw interface. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: John Snow Reviewed-by: Max Reitz --- block/qcow2-bitmap.c | 61 ++++++++++++++++++++++++++++++++++++++++++++++++= ++++ block/qcow2.c | 2 ++ block/qcow2.h | 1 + 3 files changed, 64 insertions(+) diff --git a/block/qcow2-bitmap.c b/block/qcow2-bitmap.c index 2c7b057e21..a21fab8ce8 100644 --- a/block/qcow2-bitmap.c +++ b/block/qcow2-bitmap.c @@ -826,3 +826,64 @@ fail: =20 return false; } + +int qcow2_reopen_bitmaps_rw(BlockDriverState *bs, Error **errp) +{ + BDRVQcow2State *s =3D bs->opaque; + Qcow2BitmapList *bm_list; + Qcow2Bitmap *bm; + GSList *ro_dirty_bitmaps =3D NULL; + int ret =3D 0; + + if (s->nb_bitmaps =3D=3D 0) { + /* No bitmaps - nothing to do */ + return 0; + } + + if (!can_write(bs)) { + error_setg(errp, "Can't write to the image on reopening bitmaps rw= "); + return -EINVAL; + } + + bm_list =3D bitmap_list_load(bs, s->bitmap_directory_offset, + s->bitmap_directory_size, errp); + if (bm_list =3D=3D NULL) { + return -EINVAL; + } + + QSIMPLEQ_FOREACH(bm, bm_list, entry) { + if (!(bm->flags & BME_FLAG_IN_USE)) { + BdrvDirtyBitmap *bitmap =3D bdrv_find_dirty_bitmap(bs, bm->nam= e); + if (bitmap =3D=3D NULL) { + continue; + } + + if (!bdrv_dirty_bitmap_readonly(bitmap)) { + error_setg(errp, "Bitmap %s is not readonly but not marked" + "'IN_USE' in the image. Something went wr= ong," + "all the bitmaps may be corrupted", bm->n= ame); + ret =3D -EINVAL; + goto out; + } + + bm->flags |=3D BME_FLAG_IN_USE; + ro_dirty_bitmaps =3D g_slist_append(ro_dirty_bitmaps, bitmap); + } + } + + if (ro_dirty_bitmaps !=3D NULL) { + /* in_use flags must be updated */ + ret =3D update_ext_header_and_dir_in_place(bs, bm_list); + if (ret < 0) { + error_setg_errno(errp, -ret, "Can't update bitmap directory"); + goto out; + } + g_slist_foreach(ro_dirty_bitmaps, set_readonly_helper, false); + } + +out: + g_slist_free(ro_dirty_bitmaps); + bitmap_list_free(bm_list); + + return ret; +} diff --git a/block/qcow2.c b/block/qcow2.c index 92e8ff064d..8f070b12a2 100644 --- a/block/qcow2.c +++ b/block/qcow2.c @@ -3595,6 +3595,8 @@ BlockDriver bdrv_qcow2 =3D { =20 .bdrv_detach_aio_context =3D qcow2_detach_aio_context, .bdrv_attach_aio_context =3D qcow2_attach_aio_context, + + .bdrv_reopen_bitmaps_rw =3D qcow2_reopen_bitmaps_rw, }; =20 static void bdrv_qcow2_init(void) diff --git a/block/qcow2.h b/block/qcow2.h index 67c61de008..3e23bb7361 100644 --- a/block/qcow2.h +++ b/block/qcow2.h @@ -630,5 +630,6 @@ int qcow2_check_bitmaps_refcounts(BlockDriverState *bs,= BdrvCheckResult *res, void **refcount_table, int64_t *refcount_table_size); bool qcow2_load_autoloading_dirty_bitmaps(BlockDriverState *bs, Error **er= rp); +int qcow2_reopen_bitmaps_rw(BlockDriverState *bs, Error **errp); =20 #endif --=20 2.11.1 From nobody Sat May 4 15:42:58 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 149865226750255.172284859915294; Wed, 28 Jun 2017 05:17:47 -0700 (PDT) Received: from localhost ([::1]:32878 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dQBuh-0000XD-Sf for importer@patchew.org; Wed, 28 Jun 2017 08:17:43 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:60292) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dQBj5-0006QE-2X for qemu-devel@nongnu.org; Wed, 28 Jun 2017 08:05:50 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1dQBj2-00031d-J9 for qemu-devel@nongnu.org; Wed, 28 Jun 2017 08:05:42 -0400 Received: from mailhub.sw.ru ([195.214.232.25]:41998 helo=relay.sw.ru) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1dQBj2-0002w2-5C for qemu-devel@nongnu.org; Wed, 28 Jun 2017 08:05:40 -0400 Received: from kvm.sw.ru (msk-vpn.virtuozzo.com [195.214.232.6]) by relay.sw.ru (8.13.4/8.13.4) with ESMTP id v5SC5UUH025017; Wed, 28 Jun 2017 15:05:32 +0300 (MSK) From: Vladimir Sementsov-Ogievskiy To: qemu-block@nongnu.org, qemu-devel@nongnu.org Date: Wed, 28 Jun 2017 15:05:15 +0300 Message-Id: <20170628120530.31251-16-vsementsov@virtuozzo.com> X-Mailer: git-send-email 2.11.1 In-Reply-To: <20170628120530.31251-1-vsementsov@virtuozzo.com> References: <20170628120530.31251-1-vsementsov@virtuozzo.com> X-detected-operating-system: by eggs.gnu.org: OpenBSD 3.x [fuzzy] X-Received-From: 195.214.232.25 Subject: [Qemu-devel] [PATCH v22 15/30] block/dirty-bitmap: add autoload field to BdrvDirtyBitmap X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, vsementsov@virtuozzo.com, famz@redhat.com, armbru@redhat.com, mreitz@redhat.com, stefanha@redhat.com, pbonzini@redhat.com, den@openvz.org, jsnow@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Mirror AUTO flag from Qcow2 bitmap in BdrvDirtyBitmap. This will be needed in future, to save this flag back to Qcow2 for persistent bitmaps. Signed-off-by: Vladimir Sementsov-Ogievskiy --- block/dirty-bitmap.c | 18 ++++++++++++++++++ block/qcow2-bitmap.c | 2 ++ include/block/dirty-bitmap.h | 2 ++ 3 files changed, 22 insertions(+) diff --git a/block/dirty-bitmap.c b/block/dirty-bitmap.c index 17d3068336..06dc7a3ac9 100644 --- a/block/dirty-bitmap.c +++ b/block/dirty-bitmap.c @@ -52,6 +52,8 @@ struct BdrvDirtyBitmap { Such operations must fail and both the = image and this bitmap must remain unchanged w= hile this flag is set. */ + bool autoload; /* For persistent bitmaps: bitmap must be + autoloaded on image opening */ QLIST_ENTRY(BdrvDirtyBitmap) list; }; =20 @@ -100,6 +102,7 @@ void bdrv_dirty_bitmap_make_anon(BdrvDirtyBitmap *bitma= p) assert(!bdrv_dirty_bitmap_frozen(bitmap)); g_free(bitmap->name); bitmap->name =3D NULL; + bitmap->autoload =3D false; } =20 /* Called with BQL taken. */ @@ -296,6 +299,8 @@ BdrvDirtyBitmap *bdrv_dirty_bitmap_abdicate(BlockDriver= State *bs, bitmap->name =3D NULL; successor->name =3D name; bitmap->successor =3D NULL; + successor->autoload =3D bitmap->autoload; + bitmap->autoload =3D false; bdrv_release_dirty_bitmap(bs, bitmap); =20 return successor; @@ -671,3 +676,16 @@ bool bdrv_has_readonly_bitmaps(BlockDriverState *bs) =20 return false; } + +/* Called with BQL taken. */ +void bdrv_dirty_bitmap_set_autoload(BdrvDirtyBitmap *bitmap, bool autoload) +{ + qemu_mutex_lock(bitmap->mutex); + bitmap->autoload =3D autoload; + qemu_mutex_unlock(bitmap->mutex); +} + +bool bdrv_dirty_bitmap_get_autoload(const BdrvDirtyBitmap *bitmap) +{ + return bitmap->autoload; +} diff --git a/block/qcow2-bitmap.c b/block/qcow2-bitmap.c index a21fab8ce8..ee6d8f75a9 100644 --- a/block/qcow2-bitmap.c +++ b/block/qcow2-bitmap.c @@ -793,6 +793,8 @@ bool qcow2_load_autoloading_dirty_bitmaps(BlockDriverSt= ate *bs, Error **errp) if (bitmap =3D=3D NULL) { goto fail; } + + bdrv_dirty_bitmap_set_autoload(bitmap, true); bm->flags |=3D BME_FLAG_IN_USE; created_dirty_bitmaps =3D g_slist_append(created_dirty_bitmaps, bitmap); diff --git a/include/block/dirty-bitmap.h b/include/block/dirty-bitmap.h index cb43fa37e2..e2fea12b94 100644 --- a/include/block/dirty-bitmap.h +++ b/include/block/dirty-bitmap.h @@ -72,6 +72,7 @@ void bdrv_dirty_bitmap_deserialize_ones(BdrvDirtyBitmap *= bitmap, void bdrv_dirty_bitmap_deserialize_finish(BdrvDirtyBitmap *bitmap); =20 void bdrv_dirty_bitmap_set_readonly(BdrvDirtyBitmap *bitmap, bool value); +void bdrv_dirty_bitmap_set_autoload(BdrvDirtyBitmap *bitmap, bool autoload= ); =20 /* Functions that require manual locking. */ void bdrv_dirty_bitmap_lock(BdrvDirtyBitmap *bitmap); @@ -89,5 +90,6 @@ int64_t bdrv_get_meta_dirty_count(BdrvDirtyBitmap *bitmap= ); void bdrv_dirty_bitmap_truncate(BlockDriverState *bs); bool bdrv_dirty_bitmap_readonly(const BdrvDirtyBitmap *bitmap); bool bdrv_has_readonly_bitmaps(BlockDriverState *bs); +bool bdrv_dirty_bitmap_get_autoload(const BdrvDirtyBitmap *bitmap); =20 #endif --=20 2.11.1 From nobody Sat May 4 15:42:58 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1498651816013166.90104174954786; Wed, 28 Jun 2017 05:10:16 -0700 (PDT) Received: from localhost ([::1]:32843 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dQBnS-00020D-Eb for importer@patchew.org; Wed, 28 Jun 2017 08:10:14 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:60305) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dQBj5-0006QI-5h for qemu-devel@nongnu.org; Wed, 28 Jun 2017 08:05:48 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1dQBj2-00031u-Lz for qemu-devel@nongnu.org; Wed, 28 Jun 2017 08:05:42 -0400 Received: from mailhub.sw.ru ([195.214.232.25]:45424 helo=relay.sw.ru) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1dQBj2-0002vM-9W for qemu-devel@nongnu.org; Wed, 28 Jun 2017 08:05:40 -0400 Received: from kvm.sw.ru (msk-vpn.virtuozzo.com [195.214.232.6]) by relay.sw.ru (8.13.4/8.13.4) with ESMTP id v5SC5UUI025017; Wed, 28 Jun 2017 15:05:32 +0300 (MSK) From: Vladimir Sementsov-Ogievskiy To: qemu-block@nongnu.org, qemu-devel@nongnu.org Date: Wed, 28 Jun 2017 15:05:16 +0300 Message-Id: <20170628120530.31251-17-vsementsov@virtuozzo.com> X-Mailer: git-send-email 2.11.1 In-Reply-To: <20170628120530.31251-1-vsementsov@virtuozzo.com> References: <20170628120530.31251-1-vsementsov@virtuozzo.com> X-detected-operating-system: by eggs.gnu.org: OpenBSD 3.x [fuzzy] X-Received-From: 195.214.232.25 Subject: [Qemu-devel] [PATCH v22 16/30] block: bdrv_close: release bitmaps after drv->bdrv_close X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, vsementsov@virtuozzo.com, famz@redhat.com, armbru@redhat.com, mreitz@redhat.com, stefanha@redhat.com, pbonzini@redhat.com, den@openvz.org, jsnow@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Release bitmaps after 'if (bs->drv) { ... }' block. This will allow format driver to save persistent bitmaps, which will appear in following commits. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Max Reitz --- block.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/block.c b/block.c index 3f83da178d..c649afec91 100644 --- a/block.c +++ b/block.c @@ -3061,9 +3061,6 @@ static void bdrv_close(BlockDriverState *bs) bdrv_flush(bs); bdrv_drain(bs); /* in case flush left pending I/O */ =20 - bdrv_release_named_dirty_bitmaps(bs); - assert(QLIST_EMPTY(&bs->dirty_bitmaps)); - if (bs->drv) { BdrvChild *child, *next; =20 @@ -3102,6 +3099,9 @@ static void bdrv_close(BlockDriverState *bs) bs->full_open_options =3D NULL; } =20 + bdrv_release_named_dirty_bitmaps(bs); + assert(QLIST_EMPTY(&bs->dirty_bitmaps)); + QLIST_FOREACH_SAFE(ban, &bs->aio_notifiers, list, ban_next) { g_free(ban); } --=20 2.11.1 From nobody Sat May 4 15:42:58 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1498653153149454.3863190812923; Wed, 28 Jun 2017 05:32:33 -0700 (PDT) Received: from localhost ([::1]:32992 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dQC8z-0006RT-Sk for importer@patchew.org; Wed, 28 Jun 2017 08:32:29 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:60539) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dQBjB-0006SW-Rk for qemu-devel@nongnu.org; Wed, 28 Jun 2017 08:05:51 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1dQBj3-000336-6V for qemu-devel@nongnu.org; Wed, 28 Jun 2017 08:05:49 -0400 Received: from mailhub.sw.ru ([195.214.232.25]:42162 helo=relay.sw.ru) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1dQBj2-0002wQ-Px for qemu-devel@nongnu.org; Wed, 28 Jun 2017 08:05:41 -0400 Received: from kvm.sw.ru (msk-vpn.virtuozzo.com [195.214.232.6]) by relay.sw.ru (8.13.4/8.13.4) with ESMTP id v5SC5UUJ025017; Wed, 28 Jun 2017 15:05:33 +0300 (MSK) From: Vladimir Sementsov-Ogievskiy To: qemu-block@nongnu.org, qemu-devel@nongnu.org Date: Wed, 28 Jun 2017 15:05:17 +0300 Message-Id: <20170628120530.31251-18-vsementsov@virtuozzo.com> X-Mailer: git-send-email 2.11.1 In-Reply-To: <20170628120530.31251-1-vsementsov@virtuozzo.com> References: <20170628120530.31251-1-vsementsov@virtuozzo.com> X-detected-operating-system: by eggs.gnu.org: OpenBSD 3.x [fuzzy] X-Received-From: 195.214.232.25 Subject: [Qemu-devel] [PATCH v22 17/30] block: introduce persistent dirty bitmaps X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, vsementsov@virtuozzo.com, famz@redhat.com, armbru@redhat.com, mreitz@redhat.com, stefanha@redhat.com, pbonzini@redhat.com, den@openvz.org, jsnow@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" New field BdrvDirtyBitmap.persistent means, that bitmap should be saved by format driver in .bdrv_close and .bdrv_inactivate. No format driver supports it for now. Signed-off-by: Vladimir Sementsov-Ogievskiy --- block/dirty-bitmap.c | 29 +++++++++++++++++++++++++++++ block/qcow2-bitmap.c | 1 + include/block/dirty-bitmap.h | 4 ++++ 3 files changed, 34 insertions(+) diff --git a/block/dirty-bitmap.c b/block/dirty-bitmap.c index 06dc7a3ac9..3c17c452ae 100644 --- a/block/dirty-bitmap.c +++ b/block/dirty-bitmap.c @@ -54,6 +54,7 @@ struct BdrvDirtyBitmap { this flag is set. */ bool autoload; /* For persistent bitmaps: bitmap must be autoloaded on image opening */ + bool persistent; /* bitmap must be saved to owner disk imag= e */ QLIST_ENTRY(BdrvDirtyBitmap) list; }; =20 @@ -102,6 +103,7 @@ void bdrv_dirty_bitmap_make_anon(BdrvDirtyBitmap *bitma= p) assert(!bdrv_dirty_bitmap_frozen(bitmap)); g_free(bitmap->name); bitmap->name =3D NULL; + bitmap->persistent =3D false; bitmap->autoload =3D false; } =20 @@ -299,6 +301,8 @@ BdrvDirtyBitmap *bdrv_dirty_bitmap_abdicate(BlockDriver= State *bs, bitmap->name =3D NULL; successor->name =3D name; bitmap->successor =3D NULL; + successor->persistent =3D bitmap->persistent; + bitmap->persistent =3D false; successor->autoload =3D bitmap->autoload; bitmap->autoload =3D false; bdrv_release_dirty_bitmap(bs, bitmap); @@ -689,3 +693,28 @@ bool bdrv_dirty_bitmap_get_autoload(const BdrvDirtyBit= map *bitmap) { return bitmap->autoload; } + +/* Called with BQL taken. */ +void bdrv_dirty_bitmap_set_persistance(BdrvDirtyBitmap *bitmap, bool persi= stent) +{ + qemu_mutex_lock(bitmap->mutex); + bitmap->persistent =3D persistent; + qemu_mutex_unlock(bitmap->mutex); +} + +bool bdrv_dirty_bitmap_get_persistance(BdrvDirtyBitmap *bitmap) +{ + return bitmap->persistent; +} + +bool bdrv_has_changed_persistent_bitmaps(BlockDriverState *bs) +{ + BdrvDirtyBitmap *bm; + QLIST_FOREACH(bm, &bs->dirty_bitmaps, list) { + if (bm->persistent && !bm->readonly) { + return true; + } + } + + return false; +} diff --git a/block/qcow2-bitmap.c b/block/qcow2-bitmap.c index ee6d8f75a9..52e4616b8c 100644 --- a/block/qcow2-bitmap.c +++ b/block/qcow2-bitmap.c @@ -794,6 +794,7 @@ bool qcow2_load_autoloading_dirty_bitmaps(BlockDriverSt= ate *bs, Error **errp) goto fail; } =20 + bdrv_dirty_bitmap_set_persistance(bitmap, true); bdrv_dirty_bitmap_set_autoload(bitmap, true); bm->flags |=3D BME_FLAG_IN_USE; created_dirty_bitmaps =3D diff --git a/include/block/dirty-bitmap.h b/include/block/dirty-bitmap.h index e2fea12b94..3995789218 100644 --- a/include/block/dirty-bitmap.h +++ b/include/block/dirty-bitmap.h @@ -73,6 +73,8 @@ void bdrv_dirty_bitmap_deserialize_finish(BdrvDirtyBitmap= *bitmap); =20 void bdrv_dirty_bitmap_set_readonly(BdrvDirtyBitmap *bitmap, bool value); void bdrv_dirty_bitmap_set_autoload(BdrvDirtyBitmap *bitmap, bool autoload= ); +void bdrv_dirty_bitmap_set_persistance(BdrvDirtyBitmap *bitmap, + bool persistent); =20 /* Functions that require manual locking. */ void bdrv_dirty_bitmap_lock(BdrvDirtyBitmap *bitmap); @@ -91,5 +93,7 @@ void bdrv_dirty_bitmap_truncate(BlockDriverState *bs); bool bdrv_dirty_bitmap_readonly(const BdrvDirtyBitmap *bitmap); bool bdrv_has_readonly_bitmaps(BlockDriverState *bs); bool bdrv_dirty_bitmap_get_autoload(const BdrvDirtyBitmap *bitmap); +bool bdrv_dirty_bitmap_get_persistance(BdrvDirtyBitmap *bitmap); +bool bdrv_has_changed_persistent_bitmaps(BlockDriverState *bs); =20 #endif --=20 2.11.1 From nobody Sat May 4 15:42:58 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1498652423798314.7557794669691; Wed, 28 Jun 2017 05:20:23 -0700 (PDT) Received: from localhost ([::1]:32889 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dQBxC-0003FT-EO for importer@patchew.org; Wed, 28 Jun 2017 08:20:18 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:60396) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dQBj7-0006QV-36 for qemu-devel@nongnu.org; Wed, 28 Jun 2017 08:05:48 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1dQBj3-00033X-9y for qemu-devel@nongnu.org; Wed, 28 Jun 2017 08:05:45 -0400 Received: from mailhub.sw.ru ([195.214.232.25]:15456 helo=relay.sw.ru) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1dQBj2-0002vG-Ti for qemu-devel@nongnu.org; Wed, 28 Jun 2017 08:05:41 -0400 Received: from kvm.sw.ru (msk-vpn.virtuozzo.com [195.214.232.6]) by relay.sw.ru (8.13.4/8.13.4) with ESMTP id v5SC5UUK025017; Wed, 28 Jun 2017 15:05:33 +0300 (MSK) From: Vladimir Sementsov-Ogievskiy To: qemu-block@nongnu.org, qemu-devel@nongnu.org Date: Wed, 28 Jun 2017 15:05:18 +0300 Message-Id: <20170628120530.31251-19-vsementsov@virtuozzo.com> X-Mailer: git-send-email 2.11.1 In-Reply-To: <20170628120530.31251-1-vsementsov@virtuozzo.com> References: <20170628120530.31251-1-vsementsov@virtuozzo.com> X-detected-operating-system: by eggs.gnu.org: OpenBSD 3.x [fuzzy] X-Received-From: 195.214.232.25 Subject: [Qemu-devel] [PATCH v22 18/30] block/dirty-bitmap: add bdrv_dirty_bitmap_next() X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, vsementsov@virtuozzo.com, famz@redhat.com, armbru@redhat.com, mreitz@redhat.com, stefanha@redhat.com, pbonzini@redhat.com, den@openvz.org, jsnow@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Max Reitz Reviewed-by: John Snow --- block/dirty-bitmap.c | 7 +++++++ include/block/dirty-bitmap.h | 2 ++ 2 files changed, 9 insertions(+) diff --git a/block/dirty-bitmap.c b/block/dirty-bitmap.c index 3c17c452ae..d1469418e6 100644 --- a/block/dirty-bitmap.c +++ b/block/dirty-bitmap.c @@ -718,3 +718,10 @@ bool bdrv_has_changed_persistent_bitmaps(BlockDriverSt= ate *bs) =20 return false; } + +BdrvDirtyBitmap *bdrv_dirty_bitmap_next(BlockDriverState *bs, + BdrvDirtyBitmap *bitmap) +{ + return bitmap =3D=3D NULL ? QLIST_FIRST(&bs->dirty_bitmaps) : + QLIST_NEXT(bitmap, list); +} diff --git a/include/block/dirty-bitmap.h b/include/block/dirty-bitmap.h index 3995789218..ccf2f81640 100644 --- a/include/block/dirty-bitmap.h +++ b/include/block/dirty-bitmap.h @@ -95,5 +95,7 @@ bool bdrv_has_readonly_bitmaps(BlockDriverState *bs); bool bdrv_dirty_bitmap_get_autoload(const BdrvDirtyBitmap *bitmap); bool bdrv_dirty_bitmap_get_persistance(BdrvDirtyBitmap *bitmap); bool bdrv_has_changed_persistent_bitmaps(BlockDriverState *bs); +BdrvDirtyBitmap *bdrv_dirty_bitmap_next(BlockDriverState *bs, + BdrvDirtyBitmap *bitmap); =20 #endif --=20 2.11.1 From nobody Sat May 4 15:42:58 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 149865326659851.97133436675563; Wed, 28 Jun 2017 05:34:26 -0700 (PDT) Received: from localhost ([::1]:33004 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dQCAp-0007pC-Ml for importer@patchew.org; Wed, 28 Jun 2017 08:34:23 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:60494) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dQBj9-0006Qx-F0 for qemu-devel@nongnu.org; Wed, 28 Jun 2017 08:05:51 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1dQBj3-00033k-Bj for qemu-devel@nongnu.org; Wed, 28 Jun 2017 08:05:47 -0400 Received: from mailhub.sw.ru ([195.214.232.25]:6716 helo=relay.sw.ru) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1dQBj2-0002x1-JJ for qemu-devel@nongnu.org; Wed, 28 Jun 2017 08:05:41 -0400 Received: from kvm.sw.ru (msk-vpn.virtuozzo.com [195.214.232.6]) by relay.sw.ru (8.13.4/8.13.4) with ESMTP id v5SC5UUL025017; Wed, 28 Jun 2017 15:05:33 +0300 (MSK) From: Vladimir Sementsov-Ogievskiy To: qemu-block@nongnu.org, qemu-devel@nongnu.org Date: Wed, 28 Jun 2017 15:05:19 +0300 Message-Id: <20170628120530.31251-20-vsementsov@virtuozzo.com> X-Mailer: git-send-email 2.11.1 In-Reply-To: <20170628120530.31251-1-vsementsov@virtuozzo.com> References: <20170628120530.31251-1-vsementsov@virtuozzo.com> X-detected-operating-system: by eggs.gnu.org: OpenBSD 3.x [fuzzy] X-Received-From: 195.214.232.25 Subject: [Qemu-devel] [PATCH v22 19/30] qcow2: add persistent dirty bitmaps support X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, vsementsov@virtuozzo.com, famz@redhat.com, armbru@redhat.com, mreitz@redhat.com, stefanha@redhat.com, pbonzini@redhat.com, den@openvz.org, jsnow@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Store persistent dirty bitmaps in qcow2 image. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Max Reitz --- block/qcow2-bitmap.c | 475 +++++++++++++++++++++++++++++++++++++++++++++++= ++++ block/qcow2.c | 9 + block/qcow2.h | 1 + 3 files changed, 485 insertions(+) diff --git a/block/qcow2-bitmap.c b/block/qcow2-bitmap.c index 52e4616b8c..5f53486b22 100644 --- a/block/qcow2-bitmap.c +++ b/block/qcow2-bitmap.c @@ -27,6 +27,7 @@ =20 #include "qemu/osdep.h" #include "qapi/error.h" +#include "qemu/cutils.h" =20 #include "block/block_int.h" #include "block/qcow2.h" @@ -42,6 +43,10 @@ #define BME_MIN_GRANULARITY_BITS 9 #define BME_MAX_NAME_SIZE 1023 =20 +#if BME_MAX_TABLE_SIZE * 8ULL > INT_MAX +#error In the code bitmap table physical size assumed to fit into int +#endif + /* Bitmap directory entry flags */ #define BME_RESERVED_FLAGS 0xfffffffcU #define BME_FLAG_IN_USE (1U << 0) @@ -72,6 +77,8 @@ typedef struct Qcow2BitmapTable { uint32_t size; /* number of 64bit entries */ QSIMPLEQ_ENTRY(Qcow2BitmapTable) entry; } Qcow2BitmapTable; +typedef QSIMPLEQ_HEAD(Qcow2BitmapTableList, Qcow2BitmapTable) + Qcow2BitmapTableList; =20 typedef struct Qcow2Bitmap { Qcow2BitmapTable table; @@ -79,6 +86,8 @@ typedef struct Qcow2Bitmap { uint8_t granularity_bits; char *name; =20 + BdrvDirtyBitmap *dirty_bitmap; + QSIMPLEQ_ENTRY(Qcow2Bitmap) entry; } Qcow2Bitmap; typedef QSIMPLEQ_HEAD(Qcow2BitmapList, Qcow2Bitmap) Qcow2BitmapList; @@ -104,6 +113,15 @@ static int update_header_sync(BlockDriverState *bs) return bdrv_flush(bs); } =20 +static inline void bitmap_table_to_be(uint64_t *bitmap_table, size_t size) +{ + size_t i; + + for (i =3D 0; i < size; ++i) { + cpu_to_be64s(&bitmap_table[i]); + } +} + static int check_table_entry(uint64_t entry, int cluster_size) { uint64_t offset; @@ -127,6 +145,70 @@ static int check_table_entry(uint64_t entry, int clust= er_size) return 0; } =20 +static int check_constraints_on_bitmap(BlockDriverState *bs, + const char *name, + uint32_t granularity, + Error **errp) +{ + BDRVQcow2State *s =3D bs->opaque; + int granularity_bits =3D ctz32(granularity); + int64_t len =3D bdrv_getlength(bs); + + assert(granularity > 0); + assert((granularity & (granularity - 1)) =3D=3D 0); + + if (len < 0) { + error_setg_errno(errp, -len, "Failed to get size of '%s'", + bdrv_get_device_or_node_name(bs)); + return len; + } + + if (granularity_bits > BME_MAX_GRANULARITY_BITS) { + error_setg(errp, "Granularity exceeds maximum (%llu bytes)", + 1ULL << BME_MAX_GRANULARITY_BITS); + return -EINVAL; + } + if (granularity_bits < BME_MIN_GRANULARITY_BITS) { + error_setg(errp, "Granularity is under minimum (%llu bytes)", + 1ULL << BME_MIN_GRANULARITY_BITS); + return -EINVAL; + } + + if ((len > (uint64_t)BME_MAX_PHYS_SIZE << granularity_bits) || + (len > (uint64_t)BME_MAX_TABLE_SIZE * s->cluster_size << + granularity_bits)) + { + error_setg(errp, "Too much space will be occupied by the bitmap. " + "Use larger granularity"); + return -EINVAL; + } + + if (strlen(name) > BME_MAX_NAME_SIZE) { + error_setg(errp, "Name length exceeds maximum (%u characters)", + BME_MAX_NAME_SIZE); + return -EINVAL; + } + + return 0; +} + +static void clear_bitmap_table(BlockDriverState *bs, uint64_t *bitmap_tabl= e, + uint32_t bitmap_table_size) +{ + BDRVQcow2State *s =3D bs->opaque; + int i; + + for (i =3D 0; i < bitmap_table_size; ++i) { + uint64_t addr =3D bitmap_table[i] & BME_TABLE_ENTRY_OFFSET_MASK; + if (!addr) { + continue; + } + + qcow2_free_clusters(bs, addr, s->cluster_size, QCOW2_DISCARD_OTHER= ); + bitmap_table[i] =3D 0; + } +} + static int bitmap_table_load(BlockDriverState *bs, Qcow2BitmapTable *tb, uint64_t **bitmap_table) { @@ -165,6 +247,28 @@ fail: return ret; } =20 +static int free_bitmap_clusters(BlockDriverState *bs, Qcow2BitmapTable *tb) +{ + int ret; + uint64_t *bitmap_table; + + ret =3D bitmap_table_load(bs, tb, &bitmap_table); + if (ret < 0) { + assert(bitmap_table =3D=3D NULL); + return ret; + } + + clear_bitmap_table(bs, bitmap_table, tb->size); + qcow2_free_clusters(bs, tb->offset, tb->size * sizeof(uint64_t), + QCOW2_DISCARD_OTHER); + g_free(bitmap_table); + + tb->offset =3D 0; + tb->size =3D 0; + + return 0; +} + /* This function returns the number of disk sectors covered by a single qc= ow2 * cluster of bitmap data. */ static uint64_t sectors_covered_by_bitmap_cluster(const BDRVQcow2State *s, @@ -748,6 +852,69 @@ static int update_ext_header_and_dir_in_place(BlockDri= verState *bs, */ } =20 +static int update_ext_header_and_dir(BlockDriverState *bs, + Qcow2BitmapList *bm_list) +{ + BDRVQcow2State *s =3D bs->opaque; + int ret; + uint64_t new_offset =3D 0; + uint64_t new_size =3D 0; + uint32_t new_nb_bitmaps =3D 0; + uint64_t old_offset =3D s->bitmap_directory_offset; + uint64_t old_size =3D s->bitmap_directory_size; + uint32_t old_nb_bitmaps =3D s->nb_bitmaps; + uint64_t old_autocl =3D s->autoclear_features; + + if (bm_list !=3D NULL && !QSIMPLEQ_EMPTY(bm_list)) { + new_nb_bitmaps =3D bitmap_list_count(bm_list); + + if (new_nb_bitmaps > QCOW2_MAX_BITMAPS) { + return -EINVAL; + } + + ret =3D bitmap_list_store(bs, bm_list, &new_offset, &new_size, fal= se); + if (ret < 0) { + return ret; + } + + ret =3D bdrv_flush(bs->file->bs); + if (ret < 0) { + goto fail; + } + + s->autoclear_features |=3D QCOW2_AUTOCLEAR_BITMAPS; + } else { + s->autoclear_features &=3D ~(uint64_t)QCOW2_AUTOCLEAR_BITMAPS; + } + + s->bitmap_directory_offset =3D new_offset; + s->bitmap_directory_size =3D new_size; + s->nb_bitmaps =3D new_nb_bitmaps; + + ret =3D update_header_sync(bs); + if (ret < 0) { + goto fail; + } + + if (old_size > 0) { + qcow2_free_clusters(bs, old_offset, old_size, QCOW2_DISCARD_OTHER); + } + + return 0; + +fail: + if (new_offset > 0) { + qcow2_free_clusters(bs, new_offset, new_size, QCOW2_DISCARD_OTHER); + } + + s->bitmap_directory_offset =3D old_offset; + s->bitmap_directory_size =3D old_size; + s->nb_bitmaps =3D old_nb_bitmaps; + s->autoclear_features =3D old_autocl; + + return ret; +} + /* for g_slist_foreach for GSList of BdrvDirtyBitmap* elements */ static void release_dirty_bitmap_helper(gpointer bitmap, gpointer bs) @@ -890,3 +1057,311 @@ out: =20 return ret; } + +/* store_bitmap_data() + * Store bitmap to image, filling bitmap table accordingly. + */ +static uint64_t *store_bitmap_data(BlockDriverState *bs, + BdrvDirtyBitmap *bitmap, + uint32_t *bitmap_table_size, Error **er= rp) +{ + int ret; + BDRVQcow2State *s =3D bs->opaque; + int64_t sector; + uint64_t sbc; + uint64_t bm_size =3D bdrv_dirty_bitmap_size(bitmap); + const char *bm_name =3D bdrv_dirty_bitmap_name(bitmap); + uint8_t *buf =3D NULL; + BdrvDirtyBitmapIter *dbi; + uint64_t *tb; + uint64_t tb_size =3D + size_to_clusters(s, + bdrv_dirty_bitmap_serialization_size(bitmap, 0, bm_size)); + + if (tb_size > BME_MAX_TABLE_SIZE || + tb_size * s->cluster_size > BME_MAX_PHYS_SIZE) + { + error_setg(errp, "Bitmap '%s' is too big", bm_name); + return NULL; + } + + tb =3D g_try_new0(uint64_t, tb_size); + if (tb =3D=3D NULL) { + error_setg(errp, "No memory"); + return NULL; + } + + dbi =3D bdrv_dirty_iter_new(bitmap, 0); + buf =3D g_malloc(s->cluster_size); + sbc =3D sectors_covered_by_bitmap_cluster(s, bitmap); + assert(DIV_ROUND_UP(bm_size, sbc) =3D=3D tb_size); + + while ((sector =3D bdrv_dirty_iter_next(dbi)) !=3D -1) { + uint64_t cluster =3D sector / sbc; + uint64_t end, write_size; + int64_t off; + + sector =3D cluster * sbc; + end =3D MIN(bm_size, sector + sbc); + write_size =3D + bdrv_dirty_bitmap_serialization_size(bitmap, sector, end - sec= tor); + assert(write_size <=3D s->cluster_size); + + off =3D qcow2_alloc_clusters(bs, s->cluster_size); + if (off < 0) { + error_setg_errno(errp, -off, + "Failed to allocate clusters for bitmap '%s'", + bm_name); + goto fail; + } + tb[cluster] =3D off; + + bdrv_dirty_bitmap_serialize_part(bitmap, buf, sector, end - sector= ); + if (write_size < s->cluster_size) { + memset(buf + write_size, 0, s->cluster_size - write_size); + } + + ret =3D qcow2_pre_write_overlap_check(bs, 0, off, s->cluster_size); + if (ret < 0) { + error_setg_errno(errp, -ret, "Qcow2 overlap check failed"); + goto fail; + } + + ret =3D bdrv_pwrite(bs->file, off, buf, s->cluster_size); + if (ret < 0) { + error_setg_errno(errp, -ret, "Failed to write bitmap '%s' to f= ile", + bm_name); + goto fail; + } + + if (end >=3D bm_size) { + break; + } + + bdrv_set_dirty_iter(dbi, end); + } + + *bitmap_table_size =3D tb_size; + g_free(buf); + bdrv_dirty_iter_free(dbi); + + return tb; + +fail: + clear_bitmap_table(bs, tb, tb_size); + g_free(buf); + bdrv_dirty_iter_free(dbi); + g_free(tb); + + return NULL; +} + +/* store_bitmap() + * Store bm->dirty_bitmap to qcow2. + * Set bm->table_offset and bm->table_size accordingly. + */ +static int store_bitmap(BlockDriverState *bs, Qcow2Bitmap *bm, Error **err= p) +{ + int ret; + uint64_t *tb; + int64_t tb_offset; + uint32_t tb_size; + BdrvDirtyBitmap *bitmap =3D bm->dirty_bitmap; + const char *bm_name; + + assert(bitmap !=3D NULL); + + bm_name =3D bdrv_dirty_bitmap_name(bitmap); + + tb =3D store_bitmap_data(bs, bitmap, &tb_size, errp); + if (tb =3D=3D NULL) { + return -EINVAL; + } + + assert(tb_size <=3D BME_MAX_TABLE_SIZE); + tb_offset =3D qcow2_alloc_clusters(bs, tb_size * sizeof(tb[0])); + if (tb_offset < 0) { + error_setg_errno(errp, -tb_offset, + "Failed to allocate clusters for bitmap '%s'", + bm_name); + goto fail; + } + + ret =3D qcow2_pre_write_overlap_check(bs, 0, tb_offset, + tb_size * sizeof(tb[0])); + if (ret < 0) { + error_setg_errno(errp, -ret, "Qcow2 overlap check failed"); + goto fail; + } + + bitmap_table_to_be(tb, tb_size); + ret =3D bdrv_pwrite(bs->file, tb_offset, tb, tb_size * sizeof(tb[0])); + if (ret < 0) { + error_setg_errno(errp, -ret, "Failed to write bitmap '%s' to file", + bm_name); + goto fail; + } + + g_free(tb); + + bm->table.offset =3D tb_offset; + bm->table.size =3D tb_size; + + return 0; + +fail: + clear_bitmap_table(bs, tb, tb_size); + + if (tb_offset > 0) { + qcow2_free_clusters(bs, tb_offset, tb_size * sizeof(tb[0]), + QCOW2_DISCARD_OTHER); + } + + g_free(tb); + + return ret; +} + +static Qcow2Bitmap *find_bitmap_by_name(Qcow2BitmapList *bm_list, + const char *name) +{ + Qcow2Bitmap *bm; + + QSIMPLEQ_FOREACH(bm, bm_list, entry) { + if (strcmp(name, bm->name) =3D=3D 0) { + return bm; + } + } + + return NULL; +} + +void qcow2_store_persistent_dirty_bitmaps(BlockDriverState *bs, Error **er= rp) +{ + BdrvDirtyBitmap *bitmap; + BDRVQcow2State *s =3D bs->opaque; + uint32_t new_nb_bitmaps =3D s->nb_bitmaps; + uint64_t new_dir_size =3D s->bitmap_directory_size; + int ret; + Qcow2BitmapList *bm_list; + Qcow2Bitmap *bm; + Qcow2BitmapTableList drop_tables; + Qcow2BitmapTable *tb, *tb_next; + + if (!bdrv_has_changed_persistent_bitmaps(bs)) { + /* nothing to do */ + return; + } + + if (!can_write(bs)) { + error_setg(errp, "No write access"); + return; + } + + QSIMPLEQ_INIT(&drop_tables); + + if (s->nb_bitmaps =3D=3D 0) { + bm_list =3D bitmap_list_new(); + } else { + bm_list =3D bitmap_list_load(bs, s->bitmap_directory_offset, + s->bitmap_directory_size, errp); + if (bm_list =3D=3D NULL) { + return; + } + } + + /* check constraints and names */ + for (bitmap =3D bdrv_dirty_bitmap_next(bs, NULL); bitmap !=3D NULL; + bitmap =3D bdrv_dirty_bitmap_next(bs, bitmap)) + { + const char *name =3D bdrv_dirty_bitmap_name(bitmap); + uint32_t granularity =3D bdrv_dirty_bitmap_granularity(bitmap); + Qcow2Bitmap *bm; + + if (!bdrv_dirty_bitmap_get_persistance(bitmap) || + bdrv_dirty_bitmap_readonly(bitmap)) + { + continue; + } + + if (check_constraints_on_bitmap(bs, name, granularity, errp) < 0) { + error_prepend(errp, "Bitmap '%s' doesn't satisfy the constrain= ts: ", + name); + goto fail; + } + + bm =3D find_bitmap_by_name(bm_list, name); + if (bm =3D=3D NULL) { + if (++new_nb_bitmaps > QCOW2_MAX_BITMAPS) { + error_setg(errp, "Too many persistent bitmaps"); + goto fail; + } + + new_dir_size +=3D calc_dir_entry_size(strlen(name), 0); + if (new_dir_size > QCOW2_MAX_BITMAP_DIRECTORY_SIZE) { + error_setg(errp, "Bitmap directory is too large"); + goto fail; + } + + bm =3D g_new0(Qcow2Bitmap, 1); + bm->name =3D g_strdup(name); + QSIMPLEQ_INSERT_TAIL(bm_list, bm, entry); + } else { + if (!(bm->flags & BME_FLAG_IN_USE)) { + error_setg(errp, "Bitmap '%s' already exists in the image", + name); + goto fail; + } + tb =3D g_memdup(&bm->table, sizeof(bm->table)); + bm->table.offset =3D 0; + bm->table.size =3D 0; + QSIMPLEQ_INSERT_TAIL(&drop_tables, tb, entry); + } + bm->flags =3D bdrv_dirty_bitmap_get_autoload(bitmap) ? BME_FLAG_AU= TO : 0; + bm->granularity_bits =3D ctz32(bdrv_dirty_bitmap_granularity(bitma= p)); + bm->dirty_bitmap =3D bitmap; + } + + /* allocate clusters and store bitmaps */ + QSIMPLEQ_FOREACH(bm, bm_list, entry) { + if (bm->dirty_bitmap =3D=3D NULL) { + continue; + } + + ret =3D store_bitmap(bs, bm, errp); + if (ret < 0) { + goto fail; + } + } + + ret =3D update_ext_header_and_dir(bs, bm_list); + if (ret < 0) { + error_setg_errno(errp, -ret, "Failed to update bitmap extension"); + goto fail; + } + + /* Bitmap directory was successfully updated, so, old data can be drop= ped. + * TODO it is better to reuse these clusters */ + QSIMPLEQ_FOREACH_SAFE(tb, &drop_tables, entry, tb_next) { + free_bitmap_clusters(bs, tb); + g_free(tb); + } + + bitmap_list_free(bm_list); + return; + +fail: + QSIMPLEQ_FOREACH(bm, bm_list, entry) { + if (bm->dirty_bitmap =3D=3D NULL || bm->table.offset =3D=3D 0) { + continue; + } + + free_bitmap_clusters(bs, &bm->table); + } + + QSIMPLEQ_FOREACH_SAFE(tb, &drop_tables, entry, tb_next) { + g_free(tb); + } + + bitmap_list_free(bm_list); +} diff --git a/block/qcow2.c b/block/qcow2.c index 8f070b12a2..365298d4ce 100644 --- a/block/qcow2.c +++ b/block/qcow2.c @@ -1824,6 +1824,7 @@ static int qcow2_inactivate(BlockDriverState *bs) { BDRVQcow2State *s =3D bs->opaque; int ret, result =3D 0; + Error *local_err =3D NULL; =20 ret =3D qcow2_cache_flush(bs, s->l2_table_cache); if (ret) { @@ -1839,6 +1840,14 @@ static int qcow2_inactivate(BlockDriverState *bs) strerror(-ret)); } =20 + qcow2_store_persistent_dirty_bitmaps(bs, &local_err); + if (local_err !=3D NULL) { + result =3D -EINVAL; + error_report_err(local_err); + error_report("Persistent bitmaps are lost for node '%s'", + bdrv_get_device_or_node_name(bs)); + } + if (result =3D=3D 0) { qcow2_mark_clean(bs); } diff --git a/block/qcow2.h b/block/qcow2.h index 3e23bb7361..0594551237 100644 --- a/block/qcow2.h +++ b/block/qcow2.h @@ -631,5 +631,6 @@ int qcow2_check_bitmaps_refcounts(BlockDriverState *bs,= BdrvCheckResult *res, int64_t *refcount_table_size); bool qcow2_load_autoloading_dirty_bitmaps(BlockDriverState *bs, Error **er= rp); int qcow2_reopen_bitmaps_rw(BlockDriverState *bs, Error **errp); +void qcow2_store_persistent_dirty_bitmaps(BlockDriverState *bs, Error **er= rp); =20 #endif --=20 2.11.1 From nobody Sat May 4 15:42:58 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1498651664751194.41876150367227; Wed, 28 Jun 2017 05:07:44 -0700 (PDT) Received: from localhost ([::1]:32833 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dQBkx-0007lr-7I for importer@patchew.org; Wed, 28 Jun 2017 08:07:39 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:60393) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dQBj7-0006QT-14 for qemu-devel@nongnu.org; Wed, 28 Jun 2017 08:05:48 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1dQBj3-00033A-6u for qemu-devel@nongnu.org; Wed, 28 Jun 2017 08:05:44 -0400 Received: from mailhub.sw.ru ([195.214.232.25]:47796 helo=relay.sw.ru) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1dQBj2-0002vR-OY for qemu-devel@nongnu.org; Wed, 28 Jun 2017 08:05:41 -0400 Received: from kvm.sw.ru (msk-vpn.virtuozzo.com [195.214.232.6]) by relay.sw.ru (8.13.4/8.13.4) with ESMTP id v5SC5UUM025017; Wed, 28 Jun 2017 15:05:33 +0300 (MSK) From: Vladimir Sementsov-Ogievskiy To: qemu-block@nongnu.org, qemu-devel@nongnu.org Date: Wed, 28 Jun 2017 15:05:20 +0300 Message-Id: <20170628120530.31251-21-vsementsov@virtuozzo.com> X-Mailer: git-send-email 2.11.1 In-Reply-To: <20170628120530.31251-1-vsementsov@virtuozzo.com> References: <20170628120530.31251-1-vsementsov@virtuozzo.com> X-detected-operating-system: by eggs.gnu.org: OpenBSD 3.x [fuzzy] X-Received-From: 195.214.232.25 Subject: [Qemu-devel] [PATCH v22 20/30] qcow2: store bitmaps on reopening image as read-only X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, vsementsov@virtuozzo.com, famz@redhat.com, armbru@redhat.com, mreitz@redhat.com, stefanha@redhat.com, pbonzini@redhat.com, den@openvz.org, jsnow@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Store bitmaps and mark them read-only on reopening image as read-only. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Max Reitz --- block/qcow2-bitmap.c | 22 ++++++++++++++++++++++ block/qcow2.c | 5 +++++ block/qcow2.h | 1 + 3 files changed, 28 insertions(+) diff --git a/block/qcow2-bitmap.c b/block/qcow2-bitmap.c index 5f53486b22..7912a82c8c 100644 --- a/block/qcow2-bitmap.c +++ b/block/qcow2-bitmap.c @@ -1365,3 +1365,25 @@ fail: =20 bitmap_list_free(bm_list); } + +int qcow2_reopen_bitmaps_ro(BlockDriverState *bs, Error **errp) +{ + BdrvDirtyBitmap *bitmap; + Error *local_err =3D NULL; + + qcow2_store_persistent_dirty_bitmaps(bs, &local_err); + if (local_err !=3D NULL) { + error_propagate(errp, local_err); + return -EINVAL; + } + + for (bitmap =3D bdrv_dirty_bitmap_next(bs, NULL); bitmap !=3D NULL; + bitmap =3D bdrv_dirty_bitmap_next(bs, bitmap)) + { + if (bdrv_dirty_bitmap_get_persistance(bitmap)) { + bdrv_dirty_bitmap_set_readonly(bitmap, true); + } + } + + return 0; +} diff --git a/block/qcow2.c b/block/qcow2.c index 365298d4ce..b68e04766f 100644 --- a/block/qcow2.c +++ b/block/qcow2.c @@ -1382,6 +1382,11 @@ static int qcow2_reopen_prepare(BDRVReopenState *sta= te, =20 /* We need to write out any unwritten data if we reopen read-only. */ if ((state->flags & BDRV_O_RDWR) =3D=3D 0) { + ret =3D qcow2_reopen_bitmaps_ro(state->bs, errp); + if (ret < 0) { + goto fail; + } + ret =3D bdrv_flush(state->bs); if (ret < 0) { goto fail; diff --git a/block/qcow2.h b/block/qcow2.h index 0594551237..7d0a20c053 100644 --- a/block/qcow2.h +++ b/block/qcow2.h @@ -632,5 +632,6 @@ int qcow2_check_bitmaps_refcounts(BlockDriverState *bs,= BdrvCheckResult *res, bool qcow2_load_autoloading_dirty_bitmaps(BlockDriverState *bs, Error **er= rp); int qcow2_reopen_bitmaps_rw(BlockDriverState *bs, Error **errp); void qcow2_store_persistent_dirty_bitmaps(BlockDriverState *bs, Error **er= rp); +int qcow2_reopen_bitmaps_ro(BlockDriverState *bs, Error **errp); =20 #endif --=20 2.11.1 From nobody Sat May 4 15:42:58 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1498652267881783.4965151123812; Wed, 28 Jun 2017 05:17:47 -0700 (PDT) Received: from localhost ([::1]:32880 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dQBuj-0000Yo-GD for importer@patchew.org; Wed, 28 Jun 2017 08:17:45 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:60431) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dQBj7-0006Qm-VD for qemu-devel@nongnu.org; Wed, 28 Jun 2017 08:05:48 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1dQBj3-00033m-C9 for qemu-devel@nongnu.org; Wed, 28 Jun 2017 08:05:45 -0400 Received: from mailhub.sw.ru ([195.214.232.25]:37241 helo=relay.sw.ru) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1dQBj2-0002xh-Vz for qemu-devel@nongnu.org; Wed, 28 Jun 2017 08:05:41 -0400 Received: from kvm.sw.ru (msk-vpn.virtuozzo.com [195.214.232.6]) by relay.sw.ru (8.13.4/8.13.4) with ESMTP id v5SC5UUN025017; Wed, 28 Jun 2017 15:05:33 +0300 (MSK) From: Vladimir Sementsov-Ogievskiy To: qemu-block@nongnu.org, qemu-devel@nongnu.org Date: Wed, 28 Jun 2017 15:05:21 +0300 Message-Id: <20170628120530.31251-22-vsementsov@virtuozzo.com> X-Mailer: git-send-email 2.11.1 In-Reply-To: <20170628120530.31251-1-vsementsov@virtuozzo.com> References: <20170628120530.31251-1-vsementsov@virtuozzo.com> X-detected-operating-system: by eggs.gnu.org: OpenBSD 3.x [fuzzy] X-Received-From: 195.214.232.25 Subject: [Qemu-devel] [PATCH v22 21/30] block: add bdrv_can_store_new_dirty_bitmap X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, vsementsov@virtuozzo.com, famz@redhat.com, armbru@redhat.com, mreitz@redhat.com, stefanha@redhat.com, pbonzini@redhat.com, den@openvz.org, jsnow@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" This will be needed to check some restrictions before making bitmap persistent in qmp-block-dirty-bitmap-add (this functionality will be added by future patch) Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Max Reitz Reviewed-by: John Snow --- block.c | 22 ++++++++++++++++++++++ include/block/block.h | 3 +++ include/block/block_int.h | 4 ++++ 3 files changed, 29 insertions(+) diff --git a/block.c b/block.c index c649afec91..b2719bceff 100644 --- a/block.c +++ b/block.c @@ -4954,3 +4954,25 @@ void bdrv_del_child(BlockDriverState *parent_bs, Bdr= vChild *child, Error **errp) =20 parent_bs->drv->bdrv_del_child(parent_bs, child, errp); } + +bool bdrv_can_store_new_dirty_bitmap(BlockDriverState *bs, const char *nam= e, + uint32_t granularity, Error **errp) +{ + BlockDriver *drv =3D bs->drv; + + if (!drv) { + error_setg_errno(errp, ENOMEDIUM, + "Can't store persistent bitmaps to %s", + bdrv_get_device_or_node_name(bs)); + return false; + } + + if (!drv->bdrv_can_store_new_dirty_bitmap) { + error_setg_errno(errp, ENOTSUP, + "Can't store persistent bitmaps to %s", + bdrv_get_device_or_node_name(bs)); + return false; + } + + return drv->bdrv_can_store_new_dirty_bitmap(bs, name, granularity, err= p); +} diff --git a/include/block/block.h b/include/block/block.h index a4f09df95a..1daf9a0882 100644 --- a/include/block/block.h +++ b/include/block/block.h @@ -630,4 +630,7 @@ void bdrv_add_child(BlockDriverState *parent, BlockDriv= erState *child, Error **errp); void bdrv_del_child(BlockDriverState *parent, BdrvChild *child, Error **er= rp); =20 +bool bdrv_can_store_new_dirty_bitmap(BlockDriverState *bs, const char *nam= e, + uint32_t granularity, Error **errp); + #endif diff --git a/include/block/block_int.h b/include/block/block_int.h index 4ad8eec2dd..009b4d41df 100644 --- a/include/block/block_int.h +++ b/include/block/block_int.h @@ -387,6 +387,10 @@ struct BlockDriver { * field of BlockDirtyBitmap's in case of success. */ int (*bdrv_reopen_bitmaps_rw)(BlockDriverState *bs, Error **errp); + bool (*bdrv_can_store_new_dirty_bitmap)(BlockDriverState *bs, + const char *name, + uint32_t granularity, + Error **errp); =20 QLIST_ENTRY(BlockDriver) list; }; --=20 2.11.1 From nobody Sat May 4 15:42:58 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1498651663028681.4047363787912; Wed, 28 Jun 2017 05:07:43 -0700 (PDT) Received: from localhost ([::1]:32834 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dQBky-0007mC-Il for importer@patchew.org; Wed, 28 Jun 2017 08:07:40 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:60290) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dQBj5-0006QD-1T for qemu-devel@nongnu.org; Wed, 28 Jun 2017 08:05:45 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1dQBj2-00031w-MG for qemu-devel@nongnu.org; Wed, 28 Jun 2017 08:05:42 -0400 Received: from mailhub.sw.ru ([195.214.232.25]:2603 helo=relay.sw.ru) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1dQBj2-0002v4-9b for qemu-devel@nongnu.org; Wed, 28 Jun 2017 08:05:40 -0400 Received: from kvm.sw.ru (msk-vpn.virtuozzo.com [195.214.232.6]) by relay.sw.ru (8.13.4/8.13.4) with ESMTP id v5SC5UUO025017; Wed, 28 Jun 2017 15:05:33 +0300 (MSK) From: Vladimir Sementsov-Ogievskiy To: qemu-block@nongnu.org, qemu-devel@nongnu.org Date: Wed, 28 Jun 2017 15:05:22 +0300 Message-Id: <20170628120530.31251-23-vsementsov@virtuozzo.com> X-Mailer: git-send-email 2.11.1 In-Reply-To: <20170628120530.31251-1-vsementsov@virtuozzo.com> References: <20170628120530.31251-1-vsementsov@virtuozzo.com> X-detected-operating-system: by eggs.gnu.org: OpenBSD 3.x [fuzzy] X-Received-From: 195.214.232.25 Subject: [Qemu-devel] [PATCH v22 22/30] qcow2: add .bdrv_can_store_new_dirty_bitmap X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, vsementsov@virtuozzo.com, famz@redhat.com, armbru@redhat.com, mreitz@redhat.com, stefanha@redhat.com, pbonzini@redhat.com, den@openvz.org, jsnow@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Realize .bdrv_can_store_new_dirty_bitmap interface. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: John Snow Reviewed-by: Max Reitz --- block/qcow2-bitmap.c | 51 ++++++++++++++++++++++++++++++++++++++++++++++++= +++ block/qcow2.c | 1 + block/qcow2.h | 4 ++++ 3 files changed, 56 insertions(+) diff --git a/block/qcow2-bitmap.c b/block/qcow2-bitmap.c index 7912a82c8c..f45324e584 100644 --- a/block/qcow2-bitmap.c +++ b/block/qcow2-bitmap.c @@ -1387,3 +1387,54 @@ int qcow2_reopen_bitmaps_ro(BlockDriverState *bs, Er= ror **errp) =20 return 0; } + +bool qcow2_can_store_new_dirty_bitmap(BlockDriverState *bs, + const char *name, + uint32_t granularity, + Error **errp) +{ + BDRVQcow2State *s =3D bs->opaque; + bool found; + Qcow2BitmapList *bm_list; + + if (check_constraints_on_bitmap(bs, name, granularity, errp) !=3D 0) { + goto fail; + } + + if (s->nb_bitmaps =3D=3D 0) { + return true; + } + + if (s->nb_bitmaps >=3D QCOW2_MAX_BITMAPS) { + error_setg(errp, + "Maximum number of persistent bitmaps is already reache= d"); + goto fail; + } + + if (s->bitmap_directory_size + calc_dir_entry_size(strlen(name), 0) > + QCOW2_MAX_BITMAP_DIRECTORY_SIZE) + { + error_setg(errp, "Not enough space in the bitmap directory"); + goto fail; + } + + bm_list =3D bitmap_list_load(bs, s->bitmap_directory_offset, + s->bitmap_directory_size, errp); + if (bm_list =3D=3D NULL) { + goto fail; + } + + found =3D find_bitmap_by_name(bm_list, name); + bitmap_list_free(bm_list); + if (found) { + error_setg(errp, "Bitmap with the same name is already stored"); + goto fail; + } + + return true; + +fail: + error_prepend(errp, "Can't make bitmap '%s' persistent in '%s': ", + name, bdrv_get_device_or_node_name(bs)); + return false; +} diff --git a/block/qcow2.c b/block/qcow2.c index b68e04766f..fc1f69cead 100644 --- a/block/qcow2.c +++ b/block/qcow2.c @@ -3611,6 +3611,7 @@ BlockDriver bdrv_qcow2 =3D { .bdrv_attach_aio_context =3D qcow2_attach_aio_context, =20 .bdrv_reopen_bitmaps_rw =3D qcow2_reopen_bitmaps_rw, + .bdrv_can_store_new_dirty_bitmap =3D qcow2_can_store_new_dirty_bitmap, }; =20 static void bdrv_qcow2_init(void) diff --git a/block/qcow2.h b/block/qcow2.h index 7d0a20c053..8b2f66f8b6 100644 --- a/block/qcow2.h +++ b/block/qcow2.h @@ -633,5 +633,9 @@ bool qcow2_load_autoloading_dirty_bitmaps(BlockDriverSt= ate *bs, Error **errp); int qcow2_reopen_bitmaps_rw(BlockDriverState *bs, Error **errp); void qcow2_store_persistent_dirty_bitmaps(BlockDriverState *bs, Error **er= rp); int qcow2_reopen_bitmaps_ro(BlockDriverState *bs, Error **errp); +bool qcow2_can_store_new_dirty_bitmap(BlockDriverState *bs, + const char *name, + uint32_t granularity, + Error **errp); =20 #endif --=20 2.11.1 From nobody Sat May 4 15:42:58 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1498651666020299.17222822523775; Wed, 28 Jun 2017 05:07:46 -0700 (PDT) Received: from localhost ([::1]:32836 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dQBkz-0007nr-LO for importer@patchew.org; Wed, 28 Jun 2017 08:07:41 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:60377) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dQBj6-0006QO-PS for qemu-devel@nongnu.org; Wed, 28 Jun 2017 08:05:48 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1dQBj3-00033L-Al for qemu-devel@nongnu.org; Wed, 28 Jun 2017 08:05:44 -0400 Received: from mailhub.sw.ru ([195.214.232.25]:18504 helo=relay.sw.ru) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1dQBj2-0002y4-Rq for qemu-devel@nongnu.org; Wed, 28 Jun 2017 08:05:41 -0400 Received: from kvm.sw.ru (msk-vpn.virtuozzo.com [195.214.232.6]) by relay.sw.ru (8.13.4/8.13.4) with ESMTP id v5SC5UUP025017; Wed, 28 Jun 2017 15:05:33 +0300 (MSK) From: Vladimir Sementsov-Ogievskiy To: qemu-block@nongnu.org, qemu-devel@nongnu.org Date: Wed, 28 Jun 2017 15:05:23 +0300 Message-Id: <20170628120530.31251-24-vsementsov@virtuozzo.com> X-Mailer: git-send-email 2.11.1 In-Reply-To: <20170628120530.31251-1-vsementsov@virtuozzo.com> References: <20170628120530.31251-1-vsementsov@virtuozzo.com> X-detected-operating-system: by eggs.gnu.org: OpenBSD 3.x [fuzzy] X-Received-From: 195.214.232.25 Subject: [Qemu-devel] [PATCH v22 23/30] qmp: add persistent flag to block-dirty-bitmap-add X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, vsementsov@virtuozzo.com, famz@redhat.com, armbru@redhat.com, mreitz@redhat.com, stefanha@redhat.com, pbonzini@redhat.com, den@openvz.org, jsnow@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Add optional 'persistent' flag to qmp command block-dirty-bitmap-add. Default is false. Signed-off-by: Vladimir Sementsov-Ogievskiy Signed-off-by: Denis V. Lunev Reviewed-by: Max Reitz Reviewed-by: John Snow --- blockdev.c | 18 +++++++++++++++++- qapi/block-core.json | 8 +++++++- 2 files changed, 24 insertions(+), 2 deletions(-) diff --git a/blockdev.c b/blockdev.c index 64e03c0caf..125acabc07 100644 --- a/blockdev.c +++ b/blockdev.c @@ -1973,6 +1973,7 @@ static void block_dirty_bitmap_add_prepare(BlkActionS= tate *common, /* AIO context taken and released within qmp_block_dirty_bitmap_add */ qmp_block_dirty_bitmap_add(action->node, action->name, action->has_granularity, action->granularit= y, + action->has_persistent, action->persistent, &local_err); =20 if (!local_err) { @@ -2720,9 +2721,11 @@ out: =20 void qmp_block_dirty_bitmap_add(const char *node, const char *name, bool has_granularity, uint32_t granularity, + bool has_persistent, bool persistent, Error **errp) { BlockDriverState *bs; + BdrvDirtyBitmap *bitmap; =20 if (!name || name[0] =3D=3D '\0') { error_setg(errp, "Bitmap name cannot be empty"); @@ -2745,7 +2748,20 @@ void qmp_block_dirty_bitmap_add(const char *node, co= nst char *name, granularity =3D bdrv_get_default_bitmap_granularity(bs); } =20 - bdrv_create_dirty_bitmap(bs, granularity, name, errp); + if (!has_persistent) { + persistent =3D false; + } + + if (persistent && + !bdrv_can_store_new_dirty_bitmap(bs, name, granularity, errp)) + { + return; + } + + bitmap =3D bdrv_create_dirty_bitmap(bs, granularity, name, errp); + if (bitmap !=3D NULL) { + bdrv_dirty_bitmap_set_persistance(bitmap, persistent); + } } =20 void qmp_block_dirty_bitmap_remove(const char *node, const char *name, diff --git a/qapi/block-core.json b/qapi/block-core.json index f85c2235c7..13f98ec146 100644 --- a/qapi/block-core.json +++ b/qapi/block-core.json @@ -1561,10 +1561,16 @@ # @granularity: the bitmap granularity, default is 64k for # block-dirty-bitmap-add # +# @persistent: the bitmap is persistent, i.e. it will be saved to the +# corresponding block device image file on its close. For now= only +# Qcow2 disks support persistent bitmaps. Default is false for +# block-dirty-bitmap-add. (Since: 2.10) +# # Since: 2.4 ## { 'struct': 'BlockDirtyBitmapAdd', - 'data': { 'node': 'str', 'name': 'str', '*granularity': 'uint32' } } + 'data': { 'node': 'str', 'name': 'str', '*granularity': 'uint32', + '*persistent': 'bool' } } =20 ## # @block-dirty-bitmap-add: --=20 2.11.1 From nobody Sat May 4 15:42:58 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1498651817889409.9194706334072; Wed, 28 Jun 2017 05:10:17 -0700 (PDT) Received: from localhost ([::1]:32841 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dQBnR-0001z4-6H for importer@patchew.org; Wed, 28 Jun 2017 08:10:13 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:60285) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dQBj5-0006Q8-0l for qemu-devel@nongnu.org; Wed, 28 Jun 2017 08:05:48 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1dQBj2-00031H-Ey for qemu-devel@nongnu.org; Wed, 28 Jun 2017 08:05:42 -0400 Received: from mailhub.sw.ru ([195.214.232.25]:35352 helo=relay.sw.ru) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1dQBj2-0002v8-33 for qemu-devel@nongnu.org; Wed, 28 Jun 2017 08:05:40 -0400 Received: from kvm.sw.ru (msk-vpn.virtuozzo.com [195.214.232.6]) by relay.sw.ru (8.13.4/8.13.4) with ESMTP id v5SC5UUQ025017; Wed, 28 Jun 2017 15:05:33 +0300 (MSK) From: Vladimir Sementsov-Ogievskiy To: qemu-block@nongnu.org, qemu-devel@nongnu.org Date: Wed, 28 Jun 2017 15:05:24 +0300 Message-Id: <20170628120530.31251-25-vsementsov@virtuozzo.com> X-Mailer: git-send-email 2.11.1 In-Reply-To: <20170628120530.31251-1-vsementsov@virtuozzo.com> References: <20170628120530.31251-1-vsementsov@virtuozzo.com> X-detected-operating-system: by eggs.gnu.org: OpenBSD 3.x [fuzzy] X-Received-From: 195.214.232.25 Subject: [Qemu-devel] [PATCH v22 24/30] qmp: add autoload parameter to block-dirty-bitmap-add X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, vsementsov@virtuozzo.com, famz@redhat.com, armbru@redhat.com, mreitz@redhat.com, stefanha@redhat.com, pbonzini@redhat.com, den@openvz.org, jsnow@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Optional. Default is false. Signed-off-by: Vladimir Sementsov-Ogievskiy Signed-off-by: Denis V. Lunev Reviewed-by: Max Reitz Reviewed-by: John Snow --- blockdev.c | 18 ++++++++++++++++-- qapi/block-core.json | 6 +++++- 2 files changed, 21 insertions(+), 3 deletions(-) diff --git a/blockdev.c b/blockdev.c index 125acabc07..4bb7033994 100644 --- a/blockdev.c +++ b/blockdev.c @@ -1974,6 +1974,7 @@ static void block_dirty_bitmap_add_prepare(BlkActionS= tate *common, qmp_block_dirty_bitmap_add(action->node, action->name, action->has_granularity, action->granularit= y, action->has_persistent, action->persistent, + action->has_autoload, action->autoload, &local_err); =20 if (!local_err) { @@ -2722,6 +2723,7 @@ out: void qmp_block_dirty_bitmap_add(const char *node, const char *name, bool has_granularity, uint32_t granularity, bool has_persistent, bool persistent, + bool has_autoload, bool autoload, Error **errp) { BlockDriverState *bs; @@ -2751,6 +2753,15 @@ void qmp_block_dirty_bitmap_add(const char *node, co= nst char *name, if (!has_persistent) { persistent =3D false; } + if (!has_autoload) { + autoload =3D false; + } + + if (has_autoload && !persistent) { + error_setg(errp, "Autoload flag must be used only for persistent " + "bitmaps"); + return; + } =20 if (persistent && !bdrv_can_store_new_dirty_bitmap(bs, name, granularity, errp)) @@ -2759,9 +2770,12 @@ void qmp_block_dirty_bitmap_add(const char *node, co= nst char *name, } =20 bitmap =3D bdrv_create_dirty_bitmap(bs, granularity, name, errp); - if (bitmap !=3D NULL) { - bdrv_dirty_bitmap_set_persistance(bitmap, persistent); + if (bitmap =3D=3D NULL) { + return; } + + bdrv_dirty_bitmap_set_persistance(bitmap, persistent); + bdrv_dirty_bitmap_set_autoload(bitmap, autoload); } =20 void qmp_block_dirty_bitmap_remove(const char *node, const char *name, diff --git a/qapi/block-core.json b/qapi/block-core.json index 13f98ec146..5c42cc7790 100644 --- a/qapi/block-core.json +++ b/qapi/block-core.json @@ -1566,11 +1566,15 @@ # Qcow2 disks support persistent bitmaps. Default is false for # block-dirty-bitmap-add. (Since: 2.10) # +# @autoload: the bitmap will be automatically loaded when the image it is = stored +# in is opened. This flag may only be specified for persistent +# bitmaps. Default is false for block-dirty-bitmap-add. (Since:= 2.10) +# # Since: 2.4 ## { 'struct': 'BlockDirtyBitmapAdd', 'data': { 'node': 'str', 'name': 'str', '*granularity': 'uint32', - '*persistent': 'bool' } } + '*persistent': 'bool', '*autoload': 'bool' } } =20 ## # @block-dirty-bitmap-add: --=20 2.11.1 From nobody Sat May 4 15:42:58 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1498652109383160.3612920629655; Wed, 28 Jun 2017 05:15:09 -0700 (PDT) Received: from localhost ([::1]:32865 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dQBsC-0006Q9-0c for importer@patchew.org; Wed, 28 Jun 2017 08:15:08 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:60308) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dQBj5-0006QK-8v for qemu-devel@nongnu.org; Wed, 28 Jun 2017 08:05:48 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1dQBj2-000327-NC for qemu-devel@nongnu.org; Wed, 28 Jun 2017 08:05:43 -0400 Received: from mailhub.sw.ru ([195.214.232.25]:6233 helo=relay.sw.ru) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1dQBj2-0002wd-4z for qemu-devel@nongnu.org; Wed, 28 Jun 2017 08:05:40 -0400 Received: from kvm.sw.ru (msk-vpn.virtuozzo.com [195.214.232.6]) by relay.sw.ru (8.13.4/8.13.4) with ESMTP id v5SC5UUR025017; Wed, 28 Jun 2017 15:05:33 +0300 (MSK) From: Vladimir Sementsov-Ogievskiy To: qemu-block@nongnu.org, qemu-devel@nongnu.org Date: Wed, 28 Jun 2017 15:05:25 +0300 Message-Id: <20170628120530.31251-26-vsementsov@virtuozzo.com> X-Mailer: git-send-email 2.11.1 In-Reply-To: <20170628120530.31251-1-vsementsov@virtuozzo.com> References: <20170628120530.31251-1-vsementsov@virtuozzo.com> X-detected-operating-system: by eggs.gnu.org: OpenBSD 3.x [fuzzy] X-Received-From: 195.214.232.25 Subject: [Qemu-devel] [PATCH v22 25/30] qmp: add x-debug-block-dirty-bitmap-sha256 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, vsementsov@virtuozzo.com, famz@redhat.com, armbru@redhat.com, mreitz@redhat.com, stefanha@redhat.com, pbonzini@redhat.com, den@openvz.org, jsnow@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Signed-off-by: Vladimir Sementsov-Ogievskiy --- block/dirty-bitmap.c | 5 +++++ blockdev.c | 25 +++++++++++++++++++++++++ include/block/dirty-bitmap.h | 1 + include/qemu/hbitmap.h | 8 ++++++++ qapi/block-core.json | 27 +++++++++++++++++++++++++++ tests/Makefile.include | 2 +- util/hbitmap.c | 11 +++++++++++ 7 files changed, 78 insertions(+), 1 deletion(-) diff --git a/block/dirty-bitmap.c b/block/dirty-bitmap.c index d1469418e6..5fcf917707 100644 --- a/block/dirty-bitmap.c +++ b/block/dirty-bitmap.c @@ -725,3 +725,8 @@ BdrvDirtyBitmap *bdrv_dirty_bitmap_next(BlockDriverStat= e *bs, return bitmap =3D=3D NULL ? QLIST_FIRST(&bs->dirty_bitmaps) : QLIST_NEXT(bitmap, list); } + +char *bdrv_dirty_bitmap_sha256(const BdrvDirtyBitmap *bitmap, Error **errp) +{ + return hbitmap_sha256(bitmap->bitmap, errp); +} diff --git a/blockdev.c b/blockdev.c index 4bb7033994..3c8fb75208 100644 --- a/blockdev.c +++ b/blockdev.c @@ -2832,6 +2832,31 @@ void qmp_block_dirty_bitmap_clear(const char *node, = const char *name, bdrv_clear_dirty_bitmap(bitmap, NULL); } =20 +BlockDirtyBitmapSha256 *qmp_x_debug_block_dirty_bitmap_sha256(const char *= node, + const char *= name, + Error **errp) +{ + BdrvDirtyBitmap *bitmap; + BlockDriverState *bs; + BlockDirtyBitmapSha256 *ret =3D NULL; + char *sha256; + + bitmap =3D block_dirty_bitmap_lookup(node, name, &bs, errp); + if (!bitmap || !bs) { + return NULL; + } + + sha256 =3D bdrv_dirty_bitmap_sha256(bitmap, errp); + if (sha256 =3D=3D NULL) { + return NULL; + } + + ret =3D g_new(BlockDirtyBitmapSha256, 1); + ret->sha256 =3D sha256; + + return ret; +} + void hmp_drive_del(Monitor *mon, const QDict *qdict) { const char *id =3D qdict_get_str(qdict, "id"); diff --git a/include/block/dirty-bitmap.h b/include/block/dirty-bitmap.h index ccf2f81640..744479bc76 100644 --- a/include/block/dirty-bitmap.h +++ b/include/block/dirty-bitmap.h @@ -97,5 +97,6 @@ bool bdrv_dirty_bitmap_get_persistance(BdrvDirtyBitmap *b= itmap); bool bdrv_has_changed_persistent_bitmaps(BlockDriverState *bs); BdrvDirtyBitmap *bdrv_dirty_bitmap_next(BlockDriverState *bs, BdrvDirtyBitmap *bitmap); +char *bdrv_dirty_bitmap_sha256(const BdrvDirtyBitmap *bitmap, Error **errp= ); =20 #endif diff --git a/include/qemu/hbitmap.h b/include/qemu/hbitmap.h index b52304ac29..d3a74a21fc 100644 --- a/include/qemu/hbitmap.h +++ b/include/qemu/hbitmap.h @@ -253,6 +253,14 @@ void hbitmap_deserialize_ones(HBitmap *hb, uint64_t st= art, uint64_t count, void hbitmap_deserialize_finish(HBitmap *hb); =20 /** + * hbitmap_sha256: + * @bitmap: HBitmap to operate on. + * + * Returns SHA256 hash of the last level. + */ +char *hbitmap_sha256(const HBitmap *bitmap, Error **errp); + +/** * hbitmap_free: * @hb: HBitmap to operate on. * diff --git a/qapi/block-core.json b/qapi/block-core.json index 5c42cc7790..6ad8585400 100644 --- a/qapi/block-core.json +++ b/qapi/block-core.json @@ -1644,6 +1644,33 @@ 'data': 'BlockDirtyBitmap' } =20 ## +# @BlockDirtyBitmapSha256: +# +# SHA256 hash of dirty bitmap data +# +# @sha256: ASCII representation of SHA256 bitmap hash +# +# Since: 2.10 +## + { 'struct': 'BlockDirtyBitmapSha256', + 'data': {'sha256': 'str'} } + +## +# @x-debug-block-dirty-bitmap-sha256: +# +# Get bitmap SHA256 +# +# Returns: BlockDirtyBitmapSha256 on success +# If @node is not a valid block device, DeviceNotFound +# If @name is not found or if hashing has failed, GenericError wi= th an +# explanation +# +# Since: 2.10 +## + { 'command': 'x-debug-block-dirty-bitmap-sha256', + 'data': 'BlockDirtyBitmap', 'returns': 'BlockDirtyBitmapSha256' } + +## # @blockdev-mirror: # # Start mirroring a block device's writes to a new destination. diff --git a/tests/Makefile.include b/tests/Makefile.include index ae889cae02..c738e92673 100644 --- a/tests/Makefile.include +++ b/tests/Makefile.include @@ -553,7 +553,7 @@ tests/test-blockjob$(EXESUF): tests/test-blockjob.o $(t= est-block-obj-y) $(test-u tests/test-blockjob-txn$(EXESUF): tests/test-blockjob-txn.o $(test-block-o= bj-y) $(test-util-obj-y) tests/test-thread-pool$(EXESUF): tests/test-thread-pool.o $(test-block-obj= -y) tests/test-iov$(EXESUF): tests/test-iov.o $(test-util-obj-y) -tests/test-hbitmap$(EXESUF): tests/test-hbitmap.o $(test-util-obj-y) +tests/test-hbitmap$(EXESUF): tests/test-hbitmap.o $(test-util-obj-y) $(tes= t-crypto-obj-y) tests/test-x86-cpuid$(EXESUF): tests/test-x86-cpuid.o tests/test-xbzrle$(EXESUF): tests/test-xbzrle.o migration/xbzrle.o migrati= on/page_cache.o $(test-util-obj-y) tests/test-cutils$(EXESUF): tests/test-cutils.o util/cutils.o diff --git a/util/hbitmap.c b/util/hbitmap.c index 0c1591a594..21535cc90b 100644 --- a/util/hbitmap.c +++ b/util/hbitmap.c @@ -13,6 +13,7 @@ #include "qemu/hbitmap.h" #include "qemu/host-utils.h" #include "trace.h" +#include "crypto/hash.h" =20 /* HBitmaps provides an array of bits. The bits are stored as usual in an * array of unsigned longs, but HBitmap is also optimized to provide fast @@ -727,3 +728,13 @@ void hbitmap_free_meta(HBitmap *hb) hbitmap_free(hb->meta); hb->meta =3D NULL; } + +char *hbitmap_sha256(const HBitmap *bitmap, Error **errp) +{ + size_t size =3D bitmap->sizes[HBITMAP_LEVELS - 1] * sizeof(unsigned lo= ng); + char *data =3D (char *)bitmap->levels[HBITMAP_LEVELS - 1]; + char *hash =3D NULL; + qcrypto_hash_digest(QCRYPTO_HASH_ALG_SHA256, data, size, &hash, errp); + + return hash; +} --=20 2.11.1 From nobody Sat May 4 15:42:58 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1498652723541619.5232746755873; Wed, 28 Jun 2017 05:25:23 -0700 (PDT) Received: from localhost ([::1]:32942 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dQC24-000866-6t for importer@patchew.org; Wed, 28 Jun 2017 08:25:20 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:60317) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dQBj5-0006QL-S3 for qemu-devel@nongnu.org; Wed, 28 Jun 2017 08:05:48 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1dQBj2-00032T-UM for qemu-devel@nongnu.org; Wed, 28 Jun 2017 08:05:43 -0400 Received: from mailhub.sw.ru ([195.214.232.25]:33625 helo=relay.sw.ru) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1dQBj2-0002v5-H1 for qemu-devel@nongnu.org; Wed, 28 Jun 2017 08:05:40 -0400 Received: from kvm.sw.ru (msk-vpn.virtuozzo.com [195.214.232.6]) by relay.sw.ru (8.13.4/8.13.4) with ESMTP id v5SC5UUS025017; Wed, 28 Jun 2017 15:05:33 +0300 (MSK) From: Vladimir Sementsov-Ogievskiy To: qemu-block@nongnu.org, qemu-devel@nongnu.org Date: Wed, 28 Jun 2017 15:05:26 +0300 Message-Id: <20170628120530.31251-27-vsementsov@virtuozzo.com> X-Mailer: git-send-email 2.11.1 In-Reply-To: <20170628120530.31251-1-vsementsov@virtuozzo.com> References: <20170628120530.31251-1-vsementsov@virtuozzo.com> X-detected-operating-system: by eggs.gnu.org: OpenBSD 3.x [fuzzy] X-Received-From: 195.214.232.25 Subject: [Qemu-devel] [PATCH v22 26/30] iotests: test qcow2 persistent dirty bitmap X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, vsementsov@virtuozzo.com, famz@redhat.com, armbru@redhat.com, mreitz@redhat.com, stefanha@redhat.com, pbonzini@redhat.com, den@openvz.org, jsnow@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Max Reitz --- tests/qemu-iotests/165 | 105 +++++++++++++++++++++++++++++++++++++++++= ++++ tests/qemu-iotests/165.out | 5 +++ tests/qemu-iotests/group | 1 + 3 files changed, 111 insertions(+) create mode 100755 tests/qemu-iotests/165 create mode 100644 tests/qemu-iotests/165.out diff --git a/tests/qemu-iotests/165 b/tests/qemu-iotests/165 new file mode 100755 index 0000000000..74d7b79a0b --- /dev/null +++ b/tests/qemu-iotests/165 @@ -0,0 +1,105 @@ +#!/usr/bin/env python +# +# Tests for persistent dirty bitmaps. +# +# Copyright: Vladimir Sementsov-Ogievskiy 2015-2017 +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program. If not, see . +# + +import os +import re +import iotests +from iotests import qemu_img + +disk =3D os.path.join(iotests.test_dir, 'disk') +disk_size =3D 0x40000000 # 1G + +# regions for qemu_io: (start, count) in bytes +regions1 =3D ((0, 0x100000), + (0x200000, 0x100000)) + +regions2 =3D ((0x10000000, 0x20000), + (0x3fff0000, 0x10000)) + +class TestPersistentDirtyBitmap(iotests.QMPTestCase): + + def setUp(self): + qemu_img('create', '-f', iotests.imgfmt, disk, str(disk_size)) + + def tearDown(self): + os.remove(disk) + + def mkVm(self): + return iotests.VM().add_drive(disk) + + def mkVmRo(self): + return iotests.VM().add_drive(disk, opts=3D'readonly=3Don') + + def getSha256(self): + result =3D self.vm.qmp('x-debug-block-dirty-bitmap-sha256', + node=3D'drive0', name=3D'bitmap0') + return result['return']['sha256'] + + def checkBitmap(self, sha256): + result =3D self.vm.qmp('x-debug-block-dirty-bitmap-sha256', + node=3D'drive0', name=3D'bitmap0') + self.assert_qmp(result, 'return/sha256', sha256); + + def writeRegions(self, regions): + for r in regions: + self.vm.hmp_qemu_io('drive0', + 'write %d %d' % r) + + def qmpAddBitmap(self): + self.vm.qmp('block-dirty-bitmap-add', node=3D'drive0', + name=3D'bitmap0', persistent=3DTrue, autoload=3DTrue) + + def test_persistent(self): + self.vm =3D self.mkVm() + self.vm.launch() + self.qmpAddBitmap() + + self.writeRegions(regions1) + sha256 =3D self.getSha256() + + self.vm.shutdown() + + self.vm =3D self.mkVmRo() + self.vm.launch() + self.vm.shutdown() + + #catch 'Persistent bitmaps are lost' possible error + log =3D self.vm.get_log() + log =3D re.sub(r'^\[I \d+\.\d+\] OPENED\n', '', log) + log =3D re.sub(r'\[I \+\d+\.\d+\] CLOSED\n?$', '', log) + if log: + print log + + self.vm =3D self.mkVm() + self.vm.launch() + + self.checkBitmap(sha256) + self.writeRegions(regions2) + sha256 =3D self.getSha256() + + self.vm.shutdown() + self.vm.launch() + + self.checkBitmap(sha256) + + self.vm.shutdown() + +if __name__ =3D=3D '__main__': + iotests.main(supported_fmts=3D['qcow2']) diff --git a/tests/qemu-iotests/165.out b/tests/qemu-iotests/165.out new file mode 100644 index 0000000000..ae1213e6f8 --- /dev/null +++ b/tests/qemu-iotests/165.out @@ -0,0 +1,5 @@ +. +---------------------------------------------------------------------- +Ran 1 tests + +OK diff --git a/tests/qemu-iotests/group b/tests/qemu-iotests/group index a6acafffd7..ad09b8005b 100644 --- a/tests/qemu-iotests/group +++ b/tests/qemu-iotests/group @@ -163,6 +163,7 @@ 159 rw auto quick 160 rw auto quick 162 auto quick +165 rw auto quick 170 rw auto quick 171 rw auto quick 172 auto --=20 2.11.1 From nobody Sat May 4 15:42:58 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 149865196883730.642059972820675; Wed, 28 Jun 2017 05:12:48 -0700 (PDT) Received: from localhost ([::1]:32854 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dQBpr-00047v-Dt for importer@patchew.org; Wed, 28 Jun 2017 08:12:43 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:60387) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dQBj6-0006QR-Su for qemu-devel@nongnu.org; Wed, 28 Jun 2017 08:05:48 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1dQBj4-00034P-GA for qemu-devel@nongnu.org; Wed, 28 Jun 2017 08:05:44 -0400 Received: from mailhub.sw.ru ([195.214.232.25]:48346 helo=relay.sw.ru) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1dQBj2-0002vy-H8 for qemu-devel@nongnu.org; Wed, 28 Jun 2017 08:05:42 -0400 Received: from kvm.sw.ru (msk-vpn.virtuozzo.com [195.214.232.6]) by relay.sw.ru (8.13.4/8.13.4) with ESMTP id v5SC5UUT025017; Wed, 28 Jun 2017 15:05:33 +0300 (MSK) From: Vladimir Sementsov-Ogievskiy To: qemu-block@nongnu.org, qemu-devel@nongnu.org Date: Wed, 28 Jun 2017 15:05:27 +0300 Message-Id: <20170628120530.31251-28-vsementsov@virtuozzo.com> X-Mailer: git-send-email 2.11.1 In-Reply-To: <20170628120530.31251-1-vsementsov@virtuozzo.com> References: <20170628120530.31251-1-vsementsov@virtuozzo.com> X-detected-operating-system: by eggs.gnu.org: OpenBSD 3.x [fuzzy] X-Received-From: 195.214.232.25 Subject: [Qemu-devel] [PATCH v22 27/30] block/dirty-bitmap: add bdrv_remove_persistent_dirty_bitmap X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, vsementsov@virtuozzo.com, famz@redhat.com, armbru@redhat.com, mreitz@redhat.com, stefanha@redhat.com, pbonzini@redhat.com, den@openvz.org, jsnow@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Interface for removing persistent bitmap from its storage. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Max Reitz Reviewed-by: John Snow --- block/dirty-bitmap.c | 18 ++++++++++++++++++ include/block/block_int.h | 3 +++ include/block/dirty-bitmap.h | 3 +++ 3 files changed, 24 insertions(+) diff --git a/block/dirty-bitmap.c b/block/dirty-bitmap.c index 5fcf917707..b2ca78b4d0 100644 --- a/block/dirty-bitmap.c +++ b/block/dirty-bitmap.c @@ -395,6 +395,7 @@ void bdrv_release_dirty_bitmap(BlockDriverState *bs, Bd= rvDirtyBitmap *bitmap) /** * Release all named dirty bitmaps attached to a BDS (for use in bdrv_clos= e()). * There must not be any frozen bitmaps attached. + * This function does not remove persistent bitmaps from the storage. * Called with BQL taken. */ void bdrv_release_named_dirty_bitmaps(BlockDriverState *bs) @@ -402,6 +403,23 @@ void bdrv_release_named_dirty_bitmaps(BlockDriverState= *bs) bdrv_do_release_matching_dirty_bitmap(bs, NULL, true); } =20 +/** + * Remove persistent dirty bitmap from the storage if it exists. + * Absence of bitmap is not an error, because we have the following scenar= io: + * BdrvDirtyBitmap can have .persistent =3D true but not yet saved and hav= e no + * stored version. For such bitmap bdrv_remove_persistent_dirty_bitmap() s= hould + * not fail. + * This function doesn't release corresponding BdrvDirtyBitmap. + */ +void bdrv_remove_persistent_dirty_bitmap(BlockDriverState *bs, + const char *name, + Error **errp) +{ + if (bs->drv && bs->drv->bdrv_remove_persistent_dirty_bitmap) { + bs->drv->bdrv_remove_persistent_dirty_bitmap(bs, name, errp); + } +} + /* Called with BQL taken. */ void bdrv_disable_dirty_bitmap(BdrvDirtyBitmap *bitmap) { diff --git a/include/block/block_int.h b/include/block/block_int.h index 009b4d41df..b3be797a96 100644 --- a/include/block/block_int.h +++ b/include/block/block_int.h @@ -391,6 +391,9 @@ struct BlockDriver { const char *name, uint32_t granularity, Error **errp); + void (*bdrv_remove_persistent_dirty_bitmap)(BlockDriverState *bs, + const char *name, + Error **errp); =20 QLIST_ENTRY(BlockDriver) list; }; diff --git a/include/block/dirty-bitmap.h b/include/block/dirty-bitmap.h index 744479bc76..d38233efd8 100644 --- a/include/block/dirty-bitmap.h +++ b/include/block/dirty-bitmap.h @@ -25,6 +25,9 @@ BdrvDirtyBitmap *bdrv_find_dirty_bitmap(BlockDriverState = *bs, void bdrv_dirty_bitmap_make_anon(BdrvDirtyBitmap *bitmap); void bdrv_release_dirty_bitmap(BlockDriverState *bs, BdrvDirtyBitmap *bitm= ap); void bdrv_release_named_dirty_bitmaps(BlockDriverState *bs); +void bdrv_remove_persistent_dirty_bitmap(BlockDriverState *bs, + const char *name, + Error **errp); void bdrv_disable_dirty_bitmap(BdrvDirtyBitmap *bitmap); void bdrv_enable_dirty_bitmap(BdrvDirtyBitmap *bitmap); BlockDirtyInfoList *bdrv_query_dirty_bitmaps(BlockDriverState *bs); --=20 2.11.1 From nobody Sat May 4 15:42:58 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1498651967395352.52054780334765; Wed, 28 Jun 2017 05:12:47 -0700 (PDT) Received: from localhost ([::1]:32856 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dQBps-00048p-5G for importer@patchew.org; Wed, 28 Jun 2017 08:12:44 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:60397) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dQBj7-0006QW-3X for qemu-devel@nongnu.org; Wed, 28 Jun 2017 08:05:48 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1dQBj3-00032t-4t for qemu-devel@nongnu.org; Wed, 28 Jun 2017 08:05:45 -0400 Received: from mailhub.sw.ru ([195.214.232.25]:9340 helo=relay.sw.ru) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1dQBj2-0002vQ-OI for qemu-devel@nongnu.org; Wed, 28 Jun 2017 08:05:41 -0400 Received: from kvm.sw.ru (msk-vpn.virtuozzo.com [195.214.232.6]) by relay.sw.ru (8.13.4/8.13.4) with ESMTP id v5SC5UUU025017; Wed, 28 Jun 2017 15:05:33 +0300 (MSK) From: Vladimir Sementsov-Ogievskiy To: qemu-block@nongnu.org, qemu-devel@nongnu.org Date: Wed, 28 Jun 2017 15:05:28 +0300 Message-Id: <20170628120530.31251-29-vsementsov@virtuozzo.com> X-Mailer: git-send-email 2.11.1 In-Reply-To: <20170628120530.31251-1-vsementsov@virtuozzo.com> References: <20170628120530.31251-1-vsementsov@virtuozzo.com> X-detected-operating-system: by eggs.gnu.org: OpenBSD 3.x [fuzzy] X-Received-From: 195.214.232.25 Subject: [Qemu-devel] [PATCH v22 28/30] qcow2: add .bdrv_remove_persistent_dirty_bitmap X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, vsementsov@virtuozzo.com, famz@redhat.com, armbru@redhat.com, mreitz@redhat.com, stefanha@redhat.com, pbonzini@redhat.com, den@openvz.org, jsnow@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Realize .bdrv_remove_persistent_dirty_bitmap interface. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Max Reitz Reviewed-by: John Snow --- block/qcow2-bitmap.c | 41 +++++++++++++++++++++++++++++++++++++++++ block/qcow2.c | 1 + block/qcow2.h | 3 +++ 3 files changed, 45 insertions(+) diff --git a/block/qcow2-bitmap.c b/block/qcow2-bitmap.c index f45324e584..8448bec46d 100644 --- a/block/qcow2-bitmap.c +++ b/block/qcow2-bitmap.c @@ -1236,6 +1236,47 @@ static Qcow2Bitmap *find_bitmap_by_name(Qcow2BitmapL= ist *bm_list, return NULL; } =20 +void qcow2_remove_persistent_dirty_bitmap(BlockDriverState *bs, + const char *name, + Error **errp) +{ + int ret; + BDRVQcow2State *s =3D bs->opaque; + Qcow2Bitmap *bm; + Qcow2BitmapList *bm_list; + + if (s->nb_bitmaps =3D=3D 0) { + /* Absence of the bitmap is not an error: see explanation above + * bdrv_remove_persistent_dirty_bitmap() definition. */ + return; + } + + bm_list =3D bitmap_list_load(bs, s->bitmap_directory_offset, + s->bitmap_directory_size, errp); + if (bm_list =3D=3D NULL) { + return; + } + + bm =3D find_bitmap_by_name(bm_list, name); + if (bm =3D=3D NULL) { + goto fail; + } + + QSIMPLEQ_REMOVE(bm_list, bm, Qcow2Bitmap, entry); + + ret =3D update_ext_header_and_dir(bs, bm_list); + if (ret < 0) { + error_setg_errno(errp, -ret, "Failed to update bitmap extension"); + goto fail; + } + + free_bitmap_clusters(bs, &bm->table); + +fail: + bitmap_free(bm); + bitmap_list_free(bm_list); +} + void qcow2_store_persistent_dirty_bitmaps(BlockDriverState *bs, Error **er= rp) { BdrvDirtyBitmap *bitmap; diff --git a/block/qcow2.c b/block/qcow2.c index fc1f69cead..b836b8c831 100644 --- a/block/qcow2.c +++ b/block/qcow2.c @@ -3612,6 +3612,7 @@ BlockDriver bdrv_qcow2 =3D { =20 .bdrv_reopen_bitmaps_rw =3D qcow2_reopen_bitmaps_rw, .bdrv_can_store_new_dirty_bitmap =3D qcow2_can_store_new_dirty_bitmap, + .bdrv_remove_persistent_dirty_bitmap =3D qcow2_remove_persistent_dirty= _bitmap, }; =20 static void bdrv_qcow2_init(void) diff --git a/block/qcow2.h b/block/qcow2.h index 8b2f66f8b6..ffb951df52 100644 --- a/block/qcow2.h +++ b/block/qcow2.h @@ -637,5 +637,8 @@ bool qcow2_can_store_new_dirty_bitmap(BlockDriverState = *bs, const char *name, uint32_t granularity, Error **errp); +void qcow2_remove_persistent_dirty_bitmap(BlockDriverState *bs, + const char *name, + Error **errp); =20 #endif --=20 2.11.1 From nobody Sat May 4 15:42:58 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1498651971804632.1369260041843; Wed, 28 Jun 2017 05:12:51 -0700 (PDT) Received: from localhost ([::1]:32857 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dQBps-000497-Fu for importer@patchew.org; Wed, 28 Jun 2017 08:12:44 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:60407) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dQBj7-0006Qb-DT for qemu-devel@nongnu.org; Wed, 28 Jun 2017 08:05:48 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1dQBj3-00033d-CL for qemu-devel@nongnu.org; Wed, 28 Jun 2017 08:05:45 -0400 Received: from mailhub.sw.ru ([195.214.232.25]:15228 helo=relay.sw.ru) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1dQBj2-0002yD-Tt for qemu-devel@nongnu.org; Wed, 28 Jun 2017 08:05:41 -0400 Received: from kvm.sw.ru (msk-vpn.virtuozzo.com [195.214.232.6]) by relay.sw.ru (8.13.4/8.13.4) with ESMTP id v5SC5UUV025017; Wed, 28 Jun 2017 15:05:33 +0300 (MSK) From: Vladimir Sementsov-Ogievskiy To: qemu-block@nongnu.org, qemu-devel@nongnu.org Date: Wed, 28 Jun 2017 15:05:29 +0300 Message-Id: <20170628120530.31251-30-vsementsov@virtuozzo.com> X-Mailer: git-send-email 2.11.1 In-Reply-To: <20170628120530.31251-1-vsementsov@virtuozzo.com> References: <20170628120530.31251-1-vsementsov@virtuozzo.com> X-detected-operating-system: by eggs.gnu.org: OpenBSD 3.x [fuzzy] X-Received-From: 195.214.232.25 Subject: [Qemu-devel] [PATCH v22 29/30] qmp: block-dirty-bitmap-remove: remove persistent X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, vsementsov@virtuozzo.com, famz@redhat.com, armbru@redhat.com, mreitz@redhat.com, stefanha@redhat.com, pbonzini@redhat.com, den@openvz.org, jsnow@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Remove persistent bitmap from the storage on block-dirty-bitmap-remove. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Max Reitz Reviewed-by: John Snow --- blockdev.c | 10 ++++++++++ qapi/block-core.json | 3 ++- 2 files changed, 12 insertions(+), 1 deletion(-) diff --git a/blockdev.c b/blockdev.c index 3c8fb75208..122a936719 100644 --- a/blockdev.c +++ b/blockdev.c @@ -2783,6 +2783,7 @@ void qmp_block_dirty_bitmap_remove(const char *node, = const char *name, { BlockDriverState *bs; BdrvDirtyBitmap *bitmap; + Error *local_err =3D NULL; =20 bitmap =3D block_dirty_bitmap_lookup(node, name, &bs, errp); if (!bitmap || !bs) { @@ -2795,6 +2796,15 @@ void qmp_block_dirty_bitmap_remove(const char *node,= const char *name, name); return; } + + if (bdrv_dirty_bitmap_get_persistance(bitmap)) { + bdrv_remove_persistent_dirty_bitmap(bs, name, &local_err); + if (local_err !=3D NULL) { + error_propagate(errp, local_err); + return; + } + } + bdrv_dirty_bitmap_make_anon(bitmap); bdrv_release_dirty_bitmap(bs, bitmap); } diff --git a/qapi/block-core.json b/qapi/block-core.json index 6ad8585400..e471efa1b4 100644 --- a/qapi/block-core.json +++ b/qapi/block-core.json @@ -1601,7 +1601,8 @@ # @block-dirty-bitmap-remove: # # Stop write tracking and remove the dirty bitmap that was created -# with block-dirty-bitmap-add. +# with block-dirty-bitmap-add. If the bitmap is persistent, remove it from= its +# storage too. # # Returns: nothing on success # If @node is not a valid block device or node, DeviceNotFound --=20 2.11.1 From nobody Sat May 4 15:42:58 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1498652110333679.3272548914863; Wed, 28 Jun 2017 05:15:10 -0700 (PDT) Received: from localhost ([::1]:32864 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dQBsB-0006Pf-3z for importer@patchew.org; Wed, 28 Jun 2017 08:15:07 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:60395) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dQBj7-0006QU-2w for qemu-devel@nongnu.org; Wed, 28 Jun 2017 08:05:48 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1dQBj3-00033S-9v for qemu-devel@nongnu.org; Wed, 28 Jun 2017 08:05:44 -0400 Received: from mailhub.sw.ru ([195.214.232.25]:43849 helo=relay.sw.ru) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1dQBj2-0002wH-Sx for qemu-devel@nongnu.org; Wed, 28 Jun 2017 08:05:41 -0400 Received: from kvm.sw.ru (msk-vpn.virtuozzo.com [195.214.232.6]) by relay.sw.ru (8.13.4/8.13.4) with ESMTP id v5SC5UUW025017; Wed, 28 Jun 2017 15:05:33 +0300 (MSK) From: Vladimir Sementsov-Ogievskiy To: qemu-block@nongnu.org, qemu-devel@nongnu.org Date: Wed, 28 Jun 2017 15:05:30 +0300 Message-Id: <20170628120530.31251-31-vsementsov@virtuozzo.com> X-Mailer: git-send-email 2.11.1 In-Reply-To: <20170628120530.31251-1-vsementsov@virtuozzo.com> References: <20170628120530.31251-1-vsementsov@virtuozzo.com> X-detected-operating-system: by eggs.gnu.org: OpenBSD 3.x [fuzzy] X-Received-From: 195.214.232.25 Subject: [Qemu-devel] [PATCH v22 30/30] block: release persistent bitmaps on inactivate X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, vsementsov@virtuozzo.com, famz@redhat.com, armbru@redhat.com, mreitz@redhat.com, stefanha@redhat.com, pbonzini@redhat.com, den@openvz.org, jsnow@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" We should release them here to reload on invalidate cache. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Max Reitz --- block.c | 4 ++++ block/dirty-bitmap.c | 29 +++++++++++++++++++++++------ include/block/dirty-bitmap.h | 1 + 3 files changed, 28 insertions(+), 6 deletions(-) diff --git a/block.c b/block.c index b2719bceff..acc6e4de1c 100644 --- a/block.c +++ b/block.c @@ -4156,6 +4156,10 @@ static int bdrv_inactivate_recurse(BlockDriverState = *bs, } } =20 + /* At this point persistent bitmaps should be already stored by the fo= rmat + * driver */ + bdrv_release_persistent_dirty_bitmaps(bs); + return 0; } =20 diff --git a/block/dirty-bitmap.c b/block/dirty-bitmap.c index b2ca78b4d0..543bddb9b5 100644 --- a/block/dirty-bitmap.c +++ b/block/dirty-bitmap.c @@ -356,15 +356,20 @@ void bdrv_dirty_bitmap_truncate(BlockDriverState *bs) bdrv_dirty_bitmaps_unlock(bs); } =20 +static bool bdrv_dirty_bitmap_has_name(BdrvDirtyBitmap *bitmap) +{ + return !!bdrv_dirty_bitmap_name(bitmap); +} + /* Called with BQL taken. */ -static void bdrv_do_release_matching_dirty_bitmap(BlockDriverState *bs, - BdrvDirtyBitmap *bitmap, - bool only_named) +static void bdrv_do_release_matching_dirty_bitmap( + BlockDriverState *bs, BdrvDirtyBitmap *bitmap, + bool (*cond)(BdrvDirtyBitmap *bitmap)) { BdrvDirtyBitmap *bm, *next; bdrv_dirty_bitmaps_lock(bs); QLIST_FOREACH_SAFE(bm, &bs->dirty_bitmaps, list, next) { - if ((!bitmap || bm =3D=3D bitmap) && (!only_named || bm->name)) { + if ((!bitmap || bm =3D=3D bitmap) && (!cond || cond(bm))) { assert(!bm->active_iterators); assert(!bdrv_dirty_bitmap_frozen(bm)); assert(!bm->meta); @@ -389,7 +394,7 @@ out: /* Called with BQL taken. */ void bdrv_release_dirty_bitmap(BlockDriverState *bs, BdrvDirtyBitmap *bitm= ap) { - bdrv_do_release_matching_dirty_bitmap(bs, bitmap, false); + bdrv_do_release_matching_dirty_bitmap(bs, bitmap, NULL); } =20 /** @@ -400,7 +405,19 @@ void bdrv_release_dirty_bitmap(BlockDriverState *bs, B= drvDirtyBitmap *bitmap) */ void bdrv_release_named_dirty_bitmaps(BlockDriverState *bs) { - bdrv_do_release_matching_dirty_bitmap(bs, NULL, true); + bdrv_do_release_matching_dirty_bitmap(bs, NULL, bdrv_dirty_bitmap_has_= name); +} + +/** + * Release all persistent dirty bitmaps attached to a BDS (for use in + * bdrv_inactivate_recurse()). + * There must not be any frozen bitmaps attached. + * This function does not remove persistent bitmaps from the storage. + */ +void bdrv_release_persistent_dirty_bitmaps(BlockDriverState *bs) +{ + bdrv_do_release_matching_dirty_bitmap(bs, NULL, + bdrv_dirty_bitmap_get_persistanc= e); } =20 /** diff --git a/include/block/dirty-bitmap.h b/include/block/dirty-bitmap.h index d38233efd8..cbd9704e6a 100644 --- a/include/block/dirty-bitmap.h +++ b/include/block/dirty-bitmap.h @@ -25,6 +25,7 @@ BdrvDirtyBitmap *bdrv_find_dirty_bitmap(BlockDriverState = *bs, void bdrv_dirty_bitmap_make_anon(BdrvDirtyBitmap *bitmap); void bdrv_release_dirty_bitmap(BlockDriverState *bs, BdrvDirtyBitmap *bitm= ap); void bdrv_release_named_dirty_bitmaps(BlockDriverState *bs); +void bdrv_release_persistent_dirty_bitmaps(BlockDriverState *bs); void bdrv_remove_persistent_dirty_bitmap(BlockDriverState *bs, const char *name, Error **errp); --=20 2.11.1