From nobody Fri May 3 21:45:48 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 148595445369728.18139351216041; Wed, 1 Feb 2017 05:07:33 -0800 (PST) Received: from localhost ([::1]:50555 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cYudH-0000x7-U4 for importer@patchew.org; Wed, 01 Feb 2017 08:07:31 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:57190) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cYuBc-0000Nx-PB for qemu-devel@nongnu.org; Wed, 01 Feb 2017 07:38:58 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1cYuBY-0002rZ-RW for qemu-devel@nongnu.org; Wed, 01 Feb 2017 07:38:56 -0500 Received: from smtp3.mundo-r.com ([212.51.32.191]:24384 helo=smtp4.mundo-r.com) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cYuBY-0002rF-Ck; Wed, 01 Feb 2017 07:38:52 -0500 Received: from fanzine.igalia.com ([91.117.99.155]) by smtp4.mundo-r.com with ESMTP; 01 Feb 2017 13:38:50 +0100 Received: from [194.100.51.2] (helo=perseus.local) by fanzine.igalia.com with esmtpsa (Cipher TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim) id 1cYuBV-00012p-RB; Wed, 01 Feb 2017 13:38:50 +0100 Received: from berto by perseus.local with local (Exim 4.88) (envelope-from ) id 1cYuBD-0001GP-4I; Wed, 01 Feb 2017 14:38:31 +0200 IronPort-PHdr: =?us-ascii?q?9a23=3ATanF4BSIcEWJRj95LMwt45LrR9psv+yvbD5Q0YIu?= =?us-ascii?q?jvd0So/mwa6zYxWN2/xhgRfzUJnB7Loc0qyN4vymBzNLvc7JmUtBWaQEbwUCh8?= =?us-ascii?q?QSkl5oK+++Imq/EsTXaTcnFt9JTl5v8iLzG0FUHMHjew+a+SXqvnYdFRrlKAV6?= =?us-ascii?q?OPn+FJLMgMSrzeCy/IDYbxlViDanb75/KBq7oR/Qu8QUjodvJbo9wQbVr3VVfO?= =?us-ascii?q?hb2XlmLk+JkRbm4cew8p9j8yBOtP8k6sVNT6b0cbkmQLJBFDgpPHw768PttRnY?= =?us-ascii?q?UAuA/WAcXXkMkhpJGAfK8hf3VYrsvyTgt+p93C6aPdDqTb0xRD+v4btnRAPuhS?= =?us-ascii?q?waLDMy7n3ZhdJsg6JauBKhpgJww4jIYIGOKfFyerrRcc4GSWZdW8pcUTFKDIGh?= =?us-ascii?q?YIsVF+cPPfhWoZTzqVUNqhWzHhWgCP/1xzNUnHL6wbE23uI8Gg/GxgwgGNcOvW?= =?us-ascii?q?zIodXxL6gdT/q1zLXPzT7ebv1Zwy3955TJch87vPqBWql/ftTLyUkzEAPIlVSQ?= =?us-ascii?q?qYP/MDOR1+QCqXKX4PZnVeKqkmMqrRx6rDu3xso0joTFm5gZxk3Z+Sh72oo4Id?= =?us-ascii?q?m1RFRmbdOkEJZcrzyWOo95T884Xm1kpDo2xqcGtJKmZiQG1pIqzAPFZfOdaYiH?= =?us-ascii?q?+BfjWf6UITd/mX1qZqqyhw238Ui80u38UdS00EpSoipFjNbMsncN2gTd6sedTP?= =?us-ascii?q?t8/0ah2TCR2AzJ6+FELlo7la7aK5E72LI/ip0TsUHbEi/3nkX5krOWe10g9+S1?= =?us-ascii?q?8ejrf6jqq52GO4JwjgzyKLkil8y/DOggNwgBRWmb+eCy1L35+k35Ra1HgeYona?= =?us-ascii?q?nDrJDaIt8Wpq+2AwBP1oYj6gywAy2639QfmHkLNEhFdw6fj4j1J1HOJ+j1De6h?= =?us-ascii?q?jFSpjTdn3u3JMaP/AprTKnjOi7HhfbF7605Tzgoz0MpT55VOCrEOOPjzQFP+tM?= =?us-ascii?q?TEDh8lNAy52//nB8t41oMDQG6AGauZMKTOvl+L/e8vJe6MZIkauDb7Nvgp/fnu?= =?us-ascii?q?jWU2mVUFZ6mmwYMXaGykHvRhO0iZYnTtgtAFEWcEpQc+VvLlh0CCUTFJe3a+Rb?= =?us-ascii?q?4z5jY+CIi+F4fMWpitgKCd3Ce8BpBWY3pGBU6NEXf0doSJQO0MZzyPLc9hiDME?= =?us-ascii?q?SaKtRJMm1RGrrAX60aZoLvLI+i0EspLuzMZ66PbXlR4o9jx7Ed+Q03uTQG5pnm?= =?us-ascii?q?MHXSM20Lpjrkx6z1fQmZR/1uVVEMEW6/5XXwMSM5nawOpnTdfoVVHvZNCMHX+i?= =?us-ascii?q?SNi8HTZ5YdsryN4HKxJ3Fs++iTjC0COuArZTnLuOUs9nupnA1mT8cp4ug03N07?= =?us-ascii?q?Ms2hx/GpNC?= X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: =?us-ascii?q?A2DvBADR1ZFY/5tjdVtdHQEFAQsBGAEFA?= =?us-ascii?q?QsBgn0rJ4FDjWCQVQgBAQEFAYEdkyqCD4INGoYIgjk/GAEBAQEBAQEBAQEBYSi?= =?us-ascii?q?CMxsBgkhST040iWURAa42OoszMoYHiVAchXkFm1yKA4dzDYF5iD+GRYgnilofO?= =?us-ascii?q?IEbEwgVFYQvAU4cgWNzhWEBJYIWAQEB?= X-IPAS-Result: =?us-ascii?q?A2DvBADR1ZFY/5tjdVtdHQEFAQsBGAEFAQsBgn0rJ4FDjWC?= =?us-ascii?q?QVQgBAQEFAYEdkyqCD4INGoYIgjk/GAEBAQEBAQEBAQEBYSiCMxsBgkhST040i?= =?us-ascii?q?WURAa42OoszMoYHiVAchXkFm1yKA4dzDYF5iD+GRYgnilofOIEbEwgVFYQvAU4?= =?us-ascii?q?cgWNzhWEBJYIWAQEB?= X-IronPort-AV: E=Sophos;i="5.33,320,1477954800"; d="scan'208";a="105918737" From: Alberto Garcia To: qemu-devel@nongnu.org Date: Wed, 1 Feb 2017 14:38:28 +0200 Message-Id: <20170201123828.4815-1-berto@igalia.com> X-Mailer: git-send-email 2.11.0 X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 212.51.32.191 Subject: [Qemu-devel] [PATCH v2] qcow2: Optimize the refcount-block overlap check X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Alberto Garcia , qemu-block@nongnu.org, Max Reitz Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" The metadata overlap checks introduced in a40f1c2add help detect corruption in the qcow2 image by verifying that data writes don't overlap with existing metadata sections. The 'refcount-block' check in particular iterates over the refcount table in order to get the addresses of all refcount blocks and check that none of them overlap with the region where we want to write. The problem with the refcount table is that since it always occupies complete clusters its size is usually very big. With the default values of cluster_size=3D64KB and refcount_bits=3D16 this table holds 8192 entries, each one of them enough to map 2GB worth of host clusters. So unless we're using images with several TB of allocated data this table is going to be mostly empty, and iterating over it is a waste of CPU. If the storage backend is fast enough this can have an effect on I/O performance. This patch keeps the index of the last used (i.e. non-zero) entry in the refcount table and updates it every time the table changes. The refcount-block overlap check then uses that index instead of reading the whole table. In my tests with a 4GB qcow2 file stored in RAM this doubles the amount of write IOPS. Signed-off-by: Alberto Garcia --- Changes: v2: - Handle tables with holes correctly in alloc_refcount_block() block/qcow2-refcount.c | 24 +++++++++++++++++++++++- block/qcow2.c | 1 + block/qcow2.h | 1 + 3 files changed, 25 insertions(+), 1 deletion(-) diff --git a/block/qcow2-refcount.c b/block/qcow2-refcount.c index cbfb3fe064..3dbde18612 100644 --- a/block/qcow2-refcount.c +++ b/block/qcow2-refcount.c @@ -83,6 +83,16 @@ static Qcow2SetRefcountFunc *const set_refcount_funcs[] = =3D { /*********************************************************/ /* refcount handling */ =20 +static void update_max_refcount_table_index(BDRVQcow2State *s) +{ + unsigned i =3D s->refcount_table_size - 1; + while (i > 0 && (s->refcount_table[i] & REFT_OFFSET_MASK) =3D=3D 0) { + i--; + } + /* Set s->max_refcount_table_index to the index of the last used entry= */ + s->max_refcount_table_index =3D i; +} + int qcow2_refcount_init(BlockDriverState *bs) { BDRVQcow2State *s =3D bs->opaque; @@ -111,6 +121,7 @@ int qcow2_refcount_init(BlockDriverState *bs) } for(i =3D 0; i < s->refcount_table_size; i++) be64_to_cpus(&s->refcount_table[i]); + update_max_refcount_table_index(s); } return 0; fail: @@ -439,6 +450,10 @@ static int alloc_refcount_block(BlockDriverState *bs, } =20 s->refcount_table[refcount_table_index] =3D new_block; + /* If there's a hole in s->refcount_table then it can happen + * that refcount_table_index < s->max_refcount_table_index */ + s->max_refcount_table_index =3D + MAX(s->max_refcount_table_index, refcount_table_index); =20 /* The new refcount block may be where the caller intended to put = its * data, so let it restart the search. */ @@ -580,6 +595,7 @@ static int alloc_refcount_block(BlockDriverState *bs, s->refcount_table =3D new_table; s->refcount_table_size =3D table_size; s->refcount_table_offset =3D table_offset; + update_max_refcount_table_index(s); =20 /* Free old table. */ qcow2_free_clusters(bs, old_table_offset, old_table_size * sizeof(uint= 64_t), @@ -2171,6 +2187,7 @@ write_refblocks: s->refcount_table =3D on_disk_reftable; s->refcount_table_offset =3D reftable_offset; s->refcount_table_size =3D reftable_size; + update_max_refcount_table_index(s); =20 return 0; =20 @@ -2383,7 +2400,11 @@ int qcow2_check_metadata_overlap(BlockDriverState *b= s, int ign, int64_t offset, } =20 if ((chk & QCOW2_OL_REFCOUNT_BLOCK) && s->refcount_table) { - for (i =3D 0; i < s->refcount_table_size; i++) { + unsigned last_entry =3D s->max_refcount_table_index; + assert(last_entry < s->refcount_table_size); + assert(last_entry + 1 =3D=3D s->refcount_table_size || + (s->refcount_table[last_entry + 1] & REFT_OFFSET_MASK) =3D= =3D 0); + for (i =3D 0; i <=3D last_entry; i++) { if ((s->refcount_table[i] & REFT_OFFSET_MASK) && overlaps_with(s->refcount_table[i] & REFT_OFFSET_MASK, s->cluster_size)) { @@ -2871,6 +2892,7 @@ int qcow2_change_refcount_order(BlockDriverState *bs,= int refcount_order, /* Now update the rest of the in-memory information */ old_reftable =3D s->refcount_table; s->refcount_table =3D new_reftable; + update_max_refcount_table_index(s); =20 s->refcount_bits =3D 1 << refcount_order; s->refcount_max =3D UINT64_C(1) << (s->refcount_bits - 1); diff --git a/block/qcow2.c b/block/qcow2.c index 96fb8a8f16..3e274bd1ba 100644 --- a/block/qcow2.c +++ b/block/qcow2.c @@ -2743,6 +2743,7 @@ static int make_completely_empty(BlockDriverState *bs) =20 s->refcount_table_offset =3D s->cluster_size; s->refcount_table_size =3D s->cluster_size / sizeof(uint64_t); + s->max_refcount_table_index =3D 0; =20 g_free(s->refcount_table); s->refcount_table =3D new_reftable; diff --git a/block/qcow2.h b/block/qcow2.h index 182341483a..f8aeb08794 100644 --- a/block/qcow2.h +++ b/block/qcow2.h @@ -251,6 +251,7 @@ typedef struct BDRVQcow2State { uint64_t *refcount_table; uint64_t refcount_table_offset; uint32_t refcount_table_size; + uint32_t max_refcount_table_index; /* Last used entry in refcount_tabl= e */ uint64_t free_cluster_index; uint64_t free_byte_offset; =20 --=20 2.11.0