From nobody Wed Apr 24 08:13:39 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=virtuozzo.com ARC-Seal: i=1; a=rsa-sha256; t=1557842093; cv=none; d=zoho.com; s=zohoarc; b=Z090V0pjpRVqaj1452meRwsblofQwgb65Cn41Q4anD7Dj4JlfrRe7+TOU+1ayXDPn0+XfoXr0yW7mP6nz3oBWxl3UNbNmePJ2EFjg8Ox9DI/APTqHY1f2+t6Bv4VfjXtFWiEr1EL/gCS8dXB79SwzVlldWZXyFwtGILARhRd1PQ= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1557842093; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results; bh=lWJPNZ81l0NINNTMZx7fq7AkA6QLGg3ax6o7wxhNWto=; b=Ay+dJOeHlzWP0co74ap9VP4z6YK4l0fkmndq5x5QeD8vBZ7vGWOaiffPbUCVpAuOpnOy/EfoyvLxDNgHoG/sYbN4rwACWTZV2IwOCFINzwVXdPm3afMefdUEse5SLuSlAWeToRtnChujtntOeIOSMSa6vX1giXCOcRSzyCI/cyQ= ARC-Authentication-Results: i=1; mx.zoho.com; dkim=fail; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (209.51.188.17 [209.51.188.17]) by mx.zohomail.com with SMTPS id 1557842092810789.4552711514526; Tue, 14 May 2019 06:54:52 -0700 (PDT) Received: from localhost ([127.0.0.1]:48547 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hQXtD-0004Fs-Cg for importer@patchew.org; Tue, 14 May 2019 09:54:43 -0400 Received: from eggs.gnu.org ([209.51.188.92]:58028) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hQXrk-0003PU-Rc for qemu-devel@nongnu.org; Tue, 14 May 2019 09:53:14 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hQXrj-0000vd-8E for qemu-devel@nongnu.org; Tue, 14 May 2019 09:53:12 -0400 Received: from mail-eopbgr150113.outbound.protection.outlook.com ([40.107.15.113]:48710 helo=EUR01-DB5-obe.outbound.protection.outlook.com) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1hQXrf-0000nf-08; Tue, 14 May 2019 09:53:07 -0400 Received: from VI1PR08MB4078.eurprd08.prod.outlook.com (20.178.127.92) by VI1PR08MB3117.eurprd08.prod.outlook.com (52.133.15.22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.1878.22; Tue, 14 May 2019 13:53:04 +0000 Received: from VI1PR08MB4078.eurprd08.prod.outlook.com ([fe80::8ce8:ffb4:fe9:4900]) by VI1PR08MB4078.eurprd08.prod.outlook.com ([fe80::8ce8:ffb4:fe9:4900%3]) with mapi id 15.20.1878.024; Tue, 14 May 2019 13:53:04 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=virtuozzo.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=lWJPNZ81l0NINNTMZx7fq7AkA6QLGg3ax6o7wxhNWto=; b=MeaQzZCscFpTTs9exInRf54ZPmtD1GxX+QlLxmqC4+1OD3zq5Y3vFAi4Anz3Nip++WEA85vgdjukZ0qPBYnqHzk6Vcp7GPZJ9htwuQ7Pzjp4mPdRSjnbATXlooRcGaPpV7sqWgnqvSoA+V7jE01GugQoqfiLyGzXhgtTF/N3QFA= From: Anton Nefedov To: "qemu-block@nongnu.org" Thread-Topic: [PATCH v13 1/1] qcow2: skip writing zero buffers to empty COW areas Thread-Index: AQHVClxaGdfW0rSLu0+JzTYeVhXcZw== Date: Tue, 14 May 2019 13:53:04 +0000 Message-ID: <20190514135248.34146-2-anton.nefedov@virtuozzo.com> References: <20190514135248.34146-1-anton.nefedov@virtuozzo.com> In-Reply-To: <20190514135248.34146-1-anton.nefedov@virtuozzo.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-clientproxiedby: HE1PR0901CA0064.eurprd09.prod.outlook.com (2603:10a6:3:45::32) To VI1PR08MB4078.eurprd08.prod.outlook.com (2603:10a6:803:e5::28) authentication-results: spf=none (sender IP is ) smtp.mailfrom=anton.nefedov@virtuozzo.com; x-ms-exchange-messagesentrepresentingtype: 1 x-mailer: git-send-email 2.17.1 x-originating-ip: [185.231.240.5] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 6b7491c0-2074-440b-9399-08d6d8737cc1 x-microsoft-antispam: BCL:0; PCL:0; RULEID:(2390118)(7020095)(4652040)(8989299)(4534185)(4627221)(201703031133081)(201702281549075)(8990200)(5600141)(711020)(4605104)(2017052603328)(7193020); SRVR:VI1PR08MB3117; x-ms-traffictypediagnostic: VI1PR08MB3117: x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:6430; x-forefront-prvs: 0037FD6480 x-forefront-antispam-report: SFV:NSPM; SFS:(10019020)(396003)(366004)(376002)(346002)(136003)(39840400004)(189003)(199004)(478600001)(316002)(68736007)(54906003)(86362001)(66066001)(1076003)(36756003)(5660300002)(6436002)(6116002)(2906002)(3846002)(6486002)(64756008)(66446008)(66476007)(73956011)(66946007)(66556008)(5640700003)(71190400001)(71200400001)(2501003)(446003)(44832011)(476003)(2616005)(486006)(11346002)(8936002)(81166006)(386003)(14454004)(99286004)(6506007)(53936002)(81156014)(107886003)(6512007)(4326008)(76176011)(52116002)(2351001)(8676002)(14444005)(256004)(102836004)(25786009)(7736002)(6916009)(186003)(50226002)(305945005)(26005)(21314003); DIR:OUT; SFP:1102; SCL:1; SRVR:VI1PR08MB3117; H:VI1PR08MB4078.eurprd08.prod.outlook.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; MX:1; A:1; Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; received-spf: None (protection.outlook.com: virtuozzo.com does not designate permitted sender hosts) x-ms-exchange-senderadcheck: 1 x-microsoft-antispam-message-info: vAqzbb5f/r5VdrwFaq7tz1BsCYvkYwJpw1r5A+Zt2rMt/L8F1QyH/MuX7v6cqNyjKOmqB+y7UxRBeK4bgKis7wj86ZrcaNe+QmxEICGrsrN5HGbSs/c5/PsGb7mc6fkoxCje+9nNa7wNEHehUPFPVyBl7lORbVh4DaymtV7vzC1JevBKx56TzMvmsYY0efIInpJGMQYnGSMC7AAZemkblr74htRLvODgIHN1nK2czyLq89WXj/UkU139br6Bzuo3Utfcbs+G6AzuJoD6SNJ3vGdPHqZ7jPw45SIwaY0cwKoG0t6Ry3vs44i8z/lA53XzAm5fMQbPmvRQYYYE6cgrKCjjvV6TJljNQlM01sbMG3h/xinNe9x/O9dtKXp+ymAfEGLuw3RJx/I1zjAWbPGBeV2jBa2aZeeD/LwTmiR2+Vg= Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: virtuozzo.com X-MS-Exchange-CrossTenant-Network-Message-Id: 6b7491c0-2074-440b-9399-08d6d8737cc1 X-MS-Exchange-CrossTenant-originalarrivaltime: 14 May 2019 13:53:04.7885 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 0bc7f26d-0264-416e-a6fc-8352af79c58f X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR08MB3117 X-detected-operating-system: by eggs.gnu.org: Windows 7 or 8 [fuzzy] X-Received-From: 40.107.15.113 Subject: [Qemu-devel] [PATCH v13 1/1] qcow2: skip writing zero buffers to empty COW areas X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: "kwolf@redhat.com" , Vladimir Sementsov-Ogievskiy , "berto@igalia.com" , Denis Lunev , "qemu-devel@nongnu.org" , "mreitz@redhat.com" , Anton Nefedov Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) If COW areas of the newly allocated clusters are zeroes on the backing image, efficient bdrv_write_zeroes(flags=3DBDRV_REQ_NO_FALLBACK) can be used on the whole cluster instead of writing explicit zero buffers later in perform_cow(). iotest 060: write to the discarded cluster does not trigger COW anymore. Use a backing image instead. Signed-off-by: Anton Nefedov --- qapi/block-core.json | 4 +- block/qcow2.h | 6 +++ block/qcow2-cluster.c | 2 +- block/qcow2.c | 93 +++++++++++++++++++++++++++++++++++++- block/trace-events | 1 + tests/qemu-iotests/060 | 7 ++- tests/qemu-iotests/060.out | 5 +- 7 files changed, 112 insertions(+), 6 deletions(-) diff --git a/qapi/block-core.json b/qapi/block-core.json index 7ccbfff9d0..3e4042be7f 100644 --- a/qapi/block-core.json +++ b/qapi/block-core.json @@ -3215,6 +3215,8 @@ # # @cor_write: a write due to copy-on-read (since 2.11) # +# @cluster_alloc_space: an allocation of file space for a cluster (since 4= .1) +# # Since: 2.9 ## { 'enum': 'BlkdebugEvent', 'prefix': 'BLKDBG', @@ -3233,7 +3235,7 @@ 'pwritev_rmw_tail', 'pwritev_rmw_after_tail', 'pwritev', 'pwritev_zero', 'pwritev_done', 'empty_image_prepare', 'l1_shrink_write_table', 'l1_shrink_free_l2_clusters', - 'cor_write'] } + 'cor_write', 'cluster_alloc_space'] } =20 ## # @BlkdebugInjectErrorOptions: diff --git a/block/qcow2.h b/block/qcow2.h index e62508d1ce..07b18a733b 100644 --- a/block/qcow2.h +++ b/block/qcow2.h @@ -398,6 +398,12 @@ typedef struct QCowL2Meta */ Qcow2COWRegion cow_end; =20 + /* + * Indicates that COW regions are already handled and do not require + * any more processing. + */ + bool skip_cow; + /** * The I/O vector with the data from the actual guest write request. * If non-NULL, this is meant to be merged together with the data diff --git a/block/qcow2-cluster.c b/block/qcow2-cluster.c index 974a4e8656..79d4651603 100644 --- a/block/qcow2-cluster.c +++ b/block/qcow2-cluster.c @@ -832,7 +832,7 @@ static int perform_cow(BlockDriverState *bs, QCowL2Meta= *m) assert(start->offset + start->nb_bytes <=3D end->offset); assert(!m->data_qiov || m->data_qiov->size =3D=3D data_bytes); =20 - if (start->nb_bytes =3D=3D 0 && end->nb_bytes =3D=3D 0) { + if ((start->nb_bytes =3D=3D 0 && end->nb_bytes =3D=3D 0) || m->skip_co= w) { return 0; } =20 diff --git a/block/qcow2.c b/block/qcow2.c index 8e024007db..e6b1293ddf 100644 --- a/block/qcow2.c +++ b/block/qcow2.c @@ -2120,6 +2120,11 @@ static bool merge_cow(uint64_t offset, unsigned byte= s, continue; } =20 + /* If COW regions are handled already, skip this too */ + if (m->skip_cow) { + continue; + } + /* The data (middle) region must be immediately after the * start region */ if (l2meta_cow_start(m) + m->cow_start.nb_bytes !=3D offset) { @@ -2145,6 +2150,80 @@ static bool merge_cow(uint64_t offset, unsigned byte= s, return false; } =20 +static bool is_unallocated(BlockDriverState *bs, int64_t offset, int64_t b= ytes) +{ + int64_t nr; + return !bytes || + (!bdrv_is_allocated_above(bs, NULL, offset, bytes, &nr) && nr =3D= =3D bytes); +} + +static bool is_zero_cow(BlockDriverState *bs, QCowL2Meta *m) +{ + /* + * This check is designed for optimization shortcut so it must be + * efficient. + * Instead of is_zero(), use is_unallocated() as it is faster (but not + * as accurate and can result in false negatives). + */ + return is_unallocated(bs, m->offset + m->cow_start.offset, + m->cow_start.nb_bytes) && + is_unallocated(bs, m->offset + m->cow_end.offset, + m->cow_end.nb_bytes); +} + +static int handle_alloc_space(BlockDriverState *bs, QCowL2Meta *l2meta) +{ + BDRVQcow2State *s =3D bs->opaque; + QCowL2Meta *m; + + if (!(s->data_file->bs->supported_zero_flags & BDRV_REQ_NO_FALLBACK)) { + return 0; + } + + if (bs->encrypted) { + return 0; + } + + for (m =3D l2meta; m !=3D NULL; m =3D m->next) { + int ret; + + if (!m->cow_start.nb_bytes && !m->cow_end.nb_bytes) { + continue; + } + + if (!is_zero_cow(bs, m)) { + continue; + } + + /* + * instead of writing zero COW buffers, + * efficiently zero out the whole clusters + */ + + ret =3D qcow2_pre_write_overlap_check(bs, 0, m->alloc_offset, + m->nb_clusters * s->cluster_si= ze, + true); + if (ret < 0) { + return ret; + } + + BLKDBG_EVENT(bs->file, BLKDBG_CLUSTER_ALLOC_SPACE); + ret =3D bdrv_co_pwrite_zeroes(s->data_file, m->alloc_offset, + m->nb_clusters * s->cluster_size, + BDRV_REQ_NO_FALLBACK); + if (ret < 0) { + if (ret !=3D -ENOTSUP && ret !=3D -EAGAIN) { + return ret; + } + continue; + } + + trace_qcow2_skip_cow(qemu_coroutine_self(), m->offset, m->nb_clust= ers); + m->skip_cow =3D true; + } + return 0; +} + static coroutine_fn int qcow2_co_pwritev(BlockDriverState *bs, uint64_t of= fset, uint64_t bytes, QEMUIOVector *qio= v, int flags) @@ -2225,24 +2304,34 @@ static coroutine_fn int qcow2_co_pwritev(BlockDrive= rState *bs, uint64_t offset, goto fail; } =20 + qemu_co_mutex_unlock(&s->lock); + + /* Try to efficiently initialize the physical space with zeroes */ + ret =3D handle_alloc_space(bs, l2meta); + if (ret < 0) { + qemu_co_mutex_lock(&s->lock); + goto fail; + } + /* If we need to do COW, check if it's possible to merge the * writing of the guest data together with that of the COW regions. * If it's not possible (or not necessary) then write the * guest data now. */ if (!merge_cow(offset, cur_bytes, &hd_qiov, l2meta)) { - qemu_co_mutex_unlock(&s->lock); BLKDBG_EVENT(bs->file, BLKDBG_WRITE_AIO); trace_qcow2_writev_data(qemu_coroutine_self(), cluster_offset + offset_in_cluster); ret =3D bdrv_co_pwritev(s->data_file, cluster_offset + offset_in_cluster, cur_bytes, &hd_qiov, 0); - qemu_co_mutex_lock(&s->lock); if (ret < 0) { + qemu_co_mutex_lock(&s->lock); goto fail; } } =20 + qemu_co_mutex_lock(&s->lock); + ret =3D qcow2_handle_l2meta(bs, &l2meta, true); if (ret) { goto fail; diff --git a/block/trace-events b/block/trace-events index 79ccd8d824..1e0653ce6d 100644 --- a/block/trace-events +++ b/block/trace-events @@ -68,6 +68,7 @@ qcow2_writev_done_part(void *co, int cur_bytes) "co %p cu= r_bytes %d" qcow2_writev_data(void *co, uint64_t offset) "co %p offset 0x%" PRIx64 qcow2_pwrite_zeroes_start_req(void *co, int64_t offset, int count) "co %p = offset 0x%" PRIx64 " count %d" qcow2_pwrite_zeroes(void *co, int64_t offset, int count) "co %p offset 0x%= " PRIx64 " count %d" +qcow2_skip_cow(void *co, uint64_t offset, int nb_clusters) "co %p offset 0= x%" PRIx64 " nb_clusters %d" =20 # qcow2-cluster.c qcow2_alloc_clusters_offset(void *co, uint64_t offset, int bytes) "co %p o= ffset 0x%" PRIx64 " bytes %d" diff --git a/tests/qemu-iotests/060 b/tests/qemu-iotests/060 index 89e911400c..b91d8321bb 100755 --- a/tests/qemu-iotests/060 +++ b/tests/qemu-iotests/060 @@ -150,10 +150,15 @@ $QEMU_IO -c "$OPEN_RO" -c "read -P 1 0 512" | _filter= _qemu_io echo echo "=3D=3D=3D Testing overlap while COW is in flight =3D=3D=3D" echo +BACKING_IMG=3D$TEST_IMG.base +TEST_IMG=3D$BACKING_IMG _make_test_img 1G + +$QEMU_IO -c 'write 0k 64k' "$BACKING_IMG" | _filter_qemu_io + # compat=3D0.10 is required in order to make the following discard actually # unallocate the sector rather than make it a zero sector - we want COW, a= fter # all. -IMGOPTS=3D'compat=3D0.10' _make_test_img 1G +IMGOPTS=3D'compat=3D0.10' _make_test_img -b "$BACKING_IMG" 1G # Write two clusters, the second one enforces creation of an L2 table after # the first data cluster. $QEMU_IO -c 'write 0k 64k' -c 'write 512M 64k' "$TEST_IMG" | _filter_qemu_= io diff --git a/tests/qemu-iotests/060.out b/tests/qemu-iotests/060.out index e42bf8c5a9..0f6b0658a1 100644 --- a/tests/qemu-iotests/060.out +++ b/tests/qemu-iotests/060.out @@ -97,7 +97,10 @@ read 512/512 bytes at offset 0 =20 =3D=3D=3D Testing overlap while COW is in flight =3D=3D=3D =20 -Formatting 'TEST_DIR/t.IMGFMT', fmt=3DIMGFMT size=3D1073741824 +Formatting 'TEST_DIR/t.IMGFMT.base', fmt=3DIMGFMT size=3D1073741824 +wrote 65536/65536 bytes at offset 0 +64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +Formatting 'TEST_DIR/t.IMGFMT', fmt=3DIMGFMT size=3D1073741824 backing_fil= e=3DTEST_DIR/t.IMGFMT.base wrote 65536/65536 bytes at offset 0 64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) wrote 65536/65536 bytes at offset 536870912 --=20 2.17.1