From nobody Fri May 3 10:08:46 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=dupond.be ARC-Seal: i=1; a=rsa-sha256; t=1685710142; cv=none; d=zohomail.com; s=zohoarc; b=nuwRNOZIe9JdnJjcfun5hsE7rKWCjt5yZLjPBQ9ELQB+3YuqA6+o0zG3/ZOmrAiW6ibDGApmogXUOb+aiBdqh9VX0DxCwHAf/JCkX1Z6QiKQr5t9AAwHRX+Yp8m7gpQ0CC8eyhYeHWwxmAz5y43VlfQKqzbJRN6n2hxhNhIsA8Q= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1685710142; h=Content-Transfer-Encoding:Cc:Date:From:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:Sender:Subject:To; bh=iLUc3eKVkILE/2K+nX4hTEpQEVLRgFe4D9YAajPooXE=; b=eg7hifVbvLzcOZekhfHpiCpj46CrV/AdraCeJ2gHLNYiTvthMQcMW6th540DFEtnvfBGNtoU6VHJ5RO97Rt7d4RzTzaY2gr9n3UVJdgdnAKtHFQg50Gyib5H14YylGye67Kyd23cIvIlWQfSEc1PnM+8PNhhrW6JXrRAtXFn/hg= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1685710142060426.2734045845183; Fri, 2 Jun 2023 05:49:02 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1q54Cm-0005EK-8t; Fri, 02 Jun 2023 08:48:32 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1q54Cd-0005Ds-UC for qemu-devel@nongnu.org; Fri, 02 Jun 2023 08:48:26 -0400 Received: from apollo.dupie.be ([2001:bc8:3f2a:101::1]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1q54CZ-00049U-QK for qemu-devel@nongnu.org; Fri, 02 Jun 2023 08:48:23 -0400 Received: from localhost.localdomain (unknown [IPv6:2a02:a03f:eaf7:ff01:cc6b:6666:e19c:b63f]) by apollo.dupie.be (Postfix) with ESMTPSA id 52AF91520DD7; Fri, 2 Jun 2023 14:48:13 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=dupond.be; s=dkim; t=1685710094; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding; bh=iLUc3eKVkILE/2K+nX4hTEpQEVLRgFe4D9YAajPooXE=; b=K88mb9Ftj7hbmHE+h2jS46u0KtG/1hijDIQRpvyOuKoXdtMN1x2alB6YcGP4IoV9IhJYUj 977gFFu+wxH6ZprKf+79e0P7UW9OP59Jo0iO3mj8sAuglXOL2YCTkka6ZJalp4SPFHypB0 lmWdhJ1Ft+5My02rycxNJ729cSzW3TqC08yZ+wQ5sjEdeNxDtPapIdMEn0IfbWOOFeEDcw 9kbjiQgGkH/bMZesXT5547n+fQMkq+1M24wPhSF/O2uJKDJmo+MGSuOPKlK7yk8t2IlQ90 vqN58fzVfLFfeY8Lt24gvNmWMGpQXm7w5SM7i89le+EiMrjz4PetfOGUACILCQ== From: Jean-Louis Dupond To: qemu-devel@nongnu.org, kwolf@redhat.com, hreitz@redhat.com Cc: Jean-Louis Dupond Subject: [PATCH v2] qcow2: add discard-no-unref option Date: Fri, 2 Jun 2023 14:47:48 +0200 Message-Id: <20230602124747.1544077-1-jean-louis@dupond.be> X-Mailer: git-send-email 2.40.1 MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2001:bc8:3f2a:101::1; envelope-from=jean-louis@dupond.be; helo=apollo.dupie.be X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @dupond.be) X-ZM-MESSAGEID: 1685710144758100005 Content-Type: text/plain; charset="utf-8" When we for example have a sparse qcow2 image and discard: unmap is enabled, there can be a lot of fragmentation in the image after some time. Especiall= y on VM's that do a lot of writes/deletes. This causes the qcow2 image to grow even over 110% of its virtual size, because the free gaps in the image get too small to allocate new continuous clusters. So it allocates new space at the end of the image. Disabling discard is not an option, as discard is needed to keep the incremental backup size as low as possible. Without discard, the incremental backups would become large, as qemu thinks it's just dirty blocks but it doesn't know the blocks are unneeded. So we need to avoid fragmentation but also 'empty' the unneeded blocks in the image to have a small incremental backup. In addition, we also want to send the discards further down the stack, so the underlying blocks are still discarded. Therefor we introduce a new qcow2 option "discard-no-unref". When setting this option to true, discards will no longer have the qcow2 driver relinquish cluster allocations. Other than that, the request is handled as normal: All clusters in range are marked as zero, and, if pass-discard-request is true, it is passed further down the stack. The only difference is that the now-zero clusters are preallocated instead of being unallocated. This will avoid fragmentation on the qcow2 image. Fixes: https://gitlab.com/qemu-project/qemu/-/issues/1621 Signed-off-by: Jean-Louis Dupond --- block/qcow2-cluster.c | 32 ++++++++++++++++++++++++++++---- block/qcow2.c | 18 ++++++++++++++++++ block/qcow2.h | 3 +++ qapi/block-core.json | 12 ++++++++++++ qemu-options.hx | 12 ++++++++++++ 5 files changed, 73 insertions(+), 4 deletions(-) diff --git a/block/qcow2-cluster.c b/block/qcow2-cluster.c index 39cda7f907..1f130c6ab9 100644 --- a/block/qcow2-cluster.c +++ b/block/qcow2-cluster.c @@ -1894,6 +1894,7 @@ again: return 0; } =20 + /* * This discards as many clusters of nb_clusters as possible at once (i.e. * all clusters in the same L2 slice) and returns the number of discarded @@ -1925,6 +1926,9 @@ static int discard_in_l2_slice(BlockDriverState *bs, = uint64_t offset, uint64_t new_l2_bitmap =3D old_l2_bitmap; QCow2ClusterType cluster_type =3D qcow2_get_cluster_type(bs, old_l2_entry); + bool keep_reference =3D (cluster_type !=3D QCOW2_CLUSTER_COMPRESSE= D) && + (s->discard_no_unref && + type =3D=3D QCOW2_DISCARD_REQUEST); =20 /* * If full_discard is true, the cluster should not read back as ze= roes, @@ -1943,10 +1947,22 @@ static int discard_in_l2_slice(BlockDriverState *bs= , uint64_t offset, new_l2_entry =3D new_l2_bitmap =3D 0; } else if (bs->backing || qcow2_cluster_is_allocated(cluster_type)= ) { if (has_subclusters(s)) { - new_l2_entry =3D 0; + if (keep_reference) { + new_l2_entry =3D old_l2_entry; + } else { + new_l2_entry =3D 0; + } new_l2_bitmap =3D QCOW_L2_BITMAP_ALL_ZEROES; } else { - new_l2_entry =3D s->qcow_version >=3D 3 ? QCOW_OFLAG_ZERO = : 0; + if (s->qcow_version >=3D 3) { + if (keep_reference) { + new_l2_entry |=3D QCOW_OFLAG_ZERO; + } else { + new_l2_entry =3D QCOW_OFLAG_ZERO; + } + } else { + new_l2_entry =3D 0; + } } } =20 @@ -1960,8 +1976,16 @@ static int discard_in_l2_slice(BlockDriverState *bs,= uint64_t offset, if (has_subclusters(s)) { set_l2_bitmap(s, l2_slice, l2_index + i, new_l2_bitmap); } - /* Then decrease the refcount */ - qcow2_free_any_cluster(bs, old_l2_entry, type); + if (!keep_reference) { + /* Then decrease the refcount */ + qcow2_free_any_cluster(bs, old_l2_entry, type); + } else if (s->discard_passthrough[type] && + (cluster_type =3D=3D QCOW2_CLUSTER_NORMAL || + cluster_type =3D=3D QCOW2_CLUSTER_ZERO_ALLOC)) { + /* If we keep the reference, pass on the discard still */ + bdrv_pdiscard(s->data_file, new_l2_entry & L2E_OFFSET_MASK, + s->cluster_size); + } } =20 qcow2_cache_put(s->l2_table_cache, (void **) &l2_slice); diff --git a/block/qcow2.c b/block/qcow2.c index 7f3948360d..e23edd48c2 100644 --- a/block/qcow2.c +++ b/block/qcow2.c @@ -682,6 +682,7 @@ static const char *const mutable_opts[] =3D { QCOW2_OPT_DISCARD_REQUEST, QCOW2_OPT_DISCARD_SNAPSHOT, QCOW2_OPT_DISCARD_OTHER, + QCOW2_OPT_DISCARD_NO_UNREF, QCOW2_OPT_OVERLAP, QCOW2_OPT_OVERLAP_TEMPLATE, QCOW2_OPT_OVERLAP_MAIN_HEADER, @@ -726,6 +727,11 @@ static QemuOptsList qcow2_runtime_opts =3D { .type =3D QEMU_OPT_BOOL, .help =3D "Generate discard requests when other clusters are f= reed", }, + { + .name =3D QCOW2_OPT_DISCARD_NO_UNREF, + .type =3D QEMU_OPT_BOOL, + .help =3D "Do not unreference discarded clusters", + }, { .name =3D QCOW2_OPT_OVERLAP, .type =3D QEMU_OPT_STRING, @@ -969,6 +975,7 @@ typedef struct Qcow2ReopenState { bool use_lazy_refcounts; int overlap_check; bool discard_passthrough[QCOW2_DISCARD_MAX]; + bool discard_no_unref; uint64_t cache_clean_interval; QCryptoBlockOpenOptions *crypto_opts; /* Disk encryption runtime optio= ns */ } Qcow2ReopenState; @@ -1140,6 +1147,15 @@ static int qcow2_update_options_prepare(BlockDriverS= tate *bs, r->discard_passthrough[QCOW2_DISCARD_OTHER] =3D qemu_opt_get_bool(opts, QCOW2_OPT_DISCARD_OTHER, false); =20 + r->discard_no_unref =3D qemu_opt_get_bool(opts, QCOW2_OPT_DISCARD_NO_U= NREF, + false); + if (r->discard_no_unref && s->qcow_version < 3) { + error_setg(errp, + "discard-no-unref is only supported since qcow2 version= 3"); + ret =3D -EINVAL; + goto fail; + } + switch (s->crypt_method_header) { case QCOW_CRYPT_NONE: if (encryptfmt) { @@ -1220,6 +1236,8 @@ static void qcow2_update_options_commit(BlockDriverSt= ate *bs, s->discard_passthrough[i] =3D r->discard_passthrough[i]; } =20 + s->discard_no_unref =3D r->discard_no_unref; + if (s->cache_clean_interval !=3D r->cache_clean_interval) { cache_clean_timer_del(bs); s->cache_clean_interval =3D r->cache_clean_interval; diff --git a/block/qcow2.h b/block/qcow2.h index 4f67eb912a..ea9adb5706 100644 --- a/block/qcow2.h +++ b/block/qcow2.h @@ -133,6 +133,7 @@ #define QCOW2_OPT_DISCARD_REQUEST "pass-discard-request" #define QCOW2_OPT_DISCARD_SNAPSHOT "pass-discard-snapshot" #define QCOW2_OPT_DISCARD_OTHER "pass-discard-other" +#define QCOW2_OPT_DISCARD_NO_UNREF "discard-no-unref" #define QCOW2_OPT_OVERLAP "overlap-check" #define QCOW2_OPT_OVERLAP_TEMPLATE "overlap-check.template" #define QCOW2_OPT_OVERLAP_MAIN_HEADER "overlap-check.main-header" @@ -385,6 +386,8 @@ typedef struct BDRVQcow2State { =20 bool discard_passthrough[QCOW2_DISCARD_MAX]; =20 + bool discard_no_unref; + int overlap_check; /* bitmask of Qcow2MetadataOverlap values */ bool signaled_corruption; =20 diff --git a/qapi/block-core.json b/qapi/block-core.json index 98d9116dae..7e9446e49b 100644 --- a/qapi/block-core.json +++ b/qapi/block-core.json @@ -3478,6 +3478,17 @@ # @pass-discard-other: whether discard requests for the data source # should be issued on other occasions where a cluster gets freed # +# @discard-no-unref: when enabled, discards from the guest will not cause +# cluster allocations to be relinquished. This prevents qcow2 fragment= ation +# that would be caused by such discards. Besides potential +# performance degradation, such fragmentation can lead to increased +# allocation of clusters past the end of the image file, +# resulting in image files whose file length can grow much larger +# than their guest disk size would suggest. +# If image file length is of concern (e.g. when storing qcow2 +# images directly on block devices), you should consider enabling +# this option. (since 8.1) +# # @overlap-check: which overlap checks to perform for writes to the # image, defaults to 'cached' (since 2.2) # @@ -3516,6 +3527,7 @@ '*pass-discard-request': 'bool', '*pass-discard-snapshot': 'bool', '*pass-discard-other': 'bool', + '*discard-no-unref': 'bool', '*overlap-check': 'Qcow2OverlapChecks', '*cache-size': 'int', '*l2-cache-size': 'int', diff --git a/qemu-options.hx b/qemu-options.hx index b37eb9662b..b57489d7ca 100644 --- a/qemu-options.hx +++ b/qemu-options.hx @@ -1431,6 +1431,18 @@ SRST issued on other occasions where a cluster gets freed (on/off; default: off) =20 + ``discard-no-unref`` + When enabled, discards from the guest will not cause cluster + allocations to be relinquished. This prevents qcow2 fragmentat= ion + that would be caused by such discards. Besides potential + performance degradation, such fragmentation can lead to increa= sed + allocation of clusters past the end of the image file, + resulting in image files whose file length can grow much larger + than their guest disk size would suggest. + If image file length is of concern (e.g. when storing qcow2 + images directly on block devices), you should consider enabling + this option. + ``overlap-check`` Which overlap checks to perform for writes to the image (none/constant/cached/all; default: cached). For details or --=20 2.40.1