From nobody Mon Feb 9 14:34:13 2026 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) client-ip=209.132.183.28; envelope-from=libvir-list-bounces@redhat.com; helo=mx1.redhat.com; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com Return-Path: Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by mx.zohomail.com with SMTPS id 1504766278091838.6039350193837; Wed, 6 Sep 2017 23:37:58 -0700 (PDT) Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 3AA94DF0E; Thu, 7 Sep 2017 06:37:56 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.21]) by smtp.corp.redhat.com (Postfix) with ESMTPS id CF2A06031A; Thu, 7 Sep 2017 06:37:55 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id 46CCC3FC73; Thu, 7 Sep 2017 06:37:54 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id v876bqXe020493 for ; Thu, 7 Sep 2017 02:37:52 -0400 Received: by smtp.corp.redhat.com (Postfix) id AE0D9779D1; Thu, 7 Sep 2017 06:37:52 +0000 (UTC) Received: from mx1.redhat.com (ext-mx04.extmail.prod.ext.phx2.redhat.com [10.5.110.28]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 620B180AB9; Thu, 7 Sep 2017 06:37:49 +0000 (UTC) Received: from mx1.ioa.cn (mx0.ioa.cn [103.231.69.96]) by mx1.redhat.com (Postfix) with ESMTP id D6D0A8046E; Thu, 7 Sep 2017 06:37:43 +0000 (UTC) Received: by xxx.smtp.chinacmail.com (Postfix, from userid 12301) id 02BB0C05D4; Thu, 7 Sep 2017 14:07:07 +0800 (CST) Received: from host-172-16-90-85.localdomain (unknown [221.228.240.122]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by xxx.smtp.chinacmail.com (Postfix) with ESMTPSA id C7EB2C041E; Thu, 7 Sep 2017 14:07:06 +0800 (CST) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com 3AA94DF0E Authentication-Results: ext-mx06.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=huayun.com Authentication-Results: ext-mx06.extmail.prod.ext.phx2.redhat.com; spf=fail smtp.mailfrom=libvir-list-bounces@redhat.com DKIM-Filter: OpenDKIM Filter v2.11.0 mx1.redhat.com 3AA94DF0E DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com D6D0A8046E Authentication-Results: ext-mx04.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=huayun.com Authentication-Results: ext-mx04.extmail.prod.ext.phx2.redhat.com; spf=none smtp.mailfrom=liuqing@huayun.com DKIM-Filter: OpenDKIM Filter v2.11.0 mx1.redhat.com D6D0A8046E X-HY-TRUST: YES X-HY-SID: 17182.103.1504764452.658990 X-HY-SPAM: NO From: Liu Qing To: libvir-list@redhat.com Date: Thu, 7 Sep 2017 14:09:09 +0800 Message-Id: <1504764550-14433-2-git-send-email-liuqing@huayun.com> In-Reply-To: <1504764550-14433-1-git-send-email-liuqing@huayun.com> References: <1504764550-14433-1-git-send-email-liuqing@huayun.com> X-Greylist: Delayed for 00:30:07 by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.28]); Thu, 07 Sep 2017 06:37:47 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.28]); Thu, 07 Sep 2017 06:37:47 +0000 (UTC) for IP:'103.231.69.96' DOMAIN:'mx0.ioa.cn' HELO:'mx1.ioa.cn' FROM:'liuqing@huayun.com' RCPT:'' X-RedHat-Spam-Score: 0 () 103.231.69.96 mx0.ioa.cn 103.231.69.96 mx0.ioa.cn X-Scanned-By: MIMEDefang 2.78 on 10.5.110.28 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 X-loop: libvir-list@redhat.com Cc: Liu Qing Subject: [libvirt] [PATCH 1/2] conf, docs: Add qcow2 cache configuration support X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.30]); Thu, 07 Sep 2017 06:37:56 +0000 (UTC) X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Type: text/plain; charset="utf-8" Random write IOPS will drop dramatically if qcow2 l2 cache could not cover the whole disk. This patch gives libvirt user a chance to adjust the qcow2 cache configuration. Three new qcow2 driver parameters are added. They are l2-cache-size, refcount-cache-size and cache-clean-interval. The following are from qcow2-cache.txt. The amount of virtual disk that can be mapped by the L2 and refcount caches (in bytes) is: disk_size =3D l2_cache_size * cluster_size / 8 disk_size =3D refcount_cache_size * cluster_size * 8 / refcount_bits The parameter "cache-clean-interval" defines an interval (in seconds). All cache entries that haven't been accessed during that interval are removed from memory. Signed-off-by: Liu Qing --- a) add new element qcow2 as drive's child, if any qcow2 specific configuration exists. b) merge the three test cases to one. c) update docs. cache size only support bytes. d) move the parameters checking to virDomainDiskDefParseValidate=20 docs/formatdomain.html.in | 43 ++++++++++ docs/schemas/domaincommon.rng | 35 ++++++++ src/conf/domain_conf.c | 95 ++++++++++++++++++= +++- src/qemu/qemu_command.c | 6 ++ src/util/virstoragefile.c | 3 + src/util/virstoragefile.h | 6 ++ .../qemuxml2argv-disk-drive-qcow2-cache.xml | 43 ++++++++++ .../qemuxml2xmlout-disk-drive-qcow2-cache.xml | 43 ++++++++++ tests/qemuxml2xmltest.c | 1 + 9 files changed, 274 insertions(+), 1 deletion(-) create mode 100644 tests/qemuxml2argvdata/qemuxml2argv-disk-drive-qcow2-ca= che.xml create mode 100644 tests/qemuxml2xmloutdata/qemuxml2xmlout-disk-drive-qcow= 2-cache.xml diff --git a/docs/formatdomain.html.in b/docs/formatdomain.html.in index 8ca7637..245d5c4 100644 --- a/docs/formatdomain.html.in +++ b/docs/formatdomain.html.in @@ -3036,6 +3036,49 @@ set. (Since 3.5.0) + The drive element may contain a qcow2 sub element, wh= ich + allows to specifying further details related to qcow2 driver type. + Since 3.8.0 +
    +
  • + The optional l2_cache_size attribute controls how= much + memory will be consumed by qcow2 l2 table cache in bytes. This + option is only valid when the driver type is qcow2. The default + size is 2097152. The amount of virtual disk that can be mapped= by + the L2 caches (in bytes) is: + disk_size =3D l2_cache_size * cluster_size / 8 + Since 3.8.0 + + In general you should leave this option alone, unless you + are very certain you know what you are doing. +
    • +
  • + The optional refcount_cache_size attribute contro= ls + how much memory will be consumed by qcow2 reference count table + cache in bytes. This option is only valid when the driver type= is + qcow2. The default size is 262144. The amount of virtual disk = that + can be mapped by the refcount caches (in bytes) is: + disk_size =3D refcount_cache_size * cluster_size * 8 / refco= unt_bits + Since 3.8.0 + + In general you should leave this option alone, unless you + are very certain you know what you are doing. +
    • +
  • + The optional cache_clean_interval attribute defin= es + an interval (in seconds). All cache entries that haven't been + accessed during that interval are removed from memory. This op= tion + is only valid when the driver type is qcow2. The default + value is 0, which disables this feature. If the interval is too + short, it will cause frequent cache write back and load, which + impact performance. If the interval is too long, unused cache + will still consume memory until it's been written back to disk. + Since 3.8.0 + + In general you should leave this option alone, unless you + are very certain you know what you are doing. +
    • +
backenddomain
The optional backenddomain element allows specifyin= g a diff --git a/docs/schemas/domaincommon.rng b/docs/schemas/domaincommon.rng index 06c5a91..ec44e58 100644 --- a/docs/schemas/domaincommon.rng +++ b/docs/schemas/domaincommon.rng @@ -1762,6 +1762,23 @@ + + + + + + + + + + + + + + + @@ -1800,6 +1817,9 @@ + + + @@ -1895,6 +1915,21 @@ + + + + + + + + + + + + + + + diff --git a/src/conf/domain_conf.c b/src/conf/domain_conf.c index 2fc1fc3..c7d4b9b 100644 --- a/src/conf/domain_conf.c +++ b/src/conf/domain_conf.c @@ -5673,6 +5673,28 @@ virDomainDeviceLoadparmIsValid(const char *loadparm) =20 =20 static void +virDoaminQcow2CacheOptionsFormat(virBufferPtr buf, + virDomainDiskDefPtr def) +{ + virBuffer qcow2Buff =3D VIR_BUFFER_INITIALIZER; + if (def->src->l2_cache_size > 0) + virBufferAsprintf(&qcow2Buff, " l2_cache_size=3D'%llu'", + def->src->l2_cache_size); + if (def->src->refcount_cache_size > 0) + virBufferAsprintf(&qcow2Buff, " refcount_cache_size=3D'%llu'", + def->src->refcount_cache_size); + if (def->src->cache_clean_interval > 0) + virBufferAsprintf(&qcow2Buff, " cache_clean_interval=3D'%llu'", + def->src->cache_clean_interval); + + if (virBufferUse(&qcow2Buff)) { + virBufferAddLit(buf, "\n"); + } +} + +static void virDomainVirtioOptionsFormat(virBufferPtr buf, virDomainVirtioOptionsPtr virtio) { @@ -8511,15 +8533,69 @@ virDomainDiskDefParseValidate(const virDomainDiskDe= f *def) } } =20 + if (def->src->format !=3D VIR_STORAGE_FILE_QCOW2 && + (def->src->l2_cache_size > 0 || def->src->refcount_cache_size > 0 = || + def->src->cache_clean_interval > 0)) { + virReportError(VIR_ERR_XML_ERROR, + _("Setting l2_cache_size, refcount_cache_size, " + "cache_clean_interval is not allowed for types " + "other than QCOW2")); + return -1; + } + return 0; } =20 =20 static int +virDomainDiskDefQcow2ParseXML(virDomainDiskDefPtr def, + xmlNodePtr cur) +{ + char *tmp =3D NULL; + int ret =3D -1; + + if ((tmp =3D virXMLPropString(cur, "l2_cache_size")) && + (virStrToLong_ullp(tmp, NULL, 10, &def->src->l2_cache_size) < 0)) { + virReportError(VIR_ERR_XML_ERROR, + _("Invalid l2_cache_size attribute in disk " + "driver element: %s"), tmp); + goto cleanup; + } + VIR_FREE(tmp); + + if ((tmp =3D virXMLPropString(cur, "refcount_cache_size")) && + (virStrToLong_ullp(tmp, NULL, 10, &def->src->refcount_cache_size) = < 0)) { + virReportError(VIR_ERR_XML_ERROR, + _("Invalid refcount_cache_size attribute in disk " + "driver element: %s"), tmp); + goto cleanup; + } + VIR_FREE(tmp); + + if ((tmp =3D virXMLPropString(cur, "cache_clean_interval")) && + (virStrToLong_ullp(tmp, NULL, 10, &def->src->cache_clean_interval)= < 0)) { + virReportError(VIR_ERR_XML_ERROR, + _("Invalid cache_clean_interval attribute in " + "disk driver element: %s"), tmp); + goto cleanup; + } + VIR_FREE(tmp); + + ret =3D 0; + + cleanup: + VIR_FREE(tmp); + + return ret; +} + + +static int virDomainDiskDefDriverParseXML(virDomainDiskDefPtr def, xmlNodePtr cur) { char *tmp =3D NULL; + xmlNodePtr child; int ret =3D -1; =20 def->src->driverName =3D virXMLPropString(cur, "name"); @@ -8622,6 +8698,12 @@ virDomainDiskDefDriverParseXML(virDomainDiskDefPtr d= ef, } VIR_FREE(tmp); =20 + for (child =3D cur->children; child !=3D NULL; child =3D child->next) { + if (virXMLNodeNameEqual(child, "qcow2") && + virDomainDiskDefQcow2ParseXML(def, child) < 0) { + goto cleanup; + } + } ret =3D 0; =20 cleanup: @@ -21908,7 +21990,18 @@ virDomainDiskDefFormat(virBufferPtr buf, if (virBufferUse(&driverBuf)) { virBufferAddLit(buf, "\n"); + + if (def->src->l2_cache_size =3D=3D 0 && + def->src->refcount_cache_size =3D=3D 0 && + def->src->cache_clean_interval =3D=3D 0) { + virBufferAddLit(buf, "/>\n"); + } else { + virBufferAddLit(buf, ">\n"); + virBufferAdjustIndent(buf, 2); + virDoaminQcow2CacheOptionsFormat(buf, def); + virBufferAdjustIndent(buf, -2); + virBufferAddLit(buf, "\n"); + } } =20 if (def->src->auth) { diff --git a/src/qemu/qemu_command.c b/src/qemu/qemu_command.c index ed8cb6e..391aaba 100644 --- a/src/qemu/qemu_command.c +++ b/src/qemu/qemu_command.c @@ -1433,6 +1433,12 @@ qemuBuildDriveSourceStr(virDomainDiskDefPtr disk, qemuformat =3D "luks"; virBufferAsprintf(buf, "format=3D%s,", qemuformat); } + if (disk->src->l2_cache_size > 0) + virBufferAsprintf(buf, "l2-cache-size=3D%llu,", disk->src->l2_cach= e_size); + if (disk->src->refcount_cache_size > 0) + virBufferAsprintf(buf, "refcount-cache-size=3D%llu,", disk->src->r= efcount_cache_size); + if (disk->src->cache_clean_interval > 0) + virBufferAsprintf(buf, "cache-clean-interval=3D%llu,", disk->src->= cache_clean_interval); =20 ret =3D 0; =20 diff --git a/src/util/virstoragefile.c b/src/util/virstoragefile.c index fbc8245..c685331 100644 --- a/src/util/virstoragefile.c +++ b/src/util/virstoragefile.c @@ -2038,6 +2038,9 @@ virStorageSourceCopy(const virStorageSource *src, ret->physical =3D src->physical; ret->readonly =3D src->readonly; ret->shared =3D src->shared; + ret->l2_cache_size =3D src->l2_cache_size; + ret->refcount_cache_size =3D src->refcount_cache_size; + ret->cache_clean_interval =3D src->cache_clean_interval; =20 /* storage driver metadata are not copied */ ret->drv =3D NULL; diff --git a/src/util/virstoragefile.h b/src/util/virstoragefile.h index 6c388b1..9b5a5f3 100644 --- a/src/util/virstoragefile.h +++ b/src/util/virstoragefile.h @@ -280,6 +280,12 @@ struct _virStorageSource { /* metadata that allows identifying given storage source */ char *nodeformat; /* name of the format handler object */ char *nodestorage; /* name of the storage object */ + + unsigned long long l2_cache_size; /* qcow2 l2 cache size */ + /* qcow2 reference count table cache size */ + unsigned long long refcount_cache_size; + /* clean unused cache entries interval */ + unsigned long long cache_clean_interval; }; =20 =20 diff --git a/tests/qemuxml2argvdata/qemuxml2argv-disk-drive-qcow2-cache.xml= b/tests/qemuxml2argvdata/qemuxml2argv-disk-drive-qcow2-cache.xml new file mode 100644 index 0000000..3f464db --- /dev/null +++ b/tests/qemuxml2argvdata/qemuxml2argv-disk-drive-qcow2-cache.xml @@ -0,0 +1,43 @@ + + QEMUGuest1 + c7a5fdbd-edaf-9455-926a-d65c16db1809 + 219136 + 219136 + 1 + + hvm + + + + destroy + restart + destroy + + /usr/bin/qemu-system-i686 + + + + + + +
+ + + + + + +
+ + +
+ + +
+ + + + + + + diff --git a/tests/qemuxml2xmloutdata/qemuxml2xmlout-disk-drive-qcow2-cache= .xml b/tests/qemuxml2xmloutdata/qemuxml2xmlout-disk-drive-qcow2-cache.xml new file mode 100644 index 0000000..3f464db --- /dev/null +++ b/tests/qemuxml2xmloutdata/qemuxml2xmlout-disk-drive-qcow2-cache.xml @@ -0,0 +1,43 @@ + + QEMUGuest1 + c7a5fdbd-edaf-9455-926a-d65c16db1809 + 219136 + 219136 + 1 + + hvm + + + + destroy + restart + destroy + + /usr/bin/qemu-system-i686 + + + + + + +
+ + + + + + +
+ + +
+ + +
+ + + + + + + diff --git a/tests/qemuxml2xmltest.c b/tests/qemuxml2xmltest.c index 97ff36c..1ffb831 100644 --- a/tests/qemuxml2xmltest.c +++ b/tests/qemuxml2xmltest.c @@ -461,6 +461,7 @@ mymain(void) DO_TEST("disk-drive-cache-v2-none", NONE); DO_TEST("disk-drive-cache-directsync", NONE); DO_TEST("disk-drive-cache-unsafe", NONE); + DO_TEST("disk-drive-qcow2-cache", NONE); DO_TEST("disk-drive-network-nbd", NONE); DO_TEST("disk-drive-network-nbd-export", NONE); DO_TEST("disk-drive-network-nbd-ipv6", NONE); --=20 1.8.3.1 -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list