From nobody Mon Feb 9 10:32:36 2026 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 170.10.129.124 as permitted sender) client-ip=170.10.129.124; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-delivery-124.mimecast.com; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1677768013; cv=none; d=zohomail.com; s=zohoarc; b=LfA+Yx8Z73AqPYVHYhv701ZaP5UYn5MGtNhPkK/njLFcmLPZ5rUMlsY2w0/H9Z2x80C2Sq0RiaB1nhyn/a6V3/nnyJDKTQvUGg2Kn2vMoGVCuHFkNzlMl7OvnrKMDieLj/N6Pm6a9MGiAl1FurCiIYQoQeMV4MAG8uSD1FSYSiI= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1677768013; h=Content-Type:Content-Transfer-Encoding:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=NtGd6lsyFCzI8tTZz1iFhrcCJuRhqa5BZgJlXBhchsk=; b=cSZ9GCm/blY0+1k04UvfpQxo+kc3GLq59+BIP9m/p8dy44QZSKl4jCH6zaWt3KxDdDlIS1R0APncNrWDVYfjXDEHGTr9syXqQvrEclaxOBLkf67Untp8GtbLPsV+Ogx19Fd+q24XOnqejMO/T3wMCaSItbJntjejgXDWYIxm6ds= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by mx.zohomail.com with SMTPS id 1677768013283838.2425665864658; Thu, 2 Mar 2023 06:40:13 -0800 (PST) Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-219-s6pOHhrBPlCRiLiDh_5U0g-1; Thu, 02 Mar 2023 09:39:01 -0500 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.rdu2.redhat.com [10.11.54.1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 0124E8057CC; Thu, 2 Mar 2023 14:38:31 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (unknown [10.30.29.100]) by smtp.corp.redhat.com (Postfix) with ESMTP id DF70B40C83B6; Thu, 2 Mar 2023 14:38:30 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (localhost [IPv6:::1]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id BB8651948672; Thu, 2 Mar 2023 14:38:30 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.rdu2.redhat.com [10.11.54.7]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id 8093E19452D0 for ; Thu, 2 Mar 2023 14:38:29 +0000 (UTC) Received: by smtp.corp.redhat.com (Postfix) id 645E4145888F; Thu, 2 Mar 2023 14:38:29 +0000 (UTC) Received: from speedmetal.lan (unknown [10.45.242.7]) by smtp.corp.redhat.com (Postfix) with ESMTP id B6957140EBF6 for ; Thu, 2 Mar 2023 14:38:28 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1677768012; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=NtGd6lsyFCzI8tTZz1iFhrcCJuRhqa5BZgJlXBhchsk=; b=LfLdI1b4Pr0VKLZqWRVBAFFGwxfu9NkeAMzcxUPBsojxEZvg9QzWnPPpZCrfwFCwiMkcF0 Cjo3L36lhvY2JLM9O/cLL8mtim6bgtQG664zbsNpMaMgQuTqymyvp7/RJ6tWhxkB/UzzEI VVad1WcU0kaN0W6lbiIq25XKB6O+nTk= X-MC-Unique: s6pOHhrBPlCRiLiDh_5U0g-1 X-Original-To: libvir-list@listman.corp.redhat.com From: Peter Krempa To: libvir-list@redhat.com Subject: [PATCH RFC 20/21] conf: Add possibility to configure multiple iothreads per disk Date: Thu, 2 Mar 2023 15:38:02 +0100 Message-Id: In-Reply-To: References: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.7 X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libvir-list-bounces@redhat.com Sender: "libvir-list" X-Scanned-By: MIMEDefang 3.1 on 10.11.54.1 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1677768014141100001 Content-Type: text/plain; charset="utf-8" Introduce a new sub-element of disk's which will allow configuring multiple iothreads and also map them to specific virt-queues of virtio devices. Signed-off-by: Peter Krempa --- docs/formatdomain.rst | 23 +++++- src/conf/domain_conf.c | 76 +++++++++++++++++++ src/conf/domain_conf.h | 14 ++++ src/conf/domain_validate.c | 8 ++ src/conf/schemas/domaincommon.rng | 47 +++++++++--- .../iothreads-disk.x86_64-latest.args | 13 +++- tests/qemuxml2argvdata/iothreads-disk.xml | 25 +++++- .../iothreads-disk.x86_64-latest.xml | 26 ++++++- 8 files changed, 211 insertions(+), 21 deletions(-) diff --git a/docs/formatdomain.rst b/docs/formatdomain.rst index d831c1e527..4940f3d857 100644 --- a/docs/formatdomain.rst +++ b/docs/formatdomain.rst @@ -3233,7 +3233,28 @@ paravirtualized driver is specified via the ``disk``= element. assigned to the same IOThread and are numbered from 1 to the domain iothreads value. Available for a disk device ``target`` configured t= o use "virtio" ``bus`` and "pci" or "ccw" ``address`` types. :since:`Since= 1.2.8 - (QEMU 2.1)` + (QEMU 2.1)` *Note:* ``iothread`` is mutually exclusive with ``iothre= ads``. + - The optional ``iothreads`` sub-element allows specifying multiple IO= Threads + via the ``iothread`` sub-element with attribute ``id`` the disk wil= l use + for I/O operations. Optionally the ``iothread`` element can have mul= tiple + ``queue`` subelements specifying that given iothread should be used = to + handle given queues. :since:`Since XXXXXX`. + Example:: + + + + + + + + + + + + + + + - The optional ``queues`` attribute specifies the number of virt queue= s for virtio-blk. ( :since:`Since 3.9.0` ) - The optional ``queue_size`` attribute specifies the size of each virt diff --git a/src/conf/domain_conf.c b/src/conf/domain_conf.c index 9426d55f8d..6904a71c80 100644 --- a/src/conf/domain_conf.c +++ b/src/conf/domain_conf.c @@ -2333,6 +2333,17 @@ virDomainDefGetVcpusTopology(const virDomainDef *def, } +void +virDomainDiskIothreadDefFree(virDomainDiskIothreadDef *def) +{ + if (!def) + return; + + g_free(def->queues); + g_free(def); +} + + static virDomainDiskDef * virDomainDiskDefNewSource(virDomainXMLOption *xmlopt, virStorageSource **src) @@ -2381,6 +2392,7 @@ virDomainDiskDefFree(virDomainDiskDef *def) g_free(def->virtio); virDomainDeviceInfoClear(&def->info); virObjectUnref(def->privateData); + g_slist_free_full(def->iothreads, (GDestroyNotify) virDomainDiskIothre= adDefFree); g_free(def); } @@ -7745,6 +7757,8 @@ static int virDomainDiskDefDriverParseXML(virDomainDiskDef *def, xmlNodePtr cur) { + xmlNodePtr iothreadsNode; + def->driverName =3D virXMLPropString(cur, "name"); if (virXMLPropEnum(cur, "cache", virDomainDiskCacheTypeFromString, @@ -7791,6 +7805,44 @@ virDomainDiskDefDriverParseXML(virDomainDiskDef *def, if (virXMLPropUInt(cur, "iothread", 10, VIR_XML_PROP_NONZERO, &def->io= thread) < 0) return -1; + if ((iothreadsNode =3D virXMLNodeGetSubelement(cur, "iothreads"))) { + g_autoslist(virDomainDiskIothreadDef) ioth =3D NULL; + g_autoptr(GPtrArray) iothreadNodes =3D NULL; + + if ((iothreadNodes =3D virXMLNodeGetSubelementList(iothreadsNode, = "iothread"))) { + size_t i; + + for (i =3D 0; i < iothreadNodes->len; i++) { + xmlNodePtr iothNode =3D g_ptr_array_index(iothreadNodes, i= ); + g_autoptr(virDomainDiskIothreadDef) iothdef =3D g_new0(vir= DomainDiskIothreadDef, 1); + g_autoptr(GPtrArray) queueNodes =3D NULL; + + if (virXMLPropUInt(iothNode, "id", 10, VIR_XML_PROP_REQUIR= ED, + &iothdef->id) < 0) + return -1; + + if ((queueNodes =3D virXMLNodeGetSubelementList(iothNode, = "queue"))) { + size_t q; + + iothdef->queues =3D g_new0(unsigned int, queueNodes->l= en); + iothdef->nqueues =3D queueNodes->len; + + for (q =3D 0; q < queueNodes->len; q++) { + xmlNodePtr queueNode =3D g_ptr_array_index(queueNo= des, q); + + if (virXMLPropUInt(queueNode, "id", 10, VIR_XML_PR= OP_REQUIRED, + &(iothdef->queues[q])) < 0) + return -1; + } + } + + ioth =3D g_slist_prepend(ioth, g_steal_pointer(&iothdef)); + } + + def->iothreads =3D g_slist_reverse(g_steal_pointer(&ioth)); + } + } + if (virXMLPropEnum(cur, "detect_zeroes", virDomainDiskDetectZeroesTypeFromString, VIR_XML_PROP_NONZERO, &def->detect_zeroes) < 0) @@ -22513,6 +22565,30 @@ virDomainDiskDefFormatDriver(virBuffer *buf, virXMLFormatElement(&childBuf, "metadata_cache", NULL, &metadataCa= cheChildBuf); } + if (disk->iothreads) { + g_auto(virBuffer) iothreadsChildBuf =3D VIR_BUFFER_INIT_CHILD(&chi= ldBuf); + GSList *n; + + for (n =3D disk->iothreads; n; n =3D n->next) { + virDomainDiskIothreadDef *iothDef =3D n->data; + g_auto(virBuffer) iothreadAttrBuf =3D VIR_BUFFER_INITIALIZER; + g_auto(virBuffer) iothreadChildBuf =3D VIR_BUFFER_INIT_CHILD(&= iothreadsChildBuf); + + virBufferAsprintf(&iothreadAttrBuf, " id=3D'%u'", iothDef->id); + + if (iothDef->queues) { + size_t q; + + for (q =3D 0; q < iothDef->nqueues; q++) + virBufferAsprintf(&iothreadChildBuf, "\n", iothDef->queues[q]); + } + + virXMLFormatElement(&iothreadsChildBuf, "iothread", &iothreadA= ttrBuf, &iothreadChildBuf); + } + + virXMLFormatElement(&childBuf, "iothreads", NULL, &iothreadsChildB= uf); + } + virXMLFormatElement(buf, "driver", &attrBuf, &childBuf); } diff --git a/src/conf/domain_conf.h b/src/conf/domain_conf.h index f6dade62fc..708993174a 100644 --- a/src/conf/domain_conf.h +++ b/src/conf/domain_conf.h @@ -552,6 +552,19 @@ typedef enum { VIR_ENUM_DECL(virDomainSnapshotLocation); +struct _virDomainDiskIothreadDef { + unsigned int id; + + /* optional list of virtqueues the iothread should handle */ + unsigned int *queues; + size_t nqueues; +}; + +typedef struct _virDomainDiskIothreadDef virDomainDiskIothreadDef; +void virDomainDiskIothreadDefFree(virDomainDiskIothreadDef *def); +G_DEFINE_AUTOPTR_CLEANUP_FUNC(virDomainDiskIothreadDef, virDomainDiskIothr= eadDefFree); + + /* Stores the virtual disk configuration */ struct _virDomainDiskDef { virStorageSource *src; /* non-NULL. XXX Allow NULL for empty cdrom? */ @@ -605,6 +618,7 @@ struct _virDomainDiskDef { virDomainDeviceSGIO sgio; virDomainDiskDiscard discard; unsigned int iothread; /* unused =3D 0, > 0 specific thread # */ + GSList *iothreads; /* List of virDomainDiskIothreadsDef */ virDomainDiskDetectZeroes detect_zeroes; char *domain_name; /* backend domain name */ unsigned int queues; diff --git a/src/conf/domain_validate.c b/src/conf/domain_validate.c index 5559a71e14..2aba645279 100644 --- a/src/conf/domain_validate.c +++ b/src/conf/domain_validate.c @@ -914,6 +914,14 @@ virDomainDiskDefValidate(const virDomainDef *def, return -1; } + /* configuring both and it's sub-e= lement + * isn't supported */ + if (disk->iothread && disk->iothreads) { + virReportError(VIR_ERR_XML_ERROR, "%s", + _("disk driver 'iothread' attribute can't be used t= ogether with 'iothreads' subelement")); + return -1; + } + return 0; } diff --git a/src/conf/schemas/domaincommon.rng b/src/conf/schemas/domaincom= mon.rng index 5a1d79672f..3e54091da2 100644 --- a/src/conf/schemas/domaincommon.rng +++ b/src/conf/schemas/domaincommon.rng @@ -2467,9 +2467,26 @@ - + + + + + + + + + + + + + + + + + + + + @@ -2516,17 +2533,23 @@ - - - - - - - - - + + + + + + + + + + + + + + + diff --git a/tests/qemuxml2argvdata/iothreads-disk.x86_64-latest.args b/tes= ts/qemuxml2argvdata/iothreads-disk.x86_64-latest.args index 02fb74d945..d1953327a7 100644 --- a/tests/qemuxml2argvdata/iothreads-disk.x86_64-latest.args +++ b/tests/qemuxml2argvdata/iothreads-disk.x86_64-latest.args @@ -19,6 +19,8 @@ XDG_CONFIG_HOME=3D/tmp/lib/domain--1-QEMUGuest1/.config \ -smp 2,sockets=3D2,cores=3D1,threads=3D1 \ -object '{"qom-type":"iothread","id":"iothread1"}' \ -object '{"qom-type":"iothread","id":"iothread2"}' \ +-object '{"qom-type":"iothread","id":"iothread3"}' \ +-object '{"qom-type":"iothread","id":"iothread4"}' \ -uuid c7a5fdbd-edaf-9455-926a-d65c16db1809 \ -display none \ -no-user-config \ @@ -30,12 +32,15 @@ XDG_CONFIG_HOME=3D/tmp/lib/domain--1-QEMUGuest1/.config= \ -no-acpi \ -boot strict=3Don \ -device '{"driver":"piix3-usb-uhci","id":"usb","bus":"pci.0","addr":"0x1.0= x2"}' \ --blockdev '{"driver":"file","filename":"/var/lib/libvirt/images/iothrtest1= .img","node-name":"libvirt-2-storage","auto-read-only":true,"discard":"unma= p"}' \ +-blockdev '{"driver":"file","filename":"/var/lib/libvirt/images/iothrtest1= .img","node-name":"libvirt-3-storage","auto-read-only":true,"discard":"unma= p"}' \ +-blockdev '{"node-name":"libvirt-3-format","read-only":false,"driver":"raw= ","file":"libvirt-3-storage"}' \ +-device '{"driver":"virtio-blk-pci","iothread":"iothread1","bus":"pci.0","= addr":"0x4","drive":"libvirt-3-format","id":"virtio-disk1","bootindex":1}' \ +-blockdev '{"driver":"file","filename":"/var/lib/libvirt/images/iothrtest2= .img","node-name":"libvirt-2-storage","auto-read-only":true,"discard":"unma= p"}' \ -blockdev '{"node-name":"libvirt-2-format","read-only":false,"driver":"raw= ","file":"libvirt-2-storage"}' \ --device '{"driver":"virtio-blk-pci","iothread":"iothread1","bus":"pci.0","= addr":"0x4","drive":"libvirt-2-format","id":"virtio-disk1","bootindex":1}' \ --blockdev '{"driver":"file","filename":"/var/lib/libvirt/images/iothrtest2= .img","node-name":"libvirt-1-storage","auto-read-only":true,"discard":"unma= p"}' \ +-device '{"driver":"virtio-blk-pci","num-queues":4,"bus":"pci.0","addr":"0= x2","drive":"libvirt-2-format","id":"virtio-disk2"}' \ +-blockdev '{"driver":"file","filename":"/var/lib/libvirt/images/iothrtest3= .img","node-name":"libvirt-1-storage","auto-read-only":true,"discard":"unma= p"}' \ -blockdev '{"node-name":"libvirt-1-format","read-only":false,"driver":"raw= ","file":"libvirt-1-storage"}' \ --device '{"driver":"virtio-blk-pci","iothread":"iothread2","bus":"pci.0","= addr":"0x2","drive":"libvirt-1-format","id":"virtio-disk2"}' \ +-device '{"driver":"virtio-blk-pci","bus":"pci.0","addr":"0x3","drive":"li= bvirt-1-format","id":"virtio-disk3"}' \ -audiodev '{"id":"audio1","driver":"none"}' \ -sandbox on,obsolete=3Ddeny,elevateprivileges=3Ddeny,spawn=3Ddeny,resource= control=3Ddeny \ -msg timestamp=3Don diff --git a/tests/qemuxml2argvdata/iothreads-disk.xml b/tests/qemuxml2argv= data/iothreads-disk.xml index ad0731c79c..7ce25f559e 100644 --- a/tests/qemuxml2argvdata/iothreads-disk.xml +++ b/tests/qemuxml2argvdata/iothreads-disk.xml @@ -4,7 +4,7 @@ 219136 219136 2 - 2 + 4 hvm @@ -22,10 +22,31 @@
- + + + + + + + + + + + + + + + + + + + + + + diff --git a/tests/qemuxml2xmloutdata/iothreads-disk.x86_64-latest.xml b/te= sts/qemuxml2xmloutdata/iothreads-disk.x86_64-latest.xml index ae1da9ec2a..94864feb85 100644 --- a/tests/qemuxml2xmloutdata/iothreads-disk.x86_64-latest.xml +++ b/tests/qemuxml2xmloutdata/iothreads-disk.x86_64-latest.xml @@ -4,7 +4,7 @@ 219136 219136 2 - 2 + 4 hvm @@ -25,11 +25,33 @@
- + + + + + + + + + + + +
+ + + + + + + + + +
+
--=20 2.39.2