From nobody Sat Apr 27 05:01:50 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 170.10.133.124 as permitted sender) client-ip=170.10.133.124; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-delivery-124.mimecast.com; Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=fail(p=none dis=none) header.from=nvidia.com ARC-Seal: i=1; a=rsa-sha256; t=1619695242; cv=none; d=zohomail.com; s=zohoarc; b=CHfIxDiiodc48vT6TsReamnT7hj9PYcjb/tpTr/Y4dGvdbZVkTNaKcFcXSnDdJNtLOwVnuPadlksKtKjqfhm1fOHcSRUHh02v/fn0+xQBWJbL+m+f9PI2uIpccFBI8QHbTWy06RgRp90lBC4DxzhjD/iCdzlbx1QF9zEUHMdro0= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1619695242; h=Content-Type:Cc:Date:From:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:Message-ID:Sender:Subject:To; bh=EowEXdrO7OK6vOELf6sOMaCEaSmRFj0FqMLu6xEplWI=; b=dHrpwf/z8zMoANsAtlcPRW2FJ+IgAvJk4IlzPrbNQOidaWlQO9xihiMgETuPQyJ2DzsjOtyt6w+c89DBG3g+6rBfQPKqTz84RpEJ11r2C6OQEShyUeIXHnnB67pZl3JhNn3jOug85y/o/fBDAkxBq7wtRAsqNMFMKAD2g9+BPN4= ARC-Authentication-Results: i=1; mx.zohomail.com; spf=pass (zohomail.com: domain of redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by mx.zohomail.com with SMTPS id 1619695242639927.641891347903; Thu, 29 Apr 2021 04:20:42 -0700 (PDT) Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-146-0JRvTyMoO5SoHOOYXhypzg-1; Thu, 29 Apr 2021 07:20:38 -0400 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 089B58C8305; Thu, 29 Apr 2021 11:20:23 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.20]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 308836E52F; Thu, 29 Apr 2021 11:20:20 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id 7FC901809C82; Thu, 29 Apr 2021 11:20:16 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id 13TBKE1A016190 for ; Thu, 29 Apr 2021 07:20:14 -0400 Received: by smtp.corp.redhat.com (Postfix) id 743DD7013C; Thu, 29 Apr 2021 11:20:14 +0000 (UTC) Received: from mimecast-mx01.redhat.com (mimecast03.extmail.prod.ext.phx2.redhat.com [10.5.110.60]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 6D01F70134 for ; Thu, 29 Apr 2021 11:19:50 +0000 (UTC) Received: from us-smtp-1.mimecast.com (us-smtp-delivery-1.mimecast.com [205.139.110.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 1A0521B18BC5 for ; Thu, 29 Apr 2021 11:19:50 +0000 (UTC) Received: from mellanox.co.il (mail-il-dmz.mellanox.com [193.47.165.129]) by relay.mimecast.com with ESMTP id us-mta-240-2VBYLcP2OZ6dB7xfPutDFQ-1; Thu, 29 Apr 2021 07:19:47 -0400 Received: from Internal Mail-Server by MTLPINE1 (envelope-from gavi@nvidia.com) with SMTP; 29 Apr 2021 14:12:57 +0300 Received: from nps-server-13.mtl.labs.mlnx. (nps-server-13.mtl.labs.mlnx [10.237.248.1]) by labmailer.mlnx (8.13.8/8.13.8) with ESMTP id 13TBCv87011540; Thu, 29 Apr 2021 14:12:57 +0300 X-MC-Unique: 0JRvTyMoO5SoHOOYXhypzg-1 X-MC-Unique: 2VBYLcP2OZ6dB7xfPutDFQ-1 From: Gavi Teitz To: libvir-list@redhat.com Subject: [PATCH] Add page_per_vq flag to the 'driver' element of virtio devices Date: Thu, 29 Apr 2021 14:12:44 +0300 Message-Id: <1619694764-5190-1-git-send-email-gavi@nvidia.com> X-Mimecast-Impersonation-Protect: Policy=CLT - Impersonation Protection Definition; Similar Internal Domain=false; Similar Monitored External Domain=false; Custom External Domain=false; Mimecast External Domain=false; Newly Observed Domain=false; Internal User Name=false; Custom Display Name List=false; Reply-to Address Mismatch=false; Targeted Threat Dictionary=false; Mimecast Threat Dictionary=false; Custom Threat Dictionary=false X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 X-loop: libvir-list@redhat.com Cc: moshele@nvidia.com, gavi@nvidia.com X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=libvir-list-bounces@redhat.com X-Mimecast-Spam-Score: 2 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" https://bugzilla.redhat.com/show_bug.cgi?id=3D1925363 Add support for setting the page-per-vq flag, which is important for vdpa with vhost-user performance. Signed-off-by: Gavi Teitz --- docs/formatdomain.rst | 11 +++++- docs/schemas/domaincommon.rng | 5 +++ src/conf/domain_conf.c | 15 ++++++++ src/conf/domain_conf.h | 1 + src/qemu/qemu_capabilities.c | 2 + src/qemu/qemu_capabilities.h | 1 + src/qemu/qemu_command.c | 5 +++ src/qemu/qemu_hotplug.c | 1 + tests/qemuxml2argvdata/net-virtio-page-per-vq.args | 31 ++++++++++++++++ tests/qemuxml2argvdata/net-virtio-page-per-vq.xml | 29 +++++++++++++++ tests/qemuxml2argvtest.c | 2 + .../qemuxml2xmloutdata/net-virtio-page-per-vq.xml | 43 ++++++++++++++++++= ++++ tests/qemuxml2xmltest.c | 2 + 13 files changed, 147 insertions(+), 1 deletion(-) create mode 100644 tests/qemuxml2argvdata/net-virtio-page-per-vq.args create mode 100644 tests/qemuxml2argvdata/net-virtio-page-per-vq.xml create mode 100644 tests/qemuxml2xmloutdata/net-virtio-page-per-vq.xml diff --git a/docs/formatdomain.rst b/docs/formatdomain.rst index 282176c..f5a9c70 100644 --- a/docs/formatdomain.rst +++ b/docs/formatdomain.rst @@ -5120,7 +5120,7 @@ Setting NIC driver-specific options - + @@ -5215,6 +5215,15 @@ following attributes are available for the ``"virtio= "`` NIC driver: only for ``vhostuser`` type. :since:`Since 3.7.0 (QEMU and KVM only)` **In general you should leave this option alone, unless you are very ce= rtain you know what you are doing.** +``page_per_vq`` + This optional attribute controls the layout of the notification capabil= ities + exposed to the guest. When enabled, each virtio queue will have a dedic= ated + page on the device BAR exposed to the guest. It is recommended to be us= ed when + vDPA is enabled on the hypervisor, as it enables mapping the notificati= on area + to the physical device, which is only supported in page granularity. The + default is determined by QEMU; as off. :since:`Since 2.8.0 (QEMU only)` + **In general you should leave this option alone, unless you are very ce= rtain + you know what you are doing.** virtio options For virtio interfaces, `Virtio-specific options <#elementsVirtio>`__ ca= n also be set. ( :since:`Since 3.5.0` ) diff --git a/docs/schemas/domaincommon.rng b/docs/schemas/domaincommon.rng index a2e5c50..e61ad67 100644 --- a/docs/schemas/domaincommon.rng +++ b/docs/schemas/domaincommon.rng @@ -3463,6 +3463,11 @@ + + + + + iothread diff --git a/src/conf/domain_conf.c b/src/conf/domain_conf.c index 9d98f48..0350fde 100644 --- a/src/conf/domain_conf.c +++ b/src/conf/domain_conf.c @@ -10446,6 +10446,7 @@ virDomainNetDefParseXML(virDomainXMLOption *xmlopt, g_autofree char *queues =3D NULL; g_autofree char *rx_queue_size =3D NULL; g_autofree char *tx_queue_size =3D NULL; + g_autofree char *page_per_vq =3D NULL; g_autofree char *filter =3D NULL; g_autofree char *internal =3D NULL; g_autofree char *mode =3D NULL; @@ -10615,6 +10616,7 @@ virDomainNetDefParseXML(virDomainXMLOption *xmlopt, queues =3D virXMLPropString(cur, "queues"); rx_queue_size =3D virXMLPropString(cur, "rx_queue_size"); tx_queue_size =3D virXMLPropString(cur, "tx_queue_size"); + page_per_vq =3D virXMLPropString(cur, "page_per_vq"); =20 if (virDomainVirtioOptionsParseXML(cur, &def->virtio) < 0) goto error; @@ -11041,6 +11043,15 @@ virDomainNetDefParseXML(virDomainXMLOption *xmlopt, } def->driver.virtio.tx_queue_size =3D q; } + if (page_per_vq) { + if ((val =3D virTristateSwitchTypeFromString(page_per_vq)) <= =3D 0) { + virReportError(VIR_ERR_CONFIG_UNSUPPORTED, + _("unknown page_per_vq mode '%s'"), + page_per_vq); + goto error; + } + def->driver.virtio.page_per_vq =3D val; + } =20 if ((tmpNode =3D virXPathNode("./driver/host", ctxt))) { if (virXMLPropTristateSwitch(tmpNode, "csum", VIR_XML_PROP_NON= E, @@ -25453,6 +25464,10 @@ virDomainVirtioNetDriverFormat(virBuffer *buf, if (def->driver.virtio.tx_queue_size) virBufferAsprintf(buf, " tx_queue_size=3D'%u'", def->driver.virtio.tx_queue_size); + if (def->driver.virtio.page_per_vq) { + virBufferAsprintf(buf, " page_per_vq=3D'%s'", + virTristateSwitchTypeToString(def->driver.virtio= .page_per_vq)); + } =20 virDomainVirtioOptionsFormat(buf, def->virtio); } diff --git a/src/conf/domain_conf.h b/src/conf/domain_conf.h index 85c318d..7aab565 100644 --- a/src/conf/domain_conf.h +++ b/src/conf/domain_conf.h @@ -1027,6 +1027,7 @@ struct _virDomainNetDef { virDomainNetVirtioTxModeType txmode; virTristateSwitch ioeventfd; virTristateSwitch event_idx; + virTristateSwitch page_per_vq; unsigned int queues; /* Multiqueue virtio-net */ unsigned int rx_queue_size; unsigned int tx_queue_size; diff --git a/src/qemu/qemu_capabilities.c b/src/qemu/qemu_capabilities.c index 7971a9c..e5646cb 100644 --- a/src/qemu/qemu_capabilities.c +++ b/src/qemu/qemu_capabilities.c @@ -629,6 +629,7 @@ VIR_ENUM_IMPL(virQEMUCaps, /* 400 */ "compat-deprecated", "acpi-index", + "virtio-net.page_per_vq", ); =20 =20 @@ -1405,6 +1406,7 @@ static struct virQEMUCapsDevicePropsFlags virQEMUCaps= DevicePropsVirtioNet[] =3D { { "event_idx", QEMU_CAPS_VIRTIO_NET_EVENT_IDX, NULL }, { "rx_queue_size", QEMU_CAPS_VIRTIO_NET_RX_QUEUE_SIZE, NULL }, { "tx_queue_size", QEMU_CAPS_VIRTIO_NET_TX_QUEUE_SIZE, NULL }, + { "page_per_vq", QEMU_CAPS_VIRTIO_NET_PAGE_PER_VQ, NULL }, { "host_mtu", QEMU_CAPS_VIRTIO_NET_HOST_MTU, NULL }, { "disable-legacy", QEMU_CAPS_VIRTIO_PCI_DISABLE_LEGACY, NULL }, { "iommu_platform", QEMU_CAPS_VIRTIO_PCI_IOMMU_PLATFORM, NULL }, diff --git a/src/qemu/qemu_capabilities.h b/src/qemu/qemu_capabilities.h index f54aad5..75feed4 100644 --- a/src/qemu/qemu_capabilities.h +++ b/src/qemu/qemu_capabilities.h @@ -609,6 +609,7 @@ typedef enum { /* virQEMUCapsFlags grouping marker for = syntax-check */ /* 400 */ QEMU_CAPS_COMPAT_DEPRECATED, /* -compat deprecated-(input|output) is s= upported */ QEMU_CAPS_ACPI_INDEX, /* PCI device 'acpi-index' property */ + QEMU_CAPS_VIRTIO_NET_PAGE_PER_VQ, /* virtio-net-*.page_per_vq */ =20 QEMU_CAPS_LAST /* this must always be the last item */ } virQEMUCapsFlags; diff --git a/src/qemu/qemu_command.c b/src/qemu/qemu_command.c index d7f1c71..7b5834f 100644 --- a/src/qemu/qemu_command.c +++ b/src/qemu/qemu_command.c @@ -3630,6 +3630,11 @@ qemuBuildNicDevStr(virDomainDef *def, if (net->driver.virtio.tx_queue_size) virBufferAsprintf(&buf, ",tx_queue_size=3D%u", net->driver.vir= tio.tx_queue_size); =20 + if (net->driver.virtio.page_per_vq) { + virBufferAsprintf(&buf, ",page-per-vq=3D%s", + virTristateSwitchTypeToString(net->driver.vi= rtio.page_per_vq)); + } + if (net->mtu) virBufferAsprintf(&buf, ",host_mtu=3D%u", net->mtu); =20 diff --git a/src/qemu/qemu_hotplug.c b/src/qemu/qemu_hotplug.c index 4344edc..f79c3d8 100644 --- a/src/qemu/qemu_hotplug.c +++ b/src/qemu/qemu_hotplug.c @@ -3578,6 +3578,7 @@ qemuDomainChangeNet(virQEMUDriver *driver, olddev->driver.virtio.queues !=3D newdev->driver.virtio.queues || olddev->driver.virtio.rx_queue_size !=3D newdev->driver.virtio.rx= _queue_size || olddev->driver.virtio.tx_queue_size !=3D newdev->driver.virtio.tx= _queue_size || + olddev->driver.virtio.page_per_vq !=3D newdev->driver.virtio.page= _per_vq || olddev->driver.virtio.host.csum !=3D newdev->driver.virtio.host.c= sum || olddev->driver.virtio.host.gso !=3D newdev->driver.virtio.host.gs= o || olddev->driver.virtio.host.tso4 !=3D newdev->driver.virtio.host.t= so4 || diff --git a/tests/qemuxml2argvdata/net-virtio-page-per-vq.args b/tests/qem= uxml2argvdata/net-virtio-page-per-vq.args new file mode 100644 index 0000000..d7db71f --- /dev/null +++ b/tests/qemuxml2argvdata/net-virtio-page-per-vq.args @@ -0,0 +1,31 @@ +LC_ALL=3DC \ +PATH=3D/bin \ +HOME=3D/tmp/lib/domain--1-QEMUGuest1 \ +USER=3Dtest \ +LOGNAME=3Dtest \ +XDG_DATA_HOME=3D/tmp/lib/domain--1-QEMUGuest1/.local/share \ +XDG_CACHE_HOME=3D/tmp/lib/domain--1-QEMUGuest1/.cache \ +XDG_CONFIG_HOME=3D/tmp/lib/domain--1-QEMUGuest1/.config \ +QEMU_AUDIO_DRV=3Dnone \ +/usr/bin/qemu-system-i386 \ +-name QEMUGuest1 \ +-S \ +-machine pc,accel=3Dtcg,usb=3Doff,dump-guest-core=3Doff \ +-m 214 \ +-realtime mlock=3Doff \ +-smp 1,sockets=3D1,cores=3D1,threads=3D1 \ +-uuid c7a5fdbd-edaf-9455-926a-d65c16db1809 \ +-display none \ +-no-user-config \ +-nodefaults \ +-chardev socket,id=3Dcharmonitor,path=3D/tmp/lib/domain--1-QEMUGuest1/moni= tor.sock,server=3Don,wait=3Doff \ +-mon chardev=3Dcharmonitor,id=3Dmonitor,mode=3Dcontrol \ +-rtc base=3Dutc \ +-no-shutdown \ +-no-acpi \ +-usb \ +-drive file=3D/dev/HostVG/QEMUGuest1,format=3Draw,if=3Dnone,id=3Ddrive-ide= 0-0-0 \ +-device ide-hd,bus=3Dide.0,unit=3D0,drive=3Ddrive-ide0-0-0,id=3Dide0-0-0,b= ootindex=3D1 \ +-netdev user,id=3Dhostnet0 \ +-device virtio-net-pci,page-per-vq=3Don,netdev=3Dhostnet0,id=3Dnet0,mac=3D= 00:11:22:33:44:55,bus=3Dpci.0,addr=3D0x3 \ +-device virtio-balloon-pci,id=3Dballoon0,bus=3Dpci.0,addr=3D0x4 diff --git a/tests/qemuxml2argvdata/net-virtio-page-per-vq.xml b/tests/qemu= xml2argvdata/net-virtio-page-per-vq.xml new file mode 100644 index 0000000..e22ecd6 --- /dev/null +++ b/tests/qemuxml2argvdata/net-virtio-page-per-vq.xml @@ -0,0 +1,29 @@ + + QEMUGuest1 + c7a5fdbd-edaf-9455-926a-d65c16db1809 + 219100 + 219100 + 1 + + hvm + + + + destroy + restart + destroy + + /usr/bin/qemu-system-i386 + + + + + + + + + + + + + diff --git a/tests/qemuxml2argvtest.c b/tests/qemuxml2argvtest.c index f0efe98..f6b8f34 100644 --- a/tests/qemuxml2argvtest.c +++ b/tests/qemuxml2argvtest.c @@ -1654,6 +1654,8 @@ mymain(void) QEMU_CAPS_VIRTIO_NET_TX_QUEUE_SIZE); DO_TEST_PARSE_ERROR("net-virtio-rxqueuesize-invalid-size", QEMU_CAPS_VIRTIO_NET_RX_QUEUE_SIZE); + DO_TEST("net-virtio-page-per-vq", + QEMU_CAPS_VIRTIO_NET_PAGE_PER_VQ); DO_TEST("net-virtio-teaming", QEMU_CAPS_VIRTIO_NET_FAILOVER, QEMU_CAPS_DEVICE_VFIO_PCI); diff --git a/tests/qemuxml2xmloutdata/net-virtio-page-per-vq.xml b/tests/qe= muxml2xmloutdata/net-virtio-page-per-vq.xml new file mode 100644 index 0000000..a35ed8a --- /dev/null +++ b/tests/qemuxml2xmloutdata/net-virtio-page-per-vq.xml @@ -0,0 +1,43 @@ + + QEMUGuest1 + c7a5fdbd-edaf-9455-926a-d65c16db1809 + 219100 + 219100 + 1 + + hvm + + + + destroy + restart + destroy + + /usr/bin/qemu-system-i386 + + + + +
+ + +
+ + + +
+ + + + + +
+ + + +