From nobody Tue May 21 01:59:14 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 170.10.133.124 as permitted sender) client-ip=170.10.133.124; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-delivery-124.mimecast.com; Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=fail(p=none dis=none) header.from=nvidia.com ARC-Seal: i=1; a=rsa-sha256; t=1620199545; cv=none; d=zohomail.com; s=zohoarc; b=Gu4OE3oolsUct0q0ExYou5+Mnksr+z72U5LLgG4ewycYLUIqyx9qXgRCGxcDyldUluiSHHlen41IQ/QmJoqbN5XCxOettJ3bxpE36MHNj5k6h9kWRXDwdnhXJWDb5koAkoB2qAu799FBHBUIY0b0Bn94MzmCA6tXHz3rtRFmuzM= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1620199545; h=Content-Type:Cc:Date:From:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:Message-ID:Sender:Subject:To; bh=JiAReWgJMvaNbwLrk3Z4TlMFrxDOx27k/EmItOK1GJ4=; b=KK9ALKrHt6k8+yVl31DxzB3iB/sHwSkAW2JjJd6TIlg26x7AA4ZWdH55i326c9sSylaZGCl61wu2FzCBpvp02thLdVdSduytzT1fJppAZzZrUXx4E3gcqyGDeEZpgkLducYDn1kZA+g9cqcKapCsOmFxZpjeEC8WUkoOR3FAF6Q= ARC-Authentication-Results: i=1; mx.zohomail.com; spf=pass (zohomail.com: domain of redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by mx.zohomail.com with SMTPS id 162019954540024.28850218952425; Wed, 5 May 2021 00:25:45 -0700 (PDT) Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-466-jqrHl5HVPqexAh65AcvbTA-1; Wed, 05 May 2021 03:25:42 -0400 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id C9CAD8049D8; Wed, 5 May 2021 07:25:37 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.20]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 7E25119CAB; Wed, 5 May 2021 07:25:37 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id 9AF491800BBE; Wed, 5 May 2021 07:25:33 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id 1457PVZI005271 for ; Wed, 5 May 2021 03:25:32 -0400 Received: by smtp.corp.redhat.com (Postfix) id D58F220F8C1A; Wed, 5 May 2021 07:25:31 +0000 (UTC) Received: from mimecast-mx02.redhat.com (mimecast01.extmail.prod.ext.rdu2.redhat.com [10.11.55.17]) by smtp.corp.redhat.com (Postfix) with ESMTPS id D1C2B212AAB0 for ; Wed, 5 May 2021 07:25:29 +0000 (UTC) Received: from us-smtp-1.mimecast.com (us-smtp-2.mimecast.com [205.139.110.61]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 440A6858F0D for ; Wed, 5 May 2021 07:25:29 +0000 (UTC) Received: from mellanox.co.il (mail-il-dmz.mellanox.com [193.47.165.129]) by relay.mimecast.com with ESMTP id us-mta-150-O2WrrQNtOoO2qHlWQOJ5ew-1; Wed, 05 May 2021 03:25:27 -0400 Received: from Internal Mail-Server by MTLPINE1 (envelope-from gavi@nvidia.com) with SMTP; 5 May 2021 10:25:23 +0300 Received: from nps-server-13.mtl.labs.mlnx. (nps-server-13.mtl.labs.mlnx [10.237.248.1]) by labmailer.mlnx (8.13.8/8.13.8) with ESMTP id 1457PMVR024771; Wed, 5 May 2021 10:25:22 +0300 X-MC-Unique: jqrHl5HVPqexAh65AcvbTA-1 X-MC-Unique: O2WrrQNtOoO2qHlWQOJ5ew-1 From: Gavi Teitz To: libvir-list@redhat.com Subject: [PATCH v2] Add page_per_vq flag to the 'driver' element of virtio devices Date: Wed, 5 May 2021 10:25:06 +0300 Message-Id: <1620199506-3748-1-git-send-email-gavi@nvidia.com> X-Mimecast-Impersonation-Protect: Policy=CLT - Impersonation Protection Definition; Similar Internal Domain=false; Similar Monitored External Domain=false; Custom External Domain=false; Mimecast External Domain=false; Newly Observed Domain=false; Internal User Name=false; Custom Display Name List=false; Reply-to Address Mismatch=false; Targeted Threat Dictionary=false; Mimecast Threat Dictionary=false; Custom Threat Dictionary=false X-Scanned-By: MIMEDefang 2.78 on 10.11.54.4 X-loop: libvir-list@redhat.com Cc: moshele@nvidia.com, Gavi Teitz X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=libvir-list-bounces@redhat.com X-Mimecast-Spam-Score: 2 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" https://bugzilla.redhat.com/show_bug.cgi?id=3D1925363 Add support for setting the page-per-vq flag, which is important for vdpa with vhost-user performance. Signed-off-by: Gavi Teitz --- docs/formatdomain.rst | 11 +++++- docs/schemas/domaincommon.rng | 5 +++ src/conf/domain_conf.c | 10 +++++ src/conf/domain_conf.h | 1 + src/qemu/qemu_command.c | 5 +++ src/qemu/qemu_hotplug.c | 1 + .../net-virtio-page-per-vq.x86_64-latest.args | 38 ++++++++++++++++++ tests/qemuxml2argvdata/net-virtio-page-per-vq.xml | 29 ++++++++++++++ tests/qemuxml2argvtest.c | 1 + .../net-virtio-page-per-vq.x86_64-latest.xml | 46 ++++++++++++++++++= ++++ tests/qemuxml2xmltest.c | 1 + 11 files changed, 147 insertions(+), 1 deletion(-) create mode 100644 tests/qemuxml2argvdata/net-virtio-page-per-vq.x86_64-la= test.args create mode 100644 tests/qemuxml2argvdata/net-virtio-page-per-vq.xml create mode 100644 tests/qemuxml2xmloutdata/net-virtio-page-per-vq.x86_64-= latest.xml diff --git a/docs/formatdomain.rst b/docs/formatdomain.rst index fa5c14f..ce7a537 100644 --- a/docs/formatdomain.rst +++ b/docs/formatdomain.rst @@ -5120,7 +5120,7 @@ Setting NIC driver-specific options - + @@ -5215,6 +5215,15 @@ following attributes are available for the ``"virtio= "`` NIC driver: only for ``vhostuser`` type. :since:`Since 3.7.0 (QEMU and KVM only)` **In general you should leave this option alone, unless you are very ce= rtain you know what you are doing.** +``page_per_vq`` + This optional attribute controls the layout of the notification capabil= ities + exposed to the guest. When enabled, each virtio queue will have a dedic= ated + page on the device BAR exposed to the guest. It is recommended to be us= ed when + vDPA is enabled on the hypervisor, as it enables mapping the notificati= on area + to the physical device, which is only supported in page granularity. The + default is determined by QEMU; as off. :since:`Since 7.4.0` + **In general you should leave this option alone, unless you are very ce= rtain + you know what you are doing.** virtio options For virtio interfaces, `Virtio-specific options <#elementsVirtio>`__ ca= n also be set. ( :since:`Since 3.5.0` ) diff --git a/docs/schemas/domaincommon.rng b/docs/schemas/domaincommon.rng index a2e5c50..e61ad67 100644 --- a/docs/schemas/domaincommon.rng +++ b/docs/schemas/domaincommon.rng @@ -3463,6 +3463,11 @@ + + + + + iothread diff --git a/src/conf/domain_conf.c b/src/conf/domain_conf.c index cb668d3..0e219ed 100644 --- a/src/conf/domain_conf.c +++ b/src/conf/domain_conf.c @@ -10948,6 +10948,12 @@ virDomainNetDefParseXML(virDomainXMLOption *xmlopt, def->driver.virtio.tx_queue_size =3D q; } =20 + if ((tmpNode =3D virXPathNode("./driver", ctxt))) { + if (virXMLPropTristateSwitch(tmpNode, "page_per_vq", VIR_XML_P= ROP_NONE, + &def->driver.virtio.page_per_vq) = < 0) + goto error; + } + if ((tmpNode =3D virXPathNode("./driver/host", ctxt))) { if (virXMLPropTristateSwitch(tmpNode, "csum", VIR_XML_PROP_NON= E, &def->driver.virtio.host.csum) < = 0) @@ -25221,6 +25227,10 @@ virDomainVirtioNetDriverFormat(virBuffer *buf, if (def->driver.virtio.tx_queue_size) virBufferAsprintf(buf, " tx_queue_size=3D'%u'", def->driver.virtio.tx_queue_size); + if (def->driver.virtio.page_per_vq) { + virBufferAsprintf(buf, " page_per_vq=3D'%s'", + virTristateSwitchTypeToString(def->driver.virtio= .page_per_vq)); + } =20 virDomainVirtioOptionsFormat(buf, def->virtio); } diff --git a/src/conf/domain_conf.h b/src/conf/domain_conf.h index 85c318d..7aab565 100644 --- a/src/conf/domain_conf.h +++ b/src/conf/domain_conf.h @@ -1027,6 +1027,7 @@ struct _virDomainNetDef { virDomainNetVirtioTxModeType txmode; virTristateSwitch ioeventfd; virTristateSwitch event_idx; + virTristateSwitch page_per_vq; unsigned int queues; /* Multiqueue virtio-net */ unsigned int rx_queue_size; unsigned int tx_queue_size; diff --git a/src/qemu/qemu_command.c b/src/qemu/qemu_command.c index ca2265c..3545ee9 100644 --- a/src/qemu/qemu_command.c +++ b/src/qemu/qemu_command.c @@ -3629,6 +3629,11 @@ qemuBuildNicDevStr(virDomainDef *def, if (net->driver.virtio.tx_queue_size) virBufferAsprintf(&buf, ",tx_queue_size=3D%u", net->driver.vir= tio.tx_queue_size); =20 + if (net->driver.virtio.page_per_vq) { + virBufferAsprintf(&buf, ",page-per-vq=3D%s", + virTristateSwitchTypeToString(net->driver.vi= rtio.page_per_vq)); + } + if (net->mtu) virBufferAsprintf(&buf, ",host_mtu=3D%u", net->mtu); =20 diff --git a/src/qemu/qemu_hotplug.c b/src/qemu/qemu_hotplug.c index 444d89d..a244709 100644 --- a/src/qemu/qemu_hotplug.c +++ b/src/qemu/qemu_hotplug.c @@ -3578,6 +3578,7 @@ qemuDomainChangeNet(virQEMUDriver *driver, olddev->driver.virtio.queues !=3D newdev->driver.virtio.queues || olddev->driver.virtio.rx_queue_size !=3D newdev->driver.virtio.rx= _queue_size || olddev->driver.virtio.tx_queue_size !=3D newdev->driver.virtio.tx= _queue_size || + olddev->driver.virtio.page_per_vq !=3D newdev->driver.virtio.page= _per_vq || olddev->driver.virtio.host.csum !=3D newdev->driver.virtio.host.c= sum || olddev->driver.virtio.host.gso !=3D newdev->driver.virtio.host.gs= o || olddev->driver.virtio.host.tso4 !=3D newdev->driver.virtio.host.t= so4 || diff --git a/tests/qemuxml2argvdata/net-virtio-page-per-vq.x86_64-latest.ar= gs b/tests/qemuxml2argvdata/net-virtio-page-per-vq.x86_64-latest.args new file mode 100644 index 0000000..5a52595 --- /dev/null +++ b/tests/qemuxml2argvdata/net-virtio-page-per-vq.x86_64-latest.args @@ -0,0 +1,38 @@ +LC_ALL=3DC \ +PATH=3D/bin \ +HOME=3D/tmp/lib/domain--1-QEMUGuest1 \ +USER=3Dtest \ +LOGNAME=3Dtest \ +XDG_DATA_HOME=3D/tmp/lib/domain--1-QEMUGuest1/.local/share \ +XDG_CACHE_HOME=3D/tmp/lib/domain--1-QEMUGuest1/.cache \ +XDG_CONFIG_HOME=3D/tmp/lib/domain--1-QEMUGuest1/.config \ +/usr/bin/qemu-system-i386 \ +-name guest=3DQEMUGuest1,debug-threads=3Don \ +-S \ +-object '{"qom-type":"secret","id":"masterKey0","format":"raw","file":"/tm= p/lib/domain--1-QEMUGuest1/master-key.aes"}' \ +-machine pc,accel=3Dtcg,usb=3Doff,dump-guest-core=3Doff,memory-backend=3Dp= c.ram \ +-cpu qemu64 \ +-m 214 \ +-object '{"qom-type":"memory-backend-ram","id":"pc.ram","size":224395264}'= \ +-overcommit mem-lock=3Doff \ +-smp 1,sockets=3D1,cores=3D1,threads=3D1 \ +-uuid c7a5fdbd-edaf-9455-926a-d65c16db1809 \ +-display none \ +-no-user-config \ +-nodefaults \ +-chardev socket,id=3Dcharmonitor,fd=3D1729,server=3Don,wait=3Doff \ +-mon chardev=3Dcharmonitor,id=3Dmonitor,mode=3Dcontrol \ +-rtc base=3Dutc \ +-no-shutdown \ +-no-acpi \ +-boot strict=3Don \ +-device piix3-usb-uhci,id=3Dusb,bus=3Dpci.0,addr=3D0x1.0x2 \ +-blockdev '{"driver":"host_device","filename":"/dev/HostVG/QEMUGuest1","no= de-name":"libvirt-1-storage","auto-read-only":true,"discard":"unmap"}' \ +-blockdev '{"node-name":"libvirt-1-format","read-only":false,"driver":"raw= ","file":"libvirt-1-storage"}' \ +-device ide-hd,bus=3Dide.0,unit=3D0,drive=3Dlibvirt-1-format,id=3Dide0-0-0= ,bootindex=3D1 \ +-netdev user,id=3Dhostnet0 \ +-device virtio-net-pci,page-per-vq=3Don,netdev=3Dhostnet0,id=3Dnet0,mac=3D= 00:11:22:33:44:55,bus=3Dpci.0,addr=3D0x2 \ +-audiodev id=3Daudio1,driver=3Dnone \ +-device virtio-balloon-pci,id=3Dballoon0,bus=3Dpci.0,addr=3D0x3 \ +-sandbox on,obsolete=3Ddeny,elevateprivileges=3Ddeny,spawn=3Ddeny,resource= control=3Ddeny \ +-msg timestamp=3Don diff --git a/tests/qemuxml2argvdata/net-virtio-page-per-vq.xml b/tests/qemu= xml2argvdata/net-virtio-page-per-vq.xml new file mode 100644 index 0000000..e22ecd6 --- /dev/null +++ b/tests/qemuxml2argvdata/net-virtio-page-per-vq.xml @@ -0,0 +1,29 @@ + + QEMUGuest1 + c7a5fdbd-edaf-9455-926a-d65c16db1809 + 219100 + 219100 + 1 + + hvm + + + + destroy + restart + destroy + + /usr/bin/qemu-system-i386 + + + + + + + + + + + + + diff --git a/tests/qemuxml2argvtest.c b/tests/qemuxml2argvtest.c index a9dafe2..3dc866a 100644 --- a/tests/qemuxml2argvtest.c +++ b/tests/qemuxml2argvtest.c @@ -1634,6 +1634,7 @@ mymain(void) QEMU_CAPS_VIRTIO_NET_TX_QUEUE_SIZE); DO_TEST_PARSE_ERROR("net-virtio-rxqueuesize-invalid-size", QEMU_CAPS_VIRTIO_NET_RX_QUEUE_SIZE); + DO_TEST_CAPS_LATEST("net-virtio-page-per-vq"); DO_TEST("net-virtio-teaming", QEMU_CAPS_VIRTIO_NET_FAILOVER, QEMU_CAPS_DEVICE_VFIO_PCI); diff --git a/tests/qemuxml2xmloutdata/net-virtio-page-per-vq.x86_64-latest.= xml b/tests/qemuxml2xmloutdata/net-virtio-page-per-vq.x86_64-latest.xml new file mode 100644 index 0000000..34f7ee5 --- /dev/null +++ b/tests/qemuxml2xmloutdata/net-virtio-page-per-vq.x86_64-latest.xml @@ -0,0 +1,46 @@ + + QEMUGuest1 + c7a5fdbd-edaf-9455-926a-d65c16db1809 + 219100 + 219100 + 1 + + hvm + + + + qemu64 + + + destroy + restart + destroy + + /usr/bin/qemu-system-i386 + + + + +
+ + +
+ + + +
+ + + + + +
+ + + +