From nobody Mon Feb 9 10:29:15 2026 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) client-ip=209.132.183.28; envelope-from=libvir-list-bounces@redhat.com; helo=mx1.redhat.com; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com Return-Path: Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by mx.zohomail.com with SMTPS id 1500307249662679.0638443218819; Mon, 17 Jul 2017 09:00:49 -0700 (PDT) Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 1A8F1624C9; Mon, 17 Jul 2017 16:00:48 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.20]) by smtp.corp.redhat.com (Postfix) with ESMTPS id B5EA35C8BA; Mon, 17 Jul 2017 16:00:47 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id 471681803B26; Mon, 17 Jul 2017 16:00:47 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id v6HG0kAA001581 for ; Mon, 17 Jul 2017 12:00:46 -0400 Received: by smtp.corp.redhat.com (Postfix) id 7CE087EBEA; Mon, 17 Jul 2017 16:00:46 +0000 (UTC) Received: from mx1.redhat.com (ext-mx04.extmail.prod.ext.phx2.redhat.com [10.5.110.28]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 6BEDB7EDA7 for ; Mon, 17 Jul 2017 16:00:35 +0000 (UTC) Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 84BC67A16F for ; Mon, 17 Jul 2017 16:00:33 +0000 (UTC) Received: from pps.filterd (m0098396.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.21/8.16.0.21) with SMTP id v6HFxdmW146738 for ; Mon, 17 Jul 2017 12:00:33 -0400 Received: from e23smtp04.au.ibm.com (e23smtp04.au.ibm.com [202.81.31.146]) by mx0a-001b2d01.pphosted.com with ESMTP id 2brr0fy5xk-1 (version=TLSv1.2 cipher=AES256-SHA bits=256 verify=NOT) for ; Mon, 17 Jul 2017 12:00:32 -0400 Received: from localhost by e23smtp04.au.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Tue, 18 Jul 2017 02:00:29 +1000 Received: from d23relay07.au.ibm.com (202.81.31.226) by e23smtp04.au.ibm.com (202.81.31.210) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted; Tue, 18 Jul 2017 02:00:27 +1000 Received: from d23av01.au.ibm.com (d23av01.au.ibm.com [9.190.234.96]) by d23relay07.au.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id v6HG0R1x4718708; Tue, 18 Jul 2017 02:00:27 +1000 Received: from d23av01.au.ibm.com (localhost [127.0.0.1]) by d23av01.au.ibm.com (8.14.4/8.14.4/NCO v10.0 AVout) with ESMTP id v6HG0Qpq009819; Tue, 18 Jul 2017 02:00:26 +1000 Received: from lep8c.aus.stglabs.ibm.com (lep8c.aus.stglabs.ibm.com [9.40.192.207]) by d23av01.au.ibm.com (8.14.4/8.14.4/NCO v10.0 AVin) with ESMTP id v6HG0P8Q009699; Tue, 18 Jul 2017 02:00:26 +1000 DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com 1A8F1624C9 Authentication-Results: ext-mx10.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=linux.vnet.ibm.com Authentication-Results: ext-mx10.extmail.prod.ext.phx2.redhat.com; spf=pass smtp.mailfrom=libvir-list-bounces@redhat.com DKIM-Filter: OpenDKIM Filter v2.11.0 mx1.redhat.com 1A8F1624C9 DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com 84BC67A16F Authentication-Results: ext-mx04.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=linux.vnet.ibm.com Authentication-Results: ext-mx04.extmail.prod.ext.phx2.redhat.com; spf=none smtp.mailfrom=sbhat@linux.vnet.ibm.com DKIM-Filter: OpenDKIM Filter v2.11.0 mx1.redhat.com 84BC67A16F From: Shivaprasad G Bhat To: libvir-list@redhat.com Date: Mon, 17 Jul 2017 21:29:15 +0530 In-Reply-To: <20170717155221.2984.46275.stgit@lep8c.aus.stglabs.ibm.com> References: <20170717155221.2984.46275.stgit@lep8c.aus.stglabs.ibm.com> User-Agent: StGit/0.17.1-dirty MIME-Version: 1.0 X-TM-AS-MML: disable x-cbid: 17071716-0012-0000-0000-00000257395B X-IBM-AV-DETECTION: SAVI=unused REMOTE=unused XFE=unused x-cbparentid: 17071716-0013-0000-0000-00000770EA85 Message-Id: <20170717155856.2984.10603.stgit@lep8c.aus.stglabs.ibm.com> X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:, , definitions=2017-07-17_12:, , signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 spamscore=0 suspectscore=15 malwarescore=0 phishscore=0 adultscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1706020000 definitions=main-1707170255 X-Greylist: Sender passed SPF test, Sender IP whitelisted by DNSRBL, ACL 200 matched, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.28]); Mon, 17 Jul 2017 16:00:33 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.28]); Mon, 17 Jul 2017 16:00:33 +0000 (UTC) for IP:'148.163.156.1' DOMAIN:'mx0a-001b2d01.pphosted.com' HELO:'mx0a-001b2d01.pphosted.com' FROM:'sbhat@linux.vnet.ibm.com' RCPT:'' X-RedHat-Spam-Score: -2.7 (BAYES_50, RCVD_IN_DNSWL_LOW, RCVD_IN_MSPIKE_H2) 148.163.156.1 mx0a-001b2d01.pphosted.com 148.163.156.1 mx0a-001b2d01.pphosted.com X-Scanned-By: MIMEDefang 2.78 on 10.5.110.28 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 X-loop: libvir-list@redhat.com Cc: abologna@redhat.com Subject: [libvirt] [PATCH v2 3/3] qemu: Enable NUMA node tag in pci-root for PPC64 X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Transfer-Encoding: quoted-printable Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.39]); Mon, 17 Jul 2017 16:00:48 +0000 (UTC) X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Type: text/plain; charset="utf-8" This patch addresses the same aspects on PPC the bug 1103314 addressed on x86. PCI expander bus creates multiple primary PCI busses, where each of these busses can be assigned a specific NUMA affinity, which, on x86 is advertised through ACPI on a per-bus basis. For SPAPR, a PHB's NUMA affinities are assigned on a per-PHB basis, and there is no mechanism for advertising NUMA affinities to a guest on a per-bus basis. So, even if qemu-ppc manages to get some sort of multi-bus topology working using PXB, there is no way to expose the affinities of these busses to the guest. It can only be exposed on a per-PHB/per-domain basis. So patch enables NUMA node tag in pci-root controller on PPC. The way to set the NUMA node is through the numa_node option of spapr-pci-host-bridge device. However for the implicit PHB, the only way to set the numa_node is from the -global option. The -global option applies to all the PHBs unless explicitly specified with the option on the respective PHB of CLI. The default PHB has the emulated devices only, so the patch prevents setting the NUMA node for the default PHB. Signed-off-by: Shivaprasad G Bhat --- docs/formatdomain.html.in | 5 ++ src/conf/domain_conf.c | 6 ++ src/qemu/qemu_command.c | 10 ++++ src/qemu/qemu_domain.c | 9 +++ ...muxml2argv-spapr-pci-host-bridge-numa-node.args | 29 +++++++++++ ...emuxml2argv-spapr-pci-host-bridge-numa-node.xml | 54 ++++++++++++++++= ++++ tests/qemuxml2argvtest.c | 3 + 7 files changed, 114 insertions(+), 2 deletions(-) create mode 100644 tests/qemuxml2argvdata/qemuxml2argv-spapr-pci-host-brid= ge-numa-node.args create mode 100644 tests/qemuxml2argvdata/qemuxml2argv-spapr-pci-host-brid= ge-numa-node.xml diff --git a/docs/formatdomain.html.in b/docs/formatdomain.html.in index c12efcf..64fe241 100644 --- a/docs/formatdomain.html.in +++ b/docs/formatdomain.html.in @@ -3786,6 +3786,11 @@ part of the specified NUMA node (it is up to the user of the libvirt API to attach host devices to the correct pci-expander-bus when assigning them to the domain). + On PPC64, the PCI devices can be specified to be part of a NUMA + node using only the pci-root controller with an optional + <node> subelement within the + <target> subelement. The PCI devices on the + given pci-root controller will be part of the specified NUMA node.
index
diff --git a/src/conf/domain_conf.c b/src/conf/domain_conf.c index 3feeccb..b8cdcd1 100644 --- a/src/conf/domain_conf.c +++ b/src/conf/domain_conf.c @@ -9457,6 +9457,12 @@ virDomainControllerDefParseXML(xmlNodePtr node, goto error; } } + if (def->idx =3D=3D 0 && numaNode >=3D 0) { + virReportError(VIR_ERR_XML_ERROR, "%s", + _("Only the PCI controller with index !=3D 0 ca= n " + "have NUMA node property specified")); + goto error; + } if (numaNode >=3D 0) def->opts.pciopts.numaNode =3D numaNode; break; diff --git a/src/qemu/qemu_command.c b/src/qemu/qemu_command.c index 6ac26af..83b277b 100644 --- a/src/qemu/qemu_command.c +++ b/src/qemu/qemu_command.c @@ -3038,6 +3038,16 @@ qemuBuildControllerDevStr(const virDomainDef *domain= Def, virBufferAsprintf(&buf, "%s,index=3D%d,id=3D%s", modelName, def->opts.pciopts.targetIndex, def->info.alias); + + if (def->opts.pciopts.numaNode !=3D -1) { + if (!virQEMUCapsGet(qemuCaps, QEMU_CAPS_SPAPR_PCI_HOST_BRI= DGE_NUMA_NODE)) { + virReportError(VIR_ERR_CONFIG_UNSUPPORTED, "%s", + _("the spapr-pci-host-bridge controller= " + "doesn't support numa_node on this QE= MU binary")); + goto error; + } + virBufferAsprintf(&buf, ",numa_node=3D%d", def->opts.pciop= ts.numaNode); + } break; case VIR_DOMAIN_CONTROLLER_MODEL_PCIE_ROOT: case VIR_DOMAIN_CONTROLLER_MODEL_PCI_LAST: diff --git a/src/qemu/qemu_domain.c b/src/qemu/qemu_domain.c index 464d3a1..7732fa1 100644 --- a/src/qemu/qemu_domain.c +++ b/src/qemu/qemu_domain.c @@ -3458,9 +3458,14 @@ qemuDomainControllerDefPostParse(virDomainController= DefPtr cont, * that NUMA node is configured in the guest * array. NUMA cell id's in this array are numbered * from 0 .. size-1. + * + * On PSeries, the NUMA node is set at the PHB. */ - if ((cont->model =3D=3D VIR_DOMAIN_CONTROLLER_MODEL_PCI_EXPANDER_B= US || - cont->model =3D=3D VIR_DOMAIN_CONTROLLER_MODEL_PCIE_EXPANDER_= BUS) && + if (((qemuDomainIsPSeries(def) && + cont->type =3D=3D VIR_DOMAIN_CONTROLLER_TYPE_PCI && + cont->model =3D=3D VIR_DOMAIN_CONTROLLER_MODEL_PCI_ROOT) || + (cont->model =3D=3D VIR_DOMAIN_CONTROLLER_MODEL_PCI_EXPANDER_= BUS || + cont->model =3D=3D VIR_DOMAIN_CONTROLLER_MODEL_PCIE_EXPANDER= _BUS)) && (int) virDomainNumaGetNodeCount(def->numa) <=3D cont->opts.pciopts.numaNode) { virReportError(VIR_ERR_XML_ERROR, diff --git a/tests/qemuxml2argvdata/qemuxml2argv-spapr-pci-host-bridge-numa= -node.args b/tests/qemuxml2argvdata/qemuxml2argv-spapr-pci-host-bridge-numa= -node.args new file mode 100644 index 0000000..3e29d73 --- /dev/null +++ b/tests/qemuxml2argvdata/qemuxml2argv-spapr-pci-host-bridge-numa-node.a= rgs @@ -0,0 +1,29 @@ +LC_ALL=3DC \ +PATH=3D/bin \ +HOME=3D/home/test \ +USER=3Dtest \ +LOGNAME=3Dtest \ +QEMU_AUDIO_DRV=3Dnone \ +/usr/bin/qemu-system-ppc64 \ +-name QEMUGuest1 \ +-S \ +-M pseries \ +-m 2048 \ +-smp 8,sockets=3D3,cores=3D1,threads=3D8 \ +-numa node,nodeid=3D0,cpus=3D0-3,mem=3D1024 \ +-numa node,nodeid=3D1,cpus=3D4-7,mem=3D1024 \ +-uuid 87eedafe-eedc-4336-8130-ed9fe5dc90c8 \ +-nographic \ +-nodefaults \ +-chardev socket,id=3Dcharmonitor,path=3D/tmp/lib/domain--1-QEMUGuest1/moni= tor.sock,\ +server,nowait \ +-mon chardev=3Dcharmonitor,id=3Dmonitor,mode=3Dreadline \ +-boot c \ +-device spapr-pci-host-bridge,index=3D1,id=3Dpci.1,numa_node=3D1 \ +-device spapr-pci-host-bridge,index=3D2,id=3Dpci.2 \ +-device spapr-pci-host-bridge,index=3D3,id=3Dpci.3,numa_node=3D0 \ +-device spapr-vscsi,id=3Dscsi0,reg=3D0x2000 \ +-usb \ +-drive file=3D/dev/HostVG/QEMUGuest1,format=3Draw,if=3Dnone,id=3Ddrive-scs= i0-0-0-0 \ +-device scsi-disk,bus=3Dscsi0.0,channel=3D0,scsi-id=3D0,lun=3D0,\ +drive=3Ddrive-scsi0-0-0-0,id=3Dscsi0-0-0-0 diff --git a/tests/qemuxml2argvdata/qemuxml2argv-spapr-pci-host-bridge-numa= -node.xml b/tests/qemuxml2argvdata/qemuxml2argv-spapr-pci-host-bridge-numa-= node.xml new file mode 100644 index 0000000..566879c --- /dev/null +++ b/tests/qemuxml2argvdata/qemuxml2argv-spapr-pci-host-bridge-numa-node.x= ml @@ -0,0 +1,54 @@ + + QEMUGuest1 + 87eedafe-eedc-4336-8130-ed9fe5dc90c8 + 2097152 + 2048 + 8 + + + + + + + + + + + + hvm + + + + destroy + restart + destroy + + /usr/bin/qemu-system-ppc64 + + + + +
+ + + + + + + + + 1 + + + + + + + + 0 + + + + + + diff --git a/tests/qemuxml2argvtest.c b/tests/qemuxml2argvtest.c index b95ea46..5cfad39 100644 --- a/tests/qemuxml2argvtest.c +++ b/tests/qemuxml2argvtest.c @@ -2739,6 +2739,9 @@ mymain(void) DO_TEST_PARSE_ERROR("cpu-cache-emulate-l2", QEMU_CAPS_KVM); DO_TEST_PARSE_ERROR("cpu-cache-passthrough3", QEMU_CAPS_KVM); DO_TEST_PARSE_ERROR("cpu-cache-passthrough-l3", QEMU_CAPS_KVM); + DO_TEST("spapr-pci-host-bridge-numa-node", QEMU_CAPS_NUMA, + QEMU_CAPS_DEVICE_SPAPR_PCI_HOST_BRIDGE, + QEMU_CAPS_SPAPR_PCI_HOST_BRIDGE_NUMA_NODE); =20 if (getenv("LIBVIRT_SKIP_CLEANUP") =3D=3D NULL) virFileDeleteTree(fakerootdir); -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list