From nobody Thu Nov 28 16:41:36 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) client-ip=209.132.183.28; envelope-from=libvir-list-bounces@redhat.com; helo=mx1.redhat.com; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com Return-Path: Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by mx.zohomail.com with SMTPS id 1521099435858927.0152499873662; Thu, 15 Mar 2018 00:37:15 -0700 (PDT) Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 676253DBED; Thu, 15 Mar 2018 07:37:14 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.21]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 32E1E17243; Thu, 15 Mar 2018 07:37:14 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id F06044CAB8; Thu, 15 Mar 2018 07:37:13 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id w2EHGadb024085 for ; Wed, 14 Mar 2018 13:16:48 -0400 Received: by smtp.corp.redhat.com (Postfix) id BB71A60A98; Wed, 14 Mar 2018 17:16:36 +0000 (UTC) Received: from mx1.redhat.com (ext-mx07.extmail.prod.ext.phx2.redhat.com [10.5.110.31]) by smtp.corp.redhat.com (Postfix) with ESMTPS id AD8C06090B for ; Wed, 14 Mar 2018 17:16:33 +0000 (UTC) Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 21BF9C0528C2 for ; Wed, 14 Mar 2018 17:16:32 +0000 (UTC) Received: from pps.filterd (m0098409.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.22/8.16.0.22) with SMTP id w2EHFwfe060368 for ; Wed, 14 Mar 2018 13:16:31 -0400 Received: from e06smtp10.uk.ibm.com (e06smtp10.uk.ibm.com [195.75.94.106]) by mx0a-001b2d01.pphosted.com with ESMTP id 2gq6y3ax97-1 (version=TLSv1.2 cipher=AES256-SHA256 bits=256 verify=NOT) for ; Wed, 14 Mar 2018 13:16:30 -0400 Received: from localhost by e06smtp10.uk.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Wed, 14 Mar 2018 17:16:28 -0000 Received: from b06cxnps3075.portsmouth.uk.ibm.com (9.149.109.195) by e06smtp10.uk.ibm.com (192.168.101.140) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted; Wed, 14 Mar 2018 17:16:26 -0000 Received: from d06av26.portsmouth.uk.ibm.com (d06av26.portsmouth.uk.ibm.com [9.149.105.62]) by b06cxnps3075.portsmouth.uk.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id w2EHGP0M59506866 for ; Wed, 14 Mar 2018 17:16:25 GMT Received: from d06av26.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id A47CAAE053 for ; Wed, 14 Mar 2018 17:06:52 +0000 (GMT) Received: from d06av26.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id E995CAE057 for ; Wed, 14 Mar 2018 17:06:51 +0000 (GMT) Received: from localhost.localdomain (unknown [9.79.182.166]) by d06av26.portsmouth.uk.ibm.com (Postfix) with ESMTP for ; Wed, 14 Mar 2018 17:06:51 +0000 (GMT) From: Shivaprasad G Bhat To: libvir-list@redhat.com Date: Wed, 14 Mar 2018 22:46:23 +0530 In-Reply-To: <152104711186.10112.1077788328340024644.stgit@localhost.localdomain> References: <152104711186.10112.1077788328340024644.stgit@localhost.localdomain> User-Agent: StGit/0.17.1-dirty MIME-Version: 1.0 X-TM-AS-GCONF: 00 x-cbid: 18031417-0040-0000-0000-00000420797D X-IBM-AV-DETECTION: SAVI=unused REMOTE=unused XFE=unused x-cbparentid: 18031417-0041-0000-0000-000026237EE4 Message-Id: <152104777699.10112.15553401507045394764.stgit@localhost.localdomain> X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:, , definitions=2018-03-14_09:, , signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 suspectscore=3 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 impostorscore=0 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1709140000 definitions=main-1803140192 X-Greylist: Sender passed SPF test, Sender IP whitelisted by DNSRBL, ACL 207 matched, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.31]); Wed, 14 Mar 2018 17:16:32 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.31]); Wed, 14 Mar 2018 17:16:32 +0000 (UTC) for IP:'148.163.156.1' DOMAIN:'mx0a-001b2d01.pphosted.com' HELO:'mx0a-001b2d01.pphosted.com' FROM:'sbhat@linux.vnet.ibm.com' RCPT:'' X-RedHat-Spam-Score: -0.701 (RCVD_IN_DNSWL_LOW, RCVD_IN_MSPIKE_H2) 148.163.156.1 mx0a-001b2d01.pphosted.com 148.163.156.1 mx0a-001b2d01.pphosted.com X-Scanned-By: MIMEDefang 2.78 on 10.5.110.31 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 X-loop: libvir-list@redhat.com Subject: [libvirt] [RFC PATCH 03/28] tests: pci: Mock the iommu groups and vfio X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Transfer-Encoding: quoted-printable Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.30]); Thu, 15 Mar 2018 07:37:14 +0000 (UTC) X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Type: text/plain; charset="utf-8" The iommu group, /dev/vfio/ behaviours of the host are mocked. This patch implments support for multifunction/multiple devices per iommu groups and emulates the /dev/vfio/ file correctly. This code helps adding necessary testcases for pci-hotplug code. Signed-off-by: Shivaprasad G Bhat --- tests/virpcimock.c | 178 ++++++++++++++++++++++++++= +--- tests/virpcitestdata/0005-90-01.1.config | Bin tests/virpcitestdata/0005-90-01.2.config | Bin tests/virpcitestdata/0005-90-01.3.config | Bin 4 files changed, 162 insertions(+), 16 deletions(-) create mode 100644 tests/virpcitestdata/0005-90-01.3.config diff --git a/tests/virpcimock.c b/tests/virpcimock.c index 2a7e9216b2..22adc740b8 100644 --- a/tests/virpcimock.c +++ b/tests/virpcimock.c @@ -51,7 +51,13 @@ static DIR * (*real_opendir)(const char *name); char *fakerootdir; char *fakesysfspcidir; =20 +struct pciIommuGroup { + int iommu; + size_t nDevicesBoundToVFIO; /* Indicates the devices in the gro= up */ +}; + # define SYSFS_PCI_PREFIX "/sys/bus/pci/" +# define SYSFS_KERNEL_PREFIX "/sys/kernel/" =20 # define STDERR(...) \ fprintf(stderr, "%s %zu: ", __FUNCTION__, (size_t) __LINE__); \ @@ -142,6 +148,9 @@ size_t nPCIDevices =3D 0; struct pciDriver **pciDrivers =3D NULL; size_t nPCIDrivers =3D 0; =20 +struct pciIommuGroup **pciIommuGroups =3D NULL; +size_t npciIommuGroups =3D 0; + struct fdCallback *callbacks =3D NULL; size_t nCallbacks =3D 0; =20 @@ -191,7 +200,7 @@ make_file(const char *path, VIR_FREE(filepath); } =20 -static void +static void ATTRIBUTE_UNUSED make_symlink(const char *path, const char *name, const char *target) @@ -255,6 +264,13 @@ getrealpath(char **newpath, errno =3D ENOMEM; return -1; } + } else if (STRPREFIX(path, SYSFS_KERNEL_PREFIX)) { + if (virAsprintfQuiet(newpath, "%s/%s", + fakerootdir, + path) < 0) { + errno =3D ENOMEM; + return -1; + } } else { if (VIR_STRDUP_QUIET(*newpath, path) < 0) return -1; @@ -465,6 +481,101 @@ pci_device_autobind(struct pciDevice *dev) return pci_driver_bind(driver, dev); } =20 +static void +pci_iommu_new(int num) +{ + char *iommupath, *kerneldir; + struct pciIommuGroup *iommuGroup; + + if (VIR_ALLOC_QUIET(iommuGroup) < 0) + ABORT_OOM(); + + iommuGroup->iommu =3D num; + iommuGroup->nDevicesBoundToVFIO =3D 0; /* No device bound to vfio by d= efault */ + + if (virAsprintfQuiet(&kerneldir, "%s%s", + fakerootdir, SYSFS_KERNEL_PREFIX) < 0) + ABORT_OOM(); + + if (virAsprintfQuiet(&iommupath, "%s/iommu_groups/%d/devices", kerneld= ir, num) < 0) + ABORT_OOM(); + VIR_FREE(kerneldir); + + if (virFileMakePath(iommupath) < 0) + ABORT("Unable to create: %s", iommupath); + VIR_FREE(iommupath); + + if (VIR_APPEND_ELEMENT_QUIET(pciIommuGroups, npciIommuGroups, iommuGro= up) < 0) + ABORT_OOM(); +} + +static int +pci_vfio_release_iommu(struct pciDevice *device) +{ + char *vfiopath =3D NULL; + int ret =3D -1; + size_t i =3D 0; + + for (i =3D 0; i < npciIommuGroups; i++) { + if (pciIommuGroups[i]->iommu =3D=3D device->iommuGroup) + break; + } + + if (i !=3D npciIommuGroups) { + if (pciIommuGroups[i]->nDevicesBoundToVFIO =3D=3D 0) { + ret =3D 0; + goto cleanup; + } + pciIommuGroups[i]->nDevicesBoundToVFIO--; + if (!pciIommuGroups[i]->nDevicesBoundToVFIO) { + if (virAsprintfQuiet(&vfiopath, "%s/dev/vfio/%d", + fakesysfspcidir, device->iommuGroup) < 0) { + errno =3D ENOMEM; + goto cleanup; + } + if (unlink(vfiopath) < 0) + goto cleanup; + } + } + + ret =3D 0; + cleanup: + VIR_FREE(vfiopath); + return ret; +} + +static int +pci_vfio_lock_iommu(struct pciDevice *device) +{ + char *vfiopath =3D NULL; + int ret =3D -1; + size_t i =3D 0; + int fd =3D -1; + + for (i =3D 0; i < npciIommuGroups; i++) { + if (pciIommuGroups[i]->iommu =3D=3D device->iommuGroup) + break; + } + + if (i !=3D npciIommuGroups) { + if (!pciIommuGroups[i]->nDevicesBoundToVFIO) { + if (virAsprintfQuiet(&vfiopath, "%s/dev/vfio/%d", + fakesysfspcidir, device->iommuGroup) < 0) { + errno =3D ENOMEM; + goto cleanup; + } + if ((fd =3D real_open(vfiopath, O_CREAT)) < 0) + goto cleanup; + } + pciIommuGroups[i]->nDevicesBoundToVFIO++; + } + + ret =3D 0; + cleanup: + real_close(fd); + VIR_FREE(vfiopath); + return ret; +} =20 /* * PCI Driver functions @@ -588,6 +699,10 @@ pci_driver_bind(struct pciDriver *driver, if (symlink(devpath, driverpath) < 0) goto cleanup; =20 + if (STREQ(driver->name, "vfio-pci")) + if (pci_vfio_lock_iommu(dev) < 0) + goto cleanup; + dev->driver =3D driver; ret =3D 0; cleanup: @@ -622,6 +737,10 @@ pci_driver_unbind(struct pciDriver *driver, unlink(driverpath) < 0) goto cleanup; =20 + if (STREQ(driver->name, "vfio-pci")) + if (pci_vfio_release_iommu(dev) < 0) + goto cleanup; + dev->driver =3D NULL; ret =3D 0; cleanup: @@ -819,6 +938,8 @@ init_syms(void) static void init_env(void) { + char *devvfio; + if (fakerootdir && fakesysfspcidir) return; =20 @@ -833,6 +954,28 @@ init_env(void) ABORT("Unable to create: %s", fakesysfspcidir); =20 make_file(fakesysfspcidir, "drivers_probe", NULL, -1); + if (virAsprintfQuiet(&devvfio, "%s/dev/vfio", fakesysfspcidir) < 0) + ABORT_OOM(); + + if (virFileMakePath(devvfio) < 0) + ABORT("Unable to create: %s", devvfio); + + /* Create /dev/vfio/vfio file */ + make_file(devvfio, "vfio", NULL, -1); + VIR_FREE(devvfio); + + pci_iommu_new(0); + pci_iommu_new(1); + pci_iommu_new(2); + pci_iommu_new(3); + pci_iommu_new(4); + pci_iommu_new(5); + pci_iommu_new(6); + pci_iommu_new(7); + pci_iommu_new(8); + pci_iommu_new(9); + pci_iommu_new(10); + pci_iommu_new(11); =20 # define MAKE_PCI_DRIVER(name, ...) \ pci_driver_new(name, 0, __VA_ARGS__, -1, -1) @@ -842,6 +985,7 @@ init_env(void) MAKE_PCI_DRIVER("pci-stub", -1, -1); pci_driver_new("vfio-pci", PCI_ACTION_BIND, -1, -1); =20 + # define MAKE_PCI_DEVICE(Id, Vendor, Device, ...) \ do { \ struct pciDevice dev =3D {.id =3D (char *)Id, .vendor =3D Vendor, \ @@ -849,20 +993,21 @@ init_env(void) pci_device_new_from_stub(&dev); \ } while (0) =20 - MAKE_PCI_DEVICE("0000:00:00.0", 0x8086, 0x0044); - MAKE_PCI_DEVICE("0000:00:01.0", 0x8086, 0x0044); - MAKE_PCI_DEVICE("0000:00:02.0", 0x8086, 0x0046); - MAKE_PCI_DEVICE("0000:00:03.0", 0x8086, 0x0048); - MAKE_PCI_DEVICE("0001:00:00.0", 0x1014, 0x03b9, .class =3D 0x060400); - MAKE_PCI_DEVICE("0001:01:00.0", 0x8086, 0x105e, .iommuGroup =3D 0); - MAKE_PCI_DEVICE("0001:01:00.1", 0x8086, 0x105e, .iommuGroup =3D 0); - MAKE_PCI_DEVICE("0005:80:00.0", 0x10b5, 0x8112, .class =3D 0x060400); - MAKE_PCI_DEVICE("0005:90:01.0", 0x1033, 0x0035, .iommuGroup =3D 1); - MAKE_PCI_DEVICE("0005:90:01.1", 0x1033, 0x0035, .iommuGroup =3D 1); - MAKE_PCI_DEVICE("0005:90:01.2", 0x1033, 0x00e0, .iommuGroup =3D 1); - MAKE_PCI_DEVICE("0000:0a:01.0", 0x8086, 0x0047); - MAKE_PCI_DEVICE("0000:0a:02.0", 0x8286, 0x0048); - MAKE_PCI_DEVICE("0000:0a:03.0", 0x8386, 0x0048); + MAKE_PCI_DEVICE("0000:00:00.0", 0x8086, 0x0044, .iommuGroup =3D 0); + MAKE_PCI_DEVICE("0000:00:01.0", 0x8086, 0x0044, .iommuGroup =3D 1); + MAKE_PCI_DEVICE("0000:00:02.0", 0x8086, 0x0046, .iommuGroup =3D 2); + MAKE_PCI_DEVICE("0000:00:03.0", 0x8086, 0x0048, .iommuGroup =3D 3); + MAKE_PCI_DEVICE("0001:00:00.0", 0x1014, 0x03b9, .class =3D 0x060400, .= iommuGroup =3D 4); + MAKE_PCI_DEVICE("0001:01:00.0", 0x8086, 0x105e, .iommuGroup =3D 5); + MAKE_PCI_DEVICE("0001:01:00.1", 0x8086, 0x105e, .iommuGroup =3D 5); + MAKE_PCI_DEVICE("0005:80:00.0", 0x10b5, 0x8112, .class =3D 0x060400, .= iommuGroup =3D 6); + MAKE_PCI_DEVICE("0005:90:01.0", 0x1033, 0x0035, .iommuGroup =3D 7); + MAKE_PCI_DEVICE("0005:90:01.1", 0x1033, 0x0035, .iommuGroup =3D 7); + MAKE_PCI_DEVICE("0005:90:01.2", 0x1033, 0x00e0, .iommuGroup =3D 7); + MAKE_PCI_DEVICE("0005:90:01.3", 0x1033, 0x00e0, .iommuGroup =3D 7); + MAKE_PCI_DEVICE("0000:0a:01.0", 0x8086, 0x0047, .iommuGroup =3D 8); + MAKE_PCI_DEVICE("0000:0a:02.0", 0x8286, 0x0048, .iommuGroup =3D 8); + MAKE_PCI_DEVICE("0000:0a:03.0", 0x8386, 0x0048, .iommuGroup =3D 8); } =20 =20 @@ -1029,7 +1174,8 @@ opendir(const char *path) =20 init_syms(); =20 - if (STRPREFIX(path, SYSFS_PCI_PREFIX) && + if ((STRPREFIX(path, SYSFS_PCI_PREFIX) || + STRPREFIX(path, SYSFS_KERNEL_PREFIX)) && getrealpath(&newpath, path) < 0) return NULL; =20 diff --git a/tests/virpcitestdata/0005-90-01.1.config b/tests/virpcitestdat= a/0005-90-01.1.config index beee76534041a7020c08ae9ac03d9a349c6ea12e..a60599bd342d3ebcdc7b8367ca3= 6ad337f602fde 100644 GIT binary patch delta 44 ycmZo*YG4vE7BFRCV-R3+7GUOK;Amg~f~JXq5)*X<85t+qEn;Cc-jFjfPzC^<7YL>R delta 39 ucmZo*YG4vE7BFRCV-R3+7GUOK;9y{25MXGU7$`AON05CJ=3D!Q7z5RUfCHEW5{!&mj0{Y5Fz#TaS&cX3;ByxM DCDjDB literal 256 zcmXpOc)-BMAi%_;z|6zo!oa|wz|aIFu>xbDS`csmlR$!5K#7rosSd`)Mk^@TV-u#E T7_0Gy9FS#<5E~CbCCJ=3D!Q7z5RUfCHEW5{!&mj0{Y5Fz#TaS&cX3;ByxM DCDjDB literal 0 HcmV?d00001 -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list