From nobody Sat Feb 7 17:14:00 2026 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) client-ip=209.132.183.28; envelope-from=libvir-list-bounces@redhat.com; helo=mx1.redhat.com; Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=fail(p=none dis=none) header.from=gmail.com ARC-Seal: i=1; a=rsa-sha256; t=1567106363; cv=none; d=zoho.com; s=zohoarc; b=YNoNoocOe2XC0jr1PLPs0KZt3x5b3phtnSSRSmLXqCbhd7O7Z+wz55h/Li5jap+/I0ZO+1ZNAnPjoc9rhKZqLmu+BmtC47M9ICOk/kio2L2xhWiV/2mmPFNx154iluSxHrO6hAfKOMe7wC6LiqpbMWtR3j7Fw1HZsXOomCy6g88= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1567106363; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results; bh=9LwwHTaZqJyRVIFZqxl46JEkLBelfs49nsrQng/5d7o=; b=P3zgUaR7+mOtzQbd2YdGX7Wuij4km5REuvaAFYkcYU8Q9svbwR7ieG2mP9ntd1KgQDsCU4JRyCgoQBggB074JDBm1/NFLv54hghMLETsE+CJO9HOLrcmT8ENlSLXHFSR9DoQtrGPRRQvpMBwXiNPN24If6Sb3Fn7ZHhjQw/yuw8= ARC-Authentication-Results: i=1; mx.zoho.com; dkim=fail; spf=pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by mx.zohomail.com with SMTPS id 1567106363719132.0245737505353; Thu, 29 Aug 2019 12:19:23 -0700 (PDT) Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id A415218001AD; Thu, 29 Aug 2019 19:19:20 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.21]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 1744E19D7A; Thu, 29 Aug 2019 19:19:20 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id A2B5424F32; Thu, 29 Aug 2019 19:19:17 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.phx2.redhat.com [10.5.11.23]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id x7TJJG4D023544 for ; Thu, 29 Aug 2019 15:19:16 -0400 Received: by smtp.corp.redhat.com (Postfix) id 06F1D196B2; Thu, 29 Aug 2019 19:19:16 +0000 (UTC) Received: from mx1.redhat.com (ext-mx13.extmail.prod.ext.phx2.redhat.com [10.5.110.42]) by smtp.corp.redhat.com (Postfix) with ESMTPS id ECF7F9D4F; Thu, 29 Aug 2019 19:19:15 +0000 (UTC) Received: from mail-qt1-f195.google.com (mail-qt1-f195.google.com [209.85.160.195]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id A65DA308212A; Thu, 29 Aug 2019 19:19:14 +0000 (UTC) Received: by mail-qt1-f195.google.com with SMTP id k13so4923047qtm.12; Thu, 29 Aug 2019 12:19:14 -0700 (PDT) Received: from rekt.ibmuc.com ([2804:431:c7c6:9589:1351:c7eb:87b:c9d5]) by smtp.gmail.com with ESMTPSA id m19sm1827976qke.22.2019.08.29.12.19.11 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 29 Aug 2019 12:19:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=/aO/tozzYJFIgcsnvVu2fR3ECXvNUuvqD1omkkavOi8=; b=omIxBUsqZPagWFmYkbLCI3z66fAUT19x/tRLX1MUj6zHOAc5nJBy+bFJ3RRPT2lUZJ JRe590qNHbgWtr1TTFCAfH8FVVOZp7V8FxvhaXustnhNM/9lmNEmTShvmkChEBFTouJP QVfH9ie5uPsi0xGBF/QgvXnRdJjJpSuzct6kBBxfA8i1MZ07vL4LHG92BZtV2BBi6Ag9 gD2Erjd8OrGhcZ3CmE2Zh2d1agUqCUnigG/BSUBUM0DgCqBKW/TxxsHJQbWVnGFFutUh jMEfL4xlJMUCo53daN+Ub3UyO+/HHdX+Kbh59nz+4mLvdNrDwpCDV5bBcBlDsHkHvzgc zY5A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=/aO/tozzYJFIgcsnvVu2fR3ECXvNUuvqD1omkkavOi8=; b=HPDUrsbipkd0MFlQE92NxBqBCF5liTb3sradg7rFCkxI1rj6cFne8xYF9hgT15BHTl FS+4RE5nYsHzsHeY6jlRhthBvyD5Xmsf/rqcPEb3MKaKHMYlLyz6g+EFVzIv0QGmXhMJ /NRYSjw8pYl7Rx/KeAZVkMPSJTTUV2ZAREhBcVlM41LdZiDKfmTMKHlWycjatCCZl76V TLvNGKunznsiC/+k0NvqKz0UmrEZTKhAAqozs+Gh/H+4aUQ6n2Zdr51H6k1GP6UCvHbN oYK3Ys4TSZrUke1By7g0VanShepfVwz+k8RtLaB95mo3XIFKeFkBrNP3OtNK8othG0+t 4xgw== X-Gm-Message-State: APjAAAXgPUPULaYi4gPa6Btz90qdTXHg57VdhDj+Xceoy0kRsyiNnvis iIIoh0P6yhnKdCDHhN6dM7nS7j++ X-Google-Smtp-Source: APXvYqwHI9j/xIeVeoOK0KNgJeF6SvT2qCJ5xiTnlpY/Pm5SvRtidKrBhJRHWtMOc3k8QVmd/u1Inw== X-Received: by 2002:ac8:1488:: with SMTP id l8mr11689816qtj.50.1567106353585; Thu, 29 Aug 2019 12:19:13 -0700 (PDT) From: Daniel Henrique Barboza To: libvir-list@redhat.com Date: Thu, 29 Aug 2019 16:18:58 -0300 Message-Id: <20190829191902.10084-2-danielhb413@gmail.com> In-Reply-To: <20190829191902.10084-1-danielhb413@gmail.com> References: <20190829191902.10084-1-danielhb413@gmail.com> MIME-Version: 1.0 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.42]); Thu, 29 Aug 2019 19:19:14 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.42]); Thu, 29 Aug 2019 19:19:14 +0000 (UTC) for IP:'209.85.160.195' DOMAIN:'mail-qt1-f195.google.com' HELO:'mail-qt1-f195.google.com' FROM:'danielhb413@gmail.com' RCPT:'' X-RedHat-Spam-Score: 0.151 (DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, FREEMAIL_ENVFROM_END_DIGIT, FREEMAIL_FROM, RCVD_IN_DNSWL_NONE, SPF_HELO_NONE, SPF_PASS) 209.85.160.195 mail-qt1-f195.google.com 209.85.160.195 mail-qt1-f195.google.com X-Scanned-By: MIMEDefang 2.84 on 10.5.110.42 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.23 X-loop: libvir-list@redhat.com Cc: mprivozn@redhat.com, Daniel Henrique Barboza Subject: [libvirt] [PATCH v3 1/5] virpcimock.c: mock /dev/vfio X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Transfer-Encoding: quoted-printable Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.84 on 10.5.11.23 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.6.2 (mx1.redhat.com [10.5.110.70]); Thu, 29 Aug 2019 19:19:22 +0000 (UTC) X-ZohoMail-DKIM: fail (Header signature does not verify) Content-Type: text/plain; charset="utf-8" This patch adds mock of the /dev/vfio path, needed for proper implementation of the support for multifunction/multiple devices per iommu groups. To do that, the existing bind and unbind operations were adapted to operate with the mocked filesystem as well. Signed-off-by: Daniel Henrique Barboza --- tests/virpcimock.c | 159 ++++++++++++++++++++++++++++++++++++++++----- 1 file changed, 143 insertions(+), 16 deletions(-) diff --git a/tests/virpcimock.c b/tests/virpcimock.c index a5045ed97c..e9440e7910 100644 --- a/tests/virpcimock.c +++ b/tests/virpcimock.c @@ -47,6 +47,7 @@ char *fakerootdir; =20 # define SYSFS_PCI_PREFIX "/sys/bus/pci/" =20 + # define STDERR(...) \ fprintf(stderr, "%s %zu: ", __FUNCTION__, (size_t) __LINE__); \ fprintf(stderr, __VA_ARGS__); \ @@ -105,6 +106,11 @@ struct pciDriver { size_t len; /* @len is used for both @vendor and @device */ }; =20 +struct pciIommuGroup { + int iommu; + size_t nDevicesBoundToVFIO; /* Indicates the devices in the group */ +}; + struct pciDeviceAddress { unsigned int domain; unsigned int bus; @@ -133,6 +139,9 @@ size_t nPCIDevices =3D 0; struct pciDriver **pciDrivers =3D NULL; size_t nPCIDrivers =3D 0; =20 +struct pciIommuGroup **pciIommuGroups =3D NULL; +size_t npciIommuGroups =3D 0; + struct fdCallback *callbacks =3D NULL; size_t nCallbacks =3D 0; =20 @@ -254,6 +263,15 @@ getrealpath(char **newpath, errno =3D ENOMEM; return -1; } + } else if (STRPREFIX(path, "/sys/kernel/") || + STRPREFIX(path, "/dev/vfio/")) { + + if (virAsprintfQuiet(newpath, "%s/%s", + fakerootdir, + path) < 0) { + errno =3D ENOMEM; + return -1; + } } else { if (VIR_STRDUP_QUIET(*newpath, path) < 0) return -1; @@ -389,8 +407,10 @@ static void pci_device_create_iommu(const struct pciDevice *dev, const char *devid) { + struct pciIommuGroup *iommuGroup; VIR_AUTOFREE(char *) iommuPath =3D NULL; char tmp[256]; + size_t i; =20 if (virAsprintfQuiet(&iommuPath, "%s/sys/kernel/iommu_groups/%d/device= s/", fakerootdir, dev->iommuGroup) < 0) @@ -406,6 +426,23 @@ pci_device_create_iommu(const struct pciDevice *dev, } =20 make_symlink(iommuPath, devid, tmp); + + /* pci_device_create_iommu can be called more than one for the + * same iommuGroup. Bail out here if the iommuGroup was already + * created beforehand. */ + for (i =3D 0; i < npciIommuGroups; i++) + if (pciIommuGroups[i]->iommu =3D=3D dev->iommuGroup) + return; + + if (VIR_ALLOC_QUIET(iommuGroup) < 0) + ABORT_OOM(); + + iommuGroup->iommu =3D dev->iommuGroup; + iommuGroup->nDevicesBoundToVFIO =3D 0; /* No device bound to VFIO by d= efault */ + + if (VIR_APPEND_ELEMENT_QUIET(pciIommuGroups, npciIommuGroups, + iommuGroup) < 0) + ABORT_OOM(); } =20 =20 @@ -558,6 +595,77 @@ pci_device_autobind(struct pciDevice *dev) return pci_driver_bind(driver, dev); } =20 +static int +pci_vfio_release_iommu(struct pciDevice *device) +{ + VIR_AUTOFREE(char *) vfiopath =3D NULL; + int ret =3D -1; + size_t i =3D 0; + + for (i =3D 0; i < npciIommuGroups; i++) { + if (pciIommuGroups[i]->iommu =3D=3D device->iommuGroup) { + + if (pciIommuGroups[i]->nDevicesBoundToVFIO =3D=3D 0) { + ret =3D 0; + goto cleanup; + } + + pciIommuGroups[i]->nDevicesBoundToVFIO--; + + if (!pciIommuGroups[i]->nDevicesBoundToVFIO) { + + if (virAsprintfQuiet(&vfiopath, "%s/dev/vfio/%d", + fakerootdir, + device->iommuGroup) < 0) { + errno =3D ENOMEM; + goto cleanup; + } + + if (unlink(vfiopath) < 0) + goto cleanup; + } + break; + } + } + + ret =3D 0; + + cleanup: + return ret; +} + +static int +pci_vfio_lock_iommu(struct pciDevice *device) +{ + VIR_AUTOFREE(char *) vfiopath =3D NULL; + int ret =3D -1; + size_t i =3D 0; + int fd =3D -1; + + for (i =3D 0; i < npciIommuGroups; i++) { + if (pciIommuGroups[i]->iommu =3D=3D device->iommuGroup) { + if (!pciIommuGroups[i]->nDevicesBoundToVFIO) { + if (virAsprintfQuiet(&vfiopath, "%s/dev/vfio/%d", + fakerootdir, + device->iommuGroup) < 0) { + errno =3D ENOMEM; + goto cleanup; + } + if ((fd =3D real_open(vfiopath, O_CREAT)) < 0) + goto cleanup; + + pciIommuGroups[i]->nDevicesBoundToVFIO++; + } + break; + } + } + + ret =3D 0; + + cleanup: + real_close(fd); + return ret; +} =20 /* * PCI Driver functions @@ -719,6 +827,10 @@ pci_driver_bind(struct pciDriver *driver, if (symlink(devpath, driverpath) < 0) return -1; =20 + if (STREQ(driver->name, "vfio-pci")) + if (pci_vfio_lock_iommu(dev) < 0) + return -1; + dev->driver =3D driver; return 0; } @@ -749,6 +861,10 @@ pci_driver_unbind(struct pciDriver *driver, unlink(driverpath) < 0) return -1; =20 + if (STREQ(driver->name, "vfio-pci")) + if (pci_vfio_release_iommu(dev) < 0) + return -1; + dev->driver =3D NULL; return 0; } @@ -865,6 +981,15 @@ init_env(void) make_dir(tmp, "drivers"); make_file(tmp, "drivers_probe", NULL, -1); =20 + /* Create /dev/vfio/ dir and /dev/vfio/vfio file */ + if (virAsprintfQuiet(&tmp, "%s/dev/vfio", fakerootdir) < 0) + ABORT_OOM(); + + if (virFileMakePath(tmp) < 0) + ABORT("Unable to create: %s", tmp); + + make_file(tmp, "vfio", NULL, -1); + # define MAKE_PCI_DRIVER(name, ...) \ pci_driver_new(name, __VA_ARGS__, -1, -1) =20 @@ -872,29 +997,31 @@ init_env(void) MAKE_PCI_DRIVER("i915", 0x8086, 0x0046, 0x8086, 0x0047); MAKE_PCI_DRIVER("vfio-pci", -1, -1); =20 -# define MAKE_PCI_DEVICE(Id, Vendor, Device, ...) \ +# define MAKE_PCI_DEVICE(Id, Vendor, Device, IommuGroup, ...) \ do { \ struct pciDevice dev =3D {.vendor =3D Vendor, \ - .device =3D Device, __VA_ARGS__}; \ + .device =3D Device, \ + .iommuGroup =3D IommuGroup, __VA_ARGS__}; \ if (pci_address_parse(&dev.addr, Id) < 0) \ ABORT("Unable to parse PCI address " Id); \ pci_device_new_from_stub(&dev); \ } while (0) =20 - MAKE_PCI_DEVICE("0000:00:00.0", 0x8086, 0x0044); - MAKE_PCI_DEVICE("0000:00:01.0", 0x8086, 0x0044); - MAKE_PCI_DEVICE("0000:00:02.0", 0x8086, 0x0046); - MAKE_PCI_DEVICE("0000:00:03.0", 0x8086, 0x0048); - MAKE_PCI_DEVICE("0001:00:00.0", 0x1014, 0x03b9, .klass =3D 0x060400); - MAKE_PCI_DEVICE("0001:01:00.0", 0x8086, 0x105e, .iommuGroup =3D 0); - MAKE_PCI_DEVICE("0001:01:00.1", 0x8086, 0x105e, .iommuGroup =3D 0); - MAKE_PCI_DEVICE("0005:80:00.0", 0x10b5, 0x8112, .klass =3D 0x060400); - MAKE_PCI_DEVICE("0005:90:01.0", 0x1033, 0x0035, .iommuGroup =3D 1); - MAKE_PCI_DEVICE("0005:90:01.1", 0x1033, 0x0035, .iommuGroup =3D 1); - MAKE_PCI_DEVICE("0005:90:01.2", 0x1033, 0x00e0, .iommuGroup =3D 1); - MAKE_PCI_DEVICE("0000:0a:01.0", 0x8086, 0x0047); - MAKE_PCI_DEVICE("0000:0a:02.0", 0x8286, 0x0048); - MAKE_PCI_DEVICE("0000:0a:03.0", 0x8386, 0x0048); + MAKE_PCI_DEVICE("0000:00:00.0", 0x8086, 0x0044, 0); + MAKE_PCI_DEVICE("0000:00:01.0", 0x8086, 0x0044, 1); + MAKE_PCI_DEVICE("0000:00:02.0", 0x8086, 0x0046, 2); + MAKE_PCI_DEVICE("0000:00:03.0", 0x8086, 0x0048, 3); + MAKE_PCI_DEVICE("0001:00:00.0", 0x1014, 0x03b9, 4, .klass =3D 0x060400= ); + MAKE_PCI_DEVICE("0001:01:00.0", 0x8086, 0x105e, 5); + MAKE_PCI_DEVICE("0001:01:00.1", 0x8086, 0x105e, 5); + MAKE_PCI_DEVICE("0005:80:00.0", 0x10b5, 0x8112, 6, .klass =3D 0x060400= ); + MAKE_PCI_DEVICE("0005:90:01.0", 0x1033, 0x0035, 7); + MAKE_PCI_DEVICE("0005:90:01.1", 0x1033, 0x0035, 7); + MAKE_PCI_DEVICE("0005:90:01.2", 0x1033, 0x00e0, 7); + MAKE_PCI_DEVICE("0005:90:01.3", 0x1033, 0x00e0, 7); + MAKE_PCI_DEVICE("0000:0a:01.0", 0x8086, 0x0047, 8); + MAKE_PCI_DEVICE("0000:0a:02.0", 0x8286, 0x0048, 8); + MAKE_PCI_DEVICE("0000:0a:03.0", 0x8386, 0x0048, 8); } =20 =20 --=20 2.21.0 -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list