From nobody Wed Nov 5 05:19:03 2025 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1505155101744275.88290827088997; Mon, 11 Sep 2017 11:38:21 -0700 (PDT) Received: from localhost ([::1]:59929 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1drTbA-0002ed-VH for importer@patchew.org; Mon, 11 Sep 2017 14:38:21 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:42353) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1drTZi-0001RP-6r for qemu-devel@nongnu.org; Mon, 11 Sep 2017 14:36:51 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1drTZd-0005iv-Co for qemu-devel@nongnu.org; Mon, 11 Sep 2017 14:36:50 -0400 Received: from mx1.redhat.com ([209.132.183.28]:33648) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1drTZd-0005ih-4I for qemu-devel@nongnu.org; Mon, 11 Sep 2017 14:36:45 -0400 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 2F897750EE for ; Mon, 11 Sep 2017 18:36:44 +0000 (UTC) Received: from Wei-Dev.nay.redhat.com (dhcp-15-231.nay.redhat.com [10.66.15.231]) by smtp.corp.redhat.com (Postfix) with ESMTP id 841F2600C8; Mon, 11 Sep 2017 18:36:42 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com 2F897750EE Authentication-Results: ext-mx10.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx10.extmail.prod.ext.phx2.redhat.com; spf=fail smtp.mailfrom=wexu@redhat.com From: wexu@redhat.com To: qemu-devel@nongnu.org Date: Tue, 12 Sep 2017 02:56:30 +0800 Message-Id: <1505156192-18994-2-git-send-email-wexu@redhat.com> In-Reply-To: <1505156192-18994-1-git-send-email-wexu@redhat.com> References: <1505156192-18994-1-git-send-email-wexu@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.39]); Mon, 11 Sep 2017 18:36:44 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [Patch 1/3] vfio: reusing address space for the same IOMMU group devices X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: marcel@redhat.com, jasowang@redhat.com, alex.williamson@redhat.com, Wei Xu Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: Wei Xu Currently address space of a vfio device is selected by directly looking up pci device IOMMU address space during realizing, this usually works for most none separate address space targeted cases since they are using the system address space, i.e. a q35 machine without virtual IOMMU. Unfortunately, when it comes down to the case having a virtual IOMMU(x86 vtd in this case) and two vfio devices in the same IOMMU group, the virtual IOMMU(vtd) creates two separate address space for each device, this breaks the minimum granularity for vfio, and the device fails realizing by prompting 'group xxx used in multiple address spaces'. This patch is a helper looking up the same IOMMU device before invoking creating an new address space for a device, thus fixes the issue. As a side work for the all groups/devices loop, also it checks if the device has been assigned to the guest twice before creating an extra group and removing it later which is not necessary. Signed-off-by: Wei Xu --- hw/vfio/common.c | 28 ++++++++++++++++++++++++++++ include/hw/vfio/vfio-common.h | 1 + 2 files changed, 29 insertions(+) diff --git a/hw/vfio/common.c b/hw/vfio/common.c index 7b2924c..63c3609 100644 --- a/hw/vfio/common.c +++ b/hw/vfio/common.c @@ -35,6 +35,7 @@ #include "sysemu/kvm.h" #include "trace.h" #include "qapi/error.h" +#include "hw/vfio/pci.h" =20 struct vfio_group_head vfio_group_list =3D QLIST_HEAD_INITIALIZER(vfio_group_list); @@ -1183,6 +1184,33 @@ static void vfio_disconnect_container(VFIOGroup *gro= up) } } =20 +AddressSpace *vfio_lookup_as(int groupid, PCIDevice *pdev, Error **errp) +{ + VFIOGroup *group; + VFIODevice *vbasedev_iter; + VFIOPCIDevice *vdev, *vd; + + vdev =3D DO_UPCAST(VFIOPCIDevice, pdev, pdev); + QLIST_FOREACH(group, &vfio_group_list, next) { + QLIST_FOREACH(vbasedev_iter, &group->device_list, next) { + if (strcmp(vbasedev_iter->name, vdev->vbasedev.name) =3D=3D 0)= { + error_setg(errp, "device is already attached"); + return 0; + } + + if (vbasedev_iter->group->groupid =3D=3D groupid) { + vd =3D container_of(vbasedev_iter, VFIOPCIDevice, vbasedev= ); + + if (vd->pdev.bus =3D=3D pdev->bus) { + return vbasedev_iter->group->container->space->as; + } + } + } + } + + return pci_device_iommu_address_space(pdev); +} + VFIOGroup *vfio_get_group(int groupid, AddressSpace *as, Error **errp) { VFIOGroup *group; diff --git a/include/hw/vfio/vfio-common.h b/include/hw/vfio/vfio-common.h index f3a2ac9..5b4827b 100644 --- a/include/hw/vfio/vfio-common.h +++ b/include/hw/vfio/vfio-common.h @@ -157,6 +157,7 @@ void vfio_region_mmaps_set_enabled(VFIORegion *region, = bool enabled); void vfio_region_exit(VFIORegion *region); void vfio_region_finalize(VFIORegion *region); void vfio_reset_handler(void *opaque); +AddressSpace *vfio_lookup_as(int groupid, PCIDevice *pdev, Error **errp); VFIOGroup *vfio_get_group(int groupid, AddressSpace *as, Error **errp); void vfio_put_group(VFIOGroup *group); int vfio_get_device(VFIOGroup *group, const char *name, --=20 1.8.3.1 From nobody Wed Nov 5 05:19:03 2025 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 150515510278030.66745583625584; Mon, 11 Sep 2017 11:38:22 -0700 (PDT) Received: from localhost ([::1]:59931 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1drTbB-0002gW-MR for importer@patchew.org; Mon, 11 Sep 2017 14:38:21 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:42354) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1drTZi-0001RQ-7U for qemu-devel@nongnu.org; Mon, 11 Sep 2017 14:36:51 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1drTZh-0005kB-0w for qemu-devel@nongnu.org; Mon, 11 Sep 2017 14:36:50 -0400 Received: from mx1.redhat.com ([209.132.183.28]:53316) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1drTZg-0005jo-QU for qemu-devel@nongnu.org; Mon, 11 Sep 2017 14:36:48 -0400 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id D7DC2C07F67E for ; Mon, 11 Sep 2017 18:36:47 +0000 (UTC) Received: from Wei-Dev.nay.redhat.com (dhcp-15-231.nay.redhat.com [10.66.15.231]) by smtp.corp.redhat.com (Postfix) with ESMTP id AB90A600C4; Mon, 11 Sep 2017 18:36:44 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com D7DC2C07F67E Authentication-Results: ext-mx07.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx07.extmail.prod.ext.phx2.redhat.com; spf=fail smtp.mailfrom=wexu@redhat.com From: wexu@redhat.com To: qemu-devel@nongnu.org Date: Tue, 12 Sep 2017 02:56:31 +0800 Message-Id: <1505156192-18994-3-git-send-email-wexu@redhat.com> In-Reply-To: <1505156192-18994-1-git-send-email-wexu@redhat.com> References: <1505156192-18994-1-git-send-email-wexu@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.31]); Mon, 11 Sep 2017 18:36:48 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [Patch 2/3] vfio: invoke looking up address space. X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: marcel@redhat.com, jasowang@redhat.com, alex.williamson@redhat.com, Wei Xu Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: Wei Xu Invoke looking up correct address space before getting an IOMMU group. Signed-off-by: Wei Xu --- hw/vfio/pci.c | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/hw/vfio/pci.c b/hw/vfio/pci.c index 31e1edf..856cefd 100644 --- a/hw/vfio/pci.c +++ b/hw/vfio/pci.c @@ -2640,6 +2640,7 @@ static void vfio_realize(PCIDevice *pdev, Error **err= p) struct stat st; int groupid; int i, ret; + AddressSpace *as; =20 if (!vdev->vbasedev.sysfsdev) { if (!(~vdev->host.domain || ~vdev->host.bus || @@ -2686,7 +2687,12 @@ static void vfio_realize(PCIDevice *pdev, Error **er= rp) =20 trace_vfio_realize(vdev->vbasedev.name, groupid); =20 - group =3D vfio_get_group(groupid, pci_device_iommu_address_space(pdev)= , errp); + as =3D vfio_lookup_as(groupid, pdev, errp); + if (!as) { + goto error; + } + + group =3D vfio_get_group(groupid, as, errp); if (!group) { goto error; } --=20 1.8.3.1 From nobody Wed Nov 5 05:19:03 2025 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1505155214230196.6247935080811; Mon, 11 Sep 2017 11:40:14 -0700 (PDT) Received: from localhost ([::1]:59938 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1drTcz-0004Ps-Fn for importer@patchew.org; Mon, 11 Sep 2017 14:40:13 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:42385) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1drTZk-0001Rq-8A for qemu-devel@nongnu.org; Mon, 11 Sep 2017 14:36:53 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1drTZj-0005lL-Gj for qemu-devel@nongnu.org; Mon, 11 Sep 2017 14:36:52 -0400 Received: from mx1.redhat.com ([209.132.183.28]:54212) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1drTZj-0005ky-AZ for qemu-devel@nongnu.org; Mon, 11 Sep 2017 14:36:51 -0400 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 5F0C58AE7F for ; Mon, 11 Sep 2017 18:36:50 +0000 (UTC) Received: from Wei-Dev.nay.redhat.com (dhcp-15-231.nay.redhat.com [10.66.15.231]) by smtp.corp.redhat.com (Postfix) with ESMTP id 6175E600C4; Mon, 11 Sep 2017 18:36:48 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com 5F0C58AE7F Authentication-Results: ext-mx03.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx03.extmail.prod.ext.phx2.redhat.com; spf=fail smtp.mailfrom=wexu@redhat.com From: wexu@redhat.com To: qemu-devel@nongnu.org Date: Tue, 12 Sep 2017 02:56:32 +0800 Message-Id: <1505156192-18994-4-git-send-email-wexu@redhat.com> In-Reply-To: <1505156192-18994-1-git-send-email-wexu@redhat.com> References: <1505156192-18994-1-git-send-email-wexu@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.27]); Mon, 11 Sep 2017 18:36:50 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [Patch 3/3] vfio: remove checking duplicated vfio device X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: marcel@redhat.com, jasowang@redhat.com, alex.williamson@redhat.com, Wei Xu Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: Wei Xu This has been done when introducing 'vfio_lookup_as()' patch as a side work to reuse the loop. Signed-off-by: Wei Xu --- hw/vfio/pci.c | 9 --------- 1 file changed, 9 deletions(-) diff --git a/hw/vfio/pci.c b/hw/vfio/pci.c index 856cefd..d78f756 100644 --- a/hw/vfio/pci.c +++ b/hw/vfio/pci.c @@ -2632,7 +2632,6 @@ static void vfio_unregister_req_notifier(VFIOPCIDevic= e *vdev) static void vfio_realize(PCIDevice *pdev, Error **errp) { VFIOPCIDevice *vdev =3D DO_UPCAST(VFIOPCIDevice, pdev, pdev); - VFIODevice *vbasedev_iter; VFIOGroup *group; char *tmp, group_path[PATH_MAX], *group_name; Error *err =3D NULL; @@ -2697,14 +2696,6 @@ static void vfio_realize(PCIDevice *pdev, Error **er= rp) goto error; } =20 - QLIST_FOREACH(vbasedev_iter, &group->device_list, next) { - if (strcmp(vbasedev_iter->name, vdev->vbasedev.name) =3D=3D 0) { - error_setg(errp, "device is already attached"); - vfio_put_group(group); - goto error; - } - } - ret =3D vfio_get_device(group, vdev->vbasedev.name, &vdev->vbasedev, e= rrp); if (ret) { vfio_put_group(group); --=20 1.8.3.1