From nobody Tue Feb 10 00:40:00 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=intel.com ARC-Seal: i=1; a=rsa-sha256; t=1614689878; cv=none; d=zohomail.com; s=zohoarc; b=W3XVZ2TT0DobDotQ1eTfurv+VMD8AGXS8aIa9X1GR8/+3Nf5Y0GovCcvl14aqhw5WhuEnVX3lBDhCYU7rWbDhFbv+iXBVNyIAgUHW5F4F444/YFEO/BsBWfj4vVstBe7AXq1qT/XCkqKFVCcyv3+mrTVRVCQ0P/wOsz9ndTTaBo= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1614689878; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=c3hUPo2X6i16ZZQC6gbhnNZPWixruM4yrKwWYS/mQ6A=; b=InbfUGUDB0fU4lFXzmmeWV1ENAonvVJFnsmmaB2CKHckJjtRmn9SRvyYYgWvEkOTLtOAfZVE/q/at0qLmpEI7KQSsYjEzOnad8Pv3cY1gIETN1gQvMbwzaMjj5mEVL2DOSCHHJixtxASH+EJMSIBeUrEokJBiBjN2ETKCSiWGDY= ARC-Authentication-Results: i=1; mx.zohomail.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 161468987810532.49914271161583; Tue, 2 Mar 2021 04:57:58 -0800 (PST) Received: from localhost ([::1]:56180 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lH4b6-0003Gr-Ro for importer@patchew.org; Tue, 02 Mar 2021 07:57:56 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:33048) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lH4KG-0007sh-O8 for qemu-devel@nongnu.org; Tue, 02 Mar 2021 07:40:32 -0500 Received: from mga05.intel.com ([192.55.52.43]:62468) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lH4KE-0002nT-CB for qemu-devel@nongnu.org; Tue, 02 Mar 2021 07:40:32 -0500 Received: from fmsmga004.fm.intel.com ([10.253.24.48]) by fmsmga105.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 02 Mar 2021 04:40:29 -0800 Received: from yiliu-dev.bj.intel.com (HELO dual-ub.bj.intel.com) ([10.238.156.135]) by fmsmga004.fm.intel.com with ESMTP; 02 Mar 2021 04:40:25 -0800 IronPort-SDR: 8wC4Z6XACe+XY47Sq2mfk2GtnvQakkmx7AXxdFasE4noT+5FaLCjrFYsmtzuF3UtAyVL+m9pM6 hoOfjRDL3QTQ== X-IronPort-AV: E=McAfee;i="6000,8403,9910"; a="271803131" X-IronPort-AV: E=Sophos;i="5.81,216,1610438400"; d="scan'208";a="271803131" IronPort-SDR: 2rBwXi9CGAqzBAfgjGa2/z9Z7RLcfK9+aaBOHTgVIBWhWDpMD0yV419aGiITwQLR0Jexxae9Nv r47dGVFbNZGQ== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.81,216,1610438400"; d="scan'208";a="427472958" From: Liu Yi L To: qemu-devel@nongnu.org, alex.williamson@redhat.com, peterx@redhat.com, jasowang@redhat.com Subject: [RFC v11 13/25] vfio: init HostIOMMUContext per-container Date: Wed, 3 Mar 2021 04:38:15 +0800 Message-Id: <20210302203827.437645-14-yi.l.liu@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210302203827.437645-1-yi.l.liu@intel.com> References: <20210302203827.437645-1-yi.l.liu@intel.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=192.55.52.43; envelope-from=yi.l.liu@intel.com; helo=mga05.intel.com X-Spam_score_int: -22 X-Spam_score: -2.3 X-Spam_bar: -- X-Spam_report: (-2.3 / 5.0 requ) BAYES_00=-1.9, DATE_IN_FUTURE_06_12=1.947, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: jean-philippe@linaro.org, kevin.tian@intel.com, yi.l.liu@intel.com, Yi Sun , kvm@vger.kernel.org, mst@redhat.com, jun.j.tian@intel.com, eric.auger@redhat.com, yi.y.sun@intel.com, Jacob Pan , pbonzini@redhat.com, Lingshan.Zhu@intel.com, hao.wu@intel.com, david@gibson.dropbear.id.au Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Type: text/plain; charset="utf-8" In this patch, QEMU firstly gets iommu info from kernel to check the supported capabilities by a VFIO_IOMMU_TYPE1_NESTING iommu. And inits HostIOMMUContet instance. For vfio-pci devices, it could use pci_device_set/unset_iommu() to expose host iommu context to vIOMMU emulators. vIOMMU emulators could make use of the methods provided by host iommu context. e.g. propagate requests to host iommu. Cc: Kevin Tian Cc: Jacob Pan Cc: Peter Xu Cc: Eric Auger Cc: Yi Sun Cc: David Gibson Cc: Alex Williamson Signed-off-by: Liu Yi L --- hw/vfio/common.c | 135 +++++++++++++++++++++++++--------- hw/vfio/pci.c | 17 +++++ include/hw/vfio/vfio-common.h | 1 + 3 files changed, 118 insertions(+), 35 deletions(-) diff --git a/hw/vfio/common.c b/hw/vfio/common.c index 433938c245..a12708bcb7 100644 --- a/hw/vfio/common.c +++ b/hw/vfio/common.c @@ -1623,41 +1623,11 @@ static int vfio_host_iommu_ctx_unbind_stage1_pgtbl(= HostIOMMUContext *iommu_ctx, return ret; } =20 -static int vfio_init_container(VFIOContainer *container, int group_fd, - bool want_nested, Error **errp) -{ - int iommu_type, ret; - - iommu_type =3D vfio_get_iommu_type(container, want_nested, errp); - if (iommu_type < 0) { - return iommu_type; - } - - ret =3D ioctl(group_fd, VFIO_GROUP_SET_CONTAINER, &container->fd); - if (ret) { - error_setg_errno(errp, errno, "Failed to set group container"); - return -errno; - } - - while (ioctl(container->fd, VFIO_SET_IOMMU, iommu_type)) { - if (iommu_type =3D=3D VFIO_SPAPR_TCE_v2_IOMMU) { - /* - * On sPAPR, despite the IOMMU subdriver always advertises v1 = and - * v2, the running platform may not support v2 and there is no - * way to guess it until an IOMMU group gets added to the cont= ainer. - * So in case it fails with v2, try v1 as a fallback. - */ - iommu_type =3D VFIO_SPAPR_TCE_IOMMU; - continue; - } - error_setg_errno(errp, errno, "Failed to set iommu for container"); - return -errno; - } - - container->iommu_type =3D iommu_type; - return 0; -} - +/** + * Get iommu info from host. Caller of this funcion should free + * the memory pointed by the returned pointer stored in @info + * after a successful calling when finished its usage. + */ static int vfio_get_iommu_info(VFIOContainer *container, struct vfio_iommu_type1_info **info) { @@ -1702,6 +1672,101 @@ vfio_get_iommu_info_cap(struct vfio_iommu_type1_inf= o *info, uint16_t id) return NULL; } =20 +static int vfio_get_nesting_iommu_cap(VFIOContainer *container, + struct vfio_iommu_type1_info_cap_nesting **cap_nesting) +{ + struct vfio_iommu_type1_info *info; + struct vfio_info_cap_header *hdr; + struct vfio_iommu_type1_info_cap_nesting *cap; + struct iommu_nesting_info *nest_info; + int ret; + uint32_t minsz, cap_size; + + ret =3D vfio_get_iommu_info(container, &info); + if (ret) { + return ret; + } + + hdr =3D vfio_get_iommu_info_cap(info, + VFIO_IOMMU_TYPE1_INFO_CAP_NESTING); + if (!hdr) { + g_free(info); + return -EINVAL; + } + + cap =3D container_of(hdr, + struct vfio_iommu_type1_info_cap_nesting, header); + + nest_info =3D &cap->info; + minsz =3D offsetof(struct iommu_nesting_info, vendor); + if (nest_info->argsz < minsz) { + g_free(info); + return -EINVAL; + } + + cap_size =3D offsetof(struct vfio_iommu_type1_info_cap_nesting, info) + + nest_info->argsz; + *cap_nesting =3D g_malloc0(cap_size); + memcpy(*cap_nesting, cap, cap_size); + + g_free(info); + return 0; +} + +static int vfio_init_container(VFIOContainer *container, int group_fd, + bool want_nested, Error **errp) +{ + int iommu_type, ret; + + iommu_type =3D vfio_get_iommu_type(container, want_nested, errp); + if (iommu_type < 0) { + return iommu_type; + } + + ret =3D ioctl(group_fd, VFIO_GROUP_SET_CONTAINER, &container->fd); + if (ret) { + error_setg_errno(errp, errno, "Failed to set group container"); + return -errno; + } + + while (ioctl(container->fd, VFIO_SET_IOMMU, iommu_type)) { + if (iommu_type =3D=3D VFIO_SPAPR_TCE_v2_IOMMU) { + /* + * On sPAPR, despite the IOMMU subdriver always advertises v1 = and + * v2, the running platform may not support v2 and there is no + * way to guess it until an IOMMU group gets added to the cont= ainer. + * So in case it fails with v2, try v1 as a fallback. + */ + iommu_type =3D VFIO_SPAPR_TCE_IOMMU; + continue; + } + error_setg_errno(errp, errno, "Failed to set iommu for container"); + return -errno; + } + + if (iommu_type =3D=3D VFIO_TYPE1_NESTING_IOMMU) { + struct vfio_iommu_type1_info_cap_nesting *nesting =3D NULL; + struct iommu_nesting_info *nest_info; + + ret =3D vfio_get_nesting_iommu_cap(container, &nesting); + if (ret) { + error_setg_errno(errp, -ret, + "Failed to get nesting iommu cap"); + return ret; + } + + nest_info =3D (struct iommu_nesting_info *) &nesting->info; + host_iommu_ctx_init(&container->iommu_ctx, + sizeof(container->iommu_ctx), + TYPE_VFIO_HOST_IOMMU_CONTEXT, + nest_info); + g_free(nesting); + } + + container->iommu_type =3D iommu_type; + return 0; +} + static void vfio_get_iommu_info_migration(VFIOContainer *container, struct vfio_iommu_type1_info *inf= o) { diff --git a/hw/vfio/pci.c b/hw/vfio/pci.c index 437f51338e..f5363589b6 100644 --- a/hw/vfio/pci.c +++ b/hw/vfio/pci.c @@ -2764,6 +2764,7 @@ static void vfio_realize(PCIDevice *pdev, Error **err= p) VFIOPCIDevice *vdev =3D VFIO_PCI(pdev); VFIODevice *vbasedev_iter; VFIOGroup *group; + VFIOContainer *container; char *tmp, *subsys, group_path[PATH_MAX], *group_name; Error *err =3D NULL; ssize_t len; @@ -2829,6 +2830,15 @@ static void vfio_realize(PCIDevice *pdev, Error **er= rp) goto error; } =20 + container =3D group->container; + if (container->iommu_ctx.initialized && + pci_device_set_iommu_context(pdev, &container->iommu_ctx)) { + error_setg(errp, "device attachment is denied by vIOMMU, " + "please check host IOMMU nesting capability"); + vfio_put_group(group); + goto error; + } + QLIST_FOREACH(vbasedev_iter, &group->device_list, next) { if (strcmp(vbasedev_iter->name, vdev->vbasedev.name) =3D=3D 0) { error_setg(errp, "device is already attached"); @@ -3112,9 +3122,16 @@ static void vfio_instance_finalize(Object *obj) static void vfio_exitfn(PCIDevice *pdev) { VFIOPCIDevice *vdev =3D VFIO_PCI(pdev); + VFIOContainer *container; =20 vfio_unregister_req_notifier(vdev); vfio_unregister_err_notifier(vdev); + + container =3D vdev->vbasedev.group->container; + if (container->iommu_ctx.initialized) { + pci_device_unset_iommu_context(pdev); + } + pci_device_set_intx_routing_notifier(&vdev->pdev, NULL); if (vdev->irqchip_change_notifier.notify) { kvm_irqchip_remove_change_notifier(&vdev->irqchip_change_notifier); diff --git a/include/hw/vfio/vfio-common.h b/include/hw/vfio/vfio-common.h index 55241ee270..5a9f2b6325 100644 --- a/include/hw/vfio/vfio-common.h +++ b/include/hw/vfio/vfio-common.h @@ -85,6 +85,7 @@ typedef struct VFIOContainer { MemoryListener listener; MemoryListener prereg_listener; unsigned iommu_type; + HostIOMMUContext iommu_ctx; Error *error; bool initialized; bool dirty_pages_supported; --=20 2.25.1