From nobody Mon Feb 9 11:23:25 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass header.i=@intel.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=intel.com ARC-Seal: i=1; a=rsa-sha256; t=1655101642; cv=none; d=zohomail.com; s=zohoarc; b=ZiuH4OfKvC701yM5uDQTn6rcExOMC81MDF8kKMxRtRHGiHjJtJYYbHbAJekA2BOtUHpTBy++No00kMGiCPtnXaVn+sVxadBT9VGpxqiK519Ki+jWDqIp3a+YnBjMzx9SB4ty+sDmnXUZdOJ1IrwW4/awqm6aey/4SY+KwsJKyTo= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1655101642; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=mdmRVqtZMWWInL8JKKllwAjA4aOKsjtH6TKShK+cuHo=; b=Ub20Kz297U9TW3EWyUaZLL8z78avwZ8uqnNP7VT9PNH/gN95wo8Zbn60k2spsCXEykP2epkN1q2cqS5+u7LHe/03UAbmHLwbu/FqcK5FVglyIMaKrFxtqgU0NucTibHaCPAVUbAzafhAZejWFHXOVi9C3NzXa0xs+2mMX45MZoM= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass header.i=@intel.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 165510164283663.11953666842987; Sun, 12 Jun 2022 23:27:22 -0700 (PDT) Received: from localhost ([::1]:59972 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1o0dXk-0008JC-VW for importer@patchew.org; Mon, 13 Jun 2022 02:27:21 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:35508) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1o0dOE-0000O8-HH for qemu-devel@nongnu.org; Mon, 13 Jun 2022 02:17:35 -0400 Received: from mga03.intel.com ([134.134.136.65]:32802) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1o0dO7-0005Vn-DL for qemu-devel@nongnu.org; Mon, 13 Jun 2022 02:17:26 -0400 Received: from fmsmga005.fm.intel.com ([10.253.24.32]) by orsmga103.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 12 Jun 2022 23:17:16 -0700 Received: from duan-server-s2600bt.bj.intel.com ([10.240.192.123]) by fmsmga005-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 12 Jun 2022 23:17:13 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1655101043; x=1686637043; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=G1zy6HmWRUDuak6diXLvKP8K5XiF2nvpwOl/f8NnAH4=; b=YxJpKH7NKW4KniLFTbsG9BsxHiWRHOeA9AC5vkSJFHrQcN/Kh11S4orR ZdpqTplB2ByXMhXlS0jtr6eOBDhLKNaX/46p4Lm5nXszV35P63VuFOkZo ptAeeSjwnsCj6DwfI8haaw92FN8ErqY1ah8a4oJRdjS3DyZitobzXubFS 8vRJC/+BXnCnnr6GnDefqaMSy5i8/pJtIPpU5G2+/BUwm5v/PdWl5rINk s1AO/WWpmgTFYt7nXI2ljWyiJ9tdUGNjTajf1zz+8c3LYgISl46ZKgO0d Pc3EYe3nOIzxdUEjqHd7UMXAESBoaWtCVtROTmM3OXVrWaNZGCKx6Z0xi w==; X-IronPort-AV: E=McAfee;i="6400,9594,10376"; a="279229906" X-IronPort-AV: E=Sophos;i="5.91,296,1647327600"; d="scan'208";a="279229906" X-IronPort-AV: E=Sophos;i="5.91,296,1647327600"; d="scan'208";a="910205018" From: Zhenzhong Duan To: eric.auger@redhat.com Cc: qemu-devel@nongnu.org, mst@redhat.com, jean-philippe@linaro.org, pbonzini@redhat.com, yu.c.zhang@intel.com, chuanxiao.dong@intel.com, tina.zhang@intel.com Subject: [PATCH 1/3] virtio-iommu: Add bypass mode support to assigned device Date: Mon, 13 Jun 2022 14:10:08 +0800 Message-Id: <20220613061010.2674054-2-zhenzhong.duan@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220613061010.2674054-1-zhenzhong.duan@intel.com> References: <20220613061010.2674054-1-zhenzhong.duan@intel.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=134.134.136.65; envelope-from=zhenzhong.duan@intel.com; helo=mga03.intel.com X-Spam_score_int: -44 X-Spam_score: -4.5 X-Spam_bar: ---- X-Spam_report: (-4.5 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.082, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: pass (identity @intel.com) X-ZM-MESSAGEID: 1655101644751100001 Content-Type: text/plain; charset="utf-8" Currently assigned devices can not work in virtio-iommu bypass mode. Guest driver fails to probe the device due to DMA failure. And the reason is because of lacking GPA -> HPA mappings when VM is created. Add a root container memory region to hold both bypass memory region and iommu memory region, so the switch between them is supported just like the implementation in virtual VT-d. Signed-off-by: Zhenzhong Duan --- hw/virtio/trace-events | 1 + hw/virtio/virtio-iommu.c | 115 ++++++++++++++++++++++++++++++- include/hw/virtio/virtio-iommu.h | 2 + 3 files changed, 116 insertions(+), 2 deletions(-) diff --git a/hw/virtio/trace-events b/hw/virtio/trace-events index ab8e095b73fa..20af2e7ebd78 100644 --- a/hw/virtio/trace-events +++ b/hw/virtio/trace-events @@ -124,6 +124,7 @@ virtio_iommu_remap(const char *name, uint64_t virt_star= t, uint64_t virt_end, uin virtio_iommu_set_page_size_mask(const char *name, uint64_t old, uint64_t n= ew) "mr=3D%s old_mask=3D0x%"PRIx64" new_mask=3D0x%"PRIx64 virtio_iommu_notify_flag_add(const char *name) "add notifier to mr %s" virtio_iommu_notify_flag_del(const char *name) "del notifier from mr %s" +virtio_iommu_switch_address_space(uint8_t bus, uint8_t slot, uint8_t fn, b= ool on) "Device %02x:%02x.%x switching address space (iommu enabled=3D%d)" =20 # virtio-mem.c virtio_mem_send_response(uint16_t type) "type=3D%" PRIu16 diff --git a/hw/virtio/virtio-iommu.c b/hw/virtio/virtio-iommu.c index 2597e166f939..ff718107ee02 100644 --- a/hw/virtio/virtio-iommu.c +++ b/hw/virtio/virtio-iommu.c @@ -69,6 +69,77 @@ static inline uint16_t virtio_iommu_get_bdf(IOMMUDevice = *dev) return PCI_BUILD_BDF(pci_bus_num(dev->bus), dev->devfn); } =20 +static bool virtio_iommu_device_bypassed(IOMMUDevice *sdev) +{ + uint32_t sid; + bool bypassed; + VirtIOIOMMU *s =3D sdev->viommu; + VirtIOIOMMUEndpoint *ep; + + sid =3D virtio_iommu_get_bdf(sdev); + + qemu_mutex_lock(&s->mutex); + /* need to check bypass before system reset */ + if (!s->endpoints) { + bypassed =3D s->config.bypass; + goto unlock; + } + + ep =3D g_tree_lookup(s->endpoints, GUINT_TO_POINTER(sid)); + if (!ep || !ep->domain) { + bypassed =3D s->config.bypass; + } else { + bypassed =3D ep->domain->bypass; + } + +unlock: + qemu_mutex_unlock(&s->mutex); + return bypassed; +} + +/* Return whether the device is using IOMMU translation. */ +static bool virtio_iommu_switch_address_space(IOMMUDevice *sdev) +{ + bool use_remapping; + + assert(sdev); + + use_remapping =3D !virtio_iommu_device_bypassed(sdev); + + trace_virtio_iommu_switch_address_space(pci_bus_num(sdev->bus), + PCI_SLOT(sdev->devfn), + PCI_FUNC(sdev->devfn), + use_remapping); + + /* Turn off first then on the other */ + if (use_remapping) { + memory_region_set_enabled(&sdev->bypass_mr, false); + memory_region_set_enabled(MEMORY_REGION(&sdev->iommu_mr), true); + } else { + memory_region_set_enabled(MEMORY_REGION(&sdev->iommu_mr), false); + memory_region_set_enabled(&sdev->bypass_mr, true); + } + + return use_remapping; +} + +static void virtio_iommu_switch_address_space_all(VirtIOIOMMU *s) +{ + GHashTableIter iter; + IOMMUPciBus *iommu_pci_bus; + int i; + + g_hash_table_iter_init(&iter, s->as_by_busptr); + while (g_hash_table_iter_next(&iter, NULL, (void **)&iommu_pci_bus)) { + for (i =3D 0; i < PCI_DEVFN_MAX; i++) { + if (!iommu_pci_bus->pbdev[i]) { + continue; + } + virtio_iommu_switch_address_space(iommu_pci_bus->pbdev[i]); + } + } +} + /** * The bus number is used for lookup when SID based operations occur. * In that case we lazily populate the IOMMUPciBus array from the bus hash @@ -213,6 +284,7 @@ static gboolean virtio_iommu_notify_map_cb(gpointer key= , gpointer value, static void virtio_iommu_detach_endpoint_from_domain(VirtIOIOMMUEndpoint *= ep) { VirtIOIOMMUDomain *domain =3D ep->domain; + IOMMUDevice *sdev =3D container_of(ep->iommu_mr, IOMMUDevice, iommu_mr= ); =20 if (!ep->domain) { return; @@ -221,6 +293,7 @@ static void virtio_iommu_detach_endpoint_from_domain(Vi= rtIOIOMMUEndpoint *ep) ep->iommu_mr); QLIST_REMOVE(ep, next); ep->domain =3D NULL; + virtio_iommu_switch_address_space(sdev); } =20 static VirtIOIOMMUEndpoint *virtio_iommu_get_endpoint(VirtIOIOMMU *s, @@ -323,12 +396,39 @@ static AddressSpace *virtio_iommu_find_add_as(PCIBus = *bus, void *opaque, =20 trace_virtio_iommu_init_iommu_mr(name); =20 + memory_region_init(&sdev->root, OBJECT(s), name, UINT64_MAX); + address_space_init(&sdev->as, &sdev->root, TYPE_VIRTIO_IOMMU); + + /* + * Build the IOMMU disabled container with aliases to the + * shared MRs. Note that aliasing to a shared memory region + * could help the memory API to detect same FlatViews so we + * can have devices to share the same FlatView when in bypass + * mode. (either by not configuring virtio-iommu driver or with + * "iommu=3Dpt"). It will greatly reduce the total number of + * FlatViews of the system hence VM runs faster. + */ + memory_region_init_alias(&sdev->bypass_mr, OBJECT(s), + "system", get_system_memory(), 0, + memory_region_size(get_system_memory())); + memory_region_init_iommu(&sdev->iommu_mr, sizeof(sdev->iommu_mr), TYPE_VIRTIO_IOMMU_MEMORY_REGION, OBJECT(s), name, UINT64_MAX); - address_space_init(&sdev->as, - MEMORY_REGION(&sdev->iommu_mr), TYPE_VIRTIO_IOM= MU); + + /* + * Hook both the containers under the root container, we + * switch between iommu & bypass MRs by enable/disable + * corresponding sub-containers + */ + memory_region_add_subregion_overlap(&sdev->root, 0, + MEMORY_REGION(&sdev->iommu_mr), + 0); + memory_region_add_subregion_overlap(&sdev->root, 0, + &sdev->bypass_mr, 0); + + virtio_iommu_switch_address_space(sdev); g_free(name); } return &sdev->as; @@ -342,6 +442,7 @@ static int virtio_iommu_attach(VirtIOIOMMU *s, uint32_t flags =3D le32_to_cpu(req->flags); VirtIOIOMMUDomain *domain; VirtIOIOMMUEndpoint *ep; + IOMMUDevice *sdev; =20 trace_virtio_iommu_attach(domain_id, ep_id); =20 @@ -375,6 +476,8 @@ static int virtio_iommu_attach(VirtIOIOMMU *s, QLIST_INSERT_HEAD(&domain->endpoint_list, ep, next); =20 ep->domain =3D domain; + sdev =3D container_of(ep->iommu_mr, IOMMUDevice, iommu_mr); + virtio_iommu_switch_address_space(sdev); =20 /* Replay domain mappings on the associated memory region */ g_tree_foreach(domain->mappings, virtio_iommu_notify_map_cb, @@ -887,6 +990,7 @@ static void virtio_iommu_set_config(VirtIODevice *vdev, return; } dev_config->bypass =3D in_config->bypass; + virtio_iommu_switch_address_space_all(dev); } =20 trace_virtio_iommu_set_config(in_config->bypass); @@ -1026,6 +1130,8 @@ static void virtio_iommu_system_reset(void *opaque) * system reset */ s->config.bypass =3D s->boot_bypass; + virtio_iommu_switch_address_space_all(s); + } =20 static void virtio_iommu_device_realize(DeviceState *dev, Error **errp) @@ -1041,6 +1147,11 @@ static void virtio_iommu_device_realize(DeviceState = *dev, Error **errp) virtio_iommu_handle_command); s->event_vq =3D virtio_add_queue(vdev, VIOMMU_DEFAULT_QUEUE_SIZE, NULL= ); =20 + /* + * config.bypass is needed to get initial address space early, such as + * in vfio realize + */ + s->config.bypass =3D s->boot_bypass; s->config.page_size_mask =3D TARGET_PAGE_MASK; s->config.input_range.end =3D UINT64_MAX; s->config.domain_range.end =3D UINT32_MAX; diff --git a/include/hw/virtio/virtio-iommu.h b/include/hw/virtio/virtio-io= mmu.h index 84391f844826..102eeefa731d 100644 --- a/include/hw/virtio/virtio-iommu.h +++ b/include/hw/virtio/virtio-iommu.h @@ -37,6 +37,8 @@ typedef struct IOMMUDevice { int devfn; IOMMUMemoryRegion iommu_mr; AddressSpace as; + MemoryRegion root; /* The root container of the device */ + MemoryRegion bypass_mr; /* The alias of shared memory MR */ } IOMMUDevice; =20 typedef struct IOMMUPciBus { --=20 2.25.1 From nobody Mon Feb 9 11:23:25 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass header.i=@intel.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=intel.com ARC-Seal: i=1; a=rsa-sha256; t=1655101336; cv=none; d=zohomail.com; s=zohoarc; b=e/DMbH9x0pEwzKC/K7Kq5xjhnHcNEChgIrxi2lyRcfdk3sugpvbyvmn+1B/KJb4laY/ACfT8GTX6GxM6m5fx5ey1d9lSMMExtueF0lK5LKVEbRBveq7NrLSccxKWodtDVeyNu/wzPlYP9m4ir4Ed38prHmjkJh/fx6ByY3QCWBI= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1655101336; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=qBfhOsnBK+ncKg9kMGOYygKe2wanzvhcnY5+3t3UNKU=; b=S7GibM2pHgW03qNM4zleI/VHtMhbsmq2zmO0/oC4NZNyw5aHxxQsEA5WyPK3zv2ZlptVaM+0igW9k/UvyFcd/bj29Zu4CJYQPTTMLgA+HAgwlAspVlYKugwJUioxQHbHE2Hz4W4gWBn9+h+DTJyeW2SmMGmXst0BapRe2G+MBao= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass header.i=@intel.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1655101336218135.99669723406328; Sun, 12 Jun 2022 23:22:16 -0700 (PDT) Received: from localhost ([::1]:52838 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1o0dSo-0002nv-Kb for importer@patchew.org; Mon, 13 Jun 2022 02:22:14 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:35506) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1o0dOE-0000O7-HF for qemu-devel@nongnu.org; Mon, 13 Jun 2022 02:17:35 -0400 Received: from mga03.intel.com ([134.134.136.65]:32806) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1o0dO7-0005Vs-JW for qemu-devel@nongnu.org; Mon, 13 Jun 2022 02:17:26 -0400 Received: from fmsmga005.fm.intel.com ([10.253.24.32]) by orsmga103.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 12 Jun 2022 23:17:19 -0700 Received: from duan-server-s2600bt.bj.intel.com ([10.240.192.123]) by fmsmga005-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 12 Jun 2022 23:17:16 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1655101043; x=1686637043; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=GP51qLt1vIw10Qn2ckQiyo2AsRIDCvKVJfy/CW5j34Y=; b=kntSfnd65TvDPGTCV9ZCJKmtrsSdhnGy6QKVozBO5hqu+ASm0r1+vNTM jZdB6qTrqvdCS21d/70zbvhw8rdnOkVQ/vlzgLu3fKgcEZuF4vkWe5QQg K1VrUWQROmCkmjSBEIPdcYw3W7X7Y/TQ+SHDduu41XElvmP4II7x89Fp4 Q7h1ga5uK1VSGvFeMZEiNtMec7SRwnq0czytpzWMAGy42D6iVxca4UprH wccyIijgCeR+aecgkDg8n0/BxsvlutTEnIRLaNyXGFNZOD7oy03VrkyiK f/Siyq4XDGz9bvlSXnZtk2bgKSNYaraSUGmr/34mYIVt7KWdNK/Ef4LYU g==; X-IronPort-AV: E=McAfee;i="6400,9594,10376"; a="279229916" X-IronPort-AV: E=Sophos;i="5.91,296,1647327600"; d="scan'208";a="279229916" X-IronPort-AV: E=Sophos;i="5.91,296,1647327600"; d="scan'208";a="910205031" From: Zhenzhong Duan To: eric.auger@redhat.com Cc: qemu-devel@nongnu.org, mst@redhat.com, jean-philippe@linaro.org, pbonzini@redhat.com, yu.c.zhang@intel.com, chuanxiao.dong@intel.com, tina.zhang@intel.com Subject: [PATCH 2/3] virtio-iommu: Use recursive lock to avoid deadlock Date: Mon, 13 Jun 2022 14:10:09 +0800 Message-Id: <20220613061010.2674054-3-zhenzhong.duan@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220613061010.2674054-1-zhenzhong.duan@intel.com> References: <20220613061010.2674054-1-zhenzhong.duan@intel.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=134.134.136.65; envelope-from=zhenzhong.duan@intel.com; helo=mga03.intel.com X-Spam_score_int: -44 X-Spam_score: -4.5 X-Spam_bar: ---- X-Spam_report: (-4.5 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.082, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: pass (identity @intel.com) X-ZM-MESSAGEID: 1655101338550100003 Content-Type: text/plain; charset="utf-8" When switching address space with mutex lock hold, mapping will be replayed for assigned device. This will trigger relock deadlock. Also release the mutex resource in unrealize routine. Signed-off-by: Zhenzhong Duan --- hw/virtio/virtio-iommu.c | 20 +++++++++++--------- include/hw/virtio/virtio-iommu.h | 2 +- 2 files changed, 12 insertions(+), 10 deletions(-) diff --git a/hw/virtio/virtio-iommu.c b/hw/virtio/virtio-iommu.c index ff718107ee02..73d5bde9d122 100644 --- a/hw/virtio/virtio-iommu.c +++ b/hw/virtio/virtio-iommu.c @@ -78,7 +78,7 @@ static bool virtio_iommu_device_bypassed(IOMMUDevice *sde= v) =20 sid =3D virtio_iommu_get_bdf(sdev); =20 - qemu_mutex_lock(&s->mutex); + qemu_rec_mutex_lock(&s->mutex); /* need to check bypass before system reset */ if (!s->endpoints) { bypassed =3D s->config.bypass; @@ -93,7 +93,7 @@ static bool virtio_iommu_device_bypassed(IOMMUDevice *sde= v) } =20 unlock: - qemu_mutex_unlock(&s->mutex); + qemu_rec_mutex_unlock(&s->mutex); return bypassed; } =20 @@ -745,7 +745,7 @@ static void virtio_iommu_handle_command(VirtIODevice *v= dev, VirtQueue *vq) tail.status =3D VIRTIO_IOMMU_S_DEVERR; goto out; } - qemu_mutex_lock(&s->mutex); + qemu_rec_mutex_lock(&s->mutex); switch (head.type) { case VIRTIO_IOMMU_T_ATTACH: tail.status =3D virtio_iommu_handle_attach(s, iov, iov_cnt); @@ -774,7 +774,7 @@ static void virtio_iommu_handle_command(VirtIODevice *v= dev, VirtQueue *vq) default: tail.status =3D VIRTIO_IOMMU_S_UNSUPP; } - qemu_mutex_unlock(&s->mutex); + qemu_rec_mutex_unlock(&s->mutex); =20 out: sz =3D iov_from_buf(elem->in_sg, elem->in_num, 0, @@ -862,7 +862,7 @@ static IOMMUTLBEntry virtio_iommu_translate(IOMMUMemory= Region *mr, hwaddr addr, sid =3D virtio_iommu_get_bdf(sdev); =20 trace_virtio_iommu_translate(mr->parent_obj.name, sid, addr, flag); - qemu_mutex_lock(&s->mutex); + qemu_rec_mutex_lock(&s->mutex); =20 ep =3D g_tree_lookup(s->endpoints, GUINT_TO_POINTER(sid)); if (!ep) { @@ -946,7 +946,7 @@ static IOMMUTLBEntry virtio_iommu_translate(IOMMUMemory= Region *mr, hwaddr addr, trace_virtio_iommu_translate_out(addr, entry.translated_addr, sid); =20 unlock: - qemu_mutex_unlock(&s->mutex); + qemu_rec_mutex_unlock(&s->mutex); return entry; } =20 @@ -1035,7 +1035,7 @@ static void virtio_iommu_replay(IOMMUMemoryRegion *mr= , IOMMUNotifier *n) =20 sid =3D virtio_iommu_get_bdf(sdev); =20 - qemu_mutex_lock(&s->mutex); + qemu_rec_mutex_lock(&s->mutex); =20 if (!s->endpoints) { goto unlock; @@ -1049,7 +1049,7 @@ static void virtio_iommu_replay(IOMMUMemoryRegion *mr= , IOMMUNotifier *n) g_tree_foreach(ep->domain->mappings, virtio_iommu_remap, mr); =20 unlock: - qemu_mutex_unlock(&s->mutex); + qemu_rec_mutex_unlock(&s->mutex); } =20 static int virtio_iommu_notify_flag_changed(IOMMUMemoryRegion *iommu_mr, @@ -1167,7 +1167,7 @@ static void virtio_iommu_device_realize(DeviceState *= dev, Error **errp) virtio_add_feature(&s->features, VIRTIO_IOMMU_F_PROBE); virtio_add_feature(&s->features, VIRTIO_IOMMU_F_BYPASS_CONFIG); =20 - qemu_mutex_init(&s->mutex); + qemu_rec_mutex_init(&s->mutex); =20 s->as_by_busptr =3D g_hash_table_new_full(NULL, NULL, NULL, g_free); =20 @@ -1195,6 +1195,8 @@ static void virtio_iommu_device_unrealize(DeviceState= *dev) g_tree_destroy(s->endpoints); } =20 + qemu_rec_mutex_destroy(&s->mutex); + virtio_delete_queue(s->req_vq); virtio_delete_queue(s->event_vq); virtio_cleanup(vdev); diff --git a/include/hw/virtio/virtio-iommu.h b/include/hw/virtio/virtio-io= mmu.h index 102eeefa731d..2ad5ee320be9 100644 --- a/include/hw/virtio/virtio-iommu.h +++ b/include/hw/virtio/virtio-iommu.h @@ -58,7 +58,7 @@ struct VirtIOIOMMU { ReservedRegion *reserved_regions; uint32_t nb_reserved_regions; GTree *domains; - QemuMutex mutex; + QemuRecMutex mutex; GTree *endpoints; bool boot_bypass; }; --=20 2.25.1 From nobody Mon Feb 9 11:23:25 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass header.i=@intel.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=intel.com ARC-Seal: i=1; a=rsa-sha256; t=1655101644; cv=none; d=zohomail.com; s=zohoarc; b=LI5hsG2VjkROTI68B4Q8L0EAzOMathi6vFGUz2+GlX8qpYramFVEwM8YV73wWiG1o/a8wm6iQr86rGDOKu18E2vnWjiWio1g5m6W8eIWUy2fyx5BDVEUjVM+C0OsnSWDGs6u04BrcOb4Ry2vZpEoS6B2BkV+SmtWzMSY/Cv5rI0= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1655101644; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=5TRw3K+2SS8JJ3AZCDxDT7qkyVrtHOBgbmfT7Ng2Ubw=; b=Vg+jWFspmdMChnpk9Y4pKzSt4eyJgXD65b15g7lVYxxEEvSKKn4FYsi5g4J43N+/SpJSXADOhcKGGV2OoptCE/f2JWGxVF1kdpokVlEof6Q7yMmt/0vMtxzzNMGYH+CsVK3TMoxiYFGYoZWTR07+c3C5NNcSK19781giwjUcRRg= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass header.i=@intel.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1655101644363959.8795782047822; Sun, 12 Jun 2022 23:27:24 -0700 (PDT) Received: from localhost ([::1]:59964 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1o0dXn-0008In-8W for importer@patchew.org; Mon, 13 Jun 2022 02:27:23 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:35526) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1o0dOG-0000P2-Q3 for qemu-devel@nongnu.org; Mon, 13 Jun 2022 02:17:41 -0400 Received: from mga03.intel.com ([134.134.136.65]:32802) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1o0dOE-0005Vn-Q9 for qemu-devel@nongnu.org; Mon, 13 Jun 2022 02:17:32 -0400 Received: from fmsmga005.fm.intel.com ([10.253.24.32]) by orsmga103.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 12 Jun 2022 23:17:21 -0700 Received: from duan-server-s2600bt.bj.intel.com ([10.240.192.123]) by fmsmga005-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 12 Jun 2022 23:17:19 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1655101050; x=1686637050; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=nr+MVTE7DGnhDyn8aP7rASCoalYk1aow0qsTuuKteKY=; b=id5EVLT6sr+KE1NrxuCiE1wvHrJw8ThGmsKtNQG2n4CIlcQWHet8RAH/ 9Bc78YA+UT7eq2VoZcFN39NKdLTD01v1kYe20q/rAus4ZzGbEKyWfrohs bUTi17comZBNliExEdtvcmNnB5nP3tDwn9j1nL4zgapt5OeNN2eKqz7ot 8G/5eqw4n0OiFappDKTeZil2bgReVBA+R+s1XgqtoePeP8fAKx/Bw/18c YJlaVxB1CcwI0C0L5m7W6upYkT68q93T/5JzBxi2rmn2oQu3uUK3HMrcE EyC5BNZ26GtXAkObP2aLtMsWlQ2vv9VWyP/AtXTROTLVNG0H6/DTj/QNR w==; X-IronPort-AV: E=McAfee;i="6400,9594,10376"; a="279229926" X-IronPort-AV: E=Sophos;i="5.91,296,1647327600"; d="scan'208";a="279229926" X-IronPort-AV: E=Sophos;i="5.91,296,1647327600"; d="scan'208";a="910205046" From: Zhenzhong Duan To: eric.auger@redhat.com Cc: qemu-devel@nongnu.org, mst@redhat.com, jean-philippe@linaro.org, pbonzini@redhat.com, yu.c.zhang@intel.com, chuanxiao.dong@intel.com, tina.zhang@intel.com Subject: [PATCH 3/3] virtio-iommu: Add an assert check in translate routine Date: Mon, 13 Jun 2022 14:10:10 +0800 Message-Id: <20220613061010.2674054-4-zhenzhong.duan@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220613061010.2674054-1-zhenzhong.duan@intel.com> References: <20220613061010.2674054-1-zhenzhong.duan@intel.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=134.134.136.65; envelope-from=zhenzhong.duan@intel.com; helo=mga03.intel.com X-Spam_score_int: -44 X-Spam_score: -4.5 X-Spam_bar: ---- X-Spam_report: (-4.5 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.082, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: pass (identity @intel.com) X-ZM-MESSAGEID: 1655101646410100003 Content-Type: text/plain; charset="utf-8" With address space switch supported, dma access translation only happen after endpoint is attached to a non-bypass domain. Signed-off-by: Zhenzhong Duan --- hw/virtio/virtio-iommu.c | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/hw/virtio/virtio-iommu.c b/hw/virtio/virtio-iommu.c index 73d5bde9d122..7c122ab95780 100644 --- a/hw/virtio/virtio-iommu.c +++ b/hw/virtio/virtio-iommu.c @@ -865,6 +865,10 @@ static IOMMUTLBEntry virtio_iommu_translate(IOMMUMemor= yRegion *mr, hwaddr addr, qemu_rec_mutex_lock(&s->mutex); =20 ep =3D g_tree_lookup(s->endpoints, GUINT_TO_POINTER(sid)); + + if (bypass_allowed) + assert(ep && ep->domain && !ep->domain->bypass); + if (!ep) { if (!bypass_allowed) { error_report_once("%s sid=3D%d is not known!!", __func__, sid); --=20 2.25.1