From nobody Fri May 17 05:26:25 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=huawei.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1625749260510649.4822064692221; Thu, 8 Jul 2021 06:01:00 -0700 (PDT) Received: from localhost ([::1]:36376 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1m1TeF-0005qE-Gf for importer@patchew.org; Thu, 08 Jul 2021 09:00:59 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:41366) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1m1TZL-0004Pz-Oq; Thu, 08 Jul 2021 08:55:55 -0400 Received: from szxga01-in.huawei.com ([45.249.212.187]:2497) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1m1TZF-0005Z4-8s; Thu, 08 Jul 2021 08:55:55 -0400 Received: from dggemv711-chm.china.huawei.com (unknown [172.30.72.53]) by szxga01-in.huawei.com (SkyGuard) with ESMTP id 4GLGN55CtTzXpf0; Thu, 8 Jul 2021 20:49:57 +0800 (CST) Received: from dggpemm500009.china.huawei.com (7.185.36.225) by dggemv711-chm.china.huawei.com (10.1.198.66) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2176.2; Thu, 8 Jul 2021 20:55:28 +0800 Received: from huawei.com (10.174.185.226) by dggpemm500009.china.huawei.com (7.185.36.225) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2176.2; Thu, 8 Jul 2021 20:55:27 +0800 From: Wang Xingang To: , , , , , , , , , , Subject: [PATCH v5 1/9] hw/pci/pci_host: Allow PCI host to bypass iommu Date: Thu, 8 Jul 2021 12:55:11 +0000 Message-ID: <1625748919-52456-2-git-send-email-wangxingang5@huawei.com> X-Mailer: git-send-email 2.6.4.windows.1 In-Reply-To: <1625748919-52456-1-git-send-email-wangxingang5@huawei.com> References: <1625748919-52456-1-git-send-email-wangxingang5@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.174.185.226] X-ClientProxiedBy: dggems704-chm.china.huawei.com (10.3.19.181) To dggpemm500009.china.huawei.com (7.185.36.225) X-CFilter-Loop: Reflected Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=45.249.212.187; envelope-from=wangxingang5@huawei.com; helo=szxga01-in.huawei.com X-Spam_score_int: -41 X-Spam_score: -4.2 X-Spam_bar: ---- X-Spam_report: (-4.2 / 5.0 requ) BAYES_00=-1.9, RCVD_IN_DNSWL_MED=-2.3, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: xieyingtai@huawei.com, wangxingang5@huawei.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZM-MESSAGEID: 1625749270412100005 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Xingang Wang Add a new bypass_iommu property for PCI host and use it to check whether devices attached to the PCI root bus will bypass iommu. In pci_device_iommu_address_space(), check the property and avoid getting iommu address space for devices bypass iommu. Signed-off-by: Xingang Wang Reviewed-by: Eric Auger --- hw/pci/pci.c | 18 +++++++++++++++++- hw/pci/pci_host.c | 1 + include/hw/pci/pci.h | 1 + include/hw/pci/pci_host.h | 1 + 4 files changed, 20 insertions(+), 1 deletion(-) diff --git a/hw/pci/pci.c b/hw/pci/pci.c index 377084f1a8..27d588e268 100644 --- a/hw/pci/pci.c +++ b/hw/pci/pci.c @@ -416,6 +416,22 @@ const char *pci_root_bus_path(PCIDevice *dev) return rootbus->qbus.name; } =20 +bool pci_bus_bypass_iommu(PCIBus *bus) +{ + PCIBus *rootbus =3D bus; + PCIHostState *host_bridge; + + if (!pci_bus_is_root(bus)) { + rootbus =3D pci_device_root_bus(bus->parent_dev); + } + + host_bridge =3D PCI_HOST_BRIDGE(rootbus->qbus.parent); + + assert(host_bridge->bus =3D=3D rootbus); + + return host_bridge->bypass_iommu; +} + static void pci_root_bus_init(PCIBus *bus, DeviceState *parent, MemoryRegion *address_space_mem, MemoryRegion *address_space_io, @@ -2718,7 +2734,7 @@ AddressSpace *pci_device_iommu_address_space(PCIDevic= e *dev) =20 iommu_bus =3D parent_bus; } - if (iommu_bus && iommu_bus->iommu_fn) { + if (!pci_bus_bypass_iommu(bus) && iommu_bus && iommu_bus->iommu_fn) { return iommu_bus->iommu_fn(bus, iommu_bus->iommu_opaque, devfn); } return &address_space_memory; diff --git a/hw/pci/pci_host.c b/hw/pci/pci_host.c index 8ca5fadcbd..cf02f0d6a5 100644 --- a/hw/pci/pci_host.c +++ b/hw/pci/pci_host.c @@ -222,6 +222,7 @@ const VMStateDescription vmstate_pcihost =3D { static Property pci_host_properties_common[] =3D { DEFINE_PROP_BOOL("x-config-reg-migration-enabled", PCIHostState, mig_enabled, true), + DEFINE_PROP_BOOL("bypass-iommu", PCIHostState, bypass_iommu, false), DEFINE_PROP_END_OF_LIST(), }; =20 diff --git a/include/hw/pci/pci.h b/include/hw/pci/pci.h index 6be4e0c460..f4d51b672b 100644 --- a/include/hw/pci/pci.h +++ b/include/hw/pci/pci.h @@ -480,6 +480,7 @@ void pci_for_each_bus(PCIBus *bus, =20 PCIBus *pci_device_root_bus(const PCIDevice *d); const char *pci_root_bus_path(PCIDevice *dev); +bool pci_bus_bypass_iommu(PCIBus *bus); PCIDevice *pci_find_device(PCIBus *bus, int bus_num, uint8_t devfn); int pci_qdev_find_device(const char *id, PCIDevice **pdev); void pci_bus_get_w64_range(PCIBus *bus, Range *range); diff --git a/include/hw/pci/pci_host.h b/include/hw/pci/pci_host.h index 52e038c019..c6f4eb4585 100644 --- a/include/hw/pci/pci_host.h +++ b/include/hw/pci/pci_host.h @@ -43,6 +43,7 @@ struct PCIHostState { uint32_t config_reg; bool mig_enabled; PCIBus *bus; + bool bypass_iommu; =20 QLIST_ENTRY(PCIHostState) next; }; --=20 2.19.1 From nobody Fri May 17 05:26:25 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=huawei.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1625749059199353.0968477832172; Thu, 8 Jul 2021 05:57:39 -0700 (PDT) Received: from localhost ([::1]:53794 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1m1Taz-00076Z-F6 for importer@patchew.org; Thu, 08 Jul 2021 08:57:37 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:41292) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1m1TZI-0004LF-6Y; Thu, 08 Jul 2021 08:55:52 -0400 Received: from szxga01-in.huawei.com ([45.249.212.187]:2432) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1m1TZE-0005Z2-Jx; Thu, 08 Jul 2021 08:55:51 -0400 Received: from dggemv704-chm.china.huawei.com (unknown [172.30.72.53]) by szxga01-in.huawei.com (SkyGuard) with ESMTP id 4GLGQm05h5zbbYg; Thu, 8 Jul 2021 20:52:16 +0800 (CST) Received: from dggpemm500009.china.huawei.com (7.185.36.225) by dggemv704-chm.china.huawei.com (10.3.19.47) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2176.2; Thu, 8 Jul 2021 20:55:28 +0800 Received: from huawei.com (10.174.185.226) by dggpemm500009.china.huawei.com (7.185.36.225) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2176.2; Thu, 8 Jul 2021 20:55:28 +0800 From: Wang Xingang To: , , , , , , , , , , Subject: [PATCH v5 2/9] hw/pxb: Add a bypass iommu property Date: Thu, 8 Jul 2021 12:55:12 +0000 Message-ID: <1625748919-52456-3-git-send-email-wangxingang5@huawei.com> X-Mailer: git-send-email 2.6.4.windows.1 In-Reply-To: <1625748919-52456-1-git-send-email-wangxingang5@huawei.com> References: <1625748919-52456-1-git-send-email-wangxingang5@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.174.185.226] X-ClientProxiedBy: dggems704-chm.china.huawei.com (10.3.19.181) To dggpemm500009.china.huawei.com (7.185.36.225) X-CFilter-Loop: Reflected Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=45.249.212.187; envelope-from=wangxingang5@huawei.com; helo=szxga01-in.huawei.com X-Spam_score_int: -41 X-Spam_score: -4.2 X-Spam_bar: ---- X-Spam_report: (-4.2 / 5.0 requ) BAYES_00=-1.9, RCVD_IN_DNSWL_MED=-2.3, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: xieyingtai@huawei.com, wangxingang5@huawei.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZM-MESSAGEID: 1625749060727100003 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Xingang Wang Add a bypass_iommu property for pci_expander_bridge, the property is used to indicate whether pxb root bus will bypass iommu. By default the bypass_iommu is disabled, and it can be enabled with: qemu -device pxb-pcie,bus_nr=3D0x10,addr=3D0x1,bypass_iommu=3Dtrue Signed-off-by: Xingang Wang Reviewed-by: Eric Auger --- hw/pci-bridge/pci_expander_bridge.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/hw/pci-bridge/pci_expander_bridge.c b/hw/pci-bridge/pci_expand= er_bridge.c index aedded1064..7112dc3062 100644 --- a/hw/pci-bridge/pci_expander_bridge.c +++ b/hw/pci-bridge/pci_expander_bridge.c @@ -57,6 +57,7 @@ struct PXBDev { =20 uint8_t bus_nr; uint16_t numa_node; + bool bypass_iommu; }; =20 static PXBDev *convert_to_pxb(PCIDevice *dev) @@ -255,6 +256,7 @@ static void pxb_dev_realize_common(PCIDevice *dev, bool= pcie, Error **errp) bus->map_irq =3D pxb_map_irq_fn; =20 PCI_HOST_BRIDGE(ds)->bus =3D bus; + PCI_HOST_BRIDGE(ds)->bypass_iommu =3D pxb->bypass_iommu; =20 pxb_register_bus(dev, bus, &local_err); if (local_err) { @@ -301,6 +303,7 @@ static Property pxb_dev_properties[] =3D { /* Note: 0 is not a legal PXB bus number. */ DEFINE_PROP_UINT8("bus_nr", PXBDev, bus_nr, 0), DEFINE_PROP_UINT16("numa_node", PXBDev, numa_node, NUMA_NODE_UNASSIGNE= D), + DEFINE_PROP_BOOL("bypass_iommu", PXBDev, bypass_iommu, false), DEFINE_PROP_END_OF_LIST(), }; =20 --=20 2.19.1 From nobody Fri May 17 05:26:25 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=huawei.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1625749314851281.79661158585884; Thu, 8 Jul 2021 06:01:54 -0700 (PDT) Received: from localhost ([::1]:37852 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1m1Tf7-0006wk-Hz for importer@patchew.org; Thu, 08 Jul 2021 09:01:53 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:41344) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1m1TZK-0004Mq-UD; Thu, 08 Jul 2021 08:55:54 -0400 Received: from szxga02-in.huawei.com ([45.249.212.188]:2061) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1m1TZF-0005eF-8A; Thu, 08 Jul 2021 08:55:54 -0400 Received: from dggemv703-chm.china.huawei.com (unknown [172.30.72.55]) by szxga02-in.huawei.com (SkyGuard) with ESMTP id 4GLGQt0pytzcb9f; Thu, 8 Jul 2021 20:52:22 +0800 (CST) Received: from dggpemm500009.china.huawei.com (7.185.36.225) by dggemv703-chm.china.huawei.com (10.3.19.46) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2176.2; Thu, 8 Jul 2021 20:55:29 +0800 Received: from huawei.com (10.174.185.226) by dggpemm500009.china.huawei.com (7.185.36.225) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2176.2; Thu, 8 Jul 2021 20:55:28 +0800 From: Wang Xingang To: , , , , , , , , , , Subject: [PATCH v5 3/9] hw/arm/virt: Add default_bus_bypass_iommu machine option Date: Thu, 8 Jul 2021 12:55:13 +0000 Message-ID: <1625748919-52456-4-git-send-email-wangxingang5@huawei.com> X-Mailer: git-send-email 2.6.4.windows.1 In-Reply-To: <1625748919-52456-1-git-send-email-wangxingang5@huawei.com> References: <1625748919-52456-1-git-send-email-wangxingang5@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.174.185.226] X-ClientProxiedBy: dggems704-chm.china.huawei.com (10.3.19.181) To dggpemm500009.china.huawei.com (7.185.36.225) X-CFilter-Loop: Reflected Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=45.249.212.188; envelope-from=wangxingang5@huawei.com; helo=szxga02-in.huawei.com X-Spam_score_int: -41 X-Spam_score: -4.2 X-Spam_bar: ---- X-Spam_report: (-4.2 / 5.0 requ) BAYES_00=-1.9, RCVD_IN_DNSWL_MED=-2.3, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: xieyingtai@huawei.com, wangxingang5@huawei.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZM-MESSAGEID: 1625749331062100001 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Xingang Wang Add a default_bus_bypass_iommu machine option to enable/disable bypass_iommu for default root bus. The option is disabled by default and can be enabled with: $QEMU -machine virt,iommu=3Dsmmuv3,default_bus_bypass_iommu=3Dtrue Signed-off-by: Xingang Wang --- hw/arm/virt.c | 26 ++++++++++++++++++++++++++ include/hw/arm/virt.h | 1 + 2 files changed, 27 insertions(+) diff --git a/hw/arm/virt.c b/hw/arm/virt.c index 4b96f06014..a2805977b8 100644 --- a/hw/arm/virt.c +++ b/hw/arm/virt.c @@ -1364,6 +1364,7 @@ static void create_pcie(VirtMachineState *vms) } =20 pci =3D PCI_HOST_BRIDGE(dev); + pci->bypass_iommu =3D vms->default_bus_bypass_iommu; vms->bus =3D pci->bus; if (vms->bus) { for (i =3D 0; i < nb_nics; i++) { @@ -2319,6 +2320,21 @@ static void virt_set_iommu(Object *obj, const char *= value, Error **errp) } } =20 +static bool virt_get_default_bus_bypass_iommu(Object *obj, Error **errp) +{ + VirtMachineState *vms =3D VIRT_MACHINE(obj); + + return vms->default_bus_bypass_iommu; +} + +static void virt_set_default_bus_bypass_iommu(Object *obj, bool value, + Error **errp) +{ + VirtMachineState *vms =3D VIRT_MACHINE(obj); + + vms->default_bus_bypass_iommu =3D value; +} + static CpuInstanceProperties virt_cpu_index_to_props(MachineState *ms, unsigned cpu_index) { @@ -2658,6 +2674,13 @@ static void virt_machine_class_init(ObjectClass *oc,= void *data) "Set the IOMMU type. " "Valid values are none and smmuv= 3"); =20 + object_class_property_add_bool(oc, "default_bus_bypass_iommu", + virt_get_default_bus_bypass_iommu, + virt_set_default_bus_bypass_iommu); + object_class_property_set_description(oc, "default_bus_bypass_iommu", + "Set on/off to enable/disable " + "bypass_iommu for default root b= us"); + object_class_property_add_bool(oc, "ras", virt_get_ras, virt_set_ras); object_class_property_set_description(oc, "ras", @@ -2725,6 +2748,9 @@ static void virt_instance_init(Object *obj) /* Default disallows iommu instantiation */ vms->iommu =3D VIRT_IOMMU_NONE; =20 + /* The default root bus is attached to iommu by default */ + vms->default_bus_bypass_iommu =3D false; + /* Default disallows RAS instantiation */ vms->ras =3D false; =20 diff --git a/include/hw/arm/virt.h b/include/hw/arm/virt.h index 921416f918..9661c46699 100644 --- a/include/hw/arm/virt.h +++ b/include/hw/arm/virt.h @@ -147,6 +147,7 @@ struct VirtMachineState { OnOffAuto acpi; VirtGICType gic_version; VirtIOMMUType iommu; + bool default_bus_bypass_iommu; VirtMSIControllerType msi_controller; uint16_t virtio_iommu_bdf; struct arm_boot_info bootinfo; --=20 2.19.1 From nobody Fri May 17 05:26:25 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=huawei.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1625749451423573.6719358766587; Thu, 8 Jul 2021 06:04:11 -0700 (PDT) Received: from localhost ([::1]:45432 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1m1ThK-0003fT-C2 for importer@patchew.org; Thu, 08 Jul 2021 09:04:10 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:41378) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1m1TZN-0004Yz-Vo; Thu, 08 Jul 2021 08:55:58 -0400 Received: from szxga02-in.huawei.com ([45.249.212.188]:2158) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1m1TZF-0005Z3-AV; Thu, 08 Jul 2021 08:55:57 -0400 Received: from dggemv711-chm.china.huawei.com (unknown [172.30.72.54]) by szxga02-in.huawei.com (SkyGuard) with ESMTP id 4GLGQT4mHkz77mk; Thu, 8 Jul 2021 20:52:01 +0800 (CST) Received: from dggpemm500009.china.huawei.com (7.185.36.225) by dggemv711-chm.china.huawei.com (10.1.198.66) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2176.2; Thu, 8 Jul 2021 20:55:30 +0800 Received: from huawei.com (10.174.185.226) by dggpemm500009.china.huawei.com (7.185.36.225) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2176.2; Thu, 8 Jul 2021 20:55:29 +0800 From: Wang Xingang To: , , , , , , , , , , Subject: [PATCH v5 4/9] hw/i386: Add a default_bus_bypass_iommu pc machine option Date: Thu, 8 Jul 2021 12:55:14 +0000 Message-ID: <1625748919-52456-5-git-send-email-wangxingang5@huawei.com> X-Mailer: git-send-email 2.6.4.windows.1 In-Reply-To: <1625748919-52456-1-git-send-email-wangxingang5@huawei.com> References: <1625748919-52456-1-git-send-email-wangxingang5@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.174.185.226] X-ClientProxiedBy: dggems704-chm.china.huawei.com (10.3.19.181) To dggpemm500009.china.huawei.com (7.185.36.225) X-CFilter-Loop: Reflected Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=45.249.212.188; envelope-from=wangxingang5@huawei.com; helo=szxga02-in.huawei.com X-Spam_score_int: -41 X-Spam_score: -4.2 X-Spam_bar: ---- X-Spam_report: (-4.2 / 5.0 requ) BAYES_00=-1.9, RCVD_IN_DNSWL_MED=-2.3, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: xieyingtai@huawei.com, wangxingang5@huawei.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZM-MESSAGEID: 1625749457667100001 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Xingang Wang Add a default_bus_bypass_iommu pc machine option to enable/disable bypass_iommu for default root bus. The option is disabled by default and can be enabled with: $QEMU -machine q35,default_bus_bypass_iommu=3Dtrue Signed-off-by: Xingang Wang --- hw/i386/pc.c | 20 ++++++++++++++++++++ hw/pci-host/q35.c | 2 ++ include/hw/i386/pc.h | 1 + 3 files changed, 23 insertions(+) diff --git a/hw/i386/pc.c b/hw/i386/pc.c index 8e1220db72..257d70f6e9 100644 --- a/hw/i386/pc.c +++ b/hw/i386/pc.c @@ -1522,6 +1522,21 @@ static void pc_machine_set_hpet(Object *obj, bool va= lue, Error **errp) pcms->hpet_enabled =3D value; } =20 +static bool pc_machine_get_default_bus_bypass_iommu(Object *obj, Error **e= rrp) +{ + PCMachineState *pcms =3D PC_MACHINE(obj); + + return pcms->default_bus_bypass_iommu; +} + +static void pc_machine_set_default_bus_bypass_iommu(Object *obj, bool valu= e, + Error **errp) +{ + PCMachineState *pcms =3D PC_MACHINE(obj); + + pcms->default_bus_bypass_iommu =3D value; +} + static void pc_machine_get_max_ram_below_4g(Object *obj, Visitor *v, const char *name, void *opaque, Error **errp) @@ -1621,6 +1636,7 @@ static void pc_machine_initfn(Object *obj) #ifdef CONFIG_HPET pcms->hpet_enabled =3D true; #endif + pcms->default_bus_bypass_iommu =3D false; =20 pc_system_flash_create(pcms); pcms->pcspk =3D isa_new(TYPE_PC_SPEAKER); @@ -1745,6 +1761,10 @@ static void pc_machine_class_init(ObjectClass *oc, v= oid *data) object_class_property_add_bool(oc, "hpet", pc_machine_get_hpet, pc_machine_set_hpet); =20 + object_class_property_add_bool(oc, "default_bus_bypass_iommu", + pc_machine_get_default_bus_bypass_iommu, + pc_machine_set_default_bus_bypass_iommu); + object_class_property_add(oc, PC_MACHINE_MAX_FW_SIZE, "size", pc_machine_get_max_fw_size, pc_machine_set_max_fw_size, NULL, NULL); diff --git a/hw/pci-host/q35.c b/hw/pci-host/q35.c index 2eb729dff5..826a05d7f3 100644 --- a/hw/pci-host/q35.c +++ b/hw/pci-host/q35.c @@ -64,6 +64,8 @@ static void q35_host_realize(DeviceState *dev, Error **er= rp) s->mch.address_space_io, 0, TYPE_PCIE_BUS); PC_MACHINE(qdev_get_machine())->bus =3D pci->bus; + pci->bypass_iommu =3D + PC_MACHINE(qdev_get_machine())->default_bus_bypass_iommu; qdev_realize(DEVICE(&s->mch), BUS(pci->bus), &error_fatal); } =20 diff --git a/include/hw/i386/pc.h b/include/hw/i386/pc.h index 87294f2632..fd741119fa 100644 --- a/include/hw/i386/pc.h +++ b/include/hw/i386/pc.h @@ -44,6 +44,7 @@ typedef struct PCMachineState { bool sata_enabled; bool pit_enabled; bool hpet_enabled; + bool default_bus_bypass_iommu; uint64_t max_fw_size; =20 /* NUMA information: */ --=20 2.19.1 From nobody Fri May 17 05:26:25 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=huawei.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1625749068822284.98552756923254; Thu, 8 Jul 2021 05:57:48 -0700 (PDT) Received: from localhost ([::1]:54874 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1m1Tb9-0007pa-RB for importer@patchew.org; Thu, 08 Jul 2021 08:57:47 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:41310) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1m1TZJ-0004Lh-Gg; Thu, 08 Jul 2021 08:55:53 -0400 Received: from szxga08-in.huawei.com ([45.249.212.255]:2248) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1m1TZF-0005ZY-0W; Thu, 08 Jul 2021 08:55:53 -0400 Received: from dggemv704-chm.china.huawei.com (unknown [172.30.72.57]) by szxga08-in.huawei.com (SkyGuard) with ESMTP id 4GLGN843d1z1CGSF; Thu, 8 Jul 2021 20:50:00 +0800 (CST) Received: from dggpemm500009.china.huawei.com (7.185.36.225) by dggemv704-chm.china.huawei.com (10.3.19.47) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2176.2; Thu, 8 Jul 2021 20:55:30 +0800 Received: from huawei.com (10.174.185.226) by dggpemm500009.china.huawei.com (7.185.36.225) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2176.2; Thu, 8 Jul 2021 20:55:30 +0800 From: Wang Xingang To: , , , , , , , , , , Subject: [PATCH v5 5/9] hw/pci: Add pci_bus_range() to get PCI bus number range Date: Thu, 8 Jul 2021 12:55:15 +0000 Message-ID: <1625748919-52456-6-git-send-email-wangxingang5@huawei.com> X-Mailer: git-send-email 2.6.4.windows.1 In-Reply-To: <1625748919-52456-1-git-send-email-wangxingang5@huawei.com> References: <1625748919-52456-1-git-send-email-wangxingang5@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.174.185.226] X-ClientProxiedBy: dggems704-chm.china.huawei.com (10.3.19.181) To dggpemm500009.china.huawei.com (7.185.36.225) X-CFilter-Loop: Reflected Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=45.249.212.255; envelope-from=wangxingang5@huawei.com; helo=szxga08-in.huawei.com X-Spam_score_int: -41 X-Spam_score: -4.2 X-Spam_bar: ---- X-Spam_report: (-4.2 / 5.0 requ) BAYES_00=-1.9, RCVD_IN_DNSWL_MED=-2.3, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: xieyingtai@huawei.com, wangxingang5@huawei.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZM-MESSAGEID: 1625749069752100001 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Xingang Wang This helps to get the min and max bus number of a PCI bus hierarchy. Signed-off-by: Xingang Wang Reviewed-by: Eric Auger --- hw/pci/pci.c | 16 ++++++++++++++++ include/hw/pci/pci.h | 1 + 2 files changed, 17 insertions(+) diff --git a/hw/pci/pci.c b/hw/pci/pci.c index 27d588e268..23d2ae2ab2 100644 --- a/hw/pci/pci.c +++ b/hw/pci/pci.c @@ -537,6 +537,22 @@ int pci_bus_num(PCIBus *s) return PCI_BUS_GET_CLASS(s)->bus_num(s); } =20 +/* Returns the min and max bus numbers of a PCI bus hierarchy */ +void pci_bus_range(PCIBus *bus, int *min_bus, int *max_bus) +{ + int i; + *min_bus =3D *max_bus =3D pci_bus_num(bus); + + for (i =3D 0; i < ARRAY_SIZE(bus->devices); ++i) { + PCIDevice *dev =3D bus->devices[i]; + + if (dev && PCI_DEVICE_GET_CLASS(dev)->is_bridge) { + *min_bus =3D MIN(*min_bus, dev->config[PCI_SECONDARY_BUS]); + *max_bus =3D MAX(*max_bus, dev->config[PCI_SUBORDINATE_BUS]); + } + } +} + int pci_bus_numa_node(PCIBus *bus) { return PCI_BUS_GET_CLASS(bus)->numa_node(bus); diff --git a/include/hw/pci/pci.h b/include/hw/pci/pci.h index f4d51b672b..d0f4266e37 100644 --- a/include/hw/pci/pci.h +++ b/include/hw/pci/pci.h @@ -450,6 +450,7 @@ static inline PCIBus *pci_get_bus(const PCIDevice *dev) return PCI_BUS(qdev_get_parent_bus(DEVICE(dev))); } int pci_bus_num(PCIBus *s); +void pci_bus_range(PCIBus *bus, int *min_bus, int *max_bus); static inline int pci_dev_bus_num(const PCIDevice *dev) { return pci_bus_num(pci_get_bus(dev)); --=20 2.19.1 From nobody Fri May 17 05:26:25 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=huawei.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1625749195869202.04965880686632; Thu, 8 Jul 2021 05:59:55 -0700 (PDT) Received: from localhost ([::1]:34404 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1m1TdC-0004Wo-K1 for importer@patchew.org; Thu, 08 Jul 2021 08:59:54 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:41306) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1m1TZJ-0004LY-AV; Thu, 08 Jul 2021 08:55:53 -0400 Received: from szxga02-in.huawei.com ([45.249.212.188]:2063) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1m1TZE-0005eO-R0; Thu, 08 Jul 2021 08:55:53 -0400 Received: from dggemv703-chm.china.huawei.com (unknown [172.30.72.55]) by szxga02-in.huawei.com (SkyGuard) with ESMTP id 4GLGQt3jrmzcb9v; Thu, 8 Jul 2021 20:52:22 +0800 (CST) Received: from dggpemm500009.china.huawei.com (7.185.36.225) by dggemv703-chm.china.huawei.com (10.3.19.46) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2176.2; Thu, 8 Jul 2021 20:55:31 +0800 Received: from huawei.com (10.174.185.226) by dggpemm500009.china.huawei.com (7.185.36.225) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2176.2; Thu, 8 Jul 2021 20:55:30 +0800 From: Wang Xingang To: , , , , , , , , , , Subject: [PATCH v5 6/9] hw/arm/virt-acpi-build: Add IORT support to bypass SMMUv3 Date: Thu, 8 Jul 2021 12:55:16 +0000 Message-ID: <1625748919-52456-7-git-send-email-wangxingang5@huawei.com> X-Mailer: git-send-email 2.6.4.windows.1 In-Reply-To: <1625748919-52456-1-git-send-email-wangxingang5@huawei.com> References: <1625748919-52456-1-git-send-email-wangxingang5@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.174.185.226] X-ClientProxiedBy: dggems704-chm.china.huawei.com (10.3.19.181) To dggpemm500009.china.huawei.com (7.185.36.225) X-CFilter-Loop: Reflected Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=45.249.212.188; envelope-from=wangxingang5@huawei.com; helo=szxga02-in.huawei.com X-Spam_score_int: -41 X-Spam_score: -4.2 X-Spam_bar: ---- X-Spam_report: (-4.2 / 5.0 requ) BAYES_00=-1.9, RCVD_IN_DNSWL_MED=-2.3, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: xieyingtai@huawei.com, wangxingang5@huawei.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZM-MESSAGEID: 1625749197174100001 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Xingang Wang When we build IORT table with SMMUv3 and bypass iommu feature enabled, we can no longer setup one map from RC to SMMUv3 covering the whole RIDs. We need to walk the PCI bus and check whether the root bus will bypass iommu, setup RC -> SMMUv3 -> ITS map for RC which will not bypass iommu. When a SMMUv3 node exist, we setup the idmap from SMMUv3 to ITS covering the whole RIDs, and only modify the map from RC to SMMUv3. We build RC -> SMMUv3 -> ITS map for root bus with bypass_iommu disabled, and build idmap from RC to ITS directly for the rest of the whole RID space. For example we run qemu with command line: qemu/build/aarch64-softmmu/qemu-system-aarch64 \ -kernel arch/arm64/boot/Image \ -enable-kvm \ -cpu host \ -m 8G \ -smp 8,sockets=3D2,cores=3D4,threads=3D1 \ -machine virt,kernel_irqchip=3Don,gic-version=3D3,iommu=3Dsmmuv3,default_b= us_bypass_iommu=3Dtrue \ -drive file=3D./QEMU_EFI-pflash.raw,if=3Dpflash,format=3Draw,unit=3D0,read= only=3Don \ -device pxb-pcie,bus_nr=3D0x10,id=3Dpci.10,bus=3Dpcie.0,addr=3D0x3.0x1 \ -device pxb-pcie,bus_nr=3D0x20,id=3Dpci.20,bus=3Dpcie.0,addr=3D0x3.0x2,byp= ass_iommu=3Dtrue \ -device pcie-root-port,port=3D0x20,chassis=3D1,id=3Dpci.1,bus=3Dpcie.0,add= r=3D0x2 \ -device pcie-root-port,port=3D0x20,chassis=3D11,id=3Dpci.11,bus=3Dpci.10,a= ddr=3D0x1 \ -device pcie-root-port,port=3D0x20,chassis=3D21,id=3Dpci.21,bus=3Dpci.20,a= ddr=3D0x1 \ -device virtio-scsi-pci,id=3Dscsi0,bus=3Dpci.1,addr=3D0x1 \ -device virtio-scsi-pci,id=3Dscsi1,bus=3Dpci.11,addr=3D0x1 \ -device virtio-scsi-pci,id=3Dscsi2,bus=3Dpci.21,addr=3D0x1 \ -initrd /mnt/davinci/wxg/kill-linux/rootfs/mfs.cpio.gz \ -nographic \ -append "rdinit=3Dinit console=3DttyAMA0 earlycon=3Dpl011,0x9000000 nokasl= r" \ And we get guest configuration: -+-[0000:20]---01.0-[21]-- +-[0000:10]---01.0-[11]-- \-[0000:00]-+-00.0 Device 1b36:0008 +-01.0 Device 1af4:1000 \-02.0-[01]-- With bypass_iommu enabled, the attached devices will bypass iommu. /sys/class/iommu/smmu3.0x0000000009050000/ |-- device -> ../../../arm-smmu-v3.0.auto |-- devices | `-- 0000:10:01.0 -> ../../../../../pci0000:10/0000:10:01.0 Signed-off-by: Xingang Wang --- hw/arm/virt-acpi-build.c | 114 +++++++++++++++++++++++++++++++++++---- 1 file changed, 103 insertions(+), 11 deletions(-) diff --git a/hw/arm/virt-acpi-build.c b/hw/arm/virt-acpi-build.c index f1024843dd..037cc1fd82 100644 --- a/hw/arm/virt-acpi-build.c +++ b/hw/arm/virt-acpi-build.c @@ -44,6 +44,7 @@ #include "hw/acpi/tpm.h" #include "hw/pci/pcie_host.h" #include "hw/pci/pci.h" +#include "hw/pci/pci_bus.h" #include "hw/pci-host/gpex.h" #include "hw/arm/virt.h" #include "hw/mem/nvdimm.h" @@ -239,23 +240,89 @@ static void acpi_dsdt_add_tpm(Aml *scope, VirtMachine= State *vms) } #endif =20 +/* Build the iort ID mapping to SMMUv3 for a given PCI host bridge */ +static int +iort_host_bridges(Object *obj, void *opaque) +{ + GArray *idmap_blob =3D opaque; + + if (object_dynamic_cast(obj, TYPE_PCI_HOST_BRIDGE)) { + PCIBus *bus =3D PCI_HOST_BRIDGE(obj)->bus; + + if (bus && !pci_bus_bypass_iommu(bus)) { + int min_bus, max_bus; + + pci_bus_range(bus, &min_bus, &max_bus); + + AcpiIortIdMapping idmap =3D { + .input_base =3D min_bus << 8, + .id_count =3D (max_bus - min_bus + 1) << 8, + }; + g_array_append_val(idmap_blob, idmap); + } + } + + return 0; +} + +static int iort_idmap_compare(gconstpointer a, gconstpointer b) +{ + AcpiIortIdMapping *idmap_a =3D (AcpiIortIdMapping *)a; + AcpiIortIdMapping *idmap_b =3D (AcpiIortIdMapping *)b; + + return idmap_a->input_base - idmap_b->input_base; +} + static void build_iort(GArray *table_data, BIOSLinker *linker, VirtMachineState *vms) { - int nb_nodes, iort_start =3D table_data->len; + int i, nb_nodes, rc_mapping_count, iort_start =3D table_data->len; AcpiIortIdMapping *idmap; AcpiIortItsGroup *its; AcpiIortTable *iort; AcpiIortSmmu3 *smmu; size_t node_size, iort_node_offset, iort_length, smmu_offset =3D 0; AcpiIortRC *rc; + GArray *smmu_idmaps =3D g_array_new(false, true, sizeof(AcpiIortIdMapp= ing)); + GArray *its_idmaps =3D g_array_new(false, true, sizeof(AcpiIortIdMappi= ng)); =20 iort =3D acpi_data_push(table_data, sizeof(*iort)); =20 if (vms->iommu =3D=3D VIRT_IOMMU_SMMUV3) { + AcpiIortIdMapping next_range =3D {0}; + + object_child_foreach_recursive(object_get_root(), + iort_host_bridges, smmu_idmaps); + + /* Sort the smmu idmap by input_base */ + g_array_sort(smmu_idmaps, iort_idmap_compare); + + /* + * Split the whole RIDs by mapping from RC to SMMU, + * build the ID mapping from RC to ITS directly. + */ + for (i =3D 0; i < smmu_idmaps->len; i++) { + idmap =3D &g_array_index(smmu_idmaps, AcpiIortIdMapping, i); + + if (next_range.input_base < idmap->input_base) { + next_range.id_count =3D idmap->input_base - next_range.inp= ut_base; + g_array_append_val(its_idmaps, next_range); + } + + next_range.input_base =3D idmap->input_base + idmap->id_count; + } + + /* Append the last RC -> ITS ID mapping */ + if (next_range.input_base < 0xFFFF) { + next_range.id_count =3D 0xFFFF - next_range.input_base; + g_array_append_val(its_idmaps, next_range); + } + nb_nodes =3D 3; /* RC, ITS, SMMUv3 */ + rc_mapping_count =3D smmu_idmaps->len + its_idmaps->len; } else { nb_nodes =3D 2; /* RC, ITS */ + rc_mapping_count =3D 1; } =20 iort_length =3D sizeof(*iort); @@ -307,13 +374,13 @@ build_iort(GArray *table_data, BIOSLinker *linker, Vi= rtMachineState *vms) } =20 /* Root Complex Node */ - node_size =3D sizeof(*rc) + sizeof(*idmap); + node_size =3D sizeof(*rc) + sizeof(*idmap) * rc_mapping_count; iort_length +=3D node_size; rc =3D acpi_data_push(table_data, node_size); =20 rc->type =3D ACPI_IORT_NODE_PCI_ROOT_COMPLEX; rc->length =3D cpu_to_le16(node_size); - rc->mapping_count =3D cpu_to_le32(1); + rc->mapping_count =3D cpu_to_le32(rc_mapping_count); rc->mapping_offset =3D cpu_to_le32(sizeof(*rc)); =20 /* fully coherent device */ @@ -321,20 +388,45 @@ build_iort(GArray *table_data, BIOSLinker *linker, Vi= rtMachineState *vms) rc->memory_properties.memory_flags =3D 0x3; /* CCA =3D CPM =3D DCAS = =3D 1 */ rc->pci_segment_number =3D 0; /* MCFG pci_segment */ =20 - /* Identity RID mapping covering the whole input RID range */ - idmap =3D &rc->id_mapping_array[0]; - idmap->input_base =3D 0; - idmap->id_count =3D cpu_to_le32(0xFFFF); - idmap->output_base =3D 0; - if (vms->iommu =3D=3D VIRT_IOMMU_SMMUV3) { - /* output IORT node is the smmuv3 node */ - idmap->output_reference =3D cpu_to_le32(smmu_offset); + AcpiIortIdMapping *range; + + /* translated RIDs connect to SMMUv3 node: RC -> SMMUv3 -> ITS */ + for (i =3D 0; i < smmu_idmaps->len; i++) { + idmap =3D &rc->id_mapping_array[i]; + range =3D &g_array_index(smmu_idmaps, AcpiIortIdMapping, i); + + idmap->input_base =3D cpu_to_le32(range->input_base); + idmap->id_count =3D cpu_to_le32(range->id_count); + idmap->output_base =3D cpu_to_le32(range->input_base); + /* output IORT node is the smmuv3 node */ + idmap->output_reference =3D cpu_to_le32(smmu_offset); + } + + /* bypassed RIDs connect to ITS group node directly: RC -> ITS */ + for (i =3D 0; i < its_idmaps->len; i++) { + idmap =3D &rc->id_mapping_array[smmu_idmaps->len + i]; + range =3D &g_array_index(its_idmaps, AcpiIortIdMapping, i); + + idmap->input_base =3D cpu_to_le32(range->input_base); + idmap->id_count =3D cpu_to_le32(range->id_count); + idmap->output_base =3D cpu_to_le32(range->input_base); + /* output IORT node is the ITS group node (the first node) */ + idmap->output_reference =3D cpu_to_le32(iort_node_offset); + } } else { + /* Identity RID mapping covering the whole input RID range */ + idmap =3D &rc->id_mapping_array[0]; + idmap->input_base =3D cpu_to_le32(0); + idmap->id_count =3D cpu_to_le32(0xFFFF); + idmap->output_base =3D cpu_to_le32(0); /* output IORT node is the ITS group node (the first node) */ idmap->output_reference =3D cpu_to_le32(iort_node_offset); } =20 + g_array_free(smmu_idmaps, true); + g_array_free(its_idmaps, true); + /* * Update the pointer address in case table_data->data moves during ab= ove * acpi_data_push operations. --=20 2.19.1 From nobody Fri May 17 05:26:25 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=huawei.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1625749060402558.8616225967979; Thu, 8 Jul 2021 05:57:40 -0700 (PDT) Received: from localhost ([::1]:53852 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1m1Tb0-00078v-Rs for importer@patchew.org; Thu, 08 Jul 2021 08:57:38 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:41302) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1m1TZI-0004LM-OF; Thu, 08 Jul 2021 08:55:53 -0400 Received: from szxga03-in.huawei.com ([45.249.212.189]:2189) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1m1TZE-0005bS-Qj; Thu, 08 Jul 2021 08:55:52 -0400 Received: from dggemv711-chm.china.huawei.com (unknown [172.30.72.53]) by szxga03-in.huawei.com (SkyGuard) with ESMTP id 4GLGPT3y4dz77Qk; Thu, 8 Jul 2021 20:51:09 +0800 (CST) Received: from dggpemm500009.china.huawei.com (7.185.36.225) by dggemv711-chm.china.huawei.com (10.1.198.66) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2176.2; Thu, 8 Jul 2021 20:55:32 +0800 Received: from huawei.com (10.174.185.226) by dggpemm500009.china.huawei.com (7.185.36.225) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2176.2; Thu, 8 Jul 2021 20:55:31 +0800 From: Wang Xingang To: , , , , , , , , , , Subject: [PATCH v5 7/9] hw/i386/acpi-build: Add DMAR support to bypass iommu Date: Thu, 8 Jul 2021 12:55:17 +0000 Message-ID: <1625748919-52456-8-git-send-email-wangxingang5@huawei.com> X-Mailer: git-send-email 2.6.4.windows.1 In-Reply-To: <1625748919-52456-1-git-send-email-wangxingang5@huawei.com> References: <1625748919-52456-1-git-send-email-wangxingang5@huawei.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Originating-IP: [10.174.185.226] X-ClientProxiedBy: dggems704-chm.china.huawei.com (10.3.19.181) To dggpemm500009.china.huawei.com (7.185.36.225) X-CFilter-Loop: Reflected Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=45.249.212.189; envelope-from=wangxingang5@huawei.com; helo=szxga03-in.huawei.com X-Spam_score_int: -41 X-Spam_score: -4.2 X-Spam_bar: ---- X-Spam_report: (-4.2 / 5.0 requ) BAYES_00=-1.9, RCVD_IN_DNSWL_MED=-2.3, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: xieyingtai@huawei.com, wangxingang5@huawei.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZM-MESSAGEID: 1625749062543100005 From: Xingang Wang In DMAR table, the drhd is set to cover all PCI devices when intel_iommu is on. To support bypass iommu feature, we need to walk the PCI bus with bypass_iommu disabled and add explicit scope data in DMAR drhd structure. /mnt/sdb/wxg/qemu-next/qemu/build/x86_64-softmmu/qemu-system-x86_64 \ -machine q35,accel=3Dkvm,default_bus_bypass_iommu=3Dtrue \ -cpu host \ -m 16G \ -smp 36,sockets=3D2,cores=3D18,threads=3D1 \ -device pxb-pcie,bus_nr=3D0x10,id=3Dpci.10,bus=3Dpcie.0,addr=3D0x3 \ -device pxb-pcie,bus_nr=3D0x20,id=3Dpci.20,bus=3Dpcie.0,addr=3D0x4,bypass_= iommu=3Dtrue \ -device pcie-root-port,port=3D0x1,chassis=3D1,id=3Dpci.11,bus=3Dpci.10,add= r=3D0x0 \ -device pcie-root-port,port=3D0x2,chassis=3D2,id=3Dpci.21,bus=3Dpci.20,add= r=3D0x0 \ -device virtio-scsi-pci,id=3Dscsi0,bus=3Dpci.11,addr=3D0x0 \ -device virtio-scsi-pci,id=3Dscsi1,bus=3Dpci.21,addr=3D0x0 \ -drive file=3D/mnt/sdb/wxg/fedora-48g.qcow2,format=3Dqcow2,if=3Dnone,id=3D= drive-scsi0-0-0-0,cache=3Dnone,aio=3Dnative \ -device scsi-hd,bus=3Dscsi1.0,channel=3D0,scsi-id=3D0,lun=3D0,drive=3Ddriv= e-scsi0-0-0-0,id=3Dscsi0-0-0-0,bootindex=3D1 \ -device intel-iommu \ -nographic \ And we get the guest configuration: ~ lspci -vt -+-[0000:20]---00.0-[21]----00.0 Red Hat, Inc. Virtio SCSI +-[0000:10]---00.0-[11]----00.0 Red Hat, Inc. Virtio SCSI \-[0000:00]-+-00.0 Intel Corporation 82G33/G31/P35/P31 Express DRAM Contr= oller +-01.0 Device 1234:1111 +-02.0 Intel Corporation 82574L Gigabit Network Connection +-03.0 Red Hat, Inc. QEMU PCIe Expander bridge +-04.0 Red Hat, Inc. QEMU PCIe Expander bridge +-1f.0 Intel Corporation 82801IB (ICH9) LPC Interface Control= ler +-1f.2 Intel Corporation 82801IR/IO/IH (ICH9R/DO/DH) 6 port S= ATA Controller [AHCI mode] \-1f.3 Intel Corporation 82801I (ICH9 Family) SMBus Controller With bypass_iommu enabled on root bus, the attached devices will bypass iom= mu: /sys/class/iommu/dmar0 =E2=94=9C=E2=94=80=E2=94=80 devices =E2=94=82=C2=A0=C2=A0 =E2=94=9C=E2=94=80=E2=94=80 0000:10:00.0 -> ../../../= ../pci0000:10/0000:10:00.0 =E2=94=82=C2=A0=C2=A0 =E2=94=94=E2=94=80=E2=94=80 0000:11:00.0 -> ../../../= ../pci0000:10/0000:10:00.0/0000:11:00.0 Signed-off-by: Xingang Wang --- hw/i386/acpi-build.c | 68 ++++++++++++++++++++++++++++++++++++++++++-- 1 file changed, 66 insertions(+), 2 deletions(-) diff --git a/hw/i386/acpi-build.c b/hw/i386/acpi-build.c index 796ffc6f5c..1b972b4913 100644 --- a/hw/i386/acpi-build.c +++ b/hw/i386/acpi-build.c @@ -2005,6 +2005,56 @@ build_srat(GArray *table_data, BIOSLinker *linker, M= achineState *machine) x86ms->oem_table_id); } =20 +/* + * Insert DMAR scope for PCI bridges and endpoint devcie + */ +static void +insert_scope(PCIBus *bus, PCIDevice *dev, void *opaque) +{ + GArray *scope_blob =3D opaque; + AcpiDmarDeviceScope *scope =3D NULL; + + if (object_dynamic_cast(OBJECT(dev), TYPE_PCI_BRIDGE)) { + /* Dmar Scope Type: 0x02 for PCI Bridge */ + build_append_int_noprefix(scope_blob, 0x02, 1); + } else { + /* Dmar Scope Type: 0x01 for PCI Endpoint Device */ + build_append_int_noprefix(scope_blob, 0x01, 1); + } + + /* length */ + build_append_int_noprefix(scope_blob, + sizeof(*scope) + sizeof(scope->path[0]), 1); + /* reserved */ + build_append_int_noprefix(scope_blob, 0, 2); + /* enumeration_id */ + build_append_int_noprefix(scope_blob, 0, 1); + /* bus */ + build_append_int_noprefix(scope_blob, pci_bus_num(bus), 1); + /* device */ + build_append_int_noprefix(scope_blob, PCI_SLOT(dev->devfn), 1); + /* function */ + build_append_int_noprefix(scope_blob, PCI_FUNC(dev->devfn), 1); +} + +/* For a given PCI host bridge, walk and insert DMAR scope */ +static int +dmar_host_bridges(Object *obj, void *opaque) +{ + GArray *scope_blob =3D opaque; + + if (object_dynamic_cast(obj, TYPE_PCI_HOST_BRIDGE)) { + PCIBus *bus =3D PCI_HOST_BRIDGE(obj)->bus; + + if (bus && !pci_bus_bypass_iommu(bus)) { + pci_for_each_device(bus, pci_bus_num(bus), insert_scope, + scope_blob); + } + } + + return 0; +} + /* * VT-d spec 8.1 DMA Remapping Reporting Structure * (version Oct. 2014 or later) @@ -2024,6 +2074,15 @@ build_dmar_q35(GArray *table_data, BIOSLinker *linke= r, const char *oem_id, /* Root complex IOAPIC use one path[0] only */ size_t ioapic_scope_size =3D sizeof(*scope) + sizeof(scope->path[0]); IntelIOMMUState *intel_iommu =3D INTEL_IOMMU_DEVICE(iommu); + GArray *scope_blob =3D g_array_new(false, true, 1); + + /* + * A PCI bus walk, for each PCI host bridge. + * Insert scope for each PCI bridge and endpoint device which + * is attached to a bus with iommu enabled. + */ + object_child_foreach_recursive(object_get_root(), + dmar_host_bridges, scope_blob); =20 assert(iommu); if (x86_iommu_ir_supported(iommu)) { @@ -2037,8 +2096,9 @@ build_dmar_q35(GArray *table_data, BIOSLinker *linker= , const char *oem_id, /* DMAR Remapping Hardware Unit Definition structure */ drhd =3D acpi_data_push(table_data, sizeof(*drhd) + ioapic_scope_size); drhd->type =3D cpu_to_le16(ACPI_DMAR_TYPE_HARDWARE_UNIT); - drhd->length =3D cpu_to_le16(sizeof(*drhd) + ioapic_scope_size); - drhd->flags =3D ACPI_DMAR_INCLUDE_PCI_ALL; + drhd->length =3D + cpu_to_le16(sizeof(*drhd) + ioapic_scope_size + scope_blob->len); + drhd->flags =3D 0; /* Don't include all pci device */ drhd->pci_segment =3D cpu_to_le16(0); drhd->address =3D cpu_to_le64(Q35_HOST_BRIDGE_IOMMU_ADDR); =20 @@ -2052,6 +2112,10 @@ build_dmar_q35(GArray *table_data, BIOSLinker *linke= r, const char *oem_id, scope->path[0].device =3D PCI_SLOT(Q35_PSEUDO_DEVFN_IOAPIC); scope->path[0].function =3D PCI_FUNC(Q35_PSEUDO_DEVFN_IOAPIC); =20 + /* Add scope found above */ + g_array_append_vals(table_data, scope_blob->data, scope_blob->len); + g_array_free(scope_blob, true); + if (iommu->dt_supported) { atsr =3D acpi_data_push(table_data, sizeof(*atsr)); atsr->type =3D cpu_to_le16(ACPI_DMAR_TYPE_ATSR); --=20 2.19.1 From nobody Fri May 17 05:26:25 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=huawei.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1625749430257126.08420637296865; Thu, 8 Jul 2021 06:03:50 -0700 (PDT) Received: from localhost ([::1]:44144 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1m1Tgy-0002oo-SK for importer@patchew.org; Thu, 08 Jul 2021 09:03:48 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:41334) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1m1TZK-0004M4-CD; Thu, 08 Jul 2021 08:55:54 -0400 Received: from szxga01-in.huawei.com ([45.249.212.187]:2498) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1m1TZF-0005bX-18; Thu, 08 Jul 2021 08:55:54 -0400 Received: from dggemv704-chm.china.huawei.com (unknown [172.30.72.53]) by szxga01-in.huawei.com (SkyGuard) with ESMTP id 4GLGNB2QjCzXq4V; Thu, 8 Jul 2021 20:50:02 +0800 (CST) Received: from dggpemm500009.china.huawei.com (7.185.36.225) by dggemv704-chm.china.huawei.com (10.3.19.47) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2176.2; Thu, 8 Jul 2021 20:55:32 +0800 Received: from huawei.com (10.174.185.226) by dggpemm500009.china.huawei.com (7.185.36.225) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2176.2; Thu, 8 Jul 2021 20:55:31 +0800 From: Wang Xingang To: , , , , , , , , , , Subject: [PATCH v5 8/9] hw/i386/acpi-build: Add IVRS support to bypass iommu Date: Thu, 8 Jul 2021 12:55:18 +0000 Message-ID: <1625748919-52456-9-git-send-email-wangxingang5@huawei.com> X-Mailer: git-send-email 2.6.4.windows.1 In-Reply-To: <1625748919-52456-1-git-send-email-wangxingang5@huawei.com> References: <1625748919-52456-1-git-send-email-wangxingang5@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.174.185.226] X-ClientProxiedBy: dggems704-chm.china.huawei.com (10.3.19.181) To dggpemm500009.china.huawei.com (7.185.36.225) X-CFilter-Loop: Reflected Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=45.249.212.187; envelope-from=wangxingang5@huawei.com; helo=szxga01-in.huawei.com X-Spam_score_int: -41 X-Spam_score: -4.2 X-Spam_bar: ---- X-Spam_report: (-4.2 / 5.0 requ) BAYES_00=-1.9, RCVD_IN_DNSWL_MED=-2.3, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: xieyingtai@huawei.com, wangxingang5@huawei.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZM-MESSAGEID: 1625749438513100001 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Xingang Wang Check bypass_iommu to exclude the devices which will bypass iommu. Signed-off-by: Xingang Wang --- hw/i386/acpi-build.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/hw/i386/acpi-build.c b/hw/i386/acpi-build.c index 1b972b4913..7c888f05bf 100644 --- a/hw/i386/acpi-build.c +++ b/hw/i386/acpi-build.c @@ -2246,7 +2246,7 @@ ivrs_host_bridges(Object *obj, void *opaque) if (object_dynamic_cast(obj, TYPE_PCI_HOST_BRIDGE)) { PCIBus *bus =3D PCI_HOST_BRIDGE(obj)->bus; =20 - if (bus) { + if (bus && !pci_bus_bypass_iommu(bus)) { pci_for_each_device(bus, pci_bus_num(bus), insert_ivhd, ivhd_b= lob); } } --=20 2.19.1 From nobody Fri May 17 05:26:25 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=huawei.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1625749107621612.0240144029327; Thu, 8 Jul 2021 05:58:27 -0700 (PDT) Received: from localhost ([::1]:56286 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1m1Tbl-0000Kh-Nm for importer@patchew.org; Thu, 08 Jul 2021 08:58:25 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:41338) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1m1TZK-0004MM-M5; Thu, 08 Jul 2021 08:55:54 -0400 Received: from szxga02-in.huawei.com ([45.249.212.188]:2064) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1m1TZF-0005eV-8V; Thu, 08 Jul 2021 08:55:54 -0400 Received: from dggemv703-chm.china.huawei.com (unknown [172.30.72.55]) by szxga02-in.huawei.com (SkyGuard) with ESMTP id 4GLGQt46Gfzcb9x; Thu, 8 Jul 2021 20:52:22 +0800 (CST) Received: from dggpemm500009.china.huawei.com (7.185.36.225) by dggemv703-chm.china.huawei.com (10.3.19.46) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2176.2; Thu, 8 Jul 2021 20:55:33 +0800 Received: from huawei.com (10.174.185.226) by dggpemm500009.china.huawei.com (7.185.36.225) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2176.2; Thu, 8 Jul 2021 20:55:32 +0800 From: Wang Xingang To: , , , , , , , , , , Subject: [PATCH v5 9/9] docs: Add documentation for iommu bypass Date: Thu, 8 Jul 2021 12:55:19 +0000 Message-ID: <1625748919-52456-10-git-send-email-wangxingang5@huawei.com> X-Mailer: git-send-email 2.6.4.windows.1 In-Reply-To: <1625748919-52456-1-git-send-email-wangxingang5@huawei.com> References: <1625748919-52456-1-git-send-email-wangxingang5@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.174.185.226] X-ClientProxiedBy: dggems704-chm.china.huawei.com (10.3.19.181) To dggpemm500009.china.huawei.com (7.185.36.225) X-CFilter-Loop: Reflected Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=45.249.212.188; envelope-from=wangxingang5@huawei.com; helo=szxga02-in.huawei.com X-Spam_score_int: -41 X-Spam_score: -4.2 X-Spam_bar: ---- X-Spam_report: (-4.2 / 5.0 requ) BAYES_00=-1.9, RCVD_IN_DNSWL_MED=-2.3, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: xieyingtai@huawei.com, wangxingang5@huawei.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZM-MESSAGEID: 1625749108325100001 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Xingang Wang Signed-off-by: Xingang Wang --- docs/bypass-iommu.txt | 89 +++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 89 insertions(+) create mode 100644 docs/bypass-iommu.txt diff --git a/docs/bypass-iommu.txt b/docs/bypass-iommu.txt new file mode 100644 index 0000000000..e6677bddd3 --- /dev/null +++ b/docs/bypass-iommu.txt @@ -0,0 +1,89 @@ +BYPASS IOMMU PROPERTY +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D + +Description +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D +Traditionally, there is a global switch to enable/disable vIOMMU. All +devices in the system can only support go through vIOMMU or not, which +is not flexible. We introduce this bypass iommu property to support +coexist of devices go through vIOMMU and devices not. This is useful to +passthrough devices with no-iommu mode and devices go through vIOMMU in +the same virtual machine. + +PCI host bridges have a bypass_iommu property. This property is used to +determine whether the devices attached on the PCI host bridge will bypass +virtual iommu. The bypass_iommu property is valid only when there is a +virtual iommu in the system, it is implemented to allow some devices to +bypass vIOMMU. When bypass_iommu property is not set for a host bridge, +the attached devices will go through vIOMMU by default. + +Usage +=3D=3D=3D=3D=3D +The bypass iommu feature support PXB host bridge and default main host +bridge, we add a bypass_iommu property for PXB and default_bus_bypass_iommu +for machine. Note that default_bus_bypass_iommu is available only when +the 'q35' machine type on x86 architecture and the 'virt' machine type +on AArch64. Other machine types do not support bypass iommu for default +root bus. + +1. The following is the bypass iommu options: + (1) PCI expander bridge + qemu -device pxb-pcie,bus_nr=3D0x10,addr=3D0x1,bypass_iommu=3Dtrue + (2) Arm default host bridge + qemu -machine virt,iommu=3Dsmmuv3,default_bus_bypass_iommu=3Dtrue + (3) X86 default root bus bypass iommu: + qemu -machine q35,default_bus_bypass_iommu=3Dtrue + +2. Here is the detailed qemu command line for 'virt' machine with PXB on +AArch64: + +qemu-system-aarch64 \ + -machine virt,kernel_irqchip=3Don,iommu=3Dsmmuv3,default_bus_bypass_iommu= =3Dtrue \ + -device pxb-pcie,bus_nr=3D0x10,id=3Dpci.10,bus=3Dpcie.0,addr=3D0x3.0x1 \ + -device pxb-pcie,bus_nr=3D0x20,id=3Dpci.20,bus=3Dpcie.0,addr=3D0x3.0x2,by= pass_iommu=3Dtrue \ + +And we got: + - a default host bridge which bypass SMMUv3 + - a pxb host bridge which go through SMMUv3 + - a pxb host bridge which bypass SMMUv3 + +3. Here is the detailed qemu command line for 'q35' machine with PXB on +x86 architecture: + +qemu-system-x86_64 \ + -machine q35,accel=3Dkvm,default_bus_bypass_iommu=3Dtrue \ + -device pxb-pcie,bus_nr=3D0x10,id=3Dpci.10,bus=3Dpcie.0,addr=3D0x3 \ + -device pxb-pcie,bus_nr=3D0x20,id=3Dpci.20,bus=3Dpcie.0,addr=3D0x4,bypass= _iommu=3Dtrue \ + -device intel-iommu \ + +And we got: + - a default host bridge which bypass iommu + - a pxb host bridge which go through iommu + - a pxb host bridge which bypass iommu + +Limitations +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D +There might be potential security risk when devices bypass iommu, because +devices might send malicious dma request to virtual machine if there is no +iommu isolation. So it would be necessary to only bypass iommu for trusted +device. + +Implementation +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D +The bypass iommu feature includes: + - Address space + Add bypass iommu property check of PCI Host and do not get iommu address + space for devices bypass iommu. + - Arm SMMUv3 support + We traverse all PCI root bus and get bus number ranges, then build expl= icit + RID mapping for devices which do not bypass iommu. + - X86 IOMMU support + To support Intel iommu, we traverse all PCI host bridge and get informa= tion + of devices which do not bypass iommu, then fill the DMAR drhd struct wi= th + explicit device scope info. To support AMD iommu, add check of bypass i= ommu + when traverse the PCI hsot bridge. + - Machine and PXB options + We add bypass iommu options in machine option for default root bus, and= add + option for PXB also. Note that the default value of bypass iommu is fal= se, + so that the devices will by default go through iommu if there exist one. + --=20 2.19.1