From nobody Fri Nov 29 14:52:30 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=fail(p=none dis=none) header.from=arm.com Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1634724370334641.4117125388478; Wed, 20 Oct 2021 03:06:10 -0700 (PDT) Received: from list by lists.xenproject.org with outflank-mailman.213760.372087 (Exim 4.92) (envelope-from ) id 1md8Tq-0007uG-Rs; Wed, 20 Oct 2021 10:05:54 +0000 Received: by outflank-mailman (output) from mailman id 213760.372087; Wed, 20 Oct 2021 10:05:54 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1md8Tq-0007u9-Oh; Wed, 20 Oct 2021 10:05:54 +0000 Received: by outflank-mailman (input) for mailman id 213760; Wed, 20 Oct 2021 10:05:52 +0000 Received: from us1-rack-iad1.inumbo.com ([172.99.69.81]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1md8To-0007sq-Ox for xen-devel@lists.xenproject.org; Wed, 20 Oct 2021 10:05:52 +0000 Received: from foss.arm.com (unknown [217.140.110.172]) by us1-rack-iad1.inumbo.com (Halon) with ESMTP id c43af9f5-e5cc-4bd9-91d2-639f0220f85b; Wed, 20 Oct 2021 10:05:51 +0000 (UTC) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 8BB1F106F; Wed, 20 Oct 2021 03:05:51 -0700 (PDT) Received: from e109506.cambridge.arm.com (e109506.cambridge.arm.com [10.1.199.62]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 088FA3F70D; Wed, 20 Oct 2021 03:05:49 -0700 (PDT) X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: c43af9f5-e5cc-4bd9-91d2-639f0220f85b From: Bertrand Marquis To: xen-devel@lists.xenproject.org Cc: iwj@xenproject.org, Oleksandr_Andrushchenko@epam.com, Stefano Stabellini , Julien Grall , Volodymyr Babchuk , Jan Beulich , Andrew Cooper , =?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= , Wei Liu , Paul Durrant Subject: [PATCH v3 1/1] xen/pci: Install vpci handlers on x86 and fix error paths Date: Wed, 20 Oct 2021 11:05:37 +0100 Message-Id: X-Mailer: git-send-email 2.25.1 In-Reply-To: References: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ZM-MESSAGEID: 1634724373584100001 Content-Type: text/plain; charset="utf-8" Xen might not be able to discover at boot time all devices or some devices might appear after specific actions from dom0. In this case dom0 can use the PHYSDEVOP_pci_device_add to signal some PCI devices to Xen. As those devices where not known from Xen before, the vpci handlers must be properly installed during pci_device_add for x86 PVH Dom0, in the same way as what is done currently on arm (where Xen does not detect PCI devices but relies on Dom0 to declare them all the time). So this patch is removing the ifdef protecting the call to vpci_add_handlers and the comment which was arm specific. vpci_add_handlers is called on during pci_device_add which can be called at runtime through hypercall physdev_op. Remove __hwdom_init as the call is not limited anymore to hardware domain init and fix linker script to only keep vpci_array in rodata section. Add missing vpci handlers cleanup during pci_device_remove and in case of error with iommu during pci_device_add. Move code adding the domain to the pdev domain_list as vpci_add_handlers needs this to be set and remove it from the list in the error path. Exit early of vpci_remove_device if the domain has no vpci support. Add empty static inline for vpci_remove_device when CONFIG_VPCI is not defined. Add an ASSERT in vpci_add_handlers to check that the function is not called twice for the same device. Fixes: d59168dc05 ("xen/arm: Enable the existing x86 virtual PCI support for ARM") Suggested-by: Jan Beulich Signed-off-by: Bertrand Marquis Acked-by: Julien Grall Reviewed-by: Jan Beulich --- Changes in v3 - change title (s/exit/error/ and s/path/paths) - add early exit in vpci_remove_device if the domain has no vpci support - add ASSERT in vpci_add_handlers to check that the call is only made once per device - move the call adding the domain in the pdev domain list and remove it in the error path Changes in v2 - add comment suggested by Jan on top of vpci_add_handlers call - merge the 3 patches of the serie in one patch and renamed it - fix x86 and arm linker script to only keep vpci_array in rodata and only when CONFIG_VPCI is set. --- xen/arch/arm/xen.lds.S | 9 +-------- xen/arch/x86/xen.lds.S | 9 +-------- xen/drivers/passthrough/pci.c | 14 ++++++++------ xen/drivers/vpci/vpci.c | 8 +++++++- xen/include/xen/vpci.h | 2 ++ 5 files changed, 19 insertions(+), 23 deletions(-) diff --git a/xen/arch/arm/xen.lds.S b/xen/arch/arm/xen.lds.S index b773f91f1c..08016948ab 100644 --- a/xen/arch/arm/xen.lds.S +++ b/xen/arch/arm/xen.lds.S @@ -60,7 +60,7 @@ SECTIONS *(.proc.info) __proc_info_end =3D .; =20 -#if defined(CONFIG_HAS_VPCI) && defined(CONFIG_LATE_HWDOM) +#ifdef CONFIG_HAS_VPCI . =3D ALIGN(POINTER_ALIGN); __start_vpci_array =3D .; *(SORT(.data.vpci.*)) @@ -189,13 +189,6 @@ SECTIONS *(.init_array) *(SORT(.init_array.*)) __ctors_end =3D .; - -#if defined(CONFIG_HAS_VPCI) && !defined(CONFIG_LATE_HWDOM) - . =3D ALIGN(POINTER_ALIGN); - __start_vpci_array =3D .; - *(SORT(.data.vpci.*)) - __end_vpci_array =3D .; -#endif } :text __init_end_efi =3D .; . =3D ALIGN(STACK_SIZE); diff --git a/xen/arch/x86/xen.lds.S b/xen/arch/x86/xen.lds.S index 11b1da2154..87e344d4dd 100644 --- a/xen/arch/x86/xen.lds.S +++ b/xen/arch/x86/xen.lds.S @@ -134,7 +134,7 @@ SECTIONS *(.ex_table.pre) __stop___pre_ex_table =3D .; =20 -#if defined(CONFIG_HAS_VPCI) && defined(CONFIG_LATE_HWDOM) +#ifdef CONFIG_HAS_VPCI . =3D ALIGN(POINTER_ALIGN); __start_vpci_array =3D .; *(SORT(.data.vpci.*)) @@ -247,13 +247,6 @@ SECTIONS *(.init_array) *(SORT(.init_array.*)) __ctors_end =3D .; - -#if defined(CONFIG_HAS_VPCI) && !defined(CONFIG_LATE_HWDOM) - . =3D ALIGN(POINTER_ALIGN); - __start_vpci_array =3D .; - *(SORT(.data.vpci.*)) - __end_vpci_array =3D .; -#endif } PHDR(text) =20 . =3D ALIGN(SECTION_ALIGN); diff --git a/xen/drivers/passthrough/pci.c b/xen/drivers/passthrough/pci.c index 35e0190796..0d8ab2e716 100644 --- a/xen/drivers/passthrough/pci.c +++ b/xen/drivers/passthrough/pci.c @@ -756,27 +756,28 @@ int pci_add_device(u16 seg, u8 bus, u8 devfn, if ( !pdev->domain ) { pdev->domain =3D hardware_domain; -#ifdef CONFIG_ARM + list_add(&pdev->domain_list, &hardware_domain->pdev_list); + /* - * On ARM PCI devices discovery will be done by Dom0. Add vpci han= dler - * when Dom0 inform XEN to add the PCI devices in XEN. + * For devices not discovered by Xen during boot, add vPCI handlers + * when Dom0 first informs Xen about such devices. */ ret =3D vpci_add_handlers(pdev); if ( ret ) { printk(XENLOG_ERR "Setup of vPCI failed: %d\n", ret); + list_del(&pdev->domain_list); pdev->domain =3D NULL; goto out; } -#endif ret =3D iommu_add_device(pdev); if ( ret ) { + vpci_remove_device(pdev); + list_del(&pdev->domain_list); pdev->domain =3D NULL; goto out; } - - list_add(&pdev->domain_list, &hardware_domain->pdev_list); } else iommu_enable_device(pdev); @@ -819,6 +820,7 @@ int pci_remove_device(u16 seg, u8 bus, u8 devfn) list_for_each_entry ( pdev, &pseg->alldevs_list, alldevs_list ) if ( pdev->bus =3D=3D bus && pdev->devfn =3D=3D devfn ) { + vpci_remove_device(pdev); pci_cleanup_msi(pdev); ret =3D iommu_remove_device(pdev); if ( pdev->domain ) diff --git a/xen/drivers/vpci/vpci.c b/xen/drivers/vpci/vpci.c index decf7d87a1..657697fe34 100644 --- a/xen/drivers/vpci/vpci.c +++ b/xen/drivers/vpci/vpci.c @@ -37,6 +37,9 @@ extern vpci_register_init_t *const __end_vpci_array[]; =20 void vpci_remove_device(struct pci_dev *pdev) { + if ( !has_vpci(pdev->domain) ) + return; + spin_lock(&pdev->vpci->lock); while ( !list_empty(&pdev->vpci->handlers) ) { @@ -54,7 +57,7 @@ void vpci_remove_device(struct pci_dev *pdev) pdev->vpci =3D NULL; } =20 -int __hwdom_init vpci_add_handlers(struct pci_dev *pdev) +int vpci_add_handlers(struct pci_dev *pdev) { unsigned int i; int rc =3D 0; @@ -62,6 +65,9 @@ int __hwdom_init vpci_add_handlers(struct pci_dev *pdev) if ( !has_vpci(pdev->domain) ) return 0; =20 + /* We should not get here twice for the same device. */ + ASSERT(!pdev->vpci); + pdev->vpci =3D xzalloc(struct vpci); if ( !pdev->vpci ) return -ENOMEM; diff --git a/xen/include/xen/vpci.h b/xen/include/xen/vpci.h index 6746c2589a..9ea66e033f 100644 --- a/xen/include/xen/vpci.h +++ b/xen/include/xen/vpci.h @@ -230,6 +230,8 @@ static inline int vpci_add_handlers(struct pci_dev *pde= v) return 0; } =20 +static inline void vpci_remove_device(struct pci_dev *pdev) { } + static inline void vpci_dump_msi(void) { } =20 static inline uint32_t vpci_read(pci_sbdf_t sbdf, unsigned int reg, --=20 2.25.1