From nobody Fri Apr 19 00:24:29 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=none (zoho.com: 192.237.175.120 is neither permitted nor denied by domain of lists.xenproject.org) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=fail(p=none dis=none) header.from=citrix.com ARC-Seal: i=1; a=rsa-sha256; t=1570707337; cv=none; d=zoho.com; s=zohoarc; b=j2ODk3bzF2mQjQ3dYkuYTXmn5bmolAXa7AQ5od6mV+g12mgSLSYrIdgp2jR6MHs6xn14+Hctiv/DqE83YbUNjd3XmoX3veGc2co+NVkHZ1/r6CjcvEfBryzerttV3/cY8ukdTZuHhp66Rd/Ls4ge3I0wNf/JIbnf96ZUosR7VkQ= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1570707337; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=6MMJX251P2JE4HmfOv5uAwbI4BL/rd+wf1J1FNrB7Yg=; b=FPPawS51MMNdfj9fQ0e9RYx6XM+CgfL1cnP+7UUsJ0xu0ZRAxZIBDrZgCOW2deiJ9P0PHhj6it9pakgKlh1PJUpfdn48t6pYDielws88E0+tzgW2iFcr+cyn6ltZZ6SKV2s/WSsX6Km7foMq1/Zp1r7lMkMJDHBpc0J424xhxOA= ARC-Authentication-Results: i=1; mx.zoho.com; dkim=fail; spf=none (zoho.com: 192.237.175.120 is neither permitted nor denied by domain of lists.xenproject.org) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1570707337930117.92570142712248; Thu, 10 Oct 2019 04:35:37 -0700 (PDT) Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1iIWi1-0001dK-UT; Thu, 10 Oct 2019 11:34:17 +0000 Received: from us1-rack-iad1.inumbo.com ([172.99.69.81]) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1iIWi0-0001cy-9Y for xen-devel@lists.xenproject.org; Thu, 10 Oct 2019 11:34:16 +0000 Received: from esa3.hc3370-68.iphmx.com (unknown [216.71.145.155]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS id dfa8a69e-eb51-11e9-80e3-bc764e2007e4; Thu, 10 Oct 2019 11:34:09 +0000 (UTC) X-Inumbo-ID: dfa8a69e-eb51-11e9-80e3-bc764e2007e4 DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=citrix.com; s=securemail; t=1570707250; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=VJ+eNzJg3w6G9LgrbYkzOMKvORC0WkpX/C+Ntg/gB28=; b=b5UOKN9hCilz4SKUhYbNHd+dQs8cD7pYSJYJbrSO6KhnAMaToqoG73pm TkGVKjgXzCB+YJe5hJqKC8LMM+xtsnGK0EoEGQqF4L2Q7aQ+Rzri/zDhK 7Ija9tnPWoTfe9gbj3GRq6ef6AB0ZCYyCoKrrHvrUurL1A51At5X/ysUn 8=; Authentication-Results: esa3.hc3370-68.iphmx.com; dkim=none (message not signed) header.i=none; spf=None smtp.pra=roger.pau@citrix.com; spf=Pass smtp.mailfrom=roger.pau@citrix.com; spf=None smtp.helo=postmaster@mail.citrix.com Received-SPF: none (zoho.com: 192.237.175.120 is neither permitted nor denied by domain of lists.xenproject.org) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Received-SPF: None (esa3.hc3370-68.iphmx.com: no sender authenticity information available from domain of roger.pau@citrix.com) identity=pra; client-ip=162.221.158.21; receiver=esa3.hc3370-68.iphmx.com; envelope-from="roger.pau@citrix.com"; x-sender="roger.pau@citrix.com"; x-conformance=sidf_compatible Received-SPF: Pass (esa3.hc3370-68.iphmx.com: domain of roger.pau@citrix.com designates 162.221.158.21 as permitted sender) identity=mailfrom; client-ip=162.221.158.21; receiver=esa3.hc3370-68.iphmx.com; envelope-from="roger.pau@citrix.com"; x-sender="roger.pau@citrix.com"; x-conformance=sidf_compatible; x-record-type="v=spf1"; x-record-text="v=spf1 ip4:209.167.231.154 ip4:178.63.86.133 ip4:195.66.111.40/30 ip4:85.115.9.32/28 ip4:199.102.83.4 ip4:192.28.146.160 ip4:192.28.146.107 ip4:216.52.6.88 ip4:216.52.6.188 ip4:162.221.158.21 ip4:162.221.156.83 ip4:168.245.78.127 ~all" Received-SPF: None (esa3.hc3370-68.iphmx.com: no sender authenticity information available from domain of postmaster@mail.citrix.com) identity=helo; client-ip=162.221.158.21; receiver=esa3.hc3370-68.iphmx.com; envelope-from="roger.pau@citrix.com"; x-sender="postmaster@mail.citrix.com"; x-conformance=sidf_compatible IronPort-SDR: WSBXiwoDfoZ0EYVLC6mPMbq8xO2PlionKUv57DbfifhUy1ByA3yTvSAv/Y35ONS7bLW9onmSPb dndrWDe2JBcRbM1c+8UkdPXr5EUfD7qgz8VVdNESF+CaXiQNHJ2YhVCzcmB2TzOaw7ZClAX3Sz IaglbiAEbyQNlNnIB46f1xVmbDkWYv8PU07c7IDtkWQNip3qzh12YcbUlXtVU4SOwM4PSVnk1j K5bHZujt+nqgoVl6s8LaeLKt+tovo86xaGEu0JCtDL/5oBKAl2SL3WrYonkEyAs9CgkoFlfGui IWg= X-SBRS: 2.7 X-MesageID: 6709785 X-Ironport-Server: esa3.hc3370-68.iphmx.com X-Remote-IP: 162.221.158.21 X-Policy: $RELAYED X-IronPort-AV: E=Sophos;i="5.67,280,1566878400"; d="scan'208";a="6709785" From: Roger Pau Monne To: Date: Thu, 10 Oct 2019 13:33:45 +0200 Message-ID: <20191010113345.6867-3-roger.pau@citrix.com> X-Mailer: git-send-email 2.23.0 In-Reply-To: <20191010113345.6867-1-roger.pau@citrix.com> References: <20191010113345.6867-1-roger.pau@citrix.com> MIME-Version: 1.0 Subject: [Xen-devel] [PATCH v2 2/2] iommu: translate IO-APIC pins when enabling interrupt remapping X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Juergen Gross , Suravee Suthikulpanit , Wei Liu , Andrew Cooper , Jan Beulich , Roger Pau Monne Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) On Intel hardware there's currently no translation of already enabled IO-APIC pins when interrupt remapping is enabled on the IOMMU, hence introduce a logic similar to the one used in x2apic_bsp_setup in order to save and mask all IO-APIC pins, and then translate and restore them after interrupt remapping has been enabled. With this change the AMD specific logic to deal with enabled pins (amd_iommu_setup_ioapic_remapping) can be removed, thus unifying the handling of IO-APIC when enabling interrupt remapping regardless of the IOMMU vendor. Reported-by: Andrew Cooper Signed-off-by: Roger Pau Monn=C3=A9 Reviewed-by: Jan Beulich --- Cc: Juergen Gross --- xen/drivers/passthrough/amd/iommu_init.c | 11 ++- xen/drivers/passthrough/amd/iommu_intr.c | 90 +------------------ xen/drivers/passthrough/x86/iommu.c | 34 ++++++- xen/include/asm-x86/hvm/svm/amd-iommu-proto.h | 1 + 4 files changed, 40 insertions(+), 96 deletions(-) diff --git a/xen/drivers/passthrough/amd/iommu_init.c b/xen/drivers/passthr= ough/amd/iommu_init.c index 6f53c7ec08..3c244619b9 100644 --- a/xen/drivers/passthrough/amd/iommu_init.c +++ b/xen/drivers/passthrough/amd/iommu_init.c @@ -19,6 +19,7 @@ =20 #include #include +#include #include #include #include @@ -1435,12 +1436,6 @@ int __init amd_iommu_init(bool xt) if ( rc ) goto error_out; =20 - /* initialize io-apic interrupt remapping entries */ - if ( iommu_intremap ) - rc =3D amd_iommu_setup_ioapic_remapping(); - if ( rc ) - goto error_out; - /* Allocate and initialize device table(s). */ pci_init =3D !xt; rc =3D iterate_ivrs_mappings(amd_iommu_setup_device_table); @@ -1469,6 +1464,10 @@ int __init amd_iommu_init(bool xt) goto error_out; } =20 + if ( iommu_intremap ) + register_keyhandler('V', &amd_iommu_dump_intremap_tables, + "dump IOMMU intremap tables", 0); + return 0; =20 error_out: diff --git a/xen/drivers/passthrough/amd/iommu_intr.c b/xen/drivers/passthr= ough/amd/iommu_intr.c index fb71073c84..1eed60f265 100644 --- a/xen/drivers/passthrough/amd/iommu_intr.c +++ b/xen/drivers/passthrough/amd/iommu_intr.c @@ -21,7 +21,6 @@ #include #include #include -#include #include =20 union irte32 { @@ -79,8 +78,6 @@ unsigned long *shared_intremap_inuse; static DEFINE_SPINLOCK(shared_intremap_lock); unsigned int nr_ioapic_sbdf; =20 -static void dump_intremap_tables(unsigned char key); - #define intremap_page_order(irt) PFN_ORDER(virt_to_page(irt)) =20 unsigned int amd_iommu_intremap_table_order( @@ -354,91 +351,6 @@ static int update_intremap_entry_from_ioapic( return 0; } =20 -int __init amd_iommu_setup_ioapic_remapping(void) -{ - struct IO_APIC_route_entry rte; - unsigned long flags; - union irte_ptr entry; - int apic, pin; - u8 delivery_mode, dest, vector, dest_mode; - u16 seg, bdf, req_id; - struct amd_iommu *iommu; - spinlock_t *lock; - unsigned int offset; - - /* Read ioapic entries and update interrupt remapping table accordingl= y */ - for ( apic =3D 0; apic < nr_ioapics; apic++ ) - { - for ( pin =3D 0; pin < nr_ioapic_entries[apic]; pin++ ) - { - unsigned int idx; - - rte =3D __ioapic_read_entry(apic, pin, 1); - if ( rte.mask =3D=3D 1 ) - continue; - - /* get device id of ioapic devices */ - idx =3D ioapic_id_to_index(IO_APIC_ID(apic)); - if ( idx =3D=3D MAX_IO_APICS ) - return -EINVAL; - - bdf =3D ioapic_sbdf[idx].bdf; - seg =3D ioapic_sbdf[idx].seg; - iommu =3D find_iommu_for_device(seg, bdf); - if ( !iommu ) - { - AMD_IOMMU_DEBUG("Fail to find iommu for ioapic " - "device id =3D %04x:%04x\n", seg, bdf); - continue; - } - - req_id =3D get_intremap_requestor_id(iommu->seg, bdf); - lock =3D get_intremap_lock(iommu->seg, req_id); - - delivery_mode =3D rte.delivery_mode; - vector =3D rte.vector; - dest_mode =3D rte.dest_mode; - dest =3D rte.dest.logical.logical_dest; - - if ( iommu->ctrl.xt_en ) - { - /* - * In x2APIC mode we have no way of discovering the high 24 - * bits of the destination of an already enabled interrupt. - * We come here earlier than for xAPIC mode, so no interru= pts - * should have been set up before. - */ - AMD_IOMMU_DEBUG("Unmasked IO-APIC#%u entry %u in x2APIC mo= de\n", - IO_APIC_ID(apic), pin); - } - - spin_lock_irqsave(lock, flags); - offset =3D alloc_intremap_entry(iommu, req_id, 1); - BUG_ON(offset >=3D INTREMAP_MAX_ENTRIES); - entry =3D get_intremap_entry(iommu, req_id, offset); - update_intremap_entry(iommu, entry, vector, - delivery_mode, dest_mode, dest); - spin_unlock_irqrestore(lock, flags); - - set_rte_index(&rte, offset); - ioapic_sbdf[idx].pin_2_idx[pin] =3D offset; - __ioapic_write_entry(apic, pin, 1, rte); - - if ( iommu->enabled ) - { - spin_lock_irqsave(&iommu->lock, flags); - amd_iommu_flush_intremap(iommu, req_id); - spin_unlock_irqrestore(&iommu->lock, flags); - } - } - } - - register_keyhandler('V', &dump_intremap_tables, - "dump IOMMU intremap tables", 0); - - return 0; -} - void amd_iommu_ioapic_update_ire( unsigned int apic, unsigned int reg, unsigned int value) { @@ -982,7 +894,7 @@ static int dump_intremap_mapping(const struct amd_iommu= *iommu, return 0; } =20 -static void dump_intremap_tables(unsigned char key) +void amd_iommu_dump_intremap_tables(unsigned char key) { if ( !shared_intremap_table ) { diff --git a/xen/drivers/passthrough/x86/iommu.c b/xen/drivers/passthrough/= x86/iommu.c index 59905629e1..2cf528e760 100644 --- a/xen/drivers/passthrough/x86/iommu.c +++ b/xen/drivers/passthrough/x86/iommu.c @@ -21,6 +21,7 @@ #include =20 #include +#include #include =20 const struct iommu_init_ops *__initdata iommu_init_ops; @@ -28,6 +29,7 @@ struct iommu_ops __read_mostly iommu_ops; =20 int __init iommu_hardware_setup(void) { + struct IO_APIC_route_entry **ioapic_entries =3D NULL; int rc; =20 if ( !iommu_init_ops ) @@ -43,7 +45,37 @@ int __init iommu_hardware_setup(void) /* x2apic setup may have previously initialised the struct. */ ASSERT(iommu_ops.init =3D=3D iommu_init_ops->ops->init); =20 - return iommu_init_ops->setup(); + if ( !x2apic_enabled && iommu_intremap ) + { + /* + * If x2APIC is enabled interrupt remapping is already enabled, so + * there's no need to mess with the IO-APIC because the remapping + * entries are already correctly setup by x2apic_bsp_setup. + */ + ioapic_entries =3D alloc_ioapic_entries(); + if ( !ioapic_entries ) + return -ENOMEM; + rc =3D save_IO_APIC_setup(ioapic_entries); + if ( rc ) + { + free_ioapic_entries(ioapic_entries); + return rc; + } + + mask_8259A(); + mask_IO_APIC_setup(ioapic_entries); + } + + rc =3D iommu_init_ops->setup(); + + if ( ioapic_entries ) + { + restore_IO_APIC_setup(ioapic_entries, rc); + unmask_8259A(); + free_ioapic_entries(ioapic_entries); + } + + return rc; } =20 int iommu_enable_x2apic(void) diff --git a/xen/include/asm-x86/hvm/svm/amd-iommu-proto.h b/xen/include/as= m-x86/hvm/svm/amd-iommu-proto.h index 07d25a585d..8ed9482791 100644 --- a/xen/include/asm-x86/hvm/svm/amd-iommu-proto.h +++ b/xen/include/asm-x86/hvm/svm/amd-iommu-proto.h @@ -114,6 +114,7 @@ int amd_iommu_msi_msg_update_ire( void amd_iommu_read_msi_from_ire( struct msi_desc *msi_desc, struct msi_msg *msg); int amd_setup_hpet_msi(struct msi_desc *msi_desc); +void amd_iommu_dump_intremap_tables(unsigned char key); =20 extern struct ioapic_sbdf { u16 bdf, seg; --=20 2.23.0 _______________________________________________ Xen-devel mailing list Xen-devel@lists.xenproject.org https://lists.xenproject.org/mailman/listinfo/xen-devel