From nobody Sun Nov 24 03:13:01 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass(p=reject dis=none) header.from=citrix.com ARC-Seal: i=1; a=rsa-sha256; t=1729238933; cv=none; d=zohomail.com; s=zohoarc; b=WnnRu02KI8D5IiDI6tEPGk6Q2qPIQ02zkH4vQk+FKm9oHwpBe97QY/zz/ZjGdemd/j/PS+kjLUPA/0gsNnG9vAMJsTp/Wd9btfCX8yR96JNJTUEknQOYL8DnDiwlXyQYuqNQ9ilBMkOBiMdyR5rWK48T746FRINusZEabfmiLYY= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1729238933; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=SWO0FmBLFHJZH9jt972i/jtzdUO5iTIlLeJ7XupzTFg=; b=SicAOZaWJ36IRTf9Rt+Ki7oNx3CQV52XBOdaykWRRL6CJDjITvuhmUeO6uiV65LKWkPVvXY1Y6ui33URgcvrznNZf+mWas6tjKalSj4MCEaJKaaEAvHHFZoEV7KAP1/wvtC1Mdhd3fBVIgDSWbuMTkZgM4KtMXCIjBL2tiaPh44= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass header.from= (p=reject dis=none) Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1729238933773604.7811789046755; Fri, 18 Oct 2024 01:08:53 -0700 (PDT) Received: from list by lists.xenproject.org with outflank-mailman.821658.1235570 (Exim 4.92) (envelope-from ) id 1t1i25-00011r-Ga; Fri, 18 Oct 2024 08:08:25 +0000 Received: by outflank-mailman (output) from mailman id 821658.1235570; Fri, 18 Oct 2024 08:08:25 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1t1i25-00011k-E4; Fri, 18 Oct 2024 08:08:25 +0000 Received: by outflank-mailman (input) for mailman id 821658; Fri, 18 Oct 2024 08:08:24 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1t1i24-00011d-GI for xen-devel@lists.xenproject.org; Fri, 18 Oct 2024 08:08:24 +0000 Received: from mail-ed1-x52e.google.com (mail-ed1-x52e.google.com [2a00:1450:4864:20::52e]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id 244a80c8-8d28-11ef-99a3-01e77a169b0f; Fri, 18 Oct 2024 10:08:22 +0200 (CEST) Received: by mail-ed1-x52e.google.com with SMTP id 4fb4d7f45d1cf-5c9625cfe4dso2210806a12.0 for ; Fri, 18 Oct 2024 01:08:22 -0700 (PDT) Received: from localhost ([213.195.115.182]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-a9a68bf7669sm61338066b.171.2024.10.18.01.08.20 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 18 Oct 2024 01:08:20 -0700 (PDT) X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 244a80c8-8d28-11ef-99a3-01e77a169b0f DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=citrix.com; s=google; t=1729238901; x=1729843701; darn=lists.xenproject.org; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:from:to:cc:subject:date:message-id:reply-to; bh=SWO0FmBLFHJZH9jt972i/jtzdUO5iTIlLeJ7XupzTFg=; b=sYwylxeXq1S/z1Xlx0uXQKN+LTUcdtm+jR7eeITAmGEC8Fe6oimTKXJ0IHLOsdJRU6 5rxEuKmNTmJLgt2sXOMRCjnRoVnITOj9bpbu0BcRDGb+0s4X0i1HyYwDQUORqqBb6Jta WiBE58DJUv7WChzvCZpZeM38EnWM6s9eztNvE= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1729238901; x=1729843701; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=SWO0FmBLFHJZH9jt972i/jtzdUO5iTIlLeJ7XupzTFg=; b=UE5+6OZ3BMy0osQ3Z6ud37SYB4GZQNt4kj3eOO/n7Xl3emrIXrzGt9Z1URf+FfxXhZ mpnkEOL7Av7vjBDW8tpW57F9axvBfDM7ygcIT6wS11JvZY8F34Bv2cx+CaV0xo76eR4N fobpv82C49p0iBv1s3L1w0nt9K9hHvjgW4CegJb6SDWbkmmW+TEZjWPr82gjUBZabz4w BjH8b42AKiNe3Cb75tHcvXfeCzG5NqtA72Dr35WIz94jNohSUg2oUpQKzNiOip6FWv+l UqrZiCd5mgcm64bpqF49qPJL/8rSrc4LNeBHdSYD4F2Jq4e+9AuOwKGbx5pdYrUZk2+m Xt4A== X-Gm-Message-State: AOJu0Yy0Vg0GwDzTDbw5z6lyS367NPP2Wains01cugIymdDs9V7yutTs Z97IinLfkN6S/q9utzraiKI1IVpSILJWnc6XASkIeNpjqLdCoceO5Exj6oMNYiW0M4tFTUyUIRm B X-Google-Smtp-Source: AGHT+IF+jgDYwQM9HweZrGlbsiGa6znuoD6fPzkUgdsAhPubJQJ3h9X8ir+y0rKvW08wm/Qc3I6TIQ== X-Received: by 2002:a17:907:7f87:b0:a9a:41c6:1d34 with SMTP id a640c23a62f3a-a9a69a66b40mr125532466b.21.1729238900579; Fri, 18 Oct 2024 01:08:20 -0700 (PDT) From: Roger Pau Monne To: xen-devel@lists.xenproject.org Cc: Roger Pau Monne , Jan Beulich , Andrew Cooper , Willi Junga , David Woodhouse Subject: [PATCH] x86/io-apic: fix directed EOI when using AMd-Vi interrupt remapping Date: Fri, 18 Oct 2024 10:08:13 +0200 Message-ID: <20241018080813.45759-1-roger.pau@citrix.com> X-Mailer: git-send-email 2.46.0 MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @citrix.com) X-ZM-MESSAGEID: 1729238934570116600 When using AMD-VI interrupt remapping the vector field in the IO-APIC RTE is repurposed to contain part of the offset into the remapping table. Previou= s to 2ca9fbd739b8 Xen had logic so that the offset into the interrupt remapping table would match the vector. Such logic was mandatory for end of interrup= t to work, since the vector field (even when not containing a vector) is used by= the IO-APIC to find for which pin the EOI must be performed. Introduce a table to store the EOI handlers when using interrupt remapping,= so that the IO-APIC driver can translate pins into EOI handlers without having= to read the IO-APIC RTE entry. Note that to simplify the logic such table is = used unconditionally when interrupt remapping is enabled, even if strictly it wo= uld only be required for AMD-Vi. Reported-by: Willi Junga Suggested-by: David Woodhouse Fixes: 2ca9fbd739b8 ('AMD IOMMU: allocate IRTE entries instead of using a s= tatic mapping') Signed-off-by: Roger Pau Monn=C3=A9 Tested-by: Marek Marczykowski-G=C3=B3recki --- xen/arch/x86/io_apic.c | 47 ++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 47 insertions(+) diff --git a/xen/arch/x86/io_apic.c b/xen/arch/x86/io_apic.c index e40d2f7dbd75..8856eb29d275 100644 --- a/xen/arch/x86/io_apic.c +++ b/xen/arch/x86/io_apic.c @@ -71,6 +71,22 @@ static int apic_pin_2_gsi_irq(int apic, int pin); =20 static vmask_t *__read_mostly vector_map[MAX_IO_APICS]; =20 +/* + * Store the EOI handle when using interrupt remapping. + * + * If using AMD-Vi interrupt remapping the IO-APIC redirection entry remap= ped + * format repurposes the vector field to store the offset into the Interru= pt + * Remap table. This causes directed EOI to longer work, as the CPU vecto= r no + * longer matches the contents of the RTE vector field. Add a translation + * table so that directed EOI uses the value in the RTE vector field when + * interrupt remapping is enabled. + * + * Note Intel VT-d Xen code still stores the CPU vector in the RTE vector = field + * when using the remapped format, but use the translation table uniformly= in + * order to avoid extra logic to differentiate between VT-d and AMD-Vi. + */ +static unsigned int **apic_pin_eoi; + static void share_vector_maps(unsigned int src, unsigned int dst) { unsigned int pin; @@ -273,6 +289,13 @@ void __ioapic_write_entry( { __io_apic_write(apic, 0x11 + 2 * pin, eu.w2); __io_apic_write(apic, 0x10 + 2 * pin, eu.w1); + /* + * Might be called before apic_pin_eoi is allocated. Entry will be + * updated once the array is allocated and there's an EOI or write + * against the pin. + */ + if ( apic_pin_eoi ) + apic_pin_eoi[apic][pin] =3D e.vector; } else iommu_update_ire_from_apic(apic, pin, e.raw); @@ -298,9 +321,17 @@ static void __io_apic_eoi(unsigned int apic, unsigned = int vector, unsigned int p /* Prefer the use of the EOI register if available */ if ( ioapic_has_eoi_reg(apic) ) { + if ( apic_pin_eoi ) + vector =3D apic_pin_eoi[apic][pin]; + /* If vector is unknown, read it from the IO-APIC */ if ( vector =3D=3D IRQ_VECTOR_UNASSIGNED ) + { vector =3D __ioapic_read_entry(apic, pin, true).vector; + if ( apic_pin_eoi ) + /* Update cached value so further EOI don't need to fetch = it. */ + apic_pin_eoi[apic][pin] =3D vector; + } =20 *(IO_APIC_BASE(apic)+16) =3D vector; } @@ -1022,7 +1053,23 @@ static void __init setup_IO_APIC_irqs(void) =20 apic_printk(APIC_VERBOSE, KERN_DEBUG "init IO_APIC IRQs\n"); =20 + if ( iommu_intremap ) + { + apic_pin_eoi =3D xzalloc_array(typeof(*apic_pin_eoi), nr_ioapics); + BUG_ON(!apic_pin_eoi); + } + for (apic =3D 0; apic < nr_ioapics; apic++) { + if ( iommu_intremap ) + { + apic_pin_eoi[apic] =3D xmalloc_array(typeof(**apic_pin_eoi), + nr_ioapic_entries[apic]); + BUG_ON(!apic_pin_eoi[apic]); + + for ( pin =3D 0; pin < nr_ioapic_entries[apic]; pin++ ) + apic_pin_eoi[apic][pin] =3D IRQ_VECTOR_UNASSIGNED; + } + for (pin =3D 0; pin < nr_ioapic_entries[apic]; pin++) { /* * add it to the IO-APIC irq-routing table: --=20 2.46.0