From nobody Sat Nov 23 10:19:35 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass(p=reject dis=none) header.from=citrix.com ARC-Seal: i=1; a=rsa-sha256; t=1729784965; cv=none; d=zohomail.com; s=zohoarc; b=Po/79nD7Ysx/+YoTC+FFxaFZx6NOD5JhoWmfbPbLCcncuNjcCnMil4IDN3CAr2hyRZAJixKjw4addwqrXaO8r143lJ6dg3axRPblzH3YWxefN6R5KQvpS2coA8LHH2HNT/KHNS0/ec5rGD+lkCt58oO28vsx/T1YFp9OtiKPNQ8= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1729784965; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=zlP7lw2ONsl7bRLNlv3YWePSjv/4K3vFG/1/1cHtx+s=; b=H1pCaLkL+KUFmhYitey6HusOcL45ggsEKmXFhHkz4/JAUiskWlFQFKPHudRis5nJMC4p8z1036ppV0i3sX5Bq2N7A39HufnH6SI8qEgQ+QllwuR79yHHk8BHh/+phhIwM+V5rXEg1q+uQ00pjde8kBQzOTQiAYJntNpOelZyW9Q= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass header.from= (p=reject dis=none) Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1729784965803969.3384112789222; Thu, 24 Oct 2024 08:49:25 -0700 (PDT) Received: from list by lists.xenproject.org with outflank-mailman.825460.1239687 (Exim 4.92) (envelope-from ) id 1t4055-0007IE-Gz; Thu, 24 Oct 2024 15:48:59 +0000 Received: by outflank-mailman (output) from mailman id 825460.1239687; Thu, 24 Oct 2024 15:48:59 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1t4055-0007I7-D1; Thu, 24 Oct 2024 15:48:59 +0000 Received: by outflank-mailman (input) for mailman id 825460; Thu, 24 Oct 2024 15:48:58 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1t4054-0007Hz-2m for xen-devel@lists.xenproject.org; Thu, 24 Oct 2024 15:48:58 +0000 Received: from mail-ej1-x634.google.com (mail-ej1-x634.google.com [2a00:1450:4864:20::634]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id 79cdbc6b-921f-11ef-99a3-01e77a169b0f; Thu, 24 Oct 2024 17:48:56 +0200 (CEST) Received: by mail-ej1-x634.google.com with SMTP id a640c23a62f3a-a9a2209bd7fso131438766b.2 for ; Thu, 24 Oct 2024 08:48:56 -0700 (PDT) Received: from localhost ([217.156.233.154]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-a9a91598f11sm632275766b.193.2024.10.24.08.48.54 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 24 Oct 2024 08:48:54 -0700 (PDT) X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 79cdbc6b-921f-11ef-99a3-01e77a169b0f DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=citrix.com; s=google; t=1729784935; x=1730389735; darn=lists.xenproject.org; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:from:to:cc:subject:date:message-id:reply-to; bh=zlP7lw2ONsl7bRLNlv3YWePSjv/4K3vFG/1/1cHtx+s=; b=fZTdUC9CQ1WVklXGb4NqGvQLVcKl3dU0vdH0pExS2UjyCL0bcD6VsnGLVTjyKhRuHU ujbsVBwng4/A6urNzyHcWeX5PetuM3dWoyixk6c0CNE/dOP03sfinZKaRoqq2lllybkU amy8SXqxL5yGGiN5emzQ+c+CLLLoqZm5FR388= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1729784935; x=1730389735; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=zlP7lw2ONsl7bRLNlv3YWePSjv/4K3vFG/1/1cHtx+s=; b=BOBSnktFwYZsgTQnrBPFCAThhLpl4/tCtaZk7aRTTqdlG4fr1azBB1zUOUl6EKc8CT oObeFiYAZZsDOo5MgP61S6iHOERGp7vWjBEjyPTiSSmoW3pHssRThukY8QKPW1PQ1NWf jJ9pdW0INsbCmRzkO01ySjVuB0zo+KyuPdjwGohH45iRecvOJli9dYTPgwBVeK/ctZa6 gIbrz3Ch/uUyJnSHQwZnaPs8XdlPx/NU5sgcuPAwDkekkRf3Y6cTrPX9UtVKXmJPYPUh xZ6z0LpyTPEZJH5euKw2Ny+D5/RINRmYuBiMuKBjwIaCcYcuesEy5gJ3X/0uRvtuSwgY f/KQ== X-Gm-Message-State: AOJu0YwbL2/a7eIaL7X4BK3Pqx67wuJS5h9T+iGGlbE4sCUC6nT5/J+K NVYUnZRlyRV0N5wiZDGPwBDPKsZ3FqHYosJSgb4ZfoUeIrOKCFoc253v2sSAQzUyGH1RCwaduWy J X-Google-Smtp-Source: AGHT+IHID7dLKQqhXr2eMpBAMzkmGLcY1ZYpk9S1NupJKVozmgu04MsJiLxgAuJ4E9PhYYc+8zdH0Q== X-Received: by 2002:a17:906:f592:b0:a9a:a891:b43e with SMTP id a640c23a62f3a-a9ad28158e1mr243753466b.50.1729784934884; Thu, 24 Oct 2024 08:48:54 -0700 (PDT) From: Roger Pau Monne To: xen-devel@lists.xenproject.org Cc: Roger Pau Monne , Jan Beulich , Andrew Cooper , Willi Junga , David Woodhouse Subject: [PATCH v2] x86/io-apic: fix directed EOI when using AMD-Vi interrupt remapping Date: Thu, 24 Oct 2024 16:48:44 +0100 Message-ID: <20241024154844.8652-1-roger.pau@citrix.com> X-Mailer: git-send-email 2.46.0 MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @citrix.com) X-ZM-MESSAGEID: 1729784966555116600 When using AMD-VI interrupt remapping the vector field in the IO-APIC RTE is repurposed to contain part of the offset into the remapping table. Previou= s to 2ca9fbd739b8 Xen had logic so that the offset into the interrupt remapping table would match the vector. Such logic was mandatory for end of interrup= t to work, since the vector field (even when not containing a vector) is used by= the IO-APIC to find for which pin the EOI must be performed. Introduce a table to store the EOI handlers when using interrupt remapping,= so that the IO-APIC driver can translate pins into EOI handlers without having= to read the IO-APIC RTE entry. Note that to simplify the logic such table is = used unconditionally when interrupt remapping is enabled, even if strictly it wo= uld only be required for AMD-Vi. Reported-by: Willi Junga Suggested-by: David Woodhouse Fixes: 2ca9fbd739b8 ('AMD IOMMU: allocate IRTE entries instead of using a s= tatic mapping') Signed-off-by: Roger Pau Monn=C3=A9 --- Changes since v1: - s/apic_pin_eoi/io_apic_pin_eoi/. - Expand comment about io_apic_pin_eoi usage and layout. - Use uint8_t instead of unsigned int as array type. - Do not use a sentinel value. --- xen/arch/x86/io_apic.c | 41 +++++++++++++++++++++++++++++++++++++++++ 1 file changed, 41 insertions(+) diff --git a/xen/arch/x86/io_apic.c b/xen/arch/x86/io_apic.c index e40d2f7dbd75..e3cdfab6359a 100644 --- a/xen/arch/x86/io_apic.c +++ b/xen/arch/x86/io_apic.c @@ -71,6 +71,24 @@ static int apic_pin_2_gsi_irq(int apic, int pin); =20 static vmask_t *__read_mostly vector_map[MAX_IO_APICS]; =20 +/* + * Store the EOI handle when using interrupt remapping. + * + * If using AMD-Vi interrupt remapping the IO-APIC redirection entry remap= ped + * format repurposes the vector field to store the offset into the Interru= pt + * Remap table. This causes directed EOI to longer work, as the CPU vecto= r no + * longer matches the contents of the RTE vector field. Add a translation + * table so that directed EOI uses the value in the RTE vector field when + * interrupt remapping is enabled. + * + * Note Intel VT-d Xen code still stores the CPU vector in the RTE vector = field + * when using the remapped format, but use the translation table uniformly= in + * order to avoid extra logic to differentiate between VT-d and AMD-Vi. + * + * The matrix is accessed as [#io-apic][#pin]. + */ +static uint8_t **io_apic_pin_eoi; + static void share_vector_maps(unsigned int src, unsigned int dst) { unsigned int pin; @@ -273,6 +291,13 @@ void __ioapic_write_entry( { __io_apic_write(apic, 0x11 + 2 * pin, eu.w2); __io_apic_write(apic, 0x10 + 2 * pin, eu.w1); + /* + * Called in clear_IO_APIC_pin() before io_apic_pin_eoi is allocat= ed. + * Entry will be updated once the array is allocated and there's a + * write against the pin. + */ + if ( io_apic_pin_eoi ) + io_apic_pin_eoi[apic][pin] =3D e.vector; } else iommu_update_ire_from_apic(apic, pin, e.raw); @@ -298,6 +323,9 @@ static void __io_apic_eoi(unsigned int apic, unsigned i= nt vector, unsigned int p /* Prefer the use of the EOI register if available */ if ( ioapic_has_eoi_reg(apic) ) { + if ( io_apic_pin_eoi ) + vector =3D io_apic_pin_eoi[apic][pin]; + /* If vector is unknown, read it from the IO-APIC */ if ( vector =3D=3D IRQ_VECTOR_UNASSIGNED ) vector =3D __ioapic_read_entry(apic, pin, true).vector; @@ -1022,7 +1050,20 @@ static void __init setup_IO_APIC_irqs(void) =20 apic_printk(APIC_VERBOSE, KERN_DEBUG "init IO_APIC IRQs\n"); =20 + if ( iommu_intremap ) + { + io_apic_pin_eoi =3D xzalloc_array(typeof(*io_apic_pin_eoi), nr_ioa= pics); + BUG_ON(!io_apic_pin_eoi); + } + for (apic =3D 0; apic < nr_ioapics; apic++) { + if ( iommu_intremap ) + { + io_apic_pin_eoi[apic] =3D xzalloc_array(typeof(**io_apic_pin_e= oi), + nr_ioapic_entries[apic]); + BUG_ON(!io_apic_pin_eoi[apic]); + } + for (pin =3D 0; pin < nr_ioapic_entries[apic]; pin++) { /* * add it to the IO-APIC irq-routing table: --=20 2.46.0