From nobody Thu May 16 06:32:47 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=none (zohomail.com: 192.237.175.120 is neither permitted nor denied by domain of lists.xenproject.org) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=fail(p=none dis=none) header.from=citrix.com ARC-Seal: i=1; a=rsa-sha256; t=1575476482; cv=none; d=zohomail.com; s=zohoarc; b=OJYN4pCDICrIechQ5KU/2rVjzKvDgpUJH/o+ONo14bVLjvCefjywW5QGXDnHMicHWy4rTP3IN63ks+H+giMNksteEbzUU4VVsjKCUWg2rTw7+xSF3Y/LpK2W9+MGg0ZJSGQ2runpAhvYdj4PYIHyI+zHWxxPed1cnr9iPHWjybg= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1575476482; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=O7tmIZzHZ7agyZOxhz15sxJkijA2VH+1M0lSfOboecI=; b=C8gm8PXOA11yWeHJ4lidMyi5htlfkE5dv/FeabNALL14mb0s3lwtDhcIBzCkTbiyxA8+mFTUdajIndatDjahGoWBL71GFdqYEjHu0Q4mj/X6p8jrei3PSbmkKvxRy/mUsK3ucfUm5ghavj0dmaNvAuzBB5wybY8gwM0h1ltzKZw= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=none (zohomail.com: 192.237.175.120 is neither permitted nor denied by domain of lists.xenproject.org) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1575476482286749.4564059601021; Wed, 4 Dec 2019 08:21:22 -0800 (PST) Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1icXOQ-0006qY-TV; Wed, 04 Dec 2019 16:20:46 +0000 Received: from us1-rack-iad1.inumbo.com ([172.99.69.81]) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1icXOP-0006qG-RM for xen-devel@lists.xenproject.org; Wed, 04 Dec 2019 16:20:45 +0000 Received: from esa6.hc3370-68.iphmx.com (unknown [216.71.155.175]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS id 05dce66e-16b2-11ea-a0d2-bc764e2007e4; Wed, 04 Dec 2019 16:20:45 +0000 (UTC) X-Inumbo-ID: 05dce66e-16b2-11ea-a0d2-bc764e2007e4 DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=citrix.com; s=securemail; t=1575476445; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=a3VbCfnmF7tpj2vAyuyl6JFYJVMSgRm44pU8P/N40ZM=; b=SPPWUkKi48vySeIiQieUf5AYsjgiE8hpl2BVi/EntYujdZgLvxW/+WmL 7f5lXX6qgjC+DF0PCrMaFnuapvUOFmUuLikaRCaAem9WGsSBcutPfvBZ1 jVD/RNc8Oa5qrbcuhVXnfwmQv7dmSYHmDLGVVPZ+KNVkDE3ouL/yqb6lH A=; Authentication-Results: esa6.hc3370-68.iphmx.com; dkim=none (message not signed) header.i=none; spf=None smtp.pra=roger.pau@citrix.com; spf=Pass smtp.mailfrom=roger.pau@citrix.com; spf=None smtp.helo=postmaster@mail.citrix.com Received-SPF: none (zohomail.com: 192.237.175.120 is neither permitted nor denied by domain of lists.xenproject.org) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Received-SPF: None (esa6.hc3370-68.iphmx.com: no sender authenticity information available from domain of roger.pau@citrix.com) identity=pra; client-ip=162.221.158.21; receiver=esa6.hc3370-68.iphmx.com; envelope-from="roger.pau@citrix.com"; x-sender="roger.pau@citrix.com"; x-conformance=sidf_compatible Received-SPF: Pass (esa6.hc3370-68.iphmx.com: domain of roger.pau@citrix.com designates 162.221.158.21 as permitted sender) identity=mailfrom; client-ip=162.221.158.21; receiver=esa6.hc3370-68.iphmx.com; envelope-from="roger.pau@citrix.com"; x-sender="roger.pau@citrix.com"; x-conformance=sidf_compatible; x-record-type="v=spf1"; x-record-text="v=spf1 ip4:209.167.231.154 ip4:178.63.86.133 ip4:195.66.111.40/30 ip4:85.115.9.32/28 ip4:199.102.83.4 ip4:192.28.146.160 ip4:192.28.146.107 ip4:216.52.6.88 ip4:216.52.6.188 ip4:162.221.158.21 ip4:162.221.156.83 ip4:168.245.78.127 ~all" Received-SPF: None (esa6.hc3370-68.iphmx.com: no sender authenticity information available from domain of postmaster@mail.citrix.com) identity=helo; client-ip=162.221.158.21; receiver=esa6.hc3370-68.iphmx.com; envelope-from="roger.pau@citrix.com"; x-sender="postmaster@mail.citrix.com"; x-conformance=sidf_compatible IronPort-SDR: DiTWPkgppH4Ncg4m7tBrUmjWkmgJQ8GZG5ChhVNrQRR00qkdsmLWmhlh4D0jRysgVsBI8IYbf0 iQXsLP0DMnxFjEUaWE70D6Baq7DRp+W8wvu45kTGCseiOoQxAwBfE+C2uWWmULEqM0lTRp6zb1 yRPbySQfqIv5zEpW7YfOkxEMP4S0SRIUoBrz3acpnqU9TW7ADN45ukJf04GTU4OWYvy4uUhnt4 hcDPw6ba6Y7QxTcqnr57I6v7baHEOgRwL4A8vD+J0RxslVMoVNPtTGlDvqAbE8vSuYZptuFRAU dVE= X-SBRS: 2.7 X-MesageID: 9607271 X-Ironport-Server: esa6.hc3370-68.iphmx.com X-Remote-IP: 162.221.158.21 X-Policy: $RELAYED X-IronPort-AV: E=Sophos;i="5.69,277,1571716800"; d="scan'208";a="9607271" From: Roger Pau Monne To: Date: Wed, 4 Dec 2019 17:20:22 +0100 Message-ID: <20191204162025.37510-2-roger.pau@citrix.com> X-Mailer: git-send-email 2.24.0 In-Reply-To: <20191204162025.37510-1-roger.pau@citrix.com> References: <20191204162025.37510-1-roger.pau@citrix.com> MIME-Version: 1.0 Subject: [Xen-devel] [PATCH v3 1/4] x86/ioapic: only use dest32 with x2apic and interrupt remapping enabled X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Andrew Cooper , Wei Liu , Jan Beulich , Roger Pau Monne Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) The IO-APIC code assumes that x2apic being enabled also implies interrupt remapping being enabled, and hence will use the 32bit destination field in the IO-APIC entry. This is safe now, but there's no reason to not enable x2APIC even without interrupt remapping, and hence the IO-APIC code needs to use the 32 bit destination field only when both interrupt remapping and x2APIC are enabled. Signed-off-by: Roger Pau Monn=C3=A9 Reviewed-by: Jan Beulich --- Changes since v1: - Fix set_ioapic_affinity_irq. --- xen/arch/x86/io_apic.c | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/xen/arch/x86/io_apic.c b/xen/arch/x86/io_apic.c index 97cb2d154a..6238df494b 100644 --- a/xen/arch/x86/io_apic.c +++ b/xen/arch/x86/io_apic.c @@ -562,7 +562,7 @@ set_ioapic_affinity_irq(struct irq_desc *desc, const cp= umask_t *mask) =20 dest =3D set_desc_affinity(desc, mask); if (dest !=3D BAD_APICID) { - if ( !x2apic_enabled ) + if ( !iommu_intremap || !x2apic_enabled ) dest =3D SET_APIC_LOGICAL_ID(dest); entry =3D irq_2_pin + irq; for (;;) { @@ -964,7 +964,7 @@ static hw_irq_controller ioapic_edge_type; #define IOAPIC_LEVEL 1 =20 #define SET_DEST(ent, mode, val) do { \ - if (x2apic_enabled) \ + if (x2apic_enabled && iommu_intremap) \ (ent).dest.dest32 =3D (val); \ else \ (ent).dest.mode.mode##_dest =3D (val); \ @@ -1194,14 +1194,14 @@ static void /*__init*/ __print_IO_APIC(bool boot) printk(KERN_DEBUG ".... IRQ redirection table:\n"); =20 printk(KERN_DEBUG " NR %s Msk Trg IRR Pol Stat DstM DelM Vec\n", - x2apic_enabled ? " DestID" : "Dst"); + (x2apic_enabled && iommu_intremap) ? " DestID" : "Dst"); =20 for (i =3D 0; i <=3D reg_01.bits.entries; i++) { struct IO_APIC_route_entry entry; =20 entry =3D ioapic_read_entry(apic, i, 0); =20 - if ( x2apic_enabled ) + if ( x2apic_enabled && iommu_intremap ) printk(KERN_DEBUG " %02x %08x", i, entry.dest.dest32); else printk(KERN_DEBUG " %02x %02x ", i, @@ -2504,9 +2504,9 @@ void dump_ioapic_irq_info(void) rte.dest_mode ? 'L' : 'P', rte.delivery_status, rte.polarity, rte.irr, rte.trigger ? 'L' : 'E', rte.mask, - x2apic_enabled ? 8 : 2, - x2apic_enabled ? rte.dest.dest32 - : rte.dest.logical.logical_dest); + (x2apic_enabled && iommu_intremap) ? 8 : 2, + (x2apic_enabled && iommu_intremap) ? + rte.dest.dest32 : rte.dest.logical.logical_dest); =20 if ( entry->next =3D=3D 0 ) break; --=20 2.24.0 _______________________________________________ Xen-devel mailing list Xen-devel@lists.xenproject.org https://lists.xenproject.org/mailman/listinfo/xen-devel From nobody Thu May 16 06:32:47 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=none (zohomail.com: 192.237.175.120 is neither permitted nor denied by domain of lists.xenproject.org) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=fail(p=none dis=none) header.from=citrix.com ARC-Seal: i=1; a=rsa-sha256; t=1575476480; cv=none; d=zohomail.com; s=zohoarc; b=nhpXEsc+j9TU5h8fWc4V2L5yz4kXCVGj4n2XZ2tYikyIpZbcvJyyiAJ/7ICiCPiyryjHP2IT1BdjZETuuS9U67QKsEbnUlUva8UvVJ2AQVOkKuaJjzumcC1BSkmi+bPCh+eEj5fLkVaL49DRmzYDLwT+/pR/Oj/ex3E5RsAqMrI= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1575476480; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=QnCTWSzVvl6YVZxFP2g4u3yf6UgGTTMRjhK3OVzefPc=; b=IzNZyZoMpM2U03LBMhrE3dZDtykMc3HYHbuDxHLy59tJNMqEzgzaAdoEb+EwUYVNXzTe0AWHxxf05pHbfewCfqneN6FTekX9U/KIFZ7JpMLDs+33OEpOdqUFX4jJX6FyczsGXfw9Pwq75go0HKPtgHkh7HfED/5OY4C3saCTKe0= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=none (zohomail.com: 192.237.175.120 is neither permitted nor denied by domain of lists.xenproject.org) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1575476480925733.4332692401271; Wed, 4 Dec 2019 08:21:20 -0800 (PST) Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1icXOW-0006sK-7K; Wed, 04 Dec 2019 16:20:52 +0000 Received: from us1-rack-iad1.inumbo.com ([172.99.69.81]) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1icXOU-0006rs-Qj for xen-devel@lists.xenproject.org; Wed, 04 Dec 2019 16:20:50 +0000 Received: from esa1.hc3370-68.iphmx.com (unknown [216.71.145.142]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS id 080175d6-16b2-11ea-99dd-bc764e2007e4; Wed, 04 Dec 2019 16:20:48 +0000 (UTC) X-Inumbo-ID: 080175d6-16b2-11ea-99dd-bc764e2007e4 DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=citrix.com; s=securemail; t=1575476449; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=8K929pLKGjsKHWuZReFQyGG9sX73qV7AkheF8i5ZCQo=; b=BGaY7g5QnRSOjNcMnFz0tAKXKt6nQMhfH1W1W2h6Eiaa9mm46dgLtZ86 InFFUB1TKs0ycnxHr4GoWXubGiQDbhyQfbSqVCvUoowLwSh3ATCwTXjKC WR6vlJk+DFRF+7l5BhsCIwLMFAAlKnnxV0BFuJvs/0pTVG7TX7yo4tzJR 8=; Authentication-Results: esa1.hc3370-68.iphmx.com; dkim=none (message not signed) header.i=none; spf=None smtp.pra=roger.pau@citrix.com; spf=Pass smtp.mailfrom=roger.pau@citrix.com; spf=None smtp.helo=postmaster@mail.citrix.com Received-SPF: none (zohomail.com: 192.237.175.120 is neither permitted nor denied by domain of lists.xenproject.org) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Received-SPF: None (esa1.hc3370-68.iphmx.com: no sender authenticity information available from domain of roger.pau@citrix.com) identity=pra; client-ip=162.221.158.21; receiver=esa1.hc3370-68.iphmx.com; envelope-from="roger.pau@citrix.com"; x-sender="roger.pau@citrix.com"; x-conformance=sidf_compatible Received-SPF: Pass (esa1.hc3370-68.iphmx.com: domain of roger.pau@citrix.com designates 162.221.158.21 as permitted sender) identity=mailfrom; client-ip=162.221.158.21; receiver=esa1.hc3370-68.iphmx.com; envelope-from="roger.pau@citrix.com"; x-sender="roger.pau@citrix.com"; x-conformance=sidf_compatible; x-record-type="v=spf1"; x-record-text="v=spf1 ip4:209.167.231.154 ip4:178.63.86.133 ip4:195.66.111.40/30 ip4:85.115.9.32/28 ip4:199.102.83.4 ip4:192.28.146.160 ip4:192.28.146.107 ip4:216.52.6.88 ip4:216.52.6.188 ip4:162.221.158.21 ip4:162.221.156.83 ip4:168.245.78.127 ~all" Received-SPF: None (esa1.hc3370-68.iphmx.com: no sender authenticity information available from domain of postmaster@mail.citrix.com) identity=helo; client-ip=162.221.158.21; receiver=esa1.hc3370-68.iphmx.com; envelope-from="roger.pau@citrix.com"; x-sender="postmaster@mail.citrix.com"; x-conformance=sidf_compatible IronPort-SDR: 7UcfKnWASAzwB9/pYSSpoZO1tznWUGYz+WeZuoViguPoOqnSbC36nZzsOLtI7TOS21vC70lc4A dG8Enu4LGmkVUF/qMl/EPA44fYz6lkbHG8yIlXpN87E2IsYnjbBswjaUlrEe/j/FO74YbOsXkY FNBZluS0O8QvVgKTBbFqnBrzXnQZ/6/DHudja3KAFTh4BABpUVDBXgmSgxyutxEayWqJBLnQgu sSzVJO40m9Esi0aQWiAZVQDCULB8BDmWIX+Ih0JYpx/WXfwOo8HJzWhzYMwJ5vwnfVgHpqsn48 UJI= X-SBRS: 2.7 X-MesageID: 9323386 X-Ironport-Server: esa1.hc3370-68.iphmx.com X-Remote-IP: 162.221.158.21 X-Policy: $RELAYED X-IronPort-AV: E=Sophos;i="5.69,277,1571716800"; d="scan'208";a="9323386" From: Roger Pau Monne To: Date: Wed, 4 Dec 2019 17:20:23 +0100 Message-ID: <20191204162025.37510-3-roger.pau@citrix.com> X-Mailer: git-send-email 2.24.0 In-Reply-To: <20191204162025.37510-1-roger.pau@citrix.com> References: <20191204162025.37510-1-roger.pau@citrix.com> MIME-Version: 1.0 Subject: [Xen-devel] [PATCH v3 2/4] x86/apic: force phys mode if interrupt remapping is disabled X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Stefano Stabellini , Julien Grall , Wei Liu , Konrad Rzeszutek Wilk , George Dunlap , Andrew Cooper , Ian Jackson , Jan Beulich , Roger Pau Monne Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) Cluster mode can only be used with interrupt remapping support, since the top 16bits of the APIC ID are filled with the cluster ID, and hence on systems where the physical ID is still smaller than 255 the cluster ID is not. Force x2APIC to use physical mode if there's no interrupt remapping support. Note that this requires a further patch in order to enable x2APIC without interrupt remapping support. Signed-off-by: Roger Pau Monn=C3=A9 Reviewed-by: Jan Beulich --- Changes since v2: - Update command line doc. - Fix logic to set x2apic_phys if user has specified a value. - Force phys mode if no interrupt remapping support. Changes since v1: - New in this version. --- docs/misc/xen-command-line.pandoc | 3 ++- xen/arch/x86/genapic/x2apic.c | 18 +++++++++++++++++- 2 files changed, 19 insertions(+), 2 deletions(-) diff --git a/docs/misc/xen-command-line.pandoc b/docs/misc/xen-command-line= .pandoc index 891d2d439f..d9495ef6b9 100644 --- a/docs/misc/xen-command-line.pandoc +++ b/docs/misc/xen-command-line.pandoc @@ -2303,7 +2303,8 @@ Permit use of x2apic setup for SMP environments. ### x2apic_phys (x86) > `=3D ` =20 -> Default: `true` if **FADT** mandates physical mode, `false` otherwise. +> Default: `true` if **FADT** mandates physical mode or if interrupt remap= ping +> is not available, `false` otherwise. =20 In the case that x2apic is in use, this option switches between physical a= nd clustered mode. The default, given no hint from the **FADT**, is cluster diff --git a/xen/arch/x86/genapic/x2apic.c b/xen/arch/x86/genapic/x2apic.c index d5a17f10d5..79b6c07329 100644 --- a/xen/arch/x86/genapic/x2apic.c +++ b/xen/arch/x86/genapic/x2apic.c @@ -226,7 +226,23 @@ boolean_param("x2apic_phys", x2apic_phys); const struct genapic *__init apic_x2apic_probe(void) { if ( x2apic_phys < 0 ) - x2apic_phys =3D !!(acpi_gbl_FADT.flags & ACPI_FADT_APIC_PHYSICAL); + { + if ( !iommu_intremap ) + /* + * Force physical mode if there's no interrupt remapping suppo= rt: + * the ID in clustered mode requires a 32 bit destination fiel= d due + * to the usage of the high 16 bits to store the cluster ID. + */ + x2apic_phys =3D true; + else + x2apic_phys =3D !!(acpi_gbl_FADT.flags & ACPI_FADT_APIC_PHYSIC= AL); + } + else if ( !x2apic_phys && !iommu_intremap ) + { + printk("WARNING: x2APIC cluster mode is not supported without inte= rrupt remapping\n" + "x2APIC: forcing phys mode\n"); + x2apic_phys =3D true; + } =20 if ( x2apic_phys ) return &apic_x2apic_phys; --=20 2.24.0 _______________________________________________ Xen-devel mailing list Xen-devel@lists.xenproject.org https://lists.xenproject.org/mailman/listinfo/xen-devel From nobody Thu May 16 06:32:47 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=none (zohomail.com: 192.237.175.120 is neither permitted nor denied by domain of lists.xenproject.org) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=fail(p=none dis=none) header.from=citrix.com ARC-Seal: i=1; a=rsa-sha256; t=1575476490; cv=none; d=zohomail.com; s=zohoarc; b=fEZw8Amwa2LrUp5+1bkExD1WW4CwkA93vM1swtv7BkzRra2ffSMr+to/9ETVX1T72J8WlkPtjsDl9glOrhY4tA87jOtGxo2RqKvGui0CWLMSVUtXZ5ce3FrPlY9UaUIUiy5iKzWVlEeA46ZmZ0bT+VI1poJDXKILovXPejQubL4= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1575476490; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=dB/3Y9bIH1a14vLoEF0LLZxhxNrVzaJ1z3xxO50ziT4=; b=GVIdlsxA/44b98RF2p8GCHS7EEiSdQk+IJvibIyEIiCDr1hkw50uyuEXN5MKBbK3NCu7J+VEEOtlOKTF4/loN9z0NDKOq1fPunJKySbSLr+buqQCXaPpr9nlIdnrL3Ah1Pc9Rssk9oUUdV0S4eHCfkv7amulwFOs0U+rZApH1C0= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=none (zohomail.com: 192.237.175.120 is neither permitted nor denied by domain of lists.xenproject.org) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 157547649007886.36166917875425; Wed, 4 Dec 2019 08:21:30 -0800 (PST) Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1icXOW-0006sV-IT; Wed, 04 Dec 2019 16:20:52 +0000 Received: from all-amaz-eas1.inumbo.com ([34.197.232.57] helo=us1-amaz-eas2.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1icXOV-0006s1-4u for xen-devel@lists.xenproject.org; Wed, 04 Dec 2019 16:20:51 +0000 Received: from esa2.hc3370-68.iphmx.com (unknown [216.71.145.153]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS id 08ef6ff3-16b2-11ea-8206-12813bfff9fa; Wed, 04 Dec 2019 16:20:50 +0000 (UTC) X-Inumbo-ID: 08ef6ff3-16b2-11ea-8206-12813bfff9fa DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=citrix.com; s=securemail; t=1575476451; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=80NBp/22T/CAr+4PPAbMOKrqqzLlXhTyFbDherA9Olo=; b=PQGZsW9bXnhdsa/DeprYU7x7pcqbUKQ2HJs9xtmPyeqTFe7mkNKxrBO0 phxrEUCDGTBHh6QfCbEBoqQQydnrHl70ujqlP6v1S8gKe4thRsqai5t6d a1vxsqIh2ocfMWksip3VchSKa/DfD7BCxsNONkGurU7v6hHgaj6hKbl9d 8=; Authentication-Results: esa2.hc3370-68.iphmx.com; dkim=none (message not signed) header.i=none; spf=None smtp.pra=roger.pau@citrix.com; spf=Pass smtp.mailfrom=roger.pau@citrix.com; spf=None smtp.helo=postmaster@mail.citrix.com Received-SPF: none (zohomail.com: 192.237.175.120 is neither permitted nor denied by domain of lists.xenproject.org) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Received-SPF: None (esa2.hc3370-68.iphmx.com: no sender authenticity information available from domain of roger.pau@citrix.com) identity=pra; client-ip=162.221.158.21; receiver=esa2.hc3370-68.iphmx.com; envelope-from="roger.pau@citrix.com"; x-sender="roger.pau@citrix.com"; x-conformance=sidf_compatible Received-SPF: Pass (esa2.hc3370-68.iphmx.com: domain of roger.pau@citrix.com designates 162.221.158.21 as permitted sender) identity=mailfrom; client-ip=162.221.158.21; receiver=esa2.hc3370-68.iphmx.com; envelope-from="roger.pau@citrix.com"; x-sender="roger.pau@citrix.com"; x-conformance=sidf_compatible; x-record-type="v=spf1"; x-record-text="v=spf1 ip4:209.167.231.154 ip4:178.63.86.133 ip4:195.66.111.40/30 ip4:85.115.9.32/28 ip4:199.102.83.4 ip4:192.28.146.160 ip4:192.28.146.107 ip4:216.52.6.88 ip4:216.52.6.188 ip4:162.221.158.21 ip4:162.221.156.83 ip4:168.245.78.127 ~all" Received-SPF: None (esa2.hc3370-68.iphmx.com: no sender authenticity information available from domain of postmaster@mail.citrix.com) identity=helo; client-ip=162.221.158.21; receiver=esa2.hc3370-68.iphmx.com; envelope-from="roger.pau@citrix.com"; x-sender="postmaster@mail.citrix.com"; x-conformance=sidf_compatible IronPort-SDR: SY9ZnMe4o8+zOGEwLQAliZ60XRsR9Gb/WXmtgdJNxGcHwFeuBOoohiCHDKx/PUOLK1yjNtjzB/ 2fUj1hHmyTNTcXDm8eoHqHYsN/hT6DAPFPN0w1xLz7qQfwXD6TgzcXXqEgV8xHhk0qifXyvFJ2 HlIIQcigKfTJUAIAAqJJE4lxZ561VHOAOW1cULB1x4iKZ3AynO0lAMTkFsiZIh92//IRdoksuZ AwlVAQgAMbW0lnuu3CvEUv+9wkRIG3XcKYi73wOCvJDBKfalx/vyDPSvTRcGKmt3v44hU+CZHX 5yU= X-SBRS: 2.7 X-MesageID: 9200817 X-Ironport-Server: esa2.hc3370-68.iphmx.com X-Remote-IP: 162.221.158.21 X-Policy: $RELAYED X-IronPort-AV: E=Sophos;i="5.69,277,1571716800"; d="scan'208";a="9200817" From: Roger Pau Monne To: Date: Wed, 4 Dec 2019 17:20:24 +0100 Message-ID: <20191204162025.37510-4-roger.pau@citrix.com> X-Mailer: git-send-email 2.24.0 In-Reply-To: <20191204162025.37510-1-roger.pau@citrix.com> References: <20191204162025.37510-1-roger.pau@citrix.com> MIME-Version: 1.0 Subject: [Xen-devel] [PATCH v3 3/4] x86/smp: check APIC ID on AP bringup X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Andrew Cooper , Wei Liu , Jan Beulich , Roger Pau Monne Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) Check that the processor to be woken up APIC ID is addressable in the current APIC mode. Note that in practice systems with APIC IDs > 255 should already have x2APIC enabled by the firmware, and hence this is mostly a safety belt. Signed-off-by: Roger Pau Monn=C3=A9 Reviewed-by: Jan Beulich --- Changes since v2: - Reword error message. --- xen/arch/x86/smpboot.c | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/xen/arch/x86/smpboot.c b/xen/arch/x86/smpboot.c index fa691b6ba0..8cbb7173a4 100644 --- a/xen/arch/x86/smpboot.c +++ b/xen/arch/x86/smpboot.c @@ -1317,6 +1317,13 @@ int __cpu_up(unsigned int cpu) if ( (apicid =3D x86_cpu_to_apicid[cpu]) =3D=3D BAD_APICID ) return -ENODEV; =20 + if ( (!x2apic_enabled || !iommu_intremap) && (apicid >> 8) ) + { + printk("Unsupported: APIC ID %#x in xAPIC mode w/o interrupt remap= ping", + apicid); + return -EINVAL; + } + if ( (ret =3D do_boot_cpu(apicid, cpu)) !=3D 0 ) return ret; =20 --=20 2.24.0 _______________________________________________ Xen-devel mailing list Xen-devel@lists.xenproject.org https://lists.xenproject.org/mailman/listinfo/xen-devel From nobody Thu May 16 06:32:47 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=none (zohomail.com: 192.237.175.120 is neither permitted nor denied by domain of lists.xenproject.org) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=fail(p=none dis=none) header.from=citrix.com ARC-Seal: i=1; a=rsa-sha256; t=1575476489; cv=none; d=zohomail.com; s=zohoarc; b=krHgwcowsRnj102soIcQix4HQ/NvWJLNsWU3MIG0EsvgtCShEkbDdhA1YHoaRRhX+9WBHn3XCZZQdD/Ze0A2zPPno1RgQWbR3KQQM1wW1izKnkjeBsk807BwkM/kA42y/2fOdZhz/Ydi0kyA9cnKI/sSzqk57nUwOzGw6mOd+bI= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1575476489; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=Inz+S1kWlYl4Y4Cb7mC0tFjSJvtKnChdEEJtwg57eOg=; b=OUg7K8hgBnWICgzGoBf+4+Y1eIh5DZZSd7z51HZJGGwUxRIkomWsS4WumRDNkDdH1hk8fdfIHO+j0GYRsVWAtpUyR1nIEdCfpyTX8eH/acPVz1gBhNFNUma2FzbD4TW6lcQJW+Uu+xxCrlIevfAAWg588YW6EJl+EComC7LP2d0= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=none (zohomail.com: 192.237.175.120 is neither permitted nor denied by domain of lists.xenproject.org) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1575476489197357.06526448620025; Wed, 4 Dec 2019 08:21:29 -0800 (PST) Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1icXOa-0006vU-TS; Wed, 04 Dec 2019 16:20:56 +0000 Received: from us1-rack-iad1.inumbo.com ([172.99.69.81]) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1icXOZ-0006v0-RX for xen-devel@lists.xenproject.org; Wed, 04 Dec 2019 16:20:55 +0000 Received: from esa1.hc3370-68.iphmx.com (unknown [216.71.145.142]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS id 0a218c48-16b2-11ea-99dd-bc764e2007e4; Wed, 04 Dec 2019 16:20:51 +0000 (UTC) X-Inumbo-ID: 0a218c48-16b2-11ea-99dd-bc764e2007e4 DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=citrix.com; s=securemail; t=1575476452; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=4k3gp64Eb41SEtR8x1EWU9UW9MTRECYJ23Ww9hRv8as=; b=ROGMLSu8duZXfs11+ftMtWjQ+0OllEKHLsE/oCj1WD3OSquLS8u42j43 y4hc7JpONzQkub4m0SCReZRuKOaoBv1DFMkcXKABL91vLMfirkqT8rPQ/ OBhBD9l2m+x0eVo5kFe9tT5Wj5wnrbBS2ZKS3puQaK+XvWfxmT5vdsM3C 8=; Authentication-Results: esa1.hc3370-68.iphmx.com; dkim=none (message not signed) header.i=none; spf=None smtp.pra=roger.pau@citrix.com; spf=Pass smtp.mailfrom=roger.pau@citrix.com; spf=None smtp.helo=postmaster@mail.citrix.com Received-SPF: none (zohomail.com: 192.237.175.120 is neither permitted nor denied by domain of lists.xenproject.org) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Received-SPF: None (esa1.hc3370-68.iphmx.com: no sender authenticity information available from domain of roger.pau@citrix.com) identity=pra; client-ip=162.221.158.21; receiver=esa1.hc3370-68.iphmx.com; envelope-from="roger.pau@citrix.com"; x-sender="roger.pau@citrix.com"; x-conformance=sidf_compatible Received-SPF: Pass (esa1.hc3370-68.iphmx.com: domain of roger.pau@citrix.com designates 162.221.158.21 as permitted sender) identity=mailfrom; client-ip=162.221.158.21; receiver=esa1.hc3370-68.iphmx.com; envelope-from="roger.pau@citrix.com"; x-sender="roger.pau@citrix.com"; x-conformance=sidf_compatible; x-record-type="v=spf1"; x-record-text="v=spf1 ip4:209.167.231.154 ip4:178.63.86.133 ip4:195.66.111.40/30 ip4:85.115.9.32/28 ip4:199.102.83.4 ip4:192.28.146.160 ip4:192.28.146.107 ip4:216.52.6.88 ip4:216.52.6.188 ip4:162.221.158.21 ip4:162.221.156.83 ip4:168.245.78.127 ~all" Received-SPF: None (esa1.hc3370-68.iphmx.com: no sender authenticity information available from domain of postmaster@mail.citrix.com) identity=helo; client-ip=162.221.158.21; receiver=esa1.hc3370-68.iphmx.com; envelope-from="roger.pau@citrix.com"; x-sender="postmaster@mail.citrix.com"; x-conformance=sidf_compatible IronPort-SDR: M9QDGibj8NxtLJpgf2GP/+LrJpM24eyQmuKYkHKy1LLPe13bSzG21Y91bOka1G44LHPBWdzvkO 36rUuZ0Lzn81k8PKj0TaRqKgTFZNSrzcVUJIwNt1rpm47lfbPDEnYb29Klnnl7TOSNwSaQGLaS M4BRWwmQ48BaRyQg8hiZSuw7GmTa9ea1WKbIcquhT9+CCCjHmhR3h71EbeALbMFxbRnk8GAvvQ Gsrxe9uF66zx5BAcBHvYz5u6eoKoTITsAXg1tQo1mfILmXyD+BELsIAv1oBvVoPL/iJDsgS59n xbw= X-SBRS: 2.7 X-MesageID: 9323390 X-Ironport-Server: esa1.hc3370-68.iphmx.com X-Remote-IP: 162.221.158.21 X-Policy: $RELAYED X-IronPort-AV: E=Sophos;i="5.69,277,1571716800"; d="scan'208";a="9323390" From: Roger Pau Monne To: Date: Wed, 4 Dec 2019 17:20:25 +0100 Message-ID: <20191204162025.37510-5-roger.pau@citrix.com> X-Mailer: git-send-email 2.24.0 In-Reply-To: <20191204162025.37510-1-roger.pau@citrix.com> References: <20191204162025.37510-1-roger.pau@citrix.com> MIME-Version: 1.0 Subject: [Xen-devel] [PATCH v3 4/4] x86/apic: allow enabling x2APIC mode regardless of interrupt remapping X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Andrew Cooper , Wei Liu , Jan Beulich , Roger Pau Monne Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) x2APIC mode doesn't mandate interrupt remapping, and hence can be enabled independently. This patch enables x2APIC when available, regardless of whether there's interrupt remapping support. This is beneficial specially when running on virtualized environments, since it reduces the amount of vmexits. For example when sending an IPI in xAPIC mode Xen performs at least 3 different accesses to the APIC MMIO region, while when using x2APIC mode a single wrmsr is used. The following numbers are from a lock profiling of a Xen PV shim running a Linux PV kernel with 32 vCPUs and xAPIC mode: (XEN) Global lock flush_lock: addr=3Dffff82d0804af1c0, lockval=3D03190319, = not locked (XEN) lock:656153(892606463454), block:602183(9495067321843) Average lock time: 1360363ns Average block time: 15767743ns While the following are from the same configuration but with the shim using x2APIC mode: (XEN) Global lock flush_lock: addr=3Dffff82d0804b01c0, lockval=3D1adb1adb, = not locked (XEN) lock:1841883(1375128998543), block:1658716(10193054890781) Average lock time: 746588ns Average block time: 6145147ns Enabling x2APIC has halved the average lock time, thus reducing contention. Signed-off-by: Roger Pau Monn=C3=A9 Reviewed-by: Jan Beulich --- Changes since v2: - Cache the result of iommu_enable_x2apic so it can be used in the lapic suspend/resume paths. Changes since v1: - Fix error paths of iommu_enable_x2apic call in x2apic_bsp_setup. --- NB: should enabling x2APIC without interrupt remapping be limited to running on virtualized environments? The bigger performance benefit is indeed achieved when using x2APIC on virt environments, but I also don't see why we wouldn't want to try using it everywhere where it's supported. --- xen/arch/x86/apic.c | 94 ++++++++++++++++++++++----------------------- 1 file changed, 45 insertions(+), 49 deletions(-) diff --git a/xen/arch/x86/apic.c b/xen/arch/x86/apic.c index a8ee18636f..333115bc88 100644 --- a/xen/arch/x86/apic.c +++ b/xen/arch/x86/apic.c @@ -44,6 +44,8 @@ static bool __read_mostly tdt_enabled; static bool __initdata tdt_enable =3D true; boolean_param("tdt", tdt_enable); =20 +static bool __read_mostly iommu_x2apic_enabled; + static struct { int active; /* r/w apic fields */ @@ -492,7 +494,8 @@ static void __enable_x2apic(void) =20 static void resume_x2apic(void) { - iommu_enable_x2apic(); + if ( iommu_x2apic_enabled ) + iommu_enable_x2apic(); __enable_x2apic(); } =20 @@ -695,7 +698,8 @@ int lapic_suspend(void) =20 local_irq_save(flags); disable_local_APIC(); - iommu_disable_x2apic(); + if ( iommu_x2apic_enabled ) + iommu_disable_x2apic(); local_irq_restore(flags); return 0; } @@ -860,7 +864,6 @@ void __init x2apic_bsp_setup(void) { struct IO_APIC_route_entry **ioapic_entries =3D NULL; const char *orig_name; - bool intremap_enabled; =20 if ( !cpu_has_x2apic ) return; @@ -875,56 +878,46 @@ void __init x2apic_bsp_setup(void) printk("x2APIC: Already enabled by BIOS: Ignoring cmdline disable.= \n"); } =20 - if ( !iommu_supports_x2apic() ) + if ( iommu_supports_x2apic() ) { - if ( !x2apic_enabled ) + if ( (ioapic_entries =3D alloc_ioapic_entries()) =3D=3D NULL ) { - printk("Not enabling x2APIC: depends on IOMMU support\n"); - return; + printk("Allocate ioapic_entries failed\n"); + goto out; } - panic("x2APIC: already enabled by BIOS, but no IOMMU support\n"); - } - - if ( (ioapic_entries =3D alloc_ioapic_entries()) =3D=3D NULL ) - { - printk("Allocate ioapic_entries failed\n"); - goto out; - } =20 - if ( save_IO_APIC_setup(ioapic_entries) ) - { - printk("Saving IO-APIC state failed\n"); - goto out; - } + if ( save_IO_APIC_setup(ioapic_entries) ) + { + printk("Saving IO-APIC state failed\n"); + goto out; + } =20 - mask_8259A(); - mask_IO_APIC_setup(ioapic_entries); + mask_8259A(); + mask_IO_APIC_setup(ioapic_entries); =20 - switch ( iommu_enable_x2apic() ) - { - case 0: - intremap_enabled =3D true; - break; - case -ENXIO: /* ACPI_DMAR_X2APIC_OPT_OUT set */ - if ( !x2apic_enabled ) + switch ( iommu_enable_x2apic() ) { + case 0: + iommu_x2apic_enabled =3D true; + break; + + case -ENXIO: /* ACPI_DMAR_X2APIC_OPT_OUT set */ + if ( x2apic_enabled ) + panic("IOMMU requests xAPIC mode, but x2APIC already enabl= ed by firmware\n"); + printk("Not enabling x2APIC (upon firmware request)\n"); - intremap_enabled =3D false; + iommu_x2apic_enabled =3D false; goto restore_out; + + default: + printk(XENLOG_ERR "Failed to enable Interrupt Remapping\n"); + iommu_x2apic_enabled =3D false; + break; } - /* fall through */ - default: - if ( x2apic_enabled ) - panic("Interrupt remapping could not be enabled while " - "x2APIC is already enabled by BIOS\n"); - - printk(XENLOG_ERR - "Failed to enable Interrupt Remapping: Will not enable x2AP= IC.\n"); - intremap_enabled =3D false; - goto restore_out; - } =20 - force_iommu =3D 1; + if ( iommu_x2apic_enabled ) + force_iommu =3D 1; + } =20 if ( !x2apic_enabled ) { @@ -938,13 +931,16 @@ void __init x2apic_bsp_setup(void) printk("Switched to APIC driver %s\n", genapic.name); =20 restore_out: - /* - * NB: do not use raw mode when restoring entries if the iommu has been - * enabled during the process, because the entries need to be translat= ed - * and added to the remapping table in that case. - */ - restore_IO_APIC_setup(ioapic_entries, !intremap_enabled); - unmask_8259A(); + if ( iommu_supports_x2apic() ) + { + /* + * NB: do not use raw mode when restoring entries if the iommu has= been + * enabled during the process, because the entries need to be tran= slated + * and added to the remapping table in that case. + */ + restore_IO_APIC_setup(ioapic_entries, !iommu_x2apic_enabled); + unmask_8259A(); + } =20 out: if ( ioapic_entries ) --=20 2.24.0 _______________________________________________ Xen-devel mailing list Xen-devel@lists.xenproject.org https://lists.xenproject.org/mailman/listinfo/xen-devel