From nobody Mon Jun 22 22:43:51 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5DA5CC433EF for ; Tue, 15 Mar 2022 16:50:49 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1350222AbiCOQv7 (ORCPT ); Tue, 15 Mar 2022 12:51:59 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57982 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1350192AbiCOQv4 (ORCPT ); Tue, 15 Mar 2022 12:51:56 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 331945715E; Tue, 15 Mar 2022 09:50:44 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id B0ECF6150C; Tue, 15 Mar 2022 16:50:43 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 1339CC340F4; Tue, 15 Mar 2022 16:50:43 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1647363043; bh=DLkgOaHA7X05ej4y1HzFD8hVwJioUhOuvlYG1ioWxlU=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=nGdIHrwzla74JRUbBhrpwH1GfAcq/po8n5QsxHfrGUifnI0RZgs5B3iZqJKEcD+J8 fPiWLvFudoImXGBqYRLB1ff8rTfzXswdEt67KIc2Bdu+3wOlGQjhZd2tBA4vIJVP0t A0nnJIrAeyl+t+N+MlCy4Qh9iQH+pB4SFbqYgmAZbiMYoitSjmN9V3FGAaNmXAe/78 sxxDre9/nDbWZN49kXToY70QlDCF8m+N0gDiGyAi2dSKzF76hfuaRPcC6xyw2qooSg vxbV2xxrKwbZsDdgeDS5U+bnWJOcMV2gQCbqEruHLwwcGaj/4EH74CEGWSvSZZUYaR tFZrbjJppoiIw== Received: from sofa.misterjones.org ([185.219.108.64] helo=why.lan) by disco-boy.misterjones.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1nUANc-00EhkO-W5; Tue, 15 Mar 2022 16:50:41 +0000 From: Marc Zyngier To: linux-kernel@vger.kernel.org Cc: Lorenzo Pieralisi , Andre Przywara , Thomas Gleixner , Eric Auger , stable@vger.kernel.org Subject: [PATCH 1/3] irqchip/gic-v3: Fix GICR_CTLR.RWP polling Date: Tue, 15 Mar 2022 16:50:32 +0000 Message-Id: <20220315165034.794482-2-maz@kernel.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220315165034.794482-1-maz@kernel.org> References: <20220315165034.794482-1-maz@kernel.org> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-SA-Exim-Connect-IP: 185.219.108.64 X-SA-Exim-Rcpt-To: linux-kernel@vger.kernel.org, lorenzo.pieralisi@arm.com, andre.przywara@arm.com, tglx@linutronix.de, eric.auger@redhat.com, stable@vger.kernel.org X-SA-Exim-Mail-From: maz@kernel.org X-SA-Exim-Scanned: No (on disco-boy.misterjones.org); SAEximRunCond expanded to false Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" It turns out that our polling of RWP is totally wrong when checking for it in the redistributors, as we test the *distributor* bit index, whereas it is a different bit number in the RDs... Oopsie boo. This is embarassing. Not only because it is wrong, but also because it took *8 years* to notice the blunder... Just fix the damn thing. Fixes: 021f653791ad ("irqchip: gic-v3: Initial support for GICv3") Signed-off-by: Marc Zyngier Cc: stable@vger.kernel.org Reviewed-by: Andre Przywara Reviewed-by: Lorenzo Pieralisi --- drivers/irqchip/irq-gic-v3.c | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/drivers/irqchip/irq-gic-v3.c b/drivers/irqchip/irq-gic-v3.c index 5e935d97207d..736163d36b13 100644 --- a/drivers/irqchip/irq-gic-v3.c +++ b/drivers/irqchip/irq-gic-v3.c @@ -206,11 +206,11 @@ static inline void __iomem *gic_dist_base(struct irq_= data *d) } } =20 -static void gic_do_wait_for_rwp(void __iomem *base) +static void gic_do_wait_for_rwp(void __iomem *base, u32 bit) { u32 count =3D 1000000; /* 1s! */ =20 - while (readl_relaxed(base + GICD_CTLR) & GICD_CTLR_RWP) { + while (readl_relaxed(base + GICD_CTLR) & bit) { count--; if (!count) { pr_err_ratelimited("RWP timeout, gone fishing\n"); @@ -224,13 +224,13 @@ static void gic_do_wait_for_rwp(void __iomem *base) /* Wait for completion of a distributor change */ static void gic_dist_wait_for_rwp(void) { - gic_do_wait_for_rwp(gic_data.dist_base); + gic_do_wait_for_rwp(gic_data.dist_base, GICD_CTLR_RWP); } =20 /* Wait for completion of a redistributor change */ static void gic_redist_wait_for_rwp(void) { - gic_do_wait_for_rwp(gic_data_rdist_rd_base()); + gic_do_wait_for_rwp(gic_data_rdist_rd_base(), GICR_CTLR_RWP); } =20 #ifdef CONFIG_ARM64 --=20 2.34.1 From nobody Mon Jun 22 22:43:51 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3CE99C433F5 for ; Tue, 15 Mar 2022 16:50:55 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1350240AbiCOQwF (ORCPT ); Tue, 15 Mar 2022 12:52:05 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58004 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1350213AbiCOQv6 (ORCPT ); Tue, 15 Mar 2022 12:51:58 -0400 Received: from ams.source.kernel.org (ams.source.kernel.org [IPv6:2604:1380:4601:e00::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id DF3625715E for ; Tue, 15 Mar 2022 09:50:45 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id 9682BB81802 for ; Tue, 15 Mar 2022 16:50:44 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 452F9C340F5; Tue, 15 Mar 2022 16:50:43 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1647363043; bh=xhdjH/t9SclaiYQ8bmgF90cNseOgXjX/v15wc5UmvGs=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=gIrhDTk5zYaHoLojixfhOvB5k7Z+il885Q2CzsDI9Q1J/e1nhwfurdh5+BlUD7At1 EzanIHegv0MDssYkOtfyf9eXZ3Z8xyfGLDCpQT1WymMJnim82AGcJnfcH6LaFslUSi 4qwc4fRYo2tx3jeD79Bv+qzC2wmHMgpnCkpu+IV1carWCZMYihgRIuSxNyXaLWneKB PXHletRfc/KdrlEe6ICCaZUZr82rn+P6qrgorjeIOx+SNiPRyYVs+0Di5IAY2WTGTd kgXU6jQRToK+WOdVxd98i/Uh9e0Oelqx9ehF/O3zb6pb6Tv2mq6l+vO05Zt2brEGs+ 59TfMsDPV/b8w== Received: from sofa.misterjones.org ([185.219.108.64] helo=why.lan) by disco-boy.misterjones.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1nUANd-00EhkO-6w; Tue, 15 Mar 2022 16:50:41 +0000 From: Marc Zyngier To: linux-kernel@vger.kernel.org Cc: Lorenzo Pieralisi , Andre Przywara , Thomas Gleixner , Eric Auger Subject: [PATCH 2/3] irqchip/gic-v3: Detect LPI invalidation MMIO registers Date: Tue, 15 Mar 2022 16:50:33 +0000 Message-Id: <20220315165034.794482-3-maz@kernel.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220315165034.794482-1-maz@kernel.org> References: <20220315165034.794482-1-maz@kernel.org> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-SA-Exim-Connect-IP: 185.219.108.64 X-SA-Exim-Rcpt-To: linux-kernel@vger.kernel.org, lorenzo.pieralisi@arm.com, andre.przywara@arm.com, tglx@linutronix.de, eric.auger@redhat.com X-SA-Exim-Mail-From: maz@kernel.org X-SA-Exim-Scanned: No (on disco-boy.misterjones.org); SAEximRunCond expanded to false Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Since GICv4.1, an implementation can offer the same MMIO-based implementation as DirectLPI, only with an ITS. Given that this can be hugely beneficial for workloads that are very LPI masking heavy (although these workloads are admitedly a bit odd). Interestingly, this is independent of RVPEI, which only *implies* the functionnality. So let's detect whether the implementation has GICR_CTLR.IR set, and propagate this as DirectLPI to the ITS driver. Signed-off-by: Marc Zyngier --- drivers/irqchip/irq-gic-v3.c | 15 +++++++++++---- include/linux/irqchip/arm-gic-v3.h | 2 ++ 2 files changed, 13 insertions(+), 4 deletions(-) diff --git a/drivers/irqchip/irq-gic-v3.c b/drivers/irqchip/irq-gic-v3.c index 736163d36b13..363bfe172033 100644 --- a/drivers/irqchip/irq-gic-v3.c +++ b/drivers/irqchip/irq-gic-v3.c @@ -918,7 +918,11 @@ static int gic_populate_rdist(void) static int __gic_update_rdist_properties(struct redist_region *region, void __iomem *ptr) { - u64 typer =3D gic_read_typer(ptr + GICR_TYPER); + u64 typer; + u32 ctlr; + + typer =3D gic_read_typer(ptr + GICR_TYPER); + ctlr =3D readl_relaxed(ptr + GICR_CTLR); =20 /* Boot-time cleanip */ if ((typer & GICR_TYPER_VLPIS) && (typer & GICR_TYPER_RVPEID)) { @@ -941,6 +945,7 @@ static int __gic_update_rdist_properties(struct redist_= region *region, /* RVPEID implies some form of DirectLPI, no matter what the doc says... = :-/ */ gic_data.rdists.has_rvpeid &=3D !!(typer & GICR_TYPER_RVPEID); gic_data.rdists.has_direct_lpi &=3D (!!(typer & GICR_TYPER_DirectLPIS) | + !!(ctlr & GICR_CTLR_IR) | gic_data.rdists.has_rvpeid); gic_data.rdists.has_vpend_valid_dirty &=3D !!(typer & GICR_TYPER_DIRTY); =20 @@ -962,7 +967,11 @@ static void gic_update_rdist_properties(void) gic_iterate_rdists(__gic_update_rdist_properties); if (WARN_ON(gic_data.ppi_nr =3D=3D UINT_MAX)) gic_data.ppi_nr =3D 0; - pr_info("%d PPIs implemented\n", gic_data.ppi_nr); + pr_info("GICv3 features: %d PPIs, %s%s\n", + gic_data.ppi_nr, + gic_data.has_rss ? "RSS " : "", + gic_data.rdists.has_direct_lpi ? "DirectLPI " : ""); +=09 if (gic_data.rdists.has_vlpis) pr_info("GICv4 features: %s%s%s\n", gic_data.rdists.has_direct_lpi ? "DirectLPI " : "", @@ -1797,8 +1806,6 @@ static int __init gic_init_bases(void __iomem *dist_b= ase, irq_domain_update_bus_token(gic_data.domain, DOMAIN_BUS_WIRED); =20 gic_data.has_rss =3D !!(typer & GICD_TYPER_RSS); - pr_info("Distributor has %sRange Selector support\n", - gic_data.has_rss ? "" : "no "); =20 if (typer & GICD_TYPER_MBIS) { err =3D mbi_init(handle, gic_data.domain); diff --git a/include/linux/irqchip/arm-gic-v3.h b/include/linux/irqchip/arm= -gic-v3.h index 12d91f0dedf9..aeb8ced53880 100644 --- a/include/linux/irqchip/arm-gic-v3.h +++ b/include/linux/irqchip/arm-gic-v3.h @@ -127,6 +127,8 @@ #define GICR_PIDR2 GICD_PIDR2 =20 #define GICR_CTLR_ENABLE_LPIS (1UL << 0) +#define GICR_CTLR_IR (1UL << 1) +#define GICR_CTLR_CES (1UL << 2) #define GICR_CTLR_RWP (1UL << 3) =20 #define GICR_TYPER_CPU_NUMBER(r) (((r) >> 8) & 0xffff) --=20 2.34.1 From nobody Mon Jun 22 22:43:51 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2AE80C433EF for ; Tue, 15 Mar 2022 16:50:51 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1350227AbiCOQwB (ORCPT ); Tue, 15 Mar 2022 12:52:01 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58002 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1350211AbiCOQv6 (ORCPT ); Tue, 15 Mar 2022 12:51:58 -0400 Received: from ams.source.kernel.org (ams.source.kernel.org [IPv6:2604:1380:4601:e00::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D7AD7517D0 for ; Tue, 15 Mar 2022 09:50:45 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id 87EB9B8122D for ; Tue, 15 Mar 2022 16:50:44 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 4ACD9C340F6; Tue, 15 Mar 2022 16:50:43 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1647363043; bh=QZZYi5QmeQzKihh20U0K1CiBaYfurFjvlWB0iX29PWE=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=i/uqp3zcxOehHaa7hoOUopBn2FCbFcxG3GCi/ZnPLrRhDkCenlnMolZuTRkr9VUXg VfgHfdnzwqA1VCpUuHdZE4fO3Op4e4ePrEdnFvLB91/kCH5xtcm++Q8fc25A4mMTcJ btwcQrDIAxZu36mYtCd748A+jCQCj36dYjkzRoe9O6uJbc+ZfIZ8FCL2UkSpTEpttH G5M1ndAZxNAeqcuJI5KFnqOHHbGToOixEi7kEMnP9+DaNCABthU9ZEP3vza64dJIYw ZGen5bKX4l+WqSDa5YA9h1kWr4LsxTf6smZa2E6p0lwlsTsyKkyKCcxPjSqaQ8jSDt ADsnWHFr/WhPQ== Received: from sofa.misterjones.org ([185.219.108.64] helo=why.lan) by disco-boy.misterjones.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1nUANd-00EhkO-Cm; Tue, 15 Mar 2022 16:50:41 +0000 From: Marc Zyngier To: linux-kernel@vger.kernel.org Cc: Lorenzo Pieralisi , Andre Przywara , Thomas Gleixner , Eric Auger Subject: [PATCH 3/3] irqchip/gic-v3: Relax polling of GIC{R,D}_CTLR.RWP Date: Tue, 15 Mar 2022 16:50:34 +0000 Message-Id: <20220315165034.794482-4-maz@kernel.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220315165034.794482-1-maz@kernel.org> References: <20220315165034.794482-1-maz@kernel.org> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-SA-Exim-Connect-IP: 185.219.108.64 X-SA-Exim-Rcpt-To: linux-kernel@vger.kernel.org, lorenzo.pieralisi@arm.com, andre.przywara@arm.com, tglx@linutronix.de, eric.auger@redhat.com X-SA-Exim-Mail-From: maz@kernel.org X-SA-Exim-Scanned: No (on disco-boy.misterjones.org); SAEximRunCond expanded to false Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Recent work on the KVM GIC emulation has revealed that the GICv3 driver is a bit RWP-happy, as it polls this bit for each and every write MMIO access involving a single interrupt. As it turns out, polling RWP is only required when: - Disabling an SGI, PPI or SPI - Disabling LPIs at the redistributor level - Disabling groups - Enabling ARE - Dealing with DPG* Simplify the driver by removing all the other instances of RWP polling, and add the one that was missing when enabling the distributor (as that's where we set ARE). Signed-off-by: Marc Zyngier --- drivers/irqchip/irq-gic-v3.c | 28 +++++++++++----------------- 1 file changed, 11 insertions(+), 17 deletions(-) diff --git a/drivers/irqchip/irq-gic-v3.c b/drivers/irqchip/irq-gic-v3.c index 363bfe172033..05ff7fef64cb 100644 --- a/drivers/irqchip/irq-gic-v3.c +++ b/drivers/irqchip/irq-gic-v3.c @@ -352,28 +352,27 @@ static int gic_peek_irq(struct irq_data *d, u32 offse= t) =20 static void gic_poke_irq(struct irq_data *d, u32 offset) { - void (*rwp_wait)(void); void __iomem *base; u32 index, mask; =20 offset =3D convert_offset_index(d, offset, &index); mask =3D 1 << (index % 32); =20 - if (gic_irq_in_rdist(d)) { + if (gic_irq_in_rdist(d)) base =3D gic_data_rdist_sgi_base(); - rwp_wait =3D gic_redist_wait_for_rwp; - } else { + else base =3D gic_data.dist_base; - rwp_wait =3D gic_dist_wait_for_rwp; - } =20 writel_relaxed(mask, base + offset + (index / 32) * 4); - rwp_wait(); } =20 static void gic_mask_irq(struct irq_data *d) { gic_poke_irq(d, GICD_ICENABLER); + if (gic_irq_in_rdist(d)) + gic_redist_wait_for_rwp(); + else + gic_dist_wait_for_rwp(); } =20 static void gic_eoimode1_mask_irq(struct irq_data *d) @@ -574,7 +573,6 @@ static int gic_set_type(struct irq_data *d, unsigned in= t type) { enum gic_intid_range range; unsigned int irq =3D gic_irq(d); - void (*rwp_wait)(void); void __iomem *base; u32 offset, index; int ret; @@ -590,17 +588,14 @@ static int gic_set_type(struct irq_data *d, unsigned = int type) type !=3D IRQ_TYPE_LEVEL_HIGH && type !=3D IRQ_TYPE_EDGE_RISING) return -EINVAL; =20 - if (gic_irq_in_rdist(d)) { + if (gic_irq_in_rdist(d)) base =3D gic_data_rdist_sgi_base(); - rwp_wait =3D gic_redist_wait_for_rwp; - } else { + else base =3D gic_data.dist_base; - rwp_wait =3D gic_dist_wait_for_rwp; - } =20 offset =3D convert_offset_index(d, GICD_ICFGR, &index); =20 - ret =3D gic_configure_irq(index, type, base + offset, rwp_wait); + ret =3D gic_configure_irq(index, type, base + offset, NULL); if (ret && (range =3D=3D PPI_RANGE || range =3D=3D EPPI_RANGE)) { /* Misconfigured PPIs are usually not fatal */ pr_warn("GIC: PPI INTID%d is secure or misconfigured\n", irq); @@ -808,7 +803,7 @@ static void __init gic_dist_init(void) writel_relaxed(GICD_INT_DEF_PRI_X4, base + GICD_IPRIORITYRnE + i); =20 /* Now do the common stuff, and wait for the distributor to drain */ - gic_dist_config(base, GIC_LINE_NR, gic_dist_wait_for_rwp); + gic_dist_config(base, GIC_LINE_NR, NULL); =20 val =3D GICD_CTLR_ARE_NS | GICD_CTLR_ENABLE_G1A | GICD_CTLR_ENABLE_G1; if (gic_data.rdists.gicd_typer2 & GICD_TYPER2_nASSGIcap) { @@ -818,6 +813,7 @@ static void __init gic_dist_init(void) =20 /* Enable distributor with ARE, Group1 */ writel_relaxed(val, base + GICD_CTLR); + gic_dist_wait_for_rwp(); =20 /* * Set all global interrupts to the boot CPU only. ARE must be @@ -1293,8 +1289,6 @@ static int gic_set_affinity(struct irq_data *d, const= struct cpumask *mask_val, */ if (enabled) gic_unmask_irq(d); - else - gic_dist_wait_for_rwp(); =20 irq_data_update_effective_affinity(d, cpumask_of(cpu)); =20 --=20 2.34.1