From nobody Mon Nov 25 02:27:28 2024 Received: from galois.linutronix.de (Galois.linutronix.de [193.142.43.55]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2C0271F4717; Wed, 30 Oct 2024 14:49:53 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=193.142.43.55 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1730299797; cv=none; b=pEI9PMgRYFJpejw48xspMxtxwp4EwHwLo1b+YFlWGYXE70yQEMLM/mdxgn9SfsUXmnQcIQ0DWV/tQYkjPTLJIB3AiRgDtAiISjslH0dfQTBlBvtF4KFfUgJ9bwOy/r92rXSYudgYiUEpO4e4GMT5pqY8OYaL38vbj05JTIe2L3c= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1730299797; c=relaxed/simple; bh=3lPWkW32hWl8AfexOGbICS5dr1e408tLuvniNicjBc4=; h=Date:From:To:Subject:Cc:In-Reply-To:References:MIME-Version: Message-ID:Content-Type; b=mrsFZJ6aop4vbAJeqJoIPLHckeV6LMtB7Y0qkIifTyYzUig/VrxGLp7L/x0kHxeeuT3GEap25g0iuhQcNu4iNUQLt1YvqEAXr7qmIX/Yke9b7XBgj8bAurxlaf3GJUCabyIqVij5Jyt1voCtFekPyXf4rVq7T3a+gI2xbhy6N9E= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linutronix.de; spf=pass smtp.mailfrom=linutronix.de; dkim=pass (2048-bit key) header.d=linutronix.de header.i=@linutronix.de header.b=u6K31HDk; dkim=permerror (0-bit key) header.d=linutronix.de header.i=@linutronix.de header.b=1gGB/6dO; arc=none smtp.client-ip=193.142.43.55 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linutronix.de Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linutronix.de Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=linutronix.de header.i=@linutronix.de header.b="u6K31HDk"; dkim=permerror (0-bit key) header.d=linutronix.de header.i=@linutronix.de header.b="1gGB/6dO" Date: Wed, 30 Oct 2024 14:49:51 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1730299792; h=from:from:sender:sender:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=fhw5TDkdI/wm0hFEphOAnFv1KynVGxXFayg/h3+LrgY=; b=u6K31HDkxbTjcIFDNhoDp1GA1QMUfrdnaXBpsh/3bAfh7LsJrm604LAZtuKeKAkUDkNNS2 SZC26uKSx0imCxYGN20VqXjFPIOaswIcCDsnjMH0VI1GHe/ItpuOXzSFRBC5HAnKLot110 DG1eZPq7mzV/aQuk3yWR7bF1sIvd8n9TvUsv/MaqHhO8D2KwZJk6m0y/kBr6i0bSp+iheP iPDtFxM3EjaqPGX30mIkRLql1c3cHAH9/Mkkh/amdGFReUmF3f18F+eEB6FAkpdcHB94bs A+7oOh0VeqPUs2gxPfAMux4yiRsYfOVx0UPHNSQJ/rNI06/TdcSxePA+1VFS8Q== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1730299792; h=from:from:sender:sender:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=fhw5TDkdI/wm0hFEphOAnFv1KynVGxXFayg/h3+LrgY=; b=1gGB/6dOaXXjzhZDQXRLNx2uMSYrHsOyOomyq+juSzsW1tPRS+oG9qe59cDWXCd+TXwY3Q jdjAHv42kLYehlCg== From: "tip-bot2 for Paul Burton" Sender: tip-bot2@linutronix.de Reply-to: linux-kernel@vger.kernel.org To: linux-tip-commits@vger.kernel.org Subject: [tip: irq/core] irqchip/mips-gic: Multi-cluster support Cc: Paul Burton , "Chao-ying Fu" , Dragan Mladjenovic , Aleksandar Rikalo , Thomas Gleixner , Serge Semin , Gregory CLEMENT , x86@kernel.org, linux-kernel@vger.kernel.org, maz@kernel.org In-Reply-To: <20241028175935.51250-5-arikalo@gmail.com> References: <20241028175935.51250-5-arikalo@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-ID: <173029979171.3137.5721964738341029557.tip-bot2@tip-bot2> Robot-ID: Robot-Unsubscribe: Contact to get blacklisted from these emails Precedence: bulk Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable The following commit has been merged into the irq/core branch of tip: Commit-ID: 322a9063876890895cb8308cc6f59de312e8aad5 Gitweb: https://git.kernel.org/tip/322a9063876890895cb8308cc6f59de31= 2e8aad5 Author: Paul Burton AuthorDate: Mon, 28 Oct 2024 18:59:26 +01:00 Committer: Thomas Gleixner CommitterDate: Wed, 30 Oct 2024 15:41:18 +01:00 irqchip/mips-gic: Multi-cluster support The MIPS I6500 CPU & CM (Coherence Manager) 3.5 introduce the concept of multiple clusters to the system. In these systems, each cluster contains its own GIC, so the GIC isn't truly global any longer. Access to registers in the GICs of remote clusters is possible using a redirect register block much like the redirect register blocks provided by the CM & CPC, and configured through the same GCR_REDIRECT register that mips_cm_lock_other() abstraction builds upon. It is expected that external interrupts are connected identically on all clusters. That is, if there is a device providing an interrupt connected to GIC interrupt pin 0 then it should be connected to pin 0 of every GIC in the system. For the most part, the GIC can be treated as though it is still truly global, so long as interrupts in the cluster are configured properly. Introduce support for such multi-cluster systems in the MIPS GIC irqchip driver. A newly introduced gic_irq_lock_cluster() function allows: 1) Configure access to a GIC in a remote cluster via the redirect register block, using mips_cm_lock_other(). Or: 2) Detect that the interrupt in question is affine to the local cluster and plain old GIC register access to the GIC in the local cluster should be used. It is possible to access the local cluster's GIC registers via the redirect block, but keeping the special case for them is both good for performance (because we avoid the locking & indirection overhead of using the redirect block) and necessary to maintain compatibility with systems using CM revisions prior to 3.5 which don't support the redirect block. The gic_irq_lock_cluster() function relies upon an IRQs effective affinity in order to discover which cluster the IRQ is affine to. In order to track this & allow it to be updated at an appropriate point during gic_set_affinity() select the generic support for effective affinity using CONFIG_GENERIC_IRQ_EFFECTIVE_AFF_MASK. gic_set_affinity() is the one function which gains much complexity. It now deconfigures routing to any VP(E), ie. CPU, on the old cluster when moving affinity to a new cluster. gic_shared_irq_domain_map() moves its update of the IRQs effective affinity to before its use of gic_irq_lock_cluster(), to ensure that operation is on the cluster the IRQ is affine to. The remaining changes are straightforward use of the gic_irq_lock_cluster() function to select between local cluster & remote cluster code-paths when configuring interrupts. Signed-off-by: Paul Burton Signed-off-by: Chao-ying Fu Signed-off-by: Dragan Mladjenovic Signed-off-by: Aleksandar Rikalo Signed-off-by: Thomas Gleixner Tested-by: Serge Semin Tested-by: Gregory CLEMENT Link: https://lore.kernel.org/all/20241028175935.51250-5-arikalo@gmail.com --- drivers/irqchip/Kconfig | 1 +- drivers/irqchip/irq-mips-gic.c | 161 ++++++++++++++++++++++++++++---- 2 files changed, 143 insertions(+), 19 deletions(-) diff --git a/drivers/irqchip/Kconfig b/drivers/irqchip/Kconfig index 006a47a..f20adf7 100644 --- a/drivers/irqchip/Kconfig +++ b/drivers/irqchip/Kconfig @@ -352,6 +352,7 @@ config KEYSTONE_IRQ =20 config MIPS_GIC bool + select GENERIC_IRQ_EFFECTIVE_AFF_MASK select GENERIC_IRQ_IPI if SMP select IRQ_DOMAIN_HIERARCHY select MIPS_CM diff --git a/drivers/irqchip/irq-mips-gic.c b/drivers/irqchip/irq-mips-gic.c index d93a076..f42f69b 100644 --- a/drivers/irqchip/irq-mips-gic.c +++ b/drivers/irqchip/irq-mips-gic.c @@ -111,6 +111,41 @@ static inline void gic_unlock_cluster(void) gic_unlock_cluster(), \ (cpu) =3D __gic_with_next_online_cpu(cpu)) =20 +/** + * gic_irq_lock_cluster() - Lock redirect block access to IRQ's cluster + * @d: struct irq_data corresponding to the interrupt we're interested in + * + * Locks redirect register block access to the global register block of th= e GIC + * within the remote cluster that the IRQ corresponding to @d is affine to, + * returning true when this redirect block setup & locking has been perfor= med. + * + * If @d is affine to the local cluster then no locking is performed and t= his + * function will return false, indicating to the caller that it should acc= ess + * the local clusters registers without the overhead of indirection throug= h the + * redirect block. + * + * In summary, if this function returns true then the caller should access= GIC + * registers using redirect register block accessors & then call + * mips_cm_unlock_other() when done. If this function returns false then t= he + * caller should trivially access GIC registers in the local cluster. + * + * Returns true if locking performed, else false. + */ +static bool gic_irq_lock_cluster(struct irq_data *d) +{ + unsigned int cpu, cl; + + cpu =3D cpumask_first(irq_data_get_effective_affinity_mask(d)); + BUG_ON(cpu >=3D NR_CPUS); + + cl =3D cpu_cluster(&cpu_data[cpu]); + if (cl =3D=3D cpu_cluster(¤t_cpu_data)) + return false; + + mips_cm_lock_other(cl, 0, 0, CM_GCR_Cx_OTHER_BLOCK_GLOBAL); + return true; +} + static void gic_clear_pcpu_masks(unsigned int intr) { unsigned int i; @@ -157,7 +192,12 @@ static void gic_send_ipi(struct irq_data *d, unsigned = int cpu) { irq_hw_number_t hwirq =3D GIC_HWIRQ_TO_SHARED(irqd_to_hwirq(d)); =20 - write_gic_wedge(GIC_WEDGE_RW | hwirq); + if (gic_irq_lock_cluster(d)) { + write_gic_redir_wedge(GIC_WEDGE_RW | hwirq); + mips_cm_unlock_other(); + } else { + write_gic_wedge(GIC_WEDGE_RW | hwirq); + } } =20 int gic_get_c0_compare_int(void) @@ -225,7 +265,13 @@ static void gic_mask_irq(struct irq_data *d) { unsigned int intr =3D GIC_HWIRQ_TO_SHARED(d->hwirq); =20 - write_gic_rmask(intr); + if (gic_irq_lock_cluster(d)) { + write_gic_redir_rmask(intr); + mips_cm_unlock_other(); + } else { + write_gic_rmask(intr); + } + gic_clear_pcpu_masks(intr); } =20 @@ -234,7 +280,12 @@ static void gic_unmask_irq(struct irq_data *d) unsigned int intr =3D GIC_HWIRQ_TO_SHARED(d->hwirq); unsigned int cpu; =20 - write_gic_smask(intr); + if (gic_irq_lock_cluster(d)) { + write_gic_redir_smask(intr); + mips_cm_unlock_other(); + } else { + write_gic_smask(intr); + } =20 gic_clear_pcpu_masks(intr); cpu =3D cpumask_first(irq_data_get_effective_affinity_mask(d)); @@ -245,7 +296,12 @@ static void gic_ack_irq(struct irq_data *d) { unsigned int irq =3D GIC_HWIRQ_TO_SHARED(d->hwirq); =20 - write_gic_wedge(irq); + if (gic_irq_lock_cluster(d)) { + write_gic_redir_wedge(irq); + mips_cm_unlock_other(); + } else { + write_gic_wedge(irq); + } } =20 static int gic_set_type(struct irq_data *d, unsigned int type) @@ -285,9 +341,16 @@ static int gic_set_type(struct irq_data *d, unsigned i= nt type) break; } =20 - change_gic_pol(irq, pol); - change_gic_trig(irq, trig); - change_gic_dual(irq, dual); + if (gic_irq_lock_cluster(d)) { + change_gic_redir_pol(irq, pol); + change_gic_redir_trig(irq, trig); + change_gic_redir_dual(irq, dual); + mips_cm_unlock_other(); + } else { + change_gic_pol(irq, pol); + change_gic_trig(irq, trig); + change_gic_dual(irq, dual); + } =20 if (trig =3D=3D GIC_TRIG_EDGE) irq_set_chip_handler_name_locked(d, &gic_edge_irq_controller, @@ -305,25 +368,72 @@ static int gic_set_affinity(struct irq_data *d, const= struct cpumask *cpumask, bool force) { unsigned int irq =3D GIC_HWIRQ_TO_SHARED(d->hwirq); + unsigned int cpu, cl, old_cpu, old_cl; unsigned long flags; - unsigned int cpu; =20 + /* + * The GIC specifies that we can only route an interrupt to one VP(E), + * ie. CPU in Linux parlance, at a time. Therefore we always route to + * the first online CPU in the mask. + */ cpu =3D cpumask_first_and(cpumask, cpu_online_mask); if (cpu >=3D NR_CPUS) return -EINVAL; =20 - /* Assumption : cpumask refers to a single CPU */ - raw_spin_lock_irqsave(&gic_lock, flags); + old_cpu =3D cpumask_first(irq_data_get_effective_affinity_mask(d)); + old_cl =3D cpu_cluster(&cpu_data[old_cpu]); + cl =3D cpu_cluster(&cpu_data[cpu]); =20 - /* Re-route this IRQ */ - write_gic_map_vp(irq, BIT(mips_cm_vp_id(cpu))); + raw_spin_lock_irqsave(&gic_lock, flags); =20 - /* Update the pcpu_masks */ - gic_clear_pcpu_masks(irq); - if (read_gic_mask(irq)) - set_bit(irq, per_cpu_ptr(pcpu_masks, cpu)); + /* + * If we're moving affinity between clusters, stop routing the + * interrupt to any VP(E) in the old cluster. + */ + if (cl !=3D old_cl) { + if (gic_irq_lock_cluster(d)) { + write_gic_redir_map_vp(irq, 0); + mips_cm_unlock_other(); + } else { + write_gic_map_vp(irq, 0); + } + } =20 + /* + * Update effective affinity - after this gic_irq_lock_cluster() will + * begin operating on the new cluster. + */ irq_data_update_effective_affinity(d, cpumask_of(cpu)); + + /* + * If we're moving affinity between clusters, configure the interrupt + * trigger type in the new cluster. + */ + if (cl !=3D old_cl) + gic_set_type(d, irqd_get_trigger_type(d)); + + /* Route the interrupt to its new VP(E) */ + if (gic_irq_lock_cluster(d)) { + write_gic_redir_map_pin(irq, + GIC_MAP_PIN_MAP_TO_PIN | gic_cpu_pin); + write_gic_redir_map_vp(irq, BIT(mips_cm_vp_id(cpu))); + + /* Update the pcpu_masks */ + gic_clear_pcpu_masks(irq); + if (read_gic_redir_mask(irq)) + set_bit(irq, per_cpu_ptr(pcpu_masks, cpu)); + + mips_cm_unlock_other(); + } else { + write_gic_map_pin(irq, GIC_MAP_PIN_MAP_TO_PIN | gic_cpu_pin); + write_gic_map_vp(irq, BIT(mips_cm_vp_id(cpu))); + + /* Update the pcpu_masks */ + gic_clear_pcpu_masks(irq); + if (read_gic_mask(irq)) + set_bit(irq, per_cpu_ptr(pcpu_masks, cpu)); + } + raw_spin_unlock_irqrestore(&gic_lock, flags); =20 return IRQ_SET_MASK_OK; @@ -471,11 +581,21 @@ static int gic_shared_irq_domain_map(struct irq_domai= n *d, unsigned int virq, unsigned long flags; =20 data =3D irq_get_irq_data(virq); + irq_data_update_effective_affinity(data, cpumask_of(cpu)); =20 raw_spin_lock_irqsave(&gic_lock, flags); - write_gic_map_pin(intr, GIC_MAP_PIN_MAP_TO_PIN | gic_cpu_pin); - write_gic_map_vp(intr, BIT(mips_cm_vp_id(cpu))); - irq_data_update_effective_affinity(data, cpumask_of(cpu)); + + /* Route the interrupt to its VP(E) */ + if (gic_irq_lock_cluster(data)) { + write_gic_redir_map_pin(intr, + GIC_MAP_PIN_MAP_TO_PIN | gic_cpu_pin); + write_gic_redir_map_vp(intr, BIT(mips_cm_vp_id(cpu))); + mips_cm_unlock_other(); + } else { + write_gic_map_pin(intr, GIC_MAP_PIN_MAP_TO_PIN | gic_cpu_pin); + write_gic_map_vp(intr, BIT(mips_cm_vp_id(cpu))); + } + raw_spin_unlock_irqrestore(&gic_lock, flags); =20 return 0; @@ -651,6 +771,9 @@ static int gic_ipi_domain_alloc(struct irq_domain *d, u= nsigned int virq, if (ret) goto error; =20 + /* Set affinity to cpu. */ + irq_data_update_effective_affinity(irq_get_irq_data(virq + i), + cpumask_of(cpu)); ret =3D irq_set_irq_type(virq + i, IRQ_TYPE_EDGE_RISING); if (ret) goto error;