From nobody Thu Sep 18 23:34:42 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id ED557C47088 for ; Thu, 1 Dec 2022 13:02:55 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231381AbiLANCh (ORCPT ); Thu, 1 Dec 2022 08:02:37 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42988 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231247AbiLANCA (ORCPT ); Thu, 1 Dec 2022 08:02:00 -0500 Received: from mail-pg1-x532.google.com (mail-pg1-x532.google.com [IPv6:2607:f8b0:4864:20::532]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id AB8C2BB7E5 for ; Thu, 1 Dec 2022 05:01:59 -0800 (PST) Received: by mail-pg1-x532.google.com with SMTP id 62so1585960pgb.13 for ; Thu, 01 Dec 2022 05:01:59 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ventanamicro.com; s=google; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=H/HSAfRl6AKxOrRE0JEbET+E+vFKUld6EeEf9bI00fs=; b=M7PuHzfiXSfdhv0tm5LFBjVrBaVMI6DaaQ7Pc1uxzJVMLBAyABGYiO5pQqYxxsmJ2n XNUAH13NrjW9a+kf3t4YDoPX3/rFi/kXQe7L0okkTG8+czurk83pgzP49XJWCKscppDf HovR99A/JULd09rk4CnMEUIJOxaGhDt+hhRSlizgrGZnXfI3V9wQYL6sCBTG0KAEKXh4 mnUGjsJmtPdmLh6G3OFyP2Z3xvXGFS6spLq4+3laUTbxkcA7KFFj/pSZPNelgA285wYr WJg9yp9GPUTVl8jjSJE9Gr+t8KD8IwqEV91z7TIigC997aLDoMSnVY3gafQFmY7o5II5 GUIg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=H/HSAfRl6AKxOrRE0JEbET+E+vFKUld6EeEf9bI00fs=; b=TpK0tTfgwXVjhFu7aRsZJPNAjfnbcG7MdrkrZcqvZBy2mI0SX11lGfXMFrb4GorgYT J6x1kb2sQmhGRqxcY3pCmZYBejsjH8GqKL0hYb9Eu49U539fCVaBUqvjRS3eJ7/MM5YE SDecjRvhY8Zylkcj01LoyQO01NRy7kstYcj5bN4doPcN1FVVLAz+puLjhnWraBUzB3CS iVRkFgiYkJn3mWLB+yooLB1pIH2XW9fmqsF556zizVdavjLwJyETA3o8OBVRs9xqFi2d SRVo45NBeA6/ku7EFF1w3rnGH8BBuc/9VDzeZl1GZ77M+/qOS128kUKzkd0v+1X1tUnF lbVg== X-Gm-Message-State: ANoB5plFx+bQq+KD5G0AAaXiKS8cg8s2tgrn0ssc31/1o2+tXbgB/Ab4 mcSrQS2YQ4MKFE54+QR4z6G0xA== X-Google-Smtp-Source: AA0mqf5D3j9ipSlTSkY/Z59EA1lVTCN3KjTMIgNrwJggjGP6V2hfzJ4Y8nrjZYOEfOtw1ApWaJN7og== X-Received: by 2002:a63:dc04:0:b0:46f:1262:8df7 with SMTP id s4-20020a63dc04000000b0046f12628df7mr51214679pgg.39.1669899719123; Thu, 01 Dec 2022 05:01:59 -0800 (PST) Received: from anup-ubuntu-vm.localdomain ([171.76.81.69]) by smtp.gmail.com with ESMTPSA id l4-20020a17090a384400b00212c27abcaesm4855856pjf.17.2022.12.01.05.01.55 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 01 Dec 2022 05:01:58 -0800 (PST) From: Anup Patel To: Palmer Dabbelt , Paul Walmsley , Thomas Gleixner , Marc Zyngier , Daniel Lezcano Cc: Atish Patra , Alistair Francis , Anup Patel , linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, Anup Patel Subject: [PATCH v14 3/8] genirq: Add mechanism to multiplex a single HW IPI Date: Thu, 1 Dec 2022 18:31:30 +0530 Message-Id: <20221201130135.1115380-4-apatel@ventanamicro.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20221201130135.1115380-1-apatel@ventanamicro.com> References: <20221201130135.1115380-1-apatel@ventanamicro.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" All RISC-V platforms have a single HW IPI provided by the INTC local interrupt controller. The HW method to trigger INTC IPI can be through external irqchip (e.g. RISC-V AIA), through platform specific device (e.g. SiFive CLINT timer), or through firmware (e.g. SBI IPI call). To support multiple IPIs on RISC-V, we add a generic IPI multiplexing mechanism which help us create multiple virtual IPIs using a single HW IPI. This generic IPI multiplexing is inspired from the Apple AIC irqchip driver and it is shared by various RISC-V irqchip drivers. Signed-off-by: Anup Patel --- include/linux/irq.h | 3 + kernel/irq/Kconfig | 5 ++ kernel/irq/Makefile | 1 + kernel/irq/ipi-mux.c | 199 +++++++++++++++++++++++++++++++++++++++++++ 4 files changed, 208 insertions(+) create mode 100644 kernel/irq/ipi-mux.c diff --git a/include/linux/irq.h b/include/linux/irq.h index c3eb89606c2b..b1b28affb32a 100644 --- a/include/linux/irq.h +++ b/include/linux/irq.h @@ -1266,6 +1266,9 @@ int __ipi_send_mask(struct irq_desc *desc, const stru= ct cpumask *dest); int ipi_send_single(unsigned int virq, unsigned int cpu); int ipi_send_mask(unsigned int virq, const struct cpumask *dest); =20 +void ipi_mux_process(void); +int ipi_mux_create(unsigned int nr_ipi, void (*mux_send)(unsigned int cpu)= ); + #ifdef CONFIG_GENERIC_IRQ_MULTI_HANDLER /* * Registers a generic IRQ handling function as the top-level IRQ handler = in diff --git a/kernel/irq/Kconfig b/kernel/irq/Kconfig index db3d174c53d4..df17dbc54b02 100644 --- a/kernel/irq/Kconfig +++ b/kernel/irq/Kconfig @@ -86,6 +86,11 @@ config GENERIC_IRQ_IPI depends on SMP select IRQ_DOMAIN_HIERARCHY =20 +# Generic IRQ IPI Mux support +config GENERIC_IRQ_IPI_MUX + bool + depends on SMP + # Generic MSI interrupt support config GENERIC_MSI_IRQ bool diff --git a/kernel/irq/Makefile b/kernel/irq/Makefile index b4f53717d143..f19d3080bf11 100644 --- a/kernel/irq/Makefile +++ b/kernel/irq/Makefile @@ -15,6 +15,7 @@ obj-$(CONFIG_GENERIC_IRQ_MIGRATION) +=3D cpuhotplug.o obj-$(CONFIG_PM_SLEEP) +=3D pm.o obj-$(CONFIG_GENERIC_MSI_IRQ) +=3D msi.o obj-$(CONFIG_GENERIC_IRQ_IPI) +=3D ipi.o +obj-$(CONFIG_GENERIC_IRQ_IPI_MUX) +=3D ipi-mux.o obj-$(CONFIG_SMP) +=3D affinity.o obj-$(CONFIG_GENERIC_IRQ_DEBUGFS) +=3D debugfs.o obj-$(CONFIG_GENERIC_IRQ_MATRIX_ALLOCATOR) +=3D matrix.o diff --git a/kernel/irq/ipi-mux.c b/kernel/irq/ipi-mux.c new file mode 100644 index 000000000000..1a6ba19b736c --- /dev/null +++ b/kernel/irq/ipi-mux.c @@ -0,0 +1,199 @@ +// SPDX-License-Identifier: GPL-2.0-or-later +/* + * Multiplex several virtual IPIs over a single HW IPI. + * + * Copyright The Asahi Linux Contributors + * Copyright (c) 2022 Ventana Micro Systems Inc. + */ + +#define pr_fmt(fmt) "ipi-mux: " fmt +#include +#include +#include +#include +#include +#include +#include +#include +#include + +struct ipi_mux_cpu { + atomic_t enable; + atomic_t bits; +}; + +static struct ipi_mux_cpu __percpu *ipi_mux_pcpu; +static struct irq_domain *ipi_mux_domain; +static void (*ipi_mux_send)(unsigned int cpu); + +static void ipi_mux_mask(struct irq_data *d) +{ + struct ipi_mux_cpu *icpu =3D this_cpu_ptr(ipi_mux_pcpu); + + atomic_andnot(BIT(irqd_to_hwirq(d)), &icpu->enable); +} + +static void ipi_mux_unmask(struct irq_data *d) +{ + u32 ibit =3D BIT(irqd_to_hwirq(d)); + struct ipi_mux_cpu *icpu =3D this_cpu_ptr(ipi_mux_pcpu); + + atomic_or(ibit, &icpu->enable); + + /* + * The atomic_or() above must complete before the atomic_read() + * below to avoid racing ipi_mux_send_mask(). + */ + smp_mb__after_atomic(); + + /* If a pending IPI was unmasked, raise a parent IPI immediately. */ + if (atomic_read(&icpu->bits) & ibit) + ipi_mux_send(smp_processor_id()); +} + +static void ipi_mux_send_mask(struct irq_data *d, const struct cpumask *ma= sk) +{ + u32 ibit =3D BIT(irqd_to_hwirq(d)); + struct ipi_mux_cpu *icpu =3D this_cpu_ptr(ipi_mux_pcpu); + unsigned long pending; + int cpu; + + for_each_cpu(cpu, mask) { + icpu =3D per_cpu_ptr(ipi_mux_pcpu, cpu); + pending =3D atomic_fetch_or_release(ibit, &icpu->bits); + + /* + * The atomic_fetch_or_release() above must complete + * before the atomic_read() below to avoid racing with + * ipi_mux_unmask(). + */ + smp_mb__after_atomic(); + + /* + * The flag writes must complete before the physical IPI is + * issued to another CPU. This is implied by the control + * dependency on the result of atomic_read() below, which is + * itself already ordered after the vIPI flag write. + */ + if (!(pending & ibit) && (atomic_read(&icpu->enable) & ibit)) + ipi_mux_send(cpu); + } +} + +static const struct irq_chip ipi_mux_chip =3D { + .name =3D "IPI Mux", + .irq_mask =3D ipi_mux_mask, + .irq_unmask =3D ipi_mux_unmask, + .ipi_send_mask =3D ipi_mux_send_mask, +}; + +static int ipi_mux_domain_alloc(struct irq_domain *d, unsigned int virq, + unsigned int nr_irqs, void *arg) +{ + int i; + + for (i =3D 0; i < nr_irqs; i++) { + irq_set_percpu_devid(virq + i); + irq_domain_set_info(d, virq + i, i, &ipi_mux_chip, NULL, + handle_percpu_devid_irq, NULL, NULL); + } + + return 0; +} + +static const struct irq_domain_ops ipi_mux_domain_ops =3D { + .alloc =3D ipi_mux_domain_alloc, + .free =3D irq_domain_free_irqs_top, +}; + +/** + * ipi_mux_process - Process multiplexed virtual IPIs + */ +void ipi_mux_process(void) +{ + struct ipi_mux_cpu *icpu =3D this_cpu_ptr(ipi_mux_pcpu); + irq_hw_number_t hwirq; + unsigned long ipis; + unsigned int en; + + /* + * Reading enable mask does not need to be ordered as long as + * this function called from interrupt handler because only + * the CPU itself can change it's own enable mask. + */ + en =3D atomic_read(&icpu->enable); + + /* + * Clear the IPIs we are about to handle. This pairs with the + * atomic_fetch_or_release() in ipi_mux_send_mask(). + */ + ipis =3D atomic_fetch_andnot(en, &icpu->bits) & en; + + for_each_set_bit(hwirq, &ipis, BITS_PER_TYPE(int)) + generic_handle_domain_irq(ipi_mux_domain, hwirq); +} + +/** + * ipi_mux_create - Create virtual IPIs multiplexed on top of a single + * parent IPI. + * @nr_ipi: number of virtual IPIs to create. This should + * be <=3D BITS_PER_TYPE(int) + * @mux_send: callback to trigger parent IPI for a particular CPU + * + * Returns first virq of the newly created virtual IPIs upon success + * or <=3D0 upon failure + */ +int ipi_mux_create(unsigned int nr_ipi, void (*mux_send)(unsigned int cpu)) +{ + struct fwnode_handle *fwnode; + struct irq_domain *domain; + int rc; + + if (ipi_mux_domain) + return -EEXIST; + + if (BITS_PER_TYPE(int) < nr_ipi || !mux_send) + return -EINVAL; + + ipi_mux_pcpu =3D alloc_percpu(typeof(*ipi_mux_pcpu)); + if (!ipi_mux_pcpu) + return -ENOMEM; + + fwnode =3D irq_domain_alloc_named_fwnode("IPI-Mux"); + if (!fwnode) { + pr_err("unable to create IPI Mux fwnode\n"); + rc =3D -ENOMEM; + goto fail_free_cpu; + } + + domain =3D irq_domain_create_linear(fwnode, nr_ipi, + &ipi_mux_domain_ops, NULL); + if (!domain) { + pr_err("unable to add IPI Mux domain\n"); + rc =3D -ENOMEM; + goto fail_free_fwnode; + } + + domain->flags |=3D IRQ_DOMAIN_FLAG_IPI_SINGLE; + irq_domain_update_bus_token(domain, DOMAIN_BUS_IPI); + + rc =3D __irq_domain_alloc_irqs(domain, -1, nr_ipi, + NUMA_NO_NODE, NULL, false, NULL); + if (rc <=3D 0) { + pr_err("unable to alloc IRQs from IPI Mux domain\n"); + goto fail_free_domain; + } + + ipi_mux_domain =3D domain; + ipi_mux_send =3D mux_send; + + return rc; + +fail_free_domain: + irq_domain_remove(domain); +fail_free_fwnode: + irq_domain_free_fwnode(fwnode); +fail_free_cpu: + free_percpu(ipi_mux_pcpu); + return rc; +} --=20 2.34.1