irqchip: riscv-imsic: Add kernel parameter to disable IPIs

[PATCH] irqchip: riscv-imsic: Add kernel parameter to disable IPIs

Posted by Anup Patel 3 months, 2 weeks ago

When injecting IPIs to a set of harts, the IMSIC IPI support will
do a separate MMIO write to SETIPNUM_LE register of each target
hart. This means on a platform where IMSIC is trap-n-emulated,
there will be N MMIO traps when injecting IPI to N target harts
hence IPIs based on IMSIC software injected MSI is slow compared
to the SBI IPI extension.

Add a kernel parameter to disable IPIs in IMSIC driver for platforms
with trap-n-emulated IMSIC.

Signed-off-by: Anup Patel <apatel@ventanamicro.com>
---
 Documentation/admin-guide/kernel-parameters.txt |  7 +++++++
 drivers/irqchip/irq-riscv-imsic-early.c         | 12 ++++++++++++
 2 files changed, 19 insertions(+)

diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt
index f1f2c0874da9..7f0e12d0d260 100644
--- a/Documentation/admin-guide/kernel-parameters.txt
+++ b/Documentation/admin-guide/kernel-parameters.txt
@@ -2538,6 +2538,13 @@
 			requires the kernel to be built with
 			CONFIG_ARM64_PSEUDO_NMI.
 
+	irqchip.riscv_imsic_noipi
+			[RISC-V,EARLY]
+			Force the kernel to not use IMSIC software injected MSIs
+			as IPIs. Intended for system where IMSIC is trap-n-emulated,
+			and thus want to reduce MMIO traps when triggering IPIs
+			to multiple harts.
+
 	irqfixup	[HW]
 			When an interrupt is not handled search all handlers
 			for it. Intended to get systems with badly broken
diff --git a/drivers/irqchip/irq-riscv-imsic-early.c b/drivers/irqchip/irq-riscv-imsic-early.c
index 1dbc41d7fe80..c6fba92dd5a9 100644
--- a/drivers/irqchip/irq-riscv-imsic-early.c
+++ b/drivers/irqchip/irq-riscv-imsic-early.c
@@ -9,6 +9,7 @@
 #include <linux/cpu.h>
 #include <linux/export.h>
 #include <linux/interrupt.h>
+#include <linux/init.h>
 #include <linux/io.h>
 #include <linux/irq.h>
 #include <linux/irqchip.h>
@@ -22,6 +23,14 @@
 #include "irq-riscv-imsic-state.h"
 
 static int imsic_parent_irq;
+static bool imsic_noipi;
+
+static int __init imsic_noipi_cfg(char *buf)
+{
+	imsic_noipi = true;
+	return 0;
+}
+early_param("irqchip.riscv_imsic_noipi", imsic_noipi_cfg);
 
 #ifdef CONFIG_SMP
 static void imsic_ipi_send(unsigned int cpu)
@@ -47,6 +56,9 @@ static int __init imsic_ipi_domain_init(void)
 {
 	int virq;
 
+	if (imsic_noipi)
+		return 0;
+
 	/* Create IMSIC IPI multiplexing */
 	virq = ipi_mux_create(IMSIC_NR_IPI, imsic_ipi_send);
 	if (virq <= 0)
-- 
2.43.0

Re: [PATCH] irqchip: riscv-imsic: Add kernel parameter to disable IPIs

Posted by Thomas Gleixner 3 months, 1 week ago

On Wed, Jun 25 2025 at 21:47, Anup Patel wrote:

$Subject...

https://www.kernel.org/doc/html/latest/process/maintainer-tip.html#patch-subject

Is it that hard?

> When injecting IPIs to a set of harts, the IMSIC IPI support will
> do a separate MMIO write to SETIPNUM_LE register of each target
> hart. This means on a platform where IMSIC is trap-n-emulated,
> there will be N MMIO traps when injecting IPI to N target harts
> hence IPIs based on IMSIC software injected MSI is slow compared
> to the SBI IPI extension.
>
> Add a kernel parameter to disable IPIs in IMSIC driver for platforms
> with trap-n-emulated IMSIC.

Why do you need a kernel parameter for that. If the platform uses trap-n
emulation, then disable the IPI muck automatically, no?

Thanks,

        tglx

Re: [PATCH] irqchip: riscv-imsic: Add kernel parameter to disable IPIs

Posted by Anup Patel 3 months, 1 week ago

On Mon, Jun 30, 2025 at 9:39 PM Thomas Gleixner <tglx@linutronix.de> wrote:
>
> On Wed, Jun 25 2025 at 21:47, Anup Patel wrote:
>
> $Subject...
>
> https://www.kernel.org/doc/html/latest/process/maintainer-tip.html#patch-subject
>
> Is it that hard?

Not sure why I chose a different subject prefix this time. I will update
this in the next revision.

>
> > When injecting IPIs to a set of harts, the IMSIC IPI support will
> > do a separate MMIO write to SETIPNUM_LE register of each target
> > hart. This means on a platform where IMSIC is trap-n-emulated,
> > there will be N MMIO traps when injecting IPI to N target harts
> > hence IPIs based on IMSIC software injected MSI is slow compared
> > to the SBI IPI extension.
> >
> > Add a kernel parameter to disable IPIs in IMSIC driver for platforms
> > with trap-n-emulated IMSIC.
>
> Why do you need a kernel parameter for that. If the platform uses trap-n
> emulation, then disable the IPI muck automatically, no?
>

Unfortunately, we don't have DT, ACPI, or any other way of discovering
whether underlying IMSIC is trap-n-emulated. In fact, the DT or ACPI
passed to a KVM Guest is the same irrespective of whether underlying
IMSIC is trap-n-emulated or backed by hardware IMSIC VS-file.

Using software injected MSIs as IPIs is purely a software choice in the
IMSIC driver so this new kernel parameter allows users to override it.

Regards,
Anup

Re: [PATCH] irqchip: riscv-imsic: Add kernel parameter to disable IPIs

Posted by Thomas Gleixner 3 months, 1 week ago

On Tue, Jul 01 2025 at 12:00, Anup Patel wrote:
> On Mon, Jun 30, 2025 at 9:39 PM Thomas Gleixner <tglx@linutronix.de> wrote:
>> > When injecting IPIs to a set of harts, the IMSIC IPI support will
>> > do a separate MMIO write to SETIPNUM_LE register of each target
>> > hart. This means on a platform where IMSIC is trap-n-emulated,
>> > there will be N MMIO traps when injecting IPI to N target harts
>> > hence IPIs based on IMSIC software injected MSI is slow compared
>> > to the SBI IPI extension.
>> >
>> > Add a kernel parameter to disable IPIs in IMSIC driver for platforms
>> > with trap-n-emulated IMSIC.
>>
>> Why do you need a kernel parameter for that. If the platform uses trap-n
>> emulation, then disable the IPI muck automatically, no?
>>
> Unfortunately, we don't have DT, ACPI, or any other way of discovering
> whether underlying IMSIC is trap-n-emulated. In fact, the DT or ACPI
> passed to a KVM Guest is the same irrespective of whether underlying
> IMSIC is trap-n-emulated or backed by hardware IMSIC VS-file.

Sigh.

> Using software injected MSIs as IPIs is purely a software choice in the
> IMSIC driver so this new kernel parameter allows users to override it.

Please add that information to the change log.

Thanks,

        tglx

Re: [PATCH] irqchip: riscv-imsic: Add kernel parameter to disable IPIs

Posted by Anup Patel 3 months, 1 week ago

On Tue, Jul 1, 2025 at 1:04 PM Thomas Gleixner <tglx@linutronix.de> wrote:
>
> On Tue, Jul 01 2025 at 12:00, Anup Patel wrote:
> > On Mon, Jun 30, 2025 at 9:39 PM Thomas Gleixner <tglx@linutronix.de> wrote:
> >> > When injecting IPIs to a set of harts, the IMSIC IPI support will
> >> > do a separate MMIO write to SETIPNUM_LE register of each target
> >> > hart. This means on a platform where IMSIC is trap-n-emulated,
> >> > there will be N MMIO traps when injecting IPI to N target harts
> >> > hence IPIs based on IMSIC software injected MSI is slow compared
> >> > to the SBI IPI extension.
> >> >
> >> > Add a kernel parameter to disable IPIs in IMSIC driver for platforms
> >> > with trap-n-emulated IMSIC.
> >>
> >> Why do you need a kernel parameter for that. If the platform uses trap-n
> >> emulation, then disable the IPI muck automatically, no?
> >>
> > Unfortunately, we don't have DT, ACPI, or any other way of discovering
> > whether underlying IMSIC is trap-n-emulated. In fact, the DT or ACPI
> > passed to a KVM Guest is the same irrespective of whether underlying
> > IMSIC is trap-n-emulated or backed by hardware IMSIC VS-file.
>
> Sigh.
>
> > Using software injected MSIs as IPIs is purely a software choice in the
> > IMSIC driver so this new kernel parameter allows users to override it.
>
> Please add that information to the change log.
>

Okay, I will add these details to the patch description.

Regards,
Anup