[PATCH v5 0/7] Global Software Interrupt Moderation (GSIM)

Luigi Rizzo posted 7 patches 1 month, 1 week ago
drivers/irqchip/irq-msi-lib.c |   8 +
drivers/pci/msi/irqdomain.c   |  20 +
include/linux/irq.h           |  11 +-
include/linux/irqdesc.h       |  12 +
kernel/irq/Kconfig            |  11 +
kernel/irq/Makefile           |   1 +
kernel/irq/chip.c             |  14 +
kernel/irq/debugfs.c          |   3 +
kernel/irq/internals.h        |  20 +
kernel/irq/irq_moderation.c   | 959 ++++++++++++++++++++++++++++++++++
kernel/irq/irq_moderation.h   | 156 ++++++
kernel/irq/irqdesc.c          |   1 +
kernel/irq/manage.c           |  10 +
kernel/irq/proc.c             |   2 +
kernel/irq/settings.h         |  17 +
15 files changed, 1244 insertions(+), 1 deletion(-)
create mode 100644 kernel/irq/irq_moderation.c
create mode 100644 kernel/irq/irq_moderation.h
[PATCH v5 0/7] Global Software Interrupt Moderation (GSIM)
Posted by Luigi Rizzo 1 month, 1 week ago
Global Software Interrupt Moderation (GSIM) addresses a limitation of
platforms, from many vendors, whose I/O performance drops significantly
when the total rate of MSI-X interrupts is too high (e.g 1..3M intr/s
depending on the platform).

Conventional interrupt moderation, typically implemented in hardware
by NICs or storage devices, operates separately on each source (e.g. a
completion queue). Large servers can have hundreds of sources, and
without knowledge of global activity, keeping the total rate bounded would
require moderation delays of 100-200us. Per-source adaptive moderation
would reach those delays with as little as 10K intr/s per source. These
values (high delay triggering very early) are unacceptable for RPC or
transactional workloads.

To address this problem, GSIM measures efficiently the total and per-CPU
interrupt rates, so that individual moderation delays can be dynamically
adjusted based on current global and local load. This way, delays are
normally 0 or very small except during actual local/global overload.

As an additional benefit, GSIM also monitors the percentage of time
spent by each CPU in hardirq, and can use moderation to reserve some
time for other, lower priority, tasks.

Configuration is easy and robust. System administrators specify the
maximum targets (moderation delay; interrupt rate; percentage of time
spent in hardirq), and which interrupt sources should be moderated.
Independent per-CPU control loops adjust actual delays to try and keep
metrics within the targets.

The system is adaptive. Moderation affects only latency and only in
high load scenarios.  Throughput and CPU efficiencly generally benefits
significantly.  Targets don't need to match precisely the platform
limits, and one can make conservative and robust choices. Values like
delay_us=100, target_intr_rate=1000000, hardirq_percent=70 are a very
good starting point.

GSIM does not rely on any special hardware feature.

Global parameters can be modified at runtime with

    echo ${VALUE} | sudo tee /proc/irq/sw_moderation/${NAME}

/proc/irq/sw_moderation/stats exports statistics when enabled.

and moderation on individual interrupts can be turned on/off at runtime with

    echo 1 | sudo tee /proc/irq/NN/allow_moderation  # use 0 to disable

EXAMPLE:
    # global configuration: 50us, target max 1M intr/s, and 70% in intr
    echo 50 | sudo tee /proc/irq/sw_moderation/delay_us
    echo 1000000 | sudo tee /proc/irq/sw_moderation/target_intr_rate
    echo 70 | sudo tee /proc/irq/sw_moderation/hardirq_percent

    # allow moderation on all interrupts that support it
    # remember to periodically check and set the flag for dynamically
    # created interrupts since the default is 0
    echo 1 | sudo tee /proc/irq/*/allow_moderation
    
    #  check the status
    grep -r . /proc/irq/*/*/../allow_moderation

    # look at statistics
    less /proc/irq/sw_moderation/stats

PERFORMANCE BENEFITS:
Below are some experimental results under high load comparing conventional
moderation with GSIM:

- 100Gbps NIC, 32 queues: rx goes from 50 Gbps to 92.8 Gbps (line rate).
- 200Gbps NIC, 10 VMs (total 160 queues): rx goes from 30 Gbps to 190 Gbps (line rate).
- 12 SSD, 96 queues: 4K random read goes from 6M to 20.5M IOPS (device max).

In all cases, with adaptive moderatrion, latency up to p95 is unaffected
at low/moderate load, even if compared with no moderation at all.

Changes in v5:
- refactored the commits based on previous feedback
- various cleanups
- conditionally reverted parent IRQ mask/unmask, which would completely
  defeat the mechanism GSIM is based on.

Changes in v4:
- added irqdesc and irqdata flags as suggested by maintainer
- parameters are only configured via independent procfs entries.
  No module parameters anymore.
- merged control and interrupt functions back into a single header/C files
- applied various annotations (lockdep, data_race())
- formatting and various renaming as suggested by maintainer.
- added performance measurements with adaptive moderation.
- removed the mechanism to conditionally enable moderation at interrupt
  creation. This can be done in userspace and suitable udev extensions
  will be handled separately.

Changes in v3:
- clearly documented architecture in kernel/irq/irq_moderation.c
  including how to handle enable/disable/mask, interrupt migration,
  hotplug and suspend.
- split implementation in 4 files irq_moderation.[ch] and
  irq_moderation_hook.[ch] for better separation of control plane and
  "dataplane" (functions ran on each interrupt)
- limited scope to handle_edge_irq() and handle_fasteoi_irq() which
  have been tested on actual hardware.
- tested on Intel (also with intremap=posted_msi), AMD, ARM, with NIC,
  nvme, vfio

Changes in v2:
- many style fixes (mostly on comments) based on reviewers' comments on v1
- removed background from Documentation/core-api/irq/irq-moderation.rst
- split procfs handlers
- moved internal details to kernel/irq/irq_moderation.h
- use cpu hotplug for per-CPU setup, removed unnecessary arch-specific changes
- select suitable irqs based on !irqd_is_level_type(irqd) && irqd_is_single_target(irqd)
- use a static_key to enable/disable the feature



Luigi Rizzo (7):
  genirq: Add flags for software interrupt moderation.
  genirq: Add GSIM infrastructure
  genirq: Implement core GSIM moderation logic
  genirq: Integrate GSIM into interrupt flow
  genirq: Add GSIM user space configuration (procfs)
  genirq: Adaptive Global Software Interrupt Moderation (GSIM).
  PCI/MSI: re-enable conditional parent mask/unmask with sw moderation

 drivers/irqchip/irq-msi-lib.c |   8 +
 drivers/pci/msi/irqdomain.c   |  20 +
 include/linux/irq.h           |  11 +-
 include/linux/irqdesc.h       |  12 +
 kernel/irq/Kconfig            |  11 +
 kernel/irq/Makefile           |   1 +
 kernel/irq/chip.c             |  14 +
 kernel/irq/debugfs.c          |   3 +
 kernel/irq/internals.h        |  20 +
 kernel/irq/irq_moderation.c   | 959 ++++++++++++++++++++++++++++++++++
 kernel/irq/irq_moderation.h   | 156 ++++++
 kernel/irq/irqdesc.c          |   1 +
 kernel/irq/manage.c           |  10 +
 kernel/irq/proc.c             |   2 +
 kernel/irq/settings.h         |  17 +
 15 files changed, 1244 insertions(+), 1 deletion(-)
 create mode 100644 kernel/irq/irq_moderation.c
 create mode 100644 kernel/irq/irq_moderation.h

-- 
2.55.0.737.g08866a6d13-goog
Re: [PATCH v5 0/7] Global Software Interrupt Moderation (GSIM)
Posted by Christoph Hellwig 1 month, 1 week ago
On Wed, Aug 19, 2026 at 12:43:34PM +0000, Luigi Rizzo wrote:
> Configuration is easy and robust. System administrators specify the
> maximum targets (moderation delay; interrupt rate; percentage of time
> spent in hardirq), and which interrupt sources should be moderated.
> Independent per-CPU control loops adjust actual delays to try and keep
> metrics within the targets.

Can we find a way to autodetect and autoenable this?  A magic go faster
mode that needs very specific tuning is annoying compare to sensible
defaults.

Also how does this interact with adaptive polling code inside drivers
like NAPI or the upcoming nvme variant?

> 
> The system is adaptive. Moderation affects only latency and only in
> high load scenarios.  Throughput and CPU efficiencly generally benefits
> significantly.  Targets don't need to match precisely the platform
> limits, and one can make conservative and robust choices. Values like
> delay_us=100, target_intr_rate=1000000, hardirq_percent=70 are a very
> good starting point.
> 
> GSIM does not rely on any special hardware feature.
> 
> Global parameters can be modified at runtime with
> 
>     echo ${VALUE} | sudo tee /proc/irq/sw_moderation/${NAME}
> 
> /proc/irq/sw_moderation/stats exports statistics when enabled.
> 
> and moderation on individual interrupts can be turned on/off at runtime with
> 
>     echo 1 | sudo tee /proc/irq/NN/allow_moderation  # use 0 to disable
> 
> EXAMPLE:
>     # global configuration: 50us, target max 1M intr/s, and 70% in intr
>     echo 50 | sudo tee /proc/irq/sw_moderation/delay_us
>     echo 1000000 | sudo tee /proc/irq/sw_moderation/target_intr_rate
>     echo 70 | sudo tee /proc/irq/sw_moderation/hardirq_percent
> 
>     # allow moderation on all interrupts that support it
>     # remember to periodically check and set the flag for dynamically
>     # created interrupts since the default is 0
>     echo 1 | sudo tee /proc/irq/*/allow_moderation
>     
>     #  check the status
>     grep -r . /proc/irq/*/*/../allow_moderation
> 
>     # look at statistics
>     less /proc/irq/sw_moderation/stats
> 
> PERFORMANCE BENEFITS:
> Below are some experimental results under high load comparing conventional
> moderation with GSIM:
> 
> - 100Gbps NIC, 32 queues: rx goes from 50 Gbps to 92.8 Gbps (line rate).
> - 200Gbps NIC, 10 VMs (total 160 queues): rx goes from 30 Gbps to 190 Gbps (line rate).
> - 12 SSD, 96 queues: 4K random read goes from 6M to 20.5M IOPS (device max).
> 
> In all cases, with adaptive moderatrion, latency up to p95 is unaffected
> at low/moderate load, even if compared with no moderation at all.
> 
> Changes in v5:
> - refactored the commits based on previous feedback
> - various cleanups
> - conditionally reverted parent IRQ mask/unmask, which would completely
>   defeat the mechanism GSIM is based on.
> 
> Changes in v4:
> - added irqdesc and irqdata flags as suggested by maintainer
> - parameters are only configured via independent procfs entries.
>   No module parameters anymore.
> - merged control and interrupt functions back into a single header/C files
> - applied various annotations (lockdep, data_race())
> - formatting and various renaming as suggested by maintainer.
> - added performance measurements with adaptive moderation.
> - removed the mechanism to conditionally enable moderation at interrupt
>   creation. This can be done in userspace and suitable udev extensions
>   will be handled separately.
> 
> Changes in v3:
> - clearly documented architecture in kernel/irq/irq_moderation.c
>   including how to handle enable/disable/mask, interrupt migration,
>   hotplug and suspend.
> - split implementation in 4 files irq_moderation.[ch] and
>   irq_moderation_hook.[ch] for better separation of control plane and
>   "dataplane" (functions ran on each interrupt)
> - limited scope to handle_edge_irq() and handle_fasteoi_irq() which
>   have been tested on actual hardware.
> - tested on Intel (also with intremap=posted_msi), AMD, ARM, with NIC,
>   nvme, vfio
> 
> Changes in v2:
> - many style fixes (mostly on comments) based on reviewers' comments on v1
> - removed background from Documentation/core-api/irq/irq-moderation.rst
> - split procfs handlers
> - moved internal details to kernel/irq/irq_moderation.h
> - use cpu hotplug for per-CPU setup, removed unnecessary arch-specific changes
> - select suitable irqs based on !irqd_is_level_type(irqd) && irqd_is_single_target(irqd)
> - use a static_key to enable/disable the feature
> 
> 
> 
> Luigi Rizzo (7):
>   genirq: Add flags for software interrupt moderation.
>   genirq: Add GSIM infrastructure
>   genirq: Implement core GSIM moderation logic
>   genirq: Integrate GSIM into interrupt flow
>   genirq: Add GSIM user space configuration (procfs)
>   genirq: Adaptive Global Software Interrupt Moderation (GSIM).
>   PCI/MSI: re-enable conditional parent mask/unmask with sw moderation
> 
>  drivers/irqchip/irq-msi-lib.c |   8 +
>  drivers/pci/msi/irqdomain.c   |  20 +
>  include/linux/irq.h           |  11 +-
>  include/linux/irqdesc.h       |  12 +
>  kernel/irq/Kconfig            |  11 +
>  kernel/irq/Makefile           |   1 +
>  kernel/irq/chip.c             |  14 +
>  kernel/irq/debugfs.c          |   3 +
>  kernel/irq/internals.h        |  20 +
>  kernel/irq/irq_moderation.c   | 959 ++++++++++++++++++++++++++++++++++
>  kernel/irq/irq_moderation.h   | 156 ++++++
>  kernel/irq/irqdesc.c          |   1 +
>  kernel/irq/manage.c           |  10 +
>  kernel/irq/proc.c             |   2 +
>  kernel/irq/settings.h         |  17 +
>  15 files changed, 1244 insertions(+), 1 deletion(-)
>  create mode 100644 kernel/irq/irq_moderation.c
>  create mode 100644 kernel/irq/irq_moderation.h
> 
> -- 
> 2.55.0.737.g08866a6d13-goog
> 
> 
---end quoted text---
Re: [PATCH v5 0/7] Global Software Interrupt Moderation (GSIM)
Posted by Luigi Rizzo 1 month, 1 week ago
On Thu, Aug 20, 2026 at 9:09 AM Christoph Hellwig <hch@infradead.org> wrote:
>
> On Wed, Aug 19, 2026 at 12:43:34PM +0000, Luigi Rizzo wrote:
> > Configuration is easy and robust. System administrators specify the
> > maximum targets (moderation delay; interrupt rate; percentage of time
> > spent in hardirq), and which interrupt sources should be moderated.
> > Independent per-CPU control loops adjust actual delays to try and keep
> > metrics within the targets.
>
> Can we find a way to autodetect and autoenable this?  A magic go faster
> mode that needs very specific tuning is annoying compare to sensible
> defaults.

As mentioned later, the tunings do not need to be specific,
because the control loop adjusts the delay to stay within the target,
and the target is generally just a "don't overload me" value with little
impact on performance.

Of course there are many cases (eg small systems with little I/O) where
moderation is not needed at all so at least one bit of user input is necessary.

What I normally do is use another small patch to pass the initial settings via
module parameters, using values similar to those below
(enable on all interrupts, delay_us=100, target_intr_rate=1000000,
hardirq_percent=70)

> Also how does this interact with adaptive polling code inside drivers
> like NAPI or the upcoming nvme variant?

GSIM acts at hardirq level (so below NAPI, and above device moderation e.g.
the one in ethtool -C ...) and is completely orthogonal to those other
mechanisms.

If NAPI etc manage to keep the interrupt rate/load below the targets,
then the adaptive controller in GSIM ends up using zero additional delay.
Otherwise, it gently adjusts the moderation delay so the combination suffices
to stay within the target.

All the above assumes the parameters are sensible.
GSIM or any HW moderation allows up to 1/delay_us interrupts per second, so
setting delay_us=10 allows up to 100Kintr/s per CPU handling interrupts,
so there is no way we can throttle interrupts below 100K/s)

>
> >
> > The system is adaptive. Moderation affects only latency and only in
> > high load scenarios.  Throughput and CPU efficiencly generally benefits
> > significantly.  Targets don't need to match precisely the platform
> > limits, and one can make conservative and robust choices. Values like
> > delay_us=100, target_intr_rate=1000000, hardirq_percent=70 are a very
> > good starting point.

cheers
luigi
Re: [PATCH v5 0/7] Global Software Interrupt Moderation (GSIM)
Posted by changfengnan 1 month, 1 week ago
> From: "Luigi Rizzo"<lrizzo@google.com>
> Date:  Thu, Aug 20, 2026, 15:35
> Subject:  Re: [PATCH v5 0/7] Global Software Interrupt Moderation (GSIM)
> To: "Christoph Hellwig"<hch@infradead.org>
> Cc: "Thomas Gleixner"<tglx@linutronix.de>, "Marc Zyngier"<maz@kernel.org>, "Luigi Rizzo"<rizzo.unipi@gmail.com>, "Paolo Abeni"<pabeni@redhat.com>, <linux-kernel@vger.kernel.org>, <linux-pci@vger.kernel.org>, "Bjorn Helgaas"<bhelgaas@google.com>, <netdev@vger.kernel.org>, <linux-nvme@lists.infradead.org>, "Fengnan Chang"<changfengnan@bytedance.com>
> On Thu, Aug 20, 2026 at 9:09 AM Christoph Hellwig <hch@infradead.org> wrote:
> >
> > On Wed, Aug 19, 2026 at 12:43:34PM +0000, Luigi Rizzo wrote:
> > > Configuration is easy and robust. System administrators specify the
> > > maximum targets (moderation delay; interrupt rate; percentage of time
> > > spent in hardirq), and which interrupt sources should be moderated.
> > > Independent per-CPU control loops adjust actual delays to try and keep
> > > metrics within the targets.
> >
> > Can we find a way to autodetect and autoenable this?  A magic go faster
> > mode that needs very specific tuning is annoying compare to sensible
> > defaults.
> 
> As mentioned later, the tunings do not need to be specific,
> because the control loop adjusts the delay to stay within the target,
> and the target is generally just a "don't overload me" value with little
> impact on performance.
> 
> Of course there are many cases (eg small systems with little I/O) where
> moderation is not needed at all so at least one bit of user input is necessary.
> 
> What I normally do is use another small patch to pass the initial settings via
> module parameters, using values similar to those below
> (enable on all interrupts, delay_us=100, target_intr_rate=1000000,
> hardirq_percent=70)
> 
> > Also how does this interact with adaptive polling code inside drivers
> > like NAPI or the upcoming nvme variant?
> 
> GSIM acts at hardirq level (so below NAPI, and above device moderation e.g.
> the one in ethtool -C ...) and is completely orthogonal to those other
> mechanisms.

Hi Luigi:
The NVMe variant that Christoph mentioned is this patch of mine.
https://lore.kernel.org/linux-nvme/d9210bcdf73fbe1ac8b6ec132865609a3ed68688.99d43a8b.dfff.41d1.b07d.0592a12473d2@bytedance.com/T/#m561d136611962055641f331e600e719de1e7c573
I took a quick look at your implementation, and I wouldn't say our two
approaches are completely independent.
I’ve run some tests, the test methods, environment, and data are detailed in
this document:
https://docs.google.com/spreadsheets/d/1p3XqVpKgx18QhqTFb0KlJfEW9giWDGSUuoKqCZzdZLg/edit?gid=1100001004#gid=1100001004

it appears that GSIM is only effective in scenarios where multi disks at very high
IOPS; in some cases, there was a noticeable performance regression. 
If there’s something wrong with my configuration, please correct me.

Thanks.

> 
> If NAPI etc manage to keep the interrupt rate/load below the targets,
> then the adaptive controller in GSIM ends up using zero additional delay.
> Otherwise, it gently adjusts the moderation delay so the combination suffices
> to stay within the target.
> 
> All the above assumes the parameters are sensible.
> GSIM or any HW moderation allows up to 1/delay_us interrupts per second, so
> setting delay_us=10 allows up to 100Kintr/s per CPU handling interrupts,
> so there is no way we can throttle interrupts below 100K/s)
> 
> >
> > >
> > > The system is adaptive. Moderation affects only latency and only in
> > > high load scenarios.  Throughput and CPU efficiencly generally benefits
> > > significantly.  Targets don't need to match precisely the platform
> > > limits, and one can make conservative and robust choices. Values like
> > > delay_us=100, target_intr_rate=1000000, hardirq_percent=70 are a very
> > > good starting point.
> 
> cheers
> luigi
> 
Re: [PATCH v5 0/7] Global Software Interrupt Moderation (GSIM)
Posted by Thomas Gleixner 3 weeks, 1 day ago
On Thu, Aug 20 2026 at 19:46, changfengnan@bytedance.com wrote:
>> From: "Luigi Rizzo"<lrizzo@google.com>
>> Date:  Thu, Aug 20, 2026, 15:35
>> Subject:  Re: [PATCH v5 0/7] Global Software Interrupt Moderation (GSIM)
>> To: "Christoph Hellwig"<hch@infradead.org>
>> Cc: "Thomas Gleixner"<tglx@linutronix.de>, "Marc Zyngier"<maz@kernel.org>, "Luigi Rizzo"<rizzo.unipi@gmail.com>, "Paolo Abeni"<pabeni@redhat.com>, <linux-kernel@vger.kernel.org>, <linux-pci@vger.kernel.org>, "Bjorn Helgaas"<bhelgaas@google.com>, <netdev@vger.kernel.org>, <linux-nvme@lists.infradead.org>, "Fengnan Chang"<changfengnan@bytedance.com>
>> On Thu, Aug 20, 2026 at 9:09 AM Christoph Hellwig <hch@infradead.org> wrote:

Can you please fix yuur mail client so it does not copy all that
redundant headers into your reply?

>> > Also how does this interact with adaptive polling code inside drivers
>> > like NAPI or the upcoming nvme variant?
>> 
>> GSIM acts at hardirq level (so below NAPI, and above device moderation e.g.
>> the one in ethtool -C ...) and is completely orthogonal to those other
>> mechanisms.
>
> Hi Luigi:
> The NVMe variant that Christoph mentioned is this patch of mine.
> https://lore.kernel.org/linux-nvme/d9210bcdf73fbe1ac8b6ec132865609a3ed68688.99d43a8b.dfff.41d1.b07d.0592a12473d2@bytedance.com/T/#m561d136611962055641f331e600e719de1e7c573
> I took a quick look at your implementation, and I wouldn't say our two
> approaches are completely independent.
> I’ve run some tests, the test methods, environment, and data are detailed in
> this document:
> https://docs.google.com/spreadsheets/d/1p3XqVpKgx18QhqTFb0KlJfEW9giWDGSUuoKqCZzdZLg/edit?gid=1100001004#gid=1100001004
>
> it appears that GSIM is only effective in scenarios where multi disks at very high
> IOPS; in some cases, there was a noticeable performance regression. 
> If there’s something wrong with my configuration, please correct me.

So we have a NVME specific mechanism to tackle the same problem and a
more generic version which is subsystem "independent".

Can you folks please coordinate and get your act together so that we
don't end up with two competing mechanisms which make things worse than
they are now.

TBH. I despise the NVME is special approach because it's fricking
obvious that this is _NOT_ a NVME specific issue. But sure NVME is
special as all other subsystems are special.

Q: When do driver folks actually start to look beyond the brim of their
   tea cup?

A: Probably never ...

Thanks,

        tglx
Re: [PATCH v5 0/7] Global Software Interrupt Moderation (GSIM)
Posted by Christoph Hellwig 2 weeks, 5 days ago
On Sat, Sep 05, 2026 at 10:30:22PM +0200, Thomas Gleixner wrote:
> > it appears that GSIM is only effective in scenarios where multi disks at very high
> > IOPS; in some cases, there was a noticeable performance regression. 
> > If there’s something wrong with my configuration, please correct me.
> 
> So we have a NVME specific mechanism to tackle the same problem and a
> more generic version which is subsystem "independent".

I'm not sure they tackle the entirely same problem, although they are
very related.

> Can you folks please coordinate and get your act together so that we
> don't end up with two competing mechanisms which make things worse than
> they are now.

This is what I'm trying to get done here.  This is the first time I've
seen GSIM as I still try to read lkml, although I usuall fail.
Unfortunately neither the nvme nor block lists were Cced on it,
despite most of the numbers involving NVMe.

> TBH. I despise the NVME is special approach because it's fricking
> obvious that this is _NOT_ a NVME specific issue. But sure NVME is
> special as all other subsystems are special.

Note that we tried to look into generic helpers, Keith tried various
versions using DIMLIB, and we've also considered doing more work in the
block core similar what networking does with NAPI.  But we're always
interested common code if it works, glad someone is looking.  Although
somewhat more productive suggestions would be helpful.

Re: [PATCH v5 0/7] Global Software Interrupt Moderation (GSIM)
Posted by Luigi Rizzo 2 weeks, 5 days ago
On Wed, Sep 9, 2026 at 8:25 AM Christoph Hellwig <hch@infradead.org> wrote:
>
> On Sat, Sep 05, 2026 at 10:30:22PM +0200, Thomas Gleixner wrote:
> > > it appears that GSIM is only effective in scenarios where multi disks at very high
> > > IOPS; in some cases, there was a noticeable performance regression.
> > > If there’s something wrong with my configuration, please correct me.
> >
> > So we have a NVME specific mechanism to tackle the same problem and a
> > more generic version which is subsystem "independent".
>
> I'm not sure they tackle the entirely same problem, although they are
> very related.

There is overlap but it is accidental.
The difference between the two mechanism is the following:

- GSIM addresses hardware limitations (SoCs that create huge PCIe
  backpressure on each MSIx interrupt, eventually reducing the available
  bandwidth too much, regardless of how many CPUs can process interrupts).
  The goal here is to throttle the total MSIx rate, blocking them at
  the source (PCIe device) rather than later in the interrupt controller.

- the NVME mechanism (and NAPI) aim to amortize interrupt processing
  overhead. It does so by recreating some fine grained single-source
  interrupt moderation (something that NICs do in hardware, but NVME
  does not, with 100us being way too coarse for modern SSDs) and an
  adaptive scheme to adjust the delay and decide when to kick in.

I think there is a reason for both.

GSIM because it is more general and addresses an issue the others cannot handle
the subsystem-specific scheme because they can be more efficient
(eg they can hook into existing helpers like the threaded irq handler,
call directly the handler without going through the full irq chain,
use more targeted heuristics to decide when to kick in).

cheers
luigi
Re: [PATCH v5 0/7] Global Software Interrupt Moderation (GSIM)
Posted by Fengnan 2 weeks, 5 days ago
在 2026/9/9 16:06, Luigi Rizzo 写道:
> On Wed, Sep 9, 2026 at 8:25 AM Christoph Hellwig <hch@infradead.org> wrote:
>> On Sat, Sep 05, 2026 at 10:30:22PM +0200, Thomas Gleixner wrote:
>>>> it appears that GSIM is only effective in scenarios where multi disks at very high
>>>> IOPS; in some cases, there was a noticeable performance regression.
>>>> If there’s something wrong with my configuration, please correct me.
>>> So we have a NVME specific mechanism to tackle the same problem and a
>>> more generic version which is subsystem "independent".
>> I'm not sure they tackle the entirely same problem, although they are
>> very related.
> There is overlap but it is accidental.
> The difference between the two mechanism is the following:
>
> - GSIM addresses hardware limitations (SoCs that create huge PCIe
>    backpressure on each MSIx interrupt, eventually reducing the available
>    bandwidth too much, regardless of how many CPUs can process interrupts).
>    The goal here is to throttle the total MSIx rate, blocking them at
>    the source (PCIe device) rather than later in the interrupt controller.
>
> - the NVME mechanism (and NAPI) aim to amortize interrupt processing
>    overhead. It does so by recreating some fine grained single-source
>    interrupt moderation (something that NICs do in hardware, but NVME
>    does not, with 100us being way too coarse for modern SSDs) and an
>    adaptive scheme to adjust the delay and decide when to kick in.
That's a great summary.

As for generic solutions, I've tried several approaches before, but none 
of them
could achieve the same or close results as NVMe adaptive interrupt polling.
I'm still experimenting with different options.


>
> I think there is a reason for both.
>
> GSIM because it is more general and addresses an issue the others cannot handle
> the subsystem-specific scheme because they can be more efficient
> (eg they can hook into existing helpers like the threaded irq handler,
> call directly the handler without going through the full irq chain,
> use more targeted heuristics to decide when to kick in).
>
> cheers
> luigi