[v1] net: stmmac: xgmac2: disable RBUE in default RX interrupt mask

[PATCH] net: stmmac: xgmac2: disable RBUE in default RX interrupt mask

Posted by muhammad.nazim.amirul.nazle.asmade@altera.com 1 week, 3 days ago

From: Nazim Amirul <muhammad.nazim.amirul.nazle.asmade@altera.com>

Enabling the RX Buffer Unavailable (RBUE) interrupt is counterproductive
and can trigger a MAC interrupt storm under heavy RX pressure. When the
DMA runs out of RX descriptors it fires RBUE continuously until software
refills the ring.

However, RBUE is redundant: the normal RX completion interrupt (RIE)
already triggers NAPI, which processes completed descriptors and refills
the ring, causing the DMA to resume. The RBUE handler itself only sets
handle_rx - the same outcome as RIE.

Remove RBUE from XGMAC_DMA_INT_DEFAULT_EN and XGMAC_DMA_INT_DEFAULT_RX
to prevent the interrupt storm while keeping normal RX handling intact.

Fixes: d6ddfacd95c7 ("net: stmmac: Add DMA related callbacks for XGMAC2")
Signed-off-by: Nazim Amirul <muhammad.nazim.amirul.nazle.asmade@altera.com>
---
 drivers/net/ethernet/stmicro/stmmac/dwxgmac2.h | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/drivers/net/ethernet/stmicro/stmmac/dwxgmac2.h b/drivers/net/ethernet/stmicro/stmmac/dwxgmac2.h
index 51943705a2b0..b5fdc253a2f2 100644
--- a/drivers/net/ethernet/stmicro/stmmac/dwxgmac2.h
+++ b/drivers/net/ethernet/stmicro/stmmac/dwxgmac2.h
@@ -379,9 +379,9 @@
 #define XGMAC_RIE			BIT(6)
 #define XGMAC_TBUE			BIT(2)
 #define XGMAC_TIE			BIT(0)
-#define XGMAC_DMA_INT_DEFAULT_EN	(XGMAC_NIE | XGMAC_AIE | XGMAC_RBUE | \
+#define XGMAC_DMA_INT_DEFAULT_EN	(XGMAC_NIE | XGMAC_AIE | \
 					XGMAC_RIE | XGMAC_TIE)
-#define XGMAC_DMA_INT_DEFAULT_RX	(XGMAC_RBUE | XGMAC_RIE)
+#define XGMAC_DMA_INT_DEFAULT_RX	(XGMAC_RIE)
 #define XGMAC_DMA_INT_DEFAULT_TX	(XGMAC_TIE)
 #define XGMAC_DMA_CH_Rx_WATCHDOG(x)	(0x0000313c + (0x80 * (x)))
 #define XGMAC_RWT			GENMASK(7, 0)
-- 
2.43.7

Re: [PATCH] net: stmmac: xgmac2: disable RBUE in default RX interrupt mask

Posted by Paolo Abeni 4 days, 3 hours ago

From: AI Reviewer <ai@example.com>

This is an AI-generated review of your patch. The human sending this
email has considered the AI review valid, or at least plausible.
Full review at: https://sashiko.dev/
---
net: stmmac: xgmac2: disable RBUE in default RX interrupt mask

This commit removes the Receive Buffer Unavailable (RBUE) interrupt from
the default RX and DMA interrupt masks. The goal is to prevent MAC
interrupt storms when the DMA runs out of descriptors under heavy
RX pressure.

> diff --git a/drivers/net/ethernet/stmicro/stmmac/dwxgmac2.h b/drivers/net/ethernet/stmicro/stmmac/dwxgmac2.h
> index 51943705a2b03..b5fdc253a2f24 100644
> --- a/drivers/net/ethernet/stmicro/stmmac/dwxgmac2.h
> +++ b/drivers/net/ethernet/stmicro/stmmac/dwxgmac2.h
[ ... ]
> @@ -379,9 +379,9 @@
>  #define XGMAC_RIE			BIT(6)
>  #define XGMAC_TBUE			BIT(2)
>  #define XGMAC_TIE			BIT(0)
> -#define XGMAC_DMA_INT_DEFAULT_EN	(XGMAC_NIE | XGMAC_AIE | XGMAC_RBUE | \
> +#define XGMAC_DMA_INT_DEFAULT_EN	(XGMAC_NIE | XGMAC_AIE | \
>  					XGMAC_RIE | XGMAC_TIE)

[Severity: Medium]
Does removing XGMAC_RBUE from XGMAC_DMA_INT_DEFAULT_EN prevent the
XGMAC_RBU status bit from being cleared in dwxgmac2_dma_interrupt()?

If this default mask is used to clear pending interrupts, writing a 0 to
the XGMAC_RBU position would fail to clear it. Would this cause the bit
to latch indefinitely, triggering redundant NAPI schedules on unrelated
abnormal interrupts?

> -#define XGMAC_DMA_INT_DEFAULT_RX	(XGMAC_RBUE | XGMAC_RIE)
> +#define XGMAC_DMA_INT_DEFAULT_RX	(XGMAC_RIE)

[Severity: High]
Does disabling RBUE entirely break RX out-of-memory recovery?

If the RX ring becomes completely empty during memory pressure because
stmmac_rx_refill() fails to allocate SKBs, the MAC DMA will suspend. Once
a new packet arrives, the MAC drops it and sets the XGMAC_RBU status.

Since RIE is only asserted when a packet is successfully received and
transferred to a descriptor, and RBUE is now disabled, it appears no
interrupt will fire at all in this state. Could this leave NAPI
unscheduled and result in a permanent RX path lockup even after memory
becomes available again?
-- 
This is an AI-generated review.

Re: [PATCH] net: stmmac: xgmac2: disable RBUE in default RX interrupt mask

Posted by Maxime Chevallier 1 week, 2 days ago

Hi,

On 5/29/26 09:17, muhammad.nazim.amirul.nazle.asmade@altera.com wrote:
> From: Nazim Amirul <muhammad.nazim.amirul.nazle.asmade@altera.com>
> 
> Enabling the RX Buffer Unavailable (RBUE) interrupt is counterproductive
> and can trigger a MAC interrupt storm under heavy RX pressure. When the
> DMA runs out of RX descriptors it fires RBUE continuously until software
> refills the ring.
> 
> However, RBUE is redundant: the normal RX completion interrupt (RIE)
> already triggers NAPI, which processes completed descriptors and refills
> the ring, causing the DMA to resume. The RBUE handler itself only sets
> handle_rx - the same outcome as RIE.
> 
> Remove RBUE from XGMAC_DMA_INT_DEFAULT_EN and XGMAC_DMA_INT_DEFAULT_RX
> to prevent the interrupt storm while keeping normal RX handling intact.
> 
> Fixes: d6ddfacd95c7 ("net: stmmac: Add DMA related callbacks for XGMAC2")
> Signed-off-by: Nazim Amirul <muhammad.nazim.amirul.nazle.asmade@altera.com>

We had an opposite discussion about enabling rx buf unavailable
interrupts for other DWMAC versions :

https://lore.kernel.org/netdev/E1wBBaR-0000000GZHR-1dbM@rmk-PC.armlinux.org.uk/

While your patch makes sense, I think it's important to note that it'll
now make the "rx_buf_unav_irq" stat (that we can query from userspace)
stuck at 0 now.

That's a user-visible behaviour change, for a stat that can be useful,
one could consider that a uAPI change...

Do you have some numbers on the perf impact you see when buffer
starvation hits to see what we are gaining/losing here ?

Maxime

Re: [PATCH] net: stmmac: xgmac2: disable RBUE in default RX interrupt mask

Posted by Nazle Asmade, Muhammad Nazim Amirul 4 days, 3 hours ago

On 30/5/2026 3:19 am, Maxime Chevallier wrote:
> Hi,
> 
> On 5/29/26 09:17, muhammad.nazim.amirul.nazle.asmade@altera.com wrote:
>> From: Nazim Amirul <muhammad.nazim.amirul.nazle.asmade@altera.com>
>>
>> Enabling the RX Buffer Unavailable (RBUE) interrupt is counterproductive
>> and can trigger a MAC interrupt storm under heavy RX pressure. When the
>> DMA runs out of RX descriptors it fires RBUE continuously until software
>> refills the ring.
>>
>> However, RBUE is redundant: the normal RX completion interrupt (RIE)
>> already triggers NAPI, which processes completed descriptors and refills
>> the ring, causing the DMA to resume. The RBUE handler itself only sets
>> handle_rx - the same outcome as RIE.
>>
>> Remove RBUE from XGMAC_DMA_INT_DEFAULT_EN and XGMAC_DMA_INT_DEFAULT_RX
>> to prevent the interrupt storm while keeping normal RX handling intact.
>>
>> Fixes: d6ddfacd95c7 ("net: stmmac: Add DMA related callbacks for XGMAC2")
>> Signed-off-by: Nazim Amirul 
>> <muhammad.nazim.amirul.nazle.asmade@altera.com>
> 
> We had an opposite discussion about enabling rx buf unavailable
> interrupts for other DWMAC versions :
> 
> https://nam10.safelinks.protection.outlook.com/? 
> url=https%3A%2F%2Flore.kernel.org%2Fnetdev%2FE1wBBaR-0000000GZHR-1dbM%40rmk-PC.armlinux.org.uk%2F&data=05%7C02%7Cmuhammad.nazim.amirul.nazle.asmade%40altera.com%7Cc3e0a35946944b7cebc508debdb729d1%7Cfbd72e03d4a54110adce614d51f2077a%7C0%7C0%7C639156791573135216%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=zgOd9DakLAP%2BFIOGSLYxSvXejHxFae%2BhnktdH7T1UUs%3D&reserved=0
> 
> While your patch makes sense, I think it's important to note that it'll
> now make the "rx_buf_unav_irq" stat (that we can query from userspace)
> stuck at 0 now.
> 
> That's a user-visible behaviour change, for a stat that can be useful,
> one could consider that a uAPI change...
> 
> Do you have some numbers on the perf impact you see when buffer
> starvation hits to see what we are gaining/losing here ?
> 
> Maxime
> 
Hi Maxime,

Thank you for the review and for pointing out the rx_buf_unav_irq concern.

We do have interrupt storm data to share. On Agilex5, under heavy RX 
pressure we observed:

Under heavy RX pressure we observed eth0:mac firing 1,821,811,555 times 
vs 2,618,627 actual RX completions on eth0:rx-0 — a ~695x ratio.

BR,
Nazim

Re: [PATCH] net: stmmac: xgmac2: disable RBUE in default RX interrupt mask

Posted by Maxime Chevallier 4 days, 3 hours ago

Hi,

>> to note that it'll
>> now make the "rx_buf_unav_irq" stat (that we can query from userspace)
>> stuck at 0 now.
>>
>> That's a user-visible behaviour change, for a stat that can be useful,
>> one could consider that a uAPI change...
>>
>> Do you have some numbers on the perf impact you see when buffer
>> starvation hits to see what we are gaining/losing here ?
>>
>> Maxime
>>
> Hi Maxime,
> 
> Thank you for the review and for pointing out the rx_buf_unav_irq concern.
> 
> We do have interrupt storm data to share. On Agilex5, under heavy RX 
> pressure we observed:
> 
> Under heavy RX pressure we observed eth0:mac firing 1,821,811,555 times 
> vs 2,618,627 actual RX completions on eth0:rx-0 — a ~695x ratio.

Ouch indeed. The stat itself has to stay, but the behaviour of that
counter is already different depending on the DWMAC core version.

I think we can get this patch in, but please state in the commit message
the performance measurements you're mentioning, as well as the fact
that the counter will now always show at 0 on XGMAC2 devices.

Maybe we'll eventually need some switch to toggle these counters, as it's
some valuable information when debugging, but has a significant impact on
speeds.

With the commit log updates,

Reviewed-by: Maxime Chevallier <maxime.chevallier@bootlin.com>

> 
> BR,
> Nazim

Re: [PATCH] net: stmmac: xgmac2: disable RBUE in default RX interrupt mask

Posted by Nazle Asmade, Muhammad Nazim Amirul 4 days, 2 hours ago

On 4/6/2026 5:08 pm, Maxime Chevallier wrote:
> Hi,
> 
>>> to note that it'll
>>> now make the "rx_buf_unav_irq" stat (that we can query from userspace)
>>> stuck at 0 now.
>>>
>>> That's a user-visible behaviour change, for a stat that can be useful,
>>> one could consider that a uAPI change...
>>>
>>> Do you have some numbers on the perf impact you see when buffer
>>> starvation hits to see what we are gaining/losing here ?
>>>
>>> Maxime
>>>
>> Hi Maxime,
>>
>> Thank you for the review and for pointing out the rx_buf_unav_irq concern.
>>
>> We do have interrupt storm data to share. On Agilex5, under heavy RX
>> pressure we observed:
>>
>> Under heavy RX pressure we observed eth0:mac firing 1,821,811,555 times
>> vs 2,618,627 actual RX completions on eth0:rx-0 — a ~695x ratio.
> 
> Ouch indeed. The stat itself has to stay, but the behaviour of that
> counter is already different depending on the DWMAC core version.
> 
> I think we can get this patch in, but please state in the commit message
> the performance measurements you're mentioning, as well as the fact
> that the counter will now always show at 0 on XGMAC2 devices.
> 
> Maybe we'll eventually need some switch to toggle these counters, as it's
> some valuable information when debugging, but has a significant impact on
> speeds.
> 
> With the commit log updates,
> 
> Reviewed-by: Maxime Chevallier <maxime.chevallier@bootlin.com>
> 
>>
>> BR,
>> Nazim
> 
Hi Maxime,

Thanks for your review! Commit message now updated in v2
https://lore.kernel.org/all/20260604094512.21243-1-muhammad.nazim.amirul.nazle.asmade@altera.com/

BR,
Nazim