drivers/pci: Allow attaching AER to non-RP devices that support MSI

[PATCH] drivers/pci: Allow attaching AER to non-RP devices that support MSI

Posted by Darshit Shah 2 months, 1 week ago

Previously portdrv tried to prevent non-Root Port (RP) and non-Root
Complex Event Collector (RCEC) devices from enabling AER capability.
This was done because some switches enable AER but do not support MSI.
Hence, trying to initialize the AER IRQ for such devices would fail and
Linux would fail to claim the switch port entirely.

However, it is possible to have switches upstream of an endpoint that
support MSI and AER. Without AER capability being enabled on such
a switch, portdrv will refuse to enable the DPC capability as well,
preventing a PCIe error on an endpoint from being handled by the switch.

Allow enabling the AER service on non-RP, non-RCEC devices if they still
support both AER and MSI. This allows switches upstream of an endpoint
to generate and handle DPC events.
Fixes: d8d2b65a940b ("PCI/portdrv: Allow AER service only for Root Ports & RCECs")
Signed-off-by: Darshit Shah <darnshah@amazon.de>
---
 drivers/pci/pcie/portdrv.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/drivers/pci/pcie/portdrv.c b/drivers/pci/pcie/portdrv.c
index d1b68c18444f..41326bbcd295 100644
--- a/drivers/pci/pcie/portdrv.c
+++ b/drivers/pci/pcie/portdrv.c
@@ -237,8 +237,8 @@ static int get_port_device_capability(struct pci_dev *dev)
 	}
 
 #ifdef CONFIG_PCIEAER
-	if ((pci_pcie_type(dev) == PCI_EXP_TYPE_ROOT_PORT ||
-             pci_pcie_type(dev) == PCI_EXP_TYPE_RC_EC) &&
+	if ((dev->msi_cap || pci_pcie_type(dev) == PCI_EXP_TYPE_ROOT_PORT ||
+	     pci_pcie_type(dev) == PCI_EXP_TYPE_RC_EC) &&
 	    dev->aer_cap && pci_aer_available() &&
 	    (pcie_ports_native || host->native_aer))
 		services |= PCIE_PORT_SERVICE_AER;
-- 
2.47.3




Amazon Web Services Development Center Germany GmbH
Tamara-Danz-Str. 13
10243 Berlin
Geschaeftsfuehrung: Christian Schlaeger, Christof Hellmis
Eingetragen am Amtsgericht Charlottenburg unter HRB 257764 B
Sitz: Berlin
Ust-ID: DE 365 538 597

Re: [PATCH] drivers/pci: Allow attaching AER to non-RP devices that support MSI

Posted by Bjorn Helgaas 2 months ago

On Fri, Nov 28, 2025 at 12:20:53PM +0000, Darshit Shah wrote:
> Previously portdrv tried to prevent non-Root Port (RP) and non-Root
> Complex Event Collector (RCEC) devices from enabling AER capability.
> This was done because some switches enable AER but do not support MSI.
> Hence, trying to initialize the AER IRQ for such devices would fail and
> Linux would fail to claim the switch port entirely.

The d8d2b65a940b commit log could have been clearer.

Some switches advertise an AER capability, but they can't generate AER
interrupts, regardless of whether they support MSI.  Only RPs and
RCECs can generate AER interrupts.

> However, it is possible to have switches upstream of an endpoint that
> support MSI and AER. Without AER capability being enabled on such
> a switch, portdrv will refuse to enable the DPC capability as well,
> preventing a PCIe error on an endpoint from being handled by the switch.
> 
> Allow enabling the AER service on non-RP, non-RCEC devices if they still
> support both AER and MSI. This allows switches upstream of an endpoint
> to generate and handle DPC events.
> Fixes: d8d2b65a940b ("PCI/portdrv: Allow AER service only for Root Ports & RCECs")
> Signed-off-by: Darshit Shah <darnshah@amazon.de>
> ---
>  drivers/pci/pcie/portdrv.c | 4 ++--
>  1 file changed, 2 insertions(+), 2 deletions(-)
> 
> diff --git a/drivers/pci/pcie/portdrv.c b/drivers/pci/pcie/portdrv.c
> index d1b68c18444f..41326bbcd295 100644
> --- a/drivers/pci/pcie/portdrv.c
> +++ b/drivers/pci/pcie/portdrv.c
> @@ -237,8 +237,8 @@ static int get_port_device_capability(struct pci_dev *dev)
>  	}
>  
>  #ifdef CONFIG_PCIEAER
> -	if ((pci_pcie_type(dev) == PCI_EXP_TYPE_ROOT_PORT ||
> -             pci_pcie_type(dev) == PCI_EXP_TYPE_RC_EC) &&
> +	if ((dev->msi_cap || pci_pcie_type(dev) == PCI_EXP_TYPE_ROOT_PORT ||
> +	     pci_pcie_type(dev) == PCI_EXP_TYPE_RC_EC) &&
>  	    dev->aer_cap && pci_aer_available() &&
>  	    (pcie_ports_native || host->native_aer))
>  		services |= PCIE_PORT_SERVICE_AER;
> -- 
> 2.47.3
> 
> 
> 
> 
> Amazon Web Services Development Center Germany GmbH
> Tamara-Danz-Str. 13
> 10243 Berlin
> Geschaeftsfuehrung: Christian Schlaeger, Christof Hellmis
> Eingetragen am Amtsgericht Charlottenburg unter HRB 257764 B
> Sitz: Berlin
> Ust-ID: DE 365 538 597
>

Re: [PATCH] drivers/pci: Allow attaching AER to non-RP devices that support MSI

Posted by Lukas Wunner 2 months, 1 week ago

On Fri, Nov 28, 2025 at 12:20:53PM +0000, Darshit Shah wrote:
> Previously portdrv tried to prevent non-Root Port (RP) and non-Root
> Complex Event Collector (RCEC) devices from enabling AER capability.
> This was done because some switches enable AER but do not support MSI.

The AER driver only binds to RPs and RCECs, see aer_probe():

	if ((pci_pcie_type(port) != PCI_EXP_TYPE_RC_EC) &&
	    (pci_pcie_type(port) != PCI_EXP_TYPE_ROOT_PORT))
		return -ENODEV;

So there's no point in adding PCIE_PORT_SERVICE_AER to "services"
for other port types (as your patch does).

> However, it is possible to have switches upstream of an endpoint that
> support MSI and AER. Without AER capability being enabled on such
> a switch, portdrv will refuse to enable the DPC capability as well,
> preventing a PCIe error on an endpoint from being handled by the switch.

I assume you're referring to this clause in get_port_device_capability():

	if (pci_find_ext_capability(dev, PCI_EXT_CAP_ID_DPC) &&
	    pci_aer_available() &&
	    (pcie_ports_dpc_native || (services & PCIE_PORT_SERVICE_AER)))
		services |= PCIE_PORT_SERVICE_DPC;

Presumably on your system, BIOS doesn't grant AER handling to the OS
upon _OSC negotiation?  Is there a BIOS knob to change that?
Alternatively, does passing "pcie_ports=dpc-native" fix the issue?
If it does, why do you need the patch instead of using the command line
option?

> Allow enabling the AER service on non-RP, non-RCEC devices if they still
> support both AER and MSI. This allows switches upstream of an endpoint
> to generate and handle DPC events.

Per PCIe r7.0 sec 7.8.4.9, regarding the Root Error Command Register:

   "For Functions other than Root Ports and Root Complex Event Collectors:
   when End-End TLP Prefix Supported is Set or Flit Mode Supported is Set,
   this register is RsvdP, otherwise [...] this register is not required
   to be implemented."

Hence we can't enable AER handling on anything else than RPs and RCECs.

Thanks,

Lukas

[PATCH v2 0/1] Re: [PATCH] drivers/pci: Allow attaching AER to non-RP devices that support MSI

Posted by Darshit Shah 2 months ago

On 28.11.25 18:07, Lukas Wunner wrote:
> On Fri, Nov 28, 2025 at 12:20:53PM +0000, Darshit Shah wrote:
>> Previously portdrv tried to prevent non-Root Port (RP) and non-Root
>> Complex Event Collector (RCEC) devices from enabling AER capability.
>> This was done because some switches enable AER but do not support MSI.
> 
> The AER driver only binds to RPs and RCECs, see aer_probe():
> 
> 	if ((pci_pcie_type(port) != PCI_EXP_TYPE_RC_EC) &&
> 	    (pci_pcie_type(port) != PCI_EXP_TYPE_ROOT_PORT))
> 		return -ENODEV;
> 
> So there's no point in adding PCIE_PORT_SERVICE_AER to "services"
> for other port types (as your patch does).

I agree that adding PCIE_PORT_SERVICE_AER to "services" for switches is
not useful.

> 
>> However, it is possible to have switches upstream of an endpoint that
>> support MSI and AER. Without AER capability being enabled on such
>> a switch, portdrv will refuse to enable the DPC capability as well,
>> preventing a PCIe error on an endpoint from being handled by the switch.
> 
> I assume you're referring to this clause in get_port_device_capability():
> 
> 	if (pci_find_ext_capability(dev, PCI_EXT_CAP_ID_DPC) &&
> 	    pci_aer_available() &&
> 	    (pcie_ports_dpc_native || (services & PCIE_PORT_SERVICE_AER)))
> 		services |= PCIE_PORT_SERVICE_DPC;
> 
> Presumably on your system, BIOS doesn't grant AER handling to the OS
> upon _OSC negotiation?  Is there a BIOS knob to change that?

On my system, BIOS does grant AER handling to the OS with _OSC negotiation.

2025-12-06 23:15:40.172000 kern INFO [    0.590601] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]
2025-12-06 23:15:40.172000 kern INFO [    0.590607] acpi PNP0A08:00: _OSC: platform does not support [LTR]
2025-12-06 23:15:40.172000 kern INFO [    0.590610] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]

> Alternatively, does passing "pcie_ports=dpc-native" fix the issue?
> If it does, why do you need the patch instead of using the command line
> option?

Given that my firmware correctly negotiates and hands over control of AER
to Linux, we should not need to use a quirk to enable DPC support.

What seems to be happening instead is that portdrv has misinterpreted the
Implementation Note in PCIe r5.0 6.2.10 that states:
  "It is recommended that platform firmware and operating systems always link
  the control of DPC to the control of Advanced Error Reporting"

This does not mean that AER is working on each PCIe device that wants to enable
DPC. Rather that the OS is considered to have control over DPC, if it also has
control over AER for the host bridge upstream of the device.

Thus, I claim that the following checks are incorrect:

	if (pci_find_ext_capability(dev, PCI_EXT_CAP_ID_DPC) &&
	    pci_aer_available() &&
	    (pcie_ports_dpc_native || (services & PCIE_PORT_SERVICE_AER)))
		services |= PCIE_PORT_SERVICE_DPC;

PCIE_PORT_SERVICE_AER will only ever be enabled on RP and RCEC devices.
However, PCIE_PORT_SERVICE_DPC should be allowed on non-RP devices. In fact,
that is exactly where it is needed, since the switch upstream of an endpoint
device is required to generate the DPC signal. A fixed version of this check
would instead look like:

	if (pci_find_ext_capability(dev, PCI_EXT_CAP_ID_DPC) &&
	    pci_aer_available() &&
	    (pcie_ports_dpc_native || host->native_aer)
		services |= PCIE_PORT_SERVICE_DPC;

That is, we should allow binding PCIE_PORT_SERVICE_DPC to a device, if:

* The device advertises the DPC capability
* AER has not been disabled for Linux through the command line
* Linux has control of AER (and DPC) for the host bridge above the device
  * _OR_ `pcie_ports=dpc-native` was used on the command line

Coming next is an updated patch based on this.

-- 
2.47.3

Amazon Web Services Development Center Germany GmbH
Tamara-Danz-Str. 13
10243 Berlin
Geschaeftsfuehrung: Christian Schlaeger, Christof Hellmis
Eingetragen am Amtsgericht Charlottenburg unter HRB 257764 B
Sitz: Berlin
Ust-ID: DE 365 538 597

[PATCH v2 1/1] drivers/pci: Decouple DPC from AER service

Posted by Darshit Shah 2 months ago

According to [1] it is recommended that the Operating System link the
enablement of Downstream Port Containment (DPC) to the enablement of
Advanced Error Reporting (AER).

However, AER is advertised only on Root Port (RP) or Root Complex Event
Collector (RCEC) devices. On the other hand, DPC may be advertised on
any PCIe device in the hierarchy. In fact, since the main usecase of DPC
is for the switch upstream of an endpoint device to trigger a signal to
the host-bridge, it is imperative that it be supported on non-RP,
non-RCEC devices.

Previously portdrv has interpreted [1] to mean that the AER service must
be available on the same device in order for DPC to be available. This is
not what the implementation note in [1] meant to imply. If the firmware
hands the OS control of AER via _OSC on the host bridge upstream of the
device, then the OS should be allowed to assume control of DPC on that device.

The comment above this check alludes to this, by saying:
 > With dpc-native, allow Linux to use DPC even if it doesn't have permission
 > to use AER.
However, permission to use AER is negotiated at the host bridge, not
per-device. So we should not link DPC to enabling AER at the device.
Instead, DPC should be enabled if the OS has control of AER for the
host bridge that is upstream of the device in question, or if dpc-native
was set on the command line.

[1]: PCI Express Base Specification Revision 5.0 Version 1.0, Sec. 6.2.10

Signed-off-by: Darshit Shah <darnshah@amazon.de>
---
Changes in patch v2:
  * Don't touch the PCIE_PORT_SERVICE_AER attachment
  * Stop relying on PCIE_PORT_SERVICE_AER for enabling DPC
  * Instead test that OS has control of AER at parent host bridge

 drivers/pci/pcie/portdrv.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/pci/pcie/portdrv.c b/drivers/pci/pcie/portdrv.c
index 38a41ccf79b9..8db2fa140ae2 100644
--- a/drivers/pci/pcie/portdrv.c
+++ b/drivers/pci/pcie/portdrv.c
@@ -264,7 +264,7 @@ static int get_port_device_capability(struct pci_dev *dev)
 	 */
 	if (pci_find_ext_capability(dev, PCI_EXT_CAP_ID_DPC) &&
 	    pci_aer_available() &&
-	    (pcie_ports_dpc_native || (services & PCIE_PORT_SERVICE_AER)))
+	    (host->native_aer || pcie_ports_dpc_native))
 		services |= PCIE_PORT_SERVICE_DPC;
 
 	/* Enable bandwidth control if more than one speed is supported. */
-- 
2.47.3




Amazon Web Services Development Center Germany GmbH
Tamara-Danz-Str. 13
10243 Berlin
Geschaeftsfuehrung: Christian Schlaeger, Christof Hellmis
Eingetragen am Amtsgericht Charlottenburg unter HRB 257764 B
Sitz: Berlin
Ust-ID: DE 365 538 597

Re: [PATCH v2 1/1] drivers/pci: Decouple DPC from AER service

Posted by Sathyanarayanan Kuppuswamy 1 month, 4 weeks ago

Hi,

On 12/8/25 3:25 AM, Darshit Shah wrote:
> According to [1] it is recommended that the Operating System link the
> enablement of Downstream Port Containment (DPC) to the enablement of
> Advanced Error Reporting (AER).
>
> However, AER is advertised only on Root Port (RP) or Root Complex Event
> Collector (RCEC) devices. On the other hand, DPC may be advertised on
> any PCIe device in the hierarchy. In fact, since the main usecase of DPC
> is for the switch upstream of an endpoint device to trigger a signal to
> the host-bridge, it is imperative that it be supported on non-RP,
> non-RCEC devices.
>
> Previously portdrv has interpreted [1] to mean that the AER service must
> be available on the same device in order for DPC to be available. This is
> not what the implementation note in [1] meant to imply. If the firmware
> hands the OS control of AER via _OSC on the host bridge upstream of the
> device, then the OS should be allowed to assume control of DPC on that device.
>
> The comment above this check alludes to this, by saying:
>   > With dpc-native, allow Linux to use DPC even if it doesn't have permission
>   > to use AER.
> However, permission to use AER is negotiated at the host bridge, not
> per-device. So we should not link DPC to enabling AER at the device.
> Instead, DPC should be enabled if the OS has control of AER for the
> host bridge that is upstream of the device in question, or if dpc-native
> was set on the command line.
>
> [1]: PCI Express Base Specification Revision 5.0 Version 1.0, Sec. 6.2.10
>
> Signed-off-by: Darshit Shah <darnshah@amazon.de>
> ---

Looks fine to me.

Reviewed-by: Kuppuswamy Sathyanarayanan <sathyanarayanan.kuppuswamy@linux.intel.com>

> Changes in patch v2:
>    * Don't touch the PCIE_PORT_SERVICE_AER attachment
>    * Stop relying on PCIE_PORT_SERVICE_AER for enabling DPC
>    * Instead test that OS has control of AER at parent host bridge
>
>   drivers/pci/pcie/portdrv.c | 2 +-
>   1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/drivers/pci/pcie/portdrv.c b/drivers/pci/pcie/portdrv.c
> index 38a41ccf79b9..8db2fa140ae2 100644
> --- a/drivers/pci/pcie/portdrv.c
> +++ b/drivers/pci/pcie/portdrv.c
> @@ -264,7 +264,7 @@ static int get_port_device_capability(struct pci_dev *dev)
>   	 */
>   	if (pci_find_ext_capability(dev, PCI_EXT_CAP_ID_DPC) &&
>   	    pci_aer_available() &&
> -	    (pcie_ports_dpc_native || (services & PCIE_PORT_SERVICE_AER)))
> +	    (host->native_aer || pcie_ports_dpc_native))
>   		services |= PCIE_PORT_SERVICE_DPC;
>   
>   	/* Enable bandwidth control if more than one speed is supported. */

-- 
Sathyanarayanan Kuppuswamy
Linux Kernel Developer

Re: [PATCH v2 1/1] drivers/pci: Decouple DPC from AER service

Posted by Lukas Wunner 2 months ago

[cc += Keith, Olaf; start of thread is here:
https://lore.kernel.org/all/20251208112545.21315-1-darnshah@amazon.de/
]

On Mon, Dec 08, 2025 at 11:25:45AM +0000, Darshit Shah wrote:
> According to [1] it is recommended that the Operating System link the

Minor stylistic issue:  We generally refer to the latest spec revision,
the reference at [1] doesn't mention the title of the Implementation Note
and personally I find it easier to read the commit message if references
are provided inline, so I'd recommend:

    According to PCIe r7.0, sec 6.2.11, "Implementation Note: Determination
    of DPC Control", it is recommended that ...

> +++ b/drivers/pci/pcie/portdrv.c
> @@ -264,7 +264,7 @@ static int get_port_device_capability(struct pci_dev *dev)
>  	 */
>  	if (pci_find_ext_capability(dev, PCI_EXT_CAP_ID_DPC) &&
>  	    pci_aer_available() &&
> -	    (pcie_ports_dpc_native || (services & PCIE_PORT_SERVICE_AER)))
> +	    (host->native_aer || pcie_ports_dpc_native))
>  		services |= PCIE_PORT_SERVICE_DPC;
>  
>  	/* Enable bandwidth control if more than one speed is supported. */

Somewhat tangential, I note that the pci_aer_available() check results
in DPC not being handled by Linux if pci=nomsi is specified on the
command line.  However PCIe r7.0 sec 6.2.11.1 explicitly mentions
use of INTx for DPC, suggesting that MSI support is not required.

Either case, your change looks reasonable to me, so:

Reviewed-by: Lukas Wunner <lukas@wunner.de>

Thanks,

Lukas

[PATCH v3] drivers/pci: Decouple DPC from AER service

Posted by Darshit Shah 1 month, 3 weeks ago

According to PCIe r7.0, sec. 6.2.11, "Implementation Note: Determination
of DPC Control", it is recommended that the Operating System link the
enablement of Downstream Port Containment (DPC) to the enablement of
Advanced Error Reporting (AER).

However, AER is advertised only on Root Port (RP) or Root Complex Event
Collector (RCEC) devices. On the other hand, DPC may be advertised on
any PCIe device in the hierarchy. In fact, since the main usecase of DPC
is for the switch upstream of an endpoint device to trigger a signal to
the host-bridge, it is imperative that it be supported on non-RP,
non-RCEC devices.

Previously portdrv has interpreted the spec to mean that the AER service
must be available on the same device in order for DPC to be available.
This is not what the implementation note meant to imply. If the firmware
hands Linux control of AER via _OSC on the host bridge upstream of the
device, then Linux should be allowed to assume control of DPC on the device.

The comment above this check alludes to this, by saying:
  > With dpc-native, allow Linux to use DPC even if it doesn't have
  > permission to use AER.

However, permission to use AER is negotiated at the host bridge, not
per-device. So we should not link DPC to enabling AER at the device.
Instead, DPC should be enabled if the OS has control of AER for the
host bridge that is upstream of the device in question, or if dpc-native
was set on the command line.

Cc: stable@vger.kernel.org
Signed-off-by: Darshit Shah <darnshah@amazon.de>
Reviewed-by: Lukas Wunner <lukas@wunner.de>
Reviewed-by: Kuppuswamy Sathyanarayanan <sathyanarayanan.kuppuswamy@linux.intel.com>
---
Changes from v2:
  * + stable@vger.kernel.org since moving to v6.2+ breaks DPC on our systems
  * Minor stylistic changes to commit message
  * NO functional changes

 drivers/pci/pcie/portdrv.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/pci/pcie/portdrv.c b/drivers/pci/pcie/portdrv.c
index 38a41ccf79b9..8db2fa140ae2 100644
--- a/drivers/pci/pcie/portdrv.c
+++ b/drivers/pci/pcie/portdrv.c
@@ -264,7 +264,7 @@ static int get_port_device_capability(struct pci_dev *dev)
 	 */
 	if (pci_find_ext_capability(dev, PCI_EXT_CAP_ID_DPC) &&
 	    pci_aer_available() &&
-	    (pcie_ports_dpc_native || (services & PCIE_PORT_SERVICE_AER)))
+	    (host->native_aer || pcie_ports_dpc_native))
 		services |= PCIE_PORT_SERVICE_DPC;
 
 	/* Enable bandwidth control if more than one speed is supported. */
-- 
2.47.3




Amazon Web Services Development Center Germany GmbH
Tamara-Danz-Str. 13
10243 Berlin
Geschaeftsfuehrung: Christof Hellmis, Andreas Stieger
Eingetragen am Amtsgericht Charlottenburg unter HRB 257764 B
Sitz: Berlin
Ust-ID: DE 365 538 597