From nobody Mon Feb 9 15:13:09 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5EF67C64EC3 for ; Sun, 5 Feb 2023 15:49:07 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229787AbjBEPtE (ORCPT ); Sun, 5 Feb 2023 10:49:04 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:43810 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229457AbjBEPtC (ORCPT ); Sun, 5 Feb 2023 10:49:02 -0500 Received: from angie.orcam.me.uk (angie.orcam.me.uk [IPv6:2001:4190:8020::34]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id 630A3EC69; Sun, 5 Feb 2023 07:49:01 -0800 (PST) Received: by angie.orcam.me.uk (Postfix, from userid 500) id A6BC192009E; Sun, 5 Feb 2023 16:48:59 +0100 (CET) Received: from localhost (localhost [127.0.0.1]) by angie.orcam.me.uk (Postfix) with ESMTP id A23BE92009D; Sun, 5 Feb 2023 15:48:59 +0000 (GMT) Date: Sun, 5 Feb 2023 15:48:59 +0000 (GMT) From: "Maciej W. Rozycki" To: Bjorn Helgaas , Mahesh J Salgaonkar , Oliver O'Halloran , Michael Ellerman , Nicholas Piggin , Christophe Leroy , Saeed Mahameed , Leon Romanovsky , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni cc: Alex Williamson , Lukas Wunner , Mika Westerberg , Stefan Roese , Jim Wilson , David Abdurachmanov , =?UTF-8?Q?Pali_Roh=C3=A1r?= , linux-pci@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-rdma@vger.kernel.org, netdev@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH v6 1/7] PCI: Export PCI link retrain timeout In-Reply-To: Message-ID: References: User-Agent: Alpine 2.21 (DEB 202 2017-01-01) MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Rename LINK_RETRAIN_TIMEOUT to PCIE_LINK_RETRAIN_TIMEOUT and make it available via "pci.h" for PCI drivers to use. Signed-off-by: Maciej W. Rozycki --- No change from v5. New change in v5. --- drivers/pci/pci.h | 2 ++ drivers/pci/pcie/aspm.c | 4 +--- 2 files changed, 3 insertions(+), 3 deletions(-) linux-pcie-link-retrain-timeout.diff Index: linux-macro/drivers/pci/pci.h =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D --- linux-macro.orig/drivers/pci/pci.h +++ linux-macro/drivers/pci/pci.h @@ -11,6 +11,8 @@ =20 #define PCI_VSEC_ID_INTEL_TBT 0x1234 /* Thunderbolt */ =20 +#define PCIE_LINK_RETRAIN_TIMEOUT HZ + extern const unsigned char pcie_link_speed[]; extern bool pci_early_dump; =20 Index: linux-macro/drivers/pci/pcie/aspm.c =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D --- linux-macro.orig/drivers/pci/pcie/aspm.c +++ linux-macro/drivers/pci/pcie/aspm.c @@ -90,8 +90,6 @@ static const char *policy_str[] =3D { [POLICY_POWER_SUPERSAVE] =3D "powersupersave" }; =20 -#define LINK_RETRAIN_TIMEOUT HZ - /* * The L1 PM substate capability is only implemented in function 0 in a * multi function device. @@ -213,7 +211,7 @@ static bool pcie_retrain_link(struct pci } =20 /* Wait for link training end. Break out after waiting for timeout */ - end_jiffies =3D jiffies + LINK_RETRAIN_TIMEOUT; + end_jiffies =3D jiffies + PCIE_LINK_RETRAIN_TIMEOUT; do { pcie_capability_read_word(parent, PCI_EXP_LNKSTA, ®16); if (!(reg16 & PCI_EXP_LNKSTA_LT)) From nobody Mon Feb 9 15:13:09 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id D3D7BC636CD for ; Sun, 5 Feb 2023 15:49:13 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229812AbjBEPtL (ORCPT ); Sun, 5 Feb 2023 10:49:11 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:43880 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229791AbjBEPtF (ORCPT ); Sun, 5 Feb 2023 10:49:05 -0500 Received: from angie.orcam.me.uk (angie.orcam.me.uk [78.133.224.34]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id DA5441A969 for ; Sun, 5 Feb 2023 07:49:04 -0800 (PST) Received: by angie.orcam.me.uk (Postfix, from userid 500) id 2C0899200B4; Sun, 5 Feb 2023 16:49:04 +0100 (CET) Received: from localhost (localhost [127.0.0.1]) by angie.orcam.me.uk (Postfix) with ESMTP id 2908F9200B3; Sun, 5 Feb 2023 15:49:04 +0000 (GMT) Date: Sun, 5 Feb 2023 15:49:04 +0000 (GMT) From: "Maciej W. Rozycki" To: Bjorn Helgaas , Mahesh J Salgaonkar , Oliver O'Halloran , Michael Ellerman , Nicholas Piggin , Christophe Leroy , Saeed Mahameed , Leon Romanovsky , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni cc: Alex Williamson , Lukas Wunner , Mika Westerberg , Stefan Roese , Jim Wilson , David Abdurachmanov , =?UTF-8?Q?Pali_Roh=C3=A1r?= , linux-pci@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-rdma@vger.kernel.org, netdev@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH v6 2/7] PCI: Execute `quirk_enable_clear_retrain_link' earlier In-Reply-To: Message-ID: References: User-Agent: Alpine 2.21 (DEB 202 2017-01-01) MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Make `quirk_enable_clear_retrain_link' `pci_fixup_early' so that any later=20 fixups can rely on `clear_retrain_link' to have been already initialised. Signed-off-by: Maciej W. Rozycki --- No change from v5. New change in v5. --- drivers/pci/quirks.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) linux-pcie-clear-retrain-link-early.diff Index: linux-macro/drivers/pci/quirks.c =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D --- linux-macro.orig/drivers/pci/quirks.c +++ linux-macro/drivers/pci/quirks.c @@ -2407,9 +2407,9 @@ static void quirk_enable_clear_retrain_l dev->clear_retrain_link =3D 1; pci_info(dev, "Enable PCIe Retrain Link quirk\n"); } -DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_PERICOM, 0xe110, quirk_enable_clear= _retrain_link); -DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_PERICOM, 0xe111, quirk_enable_clear= _retrain_link); -DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_PERICOM, 0xe130, quirk_enable_clear= _retrain_link); +DECLARE_PCI_FIXUP_EARLY(PCI_VENDOR_ID_PERICOM, 0xe110, quirk_enable_clear_= retrain_link); +DECLARE_PCI_FIXUP_EARLY(PCI_VENDOR_ID_PERICOM, 0xe111, quirk_enable_clear_= retrain_link); +DECLARE_PCI_FIXUP_EARLY(PCI_VENDOR_ID_PERICOM, 0xe130, quirk_enable_clear_= retrain_link); =20 static void fixup_rev1_53c810(struct pci_dev *dev) { From nobody Mon Feb 9 15:13:09 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id D1F94C63797 for ; Sun, 5 Feb 2023 15:49:23 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229866AbjBEPtV (ORCPT ); Sun, 5 Feb 2023 10:49:21 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44048 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229803AbjBEPtK (ORCPT ); Sun, 5 Feb 2023 10:49:10 -0500 Received: from angie.orcam.me.uk (angie.orcam.me.uk [78.133.224.34]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id 5366C1A4BE for ; Sun, 5 Feb 2023 07:49:09 -0800 (PST) Received: by angie.orcam.me.uk (Postfix, from userid 500) id 755979200BC; Sun, 5 Feb 2023 16:49:08 +0100 (CET) Received: from localhost (localhost [127.0.0.1]) by angie.orcam.me.uk (Postfix) with ESMTP id 716119200BB; Sun, 5 Feb 2023 15:49:08 +0000 (GMT) Date: Sun, 5 Feb 2023 15:49:08 +0000 (GMT) From: "Maciej W. Rozycki" To: Bjorn Helgaas , Mahesh J Salgaonkar , Oliver O'Halloran , Michael Ellerman , Nicholas Piggin , Christophe Leroy , Saeed Mahameed , Leon Romanovsky , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni cc: Alex Williamson , Lukas Wunner , Mika Westerberg , Stefan Roese , Jim Wilson , David Abdurachmanov , =?UTF-8?Q?Pali_Roh=C3=A1r?= , linux-pci@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-rdma@vger.kernel.org, netdev@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH v6 3/7] PCI: Initialize `link_active_reporting' earlier In-Reply-To: Message-ID: References: User-Agent: Alpine 2.21 (DEB 202 2017-01-01) MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Determine whether Data Link Layer Link Active Reporting is available=20 ahead of calling any fixups so that the cached value can be used there=20 and later on. Signed-off-by: Maciej W. Rozycki --- New change in v6. --- drivers/pci/probe.c | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) linux-pcie-link-active-reporting-early.diff Index: linux-macro/drivers/pci/probe.c =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D --- linux-macro.orig/drivers/pci/probe.c +++ linux-macro/drivers/pci/probe.c @@ -819,7 +819,6 @@ static void pci_set_bus_speed(struct pci =20 pcie_capability_read_dword(bridge, PCI_EXP_LNKCAP, &linkcap); bus->max_bus_speed =3D pcie_link_speed[linkcap & PCI_EXP_LNKCAP_SLS]; - bridge->link_active_reporting =3D !!(linkcap & PCI_EXP_LNKCAP_DLLLARC); =20 pcie_capability_read_word(bridge, PCI_EXP_LNKSTA, &linksta); pcie_update_link_speed(bus, linksta); @@ -1828,6 +1827,7 @@ int pci_setup_device(struct pci_dev *dev int pos =3D 0; struct pci_bus_region region; struct resource *res; + u32 linkcap; =20 hdr_type =3D pci_hdr_type(dev); =20 @@ -1873,6 +1873,10 @@ int pci_setup_device(struct pci_dev *dev /* "Unknown power state" */ dev->current_state =3D PCI_UNKNOWN; =20 + /* Set it early to make it available to fixups, etc. */ + pcie_capability_read_dword(dev, PCI_EXP_LNKCAP, &linkcap); + dev->link_active_reporting =3D !!(linkcap & PCI_EXP_LNKCAP_DLLLARC); + /* Early fixups, before probing the BARs */ pci_fixup_device(pci_fixup_early, dev); From nobody Mon Feb 9 15:13:09 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id CD20DC636CC for ; Sun, 5 Feb 2023 15:49:43 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229894AbjBEPtm (ORCPT ); Sun, 5 Feb 2023 10:49:42 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44656 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229788AbjBEPtY (ORCPT ); Sun, 5 Feb 2023 10:49:24 -0500 Received: from angie.orcam.me.uk (angie.orcam.me.uk [IPv6:2001:4190:8020::34]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id 0C2961F4B0; Sun, 5 Feb 2023 07:49:13 -0800 (PST) Received: by angie.orcam.me.uk (Postfix, from userid 500) id 97BD39200C0; Sun, 5 Feb 2023 16:49:12 +0100 (CET) Received: from localhost (localhost [127.0.0.1]) by angie.orcam.me.uk (Postfix) with ESMTP id 910D59200BF; Sun, 5 Feb 2023 15:49:12 +0000 (GMT) Date: Sun, 5 Feb 2023 15:49:12 +0000 (GMT) From: "Maciej W. Rozycki" To: Bjorn Helgaas , Mahesh J Salgaonkar , Oliver O'Halloran , Michael Ellerman , Nicholas Piggin , Christophe Leroy , Saeed Mahameed , Leon Romanovsky , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni cc: Alex Williamson , Lukas Wunner , Mika Westerberg , Stefan Roese , Jim Wilson , David Abdurachmanov , =?UTF-8?Q?Pali_Roh=C3=A1r?= , linux-pci@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-rdma@vger.kernel.org, netdev@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH v6 4/7] powerpc/eeh: Rely on `link_active_reporting' In-Reply-To: Message-ID: References: User-Agent: Alpine 2.21 (DEB 202 2017-01-01) MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Use `link_active_reporting' to determine whether Data Link Layer Link=20 Active Reporting is available rather than re-retrieving the capability. Signed-off-by: Maciej W. Rozycki --- NB this has been compile-tested only with a PPC64LE configuration. New change in v6. --- arch/powerpc/kernel/eeh_pe.c | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) linux-pcie-link-active-reporting-eeh.diff Index: linux-macro/arch/powerpc/kernel/eeh_pe.c =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D --- linux-macro.orig/arch/powerpc/kernel/eeh_pe.c +++ linux-macro/arch/powerpc/kernel/eeh_pe.c @@ -671,9 +671,8 @@ static void eeh_bridge_check_link(struct eeh_ops->write_config(edev, cap + PCI_EXP_LNKCTL, 2, val); =20 /* Check link */ - eeh_ops->read_config(edev, cap + PCI_EXP_LNKCAP, 4, &val); - if (!(val & PCI_EXP_LNKCAP_DLLLARC)) { - eeh_edev_dbg(edev, "No link reporting capability (0x%08x) \n", val); + if (!edev->pdev->link_active_reporting) { + eeh_edev_dbg(edev, "No link reporting capability\n"); msleep(1000); return; } From nobody Mon Feb 9 15:13:09 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2148AC636CD for ; Sun, 5 Feb 2023 15:49:55 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229870AbjBEPtx (ORCPT ); Sun, 5 Feb 2023 10:49:53 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44936 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229873AbjBEPth (ORCPT ); Sun, 5 Feb 2023 10:49:37 -0500 Received: from angie.orcam.me.uk (angie.orcam.me.uk [78.133.224.34]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id 347A71ADDB for ; Sun, 5 Feb 2023 07:49:17 -0800 (PST) Received: by angie.orcam.me.uk (Postfix, from userid 500) id 6D2B29200C4; Sun, 5 Feb 2023 16:49:16 +0100 (CET) Received: from localhost (localhost [127.0.0.1]) by angie.orcam.me.uk (Postfix) with ESMTP id 6A1A89200C3; Sun, 5 Feb 2023 15:49:16 +0000 (GMT) Date: Sun, 5 Feb 2023 15:49:16 +0000 (GMT) From: "Maciej W. Rozycki" To: Bjorn Helgaas , Mahesh J Salgaonkar , Oliver O'Halloran , Michael Ellerman , Nicholas Piggin , Christophe Leroy , Saeed Mahameed , Leon Romanovsky , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni cc: Alex Williamson , Lukas Wunner , Mika Westerberg , Stefan Roese , Jim Wilson , David Abdurachmanov , =?UTF-8?Q?Pali_Roh=C3=A1r?= , linux-pci@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-rdma@vger.kernel.org, netdev@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH v6 5/7] net/mlx5: Rely on `link_active_reporting' In-Reply-To: Message-ID: References: User-Agent: Alpine 2.21 (DEB 202 2017-01-01) MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Use `link_active_reporting' to determine whether Data Link Layer Link=20 Active Reporting is available rather than re-retrieving the capability. Signed-off-by: Maciej W. Rozycki --- NB this has been compile-tested only with PPC64LE and x86-64=20 configurations. New change in v6. --- drivers/net/ethernet/mellanox/mlx5/core/fw_reset.c | 8 ++------ 1 file changed, 2 insertions(+), 6 deletions(-) linux-pcie-link-active-reporting-mlx5.diff Index: linux-macro/drivers/net/ethernet/mellanox/mlx5/core/fw_reset.c =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D --- linux-macro.orig/drivers/net/ethernet/mellanox/mlx5/core/fw_reset.c +++ linux-macro/drivers/net/ethernet/mellanox/mlx5/core/fw_reset.c @@ -294,7 +294,6 @@ static int mlx5_pci_link_toggle(struct m unsigned long timeout; struct pci_dev *sdev; int cap, err; - u32 reg32; =20 /* Check that all functions under the pci bridge are PFs of * this device otherwise fail this function. @@ -333,11 +332,8 @@ static int mlx5_pci_link_toggle(struct m return err; =20 /* Check link */ - err =3D pci_read_config_dword(bridge, cap + PCI_EXP_LNKCAP, ®32); - if (err) - return err; - if (!(reg32 & PCI_EXP_LNKCAP_DLLLARC)) { - mlx5_core_warn(dev, "No PCI link reporting capability (0x%08x)\n", reg32= ); + if (!bridge->link_active_reporting) { + mlx5_core_warn(dev, "No PCI link reporting capability\n"); msleep(1000); goto restore; } From nobody Mon Feb 9 15:13:09 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 96BCDC636D6 for ; Sun, 5 Feb 2023 15:50:01 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229831AbjBEPuA (ORCPT ); Sun, 5 Feb 2023 10:50:00 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44880 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229841AbjBEPtu (ORCPT ); Sun, 5 Feb 2023 10:49:50 -0500 Received: from angie.orcam.me.uk (angie.orcam.me.uk [IPv6:2001:4190:8020::34]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id 0C7511ADE9; Sun, 5 Feb 2023 07:49:23 -0800 (PST) Received: by angie.orcam.me.uk (Postfix, from userid 500) id 60B849200CC; Sun, 5 Feb 2023 16:49:21 +0100 (CET) Received: from localhost (localhost [127.0.0.1]) by angie.orcam.me.uk (Postfix) with ESMTP id 5A74C9200CB; Sun, 5 Feb 2023 15:49:21 +0000 (GMT) Date: Sun, 5 Feb 2023 15:49:21 +0000 (GMT) From: "Maciej W. Rozycki" To: Bjorn Helgaas , Mahesh J Salgaonkar , Oliver O'Halloran , Michael Ellerman , Nicholas Piggin , Christophe Leroy , Saeed Mahameed , Leon Romanovsky , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni cc: Alex Williamson , Lukas Wunner , Mika Westerberg , Stefan Roese , Jim Wilson , David Abdurachmanov , =?UTF-8?Q?Pali_Roh=C3=A1r?= , linux-pci@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-rdma@vger.kernel.org, netdev@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH v6 6/7] PCI: pciehp: Rely on `link_active_reporting' In-Reply-To: Message-ID: References: User-Agent: Alpine 2.21 (DEB 202 2017-01-01) MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Use `link_active_reporting' to determine whether Data Link Layer Link=20 Active Reporting is available rather than re-retrieving the capability. Signed-off-by: Maciej W. Rozycki Reviewed-by: Lukas Wunner --- NB this has been compile-tested only with PPC64LE and x86-64 configurations. New change in v6. --- drivers/pci/hotplug/pciehp_hpc.c | 7 ++----- 1 file changed, 2 insertions(+), 5 deletions(-) linux-pcie-link-active-reporting-hpc.diff Index: linux-macro/drivers/pci/hotplug/pciehp_hpc.c =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D --- linux-macro.orig/drivers/pci/hotplug/pciehp_hpc.c +++ linux-macro/drivers/pci/hotplug/pciehp_hpc.c @@ -984,7 +984,7 @@ static inline int pcie_hotplug_depth(str struct controller *pcie_init(struct pcie_device *dev) { struct controller *ctrl; - u32 slot_cap, slot_cap2, link_cap; + u32 slot_cap, slot_cap2; u8 poweron; struct pci_dev *pdev =3D dev->port; struct pci_bus *subordinate =3D pdev->subordinate; @@ -1030,9 +1030,6 @@ struct controller *pcie_init(struct pcie if (dmi_first_match(inband_presence_disabled_dmi_table)) ctrl->inband_presence_disabled =3D 1; =20 - /* Check if Data Link Layer Link Active Reporting is implemented */ - pcie_capability_read_dword(pdev, PCI_EXP_LNKCAP, &link_cap); - /* Clear all remaining event bits in Slot Status register. */ pcie_capability_write_word(pdev, PCI_EXP_SLTSTA, PCI_EXP_SLTSTA_ABP | PCI_EXP_SLTSTA_PFD | @@ -1051,7 +1048,7 @@ struct controller *pcie_init(struct pcie FLAG(slot_cap, PCI_EXP_SLTCAP_EIP), FLAG(slot_cap, PCI_EXP_SLTCAP_NCCS), FLAG(slot_cap2, PCI_EXP_SLTCAP2_IBPD), - FLAG(link_cap, PCI_EXP_LNKCAP_DLLLARC), + FLAG(pdev->link_active_reporting, true), pdev->broken_cmd_compl ? " (with Cmd Compl erratum)" : ""); =20 /* From nobody Mon Feb 9 15:13:09 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 22790C6379F for ; Sun, 5 Feb 2023 15:54:54 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229666AbjBEPyv (ORCPT ); Sun, 5 Feb 2023 10:54:51 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49566 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229519AbjBEPyp (ORCPT ); Sun, 5 Feb 2023 10:54:45 -0500 Received: from angie.orcam.me.uk (angie.orcam.me.uk [IPv6:2001:4190:8020::34]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id CC1AC171D; Sun, 5 Feb 2023 07:54:43 -0800 (PST) Received: by angie.orcam.me.uk (Postfix, from userid 500) id 90ECE9200C3; Sun, 5 Feb 2023 16:49:27 +0100 (CET) Received: from localhost (localhost [127.0.0.1]) by angie.orcam.me.uk (Postfix) with ESMTP id 8A5369200BF; Sun, 5 Feb 2023 15:49:27 +0000 (GMT) Date: Sun, 5 Feb 2023 15:49:27 +0000 (GMT) From: "Maciej W. Rozycki" To: Bjorn Helgaas , Mahesh J Salgaonkar , Oliver O'Halloran , Michael Ellerman , Nicholas Piggin , Christophe Leroy , Saeed Mahameed , Leon Romanovsky , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni cc: Alex Williamson , Lukas Wunner , Mika Westerberg , Stefan Roese , Jim Wilson , David Abdurachmanov , =?UTF-8?Q?Pali_Roh=C3=A1r?= , linux-pci@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-rdma@vger.kernel.org, netdev@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH v6 7/7] PCI: Work around PCIe link training failures In-Reply-To: Message-ID: References: User-Agent: Alpine 2.21 (DEB 202 2017-01-01) MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Attempt to handle cases such as with a downstream port of the ASMedia=20 ASM2824 PCIe switch where link training never completes and the link=20 continues switching between speeds indefinitely with the data link layer=20 never reaching the active state. It has been observed with a downstream port of the ASMedia ASM2824 Gen 3=20 switch wired to the upstream port of the Pericom PI7C9X2G304 Gen 2=20 switch, using a Delock Riser Card PCI Express x1 > 2 x PCIe x1 device,=20 P/N 41433, wired to a SiFive HiFive Unmatched board. In this setup the=20 switches are supposed to negotiate the link speed of preferably 5.0GT/s,=20 falling back to 2.5GT/s. Instead the link continues oscillating between the two speeds, at the=20 rate of 34-35 times per second, with link training reported repeatedly=20 active ~84% of the time. Forcibly limiting the target link speed to=20 2.5GT/s with the upstream ASM2824 device however makes the two switches=20 communicate correctly. Removing the speed restriction afterwards makes=20 the two devices switch to 5.0GT/s then. Make use of these observations then and detect the inability to train=20 the link, by checking for the Data Link Layer Link Active status bit=20 being off while the Link Bandwidth Management Status indicating that=20 hardware has changed the link speed or width in an attempt to correct=20 unreliable link operation. Restrict the speed to 2.5GT/s then with the Target Link Speed field,=20 request a retrain and wait 200ms for the data link to go up. If this=20 turns out successful, then lift the restriction, letting the devices=20 negotiate a higher speed. Also check for a 2.5GT/s speed restriction the firmware may have already=20 arranged and lift it too with ports of devices known to continue working=20 afterwards, currently the ASM2824 only, that already report their data=20 link being up. Signed-off-by: Maciej W. Rozycki Link: https://lore.kernel.org/lkml/alpine.DEB.2.21.2203022037020.56670@angi= e.orcam.me.uk/ Link: https://source.denx.de/u-boot/u-boot/-/commit/a398a51ccc68 --- Changes from v5: - Move from a quirk into PCI core and call at device probing, hot-plug, reset and resume. Keep the ASMedia part under CONFIG_PCI_QUIRKS. - Rely on `dev->link_active_reporting' rather than re-retrieving the=20 capability. Changes from v4: - Remove inclusion no longer needed. - Make the quirk generic based on probing device features rather than=20 specific to the ASM2824 part only; take the Retrain Link bit erratum=20 into account. - Still lift the 2.5GT/s speed restriction with the ASM2824 only. - Increase retrain timeout from 200ms to 1s (PCIE_LINK_RETRAIN_TIMEOUT). - Remove retrain success notification. - Use PCIe helpers rather than generic PCI functions throughout. - Trim down and update the wording of the change description for the=20 switch from an ASM2824-specific to a generic fixup. Changes from v3: - Remove the entry for the ASM2824. Changes from v2: - Regenerate for 5.17-rc2 for a merge conflict. - Replace BUG_ON for a missing PCI Express capability with WARN_ON and an early return. Changes from v1: - Regenerate for a merge conflict. --- drivers/pci/pci.c | 154 +++++++++++++++++++++++++++++++++++++++++++++++= +++-- drivers/pci/pci.h | 1=20 drivers/pci/probe.c | 2=20 3 files changed, 152 insertions(+), 5 deletions(-) linux-pcie-asm2824-manual-retrain.diff Index: linux-macro/drivers/pci/pci.c =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D --- linux-macro.orig/drivers/pci/pci.c +++ linux-macro/drivers/pci/pci.c @@ -859,6 +859,132 @@ int pci_wait_for_pending(struct pci_dev return 0; } =20 +/* + * Retrain the link of a downstream PCIe port by hand if necessary. + * + * This is needed at least where a downstream port of the ASMedia ASM2824 + * Gen 3 switch is wired to the upstream port of the Pericom PI7C9X2G304 + * Gen 2 switch, and observed with the Delock Riser Card PCI Express x1 > + * 2 x PCIe x1 device, P/N 41433, plugged into the SiFive HiFive Unmatched + * board. + * + * In such a configuration the switches are supposed to negotiate the link + * speed of preferably 5.0GT/s, falling back to 2.5GT/s. However the link + * continues switching between the two speeds indefinitely and the data + * link layer never reaches the active state, with link training reported + * repeatedly active ~84% of the time. Forcing the target link speed to + * 2.5GT/s with the upstream ASM2824 device makes the two switches talk to + * each other correctly however. And more interestingly retraining with a + * higher target link speed afterwards lets the two successfully negotiate + * 5.0GT/s. + * + * With the ASM2824 we can rely on the otherwise optional Data Link Layer + * Link Active status bit and in the failed link training scenario it will + * be off along with the Link Bandwidth Management Status indicating that + * hardware has changed the link speed or width in an attempt to correct + * unreliable link operation. For a port that has been left unconnected + * both bits will be clear. So use this information to detect the problem + * rather than polling the Link Training bit and watching out for flips or + * at least the active status. + * + * Since the exact nature of the problem isn't known and in principle this + * could trigger where an ASM2824 device is downstream rather upstream, + * apply this erratum workaround to any downstream ports as long as they + * support Link Active reporting and have the Link Control 2 register. + * Restrict the speed to 2.5GT/s then with the Target Link Speed field, + * request a retrain and wait 200ms for the data link to go up. + * + * If this turns out successful and we know by the Vendor:Device ID it is + * safe to do so, then lift the restriction, letting the devices negotiate + * a higher speed. Also check for a similar 2.5GT/s speed restriction the + * firmware may have already arranged and lift it with ports that already + * report their data link being up. + * + * Return 0 if the link has been successfully retrained, otherwise -1. + */ +int pcie_downstream_link_retrain(struct pci_dev *dev) +{ + static const struct pci_device_id ids[] =3D { + { PCI_VDEVICE(ASMEDIA, 0x2824) }, /* ASMedia ASM2824 */ + {} + }; + u16 lnksta, lnkctl2; + + if (!pci_is_pcie(dev) || !pcie_downstream_port(dev) || + !pcie_cap_has_lnkctl2(dev) || !dev->link_active_reporting) + return -1; + + pcie_capability_read_word(dev, PCI_EXP_LNKCTL2, &lnkctl2); + pcie_capability_read_word(dev, PCI_EXP_LNKSTA, &lnksta); + if ((lnksta & (PCI_EXP_LNKSTA_LBMS | PCI_EXP_LNKSTA_DLLLA)) =3D=3D + PCI_EXP_LNKSTA_LBMS) { + unsigned long timeout; + u16 lnkctl; + + pci_info(dev, "broken device, retraining non-functional downstream link = at 2.5GT/s\n"); + + pcie_capability_read_word(dev, PCI_EXP_LNKCTL, &lnkctl); + lnkctl |=3D PCI_EXP_LNKCTL_RL; + lnkctl2 &=3D ~PCI_EXP_LNKCTL2_TLS; + lnkctl2 |=3D PCI_EXP_LNKCTL2_TLS_2_5GT; + pcie_capability_write_word(dev, PCI_EXP_LNKCTL2, lnkctl2); + pcie_capability_write_word(dev, PCI_EXP_LNKCTL, lnkctl); + /* + * Due to an erratum in some devices the Retrain Link bit + * needs to be cleared again manually to allow the link + * training to succeed. + */ + lnkctl &=3D ~PCI_EXP_LNKCTL_RL; + if (dev->clear_retrain_link) + pcie_capability_write_word(dev, PCI_EXP_LNKCTL, + lnkctl); + + timeout =3D jiffies + PCIE_LINK_RETRAIN_TIMEOUT; + do { + pcie_capability_read_word(dev, PCI_EXP_LNKSTA, + &lnksta); + if (lnksta & PCI_EXP_LNKSTA_DLLLA) + break; + usleep_range(10000, 20000); + } while (time_before(jiffies, timeout)); + + if (!(lnksta & PCI_EXP_LNKSTA_DLLLA)) { + pci_info(dev, "retraining failed\n"); + return -1; + } + } + + if (IS_ENABLED(CONFIG_PCI_QUIRKS) && (lnksta & PCI_EXP_LNKSTA_DLLLA) && + (lnkctl2 & PCI_EXP_LNKCTL2_TLS) =3D=3D PCI_EXP_LNKCTL2_TLS_2_5GT && + pci_match_id(ids, dev)) { + u32 lnkcap; + u16 lnkctl; + + pci_info(dev, "removing 2.5GT/s downstream link speed restriction\n"); + pcie_capability_read_dword(dev, PCI_EXP_LNKCAP, &lnkcap); + pcie_capability_read_word(dev, PCI_EXP_LNKCTL, &lnkctl); + lnkctl |=3D PCI_EXP_LNKCTL_RL; + lnkctl2 &=3D ~PCI_EXP_LNKCTL2_TLS; + lnkctl2 |=3D lnkcap & PCI_EXP_LNKCAP_SLS; + pcie_capability_write_word(dev, PCI_EXP_LNKCTL2, lnkctl2); + pcie_capability_write_word(dev, PCI_EXP_LNKCTL, lnkctl); + } + + return 0; +} + +/* Same as above, but called for a downstream device. */ +static int pcie_upstream_link_retrain(struct pci_dev *dev) +{ + struct pci_dev *bridge; + + bridge =3D pci_upstream_bridge(dev); + if (bridge) + return pcie_downstream_link_retrain(bridge); + else + return -1; +} + static int pci_acs_enable; =20 /** @@ -1148,8 +1274,8 @@ void pci_resume_bus(struct pci_bus *bus) =20 static int pci_dev_wait(struct pci_dev *dev, char *reset_type, int timeout) { + int retrain =3D 0; int delay =3D 1; - u32 id; =20 /* * After reset, the device should not silently discard config @@ -1163,21 +1289,37 @@ static int pci_dev_wait(struct pci_dev * * Command register instead of Vendor ID so we don't have to * contend with the CRS SV value. */ - pci_read_config_dword(dev, PCI_COMMAND, &id); - while (PCI_POSSIBLE_ERROR(id)) { + for (;;) { + u32 id; + + pci_read_config_dword(dev, PCI_COMMAND, &id); + if (!PCI_POSSIBLE_ERROR(id)) { + if (delay > PCI_RESET_WAIT) + pci_info(dev, "ready %dms after %s\n", + delay - 1, reset_type); + break; + } + if (delay > timeout) { pci_warn(dev, "not ready %dms after %s; giving up\n", delay - 1, reset_type); return -ENOTTY; } =20 - if (delay > PCI_RESET_WAIT) + if (delay > PCI_RESET_WAIT) { + if (!retrain) { + retrain =3D 1; + if (pcie_upstream_link_retrain(dev) =3D=3D 0) { + delay =3D 1; + continue; + } + } pci_info(dev, "not ready %dms after %s; waiting\n", delay - 1, reset_type); + } =20 msleep(delay); delay *=3D 2; - pci_read_config_dword(dev, PCI_COMMAND, &id); } =20 if (delay > PCI_RESET_WAIT) @@ -4901,6 +5043,8 @@ static bool pcie_wait_for_link_delay(str msleep(10); timeout -=3D 10; } + if (active && !ret) + ret =3D pcie_downstream_link_retrain(pdev) =3D=3D 0; if (active && ret) msleep(delay); =20 Index: linux-macro/drivers/pci/pci.h =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D --- linux-macro.orig/drivers/pci/pci.h +++ linux-macro/drivers/pci/pci.h @@ -37,6 +37,7 @@ int pci_mmap_fits(struct pci_dev *pdev, enum pci_mmap_api mmap_api); =20 bool pci_reset_supported(struct pci_dev *dev); +int pcie_downstream_link_retrain(struct pci_dev *dev); void pci_init_reset_methods(struct pci_dev *dev); int pci_bridge_secondary_bus_reset(struct pci_dev *dev); int pci_bus_error_reset(struct pci_dev *dev); Index: linux-macro/drivers/pci/probe.c =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D --- linux-macro.orig/drivers/pci/probe.c +++ linux-macro/drivers/pci/probe.c @@ -2546,6 +2546,8 @@ void pci_device_add(struct pci_dev *dev, dma_set_max_seg_size(&dev->dev, 65536); dma_set_seg_boundary(&dev->dev, 0xffffffff); =20 + pcie_downstream_link_retrain(dev); + /* Fix up broken headers */ pci_fixup_device(pci_fixup_header, dev);