From nobody Wed Dec 24 01:48:49 2025 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.13]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 794C65BAD2; Mon, 29 Jan 2024 11:28:20 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.13 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706527702; cv=none; b=O9A///ks9hxO2aRlNSiEafpWG7tkdsI4j9wwwmj9p5nN01QqwDBETrjqnoDvEK7grDn1Pb9Lz8nrIZYncKVWlZvqAqIC8VJxCqAH15fuHeKdBTeenSaHguSp88OnY0ocPetGISyr/inyQYmeNfmGOYtr9DHlqAS6lP1FdQH94LU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706527702; c=relaxed/simple; bh=vj5fBWYj/huccK5UtSPMJRYI869/VeBvWXPwIF4xpbo=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version:Content-Type; b=lYErmWoRvuLQ8b7KgiSsxfsaihWwHQixHN8MnToJ0ka6MiCU7bjQ0WCzDXRUdFRjWGA9VVhTVou/tVTuhDF9jlbfv7JSPGJV40K3WTv9UZ02J2TmckNf5cjiq4TbfwyMpTTHInwYyMxfQxapaZclqyQU2lxhyBsPpUn8QbER07U= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com; spf=none smtp.mailfrom=linux.intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=HvXzSg6W; arc=none smtp.client-ip=192.198.163.13 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.mailfrom=linux.intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="HvXzSg6W" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1706527700; x=1738063700; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=vj5fBWYj/huccK5UtSPMJRYI869/VeBvWXPwIF4xpbo=; b=HvXzSg6We+qdRerhxyjxwDP6mDeU1GO8EmlP4GocUIKNavC/a+ciyVCT XtrBz54t12k1P3yo2Lol8vGN+r5HtopeA/m3mWJm9fQutvSoRZ6XF8atb +cWwvlK5ZaqvRmHO7BYVXZmOsu4ZQ3+havEmVJNC1kR+1BdFcwAtqr5aP tKq1bX87yiA6fU8o6mukVPS5GhhbXAy1lc04nxsbLp21ZK+jLquhTN88D TyrOocQ3NZXeHSgBciMh8BwVe2LT/90Wvb2U8rx/TUA5NTaa0EbHmVhtA trvfDzJ1UK0RMxSzzju2X5boxUUM/HzdVDkm2l6UBe4HQaApd1NcBHZFn g==; X-IronPort-AV: E=McAfee;i="6600,9927,10967"; a="1857085" X-IronPort-AV: E=Sophos;i="6.05,227,1701158400"; d="scan'208";a="1857085" Received: from fmsmga004.fm.intel.com ([10.253.24.48]) by fmvoesa107.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 29 Jan 2024 03:28:19 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10967"; a="858070352" X-IronPort-AV: E=Sophos;i="6.05,227,1701158400"; d="scan'208";a="858070352" Received: from ijarvine-desk1.ger.corp.intel.com (HELO localhost) ([10.94.253.213]) by fmsmga004-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 29 Jan 2024 03:28:13 -0800 From: =?UTF-8?q?Ilpo=20J=C3=A4rvinen?= To: Bjorn Helgaas , linux-pci@vger.kernel.org, "Maciej W. Rozycki" , linux-kernel@vger.kernel.org Cc: Mika Westerberg , =?UTF-8?q?Ilpo=20J=C3=A4rvinen?= Subject: [PATCH 1/2] PCI: Clear LBMS on resume to avoid Target Speed quirk Date: Mon, 29 Jan 2024 13:27:09 +0200 Message-Id: <20240129112710.2852-2-ilpo.jarvinen@linux.intel.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20240129112710.2852-1-ilpo.jarvinen@linux.intel.com> References: <20240129112710.2852-1-ilpo.jarvinen@linux.intel.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable While a device is runtime suspended along with its PCIe hierarchy, the device could get disconnected. Because of the suspend, the device disconnection cannot be detected until portdrv/hotplug have resumed. On runtime resume, pcie_wait_for_link_delay() is called: pci_pm_runtime_resume() pci_pm_bridge_power_up_actions() pci_bridge_wait_for_secondary_bus() pcie_wait_for_link_delay() Because the device is already disconnected, this results in cascading failures: 1. pcie_wait_for_link_status() returns -ETIMEDOUT. 2. After the commit a89c82249c37 ("PCI: Work around PCIe link training failures"), pcie_failed_link_retrain() spuriously detects this failure as a Link Retraining failure and attempts the Target Speed trick, which also fails. 3. pci_bridge_wait_for_secondary_bus() then calls pci_dev_wait() which cannot succeed (but waits ~1 minute, delaying the resume). The Target Speed trick (in step 2) is only used if LBMS bit (PCIe r6.1 sec 7.5.3.8) is set. For links that have been operational before suspend, it is well possible that LBMS has been set at the bridge and remains on. Thus, after resume, LBMS does not indicate the link needs the Target Speed quirk. Clear LBMS on resume for bridges to avoid the issue. Fixes: a89c82249c37 ("PCI: Work around PCIe link training failures") Reported-by: Mika Westerberg Tested-by: Mika Westerberg Signed-off-by: Ilpo J=C3=A4rvinen --- drivers/pci/pci-driver.c | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/drivers/pci/pci-driver.c b/drivers/pci/pci-driver.c index 51ec9e7e784f..05a114962df3 100644 --- a/drivers/pci/pci-driver.c +++ b/drivers/pci/pci-driver.c @@ -574,6 +574,12 @@ static void pci_pm_bridge_power_up_actions(struct pci_= dev *pci_dev) { int ret; =20 + /* + * Clear LBMS on resume to avoid spuriously triggering Target Speed + * quirk in pcie_failed_link_retrain(). + */ + pcie_capability_write_word(pci_dev, PCI_EXP_LNKSTA, PCI_EXP_LNKSTA_LBMS); + ret =3D pci_bridge_wait_for_secondary_bus(pci_dev, "resume"); if (ret) { /* --=20 2.39.2 From nobody Wed Dec 24 01:48:49 2025 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.10]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id ED17B5B5B2; Mon, 29 Jan 2024 11:29:49 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.175.65.10 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706527791; cv=none; b=DkG8Qm8CkK2rZzvnAUwBJw8Etb3BciaMpcqEAt8QZu+KS9ntZVY2WCidAQSOk7CHDpWSzZ8D5H47jE93zedpizfcU4B5ECYrXhDN0/p6KvsTchGcxq2tJIJYp/AdRHjoDeVLpg0R4nypOn+3JDmbwk9SljHsDPL98A62CgVxMVo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706527791; c=relaxed/simple; bh=C1bZO06qv1Eiozo75zTNq7EL0T6h5w//IZNDQaGVD68=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version:Content-Type; b=ONnMykk2rqbQeP9XbVl/txQ8XXcEMCV+RC4dUXtZF7znEMRhWKD15Pgnj5dyoljHx//CgWagZKQa2kBUmIxgMkdqPvkHU/5G1btmjlzZY9b1I+rWfyttYTmNx+qk+IHyT1kfUISVOSiGpe9TY96q/0QZBCVibS/mjx1BJkHTc9E= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com; spf=none smtp.mailfrom=linux.intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=YPY9ehdi; arc=none smtp.client-ip=198.175.65.10 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.mailfrom=linux.intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="YPY9ehdi" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1706527790; x=1738063790; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=C1bZO06qv1Eiozo75zTNq7EL0T6h5w//IZNDQaGVD68=; b=YPY9ehdi779HyVu/ug4cNFRoALxjw5V+hY3VV8MRtC0KN8fUHppzqCCv B4x7iLNGA3XtDUx0PBvj/0roMj80r4i/ECKbiSzfox+McX6tHwjTiZ352 FisRAyFZL9O0J3mscofoj4BfLGcmvVLq6Sib+VRcom94Onj/mPlPz183R CXEJGqbCwtqXpnjQTZCtKFuPpTpkf6lF9FMSgMsRQBdk55Jx028zYSXXQ Upi4842QMaCse9mZ4yJK0wR8wjiynnpNXCMh7VjYjdcOA81g29iUb/JnK nszDdKiTrX9tnmatsU1SWtZN1Mq0aKlAXDiLvtjdU7JDL89r0sTE5NOwg Q==; X-IronPort-AV: E=McAfee;i="6600,9927,10967"; a="16284307" X-IronPort-AV: E=Sophos;i="6.05,227,1701158400"; d="scan'208";a="16284307" Received: from orsmga005.jf.intel.com ([10.7.209.41]) by orvoesa102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 29 Jan 2024 03:29:45 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10967"; a="960868400" X-IronPort-AV: E=Sophos;i="6.05,227,1701158400"; d="scan'208";a="960868400" Received: from ijarvine-desk1.ger.corp.intel.com (HELO localhost) ([10.94.253.213]) by orsmga005-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 29 Jan 2024 03:29:18 -0800 From: =?UTF-8?q?Ilpo=20J=C3=A4rvinen?= To: Bjorn Helgaas , linux-pci@vger.kernel.org, "Maciej W. Rozycki" , linux-kernel@vger.kernel.org Cc: Mika Westerberg , =?UTF-8?q?Ilpo=20J=C3=A4rvinen?= Subject: [PATCH 2/2] PCI: Do not wait for disconnected devices when resuming Date: Mon, 29 Jan 2024 13:27:10 +0200 Message-Id: <20240129112710.2852-3-ilpo.jarvinen@linux.intel.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20240129112710.2852-1-ilpo.jarvinen@linux.intel.com> References: <20240129112710.2852-1-ilpo.jarvinen@linux.intel.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable On runtime resume, pci_dev_wait() is called: pci_pm_runtime_resume() pci_pm_bridge_power_up_actions() pci_bridge_wait_for_secondary_bus() pci_dev_wait() While a device is runtime suspended along with its PCIe hierarchy, the device could get disconnected. In such case, the link will not come up no matter how log pci_dev_wait() waits for it. Besides the above mentioned case, there could be other ways to get the device disconnected while pci_dev_wait() is waiting for the link to come up. Make pci_dev_wait() to exit if the device is already disconnected to avoid unnecessary delay. As disconnected device is not really even a failure in the same sense as link failing to come up for whatever reason, return 0 instead of errno. Also make sure compiler does not become too clever with dev->error_state and use READ_ONCE() to force a fetch for the up-to-date value. Reported-by: Mika Westerberg Tested-by: Mika Westerberg Signed-off-by: Ilpo J=C3=A4rvinen --- drivers/pci/pci.c | 5 +++++ drivers/pci/pci.h | 4 +++- 2 files changed, 8 insertions(+), 1 deletion(-) diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c index d8f11a078924..ec9bf6c90312 100644 --- a/drivers/pci/pci.c +++ b/drivers/pci/pci.c @@ -1250,6 +1250,11 @@ static int pci_dev_wait(struct pci_dev *dev, char *r= eset_type, int timeout) for (;;) { u32 id; =20 + if (pci_dev_is_disconnected(dev)) { + pci_dbg(dev, "disconnected; not waiting\n"); + return 0; + } + pci_read_config_dword(dev, PCI_COMMAND, &id); if (!PCI_POSSIBLE_ERROR(id)) break; diff --git a/drivers/pci/pci.h b/drivers/pci/pci.h index 2336a8d1edab..563a275dff67 100644 --- a/drivers/pci/pci.h +++ b/drivers/pci/pci.h @@ -4,6 +4,8 @@ =20 #include =20 +#include + /* Number of possible devfns: 0.0 to 1f.7 inclusive */ #define MAX_NR_DEVFNS 256 =20 @@ -370,7 +372,7 @@ static inline int pci_dev_set_disconnected(struct pci_d= ev *dev, void *unused) =20 static inline bool pci_dev_is_disconnected(const struct pci_dev *dev) { - return dev->error_state =3D=3D pci_channel_io_perm_failure; + return READ_ONCE(dev->error_state) =3D=3D pci_channel_io_perm_failure; } =20 /* pci_dev priv_flags */ --=20 2.39.2