From nobody Wed Dec 17 23:53:19 2025 Received: from mail-pg1-f172.google.com (mail-pg1-f172.google.com [209.85.215.172]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 76951175AA for ; Wed, 26 Jun 2024 09:35:32 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.215.172 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1719394534; cv=none; b=d5fV+xFYW/MgoK+p5H4rH1aFkKE4Vyyx+5cec+or0E8i/CbROuP2cu1tcj47vMYBkdNEuLj45IdliMhL8sn9eskyWJRTjX/rjdE3yiQe97wOTYbX396f+z5WcTw7JKzX6nDjdgR/xrLvTWphoqCYjuPdiWAK+6fxFTLcY4EIsEg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1719394534; c=relaxed/simple; bh=K78X87jtcDtrl1rBAgUC04sDiKheqNq5YAD6b2AkW5Q=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=HW+LJl5WywZFoTFTlbhivgiT7o28kPBF4fv56Wby6enopvPltnEthheZ8RynZ8HYM4MRyQxYH16l0zj9hWrzcIDwgRCiqUvTJMlfmdOGIzU0SMLEYUdFIjnw7o4wGo8vpXkAh/OoD8eahKAqFEAE5XO9iiyZ8/97D+AnqitPA6Y= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=endlessos.org; spf=pass smtp.mailfrom=endlessos.org; dkim=pass (2048-bit key) header.d=endlessos.org header.i=@endlessos.org header.b=tIC3L1QW; arc=none smtp.client-ip=209.85.215.172 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=endlessos.org Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=endlessos.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=endlessos.org header.i=@endlessos.org header.b="tIC3L1QW" Received: by mail-pg1-f172.google.com with SMTP id 41be03b00d2f7-6c5bcb8e8edso4712659a12.2 for ; Wed, 26 Jun 2024 02:35:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=endlessos.org; s=google; t=1719394532; x=1719999332; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=BpeNWWySkoOjmL9GG/yU7x4/CpRAcKWqGhNrfk1bvDE=; b=tIC3L1QWXUwdgLLXXTZbEgOCFjqA0hmmMeq3fD5UL8VtQEdIWJOWDuPyg3a4GLMmS0 dxPUmyKA2grVEePoNu27vhZ2CqNErdvu5OdMSr2fcYDGVp6T1QMqtEY6y+XlLl4Pa7pH 9/nWoYOatYuQ9sxz4K8XpVvO1Tdm0v8Lz/h1D3IUnpAMDiCCiEfUpltpvVRY9u5j/ZOM PwXhMwIQr605rLjRoBN1gjjLdJ09Cs08Fed4prBOEW8mvYj18x33BmYkEJg3kWeDKlf6 XUPS1eUjHBmmkDumDzdzeyrHTwXBL8/3BZYLNAkYz3SN+gVJpXLQ1Wkko6q0gT+Sy5X+ RBfA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1719394532; x=1719999332; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=BpeNWWySkoOjmL9GG/yU7x4/CpRAcKWqGhNrfk1bvDE=; b=spwuOkw4jQsx2LI+b743Es5nBtZfzseRpaFhMriymP9zS8tzKOrl3zXgZi7wm3PVOC fqctJuoK4AYjgCdohCFqYEIIQC5apsiyRZteoZgu8ePjhnX0ITPrROk59KhI304JwqCZ iJE+5BVjqDySdz6GS/bDO40RfDH1h63W9M5VAxyihognX7J2wDi4pK0BtHdi4cv7vjCc 0FVsLKe1GOmGb6n9pL2XMZBQI2K2a9FJ7r9WbyDV1favGGxCV1vLIB3RDspo4Upzd25G SXoR72goYVkL/7ulPhSNo1lu/4XaBsih1uC+X4rswl/fXNP6QSidcwl6ru3mSyTkTkO2 IIJg== X-Forwarded-Encrypted: i=1; AJvYcCVdK3bwPvIEgSBZ/vF22edRyfnIzZPy5I6Wl6ga2c9nU9TozrPcHAfBGltGUuHq2Dk2nUzeqQXqG5Lv8reSTIIhnVC8QvBnFw1IWBbo X-Gm-Message-State: AOJu0YyofxLj0EJoDpO+/hEeA0j0D5SC/Iok0UpTJ2KYAL45qvoe9JpT SqAM4TJ3X9MuqKF++zJcdN7wWuCcoetibS04Nu9+0C35eWKk6vzS6SSrgYzb8Fc= X-Google-Smtp-Source: AGHT+IGj8QfzxmFDjPa6B3ZrPipsx8DPw9AmRrI7+EDboHGSILtWSRt9gEHH8rAFh/SzcNpUcGsI0g== X-Received: by 2002:a17:90b:1043:b0:2c8:db8b:7247 with SMTP id 98e67ed59e1d1-2c8db8b72bemr1101586a91.9.1719394531645; Wed, 26 Jun 2024 02:35:31 -0700 (PDT) Received: from localhost.localdomain ([123.51.167.56]) by smtp.googlemail.com with ESMTPSA id 98e67ed59e1d1-2c8d7e58fb5sm1201499a91.12.2024.06.26.02.35.29 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 26 Jun 2024 02:35:31 -0700 (PDT) From: Jian-Hong Pan To: Bjorn Helgaas Cc: Johan Hovold , David Box , =?UTF-8?q?Ilpo=20J=C3=A4rvinen?= , Kuppuswamy Sathyanarayanan , Mika Westerberg , Damien Le Moal , Nirmal Patel , Jonathan Derrick , linux-pci@vger.kernel.org, linux-kernel@vger.kernel.org, linux@endlessos.org, Jian-Hong Pan Subject: [PATCH v7 1/4] PCI: vmd: Set PCI devices to D0 before enable PCI PM's L1 substates Date: Wed, 26 Jun 2024 17:34:15 +0800 Message-ID: <20240626093414.14226-2-jhp@endlessos.org> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20240626092821.14158-2-jhp@endlessos.org> References: <20240626092821.14158-2-jhp@endlessos.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" The remapped PCIe Root Port and the child device have PCI PM L1 substates capability, but they are disabled originally. Here is a failed example on ASUS B1400CEAE: Capabilities: [900 v1] L1 PM Substates L1SubCap: PCI-PM_L1.2+ PCI-PM_L1.1- ASPM_L1.2+ ASPM_L1.1- L1_PM_Sub= states+ PortCommonModeRestoreTime=3D32us PortTPowerOnTime=3D10us L1SubCtl1: PCI-PM_L1.2- PCI-PM_L1.1- ASPM_L1.2+ ASPM_L1.1- T_CommonMode=3D0us LTR1.2_Threshold=3D0ns L1SubCtl2: T_PwrOn=3D10us Power on all of the VMD remapped PCI devices to D0 before enable PCI-PM L1 PM Substates by following "PCIe r6.0, sec 5.5.4". Link: https://bugzilla.kernel.org/show_bug.cgi?id=3D218394 Signed-off-by: Jian-Hong Pan Reviewed-by: Kuppuswamy Sathyanarayanan --- v2: - Power on the VMD remapped devices with pci_set_power_state_locked() - Prepare the PCIe LTR parameters before enable L1 Substates - Add note into the comments of both pci_enable_link_state() and pci_enable_link_state_locked() for kernel-doc. - The original patch set can be split as individual patches. v3: - Re-send for the missed version information. - Split drivers/pci/pcie/aspm.c modification into following patches. - Fix the comment for enasuring the PCI devices in D0. v4: - The same v5: - Tweak the commit title and message - Change the goto label from out_enable_link_state to out_state_change v6, v7: - The same drivers/pci/controller/vmd.c | 13 +++++++++---- 1 file changed, 9 insertions(+), 4 deletions(-) diff --git a/drivers/pci/controller/vmd.c b/drivers/pci/controller/vmd.c index 87b7856f375a..5309afbe31f9 100644 --- a/drivers/pci/controller/vmd.c +++ b/drivers/pci/controller/vmd.c @@ -751,11 +751,9 @@ static int vmd_pm_enable_quirk(struct pci_dev *pdev, v= oid *userdata) if (!(features & VMD_FEAT_BIOS_PM_QUIRK)) return 0; =20 - pci_enable_link_state_locked(pdev, PCIE_LINK_STATE_ALL); - pos =3D pci_find_ext_capability(pdev, PCI_EXT_CAP_ID_LTR); if (!pos) - return 0; + goto out_state_change; =20 /* * Skip if the max snoop LTR is non-zero, indicating BIOS has set it @@ -763,7 +761,7 @@ static int vmd_pm_enable_quirk(struct pci_dev *pdev, vo= id *userdata) */ pci_read_config_dword(pdev, pos + PCI_LTR_MAX_SNOOP_LAT, <r_reg); if (!!(ltr_reg & (PCI_LTR_VALUE_MASK | PCI_LTR_SCALE_MASK))) - return 0; + goto out_state_change; =20 /* * Set the default values to the maximum required by the platform to @@ -775,6 +773,13 @@ static int vmd_pm_enable_quirk(struct pci_dev *pdev, v= oid *userdata) pci_write_config_dword(pdev, pos + PCI_LTR_MAX_SNOOP_LAT, ltr_reg); pci_info(pdev, "VMD: Default LTR value set by driver\n"); =20 +out_state_change: + /* + * Ensure devices are in D0 before enabling PCI-PM L1 PM Substates, per + * PCIe r6.0, sec 5.5.4. + */ + pci_set_power_state_locked(pdev, PCI_D0); + pci_enable_link_state_locked(pdev, PCIE_LINK_STATE_ALL); return 0; } =20 --=20 2.45.2 From nobody Wed Dec 17 23:53:19 2025 Received: from mail-pg1-f171.google.com (mail-pg1-f171.google.com [209.85.215.171]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 44BCF12B144 for ; Wed, 26 Jun 2024 09:38:25 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.215.171 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1719394706; cv=none; b=lyUoqJ8exVU6Ie84ymDkLDmuINE3RvVhUF0jW05f1XcrUesC3mAhqcA/n+eVvWeCjfTDP4G3MCWZZJZ6XAporUOUkmpMVeTm8UMias1mSOGJjXErNK5CJfn67GORMEZaV4r8cCaBsxMxrHh11aRwt5rgUSIm5ho03oGGmGrk368= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1719394706; c=relaxed/simple; bh=VMgxm/Ef7XHNkgrpqem5t2A+iBBaltsqNuNFFFlZROE=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=rTyEb5M8mWzt+uUaRdwXf71lSUj0hY+SMfcjPSEm98AaMHS78xpg7zDHq0I1LEP17OYlVNiMOsc7LOFmLUrt1M2xFMfbk2+8V4qTjDugY3F/mv4Z7M5mpyIS9RpmAxixnk7VS9RAfTSf2LxYapH1FyYTSbCVc+XJyPpNuwUYtYw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=endlessos.org; spf=pass smtp.mailfrom=endlessos.org; dkim=pass (2048-bit key) header.d=endlessos.org header.i=@endlessos.org header.b=hYPbyF7h; arc=none smtp.client-ip=209.85.215.171 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=endlessos.org Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=endlessos.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=endlessos.org header.i=@endlessos.org header.b="hYPbyF7h" Received: by mail-pg1-f171.google.com with SMTP id 41be03b00d2f7-70b2421471aso4601238a12.0 for ; Wed, 26 Jun 2024 02:38:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=endlessos.org; s=google; t=1719394704; x=1719999504; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=K8+v3noNoyPjynJix5Z6KWWDUhj0ABP/SpdUpbNFskg=; b=hYPbyF7hEXoNJRpZ0k0dZdUb3ZqCMxZenL9iP+0zd3armOWxzF+voEOFMx3ZvG+A7M 0rICP3hI1gTiAj2RRGphCUY3uZ55dx830DAM6lXhHiWQctnuU/N4AH1XKK5+0Xy2ckzd aAf+w0Q/O++2DSWO30h2PWykXH/QORroUdKKZpY1gIol7WwiOgiDpe589GVfjCTkBoM7 FXb30Pm8aoiShIqHQPykabZiJ935Q7kT95mEX5NpFuAh0+8L19ToHVa5JtjBE1Vw2Ipj LU/879XXLhIZj1stAqLB/tGrFJ8uq7csopfLODifEePZ0Zbr8EMEpVWByC8rd+4m+b1J Qs8Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1719394704; x=1719999504; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=K8+v3noNoyPjynJix5Z6KWWDUhj0ABP/SpdUpbNFskg=; b=T5QOApAStu44eiVnOehvdZqjYy44xapQrfvbOGe3YfHW65TuRGZpSOVIKdmp2MssBl EGkbRJgBjp/ZndTJ3AbdfkHARlvbR8JiLO/XXS+dj3cljPW/JusjmwhLLBTVnncKhujP fUbGKAMp+CtuZRcf93BzSm/zncf4pC/bL2QSCjZVdQSYSzG9EKWjYl9t5lfFvHVWI2g8 qnV1Z6wph09Vpm4FU3YHxWn+46kkvwH81dwm7RugvmAW9GHype+xBlodse+W5u8Zcm4R c7k6Z8Gk2RF+NK7ScAdcBEVscGwawmCe5GNLuH83Fs2samzRws4vIpkBpcQr6ai1Gz2F 6U6A== X-Forwarded-Encrypted: i=1; AJvYcCX4U9yeTTSSep2Bdt02nARITdq/DqshJGn0wMt7ohI0Ppu18O+Ok+TdcXpMFFEDgMDbdqRgPNkKtCeF4zu+BDcrCFx0kT7zzzzfKNBI X-Gm-Message-State: AOJu0YyDkzKa5td0Bx+rO4TVfV1U2fTVbhEcqF6eDyuyPASNW/lMY5br UwsX8s++shNSSbNCuBK8JQG0zAmqMlQrfjVsuuaMc77ecMcejZXNMTzjPw90RYc= X-Google-Smtp-Source: AGHT+IEC3KSzdYFibbSuDaV54ZQfYOq+YaBYF/YhFaYzGn4bpoJpyTfuPOdLMIDcdRzSetN4TJ0NrA== X-Received: by 2002:a05:6a20:4f24:b0:1bd:1972:370e with SMTP id adf61e73a8af0-1bd19723884mr4741524637.48.1719394704435; Wed, 26 Jun 2024 02:38:24 -0700 (PDT) Received: from localhost.localdomain ([123.51.167.56]) by smtp.googlemail.com with ESMTPSA id d9443c01a7336-1f9eb3c6d47sm95676685ad.178.2024.06.26.02.38.22 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 26 Jun 2024 02:38:24 -0700 (PDT) From: Jian-Hong Pan To: Bjorn Helgaas Cc: Johan Hovold , David Box , =?UTF-8?q?Ilpo=20J=C3=A4rvinen?= , Kuppuswamy Sathyanarayanan , Mika Westerberg , Damien Le Moal , Nirmal Patel , Jonathan Derrick , linux-pci@vger.kernel.org, linux-kernel@vger.kernel.org, linux@endlessos.org, Jian-Hong Pan , Bjorn Helgaas Subject: [PATCH v7 2/4] PCI/ASPM: Add notes about enabling PCI-PM L1SS to pci_enable_link_state(_locked) Date: Wed, 26 Jun 2024 17:37:00 +0800 Message-ID: <20240626093659.14346-2-jhp@endlessos.org> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20240626092821.14158-2-jhp@endlessos.org> References: <20240626092821.14158-2-jhp@endlessos.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" According to "PCIe r6.0, sec 5.5.4", add note about D0 requirement in pci_enable_link_state() kernel-doc. Signed-off-by: Jian-Hong Pan Acked-by: Bjorn Helgaas Reviewed-by: Kuppuswamy Sathyanarayanan --- v3: - Fix as readable comments v4: - The same v5: - Tweak and simplify the commit message v6, v7: - The same drivers/pci/pcie/aspm.c | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/drivers/pci/pcie/aspm.c b/drivers/pci/pcie/aspm.c index cee2365e54b8..bd0a8a05647e 100644 --- a/drivers/pci/pcie/aspm.c +++ b/drivers/pci/pcie/aspm.c @@ -1442,6 +1442,9 @@ static int __pci_enable_link_state(struct pci_dev *pd= ev, int state, bool locked) * touch the LNKCTL register. Also note that this does not enable states * disabled by pci_disable_link_state(). Return 0 or a negative errno. * + * Note: Ensure devices are in D0 before enabling PCI-PM L1 PM Substates, = per + * PCIe r6.0, sec 5.5.4. + * * @pdev: PCI device * @state: Mask of ASPM link states to enable */ @@ -1458,6 +1461,9 @@ EXPORT_SYMBOL(pci_enable_link_state); * can't touch the LNKCTL register. Also note that this does not enable st= ates * disabled by pci_disable_link_state(). Return 0 or a negative errno. * + * Note: Ensure devices are in D0 before enabling PCI-PM L1 PM Substates, = per + * PCIe r6.0, sec 5.5.4. + * * @pdev: PCI device * @state: Mask of ASPM link states to enable * --=20 2.45.2 From nobody Wed Dec 17 23:53:19 2025 Received: from mail-pj1-f44.google.com (mail-pj1-f44.google.com [209.85.216.44]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2A19761FEA for ; Wed, 26 Jun 2024 09:40:14 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.44 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1719394816; cv=none; b=Zur1CX5wgYWbW59TZfTWCg3qgjaxY6x+6brmwG+7BLPEhb+vIIKZEzLP51WvSuqU27N236OHWULbtPYdg9hAriByajengz9v+Ug5hHnqtFKFB9hQQqHqVP3BPHgDtzc/5hjIlfXeGlFFH0T7Zo9d2bo6RXDR1/CEaL7giUEODrU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1719394816; c=relaxed/simple; bh=t3x+yPoWJqcunaS/pKWvxbPII/x2hQ1qiSygdYUHM84=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=jyU0cBTd3akyLKJBtthNoY+3SeNU9A6boYfwYkbBsNe0yi8zyCP24oSU8nVZqWdFhZg3DLzaEKx/BGa75SLH3XM9/cUln7mb1zhilNludQngWgGbK9Q1a8wl/NVIJYPq+PJ9NgPyxhf/F1ljQC+Cu2EAgL8Vavf1plnjiATE0do= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=endlessos.org; spf=pass smtp.mailfrom=endlessos.org; dkim=pass (2048-bit key) header.d=endlessos.org header.i=@endlessos.org header.b=N0uEqOZa; arc=none smtp.client-ip=209.85.216.44 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=endlessos.org Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=endlessos.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=endlessos.org header.i=@endlessos.org header.b="N0uEqOZa" Received: by mail-pj1-f44.google.com with SMTP id 98e67ed59e1d1-2c80637ee79so4570537a91.0 for ; Wed, 26 Jun 2024 02:40:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=endlessos.org; s=google; t=1719394814; x=1719999614; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=EYkL/+OYvrPefgy3yyZJb4UdPCVnAP9ikoloQG2+LAg=; b=N0uEqOZaTh/aqHFkpOfdHSZQWREJGX+6QQIzadsbBdSCQvDDRqtnRjp23XHMQk8Nz0 vIZ3v1L1WC4j7oLxMsVSyOSTqwpAIcqVgbcz+jmJOA3cCcMBadMA4S03Gr2ygGXbE0xR U3E3rFigujCbNtXC6/yziIkTR6Q1btkyync3zxI4op/ZtciQGlYXCJzGEPoVhQpyB4uR eyrvbJ/DumIgg0qfJxhWUqFMcNzAmidwRLGmRfz1AVJKDwvlLv8Jq3FUuDYTYtk6s3ME B2eHBH1QEmhUsMYNKV+MnVGYus3dMF9jgDrd9VA42ngRuxaPfTsv1FgE2QnjZ/Z/j8tb GG4w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1719394814; x=1719999614; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=EYkL/+OYvrPefgy3yyZJb4UdPCVnAP9ikoloQG2+LAg=; b=GXbjgQi6mg1Am4XxwfNmhBWDOsMMUuHRCNaUaOsr0OKaZ5xKOpgm3S9jCBO4MNuDhb JlWxc8sCvCwMVkw7gkGulOeBntE/zGBLRUn2jGVjZKd04GncE7GEFxl4RnPx6+XK/7Nn 0/BnPufgIfsYWP601ogtoCT43kiEcQtflJNX9bjJ5I9EUrJaFutf2eqMkzAdbPScyarE k76Wsc9zzoHWUWgjtFdUm8JB9aHT5JnkPlgu2fk8YDqKTjGox3v7YuZJqR97sz30PxNN qHjXds5Bc5R+F15Qomh4b812PdGu/LCcPV/Ogc86deEtmaVOeVjAoh0gXkX0KEHnWpU4 Xnbw== X-Forwarded-Encrypted: i=1; AJvYcCUA6iiyNB9AMOWunn7xNkMfyxhGSO2We72schptKkVVAt1sLQ/m39B8iL36Lp4j/eYHzTKpb2mgC8T/j8I/YjjI7jBeezfQv2AxsoZE X-Gm-Message-State: AOJu0YwdyyCy69m6CQPszUN/mth9loCc2LOM4qNLtXbPzygG2ASjdeI5 GJrX4kKSJNnVjYO9BnGISEBstrW8wvevM1A115xDfdaV69pYxMdBo+HqZYx53dE= X-Google-Smtp-Source: AGHT+IG3oKwCrQlC6wo/9oxGXcDpBv5N7NMtxjzsIJEcW7oGLQArvKeicnb3ZhGwKn3EmwKnwHhuvQ== X-Received: by 2002:a17:90b:4c8f:b0:2c8:a8f:c97 with SMTP id 98e67ed59e1d1-2c86146c865mr7628141a91.37.1719394814219; Wed, 26 Jun 2024 02:40:14 -0700 (PDT) Received: from localhost.localdomain ([123.51.167.56]) by smtp.googlemail.com with ESMTPSA id 98e67ed59e1d1-2c8d806f8f7sm1199096a91.35.2024.06.26.02.40.12 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 26 Jun 2024 02:40:13 -0700 (PDT) From: Jian-Hong Pan To: Bjorn Helgaas Cc: Johan Hovold , David Box , =?UTF-8?q?Ilpo=20J=C3=A4rvinen?= , Kuppuswamy Sathyanarayanan , Mika Westerberg , Damien Le Moal , Nirmal Patel , Jonathan Derrick , linux-pci@vger.kernel.org, linux-kernel@vger.kernel.org, linux@endlessos.org, Jian-Hong Pan Subject: [PATCH v7 3/4] PCI/ASPM: Introduce aspm_get_l1ss_cap() Date: Wed, 26 Jun 2024 17:39:12 +0800 Message-ID: <20240626093911.14435-2-jhp@endlessos.org> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20240626092821.14158-2-jhp@endlessos.org> References: <20240626092821.14158-2-jhp@endlessos.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Introduce aspm_get_l1ss_cap() which is extracted from aspm_l1ss_init() to get the PCIe's L1SS capability. This does not change any behavior, but aspm_get_l1ss_cap() can be reused later. Link: https://bugzilla.kernel.org/show_bug.cgi?id=3D218394 Signed-off-by: Jian-Hong Pan Reviewed-by: David E. Box --- v6: - Skipped v7: - Pick back drivers/pci/pcie/aspm.c | 23 ++++++++++++++--------- 1 file changed, 14 insertions(+), 9 deletions(-) diff --git a/drivers/pci/pcie/aspm.c b/drivers/pci/pcie/aspm.c index bd0a8a05647e..5db1044c9895 100644 --- a/drivers/pci/pcie/aspm.c +++ b/drivers/pci/pcie/aspm.c @@ -611,6 +611,18 @@ static void pcie_aspm_check_latency(struct pci_dev *en= dpoint) } } =20 +static u32 aspm_get_l1ss_cap(struct pci_dev *pdev) +{ + u32 l1ss_cap; + + pci_read_config_dword(pdev, pdev->l1ss + PCI_L1SS_CAP, &l1ss_cap); + + if (!(l1ss_cap & PCI_L1SS_CAP_L1_PM_SS)) + l1ss_cap =3D 0; + + return l1ss_cap; +} + /* Calculate L1.2 PM substate timing parameters */ static void aspm_calc_l12_info(struct pcie_link_state *link, u32 parent_l1ss_cap, u32 child_l1ss_cap) @@ -721,15 +733,8 @@ static void aspm_l1ss_init(struct pcie_link_state *lin= k) return; =20 /* Setup L1 substate */ - pci_read_config_dword(parent, parent->l1ss + PCI_L1SS_CAP, - &parent_l1ss_cap); - pci_read_config_dword(child, child->l1ss + PCI_L1SS_CAP, - &child_l1ss_cap); - - if (!(parent_l1ss_cap & PCI_L1SS_CAP_L1_PM_SS)) - parent_l1ss_cap =3D 0; - if (!(child_l1ss_cap & PCI_L1SS_CAP_L1_PM_SS)) - child_l1ss_cap =3D 0; + parent_l1ss_cap =3D aspm_get_l1ss_cap(parent); + child_l1ss_cap =3D aspm_get_l1ss_cap(child); =20 /* * If we don't have LTR for the entire path from the Root Complex --=20 2.45.2 From nobody Wed Dec 17 23:53:19 2025 Received: from mail-pl1-f175.google.com (mail-pl1-f175.google.com [209.85.214.175]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id DCD1016CD11 for ; Wed, 26 Jun 2024 09:43:26 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.175 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1719395008; cv=none; b=WwbD7cRv+8ld2oNC8b0TwqkDiPBki/jZm/58uqOCUX4H8cTGTkqXRXFHoXI7ksfNu5hUXtjYbKIQSTvEKPPsJVCsCf8vED161FeG9DZroLmG0czMiV/U+PkALuSk+zI14/2DRkU12RppXw3dmkGMgw9aSAi6oYedcWxcDEVM090= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1719395008; c=relaxed/simple; bh=vDLv0eUVGJMmwyg+4UxkWlAksU175EmQrjDIno/a+Gs=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=se+CBTyiKXh4YPchytjF/sTMafiztJLI16fAFH/bJMCbD/VG1gqPAgES/LgbN8AbkySaX9nfeQGX8j5zpWeiH5+7T3FJFqG4GCp+B8FFqIpIgp/Upeq8WadgwCOdnyxDPAX2ECEr97oMcnJxllp3C17mnX6czqyu6+JA5teARDg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=endlessos.org; spf=pass smtp.mailfrom=endlessos.org; dkim=pass (2048-bit key) header.d=endlessos.org header.i=@endlessos.org header.b=vwpfgZQe; arc=none smtp.client-ip=209.85.214.175 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=endlessos.org Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=endlessos.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=endlessos.org header.i=@endlessos.org header.b="vwpfgZQe" Received: by mail-pl1-f175.google.com with SMTP id d9443c01a7336-1f47f07acd3so53363565ad.0 for ; Wed, 26 Jun 2024 02:43:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=endlessos.org; s=google; t=1719395006; x=1719999806; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=jWFJ83bKSjpBaUA+SG/yPoduHd16W9GZx5KqGR4AgEA=; b=vwpfgZQeL49q6ymP0Kpc4x4eSMcAgLK+4DK6siPxGI2rrqqD38Hb0IyQZCAs+EwhB+ ZzSlEArQj1AE1RzTSYetjRvmXQ6VqBFE1LEis13WUE6PCLtJ9mnbNEpbutt0SlSeAoXt F1OH/iDOZxJkt9TSIKrBSwc1dWSGUi95mAeMmO13EIbvH/SYKF7PqmNJmAHgoSUDEO8T mRIRIGNgxBM1g4LwDjLYUxh25ku3Lgzrk7BJTsht+TETRWogr1qmkF5+cih+nHfVAoEP y8iGYH5nRWgSQonc8Dlko6skMXdqrmQLkqx0b4K6IQbG/ZH3gBICbw71FO7IRQMDFXWj fMKQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1719395006; x=1719999806; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=jWFJ83bKSjpBaUA+SG/yPoduHd16W9GZx5KqGR4AgEA=; b=EvGRiA8MM8MQdfgWDs+roOFSBO57wHic4UGjq+3RDbou9CtEXRJCgTg2w2lLDzdhjs OKwIQUetJAzGWThrFiCHno+zVOZA9OICqq32JvD6BUhz+mH0V6dqtldxO5I3sep3aTMJ KzjwvMI1gPhwB+ZFIsYuZsCwZAhwytTl5klW7HzHIrEWlio877xKaioxNehKKqjdMiX+ v4uSfB9efn5jQhMy3qyDJjG5FK7MdoTXB3nIO+m3F2PSG7CgnBpcyDp3RevLTcniMTfC 0AvO5i3uzDT/v4J1Xq9rtD1LFrBEHV5Yd5jOOrDdiU6/eaQURfTfAWB+xjbitzDFeIqB hxVw== X-Forwarded-Encrypted: i=1; AJvYcCXnWujWUGUCNq+P4/GBT9WbyUmg+w+KE+walCssmATowuFNFWCbyawElGMS8LDdDFl56l4QV6ggeXy5DXeihhMlMkgWDDIbhDDXf2Iw X-Gm-Message-State: AOJu0YxjC3UjQBOkWPUgFPkbE6GbYjcoSQ9b67uEqI53vxSSTRr8nUaD lFvH1oQhKavDAH85b9SqLkLvsgHq/v9T33WGF94ilyDbM23z42UH63qK6Llglmw= X-Google-Smtp-Source: AGHT+IE2etjp6SLgpYIk6NOvVt25jF7m5xfwbrxRBUDv9rWsMCAEdZz8pW2+kWTCkFtQsPXiOao3DA== X-Received: by 2002:a17:902:ecc8:b0:1f9:c289:737c with SMTP id d9443c01a7336-1fa1d683c1emr116520375ad.60.1719395006080; Wed, 26 Jun 2024 02:43:26 -0700 (PDT) Received: from localhost.localdomain ([123.51.167.56]) by smtp.googlemail.com with ESMTPSA id d9443c01a7336-1f9eb5e030dsm95700375ad.218.2024.06.26.02.43.23 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 26 Jun 2024 02:43:25 -0700 (PDT) From: Jian-Hong Pan To: Bjorn Helgaas Cc: Johan Hovold , David Box , =?UTF-8?q?Ilpo=20J=C3=A4rvinen?= , Kuppuswamy Sathyanarayanan , Mika Westerberg , Damien Le Moal , Nirmal Patel , Jonathan Derrick , Paul M Stillwell Jr , linux-pci@vger.kernel.org, linux-kernel@vger.kernel.org, linux@endlessos.org, Jian-Hong Pan Subject: [PATCH v7 4/4] PCI/ASPM: Fix L1.2 parameters when enable link state Date: Wed, 26 Jun 2024 17:41:16 +0800 Message-ID: <20240626094115.14470-2-jhp@endlessos.org> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20240626092821.14158-2-jhp@endlessos.org> References: <20240626092821.14158-2-jhp@endlessos.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Currently, when enable link's L1.2 features with __pci_enable_link_state(), it configs the link directly without ensuring related L1.2 parameters, such as T_POWER_ON, Common_Mode_Restore_Time, and LTR_L1.2_THRESHOLD have been programmed. This leads the link's L1.2 between PCIe Root Port and child device gets wrong configs when a caller tries to enabled it. Here is a failed example on ASUS B1400CEAE with enabled VMD: 10000:e0:06.0 PCI bridge: Intel Corporation 11th Gen Core Processor PCIe Co= ntroller (rev 01) (prog-if 00 [Normal decode]) ... Capabilities: [200 v1] L1 PM Substates L1SubCap: PCI-PM_L1.2+ PCI-PM_L1.1+ ASPM_L1.2+ ASPM_L1.1+ L1_PM_Sub= states+ PortCommonModeRestoreTime=3D45us PortTPowerOnTime=3D50us L1SubCtl1: PCI-PM_L1.2- PCI-PM_L1.1- ASPM_L1.2+ ASPM_L1.1- T_CommonMode=3D45us LTR1.2_Threshold=3D101376ns L1SubCtl2: T_PwrOn=3D50us 10000:e1:00.0 Non-Volatile memory controller: Sandisk Corp WD Blue SN550 NV= Me SSD (rev 01) (prog-if 02 [NVM Express]) ... Capabilities: [900 v1] L1 PM Substates L1SubCap: PCI-PM_L1.2+ PCI-PM_L1.1- ASPM_L1.2+ ASPM_L1.1- L1_PM_Sub= states+ PortCommonModeRestoreTime=3D32us PortTPowerOnTime=3D10us L1SubCtl1: PCI-PM_L1.2- PCI-PM_L1.1- ASPM_L1.2+ ASPM_L1.1- T_CommonMode=3D0us LTR1.2_Threshold=3D0ns L1SubCtl2: T_PwrOn=3D10us According to "PCIe r6.0, sec 5.5.4", before enabling ASPM L1.2 on the PCIe Root Port and the child NVMe, they should be programmed with the same LTR1.2_Threshold value. However, they have different values in this case. Invoke aspm_calc_l12_info() to program the L1.2 parameters properly before enable L1.2 bits of L1 PM Substates Control Register in __pci_enable_link_state(). Link: https://bugzilla.kernel.org/show_bug.cgi?id=3D218394 Signed-off-by: Jian-Hong Pan --- v2: - Prepare the PCIe LTR parameters before enable L1 Substates v3: - Only enable supported features for the L1 Substates part v4: - Focus on fixing L1.2 parameters, instead of re-initializing whole L1SS v5: - Fix typo and commit message - Split introducing aspm_get_l1ss_cap() to "PCI/ASPM: Introduce aspm_get_l1ss_cap()" v6: - Skipped v7: - Pick back and rebase on the new version kernel - Drop the link state flag check. And, always config link state's timing parameters drivers/pci/pcie/aspm.c | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/drivers/pci/pcie/aspm.c b/drivers/pci/pcie/aspm.c index 5db1044c9895..7f2cdda259dc 100644 --- a/drivers/pci/pcie/aspm.c +++ b/drivers/pci/pcie/aspm.c @@ -1411,6 +1411,8 @@ EXPORT_SYMBOL(pci_disable_link_state); static int __pci_enable_link_state(struct pci_dev *pdev, int state, bool l= ocked) { struct pcie_link_state *link =3D pcie_aspm_get_link(pdev); + struct pci_dev *child =3D link->downstream, *parent =3D link->pdev; + u32 parent_l1ss_cap, child_l1ss_cap; =20 if (!link) return -EINVAL; @@ -1428,6 +1430,15 @@ static int __pci_enable_link_state(struct pci_dev *p= dev, int state, bool locked) if (!locked) down_read(&pci_bus_sem); mutex_lock(&aspm_lock); + /* + * Ensure L1.2 parameters: Common_Mode_Restore_Times, T_POWER_ON and + * LTR_L1.2_THRESHOLD are programmed properly before enable bits for + * L1.2, per PCIe r6.0, sec 5.5.4. + */ + parent_l1ss_cap =3D aspm_get_l1ss_cap(parent); + child_l1ss_cap =3D aspm_get_l1ss_cap(child); + aspm_calc_l12_info(link, parent_l1ss_cap, child_l1ss_cap); + link->aspm_default =3D pci_calc_aspm_enable_mask(state); pcie_config_aspm_link(link, policy_to_aspm_state(link)); =20 --=20 2.45.2