From nobody Fri Sep 25 16:56:34 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 216844749E0; Thu, 10 Sep 2026 11:32:27 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789039950; cv=none; b=mTCxa+jNRCBqpaFjQ/l/xxhhxZy2Ys2HWiyHk8ghagRPGtLyt/MipO6+Wf0J1lmP0d3hQOE7S/0mzG/ECErWIXz/Tk4hm6QntN4qZrrw3e1gpQ3EBPhhdRpsRPgwsxxcNhDQDxJ/D4j1hlnBG2JtMMtXQVpfTyoh+Tv3Uq0Js0E= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789039950; c=relaxed/simple; bh=sNSKCsf/sqMm/uW6DJDsKzV+8NFWqca8nOKcDmgRicE=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=NptTr0JB+IqEN/QURvMQfioRuIOQTwN8IcJWTXe/sRRxheiUHFL0oWM1PDfVSmRyZStoOc4mBR0KNDBuT8JMgf6fgzodd+DODdnq6WvEdkiMrw/GHzpJ655x814I/i6V+PnfOTBoIjUqoQeVeFwi+XgYqSXNVBVxkPJzlor2WM4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=Ra4n7vOl; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="Ra4n7vOl" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 16EE31F000FF; Thu, 10 Sep 2026 11:32:26 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1789039946; bh=mewnXaBKzi2RuajAdR+1gFX2UcD8vJrlfBBh0mRM42E=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=Ra4n7vOl0p6AbaxDDTzha13CuM7FIoHC/LtcZWhlEZa3wjszkfLr9DgxKlCPXomjr EAacQV1plKqvRzGJZDjID8gASAiPZgNBga6H67V9izFK5wOx8pISTAm3C/Pk3Shorb N9RX7Q7UmObdzlwxXhaYViVa3ILf7rYS9cst/BpbxlfD0Fhe7hTQ/dSSX7vVEympXe Z4rzdinSgRHzE+M3wS3EyaHpU/hWIXIFUBzIiQfwYNxaZjRLk+lPJlD8on5WgVncUm EHpGKv3jl9epUY3zUhPzUuEC0+7s+V3zFm3q/ho/qMua8JQ9YqD02sD9LC0Hx2VWSe Oq3wMMCdxpqCg== From: Leon Romanovsky To: Bjorn Helgaas , Logan Gunthorpe , Chaitanya Kulkarni , Greg Kroah-Hartman , Jens Axboe , Alex Williamson , Leon Romanovsky , Ankit Agrawal , Jason Gunthorpe , Jonathan Corbet , Shuah Khan , "Joerg Roedel (AMD)" , Will Deacon , Robin Murphy , Randy Dunlap , Sumit Semwal , =?utf-8?q?Christian_K=C3=B6nig?= Cc: linux-pci@vger.kernel.org, linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, iommu@lists.linux.dev, Tushar Dave , linux-media@vger.kernel.org, dri-devel@lists.freedesktop.org, linaro-mm-sig@lists.linaro.org, linux-rdma@vger.kernel.org, kvm@vger.kernel.org, Matt Evans Subject: [PATCH v5 01/18] PCI/P2PDMA: Document pdev->p2pdma lifetime rules Date: Thu, 10 Sep 2026 14:31:56 +0300 Message-ID: <20260910-fix-p2p-acs-v4-0-v5-1-856087f63c0d@nvidia.com> X-Mailer: git-send-email 2.55.0 In-Reply-To: <20260910-fix-p2p-acs-v4-0-v5-0-856087f63c0d@nvidia.com> References: <20260910-fix-p2p-acs-v4-0-v5-0-856087f63c0d@nvidia.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" X-Mailer: b4 0.15-dev-18f8f Content-Transfer-Encoding: quoted-printable From: Leon Romanovsky pdev->p2pdma is normally accessed through provider-owned references, but pci_p2pmem_find_many() and the p2pmem sysfs attributes may race with driver unbind. Document initialization, teardown, and how the release grace period protects both the P2PDMA state and its optional allocation pool. Cc: Alex Williamson Cc: Matt Evans Signed-off-by: Leon Romanovsky --- drivers/pci/p2pdma.c | 23 +++++++++++++++++++++++ 1 file changed, 23 insertions(+) diff --git a/drivers/pci/p2pdma.c b/drivers/pci/p2pdma.c index 4e4d2df17a45..d8c9d605b828 100644 --- a/drivers/pci/p2pdma.c +++ b/drivers/pci/p2pdma.c @@ -21,6 +21,29 @@ #include #include =20 +/* + * Lifetime and RCU usage + * + * Within one driver bind, pdev->p2pdma is set exactly once, + * by pcim_p2pdma_init(), and cleared exactly once, by the pci_p2pdma_rele= ase() + * devres action that the same function installs. It is never changed and = follow + * same lifetime as parent pdev. + * + * Most exported entry points reach pdev->p2pdma through a struct pci_dev = or a + * struct p2pdma_provider owned by the provider driver, and + * pcim_p2pdma_provider() requires callers to drop those references before= the + * driver's remove() completes. Those cannot run concurrently with + * pci_p2pdma_release(), and their rcu_dereference() calls are simply how = an + * __rcu pointer is read. + * + * pci_p2pmem_find_many() and the p2pmem sysfs attributes are the exceptio= ns. + * The first walks every PCI device, so it can reach a provider whose driv= er is + * unbinding: pci_get_device() pins the struct pci_dev, not the driver. The + * second is reachable from userspace until sysfs_remove_group() runs at t= he end + * of the release. pci_has_p2pmem() must dereference the object to determi= ne + * whether it owns a gen_pool, so even a device without a pool must remain= alive + * until that RCU reader exits. + */ struct pci_p2pdma { struct gen_pool *pool; bool p2pmem_published; --=20 2.55.0 From nobody Fri Sep 25 16:56:34 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2F681420496; Thu, 10 Sep 2026 11:32:23 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789039947; cv=none; b=GLVX3RBydzY4dNrYhOeiGRFZb74MAQ5jBFwT47Uyh43GaBEKbfI2oiTWnW8GVojRfUAw+sHJPKviM6m9sjhhHp6Ws8n2pmM12Y+x4GDIXn6jDGeKVP70JWzMAbkPSROw19I+qtR18Vj3cauxiLgWz5sZ/WSDFAWeh5sCvkO+Zek= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789039947; c=relaxed/simple; bh=Lf8pxsTHkin8iU9aiJA0CK4BtBgEr+wkCX1jxEk/GQE=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=DfP3cqH5b6DeVkiLwgrtPtUG8aFIWiMvx405hEHV10b93hZZ+GO1MTE0R79uZidy8BWNL0eKkmFDIjvF3oqKsmAEUjshbARoEa/EE/ffHbNxAMkBPiJ4dxcgIuCH+8RxR3Pard9QD6qLxn81UJU7dUO7Kc/1yKVTTOTA85gVIY8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=MvR7VdmN; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="MvR7VdmN" Received: by smtp.kernel.org (Postfix) with ESMTPSA id E7CB71F00893; Thu, 10 Sep 2026 11:32:22 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1789039943; bh=3YMYg7jFStAT+n5lMS0MTimOeAZfs69FZvv3CJKO2+g=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=MvR7VdmN1scind3AdnaOAELtbyQmi0B+dJa+PqXOYsuDjSEO9ZJ+87W81CHqQBNmV x3ZJd9VYw1Zd7r389htNRzHSb00pDhWon6/15rgw++/0Gx+maCoPq6UMiULdrcTkfn TfTtvSRbrBxxY4ZnYLjGaCvSTaT0KCT8+2qGJF+1Hp6tcpp0DZYeDFtuJv1mxl8pKy 3WX2W53XMXo4p7GIxY0HyBf1p86SSsJtxeRSn/fm9b3QgLUM9nQK/hE3df+HajrOns azfEOHyXCTxaerEr97OWEj9U5F8UbII4ztD91+fkxI1iC16Q4RxZmnDCArmH0aBMDf 1OXrqK9EtOoqA== From: Leon Romanovsky To: Bjorn Helgaas , Logan Gunthorpe , Chaitanya Kulkarni , Greg Kroah-Hartman , Jens Axboe , Alex Williamson , Leon Romanovsky , Ankit Agrawal , Jason Gunthorpe , Jonathan Corbet , Shuah Khan , "Joerg Roedel (AMD)" , Will Deacon , Robin Murphy , Randy Dunlap , Sumit Semwal , =?utf-8?q?Christian_K=C3=B6nig?= Cc: linux-pci@vger.kernel.org, linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, iommu@lists.linux.dev, Tushar Dave , linux-media@vger.kernel.org, dri-devel@lists.freedesktop.org, linaro-mm-sig@lists.linaro.org, linux-rdma@vger.kernel.org, kvm@vger.kernel.org Subject: [PATCH v5 02/18] PCI/P2PDMA: Document the TLP attribute assumptions Date: Thu, 10 Sep 2026 14:31:57 +0300 Message-ID: <20260910-fix-p2p-acs-v4-0-v5-2-856087f63c0d@nvidia.com> X-Mailer: git-send-email 2.55.0 In-Reply-To: <20260910-fix-p2p-acs-v4-0-v5-0-856087f63c0d@nvidia.com> References: <20260910-fix-p2p-acs-v4-0-v5-0-856087f63c0d@nvidia.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" X-Mailer: b4 0.15-dev-18f8f Content-Transfer-Encoding: quoted-printable From: Leon Romanovsky P2PDMA selects a mapping without receiving the Request's ordering or Address Type attributes. Its ACS handles only strictly ordered Requests carrying an Untranslated address. Document that the result is not defined for Relaxed Ordering or ATS-translated Requests because those TLP attributes can select different routes through the fabric. Reviewed-by: Logan Gunthorpe Signed-off-by: Leon Romanovsky --- Documentation/driver-api/pci/p2pdma.rst | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/Documentation/driver-api/pci/p2pdma.rst b/Documentation/driver= -api/pci/p2pdma.rst index 63cff9e4d2c9..80f8fec9b0e9 100644 --- a/Documentation/driver-api/pci/p2pdma.rst +++ b/Documentation/driver-api/pci/p2pdma.rst @@ -15,6 +15,13 @@ then based on the ACS settings the transaction can route= entirely within the PCIe hierarchy and never reach the root port. The kernel will evaluate the PCIe topology and always permit P2P in these well-defined cases. =20 +This evaluation assumes clients issue strictly ordered Requests carrying an +Untranslated address. Its result is not defined when clients use Relaxed +Ordering or issue ATS-translated Requests because those TLP attributes can +select different routes through the fabric. Unless ACS Translation Blocking +is enabled, a Port with ACS Direct Translated P2P enabled routes a +Translated Request directly to the peer regardless of the redirect control= s. + However, if the P2P transaction reaches the host bridge then it might have= to hairpin back out the same root port, be routed inside the CPU SOC to anoth= er PCIe root port, or routed internally to the SOC. --=20 2.55.0 From nobody Fri Sep 25 16:56:34 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id AD4E54749DB; Thu, 10 Sep 2026 11:32:45 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789039968; cv=none; b=jGOdeGjABYQhP31xk9wB3F3vVsYhUAbpOdpkJYScZj8H1dr0QGzDQ26RTFuFIiDfjzD2C/CykyK9Ff4yVBbZFyLHkT4bPicowuEtW1cW/7e+qLLGiaqxMhsfU88qsFHzAJ5zdeVmF0M1FV5G2cJfwQJom5Nkku7jVwG4QzckdSA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789039968; c=relaxed/simple; bh=iAVncddmE5MxAK6kUwA7UXhbZGTwi7fd4RHqWbBg4PM=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=r1SRMSkFA089x0Wu7YJC9Bg9P9SSmwrPGy0mk73PrBWPMYaRCWpOUox9MDpiQfaI5wL7NgCVr6DTtuxrO0WKKoP9wWTFUJNmJFDs9e0cxO6mXW8kwabAQ3H2HxNnZRIDReZnPcUHGzYdYuaGfudWQEYAbYLqeZFXqDILxT9N/Fc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=YJoM5oNQ; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="YJoM5oNQ" Received: by smtp.kernel.org (Postfix) with ESMTPSA id C4D4A1F00898; Thu, 10 Sep 2026 11:32:43 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1789039964; bh=OwvFFwMONbtdqqV/gBRe9/xM6CrDdlgyRpvDVfwgLec=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=YJoM5oNQ9mMwWte5t2arTUdDYL3nW5CcnZZQ8kDNNglxugCnRx79wQm3RRrzApPQ0 8KrPglOzC7ZvDW8/m/hDw9MBfYYSnbpZtPPNxiChUrjdoRP8aA4OK1GoP+inJNfu0w NWDfSjpb79dqvCRqcr7VUltMt3cMsy6mKFTUIu4tVe6KC9p12ql6AkuNDLhXUCcFoz SYtddo1cMAH46bDsExnfeCz4MqytKQuAaDwsk2RVKPk4Il8k4pJg5vSeUxHH52ghgT fkKHIpwfMcPIHt+/DG9Fas7OiJ373kvlPuqn4qjY7aW9+m+tlnbnp9Z+PRRoim+9qT L+skzzSz27OQg== From: Leon Romanovsky To: Bjorn Helgaas , Logan Gunthorpe , Chaitanya Kulkarni , Greg Kroah-Hartman , Jens Axboe , Alex Williamson , Leon Romanovsky , Ankit Agrawal , Jason Gunthorpe , Jonathan Corbet , Shuah Khan , "Joerg Roedel (AMD)" , Will Deacon , Robin Murphy , Randy Dunlap , Sumit Semwal , =?utf-8?q?Christian_K=C3=B6nig?= Cc: linux-pci@vger.kernel.org, linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, iommu@lists.linux.dev, Tushar Dave , linux-media@vger.kernel.org, dri-devel@lists.freedesktop.org, linaro-mm-sig@lists.linaro.org, linux-rdma@vger.kernel.org, kvm@vger.kernel.org Subject: [PATCH v5 03/18] PCI/P2PDMA: Derive routing from directional ACS controls Date: Thu, 10 Sep 2026 14:31:58 +0300 Message-ID: <20260910-fix-p2p-acs-v4-0-v5-3-856087f63c0d@nvidia.com> X-Mailer: git-send-email 2.55.0 In-Reply-To: <20260910-fix-p2p-acs-v4-0-v5-0-856087f63c0d@nvidia.com> References: <20260910-fix-p2p-acs-v4-0-v5-0-856087f63c0d@nvidia.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" X-Mailer: b4 0.15-dev-18f8f Content-Transfer-Encoding: quoted-printable From: Leon Romanovsky pci_bridge_has_acs_redir() treats Request and Completion Redirect as interchangeable. On asymmetric fabrics, a control for only the reverse TLP direction can unnecessarily force P2PDMA through the host bridge. Evaluate Request Redirect for client Requests and Completion Redirect for provider read Completions. Continue treating enabled Egress Control conservatively as a Request redirect. Fixes: 52916982af48 ("PCI/P2PDMA: Support peer-to-peer memory") Signed-off-by: Leon Romanovsky --- drivers/pci/p2pdma.c | 75 ++++++++++++++++++++++++++++++++++++++++--------= ---- 1 file changed, 58 insertions(+), 17 deletions(-) diff --git a/drivers/pci/p2pdma.c b/drivers/pci/p2pdma.c index d8c9d605b828..6da68ec99601 100644 --- a/drivers/pci/p2pdma.c +++ b/drivers/pci/p2pdma.c @@ -21,6 +21,8 @@ #include #include =20 +#include "pci.h" + /* * Lifetime and RCU usage * @@ -513,26 +515,56 @@ static struct pci_dev *find_parent_pci_dev(struct dev= ice *dev) return NULL; } =20 +enum pci_acs_p2pdma_state { + PCI_ACS_P2PDMA_DIRECT, + PCI_ACS_P2PDMA_REDIRECT, +}; + /* - * Check if a PCI bridge has its ACS redirection bits set to redirect P2P - * TLPs upstream via ACS. Returns 1 if the packets will be redirected - * upstream, 0 otherwise. + * Decide how a peer-to-peer Request at an ACS-capable ingress port routes, + * from that port's ACS Control register. + * + * Linux does not read the Egress Control Vector, so Egress Control is tre= ated + * conservatively as a redirect. Per PCIe r7.0 Table 6-11 the outcomes it + * selects are a direct route and an ACS Violation, and neither one lets p= eer + * bus addressing be assumed. */ -static int pci_bridge_has_acs_redir(struct pci_dev *pdev) +static enum pci_acs_p2pdma_state +pci_acs_p2pdma_request(u16 ctrl) { - int pos; - u16 ctrl; + return ctrl & (PCI_ACS_RR | PCI_ACS_EC) ? + PCI_ACS_P2PDMA_REDIRECT : PCI_ACS_P2PDMA_DIRECT; +} =20 - pos =3D pdev->acs_cap; - if (!pos) - return 0; +/* + * Decide how a peer-to-peer Completion at an ACS-capable ingress port rou= tes. + * PCIe r7.0 sec 6.12.1.1: no ACS control other than P2P Completion Redire= ct + * affects a Completion. + */ +static enum pci_acs_p2pdma_state +pci_acs_p2pdma_completion(u16 ctrl) +{ + return ctrl & PCI_ACS_CR ? PCI_ACS_P2PDMA_REDIRECT : + PCI_ACS_P2PDMA_DIRECT; +} =20 - pci_read_config_word(pdev, pos + PCI_ACS_CTRL, &ctrl); +/* + * Read @pdev's ACS Control register. A device without an ACS capability h= as + * no peer-to-peer controls at all, which routes the same as having them a= ll + * clear. Returns false when the register is present but cannot be read; @= ctrl + * is then meaningless. + */ +static bool pci_acs_p2pdma_ctrl(struct pci_dev *pdev, u16 *ctrl) +{ + int pos; =20 - if (ctrl & (PCI_ACS_RR | PCI_ACS_CR | PCI_ACS_EC)) - return 1; + pos =3D pdev->acs_cap; + if (!pos) { + *ctrl =3D 0; + return true; + } =20 - return 0; + return !pci_read_config_word(pdev, pos + PCI_ACS_CTRL, ctrl); } =20 static void seq_buf_print_bus_devfn(struct seq_buf *buf, struct pci_dev *p= dev) @@ -721,6 +753,10 @@ static unsigned long map_types_idx(struct pci_dev *cli= ent) * then to Device B. The mapping type returned depends on the ACS * redirection setting of the ports along the path. * + * The client initiates Requests to provider memory. Check Request Redirect + * on the client path and Completion Redirect for read Completions on the + * provider path. + * * If ACS redirect is set on any port in the path, traffic between the * devices will go through the host bridge, so return * PCI_P2PDMA_MAP_THRU_HOST_BRIDGE; otherwise return @@ -744,6 +780,7 @@ calc_map_type_and_dist(struct pci_dev *provider, struct= pci_dev *client, int dist_a =3D 0; int dist_b =3D 0; char buf[128]; + u16 ctrl; =20 seq_buf_init(&acs_list, buf, sizeof(buf)); =20 @@ -755,7 +792,9 @@ calc_map_type_and_dist(struct pci_dev *provider, struct= pci_dev *client, while (a) { dist_b =3D 0; =20 - if (pci_bridge_has_acs_redir(a)) { + if (!pci_acs_p2pdma_ctrl(a, &ctrl) || + pci_acs_p2pdma_completion(ctrl) =3D=3D + PCI_ACS_P2PDMA_REDIRECT) { seq_buf_print_bus_devfn(&acs_list, a); acs_cnt++; } @@ -784,7 +823,9 @@ calc_map_type_and_dist(struct pci_dev *provider, struct= pci_dev *client, if (a =3D=3D bb) break; =20 - if (pci_bridge_has_acs_redir(bb)) { + if (!pci_acs_p2pdma_ctrl(bb, &ctrl) || + pci_acs_p2pdma_request(ctrl) =3D=3D + PCI_ACS_P2PDMA_REDIRECT) { seq_buf_print_bus_devfn(&acs_list, bb); acs_cnt++; } @@ -1132,10 +1173,10 @@ EXPORT_SYMBOL_GPL(pci_p2pmem_publish); /** * pci_p2pdma_map_type - Determine the mapping type for P2PDMA transfers * @provider: P2PDMA provider structure - * @dev: Target device for the transfer + * @dev: Client device that initiates the transfer * * Determines how peer-to-peer DMA transfers should be mapped between - * the provider and the target device. The mapping type indicates whether + * the provider and the client device. The mapping type indicates whether * the transfer can be done directly through PCI switches or must go * through the host bridge. */ --=20 2.55.0 From nobody Fri Sep 25 16:56:34 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E3F7E47534D; Thu, 10 Sep 2026 11:32:30 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789039953; cv=none; b=ONtIgDhguXNercaGuBPt0uzBiXLmzCg/GtfF0tOjHkRu2E8mrvr10PE4x/Hii5jL4c5WQzCB4LS4FKhHQ0WIeGYef6hieOkIrNW9xbAkc1BWTgOoKtu/HnIkr85scOi2xJI8Y50/D4LHYCKkM2KKAiz2oeIqBZ+S1VXkZ6K2Fho= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789039953; c=relaxed/simple; bh=YHMSGoWo12kqHH/9jpnEItTIAqyqxLhgK1rYGVpBOuQ=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=nGcKnRiAfhELSvwArAUL37mjyGdzo9MYGYtIX0X9IsnpKuVy37r5PMUJkAxzRYzfFCH6ql3tuBUyt/3CffTd53KfOWfmHt5KeJMg/4xZzU7VXH+zXLFD0N0uwzXUqFc08XgmaaPuQQLOgeZhWTxiAWApY+KV97dN9CcVKEYYG8E= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=i6Kz/EyJ; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="i6Kz/EyJ" Received: by smtp.kernel.org (Postfix) with ESMTPSA id A3AD81F00898; Thu, 10 Sep 2026 11:32:29 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1789039950; bh=EQ+G8TpB9QqK14VWN1C6DtN6XM6oOCSfCS60D48WQ4Q=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=i6Kz/EyJE544HHR5myLA2OC1c9bhuT8hjZgnQYVGM15EoE0RqhkZocMi0v6ldF9RN Jx2bO7tPtvmGM5ZwjKOg/PMPvHruMB1H/4zly6nR0DMwiSC7JqmMDguwLSjAfxhjR6 K/zdkLV/jUCA/dKuMc/qOoaP52D2BvYsNAbVI/sLipCjONekr6OyhVdpYvXtUIoyk2 SklsiBFvjA4NqtRBn+cdp4Hz9Jjhszh6ftjepwhsgHUggT1POKyN0YVdc+6+YnTGv5 +CCjPDcaL7MD9+N6R1AnNrCiaXlEXF436bOSYyhetdxQMpHRmQe5tQP2ClRR3obBql 4M2coZDzSfzMw== From: Leon Romanovsky To: Bjorn Helgaas , Logan Gunthorpe , Chaitanya Kulkarni , Greg Kroah-Hartman , Jens Axboe , Alex Williamson , Leon Romanovsky , Ankit Agrawal , Jason Gunthorpe , Jonathan Corbet , Shuah Khan , "Joerg Roedel (AMD)" , Will Deacon , Robin Murphy , Randy Dunlap , Sumit Semwal , =?utf-8?q?Christian_K=C3=B6nig?= Cc: linux-pci@vger.kernel.org, linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, iommu@lists.linux.dev, Tushar Dave , linux-media@vger.kernel.org, dri-devel@lists.freedesktop.org, linaro-mm-sig@lists.linaro.org, linux-rdma@vger.kernel.org, kvm@vger.kernel.org Subject: [PATCH v5 04/18] PCI: Reject unreadable ACS controls in isolation checks Date: Thu, 10 Sep 2026 14:31:59 +0300 Message-ID: <20260910-fix-p2p-acs-v4-0-v5-4-856087f63c0d@nvidia.com> X-Mailer: git-send-email 2.55.0 In-Reply-To: <20260910-fix-p2p-acs-v4-0-v5-0-856087f63c0d@nvidia.com> References: <20260910-fix-p2p-acs-v4-0-v5-0-856087f63c0d@nvidia.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" X-Mailer: b4 0.15-dev-18f8f Content-Transfer-Encoding: quoted-printable From: Leon Romanovsky pci_acs_flags_enabled() and the Intel SPT PCH quirk use ACS registers without checking config-space read errors. A failed read may leave control state indeterminate yet allow the device to satisfy requested isolation controls. Return false when either ACS capability or control state cannot be read. An unknown state cannot prove isolation. Signed-off-by: Leon Romanovsky --- drivers/pci/pci.c | 3 ++- drivers/pci/quirks.c | 6 ++++-- 2 files changed, 6 insertions(+), 3 deletions(-) diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c index b2879a6be5f8..f7d94ecf9157 100644 --- a/drivers/pci/pci.c +++ b/drivers/pci/pci.c @@ -3594,7 +3594,8 @@ static bool pci_acs_flags_enabled(struct pci_dev *pde= v, u16 acs_flags) */ acs_flags &=3D (pdev->acs_capabilities | PCI_ACS_EC); =20 - pci_read_config_word(pdev, pos + PCI_ACS_CTRL, &ctrl); + if (pci_read_config_word(pdev, pos + PCI_ACS_CTRL, &ctrl)) + return false; return (ctrl & acs_flags) =3D=3D acs_flags; } =20 diff --git a/drivers/pci/quirks.c b/drivers/pci/quirks.c index de9bbccda21f..d5c3e6802840 100644 --- a/drivers/pci/quirks.c +++ b/drivers/pci/quirks.c @@ -4992,10 +4992,12 @@ static int pci_quirk_intel_spt_pch_acs(struct pci_d= ev *dev, u16 acs_flags) return -ENOTTY; =20 /* see pci_acs_flags_enabled() */ - pci_read_config_dword(dev, pos + PCI_ACS_CAP, &cap); + if (pci_read_config_dword(dev, pos + PCI_ACS_CAP, &cap)) + return 0; acs_flags &=3D (cap | PCI_ACS_EC); =20 - pci_read_config_dword(dev, pos + INTEL_SPT_ACS_CTRL, &ctrl); + if (pci_read_config_dword(dev, pos + INTEL_SPT_ACS_CTRL, &ctrl)) + return 0; =20 return pci_acs_ctrl_enabled(acs_flags, ctrl); } --=20 2.55.0 From nobody Fri Sep 25 16:56:34 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 640F2476070; Thu, 10 Sep 2026 11:32:34 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789039957; cv=none; b=kDiKmjzTfO0BiIcC8Tdp3hgV56+1k69D8t3vOiTlnAe4/J1lrItrSPx+5aJJIvkMRON2x+Gc9zU85BjWK1XAtuKCEx0QzFjL83FXJ0DmUq0wHUtCYt/HrVEuS6z3nA2bbhVNLwy3sVMRG0AaFH6jd8OOawVhHafG3QqYGnqMVBk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789039957; c=relaxed/simple; bh=uUPkLTIdOcJG98wcuOEorHeRIFvb1d4Zm1COLXqc4b4=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=laJPAgEVOnQ/ZkZM+fhcyKBM7yBpbcHdIhT2+U1CVNtriCGGXLOQjwt7gMRVbYze1bpmkJgMxbFb30p+XLzeSWbHcaeS4Wvm1Eam1PO0gWnKgNSWjM+t0UpvN3UeEPotsO4bppxxULHdrY4mjMP8Vyb4c0fZMGMFeZDADMoSsFA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=RM/xxFJQ; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="RM/xxFJQ" Received: by smtp.kernel.org (Postfix) with ESMTPSA id D14751F000FF; Thu, 10 Sep 2026 11:32:32 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1789039953; bh=A4ZAZK3Ia0RfkaGkN5vfdYSNwiPoFjQt6i6vVfKhRQI=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=RM/xxFJQubRp+Ub0lnLKlR31/b3/KPG9xPnyMT7lfCSYvtzznGrqGTGGExLgVbRp/ q96QCamnr73ItYNPNgFWAWw4IVfFfvcUg7ss6FOfISl/Io7LIofY8LXfAJ6NNS8+vi c/0AMtxUzohddBdKLqQuS0w/21BYm/c8HBf0cKtcG1WWh/ycQYlkYyplfDXQJtDMZr xa1EPjXzueTNtXabAhQ0oyx8iG7PtaK/Lek59pugsl4T4ttbQof2wliZIZdqgTlVye 10qG9TnFCm1Nm+J4L55fBfuWQxbujiojNlLsPZ67qh69b33TCljSXXrhDxe9MW4MyP XiOhHwmPPynfw== From: Leon Romanovsky To: Bjorn Helgaas , Logan Gunthorpe , Chaitanya Kulkarni , Greg Kroah-Hartman , Jens Axboe , Alex Williamson , Leon Romanovsky , Ankit Agrawal , Jason Gunthorpe , Jonathan Corbet , Shuah Khan , "Joerg Roedel (AMD)" , Will Deacon , Robin Murphy , Randy Dunlap , Sumit Semwal , =?utf-8?q?Christian_K=C3=B6nig?= Cc: linux-pci@vger.kernel.org, linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, iommu@lists.linux.dev, Tushar Dave , linux-media@vger.kernel.org, dri-devel@lists.freedesktop.org, linaro-mm-sig@lists.linaro.org, linux-rdma@vger.kernel.org, kvm@vger.kernel.org Subject: [PATCH v5 05/18] PCI/P2PDMA: Evaluate ACS controls at the path divergence Date: Thu, 10 Sep 2026 14:32:00 +0300 Message-ID: <20260910-fix-p2p-acs-v4-0-v5-5-856087f63c0d@nvidia.com> X-Mailer: git-send-email 2.55.0 In-Reply-To: <20260910-fix-p2p-acs-v4-0-v5-0-856087f63c0d@nvidia.com> References: <20260910-fix-p2p-acs-v4-0-v5-0-856087f63c0d@nvidia.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" X-Mailer: b4 0.15-dev-18f8f Content-Transfer-Encoding: quoted-printable From: Leon Romanovsky ACS redirect controls choose between peer and upstream routes only at the path divergence. Applying them below that point rejects valid nested topologies because traffic already has only an upstream route. Evaluate Request controls on the client-side divergence port and Completion Redirect on the provider-side port and reject an unreadable ACS Control register. Fixes: 52916982af48 ("PCI/P2PDMA: Support peer-to-peer memory") Signed-off-by: Leon Romanovsky --- drivers/pci/p2pdma.c | 101 +++++++++++++++++++++++++++++------------= ---- include/linux/pci-p2pdma.h | 8 ++-- 2 files changed, 70 insertions(+), 39 deletions(-) diff --git a/drivers/pci/p2pdma.c b/drivers/pci/p2pdma.c index 6da68ec99601..43219a67aaf6 100644 --- a/drivers/pci/p2pdma.c +++ b/drivers/pci/p2pdma.c @@ -516,6 +516,7 @@ static struct pci_dev *find_parent_pci_dev(struct devic= e *dev) } =20 enum pci_acs_p2pdma_state { + PCI_ACS_P2PDMA_NOT_SUPPORTED, PCI_ACS_P2PDMA_DIRECT, PCI_ACS_P2PDMA_REDIRECT, }; @@ -753,13 +754,13 @@ static unsigned long map_types_idx(struct pci_dev *cl= ient) * then to Device B. The mapping type returned depends on the ACS * redirection setting of the ports along the path. * - * The client initiates Requests to provider memory. Check Request Redirect - * on the client path and Completion Redirect for read Completions on the - * provider path. + * The client initiates Requests to provider memory. At the path divergenc= e, + * check Request Redirect and Egress Control on the client-side port, and + * Completion Redirect for read Completions on the provider-side port. * - * If ACS redirect is set on any port in the path, traffic between the - * devices will go through the host bridge, so return - * PCI_P2PDMA_MAP_THRU_HOST_BRIDGE; otherwise return + * If ACS redirects traffic at either divergence port, return + * PCI_P2PDMA_MAP_THRU_HOST_BRIDGE. If the ACS Control register cannot be + * read, return PCI_P2PDMA_MAP_NOT_SUPPORTED. Otherwise, return * PCI_P2PDMA_MAP_BUS_ADDR. * * Any two devices that have a data path that goes through the host bridge @@ -773,10 +774,13 @@ calc_map_type_and_dist(struct pci_dev *provider, stru= ct pci_dev *client, int *dist, bool verbose) { enum pci_p2pdma_map_type map_type =3D PCI_P2PDMA_MAP_THRU_HOST_BRIDGE; + enum pci_acs_p2pdma_state state =3D PCI_ACS_P2PDMA_NOT_SUPPORTED; struct pci_dev *a =3D provider, *b =3D client, *bb; + struct pci_dev *a_child =3D NULL, *b_child =3D NULL; + struct pci_dev *acs_unreadable =3D NULL; struct pci_p2pdma *p2pdma; struct seq_buf acs_list; - int acs_cnt =3D 0; + int acs_redirect_cnt =3D 0; int dist_a =3D 0; int dist_b =3D 0; char buf[128]; @@ -791,51 +795,67 @@ calc_map_type_and_dist(struct pci_dev *provider, stru= ct pci_dev *client, */ while (a) { dist_b =3D 0; - - if (!pci_acs_p2pdma_ctrl(a, &ctrl) || - pci_acs_p2pdma_completion(ctrl) =3D=3D - PCI_ACS_P2PDMA_REDIRECT) { - seq_buf_print_bus_devfn(&acs_list, a); - acs_cnt++; - } - + b_child =3D NULL; bb =3D b; =20 while (bb) { if (a =3D=3D bb) - goto check_b_path_acs; + goto check_paths_acs; =20 + b_child =3D bb; bb =3D pci_upstream_bridge(bb); dist_b++; } =20 + a_child =3D a; a =3D pci_upstream_bridge(a); dist_a++; } =20 + /* + * The paths share no upstream bridge, so there is no direct path for + * ACS to gate: PCI_P2PDMA_MAP_BUS_ADDR is not reachable here and the + * request can only get to the peer through the host bridge. + */ *dist =3D dist_a + dist_b; goto map_through_host_bridge; =20 -check_b_path_acs: - bb =3D b; - - while (bb) { - if (a =3D=3D bb) - break; +check_paths_acs: + *dist =3D dist_a + dist_b; =20 - if (!pci_acs_p2pdma_ctrl(bb, &ctrl) || - pci_acs_p2pdma_request(ctrl) =3D=3D - PCI_ACS_P2PDMA_REDIRECT) { - seq_buf_print_bus_devfn(&acs_list, bb); - acs_cnt++; + /* + * ACS P2P routing controls apply where a TLP can route toward the peer + * or upstream. Below that divergence, its only route toward the other + * branch is upstream, so redirect controls do not affect the path. + */ + if (a_child && b_child) { + if (pci_acs_p2pdma_ctrl(a_child, &ctrl)) + state =3D pci_acs_p2pdma_completion(ctrl); + if (state !=3D PCI_ACS_P2PDMA_DIRECT) { + seq_buf_print_bus_devfn(&acs_list, a_child); + if (state =3D=3D PCI_ACS_P2PDMA_REDIRECT) + acs_redirect_cnt++; + else if (!acs_unreadable) + acs_unreadable =3D a_child; } =20 - bb =3D pci_upstream_bridge(bb); + state =3D PCI_ACS_P2PDMA_NOT_SUPPORTED; + if (pci_acs_p2pdma_ctrl(b_child, &ctrl)) + state =3D pci_acs_p2pdma_request(ctrl); + if (state !=3D PCI_ACS_P2PDMA_DIRECT) { + seq_buf_print_bus_devfn(&acs_list, b_child); + if (state =3D=3D PCI_ACS_P2PDMA_REDIRECT) + acs_redirect_cnt++; + else if (!acs_unreadable) + acs_unreadable =3D b_child; + } } =20 - *dist =3D dist_a + dist_b; - - if (!acs_cnt) { + /* + * Below a shared upstream bridge, a path whose divergence ports do not + * redirect routes the request directly. + */ + if (!acs_unreadable && !acs_redirect_cnt) { map_type =3D PCI_P2PDMA_MAP_BUS_ADDR; goto done; } @@ -844,10 +864,21 @@ calc_map_type_and_dist(struct pci_dev *provider, stru= ct pci_dev *client, /* Drop the final semicolon; the list is not empty here. */ if (!seq_buf_has_overflowed(&acs_list)) acs_list.buffer[acs_list.len - 1] =3D '\0'; - pci_warn(client, "ACS redirect is set between the client and provider (%= s)\n", - pci_name(provider)); - pci_warn(client, "to disable ACS redirect for this path, add the kernel = parameter: pci=3Ddisable_acs_redir=3D%s\n", - seq_buf_str(&acs_list)); + if (acs_unreadable) + pci_warn(client, "ACS Control is unreadable for provider %s at %s\n", + pci_name(provider), pci_name(acs_unreadable)); + else { + pci_warn(client, "ACS redirect is set between the client and provider (= %s)\n", + pci_name(provider)); + pci_warn(client, "to disable ACS controls for this path, add the kernel= parameter: pci=3Ddisable_acs_redir=3D%s\n", + seq_buf_str(&acs_list)); + } + } + + /* An unreadable control does not establish an upstream redirect. */ + if (acs_unreadable) { + map_type =3D PCI_P2PDMA_MAP_NOT_SUPPORTED; + goto done; } =20 map_through_host_bridge: diff --git a/include/linux/pci-p2pdma.h b/include/linux/pci-p2pdma.h index 873de20a2247..dd17501ba1b6 100644 --- a/include/linux/pci-p2pdma.h +++ b/include/linux/pci-p2pdma.h @@ -42,10 +42,10 @@ enum pci_p2pdma_map_type { PCI_P2PDMA_MAP_NONE, =20 /* - * PCI_P2PDMA_MAP_NOT_SUPPORTED: Indicates the transaction will - * traverse the host bridge and the host bridge is not in the - * allowlist. DMA Mapping routines should return an error when - * this is returned. + * PCI_P2PDMA_MAP_NOT_SUPPORTED: Indicates no safe mapping is available, + * for example because ACS blocks the direct path or the required host + * bridge is not in the allowlist. DMA Mapping routines should return an + * error when this is returned. */ PCI_P2PDMA_MAP_NOT_SUPPORTED, =20 --=20 2.55.0 From nobody Fri Sep 25 16:56:34 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 79798477293; Thu, 10 Sep 2026 11:32:37 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789039959; cv=none; b=W+BNLjlpG5qZ0vzuaBZ0BeBtg7zi5CfOgAIrGIQU9xP6DjNriZ/B3YztckhsJXSbGo7LHoflBVIGpb+itSVHD1H7hUimDUIiwnuHsIAsKCLpuXLWvwlnFTpTzG1ViG4g/4j71noWYcm+Nqhomj0ZIi8F7oE93hmqEVtRCS0cJTo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789039959; c=relaxed/simple; bh=3mMax6tEtS6HXskdbg5O/HITyZZjE2ilSYcvfQJ/HWg=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=A6FY7b3OCaKmyGsJhIWXt7lZ8Ogrbt/EUi06dZ+QTpB7Y/QG1CjCGpJplG8YBdJaQHDnAnzrilXUSDbYcfrvQ4FhFau4S1M+2BC8U2hLV3xn7dcaP9x0WdhpJSPAa6bcMtNj4SooVr7MxMFgrS5H6+aQWELTFMxxILd/ti+kNb8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=nbZTW7V3; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="nbZTW7V3" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 4B9751F00893; Thu, 10 Sep 2026 11:32:36 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1789039957; bh=Ry6T6D/pwfGxSb4X1rqr80n4nBDGwc9qJPxAeSwSHa4=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=nbZTW7V3P5yPRRsbgyZgxp7OUOwRc67JgbPEYDoLs0qcYrwwKEvORuFN3a1YH35eP fY0Hqlx4wkcBYPFEfEEVhvwSTaLyJdixCB6jJ0IaKvKX80I6qYYAl5OiRZ4dRHglT3 dmjXp1St7OGpHDo/y7wA+xhlrYqpTTvUCy4mT2W7LGh6gFJ3HfHt5fCIXJ8bsehACx hjX/MSbz/dd26fp/xSrAjo3F4Jdmf5zOy9rvqQmAP3/BdoTKIq4hHbMe9bH5kvcrY0 t35VnJkLE8MKgx9ewS8rB0mlcwImuyzmaC0TETOPxFl/QZu3nxDkxTXmPKT5OfRXjK lGREuBycRNLcw== From: Leon Romanovsky To: Bjorn Helgaas , Logan Gunthorpe , Chaitanya Kulkarni , Greg Kroah-Hartman , Jens Axboe , Alex Williamson , Leon Romanovsky , Ankit Agrawal , Jason Gunthorpe , Jonathan Corbet , Shuah Khan , "Joerg Roedel (AMD)" , Will Deacon , Robin Murphy , Randy Dunlap , Sumit Semwal , =?utf-8?q?Christian_K=C3=B6nig?= Cc: linux-pci@vger.kernel.org, linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, iommu@lists.linux.dev, Tushar Dave , linux-media@vger.kernel.org, dri-devel@lists.freedesktop.org, linaro-mm-sig@lists.linaro.org, linux-rdma@vger.kernel.org, kvm@vger.kernel.org Subject: [PATCH v5 06/18] PCI/P2PDMA: Document directional ACS routing Date: Thu, 10 Sep 2026 14:32:01 +0300 Message-ID: <20260910-fix-p2p-acs-v4-0-v5-6-856087f63c0d@nvidia.com> X-Mailer: git-send-email 2.55.0 In-Reply-To: <20260910-fix-p2p-acs-v4-0-v5-0-856087f63c0d@nvidia.com> References: <20260910-fix-p2p-acs-v4-0-v5-0-856087f63c0d@nvidia.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" X-Mailer: b4 0.15-dev-18f8f Content-Transfer-Encoding: quoted-printable From: Leon Romanovsky P2PDMA documentation describes ACS controls as path-wide, although Request and Completion controls apply to different transaction directions and only affect peer-versus-upstream decisions at the path divergence. Document the fixed client and provider roles, the divergence port checked for each TLP direction, and the conservative handling of unreadable ACS state. Clarify which controls disable_acs_redir changes. Signed-off-by: Leon Romanovsky --- Documentation/admin-guide/kernel-parameters.txt | 9 +++++---- Documentation/driver-api/pci/p2pdma.rst | 13 +++++++++++++ 2 files changed, 18 insertions(+), 4 deletions(-) diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentatio= n/admin-guide/kernel-parameters.txt index 68647ff4bdd2..43a25f3f94fd 100644 --- a/Documentation/admin-guide/kernel-parameters.txt +++ b/Documentation/admin-guide/kernel-parameters.txt @@ -5291,10 +5291,11 @@ Kernel parameters disable_acs_redir=3D[; ...] Specify one or more PCI devices (in the format specified above) separated by semicolons. - Each device specified will have the PCI ACS - redirect capabilities forced off which will - allow P2P traffic between devices through - bridges without forcing it upstream. Note: + Each device specified will have the PCI ACS P2P + Request Redirect, Completion Redirect, and Egress + Control features forced off. This may allow P2P + traffic through bridges that would otherwise be + redirected upstream. Note: this removes isolation between devices and may put more devices in an IOMMU group. config_acs=3D diff --git a/Documentation/driver-api/pci/p2pdma.rst b/Documentation/driver= -api/pci/p2pdma.rst index 80f8fec9b0e9..42b18610bf7d 100644 --- a/Documentation/driver-api/pci/p2pdma.rst +++ b/Documentation/driver-api/pci/p2pdma.rst @@ -15,6 +15,19 @@ then based on the ACS settings the transaction can route= entirely within the PCIe hierarchy and never reach the root port. The kernel will evaluate the PCIe topology and always permit P2P in these well-defined cases. =20 +The client remains the PCIe requester when it reads or writes provider mem= ory. +Where the paths diverge, the kernel therefore evaluates P2P Request Redire= ct +and Egress Control on the client-side port, and P2P Completion Redirect on= the +provider-side port for completions from a read. An enabled Egress Control = is +conservatively treated as a Request redirect. + +Below the divergence, the route toward the other branch is already upstrea= m, +so those P2P redirect controls do not affect it. Redirect controls for the +reverse transaction directions do not affect the mapping. P2P DMA is routed +through the host bridge when either applicable port redirects. If an ACS +Control register cannot be read, P2P DMA is rejected because the kernel ca= nnot +establish a usable route. + This evaluation assumes clients issue strictly ordered Requests carrying an Untranslated address. Its result is not defined when clients use Relaxed Ordering or issue ATS-translated Requests because those TLP attributes can --=20 2.55.0 From nobody Fri Sep 25 16:56:34 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7E0C44749C9; Thu, 10 Sep 2026 11:32:41 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789039963; cv=none; b=tzXm1Aui78n+zaMb1LnUNs91Fk2XwaCCaXebE1Kb2qMnYuITHnhQpj+716NK4repbLhiDhvJP3gjn6QKdNu15j+0AGUJuWjmXt5JMDgk4j6bj/RtZUM2rs3zLQl7pMDWml/Gn2Earx4qJHS/00hw7HFXmYW62FmcSWOsunxjHJA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789039963; c=relaxed/simple; bh=Uv5KHKoiHDpKGepAN/SRZp2YwTtspNNmDJBxWTGTYuk=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=qowE1p7M/UW8WtpF0Fo754OR1BSkBGHXeMLLOZ0m4G+nETzRusAXpxYYE8q3NlF2aH1AKqB1d6ax9cUW+ULyUvQ5P2YRmqooRPxkX6J02h6sm9uuRNhj3ICnE3Z9MJ5Je4U4CLNd5fzuvrJPl0XA8d66+lp625SZkj7oJlH13ps= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=gA7evxfJ; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="gA7evxfJ" Received: by smtp.kernel.org (Postfix) with ESMTPSA id C82FD1F00899; Thu, 10 Sep 2026 11:32:39 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1789039960; bh=76WYju94Ls/jEg6VJCnypCSeojbnUYpLisq3GOhkJO0=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=gA7evxfJNkAyc9tlZBtOk6Qyh/tlVAaI+ZFNKLG22NH8hwBFohC1vssuByg/qUYrQ iXC1CocluuAWpeNDil5dizPm3lky45QSSuEUfA9BLlBm7mz1Xy5NmRtb4bgwg9R4Hq nfs1uyRHbiipfdpQ1n6bS/SnId2BsegdWeNehWfD11QBAZeyJ0AJwqazM7nTU1urfj UAdoFt7jHMPC2q0s5afSAZ67/GqmiolS1dsEG9412885smE+Y+apMqNo3s8h7auH6J Mp1IEq0tun3dnxiDuy0Z1l0oqSH3/IgOlwSCOcuP7C7R7/R4NRq6SLxQ3RAH4dldJg 1DyPV1URoH9LA== From: Leon Romanovsky To: Bjorn Helgaas , Logan Gunthorpe , Chaitanya Kulkarni , Greg Kroah-Hartman , Jens Axboe , Alex Williamson , Leon Romanovsky , Ankit Agrawal , Jason Gunthorpe , Jonathan Corbet , Shuah Khan , "Joerg Roedel (AMD)" , Will Deacon , Robin Murphy , Randy Dunlap , Sumit Semwal , =?utf-8?q?Christian_K=C3=B6nig?= Cc: linux-pci@vger.kernel.org, linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, iommu@lists.linux.dev, Tushar Dave , linux-media@vger.kernel.org, dri-devel@lists.freedesktop.org, linaro-mm-sig@lists.linaro.org, linux-rdma@vger.kernel.org, kvm@vger.kernel.org Subject: [PATCH v5 07/18] PCI/P2PDMA: Collect the path's ACS controls before deciding Date: Thu, 10 Sep 2026 14:32:02 +0300 Message-ID: <20260910-fix-p2p-acs-v4-0-v5-7-856087f63c0d@nvidia.com> X-Mailer: git-send-email 2.55.0 In-Reply-To: <20260910-fix-p2p-acs-v4-0-v5-0-856087f63c0d@nvidia.com> References: <20260910-fix-p2p-acs-v4-0-v5-0-856087f63c0d@nvidia.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" X-Mailer: b4 0.15-dev-18f8f Content-Transfer-Encoding: quoted-printable From: Leon Romanovsky calc_map_type_and_dist() reads each divergence port's ACS Control register and folds the result into running counters as it goes. Any routing property that depends on the kind of TLP being routed would have to be threaded through that code, so there is nowhere to put one without reading the registers again for each kind. Collect the two ports' ACS Control values into struct pci_p2pdma_acs_path first, then decide from it. pci_p2pdma_route() applies the same rule as before: a path routes directly only when both directions do. Signed-off-by: Leon Romanovsky --- drivers/pci/p2pdma.c | 148 +++++++++++++++++++++++++++++++++--------------= ---- 1 file changed, 96 insertions(+), 52 deletions(-) diff --git a/drivers/pci/p2pdma.c b/drivers/pci/p2pdma.c index 43219a67aaf6..3a14abf5fe84 100644 --- a/drivers/pci/p2pdma.c +++ b/drivers/pci/p2pdma.c @@ -576,6 +576,80 @@ static void seq_buf_print_bus_devfn(struct seq_buf *bu= f, struct pci_dev *pdev) seq_buf_printf(buf, "%s;", pci_name(pdev)); } =20 +/* + * What the topology walk found out about one provider/client path. Produc= ing + * this costs a walk and one config read per divergence port, none of which + * depends on the TLP being routed. + * + * @req_ctrl: ACS Control of the client-side divergence port. That is the + * first port at which a Request can route toward the peer rather + * than upstream, so it is where the Request controls apply. + * @cpl_ctrl: ACS Control of the provider-side divergence port, likewise f= or + * the Completions travelling back. + * @unreadable: First port whose ACS Control could not be read, if any. + */ +struct pci_p2pdma_acs_path { + u16 req_ctrl; + u16 cpl_ctrl; + struct pci_dev *unreadable; +}; + +/* + * Combine both directions into a mapping type. Only a path that routes the + * Request and the Completions it generates directly can be programmed with + * the peer's bus addresses. + */ +static enum pci_p2pdma_map_type +pci_p2pdma_route(const struct pci_p2pdma_acs_path *path) +{ + if (path->unreadable) + return PCI_P2PDMA_MAP_NOT_SUPPORTED; + + if (pci_acs_p2pdma_request(path->req_ctrl) =3D=3D PCI_ACS_P2PDMA_DIRECT && + pci_acs_p2pdma_completion(path->cpl_ctrl) =3D=3D PCI_ACS_P2PDMA_DIREC= T) + return PCI_P2PDMA_MAP_BUS_ADDR; + + return PCI_P2PDMA_MAP_THRU_HOST_BRIDGE; +} + +/* + * Name the ports that keep this path off a direct route, so that the admin + * can hand them to pci=3Ddisable_acs_redir=3D. + */ +static void pci_p2pdma_warn_path(struct pci_dev *client, + struct pci_dev *provider, + const struct pci_p2pdma_acs_path *path, + struct pci_dev *a_child, + struct pci_dev *b_child) +{ + struct seq_buf acs_list; + char buf[128]; + + if (path->unreadable) { + pci_warn(client, + "ACS Control is unreadable for provider %s at %s\n", + pci_name(provider), pci_name(path->unreadable)); + return; + } + + seq_buf_init(&acs_list, buf, sizeof(buf)); + if (pci_acs_p2pdma_completion(path->cpl_ctrl) !=3D PCI_ACS_P2PDMA_DIRECT) + seq_buf_print_bus_devfn(&acs_list, a_child); + if (pci_acs_p2pdma_request(path->req_ctrl) !=3D PCI_ACS_P2PDMA_DIRECT) + seq_buf_print_bus_devfn(&acs_list, b_child); + + /* Drop the final semicolon; the list is not empty here. */ + if (!seq_buf_has_overflowed(&acs_list)) + acs_list.buffer[acs_list.len - 1] =3D '\0'; + + pci_warn(client, + "ACS redirect is set between the client and provider (%s)\n", + pci_name(provider)); + pci_warn(client, + "to disable ACS controls for this path, add the kernel parameter: pci= =3Ddisable_acs_redir=3D%s\n", + seq_buf_str(&acs_list)); +} + static bool cpu_supports_p2pdma(void) { #ifdef CONFIG_X86 @@ -774,19 +848,13 @@ calc_map_type_and_dist(struct pci_dev *provider, stru= ct pci_dev *client, int *dist, bool verbose) { enum pci_p2pdma_map_type map_type =3D PCI_P2PDMA_MAP_THRU_HOST_BRIDGE; - enum pci_acs_p2pdma_state state =3D PCI_ACS_P2PDMA_NOT_SUPPORTED; struct pci_dev *a =3D provider, *b =3D client, *bb; struct pci_dev *a_child =3D NULL, *b_child =3D NULL; - struct pci_dev *acs_unreadable =3D NULL; + struct pci_p2pdma_acs_path path =3D {}; struct pci_p2pdma *p2pdma; - struct seq_buf acs_list; - int acs_redirect_cnt =3D 0; + bool cpu_p2pdma, host_whitelisted =3D false; int dist_a =3D 0; int dist_b =3D 0; - char buf[128]; - u16 ctrl; - - seq_buf_init(&acs_list, buf, sizeof(buf)); =20 /* * Note, we don't need to take references to devices returned by @@ -829,61 +897,35 @@ calc_map_type_and_dist(struct pci_dev *provider, stru= ct pci_dev *client, * branch is upstream, so redirect controls do not affect the path. */ if (a_child && b_child) { - if (pci_acs_p2pdma_ctrl(a_child, &ctrl)) - state =3D pci_acs_p2pdma_completion(ctrl); - if (state !=3D PCI_ACS_P2PDMA_DIRECT) { - seq_buf_print_bus_devfn(&acs_list, a_child); - if (state =3D=3D PCI_ACS_P2PDMA_REDIRECT) - acs_redirect_cnt++; - else if (!acs_unreadable) - acs_unreadable =3D a_child; - } - - state =3D PCI_ACS_P2PDMA_NOT_SUPPORTED; - if (pci_acs_p2pdma_ctrl(b_child, &ctrl)) - state =3D pci_acs_p2pdma_request(ctrl); - if (state !=3D PCI_ACS_P2PDMA_DIRECT) { - seq_buf_print_bus_devfn(&acs_list, b_child); - if (state =3D=3D PCI_ACS_P2PDMA_REDIRECT) - acs_redirect_cnt++; - else if (!acs_unreadable) - acs_unreadable =3D b_child; - } + if (!pci_acs_p2pdma_ctrl(a_child, &path.cpl_ctrl)) + path.unreadable =3D a_child; + if (!pci_acs_p2pdma_ctrl(b_child, &path.req_ctrl) && + !path.unreadable) + path.unreadable =3D b_child; } =20 /* * Below a shared upstream bridge, a path whose divergence ports do not * redirect routes the request directly. */ - if (!acs_unreadable && !acs_redirect_cnt) { - map_type =3D PCI_P2PDMA_MAP_BUS_ADDR; + map_type =3D pci_p2pdma_route(&path); + if (map_type =3D=3D PCI_P2PDMA_MAP_BUS_ADDR) goto done; - } =20 - if (verbose) { - /* Drop the final semicolon; the list is not empty here. */ - if (!seq_buf_has_overflowed(&acs_list)) - acs_list.buffer[acs_list.len - 1] =3D '\0'; - if (acs_unreadable) - pci_warn(client, "ACS Control is unreadable for provider %s at %s\n", - pci_name(provider), pci_name(acs_unreadable)); - else { - pci_warn(client, "ACS redirect is set between the client and provider (= %s)\n", - pci_name(provider)); - pci_warn(client, "to disable ACS controls for this path, add the kernel= parameter: pci=3Ddisable_acs_redir=3D%s\n", - seq_buf_str(&acs_list)); - } - } + if (verbose) + pci_p2pdma_warn_path(client, provider, &path, a_child, b_child); =20 /* An unreadable control does not establish an upstream redirect. */ - if (acs_unreadable) { - map_type =3D PCI_P2PDMA_MAP_NOT_SUPPORTED; + if (path.unreadable) goto done; - } =20 map_through_host_bridge: - if (!cpu_supports_p2pdma() && - !host_bridge_whitelist(provider, client, verbose)) { + cpu_p2pdma =3D cpu_supports_p2pdma(); + if (!cpu_p2pdma) + host_whitelisted =3D host_bridge_whitelist(provider, client, + verbose); + + if (!cpu_p2pdma && !host_whitelisted) { if (verbose) pci_warn(client, "cannot be used for peer-to-peer DMA as the client and= provider (%s) do not share an upstream bridge or whitelisted host bridge\n= ", pci_name(provider)); @@ -1216,8 +1258,9 @@ enum pci_p2pdma_map_type pci_p2pdma_map_type(struct p= 2pdma_provider *provider, { enum pci_p2pdma_map_type type =3D PCI_P2PDMA_MAP_NOT_SUPPORTED; struct pci_dev *pdev =3D to_pci_dev(provider->owner); - struct pci_dev *client; struct pci_p2pdma *p2pdma; + unsigned long cache_index; + struct pci_dev *client; int dist; =20 if (!pdev->p2pdma) @@ -1227,13 +1270,14 @@ enum pci_p2pdma_map_type pci_p2pdma_map_type(struct= p2pdma_provider *provider, return PCI_P2PDMA_MAP_NOT_SUPPORTED; =20 client =3D to_pci_dev(dev); + cache_index =3D map_types_idx(client); =20 rcu_read_lock(); p2pdma =3D rcu_dereference(pdev->p2pdma); =20 if (p2pdma) type =3D xa_to_value(xa_load(&p2pdma->map_types, - map_types_idx(client))); + cache_index)); rcu_read_unlock(); =20 if (type =3D=3D PCI_P2PDMA_MAP_UNKNOWN) --=20 2.55.0 From nobody Fri Sep 25 16:56:34 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id BE57C47DFAF; Thu, 10 Sep 2026 11:33:02 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789039986; cv=none; b=hlyOunP+zVoHKAKAW+n1O+n/dVaTzgCxDgs5g+LwH5FmuWBlMZZIIM/qgN1VyU/m+SKMR/pcxE6gil82rjISUE6J5Qw0anK1VmjximzF+9Zw9I0axQ7kyWH3Wnxdr7wD5D+rDpwYoN/h4XmeOA1WDBlQNQn0xctHaBgmpny1zy4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789039986; c=relaxed/simple; bh=J3hnBZTraVcqUV0dnkw2YKaSom0sgsYElIGHb5wOQOE=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=FTS4dJ7zJrkzg44dvzOx3QHI7JU4k2jnuoqcxmAHrGD8chsxBQeVJep84Ah/1lWavkyRUGYJHLuV8Asx0OgkXtYe7U0xoRDBYJ8B3n3VhCpY94HcUINjZBXlQCS1cKVNJvQdh7Y+SP9DYzZKxXVnSgnaHPMupA4gw1DBasyB7Zw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=Dp/es9dx; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="Dp/es9dx" Received: by smtp.kernel.org (Postfix) with ESMTPSA id DE4EA1F000FF; Thu, 10 Sep 2026 11:33:00 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1789039981; bh=ARRhQFUHv0KrZ0GZtzk706qAcJDNAlE6Cjev/pw1Dbg=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=Dp/es9dxjBsg6LpqpKJkEHNFNuKKeII5ZJoSrSemS7AfYilyqwxE8bmu5MxRHnS6z yXbpczNMWbUk9cqYPMBKIqQd1mMgxD9msuUL87suebsp4Xvb4V3JAfsK/VjOQgeQKM sljQ+DZ4lTQ2BNFN59hxyhTwIxyx0JfYWm8UnUD0UrpEfSg7LwmVN5+ozOPUBwK/hl kkTKWxFFaaWFj+6L0ExRD2ksupEUlzT/MjSuV5yeyruI1c9tsH/OtYoS2iQrMtcQV5 Ffbh83xaJqtI8EaCLeoNnDn6CPzcVAdL59jg2Ue7FQO1svngNSTxBY2aR1PeioRWfU c0v2qMIo0f0QA== From: Leon Romanovsky To: Bjorn Helgaas , Logan Gunthorpe , Chaitanya Kulkarni , Greg Kroah-Hartman , Jens Axboe , Alex Williamson , Leon Romanovsky , Ankit Agrawal , Jason Gunthorpe , Jonathan Corbet , Shuah Khan , "Joerg Roedel (AMD)" , Will Deacon , Robin Murphy , Randy Dunlap , Sumit Semwal , =?utf-8?q?Christian_K=C3=B6nig?= Cc: linux-pci@vger.kernel.org, linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, iommu@lists.linux.dev, Tushar Dave , linux-media@vger.kernel.org, dri-devel@lists.freedesktop.org, linaro-mm-sig@lists.linaro.org, linux-rdma@vger.kernel.org, kvm@vger.kernel.org Subject: [PATCH v5 08/18] PCI/P2PDMA: Answer routing per TLP class Date: Thu, 10 Sep 2026 14:32:03 +0300 Message-ID: <20260910-fix-p2p-acs-v4-0-v5-8-856087f63c0d@nvidia.com> X-Mailer: git-send-email 2.55.0 In-Reply-To: <20260910-fix-p2p-acs-v4-0-v5-0-856087f63c0d@nvidia.com> References: <20260910-fix-p2p-acs-v4-0-v5-0-856087f63c0d@nvidia.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" X-Mailer: b4 0.15-dev-18f8f Content-Transfer-Encoding: quoted-printable From: Leon Romanovsky calc_map_type_and_dist() returns one mapping type per provider and client, valid only for strictly ordered Requests carrying an Untranslated address. Clients that use Relaxed Ordering or ATS cannot ask what the fabric would do with their traffic. Add enum pci_p2pdma_tlp_flags to name a class and pci_p2pdma_map_type_tlp() to ask about one. The topology walk and the ACS Control reads do not depend on the class, so decide all of them from the one walk and cache them together, four bits each. Every class still answers alike; the controls that tell them apart come next. Signed-off-by: Leon Romanovsky --- drivers/pci/p2pdma.c | 121 +++++++++++++++++++++++++++++++++--------= ---- include/linux/pci-p2pdma.h | 49 ++++++++++++++++-- 2 files changed, 136 insertions(+), 34 deletions(-) diff --git a/drivers/pci/p2pdma.c b/drivers/pci/p2pdma.c index 3a14abf5fe84..fac765d0a16f 100644 --- a/drivers/pci/p2pdma.c +++ b/drivers/pci/p2pdma.c @@ -531,7 +531,7 @@ enum pci_acs_p2pdma_state { * bus addressing be assumed. */ static enum pci_acs_p2pdma_state -pci_acs_p2pdma_request(u16 ctrl) +pci_acs_p2pdma_request(u16 ctrl, unsigned int tlp_flags) { return ctrl & (PCI_ACS_RR | PCI_ACS_EC) ? PCI_ACS_P2PDMA_REDIRECT : PCI_ACS_P2PDMA_DIRECT; @@ -543,7 +543,7 @@ pci_acs_p2pdma_request(u16 ctrl) * affects a Completion. */ static enum pci_acs_p2pdma_state -pci_acs_p2pdma_completion(u16 ctrl) +pci_acs_p2pdma_completion(u16 ctrl, unsigned int tlp_flags) { return ctrl & PCI_ACS_CR ? PCI_ACS_P2PDMA_REDIRECT : PCI_ACS_P2PDMA_DIRECT; @@ -600,13 +600,16 @@ struct pci_p2pdma_acs_path { * the peer's bus addresses. */ static enum pci_p2pdma_map_type -pci_p2pdma_route(const struct pci_p2pdma_acs_path *path) +pci_p2pdma_route(const struct pci_p2pdma_acs_path *path, + unsigned int tlp_flags) { if (path->unreadable) return PCI_P2PDMA_MAP_NOT_SUPPORTED; =20 - if (pci_acs_p2pdma_request(path->req_ctrl) =3D=3D PCI_ACS_P2PDMA_DIRECT && - pci_acs_p2pdma_completion(path->cpl_ctrl) =3D=3D PCI_ACS_P2PDMA_DIREC= T) + if (pci_acs_p2pdma_request(path->req_ctrl, tlp_flags) =3D=3D + PCI_ACS_P2PDMA_DIRECT && + pci_acs_p2pdma_completion(path->cpl_ctrl, tlp_flags) =3D=3D + PCI_ACS_P2PDMA_DIRECT) return PCI_P2PDMA_MAP_BUS_ADDR; =20 return PCI_P2PDMA_MAP_THRU_HOST_BRIDGE; @@ -620,7 +623,8 @@ static void pci_p2pdma_warn_path(struct pci_dev *client, struct pci_dev *provider, const struct pci_p2pdma_acs_path *path, struct pci_dev *a_child, - struct pci_dev *b_child) + struct pci_dev *b_child, + unsigned int tlp_flags) { struct seq_buf acs_list; char buf[128]; @@ -633,9 +637,11 @@ static void pci_p2pdma_warn_path(struct pci_dev *clien= t, } =20 seq_buf_init(&acs_list, buf, sizeof(buf)); - if (pci_acs_p2pdma_completion(path->cpl_ctrl) !=3D PCI_ACS_P2PDMA_DIRECT) + if (pci_acs_p2pdma_completion(path->cpl_ctrl, tlp_flags) !=3D + PCI_ACS_P2PDMA_DIRECT) seq_buf_print_bus_devfn(&acs_list, a_child); - if (pci_acs_p2pdma_request(path->req_ctrl) !=3D PCI_ACS_P2PDMA_DIRECT) + if (pci_acs_p2pdma_request(path->req_ctrl, tlp_flags) !=3D + PCI_ACS_P2PDMA_DIRECT) seq_buf_print_bus_devfn(&acs_list, b_child); =20 /* Drop the final semicolon; the list is not empty here. */ @@ -803,6 +809,31 @@ static unsigned long map_types_idx(struct pci_dev *cli= ent) return (pci_domain_nr(client->bus) << 16) | pci_dev_id(client); } =20 +/* + * One cache entry holds the routing of every TLP class, four bits each, + * indexed by the &enum pci_p2pdma_tlp_flags combination that selects it. = An + * absent entry reads back as PCI_P2PDMA_MAP_UNKNOWN in every class. + */ +static_assert(PCI_P2PDMA_MAP_THRU_HOST_BRIDGE < 16); + +static unsigned long +pci_p2pdma_map_types_pack(const enum pci_p2pdma_map_type *type) +{ + unsigned long val =3D 0; + unsigned int flags; + + for (flags =3D 0; flags < PCI_P2PDMA_TLP_CLASSES; flags++) + val |=3D (unsigned long)type[flags] << (flags * 4); + + return val; +} + +static enum pci_p2pdma_map_type +pci_p2pdma_map_types_unpack(unsigned long val, unsigned int tlp_flags) +{ + return (val >> (tlp_flags * 4)) & 0xf; +} + /* * Calculate the P2PDMA mapping type and distance between two PCI devices. * @@ -832,6 +863,10 @@ static unsigned long map_types_idx(struct pci_dev *cli= ent) * check Request Redirect and Egress Control on the client-side port, and * Completion Redirect for read Completions on the provider-side port. * + * Those controls apply to different TLPs, so every class named by &enum + * pci_p2pdma_tlp_flags is decided from the one walk and cached together; + * @tlp_flags selects which one is returned. + * * If ACS redirects traffic at either divergence port, return * PCI_P2PDMA_MAP_THRU_HOST_BRIDGE. If the ACS Control register cannot be * read, return PCI_P2PDMA_MAP_NOT_SUPPORTED. Otherwise, return @@ -845,14 +880,16 @@ static unsigned long map_types_idx(struct pci_dev *cl= ient) */ static enum pci_p2pdma_map_type calc_map_type_and_dist(struct pci_dev *provider, struct pci_dev *client, - int *dist, bool verbose) + int *dist, unsigned int tlp_flags, bool verbose) { - enum pci_p2pdma_map_type map_type =3D PCI_P2PDMA_MAP_THRU_HOST_BRIDGE; + enum pci_p2pdma_map_type map_type[PCI_P2PDMA_TLP_CLASSES]; struct pci_dev *a =3D provider, *b =3D client, *bb; struct pci_dev *a_child =3D NULL, *b_child =3D NULL; struct pci_p2pdma_acs_path path =3D {}; struct pci_p2pdma *p2pdma; bool cpu_p2pdma, host_whitelisted =3D false; + bool host_fallback =3D false; + unsigned int flags; int dist_a =3D 0; int dist_b =3D 0; =20 @@ -886,6 +923,8 @@ calc_map_type_and_dist(struct pci_dev *provider, struct= pci_dev *client, * request can only get to the peer through the host bridge. */ *dist =3D dist_a + dist_b; + for (flags =3D 0; flags < PCI_P2PDMA_TLP_CLASSES; flags++) + map_type[flags] =3D PCI_P2PDMA_MAP_THRU_HOST_BRIDGE; goto map_through_host_bridge; =20 check_paths_acs: @@ -905,18 +944,24 @@ calc_map_type_and_dist(struct pci_dev *provider, stru= ct pci_dev *client, } =20 /* - * Below a shared upstream bridge, a path whose divergence ports do not - * redirect routes the request directly. + * The walk and the config reads above serve every class; only the + * decision below depends on the kind of TLP being routed. */ - map_type =3D pci_p2pdma_route(&path); - if (map_type =3D=3D PCI_P2PDMA_MAP_BUS_ADDR) - goto done; + for (flags =3D 0; flags < PCI_P2PDMA_TLP_CLASSES; flags++) { + map_type[flags] =3D pci_p2pdma_route(&path, flags); + if (map_type[flags] =3D=3D PCI_P2PDMA_MAP_THRU_HOST_BRIDGE) + host_fallback =3D true; + } =20 - if (verbose) - pci_p2pdma_warn_path(client, provider, &path, a_child, b_child); + if (verbose && map_type[0] !=3D PCI_P2PDMA_MAP_BUS_ADDR) + pci_p2pdma_warn_path(client, provider, &path, a_child, + b_child, 0); =20 - /* An unreadable control does not establish an upstream redirect. */ - if (path.unreadable) + /* + * Nothing needs the host bridge: the classes that did not get a direct + * route have no fallback that would use it. + */ + if (!host_fallback) goto done; =20 map_through_host_bridge: @@ -929,16 +974,19 @@ calc_map_type_and_dist(struct pci_dev *provider, stru= ct pci_dev *client, if (verbose) pci_warn(client, "cannot be used for peer-to-peer DMA as the client and= provider (%s) do not share an upstream bridge or whitelisted host bridge\n= ", pci_name(provider)); - map_type =3D PCI_P2PDMA_MAP_NOT_SUPPORTED; + for (flags =3D 0; flags < PCI_P2PDMA_TLP_CLASSES; flags++) + if (map_type[flags] =3D=3D PCI_P2PDMA_MAP_THRU_HOST_BRIDGE) + map_type[flags] =3D PCI_P2PDMA_MAP_NOT_SUPPORTED; } done: rcu_read_lock(); p2pdma =3D rcu_dereference(provider->p2pdma); if (p2pdma) xa_store(&p2pdma->map_types, map_types_idx(client), - xa_mk_value(map_type), GFP_ATOMIC); + xa_mk_value(pci_p2pdma_map_types_pack(map_type)), + GFP_ATOMIC); rcu_read_unlock(); - return map_type; + return map_type[tlp_flags]; } =20 /** @@ -979,7 +1027,7 @@ int pci_p2pdma_distance_many(struct pci_dev *provider,= struct device **clients, return -1; } =20 - map =3D calc_map_type_and_dist(provider, pci_client, &distance, + map =3D calc_map_type_and_dist(provider, pci_client, &distance, 0, verbose); =20 pci_dev_put(pci_client); @@ -1244,25 +1292,34 @@ void pci_p2pmem_publish(struct pci_dev *pdev, bool = publish) EXPORT_SYMBOL_GPL(pci_p2pmem_publish); =20 /** - * pci_p2pdma_map_type - Determine the mapping type for P2PDMA transfers + * pci_p2pdma_map_type_tlp - Determine the mapping type for P2PDMA transfe= rs * @provider: P2PDMA provider structure * @dev: Client device that initiates the transfer + * @tlp_flags: &enum pci_p2pdma_tlp_flags describing the TLPs @dev will is= sue * * Determines how peer-to-peer DMA transfers should be mapped between * the provider and the client device. The mapping type indicates whether * the transfer can be done directly through PCI switches or must go * through the host bridge. + * + * ACS routes a peer-to-peer transaction by the attributes its TLPs carry,= so + * the answer depends on @tlp_flags. A caller that passes flags its traffic + * does not match gets a mapping the fabric will not deliver. */ -enum pci_p2pdma_map_type pci_p2pdma_map_type(struct p2pdma_provider *provi= der, - struct device *dev) +enum pci_p2pdma_map_type +pci_p2pdma_map_type_tlp(struct p2pdma_provider *provider, struct device *d= ev, + unsigned int tlp_flags) { - enum pci_p2pdma_map_type type =3D PCI_P2PDMA_MAP_NOT_SUPPORTED; struct pci_dev *pdev =3D to_pci_dev(provider->owner); + unsigned long cache_index, cached =3D 0; + enum pci_p2pdma_map_type type; struct pci_p2pdma *p2pdma; - unsigned long cache_index; struct pci_dev *client; int dist; =20 + if (WARN_ON_ONCE(tlp_flags >=3D PCI_P2PDMA_TLP_CLASSES)) + return PCI_P2PDMA_MAP_NOT_SUPPORTED; + if (!pdev->p2pdma) return PCI_P2PDMA_MAP_NOT_SUPPORTED; =20 @@ -1276,12 +1333,14 @@ enum pci_p2pdma_map_type pci_p2pdma_map_type(struct= p2pdma_provider *provider, p2pdma =3D rcu_dereference(pdev->p2pdma); =20 if (p2pdma) - type =3D xa_to_value(xa_load(&p2pdma->map_types, - cache_index)); + cached =3D xa_to_value(xa_load(&p2pdma->map_types, + cache_index)); rcu_read_unlock(); + type =3D pci_p2pdma_map_types_unpack(cached, tlp_flags); =20 if (type =3D=3D PCI_P2PDMA_MAP_UNKNOWN) - return calc_map_type_and_dist(pdev, client, &dist, true); + return calc_map_type_and_dist(pdev, client, &dist, tlp_flags, + true); =20 return type; } diff --git a/include/linux/pci-p2pdma.h b/include/linux/pci-p2pdma.h index dd17501ba1b6..36045b1b730c 100644 --- a/include/linux/pci-p2pdma.h +++ b/include/linux/pci-p2pdma.h @@ -28,6 +28,33 @@ struct p2pdma_provider { u64 bus_offset; }; =20 +/** + * enum pci_p2pdma_tlp_flags - Properties of the TLPs a client will issue + * + * These describe the traffic rather than the topology, and select which A= CS + * controls apply along the peer-to-peer path. A value of 0 means strictly + * ordered Requests carrying an Untranslated address. + * + * @PCI_P2PDMA_TLP_TRANSLATED: Requests carry an ATS Translated address. P= CIe + * r7.0 sec 6.12.3 routes those to the peer regardless of ACS P2P Request + * Redirect and ACS P2P Egress Control wherever ACS Direct Translated P2P + * is enabled. + * @PCI_P2PDMA_TLP_RELAXED_CPL: The provider returns Completions with the + * Relaxed Ordering attribute set. PCIe r7.0 sec 6.12.1.1 never redirects + * those, so ACS P2P Completion Redirect does not gate the path. The + * Completer chooses this attribute and the specification does not require + * it to copy Relaxed Ordering from the Request into the Completion, so a + * caller passing this flag asserts that its provider does. + */ +enum pci_p2pdma_tlp_flags { + PCI_P2PDMA_TLP_TRANSLATED =3D 1 << 0, + PCI_P2PDMA_TLP_RELAXED_CPL =3D 1 << 1, +}; + +/* Every combination of the flags above selects one routing class. */ +#define PCI_P2PDMA_TLP_CLASSES \ + ((PCI_P2PDMA_TLP_TRANSLATED | PCI_P2PDMA_TLP_RELAXED_CPL) + 1) + enum pci_p2pdma_map_type { /* * PCI_P2PDMA_MAP_UNKNOWN: Used internally as an initial state before @@ -86,8 +113,9 @@ int pci_p2pdma_enable_store(const char *page, struct pci= _dev **p2p_dev, bool *use_p2pdma); ssize_t pci_p2pdma_enable_show(char *page, struct pci_dev *p2p_dev, bool use_p2pdma); -enum pci_p2pdma_map_type pci_p2pdma_map_type(struct p2pdma_provider *provi= der, - struct device *dev); +enum pci_p2pdma_map_type +pci_p2pdma_map_type_tlp(struct p2pdma_provider *provider, struct device *d= ev, + unsigned int tlp_flags); #else /* CONFIG_PCI_P2PDMA */ static inline int pcim_p2pdma_init(struct pci_dev *pdev) { @@ -150,7 +178,8 @@ static inline ssize_t pci_p2pdma_enable_show(char *page, return sprintf(page, "none\n"); } static inline enum pci_p2pdma_map_type -pci_p2pdma_map_type(struct p2pdma_provider *provider, struct device *dev) +pci_p2pdma_map_type_tlp(struct p2pdma_provider *provider, struct device *d= ev, + unsigned int tlp_flags) { return PCI_P2PDMA_MAP_NOT_SUPPORTED; } @@ -168,6 +197,20 @@ static inline struct pci_dev *pci_p2pmem_find(struct d= evice *client) return pci_p2pmem_find_many(&client, 1); } =20 +/** + * pci_p2pdma_map_type - Determine the mapping type for P2PDMA transfers + * @provider: P2PDMA provider structure + * @dev: Client device that initiates the transfer + * + * Same as pci_p2pdma_map_type_tlp() for a client issuing strictly ordered + * Requests that carry an Untranslated address. + */ +static inline enum pci_p2pdma_map_type +pci_p2pdma_map_type(struct p2pdma_provider *provider, struct device *dev) +{ + return pci_p2pdma_map_type_tlp(provider, dev, 0); +} + struct pci_p2pdma_map_state { struct p2pdma_provider *mem; enum pci_p2pdma_map_type map; --=20 2.55.0 From nobody Fri Sep 25 16:56:34 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id DA9C447A87F; Thu, 10 Sep 2026 11:32:48 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789039970; cv=none; b=dpH4KDQEv85Ymivm6xsL/ZyRs1eXpjGHRny8XtPhguZtTGVqLWa0UczorbE4+JmLIDQiSklDcBfmXPWLpuiSVKhpYLLr8jKnzbzWl+6uCMguyOUOEIav7OwCU71/eOgTf+rzHlpGalV+TsGdmWRQHqLS8a/rxhnys1uH1huMIBs= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789039970; c=relaxed/simple; bh=P2xPREMOgoBKZLanV7o7qaQOyqC/Ow4GhPacLX4idWs=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=DguLyzQp821EA2P9VtzG4yKVDMsAD4BIDn1W7Yde2CHB+xx0bFzeLCuA73HrD2detHcrDN0rxyvNsOe7YcP3D2+pVemMYIpmZvjpHn1QFzPcLqbxFqw7vnZMqcPFDb+vDvHVTaL2Zr/TU+Ny1ATDugssGWJP0cj3wCpGFi8mVRE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=FYYnw0pN; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="FYYnw0pN" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 75EDD1F00893; Thu, 10 Sep 2026 11:32:47 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1789039968; bh=6iVQXJrfzqj9H9Mn6XbNg8yLfGKcJ/HPEopWBqbnUfE=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=FYYnw0pNDucrBfisYbPo4EzYmBSrkTBtO/V4OVKYu8xDzGF6uagI6O4ssyReZzKf8 70mVSOp+v0P4nTizehY4D9rSn84D9kDG7FiqBGMFa3cLyoZYco6daNHoRwrDqeoNmQ CF4OoJi+rNBFbGLedqlWhbAavPU2J6htZapHdT4fHAJgmslsRt6zo6Ln/7SW4td6hW RgviG7nlCs0Nt7+c/mRmR8V7qjQ42T1ypWVlmMSuRy6mjrbAB+EGOLwY2t7fFO8Mni NskOAtwsOuRULfejMPkTLeF+4MkZjF4tAH391H0RZXl5fqEMJAOGQT7VpH3Ng1FSc3 pTDTxZv5ChKpQ== From: Leon Romanovsky To: Bjorn Helgaas , Logan Gunthorpe , Chaitanya Kulkarni , Greg Kroah-Hartman , Jens Axboe , Alex Williamson , Leon Romanovsky , Ankit Agrawal , Jason Gunthorpe , Jonathan Corbet , Shuah Khan , "Joerg Roedel (AMD)" , Will Deacon , Robin Murphy , Randy Dunlap , Sumit Semwal , =?utf-8?q?Christian_K=C3=B6nig?= Cc: linux-pci@vger.kernel.org, linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, iommu@lists.linux.dev, Tushar Dave , linux-media@vger.kernel.org, dri-devel@lists.freedesktop.org, linaro-mm-sig@lists.linaro.org, linux-rdma@vger.kernel.org, kvm@vger.kernel.org Subject: [PATCH v5 09/18] PCI/P2PDMA: Route Relaxed Ordering Completions directly Date: Thu, 10 Sep 2026 14:32:04 +0300 Message-ID: <20260910-fix-p2p-acs-v4-0-v5-9-856087f63c0d@nvidia.com> X-Mailer: git-send-email 2.55.0 In-Reply-To: <20260910-fix-p2p-acs-v4-0-v5-0-856087f63c0d@nvidia.com> References: <20260910-fix-p2p-acs-v4-0-v5-0-856087f63c0d@nvidia.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" X-Mailer: b4 0.15-dev-18f8f Content-Transfer-Encoding: quoted-printable From: Leon Romanovsky ACS P2P Completion Redirect leaves Completions carrying the Relaxed Ordering attribute alone. PCIe r7.0 sec 6.12.1.1 redirects only those "that do not have the Relaxed Ordering Attribute bit set", and sec 7.7.12.5 describes the enable bit as "applicable only to Completions whose Relaxed Ordering Attribute is clear". P2PDMA reports one answer for every kind of TLP, so a client whose provider returns such Completions is sent through the host bridge for a redirect that never happens to it. Add enum pci_p2pdma_tlp_flags and let a caller state that property. Signed-off-by: Leon Romanovsky --- drivers/pci/p2pdma.c | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/drivers/pci/p2pdma.c b/drivers/pci/p2pdma.c index fac765d0a16f..327b6a14e07d 100644 --- a/drivers/pci/p2pdma.c +++ b/drivers/pci/p2pdma.c @@ -540,11 +540,15 @@ pci_acs_p2pdma_request(u16 ctrl, unsigned int tlp_fla= gs) /* * Decide how a peer-to-peer Completion at an ACS-capable ingress port rou= tes. * PCIe r7.0 sec 6.12.1.1: no ACS control other than P2P Completion Redire= ct - * affects a Completion. + * affects a Completion, and that one leaves Completions carrying the Rela= xed + * Ordering attribute alone. */ static enum pci_acs_p2pdma_state pci_acs_p2pdma_completion(u16 ctrl, unsigned int tlp_flags) { + if (tlp_flags & PCI_P2PDMA_TLP_RELAXED_CPL) + return PCI_ACS_P2PDMA_DIRECT; + return ctrl & PCI_ACS_CR ? PCI_ACS_P2PDMA_REDIRECT : PCI_ACS_P2PDMA_DIRECT; } --=20 2.55.0 From nobody Fri Sep 25 16:56:34 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id F2B704749DD; Thu, 10 Sep 2026 11:32:51 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789039974; cv=none; b=makn3WuBynTHJnDO4FAGCS3G5qwYsqWRstatBpVpVmU64RlIm1b9khvzRaALdrgRog5luUtqGhnq/NqknZFJRrtyiul21MmW6Px8e2fHkTcYaaoSMcl5w10vXuLgzjvVUwpmoA1CYdWpD9G3Njjl/gFoR3iOLncwf5miVWgtVWk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789039974; c=relaxed/simple; bh=/t+DL64LGojkS4yQIOZuTZFiWTkp2eiaWsTM89A9nnY=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=FLoO5yW1XOR0+RHEavbykjN0nhrW8WJjCZNeXxOXszTOxFbO854rzYjBw/Ml3aCxKR83G5/fBBKuZjh12eMH5QNxKDpI08KdS0dYCga8vtQMbgT41d9DqTkxFeTFd2LYOLZB05YjpYUPka/hWhnUupxPoYOqH3JuMQlyxU1qMVs= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=V3xWakNj; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="V3xWakNj" Received: by smtp.kernel.org (Postfix) with ESMTPSA id B8A2D1F000FF; Thu, 10 Sep 2026 11:32:50 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1789039971; bh=Z4R3ZMvcrefwd60N4dKDap5+w8BxOMwsC13REnzGF7c=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=V3xWakNjoNFwTQ6xPwyw5rY/wSXnVQaYG8eQ2yGGTeoNpWu7v+0pQo/1kUP7RfVeD s5TwvpyLG73eXnOj4K7Rjbd9KTMcmvfEjEqhOUs10AtudStObQr0VyfeNwPPJAc7yk gzq34h3ogSzMd0zenmwMsPXIY+CON6nABzPWwHFVs92xU8YE37RzAVpUa+MGJYzVbp wYBu6px0lvqYIYaDaSVxT9JHP04zUpiZsJaYSFP2lTv8W3a8wP9BU+ZqF0KOnukomE 7OpP8E3EWhUsEmrZ90H4nqICcAiVec8KDbop9h1QGxhRtGqGgfKst1pRGypHlcjjXb Dfrb2UxYGgVPQ== From: Leon Romanovsky To: Bjorn Helgaas , Logan Gunthorpe , Chaitanya Kulkarni , Greg Kroah-Hartman , Jens Axboe , Alex Williamson , Leon Romanovsky , Ankit Agrawal , Jason Gunthorpe , Jonathan Corbet , Shuah Khan , "Joerg Roedel (AMD)" , Will Deacon , Robin Murphy , Randy Dunlap , Sumit Semwal , =?utf-8?q?Christian_K=C3=B6nig?= Cc: linux-pci@vger.kernel.org, linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, iommu@lists.linux.dev, Tushar Dave , linux-media@vger.kernel.org, dri-devel@lists.freedesktop.org, linaro-mm-sig@lists.linaro.org, linux-rdma@vger.kernel.org, kvm@vger.kernel.org Subject: [PATCH v5 10/18] PCI/P2PDMA: Reject Translated Requests blocked by Translation Blocking Date: Thu, 10 Sep 2026 14:32:05 +0300 Message-ID: <20260910-fix-p2p-acs-v4-0-v5-10-856087f63c0d@nvidia.com> X-Mailer: git-send-email 2.55.0 In-Reply-To: <20260910-fix-p2p-acs-v4-0-v5-0-856087f63c0d@nvidia.com> References: <20260910-fix-p2p-acs-v4-0-v5-0-856087f63c0d@nvidia.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" X-Mailer: b4 0.15-dev-18f8f Content-Transfer-Encoding: quoted-printable From: Leon Romanovsky A Downstream Port with ACS Translation Blocking enabled treats every Upstream Memory Request whose Address Type is not Untranslated as an ACS Violation, ahead of "any applicable ACS P2P control mechanisms" per PCIe r7.0 sec 6.12.1.1. P2PDMA never looks at that bit, so it reports a bus-addressable path where an ATS client's Requests would be rejected. Add PCI_ACS_P2PDMA_BLOCKED, and because blocking is not a routing control, scan the whole client-side path for it rather than the divergence port alone. A blocked Request has no host bridge fallback, since the Address Type is rejected wherever the Request is addressed. Signed-off-by: Leon Romanovsky --- drivers/pci/p2pdma.c | 105 ++++++++++++++++++++++++++++++++++++++++++++++-= ---- 1 file changed, 96 insertions(+), 9 deletions(-) diff --git a/drivers/pci/p2pdma.c b/drivers/pci/p2pdma.c index 327b6a14e07d..94c0760f27e5 100644 --- a/drivers/pci/p2pdma.c +++ b/drivers/pci/p2pdma.c @@ -519,11 +519,12 @@ enum pci_acs_p2pdma_state { PCI_ACS_P2PDMA_NOT_SUPPORTED, PCI_ACS_P2PDMA_DIRECT, PCI_ACS_P2PDMA_REDIRECT, + PCI_ACS_P2PDMA_BLOCKED, }; =20 /* * Decide how a peer-to-peer Request at an ACS-capable ingress port routes, - * from that port's ACS Control register. + * from that port's ACS Control register and the Request's Address Type. * * Linux does not read the Egress Control Vector, so Egress Control is tre= ated * conservatively as a redirect. Per PCIe r7.0 Table 6-11 the outcomes it @@ -533,6 +534,18 @@ enum pci_acs_p2pdma_state { static enum pci_acs_p2pdma_state pci_acs_p2pdma_request(u16 ctrl, unsigned int tlp_flags) { + if (tlp_flags & PCI_P2PDMA_TLP_TRANSLATED) { + /* + * PCIe r7.0 sec 6.12.1.1: Translation Blocking makes every + * Upstream Memory Request whose Address Type is not + * Untranslated an ACS Violation, taking precedence over the + * P2P controls. Sec 7.7.12.5: Direct Translated P2P "is + * ignored if ACS Translation Blocking Enable is 1b". + */ + if (ctrl & PCI_ACS_TB) + return PCI_ACS_P2PDMA_BLOCKED; + } + return ctrl & (PCI_ACS_RR | PCI_ACS_EC) ? PCI_ACS_P2PDMA_REDIRECT : PCI_ACS_P2PDMA_DIRECT; } @@ -572,6 +585,35 @@ static bool pci_acs_p2pdma_ctrl(struct pci_dev *pdev, = u16 *ctrl) return !pci_read_config_word(pdev, pos + PCI_ACS_CTRL, ctrl); } =20 +/* + * Report whether any port between @client and @divergence rejects Transla= ted + * addresses. @common bounds the walk; @divergence itself is read along wi= th + * the routing controls. A port whose ACS Control cannot be read counts as + * blocking, which withdraws only the Translated classes because an + * Untranslated Request is routed at the divergence. + */ +static bool pci_p2pdma_path_blocks_translation(struct pci_dev *client, + struct pci_dev *divergence, + struct pci_dev *common) +{ + struct pci_dev *pdev; + u16 ctrl; + + for (pdev =3D pci_upstream_bridge(client); pdev && pdev !=3D common; + pdev =3D pci_upstream_bridge(pdev)) { + if (pdev =3D=3D divergence) + continue; + + if (!pci_acs_p2pdma_ctrl(pdev, &ctrl)) + return true; + + if (ctrl & PCI_ACS_TB) + return true; + } + + return false; +} + static void seq_buf_print_bus_devfn(struct seq_buf *buf, struct pci_dev *p= dev) { if (!buf) @@ -590,14 +632,35 @@ static void seq_buf_print_bus_devfn(struct seq_buf *b= uf, struct pci_dev *pdev) * than upstream, so it is where the Request controls apply. * @cpl_ctrl: ACS Control of the provider-side divergence port, likewise f= or * the Completions travelling back. + * @tb_below: A port between the client and the divergence blocks Translat= ed + * addresses. * @unreadable: First port whose ACS Control could not be read, if any. */ struct pci_p2pdma_acs_path { u16 req_ctrl; u16 cpl_ctrl; + bool tb_below; struct pci_dev *unreadable; }; =20 +/* + * ACS Translation Blocking is not a routing control, so unlike the redire= ct + * controls it is not decided at the divergence alone. PCIe r7.0 sec 6.12.= 1.1 + * has every Downstream Port check the Address Type of each Upstream Memory + * Request it receives, ahead of "any applicable ACS P2P control mechanism= s". + * A port below the divergence cannot redirect the Request anywhere it was= not + * already going, but it can still reject a Translated address. + */ +static enum pci_acs_p2pdma_state +pci_p2pdma_request_state(const struct pci_p2pdma_acs_path *path, + unsigned int tlp_flags) +{ + if (tlp_flags & PCI_P2PDMA_TLP_TRANSLATED && path->tb_below) + return PCI_ACS_P2PDMA_BLOCKED; + + return pci_acs_p2pdma_request(path->req_ctrl, tlp_flags); +} + /* * Combine both directions into a mapping type. Only a path that routes the * Request and the Completions it generates directly can be programmed with @@ -607,11 +670,23 @@ static enum pci_p2pdma_map_type pci_p2pdma_route(const struct pci_p2pdma_acs_path *path, unsigned int tlp_flags) { + enum pci_acs_p2pdma_state req; + if (path->unreadable) return PCI_P2PDMA_MAP_NOT_SUPPORTED; =20 - if (pci_acs_p2pdma_request(path->req_ctrl, tlp_flags) =3D=3D - PCI_ACS_P2PDMA_DIRECT && + req =3D pci_p2pdma_request_state(path, tlp_flags); + + /* + * Translation Blocking rejects the Address Type rather than the + * target, so a blocked Request stays blocked however it is addressed. + * No host bridge fallback keeps a Translated address working; the + * caller has to issue a different kind of Request instead. + */ + if (req =3D=3D PCI_ACS_P2PDMA_BLOCKED) + return PCI_P2PDMA_MAP_NOT_SUPPORTED; + + if (req =3D=3D PCI_ACS_P2PDMA_DIRECT && pci_acs_p2pdma_completion(path->cpl_ctrl, tlp_flags) =3D=3D PCI_ACS_P2PDMA_DIRECT) return PCI_P2PDMA_MAP_BUS_ADDR; @@ -640,12 +715,19 @@ static void pci_p2pdma_warn_path(struct pci_dev *clie= nt, return; } =20 + if (pci_p2pdma_request_state(path, tlp_flags) =3D=3D + PCI_ACS_P2PDMA_BLOCKED) { + pci_warn(client, + "ACS Translation Blocking rejects Translated Requests to provider %s\n= ", + pci_name(provider)); + return; + } + seq_buf_init(&acs_list, buf, sizeof(buf)); if (pci_acs_p2pdma_completion(path->cpl_ctrl, tlp_flags) !=3D PCI_ACS_P2PDMA_DIRECT) seq_buf_print_bus_devfn(&acs_list, a_child); - if (pci_acs_p2pdma_request(path->req_ctrl, tlp_flags) !=3D - PCI_ACS_P2PDMA_DIRECT) + if (pci_p2pdma_request_state(path, tlp_flags) !=3D PCI_ACS_P2PDMA_DIRECT) seq_buf_print_bus_devfn(&acs_list, b_child); =20 /* Drop the final semicolon; the list is not empty here. */ @@ -864,8 +946,11 @@ pci_p2pdma_map_types_unpack(unsigned long val, unsigne= d int tlp_flags) * redirection setting of the ports along the path. * * The client initiates Requests to provider memory. At the path divergenc= e, - * check Request Redirect and Egress Control on the client-side port, and - * Completion Redirect for read Completions on the provider-side port. + * check Request Redirect, Egress Control, Translation Blocking and Direct + * Translated P2P on the client-side port, and Completion Redirect for read + * Completions on the provider-side port. Translation Blocking is checked = on + * every client-side port instead, because it rejects a Request rather than + * routing it. * * Those controls apply to different TLPs, so every class named by &enum * pci_p2pdma_tlp_flags is decided from the one walk and cached together; @@ -873,8 +958,8 @@ pci_p2pdma_map_types_unpack(unsigned long val, unsigned= int tlp_flags) * * If ACS redirects traffic at either divergence port, return * PCI_P2PDMA_MAP_THRU_HOST_BRIDGE. If the ACS Control register cannot be - * read, return PCI_P2PDMA_MAP_NOT_SUPPORTED. Otherwise, return - * PCI_P2PDMA_MAP_BUS_ADDR. + * read, or Translation Blocking rejects the class being asked about, retu= rn + * PCI_P2PDMA_MAP_NOT_SUPPORTED. Otherwise, return PCI_P2PDMA_MAP_BUS_ADDR. * * Any two devices that have a data path that goes through the host bridge * will consult a whitelist. If the host bridge is in the whitelist, return @@ -945,6 +1030,8 @@ calc_map_type_and_dist(struct pci_dev *provider, struc= t pci_dev *client, if (!pci_acs_p2pdma_ctrl(b_child, &path.req_ctrl) && !path.unreadable) path.unreadable =3D b_child; + path.tb_below =3D pci_p2pdma_path_blocks_translation(client, + b_child, a); } =20 /* --=20 2.55.0 From nobody Fri Sep 25 16:56:34 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E748F47CA9F; Thu, 10 Sep 2026 11:32:54 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789039976; cv=none; b=djVODIeOGM4rPv4jq/92i3yu/QI61mWIl/bJc9YpVQbluJ8OleqPKWrWukG+bFCxLEWwB8IFRPu9IJC0VVSO3s8hrJJYBkDJC7TfTQEphb+xbZ/b7ntTlP98BiFdjUau+N4X6a5JGog9GSOkijV1Qlbhb5IiBUNkCW9ck6/CX/M= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789039976; c=relaxed/simple; bh=XiMBk2BktCZForHPeZMnVPWr9BtHOLFfS1HSzNZLaz4=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=ld0/cJzCISebp75/F0dp99oMhgTzncXMCJ+SDZtH00JObvSllEMpJfNAnYv3HYhHs23WpxfWsaZXH6fAF+M/2L/6wnkWNlBFauBgfTiCwq+N1ePvT8hDfDuaKwlJeP+s45eCoCUTGD18eeapU0lhfb/cXLBAI7jGNsoncZjgAxc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=Ccb3Piqx; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="Ccb3Piqx" Received: by smtp.kernel.org (Postfix) with ESMTPSA id D45BE1F00893; Thu, 10 Sep 2026 11:32:53 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1789039974; bh=cAJyIat4limGWSmrmndSZaY7FfmdAnSZ8M1EcuzJ9BU=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=Ccb3PiqxnrWAaVZDisld7hOwJ9PKqmHnfV7oR7h+sntAWIYV5I6F9QhyJx+Cd++XI VLT0kOSwl5CbzDhctzj2GZk1oFXizQ8rV6z1ab8LzJZk5IuQALQDCQ7oRnRA2ERZeb IqlU0Ymq6XfdOd016tt6R6ShrqgBdiv8/9Wez3ja9XOHul10Gyfx8U595ENSy3unZJ IuJOs3Nm/J1cLFudydh3Corm0GIHsHBXWdQFIfq+4QuDMt648o3IrOfRCYKoJr/AuB xsdNJrT8DZ0WBLJFEgJh7BDLzz5glE3mVHcuctp4rdv7R9tZ301S5rWBaENzwGzwMh XFqn/HhhQpB4w== From: Leon Romanovsky To: Bjorn Helgaas , Logan Gunthorpe , Chaitanya Kulkarni , Greg Kroah-Hartman , Jens Axboe , Alex Williamson , Leon Romanovsky , Ankit Agrawal , Jason Gunthorpe , Jonathan Corbet , Shuah Khan , "Joerg Roedel (AMD)" , Will Deacon , Robin Murphy , Randy Dunlap , Sumit Semwal , =?utf-8?q?Christian_K=C3=B6nig?= Cc: linux-pci@vger.kernel.org, linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, iommu@lists.linux.dev, Tushar Dave , linux-media@vger.kernel.org, dri-devel@lists.freedesktop.org, linaro-mm-sig@lists.linaro.org, linux-rdma@vger.kernel.org, kvm@vger.kernel.org Subject: [PATCH v5 11/18] PCI/P2PDMA: Route Translated Requests under Direct Translated P2P Date: Thu, 10 Sep 2026 14:32:06 +0300 Message-ID: <20260910-fix-p2p-acs-v4-0-v5-11-856087f63c0d@nvidia.com> X-Mailer: git-send-email 2.55.0 In-Reply-To: <20260910-fix-p2p-acs-v4-0-v5-0-856087f63c0d@nvidia.com> References: <20260910-fix-p2p-acs-v4-0-v5-0-856087f63c0d@nvidia.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" X-Mailer: b4 0.15-dev-18f8f Content-Transfer-Encoding: quoted-printable From: Leon Romanovsky A Downstream Port with ACS Direct Translated P2P enabled routes a Request whose Address Type is Translated "to the peer Egress Port without redirection, regardless of ACS P2P Request Redirect and ACS P2P Egress Control", per PCIe r7.0 sec 6.12.3. P2PDMA assumes every Request carries an Untranslated address, so it sends an ATS client through the host bridge even where the fabric would route it straight to the peer. Add PCI_P2PDMA_TLP_TRANSLATED and consult Direct Translated P2P for the Requests it describes. Signed-off-by: Leon Romanovsky --- drivers/pci/p2pdma.c | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/drivers/pci/p2pdma.c b/drivers/pci/p2pdma.c index 94c0760f27e5..77ff5cda8292 100644 --- a/drivers/pci/p2pdma.c +++ b/drivers/pci/p2pdma.c @@ -544,6 +544,15 @@ pci_acs_p2pdma_request(u16 ctrl, unsigned int tlp_flag= s) */ if (ctrl & PCI_ACS_TB) return PCI_ACS_P2PDMA_BLOCKED; + + /* + * PCIe r7.0 sec 6.12.3: ACS Direct Translated P2P routes a + * Request carrying a Translated address to the peer "without + * redirection, regardless of ACS P2P Request Redirect and ACS + * P2P Egress Control settings". + */ + if (ctrl & PCI_ACS_DT) + return PCI_ACS_P2PDMA_DIRECT; } =20 return ctrl & (PCI_ACS_RR | PCI_ACS_EC) ? --=20 2.55.0 From nobody Fri Sep 25 16:56:34 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 54E5947DD4F; Thu, 10 Sep 2026 11:32:57 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789039981; cv=none; b=BfohbSXfPnh/+zm37J2HcHIWjBDzjGPAJMNVQ+S7Md52lqds1s3PPqWXj27X4G4k6a5VX6j9ZX5FtL/42NhcW3CC7rOZUANabd+2hYklGEHkM1UQlZehEuHBSqbYnKjkiyIz8HMv2CWPe5rVJCoT6/5kxNyT/neXK8puZ3NpHAM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789039981; c=relaxed/simple; bh=3J/BAkSnt/TldpWEz6Jd4QgDRRvsWZgpbn0+fS0THXQ=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=aNvTy6wFHcSO/jVxagJIFUxOU0b/3fcOl7ph6cN6j5IXBFC2dD+ZFHrcU3fL54JqDMpQJ1aVr2AS3Ps/0ide+XS9ogxaUj78LTky3s14FBDWBuSDpEP86H4B7MJ9xTn5L3N6kmpZGw+2ETigkv06Yw+KMDyobQtMQn6gGX87QmY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=QKwUFI1V; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="QKwUFI1V" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 5124C1F00893; Thu, 10 Sep 2026 11:32:57 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1789039977; bh=PX+KsmKZS0DEpR9HsspFfs6iCGAAzzB54XY2c9EF4hc=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=QKwUFI1VIp47JzviFnNfgPEuqtzx2yc+UTyukN3Oka/tWsAdFMBZHb0U/ByhPk3Iu 95+XLRMLeq/2qfq/LJuO5is+MK9VdP8e2NpQB513WsVYafpGLweAW+ckNj1nSL65Al 0xNrpP2oJ+hcaOle+EjaCy5JtVjtvIZnHP0ojfZOrtbyBnP07ynPrVtwVePQVlMo31 DqOZ54igdzzvDNCqalv2kI8yjUylIlMLDeizcGn/GZn+ShPFa71hvFJJcz/juSSlDC LxOa1O2s/idxsCsj3Hvv6kObBli6zQjAYgbqfTQ99VDLxt6OYqncrdhQ5vbuCPqCes B1p02sWQOmqOQ== From: Leon Romanovsky To: Bjorn Helgaas , Logan Gunthorpe , Chaitanya Kulkarni , Greg Kroah-Hartman , Jens Axboe , Alex Williamson , Leon Romanovsky , Ankit Agrawal , Jason Gunthorpe , Jonathan Corbet , Shuah Khan , "Joerg Roedel (AMD)" , Will Deacon , Robin Murphy , Randy Dunlap , Sumit Semwal , =?utf-8?q?Christian_K=C3=B6nig?= Cc: linux-pci@vger.kernel.org, linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, iommu@lists.linux.dev, Tushar Dave , linux-media@vger.kernel.org, dri-devel@lists.freedesktop.org, linaro-mm-sig@lists.linaro.org, linux-rdma@vger.kernel.org, kvm@vger.kernel.org Subject: [PATCH v5 12/18] PCI/P2PDMA: Log detailed ACS routing diagnostics Date: Thu, 10 Sep 2026 14:32:07 +0300 Message-ID: <20260910-fix-p2p-acs-v4-0-v5-12-856087f63c0d@nvidia.com> X-Mailer: git-send-email 2.55.0 In-Reply-To: <20260910-fix-p2p-acs-v4-0-v5-0-856087f63c0d@nvidia.com> References: <20260910-fix-p2p-acs-v4-0-v5-0-856087f63c0d@nvidia.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" X-Mailer: b4 0.15-dev-18f8f Content-Transfer-Encoding: quoted-printable From: Leon Romanovsky When P2PDMA rejects a mapping, existing warnings identify only the final ACS or host-bridge result. They omit topology, live controls, divergence ports, cache state, and intermediate routing decisions. Emit debug-level messages for verbose calculations and cache lookups. Report both paths, decoded ACS controls, directional decisions, host fallback, and the final mapping. This keeps incidental unsupported probes quiet while allowing the diagnostics to be enabled when needed. Signed-off-by: Leon Romanovsky --- drivers/pci/p2pdma.c | 227 +++++++++++++++++++++++++++++++++++++++++++++++= ---- 1 file changed, 212 insertions(+), 15 deletions(-) diff --git a/drivers/pci/p2pdma.c b/drivers/pci/p2pdma.c index 77ff5cda8292..60bd9ec5f993 100644 --- a/drivers/pci/p2pdma.c +++ b/drivers/pci/p2pdma.c @@ -575,23 +575,128 @@ pci_acs_p2pdma_completion(u16 ctrl, unsigned int tlp= _flags) PCI_ACS_P2PDMA_DIRECT; } =20 +static const char *pci_acs_p2pdma_state_name(enum pci_acs_p2pdma_state sta= te) +{ + switch (state) { + case PCI_ACS_P2PDMA_DIRECT: + return "direct"; + case PCI_ACS_P2PDMA_REDIRECT: + return "redirect"; + case PCI_ACS_P2PDMA_BLOCKED: + return "blocked"; + case PCI_ACS_P2PDMA_NOT_SUPPORTED: + return "not-supported"; + } + + return "invalid"; +} + +static const char *pci_p2pdma_map_type_name(enum pci_p2pdma_map_type type) +{ + switch (type) { + case PCI_P2PDMA_MAP_UNKNOWN: + return "unknown"; + case PCI_P2PDMA_MAP_NONE: + return "none"; + case PCI_P2PDMA_MAP_NOT_SUPPORTED: + return "not-supported"; + case PCI_P2PDMA_MAP_BUS_ADDR: + return "bus-address"; + case PCI_P2PDMA_MAP_THRU_HOST_BRIDGE: + return "through-host-bridge"; + } + + return "invalid"; +} + /* * Read @pdev's ACS Control register. A device without an ACS capability h= as * no peer-to-peer controls at all, which routes the same as having them a= ll * clear. Returns false when the register is present but cannot be read; @= ctrl * is then meaningless. */ -static bool pci_acs_p2pdma_ctrl(struct pci_dev *pdev, u16 *ctrl) +static bool pci_acs_p2pdma_ctrl(struct pci_dev *pdev, const char *what, + u16 *ctrl, bool verbose) { - int pos; + int pos, ret; =20 pos =3D pdev->acs_cap; if (!pos) { + if (verbose) + pci_dbg(pdev, + "P2PDMA ACS: %s has no ACS capability\n", what); *ctrl =3D 0; return true; } =20 - return !pci_read_config_word(pdev, pos + PCI_ACS_CTRL, ctrl); + ret =3D pci_read_config_word(pdev, pos + PCI_ACS_CTRL, ctrl); + if (ret) { + if (verbose) + pci_dbg(pdev, + "P2PDMA ACS: %s ACS Control read failed at %#x: %#x\n", + what, pos + PCI_ACS_CTRL, ret); + return false; + } + + if (verbose) { + pci_dbg(pdev, + "P2PDMA ACS: %s cap=3D%#x caps=3D%#06x ctrl=3D%#06x\n", + what, pos, pdev->acs_capabilities, *ctrl); + pci_dbg(pdev, + "P2PDMA ACS: control bits SV=3D%u TB=3D%u RR=3D%u CR=3D%u UF=3D%u EC= =3D%u DT=3D%u\n", + !!(*ctrl & PCI_ACS_SV), !!(*ctrl & PCI_ACS_TB), + !!(*ctrl & PCI_ACS_RR), !!(*ctrl & PCI_ACS_CR), + !!(*ctrl & PCI_ACS_UF), !!(*ctrl & PCI_ACS_EC), + !!(*ctrl & PCI_ACS_DT)); + } + + return true; +} + +static void pci_p2pdma_log_path(const char *name, struct pci_dev *start, + struct pci_dev *common) +{ + struct pci_dev *pdev, *upstream; + int hop =3D 0, ret, type; + u16 ctrl; + + for (pdev =3D start; pdev; pdev =3D upstream, hop++) { + upstream =3D pci_upstream_bridge(pdev); + type =3D pci_is_pcie(pdev) ? pci_pcie_type(pdev) : -1; + pci_dbg(pdev, + "P2PDMA ACS: %s path hop=3D%d common=3D%u pcie=3D%u type=3D%d class=3D= %#08x vendor=3D%04x device=3D%04x upstream=3D%s\n", + name, hop, pdev =3D=3D common, pci_is_pcie(pdev), type, + pdev->class, pdev->vendor, pdev->device, + upstream ? pci_name(upstream) : ""); + + if (pdev->subordinate) + pci_dbg(pdev, + "P2PDMA ACS: bridge bus range=3D%02llx-%02llx\n", + (unsigned long long)pdev->subordinate->busn_res.start, + (unsigned long long)pdev->subordinate->busn_res.end); + + if (!pdev->acs_cap) { + pci_dbg(pdev, "P2PDMA ACS: ACS capability absent\n"); + continue; + } + + ret =3D pci_read_config_word(pdev, pdev->acs_cap + PCI_ACS_CTRL, + &ctrl); + if (ret) { + pci_dbg(pdev, + "P2PDMA ACS: ACS cap=3D%#x caps=3D%#06x Control read failed: %#x\n", + pdev->acs_cap, pdev->acs_capabilities, ret); + continue; + } + + pci_dbg(pdev, + "P2PDMA ACS: ACS cap=3D%#x caps=3D%#06x ctrl=3D%#06x SV=3D%u TB=3D%u RR= =3D%u CR=3D%u UF=3D%u EC=3D%u DT=3D%u\n", + pdev->acs_cap, pdev->acs_capabilities, ctrl, + !!(ctrl & PCI_ACS_SV), !!(ctrl & PCI_ACS_TB), + !!(ctrl & PCI_ACS_RR), !!(ctrl & PCI_ACS_CR), + !!(ctrl & PCI_ACS_UF), !!(ctrl & PCI_ACS_EC), + !!(ctrl & PCI_ACS_DT)); + } } =20 /* @@ -603,7 +708,8 @@ static bool pci_acs_p2pdma_ctrl(struct pci_dev *pdev, u= 16 *ctrl) */ static bool pci_p2pdma_path_blocks_translation(struct pci_dev *client, struct pci_dev *divergence, - struct pci_dev *common) + struct pci_dev *common, + bool verbose) { struct pci_dev *pdev; u16 ctrl; @@ -613,11 +719,15 @@ static bool pci_p2pdma_path_blocks_translation(struct= pci_dev *client, if (pdev =3D=3D divergence) continue; =20 - if (!pci_acs_p2pdma_ctrl(pdev, &ctrl)) + if (!pci_acs_p2pdma_ctrl(pdev, "path hop", &ctrl, verbose)) return true; =20 - if (ctrl & PCI_ACS_TB) + if (ctrl & PCI_ACS_TB) { + if (verbose) + pci_dbg(pdev, + "P2PDMA ACS: Translation Blocking rejects Translated Requests on this= path\n"); return true; + } } =20 return false; @@ -983,14 +1093,22 @@ calc_map_type_and_dist(struct pci_dev *provider, str= uct pci_dev *client, enum pci_p2pdma_map_type map_type[PCI_P2PDMA_TLP_CLASSES]; struct pci_dev *a =3D provider, *b =3D client, *bb; struct pci_dev *a_child =3D NULL, *b_child =3D NULL; + struct pci_host_bridge *provider_host, *client_host; struct pci_p2pdma_acs_path path =3D {}; struct pci_p2pdma *p2pdma; bool cpu_p2pdma, host_whitelisted =3D false; + bool cache_store =3D false; bool host_fallback =3D false; unsigned int flags; int dist_a =3D 0; int dist_b =3D 0; =20 + if (verbose) + pci_dbg(client, + "P2PDMA ACS: begin provider=3D%s client=3D%s cache-index=3D%#lx\n", + pci_name(provider), pci_name(client), + map_types_idx(client)); + /* * Note, we don't need to take references to devices returned by * pci_upstream_bridge() seeing we hold a reference to a child @@ -1021,12 +1139,29 @@ calc_map_type_and_dist(struct pci_dev *provider, st= ruct pci_dev *client, * request can only get to the peer through the host bridge. */ *dist =3D dist_a + dist_b; + if (verbose) { + pci_dbg(client, + "P2PDMA ACS: no common upstream bridge provider-distance=3D%d client-di= stance=3D%d total=3D%d\n", + dist_a, dist_b, *dist); + pci_p2pdma_log_path("provider", provider, NULL); + pci_p2pdma_log_path("client", client, NULL); + } for (flags =3D 0; flags < PCI_P2PDMA_TLP_CLASSES; flags++) map_type[flags] =3D PCI_P2PDMA_MAP_THRU_HOST_BRIDGE; goto map_through_host_bridge; =20 check_paths_acs: *dist =3D dist_a + dist_b; + if (verbose) { + pci_dbg(client, + "P2PDMA ACS: common=3D%s provider-divergence=3D%s client-divergence=3D%= s provider-distance=3D%d client-distance=3D%d total=3D%d\n", + pci_name(a), + a_child ? pci_name(a_child) : "", + b_child ? pci_name(b_child) : "", + dist_a, dist_b, *dist); + pci_p2pdma_log_path("provider", provider, a); + pci_p2pdma_log_path("client", client, a); + } =20 /* * ACS P2P routing controls apply where a TLP can route toward the peer @@ -1034,13 +1169,29 @@ calc_map_type_and_dist(struct pci_dev *provider, st= ruct pci_dev *client, * branch is upstream, so redirect controls do not affect the path. */ if (a_child && b_child) { - if (!pci_acs_p2pdma_ctrl(a_child, &path.cpl_ctrl)) + if (!pci_acs_p2pdma_ctrl(a_child, "completion", &path.cpl_ctrl, + verbose)) path.unreadable =3D a_child; - if (!pci_acs_p2pdma_ctrl(b_child, &path.req_ctrl) && - !path.unreadable) + if (!pci_acs_p2pdma_ctrl(b_child, "request", &path.req_ctrl, + verbose) && !path.unreadable) path.unreadable =3D b_child; path.tb_below =3D pci_p2pdma_path_blocks_translation(client, - b_child, a); + b_child, a, + verbose); + + if (verbose && !path.unreadable) + pci_dbg(client, + "P2PDMA ACS: request=3D%s at %s completion=3D%s at %s\n", + pci_acs_p2pdma_state_name( + pci_p2pdma_request_state(&path, 0)), + pci_name(b_child), + pci_acs_p2pdma_state_name( + pci_acs_p2pdma_completion(path.cpl_ctrl, + 0)), + pci_name(a_child)); + } else if (verbose) { + pci_dbg(client, + "P2PDMA ACS: peer divergence is incomplete; no ACS peer-routing control= s evaluated\n"); } =20 /* @@ -1070,6 +1221,19 @@ calc_map_type_and_dist(struct pci_dev *provider, str= uct pci_dev *client, host_whitelisted =3D host_bridge_whitelist(provider, client, verbose); =20 + if (verbose) { + provider_host =3D pci_find_host_bridge(provider->bus); + client_host =3D pci_find_host_bridge(client->bus); + pci_dbg(client, + "P2PDMA ACS: host fallback cpu-support=3D%u whitelist=3D%s provider-hos= t=3D%s client-host=3D%s same-host=3D%u\n", + cpu_p2pdma, + cpu_p2pdma ? "not-consulted" : + (host_whitelisted ? "yes" : "no"), + provider_host ? dev_name(&provider_host->dev) : "", + client_host ? dev_name(&client_host->dev) : "", + provider_host && provider_host =3D=3D client_host); + } + if (!cpu_p2pdma && !host_whitelisted) { if (verbose) pci_warn(client, "cannot be used for peer-to-peer DMA as the client and= provider (%s) do not share an upstream bridge or whitelisted host bridge\n= ", @@ -1081,11 +1245,31 @@ calc_map_type_and_dist(struct pci_dev *provider, st= ruct pci_dev *client, done: rcu_read_lock(); p2pdma =3D rcu_dereference(provider->p2pdma); - if (p2pdma) + if (p2pdma) { xa_store(&p2pdma->map_types, map_types_idx(client), - xa_mk_value(pci_p2pdma_map_types_pack(map_type)), - GFP_ATOMIC); + xa_mk_value(pci_p2pdma_map_types_pack(map_type)), GFP_ATOMIC); + cache_store =3D true; + } rcu_read_unlock(); + if (verbose) { + pci_dbg(client, + "P2PDMA ACS: final provider=3D%s result=3D%s(%d) tlp-flags=3D%#x distan= ce=3D%d unreadable=3D%s cache-store=3D%u index=3D%#lx\n", + pci_name(provider), + pci_p2pdma_map_type_name(map_type[tlp_flags]), + map_type[tlp_flags], tlp_flags, *dist, + path.unreadable ? pci_name(path.unreadable) : "", + cache_store, map_types_idx(client)); + pci_dbg(client, + "P2PDMA ACS: classes strict=3D%s relaxed=3D%s translated=3D%s translate= d+relaxed=3D%s\n", + pci_p2pdma_map_type_name(map_type[0]), + pci_p2pdma_map_type_name( + map_type[PCI_P2PDMA_TLP_RELAXED_CPL]), + pci_p2pdma_map_type_name( + map_type[PCI_P2PDMA_TLP_TRANSLATED]), + pci_p2pdma_map_type_name( + map_type[PCI_P2PDMA_TLP_TRANSLATED | + PCI_P2PDMA_TLP_RELAXED_CPL])); + } return map_type[tlp_flags]; } =20 @@ -1415,16 +1599,24 @@ pci_p2pdma_map_type_tlp(struct p2pdma_provider *pro= vider, struct device *dev, enum pci_p2pdma_map_type type; struct pci_p2pdma *p2pdma; struct pci_dev *client; + bool provider_state; int dist; =20 if (WARN_ON_ONCE(tlp_flags >=3D PCI_P2PDMA_TLP_CLASSES)) return PCI_P2PDMA_MAP_NOT_SUPPORTED; =20 - if (!pdev->p2pdma) + if (!pdev->p2pdma) { + pci_dbg(pdev, + "P2PDMA ACS: map lookup rejected; provider state is absent\n"); return PCI_P2PDMA_MAP_NOT_SUPPORTED; + } =20 - if (!dev_is_pci(dev)) + if (!dev_is_pci(dev)) { + dev_dbg(dev, + "P2PDMA ACS: provider=3D%s map lookup rejected; client is not PCI\n", + pci_name(pdev)); return PCI_P2PDMA_MAP_NOT_SUPPORTED; + } =20 client =3D to_pci_dev(dev); cache_index =3D map_types_idx(client); @@ -1435,8 +1627,13 @@ pci_p2pdma_map_type_tlp(struct p2pdma_provider *prov= ider, struct device *dev, if (p2pdma) cached =3D xa_to_value(xa_load(&p2pdma->map_types, cache_index)); + provider_state =3D !!p2pdma; rcu_read_unlock(); type =3D pci_p2pdma_map_types_unpack(cached, tlp_flags); + pci_dbg(client, + "P2PDMA ACS: map lookup provider=3D%s index=3D%#lx tlp-flags=3D%#x cach= ed=3D%s(%d) provider-state=3D%u\n", + pci_name(pdev), cache_index, tlp_flags, + pci_p2pdma_map_type_name(type), type, provider_state); =20 if (type =3D=3D PCI_P2PDMA_MAP_UNKNOWN) return calc_map_type_and_dist(pdev, client, &dist, tlp_flags, --=20 2.55.0 From nobody Fri Sep 25 16:56:34 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 29606481FC2; Thu, 10 Sep 2026 11:33:16 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789039999; cv=none; b=WdGMDHRD2AZbXuCfhP+71u8+G6BHg+GeqmC4fl/0oXFkXkhnAipfjlOIx5dlUhZatK9X6k2Hg2WQzHJ+xkfETgqw3C9UIfgBzUn2arDN0DR22d2C3xn/KiawGq26XzFLN2A2J3bNL+zB2t/WyUDc3eTyHTuUqP7GwOSkqIEx4aI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789039999; c=relaxed/simple; bh=AhSwuYq8izIYeDSBz6OCwbM12Cifd8T1NxqOGMNPwsU=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=ojXmwB4jj6kETCZK3wyPEKa6eifZIpqz76sNLTqGWvFovH30oJx6xFb4tOrbU1zNAPdX6yw8gWeXNcwH4M0MvYgGYyrI5udxYWZFMRjjEnOjoLSJSoe42PLC3W7rGUkUbHEE08HQ3gN/Xw3jH40KuB5ABUvUE6OXwy4Z4TuwT7Q= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=YD9lGSLI; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="YD9lGSLI" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 385781F00893; Thu, 10 Sep 2026 11:33:15 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1789039995; bh=YwVvJhRoi+Lr5LSgnNr8iWhw0NvaGa2ylJrrKFpMPaI=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=YD9lGSLIf498vHtkHjXYHrg3tRoHnA6KLALmZyJN6hSvhOx6t/ZghnY+QyRBpQTWB bjxuZaJ3tUwLZIHj8yrTZ3mT46d6mZ2/ecVpNMtZXZq2X8E3Rs6Axl3VCA1Hfxg1cO 8n/rkApNJZahESrfTKQMbbH9nNPc6jyuFkBgJ/5OO+CZVXhp4hAP9jh63/A0K80idj UXGVWvgGr5M71T6MFnSbVM/Jz4NOjVln/M8pYR7qqZVdjDQKQIr6ODydAMNgYR8tLH xnnwbdMsLSAu91VQolijkZf870hqnAg5hAEXpB6dvoUV+K2e2fZ9vIJ4l0QgEAvHtT w1yX+Qgkd0jRQ== From: Leon Romanovsky To: Bjorn Helgaas , Logan Gunthorpe , Chaitanya Kulkarni , Greg Kroah-Hartman , Jens Axboe , Alex Williamson , Leon Romanovsky , Ankit Agrawal , Jason Gunthorpe , Jonathan Corbet , Shuah Khan , "Joerg Roedel (AMD)" , Will Deacon , Robin Murphy , Randy Dunlap , Sumit Semwal , =?utf-8?q?Christian_K=C3=B6nig?= Cc: linux-pci@vger.kernel.org, linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, iommu@lists.linux.dev, Tushar Dave , linux-media@vger.kernel.org, dri-devel@lists.freedesktop.org, linaro-mm-sig@lists.linaro.org, linux-rdma@vger.kernel.org, kvm@vger.kernel.org Subject: [PATCH v5 13/18] PCI/P2PDMA: Add KUnit tests for the ACS routing decisions Date: Thu, 10 Sep 2026 14:32:08 +0300 Message-ID: <20260910-fix-p2p-acs-v4-0-v5-13-856087f63c0d@nvidia.com> X-Mailer: git-send-email 2.55.0 In-Reply-To: <20260910-fix-p2p-acs-v4-0-v5-0-856087f63c0d@nvidia.com> References: <20260910-fix-p2p-acs-v4-0-v5-0-856087f63c0d@nvidia.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" X-Mailer: b4 0.15-dev-18f8f Content-Transfer-Encoding: quoted-printable From: Leon Romanovsky pci_acs_p2pdma_request() and pci_acs_p2pdma_completion() turn an ACS Control register and a TLP class into a routing decision. Which bits apply to which direction and which class is easy to get wrong, and hardware that exposes a given combination may not be at hand. Drive both from a table of register values and classes, covering the redirect controls per direction and Translation Blocking, Direct Translated P2P and Relaxed Ordering. Direct Translated P2P gets a case with and without a redirect to override, since it changes nothing without one. Exposing the two helpers moves their state enum into pci.h. Signed-off-by: Leon Romanovsky --- drivers/pci/Kconfig | 15 ++++++ drivers/pci/Makefile | 1 + drivers/pci/p2pdma.c | 13 ++--- drivers/pci/pci.h | 16 ++++++ drivers/pci/pci_acs_test.c | 121 +++++++++++++++++++++++++++++++++++++++++= ++++ 5 files changed, 157 insertions(+), 9 deletions(-) diff --git a/drivers/pci/Kconfig b/drivers/pci/Kconfig index 0c7408509ba2..7a3eb5beb328 100644 --- a/drivers/pci/Kconfig +++ b/drivers/pci/Kconfig @@ -226,6 +226,21 @@ config PCI_P2PDMA =20 If unsure, say N. =20 +config PCI_ACS_KUNIT_TEST + tristate "KUnit tests for PCI ACS P2P routing" if !KUNIT_ALL_TESTS + depends on PCI_P2PDMA && KUNIT + default KUNIT_ALL_TESTS + help + Enable KUnit tests for the PCI ACS peer-to-peer routing decision + logic, including direction-specific Request and Completion + controls that cannot all be exercised on typical peer-to-peer + hardware. + + For more information on KUnit and unit tests in general, refer to + the KUnit documentation in Documentation/dev-tools/kunit/. + + If unsure, say N. + config PCI_LABEL def_bool y if (DMI || ACPI) select NLS diff --git a/drivers/pci/Makefile b/drivers/pci/Makefile index 41ebc3b9a518..6305d128d3df 100644 --- a/drivers/pci/Makefile +++ b/drivers/pci/Makefile @@ -31,6 +31,7 @@ obj-$(CONFIG_PCI_STUB) +=3D pci-stub.o obj-$(CONFIG_PCI_PF_STUB) +=3D pci-pf-stub.o obj-$(CONFIG_PCI_ECAM) +=3D ecam.o obj-$(CONFIG_PCI_P2PDMA) +=3D p2pdma.o +obj-$(CONFIG_PCI_ACS_KUNIT_TEST) +=3D pci_acs_test.o obj-$(CONFIG_XEN_PCIDEV_FRONTEND) +=3D xen-pcifront.o obj-$(CONFIG_VGA_ARB) +=3D vgaarb.o obj-$(CONFIG_PCI_DOE) +=3D doe.o diff --git a/drivers/pci/p2pdma.c b/drivers/pci/p2pdma.c index 60bd9ec5f993..955e3b41ec1f 100644 --- a/drivers/pci/p2pdma.c +++ b/drivers/pci/p2pdma.c @@ -515,13 +515,6 @@ static struct pci_dev *find_parent_pci_dev(struct devi= ce *dev) return NULL; } =20 -enum pci_acs_p2pdma_state { - PCI_ACS_P2PDMA_NOT_SUPPORTED, - PCI_ACS_P2PDMA_DIRECT, - PCI_ACS_P2PDMA_REDIRECT, - PCI_ACS_P2PDMA_BLOCKED, -}; - /* * Decide how a peer-to-peer Request at an ACS-capable ingress port routes, * from that port's ACS Control register and the Request's Address Type. @@ -531,7 +524,7 @@ enum pci_acs_p2pdma_state { * selects are a direct route and an ACS Violation, and neither one lets p= eer * bus addressing be assumed. */ -static enum pci_acs_p2pdma_state +VISIBLE_IF_KUNIT enum pci_acs_p2pdma_state pci_acs_p2pdma_request(u16 ctrl, unsigned int tlp_flags) { if (tlp_flags & PCI_P2PDMA_TLP_TRANSLATED) { @@ -558,6 +551,7 @@ pci_acs_p2pdma_request(u16 ctrl, unsigned int tlp_flags) return ctrl & (PCI_ACS_RR | PCI_ACS_EC) ? PCI_ACS_P2PDMA_REDIRECT : PCI_ACS_P2PDMA_DIRECT; } +EXPORT_SYMBOL_IF_KUNIT(pci_acs_p2pdma_request); =20 /* * Decide how a peer-to-peer Completion at an ACS-capable ingress port rou= tes. @@ -565,7 +559,7 @@ pci_acs_p2pdma_request(u16 ctrl, unsigned int tlp_flags) * affects a Completion, and that one leaves Completions carrying the Rela= xed * Ordering attribute alone. */ -static enum pci_acs_p2pdma_state +VISIBLE_IF_KUNIT enum pci_acs_p2pdma_state pci_acs_p2pdma_completion(u16 ctrl, unsigned int tlp_flags) { if (tlp_flags & PCI_P2PDMA_TLP_RELAXED_CPL) @@ -574,6 +568,7 @@ pci_acs_p2pdma_completion(u16 ctrl, unsigned int tlp_fl= ags) return ctrl & PCI_ACS_CR ? PCI_ACS_P2PDMA_REDIRECT : PCI_ACS_P2PDMA_DIRECT; } +EXPORT_SYMBOL_IF_KUNIT(pci_acs_p2pdma_completion); =20 static const char *pci_acs_p2pdma_state_name(enum pci_acs_p2pdma_state sta= te) { diff --git a/drivers/pci/pci.h b/drivers/pci/pci.h index ba3c3fddddc2..c0d2711d73f9 100644 --- a/drivers/pci/pci.h +++ b/drivers/pci/pci.h @@ -2,6 +2,7 @@ #ifndef DRIVERS_PCI_H #define DRIVERS_PCI_H =20 +#include #include #include #include @@ -1093,6 +1094,21 @@ resource_size_t pci_min_window_alignment(struct pci_= bus *bus, =20 void pci_acs_init(struct pci_dev *dev); void pci_enable_acs(struct pci_dev *dev); + +enum pci_acs_p2pdma_state { + PCI_ACS_P2PDMA_NOT_SUPPORTED, + PCI_ACS_P2PDMA_DIRECT, + PCI_ACS_P2PDMA_REDIRECT, + PCI_ACS_P2PDMA_BLOCKED, +}; + +#if IS_ENABLED(CONFIG_KUNIT) +enum pci_acs_p2pdma_state pci_acs_p2pdma_request(u16 ctrl, + unsigned int tlp_flags); +enum pci_acs_p2pdma_state pci_acs_p2pdma_completion(u16 ctrl, + unsigned int tlp_flags); +#endif + #ifdef CONFIG_PCI_QUIRKS int pci_dev_specific_acs_enabled(struct pci_dev *dev, u16 acs_flags); int pci_dev_specific_enable_acs(struct pci_dev *dev); diff --git a/drivers/pci/pci_acs_test.c b/drivers/pci/pci_acs_test.c new file mode 100644 index 000000000000..ce6b9375da36 --- /dev/null +++ b/drivers/pci/pci_acs_test.c @@ -0,0 +1,121 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * KUnit tests for PCI ACS peer-to-peer routing decisions. + * + * These exercise Request and Completion routing independently of the ACS + * settings exposed by available PCIe hardware. + */ +#include + +#include +#include +#include + +#include "pci.h" + +struct acs_decision_case { + const char *desc; + u16 ctrl; + unsigned int tlp_flags; + enum pci_acs_p2pdma_state expect; +}; + +/* Shorthands to keep the tables below readable. */ +#define ACS_DIRECT PCI_ACS_P2PDMA_DIRECT +#define ACS_REDIR PCI_ACS_P2PDMA_REDIRECT +#define ACS_RO PCI_P2PDMA_TLP_RELAXED_CPL +#define ACS_AT PCI_P2PDMA_TLP_TRANSLATED +#define ACS_BLOCK PCI_ACS_P2PDMA_BLOCKED + +/* Request routing ignores Completion Redirect. */ +static const struct acs_decision_case acs_request_cases[] =3D { + { "req/none", 0, 0, ACS_DIRECT }, + { "req/rr", PCI_ACS_RR, 0, ACS_REDIR }, + { "req/cr", PCI_ACS_CR, 0, ACS_DIRECT }, + { "req/rr_cr", PCI_ACS_RR | PCI_ACS_CR, 0, ACS_REDIR }, + { "req/ec", PCI_ACS_EC, 0, ACS_REDIR }, + { "req/ec_cr", PCI_ACS_EC | PCI_ACS_CR, 0, ACS_REDIR }, + + /* + * Direct Translated P2P overrides the redirect controls, but only for + * a Request that actually carries a Translated address. + */ + { "req/dt", PCI_ACS_DT, 0, ACS_DIRECT }, + { "req/dt_rr", PCI_ACS_DT | PCI_ACS_RR, 0, ACS_REDIR }, + { "req/at", 0, ACS_AT, ACS_DIRECT }, + { "req/at_rr", PCI_ACS_RR, ACS_AT, ACS_REDIR }, + { "req/at_dt_rr", PCI_ACS_DT | PCI_ACS_RR, ACS_AT, ACS_DIRECT }, + { "req/at_dt_ec", PCI_ACS_DT | PCI_ACS_EC, ACS_AT, ACS_DIRECT }, + + /* + * Translation Blocking rejects a Translated address outright, and + * makes the port ignore Direct Translated P2P. + */ + { "req/tb", PCI_ACS_TB, 0, ACS_DIRECT }, + { "req/tb_rr", PCI_ACS_TB | PCI_ACS_RR, 0, ACS_REDIR }, + { "req/at_tb", PCI_ACS_TB, ACS_AT, ACS_BLOCK }, + { "req/at_tb_dt", PCI_ACS_TB | PCI_ACS_DT, ACS_AT, ACS_BLOCK }, +}; + +/* Completion routing depends only on Completion Redirect. */ +static const struct acs_decision_case acs_completion_cases[] =3D { + { "cpl/none", 0, 0, ACS_DIRECT }, + { "cpl/rr", PCI_ACS_RR, 0, ACS_DIRECT }, + { "cpl/cr", PCI_ACS_CR, 0, ACS_REDIR }, + { "cpl/rr_cr", PCI_ACS_RR | PCI_ACS_CR, 0, ACS_REDIR }, + { "cpl/ec", PCI_ACS_EC, 0, ACS_DIRECT }, + { "cpl/ec_cr", PCI_ACS_EC | PCI_ACS_CR, 0, ACS_REDIR }, + + /* Relaxed Ordering Completions are never redirected. */ + { "cpl/ro", 0, ACS_RO, ACS_DIRECT }, + { "cpl/ro_cr", PCI_ACS_CR, ACS_RO, ACS_DIRECT }, + { "cpl/ro_rr_cr", PCI_ACS_RR | PCI_ACS_CR, ACS_RO, ACS_DIRECT }, +}; + +#undef ACS_DIRECT +#undef ACS_REDIR +#undef ACS_RO +#undef ACS_AT +#undef ACS_BLOCK + +static void acs_decision_desc(const struct acs_decision_case *c, char *des= c) +{ + strscpy(desc, c->desc, KUNIT_PARAM_DESC_SIZE); +} + +KUNIT_ARRAY_PARAM(acs_request, acs_request_cases, acs_decision_desc); +KUNIT_ARRAY_PARAM(acs_completion, acs_completion_cases, acs_decision_desc); + +static void pci_acs_p2pdma_request_test(struct kunit *test) +{ + const struct acs_decision_case *c =3D test->param_value; + + KUNIT_EXPECT_EQ(test, pci_acs_p2pdma_request(c->ctrl, c->tlp_flags), + c->expect); +} + +static void pci_acs_p2pdma_completion_test(struct kunit *test) +{ + const struct acs_decision_case *c =3D test->param_value; + + KUNIT_EXPECT_EQ(test, pci_acs_p2pdma_completion(c->ctrl, c->tlp_flags), + c->expect); +} + +static struct kunit_case pci_acs_test_cases[] =3D { + KUNIT_CASE_PARAM(pci_acs_p2pdma_request_test, + acs_request_gen_params), + KUNIT_CASE_PARAM(pci_acs_p2pdma_completion_test, + acs_completion_gen_params), + {} +}; + +static struct kunit_suite pci_acs_test_suite =3D { + .name =3D "pci_acs", + .test_cases =3D pci_acs_test_cases, +}; +kunit_test_suite(pci_acs_test_suite); + +MODULE_IMPORT_NS("EXPORTED_FOR_KUNIT_TESTING"); +MODULE_LICENSE("GPL"); +MODULE_DESCRIPTION("KUnit tests for PCI ACS peer-to-peer routing decisions= "); --=20 2.55.0 From nobody Fri Sep 25 16:56:34 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E3F0E475349; Thu, 10 Sep 2026 11:33:05 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789039988; cv=none; b=EBs51uPvUt1yxhUG8PSOQQY5I1KqAjCOBVYAr3Uax+7GlIaGXugPmo8upRO2f/Wydx+Q0HZxnviHHR5t1BOf63SfQJVdEu2di0m20ChvTExUagfDzpqcm68+JzA0skmNoM715hLHzS6Oy4jl/7lQ9/LTEO/RrjHni3QM474SCpI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789039988; c=relaxed/simple; bh=IsGvpzHxKgGZ1avS9Y2ZIY62aFibIVbZXXmCdjyKlyE=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=PobKA42gji/rHU+3JjZHYqImJesk04nVnEHTQfuwNMJUdIcKWa7YLTWF8atRLWJMNvY6VOkbbBndHW4CbgffZvfETPghTqX3ErDKZu2zBE9OT7uFn4FK7QnVy+q8uZgAOgFvSZ+Qa/YuuEHoYPxomyXnUOD3WBoYVm+D1cu723Y= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=JtzKKkuY; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="JtzKKkuY" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 7DBCB1F00893; Thu, 10 Sep 2026 11:33:04 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1789039985; bh=0kbSch0SERCu06PJMYS0TfO7WiOJ0Ulzs7nJ+u2rzoQ=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=JtzKKkuY8LaaQSbg6R8dGLJqSpB0tTodqN9wYUiMDBsQCwfIuxY3Rh8vaZyzZDu2a eWjiEYEvCqjP0NX3uRaE0toa5NxkPkwX5/VSnKolX6t0w+z5Zr7z0/3tcRwMBjapXl LawOPqK5TENTwMNGD0tgekJ9XJovYdwbUaQaUW1oT3lQlKPowmfcAWhaIMLZKW4d3e T6jq+14eB4vqaCAbeu6YwWlqDuaVa2ct4uoTHc8MfMbRMu6XfRIGLQnWb4R6dEmjmS qE6LXCw6GO8beVxOjZGSPYSt3ZKpx2YUaV+ig1PJE5irDKAjktfoPa6kAw6k/SCba5 /ATHsMZ78G7fQ== From: Leon Romanovsky To: Bjorn Helgaas , Logan Gunthorpe , Chaitanya Kulkarni , Greg Kroah-Hartman , Jens Axboe , Alex Williamson , Leon Romanovsky , Ankit Agrawal , Jason Gunthorpe , Jonathan Corbet , Shuah Khan , "Joerg Roedel (AMD)" , Will Deacon , Robin Murphy , Randy Dunlap , Sumit Semwal , =?utf-8?q?Christian_K=C3=B6nig?= Cc: linux-pci@vger.kernel.org, linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, iommu@lists.linux.dev, Tushar Dave , linux-media@vger.kernel.org, dri-devel@lists.freedesktop.org, linaro-mm-sig@lists.linaro.org, linux-rdma@vger.kernel.org, kvm@vger.kernel.org Subject: [PATCH v5 14/18] PCI/P2PDMA: Test the ACS P2P routing walk Date: Thu, 10 Sep 2026 14:32:09 +0300 Message-ID: <20260910-fix-p2p-acs-v4-0-v5-14-856087f63c0d@nvidia.com> X-Mailer: git-send-email 2.55.0 In-Reply-To: <20260910-fix-p2p-acs-v4-0-v5-0-856087f63c0d@nvidia.com> References: <20260910-fix-p2p-acs-v4-0-v5-0-856087f63c0d@nvidia.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" X-Mailer: b4 0.15-dev-18f8f Content-Transfer-Encoding: quoted-printable From: Leon Romanovsky calc_map_type_and_dist() decides which ports along a path carry the routing controls, and holds every class's answer in one cache entry. Neither depends on a single register, so a table of them cannot reach the walk itself. Drive the walk over a fabricated fabric of two devices below a switch, with fake config space supplying the ACS Control registers. Cover the ports below the divergence, the three cases where two classes of one path disagree, and the packed cache, which the fabric has no provider state to exercise indirectly. Signed-off-by: Leon Romanovsky --- drivers/pci/p2pdma.c | 9 +- drivers/pci/pci.h | 9 ++ drivers/pci/pci_acs_test.c | 350 +++++++++++++++++++++++++++++++++++++++++= ++++ 3 files changed, 365 insertions(+), 3 deletions(-) diff --git a/drivers/pci/p2pdma.c b/drivers/pci/p2pdma.c index 955e3b41ec1f..e522be1372f7 100644 --- a/drivers/pci/p2pdma.c +++ b/drivers/pci/p2pdma.c @@ -1016,7 +1016,7 @@ static unsigned long map_types_idx(struct pci_dev *cl= ient) */ static_assert(PCI_P2PDMA_MAP_THRU_HOST_BRIDGE < 16); =20 -static unsigned long +VISIBLE_IF_KUNIT unsigned long pci_p2pdma_map_types_pack(const enum pci_p2pdma_map_type *type) { unsigned long val =3D 0; @@ -1027,12 +1027,14 @@ pci_p2pdma_map_types_pack(const enum pci_p2pdma_map= _type *type) =20 return val; } +EXPORT_SYMBOL_IF_KUNIT(pci_p2pdma_map_types_pack); =20 -static enum pci_p2pdma_map_type +VISIBLE_IF_KUNIT enum pci_p2pdma_map_type pci_p2pdma_map_types_unpack(unsigned long val, unsigned int tlp_flags) { return (val >> (tlp_flags * 4)) & 0xf; } +EXPORT_SYMBOL_IF_KUNIT(pci_p2pdma_map_types_unpack); =20 /* * Calculate the P2PDMA mapping type and distance between two PCI devices. @@ -1081,7 +1083,7 @@ pci_p2pdma_map_types_unpack(unsigned long val, unsign= ed int tlp_flags) * ports per above. If the device is not in the whitelist, return * PCI_P2PDMA_MAP_NOT_SUPPORTED. */ -static enum pci_p2pdma_map_type +VISIBLE_IF_KUNIT enum pci_p2pdma_map_type calc_map_type_and_dist(struct pci_dev *provider, struct pci_dev *client, int *dist, unsigned int tlp_flags, bool verbose) { @@ -1267,6 +1269,7 @@ calc_map_type_and_dist(struct pci_dev *provider, stru= ct pci_dev *client, } return map_type[tlp_flags]; } +EXPORT_SYMBOL_IF_KUNIT(calc_map_type_and_dist); =20 /** * pci_p2pdma_distance_many - Determine the cumulative distance between diff --git a/drivers/pci/pci.h b/drivers/pci/pci.h index c0d2711d73f9..56f821e40637 100644 --- a/drivers/pci/pci.h +++ b/drivers/pci/pci.h @@ -7,6 +7,7 @@ #include #include #include +#include #include =20 struct pcie_tlp_log; @@ -1107,6 +1108,14 @@ enum pci_acs_p2pdma_state pci_acs_p2pdma_request(u16= ctrl, unsigned int tlp_flags); enum pci_acs_p2pdma_state pci_acs_p2pdma_completion(u16 ctrl, unsigned int tlp_flags); +unsigned long pci_p2pdma_map_types_pack(const enum pci_p2pdma_map_type *ty= pe); +enum pci_p2pdma_map_type pci_p2pdma_map_types_unpack(unsigned long val, + unsigned int tlp_flags); +enum pci_p2pdma_map_type calc_map_type_and_dist(struct pci_dev *provider, + struct pci_dev *client, + int *dist, + unsigned int tlp_flags, + bool verbose); #endif =20 #ifdef CONFIG_PCI_QUIRKS diff --git a/drivers/pci/pci_acs_test.c b/drivers/pci/pci_acs_test.c index ce6b9375da36..28eced6dd672 100644 --- a/drivers/pci/pci_acs_test.c +++ b/drivers/pci/pci_acs_test.c @@ -102,11 +102,361 @@ static void pci_acs_p2pdma_completion_test(struct ku= nit *test) c->expect); } =20 +/* + * Drive calc_map_type_and_dist() over a fabricated PCIe fabric matching t= he + * canonical topology of two devices below one switch: + * + * host bridge / root bus + * Root Port + * Switch Upstream Port + * Switch Downstream Port 0 + * Nested Switch -- provider + * Switch Downstream Port 1 + * Nested Switch -- client + * + * Fake config-space operations supply the ACS Control registers. This lets + * the cases vary both divergence ports and controls below the divergence + * without depending on real hardware. + */ +struct acs_port_cfg { + u16 ctrl; + bool fail_read; +}; + +struct acs_fabric { + struct pci_dev *provider; + struct pci_dev *client; + struct pci_dev *dn0; /* Downstream Port 0 (provider side) */ + struct pci_dev *dn1; /* Downstream Port 1 (client side) */ + struct pci_dev *provider_leaf; + struct pci_dev *client_leaf; + struct acs_port_cfg dn0_cfg; + struct acs_port_cfg dn1_cfg; + struct acs_port_cfg provider_leaf_cfg; + struct acs_port_cfg client_leaf_cfg; +}; + +static int acs_port_read(struct pci_dev *port, struct acs_port_cfg *cfg, + int where, int size, u32 *val) +{ + if (port->acs_cap && size =3D=3D 2 && + where =3D=3D port->acs_cap + PCI_ACS_CTRL) { + if (cfg->fail_read) + return PCIBIOS_DEVICE_NOT_FOUND; + *val =3D cfg->ctrl; + } + + return PCIBIOS_SUCCESSFUL; +} + +static int acs_fabric_read(struct pci_bus *bus, unsigned int devfn, + int where, int size, u32 *val) +{ + struct acs_fabric *f =3D bus->sysdata; + + *val =3D 0; + if (bus =3D=3D f->dn0->bus && devfn =3D=3D f->dn0->devfn) + return acs_port_read(f->dn0, &f->dn0_cfg, where, size, val); + if (bus =3D=3D f->dn1->bus && devfn =3D=3D f->dn1->devfn) + return acs_port_read(f->dn1, &f->dn1_cfg, where, size, val); + if (bus =3D=3D f->provider_leaf->bus && + devfn =3D=3D f->provider_leaf->devfn) + return acs_port_read(f->provider_leaf, &f->provider_leaf_cfg, + where, size, val); + if (bus =3D=3D f->client_leaf->bus && devfn =3D=3D f->client_leaf->devfn) + return acs_port_read(f->client_leaf, &f->client_leaf_cfg, + where, size, val); + + return PCIBIOS_SUCCESSFUL; +} + +static int acs_fabric_write(struct pci_bus *bus, unsigned int devfn, + int where, int size, u32 val) +{ + return PCIBIOS_SUCCESSFUL; +} + +static struct pci_ops acs_fabric_ops =3D { + .read =3D acs_fabric_read, + .write =3D acs_fabric_write, +}; + +static struct pci_bus *acs_add_bus(struct kunit *test, struct pci_bus *par= ent, + struct pci_dev *self, u8 nr, void *sysdata) +{ + struct pci_bus *bus =3D kunit_kzalloc(test, sizeof(*bus), GFP_KERNEL); + + KUNIT_ASSERT_NOT_NULL(test, bus); + bus->parent =3D parent; + bus->self =3D self; + bus->number =3D nr; + bus->ops =3D &acs_fabric_ops; + bus->sysdata =3D sysdata; + INIT_LIST_HEAD(&bus->devices); + return bus; +} + +static struct pci_dev *acs_add_dev(struct kunit *test, struct pci_bus *bus, + unsigned int devfn, int pcie_type) +{ + struct pci_dev *dev =3D kunit_kzalloc(test, sizeof(*dev), GFP_KERNEL); + + KUNIT_ASSERT_NOT_NULL(test, dev); + dev->bus =3D bus; + dev->devfn =3D devfn; + dev->pcie_cap =3D 0x40; + dev->pcie_flags_reg =3D (pcie_type << 4) | 0x2; + list_add_tail(&dev->bus_list, &bus->devices); + return dev; +} + +static void acs_build_fabric(struct kunit *test, struct acs_fabric *f) +{ + struct pci_bus *bus0, *bus1, *bus2, *bus3, *bus4, *bus5, *bus6; + struct pci_bus *bus7, *bus8; + struct pci_dev *rootport, *swup, *provider_swup, *client_swup; + struct pci_host_bridge *host; + + host =3D kunit_kzalloc(test, sizeof(*host), GFP_KERNEL); + KUNIT_ASSERT_NOT_NULL(test, host); + + bus0 =3D acs_add_bus(test, NULL, NULL, 0, f); + /* The Root Port doubles as the whitelisted host-bridge device. */ + rootport =3D acs_add_dev(test, bus0, PCI_DEVFN(0, 0), + PCI_EXP_TYPE_ROOT_PORT); + rootport->vendor =3D PCI_VENDOR_ID_GOOGLE; + rootport->device =3D 0x1234; + host->bus =3D bus0; + bus0->bridge =3D &host->dev; + + bus1 =3D acs_add_bus(test, bus0, rootport, 1, f); + swup =3D acs_add_dev(test, bus1, PCI_DEVFN(0, 0), + PCI_EXP_TYPE_UPSTREAM); + + bus2 =3D acs_add_bus(test, bus1, swup, 2, f); + f->dn0 =3D acs_add_dev(test, bus2, PCI_DEVFN(0, 0), + PCI_EXP_TYPE_DOWNSTREAM); + f->dn1 =3D acs_add_dev(test, bus2, PCI_DEVFN(1, 0), + PCI_EXP_TYPE_DOWNSTREAM); + + bus3 =3D acs_add_bus(test, bus2, f->dn0, 3, f); + provider_swup =3D acs_add_dev(test, bus3, PCI_DEVFN(0, 0), + PCI_EXP_TYPE_UPSTREAM); + bus5 =3D acs_add_bus(test, bus3, provider_swup, 5, f); + f->provider_leaf =3D acs_add_dev(test, bus5, PCI_DEVFN(0, 0), + PCI_EXP_TYPE_DOWNSTREAM); + bus7 =3D acs_add_bus(test, bus5, f->provider_leaf, 7, f); + f->provider =3D acs_add_dev(test, bus7, PCI_DEVFN(0, 0), + PCI_EXP_TYPE_ENDPOINT); + + bus4 =3D acs_add_bus(test, bus2, f->dn1, 4, f); + client_swup =3D acs_add_dev(test, bus4, PCI_DEVFN(0, 0), + PCI_EXP_TYPE_UPSTREAM); + bus6 =3D acs_add_bus(test, bus4, client_swup, 6, f); + f->client_leaf =3D acs_add_dev(test, bus6, PCI_DEVFN(0, 0), + PCI_EXP_TYPE_DOWNSTREAM); + bus8 =3D acs_add_bus(test, bus6, f->client_leaf, 8, f); + f->client =3D acs_add_dev(test, bus8, PCI_DEVFN(0, 0), + PCI_EXP_TYPE_ENDPOINT); +} + +static enum pci_p2pdma_map_type acs_walk_map(struct acs_fabric *f, + unsigned int tlp_flags) +{ + int dist; + + return calc_map_type_and_dist(f->provider, f->client, &dist, tlp_flags, + false); +} + +static void acs_walk_bus_addr_test(struct kunit *test) +{ + struct acs_fabric f =3D {}; + + acs_build_fabric(test, &f); + KUNIT_EXPECT_EQ(test, acs_walk_map(&f, 0), PCI_P2PDMA_MAP_BUS_ADDR); +} + +static void acs_walk_request_redirect_test(struct kunit *test) +{ + struct acs_fabric f =3D {}; + + acs_build_fabric(test, &f); + f.dn1->acs_cap =3D 0x100; + f.dn1->acs_capabilities =3D PCI_ACS_RR; + f.dn1_cfg.ctrl =3D PCI_ACS_RR; + + KUNIT_EXPECT_EQ(test, acs_walk_map(&f, 0), + PCI_P2PDMA_MAP_THRU_HOST_BRIDGE); +} + +static void acs_walk_completion_redirect_test(struct kunit *test) +{ + struct acs_fabric f =3D {}; + + acs_build_fabric(test, &f); + f.dn0->acs_cap =3D 0x100; + f.dn0->acs_capabilities =3D PCI_ACS_CR; + f.dn0_cfg.ctrl =3D PCI_ACS_CR; + + KUNIT_EXPECT_EQ(test, acs_walk_map(&f, 0), + PCI_P2PDMA_MAP_THRU_HOST_BRIDGE); +} + +static void acs_walk_egress_control_test(struct kunit *test) +{ + struct acs_fabric f =3D {}; + + acs_build_fabric(test, &f); + f.dn1->acs_cap =3D 0x100; + f.dn1->acs_capabilities =3D PCI_ACS_EC; + f.dn1_cfg.ctrl =3D PCI_ACS_EC; + + KUNIT_EXPECT_EQ(test, acs_walk_map(&f, 0), + PCI_P2PDMA_MAP_THRU_HOST_BRIDGE); +} + +static void acs_walk_asymmetric_direct_test(struct kunit *test) +{ + struct acs_fabric f =3D {}; + + acs_build_fabric(test, &f); + /* These controls affect only the reverse transaction directions. */ + f.dn0->acs_cap =3D 0x100; + f.dn0->acs_capabilities =3D PCI_ACS_RR | PCI_ACS_EC; + f.dn0_cfg.ctrl =3D PCI_ACS_RR | PCI_ACS_EC; + f.dn1->acs_cap =3D 0x100; + f.dn1->acs_capabilities =3D PCI_ACS_CR; + f.dn1_cfg.ctrl =3D PCI_ACS_CR; + + KUNIT_EXPECT_EQ(test, acs_walk_map(&f, 0), PCI_P2PDMA_MAP_BUS_ADDR); +} + +static void acs_walk_nested_completion_redirect_test(struct kunit *test) +{ + struct acs_fabric f =3D {}; + + acs_build_fabric(test, &f); + f.provider_leaf->acs_cap =3D 0x100; + f.provider_leaf->acs_capabilities =3D PCI_ACS_CR; + f.provider_leaf_cfg.ctrl =3D PCI_ACS_CR; + + KUNIT_EXPECT_EQ(test, acs_walk_map(&f, 0), PCI_P2PDMA_MAP_BUS_ADDR); +} + +static void acs_walk_nested_request_redirect_test(struct kunit *test) +{ + struct acs_fabric f =3D {}; + + acs_build_fabric(test, &f); + f.client_leaf->acs_cap =3D 0x100; + f.client_leaf->acs_capabilities =3D PCI_ACS_RR; + f.client_leaf_cfg.ctrl =3D PCI_ACS_RR; + + KUNIT_EXPECT_EQ(test, acs_walk_map(&f, 0), PCI_P2PDMA_MAP_BUS_ADDR); +} + +static void acs_walk_translation_blocking_test(struct kunit *test) +{ + struct acs_fabric f =3D {}; + + acs_build_fabric(test, &f); + f.client_leaf->acs_cap =3D 0x100; + f.client_leaf->acs_capabilities =3D PCI_ACS_TB; + f.client_leaf_cfg.ctrl =3D PCI_ACS_TB; + + /* Untranslated Requests are unaffected by Translation Blocking. */ + KUNIT_EXPECT_EQ(test, acs_walk_map(&f, 0), PCI_P2PDMA_MAP_BUS_ADDR); + KUNIT_EXPECT_EQ(test, acs_walk_map(&f, PCI_P2PDMA_TLP_TRANSLATED), + PCI_P2PDMA_MAP_NOT_SUPPORTED); +} + +static void acs_walk_relaxed_completion_test(struct kunit *test) +{ + struct acs_fabric f =3D {}; + + acs_build_fabric(test, &f); + f.dn0->acs_cap =3D 0x100; + f.dn0->acs_capabilities =3D PCI_ACS_CR; + f.dn0_cfg.ctrl =3D PCI_ACS_CR; + + KUNIT_EXPECT_EQ(test, acs_walk_map(&f, 0), + PCI_P2PDMA_MAP_THRU_HOST_BRIDGE); + KUNIT_EXPECT_EQ(test, acs_walk_map(&f, PCI_P2PDMA_TLP_RELAXED_CPL), + PCI_P2PDMA_MAP_BUS_ADDR); +} + +static void acs_walk_direct_translated_test(struct kunit *test) +{ + struct acs_fabric f =3D {}; + + acs_build_fabric(test, &f); + f.dn1->acs_cap =3D 0x100; + f.dn1->acs_capabilities =3D PCI_ACS_RR | PCI_ACS_DT; + f.dn1_cfg.ctrl =3D PCI_ACS_RR | PCI_ACS_DT; + + KUNIT_EXPECT_EQ(test, acs_walk_map(&f, 0), + PCI_P2PDMA_MAP_THRU_HOST_BRIDGE); + KUNIT_EXPECT_EQ(test, acs_walk_map(&f, PCI_P2PDMA_TLP_TRANSLATED), + PCI_P2PDMA_MAP_BUS_ADDR); +} + +/* + * The cache stores one packed value per client, so every class has to come + * back out under the flags that selected it. + */ +static void acs_map_types_pack_test(struct kunit *test) +{ + static const enum pci_p2pdma_map_type type[PCI_P2PDMA_TLP_CLASSES] =3D { + [0] =3D PCI_P2PDMA_MAP_THRU_HOST_BRIDGE, + [PCI_P2PDMA_TLP_TRANSLATED] =3D PCI_P2PDMA_MAP_NOT_SUPPORTED, + [PCI_P2PDMA_TLP_RELAXED_CPL] =3D PCI_P2PDMA_MAP_BUS_ADDR, + [PCI_P2PDMA_TLP_TRANSLATED | PCI_P2PDMA_TLP_RELAXED_CPL] =3D + PCI_P2PDMA_MAP_UNKNOWN, + }; + unsigned long packed =3D pci_p2pdma_map_types_pack(type); + unsigned int flags; + + for (flags =3D 0; flags < PCI_P2PDMA_TLP_CLASSES; flags++) + KUNIT_EXPECT_EQ(test, + pci_p2pdma_map_types_unpack(packed, flags), + type[flags]); + + /* An absent cache entry reads back as unknown in every class. */ + for (flags =3D 0; flags < PCI_P2PDMA_TLP_CLASSES; flags++) + KUNIT_EXPECT_EQ(test, pci_p2pdma_map_types_unpack(0, flags), + PCI_P2PDMA_MAP_UNKNOWN); +} + +static void acs_walk_unreadable_control_test(struct kunit *test) +{ + struct acs_fabric f =3D {}; + + acs_build_fabric(test, &f); + f.dn1->acs_cap =3D 0x100; + f.dn1_cfg.fail_read =3D true; + + KUNIT_EXPECT_EQ(test, acs_walk_map(&f, 0), + PCI_P2PDMA_MAP_NOT_SUPPORTED); +} + static struct kunit_case pci_acs_test_cases[] =3D { KUNIT_CASE_PARAM(pci_acs_p2pdma_request_test, acs_request_gen_params), KUNIT_CASE_PARAM(pci_acs_p2pdma_completion_test, acs_completion_gen_params), + KUNIT_CASE(acs_walk_bus_addr_test), + KUNIT_CASE(acs_walk_request_redirect_test), + KUNIT_CASE(acs_walk_completion_redirect_test), + KUNIT_CASE(acs_walk_egress_control_test), + KUNIT_CASE(acs_walk_asymmetric_direct_test), + KUNIT_CASE(acs_walk_nested_completion_redirect_test), + KUNIT_CASE(acs_walk_nested_request_redirect_test), + KUNIT_CASE(acs_walk_translation_blocking_test), + KUNIT_CASE(acs_walk_relaxed_completion_test), + KUNIT_CASE(acs_walk_direct_translated_test), + KUNIT_CASE(acs_walk_unreadable_control_test), + KUNIT_CASE(acs_map_types_pack_test), {} }; =20 --=20 2.55.0 From nobody Fri Sep 25 16:56:34 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 46584480962; Thu, 10 Sep 2026 11:33:08 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789039991; cv=none; b=hTrW4mt7KLAiaQoSFIuFt+mO9G9ZTxDJgh0bMk+lNzzZo2LA8npWg+HfAeyxoLOLymFQXMB3pkjT5oXUul/R9k6vAouY9ly7BC/CctZVWydCMHeqeY++K5y9qHrWe3kgbYQnLpW5PX7c0tprS78gPT9gGGORGRHN47CDGSI/rZ8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789039991; c=relaxed/simple; bh=0K5rUpBpGWQFTY3uzLBKiWWTUeOLwszMt+xh11hdHPc=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=I6zjvlp+cvppz1GbZYVuc+pO+QFYrJulTnmE83lAA0/5jdIu3lbcqJBfSp5y/ce/wTUn/XbCcXbPva0OH6krzwwM2ELeVOpZlbA6TXOIVz1u9oGJW9P0iu2EKpTPTb1IaiA0Px5J7WCTS75UaYCuNkvv9cJG4FBLJJq5MaWEOvk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=V/Yf24Jp; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="V/Yf24Jp" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 358971F0089A; Thu, 10 Sep 2026 11:33:08 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1789039988; bh=ezjTrl262v9AlY/XGNLFM9lj31ECNwVLoicbe3EAPpQ=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=V/Yf24JpSkv1iNBpprcI6ILu+GMcaZ6rb0ZzfjHxP7UPTP1eYU3alkHH+c1H//Ny5 nmFEbIXMhczAQO6KfVyB1RmfmUIxxnt7wG3GNs9HnN4ua+sWo63ZTD9Jf+vlMQxfqI GcaoKRDwQ7V2A3mXwRx5ZIR7LiCCZ5FczMge9l+2TT1lL0UQCMUzvqahaRjercBVho iBrWvuNBHfQYpAnbHjvqkorEQtqkc3Z594aZsWSyBe1SPbc60RzshkKUECzDk5Ai+8 iXR5Eqgg1ntr8nJ7DWiaskUPu2VLor1AlKVqrMyvwbzrT2DrUUEahwrZ5+p9iHFClt Mfjlw6+CEJVZw== From: Leon Romanovsky To: Bjorn Helgaas , Logan Gunthorpe , Chaitanya Kulkarni , Greg Kroah-Hartman , Jens Axboe , Alex Williamson , Leon Romanovsky , Ankit Agrawal , Jason Gunthorpe , Jonathan Corbet , Shuah Khan , "Joerg Roedel (AMD)" , Will Deacon , Robin Murphy , Randy Dunlap , Sumit Semwal , =?utf-8?q?Christian_K=C3=B6nig?= Cc: linux-pci@vger.kernel.org, linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, iommu@lists.linux.dev, Tushar Dave , linux-media@vger.kernel.org, dri-devel@lists.freedesktop.org, linaro-mm-sig@lists.linaro.org, linux-rdma@vger.kernel.org, kvm@vger.kernel.org Subject: [PATCH v5 15/18] PCI: Add KUnit coverage for ACS isolation checks Date: Thu, 10 Sep 2026 14:32:10 +0300 Message-ID: <20260910-fix-p2p-acs-v4-0-v5-15-856087f63c0d@nvidia.com> X-Mailer: git-send-email 2.55.0 In-Reply-To: <20260910-fix-p2p-acs-v4-0-v5-0-856087f63c0d@nvidia.com> References: <20260910-fix-p2p-acs-v4-0-v5-0-856087f63c0d@nvidia.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" X-Mailer: b4 0.15-dev-18f8f Content-Transfer-Encoding: quoted-printable From: Leon Romanovsky Direct Translated P2P does not weaken IOMMU isolation because a Translated Request carries an address supplied by the IOMMU. Config-space read failures, however, leave ACS state unknown and must not report isolation. Exercise both cases with fake config-space operations. Also cover missing and unrequested controls and a missing ACS capability. Signed-off-by: Leon Romanovsky --- drivers/pci/pci.c | 4 +- drivers/pci/pci.h | 1 + drivers/pci/pci_acs_test.c | 138 +++++++++++++++++++++++++++++++++++++++++= ++++ 3 files changed, 142 insertions(+), 1 deletion(-) diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c index f7d94ecf9157..4a9ab3882aac 100644 --- a/drivers/pci/pci.c +++ b/drivers/pci/pci.c @@ -3578,7 +3578,8 @@ void pci_configure_ari(struct pci_dev *dev) } } =20 -static bool pci_acs_flags_enabled(struct pci_dev *pdev, u16 acs_flags) +VISIBLE_IF_KUNIT +bool pci_acs_flags_enabled(struct pci_dev *pdev, u16 acs_flags) { int pos; u16 ctrl; @@ -3598,6 +3599,7 @@ static bool pci_acs_flags_enabled(struct pci_dev *pde= v, u16 acs_flags) return false; return (ctrl & acs_flags) =3D=3D acs_flags; } +EXPORT_SYMBOL_IF_KUNIT(pci_acs_flags_enabled); =20 /** * pci_acs_enabled - test ACS against required flags for a given device diff --git a/drivers/pci/pci.h b/drivers/pci/pci.h index 56f821e40637..5bc703ff0c86 100644 --- a/drivers/pci/pci.h +++ b/drivers/pci/pci.h @@ -1104,6 +1104,7 @@ enum pci_acs_p2pdma_state { }; =20 #if IS_ENABLED(CONFIG_KUNIT) +bool pci_acs_flags_enabled(struct pci_dev *pdev, u16 acs_flags); enum pci_acs_p2pdma_state pci_acs_p2pdma_request(u16 ctrl, unsigned int tlp_flags); enum pci_acs_p2pdma_state pci_acs_p2pdma_completion(u16 ctrl, diff --git a/drivers/pci/pci_acs_test.c b/drivers/pci/pci_acs_test.c index 28eced6dd672..880fc810080a 100644 --- a/drivers/pci/pci_acs_test.c +++ b/drivers/pci/pci_acs_test.c @@ -102,6 +102,140 @@ static void pci_acs_p2pdma_completion_test(struct kun= it *test) c->expect); } =20 +/* Flags an IOMMU asks for; see REQ_ACS_FLAGS in drivers/iommu/iommu.c. */ +#define ACS_REQ_FLAGS (PCI_ACS_SV | PCI_ACS_RR | PCI_ACS_CR | PCI_ACS_UF) +#define ACS_ALL_CAPS (PCI_ACS_SV | PCI_ACS_TB | PCI_ACS_RR | PCI_ACS_CR | \ + PCI_ACS_UF | PCI_ACS_DT) +#define ACS_TEST_CAP 0x100 + +struct acs_ctrl_cfg { + unsigned int devfn; + u16 cap; /* Offset where the ACS capability responds */ + u16 ctrl; + bool fail_read; +}; + +static int acs_ctrl_read(struct pci_bus *bus, unsigned int devfn, + int where, int size, u32 *val) +{ + struct acs_ctrl_cfg *cfg =3D bus->sysdata; + + *val =3D 0; + if (cfg->fail_read) + return PCIBIOS_DEVICE_NOT_FOUND; + + if (devfn =3D=3D cfg->devfn && size =3D=3D 2 && + where =3D=3D cfg->cap + PCI_ACS_CTRL) + *val =3D cfg->ctrl; + return PCIBIOS_SUCCESSFUL; +} + +static int acs_ctrl_write(struct pci_bus *bus, unsigned int devfn, + int where, int size, u32 val) +{ + return PCIBIOS_SUCCESSFUL; +} + +static struct pci_ops acs_ctrl_ops =3D { + .read =3D acs_ctrl_read, + .write =3D acs_ctrl_write, +}; + +struct acs_isolation_case { + const char *desc; + u16 ctrl; + u16 req; + bool expect; +}; + +static const struct acs_isolation_case acs_isolation_cases[] =3D { + { "all_enabled", ACS_REQ_FLAGS, ACS_REQ_FLAGS, true }, + /* Translated Requests remain isolated by their IOMMU translation. */ + { "dt", ACS_REQ_FLAGS | PCI_ACS_DT, ACS_REQ_FLAGS, true }, + { "rr_not_enabled", PCI_ACS_SV | PCI_ACS_CR | PCI_ACS_UF, + ACS_REQ_FLAGS, false }, + { "rr_not_required", PCI_ACS_SV | PCI_ACS_CR | PCI_ACS_UF, + PCI_ACS_SV | PCI_ACS_CR | PCI_ACS_UF, true }, +}; + +static void acs_isolation_desc(const struct acs_isolation_case *c, char *d= esc) +{ + strscpy(desc, c->desc, KUNIT_PARAM_DESC_SIZE); +} + +KUNIT_ARRAY_PARAM(acs_isolation, acs_isolation_cases, acs_isolation_desc); + +static void pci_acs_flags_enabled_test(struct kunit *test) +{ + const struct acs_isolation_case *c =3D test->param_value; + struct acs_ctrl_cfg cfg =3D { + .devfn =3D PCI_DEVFN(0, 0), + .cap =3D ACS_TEST_CAP, + .ctrl =3D c->ctrl, + }; + struct pci_bus *bus =3D kunit_kzalloc(test, sizeof(*bus), GFP_KERNEL); + struct pci_dev *pdev =3D kunit_kzalloc(test, sizeof(*pdev), GFP_KERNEL); + + KUNIT_ASSERT_NOT_NULL(test, bus); + KUNIT_ASSERT_NOT_NULL(test, pdev); + + bus->ops =3D &acs_ctrl_ops; + bus->sysdata =3D &cfg; + + pdev->bus =3D bus; + pdev->devfn =3D cfg.devfn; + pdev->acs_cap =3D ACS_TEST_CAP; + pdev->acs_capabilities =3D ACS_ALL_CAPS; + + KUNIT_EXPECT_EQ(test, pci_acs_flags_enabled(pdev, c->req), c->expect); +} + +static bool acs_isolated(struct kunit *test, struct acs_ctrl_cfg *cfg, + u16 acs_cap, u16 acs_flags) +{ + struct pci_bus *bus =3D kunit_kzalloc(test, sizeof(*bus), GFP_KERNEL); + struct pci_dev *pdev =3D kunit_kzalloc(test, sizeof(*pdev), GFP_KERNEL); + + KUNIT_ASSERT_NOT_NULL(test, bus); + KUNIT_ASSERT_NOT_NULL(test, pdev); + + bus->ops =3D &acs_ctrl_ops; + bus->sysdata =3D cfg; + + pdev->bus =3D bus; + pdev->devfn =3D cfg->devfn; + pdev->acs_cap =3D acs_cap; + pdev->acs_capabilities =3D ACS_ALL_CAPS; + + return pci_acs_flags_enabled(pdev, acs_flags); +} + +static void pci_acs_flags_no_cap_test(struct kunit *test) +{ + struct acs_ctrl_cfg cfg =3D { + .devfn =3D PCI_DEVFN(0, 0), + .cap =3D 0, + .ctrl =3D ACS_REQ_FLAGS, + }; + + KUNIT_EXPECT_FALSE(test, acs_isolated(test, &cfg, 0, ACS_REQ_FLAGS)); +} + +static void pci_acs_flags_read_fails_test(struct kunit *test) +{ + u16 no_rr =3D ACS_REQ_FLAGS & ~PCI_ACS_RR; + struct acs_ctrl_cfg cfg =3D { + .devfn =3D PCI_DEVFN(0, 0), + .cap =3D ACS_TEST_CAP, + .ctrl =3D ACS_REQ_FLAGS, + }; + + KUNIT_EXPECT_TRUE(test, acs_isolated(test, &cfg, ACS_TEST_CAP, no_rr)); + + cfg.fail_read =3D true; + KUNIT_EXPECT_FALSE(test, acs_isolated(test, &cfg, ACS_TEST_CAP, no_rr)); +} + /* * Drive calc_map_type_and_dist() over a fabricated PCIe fabric matching t= he * canonical topology of two devices below one switch: @@ -445,6 +579,10 @@ static struct kunit_case pci_acs_test_cases[] =3D { acs_request_gen_params), KUNIT_CASE_PARAM(pci_acs_p2pdma_completion_test, acs_completion_gen_params), + KUNIT_CASE_PARAM(pci_acs_flags_enabled_test, + acs_isolation_gen_params), + KUNIT_CASE(pci_acs_flags_no_cap_test), + KUNIT_CASE(pci_acs_flags_read_fails_test), KUNIT_CASE(acs_walk_bus_addr_test), KUNIT_CASE(acs_walk_request_redirect_test), KUNIT_CASE(acs_walk_completion_redirect_test), --=20 2.55.0 From nobody Fri Sep 25 16:56:34 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A2417475324; Thu, 10 Sep 2026 11:33:12 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789039995; cv=none; b=pfB/PLM6RY34H1P/TESsKX0Af5TxwUT9MqcxAVdK4Lhq1+CdBdB5TYxRwQZf9TV9Cg2HU7gcrlnsbdUE8cf/NNQu3xLgmsjMbWyqSUM0zL9VFm7/oAwWtpQYEvOpSUV/uXrlHu8IZlPh9pyBxmw4Kt+w+ynm1ISCGfwGqqses/w= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789039995; c=relaxed/simple; bh=OmXiaQz8oAAg8BSkVKc3iM7ZxXZ/pDOr0Gp5QWWZ59o=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=lMCjwHgUb2sXNO6VXEXC9UVTE/Jx/f94rzu6DO8Gyp0quXNXodWeH6O8MUDQ6B3AzrtFFHs6A9sJlsHxS58aFWmRZLDDxMszw8hpils+P2lAsCHrVlE8wTGSlRQYNX+KfooBGQ6GAWD9Qu46xtzoCXhXUsZZOHtuyYmGwq7XBHw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=ZN5EAYhJ; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="ZN5EAYhJ" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 8648B1F000FF; Thu, 10 Sep 2026 11:33:11 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1789039992; bh=xO95Pk5XRLE0oFtxojC3cgHS1ScHTkbYcAUHrZA8bgM=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=ZN5EAYhJjwCjOY9/yo7w3n2G2lH0exsEe4O73m1Jjd6DlV5P2Xj7veAIdkbLElx4W E/1c/rnjyXukT5dKO09N0NpsnjZep/THUcvVQuaIDMgSMU21C8AjimyF4wRLy+kd3T x5yDTyWmj8ziM03a5lBX1Kwvi9w5wHXO3dnGoFIRizOJwySJbYEW5lqw1BCx0oiJHr M8eTKaCeoG6Y2PxhCIcEIu9Xug7+80DCtMoilgqPRB962FJS3UQr/BMOcwVF5GxUEy Tj8rFdUIAYYW/KrvYYvpUaRjX5PCfoEoSeV/oWIY5AGthVy2Jds8a5GPFQtjZbdTnQ XGuByM6VyvNgQ== From: Leon Romanovsky To: Bjorn Helgaas , Logan Gunthorpe , Chaitanya Kulkarni , Greg Kroah-Hartman , Jens Axboe , Alex Williamson , Leon Romanovsky , Ankit Agrawal , Jason Gunthorpe , Jonathan Corbet , Shuah Khan , "Joerg Roedel (AMD)" , Will Deacon , Robin Murphy , Randy Dunlap , Sumit Semwal , =?utf-8?q?Christian_K=C3=B6nig?= Cc: linux-pci@vger.kernel.org, linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, iommu@lists.linux.dev, Tushar Dave , linux-media@vger.kernel.org, dri-devel@lists.freedesktop.org, linaro-mm-sig@lists.linaro.org, linux-rdma@vger.kernel.org, kvm@vger.kernel.org Subject: [PATCH v5 16/18] PCI/P2PDMA: Document TLP-class routing Date: Thu, 10 Sep 2026 14:32:11 +0300 Message-ID: <20260910-fix-p2p-acs-v4-0-v5-16-856087f63c0d@nvidia.com> X-Mailer: git-send-email 2.55.0 In-Reply-To: <20260910-fix-p2p-acs-v4-0-v5-0-856087f63c0d@nvidia.com> References: <20260910-fix-p2p-acs-v4-0-v5-0-856087f63c0d@nvidia.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" X-Mailer: b4 0.15-dev-18f8f Content-Transfer-Encoding: quoted-printable From: Leon Romanovsky The P2PDMA documentation stated that the mapping result is not defined for Relaxed Ordering or ATS-translated Requests. It now is. Replace that paragraph with what the three TLP-sensitive ACS controls do, the table of outcomes per class, and the interface a client uses to name its traffic. Record that a caller claiming Relaxed Ordering Completions is asserting provider behaviour the PCIe specification leaves optional. Signed-off-by: Leon Romanovsky --- Documentation/driver-api/pci/p2pdma.rst | 61 +++++++++++++++++++++++++++++= ---- 1 file changed, 55 insertions(+), 6 deletions(-) diff --git a/Documentation/driver-api/pci/p2pdma.rst b/Documentation/driver= -api/pci/p2pdma.rst index 42b18610bf7d..96a4c1d1b0fb 100644 --- a/Documentation/driver-api/pci/p2pdma.rst +++ b/Documentation/driver-api/pci/p2pdma.rst @@ -28,12 +28,61 @@ through the host bridge when either applicable port red= irects. If an ACS Control register cannot be read, P2P DMA is rejected because the kernel ca= nnot establish a usable route. =20 -This evaluation assumes clients issue strictly ordered Requests carrying an -Untranslated address. Its result is not defined when clients use Relaxed -Ordering or issue ATS-translated Requests because those TLP attributes can -select different routes through the fabric. Unless ACS Translation Blocking -is enabled, a Port with ACS Direct Translated P2P enabled routes a -Translated Request directly to the peer regardless of the redirect control= s. +Three of those controls act on TLP attributes that the client chooses rath= er +than on the topology, so the same path routes differently for different +traffic. ACS Translation Blocking rejects any Request whose Address Type is +not Untranslated, and takes precedence over every other P2P control. ACS +Direct Translated P2P routes a Translated Request to the peer regardless of +Request Redirect and Egress Control. ACS Completion Redirect leaves alone +Completions that carry the Relaxed Ordering attribute. + +A client therefore describes its traffic with ``enum pci_p2pdma_tlp_flags`` +and asks ``pci_p2pdma_map_type_tlp()``. ``pci_p2pdma_map_type()`` answers = for +the default: strictly ordered Requests carrying an Untranslated address. + +The two directions are decided independently. Translation Blocking (TB), +Direct Translated P2P (DT), Request Redirect (RR) and Egress Control (EC) = on +the client-side port decide the Request: + +=3D=3D=3D=3D=3D =3D=3D=3D=3D=3D =3D=3D=3D=3D=3D=3D=3D =3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D +TB DT RR/EC TLP class Request +=3D=3D=3D=3D=3D =3D=3D=3D=3D=3D =3D=3D=3D=3D=3D=3D=3D =3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D +set x x translated blocked +clear set x translated direct +clear clear clear translated direct +clear clear set translated redirected +x x clear untranslated direct +x x set untranslated redirected +=3D=3D=3D=3D=3D =3D=3D=3D=3D=3D =3D=3D=3D=3D=3D=3D=3D =3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D + +Completion Redirect (CR) on the provider-side port decides the Completions: + +=3D=3D=3D=3D=3D =3D=3D=3D=3D=3D=3D=3D=3D=3D =3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D +CR TLP class Completion +=3D=3D=3D=3D=3D =3D=3D=3D=3D=3D=3D=3D=3D=3D =3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D +x relaxed direct +clear strict direct +set strict redirected +=3D=3D=3D=3D=3D =3D=3D=3D=3D=3D=3D=3D=3D=3D =3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D + +A path is bus addressable only where both directions route directly, so wi= th +nothing enabled every class is. A blocked Request is not supported, because +Translation Blocking rejects the Address Type wherever the Request is +addressed. Anything else goes through the host bridge. + +Note that DT only matters where RR or EC would otherwise redirect: it +overrides them for a Translated address rather than granting a direct route +that was not already there. + +Translation Blocking is not a routing control, so it is evaluated on every +client-side port rather than at the divergence alone. A Request it rejects +has no host bridge fallback, because the Address Type is rejected wherever +the Request is addressed. + +The Completer chooses whether a Completion carries Relaxed Ordering, and t= he +PCIe specification does not require it to copy that attribute from the +Request. A caller passing ``PCI_P2PDMA_TLP_RELAXED_CPL`` is asserting that +its provider does. =20 However, if the P2P transaction reaches the host bridge then it might have= to hairpin back out the same root port, be routed inside the CPU SOC to anoth= er --=20 2.55.0 From nobody Fri Sep 25 16:56:34 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 44A714825CD; Thu, 10 Sep 2026 11:33:19 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789040001; cv=none; b=U8WoIYUImaazPXnuSD29Drv5nq/XaNk4ORd3zo1GPPJiqMXLDalPtjPy+63ZWAf1PBtLr4Indr/gtGxEmLwAQqNbkIiE4JMjMo0ENW8YsmnpJPxWEi0GIGa9R7O4JQNGYBDxljWfQttIjK1jLPCgQ/6U16S2iJOktK6ykUdrD6s= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789040001; c=relaxed/simple; bh=HnOE7KxRMQDSoSWzdw0jEvJl+vqEH71USKekBD/2CmA=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=g7kg4U8wbHXes+3nJ7phPogiOiVgTZXORRg3xQ4w4UPnsca6UF2DrBG2Al8IMSwc4aJHo8sBJwGDEmaPA9rSXmw6H4G51cmFs6eTuHFTDvDPavYr2ucbkBiyA8bTx1D2rOcjlO9OvkfGOX+j0SjQDjmuQMm7Hpq5Eq+szr1PRdc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=L2DWUUe3; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="L2DWUUe3" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 7D5DD1F00898; Thu, 10 Sep 2026 11:33:18 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1789039999; bh=tVEndtM1ZMkTnl3oy8jWZNBW5Ka8c12aDzrffOz7F7s=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=L2DWUUe36TRIQVb7AMqWGmMpNntSEztZ0qry9hebj42vUWTDSaF9MS6n34PaQ4fyW DM+TFYDAVsOJaQx3QfWDxaq+BC4XZATCAMO18311no75oSZV+WJJpqEHS3a9s78hu8 cOJF4/J7tlNNUVHYFr3e97shW8aifqWEfL6YBTjQaUUw1pskfaIg2VK0K2vx0O26wT ASy/sRXS/fBTx3orHKl8ZHvUvsLkXBLQ6HlKVtpztJoSiguowdmqebRUdvaY/o4Ass YaRX5Fng3uGUE555P19PJ7EXWQO2I/57eS0MCsO/b7FONX5rtdHtevKIHUYMfK9cQh muqwvD3GleIoA== From: Leon Romanovsky To: Bjorn Helgaas , Logan Gunthorpe , Chaitanya Kulkarni , Greg Kroah-Hartman , Jens Axboe , Alex Williamson , Leon Romanovsky , Ankit Agrawal , Jason Gunthorpe , Jonathan Corbet , Shuah Khan , "Joerg Roedel (AMD)" , Will Deacon , Robin Murphy , Randy Dunlap , Sumit Semwal , =?utf-8?q?Christian_K=C3=B6nig?= Cc: linux-pci@vger.kernel.org, linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, iommu@lists.linux.dev, Tushar Dave , linux-media@vger.kernel.org, dri-devel@lists.freedesktop.org, linaro-mm-sig@lists.linaro.org, linux-rdma@vger.kernel.org, kvm@vger.kernel.org Subject: [PATCH v5 17/18] dma-buf: Let importers ask how peer-to-peer traffic is routed Date: Thu, 10 Sep 2026 14:32:12 +0300 Message-ID: <20260910-fix-p2p-acs-v4-0-v5-17-856087f63c0d@nvidia.com> X-Mailer: git-send-email 2.55.0 In-Reply-To: <20260910-fix-p2p-acs-v4-0-v5-0-856087f63c0d@nvidia.com> References: <20260910-fix-p2p-acs-v4-0-v5-0-856087f63c0d@nvidia.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" X-Mailer: b4 0.15-dev-18f8f Content-Transfer-Encoding: quoted-printable From: Leon Romanovsky Exporters keep the &struct p2pdma_provider in their own private data and hand it to dma_buf_phys_vec_to_sgt() on every map. An importer cannot reach it, so it has no way to learn how its own peer-to-peer traffic would be routed before it programs its hardware. Publish the provider through &struct dma_buf_export_info instead, and add dma_buf_p2pdma_map_type() for importers to query it by TLP class. It is fixed at export time, so dma_buf_phys_vec_to_sgt() no longer needs it as an argument and the two exporters no longer need a copy of it. Signed-off-by: Leon Romanovsky --- drivers/dma-buf/dma-buf-mapping.c | 41 +++++++++++++++++++= ---- drivers/dma-buf/dma-buf.c | 1 + drivers/infiniband/core/uverbs.h | 1 - drivers/infiniband/core/uverbs_std_types_dmabuf.c | 7 ++-- drivers/vfio/pci/vfio_pci_dmabuf.c | 8 ++--- include/linux/dma-buf-mapping.h | 4 ++- include/linux/dma-buf.h | 5 +++ 7 files changed, 49 insertions(+), 18 deletions(-) diff --git a/drivers/dma-buf/dma-buf-mapping.c b/drivers/dma-buf/dma-buf-ma= pping.c index 794acff2546a..8b431000e906 100644 --- a/drivers/dma-buf/dma-buf-mapping.c +++ b/drivers/dma-buf/dma-buf-mapping.c @@ -6,6 +6,32 @@ #include #include =20 +/** + * dma_buf_p2pdma_map_type - How peer-to-peer traffic to a buffer is routed + * @attach: attachment of the importer that will issue the traffic + * @tlp_flags: &enum pci_p2pdma_tlp_flags describing the TLPs it will issue + * + * Reports how the PCIe fabric routes @tlp_flags traffic between the buffer + * behind @attach and the importer attached to it, so that an importer can + * choose the TLP attributes that earn it a direct route before it programs + * its hardware. + * + * Return: the mapping type for @tlp_flags traffic, or PCI_P2PDMA_MAP_NONE + * when the exporter named no &struct p2pdma_provider and nothing is known + * about the route. + */ +enum pci_p2pdma_map_type +dma_buf_p2pdma_map_type(struct dma_buf_attachment *attach, + unsigned int tlp_flags) +{ + if (!attach->dmabuf->provider) + return PCI_P2PDMA_MAP_NONE; + + return pci_p2pdma_map_type_tlp(attach->dmabuf->provider, attach->dev, + tlp_flags); +} +EXPORT_SYMBOL_NS_GPL(dma_buf_p2pdma_map_type, "DMA_BUF"); + static struct scatterlist *fill_sg_entry(struct scatterlist *sgl, size_t l= ength, dma_addr_t addr) { @@ -71,7 +97,6 @@ struct dma_buf_dma { * from arrays of physical vectors. This funciton is intended for MMIO mem= ory * only. * @attach: [in] attachment whose scatterlist is to be returned - * @provider: [in] p2pdma provider * @phys_vec: [in] array of physical vectors * @nr_ranges: [in] number of entries in phys_vec array * @size: [in] total size of phys_vec @@ -85,16 +110,17 @@ struct dma_buf_dma { * * A mapping must be unmapped by using dma_buf_free_sgt(). * - * NOTE: This function is intended for exporters. If direct traffic routin= g is - * mandatory exporter should call routing pci_p2pdma_map_type() before cal= ling - * this function. + * NOTE: This function is intended for exporters, and works on MMIO memory + * only, so &dma_buf.provider must have been set at export time. If direct + * traffic routing is mandatory the exporter should call + * pci_p2pdma_map_type() before calling this function. */ struct sg_table *dma_buf_phys_vec_to_sgt(struct dma_buf_attachment *attach, - struct p2pdma_provider *provider, struct phys_vec *phys_vec, size_t nr_ranges, size_t size, enum dma_data_direction dir) { + struct p2pdma_provider *provider; unsigned int nents, mapped_len =3D 0; struct dma_buf_dma *dma; struct scatterlist *sgl; @@ -104,10 +130,11 @@ struct sg_table *dma_buf_phys_vec_to_sgt(struct dma_b= uf_attachment *attach, =20 dma_resv_assert_held(attach->dmabuf->resv); =20 - if (WARN_ON(!attach || !attach->dmabuf || !provider)) - /* This function is supposed to work on MMIO memory only */ + if (WARN_ON(!attach || !attach->dmabuf || !attach->dmabuf->provider)) return ERR_PTR(-EINVAL); =20 + provider =3D attach->dmabuf->provider; + dma =3D kzalloc_obj(*dma); if (!dma) return ERR_PTR(-ENOMEM); diff --git a/drivers/dma-buf/dma-buf.c b/drivers/dma-buf/dma-buf.c index d504c636dc29..c2b9944e9659 100644 --- a/drivers/dma-buf/dma-buf.c +++ b/drivers/dma-buf/dma-buf.c @@ -747,6 +747,7 @@ struct dma_buf *dma_buf_export(const struct dma_buf_exp= ort_info *exp_info) dmabuf->size =3D exp_info->size; dmabuf->exp_name =3D exp_info->exp_name; dmabuf->owner =3D exp_info->owner; + dmabuf->provider =3D exp_info->provider; spin_lock_init(&dmabuf->name_lock); init_waitqueue_head(&dmabuf->poll); dmabuf->cb_in.poll =3D dmabuf->cb_out.poll =3D &dmabuf->poll; diff --git a/drivers/infiniband/core/uverbs.h b/drivers/infiniband/core/uve= rbs.h index c64dd6b94e10..fbdac18b69ca 100644 --- a/drivers/infiniband/core/uverbs.h +++ b/drivers/infiniband/core/uverbs.h @@ -139,7 +139,6 @@ struct ib_uverbs_dmabuf_file { struct list_head dmabufs_elm; struct rdma_user_mmap_entry *mmap_entry; struct phys_vec phys_vec; - struct p2pdma_provider *provider; struct kref kref; struct completion comp; u8 revoked :1; diff --git a/drivers/infiniband/core/uverbs_std_types_dmabuf.c b/drivers/in= finiband/core/uverbs_std_types_dmabuf.c index 2411ebee69e2..94d3719a34da 100644 --- a/drivers/infiniband/core/uverbs_std_types_dmabuf.c +++ b/drivers/infiniband/core/uverbs_std_types_dmabuf.c @@ -33,9 +33,8 @@ uverbs_dmabuf_map(struct dma_buf_attachment *attachment, if (priv->revoked) return ERR_PTR(-ENODEV); =20 - ret =3D dma_buf_phys_vec_to_sgt(attachment, priv->provider, - &priv->phys_vec, 1, priv->phys_vec.len, - dir); + ret =3D dma_buf_phys_vec_to_sgt(attachment, &priv->phys_vec, 1, + priv->phys_vec.len, dir); if (IS_ERR(ret)) return ret; =20 @@ -108,7 +107,7 @@ static int UVERBS_HANDLER(UVERBS_METHOD_DMABUF_ALLOC)( return -EINVAL; =20 ret =3D ib_dev->ops.mmap_get_pfns(mmap_entry, &uverbs_dmabuf->phys_vec, - &uverbs_dmabuf->provider); + &exp_info.provider); if (ret) goto err; =20 diff --git a/drivers/vfio/pci/vfio_pci_dmabuf.c b/drivers/vfio/pci/vfio_pci= _dmabuf.c index c16f460c01d6..10f47992e7f5 100644 --- a/drivers/vfio/pci/vfio_pci_dmabuf.c +++ b/drivers/vfio/pci/vfio_pci_dmabuf.c @@ -15,7 +15,6 @@ struct vfio_pci_dma_buf { struct list_head dmabufs_elm; size_t size; struct phys_vec *phys_vec; - struct p2pdma_provider *provider; u32 nr_ranges; struct kref kref; struct completion comp; @@ -59,9 +58,8 @@ vfio_pci_dma_buf_map(struct dma_buf_attachment *attachmen= t, if (priv->revoked) return ERR_PTR(-ENODEV); =20 - ret =3D dma_buf_phys_vec_to_sgt(attachment, priv->provider, - priv->phys_vec, priv->nr_ranges, - priv->size, dir); + ret =3D dma_buf_phys_vec_to_sgt(attachment, priv->phys_vec, + priv->nr_ranges, priv->size, dir); if (IS_ERR(ret)) return ret; =20 @@ -274,7 +272,7 @@ int vfio_pci_core_feature_dma_buf(struct vfio_pci_core_= device *vdev, u32 flags, priv->vdev =3D vdev; priv->nr_ranges =3D get_dma_buf.nr_ranges; priv->size =3D length; - ret =3D vdev->pci_ops->get_dmabuf_phys(vdev, &priv->provider, + ret =3D vdev->pci_ops->get_dmabuf_phys(vdev, &exp_info.provider, get_dma_buf.region_index, priv->phys_vec, dma_ranges, priv->nr_ranges); diff --git a/include/linux/dma-buf-mapping.h b/include/linux/dma-buf-mappin= g.h index 09bde3f748e4..d63ac4d52aa5 100644 --- a/include/linux/dma-buf-mapping.h +++ b/include/linux/dma-buf-mapping.h @@ -7,8 +7,10 @@ #define __DMA_BUF_MAPPING_H__ #include =20 +enum pci_p2pdma_map_type +dma_buf_p2pdma_map_type(struct dma_buf_attachment *attach, + unsigned int tlp_flags); struct sg_table *dma_buf_phys_vec_to_sgt(struct dma_buf_attachment *attach, - struct p2pdma_provider *provider, struct phys_vec *phys_vec, size_t nr_ranges, size_t size, enum dma_data_direction dir); diff --git a/include/linux/dma-buf.h b/include/linux/dma-buf.h index d1203da56fc5..81c136fcee6e 100644 --- a/include/linux/dma-buf.h +++ b/include/linux/dma-buf.h @@ -368,6 +368,9 @@ struct dma_buf { /** @priv: exporter specific private data for this buffer object. */ void *priv; =20 + /** @provider: The peer-to-peer provider backing this buffer. */ + struct p2pdma_provider *provider; + /** * @resv: * @@ -501,6 +504,7 @@ struct dma_buf_attachment { * @flags: mode flags for the file * @resv: reservation-object, NULL to allocate default one * @priv: Attach private data of allocator to this buffer + * @provider: Peer-to-peer provider backing the buffer * * This structure holds the information required to export the buffer. Used * with dma_buf_export() only. @@ -512,6 +516,7 @@ struct dma_buf_export_info { size_t size; int flags; struct dma_resv *resv; + struct p2pdma_provider *provider; void *priv; }; =20 --=20 2.55.0 From nobody Fri Sep 25 16:56:34 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0E953484238; Thu, 10 Sep 2026 11:33:23 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789040006; cv=none; b=N4nLdz3cvJ1p9Wy/s2kbsOxfFvyVd1MxUDPGqsfiYYd9mXMJdhjD0kOJgg7fELRnnQ6+tO40cx/CBXydYk8JJX9eN+VApE8tOzRRo5GHeBZMAa2wUafwtiXvgHrpwQHM31MDTwecyQnT3eMPp96/lcb1OzWmvQJsK7IjgeYyiXo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789040006; c=relaxed/simple; bh=hvVwt6ohh/ZS5jfoRHiM6qKZdBfoz9Vd2295Ug5sbr8=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=pS45pnZauaRlbg63kpUX9C4UwPUHz5rFBtY21XXDKdvsVKTPDr50VwRTprki8cOMYgZKMGA7aQjBO3gBRK010+iKs0sX7gDDwZ9wGkT8MCQU+r2i8ellAarU02KyHgC6U0fnBC1SLaSe3vQ9BASy4qHYumpx5oolJhYoTqvv4yM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=VJQVGUTL; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="VJQVGUTL" Received: by smtp.kernel.org (Postfix) with ESMTPSA id A69031F000FF; Thu, 10 Sep 2026 11:33:21 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1789040002; bh=YtIACyToojkPPw7wMDUQ/Ckhd+O2kfDRIKTR1tO2oXk=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=VJQVGUTLv8gM7bjQGnsZH2u0qxwm3dviTA391P/zKAGLCSCpght/oHVmkHaPsQoyG ZRgX9lWQUZXGOSCkFdEocP+IO5b88+kg7x/1mBmDwqMol3y9jajpqGzofXy7V6yt+Y OA9F4YNfuNGZqPIM7bK4r81u1Hw6sUMA/ZABwup4ghE4BLmJHT7nq8unPWL1TAuofO ja9w9yv7/7z0GOqAJi4MmGzhF3qhczbIuo5n5ETv7Qvi4FHkdKuWXx1+0/bXnCiNrT 4Vnp7cfAt20DE1FyPVfN3+muefwGix6mQCiO5ZJrCmA7ffCTn+gnm9cg4ZW5sDF816 7Zw+TC6+jPI9A== From: Leon Romanovsky To: Bjorn Helgaas , Logan Gunthorpe , Chaitanya Kulkarni , Greg Kroah-Hartman , Jens Axboe , Alex Williamson , Leon Romanovsky , Ankit Agrawal , Jason Gunthorpe , Jonathan Corbet , Shuah Khan , "Joerg Roedel (AMD)" , Will Deacon , Robin Murphy , Randy Dunlap , Sumit Semwal , =?utf-8?q?Christian_K=C3=B6nig?= Cc: linux-pci@vger.kernel.org, linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, iommu@lists.linux.dev, Tushar Dave , linux-media@vger.kernel.org, dri-devel@lists.freedesktop.org, linaro-mm-sig@lists.linaro.org, linux-rdma@vger.kernel.org, kvm@vger.kernel.org Subject: [PATCH v5 18/18] RDMA/mlx5: Ask P2PDMA whether ATS takes a direct peer-to-peer route Date: Thu, 10 Sep 2026 14:32:13 +0300 Message-ID: <20260910-fix-p2p-acs-v4-0-v5-18-856087f63c0d@nvidia.com> X-Mailer: git-send-email 2.55.0 In-Reply-To: <20260910-fix-p2p-acs-v4-0-v5-0-856087f63c0d@nvidia.com> References: <20260910-fix-p2p-acs-v4-0-v5-0-856087f63c0d@nvidia.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" X-Mailer: b4 0.15-dev-18f8f Content-Transfer-Encoding: quoted-printable From: Leon Romanovsky mlx5_umem_needs_ats() enables ATS for any dma-buf whose caller asked for Relaxed Ordering, on the assumption that a switch in the path has CR, RR and DT all set. It also enables it for a buffer already mapped with the peer's bus addresses, which are not translatable at all. P2PDMA has read the ACS controls, so ask it through dma_buf_p2pdma_map_type(): enable ATS only where the path is not routed directly as it stands, but would be for a Translated Request whose Completions carry Relaxed Ordering. Exporters that name no provider keep the old assumption, since their ACS settings remain hidden. Signed-off-by: Leon Romanovsky --- drivers/infiniband/hw/mlx5/mlx5_ib.h | 36 ++------------------------------ drivers/infiniband/hw/mlx5/mr.c | 40 ++++++++++++++++++++++++++++++++= ++++ 2 files changed, 42 insertions(+), 34 deletions(-) diff --git a/drivers/infiniband/hw/mlx5/mlx5_ib.h b/drivers/infiniband/hw/m= lx5/mlx5_ib.h index e9ddf2e97a76..ab32742b2180 100644 --- a/drivers/infiniband/hw/mlx5/mlx5_ib.h +++ b/drivers/infiniband/hw/mlx5/mlx5_ib.h @@ -1646,40 +1646,8 @@ static inline bool rt_supported(int ts_cap) ts_cap =3D=3D MLX5_TIMESTAMP_FORMAT_CAP_FREE_RUNNING_AND_REAL_TIME; } =20 -/* - * PCI Peer to Peer is a trainwreck. If no switch is present then things - * sometimes work, depending on the pci_distance_p2p logic for excluding b= roken - * root complexes. However if a switch is present in the path, then things= get - * really ugly depending on how the switch is setup. This table assumes th= at the - * root complex is strict and is validating that all req/reps are matches - * perfectly - so any scenario where it sees only half the transaction is a - * failure. - * - * CR/RR/DT ATS RO P2P - * 00X X X OK - * 010 X X fails (request is routed to root but root never sees c= omp) - * 011 0 X fails (request is routed to root but root never sees c= omp) - * 011 1 X OK - * 10X X 1 OK - * 101 X 0 fails (completion is routed to root but root didn't se= e req) - * 110 X 0 SLOW - * 111 0 0 SLOW - * 111 1 0 fails (completion is routed to root but root didn't se= e req) - * 111 1 1 OK - * - * Unfortunately we cannot reliably know if a switch is present or what the - * CR/RR/DT ACS settings are, as in a VM that is all hidden. Assume that - * CR/RR/DT is 111 if the ATS cap is enabled and follow the last three row= s. - * - * For now assume if the umem is a dma_buf then it is P2P. - */ -static inline bool mlx5_umem_needs_ats(struct mlx5_ib_dev *dev, - struct ib_umem *umem, int access_flags) -{ - if (!MLX5_CAP_GEN(dev->mdev, ats) || !umem->is_dmabuf) - return false; - return access_flags & IB_ACCESS_RELAXED_ORDERING; -} +bool mlx5_umem_needs_ats(struct mlx5_ib_dev *dev, struct ib_umem *umem, + int access_flags); =20 int set_roce_addr(struct mlx5_ib_dev *dev, u32 port_num, unsigned int index, const union ib_gid *gid, diff --git a/drivers/infiniband/hw/mlx5/mr.c b/drivers/infiniband/hw/mlx5/m= r.c index 00e13028762a..286f372e5b0c 100644 --- a/drivers/infiniband/hw/mlx5/mr.c +++ b/drivers/infiniband/hw/mlx5/mr.c @@ -38,6 +38,7 @@ #include #include #include +#include #include #include #include @@ -47,6 +48,45 @@ #include "data_direct.h" #include "dmah.h" =20 +MODULE_IMPORT_NS("DMA_BUF"); + +bool mlx5_umem_needs_ats(struct mlx5_ib_dev *dev, struct ib_umem *umem, + int access_flags) +{ + struct dma_buf_attachment *attach; + + if (!MLX5_CAP_GEN(dev->mdev, ats) || !umem->is_dmabuf) + return false; + + /* + * The Completer decides whether its Completions carry Relaxed + * Ordering, and only a Request that asked for it can expect them to. + */ + if (!(access_flags & IB_ACCESS_RELAXED_ORDERING)) + return false; + + attach =3D to_ib_umem_dmabuf(umem)->attach; + switch (dma_buf_p2pdma_map_type(attach, 0)) { + case PCI_P2PDMA_MAP_NONE: + /* Nothing is known about the route, so fall back to the bet. */ + return true; + case PCI_P2PDMA_MAP_BUS_ADDR: + /* + * The path is routed directly already and is programmed with + * the peer's bus addresses. Those are not translatable, so + * ATS would be wrong as well as pointless. + */ + return false; + default: + break; + } + + return dma_buf_p2pdma_map_type(attach, + PCI_P2PDMA_TLP_TRANSLATED | + PCI_P2PDMA_TLP_RELAXED_CPL) =3D=3D + PCI_P2PDMA_MAP_BUS_ADDR; +} + static int mkey_max_umr_order(struct mlx5_ib_dev *dev) { if (MLX5_CAP_GEN(dev->mdev, umr_extended_translation_offset)) --=20 2.55.0