From nobody Sat Sep 26 20:30:26 2026 Received: from OS0P286CU010.outbound.protection.outlook.com (mail-japanwestazon11021129.outbound.protection.outlook.com [40.107.74.129]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3F8D838D41A; Sun, 30 Aug 2026 15:20:14 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.74.129 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788103216; cv=fail; b=C4M9TmMIdBPPWUkCrRjT6hxbfG0596pqCedmqLp5M25+20LaXpUhdyNFC4SRnxqYVUgrzz2x8sEFHjfAlyWNZPekuLEkU1llUO2oU2Wlbu6rMGGvEbMd7oThhXsJslQokbc0ICt39UcHnYU7ON9vMbq9FQ+54ouE3vI/yuC22Ic= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788103216; c=relaxed/simple; bh=lkRAWyjdPw2iqLJlUEbDeUYLnC9HgHV/l/FTXyJ9vVM=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: Content-Type:MIME-Version; b=bDRy1xlUj9yGUAQ2hAhRncyFRw84t1z1a3U1oDjw/jHtN9U8k5JYCkLDT/umcXinLoDTYU4K1rgU4CesuAEj0R6fqRMto8DisKlDdSD7qrCTzu/6licnVlVyFlu6VcOpYTXsudD9bD/G6DvnHcZjOnXR0OqOZK/8tBtu9XcZITI= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=valinux.co.jp; spf=pass smtp.mailfrom=valinux.co.jp; dkim=pass (1024-bit key) header.d=valinux.co.jp header.i=@valinux.co.jp header.b=aRoVSPCD; arc=fail smtp.client-ip=40.107.74.129 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=valinux.co.jp Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=valinux.co.jp Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=valinux.co.jp header.i=@valinux.co.jp header.b="aRoVSPCD" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=KtgXotv68eBXx9rTBlaeV7/RSLntb+WWfnHnOsquIlhxV6AN1YjxgW59EmJZck0YnObRZMgTU5sdlin8YMd94xQmT4DZN+cdnyGo4OjKoouu8ArdpQpnyOMqfQvNlkI+PUUQriK7RwpvQcaObAi54aSB5wTXaeQ2nlLgHXKhQIhsQXWwg8e6IW5NdP7009cwmffS1JMDfSdgqvwvHgJ522690ueOhSUC9BX/WQuOl0ZbNJUoLmjDV5zSdVPRSdRoTXvNe5E5QGxjltkeo7TOBsPNYPAAJ2OfmIyjfQh0FwRxadKZgmxfXZxDAilIWkQetF1HnEJ8Rxq7vkhJepzUGw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=RtjacCaigq2Si6y5XiwB8SZ6VkEcuottmHl2Lg8WU18=; b=P9KrTfMoat5iFiA1QUVZ4SAsuomhgcHonc2avZZYJwzULrUbeqYt6nU/kXqfLggiPg81KcrjNSWECclBdEnplzfQ1uE5YhnYDWT7rGrASsquMrXZhO0yUhxowH2GB38CPquiOMbcWxckSGrFlcD+D4KojGFbiQ17JK+8hiw2Xbqe8R8URzd9Sdq7tyJyQ/2G5bh0+jtfvVGE+1wuPJjVV0UCRY2M7o4odqQCoR9VnWwFiT380bbhGGnKOliCphzyseK2ROkHihpTktWTB1ZBFZTzPmksTlmZzzR9kvpObPrGHjQO8afXfAXzMEjYpt7AiN2S1j0/4cnZNoZXBWXtNw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=valinux.co.jp; dmarc=pass action=none header.from=valinux.co.jp; dkim=pass header.d=valinux.co.jp; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=valinux.co.jp; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=RtjacCaigq2Si6y5XiwB8SZ6VkEcuottmHl2Lg8WU18=; b=aRoVSPCD5y1r1oUAVxzCDTFLIg4yD80P11J+AYQkVNDCGfzyPm9aOGul6X+zjaBDUri740S1aBu/bMobnpfL14+p079ONAZNt0B4HqDdPeu0ByVTgCnHePiL9WGJZ7AM6zK565ElH19e/Ltr8w8bEefhJ88TR7iV/GDbqtVvAhM= Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=valinux.co.jp; Received: from TY7P286MB7722.JPNP286.PROD.OUTLOOK.COM (2603:1096:405:38f::10) by OS3P286MB3211.JPNP286.PROD.OUTLOOK.COM (2603:1096:604:214::9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.360.13; Sun, 30 Aug 2026 15:20:11 +0000 Received: from TY7P286MB7722.JPNP286.PROD.OUTLOOK.COM ([fe80::2305:327c:28ec:9b32]) by TY7P286MB7722.JPNP286.PROD.OUTLOOK.COM ([fe80::2305:327c:28ec:9b32%4]) with mapi id 15.21.0360.008; Sun, 30 Aug 2026 15:20:11 +0000 From: Koichiro Den To: Manivannan Sadhasivam , =?UTF-8?q?Krzysztof=20Wilczy=C5=84ski?= , Kishon Vijay Abraham I , Frank Li , Bjorn Helgaas , Jingoo Han , Niklas Cassel Cc: Lorenzo Pieralisi , Rob Herring , Aksh Garg , Christoph Hellwig , Sagi Grimberg , Chaitanya Kulkarni , Jon Mason , Dave Jiang , Allen Hubbe , Heiko Stuebner , Shawn Lin , Manikanta Maddireddy , Shin'ichiro Kawasaki , linux-pci@vger.kernel.org, linux-nvme@lists.infradead.org, ntb@lists.linux.dev, linux-rockchip@lists.infradead.org, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org Subject: [PATCH v2 1/3] PCI: endpoint: Support hardware-owned MSI-X table and PBA Date: Mon, 31 Aug 2026 00:19:46 +0900 Message-ID: <20260830151948.3547577-2-den@valinux.co.jp> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20260830151948.3547577-1-den@valinux.co.jp> References: <20260830151948.3547577-1-den@valinux.co.jp> Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: TYWPR01CA0026.jpnprd01.prod.outlook.com (2603:1096:400:aa::13) To TY7P286MB7722.JPNP286.PROD.OUTLOOK.COM (2603:1096:405:38f::10) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: TY7P286MB7722:EE_|OS3P286MB3211:EE_ X-MS-Office365-Filtering-Correlation-Id: 48833f31-2405-40ce-b288-08df06aa2eea X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|376014|23010399003|7416014|1800799024|10070799003|366016|6133799003|10067099003|5023799004|56012099006|18002099003|22082099003; X-Microsoft-Antispam-Message-Info: 0jyzdUUynSh7GskkS5jZVuveViS9zFXfmejUdtTojm9KtdyMQfjZJWWZFhsSY1NhcAYaRiIyXwtxyTaSH2pse/uip4z++IriY9CyKqMCfBPIR9ECLCK+KjyQ41rgKa5T3fZpdPXcgjtSSmPhSsbiXAuVKlWeGRToX+Y8xO17BsaRRsSJwL+Iryhx6MHN+99YOwIOPkQTS7DvVnOFD7QVn7AxrRAW3mJUnwCVlHem41Zi0SGPsTr66r7cg9+pSAwIeDrX+WCeMPS11Qj8ybmFeZqbSki0yMZqGJ8lgtABLp6BCsQ8UtAGzH8PAa9uqm2Y7XtpuVyZ0RJACFNPJdgmrU/9qwf3ly9siusEpPw8aJPaKU7wns321zesxDrzWlXKsb/kCsbRmm+2+eHzbSGRY9rS+L1rsSh2YCrxpXQL3bQv/Gy5rXkykXMMVyqn5qXUCcWRJfoLq5igaN2Y/J69A36kRPSqbp/hp9CwC+qWANIEkN/+ksd/1Z21K+vQoPoTSUddjQR2zc+3Lp1mfpJ6jiIvzYRMJCnFCssaFABuzX0UQghyRkTv7PfSA7frHs5dYHRt+pxu1GzieusqHkNBfIzzNHO6OX19bN5pTAYCuclz25gNqJbMzPjjSdbjKMfqi4oXI3jsO/Y9rO6f8D1yi9AN5EH4GpKNsH0FKOlr6BA= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:TY7P286MB7722.JPNP286.PROD.OUTLOOK.COM;PTR:;CAT:NONE;SFS:(13230040)(376014)(23010399003)(7416014)(1800799024)(10070799003)(366016)(6133799003)(10067099003)(5023799004)(56012099006)(18002099003)(22082099003);DIR:OUT;SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?FS1XOaDBuROkbrpeVgvdCUBNB6puXP+uc7TueYCsMuvfKIp2PggJqE2Nvhi6?= =?us-ascii?Q?sfn7FwLjuXL2ods67T4Ops7VOAT0u0kNlEw6BfLsv8Z/4g20xjJL59/EQos6?= =?us-ascii?Q?2JkgJX0kpay40euYARY5g+D6Yqg+JRwSjXogAi626nY7c7bQhCKN1K9Tf0fZ?= =?us-ascii?Q?EPvCaPNEgbuk229D0b7fu578LpLFYEQfGaiAnepYbj8KJck66ogxq8Y24rh8?= =?us-ascii?Q?xUSo9YmZzAhN+Do9TQucEGVeFdWZ8dzGWDEYnsT4DTHtUm8HEZVBnvSlpu8i?= =?us-ascii?Q?SxXJ63iBKLWKZw+LnhhkBd29NWSFG9anWzLlss1fNytcUorhulBCGfoOjtcM?= =?us-ascii?Q?1utfQ4g/X3DbdYKerd/3qULjor4hbXlYywb0rQYHN/UL/47GlupsaCLdtAPC?= =?us-ascii?Q?ZQCNXZXvfrIVafnZHYxiA19pe/J7/w+qSpuzu+HPzKqiEB+mZMLRSW+DomF9?= =?us-ascii?Q?5g2fAXzl/GxzRJYwYRlNV3thNZSEyrdy/A9G2iWDsI0LuwFGjJ99ROHRVju6?= =?us-ascii?Q?2bB5X6OuRasYCjxaRZ3Nh5BdPzI8BMkUtVNArCS5tG0BPkeItGpNOypu36Ty?= =?us-ascii?Q?5YeMONqxfXDDFPsKPtLb2zZVn7lB0dulRY3DMI2niFJjWt4gtQpP43TNb4yb?= =?us-ascii?Q?WALcCOcP2s33Bi/Pjz43dNbl+xJHgPZjOd7RQRuIEvC++xpzrfjIHjrguONc?= =?us-ascii?Q?+prAYE+V6idwS51hAp9buzhfVeFyVegdfcBCwx6s8P2DB9qbAjUMzE8gukjz?= =?us-ascii?Q?CmGkx0tEUgFaKeozk23moOC3Gibs34o6M1P+VGm8NNU0cnrD8ohLAZqCGUwT?= =?us-ascii?Q?ttmamNGjgHg8DV0o6+X8juYjFUwWdzj5PDeRD6HT8HOrlx402cWSbG+5TRla?= =?us-ascii?Q?7rlnIpHq1W4yvwdsS6Idw5Ji0Uuj38qKwCz0ihS598/s088sEZ1uIPT0m/62?= =?us-ascii?Q?0qQMZcW97o8loBuI1JVVDFb/iOT4izu9Y5x9Qk/TJFU2CM0iJWR+ZXhEQcT+?= =?us-ascii?Q?RsZOvFv9LyP7K0SwqSI9+MMUE2pEUwtp38toUunWyr1nk15Oc2417bleednk?= =?us-ascii?Q?YCZeSf/9zUqA/4Lvay83C6tkk5wvEW8JFpD81pqMdibNy37J0+b/R+e+p08/?= =?us-ascii?Q?30Gjs/QPGmTFxHP9jH5NjPp2XveB5O6j2xTjOJCVzNmiiO74GEGq7Cww2q+E?= =?us-ascii?Q?NWWWr6+/dvxCWlB3HIOnbmCcRTq8Uarv3ARmoMD9kv43hrfb09JwLINj+nW8?= =?us-ascii?Q?w1ljXvDF81Yx5+L8AoxVVdAU5oqkWvwoMLew2AEqoZ8FTby9kwmqlj3WC+7U?= =?us-ascii?Q?X1b9dV6/23HArenAU5t77tz3WkXvNoE+i4VyHvHxlTmQBzO68eio0jfcoBcH?= =?us-ascii?Q?1Z5I7tTmaAGe2HE9Lw6ZQYsa1XI5N8U6Yz6H/COLoZkpEpmgt4ENqePZVVX4?= =?us-ascii?Q?zEEzDSYefsN8eRQ/a1kLnh2eT4mCDowI8hS1pspfsD6mGrL8o8LT/6Mt8eGc?= =?us-ascii?Q?syC+cLcOjBjrrS71te95O6hGEjVlaZeAqm4vyuZ0AQrf5C5UY//9U37CPoEJ?= =?us-ascii?Q?RV/q/8XoLifa0Eg+d4UtdDNeGL4FoxPzR1WlDJekhSLmLL0m/tIgvE1McDje?= =?us-ascii?Q?JoWImVyl+LQ2u5oqG8nNeC/YXCK5+CJuNbY06sthFruyS7AD02ylZMCS5uJx?= =?us-ascii?Q?uirLSyiMlwqH6J89Dh1HByTQmDjKHyCoL8A9dbWr/W9QF58dKvHX3N7bYr4m?= =?us-ascii?Q?qY5R3oXg1C8vDvsm+XQXe8ZXMQLNSk/KOCUPBY6hTqhCuk31bmF8?= X-OriginatorOrg: valinux.co.jp X-MS-Exchange-CrossTenant-Network-Message-Id: 48833f31-2405-40ce-b288-08df06aa2eea X-MS-Exchange-CrossTenant-AuthSource: TY7P286MB7722.JPNP286.PROD.OUTLOOK.COM X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 Aug 2026 15:20:11.1943 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 7a57bee8-f73d-4c5f-a4f7-d72c91c8c111 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 0iV+CsSQzTFXuQDZaSmOlE8ZJxicd7fA0PbjR/y2YXBtd9wiO35jNVDeOCff0CVs8gEGYdz1m9YlP+qWDu91vw== X-MS-Exchange-Transport-CrossTenantHeadersStamped: OS3P286MB3211 Content-Type: text/plain; charset="utf-8" Some endpoint controllers expose the MSI-X table and Pending Bit Array (PBA) in fixed, hardware-owned BAR regions. The EPC set_msix() callback currently receives only the table BAR and offset and assumes that the PBA immediately follows the table in the same BAR. It cannot describe a fixed layout whose table and PBA have independent locations. Add struct pci_epc_msix_layout and pass the full layout to pci_epc_set_msix(). Leave layout selection to EPF implementations. For instance, pci-epf-ntb reads the MSI-X Table from its own BAR. Using a fixed layout there would require a way to read hardware-owned Table entries. Add pci_epc_get_hw_msix_layout() for EPFs that want the fixed layout. Update the existing callers and validate the supplied layout. Signed-off-by: Koichiro Den --- drivers/nvme/target/pci-epf.c | 16 ++- .../pci/controller/cadence/pcie-cadence-ep.c | 9 +- .../pci/controller/dwc/pcie-designware-ep.c | 7 +- drivers/pci/endpoint/functions/pci-epf-ntb.c | 30 +++--- drivers/pci/endpoint/functions/pci-epf-test.c | 17 +-- drivers/pci/endpoint/pci-epc-core.c | 102 +++++++++++++++++- include/linux/pci-epc.h | 25 ++++- 7 files changed, 168 insertions(+), 38 deletions(-) diff --git a/drivers/nvme/target/pci-epf.c b/drivers/nvme/target/pci-epf.c index 4e9db96ebfec..37182f6d29b1 100644 --- a/drivers/nvme/target/pci-epf.c +++ b/drivers/nvme/target/pci-epf.c @@ -201,7 +201,7 @@ struct nvmet_pci_epf { const struct pci_epc_features *epc_features; =20 void *reg_bar; - size_t msix_table_offset; + struct pci_epc_msix_layout msix_layout; =20 unsigned int irq_type; unsigned int nr_vectors; @@ -2187,8 +2187,14 @@ static int nvmet_pci_epf_configure_bar(struct nvmet_= pci_epf *nvme_epf) size_t pba_size; =20 msix_table_size =3D PCI_MSIX_ENTRY_SIZE * epf->msix_interrupts; - nvme_epf->msix_table_offset =3D reg_size; - pba_size =3D ALIGN(DIV_ROUND_UP(epf->msix_interrupts, 8), 8); + pba_size =3D BITS_TO_U64(epf->msix_interrupts) * sizeof(u64); + + nvme_epf->msix_layout.table_bar =3D BAR_0; + nvme_epf->msix_layout.table_offset =3D reg_size; + nvme_epf->msix_layout.table_size =3D msix_table_size; + nvme_epf->msix_layout.pba_bar =3D BAR_0; + nvme_epf->msix_layout.pba_offset =3D reg_size + msix_table_size; + nvme_epf->msix_layout.pba_size =3D pba_size; =20 reg_size +=3D msix_table_size + pba_size; } @@ -2245,8 +2251,8 @@ static int nvmet_pci_epf_init_irq(struct nvmet_pci_ep= f *nvme_epf) /* Enable MSI-X if supported, otherwise, use MSI. */ if (epc_features->msix_capable && epf->msix_interrupts) { ret =3D pci_epc_set_msix(epf->epc, epf->func_no, epf->vfunc_no, - epf->msix_interrupts, BAR_0, - nvme_epf->msix_table_offset); + epf->msix_interrupts, + &nvme_epf->msix_layout); if (ret) { dev_err(&epf->dev, "Failed to configure MSI-X\n"); return ret; diff --git a/drivers/pci/controller/cadence/pcie-cadence-ep.c b/drivers/pci= /controller/cadence/pcie-cadence-ep.c index c0e1194a936b..2b69ea88aed2 100644 --- a/drivers/pci/controller/cadence/pcie-cadence-ep.c +++ b/drivers/pci/controller/cadence/pcie-cadence-ep.c @@ -293,7 +293,8 @@ static int cdns_pcie_ep_get_msix(struct pci_epc *epc, u= 8 func_no, u8 vfunc_no) } =20 static int cdns_pcie_ep_set_msix(struct pci_epc *epc, u8 fn, u8 vfn, - u16 nr_irqs, enum pci_barno bir, u32 offset) + u16 nr_irqs, + const struct pci_epc_msix_layout *layout) { struct cdns_pcie_ep *ep =3D epc_get_drvdata(epc); struct cdns_pcie *pcie =3D &ep->pcie; @@ -311,12 +312,12 @@ static int cdns_pcie_ep_set_msix(struct pci_epc *epc,= u8 fn, u8 vfn, =20 /* Set MSI-X BAR and offset */ reg =3D cap + PCI_MSIX_TABLE; - val =3D offset | bir; + val =3D layout->table_offset | layout->table_bar; cdns_pcie_ep_fn_writel(pcie, fn, reg, val); =20 - /* Set PBA BAR and offset. BAR must match MSI-X BAR */ + /* Set PBA BAR and offset */ reg =3D cap + PCI_MSIX_PBA; - val =3D (offset + (nr_irqs * PCI_MSIX_ENTRY_SIZE)) | bir; + val =3D layout->pba_offset | layout->pba_bar; cdns_pcie_ep_fn_writel(pcie, fn, reg, val); =20 return 0; diff --git a/drivers/pci/controller/dwc/pcie-designware-ep.c b/drivers/pci/= controller/dwc/pcie-designware-ep.c index 7d2794945704..147b043589f0 100644 --- a/drivers/pci/controller/dwc/pcie-designware-ep.c +++ b/drivers/pci/controller/dwc/pcie-designware-ep.c @@ -737,7 +737,8 @@ static int dw_pcie_ep_get_msix(struct pci_epc *epc, u8 = func_no, u8 vfunc_no) } =20 static int dw_pcie_ep_set_msix(struct pci_epc *epc, u8 func_no, u8 vfunc_n= o, - u16 nr_irqs, enum pci_barno bir, u32 offset) + u16 nr_irqs, + const struct pci_epc_msix_layout *layout) { struct dw_pcie_ep *ep =3D epc_get_drvdata(epc); struct dw_pcie *pci =3D to_dw_pcie_from_ep(ep); @@ -757,11 +758,11 @@ static int dw_pcie_ep_set_msix(struct pci_epc *epc, u= 8 func_no, u8 vfunc_no, dw_pcie_ep_writew_dbi(ep, func_no, reg, val); =20 reg =3D ep_func->msix_cap + PCI_MSIX_TABLE; - val =3D offset | bir; + val =3D layout->table_offset | layout->table_bar; dw_pcie_ep_writel_dbi(ep, func_no, reg, val); =20 reg =3D ep_func->msix_cap + PCI_MSIX_PBA; - val =3D (offset + (nr_irqs * PCI_MSIX_ENTRY_SIZE)) | bir; + val =3D layout->pba_offset | layout->pba_bar; dw_pcie_ep_writel_dbi(ep, func_no, reg, val); =20 dw_pcie_dbi_ro_wr_dis(pci); diff --git a/drivers/pci/endpoint/functions/pci-epf-ntb.c b/drivers/pci/end= point/functions/pci-epf-ntb.c index 5314aca2188a..f3e0e1b3ffb9 100644 --- a/drivers/pci/endpoint/functions/pci-epf-ntb.c +++ b/drivers/pci/endpoint/functions/pci-epf-ntb.c @@ -90,12 +90,11 @@ struct epf_ntb_epc { u8 vfunc_no; bool linkup; bool is_msix; - int msix_bar; u32 spad_size; struct pci_epc *epc; struct epf_ntb *epf_ntb; void __iomem *mw_addr[6]; - size_t msix_table_offset; + struct pci_epc_msix_layout msix_layout; struct epf_ntb_ctrl *reg; struct pci_epf_bar *epf_bar; enum pci_barno epf_ntb_bar[6]; @@ -475,9 +474,9 @@ static int epf_ntb_configure_msi(struct epf_ntb *ntb, * * The MSI-X address is in the MSI-X table of EP CONTROLLER 2 and * the count of doorbell is in ctrl->argument of epf_ntb_epc that is conne= cted - * to HOST2. MSI-X table is stored memory mapped to ntb_epc->msix_bar and = the - * offset is in ntb_epc->msix_table_offset. From this epf_ntb_configure_ms= ix() - * gets the MSI-X address and data. + * to HOST2. The location of the memory-mapped MSI-X table is described by + * ntb_epc->msix_layout. From this epf_ntb_configure_msix() gets the MSI-X + * address and data. * * epf_ntb_configure_msix() also stores the MSI-X data to raise each inter= rupt * in db_data of the peer's control region. This helps the peer to raise @@ -505,8 +504,8 @@ static int epf_ntb_configure_msix(struct epf_ntb *ntb, ntb_epc =3D ntb->epc[type]; epc =3D ntb_epc->epc; =20 - epf_bar =3D &ntb_epc->epf_bar[ntb_epc->msix_bar]; - msix_tbl =3D epf_bar->addr + ntb_epc->msix_table_offset; + epf_bar =3D &ntb_epc->epf_bar[ntb_epc->msix_layout.table_bar]; + msix_tbl =3D epf_bar->addr + ntb_epc->msix_layout.table_offset; =20 peer_ntb_epc =3D ntb->epc[!type]; peer_barno =3D peer_ntb_epc->epf_ntb_bar[BAR_DB_MW1]; @@ -1036,10 +1035,14 @@ static int epf_ntb_config_spad_bar_alloc(struct epf= _ntb *ntb, if (msix_capable) { msix_table_size =3D PCI_MSIX_ENTRY_SIZE * ntb->db_count; ctrl_size =3D ALIGN(ctrl_size, 8); - ntb_epc->msix_table_offset =3D ctrl_size; - ntb_epc->msix_bar =3D barno; - /* Align to QWORD or 8 Bytes */ - pba_size =3D ALIGN(DIV_ROUND_UP(ntb->db_count, 8), 8); + pba_size =3D BITS_TO_U64(ntb->db_count) * sizeof(u64); + + ntb_epc->msix_layout.table_bar =3D barno; + ntb_epc->msix_layout.table_offset =3D ctrl_size; + ntb_epc->msix_layout.table_size =3D msix_table_size; + ntb_epc->msix_layout.pba_bar =3D barno; + ntb_epc->msix_layout.pba_offset =3D ctrl_size + msix_table_size; + ntb_epc->msix_layout.pba_size =3D pba_size; ctrl_size =3D ctrl_size + msix_table_size + pba_size; } =20 @@ -1317,10 +1320,9 @@ static int epf_ntb_configure_interrupt(struct epf_nt= b *ntb, =20 if (msix_capable) { ret =3D pci_epc_set_msix(epc, func_no, vfunc_no, ntb->db_count, - ntb_epc->msix_bar, - ntb_epc->msix_table_offset); + &ntb_epc->msix_layout); if (ret) { - dev_err(dev, "MSI configuration failed\n"); + dev_err(dev, "MSI-X configuration failed\n"); return ret; } } diff --git a/drivers/pci/endpoint/functions/pci-epf-test.c b/drivers/pci/en= dpoint/functions/pci-epf-test.c index 4802d4f80f78..e48292e0fb37 100644 --- a/drivers/pci/endpoint/functions/pci-epf-test.c +++ b/drivers/pci/endpoint/functions/pci-epf-test.c @@ -82,7 +82,7 @@ struct pci_epf_test { struct pci_epf *epf; struct config_group group; enum pci_barno test_reg_bar; - size_t msix_table_offset; + struct pci_epc_msix_layout msix_layout; struct delayed_work cmd_handler; struct dma_chan *dma_chan_tx; struct dma_chan *dma_chan_rx; @@ -1221,8 +1221,7 @@ static int pci_epf_test_epc_init(struct pci_epf *epf) if (epc_features->msix_capable) { ret =3D pci_epc_set_msix(epc, epf->func_no, epf->vfunc_no, epf->msix_interrupts, - epf_test->test_reg_bar, - epf_test->msix_table_offset); + &epf_test->msix_layout); if (ret) { dev_err(dev, "MSI-X configuration failed\n"); return ret; @@ -1288,9 +1287,15 @@ static int pci_epf_test_alloc_space(struct pci_epf *= epf) =20 if (epc_features->msix_capable) { msix_table_size =3D PCI_MSIX_ENTRY_SIZE * epf->msix_interrupts; - epf_test->msix_table_offset =3D test_reg_bar_size; - /* Align to QWORD or 8 Bytes */ - pba_size =3D ALIGN(DIV_ROUND_UP(epf->msix_interrupts, 8), 8); + pba_size =3D BITS_TO_U64(epf->msix_interrupts) * sizeof(u64); + + epf_test->msix_layout.table_bar =3D test_reg_bar; + epf_test->msix_layout.table_offset =3D test_reg_bar_size; + epf_test->msix_layout.table_size =3D msix_table_size; + epf_test->msix_layout.pba_bar =3D test_reg_bar; + epf_test->msix_layout.pba_offset =3D test_reg_bar_size + + msix_table_size; + epf_test->msix_layout.pba_size =3D pba_size; } test_reg_size =3D test_reg_bar_size + msix_table_size + pba_size; =20 diff --git a/drivers/pci/endpoint/pci-epc-core.c b/drivers/pci/endpoint/pci= -epc-core.c index 831b40458dcd..4a0ceb62f38f 100644 --- a/drivers/pci/endpoint/pci-epc-core.c +++ b/drivers/pci/endpoint/pci-epc-core.c @@ -6,6 +6,7 @@ * Author: Kishon Vijay Abraham I */ =20 +#include #include #include #include @@ -113,6 +114,84 @@ enum pci_barno pci_epc_get_next_free_bar(const struct = pci_epc_features } EXPORT_SYMBOL_GPL(pci_epc_get_next_free_bar); =20 +static int pci_epc_get_msix_region(const struct pci_epc_features *epc_feat= ures, + enum pci_epc_bar_rsvd_region_type type, + enum pci_barno *bar, u32 *offset, + resource_size_t *size) +{ + const struct pci_epc_bar_rsvd_region *region; + const struct pci_epc_bar_desc *bar_desc; + bool found =3D false; + int i, j; + + if (!epc_features) + return -ENOENT; + + for (i =3D BAR_0; i < PCI_STD_NUM_BARS; i++) { + bar_desc =3D &epc_features->bar[i]; + if (bar_desc->nr_rsvd_regions && !bar_desc->rsvd_regions) + return -EINVAL; + + for (j =3D 0; j < bar_desc->nr_rsvd_regions; j++) { + region =3D &bar_desc->rsvd_regions[j]; + if (region->type !=3D type) + continue; + + if (found || bar_desc->type !=3D BAR_RESERVED || !region->size || + region->offset > PCI_MSIX_TABLE_OFFSET || + !IS_ALIGNED(region->offset, 8)) + return -EINVAL; + + found =3D true; + *bar =3D i; + *offset =3D region->offset; + *size =3D region->size; + } + } + + return found ? 0 : -ENOENT; +} + +/** + * pci_epc_get_hw_msix_layout() - get a hardware-owned MSI-X table and PBA= layout + * @epc_features: features provided by an EPC for an endpoint function + * @layout: layout to populate + * + * Return: 0 if the EPC describes both hardware-owned MSI-X regions, -ENOE= NT if + * neither region is described, or an error if the description is invalid. + */ +int pci_epc_get_hw_msix_layout(const struct pci_epc_features *epc_features, + struct pci_epc_msix_layout *layout) +{ + struct pci_epc_msix_layout hw_layout; + int table_ret, pba_ret; + + if (!layout) + return -EINVAL; + + table_ret =3D pci_epc_get_msix_region(epc_features, + PCI_EPC_BAR_RSVD_MSIX_TBL_RAM, + &hw_layout.table_bar, + &hw_layout.table_offset, + &hw_layout.table_size); + pba_ret =3D pci_epc_get_msix_region(epc_features, + PCI_EPC_BAR_RSVD_MSIX_PBA_RAM, + &hw_layout.pba_bar, + &hw_layout.pba_offset, + &hw_layout.pba_size); + + if (table_ret =3D=3D -ENOENT && pba_ret =3D=3D -ENOENT) + return -ENOENT; + + if (table_ret || pba_ret) + return -EINVAL; + + *layout =3D hw_layout; + + return 0; +} +EXPORT_SYMBOL_GPL(pci_epc_get_hw_msix_layout); + static bool pci_epc_function_is_valid(struct pci_epc *epc, u8 func_no, u8 vfunc_no) { @@ -443,14 +522,14 @@ EXPORT_SYMBOL_GPL(pci_epc_get_msix); * @func_no: the physical endpoint function number in the EPC device * @vfunc_no: the virtual endpoint function number in the physical function * @nr_irqs: number of MSI-X interrupts required by the EPF - * @bir: BAR where the MSI-X table resides - * @offset: Offset pointing to the start of MSI-X table + * @layout: MSI-X table and PBA layout selected by the EPF * * Invoke to set the required number of MSI-X interrupts. */ int pci_epc_set_msix(struct pci_epc *epc, u8 func_no, u8 vfunc_no, u16 nr_= irqs, - enum pci_barno bir, u32 offset) + const struct pci_epc_msix_layout *layout) { + size_t table_size, pba_size; int ret; =20 if (!pci_epc_function_is_valid(epc, func_no, vfunc_no)) @@ -459,11 +538,26 @@ int pci_epc_set_msix(struct pci_epc *epc, u8 func_no,= u8 vfunc_no, u16 nr_irqs, if (nr_irqs < 1 || nr_irqs > 2048) return -EINVAL; =20 + if (!layout || layout->table_bar < BAR_0 || + layout->table_bar >=3D PCI_STD_NUM_BARS || + layout->pba_bar < BAR_0 || layout->pba_bar >=3D PCI_STD_NUM_BARS || + !IS_ALIGNED(layout->table_offset, 8) || + !IS_ALIGNED(layout->pba_offset, 8) || + layout->table_offset > PCI_MSIX_TABLE_OFFSET || + layout->pba_offset > PCI_MSIX_PBA_OFFSET) + return -EINVAL; + + table_size =3D nr_irqs * PCI_MSIX_ENTRY_SIZE; + pba_size =3D BITS_TO_U64(nr_irqs) * sizeof(u64); + + if (layout->table_size < table_size || layout->pba_size < pba_size) + return -ENOSPC; + if (!epc->ops->set_msix) return 0; =20 mutex_lock(&epc->lock); - ret =3D epc->ops->set_msix(epc, func_no, vfunc_no, nr_irqs, bir, offset); + ret =3D epc->ops->set_msix(epc, func_no, vfunc_no, nr_irqs, layout); mutex_unlock(&epc->lock); =20 return ret; diff --git a/include/linux/pci-epc.h b/include/linux/pci-epc.h index f247cf9bcf1a..3d28231f092a 100644 --- a/include/linux/pci-epc.h +++ b/include/linux/pci-epc.h @@ -102,6 +102,24 @@ struct pci_epc_aux_resource { } u; }; =20 +/** + * struct pci_epc_msix_layout - layout of an MSI-X table and PBA + * @table_bar: BAR containing the MSI-X table + * @table_offset: offset of the MSI-X table within @table_bar + * @table_size: size of the MSI-X table region + * @pba_bar: BAR containing the MSI-X Pending Bit Array (PBA) + * @pba_offset: offset of the MSI-X PBA within @pba_bar + * @pba_size: size of the MSI-X PBA region + */ +struct pci_epc_msix_layout { + enum pci_barno table_bar; + u32 table_offset; + resource_size_t table_size; + enum pci_barno pba_bar; + u32 pba_offset; + resource_size_t pba_size; +}; + /** * struct pci_epc_ops - set of function pointers for performing EPC operat= ions * @write_header: ops to populate configuration space header @@ -147,7 +165,8 @@ struct pci_epc_ops { u8 nr_irqs); int (*get_msi)(struct pci_epc *epc, u8 func_no, u8 vfunc_no); int (*set_msix)(struct pci_epc *epc, u8 func_no, u8 vfunc_no, - u16 nr_irqs, enum pci_barno, u32 offset); + u16 nr_irqs, + const struct pci_epc_msix_layout *layout); int (*get_msix)(struct pci_epc *epc, u8 func_no, u8 vfunc_no); int (*raise_irq)(struct pci_epc *epc, u8 func_no, u8 vfunc_no, unsigned int type, u16 interrupt_num); @@ -381,8 +400,10 @@ void pci_epc_unmap_addr(struct pci_epc *epc, u8 func_n= o, u8 vfunc_no, int pci_epc_set_msi(struct pci_epc *epc, u8 func_no, u8 vfunc_no, u8 nr_ir= qs); int pci_epc_get_msi(struct pci_epc *epc, u8 func_no, u8 vfunc_no); int pci_epc_set_msix(struct pci_epc *epc, u8 func_no, u8 vfunc_no, u16 nr_= irqs, - enum pci_barno, u32 offset); + const struct pci_epc_msix_layout *layout); int pci_epc_get_msix(struct pci_epc *epc, u8 func_no, u8 vfunc_no); +int pci_epc_get_hw_msix_layout(const struct pci_epc_features *epc_features, + struct pci_epc_msix_layout *layout); int pci_epc_map_msi_irq(struct pci_epc *epc, u8 func_no, u8 vfunc_no, phys_addr_t phys_addr, u8 interrupt_num, u32 entry_size, u32 *msi_data, u32 *msi_addr_offset); --=20 2.51.0 From nobody Sat Sep 26 20:30:26 2026 Received: from OS0P286CU010.outbound.protection.outlook.com (mail-japanwestazon11021129.outbound.protection.outlook.com [40.107.74.129]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 93E673B19D0; Sun, 30 Aug 2026 15:20:16 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.74.129 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788103218; cv=fail; b=laXp6dvE4m3EFst54RwZblFlQ2lST9hElmMgA/lUk7KQ6vanioj9EJ0G4FRkkngz1RVoJr2ztIrKiSnFc3gN45DwtkuqJM15h+C7gu70PI2udUwpwtL+Xg6CWp9LuD/wI/pXhCOT9wsbwyl0AfwOMg7Nbj7NDkh8BkFsTTlVirk= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788103218; c=relaxed/simple; bh=ps/WPtQaYmiRassaH0Mf2DwEy+hbbFES8CNqTT5a5r0=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: Content-Type:MIME-Version; b=VKbsW9+tbHzvxIrCICYbNN1SUEReOJY3lEYd3yPVxo9U9xKensz9VVDEZSG487shW+xD6VUwMF08eHDCNI+uPWiLewwuWILlGWKaFCJD9dY9PXfDx4DMAR2CxdnCourAzb9eQEEFe2Tq1Wtfs3JOXmcItAXZZxQROI/9f2dDq08= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=valinux.co.jp; spf=pass smtp.mailfrom=valinux.co.jp; dkim=pass (1024-bit key) header.d=valinux.co.jp header.i=@valinux.co.jp header.b=bz+Kyq3V; arc=fail smtp.client-ip=40.107.74.129 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=valinux.co.jp Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=valinux.co.jp Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=valinux.co.jp header.i=@valinux.co.jp header.b="bz+Kyq3V" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=luXGWPbZSMfIJVr1ktfwJ7yG11MhLZW731FWjjKbRB+psX2tU1Gs0fTiqOBMHUUaBHTP+QoqQ++EREGQp0sZv42XLcPrJvKVyoYZlAhrbX/tVelg8VBdcO/Jgt1GcWdJGp52UAfPN9zspe6gOOp7oTTnhzouA9+G9bcKT9v6chjrStoYKVmq5qcE018zMboVjoUhNPsfwEbltgY1+yWLSVe3A0lvtre7HYwaKcH4Dtahhj/grONEcyWdlZROU4waVeTDgQqn4C2gSUEd/WA20eTACj3gLHjAvUT5ydAcYmTxsngLBys/CiNx6ThztqRfZWbP6LQB/ZHHVWaPg9ExKA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=9K08nhjpY6VLETgYGFjZK7o3d80NLxxcir9aDigmJ7g=; b=Rw2sYagFwA7XEOi6glYlCAKI8BSuMcSQKdbRp4uPTfBWFaJTJQ6SNTbGm7twqRSRV4b2EhLt2Zzc53FPd6IHsVo+4XFiHRvB9r6jNqH7UUDKC1VtscjUQwESK0bzhAvFbVqFuvaSC2PvI6hv3xiiScRdmwnCQgtllOwzz7Gi0fdMTvZ5bR2F1/3IE6DJbP2SoXl1le+wnB73WM3Eu7QOmpaFzl2LXqzZZRqcOxRgm0mcyyTM12EGqfGlYVshZpJWDq6aO2ybiublEL810HlCiBkzLh3vP58i9tU1SX9Qn4M29NDSlKfx9rLSmOcEscIxJrtSl3C+FGewUGD+sk6flw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=valinux.co.jp; dmarc=pass action=none header.from=valinux.co.jp; dkim=pass header.d=valinux.co.jp; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=valinux.co.jp; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=9K08nhjpY6VLETgYGFjZK7o3d80NLxxcir9aDigmJ7g=; b=bz+Kyq3VFffy2CniAYGyjKgYxcseYgc/+NqbHQYWiJUVndQzb0Aw9821TpycuUKfO6q7ZgtxF8UTZy/LrpFcaRRLibA/ts7CvCxip7z+uknfUTI/UtHYJqoeUkWP03Un8UgNkByboANt9JsVE8u5nTglgkTwfhXxD1bhDii46mY= Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=valinux.co.jp; Received: from TY7P286MB7722.JPNP286.PROD.OUTLOOK.COM (2603:1096:405:38f::10) by OS3P286MB3211.JPNP286.PROD.OUTLOOK.COM (2603:1096:604:214::9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.360.13; Sun, 30 Aug 2026 15:20:12 +0000 Received: from TY7P286MB7722.JPNP286.PROD.OUTLOOK.COM ([fe80::2305:327c:28ec:9b32]) by TY7P286MB7722.JPNP286.PROD.OUTLOOK.COM ([fe80::2305:327c:28ec:9b32%4]) with mapi id 15.21.0360.008; Sun, 30 Aug 2026 15:20:12 +0000 From: Koichiro Den To: Manivannan Sadhasivam , =?UTF-8?q?Krzysztof=20Wilczy=C5=84ski?= , Kishon Vijay Abraham I , Frank Li , Bjorn Helgaas , Jingoo Han , Niklas Cassel Cc: Lorenzo Pieralisi , Rob Herring , Aksh Garg , Christoph Hellwig , Sagi Grimberg , Chaitanya Kulkarni , Jon Mason , Dave Jiang , Allen Hubbe , Heiko Stuebner , Shawn Lin , Manikanta Maddireddy , Shin'ichiro Kawasaki , linux-pci@vger.kernel.org, linux-nvme@lists.infradead.org, ntb@lists.linux.dev, linux-rockchip@lists.infradead.org, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org Subject: [PATCH v2 2/3] PCI: dw-rockchip: Support fixed MSI-X table and PBA on RK3588 Date: Mon, 31 Aug 2026 00:19:47 +0900 Message-ID: <20260830151948.3547577-3-den@valinux.co.jp> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20260830151948.3547577-1-den@valinux.co.jp> References: <20260830151948.3547577-1-den@valinux.co.jp> Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: TYWPR01CA0025.jpnprd01.prod.outlook.com (2603:1096:400:aa::12) To TY7P286MB7722.JPNP286.PROD.OUTLOOK.COM (2603:1096:405:38f::10) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: TY7P286MB7722:EE_|OS3P286MB3211:EE_ X-MS-Office365-Filtering-Correlation-Id: 33102875-61be-48ec-a54c-08df06aa2f7d X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|376014|23010399003|7416014|1800799024|10070799003|366016|6133799003|10067099003|56012099006|18002099003|22082099003; X-Microsoft-Antispam-Message-Info: S6PaoyqQuo00fda0S2WNBI/zb1qZklJq5d2lyn76sQvHuexELXezNzOKbAyT4MY1RxsfL5KG9auIX/2FS3yAQj/hGkd4fPeXOfUiSwwbm0YYpozvRUoGvbZohiVKLeqsSzisQhngz22rKRYiD2yPIDx1qsYBeX8ZM/i7vurjIpEt34zlK4DMbEYQC1AyNDLIA5jsgTMlht4qcgbyDqO1cj3gLs2n3yintm0B0zj8POaVtC2TvuhZ1PizGVgFLPAqdl/rTMo/iV15E2yA2k/pustCsszmcTnPWzOKWQpKBFu947bO491UnRbZl1hscb8swJSME0QuE8VEpIMOOHeAdVKnGGAZEWqFjEDh53ka/gRfDLACxntS8ui7RfYFxCGtllQpGwnNhFxdM8ue7tdnM42+wHfNmIk4/MOIO96kheKX1XiXK8c6iqNdx55xicR8GFjvvi91ixn1sVejunGMcKGzhP6qdLj40o1Rrw8ZLvTQ3hv0PxWeo7VHyK57UOu/46IxTWfxfZo4nj/SFFjV7ckYF4N+yXaMqKLiO7OgysR52OLOhYhbuF2CLvXw7AV2M6Slg4HgBzzQmBTAAvhtC0QfaONKvibKUZL3Yffw7ROlYTAnTZ+YbsqTBIgYbHMzc++8S8Fe78umV5wLgnZCpciUKg/JlxGGZdS0H/Y4SdA= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:TY7P286MB7722.JPNP286.PROD.OUTLOOK.COM;PTR:;CAT:NONE;SFS:(13230040)(376014)(23010399003)(7416014)(1800799024)(10070799003)(366016)(6133799003)(10067099003)(56012099006)(18002099003)(22082099003);DIR:OUT;SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?bi7cQ28ksk36R/Wy85otbj+vayhgagTmY7BkGf86tecYqP+tt+R5WrOXJnFx?= =?us-ascii?Q?DBvwfXa6Cz560j2vdAKS9shWub1YvJlSuToO5xG+IREjGCkTHa6jgjLcLjKe?= =?us-ascii?Q?yCSyBKaeEZnoxuJ/aMyvc6BknBmaL3c0BuDrhgUbR+YXtcVE13i7f36wmjDc?= =?us-ascii?Q?HhWAd3unU9OCliBSzpxYJQxtfkD7xFZqbP9H4kwzh9ET+jqDnIJ42hPfgD4D?= =?us-ascii?Q?GxkoCbM0WU8mkwW2atfvnbXYFmP2kH3q/KiMcyzlY+1AP1QZylqTGTYIkWMC?= =?us-ascii?Q?eECy7QqMIgsrI+NvOglx7lgnIXDvJba9fsnu7lZ7ENY5XlKMXabIDzqrINCG?= =?us-ascii?Q?YguWl1iQrfPbe4OrS61DVnVsAe3sqNxwexuGg3kAKjM8PiQBRqAsg7BMlJJt?= =?us-ascii?Q?7225z+OTkQgOQZsPfHLa8gwc4gqmk4KYwnMDRIrTsuwkzJXaZpt1Zwh1fAmU?= =?us-ascii?Q?9KTuk6cAElBz4rS/oMoglpVYadFlLxf364mkcwX5v86lJsTnb2O23fLwZ20W?= =?us-ascii?Q?MB3YrRDNva0lRvtb16azo4VIE/mmMfaM757o5HsW2AB2inusne9/DXf9RdYI?= =?us-ascii?Q?f7moZxTrVVagICMOQELrwke7x/3iqD0vwNwr/W4pw5hqbNpW7HJLNqj1Wrca?= =?us-ascii?Q?LcpNlta5XXAq/NK3YcnIu9lU37tX6GHpM+1r3F/yldV6GCRKIso7MaIBycwb?= =?us-ascii?Q?iWBL42uEZQ0xHBTsXO2HSCPpaxAo8+ObocWMNv94rr0TGgyzoEvgUG6U91XN?= =?us-ascii?Q?KqWPVmotYOKRajdPOeGi/5Uk2TdsxgcV4s8GfhrEFNq1p2ULHg1o4PB/tGsR?= =?us-ascii?Q?WEKxVJjzcq5pisTP0k6qWtqMuCTLXw3YrNQ+Bjdk0bkHmlCA9n8ngqAOjaz6?= =?us-ascii?Q?42QzBOYyLdsZRkOhxPkIp7yX59VZhBkEVQ3PQQhqpbyhhXQEgevRzB2EA98t?= =?us-ascii?Q?5TFjyxd2y/bFchK4vJ+mXwKEAuLAuYkgBvxqRAfVqkJw0RDp+B8dYzEMQluT?= =?us-ascii?Q?fSNW4nU+ghWv7oxf9DwJg8THLGS1vBtxe6HBzTwULL/134dBQGkXVrWxyrp7?= =?us-ascii?Q?ZpixfWUn1yEflaP3qFz+x5iDk7UUdfxeTJmAzX+DTT07xtXmn5LdnHZpnTPN?= =?us-ascii?Q?0pexa6tHN7XjhlC+HE/R/UePrW1sDQUfxQw9jb9Z5aPutyFdMTzcsiTDzVDy?= =?us-ascii?Q?CckSfmVg0iEeZT8dvU48pohZKrZHbxGn40LtA10uaWo5+VlVREt+sZaG/rsq?= =?us-ascii?Q?LV3OWY8VyLMvJayxw79KAXX9GVkKADCXElJrZjXD7Durjj7erfDyP3wN1lZR?= =?us-ascii?Q?DJGSgDIlEdVwEuOMEQ7na38QpLsmDeW/7a6Eg3cN+hwr0ul+A7nZ2+ROcVfA?= =?us-ascii?Q?7tPUfERJFEImJ2VEyiN/gnEdpCUu6x+fcrOEpDjHAFxL1llPTHG0/F0nC0z3?= =?us-ascii?Q?qhabcWT+BRezHxVVJ2PqZf5OQcOW3qUk/sinRQqwu2qbl8zdNoyT5LB/HZm6?= =?us-ascii?Q?VaBSqF7YYzs49VWQ5MIzKbgThAwW1w1NwEt0ej45SrsfoCIfHaAnjJMzzATX?= =?us-ascii?Q?vSNksgKMOeEEqmEfjjDqdta/OViUmKSvw8Q7+C5fXExOEbS7WQHgsZv4lrz8?= =?us-ascii?Q?Hs2wIyFZMJgkm+BDegLy+TYBYVthZGjKI8pxuWiLHAJboAAT0w6lanJClFog?= =?us-ascii?Q?zDvK7ympop+KEG0I1O6orCUwx3dqNPudZDKGgyshO0jvxGaPfZoiENp5byyM?= =?us-ascii?Q?mFD2KcbS/lyIxBT+zdFJNqRc8rXN5W+TZlvsV3JGZAMHu+OoKcbb?= X-OriginatorOrg: valinux.co.jp X-MS-Exchange-CrossTenant-Network-Message-Id: 33102875-61be-48ec-a54c-08df06aa2f7d X-MS-Exchange-CrossTenant-AuthSource: TY7P286MB7722.JPNP286.PROD.OUTLOOK.COM X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 Aug 2026 15:20:12.1357 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 7a57bee8-f73d-4c5f-a4f7-d72c91c8c111 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: Tvjnyl4yjj5mmqVEuhvBl18eXddvC5wRPJTzOCCQzD7khjkwe4FBEzm8OAGbxxCT1272ul4bY3NcyvISgJ2mJQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: OS3P286MB3211 Content-Type: text/plain; charset="utf-8" The RK3588 PCIe endpoint controller exposes its MSI-X table at BAR4+0x4000 and its PBA at BAR4+0x5000. Describe both as reserved regions so EPF drivers can select the hardware-owned layout. The regular DesignWare MSI-X interrupt path reads an EPF-owned table through epf_bar[]. A reserved, hardware-owned table has no such backing. Record whether the layout selected for each function matches the hardware-owned layout, and use the controller MSI-X doorbell on RK3588 only in that case. Continue to use the regular path for EPF-owned layouts. Signed-off-by: Koichiro Den --- Changes in v2: - Return an error if the regular MSI-X helper is called for a hardware-owned layout. (Sashiko) .../pci/controller/dwc/pcie-designware-ep.c | 23 +++++++++++++++ drivers/pci/controller/dwc/pcie-designware.h | 1 + drivers/pci/controller/dwc/pcie-dw-rockchip.c | 28 +++++++++++++++++-- 3 files changed, 49 insertions(+), 3 deletions(-) diff --git a/drivers/pci/controller/dwc/pcie-designware-ep.c b/drivers/pci/= controller/dwc/pcie-designware-ep.c index 147b043589f0..96726afec564 100644 --- a/drivers/pci/controller/dwc/pcie-designware-ep.c +++ b/drivers/pci/controller/dwc/pcie-designware-ep.c @@ -736,6 +736,26 @@ static int dw_pcie_ep_get_msix(struct pci_epc *epc, u8= func_no, u8 vfunc_no) return val + 1; } =20 +static bool +dw_pcie_ep_msix_layout_is_hw_owned(struct dw_pcie_ep *ep, + const struct pci_epc_msix_layout *layout) +{ + const struct pci_epc_features *features; + struct pci_epc_msix_layout hw_layout; + + if (!ep->ops->get_features) + return false; + + features =3D ep->ops->get_features(ep); + if (pci_epc_get_hw_msix_layout(features, &hw_layout)) + return false; + + return layout->table_bar =3D=3D hw_layout.table_bar && + layout->table_offset =3D=3D hw_layout.table_offset && + layout->pba_bar =3D=3D hw_layout.pba_bar && + layout->pba_offset =3D=3D hw_layout.pba_offset; +} + static int dw_pcie_ep_set_msix(struct pci_epc *epc, u8 func_no, u8 vfunc_n= o, u16 nr_irqs, const struct pci_epc_msix_layout *layout) @@ -764,6 +784,7 @@ static int dw_pcie_ep_set_msix(struct pci_epc *epc, u8 = func_no, u8 vfunc_no, reg =3D ep_func->msix_cap + PCI_MSIX_PBA; val =3D layout->pba_offset | layout->pba_bar; dw_pcie_ep_writel_dbi(ep, func_no, reg, val); + ep_func->msix_hw_owned =3D dw_pcie_ep_msix_layout_is_hw_owned(ep, layout); =20 dw_pcie_dbi_ro_wr_dis(pci); =20 @@ -1109,6 +1130,8 @@ int dw_pcie_ep_raise_msix_irq(struct dw_pcie_ep *ep, = u8 func_no, ep_func =3D dw_pcie_ep_get_func_from_ep(ep, func_no); if (!ep_func || !ep_func->msix_cap) return -EINVAL; + if (ep_func->msix_hw_owned) + return -EOPNOTSUPP; =20 reg =3D ep_func->msix_cap + PCI_MSIX_TABLE; tbl_offset =3D dw_pcie_ep_readl_dbi(ep, func_no, reg); diff --git a/drivers/pci/controller/dwc/pcie-designware.h b/drivers/pci/con= troller/dwc/pcie-designware.h index de4b245b1758..043c39b5881d 100644 --- a/drivers/pci/controller/dwc/pcie-designware.h +++ b/drivers/pci/controller/dwc/pcie-designware.h @@ -495,6 +495,7 @@ struct dw_pcie_ep_func { u8 func_no; u8 msi_cap; /* MSI capability offset */ u8 msix_cap; /* MSI-X capability offset */ + bool msix_hw_owned; u8 bar_to_atu[PCI_STD_NUM_BARS]; struct pci_epf_bar *epf_bar[PCI_STD_NUM_BARS]; =20 diff --git a/drivers/pci/controller/dwc/pcie-dw-rockchip.c b/drivers/pci/co= ntroller/dwc/pcie-dw-rockchip.c index 731d93663cca..d622723dfcf8 100644 --- a/drivers/pci/controller/dwc/pcie-dw-rockchip.c +++ b/drivers/pci/controller/dwc/pcie-dw-rockchip.c @@ -120,6 +120,7 @@ struct rockchip_pcie { struct rockchip_pcie_of_data { enum dw_pcie_device_mode mode; const struct pci_epc_features *epc_features; + bool msix_doorbell; }; =20 static int rockchip_pcie_readl_apb(struct rockchip_pcie *rockchip, u32 reg) @@ -481,6 +482,8 @@ static int rockchip_pcie_raise_irq(struct dw_pcie_ep *e= p, u8 func_no, unsigned int type, u16 interrupt_num) { struct dw_pcie *pci =3D to_dw_pcie_from_ep(ep); + struct rockchip_pcie *rockchip =3D to_rockchip_pcie(pci); + struct dw_pcie_ep_func *ep_func; =20 switch (type) { case PCI_IRQ_INTX: @@ -488,6 +491,12 @@ static int rockchip_pcie_raise_irq(struct dw_pcie_ep *= ep, u8 func_no, case PCI_IRQ_MSI: return dw_pcie_ep_raise_msi_irq(ep, func_no, interrupt_num); case PCI_IRQ_MSIX: + ep_func =3D dw_pcie_ep_get_func_from_ep(ep, func_no); + if (rockchip->data->msix_doorbell && ep_func && + ep_func->msix_hw_owned) + return dw_pcie_ep_raise_msix_irq_doorbell(ep, func_no, + interrupt_num); + return dw_pcie_ep_raise_msix_irq(ep, func_no, interrupt_num); default: dev_err(pci->dev, "UNKNOWN IRQ type\n"); @@ -517,12 +526,24 @@ static const struct pci_epc_bar_rsvd_region rk3588_ba= r4_rsvd[] =3D { .offset =3D 0x0, .size =3D 0x2000, }, + { + /* MSI-X Table (BAR4: MSI-X Table) */ + .type =3D PCI_EPC_BAR_RSVD_MSIX_TBL_RAM, + .offset =3D 0x4000, + .size =3D SZ_4K, + }, + { + /* MSI-X PBA (BAR4: MSI-X PBA) */ + .type =3D PCI_EPC_BAR_RSVD_MSIX_PBA_RAM, + .offset =3D 0x5000, + .size =3D SZ_4K, + }, }; =20 /* - * BAR4 on rk3588 exposes the ATU Port Logic Structure to the host regardl= ess of - * iATU settings for BAR4. This means that BAR4 cannot be used by an EPF d= river, - * so mark it as RESERVED. + * BAR4 on RK3588 exposes the DMA and ATU Port Logic Structures and the MS= I-X + * table and PBA to the host regardless of iATU settings for BAR4. This me= ans + * that BAR4 cannot be used by an EPF driver, so mark it as RESERVED. */ static const struct pci_epc_features rockchip_pcie_epc_features_rk3588 =3D= { DWC_EPC_COMMON_FEATURES, @@ -841,6 +862,7 @@ static const struct rockchip_pcie_of_data rockchip_pcie= _ep_of_data_rk3568 =3D { static const struct rockchip_pcie_of_data rockchip_pcie_ep_of_data_rk3588 = =3D { .mode =3D DW_PCIE_EP_TYPE, .epc_features =3D &rockchip_pcie_epc_features_rk3588, + .msix_doorbell =3D true, }; =20 static const struct of_device_id rockchip_pcie_of_match[] =3D { --=20 2.51.0 From nobody Sat Sep 26 20:30:26 2026 Received: from OS0P286CU010.outbound.protection.outlook.com (mail-japanwestazon11021129.outbound.protection.outlook.com [40.107.74.129]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 58B2E25B0AC; Sun, 30 Aug 2026 15:20:18 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.74.129 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788103220; cv=fail; b=m25QPD8++sfIOYU5G/4lncvsI/+rWLfrvyJDIAwATYmC6qztNovNqL6AiTPuEmVNP7gFEzA24U6OR4pmwgGH7OIq6D1VjyKUCttxmYzR4vKO57v+NkfTbZCy0j4/ObACF24yeY69aY+ijC/IQjLiwzAgckfGZ7MGlw91kkhsAfg= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788103220; c=relaxed/simple; bh=IFgLUanqYYV5Dr5L0Bys/vClx7EP6Ed+ddeWc0GS4DM=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: Content-Type:MIME-Version; b=QXTZcHvJfOdVf/go9AD88k/t16+HQFW0Pflk71tUSYOFoJgBJcii5/xv3D6lKn9vI7PMAET7w/897FCd+YH1Q51HboLEAbj7v4gpWQyioJayb0vbZB4zqQlJsL3wgg1e3PlqUBLxvzfL7q6sD9EN7M6Q1lEBxOcyHBU7LwCqmpw= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=valinux.co.jp; spf=pass smtp.mailfrom=valinux.co.jp; dkim=pass (1024-bit key) header.d=valinux.co.jp header.i=@valinux.co.jp header.b=kIsgxm+0; arc=fail smtp.client-ip=40.107.74.129 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=valinux.co.jp Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=valinux.co.jp Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=valinux.co.jp header.i=@valinux.co.jp header.b="kIsgxm+0" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=U3AKWDBsVpwPVupFf7sapjS6WmrSRyu2onw6GVLrUErfYiUpiDk+afAlsUcwX2XS6WJRV59VyI4c/fiCUUV5ji6rsw5BxwYH9AYFzHEXB69TKBksnnZAWvVPGMx5zQrKMxjNT2kBPFmtWquGR98R/8lNCR27+aUM2/yim8FeRdo86cemd9Rs+fxy9Pt9Z2LrNRGcOX1sunBkacqC5yjzSVzV6WwN2RSUsEpTdeIFffXlB1RX3EDbAbhET4ovwTLvVc9UPyv2rU19oxdZqMGegA20IIXVaoTRG0cMGlQNWANMXdV9AerklWuGePALHV7xftcrMvxvTMoJ7YQrXTX71g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=efawzy7rURFDohDhxVauA3Ud/GkzLNSSI2pNYfHegKs=; b=WjpSvGqFTml7IZthR6DJmpN+1yE19tWqWxAkdSN50R28qCSLQ1Wu8nfS5F626h3dWHIiQlc29m+Yv+w3JCLyeWQUeo/ltLmyRVIj/YGarvTUMGNVGHYBozosdgNoXEjG2DUxGGeKfwnDRMBfNB35FLgxmmilbtoe6mEzXDEI88jym3Jx/fBjAsYsyKPwqvqpP7ka5VJ/vlHx6/AeY0wM5X5PIKifdWZvYZioTYQxQclWt33WZKXy+1RUuahEXvLKVYpcBYtvQXjkugueFgLyASPKRs3Xw24cRsmpDA2zi7OlwKAgiasnJlMfNKfqNyqNZ53FVgJhDwupIx/v7rl+kA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=valinux.co.jp; dmarc=pass action=none header.from=valinux.co.jp; dkim=pass header.d=valinux.co.jp; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=valinux.co.jp; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=efawzy7rURFDohDhxVauA3Ud/GkzLNSSI2pNYfHegKs=; b=kIsgxm+0LwsJ13FFLdiFLTAL20fPWItY3gGD2t3++UIR/toL+nx70UmhSy398cLBOivLQK8sdR/MSBqiCen4J0f6dJ/6A8S4/BrziZHicau9FGYaWs8u6Cd4poiKqqNKWDtgkkKE5WXx1GuD+fmvYbsA7zesE1AP39DIEXy91Ew= Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=valinux.co.jp; Received: from TY7P286MB7722.JPNP286.PROD.OUTLOOK.COM (2603:1096:405:38f::10) by OS3P286MB3211.JPNP286.PROD.OUTLOOK.COM (2603:1096:604:214::9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.360.13; Sun, 30 Aug 2026 15:20:13 +0000 Received: from TY7P286MB7722.JPNP286.PROD.OUTLOOK.COM ([fe80::2305:327c:28ec:9b32]) by TY7P286MB7722.JPNP286.PROD.OUTLOOK.COM ([fe80::2305:327c:28ec:9b32%4]) with mapi id 15.21.0360.008; Sun, 30 Aug 2026 15:20:13 +0000 From: Koichiro Den To: Manivannan Sadhasivam , =?UTF-8?q?Krzysztof=20Wilczy=C5=84ski?= , Kishon Vijay Abraham I , Frank Li , Bjorn Helgaas , Jingoo Han , Niklas Cassel Cc: Lorenzo Pieralisi , Rob Herring , Aksh Garg , Christoph Hellwig , Sagi Grimberg , Chaitanya Kulkarni , Jon Mason , Dave Jiang , Allen Hubbe , Heiko Stuebner , Shawn Lin , Manikanta Maddireddy , Shin'ichiro Kawasaki , linux-pci@vger.kernel.org, linux-nvme@lists.infradead.org, ntb@lists.linux.dev, linux-rockchip@lists.infradead.org, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org Subject: [PATCH v2 3/3] PCI: endpoint: pci-epf-vntb: Honor MSI-X selection Date: Mon, 31 Aug 2026 00:19:48 +0900 Message-ID: <20260830151948.3547577-4-den@valinux.co.jp> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20260830151948.3547577-1-den@valinux.co.jp> References: <20260830151948.3547577-1-den@valinux.co.jp> Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: TY4P301CA0114.JPNP301.PROD.OUTLOOK.COM (2603:1096:405:37e::12) To TY7P286MB7722.JPNP286.PROD.OUTLOOK.COM (2603:1096:405:38f::10) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: TY7P286MB7722:EE_|OS3P286MB3211:EE_ X-MS-Office365-Filtering-Correlation-Id: 1f380272-99a6-4b70-a14a-08df06aa3009 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|376014|23010399003|7416014|1800799024|10070799003|366016|6133799003|10067099003|5023799004|56012099006|18002099003|22082099003; X-Microsoft-Antispam-Message-Info: 9GK654R7IIjXgowQAZo4nlW6dL0vbc8YMcHksLk458eimsnGcKhJKSvf6UWHeSABD6LT0XLhuEjW1567CQ5ed8l4aNrk03uqavCYxeQkjPT7p8stN2ZCcGG96pVnLUNOYu7mbdYSVcDiK64/Im1v2zNNCsRot66kMO7s2AhuuwIWIW4KZqZF7T3bWu/SY7k45+hWGqzr3ZE+ezaAwLKVw4ptY/R0VJYWqC6nj/kIgDp35ftuzdSSlL9UoXVNEp6XbLxy/QaZURqbP9JkCRaIUl+WQoDU1FEmmqTdMXIKQpqHWC9FLSAfuE3WGfqdqMPKjvo0cpgR81mS9LlfimwOcEQxKLrXIfrijqbnBuc6APZsH+5vXr0KmnVk73goawRxWcgVqpqog9rxcwAYUrTge7rRy9LaWbDPFfZ7mqz24ZDzY5IU5MtAkan8qvcBzy8V5ZEi3tWlNcsEA56DB3CViyTE+hw87/TEwq9YlBKpI9HCVkXnOsYEmUtvBMKYHYkVEuFXeOeBo/mIlnbU+n3MouOEgOIiftgs619yenf/LQfj+axsGxrdSmR6ssJGvnjJJSZBA+XgqbFp3dNbu2NdKlU1iKSMIQnzMlFUhuPVPBeCMqtbFYS9MDPvyxZqsfuhzYbiUowPEyQIeY2bChjaUzIlYzuTZjDCKmRf79XxUoE= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:TY7P286MB7722.JPNP286.PROD.OUTLOOK.COM;PTR:;CAT:NONE;SFS:(13230040)(376014)(23010399003)(7416014)(1800799024)(10070799003)(366016)(6133799003)(10067099003)(5023799004)(56012099006)(18002099003)(22082099003);DIR:OUT;SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?6gpwNXxzSKsBml7tfJc7Wp47GhFeHr6xzNU0k2B7dt4xddB/X6YPXGKh10nz?= =?us-ascii?Q?B4Faf7oRZU+Vfk9TqCNsVGZ8CNe6GcAro5FQJQVMMGBhkcLsJafPfhfXCbmn?= =?us-ascii?Q?byPzpuLFQpsITrIi514vQn1uCSIXFAmbFsFXs0r+9QPMYrmXe6u5BFX4QrYY?= =?us-ascii?Q?YUjBipRh/tMGPgQ9Au3KmiMnNVijWvHH1D39eTbYwjfgwxd2jzljgCEae4EK?= =?us-ascii?Q?2rCA6EeEhagW3etU0Wx6FPSrNnKRN9fdgwaAr+VL8nWGJC+uDkaPBTn9wc3r?= =?us-ascii?Q?31rJ8EcBMAxUMY3VGkU/a9KrGJ52a+0/Rf3xUn0Wg2AWjbp2IySMigE2ZHBD?= =?us-ascii?Q?9ze347VYbleeCEZ/27cL7r/O0gPg4VXg8O5PdfFk6YvoJXRtst4z+VrH/Rps?= =?us-ascii?Q?5+cizfVgI+XrYl97K2Z3WD1aajq3+rDBhWhd7mJ+52b3dhg+C4EKYUz5q+qO?= =?us-ascii?Q?oWZKP53v1dZ892Oma2cSqahoxesX1OJ/kBB+ueY5bpNWjZIbArM5EnpLNZcL?= =?us-ascii?Q?LijoVTwltse8hXsmKMMPaRJU1OH9sAwTuy40wcaUlVgoaWjc86La9aaYNQ2i?= =?us-ascii?Q?u/eRaPyEoZvzoKg1NqCA2T2t9zajU4W1yScJtfR01EZ5uQWbN+Z21Lv6GDKW?= =?us-ascii?Q?aGyWjucFQsE5dmzMt9YGbC8WVSt9ujlWICuP5CwyVSLrsCCJDB7HWP87tOKC?= =?us-ascii?Q?Iqpcekjz/vbWNiO3PofyJync2xdaPwV9ilF0OJJQr5GMJR8ZKrBgbudXBBdJ?= =?us-ascii?Q?JyNTU+XaQNbGNBXDth5etqHMZBGEJhPSpQRlxzLFvOesI0+o8f2vv8DV7rC9?= =?us-ascii?Q?ycyuEu75dyuSldzabVHG/Wiew6S1t2amvXpHpCPcDw8t9sCepaPZJ5DJvVPs?= =?us-ascii?Q?LBEg7xkgsirUMHtc0rypXafXqodC+q4GA8kOy4XSPm8ccBGd1m0a2KSOo8FW?= =?us-ascii?Q?21gZmr8tfEMkoA43V1t3CBDQYOog1OIfN3B0cEJqrJTKsitH27GFnm9YNYgU?= =?us-ascii?Q?PA99fWbzyc5BmUhMsaOBow1MyJc3m9C44bYq99vldbLWmkrEgrkDb4WncZfr?= =?us-ascii?Q?NqBdhAxQPTT7hOwazVJqpGOkelGAtzRa0Lu6028ug/+VjfiUFlOuvaFgMpS5?= =?us-ascii?Q?y/lM5XAHSKwyFXdJ6C6IRPhGT+yx9HNxyeoYET+aFxxh9H7FueRIdWnwY2TI?= =?us-ascii?Q?MC6WGSsjzsTR5ECqy8XegvQXew6x1ilvJyYx6QUaYu+znANqCQKktrPXmoDi?= =?us-ascii?Q?Csdkb2+8gUqnm4bHH+Wo83RzjRyMMNPgKiu99ReYhEx8HXJri3jEjaeunvqq?= =?us-ascii?Q?kF+/P5yUdfB+i4qC4IzcB9CRExSBdnCcKuouhWy+RKVy8CKy77gzKJgoP515?= =?us-ascii?Q?+aF+dS2XyCR6PRhAoFIbmNKF962/TmfbyA5nsZNJ0D05h8X4AVZ1qm+wslQ3?= =?us-ascii?Q?KQjYBb85MR/Wh1ppPECyeCZgA28+a3Mla6B6OmR37MAzRjZsrAXr9YngczXv?= =?us-ascii?Q?48xDw/hRsCLv7sHwpwZCjAKJ1TSaXgjN406HmwfKze4MXXdlSOePU9HWYz3u?= =?us-ascii?Q?6cJX9X8Rv9op4Ib8t/Ps4OJCuoK7lMcZy49wNf5HBGjCdNftVB+uk7ZiNjLj?= =?us-ascii?Q?51Gfzw2VsMTuPrrBIP8YgWsuHFPHyXqNYPFuUy2YovbLTo9ic6jLjB49Dlyo?= =?us-ascii?Q?MlOU0+dktm8UpRJ3sQg3ppvEudxf2tLFFQ39WcNUIRJfcyDilnVtEweccIop?= =?us-ascii?Q?/EpA0HI6NCuc5kB8aQdIugCtdnCjnWgLCmPWHchCrkw58Y3ZxeVd?= X-OriginatorOrg: valinux.co.jp X-MS-Exchange-CrossTenant-Network-Message-Id: 1f380272-99a6-4b70-a14a-08df06aa3009 X-MS-Exchange-CrossTenant-AuthSource: TY7P286MB7722.JPNP286.PROD.OUTLOOK.COM X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 Aug 2026 15:20:13.0577 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 7a57bee8-f73d-4c5f-a4f7-d72c91c8c111 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: VhG8ePq89t9h8U7S68BoAfxOgMXfmJl7CEr1nd3OnuP9O0vZeob1eSHYDVZQiKP8l6tUuC2oyFZ0cCBmXJdyCg== X-MS-Exchange-Transport-CrossTenantHeadersStamped: OS3P286MB3211 Content-Type: text/plain; charset="utf-8" ntb_hw_epf tries MSI-X first and falls back to MSI. It reports the result in COMMAND_CONFIGURE_DOORBELL. pci-epf-vntb ignores MSIX_ENABLE, configures only MSI, and always raises peer doorbells with PCI_IRQ_MSI. When MSI-X is selected, the host does not program MSI, so raising it can issue a write to an invalid address. This was observed with an IOMMU enabled on the RC. Configure MSI-X when supported and use the selected type for peer doorbells. Use the hardware-owned layout when available. Otherwise allocate an EPF-owned Table and PBA in the config BAR. Configure db_count entries to cover the link event, the reserved slot, and the doorbell slots. Fixes: e35f56bb0330 ("PCI: endpoint: Support NTB transfer between RC and EP= ") Reviewed-by: Frank Li Signed-off-by: Koichiro Den --- drivers/pci/endpoint/functions/pci-epf-vntb.c | 67 ++++++++++++++----- 1 file changed, 51 insertions(+), 16 deletions(-) diff --git a/drivers/pci/endpoint/functions/pci-epf-vntb.c b/drivers/pci/en= dpoint/functions/pci-epf-vntb.c index c3caec927d74..d9622a5d4710 100644 --- a/drivers/pci/endpoint/functions/pci-epf-vntb.c +++ b/drivers/pci/endpoint/functions/pci-epf-vntb.c @@ -148,6 +148,7 @@ struct epf_ntb { u16 vntb_vid; =20 bool linkup; + bool peer_msix; =20 /* * True when doorbells are interrupt-driven (MSI or embedded), false @@ -155,6 +156,7 @@ struct epf_ntb { */ bool msi_doorbell; u32 spad_size; + struct pci_epc_msix_layout msix_layout; =20 enum pci_barno epf_ntb_bar[VNTB_BAR_NUM]; =20 @@ -303,6 +305,7 @@ static void epf_ntb_cmd_handler(struct work_struct *wor= k) =20 switch (command) { case COMMAND_CONFIGURE_DOORBELL: + WRITE_ONCE(ntb->peer_msix, argument & MSIX_ENABLE); ctrl->command_status =3D COMMAND_STATUS_OK; break; case COMMAND_TEARDOWN_DOORBELL: @@ -439,9 +442,9 @@ static void epf_ntb_config_spad_bar_free(struct epf_ntb= *ntb) * region * @ntb: NTB device that facilitates communication between HOST and VHOST * - * Allocate the Local Memory mentioned in the above diagram. The size of - * CONFIG REGION is sizeof(struct epf_ntb_ctrl) and size of SCRATCHPAD REG= ION - * is obtained from "spad-count" configfs entry. + * Allocate the control and scratchpad regions described in the above diag= ram. + * If the EPC does not provide a hardware-owned MSI-X table and PBA, alloc= ate + * space for them between the control and scratchpad regions. * * Returns: Zero for success, or an error code in case of failure */ @@ -454,7 +457,7 @@ static int epf_ntb_config_spad_bar_alloc(struct epf_ntb= *ntb) struct device *dev =3D &epf->dev; u32 spad_count; void *base; - int i; + int i, ret; const struct pci_epc_features *epc_features =3D pci_epc_get_features(epf-= >epc, epf->func_no, epf->vfunc_no); @@ -462,6 +465,29 @@ static int epf_ntb_config_spad_bar_alloc(struct epf_nt= b *ntb) spad_count =3D ntb->spad_count; =20 ctrl_size =3D ALIGN(sizeof(struct epf_ntb_ctrl), sizeof(u32)); + if (epc_features->msix_capable) { + ret =3D pci_epc_get_hw_msix_layout(epc_features, + &ntb->msix_layout); + if (ret && ret !=3D -ENOENT) { + dev_err(dev, "Invalid hardware-owned MSI-X layout\n"); + return ret; + } + + if (ret =3D=3D -ENOENT) { + ntb->msix_layout.table_bar =3D barno; + ntb->msix_layout.table_offset =3D ALIGN(ctrl_size, 8); + ntb->msix_layout.table_size =3D + ntb->db_count * PCI_MSIX_ENTRY_SIZE; + ntb->msix_layout.pba_bar =3D barno; + ntb->msix_layout.pba_offset =3D + ntb->msix_layout.table_offset + + ntb->msix_layout.table_size; + ntb->msix_layout.pba_size =3D + BITS_TO_U64(ntb->db_count) * sizeof(u64); + ctrl_size =3D ntb->msix_layout.pba_offset + + ntb->msix_layout.pba_size; + } + } spad_size =3D 2 * spad_count * sizeof(u32); =20 base =3D pci_epf_alloc_space(epf, ctrl_size + spad_size, @@ -502,6 +528,7 @@ static int epf_ntb_config_spad_bar_alloc(struct epf_ntb= *ntb) static int epf_ntb_configure_interrupt(struct epf_ntb *ntb) { const struct pci_epc_features *epc_features; + struct pci_epf *epf =3D ntb->epf; struct device *dev; int ret; =20 @@ -521,16 +548,22 @@ static int epf_ntb_configure_interrupt(struct epf_ntb= *ntb) } =20 if (epc_features->msi_capable) { - ret =3D pci_epc_set_msi(ntb->epf->epc, - ntb->epf->func_no, - ntb->epf->vfunc_no, - 16); + ret =3D pci_epc_set_msi(epf->epc, epf->func_no, epf->vfunc_no, 16); if (ret) { dev_err(dev, "MSI configuration failed\n"); return ret; } } =20 + if (epc_features->msix_capable) { + ret =3D pci_epc_set_msix(epf->epc, epf->func_no, epf->vfunc_no, + ntb->db_count, &ntb->msix_layout); + if (ret) { + dev_err(dev, "MSI-X configuration failed\n"); + return ret; + } + } + return 0; } =20 @@ -1512,6 +1545,7 @@ static void vntb_epf_peer_db_work(struct work_struct = *work) struct epf_ntb *ntb =3D container_of(work, struct epf_ntb, peer_db_work); struct pci_epf *epf =3D ntb->epf; unsigned int budget =3D VNTB_PEER_DB_WORK_BUDGET; + unsigned int irq_type; u8 func_no, vfunc_no; unsigned int db_bit; u32 interrupt_num; @@ -1523,6 +1557,7 @@ static void vntb_epf_peer_db_work(struct work_struct = *work) =20 func_no =3D epf->func_no; vfunc_no =3D epf->vfunc_no; + irq_type =3D READ_ONCE(ntb->peer_msix) ? PCI_IRQ_MSIX : PCI_IRQ_MSI; =20 /* * Drain doorbells from peer_db_pending in snapshots (atomic64_xchg()). @@ -1536,16 +1571,16 @@ static void vntb_epf_peer_db_work(struct work_struc= t *work) =20 while (db_bits) { /* - * pci_epc_raise_irq() for MSI expects a 1-based - * interrupt number. The first usable doorbell starts - * at EPF_IRQ_DB_START in the legacy slot layout. + * pci_epc_raise_irq() expects a 1-based interrupt + * number for MSI and MSI-X. The first usable doorbell + * starts at EPF_IRQ_DB_START in the legacy slot layout. * * Legacy mapping (kept for compatibility): * - * MSI #1 : link event (reserved) - * MSI #2 : unused (historical offset) - * MSI #3 : doorbell bit 0 (DB#0) - * MSI #4 : doorbell bit 1 (DB#1) + * IRQ #1 : link event (reserved) + * IRQ #2 : unused (historical offset) + * IRQ #3 : doorbell bit 0 (DB#0) + * IRQ #4 : doorbell bit 1 (DB#1) * ... * * Do not change this mapping to avoid breaking @@ -1556,7 +1591,7 @@ static void vntb_epf_peer_db_work(struct work_struct = *work) db_bits &=3D ~BIT_ULL(db_bit); =20 ret =3D pci_epc_raise_irq(epf->epc, func_no, vfunc_no, - PCI_IRQ_MSI, interrupt_num); + irq_type, interrupt_num); if (ret) dev_err(&ntb->ntb.dev, "Failed to raise IRQ for interrupt_num %u: %d\n", --=20 2.51.0