From nobody Sun Sep 27 00:39:56 2026 Received: from OS0P286CU010.outbound.protection.outlook.com (mail-japanwestazon11021112.outbound.protection.outlook.com [40.107.74.112]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9771A48989D; Thu, 27 Aug 2026 18:20:21 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.74.112 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1787854824; cv=fail; b=EqolcGXHdYKEBqt/G8takaSxRckp8oFTNZWaG/kRokWSvbwj1sxBsQHam1wYKHsnvGVVhmcXufx7+S8QpTiAgbB7Cl7zUbhtlSCD+6hc5EVLC/6SGUszN6TzgywiRotyscA00Crk7oLj515feXC0Nmo4Nd2Cyrczlw3Ui5DMjkg= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1787854824; c=relaxed/simple; bh=lkRAWyjdPw2iqLJlUEbDeUYLnC9HgHV/l/FTXyJ9vVM=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: Content-Type:MIME-Version; b=rk85cPxkzg/7YyJiMZESzluriatIbsCHjqLSVpREq+PBdyHayCVUVSzD3LQNX/Epun9/8r2ClruJTLnKoaKYM2DtzQz6oMfFbwplzMVG/9gITln2z6Z4InABrI8T47YnzZK/f4F+uN5+UXIhtr+6+JnV+LPRweW4DnxuMDt6AQo= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=valinux.co.jp; spf=pass smtp.mailfrom=valinux.co.jp; dkim=pass (1024-bit key) header.d=valinux.co.jp header.i=@valinux.co.jp header.b=o6i2oFQc; arc=fail smtp.client-ip=40.107.74.112 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=valinux.co.jp Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=valinux.co.jp Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=valinux.co.jp header.i=@valinux.co.jp header.b="o6i2oFQc" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=dLR2N2SQKhh0WBbRo23RPjGjLr6Sgj7A77EFKCPVBm80c0Gl18Qzj0l218XpY3CeeHa8jM8Ooubq/S6OB1eOgu8dYpIaiZtnYzGZolXrBICfqflBd8EhiL57RB5l1tIPWFM3D/Bzd9xkq667MkTBGzqRqXwVZcyJFhC+fC+hRVIuGHIBeLB8GSwOXeE4LyJC6wrcXxSNzNxvcSVJNY/v7P7/hthmgLkVRZUD4DItvVJVNUpYEUjpdi970eAsNRkEKsSIfIizDss9z9giCAHOVDT5CEehEVNZ1LzYCfUPRDhV2LYKseoDn5JeWJCRFjHNaUnQHhhHRdJq3gwj4A1APQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=RtjacCaigq2Si6y5XiwB8SZ6VkEcuottmHl2Lg8WU18=; b=oJfhf0LYvmUnhJbe8cxPyvhNAfeDxoauunGsbvr2XGvLof23B5VLp/f+Sn0Ko99rjqDLb/R5FqMxpUcruW1w0ceIBjngdc2Bo9mLZuTnqcWBj3PO7Gdg1MbtNU+zCZ1kQ14QJi0ichUeWK6ivpM3fO2gla85FBzwD5Sw8GqVh2lq1TG5zrZ7WqPCxyAytUgT72RDE/rIifF30CEXQYoMxAp4XpBnEWTcydIdscC4go7uJkjyKS+0A0L3op8jD2y0q8Owfwf6u+lmX5wsvzNfgu7jcMCdzYqB79yG0GeRAouI4cLHwyTBltkJlNnWf1c3QGTZd/Fw6aIJpA4CnEALIA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=valinux.co.jp; dmarc=pass action=none header.from=valinux.co.jp; dkim=pass header.d=valinux.co.jp; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=valinux.co.jp; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=RtjacCaigq2Si6y5XiwB8SZ6VkEcuottmHl2Lg8WU18=; b=o6i2oFQciJ5mBtIr/NFw+umWYlXS+rx+e49VvylPUsJluFbvgysPSX66ksvRMIO164GDVt39NSuRaCJLUnH2MCa8tFc4FBiymypaILrTRCSiDKHsqD2RdpmJgwdzlbPMEjasemNkFyyNijSQMh1K/t3DvYMT62Tk4b2i8WBoOjc= Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=valinux.co.jp; Received: from TY7P286MB7722.JPNP286.PROD.OUTLOOK.COM (2603:1096:405:38f::10) by TY7P286MB7058.JPNP286.PROD.OUTLOOK.COM (2603:1096:405:31d::13) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.360.10; Thu, 27 Aug 2026 18:20:18 +0000 Received: from TY7P286MB7722.JPNP286.PROD.OUTLOOK.COM ([fe80::2305:327c:28ec:9b32]) by TY7P286MB7722.JPNP286.PROD.OUTLOOK.COM ([fe80::2305:327c:28ec:9b32%4]) with mapi id 15.21.0360.008; Thu, 27 Aug 2026 18:20:18 +0000 From: Koichiro Den To: Manivannan Sadhasivam , =?UTF-8?q?Krzysztof=20Wilczy=C5=84ski?= , Kishon Vijay Abraham I , Frank Li , Bjorn Helgaas , Jingoo Han , Niklas Cassel Cc: Lorenzo Pieralisi , Rob Herring , Aksh Garg , Christoph Hellwig , Sagi Grimberg , Chaitanya Kulkarni , Jon Mason , Dave Jiang , Allen Hubbe , Heiko Stuebner , Shawn Lin , Manikanta Maddireddy , Shin'ichiro Kawasaki , linux-pci@vger.kernel.org, linux-nvme@lists.infradead.org, ntb@lists.linux.dev, linux-rockchip@lists.infradead.org, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org Subject: [PATCH 1/3] PCI: endpoint: Support hardware-owned MSI-X table and PBA Date: Fri, 28 Aug 2026 03:20:10 +0900 Message-ID: <20260827182012.1984960-2-den@valinux.co.jp> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20260827182012.1984960-1-den@valinux.co.jp> References: <20260827182012.1984960-1-den@valinux.co.jp> Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: TYCP286CA0335.JPNP286.PROD.OUTLOOK.COM (2603:1096:400:38e::13) To TY7P286MB7722.JPNP286.PROD.OUTLOOK.COM (2603:1096:405:38f::10) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: TY7P286MB7722:EE_|TY7P286MB7058:EE_ X-MS-Office365-Filtering-Correlation-Id: 535128d3-088b-466e-4151-08df0467d921 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|1800799024|7416014|376014|23010399003|10070799003|366016|6133799003|10067099003|56012099006|5023799004|22082099003|18002099003; X-Microsoft-Antispam-Message-Info: QKNqAj2eaaIm4GAWuaYYBrTmivzG4bxCOkdzubbvw3mGfZwBX73l7aL0O+3Ca6byp1XVjW/0eQPlP+jHfAzCXVQUunlVyjdIDFp3lv4XO/1xReSQpbcA4n6ZGmB+2tklWgcNcOzAPn2NuqJm86dto0xvWedSzFHgDDegUN83L665nEuJQVYJ07ewpCH8d0EHxMg8GL/doOzvkVwa6qXGqqYwxAfpgYbBXC6Ls8sqC7DY+PrSeYd+dlZCdA53x4VTEpTycCpqC/lX3e2iOBBrn8F855SHCzsar/eQ68Yo+uu6Bq6VDyjrHgWjXINNK1m60Gik0fVqrOWUr5REuFUbHvD4MjCUJJgL+iihj5nGre3E3q7iaHOEE+zn2iNjCsQ+oU18dxeEuqpxFRiBctt+ojezGZey2e7GjB0shpxWrW1GPE2yqE95mJXjSg8lsf8KKRYo81aciDmjAq4cIDxWnajw03q1LpvPLZ2e77AOSt0fNRCUUyoIYOm1nW9xXikFVDz4krllmQ8lsMHjSaN1nacjO7s9AjE6HqhAIWQ+BiwKvlTbq1jSCuLXFID3SYsAMawAh6eEFm14Y0Me/jUPg+02ZQQhA9Bz4IJvDHWnotgVzflFngVoteelHyy08kJEAKqogbPgxKNkoceLjo70MOV6svl15Fz+Yr4wQo5sCmQ= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:TY7P286MB7722.JPNP286.PROD.OUTLOOK.COM;PTR:;CAT:NONE;SFS:(13230040)(1800799024)(7416014)(376014)(23010399003)(10070799003)(366016)(6133799003)(10067099003)(56012099006)(5023799004)(22082099003)(18002099003);DIR:OUT;SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?skcvwv9xugQ5Xs6bVYfwMTYPOp92qM4DRjzDhauR8YWTBQcrCDNufs+n0tjC?= =?us-ascii?Q?hWoYwzITvYvpOJ4L2aYciBk2JKam5jCWmLKW4p6PhN2ltzxhJQKKyrWpIBQ7?= =?us-ascii?Q?ZTbRgcsWgJgTZyTn6JDZWFHb0yVtJ0yv3b/NpxWYd+ts4vInjzjjAfYZ+JIe?= =?us-ascii?Q?pBJMFLmcWuuBgyGK9JcdXVJjyhCHC2KhEuRcnAhMbSxqyZXSuz/Tdj7YbJJe?= =?us-ascii?Q?7bdYG78njEzc7ihUujag4gggJ44csns0GNrVvK/D77O+H37vUOP5VNmWfLII?= =?us-ascii?Q?UN43yk3YNPgoA8UrCvSzthyc46PkfCW9n5Np6dsi1piKoxvgOs9/u6+Mvx6i?= =?us-ascii?Q?ZdUmMXtGzHr/eMtnI2VdKFN9UK/YjHxNVIE+hPxlIq/0tFoS5dvtYnuOdyWR?= =?us-ascii?Q?AMoCX6/4//GUTF0mpw2lw/OLiokS8H9y0T3FEbqx78TUvk2ChVI55aLZujcv?= =?us-ascii?Q?Ce+TSGySkr2QmEtvKtGnCk5nHzSJRdmAHCuJaTVcEcfSN0lfx2lJuzwFcFuC?= =?us-ascii?Q?kyosoIiwUL5igxWToMAvqu0zqtL3lCwxKlvhH6/XRk/jx/BA1ukOjhRDmbTI?= =?us-ascii?Q?sj3KLFPsrt+VoWlCqvC/R8g6hVmYa55/s11U4hV0qRhwNbvkpC4A+d1M23RD?= =?us-ascii?Q?YKVdO0iTjH8LRcib2koLiBNjxNAvlmfh82TlBOhBvKOZ1XHodtgOM/cpyUd5?= =?us-ascii?Q?XCRkCFTbqdpx7HyvC3GblVE8FI3L7dMJYMKPuvBBlmyXOYzHKysxZc+HwICl?= =?us-ascii?Q?3A6CcMXL+fP/gcpz6wYznryZbsinMfAE7zzuYjyh66qk1Xkwq+bO9BbDuTww?= =?us-ascii?Q?hsb/j0YQBA9HCt0GVEUBXTW5Zqm7fQiiXZMYJhNkWc9tJ/mzR2j6rvCCpD1l?= =?us-ascii?Q?836re8anQkfhVTNCjdObJm4ytsE0NzOWmwY20OySKqPhVnbkGnIQRrBMjpkS?= =?us-ascii?Q?1Y67lY1tItavjKW4hTjSjo7KUVPUQyF8fVXxpve83oMLrz5GsVdhEo93wo7j?= =?us-ascii?Q?3LC7cJfHAA954M7AlHVUZcoF76shz6f9HuJTCawCaaYNfihwvr/Nkwf83QRH?= =?us-ascii?Q?rWr3idYQzQMN12o09n+Kd7P8RvJnhyhRYybZuXRMINwxdKtCqfJEo7lKaA6Q?= =?us-ascii?Q?/IXtphK9UIlzJ2+WmBwqBVoyuGy1dcjgglpy5UhXmu2oMmZe4yaSen27NFs2?= =?us-ascii?Q?5PohUCIrLiqgc6v4aOGTHTy7hr30lhk2kH17LWLfZXVbg/S4NU1gAcxCjAkD?= =?us-ascii?Q?uolQolEg/y79TGGzMG4y6i3IspTbfebn57d4zYzFj+sYRT+TZZQ2WENA7Kpj?= =?us-ascii?Q?Kn+tTGwhgi/uaw5UPwPG3ZCNKcF/3AJbgSn/VmdWDBGCLVSMBvYkTrPtv36N?= =?us-ascii?Q?AGsHi5urPM/ZAVa25UKv5zAYI/sy46z2Z5OZY0rAuJOlE+/AuLPiCbOE775M?= =?us-ascii?Q?qxxZSmaChgxthLpRAk60+NyXyPsG1tmoOGR/WWNR9ZMjhwQvXscL1KKV3PRU?= =?us-ascii?Q?N7a47BH3QYaAhv3DqIzLIO6WKxRoAx7OF/aUUauFtN2RTYSvB4lmXt+j54V/?= =?us-ascii?Q?E2JN1BjyIpbNf3L/Ov2AITU7J+VcPYuo0VMjxHogAITgytzbZEb/x3BXNquJ?= =?us-ascii?Q?voXKUTT5xmo8+c/7lxKRShLVoszPnUmtd5PpOzlB3PDyZ4i2nJFbgZxbn9Bt?= =?us-ascii?Q?pJutUFtiTPCjf5URPUNToyolXehMo2G4MA2pKaf2B6Z7G9N36y88weQJfcR0?= =?us-ascii?Q?l0VHf7/dIX+0KgluEo5TYbut6jUzV8STSo84B/3ZMpL3/qywfmyr?= X-OriginatorOrg: valinux.co.jp X-MS-Exchange-CrossTenant-Network-Message-Id: 535128d3-088b-466e-4151-08df0467d921 X-MS-Exchange-CrossTenant-AuthSource: TY7P286MB7722.JPNP286.PROD.OUTLOOK.COM X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 27 Aug 2026 18:20:18.1597 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 7a57bee8-f73d-4c5f-a4f7-d72c91c8c111 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 1ExQJ+a9CfGMHNXIrbylPKqqeadC6FPAj8/PkmEY2aRRHUOG9E2cdNM8z+9/TSFOrmspjMeglOAgfoxEPXvnMg== X-MS-Exchange-Transport-CrossTenantHeadersStamped: TY7P286MB7058 Content-Type: text/plain; charset="utf-8" Some endpoint controllers expose the MSI-X table and Pending Bit Array (PBA) in fixed, hardware-owned BAR regions. The EPC set_msix() callback currently receives only the table BAR and offset and assumes that the PBA immediately follows the table in the same BAR. It cannot describe a fixed layout whose table and PBA have independent locations. Add struct pci_epc_msix_layout and pass the full layout to pci_epc_set_msix(). Leave layout selection to EPF implementations. For instance, pci-epf-ntb reads the MSI-X Table from its own BAR. Using a fixed layout there would require a way to read hardware-owned Table entries. Add pci_epc_get_hw_msix_layout() for EPFs that want the fixed layout. Update the existing callers and validate the supplied layout. Signed-off-by: Koichiro Den --- drivers/nvme/target/pci-epf.c | 16 ++- .../pci/controller/cadence/pcie-cadence-ep.c | 9 +- .../pci/controller/dwc/pcie-designware-ep.c | 7 +- drivers/pci/endpoint/functions/pci-epf-ntb.c | 30 +++--- drivers/pci/endpoint/functions/pci-epf-test.c | 17 +-- drivers/pci/endpoint/pci-epc-core.c | 102 +++++++++++++++++- include/linux/pci-epc.h | 25 ++++- 7 files changed, 168 insertions(+), 38 deletions(-) diff --git a/drivers/nvme/target/pci-epf.c b/drivers/nvme/target/pci-epf.c index 4e9db96ebfec..37182f6d29b1 100644 --- a/drivers/nvme/target/pci-epf.c +++ b/drivers/nvme/target/pci-epf.c @@ -201,7 +201,7 @@ struct nvmet_pci_epf { const struct pci_epc_features *epc_features; =20 void *reg_bar; - size_t msix_table_offset; + struct pci_epc_msix_layout msix_layout; =20 unsigned int irq_type; unsigned int nr_vectors; @@ -2187,8 +2187,14 @@ static int nvmet_pci_epf_configure_bar(struct nvmet_= pci_epf *nvme_epf) size_t pba_size; =20 msix_table_size =3D PCI_MSIX_ENTRY_SIZE * epf->msix_interrupts; - nvme_epf->msix_table_offset =3D reg_size; - pba_size =3D ALIGN(DIV_ROUND_UP(epf->msix_interrupts, 8), 8); + pba_size =3D BITS_TO_U64(epf->msix_interrupts) * sizeof(u64); + + nvme_epf->msix_layout.table_bar =3D BAR_0; + nvme_epf->msix_layout.table_offset =3D reg_size; + nvme_epf->msix_layout.table_size =3D msix_table_size; + nvme_epf->msix_layout.pba_bar =3D BAR_0; + nvme_epf->msix_layout.pba_offset =3D reg_size + msix_table_size; + nvme_epf->msix_layout.pba_size =3D pba_size; =20 reg_size +=3D msix_table_size + pba_size; } @@ -2245,8 +2251,8 @@ static int nvmet_pci_epf_init_irq(struct nvmet_pci_ep= f *nvme_epf) /* Enable MSI-X if supported, otherwise, use MSI. */ if (epc_features->msix_capable && epf->msix_interrupts) { ret =3D pci_epc_set_msix(epf->epc, epf->func_no, epf->vfunc_no, - epf->msix_interrupts, BAR_0, - nvme_epf->msix_table_offset); + epf->msix_interrupts, + &nvme_epf->msix_layout); if (ret) { dev_err(&epf->dev, "Failed to configure MSI-X\n"); return ret; diff --git a/drivers/pci/controller/cadence/pcie-cadence-ep.c b/drivers/pci= /controller/cadence/pcie-cadence-ep.c index c0e1194a936b..2b69ea88aed2 100644 --- a/drivers/pci/controller/cadence/pcie-cadence-ep.c +++ b/drivers/pci/controller/cadence/pcie-cadence-ep.c @@ -293,7 +293,8 @@ static int cdns_pcie_ep_get_msix(struct pci_epc *epc, u= 8 func_no, u8 vfunc_no) } =20 static int cdns_pcie_ep_set_msix(struct pci_epc *epc, u8 fn, u8 vfn, - u16 nr_irqs, enum pci_barno bir, u32 offset) + u16 nr_irqs, + const struct pci_epc_msix_layout *layout) { struct cdns_pcie_ep *ep =3D epc_get_drvdata(epc); struct cdns_pcie *pcie =3D &ep->pcie; @@ -311,12 +312,12 @@ static int cdns_pcie_ep_set_msix(struct pci_epc *epc,= u8 fn, u8 vfn, =20 /* Set MSI-X BAR and offset */ reg =3D cap + PCI_MSIX_TABLE; - val =3D offset | bir; + val =3D layout->table_offset | layout->table_bar; cdns_pcie_ep_fn_writel(pcie, fn, reg, val); =20 - /* Set PBA BAR and offset. BAR must match MSI-X BAR */ + /* Set PBA BAR and offset */ reg =3D cap + PCI_MSIX_PBA; - val =3D (offset + (nr_irqs * PCI_MSIX_ENTRY_SIZE)) | bir; + val =3D layout->pba_offset | layout->pba_bar; cdns_pcie_ep_fn_writel(pcie, fn, reg, val); =20 return 0; diff --git a/drivers/pci/controller/dwc/pcie-designware-ep.c b/drivers/pci/= controller/dwc/pcie-designware-ep.c index 7d2794945704..147b043589f0 100644 --- a/drivers/pci/controller/dwc/pcie-designware-ep.c +++ b/drivers/pci/controller/dwc/pcie-designware-ep.c @@ -737,7 +737,8 @@ static int dw_pcie_ep_get_msix(struct pci_epc *epc, u8 = func_no, u8 vfunc_no) } =20 static int dw_pcie_ep_set_msix(struct pci_epc *epc, u8 func_no, u8 vfunc_n= o, - u16 nr_irqs, enum pci_barno bir, u32 offset) + u16 nr_irqs, + const struct pci_epc_msix_layout *layout) { struct dw_pcie_ep *ep =3D epc_get_drvdata(epc); struct dw_pcie *pci =3D to_dw_pcie_from_ep(ep); @@ -757,11 +758,11 @@ static int dw_pcie_ep_set_msix(struct pci_epc *epc, u= 8 func_no, u8 vfunc_no, dw_pcie_ep_writew_dbi(ep, func_no, reg, val); =20 reg =3D ep_func->msix_cap + PCI_MSIX_TABLE; - val =3D offset | bir; + val =3D layout->table_offset | layout->table_bar; dw_pcie_ep_writel_dbi(ep, func_no, reg, val); =20 reg =3D ep_func->msix_cap + PCI_MSIX_PBA; - val =3D (offset + (nr_irqs * PCI_MSIX_ENTRY_SIZE)) | bir; + val =3D layout->pba_offset | layout->pba_bar; dw_pcie_ep_writel_dbi(ep, func_no, reg, val); =20 dw_pcie_dbi_ro_wr_dis(pci); diff --git a/drivers/pci/endpoint/functions/pci-epf-ntb.c b/drivers/pci/end= point/functions/pci-epf-ntb.c index 5314aca2188a..f3e0e1b3ffb9 100644 --- a/drivers/pci/endpoint/functions/pci-epf-ntb.c +++ b/drivers/pci/endpoint/functions/pci-epf-ntb.c @@ -90,12 +90,11 @@ struct epf_ntb_epc { u8 vfunc_no; bool linkup; bool is_msix; - int msix_bar; u32 spad_size; struct pci_epc *epc; struct epf_ntb *epf_ntb; void __iomem *mw_addr[6]; - size_t msix_table_offset; + struct pci_epc_msix_layout msix_layout; struct epf_ntb_ctrl *reg; struct pci_epf_bar *epf_bar; enum pci_barno epf_ntb_bar[6]; @@ -475,9 +474,9 @@ static int epf_ntb_configure_msi(struct epf_ntb *ntb, * * The MSI-X address is in the MSI-X table of EP CONTROLLER 2 and * the count of doorbell is in ctrl->argument of epf_ntb_epc that is conne= cted - * to HOST2. MSI-X table is stored memory mapped to ntb_epc->msix_bar and = the - * offset is in ntb_epc->msix_table_offset. From this epf_ntb_configure_ms= ix() - * gets the MSI-X address and data. + * to HOST2. The location of the memory-mapped MSI-X table is described by + * ntb_epc->msix_layout. From this epf_ntb_configure_msix() gets the MSI-X + * address and data. * * epf_ntb_configure_msix() also stores the MSI-X data to raise each inter= rupt * in db_data of the peer's control region. This helps the peer to raise @@ -505,8 +504,8 @@ static int epf_ntb_configure_msix(struct epf_ntb *ntb, ntb_epc =3D ntb->epc[type]; epc =3D ntb_epc->epc; =20 - epf_bar =3D &ntb_epc->epf_bar[ntb_epc->msix_bar]; - msix_tbl =3D epf_bar->addr + ntb_epc->msix_table_offset; + epf_bar =3D &ntb_epc->epf_bar[ntb_epc->msix_layout.table_bar]; + msix_tbl =3D epf_bar->addr + ntb_epc->msix_layout.table_offset; =20 peer_ntb_epc =3D ntb->epc[!type]; peer_barno =3D peer_ntb_epc->epf_ntb_bar[BAR_DB_MW1]; @@ -1036,10 +1035,14 @@ static int epf_ntb_config_spad_bar_alloc(struct epf= _ntb *ntb, if (msix_capable) { msix_table_size =3D PCI_MSIX_ENTRY_SIZE * ntb->db_count; ctrl_size =3D ALIGN(ctrl_size, 8); - ntb_epc->msix_table_offset =3D ctrl_size; - ntb_epc->msix_bar =3D barno; - /* Align to QWORD or 8 Bytes */ - pba_size =3D ALIGN(DIV_ROUND_UP(ntb->db_count, 8), 8); + pba_size =3D BITS_TO_U64(ntb->db_count) * sizeof(u64); + + ntb_epc->msix_layout.table_bar =3D barno; + ntb_epc->msix_layout.table_offset =3D ctrl_size; + ntb_epc->msix_layout.table_size =3D msix_table_size; + ntb_epc->msix_layout.pba_bar =3D barno; + ntb_epc->msix_layout.pba_offset =3D ctrl_size + msix_table_size; + ntb_epc->msix_layout.pba_size =3D pba_size; ctrl_size =3D ctrl_size + msix_table_size + pba_size; } =20 @@ -1317,10 +1320,9 @@ static int epf_ntb_configure_interrupt(struct epf_nt= b *ntb, =20 if (msix_capable) { ret =3D pci_epc_set_msix(epc, func_no, vfunc_no, ntb->db_count, - ntb_epc->msix_bar, - ntb_epc->msix_table_offset); + &ntb_epc->msix_layout); if (ret) { - dev_err(dev, "MSI configuration failed\n"); + dev_err(dev, "MSI-X configuration failed\n"); return ret; } } diff --git a/drivers/pci/endpoint/functions/pci-epf-test.c b/drivers/pci/en= dpoint/functions/pci-epf-test.c index 4802d4f80f78..e48292e0fb37 100644 --- a/drivers/pci/endpoint/functions/pci-epf-test.c +++ b/drivers/pci/endpoint/functions/pci-epf-test.c @@ -82,7 +82,7 @@ struct pci_epf_test { struct pci_epf *epf; struct config_group group; enum pci_barno test_reg_bar; - size_t msix_table_offset; + struct pci_epc_msix_layout msix_layout; struct delayed_work cmd_handler; struct dma_chan *dma_chan_tx; struct dma_chan *dma_chan_rx; @@ -1221,8 +1221,7 @@ static int pci_epf_test_epc_init(struct pci_epf *epf) if (epc_features->msix_capable) { ret =3D pci_epc_set_msix(epc, epf->func_no, epf->vfunc_no, epf->msix_interrupts, - epf_test->test_reg_bar, - epf_test->msix_table_offset); + &epf_test->msix_layout); if (ret) { dev_err(dev, "MSI-X configuration failed\n"); return ret; @@ -1288,9 +1287,15 @@ static int pci_epf_test_alloc_space(struct pci_epf *= epf) =20 if (epc_features->msix_capable) { msix_table_size =3D PCI_MSIX_ENTRY_SIZE * epf->msix_interrupts; - epf_test->msix_table_offset =3D test_reg_bar_size; - /* Align to QWORD or 8 Bytes */ - pba_size =3D ALIGN(DIV_ROUND_UP(epf->msix_interrupts, 8), 8); + pba_size =3D BITS_TO_U64(epf->msix_interrupts) * sizeof(u64); + + epf_test->msix_layout.table_bar =3D test_reg_bar; + epf_test->msix_layout.table_offset =3D test_reg_bar_size; + epf_test->msix_layout.table_size =3D msix_table_size; + epf_test->msix_layout.pba_bar =3D test_reg_bar; + epf_test->msix_layout.pba_offset =3D test_reg_bar_size + + msix_table_size; + epf_test->msix_layout.pba_size =3D pba_size; } test_reg_size =3D test_reg_bar_size + msix_table_size + pba_size; =20 diff --git a/drivers/pci/endpoint/pci-epc-core.c b/drivers/pci/endpoint/pci= -epc-core.c index 831b40458dcd..4a0ceb62f38f 100644 --- a/drivers/pci/endpoint/pci-epc-core.c +++ b/drivers/pci/endpoint/pci-epc-core.c @@ -6,6 +6,7 @@ * Author: Kishon Vijay Abraham I */ =20 +#include #include #include #include @@ -113,6 +114,84 @@ enum pci_barno pci_epc_get_next_free_bar(const struct = pci_epc_features } EXPORT_SYMBOL_GPL(pci_epc_get_next_free_bar); =20 +static int pci_epc_get_msix_region(const struct pci_epc_features *epc_feat= ures, + enum pci_epc_bar_rsvd_region_type type, + enum pci_barno *bar, u32 *offset, + resource_size_t *size) +{ + const struct pci_epc_bar_rsvd_region *region; + const struct pci_epc_bar_desc *bar_desc; + bool found =3D false; + int i, j; + + if (!epc_features) + return -ENOENT; + + for (i =3D BAR_0; i < PCI_STD_NUM_BARS; i++) { + bar_desc =3D &epc_features->bar[i]; + if (bar_desc->nr_rsvd_regions && !bar_desc->rsvd_regions) + return -EINVAL; + + for (j =3D 0; j < bar_desc->nr_rsvd_regions; j++) { + region =3D &bar_desc->rsvd_regions[j]; + if (region->type !=3D type) + continue; + + if (found || bar_desc->type !=3D BAR_RESERVED || !region->size || + region->offset > PCI_MSIX_TABLE_OFFSET || + !IS_ALIGNED(region->offset, 8)) + return -EINVAL; + + found =3D true; + *bar =3D i; + *offset =3D region->offset; + *size =3D region->size; + } + } + + return found ? 0 : -ENOENT; +} + +/** + * pci_epc_get_hw_msix_layout() - get a hardware-owned MSI-X table and PBA= layout + * @epc_features: features provided by an EPC for an endpoint function + * @layout: layout to populate + * + * Return: 0 if the EPC describes both hardware-owned MSI-X regions, -ENOE= NT if + * neither region is described, or an error if the description is invalid. + */ +int pci_epc_get_hw_msix_layout(const struct pci_epc_features *epc_features, + struct pci_epc_msix_layout *layout) +{ + struct pci_epc_msix_layout hw_layout; + int table_ret, pba_ret; + + if (!layout) + return -EINVAL; + + table_ret =3D pci_epc_get_msix_region(epc_features, + PCI_EPC_BAR_RSVD_MSIX_TBL_RAM, + &hw_layout.table_bar, + &hw_layout.table_offset, + &hw_layout.table_size); + pba_ret =3D pci_epc_get_msix_region(epc_features, + PCI_EPC_BAR_RSVD_MSIX_PBA_RAM, + &hw_layout.pba_bar, + &hw_layout.pba_offset, + &hw_layout.pba_size); + + if (table_ret =3D=3D -ENOENT && pba_ret =3D=3D -ENOENT) + return -ENOENT; + + if (table_ret || pba_ret) + return -EINVAL; + + *layout =3D hw_layout; + + return 0; +} +EXPORT_SYMBOL_GPL(pci_epc_get_hw_msix_layout); + static bool pci_epc_function_is_valid(struct pci_epc *epc, u8 func_no, u8 vfunc_no) { @@ -443,14 +522,14 @@ EXPORT_SYMBOL_GPL(pci_epc_get_msix); * @func_no: the physical endpoint function number in the EPC device * @vfunc_no: the virtual endpoint function number in the physical function * @nr_irqs: number of MSI-X interrupts required by the EPF - * @bir: BAR where the MSI-X table resides - * @offset: Offset pointing to the start of MSI-X table + * @layout: MSI-X table and PBA layout selected by the EPF * * Invoke to set the required number of MSI-X interrupts. */ int pci_epc_set_msix(struct pci_epc *epc, u8 func_no, u8 vfunc_no, u16 nr_= irqs, - enum pci_barno bir, u32 offset) + const struct pci_epc_msix_layout *layout) { + size_t table_size, pba_size; int ret; =20 if (!pci_epc_function_is_valid(epc, func_no, vfunc_no)) @@ -459,11 +538,26 @@ int pci_epc_set_msix(struct pci_epc *epc, u8 func_no,= u8 vfunc_no, u16 nr_irqs, if (nr_irqs < 1 || nr_irqs > 2048) return -EINVAL; =20 + if (!layout || layout->table_bar < BAR_0 || + layout->table_bar >=3D PCI_STD_NUM_BARS || + layout->pba_bar < BAR_0 || layout->pba_bar >=3D PCI_STD_NUM_BARS || + !IS_ALIGNED(layout->table_offset, 8) || + !IS_ALIGNED(layout->pba_offset, 8) || + layout->table_offset > PCI_MSIX_TABLE_OFFSET || + layout->pba_offset > PCI_MSIX_PBA_OFFSET) + return -EINVAL; + + table_size =3D nr_irqs * PCI_MSIX_ENTRY_SIZE; + pba_size =3D BITS_TO_U64(nr_irqs) * sizeof(u64); + + if (layout->table_size < table_size || layout->pba_size < pba_size) + return -ENOSPC; + if (!epc->ops->set_msix) return 0; =20 mutex_lock(&epc->lock); - ret =3D epc->ops->set_msix(epc, func_no, vfunc_no, nr_irqs, bir, offset); + ret =3D epc->ops->set_msix(epc, func_no, vfunc_no, nr_irqs, layout); mutex_unlock(&epc->lock); =20 return ret; diff --git a/include/linux/pci-epc.h b/include/linux/pci-epc.h index f247cf9bcf1a..3d28231f092a 100644 --- a/include/linux/pci-epc.h +++ b/include/linux/pci-epc.h @@ -102,6 +102,24 @@ struct pci_epc_aux_resource { } u; }; =20 +/** + * struct pci_epc_msix_layout - layout of an MSI-X table and PBA + * @table_bar: BAR containing the MSI-X table + * @table_offset: offset of the MSI-X table within @table_bar + * @table_size: size of the MSI-X table region + * @pba_bar: BAR containing the MSI-X Pending Bit Array (PBA) + * @pba_offset: offset of the MSI-X PBA within @pba_bar + * @pba_size: size of the MSI-X PBA region + */ +struct pci_epc_msix_layout { + enum pci_barno table_bar; + u32 table_offset; + resource_size_t table_size; + enum pci_barno pba_bar; + u32 pba_offset; + resource_size_t pba_size; +}; + /** * struct pci_epc_ops - set of function pointers for performing EPC operat= ions * @write_header: ops to populate configuration space header @@ -147,7 +165,8 @@ struct pci_epc_ops { u8 nr_irqs); int (*get_msi)(struct pci_epc *epc, u8 func_no, u8 vfunc_no); int (*set_msix)(struct pci_epc *epc, u8 func_no, u8 vfunc_no, - u16 nr_irqs, enum pci_barno, u32 offset); + u16 nr_irqs, + const struct pci_epc_msix_layout *layout); int (*get_msix)(struct pci_epc *epc, u8 func_no, u8 vfunc_no); int (*raise_irq)(struct pci_epc *epc, u8 func_no, u8 vfunc_no, unsigned int type, u16 interrupt_num); @@ -381,8 +400,10 @@ void pci_epc_unmap_addr(struct pci_epc *epc, u8 func_n= o, u8 vfunc_no, int pci_epc_set_msi(struct pci_epc *epc, u8 func_no, u8 vfunc_no, u8 nr_ir= qs); int pci_epc_get_msi(struct pci_epc *epc, u8 func_no, u8 vfunc_no); int pci_epc_set_msix(struct pci_epc *epc, u8 func_no, u8 vfunc_no, u16 nr_= irqs, - enum pci_barno, u32 offset); + const struct pci_epc_msix_layout *layout); int pci_epc_get_msix(struct pci_epc *epc, u8 func_no, u8 vfunc_no); +int pci_epc_get_hw_msix_layout(const struct pci_epc_features *epc_features, + struct pci_epc_msix_layout *layout); int pci_epc_map_msi_irq(struct pci_epc *epc, u8 func_no, u8 vfunc_no, phys_addr_t phys_addr, u8 interrupt_num, u32 entry_size, u32 *msi_data, u32 *msi_addr_offset); --=20 2.51.0 From nobody Sun Sep 27 00:39:56 2026 Received: from OS0P286CU010.outbound.protection.outlook.com (mail-japanwestazon11021112.outbound.protection.outlook.com [40.107.74.112]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 30792499F17; Thu, 27 Aug 2026 18:20:24 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.74.112 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1787854826; cv=fail; b=aiW/FW6vp3TaP0lBtYRzHlFA6hBToIhDF66Ycc7+5+NrnlLLkkllOzOcTXNQpP0XfK+VOJly9VztJvlAvK4LRMzsoqCKxiw/TknIqsfyKro9jlD1XUgL/34tPKTdbkyzTCW5B+FCo4x3tXRjh4otmDdFRPP1ONBz/rYYQL/w8kk= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1787854826; c=relaxed/simple; bh=gSEV84/4vIvOBMKd4OvjxLQ9yjY+Y+YjfuL2M8EkUtA=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: Content-Type:MIME-Version; b=f7LSw+XlY0ICAOL2HtSn0F7faX570y952XK7JUdY5gNjZ9PU1VUcd7zs5QzfB8mM30vneq9gwvAS0365eBGpH2nwHUoXqJymejbHSrTqCxntotjo6lEZj5WkfU4jCBcpauqwyZnm7d8i170vZY4V4pCi7bgYYol1IwtCrUIshyM= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=valinux.co.jp; spf=pass smtp.mailfrom=valinux.co.jp; dkim=pass (1024-bit key) header.d=valinux.co.jp header.i=@valinux.co.jp header.b=q/COslAs; arc=fail smtp.client-ip=40.107.74.112 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=valinux.co.jp Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=valinux.co.jp Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=valinux.co.jp header.i=@valinux.co.jp header.b="q/COslAs" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=eq559rzPigeeZU5y2uxvzMiUUNuvzNRzaLUx3eXLGO1bLVhfTSyEVysKHktRwm0bQtSqxzeMGU0488kVpgdfqVxvor5buQGyS4G5HhjIt2rYTueOCJtEUNbsDWvr7X6BK4C357m/92+kP2ZxpIItEqfDbkXvMlbLbyuV4C7gKDDhSMYOdc9EYgqBTG3jubVINdFvXWfQSu861cUW8r7ss41GjsqvpLUdFLBk9q7I8rsZyl9fLZlKcAw3mipNBI0J8dSrDawXA+0WSXa8+tDLP7BQCPrBPwtlG9vvSdcg9eP+1RbPdKNMAd/TeBjypxQXPiP6S8pYeWZk8ROaXRMUDA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=Adj1PhWejpLIjLcRuT8yyvBm2g+bncsxMaqqrV/lZIA=; b=DetJA3BqMspZN3LB4JAf9RDT34ntuAZzxiWKfUGx6kw67rBpgiPHGpazDtmRIEab1b8b2LOWnS66YwV9YgIbUvtqpZwL5356ydHrpgoWL9CHLFAk3ToxyCAfkLgAoiyPH+89paCXHxmPn5D5JHALzrqb13YaoaoiCfQxDC/jqa+DwiiXpYN9l6whXGEuBMQMe9/evif+wsgDiZOuhDCZ0hv1Cm1RyOAHnuhZIvt5Jn/sKfpOs5WSTGrDX844MTc1DZU+LT72jondn1JqPoUcL3gNgTMqHDAyiDwke4fOFsWZigBlExLabdBX1APoBY/4BG5KaR+0ZsI/hWaFfARoCQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=valinux.co.jp; dmarc=pass action=none header.from=valinux.co.jp; dkim=pass header.d=valinux.co.jp; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=valinux.co.jp; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Adj1PhWejpLIjLcRuT8yyvBm2g+bncsxMaqqrV/lZIA=; b=q/COslAslbdcofgxRY+pu1LE30mBMBjokib4rnbZVupBVcMlQPd/k006rabZQBXGsodbhoXJvBfS2xUMD3jaCBVv5RjSuxNp3uPN5dwYtDiQ6dfNljxwdZx5U9wgFJFXc7P5IQv/am5eD/qYX07onDXlVZrhVcqRK6T/gLLEQZo= Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=valinux.co.jp; Received: from TY7P286MB7722.JPNP286.PROD.OUTLOOK.COM (2603:1096:405:38f::10) by TY7P286MB7058.JPNP286.PROD.OUTLOOK.COM (2603:1096:405:31d::13) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.360.10; Thu, 27 Aug 2026 18:20:19 +0000 Received: from TY7P286MB7722.JPNP286.PROD.OUTLOOK.COM ([fe80::2305:327c:28ec:9b32]) by TY7P286MB7722.JPNP286.PROD.OUTLOOK.COM ([fe80::2305:327c:28ec:9b32%4]) with mapi id 15.21.0360.008; Thu, 27 Aug 2026 18:20:19 +0000 From: Koichiro Den To: Manivannan Sadhasivam , =?UTF-8?q?Krzysztof=20Wilczy=C5=84ski?= , Kishon Vijay Abraham I , Frank Li , Bjorn Helgaas , Jingoo Han , Niklas Cassel Cc: Lorenzo Pieralisi , Rob Herring , Aksh Garg , Christoph Hellwig , Sagi Grimberg , Chaitanya Kulkarni , Jon Mason , Dave Jiang , Allen Hubbe , Heiko Stuebner , Shawn Lin , Manikanta Maddireddy , Shin'ichiro Kawasaki , linux-pci@vger.kernel.org, linux-nvme@lists.infradead.org, ntb@lists.linux.dev, linux-rockchip@lists.infradead.org, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org Subject: [PATCH 2/3] PCI: dw-rockchip: Support fixed MSI-X table and PBA on RK3588 Date: Fri, 28 Aug 2026 03:20:11 +0900 Message-ID: <20260827182012.1984960-3-den@valinux.co.jp> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20260827182012.1984960-1-den@valinux.co.jp> References: <20260827182012.1984960-1-den@valinux.co.jp> Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: TY4P301CA0070.JPNP301.PROD.OUTLOOK.COM (2603:1096:405:36a::8) To TY7P286MB7722.JPNP286.PROD.OUTLOOK.COM (2603:1096:405:38f::10) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: TY7P286MB7722:EE_|TY7P286MB7058:EE_ X-MS-Office365-Filtering-Correlation-Id: cbd3733c-38f5-4126-ec2e-08df0467d9a1 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|1800799024|7416014|376014|23010399003|10070799003|366016|6133799003|10067099003|56012099006|22082099003|18002099003; X-Microsoft-Antispam-Message-Info: zh4QOejTjREtvY9k+AIV3gmWynGRmclPZM67f7Wr34JFv02kcvtDRr1bHnZNTx3KJgUq6UeU20xlCuP4qLx1RcKxLBPSxYUq3WoY7pvVkoAqv9WajP7QcBchioJiRHdC3W6jrXmCKRPC8cAgWOkmGgA54HKh4k6zovyO7QjZelFXhnnqsN3gXts3S7Vp49LQIZEnt0AxJxScMs9LZNwTSMD/G+av0IB2OJ7+bkcSIzsYfPW102DecatZY18djg4h1j1yvgzMyH7b+D0e8cVG90TjP/N+NJZVoanL/EsnVEeRpKZY+LIWCwEtzcG1hG/2MLJ5GWhyk4sajxYjOOXStN/lBcqnCO0iVBhYlmy/VKmRbCFAOQWGqHqQq6oIAhFqCRilyBtEzh/6vbHDpGnDwPIZ8tiMNoWmQjUwbt+A8YnNNpE8751QKrJjc+tm2hHtgWcNVCnuFTkv6p1ikoGBIRL5gInvvo01dDkW0+y/BC3i3Ra8MX/y5yo8EOhCKE3frXgSOKaxDdCotg6n5Pzfwd2pLptVad+B6gGF2g+98EsHB9sDmCLaVHO/Udv5JSsGv5XIxfTtd2U2604vhbKjuoMGFJyLaPkDxfI09LtpHyjb99EeOQJ47uiJw2ekvmIHMePRtW1uxg4qTHDUuhWkZ39HeGoNYm00oF83YJpTuiw= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:TY7P286MB7722.JPNP286.PROD.OUTLOOK.COM;PTR:;CAT:NONE;SFS:(13230040)(1800799024)(7416014)(376014)(23010399003)(10070799003)(366016)(6133799003)(10067099003)(56012099006)(22082099003)(18002099003);DIR:OUT;SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?kAlar234Wmg5xy7rSn9XCtO8e/b6mM7siNYQewX3zayUPhSSVWCMBX1LxHfY?= =?us-ascii?Q?FnXk7NhNgnkJHthOhGaMM3AH86y6plgNysvUwM+/9Dkw239Q4VqtMT/EOLJ2?= =?us-ascii?Q?yEv8Ct/bcJ83r1Q2ewG8kVZjHtjEoNaem4rZRtb7scnnm4iEpoTDJpFntzpc?= =?us-ascii?Q?fGfTOr+nDnKqLiOTe46SLAMbTffyKJUjN5C50KzOkazS9p1r6A0/ZmsvNXQ2?= =?us-ascii?Q?i0JWIJ+60aROIVF1wAoi6vFzwMXZLQGjfkthAwStiyRC6plDzJlGq0wqPPZa?= =?us-ascii?Q?sxAo9jOLPVkV3qeWxRSGOmyddmWeyjDcM7Ac84dJskOh0sh6LwSgxCsnLYnU?= =?us-ascii?Q?0jlh5N8wL+Sn7gwv0qGL5VAnCfy12hsd1t0VA6vDYz213lWaqA5nl9OXLs7S?= =?us-ascii?Q?HrNJJk+dnaaqJhsXuwEDxk8RYEagGfF/45Ri/8gv0wq7dFRk9ChU1Gf4aYyI?= =?us-ascii?Q?1g1Qm23y+x1MbKIif5MrBOyNeM2w2di2NmGQm46j8hncc1y1jDJDIKMh/Y/8?= =?us-ascii?Q?31dpdbCgSaalXFbIemlloepeh/ptnW1A22sXPNABESzmozCBj4yiCyWBekJQ?= =?us-ascii?Q?UNEwZfpQKRGOJOFY3o7tXgEEfC415b4k28G0iRdYYxoEJohdJLoB0vP8jHAQ?= =?us-ascii?Q?hLN5844UNgQuBndy76LhOlbr9aMoqSpPA6uP3VaC+E1997CTXqEktkkkF656?= =?us-ascii?Q?wYO8a7zgVOzu/LBAZnvK+d++wEWDIdn45OH5X+WAr4h4QaVHtXn+PS/OsqB6?= =?us-ascii?Q?u7wU+xTFfiCjwrtaWpzLZ9QKnTAigqi7d5KiGho9uzJaBCD268SWVKs3Ngke?= =?us-ascii?Q?9qnk/nQUMXg1RcJbEeCkd9QE/56OpbMWSMskDoKwVq3XzKmQ/o03evxjASgW?= =?us-ascii?Q?fjFoN2hN9HeBR/NHtfXFXI8TqD9bgglheBFJshgVymiHoQatXcWILDoZbR/O?= =?us-ascii?Q?WiPs+/sBPPRnuNLXG/iKE8CoaXrwHGY5QQXTRCGhMP/NGxMeRcL7Zfc+nA1P?= =?us-ascii?Q?t6GXdLdz2hn9Z5mLGWZLNdlepKFN1YUf6OhayQkhTTGjDTf/kAJCF69OHmKp?= =?us-ascii?Q?P4Hu8ONpx1wV4N84gbeSkdp5MKGVM88TVncgi5mEZboLRGQkXt7rMfOdon21?= =?us-ascii?Q?aj95LQvGTg5fZBnXhQ777X06HFfsVnDmw3Z4/utjLfRWFi5qJb/EFj8S1rpM?= =?us-ascii?Q?RBFN+BOUAiSaxdnkVCMDa7AENpvD8Zus+OU2pCLXafylP9aDsGsEz/fSu6RR?= =?us-ascii?Q?cptnm51d8OB7J51sqOk9SA/6mHNGKM14CzRVtLsGOg8x8YGP/eeiWn8qx93q?= =?us-ascii?Q?XPOyfZbJZ6pV/qRKDCeOE3flPK+ADwv+CzSmxFRfoNVLyIMCMzpYslXHklUv?= =?us-ascii?Q?/11fH5Ne2FIGlFQeRPgpAwLWm7VrpKd9aUWNkxc0UuYWMkYNmG4NuFnP6haF?= =?us-ascii?Q?C1aW6nAdEQank5MEXuZf+N77M827DOWCNzPclAgO6FR3xPvMQmiMGTkOGEB7?= =?us-ascii?Q?uJ82fG4KhSwQLtCKJgbF2wUPVXBRx/KoqtFEOYD8qgxnnMm3Wh3NAIKPuZIS?= =?us-ascii?Q?nvdxYXzMYUqTu6x/zcBTXT8k1zNLYIuATdYDL/KjeLPmLSaxH1VcF4W0WJuY?= =?us-ascii?Q?frTTIafdeF+46mk9dg3zsniqjw24vvqQPadbr6gJiFghWYUTEmnTVqQRE7w9?= =?us-ascii?Q?qR0pxwAntaJsbdhpboFMDpG+mtQY8x+smRBD9qPCWbQqR4uJGxKyPgZ1Sw8/?= =?us-ascii?Q?M4gjiYnnvzx2u5+9hG6whY+1sGMYlwv5zHToh5dWWJvk/8xcsVwk?= X-OriginatorOrg: valinux.co.jp X-MS-Exchange-CrossTenant-Network-Message-Id: cbd3733c-38f5-4126-ec2e-08df0467d9a1 X-MS-Exchange-CrossTenant-AuthSource: TY7P286MB7722.JPNP286.PROD.OUTLOOK.COM X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 27 Aug 2026 18:20:18.9986 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 7a57bee8-f73d-4c5f-a4f7-d72c91c8c111 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: P7W3R9kie6C8O/d8rh/6NZ7A6Uc/SrdOKeDtvM9GnBsC8pZmT3Aroo7WigtTHJCGdj6BH8E4309IggKGUgIQHw== X-MS-Exchange-Transport-CrossTenantHeadersStamped: TY7P286MB7058 Content-Type: text/plain; charset="utf-8" The RK3588 PCIe endpoint controller exposes its MSI-X table at BAR4+0x4000 and its PBA at BAR4+0x5000. Describe both as reserved regions so EPF drivers can select the hardware-owned layout. The regular DesignWare MSI-X interrupt path reads an EPF-owned table through epf_bar[]. A reserved, hardware-owned table has no such backing. Record whether the layout selected for each function matches the hardware-owned layout, and use the controller MSI-X doorbell on RK3588 only in that case. Continue to use the regular path for EPF-owned layouts. Signed-off-by: Koichiro Den --- .../pci/controller/dwc/pcie-designware-ep.c | 21 ++++++++++++++ drivers/pci/controller/dwc/pcie-designware.h | 1 + drivers/pci/controller/dwc/pcie-dw-rockchip.c | 28 +++++++++++++++++-- 3 files changed, 47 insertions(+), 3 deletions(-) diff --git a/drivers/pci/controller/dwc/pcie-designware-ep.c b/drivers/pci/= controller/dwc/pcie-designware-ep.c index 147b043589f0..bc2b63814dea 100644 --- a/drivers/pci/controller/dwc/pcie-designware-ep.c +++ b/drivers/pci/controller/dwc/pcie-designware-ep.c @@ -736,6 +736,26 @@ static int dw_pcie_ep_get_msix(struct pci_epc *epc, u8= func_no, u8 vfunc_no) return val + 1; } =20 +static bool +dw_pcie_ep_msix_layout_is_hw_owned(struct dw_pcie_ep *ep, + const struct pci_epc_msix_layout *layout) +{ + const struct pci_epc_features *features; + struct pci_epc_msix_layout hw_layout; + + if (!ep->ops->get_features) + return false; + + features =3D ep->ops->get_features(ep); + if (pci_epc_get_hw_msix_layout(features, &hw_layout)) + return false; + + return layout->table_bar =3D=3D hw_layout.table_bar && + layout->table_offset =3D=3D hw_layout.table_offset && + layout->pba_bar =3D=3D hw_layout.pba_bar && + layout->pba_offset =3D=3D hw_layout.pba_offset; +} + static int dw_pcie_ep_set_msix(struct pci_epc *epc, u8 func_no, u8 vfunc_n= o, u16 nr_irqs, const struct pci_epc_msix_layout *layout) @@ -764,6 +784,7 @@ static int dw_pcie_ep_set_msix(struct pci_epc *epc, u8 = func_no, u8 vfunc_no, reg =3D ep_func->msix_cap + PCI_MSIX_PBA; val =3D layout->pba_offset | layout->pba_bar; dw_pcie_ep_writel_dbi(ep, func_no, reg, val); + ep_func->msix_hw_owned =3D dw_pcie_ep_msix_layout_is_hw_owned(ep, layout); =20 dw_pcie_dbi_ro_wr_dis(pci); =20 diff --git a/drivers/pci/controller/dwc/pcie-designware.h b/drivers/pci/con= troller/dwc/pcie-designware.h index de4b245b1758..043c39b5881d 100644 --- a/drivers/pci/controller/dwc/pcie-designware.h +++ b/drivers/pci/controller/dwc/pcie-designware.h @@ -495,6 +495,7 @@ struct dw_pcie_ep_func { u8 func_no; u8 msi_cap; /* MSI capability offset */ u8 msix_cap; /* MSI-X capability offset */ + bool msix_hw_owned; u8 bar_to_atu[PCI_STD_NUM_BARS]; struct pci_epf_bar *epf_bar[PCI_STD_NUM_BARS]; =20 diff --git a/drivers/pci/controller/dwc/pcie-dw-rockchip.c b/drivers/pci/co= ntroller/dwc/pcie-dw-rockchip.c index 731d93663cca..d622723dfcf8 100644 --- a/drivers/pci/controller/dwc/pcie-dw-rockchip.c +++ b/drivers/pci/controller/dwc/pcie-dw-rockchip.c @@ -120,6 +120,7 @@ struct rockchip_pcie { struct rockchip_pcie_of_data { enum dw_pcie_device_mode mode; const struct pci_epc_features *epc_features; + bool msix_doorbell; }; =20 static int rockchip_pcie_readl_apb(struct rockchip_pcie *rockchip, u32 reg) @@ -481,6 +482,8 @@ static int rockchip_pcie_raise_irq(struct dw_pcie_ep *e= p, u8 func_no, unsigned int type, u16 interrupt_num) { struct dw_pcie *pci =3D to_dw_pcie_from_ep(ep); + struct rockchip_pcie *rockchip =3D to_rockchip_pcie(pci); + struct dw_pcie_ep_func *ep_func; =20 switch (type) { case PCI_IRQ_INTX: @@ -488,6 +491,12 @@ static int rockchip_pcie_raise_irq(struct dw_pcie_ep *= ep, u8 func_no, case PCI_IRQ_MSI: return dw_pcie_ep_raise_msi_irq(ep, func_no, interrupt_num); case PCI_IRQ_MSIX: + ep_func =3D dw_pcie_ep_get_func_from_ep(ep, func_no); + if (rockchip->data->msix_doorbell && ep_func && + ep_func->msix_hw_owned) + return dw_pcie_ep_raise_msix_irq_doorbell(ep, func_no, + interrupt_num); + return dw_pcie_ep_raise_msix_irq(ep, func_no, interrupt_num); default: dev_err(pci->dev, "UNKNOWN IRQ type\n"); @@ -517,12 +526,24 @@ static const struct pci_epc_bar_rsvd_region rk3588_ba= r4_rsvd[] =3D { .offset =3D 0x0, .size =3D 0x2000, }, + { + /* MSI-X Table (BAR4: MSI-X Table) */ + .type =3D PCI_EPC_BAR_RSVD_MSIX_TBL_RAM, + .offset =3D 0x4000, + .size =3D SZ_4K, + }, + { + /* MSI-X PBA (BAR4: MSI-X PBA) */ + .type =3D PCI_EPC_BAR_RSVD_MSIX_PBA_RAM, + .offset =3D 0x5000, + .size =3D SZ_4K, + }, }; =20 /* - * BAR4 on rk3588 exposes the ATU Port Logic Structure to the host regardl= ess of - * iATU settings for BAR4. This means that BAR4 cannot be used by an EPF d= river, - * so mark it as RESERVED. + * BAR4 on RK3588 exposes the DMA and ATU Port Logic Structures and the MS= I-X + * table and PBA to the host regardless of iATU settings for BAR4. This me= ans + * that BAR4 cannot be used by an EPF driver, so mark it as RESERVED. */ static const struct pci_epc_features rockchip_pcie_epc_features_rk3588 =3D= { DWC_EPC_COMMON_FEATURES, @@ -841,6 +862,7 @@ static const struct rockchip_pcie_of_data rockchip_pcie= _ep_of_data_rk3568 =3D { static const struct rockchip_pcie_of_data rockchip_pcie_ep_of_data_rk3588 = =3D { .mode =3D DW_PCIE_EP_TYPE, .epc_features =3D &rockchip_pcie_epc_features_rk3588, + .msix_doorbell =3D true, }; =20 static const struct of_device_id rockchip_pcie_of_match[] =3D { --=20 2.51.0 From nobody Sun Sep 27 00:39:56 2026 Received: from OS0P286CU010.outbound.protection.outlook.com (mail-japanwestazon11021112.outbound.protection.outlook.com [40.107.74.112]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8FF994C6F1E; Thu, 27 Aug 2026 18:20:26 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.74.112 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1787854828; cv=fail; b=qX6ZCO+UhHeBwAOPEtVhNv0ZJW7aNTLqJj0LKNLPh0IfVJaMHNm92MHaqhdLjOdKRF/5cMVTk7mGUd+PFXMDQAEPBAPzaa7Ha1CYdQrrx5nwnXXZ6ngg9UmzIaXlWc3IHiwmNieEce/P/BJ3HSRzZa5BXNMBebgwrFO1stKKkLs= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1787854828; c=relaxed/simple; bh=A6SuPD/G58XP3pEsWBX9gkn9tiyUSBeOHpQy9q+mkxg=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: Content-Type:MIME-Version; b=jeivVMn2U6tBH5+rCOfd+7PEqRzq5QDIGG13BTUIaUHHoLIq+9F71mqzUsSGNKoeUB32TbZ7RsKMRym9VRu4XcScByEACHCiFNrejTwFcoopj4JfuO9c20xePDm3qW9zEPrAwk/Lb0M3lJOfM/7/TtsshAXdJnnUB/VqYthYKs0= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=valinux.co.jp; spf=pass smtp.mailfrom=valinux.co.jp; dkim=pass (1024-bit key) header.d=valinux.co.jp header.i=@valinux.co.jp header.b=LDJ2wCVc; arc=fail smtp.client-ip=40.107.74.112 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=valinux.co.jp Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=valinux.co.jp Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=valinux.co.jp header.i=@valinux.co.jp header.b="LDJ2wCVc" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=cN6PAQnLJXYG7vTfsNJqAm9Xg/cpJal2JhxyCH8K5OOupI5jt0YGwaywxVfbAykbga940RZTAm94MANbTkxa9FCP+Hh+VqMpMuC/3kN2WFgD+omffVgzICLMBu9obaft6onzmG/xEi5b7hU6pkIyheamhr/wBsnKRVGhz7exjuH4k0Fc9/7A3EDgj5UF9lKXOKB9v/oOiEAznyt9Kgmnv+mpqzfLnDNXL4ociZCX52qiZtD0fu6jmIIdSuppeFDAnJKFMITUQOswPSvQ56zExo/GjFxzrwWZyUu4TQU3J8pSFDmVvI5rP5lyoMjSCTs9Sc9AI+jnO9v9SsDNrF7DLQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=nuQoURfAXs9A5j0AqOjbcqJ/uEhP2cikdtEm7rYNDss=; b=RCm/YwB6i8xLFC5JeF768qjZz8BxAE0xZhbQehk/9+AmG9I2eHcr++w4lezkbM8bc6aOZ6BMKBQVLoV3qQ+1GsGnkO6wwEnB4uiYIFzCJ+wxMrPCztBnSZiDVx00HWzIkCOAX7eT5hy24v071dLi8KmEmiieklK9+m6371dsl6omyLirpPaaMvdXUnkBbf4d4kBLD7xZBDRpdreo3gjclnJaBafMgYQfACYZahNZpzgxvfdKFdUTU9+jPYxhE352CqEDGc5DbnOs0Q1IdKIve37+tEshQwAGeVOsFJ9i/wFDPC76gEbmYlYREpzTNmxHxVOszPHXwC3vA20tHauPqA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=valinux.co.jp; dmarc=pass action=none header.from=valinux.co.jp; dkim=pass header.d=valinux.co.jp; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=valinux.co.jp; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=nuQoURfAXs9A5j0AqOjbcqJ/uEhP2cikdtEm7rYNDss=; b=LDJ2wCVc1OqSXxdOkVj8bGxNpR/ZHPdZcLSLXBlUZj/fVNpNCepS8ZvgXzl/z59OWw8KuVagb9HiQlPpuT/f7QpfhU6p6uXSV2mdM5vIef4n9qmvD23Atdwi0sQY/pgcosoZE1aWof5GZV9jVckm4PvhMDUxu6aBJE6JzJo1/jA= Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=valinux.co.jp; Received: from TY7P286MB7722.JPNP286.PROD.OUTLOOK.COM (2603:1096:405:38f::10) by TY7P286MB7058.JPNP286.PROD.OUTLOOK.COM (2603:1096:405:31d::13) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.360.10; Thu, 27 Aug 2026 18:20:19 +0000 Received: from TY7P286MB7722.JPNP286.PROD.OUTLOOK.COM ([fe80::2305:327c:28ec:9b32]) by TY7P286MB7722.JPNP286.PROD.OUTLOOK.COM ([fe80::2305:327c:28ec:9b32%4]) with mapi id 15.21.0360.008; Thu, 27 Aug 2026 18:20:19 +0000 From: Koichiro Den To: Manivannan Sadhasivam , =?UTF-8?q?Krzysztof=20Wilczy=C5=84ski?= , Kishon Vijay Abraham I , Frank Li , Bjorn Helgaas , Jingoo Han , Niklas Cassel Cc: Lorenzo Pieralisi , Rob Herring , Aksh Garg , Christoph Hellwig , Sagi Grimberg , Chaitanya Kulkarni , Jon Mason , Dave Jiang , Allen Hubbe , Heiko Stuebner , Shawn Lin , Manikanta Maddireddy , Shin'ichiro Kawasaki , linux-pci@vger.kernel.org, linux-nvme@lists.infradead.org, ntb@lists.linux.dev, linux-rockchip@lists.infradead.org, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org Subject: [PATCH 3/3] PCI: endpoint: pci-epf-vntb: Honor MSI-X selection Date: Fri, 28 Aug 2026 03:20:12 +0900 Message-ID: <20260827182012.1984960-4-den@valinux.co.jp> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20260827182012.1984960-1-den@valinux.co.jp> References: <20260827182012.1984960-1-den@valinux.co.jp> Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: TY4P301CA0062.JPNP301.PROD.OUTLOOK.COM (2603:1096:405:36a::13) To TY7P286MB7722.JPNP286.PROD.OUTLOOK.COM (2603:1096:405:38f::10) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: TY7P286MB7722:EE_|TY7P286MB7058:EE_ X-MS-Office365-Filtering-Correlation-Id: 767b165d-8e99-4854-2961-08df0467da29 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|1800799024|7416014|376014|23010399003|10070799003|366016|6133799003|10067099003|56012099006|5023799004|22082099003|18002099003; X-Microsoft-Antispam-Message-Info: 1/u3Su6XtZGCUL943Dkk90/z5sN4Gm6VnMMmas8DpKYpc4QqgOtplnpfwZ0xAwl3dMRFnb0205LjmFeCb0mhHZmVMZ/7O4xvIu0Gq2GFQm+nhRfiZR9ARm/B++oq2hZl2mP2DvWG/1N50rjdrmEjdZZbSVvWiwMOGwV72N5M2rw5pSTstWxqqjdEd8yFn9Qf/XayUXCh/GaFvrCc519vj5FW2ipKHWaCuDgjCweepSwGT8hqJwmEdn8IeI1/P1mmtDdtBdhAEmtI7W2PD277kKEHS8MZ+RzQljNM7LLMQBqMvewkd4UJGiIIaAx0zQPvLUZP2AZQRyiw34fywEWcvYv7UHs1UvJACCTj2SE3fDN5Puq/KxZHytp69KH1UPTHd7R2WcljG0bfci6PrEBd8M7XxLWy5ZPd4l1t2ReoTu4rBn97NXxFwOQbv3MCfp9glsGPNGcQNitiVzugrXUhwg0qW9wXr3Ve3ncpp4AkraMasn4dyBtTG5WEy9m7YcUrGFcjQSSWcNTG5R5proy2dFPUwHh5A/sXnUq6z8TLRBeawi9o2uK4PVZlx5C/mKFWI+20rofxct972tyN6MpqXervlJyWS0Wi/cwb0mqexiKjvlpwJ6tP4mRJzb7UZiuulhHHJwgobhmtlXGXjW+VZUs1L1enyu7Qas1uEBlqwOU= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:TY7P286MB7722.JPNP286.PROD.OUTLOOK.COM;PTR:;CAT:NONE;SFS:(13230040)(1800799024)(7416014)(376014)(23010399003)(10070799003)(366016)(6133799003)(10067099003)(56012099006)(5023799004)(22082099003)(18002099003);DIR:OUT;SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?GHzg8pCWiJgjweSBvsTYhvji8HHDPH90XYAADlBs2lWdBokaTiMJDK/2K2K8?= =?us-ascii?Q?JghxJexBwzHNfJ2VSLw1o/pnrt1Eyr/jU4T+y/ijFduL5DcQ4zhSjLCoPgGw?= =?us-ascii?Q?SLLaRbTdHFA5jjyT7C+V26EGJJrxYL15d0/GhOOlESQfU/BDFuE78XCkaDHv?= =?us-ascii?Q?txt2Md2h+FKqttnvV8gqpJV3j7c3QV9KxAk0MQm067GvHQkRl96sQqbt9RZf?= =?us-ascii?Q?cMijG6yRkuKU9MpAfkfybpaMKH1aMV5NvUgjLkjxMd2iVBm62iPe52RCNzwX?= =?us-ascii?Q?sM0jkHEXURxGnjIlfYuF7YdzH6YD6XobIjYcnTAXjjIh9D8ThqB6WGgmpDaQ?= =?us-ascii?Q?ZnI3vj4wINi+N6zg0GiIWEXBS6gFKPLI67WPUdaS9XleXxWFZ2CMZRnAMNJr?= =?us-ascii?Q?07FoSxJlxyG7OgXntrr6YCE+sxmnXKN3aYGsrZH6BsohA0r4c28atci9fpzy?= =?us-ascii?Q?GFYo6kvHc97g6lNRoSTlRq+HPoH1zmSAp/wZ51I9HZgITGFP8jfpr5bIcMGv?= =?us-ascii?Q?VpnsKfLOey8yy7BkM4Leh/IAQfn6fqa8wyMUuPwSRSCelVJA3vn1b+QvYR6r?= =?us-ascii?Q?XwOivgQzJH6j0+kjG9inlVjiq4yqd9BX/s7j2rWj02N1WVrXWq2doKkh7AUO?= =?us-ascii?Q?dOMHced7mp28xfYHf/SrrCl4yS8UtxqjPaAReley6ZShTq3c1ZsEW2NAZ7EW?= =?us-ascii?Q?pt/iXsyHb73NoJ2erQHfnLXnGOgs/M1MCaj3vuKfilbsDsJ0e/xNPiC0lZOf?= =?us-ascii?Q?MFcHH6teE6lwly4v00maqZefN6yyGQAQDthGp1Va0NkRabGHSh4ze2aqfbTi?= =?us-ascii?Q?uVk7rZF3sp9/vaA80LUSCM8nwQDLugDFXPTsdECMwlXBNVymL3a1bNBOilhb?= =?us-ascii?Q?WUQpVxHnVbh5ZFr/dZaHWCI1n7TvxA35X1xpg9p25SpYbnYNxUtzlKFsuQXU?= =?us-ascii?Q?1WXjxvNN5B/+rxFEsEjixh7e/ugGG05Dwt2XlyyAifWrHVJ68zlUnUQeLnbT?= =?us-ascii?Q?jQB57jSaSA4w2Rd1HbMa8PWikdS+gqaAfYfe7CVamsJ0FexgTh4FDdI8YB9E?= =?us-ascii?Q?ESxUpcAefKCuUnx3V1uqcFJxgGXR0fGTtxpi8SwUuXRkG3dqqgL+8mXI7ot8?= =?us-ascii?Q?nCfKjOVMQ6R3GEKPnz/gEATrWPtA7JV5wZ03Ypy9W7I87Jda2LDhHDLl5Aw9?= =?us-ascii?Q?P/+loA94WbEn22WoJ3gUWGLrPNkcq0G3QScsr3PIE5hgZD5b+6GHpZUTomc+?= =?us-ascii?Q?ItG3nTHuSQrOsu+MuRhRU+DJoHgA/JfIRsjQUe8Z84aSegokYZWtICis4MBk?= =?us-ascii?Q?e/G+s33ir5Kq1AYBiadNe8tKfPv5JjcSK1aRzU/95QQ/ciCkBRvR3uNfmFbs?= =?us-ascii?Q?/B05TL7PXDMRbg1xRJH0K+8I2g9PaWfaZFs9WHyB4q+ksuAlNkZUmfQrhhs4?= =?us-ascii?Q?BNMKKJkWufeMMrX9IE3O9C/tuZNQbY9FJGsO7xZs55JN8P+PDKCHpg08hNft?= =?us-ascii?Q?Np4N0PDpY7L65V0N5gDOpwYtGNsbg7wrqOVIw/Ckuye+Swv7Jr6cTh8w/9Xd?= =?us-ascii?Q?+sd8awpx1HDnXEv1tegl0AGNoGjMxYUBVkaOYQXjsvHSSqnmPmsSdAEHLnET?= =?us-ascii?Q?SFaYyxH31Gzq1jkeTD9oNNK3xmPEC+2pYP0R2xqXXkDB0d2qYBn0xW/aHak0?= =?us-ascii?Q?3WuCQhdUXee0GfTGINplLfa74JGXy8MmKkcp9cTXfJSwEzRCl2JS/686/EDv?= =?us-ascii?Q?NCZeTK2WdaypgC3p+ayXU4KBfvie5m1nrscKbcZ/c8naQX+D6z82?= X-OriginatorOrg: valinux.co.jp X-MS-Exchange-CrossTenant-Network-Message-Id: 767b165d-8e99-4854-2961-08df0467da29 X-MS-Exchange-CrossTenant-AuthSource: TY7P286MB7722.JPNP286.PROD.OUTLOOK.COM X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 27 Aug 2026 18:20:19.9167 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 7a57bee8-f73d-4c5f-a4f7-d72c91c8c111 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: Qfya+bfu86m+Mn/Cdq2Zg3/x1fdSTrAmMqmwSUmDPnRxBjJJc9BdADPLIO02DWAPFds+yvbYgKEx7AolzebFDQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: TY7P286MB7058 Content-Type: text/plain; charset="utf-8" ntb_hw_epf tries MSI-X first and falls back to MSI. It reports the result in COMMAND_CONFIGURE_DOORBELL. pci-epf-vntb ignores MSIX_ENABLE, configures only MSI, and always raises peer doorbells with PCI_IRQ_MSI. When MSI-X is selected, the host does not program MSI, so raising it can issue a write to an invalid address. This was observed with an IOMMU enabled on the RC. Configure MSI-X when supported and use the selected type for peer doorbells. Use the hardware-owned layout when available. Otherwise allocate an EPF-owned Table and PBA in the config BAR. Configure db_count entries to cover the link event, the reserved slot, and the doorbell slots. Fixes: e35f56bb0330 ("PCI: endpoint: Support NTB transfer between RC and EP= ") Signed-off-by: Koichiro Den Reviewed-by: Frank Li --- drivers/pci/endpoint/functions/pci-epf-vntb.c | 67 ++++++++++++++----- 1 file changed, 51 insertions(+), 16 deletions(-) diff --git a/drivers/pci/endpoint/functions/pci-epf-vntb.c b/drivers/pci/en= dpoint/functions/pci-epf-vntb.c index c3caec927d74..d9622a5d4710 100644 --- a/drivers/pci/endpoint/functions/pci-epf-vntb.c +++ b/drivers/pci/endpoint/functions/pci-epf-vntb.c @@ -148,6 +148,7 @@ struct epf_ntb { u16 vntb_vid; =20 bool linkup; + bool peer_msix; =20 /* * True when doorbells are interrupt-driven (MSI or embedded), false @@ -155,6 +156,7 @@ struct epf_ntb { */ bool msi_doorbell; u32 spad_size; + struct pci_epc_msix_layout msix_layout; =20 enum pci_barno epf_ntb_bar[VNTB_BAR_NUM]; =20 @@ -303,6 +305,7 @@ static void epf_ntb_cmd_handler(struct work_struct *wor= k) =20 switch (command) { case COMMAND_CONFIGURE_DOORBELL: + WRITE_ONCE(ntb->peer_msix, argument & MSIX_ENABLE); ctrl->command_status =3D COMMAND_STATUS_OK; break; case COMMAND_TEARDOWN_DOORBELL: @@ -439,9 +442,9 @@ static void epf_ntb_config_spad_bar_free(struct epf_ntb= *ntb) * region * @ntb: NTB device that facilitates communication between HOST and VHOST * - * Allocate the Local Memory mentioned in the above diagram. The size of - * CONFIG REGION is sizeof(struct epf_ntb_ctrl) and size of SCRATCHPAD REG= ION - * is obtained from "spad-count" configfs entry. + * Allocate the control and scratchpad regions described in the above diag= ram. + * If the EPC does not provide a hardware-owned MSI-X table and PBA, alloc= ate + * space for them between the control and scratchpad regions. * * Returns: Zero for success, or an error code in case of failure */ @@ -454,7 +457,7 @@ static int epf_ntb_config_spad_bar_alloc(struct epf_ntb= *ntb) struct device *dev =3D &epf->dev; u32 spad_count; void *base; - int i; + int i, ret; const struct pci_epc_features *epc_features =3D pci_epc_get_features(epf-= >epc, epf->func_no, epf->vfunc_no); @@ -462,6 +465,29 @@ static int epf_ntb_config_spad_bar_alloc(struct epf_nt= b *ntb) spad_count =3D ntb->spad_count; =20 ctrl_size =3D ALIGN(sizeof(struct epf_ntb_ctrl), sizeof(u32)); + if (epc_features->msix_capable) { + ret =3D pci_epc_get_hw_msix_layout(epc_features, + &ntb->msix_layout); + if (ret && ret !=3D -ENOENT) { + dev_err(dev, "Invalid hardware-owned MSI-X layout\n"); + return ret; + } + + if (ret =3D=3D -ENOENT) { + ntb->msix_layout.table_bar =3D barno; + ntb->msix_layout.table_offset =3D ALIGN(ctrl_size, 8); + ntb->msix_layout.table_size =3D + ntb->db_count * PCI_MSIX_ENTRY_SIZE; + ntb->msix_layout.pba_bar =3D barno; + ntb->msix_layout.pba_offset =3D + ntb->msix_layout.table_offset + + ntb->msix_layout.table_size; + ntb->msix_layout.pba_size =3D + BITS_TO_U64(ntb->db_count) * sizeof(u64); + ctrl_size =3D ntb->msix_layout.pba_offset + + ntb->msix_layout.pba_size; + } + } spad_size =3D 2 * spad_count * sizeof(u32); =20 base =3D pci_epf_alloc_space(epf, ctrl_size + spad_size, @@ -502,6 +528,7 @@ static int epf_ntb_config_spad_bar_alloc(struct epf_ntb= *ntb) static int epf_ntb_configure_interrupt(struct epf_ntb *ntb) { const struct pci_epc_features *epc_features; + struct pci_epf *epf =3D ntb->epf; struct device *dev; int ret; =20 @@ -521,16 +548,22 @@ static int epf_ntb_configure_interrupt(struct epf_ntb= *ntb) } =20 if (epc_features->msi_capable) { - ret =3D pci_epc_set_msi(ntb->epf->epc, - ntb->epf->func_no, - ntb->epf->vfunc_no, - 16); + ret =3D pci_epc_set_msi(epf->epc, epf->func_no, epf->vfunc_no, 16); if (ret) { dev_err(dev, "MSI configuration failed\n"); return ret; } } =20 + if (epc_features->msix_capable) { + ret =3D pci_epc_set_msix(epf->epc, epf->func_no, epf->vfunc_no, + ntb->db_count, &ntb->msix_layout); + if (ret) { + dev_err(dev, "MSI-X configuration failed\n"); + return ret; + } + } + return 0; } =20 @@ -1512,6 +1545,7 @@ static void vntb_epf_peer_db_work(struct work_struct = *work) struct epf_ntb *ntb =3D container_of(work, struct epf_ntb, peer_db_work); struct pci_epf *epf =3D ntb->epf; unsigned int budget =3D VNTB_PEER_DB_WORK_BUDGET; + unsigned int irq_type; u8 func_no, vfunc_no; unsigned int db_bit; u32 interrupt_num; @@ -1523,6 +1557,7 @@ static void vntb_epf_peer_db_work(struct work_struct = *work) =20 func_no =3D epf->func_no; vfunc_no =3D epf->vfunc_no; + irq_type =3D READ_ONCE(ntb->peer_msix) ? PCI_IRQ_MSIX : PCI_IRQ_MSI; =20 /* * Drain doorbells from peer_db_pending in snapshots (atomic64_xchg()). @@ -1536,16 +1571,16 @@ static void vntb_epf_peer_db_work(struct work_struc= t *work) =20 while (db_bits) { /* - * pci_epc_raise_irq() for MSI expects a 1-based - * interrupt number. The first usable doorbell starts - * at EPF_IRQ_DB_START in the legacy slot layout. + * pci_epc_raise_irq() expects a 1-based interrupt + * number for MSI and MSI-X. The first usable doorbell + * starts at EPF_IRQ_DB_START in the legacy slot layout. * * Legacy mapping (kept for compatibility): * - * MSI #1 : link event (reserved) - * MSI #2 : unused (historical offset) - * MSI #3 : doorbell bit 0 (DB#0) - * MSI #4 : doorbell bit 1 (DB#1) + * IRQ #1 : link event (reserved) + * IRQ #2 : unused (historical offset) + * IRQ #3 : doorbell bit 0 (DB#0) + * IRQ #4 : doorbell bit 1 (DB#1) * ... * * Do not change this mapping to avoid breaking @@ -1556,7 +1591,7 @@ static void vntb_epf_peer_db_work(struct work_struct = *work) db_bits &=3D ~BIT_ULL(db_bit); =20 ret =3D pci_epc_raise_irq(epf->epc, func_no, vfunc_no, - PCI_IRQ_MSI, interrupt_num); + irq_type, interrupt_num); if (ret) dev_err(&ntb->ntb.dev, "Failed to raise IRQ for interrupt_num %u: %d\n", --=20 2.51.0