From nobody Sun May 24 21:40:06 2026 Received: from TYVP286CU001.outbound.protection.outlook.com (mail-japaneastazon11021086.outbound.protection.outlook.com [52.101.125.86]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E7A13221F20; Thu, 21 May 2026 06:36:49 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=52.101.125.86 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779345411; cv=fail; b=lwsqMWUicSJM+HFlHDH1PUo0LCSwzZ4pzP62yhwqBYrwjBHFhg8pV8OICN/LZ9vmwm/xp269uOposgGzXOgalXm0kvoZo+nkMoOv40GyVuZlJmSVFDwFJxOyM4y9q4MyLajsw4lnBSMmKd3HXN0KIFfC4dUxW49QBd430NkxGzo= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779345411; c=relaxed/simple; bh=FB+witlToWXxcpiezfExdkTW4NijAxT45BJfM3QyChM=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: Content-Type:MIME-Version; b=uFzhMKhfUfWfFU7ENhlwbCpGiH5sOuA/09CbO3WwTDwzxAVOJ207E1zWNUJur7Dpg+A1hznp9pTVjNCgjaVsVQnGLZWhcPukDbTbMYqEPaBPS8HQ7UFAOIfeHgLURXey5I1A0UaeFgHcRbVq+hAxB9j1vJ9lzIHVlQpzukEuKDA= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=valinux.co.jp; spf=pass smtp.mailfrom=valinux.co.jp; dkim=pass (1024-bit key) header.d=valinux.co.jp header.i=@valinux.co.jp header.b=PJMqogCQ; arc=fail smtp.client-ip=52.101.125.86 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=valinux.co.jp Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=valinux.co.jp Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=valinux.co.jp header.i=@valinux.co.jp header.b="PJMqogCQ" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=AVD1KPIUHGN/xl3ONk7lm9qsQ5cwPejyuRRB1lKS4x/O+hfWqImdctur06++bS4rju9GRflAtCBdqI9S0qQ2kefpY7pcPkdwnLgh9w738CofVzRe/KA+vnWXSbSp/BS261FerORUUVkFalMZqbj07VVc0rltwK0D6syyl5QiH+YCPWXrUGINv3u269zzWBuQYWmbsZP5Ezn7M8Ty8fYDXSzhDYIrLC+6i0cfnSv+FuZkANw+DM1D77R6586bNPKMqfcvEamc9lLM2EcXnMvzt5zyDdmCFJz9fWZ+9jLPDSgERuZN+R816nuPsKhj8TqjMp5RuvhUtoI8Bp20MfHzMA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=dop1qI1keaJjQAunDvv5olkP8OB6yiyu1DY9uS5N5/Y=; b=cMQ+5b5oZ72eTwx1P4HVUHUxyBrBbbn3r0LSjth8yfOeOmNWPBDLH9oSZlK0n0znX1HXshCBdVO/n2rNeJEeLxhHmcsjqwOrtCWD/WfFTElYHWxnhq28ox8SNI3cHMT7I5QkYpL0jsGrP5AZ0kupxGWMxbdubTBVe6QycFjvH+0h5IgoDr35rWKpsqOsPcAlN8CUKCXa0ekiv+1GArhYYKo3e9n2qd9PCQi0FlOxMdbbapsEMOWpuolnm/o55tflXIY3nLjrHkx2hozQcGMyHHoaFtbsC8qx5rLiJhUr8EIvEVOMN/fjKERlu0xVpRPAFQg+g3BMKiSURPSAoJmIlg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=valinux.co.jp; dmarc=pass action=none header.from=valinux.co.jp; dkim=pass header.d=valinux.co.jp; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=valinux.co.jp; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=dop1qI1keaJjQAunDvv5olkP8OB6yiyu1DY9uS5N5/Y=; b=PJMqogCQfZ/hvuH9ExK08RwFVmyZgoD4qudHP6PyekHm+ES4T4Ldiw/dHkgUUk+nmTcoK3lQ8HjCFLIa+EBbx3DMod8sCxEiEbjkSLnTbP0D47MoCBQ/hnIiOOn+jTfusSO1n1hZ5vV/fW2bdEiyQyDDQz6OJJTbeVi2XBFP+Bg= Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=valinux.co.jp; Received: from TY7P286MB7722.JPNP286.PROD.OUTLOOK.COM (2603:1096:405:38f::10) by TY6P286MB7399.JPNP286.PROD.OUTLOOK.COM (2603:1096:405:35c::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.48.16; Thu, 21 May 2026 06:36:46 +0000 Received: from TY7P286MB7722.JPNP286.PROD.OUTLOOK.COM ([fe80::2305:327c:28ec:9b32]) by TY7P286MB7722.JPNP286.PROD.OUTLOOK.COM ([fe80::2305:327c:28ec:9b32%5]) with mapi id 15.21.0048.016; Thu, 21 May 2026 06:36:44 +0000 From: Koichiro Den To: Manivannan Sadhasivam , =?UTF-8?q?Krzysztof=20Wilczy=C5=84ski?= , Kishon Vijay Abraham I , Bjorn Helgaas , Jonathan Corbet , Shuah Khan , Vinod Koul , Frank Li , Arnd Bergmann , Damien Le Moal , Niklas Cassel Cc: Marek Vasut , Yoshihiro Shimoda , linux-pci@vger.kernel.org, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, dmaengine@vger.kernel.org Subject: [PATCH 1/3] dmaengine: dw-edma-pcie: Discover endpoint DMA metadata Date: Thu, 21 May 2026 15:36:36 +0900 Message-ID: <20260521063638.2843021-2-den@valinux.co.jp> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20260521063638.2843021-1-den@valinux.co.jp> References: <20260521063638.2843021-1-den@valinux.co.jp> Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: TY4P286CA0026.JPNP286.PROD.OUTLOOK.COM (2603:1096:405:2b0::11) To TY7P286MB7722.JPNP286.PROD.OUTLOOK.COM (2603:1096:405:38f::10) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: TY7P286MB7722:EE_|TY6P286MB7399:EE_ X-MS-Office365-Filtering-Correlation-Id: 9c96c2be-fc86-4447-bec0-08deb7035371 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|1800799024|10070799003|376014|7416014|366016|921020|18002099003|56012099003|22082099003|3023799007; X-Microsoft-Antispam-Message-Info: VyPxx/zEGX+jXGQN1knC/4UohEnbowhFgsQXc878kY3AIqRiUawK91TmQc5XvYZ7vYlcKktcJDPBGZariHnIG0l6b9xwVyHR0cWr3JQ+Q3aKK+fymMSFOgTgWmhnRNrQd4DgxsjkV6CSHSOzbz18MthtG6H+QpRaXSg1QNRLCOMURKmzPIXHgAJeKXRM6BbsRuVsbKk3cNGCiERpaDysVvF9VLb/KIWZMVt9/r4UokWQGfeXNJae0bDf0hTs+hvxQ+epylONR8Do+7h7s5riekAylq6Cj8pjYoF0J81T95asAn6TAxUOood6yc/8fNHupfzQh4t076PCtaEnw79hSSQNMnyMIOLa1Pw+KSx6wT1q8RjmldHPi1Y4HZK3/grmNRkUvBp9Le4QNedq6mclu8rw4/ahicWMoPiFLf1Q42qHocWgVe3Arymohg2gTgsEeLHgWUdjrmqOzyFQfdDHk/AfO/d6hkIrAtZSG7q6ZIKjtRQF9eBF+cgaGDKIDRTwKLLS47KJpukHi0UIzxB4RWAl8e0T8nMPM50A7faf1T0xt6TG8qAJZMl9Z62jvN2gMbKvGgYKcXAxJBXc4pUg5RkYKF9KVbaP5QyDDpe1+3FO8sV4gdBL7zCAsEy4ooD+dRLaDdURZZnl1r2xtewAby8NxUh3usOoK9TgjZbLK8lCokV2iIPz2hK8FH6MFWlonWI5oiDJxs5OYr0YWacVLj8QrxbL1wXi93+YUihcZd8= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:TY7P286MB7722.JPNP286.PROD.OUTLOOK.COM;PTR:;CAT:NONE;SFS:(13230040)(1800799024)(10070799003)(376014)(7416014)(366016)(921020)(18002099003)(56012099003)(22082099003)(3023799007);DIR:OUT;SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?OQcZmX7EMWFpT7f5unYEAwdh001y1l7Cb3Rn3ytchL+Nlyk7ACQibPap+4lb?= =?us-ascii?Q?Nv8HDr2NSWPu8E8tJ3PY6UJ+Rr6KziO8Lvqw0X9HIWvaJkh+kLp1LcDgemOo?= =?us-ascii?Q?ry9VTs9pSZVezUQKgApr8YL/li9Jn1NFoJs4CUbmTzdV3CWNhd+WRH9HhLD6?= =?us-ascii?Q?pjtWKG6KpRYVt84ycDzQ4wPDGj3I2IrACA/wjLUOmJQNEKb78b8wsTw0Q/yJ?= =?us-ascii?Q?gLgcV9EFouzukk+5cIGrEyRZAUhCSPY26duDKpHPbHaUsktQV6o0D084jHxX?= =?us-ascii?Q?oE+TwiOpi9ICJnFBr3919oR1S4siBwUxKHn3w7+BISchms0TcdWjBUgaakg8?= =?us-ascii?Q?33oQRkndCJ3QjsxOjRZ6y2F1HOyLuZY7OdSSotoj1l459dQWvoJdy3acZetj?= =?us-ascii?Q?amxDtUGkg1TXz/iDyco4gi8dm2PA7DsbZk2yXMK8Wot0CZngquhPfTf5jbUk?= =?us-ascii?Q?8T4MV9uo8qJ40REdsffBeF7PKzL0DcratxhIe9a+ShKiJuJdp6qR/dZeWM0v?= =?us-ascii?Q?rZXcbj5hGsarvQ7mVywWxDnhIHTJTiryBgg48Tr7FsqrTWxcO0IOUD8AkQ2Q?= =?us-ascii?Q?WPcJeisZym5Lbi7PgJXmfHpZ8xy4RuIGTx5rcuMdnmSWUleKrWGZKRzqMRKx?= =?us-ascii?Q?BPCvieRQiw4AzYVT1iqgi+3E0rFvYKdjOmm8yA6Q+DoU4Ts3Py9Ic4Ovpads?= =?us-ascii?Q?uTC1fQDcHa0nNTs+VU4RrHHdplyNGfglDKWKKKpaezqBMRw+Iq5tl+9f6+jp?= =?us-ascii?Q?amQ73Y9XHPsJjRYeIjhGmY78eTbSE6e4V0mdYav3lRm2X07Lp7z5etjYBFAy?= =?us-ascii?Q?vKHqmtGnMGRNJ4/N/nOAbjcCVUXcX3JNMK7UhBSPyadFLW1bmH/WLuuuiT1x?= =?us-ascii?Q?0Dw+5i5Oy5eiXMuy/aYLZZD9o8bUnIp62gIA8zuFj6MsUqqRzdUC3xd3gtno?= =?us-ascii?Q?pR56Xqc4DZS7TclAKS+zRWGs5nFXKPAM8H0XyLTQaFJKAV18ZEpSIh8Ob5+1?= =?us-ascii?Q?caelWR68GDiDnh6tYUoU8VB0p/cb+84oq4JJAZxKZYURqzxJ3+SVoXsyea9R?= =?us-ascii?Q?HnsePimgEPTu4TKTwLOQ184dmmwHtbg208JN0Xd2Pl1zbQog1IfHXndxpupc?= =?us-ascii?Q?vbUWiX98RAwOYDuM7tZxioBJB7akKJ+yFQnCnxtVYQuH8rd5BFlqnk9uU6wQ?= =?us-ascii?Q?xjecKo8aRA2xtkLqY52ANmapy9KkMBiAU1YAtko68Da5qGLAGaI120dDEFhi?= =?us-ascii?Q?8bK2+jDoEA3lPz24AuL14AE5itDJA5A8RyoJQznNVuSQ9me2ElJbGPgTKMpL?= =?us-ascii?Q?V2l2zwXnkSlqT26vpTPfn2g8ROZUmpUmDBStvKnK70BjZNBzddUpNDm7A3df?= =?us-ascii?Q?5yODkZhNg8csDFAJFt6UTwJq3QmBsYcX7AJBpoct5xOm7XDihv8cME8SSCTA?= =?us-ascii?Q?GFLYDfZJyFeDpxn+knQsvrOacR6kxOqIXAk90uAJQawIujZodzpdg9BAw2cT?= =?us-ascii?Q?8hFOdJ/bqYHncQS4IqIiATnmhirwn21KXHWn2Rwki1JSTwT40T4yDX2KUEfx?= =?us-ascii?Q?bgID18rL7QvPGIF3JdmQN2UVpMF3JgEuBMXedGy0E1nsn9uliMQt1E8cYmO+?= =?us-ascii?Q?h7JFnuBM6w11WuBt59+DKZKlCNfQwwwigEq6Y6HjYxvuE96b2F51RYBUFjs3?= =?us-ascii?Q?ezqVYOpQZCOAlBymOzdOuzlUHJgbLfeanLKBPQnRK4TYTcT4mgnTt8CXN6uV?= =?us-ascii?Q?WglotSw8eLfIFwufEc3Ipz4ErjNfIyebqoY/BD9GqDzwM24IR8Yl?= X-OriginatorOrg: valinux.co.jp X-MS-Exchange-CrossTenant-Network-Message-Id: 9c96c2be-fc86-4447-bec0-08deb7035371 X-MS-Exchange-CrossTenant-AuthSource: TY7P286MB7722.JPNP286.PROD.OUTLOOK.COM X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 21 May 2026 06:36:44.7015 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 7a57bee8-f73d-4c5f-a4f7-d72c91c8c111 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: OIoEieaMFPoOrs4kRX6mNuSDt2MxKI+YLfnNn+CGRdCH8KKgEti+Nw4CUiS/fbe703ZfmTGAeH5p44zlS0ofvg== X-MS-Exchange-Transport-CrossTenantHeadersStamped: TY6P286MB7399 Content-Type: text/plain; charset="utf-8" Teach dw-edma-pcie to discover a PCI endpoint DMA function from BAR-resident metadata. The metadata supplies the DMA register window, channel counts, descriptor windows, optional auxiliary windows, and endpoint-local descriptor and auxiliary addresses. Endpoint-provided DMA channels use raw slave addresses because the host programs transfers against endpoint physical addresses, not PCI BAR addresses. Scope the default remote interrupt mode to the endpoint DMA metadata match entry so EDDA and MDB keep their existing local interrupt behavior. Endpoint DMA metadata can be discovered after an explicit bind through driver_override or a dynamic ID. For such binds, there is no static match data, so the driver falls back to the generic endpoint DMA metadata parser. The endpoint polls HOST_REQ at a low idle rate before programming DMA window submaps and setting READY. Let the host wait for several endpoint poll periods before treating the READY handshake as timed out. Signed-off-by: Koichiro Den --- drivers/dma/dw-edma/dw-edma-pcie.c | 369 ++++++++++++++++++++++++++++- 1 file changed, 368 insertions(+), 1 deletion(-) diff --git a/drivers/dma/dw-edma/dw-edma-pcie.c b/drivers/dma/dw-edma/dw-ed= ma-pcie.c index 2f752e8fb999..d4ae6df36858 100644 --- a/drivers/dma/dw-edma/dw-edma-pcie.c +++ b/drivers/dma/dw-edma/dw-edma-pcie.c @@ -11,9 +11,13 @@ #include #include #include +#include #include #include #include +#include +#include +#include #include =20 #include "dw-edma-core.h" @@ -44,6 +48,9 @@ #define DW_PCIE_XILINX_MDB_DT_OFF_GAP 0x100000 #define DW_PCIE_XILINX_MDB_DT_SIZE 0x800 =20 +#define DW_PCIE_EP_DMA_READY_POLL_US 1000 +#define DW_PCIE_EP_DMA_READY_TIMEOUT_US 2000000 + #define DW_BLOCK(a, b, c) \ { \ .bar =3D a, \ @@ -93,6 +100,12 @@ struct dw_edma_pcie_match_data { #define DW_EDMA_PCIE_F_RAW_SLAVE_ADDR BIT(1) #define DW_EDMA_PCIE_F_REG_OFFSET BIT(2) =20 +struct dw_edma_pcie_ep_dma_view { + struct pci_dev *pdev; + void __iomem *base; + resource_size_t limit; +}; + static const struct dw_edma_pcie_data snps_edda_data =3D { /* eDMA registers location */ .rg.bar =3D BAR_0, @@ -144,6 +157,13 @@ static const struct dw_edma_pcie_data xilinx_mdb_data = =3D { .rd_ch_cnt =3D 8, }; =20 +static const struct dw_edma_pcie_data ep_dma_data =3D { + .mf =3D EDMA_MF_EDMA_UNROLL, + .irqs =3D EDMA_MAX_WR_CH + EDMA_MAX_RD_CH, + .wr_ch_cnt =3D EDMA_MAX_WR_CH, + .rd_ch_cnt =3D EDMA_MAX_RD_CH, +}; + static void dw_edma_set_chan_region_offset(struct dw_edma_pcie_data *pdata, enum pci_barno bar, off_t start_off, off_t ll_off_gap, size_t ll_size, @@ -217,6 +237,82 @@ static const struct dw_edma_plat_ops dw_edma_pcie_raw_= addr_plat_ops =3D { .irq_vector =3D dw_edma_pcie_irq_vector, }; =20 +static bool dw_edma_pcie_valid_bar(enum pci_barno bar) +{ + return bar >=3D BAR_0 && bar <=3D BAR_5; +} + +static bool dw_edma_pcie_valid_bar_range(struct pci_dev *pdev, + enum pci_barno bar, u64 off, + size_t sz) +{ + resource_size_t bar_len; + + if (!dw_edma_pcie_valid_bar(bar) || !sz) + return false; + + bar_len =3D pci_resource_len(pdev, bar); + + return off <=3D bar_len && sz <=3D bar_len - off; +} + +static bool dw_edma_pcie_valid_block(struct pci_dev *pdev, + const struct dw_edma_block *block) +{ + return dw_edma_pcie_valid_bar_range(pdev, block->bar, block->off, + block->sz); +} + +static bool dw_edma_pcie_ep_dma_bar_scannable(struct pci_dev *pdev, + enum pci_barno bar) +{ + unsigned long flags =3D pci_resource_flags(pdev, bar); + + if (!(flags & IORESOURCE_MEM)) + return false; + + if (flags & (IORESOURCE_UNSET | IORESOURCE_DISABLED)) + return false; + + return pci_resource_len(pdev, bar) >=3D PCI_EP_DMA_METADATA_HDR_LEN; +} + +static u32 dw_edma_pcie_ep_dma_readl(struct dw_edma_pcie_ep_dma_view *view, + u16 off) +{ + return readl(view->base + off); +} + +static void dw_edma_pcie_ep_dma_writel(struct dw_edma_pcie_ep_dma_view *vi= ew, + u16 off, u32 val) +{ + writel(val, view->base + off); +} + +static u64 dw_edma_pcie_ep_dma_read64(struct dw_edma_pcie_ep_dma_view *vie= w, + u16 lo, u16 hi) +{ + u64 val; + + val =3D dw_edma_pcie_ep_dma_readl(view, hi); + + return (val << 32) | dw_edma_pcie_ep_dma_readl(view, lo); +} + +static int dw_edma_pcie_ep_dma_read_off(struct dw_edma_pcie_ep_dma_view *v= iew, + u16 lo, u16 hi, off_t *off) +{ + u64 val; + + val =3D dw_edma_pcie_ep_dma_read64(view, lo, hi); + if (val > type_max(*off)) + return -EINVAL; + + *off =3D val; + + return 0; +} + static void dw_edma_pcie_get_synopsys_dma_data(struct pci_dev *pdev, struct dw_edma_pcie_data *pdata) { @@ -318,6 +414,265 @@ static void dw_edma_pcie_get_xilinx_dma_data(struct p= ci_dev *pdev, pdata->devmem_phys_off =3D off; } =20 +static int +dw_edma_pcie_parse_ep_dma_ch_table(struct dw_edma_pcie_ep_dma_view *view, + struct dw_edma_pcie_data *pdata, + u16 table_off, u16 entry_size, u16 ch_cnt, + bool write) +{ + struct dw_edma_block *desc_blocks =3D write ? pdata->ll_wr : pdata->ll_rd; + struct dw_edma_block *data_blocks =3D write ? pdata->dt_wr : pdata->dt_rd; + u32 ctrl; + u16 i; + int ret; + + for (i =3D 0; i < ch_cnt; i++) { + struct dw_edma_block *desc_block =3D &desc_blocks[i]; + struct dw_edma_block *data_block =3D &data_blocks[i]; + u16 off =3D table_off + i * entry_size; + u16 field, lo, hi; + + field =3D off + PCI_EP_DMA_METADATA_CH_CTRL; + ctrl =3D dw_edma_pcie_ep_dma_readl(view, field); + if (FIELD_GET(PCI_EP_DMA_METADATA_CH_CTRL_HW_CH, ctrl) !=3D i) + return -EOPNOTSUPP; + + desc_block->bar =3D + FIELD_GET(PCI_EP_DMA_METADATA_CH_CTRL_DESC_BAR, ctrl); + lo =3D off + PCI_EP_DMA_METADATA_CH_DESC_OFF_LO; + hi =3D off + PCI_EP_DMA_METADATA_CH_DESC_OFF_HI; + ret =3D dw_edma_pcie_ep_dma_read_off(view, lo, hi, + &desc_block->off); + if (ret) + return ret; + field =3D off + PCI_EP_DMA_METADATA_CH_DESC_SIZE; + desc_block->sz =3D dw_edma_pcie_ep_dma_readl(view, field); + lo =3D off + PCI_EP_DMA_METADATA_CH_DESC_ADDR_LO; + hi =3D off + PCI_EP_DMA_METADATA_CH_DESC_ADDR_HI; + desc_block->paddr =3D + dw_edma_pcie_ep_dma_read64(view, lo, hi); + desc_block->paddr_valid =3D true; + if (!dw_edma_pcie_valid_block(view->pdev, desc_block)) + return -EINVAL; + + *data_block =3D (struct dw_edma_block) { .bar =3D NO_BAR }; + if (!(ctrl & PCI_EP_DMA_METADATA_CH_CTRL_AUX_VALID)) + continue; + + data_block->bar =3D + FIELD_GET(PCI_EP_DMA_METADATA_CH_CTRL_AUX_BAR, ctrl); + lo =3D off + PCI_EP_DMA_METADATA_CH_AUX_OFF_LO; + hi =3D off + PCI_EP_DMA_METADATA_CH_AUX_OFF_HI; + ret =3D dw_edma_pcie_ep_dma_read_off(view, lo, hi, + &data_block->off); + if (ret) + return ret; + field =3D off + PCI_EP_DMA_METADATA_CH_AUX_SIZE; + data_block->sz =3D dw_edma_pcie_ep_dma_readl(view, field); + lo =3D off + PCI_EP_DMA_METADATA_CH_AUX_ADDR_LO; + hi =3D off + PCI_EP_DMA_METADATA_CH_AUX_ADDR_HI; + data_block->paddr =3D + dw_edma_pcie_ep_dma_read64(view, lo, hi); + data_block->paddr_valid =3D true; + if (!dw_edma_pcie_valid_block(view->pdev, data_block)) + return -EINVAL; + } + + return 0; +} + +static int +dw_edma_pcie_ep_dma_wait_ready(struct dw_edma_pcie_ep_dma_view *view) +{ + u32 val; + + return read_poll_timeout(dw_edma_pcie_ep_dma_readl, val, + val & PCI_EP_DMA_METADATA_CTRL_READY, + DW_PCIE_EP_DMA_READY_POLL_US, + DW_PCIE_EP_DMA_READY_TIMEOUT_US, false, + view, PCI_EP_DMA_METADATA_CTRL); +} + +static int +dw_edma_pcie_validate_ep_dma_metadata(struct dw_edma_pcie_ep_dma_view *vie= w, + u32 *metadata_ctrl, u8 *reg_layout_data) +{ + size_t table_size, table_end; + enum pci_barno reg_bar; + u16 len, entry_size; + u16 wr_ch_cnt, rd_ch_cnt; + u8 layout, layout_data; + u32 val; + + val =3D dw_edma_pcie_ep_dma_readl(view, 0); + if (val !=3D PCI_EP_DMA_METADATA_MAGIC) + return -ENODEV; + + val =3D dw_edma_pcie_ep_dma_readl(view, PCI_EP_DMA_METADATA_HDR); + if (FIELD_GET(PCI_EP_DMA_METADATA_HDR_REV, val) !=3D + PCI_EP_DMA_METADATA_REV) + return -EINVAL; + + len =3D FIELD_GET(PCI_EP_DMA_METADATA_HDR_LEN_FIELD, val); + if (len < PCI_EP_DMA_METADATA_HDR_LEN) + return -EINVAL; + if (len > view->limit) + return -EINVAL; + + val =3D dw_edma_pcie_ep_dma_readl(view, PCI_EP_DMA_METADATA_REG_LAYOUT); + layout =3D FIELD_GET(PCI_EP_DMA_METADATA_REG_LAYOUT_ID, val); + if (layout !=3D PCI_EP_DMA_METADATA_REG_LAYOUT_DW_EDMA) + return -EOPNOTSUPP; + + layout_data =3D FIELD_GET(PCI_EP_DMA_METADATA_REG_LAYOUT_DATA, val); + if (layout_data =3D=3D EDMA_MF_EDMA_LEGACY || + layout_data =3D=3D EDMA_MF_HDMA_NATIVE) + return -EOPNOTSUPP; + if (layout_data !=3D EDMA_MF_EDMA_UNROLL && + layout_data !=3D EDMA_MF_HDMA_COMPAT) + return -EINVAL; + + val =3D dw_edma_pcie_ep_dma_readl(view, PCI_EP_DMA_METADATA_CTRL); + reg_bar =3D FIELD_GET(PCI_EP_DMA_METADATA_CTRL_REG_BAR, val); + if (!dw_edma_pcie_valid_bar(reg_bar)) + return -EINVAL; + + wr_ch_cnt =3D FIELD_GET(PCI_EP_DMA_METADATA_CTRL_WR_CH_COUNT, val); + rd_ch_cnt =3D FIELD_GET(PCI_EP_DMA_METADATA_CTRL_RD_CH_COUNT, val); + if (!wr_ch_cnt && !rd_ch_cnt) + return -EINVAL; + if (wr_ch_cnt > EDMA_MAX_WR_CH || rd_ch_cnt > EDMA_MAX_RD_CH) + return -EINVAL; + + entry_size =3D FIELD_GET(PCI_EP_DMA_METADATA_CTRL_CH_ENTRY_SIZE, val); + if (entry_size < PCI_EP_DMA_METADATA_CH_ENTRY_SIZE || + entry_size % sizeof(u32)) + return -EINVAL; + + if (check_mul_overflow((size_t)(wr_ch_cnt + rd_ch_cnt), + (size_t)entry_size, &table_size) || + check_add_overflow((size_t)PCI_EP_DMA_METADATA_HDR_LEN, + table_size, &table_end) || + table_end > len) + return -EINVAL; + + if (metadata_ctrl) + *metadata_ctrl =3D val; + if (reg_layout_data) + *reg_layout_data =3D layout_data; + + return 0; +} + +static int +dw_edma_pcie_parse_ep_dma_data(struct dw_edma_pcie_ep_dma_view *view, + struct dw_edma_pcie_data *pdata) +{ + u32 ctrl, reg_sz; + u8 reg_layout_data; + u64 reg_off; + u16 wr_table, rd_table, entry_size; + u16 wr_ch_cnt, rd_ch_cnt; + int ret; + + ret =3D dw_edma_pcie_validate_ep_dma_metadata(view, &ctrl, + ®_layout_data); + if (ret) + return ret; + + pci_dbg(view->pdev, "Detected PCI endpoint DMA BAR metadata\n"); + + pdata->mf =3D reg_layout_data; + pdata->rg.bar =3D FIELD_GET(PCI_EP_DMA_METADATA_CTRL_REG_BAR, ctrl); + + wr_ch_cnt =3D FIELD_GET(PCI_EP_DMA_METADATA_CTRL_WR_CH_COUNT, ctrl); + rd_ch_cnt =3D FIELD_GET(PCI_EP_DMA_METADATA_CTRL_RD_CH_COUNT, ctrl); + pdata->wr_ch_cnt =3D min_t(u16, pdata->wr_ch_cnt, wr_ch_cnt); + pdata->rd_ch_cnt =3D min_t(u16, pdata->rd_ch_cnt, rd_ch_cnt); + pdata->irqs =3D pdata->wr_ch_cnt + pdata->rd_ch_cnt; + reg_off =3D dw_edma_pcie_ep_dma_read64(view, + PCI_EP_DMA_METADATA_REG_OFF_LO, + PCI_EP_DMA_METADATA_REG_OFF_HI); + reg_sz =3D dw_edma_pcie_ep_dma_readl(view, PCI_EP_DMA_METADATA_REG_SIZE); + if (reg_off > type_max(pdata->rg.off) || + !dw_edma_pcie_valid_bar_range(view->pdev, pdata->rg.bar, + reg_off, reg_sz)) + return -EINVAL; + pdata->rg.off =3D reg_off; + pdata->rg.sz =3D reg_sz; + + entry_size =3D FIELD_GET(PCI_EP_DMA_METADATA_CTRL_CH_ENTRY_SIZE, ctrl); + wr_table =3D PCI_EP_DMA_METADATA_HDR_LEN; + rd_table =3D PCI_EP_DMA_METADATA_HDR_LEN + wr_ch_cnt * entry_size; + + ret =3D dw_edma_pcie_parse_ep_dma_ch_table(view, pdata, wr_table, + entry_size, pdata->wr_ch_cnt, + true); + if (ret) + return ret; + + return dw_edma_pcie_parse_ep_dma_ch_table(view, pdata, rd_table, + entry_size, + pdata->rd_ch_cnt, false); +} + +static int +dw_edma_pcie_parse_ep_dma_caps(struct pci_dev *pdev, + struct dw_edma_pcie_data *pdata, bool *non_ll) +{ + struct dw_edma_pcie_ep_dma_view metadata_view; + void __iomem *base; + resource_size_t bar_len; + enum pci_barno bar; + u32 ctrl; + int ret; + + for (bar =3D BAR_0; bar < PCI_STD_NUM_BARS; bar++) { + if (!dw_edma_pcie_ep_dma_bar_scannable(pdev, bar)) + continue; + + bar_len =3D pci_resource_len(pdev, bar); + base =3D pci_iomap_range(pdev, bar, 0, 0); + if (!base) + continue; + + metadata_view =3D (struct dw_edma_pcie_ep_dma_view) { + .pdev =3D pdev, + .base =3D base, + .limit =3D bar_len, + }; + ret =3D dw_edma_pcie_validate_ep_dma_metadata(&metadata_view, + NULL, NULL); + if (ret =3D=3D -ENODEV) { + pci_iounmap(metadata_view.pdev, base); + continue; + } + if (ret) { + pci_iounmap(metadata_view.pdev, base); + return ret; + } + + ctrl =3D dw_edma_pcie_ep_dma_readl(&metadata_view, + PCI_EP_DMA_METADATA_CTRL); + ctrl |=3D PCI_EP_DMA_METADATA_CTRL_HOST_REQ; + dw_edma_pcie_ep_dma_writel(&metadata_view, + PCI_EP_DMA_METADATA_CTRL, ctrl); + + ret =3D dw_edma_pcie_ep_dma_wait_ready(&metadata_view); + if (ret) { + pci_iounmap(metadata_view.pdev, base); + return ret; + } + + ret =3D dw_edma_pcie_parse_ep_dma_data(&metadata_view, pdata); + pci_iounmap(metadata_view.pdev, base); + + return ret; + } + + return -ENODEV; +} + static int dw_edma_pcie_parse_synopsys_caps(struct pci_dev *pdev, struct dw_edma_pcie_data *pdata, bool *non_ll) @@ -357,6 +712,14 @@ dw_edma_pcie_parse_xilinx_caps(struct pci_dev *pdev, return 0; } =20 +static const struct dw_edma_pcie_match_data ep_dma_match_data =3D { + .data =3D &ep_dma_data, + .parse_caps =3D dw_edma_pcie_parse_ep_dma_caps, + .flags =3D DW_EDMA_PCIE_F_REG_OFFSET | DW_EDMA_PCIE_F_RAW_SLAVE_ADDR, + .chip_flags =3D DW_EDMA_CHIP_PARTIAL, + .default_irq_mode =3D DW_EDMA_CH_IRQ_REMOTE, +}; + static u64 dw_edma_get_phys_addr(struct pci_dev *pdev, const struct dw_edma_pcie_match_data *match, struct dw_edma_pcie_data *pdata, @@ -384,7 +747,7 @@ static int dw_edma_pcie_probe(struct pci_dev *pdev, const struct pci_device_id *pid) { const struct dw_edma_pcie_match_data *match =3D (void *)pid->driver_data; - const struct dw_edma_pcie_data *pdata =3D match->data; + const struct dw_edma_pcie_data *pdata; struct device *dev =3D &pdev->dev; struct dw_edma_chip *chip; int err, nr_irqs; @@ -398,6 +761,10 @@ static int dw_edma_pcie_probe(struct pci_dev *pdev, return err; } =20 + if (!match) + match =3D &ep_dma_match_data; + pdata =3D match->data; + struct dw_edma_pcie_data *dma_data __free(kfree) =3D kmemdup(pdata, sizeof(*dma_data), GFP_KERNEL); if (!dma_data) --=20 2.51.0 From nobody Sun May 24 21:40:06 2026 Received: from TYVP286CU001.outbound.protection.outlook.com (mail-japaneastazon11021086.outbound.protection.outlook.com [52.101.125.86]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id ECEF8394EBA; Thu, 21 May 2026 06:36:51 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=52.101.125.86 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779345414; cv=fail; b=oKSAQX6N+hatIilP5yi8HpGErR6g24qn8rcs+1HwAYiTgztc2TIsqkxQ6GMLNVSRx4MSx6PXptF6Rbm4aGcS4JAHsYU8WKhjt3zEtmXnPyuP4hZ/0FHca6/dTwGP7aGg6O2NVdq8ubY1FlaZvPYfQGhjBBu2XkWYt3N60EP7xhU= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779345414; c=relaxed/simple; bh=FyXeL25ihYvmtA+thu4IDcLl1OmvSnQSceKaS4iNsXA=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: Content-Type:MIME-Version; b=UbuvEW1RCVRBwUeAb09h02T5LRUKfSMSqCV3YFdbnyCDCay9hgoeJxl9w6+5NqT0iCsCM4JIF8Y0mkqHofQQlur4iS9siplnWR/aPytKcdWEcZ91C9uMkiNLyMSUIWGdZRXMKdRhTMSufAqG/975QwglCVtPERbsx1/SeAGDOFE= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=valinux.co.jp; spf=pass smtp.mailfrom=valinux.co.jp; dkim=pass (1024-bit key) header.d=valinux.co.jp header.i=@valinux.co.jp header.b=LT3YLhsf; arc=fail smtp.client-ip=52.101.125.86 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=valinux.co.jp Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=valinux.co.jp Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=valinux.co.jp header.i=@valinux.co.jp header.b="LT3YLhsf" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=Vm40JAD2COu3jUD1m7IU+26F19w0cYH4g9usPgTegHHzWFCdX9tnGLOlfjOkWIL95Za4ZtLSzNIMRsbNjAKcb8aviuWWMFERZDxWoJkaB7ay/dpazC0sWPcqsK0cUXfMPMIAMSCmhg8zmvm+b0LBJ9HolGY2ej8wRPlqjFUildQkpST8OU5/FLqlc35JOAEk8E1SZcVhXWtd4QnBymVzUbwxo53E4qdpLYWwl0rKuPYRnAEN2sAPEaG5OJTk/r6TV17kA5mi4Cbojv/Jnb4gogUorZHbakVmHhq+C5cQ3w194EWKKYB7OU6dmvjdFHx065uLYDW/DLgT/a3GVGEAxw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=a4BgLifLazLHnpHnDCNCCP0k2md3auKf/cVubtuN6Pw=; b=fRAazZSWx2txqYWQzeCaR7/DOgBFC/onIKZX7F7xTLul2Tumc61K0jeSxUcN0CTWk/tRKlDHLyvBsC0E20eoH7bzGsnHEoSAUU7nu4Y/ipwASW2Bt+mhApGhwU9FuhIQIxdeogFxYsHBFvxve2vipFSH6TMrT/Zllm5k7RInHgFCceqX4leIQ1kP0UUYUyXxMvAqh1AwEi8FqRD/a5Q/06HJxLXofOIvos0D6ysDmSw4YUst4ogOtztLvir4fOaKBzSngmDvIfirpYLHyrxaFaWvS3MVqeH0oqrHKeEZ4pwo4cTrU4fJt5hAY2zADjhnPIPQcSUrVlEG9xkpb4sp4A== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=valinux.co.jp; dmarc=pass action=none header.from=valinux.co.jp; dkim=pass header.d=valinux.co.jp; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=valinux.co.jp; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=a4BgLifLazLHnpHnDCNCCP0k2md3auKf/cVubtuN6Pw=; b=LT3YLhsf5Of04zaxxI6OEIFLIjGaOisN11JGr7lvUFbnLbTtv7kf1Ytb6kMg9x+ub2HOVDVXXJZrHOT+MSfNRbNMDPrj8Vv8T5L3HfyW4huhRiZeOg6RJJ8dIaJNFud7HWkvNNGRMI5cyAuW7jgbIXUFOlLaeRj/gHUZV0bDWJY= Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=valinux.co.jp; Received: from TY7P286MB7722.JPNP286.PROD.OUTLOOK.COM (2603:1096:405:38f::10) by TY6P286MB7399.JPNP286.PROD.OUTLOOK.COM (2603:1096:405:35c::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.48.16; Thu, 21 May 2026 06:36:46 +0000 Received: from TY7P286MB7722.JPNP286.PROD.OUTLOOK.COM ([fe80::2305:327c:28ec:9b32]) by TY7P286MB7722.JPNP286.PROD.OUTLOOK.COM ([fe80::2305:327c:28ec:9b32%5]) with mapi id 15.21.0048.016; Thu, 21 May 2026 06:36:46 +0000 From: Koichiro Den To: Manivannan Sadhasivam , =?UTF-8?q?Krzysztof=20Wilczy=C5=84ski?= , Kishon Vijay Abraham I , Bjorn Helgaas , Jonathan Corbet , Shuah Khan , Vinod Koul , Frank Li , Arnd Bergmann , Damien Le Moal , Niklas Cassel Cc: Marek Vasut , Yoshihiro Shimoda , linux-pci@vger.kernel.org, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, dmaengine@vger.kernel.org Subject: [PATCH 2/3] PCI: endpoint: Add DMA endpoint function Date: Thu, 21 May 2026 15:36:37 +0900 Message-ID: <20260521063638.2843021-3-den@valinux.co.jp> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20260521063638.2843021-1-den@valinux.co.jp> References: <20260521063638.2843021-1-den@valinux.co.jp> Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: TY4PR01CA0032.jpnprd01.prod.outlook.com (2603:1096:405:2bd::13) To TY7P286MB7722.JPNP286.PROD.OUTLOOK.COM (2603:1096:405:38f::10) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: TY7P286MB7722:EE_|TY6P286MB7399:EE_ X-MS-Office365-Filtering-Correlation-Id: e9aa5c34-ef8d-4cdb-72d4-08deb703541b X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|1800799024|10070799003|376014|7416014|366016|921020|18002099003|56012099003|22082099003|6133799003|5023799004; X-Microsoft-Antispam-Message-Info: rcJKVlYQB1EqWA5EvE858IOJz2pVQnt4hqWiLOsoiTY8ypu/Y30iqASwHnWf6LO3Ha3UzQxN/rVF64V7/KRVQUbVcHJwtCSgjp2lAcUYLbRhYKIwMi7vfJTHlSXZJU4gb6mzr6vcv6Sain4l0gpySdc1k4K7hALDqoNC4djbevW8qAmLUfTmQbw+w+sn3XCOHrlWcIExkTs9DsG8qfiU5bEQsvBS2Et2t2p7RRnlLLcsW2+xL/0RrNBX9JiU5cQJ4ie6McHN/UxoL6FMD+VVIpMcfSP4k48goRb4vw+9S02ZL1kCGnDUao78akPTi0k8FSGxs6BwSPTmibC3BFh82tgrK50/Nskd4dc8pc0jQxK5u7OWo8bbkxR0Y2f8zSzwCQjtD+fNDfA8VUTTJS4ybnyKkmy3oT56gpSMEJ88drt8B8Kbc9VHNdlGoGzGL472sSbl1m3eQRd7NGE2ao4DUpzA2BVXfVu+b9/N2P0FHtRyeiKuFlMjv8aQOkP5j3Mwb1FiF/XCDHJ418vadoBQn91EKC3FprGPr4kNVEfXOSLT4MRwSB2TAxac5Zgt12F9Bl1qPO+RtFQayaMtq2svvxaWr+2a2t0aDwHSk/ZVLJOO4kdeJh3ELNf6ECAH87NAP7GzL7kKXyTu0ObuPMUgzwQJPzSPMzmKXjH+lDfXjZnP6hEyAbp/NaQecIWtriw8jBw4RZajP5kDZza+UH7pvBHI+pLbrXdqKFGk93beDAo= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:TY7P286MB7722.JPNP286.PROD.OUTLOOK.COM;PTR:;CAT:NONE;SFS:(13230040)(1800799024)(10070799003)(376014)(7416014)(366016)(921020)(18002099003)(56012099003)(22082099003)(6133799003)(5023799004);DIR:OUT;SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?BLKd/74PtDbFC4Y0GmJdX8RhQsWX1xkeQcqwFVHALI1/EF5lSoAgVFV6207Q?= =?us-ascii?Q?5Td6VKJHxpCuiq96Eo8Pt4mIseZRka5mfgDFP47JbdmdXotmcddvOwanPukG?= =?us-ascii?Q?ciIeeMzVfO+ezxFh6xonhiEdS84YG00BYFn2g9IJjpc2uj1+pEHdCMLRZ0d6?= =?us-ascii?Q?KmXvde8Uz9QxfKGsfJslW5JT12axzoKlHscqwtp1vkbfI+ffFMHx8N+UcGw7?= =?us-ascii?Q?rs6g770djhrKQrNZRxOWzzpWDZLwNCUnBmr0lB1e9SQKtk1mqP6orqgtiM3f?= =?us-ascii?Q?oKIHviaMh/AFctuCVCwTGesYxkK08Pgr+ms7shngi0xknP1ATf9Ze/e4n1JN?= =?us-ascii?Q?IrTF9QI17JroMW63vIPBqRm+637Bylfm3qY5OjBZuvMCCc+VK7VwraLORHlO?= =?us-ascii?Q?Q0UHY5ti0byEJsIgDXd1D60k07xl6JyUCfCfw25NILb6rOwDrkTf/zeyy/6A?= =?us-ascii?Q?L/LyMTIQHXvetT1N2LHocGs2+ptB9iQ2NdNi4JHTNCMJ4F0Wletdueo3l1bW?= =?us-ascii?Q?XrlK1NyzjgB75/Zh1TtBLVR0kTu8+zyLNYquAz/N9V2AY8CLr6kvhShW557g?= =?us-ascii?Q?mfL82ZSFybYlXzDkwCJl8YyAzF6BiQINAgLX59BXghJA2QDnXSrhMYQ/fHjU?= =?us-ascii?Q?iayYi/U2RoYXOA8iuIVhFu4V3A3auFHIIiOE8wxFARE96PcbiCwgpF8PFxPl?= =?us-ascii?Q?TdW7v+v+9RSz9TP0eqsiy4iz1u39FJ96iJwhkqo8AcDnKX+dYA/x9g0fwMG/?= =?us-ascii?Q?loOwwxyyn8WwSAYSpGxQuLwJFtfxTH5bid+JqucfMFda+047HJFwfU0+ABiY?= =?us-ascii?Q?a5uLaZeiSZZiq4WVUp6Eycj9ffYAAqoe1LPfV2TsyxILwCrOl8Rzho13QZdq?= =?us-ascii?Q?uC/z+aGOGd/oW9WYACv3O1YbhITzKK/JXsbU/drOFbnQK3TbpquSrHDjBJLR?= =?us-ascii?Q?qzCiPdDTmekhOEmUO8ofLykw0sDQquOHvGV+NbYzaBkZHf4m1pxzbncxnwNb?= =?us-ascii?Q?6FlM/SnpVQSwm6rFRTWlSmj2XyZOI49/Vi8NEmxfH2Vb9Tx+MXHeFpWrDABJ?= =?us-ascii?Q?Tx5z09vSTfgp465hqUWr7q639WMRnqSPiGNTCoF8V1YWvzO4tSceslPku8z1?= =?us-ascii?Q?c7wP7VHffgAdW4ww9zkt6dj5qDE1B9kGedOxgPJrQIxXVw7n+LUwlHLCaARK?= =?us-ascii?Q?EQ+ibSwNW3nZ9VSOg/OfA5SgmAOrtjENtgMKiCb9TD4MSgFR8k0724F5vd3m?= =?us-ascii?Q?9t48jQnq9rGhDtE0M7AmvTw2fGPEiZesL5zLeZZhT2bZgnsDo7Sa9/8OFw3i?= =?us-ascii?Q?ubD01b2sf/ZA5KsMNhMyCvLVotbzO7Tjf1/lZntgBwU7b77lIk9hGUZgis73?= =?us-ascii?Q?T5L8pv4ydMJliQ4B9SzK1jbgy/R6R8vd9z1BlP1e0YmylRPMGKsIEjDvg7q/?= =?us-ascii?Q?efGgYZ0AENZpXDCxw+XkWohSCP/VnnqCAR5RNeakhxwgFkAarm4nC22uwxaK?= =?us-ascii?Q?mXqM9BV6SE7ZNvIelZHerj13f+bWImsnGeE2+KpH32R1cHFKgIQkXy5qbkF3?= =?us-ascii?Q?qmT/KIpxuEY0UlAI9aV5lc/pv8hvBJUoJGmYaxtp5oQnqpJu91JUOSq8Nxky?= =?us-ascii?Q?jHVXoFxeetz4ZA40UcJC/neTIdDajVN7yzEXiDrv+4DlGH30V/j0ENvmrVjL?= =?us-ascii?Q?/OZvmvv/bruO1kCmlHisQ1KWcOATlDsbRvja/fO+nHXh9rQaFZxLpSOrLuw0?= =?us-ascii?Q?P2F2yFvtuD1xH/3maPCtF3LKj1t3sc4KMNXFNIvcLrLrnW9jWsrN?= X-OriginatorOrg: valinux.co.jp X-MS-Exchange-CrossTenant-Network-Message-Id: e9aa5c34-ef8d-4cdb-72d4-08deb703541b X-MS-Exchange-CrossTenant-AuthSource: TY7P286MB7722.JPNP286.PROD.OUTLOOK.COM X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 21 May 2026 06:36:45.8872 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 7a57bee8-f73d-4c5f-a4f7-d72c91c8c111 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 1QAvt+y4YSHRpzfY/aO21B1/P9sXyRG7e34l+JK8Ba98RYrZKDiFHwfxL/l6dWq5hNxDDZasGZfGw+w4bz9u4Q== X-MS-Exchange-Transport-CrossTenantHeadersStamped: TY6P286MB7399 Content-Type: text/plain; charset="utf-8" Add pci-epf-dma, an endpoint function that exposes selected endpoint-integrated DMA channels as a separate PCI DMA controller function. The function consumes EPC auxiliary DMA resources, publishes a stable metadata BAR for host discovery, and uses a DMA window BAR for DMA resources that are not already host-visible. After the host-side driver finds the metadata and requests the final layout, the endpoint function programs DMA window BAR submaps and marks the metadata ready. The endpoint function does not bake in a vendor/device ID. As with other generic endpoint functions, users provide the PCI IDs through the common EPF configfs header attributes. Signed-off-by: Koichiro Den --- drivers/pci/endpoint/functions/Kconfig | 14 + drivers/pci/endpoint/functions/Makefile | 1 + drivers/pci/endpoint/functions/pci-epf-dma.c | 1361 ++++++++++++++++++ 3 files changed, 1376 insertions(+) create mode 100644 drivers/pci/endpoint/functions/pci-epf-dma.c diff --git a/drivers/pci/endpoint/functions/Kconfig b/drivers/pci/endpoint/= functions/Kconfig index bb5a23994288..078ac19dc772 100644 --- a/drivers/pci/endpoint/functions/Kconfig +++ b/drivers/pci/endpoint/functions/Kconfig @@ -39,6 +39,20 @@ config PCI_EPF_VNTB =20 If in doubt, say "N" to disable Endpoint NTB driver. =20 +config PCI_EPF_DMA + tristate "PCI Endpoint DMA driver" + depends on PCI_ENDPOINT + select CONFIGFS_FS + select DMA_ENGINE + help + Select this configuration option to expose an endpoint-integrated + DMA controller as a PCI endpoint function. The function advertises + the DMA controller layout to the host using BAR-resident metadata + and maps resources that are not already host-visible into the + DMA window BAR. + + If in doubt, say "N" to disable Endpoint DMA driver. + config PCI_EPF_MHI tristate "PCI Endpoint driver for MHI bus" depends on PCI_ENDPOINT && MHI_BUS_EP diff --git a/drivers/pci/endpoint/functions/Makefile b/drivers/pci/endpoint= /functions/Makefile index 696473fce50e..de92f6897b8f 100644 --- a/drivers/pci/endpoint/functions/Makefile +++ b/drivers/pci/endpoint/functions/Makefile @@ -6,4 +6,5 @@ obj-$(CONFIG_PCI_EPF_TEST) +=3D pci-epf-test.o obj-$(CONFIG_PCI_EPF_NTB) +=3D pci-epf-ntb.o obj-$(CONFIG_PCI_EPF_VNTB) +=3D pci-epf-vntb.o +obj-$(CONFIG_PCI_EPF_DMA) +=3D pci-epf-dma.o obj-$(CONFIG_PCI_EPF_MHI) +=3D pci-epf-mhi.o diff --git a/drivers/pci/endpoint/functions/pci-epf-dma.c b/drivers/pci/end= point/functions/pci-epf-dma.c new file mode 100644 index 000000000000..d7761966eca2 --- /dev/null +++ b/drivers/pci/endpoint/functions/pci-epf-dma.c @@ -0,0 +1,1361 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * PCI endpoint function that exposes an endpoint-integrated DMA controller + * to the PCI host. + * + * The host-side dw-edma-pcie driver consumes the BAR metadata published + * by this function. + */ + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +/* HOST_REQ is set by the host driver, so poll it at a low idle rate. */ +#define PCI_EPF_DMA_HOST_REQ_POLL_MS 500 + +struct pci_epf_dma_bar_map { + const struct pci_epc_aux_resource *res; + enum pci_barno bar; + u64 res_offset_in_bar; + u64 submap_offset_in_bar; + dma_addr_t phys_addr; + size_t map_size; + bool needs_submap; +}; + +struct pci_epf_dma { + struct pci_epf *epf; + struct config_group group; + struct delayed_work map_work; + + enum pci_barno metadata_bar; + enum pci_barno dma_window_bar; + u16 wr_chans; + u16 rd_chans; + u8 reg_layout; + u8 reg_layout_data; + + /* Backing storage for ctrl and descriptor resource pointers. */ + struct pci_epc_aux_resource *resources; + unsigned int num_resources; + const struct pci_epc_aux_resource *ctrl; + const struct pci_epc_aux_resource *ep_to_rc_desc[EDMA_MAX_WR_CH]; + const struct pci_epc_aux_resource *rc_to_ep_desc[EDMA_MAX_RD_CH]; + + /* Local DMAengine reservations for channels delegated to the host. */ + struct dma_chan *ep_to_rc_chan[EDMA_MAX_WR_CH]; + struct dma_chan *rc_to_ep_chan[EDMA_MAX_RD_CH]; + + void *metadata_addr; + void *dma_window_addr; + size_t msix_table_offset; + struct pci_epf_dma_bar_map *bar_maps; + unsigned int num_bar_maps; + struct pci_epf_bar_submap *submaps; + unsigned int num_submaps; + + /* Cleared when a later event should retry programming the submaps. */ + bool submaps_programmed; +}; + +#define to_epf_dma(epf_group) container_of((epf_group), struct pci_epf_dma= , group) + +static struct pci_epf_header pci_epf_dma_header =3D { + .vendorid =3D PCI_ANY_ID, + .deviceid =3D PCI_ANY_ID, + .baseclass_code =3D PCI_BASE_CLASS_SYSTEM, + .subclass_code =3D PCI_CLASS_SYSTEM_DMA & 0xff, + .interrupt_pin =3D PCI_INTERRUPT_INTA, +}; + +static void pci_epf_dma_release_channels(struct pci_epf_dma *epf_dma) +{ + unsigned int i; + + for (i =3D 0; i < ARRAY_SIZE(epf_dma->ep_to_rc_chan); i++) { + if (!epf_dma->ep_to_rc_chan[i]) + continue; + + dma_release_channel(epf_dma->ep_to_rc_chan[i]); + epf_dma->ep_to_rc_chan[i] =3D NULL; + } + + for (i =3D 0; i < ARRAY_SIZE(epf_dma->rc_to_ep_chan); i++) { + if (!epf_dma->rc_to_ep_chan[i]) + continue; + + dma_release_channel(epf_dma->rc_to_ep_chan[i]); + epf_dma->rc_to_ep_chan[i] =3D NULL; + } +} + +static int pci_epf_dma_claim_channel(struct pci_epf_dma *epf_dma, + const struct pci_epc_aux_resource *res, + struct dma_chan **chan) +{ + struct device *dev =3D &epf_dma->epf->dev; + struct dma_chan *dma_chan; + + if (!res->u.dma_desc.dma_chan) { + dev_err(dev, "DMA channel %u cannot be reserved\n", + res->u.dma_desc.hw_ch); + return -EOPNOTSUPP; + } + + dma_chan =3D dma_get_slave_channel(res->u.dma_desc.dma_chan); + if (!dma_chan) { + dev_err(dev, "DMA channel %u is already in use\n", + res->u.dma_desc.hw_ch); + return -EBUSY; + } + + *chan =3D dma_chan; + + return 0; +} + +static int +pci_epf_dma_validate_dw_edma_ctrl(struct pci_epf_dma *epf_dma, + const struct pci_epc_aux_resource *ctrl) +{ + struct device *dev =3D &epf_dma->epf->dev; + enum dw_edma_map_format map =3D ctrl->u.dma_ctrl.reg_layout_data; + u16 total_wr_chans =3D ctrl->u.dma_ctrl.ep_to_rc_ch_cnt; + u16 total_rd_chans =3D ctrl->u.dma_ctrl.rc_to_ep_ch_cnt; + + switch (map) { + case EDMA_MF_EDMA_LEGACY: + dev_err(dev, "legacy DesignWare eDMA layout cannot be delegated\n"); + return -EOPNOTSUPP; + case EDMA_MF_EDMA_UNROLL: + case EDMA_MF_HDMA_COMPAT: + if ((epf_dma->wr_chans && epf_dma->wr_chans !=3D total_wr_chans) || + (epf_dma->rd_chans && epf_dma->rd_chans !=3D total_rd_chans)) { + dev_err(dev, "DesignWare eDMA v0 delegation must cover the whole direct= ion\n"); + return -EOPNOTSUPP; + } + return 0; + case EDMA_MF_HDMA_NATIVE: + dev_err(dev, "DesignWare HDMA native layout cannot be delegated\n"); + return -EOPNOTSUPP; + default: + return -EINVAL; + } +} + +static bool pci_epf_dma_bar_usable(const struct pci_epc_features *epc_feat= ures, + enum pci_barno bar) +{ + if (bar < BAR_0 || bar >=3D PCI_STD_NUM_BARS) + return false; + + return epc_features->bar[bar].type !=3D BAR_RESERVED && + epc_features->bar[bar].type !=3D BAR_DISABLED; +} + +static bool pci_epf_dma_bar_has_fixed_resource(struct pci_epf_dma *epf_dma, + enum pci_barno bar) +{ + unsigned int i; + + for (i =3D 0; i < epf_dma->num_resources; i++) { + if (epf_dma->resources[i].bar =3D=3D bar) + return true; + } + + return false; +} + +static enum pci_barno +pci_epf_dma_first_usable_bar(struct pci_epf_dma *epf_dma, + const struct pci_epc_features *epc_features, + enum pci_barno exclude) +{ + enum pci_barno bar; + + for (bar =3D BAR_0; bar < PCI_STD_NUM_BARS; bar++) { + bar =3D pci_epc_get_next_free_bar(epc_features, bar); + if (bar =3D=3D NO_BAR) + return NO_BAR; + if (bar !=3D exclude && + !pci_epf_dma_bar_has_fixed_resource(epf_dma, bar)) + return bar; + } + + return NO_BAR; +} + +static size_t pci_epf_dma_align_size(size_t size, size_t align) +{ + if (!align) + return size; + + return ALIGN(size, align); +} + +static int pci_epf_dma_reuse_submap(struct pci_epf_dma *epf_dma, + unsigned int map_count, + dma_addr_t phys_addr, size_t map_size, + size_t offset, size_t *next_offset_in_bar, + u64 *res_offset_in_bar) +{ + struct pci_epf_dma_bar_map *map; + u64 delta; + size_t merged_size, next; + u64 res_map_end, submap_bar_end, submap_phys_end; + unsigned int i; + + if (check_add_overflow(phys_addr, map_size, &res_map_end)) + return -EOVERFLOW; + + for (i =3D 0; i < map_count; i++) { + map =3D &epf_dma->bar_maps[i]; + if (!map->needs_submap || map->bar !=3D epf_dma->dma_window_bar) + continue; + + if (check_add_overflow(map->phys_addr, map->map_size, + &submap_phys_end) || + check_add_overflow(map->submap_offset_in_bar, + map->map_size, &submap_bar_end)) + return -EOVERFLOW; + + /* + * Reuse a submap that already covers this aligned resource + * window. + */ + if (phys_addr >=3D map->phys_addr && + res_map_end <=3D submap_phys_end) { + if (check_add_overflow(phys_addr - map->phys_addr, + offset, &delta) || + check_add_overflow(map->submap_offset_in_bar, + delta, res_offset_in_bar)) + return -EOVERFLOW; + return 1; + } + + /* + * Extend only the BAR-tail submap when the physical ranges are + * contiguous. + */ + if (submap_phys_end =3D=3D phys_addr && + submap_bar_end =3D=3D *next_offset_in_bar) { + if (check_add_overflow(map->map_size, map_size, + &merged_size) || + check_add_overflow(*next_offset_in_bar, map_size, + &next) || + check_add_overflow(*next_offset_in_bar, offset, + res_offset_in_bar)) + return -EOVERFLOW; + + map->map_size =3D merged_size; + *next_offset_in_bar =3D next; + return 1; + } + } + + return 0; +} + +static int pci_epf_dma_add_map(struct pci_epf_dma *epf_dma, + const struct pci_epc_aux_resource *res, + size_t align, size_t *next_offset_in_bar, + unsigned int *map_idx) +{ + dma_addr_t phys_addr; + size_t map_size, offset =3D 0, next; + u64 res_offset_in_bar; + int ret; + + if (!res || !res->size) + return -EINVAL; + + if (res->bar !=3D NO_BAR) { + if (res->bar < BAR_0 || res->bar >=3D PCI_STD_NUM_BARS) + return -EINVAL; + if (res->bar =3D=3D epf_dma->metadata_bar || + res->bar =3D=3D epf_dma->dma_window_bar) + return -EINVAL; + + epf_dma->bar_maps[*map_idx] =3D (struct pci_epf_dma_bar_map) { + .res =3D res, + .bar =3D res->bar, + .res_offset_in_bar =3D res->bar_offset, + .map_size =3D res->size, + }; + (*map_idx)++; + + return 0; + } + + if (epf_dma->dma_window_bar =3D=3D NO_BAR) + return -EOPNOTSUPP; + + phys_addr =3D res->phys_addr; + /* Map the aligned window that contains this resource. */ + if (align) { + phys_addr =3D ALIGN_DOWN(res->phys_addr, align); + offset =3D res->phys_addr - phys_addr; + } + + if (check_add_overflow(res->size, offset, &map_size)) + return -EOVERFLOW; + map_size =3D pci_epf_dma_align_size(map_size, align); + + ret =3D pci_epf_dma_reuse_submap(epf_dma, *map_idx, phys_addr, map_size, + offset, next_offset_in_bar, + &res_offset_in_bar); + if (ret < 0) + return ret; + if (ret) { + epf_dma->bar_maps[*map_idx] =3D (struct pci_epf_dma_bar_map) { + .res =3D res, + .bar =3D epf_dma->dma_window_bar, + .res_offset_in_bar =3D res_offset_in_bar, + .phys_addr =3D res->phys_addr, + .map_size =3D res->size, + }; + + (*map_idx)++; + + return 0; + } + + if (check_add_overflow(*next_offset_in_bar, map_size, &next)) + return -EOVERFLOW; + if (check_add_overflow(*next_offset_in_bar, offset, &res_offset_in_bar)) + return -EOVERFLOW; + + epf_dma->bar_maps[*map_idx] =3D (struct pci_epf_dma_bar_map) { + .res =3D res, + .bar =3D epf_dma->dma_window_bar, + .res_offset_in_bar =3D res_offset_in_bar, + .submap_offset_in_bar =3D *next_offset_in_bar, + .phys_addr =3D phys_addr, + .map_size =3D map_size, + .needs_submap =3D true, + }; + + *next_offset_in_bar =3D next; + (*map_idx)++; + + return 0; +} + +static const struct pci_epf_dma_bar_map * +pci_epf_dma_find_map(struct pci_epf_dma *epf_dma, + const struct pci_epc_aux_resource *res) +{ + unsigned int i; + + for (i =3D 0; i < epf_dma->num_bar_maps; i++) { + if (epf_dma->bar_maps[i].res =3D=3D res) + return &epf_dma->bar_maps[i]; + } + + return NULL; +} + +static bool pci_epf_dma_needs_dma_window(struct pci_epf_dma *epf_dma) +{ + unsigned int i; + + if (epf_dma->ctrl && epf_dma->ctrl->bar =3D=3D NO_BAR) + return true; + + for (i =3D 0; i < epf_dma->wr_chans; i++) { + if (epf_dma->ep_to_rc_desc[i] && + epf_dma->ep_to_rc_desc[i]->bar =3D=3D NO_BAR) + return true; + } + + for (i =3D 0; i < epf_dma->rd_chans; i++) { + if (epf_dma->rc_to_ep_desc[i] && + epf_dma->rc_to_ep_desc[i]->bar =3D=3D NO_BAR) + return true; + } + + return false; +} + +static int pci_epf_dma_collect_resources(struct pci_epf_dma *epf_dma) +{ + const struct pci_epc_aux_resource *ep_to_rc_desc[EDMA_MAX_WR_CH] =3D {}; + const struct pci_epc_aux_resource *rc_to_ep_desc[EDMA_MAX_RD_CH] =3D {}; + const struct pci_epc_aux_resource *ctrl =3D NULL; + struct pci_epf *epf =3D epf_dma->epf; + struct pci_epc *epc =3D epf->epc; + struct device *dev =3D &epf->dev; + int count, i, ret; + + count =3D pci_epc_get_aux_resources_count(epc, epf->func_no, + epf->vfunc_no); + if (count <=3D 0) + return count ?: -ENODEV; + + struct pci_epc_aux_resource *res __free(kfree) =3D + kzalloc_objs(*res, count); + if (!res) + return -ENOMEM; + + ret =3D pci_epc_get_aux_resources(epc, epf->func_no, epf->vfunc_no, + res, count); + if (ret) + return ret; + + for (i =3D 0; i < count; i++) { + switch (res[i].type) { + case PCI_EPC_AUX_DMA_CTRL_MMIO: + if (ctrl) + return -EINVAL; + ctrl =3D &res[i]; + break; + case PCI_EPC_AUX_DMA_DESC_MEM: { + u16 hw_ch =3D res[i].u.dma_desc.hw_ch; + + switch (res[i].u.dma_desc.dir) { + case PCI_EPC_AUX_DMA_EP_TO_RC: + if (hw_ch >=3D EDMA_MAX_WR_CH || + ep_to_rc_desc[hw_ch]) + return -EINVAL; + ep_to_rc_desc[hw_ch] =3D &res[i]; + break; + case PCI_EPC_AUX_DMA_RC_TO_EP: + if (hw_ch >=3D EDMA_MAX_RD_CH || + rc_to_ep_desc[hw_ch]) + return -EINVAL; + rc_to_ep_desc[hw_ch] =3D &res[i]; + break; + default: + return -EINVAL; + } + break; + } + default: + continue; + } + } + + if (!ctrl) + return -ENODEV; + + if (!epf_dma->wr_chans && !epf_dma->rd_chans) + return -EINVAL; + + if (epf_dma->wr_chans > ctrl->u.dma_ctrl.ep_to_rc_ch_cnt || + epf_dma->rd_chans > ctrl->u.dma_ctrl.rc_to_ep_ch_cnt) + return -EINVAL; + + switch (ctrl->u.dma_ctrl.reg_layout) { + case PCI_EPC_AUX_DMA_REG_LAYOUT_DW_EDMA: + ret =3D pci_epf_dma_validate_dw_edma_ctrl(epf_dma, ctrl); + if (ret) + return ret; + epf_dma->reg_layout =3D PCI_EP_DMA_METADATA_REG_LAYOUT_DW_EDMA; + epf_dma->reg_layout_data =3D ctrl->u.dma_ctrl.reg_layout_data; + break; + default: + return -EOPNOTSUPP; + } + + for (i =3D 0; i < epf_dma->wr_chans; i++) { + if (!ep_to_rc_desc[i]) { + dev_err(dev, "missing dense write DMA channel %d\n", i); + return -EINVAL; + } + } + + for (i =3D 0; i < epf_dma->rd_chans; i++) { + if (!rc_to_ep_desc[i]) { + dev_err(dev, "missing dense read DMA channel %d\n", i); + return -EINVAL; + } + } + + for (i =3D 0; i < epf_dma->wr_chans; i++) { + ret =3D pci_epf_dma_claim_channel(epf_dma, ep_to_rc_desc[i], + &epf_dma->ep_to_rc_chan[i]); + if (ret) + goto err_release_channels; + } + + for (i =3D 0; i < epf_dma->rd_chans; i++) { + ret =3D pci_epf_dma_claim_channel(epf_dma, rc_to_ep_desc[i], + &epf_dma->rc_to_ep_chan[i]); + if (ret) + goto err_release_channels; + } + + epf_dma->resources =3D no_free_ptr(res); + epf_dma->num_resources =3D count; + epf_dma->ctrl =3D ctrl; + memcpy(epf_dma->ep_to_rc_desc, ep_to_rc_desc, sizeof(ep_to_rc_desc)); + memcpy(epf_dma->rc_to_ep_desc, rc_to_ep_desc, sizeof(rc_to_ep_desc)); + + return 0; + +err_release_channels: + pci_epf_dma_release_channels(epf_dma); + + return ret; +} + +static void pci_epf_dma_metadata_write(__le32 *metadata, u16 metadata_off, + u32 val) +{ + metadata[metadata_off / sizeof(*metadata)] =3D cpu_to_le32(val); +} + +static void pci_epf_dma_metadata_write64(__le32 *metadata, u16 metadata_of= f, + u64 val) +{ + pci_epf_dma_metadata_write(metadata, metadata_off, lower_32_bits(val)); + pci_epf_dma_metadata_write(metadata, metadata_off + sizeof(u32), + upper_32_bits(val)); +} + +static int pci_epf_dma_build_ch_entry(const struct pci_epf_dma_bar_map *ma= p, + __le32 *metadata, u16 entry) +{ + const struct pci_epc_aux_resource *res =3D map->res; + u32 ctrl; + + if (res->size > U32_MAX) + return -EOVERFLOW; + + ctrl =3D FIELD_PREP(PCI_EP_DMA_METADATA_CH_CTRL_HW_CH, + res->u.dma_desc.hw_ch) | + FIELD_PREP(PCI_EP_DMA_METADATA_CH_CTRL_DESC_BAR, map->bar); + + pci_epf_dma_metadata_write(metadata, entry + PCI_EP_DMA_METADATA_CH_CTRL, + ctrl); + pci_epf_dma_metadata_write64(metadata, + entry + PCI_EP_DMA_METADATA_CH_DESC_OFF_LO, + map->res_offset_in_bar); + pci_epf_dma_metadata_write(metadata, + entry + PCI_EP_DMA_METADATA_CH_DESC_SIZE, + (u32)res->size); + pci_epf_dma_metadata_write64(metadata, + entry + PCI_EP_DMA_METADATA_CH_DESC_ADDR_LO, + res->phys_addr); + + return 0; +} + +static void pci_epf_dma_set_metadata_ready(struct pci_epf_dma *epf_dma, + bool ready) +{ + __le32 *metadata =3D epf_dma->metadata_addr; + __le32 *ctrl_ptr; + u32 ctrl; + + if (!metadata) + return; + + ctrl_ptr =3D &metadata[PCI_EP_DMA_METADATA_CTRL / sizeof(*metadata)]; + ctrl =3D le32_to_cpu(READ_ONCE(*ctrl_ptr)); + if (ready) { + dma_wmb(); + ctrl |=3D PCI_EP_DMA_METADATA_CTRL_READY; + } else { + ctrl &=3D ~PCI_EP_DMA_METADATA_CTRL_READY; + } + WRITE_ONCE(*ctrl_ptr, cpu_to_le32(ctrl)); +} + +static bool pci_epf_dma_metadata_host_requested(struct pci_epf_dma *epf_dm= a) +{ + __le32 *metadata =3D epf_dma->metadata_addr; + u32 ctrl; + + if (!metadata) + return false; + + ctrl =3D le32_to_cpu(READ_ONCE(metadata[PCI_EP_DMA_METADATA_CTRL / + sizeof(*metadata)])); + + return ctrl & PCI_EP_DMA_METADATA_CTRL_HOST_REQ; +} + +static void pci_epf_dma_clear_metadata_status(struct pci_epf_dma *epf_dma) +{ + __le32 *metadata =3D epf_dma->metadata_addr; + __le32 *ctrl_ptr; + u32 ctrl; + + if (!metadata) + return; + + ctrl_ptr =3D &metadata[PCI_EP_DMA_METADATA_CTRL / sizeof(*metadata)]; + ctrl =3D le32_to_cpu(READ_ONCE(*ctrl_ptr)); + ctrl &=3D ~(PCI_EP_DMA_METADATA_CTRL_HOST_REQ | + PCI_EP_DMA_METADATA_CTRL_READY); + WRITE_ONCE(*ctrl_ptr, cpu_to_le32(ctrl)); +} + +static int pci_epf_dma_build_metadata(struct pci_epf_dma *epf_dma) +{ + const struct pci_epf_dma_bar_map *ctrl_map; + u16 entry_size =3D PCI_EP_DMA_METADATA_CH_ENTRY_SIZE; + u16 wr_table, rd_table, total_len; + __le32 *metadata =3D epf_dma->metadata_addr; + unsigned int i; + int ret; + + if (!metadata) + return -EINVAL; + + ctrl_map =3D pci_epf_dma_find_map(epf_dma, epf_dma->ctrl); + if (!ctrl_map) + return -EINVAL; + if (epf_dma->wr_chans > FIELD_MAX(PCI_EP_DMA_METADATA_CTRL_WR_CH_COUNT) || + epf_dma->rd_chans > FIELD_MAX(PCI_EP_DMA_METADATA_CTRL_RD_CH_COUNT) || + entry_size > FIELD_MAX(PCI_EP_DMA_METADATA_CTRL_CH_ENTRY_SIZE) || + ctrl_map->res->size > U32_MAX) + return -EOVERFLOW; + + wr_table =3D epf_dma->wr_chans ? PCI_EP_DMA_METADATA_HDR_LEN : 0; + rd_table =3D epf_dma->rd_chans ? + PCI_EP_DMA_METADATA_HDR_LEN + epf_dma->wr_chans * entry_size : 0; + total_len =3D PCI_EP_DMA_METADATA_HDR_LEN + + (epf_dma->wr_chans + epf_dma->rd_chans) * entry_size; + + memset(metadata, 0, total_len); + + pci_epf_dma_metadata_write(metadata, 0, PCI_EP_DMA_METADATA_MAGIC); + pci_epf_dma_metadata_write(metadata, PCI_EP_DMA_METADATA_HDR, + FIELD_PREP(PCI_EP_DMA_METADATA_HDR_REV, + PCI_EP_DMA_METADATA_REV) | + FIELD_PREP(PCI_EP_DMA_METADATA_HDR_LEN_FIELD, + total_len)); + pci_epf_dma_metadata_write(metadata, PCI_EP_DMA_METADATA_CTRL, + FIELD_PREP(PCI_EP_DMA_METADATA_CTRL_REG_BAR, + ctrl_map->bar) | + FIELD_PREP(PCI_EP_DMA_METADATA_CTRL_WR_CH_COUNT, + epf_dma->wr_chans) | + FIELD_PREP(PCI_EP_DMA_METADATA_CTRL_RD_CH_COUNT, + epf_dma->rd_chans) | + FIELD_PREP(PCI_EP_DMA_METADATA_CTRL_CH_ENTRY_SIZE, + entry_size)); + pci_epf_dma_metadata_write64(metadata, + PCI_EP_DMA_METADATA_REG_OFF_LO, + ctrl_map->res_offset_in_bar); + pci_epf_dma_metadata_write(metadata, PCI_EP_DMA_METADATA_REG_LAYOUT, + FIELD_PREP(PCI_EP_DMA_METADATA_REG_LAYOUT_ID, + epf_dma->reg_layout) | + FIELD_PREP(PCI_EP_DMA_METADATA_REG_LAYOUT_DATA, + epf_dma->reg_layout_data)); + pci_epf_dma_metadata_write(metadata, PCI_EP_DMA_METADATA_REG_SIZE, + (u32)ctrl_map->res->size); + + for (i =3D 0; i < epf_dma->wr_chans; i++) { + const struct pci_epf_dma_bar_map *map; + + map =3D pci_epf_dma_find_map(epf_dma, + epf_dma->ep_to_rc_desc[i]); + if (!map) + return -EINVAL; + ret =3D pci_epf_dma_build_ch_entry(map, metadata, + wr_table + i * entry_size); + if (ret) + return ret; + } + + for (i =3D 0; i < epf_dma->rd_chans; i++) { + const struct pci_epf_dma_bar_map *map; + + map =3D pci_epf_dma_find_map(epf_dma, + epf_dma->rc_to_ep_desc[i]); + if (!map) + return -EINVAL; + ret =3D pci_epf_dma_build_ch_entry(map, metadata, + rd_table + i * entry_size); + if (ret) + return ret; + } + + return 0; +} + +static int pci_epf_dma_reserve_msix(struct pci_epf_dma *epf_dma, + const struct pci_epc_features *epc_features, + size_t *backing_size) +{ + struct pci_epf *epf =3D epf_dma->epf; + size_t msix_table_size, pba_size, next; + unsigned int nvec =3D epf->msix_interrupts; + + epf_dma->msix_table_offset =3D 0; + + if (!epc_features->msix_capable || !nvec) + return 0; + + next =3D ALIGN(*backing_size, 8); + if (next > U32_MAX) + return -EOVERFLOW; + epf_dma->msix_table_offset =3D next; + + if (check_mul_overflow(PCI_MSIX_ENTRY_SIZE, nvec, &msix_table_size)) + return -EOVERFLOW; + + pba_size =3D ALIGN(DIV_ROUND_UP(nvec, 8), 8); + if (check_add_overflow(next, msix_table_size, &next) || + next > U32_MAX || + check_add_overflow(next, pba_size, &next)) + return -EOVERFLOW; + + *backing_size =3D next; + + return 0; +} + +static int pci_epf_dma_build_layout(struct pci_epf_dma *epf_dma, + const struct pci_epc_features *epc_features) +{ + struct pci_epf *epf =3D epf_dma->epf; + struct device *dev =3D &epf->dev; + struct pci_epf_bar *bar; + unsigned int max_maps, map_idx =3D 0, sub_idx =3D 0; + size_t align =3D epc_features->align; + size_t metadata_size, metadata_backing_size, metadata_bar_size; + size_t mapped_size =3D 0, dma_window_bar_size; + int i, ret; + + metadata_size =3D PCI_EP_DMA_METADATA_HDR_LEN; + metadata_size +=3D (epf_dma->wr_chans + epf_dma->rd_chans) * + PCI_EP_DMA_METADATA_CH_ENTRY_SIZE; + metadata_backing_size =3D metadata_size; + ret =3D pci_epf_dma_reserve_msix(epf_dma, epc_features, + &metadata_backing_size); + if (ret) + return ret; + metadata_bar_size =3D pci_epf_dma_align_size(metadata_backing_size, + align); + + epf_dma->metadata_addr =3D pci_epf_alloc_space(epf, metadata_bar_size, + epf_dma->metadata_bar, + epc_features, + PRIMARY_INTERFACE); + if (!epf_dma->metadata_addr) { + dev_err(dev, "failed to allocate BAR%d metadata space\n", + epf_dma->metadata_bar); + return -ENOMEM; + } + memset(epf_dma->metadata_addr, 0, epf->bar[epf_dma->metadata_bar].size); + + /* One map for DMA controller registers, plus one per channel. */ + max_maps =3D 1 + epf_dma->wr_chans + epf_dma->rd_chans; + epf_dma->bar_maps =3D kzalloc_objs(*epf_dma->bar_maps, max_maps); + if (!epf_dma->bar_maps) + return -ENOMEM; + + ret =3D pci_epf_dma_add_map(epf_dma, epf_dma->ctrl, align, + &mapped_size, &map_idx); + if (ret) + return ret; + + for (i =3D 0; i < epf_dma->wr_chans; i++) { + ret =3D pci_epf_dma_add_map(epf_dma, + epf_dma->ep_to_rc_desc[i], align, + &mapped_size, &map_idx); + if (ret) + return ret; + } + + for (i =3D 0; i < epf_dma->rd_chans; i++) { + ret =3D pci_epf_dma_add_map(epf_dma, + epf_dma->rc_to_ep_desc[i], align, + &mapped_size, &map_idx); + if (ret) + return ret; + } + + epf_dma->num_bar_maps =3D map_idx; + + ret =3D pci_epf_dma_build_metadata(epf_dma); + if (ret) + return ret; + + /* Some DMA resources may already be visible through another map. */ + for (i =3D 0; i < epf_dma->num_bar_maps; i++) { + if (epf_dma->bar_maps[i].needs_submap) + epf_dma->num_submaps++; + } + if (!epf_dma->num_submaps) + return 0; + + dma_window_bar_size =3D mapped_size; + epf_dma->dma_window_addr =3D + pci_epf_alloc_space(epf, dma_window_bar_size, + epf_dma->dma_window_bar, epc_features, + PRIMARY_INTERFACE); + if (!epf_dma->dma_window_addr) { + dev_err(dev, "failed to allocate BAR%d DMA window space\n", + epf_dma->dma_window_bar); + return -ENOMEM; + } + bar =3D &epf->bar[epf_dma->dma_window_bar]; + memset(epf_dma->dma_window_addr, 0, bar->size); + + if (bar->size > mapped_size) + epf_dma->num_submaps++; + + epf_dma->submaps =3D kzalloc_objs(*epf_dma->submaps, epf_dma->num_submaps= ); + if (!epf_dma->submaps) + return -ENOMEM; + + for (i =3D 0; i < epf_dma->num_bar_maps; i++) { + if (!epf_dma->bar_maps[i].needs_submap) + continue; + + epf_dma->submaps[sub_idx++] =3D (struct pci_epf_bar_submap) { + .phys_addr =3D epf_dma->bar_maps[i].phys_addr, + .size =3D epf_dma->bar_maps[i].map_size, + }; + } + + /* Cover any BAR tail padding with the allocated scratch space. */ + if (bar->size > mapped_size) { + epf_dma->submaps[sub_idx++] =3D (struct pci_epf_bar_submap) { + .phys_addr =3D bar->phys_addr + mapped_size, + .size =3D bar->size - mapped_size, + }; + } + + return 0; +} + +static void pci_epf_dma_free_layout(struct pci_epf_dma *epf_dma) +{ + struct pci_epf *epf =3D epf_dma->epf; + struct pci_epf_bar *bar; + + if (epf_dma->dma_window_addr) { + bar =3D &epf->bar[epf_dma->dma_window_bar]; + bar->submap =3D NULL; + bar->num_submap =3D 0; + } + epf_dma->submaps_programmed =3D false; + + kfree(epf_dma->submaps); + epf_dma->submaps =3D NULL; + epf_dma->num_submaps =3D 0; + + kfree(epf_dma->bar_maps); + epf_dma->bar_maps =3D NULL; + epf_dma->num_bar_maps =3D 0; + + pci_epf_dma_release_channels(epf_dma); + + kfree(epf_dma->resources); + epf_dma->resources =3D NULL; + epf_dma->num_resources =3D 0; + epf_dma->ctrl =3D NULL; + memset(epf_dma->ep_to_rc_desc, 0, sizeof(epf_dma->ep_to_rc_desc)); + memset(epf_dma->rc_to_ep_desc, 0, sizeof(epf_dma->rc_to_ep_desc)); + + if (epf_dma->dma_window_addr) { + pci_epf_free_space(epf, epf_dma->dma_window_addr, + epf_dma->dma_window_bar, + PRIMARY_INTERFACE); + epf_dma->dma_window_addr =3D NULL; + } + + if (epf_dma->metadata_addr) { + pci_epf_free_space(epf, epf_dma->metadata_addr, + epf_dma->metadata_bar, + PRIMARY_INTERFACE); + epf_dma->metadata_addr =3D NULL; + } + epf_dma->msix_table_offset =3D 0; +} + +static int pci_epf_dma_program_submaps(struct pci_epf_dma *epf_dma) +{ + struct pci_epf *epf =3D epf_dma->epf; + struct pci_epf_bar *bar; + int ret; + + if (!epf_dma->dma_window_addr) { + pci_epf_dma_set_metadata_ready(epf_dma, true); + return 0; + } + + if (epf_dma->submaps_programmed) + return 0; + + bar =3D &epf->bar[epf_dma->dma_window_bar]; + bar->submap =3D epf_dma->submaps; + bar->num_submap =3D epf_dma->num_submaps; + + ret =3D pci_epc_set_bar(epf->epc, epf->func_no, epf->vfunc_no, bar); + if (ret) { + bar->submap =3D NULL; + bar->num_submap =3D 0; + return ret; + } + + epf_dma->submaps_programmed =3D true; + pci_epf_dma_set_metadata_ready(epf_dma, true); + + return 0; +} + +static void pci_epf_dma_map_work(struct work_struct *work) +{ + struct pci_epf_dma *epf_dma =3D + container_of(to_delayed_work(work), struct pci_epf_dma, + map_work); + struct pci_epf *epf =3D epf_dma->epf; + int ret; + + if (!epf->epc) + return; + + if (!epf->epc->init_complete) { + schedule_delayed_work(&epf_dma->map_work, + msecs_to_jiffies(PCI_EPF_DMA_HOST_REQ_POLL_MS)); + return; + } + + if (!pci_epf_dma_metadata_host_requested(epf_dma)) { + schedule_delayed_work(&epf_dma->map_work, + msecs_to_jiffies(PCI_EPF_DMA_HOST_REQ_POLL_MS)); + return; + } + + ret =3D pci_epf_dma_program_submaps(epf_dma); + if (ret) + dev_err(&epf->dev, "failed to program DMA window BAR submaps: %d\n", + ret); +} + +static int pci_epf_dma_epc_init(struct pci_epf *epf) +{ + struct pci_epf_dma *epf_dma =3D epf_get_drvdata(epf); + const struct pci_epc_features *epc_features; + struct pci_epc *epc =3D epf->epc; + struct device *dev =3D &epf->dev; + int ret; + + epc_features =3D pci_epc_get_features(epc, epf->func_no, epf->vfunc_no); + if (!epc_features) + return -EOPNOTSUPP; + + pci_epf_dma_clear_metadata_status(epf_dma); + + ret =3D pci_epc_write_header(epc, epf->func_no, epf->vfunc_no, + epf->header); + if (ret) { + dev_err(dev, "configuration header write failed\n"); + return ret; + } + + ret =3D pci_epc_set_bar(epc, epf->func_no, epf->vfunc_no, + &epf->bar[epf_dma->metadata_bar]); + if (ret) { + dev_err(dev, "BAR%d setup failed: %d\n", + epf_dma->metadata_bar, ret); + return ret; + } + + if (epf_dma->dma_window_addr) { + ret =3D pci_epc_set_bar(epc, epf->func_no, epf->vfunc_no, + &epf->bar[epf_dma->dma_window_bar]); + if (ret) { + dev_err(dev, "BAR%d setup failed: %d\n", + epf_dma->dma_window_bar, ret); + goto err_clear_metadata_bar; + } + } + + if (epc_features->msi_capable && epf->msi_interrupts) { + ret =3D pci_epc_set_msi(epc, epf->func_no, epf->vfunc_no, + epf->msi_interrupts); + if (ret) { + dev_err(dev, "MSI setup failed: %d\n", ret); + goto err_clear_dma_window_bar; + } + } + + if (epc_features->msix_capable && epf->msix_interrupts) { + ret =3D pci_epc_set_msix(epc, epf->func_no, epf->vfunc_no, + epf->msix_interrupts, + epf_dma->metadata_bar, + epf_dma->msix_table_offset); + if (ret) { + dev_err(dev, "MSI-X setup failed: %d\n", ret); + goto err_clear_dma_window_bar; + } + } + + schedule_delayed_work(&epf_dma->map_work, 0); + + return 0; + +err_clear_dma_window_bar: + if (epf_dma->dma_window_addr) + pci_epc_clear_bar(epc, epf->func_no, epf->vfunc_no, + &epf->bar[epf_dma->dma_window_bar]); +err_clear_metadata_bar: + pci_epc_clear_bar(epc, epf->func_no, epf->vfunc_no, + &epf->bar[epf_dma->metadata_bar]); + pci_epf_dma_clear_metadata_status(epf_dma); + + return ret; +} + +static void pci_epf_dma_epc_deinit(struct pci_epf *epf) +{ + struct pci_epf_dma *epf_dma =3D epf_get_drvdata(epf); + struct pci_epf_bar *bar; + + cancel_delayed_work_sync(&epf_dma->map_work); + + if (!epf_dma->metadata_addr) + return; + + pci_epf_dma_clear_metadata_status(epf_dma); + if (epf_dma->dma_window_addr) { + bar =3D &epf->bar[epf_dma->dma_window_bar]; + pci_epc_clear_bar(epf->epc, epf->func_no, epf->vfunc_no, bar); + bar->submap =3D NULL; + bar->num_submap =3D 0; + } + pci_epc_clear_bar(epf->epc, epf->func_no, epf->vfunc_no, + &epf->bar[epf_dma->metadata_bar]); + epf_dma->submaps_programmed =3D false; +} + +static int pci_epf_dma_link_up(struct pci_epf *epf) +{ + struct pci_epf_dma *epf_dma =3D epf_get_drvdata(epf); + + schedule_delayed_work(&epf_dma->map_work, 0); + + return 0; +} + +static int pci_epf_dma_link_down(struct pci_epf *epf) +{ + struct pci_epf_dma *epf_dma =3D epf_get_drvdata(epf); + + cancel_delayed_work_sync(&epf_dma->map_work); + pci_epf_dma_clear_metadata_status(epf_dma); + /* + * Link down can invalidate non-sticky inbound ATU state without going + * through pci_epc_clear_bar(). Keep the BAR/submap description intact, + * but force the next link-up path to reprogram the subrange mappings. + */ + epf_dma->submaps_programmed =3D false; + + return 0; +} + +static const struct pci_epc_event_ops pci_epf_dma_event_ops =3D { + .epc_init =3D pci_epf_dma_epc_init, + .epc_deinit =3D pci_epf_dma_epc_deinit, + .link_up =3D pci_epf_dma_link_up, + .link_down =3D pci_epf_dma_link_down, +}; + +static int pci_epf_dma_bind(struct pci_epf *epf) +{ + struct pci_epf_dma *epf_dma =3D epf_get_drvdata(epf); + const struct pci_epc_features *epc_features; + struct pci_epc *epc =3D epf->epc; + bool needs_dma_window; + int ret; + + if (WARN_ON_ONCE(!epc)) + return -EINVAL; + + epc_features =3D pci_epc_get_features(epc, epf->func_no, epf->vfunc_no); + if (!epc_features) + return -EOPNOTSUPP; + + if (!epc_features->msi_capable && !epc_features->msix_capable) + return -EOPNOTSUPP; + + if ((!epc_features->msi_capable || !epf->msi_interrupts) && + (!epc_features->msix_capable || !epf->msix_interrupts)) + return -EINVAL; + + ret =3D pci_epf_dma_collect_resources(epf_dma); + if (ret) + return ret; + + if (epf_dma->metadata_bar =3D=3D NO_BAR) + epf_dma->metadata_bar =3D + pci_epf_dma_first_usable_bar(epf_dma, epc_features, + NO_BAR); + + if (epf_dma->metadata_bar =3D=3D NO_BAR || + !pci_epf_dma_bar_usable(epc_features, epf_dma->metadata_bar) || + pci_epf_dma_bar_has_fixed_resource(epf_dma, epf_dma->metadata_bar)) { + ret =3D -EINVAL; + goto err_free; + } + + needs_dma_window =3D pci_epf_dma_needs_dma_window(epf_dma); + if (needs_dma_window) { + if (!epc_features->subrange_mapping || + !epc_features->dynamic_inbound_mapping) { + ret =3D -EOPNOTSUPP; + goto err_free; + } + + if (epf_dma->dma_window_bar =3D=3D NO_BAR) + epf_dma->dma_window_bar =3D + pci_epf_dma_first_usable_bar(epf_dma, epc_features, + epf_dma->metadata_bar); + if (epf_dma->dma_window_bar =3D=3D NO_BAR) { + ret =3D -EOPNOTSUPP; + goto err_free; + } + } + + if (epf_dma->dma_window_bar !=3D NO_BAR) { + if (!pci_epf_dma_bar_usable(epc_features, + epf_dma->dma_window_bar)) { + ret =3D -EINVAL; + goto err_free; + } + if (epf_dma->metadata_bar =3D=3D epf_dma->dma_window_bar || + pci_epf_dma_bar_has_fixed_resource(epf_dma, + epf_dma->dma_window_bar)) { + ret =3D -EINVAL; + goto err_free; + } + } + + ret =3D pci_epf_dma_build_layout(epf_dma, epc_features); + if (ret) + goto err_free; + + return 0; + +err_free: + pci_epf_dma_free_layout(epf_dma); + + return ret; +} + +static void pci_epf_dma_unbind(struct pci_epf *epf) +{ + struct pci_epf_dma *epf_dma =3D epf_get_drvdata(epf); + + cancel_delayed_work_sync(&epf_dma->map_work); + if (epf->epc && epf->epc->init_complete) + pci_epf_dma_epc_deinit(epf); + pci_epf_dma_free_layout(epf_dma); +} + +#define PCI_EPF_DMA_SHOW(_name, _fmt, _val) \ +static ssize_t pci_epf_dma_##_name##_show(struct config_item *item, \ + char *page) \ +{ \ + struct config_group *group =3D to_config_group(item); \ + struct pci_epf_dma *epf_dma =3D to_epf_dma(group); \ + \ + return sysfs_emit(page, _fmt "\n", (_val)); \ +} + +PCI_EPF_DMA_SHOW(metadata_bar, "%d", (int)epf_dma->metadata_bar) +PCI_EPF_DMA_SHOW(dma_window_bar, "%d", (int)epf_dma->dma_window_bar) + +static ssize_t pci_epf_dma_metadata_bar_store(struct config_item *item, co= nst char *page, + size_t len) +{ + struct config_group *group =3D to_config_group(item); + struct pci_epf_dma *epf_dma =3D to_epf_dma(group); + int bar, ret; + + if (epf_dma->epf->epc) + return -EOPNOTSUPP; + + ret =3D kstrtoint(page, 0, &bar); + if (ret) + return ret; + + if (bar !=3D NO_BAR && (bar < BAR_0 || bar >=3D PCI_STD_NUM_BARS)) + return -EINVAL; + if (bar !=3D NO_BAR && bar =3D=3D epf_dma->dma_window_bar) + return -EINVAL; + + epf_dma->metadata_bar =3D bar; + + return len; +} + +static ssize_t pci_epf_dma_dma_window_bar_store(struct config_item *item, + const char *page, size_t len) +{ + struct config_group *group =3D to_config_group(item); + struct pci_epf_dma *epf_dma =3D to_epf_dma(group); + int bar, ret; + + if (epf_dma->epf->epc) + return -EOPNOTSUPP; + + ret =3D kstrtoint(page, 0, &bar); + if (ret) + return ret; + + if (bar !=3D NO_BAR && (bar < BAR_0 || bar >=3D PCI_STD_NUM_BARS)) + return -EINVAL; + if (bar !=3D NO_BAR && bar =3D=3D epf_dma->metadata_bar) + return -EINVAL; + + epf_dma->dma_window_bar =3D bar; + + return len; +} + +PCI_EPF_DMA_SHOW(wr_chans, "%u", (unsigned int)epf_dma->wr_chans) + +static ssize_t pci_epf_dma_wr_chans_store(struct config_item *item, + const char *page, size_t len) +{ + struct config_group *group =3D to_config_group(item); + struct pci_epf_dma *epf_dma =3D to_epf_dma(group); + u16 val; + int ret; + + if (epf_dma->epf->epc) + return -EOPNOTSUPP; + + ret =3D kstrtou16(page, 0, &val); + if (ret) + return ret; + if (val > EDMA_MAX_WR_CH) + return -EINVAL; + + epf_dma->wr_chans =3D val; + + return len; +} + +PCI_EPF_DMA_SHOW(rd_chans, "%u", (unsigned int)epf_dma->rd_chans) + +static ssize_t pci_epf_dma_rd_chans_store(struct config_item *item, + const char *page, size_t len) +{ + struct config_group *group =3D to_config_group(item); + struct pci_epf_dma *epf_dma =3D to_epf_dma(group); + u16 val; + int ret; + + if (epf_dma->epf->epc) + return -EOPNOTSUPP; + + ret =3D kstrtou16(page, 0, &val); + if (ret) + return ret; + if (val > EDMA_MAX_RD_CH) + return -EINVAL; + + epf_dma->rd_chans =3D val; + + return len; +} + +CONFIGFS_ATTR(pci_epf_dma_, metadata_bar); +CONFIGFS_ATTR(pci_epf_dma_, dma_window_bar); +CONFIGFS_ATTR(pci_epf_dma_, wr_chans); +CONFIGFS_ATTR(pci_epf_dma_, rd_chans); + +static struct configfs_attribute *pci_epf_dma_attrs[] =3D { + &pci_epf_dma_attr_metadata_bar, + &pci_epf_dma_attr_dma_window_bar, + &pci_epf_dma_attr_wr_chans, + &pci_epf_dma_attr_rd_chans, + NULL, +}; + +static const struct config_item_type pci_epf_dma_group_type =3D { + .ct_attrs =3D pci_epf_dma_attrs, + .ct_owner =3D THIS_MODULE, +}; + +static struct config_group *pci_epf_dma_add_cfs(struct pci_epf *epf, + struct config_group *group) +{ + struct pci_epf_dma *epf_dma =3D epf_get_drvdata(epf); + struct config_group *epf_group =3D &epf_dma->group; + struct device *dev =3D &epf->dev; + + config_group_init_type_name(epf_group, dev_name(dev), + &pci_epf_dma_group_type); + + return epf_group; +} + +static const struct pci_epf_device_id pci_epf_dma_ids[] =3D { + { + .name =3D "pci_epf_dma", + }, + {}, +}; + +static int pci_epf_dma_probe(struct pci_epf *epf, + const struct pci_epf_device_id *id) +{ + struct pci_epf_dma *epf_dma; + + epf_dma =3D devm_kzalloc(&epf->dev, sizeof(*epf_dma), GFP_KERNEL); + if (!epf_dma) + return -ENOMEM; + + epf->header =3D &pci_epf_dma_header; + epf->event_ops =3D &pci_epf_dma_event_ops; + + epf_dma->epf =3D epf; + epf_dma->metadata_bar =3D NO_BAR; + epf_dma->dma_window_bar =3D NO_BAR; + INIT_DELAYED_WORK(&epf_dma->map_work, pci_epf_dma_map_work); + + epf_set_drvdata(epf, epf_dma); + + return 0; +} + +static const struct pci_epf_ops pci_epf_dma_ops =3D { + .unbind =3D pci_epf_dma_unbind, + .bind =3D pci_epf_dma_bind, + .add_cfs =3D pci_epf_dma_add_cfs, +}; + +static struct pci_epf_driver pci_epf_dma_driver =3D { + .driver.name =3D "pci_epf_dma", + .probe =3D pci_epf_dma_probe, + .id_table =3D pci_epf_dma_ids, + .ops =3D &pci_epf_dma_ops, + .owner =3D THIS_MODULE, +}; + +static int __init pci_epf_dma_init(void) +{ + return pci_epf_register_driver(&pci_epf_dma_driver); +} +module_init(pci_epf_dma_init); + +static void __exit pci_epf_dma_exit(void) +{ + pci_epf_unregister_driver(&pci_epf_dma_driver); +} +module_exit(pci_epf_dma_exit); + +MODULE_DESCRIPTION("PCI EPF DMA DRIVER"); +MODULE_AUTHOR("Koichiro Den "); +MODULE_LICENSE("GPL"); --=20 2.51.0 From nobody Sun May 24 21:40:06 2026 Received: from TYVP286CU001.outbound.protection.outlook.com (mail-japaneastazon11021086.outbound.protection.outlook.com [52.101.125.86]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id EE042395273; Thu, 21 May 2026 06:36:54 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=52.101.125.86 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779345417; cv=fail; b=ux4OGK0mW5pG29NOhQHQB07w9iEZOR16YxnF0tFOzNpoi6NHqpya9RU8itrYDp/m0j/nwxP1BzvZvrncdJSBmNdPZ9Y7cZXjTk7PaXd8PQ2K1a5pCCs9fvPSwnXp4A9SXYyVo+RtXCtJyBT7L5zJtScpAwFwzOsbGT+mv+mSPO8= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779345417; c=relaxed/simple; bh=xupSMoh7sJH84KOQ9LPKMbX2M5FVkLAxsVYm9kAjFnY=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: Content-Type:MIME-Version; b=MfLYvF9/UrGgdrtp8oJls0dGNpoTHWBtrTM+0rQnAG5e/dDcQndfBAEnCn6+TdYYCyQYP7Qy8dTlv+ajpDT5iFvRpGsOya6w5WYB3vG0rnGM7HTbjzoZAwHDQtkzciJAPYNMHr3hDQVsLpDaIcmqKDJTUVttICPIlxCtvFr1n7U= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=valinux.co.jp; spf=pass smtp.mailfrom=valinux.co.jp; dkim=pass (1024-bit key) header.d=valinux.co.jp header.i=@valinux.co.jp header.b=RHVFk94y; arc=fail smtp.client-ip=52.101.125.86 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=valinux.co.jp Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=valinux.co.jp Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=valinux.co.jp header.i=@valinux.co.jp header.b="RHVFk94y" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=OQMqovipy871YuovvkEZLCG9//WwhEzlqHAuaBM6rIBOEZgYTMODJKfDi2OdcLuLNUkFKmPpyAcuATneMTub+NyzdHoLQsd0lVpPKiP8zMNZ4ZiJXDWPDAq+2ODo/p6vQjpAli0+xpT3RwR8gw/CKy9a4zSOeteWyW8kirLoHnsN9VhEAAgPLEQU7tq0G2WCxj9XR74eKCWaIqeEqUOzFDp9FTtAZS0bd9y8ndEQpSHTiDy/4d5+XWPZl2DXljrOuy96XR6AIDLCj1dEV29klwrfpEtTs6WUovUqfmXw05feCL7re9Dczs1jTV4O0Gh8PbuPsPDfhiAGIdolU/fBzQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=h8Kd5PMKJ1p9ZX2WiTOp3478oz/MWFzaycfRVmo/Rio=; b=IAOGlt58JmTHm1wNnJpeB7tgbhsXNOR/vWJq2Z4QjDgiaTYQqjyVOZ/O2mdFIRlYnwgI3xI3vkBsBM/I2vMz9W0rFE5bl6RyM3QrAzS+d4cOP06/U6GRXx/V05sFzZttsyByUDFXNDMN8pRKkeu7u1Co8CXQPbaxaCeICnATvZLaDX5NAOS94mYl/xdvdMlqGV7F9XzMkYkV05p0BERDCUNkdMIvWiJgMkvoKP4F/2rlRSMTXgWYPSIPQbBbmJEXWi1Ec3F+jBHI70gRGs6B0Oc9sWORBFJEdZREN0ZkyjmW0RXPvokFO8UU8uqxyuf+DRpl4ZcRhqLK3Mk1rggngw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=valinux.co.jp; dmarc=pass action=none header.from=valinux.co.jp; dkim=pass header.d=valinux.co.jp; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=valinux.co.jp; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=h8Kd5PMKJ1p9ZX2WiTOp3478oz/MWFzaycfRVmo/Rio=; b=RHVFk94yrgvYB65/0whM/zenD23DBtuJ9XKLPeQy3zI8qeehM2Nv1MMhHK2f+J6a3N5JxCBwjX16L5/ZBPSTtFPdL6aiedKGPjd3+uPigAwywEzV1aiAIUfxugPV+pu7JyEY1l8Rw3QRvxrGqY3Wf+p9dhueXkNlF82gUf36+pI= Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=valinux.co.jp; Received: from TY7P286MB7722.JPNP286.PROD.OUTLOOK.COM (2603:1096:405:38f::10) by TY6P286MB7399.JPNP286.PROD.OUTLOOK.COM (2603:1096:405:35c::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.48.16; Thu, 21 May 2026 06:36:46 +0000 Received: from TY7P286MB7722.JPNP286.PROD.OUTLOOK.COM ([fe80::2305:327c:28ec:9b32]) by TY7P286MB7722.JPNP286.PROD.OUTLOOK.COM ([fe80::2305:327c:28ec:9b32%5]) with mapi id 15.21.0048.016; Thu, 21 May 2026 06:36:46 +0000 From: Koichiro Den To: Manivannan Sadhasivam , =?UTF-8?q?Krzysztof=20Wilczy=C5=84ski?= , Kishon Vijay Abraham I , Bjorn Helgaas , Jonathan Corbet , Shuah Khan , Vinod Koul , Frank Li , Arnd Bergmann , Damien Le Moal , Niklas Cassel Cc: Marek Vasut , Yoshihiro Shimoda , linux-pci@vger.kernel.org, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, dmaengine@vger.kernel.org Subject: [PATCH 3/3] Documentation: PCI: Add PCI DMA endpoint function documentation Date: Thu, 21 May 2026 15:36:38 +0900 Message-ID: <20260521063638.2843021-4-den@valinux.co.jp> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20260521063638.2843021-1-den@valinux.co.jp> References: <20260521063638.2843021-1-den@valinux.co.jp> Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: TY4P286CA0015.JPNP286.PROD.OUTLOOK.COM (2603:1096:405:2b0::15) To TY7P286MB7722.JPNP286.PROD.OUTLOOK.COM (2603:1096:405:38f::10) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: TY7P286MB7722:EE_|TY6P286MB7399:EE_ X-MS-Office365-Filtering-Correlation-Id: 5e31fd08-46b3-4b0f-07b4-08deb70354aa X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|1800799024|10070799003|376014|7416014|366016|921020|18002099003|56012099003|22082099003|6133799003|3023799007; X-Microsoft-Antispam-Message-Info: JlYtWlACdsLM7vNclF9RvY60nmFIfJ7zWnzk83G1u7eEifTd11wfHnoaO98NJhiLtULIceohQFVv2Arti3orSCPP2Dmkbaxyh7oVxwrNxAxd2wqENfIrdSW0SkyWKSipCi2dQtZtyjeZcGgovvWkKBUn4MdeLL1AdlCwlrpBgcbG4YMruYYWhcVznufDRE+HwK+eQO78eGmgTDydrjvJ37XQF6VNc13TDjw/KCz0hZkEkwQb7lvelV8njfqpDf7n41UwXLoDZiRr+Ra8JOObgee+uBvk4wXOexqj86q6HHJrzN74x9da8PXXCAC/h+NwsnjAUFoeH0y7DfZOyZQaokz3ZKlLP15iXLo+3yRe+qkztI0uPEXG0ca+MixDJRBAoH/orANL9+gUYkl3uAfl8E7wheBd68747Gk0vJsTQTyVHb8dBt0YkZHRQkmW+N+9SPBcH6/cWPoL8vKgTvzyfy7xI1hbgw7Tp03LqrYokh5NjyqcUUQkPy4PLGPPG6MlSKQiBaVDwqzB+0KK06DzDtH07oxvfjdRLPCHEnQ7qY/NP+ZsrQRvXEDlA1i7ULEugyTbSenIebGCbSMP+/BJ6l22ATiWZBw/f/Mq9u5v8+hzZYSXfaRC8Vok7Y4BnMKONoIuNFu83YuTDcHLzh/4mVRIKXgYqSbc+jOmwlqyn+j68CH9Pgzc7j+yCGuEZci+sdjrcrQ6VcQbcZTYEkJ9pVMCu70BHObDffVVu0JVhos= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:TY7P286MB7722.JPNP286.PROD.OUTLOOK.COM;PTR:;CAT:NONE;SFS:(13230040)(1800799024)(10070799003)(376014)(7416014)(366016)(921020)(18002099003)(56012099003)(22082099003)(6133799003)(3023799007);DIR:OUT;SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?FCUJjeyp6T0yJTLQYGW+0z5/+s7J7wX+QQz8Glhy3Jg59PVSGl6ruCqo/Q1V?= =?us-ascii?Q?UXSce9AaCEcDu6FTkMJeDLMG4pbmIHjBPlTi6hqThW5k7lkV1OkoCPv1KLiz?= =?us-ascii?Q?smzny15l8LnH2dMPzSZAh0ZLZR3uJzNDJYC0OEmUQVMmVo9NjHbfNeIMhqEf?= =?us-ascii?Q?CetinTx9HoYIcmXFX4Tqs7EXaaLsx6a3pcnFX0f7USBsH+S+uyz+7FrnpsSo?= =?us-ascii?Q?cHgnn0qE+E+uNn7qedvt0mkaMz0gsEtu68J32C4+l6GiTaINpVrkVCjBh9lg?= =?us-ascii?Q?HbRP3N4P5TkzAbdgPhS8y1GSp913WgFZAeEnznjzlV55tsg7kqvDjZx2M3a+?= =?us-ascii?Q?gXsJ6bOohpFYcMFwgfGxkg+19CtzwHl8n8yjdK7+RueDCpDSFcefC/Za0qIX?= =?us-ascii?Q?jbPPASMJNfn5dR6MqH2hi9Mmg5a9VTNdVp5KfQmi3H58x9L31tkB1E0GvJmV?= =?us-ascii?Q?z1LOzjnh/kQOcC+IG9LnRJhIoa7R/8wOj9ifsuSo9dDibWUgwRm9h7g79p8B?= =?us-ascii?Q?2TbKYxxbCGBATiOIPkYm75p5GrtOvxup8ByO1V+n6AyPMnoOJhkbQxPPsb5t?= =?us-ascii?Q?8W8PoLMNqu9JKlGI+9GSlnCdazInSZdpUhGyPGSm9nXalj7VAeDpwy9VyWxJ?= =?us-ascii?Q?BGRaNS9+Rq929C6uZ016e/hjeBwXrvamjzsJiuoqxWOE296zn4MJzWpbhQh7?= =?us-ascii?Q?FeX2ruA66Bx7HBvK9ADnDiUTlNQ/+5WtGygyWCjYeYwAlCHHLWIsWDK9lWiR?= =?us-ascii?Q?WPQUmsEDOEHOt8OUYt4QQqRGHgX3J+5rdMVlzGuLXT5axkVAAfmCkyk7wLzz?= =?us-ascii?Q?mhDBX5YcSV9fBDn5iRCWlp+cfOLuEPMP8994vnkaolxlRaYNEXy45CqYUKmJ?= =?us-ascii?Q?kJzb7pgXY2QP53SNj8M9R7DkpzK4u2gb3qqALnWG+HyTB1VfnF0+MaqkWCgM?= =?us-ascii?Q?VNCPwWz/qensggEZL8k2OxxFwwr+Y2WgFaVCOYHqdoXGMkQrdUAnFK4u1ZEA?= =?us-ascii?Q?185b7GaaDSvXXd1/hl/ykKd1bofwdVtKfVaIS4Lbbp7OTTvnTSspMKzrWbul?= =?us-ascii?Q?jvGThtscLXpgtNbVrHG/qVZeOFyjfDj84XGAy+o2e4eDuxSj8wgSUZOGS/sH?= =?us-ascii?Q?9sgDWFcL+a5jOarxfLmSE+SMv2/SVRsx3UfULQz7qnZBQ/jWyOGZ4brSI5fl?= =?us-ascii?Q?EvFqMwz05pq7ESf9+eTK4wpyt6o8Rm5FZixtNHUnanbSs8y1ZsOSvsjddpS9?= =?us-ascii?Q?PnEXIq8BkdOEMThyRiaVxFGnh2B8mzPDwdPyLVbgqAmPc1Uu3DbnXQDU8Gmd?= =?us-ascii?Q?9wOMp+uu2GCt2nRofBxMWgrbzerbQt3tbWkjP+hxjCEN0IceWT7icYHnAndH?= =?us-ascii?Q?CVXJj0BJQ+UUMbrKUAZrP0r7XRR36OqS7xhJUt6he9cNqAVEdy1VlRPabLdm?= =?us-ascii?Q?7y2Bnzs8qm1ieJNdKCrbtVsftQls15cYq9mcMys8cK3r3l3AMx4LSXyjVzLw?= =?us-ascii?Q?V6bMPI5u5w9Dp8w74D5oNmCy3QSlOTya4xhiOi07FOia7zLBPfqcw/kw6oe6?= =?us-ascii?Q?6n9RuhucWSeoWIrXutH992xO4FCBKEhJf18nHyWqnjbDV6idrGX3BP21Zz5j?= =?us-ascii?Q?P/lakmNZ/FCLx1gcdTRPp9K943oGDkc17fHCDhoTiixCX0DNU14a8nE+zNBd?= =?us-ascii?Q?mqHOjEkN197dRA4KbpTcJFH32Vnx6qmlVwm0vbrgrjn/Yj4hVKfHM6TLMfCK?= =?us-ascii?Q?T25FAE1YGdLZlT5muPaQ9z6uVBaUmhkegoVW18xf/h3OchdpLNNU?= X-OriginatorOrg: valinux.co.jp X-MS-Exchange-CrossTenant-Network-Message-Id: 5e31fd08-46b3-4b0f-07b4-08deb70354aa X-MS-Exchange-CrossTenant-AuthSource: TY7P286MB7722.JPNP286.PROD.OUTLOOK.COM X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 21 May 2026 06:36:46.7806 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 7a57bee8-f73d-4c5f-a4f7-d72c91c8c111 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: ME6igPkbXQDl2gC9wgWxd9spLuQ1ECLxH1vfDPH1nDoQITOXpdVxtoRt+cVySutF83uvNQ00prUxRHEU3Cu+VQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: TY6P286MB7399 Content-Type: text/plain; charset="utf-8" Add a function description and a user guide for pci-epf-dma. Describe the BAR-resident metadata consumed by dw-edma-pcie, the configfs attributes, endpoint controller requirements and the host-side DMAengine usage model. Signed-off-by: Koichiro Den --- Documentation/PCI/endpoint/index.rst | 2 + .../PCI/endpoint/pci-dma-function.rst | 182 ++++++++++++++++ Documentation/PCI/endpoint/pci-dma-howto.rst | 200 ++++++++++++++++++ 3 files changed, 384 insertions(+) create mode 100644 Documentation/PCI/endpoint/pci-dma-function.rst create mode 100644 Documentation/PCI/endpoint/pci-dma-howto.rst diff --git a/Documentation/PCI/endpoint/index.rst b/Documentation/PCI/endpo= int/index.rst index dd1f62e731c9..cd4107e02ec2 100644 --- a/Documentation/PCI/endpoint/index.rst +++ b/Documentation/PCI/endpoint/index.rst @@ -15,6 +15,8 @@ PCI Endpoint Framework pci-ntb-howto pci-vntb-function pci-vntb-howto + pci-dma-function + pci-dma-howto pci-nvme-function =20 function/binding/pci-test diff --git a/Documentation/PCI/endpoint/pci-dma-function.rst b/Documentatio= n/PCI/endpoint/pci-dma-function.rst new file mode 100644 index 000000000000..54caf4fafe00 --- /dev/null +++ b/Documentation/PCI/endpoint/pci-dma-function.rst @@ -0,0 +1,182 @@ +.. SPDX-License-Identifier: GPL-2.0 + +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D +PCI DMA Function +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D + +:Author: Koichiro Den + +The PCI DMA endpoint function exposes an endpoint-integrated DMA controller +to the PCI host as a PCI DMA controller. A matching host-side driver +discovers the endpoint DMA metadata and registers the delegated channels w= ith +the Linux DMAengine framework, so host DMAengine clients can submit +transfers. + +An endpoint Linux system can already use an endpoint-integrated DMA +controller locally through the normal DMAengine API, for example to transf= er +data between endpoint memory and host addresses reachable over PCI. The P= CI +DMA function provides a different ownership model: it delegates selected +local DMA channels to the host, so a host DMAengine client can request and +program those endpoint-side channels through the host's DMAengine API. + +To make that possible, the endpoint function publishes the DMA controller +register window and descriptor memory layout to the host, reserves the +selected local DMA channels on the endpoint side, and lets the host program +those channels directly. + +Constructs Used for Implementing DMA +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D + +The PCI DMA function uses the following endpoint-side resources and +configuration: + + 1) DMA controller register window + 2) DMA descriptor memory for endpoint-to-RC channels + 3) DMA descriptor memory for RC-to-endpoint channels + 4) MSI or MSI-X interrupt vectors selected through configfs + 5) One endpoint BAR used to publish metadata + 6) If needed, one endpoint BAR used for dynamically mapped DMA windows + +The endpoint controller reports the DMA controller register and descriptor +resources through the endpoint auxiliary resource interface. The PCI DMA +function uses those descriptions to build the host-visible metadata and to= map +resources that are not already visible to the host. + +DMA Controller Register Window: +------------------------------- + +It contains the DMA controller registers programmed by the host-side driver +to submit transfers, control channels and handle DMA interrupts. + +DMA Descriptor Memory: +---------------------- + +It contains the descriptor memory used by the DMA controller. The PCI DMA +function exposes descriptor memory for the delegated endpoint-to-RC and +RC-to-endpoint channels. + +MSI/MSI-X Interrupt Vectors: +---------------------------- + +They are used by the delegated DMA channels to signal completion and error +conditions to the host-side driver. + +Metadata BAR: +------------- + +It is the endpoint BAR used to publish the endpoint DMA metadata and hands= hake +bits. The BAR remains stable while the endpoint function programs the DMA +windows. + +DMA Window BAR: +--------------- + +It is the endpoint BAR used for DMA resources that are not already visible +through a fixed BAR. The endpoint function may switch this BAR to subrange +mapping after the host-side driver has found the metadata BAR. + +BAR Metadata +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D + +The endpoint function places a small metadata block at the beginning of the +selected metadata BAR. The format is defined in +``include/linux/pci-ep-dma.h``. + +The host-side driver scans the function's assigned memory BARs, looks for = the +endpoint DMA metadata magic, requests DMA window programming, waits for the +READY bit, and then parses the metadata to find the DMA register window and +descriptor windows. + +:: + + +----------------------+ metadata BAR offset 0 + | endpoint DMA metadata| + +----------------------+ + | optional padding | + +----------------------+ + + +----------------------+ DMA window BAR offset 0 + | mapped DMA resources | + +----------------------+ + | optional padding | + +----------------------+ + +The metadata can also reference resources that are already host-visible +through fixed BARs. For example, an endpoint controller may expose the DMA +controller register window at a fixed BAR offset while descriptor memories +are mapped into the DMA window BAR by the endpoint function. + +The metadata is BAR-resident instead of a self-contained PCI Vendor-Specif= ic +Extended Capability (VSEC). Some endpoint controllers do not provide writ= able +configuration-space backing storage large enough for a new VSEC payload, w= hile +they can map endpoint memory and controller resources into a BAR. + +Channel Ownership +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D + +The ``wr_chans`` attribute exposes endpoint-to-RC DMA write channels. The +``rd_chans`` attribute exposes RC-to-endpoint DMA read channels. The func= tion +reserves the selected endpoint-side DMAengine channels so that endpoint-si= de +DMAengine clients cannot allocate and use the same hardware channels while +they are delegated to the host. + +The current metadata revision describes channels in dense, zero-based orde= r. +For example, ``wr_chans =3D 2`` exposes write channels 0 and 1. Skipping a +hardware channel in the middle of the exposed range is not supported. + +The current DesignWare eDMA unroll and HDMA compatible support also requir= es +each exposed direction to be delegated as a whole. For example, on a cont= roller +with two write channels, ``wr_chans`` must be either 0 or 2. + +Interrupts +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D + +The PCI DMA function exposes DMA interrupts through MSI or MSI-X. The com= mon +endpoint function ``msi_interrupts`` and ``msix_interrupts`` configfs attr= ibutes +select the interrupt vector counts programmed into endpoint config space. = At +least one MSI or MSI-X vector must be configured before the function is bo= und +to an endpoint controller. + +Transfer Addressing +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D + +The host-side DMAengine client supplies the endpoint memory address as the +DMA slave address. For example, the ``dw-edma-pcie`` endpoint DMA metadata +parser passes that slave address to the DMA controller as a raw endpoint-s= ide +address instead of translating it through a host PCI BAR resource. + +The host memory buffer used as the other side of the transfer is still map= ped +using the normal DMA mapping API on the host. + +Endpoint Controller Requirements +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D + +The endpoint controller driver must expose the DMA controller register +window and per-channel descriptor memories through the endpoint auxiliary +resource API. Endpoint controllers with other DMA register layouts also n= eed +matching metadata and host-side DMAengine driver support. + +If any DMA resource is not already host-visible through a fixed BAR, the +endpoint controller must also support BAR subrange mapping and dynamic inb= ound +mapping, because the DMA window BAR is assembled from those resources. + +Current Support +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D + +The current host-side support is implemented in ``dw-edma-pcie`` for +DesignWare eDMA unroll and HDMA compatible layouts. Other PCIe controller +DMA implementations need corresponding host-side DMAengine driver support. + +The ``dw-edma-pcie`` PCI ID table does not contain a generic endpoint DMA = PCI +ID entry. Users need to bind the host-side driver explicitly using +``driver_override``. + +The current metadata revision requires the exposed channels to be a dense +prefix of the hardware channel numbers. + +Security Model +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D + +The interface is intended for trusted endpoint/host deployments. A delega= ted +DMA channel can access endpoint memory addresses supplied by a host DMAeng= ine +client. diff --git a/Documentation/PCI/endpoint/pci-dma-howto.rst b/Documentation/P= CI/endpoint/pci-dma-howto.rst new file mode 100644 index 000000000000..84f322881aa7 --- /dev/null +++ b/Documentation/PCI/endpoint/pci-dma-howto.rst @@ -0,0 +1,200 @@ +.. SPDX-License-Identifier: GPL-2.0 + +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D +PCI DMA Endpoint Function (EPF) User Guide +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D + +:Author: Koichiro Den + +This guide shows how to configure the ``pci-epf-dma`` endpoint function dr= iver. +It uses ``dw-edma-pcie`` as the currently available host-side driver. For= the +hardware model and layout see Documentation/PCI/endpoint/pci-dma-function.= rst. + +Endpoint Device +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D + +Endpoint Controller Devices +--------------------------- + +To find the list of endpoint controller devices in the system:: + + # ls /sys/class/pci_epc/ + e65d0000.pcie-ep + +If ``PCI_ENDPOINT_CONFIGFS`` is enabled:: + + # ls /sys/kernel/config/pci_ep/controllers + e65d0000.pcie-ep + +Endpoint Function Drivers +------------------------- + +To find the list of endpoint function drivers in the system:: + + # ls /sys/bus/pci-epf/drivers + pci_epf_dma pci_epf_test + +If ``PCI_ENDPOINT_CONFIGFS`` is enabled:: + + # ls /sys/kernel/config/pci_ep/functions + pci_epf_dma pci_epf_test + +Creating pci-epf-dma Device +--------------------------- + +Create a ``pci-epf-dma`` device with configfs:: + + # mount -t configfs none /sys/kernel/config + # cd /sys/kernel/config/pci_ep/ + # mkdir functions/pci_epf_dma/dma0 + +The "mkdir dma0" above creates the ``pci-epf-dma`` function device that wi= ll +be probed by the ``pci_epf_dma`` driver. + +The PCI endpoint framework populates the directory with the common +configurable fields:: + + # ls functions/pci_epf_dma/dma0 + baseclass_code msi_interrupts progif_code subsys_id + cache_line_size msix_interrupts revid subsys_vendor_id + deviceid pci_epf_dma.0 secondary vendorid + interrupt_pin primary subclass_code + +The PCI DMA function driver also creates a function-specific sub-directory. +The numeric suffix depends on the endpoint function instance number:: + + # ls functions/pci_epf_dma/dma0/pci_epf_dma.0/ + dma_window_bar metadata_bar rd_chans wr_chans + +Configuring pci-epf-dma Device +------------------------------ + +The host-side ``dw-edma-pcie`` PCI ID table does not contain a generic +endpoint DMA PCI ID entry. Choose a PCI vendor/device ID for the endpoint +device:: + + # echo > functions/pci_epf_dma/dma0/vendorid + # echo > functions/pci_epf_dma/dma0/deviceid + # echo 1 > functions/pci_epf_dma/dma0/msi_interrupts + +The PCI class defaults to ``PCI_BASE_CLASS_SYSTEM`` and +``PCI_CLASS_SYSTEM_DMA``. + +The function-specific attributes are: + +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D +Attribute Description +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D +metadata_bar BAR used to publish the endpoint DMA metadata and handshake + bits. It is kept as a stable BAR while the DMA windows are + programmed. If this is left unset, the first usable BAR th= at + does not already contain a fixed DMA resource is used. +dma_window_bar BAR used for DMA resources that are not already host-visibl= e, + such as the DMA register window or descriptor windows. This + BAR may be switched to subrange mapping after the host driv= er + has found the metadata. If this is left unset and a DMA + window is needed, the first usable BAR different from + ``metadata_bar`` and not already occupied by a fixed DMA + resource is used. +wr_chans Number of endpoint-to-RC DMA write channels to expose. +rd_chans Number of RC-to-endpoint DMA read channels to expose. +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D + +A sample configuration for a DesignWare eDMA/HDMA compatible controller wi= th +two write channels and two read channels is given below:: + + # echo 0 > functions/pci_epf_dma/dma0/pci_epf_dma.0/metadata_bar + # echo 2 > functions/pci_epf_dma/dma0/pci_epf_dma.0/dma_window_bar + # echo 2 > functions/pci_epf_dma/dma0/pci_epf_dma.0/wr_chans + # echo 2 > functions/pci_epf_dma/dma0/pci_epf_dma.0/rd_chans + +``wr_chans`` and ``rd_chans`` default to 0. At least one channel direction +must be configured. The selected channels are exposed in dense, zero-based +order; for example, ``wr_chans =3D 2`` exposes write channels 0 and 1. +Current DesignWare eDMA unroll and HDMA compatible support requires each +exposed direction to be delegated as a whole, so set a direction to either= 0 or +the number of hardware channels in that direction. If ``dma_window_bar`` = is +configured, it must be different from ``metadata_bar``. + +The common ``msi_interrupts`` and ``msix_interrupts`` attributes select the +number of MSI and MSI-X vectors exposed to the host. At least one MSI or +MSI-X vector must be configured. + +The function-specific attributes can only be changed before the endpoint +function is bound to an endpoint controller. + +Binding pci-epf-dma Device to EP Controller +------------------------------------------- + +The DMA function device should be attached to a PCI endpoint controller +connected to the host:: + + # ln -s controllers/e65d0000.pcie-ep \ + functions/pci_epf_dma/dma0/primary/ + +Once the above step is completed, the PCI endpoint controller is ready to +establish a link with the host. + +Start the Link +-------------- + +Start the endpoint controller by writing 1 to ``start``:: + + # echo 1 > controllers/e65d0000.pcie-ep/start + +Root Complex Device +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D + +lspci Output +------------ + +Note that the device listed here corresponds to the values populated in the +endpoint configuration above:: + + # lspci -nk + 01:00.1 0801: : + +If the host was already running while the endpoint function was configured, +rescan the PCI bus after the endpoint side has completed the configfs setup +and started the endpoint controller, if the platform supports it. + +Bind the endpoint DMA function to ``dw-edma-pcie`` explicitly with +``driver_override``:: + + # modprobe dw_edma_pcie + # echo dw-edma-pcie > /sys/bus/pci/devices/0000:01:00.1/driver_override + # echo 0000:01:00.1 > /sys/bus/pci/drivers_probe + +The device should then be bound to ``dw-edma-pcie``:: + + # lspci -nk -s 01:00.1 + 01:00.1 0801: : + Kernel driver in use: dw-edma-pcie + +Using pci-epf-dma Device +------------------------ + +The host side software uses the standard Linux DMAengine API. A DMAengine +client driver running on the host must request one of the channels provide= d by +``dw-edma-pcie`` and submit a transfer. + +For an endpoint-to-RC write transfer, the DMAengine client uses a host DMA +buffer as the destination and an endpoint-side address as the slave source +address. For an RC-to-endpoint read transfer, the DMAengine client uses a +host DMA buffer as the source and an endpoint-side address as the slave +destination address. + +Troubleshooting +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D + +``pci-epf-dma`` requires endpoint controller support for DMA auxiliary +resources and MSI or MSI-X. If any DMA resource must be mapped dynamicall= y, +the endpoint controller must also support BAR subrange mapping and dynamic +inbound mapping. Binding the function to an endpoint controller fails if = the +required capabilities are not available, or if both ``msi_interrupts`` and +``msix_interrupts`` are zero. + +If ``dw-edma-pcie`` fails to probe on the host, check that the endpoint was +bound to the host driver, that the endpoint BARs were assigned by PCI +enumeration, and that the endpoint DMA metadata READY bit was set after any +DMA window BAR submaps were programmed. --=20 2.51.0