From nobody Tue Sep 29 04:14:12 2026 Received: from TY3P286CU002.outbound.protection.outlook.com (mail-japaneastazon11020112.outbound.protection.outlook.com [52.101.229.112]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A026B3002C8; Wed, 12 Aug 2026 15:57:29 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=52.101.229.112 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1786550252; cv=fail; b=j2jLRp8Zm0nuhZvHVV1ueK1iwbbifhsRX36FpfrjraWa5zYWzB9vnc5LQRO19IkGl+fwKqCvPhtcp+PvBwhysCxOYDkUwN0/4htvYurq2dRAhNK9R+DgJxSobs3Iz2poGlogRoa4SYR8MkEc9l/7+9CaA0TahDafmuNiFbXSYf8= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1786550252; c=relaxed/simple; bh=4EvJO7PYu90QVmc80jjL+A6MWWAtwReJdIbPx4u/zpA=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: Content-Type:MIME-Version; b=aRIMKDDYKcthA0zlfvS0brXw/KArKg54F3Cib4pwkz/B0vuIGWeYPlFzcFl0RoMo0raC6ciYTRQMDtC3uCnN6pk+exV0YPxsRfryqGVpjnFF28MoG6in7H817j5of94FQDwr/tcU+RwkkTpM87jgwdjNSr1TdUBnp7RRUiYVJAA= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=valinux.co.jp; spf=pass smtp.mailfrom=valinux.co.jp; dkim=pass (1024-bit key) header.d=valinux.co.jp header.i=@valinux.co.jp header.b=hkskB0Se; arc=fail smtp.client-ip=52.101.229.112 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=valinux.co.jp Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=valinux.co.jp Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=valinux.co.jp header.i=@valinux.co.jp header.b="hkskB0Se" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=bxuUDkXTR/+NPgMUGRtKqEopwbXCrUGu4LvWSyVZLHtWPLC1uraSDl6JyEYy7CGOS7+x3AVIlZN8WOcGKqi8r2cmLh0qOqS8vv8fSR+p/3l+piARvS+0cB7Hdnhp92Vz4IgUA74mUzER/F3bzpcGOX6ZvKpABb6/Oh+XZWYxfzdsGj1VYpV0wMf56n4hFIvnvryNFJjHh9rI69wJAiP49GGa+vm7sVuWhcRAr//e41ntTmrXdzS9Slv0YSjwykyyhkDLtDk4SeuUU4eZ+vFsbiicwPSo40UMcQfGAfkD1lf5ZFrFlCyvMcgbAQlE7pXUAJ7RflWUOnvwZtNqqBhDcw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=N56QZWCdFxARArrJHmKUqw0An/HbFxiNWvpcQ3aXTnE=; b=FsziXVu92/vg/beIEtua0chvFsvvIrOSXT5kW1oTJfaHMXa9clfuTXpE8kWjp8z3nmCk/WoReqFTFodVWbBhEx2MNe4Jaj8dUIH0fzU21XldZ3OBWB6+gTZNkhgSW7u8Drsnmu2EFz+ysRifz89AwgOM64k1AhqBlmqYbON7t22jjpmNHL8Iv6WFW3Jsk8RnWGtLBVshiKigp2t0fz2Ue8HlUl7PNlMtdZxwO8Kolg2XEoKc1RP/CvX8IS7tsa3HVxZGMyWpfW5ejtp1jxHmBcPyxrfpIgO+6c1hvRYRGEnouQ1nzs3WDIYdnJaGVQiocW+dvnETjW93Qt5VCChWYQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=valinux.co.jp; dmarc=pass action=none header.from=valinux.co.jp; dkim=pass header.d=valinux.co.jp; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=valinux.co.jp; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=N56QZWCdFxARArrJHmKUqw0An/HbFxiNWvpcQ3aXTnE=; b=hkskB0SeHynMm3oa6ahODeURCKa3/2Jc2U7zPyeQNniiCap5ZD8PfSv964shuUfYpJPUgErlOvqqmscR20AOiy/DZsMo1CdPYE7b00FXVZGhNCA2EvMH7FpMLlUmyXpx6wVmAuLxpup+Mk8h5+5i0C1/DozSQIIjD58JEH9Pn54= Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=valinux.co.jp; Received: from TY7P286MB7722.JPNP286.PROD.OUTLOOK.COM (2603:1096:405:38f::10) by OS7P286MB7115.JPNP286.PROD.OUTLOOK.COM (2603:1096:604:42f::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.315.14; Wed, 12 Aug 2026 15:57:27 +0000 Received: from TY7P286MB7722.JPNP286.PROD.OUTLOOK.COM ([fe80::2305:327c:28ec:9b32]) by TY7P286MB7722.JPNP286.PROD.OUTLOOK.COM ([fe80::2305:327c:28ec:9b32%3]) with mapi id 15.21.0292.024; Wed, 12 Aug 2026 15:57:27 +0000 From: Koichiro Den To: Vinod Koul , Frank Li , Manivannan Sadhasivam , Gustavo Pimentel , Kees Cook , =?UTF-8?q?Krzysztof=20Wilczy=C5=84ski?= , Kishon Vijay Abraham I , Bjorn Helgaas , Christoph Hellwig , Serge Semin , Cai Huoqing , Niklas Cassel Cc: Devendra K Verma , dmaengine@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH v5 01/24] dmaengine: dw-edma: Add dw_edma_core_ll_cur_idx() to get current LL entry index Date: Thu, 13 Aug 2026 00:56:58 +0900 Message-ID: <20260812155721.2807506-2-den@valinux.co.jp> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20260812155721.2807506-1-den@valinux.co.jp> References: <20260812155721.2807506-1-den@valinux.co.jp> Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: TY4P286CA0102.JPNP286.PROD.OUTLOOK.COM (2603:1096:405:380::16) To TY7P286MB7722.JPNP286.PROD.OUTLOOK.COM (2603:1096:405:38f::10) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: TY7P286MB7722:EE_|OS7P286MB7115:EE_ X-MS-Office365-Filtering-Correlation-Id: 5179e920-178b-4a2b-2ce1-08def88a6833 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|376014|7416014|366016|1800799024|23010399003|10070799003|18002099003|22082099003|56012099006|6133799003|10067099003|921020; X-Microsoft-Antispam-Message-Info: WQsKBwaW/OW46xoi126RGyhlYxn7SIYmnBmfogs2oTsqKlZzKoxsPIf3v+8Uy4oPgTZonc41GAXBhkvCEw+3p8fmp8ZMFAtgeWh63x25cWzpKIZ6ZvGqdWRqMIRI3jgLUVfPvczFoVQdU1atgIJXkt16SMhHnJ0PDyltSTwJODoI/E+IA398LdkpED4ZmSZVqlj8+QJnXg9LNn/NvZfH8+KpQ2PkMXJsiMQFm2EBz28gjMm4rmVa1tgGrFjDiiU/3mOT38thU04WnzBbK5dVtmvT0fBDKx90S07WFX7YubT0zp1o7EEB/ofHmkt7xyXbhhhheAlbbCcaiBmdwAdsvY7oE3lpQOkxVtl36BG/CTwauBy8/NtTLk507VQHL90hj5hDfuO9D30j/uIq83MvNobpPpToGys1UGIPEJRQTOZUchvfxgv1HhC5EmuDWF4Ms6a8BxvBscKs061VrpwvYMtuPpJxyj4KW53uP/wpXWZU1ggp2Fo3V3r6PkdHnI9sDv4dYJ+q7r66gW8tGxzgaPJQFAKc4qbBYCNoLkjDp9DEQ3dRRpfRPAxL4ezsY4QEV9GeNHmsL7GqVOKBF/xm46/UBHaAWQdjIbJvZtl6pmu5lV3LgTuy/1zmXJTFP781jxFzwczvG5xdQQHQzzWcgqdfgEpkMuqQvVEKJ5aKouHEEISdWMd0zXHU5lMk4ySFqgcXVSfypoZ4wmyeUnHU7A== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:TY7P286MB7722.JPNP286.PROD.OUTLOOK.COM;PTR:;CAT:NONE;SFS:(13230040)(376014)(7416014)(366016)(1800799024)(23010399003)(10070799003)(18002099003)(22082099003)(56012099006)(6133799003)(10067099003)(921020);DIR:OUT;SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?3M1dsZJLG53H/HoIZQogPXLf0pgYx4wXnPc1oRGmh/VMRQGSjHWersC9XY2I?= =?us-ascii?Q?3zgBdyCXZ99MmVP87UEZnqjItDSnjq5Odu2NN6XXGDqrcslCc4AV1GFov/9P?= =?us-ascii?Q?OSQw10d4fJKEWD3dKRFL72vYPWrKbsQURyEtH6pm5EQfitnB3FAMoQPDCJw0?= =?us-ascii?Q?jJKnBICQusEsD6lnri8jWDlMU9FNlmt1TnKU6HNmpe9nDCAmEHe17kNmYF3t?= =?us-ascii?Q?IQ2xogbiwcj3FW3tI++0r4+Sms5KS+u1zCeAnkDEp+3kRa0C5NNxzx31TuKz?= =?us-ascii?Q?O5zg1xh74x7BgttsMeoWJ9dGRyMRks9DFrw1P5Z1g3uBdb3mug8nfNvNo76b?= =?us-ascii?Q?x9GAj5+84Rd07Nupw+YfV0WxdESweuxuv9SUmFsWLSTYc8+21LYAWYKVNnHk?= =?us-ascii?Q?jH84Y369n0q+isyQIDEBrm1HvxKruaBPe6So5DLSVAGdBGFlM+cKxFK29JXg?= =?us-ascii?Q?Ot/N27wDXRJTOwXbP0bsn9IxE6skE5Ayne9/ehKyAuxIX10JydlE9wVk9FAG?= =?us-ascii?Q?Ku5EX05CaPsZbDrvOyN0w7112n1tkzFssj87ZbGh823hacBROheze9plyWOt?= =?us-ascii?Q?Qf/NFr+TIjGYTzn4Rw9lNpE/LyjUs9Jv0UWii+OviioRk9zbtOu4FxgnroyG?= =?us-ascii?Q?v2PagRLBxKe+oioGtX2ctu4uVujZnzfFwaMXLIruWUMDzlTWAMjzqgSILS8U?= =?us-ascii?Q?xwrulaHG0wEM4drDuEppHsNXLNUAtIDYH3YD7bQBhkaBPV2bN+u4frLw/F2S?= =?us-ascii?Q?o2Vw3e2sfmCtFeHzvZBBHyrvo6/aHMTzwHrxEVGY6RO2ArZmThJhBn541y8D?= =?us-ascii?Q?O8FB2YN2K1ZxL6W9/AuJXbVwjidTGt1VGQyzu95AejsPd9cBC08+tAUNNXy0?= =?us-ascii?Q?e/spC5iR6xfUFZIHCXSFDJkUVMwb2rPO9h9RY7U1vZP+1nUlHO0RiQfHUoI+?= =?us-ascii?Q?UB9m6SsuxLItBgbH6O1pMK4OKTJ/kiW+ZjsTXBgqd5Wu/S9azJTU8ZYLFjKr?= =?us-ascii?Q?ZxzkAz2hIZDmDKdv8YhydeOBIyc7ghQjdbT5paYZLjacj1AaHYp7UEaFOsES?= =?us-ascii?Q?dEdhiNZhucOIOO0rskQlTLse/q/0C8AC3wVSUuHoSarXIg5QvNCgXkHMUTvi?= =?us-ascii?Q?pFyZhXt98svmgwd0CC9fI0yUQcdsiLq0SPtAvlN8tRJHMq9s9OX1m5Rb7BY8?= =?us-ascii?Q?dnlYBVV9V3GqVTjw2USKWOnwTnoeqlJmE3mJEaa05UJfv8CP5g57HtwNFipi?= =?us-ascii?Q?w54b8ROOJKWOxVjG70r/8maoGPf16LVfCiub4tU9sMj6K5vFk3QFwiDfiLfY?= =?us-ascii?Q?7FpsoTkkkoSucf8O1//I+i3ECZ+9DCi30lZ04pqgDgaCqMKJHICZDh3aEkjz?= =?us-ascii?Q?VtHZ1QrDMlbQSEhOH8Nd1xgXJ9e3620ZRtlJKV1ieq3jMVgFZenE4ZYvDm/J?= =?us-ascii?Q?yfAGg2xp963Qb7FjNJ7gP4qRJkvqAwf15j3iXOv3r5jUAOjqFw5VACa2GJSO?= =?us-ascii?Q?MVyEwb/b0HmRk/DT8kYu33tpmLuqABYlPm215phF+sca1oFe163SzBp+OOdP?= =?us-ascii?Q?X/mP35K1tRZw9WGE2X6/W/lbl3hhWgwq5GVrErQ4ReQKkfWrL0Rr/f1EoEnm?= =?us-ascii?Q?QK81bK76n7ChYAX916USM8eP1sWrr7xVWtYUBwp5iwg4K3FK8+E+KjBYPBaR?= =?us-ascii?Q?RioeaJTj5Ql96rJV5+a0eAWSwc86jD85k0CyQyHnYLoeJbB8qmTR1vwkG14D?= =?us-ascii?Q?3lxzfVOxpuPkICcmKGgsFgz0OSPM/3yFvFmhij2Z811KP8UbKbbo?= X-OriginatorOrg: valinux.co.jp X-MS-Exchange-CrossTenant-Network-Message-Id: 5179e920-178b-4a2b-2ce1-08def88a6833 X-MS-Exchange-CrossTenant-AuthSource: TY7P286MB7722.JPNP286.PROD.OUTLOOK.COM X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 12 Aug 2026 15:57:27.1096 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 7a57bee8-f73d-4c5f-a4f7-d72c91c8c111 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: MfHCabYO6o0vJ7lNRVF1euD4fWom9R5jmipYw8yrc42HPGxikbsD46SfQPD0EUVIUUZqP6uGNOtF2tSC8bDzCw== X-MS-Exchange-Transport-CrossTenantHeadersStamped: OS7P286MB7115 Content-Type: text/plain; charset="utf-8" From: Frank Li Add dw_edma_core_ll_cur_idx() to get the current LL entry index and prepare for dynamic addition of DMA requests while the DMA engine is running. Signed-off-by: Frank Li Reviewed-by: Frank Li Co-developed-by: Koichiro Den Signed-off-by: Koichiro Den --- Changes in v5: - Reject an all-ones DMA_LLP value before converting it to an LL index. (Sashiko) @Frank, I kept your R-b tag since the v5 change is only a small validation check. Please let me know if you prefer me to drop it. drivers/dma/dw-edma/dw-edma-core.h | 10 ++++++++++ drivers/dma/dw-edma/dw-edma-v0-core.c | 20 ++++++++++++++++++++ drivers/dma/dw-edma/dw-hdma-v0-core.c | 20 ++++++++++++++++++++ include/linux/dma/edma.h | 2 +- 4 files changed, 51 insertions(+), 1 deletion(-) diff --git a/drivers/dma/dw-edma/dw-edma-core.h b/drivers/dma/dw-edma/dw-ed= ma-core.h index f6a5ad317567..e50fa7965778 100644 --- a/drivers/dma/dw-edma/dw-edma-core.h +++ b/drivers/dma/dw-edma/dw-edma-core.h @@ -138,6 +138,7 @@ struct dw_edma_core_ops { void (*ll_data)(struct dw_edma_chan *chan, struct dw_edma_burst *burst, u32 idx, bool cb, bool irq); void (*ll_link)(struct dw_edma_chan *chan, u32 idx, bool cb, u64 addr); + int (*ll_cur_idx)(struct dw_edma_chan *chan); void (*ch_doorbell)(struct dw_edma_chan *chan); void (*ch_enable)(struct dw_edma_chan *chan); void (*ch_config)(struct dw_edma_chan *chan); @@ -181,6 +182,15 @@ struct dw_edma_chan *dchan2dw_edma_chan(struct dma_cha= n *dchan) return vc2dw_edma_chan(to_virt_chan(dchan)); } =20 +/* + * Return the current LL entry index. A negative value means that the chan= nel + * context is not initialized or was lost after a link reset. + */ +static inline int dw_edma_core_ll_cur_idx(struct dw_edma_chan *chan) +{ + return chan->dw->core->ll_cur_idx(chan); +} + static inline u64 dw_edma_core_get_ll_paddr(struct dw_edma_chan *chan) { if (chan->dir =3D=3D EDMA_DIR_WRITE) diff --git a/drivers/dma/dw-edma/dw-edma-v0-core.c b/drivers/dma/dw-edma/dw= -edma-v0-core.c index abc1bd4c0be2..cc8059182940 100644 --- a/drivers/dma/dw-edma/dw-edma-v0-core.c +++ b/drivers/dma/dw-edma/dw-edma-v0-core.c @@ -616,6 +616,25 @@ static void dw_edma_v0_core_ch_doorbell(struct dw_edma= _chan *chan) FIELD_PREP(EDMA_V0_DOORBELL_CH_MASK, chan->id)); } =20 +static int dw_edma_v0_core_ll_cur_idx(struct dw_edma_chan *chan) +{ + u32 base, val; + + val =3D GET_CH_32(chan->dw, chan->dir, chan->id, llp.lsb); + base =3D lower_32_bits(dw_edma_core_get_ll_paddr(chan)); + + /* + * LL regions stay within one 4 GiB address window. Reject an all-ones + * MMIO value. If the low word is zero, use the high word to distinguish + * a nonzero boundary address from an unprogrammed all-zero context. + */ + if (val =3D=3D U32_MAX || + (!val && !GET_CH_32(chan->dw, chan->dir, chan->id, llp.msb))) + return -EINVAL; + + return (val - base) / EDMA_LL_SZ; +} + /* eDMA debugfs callbacks */ static void dw_edma_v0_core_debugfs_on(struct dw_edma *dw) { @@ -650,6 +669,7 @@ static const struct dw_edma_core_ops dw_edma_v0_core = =3D { .handle_int =3D dw_edma_v0_core_handle_int, .ll_data =3D dw_edma_v0_core_ll_data, .ll_link =3D dw_edma_v0_core_ll_link, + .ll_cur_idx =3D dw_edma_v0_core_ll_cur_idx, .ch_doorbell =3D dw_edma_v0_core_ch_doorbell, .ch_enable =3D dw_edma_v0_core_ch_enable, .ch_config =3D dw_edma_v0_core_ch_config, diff --git a/drivers/dma/dw-edma/dw-hdma-v0-core.c b/drivers/dma/dw-edma/dw= -hdma-v0-core.c index 36ee72efcd31..19da1c8fd615 100644 --- a/drivers/dma/dw-edma/dw-hdma-v0-core.c +++ b/drivers/dma/dw-edma/dw-hdma-v0-core.c @@ -384,6 +384,25 @@ static void dw_hdma_v0_core_ch_doorbell(struct dw_edma= _chan *chan) SET_CH_32(dw, chan->dir, chan->id, doorbell, HDMA_V0_DOORBELL_START); } =20 +static int dw_hdma_v0_core_ll_cur_idx(struct dw_edma_chan *chan) +{ + u32 base, val; + + val =3D GET_CH_32(chan->dw, chan->dir, chan->id, llp.lsb); + base =3D lower_32_bits(dw_edma_core_get_ll_paddr(chan)); + + /* + * LL regions stay within one 4 GiB address window. Reject an all-ones + * MMIO value. If the low word is zero, use the high word to distinguish + * a nonzero boundary address from an unprogrammed all-zero context. + */ + if (val =3D=3D U32_MAX || + (!val && !GET_CH_32(chan->dw, chan->dir, chan->id, llp.msb))) + return -EINVAL; + + return (val - base) / EDMA_LL_SZ; +} + /* HDMA debugfs callbacks */ static void dw_hdma_v0_core_debugfs_on(struct dw_edma *dw) { @@ -406,6 +425,7 @@ static const struct dw_edma_core_ops dw_hdma_v0_core = =3D { .non_ll_start =3D dw_hdma_v0_core_non_ll_start, .ll_data =3D dw_hdma_v0_core_ll_data, .ll_link =3D dw_hdma_v0_core_ll_link, + .ll_cur_idx =3D dw_hdma_v0_core_ll_cur_idx, .ch_doorbell =3D dw_hdma_v0_core_ch_doorbell, .ch_enable =3D dw_hdma_v0_core_ch_enable, .ch_config =3D dw_hdma_v0_core_ch_config, diff --git a/include/linux/dma/edma.h b/include/linux/dma/edma.h index 3c8e2ef9dee0..a4e5aab8fddc 100644 --- a/include/linux/dma/edma.h +++ b/include/linux/dma/edma.h @@ -130,7 +130,7 @@ struct dw_edma_chip { =20 u16 ll_wr_cnt; u16 ll_rd_cnt; - /* link list address */ + /* Linked-list regions must not cross a 4 GiB address boundary. */ struct dw_edma_region ll_region_wr[HDMA_MAX_WR_CH]; struct dw_edma_region ll_region_rd[HDMA_MAX_RD_CH]; =20 --=20 2.51.0 From nobody Tue Sep 29 04:14:12 2026 Received: from TY3P286CU002.outbound.protection.outlook.com (mail-japaneastazon11020119.outbound.protection.outlook.com [52.101.229.119]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A995743637C; Wed, 12 Aug 2026 15:57:33 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=52.101.229.119 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1786550255; cv=fail; b=kWF8O8IsphyC//T0L84NdNe9+RIc2ihnPsF2du9STdyvcuQYK05ZnN8n8gnAe1Xq19DtP/BsjtSZXpc/hWzwllbLzQ5cYNWX/SRk6v6cPawie84HfNTFcC7xub33nXI95MBy8I9PjX+Sy31qhK0osnVse1vCjxwo+iakAHXl2PI= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1786550255; c=relaxed/simple; bh=GslbdYxP0s9xx0WG4fv6FcvT7dzHRwxtTAl51n9r4f4=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: Content-Type:MIME-Version; b=b4+UF7Wrrt4B+sIdJwwjGVJhuNpXXT0NZaqU1b2Yz6biJwjiBMT4yenlbvmXZFkrbbu77vYEnbpNT8OnrjYuwcvVnjdcoK2sgNvz/C9AvdBsPjb2ytrdRLRpZCQc77jz4ighdM1T+GFn1ApjVBXaIks5CgVrW+zj56TyspLkrxg= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=valinux.co.jp; spf=pass smtp.mailfrom=valinux.co.jp; dkim=pass (1024-bit key) header.d=valinux.co.jp header.i=@valinux.co.jp header.b=FNXv2eOA; arc=fail smtp.client-ip=52.101.229.119 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=valinux.co.jp Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=valinux.co.jp Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=valinux.co.jp header.i=@valinux.co.jp header.b="FNXv2eOA" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=Pt6HFB3X/jvK3720eTHkA/NFNbBCA5U7N0eTHZAgPeqRDlVUdq2c+U8witzY6FsmcSHBLAW12BVUiuWiMNiGCaMTfwpKB3noNmnD0hm5E43WJ9kWNS0js/MDSNJILcwwVOm5wT35idz+LAjFc9hdhmBCaqIRNoNrF4W1DknTx5oHFVoL2kD/84ro1fxqOSTCav0UjZjGeFgQtmOOU+7FEKOmjkwS13Azr1ipCztz/D/DUqTTY2xguSQGVq6RL9+wEnII+YpIBWeu40wP0zXdJ3hkWKGkoSmd9gqCgWDjyo4r4jSmXOaH+aopxzxGijf1J4UsdVCb0AgVnp65Dzw0Ew== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=QOV6a6QQoqKG34gqy9uvJO8zPzsBjLbUk1ONw63w9ds=; b=NrYEnS+8bAGA+vo/ijBIT37JaT1jGuCyMXwmW3ylVVeFrF3iQ7nAHk8OWyNr2z+VehPf5Vci/S/VV262FNgrj5bCqY0ajmMshqULsoVd5QPYXzEZ5uj+acrCuc/HLB1ixgNhsAZJDdGVYJibyVeiPOvZH5KOrty9Oi4PWHyc39HzhFpaBBRgY+BtSv0SlFDawOZv4DncoiaZ33I6Ffmki2QYCDv52Gmaw0Gpxfync+QC5wN0GMDOgUDqSgGeD7qX7/dH6ldlY9yA5YzFLx1qeYCpoHlpvf5m+J62QnQ2ydi5TaWvwc0WYYkDe93BZD7pHbFh9j1wA6PcdbgkxGCdKA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=valinux.co.jp; dmarc=pass action=none header.from=valinux.co.jp; dkim=pass header.d=valinux.co.jp; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=valinux.co.jp; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=QOV6a6QQoqKG34gqy9uvJO8zPzsBjLbUk1ONw63w9ds=; b=FNXv2eOA9muDS9u1DTjoFzC8pcEg08sZYYdbxOAsvNVuQKzHcehJQ6iZNuW2G8qpb4hd3CDxjlSWoOUVawLR9c+bHIwXADDO+JGpoO0dCWgYM6gK99McYCzpVo2WnPWvp3LXWIOglqvBXHXobxrnzSCJ9R8osXnubt+8RSqQU64= Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=valinux.co.jp; Received: from TY7P286MB7722.JPNP286.PROD.OUTLOOK.COM (2603:1096:405:38f::10) by OS7P286MB7115.JPNP286.PROD.OUTLOOK.COM (2603:1096:604:42f::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.315.14; Wed, 12 Aug 2026 15:57:27 +0000 Received: from TY7P286MB7722.JPNP286.PROD.OUTLOOK.COM ([fe80::2305:327c:28ec:9b32]) by TY7P286MB7722.JPNP286.PROD.OUTLOOK.COM ([fe80::2305:327c:28ec:9b32%3]) with mapi id 15.21.0292.024; Wed, 12 Aug 2026 15:57:27 +0000 From: Koichiro Den To: Vinod Koul , Frank Li , Manivannan Sadhasivam , Gustavo Pimentel , Kees Cook , =?UTF-8?q?Krzysztof=20Wilczy=C5=84ski?= , Kishon Vijay Abraham I , Bjorn Helgaas , Christoph Hellwig , Serge Semin , Cai Huoqing , Niklas Cassel Cc: Devendra K Verma , dmaengine@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH v5 02/24] dmaengine: dw-edma: Add dw_edma_core_ll_clear() to clear LL control-word Date: Thu, 13 Aug 2026 00:56:59 +0900 Message-ID: <20260812155721.2807506-3-den@valinux.co.jp> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20260812155721.2807506-1-den@valinux.co.jp> References: <20260812155721.2807506-1-den@valinux.co.jp> Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: TYCP286CA0003.JPNP286.PROD.OUTLOOK.COM (2603:1096:400:26c::10) To TY7P286MB7722.JPNP286.PROD.OUTLOOK.COM (2603:1096:405:38f::10) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: TY7P286MB7722:EE_|OS7P286MB7115:EE_ X-MS-Office365-Filtering-Correlation-Id: 0cb9e6fc-fb5a-4efc-44e7-08def88a68ab X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|376014|7416014|366016|1800799024|23010399003|10070799003|18002099003|22082099003|56012099006|10067099003|921020; X-Microsoft-Antispam-Message-Info: j64c8KZx8tb9lEBw3wLG8HV+GpC4Ww8aiqSc5UJPgGghnESY9LtkaRbdsI87HBMdoxbud2y+h/LoPFchTzTOebtyiq+bVpFNqVJIl9/AKV/AyZ/NxAG7+GGgf22s0ZlyZn/59nKacpSx2RweMGitesPerFQc7mltqmGpCmalnuDIzB2AptnBlHhgJatABOqgfqr50PJpJht1zDOGnzO1C9dhCsCKyj5D8UdxMiY6KXrdL6pAhYVasDpxHqnZhq4prH9INJmY52ol39pwgSgkhTSDJpgGPiToS0lZI23qIth8P4Ks3rUFUmCQBfTdO4OlyYe8vJfPxCsQyN2e58sNm/NU0nJg2LrRMurGxNIxQq8NhyKfqaSN4CHFpIt7rBOOu35CpInRv3CbbC578xvymxBlgXHojIkB3uM6xLt9NNu2sPTFVW/4/OjNnJGC7QaQOWEOGkRFJDJcVAVo5p7zBXmJIjVRTlNAytqK1I6DM0aqolE2pOlpoqiixR2M6HYqoCbv6ZSJ7WJEk62dWAEKG7TOLYdZWe/8Fnj68Cfjjxvgn4FBJrLKfUrnE/foJ3atufLcM/2/B0AVh5wGO4/TBpSodeGEpr4dtuUFJgUv5QPZ4w2dbdQjO9vWCI6PfVftxBFwutcO5Wx9R8OW9teh0xuhzRBpFhNz6ZY8FmFy8mut3Lj8ARuduMsRThy+QUS2r7lc8tMMbsM2mZGT3TnkZw== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:TY7P286MB7722.JPNP286.PROD.OUTLOOK.COM;PTR:;CAT:NONE;SFS:(13230040)(376014)(7416014)(366016)(1800799024)(23010399003)(10070799003)(18002099003)(22082099003)(56012099006)(10067099003)(921020);DIR:OUT;SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?ubFnlGklSmiqWQ+zVrvhZHUcaEpIapdnfD19Obf4qdB0U38I5zr3epOqIA38?= =?us-ascii?Q?fK2ltvPo/sBOr4JsUa5CgkwsO/tCCxClw5RriUFmXkk7VACqXavaRB2Bwuxq?= =?us-ascii?Q?b26zz2y0IBxOtP+fSKxDZhJKT7ilKH6mbJNzFio59pRGQvfNbHwRD626aFcS?= =?us-ascii?Q?iJ7yrNLX70CiUt4HLNMus8BqlktWn9LmYsa/nWpMo7/3E8YYGTZlks//2aYj?= =?us-ascii?Q?nZK/pnMR4f01LB0xatGchI700oyqEWlW4bDg0FX8UqckKXC8qFsEL9ioHf+7?= =?us-ascii?Q?ob8ddnqcnjyWslpoUka9NQqyXDU+VHsE5KL8+cwfkYXGHkOrFPSYsa9xN9t1?= =?us-ascii?Q?aJbV9udk/qLL8rtcUmSOBqwiMxeXIuMIrIeg0wDU430SgEE50VQ0R/ZtoCK1?= =?us-ascii?Q?XZfDZL0NN6KkWx4E2kGnumbFsNpwvJQw9KcVF1as09s/BaEF3Pmw5oqXWVwY?= =?us-ascii?Q?oUQdhyKh5YJbQ+aP+3FSECCo53tNuA8pwWsb61t6GXwR5iqFHW+0GMy6afpQ?= =?us-ascii?Q?40QTyYp5VmTCCuF2dON0sB7cGy1NSKBtZWeHR1epAooqhIjyoxOYMlxfIC/z?= =?us-ascii?Q?e8PZGP3au+EPetrF/P56ugtCRj+iBbqwAQkbJo/Iy2HpARefA+ntCOjPJFEL?= =?us-ascii?Q?tE+I2Nj9TZJNIhwDAmvCGIFqO8cuhdOafSrsHJTNtMEk3KUec3fn90xgZyiG?= =?us-ascii?Q?XwyiEIsA+TqSfuaBQp1/TQDqe7g2W/eZZEUZsKTwzXx/RjrGHN2H1aJsdWF5?= =?us-ascii?Q?XFT0q9ur26H1E4dRHrfM6APYzzsLFFmgJidd9RIXfVgBAg52yKIY7oFnBz0C?= =?us-ascii?Q?sm9CpYJ8AS+aQGXst37+7fe8TMKI5VFTElOogl9TH2RkW9V1pqfDTmObESip?= =?us-ascii?Q?GSUbAeRKwI8w2EEp9eMId17gd1z4dmK0FFHoi8EmL2na5GgMN7a/PN3y4jgA?= =?us-ascii?Q?zwhV2QCOIjF61m1VDzs++ROfc7PsCXG1pEMwmESO4kAYBEIzWmnX1let6/2Y?= =?us-ascii?Q?LlrE/foN+yF1uXeAHSrYPkQywdCCJRZY5FQvHoTNWfqK60zCmhsIS+3OmDfp?= =?us-ascii?Q?ngprDHSUwP3xl6KguPxeRT8A/lb/T0e9Pra4AOOlGPpyJE4KwzUrfq93ITCo?= =?us-ascii?Q?dQoT0CWHJRWeNin0dyaB3PwKwjJPOEfMIeRQqQQMBwFH4G9O9/3wYTaKtwYa?= =?us-ascii?Q?/y+37dyXVy/G0auCt3OUmz0uiZWHfPrhMNewiDZW6n++IZt4ykRD3nsfTgR7?= =?us-ascii?Q?DVoJLvpnh6bzHvxZwcVECcKDBGkMd6V7yQvcvV2R0Oq7AVKGtAmVBOZSoVGw?= =?us-ascii?Q?1oSBBVIpyeLebN3nXcQQw7MwHnCzq2CKhHswHc8PL9PXTJOg9/I6IwW7bGe8?= =?us-ascii?Q?okGDSDBhpSLWh36b8e1d6hdFepjlsq/jeRH+lwCBfqz8CDG3t8yE8BhLrz2y?= =?us-ascii?Q?Nc9eeEsFGHJQGKNxK4CliGPMTShKWyNFIyb7oO39uUaA2NxjVU8AR5y1iDUP?= =?us-ascii?Q?AuH66Hv9QDi96xXy3+lrboA0t+U2C50wA+Z1YTqEh4LVLTnvHTyUmKCH2Nbs?= =?us-ascii?Q?LCmH4uFjgW1m6aOjGZToCFtdFCG2k9FXAvyGcuWwFoW+XCbX1el+gfXRI6Ir?= =?us-ascii?Q?0enoV1Z/oEr3wmoOIhJSjSNY9D6OaUE+fZMYblfIw8JxIohCbfYeazsMU57I?= =?us-ascii?Q?JF7BvRa8bjMiTMZVmlmpwniEDqX6HXAVKh6Iys5cs5FcRwQG7v00cT+z8mlK?= =?us-ascii?Q?oe6lBHVTHYHu+JnlH6k8kMNyAFB1wYuTCN92iS2KT5Qpy4RWu93D?= X-OriginatorOrg: valinux.co.jp X-MS-Exchange-CrossTenant-Network-Message-Id: 0cb9e6fc-fb5a-4efc-44e7-08def88a68ab X-MS-Exchange-CrossTenant-AuthSource: TY7P286MB7722.JPNP286.PROD.OUTLOOK.COM X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 12 Aug 2026 15:57:27.8930 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 7a57bee8-f73d-4c5f-a4f7-d72c91c8c111 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: sT6VBTdEY4dZhMXU6ONe3ssJkC+TK2lNC7wLDx3sBz0f53utgmdBihEFHDw/giSWgDoIj9jolbrAbmcv2SLuOQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: OS7P286MB7115 Content-Type: text/plain; charset="utf-8" Add a core operation for clearing the control word of one linked-list entry. A later patch uses the operation to invalidate stale CB when resetting a circular LL ring. No functional change. Reviewed-by: Frank Li Signed-off-by: Koichiro Den --- Changes in v5: - No changes. drivers/dma/dw-edma/dw-edma-core.h | 6 ++++++ drivers/dma/dw-edma/dw-edma-v0-core.c | 16 ++++++++++++++++ drivers/dma/dw-edma/dw-hdma-v0-core.c | 16 ++++++++++++++++ 3 files changed, 38 insertions(+) diff --git a/drivers/dma/dw-edma/dw-edma-core.h b/drivers/dma/dw-edma/dw-ed= ma-core.h index e50fa7965778..089f913fd247 100644 --- a/drivers/dma/dw-edma/dw-edma-core.h +++ b/drivers/dma/dw-edma/dw-edma-core.h @@ -138,6 +138,7 @@ struct dw_edma_core_ops { void (*ll_data)(struct dw_edma_chan *chan, struct dw_edma_burst *burst, u32 idx, bool cb, bool irq); void (*ll_link)(struct dw_edma_chan *chan, u32 idx, bool cb, u64 addr); + void (*ll_clear)(struct dw_edma_chan *chan, u32 idx); int (*ll_cur_idx)(struct dw_edma_chan *chan); void (*ch_doorbell)(struct dw_edma_chan *chan); void (*ch_enable)(struct dw_edma_chan *chan); @@ -255,6 +256,11 @@ dw_edma_core_ll_link(struct dw_edma_chan *chan, u32 id= x, bool cb, u64 addr) chan->dw->core->ll_link(chan, idx, cb, addr); } =20 +static inline void dw_edma_core_ll_clear(struct dw_edma_chan *chan, u32 id= x) +{ + chan->dw->core->ll_clear(chan, idx); +} + static inline void dw_edma_core_ch_doorbell(struct dw_edma_chan *chan) { chan->dw->core->ch_doorbell(chan); diff --git a/drivers/dma/dw-edma/dw-edma-v0-core.c b/drivers/dma/dw-edma/dw= -edma-v0-core.c index cc8059182940..bb86d2f60ab5 100644 --- a/drivers/dma/dw-edma/dw-edma-v0-core.c +++ b/drivers/dma/dw-edma/dw-edma-v0-core.c @@ -605,6 +605,21 @@ dw_edma_v0_core_ll_link(struct dw_edma_chan *chan, u32= idx, bool cb, u64 addr) dw_edma_v0_write_ll_link(chan, idx, control, addr); } =20 +static void dw_edma_v0_core_ll_clear(struct dw_edma_chan *chan, u32 idx) +{ + ptrdiff_t ofs =3D idx * sizeof(struct dw_edma_v0_lli); + + if (chan->dw->chip->flags & DW_EDMA_CHIP_LOCAL) { + struct dw_edma_v0_lli *lli =3D chan->ll_region.vaddr.mem + ofs; + + lli->control =3D 0; + } else { + struct dw_edma_v0_lli __iomem *lli =3D chan->ll_region.vaddr.io + ofs; + + writel(0, &lli->control); + } +} + static void dw_edma_v0_core_ch_doorbell(struct dw_edma_chan *chan) { struct dw_edma *dw =3D chan->dw; @@ -669,6 +684,7 @@ static const struct dw_edma_core_ops dw_edma_v0_core = =3D { .handle_int =3D dw_edma_v0_core_handle_int, .ll_data =3D dw_edma_v0_core_ll_data, .ll_link =3D dw_edma_v0_core_ll_link, + .ll_clear =3D dw_edma_v0_core_ll_clear, .ll_cur_idx =3D dw_edma_v0_core_ll_cur_idx, .ch_doorbell =3D dw_edma_v0_core_ch_doorbell, .ch_enable =3D dw_edma_v0_core_ch_enable, diff --git a/drivers/dma/dw-edma/dw-hdma-v0-core.c b/drivers/dma/dw-edma/dw= -hdma-v0-core.c index 19da1c8fd615..47d7a01658eb 100644 --- a/drivers/dma/dw-edma/dw-hdma-v0-core.c +++ b/drivers/dma/dw-edma/dw-hdma-v0-core.c @@ -374,6 +374,21 @@ dw_hdma_v0_core_ll_link(struct dw_edma_chan *chan, u32= idx, bool cb, u64 addr) dw_hdma_v0_write_ll_link(chan, idx, control, addr); } =20 +static void dw_hdma_v0_core_ll_clear(struct dw_edma_chan *chan, u32 idx) +{ + ptrdiff_t ofs =3D idx * sizeof(struct dw_hdma_v0_lli); + + if (chan->dw->chip->flags & DW_EDMA_CHIP_LOCAL) { + struct dw_hdma_v0_lli *lli =3D chan->ll_region.vaddr.mem + ofs; + + lli->control =3D 0; + } else { + struct dw_hdma_v0_lli __iomem *lli =3D chan->ll_region.vaddr.io + ofs; + + writel(0, &lli->control); + } +} + static void dw_hdma_v0_core_ch_doorbell(struct dw_edma_chan *chan) { struct dw_edma *dw =3D chan->dw; @@ -425,6 +440,7 @@ static const struct dw_edma_core_ops dw_hdma_v0_core = =3D { .non_ll_start =3D dw_hdma_v0_core_non_ll_start, .ll_data =3D dw_hdma_v0_core_ll_data, .ll_link =3D dw_hdma_v0_core_ll_link, + .ll_clear =3D dw_hdma_v0_core_ll_clear, .ll_cur_idx =3D dw_hdma_v0_core_ll_cur_idx, .ch_doorbell =3D dw_hdma_v0_core_ch_doorbell, .ch_enable =3D dw_hdma_v0_core_ch_enable, --=20 2.51.0 From nobody Tue Sep 29 04:14:12 2026 Received: from TY3P286CU002.outbound.protection.outlook.com (mail-japaneastazon11020112.outbound.protection.outlook.com [52.101.229.112]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C510C45C70B; Wed, 12 Aug 2026 15:57:34 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=52.101.229.112 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1786550256; cv=fail; b=tEx11WZJb7IccCwQ+6Ai05xZj4bqXGYVCnaNsWxfeVyA/PklBYNIhNhEC9pp+87pnV65pCNP/QiYM+0BWdxteT/3XwihC2L2tKnCm03/B28d6EIfXs5qPv9i7TNRReHfVD7azn8F1j07b96T0DG8UZOUIUArGVcY9DSDnjqSJYc= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1786550256; c=relaxed/simple; bh=4OEduEHBjVr7Fs442YJBEJVKhItyTZwn87ek/Pw3ksQ=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: Content-Type:MIME-Version; b=Wr8XMBoHS/UWQXeDcqnKq0v1EJJFl0st4UwDR6QaI2ZcvUsVz5ezMe29H1DpIc8AiAkZciS07HzuOPgCgPz7vkwDsvHiqVES1vmFSK4s3rxqc2iPyzFQ6SJZ1qrwvebW2RvQ1EVpvjBWVG7FGNebAo2e+h/cOQARRz0ACPysqSY= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=valinux.co.jp; spf=pass smtp.mailfrom=valinux.co.jp; dkim=pass (1024-bit key) header.d=valinux.co.jp header.i=@valinux.co.jp header.b=Dx+qW4yV; arc=fail smtp.client-ip=52.101.229.112 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=valinux.co.jp Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=valinux.co.jp Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=valinux.co.jp header.i=@valinux.co.jp header.b="Dx+qW4yV" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=aGvqVMd8TPYqFvBH0JB8xwvC5tSylLGWcZJ2v1mM59d9Hnv+z7KqGAi7hHx8TH/DoPtWkln9GYyQhkqLXwNMa95e9y1Act0oaJhNEQccuFbrwMWCjHOaedrcVwzXq8x/lBwmnVNV/P62sW5wq6GuNZcuPFCyOJGeReEgZSgM4/s8rkpGhSBq+69UUrqL5nhBsHEw/CSGJ79P8Y/9K3aj5Vio76OLsFG5oK9VPjjzedc1Dx5Gl9gwIxrnM9OigV2RoPB2LHQpEXdBQ0LAd92UY66f0xPwMX77eAp6XPw3MdN6QJomb6iYayTK2tJfJ2XE+haHSBo3HlANEvmBeIzMog== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=EgezhNz0+YTzTaj5Hn/gMkAyBPBQaQ3mEHVuBWk6mrI=; b=Dmo6LAqO+zSEUwcwIG4K5psL9FFbPGU98VH55cCI1Aj8bnbdtaRYZ+ZB2bEoIVBdM6SxqQiUmSfu1XXxZoteYLt5Gucq0Rg3ed65m/Q6PrLN223PD9U/2A8HBnrr8wGMZMnAJF/SPtjDsMUJGNhO3Xf3O4hprvCs0CVmgn9Gb4QebPWGWJW8vQ36vKDUE6vP3IkPD7bp2+uri1Ww8BupaQML4Gk/cCjxJJzNcE/lw+79AKVW2nM+Q++thtjj/wrdxqFdTGpawTSAz2kx0AmIRjELmfelrA2PMQ8cmAOZKrgdw5l7BW6lGB/Im81vOTFADNAg9v4O6MsKGh+ygTThyA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=valinux.co.jp; dmarc=pass action=none header.from=valinux.co.jp; dkim=pass header.d=valinux.co.jp; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=valinux.co.jp; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=EgezhNz0+YTzTaj5Hn/gMkAyBPBQaQ3mEHVuBWk6mrI=; b=Dx+qW4yVScwA3bGMfalMGe1esHSvLVUKhMIyUvxkLIxn6xTc0UOebvzx9dLRzz2bFsCLHfHAyAVKexMaG9ddnk3wRxqMhTZTeWKdbivqqo7F+IZB1dCpuI868sjBCtDDkHCSsG9E0IVECdo07ydhNIm6k+ApI3MaTZk+KmdJCtI= Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=valinux.co.jp; Received: from TY7P286MB7722.JPNP286.PROD.OUTLOOK.COM (2603:1096:405:38f::10) by OS7P286MB7115.JPNP286.PROD.OUTLOOK.COM (2603:1096:604:42f::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.315.14; Wed, 12 Aug 2026 15:57:28 +0000 Received: from TY7P286MB7722.JPNP286.PROD.OUTLOOK.COM ([fe80::2305:327c:28ec:9b32]) by TY7P286MB7722.JPNP286.PROD.OUTLOOK.COM ([fe80::2305:327c:28ec:9b32%3]) with mapi id 15.21.0292.024; Wed, 12 Aug 2026 15:57:28 +0000 From: Koichiro Den To: Vinod Koul , Frank Li , Manivannan Sadhasivam , Gustavo Pimentel , Kees Cook , =?UTF-8?q?Krzysztof=20Wilczy=C5=84ski?= , Kishon Vijay Abraham I , Bjorn Helgaas , Christoph Hellwig , Serge Semin , Cai Huoqing , Niklas Cassel Cc: Devendra K Verma , dmaengine@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH v5 03/24] dmaengine: dw-edma: Factor out linked-list transfer start Date: Thu, 13 Aug 2026 00:57:00 +0900 Message-ID: <20260812155721.2807506-4-den@valinux.co.jp> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20260812155721.2807506-1-den@valinux.co.jp> References: <20260812155721.2807506-1-den@valinux.co.jp> Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: TYWPR01CA0024.jpnprd01.prod.outlook.com (2603:1096:400:aa::11) To TY7P286MB7722.JPNP286.PROD.OUTLOOK.COM (2603:1096:405:38f::10) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: TY7P286MB7722:EE_|OS7P286MB7115:EE_ X-MS-Office365-Filtering-Correlation-Id: 6bfc65db-ddfa-4ef0-bbc2-08def88a6931 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|376014|7416014|366016|1800799024|23010399003|10070799003|18002099003|22082099003|56012099006|10067099003|921020; X-Microsoft-Antispam-Message-Info: WrdiTU3qjgGiWurhJ78+inuR5A7Zm47hmuJdaZFQTuPw7Fu3m3r7EiynAVe4E7LA6IeaCE7ViuWe4ZvC/qviPAqxTlAU0j2GmndVK0rXpT6O9R6o5DuanbU7izjPcAXfQKKWeTYR9rz+54l+ONo6le3c9/9YRVuxV/6C3T+SiZzEqqBC8eWjWd9BguD6BNjTtpdxHnCXF2yrKVvULvRH9abo+4IXrs7Vf4Ihzz6i543OHWVu4qSj8V8ujyoB6Eb6Y+480jpqZIjK3riI2nMZ/n4D61Xnl86vftr+MOEK0GQSqYlZAdLgTRM+StQh5hvin0ZY5fAHJSdIrNV+Eg5fPXaSwYg9GqZ/G9JBVSBxoBtFm/dja+18t7aZgZDtQ23YpG08dQY2CgnQx3xFfQuBpWaRQBFrG2Bcf70KUozrEdzxX/eD9VHIYfBwCD+dMqxorKcl3sTw8/ZGxbRV+WESs80tBov8isLO0PEZaoCjkmXTu4bIzbyHN2XZdu2A1oJeImMm9iCUEXk5FxShqqueT+WnJI3NRulKr2Bh+Td/qTVcEcfNKU1M6jEPH/2yQU+vM/ZyPMd3fSFLwCzm94FK0ivFMLdKXJ+/Z2ND6L8p+jGNCmS8fVo0JqORQf4/aATlUociRQM2JA2volYgPPFTx8JwSvutjSVMUAUC0kkvJrQ63GrNRZ4vOV1hYskdTlsm55vwmvxz99b6qNtvvGfbmQ== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:TY7P286MB7722.JPNP286.PROD.OUTLOOK.COM;PTR:;CAT:NONE;SFS:(13230040)(376014)(7416014)(366016)(1800799024)(23010399003)(10070799003)(18002099003)(22082099003)(56012099006)(10067099003)(921020);DIR:OUT;SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?5K+YcDYpvV60er2LuRW6kIV+dMFL/u2fMwSSAqJxlIXd16/1FleSHzXlLtcq?= =?us-ascii?Q?fsDDA2/byOnj7FEs8FtSRwbY1QMxZCXCJ3ic2Q3gucST13f9m0EH4gGn6rka?= =?us-ascii?Q?Fkao63ECTZI1P9djDW+gk8wDoFCrGzC2MXCelKiwNyWAYvdUcQ48XGUAqJH9?= =?us-ascii?Q?eV/WlV6eN6MscTjbLfGqn2zJpUR/5X9WehADHG38RNyVEqE03gsfIse8Ki88?= =?us-ascii?Q?+5Gzo8JwlFm5cS/wUZJJ8jjJXpl7cWrlbTQeMrJkAvZoEnp0a65e5SvXmVK1?= =?us-ascii?Q?KT5ITtUotL9qlumSXyqE8kbMf9GSJbx6XqkR2/RyBZ/vc1iuN84+eV5/v3EK?= =?us-ascii?Q?X7lBIXNVJ5xpJjhmMat5iNMVl/bFPC6mIWcCuTdmrQzhSeVjpx2x3MT1/5ol?= =?us-ascii?Q?9CFTaB4oboUcKLqESgOTPN24JP6OxkkD+uDZKr0aC6xnrxNv2v4qxePnmL/Z?= =?us-ascii?Q?J0HyDjWs3BvXqYG9+XTc6STd7snNf3qAOG/G4nY7frgkhICuEDCf0elYzDLF?= =?us-ascii?Q?jEjTxVRRC2v7B0zE+9D1bf0bgPZlOp5dmUUZBB8zIaEEqPPSsAcDMFw5Vdv/?= =?us-ascii?Q?b1I8C1IowBBGNSs+Oc8eeoK2TezpLNamoKEoB9YHuLnpT8V2U64NBZr2XY2q?= =?us-ascii?Q?3zhLvbVwvgghohjVkxulpwWGfsCN5cWNDboYS7b9tGjl88iJaqqKtZt8gx4q?= =?us-ascii?Q?uvNcTnkoP3InFADDv6T/v2gzwU7jxuEUJqHmWojGBvPRyEhh9B5L+2XhA3lh?= =?us-ascii?Q?ZAjlDdO1QhexqOhcqMYNUmB2QWUedPckDkwCOaUxRFKo3u3S8xtqGhidbji6?= =?us-ascii?Q?Ywj8Vrc4EjnOKGypp0/z3C/w5kkFvdOAFsy7a0cqPmfsJPgKgxWOsfuu7z86?= =?us-ascii?Q?HcR3j+ssPTzmg1GBXrR1hoid4KE2Ph3SI7cItzxdqOXuARVQ/fMb0ibawCIb?= =?us-ascii?Q?M9vY/S/HO6Is3AzZ1MV17nrqzbrWaYfim9UCPOzdYCIIPhNSQzX7+wQCJV0D?= =?us-ascii?Q?8Bj+RHHBo7W66U+8uqSzuXtRRMkJtCUz3lR6Ec9gqeLzkuphBSyRjrsd2Kvg?= =?us-ascii?Q?XjWeAeuP8B1gG0kPkM73XcrQvXKGnZuZgqdKAG9YXDpck5ZScUJmPDjFxkB5?= =?us-ascii?Q?O2njn5T2H+Cj8CA/7wGXmiKf09L35rbCvl8bKZ2LrKkrhBlN5+vBY2T46HPU?= =?us-ascii?Q?fJZq+86Tbdn/v82w3Ab7hRXgHT4NGOrZ3Xkm8Iv8tDKrNqzoEUMBdxNUpqfI?= =?us-ascii?Q?HuEDxRAtnCujIC6yqgQmZJ1/1F7lJITSwtB1vM/J/kQWw5FIWdTg4nvxX2P4?= =?us-ascii?Q?muPFLkq7CBsaDJl1smzkoBe+C24hqzDPFDY367kBORnZPIPlDZNy25Bv2TSw?= =?us-ascii?Q?iprPXk1K7CcQYrowrx8SDyGKdDMuHsQbBT1eOWik7DjSDxaVHFnRQxI6w3jm?= =?us-ascii?Q?1twO76S1Ia0GGurF1/WPWKk5La9BOfHyvu+COfmucKksAxDrjx76rOo0avAz?= =?us-ascii?Q?e1PN31b8cj86Sn4GU1VshQwzwWUKsx8gMLtJL2GjQv9ElmSYz4dQhyT4koCS?= =?us-ascii?Q?i8a0Q3dJKK4UsS5TyRYSCtcWDW5JlHQO4dyF6LIkfR9+pzmrEfLnX25zzU5B?= =?us-ascii?Q?/WfCx6GgbEo3m5QWbACnw3N2Md4w12vGUkZTpgoJ6ndClhP3h3BGhdH9mAx/?= =?us-ascii?Q?z42Yq7W5raLMYK+G2pysp+p0/7rVw5jirmlhAyyDh85x7AEcyoPdxmFElmN5?= =?us-ascii?Q?6JT8U9FGxpkuRf9iiKGRqyFNs1FgRVJosD6aWWfNqnEsbzm12DwL?= X-OriginatorOrg: valinux.co.jp X-MS-Exchange-CrossTenant-Network-Message-Id: 6bfc65db-ddfa-4ef0-bbc2-08def88a6931 X-MS-Exchange-CrossTenant-AuthSource: TY7P286MB7722.JPNP286.PROD.OUTLOOK.COM X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 12 Aug 2026 15:57:28.7727 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 7a57bee8-f73d-4c5f-a4f7-d72c91c8c111 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: VizInyetycHATsrlscDvqeYQTo9k0R/k55ptMNjMdnm4gcKzCtPn0UkLUBI7S1RDTh5XIZjZSBdO76rwdOuDGQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: OS7P286MB7115 Content-Type: text/plain; charset="utf-8" Move linked-list programming into dw_edma_core_ll_start(). It can derive the first-batch state from desc->start_burst, so dw_edma_core_start() no longer needs a pass-through argument. No functional change. Reviewed-by: Frank Li Signed-off-by: Koichiro Den --- Changes in v5: - No changes. drivers/dma/dw-edma/dw-edma-core.c | 29 +++++++++++++++++++---------- 1 file changed, 19 insertions(+), 10 deletions(-) diff --git a/drivers/dma/dw-edma/dw-edma-core.c b/drivers/dma/dw-edma/dw-ed= ma-core.c index 03b2c2188351..a6bb68ffdfe2 100644 --- a/drivers/dma/dw-edma/dw-edma-core.c +++ b/drivers/dma/dw-edma/dw-edma-core.c @@ -67,17 +67,11 @@ static void vchan_free_desc(struct virt_dma_desc *vdesc) kfree(vd2dw_edma_desc(vdesc)); } =20 -static void dw_edma_core_start(struct dw_edma_desc *desc, bool first) +static void dw_edma_core_ll_start(struct dw_edma_desc *desc) { struct dw_edma_chan *chan =3D desc->chan; - size_t i =3D 0; - - if (chan->non_ll) { - chan->dw->core->non_ll_start(chan, &desc->burst[desc->start_burst]); - desc->done_burst =3D desc->start_burst; - desc->start_burst +=3D 1; - return; - } + size_t i; + bool first =3D !desc->start_burst; =20 for (i =3D 0; i + desc->start_burst < desc->nburst; i++) { u32 idx =3D i + desc->start_burst; @@ -101,6 +95,21 @@ static void dw_edma_core_start(struct dw_edma_desc *des= c, bool first) dw_edma_core_ch_doorbell(chan); } =20 +static void dw_edma_core_start(struct dw_edma_desc *desc) +{ + struct dw_edma_chan *chan =3D desc->chan; + + if (chan->non_ll) { + chan->dw->core->non_ll_start(chan, + &desc->burst[desc->start_burst]); + desc->done_burst =3D desc->start_burst; + desc->start_burst +=3D 1; + return; + } + + dw_edma_core_ll_start(desc); +} + static int dw_edma_start_transfer(struct dw_edma_chan *chan) { struct dw_edma_desc *desc; @@ -114,7 +123,7 @@ static int dw_edma_start_transfer(struct dw_edma_chan *= chan) if (!desc) return 0; =20 - dw_edma_core_start(desc, !desc->start_burst); + dw_edma_core_start(desc); =20 desc->cb =3D !desc->cb; =20 --=20 2.51.0 From nobody Tue Sep 29 04:14:12 2026 Received: from TY3P286CU002.outbound.protection.outlook.com (mail-japaneastazon11020119.outbound.protection.outlook.com [52.101.229.119]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B9D1F448D02; Wed, 12 Aug 2026 15:57:35 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=52.101.229.119 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1786550257; cv=fail; b=AX7+ITP5pl6pUwEg9zphtobzjkSZ9u5YSlP83h16z53JX45UMC+jMArBut0Pev3LMWEFdljtdCS77a9Lo12qwuiX+Bn6QDGzu+HDcsyVqPvr4ldBgGVqJcp6tFHqs6pKBMet3H92nxbPa2GNKwdBgabuJ26ZnGnZ0HyPJyA79co= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1786550257; c=relaxed/simple; bh=1mRYD+w1UVqNtDR2dkJAa+mvKSsPIrAJiTq+3AC/KSI=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: Content-Type:MIME-Version; b=n9EzBMgTkxJnIUzzAt+HvQ9UUD101HCLPsg/kJaZ4498Bq/xyqH3BnVbjVGHUigxyXwfw41oY5PEZjAdUQNlZaWEwOARnHO6eVwFCZ7ZpyC4G4xGHo8WLR3R7ePS29oZGmIu873KeHoVriUvSbLYJQIlozZQoapgGtS2RgQv67o= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=valinux.co.jp; spf=pass smtp.mailfrom=valinux.co.jp; dkim=pass (1024-bit key) header.d=valinux.co.jp header.i=@valinux.co.jp header.b=Ts970kmY; arc=fail smtp.client-ip=52.101.229.119 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=valinux.co.jp Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=valinux.co.jp Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=valinux.co.jp header.i=@valinux.co.jp header.b="Ts970kmY" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=ElrSlM/NPZaXMneyVH3xKmQqv7wogAx9y3kL20Orj7Uv8gBL2ZaRL8rhCUTkfvKhyat3QJIayckX/lP3Sa0P/DDZptDaB8JGx78ZG+A7r6RjzmH511x6FfwzleySRnDV1DN78SuKdx5MICt4NHRsnC3ZdfETr6gUqmA7TMnCKh7Rpoca117H2ON4vHzh98BxR+HXG3bWD3iaFAJzuYJDxKcJ7dnjqWQiU0vtW+98XmJ3McGofKqpyi3mxyIuNaPf5/TPED9H5RwMCOWCA5i+u6EKbfZzexVCq/LbOzzXy0zhgEjfS4j+qnvzcYdDXVncpFTQK9m2gwSgc6fl6Zqq1g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=gHUaRrwSBTJcpawM1hcDvRJJg2U0+tS3ijt6SCL8nUk=; b=GVTmJwNS8UwdznbZA8THtVZ4MLGFRHQpj1VtBxD1vz94Kz79G2JmF4s7SAzWxJ4E1Ur6LlTxDpvkfUQy3F//fpBdt912h5tt+SdzcssC63mLr4L0G0RayY0kxIXJn+0f79N1JifaclSLbY8y9Wj/ck6bYnNwYmEJOZ29HWk7Kh9wmWGJ2Tz6LADXEo4NIi7Infb6HwU6EtiF4er+9l7damYeBpBS4YY716KuNbEZ+Ef2l3wnQ7oXh/dwwajMeq+jhd4T339hyMFZhwt/vk9f2q9B4oaExbzjHF5jeu5HPqaqqxDOsKydhOF0F4et4C/6LEWp7lc2KMkjhgBEm3LaiA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=valinux.co.jp; dmarc=pass action=none header.from=valinux.co.jp; dkim=pass header.d=valinux.co.jp; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=valinux.co.jp; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=gHUaRrwSBTJcpawM1hcDvRJJg2U0+tS3ijt6SCL8nUk=; b=Ts970kmYz69gbprFfsZheq+9TD0ewJXR1IQS1/MIrOLY4InjFhoJMnwvce7925Qs7BQzoOe3paG9EO8Bzo/OG1tkFivGHX7IGwBywaodWK7bD8NYNOBFda8zUcX9SpUCAITPPFm6AQYXfjBgDwXvylkqKuvNBa3PpBzt+ilCrm0= Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=valinux.co.jp; Received: from TY7P286MB7722.JPNP286.PROD.OUTLOOK.COM (2603:1096:405:38f::10) by OS7P286MB7115.JPNP286.PROD.OUTLOOK.COM (2603:1096:604:42f::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.315.14; Wed, 12 Aug 2026 15:57:29 +0000 Received: from TY7P286MB7722.JPNP286.PROD.OUTLOOK.COM ([fe80::2305:327c:28ec:9b32]) by TY7P286MB7722.JPNP286.PROD.OUTLOOK.COM ([fe80::2305:327c:28ec:9b32%3]) with mapi id 15.21.0292.024; Wed, 12 Aug 2026 15:57:29 +0000 From: Koichiro Den To: Vinod Koul , Frank Li , Manivannan Sadhasivam , Gustavo Pimentel , Kees Cook , =?UTF-8?q?Krzysztof=20Wilczy=C5=84ski?= , Kishon Vijay Abraham I , Bjorn Helgaas , Christoph Hellwig , Serge Semin , Cai Huoqing , Niklas Cassel Cc: Devendra K Verma , dmaengine@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH v5 04/24] dmaengine: dw-edma: Make DMA link list work as a circular buffer Date: Thu, 13 Aug 2026 00:57:01 +0900 Message-ID: <20260812155721.2807506-5-den@valinux.co.jp> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20260812155721.2807506-1-den@valinux.co.jp> References: <20260812155721.2807506-1-den@valinux.co.jp> Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: TYWP286CA0014.JPNP286.PROD.OUTLOOK.COM (2603:1096:400:178::20) To TY7P286MB7722.JPNP286.PROD.OUTLOOK.COM (2603:1096:405:38f::10) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: TY7P286MB7722:EE_|OS7P286MB7115:EE_ X-MS-Office365-Filtering-Correlation-Id: 87154d23-1dce-4e82-9930-08def88a69a0 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|376014|7416014|366016|1800799024|23010399003|10070799003|18002099003|22082099003|56012099006|10067099003|921020; X-Microsoft-Antispam-Message-Info: K0sqZlqOlix20hf4AWWdRQyQ66LKSmoFuJqKWSwkmiZpvEFxeL8GVxC5Cc00Ml3vhIt0SKHldtKQODuCXKXVxl7DiTWx334+hmXmpuqr8FvjgkLNcEH7ybnBvle1fHDM7K1WX2NfZhaWXNrkLIqsrUaG85N3DG81ku91Aa5xtPkVv8vK7Tro2QLKVXntG+gusp+bTB3jWvimUhwY/DgjWvwgRQiFJL4zMxa+/7d5oP4Ldu8AJ7rQ3JxwI1oMxZBTbl221zLaNzQOtpXaU7V2mbTdFzcc9uLkRkD/8zwFFQNrAioueeAsEe/SpXwt7IOhdFhayzjPzXZdMBFKM7bdbNP8rbd3pzABJ9HASoNeNgS91IDPlNPVlDexTl6vujrhydaKM+tBuTaHGS0I8zJutdjjwgKXeRVKKgp2Uyxg/mZISukF+uiLxBYPOkV3D+GiRnT1iywQ2sz7H0r/PQ/KX5leaqvNXd1zYUtj/VUEY5opSDN9RMfQ7i7oJ7lfpU/FMq7KGCwYO3z1+ScITW7vnXU7TFeJmvWSB/sRU6/FaXtYT7a5rDmsLBhga6SSw7mA/6vW1UT3aGRK5aE25Xfrcknot2a8yMVETFsgoLH+48yvVRENrJp6mEqn2GZ660erIeTXXfZx+jx1niX+d0rqZf/xEDvnX+tEQ8Ozy/rvHspHMnEIUU3Vqw/fHQgh+9ndDx913KYFKleoSNg7vi35UQ== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:TY7P286MB7722.JPNP286.PROD.OUTLOOK.COM;PTR:;CAT:NONE;SFS:(13230040)(376014)(7416014)(366016)(1800799024)(23010399003)(10070799003)(18002099003)(22082099003)(56012099006)(10067099003)(921020);DIR:OUT;SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?gPP4mOy4N8RZdJPrwH/0+1s2D2sRitl8+s+V5gZnpHGdoETwH4YN2bv2fI3h?= =?us-ascii?Q?AqKoAW2S6181UeNdKs8xuHZkgyOKOvM4PnXQiD4lByxOOPdVPtoOSSoTuX/p?= =?us-ascii?Q?9iwK/058Z4QC5GG2TgVexPlxWx0zQ962A51TQRFqGK3TE9OTwsXYKF4n8VXs?= =?us-ascii?Q?OHf1KajLxyRRRIoJu/dnQYoSqv9/Ph4Bhvy+T/RLgluo5OF1vb+Exy6Y4FNW?= =?us-ascii?Q?XI0c2wIGyM/VEm62x5etUfc7pntnL0ok7JJqAFrF+VVgaKhSrGVoxwSGmJzR?= =?us-ascii?Q?ScL1Mww9YYorq0Ca/hjkdH/x3GuJD7xUCfx5DZYutWTJPmksYg1bmqDef169?= =?us-ascii?Q?hNSsu9MjLPiXo6UBL8MXAOeJ2U2i/OP5GU/ZTDAlaG98eU+423e5M0h8fnUh?= =?us-ascii?Q?UdGjTYedpBWaZkf80fo3x2idnIaMN3LuowNteaC/QO44rkyJc6OjmBrPgvzj?= =?us-ascii?Q?6zkdhjeB6PwKNjiE7HxWJwGn+wLuvnHA04FAhT7aj529d8QCJax3phwpRN7q?= =?us-ascii?Q?ToVxD7m2CKObpBhuknjEFNDW/TwQ+M9G1hymf269JsxQjoKwuaXri8x5YID9?= =?us-ascii?Q?Djvi05pXvGSI2VWGeq00kGoA3ZhWVQTJx9iywypUrraXC1MhPMjYMux1A/4n?= =?us-ascii?Q?oZ5OpOFnT53LFPX3O+tY/ZX8JiwLTjJL7W/tEb9JgZMcUVG4Xc6zv5qsNHbr?= =?us-ascii?Q?Wog9W7nfLzeV2TgVC+npXqF64q4pjt4ftTDGtLASkWeZD/TpJXqSoIVW9+bM?= =?us-ascii?Q?VZOad6mHCvIgV2K/XWyTDylI+9ZiB+6H/wk9QyJck6v7jG5PdDtLbGjwwyWv?= =?us-ascii?Q?QHH/JreRAEE0M2KI79mWMRtzNte4xWTgWlVYHnQqlodthlQZL302o/wPmIWs?= =?us-ascii?Q?eNJ8asFY8Ewh1JYKkJZeCuHS/8PclvY3scaiJ+d95Ec9vB3ByfY6/t0cp0Pr?= =?us-ascii?Q?PcMCff6JgkES3fiz6ANOIizc5EHAvOXueG/IhPIj2i2AgwboawGwSg6Fn4av?= =?us-ascii?Q?cTG6t3ov4V2u++tgg9Ue7LDQwr8YXRz7PS4g6ZFptxPe+Adz++Xg5GlMS4iH?= =?us-ascii?Q?wfY+QhuTU4QhAw3sY/g/Dirlt5HsvwCikzAQqaBRKHZRyYYroGjgYEarshl7?= =?us-ascii?Q?3W07u1M/KGZjx5z2SUidpBiUa/DxPGltC7Agb30cdZOipRo+JagwJL+7VaEF?= =?us-ascii?Q?bInsskm+AEQXq2yucqLEt5pwXCiZ5Ktd01EkykA8/tVdvoK7qib9Yeu0GdXC?= =?us-ascii?Q?yI5uamffrY1JOmMHTX5NALamzP66sVM45l42lIq8faa48SO47s0JdsIvQu4F?= =?us-ascii?Q?tbFhxFEb8EGtHqpnR8aPnG+AzSM2ehg0n6saZlw8wev6q93gN509Ps6uI/me?= =?us-ascii?Q?JkUszq5oy8QkTppo53Ey/DL4dISQeyaehg3u5XNeNblzqtkOsX6c3rSqWAvG?= =?us-ascii?Q?C4Q6zy9VOaoqhG04ocaMNA2eBziJLIZp7llhLH/uThBrUJoPF07pXi8LBo2A?= =?us-ascii?Q?gOAkxLgL3aFTr0X75kQ4l2VoUSzOXo/8JS5ew25scnnTimOrdp5v0FARarLe?= =?us-ascii?Q?MwBWMbY9a6jy6T436zlT4o+cIXCYn/hGvVs6VWyM5PHWyFovS2iZ9AXn8wQc?= =?us-ascii?Q?HcBGt1dTkw49p8iud5LPQxKVmquSrhh6TOgMVUS8fE35na0Dt6GWYu7/nnIG?= =?us-ascii?Q?ifrSmqijgr6oq+x/yDWCcXtir0yDgd75JOdZMw+MKi+6y85GoRlIn3wPqYGa?= =?us-ascii?Q?khNeYc6n5qABxrEl3X5AlK+GF5FSWQdmZhg7JtKv6zAm05+r6w4x?= X-OriginatorOrg: valinux.co.jp X-MS-Exchange-CrossTenant-Network-Message-Id: 87154d23-1dce-4e82-9930-08def88a69a0 X-MS-Exchange-CrossTenant-AuthSource: TY7P286MB7722.JPNP286.PROD.OUTLOOK.COM X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 12 Aug 2026 15:57:29.5097 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 7a57bee8-f73d-4c5f-a4f7-d72c91c8c111 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: zNzFgERBjlEedBz1C6o+YjexteghH3UCwe0Z/hR4AVdeaOPXrIZstIM9KNKr8WA/8WwKRj9PSqKtOX2n/nWlcw== X-MS-Exchange-Transport-CrossTenantHeadersStamped: OS7P286MB7115 Content-Type: text/plain; charset="utf-8" From: Frank Li The driver currently rebuilds the whole linked list for every transfer. Use it as a circular ring instead. Append entries at ll_head with the current cycle bit, and reserve the final entry for the link back to the start. Clear control words before first use so stale cycle bits cannot become valid entries. Reject rings without usable data slots and, until reclaim support lands, descriptors that exceed the usable ring capacity. Termination and abort can discard descriptors while ll_done still trails ll_head. Reset the ring after the channel has stopped so the next transfer does not inherit occupied slots. This prepares the driver for appending requests while the engine runs. Signed-off-by: Frank Li Reviewed-by: Frank Li Co-developed-by: Koichiro Den Signed-off-by: Koichiro Den --- Changes in v5: - No changes. drivers/dma/dw-edma/dw-edma-core.c | 125 +++++++++++++++++++++++------ drivers/dma/dw-edma/dw-edma-core.h | 27 ++++++- 2 files changed, 125 insertions(+), 27 deletions(-) diff --git a/drivers/dma/dw-edma/dw-edma-core.c b/drivers/dma/dw-edma/dw-ed= ma-core.c index a6bb68ffdfe2..1af262b6e881 100644 --- a/drivers/dma/dw-edma/dw-edma-core.c +++ b/drivers/dma/dw-edma/dw-edma-core.c @@ -51,13 +51,19 @@ dw_edma_alloc_desc(struct dw_edma_chan *chan, size_t nb= urst) { struct dw_edma_desc *desc; =20 + /* + * For now, a descriptor that does not fit would stall the channel + * forever: reject it up front. + */ + if (!chan->non_ll && nburst > chan->ll_max - 1) + return NULL; + desc =3D kzalloc_flex(*desc, burst, nburst, GFP_NOWAIT); if (unlikely(!desc)) return NULL; =20 desc->chan =3D chan; desc->nburst =3D nburst; - desc->cb =3D true; =20 return desc; } @@ -67,30 +73,75 @@ static void vchan_free_desc(struct virt_dma_desc *vdesc) kfree(vd2dw_edma_desc(vdesc)); } =20 +static void dw_edma_core_reset_ll(struct dw_edma_chan *chan) +{ + u32 i; + + chan->ll_head =3D 0; + chan->ll_done =3D 0; + /* Drop stale CB bits before reusing the circular LL ring. */ + for (i =3D 0; i < chan->ll_max; i++) + dw_edma_core_ll_clear(chan, i); + chan->cb =3D true; + + dw_edma_core_ll_link(chan, chan->ll_max, chan->cb, + chan->ll_region.paddr); + + dw_edma_core_ch_enable(chan); + chan->ll_valid =3D true; +} + +static u32 dw_edma_core_get_ll_dist(struct dw_edma_chan *chan, u32 from, u= 32 to) +{ + return (to + chan->ll_max - from) % chan->ll_max; +} + +static u32 dw_edma_core_get_used_num(struct dw_edma_chan *chan) +{ + return dw_edma_core_get_ll_dist(chan, chan->ll_done, chan->ll_head); +} + +static u32 dw_edma_core_get_free_num(struct dw_edma_chan *chan) +{ + /* Keep one data entry free so equal indices mean an empty ring. */ + return chan->ll_max - 1 - dw_edma_core_get_used_num(chan); +} + +static bool dw_edma_ll_pending(struct dw_edma_chan *chan) +{ + return chan->ll_head !=3D chan->ll_done; +} + static void dw_edma_core_ll_start(struct dw_edma_desc *desc) { struct dw_edma_chan *chan =3D desc->chan; size_t i; - bool first =3D !desc->start_burst; + u32 free; + + free =3D dw_edma_core_get_free_num(chan); + for (i =3D desc->start_burst; i < desc->nburst && free; i++, free--) { + /* + * Refresh the link element before filling the last data slot so + * the next lap has the updated CB value. + */ + if (chan->ll_head =3D=3D chan->ll_max - 1) + dw_edma_core_ll_link(chan, chan->ll_max, chan->cb, + chan->ll_region.paddr); =20 - for (i =3D 0; i + desc->start_burst < desc->nburst; i++) { - u32 idx =3D i + desc->start_burst; + dw_edma_core_ll_data(chan, &desc->burst[i], + chan->ll_head, chan->cb, + i =3D=3D desc->nburst - 1 || free =3D=3D 1); =20 - if (i =3D=3D chan->ll_max) - break; + chan->ll_head++; =20 - dw_edma_core_ll_data(chan, &desc->burst[idx], - i, desc->cb, - idx =3D=3D desc->nburst - 1 || i =3D=3D chan->ll_max - 1); + if (chan->ll_head =3D=3D chan->ll_max) { + chan->cb =3D !chan->cb; + chan->ll_head =3D 0; + } } =20 desc->done_burst =3D desc->start_burst; - desc->start_burst +=3D i; - - dw_edma_core_ll_link(chan, i, desc->cb, chan->ll_region.paddr); - - if (first) - dw_edma_core_ch_enable(chan); + desc->start_burst =3D i; =20 dw_edma_core_ch_doorbell(chan); } @@ -123,9 +174,10 @@ static int dw_edma_start_transfer(struct dw_edma_chan = *chan) if (!desc) return 0; =20 - dw_edma_core_start(desc); + if (!chan->non_ll && !chan->ll_valid) + dw_edma_core_reset_ll(chan); =20 - desc->cb =3D !desc->cb; + dw_edma_core_start(desc); =20 return 1; } @@ -159,6 +211,19 @@ static void dw_edma_terminate_all_descs(struct dw_edma= _chan *chan) dw_edma_terminate_vdesc_list(&chan->vc.desc_submitted); } =20 +/* Must be called with vc.lock held after the channel has stopped. */ +static void dw_edma_finish_termination(struct dw_edma_chan *chan) +{ + dw_edma_terminate_all_descs(chan); + + /* Preserve a clean ring; resync only if entries remain published. */ + if (!chan->non_ll && dw_edma_ll_pending(chan)) + dw_edma_core_reset_ll(chan); + + chan->request =3D EDMA_REQ_NONE; + chan->status =3D EDMA_ST_IDLE; +} + static void dw_edma_device_caps(struct dma_chan *dchan, struct dma_slave_caps *caps) { @@ -299,17 +364,15 @@ static int dw_edma_device_terminate_all(struct dma_ch= an *dchan) if (!chan->configured) { dw_edma_terminate_all_descs(chan); } else if (chan->status =3D=3D EDMA_ST_PAUSE) { - dw_edma_terminate_all_descs(chan); - chan->status =3D EDMA_ST_IDLE; + dw_edma_finish_termination(chan); } else if (chan->status =3D=3D EDMA_ST_IDLE) { - dw_edma_terminate_all_descs(chan); + dw_edma_finish_termination(chan); } else if (dw_edma_core_ch_status(chan) =3D=3D DMA_COMPLETE) { /* * The channel is in a false BUSY state, probably didn't * receive or lost an interrupt */ - dw_edma_terminate_all_descs(chan); - chan->status =3D EDMA_ST_IDLE; + dw_edma_finish_termination(chan); } else if (chan->request > EDMA_REQ_PAUSE) { err =3D -EPERM; } else { @@ -645,6 +708,8 @@ static void dw_edma_done_interrupt(struct dw_edma_chan = *chan) DMA_TRANS_NOERROR); list_del(&vd->node); vchan_cookie_complete(vd); + if (!chan->non_ll) + chan->ll_done =3D chan->ll_head; } =20 if (chan->request =3D=3D EDMA_REQ_PAUSE) { @@ -659,9 +724,7 @@ static void dw_edma_done_interrupt(struct dw_edma_chan = *chan) break; =20 case EDMA_REQ_STOP: - dw_edma_terminate_all_descs(chan); - chan->request =3D EDMA_REQ_NONE; - chan->status =3D EDMA_ST_IDLE; + dw_edma_finish_termination(chan); break; =20 default: @@ -685,6 +748,8 @@ static void dw_edma_abort_interrupt(struct dw_edma_chan= *chan) list_del(&vd->node); vchan_cookie_complete(vd); } + if (!chan->non_ll) + dw_edma_core_reset_ll(chan); chan->request =3D EDMA_REQ_NONE; chan->status =3D EDMA_ST_IDLE; spin_unlock_irqrestore(&chan->vc.lock, flags); @@ -872,6 +937,9 @@ static int dw_edma_alloc_chan_resources(struct dma_chan= *dchan) if (chan->status !=3D EDMA_ST_IDLE) return -EBUSY; =20 + /* The hardware context may have been invalidated while unowned. */ + chan->ll_valid =3D false; + return 0; } =20 @@ -963,6 +1031,13 @@ static int dw_edma_channel_setup(struct dw_edma *dw, = u32 wr_alloc, u32 rd_alloc) else chan->ll_region =3D chip->ll_region_rd[chan->id]; =20 + if (!chip->cfg_non_ll && chan->ll_region.sz < 3 * EDMA_LL_SZ) { + dev_err(dev, + "channel %s[%u]: LL region has fewer than 2 data entries\n", + str_write_read(chan->dir =3D=3D EDMA_DIR_WRITE), + chan->id); + return -EINVAL; + } chan->ll_max =3D chan->ll_region.sz / EDMA_LL_SZ - 1; =20 dev_vdbg(dev, "L. List:\tChannel %s[%u] max_cnt=3D%u\n", diff --git a/drivers/dma/dw-edma/dw-edma-core.h b/drivers/dma/dw-edma/dw-ed= ma-core.h index 089f913fd247..761a5ab4bbb5 100644 --- a/drivers/dma/dw-edma/dw-edma-core.h +++ b/drivers/dma/dw-edma/dw-edma-core.h @@ -60,7 +60,6 @@ struct dw_edma_desc { =20 size_t done_burst; size_t start_burst; - u8 cb; size_t nburst; struct dw_edma_burst burst[] __counted_by(nburst); }; @@ -72,8 +71,32 @@ struct dw_edma_chan { enum dw_edma_dir dir; u8 func_no; =20 - u32 ll_max; + /* + * New LL entries are appended at ll_head. Entries between ll_done + * and ll_head, modulo the LL ring, are owned by DMA; the rest are + * owned by software. + * + * software-owned DMA-owned software-owned + * +---------------+-------------------+---------------+ + * ^ ^ ^ + * 0 ll_done ll_head + * + * The link entry points back to the region start. ll_head =3D=3D ll_done + * means all entries are software-owned and previous DMA work is + * done. + * + * Software always keeps at least one free entry, so the ring is + * never completely DMA-owned. That keeps a hardware-reported physical + * LL index unique within the current ll_done..ll_head producer window. + */ + u32 ll_head; + u32 ll_done; + + u32 ll_max; /* Data entries */ struct dw_edma_region ll_region; /* Linked list */ + bool ll_valid; /* LL context programmed */ + + bool cb; =20 struct msi_msg msi; =20 --=20 2.51.0 From nobody Tue Sep 29 04:14:12 2026 Received: from TY3P286CU002.outbound.protection.outlook.com (mail-japaneastazon11020112.outbound.protection.outlook.com [52.101.229.112]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8999646F486; Wed, 12 Aug 2026 15:57:36 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=52.101.229.112 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1786550258; cv=fail; b=QxAljWRx+xfENZbMfe5P6lvFk+Wv4dHIs4a33DDaozd2dkAca5aqKI3BR7Ki27WWBnn1vtLZWj3lbjbr6i+JweqaKXWXJ3PjChXzeCnItnanqdcGi4LlasglUpYQo+0xN+8v+ExI2zHQ7vUrS1rnsLeCZV0oHO9u1LoTCUoYwms= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1786550258; c=relaxed/simple; bh=WRgGhUaJ9tcTgKDdHOF3WbmTB5P4DlOMs4qgD67i128=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: Content-Type:MIME-Version; b=AjKSlUKEWbsZq1lLuR9CIYidmFYl8Vsz8rce0WnHC4wj0Gq19K2RUV39cZavMixAl41E7SdJtiRERoB6PqLoOQdlrwMG6PPkNvMs4UWGb4dlFIpTfl4wvoamANuRntc1M8PgY/QmZ6Lfd6GCA2s2b9Br6rH7hQxeMYaulLBw9uk= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=valinux.co.jp; spf=pass smtp.mailfrom=valinux.co.jp; dkim=pass (1024-bit key) header.d=valinux.co.jp header.i=@valinux.co.jp header.b=uhyKFYZX; arc=fail smtp.client-ip=52.101.229.112 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=valinux.co.jp Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=valinux.co.jp Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=valinux.co.jp header.i=@valinux.co.jp header.b="uhyKFYZX" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=VVvblFY/MKgxy1HLSMkMCKoRZVbqZwqacAfrGvaCIXSKsIxAF5IKr6hcRlTNgOHbN8mlUKmDXQNHc9huwgQX7kPaW1MB6ZCMok00jkie1DibRpaS2aEzr1zyw0AZBTooNOTTOgLzcaA4bwc3hBdHHFcDO2ldAUt8BzEZ8WEvgIPmmTPBnFCdhm04Zbw3r+llfBPJEjDBRae4CWKkUcoEaMQSS1aWMs5dJ39waXDmr9+1BtRGXq4QfjT1+Nlc5sBb/rRhzWJZYy/qZ8YLp3VMy8AXZm67AJIagyBG/mqArTJCBB0JywA6HPgVXTSj2qOJZtryXGW0WkUfei6W8MjjFg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=my5jlaExIwIERKEMRvqBwy3Nw9iy0w3XnI4PIiZCu1s=; b=x5AYKRMfaB2zx1TkahYjWU93sZ0FiGU3AGPsSyoEtcouYKH4EFbNMrUDgLtPmGyUdsY4TL1ZRfErvUTktaet3ZlQaBJaEwYQiI5GXxyjoIhfUOR6gsqUIm2QXCsXJ9K/q5PYIp9qEwjq26Jf4iPS5FmiRj4bVolY5nRtV7uGF9s+9FsdyY4Xb2Z+J+S4gFfuiwnmEIhsPKyyvOpecuGmrwlM7HHkN3uAh31fsFW4dZVHV+cEwxGWWhuhdDj8gjhWH4JpDz2/uZM0K9Liy89MeyxSd/8PHpDHlzBsR04Y6fdldn7RkzbbAKL5zSOsWZJ55L6lCRo4hhSTWmBFlOXlSA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=valinux.co.jp; dmarc=pass action=none header.from=valinux.co.jp; dkim=pass header.d=valinux.co.jp; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=valinux.co.jp; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=my5jlaExIwIERKEMRvqBwy3Nw9iy0w3XnI4PIiZCu1s=; b=uhyKFYZXTBDv5E13jtHN66ZxIzQ15ltlg48x8tqUUshEeMALnYiMeW1iUbslcwwF+KrLfRVO1f1w2qsa716hqPEXEv2rCnHrvhiKVIbUa6k82yZ27JLX2kI5Rv6KrX70fqVvgPNe9UHfwk0g2w6B0ZeN/aLdt8c3FdURRwO/zyM= Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=valinux.co.jp; Received: from TY7P286MB7722.JPNP286.PROD.OUTLOOK.COM (2603:1096:405:38f::10) by OS7P286MB7115.JPNP286.PROD.OUTLOOK.COM (2603:1096:604:42f::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.315.14; Wed, 12 Aug 2026 15:57:30 +0000 Received: from TY7P286MB7722.JPNP286.PROD.OUTLOOK.COM ([fe80::2305:327c:28ec:9b32]) by TY7P286MB7722.JPNP286.PROD.OUTLOOK.COM ([fe80::2305:327c:28ec:9b32%3]) with mapi id 15.21.0292.024; Wed, 12 Aug 2026 15:57:30 +0000 From: Koichiro Den To: Vinod Koul , Frank Li , Manivannan Sadhasivam , Gustavo Pimentel , Kees Cook , =?UTF-8?q?Krzysztof=20Wilczy=C5=84ski?= , Kishon Vijay Abraham I , Bjorn Helgaas , Christoph Hellwig , Serge Semin , Cai Huoqing , Niklas Cassel Cc: Devendra K Verma , dmaengine@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH v5 05/24] dmaengine: dw-edma: Move callback result helper before LL helpers Date: Thu, 13 Aug 2026 00:57:02 +0900 Message-ID: <20260812155721.2807506-6-den@valinux.co.jp> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20260812155721.2807506-1-den@valinux.co.jp> References: <20260812155721.2807506-1-den@valinux.co.jp> Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: TYWP286CA0002.JPNP286.PROD.OUTLOOK.COM (2603:1096:400:178::17) To TY7P286MB7722.JPNP286.PROD.OUTLOOK.COM (2603:1096:405:38f::10) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: TY7P286MB7722:EE_|OS7P286MB7115:EE_ X-MS-Office365-Filtering-Correlation-Id: 4afcd15b-9dfc-4f7a-18d8-08def88a6a28 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|376014|7416014|366016|1800799024|23010399003|10070799003|18002099003|22082099003|56012099006|10067099003|921020; X-Microsoft-Antispam-Message-Info: ByySLXgVEoTzqKPR44oVwaQBNYiom+3Lszpuf9aRoZmp+XK00Zt3w0JO+lKJcgwx/S+Q7NFu6r3lZuk4udwVM70rb1H1/m9o36RlTNvY2PwIJkhqjX5n5j4/L9rsVGkklRQ112FiTWikX4S72fM+B55Ubru4CKhvh6fYJvjtwBzh7uHEi/TDcP/EQUeYqR1SkjflHKHcwWJfTv6WoqdB862MuhsTJlYBwn/1g/ETh5sbXm0A7nVUvVHPsyD6r9yTu1Kj/erDQrxPLPj7ekZspzZg3SsyDn4UTC22RNgGacrdDFHQsrUjOBH3tuH0R9o4Z1X8Ymh3y+ugO1X2A9J96hWMRSlWjWZvsKB0v+tEaDsKubI5s8TfhdHOjFi5cqS3xTRSI75SsoIJj4m1dhFxdUuLH5X5FCKPuOp3LLIY4el39s4b7mRYLiViU+k7gVIe8P0eHvwwK6e72Fi+uikdZguulRXg6cQDp1pTL0dKYM4KYd+/QoiDXw7RG10RmyPV8sKgSwYR5ECD13Ld/YyZpLLdIAKANhFq88Mf15kWXEBYKu7WgqUIv+Ydm8FVN9PayHGvY/o3bP1OLnr/anwSysPpFhgtRshjT13pD4aN7issgVqh3G0mEy4HWstNA/t/nR1elGXwlQZK6oHmAcfnBZmMm0QpxFLJQZ36V2zomJTUrqVlt9u/ZagKmBvmJbLYt88FttTArhvh/k7ysC+dWQ== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:TY7P286MB7722.JPNP286.PROD.OUTLOOK.COM;PTR:;CAT:NONE;SFS:(13230040)(376014)(7416014)(366016)(1800799024)(23010399003)(10070799003)(18002099003)(22082099003)(56012099006)(10067099003)(921020);DIR:OUT;SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?NmBoHF10cs/SywYv9gOtj4Qe+F5pr4kMLdxubnVeZtpfW7w8qDA18M94U04T?= =?us-ascii?Q?mkT43GRjQ2OC9Kx6Nr/PQgdOVbVJiWRmt+zii0OVojP52p2+hcj+KdeVb7j0?= =?us-ascii?Q?TjNn9Nh05hriHpj5TdzLBCgY7cBVs7YyDiPVsn2+UswDjCgSGJed2r0r4Zue?= =?us-ascii?Q?5s/EGx7cXvumAi4SBS2PPhPs0GY7oTdOUleYyvnfC79QabWmiPHQX7jM3rpg?= =?us-ascii?Q?BR+P7eWHS4umpDnImzKQRrQYkUKLCRc7EXu3lQv2RLT3vwTVruXXmsKvaq8E?= =?us-ascii?Q?kHHj5YuxNguViRG1nJ609Hn0LQOU5P7FHnAexdCVuFuqEWw4448oOhfJ/Sq7?= =?us-ascii?Q?58LKVq6m9itFzWQjzX1Ta64uSIWjlZehiVpMUFiQbFEo5tn+T+n/J44RvfZU?= =?us-ascii?Q?0wSqa0lPET3vB1nbEe2MhwAFqkSiflJgQ6RgA/rCRLgREsfjENh9S8Fah7FR?= =?us-ascii?Q?w6xAyXaWth3zc7Od8npyzo47W2dbdSced6Qdz8m5I/vo1e42h2IhKcNTj/aF?= =?us-ascii?Q?rYjUZvLoD7Vo0S5SrS6q1MkeRfemSBA2ExqAn7ZY8IPiINIwH/IPV7/QccWg?= =?us-ascii?Q?AAvdBrB/2FpRdTumvUQh/R6QteWKnwfmsA/Ma2CzpRi9VN3+F1QCDHSLpEoR?= =?us-ascii?Q?56bmmdGL5O7+zuA55NbEslM3j5osGXHUl5mrukswTUDxWr8FV2UTRgDyVRf0?= =?us-ascii?Q?FQ8fO3qfuOW0zUzzMLveGqd9QYIPcJohUZoAmj0STc7lQeYTnZ1hROw57qVz?= =?us-ascii?Q?R71bCq+Be6gKFsZrP+8vWoVH0thZpb0vOtem+nIQy8i5yABeFSS2ZAdHfIAX?= =?us-ascii?Q?hEgL9UR/L7zDyf15sWXMfKtokZ3Bunzu6MbrO9oCZA6m64UyWGYSg/o8bTiz?= =?us-ascii?Q?cAL7qwaCbXcAPiXTs5sAWOK034MZ5SITHqrcZP0SD+OhuyVnFWEbsmUxiRTJ?= =?us-ascii?Q?0j1P7NAvJl8HTNVHO66bvc9QPpvE7mdM6ejk1CwKvkW6ny7zHS3nkvkQbSNM?= =?us-ascii?Q?suLaSV5VneglJDwFVxsDY2gBVHTOUadsJ2Idl9BnAyhypbcCOAQpF1GOMcOW?= =?us-ascii?Q?/hf7fLwJttTo/MwZoz98XiZ+Q5NqQqUf+HWzEpuUSstcIbYkjp9ay5JO8bVF?= =?us-ascii?Q?thzkJVQziB3tJeFFU4UDO5ZEH73KZeE4cgHEXcg3SqrN86kAB/MYJKr/mKiH?= =?us-ascii?Q?AnXGsvLGN3Vlu2/n2b2d8bXH66TaLx7xe6DgM3rxWNAafthiVdGLcg/7W07N?= =?us-ascii?Q?/HVB8npX/a4lnU0FifWvjvJrICCXBM9giGtQfxI3e52NwscQmXMaFxkBuuLw?= =?us-ascii?Q?Ub4GaqgmjxpOgaqiaKLlY/6G27ZPQer+pWVtLfLes8ScwmOaPxqgiXmb2toD?= =?us-ascii?Q?Yujk3jvArWdnCBUEPFGsBr+tX1we7lmnaMTtBMMKtZPcBWpwAzioFhFeUvJK?= =?us-ascii?Q?P4O+RyLYcbt91Iuze16Pz+drsMArSilJ9QVFiaZWeTjFGA0VEeQlLgxLUqLQ?= =?us-ascii?Q?3OXY8QW10vQhfss+kIb9LkEXJJnRWCiK591neQ2oEZMf4kjjAU06GsoTrAu2?= =?us-ascii?Q?b5I9VEZJddKah1D9dVg6jQBLeWwYJBkawL6Bpq+QzONd2A1VwviLPZeSMtqW?= =?us-ascii?Q?1hUEIp7nTqPN5Q3+cEBCKcTCB88u8Q7k82JtkPECy2gLeRlE1kORvzb9AVPM?= =?us-ascii?Q?YSjzWLewUcQ5OJeH+QB9+op9QpgTiBKkRlbZStg6NVvfCC0opf8aVaa8sRkt?= =?us-ascii?Q?1LGthKI30sg3z2N7G0x15ar3s4CBP0ZEIpAKU67bQ7ynBn+oZ0Qe?= X-OriginatorOrg: valinux.co.jp X-MS-Exchange-CrossTenant-Network-Message-Id: 4afcd15b-9dfc-4f7a-18d8-08def88a6a28 X-MS-Exchange-CrossTenant-AuthSource: TY7P286MB7722.JPNP286.PROD.OUTLOOK.COM X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 12 Aug 2026 15:57:30.3877 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 7a57bee8-f73d-4c5f-a4f7-d72c91c8c111 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: GryLg5IzQRHPVv7d7agnH+OctqJe2nN18YYDq5fF5x/2y+3JrTRzvfLWtdOI4vjKbmVm1JaAPXNMUuzBTzlUUg== X-MS-Exchange-Transport-CrossTenantHeadersStamped: OS7P286MB7115 Content-Type: text/plain; charset="utf-8" From: Frank Li Move dw_hdma_set_callback_result() next to the descriptor helpers so linked-list completion and reset code can use it without a forward declaration. No functional change. Signed-off-by: Frank Li Reviewed-by: Frank Li Signed-off-by: Koichiro Den --- Changes in v5: - No changes. drivers/dma/dw-edma/dw-edma-core.c | 50 +++++++++++++++--------------- 1 file changed, 25 insertions(+), 25 deletions(-) diff --git a/drivers/dma/dw-edma/dw-edma-core.c b/drivers/dma/dw-edma/dw-ed= ma-core.c index 1af262b6e881..ab8e343f0196 100644 --- a/drivers/dma/dw-edma/dw-edma-core.c +++ b/drivers/dma/dw-edma/dw-edma-core.c @@ -73,6 +73,31 @@ static void vchan_free_desc(struct virt_dma_desc *vdesc) kfree(vd2dw_edma_desc(vdesc)); } =20 +static void dw_hdma_set_callback_result(struct virt_dma_desc *vd, + enum dmaengine_tx_result result) +{ + u32 residue =3D 0; + struct dw_edma_desc *desc; + struct dmaengine_result *res; + + if (!vd->tx.callback_result) + return; + + desc =3D vd2dw_edma_desc(vd); + if (desc) { + residue =3D desc->alloc_sz; + + if (result =3D=3D DMA_TRANS_NOERROR) + residue -=3D desc->burst[desc->start_burst - 1].xfer_sz; + else if (desc->done_burst) + residue -=3D desc->burst[desc->done_burst - 1].xfer_sz; + } + + res =3D &vd->tx_result; + res->result =3D result; + res->residue =3D residue; +} + static void dw_edma_core_reset_ll(struct dw_edma_chan *chan) { u32 i; @@ -660,31 +685,6 @@ dw_edma_device_prep_interleaved_dma(struct dma_chan *d= chan, return dw_edma_device_transfer(&xfer, dw_edma_device_get_config(dchan, NU= LL)); } =20 -static void dw_hdma_set_callback_result(struct virt_dma_desc *vd, - enum dmaengine_tx_result result) -{ - u32 residue =3D 0; - struct dw_edma_desc *desc; - struct dmaengine_result *res; - - if (!vd->tx.callback_result) - return; - - desc =3D vd2dw_edma_desc(vd); - if (desc) { - residue =3D desc->alloc_sz; - - if (result =3D=3D DMA_TRANS_NOERROR) - residue -=3D desc->burst[desc->start_burst - 1].xfer_sz; - else if (desc->done_burst) - residue -=3D desc->burst[desc->done_burst - 1].xfer_sz; - } - - res =3D &vd->tx_result; - res->result =3D result; - res->residue =3D residue; -} - static void dw_edma_done_interrupt(struct dw_edma_chan *chan) { struct dw_edma_desc *desc; --=20 2.51.0 From nobody Tue Sep 29 04:14:12 2026 Received: from TY3P286CU002.outbound.protection.outlook.com (mail-japaneastazon11020119.outbound.protection.outlook.com [52.101.229.119]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id CEDB14746C0; Wed, 12 Aug 2026 15:57:38 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=52.101.229.119 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1786550262; cv=fail; b=asoHBuPaCqA424QpoYF/VAubT3dMLvQoyltLaWRzklz3tdLvmF2Kelvbipjf2MlElBO5IzRjYANclxEMF8hWuiyCQdynPi8YwT75OB+JWFjBJ9Gg/zFfFpPOSEj8SyYONRLbL7GdlM6iRjzYNWLV9/bN0XI35cOQ7/m0MqKof0A= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1786550262; c=relaxed/simple; bh=1YkUk/bcF+2K/GGi2xXofv1kdf7pQvgWh0jDeL6mQsw=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: Content-Type:MIME-Version; b=UPRYNPR/2VPUgkcZSIAKe0Impdi/N6fICRgOUeHijULOTUATuuNxyMmGxrVmsf4xWQWiSQyx/nFvHpaYBpBA9Q0nVGhYEJh33PikHKWw91i8GrYsVLZio4yWyrnnAhd+Hcth7n3RCw3ZqmBpuNPRF7UbIV1OuktbJgBqW85XX+Y= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=valinux.co.jp; spf=pass smtp.mailfrom=valinux.co.jp; dkim=pass (1024-bit key) header.d=valinux.co.jp header.i=@valinux.co.jp header.b=hOpngMSO; arc=fail smtp.client-ip=52.101.229.119 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=valinux.co.jp Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=valinux.co.jp Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=valinux.co.jp header.i=@valinux.co.jp header.b="hOpngMSO" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=FHMMkP3iuGbvI93C/LizQ6r4a8OmDEYrdIWpDf9Z65fjoDNxF6UB3h15hFGR5VCKEAh+0++g3dntb4C9DdB5FH4NM93+qDUJB8/Amb2J9pK9FUnUmbD3w+axuJsIDF86Fk+fL5gYCB58+dD8aoRa8jYE/pSinM7J8+i3VnS7BVk9fYXR85OqCg2IZlhaY5GmoD475/DQ6ulqPMJ3uVA9XdcnYf+12dI+pjWzprrgKqtUC8avY1gZ86kQUAZDdE/ohOi4CUoaNeEIZQYg1XOEsTLWW1kVd0cpbZgS9ajyO3Jp0G1nHc668A/jwJtvRHe6e5SiKinITJAoUt6/7U67jA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=mTNpU4j1MsO/6cX0QYUM2Xqu/UmBBaocCy01N70V5IE=; b=qjaWYS1y5IkiyQ3DfL9/K3Z6AlcpLJxaIAwbnqf3VkKC0FIyscjmUdq3vkNls0z/Fg+rHZg3b2gsr3h7669WXDbyU2uLub3qdjRK/uGkpFQTKJNHOR3gKWAxSMPn0wYlnaIPHuJqr6xKNCQnKMZALvvaVcmJj5N5KOnJg8Ehh/6XwRzZNfJm4gewclq2x+aBF1IxSWD5T6oeCMpZyroLoK7DtspmSXhn49cr8QgFW5VKIC0HhUcm6O7TI2ZJika3sK4TJqEY49+UU1dAsuTYsViC9ZY90Tkgn0iX0prDVdJVXaIx6AlflH7ozVXPDmKEVb8qoQ+cHp/rtBt21M2D3A== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=valinux.co.jp; dmarc=pass action=none header.from=valinux.co.jp; dkim=pass header.d=valinux.co.jp; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=valinux.co.jp; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=mTNpU4j1MsO/6cX0QYUM2Xqu/UmBBaocCy01N70V5IE=; b=hOpngMSO/W9ZnDd6tP8sU0bevrVfwhX2QGq2rr5U110rusC+S0XZys56ZhLqU+qoVrNBgh87Ma7u4D3B7hUEaUpNXbg5uYodpL4UA81JRH/RXbWt5JX9B48zzwVxdTtdIwMMPWoEY0fzfmtmZYVF64RLFQ/G/NqOUGvpRMDOHbk= Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=valinux.co.jp; Received: from TY7P286MB7722.JPNP286.PROD.OUTLOOK.COM (2603:1096:405:38f::10) by OS7P286MB7115.JPNP286.PROD.OUTLOOK.COM (2603:1096:604:42f::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.315.14; Wed, 12 Aug 2026 15:57:31 +0000 Received: from TY7P286MB7722.JPNP286.PROD.OUTLOOK.COM ([fe80::2305:327c:28ec:9b32]) by TY7P286MB7722.JPNP286.PROD.OUTLOOK.COM ([fe80::2305:327c:28ec:9b32%3]) with mapi id 15.21.0292.024; Wed, 12 Aug 2026 15:57:31 +0000 From: Koichiro Den To: Vinod Koul , Frank Li , Manivannan Sadhasivam , Gustavo Pimentel , Kees Cook , =?UTF-8?q?Krzysztof=20Wilczy=C5=84ski?= , Kishon Vijay Abraham I , Bjorn Helgaas , Christoph Hellwig , Serge Semin , Cai Huoqing , Niklas Cassel Cc: Devendra K Verma , dmaengine@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH v5 06/24] dmaengine: dw-edma: Dispatch DONE interrupts by channel request Date: Thu, 13 Aug 2026 00:57:03 +0900 Message-ID: <20260812155721.2807506-7-den@valinux.co.jp> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20260812155721.2807506-1-den@valinux.co.jp> References: <20260812155721.2807506-1-den@valinux.co.jp> Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: TYWP286CA0023.JPNP286.PROD.OUTLOOK.COM (2603:1096:400:262::10) To TY7P286MB7722.JPNP286.PROD.OUTLOOK.COM (2603:1096:405:38f::10) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: TY7P286MB7722:EE_|OS7P286MB7115:EE_ X-MS-Office365-Filtering-Correlation-Id: 9b0c95c8-368f-4aef-1dd6-08def88a6aa3 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|376014|7416014|366016|1800799024|23010399003|10070799003|18002099003|22082099003|56012099006|10067099003|921020; X-Microsoft-Antispam-Message-Info: tlAhfgehvD9NmVptaGaHP80iipUjmG/h3maz3AcwLMezoBbxokkpXyEqzsMNtpK6B2cf8RTuQXh49EL/KjhaPI+vBDMcm3vFUKCdsUgSkELYzn+sq7NEV6/EH+8QM1+N4A/stuLofBYGrCFznJHaQJ6/yYjxERTER6Pfdu3puq8R0f76xwI5+kn+pa/+pdZgv0IbK/5Aejw87uHwxHUJpa2vv8xm2s8VdmhXzB3rz54AHAP/7yKAT6gTm3vauYqnqhYR3UbT1A1qsJrRePDTXYZcZ0JLxbQwokcGNtnoYS7Dsgjcs0zlvgjwiTgjN5Lsv4rc4EwZRyEWeOrvsMwKT+wCz0w30XQeLyCnhrUSA0bQCYt9w4OnKmrfwf4Up6Rub0xsGp1Dfho2lG79Dpfja8gumtwsvNGoFqKV4hUUUc6iJxe068so51AXjDMUC7x4PAP6aDBtrWk84pRM+f+QaRVUrndKCsH1OZG2FdH5aVBzjzScn9bkWeKuhkL09lyAtqIpLKABvFdkPpBl4552OnC4/ATxFElAaZC7o9UJo/Sk+VbeN1LqloM4tP1FFoXi9V+idMLKfXWWI3vRPiToTAHPKKMTZl+80sSPfg5+4Otvrmku5muCWmD+QVuXhepvrIJKdaJt8uExIFaAyfiUhpEME4lhEday15kEgAKZa3HtktbTfbNsgW5bsyhC8wt10xOeWbbhI4Ls7P4uPF6DzA== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:TY7P286MB7722.JPNP286.PROD.OUTLOOK.COM;PTR:;CAT:NONE;SFS:(13230040)(376014)(7416014)(366016)(1800799024)(23010399003)(10070799003)(18002099003)(22082099003)(56012099006)(10067099003)(921020);DIR:OUT;SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?Bn+oinnT5V83FtPyppwFMKQD95uDmvvIdATEiButDsuPVOxYTx78Gu2WLBrd?= =?us-ascii?Q?taO1yfrY2OzNuHWxyieznKPWJz92XA+HEQOpFbI9Lh5Sfn89ZyZRmbwmY1bc?= =?us-ascii?Q?CtIsWUlbBjckFL/Bb5RKcRVEQ96LBY4F+d0mzoTpIZ7VJNQTjHLP29y4mlpB?= =?us-ascii?Q?9nSyfXv0UrJ/ywi3F4DOo1oelWEm78ss4gMRr3jLOcdeP6OQCWhgRDmquzwr?= =?us-ascii?Q?Ecds2he0YSoy6xLVp+1fdrslpFxh6Y/12KvplLhW/jTEAFe3lgdGgwNmmPXm?= =?us-ascii?Q?3uhr0jeicQlTNHI5aUdQpf0UzPBjcN0ZtLTKdOVXPdBbCufAdkff60TIKQ0v?= =?us-ascii?Q?WIGeoLKzn1i4fzUqxQ79yMsPcRINuXG/BKgBPrHs4eP8ftlQXdY0hrPFkPaQ?= =?us-ascii?Q?yK5TCsz6HJ36WwnuMriI2yE2Y4CY+9eur9Az16nZOFQfY7hQef3Br7na8iXd?= =?us-ascii?Q?DXvxrpFlT2edMyDfmVkXxMSF8bgigY8nzpig4Lk1uoyVoViGdytEH+NE+nSU?= =?us-ascii?Q?ccQq9rFXdmPb8V1J6n/2OM6sDt0eHBm09iBejLC7uZY9GKzY6yxKf1b8XStc?= =?us-ascii?Q?Tr/uaYYuSCN4OlH+VMaqcFPPcmrd2HISJo1gH/fKzBYUguXkYxjpCHhzq4Qx?= =?us-ascii?Q?39LHBu//Mw9t04sMi4KFSpAQEH92SyCZnJYsgdMN8DLnXsQgXNKFwfUjuXe0?= =?us-ascii?Q?5SGVVoxZKBUB6Ki5LcFWtrR8XbOs4ayerYaKFItw0zHIZ7v/4bNu/ayfociu?= =?us-ascii?Q?nihqgME7u/afH152PF2eF2DzleX4UXe1XLQu6AA+fU1yMKaE2rCDUY5MH5wK?= =?us-ascii?Q?+Ns6GbKgGTR2dvKVQDWNDpk7WrO+tq2kIPSLicGqJLwzs9slgyy//Pm+ku2N?= =?us-ascii?Q?K5vTayIPYCexoRN79k2Zs3IgFD+F/QlzQMEFG7Z8NVLC4BjsxtoUBCbJIiMt?= =?us-ascii?Q?mJObqbMYyD3x2E22I3KTssSiqQbiBvn0lr0lGe5YbTEv/NEdkcvwh+gM57oe?= =?us-ascii?Q?Y4NzGw4QGmivUTl8fRZ5B0GF3V7OllIgNsnOjC37O9XVlD6K7vmtaAogtccD?= =?us-ascii?Q?TOqh1dJKKbzM5AfEDNkiJ+B0EdKQ/FSgPHu8fhOWSL3dVoAYLfRjQH0ZIVO5?= =?us-ascii?Q?qcOg8zg7N5IlPiTMELeCzqvAtm2jKhmKDctt65uEVApmWN6OcbVJoK5/mUbF?= =?us-ascii?Q?qed0RXn1XBK2GwD2JDxKUvR7VMs/vFEYbu5/LVYLsmo1rP7oZHRCppquv9Uc?= =?us-ascii?Q?SDwQSJNvmJ84zI/SsrTCqQotNK9BumIzEiIjLLJtoE56O969ddhXeo1vObTj?= =?us-ascii?Q?16U3uI0mpp9vAipJd25fWRRIEZo5EuZTAjpfETSuebmPs4fo4mNdIxqJ4Hrt?= =?us-ascii?Q?3Em3H36kRnM5PRdHfUAWPpkWdOwB8tYXA2GHLAPCsLK1nf2EwSM8XNCBz77T?= =?us-ascii?Q?IuABLlfY6Z33rUOby2C1UKy99aA/k0hrv7zsBnD+qE8ZBz8j5GYbVfXG7/tx?= =?us-ascii?Q?IYkfo5mqFDOFgr8FdVYZMqw9FMPZKu57tpOB5VzB4+0rhi3pGXmSB6NNGLLM?= =?us-ascii?Q?ZdnE3QdckLrImmLJ/5YzJ+4Y8oe3auVB2Ok5d10fR1eHJKk+4P+jS3wJiX6A?= =?us-ascii?Q?485BoFRT1vhZa6UoB5l+RIYrI9bk7tcQAKyZBNxiZtg8gKh7eyc50NWP7tAO?= =?us-ascii?Q?A5qlr2mvJlEVpzlRDjUuzg0B/9ZAy2bJib/CsI6arTA6FlLL+M9jOM+8POz7?= =?us-ascii?Q?PzpdJ1eZhz7Ju9y7jy8iBZ512KcdlRpDg0YNISPlqj2nbhWgrAUk?= X-OriginatorOrg: valinux.co.jp X-MS-Exchange-CrossTenant-Network-Message-Id: 9b0c95c8-368f-4aef-1dd6-08def88a6aa3 X-MS-Exchange-CrossTenant-AuthSource: TY7P286MB7722.JPNP286.PROD.OUTLOOK.COM X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 12 Aug 2026 15:57:31.1942 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 7a57bee8-f73d-4c5f-a4f7-d72c91c8c111 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: gPhnWEchWH2J0Zy8dAUK84Ifcyg+aT7kY2CuKXh41F0NzPw9avi8l6jS7W8n2RUyfy73yPfm01uzdwbJa1Jkkw== X-MS-Exchange-Transport-CrossTenantHeadersStamped: OS7P286MB7115 Content-Type: text/plain; charset="utf-8" Prepare for a later patch that reclaims LL progress before request handling and can remove the last issued descriptor. Move the request switch outside the common descriptor check. Keep the existing per-request descriptor checks for now. That later patch removes them when STOP/PAUSE handling becomes independent of the descriptor list. No functional change. Reviewed-by: Frank Li Signed-off-by: Koichiro Den --- Changes in v5: - No changes. drivers/dma/dw-edma/dw-edma-core.c | 56 ++++++++++++++++-------------- 1 file changed, 29 insertions(+), 27 deletions(-) diff --git a/drivers/dma/dw-edma/dw-edma-core.c b/drivers/dma/dw-edma/dw-ed= ma-core.c index ab8e343f0196..3c1f3f728875 100644 --- a/drivers/dma/dw-edma/dw-edma-core.c +++ b/drivers/dma/dw-edma/dw-edma-core.c @@ -697,39 +697,41 @@ static void dw_edma_done_interrupt(struct dw_edma_cha= n *chan) return; } =20 - vd =3D vchan_next_desc(&chan->vc); - if (vd) { - switch (chan->request) { - case EDMA_REQ_NONE: - case EDMA_REQ_PAUSE: - desc =3D vd2dw_edma_desc(vd); - if (desc->start_burst >=3D desc->nburst) { - dw_hdma_set_callback_result(vd, - DMA_TRANS_NOERROR); - list_del(&vd->node); - vchan_cookie_complete(vd); - if (!chan->non_ll) - chan->ll_done =3D chan->ll_head; - } + switch (chan->request) { + case EDMA_REQ_NONE: + case EDMA_REQ_PAUSE: + vd =3D vchan_next_desc(&chan->vc); + if (!vd) + break; =20 - if (chan->request =3D=3D EDMA_REQ_PAUSE) { - chan->request =3D EDMA_REQ_NONE; - chan->status =3D EDMA_ST_PAUSE; - break; - } + desc =3D vd2dw_edma_desc(vd); + if (desc->start_burst >=3D desc->nburst) { + dw_hdma_set_callback_result(vd, DMA_TRANS_NOERROR); + list_del(&vd->node); + vchan_cookie_complete(vd); + if (!chan->non_ll) + chan->ll_done =3D chan->ll_head; + } =20 - /* Continue transferring if there are remaining chunks or issued reques= ts. - */ - chan->status =3D dw_edma_start_transfer(chan) ? EDMA_ST_BUSY : EDMA_ST_= IDLE; + if (chan->request =3D=3D EDMA_REQ_PAUSE) { + chan->request =3D EDMA_REQ_NONE; + chan->status =3D EDMA_ST_PAUSE; break; + } =20 - case EDMA_REQ_STOP: - dw_edma_finish_termination(chan); - break; + chan->status =3D dw_edma_start_transfer(chan) ? EDMA_ST_BUSY : EDMA_ST_I= DLE; + break; =20 - default: + case EDMA_REQ_STOP: + vd =3D vchan_next_desc(&chan->vc); + if (!vd) break; - } + + dw_edma_finish_termination(chan); + break; + + default: + break; } spin_unlock_irqrestore(&chan->vc.lock, flags); } --=20 2.51.0 From nobody Tue Sep 29 04:14:12 2026 Received: from TY3P286CU002.outbound.protection.outlook.com (mail-japaneastazon11020112.outbound.protection.outlook.com [52.101.229.112]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id AA1743002C8; Wed, 12 Aug 2026 15:57:39 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=52.101.229.112 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1786550263; cv=fail; b=lRG1oqvszGjXWALIhs9RlAIIWowp8GDpOuDlFnE6Oc/kvqYtsVDDWd17gVNGIMPO70xmROH3vDoGtH6SwJKNb873esS9yeKa+no8FBUpX/etB+z2+MUuAgLI/VjCufTHNk1DE72REdN/YgW5a7J/TKhhULr1rDA6fD8Quax3Wpc= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1786550263; c=relaxed/simple; bh=kPa25jWyM9CKbEUDlQ9tY/p8a4RHvT0HsiPfuiW2WD4=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: Content-Type:MIME-Version; b=F+Cv2Gl9iRyW1GjmbsyO6N50EZ5E3FHiqRFBtFvbm7WNW/EqZ5uCgv2IrbxMxKzP3dqNmB3gow10sBhmMkgAxeAIGS1WQ4USVVV5odFNtygFZd/Xsg2yrgFlWl65xwX8IBWngY8VJOGYC9rUgz26W7CrT5da2nh3y5p0Fv/l4is= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=valinux.co.jp; spf=pass smtp.mailfrom=valinux.co.jp; dkim=pass (1024-bit key) header.d=valinux.co.jp header.i=@valinux.co.jp header.b=vYlIoPZu; arc=fail smtp.client-ip=52.101.229.112 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=valinux.co.jp Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=valinux.co.jp Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=valinux.co.jp header.i=@valinux.co.jp header.b="vYlIoPZu" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=HnS71RJbbu/DyHjbcpjl3mbecEmDzNViXrpUeW7095Gps+elAE+ARkzHKN4VNJ75apVh7zNLwZ0Iv7QooiB/M+th4Q1ckjuCZtSpzGuyGd2RaSpZsCJBytHEsKL/HaVaBs1UA04b096vJaHceQ9DQ8RVbJSi6f5z8AuxtT9G/EiCpjYak+MZ5dEzuCSB+HthTswLQJyvOJd3foNmA8fqL2iaqrqNYZBqbtVwi2OluQZs6RXQeSzVT8q3s9dnE4k2ELPGSafWD5QFWNAA4jlH/LX1kkEs/eiJSiMC+casuFOw6JGoZKLB8VqBZkFkdLyzsufzt/6Ls9bv43BO0YllCw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=egqeyCBg5xIHB/5glkEUqDDyQNn4M3mET2n8VZZ+ckA=; b=Ed/vYN4gHm/Mtlw1LFy0dxHAoDh1B41ITuoJ40hjCzi8xRwMVMseif4I3Vm7YTwznj/J+JFGZJw9zleNRhueTWBTwEypRfYXrQdpCIm5NSYM4/BFFLF5wkyUZXd/RxeKunwsZnUCRm1JW88UduVmFIqmZUkeDjIfnXQ4Z4ibBXXoH6j6WVHFj3BuCUq+mYO1ZRIM3NwVaYrzY3SU2uGL4GMeje6KI8GbrvBq2mM8NSPvUjYsL0F/LNiOJeLur6AFAIcHqEpvr3Y6co+yppL/+Nzh3OVfHueR0bnKvIoNCCAFaOFgfqmxuCNXwRUJdNWetp9GEqVX1bvl1PB2wPt2dw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=valinux.co.jp; dmarc=pass action=none header.from=valinux.co.jp; dkim=pass header.d=valinux.co.jp; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=valinux.co.jp; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=egqeyCBg5xIHB/5glkEUqDDyQNn4M3mET2n8VZZ+ckA=; b=vYlIoPZuwT32bDgHZSA7v3O9pL0+Agr4FPAazfHzpBuz5Iw3xCK5wN9TSB752LtaIWu4xckeHAqQEJYYIXMFNiwhW1HMAgyo/6vq9IoFqIMAwdUqapKOcDGLvBgtQv3615T9I3fW79fF863dOGMEBCIn9J0HIrfAbMJk6H5e0nM= Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=valinux.co.jp; Received: from TY7P286MB7722.JPNP286.PROD.OUTLOOK.COM (2603:1096:405:38f::10) by OS7P286MB7115.JPNP286.PROD.OUTLOOK.COM (2603:1096:604:42f::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.315.14; Wed, 12 Aug 2026 15:57:32 +0000 Received: from TY7P286MB7722.JPNP286.PROD.OUTLOOK.COM ([fe80::2305:327c:28ec:9b32]) by TY7P286MB7722.JPNP286.PROD.OUTLOOK.COM ([fe80::2305:327c:28ec:9b32%3]) with mapi id 15.21.0292.024; Wed, 12 Aug 2026 15:57:32 +0000 From: Koichiro Den To: Vinod Koul , Frank Li , Manivannan Sadhasivam , Gustavo Pimentel , Kees Cook , =?UTF-8?q?Krzysztof=20Wilczy=C5=84ski?= , Kishon Vijay Abraham I , Bjorn Helgaas , Christoph Hellwig , Serge Semin , Cai Huoqing , Niklas Cassel Cc: Devendra K Verma , dmaengine@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH v5 07/24] dmaengine: dw-edma: Centralize LL doorbell decisions Date: Thu, 13 Aug 2026 00:57:04 +0900 Message-ID: <20260812155721.2807506-8-den@valinux.co.jp> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20260812155721.2807506-1-den@valinux.co.jp> References: <20260812155721.2807506-1-den@valinux.co.jp> Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: TYCP286CA0024.JPNP286.PROD.OUTLOOK.COM (2603:1096:400:263::14) To TY7P286MB7722.JPNP286.PROD.OUTLOOK.COM (2603:1096:405:38f::10) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: TY7P286MB7722:EE_|OS7P286MB7115:EE_ X-MS-Office365-Filtering-Correlation-Id: 9025e07c-825a-4ba0-57d8-08def88a6b16 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|376014|7416014|366016|1800799024|23010399003|10070799003|18002099003|22082099003|56012099006|10067099003|921020; X-Microsoft-Antispam-Message-Info: nWVopu2GJJdHi4EoHUptGRGP/P60NsUfTgEe15oWfxIbv/8fe+dICQncLmVLknf5w0yZnELLF1n7I0qjJE2ZMEq/GkF/TnIrK06eAOy25KyLajX+0R/yyKPpYc14YDntOKUpP+RgJUpgdx36Y+L4otP09bCEvtKKVd4gF3Oh5gwC0fNQIZwxMnKmLT72u72KRouczVvhVwS8Wf/DmlU0MTDR+6ATT9qXFhK2Eg9wi0av0g8APZ69u1061CHyABOy7V3TITh4PJqA4DLq2aigap9na9MMKUSk/ioy61AlTNXqcODMk+5Wqzie2AmbrXtVwoO5KM8MNFOF9TS/I+L8ljpJXRZJ7z8Cu0vnR7UgTbQJSXn+e3DdXxJSU8sPCjYl4VbsJwqu9WvtTVdBd9ANX4P6wxVhnFx37PHBd16fhewVHLE+p32dHbvq/LJyKDXmV3qV9AS56MCzBJUB2OMI4YbgtWxtx0sfCU+H+CNw0nu4vtrZqRj7nhazV6vLKJHzpt7pMGh+FNccwD2ByVzwUIL83n4n2tDFZPbxyejhjL2Hr9f8mXGOacBzYE2obhWu/D8oS4Pn2K9znHeYsmL1tV37RQ5QXvIaXuC3IeDqV8FU8SgZH+R7x84AvhIyn0d5Ebbz2FpKyLy9jfrIgXN4PcFrHeCYlj04ENRhG9o0UUJ/ieldcmazbNWIlczfr9rG0n0GafBUTTIiVfOc1vB4DA== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:TY7P286MB7722.JPNP286.PROD.OUTLOOK.COM;PTR:;CAT:NONE;SFS:(13230040)(376014)(7416014)(366016)(1800799024)(23010399003)(10070799003)(18002099003)(22082099003)(56012099006)(10067099003)(921020);DIR:OUT;SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?Sg4+p1eoU6K8nK7Sc2pudLPDXflyp6FvSIgQotwDwSCuJG529Ce8YGkTA5Ec?= =?us-ascii?Q?+4H/8mj7ijtbuZem46CZF78t7ufNFa9eHJ2GJ8CKGxKHiV6ujo59QjiMHz+K?= =?us-ascii?Q?L1PN/h0pcRzD2Jq5WYMx0r8QjAcGsxbeRFf2g63eeqdujUpYu9A14c/IbYWx?= =?us-ascii?Q?VwgUca33tv/YbMXOoIyXYq6KY8UV5xjWP8bVr782ZJE78dKDzNYTLhXBhycR?= =?us-ascii?Q?E7Ujs9Ycr2yzj8zZho7T8R6u5cmLRC88OW23MzEHEh11EIGZB0CWIgf1gPjF?= =?us-ascii?Q?YtCfqQbhHZcWePNcCaWrIbqZM4BFiXLMrs3ynM1/UGgyVZ14zb9jszqYRfW/?= =?us-ascii?Q?JSJo5uktE2ggx1RMK3EgwkVS3p1eFoVdYri4Sle3CNhAE0shhwKsp4jY2r2X?= =?us-ascii?Q?NxSPOTX/tyT4ubL21SMYn8+bL89orL0qxuN/Sre3eN9ggcLlb9TmqHhrZ+rz?= =?us-ascii?Q?ALpeCTJFW4W/xfCIIb67wICfGnZPNiMiGRfnYGxxgjNuNL/WTM3iWjYGr4mX?= =?us-ascii?Q?Rkwe/RUSgSf2xW4n7HP4fUoDSMYACbjrPHbkvuyfS4kwYfNweBFLTYhL2t+P?= =?us-ascii?Q?omA6vM2cqzNVwMRMsOFxgk136t9+4W5xefUs3SH1hoyfzNc1XgkBZHQLHdRJ?= =?us-ascii?Q?bMwkKAbkT/tcA1FKgRYE2d1aqaAlatfCjrBQxsVeeuv+rteIKAyMqRoopFBB?= =?us-ascii?Q?9WGj/GH7cVP1U+Tjs9cQmwiWneVd3tFE70xUMuUz8zjxBTkypBRUfIiiuJJY?= =?us-ascii?Q?ZNSmc2JY91BHxGLUxv+vb42qXp3BuBFVR1doj2uyWiT2eHO7U5qsCCioygoY?= =?us-ascii?Q?DbCTw0jbdn8ltax7Jk6HmJRU1EgEydAEIQ6BodYutl0fbtRm4nVOmXX/0Slq?= =?us-ascii?Q?vQ8QLYAl/dQLt9+9ZEs8a3BORMyWmb+i6IC7Fil4PKavMCZ94+Q/Z+tDO+DR?= =?us-ascii?Q?Odhz9W69FbBM5pFScrAFp1ODdOHDpeGwgtakVcPRKM7pWy+5Fni9xPWsaMkc?= =?us-ascii?Q?gszFWw9r00o7DIvj/vEJVK2omzv50+Ry48ZNejicn0NHkWxvfh9WLvq9DxLn?= =?us-ascii?Q?h2wDZNIzlbonxtcxdSVqM/CSMXJ7uisdaTHOVtg8CagwRsBJILB99yVKX5Ap?= =?us-ascii?Q?KRznK7CeTzVPmgFf7MO2pY2CRxuN7KRTviomvj6Tymt4gjBBOpJDdBCMi1/P?= =?us-ascii?Q?CApNjUkQCWgX9bkHPtRnxBUrw1ZDdPjmEgAFmHq7RAst+ClA95BuJ+wwdTRr?= =?us-ascii?Q?6PD4Q35a/3ZfKd3hlPEllTsLn7yzci07fOwY8IiAjaY6xG2D8AmifPVyIGt2?= =?us-ascii?Q?UP0Ho93H4J6fkLLmtj/eYNveS/OawqLBgr1ask2bgGA2PuU4xmG1qvs0bbW7?= =?us-ascii?Q?vc9OtGjVYvOTK+nRmkQejFoctAC0X2P5/Kqzxhvu/qhyByWkPem2Yn2+3lqB?= =?us-ascii?Q?ehJDKGzIrl0eClQbtDKcgGc4zWM7nnfiWjHwty7XHWW+9WCsMqRdAyueK4TY?= =?us-ascii?Q?fAaL6YTJZ7AzztV0sC9EdKjqjuBHGhh2oYMrrFzOKyUzA1QSGqJMnTP1tT5G?= =?us-ascii?Q?twF56H11b7hO2UIi7ILCdqxPjFtD41VQB5q6BvDTY7exv3McOgTjgnpKX762?= =?us-ascii?Q?50yj84mYjDW8gias+jeATdq2UoSKKH5Qw7iEQLnheA2XnucI6g51WUA0QB03?= =?us-ascii?Q?gw+w0Ff9hBsSX5gA+eYUtSx7gwVWiyZC8slAQx7CiKYL77CWrbrBW+/F+gWA?= =?us-ascii?Q?w8vlrG1FDhKhVFknduyAI4/gcaNfU1pax7UBsMn2IbxE1aMtCA6/?= X-OriginatorOrg: valinux.co.jp X-MS-Exchange-CrossTenant-Network-Message-Id: 9025e07c-825a-4ba0-57d8-08def88a6b16 X-MS-Exchange-CrossTenant-AuthSource: TY7P286MB7722.JPNP286.PROD.OUTLOOK.COM X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 12 Aug 2026 15:57:31.9532 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 7a57bee8-f73d-4c5f-a4f7-d72c91c8c111 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 0LlpxTzYgNZG/vLWV+3VmFBgM/xWi1dI3yrjrHtvvdMFiYyGOlkU9WgctKMDnABgZKBHBZ0uoC95K6otEsCDAA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: OS7P286MB7115 Content-Type: text/plain; charset="utf-8" LL fill currently rings the doorbell every time. A running eDMA or HDMA channel follows entries whose CB matches CCS without another kick. A channel stopped at a CB mismatch needs one. Move that decision to dw_edma_core_ch_maybe_doorbell(). It rings only for a BUSY LL channel with pending entries, no EDMA_REQ_STOP or EDMA_REQ_PAUSE. The BUSY check prevents EDMA_REQ_STOP completion from restarting terminated work. issue_pending() still publishes LL entries only from IDLE. A later patch allows live appends. Non-LL channels are unchanged. Reviewed-by: Frank Li Signed-off-by: Koichiro Den --- Changes in v5: - No changes. drivers/dma/dw-edma/dw-edma-core.c | 16 ++++++++++++++-- 1 file changed, 14 insertions(+), 2 deletions(-) diff --git a/drivers/dma/dw-edma/dw-edma-core.c b/drivers/dma/dw-edma/dw-ed= ma-core.c index 3c1f3f728875..5a8407ca68a2 100644 --- a/drivers/dma/dw-edma/dw-edma-core.c +++ b/drivers/dma/dw-edma/dw-edma-core.c @@ -167,8 +167,6 @@ static void dw_edma_core_ll_start(struct dw_edma_desc *= desc) =20 desc->done_burst =3D desc->start_burst; desc->start_burst =3D i; - - dw_edma_core_ch_doorbell(chan); } =20 static void dw_edma_core_start(struct dw_edma_desc *desc) @@ -249,6 +247,16 @@ static void dw_edma_finish_termination(struct dw_edma_= chan *chan) chan->status =3D EDMA_ST_IDLE; } =20 +/* Must be called with vc.lock held. */ +static void dw_edma_core_ch_maybe_doorbell(struct dw_edma_chan *chan) +{ + if (chan->non_ll || chan->request !=3D EDMA_REQ_NONE || + chan->status !=3D EDMA_ST_BUSY || !dw_edma_ll_pending(chan)) + return; + + dw_edma_core_ch_doorbell(chan); +} + static void dw_edma_device_caps(struct dma_chan *dchan, struct dma_slave_caps *caps) { @@ -374,6 +382,7 @@ static int dw_edma_device_resume(struct dma_chan *dchan) chan->status =3D EDMA_ST_BUSY; if (!dw_edma_start_transfer(chan)) chan->status =3D EDMA_ST_IDLE; + dw_edma_core_ch_maybe_doorbell(chan); } =20 return err; @@ -420,6 +429,7 @@ static void dw_edma_device_issue_pending(struct dma_cha= n *dchan) chan->status =3D=3D EDMA_ST_IDLE) { chan->status =3D EDMA_ST_BUSY; dw_edma_start_transfer(chan); + dw_edma_core_ch_maybe_doorbell(chan); } spin_unlock_irqrestore(&chan->vc.lock, flags); } @@ -733,6 +743,8 @@ static void dw_edma_done_interrupt(struct dw_edma_chan = *chan) default: break; } + dw_edma_core_ch_maybe_doorbell(chan); + spin_unlock_irqrestore(&chan->vc.lock, flags); } =20 --=20 2.51.0 From nobody Tue Sep 29 04:14:12 2026 Received: from TY3P286CU002.outbound.protection.outlook.com (mail-japaneastazon11020119.outbound.protection.outlook.com [52.101.229.119]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8D7B54756D5; Wed, 12 Aug 2026 15:57:42 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=52.101.229.119 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1786550267; cv=fail; b=gRdnXiZH7qEnhdGYpKLImKFUzwOCEZYg52M8nfKCk0N/z8FLJn1yagqtT9UKzokc9ZIMvYug/qw47Sow33L71/iLb+mlqo6VYdvXEUXYnN8mR+WkVO+O/5ToKvePZnQoxnYxedqHW8X2Kf2rARNm1oSrAwXSsWp8uRDpvM4KBUw= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1786550267; c=relaxed/simple; bh=7ppYB0zylSxr6UEHqSqDE5h4g73fH+sZWWwMhqBOs8M=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: Content-Type:MIME-Version; b=Z9JOiEZHgh5s3edwu8P6FcX+ZdlCCwcL6HqTejt5hYWtjFjOYW1Ytp/ZKvJuL8+3WPsmfPMX74ezKoWM7CLyYG//HBhv1DZRWm9n+a5/nOqhGM44xUDRN4WIq28FPNySkjcOyJcEN+oTpFmusH/k+J8oUCd3aFgEQR32EzlEwcM= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=valinux.co.jp; spf=pass smtp.mailfrom=valinux.co.jp; dkim=pass (1024-bit key) header.d=valinux.co.jp header.i=@valinux.co.jp header.b=K2U7Nkeo; arc=fail smtp.client-ip=52.101.229.119 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=valinux.co.jp Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=valinux.co.jp Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=valinux.co.jp header.i=@valinux.co.jp header.b="K2U7Nkeo" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=b6SA7UbZ7VG3U1ay4yfL8ttuHUrxLQ7m3bpYMa1Bxbg8dV9hV8PqJgqT/Tc5WwvsnobR+Jfz3zB0VxCCMZoQqHJgK7pZGbVGcKATk/MIuTeQ1aDCqG3n3Gtftlk8uPMA5dv1sUapMtLVT/y9xt6phlh/bb1pRWM1+ZG+sUxPr6Vw56om/PnORcpEHFuPI6rJOn92JSKX2coJnHK+YAlxWfLXyFIkeQNo+jl1O/N+I6a4mEZv6TQ32Y+iLABybpXleILJ3kYD32cBOsR/BQY8wZnYdP23XF/KaTX8yZ6RCUdcYm2QjnYj+tzzlkJXa6ZQjaDy96jmo/T0lJDqWMF/Hg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=0zKxay8s2mSAPuxqXbGZqjmZ8zhyayZRqwBBKCe7m7Q=; b=OaLVsiXQ3T4g9Vx5OdJilHLmfuOxJwxJGYfA9YS7eKW5IYVbSO8njSeaxSJg9M0aOvdqH44xZWfKxD1lvQmrk0yoGc851AOByZjS7wuO3w0qjtCaONBMquerhDgzLJoGvAYlg0VyE1N2nFRBBXWJW7Ck3cAG+ngx8zp4zlm4y1dbrnJ1cm6PIgGFuvfhaERXTzRiGMohRHnXqmzO1QwhKEgbPBcTnPDHmhXfjCLyExEWuD9T/vnjtp8YAMiVzrMOMln+YJ/R1iDSzaTLCI5uCSnuW4bF2ZuvgspLNeYNjcs4Q9yE9zWwbZt1oH9K2IsnrskQfwbTaYIUmKV7EeHbcg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=valinux.co.jp; dmarc=pass action=none header.from=valinux.co.jp; dkim=pass header.d=valinux.co.jp; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=valinux.co.jp; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=0zKxay8s2mSAPuxqXbGZqjmZ8zhyayZRqwBBKCe7m7Q=; b=K2U7NkeoREqayKU+iJ0uyj/mVP7tZ6R/OHwq3sRTsnKLqsxwXUdxdml+adACV2t6jPNVtx0Kpe96DFDnSkrgTKJyqEsBTG11qeoXThGhGwqLoQZ5np8rwKQwmJtWLu5ZsChiGKZtcDS5nNPs9aZRXmulAtohVtPeDM9po3x8DMM= Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=valinux.co.jp; Received: from TY7P286MB7722.JPNP286.PROD.OUTLOOK.COM (2603:1096:405:38f::10) by OS7P286MB7115.JPNP286.PROD.OUTLOOK.COM (2603:1096:604:42f::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.315.14; Wed, 12 Aug 2026 15:57:32 +0000 Received: from TY7P286MB7722.JPNP286.PROD.OUTLOOK.COM ([fe80::2305:327c:28ec:9b32]) by TY7P286MB7722.JPNP286.PROD.OUTLOOK.COM ([fe80::2305:327c:28ec:9b32%3]) with mapi id 15.21.0292.024; Wed, 12 Aug 2026 15:57:32 +0000 From: Koichiro Den To: Vinod Koul , Frank Li , Manivannan Sadhasivam , Gustavo Pimentel , Kees Cook , =?UTF-8?q?Krzysztof=20Wilczy=C5=84ski?= , Kishon Vijay Abraham I , Bjorn Helgaas , Christoph Hellwig , Serge Semin , Cai Huoqing , Niklas Cassel Cc: Devendra K Verma , dmaengine@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH v5 08/24] dmaengine: dw-edma: Prepare LL progress event handling Date: Thu, 13 Aug 2026 00:57:05 +0900 Message-ID: <20260812155721.2807506-9-den@valinux.co.jp> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20260812155721.2807506-1-den@valinux.co.jp> References: <20260812155721.2807506-1-den@valinux.co.jp> Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: TY4P301CA0041.JPNP301.PROD.OUTLOOK.COM (2603:1096:405:2be::15) To TY7P286MB7722.JPNP286.PROD.OUTLOOK.COM (2603:1096:405:38f::10) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: TY7P286MB7722:EE_|OS7P286MB7115:EE_ X-MS-Office365-Filtering-Correlation-Id: 9a1cdc37-ee3e-4f2f-4571-08def88a6b91 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|376014|7416014|366016|1800799024|23010399003|10070799003|18002099003|22082099003|56012099006|10067099003|921020; X-Microsoft-Antispam-Message-Info: kOwlJnesBY1E6nj2w0jjt6G2lujXYjl7kxLjrANzJlSm5zQGzJi0L4slM3YnfDgJQA8FwzMcrYvVKyp380kCwyxOGZUq86PxxqPQW1cgqdMtDhb7QOCST0JFmh/PrLrTWMNlLk6KqY+xE++MecHGDYODVPih0nUXKJxQpXHotTNwflh8R5gLst+VomwWk5nQadKNzyQzosz2S+0jRHGGIRCkaRBXFBAw4xTxbv6pSvXDOJ1ossljDJb3C5oZaGT3X9A9E9ICS4iOgYg1VrpBCgEsPq0Ms3jEn+BFowU48/0fOE3ShA4B0memjtLf0TCvmtgpcPfQZUQYqLHpjMfryVDGWTho6RxSOjKoqgFsgzExYVvsH8BworvVE9zrCbyUy4lmtwuRfDZ96TxkNGAsDHhM1pYp3ZTFzx7Lgk2g6EwitSmIir78Pg9J/Vn4CiUGt7X5cgwC7pMArooYptZ3Yc9Tyo7vT2LobjA5oL5z9khXJoEUAAMeta+E7Fe1FkV7TzuZmqmoj5U3ZGsLiANdyABqU9uD0vN4nNnmBlAEI9p+OvJYL5+5D3HMyuF4jkhhtfDFBLgysq3/l/aJsOrEre6y8lSQMoCFzM1e1HnPXHa2KlxqdJheNDz1PaY4m4MYtRJ91BVU6mMOmInjHkXgts8WEYDo8Lj+qUqaU3sMwnvYi4SRO8dn9eKBjDSY+sjg3ZZHdcncavJJQ02jtlebgA== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:TY7P286MB7722.JPNP286.PROD.OUTLOOK.COM;PTR:;CAT:NONE;SFS:(13230040)(376014)(7416014)(366016)(1800799024)(23010399003)(10070799003)(18002099003)(22082099003)(56012099006)(10067099003)(921020);DIR:OUT;SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?dlD6ALACwySMrA/zhlj0YFmZFjjsm3n8Kz10cXb3BwoRP5m46zF+GnEEeYqd?= =?us-ascii?Q?c04/ai9EmtbwFY2YvxKZEAXrZjkN7AMh5sQ5f7N9ZEI15KSlMuHtVRdlq91o?= =?us-ascii?Q?Mpki9YtFnHoouuDjA8KAUJMddZHsXUJl2PPLDhuRj2Q71nnSioszz3w042lz?= =?us-ascii?Q?/WHmMYS86dnmyUYFPcDMrvRnHORw99o64RNzmrW75f+RJhZLVU7TghUY8fSz?= =?us-ascii?Q?KS5u58zYPeQsC89JPhcQsJCq2QhuCgXJcjJZKRHve/7mFqP1z8wZ8AyBDwc0?= =?us-ascii?Q?A3LphqKXAk96iasn0k+q5E+EhJmFHr+hGxiF+lcKn0szN7S/Ko6tLVa3aZcp?= =?us-ascii?Q?5nAkhNGPy9PM2gVmUon0ceFnPoFRQ7fyga4qlk26fbyWIEoXX48vOYqdd7g5?= =?us-ascii?Q?CrZrUTFzsXgm6y0WIS2Xof9BpFmoaBRbBmLlArzrRLnUD27En3hA4rsz6uP3?= =?us-ascii?Q?hUdrn5VrK/u4n6DSnAExUNBsaB0IkaGu9l6ItiJzOPyF4Q8Iu1LluBA2ge+5?= =?us-ascii?Q?/0ylEhwmWNIrItun2U2xE9siBDP4WMCFPBcNBSt0uWf6aIUojApR8wBbJX8h?= =?us-ascii?Q?LCDPTzkvyZ0vjurkSGG2kkJRso/c9vsbmOZyyjBlJfV5NmVyLCNQ9iAiheoW?= =?us-ascii?Q?a8jLtu7WuB4+676IGo24Yv7FFb5z0u+SmXXKK/QhJx7Qja4N/6j9JhWRuAv2?= =?us-ascii?Q?CE6hI7vp+Z/Wlc3y38qKPPgScxugPw0WH/kA2ps+nM6iKF0iwuCQAuRm2JQN?= =?us-ascii?Q?JmcamPK//hgPXUzE66pgNVuVJsGybvt2ICmRhbRLWAkKqWHfIejxRFTdUYwY?= =?us-ascii?Q?gn4qDjdQzRoBAKSeX4Jf6+1SFaNPpyaahJxVHy6VtG0upwnMTjO2glHsmcPW?= =?us-ascii?Q?hC4P65K/jqPUnuiUz8bvHfSjh6Cvf9WsHUJAHAWP39svkNCtzb/0JbAxrVfe?= =?us-ascii?Q?x6rEV5+EZQV7zbDfQnklrhTydbUUIo3BgZPlNHahKMikeT74JFu/gdOj0gHr?= =?us-ascii?Q?+fjoWnBs4H1fjeBQWyTUtZgaKvOmq/teEq7+/PXxG4Imr/aFJXKZZcCWWic2?= =?us-ascii?Q?h+R3rrIYjyfuJk4EGas3/D9RK03sNoy5EqUIm7v8jc1pEgPBYTZydlZFxF2i?= =?us-ascii?Q?UTAFy4sar5fjk4dqqEeTS6iIqM9Q5ClILmSGV0eYOxuoUgLlXt/v/5Kpf081?= =?us-ascii?Q?7OkYsOfEk/A1u4xmZ1Nvu5RYsAe6R5IMzJnuwK6PCQI++OOlZDJwuyFwVoXO?= =?us-ascii?Q?JeZFIQaCvfEWunMq88Ri8f34Q9tJXMcl7UCfHDlPbcn+aHOQ1ijf4VwPN055?= =?us-ascii?Q?p00rLhlEUgPIqBvQT/jQ/+ij+bVuqttSQoTT5AAm4O+7NdMJhSr48ctd3POf?= =?us-ascii?Q?e9WndBnZO8sa9mIKjKIi5VYecwAmX8IuQQQ9spW4TdFjGM/f5S2kMvqafmV5?= =?us-ascii?Q?kLZVTjKt3kpe4ywfDW9OhO63nqWViwknHrCEe8xVD+Ws9m0uQZtvH4PooDX0?= =?us-ascii?Q?yvKlpgovVVO+9EHqcRCbhpWlVnQyP6vpNSOAMWjvbpQaw/BVxLPfoBvBRsCS?= =?us-ascii?Q?YOp8Fv3gDsiFHNVsrjQp4/96ay7BL36Mnybymu+eAM+zmQw48Il1VqbVrRq0?= =?us-ascii?Q?XiMDGfDelcYZ/Ow0k0CrAz33MW34sYkhH/wvvz6yzt6JQhN6f4JVHmg2gmBc?= =?us-ascii?Q?SgolyIO3O7uiU1GtCdvfj3HjRZ7sVfBl8Og6zKllyh88fySqOnFYgHUGZP9L?= =?us-ascii?Q?hOKZ1XyVNzCrkWdmQ8+hrMjQA4pijE4HWGLx3EnzXKAigSMDrTRC?= X-OriginatorOrg: valinux.co.jp X-MS-Exchange-CrossTenant-Network-Message-Id: 9a1cdc37-ee3e-4f2f-4571-08def88a6b91 X-MS-Exchange-CrossTenant-AuthSource: TY7P286MB7722.JPNP286.PROD.OUTLOOK.COM X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 12 Aug 2026 15:57:32.7543 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 7a57bee8-f73d-4c5f-a4f7-d72c91c8c111 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: EwS3K4OvCsXyuylXKK23crXWio0sZzTbfc45xdq0m5ONXFDAsoSEq5uXeuOwna6OR7Lxw/ckzZsThZswEMY+9Q== X-MS-Exchange-Transport-CrossTenantHeadersStamped: OS7P286MB7115 Content-Type: text/plain; charset="utf-8" A later patch records each observed LL event (STOP or PROGRESS) together with the LLP sample taken in the same interrupt-handler pass. It needs a provider operation to clear stale LL interrupt status before a channel restart. Add it. Centralize runtime request updates in dw_edma_set_request(). A later patch uses it to cancel a pending LL recheck when STOP or PAUSE changes the channel policy. Rename the existing workqueue event bits to distinguish them from provider IRQ events. No functional change. Reviewed-by: Frank Li Signed-off-by: Koichiro Den --- Changes in v5: - Drop the remaining 'event_scope', which should not have been present in v4. (Frank) drivers/dma/dw-edma/dw-edma-core.c | 35 ++++++++++++++++----------- drivers/dma/dw-edma/dw-edma-core.h | 6 +++++ drivers/dma/dw-edma/dw-edma-v0-core.c | 1 + drivers/dma/dw-edma/dw-hdma-v0-core.c | 1 + 4 files changed, 29 insertions(+), 14 deletions(-) diff --git a/drivers/dma/dw-edma/dw-edma-core.c b/drivers/dma/dw-edma/dw-ed= ma-core.c index 5a8407ca68a2..bbfd82b12ab5 100644 --- a/drivers/dma/dw-edma/dw-edma-core.c +++ b/drivers/dma/dw-edma/dw-edma-core.c @@ -30,9 +30,9 @@ struct dw_edma_desc *vd2dw_edma_desc(struct virt_dma_desc= *vd) return container_of(vd, struct dw_edma_desc, vd); } =20 -enum dw_edma_irq_event { - DW_EDMA_IRQ_DONE =3D BIT(0), - DW_EDMA_IRQ_ABORT =3D BIT(1), +enum dw_edma_deferred_event { + DW_EDMA_DEFERRED_DONE =3D BIT(0), + DW_EDMA_DEFERRED_ABORT =3D BIT(1), }; =20 static inline @@ -73,6 +73,13 @@ static void vchan_free_desc(struct virt_dma_desc *vdesc) kfree(vd2dw_edma_desc(vdesc)); } =20 +/* Must be called with vc.lock held. */ +static void +dw_edma_set_request(struct dw_edma_chan *chan, enum dw_edma_request reques= t) +{ + chan->request =3D request; +} + static void dw_hdma_set_callback_result(struct virt_dma_desc *vd, enum dmaengine_tx_result result) { @@ -243,7 +250,7 @@ static void dw_edma_finish_termination(struct dw_edma_c= han *chan) if (!chan->non_ll && dw_edma_ll_pending(chan)) dw_edma_core_reset_ll(chan); =20 - chan->request =3D EDMA_REQ_NONE; + dw_edma_set_request(chan, EDMA_REQ_NONE); chan->status =3D EDMA_ST_IDLE; } =20 @@ -360,7 +367,7 @@ static int dw_edma_device_pause(struct dma_chan *dchan) else if (chan->request !=3D EDMA_REQ_NONE) err =3D -EPERM; else - chan->request =3D EDMA_REQ_PAUSE; + dw_edma_set_request(chan, EDMA_REQ_PAUSE); =20 return err; } @@ -410,10 +417,10 @@ static int dw_edma_device_terminate_all(struct dma_ch= an *dchan) } else if (chan->request > EDMA_REQ_PAUSE) { err =3D -EPERM; } else { - chan->request =3D EDMA_REQ_STOP; + dw_edma_set_request(chan, EDMA_REQ_STOP); } if (chan->status =3D=3D EDMA_ST_IDLE) - chan->request =3D EDMA_REQ_NONE; + dw_edma_set_request(chan, EDMA_REQ_NONE); =20 return err; } @@ -724,7 +731,7 @@ static void dw_edma_done_interrupt(struct dw_edma_chan = *chan) } =20 if (chan->request =3D=3D EDMA_REQ_PAUSE) { - chan->request =3D EDMA_REQ_NONE; + dw_edma_set_request(chan, EDMA_REQ_NONE); chan->status =3D EDMA_ST_PAUSE; break; } @@ -764,7 +771,7 @@ static void dw_edma_abort_interrupt(struct dw_edma_chan= *chan) } if (!chan->non_ll) dw_edma_core_reset_ll(chan); - chan->request =3D EDMA_REQ_NONE; + dw_edma_set_request(chan, EDMA_REQ_NONE); chan->status =3D EDMA_ST_IDLE; spin_unlock_irqrestore(&chan->vc.lock, flags); } @@ -778,15 +785,15 @@ static void dw_edma_irq_work(struct work_struct *work) do { events =3D atomic_xchg(&chan->irq_pending, 0); =20 - if (events & DW_EDMA_IRQ_DONE) + if (events & DW_EDMA_DEFERRED_DONE) dw_edma_done_interrupt(chan); - if (events & DW_EDMA_IRQ_ABORT) + if (events & DW_EDMA_DEFERRED_ABORT) dw_edma_abort_interrupt(chan); } while (atomic_read(&chan->irq_pending)); } =20 static void dw_edma_queue_irq_work(struct dw_edma_chan *chan, - enum dw_edma_irq_event event) + enum dw_edma_deferred_event event) { atomic_or(event, &chan->irq_pending); queue_work(chan->dw->wq, &chan->irq_work); @@ -794,12 +801,12 @@ static void dw_edma_queue_irq_work(struct dw_edma_cha= n *chan, =20 static void dw_edma_done_interrupt_deferred(struct dw_edma_chan *chan) { - dw_edma_queue_irq_work(chan, DW_EDMA_IRQ_DONE); + dw_edma_queue_irq_work(chan, DW_EDMA_DEFERRED_DONE); } =20 static void dw_edma_abort_interrupt_deferred(struct dw_edma_chan *chan) { - dw_edma_queue_irq_work(chan, DW_EDMA_IRQ_ABORT); + dw_edma_queue_irq_work(chan, DW_EDMA_DEFERRED_ABORT); } =20 static void dw_edma_emul_irq_ack(struct irq_data *d) diff --git a/drivers/dma/dw-edma/dw-edma-core.h b/drivers/dma/dw-edma/dw-ed= ma-core.h index 761a5ab4bbb5..8d993e8cccbb 100644 --- a/drivers/dma/dw-edma/dw-edma-core.h +++ b/drivers/dma/dw-edma/dw-edma-core.h @@ -163,6 +163,7 @@ struct dw_edma_core_ops { void (*ll_link)(struct dw_edma_chan *chan, u32 idx, bool cb, u64 addr); void (*ll_clear)(struct dw_edma_chan *chan, u32 idx); int (*ll_cur_idx)(struct dw_edma_chan *chan); + void (*ll_irq_clear)(struct dw_edma_chan *chan); void (*ch_doorbell)(struct dw_edma_chan *chan); void (*ch_enable)(struct dw_edma_chan *chan); void (*ch_config)(struct dw_edma_chan *chan); @@ -284,6 +285,11 @@ static inline void dw_edma_core_ll_clear(struct dw_edm= a_chan *chan, u32 idx) chan->dw->core->ll_clear(chan, idx); } =20 +static inline void dw_edma_core_ll_irq_clear(struct dw_edma_chan *chan) +{ + chan->dw->core->ll_irq_clear(chan); +} + static inline void dw_edma_core_ch_doorbell(struct dw_edma_chan *chan) { chan->dw->core->ch_doorbell(chan); diff --git a/drivers/dma/dw-edma/dw-edma-v0-core.c b/drivers/dma/dw-edma/dw= -edma-v0-core.c index bb86d2f60ab5..9053354eda67 100644 --- a/drivers/dma/dw-edma/dw-edma-v0-core.c +++ b/drivers/dma/dw-edma/dw-edma-v0-core.c @@ -686,6 +686,7 @@ static const struct dw_edma_core_ops dw_edma_v0_core = =3D { .ll_link =3D dw_edma_v0_core_ll_link, .ll_clear =3D dw_edma_v0_core_ll_clear, .ll_cur_idx =3D dw_edma_v0_core_ll_cur_idx, + .ll_irq_clear =3D dw_edma_v0_core_clear_done_int, .ch_doorbell =3D dw_edma_v0_core_ch_doorbell, .ch_enable =3D dw_edma_v0_core_ch_enable, .ch_config =3D dw_edma_v0_core_ch_config, diff --git a/drivers/dma/dw-edma/dw-hdma-v0-core.c b/drivers/dma/dw-edma/dw= -hdma-v0-core.c index 47d7a01658eb..4acbace43519 100644 --- a/drivers/dma/dw-edma/dw-hdma-v0-core.c +++ b/drivers/dma/dw-edma/dw-hdma-v0-core.c @@ -442,6 +442,7 @@ static const struct dw_edma_core_ops dw_hdma_v0_core = =3D { .ll_link =3D dw_hdma_v0_core_ll_link, .ll_clear =3D dw_hdma_v0_core_ll_clear, .ll_cur_idx =3D dw_hdma_v0_core_ll_cur_idx, + .ll_irq_clear =3D dw_hdma_v0_core_clear_done_int, .ch_doorbell =3D dw_hdma_v0_core_ch_doorbell, .ch_enable =3D dw_hdma_v0_core_ch_enable, .ch_config =3D dw_hdma_v0_core_ch_config, --=20 2.51.0 From nobody Tue Sep 29 04:14:12 2026 Received: from TY3P286CU002.outbound.protection.outlook.com (mail-japaneastazon11020112.outbound.protection.outlook.com [52.101.229.112]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0CE46476689; Wed, 12 Aug 2026 15:57:43 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=52.101.229.112 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1786550267; cv=fail; b=CShIcvR9vAre9eReJvoqLmGmHXRhkY2eF2yGzyZiqf89Q3z66K8+1csjE2whfrZ9InrlCaQUDT5siEwbg9x0lPJySgc/5Qu7iVzScjytLZbae1vhJQfCTELfQSmOc3seiQNuKpusmBV+Pz+QaHmfxaokA5hAde7N5hZIpe2ZooM= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1786550267; c=relaxed/simple; bh=cxLxcpp+oWxzVCWLTxIP5dAViK0QwHY0ItRy+fHSF6I=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: Content-Type:MIME-Version; b=LTbo837C9XZfkB5D3+7sAuWwiu/qkVUOAlZuUFr+WNFyu5sibWnuSq1FMDumnH6ioVLjAWbFdPzjx1muj7MkL+E6Mt/DP+aoLC+ly5TfcNU2j81V3yIiOTsgK7qRGHTmkOIwmsY12In+E3hqsV1nkrTKqtjfmIuB8m/M+vYLNx0= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=valinux.co.jp; spf=pass smtp.mailfrom=valinux.co.jp; dkim=pass (1024-bit key) header.d=valinux.co.jp header.i=@valinux.co.jp header.b=ZJvZoiP4; arc=fail smtp.client-ip=52.101.229.112 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=valinux.co.jp Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=valinux.co.jp Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=valinux.co.jp header.i=@valinux.co.jp header.b="ZJvZoiP4" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=fWrPw4/I+CvsAqRrBah+H9l6O2n5AzNdxC91l6RnaeAD/dhTxNsPD80dijgOg59+wm8SJ5wkiQQHUTEWN5HJH21TMTOl+wY3vg4g0JeCe7bWuIyFfj8OyC2xBRBeGyV9oqfSAXFDMng7Xc1e1FkJF8qHIslGJC7+33wsj7RKrlxIF59mIpezzPP8TTfqiO89YNHy4w/0y6kgTvo+pPVYYWXv76Aga2mBbM+g7Ess4cpYFcHeVCmO0ovVdxyoeb0TNsVGazLpXqo3WZdVMAly/JSwxAxihTxLZAyCVdmWamOj+PRNz217xHKlgaGYDDR0nG/06MDM0rQfhHttdYWPMA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=4ZKeOlJRBrTtFydbGaizfuEBVRhbMOqeIru2GMFveWI=; b=vmg3J5lDrbcRe4IiQETAoY8n38+/ap3cuZHBxW8LU5dbp+YY+V60yYv7FIHlu2MbWu/RSnSUTvbkew2utUlHXGhX5uNIJjdxtI1WpwXRqrxtvTlUl/o/XgafQDRIVbu4YQSLdFQL7Pv9hZFKy3Qn0UK19GNFqLDgqg6doT4PbaZp74lYdj0Xexp7ZNsHy3a1LVhi5sw9oORJEeMxCLxmnBYS9geiNLDl3ETGeI+tvRN1jNYrfOMpPhYy9neXuux6BG4ovU9ZzDj6WaCJpNL51Dl9tv1uytAcRjfl6BirnIMwn+99TXCwzUt92GQn0AGCc3ROjs6BBNJtVlzujRYwNw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=valinux.co.jp; dmarc=pass action=none header.from=valinux.co.jp; dkim=pass header.d=valinux.co.jp; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=valinux.co.jp; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=4ZKeOlJRBrTtFydbGaizfuEBVRhbMOqeIru2GMFveWI=; b=ZJvZoiP42Nik9vGNijki0nUqcTNka4BDkA1B0yC0TwPk6x2dU6hGj8JEFfJ/LG0tDldduFbNKi1IPPv3dXneSAoUIN2vvd+tojYpEmnN5u2aFFHZHD84EYeyPezDQuc7EF77M/VoWG2QUZSNIBqm5lCVA+PuVfLaRekGmQ0Ra1w= Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=valinux.co.jp; Received: from TY7P286MB7722.JPNP286.PROD.OUTLOOK.COM (2603:1096:405:38f::10) by OS7P286MB7115.JPNP286.PROD.OUTLOOK.COM (2603:1096:604:42f::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.315.14; Wed, 12 Aug 2026 15:57:33 +0000 Received: from TY7P286MB7722.JPNP286.PROD.OUTLOOK.COM ([fe80::2305:327c:28ec:9b32]) by TY7P286MB7722.JPNP286.PROD.OUTLOOK.COM ([fe80::2305:327c:28ec:9b32%3]) with mapi id 15.21.0292.024; Wed, 12 Aug 2026 15:57:33 +0000 From: Koichiro Den To: Vinod Koul , Frank Li , Manivannan Sadhasivam , Gustavo Pimentel , Kees Cook , =?UTF-8?q?Krzysztof=20Wilczy=C5=84ski?= , Kishon Vijay Abraham I , Bjorn Helgaas , Christoph Hellwig , Serge Semin , Cai Huoqing , Niklas Cassel Cc: Devendra K Verma , dmaengine@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH v5 09/24] dmaengine: dw-edma: Prepare deferred IRQ reporting for LL events Date: Thu, 13 Aug 2026 00:57:06 +0900 Message-ID: <20260812155721.2807506-10-den@valinux.co.jp> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20260812155721.2807506-1-den@valinux.co.jp> References: <20260812155721.2807506-1-den@valinux.co.jp> Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: TYCP286CA0238.JPNP286.PROD.OUTLOOK.COM (2603:1096:400:3c7::11) To TY7P286MB7722.JPNP286.PROD.OUTLOOK.COM (2603:1096:405:38f::10) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: TY7P286MB7722:EE_|OS7P286MB7115:EE_ X-MS-Office365-Filtering-Correlation-Id: 6d377e71-dabc-44eb-4071-08def88a6c02 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|376014|7416014|366016|1800799024|23010399003|10070799003|18002099003|22082099003|56012099006|10067099003|921020; X-Microsoft-Antispam-Message-Info: qKTnuHMe1uV1U4Hvn6kQ11k7bmitmYX02Cp5HbnIJFiNOvv+Og8dfpJmo3rYNI6qza48aJ8by4/mUJbDSDrfuZVhFkjcig4PHSJnTG7aHhaveZAHfqRUwMClH3ON4DWCHd5eHGJcWaDEZeKERw92mLls6R9GjHYL1BOaMta//RpZ9jH7Gn/X8YFYHA7tFZO4eYzN3No493b2aUs5j0Yx7ObhFXkVnFYIV4GNOAIxXoT+9CovtFnEtoEZMmeLjmEe3LgHTs2AgCptLqaeqyG33JODDPWKNNxblRoNbAqw1a9kxaiVqAamOQK+yjR60k3vxgPJQbxlptci0JqJWZFOSivx/9v2O4751iOEpfamL49O7t7ff5eCA9gxqou39Sr4Ob6VP/xTJkhjFdipre4XBwQ0lftfNVe8WU1i4x7eqr4k14RhkAxUVrZhu8NCMz6c/hb7uSe9nSrSDhEPVHfL0F6u6VA/QTxujFCRx5KUvd3ihVt31Ajbof0Kq1Pddzi4xwpv4xVGiqZqS47kFZlebZ2oE1/nig4JeJEgIAYG026k4oK9XvL87nBsWgZ7fyxr/XEGaoopXRSZGzK7Pdy1Kh9b045UInZgsVyYUM7GgNvVjZGcY+3Q3qM5h6XRyEcuCHBVXL6Cw4Gaz+VOH6VsblevtZ4ozEzEvj7a+8SjuedNn62tC/Tum2oLv/ZUd1TiJhKlH0r1lp9Qb6m5tNPn3A== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:TY7P286MB7722.JPNP286.PROD.OUTLOOK.COM;PTR:;CAT:NONE;SFS:(13230040)(376014)(7416014)(366016)(1800799024)(23010399003)(10070799003)(18002099003)(22082099003)(56012099006)(10067099003)(921020);DIR:OUT;SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?CXVQmodM6WYpcB9m43T0cMEms/WznYtNUS7iSsnu6DSRM05nZ8RL55SF3jxU?= =?us-ascii?Q?HPxcH0P335DjmqRRElL2ep5isGxeFOLrte2eoljC55RxzhPhG/VHW2ZJAHlD?= =?us-ascii?Q?PUZI/macFdX7bBhVKZaW5LgplSwFc0GDGxl7cYQFimXuTuqPAHdfE4W6xKbN?= =?us-ascii?Q?SmcFhYk1gD+ZZRE/fRkTPIXaBW03/Um0/NhxXghACfZas/I6q7hnSDnL2XG/?= =?us-ascii?Q?3qGIaPFWZT3HhwMz/Kwd/8isco3hZqqr4TrNY2QoFr5RnDCDgSQAzk4p4g3O?= =?us-ascii?Q?UZ8N6+WPQ0D/Gl6C7KdvKo0Dbz1o4+kxGDfXMuMs3icodZ7OvJCt21oFC8Sg?= =?us-ascii?Q?Bx6eUuQ+sfPuKmYPTLlNphmpz0mqdbl7gi0aqxkMaYW5naEXCw5RIqZ3Ad2E?= =?us-ascii?Q?cesPLrr5oEedPaDFWcrrVLFMhqHkiGYwjQSYolkEtgqWw4J8E+a2HJ7PF6K2?= =?us-ascii?Q?PT49ST4nNCjuxQuqOsgw4eyboVFLdKfpEbKHusumNBvq/kBOr7tvmkUkGtBs?= =?us-ascii?Q?pVKYbvCTqmoM6rrgKYdmi/TDQDGygvCOOfE6tKpdALZl3/9SbNYlSuch2Jdc?= =?us-ascii?Q?ioFmYn4mvrUCEguqWqOXK2KiW26HzI9d9yVy0pwHp77bmoq+PaJvShwySRhN?= =?us-ascii?Q?QE37qmJVUZ9+VO9GOaVlLlwN9wPLsIkbdV5XSEfGd/vjd4jCdzEiIrFnBGkN?= =?us-ascii?Q?CKyt2J443+V3s1lEdicJjYDqpOrngclg7s7Yi+IRMqbi6DSPiKwj/tM7lrZf?= =?us-ascii?Q?Rs4Y3/B6qDLd3EzkfLLGstrSgNQmErM775Ait18ZLxS8RmamLRT7k8MpjLAj?= =?us-ascii?Q?hXRxauNR6cJFDO7g8Fbgv47hDBNPU5RbWrZcgthFODXE/CwxCyu0q8aQzF+Z?= =?us-ascii?Q?vGgOFgs+gWpMkId1P9EK3kz4g9ZadPa0e7WKaL00FJALy0TUIAnyyviSNIrf?= =?us-ascii?Q?eb5D0dpcquT2X37GCW3lx60Lzyday0j0GWTll+vKdEYKmz2roJVZyTziDIK6?= =?us-ascii?Q?Mf/24j1jLrHcnYoDzSrbEEk/mvgBrjgLe9XFiPma2YHzUvBG/RK40Chpehyk?= =?us-ascii?Q?m2gmwb6obKxLCisi//N4dV5yyQpsLNrAMh/wbJ2WQ055z6QydKBIZc2LL1IC?= =?us-ascii?Q?D27XSqIxhKlIINuiVBVZpJ1emJpkjmEPE74fTx93s/pmz3bFGmTjn9jDrCSE?= =?us-ascii?Q?ctJlPXEyS6DpRsWUWhixmvrtysJETGpeVvV+OF5MakIjAGvIY4SRKbKAdnqr?= =?us-ascii?Q?DP49LeCadEUhSvNILNv3C3HhWYJxTTzBH8efJu8XSlH0paSF0VyMwBABHnD1?= =?us-ascii?Q?2Id2ah5SExw1UKdVVhqQqVs3EOMyrf0TH5ObVprX1LHbK3tKfLexAjoq56m8?= =?us-ascii?Q?UM95t5QUICk/k7Zsshr8/qHydQQokGPnQkzMx+qgR3jIUwTSM/pmrOu8+Zrk?= =?us-ascii?Q?2dl5cmW/DX0XeyVA5uAB//behJdek0PPJXE9I2RaL+4gPeUXkua66fDrzHBV?= =?us-ascii?Q?nhuYxNngbIIul0E7p2ZwWCQanpn9EufvtpUdNd15VP63Z5kOLwn/hFwtnw2f?= =?us-ascii?Q?FsmZdptyJ+F6wMgqLHgPcQE4pBnf/vjEEwop/a0oed2xK1H513HCd2iXICC2?= =?us-ascii?Q?jT4Alcs6x5ZfjmbKdjBCP2BnhBpWyAvT75Ox3H726/MaDXjR52ki5LtWLbG9?= =?us-ascii?Q?wtolXh4vds/HTJhIrqb0csNpAVmjlnKh+uCqJR0+qiwCQlJ6WF13OQAXHBfS?= =?us-ascii?Q?d3ODRiGkZj3U6DjC0Dgu9XKHmKmPgeJVWiGJPPl2FgFyxNPs9MMA?= X-OriginatorOrg: valinux.co.jp X-MS-Exchange-CrossTenant-Network-Message-Id: 6d377e71-dabc-44eb-4071-08def88a6c02 X-MS-Exchange-CrossTenant-AuthSource: TY7P286MB7722.JPNP286.PROD.OUTLOOK.COM X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 12 Aug 2026 15:57:33.4959 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 7a57bee8-f73d-4c5f-a4f7-d72c91c8c111 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 77SjfmUAZm/f7l7VJ/Gn3OSJLUZqkrdcDdVXY7HeYn0jRykW4EHnk5qdmCJWGA15oVlRsY8HQBMQMKqMwUuqgw== X-MS-Exchange-Transport-CrossTenantHeadersStamped: OS7P286MB7115 Content-Type: text/plain; charset="utf-8" A later change serializes status capture, interrupt clear, and event recording against channel restart. It must also publish all causes observed for a channel in that critical section through one callback. Otherwise, the deferred worker can consume one cause before another from the same status pass is recorded. Replace the DONE and ABORT callback pair with one callback carrying an event bitmap. For now, map the bitmap back to the existing deferred DONE and ABORT work, so behavior remains unchanged. Also split DONE handling into a lock-taking wrapper and a vc.lock-held body. The later LL event path needs to validate its recorded event under vc.lock before running the existing DONE state machine. No functional change. Reviewed-by: Frank Li Signed-off-by: Koichiro Den --- Changes in v5: - No changes. drivers/dma/dw-edma/dw-edma-core.c | 44 ++++++++++++++++----------- drivers/dma/dw-edma/dw-edma-core.h | 16 +++++++--- drivers/dma/dw-edma/dw-edma-v0-core.c | 6 ++-- drivers/dma/dw-edma/dw-hdma-v0-core.c | 6 ++-- 4 files changed, 44 insertions(+), 28 deletions(-) diff --git a/drivers/dma/dw-edma/dw-edma-core.c b/drivers/dma/dw-edma/dw-ed= ma-core.c index bbfd82b12ab5..97de0ad4489a 100644 --- a/drivers/dma/dw-edma/dw-edma-core.c +++ b/drivers/dma/dw-edma/dw-edma-core.c @@ -702,17 +702,16 @@ dw_edma_device_prep_interleaved_dma(struct dma_chan *= dchan, return dw_edma_device_transfer(&xfer, dw_edma_device_get_config(dchan, NU= LL)); } =20 -static void dw_edma_done_interrupt(struct dw_edma_chan *chan) +/* Must be called with vc.lock held. */ +static void dw_edma_done_interrupt_locked(struct dw_edma_chan *chan) { struct dw_edma_desc *desc; struct virt_dma_desc *vd; - unsigned long flags; =20 - spin_lock_irqsave(&chan->vc.lock, flags); - if (chan->status =3D=3D EDMA_ST_PAUSE) { - spin_unlock_irqrestore(&chan->vc.lock, flags); + lockdep_assert_held(&chan->vc.lock); + + if (chan->status =3D=3D EDMA_ST_PAUSE) return; - } =20 switch (chan->request) { case EDMA_REQ_NONE: @@ -751,7 +750,14 @@ static void dw_edma_done_interrupt(struct dw_edma_chan= *chan) break; } dw_edma_core_ch_maybe_doorbell(chan); +} =20 +static void dw_edma_done_interrupt(struct dw_edma_chan *chan) +{ + unsigned long flags; + + spin_lock_irqsave(&chan->vc.lock, flags); + dw_edma_done_interrupt_locked(chan); spin_unlock_irqrestore(&chan->vc.lock, flags); } =20 @@ -793,20 +799,24 @@ static void dw_edma_irq_work(struct work_struct *work) } =20 static void dw_edma_queue_irq_work(struct dw_edma_chan *chan, - enum dw_edma_deferred_event event) + unsigned int events) { - atomic_or(event, &chan->irq_pending); + atomic_or(events, &chan->irq_pending); queue_work(chan->dw->wq, &chan->irq_work); } =20 -static void dw_edma_done_interrupt_deferred(struct dw_edma_chan *chan) +static void dw_edma_record_irq(struct dw_edma_chan *chan, unsigned int eve= nts) { - dw_edma_queue_irq_work(chan, DW_EDMA_DEFERRED_DONE); -} + unsigned int pending =3D 0; =20 -static void dw_edma_abort_interrupt_deferred(struct dw_edma_chan *chan) -{ - dw_edma_queue_irq_work(chan, DW_EDMA_DEFERRED_ABORT); + if (events & (DW_EDMA_IRQ_DONE | DW_EDMA_IRQ_PROGRESS | + DW_EDMA_IRQ_STOP)) + pending |=3D DW_EDMA_DEFERRED_DONE; + if (events & DW_EDMA_IRQ_ABORT) + pending |=3D DW_EDMA_DEFERRED_ABORT; + + if (pending) + dw_edma_queue_irq_work(chan, pending); } =20 static void dw_edma_emul_irq_ack(struct irq_data *d) @@ -907,8 +917,7 @@ static inline irqreturn_t dw_edma_interrupt_write_inner= (int irq, void *data) struct dw_edma_irq *dw_irq =3D data; =20 return dw_edma_core_handle_int(dw_irq, EDMA_DIR_WRITE, - dw_edma_done_interrupt_deferred, - dw_edma_abort_interrupt_deferred); + dw_edma_record_irq); } =20 static inline irqreturn_t dw_edma_interrupt_read_inner(int irq, void *data) @@ -916,8 +925,7 @@ static inline irqreturn_t dw_edma_interrupt_read_inner(= int irq, void *data) struct dw_edma_irq *dw_irq =3D data; =20 return dw_edma_core_handle_int(dw_irq, EDMA_DIR_READ, - dw_edma_done_interrupt_deferred, - dw_edma_abort_interrupt_deferred); + dw_edma_record_irq); } =20 static inline irqreturn_t dw_edma_interrupt_write(int irq, void *data) diff --git a/drivers/dma/dw-edma/dw-edma-core.h b/drivers/dma/dw-edma/dw-ed= ma-core.h index 8d993e8cccbb..30b508eb3e3d 100644 --- a/drivers/dma/dw-edma/dw-edma-core.h +++ b/drivers/dma/dw-edma/dw-edma-core.h @@ -41,6 +41,13 @@ enum dw_edma_xfer_type { EDMA_XFER_INTERLEAVED }; =20 +enum dw_edma_irq_event { + DW_EDMA_IRQ_DONE =3D BIT(0), + DW_EDMA_IRQ_PROGRESS =3D BIT(1), + DW_EDMA_IRQ_STOP =3D BIT(2), + DW_EDMA_IRQ_ABORT =3D BIT(3), +}; + struct dw_edma_chan; struct dw_edma_chunk; =20 @@ -147,7 +154,8 @@ struct dw_edma { const struct dw_edma_core_ops *core; }; =20 -typedef void (*dw_edma_handler_t)(struct dw_edma_chan *); +typedef void (*dw_edma_handler_t)(struct dw_edma_chan *chan, + unsigned int events); =20 struct dw_edma_core_ops { void (*off)(struct dw_edma *dw); @@ -156,7 +164,7 @@ struct dw_edma_core_ops { u16 (*ch_count)(struct dw_edma *dw, enum dw_edma_dir dir); enum dma_status (*ch_status)(struct dw_edma_chan *chan); irqreturn_t (*handle_int)(struct dw_edma_irq *dw_irq, enum dw_edma_dir di= r, - dw_edma_handler_t done, dw_edma_handler_t abort); + dw_edma_handler_t handler); void (*non_ll_start)(struct dw_edma_chan *chan, struct dw_edma_burst *chi= ld); void (*ll_data)(struct dw_edma_chan *chan, struct dw_edma_burst *burst, u32 idx, bool cb, bool irq); @@ -256,9 +264,9 @@ enum dma_status dw_edma_core_ch_status(struct dw_edma_c= han *chan) =20 static inline irqreturn_t dw_edma_core_handle_int(struct dw_edma_irq *dw_irq, enum dw_edma_dir dir, - dw_edma_handler_t done, dw_edma_handler_t abort) + dw_edma_handler_t handler) { - return dw_irq->dw->core->handle_int(dw_irq, dir, done, abort); + return dw_irq->dw->core->handle_int(dw_irq, dir, handler); } =20 static inline diff --git a/drivers/dma/dw-edma/dw-edma-v0-core.c b/drivers/dma/dw-edma/dw= -edma-v0-core.c index 9053354eda67..4a1bb128fa00 100644 --- a/drivers/dma/dw-edma/dw-edma-v0-core.c +++ b/drivers/dma/dw-edma/dw-edma-v0-core.c @@ -336,7 +336,7 @@ static void dw_edma_v0_core_clear_abort_int(struct dw_e= dma_chan *chan) =20 static irqreturn_t dw_edma_v0_core_handle_int(struct dw_edma_irq *dw_irq, enum dw_edma_dir di= r, - dw_edma_handler_t done, dw_edma_handler_t abort) + dw_edma_handler_t handler) { struct dw_edma *dw =3D dw_irq->dw; unsigned long total, pos, val; @@ -375,7 +375,7 @@ dw_edma_v0_core_handle_int(struct dw_edma_irq *dw_irq, = enum dw_edma_dir dir, continue; =20 dw_edma_v0_core_clear_done_int(chan); - done(chan); + handler(chan, DW_EDMA_IRQ_DONE); =20 ret =3D IRQ_HANDLED; } @@ -389,7 +389,7 @@ dw_edma_v0_core_handle_int(struct dw_edma_irq *dw_irq, = enum dw_edma_dir dir, continue; =20 dw_edma_v0_core_clear_abort_int(chan); - abort(chan); + handler(chan, DW_EDMA_IRQ_ABORT); =20 ret =3D IRQ_HANDLED; } diff --git a/drivers/dma/dw-edma/dw-hdma-v0-core.c b/drivers/dma/dw-edma/dw= -hdma-v0-core.c index 4acbace43519..c6d7f9885aa8 100644 --- a/drivers/dma/dw-edma/dw-hdma-v0-core.c +++ b/drivers/dma/dw-edma/dw-hdma-v0-core.c @@ -174,7 +174,7 @@ static u32 dw_hdma_v0_core_status_int(struct dw_edma_ch= an *chan) =20 static irqreturn_t dw_hdma_v0_core_handle_int(struct dw_edma_irq *dw_irq, enum dw_edma_dir di= r, - dw_edma_handler_t done, dw_edma_handler_t abort) + dw_edma_handler_t handler) { struct dw_edma *dw =3D dw_irq->dw; unsigned long total, pos, val; @@ -200,14 +200,14 @@ dw_hdma_v0_core_handle_int(struct dw_edma_irq *dw_irq= , enum dw_edma_dir dir, val =3D dw_hdma_v0_core_status_int(chan); if (FIELD_GET(HDMA_V0_STOP_INT_MASK, val)) { dw_hdma_v0_core_clear_done_int(chan); - done(chan); + handler(chan, DW_EDMA_IRQ_STOP); =20 ret =3D IRQ_HANDLED; } =20 if (FIELD_GET(HDMA_V0_ABORT_INT_MASK, val)) { dw_hdma_v0_core_clear_abort_int(chan); - abort(chan); + handler(chan, DW_EDMA_IRQ_ABORT); =20 ret =3D IRQ_HANDLED; } --=20 2.51.0 From nobody Tue Sep 29 04:14:12 2026 Received: from TY3P286CU002.outbound.protection.outlook.com (mail-japaneastazon11020119.outbound.protection.outlook.com [52.101.229.119]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 146C8470425; Wed, 12 Aug 2026 15:57:47 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=52.101.229.119 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1786550271; cv=fail; b=M4QEOGNZmV/H/yUKB7VxrhCojd0qsqqj0PEs1YLbYKK9BQENCb4bLz0/DTUIKu7OtzTDHZslblOW0sOAAzhqTggNOJ7viV3ydVuxfVmNX/tC0gJL4UrperE81GFRsMh15RTwlMEwgQFzE1XAjLWAm3MpcYvzRG9vTFvZ6AV3OM4= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1786550271; c=relaxed/simple; bh=VKao61kVHOe4DX7U6v7OBspUx6ca4AL/foHEUBq9Ufw=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: Content-Type:MIME-Version; b=ZLF5fZI6xd9HW7wewrgVqgKUeJ5oTq4iS/BG9gaHynVi7BDkCYLZiKin4zT5cUAv3S3QrtMLpeaE6jvVOi1jr6xfTmUUyKsn05unJ2l21UveqBG6zknGgKf3wkKjoJ3rw/68dNW+gRYBgvIjsEIIp/KlS85K0zEJnuOeqgYbsT8= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=valinux.co.jp; spf=pass smtp.mailfrom=valinux.co.jp; dkim=pass (1024-bit key) header.d=valinux.co.jp header.i=@valinux.co.jp header.b=JairWKpS; arc=fail smtp.client-ip=52.101.229.119 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=valinux.co.jp Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=valinux.co.jp Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=valinux.co.jp header.i=@valinux.co.jp header.b="JairWKpS" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=pzZTZEQKkmXSPUst4OKWgLsiEQSjGU3hKzPC6lnVC+2KbHP9RZuer/cvRQtoLqtfCg/v/SPqgegBZQu9KxYVLADVwgO8slmBkEgnA2OPZNsf8eQGw5Km9ABHYb/Urq3dIxb54jrxQ/W81OLySZFt7Q65cUYByoXjhwmk3rPSvzIGe46R93CUAoD0zBCk8SdYqzZeb9N1RQZfg/Fh4vTGgiD/HVP4txhI6tpbBvK/XVU++uQyjy4TZNb8syN7USbX87oOSOwQb9SJuQJ9PJNRIVY8M34lpgi9Wo7IzoG4WxqeJsIWFHfr1Lx+HtMNE+lqr9n3e2XbReQsP9U4fQazNw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=Z4UdBxoqEVQxhsj4zqHh7z4Dph1UF1unDV1zV2E2g/Q=; b=PES/hWtUMbY/S4Tck5iyRRBsqsbKZHKfpw7MT8X+E2FiYX9Gcj5r3C/hdH8EWGiVO4kjFxlLBwGrLUnpVvy+tbNuMYp5hXJU/j3l6eBjwlgeiRAoWuOwR4OHcWSVyO1YmG5vgCeFbSGL5azSBQrscYx0G9qgRwsJKewNDpbi79E5ZXPvy5ytnCRAuNnfi6vQ4JkquA9yNJ9kesUmo/Pk3LGyP/eMcwJ/SRRpvGriou9D4mc5Tf33d6e+HyurJmBoWh1sDtvstdUy3Wj+no0n4lL8Algr5gksYYn48jOu1PhCzy1iWsUnUxAmXU5pvSgqmZL9xxvrtCeToR9/38fxVg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=valinux.co.jp; dmarc=pass action=none header.from=valinux.co.jp; dkim=pass header.d=valinux.co.jp; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=valinux.co.jp; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Z4UdBxoqEVQxhsj4zqHh7z4Dph1UF1unDV1zV2E2g/Q=; b=JairWKpSpW2tf84CjrSatHZIu0lbxNGI5I4spoG7PtDcVWNb7QWaNt/hbUv54kf8pfMVi7Y5K5Sc3tvHM3CHfGEjzQ+UD5ytgPMkfylDGcOtlzcdlRWiLo1n1l1kUq7nwBX3iR/ZMdSt++qNSWCrdLyTEZc6CkVjDggujcispLg= Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=valinux.co.jp; Received: from TY7P286MB7722.JPNP286.PROD.OUTLOOK.COM (2603:1096:405:38f::10) by OS7P286MB7115.JPNP286.PROD.OUTLOOK.COM (2603:1096:604:42f::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.315.14; Wed, 12 Aug 2026 15:57:34 +0000 Received: from TY7P286MB7722.JPNP286.PROD.OUTLOOK.COM ([fe80::2305:327c:28ec:9b32]) by TY7P286MB7722.JPNP286.PROD.OUTLOOK.COM ([fe80::2305:327c:28ec:9b32%3]) with mapi id 15.21.0292.024; Wed, 12 Aug 2026 15:57:34 +0000 From: Koichiro Den To: Vinod Koul , Frank Li , Manivannan Sadhasivam , Gustavo Pimentel , Kees Cook , =?UTF-8?q?Krzysztof=20Wilczy=C5=84ski?= , Kishon Vijay Abraham I , Bjorn Helgaas , Christoph Hellwig , Serge Semin , Cai Huoqing , Niklas Cassel Cc: Devendra K Verma , dmaengine@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH v5 10/24] dmaengine: dw-edma: Prepare LL kicks for event serialization Date: Thu, 13 Aug 2026 00:57:07 +0900 Message-ID: <20260812155721.2807506-11-den@valinux.co.jp> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20260812155721.2807506-1-den@valinux.co.jp> References: <20260812155721.2807506-1-den@valinux.co.jp> Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: TY6P301CA0013.JPNP301.PROD.OUTLOOK.COM (2603:1096:405:3be::11) To TY7P286MB7722.JPNP286.PROD.OUTLOOK.COM (2603:1096:405:38f::10) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: TY7P286MB7722:EE_|OS7P286MB7115:EE_ X-MS-Office365-Filtering-Correlation-Id: baedf21d-aac1-48ec-391a-08def88a6c7a X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|376014|7416014|366016|1800799024|23010399003|10070799003|18002099003|22082099003|56012099006|6133799003|10067099003|921020; X-Microsoft-Antispam-Message-Info: 3xMjAHakW1VjSMGyrN+e/92x1B/vy5ytQ1jI53X8SexAjIv9qhWOad0eyNKEQUmBD3Hew2CyE55WNl0btdjROU8+iakLbm/dLM3FZg5izNRwttHVOitlFK7TirRb4TPXWutUGB153VtyfShwyR8hjoAMogB9KnlA/XtRV0EBWvjZecoo5H+/erroeAE8eMkQgprlDu30/PGmwv1OaGaQcQLR81x17agnAcknhHK8yBY7W8VhI7UtjbIixsSVX2Ud6LrSpd/D565HjkcWX+MMn8xnSHirRvLNFNbB7Eoxtgvy5Fkn3VhAPCcIzMlpYzPUqdBJnNTfqH9ueZ/hMjEcqvP4Bd42XiFBOjJaChz3zp4HCmw7xz8+6PlKdJ342g0r5nuLGZGXx9xA3TJ5dFtF9QK5wUObdYMIZQmIIf/81lanlQGusCjv9x6uXA77b0FRa/BJ9iXBBjRwnMZ9VrHxA2Xrjo5Mcm7DIKIDqVhCY4dhBRUIzGjDsdNDOu4kWcr7k/qAbgEdwlalL/i33ORhr3Ua90Tv+iPjl3SbR89LEKdIe4BXOtCDeYw25sQYV89gQXMIYQodMkQwgTBgh7XWAdSaaPWikYzreEbF3hAz+AbnLKDm33ZbX4LYFe8gfTZAXRP1WOVkQpV5qVlmlLb5ztuzq2gQu0MjvrFBeGMdGne7pOdwVNqipibZF876nEo+jU41X0X+skvA9rjiOMeHCA== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:TY7P286MB7722.JPNP286.PROD.OUTLOOK.COM;PTR:;CAT:NONE;SFS:(13230040)(376014)(7416014)(366016)(1800799024)(23010399003)(10070799003)(18002099003)(22082099003)(56012099006)(6133799003)(10067099003)(921020);DIR:OUT;SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?6BpCEEHcKowjCE3IYxBXyqjPx4Rt3F1gnD6Gi/t3tc2gNE5VTIv8kHkDqfWC?= =?us-ascii?Q?p9XD6HA+y2MipPGY12FoFmKBaqM4bwlxG1CQ6NDiPMIkp6JcFu7HR77OHLXi?= =?us-ascii?Q?aNPyoYQp6GgurChdYRegdX1PFWrPKsxp8aai+X8y5nqlgG+l776U8jTFOcQv?= =?us-ascii?Q?MmdE552iN+C809w2p/7BZ9QeShGDYBPyU9Ls6wZND8qikis+CzauBUgCJEki?= =?us-ascii?Q?2tiPGQT+raNs3lCqIkAEp8J+BdChmR1VoUW5siU8zg7MKmtMhEz8PWkf4D38?= =?us-ascii?Q?bmN96LwL2Wgspm5p0RarF2KjqUO0FvzG8pr2r6ZQWrri329JFHJkCUbzW5mK?= =?us-ascii?Q?9mYnUCHhkYl3CNqPfvptsiYdItYcMy8GzfvVImEyaCW6v4Oi5zG0KJ++e1No?= =?us-ascii?Q?ghaMnZgE+PJszUqFDdXRgSXrOfuBwGM/yqWOjrG/0KsejQWO3Brb84U/PUXC?= =?us-ascii?Q?RYp2cVszk8MiStTq+0eQWIjg3WGDOVEFYHAflrv1/tn/aGnE6PVakzRkgBQ+?= =?us-ascii?Q?ASeMkE5FYWSVwpHCPkCoN8Dpvmao+fZl663H+vAcm4aRFV/zfsCT2PRJs73n?= =?us-ascii?Q?XPhRTR5GenWEZloLh6lGXiAtuqIRnyBCLmsXFufcDG/qhc0GCAy1G8B44Q1v?= =?us-ascii?Q?e7/kpHB01nMGJxlec5VukaSsCoqfjhyOTN/5hcCfMGrWIoK7cyDRLVxozL/+?= =?us-ascii?Q?qFmJwkQ2MjPX1jZ6Q51TlK3+4D2y6C2tF9TgCMCNiz5dZ14qp+rEhulndmxe?= =?us-ascii?Q?5F7DiqlIV4kdHl08UEg5pf+UOKPkFumLsTtub8PTslJBtR23hzWeIR51VikW?= =?us-ascii?Q?0Anvz73GIYDlXiJdfoB8WoIZggzYYvdOzILQS0tzl9tiNLsM/nDzzvRvLVjY?= =?us-ascii?Q?b+8iWB3BaKy46MYqoHzJs4ipba/UrC0C+nZx4LP6MSQ4103VkEd9zEicT27J?= =?us-ascii?Q?S25ui2Inl9/cH1VNTbKz4i532yWGD9u1e+RSTu0x9TbtJ6Tb1Drj5Zh2BuCw?= =?us-ascii?Q?eglZ96UTqVPesjyXVjgI4vsIyi6BlrfovVJrQRe/aFGJsZDrbCaksuqWR86O?= =?us-ascii?Q?VewsRXy8hGlSEW/f1vEIOUh/6GeE0Jt90bun2Cslf2TfWpibXhOVpk9QCdKo?= =?us-ascii?Q?5MZMJoq07nAZvtKsyKsiP7ws3hHbVbqfvtIjGGVASLSwDfVfVx0E27laac8r?= =?us-ascii?Q?ehAbE3i2rM9RK4ac15995x6skv6qbOPQYD96eB4mnklmVdHpm7UxRezHWLX1?= =?us-ascii?Q?ty0l22+2AvPcgeuEDuCL29PsMUahJsgvwXOwm2qbNLrpk69GM5vOJ7Q1J/RN?= =?us-ascii?Q?kXl/LoCz4JSrYGzR4vihjTcbN0zOmOnarLY8vGqT3vqPIFBUSkJqWrrqygPt?= =?us-ascii?Q?Y509B23fNm+jiPwkN6dVY7BJkPki9DbPecyqxEk4QJg2wN+PGqryVmhFA7oT?= =?us-ascii?Q?QKY5p0nvXTGffq0cILE5AB2zVa8qpdnoA8y+wEdLHUyFPQ8/h5M8hY93KtAl?= =?us-ascii?Q?q0qMEDHOoHzawT4q4sNSy5J3Vv6iblAoTPD+AhpfBNgWk5Fz/VRAXZV8lSfh?= =?us-ascii?Q?yKpCAPUJRN0Eq0UdreM4ZLxMReigByPprhL+IgLPp0ECDLH4r0AAaeK3yNh7?= =?us-ascii?Q?dJVZKF0MNoLSJUAr69lSypUAApF26sR9vcjXaxVqkaJxsGt+1KR9QIje+s9/?= =?us-ascii?Q?PmaDBU7Zc2IWlULEFmO6U2JcHXVKUfk0L5VWQaDKZiTGTHc0uEefjyNqZodE?= =?us-ascii?Q?QWUUOKiiaxXgZ6rji7gy2mu/3cLpSK3JmhVRGGfw/t/AFE2Hho0/?= X-OriginatorOrg: valinux.co.jp X-MS-Exchange-CrossTenant-Network-Message-Id: baedf21d-aac1-48ec-391a-08def88a6c7a X-MS-Exchange-CrossTenant-AuthSource: TY7P286MB7722.JPNP286.PROD.OUTLOOK.COM X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 12 Aug 2026 15:57:34.2801 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 7a57bee8-f73d-4c5f-a4f7-d72c91c8c111 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: fQ1I0psCg/Vj959qj5UHrP5BUcSyQhGj6HhMnv2aW0YF71EQZGZ1eEbgugLkLVsKlk0xuKp8k5fT+ZvhxWDhzg== X-MS-Exchange-Transport-CrossTenantHeadersStamped: OS7P286MB7115 Content-Type: text/plain; charset="utf-8" Both eDMA and HDMA perform the same remote LL read-back immediately before writing the doorbell register. A later patch serializes each raw doorbell write against IRQ event capture. Keeping the read-back in the provider callback would hold event_lock across a remote read. Move the common read-back into the high-level dw_edma_core_ch_doorbell(), where it can run before the serialized section. Name the provider write wrapper dw_edma_core_do_ch_doorbell() to distinguish the raw write, and leave dw_edma_core_ch_maybe_doorbell() to decide whether a kick is needed. No functional change. Suggested-by: Frank Li Reviewed-by: Frank Li Signed-off-by: Koichiro Den --- Changes in v5: - No changes. drivers/dma/dw-edma/dw-edma-core.c | 18 ++++++++++++++++++ drivers/dma/dw-edma/dw-edma-core.h | 2 +- drivers/dma/dw-edma/dw-edma-v0-core.c | 16 ---------------- drivers/dma/dw-edma/dw-hdma-v0-core.c | 16 ---------------- 4 files changed, 19 insertions(+), 33 deletions(-) diff --git a/drivers/dma/dw-edma/dw-edma-core.c b/drivers/dma/dw-edma/dw-ed= ma-core.c index 97de0ad4489a..e2eba6d72b53 100644 --- a/drivers/dma/dw-edma/dw-edma-core.c +++ b/drivers/dma/dw-edma/dw-edma-core.c @@ -13,6 +13,7 @@ #include #include #include +#include #include #include #include @@ -254,6 +255,23 @@ static void dw_edma_finish_termination(struct dw_edma_= chan *chan) chan->status =3D EDMA_ST_IDLE; } =20 +static void dw_edma_core_ll_sync(struct dw_edma_chan *chan) +{ + /* + * Remote controller registers and LL memory may be reached through + * different paths. Complete posted LL writes before the doorbell. + */ + if (!(chan->dw->chip->flags & DW_EDMA_CHIP_LOCAL)) + readl(chan->ll_region.vaddr.io); +} + +/* Must be called with vc.lock held for an LL channel. */ +static void dw_edma_core_ch_doorbell(struct dw_edma_chan *chan) +{ + dw_edma_core_ll_sync(chan); + dw_edma_core_do_ch_doorbell(chan); +} + /* Must be called with vc.lock held. */ static void dw_edma_core_ch_maybe_doorbell(struct dw_edma_chan *chan) { diff --git a/drivers/dma/dw-edma/dw-edma-core.h b/drivers/dma/dw-edma/dw-ed= ma-core.h index 30b508eb3e3d..76fd5a41a707 100644 --- a/drivers/dma/dw-edma/dw-edma-core.h +++ b/drivers/dma/dw-edma/dw-edma-core.h @@ -298,7 +298,7 @@ static inline void dw_edma_core_ll_irq_clear(struct dw_= edma_chan *chan) chan->dw->core->ll_irq_clear(chan); } =20 -static inline void dw_edma_core_ch_doorbell(struct dw_edma_chan *chan) +static inline void dw_edma_core_do_ch_doorbell(struct dw_edma_chan *chan) { chan->dw->core->ch_doorbell(chan); } diff --git a/drivers/dma/dw-edma/dw-edma-v0-core.c b/drivers/dma/dw-edma/dw= -edma-v0-core.c index 4a1bb128fa00..c053cb4c2844 100644 --- a/drivers/dma/dw-edma/dw-edma-v0-core.c +++ b/drivers/dma/dw-edma/dw-edma-v0-core.c @@ -480,20 +480,6 @@ static void dw_edma_v0_core_ch_enable(struct dw_edma_c= han *chan) upper_32_bits(chan->ll_region.paddr)); } =20 -static void dw_edma_v0_sync_ll_data(struct dw_edma_chan *chan) -{ - /* - * In case of remote eDMA engine setup, the DW PCIe RP/EP internal - * configuration registers and application memory are normally accessed - * over different buses. Ensure LL-data reaches the memory before the - * doorbell register is toggled by issuing the dummy-read from the remote - * LL memory in a hope that the MRd TLP will return only after the - * last MWr TLP is completed - */ - if (!(chan->dw->chip->flags & DW_EDMA_CHIP_LOCAL)) - readl(chan->ll_region.vaddr.io); -} - static void dw_edma_v0_core_ch_config(struct dw_edma_chan *chan) { struct dw_edma *dw =3D chan->dw; @@ -624,8 +610,6 @@ static void dw_edma_v0_core_ch_doorbell(struct dw_edma_= chan *chan) { struct dw_edma *dw =3D chan->dw; =20 - dw_edma_v0_sync_ll_data(chan); - /* Doorbell */ SET_RW_32(dw, chan->dir, doorbell, FIELD_PREP(EDMA_V0_DOORBELL_CH_MASK, chan->id)); diff --git a/drivers/dma/dw-edma/dw-hdma-v0-core.c b/drivers/dma/dw-edma/dw= -hdma-v0-core.c index c6d7f9885aa8..0b3c8496b0ad 100644 --- a/drivers/dma/dw-edma/dw-hdma-v0-core.c +++ b/drivers/dma/dw-edma/dw-hdma-v0-core.c @@ -282,20 +282,6 @@ static void dw_hdma_v0_core_ch_enable(struct dw_edma_c= han *chan) HDMA_V0_CONSUMER_CYCLE_STAT | HDMA_V0_CONSUMER_CYCLE_BIT); } =20 -static void dw_hdma_v0_sync_ll_data(struct dw_edma_chan *chan) -{ - /* - * In case of remote HDMA engine setup, the DW PCIe RP/EP internal - * configuration registers and application memory are normally accessed - * over different buses. Ensure LL-data reaches the memory before the - * doorbell register is toggled by issuing the dummy-read from the remote - * LL memory in a hope that the MRd TLP will return only after the - * last MWr TLP is completed - */ - if (!(chan->dw->chip->flags & DW_EDMA_CHIP_LOCAL)) - readl(chan->ll_region.vaddr.io); -} - static void dw_hdma_v0_core_non_ll_start(struct dw_edma_chan *chan, struct dw_edma_burst *child) { @@ -393,8 +379,6 @@ static void dw_hdma_v0_core_ch_doorbell(struct dw_edma_= chan *chan) { struct dw_edma *dw =3D chan->dw; =20 - dw_hdma_v0_sync_ll_data(chan); - /* Doorbell */ SET_CH_32(dw, chan->dir, chan->id, doorbell, HDMA_V0_DOORBELL_START); } --=20 2.51.0 From nobody Tue Sep 29 04:14:12 2026 Received: from TY3P286CU002.outbound.protection.outlook.com (mail-japaneastazon11020112.outbound.protection.outlook.com [52.101.229.112]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 70209476697; Wed, 12 Aug 2026 15:57:48 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=52.101.229.112 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1786550272; cv=fail; b=PAZxnvPPfffzZ6dKpKYMzZdTxqIi20KdiI0r2hGhhr8r4rzvFI0ycGUNgTPPPM/3OgAUVAA5GBLHSKJybYv5Q8aq4GE/NUr3X1sQjZjTeMd74jUJRADjmR08Qq7QKKEn4cVUcy4gsy/4okPOjoxHi8PTclxtySAddyOV6MBfksA= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1786550272; c=relaxed/simple; bh=0q/+iQ+RFZgv7TelG+NPvuFde80DfHOQrSwkdrFCGw4=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: Content-Type:MIME-Version; b=GkPYIVjuPvphZ6hZ6qG69kJuSdklTfiMC/rkZ9cbP4TVRZ0KEq5UIYZmkLRyQMriBPPjMT+U3e9XSLj3t+L4Cd5l9vojLCBGsp0uX8Go8BZ87AehKT06bsn/2BsonlciiOFwglwsCA2OGeKsAwR3lX+tbPhFr3zaJTFw50GG4MM= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=valinux.co.jp; spf=pass smtp.mailfrom=valinux.co.jp; dkim=pass (1024-bit key) header.d=valinux.co.jp header.i=@valinux.co.jp header.b=UohDVUd7; arc=fail smtp.client-ip=52.101.229.112 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=valinux.co.jp Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=valinux.co.jp Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=valinux.co.jp header.i=@valinux.co.jp header.b="UohDVUd7" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=nb6QHXfmG7iLl7jcLcDnGGpPcJSK5qyg6G+wO3Ftx/2WMYKngkJy9zFkcYVR0fLeIsoHdi656Qkp9LJuvot7IdaQXfyH1HJRHq8tcb7ASW61BOjM0kHT1LxFlzlNmZgcMooqlpaMc0Ju3wV5dYZGJXtyNVzuudq1EsP4qTGYoeL/7jbIcS/klSd0y+sqQhX5aBoje87a4Dm45GZo04m/TidFhaPgR6q2fS17ayogno8IjwMvZHymwEfJem1Hfu7nFQ5KbCTg9X0xlMaa8eWnKMu3JI1ncmE6eaFONE6NFK51M8jY+PNa3ArLKcJxbjV4/802j4kqRD+t+5EYJsuVBA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=KoDJXxh6CfK6spMKyoSpqY/RXlhSGBfrvV5iyL+pruo=; b=m+CbSFLHxfKSjvLavj6LuU3oYSGoqqMVR3GaVu3u1mwt4X+rHNaBzM8GqBvxHO/4TLiF3yCSxYG1XNNKJxzsBWSBQY3ArEguR4ppI+lIEV/sLEw4DcI8gsm8mySVRqzQkatModQv7fEryq7s+srTZVPpSo+PY7NHm95l6bwnSWDN8RbccynZ/B1c5Y5+L8anyVKq7iOJjCmQyjArqxLBgtWjeY++mZHOVYT884rwW2sX7AUdXPnA6BK5C4MY6354H1Ar138c3//wjRciCDqEtKLAPIP0DPB8s9dGCeGkAootwZMutu/Sc6LTinaYIzbseh3vKnkAx2OBtoKI1QWysQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=valinux.co.jp; dmarc=pass action=none header.from=valinux.co.jp; dkim=pass header.d=valinux.co.jp; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=valinux.co.jp; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=KoDJXxh6CfK6spMKyoSpqY/RXlhSGBfrvV5iyL+pruo=; b=UohDVUd7XYvDtdBori7B2hwalf1IimWOdAey2byLCBpmz4+BX77sozELRwsoPxwpHbVnsyHznJvtMsaJQRB1MqaJxy4wJ0TnJ8Bpd5lk8lJZqdErX6kDpE+LY6anikyM0JJ4Y0nkV6z+EyKdIh6pR/YTXvgQoVgoYR024NM2vmo= Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=valinux.co.jp; Received: from TY7P286MB7722.JPNP286.PROD.OUTLOOK.COM (2603:1096:405:38f::10) by OS7P286MB7115.JPNP286.PROD.OUTLOOK.COM (2603:1096:604:42f::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.315.14; Wed, 12 Aug 2026 15:57:35 +0000 Received: from TY7P286MB7722.JPNP286.PROD.OUTLOOK.COM ([fe80::2305:327c:28ec:9b32]) by TY7P286MB7722.JPNP286.PROD.OUTLOOK.COM ([fe80::2305:327c:28ec:9b32%3]) with mapi id 15.21.0292.024; Wed, 12 Aug 2026 15:57:35 +0000 From: Koichiro Den To: Vinod Koul , Frank Li , Manivannan Sadhasivam , Gustavo Pimentel , Kees Cook , =?UTF-8?q?Krzysztof=20Wilczy=C5=84ski?= , Kishon Vijay Abraham I , Bjorn Helgaas , Christoph Hellwig , Serge Semin , Cai Huoqing , Niklas Cassel Cc: Devendra K Verma , dmaengine@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH v5 11/24] dmaengine: dw-edma: Serialize LL event capture with channel kicks Date: Thu, 13 Aug 2026 00:57:08 +0900 Message-ID: <20260812155721.2807506-12-den@valinux.co.jp> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20260812155721.2807506-1-den@valinux.co.jp> References: <20260812155721.2807506-1-den@valinux.co.jp> Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: TY6P301CA0001.JPNP301.PROD.OUTLOOK.COM (2603:1096:405:3be::12) To TY7P286MB7722.JPNP286.PROD.OUTLOOK.COM (2603:1096:405:38f::10) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: TY7P286MB7722:EE_|OS7P286MB7115:EE_ X-MS-Office365-Filtering-Correlation-Id: f88e41a6-7cc2-4149-7ab5-08def88a6ce4 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|376014|7416014|366016|1800799024|23010399003|10070799003|18002099003|22082099003|56012099006|6133799003|5023799004|10067099003|921020; X-Microsoft-Antispam-Message-Info: +mB6LKaKmTL6XBokYMC089AqfaybKT/shwMEHtkvuSATmTMnOAJh4Pq7fVeFIGilVJeS2Fv+N6neH9sasXbCu1C1YCUW+CSKn+er1hyam3X7h7PBbb2tKsKh0WTFEZtSQ/angaDWMtro3gKIYWzpLC4wpUjPfYztTUed3LGDB3KXAapZQDYnL8JzDF1+CWHXedBnV2MprQ6CXaFwCe3jEJVHbHgbrK/hlNrXt0c9fLb3UjV8AqG0XQJ0ihnrkb4z8tpZ1ilJ7vrxWaMHL2Tg4XH60AuCJCzxrr44srSP1ilpgzyZK9YEevGKCDhAZ/a9bWi5zic5zPNBViciFeJP5ubyowC2fYE01ySlLIg+j1YXBE8uf1J1tUdQUr/V0i8+mO0qW69COuHilfr7lnQykIoNho5RHqXLxwdNrWrJybt5kom+hTJ2w/KmDidNYde43j0w/eOqBiP9UrhlRwYS2QmNyIP//12VRSIVieuoZKonSYItH96FiBWUQmCUYGBaFCU1DvGH9MYPNjCCqDpMj4tPuTNxabMVYJj6OAuKKSQeqt5ng5t33j4f9+fnWjKTrzdQ2SOUQfdotEdZS+BI0KX3zBR7VoSUsst8UvwXyG7G0QF0ukjZfWxCe5slr56CfjvH+9PrpHH+uwT0KjcaaF93vrEFa0xrBiXSauml5gqn845o0mgLad1MgJtBcR1falvOtROJdFFRHO9M0dXsMg== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:TY7P286MB7722.JPNP286.PROD.OUTLOOK.COM;PTR:;CAT:NONE;SFS:(13230040)(376014)(7416014)(366016)(1800799024)(23010399003)(10070799003)(18002099003)(22082099003)(56012099006)(6133799003)(5023799004)(10067099003)(921020);DIR:OUT;SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?aujMsjIqjgwnqDJO16+qyfTotb6/Ub19vSYnRy8wRsG5onPkd4omX/v3JS2+?= =?us-ascii?Q?jcCCULLXdecejPxsbf1bzI4ls9ydlZ+OmQNUue2454ntmAz1vexH2sas546V?= =?us-ascii?Q?tewozRkg3s1YcGnmroEdr6ifBXw11z8DJWcwnc+dQaMLGZqCdtWt5MKmO1ae?= =?us-ascii?Q?5BQs9XiDYfp/+XkAACW6+cGscK1WJP8TJkT2Mgd7sKNoK5KlslVmEZ+8MqMH?= =?us-ascii?Q?YlXpWrPjmnCglT4/F/GN4J2KFufbir37DBeLmyRedAExxVEiWmVIufOtU+CP?= =?us-ascii?Q?n14oXUBt4uOkWinKNi1mlyRbx58NgeuOfbLSwMWMLCHLsN+uBgifGzu5fMYg?= =?us-ascii?Q?3QRvK0SkOBmzrES4CrIO8/qSX1PJBEWdD/EfcDgzdzERj2C+qDYovIqoxgyj?= =?us-ascii?Q?PWCmJCmPntijtHWMc5I0O+9ssFNQnk+t70TIhLulW0ap6fSxnBBZD96xwxpU?= =?us-ascii?Q?dffKeIFgaHFQ2GulBVcohi9KSouvxW+KWBu848AB4C0I29ee7yHozz58ib7A?= =?us-ascii?Q?j0nczNdrPNCGmQnplCspL/ELp762uNOjt98jnpC2fHHy5z9uProYTKs8z4rK?= =?us-ascii?Q?Gp3WifrR0lnG2Tj8bUt8hbl4IN5W/Q8vsrMOrQDlVpaqyB3u5+ovks4Byx79?= =?us-ascii?Q?0yZYRVVFjP1YvbLq076N4tzIV97Uyz/2HYkvYfP2qO1lzsSKD/1KyfNdx7Xo?= =?us-ascii?Q?SkX0LtJ1hRDzwGVqvXQPsyNGQgd4rUVZXKSL9WOxdLt2W7JpNUKsmcA6//xR?= =?us-ascii?Q?dQUM6fBV4iWhfRMmexU6mRx+yMYDRM5FH3G7qVs0tomB0TPEKMXQO75kFT8/?= =?us-ascii?Q?34RRlT1HqlqurmUIlIEYj9rOxJndNPGbNfg9lfJmNQBc+BkqOgxJmlJeL+FE?= =?us-ascii?Q?iWMHCr50AeLntoNOjIGMVYFuT5Egom20yV1NDfA38Itzfjrho+GOHKcWv9UB?= =?us-ascii?Q?ndPAj4PsOVn1N6c31nln0NsDqh39iKfROC9UhK0ASpyzxx0Pi37rdR/BhvS6?= =?us-ascii?Q?1yFFswyZ8ybcoGcI9lgvtU7UEH3nOBgr4TC6NQubTY10L55hwkip6DgjHe1D?= =?us-ascii?Q?pm3ecVmsdJgC6ovCiOMX6Q0jCkUS+bTiP4aDo7g24bb9nvyCK6KCnpfmurKd?= =?us-ascii?Q?YISLQMyKs493MX86JiJQpZ5HYREVDoSrmgNwI1FHcwSstSCv6RLR/d0tUs6T?= =?us-ascii?Q?wg78KGq0+Al4tSXFmIha5+b3sMFOZIcUvr+l+Jzmc2SOvJ9096rq4FHnbYzp?= =?us-ascii?Q?VJrW/907gFwXpspNnebH06Ye8XLsCd5N8dYEDeM12CWmwu65JlH75kPa93e/?= =?us-ascii?Q?lZBd3xdOborCDpw+l41sAo1mby9NTv82JNtM89SWRJtf6CD94QqMnoDGBgAL?= =?us-ascii?Q?+cLdXJL+3ZI4LZdhr9KNUkS/jNYs1yUlcbDuPHvb3j3Xuwl4VtewdImchJyB?= =?us-ascii?Q?lC9a+eeRKtAZe3VFHvzt8qHCust5Y3VEhmUwPL8P1AYP3NTODdtTEb3bioKa?= =?us-ascii?Q?5A/o/tAnc4u9w7k9cpLRjBVRnrnC2V5Pr4EbdNJf08/Xzapsp1iiWa7mspFR?= =?us-ascii?Q?lEvYmsP51y1vZhVRqxq5pjHCIzcCwErIH3KWfra33Q8yMJbg+PJiKnXX36+p?= =?us-ascii?Q?Z0Hxh6Y74C17JTx3mf2qUZ38Bh5FrLVSlxljR2VIoN3RfzVQAhHKDedeT3N5?= =?us-ascii?Q?BYOchc+1ZxF8QjzKXWrT9G0shC8ScFSOMvGGxaxeEi9vNxTEedtzqxz2BmU3?= =?us-ascii?Q?R9BUVoOGnQ3EkgvBPLZBZhftFOxHBgGnoxDNysmeeYixXd+bFA0e?= X-OriginatorOrg: valinux.co.jp X-MS-Exchange-CrossTenant-Network-Message-Id: f88e41a6-7cc2-4149-7ab5-08def88a6ce4 X-MS-Exchange-CrossTenant-AuthSource: TY7P286MB7722.JPNP286.PROD.OUTLOOK.COM X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 12 Aug 2026 15:57:34.9839 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 7a57bee8-f73d-4c5f-a4f7-d72c91c8c111 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: ui9brD4V4+HCKopIlZ5xtyfh9RFoity6UhlZj3HKnR8xRcUz0CkAszeKMHAjd4QIjacTHvYVccCk6U8tdLLz0w== X-MS-Exchange-Transport-CrossTenantHeadersStamped: OS7P286MB7115 Content-Type: text/plain; charset="utf-8" The hard IRQ path clears interrupt status before deferred handling takes vc.lock. If a doorbell starts a new hardware run in between, the deferred handler can apply an event from the previous run. Use the event lock to serialize the interrupt status read and clear, snapshot recording, and LL doorbell writes. Do not write another doorbell while a snapshot is pending. A ring reset discards the snapshot under the same lock. When events coalesce, keep STOP over PROGRESS. Changing chan->request alone neither changes the ring nor writes a doorbell, so it does not invalidate the snapshot. Preserve it across request changes. A later patch uses PROGRESS and STOP to decide whether a pending STOP or PAUSE request can finish. Suggested-by: Frank Li Signed-off-by: Koichiro Den --- Changes in v5: - Drop the event_scope removal now that patch 8 no longer introduces the temporary API. (Frank) - Move the dw_edma_ll_snapshot_discard() call in issue_pending() here from the progress-reclamation patch, avoiding unnecessary removal and re-addition of the helper in between. - Take event_lock directly in dw_edma_core_reset_ll(), avoiding a tempora= ry event-discard wrapper before ll_restart_armed is added. Note: The last two changes are refactoring only and do not change behavior. drivers/dma/dw-edma/dw-edma-core.c | 110 +++++++++++++++++++++++++- drivers/dma/dw-edma/dw-edma-core.h | 37 +++++++++ drivers/dma/dw-edma/dw-edma-v0-core.c | 65 +++++++-------- drivers/dma/dw-edma/dw-hdma-v0-core.c | 32 +++++--- 4 files changed, 198 insertions(+), 46 deletions(-) diff --git a/drivers/dma/dw-edma/dw-edma-core.c b/drivers/dma/dw-edma/dw-ed= ma-core.c index e2eba6d72b53..fc83f12560a5 100644 --- a/drivers/dma/dw-edma/dw-edma-core.c +++ b/drivers/dma/dw-edma/dw-edma-core.c @@ -33,7 +33,8 @@ struct dw_edma_desc *vd2dw_edma_desc(struct virt_dma_desc= *vd) =20 enum dw_edma_deferred_event { DW_EDMA_DEFERRED_DONE =3D BIT(0), - DW_EDMA_DEFERRED_ABORT =3D BIT(1), + DW_EDMA_DEFERRED_LL =3D BIT(1), + DW_EDMA_DEFERRED_ABORT =3D BIT(2), }; =20 static inline @@ -74,6 +75,44 @@ static void vchan_free_desc(struct virt_dma_desc *vdesc) kfree(vd2dw_edma_desc(vdesc)); } =20 +static void dw_edma_ll_snapshot_discard_locked(struct dw_edma_chan *chan) +{ + lockdep_assert_held(dw_edma_event_lock(chan)); + + chan->ll_irq.event =3D DW_EDMA_LL_EVENT_NONE; +} + +static void dw_edma_ll_snapshot_discard(struct dw_edma_chan *chan) +{ + guard(spinlock_irqsave)(dw_edma_event_lock(chan)); + + dw_edma_ll_snapshot_discard_locked(chan); +} + +/* Must be called with vc.lock held. */ +static bool +dw_edma_ll_snapshot_take(struct dw_edma_chan *chan, + struct dw_edma_ll_snapshot *snapshot) +{ + guard(spinlock_irqsave)(dw_edma_event_lock(chan)); + + if (chan->ll_irq.event =3D=3D DW_EDMA_LL_EVENT_NONE) + return false; + + *snapshot =3D chan->ll_irq; + dw_edma_ll_snapshot_discard_locked(chan); + + return true; +} + +static void dw_edma_ll_event_discard_locked(struct dw_edma_chan *chan) +{ + lockdep_assert_held(dw_edma_event_lock(chan)); + + dw_edma_core_ll_irq_clear(chan); + dw_edma_ll_snapshot_discard_locked(chan); +} + /* Must be called with vc.lock held. */ static void dw_edma_set_request(struct dw_edma_chan *chan, enum dw_edma_request reques= t) @@ -120,6 +159,9 @@ static void dw_edma_core_reset_ll(struct dw_edma_chan *= chan) dw_edma_core_ll_link(chan, chan->ll_max, chan->cb, chan->ll_region.paddr); =20 + scoped_guard(spinlock_irqsave, dw_edma_event_lock(chan)) { + dw_edma_ll_event_discard_locked(chan); + } dw_edma_core_ch_enable(chan); chan->ll_valid =3D true; } @@ -269,6 +311,17 @@ static void dw_edma_core_ll_sync(struct dw_edma_chan *= chan) static void dw_edma_core_ch_doorbell(struct dw_edma_chan *chan) { dw_edma_core_ll_sync(chan); + + guard(spinlock_irqsave)(dw_edma_event_lock(chan)); + + /* + * A recorded event belongs to the current ring and must be consumed + * before starting another hardware run. + */ + if (chan->ll_irq.event !=3D DW_EDMA_LL_EVENT_NONE) + return; + + dw_edma_ll_event_discard_locked(chan); dw_edma_core_do_ch_doorbell(chan); } =20 @@ -452,6 +505,8 @@ static void dw_edma_device_issue_pending(struct dma_cha= n *dchan) if (chan->configured && vchan_issue_pending(&chan->vc) && chan->request =3D=3D EDMA_REQ_NONE && chan->status =3D=3D EDMA_ST_IDLE) { + if (!chan->non_ll && !dw_edma_ll_pending(chan)) + dw_edma_ll_snapshot_discard(chan); chan->status =3D EDMA_ST_BUSY; dw_edma_start_transfer(chan); dw_edma_core_ch_maybe_doorbell(chan); @@ -779,6 +834,18 @@ static void dw_edma_done_interrupt(struct dw_edma_chan= *chan) spin_unlock_irqrestore(&chan->vc.lock, flags); } =20 +static void dw_edma_ll_interrupt(struct dw_edma_chan *chan) +{ + struct dw_edma_ll_snapshot snapshot; + + guard(spinlock_irqsave)(&chan->vc.lock); + + if (!dw_edma_ll_snapshot_take(chan, &snapshot)) + return; + + dw_edma_done_interrupt_locked(chan); +} + static void dw_edma_abort_interrupt(struct dw_edma_chan *chan) { struct virt_dma_desc *vd; @@ -811,6 +878,8 @@ static void dw_edma_irq_work(struct work_struct *work) =20 if (events & DW_EDMA_DEFERRED_DONE) dw_edma_done_interrupt(chan); + if (events & DW_EDMA_DEFERRED_LL) + dw_edma_ll_interrupt(chan); if (events & DW_EDMA_DEFERRED_ABORT) dw_edma_abort_interrupt(chan); } while (atomic_read(&chan->irq_pending)); @@ -825,14 +894,29 @@ static void dw_edma_queue_irq_work(struct dw_edma_cha= n *chan, =20 static void dw_edma_record_irq(struct dw_edma_chan *chan, unsigned int eve= nts) { + struct dw_edma_ll_snapshot snapshot =3D { + .event =3D events & DW_EDMA_IRQ_STOP ? + DW_EDMA_LL_EVENT_STOP : DW_EDMA_LL_EVENT_PROGRESS, + }; unsigned int pending =3D 0; =20 - if (events & (DW_EDMA_IRQ_DONE | DW_EDMA_IRQ_PROGRESS | - DW_EDMA_IRQ_STOP)) - pending |=3D DW_EDMA_DEFERRED_DONE; + lockdep_assert_held(dw_edma_event_lock(chan)); + if (events & DW_EDMA_IRQ_ABORT) pending |=3D DW_EDMA_DEFERRED_ABORT; =20 + if (chan->non_ll) { + if (events & (DW_EDMA_IRQ_DONE | DW_EDMA_IRQ_STOP)) + pending |=3D DW_EDMA_DEFERRED_DONE; + } else if (events & (DW_EDMA_IRQ_DONE | DW_EDMA_IRQ_PROGRESS | + DW_EDMA_IRQ_STOP)) { + /* STOP is final for this run; do not replace it with progress. */ + if (chan->ll_irq.event !=3D DW_EDMA_LL_EVENT_STOP || + snapshot.event =3D=3D DW_EDMA_LL_EVENT_STOP) + chan->ll_irq =3D snapshot; + pending |=3D DW_EDMA_DEFERRED_LL; + } + if (pending) dw_edma_queue_irq_work(chan, pending); } @@ -1021,6 +1105,7 @@ static void dw_edma_device_synchronize(struct dma_cha= n *dchan) dw_edma_wait_termination(dchan); cancel_work_sync(&chan->irq_work); atomic_set(&chan->irq_pending, 0); + dw_edma_ll_snapshot_discard(chan); vchan_synchronize(&chan->vc); } =20 @@ -1072,6 +1157,7 @@ static int dw_edma_channel_setup(struct dw_edma *dw, = u32 wr_alloc, u32 rd_alloc) chan->irq_mode =3D dw_edma_get_default_irq_mode(chan); INIT_WORK(&chan->irq_work, dw_edma_irq_work); atomic_set(&chan->irq_pending, 0); + chan->ll_irq.event =3D DW_EDMA_LL_EVENT_NONE; =20 if (chan->dir =3D=3D EDMA_DIR_WRITE) chan->ll_region =3D chip->ll_region_wr[chan->id]; @@ -1265,6 +1351,7 @@ int dw_edma_probe(struct dw_edma_chip *chip) { struct device *dev; struct dw_edma *dw; + struct dw_edma_chan *chan; u16 hw_wr_ch_cnt; u16 hw_rd_ch_cnt; u32 wr_alloc =3D 0; @@ -1308,6 +1395,8 @@ int dw_edma_probe(struct dw_edma_chip *chip) } =20 raw_spin_lock_init(&dw->lock); + for (i =3D 0; i < ARRAY_SIZE(dw->event_lock_per_dir); i++) + spin_lock_init(&dw->event_lock_per_dir[i]); =20 /* * chip->ll_*_cnt describes the channels exposed by this instance. Keep @@ -1337,6 +1426,19 @@ int dw_edma_probe(struct dw_edma_chip *chip) if (!dw->chan) return -ENOMEM; =20 + /* Event locks must be ready before request_irq(). */ + for (i =3D 0; i < dw->wr_ch_cnt + dw->rd_ch_cnt; i++) { + enum dw_edma_dir dir =3D i < dw->wr_ch_cnt ? + EDMA_DIR_WRITE : EDMA_DIR_READ; + + chan =3D &dw->chan[i]; + spin_lock_init(&chan->event_lock_per_chan); + if (chip->mf =3D=3D EDMA_MF_HDMA_NATIVE) + chan->event_lock =3D &chan->event_lock_per_chan; + else + chan->event_lock =3D &dw->event_lock_per_dir[dir]; + } + snprintf(dw->name, sizeof(dw->name), "dw-edma-core:%s", dev_name(chip->dev)); =20 diff --git a/drivers/dma/dw-edma/dw-edma-core.h b/drivers/dma/dw-edma/dw-ed= ma-core.h index 76fd5a41a707..a3df59976a2b 100644 --- a/drivers/dma/dw-edma/dw-edma-core.h +++ b/drivers/dma/dw-edma/dw-edma-core.h @@ -48,6 +48,12 @@ enum dw_edma_irq_event { DW_EDMA_IRQ_ABORT =3D BIT(3), }; =20 +enum dw_edma_ll_event { + DW_EDMA_LL_EVENT_NONE, + DW_EDMA_LL_EVENT_PROGRESS, + DW_EDMA_LL_EVENT_STOP, +}; + struct dw_edma_chan; struct dw_edma_chunk; =20 @@ -71,6 +77,10 @@ struct dw_edma_desc { struct dw_edma_burst burst[] __counted_by(nburst); }; =20 +struct dw_edma_ll_snapshot { + enum dw_edma_ll_event event; +}; + struct dw_edma_chan { struct virt_dma_chan vc; struct dw_edma *dw; @@ -99,6 +109,16 @@ struct dw_edma_chan { u32 ll_head; u32 ll_done; =20 + /* + * LL event recorded by the hard IRQ handler. The event lock + * serializes its capture with a new hardware run; vc.lock serializes + * its consumption with LL state. + */ + struct dw_edma_ll_snapshot ll_irq; + /* Native HDMA lock storage. */ + spinlock_t event_lock_per_chan; + spinlock_t *event_lock; /* Selected event lock */ + u32 ll_max; /* Data entries */ struct dw_edma_region ll_region; /* Linked list */ bool ll_valid; /* LL context programmed */ @@ -148,6 +168,8 @@ struct dw_edma { struct workqueue_struct *wq; =20 raw_spinlock_t lock; /* Protect v0 shared registers */ + /* Per-direction lock storage for the eDMA interrupt registers. */ + spinlock_t event_lock_per_dir[2]; =20 struct dw_edma_chip *chip; =20 @@ -171,7 +193,9 @@ struct dw_edma_core_ops { void (*ll_link)(struct dw_edma_chan *chan, u32 idx, bool cb, u64 addr); void (*ll_clear)(struct dw_edma_chan *chan, u32 idx); int (*ll_cur_idx)(struct dw_edma_chan *chan); + /* Called with dw_edma_event_lock(chan) held. */ void (*ll_irq_clear)(struct dw_edma_chan *chan); + /* Called with dw_edma_event_lock(chan) held for an LL channel. */ void (*ch_doorbell)(struct dw_edma_chan *chan); void (*ch_enable)(struct dw_edma_chan *chan); void (*ch_config)(struct dw_edma_chan *chan); @@ -215,6 +239,19 @@ struct dw_edma_chan *dchan2dw_edma_chan(struct dma_cha= n *dchan) return vc2dw_edma_chan(to_virt_chan(dchan)); } =20 +/* + * Lock ordering: + * + * chan->vc.lock -> dw_edma_event_lock(chan) + * + * Interrupt providers invoke the dw_edma_handler_t callback with the event + * lock held. The callback must not take vc.lock. + */ +static inline spinlock_t *dw_edma_event_lock(struct dw_edma_chan *chan) +{ + return chan->event_lock; +} + /* * Return the current LL entry index. A negative value means that the chan= nel * context is not initialized or was lost after a link reset. diff --git a/drivers/dma/dw-edma/dw-edma-v0-core.c b/drivers/dma/dw-edma/dw= -edma-v0-core.c index c053cb4c2844..3b300c746601 100644 --- a/drivers/dma/dw-edma/dw-edma-v0-core.c +++ b/drivers/dma/dw-edma/dw-edma-v0-core.c @@ -339,8 +339,8 @@ dw_edma_v0_core_handle_int(struct dw_edma_irq *dw_irq, = enum dw_edma_dir dir, dw_edma_handler_t handler) { struct dw_edma *dw =3D dw_irq->dw; - unsigned long total, pos, val; - irqreturn_t ret =3D IRQ_NONE; + u8 events[EDMA_V0_MAX_NR_CH] =3D {}; + unsigned long total, pos, val, active =3D 0; struct dw_edma_chan *chan; unsigned long off; unsigned long *mask; @@ -356,45 +356,48 @@ dw_edma_v0_core_handle_int(struct dw_edma_irq *dw_irq= , enum dw_edma_dir dir, mask =3D dw_irq->rd_mask; } =20 - /* - * DONE and ABORT status share one register, and on remote setups - * every read is a non-posted round trip across the PCIe link. Take - * one snapshot and derive both views from it. An abort raised - * after the snapshot is deferred, not lost: only bits observed in - * the snapshot are ever cleared below, so its status remains set and - * triggers another handler pass. - */ - sts =3D GET_RW_32(dw, dir, int_status); + if (!total) + return IRQ_NONE; =20 - val =3D FIELD_GET(EDMA_V0_DONE_INT_MASK, sts); - val &=3D *mask; - for_each_set_bit(pos, &val, total) { - chan =3D &dw->chan[pos + off]; + scoped_guard(spinlock_irqsave, dw_edma_event_lock(&dw->chan[off])) { + /* + * DONE and ABORT status share one register, and on remote setups + * every read is a non-posted round trip across the PCIe link. Take + * one snapshot and derive both views from it. + */ + sts =3D GET_RW_32(dw, dir, int_status); =20 - if (unlikely(dw_edma_core_ch_ignore_irq(chan))) - continue; + val =3D FIELD_GET(EDMA_V0_DONE_INT_MASK, sts); + val &=3D *mask; + for_each_set_bit(pos, &val, total) { + chan =3D &dw->chan[pos + off]; =20 - dw_edma_v0_core_clear_done_int(chan); - handler(chan, DW_EDMA_IRQ_DONE); + if (unlikely(dw_edma_core_ch_ignore_irq(chan))) + continue; =20 - ret =3D IRQ_HANDLED; - } + events[pos] |=3D DW_EDMA_IRQ_DONE; + active |=3D BIT(pos); + dw_edma_v0_core_clear_done_int(chan); + } =20 - val =3D FIELD_GET(EDMA_V0_ABORT_INT_MASK, sts); - val &=3D *mask; - for_each_set_bit(pos, &val, total) { - chan =3D &dw->chan[pos + off]; + val =3D FIELD_GET(EDMA_V0_ABORT_INT_MASK, sts); + val &=3D *mask; + for_each_set_bit(pos, &val, total) { + chan =3D &dw->chan[pos + off]; =20 - if (unlikely(dw_edma_core_ch_ignore_irq(chan))) - continue; + if (unlikely(dw_edma_core_ch_ignore_irq(chan))) + continue; =20 - dw_edma_v0_core_clear_abort_int(chan); - handler(chan, DW_EDMA_IRQ_ABORT); + events[pos] |=3D DW_EDMA_IRQ_ABORT; + active |=3D BIT(pos); + dw_edma_v0_core_clear_abort_int(chan); + } =20 - ret =3D IRQ_HANDLED; + for_each_set_bit(pos, &active, total) + handler(&dw->chan[pos + off], events[pos]); } =20 - return ret; + return active ? IRQ_HANDLED : IRQ_NONE; } =20 static void dw_edma_v0_write_ll_data(struct dw_edma_chan *chan, int i, diff --git a/drivers/dma/dw-edma/dw-hdma-v0-core.c b/drivers/dma/dw-edma/dw= -hdma-v0-core.c index 0b3c8496b0ad..d14dfcc174fb 100644 --- a/drivers/dma/dw-edma/dw-hdma-v0-core.c +++ b/drivers/dma/dw-edma/dw-hdma-v0-core.c @@ -177,10 +177,12 @@ dw_hdma_v0_core_handle_int(struct dw_edma_irq *dw_irq= , enum dw_edma_dir dir, dw_edma_handler_t handler) { struct dw_edma *dw =3D dw_irq->dw; - unsigned long total, pos, val; + unsigned int events; + unsigned long total, pos; irqreturn_t ret =3D IRQ_NONE; struct dw_edma_chan *chan; unsigned long off, *mask; + u32 val; =20 if (dir =3D=3D EDMA_DIR_WRITE) { total =3D dw->wr_ch_cnt; @@ -197,20 +199,28 @@ dw_hdma_v0_core_handle_int(struct dw_edma_irq *dw_irq= , enum dw_edma_dir dir, if (unlikely(dw_edma_core_ch_ignore_irq(chan))) continue; =20 - val =3D dw_hdma_v0_core_status_int(chan); - if (FIELD_GET(HDMA_V0_STOP_INT_MASK, val)) { - dw_hdma_v0_core_clear_done_int(chan); - handler(chan, DW_EDMA_IRQ_STOP); + events =3D 0; + scoped_guard(spinlock_irqsave, dw_edma_event_lock(chan)) { + val =3D dw_hdma_v0_core_status_int(chan); =20 - ret =3D IRQ_HANDLED; - } + if (FIELD_GET(HDMA_V0_STOP_INT_MASK, val)) { + events |=3D DW_EDMA_IRQ_STOP; + dw_hdma_v0_core_clear_done_int(chan); + } =20 - if (FIELD_GET(HDMA_V0_ABORT_INT_MASK, val)) { - dw_hdma_v0_core_clear_abort_int(chan); - handler(chan, DW_EDMA_IRQ_ABORT); + if (FIELD_GET(HDMA_V0_ABORT_INT_MASK, val)) { + events |=3D DW_EDMA_IRQ_ABORT; + dw_hdma_v0_core_clear_abort_int(chan); + } =20 - ret =3D IRQ_HANDLED; + if (events) + handler(chan, events); } + + if (!events) + continue; + + ret =3D IRQ_HANDLED; } =20 return ret; --=20 2.51.0 From nobody Tue Sep 29 04:14:12 2026 Received: from TY3P286CU002.outbound.protection.outlook.com (mail-japaneastazon11020112.outbound.protection.outlook.com [52.101.229.112]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 992E847ECDD; Wed, 12 Aug 2026 15:57:56 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=52.101.229.112 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1786550281; cv=fail; b=FepQBMRqw0H+SclKqDoUyd0n1mUbm4zWX9kKg0gGc46YBexZSansQd49LL1wq1QlNCBaH4g7nd9qBy0xJJc8Ty3ksOMYDcjJF4JTv7H5e9P2ZpIXhyGy/NzmxSL7NvHZlDFIyPQg4y+msnyh0JAiGlH1Q9G6zpOJQCNdl3X1kXg= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1786550281; c=relaxed/simple; bh=NnU/QNoePwLFCO0RgBepZHSmX4pv2BtpCSRpuk8bVGs=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: Content-Type:MIME-Version; b=QLQLet3PqTws8/Ei8Zzxhg06PdAiJfcnd0mr+tXL0kdecajn81ZRC6lzWstZTkyucaKMcl6Qa4wER2ab2Xumdl/oYf0fdJVL2y6TjX6Fe9amh4uoT0j80Xui9H3vwBxcqS8gd2HUZmifSijkuSdaddH63KMPvdq4qAR6MHXXxto= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=valinux.co.jp; spf=pass smtp.mailfrom=valinux.co.jp; dkim=pass (1024-bit key) header.d=valinux.co.jp header.i=@valinux.co.jp header.b=FAm7mRKC; arc=fail smtp.client-ip=52.101.229.112 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=valinux.co.jp Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=valinux.co.jp Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=valinux.co.jp header.i=@valinux.co.jp header.b="FAm7mRKC" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=tbOL8fmOmKUsE8ZuguQ0EHzU1xqsz5R0o/R2sWRpmKnwc8mwoxnS+LdpqNOXkT1wsLko+Y0tWZhnZIh+kzQVsz5KyU6cQh6uKQLBR4dASMuETROZFbU4yEiBse91tzpVp8lMVLQiPlaVhbJ454Rotb8njyOzx3w+x1XEebIl5i1/hQdky6yrym8e2/ZN2Nb03mKyY4pIfXkyUKRC8xMeRxAerSU9p41ewY/B+2NxIesNjcGVhVIt+MHdqfjBMvrVmoPKMg/jZGvrBeNrqRQC5HVUeYh7KSrn29l3lcfrtZzuoy04K23wxpQi1B4gzEaHJeUfVRhq/TC0eRFpth8kYA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=BQVhUE2+ZqUIfwnPWSj5mN+CliCWAEEqMzFvRjcEO3I=; b=mzDf1ymwQr14LqJexdSOgOeJWYhHxfA5zHD2QLFxGUwUuK1cPZhBn6oDgF7OvLK/z4ulS62CKSTIc4mZXxCyu53W7duZeV0XhlbThzpQD/pLiu108abXyFb4nVyTXSbdcHB1N1n+e2IYzDNFhUfVT/SF79cqPGx7Sq9ovCqI3NizUbKGg1uA1tfY3JVBvfTziejFvOFvQgAjpgP/OBt0y3Q45TZo0Sh1/D5eOkwLbS0u5EazHT53szCk8vjpkYkvz34iLMnTm4ERg2yZP2Ubly223QyCx5hY55cC1JDc1MyOkz5dqQdqIuhOtBXDKQFjdaXCV+7Z/Ztxw7fve0iy8w== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=valinux.co.jp; dmarc=pass action=none header.from=valinux.co.jp; dkim=pass header.d=valinux.co.jp; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=valinux.co.jp; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=BQVhUE2+ZqUIfwnPWSj5mN+CliCWAEEqMzFvRjcEO3I=; b=FAm7mRKC8YMh8B6XZ+JULj2ROcIQ++g/0MxS98AdtHh1ovcFtj6yIhW6c/r7i7e7njrfFS9qBT8ncxxyGVzm1wp37jrq46Dj6Mq188kPnUxBh2pMfs+wytwGwj+KTlyTqQpZ82ifQu2Ey9d7RcnvDtCLJT6sJBLv5ufABpzz2ME= Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=valinux.co.jp; Received: from TY7P286MB7722.JPNP286.PROD.OUTLOOK.COM (2603:1096:405:38f::10) by OS7P286MB7115.JPNP286.PROD.OUTLOOK.COM (2603:1096:604:42f::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.315.14; Wed, 12 Aug 2026 15:57:35 +0000 Received: from TY7P286MB7722.JPNP286.PROD.OUTLOOK.COM ([fe80::2305:327c:28ec:9b32]) by TY7P286MB7722.JPNP286.PROD.OUTLOOK.COM ([fe80::2305:327c:28ec:9b32%3]) with mapi id 15.21.0292.024; Wed, 12 Aug 2026 15:57:35 +0000 From: Koichiro Den To: Vinod Koul , Frank Li , Manivannan Sadhasivam , Gustavo Pimentel , Kees Cook , =?UTF-8?q?Krzysztof=20Wilczy=C5=84ski?= , Kishon Vijay Abraham I , Bjorn Helgaas , Christoph Hellwig , Serge Semin , Cai Huoqing , Niklas Cassel Cc: Devendra K Verma , dmaengine@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH v5 12/24] dmaengine: dw-edma: Keep channels stopped while ABORT is pending Date: Thu, 13 Aug 2026 00:57:09 +0900 Message-ID: <20260812155721.2807506-13-den@valinux.co.jp> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20260812155721.2807506-1-den@valinux.co.jp> References: <20260812155721.2807506-1-den@valinux.co.jp> Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: TYCP286CA0148.JPNP286.PROD.OUTLOOK.COM (2603:1096:400:31b::16) To TY7P286MB7722.JPNP286.PROD.OUTLOOK.COM (2603:1096:405:38f::10) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: TY7P286MB7722:EE_|OS7P286MB7115:EE_ X-MS-Office365-Filtering-Correlation-Id: fe6bf8e0-3e24-458a-0f1b-08def88a6d50 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|23010399003|10070799003|7416014|376014|1800799024|366016|921020|10067099003|56012099006|18002099003|22082099003; X-Microsoft-Antispam-Message-Info: zOYr7uI4SmlorzHEYual10/v6NvIYUjCmuE8jNeCINwtRfROhQBD57OuSwoyNcevHM3/WS4dzqVs7VpwPyAJRSvzeUYzTtM9eql8egsf3ccFIV+pABDM82DVOjuEcYxhOWl21V47vAH3I/6aMMGYc/TVopg0bgHn6fUAPrh0bAapcDAuZ6RFAQwViVlAdd382u5h61iknOQr/cgA7iESW+auJeM7jnbdY79UaNJMfeUYQMqL+lIvID/6i+apcALISlVOm09eV984cGvdW4h3cVZdGPeB5oo1VEGsp6XaM3+biUzsLSXP0LMJwobmt9+9Zy8JCdmwOWsfBB1Z/sO68hr77fkrGhN9Vfebw4nfnMNUYg7aLU4MPv2gdwK0orVOOong85zHg303uggUQgzVlKORIqHmnh3+vADffg+68VB1I39a4zhVyFTqLmiPqP5DFGRk5z9/bE7vB3dOOUCq2S6RP9O/PLH9zV8aRYycaX+Ks05/7UH+MZUDS7YCcrXbSyrOS2kos28hzgZaOKV/pNkSQl0qQFgNaKXMtVHqvmH3ern9lxImZAtbnaVMAJwytoNw3l3YLgo24ebz4vTbfoG0QOCPybjkc7ZMPvao8nMSgIF5siyyZHQzgT3X12ewA2S+Fy60/krx8rs50E18WSpYj0I2OYPh0sRLX1iJUnF8J9x3DUlkInCuUaxn9IJ8Y929bWKG30eQISexEJoB9A== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:TY7P286MB7722.JPNP286.PROD.OUTLOOK.COM;PTR:;CAT:NONE;SFS:(13230040)(23010399003)(10070799003)(7416014)(376014)(1800799024)(366016)(921020)(10067099003)(56012099006)(18002099003)(22082099003);DIR:OUT;SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?NDqA2oM5hY3r0UUsOXXRqS6HTnuPWmLPWbjbICq5q91DHZXerxPdutap8mpf?= =?us-ascii?Q?UzvO6/Ace0Lxr/NZO3k7z7MhjXrQ3aF/MSOXBmbLOKh267aRh5B8sTZ2e59i?= =?us-ascii?Q?SyEqydrqvHDQCtCF+MG98ryBy4pvr/C/QwRpZE27WTz3GdqxKuO1kO/UV/+R?= =?us-ascii?Q?6onbTMBl2rM7s0uoNpFv0qrSVnLfFyaemU85VPUZ1NDlr9BI4ODfyk+aeHMf?= =?us-ascii?Q?+FzTqgea4MRaPOTl89/sdyweBqdo++eUg+xv6/+6Carp/UfW4zJnUgQziJDQ?= =?us-ascii?Q?joN3xPQApfLodsgJtec0uzxYvdumsiQuWqATJu6mqan+gyfh2cwo+kTMhEEB?= =?us-ascii?Q?A7fp4xctNcND3nWH+JhRlxdy47I5okSN1/cQ3gMP1VLpz9rxMtkl22BFgyqo?= =?us-ascii?Q?/UFPAblzH0AhC20Z6IXglAWQq0EyON8cBbMLfz8rN+dGgye0K4kOCUF8pypS?= =?us-ascii?Q?rGQKOkdXb+LBG+Jfh7orBi0FjDc7PNpmO6SaQJUU4Ml+Pjx1Mwrft/QMyKxk?= =?us-ascii?Q?LGhj35bCef50pRfl0G9pCC9Vvih67EbUkUq4az+vuiM/YrE2ADL0OfMehEPS?= =?us-ascii?Q?7Mc3l6mzgusSlvFBqTxMeT5K8j5q2GMYyecWBRYIvq9qLkSWk3uwrxzWCrFL?= =?us-ascii?Q?5VNZkqNcscEHpX2TUXLvm/Deqb7FRWnFVn6lfea+gPm9wJ0bLrVn+HT5dh4Z?= =?us-ascii?Q?EtDtlH20afZjYZ4ad4IJjwtzqJ9KlUEyj81FOZijAi0Zv9oea4Qj8+zYE173?= =?us-ascii?Q?m0v3M4oOM8ilIM/HznrWfn56HKrx/0cBCLZUrj4Kxct077x0OWHYQNjphixv?= =?us-ascii?Q?tGrBuvLMEHvyEd7bzw+hzF5ve7ugDZArhy8vHhMut/203fvLkD8jin1Ah6XB?= =?us-ascii?Q?mNeF8EFs9XjFizOKpX4Z3S2pJXgm/B9gt+o+XqHoGl5yDQ309y4PtDiIqep/?= =?us-ascii?Q?QWMAkuSLrHn1L6N/w64N/gB4RH32z5VbQd404Vs4sKUyOhwscKzCnvfwlmQX?= =?us-ascii?Q?4bA3sgIvVzMitYrTQRyAo/AyMnzXgPqzzKtsVI/Hi4zYbfcD14Kvs8DZkuBz?= =?us-ascii?Q?fdDtMDySAzbKKkPDadtYSzsORUzc0+c9TqU04zqn0vY4QAYmA9H9fqHRvZjI?= =?us-ascii?Q?8k3XSmKJJTeCFKKsuqykJNV9hTkZUNMtDYFTvrARgm5DtOHY49dPjRGLU2e3?= =?us-ascii?Q?KOfvksskuZawuL5Rmwf/Skifgu5Rc85ygU9ym27hXHVRv/z0SSfRnb+LiS7d?= =?us-ascii?Q?7B+TvxJPFtJXNYiVmRD+ZtsIfhI+idqCc+S3Si55/J9Q4PJ8eIp8ijijpfHF?= =?us-ascii?Q?iAjp/7cdrBg00atYUI9R1c8PV9bexBidT+/eoyTXyY19RMJtHG2lnd4GWqML?= =?us-ascii?Q?BDSC0MGbt0c8pohKfAo3iFGZxAabFvJm/bKQiLlqmLuwbFDdHQuf+ON6urEd?= =?us-ascii?Q?WiE+wU3vjlzEweAh5jleMHK+wWc114Bjx7WqrDK3ZyyIdwiej0yPrFZWpN3m?= =?us-ascii?Q?RysF0mapjccRr9QGBwT7OOn6zYP6BPy6lXwflHxRHULZ7OD6GX2hrC7Vzruo?= =?us-ascii?Q?hdr4WYqeowtUEsoDr4F3BcIKfxflChbyBSTcVuJiJ8p95WaJsUCUUgQn5Lux?= =?us-ascii?Q?XHQIWhFU0jX06Wl4x0jmVYhzzhAyh04D4seAAHiC125Sdqvi2Ii6kcxcqn5G?= =?us-ascii?Q?CRGguJ7RSVZ5VzMfkK/wq2fkghjQI+agCzFN/415UyoS26xoQWYym72qcrch?= =?us-ascii?Q?8BKdLfKSMh6gwUPjY4mTACKLWsqYii7UIWlD+l2xglyh/YLd8hKL?= X-OriginatorOrg: valinux.co.jp X-MS-Exchange-CrossTenant-Network-Message-Id: fe6bf8e0-3e24-458a-0f1b-08def88a6d50 X-MS-Exchange-CrossTenant-AuthSource: TY7P286MB7722.JPNP286.PROD.OUTLOOK.COM X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 12 Aug 2026 15:57:35.6754 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 7a57bee8-f73d-4c5f-a4f7-d72c91c8c111 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: TQREnwyn45DoWusU6M7y0HdqGOe86X18G73DrbzK9m0XU8aN9JA2IW3Uln24BH/VgN6NlsyJTZDABhxYsUYyMg== X-MS-Exchange-Transport-CrossTenantHeadersStamped: OS7P286MB7115 Content-Type: text/plain; charset="utf-8" The hard IRQ path clears the ABORT interrupt status before deferred handling. Keep the event pending until the worker has handled it; otherwise a new start can race ahead after the status has been cleared. Set abort_pending before clearing the status. Before a non-LL start or LL doorbell, check both the pending flag and hardware status while holding the event lock. This covers the case where the start path gets the lock before the IRQ handler has latched the event. Handle ABORT before coalesced DONE or LL work. Also block LL progress consumers while ABORT is pending; tx_status() can otherwise complete a descriptor before the ABORT worker runs. ABORT cleanup makes that progress stale by completing or terminating descriptors and, for an LL channel, resetting the ring. Clear abort_pending only after cleanup. If termination sees a pending ABORT, leave EDMA_REQ_STOP set until the ABORT worker finishes. This prevents dw_edma_device_synchronize() from discarding the event before cleanup. Note also that only one issued descriptor can have LL entries published at this point. Before multi-descriptor publication is enabled, the ring reset is extended to abort every descriptor with outstanding published entries. Therefore, discarding coalesced DONE or LL work cannot strand another published descriptor. Signed-off-by: Koichiro Den --- Changes in v5: - Adjust context after moving the issue_pending() snapshot discard to patch 11. dw_edma_ll_snapshot_discard() is no longer removed here. drivers/dma/dw-edma/dw-edma-core.c | 87 ++++++++++++++++++++++++--- drivers/dma/dw-edma/dw-edma-core.h | 17 ++++++ drivers/dma/dw-edma/dw-edma-v0-core.c | 9 +++ drivers/dma/dw-edma/dw-hdma-v0-core.c | 8 +++ 4 files changed, 113 insertions(+), 8 deletions(-) diff --git a/drivers/dma/dw-edma/dw-edma-core.c b/drivers/dma/dw-edma/dw-ed= ma-core.c index fc83f12560a5..1deaf2d0da91 100644 --- a/drivers/dma/dw-edma/dw-edma-core.c +++ b/drivers/dma/dw-edma/dw-edma-core.c @@ -89,6 +89,32 @@ static void dw_edma_ll_snapshot_discard(struct dw_edma_c= han *chan) dw_edma_ll_snapshot_discard_locked(chan); } =20 +static void dw_edma_irq_events_discard(struct dw_edma_chan *chan) +{ + guard(spinlock_irqsave)(dw_edma_event_lock(chan)); + + dw_edma_ll_snapshot_discard_locked(chan); + chan->abort_pending =3D false; +} + +static bool dw_edma_abort_latch_locked(struct dw_edma_chan *chan) +{ + lockdep_assert_held(dw_edma_event_lock(chan)); + + if (!chan->abort_pending && + dw_edma_core_ch_abort_int_pending(chan)) + dw_edma_abort_event_mark(chan); + + return chan->abort_pending; +} + +static bool dw_edma_abort_is_pending(struct dw_edma_chan *chan) +{ + guard(spinlock_irqsave)(dw_edma_event_lock(chan)); + + return chan->abort_pending; +} + /* Must be called with vc.lock held. */ static bool dw_edma_ll_snapshot_take(struct dw_edma_chan *chan, @@ -99,12 +125,23 @@ dw_edma_ll_snapshot_take(struct dw_edma_chan *chan, if (chan->ll_irq.event =3D=3D DW_EDMA_LL_EVENT_NONE) return false; =20 + if (dw_edma_abort_latch_locked(chan)) + return false; + *snapshot =3D chan->ll_irq; dw_edma_ll_snapshot_discard_locked(chan); =20 return true; } =20 +/* Must be called with vc.lock held. */ +static void dw_edma_abort_event_clear(struct dw_edma_chan *chan) +{ + guard(spinlock_irqsave)(dw_edma_event_lock(chan)); + + chan->abort_pending =3D false; +} + static void dw_edma_ll_event_discard_locked(struct dw_edma_chan *chan) { lockdep_assert_held(dw_edma_event_lock(chan)); @@ -247,7 +284,20 @@ static int dw_edma_start_transfer(struct dw_edma_chan = *chan) if (!desc) return 0; =20 - if (!chan->non_ll && !chan->ll_valid) + if (chan->non_ll) { + guard(spinlock_irqsave)(dw_edma_event_lock(chan)); + + if (dw_edma_abort_latch_locked(chan)) + return 0; + + dw_edma_core_start(desc); + return 1; + } + + if (dw_edma_abort_is_pending(chan)) + return 0; + + if (!chan->ll_valid) dw_edma_core_reset_ll(chan); =20 dw_edma_core_start(desc); @@ -287,6 +337,11 @@ static void dw_edma_terminate_all_descs(struct dw_edma= _chan *chan) /* Must be called with vc.lock held after the channel has stopped. */ static void dw_edma_finish_termination(struct dw_edma_chan *chan) { + if (dw_edma_abort_is_pending(chan)) { + dw_edma_set_request(chan, EDMA_REQ_STOP); + return; + } + dw_edma_terminate_all_descs(chan); =20 /* Preserve a clean ring; resync only if entries remain published. */ @@ -314,6 +369,9 @@ static void dw_edma_core_ch_doorbell(struct dw_edma_cha= n *chan) =20 guard(spinlock_irqsave)(dw_edma_event_lock(chan)); =20 + if (dw_edma_abort_latch_locked(chan)) + return; + /* * A recorded event belongs to the current ring and must be consumed * before starting another hardware run. @@ -490,7 +548,8 @@ static int dw_edma_device_terminate_all(struct dma_chan= *dchan) } else { dw_edma_set_request(chan, EDMA_REQ_STOP); } - if (chan->status =3D=3D EDMA_ST_IDLE) + if (chan->status =3D=3D EDMA_ST_IDLE && + !dw_edma_abort_is_pending(chan)) dw_edma_set_request(chan, EDMA_REQ_NONE); =20 return err; @@ -846,14 +905,19 @@ static void dw_edma_ll_interrupt(struct dw_edma_chan = *chan) dw_edma_done_interrupt_locked(chan); } =20 -static void dw_edma_abort_interrupt(struct dw_edma_chan *chan) +static bool dw_edma_abort_interrupt(struct dw_edma_chan *chan) { struct virt_dma_desc *vd; unsigned long flags; =20 spin_lock_irqsave(&chan->vc.lock, flags); + if (!dw_edma_abort_is_pending(chan)) { + spin_unlock_irqrestore(&chan->vc.lock, flags); + return false; + } + vd =3D vchan_next_desc(&chan->vc); - if (vd && chan->request =3D=3D EDMA_REQ_STOP) { + if (chan->request =3D=3D EDMA_REQ_STOP) { dw_edma_terminate_all_descs(chan); } else if (vd) { dw_hdma_set_callback_result(vd, DMA_TRANS_ABORTED); @@ -864,7 +928,11 @@ static void dw_edma_abort_interrupt(struct dw_edma_cha= n *chan) dw_edma_core_reset_ll(chan); dw_edma_set_request(chan, EDMA_REQ_NONE); chan->status =3D EDMA_ST_IDLE; + dw_edma_abort_event_clear(chan); + spin_unlock_irqrestore(&chan->vc.lock, flags); + + return true; } =20 static void dw_edma_irq_work(struct work_struct *work) @@ -876,12 +944,14 @@ static void dw_edma_irq_work(struct work_struct *work) do { events =3D atomic_xchg(&chan->irq_pending, 0); =20 + /* ABORT cleanup makes coalesced DONE and LL work stale. */ + if ((events & DW_EDMA_DEFERRED_ABORT) && + dw_edma_abort_interrupt(chan)) + continue; if (events & DW_EDMA_DEFERRED_DONE) dw_edma_done_interrupt(chan); if (events & DW_EDMA_DEFERRED_LL) dw_edma_ll_interrupt(chan); - if (events & DW_EDMA_DEFERRED_ABORT) - dw_edma_abort_interrupt(chan); } while (atomic_read(&chan->irq_pending)); } =20 @@ -902,7 +972,7 @@ static void dw_edma_record_irq(struct dw_edma_chan *cha= n, unsigned int events) =20 lockdep_assert_held(dw_edma_event_lock(chan)); =20 - if (events & DW_EDMA_IRQ_ABORT) + if ((events & DW_EDMA_IRQ_ABORT) && chan->abort_pending) pending |=3D DW_EDMA_DEFERRED_ABORT; =20 if (chan->non_ll) { @@ -1105,7 +1175,7 @@ static void dw_edma_device_synchronize(struct dma_cha= n *dchan) dw_edma_wait_termination(dchan); cancel_work_sync(&chan->irq_work); atomic_set(&chan->irq_pending, 0); - dw_edma_ll_snapshot_discard(chan); + dw_edma_irq_events_discard(chan); vchan_synchronize(&chan->vc); } =20 @@ -1158,6 +1228,7 @@ static int dw_edma_channel_setup(struct dw_edma *dw, = u32 wr_alloc, u32 rd_alloc) INIT_WORK(&chan->irq_work, dw_edma_irq_work); atomic_set(&chan->irq_pending, 0); chan->ll_irq.event =3D DW_EDMA_LL_EVENT_NONE; + chan->abort_pending =3D false; =20 if (chan->dir =3D=3D EDMA_DIR_WRITE) chan->ll_region =3D chip->ll_region_wr[chan->id]; diff --git a/drivers/dma/dw-edma/dw-edma-core.h b/drivers/dma/dw-edma/dw-ed= ma-core.h index a3df59976a2b..d709aa274300 100644 --- a/drivers/dma/dw-edma/dw-edma-core.h +++ b/drivers/dma/dw-edma/dw-edma-core.h @@ -115,6 +115,8 @@ struct dw_edma_chan { * its consumption with LL state. */ struct dw_edma_ll_snapshot ll_irq; + /* ABORT is terminal and remains pending across LL state changes. */ + bool abort_pending; /* Native HDMA lock storage. */ spinlock_t event_lock_per_chan; spinlock_t *event_lock; /* Selected event lock */ @@ -185,6 +187,8 @@ struct dw_edma_core_ops { int (*ch_quiesce)(struct dw_edma_chan *chan); u16 (*ch_count)(struct dw_edma *dw, enum dw_edma_dir dir); enum dma_status (*ch_status)(struct dw_edma_chan *chan); + /* Called with dw_edma_event_lock(chan) held. */ + bool (*ch_abort_int_pending)(struct dw_edma_chan *chan); irqreturn_t (*handle_int)(struct dw_edma_irq *dw_irq, enum dw_edma_dir di= r, dw_edma_handler_t handler); void (*non_ll_start)(struct dw_edma_chan *chan, struct dw_edma_burst *chi= ld); @@ -252,6 +256,13 @@ static inline spinlock_t *dw_edma_event_lock(struct dw= _edma_chan *chan) return chan->event_lock; } =20 +static inline void dw_edma_abort_event_mark(struct dw_edma_chan *chan) +{ + lockdep_assert_held(dw_edma_event_lock(chan)); + + chan->abort_pending =3D true; +} + /* * Return the current LL entry index. A negative value means that the chan= nel * context is not initialized or was lost after a link reset. @@ -299,6 +310,12 @@ enum dma_status dw_edma_core_ch_status(struct dw_edma_= chan *chan) return chan->dw->core->ch_status(chan); } =20 +static inline bool +dw_edma_core_ch_abort_int_pending(struct dw_edma_chan *chan) +{ + return chan->dw->core->ch_abort_int_pending(chan); +} + static inline irqreturn_t dw_edma_core_handle_int(struct dw_edma_irq *dw_irq, enum dw_edma_dir dir, dw_edma_handler_t handler) diff --git a/drivers/dma/dw-edma/dw-edma-v0-core.c b/drivers/dma/dw-edma/dw= -edma-v0-core.c index 3b300c746601..20dff21a0603 100644 --- a/drivers/dma/dw-edma/dw-edma-v0-core.c +++ b/drivers/dma/dw-edma/dw-edma-v0-core.c @@ -318,6 +318,13 @@ static enum dma_status dw_edma_v0_core_ch_status(struc= t dw_edma_chan *chan) return DMA_ERROR; } =20 +static bool dw_edma_v0_core_ch_abort_int_pending(struct dw_edma_chan *chan) +{ + u32 sts =3D GET_RW_32(chan->dw, chan->dir, int_status); + + return FIELD_GET(EDMA_V0_ABORT_INT_MASK, sts) & BIT(chan->id); +} + static void dw_edma_v0_core_clear_done_int(struct dw_edma_chan *chan) { struct dw_edma *dw =3D chan->dw; @@ -390,6 +397,7 @@ dw_edma_v0_core_handle_int(struct dw_edma_irq *dw_irq, = enum dw_edma_dir dir, =20 events[pos] |=3D DW_EDMA_IRQ_ABORT; active |=3D BIT(pos); + dw_edma_abort_event_mark(chan); dw_edma_v0_core_clear_abort_int(chan); } =20 @@ -668,6 +676,7 @@ static const struct dw_edma_core_ops dw_edma_v0_core = =3D { .ch_quiesce =3D dw_edma_v0_core_ch_quiesce, .ch_count =3D dw_edma_v0_core_ch_count, .ch_status =3D dw_edma_v0_core_ch_status, + .ch_abort_int_pending =3D dw_edma_v0_core_ch_abort_int_pending, .handle_int =3D dw_edma_v0_core_handle_int, .ll_data =3D dw_edma_v0_core_ll_data, .ll_link =3D dw_edma_v0_core_ll_link, diff --git a/drivers/dma/dw-edma/dw-hdma-v0-core.c b/drivers/dma/dw-edma/dw= -hdma-v0-core.c index d14dfcc174fb..41e51eee71cc 100644 --- a/drivers/dma/dw-edma/dw-hdma-v0-core.c +++ b/drivers/dma/dw-edma/dw-hdma-v0-core.c @@ -172,6 +172,12 @@ static u32 dw_hdma_v0_core_status_int(struct dw_edma_c= han *chan) return GET_CH_32(dw, chan->dir, chan->id, int_stat); } =20 +static bool dw_hdma_v0_core_ch_abort_int_pending(struct dw_edma_chan *chan) +{ + return FIELD_GET(HDMA_V0_ABORT_INT_MASK, + dw_hdma_v0_core_status_int(chan)); +} + static irqreturn_t dw_hdma_v0_core_handle_int(struct dw_edma_irq *dw_irq, enum dw_edma_dir di= r, dw_edma_handler_t handler) @@ -210,6 +216,7 @@ dw_hdma_v0_core_handle_int(struct dw_edma_irq *dw_irq, = enum dw_edma_dir dir, =20 if (FIELD_GET(HDMA_V0_ABORT_INT_MASK, val)) { events |=3D DW_EDMA_IRQ_ABORT; + dw_edma_abort_event_mark(chan); dw_hdma_v0_core_clear_abort_int(chan); } =20 @@ -430,6 +437,7 @@ static const struct dw_edma_core_ops dw_hdma_v0_core = =3D { .ch_quiesce =3D dw_hdma_v0_core_ch_quiesce, .ch_count =3D dw_hdma_v0_core_ch_count, .ch_status =3D dw_hdma_v0_core_ch_status, + .ch_abort_int_pending =3D dw_hdma_v0_core_ch_abort_int_pending, .handle_int =3D dw_hdma_v0_core_handle_int, .non_ll_start =3D dw_hdma_v0_core_non_ll_start, .ll_data =3D dw_hdma_v0_core_ll_data, --=20 2.51.0 From nobody Tue Sep 29 04:14:12 2026 Received: from TY3P286CU002.outbound.protection.outlook.com (mail-japaneastazon11020119.outbound.protection.outlook.com [52.101.229.119]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id F1857479876; Wed, 12 Aug 2026 15:57:51 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=52.101.229.119 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1786550275; cv=fail; b=hFOAFcrYEJMv00Lbt/WwDygmQpMleH+s0/zMrW8SZC4V7FFbswCTE4rtiDlU29+/CCScyW9l32GJXOZ0SBxzOYB2oR+skzls0jxmjFXbozzHeRBFg9ZDcTrBJPj6F6rHz3nSQoJASf3bMn1YJDT3E+gFvDYWQi8BfV0tmH9FU1s= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1786550275; c=relaxed/simple; bh=Pu8N86kNVtpHsXFhUHLqqYlb3zcDHfJg4YKK6COtUYI=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: Content-Type:MIME-Version; b=aHs2E9iHfdp0PxhJwLSJRE9aj2xvRL8v+g62WQf2IRdmAqF3jTbfoaiTuR2AarQtdOFraPVgVfDqcTh0Sqj4qAa8zaXKQMiaxSKp9cFjs7oa9UQNHFYQWEGOQsLCOltGS2NwqAvCIDCpQYIVTFc07EykCYRUEhMEXzZz9Oensto= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=valinux.co.jp; spf=pass smtp.mailfrom=valinux.co.jp; dkim=pass (1024-bit key) header.d=valinux.co.jp header.i=@valinux.co.jp header.b=ECKYiVow; arc=fail smtp.client-ip=52.101.229.119 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=valinux.co.jp Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=valinux.co.jp Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=valinux.co.jp header.i=@valinux.co.jp header.b="ECKYiVow" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=yCYh/ze+5/DjGLNe/NbMgCDUQ0w8wGoF/LdmsMUhkVNZcbUzIwW2dHvE4Krs738sT+vHM7VlA1osK9oMb99t3SYBDt2WcV9rwofVKBkh0YHoM+pCTMCUYpWZbfU1iLVEgwClfWoDqyIWvhNXnzI3VuM1TThdadUCs7lyF81x72MwVRDthRSTuGVIHau4TPGMBiGA7xubPpkewsL3waPoN6TVUdFA1YQyJnn3sPDSd5TsHoF4e4uH/XEGy3WNZ0l7BfBxpIQsdI6R6CnHoicN/dhGp2TOhJTvjjdJOGEePNvJpXYZmZ4nlKrNNzFme8hwYMJW3h+DDKz8ERGdLaSD3Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=st2GYYL/DeudcaNsVx2uvm7ia0VEeQboPdq7hmatszc=; b=dkHaqhGaPMwIeiepZHqvBHtSRYcLPCRYFIYxEVJ3G5BHSQFxTHJH0eEheFRBXpTGMyc565Eb9MPk7iGZwPGYSjqmO42HizhCMesKQ7uMQpyBX67XIIK5z5QBr1lH+BH4/oCux2AGfXM57ElR6OWVoZ1xUNSKOGm+yGViBACwYjFn19PTZjyj3GPX4FLLXDmNi5yBgud8IWFNiNJiKVWzYO9jkIvXPynDnYfkZ8deIGWP9kA8Fo4h6MWByxYy3wCgGroopJqTL6uEcISrMGxJBLburXVUR/q5tCt9CnYJZ9GWTCwCqvagM8f062GQ9QyWUC7qzd56TuiXbAmFIXDtNQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=valinux.co.jp; dmarc=pass action=none header.from=valinux.co.jp; dkim=pass header.d=valinux.co.jp; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=valinux.co.jp; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=st2GYYL/DeudcaNsVx2uvm7ia0VEeQboPdq7hmatszc=; b=ECKYiVow+lznJej+SqIjBc3nqRjeyP2evXkO8i3gJFEhtFfb0mx5ibGXROB/1ESwOzlPzkXVIx9kMmGc0lSXW25hQsN5bSUmnjfCFl+JnXJa61NvwgWoQz/zPMHYPH4reHbkdCO20gyMO8MX6zyq4fsWOOJmLkYV8keY5sT1dGg= Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=valinux.co.jp; Received: from TY7P286MB7722.JPNP286.PROD.OUTLOOK.COM (2603:1096:405:38f::10) by OS7P286MB7115.JPNP286.PROD.OUTLOOK.COM (2603:1096:604:42f::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.315.14; Wed, 12 Aug 2026 15:57:36 +0000 Received: from TY7P286MB7722.JPNP286.PROD.OUTLOOK.COM ([fe80::2305:327c:28ec:9b32]) by TY7P286MB7722.JPNP286.PROD.OUTLOOK.COM ([fe80::2305:327c:28ec:9b32%3]) with mapi id 15.21.0292.024; Wed, 12 Aug 2026 15:57:36 +0000 From: Koichiro Den To: Vinod Koul , Frank Li , Manivannan Sadhasivam , Gustavo Pimentel , Kees Cook , =?UTF-8?q?Krzysztof=20Wilczy=C5=84ski?= , Kishon Vijay Abraham I , Bjorn Helgaas , Christoph Hellwig , Serge Semin , Cai Huoqing , Niklas Cassel Cc: Devendra K Verma , dmaengine@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH v5 13/24] dmaengine: dw-edma: Reclaim issued descriptors from IRQ-paired LL progress Date: Thu, 13 Aug 2026 00:57:10 +0900 Message-ID: <20260812155721.2807506-14-den@valinux.co.jp> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20260812155721.2807506-1-den@valinux.co.jp> References: <20260812155721.2807506-1-den@valinux.co.jp> Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: TYCP286CA0137.JPNP286.PROD.OUTLOOK.COM (2603:1096:400:31b::15) To TY7P286MB7722.JPNP286.PROD.OUTLOOK.COM (2603:1096:405:38f::10) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: TY7P286MB7722:EE_|OS7P286MB7115:EE_ X-MS-Office365-Filtering-Correlation-Id: 943dfdaa-b5af-4b21-69b3-08def88a6dbe X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|23010399003|10070799003|7416014|376014|1800799024|366016|921020|6133799003|10067099003|56012099006|18002099003|22082099003; X-Microsoft-Antispam-Message-Info: Fj1HvJCPneWiVf2jugKJgk9Q+fhtB1HDBWRQ5AXu8XZH0yHkMQha8J4AhMkEgzBzK+fb5b0IjXy+AMhMuYffrrZg3vw4yPuJNfoD7O9szvbHn5+GkIw5AKdb46go5lgp7Sw4xLDf/wt6y10HSh1aQMC3pAqw9eOIDcesJsv26aNp+gr0DbhJ59zB39o/K2oYNdOVKpaI//6AQ5Lrdn2DgoWRu6FU970U5cVJyJXM0PwSJ45NT0QuYYrMZ92RohJ4teXlh6MbA4oWnw6s4RgWsnfMvgRX90jhqthLOGaY5dIt4u6TOJWaCnc+GR8iBttvlK+PVEh8jHCmijEy31Ptmivlz8k3eLiBSptDEYHMj+52EakmMJny9YFKJnUUQXtygyIaa/M6IkXGg4IvKMSqSZdlmoLekvSZ37SdQCIsT7UhOl/8jp1sLlO7kDroF493wKexcV/WJyRFr2pY207KnI4SpGCg2a1Tqv8edLvg0sqjMOulf4kSApRM9c00xBbL9l/bF5hpAiX7VG3oQNrIAVM4hjdceqTtgyIEjqnO0lMstaPqvwxNbWYusQkax82op64WRspx4Kg3s8+fKRbY8wxA+WbkJeoo0BYqSqIRr8mvSeNyA29IvVt5pYfUQkAQwo+VL5S/Ff75zcxZB8M7eGam/L14ku56fF4JalpJr6psYc+sOB0HBHtmbZfEFCzq9Z7xpcRnq3z+EgZlkR4M5w== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:TY7P286MB7722.JPNP286.PROD.OUTLOOK.COM;PTR:;CAT:NONE;SFS:(13230040)(23010399003)(10070799003)(7416014)(376014)(1800799024)(366016)(921020)(6133799003)(10067099003)(56012099006)(18002099003)(22082099003);DIR:OUT;SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?moJbzFu6JnfU0Gtp+VPrKsLP0JnG/Ii7nByQ+w+wHcO7A7wjDu/5Yy/mKSre?= =?us-ascii?Q?V3aU5kH6eoTIdxkCFjt72f8FDq7kQ9BUXwlibfG7mmNEFwLpHLsN61TpWPA0?= =?us-ascii?Q?oF9QxJlAZ7eEXSo/0s+0zrfi1AvDL5nsUrhD9U7o0S+tLUJ5/iSZXVOQnpwW?= =?us-ascii?Q?R8EhgYJxdNgMrypP4ieKmsezmna7FxEr3gmFJWusp0Dusy0VjAxwCwODxwKg?= =?us-ascii?Q?IznslG8BeQPApQSljzdhS/xfLpk34metpSeKYXDS2mPpN2lTtTs3AR3S6/hP?= =?us-ascii?Q?OvDhhdKc3bGFZie7H3iglpsD7LarPdVXqPfwE1Ns7jGoDDc2SykZyBZFWn7O?= =?us-ascii?Q?lcexFB3fKoNDT0Zscc0WjM4h6F2ZGfXJPo8bG5c3kOQrGke1sv7ILUz+gVzD?= =?us-ascii?Q?LB27rt5MB7Gj+6XoZ3ZB5EHeBVC1tIMcKGIGCILdYvHq63AG6722knozvBC/?= =?us-ascii?Q?bJeyET3QhfJCe1dDw8wtw1qSAi++D2ayd8lwl3OOTXPURTIKjfOBKpxlmao3?= =?us-ascii?Q?kk6R3IfMTDrukwo88b71CqsLk5of6Nr03MMjOem3wGTrRA9d26ZOHXXkIYW5?= =?us-ascii?Q?dr0CKXejXhlwqsPmAsiJVysNrmfF76iBA//0ncSRfjZrbulnBfWmVt349z5c?= =?us-ascii?Q?1pf3OZn4AB4NbpJCWtR1zlN4OLvbp6w+kQZY5TfNYBMAIHv2jjwEqWdS7C0+?= =?us-ascii?Q?X0sDAnlvh8he3MPHe7cHoxYnFnJHXLQVGRsxGEnTxcaBOX+pFrYHmrX/WM1u?= =?us-ascii?Q?kpDA0XMRV9YOFRvl9Vgh5PGki3ibvqtc6rRP9ch6uKZa1PIWRk6M+wyDCWVR?= =?us-ascii?Q?D7/RlMpiW69nNc1XSZSHHWaCaOa51MKucDVEG11Ub0gnr/P+pHCZz7MAl/32?= =?us-ascii?Q?dAH6DXGd9nbRp9CtAG8j71dG5/12uM3CblztM1YU3faxr1dYMvthqFWCggaz?= =?us-ascii?Q?rgOV3Oor5kYdNnLrAk7rXDuVbx8sVc8TBS78ymfqnrjbo7CUeWJzRC5R5eXj?= =?us-ascii?Q?pYtZf7v3XjRwKGA0kpS3s6/+jGYq0vGiuBTjVjn6xoiXJQ+hGmyOsX7ZOUmP?= =?us-ascii?Q?wapAtDvzmHxhNWwe2X5kq6Ho6tQRXUa5DMkoTc+dt9W7LWwiYRApmsO5C4If?= =?us-ascii?Q?4wIAUe4AJd2SthIxFRbodfjbeP6nBiY6aQgTKMI4xE0+SQbB68ljUiL/qlz4?= =?us-ascii?Q?DHa97NfRYZW6LU9M56dYGV4kzO+dotCkI+prY8AjiM+9uX4asnQi/HdQ86RN?= =?us-ascii?Q?qSKk9fWifGJmfwPAUYWkBfT6Cj66WzzaoEJp0Nkzq/Q+whq0PjBecUAQKcRy?= =?us-ascii?Q?2x/o/SSH/B+vnLfKYrHN1jTbEqA60PSddeo2ehLW1vMQ08Ozea4QGO0Mc1IJ?= =?us-ascii?Q?8NOvU5eW5JkXK7203eYwf7Jbx91GhgRUIycqQXdJmqwUtBtjifkUZrMkb5Dp?= =?us-ascii?Q?lgUSCSAz+iCOi5DWiFgPCKEZC9lheI59Rc7i95aOoupJMtlImiOreEY+Js+K?= =?us-ascii?Q?M6NedG/eiRSa+s9L20SWdRxlbI5p6JQc4i/VasHMjZMUcSBNMkcTq/2xknMG?= =?us-ascii?Q?1wZaui0aL+XoK9m81464Yd4bzgWtkIqVhuXsJizPmpN3Q7gJ3vnJAtr9UAvF?= =?us-ascii?Q?XLs2V0m7kSAZtydnVjw/R7w/jEDaQrOMNbOziazcGg4t0nDrC39We+NrlHva?= =?us-ascii?Q?Sc4Hs+9bGmlV9MvWpPzvLWtAy8z+GUqsQgxGVDiF3y+Z4yeuYILk9ZW9gN/6?= =?us-ascii?Q?2BeVP/824GIybH2xoYMHcUFwHPZPNX38NRe7JPEJ6RgC1o14klM2?= X-OriginatorOrg: valinux.co.jp X-MS-Exchange-CrossTenant-Network-Message-Id: 943dfdaa-b5af-4b21-69b3-08def88a6dbe X-MS-Exchange-CrossTenant-AuthSource: TY7P286MB7722.JPNP286.PROD.OUTLOOK.COM X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 12 Aug 2026 15:57:36.4167 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 7a57bee8-f73d-4c5f-a4f7-d72c91c8c111 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 3VUUUAKTIc8ttigNeaqBG5DQOhYXJ75IKXJpisUooiDcCAYQCUn6xxjKk5ZTvjzQyOBL0NdT9K82RiIvZbpz5Q== X-MS-Exchange-Transport-CrossTenantHeadersStamped: OS7P286MB7115 Content-Type: text/plain; charset="utf-8" Dynamic append can place entries from several descriptors in one LL ring. Track the consumed boundary in ll_done, reuse entries behind it, and complete descriptors in issue order as done_burst advances. Normalize each IRQ-paired LLP sample to the exclusive boundary used by ll_done. A stopped sample points to the next entry. Keep a running sample one entry behind the raw LLP so an entry is not recycled before payload completion is established. For the eDMA-compatible interrupt interface, treat DONE as a stopped boundary only when channel status is complete and transfer size is zero. Record the first outstanding physical LL entry in each descriptor and verify that it matches ll_done before consuming that descriptor. On a mismatch, warn and resynchronize only if the sampled boundary has reached the descriptor; otherwise stop without completing it. Reclaiming entries as they are consumed also lets a descriptor larger than the ring advance. Keep one data entry free so a physical index remains unambiguous in the active producer window. Clear any published ring state on termination or abort before starting another transfer. LL progress now has its own completion path, so fold the temporary lock-held DONE helper back into its only caller. Suggested-by: Frank Li Signed-off-by: Koichiro Den --- Changes in v5: - Adjust context after moving the issue_pending() snapshot discard to patch 11. dw_edma_ll_snapshot_discard() is no longer re-added here. - Fold dw_edma_ll_clean_pending() into dw_edma_ll_consume_progress(), removing the one-line wrapper (pure refactoring). drivers/dma/dw-edma/dw-edma-core.c | 273 +++++++++++++++++++++----- drivers/dma/dw-edma/dw-edma-core.h | 4 + drivers/dma/dw-edma/dw-edma-v0-core.c | 6 + 3 files changed, 238 insertions(+), 45 deletions(-) diff --git a/drivers/dma/dw-edma/dw-edma-core.c b/drivers/dma/dw-edma/dw-ed= ma-core.c index 1deaf2d0da91..363993c64c44 100644 --- a/drivers/dma/dw-edma/dw-edma-core.c +++ b/drivers/dma/dw-edma/dw-edma-core.c @@ -53,13 +53,6 @@ dw_edma_alloc_desc(struct dw_edma_chan *chan, size_t nbu= rst) { struct dw_edma_desc *desc; =20 - /* - * For now, a descriptor that does not fit would stall the channel - * forever: reject it up front. - */ - if (!chan->non_ll && nburst > chan->ll_max - 1) - return NULL; - desc =3D kzalloc_flex(*desc, burst, nburst, GFP_NOWAIT); if (unlikely(!desc)) return NULL; @@ -79,6 +72,7 @@ static void dw_edma_ll_snapshot_discard_locked(struct dw_= edma_chan *chan) { lockdep_assert_held(dw_edma_event_lock(chan)); =20 + chan->ll_irq.idx =3D -1; chan->ll_irq.event =3D DW_EDMA_LL_EVENT_NONE; } =20 @@ -128,6 +122,7 @@ dw_edma_ll_snapshot_take(struct dw_edma_chan *chan, if (dw_edma_abort_latch_locked(chan)) return false; =20 + /* Consume each snapshot once, even if its boundary is later rejected. */ *snapshot =3D chan->ll_irq; dw_edma_ll_snapshot_discard_locked(chan); =20 @@ -215,7 +210,13 @@ static u32 dw_edma_core_get_used_num(struct dw_edma_ch= an *chan) =20 static u32 dw_edma_core_get_free_num(struct dw_edma_chan *chan) { - /* Keep one data entry free so equal indices mean an empty ring. */ + /* + * ll_done is the consumer boundary, so only the distance from ll_done + * to ll_head is occupied. Descriptor completion is tracked separately + * with done_burst. + * + * Keep one data entry free so equal indices mean an empty ring. + */ return chan->ll_max - 1 - dw_edma_core_get_used_num(chan); } =20 @@ -224,6 +225,25 @@ static bool dw_edma_ll_pending(struct dw_edma_chan *ch= an) return chan->ll_head !=3D chan->ll_done; } =20 +static u32 dw_edma_core_ch_transfer_size(struct dw_edma_chan *chan) +{ + if (!chan->dw->core->ch_transfer_size) + return U32_MAX; + + return chan->dw->core->ch_transfer_size(chan); +} + +static bool dw_edma_ll_done_is_stopped(struct dw_edma_chan *chan) +{ + /* + * Native HDMA reports STOP separately. The eDMA-compatible interrupt + * interface uses DONE for both progress and stop, so confirm a stopped + * boundary with channel status and transfer size. + */ + return dw_edma_core_ch_status(chan) =3D=3D DMA_COMPLETE && + dw_edma_core_ch_transfer_size(chan) =3D=3D 0; +} + static void dw_edma_core_ll_start(struct dw_edma_desc *desc) { struct dw_edma_chan *chan =3D desc->chan; @@ -231,6 +251,9 @@ static void dw_edma_core_ll_start(struct dw_edma_desc *= desc) u32 free; =20 free =3D dw_edma_core_get_free_num(chan); + if (free && desc->start_burst =3D=3D desc->done_burst) + desc->ll_start =3D chan->ll_head; + for (i =3D desc->start_burst; i < desc->nburst && free; i++, free--) { /* * Refresh the link element before filling the last data slot so @@ -252,7 +275,6 @@ static void dw_edma_core_ll_start(struct dw_edma_desc *= desc) } } =20 - desc->done_burst =3D desc->start_burst; desc->start_burst =3D i; } =20 @@ -352,6 +374,127 @@ static void dw_edma_finish_termination(struct dw_edma= _chan *chan) chan->status =3D EDMA_ST_IDLE; } =20 +/* + * Must be called with vc.lock held. Consume an LL progress boundary, + * advance ll_done and complete descriptors covered by the consumed range. + * Return true if ll_done advanced. + */ +static bool dw_edma_ll_consume_progress(struct dw_edma_chan *chan, int idx) +{ + struct virt_dma_desc *vd, *_vd; + bool advanced =3D false; + u32 done, gap; + + /* Ignore invalid, duplicate or stale progress. */ + if (idx < 0 || (u32)idx >=3D chan->ll_max) + return false; + + done =3D dw_edma_core_get_ll_dist(chan, chan->ll_done, idx); + if (!done || done > dw_edma_core_get_used_num(chan)) + return false; + + list_for_each_entry_safe(vd, _vd, &chan->vc.desc_issued, node) { + struct dw_edma_desc *desc =3D vd2dw_edma_desc(vd); + u32 consumed; + + if (!done) + break; + + if (WARN_ON_ONCE(desc->done_burst > desc->start_burst || + desc->start_burst > desc->nburst)) + return advanced; + + /* + * start_burst is the next burst to append. done_burst counts + * bursts already consumed by hardware. + */ + consumed =3D desc->start_burst - desc->done_burst; + if (!consumed) + break; + + /* + * ll_start ties the descriptor counters to the physical ring. + * If accounting lost entries before this descriptor, skip them + * only after the sampled boundary has reached ll_start. + */ + if (WARN_ON_ONCE(desc->ll_start !=3D chan->ll_done)) { + gap =3D dw_edma_core_get_ll_dist(chan, chan->ll_done, + desc->ll_start); + if (gap > done) + return advanced; + + chan->ll_done =3D desc->ll_start; + done -=3D gap; + advanced =3D true; + if (!done) + break; + } + + consumed =3D min(done, consumed); + desc->done_burst +=3D consumed; + desc->ll_start =3D (desc->ll_start + consumed) % chan->ll_max; + chan->ll_done =3D desc->ll_start; + done -=3D consumed; + advanced =3D true; + + /* + * Descriptors are published and retired in strict list order. A + * later descriptor cannot complete until this one is fully consumed. + */ + if (desc->done_burst !=3D desc->nburst) + break; + + /* Hardware has consumed this descriptor's LL entries. */ + dw_hdma_set_callback_result(vd, DMA_TRANS_NOERROR); + list_del(&vd->node); + vchan_cookie_complete(vd); + } + + WARN_ON_ONCE(done); + + return advanced; +} + +static int +dw_edma_ll_recycle_idx(struct dw_edma_chan *chan, int idx, + enum dw_edma_ll_event event) +{ + if (idx < 0 || (u32)idx > chan->ll_max) + return -EINVAL; + + /* + * Convert the raw LLP index to the exclusive boundary used by ll_done. + * For both eDMA and HDMA, once the engine has stopped, LLP points to + * the next element. ll_max is the link element, hence the following + * data boundary is 0. + */ + if (event =3D=3D DW_EDMA_LL_EVENT_STOP) + return idx =3D=3D chan->ll_max ? 0 : idx; + + /* + * Moving a running index one entry back cannot represent index 0 + * without wrapping it to ll_max - 1. That could falsely consume a full + * producer window, so wait for another sample or STOP. + */ + if (!idx) + return -EINVAL; + + /* + * A running eDMA LLP can move ahead of payload completion, so keep the + * boundary one entry behind it. + * + * DWC PCIe Controller Databook 6.10a-lca06, Section 7.2.1, Table 7-3 + * describes an HDMA watermark LLP as an inclusive LLE recycling + * boundary, which would normally translate to idx + 1. During testing + * on a DWC HDMA 6.30a integration, using that boundary for DMAengine + * completion let clients release DMA mappings while hardware still + * accessed them, causing IOMMU faults. Keep the boundary one entry + * behind the raw LLP for HDMA as well. Stopped samples continue to use + * the next-entry boundary above. + */ + return idx =3D=3D chan->ll_max ? chan->ll_max - 1 : idx - 1; +} + static void dw_edma_core_ll_sync(struct dw_edma_chan *chan) { /* @@ -834,31 +977,30 @@ dw_edma_device_prep_interleaved_dma(struct dma_chan *= dchan, return dw_edma_device_transfer(&xfer, dw_edma_device_get_config(dchan, NU= LL)); } =20 -/* Must be called with vc.lock held. */ -static void dw_edma_done_interrupt_locked(struct dw_edma_chan *chan) +static void dw_edma_done_interrupt(struct dw_edma_chan *chan) { struct dw_edma_desc *desc; struct virt_dma_desc *vd; + unsigned long flags; =20 - lockdep_assert_held(&chan->vc.lock); - - if (chan->status =3D=3D EDMA_ST_PAUSE) + spin_lock_irqsave(&chan->vc.lock, flags); + if (chan->status =3D=3D EDMA_ST_PAUSE) { + spin_unlock_irqrestore(&chan->vc.lock, flags); return; + } =20 switch (chan->request) { case EDMA_REQ_NONE: case EDMA_REQ_PAUSE: vd =3D vchan_next_desc(&chan->vc); - if (!vd) - break; - - desc =3D vd2dw_edma_desc(vd); - if (desc->start_burst >=3D desc->nburst) { - dw_hdma_set_callback_result(vd, DMA_TRANS_NOERROR); - list_del(&vd->node); - vchan_cookie_complete(vd); - if (!chan->non_ll) - chan->ll_done =3D chan->ll_head; + if (vd) { + desc =3D vd2dw_edma_desc(vd); + if (desc->start_burst >=3D desc->nburst) { + dw_hdma_set_callback_result(vd, + DMA_TRANS_NOERROR); + list_del(&vd->node); + vchan_cookie_complete(vd); + } } =20 if (chan->request =3D=3D EDMA_REQ_PAUSE) { @@ -871,25 +1013,12 @@ static void dw_edma_done_interrupt_locked(struct dw_= edma_chan *chan) break; =20 case EDMA_REQ_STOP: - vd =3D vchan_next_desc(&chan->vc); - if (!vd) - break; - dw_edma_finish_termination(chan); break; =20 default: break; } - dw_edma_core_ch_maybe_doorbell(chan); -} - -static void dw_edma_done_interrupt(struct dw_edma_chan *chan) -{ - unsigned long flags; - - spin_lock_irqsave(&chan->vc.lock, flags); - dw_edma_done_interrupt_locked(chan); spin_unlock_irqrestore(&chan->vc.lock, flags); } =20 @@ -902,7 +1031,37 @@ static void dw_edma_ll_interrupt(struct dw_edma_chan = *chan) if (!dw_edma_ll_snapshot_take(chan, &snapshot)) return; =20 - dw_edma_done_interrupt_locked(chan); + if (chan->status =3D=3D EDMA_ST_PAUSE) + return; + + dw_edma_ll_consume_progress(chan, snapshot.idx); + + if (snapshot.event =3D=3D DW_EDMA_LL_EVENT_PROGRESS && + chan->request !=3D EDMA_REQ_NONE) + goto out; + + switch (chan->request) { + case EDMA_REQ_NONE: + dw_edma_start_transfer(chan); + chan->status =3D dw_edma_ll_pending(chan) ? + EDMA_ST_BUSY : EDMA_ST_IDLE; + break; + + case EDMA_REQ_PAUSE: + dw_edma_set_request(chan, EDMA_REQ_NONE); + chan->status =3D EDMA_ST_PAUSE; + break; + + case EDMA_REQ_STOP: + dw_edma_finish_termination(chan); + break; + + default: + break; + } + +out: + dw_edma_core_ch_maybe_doorbell(chan); } =20 static bool dw_edma_abort_interrupt(struct dw_edma_chan *chan) @@ -965,21 +1124,44 @@ static void dw_edma_queue_irq_work(struct dw_edma_ch= an *chan, static void dw_edma_record_irq(struct dw_edma_chan *chan, unsigned int eve= nts) { struct dw_edma_ll_snapshot snapshot =3D { - .event =3D events & DW_EDMA_IRQ_STOP ? - DW_EDMA_LL_EVENT_STOP : DW_EDMA_LL_EVENT_PROGRESS, + .idx =3D -1, + .event =3D DW_EDMA_LL_EVENT_NONE, }; unsigned int pending =3D 0; =20 lockdep_assert_held(dw_edma_event_lock(chan)); =20 + /* + * Classify the LL event before normalizing its LLP sample to the + * exclusive consumer boundary. Keep STOP even without a valid + * boundary so deferred handling still sees that the run ended. + */ + if (!chan->non_ll && + (events & (DW_EDMA_IRQ_DONE | DW_EDMA_IRQ_PROGRESS | + DW_EDMA_IRQ_STOP))) { + if ((events & DW_EDMA_IRQ_STOP) || + ((events & DW_EDMA_IRQ_DONE) && + dw_edma_ll_done_is_stopped(chan))) + snapshot.event =3D DW_EDMA_LL_EVENT_STOP; + else + snapshot.event =3D DW_EDMA_LL_EVENT_PROGRESS; + + snapshot.idx =3D dw_edma_ll_recycle_idx(chan, + dw_edma_core_ll_cur_idx(chan), + snapshot.event); + if (snapshot.idx < 0 && + snapshot.event !=3D DW_EDMA_LL_EVENT_STOP) + snapshot.event =3D DW_EDMA_LL_EVENT_NONE; + } + if ((events & DW_EDMA_IRQ_ABORT) && chan->abort_pending) pending |=3D DW_EDMA_DEFERRED_ABORT; =20 - if (chan->non_ll) { - if (events & (DW_EDMA_IRQ_DONE | DW_EDMA_IRQ_STOP)) - pending |=3D DW_EDMA_DEFERRED_DONE; - } else if (events & (DW_EDMA_IRQ_DONE | DW_EDMA_IRQ_PROGRESS | - DW_EDMA_IRQ_STOP)) { + if (chan->non_ll && + (events & (DW_EDMA_IRQ_DONE | DW_EDMA_IRQ_STOP))) + pending |=3D DW_EDMA_DEFERRED_DONE; + + if (snapshot.event !=3D DW_EDMA_LL_EVENT_NONE) { /* STOP is final for this run; do not replace it with progress. */ if (chan->ll_irq.event !=3D DW_EDMA_LL_EVENT_STOP || snapshot.event =3D=3D DW_EDMA_LL_EVENT_STOP) @@ -1227,6 +1409,7 @@ static int dw_edma_channel_setup(struct dw_edma *dw, = u32 wr_alloc, u32 rd_alloc) chan->irq_mode =3D dw_edma_get_default_irq_mode(chan); INIT_WORK(&chan->irq_work, dw_edma_irq_work); atomic_set(&chan->irq_pending, 0); + chan->ll_irq.idx =3D -1; chan->ll_irq.event =3D DW_EDMA_LL_EVENT_NONE; chan->abort_pending =3D false; =20 diff --git a/drivers/dma/dw-edma/dw-edma-core.h b/drivers/dma/dw-edma/dw-ed= ma-core.h index d709aa274300..27cab6ca5e67 100644 --- a/drivers/dma/dw-edma/dw-edma-core.h +++ b/drivers/dma/dw-edma/dw-edma-core.h @@ -71,6 +71,7 @@ struct dw_edma_desc { =20 u32 alloc_sz; =20 + u32 ll_start; /* First outstanding LL entry */ size_t done_burst; size_t start_burst; size_t nburst; @@ -78,6 +79,7 @@ struct dw_edma_desc { }; =20 struct dw_edma_ll_snapshot { + int idx; enum dw_edma_ll_event event; }; =20 @@ -113,6 +115,7 @@ struct dw_edma_chan { * LL event recorded by the hard IRQ handler. The event lock * serializes its capture with a new hardware run; vc.lock serializes * its consumption with LL state. + * Valid indices use the exclusive boundary convention of ll_done. */ struct dw_edma_ll_snapshot ll_irq; /* ABORT is terminal and remains pending across LL state changes. */ @@ -189,6 +192,7 @@ struct dw_edma_core_ops { enum dma_status (*ch_status)(struct dw_edma_chan *chan); /* Called with dw_edma_event_lock(chan) held. */ bool (*ch_abort_int_pending)(struct dw_edma_chan *chan); + u32 (*ch_transfer_size)(struct dw_edma_chan *chan); irqreturn_t (*handle_int)(struct dw_edma_irq *dw_irq, enum dw_edma_dir di= r, dw_edma_handler_t handler); void (*non_ll_start)(struct dw_edma_chan *chan, struct dw_edma_burst *chi= ld); diff --git a/drivers/dma/dw-edma/dw-edma-v0-core.c b/drivers/dma/dw-edma/dw= -edma-v0-core.c index 20dff21a0603..b9d6205157b3 100644 --- a/drivers/dma/dw-edma/dw-edma-v0-core.c +++ b/drivers/dma/dw-edma/dw-edma-v0-core.c @@ -318,6 +318,11 @@ static enum dma_status dw_edma_v0_core_ch_status(struc= t dw_edma_chan *chan) return DMA_ERROR; } =20 +static u32 dw_edma_v0_core_ch_transfer_size(struct dw_edma_chan *chan) +{ + return GET_CH_32(chan->dw, chan->dir, chan->id, transfer_size); +} + static bool dw_edma_v0_core_ch_abort_int_pending(struct dw_edma_chan *chan) { u32 sts =3D GET_RW_32(chan->dw, chan->dir, int_status); @@ -677,6 +682,7 @@ static const struct dw_edma_core_ops dw_edma_v0_core = =3D { .ch_count =3D dw_edma_v0_core_ch_count, .ch_status =3D dw_edma_v0_core_ch_status, .ch_abort_int_pending =3D dw_edma_v0_core_ch_abort_int_pending, + .ch_transfer_size =3D dw_edma_v0_core_ch_transfer_size, .handle_int =3D dw_edma_v0_core_handle_int, .ll_data =3D dw_edma_v0_core_ll_data, .ll_link =3D dw_edma_v0_core_ll_link, --=20 2.51.0 From nobody Tue Sep 29 04:14:12 2026 Received: from TY3P286CU002.outbound.protection.outlook.com (mail-japaneastazon11020112.outbound.protection.outlook.com [52.101.229.112]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 1F04847A882; Wed, 12 Aug 2026 15:57:53 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=52.101.229.112 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1786550275; cv=fail; b=cZ895doRnk/4PfIh+LLrO7nGc38kAWCFPjGBuMaGK/Br9nPedVK4mEq/ulXTbzd3cInqtwXKLt9vd+qFgCPuUVF8XHxrbk3WjZcFDBpZX54TEdcG4boJ1TWCXqNzi9t5WAuDGzwxYgCK5owuX105J8pfu/IDLZ7lf2P/7YKBl3s= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1786550275; c=relaxed/simple; bh=CaewytvyS3x0FYwP3b6BcW+6JXMOLP6Qrg8T+rv9nxM=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: Content-Type:MIME-Version; b=ZqsL/FXCskMSUQWghuHBrg3NdWZG8Va3yn9gVAX3G6FBWN1nOFyHd5IiviOs2iwLEPxeWjr1ctxYLeYrGdaN2uKyORTK3XMdODP1NQfwIdz9HUeoLFMSgOv/6ob9Rb12HoLXWhU33Uit3OT21sosy8SJcBkmdUhuci2Fgaf4Bmo= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=valinux.co.jp; spf=pass smtp.mailfrom=valinux.co.jp; dkim=pass (1024-bit key) header.d=valinux.co.jp header.i=@valinux.co.jp header.b=ukVXu9Xs; arc=fail smtp.client-ip=52.101.229.112 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=valinux.co.jp Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=valinux.co.jp Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=valinux.co.jp header.i=@valinux.co.jp header.b="ukVXu9Xs" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=cOnjMxAXB2IdXdkDTfrfLdOeDMyA4bGDUc4U7EggM/SVWPU6UJ/Sil1B/+q1fuqnNNwSgoK/oQ1vU0ud1FWM5xOzZvJntg8MD+FcLGwIkkzTF+kgM5OyCQPCZHCaua2YPtPXqIC/Tmww+Nqvh/6fThphyh58KPgu47uQJRc4PUfXvqUPqqHBWjcQXLcXkeRFVyHtWe1s6kMBYdI3KtweTUg9TQ19Pr0BpvuAGr+fDT7Hm7TXovUik2gFHsPV1qPYIczy7xo1V6ZcWVici/qcUn/cYsKYqWcLhokzqGEoz9jeYLPGpS67w5hmZpS8xljNz2NmiGgjiGPYpUmTDIY2mA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=K+t3lqVSI6RIl01Qv4N8WBaW6BsdVDNi5UDZzoHtL7s=; b=scLZ2sHx+SPlJYGURmmLX2nkxm+vKLOljJs2ORjdvjKgK55+OSXnnPTj54oVgcN8XJqslKrF027zKD26ZAgW0xxs/iM9Jry//BDbWxgjG1vki4luS6RWbbEO/kYG/tUO6w98pqGWDNz6RhHbAiqkYmIjp7toBByliV0qhDiOhDUboIbY381ZTFzPW0svr2BB18WJj9mfLIeqoZW3fGDs45SjMTl2I4+hx6rDCxp4FJAVtNd6WUX0DhG3YvLYQTiN8AGhqqpndWh9rleQw85CXsbV7aBRTONX/jRLa0+Uggxk2sHgwcqQsEKK5BN62oDnlWYH1k5on/ZTLgRWibPVIw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=valinux.co.jp; dmarc=pass action=none header.from=valinux.co.jp; dkim=pass header.d=valinux.co.jp; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=valinux.co.jp; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=K+t3lqVSI6RIl01Qv4N8WBaW6BsdVDNi5UDZzoHtL7s=; b=ukVXu9XstgTZelN9B3xr4cx6xtcvd6458/Say/cXz6Vv1S6VlQB7n/4vdg0PSZxlOD8KfJXqp8BlssjxnFFUlRa5sld95TAf5Ccfcg+5K8b0dgd0QsCjzkCnnvswa92aNcWJi3norgZtlh+PPu1vy4bunke36FosUN/UvYtskeE= Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=valinux.co.jp; Received: from TY7P286MB7722.JPNP286.PROD.OUTLOOK.COM (2603:1096:405:38f::10) by OS7P286MB7115.JPNP286.PROD.OUTLOOK.COM (2603:1096:604:42f::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.315.14; Wed, 12 Aug 2026 15:57:37 +0000 Received: from TY7P286MB7722.JPNP286.PROD.OUTLOOK.COM ([fe80::2305:327c:28ec:9b32]) by TY7P286MB7722.JPNP286.PROD.OUTLOOK.COM ([fe80::2305:327c:28ec:9b32%3]) with mapi id 15.21.0292.024; Wed, 12 Aug 2026 15:57:37 +0000 From: Koichiro Den To: Vinod Koul , Frank Li , Manivannan Sadhasivam , Gustavo Pimentel , Kees Cook , =?UTF-8?q?Krzysztof=20Wilczy=C5=84ski?= , Kishon Vijay Abraham I , Bjorn Helgaas , Christoph Hellwig , Serge Semin , Cai Huoqing , Niklas Cassel Cc: Devendra K Verma , dmaengine@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH v5 14/24] dmaengine: dw-edma: Add LL interrupt placement policy Date: Thu, 13 Aug 2026 00:57:11 +0900 Message-ID: <20260812155721.2807506-15-den@valinux.co.jp> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20260812155721.2807506-1-den@valinux.co.jp> References: <20260812155721.2807506-1-den@valinux.co.jp> Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: TYCP286CA0181.JPNP286.PROD.OUTLOOK.COM (2603:1096:400:382::19) To TY7P286MB7722.JPNP286.PROD.OUTLOOK.COM (2603:1096:405:38f::10) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: TY7P286MB7722:EE_|OS7P286MB7115:EE_ X-MS-Office365-Filtering-Correlation-Id: 8c1e23a2-a510-4099-1420-08def88a6e3b X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|23010399003|10070799003|7416014|376014|1800799024|366016|921020|10067099003|56012099006|18002099003|22082099003; X-Microsoft-Antispam-Message-Info: zjfsbOofLugBgSIfXM/SRfkM7KuyxFlb+24OvEvdvPtSkN9TG5xLPHrfBqI5ynTicCMSTHczH2A0Owb16bZsdaI9zT88+8AczzFexasQQjTW741jXZjcujZH+xBjW8wuEf/8V3IOMuewr0XVSQhnBzrSc3VTMWDrQwWYw9WHIxQ7aNDMmqd7e5FKqXu5akHKhWPYwBJe8FzfwR+E6vsI8lN/EVdCyThjyZAfDjFlQD3KX6szLwXIVyY+2fj38qIm9w8gAaOqFEQcPV/UVCWqQMODWz+Wuh6B+EABIGj7aSyzCz28JRifJb7Nax8cg4I2Y/alJkd+GLZqNOxW6iOVDOnKBUYqvy/xTHZMLKt990/9GQae2bhYzQEUPvqYOv3TOBx/6XX5FtfV3wKg1ivGlWvNeJ8Pu1lGo7x9TtomDQczYQfPcodV4KLySN1N1A8GHwLHTUjAGzftWla7Iae4E/NkYFIMLVge3PcLK0ILtuaAipv2jO4woIFukwE/h0v9RzQadpJejxtLCvxMTq8L6Ci9lQW5CI5YkBbv1/0ZPCuOa+RKGUNgZD/n11dGYeO4Q6SKKhqGxo++RZ9iETzgfD+bfjprCWs2fQ4YIh0fWY0qEFlyioiwBCNrk6ctlQsN8rovrHwGxgEZGWbTJR3Bo1KqUHbTbL32GSFMAwq5XrPT+m9aCrvSblp58KYqXnSRMyWWoTnLZJ/o/14m05Agtw== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:TY7P286MB7722.JPNP286.PROD.OUTLOOK.COM;PTR:;CAT:NONE;SFS:(13230040)(23010399003)(10070799003)(7416014)(376014)(1800799024)(366016)(921020)(10067099003)(56012099006)(18002099003)(22082099003);DIR:OUT;SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?QFSUKhldjszJ4WvyNjLMPzuz7oiCiGHc0avAaYF6ETzL6QzMTd1FJph/2Dby?= =?us-ascii?Q?RDRmxmxrfyDavspDEO217zFlEXuzaIyLP1Gp1PvEnUbuE0FDPyGsffd7nWYV?= =?us-ascii?Q?dKuxXa+Bc7Tj6p+IL9y4AdaFrAA6JxlVcM3DDw/9pnl7Op1RFZyh3x5sEj+8?= =?us-ascii?Q?YDo6NpPJ6jp1MNzgI6R4JjdEntMPFyg0uW5+dm6QntpaqEl/OQwhKLk56yC4?= =?us-ascii?Q?og0eiXrxt6V/Px3XClkOSP0OrZQ5AhPq4hd0/M2KU4gUHYloo8HYC+s+Wa62?= =?us-ascii?Q?S4Zy2NUGVRVIa96ySDSecC+yB2Qep1uqaDrkpXEASlijkC/obBXNuEotUoeN?= =?us-ascii?Q?S9rUs7MMJgyD5y+MNTQY83wWda/01WPVXTvH58mo8fZShZOx2CrQIgWIYU7n?= =?us-ascii?Q?0/NVap0ByN0n1hrPsVGIFYhENdjkjFa0Pjtj8EnIjfivlndNZ2oVZY7yOPcn?= =?us-ascii?Q?4avwL1sGC8U+8Sm1p5eW4cq/EgfpBMhm7RVzVYQWYvI8maXU8CM/RJYYstlm?= =?us-ascii?Q?i8sqBALEj5iX6wAoogsh7QxWLj0umGmJ1ne8FycEGefOIAhKmPoF5UogwVjW?= =?us-ascii?Q?KuCGZrKzqBx4eA8UyGsNXyBuwgOWLcVgcCn8DnUwaioqf/BcTZvLY3f0gGdy?= =?us-ascii?Q?K9ihngzbQLMpBbahyAQDyeW3IkPlpMpFSrGHvv8WisTLT8l8oPfBoqcQGipw?= =?us-ascii?Q?NHrX/wLCNWQwvInNdWHizHxBF5/BWmZI1COI7TkDNpnPDLBtSKN26S/5QD2o?= =?us-ascii?Q?WC+Ag6B0ajKDfGia4p05R2iu5KuZyQ0E87jFGzKesQ0AGFFVKTm86ay3N7PM?= =?us-ascii?Q?TgP8YU8CPqcLttKG6pM26KH7jn0FCC2S6JyG9i+ThjRRuJvwS0GGysijpg50?= =?us-ascii?Q?Ji2GVBXlKmSFzhNqWluLVKc5c8OdW+esgwqX+GZs3C3L19SvnCsKR9Zxhh38?= =?us-ascii?Q?vNVJcurrIANG/IDPIjavzfeOt+x2q3l1kWf9a88+Kw0SRaP1lXga7LmKwroN?= =?us-ascii?Q?nESHtU6MIR15UcmpwJKYj0NIpErMcPKpSx/ShCtVmVLRaAHyqGuIMqBk5ICn?= =?us-ascii?Q?C0TZgqf9ZX0oeqJ6vu7h2SQc9e92g7EDXhOyppiFBaJiGI5e0fwyc5ee2h3y?= =?us-ascii?Q?hy2BFxBq0s4s1hnLfD1uXn+YZo1GpVpYsdk4HjwVhe+4LTmeCb0jsgP4p1on?= =?us-ascii?Q?Ju6Y1I1y2d8tHY66eD7tHvSpaiYdw8/QZXSqDRQaW0bRvz0DIs+Vf78e3q69?= =?us-ascii?Q?oZqH0JhBjgogQMbp5rHj/zUtsXJLVC54jiSm+Jt5eHMElT0niHWVAjiY7Hxk?= =?us-ascii?Q?oi5y7AjYDeUry+kyaxETbYiOXpWRrGsf54gIf94A6qze4fz0bZVM9SeH7Ver?= =?us-ascii?Q?JcB+WMXjh55ztiNTV1230e8nLsZ/65fGmeg60fKWkf7divCKHA4xiLs7hT0L?= =?us-ascii?Q?KsusTqEICP3ehal2EGHfbvRKni3RWaHyjLPYG8Z3pyGwZHkg6MSIEseiKVIV?= =?us-ascii?Q?eBqAw/H2ZA0317Sh4Pc9xaMPRQeWlW8z+W+WfdVmd2T52zcbHcS7UCPvlz6z?= =?us-ascii?Q?jvkksSqSbQ1Rye+lwSPtbqgkW6oLUUaPNWfnk5bqmZOdJUIGt/Kj5pG5NWmO?= =?us-ascii?Q?Tgy618TtV+PpkJeDPMOOutqNQ1vuPAHhudpqfQvmP2O4/xWe2l5LzugoDYDl?= =?us-ascii?Q?hROy0v/C4sxi9oP0n6r6QHJGBe3mj/kjzEShotyVOAltXPU1tLELFEFpe8Qm?= =?us-ascii?Q?w15HjhrI6B6UVzEYieBsmzLbPUIS+bik94fwFmakkY8bAMj7wCM1?= X-OriginatorOrg: valinux.co.jp X-MS-Exchange-CrossTenant-Network-Message-Id: 8c1e23a2-a510-4099-1420-08def88a6e3b X-MS-Exchange-CrossTenant-AuthSource: TY7P286MB7722.JPNP286.PROD.OUTLOOK.COM X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 12 Aug 2026 15:57:37.2151 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 7a57bee8-f73d-4c5f-a4f7-d72c91c8c111 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: uKyYajCNq4phJyEtdK9wOr+GJt1gvwDalRN2fc7d5+xc3ByTs80ckhQhXm1jCAvdHthcXfqcPziCxn2EmOQJ1g== X-MS-Exchange-Transport-CrossTenantHeadersStamped: OS7P286MB7115 Content-Type: text/plain; charset="utf-8" Choose LL interrupt positions in the common core. Always request one at descriptor ends, at the last free slot, and before each link. Add one every four entries while the rest of the descriptor does not fit in the current free space. Four entries is an empirically chosen coalescing interval. Suggested-by: Frank Li Signed-off-by: Koichiro Den --- Changes in v5: - No changes. drivers/dma/dw-edma/dw-edma-core.c | 25 ++++++++++++++++++++++++- 1 file changed, 24 insertions(+), 1 deletion(-) diff --git a/drivers/dma/dw-edma/dw-edma-core.c b/drivers/dma/dw-edma/dw-ed= ma-core.c index 363993c64c44..13c45549e08b 100644 --- a/drivers/dma/dw-edma/dw-edma-core.c +++ b/drivers/dma/dw-edma/dw-edma-core.c @@ -25,6 +25,9 @@ #include "../dmaengine.h" #include "../virt-dma.h" =20 +/* Empirically chosen progress interval. */ +#define DW_EDMA_LL_PROGRESS_INTERVAL 4 + static inline struct dw_edma_desc *vd2dw_edma_desc(struct virt_dma_desc *vd) { @@ -244,6 +247,26 @@ static bool dw_edma_ll_done_is_stopped(struct dw_edma_= chan *chan) dw_edma_core_ch_transfer_size(chan) =3D=3D 0; } =20 +static bool dw_edma_core_enable_ll_irq(struct dw_edma_desc *desc, u32 i, + u32 free) +{ + struct dw_edma_chan *chan =3D desc->chan; + + /* Always report descriptor ends and the last free slot. */ + if (i =3D=3D desc->nburst - 1 || free =3D=3D 1) + return true; + + /* Keep one progress point per physical ring lap. */ + if (chan->ll_head =3D=3D chan->ll_max - 1) + return true; + + /* Add periodic progress points only while this descriptor does not fit. = */ + if (desc->nburst - i <=3D free) + return false; + + return (chan->ll_head + 1) % DW_EDMA_LL_PROGRESS_INTERVAL =3D=3D 0; +} + static void dw_edma_core_ll_start(struct dw_edma_desc *desc) { struct dw_edma_chan *chan =3D desc->chan; @@ -265,7 +288,7 @@ static void dw_edma_core_ll_start(struct dw_edma_desc *= desc) =20 dw_edma_core_ll_data(chan, &desc->burst[i], chan->ll_head, chan->cb, - i =3D=3D desc->nburst - 1 || free =3D=3D 1); + dw_edma_core_enable_ll_irq(desc, i, free)); =20 chan->ll_head++; =20 --=20 2.51.0 From nobody Tue Sep 29 04:14:12 2026 Received: from TY3P286CU002.outbound.protection.outlook.com (mail-japaneastazon11020119.outbound.protection.outlook.com [52.101.229.119]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B702F47DF97; Wed, 12 Aug 2026 15:57:55 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=52.101.229.119 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1786550280; cv=fail; b=UEIe5NNqIzwDyScovq8/lkNwOc9QpxAD7jJy44o4x4i0/doD5g7xoVK77Nxk5Vw+bEj5S0z2iQy2II5hiKSFXuHhehvRyQHEMVzLyx00DEv/NI4kfGbq3qepUaTImW/GcBgHBK6dFVzqOkPugGjeDXQ0dT6mBgKOXNgAIuHgUo8= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1786550280; c=relaxed/simple; bh=zX7nk+IeidK6kLETDK68ddYIR8mCkOMhDrc6u4+ryJs=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: Content-Type:MIME-Version; b=m05fO9WhdegokMJESVq6jM8IY/o2GmU/dty4xv2O5PmR9stqcbLDYRXj96f+GcCPIdkbBGad8MvZBoOpUIxwiGd42tfr5oibNajRmN0W6HntTXOKI/lW+uDsuc6RSrg7lOMR9fWM4LLlzr9v0Mks2u337o0trRE4yfx+//RmgUI= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=valinux.co.jp; spf=pass smtp.mailfrom=valinux.co.jp; dkim=pass (1024-bit key) header.d=valinux.co.jp header.i=@valinux.co.jp header.b=bBJIn4bc; arc=fail smtp.client-ip=52.101.229.119 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=valinux.co.jp Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=valinux.co.jp Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=valinux.co.jp header.i=@valinux.co.jp header.b="bBJIn4bc" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=IVLgYvtQ/ZeeEIjHLUsfHuAysWG3v6BFiZ59nnN/MTHGxYGtR1+0/CepKu8pTN1036tH0n/tcHzcg11tv0ASDGdLXcFsXcVHV3dYo/ICbS3Wh5GUKX59pN8n6k2NtSIIjOPHTmcNep/FenHlEvxW1hH4/RuhZW6kXKKh+OtTrobo77rB/UGZ+aTBN5URFpfEqgLVcDj8U9ejUi29N+BrTS9I8/8r9LGcqY0UQ8qv1aftAVZBxFJf1Wk0T2f4y8APkWCVu5mPG5xUUrVTz7Dr6ded6iavdrFnDb/9rmCFsw+t1LfL2wWvYFfuEoQHPe4RamgtKyllznc9t80uLFYRsQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=qbNXVZAuGPKJ2cUXFDLR5scwcPmTDGfYCd2H2khzBeo=; b=Ii9APjzxOM8BMq2oGSRH8DC2uyS1YhaPX4F6rBguWW7h45cYSyJ0+ZYWXLV44CqHVURBA6V4giJiWsmtXdiLenOdGyCKszuigkoOvkVDQ5a6YZ0ObTFJgZgjymSHYtf2+LHCgZQEohMO155K1MJKPaIr55Nj0qaJBkXy7wKNkeMG3viYQJOjoyW2Xtej+iZh+63t3HIiZgEun3lsSwg6cuKR9IvOyR2bgg1kHQ1JiyIcHQqa6vc7cPfCc0oH8syWdb2rLhEERry4yzEOoRTCAhn88mWPSvkEfrHGR4tzs4Ag8Ry//3fBmmjxQZs5iiKswMSmM23sJB+QCbLo7RHV9w== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=valinux.co.jp; dmarc=pass action=none header.from=valinux.co.jp; dkim=pass header.d=valinux.co.jp; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=valinux.co.jp; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=qbNXVZAuGPKJ2cUXFDLR5scwcPmTDGfYCd2H2khzBeo=; b=bBJIn4bcwjobugb34+0e/tZ0EuuC0hkrisp+GR4TDCJ1zqFDTpLj2vaOUwbjZ5DOtQYxK1xu011JRnUBq1at9VFp5xWturmMdaDuXHKBSxXlHIzVVlqAvG+aFNWU4Wmq/sm6WvVtTUyhSE0bpQmi/HFrHEztyEQDAIz7tcp0gks= Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=valinux.co.jp; Received: from TY7P286MB7722.JPNP286.PROD.OUTLOOK.COM (2603:1096:405:38f::10) by OS7P286MB7115.JPNP286.PROD.OUTLOOK.COM (2603:1096:604:42f::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.315.14; Wed, 12 Aug 2026 15:57:38 +0000 Received: from TY7P286MB7722.JPNP286.PROD.OUTLOOK.COM ([fe80::2305:327c:28ec:9b32]) by TY7P286MB7722.JPNP286.PROD.OUTLOOK.COM ([fe80::2305:327c:28ec:9b32%3]) with mapi id 15.21.0292.024; Wed, 12 Aug 2026 15:57:38 +0000 From: Koichiro Den To: Vinod Koul , Frank Li , Manivannan Sadhasivam , Gustavo Pimentel , Kees Cook , =?UTF-8?q?Krzysztof=20Wilczy=C5=84ski?= , Kishon Vijay Abraham I , Bjorn Helgaas , Christoph Hellwig , Serge Semin , Cai Huoqing , Niklas Cassel Cc: Devendra K Verma , dmaengine@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH v5 15/24] dmaengine: dw-edma: Recheck stopped LL channels before restart Date: Thu, 13 Aug 2026 00:57:12 +0900 Message-ID: <20260812155721.2807506-16-den@valinux.co.jp> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20260812155721.2807506-1-den@valinux.co.jp> References: <20260812155721.2807506-1-den@valinux.co.jp> Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: TYCP286CA0193.JPNP286.PROD.OUTLOOK.COM (2603:1096:400:382::6) To TY7P286MB7722.JPNP286.PROD.OUTLOOK.COM (2603:1096:405:38f::10) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: TY7P286MB7722:EE_|OS7P286MB7115:EE_ X-MS-Office365-Filtering-Correlation-Id: bf94577d-d565-406a-9d32-08def88a6ea7 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|23010399003|10070799003|7416014|376014|1800799024|366016|921020|6133799003|10067099003|56012099006|18002099003|22082099003; X-Microsoft-Antispam-Message-Info: RK9dHn2xYJWOvZSMzNKCu3iuJJOp0R9HCp95rBchUuM5jh8yuZGnIy5rGv1fsCUPHH+u0OszZgIRA/2JOmGg/BV3mBJAGu07PXqooRMqSywwpL08wRrFd8ngQ92OeDbwSUYu3K01U1WwdE9P/GbDofpY+TZyeCeR29HkUdfTtn5Z4mYLfAACEK+QJHDjkMxDHkQUBYzwi5p+Qq2Ip23IvuN0LxVBkgfJN0IJbB8a2ULNVQVHtEDrzX3NMbKQVKKeKZ/6IXPfo8B8igmyAfmLiX6LR6nzLJOXQXZXnZ2EonMT+KKxRsXou8irVCBzvI8Wem+JU5akeiacDiIOvamoMjjaDu1f/Y286FQrbGz6LTjcgMHOP/eklruNSaoig1/eAKQ2yCAvQ4l5qmVjK2eGB4Syns2o2E0S239oOAdnMRkZLROYi0vnFho5rUm0RamHeCj5M0x7gCM+/er9zMW+NF7j8kQJASAn58G1WvTr0AafJxswnK4BF+yqAVm/4vBcfVKHLYz+nPtKbG+htrG+ym0bLo9lZ8DvqJGCQRswk4ROZxGS4CzuZUMYFih+2VGVo3NnrRJ2jwlgq+Jl+Mq5hbNAGk7/NC13EYVbh9iUqFKWR05qCDKi+vdJWMf2H+493lJxJ+ibAocyQghX0u/TJBSJDRzmLL7pYfeEXjwZiVbcg7OCob3sKFtFJpMUvzsOhElt53QpR0O0qUrVBokF/Q== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:TY7P286MB7722.JPNP286.PROD.OUTLOOK.COM;PTR:;CAT:NONE;SFS:(13230040)(23010399003)(10070799003)(7416014)(376014)(1800799024)(366016)(921020)(6133799003)(10067099003)(56012099006)(18002099003)(22082099003);DIR:OUT;SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?0vYd5Jdeb0AOQsJHka6gttiCKhVbSw6kci5CviWWuFesgRlSA0qHbcaC3DKh?= =?us-ascii?Q?P61E8rx2A4jmWZm62vVr+mqtUpLJ2t7quXknVEkXeq5KFBNczop2KoiQFoXB?= =?us-ascii?Q?2fHUvVxXpFSg+siYS8UmGH41z7BvDMM5G0+S/E/nZaOU2A/qyjYT1Hqyf+Rt?= =?us-ascii?Q?NPu4f5Jc1gQEYw4JAnDdldUfpP9xc5wJMMcnS4sOTA6vO6zgPpDEKQHNZ2tF?= =?us-ascii?Q?DK7fkKpqbqFSfOdnR5sc/rdZE1Hk2kMUk5ndfrwAf7EUuYN1K6LbbAetob6K?= =?us-ascii?Q?fjgoRIX74E3ZOIsueywvDFJsF7ZLuXUaMIVdnf8zBLtK7HT4bJfY9AB91SXa?= =?us-ascii?Q?fG6oSHKFJPBl+9m6C+cGfVCno3bu+2MbyvAdirJJMZfwXoJqV7auwe7eIVps?= =?us-ascii?Q?0l8hBBOX3avvlhaq2hKlkleLd3WCIkJDknhcSlGm06bh5KVTuIL2f/3DiNOQ?= =?us-ascii?Q?4H8lciJtDy5TLu6oi5Yt4T14LQoQYALtXFeNVa6s2qCAm7//MftcNiXtOcQh?= =?us-ascii?Q?XJ+0fHIioBvE1PpDW/nxqJUuJxdniOxY6uZESJuoeawsjuxyaywR3PKqdj6Z?= =?us-ascii?Q?9EHyECoKENpKNzivvjv0T62yqmsSxvIbpjjtR/gilgNIUqSba+pkhLpT6CZI?= =?us-ascii?Q?keanW7LwOV20mu79S9n3YHF1UqsghWlmIiz0DTASb+vGu4eLrjI2RTS6RDN1?= =?us-ascii?Q?DDxHoSpi0HGSSiFhJskLDEwvtTIYdyw3RJ2gsauhZOeBy/Q7H8SVpl9jketk?= =?us-ascii?Q?sKBBya4w255XH+5OOJerrH17zB/9N4LOCMsp0Bv/61eW5RDqhxJydO7lWoMT?= =?us-ascii?Q?BJjLCOJuP68Ascn9PzVXMWBNfQtgWpNMd6TiG/MmsYp2Yq6lHr96R7KGZGd6?= =?us-ascii?Q?1d28h3n6ef2ggfwCVmJKpoNX7geMp4Ke7zdyVKOgqzXE5FcLzQQ8ETk9oXVB?= =?us-ascii?Q?7We4T9wVt1E3bwMOzkfWDWtzhvRqGg+2OpWL9Y4rugKSTq7anRf8Yj2+QkhB?= =?us-ascii?Q?AWUgks8CF7bnAIE1o07CHRqEdz68t3S6hdrkU4N1LbrgqXCPNAdo4xdTrhMk?= =?us-ascii?Q?cSd9Vjpb0tZ8vyx/jXxB1URM54WjiD0hG2jizVnxybty39pMP8PcyjCUuexe?= =?us-ascii?Q?xpC2W87tWIL5P98Xr/osd3Wi0z2o9lbOTD3S4cJi6XuPfCTPbkIE5jGdiyHC?= =?us-ascii?Q?heGuFb+nEmDS8M3CuRM7uzfftcaj/qmSMP4oMtYK8XS1oac2MKxET3uyfhDD?= =?us-ascii?Q?TvgkAY8JMgCPYFnj6oJyff2I1kzOk7sMajC8TIzl8RisLzOzRu0U1KFk4MKN?= =?us-ascii?Q?Genc6drdy7U9rtOXJ9DSiIxTq/V8UUygOXtzBmNJULLL9Bc+4T1ZoQy7z79d?= =?us-ascii?Q?wvjPZ0NksqDzgZ8LcSUrkZxqaJ2u2Jfbq8mqINA5ARvSda35zdZssvsx7l+5?= =?us-ascii?Q?bb6OQ6NGoyiovqx9FZhXdBM7YeBCcjkohW470EQOPc+wpUUsZnMprq0XAqcm?= =?us-ascii?Q?5OFLGKbWaFWjSKkAkyxgtdcIHbIAgKsm0ixxjGUNR8Ri53xL7OW0G5rPTFVg?= =?us-ascii?Q?0x2LzSPRilUHnEwtMzQ2DXlirazk6Ic2dINFEiCSCVY/Zar4+/ni5KEpBohJ?= =?us-ascii?Q?DxvB6yCJrCqyoEUUQGNeBYISFiy8VyVtGJY5oNP0+N6WreTl6c5J85q+nJMF?= =?us-ascii?Q?3OdsyPoXPxx/SL+0B9J761ntKh8jXWDaKtFB1t9ezKQ0zT4+TsBAIXNn7U7c?= =?us-ascii?Q?xhpYn4lBzxkHDfIyyIjMvRQnJw6ZFpbSA5CtiDg6lm0vKpvngzZ4?= X-OriginatorOrg: valinux.co.jp X-MS-Exchange-CrossTenant-Network-Message-Id: bf94577d-d565-406a-9d32-08def88a6ea7 X-MS-Exchange-CrossTenant-AuthSource: TY7P286MB7722.JPNP286.PROD.OUTLOOK.COM X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 12 Aug 2026 15:57:37.9908 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 7a57bee8-f73d-4c5f-a4f7-d72c91c8c111 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: KfpCxl4vDMaM+UMdrQaMsE1Q6XoGiVmG4WqiXhXb2cJK/fU0UH81R5jNWYaybbHpCyLTtzi9XExriAtPXb6YVA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: OS7P286MB7115 Content-Type: text/plain; charset="utf-8" A DMA_LLP sample taken for a DONE interrupt through the eDMA-compatible interface can race with the final DMA_LLP update and leave the recorded consumer boundary one entry short. If the channel stops there, no later interrupt reports the missing progress. Before restarting a stopped LL channel, sample DMA_LLP again and reclaim any newly visible progress. If the channel still reports running, do not ring another doorbell. It may stop at a CB mismatch just after the status read, so schedule one delayed recheck. Also recheck after actually kicking a stopped channel, in case no event reports the result of the kick. Native HDMA reports STOP separately and does not need this recheck. For eDMA, permit the initial kick after an LL reset, then only one restart after each software-confirmed STOP boundary. Consume the permission with the doorbell under the event lock. This prevents a newly asserted STOP from being cleared before the hard IRQ records it. Serialize the stopped-state check, LLP sample and interrupt-status clear against IRQ capture. Do not reconcile a stopped boundary while ABORT is pending. Keep stopped-boundary reconciliation separate from refilling so STOP and PAUSE drain paths can consume it without publishing more entries. Signed-off-by: Koichiro Den --- Changes in v5: - Read jiffies once for both the recheck-deadline comparison and the remaining-delay calculation. (Sashiko) - Gate eDMA kicks with a one-shot credit armed by LL reset or a software-confirmed STOP, preventing a later kick from clearing an unrecorded STOP status. (Sashiko) drivers/dma/dw-edma/dw-edma-core.c | 225 +++++++++++++++++++++++++++-- drivers/dma/dw-edma/dw-edma-core.h | 7 + 2 files changed, 220 insertions(+), 12 deletions(-) diff --git a/drivers/dma/dw-edma/dw-edma-core.c b/drivers/dma/dw-edma/dw-ed= ma-core.c index 13c45549e08b..8325e296cb2a 100644 --- a/drivers/dma/dw-edma/dw-edma-core.c +++ b/drivers/dma/dw-edma/dw-edma-core.c @@ -27,6 +27,7 @@ =20 /* Empirically chosen progress interval. */ #define DW_EDMA_LL_PROGRESS_INTERVAL 4 +#define DW_EDMA_LL_RECHECK_DELAY_MS 30 =20 static inline struct dw_edma_desc *vd2dw_edma_desc(struct virt_dma_desc *vd) @@ -112,6 +113,12 @@ static bool dw_edma_abort_is_pending(struct dw_edma_ch= an *chan) return chan->abort_pending; } =20 +static bool dw_edma_ll_uses_restart_credit(struct dw_edma_chan *chan) +{ + return chan->dw->chip->mf =3D=3D EDMA_MF_EDMA_LEGACY || + chan->dw->chip->mf =3D=3D EDMA_MF_EDMA_UNROLL; +} + /* Must be called with vc.lock held. */ static bool dw_edma_ll_snapshot_take(struct dw_edma_chan *chan, @@ -148,10 +155,33 @@ static void dw_edma_ll_event_discard_locked(struct dw= _edma_chan *chan) dw_edma_ll_snapshot_discard_locked(chan); } =20 +/* Must be called with vc.lock held. */ +static void dw_edma_ll_recheck_cancel(struct dw_edma_chan *chan) +{ + chan->ll_recheck_at =3D 0; + cancel_delayed_work(&chan->ll_recheck_work); +} + +/* Must be called with vc.lock held. */ +static void dw_edma_ll_recheck_schedule(struct dw_edma_chan *chan) +{ + unsigned long delay =3D + msecs_to_jiffies(DW_EDMA_LL_RECHECK_DELAY_MS); + + if (unlikely(READ_ONCE(chan->dw->teardown))) + return; + + /* Keep zero reserved for a cancelled recheck across jiffies wrap. */ + chan->ll_recheck_at =3D jiffies + delay ?: 1; + mod_delayed_work(chan->dw->wq, &chan->ll_recheck_work, delay); +} + /* Must be called with vc.lock held. */ static void dw_edma_set_request(struct dw_edma_chan *chan, enum dw_edma_request reques= t) { + if (!chan->non_ll && chan->request !=3D request) + dw_edma_ll_recheck_cancel(chan); chan->request =3D request; } =20 @@ -184,6 +214,8 @@ static void dw_edma_core_reset_ll(struct dw_edma_chan *= chan) { u32 i; =20 + dw_edma_ll_recheck_cancel(chan); + chan->ll_head =3D 0; chan->ll_done =3D 0; /* Drop stale CB bits before reusing the circular LL ring. */ @@ -195,6 +227,8 @@ static void dw_edma_core_reset_ll(struct dw_edma_chan *= chan) chan->ll_region.paddr); =20 scoped_guard(spinlock_irqsave, dw_edma_event_lock(chan)) { + if (dw_edma_ll_uses_restart_credit(chan)) + chan->ll_restart_armed =3D true; dw_edma_ll_event_discard_locked(chan); } dw_edma_core_ch_enable(chan); @@ -425,7 +459,7 @@ static bool dw_edma_ll_consume_progress(struct dw_edma_= chan *chan, int idx) =20 if (WARN_ON_ONCE(desc->done_burst > desc->start_burst || desc->start_burst > desc->nburst)) - return advanced; + goto out; =20 /* * start_burst is the next burst to append. done_burst counts @@ -444,7 +478,7 @@ static bool dw_edma_ll_consume_progress(struct dw_edma_= chan *chan, int idx) gap =3D dw_edma_core_get_ll_dist(chan, chan->ll_done, desc->ll_start); if (gap > done) - return advanced; + goto out; =20 chan->ll_done =3D desc->ll_start; done -=3D gap; @@ -475,6 +509,10 @@ static bool dw_edma_ll_consume_progress(struct dw_edma= _chan *chan, int idx) =20 WARN_ON_ONCE(done); =20 +out: + if (advanced) + dw_edma_ll_recheck_cancel(chan); + return advanced; } =20 @@ -518,6 +556,61 @@ dw_edma_ll_recycle_idx(struct dw_edma_chan *chan, int = idx, return idx =3D=3D chan->ll_max ? chan->ll_max - 1 : idx - 1; } =20 +static bool dw_edma_ll_has_hdma_stop_event(struct dw_edma_chan *chan) +{ + return chan->dw->chip->mf =3D=3D EDMA_MF_HDMA_NATIVE; +} + +/* + * Must be called with vc.lock held. A DONE-time DMA_LLP sample may miss + * the final burst element. For the eDMA-compatible interrupt interface, + * accept a fresh LLP sample only when status is STOPPED and transfer size + * is zero. Native HDMA reports STOP directly. + */ +static bool dw_edma_ll_reconcile_stopped(struct dw_edma_chan *chan) +{ + int idx; + + scoped_guard(spinlock_irqsave, dw_edma_event_lock(chan)) { + if (dw_edma_abort_latch_locked(chan)) + return false; + + /* + * Leave an IRQ-captured event to its worker. Otherwise pair the + * stopped boundary with the status clear before another kick. + */ + if (chan->ll_irq.event !=3D DW_EDMA_LL_EVENT_NONE || + dw_edma_core_ch_status(chan) !=3D DMA_COMPLETE) + return false; + + /* Native HDMA reports STOP without a transfer-size check. */ + if (!dw_edma_ll_has_hdma_stop_event(chan) && + dw_edma_core_ch_transfer_size(chan) !=3D 0) + return false; + + idx =3D dw_edma_ll_recycle_idx(chan, + dw_edma_core_ll_cur_idx(chan), + DW_EDMA_LL_EVENT_STOP); + dw_edma_core_ll_irq_clear(chan); + if (dw_edma_ll_uses_restart_credit(chan)) + chan->ll_restart_armed =3D true; + } + + return dw_edma_ll_consume_progress(chan, idx); +} + +static bool dw_edma_ll_reconcile_and_refill(struct dw_edma_chan *chan) +{ + if (!dw_edma_ll_reconcile_stopped(chan)) + return false; + + dw_edma_start_transfer(chan); + chan->status =3D dw_edma_ll_pending(chan) ? + EDMA_ST_BUSY : EDMA_ST_IDLE; + + return true; +} + static void dw_edma_core_ll_sync(struct dw_edma_chan *chan) { /* @@ -531,6 +624,15 @@ static void dw_edma_core_ll_sync(struct dw_edma_chan *= chan) /* Must be called with vc.lock held for an LL channel. */ static void dw_edma_core_ch_doorbell(struct dw_edma_chan *chan) { + if (unlikely(READ_ONCE(chan->dw->teardown))) + return; + + dw_edma_ll_recheck_cancel(chan); + + /* + * Complete the remote LL publication before serializing the new + * hardware run with IRQ capture. + */ dw_edma_core_ll_sync(chan); =20 guard(spinlock_irqsave)(dw_edma_event_lock(chan)); @@ -545,18 +647,85 @@ static void dw_edma_core_ch_doorbell(struct dw_edma_c= han *chan) if (chan->ll_irq.event !=3D DW_EDMA_LL_EVENT_NONE) return; =20 + /* One confirmed stop permits one eDMA restart. */ + if (dw_edma_ll_uses_restart_credit(chan)) { + if (!chan->ll_restart_armed) + return; + chan->ll_restart_armed =3D false; + } + dw_edma_ll_event_discard_locked(chan); dw_edma_core_do_ch_doorbell(chan); } =20 -/* Must be called with vc.lock held. */ -static void dw_edma_core_ch_maybe_doorbell(struct dw_edma_chan *chan) +/* + * Must be called with vc.lock held. Return true when published work is st= ill + * running and may need one later stop recheck. + */ +static bool dw_edma_core_ch_maybe_doorbell(struct dw_edma_chan *chan) { if (chan->non_ll || chan->request !=3D EDMA_REQ_NONE || - chan->status !=3D EDMA_ST_BUSY || !dw_edma_ll_pending(chan)) - return; + chan->status !=3D EDMA_ST_BUSY || !dw_edma_ll_pending(chan) || + dw_edma_abort_is_pending(chan)) + return false; + + /* + * While running, both eDMA and HDMA consume newly published + * elements without another doorbell. + */ + if (dw_edma_core_ch_status(chan) =3D=3D DMA_IN_PROGRESS) + return true; + + dw_edma_ll_reconcile_and_refill(chan); + if (!dw_edma_ll_pending(chan)) + return false; =20 dw_edma_core_ch_doorbell(chan); + if (!dw_edma_ll_has_hdma_stop_event(chan)) + dw_edma_ll_recheck_schedule(chan); + + return false; +} + +/* + * eDMA may stop at a CB mismatch just after reporting RUNNING. + * Recheck once so the stopped tail is not left pending. + */ +static void +dw_edma_core_ch_maybe_doorbell_or_recheck(struct dw_edma_chan *chan) +{ + if (dw_edma_core_ch_maybe_doorbell(chan) && + !dw_edma_ll_has_hdma_stop_event(chan)) + dw_edma_ll_recheck_schedule(chan); +} + +static void dw_edma_ll_recheck_work(struct work_struct *work) +{ + struct dw_edma_chan *chan =3D + container_of(to_delayed_work(work), struct dw_edma_chan, + ll_recheck_work); + unsigned long delay, now; + + guard(spinlock_irqsave)(&chan->vc.lock); + + if (unlikely(READ_ONCE(chan->dw->teardown))) { + chan->ll_recheck_at =3D 0; + return; + } + + if (!chan->ll_recheck_at) + return; + + now =3D jiffies; + if (time_before(now, chan->ll_recheck_at)) { + delay =3D chan->ll_recheck_at - now; + mod_delayed_work(chan->dw->wq, &chan->ll_recheck_work, delay); + return; + } + + chan->ll_recheck_at =3D 0; + if (chan->request =3D=3D EDMA_REQ_NONE) + dw_edma_core_ch_maybe_doorbell(chan); } =20 static void dw_edma_device_caps(struct dma_chan *dchan, @@ -684,7 +853,7 @@ static int dw_edma_device_resume(struct dma_chan *dchan) chan->status =3D EDMA_ST_BUSY; if (!dw_edma_start_transfer(chan)) chan->status =3D EDMA_ST_IDLE; - dw_edma_core_ch_maybe_doorbell(chan); + dw_edma_core_ch_maybe_doorbell_or_recheck(chan); } =20 return err; @@ -734,7 +903,7 @@ static void dw_edma_device_issue_pending(struct dma_cha= n *dchan) dw_edma_ll_snapshot_discard(chan); chan->status =3D EDMA_ST_BUSY; dw_edma_start_transfer(chan); - dw_edma_core_ch_maybe_doorbell(chan); + dw_edma_core_ch_maybe_doorbell_or_recheck(chan); } spin_unlock_irqrestore(&chan->vc.lock, flags); } @@ -1084,7 +1253,7 @@ static void dw_edma_ll_interrupt(struct dw_edma_chan = *chan) } =20 out: - dw_edma_core_ch_maybe_doorbell(chan); + dw_edma_core_ch_maybe_doorbell_or_recheck(chan); } =20 static bool dw_edma_abort_interrupt(struct dw_edma_chan *chan) @@ -1123,6 +1292,11 @@ static void dw_edma_irq_work(struct work_struct *wor= k) irq_work); unsigned int events; =20 + if (unlikely(READ_ONCE(chan->dw->teardown))) { + atomic_set(&chan->irq_pending, 0); + return; + } + do { events =3D atomic_xchg(&chan->irq_pending, 0); =20 @@ -1140,6 +1314,9 @@ static void dw_edma_irq_work(struct work_struct *work) static void dw_edma_queue_irq_work(struct dw_edma_chan *chan, unsigned int events) { + if (unlikely(READ_ONCE(chan->dw->teardown))) + return; + atomic_or(events, &chan->irq_pending); queue_work(chan->dw->wq, &chan->irq_work); } @@ -1189,6 +1366,9 @@ static void dw_edma_record_irq(struct dw_edma_chan *c= han, unsigned int events) if (chan->ll_irq.event !=3D DW_EDMA_LL_EVENT_STOP || snapshot.event =3D=3D DW_EDMA_LL_EVENT_STOP) chan->ll_irq =3D snapshot; + if (dw_edma_ll_uses_restart_credit(chan)) + chan->ll_restart_armed =3D + chan->ll_irq.event =3D=3D DW_EDMA_LL_EVENT_STOP; pending |=3D DW_EDMA_DEFERRED_LL; } =20 @@ -1378,6 +1558,9 @@ static void dw_edma_device_synchronize(struct dma_cha= n *dchan) struct dw_edma_chan *chan =3D dchan2dw_edma_chan(dchan); =20 dw_edma_wait_termination(dchan); + scoped_guard(spinlock_irqsave, &chan->vc.lock) + dw_edma_ll_recheck_cancel(chan); + cancel_delayed_work_sync(&chan->ll_recheck_work); cancel_work_sync(&chan->irq_work); atomic_set(&chan->irq_pending, 0); dw_edma_irq_events_discard(chan); @@ -1431,6 +1614,8 @@ static int dw_edma_channel_setup(struct dw_edma *dw, = u32 wr_alloc, u32 rd_alloc) chan->status =3D EDMA_ST_IDLE; chan->irq_mode =3D dw_edma_get_default_irq_mode(chan); INIT_WORK(&chan->irq_work, dw_edma_irq_work); + INIT_DELAYED_WORK(&chan->ll_recheck_work, + dw_edma_ll_recheck_work); atomic_set(&chan->irq_pending, 0); chan->ll_irq.idx =3D -1; chan->ll_irq.event =3D DW_EDMA_LL_EVENT_NONE; @@ -1786,23 +1971,39 @@ int dw_edma_remove(struct dw_edma_chip *chip) if (!dw) return -ENODEV; =20 + /* + * Stop new clients and asynchronous hardware access before dismantling + * their execution context. + */ + WRITE_ONCE(dw->teardown, true); + dma_async_device_unregister(&dw->dma); + + /* + * Drain channel work that may have passed the teardown gate before + * stopping the hardware. IRQ handlers remain installed while it is + * active. + */ + for (i =3D 0; i < dw->wr_ch_cnt + dw->rd_ch_cnt; i++) { + disable_delayed_work_sync(&dw->chan[i].ll_recheck_work); + cancel_work_sync(&dw->chan[i].irq_work); + } + if (chip->flags & DW_EDMA_CHIP_PARTIAL) err =3D dw_edma_core_quiesce(dw); else dw_edma_core_off(dw); =20 - /* Free irqs */ + /* No new hardware event can be raised after the quiesce. */ for (i =3D (dw->nr_irqs - 1); i >=3D 0; i--) free_irq(chip->ops->irq_vector(dev, i), &dw->irq[i]); dw_edma_emul_irq_free(dw); =20 + /* Drain IRQ work queued by a handler that raced with the gate. */ for (i =3D 0; i < dw->wr_ch_cnt + dw->rd_ch_cnt; i++) cancel_work_sync(&dw->chan[i].irq_work); =20 destroy_workqueue(dw->wq); =20 - /* Deregister eDMA device */ - dma_async_device_unregister(&dw->dma); list_for_each_entry_safe(chan, _chan, &dw->dma.channels, vc.chan.device_node) { tasklet_kill(&chan->vc.task); diff --git a/drivers/dma/dw-edma/dw-edma-core.h b/drivers/dma/dw-edma/dw-ed= ma-core.h index 27cab6ca5e67..aaaee312e973 100644 --- a/drivers/dma/dw-edma/dw-edma-core.h +++ b/drivers/dma/dw-edma/dw-edma-core.h @@ -120,10 +120,15 @@ struct dw_edma_chan { struct dw_edma_ll_snapshot ll_irq; /* ABORT is terminal and remains pending across LL state changes. */ bool abort_pending; + /* One-shot eDMA restart credit, protected by event_lock. */ + bool ll_restart_armed; /* Native HDMA lock storage. */ spinlock_t event_lock_per_chan; spinlock_t *event_lock; /* Selected event lock */ =20 + struct delayed_work ll_recheck_work; + unsigned long ll_recheck_at; + u32 ll_max; /* Data entries */ struct dw_edma_region ll_region; /* Linked list */ bool ll_valid; /* LL context programmed */ @@ -172,6 +177,8 @@ struct dw_edma { */ struct workqueue_struct *wq; =20 + bool teardown; /* Gate asynchronous hardware access */ + raw_spinlock_t lock; /* Protect v0 shared registers */ /* Per-direction lock storage for the eDMA interrupt registers. */ spinlock_t event_lock_per_dir[2]; --=20 2.51.0 From nobody Tue Sep 29 04:14:12 2026 Received: from TY3P286CU002.outbound.protection.outlook.com (mail-japaneastazon11020119.outbound.protection.outlook.com [52.101.229.119]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9E3BB480976; Wed, 12 Aug 2026 15:58:00 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=52.101.229.119 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1786550284; cv=fail; b=tNzFkwp6IUU2zVRy2z3SxcBuDsBCdWj/t2mPr7vNRNkpm0bbrknxMOb2zsSDboykP6Y/xPyUQ5gLkgWB5FnUUtyr2g2vak/PMAl0Aa1Q825veECd2lQXQt3A4HmXEoBM/rPzyhYxXn5sqsxi3SelB1daJ0WoP1fpZHuySsSwVoU= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1786550284; c=relaxed/simple; bh=3/8nBuTskyn1uFh7CbZbYf8rDiWJ+TKA+9DnZzsIlUc=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: Content-Type:MIME-Version; b=kaTSSVEpZdWyhha1Qs483GITs9/0vAy73f7Btix4L57XeVrUFAmTB08aXgV5lB1WtAQIw0h7TIC1FqM/lINPcCq2ArW29FV4TYkrYFw2p1RsUscbdOiEVBgUNsWb1k7Q4B336Pbv3RbAuPUpiZkvkTZF769b216BmCydds0H6Vw= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=valinux.co.jp; spf=pass smtp.mailfrom=valinux.co.jp; dkim=pass (1024-bit key) header.d=valinux.co.jp header.i=@valinux.co.jp header.b=EiIOqs0e; arc=fail smtp.client-ip=52.101.229.119 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=valinux.co.jp Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=valinux.co.jp Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=valinux.co.jp header.i=@valinux.co.jp header.b="EiIOqs0e" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=tQuMGNe1xIPbPYBph/vw8w57t4lT7AL9/GxlsHSd5H6xwgtBK5Km/5Qg1aL/FCp7VMMTWtWo1lLaFUJelNrD9ZeAQqUf9Wx6oDsPUj1iPGONYoBSJrJykpjG1Xxi7KozLUnGAV434e/xaKbt4evbvkUGvx/ZH2MmfrNOzdDKHVntZVUPtGZzCLqgKh7bSOqgItzxd3XG35CUgBJLMTDx1OdmRA3Ro6azRRvJQt42tHxLGNAxols30pisZfNg44kYOjR0qTk3oX5Trw/hBlVxOBW28EddOB7AbEShpx3cpFziIg72Y9DZ/ijEEZAyOmpnpHXQljtHNCn794B5bAAvIQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=3a+b6dMN97tY03hzxFgfzyK4jxgwlD2QJDDcKom7a7c=; b=m02ygwjMnT1VO9Lv8J4if/FGgjbEgyl4bU2j69banvVk320GE5Af8RMI16RFDY6NYxKWtyB5EJvGSJ2rs5/H35cQ8Cru4ft97W9heRdEvP1nIi2P9kAEIlghMd1Sszj9RjCMecis/PNEPXQnmYRZK9934ybcO2qKt3lsaJlWsm1cdtIlSV3i5vBabFpteEnDME5rOBC+ik1pNfaOeDtDVaKwSegG4JF+js9OOEIN+itJDjec8ElxM9QZDv7EGHut5JojeaC73c40MiT7bfK4unxfrGfNec/PhD2iaTkF0JojIS+jwEAj/+BGj2Gz1LdEfLfSuhGGSuEzHTtzKsmozA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=valinux.co.jp; dmarc=pass action=none header.from=valinux.co.jp; dkim=pass header.d=valinux.co.jp; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=valinux.co.jp; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=3a+b6dMN97tY03hzxFgfzyK4jxgwlD2QJDDcKom7a7c=; b=EiIOqs0eXBDiKhDR7ygebTdDW+Rk8idqSHe6B27Mh9lnM25kvminA6y5dhzq1j6exAsAau4mepg0HMyRRXdv+uy+oVLHHFaSEZCQMVeiIUwe6CXZWJNSEQxOLdk89Me2ah7KRTbme5yVS0FXbvfrjpIeH2k+5ysXJlJKMNDGTEA= Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=valinux.co.jp; Received: from TY7P286MB7722.JPNP286.PROD.OUTLOOK.COM (2603:1096:405:38f::10) by OS7P286MB7115.JPNP286.PROD.OUTLOOK.COM (2603:1096:604:42f::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.315.14; Wed, 12 Aug 2026 15:57:38 +0000 Received: from TY7P286MB7722.JPNP286.PROD.OUTLOOK.COM ([fe80::2305:327c:28ec:9b32]) by TY7P286MB7722.JPNP286.PROD.OUTLOOK.COM ([fe80::2305:327c:28ec:9b32%3]) with mapi id 15.21.0292.024; Wed, 12 Aug 2026 15:57:38 +0000 From: Koichiro Den To: Vinod Koul , Frank Li , Manivannan Sadhasivam , Gustavo Pimentel , Kees Cook , =?UTF-8?q?Krzysztof=20Wilczy=C5=84ski?= , Kishon Vijay Abraham I , Bjorn Helgaas , Christoph Hellwig , Serge Semin , Cai Huoqing , Niklas Cassel Cc: Devendra K Verma , dmaengine@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH v5 16/24] dmaengine: dw-edma: Use HDMA watermarks as progress events Date: Thu, 13 Aug 2026 00:57:13 +0900 Message-ID: <20260812155721.2807506-17-den@valinux.co.jp> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20260812155721.2807506-1-den@valinux.co.jp> References: <20260812155721.2807506-1-den@valinux.co.jp> Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: TYWP286CA0019.JPNP286.PROD.OUTLOOK.COM (2603:1096:400:262::6) To TY7P286MB7722.JPNP286.PROD.OUTLOOK.COM (2603:1096:405:38f::10) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: TY7P286MB7722:EE_|OS7P286MB7115:EE_ X-MS-Office365-Filtering-Correlation-Id: 7c9fb254-87a9-4d59-aca7-08def88a6f25 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|23010399003|10070799003|7416014|376014|1800799024|366016|921020|6133799003|10067099003|56012099006|18002099003|22082099003; X-Microsoft-Antispam-Message-Info: c1GaXt+eunww7npqQq5qZRHpa8TkIrwFMMnoIp0dkkQUBftYr/DduY7PjEAAABOoSjqTeNK5baX2rORR/GLOmX9zKryv+767NA57Lb9nY5jvkZlXLPnXtf2HdVu4mIt3JEhc89jfa5dB5uKXQUz350IEFHz6/ofznNllPTrsrPBGZqt9oCv6gsw/RNCZ7SI+0SFrIH1zMEkkWOQP4W9ePcGxahRFziea5wxBrOp4R9vlEpStd36RsVrXmYKQNOW7PTWo2S+iRBaNYFlFZnoZDUNg12zXFKF7H16CJKaflc3mhY1bT0II4nGk5cJAw9emi7WSFS8gLg3Mx2OZEQGcu6aIgVA6LudA+nmRdgjxBNVWdSLcDNYHkoUPQBQ4utD/Cv2Ub1ugPF/IrLxj93vDaTK6QfHHW6aM1u+221J2HJ293xclBP3Zn+hR2KLK/KXu+7r1eoxzWZJVBDjg7uLvVWhNFORwpaI+doNFA3Y2gGybQWhFLqxP8FgxuhhaoNjAsbOgUCO5z/Ngo7Mji8/+n7XzBkVCd0LP1b6zNLIKqCcsEi85rt62O3tNfKSazAXiw0JzmBT9CMtAex/yCeDAc1dd/n54wKzuPuv6pE2FjJkZBobtFgOWnz+Iz3K5OaXKXOQpo7z1tv5EtyPPpsOJd7uJA1BnFjJGIGD+9Q3T0VyaovFTb7yIlX7AicUrAo4+EibMgD6Gr/PoVWPY+IlHCQ== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:TY7P286MB7722.JPNP286.PROD.OUTLOOK.COM;PTR:;CAT:NONE;SFS:(13230040)(23010399003)(10070799003)(7416014)(376014)(1800799024)(366016)(921020)(6133799003)(10067099003)(56012099006)(18002099003)(22082099003);DIR:OUT;SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?BUx7u2GUTZJl06MyWt7GpwJYObfDIP/WGmYKD4CZCYLh+N/wwl7gsfM+r/1d?= =?us-ascii?Q?zJ3o257kCO9cIkWILMyUeXx3FuES/XfCIA5hMwq99bIoUhkz7KlytpyyC+Yo?= =?us-ascii?Q?wU2qAT5U198mYoH4GXXg1ONDQ0zKDzdhGiYsNEMTorF1QQ7VtPuTO+CHz6oy?= =?us-ascii?Q?iVRD+qb8zTO9ZNdlueKp6frOh9pCp/CzC+AwR/kNj4EBKkRd707UU5/A9qxv?= =?us-ascii?Q?raPMsbleQo9ftErwHS6NtzxG3QCf2kkqzoT7XjstnHMGb0pdidtFje1z/Rwd?= =?us-ascii?Q?GqtBvYsqy/DeUi+j/D1szjOxGHWrrjD4vXiJEnS4K88m+ODeqrCvVn7A88G4?= =?us-ascii?Q?wDuPXNq9IJJpOmr3cAaLbHxMVtfbCBtaiaZHXnVGBIuW+qXk10qHnWX1bsF8?= =?us-ascii?Q?21eYXTlTLMZiLkKf+NHPUN2iQpPU4gGonvWGW1rbpdcEdfNelgrXe5uUXdbl?= =?us-ascii?Q?oWpcO6Sbsqg8ewQV50ZLxM+n23u171VaS6OC/f4GCsszTRRSKgy149XubXrO?= =?us-ascii?Q?NjnSe+HSYEJa1znyIGcUzBO6vdNljNgFzn+VOe5Ctm2i7QZhkyOoJE24QKoP?= =?us-ascii?Q?bSjsPNkFlGakUC9c6c1vSjM/b+u1XXScWQMIuVafaG6YaHOs/TzrsxwRaAPS?= =?us-ascii?Q?vAVXZl7D/z8OcVAJZylXQEEzp5f87tD5y8ey6SWa7120uUwvAGauve7kRYng?= =?us-ascii?Q?VBa8BgQk1Q8GcDjBcc1rj1v9hAVapbB5xhY6LGkozcM3BWcJ1eutUz2oLWnY?= =?us-ascii?Q?nAbyujrfoY8peAWJIsFwYXd1YKoBOOgOS0cymd8ANyDRi+xFpJUvluXC5LhP?= =?us-ascii?Q?ltR1Oa8dBkPp+NAfIK/bHrmIFRofZT1tzyqiKBqY+Sfk0WFPNteet/VinuHX?= =?us-ascii?Q?zTCqr6165IjEZ8h8huonGy/LHWV3wVi+yqtCNGrHecFqYWXCVsZwUZ0NiKgb?= =?us-ascii?Q?gRTGugyVOUvphKSJC90w9x560NDM0Y4AQXwAbw+tM7aB4Rr/k3mINNjhQplv?= =?us-ascii?Q?fBk9YjMA4skADPaeowzSS8PJb7Cy5N34BRhkV+X5pmeQmAVNw6axSxj74gQW?= =?us-ascii?Q?Opcfd4NzxCFV/ORmHqE2W4KnM/RDPJ0nZLoOshC8FmBBHWoHhEanmvt1M/ao?= =?us-ascii?Q?oF9v9x8zhcEe1CNyFUsjOE2YgQobeX1c/Ud2qlKxGizUfN8L0H/4v2+WXPdT?= =?us-ascii?Q?/IDyWx7CLw6+VEoR+GRXx9HDWc/921R8gRmP6b/k0xU8dNcZd+e1E43EGqY4?= =?us-ascii?Q?zoGZ0lguRUbHFPTANp385H50f71a3b+OfUps4xlVV0LO/CTVK7i4RHitvx3D?= =?us-ascii?Q?8uSmZ+jAk/Oskjqxyrx35o7OF2kCLYsvlz8LWqbTUY64tvR5tUu50oPV5H2c?= =?us-ascii?Q?uBFmDxx+hWCuQ7ztZvjKBXeblnIYzkTE8qCIWGq4sqBcbdDPXtEOjVk/Syit?= =?us-ascii?Q?77dQxNGljemiNJGyiMtyQsNyscSPUVreepILESBjPGSVeS8LMOTw08K7/4nq?= =?us-ascii?Q?qgsVryGmgLjtAsrSiTVSZBkqgVk21bQLaD1GPORC2iHXMCYAc7dp3UbzId/N?= =?us-ascii?Q?aQ9cAsCx323iwjo2h9AV1dRZkkimvMpYmt0NwlcxCSBvJZeSzWkiGjtFjAP6?= =?us-ascii?Q?nCny3T6fDERKPCoxykl9Yj++uZopRxBeGkG+5/8vYaNNAsrrrxRzdQ/tDLn+?= =?us-ascii?Q?+EfW0rWX8Pnf23jGWKMcLoXrfMMOnFzmMJ46/8agQzR5CVWhkaO8w2Ml5wjW?= =?us-ascii?Q?8OILkHp3igHoRUlV6zVenwlmZr37B5WKCkKMcxjAaphEJU2eL7QJ?= X-OriginatorOrg: valinux.co.jp X-MS-Exchange-CrossTenant-Network-Message-Id: 7c9fb254-87a9-4d59-aca7-08def88a6f25 X-MS-Exchange-CrossTenant-AuthSource: TY7P286MB7722.JPNP286.PROD.OUTLOOK.COM X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 12 Aug 2026 15:57:38.7591 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 7a57bee8-f73d-4c5f-a4f7-d72c91c8c111 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: adqfnbboC+RkgITv6Wq7jkf3XIw+Huh/Dp/y6z5aeIptlVLZ6vExFxwWnWu3V8ay/Zl/XalPbt6dHPxLg6O90A== X-MS-Exchange-Transport-CrossTenantHeadersStamped: OS7P286MB7115 Content-Type: text/plain; charset="utf-8" HDMA reports running LL progress through WATERMARK and channel stop through STOP. Wire watermark setup, clearing, MSI programming, and LWIE/RWIE into the core interrupt path. Record WATERMARK as progress; STOP remains a stopped event. DWC PCIe Controller Databook 6.10a, Section 7.2.1, permits recycling LLEs through the watermark LLP. Testing on a 6.30a integration found that boundary unsafe for DMAengine completion: clients released mappings while hardware still accessed them. Keep completion one entry behind the raw LLP. Once the channel has stopped, LLP points to the next entry and needs no such margin. Leave EDMA_REQ_STOP and EDMA_REQ_PAUSE to the STOP path instead of handling their watermark as normal running progress. The eDMA path is unchanged. Signed-off-by: Koichiro Den --- Changes in v5: - No changes. drivers/dma/dw-edma/dw-hdma-v0-core.c | 69 +++++++++++++++++++++++---- drivers/dma/dw-edma/dw-hdma-v0-regs.h | 1 + 2 files changed, 60 insertions(+), 10 deletions(-) diff --git a/drivers/dma/dw-edma/dw-hdma-v0-core.c b/drivers/dma/dw-edma/dw= -hdma-v0-core.c index 41e51eee71cc..a0f36cbe0e6b 100644 --- a/drivers/dma/dw-edma/dw-hdma-v0-core.c +++ b/drivers/dma/dw-edma/dw-hdma-v0-core.c @@ -53,7 +53,8 @@ static u32 dw_hdma_v0_core_int_setup(struct dw_edma_chan = *chan, u32 val) { val &=3D ~(HDMA_V0_LOCAL_ABORT_INT_EN | HDMA_V0_REMOTE_ABORT_INT_EN | HDMA_V0_LOCAL_STOP_INT_EN | HDMA_V0_REMOTE_STOP_INT_EN | - HDMA_V0_ABORT_INT_MASK | HDMA_V0_STOP_INT_MASK); + HDMA_V0_ABORT_INT_MASK | HDMA_V0_WATERMARK_INT_MASK | + HDMA_V0_STOP_INT_MASK); =20 /* * HDMA_INT_STATUS.STOP and .ABORT are latched only when LSIE and @@ -63,10 +64,14 @@ static u32 dw_hdma_v0_core_int_setup(struct dw_edma_cha= n *chan, u32 val) */ val |=3D HDMA_V0_LOCAL_ABORT_INT_EN | HDMA_V0_LOCAL_STOP_INT_EN; =20 - if (chan->irq_mode =3D=3D DW_EDMA_CH_IRQ_REMOTE) + if (chan->irq_mode =3D=3D DW_EDMA_CH_IRQ_REMOTE) { + if (!chan->non_ll) + val |=3D HDMA_V0_WATERMARK_INT_MASK; + val |=3D HDMA_V0_REMOTE_ABORT_INT_EN | HDMA_V0_REMOTE_STOP_INT_EN | HDMA_V0_ABORT_INT_MASK | HDMA_V0_STOP_INT_MASK; + } =20 return val; } @@ -76,10 +81,12 @@ static void dw_hdma_v0_core_ch_off(struct dw_edma *dw, = enum dw_edma_dir dir, u16 id) { SET_CH_32(dw, dir, id, int_setup, - HDMA_V0_STOP_INT_MASK | HDMA_V0_ABORT_INT_MASK); + HDMA_V0_STOP_INT_MASK | HDMA_V0_WATERMARK_INT_MASK | + HDMA_V0_ABORT_INT_MASK); SET_CH_32(dw, dir, id, ch_en, 0); SET_CH_32(dw, dir, id, int_clear, - HDMA_V0_STOP_INT_MASK | HDMA_V0_ABORT_INT_MASK); + HDMA_V0_STOP_INT_MASK | HDMA_V0_WATERMARK_INT_MASK | + HDMA_V0_ABORT_INT_MASK); } =20 static void dw_hdma_v0_core_off(struct dw_edma *dw) @@ -90,18 +97,26 @@ static void dw_hdma_v0_core_off(struct dw_edma *dw) dir =3D EDMA_DIR_WRITE; for (id =3D 0; id < dw->wr_ch_cnt; id++) { SET_CH_32(dw, dir, id, int_setup, - HDMA_V0_STOP_INT_MASK | HDMA_V0_ABORT_INT_MASK); + HDMA_V0_STOP_INT_MASK | + HDMA_V0_WATERMARK_INT_MASK | + HDMA_V0_ABORT_INT_MASK); SET_CH_32(dw, dir, id, int_clear, - HDMA_V0_STOP_INT_MASK | HDMA_V0_ABORT_INT_MASK); + HDMA_V0_STOP_INT_MASK | + HDMA_V0_WATERMARK_INT_MASK | + HDMA_V0_ABORT_INT_MASK); SET_CH_32(dw, dir, id, ch_en, 0); } =20 dir =3D EDMA_DIR_READ; for (id =3D 0; id < dw->rd_ch_cnt; id++) { SET_CH_32(dw, dir, id, int_setup, - HDMA_V0_STOP_INT_MASK | HDMA_V0_ABORT_INT_MASK); + HDMA_V0_STOP_INT_MASK | + HDMA_V0_WATERMARK_INT_MASK | + HDMA_V0_ABORT_INT_MASK); SET_CH_32(dw, dir, id, int_clear, - HDMA_V0_STOP_INT_MASK | HDMA_V0_ABORT_INT_MASK); + HDMA_V0_STOP_INT_MASK | + HDMA_V0_WATERMARK_INT_MASK | + HDMA_V0_ABORT_INT_MASK); SET_CH_32(dw, dir, id, ch_en, 0); } } @@ -158,6 +173,22 @@ static void dw_hdma_v0_core_clear_done_int(struct dw_e= dma_chan *chan) SET_CH_32(dw, chan->dir, chan->id, int_clear, HDMA_V0_STOP_INT_MASK); } =20 +static void dw_hdma_v0_core_clear_watermark_int(struct dw_edma_chan *chan) +{ + struct dw_edma *dw =3D chan->dw; + + SET_CH_32(dw, chan->dir, chan->id, int_clear, + HDMA_V0_WATERMARK_INT_MASK); +} + +static void dw_hdma_v0_core_clear_ll_int(struct dw_edma_chan *chan) +{ + struct dw_edma *dw =3D chan->dw; + + SET_CH_32(dw, chan->dir, chan->id, int_clear, + HDMA_V0_STOP_INT_MASK | HDMA_V0_WATERMARK_INT_MASK); +} + static void dw_hdma_v0_core_clear_abort_int(struct dw_edma_chan *chan) { struct dw_edma *dw =3D chan->dw; @@ -214,6 +245,12 @@ dw_hdma_v0_core_handle_int(struct dw_edma_irq *dw_irq,= enum dw_edma_dir dir, dw_hdma_v0_core_clear_done_int(chan); } =20 + if (FIELD_GET(HDMA_V0_WATERMARK_INT_MASK, val)) { + if (!(events & DW_EDMA_IRQ_STOP)) + events |=3D DW_EDMA_IRQ_PROGRESS; + dw_hdma_v0_core_clear_watermark_int(chan); + } + if (FIELD_GET(HDMA_V0_ABORT_INT_MASK, val)) { events |=3D DW_EDMA_IRQ_ABORT; dw_edma_abort_event_mark(chan); @@ -282,7 +319,7 @@ static void dw_hdma_v0_core_ch_enable(struct dw_edma_ch= an *chan) =20 /* Enable engine */ SET_CH_32(dw, chan->dir, chan->id, ch_en, BIT(0)); - /* Interrupt unmask - stop, abort */ + /* Interrupt unmask - stop, watermark, abort */ tmp =3D GET_CH_32(dw, chan->dir, chan->id, int_setup); tmp =3D dw_hdma_v0_core_int_setup(chan, tmp); SET_CH_32(dw, chan->dir, chan->id, int_setup, tmp); @@ -343,6 +380,11 @@ static void dw_hdma_v0_core_ch_config(struct dw_edma_c= han *chan) /* MSI done addr - low, high */ SET_CH_32(dw, chan->dir, chan->id, msi_stop.lsb, chan->msi.address_lo); SET_CH_32(dw, chan->dir, chan->id, msi_stop.msb, chan->msi.address_hi); + /* MSI watermark addr - low, high */ + SET_CH_32(dw, chan->dir, chan->id, msi_watermark.lsb, + chan->msi.address_lo); + SET_CH_32(dw, chan->dir, chan->id, msi_watermark.msb, + chan->msi.address_hi); /* MSI abort addr - low, high */ SET_CH_32(dw, chan->dir, chan->id, msi_abort.lsb, chan->msi.address_lo); SET_CH_32(dw, chan->dir, chan->id, msi_abort.msb, chan->msi.address_hi); @@ -362,6 +404,13 @@ dw_hdma_v0_core_ll_data(struct dw_edma_chan *chan, str= uct dw_edma_burst *burst, if (cb) control |=3D DW_HDMA_V0_CB; =20 + if (irq) { + control |=3D DW_HDMA_V0_LWIE; + + if (chan->irq_mode =3D=3D DW_EDMA_CH_IRQ_REMOTE) + control |=3D DW_HDMA_V0_RWIE; + } + dw_hdma_v0_write_ll_data(chan, idx, control, burst->sz, burst->sar, burst->dar); } @@ -444,7 +493,7 @@ static const struct dw_edma_core_ops dw_hdma_v0_core = =3D { .ll_link =3D dw_hdma_v0_core_ll_link, .ll_clear =3D dw_hdma_v0_core_ll_clear, .ll_cur_idx =3D dw_hdma_v0_core_ll_cur_idx, - .ll_irq_clear =3D dw_hdma_v0_core_clear_done_int, + .ll_irq_clear =3D dw_hdma_v0_core_clear_ll_int, .ch_doorbell =3D dw_hdma_v0_core_ch_doorbell, .ch_enable =3D dw_hdma_v0_core_ch_enable, .ch_config =3D dw_hdma_v0_core_ch_config, diff --git a/drivers/dma/dw-edma/dw-hdma-v0-regs.h b/drivers/dma/dw-edma/dw= -hdma-v0-regs.h index 2124c162a62f..60dc054adafa 100644 --- a/drivers/dma/dw-edma/dw-hdma-v0-regs.h +++ b/drivers/dma/dw-edma/dw-hdma-v0-regs.h @@ -18,6 +18,7 @@ #define HDMA_V0_LOCAL_STOP_INT_EN BIT(4) #define HDMA_V0_REMOTE_STOP_INT_EN BIT(3) #define HDMA_V0_ABORT_INT_MASK BIT(2) +#define HDMA_V0_WATERMARK_INT_MASK BIT(1) #define HDMA_V0_STOP_INT_MASK BIT(0) #define HDMA_V0_LINKLIST_EN BIT(0) #define HDMA_V0_CONSUMER_CYCLE_STAT BIT(1) --=20 2.51.0 From nobody Tue Sep 29 04:14:12 2026 Received: from TY3P286CU002.outbound.protection.outlook.com (mail-japaneastazon11020112.outbound.protection.outlook.com [52.101.229.112]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id DE9224756DF; Wed, 12 Aug 2026 15:58:01 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=52.101.229.112 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1786550284; cv=fail; b=kWQt9bMJw48d+sJ6HkKvy/10ZXmNtOmZESdOGF32JTMHlly3TD5mxjHvnTtc2UIiBLEQ3/ylGIDnfrktG1ODW9NBAYcd1gccn6++U1r3O5XdFNp2FFISdm+lZPgdaj8JKJQyhddb8kNIqGXPEWkao2rZq5qXXu+BR/PP9A4gfCA= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1786550284; c=relaxed/simple; bh=aL0BnrD5qr+7MSXwkpQP8pnFutLCnL+tmjrkrlwEN04=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: Content-Type:MIME-Version; b=QJf4p+lq+GVwAd1ZQqOMUmVwmjLwHhooCvzP+mAQrQo5Xh+BPrrBAKfeI4EGx5o0h/YAEGLNuMg+fSmz+C0Ews/w0r7suQg3nSftC8m9tqMPsZ8nmy3jQuLUxREIgQ86Y2fxKr6Up177bMLaGeum4hQ43kKYtFIo9wn9+AguO/E= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=valinux.co.jp; spf=pass smtp.mailfrom=valinux.co.jp; dkim=pass (1024-bit key) header.d=valinux.co.jp header.i=@valinux.co.jp header.b=GCtkbuno; arc=fail smtp.client-ip=52.101.229.112 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=valinux.co.jp Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=valinux.co.jp Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=valinux.co.jp header.i=@valinux.co.jp header.b="GCtkbuno" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=Io/VEMipaE4BkWb65B6EzimH3uzkOUZoztVC0awDc4YBD8v+Q35D8KwAFax8xbGPPQ0aYOb7jHmAhtxGM96rKE7YRJkOxtBxHGXnnKKB9JFCdK/DXO3EViv97PHX/Rm8OhfJPAHSwGkBPCaortg7ebQWhsbrMiZuUi8l5T71nF6mFxGj7wkxJkrOIil2Njbjx4tM+ESx/4rb/rZ04KH7aFP6IW/iL44H7XGXywskQskFjqCI3aY1vHfW1iQWMCIceAl/Q/x9N8cZd8U6EGAcWVsVWt/Q7MLr2mQPa+LYm85hLz3N6/84zqCzq2HOjjcN2QIUurYhQu594JuhmbN1aQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=juXRn/68rZBO3D6WyenoPw2o2THQi6HyRTKCDrtz8io=; b=Nwk1UL5nMTXGGLCucLABTRpVziOvLCjlyM3hUfwBxemcPzhFX4TihwYtyhJqRK3cNiz9nQzziMwyXqFP22v2+Cj6moxRXUvzIexL90nyBdgHQKcPLS7pVScFPOuaI9YP7fpcpl/p7ub1XiMjplT41uYX7rMW2dcQToY3AbLUpCmSGqYIxRs0j2jTOklQFn7qfSz6nyv4WMAVzyPUywz6YUhJ60lqroU5O5yUlfCPBE5Q34UFoKx7D6F45bnODxGVvGpMpkExEpzFqC28jbzYrPT3991kE1eCZ+LX77S24tV11fAJgn5gFAe/fq70JZ+eNoR60tUKots/J8z4bEa6NQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=valinux.co.jp; dmarc=pass action=none header.from=valinux.co.jp; dkim=pass header.d=valinux.co.jp; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=valinux.co.jp; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=juXRn/68rZBO3D6WyenoPw2o2THQi6HyRTKCDrtz8io=; b=GCtkbunoG8IXtesOlm0xCbo+m7RBOPbOWj4q6E4PP998SnoU619LxDSDVuE8WSCN8ZylaphKzPFiG4+kn6W7f/Y5Tx8jjtZZ2gUnI3ADU+sFwk3xPZ3+9Xj7xbEh1TyvsI7gVW/yKkRygVJpER4NJ+n6MQSkjS2U7uz5HA5zdAA= Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=valinux.co.jp; Received: from TY7P286MB7722.JPNP286.PROD.OUTLOOK.COM (2603:1096:405:38f::10) by OS7P286MB7115.JPNP286.PROD.OUTLOOK.COM (2603:1096:604:42f::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.315.14; Wed, 12 Aug 2026 15:57:39 +0000 Received: from TY7P286MB7722.JPNP286.PROD.OUTLOOK.COM ([fe80::2305:327c:28ec:9b32]) by TY7P286MB7722.JPNP286.PROD.OUTLOOK.COM ([fe80::2305:327c:28ec:9b32%3]) with mapi id 15.21.0292.024; Wed, 12 Aug 2026 15:57:39 +0000 From: Koichiro Den To: Vinod Koul , Frank Li , Manivannan Sadhasivam , Gustavo Pimentel , Kees Cook , =?UTF-8?q?Krzysztof=20Wilczy=C5=84ski?= , Kishon Vijay Abraham I , Bjorn Helgaas , Christoph Hellwig , Serge Semin , Cai Huoqing , Niklas Cassel Cc: Devendra K Verma , dmaengine@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH v5 17/24] dmaengine: dw-edma: Recover stopped channels from tx_status() Date: Thu, 13 Aug 2026 00:57:14 +0900 Message-ID: <20260812155721.2807506-18-den@valinux.co.jp> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20260812155721.2807506-1-den@valinux.co.jp> References: <20260812155721.2807506-1-den@valinux.co.jp> Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: TYCP286CA0209.JPNP286.PROD.OUTLOOK.COM (2603:1096:400:385::12) To TY7P286MB7722.JPNP286.PROD.OUTLOOK.COM (2603:1096:405:38f::10) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: TY7P286MB7722:EE_|OS7P286MB7115:EE_ X-MS-Office365-Filtering-Correlation-Id: 234d9c29-195f-47eb-41eb-08def88a6f98 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|23010399003|10070799003|7416014|376014|1800799024|366016|921020|6133799003|10067099003|56012099006|18002099003|22082099003; X-Microsoft-Antispam-Message-Info: cMoFviUb1U9iogwEVbmAn4ZVmO4/BzkRw5cwdJKzxU3SJJUrbjSABZb2jLRU1piuQmh7VlT/ZhyB5BvSA9twSM46yUQO2emszmFAlMP30dwj1pYvcxgkAC4QSRCk8a5FzAIIXKNIvfTt00BACN8WrqhB5G3ZTp2Rz7qXwgUXh357hH/q+zYtFUcSYYCBf73gzZ9gtWrzjS0nnKXEGtJfEw/VGLSRzb9y6Lzh+JR36S3KfVRx1ETA4556qlqzHBRyD6IueKL+ausdPRB7+2sWLraugPIY4yD9VqtnOhld65G0WqUYbNbJAUBNFgEoV8yTJAJiU1zbSEvy0f2uBntQft4rMcSo2dwnr6qVD2GJjaJKd0e1pNiiY+X3jyMvt6x6CeD1cG0hJl1GQuBo6+uaz8abKhxdvlrlip2ESLzYXJxfEWbpDdbw/CMJNT57IEqMT9JBz9kY7vAqv3LUx478TnT5vAa4tsk6jHv2cDwavfoJ7SEKPZWbevplTqTghu4K1qnzh3zfgymqb/z0HnHxqn3BaSlESztL+gw7Vx6erzvOhXokzvp/6v/CJdJAhe7yYcVNy7pBCFUzkXln7SN9kZJx3PV8bPDJZ62OGF4f8sxcfLrVXMVyJx2qUXLGOv2JIjB0cTf3LZ+F1jSEPN1KhM1ZX0GfXzJEBXxEBOAQmWwqg07r8tum0+1tj9fqrqeV2VqPXo67iAP6n447g9x2oQ== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:TY7P286MB7722.JPNP286.PROD.OUTLOOK.COM;PTR:;CAT:NONE;SFS:(13230040)(23010399003)(10070799003)(7416014)(376014)(1800799024)(366016)(921020)(6133799003)(10067099003)(56012099006)(18002099003)(22082099003);DIR:OUT;SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?nqaHY1jX0igoPWvQCZsTN1IG+c7CtxI/prx/qgqhRiPyl6sTBwAwIu/z/xY1?= =?us-ascii?Q?Is3LDclCcxqwCrkAP7V+bvGhS28TfmcPjOEQBRPeSyvTXcH1SNuMAjSvtkIZ?= =?us-ascii?Q?27JOUfKrM8PHFOiNy2KM8zx929q7l6l+Ll58b++u5iTd1LCux5x3MZHjgBzM?= =?us-ascii?Q?/F9oQLFGlTF+4tbOt+bVg489ESIfIZ+C1rREakzK5UVVw/NZSxzY7/WWAqsC?= =?us-ascii?Q?qO/jA97nIHk5IYRVAmvrYclJtomoM1y9hdSPUXv/tLDvVqk3ISHr6eSxRkRL?= =?us-ascii?Q?je2gpa8uw/bXzegsShpEoyDgSB3SesDSAtFaIzfpHcEGoaSmOF4hNDo/A0S0?= =?us-ascii?Q?g+cy41kelyTaAYz4hHMUj6B2mXBg75hgcM5/ke2uZhxCw67dlCJizvgi5Wab?= =?us-ascii?Q?HafYW1j/DiNO0v2K4HPINpla2xBsV6pQ1flTTVOI2ZfhExqAYDCdjZj+oKxu?= =?us-ascii?Q?noPWSyFbo5/7cLnG1ZtJOWlaa8Nf4uXgoN7aXpDGQE7ASfiOGNvKVTO5uNR5?= =?us-ascii?Q?ctB9D4pdOLNumdnbVyrS15Va6RiZ5Ty0N5djcYEgMCJ2HoyUVEdOFNlcD1BN?= =?us-ascii?Q?Nq3731g2p+FUs8NBKRwChb3YArIcg1T16je/+o6Y5h7IvFsKNH4B0wXSzR9G?= =?us-ascii?Q?REn+pd/SQZRiswAWjx2V+i2M32m29/D+L8uRAstNxrBdsf1cxU+xxKp9UVYJ?= =?us-ascii?Q?dnFq7Tm+ATJLZACb9XHrmnGDR0tcEX66ijHpXl5BT2EKLWXjDpiVc8jVkLwA?= =?us-ascii?Q?ePBbuy4CvF55UTd2EM2ZLBnC/4SfhWnqDMUFb4qwm3Wtd7VNoRsLPGyoff1g?= =?us-ascii?Q?EDKHIKO8UYNdV4bwYpMx2Oh/uQX1V7IW4a26ncq7tGYVR9HSIRv3FApviSxT?= =?us-ascii?Q?bRMO+RebqUbHyrZ6OfZreDuv9YJAhZVltI6MztYOkHEXqlB183/JCRwupZ4v?= =?us-ascii?Q?Mm3yJaauYHwJooZ0Ar4o0HCAFr06BI7pFVKoIScnPbwNIw28QNX0ftiMlj5b?= =?us-ascii?Q?KcCdT07poePcpuN0OaoCF1X9T4dyoGLNHoaQTfOqSyjUXrVIJiXsS/ACFXW2?= =?us-ascii?Q?VzFRTIKV0fbxqg3LwL9CtBkgR0Tmtj/FnSTHOCXSe7W8IElemF6PpumPTqNl?= =?us-ascii?Q?xn1/e2W1uxBtRhnQsuwS87r4cS/K8BofLDM00BZCxgFSlFiXr7gPqOiq3oZO?= =?us-ascii?Q?x1OclvkWMDpUZaV3bWLSyH+ARW90ZVxq9pHnAwNwG0NEAkxFm9a4oDINOJmj?= =?us-ascii?Q?OMk6qr4dOEDIcuHH+gXY3wUFM+9f16Vfb/y0BmhNJ1dG9R/PKwUbkLlc/kt3?= =?us-ascii?Q?HKGBM46RDNrgZqEai9NVjWtOpBYnpM0P7Z0XhpyKTMd66ohoq5SEgu75sAF7?= =?us-ascii?Q?H8vj1UahFhMFs7AJpHIDJNdX8VFrEkYsUaNeb6nuSjPPpbso83lCurJfTPxg?= =?us-ascii?Q?0Azp4KPf62hgyhpIR4vnhWAK+Y8zgAWvssjeTMWKFwyN1TK+yS1ATvoChigS?= =?us-ascii?Q?AVhCTYJ1u3FnfvWt9RISTtdpp7GxX37NHQ9kQZGX9L8sd7MOfiTCDxp5LanR?= =?us-ascii?Q?J09w7O9XMnEvlQDvhuShfzLo+BN8itnouWs0RRyLsal3PsGbO5YaiVZUetRr?= =?us-ascii?Q?f8zLxMK0fNvSId2hSZvEsNrAg4MaW1qWyB9+TRdrjDfF5OppGrrIwmDz2Fz+?= =?us-ascii?Q?31dMu9q9vSUaIAR+5c/ocPknGBgLS4/NhEDfcewOChkOZuWanMt0/WcG8PsF?= =?us-ascii?Q?IsqI08rXvxyY6fSmxlTI1s8dVRZce+MzWMjZsmmTm8yI63A7LWHH?= X-OriginatorOrg: valinux.co.jp X-MS-Exchange-CrossTenant-Network-Message-Id: 234d9c29-195f-47eb-41eb-08def88a6f98 X-MS-Exchange-CrossTenant-AuthSource: TY7P286MB7722.JPNP286.PROD.OUTLOOK.COM X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 12 Aug 2026 15:57:39.5197 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 7a57bee8-f73d-4c5f-a4f7-d72c91c8c111 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: cu7AF3bz9UyKEWSXdw0BE32yVKYs5eEqpAUkdqgtIQdJqNOXH8x3mH2hRVrEtW8DzPZA2pPDZnD1LnadnuOqlA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: OS7P286MB7115 Content-Type: text/plain; charset="utf-8" An LL channel can stop after an append without another interrupt. A synchronous user may then time out with work still pending. Let tx_status() consume IRQ-recorded progress under vc.lock. If the channel is stopped with pending entries, reconcile a fresh LLP boundary and re-kick it. If it still reports running, arm the bounded eDMA tail recheck. Do not use a running LLP read as progress because it can move ahead of payload completion. Refill the ring after consuming progress and update BUSY or IDLE from the remaining entries. Signed-off-by: Koichiro Den --- Changes in v5: - No changes. drivers/dma/dw-edma/dw-edma-core.c | 40 ++++++++++++++++++++++++++++++ 1 file changed, 40 insertions(+) diff --git a/drivers/dma/dw-edma/dw-edma-core.c b/drivers/dma/dw-edma/dw-ed= ma-core.c index 8325e296cb2a..dffe3f8b8c20 100644 --- a/drivers/dma/dw-edma/dw-edma-core.c +++ b/drivers/dma/dw-edma/dw-edma-core.c @@ -301,6 +301,14 @@ static bool dw_edma_core_enable_ll_irq(struct dw_edma_= desc *desc, u32 i, return (chan->ll_head + 1) % DW_EDMA_LL_PROGRESS_INTERVAL =3D=3D 0; } =20 +static bool dw_edma_ll_recoverable_pending(struct dw_edma_chan *chan, + enum dma_status ch_status) +{ + return chan->request =3D=3D EDMA_REQ_NONE && + chan->status !=3D EDMA_ST_PAUSE && + dw_edma_ll_pending(chan) && ch_status =3D=3D DMA_COMPLETE; +} + static void dw_edma_core_ll_start(struct dw_edma_desc *desc) { struct dw_edma_chan *chan =3D desc->chan; @@ -923,6 +931,38 @@ dw_edma_device_tx_status(struct dma_chan *dchan, dma_c= ookie_t cookie, if (ret =3D=3D DMA_COMPLETE) return ret; =20 + if (!chan->non_ll) { + scoped_guard(spinlock_irqsave, &chan->vc.lock) { + struct dw_edma_ll_snapshot snapshot; + enum dma_status ch_status; + + ch_status =3D dw_edma_core_ch_status(chan); + + /* + * Do not treat raw DMA_LLP polling as normal progress here. Only + * recycle progress recorded by the IRQ pass that consumed the + * status event. + */ + if (chan->request =3D=3D EDMA_REQ_NONE && + chan->status !=3D EDMA_ST_PAUSE && + ch_status =3D=3D DMA_COMPLETE && + dw_edma_ll_snapshot_take(chan, &snapshot) && + dw_edma_ll_consume_progress(chan, snapshot.idx)) { + dw_edma_start_transfer(chan); + chan->status =3D dw_edma_ll_pending(chan) ? + EDMA_ST_BUSY : EDMA_ST_IDLE; + } + + if (dw_edma_ll_recoverable_pending(chan, ch_status)) + chan->status =3D EDMA_ST_BUSY; + dw_edma_core_ch_maybe_doorbell_or_recheck(chan); + } + + /* dw_edma_ll_consume_progress() may have completed this cookie. */ + ret =3D dma_cookie_status(dchan, cookie, txstate); + if (ret =3D=3D DMA_COMPLETE) + return ret; + } if (ret =3D=3D DMA_IN_PROGRESS && chan->status =3D=3D EDMA_ST_PAUSE) ret =3D DMA_PAUSED; =20 --=20 2.51.0 From nobody Tue Sep 29 04:14:12 2026 Received: from OS0P286CU010.outbound.protection.outlook.com (mail-japanwestazon11021092.outbound.protection.outlook.com [40.107.74.92]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id CED2647ACD3; Wed, 12 Aug 2026 15:57:53 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.74.92 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1786550276; cv=fail; b=JC/j7aRaL/UMvN7zAcWv/mCCiIPpGTKpC17ptihu80SvqHjcQZmsRenxzDulOvwQGkZddEqOWqKWA4nnorWd6YlUFVfkza81pnKGHEZnHJC/3N40L7nheoniy5JOt3o8jLswksdyO1vfcZMmY7SGNbiP/wWOe87CKhXZn034qNY= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1786550276; c=relaxed/simple; bh=qBEp0d/wkGX8sdXGz1MhtNets/tVO6xJPQali+LukXA=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: Content-Type:MIME-Version; b=cbqAeDmaihUYcQ07c7kfsOoZvY3PD4D8mGiYS0Xrvu9DOK09MZO25bq7+f1HhLrjf12A6X8MUKDx2zd2JkIUPpZpT+UtVgSQaGClcp+caRfOOpFh1DMSBxgbUl9tV9XL6bw7WtevZLMcM73raTvNaqS9hS3OxmpOwF5aXmSkHtY= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=valinux.co.jp; spf=pass smtp.mailfrom=valinux.co.jp; dkim=pass (1024-bit key) header.d=valinux.co.jp header.i=@valinux.co.jp header.b=HlxzTMlo; arc=fail smtp.client-ip=40.107.74.92 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=valinux.co.jp Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=valinux.co.jp Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=valinux.co.jp header.i=@valinux.co.jp header.b="HlxzTMlo" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=JIeyr4cgO60RHjbksqgrWV44RcP2XxxiUwlxaUxNtOpztw41xs8CSOhAQ8xVK7Cl6XiwafGa121kNiXcSAJziM6Uc9jI++NRj/9r6bMN7vIdaOev5m/ET56R35zbe2JZDYKkAZvsPWeJnevzcpNR7BbIO1441sFAQpZ3lG6wJ/pOqQyHU7VJNrPR0fkJGya9y/avzKHHjsjwG5BztZ48SRhNSlgzj9X28qWXMQFMLVE6s2klr/XfxpvRy6i+rfOTDK/YXLHxltbIDFvzLAOlBpVAuH1SQaRBzyBCxWaA0RJCcCETP40oCVhhwI0TamQVFiHAIqbLUanUQd17jPpW2g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=U2xvVOegUqhRTV/rmKVgc3QzF4fpAXJ1v10a5a+yNUs=; b=A70GMcMgiaHydwWY3Zigfy1EIemQmcM3Q09xd7xh4FRqot7mqskYbsMEWC0QdaMVfG8DGd+M1DQbUr12HatF2HhSLLIrfq3gtvtHimw9rAMCJcQyGco7zn+VVDDe2VY962EegOhvmhmV8zcHwZhOS7UnzKe7Aw9x7SWV2CFflqVfqRRsyplisAmZRtsSRyRHKXW8gIuCzMyR+tqQS0WojIB6IBLZ5LShWCin2XWpvWIUQzknA9e9PSmWBZLEkCKR3ge3MDM2x+zONPdc1n5XxMaHQ2/VX0Y88y7XQQXVDVcwgXLqCIxN5K98S00IncxrT8DgmX08wd43MXjjrrNOug== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=valinux.co.jp; dmarc=pass action=none header.from=valinux.co.jp; dkim=pass header.d=valinux.co.jp; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=valinux.co.jp; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=U2xvVOegUqhRTV/rmKVgc3QzF4fpAXJ1v10a5a+yNUs=; b=HlxzTMloa7MtRFfah6QpN2ZdZttvqYuOGuMUipuxIIZpRC3qbhGoWEImwOUgBoHZjWhe2nwISQCAap8aswL3MhK8h80U70cJKcLlLQ3cUkJ/aU99/hjPBzFaqW4j0wt1p/4J922dM8DxMoXNotKi3NNPHswyqzCTaxfrKH+FMS8= Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=valinux.co.jp; Received: from TY7P286MB7722.JPNP286.PROD.OUTLOOK.COM (2603:1096:405:38f::10) by TY4P286MB7069.JPNP286.PROD.OUTLOOK.COM (2603:1096:405:343::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.315.13; Wed, 12 Aug 2026 15:57:40 +0000 Received: from TY7P286MB7722.JPNP286.PROD.OUTLOOK.COM ([fe80::2305:327c:28ec:9b32]) by TY7P286MB7722.JPNP286.PROD.OUTLOOK.COM ([fe80::2305:327c:28ec:9b32%3]) with mapi id 15.21.0292.024; Wed, 12 Aug 2026 15:57:40 +0000 From: Koichiro Den To: Vinod Koul , Frank Li , Manivannan Sadhasivam , Gustavo Pimentel , Kees Cook , =?UTF-8?q?Krzysztof=20Wilczy=C5=84ski?= , Kishon Vijay Abraham I , Bjorn Helgaas , Christoph Hellwig , Serge Semin , Cai Huoqing , Niklas Cassel Cc: Devendra K Verma , dmaengine@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH v5 18/24] dmaengine: dw-edma: Make the LL ring reset a full channel resync Date: Thu, 13 Aug 2026 00:57:15 +0900 Message-ID: <20260812155721.2807506-19-den@valinux.co.jp> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20260812155721.2807506-1-den@valinux.co.jp> References: <20260812155721.2807506-1-den@valinux.co.jp> Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: TY4P301CA0024.JPNP301.PROD.OUTLOOK.COM (2603:1096:405:2b1::11) To TY7P286MB7722.JPNP286.PROD.OUTLOOK.COM (2603:1096:405:38f::10) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: TY7P286MB7722:EE_|TY4P286MB7069:EE_ X-MS-Office365-Filtering-Correlation-Id: 9e4fd8c0-32f0-4146-0cef-08def88a7013 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|1800799024|10070799003|366016|376014|23010399003|7416014|18002099003|22082099003|56012099006|10067099003|921020; X-Microsoft-Antispam-Message-Info: p0BMcxQKQU1uEPDq23/wwvSX7GYDiutm91nCq/VsV/ckunzEO1DExx/g3TrdOk5Y9GSulCjKN0ot73w2q5iKASbUeDhGtjAi6Cw/XXMKiN6kdQwVkg36eSBvfp/D/7MtfZdf8ykY06NbKuK21iEECybNlxn7SJkHB9XLp+adih//Zat9/nm1O/K2HlZM82oiQDEVwjgWR0KZW64pLjn1Ut3bWFwfNjtR/CpBhWHGzAvE7VaPEQyEIBGBdW7vKJvFsxtp8oYAGj9GhBHgGe8JPmTw0UQ4SyRIHya5qTnNY4IOpHEFviHXwl4KykUcejjWWh2hfV3ILSDUCFR/Jxvv3yQ6MLfTp6aQKmJKJRhZPazEYWucgf93a8bR7mjVCL3axkNGCagbdXt4I3VF16VF13kPOZxnjRFcNCR2sM6xL4mzPwu9RISqFYbIleoawr5tvOLKx8lw954fIsOzep5ne7vEWdj6FLJip4sT/mGGPr+LAhE0DczITHqCuvmmdryz4RIftnoG2Yd5nLHDhd5m9fuGvoVvjw/I2TE7vR6ECRgsHmsdPyamjkfLzSK43Rn0CS2Go4AaVw4G1lVAhK5f2hV50mqEQZmhCNrnV1kl1QCfyF2skKp/nDqfpywTF6xAKR2j73lrRE4O67F16f6jBlJHD/QqzxBoufwMsbPcXwtQPZ+UBC8NOVZOUGL9a5xWmYkiVQj2n7uGGwY2EZF0ew== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:TY7P286MB7722.JPNP286.PROD.OUTLOOK.COM;PTR:;CAT:NONE;SFS:(13230040)(1800799024)(10070799003)(366016)(376014)(23010399003)(7416014)(18002099003)(22082099003)(56012099006)(10067099003)(921020);DIR:OUT;SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?VKFoA0UPJL1tm1AsSCYLlDHBh+2ouLquP1einChRAEPPeClCaIxI8Fbq1Y9f?= =?us-ascii?Q?h0o+3X6Iepipvi1vKXYV+B5ZwXE9iV88yl3cCmZWVdYoj49zUKzjL+LY9y4a?= =?us-ascii?Q?nQ0eEmaKF9CPYjg6NYRB25G9GyhVip3KcKLErpiu2DxEu0EqUNBN4O1mLbjE?= =?us-ascii?Q?Fy4k0B2pKFSO/r9IuS1USRNnNphBoSGNU0uTBnbaIALiAL7SiSv5VkTkbYe8?= =?us-ascii?Q?vnSGonGACIsm/g7ubC4Yk9e/jpzNVEych3PjYtLfSxmm7wp4FZW4ZT5egQIh?= =?us-ascii?Q?55/qV+ug4XQdHa+X69D/JIRHpGW39tGbkpdtBhCraysjskcO3W4LUeuDPTPv?= =?us-ascii?Q?mlAGjZNfyIYmnRuLJQhTJDq75h/TdS3nGSoytPHTb3i8mSQT3T7tGhRrGzVR?= =?us-ascii?Q?9JTrI0RppIXmKOho70kxA8F5Q2f7+YMHCm5PJArneFqvlS1NbrQhPnkn3j5J?= =?us-ascii?Q?T0gax4ALq0omFUck5XaOIFORouDnNVYFP2XT7NrSNesP6zlS10xj7N2rXtYg?= =?us-ascii?Q?Gd2ksmKpPgnufAMLFXMkmamwtIe4+HbRSF/v0bAy1fNZR3LrNF48T9gG1WFK?= =?us-ascii?Q?7q/IenKLL9bsaAPoZh/v8/DVozyn0QnlsBipGPH81LQc9xqRLK0eUjL6VcgU?= =?us-ascii?Q?3rDAnBHUDUPa879Wf0qgqL3N5OYz2ufrL8rM8HBWfd+kGe7CJFM0zfdYFfiu?= =?us-ascii?Q?Hzo6aH7+6XefLeF4U1GsZkLCF+OcB+RdT1LyygAbCw0ad9OiakPLucG86Dnd?= =?us-ascii?Q?P1qKQ838QFPIXI4n6nyThMJvpdgVf8JxxzyJ54SbCMWkNHNCeg5odE9y/LbJ?= =?us-ascii?Q?lHTzrBnK7/h5QX2ZXCmIV+jh6OyGK85XIiKBvUjiKn7F5DRFde5gxJOE6LvM?= =?us-ascii?Q?FH5MoPL1O1ieBFAaEilvZjWS4W8LZ7d9oQK2gMmtCnmyRGL8QyXXltnrO5Cw?= =?us-ascii?Q?FiIsh3OE5bxmGK9vyPZsK4B7ddmH3cp9uQEXnP1XMwYvqVriDKITSG+pSLf/?= =?us-ascii?Q?7F6KifYAxo3Nfe07P+dE0s2miSrGqHuHPJrPFz2M62ewIrOZEFY4cY+L83XP?= =?us-ascii?Q?V1zMXOtFVB3mCKCHTZN7xbRyD/MOdtEBwRp3xIC6qf5s+YDh3049RSlEx1cm?= =?us-ascii?Q?v7XIo98a2+WAagtSSaGr1VXFKhgY/WyMUSZSGRd9Jv0RXVudA4ppyfhOe/P0?= =?us-ascii?Q?tZz9X7F3HUpOP9TB5vmY/qxyHUsyILQ4plPdrdDk7WZ5NCBPOcL6laXICiPB?= =?us-ascii?Q?vSzSrP4BRIp/NrPbr7aDL6s8nnafDTis8bxUBbgqfShk0EMNlqP7DHDXoAwi?= =?us-ascii?Q?WwW1efr7i5nHaF6jK+X/GQEXYT+EnEUNXxgoPO04g2utkcGunMfioKdOQUQ2?= =?us-ascii?Q?J/dh/aPepFOZRb+kVDrcA5SVDYfsUgYCBeYnNx722ODRh7joHQUw5WftnTy4?= =?us-ascii?Q?a7BKTqvj1O8kUHrKRVZWvjdL3UGcMFF7/Jh2BvqsaFF/hUCULV6jm8Zy4k21?= =?us-ascii?Q?pAPBuKqPKIMnIMA6pGzdo9Y4JOiC2/X7xjGaBmmErbhTsDMjvAnLntPDxo7I?= =?us-ascii?Q?gqFxI1oASsvSZ71C5NIJ+oaRz/Otj7dSZc74cM+XOyV/jDU2tiNGB4g352GF?= =?us-ascii?Q?jjnE/21y+ut2BXtpnG5yV1VMKC5LVPEGEt5ha7nopB6D58ZhGEMDaAZBtH66?= =?us-ascii?Q?19emaP0s78CgVt/n362Jr3bhhmxp4qo/jF/ffrZXtGi03Lt7Ed3GbwgX1GpF?= =?us-ascii?Q?8UsqDYwYDf8M3lTbU1grN9M1XzfTr5jgigislg5Oq4xYmH8xFLDv?= X-OriginatorOrg: valinux.co.jp X-MS-Exchange-CrossTenant-Network-Message-Id: 9e4fd8c0-32f0-4146-0cef-08def88a7013 X-MS-Exchange-CrossTenant-AuthSource: TY7P286MB7722.JPNP286.PROD.OUTLOOK.COM X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 12 Aug 2026 15:57:40.3152 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 7a57bee8-f73d-4c5f-a4f7-d72c91c8c111 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 7VjkW7o7Oh0O0+jQUmbjPIkSiFJP3kCqIAXbio4nu2AyK0olUDVmop7Cdp7kuqOgvqXtMuFXPlZoQxgiDSytyA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: TY4P286MB7069 Content-Type: text/plain; charset="utf-8" Reset currently clears the ring but leaves issued descriptors unchanged. A descriptor with published entries is then stranded because LL progress can no longer account for those entries. Cancel any pending stopped-tail recheck before replacing the ring state. Keep descriptors with no outstanding published entries so their unpublished work can use the fresh ring. Abort descriptors with outstanding entries because their consumption is unknown and replay is unsafe. Since resync can complete and remove descriptors, reset the ring before looking up the next issued descriptor. Signed-off-by: Koichiro Den --- Changes in v5: - No changes. drivers/dma/dw-edma/dw-edma-core.c | 32 ++++++++++++++++++++++++------ 1 file changed, 26 insertions(+), 6 deletions(-) diff --git a/drivers/dma/dw-edma/dw-edma-core.c b/drivers/dma/dw-edma/dw-ed= ma-core.c index dffe3f8b8c20..eb0fc73a4f47 100644 --- a/drivers/dma/dw-edma/dw-edma-core.c +++ b/drivers/dma/dw-edma/dw-edma-core.c @@ -212,10 +212,29 @@ static void dw_hdma_set_callback_result(struct virt_d= ma_desc *vd, =20 static void dw_edma_core_reset_ll(struct dw_edma_chan *chan) { + struct virt_dma_desc *vd, *tmp; u32 i; =20 dw_edma_ll_recheck_cancel(chan); =20 + /* + * Software cannot tell which published entries completed before the + * reset. Replaying one could duplicate a transfer after its target + * changed ownership, while keeping descriptors with uncertain entries + * would block later in-order completions. Abort descriptors with publish= ed + * entries; untouched descriptors remain available for republishing. + */ + list_for_each_entry_safe(vd, tmp, &chan->vc.desc_issued, node) { + struct dw_edma_desc *desc =3D vd2dw_edma_desc(vd); + + if (desc->start_burst =3D=3D desc->done_burst) + continue; + + dw_hdma_set_callback_result(vd, DMA_TRANS_ABORTED); + list_del(&vd->node); + vchan_cookie_complete(vd); + } + chan->ll_head =3D 0; chan->ll_done =3D 0; /* Drop stale CB bits before reusing the circular LL ring. */ @@ -363,6 +382,13 @@ static int dw_edma_start_transfer(struct dw_edma_chan = *chan) struct dw_edma_desc *desc; struct virt_dma_desc *vd; =20 + if (!chan->non_ll) { + if (dw_edma_abort_is_pending(chan)) + return 0; + if (!chan->ll_valid) + dw_edma_core_reset_ll(chan); + } + vd =3D vchan_next_desc(&chan->vc); if (!vd) return 0; @@ -381,12 +407,6 @@ static int dw_edma_start_transfer(struct dw_edma_chan = *chan) return 1; } =20 - if (dw_edma_abort_is_pending(chan)) - return 0; - - if (!chan->ll_valid) - dw_edma_core_reset_ll(chan); - dw_edma_core_start(desc); =20 return 1; --=20 2.51.0 From nobody Tue Sep 29 04:14:12 2026 Received: from TY3P286CU002.outbound.protection.outlook.com (mail-japaneastazon11020143.outbound.protection.outlook.com [52.101.229.143]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 335CB43637C; Wed, 12 Aug 2026 15:57:46 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=52.101.229.143 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1786550273; cv=fail; b=m+Yfb7lkMgxsrS5hHLE0AZss+JgVuJ/jmKr3iCEX1bKcD/LqdTxI06Pn6QRAA9S+15a4qNaL1Y8M1ZBRqhLniibcIjNVAKERNhJ6iKWd3YyttFtdiaeouX+elqvCQDvZMEuxmaZKwIIy/9gygyZaelQF7Uz6aku1NM35cf4oP98= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1786550273; c=relaxed/simple; bh=axRFgFoGwlwpC+c2dFY/BlQY/oP/sr41YkZVyCSg+1E=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: Content-Type:MIME-Version; b=cE+1JthMHCAhR/VaQJ2BteEsxMqf4eEB55Z8OND/MXLXYAjZZfr2szcRqrI25GLORa/iBTwfIaNMVqdMzDQKcdC7O+b16fm9extybOeFFPs/ktxxQZY+kE2JnUmQpsKy2sqJbDayBfBJffx/NJGmKq57sbwqKmFvCzwHRwh56uo= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=valinux.co.jp; spf=pass smtp.mailfrom=valinux.co.jp; dkim=pass (1024-bit key) header.d=valinux.co.jp header.i=@valinux.co.jp header.b=nvkbxPQE; arc=fail smtp.client-ip=52.101.229.143 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=valinux.co.jp Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=valinux.co.jp Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=valinux.co.jp header.i=@valinux.co.jp header.b="nvkbxPQE" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=FJDKyHiEXgEp1DiwIuj23S8gNwCx6Ov1HOZnMPM4Amxuuwi4XKUkh0W+7sNGAlJOPzlf3qERDVelJbQk4p2jqFKXTp58cvNul63wDHY9UTq24BafxE0IWwvNe5KCWo2stBqilGWhKI7YsuG9gXPxmBeGCVjucQKE3XZnXCveBubeJzQzrJr3bFBoPAcB/V9AJtUGgHM8cNH+WH6oFrXDkfSUtHxWueb2NJTwK/2JmZACxb2bLcKV9sn6MstormGM+yxBVj37kzDMUerPBDR36c41/1nHiWtmn4c8KpWRTulKh8j4a/oaRI8E290N2eoczMCnIDtj3ImtKn7nWn7TzA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=S7pYf015fzZPSbV1eBPVsz5fjTdfQ6c5YiEDF1QgFWc=; b=v4NCCvzPCbLPHgfsGJcbdAdA+RYAvD89vIpMp9pV3s1gvHPLTOeRcaD3SieCALE6BeJmdW9GieDA65ozMDEHCiwBfMr8SUAiw7trthv92mHlQbCzSCsrlQILLWKi1JnXEjdcaUZAKYmCS1GY6Mo3AwlXXS+zPrejxh4+jalbOVPVkaQ+XjNo7rJe+58APvFgbsyrfvgUvNlpK76LKMwVQjxnVwBEMY3EfRM+iCzhNWgoz+GukCt4eHOiO10nONs1r6TIJzMxjnMGiu5VocliS+uyR8Y0/tuSHDjhcFCDTBHylJ5bQwxJGswwvdity9HJh3fcZ4mA+GH9ngfI/Sq8lQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=valinux.co.jp; dmarc=pass action=none header.from=valinux.co.jp; dkim=pass header.d=valinux.co.jp; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=valinux.co.jp; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=S7pYf015fzZPSbV1eBPVsz5fjTdfQ6c5YiEDF1QgFWc=; b=nvkbxPQEAIP589GKdxlgJSnwTZv6Fj2tDIfGWm7i29NpBxCPu3zGlgt8OMzZwsBTfctYT7kl9fe2/iWeAg4y2NhDrpUt6AVUrGQUoEz/VN+ENaQSuteCniXmCBBxclA6/yF+c9RCCly3syknCUPYvuHJhb0HNGTmd3qKlS1AaeE= Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=valinux.co.jp; Received: from TY7P286MB7722.JPNP286.PROD.OUTLOOK.COM (2603:1096:405:38f::10) by TY4P286MB7069.JPNP286.PROD.OUTLOOK.COM (2603:1096:405:343::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.315.13; Wed, 12 Aug 2026 15:57:41 +0000 Received: from TY7P286MB7722.JPNP286.PROD.OUTLOOK.COM ([fe80::2305:327c:28ec:9b32]) by TY7P286MB7722.JPNP286.PROD.OUTLOOK.COM ([fe80::2305:327c:28ec:9b32%3]) with mapi id 15.21.0292.024; Wed, 12 Aug 2026 15:57:41 +0000 From: Koichiro Den To: Vinod Koul , Frank Li , Manivannan Sadhasivam , Gustavo Pimentel , Kees Cook , =?UTF-8?q?Krzysztof=20Wilczy=C5=84ski?= , Kishon Vijay Abraham I , Bjorn Helgaas , Christoph Hellwig , Serge Semin , Cai Huoqing , Niklas Cassel Cc: Devendra K Verma , dmaengine@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH v5 19/24] dmaengine: dw-edma: Drain LL entries for STOP and PAUSE Date: Thu, 13 Aug 2026 00:57:16 +0900 Message-ID: <20260812155721.2807506-20-den@valinux.co.jp> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20260812155721.2807506-1-den@valinux.co.jp> References: <20260812155721.2807506-1-den@valinux.co.jp> Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: TYCP286CA0330.JPNP286.PROD.OUTLOOK.COM (2603:1096:400:3b7::19) To TY7P286MB7722.JPNP286.PROD.OUTLOOK.COM (2603:1096:405:38f::10) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: TY7P286MB7722:EE_|TY4P286MB7069:EE_ X-MS-Office365-Filtering-Correlation-Id: 4e927e61-daf1-4c72-85ed-08def88a708e X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|1800799024|10070799003|366016|376014|23010399003|7416014|18002099003|22082099003|56012099006|6133799003|10067099003|921020; X-Microsoft-Antispam-Message-Info: G4QiZWldPSW22ZjFP6OQwJfEj0/Dp8azkQigK60c8LK6DV99w+AqzSdWpFMqQaw3Y7PtVpT1CYakkYcjQVZV5uu5nsUy15zYQKGwKAAbvfdF9iGvGswa5fDRGW2jNftGk17egxo3OjTHvzqlP6LujsCUlvbWT5E01k43exBYdoyFy9DHSfqwhXinU7ae1FcaZDd7Sy+0BK62tfy6s7mDd1ZrmqWmYRskvPnHR6xqn9HvXzmAcVAitE7e0nbroR3WmbK6XyxoOpz+d18Uj2e+nIcjIwMTqP4TJsJTpliHLYOBnibJZkrsya4i3puvCHxvKT4i4llc8E2DZNMafa5PNiB0NfkgHfzytRjhbEtavisG8syTk7HKKFHTUKpvJEIbrvuZInJsfv8AN/fsG/tT6Sp2MAzL9laJKk0UD+2rDDTJZSDcpCgZEQpkYcVrw+IXPmnvYj1rECL0oVfvW53u/Goivgt0XeUptWKtXKw3kAaJaNhsF65q7pALfnsoWmX+BQGjBecnYOd+qPVSu7NgWJldMnCppgsgWH3JY7SPxnVRN9411L/qkrpzAk90YJNi8Zl8aZ9/hEZ71akpYG7gPjfI+MSXQgnTntxqTpbMB1o0JzPw1Kmwaio6gPtQ/KwQ2YWeBhONnirtXFZ+RvQmNbRxUwrQ/ODvHqz7vVHNGyA4/RMLB95FZZg/SG3p0LEX+2Ovx1pSOrb63GmGWe+5Zg== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:TY7P286MB7722.JPNP286.PROD.OUTLOOK.COM;PTR:;CAT:NONE;SFS:(13230040)(1800799024)(10070799003)(366016)(376014)(23010399003)(7416014)(18002099003)(22082099003)(56012099006)(6133799003)(10067099003)(921020);DIR:OUT;SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?I8HGif8bA7AVI/06/dHMbfjfiAG+65M1emgR9g4B23fiKDbPOxOWWtZo/ADC?= =?us-ascii?Q?RszXM3a4dy8iCYRgxO14RNbVz/AvvsJE6AzUjMCdNCtZi6aKp5ts+m3/Xn8m?= =?us-ascii?Q?dmJDuxqfqqsP3lEnbGNfUqRgx46H1aj7178NRs79FhMijPJTPpr9ZZnSoE/V?= =?us-ascii?Q?Wr5OAHUbU2aFgfq/5gjjCcNU/VhHo8ouPGd2plF4vLg8pCJp8I6mm8PGmcsD?= =?us-ascii?Q?UIB0psnLl4KpqAsMrx5qaQNNY0j6SKXCCNUWeq4StMYFAMYNwVWt7zjDkknP?= =?us-ascii?Q?4Kr61vb1xemikCkyBis2tfhJjotjkgxFzCx0VBnxUcNcJforcLZqS87pdvTS?= =?us-ascii?Q?2lWWCPMmTAmUZPf8xjoZ8w1/l7QGHaCGIkCXLleDi8yQDNJwdqpTfb52/izm?= =?us-ascii?Q?7SJ8kVrE7J2oFewal/uxP8DnFwhLGh8ODQyfnkOJr8V1+kZ8uSRv5XH4CwcF?= =?us-ascii?Q?mDtlIt7+Zgsa/m9ZrmaIM+2QN5KXMwXzfneILeR+Zmi2zdjWtLbRPwFCp8v4?= =?us-ascii?Q?tsT0khDc5LStc/IBLQURBi/b2rWMdXMkPpfhDGS7Xq6fXhFu0g/iAldNgwkE?= =?us-ascii?Q?OVyM8azV59qWYcGTy4XFHeNTqsgqlDr5S66z9k9ZNVYSkRleIoPCu8m8jSCL?= =?us-ascii?Q?e3u7uzzAGnknapZzi/wVY9FzXhOoU6Dr01dk1ouMWW2ZDTE1c5mevrP6K74f?= =?us-ascii?Q?IsUBCCANjxKCg+XXBi0yRdY6EQtSp/aFPnk9PEuzc4tsRrs+AnJRSlt1IPVG?= =?us-ascii?Q?0vxHLtHhg4zIqeTdcaVlHPQZcx5/K0nvuAvT7u2EYd1wO50O48VZYkD9xvvF?= =?us-ascii?Q?TyA2BNvnwlWGQ4qCEoZojHY1wcq/0I3EYCOcQlWmhuen1pnsIWW15jlMr1ZA?= =?us-ascii?Q?FFuPRaOC85x2WLgFPHORINZrkgd8zkAYN/3hxPSe4WXCYYWwO1rFiICCOLdw?= =?us-ascii?Q?x3R5RPkKp8fs5JSIC8zkH3HbA2qrzql8Ghj+hDQgq/QdtuEr1bWka8qSs6ud?= =?us-ascii?Q?3OqmH+iIs4V1SpSeJ7C+IUCuldLU8cEHTjUqkAqRtt4X69iJ5q2eHV3vPySQ?= =?us-ascii?Q?vt4oW+5ig+cw42VkqfkC6zPc+Kr6KkYFFkGOOOMlOwPVls1FAKyl5lBL5zEv?= =?us-ascii?Q?IYn2j+vB6BYAnyfypmyD6ELw+BP5acjim+aFNowEbdQSsvAMfZQ/tNtOnF4K?= =?us-ascii?Q?RevkFnHCcMVfRMcF3UxkHd+uNiqfgJ5FVdhBwdXOFhzUYuzSnxfDHEoMis0A?= =?us-ascii?Q?B9+jIS3JY1UCUViFpM8qaJnk7fBrXl/bPD1Pige68nEjzrAZ2MGL1RBIS01l?= =?us-ascii?Q?g5wQO/mIvHI4m3X8nRJL3AAqQWqD093XLj/SC7rgPIgQz8p+FMeommqVhvmi?= =?us-ascii?Q?bW3pI5A3kng4EgJtya7HJt0BAsi/wNpuOKyrkuT741ouX8rPkW509aeUGaKL?= =?us-ascii?Q?V9R38ifP/F+lc4zIeuKJrge02aqDB41dPM5d7Eab1eXBYdAq8qE8/q6JGshY?= =?us-ascii?Q?+v+t5E+thxvxap3foBEamASwwkgG/EDYOrxuT/tcvXrwNx+rHGbhEEJ9JA6c?= =?us-ascii?Q?PlqzuD3ApSuxRjdnOnE4T7FDBLirmlaDb9uo7v1UYuZucyCOn+nZ0EL2EA2Z?= =?us-ascii?Q?3KpG7peq6vPDc9KPwnhy70+68xoaWLcrevq5xdPvhjXgjIU71agMvaKUbn69?= =?us-ascii?Q?P6dCOP9CaaYM2A1Ce9D+mnV6uG8tst0wPC1F9kHkM8Keg5xqrBVACzSKjMsj?= =?us-ascii?Q?fkquQij3/r/i95WY6j83A/64IOIItNqxm7NMSKBRME/Sl0frwulh?= X-OriginatorOrg: valinux.co.jp X-MS-Exchange-CrossTenant-Network-Message-Id: 4e927e61-daf1-4c72-85ed-08def88a708e X-MS-Exchange-CrossTenant-AuthSource: TY7P286MB7722.JPNP286.PROD.OUTLOOK.COM X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 12 Aug 2026 15:57:41.1244 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 7a57bee8-f73d-4c5f-a4f7-d72c91c8c111 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: uMv/IMsl7MGcj7BSJg2nTEtqjpLAdQSsQonsu+Ciz37CEDWkzcFZAT39EnZoGbUVDeJNt0jQp7lXJxmC1CPOLw== X-MS-Exchange-Transport-CrossTenantHeadersStamped: TY4P286MB7069 Content-Type: text/plain; charset="utf-8" An EDMA_REQ_STOP or EDMA_REQ_PAUSE can arrive while hardware still owns published LL entries. Keep the request pending until hardware consumes them instead of discarding the ring state immediately. With the eDMA-compatible interrupt interface, a DONE interrupt can report progress rather than a stop, and STOPPED status may still be visible while a doorbell-triggered start takes effect. Before kicking the remaining entries, serialize the stopped-state and zero-transfer-size checks, the DMA_LLP sample and the interrupt-status clear against IRQ capture. Consume that boundary without refilling so the producer boundary stays fixed and the kick path cannot clear its DONE status before accounting for the stop. If the channel has already stopped when the request arrives, complete the request once the stopped boundary consumes all published entries. Otherwise, kick only the entries published before the request. If status still reports the channel running, arm one delayed recheck to catch a later stop at a CB mismatch. Native HDMA reports STOP separately; non-LL completion and EDMA_ST_PAUSE also identify a stopped channel. If an ABORT is pending, leave cleanup to its handler and do not ring another doorbell. Once EDMA_REQ_STOP completes, terminate submitted and issued descriptors without callbacks. EDMA_REQ_PAUSE leaves the channel stopped for resume. Signed-off-by: Koichiro Den --- Changes in v5: - Explain why termination keeps waiting after reporting a timeout. (Sashiko) drivers/dma/dw-edma/dw-edma-core.c | 116 +++++++++++++++++++++-------- 1 file changed, 86 insertions(+), 30 deletions(-) diff --git a/drivers/dma/dw-edma/dw-edma-core.c b/drivers/dma/dw-edma/dw-ed= ma-core.c index eb0fc73a4f47..1cc9fa4866b9 100644 --- a/drivers/dma/dw-edma/dw-edma-core.c +++ b/drivers/dma/dw-edma/dw-edma-core.c @@ -459,6 +459,13 @@ static void dw_edma_finish_termination(struct dw_edma_= chan *chan) chan->status =3D EDMA_ST_IDLE; } =20 +/* Must be called with vc.lock held after the channel has stopped. */ +static void dw_edma_finish_pause(struct dw_edma_chan *chan) +{ + dw_edma_set_request(chan, EDMA_REQ_NONE); + chan->status =3D EDMA_ST_PAUSE; +} + /* * Must be called with vc.lock held. Consume an LL progress boundary, * advance ll_done and complete descriptors covered by the consumed range. @@ -692,8 +699,9 @@ static void dw_edma_core_ch_doorbell(struct dw_edma_cha= n *chan) */ static bool dw_edma_core_ch_maybe_doorbell(struct dw_edma_chan *chan) { - if (chan->non_ll || chan->request !=3D EDMA_REQ_NONE || - chan->status !=3D EDMA_ST_BUSY || !dw_edma_ll_pending(chan) || + if (chan->non_ll || + chan->status !=3D EDMA_ST_BUSY || + !dw_edma_ll_pending(chan) || dw_edma_abort_is_pending(chan)) return false; =20 @@ -704,9 +712,25 @@ static bool dw_edma_core_ch_maybe_doorbell(struct dw_e= dma_chan *chan) if (dw_edma_core_ch_status(chan) =3D=3D DMA_IN_PROGRESS) return true; =20 - dw_edma_ll_reconcile_and_refill(chan); - if (!dw_edma_ll_pending(chan)) - return false; + /* + * Normal work may refill a stopped tail. STOP and PAUSE freeze the + * producer boundary, so only kick entries published before the request. + */ + if (chan->request =3D=3D EDMA_REQ_NONE) { + dw_edma_ll_reconcile_and_refill(chan); + if (!dw_edma_ll_pending(chan)) + return false; + } else { + dw_edma_ll_reconcile_stopped(chan); + if (!dw_edma_ll_pending(chan)) { + if (chan->request =3D=3D EDMA_REQ_STOP) + dw_edma_finish_termination(chan); + else + dw_edma_finish_pause(chan); + + return false; + } + } =20 dw_edma_core_ch_doorbell(chan); if (!dw_edma_ll_has_hdma_stop_event(chan)) @@ -752,8 +776,7 @@ static void dw_edma_ll_recheck_work(struct work_struct = *work) } =20 chan->ll_recheck_at =3D 0; - if (chan->request =3D=3D EDMA_REQ_NONE) - dw_edma_core_ch_maybe_doorbell(chan); + dw_edma_core_ch_maybe_doorbell(chan); } =20 static void dw_edma_device_caps(struct dma_chan *dchan, @@ -845,6 +868,16 @@ dw_edma_device_get_config(struct dma_chan *dchan, return &chan->config; } =20 +/* Must be called with vc.lock held. */ +static void dw_edma_request_pause(struct dw_edma_chan *chan) +{ + dw_edma_set_request(chan, EDMA_REQ_PAUSE); + if (!chan->non_ll && !dw_edma_ll_pending(chan)) + dw_edma_finish_pause(chan); + else + dw_edma_core_ch_maybe_doorbell_or_recheck(chan); +} + static int dw_edma_device_pause(struct dma_chan *dchan) { struct dw_edma_chan *chan =3D dchan2dw_edma_chan(dchan); @@ -859,7 +892,7 @@ static int dw_edma_device_pause(struct dma_chan *dchan) else if (chan->request !=3D EDMA_REQ_NONE) err =3D -EPERM; else - dw_edma_set_request(chan, EDMA_REQ_PAUSE); + dw_edma_request_pause(chan); =20 return err; } @@ -887,35 +920,52 @@ static int dw_edma_device_resume(struct dma_chan *dch= an) return err; } =20 +/* Must be called with vc.lock held. */ +static bool dw_edma_may_be_active(struct dw_edma_chan *chan) +{ + if (chan->non_ll) + return dw_edma_core_ch_status(chan) =3D=3D DMA_IN_PROGRESS; + + if (!dw_edma_ll_pending(chan)) + return false; + + /* + * eDMA can report STOPPED while a doorbell-triggered start is + * still pending. Native HDMA has a dedicated STOP event. + */ + return !dw_edma_ll_has_hdma_stop_event(chan) || + dw_edma_core_ch_status(chan) =3D=3D DMA_IN_PROGRESS; +} + static int dw_edma_device_terminate_all(struct dma_chan *dchan) { struct dw_edma_chan *chan =3D dchan2dw_edma_chan(dchan); - int err =3D 0; =20 guard(spinlock_irqsave)(&chan->vc.lock); =20 if (!chan->configured) { dw_edma_terminate_all_descs(chan); + chan->request =3D EDMA_REQ_NONE; } else if (chan->status =3D=3D EDMA_ST_PAUSE) { + /* A paused channel has already stopped. */ dw_edma_finish_termination(chan); - } else if (chan->status =3D=3D EDMA_ST_IDLE) { - dw_edma_finish_termination(chan); - } else if (dw_edma_core_ch_status(chan) =3D=3D DMA_COMPLETE) { + } else if (dw_edma_may_be_active(chan)) { /* - * The channel is in a false BUSY state, probably didn't - * receive or lost an interrupt + * Keep published entries intact until an IRQ sample confirms + * that the channel stopped. An outstanding EDMA_REQ_PAUSE is + * replaced by EDMA_REQ_STOP. */ - dw_edma_finish_termination(chan); - } else if (chan->request > EDMA_REQ_PAUSE) { - err =3D -EPERM; - } else { dw_edma_set_request(chan, EDMA_REQ_STOP); + dw_edma_core_ch_maybe_doorbell_or_recheck(chan); + } else { + /* + * No LL entry is hardware-owned, or non-LL status confirms that + * the one programmed burst is not running. + */ + dw_edma_finish_termination(chan); } - if (chan->status =3D=3D EDMA_ST_IDLE && - !dw_edma_abort_is_pending(chan)) - dw_edma_set_request(chan, EDMA_REQ_NONE); =20 - return err; + return 0; } =20 static void dw_edma_device_issue_pending(struct dma_chan *dchan) @@ -1256,8 +1306,7 @@ static void dw_edma_done_interrupt(struct dw_edma_cha= n *chan) } =20 if (chan->request =3D=3D EDMA_REQ_PAUSE) { - dw_edma_set_request(chan, EDMA_REQ_NONE); - chan->status =3D EDMA_ST_PAUSE; + dw_edma_finish_pause(chan); break; } =20 @@ -1300,8 +1349,7 @@ static void dw_edma_ll_interrupt(struct dw_edma_chan = *chan) break; =20 case EDMA_REQ_PAUSE: - dw_edma_set_request(chan, EDMA_REQ_NONE); - chan->status =3D EDMA_ST_PAUSE; + dw_edma_finish_pause(chan); break; =20 case EDMA_REQ_STOP: @@ -1598,19 +1646,27 @@ static void dw_edma_wait_termination(struct dma_cha= n *dchan) /* * A STOP may be deferred to a later interrupt while the channel is still * running. Wait until that handler completes the termination. + * + * dmaengine_synchronize() cannot report an error. Returning while + * EDMA_REQ_STOP remains pending would let callers release transfer + * resources even though hardware access cannot be ruled out. Warn + * periodically, but keep waiting. */ - while (time_before(jiffies, timeout)) { + for (;;) { scoped_guard(spinlock_irqsave, &chan->vc.lock) stopping =3D chan->request =3D=3D EDMA_REQ_STOP; =20 if (!stopping) return; =20 + if (time_after_eq(jiffies, timeout)) { + dev_warn(chan->dw->chip->dev, + "timeout waiting for channel termination; still waiting\n"); + timeout =3D jiffies + msecs_to_jiffies(5000); + } + fsleep(1000); } - - dev_warn(chan->dw->chip->dev, - "timeout waiting for channel termination\n"); } =20 static void dw_edma_device_synchronize(struct dma_chan *dchan) --=20 2.51.0 From nobody Tue Sep 29 04:14:12 2026 Received: from OS0P286CU010.outbound.protection.outlook.com (mail-japanwestazon11021092.outbound.protection.outlook.com [40.107.74.92]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B49C9478E51; Wed, 12 Aug 2026 15:57:49 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.74.92 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1786550273; cv=fail; b=M3Sf+CkbpZFUsIe5b4UEEWIov2gJ40dxOAaPSc/dbEtL2QJTSGg2846l+BV8ozPiFoEtAMVcmrers0glCucQ0XAHK76+BoAuvLmiW2O7VZTX8wh6X/XnmwyTxvz4WBtnfqMO4oBHFUj5H155MCleSOY22dsfscLJzkQL+Y5IYWc= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1786550273; c=relaxed/simple; bh=frW5ABWRt/v3Fwj/W1ZVkV4P5Km6GreXGyvNEFsuHtU=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: Content-Type:MIME-Version; b=lZmC2MTOkM8pKOz9OqqJERYtVIdxXxpXIRVVM4u+1WFFNt7EZN6nEIIp03ZbLFa0zXO94izBz4cTDp8FuKPuxlf1k8/p0WzQjHjI06WKnL7ysmhXPNemAVUwWoEdOXs5ummaPhBOdsGdb4Cd/A3UkT+DJ2HbD59KKJ9vuij5Nrk= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=valinux.co.jp; spf=pass smtp.mailfrom=valinux.co.jp; dkim=pass (1024-bit key) header.d=valinux.co.jp header.i=@valinux.co.jp header.b=ac2KWyXV; arc=fail smtp.client-ip=40.107.74.92 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=valinux.co.jp Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=valinux.co.jp Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=valinux.co.jp header.i=@valinux.co.jp header.b="ac2KWyXV" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=TMehn+v8MaQT3WwSFJt79xe2qhp2CwJ/0d7D6tajRKA+NhsecdTcNrHwkmkw+JPMh/8PHOQG4MKv8Tux3AZx0Nvt/6cCehOXAT6iQctAOlj2dXtB8kD6dxfDEZMQnqHUPvbFiWRWCz0+MRsYBNMqI2SUKb/O+K/eSfxHvFfVZaXJWLUrVTi9Ig9ZjHxsLfGQ5/57byFIYsOLThpkuVHpdtqCRTw0UVQLfLOqt8mZmQUK5v075DNxyfJcQx3qozUUHlkkrWPGBpTs7/COUs9TekU3tUarOe7XtFrf8LQuL1miOnnwIe9oF3cTQ8ufePz1OXsQEyB96qU1MnbcmDzSew== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=zEDqvCGM9iLqzqaNMum+V7IEieodC8mnDQyal/XX6jg=; b=EM6SA3rmztzlZ87firK8/yH41l4ZEnalwiDiEuwnKI0l4AmemwtQob8xfl7jGJpbkzjtPLnkE2Q42LDPTzso9YgFOv88fzJvx6inZEdcDwLwMunIz1uZ5caO4+Uq9/vAk/E5i9XD1nrLwaO2KMUKJoQ8eGWxbAswLIjqCW5w2UFRTkHjlRkngxEKh7vsuOeFRE6b0jGdbEMFs7LIPbgXL0asrKEYDDTLh6F6GCUCmDhyEOTD0rcy/97TKVo0x5BSeGFRJ1OtwIyeggCHGPGbaTP6ai+OjLKl0M8d0uP7qtQPwYFgMe7YYjWUGYfT8gfwA8gwRPIWpTAylTMjSp3gig== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=valinux.co.jp; dmarc=pass action=none header.from=valinux.co.jp; dkim=pass header.d=valinux.co.jp; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=valinux.co.jp; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=zEDqvCGM9iLqzqaNMum+V7IEieodC8mnDQyal/XX6jg=; b=ac2KWyXVIw7cLtwBu5ULD7QC6XGCDkvtrdzHT2pC6SJFQUCn5QY8gxBBhqkXEA8yiMSJ1irSSAxyJqkIp6CCuPQ4gyqk9bzoOhEG0xLM2+3il7kOBlia/24cwp43IXybXoO6GHavZkcX0V0WXgoOVd82seLK3k673t2mHEguWQ0= Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=valinux.co.jp; Received: from TY7P286MB7722.JPNP286.PROD.OUTLOOK.COM (2603:1096:405:38f::10) by TY4P286MB7069.JPNP286.PROD.OUTLOOK.COM (2603:1096:405:343::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.315.13; Wed, 12 Aug 2026 15:57:41 +0000 Received: from TY7P286MB7722.JPNP286.PROD.OUTLOOK.COM ([fe80::2305:327c:28ec:9b32]) by TY7P286MB7722.JPNP286.PROD.OUTLOOK.COM ([fe80::2305:327c:28ec:9b32%3]) with mapi id 15.21.0292.024; Wed, 12 Aug 2026 15:57:41 +0000 From: Koichiro Den To: Vinod Koul , Frank Li , Manivannan Sadhasivam , Gustavo Pimentel , Kees Cook , =?UTF-8?q?Krzysztof=20Wilczy=C5=84ski?= , Kishon Vijay Abraham I , Bjorn Helgaas , Christoph Hellwig , Serge Semin , Cai Huoqing , Niklas Cassel Cc: Devendra K Verma , dmaengine@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH v5 20/24] dmaengine: dw-edma: Dynamically append requests while running Date: Thu, 13 Aug 2026 00:57:17 +0900 Message-ID: <20260812155721.2807506-21-den@valinux.co.jp> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20260812155721.2807506-1-den@valinux.co.jp> References: <20260812155721.2807506-1-den@valinux.co.jp> Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: TY4PR01CA0090.jpnprd01.prod.outlook.com (2603:1096:405:37d::8) To TY7P286MB7722.JPNP286.PROD.OUTLOOK.COM (2603:1096:405:38f::10) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: TY7P286MB7722:EE_|TY4P286MB7069:EE_ X-MS-Office365-Filtering-Correlation-Id: 725c2e77-fbe0-43ff-c9a6-08def88a70fd X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|1800799024|10070799003|366016|376014|23010399003|7416014|18002099003|22082099003|56012099006|10067099003|921020; X-Microsoft-Antispam-Message-Info: sIBY3CykhRWmZvJW0SetYjJqexvtZIzZH7ptJ9S1lYwF+9okPQ0FDkx10t0X7RsiAVfJHSAdERgnSrUo4awYt5VjSi2k0zvbxepofBKDwjTGllOpmGzt4wh/yuv0Zsf6xwWTz0SFopk6s9/5jElD84ao7h6J4ju09KdZ72uWQmifDpL9dvOYVqCILzdyqTy87sKj1tST9qgWzwXNbX1V/6sLCzvtD9UVOVCxbs2xMArJ+lV+DfYjcIXcawzOlv4oDTO4Tkc4NWHoQ3MtsynkBt17KVX6cD0jOpBeh6Na6PBOg7tVXRKN5xsJn/dqx/CcjvRKNuA0DAyUVNZ/i99VBWEEwdsLzUOJqAzKjrSXGyPLcT4CzQNT8EbopB6rZv4itbaIGnSphR9DshDmBAzFBUj71YCpnGmkyCpBClsN+TRBkPtjYA6GfzrNQcPQYcnAJosoQgltGPMu4HSY7Vs+FEaaOUdcL3vHq+VlksjQN2qafqOUihtYjfN5UKMG8cgF6eCQ26CC+49028q1k9ajz4XGc6zKscxGfUTLLkvH77AkaNa6ZJ5EnZarj7MGTL0XAx7nUTB9HLdHcCyobe1LVpHDqRmFMQodH6U46O0PIZhrAeW6GyPCrpxKdETHM1J8sreFxknBcXlwSWnUjasvYYxOSmOvaPZBAkqP5veCESbWVkbXfkZjUt8mFqjic514DlBkycBHyrweh704HBl+4g== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:TY7P286MB7722.JPNP286.PROD.OUTLOOK.COM;PTR:;CAT:NONE;SFS:(13230040)(1800799024)(10070799003)(366016)(376014)(23010399003)(7416014)(18002099003)(22082099003)(56012099006)(10067099003)(921020);DIR:OUT;SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?Bnqtxrb9s1Z2ma2kIrMoA4Y9VaFcI7Rwc6qtTgVLEtn0UX3LSiCUwUmqCZNg?= =?us-ascii?Q?hT2Z8AOEN2VMUcb03CyrwwC7qt32z7b1pfVrfbTJANV6WD6qh4qVeMLZVrOU?= =?us-ascii?Q?pc0qdIu11Zn++ysABMfhyc61ILMN22slu6Ls2NNMmX5l8hCHNJsIWzpJmpaD?= =?us-ascii?Q?PnL7cuvAqWRprTxLKcS7oLa7SU/tYzXlw2yQdv/g1Hpkoxdso9ZKzjZYwh9T?= =?us-ascii?Q?lJp+GVxz4ciba4MZXbSlGF6IcT22T6WWYaO8fgJidbHKYpP9U9VETtsRuJzg?= =?us-ascii?Q?cbwvqB5/kNC2R+BVuP/wlYii5LK1NoKgB7aPx7DcST2GC99kysxosaCcohpM?= =?us-ascii?Q?IXTeuP6PS7NmAUVaEk1HpXIm8JtJ8fKACtRLbTr90OTx/0R5tKNBavinHBMQ?= =?us-ascii?Q?BbZo1NqRVBhyzFMTXZFZwK+BC77WOOBgg9705t5yxfBIHKvoCaKlKPZn9Fre?= =?us-ascii?Q?QWeQtTJMWFyn8tCRpWQwhl9uJFEgBFYWCrg5QWssqp3FGKdoQLmce2I4RDe0?= =?us-ascii?Q?mxheHVVl6C/hq2hMd1m+pDPWbyxDJ0IXQShKGDI/AxTvRSmVM/2MMRf9F13+?= =?us-ascii?Q?ui0niEHipChLWKaaitQCBmpWzTQuMwsfueOGROC9NsLY1vfkV1fc9IfbAR1k?= =?us-ascii?Q?Z+fuWOUQAGYBknGrJhelduvaMoH50Adg+39JpqM2SlqBPT+zXbEWYOD/IElY?= =?us-ascii?Q?lvAIxzFwzzc0mtboimJ5XGLA4g2VQrN1xUO0DyPNQ2zhWrV8ocs+99RjWf1R?= =?us-ascii?Q?X48ubiHZaTKXMhyJLIjA2NjyFy5m38vqpDic4AkMhoEJscrhggibajI6rXyq?= =?us-ascii?Q?4LnIjKWm6AgqqEv7QtJ5E8sDTan8SN2MT/hNDpdyhT+OBPG+Pc0eDBKCucii?= =?us-ascii?Q?nnjkSADkLf1qCvJ6E/EkVC6Kp8VNlYcCEGXjbkZ2dGS2qsMn+C0RlXih5rGT?= =?us-ascii?Q?mJyf1NECszPNKVquP4Hen2mNDj7qAFgRmf3D6cDZyWe5vFubGhyxwL7MXq6L?= =?us-ascii?Q?cDHPKYQDUyiBlh6JMQ58qGynn5x5K3eupKw0vgBGohRML3LXiSIuL28d8IJO?= =?us-ascii?Q?WYgWvSVOY02QvJfCuqdxEhXbhhFHHRVi8mNJp8Zri+elSZVhK9ewzTDRhtE3?= =?us-ascii?Q?8BUBtHVKBch1FgpfolAEjLK434Ram+NYY6HWuHx+sqghY0ntCXo10NGcynBg?= =?us-ascii?Q?ckwyGq18CswQRYNts73ok0aFjHYB/7U7jZMnP1u3bEQX7ZdXze1nlcSmHWRV?= =?us-ascii?Q?57ueBykijuWGPxVqcf6dc57taJWYA5yYGGbl01Cm65chqLU7NtxBNnjPU4J6?= =?us-ascii?Q?lSB9ZyoidK+WZXTVsxlSnNT3XzVfk8p3g55Y9DBEBkufzL1CIpSh4Nxcmqc4?= =?us-ascii?Q?Llcsh0Te9PTy2+rmO+8ezojHaxLvgmvmB+txI7LkldzCYij2gD0uqtZFGco/?= =?us-ascii?Q?VVfDcM2o/C7m/7a9rEqrqHhW6nauEq9/ZONXGk9EvDUu2hKkw/oDZPf15qoV?= =?us-ascii?Q?YTTlshAtZrTM9hYHmB8C91Ye5O1sjdSvaCg9PkmfZ0Yj293UDdkBU68GPh4k?= =?us-ascii?Q?DJ5BJdDgdCwCgIqC7JKvg4IstIpXN/wm/d7D3HCWzIgMC2/CBq7OfdM5Ffup?= =?us-ascii?Q?P7Cqo+BcO1PEOiID9bq79q1QPM0sWQQzO+8G0YsTjF5YCTR/mUoLTJJv5yXy?= =?us-ascii?Q?LbZgyZ1BSvSOLJ48WMvGdi4QS9c7Cr+R5MmTAuazgfYOjcbL1Cird2dRzoRM?= =?us-ascii?Q?S4LQgUz685uy7OlJ10+/4ofVp6cPBT6B4+Yv7CQa8456YyeGGheT?= X-OriginatorOrg: valinux.co.jp X-MS-Exchange-CrossTenant-Network-Message-Id: 725c2e77-fbe0-43ff-c9a6-08def88a70fd X-MS-Exchange-CrossTenant-AuthSource: TY7P286MB7722.JPNP286.PROD.OUTLOOK.COM X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 12 Aug 2026 15:57:41.8705 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 7a57bee8-f73d-4c5f-a4f7-d72c91c8c111 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: m43Lf0BmT2YeGk6pEqO4RZ3CYmdraUbGaD8JaWe08jWeRA1iSOKkw4VquxEdfQTI0jAJw4X64wcfx6suGhE0wg== X-MS-Exchange-Transport-CrossTenantHeadersStamped: TY4P286MB7069 Content-Type: text/plain; charset="utf-8" From: Frank Li Publish issued descriptors into free LL entries while the channel runs. Skip descriptors already fully published and continue with later ones. Request periodic LL progress while more issued work follows, so reclaimed slots can be used to publish those later descriptors. For LL channels, dw_edma_start_transfer() reports only whether this pass published entries. Derive BUSY or IDLE from pending ring work instead. Let issue_pending() feed a BUSY LL channel when neither EDMA_REQ_STOP nor EDMA_REQ_PAUSE is pending; non-LL channels remain IDLE-only. All kicks still use the common doorbell path, so running channels continue without another kick and stopped channels use the reconciliation and restart paths. Signed-off-by: Frank Li Co-developed-by: Koichiro Den Signed-off-by: Koichiro Den --- Changes in v5: - No changes. drivers/dma/dw-edma/dw-edma-core.c | 61 +++++++++++++++++++----------- 1 file changed, 38 insertions(+), 23 deletions(-) diff --git a/drivers/dma/dw-edma/dw-edma-core.c b/drivers/dma/dw-edma/dw-ed= ma-core.c index 1cc9fa4866b9..ed5654c63933 100644 --- a/drivers/dma/dw-edma/dw-edma-core.c +++ b/drivers/dma/dw-edma/dw-edma-core.c @@ -313,8 +313,12 @@ static bool dw_edma_core_enable_ll_irq(struct dw_edma_= desc *desc, u32 i, if (chan->ll_head =3D=3D chan->ll_max - 1) return true; =20 - /* Add periodic progress points only while this descriptor does not fit. = */ - if (desc->nburst - i <=3D free) + /* + * Add periodic progress points only while this descriptor does not fit + * in the current free space or more issued work follows it. + */ + if (desc->nburst - i <=3D free && + list_is_last(&desc->vd.node, &chan->vc.desc_issued)) return false; =20 return (chan->ll_head + 1) % DW_EDMA_LL_PROGRESS_INTERVAL =3D=3D 0; @@ -381,35 +385,42 @@ static int dw_edma_start_transfer(struct dw_edma_chan= *chan) { struct dw_edma_desc *desc; struct virt_dma_desc *vd; + int ret =3D 0; =20 - if (!chan->non_ll) { - if (dw_edma_abort_is_pending(chan)) + if (chan->non_ll) { + vd =3D vchan_next_desc(&chan->vc); + if (!vd) return 0; - if (!chan->ll_valid) - dw_edma_core_reset_ll(chan); - } - - vd =3D vchan_next_desc(&chan->vc); - if (!vd) - return 0; =20 - desc =3D vd2dw_edma_desc(vd); - if (!desc) - return 0; - - if (chan->non_ll) { guard(spinlock_irqsave)(dw_edma_event_lock(chan)); =20 if (dw_edma_abort_latch_locked(chan)) return 0; =20 - dw_edma_core_start(desc); + dw_edma_core_start(vd2dw_edma_desc(vd)); return 1; } =20 - dw_edma_core_start(desc); + if (dw_edma_abort_is_pending(chan)) + return 0; + + if (!chan->ll_valid) + dw_edma_core_reset_ll(chan); =20 - return 1; + list_for_each_entry(vd, &chan->vc.desc_issued, node) { + if (!dw_edma_core_get_free_num(chan)) + break; + + desc =3D vd2dw_edma_desc(vd); + /* A fully published descriptor may still be pending in hardware. */ + if (desc->start_burst =3D=3D desc->nburst) + continue; + + dw_edma_core_start(desc); + ret =3D 1; + } + + return ret; } =20 static void dw_edma_terminate_vdesc(struct virt_dma_desc *vd) @@ -900,6 +911,7 @@ static int dw_edma_device_pause(struct dma_chan *dchan) static int dw_edma_device_resume(struct dma_chan *dchan) { struct dw_edma_chan *chan =3D dchan2dw_edma_chan(dchan); + bool active; int err =3D 0; =20 guard(spinlock_irqsave)(&chan->vc.lock); @@ -911,9 +923,10 @@ static int dw_edma_device_resume(struct dma_chan *dcha= n) } else if (chan->request !=3D EDMA_REQ_NONE) { err =3D -EPERM; } else { - chan->status =3D EDMA_ST_BUSY; - if (!dw_edma_start_transfer(chan)) - chan->status =3D EDMA_ST_IDLE; + active =3D dw_edma_start_transfer(chan); + if (!chan->non_ll) + active =3D dw_edma_ll_pending(chan); + chan->status =3D active ? EDMA_ST_BUSY : EDMA_ST_IDLE; dw_edma_core_ch_maybe_doorbell_or_recheck(chan); } =20 @@ -974,9 +987,11 @@ static void dw_edma_device_issue_pending(struct dma_ch= an *dchan) unsigned long flags; =20 spin_lock_irqsave(&chan->vc.lock, flags); + /* Only LL channels can accept work while already running. */ if (chan->configured && vchan_issue_pending(&chan->vc) && chan->request =3D=3D EDMA_REQ_NONE && - chan->status =3D=3D EDMA_ST_IDLE) { + (chan->non_ll ? chan->status =3D=3D EDMA_ST_IDLE : + chan->status !=3D EDMA_ST_PAUSE)) { if (!chan->non_ll && !dw_edma_ll_pending(chan)) dw_edma_ll_snapshot_discard(chan); chan->status =3D EDMA_ST_BUSY; --=20 2.51.0 From nobody Tue Sep 29 04:14:12 2026 Received: from TY3P286CU002.outbound.protection.outlook.com (mail-japaneastazon11020143.outbound.protection.outlook.com [52.101.229.143]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 658ED47CA7D; Wed, 12 Aug 2026 15:57:54 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=52.101.229.143 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1786550277; cv=fail; b=tjnRSd8v/Mo763hFqaHvbs7M3zLR+7B0MMMRHZJdIUn/uMDUt/NvB1cpufLSnQsWev/5IcGaPaV5/0jfF6GPp8f7xedamnhaKoRM1+jrXqHlrglrVxY7bIspNk8yCaWuTxr0ViCLgasLlUyL9FTTAiWXKlOUCWeVbSa/O0l4j8s= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1786550277; c=relaxed/simple; bh=qoWc3n5QjxbzHYbDsInNykbYUrRlqhUDifmEf5TPds4=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: Content-Type:MIME-Version; b=M3bfkYdaEbgrLbMAm8weocsun2CDJG2iOm26ePO8OiV+VNudSvNfS/Is06DSU6NsBXMoEgG0k8kBlJegOJkRgnamRN1urdhDygD/tEjaLtrk97Zq00tcBqYtQs6r2jT7WLQ/1LXix/HILCWnf81aC01dDKx6AgwlfjQ0qFvKfG0= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=valinux.co.jp; spf=pass smtp.mailfrom=valinux.co.jp; dkim=pass (1024-bit key) header.d=valinux.co.jp header.i=@valinux.co.jp header.b=dlavqJoi; arc=fail smtp.client-ip=52.101.229.143 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=valinux.co.jp Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=valinux.co.jp Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=valinux.co.jp header.i=@valinux.co.jp header.b="dlavqJoi" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=tFleaEEL8fCc65uV3uoUclP79lv/gZ9zQgKUPrZgMqn6jLJo2H35K1EqhfP7Rw9zxAxHagI+hp8jUwdsQsnrZAfs4vEBx96cb1giyqwkYVCwc7QIEY15P/piHRGgGWQ6WPAofwleYfr4/e5kcM5vHOewNaKZcGE5aXCcvrw/ovXLHnGEqtLsgnMtmbd1eMXY4MzCClCXTDD1dcuKJswC37MAPwhTGh7XJwow5VMu3ojTHwAavtzrfDS3KbJUS3Ha4FxoCGn8227fY64H3j9/0p7NdDJp75W9xtZgYg635HesvT0qFR1a3nb/nlq/VKXkrABYrBDAytCuRMcorNs4rA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=gT8vU9Av733Cgu2bm2j4FKJhiGqyHejYWMdYTPHrSQ4=; b=PI44LknW5KwPfbg0b9QjZvGqxN2rzqnyTfZi/ALffnm5JVwZYyV1qPFXGanzI/C7yD2cHXQ9CewsUq6Cl3piGMWyJVxZWGoG0ruogAt8CsBTZRT0Cj+HyO35EfyJWnQVsKw5bZfLanQ6dUORBr09iWlNRbmBl+5bu/3IMYDftuS7rgZ41cZwE0CZ8RNo5ZvznYi+YacXRbf3X4VYSTM5lM+kHkjBj4nMIyteq3DIipvDqwqUP+odPDEhR4YAQMF6MxbxEZXZ3LccANJY/qTPUUEMEXiAvS0CHv7V2qYbx0w2C+QtKHNyGN2DYvDOGZ6+kDZj/HNcu+Fq6uUTikDf8A== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=valinux.co.jp; dmarc=pass action=none header.from=valinux.co.jp; dkim=pass header.d=valinux.co.jp; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=valinux.co.jp; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=gT8vU9Av733Cgu2bm2j4FKJhiGqyHejYWMdYTPHrSQ4=; b=dlavqJoiBTdk7kvK4MYPSVV1jOTBj0/Fi9pVSVTRBtuvFbxQDbjKnSzTApYTIREbff7dPuOasKYizz43FpZk0Eb33zn8o3dmdJhuvMFQtyt0PQGyYrXiqpSUFV2ggPbsmU04OFZJadrYy1/G86wD/SEDKE+HtNU8YiMxUY1xFnw= Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=valinux.co.jp; Received: from TY7P286MB7722.JPNP286.PROD.OUTLOOK.COM (2603:1096:405:38f::10) by TY4P286MB7069.JPNP286.PROD.OUTLOOK.COM (2603:1096:405:343::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.315.13; Wed, 12 Aug 2026 15:57:42 +0000 Received: from TY7P286MB7722.JPNP286.PROD.OUTLOOK.COM ([fe80::2305:327c:28ec:9b32]) by TY7P286MB7722.JPNP286.PROD.OUTLOOK.COM ([fe80::2305:327c:28ec:9b32%3]) with mapi id 15.21.0292.024; Wed, 12 Aug 2026 15:57:42 +0000 From: Koichiro Den To: Vinod Koul , Frank Li , Manivannan Sadhasivam , Gustavo Pimentel , Kees Cook , =?UTF-8?q?Krzysztof=20Wilczy=C5=84ski?= , Kishon Vijay Abraham I , Bjorn Helgaas , Christoph Hellwig , Serge Semin , Cai Huoqing , Niklas Cassel Cc: Devendra K Verma , dmaengine@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH v5 21/24] dmaengine: dw-edma: Add engine reset and enable operations Date: Thu, 13 Aug 2026 00:57:18 +0900 Message-ID: <20260812155721.2807506-22-den@valinux.co.jp> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20260812155721.2807506-1-den@valinux.co.jp> References: <20260812155721.2807506-1-den@valinux.co.jp> Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: TY4P301CA0016.JPNP301.PROD.OUTLOOK.COM (2603:1096:405:2b1::19) To TY7P286MB7722.JPNP286.PROD.OUTLOOK.COM (2603:1096:405:38f::10) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: TY7P286MB7722:EE_|TY4P286MB7069:EE_ X-MS-Office365-Filtering-Correlation-Id: df25a785-75c0-48be-173f-08def88a717b X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|1800799024|10070799003|366016|376014|23010399003|7416014|18002099003|22082099003|56012099006|6133799003|10067099003|921020; X-Microsoft-Antispam-Message-Info: UJGvNl8VF7qxbu5nnFpUmLZAw99s9LbsJVCoRMjOfZIyUBLC9HWCvFCqd3rufZy4i8//3SVqJgdCmoX7Cu75kMtPimEjRL2hksM8AjPy8bHBy8e8a8tpJ1pYM+Fz7wrR8nXBg63DXerXm0Bv4MspxLsutZRDqp0fPOr4BtjINfqZ1A3fuIyjviVBYiOqAglOiWMEsE8lnsYWekjHq4GcOK0Yxp1CqgzhDfoBBcQ1GcCuQCXWs0oW6R5WipOLjos+Fc0VMFObKk2t7jba9ClXBfLPrhn2VaeGyvCCLk3uRAE71pxDyOEztW9ArxFNWtAVqvYNF66YmQah+Y/FGb54oa9qiuBJRvld+8x4q40nA4N9eu5dL8oWNCGAzSPUbBLx4ANvYqx4kiEgHfWRMvB7uQeSgT6ULpCwV1f3iGTLS2kes2L9duED3ctiCEy4amCEhJCsGC/lLSGkPjmnTXUMe5xAguWG+RV57sC8APpybj+BzzNBV06dsFsr8G4plVQEUuWFRABO6h+ZRybZ9j+8doU6McPSnUlePCTl5O4UxOd8LmZoBIJvYOCG9I1QeT6uZzOLx6wUmEOMiBAQGHVbp8Y1y050AalE5spA2ZzDTPDrvc1GhZq8t2l/M4ByZqXPnE/tsM8aQm4CMrhDF0kduU7dBp+R6tde0SJ3zKGqXN2+RO1UzJgKKb76dA0WikXDZV0L6TqEOJkr34GCascQWQ== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:TY7P286MB7722.JPNP286.PROD.OUTLOOK.COM;PTR:;CAT:NONE;SFS:(13230040)(1800799024)(10070799003)(366016)(376014)(23010399003)(7416014)(18002099003)(22082099003)(56012099006)(6133799003)(10067099003)(921020);DIR:OUT;SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?mFJo0m/GV5m8vCObd8Q/DWHyi/61SqaA/KLslRYAQ+EU6HY2GVm36TeTIE3q?= =?us-ascii?Q?rI+dNOZqNiYCdgInBYOwl2T5yMtMaRDu06fAhNeyGzCXRNsnbC6ZiMoSeQ2W?= =?us-ascii?Q?Fa4pSeGpB5B+cZ3afV9G8/nFW9GeWAb+bF88ZhdxP+tZg7BVRrCAuNnGTrcO?= =?us-ascii?Q?j2rtld/QlGuLhG/2E4+nLEASOceLHuFs9FfAqUfdv0+72e+tEULJ6vT13mBW?= =?us-ascii?Q?qbVK0awuBQNPL9PrGCDWa4ciGpSIXyGmVDuDNhhaFtGQ8NMLSYMOdoDq+tR9?= =?us-ascii?Q?MdHUy18RAwLkosy7fAP9xNUyX8VheBDx5c3HtBgAJHkNQqDE9HwTreMtjEXo?= =?us-ascii?Q?sk9+/JV0JpxJoLJpTAJuLVZPphljcldYRTgD24onlng22NiMhGh/5frJUGkk?= =?us-ascii?Q?DGF3znUJEA+9yn50os+j7oiYa2sNcxyur4aXqFTSaip9DMKSE0jrQ9+pj99T?= =?us-ascii?Q?5hoDlSZNzwmueztLXIpGHrhXzYhtpfvtQ/Yl8dkP2mmizKULsOfyDwf/WbWS?= =?us-ascii?Q?RXW24cvKRrISiuhdjgwrrzipicGFpaR1giegiNZ/KDTjPj7FQygIyzrX1Ft8?= =?us-ascii?Q?/rUSuSeo2RVTgkbz5dIqNCZ6FKm72nJrmpL62vNrpH7VQIPhvrZ+4YagHcd4?= =?us-ascii?Q?wSYoiit9ZZ0SrUQGjS3skaaK1UIQil9MpyN6RI6Tw9uBZ0/H/AaEEmBXxEKs?= =?us-ascii?Q?mtbqbO/7axfn7wyRRl+0AlBL8qzGcvDYbzcY8HGGXQ5etFx2LCVG0uNoo1r/?= =?us-ascii?Q?U9yrBwcubWMLT+CAJneKVDvO7w0YH34TBjllMXyLYM142sslXKJ2ch0et/kq?= =?us-ascii?Q?ntQVsKqc85FESPazHKU8jxWPjEQF4Un3Wn2H7Fl8nVfV28ajFHumXH39wimJ?= =?us-ascii?Q?hxZGjr9/L8Dc33e7CmhN+nkWOHO0HCByIECKyIBY8NkOGum7vhszj4Pj8R2r?= =?us-ascii?Q?Iy4QqYLER9P2Ovw0DgVaDM3SoOYjH4NEpSLgmP6XtTGiNxtLhtQOqTRW5zao?= =?us-ascii?Q?8I+NfHVA41x3rXu3qxmMesBzXjp61qJtn6xNw3vwvvo+YhXcAupr96Mc8gqK?= =?us-ascii?Q?haKUItBB5oJVH+4UlW3S6ni9KMMMj5duVV4281jm0K5dUMPPjBadZlas3sZ8?= =?us-ascii?Q?PR2IimQxY+h9YrnsQ6FpVvmQSE1tw0QzobB/0RlXEiYB1PQBF40IM4NbiKiN?= =?us-ascii?Q?dfuKcq3FWFiYUWvR6RX5CmC/3AVLCj5367GoU8Lj64HnzloFeoGtUcp7ZvlN?= =?us-ascii?Q?H1m15lfNPNwl/2nNUnjoQmBKRMUP30p8VW8zhBPEBvUj8c5XKKpsLTLk6bin?= =?us-ascii?Q?kGtm4vKg+uRKBzRffYSVP3oA/i5BIDq5K5SAx9h/aiIG2CjAV8Dy0UyLaQbk?= =?us-ascii?Q?zMGyVDyYhj/k8itJxkHVFYwaIwUHc7G5SLjnarbGvRBmQ5LCygL13hFgFGTX?= =?us-ascii?Q?O8xkNlrAYhUO4Dqohh6lGtGnehYKTEDtNtq3fAkwYZBU3ifdbKYHDz7PjBXN?= =?us-ascii?Q?pt4aPpRD7ysGYNjU7Qc+BV0vXzbh2L98du9oZ0TWnWb51H/lh1lasnctQNLE?= =?us-ascii?Q?znFGUIpe11V+Yn3mbHvKzCGVxqtfckKzFkPSZPPS8XvxE7GHeGx4QLJa0tNl?= =?us-ascii?Q?/4ERlI8EDsH9sty1SIYGjX06O1sy6e+lstOVuSEcX2AYKbLCrrKYgKkua2Hv?= =?us-ascii?Q?Y1ljCE95I183YHI6iGk83fuuPc/S4FQSik1jglFV7M6sFZN2yXxRKV1tgy5M?= =?us-ascii?Q?f63wr/BHWisMkS4zzgEWuu+VM1yAGbrAYcEZMY51PE/KYQ5h/AC1?= X-OriginatorOrg: valinux.co.jp X-MS-Exchange-CrossTenant-Network-Message-Id: df25a785-75c0-48be-173f-08def88a717b X-MS-Exchange-CrossTenant-AuthSource: TY7P286MB7722.JPNP286.PROD.OUTLOOK.COM X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 12 Aug 2026 15:57:42.6778 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 7a57bee8-f73d-4c5f-a4f7-d72c91c8c111 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 0+vHYWMVcCJig9vXVFEtJaRXkOZCDh8opnPCV2hp+o8nIUg/XBaYfuG5PI1EBP4X2sCnV4ouHeyHBJ6vyhhgoA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: TY4P286MB7069 Content-Type: text/plain; charset="utf-8" Direction recovery must keep the engine disabled while all exposed LL channel contexts are rebuilt. Add separate engine_reset and engine_enable operations. For eDMA, reset clears ENGINE_EN, waits for it to read back zero, then clears latched DONE and ABORT status. If ENGINE_EN does not clear, dump the state and do not try to enable it again. A separate enable operation lets the caller rebuild channel contexts first. Enable the direction after channel context programming. Reject reset and keep enable a no-op in HDMA compatibility mode, which does not expose ENGINE_EN. Signed-off-by: Koichiro Den --- Changes in v5: - No changes. drivers/dma/dw-edma/dw-edma-core.h | 5 ++ drivers/dma/dw-edma/dw-edma-v0-core.c | 67 ++++++++++++++++++++++++++- 2 files changed, 70 insertions(+), 2 deletions(-) diff --git a/drivers/dma/dw-edma/dw-edma-core.h b/drivers/dma/dw-edma/dw-ed= ma-core.h index aaaee312e973..329f02c58995 100644 --- a/drivers/dma/dw-edma/dw-edma-core.h +++ b/drivers/dma/dw-edma/dw-edma-core.h @@ -210,6 +210,9 @@ struct dw_edma_core_ops { int (*ll_cur_idx)(struct dw_edma_chan *chan); /* Called with dw_edma_event_lock(chan) held. */ void (*ll_irq_clear)(struct dw_edma_chan *chan); + /* Reset one direction, clear its IRQ status, and leave it disabled. */ + bool (*engine_reset)(struct dw_edma *dw, enum dw_edma_dir dir); + void (*engine_enable)(struct dw_edma *dw, enum dw_edma_dir dir); /* Called with dw_edma_event_lock(chan) held for an LL channel. */ void (*ch_doorbell)(struct dw_edma_chan *chan); void (*ch_enable)(struct dw_edma_chan *chan); @@ -371,6 +374,8 @@ static inline void dw_edma_core_do_ch_doorbell(struct d= w_edma_chan *chan) static inline void dw_edma_core_ch_enable(struct dw_edma_chan *chan) { chan->dw->core->ch_enable(chan); + if (chan->dw->core->engine_enable) + chan->dw->core->engine_enable(chan->dw, chan->dir); } =20 static inline diff --git a/drivers/dma/dw-edma/dw-edma-v0-core.c b/drivers/dma/dw-edma/dw= -edma-v0-core.c index b9d6205157b3..455fab945014 100644 --- a/drivers/dma/dw-edma/dw-edma-v0-core.c +++ b/drivers/dma/dw-edma/dw-edma-v0-core.c @@ -220,6 +220,69 @@ static int dw_edma_v0_core_engine_disable(struct dw_ed= ma *dw, return ret; } =20 +static void dw_edma_v0_core_engine_dump(struct dw_edma *dw, + enum dw_edma_dir dir) +{ + u16 cnt =3D dir =3D=3D EDMA_DIR_WRITE ? dw->wr_ch_cnt : dw->rd_ch_cnt; + u16 i; + + dev_warn(dw->chip->dev, "%s engine: int_status 0x%08x err_status 0x%08x\n= ", + dir =3D=3D EDMA_DIR_WRITE ? "wr" : "rd", + GET_RW_32(dw, dir, int_status), + dir =3D=3D EDMA_DIR_WRITE ? GET_32(dw, wr_err_status) : + GET_32(dw, rd_err_status.lsb)); + for (i =3D 0; i < cnt; i++) + dev_warn(dw->chip->dev, + " ch%u: ch_control1 0x%08x transfer_size 0x%08x llp.lsb 0x%08x\n", + i, GET_CH_32(dw, dir, i, ch_control1), + GET_CH_32(dw, dir, i, transfer_size), + GET_CH_32(dw, dir, i, llp.lsb)); +} + +static bool dw_edma_v0_core_engine_reset(struct dw_edma *dw, + enum dw_edma_dir dir) +{ + /* HDMA compatibility mode does not implement ENGINE_EN. */ + if (dw->chip->mf =3D=3D EDMA_MF_HDMA_COMPAT) + return false; + + /* + * The databook says clearing ENGINE_EN resets the DMA logic while + * preserving configuration registers, but does not document drain + * semantics. On the tested integration, ENGINE_EN read back as zero + * only after outstanding transactions appeared to drain; the stalled + * channel then resumed after reset. + * + * If ENGINE_EN does not clear, do not re-enable the engine. On the + * tested integration, re-enabling after such a timeout wedged the + * controller and also blocked inbound BAR accesses. Return failure so + * the caller can keep channels gated and retry. + */ + if (dw_edma_v0_core_engine_disable(dw, dir)) { + dw_edma_v0_core_engine_dump(dw, dir); + return false; + } + + /* Discard interrupt status belonging to the contexts being reset. */ + SET_RW_32(dw, dir, int_clear, + EDMA_V0_DONE_INT_MASK | EDMA_V0_ABORT_INT_MASK); + GET_RW_32(dw, dir, int_status); + + return true; +} + +/* + * Re-enabling can resume a previously running channel from preserved + * internal state without a doorbell, so the caller must rebuild every + * channel context first. + */ +static void dw_edma_v0_core_engine_enable(struct dw_edma *dw, + enum dw_edma_dir dir) +{ + if (dw->chip->mf !=3D EDMA_MF_HDMA_COMPAT) + SET_RW_32(dw, dir, engine_en, BIT(0)); +} + static int dw_edma_v0_core_dir_off(struct dw_edma *dw, enum dw_edma_dir di= r) { u16 count, id; @@ -461,8 +524,6 @@ static void dw_edma_v0_core_ch_enable(struct dw_edma_ch= an *chan) unsigned long flags; u32 tmp; =20 - /* Enable engine */ - SET_RW_32(dw, chan->dir, engine_en, BIT(0)); if (dw->chip->mf =3D=3D EDMA_MF_HDMA_COMPAT) dw_edma_v0_core_ch_power(dw, chan->dir, chan->id, true); /* Interrupt mask/unmask - done, abort */ @@ -689,6 +750,8 @@ static const struct dw_edma_core_ops dw_edma_v0_core = =3D { .ll_clear =3D dw_edma_v0_core_ll_clear, .ll_cur_idx =3D dw_edma_v0_core_ll_cur_idx, .ll_irq_clear =3D dw_edma_v0_core_clear_done_int, + .engine_reset =3D dw_edma_v0_core_engine_reset, + .engine_enable =3D dw_edma_v0_core_engine_enable, .ch_doorbell =3D dw_edma_v0_core_ch_doorbell, .ch_enable =3D dw_edma_v0_core_ch_enable, .ch_config =3D dw_edma_v0_core_ch_config, --=20 2.51.0 From nobody Tue Sep 29 04:14:12 2026 Received: from OS0P286CU010.outbound.protection.outlook.com (mail-japanwestazon11021092.outbound.protection.outlook.com [40.107.74.92]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id EAFDB47F2EE; Wed, 12 Aug 2026 15:57:56 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.74.92 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1786550281; cv=fail; b=GgLlYrZLM2SpRSrpdRg8K+7g8M9z1bx2tk4ggnPzBNtuUgZKTwF1h7PrMHtS7g2SHf+96uvYJLikZbv+EPyT9LWLQRwlOuqz7nagU3H7wzZ8KZC1FQV+EGUaye9HNm1+xCbmmSgM2Vc/NhtULsjWokpsy2nSe8EIxThws6Y4oRc= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1786550281; c=relaxed/simple; bh=ud4Jzl0KJa84LGI3caI1hPSiMlfS2krmuR0rtWS4M0g=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: Content-Type:MIME-Version; b=i04ZR3Ru/48HtxKP1hTz7DQzDFhEb998Z2QU1u/Kxa3gPi65/9FRXNNfjyRiVxWyEluICEMATpdp1PTmDoAwPKHM+DBRrORIyK22jF1AO0NIqJVY3k/h8dP3slmatwpV5jN9BxT55OqVq9SyOJifqtFlb3idt0K3FEf8cCkZS+U= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=valinux.co.jp; spf=pass smtp.mailfrom=valinux.co.jp; dkim=pass (1024-bit key) header.d=valinux.co.jp header.i=@valinux.co.jp header.b=Bkv/fnVw; arc=fail smtp.client-ip=40.107.74.92 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=valinux.co.jp Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=valinux.co.jp Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=valinux.co.jp header.i=@valinux.co.jp header.b="Bkv/fnVw" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=gJfyHU5eK/8/j4ZEA6bW83nYYvtMUcJKCj4gu1U+USTabgcwdBTAnVAFtU7wBEYtt9+Wb9qTVXbreYsVb+lGL4UnTE4sT8g26dXgz76Nvi/lt4j2vxQ1yYfkXwqmBy6wbW46OQx18xDxVY6VeR9SZfCTDa4A3rTSzkx5FCivuuokAA2eiZDyvXN0tZzXzdj58zwFH7NUsFpvaKzT8JUZ/AeHDJf6UoCKBM77S/Oy2mViylXBeTUfbEIIFRwyLDAWBZYtl5SGSB/DXtqt3OZyRa66VdLKz/AemauflOK1nz6JwEEv8qEgFSRe6Xs+JegPhmgiZySTFLybtc9Cg2Np6Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=e0C3o3VKNDrkEt7ThfP2wZuQan6sxjpQgTcuTyIin88=; b=drLLBXCKDsSVpiRp+3xWKWQd0zC+yweUGQZ72p7nI6Eag9Y6x4oczcyQv/feZMKzcouGpLb0C93Umhn2KA6vXRhO8TSwiX22qj6QYfdpi5nDeHl5bxhVeYnRiGkdaKzjVRHZpsGNOA0IHdG5oTmUw69fcwtT0v1iOKsfHaV4jb1W0wXGgo6rCVLnPG++Esv8idWsD2yLPWWKPY0X6h4ibM/0DHI1Sb9GiTaqpz//vwzyoVhBMsS+IR7g8xk1MqlY8m52clzZS4gDhhC1mnbqpE87j2tb7aTeyjHgURMwiYlQv4wjBRfmuaUDwEz/dI/ozDOeOfsP6j36jcGZGLMTOA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=valinux.co.jp; dmarc=pass action=none header.from=valinux.co.jp; dkim=pass header.d=valinux.co.jp; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=valinux.co.jp; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=e0C3o3VKNDrkEt7ThfP2wZuQan6sxjpQgTcuTyIin88=; b=Bkv/fnVwDmlFn5+5H+0516T4w/6rzOOJrwHtU2bDEjlGqORjY33hqE40h3SK7PJUq5WWvHmleFyAfCPO6dlrmHUJ1RcjLOQHc/QduCUCwxfV+ooHXw6qTJs7pyxi5epZWrl+a117UsjEs6df8MtHPinR2VqxvRSa59MME4TiVss= Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=valinux.co.jp; Received: from TY7P286MB7722.JPNP286.PROD.OUTLOOK.COM (2603:1096:405:38f::10) by TY4P286MB7069.JPNP286.PROD.OUTLOOK.COM (2603:1096:405:343::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.315.13; Wed, 12 Aug 2026 15:57:43 +0000 Received: from TY7P286MB7722.JPNP286.PROD.OUTLOOK.COM ([fe80::2305:327c:28ec:9b32]) by TY7P286MB7722.JPNP286.PROD.OUTLOOK.COM ([fe80::2305:327c:28ec:9b32%3]) with mapi id 15.21.0292.024; Wed, 12 Aug 2026 15:57:43 +0000 From: Koichiro Den To: Vinod Koul , Frank Li , Manivannan Sadhasivam , Gustavo Pimentel , Kees Cook , =?UTF-8?q?Krzysztof=20Wilczy=C5=84ski?= , Kishon Vijay Abraham I , Bjorn Helgaas , Christoph Hellwig , Serge Semin , Cai Huoqing , Niklas Cassel Cc: Devendra K Verma , dmaengine@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH v5 22/24] dmaengine: dw-edma: Add engine recovery infrastructure Date: Thu, 13 Aug 2026 00:57:19 +0900 Message-ID: <20260812155721.2807506-23-den@valinux.co.jp> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20260812155721.2807506-1-den@valinux.co.jp> References: <20260812155721.2807506-1-den@valinux.co.jp> Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: TY4PR01CA0088.jpnprd01.prod.outlook.com (2603:1096:405:37d::9) To TY7P286MB7722.JPNP286.PROD.OUTLOOK.COM (2603:1096:405:38f::10) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: TY7P286MB7722:EE_|TY4P286MB7069:EE_ X-MS-Office365-Filtering-Correlation-Id: 0798e43d-b68f-4ae5-f3be-08def88a7207 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|1800799024|10070799003|366016|376014|23010399003|7416014|18002099003|22082099003|56012099006|6133799003|10067099003|921020; X-Microsoft-Antispam-Message-Info: AHjaJvlwQm3HFrnItdGT1oqtEf5EiYeycVtHiP8CJawCg3NhrqoLZquxY6oBZVHN2hMEkHMA+EYrf8zpwMu4EUwur43vmkOKmq7lmkEaGo7NHGBGaSAdaztwJ/fRxxnt0nwMDTdX8WiQwm0N1yKK5YXLZW7eUFCptq7QikSO7SuAcIAD8rsiz8Kqcos1uCdKoOwzpPRYZw5WiVIjqcgGPKXNYrL7lUkXA4mXC12PExw4jLzkO4HKLjgpQwS9yEVkInLurxtqCXkDbme/c8WX1z1MJoayNbSfKCnC5cmcZ8Bi1mI7BG9UR87BvtMKuIAckntSD7ayLaaa4L4XxoDTAKtL4hdfGrBN6kSX4v5Shzq8rfpFWJys7CL4KOsUtcP1gpr+qOGS3Ke8bSxTa02nCFwIGik8IOp6RXL0IhQLwqPYUxvL7k23wv3MD99cx/h2XKlJ9irN3/6jwsgb3Euizu+OKN08TzjUN1O/rMtFO7Gu6nlQkpg75w9g320A3mQbg+cSQJ1csvtKV3j77Bk8OoetZyphmzsUqDj55HcVVyb764Rqof6phKoIT1cPbxJXX8/NM5ZGMJIkRvp+1/DAa3rQufUNJDGhQKGVrHXR76EITjd5XYzAtQqJLManp4+dtDU/z5MoN3BtBy+38pPx8WDkzMO4cBWCx5b/f0R1Cil+UxTaUdfMPSvIbTiAsADtPFz5g+2VxRjGbRSCCdlVzQ== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:TY7P286MB7722.JPNP286.PROD.OUTLOOK.COM;PTR:;CAT:NONE;SFS:(13230040)(1800799024)(10070799003)(366016)(376014)(23010399003)(7416014)(18002099003)(22082099003)(56012099006)(6133799003)(10067099003)(921020);DIR:OUT;SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?oTKAw3QWYUTAGsM8DogeH8IyMz/SROIlWeLyrbflUywMGTB2++ywq635iYJn?= =?us-ascii?Q?c84ri2lkR+wl43zRtJN03H6JRI/9EtJ56qDHvj++YqnE9QlcAtWH2vX07XAt?= =?us-ascii?Q?NzxlLiY82SF50x5Ls91RT6bNO2A9QSIlAXHDmvxvxhf+zz21s4FtE2bMvR9Z?= =?us-ascii?Q?6Vid6lTLGmZAmFf7UWauXtq0MWRf79UETg5Q8YZ9nXQlgwiz5wglqonoFdf9?= =?us-ascii?Q?RrOkZg11kPVdYgA5eRb8i+zVdN1/DFv+F7vHRZfLyraS1zuewy4YzsxSqwFQ?= =?us-ascii?Q?+1+IlAPOmID8otA9GOVpYWzRKYwbiSx3kreYjLMiOlL6JqwfvXjGVXJlsrNP?= =?us-ascii?Q?Lkjphr51d0rfiHsLA4/ymTymMENsxbB9O/HeH+Cht/YlPyWs6kgv/OEtuS4u?= =?us-ascii?Q?I/5yzDIwurekirUZofs/0RjG6zX5JgZwHhK9ZRM0rCjy6DoRwpq5SpDBQvVj?= =?us-ascii?Q?vMW5wR3dhtFShJeav+HeaM5Z7Hy0qad/Ly+bgoXdRjAtnfoTErYCH+646Jsa?= =?us-ascii?Q?yG9EStJblYAPLYZ6JRwJbQaFRpcESwQhKJV/6QSTyOw6Tcklp9Vqp9GeJiF1?= =?us-ascii?Q?Q41bArjOxXr4v7frmvX9zsruZrTjR+E/DX1d1gJF1YK8RNSh63ru5Pe9Jq3O?= =?us-ascii?Q?edPrNMD/hOmSg1kgyQ2oW4c+Gius6UR6IejgozV4dD3EsI3/I64DO91vXTva?= =?us-ascii?Q?03+a7EEt8ZO7jpBnWWejfMgC9yfh8flR1R8oc1cIY8f2imzvjL8X9I9xfNf1?= =?us-ascii?Q?AME7SEeDdrQz1BeWANzTXm8ptJGN1Z/r4KZTBQZsjwocc0nxktUd6BX9uBcQ?= =?us-ascii?Q?qaniKZ3edW5ow7Es9LgRG0+98ccU7wmwUmCJkXHZM8pVHKXv1XGufgeFAxRZ?= =?us-ascii?Q?Z9fAi8V9LDNWYMaJjsdYOJ2lyPSxcIpoSCIL2f8Pl4s3rwf0HRLxP+z7VOki?= =?us-ascii?Q?eCPYE47507005C9Wb08lhiJoH2Nav7BQfkrWH+s7hTFRC4VSwy4OjMLLwhWo?= =?us-ascii?Q?7lAONaC8j0qYMTe7j/RPOAmZ+lwrcWfrGmxS9/EQxnKaDJr8SFuc5Xy0qssa?= =?us-ascii?Q?Cns21VRDQDfqUD67UhTOqHEfhbWxLROTtssTgdu5OCgQ+1fNMsO9syJTSzeV?= =?us-ascii?Q?p6+TjvJfXTRbcJANgt1vZp1V9ciFaKazgc51PZoWxIpKtLG0IOw9+Xnpb0iO?= =?us-ascii?Q?BSkQATRdkceHyd/O3cmv4z3OTI2vSyOGMq08SwqNvVOTvy8PPWYG7z5HqWfr?= =?us-ascii?Q?LjfoUdNENjyTyKX+2WMEDWdeSfrl4YAdzYoKgaJSaGd+L2MD7cjstwCLSHzf?= =?us-ascii?Q?vzxOyc5mBXIbDlYVb4865UkTQexHfwtKYpVz4QcLQb3MBPRHYVHAzLzu86AX?= =?us-ascii?Q?ssJp9ink19tQDxSoFkTfqzQ2FME0audxSILMMvXBlLs+uZIg+04etWOOJVu7?= =?us-ascii?Q?DsZ0Whomjsif5+pMama7xy6DQUrVO6KDsMRiIz9cn+9c6KvGC0t70icQnl3b?= =?us-ascii?Q?lF7pKk4lG1P1nVa11XkFI84n8pW92o1VaVn8ipzm/tm3hpyK443Vq/7ZaPcx?= =?us-ascii?Q?9wnkogTvmLvfehtuStaROzjVy85HzhQwbS7xlEJb/BPgWAKb9hmCZjMI4Bm+?= =?us-ascii?Q?0xjYOtBtHonfkIow7oL7hIw56+68PTr5i9F5UfGSHjXXqnurPwS7ozG45AyK?= =?us-ascii?Q?OirWehcDguBjQMotSmNt9ZS7OdLM7VEGd9seRBx7Az4hT7DbSvYr+YnA7kCm?= =?us-ascii?Q?jf/TX/zxy90K6PhlSUokrg/z+NO4hmQV5/+DZfkvJ908CXuiqq4h?= X-OriginatorOrg: valinux.co.jp X-MS-Exchange-CrossTenant-Network-Message-Id: 0798e43d-b68f-4ae5-f3be-08def88a7207 X-MS-Exchange-CrossTenant-AuthSource: TY7P286MB7722.JPNP286.PROD.OUTLOOK.COM X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 12 Aug 2026 15:57:43.6098 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 7a57bee8-f73d-4c5f-a4f7-d72c91c8c111 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: MtffEXH8+RXZKsxo/QGxyNrXrbweCV+I7ZWcSfWQWPbSv2NCqcKfmpIYY9QRQyLLJdrK2uDkWqgJTwIRQT/J7g== X-MS-Exchange-Transport-CrossTenantHeadersStamped: TY4P286MB7069 Content-Type: text/plain; charset="utf-8" For eDMA, resetting one direction affects every channel in it. Add a worker to coordinate that recovery. Gate publication and doorbells, and prevent channel setup from re-enabling the direction during recovery. Wait for active LL channels to stop. If a transfer is still active, compare two transfer-size samples. Retry when it is moving; treat an unchanged value as frozen. For a frozen direction, drain IRQ handling so pending ABORT cleanup can finish. If recovery is still needed, reset the direction, rebuild every exposed LL context while it remains disabled, publish remote LL writes, and enable it again. Honor EDMA_REQ_STOP and EDMA_REQ_PAUSE during the rebuild; keep channels gated and retry if reset keeps failing. Providers opt in with engine_reset(), engine_enable(), and ch_transfer_size(). No caller requests recovery until the next patch. Signed-off-by: Koichiro Den --- Changes in v5: - Keep recovery active after engine_reset() times out because the directi= on may still become disabled, even if pending work completes. (Sashiko) drivers/dma/dw-edma/dw-edma-core.c | 325 ++++++++++++++++++++++++++++- drivers/dma/dw-edma/dw-edma-core.h | 22 +- 2 files changed, 342 insertions(+), 5 deletions(-) diff --git a/drivers/dma/dw-edma/dw-edma-core.c b/drivers/dma/dw-edma/dw-ed= ma-core.c index ed5654c63933..daf1fa89ef94 100644 --- a/drivers/dma/dw-edma/dw-edma-core.c +++ b/drivers/dma/dw-edma/dw-edma-core.c @@ -28,6 +28,10 @@ /* Empirically chosen progress interval. */ #define DW_EDMA_LL_PROGRESS_INTERVAL 4 #define DW_EDMA_LL_RECHECK_DELAY_MS 30 +#define DW_EDMA_ENGINE_QUIESCE_TIMEOUT_MS 250 +#define DW_EDMA_ENGINE_RESET_FAST_ATTEMPTS 5 +#define DW_EDMA_ENGINE_RESET_RETRY_DELAY_MS 5000 +#define DW_EDMA_MAX_DIR_CH MAX(HDMA_MAX_WR_CH, HDMA_MAX_RD_CH) =20 static inline struct dw_edma_desc *vd2dw_edma_desc(struct virt_dma_desc *vd) @@ -185,6 +189,22 @@ dw_edma_set_request(struct dw_edma_chan *chan, enum dw= _edma_request request) chan->request =3D request; } =20 +static void +dw_edma_engine_recovery_schedule(struct dw_edma_engine_recovery *rec, + unsigned long delay) +{ + if (unlikely(READ_ONCE(rec->dw->teardown))) + return; + + queue_delayed_work(rec->dw->wq, &rec->work, delay); +} + +static void +dw_edma_engine_recovery_queue(struct dw_edma_engine_recovery *rec) +{ + dw_edma_engine_recovery_schedule(rec, 0); +} + static void dw_hdma_set_callback_result(struct virt_dma_desc *vd, enum dmaengine_tx_result result) { @@ -237,6 +257,7 @@ static void dw_edma_core_reset_ll(struct dw_edma_chan *= chan) =20 chan->ll_head =3D 0; chan->ll_done =3D 0; + chan->ll_recovery_pending =3D false; /* Drop stale CB bits before reusing the circular LL ring. */ for (i =3D 0; i < chan->ll_max; i++) dw_edma_core_ll_clear(chan, i); @@ -401,7 +422,7 @@ static int dw_edma_start_transfer(struct dw_edma_chan *= chan) return 1; } =20 - if (dw_edma_abort_is_pending(chan)) + if (chan->ll_recovering || dw_edma_abort_is_pending(chan)) return 0; =20 if (!chan->ll_valid) @@ -556,8 +577,10 @@ static bool dw_edma_ll_consume_progress(struct dw_edma= _chan *chan, int idx) WARN_ON_ONCE(done); =20 out: - if (advanced) + if (advanced) { dw_edma_ll_recheck_cancel(chan); + chan->ll_recovery_pending =3D false; + } =20 return advanced; } @@ -711,6 +734,7 @@ static void dw_edma_core_ch_doorbell(struct dw_edma_cha= n *chan) static bool dw_edma_core_ch_maybe_doorbell(struct dw_edma_chan *chan) { if (chan->non_ll || + chan->ll_recovering || chan->status !=3D EDMA_ST_BUSY || !dw_edma_ll_pending(chan) || dw_edma_abort_is_pending(chan)) @@ -790,6 +814,292 @@ static void dw_edma_ll_recheck_work(struct work_struc= t *work) dw_edma_core_ch_maybe_doorbell(chan); } =20 +static bool +dw_edma_engine_recovery_needed(struct dw_edma_engine_recovery *rec) +{ + struct dw_edma *dw =3D rec->dw; + u16 off =3D rec->dir =3D=3D EDMA_DIR_WRITE ? 0 : dw->wr_ch_cnt; + u16 cnt =3D rec->dir =3D=3D EDMA_DIR_WRITE ? dw->wr_ch_cnt : dw->rd_ch_cn= t; + struct dw_edma_chan *chan; + bool found =3D false; + bool needed; + u16 i; + + for (i =3D 0; i < cnt; i++) { + chan =3D &dw->chan[off + i]; + scoped_guard(spinlock_irqsave, &chan->vc.lock) { + needed =3D chan->ll_recovery_pending && chan->configured && + !chan->non_ll && + chan->status !=3D EDMA_ST_PAUSE; + if (needed && chan->request =3D=3D EDMA_REQ_NONE && + dw_edma_ll_reconcile_and_refill(chan)) + dw_edma_core_ch_maybe_doorbell(chan); + needed =3D needed && chan->ll_recovery_pending && + dw_edma_ll_pending(chan); + if (!needed) + chan->ll_recovery_pending =3D false; + } + found |=3D needed; + } + + return found; +} + +static bool +dw_edma_engine_recovery_incomplete(struct dw_edma_engine_recovery *rec) +{ + bool needed =3D dw_edma_engine_recovery_needed(rec); + + /* A timed-out reset may still disable the engine before the retry. */ + return needed || rec->fails; +} + +static void +dw_edma_engine_recovery_release(struct dw_edma_engine_recovery *rec) +{ + struct dw_edma *dw =3D rec->dw; + u16 off =3D rec->dir =3D=3D EDMA_DIR_WRITE ? 0 : dw->wr_ch_cnt; + u16 cnt =3D rec->dir =3D=3D EDMA_DIR_WRITE ? dw->wr_ch_cnt : dw->rd_ch_cn= t; + struct dw_edma_chan *chan; + u16 i; + + rec->fails =3D 0; + + for (i =3D 0; i < cnt; i++) { + chan =3D &dw->chan[off + i]; + guard(spinlock_irqsave)(&chan->vc.lock); + chan->ll_recovering =3D false; + + if (!READ_ONCE(dw->teardown) && + chan->configured && !chan->non_ll && + chan->request =3D=3D EDMA_REQ_NONE && + chan->status !=3D EDMA_ST_PAUSE) { + dw_edma_start_transfer(chan); + chan->status =3D dw_edma_ll_pending(chan) ? + EDMA_ST_BUSY : EDMA_ST_IDLE; + dw_edma_core_ch_maybe_doorbell(chan); + } + } + + WRITE_ONCE(rec->active, false); +} + +static void +dw_edma_engine_recovery_drain_irqs(struct dw_edma_engine_recovery *rec) +{ + struct dw_edma *dw =3D rec->dw; + struct device *dev =3D dw->chip->dev; + u16 off =3D rec->dir =3D=3D EDMA_DIR_WRITE ? 0 : dw->wr_ch_cnt; + u16 cnt =3D rec->dir =3D=3D EDMA_DIR_WRITE ? dw->wr_ch_cnt : dw->rd_ch_cn= t; + struct dw_edma_chan *chan; + u16 i; + + /* Drain events observed before the engine context changes. */ + for (i =3D 0; i < dw->nr_irqs; i++) { + if (bitmap_empty(rec->dir =3D=3D EDMA_DIR_WRITE ? + dw->irq[i].wr_mask : dw->irq[i].rd_mask, cnt)) + continue; + + synchronize_irq(dw->chip->ops->irq_vector(dev, i)); + } + + for (i =3D 0; i < cnt; i++) { + chan =3D &dw->chan[off + i]; + flush_work(&chan->irq_work); + } +} + +static void dw_edma_engine_recovery_work(struct work_struct *work) +{ + struct dw_edma_engine_recovery *rec =3D + container_of(to_delayed_work(work), + struct dw_edma_engine_recovery, work); + struct dw_edma *dw =3D rec->dw; + const char *dir_name =3D str_write_read(rec->dir =3D=3D EDMA_DIR_WRITE); + u16 off =3D rec->dir =3D=3D EDMA_DIR_WRITE ? 0 : dw->wr_ch_cnt; + u16 cnt =3D rec->dir =3D=3D EDMA_DIR_WRITE ? dw->wr_ch_cnt : dw->rd_ch_cn= t; + unsigned long timeout =3D + jiffies + msecs_to_jiffies(DW_EDMA_ENGINE_QUIESCE_TIMEOUT_MS); + struct dw_edma_chan *chan; + bool configured_ll; + bool busy; + u16 i; + + /* + * Recovery resets a whole direction. The legacy eDMA register map does + * not support partial ownership, while unrolled eDMA partial ownership + * and delegation are direction-granular. Native HDMA permits per-channel + * ownership, but does not provide direction recovery operations. + */ + if (unlikely(READ_ONCE(dw->teardown))) + return; + + if (!dw_edma_engine_recovery_incomplete(rec)) { + if (READ_ONCE(rec->active)) + dw_edma_engine_recovery_release(rec); + return; + } + + WRITE_ONCE(rec->active, true); + + /* Gate each channel before inspecting or resetting the direction. */ + for (i =3D 0; i < cnt; i++) { + chan =3D &dw->chan[off + i]; + guard(spinlock_irqsave)(&chan->vc.lock); + chan->ll_recovering =3D true; + } + + /* + * Gated channels stop at their first unpublished element. Wait for that + * point before resetting; otherwise a restart could skip the remainder + * of an in-flight element. + */ + do { + busy =3D false; + for (i =3D 0; i < cnt; i++) { + chan =3D &dw->chan[off + i]; + scoped_guard(spinlock_irqsave, &chan->vc.lock) + configured_ll =3D chan->configured && !chan->non_ll; + if (configured_ll && + dw_edma_core_ch_status(chan) =3D=3D DMA_IN_PROGRESS) + busy =3D true; + } + if (!busy) + break; + fsleep(1000); + } while (time_before(jiffies, timeout)); + + /* Progress after queueing makes this recovery request obsolete. */ + if (!dw_edma_engine_recovery_incomplete(rec)) { + dw_edma_engine_recovery_release(rec); + return; + } + + if (busy) { + DECLARE_BITMAP(busy_mask, DW_EDMA_MAX_DIR_CH) =3D { 0 }; + u32 tsz[DW_EDMA_MAX_DIR_CH]; + bool moving =3D false; + + /* + * A transfer_size change or a transition out of DMA_IN_PROGRESS + * shows movement after the first sample. Retry while producers + * remain gated. If neither occurs, treat the channel as frozen + * within one element. + */ + for (i =3D 0; i < cnt; i++) { + chan =3D &dw->chan[off + i]; + scoped_guard(spinlock_irqsave, &chan->vc.lock) + configured_ll =3D chan->configured && !chan->non_ll; + if (configured_ll && + dw_edma_core_ch_status(chan) =3D=3D DMA_IN_PROGRESS) { + __set_bit(i, busy_mask); + tsz[i] =3D dw_edma_core_ch_transfer_size(chan); + } + } + fsleep(1000); + for_each_set_bit(i, busy_mask, cnt) { + chan =3D &dw->chan[off + i]; + if (dw_edma_core_ch_status(chan) !=3D DMA_IN_PROGRESS || + tsz[i] !=3D dw_edma_core_ch_transfer_size(chan)) + moving =3D true; + } + if (!dw_edma_engine_recovery_incomplete(rec)) { + dw_edma_engine_recovery_release(rec); + return; + } + if (moving) { + dev_warn_ratelimited(dw->chip->dev, + "%s engine quiesce timed out with transfers still progressing, r= etrying\n", + dir_name); + dw_edma_engine_recovery_queue(rec); + return; + } + + dev_warn(dw->chip->dev, + "%s engine reset with a channel frozen mid-element\n", + dir_name); + } + + if (!dw_edma_engine_recovery_incomplete(rec)) { + dw_edma_engine_recovery_release(rec); + return; + } + + /* + * Let a terminal ABORT finish its normal channel cleanup before a + * direction-wide reset. It may also make recovery unnecessary. + */ + dw_edma_engine_recovery_drain_irqs(rec); + if (!dw_edma_engine_recovery_incomplete(rec)) { + dw_edma_engine_recovery_release(rec); + return; + } + + if (!dw->core->engine_reset(dw, rec->dir)) { + unsigned long delay =3D 0; + + /* + * Keep channels gated and do not re-enable the engine unless + * ENGINE_EN clears. + */ + if (++rec->fails >=3D DW_EDMA_ENGINE_RESET_FAST_ATTEMPTS) + delay =3D msecs_to_jiffies(DW_EDMA_ENGINE_RESET_RETRY_DELAY_MS); + if (rec->fails =3D=3D DW_EDMA_ENGINE_RESET_FAST_ATTEMPTS) + dev_err(dw->chip->dev, + "%s engine did not drain after %u attempts; retrying every %u ms\n", + dir_name, rec->fails, + DW_EDMA_ENGINE_RESET_RETRY_DELAY_MS); + dw_edma_engine_recovery_schedule(rec, delay); + return; + } + + if (rec->fails) + dev_warn(dw->chip->dev, "%s engine drained after %u retries\n", + dir_name, rec->fails); + + /* Drain an event that raced with the engine reset. */ + dw_edma_engine_recovery_drain_irqs(rec); + + /* + * Keep the engine disabled until every LL channel context is rebuilt. + * On the tested integration, re-enabling a preserved context resumed an + * old element without a doorbell, causing an IOMMU fault and stalling the + * engine again. Reset each ring so it starts from a fresh context, and + * complete remote LL publication before re-enabling the direction. + */ + for (i =3D 0; i < cnt; i++) { + bool stop; + + chan =3D &dw->chan[off + i]; + guard(spinlock_irqsave)(&chan->vc.lock); + + if (chan->non_ll) + continue; + + /* Honor EDMA_REQ_STOP and EDMA_REQ_PAUSE instead of republishing. */ + stop =3D chan->request =3D=3D EDMA_REQ_STOP; + if (stop) + dw_edma_terminate_all_descs(chan); + + dw_edma_core_reset_ll(chan); + dw_edma_core_ll_sync(chan); + + if (stop) { + dw_edma_set_request(chan, EDMA_REQ_NONE); + chan->status =3D EDMA_ST_IDLE; + } else if (chan->request =3D=3D EDMA_REQ_PAUSE) { + dw_edma_set_request(chan, EDMA_REQ_NONE); + chan->status =3D EDMA_ST_PAUSE; + } + } + + dw->core->engine_enable(dw, rec->dir); + dw_edma_engine_recovery_release(rec); + + dev_warn(dw->chip->dev, "%s engine was reset to recover a stalled channel= \n", + dir_name); +} + static void dw_edma_device_caps(struct dma_chan *dchan, struct dma_slave_caps *caps) { @@ -1991,6 +2301,13 @@ int dw_edma_probe(struct dw_edma_chip *chip) for (i =3D 0; i < ARRAY_SIZE(dw->event_lock_per_dir); i++) spin_lock_init(&dw->event_lock_per_dir[i]); =20 + for (i =3D 0; i < ARRAY_SIZE(dw->eng_recovery); i++) { + dw->eng_recovery[i].dw =3D dw; + dw->eng_recovery[i].dir =3D i; + INIT_DELAYED_WORK(&dw->eng_recovery[i].work, + dw_edma_engine_recovery_work); + } + /* * chip->ll_*_cnt describes the channels exposed by this instance. Keep * the usable hardware counts separate for partial ownership checks. @@ -2109,6 +2426,10 @@ int dw_edma_remove(struct dw_edma_chip *chip) WRITE_ONCE(dw->teardown, true); dma_async_device_unregister(&dw->dma); =20 + /* A recovery worker can re-enable the engine. */ + disable_delayed_work_sync(&dw->eng_recovery[0].work); + disable_delayed_work_sync(&dw->eng_recovery[1].work); + /* * Drain channel work that may have passed the teardown gate before * stopping the hardware. IRQ handlers remain installed while it is diff --git a/drivers/dma/dw-edma/dw-edma-core.h b/drivers/dma/dw-edma/dw-ed= ma-core.h index 329f02c58995..855ffbd2c06e 100644 --- a/drivers/dma/dw-edma/dw-edma-core.h +++ b/drivers/dma/dw-edma/dw-edma-core.h @@ -128,6 +128,9 @@ struct dw_edma_chan { =20 struct delayed_work ll_recheck_work; unsigned long ll_recheck_at; + /* Per-channel recovery state. */ + bool ll_recovery_pending; + bool ll_recovering; =20 u32 ll_max; /* Data entries */ struct dw_edma_region ll_region; /* Linked list */ @@ -158,6 +161,18 @@ struct dw_edma_irq { DECLARE_BITMAP(rd_mask, HDMA_MAX_RD_CH); }; =20 +/* + * Direction-wide recovery state. active records that recovery has gated t= he + * channels; workqueue state coalesces duplicate requests. + */ +struct dw_edma_engine_recovery { + bool active; + unsigned int fails; + struct delayed_work work; + struct dw_edma *dw; + enum dw_edma_dir dir; +}; + struct dw_edma { char name[32]; =20 @@ -172,12 +187,13 @@ struct dw_edma { struct dw_edma_chan *chan; =20 /* - * WQ_HIGHPRI keeps completion processing responsive under heavy load; - * WQ_UNBOUND lets different channels run on different CPUs. + * WQ_HIGHPRI keeps completion and recovery work responsive under heavy + * load; WQ_UNBOUND lets independent work run on different CPUs. */ struct workqueue_struct *wq; =20 bool teardown; /* Gate asynchronous hardware access */ + struct dw_edma_engine_recovery eng_recovery[2]; =20 raw_spinlock_t lock; /* Protect v0 shared registers */ /* Per-direction lock storage for the eDMA interrupt registers. */ @@ -374,7 +390,7 @@ static inline void dw_edma_core_do_ch_doorbell(struct d= w_edma_chan *chan) static inline void dw_edma_core_ch_enable(struct dw_edma_chan *chan) { chan->dw->core->ch_enable(chan); - if (chan->dw->core->engine_enable) + if (!chan->ll_recovering && chan->dw->core->engine_enable) chan->dw->core->engine_enable(chan->dw, chan->dir); } =20 --=20 2.51.0 From nobody Tue Sep 29 04:14:12 2026 Received: from TY3P286CU002.outbound.protection.outlook.com (mail-japaneastazon11020143.outbound.protection.outlook.com [52.101.229.143]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 353EC463B60; Wed, 12 Aug 2026 15:57:57 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=52.101.229.143 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1786550280; cv=fail; b=M4Lo5ArNu5nlQuRJ9mLotoMsMtWMTMOJN5grLFQt2a5YgQ8Qne//k13jk/dXIyJOuT4zTn7/hg62lmXk2YdP2jyb5cdjb4BZQfB5dI5E9QlrUIVahDhix5dTL95nBf61qDcCyjCwHXwSR9vW8YlqaB4UgN4tfK38CzIAqlDGz2o= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1786550280; c=relaxed/simple; bh=2bt+YgtW9DhKrVsuqfUi5zHXTN/ZKpDDXbUIHbX9Gws=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: Content-Type:MIME-Version; b=F7MhEBzSMC5+u9p0UnY979HXrzymTJfiqgNBtLC14BruQmQWdB8Bzh5ORndq3eKHP2dwOR+RC707UPTtsAQ5n+Sj8kAnMF44civNeMM4SQzhttIenJEt2fqyALS0yO751UwnlMjze01F9UIg7YZhqEHFje6yh5NGk27AbDsvAfA= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=valinux.co.jp; spf=pass smtp.mailfrom=valinux.co.jp; dkim=pass (1024-bit key) header.d=valinux.co.jp header.i=@valinux.co.jp header.b=TafRkkTq; arc=fail smtp.client-ip=52.101.229.143 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=valinux.co.jp Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=valinux.co.jp Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=valinux.co.jp header.i=@valinux.co.jp header.b="TafRkkTq" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=pvq5RaUOcCY10yBrkImqdgKPt+e/SykNj9Z3oZ/LJqM+AWV60jFGiIBWaut/eld8Oftha858IFCds5B28H8CjJ/Y8Nu3QydOlnVcEJDNtTtQI2sp3g7AL/qSV89DsbEZsbEpuZWu0SDtgVGFHoMTZLj9Y2KuUZC34UQ8SqH8Gf3XOYttGEUnhIYWUfmHdyY0PLgJjOSuqArafPJdVkrdYyc3V8XidpgBzOR9SqKCJFh6ls8LQFpMocD4Obq25QH39v9F923TT12kmjDbg3wVuq3gNhZoil8ujaZHZy+sRqRJkMA5kjlGzVt/Vexf4as0kfiNVb8VbEO5ZZb25rHGuA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=iFt/b6EzekDQbSyojiDQMsWT9O6DGa1v3Z+DrilfavA=; b=KItIvUPeHHi1Bl18SxLcFqWcRMmHDlEnmWECx+04cJVq5YVQdTOZAqNb7N0stY4beee/FwGYbFtV+SnXUOcy8lgJw6uXdhr5go8m1NoBsOd/4JbgqXxcS8yuT6BVnKkbdOdJxi2/W3AJ+nM1jAE0dVVqKVK6wznHpxAVIfQMWukpXSVLVd2tsQeTkDqvQYhOOcTbd6rrZeuo/vMY6+IVQNlKxpG+nSY6SzCxL87KUZTJTSY5Wfn2pPQXQkON+tvOyTFqoeb5lhWdWN+qucwoNGTjWzGyCmilpizNRXAxCJPAk7jzN4PhZqoFe8w5ADGS37M/5C6dJfxDMbAOZuaBZg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=valinux.co.jp; dmarc=pass action=none header.from=valinux.co.jp; dkim=pass header.d=valinux.co.jp; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=valinux.co.jp; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=iFt/b6EzekDQbSyojiDQMsWT9O6DGa1v3Z+DrilfavA=; b=TafRkkTqIgB5+XT1hzW1zhDOgwvLcI/WjxYXjFu2am78UNZYZbSJ6ZM3IAaTqXV9Pp2Uy7au/WE0roTcf/vc3XDt5nN4/K6P9NI6DTFtHi9Sewj680heZDHAf6TY+i4dekvMI5g3W+SDa+IusTLeeFY0HKs9UALiORdyb8NyqeE= Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=valinux.co.jp; Received: from TY7P286MB7722.JPNP286.PROD.OUTLOOK.COM (2603:1096:405:38f::10) by TY4P286MB7069.JPNP286.PROD.OUTLOOK.COM (2603:1096:405:343::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.315.13; Wed, 12 Aug 2026 15:57:44 +0000 Received: from TY7P286MB7722.JPNP286.PROD.OUTLOOK.COM ([fe80::2305:327c:28ec:9b32]) by TY7P286MB7722.JPNP286.PROD.OUTLOOK.COM ([fe80::2305:327c:28ec:9b32%3]) with mapi id 15.21.0292.024; Wed, 12 Aug 2026 15:57:44 +0000 From: Koichiro Den To: Vinod Koul , Frank Li , Manivannan Sadhasivam , Gustavo Pimentel , Kees Cook , =?UTF-8?q?Krzysztof=20Wilczy=C5=84ski?= , Kishon Vijay Abraham I , Bjorn Helgaas , Christoph Hellwig , Serge Semin , Cai Huoqing , Niklas Cassel Cc: Devendra K Verma , dmaengine@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH v5 23/24] dmaengine: dw-edma: Detect and recover a stalled eDMA engine Date: Thu, 13 Aug 2026 00:57:20 +0900 Message-ID: <20260812155721.2807506-24-den@valinux.co.jp> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20260812155721.2807506-1-den@valinux.co.jp> References: <20260812155721.2807506-1-den@valinux.co.jp> Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: TY4P286CA0105.JPNP286.PROD.OUTLOOK.COM (2603:1096:405:380::9) To TY7P286MB7722.JPNP286.PROD.OUTLOOK.COM (2603:1096:405:38f::10) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: TY7P286MB7722:EE_|TY4P286MB7069:EE_ X-MS-Office365-Filtering-Correlation-Id: e713c59b-6082-463d-28e7-08def88a7298 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|1800799024|10070799003|366016|376014|23010399003|7416014|18002099003|22082099003|56012099006|10067099003|921020; X-Microsoft-Antispam-Message-Info: BdVKw24kQ7GXBExAFJ+Cwi5gWXp3Ei4XMbU6CHWVMLt/MXcfkcie+ho8ZqwsB6baNvX0hK0VAJs+b8mtInrEDYQxi+ze019LtKWKx+xLJPwBXSnFWt8qfFuLYrJ8I6+LeRqHqDOhscFAjrkpqQweaQA/kYOObcAsYq56OTJjRKRuGR4tEw0IzqoIWqHkJJUOFHT9N4vEdRu5wkkSl3aq9QHRWKVzMe9xIArAuupKCKMo+ggGzbeE2wDdK6W7pdi5pk9b45t9GB6qMTDYOQd2jGIgTyY8AJaw7s92yn49H7luZLTVlMGn8SlZrGU/s3lrsBOachqsr1m9OOvHFKhPkdwAulssLFJctnARyoehKb9IIuLe5PhHCCrEqTInyg7Z9fySqozhwGR1Iemk5McrUOr+TF6CRtyWOHBzcF7gH/wnudhHbMoy2k+1FGxgbZetMx04wH6phl302c5bNjE8XCOd65+FJ/FGm4r5oTiIPWuBBzoNIP4bwX48X3D4W+eingbFMwcV4dEGTZyh78PAe+EceIq1gIW8knAbOrVYF9MWAKm5X1EEZ+bJt20AScvR2XvoWKgR+V47PtLnXzQAYdBNInuxX6gUATH7cbRI8ObeIn9Md/bwnib3tYDDOz99mpo5lfRP+Fe86W8Et94XzwpQcIxdePI+u7lgtXdrBTl6sbwkOtFu/7UpBewmJKqpmNnzXp+0ZTlxNNpx2o17QA== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:TY7P286MB7722.JPNP286.PROD.OUTLOOK.COM;PTR:;CAT:NONE;SFS:(13230040)(1800799024)(10070799003)(366016)(376014)(23010399003)(7416014)(18002099003)(22082099003)(56012099006)(10067099003)(921020);DIR:OUT;SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?JGORlM/uFsgQVEuUlF4Lv5PAN7stmkA1WT41dnvum2jsuELbdvFiLOgX40Y+?= =?us-ascii?Q?2Q3DozeEdbX1Jf73lgp7Ra2VHm0Co3SyeJ9p+0LFaAl9aQl8+Ax68Z5KEwPM?= =?us-ascii?Q?fgkry+y7/2rKLYHFXj/OBatmPTOCGrS3XxhAkIAMXXlrc8HIkus4o0+5sUDp?= =?us-ascii?Q?2PTyx9h3Vi5TM2PbAd0szgKbCRD6m5us/rYzc/NQpuPY7qg03Yfc1e9M4A7A?= =?us-ascii?Q?9Ul9cs+RGDMSNclwlR0c0F3kS+1hI37GiWqPG26a7YpLExWV4644aCALiHPL?= =?us-ascii?Q?1DdGeWPSBA1PVHvkrjeON9jp208RQgjFVF+FknrFdTyHaQr2IK87lDMy+XCm?= =?us-ascii?Q?f+U7NX1Og0WPE6I0jY+3MLstTE3DV+fybeGJ9g9R3uf8eVpLf7yHt1jVWeL9?= =?us-ascii?Q?iQD2WMAdUVXiqIZTUyobujmekHId9WbHkcgMxLZ1g5PxfxQjbajHJ73Ny6Hj?= =?us-ascii?Q?XZlCU3bKokNszvD4/D+ArMuJjbcDuJTzxXaDRifCJRR/Wq6I1nMIjF9VNg1f?= =?us-ascii?Q?usIP4p2/B7DcD+Sh1Q2ogsEJs8sAbx8JX/v03EpDOmi/gIuzp0HmlcWvCUEg?= =?us-ascii?Q?1GZ71wnYlCSUhLJdBGKPB9+a44a8r1aS1KBre3gFjCGg+HZn2cvtLy42pcLi?= =?us-ascii?Q?q0peipHaQdjHwkY8VV6MRXzwZ5PTn33FfSLe9B4zaINf0eoDmORgzHOlRsWg?= =?us-ascii?Q?kOTFMRqxCvxRY3pJDJ81j91YAIA7fbezCP9YVPYwIHHw9l/oKGLIaasorBEl?= =?us-ascii?Q?YkaGu5coOSrTDKKUMylBomiix73h99/jca6T+drMRhKwBbImmh/agGHIsgO7?= =?us-ascii?Q?2FIe0s0M/WGBGzu7W9wm2wA0gz8fLMOaGsfmKfLYR8qQ3Qqpzabiys6Xuvcm?= =?us-ascii?Q?5Z2+7NJinM+Eai07AHuz+BuTkLPLR4IPuvWr/wQiJDV479PQRG8ImtVm32fK?= =?us-ascii?Q?wPpJPdeBQIOu2ENIuLan96kxLBj6dnVZC9+1WHn7yHVb53UYSxC2ukkGsvj5?= =?us-ascii?Q?F+mQjhrddlcXtSSI25gxG7GiTf0a/PFfECcZQ7gxkXJiffBPOQPxGjpfrnbk?= =?us-ascii?Q?3BHFJgSrXh5bEhvwrU4GjxGI0LABP4eJXy4ZcDQ6scPfKY4PcutRd65gPqm0?= =?us-ascii?Q?DexyLZDZFk6gWNS/Gldlz4USnzqh8gQ9r3kY2oIv3vNxy6dLK5cub+JfydvO?= =?us-ascii?Q?njhw6QHlZK9MSNMTbTC/MUR/T0E/iA1a5QyiGvNBsK4tUF/Ok5CETZ2Ke3Rl?= =?us-ascii?Q?eQzSizwAaPf+S5UrklE4TW8XffgxVqjTSivrRSRkIIm/aXrmie2+LBXfXHK1?= =?us-ascii?Q?URGQUQm2g8pzbnHx/Uw5qN1dkLBrcURVpIINrFUmTD0JVEC4m8/tDEess7sJ?= =?us-ascii?Q?GabqKJPgCRRJWzJMk6W2Arx6Ba99waJKvHOMRVXb8S20je8DvKj5GwfDn20C?= =?us-ascii?Q?le3ibrLzJs+JLNTNHYgPu8ttIbsV0CyQsajRqpM+l+w9HW+4FNUJLEzO9jnO?= =?us-ascii?Q?eema0LpptdgBaFG8qvfI7XyelU52L0o5cONySl3GgTq5fcw8QLviCw/q+Z0o?= =?us-ascii?Q?UgD68Bsk1EV7JoTLTQa5wpMDddtGfc7AMnJFjQpqP1wUqE4kdOExOoVqNlEE?= =?us-ascii?Q?OuRm9dwFjLko9Ux1be5gTWyKGYqZE2oRr56L6n7rVzIUBAkZ0CArjtM11Shl?= =?us-ascii?Q?zOZ3PdRcc9GyIPQ0lvy+xqRbayZ7XSh6/kMD4Dl4dy8u10xYt84z1afBaNQR?= =?us-ascii?Q?xT4qkN8RQQxVjIFRvZKvnUgwtyMX/JfvPdo6IlRFJU+1QcCaOKMc?= X-OriginatorOrg: valinux.co.jp X-MS-Exchange-CrossTenant-Network-Message-Id: e713c59b-6082-463d-28e7-08def88a7298 X-MS-Exchange-CrossTenant-AuthSource: TY7P286MB7722.JPNP286.PROD.OUTLOOK.COM X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 12 Aug 2026 15:57:44.5512 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 7a57bee8-f73d-4c5f-a4f7-d72c91c8c111 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 9Y/LJ7JZQhgX641gkOpkcefK9UbtAyYfYEHQMw6OkqGIBNwKwXUG5mAuth/TS3D8aE56Ns175Di7qjO4SAHeDw== X-MS-Exchange-Transport-CrossTenantHeadersStamped: TY4P286MB7069 Content-Type: text/plain; charset="utf-8" Under sustained multi-channel traffic on one eDMA integration, a write channel occasionally stopped with pending work, no progress, and no error. Repeated doorbells did not restart it; clearing ENGINE_EN did. LL progress accounting already schedules a bounded recheck when a DONE or STOP/PAUSE drain leaves published work running. Reuse it for passive stall detection. When a check finds the channel stopped, run the normal restart or drain path, kick it, and arm one more check. If it is still stopped with no progress 30 ms later, queue direction recovery. Queue every recovery request and let the workqueue coalesce duplicates, so a request raised while the worker releases channels is not lost. The condition uses only generic eDMA state: published work, STOPPED status, and lack of progress. Providers opt in by supplying the recovery operations; HDMA compatibility mode is excluded because it has no ENGINE_EN. Normal progress cancels the check before recovery runs. The 30 ms threshold is empirical and should be validated on more hardware. Signed-off-by: Koichiro Den --- Changes in v5: - No changes. drivers/dma/dw-edma/dw-edma-core.c | 52 +++++++++++++++++++++++++++++- drivers/dma/dw-edma/dw-edma-core.h | 4 ++- 2 files changed, 54 insertions(+), 2 deletions(-) diff --git a/drivers/dma/dw-edma/dw-edma-core.c b/drivers/dma/dw-edma/dw-ed= ma-core.c index daf1fa89ef94..8dba7e02c944 100644 --- a/drivers/dma/dw-edma/dw-edma-core.c +++ b/drivers/dma/dw-edma/dw-edma-core.c @@ -257,6 +257,7 @@ static void dw_edma_core_reset_ll(struct dw_edma_chan *= chan) =20 chan->ll_head =3D 0; chan->ll_done =3D 0; + chan->ll_stall_valid =3D false; chan->ll_recovery_pending =3D false; /* Drop stale CB bits before reusing the circular LL ring. */ for (i =3D 0; i < chan->ll_max; i++) @@ -302,6 +303,52 @@ static bool dw_edma_ll_pending(struct dw_edma_chan *ch= an) return chan->ll_head !=3D chan->ll_done; } =20 +static bool dw_edma_engine_recovery_supported(struct dw_edma *dw) +{ + /* + * DWC PCIe Controller Databook 6.10a-lca06, "Legacy DMA and HDMA + * Software Compatibility": + * + * "HDMA does not implement engine enable, that is + * DMA_[WRITE|READ]_ENGINE_EN_OFF.DMA_[WRITE|READ]_ENGINE field." + */ + if (dw->chip->mf =3D=3D EDMA_MF_HDMA_COMPAT) + return false; + + return dw->core->engine_reset && dw->core->engine_enable && + dw->core->ch_transfer_size; +} + +/* + * Called with vc.lock held for a stopped channel with LL work pending. Qu= eue + * direction recovery if repeated doorbells show no progress for one reche= ck + * interval. + */ +static void dw_edma_ll_stall_check(struct dw_edma_chan *chan) +{ + struct dw_edma_engine_recovery *rec; + + if (unlikely(READ_ONCE(chan->dw->teardown))) + return; + + if (!dw_edma_engine_recovery_supported(chan->dw)) + return; + + if (!chan->ll_stall_valid) { + chan->ll_stall_valid =3D true; + chan->ll_stall_since =3D jiffies; + return; + } + + if (time_before(jiffies, chan->ll_stall_since + + msecs_to_jiffies(DW_EDMA_LL_RECHECK_DELAY_MS))) + return; + + rec =3D &chan->dw->eng_recovery[chan->dir]; + chan->ll_recovery_pending =3D true; + dw_edma_engine_recovery_queue(rec); +} + static u32 dw_edma_core_ch_transfer_size(struct dw_edma_chan *chan) { if (!chan->dw->core->ch_transfer_size) @@ -579,6 +626,7 @@ static bool dw_edma_ll_consume_progress(struct dw_edma_= chan *chan, int idx) out: if (advanced) { dw_edma_ll_recheck_cancel(chan); + chan->ll_stall_valid =3D false; chan->ll_recovery_pending =3D false; } =20 @@ -767,8 +815,10 @@ static bool dw_edma_core_ch_maybe_doorbell(struct dw_e= dma_chan *chan) } } =20 + dw_edma_ll_stall_check(chan); dw_edma_core_ch_doorbell(chan); - if (!dw_edma_ll_has_hdma_stop_event(chan)) + if (!dw_edma_ll_has_hdma_stop_event(chan) && + !chan->ll_recovery_pending) dw_edma_ll_recheck_schedule(chan); =20 return false; diff --git a/drivers/dma/dw-edma/dw-edma-core.h b/drivers/dma/dw-edma/dw-ed= ma-core.h index 855ffbd2c06e..b721d0926de5 100644 --- a/drivers/dma/dw-edma/dw-edma-core.h +++ b/drivers/dma/dw-edma/dw-edma-core.h @@ -126,9 +126,11 @@ struct dw_edma_chan { spinlock_t event_lock_per_chan; spinlock_t *event_lock; /* Selected event lock */ =20 + /* Per-channel stall and recovery state. */ struct delayed_work ll_recheck_work; unsigned long ll_recheck_at; - /* Per-channel recovery state. */ + unsigned long ll_stall_since; + bool ll_stall_valid; bool ll_recovery_pending; bool ll_recovering; =20 --=20 2.51.0 From nobody Tue Sep 29 04:14:12 2026 Received: from TY3P286CU002.outbound.protection.outlook.com (mail-japaneastazon11020143.outbound.protection.outlook.com [52.101.229.143]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 122B9480DE0; Wed, 12 Aug 2026 15:58:00 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=52.101.229.143 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1786550286; cv=fail; b=Y/gckTYaBsfqLPLV/l3eC1LaPKr3PkNFEMqCQb45C/ySlLw/DIkq20HCcVOZqpgCFgddnSPYFxioIjmfX2dobdYiLXss/VtjpwicgUDRSrSdk0ETqr8M1i9IHCb4lmqLamVOMhPZgt+BSOj3hEeK33yt1GJcp59eb643WLTMTCk= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1786550286; c=relaxed/simple; bh=VvoVI7qFYJsVqtvVvbV+Eso3U2Otnyv2XQTPDUnNqsQ=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: Content-Type:MIME-Version; b=ktFuUU89nu5l0edl0VuL+sJ+qM93m1xb9MlCBL5YdK0l1o1UacJ/rQ/iR7MIU6nqIQIi41FxxCWU5yejzpWS4n2y8TYkUVHzuPDHmjknnmqQslF4AiuM66c6TIMK/KOiI3pYGqyp6n3saeRb6U6fwxHmp5KUyKDUR4YWsctRJJQ= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=valinux.co.jp; spf=pass smtp.mailfrom=valinux.co.jp; dkim=pass (1024-bit key) header.d=valinux.co.jp header.i=@valinux.co.jp header.b=WZI6H5QP; arc=fail smtp.client-ip=52.101.229.143 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=valinux.co.jp Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=valinux.co.jp Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=valinux.co.jp header.i=@valinux.co.jp header.b="WZI6H5QP" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=OVBlzWAPtWqniOwTBXlc6pkY0qewB1Z9p/BM/hEFNN7mRShOomkacWo7UA3qxREOrHSufl1Osaj7ORyjq2jiwHpFf6CiTHibOknkZV0/Y4VW1uT32XqQ+8Wogb8Ffn/7d3RlsYG8plBDODz7Rgrjie9g3uP3MecbV/lVCGZLMuQ12CFnLjKGHko7Us8ajn3CRZLaak52XR97J6XfwihTHHPSKErdtriDp4LqYZGCcOoX2aYrv5s0hQpqh/duv7KTm0drj083zHzbQjGsoMwLBH1QG69HgmCeWYOGfrwtmIrFJHHvUO63pJ16Zcjw0LditOlp9cK+LhWL9wn8DIyn2w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=hLBgNuhDfQwoRhWRFLlBULNeV5vtaEJOB6EAlWyp2q8=; b=OfkwAR1M0MJ4qWTPWf5YKLVj3dPwa1tXHU6zFZjpiFY0wFyEHDOhypNH4zK8Wcw+05Petm8yG3PJP7UYaRbTr3AOfhCHu5l5KCBAoDQ0CyCC3MaW78wDyvvrkGIfDELfhBYY0Syp9pLV/bpsu6HJgox442udpXp8IL21Lrc4f2vIccEKWWIz1be9vbvu4OaIk0tHhoTYabCXbgiFKRpbzhy+fjBgs69vjaOdBUYz6Wr8yR+Rxh84jUdq2euL+33CU9fiAwbxlYC2qYfIjPhqeHlwrte98aB249tWHA2Z2qbg5SPy9w+Oxykl2ViUPQJ7ehmRM9AZuU2Vq5t42Ch6aA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=valinux.co.jp; dmarc=pass action=none header.from=valinux.co.jp; dkim=pass header.d=valinux.co.jp; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=valinux.co.jp; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=hLBgNuhDfQwoRhWRFLlBULNeV5vtaEJOB6EAlWyp2q8=; b=WZI6H5QPfzHzhBV8kzYodlm3V0XQsoaWBGiXgtlU1wAnBcAGLiW/ILgNTwI+I/bJgRsGvlMm+O4PfgCDGjSdS+CMuONoEHTh8nVlNDNZLm/MxCOsrge61wlpZGL8ucowCDpVzZoYq3egBG0gMOHg/QObb+f+PR3werBpsaWH3P4= Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=valinux.co.jp; Received: from TY7P286MB7722.JPNP286.PROD.OUTLOOK.COM (2603:1096:405:38f::10) by TY4P286MB7069.JPNP286.PROD.OUTLOOK.COM (2603:1096:405:343::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.315.13; Wed, 12 Aug 2026 15:57:45 +0000 Received: from TY7P286MB7722.JPNP286.PROD.OUTLOOK.COM ([fe80::2305:327c:28ec:9b32]) by TY7P286MB7722.JPNP286.PROD.OUTLOOK.COM ([fe80::2305:327c:28ec:9b32%3]) with mapi id 15.21.0292.024; Wed, 12 Aug 2026 15:57:45 +0000 From: Koichiro Den To: Vinod Koul , Frank Li , Manivannan Sadhasivam , Gustavo Pimentel , Kees Cook , =?UTF-8?q?Krzysztof=20Wilczy=C5=84ski?= , Kishon Vijay Abraham I , Bjorn Helgaas , Christoph Hellwig , Serge Semin , Cai Huoqing , Niklas Cassel Cc: Devendra K Verma , dmaengine@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH v5 24/24] dmaengine: dw-edma: Add trace support Date: Thu, 13 Aug 2026 00:57:21 +0900 Message-ID: <20260812155721.2807506-25-den@valinux.co.jp> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20260812155721.2807506-1-den@valinux.co.jp> References: <20260812155721.2807506-1-den@valinux.co.jp> Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: TYCP286CA0263.JPNP286.PROD.OUTLOOK.COM (2603:1096:400:455::11) To TY7P286MB7722.JPNP286.PROD.OUTLOOK.COM (2603:1096:405:38f::10) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: TY7P286MB7722:EE_|TY4P286MB7069:EE_ X-MS-Office365-Filtering-Correlation-Id: 1614eb46-8e3a-410a-fdb4-08def88a7310 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|1800799024|10070799003|366016|376014|23010399003|7416014|18002099003|22082099003|56012099006|10067099003|921020; X-Microsoft-Antispam-Message-Info: b4C0v+Oh4Yxr7y0lbv5bCSPhuajg13YdlBZT2OMXTGVFEYW6vI2DTFB10Qfv3M6rfeJmLogZ1n0f34hIjsxD3OdDtEoX3t4TZLthaDZRPsS+2eYF01m+rhYO6ZT86MtkIncHF7wG2GKf/aJCWC8WfzepwsiIMBL+pE9WoBiIv/q/ksObQV+ogCjASpXB3OROSZ0tnw8ZxFYiuaIiT9TYk02XJVWiYuX6cGf/7to3qda3RF5klFthlWVJQUPPL2tUQbJgWWRhSDzxX+83sE3dgEu7HLyh5hu9wc3sxIrCvRainUcLDf/GoHetMaCugfxoD7UFXL5P0OwG/6nxE3gLrzeBr6h9IJ1bdV+4HPnquNzXiM1W2e9anxHafpH/wealr6FuvZ7dahurKACy4HPPy19mLMmR1ubvQMvKLwPHtVx5YJx8zStqfwWNaJVAfufiQttSzOKGbLk9PpTK+tMD7vzm/PPS3VWCOQF+WGDfs9mYFP4YOYNUVkFqSTSfQXXDjNMOAn21yhmaFwYwSjwm1JyvjhD1+AD6Lzs13J0mtSsz0z0UZPtTiK/3Qd1qHf8bJFw/cYK7EaTYHMUCRfPK6qV85DrPs5BWVZy1uhA2CcdcCHZgA9T7XFwgXdFWbFbqcoltuqtsaxxn8GKUaMgL36Cko2a9ZPwkgi5Or5wy5Ojc/03Vg1MXGIsmVdVA2XkxNzpf4+tDspK1AGZFEuc31A== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:TY7P286MB7722.JPNP286.PROD.OUTLOOK.COM;PTR:;CAT:NONE;SFS:(13230040)(1800799024)(10070799003)(366016)(376014)(23010399003)(7416014)(18002099003)(22082099003)(56012099006)(10067099003)(921020);DIR:OUT;SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?pni720nLNkDn0EBlhieOxQu2QggTbP9FdJTlKIHEigIW7SON1OhPMgHKUqWX?= =?us-ascii?Q?IwpeUqEJSLAGDnhOOplLtFtQmQZ+TPA+BZvrqMt4XezlkGWnTbZDM/w+ku3p?= =?us-ascii?Q?0ZygepTABZrwTnRa0xYdVY8Hw9jrJ+2dEHR2aV1mS7ts1tGpUvwUOJPw0UJv?= =?us-ascii?Q?wXFFL8MNQtBgV2xwR6ZWMohDy/olni5KOZ2A2Du9xTdNdm4TbI/dFUDoXDrI?= =?us-ascii?Q?/Ie4Fp5t0KmzEpNvra3Wx4UGVJIO1aVVDwFXUUmqHHDfCCAg01AAfwGBf623?= =?us-ascii?Q?vdOA7zk9fcE5rVD4lX9n8N/AX+yebD735si3c3IfB+3fmgHgY1uP4uXKuNeP?= =?us-ascii?Q?2JkBNtNSWuCtCEUSPdJCcbz4Hk8jf/GTCxO9RSluavBNp5dJq6mqGiyjZkGv?= =?us-ascii?Q?7vKx/1ptynYcp6cdKcPBd2xmXNeuCR5FF6UQakUiX9n6uV7p5CcglgfsTswG?= =?us-ascii?Q?L+XAr3+PJG9SpTk7Un7GXPTWoq68s4ChtDL4ntR8/mml/HUcmmnmRDw7FuwG?= =?us-ascii?Q?GMlzth1YvRG/FIq3HPxHBL7hF+KCxny2zlcKJz/3eGCOCoE+dSUHjyhR6e7m?= =?us-ascii?Q?iSKheW7Y8mi2GmIIT1JUs4oXEFg7gxBB+b9d6vZA4a4/5jJa/NbSwhGHXdcJ?= =?us-ascii?Q?F4Tr4UFn0p2ZlihMDP75DnT/ma9I8swp6ezyD/jX/uuZdWtNnZEsHjpccUEO?= =?us-ascii?Q?DvaSXcz72INPxU52cugtDdxTa3iKt6MP+rCVC4PmEA8hHcrImLhYhFKxSqPh?= =?us-ascii?Q?xLHEz9EPOXe/TBBv5FFXHgYUGcewW60v8rNebaXhC4KgXiwvnOuUE5Ijmu1Q?= =?us-ascii?Q?nzcV5w/PxFNBxRHj4VgPf9J6cPRdqdw1dl6lR1IMZmOkUAvDzhVO+rVgjfYq?= =?us-ascii?Q?Lw9gt+FFtYYbveeqSEouoSfXUPXgZDgX/PABm3zHDiQaYO8QeEMP2h5Rd9RM?= =?us-ascii?Q?jzZiuo2gjibD4oz8ADLNUJxh52ApaHNA0a3JhVz2A3nuXKNkd+aaBfJ3Hgzs?= =?us-ascii?Q?TMEl9Bo2KZ9ZZtw0QzFTqNeWtg3s5Gf72TLaa98J9i7fhJOhRcPK9/BFCnyT?= =?us-ascii?Q?9rqbjP5G8o7tLbuFXtuONQDSAtWpg6CbJ8GkAQDMUlohLMCI1mJ0Cc9qcIup?= =?us-ascii?Q?pnHlH2cb5UZIFWAdFoWn3gk+xGoniSSaEdU2SX6CE6DIO/gbHLUs/ewaSo39?= =?us-ascii?Q?lx43o+eii1nQqe4uE2GdLzqnpJ1K/ZbJvjiNeLYZ87o5qsBTJJlfyFy8EUcw?= =?us-ascii?Q?UfF41R0TdA2bvNniLBjylgfN2bVuDwxtZsfFK8j9JZkH0MVXOl939D3Rtsah?= =?us-ascii?Q?wSQdWo9GY9gePDIDA7AsLT6et7R8/7A6WvqMQqXZI7hkeKZBEw1FVnfI/g00?= =?us-ascii?Q?T3eXxwPh3TDM6Qd5SKFepjwmftrnCYTkS0nCmniLo9994UbDHVB4A2Mmp46V?= =?us-ascii?Q?0liWdu/Tq51ZNwmLXqY+05B6bxJ4O9P/aEiqVPw0jxpAhzWfXtAPZOXFKsps?= =?us-ascii?Q?vcOd9zqiD75KbvQI7tiRvYIs3IfIGzWfDpuDObSGhBryXl+rJw2/EItf8h4X?= =?us-ascii?Q?eYx6mlDJV9O0zBOiWkcaMKhlgcIRsCE54yjHvbsBSSLwDxAFLMb0krYAUY5b?= =?us-ascii?Q?SYi9FnicHiqhAvbMfT5m8MvAICz+ye0FQDcaJoMD6rURpOVddAWjwj3/kNjs?= =?us-ascii?Q?cu/B7LPZIV4fwKtGaWvDgPBtc+IuI9nJYHCiEj2tvD2K+48zMMIiLjQnTDRP?= =?us-ascii?Q?61BlHmTaXBdh+65zZwQ5dJ4LFEwVEaUrV7kQhGWLQdOHmDxiz+5b?= X-OriginatorOrg: valinux.co.jp X-MS-Exchange-CrossTenant-Network-Message-Id: 1614eb46-8e3a-410a-fdb4-08def88a7310 X-MS-Exchange-CrossTenant-AuthSource: TY7P286MB7722.JPNP286.PROD.OUTLOOK.COM X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 12 Aug 2026 15:57:45.3336 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 7a57bee8-f73d-4c5f-a4f7-d72c91c8c111 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: Y+1IOYPQLfD8Iv7ifpmfZHg45KEAX0xv8goIqbJydMgcVYscIuLUhAgBfdF25E27gX0BXTvwOzvx8dT86cR/nw== X-MS-Exchange-Transport-CrossTenantHeadersStamped: TY4P286MB7069 Content-Type: text/plain; charset="utf-8" From: Frank Li The circular LL ring has separate producer and consumer positions. Add tracepoints for LL publication, descriptor append and completion, IRQ-paired LL events, and engine recovery. The LL events cover both eDMA and HDMA. The recovery event records configured LL channels before a direction reset. When disabled, static keys leave only a branch at each trace site. Signed-off-by: Frank Li Co-developed-by: Koichiro Den Signed-off-by: Koichiro Den --- Changes in v5: - Export LL event enum values for userspace trace format parsers. (Sashiko) drivers/dma/dw-edma/Makefile | 2 + drivers/dma/dw-edma/dw-edma-core.c | 49 ++++++++ drivers/dma/dw-edma/dw-edma-core.h | 2 + drivers/dma/dw-edma/dw-edma-trace.c | 4 + drivers/dma/dw-edma/dw-edma-trace.h | 176 ++++++++++++++++++++++++++++ 5 files changed, 233 insertions(+) create mode 100644 drivers/dma/dw-edma/dw-edma-trace.c create mode 100644 drivers/dma/dw-edma/dw-edma-trace.h diff --git a/drivers/dma/dw-edma/Makefile b/drivers/dma/dw-edma/Makefile index 83ab58f87760..5f395fa1ede4 100644 --- a/drivers/dma/dw-edma/Makefile +++ b/drivers/dma/dw-edma/Makefile @@ -1,8 +1,10 @@ # SPDX-License-Identifier: GPL-2.0 =20 +CFLAGS_dw-edma-trace.o :=3D -I$(src) obj-$(CONFIG_DW_EDMA) +=3D dw-edma.o dw-edma-$(CONFIG_DEBUG_FS) :=3D dw-edma-v0-debugfs.o \ dw-hdma-v0-debugfs.o +dw-edma-$(CONFIG_TRACING) +=3D dw-edma-trace.o dw-edma-objs :=3D dw-edma-core.o \ dw-edma-v0-core.o \ dw-hdma-v0-core.o $(dw-edma-y) diff --git a/drivers/dma/dw-edma/dw-edma-core.c b/drivers/dma/dw-edma/dw-ed= ma-core.c index 8dba7e02c944..261c7e770f63 100644 --- a/drivers/dma/dw-edma/dw-edma-core.c +++ b/drivers/dma/dw-edma/dw-edma-core.c @@ -423,6 +423,12 @@ static void dw_edma_core_ll_start(struct dw_edma_desc = *desc) chan->ll_head, chan->cb, dw_edma_core_enable_ll_irq(desc, i, free)); =20 + trace_edma_fill_ll(chan, chan->ll_head, + desc->vd.tx.cookie, + desc->burst[i].sar, + desc->burst[i].dar, desc->burst[i].sz, + chan->cb); + chan->ll_head++; =20 if (chan->ll_head =3D=3D chan->ll_max) { @@ -484,6 +490,7 @@ static int dw_edma_start_transfer(struct dw_edma_chan *= chan) if (desc->start_burst =3D=3D desc->nburst) continue; =20 + trace_edma_append_desc(desc); dw_edma_core_start(desc); ret =3D 1; } @@ -618,6 +625,7 @@ static bool dw_edma_ll_consume_progress(struct dw_edma_= chan *chan, int idx) /* Hardware has consumed this descriptor's LL entries. */ dw_hdma_set_callback_result(vd, DMA_TRANS_NOERROR); list_del(&vd->node); + trace_edma_complete_desc(desc); vchan_cookie_complete(vd); } =20 @@ -959,6 +967,44 @@ dw_edma_engine_recovery_drain_irqs(struct dw_edma_engi= ne_recovery *rec) } } =20 +static void +dw_edma_trace_engine_recovery(struct dw_edma_engine_recovery *rec) +{ + struct dw_edma *dw =3D rec->dw; + u16 off =3D rec->dir =3D=3D EDMA_DIR_WRITE ? 0 : dw->wr_ch_cnt; + u16 cnt =3D rec->dir =3D=3D EDMA_DIR_WRITE ? dw->wr_ch_cnt : dw->rd_ch_cn= t; + struct dw_edma_chan *chan; + u16 i; + + if (!trace_edma_engine_recovery_enabled()) + return; + + for (i =3D 0; i < cnt; i++) { + enum dw_edma_request request; + enum dw_edma_status status; + bool configured_ll; + u32 ll_head, ll_done, pending; + + chan =3D &dw->chan[off + i]; + scoped_guard(spinlock_irqsave, &chan->vc.lock) { + configured_ll =3D chan->configured && !chan->non_ll; + if (configured_ll) { + request =3D chan->request; + status =3D chan->status; + ll_head =3D chan->ll_head; + ll_done =3D chan->ll_done; + pending =3D dw_edma_core_get_used_num(chan); + } + } + if (!configured_ll) + continue; + + trace_edma_engine_recovery(chan, dw_edma_core_ch_status(chan), + request, status, ll_head, ll_done, + pending); + } +} + static void dw_edma_engine_recovery_work(struct work_struct *work) { struct dw_edma_engine_recovery *rec =3D @@ -1085,6 +1131,8 @@ static void dw_edma_engine_recovery_work(struct work_= struct *work) return; } =20 + dw_edma_trace_engine_recovery(rec); + if (!dw->core->engine_reset(dw, rec->dir)) { unsigned long delay =3D 0; =20 @@ -1707,6 +1755,7 @@ static void dw_edma_ll_interrupt(struct dw_edma_chan = *chan) if (!dw_edma_ll_snapshot_take(chan, &snapshot)) return; =20 + trace_edma_irq(chan, &snapshot); if (chan->status =3D=3D EDMA_ST_PAUSE) return; =20 diff --git a/drivers/dma/dw-edma/dw-edma-core.h b/drivers/dma/dw-edma/dw-ed= ma-core.h index b721d0926de5..bbca2d611c95 100644 --- a/drivers/dma/dw-edma/dw-edma-core.h +++ b/drivers/dma/dw-edma/dw-edma-core.h @@ -206,6 +206,8 @@ struct dw_edma { const struct dw_edma_core_ops *core; }; =20 +#include "dw-edma-trace.h" + typedef void (*dw_edma_handler_t)(struct dw_edma_chan *chan, unsigned int events); =20 diff --git a/drivers/dma/dw-edma/dw-edma-trace.c b/drivers/dma/dw-edma/dw-e= dma-trace.c new file mode 100644 index 000000000000..2620ad61a943 --- /dev/null +++ b/drivers/dma/dw-edma/dw-edma-trace.c @@ -0,0 +1,4 @@ +// SPDX-License-Identifier: GPL-2.0 + +#define CREATE_TRACE_POINTS +#include "dw-edma-core.h" diff --git a/drivers/dma/dw-edma/dw-edma-trace.h b/drivers/dma/dw-edma/dw-e= dma-trace.h new file mode 100644 index 000000000000..cf78e299049f --- /dev/null +++ b/drivers/dma/dw-edma/dw-edma-trace.h @@ -0,0 +1,176 @@ +/* SPDX-License-Identifier: GPL-2.0-or-later */ +/* + * Copyright 2023 NXP. + */ + +#undef TRACE_SYSTEM +#define TRACE_SYSTEM dw_edma + +#if !defined(__LINUX_DW_EDMA_TRACE) || defined(TRACE_HEADER_MULTI_READ) +#define __LINUX_DW_EDMA_TRACE + +#include +#include + +TRACE_DEFINE_ENUM(DW_EDMA_LL_EVENT_NONE); +TRACE_DEFINE_ENUM(DW_EDMA_LL_EVENT_PROGRESS); +TRACE_DEFINE_ENUM(DW_EDMA_LL_EVENT_STOP); + +DECLARE_EVENT_CLASS(edma_desc_info, + TP_PROTO(struct dw_edma_desc *desc), + TP_ARGS(desc), + TP_STRUCT__entry( + __field(size_t, nburst) + __field(size_t, start_burst) + __field(dma_cookie_t, cookie) + __field(u32, id) + __field(u8, dir) + ), + TP_fast_assign( + __entry->nburst =3D desc->nburst; + __entry->start_burst =3D desc->start_burst; + __entry->id =3D desc->chan->id; + __entry->dir =3D desc->chan->dir; + __entry->cookie =3D desc->vd.tx.cookie; + ), + TP_printk("chan %u%c desc %d nburst %zu start_burst %zu", + __entry->id, + __entry->dir ? 'R' : 'W', + __entry->cookie, + __entry->nburst, + __entry->start_burst) +); + +DEFINE_EVENT(edma_desc_info, edma_append_desc, + TP_PROTO(struct dw_edma_desc *desc), + TP_ARGS(desc) +); + +DEFINE_EVENT(edma_desc_info, edma_complete_desc, + TP_PROTO(struct dw_edma_desc *desc), + TP_ARGS(desc) +); + +TRACE_EVENT(edma_irq, + TP_PROTO(struct dw_edma_chan *chan, + const struct dw_edma_ll_snapshot *snapshot), + TP_ARGS(chan, snapshot), + TP_STRUCT__entry( + __field(u32, head) + __field(u32, done) + __field(u32, total) + __field(int, index) + __field(int, event) + __field(dma_cookie_t, completed_cookie) + __field(dma_cookie_t, cookie) + __field(u32, id) + __field(u8, dir) + ), + TP_fast_assign( + __entry->head =3D chan->ll_head; + __entry->done =3D chan->ll_done; + __entry->total =3D chan->ll_max; + __entry->index =3D snapshot->idx; + __entry->event =3D snapshot->event; + __entry->completed_cookie =3D chan->vc.chan.completed_cookie; + __entry->cookie =3D chan->vc.chan.cookie; + __entry->id =3D chan->id; + __entry->dir =3D chan->dir; + ), + TP_printk("chan %u%c event %s head %u done %u total %u idx %d completed %= d cookie %d", + __entry->id, + __entry->dir ? 'R' : 'W', + __print_symbolic(__entry->event, + { DW_EDMA_LL_EVENT_NONE, "none" }, + { DW_EDMA_LL_EVENT_PROGRESS, "progress" }, + { DW_EDMA_LL_EVENT_STOP, "stop" }), + __entry->head, + __entry->done, + __entry->total, + __entry->index, + __entry->completed_cookie, + __entry->cookie) +); + +TRACE_EVENT(edma_engine_recovery, + TP_PROTO(struct dw_edma_chan *chan, enum dma_status hw_status, + enum dw_edma_request request, enum dw_edma_status status, + u32 ll_head, u32 ll_done, u32 pending), + TP_ARGS(chan, hw_status, request, status, ll_head, ll_done, pending), + TP_STRUCT__entry( + __field(u32, id) + __field(u32, head) + __field(u32, done) + __field(u32, pending) + __field(int, hw_status) + __field(int, request) + __field(int, status) + __field(u8, dir) + ), + TP_fast_assign( + __entry->id =3D chan->id; + __entry->dir =3D chan->dir; + __entry->hw_status =3D hw_status; + __entry->request =3D request; + __entry->status =3D status; + __entry->head =3D ll_head; + __entry->done =3D ll_done; + __entry->pending =3D pending; + ), + TP_printk("chan %u%c hw %d request %d status %d head %u done %u pending %= u", + __entry->id, + __entry->dir ? 'R' : 'W', + __entry->hw_status, + __entry->request, + __entry->status, + __entry->head, + __entry->done, + __entry->pending) +); + +TRACE_EVENT(edma_fill_ll, + TP_PROTO(struct dw_edma_chan *chan, u32 idx, dma_cookie_t cookie, u64 src, + u64 dest, u32 sz, bool flag), + TP_ARGS(chan, idx, cookie, src, dest, sz, flag), + TP_STRUCT__entry( + __field(u32, idx) + __field(u64, src) + __field(u64, dest) + __field(u32, sz) + __field(u32, id) + __field(dma_cookie_t, cookie) + __field(bool, flag) + __field(u8, dir) + ), + TP_fast_assign( + __entry->idx =3D idx; + __entry->src =3D src; + __entry->dest =3D dest; + __entry->sz =3D sz; + __entry->id =3D chan->id; + __entry->dir =3D chan->dir; + __entry->cookie =3D cookie; + __entry->flag =3D flag; + ), + TP_printk("chan %u%c %d [%u] %c src: %08llx dest: %08llx sz: %04x", + __entry->id, + __entry->dir ? 'R' : 'W', + __entry->cookie, + __entry->idx, + __entry->flag ? 'C' : 'c', + __entry->src, + __entry->dest, + __entry->sz) +); + +#endif + +/* This part must be outside the header guard. */ + +#undef TRACE_INCLUDE_PATH +#define TRACE_INCLUDE_PATH . + +#undef TRACE_INCLUDE_FILE +#define TRACE_INCLUDE_FILE dw-edma-trace + +#include --=20 2.51.0