From nobody Tue Feb 10 23:39:38 2026 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass(p=none dis=none) header.from=gmail.com ARC-Seal: i=1; a=rsa-sha256; t=1765478786; cv=none; d=zohomail.com; s=zohoarc; b=CvJXbXUaGVBXgKikk+fPaG61MGXNk8ERWnha2fjCr+WhXvvqDsf8n+b0Yr580OLc6DY6XHGtti4X5spUU8jpIVdGKENB9fV8LB9anY20Nz4SVg787Ul1ot7dwaBYh6zzMGFBGMJxHq8Xmw5sea8tC+QNlJLk0G6O1NXNMc66e/Q= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1765478786; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=YyX+DKhss3Tir9n7VoBzOtKgYwwNvTr5WMzP0cAUYTo=; b=R6Z1UfhenCpOeMoLC3mWKRjahNNpUKks3l9zimOpFCz0UeLJ1+B777NcQ9nW303LVBP/OE9zYdJuLBJWTfcCCHk8i3DZvQpm9jLMK74USOitAoTBFnYY4KxDDb74xHRWg2hD8b5V7q6v9lGMoXQG5jB0iz3nEF9wTzS/OiAkHsQ= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1765478786576190.6617171409771; Thu, 11 Dec 2025 10:46:26 -0800 (PST) Received: from list by lists.xenproject.org with outflank-mailman.1184552.1507024 (Exim 4.92) (envelope-from ) id 1vTlfs-00052j-EQ; Thu, 11 Dec 2025 18:46:00 +0000 Received: by outflank-mailman (output) from mailman id 1184552.1507024; Thu, 11 Dec 2025 18:46:00 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1vTlfs-00052Y-9j; Thu, 11 Dec 2025 18:46:00 +0000 Received: by outflank-mailman (input) for mailman id 1184552; Thu, 11 Dec 2025 18:45:58 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1vTlfq-0003I8-OZ for xen-devel@lists.xenproject.org; Thu, 11 Dec 2025 18:45:58 +0000 Received: from mail-ej1-x632.google.com (mail-ej1-x632.google.com [2a00:1450:4864:20::632]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id a1a96d36-d6c1-11f0-b15b-2bf370ae4941; Thu, 11 Dec 2025 19:45:57 +0100 (CET) Received: by mail-ej1-x632.google.com with SMTP id a640c23a62f3a-b734fcbf1e3so90814766b.3 for ; Thu, 11 Dec 2025 10:45:58 -0800 (PST) Received: from EPUAKYIW02F7.. (pool185-5-253-4.as6723.net. [185.5.253.4]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-b7cfa5d0b0dsm345870466b.67.2025.12.11.10.45.55 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 11 Dec 2025 10:45:56 -0800 (PST) X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: a1a96d36-d6c1-11f0-b15b-2bf370ae4941 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1765478757; x=1766083557; darn=lists.xenproject.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=YyX+DKhss3Tir9n7VoBzOtKgYwwNvTr5WMzP0cAUYTo=; b=SeqabM84oMlxwqvC9qGw216lkj/k6KV2LnAvS0TPSvHsyRLEN/t2xMUrxPRl8qqYnK o1hzMP15Q2yDfptcZKqlFrqVPiSNnyShUcryIaKEmqEsQZTImE3HgfQWMh4FbIuADwlK VynWznj/G9dqfOWi9DTiAVM76ItI21u1+dDa64zmEd/mGIuMg+X/HPppdNJWs8yaQctx OAoy69Hq+kEp87Ub/0kqNQQKm3TBk5sm3yNm0QFaisBE2sleRBSbNhmEySnP6fgKQaxB hq4CtStARO8ACsbW27XHsbkku3zm0O/71jywK6WT0hWkUBuWoEpLusFnt7yRlktkVIZ2 1lGw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1765478757; x=1766083557; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=YyX+DKhss3Tir9n7VoBzOtKgYwwNvTr5WMzP0cAUYTo=; b=bb3ayGURspc+VaQNkBWEp5NQbkBdElX37swNx56XOe41UMlVoBWwl3KmagthXQtqRc zxvVewya7kspU/ly2blIoHACz61CHQqkxv/SmZH/bk/Yg5PNEc/uHMyDB5g5bixUCN3+ 4ODA0KqkQ8sPJdlSAryzNNVGvjH59D/jAYTvbGGUbryrBY3oOR4PhOJexpY3I6XeKQkR VFHT5qKCZOPwcgQzlHz0LrRuLBIMF9G44kojDrV3eIIqJAj+I9pK1GoWl27601E3OdN6 T15FepqEPxp0pV0KBTYFnUS5YC9yg2CyXNJYQVPufM2FBvr9rhFT30wITEpFyrqDo3iZ 9wDQ== X-Gm-Message-State: AOJu0Yw9FNZwarjCXykUs1z4rZla+CjgaSezchPNPpP5xxM1Ocw70/uj pWtNBMjZnSjtTaSbGAPKTkqsSij+Y7FJriaX8vhIqfZMqJeaoPkGRxMCp0cmF5P4 X-Gm-Gg: AY/fxX5S3XQ9XxJ6wT/ztpgsEdWea2B38IJ/7aUdQkpYicv3disAl2aZLuHOna4e4by Fl5sWNHDLPVJnSlfp7WeMH1unMrE2NX1DmNnwac34criY3HeAQ/l6nU7yikr6qIXKen5rXeiW+b 4pY2thGg7BUTZufzav042/m+OUn+7H1JgwafvbCaJXjDhEMnsKZUnvwJbdvGgCcSZyH+7lEA8Kj xLAQBvPVaSMUFBFJc8GYnAG5slwyZm7ULz/UOZfqVyPYf95rCpJMJgOJcOroM0KGjkzqZZeJpsO T1B1+82rgCpsTyIt5f5QzIfFGNcnIKu0pV+hrFsUSn2HFXVucCbqgjxax9psyTH4ES4uSe1Kj6o RR03HnDoTnUGeN/1xd5hUZ24OE7yhGE1q9+EbEwQIBkrJNCJKwco+XmbWHZvpoSifXfOgcM2H55 16QuQVox02Z1uAQ31kBDKEVy0jXBqgk5679Prc1DVE X-Google-Smtp-Source: AGHT+IHqZwnZeJZ9SiPy9sMGvjH5yQTECZVUy2zQSTwn/fCCIricEV1JkHIoYJqUXa8UozBVjbaDgw== X-Received: by 2002:a17:907:6ea7:b0:b73:5c12:3f8a with SMTP id a640c23a62f3a-b7ce826e8b7mr710504466b.18.1765478756907; Thu, 11 Dec 2025 10:45:56 -0800 (PST) From: Mykola Kvach To: xen-devel@lists.xenproject.org Cc: Mykola Kvach , Mykola Kvach , Bertrand Marquis , Rahul Singh , Stefano Stabellini , Julien Grall , Michal Orzel , Volodymyr Babchuk , Pranjal Shrivastava Subject: [PATCH v7 08/12] arm/smmu-v3: add suspend/resume handlers Date: Thu, 11 Dec 2025 20:43:32 +0200 Message-ID: <58c1873d355f5ea9b5182349895905d25cb57256.1765472890.git.mykola_kvach@epam.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: References: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @gmail.com) X-ZM-MESSAGEID: 1765478788667158500 Content-Type: text/plain; charset="utf-8" From: Mykola Kvach Before we suspend SMMU, we want to ensure that all commands (especially ATC_INV) have been flushed by the CMDQ, i.e. the CMDQs are empty. The suspend callback configures the SMMU to abort new transactions, disables the main translation unit and then drains the command queue to ensure completion of any in-flight commands. The resume callback performs a full device reset via 'arm_smmu_device_reset' to bring the SMMU back to an operational state. Link: https://lore.kernel.org/linux-iommu/20251117191433.3360130-1-praan@go= ogle.com / Based-on-patch-by: Pranjal Shrivastava Signed-off-by: Mykola Kvach --- xen/drivers/passthrough/arm/smmu-v3.c | 170 ++++++++++++++++++++------ 1 file changed, 134 insertions(+), 36 deletions(-) diff --git a/xen/drivers/passthrough/arm/smmu-v3.c b/xen/drivers/passthroug= h/arm/smmu-v3.c index bf153227db..10c4c5dee0 100644 --- a/xen/drivers/passthrough/arm/smmu-v3.c +++ b/xen/drivers/passthrough/arm/smmu-v3.c @@ -1814,8 +1814,7 @@ static int arm_smmu_write_reg_sync(struct arm_smmu_de= vice *smmu, u32 val, } =20 /* GBPA is "special" */ -static int __init arm_smmu_update_gbpa(struct arm_smmu_device *smmu, - u32 set, u32 clr) +static int arm_smmu_update_gbpa(struct arm_smmu_device *smmu, u32 set, u32= clr) { int ret; u32 reg, __iomem *gbpa =3D smmu->base + ARM_SMMU_GBPA; @@ -1995,10 +1994,29 @@ err_free_evtq_irq: return ret; } =20 +static int arm_smmu_enable_irqs(struct arm_smmu_device *smmu) +{ + int ret; + u32 irqen_flags =3D IRQ_CTRL_EVTQ_IRQEN | IRQ_CTRL_GERROR_IRQEN; + + if ( smmu->features & ARM_SMMU_FEAT_PRI ) + irqen_flags |=3D IRQ_CTRL_PRIQ_IRQEN; + + /* Enable interrupt generation on the SMMU */ + ret =3D arm_smmu_write_reg_sync(smmu, irqen_flags, + ARM_SMMU_IRQ_CTRL, ARM_SMMU_IRQ_CTRLACK); + if ( ret ) + { + dev_warn(smmu->dev, "failed to enable irqs\n"); + return ret; + } + + return 0; +} + static int __init arm_smmu_setup_irqs(struct arm_smmu_device *smmu) { int ret, irq; - u32 irqen_flags =3D IRQ_CTRL_EVTQ_IRQEN | IRQ_CTRL_GERROR_IRQEN; =20 /* Disable IRQs first */ ret =3D arm_smmu_write_reg_sync(smmu, 0, ARM_SMMU_IRQ_CTRL, @@ -2028,22 +2046,7 @@ static int __init arm_smmu_setup_irqs(struct arm_smm= u_device *smmu) } } =20 - if (smmu->features & ARM_SMMU_FEAT_PRI) - irqen_flags |=3D IRQ_CTRL_PRIQ_IRQEN; - - /* Enable interrupt generation on the SMMU */ - ret =3D arm_smmu_write_reg_sync(smmu, irqen_flags, - ARM_SMMU_IRQ_CTRL, ARM_SMMU_IRQ_CTRLACK); - if (ret) { - dev_warn(smmu->dev, "failed to enable irqs\n"); - goto err_free_irqs; - } - return 0; - -err_free_irqs: - arm_smmu_free_irqs(smmu); - return ret; } =20 static int arm_smmu_device_disable(struct arm_smmu_device *smmu) @@ -2057,7 +2060,7 @@ static int arm_smmu_device_disable(struct arm_smmu_de= vice *smmu) return ret; } =20 -static int __init arm_smmu_device_reset(struct arm_smmu_device *smmu) +static int arm_smmu_device_reset(struct arm_smmu_device *smmu) { int ret; u32 reg, enables; @@ -2163,17 +2166,9 @@ static int __init arm_smmu_device_reset(struct arm_s= mmu_device *smmu) } } =20 - ret =3D arm_smmu_setup_irqs(smmu); - if (ret) { - dev_err(smmu->dev, "failed to setup irqs\n"); + ret =3D arm_smmu_enable_irqs(smmu); + if ( ret ) return ret; - } - - /* Initialize tasklets for threaded IRQs*/ - tasklet_init(&smmu->evtq_irq_tasklet, arm_smmu_evtq_tasklet, smmu); - tasklet_init(&smmu->priq_irq_tasklet, arm_smmu_priq_tasklet, smmu); - tasklet_init(&smmu->combined_irq_tasklet, arm_smmu_combined_irq_tasklet, - smmu); =20 /* Enable the SMMU interface, or ensure bypass */ if (disable_bypass) { @@ -2181,20 +2176,16 @@ static int __init arm_smmu_device_reset(struct arm_= smmu_device *smmu) } else { ret =3D arm_smmu_update_gbpa(smmu, 0, GBPA_ABORT); if (ret) - goto err_free_irqs; + return ret; } ret =3D arm_smmu_write_reg_sync(smmu, enables, ARM_SMMU_CR0, ARM_SMMU_CR0ACK); if (ret) { dev_err(smmu->dev, "failed to enable SMMU interface\n"); - goto err_free_irqs; + return ret; } =20 return 0; - -err_free_irqs: - arm_smmu_free_irqs(smmu); - return ret; } =20 static int arm_smmu_device_hw_probe(struct arm_smmu_device *smmu) @@ -2558,10 +2549,23 @@ static int __init arm_smmu_device_probe(struct plat= form_device *pdev) if (ret) goto out_free; =20 + ret =3D arm_smmu_setup_irqs(smmu); + if ( ret ) + { + dev_err(smmu->dev, "failed to setup irqs\n"); + goto out_free; + } + + /* Initialize tasklets for threaded IRQs*/ + tasklet_init(&smmu->evtq_irq_tasklet, arm_smmu_evtq_tasklet, smmu); + tasklet_init(&smmu->priq_irq_tasklet, arm_smmu_priq_tasklet, smmu); + tasklet_init(&smmu->combined_irq_tasklet, arm_smmu_combined_irq_tasklet, + smmu); + /* Reset the device */ ret =3D arm_smmu_device_reset(smmu); if (ret) - goto out_free; + goto out_free_irqs; =20 /* * Keep a list of all probed devices. This will be used to query @@ -2575,6 +2579,8 @@ static int __init arm_smmu_device_probe(struct platfo= rm_device *pdev) =20 return 0; =20 +out_free_irqs: + arm_smmu_free_irqs(smmu); =20 out_free: arm_smmu_free_structures(smmu); @@ -2855,6 +2861,94 @@ static void arm_smmu_iommu_xen_domain_teardown(struc= t domain *d) xfree(xen_domain); } =20 +#ifdef CONFIG_SYSTEM_SUSPEND + +static int arm_smmu_suspend(void) +{ + struct arm_smmu_device *smmu; + int ret =3D 0; + + list_for_each_entry(smmu, &arm_smmu_devices, devices) + { + /* Abort all transactions before disable to avoid spurious bypass */ + ret =3D arm_smmu_update_gbpa(smmu, GBPA_ABORT, 0); + if ( ret ) + goto fail; + + /* Disable the SMMU via CR0.EN and all queues except CMDQ */ + ret =3D arm_smmu_write_reg_sync(smmu, CR0_CMDQEN, ARM_SMMU_CR0, + ARM_SMMU_CR0ACK); + if ( ret ) + { + dev_err(smmu->dev, "Timed-out while disabling smmu\n"); + goto fail; + } + + /* + * At this point the SMMU is completely disabled and won't access + * any translation/config structures, even speculative accesses + * aren't performed as per the IHI0070 spec (section 6.3.9.6). + */ + + /* Wait for the CMDQs to be drained to flush any pending commands */ + ret =3D queue_poll_cons(&smmu->cmdq.q, true, 0); + if ( ret ) + { + dev_err(smmu->dev, "Draining queues timed-out\n"); + goto fail; + } + + /* Disable everything */ + ret =3D arm_smmu_device_disable(smmu); + if ( ret ) + goto fail; + + dev_dbg(smmu->dev, "Suspended smmu\n"); + } + + return 0; + + fail: + { + int rc; + + /* Reset the device that failed as well as any already-suspended ones. */ + rc =3D arm_smmu_device_reset(smmu); + if ( rc ) + dev_err(smmu->dev, "Failed to reset during resume operation: %d\n", rc); + + list_for_each_entry_continue_reverse(smmu, &arm_smmu_devices, devices) + { + rc =3D arm_smmu_device_reset(smmu); + if ( rc ) + dev_err(smmu->dev, "Failed to reset during resume operation: %d\n", rc= ); + } + } + + return ret; +} + +static void arm_smmu_resume(void) +{ + int ret; + struct arm_smmu_device *smmu; + + list_for_each_entry(smmu, &arm_smmu_devices, devices) + { + dev_dbg(smmu->dev, "Resuming device\n"); + + /* + * The reset will re-initialize all the base addresses, queues, + * prod and cons maintained within struct arm_smmu_device as well as + * re-enable the interrupts. + */ + ret =3D arm_smmu_device_reset(smmu); + if ( ret ) + dev_err(smmu->dev, "Failed to reset during resume operation: %d\n", ret= ); + } +} +#endif + static const struct iommu_ops arm_smmu_iommu_ops =3D { .page_sizes =3D PAGE_SIZE_4K, .init =3D arm_smmu_iommu_xen_domain_init, @@ -2867,6 +2961,10 @@ static const struct iommu_ops arm_smmu_iommu_ops =3D= { .unmap_page =3D arm_iommu_unmap_page, .dt_xlate =3D arm_smmu_dt_xlate, .add_device =3D arm_smmu_add_device, +#ifdef CONFIG_SYSTEM_SUSPEND + .suspend =3D arm_smmu_suspend, + .resume =3D arm_smmu_resume, +#endif }; =20 static __init int arm_smmu_dt_init(struct dt_device_node *dev, --=20 2.43.0