From nobody Wed Apr 8 07:27:03 2026 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass(p=none dis=none) header.from=gmail.com ARC-Seal: i=1; a=rsa-sha256; t=1775126880; cv=none; d=zohomail.com; s=zohoarc; b=lFJx+/jsTKWfwpOh8/agHhZyBvGYkviMSBucG6Eqr4RTufas1+swAvOfuaMG9F+BoAeT6cW7Eqw1VVwtzSil1XoZVdReOPpc8u9cgRWo4UJdolwcZmEsBRoUXBXwN8bSnWEnZ16duo1tgFITJyDg24/HytKzDBF+qR9vozNOQgY= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1775126880; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=qk9ILGeqhLaxN17ATAwS/XwCMf4ZnQc5O/7gi3JtyMs=; b=YvkVxDa8Z/ZIdmnEWEs2LKL64hSgodABIrQVbz5VMOGBbCqxi0AjaGxoXlCY6LbhqgX9AJqizeD4SNet3nC6+HE3KNwKzTh06kv29BTCxrAYe33RRArl1/iKfRo+DZOgbJNZWPrY56XHglj0d2Ou9erZfqs7piNKy1+XGy5sGkA= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1775126880571619.3623094910482; Thu, 2 Apr 2026 03:48:00 -0700 (PDT) Received: from list by lists.xenproject.org with outflank-mailman.1271525.1559674 (Exim 4.92) (envelope-from ) id 1w8FaH-00055g-PN; Thu, 02 Apr 2026 10:47:33 +0000 Received: by outflank-mailman (output) from mailman id 1271525.1559674; Thu, 02 Apr 2026 10:47:33 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1w8FaH-00054u-KI; Thu, 02 Apr 2026 10:47:33 +0000 Received: by outflank-mailman (input) for mailman id 1271525; Thu, 02 Apr 2026 10:47:32 +0000 Received: from mx.expurgate.net ([195.190.135.10]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1w8FaG-0004dR-6T for xen-devel@lists.xenproject.org; Thu, 02 Apr 2026 10:47:32 +0000 Received: from mx.expurgate.net (helo=localhost) by mx.expurgate.net with esmtp id 1w8FaF-009qA7-JE for xen-devel@lists.xenproject.org; Thu, 02 Apr 2026 12:47:31 +0200 Received: from [10.42.69.12] (helo=localhost) by localhost with ESMTP (eXpurgate MTA 0.9.1) (envelope-from ) id 69ce4932-e002-0a2a0a5209dd-0a2a450cd128-32 for ; Thu, 02 Apr 2026 12:47:31 +0200 Received: from [209.85.218.41] (helo=mail-ej1-f41.google.com) by tlsNG-d25034.mxtls.expurgate.net with ESMTPS (eXpurgate 4.56.0) (envelope-from ) id 69ce4943-f40c-0a2a450c0019-d155da29cd5e-3 for ; Thu, 02 Apr 2026 12:47:31 +0200 Received: by mail-ej1-f41.google.com with SMTP id a640c23a62f3a-b9910707d82so95092866b.1 for ; Thu, 02 Apr 2026 03:47:31 -0700 (PDT) Received: from EPUAKYIW02F7.. ([45.12.26.38]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-b9c3d028955sm76392366b.61.2026.04.02.03.47.28 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 02 Apr 2026 03:47:29 -0700 (PDT) X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" Authentication-Results: eu.smtp.expurgate.cloud; dkim=pass header.s=20251104 header.d=gmail.com header.i="@gmail.com" header.h="Content-Transfer-Encoding:MIME-Version:References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1775126850; x=1775731650; darn=lists.xenproject.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=qk9ILGeqhLaxN17ATAwS/XwCMf4ZnQc5O/7gi3JtyMs=; b=W2Opd3NOXDhqwaHjQ8eyEy6U8TOtgVfW0ynPPf7Fd9T/NOFIQ0to/UC5HxhgoxJyI3 lgzK7pP/tpks1HUR6xY9FN1X56jp6+ETN3wYpcYMjmJcrE38QhXcIKUOt8D8BlLtTYrm IdlqM1fJHt3YE+y/EmXU8biawm+fQMT+8Rxz0TrigSbwjXwwpb7q3/m0NYk8935fndcG x53BXqZy3srjMaU633N+S6int/NTI09Dqf+dTxlSq7gFDfIHVdCOGweFCJ4UOoKpPCTf VZWWtY7G6j+yVi6qpbqc1aM8gtCKFu7/KLos/96hARe0ZqWVNZJIjIOZ79Bm9Kq3HSFF gjHg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1775126850; x=1775731650; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=qk9ILGeqhLaxN17ATAwS/XwCMf4ZnQc5O/7gi3JtyMs=; b=UTu3MV1++V0XV6ubyzlS37DUYWZQEC7Hb3iONDj7F5LxaJkqnjXG0k2nBvcFyq/b75 yvKLHLfo7ZF+iVa98/dubLF+gZu23ICxRc53vAPaJHHHI1L+l9aYcfpdsT1IB6GfwCqb nWUpwgRqKvC3svXWEU34GqpFf19NaAY33blRRLhfY3vo6TrxQPcd4VzTw7veryglP7gf nZkZidzT3v8Rw8lI+1Q1+pI7CxLk//NpnLUAjfqp1MEtDE+jkcy4KZ66udj2kTdtqZcG YQGvieGENvp2DVJYSFYIoRHvWPLW1Qb4/EmZME0z2DlZlOsuJqcEQjHdSU/d3pGgGkzd J9aA== X-Gm-Message-State: AOJu0Ywicsxvk28ncZC9mwKYtqpHqA1BruQUwhxuOcAwPmW2btH+OcUT 4R+BQhjgP+CG/IvyHNGOAkWn2ne51pqgNX6VJRnOy+Ip4MYMAknCLTQ7PqCoGHET X-Gm-Gg: ATEYQzxAOeTpy4iksfHwyIe24RYxH0ZLuVgZslhLrVNIVaegd+4BDpqFZiHqZrJSMur ziNXD91CgA0HKYO7K8n8GvM3cEEIe4+Czv9aPzIZLgeIGM7iD6g5Aut4qa+rwvTp2R/AvaThuWJ J6dj1bhCEJZIcPTwcSgYAylzN2jSSzfJRf6JREyoo1oGsDgibPdNBtUHM4EyY6BSkIfg253acef Pyuc3mksGqmWexInwnrQ6VnB3IW7WmzF0FgoBqL7w4hzKocDA2OXPmFq4b5waaHE9hKEEOaDIOe bNl4x8Xs5G/JY4JBrafI1JW7j2eKTB93OoK88a6e4an0HP8ci4CT41rYbN5DgM4uVfL3mS6t7MU Pd+4H5hCboc8zZwjFpunCaQHs4AdlU7cAikkWw9UyF5H6W5letnvXOOfFCI0DsiPGlpBvn9oBMr 4+65b2aYxZHXxjEJPges2H07x22w== X-Received: by 2002:a17:907:1c85:b0:b93:7d70:20a0 with SMTP id a640c23a62f3a-b9c137a08acmr482815566b.4.1775126850107; Thu, 02 Apr 2026 03:47:30 -0700 (PDT) From: Mykola Kvach To: xen-devel@lists.xenproject.org Cc: Mykola Kvach , Stefano Stabellini , Julien Grall , Bertrand Marquis , Michal Orzel , Volodymyr Babchuk , Andrew Cooper , Anthony PERARD , Jan Beulich , =?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= Subject: [PATCH v8 05/13] xen/arm: gic-v3: add ITS suspend/resume support Date: Thu, 2 Apr 2026 13:45:06 +0300 Message-ID: <5a42b7c32fadf21262b8342f27e685916d0e5812.1775125380.git.mykola_kvach@epam.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: References: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-purgate-ID: tlsNG-d25034/1775126851-FE74EA3D-D1CDCDD3/0/0 X-purgate-type: clean X-purgate-size: 9675 X-ZohoMail-DKIM: pass (identity @gmail.com) X-ZM-MESSAGEID: 1775126882759154100 Content-Type: text/plain; charset="utf-8" From: Mykola Kvach Handle system suspend/resume for GICv3 with an ITS present so LPIs keep working after firmware powers the GIC down. Snapshot the CPU interface, distributor and last-CPU redistributor state, disable the ITS to cache its CTLR/CBASER/BASER registers, then restore everything and re-arm the collection on resume. Add list_for_each_entry_continue_reverse() in list.h for the ITS suspend error path that needs to roll back partially saved state. Based on Linux commit dba0bc7b76dc ("irqchip/gic-v3-its: Add ability to sav= e/restore ITS state") Signed-off-by: Mykola Kvach --- Changes in V8: - Reword the CBASER/CWRITER comment to match Xen and drop the stale Linux cmd_write reference. - Clarify the list_for_each_entry_continue_reverse() comment. - Factor out per-ITS helpers for collection setup and resume. - Restore each ITS and re-establish its collection mapping in the same loop, so a failed ITS resume is not followed by MAPC/SYNC on that un-restored instance. - panic in case when resume of an ITS failed - cleanup baser cache during suspend --- xen/arch/arm/gic-v3-its.c | 126 ++++++++++++++++++++++++-- xen/arch/arm/gic-v3.c | 15 ++- xen/arch/arm/include/asm/gic_v3_its.h | 23 +++++ xen/include/xen/list.h | 14 +++ 4 files changed, 166 insertions(+), 12 deletions(-) diff --git a/xen/arch/arm/gic-v3-its.c b/xen/arch/arm/gic-v3-its.c index 9ba068c46f..fe2865eac9 100644 --- a/xen/arch/arm/gic-v3-its.c +++ b/xen/arch/arm/gic-v3-its.c @@ -335,6 +335,22 @@ static int its_send_cmd_inv(struct host_its *its, return its_send_command(its, cmd); } =20 +static int gicv3_its_setup_collection_single(struct host_its *its, + unsigned int cpu) +{ + int ret; + + ret =3D its_send_cmd_mapc(its, cpu, cpu); + if ( ret ) + return ret; + + ret =3D its_send_cmd_sync(its, cpu); + if ( ret ) + return ret; + + return gicv3_its_wait_commands(its); +} + /* Set up the (1:1) collection mapping for the given host CPU. */ int gicv3_its_setup_collection(unsigned int cpu) { @@ -343,15 +359,7 @@ int gicv3_its_setup_collection(unsigned int cpu) =20 list_for_each_entry(its, &host_its_list, entry) { - ret =3D its_send_cmd_mapc(its, cpu, cpu); - if ( ret ) - return ret; - - ret =3D its_send_cmd_sync(its, cpu); - if ( ret ) - return ret; - - ret =3D gicv3_its_wait_commands(its); + ret =3D gicv3_its_setup_collection_single(its, cpu); if ( ret ) return ret; } @@ -1209,6 +1217,106 @@ int gicv3_its_init(void) return 0; } =20 +#ifdef CONFIG_SYSTEM_SUSPEND +int gicv3_its_suspend(void) +{ + struct host_its *its; + int ret; + + list_for_each_entry(its, &host_its_list, entry) + { + unsigned int i; + void __iomem *base =3D its->its_base; + + its->suspend_ctx.ctlr =3D readl_relaxed(base + GITS_CTLR); + ret =3D gicv3_disable_its(its); + if ( ret ) + { + writel_relaxed(its->suspend_ctx.ctlr, base + GITS_CTLR); + goto err; + } + + its->suspend_ctx.cbaser =3D readq_relaxed(base + GITS_CBASER); + + for (i =3D 0; i < GITS_BASER_NR_REGS; i++) + { + uint64_t baser =3D readq_relaxed(base + GITS_BASER0 + i * 8); + + its->suspend_ctx.baser[i] =3D 0; + + if ( !(baser & GITS_VALID_BIT) ) + continue; + + its->suspend_ctx.baser[i] =3D baser; + } + } + + return 0; + + err: + list_for_each_entry_continue_reverse(its, &host_its_list, entry) + writel_relaxed(its->suspend_ctx.ctlr, its->its_base + GITS_CTLR); + + return ret; +} + +static int gicv3_its_resume_single(struct host_its *its, unsigned int cpu) +{ + void __iomem *base =3D its->its_base; + unsigned int i; + int ret; + + /* + * Make sure that the ITS is disabled. If it fails to quiesce, + * don't restore it since writing to CBASER or BASER + * registers is undefined according to the GIC v3 ITS + * Specification. + */ + WARN_ON(readl_relaxed(base + GITS_CTLR) & GITS_CTLR_ENABLE); + ret =3D gicv3_disable_its(its); + if ( ret ) + return ret; + + writeq_relaxed(its->suspend_ctx.cbaser, base + GITS_CBASER); + + /* + * Writing CBASER resets CREADR to 0, so reset CWRITER to + * keep the command queue pointers aligned. + */ + writeq_relaxed(0, base + GITS_CWRITER); + + /* Restore GITS_BASER from the value cache. */ + for ( i =3D 0; i < GITS_BASER_NR_REGS; i++ ) + { + uint64_t baser =3D its->suspend_ctx.baser[i]; + + if ( !(baser & GITS_VALID_BIT) ) + continue; + + writeq_relaxed(baser, base + GITS_BASER0 + i * 8); + } + + writel_relaxed(its->suspend_ctx.ctlr, base + GITS_CTLR); + + return gicv3_its_setup_collection_single(its, cpu); +} + +void gicv3_its_resume(void) +{ + struct host_its *its; + unsigned int cpu =3D smp_processor_id(); + int ret; + + list_for_each_entry(its, &host_its_list, entry) + { + ret =3D gicv3_its_resume_single(its, cpu); + if ( ret ) + panic("GICv3: ITS@%"PRIpaddr": failed to restore during resume= : %d\n", + its->addr, ret); + } +} + +#endif /* CONFIG_SYSTEM_SUSPEND */ =20 /* * Local variables: diff --git a/xen/arch/arm/gic-v3.c b/xen/arch/arm/gic-v3.c index d182a71478..ef8318dd50 100644 --- a/xen/arch/arm/gic-v3.c +++ b/xen/arch/arm/gic-v3.c @@ -862,7 +862,7 @@ static bool gicv3_enable_lpis(void) return true; } =20 -static int __init gicv3_populate_rdist(void) +static int gicv3_populate_rdist(void) { int i; uint32_t aff; @@ -932,7 +932,7 @@ static int __init gicv3_populate_rdist(void) ret =3D gicv3_lpi_init_rdist(ptr); if ( ret && ret !=3D -ENODEV && ret !=3D -EBUSY ) { - printk("GICv3: CPU%d: Cannot initialize LPIs: %u\n= ", + printk("GICv3: CPU%d: Cannot initialize LPIs: %d\n= ", smp_processor_id(), ret); break; } @@ -2101,10 +2101,14 @@ static int gicv3_suspend(void) =20 gicv3_disable_interface(); =20 - ret =3D gicv3_disable_redist(); + ret =3D gicv3_its_suspend(); if ( ret ) goto out_enable_iface; =20 + ret =3D gicv3_disable_redist(); + if ( ret ) + goto out_its_resume; + /* Save GICR configuration */ gicv3_redist_wait_for_rwp(); =20 @@ -2140,6 +2144,9 @@ static int gicv3_suspend(void) =20 return 0; =20 + out_its_resume: + gicv3_its_resume(); + out_enable_iface: gicv3_hyp_enable(true); WRITE_SYSREG(gicv3_ctx.cpu.ctlr, ICC_CTLR_EL1); @@ -2212,6 +2219,8 @@ static void gicv3_resume(void) =20 gicv3_redist_wait_for_rwp(); =20 + gicv3_its_resume(); + WRITE_SYSREG(gicv3_ctx.cpu.sre_el2, ICC_SRE_EL2); isb(); =20 diff --git a/xen/arch/arm/include/asm/gic_v3_its.h b/xen/arch/arm/include/a= sm/gic_v3_its.h index 081bd19180..3ca74435c8 100644 --- a/xen/arch/arm/include/asm/gic_v3_its.h +++ b/xen/arch/arm/include/asm/gic_v3_its.h @@ -129,6 +129,13 @@ struct host_its { spinlock_t cmd_lock; void *cmd_buf; unsigned int flags; +#ifdef CONFIG_SYSTEM_SUSPEND + struct suspend_ctx { + uint32_t ctlr; + uint64_t cbaser; + uint64_t baser[GITS_BASER_NR_REGS]; + } suspend_ctx; +#endif }; =20 /* Map a collection for this host CPU to each host ITS. */ @@ -205,6 +212,11 @@ uint64_t gicv3_its_get_cacheability(void); uint64_t gicv3_its_get_shareability(void); unsigned int gicv3_its_get_memflags(void); =20 +#ifdef CONFIG_SYSTEM_SUSPEND +int gicv3_its_suspend(void); +void gicv3_its_resume(void); +#endif + #else =20 #ifdef CONFIG_ACPI @@ -272,6 +284,17 @@ static inline int gicv3_its_make_hwdom_dt_nodes(const = struct domain *d, return 0; } =20 +#ifdef CONFIG_SYSTEM_SUSPEND +static inline int gicv3_its_suspend(void) +{ + return 0; +} + +static inline void gicv3_its_resume(void) +{ +} +#endif + #endif /* CONFIG_HAS_ITS */ =20 #endif diff --git a/xen/include/xen/list.h b/xen/include/xen/list.h index 98d8482dab..2aab274157 100644 --- a/xen/include/xen/list.h +++ b/xen/include/xen/list.h @@ -535,6 +535,20 @@ static inline void list_splice_init(struct list_head *= list, &(pos)->member !=3D (head); = \ (pos) =3D list_entry((pos)->member.next, typeof(*(pos)), member)) =20 +/** + * list_for_each_entry_continue_reverse - iterate backwards from the given= point + * @pos: the type * to use as a loop cursor. + * @head: the head for your list. + * @member: the name of the list_head within the struct. + * + * Iterate over list of given type backwards, starting from the element pr= evious + * to the current one in list order. + */ +#define list_for_each_entry_continue_reverse(pos, head, member) \ + for ((pos) =3D list_entry((pos)->member.prev, typeof(*(pos)), member);= \ + &(pos)->member !=3D (head); = \ + (pos) =3D list_entry((pos)->member.prev, typeof(*(pos)), member)) + /** * list_for_each_entry_from - iterate over list of given type from the * current point --=20 2.43.0