From nobody Wed Apr 8 05:51:36 2026 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass(p=none dis=none) header.from=gmail.com ARC-Seal: i=1; a=rsa-sha256; t=1775126883; cv=none; d=zohomail.com; s=zohoarc; b=AvFAmYk5tIsoA1WbSasgkgNSdpr5lNeVRzmvX2pXMVMqUgy2YCPqC/KlsC7QmTFHXiCwnx39cufOSkQEkgGE4IUZQy0kLWYvooPEHBc2mXk0hq9CN0qU4KreyzluGe9BaJrvfUzGZNUXZheKmKqaTYFLUfoeW4qTImfpsOMOf/Y= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1775126883; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=Q0cUh9DunvbioAZLIRhQfJTs0EihHFj+/wkrJJoLmJs=; b=i6mq4a7moDiClUO9iawIy/xzB02dsk74kV+wdLEkBpQR3ynGeeWb9/+Lz3/lHM6KhvN0tFKmhilQmp1D/qdwqX8SR2k5zxUXMpeDZxrf9lwj/Okp3k7+JTCoAOX4stE0u0btR7ENHqxBsjF0Pf4Dv640gKDadWxZaDtMhD9ng94= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1775126883574105.47409252939372; Thu, 2 Apr 2026 03:48:03 -0700 (PDT) Received: from list by lists.xenproject.org with outflank-mailman.1271520.1559634 (Exim 4.92) (envelope-from ) id 1w8FaD-00045P-Di; Thu, 02 Apr 2026 10:47:29 +0000 Received: by outflank-mailman (output) from mailman id 1271520.1559634; Thu, 02 Apr 2026 10:47:29 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1w8FaD-00044A-Ah; Thu, 02 Apr 2026 10:47:29 +0000 Received: by outflank-mailman (input) for mailman id 1271520; Thu, 02 Apr 2026 10:47:28 +0000 Received: from mx.expurgate.net ([195.190.135.10]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1w8FaC-0003yr-3q for xen-devel@lists.xenproject.org; Thu, 02 Apr 2026 10:47:28 +0000 Received: from mx.expurgate.net (helo=localhost) by mx.expurgate.net with esmtp id 1w8FaB-009q68-FZ for xen-devel@lists.xenproject.org; Thu, 02 Apr 2026 12:47:27 +0200 Received: from [10.42.69.10] (helo=localhost) by localhost with ESMTP (eXpurgate MTA 0.9.1) (envelope-from ) id 69ce493f-e002-0a2a0a5209dd-0a2a450a8a2e-0 for ; Thu, 02 Apr 2026 12:47:27 +0200 Received: from [209.85.218.46] (helo=mail-ej1-f46.google.com) by tlsNG-4011c0.mxtls.expurgate.net with ESMTPS (eXpurgate 4.56.0) (envelope-from ) id 69ce493f-ee98-0a2a450a0019-d155da2edc71-3 for ; Thu, 02 Apr 2026 12:47:27 +0200 Received: by mail-ej1-f46.google.com with SMTP id a640c23a62f3a-b9c280322e0so86796966b.0 for ; Thu, 02 Apr 2026 03:47:27 -0700 (PDT) Received: from EPUAKYIW02F7.. ([45.12.26.38]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-b9c3d028955sm76392366b.61.2026.04.02.03.47.25 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 02 Apr 2026 03:47:25 -0700 (PDT) X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" Authentication-Results: eu.smtp.expurgate.cloud; dkim=pass header.s=20251104 header.d=gmail.com header.i="@gmail.com" header.h="Content-Transfer-Encoding:MIME-Version:References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1775126846; x=1775731646; darn=lists.xenproject.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Q0cUh9DunvbioAZLIRhQfJTs0EihHFj+/wkrJJoLmJs=; b=DgsaPB8gZzUseSYFpEQGPQeE1chIRot2lr90k5iTVS6HNsRvjL3FdDv9QcOPCABT/P A1uq2WUW68cKqRyyEpnDc/+DjEbG4e+8Ckbkvt60Xup+8z3aJXEx7pIhTO/PulvORb0M 9bEHd0M4vf8yhjWS/WAfyuhw8JOuZhfSSQfWxX+O8iYaJ63cucfQCkGwnaCorqVfJnro XKEz2jTeXX6KIgAYP8n/agVJpnNx8vREO4etwfGlzvoioHL5GeQ7LAeMqERcZ32dPl2M uiVN5oiWafmVmjIVEsmSZiJhZATbMvVdWF3+M8mT7eKayh9En1uGB2fN6AX/IyotDxQW 0MSQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1775126846; x=1775731646; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=Q0cUh9DunvbioAZLIRhQfJTs0EihHFj+/wkrJJoLmJs=; b=gZytgkB6vKqyY+dzHO/Y3UaM+13tJKlw+dFJNCQWe4CbX+lN/K3foXvpeQ1v0NimgN lDJLqERFzNfYbYX1xuADkMxedSkSoUA7UPPwuX1DGXe6V66HgXcFYRW9mEAnJl+Uvaaj PPE/OJlEmmHDhyhxcJccJQP9EoBRJp68++qQNoOe4ukupiH/xj4i/L2jxANYSmKeYusA lCgqxqfZDv/06LKUjvuX3swEwaGydpc0/JCt3oklcU24MnPUy541RFtsv0garZQfqaIb ihujNOBI4UYDa4GQF13vsd/G1WD2fcCBn8yyr64zldZTxX/pt7f8CSp61j+dHP7rcWav /Jhw== X-Gm-Message-State: AOJu0YzSR2/+zYY8IW1ws+/2d3y6DFr9rR8PE/UJX+h7fjoOpPtDg5Ed 4iLjluHt81Mwyq+cwZq5jsWhoEzgHS/iZEqWNzP+W6eHjKchjm9t7P19EdhUZkwJ X-Gm-Gg: ATEYQzycnmdYyIMhm/MJPwYLF7L3O+oWUJka+LdoaEBitdbYA/okopmmhNwrx5lgvRS h9ZJ5iOE1dUSnTPBAd2IftxPwQfdbv+pPYlBXl5xFFMa6ov9FaqLK+I3WSyNQYSDSiW2bGRKWm2 k1VbNk04LOngCGVLpGG9DyXsU/kp+P9w7wxFZbbb6K7e85F1YcTZ/zhe74v5c4rgxzJsY6KXVxX l9095o1BJELbpFn7mhDC93VWriz3fQ9YY5wZ/s1WKcqMI4NNT9+oV1Nu7jorJltelUl0To31mTv 35VK/XDlqbANtbZrFFyLXP9aPpXObSfY+r0ivWzeWzebKZYL1UYRpRX9eVd8eL2ldawfO3y0b8d crrn4WGwjHF9VP3d8uyOZlNleA61UH4Ix+ESMSKd30VA1lj+gcS+ufhAEGc6ucwQwxXrWqhEg3X T34dyHGRmq96oUkzG0CDdUvPc/K+2wrEOt2hJj X-Received: by 2002:a17:907:724d:b0:b97:98b3:67ce with SMTP id a640c23a62f3a-b9c13b3ccc9mr480119866b.32.1775126845902; Thu, 02 Apr 2026 03:47:25 -0700 (PDT) From: Mykola Kvach To: xen-devel@lists.xenproject.org Cc: Mykola Kvach , Stefano Stabellini , Julien Grall , Bertrand Marquis , Michal Orzel , Volodymyr Babchuk , Julien Grall Subject: [PATCH v8 01/13] xen/arm: Add suspend and resume timer helpers Date: Thu, 2 Apr 2026 13:45:02 +0300 Message-ID: <9b5e454f55d20d3e81771da53ae06fb511bd198e.1775125380.git.mykola_kvach@epam.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: References: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-purgate-ID: tlsNG-4011c0/1775126847-0F3490B1-23AF1613/0/0 X-purgate-type: clean X-purgate-size: 5007 X-ZohoMail-DKIM: pass (identity @gmail.com) X-ZM-MESSAGEID: 1775126885773158500 Content-Type: text/plain; charset="utf-8" From: Mirela Simonovic Timer interrupts must be disabled while the system is suspended to prevent spurious wake-ups. Suspending timers in Xen consists of disabling the physical timer and the hypervisor timer on the current CPU. The virtual timer does not need explicit handling here, as it is already disabled on vCPU context switch and its state is restored per-vCPU on the next context restore. Resuming consists of raising TIMER_SOFTIRQ, which prompts the generic timer code to reprogram the hypervisor timer with the correct timeout. Xen does not use or expose the physical timer, so it remains disabled across suspend/resume. Introduce a new helper, disable_phys_hyp_timers(), to encapsulate disabling of the physical and hypervisor timers. Signed-off-by: Mirela Simonovic Signed-off-by: Saeed Nowshadi Signed-off-by: Mykola Kvach Acked-by: Julien Grall --- Changes in V7: - Dropped EL1/EL2 wording; use "physical timer" and "hypervisor timer" - Renamed helper to disable_phys_hyp_timers() to reflect its actual scope - Clarified virtual timer handling (disabled on vCPU switch-out, restored= on context restore) and added comments in suspend/resume paths - Added resume comment explaining which timers are restored by TIMER_SOFT= IRQ --- xen/arch/arm/include/asm/time.h | 5 ++++ xen/arch/arm/time.c | 44 ++++++++++++++++++++++++++++----- 2 files changed, 43 insertions(+), 6 deletions(-) diff --git a/xen/arch/arm/include/asm/time.h b/xen/arch/arm/include/asm/tim= e.h index c194dbb9f5..9313b157ea 100644 --- a/xen/arch/arm/include/asm/time.h +++ b/xen/arch/arm/include/asm/time.h @@ -105,6 +105,11 @@ void preinit_xen_time(void); =20 void force_update_vcpu_system_time(struct vcpu *v); =20 +#ifdef CONFIG_SYSTEM_SUSPEND +void time_suspend(void); +void time_resume(void); +#endif /* CONFIG_SYSTEM_SUSPEND */ + #endif /* __ARM_TIME_H__ */ /* * Local variables: diff --git a/xen/arch/arm/time.c b/xen/arch/arm/time.c index a12912a106..f91dc64099 100644 --- a/xen/arch/arm/time.c +++ b/xen/arch/arm/time.c @@ -296,6 +296,14 @@ static void check_timer_irq_cfg(unsigned int irq, cons= t char *which) static DEFINE_PER_CPU_READ_MOSTLY(struct irqaction, irq_hyp); static DEFINE_PER_CPU_READ_MOSTLY(struct irqaction, irq_virt); =20 +/* Disable physical and hypervisor timers on the current CPU */ +static inline void disable_phys_hyp_timers(void) +{ + WRITE_SYSREG(0, CNTP_CTL_EL0); /* Physical timer disabled */ + WRITE_SYSREG(0, CNTHP_CTL_EL2); /* Hypervisor's timer disabled */ + isb(); +} + /* Set up the timer interrupt on this CPU */ void init_timer_interrupt(void) { @@ -306,9 +314,7 @@ void init_timer_interrupt(void) WRITE_SYSREG64(0, CNTVOFF_EL2); /* No VM-specific offset */ /* Do not let the VMs program the physical timer, only read the physic= al counter */ WRITE_SYSREG(CNTHCTL_EL2_EL1PCTEN, CNTHCTL_EL2); - WRITE_SYSREG(0, CNTP_CTL_EL0); /* Physical timer disabled */ - WRITE_SYSREG(0, CNTHP_CTL_EL2); /* Hypervisor's timer disabled */ - isb(); + disable_phys_hyp_timers(); =20 hyp_action->name =3D "hyptimer"; hyp_action->handler =3D htimer_interrupt; @@ -333,9 +339,7 @@ void init_timer_interrupt(void) */ static void deinit_timer_interrupt(void) { - WRITE_SYSREG(0, CNTP_CTL_EL0); /* Disable physical timer */ - WRITE_SYSREG(0, CNTHP_CTL_EL2); /* Disable hypervisor's timer */ - isb(); + disable_phys_hyp_timers(); =20 release_irq(timer_irq[TIMER_HYP_PPI], NULL); release_irq(timer_irq[TIMER_VIRT_PPI], NULL); @@ -375,6 +379,34 @@ void domain_set_time_offset(struct domain *d, int64_t = time_offset_seconds) /* XXX update guest visible wallclock time */ } =20 +#ifdef CONFIG_SYSTEM_SUSPEND + +void time_suspend(void) +{ + /* CNTV already disabled by virt_timer_save() during vcpu context swit= ch. */ + disable_phys_hyp_timers(); +} + +void time_resume(void) +{ + /* + * Raising TIMER_SOFTIRQ triggers generic timer code to reprogram the + * hypervisor timer with the correct timeout (not known here). + * + * Xen doesn't use or expose the physical timer, so it remains disabled + * across suspend/resume. + * + * The virtual timer state is restored per-vCPU on the next context sw= itch. + * + * No further action is needed to restore timekeeping after power down, + * since the system counter is unaffected. See ARM DDI 0487 L.a, D12.1= .2 + * "The system counter must be implemented in an always-on power domai= n." + */ + raise_softirq(TIMER_SOFTIRQ); +} + +#endif /* CONFIG_SYSTEM_SUSPEND */ + static int cpu_time_callback(struct notifier_block *nfb, unsigned long action, void *hcpu) --=20 2.43.0 From nobody Wed Apr 8 05:51:36 2026 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass(p=none dis=none) header.from=gmail.com ARC-Seal: i=1; a=rsa-sha256; t=1775126880; cv=none; d=zohomail.com; s=zohoarc; b=Wy4zjjjUSQs6uIWBpZKCBiokxpndY9lU2iUBu1C8ByFxUZV4KiLOzN+F8FY68/egCXP6nUa0P9mG8agIvtvQ4nJxVeD1awdbcJgsCiyzjEBMzYo4H/9hKx8ZrEle3FWUT4JaUte7jU2eRX6Dv42pB9t8BnjZWGnCXO/K9xyrP+0= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1775126880; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=C35Ftt8QbrUmubgD8i7RuPmnD1qKmnwaEWSlycGvL4g=; b=GuqU1BKgLQdULLG2kU2RhPtHX89W+7zaWY4L8o/ZNNL0uYNo6bidSX8dqviMGNa+21KTOQXYyToWoSfBNp5VSpeUUMbOf8DeAstmWHEdrUPHFE3YF8ao8sB9IgIItNer47UY+HuIqcszOX9EAUJhT2b32UXOuhcM7lEvQFb7058= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1775126880954250.52644164197602; Thu, 2 Apr 2026 03:48:00 -0700 (PDT) Received: from list by lists.xenproject.org with outflank-mailman.1271521.1559642 (Exim 4.92) (envelope-from ) id 1w8FaD-0004Ay-RV; Thu, 02 Apr 2026 10:47:29 +0000 Received: by outflank-mailman (output) from mailman id 1271521.1559642; Thu, 02 Apr 2026 10:47:29 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1w8FaD-00049E-K6; Thu, 02 Apr 2026 10:47:29 +0000 Received: by outflank-mailman (input) for mailman id 1271521; Thu, 02 Apr 2026 10:47:28 +0000 Received: from mx.expurgate.net ([195.190.135.10]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1w8FaC-0003z0-Jl for xen-devel@lists.xenproject.org; Thu, 02 Apr 2026 10:47:28 +0000 Received: from mx.expurgate.net (helo=localhost) by mx.expurgate.net with esmtp id 1w8FaC-009q68-0I for xen-devel@lists.xenproject.org; Thu, 02 Apr 2026 12:47:28 +0200 Received: from [10.42.69.7] (helo=localhost) by localhost with ESMTP (eXpurgate MTA 0.9.1) (envelope-from ) id 69ce4936-e002-0a2a0a5209dd-0a2a4507b904-28 for ; Thu, 02 Apr 2026 12:47:27 +0200 Received: from [209.85.208.45] (helo=mail-ed1-f45.google.com) by tlsNG-ef75cf.mxtls.expurgate.net with ESMTPS (eXpurgate 4.56.0) (envelope-from ) id 69ce493f-ba2d-0a2a45070019-d155d02dccda-3 for ; Thu, 02 Apr 2026 12:47:27 +0200 Received: by mail-ed1-f45.google.com with SMTP id 4fb4d7f45d1cf-66bb66db39dso1187246a12.0 for ; Thu, 02 Apr 2026 03:47:27 -0700 (PDT) Received: from EPUAKYIW02F7.. ([45.12.26.38]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-b9c3d028955sm76392366b.61.2026.04.02.03.47.26 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 02 Apr 2026 03:47:26 -0700 (PDT) X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" Authentication-Results: eu.smtp.expurgate.cloud; dkim=pass header.s=20251104 header.d=gmail.com header.i="@gmail.com" header.h="Content-Transfer-Encoding:MIME-Version:References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1775126847; x=1775731647; darn=lists.xenproject.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=C35Ftt8QbrUmubgD8i7RuPmnD1qKmnwaEWSlycGvL4g=; b=hz35lygR8MHzWP3b0MJ40QrwDo/zKjxPh3QFAgw2VXUr6o8r8OnexxoOyKpAfu2YN7 ILh50Jm7ZBOD9fsRrNYy/oHlwfQoC9dI42DyNNkWvVCqRi10TNZkw8yIq10mZ96PQUcq NbozaB2x5vpGcIiuYlEW956StTwZS7SXMi9ipEOF7/ITinfwcLbrxNHEMgInXFrjCPff YGIIwwBRhf+Kz4QfR/wMjnMS9AUXH2NwF4FQoYj2f5brfyA/F8qHw6/zdxNAmhl8eE1g MVXVyFhs5mbU1aiNq2+86vucV11Pr7stNupPBkD+j5PVJ6vKhA/WmwaFuW7LJZ5oByoC Tkug== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1775126847; x=1775731647; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=C35Ftt8QbrUmubgD8i7RuPmnD1qKmnwaEWSlycGvL4g=; b=jDcf+m3XnRP4P+Z/DHP5EHK622fDR5DWAJs7PQ3ql8gNP21b78deh5/9Bg+tFTttxQ 7jMyPA+aUUtmdObE4NXETErnpKemOwXa8DMvn6ahKxkiDI4AjmwdNILMaGAW5B2yK5rr E/1zJRJe5dhKNTk3lqsCXJ7eoPYNKqbgvkYcMGOAMrnt0dD0IjHF7BXGWP9yOTLw8yyZ 3dbWuwQgC5LHJrzCJmouYz/Mi1h7T8YaQMvLU37lnAGlu6j3UaztLYJkCtA9NymuTpMR 8oSRsmRoIkhoeibvLNprxaVtwA41U90OLzaNVqWjXD0bPy1teU/JKHemPp6R7YEcr+2c b3rw== X-Gm-Message-State: AOJu0Yyn2zBftPTsPqGSgZizwHqs8HPjW0Kv2MwmiemTAFKaThetmfuk r4dNZgfZRvalaEhqU0Jb1fzz3J8lHOJBhC3xqp7jvMItqdIyzjGwsplrBwpV8boG X-Gm-Gg: ATEYQzziXu5ZR0leLvmKpAClf/UquHRzyNgfSBfgvYbG0w+AQNP9tg9UfYF7XY0aK6e ikdLwIYo9QBjD2h9Yj0V4Mkql/Le+yflnLtc79pyDALA+EdluqAWu5abUtCqIM1RoIvPbvi3cSK rmNBgeiN3gMOEAQZibQqGdfldfCNFH0yLvjX82Q5CH1YwKaz08U+e6+dJ2RYttxwBzN2Y4CBdbs JAZ3SON0ETksgg+j7b8Q8zuJ50m0XWroQ1w1Wnfcd1WAnMnhGaiV0BG1qGwqxY2zmLQjr9YDm9P K5tiZoLIApDLHcCGu8tfcBig2BSKAfLtD4UKeJV6qH9UGuOr2xIzh+fEfafqs3SLOFF7KVQIwej PaC3z0j9AvqXyL55eg9++Jsdo6Og8ugAJ/YLrMiqOZHwsGbJ7+Yvhl9Mh9l7vZb6XJx+FyXazkC DSiLyoWvOjA0mww0lKWWhWNkZXiQ== X-Received: by 2002:a17:906:9582:b0:b97:464:956e with SMTP id a640c23a62f3a-b9c13b41d82mr319751366b.34.1775126846740; Thu, 02 Apr 2026 03:47:26 -0700 (PDT) From: Mykola Kvach To: xen-devel@lists.xenproject.org Cc: Mykola Kvach , Stefano Stabellini , Julien Grall , Bertrand Marquis , Michal Orzel , Volodymyr Babchuk Subject: [PATCH v8 02/13] xen/arm: gic-v2: Implement GIC suspend/resume functions Date: Thu, 2 Apr 2026 13:45:03 +0300 Message-ID: <5e1951e1c17037e020de9c0eecb5fba94edcd9dd.1775125380.git.mykola_kvach@epam.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: References: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-purgate-ID: tlsNG-ef75cf/1775126847-1839741E-EDA16485/0/0 X-purgate-type: clean X-purgate-size: 8460 X-ZohoMail-DKIM: pass (identity @gmail.com) X-ZM-MESSAGEID: 1775126882760154100 Content-Type: text/plain; charset="utf-8" From: Mirela Simonovic System suspend may lead to a state where GIC would be powered down. Therefore, Xen should save/restore the context of GIC on suspend/resume. Note that the context consists of states of registers which are controlled by the hypervisor. Other GIC registers which are accessible by guests are saved/restored on context switch. Signed-off-by: Mirela Simonovic Signed-off-by: Saeed Nowshadi Signed-off-by: Mykyta Poturai Signed-off-by: Mykola Kvach --- Changes in V8: - disable cpu interface + distributor before suspend - change 0xffffffff to GENMASK; - cosmetic changes; Changes in V7: - Allocate one contiguous memory block for the GICv2 dist suspend context. - gicv2_resume() no longer unconditionally re-enables the distributor/CPU i= nterface; it now writes back the saved CTLR values as-is. - gicv2_alloc_context() now returns 0 on success and panics on failure, sin= ce suspend context allocation is not recoverable. --- xen/arch/arm/gic-v2.c | 132 +++++++++++++++++++++++++++++++++ xen/arch/arm/gic.c | 29 ++++++++ xen/arch/arm/include/asm/gic.h | 12 +++ 3 files changed, 173 insertions(+) diff --git a/xen/arch/arm/gic-v2.c b/xen/arch/arm/gic-v2.c index b23e72a3d0..dbff470962 100644 --- a/xen/arch/arm/gic-v2.c +++ b/xen/arch/arm/gic-v2.c @@ -1098,6 +1098,129 @@ static int gicv2_iomem_deny_access(struct domain *d) return iomem_deny_access(d, mfn, mfn + nr); } =20 +#ifdef CONFIG_SYSTEM_SUSPEND + +/* This struct represents block of 32 IRQs */ +struct irq_block { + uint32_t icfgr[2]; /* 2 registers of 16 IRQs each */ + uint32_t ipriorityr[8]; + uint32_t isenabler; + uint32_t isactiver; + uint32_t itargetsr[8]; +}; + +/* GICv2 registers to be saved/restored on system suspend/resume */ +struct gicv2_context { + /* GICC context */ + struct cpu_ctx { + uint32_t ctlr; + uint32_t pmr; + uint32_t bpr; + } cpu; + + /* GICD context */ + struct dist_ctx { + uint32_t ctlr; + /* Includes banked SGI/PPI state for the boot CPU. */ + struct irq_block *irqs; + } dist; +}; + +static struct gicv2_context gic_ctx; + +static int gicv2_suspend(void) +{ + unsigned int i, blocks =3D DIV_ROUND_UP(gicv2_info.nr_lines, 32); + + /* Save GICC_CTLR configuration. */ + gic_ctx.cpu.ctlr =3D readl_gicc(GICC_CTLR); + + /* Quiesce the GIC CPU interface before suspend. */ + gicv2_cpu_disable(); + + /* Save GICD configuration */ + gic_ctx.dist.ctlr =3D readl_gicd(GICD_CTLR); + writel_gicd(0, GICD_CTLR); + + gic_ctx.cpu.pmr =3D readl_gicc(GICC_PMR); + gic_ctx.cpu.bpr =3D readl_gicc(GICC_BPR); + + for ( i =3D 0; i < blocks; i++ ) + { + struct irq_block *irqs =3D gic_ctx.dist.irqs + i; + size_t j, off =3D i * sizeof(irqs->isenabler); + + irqs->isenabler =3D readl_gicd(GICD_ISENABLER + off); + irqs->isactiver =3D readl_gicd(GICD_ISACTIVER + off); + + off =3D i * sizeof(irqs->ipriorityr); + for ( j =3D 0; j < ARRAY_SIZE(irqs->ipriorityr); j++ ) + { + irqs->ipriorityr[j] =3D readl_gicd(GICD_IPRIORITYR + off + j *= 4); + irqs->itargetsr[j] =3D readl_gicd(GICD_ITARGETSR + off + j * 4= ); + } + + off =3D i * sizeof(irqs->icfgr); + for ( j =3D 0; j < ARRAY_SIZE(irqs->icfgr); j++ ) + irqs->icfgr[j] =3D readl_gicd(GICD_ICFGR + off + j * 4); + } + + return 0; +} + +static void gicv2_resume(void) +{ + unsigned int i, blocks =3D DIV_ROUND_UP(gicv2_info.nr_lines, 32); + + gicv2_cpu_disable(); + /* Disable distributor */ + writel_gicd(0, GICD_CTLR); + + for ( i =3D 0; i < blocks; i++ ) + { + struct irq_block *irqs =3D gic_ctx.dist.irqs + i; + size_t j, off =3D i * sizeof(irqs->isenabler); + + writel_gicd(GENMASK(31, 0), GICD_ICENABLER + off); + writel_gicd(irqs->isenabler, GICD_ISENABLER + off); + + writel_gicd(GENMASK(31, 0), GICD_ICACTIVER + off); + writel_gicd(irqs->isactiver, GICD_ISACTIVER + off); + + off =3D i * sizeof(irqs->ipriorityr); + for ( j =3D 0; j < ARRAY_SIZE(irqs->ipriorityr); j++ ) + { + writel_gicd(irqs->ipriorityr[j], GICD_IPRIORITYR + off + j * 4= ); + writel_gicd(irqs->itargetsr[j], GICD_ITARGETSR + off + j * 4); + } + + off =3D i * sizeof(irqs->icfgr); + for ( j =3D 0; j < ARRAY_SIZE(irqs->icfgr); j++ ) + writel_gicd(irqs->icfgr[j], GICD_ICFGR + off + j * 4); + } + + /* Make sure all registers are restored and enable distributor */ + writel_gicd(gic_ctx.dist.ctlr, GICD_CTLR); + + /* Restore GIC CPU interface configuration */ + writel_gicc(gic_ctx.cpu.pmr, GICC_PMR); + writel_gicc(gic_ctx.cpu.bpr, GICC_BPR); + + /* Enable GIC CPU interface */ + writel_gicc(gic_ctx.cpu.ctlr, GICC_CTLR); +} + +static void __init gicv2_alloc_context(void) +{ + uint32_t blocks =3D DIV_ROUND_UP(gicv2_info.nr_lines, 32); + + gic_ctx.dist.irqs =3D xzalloc_array(struct irq_block, blocks); + if ( !gic_ctx.dist.irqs ) + panic("Failed to allocate memory for GICv2 suspend context\n"); +} + +#endif /* CONFIG_SYSTEM_SUSPEND */ + #ifdef CONFIG_ACPI static unsigned long gicv2_get_hwdom_extra_madt_size(const struct domain *= d) { @@ -1302,6 +1425,11 @@ static int __init gicv2_init(void) =20 spin_unlock(&gicv2.lock); =20 +#ifdef CONFIG_SYSTEM_SUSPEND + /* Allocate memory to be used for saving GIC context during the suspen= d */ + gicv2_alloc_context(); +#endif /* CONFIG_SYSTEM_SUSPEND */ + return 0; } =20 @@ -1345,6 +1473,10 @@ static const struct gic_hw_operations gicv2_ops =3D { .map_hwdom_extra_mappings =3D gicv2_map_hwdom_extra_mappings, .iomem_deny_access =3D gicv2_iomem_deny_access, .do_LPI =3D gicv2_do_LPI, +#ifdef CONFIG_SYSTEM_SUSPEND + .suspend =3D gicv2_suspend, + .resume =3D gicv2_resume, +#endif /* CONFIG_SYSTEM_SUSPEND */ }; =20 /* Set up the GIC */ diff --git a/xen/arch/arm/gic.c b/xen/arch/arm/gic.c index ee75258fc3..7727ffed5a 100644 --- a/xen/arch/arm/gic.c +++ b/xen/arch/arm/gic.c @@ -432,6 +432,35 @@ int gic_iomem_deny_access(struct domain *d) return gic_hw_ops->iomem_deny_access(d); } =20 +#ifdef CONFIG_SYSTEM_SUSPEND + +int gic_suspend(void) +{ + /* Must be called by boot CPU#0 with interrupts disabled */ + ASSERT(!local_irq_is_enabled()); + ASSERT(!smp_processor_id()); + + if ( !gic_hw_ops->suspend || !gic_hw_ops->resume ) + return -ENOSYS; + + return gic_hw_ops->suspend(); +} + +void gic_resume(void) +{ + /* + * Must be called by boot CPU#0 with interrupts disabled after gic_sus= pend + * has returned successfully. + */ + ASSERT(!local_irq_is_enabled()); + ASSERT(!smp_processor_id()); + ASSERT(gic_hw_ops->resume); + + gic_hw_ops->resume(); +} + +#endif /* CONFIG_SYSTEM_SUSPEND */ + static int cpu_gic_callback(struct notifier_block *nfb, unsigned long action, void *hcpu) diff --git a/xen/arch/arm/include/asm/gic.h b/xen/arch/arm/include/asm/gic.h index 8e713aa477..8e8f4ac4c5 100644 --- a/xen/arch/arm/include/asm/gic.h +++ b/xen/arch/arm/include/asm/gic.h @@ -280,6 +280,12 @@ extern int gicv_setup(struct domain *d); extern void gic_save_state(struct vcpu *v); extern void gic_restore_state(struct vcpu *v); =20 +#ifdef CONFIG_SYSTEM_SUSPEND +/* Suspend/resume */ +extern int gic_suspend(void); +extern void gic_resume(void); +#endif /* CONFIG_SYSTEM_SUSPEND */ + /* SGI (AKA IPIs) */ enum gic_sgi { GIC_SGI_EVENT_CHECK, @@ -423,6 +429,12 @@ struct gic_hw_operations { int (*iomem_deny_access)(struct domain *d); /* Handle LPIs, which require special handling */ void (*do_LPI)(unsigned int lpi); +#ifdef CONFIG_SYSTEM_SUSPEND + /* Save GIC configuration due to the system suspend */ + int (*suspend)(void); + /* Restore GIC configuration due to the system resume */ + void (*resume)(void); +#endif /* CONFIG_SYSTEM_SUSPEND */ }; =20 extern const struct gic_hw_operations *gic_hw_ops; --=20 2.43.0 From nobody Wed Apr 8 05:51:36 2026 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass(p=none dis=none) header.from=gmail.com ARC-Seal: i=1; a=rsa-sha256; t=1775126891; cv=none; d=zohomail.com; s=zohoarc; b=C8iIV/L/F31dK9HDSFpLOaAx1zmKKLUViMXx0kfrvhAvS+dKk7FA/yRqcGWz+mQVb48gGzb4bFXdUku3bfM8Sv47WD+Q8DlSQNWzJ7LNMH04Nj5bOpwkFdLGYC+UuRNv1JaK04K5jsplwNt2M2EjV7UcJx1lbnHothVobz3E6CU= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1775126891; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=OrY9cE1nd2J0of99KnQlF14iwjljRp7bNvn2AnZysGw=; b=KfULdtWXHdu3YPvF6ruBhr6tchH1cprVgaE3q/Gk924eTASx+3UTkTW0W+c7Q4knANAy5ZAnzx8n1RDDDsBapReZ+lBBHiUhaWsutOIZge/xZgU8x6GKkkxf4FwG/lyAjZZ69ebJt3Akjk7iyOnMrdVCt/F7D3aJMUULPoDOuu4= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1775126891621828.522707954174; Thu, 2 Apr 2026 03:48:11 -0700 (PDT) Received: from list by lists.xenproject.org with outflank-mailman.1271522.1559656 (Exim 4.92) (envelope-from ) id 1w8FaG-0004d6-04; Thu, 02 Apr 2026 10:47:32 +0000 Received: by outflank-mailman (output) from mailman id 1271522.1559656; Thu, 02 Apr 2026 10:47:31 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1w8FaF-0004cw-Sq; Thu, 02 Apr 2026 10:47:31 +0000 Received: by outflank-mailman (input) for mailman id 1271522; Thu, 02 Apr 2026 10:47:30 +0000 Received: from mx.expurgate.net ([195.190.135.10]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1w8FaD-0004Am-SA for xen-devel@lists.xenproject.org; Thu, 02 Apr 2026 10:47:29 +0000 Received: from mx.expurgate.net (helo=localhost) by mx.expurgate.net with esmtp id 1w8FaD-004haJ-8O for xen-devel@lists.xenproject.org; Thu, 02 Apr 2026 12:47:29 +0200 Received: from [10.42.69.6] (helo=localhost) by localhost with ESMTP (eXpurgate MTA 0.9.1) (envelope-from ) id 69ce4932-5cb7-0a2a0a5109dd-0a2a4506ed0e-16 for ; Thu, 02 Apr 2026 12:47:29 +0200 Received: from [209.85.218.50] (helo=mail-ej1-f50.google.com) by tlsNG-16d1c6.mxtls.expurgate.net with ESMTPS (eXpurgate 4.56.0) (envelope-from ) id 69ce4941-0df0-0a2a45060019-d155da32a5de-3 for ; Thu, 02 Apr 2026 12:47:29 +0200 Received: by mail-ej1-f50.google.com with SMTP id a640c23a62f3a-b9a0762ed5fso112563066b.1 for ; Thu, 02 Apr 2026 03:47:29 -0700 (PDT) Received: from EPUAKYIW02F7.. ([45.12.26.38]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-b9c3d028955sm76392366b.61.2026.04.02.03.47.26 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 02 Apr 2026 03:47:27 -0700 (PDT) X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" Authentication-Results: eu.smtp.expurgate.cloud; dkim=pass header.s=20251104 header.d=gmail.com header.i="@gmail.com" header.h="Content-Transfer-Encoding:MIME-Version:References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1775126848; x=1775731648; darn=lists.xenproject.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=OrY9cE1nd2J0of99KnQlF14iwjljRp7bNvn2AnZysGw=; b=L3PM+kaAyXPWdp2b4PzoPDl1+HMP/cbwZ0q3PZmcNcfboIlOZXjhh5gHQcyE7P5Ag3 Ssom/tpJUMz+g79mk12p+p11Vx/yzDQ/PZ79X8yI6ZYgzYzNGePB9wzf4pab6TSS1yEs pPN1+4yiBG+3FXDmRB+s3A5QsfDYVxv2DY5ZAMDrpzmpTP+3kp0V5CGGnBkviq3FtqFS EwNVfdnmKE8a6xZj10YIc79NCAaBfXQ/pL6BLxKmZuBe3hk7WcUaHDazhGIsTXcEKUFv w7GkjCxcuLir3BZ014pHQxCT6vVKVze2dUAoxhqK7Rb0Sb7f5mkIdRW/G8EtReLncKEM bphA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1775126848; x=1775731648; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=OrY9cE1nd2J0of99KnQlF14iwjljRp7bNvn2AnZysGw=; b=SJfaEwqtSzLmI4/MlAqA6wGSMA/USHAIzzztFcN1HXmQS58ypHK/we3qYtBOe7w+ut zXlk2zN56NiEvvc47Y5BZzYaAYzSegsRAkXlCMfmbYnqaqNDuJr+jDQRrPLfFsRmdT8M ixJy+xIlBX4kwYPna4l630sNP+gJikgaUqYzdX/RRHOmNpwdISYUL/4hGHgUE1paWlZf 3lyWFsGdUtk0ueKAIHqg/fqfOgIADkohD+/iblD/HL2M4un6KSdhIMmpIguXg/wbxmTW HKcxky7CUnhlTowi8xG0dWAO0Ta9WedlAWIQRwJTK+InccRIJrJHUzl60d8zz2phzqiA SSlg== X-Gm-Message-State: AOJu0YxrIKkKAHslIGajy3KX+gzhP7PUhodCPQS48xLpGl8C3BQL5lXM eALKluwg4M4sLXVRCFHLIL7ucUgSbNf+YLwGz3xjNqfXulJTnG172kpPt7pprG8a X-Gm-Gg: ATEYQzyeM1ISVBxTfqhC00SAKkeD1igixeMUPiyIikwYN10H0n4Z3aas8+xqG1/h9/G OHa/mEMjzeLnfMAvM27lbuLtnhacxEvwrpl+MgSrkMTiBJv1DnSm5wUcnpvgFQI4+IN1NvBegSO EgKAuORgmBidV0fJgrE7lPm0C3SEBuq4d+vxhflmrTSvYkYqjwFl/9SrmzJ1h9DBBJpFM0W+XGk Wfvj7VudvIUYOmjUi4UQt5/qYmTiw8a/b5bEW9puYX1Q23A/4KAA+7ikg8dPUtiAu4EHEwYiG5u CYpbp/KCakXscDaHmPuXM6/NVymz70/Kt31kvt6chAKvQvqLI8MikT79v01Egknr+NkFZLrn9bf XDi18Kc1z6p0sIuuGbqbb0xBCPHj6HNZi9XZWyeLy0Mi7o0jQLRWRWSqGOKSS/jdMrICrce+SXA myS+7D9WRfcKvrPs6l4xwNRzvXKg== X-Received: by 2002:a17:907:3d56:b0:b94:231f:26ca with SMTP id a640c23a62f3a-b9c46fedc37mr91352666b.20.1775126847635; Thu, 02 Apr 2026 03:47:27 -0700 (PDT) From: Mykola Kvach To: xen-devel@lists.xenproject.org Cc: Mykola Kvach , Stefano Stabellini , Julien Grall , Bertrand Marquis , Michal Orzel , Volodymyr Babchuk Subject: [PATCH v8 03/13] xen/arm: gic-v3: tolerate retained redistributor LPI state across CPU_OFF Date: Thu, 2 Apr 2026 13:45:04 +0300 Message-ID: X-Mailer: git-send-email 2.43.0 In-Reply-To: References: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-purgate-ID: tlsNG-16d1c6/1775126849-5E5303D8-0CBBBD7E/0/0 X-purgate-type: clean X-purgate-size: 8639 X-ZohoMail-DKIM: pass (identity @gmail.com) X-ZM-MESSAGEID: 1775126893817158500 Content-Type: text/plain; charset="utf-8" From: Mykola Kvach PSCI does not guarantee that a GICv3 redistributor is powered down across CPU_OFF -> CPU_ON. DEN0022F.b says CPU_OFF powers down the calling core (5.5) and CPU_ON brings the core back with a defined initial CPU state (5.6, 6.4). However, PSCI leaves interrupt migration and GIC re-initialization to the supervisory software/firmware stack: the caller must migrate interrupts away before CPU_OFF (5.5.2), and the execution context that is lost in a powerdown state must be saved and restored by software (6.8). PSCI also calls out GIC management explicitly in 6.8, including retargeting SPIs, preventing PPIs/SGIs from targeting a powered down CPU, and reinitializing the CPU interface after CPU_ON. This matches the GIC architecture. IHI0069H.b Chapter 11.1 requires the PE and CPU interface to share a power domain, but explicitly allows the associated redistributor, distributor, and ITS to remain powered while the PE and CPU interface are off. All other GIC power-management behavior is IMPLEMENTATION DEFINED. DEN0050D Chapter 4.2, "Generic Interrupt Controller (GIC)", says the GICv3 redistributor may live either in the AP core power domain or in a relatively always-on parent domain. So after CPU_OFF -> CPU_ON a secondary CPU can legitimately come back to a live redistributor with GICR_CTLR.EnableLPIs still set. Handle that case in the LPI setup path instead of assuming a fully reset redistributor. The LPI path needs special care because the GIC spec makes redistributor LPI state sticky and partially implementation defined. IHI0069H.b 5.1.1 and 5.1.2 say that changing GICR_PROPBASER or GICR_PENDBASER while GICR_CTLR.EnableLPIs =3D=3D 1 is UNPREDICTABLE. After clearing EnableLPIs, software must wait for GICR_CTLR.RWP =3D=3D 0 before touching the pending table. The architecture also permits implementations where, once EnableLPIs has been set, clearing it again is not guaranteed to work. Where an ITS is present, the spec strongly recommends moving LPIs to another redistributor before clearing EnableLPIs. Because of that, treat a retained EnableLPIs state as valid when the redistributor still points at Xen's expected PROPBASER/PENDBASER tables. Only try to clear EnableLPIs when the retained configuration does not match Xen's state, and wait for RWP before reprogramming the tables. This is also consistent with platform firmware reality: PSCI and the GIC architecture allow platform-specific redistributor power handling, and not all TF-A platforms force a full redistributor power-off through implementation-defined controls during CPU_OFF. Xen therefore needs to tolerate retained redistributor state on secondary CPU bring-up. Tested using Xen's non-boot CPU disable/enable path on Arm FVP_Base_RevC-2xAEMvA, both with and without: -C gic_distributor.allow-LPIEN-clear=3D1 -C gic_distributor.GICR-clear-enable-supported=3D1 and on Orange Pi 5. Signed-off-by: Mykola Kvach --- xen/arch/arm/gic-v3-lpi.c | 77 ++++++++++++++++++++++++++- xen/arch/arm/gic-v3.c | 15 ++++-- xen/arch/arm/include/asm/gic_v3_its.h | 1 + 3 files changed, 87 insertions(+), 6 deletions(-) diff --git a/xen/arch/arm/gic-v3-lpi.c b/xen/arch/arm/gic-v3-lpi.c index de5052e5cf..125f51e61b 100644 --- a/xen/arch/arm/gic-v3-lpi.c +++ b/xen/arch/arm/gic-v3-lpi.c @@ -81,6 +81,13 @@ static DEFINE_PER_CPU(struct lpi_redist_data, lpi_redist= ); #define MAX_NR_HOST_LPIS (lpi_data.max_host_lpi_ids - LPI_OFFSET) #define HOST_LPIS_PER_PAGE (PAGE_SIZE / sizeof(union host_lpi)) =20 +#define GICR_PROPBASER_XEN_MASK GENMASK_ULL(51, 12) +/* + * For retained redistributor state, match the pending table by address on= ly. + * Attribute bits such as PTZ may not read back with the programmed value. + */ +#define GICR_PENDBASER_XEN_MASK GENMASK_ULL(51, 16) + static union host_lpi *gic_get_host_lpi(uint32_t plpi) { union host_lpi *block; @@ -296,6 +303,60 @@ static int gicv3_lpi_set_pendtable(void __iomem *rdist= _base) return 0; } =20 +static uint64_t gicv3_lpi_expected_proptable(void) +{ + return virt_to_maddr(lpi_data.lpi_property); +} + +static uint64_t gicv3_lpi_expected_pendtable(void) +{ + return virt_to_maddr(this_cpu(lpi_redist).pending_table); +} + +static bool gicv3_lpi_tables_match(void __iomem *rdist_base) +{ + uint64_t propbase, pendbase; + + if ( !lpi_data.lpi_property || !this_cpu(lpi_redist).pending_table ) + return false; + + propbase =3D readq_relaxed(rdist_base + GICR_PROPBASER); + pendbase =3D readq_relaxed(rdist_base + GICR_PENDBASER); + + return ((propbase & GICR_PROPBASER_XEN_MASK) =3D=3D + (gicv3_lpi_expected_proptable() & GICR_PROPBASER_XEN_MASK)) && + ((pendbase & GICR_PENDBASER_XEN_MASK) =3D=3D + (gicv3_lpi_expected_pendtable() & GICR_PENDBASER_XEN_MASK)); +} + +static int gicv3_lpi_disable_lpis(void __iomem *rdist_base) +{ + uint32_t reg =3D readl_relaxed(rdist_base + GICR_CTLR); + int ret; + + if ( !(reg & GICR_CTLR_ENABLE_LPIS) ) + return 0; + + writel_relaxed(reg & ~GICR_CTLR_ENABLE_LPIS, rdist_base + GICR_CTLR); + + /* + * The spec only guarantees programmability when we have observed the = bit + * cleared. Where clearing is supported, RWP must reach 0 before touch= ing + * PROPBASER/PENDBASER again. + */ + wmb(); + + ret =3D gicv3_do_wait_for_rwp(rdist_base); + if ( ret ) + return ret; + + reg =3D readl_relaxed(rdist_base + GICR_CTLR); + if ( reg & GICR_CTLR_ENABLE_LPIS ) + return -EBUSY; + + return 0; +} + /* * Tell a redistributor about the (shared) property table, allocating one * if not already done. @@ -373,7 +434,21 @@ int gicv3_lpi_init_rdist(void __iomem * rdist_base) /* Make sure LPIs are disabled before setting up the tables. */ reg =3D readl_relaxed(rdist_base + GICR_CTLR); if ( reg & GICR_CTLR_ENABLE_LPIS ) - return -EBUSY; + { + if ( gicv3_lpi_tables_match(rdist_base) ) + return -EBUSY; + + ret =3D gicv3_lpi_disable_lpis(rdist_base); + if ( ret =3D=3D -EBUSY ) + { + printk(XENLOG_ERR + "GICv3: CPU%d: LPIs still enabled with unexpected redis= tributor tables\n", + smp_processor_id()); + return -EINVAL; + } + if ( ret ) + return ret; + } =20 ret =3D gicv3_lpi_set_pendtable(rdist_base); if ( ret ) diff --git a/xen/arch/arm/gic-v3.c b/xen/arch/arm/gic-v3.c index bc07f97c16..34fb065afc 100644 --- a/xen/arch/arm/gic-v3.c +++ b/xen/arch/arm/gic-v3.c @@ -274,8 +274,8 @@ static void gicv3_enable_sre(void) isb(); } =20 -/* Wait for completion of a distributor change */ -static void gicv3_do_wait_for_rwp(void __iomem *base) +/* Wait for completion of a distributor/redistributor write-pending change= . */ +int gicv3_do_wait_for_rwp(void __iomem *base) { uint32_t val; bool timeout =3D false; @@ -295,17 +295,22 @@ static void gicv3_do_wait_for_rwp(void __iomem *base) } while ( 1 ); =20 if ( timeout ) + { dprintk(XENLOG_ERR, "RWP timeout\n"); + return -ETIMEDOUT; + } + + return 0; } =20 static void gicv3_dist_wait_for_rwp(void) { - gicv3_do_wait_for_rwp(GICD); + (void)gicv3_do_wait_for_rwp(GICD); } =20 static void gicv3_redist_wait_for_rwp(void) { - gicv3_do_wait_for_rwp(GICD_RDIST_BASE); + (void)gicv3_do_wait_for_rwp(GICD_RDIST_BASE); } =20 static void gicv3_wait_for_rwp(int irq) @@ -925,7 +930,7 @@ static int __init gicv3_populate_rdist(void) gicv3_set_redist_address(rdist_addr, procnum); =20 ret =3D gicv3_lpi_init_rdist(ptr); - if ( ret && ret !=3D -ENODEV ) + if ( ret && ret !=3D -ENODEV && ret !=3D -EBUSY ) { printk("GICv3: CPU%d: Cannot initialize LPIs: %u\n= ", smp_processor_id(), ret); diff --git a/xen/arch/arm/include/asm/gic_v3_its.h b/xen/arch/arm/include/a= sm/gic_v3_its.h index fc5a84892c..081bd19180 100644 --- a/xen/arch/arm/include/asm/gic_v3_its.h +++ b/xen/arch/arm/include/asm/gic_v3_its.h @@ -133,6 +133,7 @@ struct host_its { =20 /* Map a collection for this host CPU to each host ITS. */ int gicv3_its_setup_collection(unsigned int cpu); +int gicv3_do_wait_for_rwp(void __iomem *base); =20 #ifdef CONFIG_HAS_ITS =20 --=20 2.43.0 From nobody Wed Apr 8 05:51:36 2026 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass(p=none dis=none) header.from=gmail.com ARC-Seal: i=1; a=rsa-sha256; t=1775126884; cv=none; d=zohomail.com; s=zohoarc; b=SZjtwI6DYsgom7keywnsVWybyr2deNBTluURGGYI20qJzB63+YUouz8CU4Arrdhb7wzYMDMCEjviyMd1SnhsVNwE7uvFalBqVqdLon9hggMSZLtgpaCrkHlcAMr7CGjce114Od2ixJG6L2CB2+inw33MKaPj7Hq4M4uSlA4P/Ak= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1775126884; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=NDTHUo0iRavupmsXxpHOoWQVA9mce1asVaUpgFiaZZQ=; b=FXc8FfkOwfxEfjVo5AVmv28pvLLRyXHDCbtKMMnxvEHip3odHR9nMLga0JI91ix86+fQc3qbfTMA2vprQPSNN2ud2uRP92T9Ifi+q5OZT7ozuRlVvFiQEzVmBUnF/JhG6dpHJIsnwyhD0Kiw+qatEZAZYpErjUCIFEpXCqoVsnM= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1775126884359613.1982381220697; Thu, 2 Apr 2026 03:48:04 -0700 (PDT) Received: from list by lists.xenproject.org with outflank-mailman.1271523.1559661 (Exim 4.92) (envelope-from ) id 1w8FaG-0004fZ-9S; Thu, 02 Apr 2026 10:47:32 +0000 Received: by outflank-mailman (output) from mailman id 1271523.1559661; Thu, 02 Apr 2026 10:47:32 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1w8FaG-0004e6-2Y; Thu, 02 Apr 2026 10:47:32 +0000 Received: by outflank-mailman (input) for mailman id 1271523; Thu, 02 Apr 2026 10:47:30 +0000 Received: from mx.expurgate.net ([195.190.135.10]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1w8FaE-0004Wm-Mn for xen-devel@lists.xenproject.org; Thu, 02 Apr 2026 10:47:30 +0000 Received: from mx.expurgate.net (helo=localhost) by mx.expurgate.net with esmtp id 1w8FaE-00Ej0u-35 for xen-devel@lists.xenproject.org; Thu, 02 Apr 2026 12:47:30 +0200 Received: from [10.42.69.8] (helo=localhost) by localhost with ESMTP (eXpurgate MTA 0.9.1) (envelope-from ) id 69ce4935-2eae-0a2a0a5409dd-0a2a4508a782-8 for ; Thu, 02 Apr 2026 12:47:30 +0200 Received: from [209.85.218.53] (helo=mail-ej1-f53.google.com) by tlsNG-c1860d.mxtls.expurgate.net with ESMTPS (eXpurgate 4.56.0) (envelope-from ) id 69ce4941-fab6-0a2a45080019-d155da35c9f8-3 for ; Thu, 02 Apr 2026 12:47:30 +0200 Received: by mail-ej1-f53.google.com with SMTP id a640c23a62f3a-b9c3a9fe80fso90505966b.3 for ; Thu, 02 Apr 2026 03:47:29 -0700 (PDT) Received: from EPUAKYIW02F7.. ([45.12.26.38]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-b9c3d028955sm76392366b.61.2026.04.02.03.47.27 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 02 Apr 2026 03:47:28 -0700 (PDT) X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" Authentication-Results: eu.smtp.expurgate.cloud; dkim=pass header.s=20251104 header.d=gmail.com header.i="@gmail.com" header.h="Content-Transfer-Encoding:MIME-Version:References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1775126849; x=1775731649; darn=lists.xenproject.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=NDTHUo0iRavupmsXxpHOoWQVA9mce1asVaUpgFiaZZQ=; b=b+ZECLIghcIgTlcgPUDvEZaEmp7ruge7IOSO7wchuTMmQKKmmoyM2zEDQt38PVqmi3 pfCUXXaqSSXYmFdPPltiuxLETvrAw71fDG7kV01HgHKuUQJUyN+YbFIVHicmRx9Xsklu Ac17lKFolu7/vjOeteXykKm9M+5linLu91nZVampW6Q8rMh32WBvVKY5IlBX6A+H1cB4 bPXZbVA1Z2yuNXpiRg4u4eI6+60z9CQdOISEwyr9k8KZaAVpDiYNHMKoJ8zDxzSqNYBv CavfFoZZOa81yYRvLIguyR5QPTWA5vqLxxO/Jv9j1mFIqWMHbj+cpxjt70GyuPYWTCO/ ujQg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1775126849; x=1775731649; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=NDTHUo0iRavupmsXxpHOoWQVA9mce1asVaUpgFiaZZQ=; b=q+Zv3eLgDusNY097ZmspCbHQ2mfoDqhHI2KRU+6qrCavDrY9aPKuOUCETumxLV1OIA u1Fi0M0MK6U0PTdOYRNornFWQags3OHS2poyo6JRaGYM6KSoDMAVjOVS4kk4/izMMJmB CJoLFZk1EOeSlJW5mIuo62xb8EeunK/vtNpgePdd62WfGcvx3hxiOZGEzCsPRBTNNkrW AsRK2hFrB9LhePH6EU+8NgbKiDBU42ocbAYfsslWD3OxQqz9M4oPM3Wm1zG5QzTB2DOM H8XcfKvBtOQDAAsbTtzrSwD1M+1PLrfWZi36PbjSKz7Zt/ui5b4FM6q1OaTGMca85/Yp 7WLw== X-Gm-Message-State: AOJu0YxoPLhabGP8oG1o8l1wQKUsww1pgLHx1xIMreINKy52JjkD0ZYp pVBkqzRMOpapKEah+b/tZ4qRanGZh+Sw01R4TSdbUqGOAp4f4e5rfkiGuFEaKP+x X-Gm-Gg: ATEYQzym8GXRk8UFptAL8aH1rUBEMVRaHHRWs1L2gCtB19F+2lZd5WSvkOabNx6K0+B 9Uz366xzxgS8SyeU3XxWKIq/rNri4S5X7nfv5buk9d+44/KxeCIvUoBFZE2QCIKtoh/CXB/JoI/ iKs4ns266TPgA0xPxctBhfUcVZFK/uyHNciXNjyxTDt31/aywZvOFeC0Wz+4FXXctqiR6Ff8N6n boWkIeqoIG32uWgqR57Dr5fk5mIKQnULBE8d/cjam2+KB/+aXORNbMitWRW6MKtbVc38IjdoToi i3iKPL8WJBPRP1EXWcubbnNTrI2NtZ7sJ135t49wWekA75bA+SHDAz+voeB5dF3PR0UAU4mxVWq k99+abtRCbInVBp8gYOw1H/uE2xAo+eoISicoDDkaNZ6DzXZthSkA/HKIKZ8l9mjB6UbPeyGLUw RsgD4Z+5lJCjJZ30YraWOl1l8d2Q== X-Received: by 2002:a17:907:c789:b0:b9c:ed9:50c8 with SMTP id a640c23a62f3a-b9c13b2c4e2mr463323366b.26.1775126848495; Thu, 02 Apr 2026 03:47:28 -0700 (PDT) From: Mykola Kvach To: xen-devel@lists.xenproject.org Cc: Mykola Kvach , Stefano Stabellini , Julien Grall , Bertrand Marquis , Michal Orzel , Volodymyr Babchuk Subject: [PATCH v8 04/13] xen/arm: gic-v3: Implement GICv3 suspend/resume functions Date: Thu, 2 Apr 2026 13:45:05 +0300 Message-ID: X-Mailer: git-send-email 2.43.0 In-Reply-To: References: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-purgate-ID: tlsNG-c1860d/1775126850-7495D497-CD7F2EA0/0/0 X-purgate-type: clean X-purgate-size: 14084 X-ZohoMail-DKIM: pass (identity @gmail.com) X-ZM-MESSAGEID: 1775126886709154100 Content-Type: text/plain; charset="utf-8" From: Mykola Kvach System suspend may lead to a state where GIC would be powered down. Therefore, Xen should save/restore the context of GIC on suspend/resume. Note that the context consists of states of registers which are controlled by the hypervisor. Other GIC registers which are accessible by guests are saved/restored on context switch. Signed-off-by: Mykola Kvach --- Changes in V8: - use right rdist base for prop/pend baser and ctrl Changes in V7: - restore LPI regs on resume - add timeout during redist disabling - squash with suspend/resume handling for GICv3 eSPI registers - drop ITS guard paths so suspend/resume always runs; switch missing ctx allocation to panic - trim TODO comments; narrow redistributor storage to PPI icfgr - keep distributor context allocation even without ITS; adjust resume to use GENMASK(31, 0) for clearing enables - drop storage of the SGI configuration register, as SGIs are always edge-triggered --- xen/arch/arm/gic-v3-lpi.c | 3 + xen/arch/arm/gic-v3.c | 321 ++++++++++++++++++++++++- xen/arch/arm/include/asm/gic_v3_defs.h | 1 + 3 files changed, 322 insertions(+), 3 deletions(-) diff --git a/xen/arch/arm/gic-v3-lpi.c b/xen/arch/arm/gic-v3-lpi.c index 125f51e61b..01120aeed9 100644 --- a/xen/arch/arm/gic-v3-lpi.c +++ b/xen/arch/arm/gic-v3-lpi.c @@ -466,6 +466,9 @@ static int cpu_callback(struct notifier_block *nfb, uns= igned long action, switch ( action ) { case CPU_UP_PREPARE: + if ( system_state =3D=3D SYS_STATE_resume ) + break; + rc =3D gicv3_lpi_allocate_pendtable(cpu); if ( rc ) printk(XENLOG_ERR "Unable to allocate the pendtable for CPU%lu= \n", diff --git a/xen/arch/arm/gic-v3.c b/xen/arch/arm/gic-v3.c index 34fb065afc..d182a71478 100644 --- a/xen/arch/arm/gic-v3.c +++ b/xen/arch/arm/gic-v3.c @@ -1072,12 +1072,12 @@ out: return res; } =20 -static void gicv3_hyp_disable(void) +static void gicv3_hyp_enable(bool enable) { register_t hcr; =20 hcr =3D READ_SYSREG(ICH_HCR_EL2); - hcr &=3D ~GICH_HCR_EN; + hcr =3D enable ? (hcr | GICH_HCR_EN) : (hcr & ~GICH_HCR_EN); WRITE_SYSREG(hcr, ICH_HCR_EL2); isb(); } @@ -1184,7 +1184,7 @@ static void gicv3_disable_interface(void) spin_lock(&gicv3.lock); =20 gicv3_cpu_disable(); - gicv3_hyp_disable(); + gicv3_hyp_enable(false); =20 spin_unlock(&gicv3.lock); } @@ -1920,6 +1920,313 @@ static bool gic_dist_supports_lpis(void) return (readl_relaxed(GICD + GICD_TYPER) & GICD_TYPE_LPIS); } =20 +#ifdef CONFIG_SYSTEM_SUSPEND + +/* This struct represent block of 32 IRQs */ +struct dist_irq_block { + uint32_t icfgr[2]; + uint32_t ipriorityr[8]; + uint64_t irouter[32]; + uint32_t isactiver; + uint32_t isenabler; +}; + +struct redist_ctx { + uint32_t ctlr; + uint32_t icfgr; /* only PPIs stored */ + uint32_t igroupr; + uint32_t ipriorityr[8]; + uint32_t isactiver; + uint32_t isenabler; + + uint64_t pendbase; + uint64_t propbase; +}; + +/* GICv3 registers to be saved/restored on system suspend/resume */ +struct gicv3_ctx { + struct dist_ctx { + uint32_t ctlr; + struct dist_irq_block *irqs, *espi_irqs; + } dist; + + /* have only one rdist structure for last running CPU during suspend */ + struct redist_ctx rdist; + + struct cpu_ctx { + uint32_t ctlr; + uint32_t pmr; + uint32_t bpr; + uint32_t sre_el2; + uint32_t grpen; + } cpu; +}; + +static struct gicv3_ctx gicv3_ctx; + +static void __init gicv3_alloc_context(void) +{ + uint32_t blocks =3D DIV_ROUND_UP(gicv3_info.nr_lines, 32); + + /* The spec allows for systems without any SPIs */ + if ( blocks > 1 ) + { + gicv3_ctx.dist.irqs =3D xzalloc_array(struct dist_irq_block, block= s - 1); + if ( !gicv3_ctx.dist.irqs ) + panic("Failed to allocate memory for GICv3 suspend context\n"); + } + +#ifdef CONFIG_GICV3_ESPI + if ( !gic_number_espis() ) + return; + + blocks =3D gic_number_espis() / 32; + gicv3_ctx.dist.espi_irqs =3D xzalloc_array(struct dist_irq_block, bloc= ks); + if ( !gicv3_ctx.dist.espi_irqs ) + panic("Failed to allocate memory for GICv3 eSPI suspend context\n"= ); +#endif +} + +static int gicv3_disable_redist(void) +{ + void __iomem *waker =3D GICD_RDIST_BASE + GICR_WAKER; + s_time_t deadline; + + /* + * Avoid infinite loop if Non-secure does not have access to GICR_WAKE= R. + * See Arm IHI 0069H.b, 12.11.42 GICR_WAKER: + * When GICD_CTLR.DS =3D=3D 0 and an access is Non-secure accesses= to this + * register are RAZ/WI. + */ + if ( !(readl_relaxed(GICD + GICD_CTLR) & GICD_CTLR_DS) ) + return 0; + + deadline =3D NOW() + MILLISECS(1000); + + writel_relaxed(readl_relaxed(waker) | GICR_WAKER_ProcessorSleep, waker= ); + while ( (readl_relaxed(waker) & GICR_WAKER_ChildrenAsleep) =3D=3D 0 ) + { + if ( NOW() > deadline ) + { + printk("GICv3: Timeout waiting for redistributor to sleep\n"); + return -ETIMEDOUT; + } + cpu_relax(); + udelay(10); + } + + return 0; +} + +#define GET_SPI_REG_OFFSET(name, is_espi) \ + ((is_espi) ? GICD_##name##nE : GICD_##name) + +static void gicv3_store_spi_irq_block(struct dist_irq_block *irqs, + unsigned int i, bool is_espi) +{ + void __iomem *base; + unsigned int irq; + + base =3D GICD + GET_SPI_REG_OFFSET(ICFGR, is_espi) + i * sizeof(irqs->= icfgr); + irqs->icfgr[0] =3D readl_relaxed(base); + irqs->icfgr[1] =3D readl_relaxed(base + 4); + + base =3D GICD + GET_SPI_REG_OFFSET(IPRIORITYR, is_espi); + base +=3D i * sizeof(irqs->ipriorityr); + for ( irq =3D 0; irq < ARRAY_SIZE(irqs->ipriorityr); irq++ ) + irqs->ipriorityr[irq] =3D readl_relaxed(base + 4 * irq); + + base =3D GICD + GET_SPI_REG_OFFSET(IROUTER, is_espi); + base +=3D i * sizeof(irqs->irouter); + for ( irq =3D 0; irq < ARRAY_SIZE(irqs->irouter); irq++ ) + irqs->irouter[irq] =3D readq_relaxed_non_atomic(base + 8 * irq); + + base =3D GICD + GET_SPI_REG_OFFSET(ISACTIVER, is_espi); + base +=3D i * sizeof(irqs->isactiver); + irqs->isactiver =3D readl_relaxed(base); + + base =3D GICD + GET_SPI_REG_OFFSET(ISENABLER, is_espi); + base +=3D i * sizeof(irqs->isenabler); + irqs->isenabler =3D readl_relaxed(base); +} + +static void gicv3_restore_spi_irq_block(struct dist_irq_block *irqs, + unsigned int i, bool is_espi) +{ + void __iomem *base; + unsigned int irq; + + base =3D GICD + GET_SPI_REG_OFFSET(ICFGR, is_espi) + i * sizeof(irqs->= icfgr); + writel_relaxed(irqs->icfgr[0], base); + writel_relaxed(irqs->icfgr[1], base + 4); + + base =3D GICD + GET_SPI_REG_OFFSET(IPRIORITYR, is_espi); + base +=3D i * sizeof(irqs->ipriorityr); + for ( irq =3D 0; irq < ARRAY_SIZE(irqs->ipriorityr); irq++ ) + writel_relaxed(irqs->ipriorityr[irq], base + 4 * irq); + + base =3D GICD + GET_SPI_REG_OFFSET(IROUTER, is_espi); + base +=3D i * sizeof(irqs->irouter); + for ( irq =3D 0; irq < ARRAY_SIZE(irqs->irouter); irq++ ) + writeq_relaxed_non_atomic(irqs->irouter[irq], base + 8 * irq); + + base =3D GICD + GET_SPI_REG_OFFSET(ICENABLER, is_espi) + i * 4; + writel_relaxed(GENMASK(31, 0), base); + + base =3D GICD + GET_SPI_REG_OFFSET(ISENABLER, is_espi); + base +=3D i * sizeof(irqs->isenabler); + writel_relaxed(irqs->isenabler, base); + + base =3D GICD + GET_SPI_REG_OFFSET(ICACTIVER, is_espi) + i * 4; + writel_relaxed(GENMASK(31, 0), base); + + base =3D GICD + GET_SPI_REG_OFFSET(ISACTIVER, is_espi); + base +=3D i * sizeof(irqs->isactiver); + writel_relaxed(irqs->isactiver, base); +} + +static int gicv3_suspend(void) +{ + unsigned int i; + void __iomem *base; + int ret; + struct redist_ctx *rdist =3D &gicv3_ctx.rdist; + + /* Save GICC configuration */ + gicv3_ctx.cpu.ctlr =3D READ_SYSREG(ICC_CTLR_EL1); + gicv3_ctx.cpu.pmr =3D READ_SYSREG(ICC_PMR_EL1); + gicv3_ctx.cpu.bpr =3D READ_SYSREG(ICC_BPR1_EL1); + gicv3_ctx.cpu.sre_el2 =3D READ_SYSREG(ICC_SRE_EL2); + gicv3_ctx.cpu.grpen =3D READ_SYSREG(ICC_IGRPEN1_EL1); + + gicv3_disable_interface(); + + ret =3D gicv3_disable_redist(); + if ( ret ) + goto out_enable_iface; + + /* Save GICR configuration */ + gicv3_redist_wait_for_rwp(); + + base =3D GICD_RDIST_BASE; + + rdist->ctlr =3D readl_relaxed(base + GICR_CTLR); + + rdist->propbase =3D readq_relaxed(base + GICR_PROPBASER); + rdist->pendbase =3D readq_relaxed(base + GICR_PENDBASER); + + base =3D GICD_RDIST_SGI_BASE; + + /* Save priority on PPI and SGI interrupts */ + for ( i =3D 0; i < NR_GIC_LOCAL_IRQS / 4; i++ ) + rdist->ipriorityr[i] =3D readl_relaxed(base + GICR_IPRIORITYR0 + 4= * i); + + rdist->isactiver =3D readl_relaxed(base + GICR_ISACTIVER0); + rdist->isenabler =3D readl_relaxed(base + GICR_ISENABLER0); + rdist->igroupr =3D readl_relaxed(base + GICR_IGROUPR0); + rdist->icfgr =3D readl_relaxed(base + GICR_ICFGR1); + + /* Save GICD configuration */ + gicv3_dist_wait_for_rwp(); + gicv3_ctx.dist.ctlr =3D readl_relaxed(GICD + GICD_CTLR); + + for ( i =3D 1; i < DIV_ROUND_UP(gicv3_info.nr_lines, 32); i++ ) + gicv3_store_spi_irq_block(gicv3_ctx.dist.irqs + i - 1, i, false); + +#ifdef CONFIG_GICV3_ESPI + for ( i =3D 0; i < gic_number_espis() / 32; i++ ) + gicv3_store_spi_irq_block(gicv3_ctx.dist.espi_irqs + i, i, true); +#endif + + return 0; + + out_enable_iface: + gicv3_hyp_enable(true); + WRITE_SYSREG(gicv3_ctx.cpu.ctlr, ICC_CTLR_EL1); + isb(); + + return ret; +} + +static void gicv3_resume(void) +{ + int ret; + unsigned int i; + void __iomem *base; + struct redist_ctx *rdist =3D &gicv3_ctx.rdist; + + writel_relaxed(0, GICD + GICD_CTLR); + + for ( i =3D NR_GIC_LOCAL_IRQS; i < gicv3_info.nr_lines; i +=3D 32 ) + writel_relaxed(GENMASK(31, 0), GICD + GICD_IGROUPR + (i / 32) * 4); + + for ( i =3D 1; i < DIV_ROUND_UP(gicv3_info.nr_lines, 32); i++ ) + gicv3_restore_spi_irq_block(gicv3_ctx.dist.irqs + i - 1, i, false); + +#ifdef CONFIG_GICV3_ESPI + for ( i =3D 0; i < gic_number_espis() / 32; i++ ) + gicv3_restore_spi_irq_block(gicv3_ctx.dist.espi_irqs + i, i, true); +#endif + + writel_relaxed(gicv3_ctx.dist.ctlr, GICD + GICD_CTLR); + gicv3_dist_wait_for_rwp(); + + ret =3D gicv3_lpi_init_rdist(GICD_RDIST_BASE); + /* + * If LPIs are already enabled, assume firmware or the still-powered + * redistributor has valid PROPBASER/PENDBASER and skip reprogramming. + * Return -EBUSY so callers can ignore this case. + */ + if ( ret && ret !=3D -ENODEV && ret !=3D -EBUSY ) + panic("GICv3: Failed to re-initialize LPIs during resume\n"); + else if ( ret =3D=3D -EBUSY ) /* extra checks, just to be sure */ + { + base =3D GICD_RDIST_BASE; + if ( readq_relaxed(base + GICR_PROPBASER) !=3D rdist->propbase || + readq_relaxed(base + GICR_PENDBASER) !=3D rdist->pendbase ) + { + panic("GICv3: LPIs already enabled with unexpected PROPBASER/P= ENDBASER during resume\n"); + } + } + + /* Restore GICR (Redistributor) configuration */ + if ( gicv3_enable_redist() ) + panic("GICv3: Failed to re-enable redistributor during resume\n"); + + base =3D GICD_RDIST_SGI_BASE; + + writel_relaxed(GENMASK(31, 0), base + GICR_ICENABLER0); + gicv3_redist_wait_for_rwp(); + + for ( i =3D 0; i < NR_GIC_LOCAL_IRQS / 4; i++ ) + writel_relaxed(rdist->ipriorityr[i], base + GICR_IPRIORITYR0 + i *= 4); + + writel_relaxed(rdist->isactiver, base + GICR_ISACTIVER0); + writel_relaxed(rdist->igroupr, base + GICR_IGROUPR0); + writel_relaxed(rdist->icfgr, base + GICR_ICFGR1); + + gicv3_redist_wait_for_rwp(); + + writel_relaxed(rdist->isenabler, base + GICR_ISENABLER0); + writel_relaxed(rdist->ctlr, GICD_RDIST_BASE + GICR_CTLR); + + gicv3_redist_wait_for_rwp(); + + WRITE_SYSREG(gicv3_ctx.cpu.sre_el2, ICC_SRE_EL2); + isb(); + + /* Restore CPU interface (System registers) */ + WRITE_SYSREG(gicv3_ctx.cpu.pmr, ICC_PMR_EL1); + WRITE_SYSREG(gicv3_ctx.cpu.bpr, ICC_BPR1_EL1); + WRITE_SYSREG(gicv3_ctx.cpu.ctlr, ICC_CTLR_EL1); + WRITE_SYSREG(gicv3_ctx.cpu.grpen, ICC_IGRPEN1_EL1); + isb(); + + gicv3_hyp_init(); +} + +#endif /* CONFIG_SYSTEM_SUSPEND */ + /* Set up the GIC */ static int __init gicv3_init(void) { @@ -1994,6 +2301,10 @@ static int __init gicv3_init(void) =20 gicv3_hyp_init(); =20 +#ifdef CONFIG_SYSTEM_SUSPEND + gicv3_alloc_context(); +#endif + out: spin_unlock(&gicv3.lock); =20 @@ -2033,6 +2344,10 @@ static const struct gic_hw_operations gicv3_ops =3D { #endif .iomem_deny_access =3D gicv3_iomem_deny_access, .do_LPI =3D gicv3_do_LPI, +#ifdef CONFIG_SYSTEM_SUSPEND + .suspend =3D gicv3_suspend, + .resume =3D gicv3_resume, +#endif }; =20 static int __init gicv3_dt_preinit(struct dt_device_node *node, const void= *data) diff --git a/xen/arch/arm/include/asm/gic_v3_defs.h b/xen/arch/arm/include/= asm/gic_v3_defs.h index c373b94d19..992c8f9c2f 100644 --- a/xen/arch/arm/include/asm/gic_v3_defs.h +++ b/xen/arch/arm/include/asm/gic_v3_defs.h @@ -94,6 +94,7 @@ #define GICD_TYPE_LPIS (1U << 17) =20 #define GICD_CTLR_RWP (1UL << 31) +#define GICD_CTLR_DS (1U << 6) #define GICD_CTLR_ARE_NS (1U << 4) #define GICD_CTLR_ENABLE_G1A (1U << 1) #define GICD_CTLR_ENABLE_G1 (1U << 0) --=20 2.43.0 From nobody Wed Apr 8 05:51:36 2026 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass(p=none dis=none) header.from=gmail.com ARC-Seal: i=1; a=rsa-sha256; t=1775126880; cv=none; d=zohomail.com; s=zohoarc; b=lFJx+/jsTKWfwpOh8/agHhZyBvGYkviMSBucG6Eqr4RTufas1+swAvOfuaMG9F+BoAeT6cW7Eqw1VVwtzSil1XoZVdReOPpc8u9cgRWo4UJdolwcZmEsBRoUXBXwN8bSnWEnZ16duo1tgFITJyDg24/HytKzDBF+qR9vozNOQgY= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1775126880; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=qk9ILGeqhLaxN17ATAwS/XwCMf4ZnQc5O/7gi3JtyMs=; b=YvkVxDa8Z/ZIdmnEWEs2LKL64hSgodABIrQVbz5VMOGBbCqxi0AjaGxoXlCY6LbhqgX9AJqizeD4SNet3nC6+HE3KNwKzTh06kv29BTCxrAYe33RRArl1/iKfRo+DZOgbJNZWPrY56XHglj0d2Ou9erZfqs7piNKy1+XGy5sGkA= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1775126880571619.3623094910482; Thu, 2 Apr 2026 03:48:00 -0700 (PDT) Received: from list by lists.xenproject.org with outflank-mailman.1271525.1559674 (Exim 4.92) (envelope-from ) id 1w8FaH-00055g-PN; Thu, 02 Apr 2026 10:47:33 +0000 Received: by outflank-mailman (output) from mailman id 1271525.1559674; Thu, 02 Apr 2026 10:47:33 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1w8FaH-00054u-KI; Thu, 02 Apr 2026 10:47:33 +0000 Received: by outflank-mailman (input) for mailman id 1271525; Thu, 02 Apr 2026 10:47:32 +0000 Received: from mx.expurgate.net ([195.190.135.10]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1w8FaG-0004dR-6T for xen-devel@lists.xenproject.org; Thu, 02 Apr 2026 10:47:32 +0000 Received: from mx.expurgate.net (helo=localhost) by mx.expurgate.net with esmtp id 1w8FaF-009qA7-JE for xen-devel@lists.xenproject.org; Thu, 02 Apr 2026 12:47:31 +0200 Received: from [10.42.69.12] (helo=localhost) by localhost with ESMTP (eXpurgate MTA 0.9.1) (envelope-from ) id 69ce4932-e002-0a2a0a5209dd-0a2a450cd128-32 for ; Thu, 02 Apr 2026 12:47:31 +0200 Received: from [209.85.218.41] (helo=mail-ej1-f41.google.com) by tlsNG-d25034.mxtls.expurgate.net with ESMTPS (eXpurgate 4.56.0) (envelope-from ) id 69ce4943-f40c-0a2a450c0019-d155da29cd5e-3 for ; Thu, 02 Apr 2026 12:47:31 +0200 Received: by mail-ej1-f41.google.com with SMTP id a640c23a62f3a-b9910707d82so95092866b.1 for ; Thu, 02 Apr 2026 03:47:31 -0700 (PDT) Received: from EPUAKYIW02F7.. ([45.12.26.38]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-b9c3d028955sm76392366b.61.2026.04.02.03.47.28 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 02 Apr 2026 03:47:29 -0700 (PDT) X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" Authentication-Results: eu.smtp.expurgate.cloud; dkim=pass header.s=20251104 header.d=gmail.com header.i="@gmail.com" header.h="Content-Transfer-Encoding:MIME-Version:References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1775126850; x=1775731650; darn=lists.xenproject.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=qk9ILGeqhLaxN17ATAwS/XwCMf4ZnQc5O/7gi3JtyMs=; b=W2Opd3NOXDhqwaHjQ8eyEy6U8TOtgVfW0ynPPf7Fd9T/NOFIQ0to/UC5HxhgoxJyI3 lgzK7pP/tpks1HUR6xY9FN1X56jp6+ETN3wYpcYMjmJcrE38QhXcIKUOt8D8BlLtTYrm IdlqM1fJHt3YE+y/EmXU8biawm+fQMT+8Rxz0TrigSbwjXwwpb7q3/m0NYk8935fndcG x53BXqZy3srjMaU633N+S6int/NTI09Dqf+dTxlSq7gFDfIHVdCOGweFCJ4UOoKpPCTf VZWWtY7G6j+yVi6qpbqc1aM8gtCKFu7/KLos/96hARe0ZqWVNZJIjIOZ79Bm9Kq3HSFF gjHg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1775126850; x=1775731650; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=qk9ILGeqhLaxN17ATAwS/XwCMf4ZnQc5O/7gi3JtyMs=; b=UTu3MV1++V0XV6ubyzlS37DUYWZQEC7Hb3iONDj7F5LxaJkqnjXG0k2nBvcFyq/b75 yvKLHLfo7ZF+iVa98/dubLF+gZu23ICxRc53vAPaJHHHI1L+l9aYcfpdsT1IB6GfwCqb nWUpwgRqKvC3svXWEU34GqpFf19NaAY33blRRLhfY3vo6TrxQPcd4VzTw7veryglP7gf nZkZidzT3v8Rw8lI+1Q1+pI7CxLk//NpnLUAjfqp1MEtDE+jkcy4KZ66udj2kTdtqZcG YQGvieGENvp2DVJYSFYIoRHvWPLW1Qb4/EmZME0z2DlZlOsuJqcEQjHdSU/d3pGgGkzd J9aA== X-Gm-Message-State: AOJu0Ywicsxvk28ncZC9mwKYtqpHqA1BruQUwhxuOcAwPmW2btH+OcUT 4R+BQhjgP+CG/IvyHNGOAkWn2ne51pqgNX6VJRnOy+Ip4MYMAknCLTQ7PqCoGHET X-Gm-Gg: ATEYQzxAOeTpy4iksfHwyIe24RYxH0ZLuVgZslhLrVNIVaegd+4BDpqFZiHqZrJSMur ziNXD91CgA0HKYO7K8n8GvM3cEEIe4+Czv9aPzIZLgeIGM7iD6g5Aut4qa+rwvTp2R/AvaThuWJ J6dj1bhCEJZIcPTwcSgYAylzN2jSSzfJRf6JREyoo1oGsDgibPdNBtUHM4EyY6BSkIfg253acef Pyuc3mksGqmWexInwnrQ6VnB3IW7WmzF0FgoBqL7w4hzKocDA2OXPmFq4b5waaHE9hKEEOaDIOe bNl4x8Xs5G/JY4JBrafI1JW7j2eKTB93OoK88a6e4an0HP8ci4CT41rYbN5DgM4uVfL3mS6t7MU Pd+4H5hCboc8zZwjFpunCaQHs4AdlU7cAikkWw9UyF5H6W5letnvXOOfFCI0DsiPGlpBvn9oBMr 4+65b2aYxZHXxjEJPges2H07x22w== X-Received: by 2002:a17:907:1c85:b0:b93:7d70:20a0 with SMTP id a640c23a62f3a-b9c137a08acmr482815566b.4.1775126850107; Thu, 02 Apr 2026 03:47:30 -0700 (PDT) From: Mykola Kvach To: xen-devel@lists.xenproject.org Cc: Mykola Kvach , Stefano Stabellini , Julien Grall , Bertrand Marquis , Michal Orzel , Volodymyr Babchuk , Andrew Cooper , Anthony PERARD , Jan Beulich , =?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= Subject: [PATCH v8 05/13] xen/arm: gic-v3: add ITS suspend/resume support Date: Thu, 2 Apr 2026 13:45:06 +0300 Message-ID: <5a42b7c32fadf21262b8342f27e685916d0e5812.1775125380.git.mykola_kvach@epam.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: References: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-purgate-ID: tlsNG-d25034/1775126851-FE74EA3D-D1CDCDD3/0/0 X-purgate-type: clean X-purgate-size: 9675 X-ZohoMail-DKIM: pass (identity @gmail.com) X-ZM-MESSAGEID: 1775126882759154100 Content-Type: text/plain; charset="utf-8" From: Mykola Kvach Handle system suspend/resume for GICv3 with an ITS present so LPIs keep working after firmware powers the GIC down. Snapshot the CPU interface, distributor and last-CPU redistributor state, disable the ITS to cache its CTLR/CBASER/BASER registers, then restore everything and re-arm the collection on resume. Add list_for_each_entry_continue_reverse() in list.h for the ITS suspend error path that needs to roll back partially saved state. Based on Linux commit dba0bc7b76dc ("irqchip/gic-v3-its: Add ability to sav= e/restore ITS state") Signed-off-by: Mykola Kvach --- Changes in V8: - Reword the CBASER/CWRITER comment to match Xen and drop the stale Linux cmd_write reference. - Clarify the list_for_each_entry_continue_reverse() comment. - Factor out per-ITS helpers for collection setup and resume. - Restore each ITS and re-establish its collection mapping in the same loop, so a failed ITS resume is not followed by MAPC/SYNC on that un-restored instance. - panic in case when resume of an ITS failed - cleanup baser cache during suspend --- xen/arch/arm/gic-v3-its.c | 126 ++++++++++++++++++++++++-- xen/arch/arm/gic-v3.c | 15 ++- xen/arch/arm/include/asm/gic_v3_its.h | 23 +++++ xen/include/xen/list.h | 14 +++ 4 files changed, 166 insertions(+), 12 deletions(-) diff --git a/xen/arch/arm/gic-v3-its.c b/xen/arch/arm/gic-v3-its.c index 9ba068c46f..fe2865eac9 100644 --- a/xen/arch/arm/gic-v3-its.c +++ b/xen/arch/arm/gic-v3-its.c @@ -335,6 +335,22 @@ static int its_send_cmd_inv(struct host_its *its, return its_send_command(its, cmd); } =20 +static int gicv3_its_setup_collection_single(struct host_its *its, + unsigned int cpu) +{ + int ret; + + ret =3D its_send_cmd_mapc(its, cpu, cpu); + if ( ret ) + return ret; + + ret =3D its_send_cmd_sync(its, cpu); + if ( ret ) + return ret; + + return gicv3_its_wait_commands(its); +} + /* Set up the (1:1) collection mapping for the given host CPU. */ int gicv3_its_setup_collection(unsigned int cpu) { @@ -343,15 +359,7 @@ int gicv3_its_setup_collection(unsigned int cpu) =20 list_for_each_entry(its, &host_its_list, entry) { - ret =3D its_send_cmd_mapc(its, cpu, cpu); - if ( ret ) - return ret; - - ret =3D its_send_cmd_sync(its, cpu); - if ( ret ) - return ret; - - ret =3D gicv3_its_wait_commands(its); + ret =3D gicv3_its_setup_collection_single(its, cpu); if ( ret ) return ret; } @@ -1209,6 +1217,106 @@ int gicv3_its_init(void) return 0; } =20 +#ifdef CONFIG_SYSTEM_SUSPEND +int gicv3_its_suspend(void) +{ + struct host_its *its; + int ret; + + list_for_each_entry(its, &host_its_list, entry) + { + unsigned int i; + void __iomem *base =3D its->its_base; + + its->suspend_ctx.ctlr =3D readl_relaxed(base + GITS_CTLR); + ret =3D gicv3_disable_its(its); + if ( ret ) + { + writel_relaxed(its->suspend_ctx.ctlr, base + GITS_CTLR); + goto err; + } + + its->suspend_ctx.cbaser =3D readq_relaxed(base + GITS_CBASER); + + for (i =3D 0; i < GITS_BASER_NR_REGS; i++) + { + uint64_t baser =3D readq_relaxed(base + GITS_BASER0 + i * 8); + + its->suspend_ctx.baser[i] =3D 0; + + if ( !(baser & GITS_VALID_BIT) ) + continue; + + its->suspend_ctx.baser[i] =3D baser; + } + } + + return 0; + + err: + list_for_each_entry_continue_reverse(its, &host_its_list, entry) + writel_relaxed(its->suspend_ctx.ctlr, its->its_base + GITS_CTLR); + + return ret; +} + +static int gicv3_its_resume_single(struct host_its *its, unsigned int cpu) +{ + void __iomem *base =3D its->its_base; + unsigned int i; + int ret; + + /* + * Make sure that the ITS is disabled. If it fails to quiesce, + * don't restore it since writing to CBASER or BASER + * registers is undefined according to the GIC v3 ITS + * Specification. + */ + WARN_ON(readl_relaxed(base + GITS_CTLR) & GITS_CTLR_ENABLE); + ret =3D gicv3_disable_its(its); + if ( ret ) + return ret; + + writeq_relaxed(its->suspend_ctx.cbaser, base + GITS_CBASER); + + /* + * Writing CBASER resets CREADR to 0, so reset CWRITER to + * keep the command queue pointers aligned. + */ + writeq_relaxed(0, base + GITS_CWRITER); + + /* Restore GITS_BASER from the value cache. */ + for ( i =3D 0; i < GITS_BASER_NR_REGS; i++ ) + { + uint64_t baser =3D its->suspend_ctx.baser[i]; + + if ( !(baser & GITS_VALID_BIT) ) + continue; + + writeq_relaxed(baser, base + GITS_BASER0 + i * 8); + } + + writel_relaxed(its->suspend_ctx.ctlr, base + GITS_CTLR); + + return gicv3_its_setup_collection_single(its, cpu); +} + +void gicv3_its_resume(void) +{ + struct host_its *its; + unsigned int cpu =3D smp_processor_id(); + int ret; + + list_for_each_entry(its, &host_its_list, entry) + { + ret =3D gicv3_its_resume_single(its, cpu); + if ( ret ) + panic("GICv3: ITS@%"PRIpaddr": failed to restore during resume= : %d\n", + its->addr, ret); + } +} + +#endif /* CONFIG_SYSTEM_SUSPEND */ =20 /* * Local variables: diff --git a/xen/arch/arm/gic-v3.c b/xen/arch/arm/gic-v3.c index d182a71478..ef8318dd50 100644 --- a/xen/arch/arm/gic-v3.c +++ b/xen/arch/arm/gic-v3.c @@ -862,7 +862,7 @@ static bool gicv3_enable_lpis(void) return true; } =20 -static int __init gicv3_populate_rdist(void) +static int gicv3_populate_rdist(void) { int i; uint32_t aff; @@ -932,7 +932,7 @@ static int __init gicv3_populate_rdist(void) ret =3D gicv3_lpi_init_rdist(ptr); if ( ret && ret !=3D -ENODEV && ret !=3D -EBUSY ) { - printk("GICv3: CPU%d: Cannot initialize LPIs: %u\n= ", + printk("GICv3: CPU%d: Cannot initialize LPIs: %d\n= ", smp_processor_id(), ret); break; } @@ -2101,10 +2101,14 @@ static int gicv3_suspend(void) =20 gicv3_disable_interface(); =20 - ret =3D gicv3_disable_redist(); + ret =3D gicv3_its_suspend(); if ( ret ) goto out_enable_iface; =20 + ret =3D gicv3_disable_redist(); + if ( ret ) + goto out_its_resume; + /* Save GICR configuration */ gicv3_redist_wait_for_rwp(); =20 @@ -2140,6 +2144,9 @@ static int gicv3_suspend(void) =20 return 0; =20 + out_its_resume: + gicv3_its_resume(); + out_enable_iface: gicv3_hyp_enable(true); WRITE_SYSREG(gicv3_ctx.cpu.ctlr, ICC_CTLR_EL1); @@ -2212,6 +2219,8 @@ static void gicv3_resume(void) =20 gicv3_redist_wait_for_rwp(); =20 + gicv3_its_resume(); + WRITE_SYSREG(gicv3_ctx.cpu.sre_el2, ICC_SRE_EL2); isb(); =20 diff --git a/xen/arch/arm/include/asm/gic_v3_its.h b/xen/arch/arm/include/a= sm/gic_v3_its.h index 081bd19180..3ca74435c8 100644 --- a/xen/arch/arm/include/asm/gic_v3_its.h +++ b/xen/arch/arm/include/asm/gic_v3_its.h @@ -129,6 +129,13 @@ struct host_its { spinlock_t cmd_lock; void *cmd_buf; unsigned int flags; +#ifdef CONFIG_SYSTEM_SUSPEND + struct suspend_ctx { + uint32_t ctlr; + uint64_t cbaser; + uint64_t baser[GITS_BASER_NR_REGS]; + } suspend_ctx; +#endif }; =20 /* Map a collection for this host CPU to each host ITS. */ @@ -205,6 +212,11 @@ uint64_t gicv3_its_get_cacheability(void); uint64_t gicv3_its_get_shareability(void); unsigned int gicv3_its_get_memflags(void); =20 +#ifdef CONFIG_SYSTEM_SUSPEND +int gicv3_its_suspend(void); +void gicv3_its_resume(void); +#endif + #else =20 #ifdef CONFIG_ACPI @@ -272,6 +284,17 @@ static inline int gicv3_its_make_hwdom_dt_nodes(const = struct domain *d, return 0; } =20 +#ifdef CONFIG_SYSTEM_SUSPEND +static inline int gicv3_its_suspend(void) +{ + return 0; +} + +static inline void gicv3_its_resume(void) +{ +} +#endif + #endif /* CONFIG_HAS_ITS */ =20 #endif diff --git a/xen/include/xen/list.h b/xen/include/xen/list.h index 98d8482dab..2aab274157 100644 --- a/xen/include/xen/list.h +++ b/xen/include/xen/list.h @@ -535,6 +535,20 @@ static inline void list_splice_init(struct list_head *= list, &(pos)->member !=3D (head); = \ (pos) =3D list_entry((pos)->member.next, typeof(*(pos)), member)) =20 +/** + * list_for_each_entry_continue_reverse - iterate backwards from the given= point + * @pos: the type * to use as a loop cursor. + * @head: the head for your list. + * @member: the name of the list_head within the struct. + * + * Iterate over list of given type backwards, starting from the element pr= evious + * to the current one in list order. + */ +#define list_for_each_entry_continue_reverse(pos, head, member) \ + for ((pos) =3D list_entry((pos)->member.prev, typeof(*(pos)), member);= \ + &(pos)->member !=3D (head); = \ + (pos) =3D list_entry((pos)->member.prev, typeof(*(pos)), member)) + /** * list_for_each_entry_from - iterate over list of given type from the * current point --=20 2.43.0 From nobody Wed Apr 8 05:51:36 2026 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass(p=none dis=none) header.from=gmail.com ARC-Seal: i=1; a=rsa-sha256; t=1775126880; cv=none; d=zohomail.com; s=zohoarc; b=SHIXjs/eR28J7U95UzP7n88EGfFt6FmNsZxnUTS+I5MY/0yMyxgcsVWOa14ExwsbepSoLAYQbIXG7vCSsX0gyuZP0wECIUK4Z5bZ5z2dRd4c2dG9CuyUjRSk4DzXjTvLOuANew0likuxC8Q3RfwjPydPYWWZAA9Ka/Qz/MCv4nY= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1775126880; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=Tm0K7gfsRpBFVrENvrubslPphiODpeHwAJ+aZ5FZLuc=; b=UTcPlm0llU/TzjWn78WxkQUDj5kju91MaUtharY9ljD+NTxDLkUuj8r5wNpA6JEMnlDOqAn2+xbbpHPfkIgNs8pKpLHiDXKLy5lQyHxoS4yZlWW6cgu6MUkk8My8Qz5cEI0jKwwYjlxxEkGjhoJV9TZloDyIQ+jIzAj7vdULpxA= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1775126880206869.2920802484858; Thu, 2 Apr 2026 03:48:00 -0700 (PDT) Received: from list by lists.xenproject.org with outflank-mailman.1271526.1559682 (Exim 4.92) (envelope-from ) id 1w8FaJ-0005NL-6z; Thu, 02 Apr 2026 10:47:35 +0000 Received: by outflank-mailman (output) from mailman id 1271526.1559682; Thu, 02 Apr 2026 10:47:35 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1w8FaJ-0005N0-3l; Thu, 02 Apr 2026 10:47:35 +0000 Received: by outflank-mailman (input) for mailman id 1271526; Thu, 02 Apr 2026 10:47:33 +0000 Received: from mx.expurgate.net ([195.190.135.10]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1w8FaH-0004vd-62 for xen-devel@lists.xenproject.org; Thu, 02 Apr 2026 10:47:33 +0000 Received: from mx.expurgate.net (helo=localhost) by mx.expurgate.net with esmtp id 1w8FaG-00Ej0u-IP for xen-devel@lists.xenproject.org; Thu, 02 Apr 2026 12:47:32 +0200 Received: from [10.42.69.11] (helo=localhost) by localhost with ESMTP (eXpurgate MTA 0.9.1) (envelope-from ) id 69ce493d-2eae-0a2a0a5409dd-0a2a450be30c-20 for ; Thu, 02 Apr 2026 12:47:32 +0200 Received: from [209.85.208.46] (helo=mail-ed1-f46.google.com) by tlsNG-42698a.mxtls.expurgate.net with ESMTPS (eXpurgate 4.56.0) (envelope-from ) id 69ce4944-bca8-0a2a450b0019-d155d02eadcf-3 for ; Thu, 02 Apr 2026 12:47:32 +0200 Received: by mail-ed1-f46.google.com with SMTP id 4fb4d7f45d1cf-65c4152313fso1006449a12.1 for ; Thu, 02 Apr 2026 03:47:32 -0700 (PDT) Received: from EPUAKYIW02F7.. ([45.12.26.38]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-b9c3d028955sm76392366b.61.2026.04.02.03.47.30 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 02 Apr 2026 03:47:30 -0700 (PDT) X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" Authentication-Results: eu.smtp.expurgate.cloud; dkim=pass header.s=20251104 header.d=gmail.com header.i="@gmail.com" header.h="Content-Transfer-Encoding:MIME-Version:References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1775126852; x=1775731652; darn=lists.xenproject.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Tm0K7gfsRpBFVrENvrubslPphiODpeHwAJ+aZ5FZLuc=; b=HFYnDybK1UebaDizt6eZk5c/4D3c+mPJXXC6AbSiViBhbZG5antkrqTM9lJUkh3ZGD neu0txMAzrokKucmGjVz+qCAU5oPoVFHgNO00ejSQdWPWJabozdybnkBVIc+46aD7KFc kRN//S1D/0uVC/AZywsm+MRozj4nEvC6ZWn2eLHKsxOcb9Zn0c4bmO9as6M5B9h/oQBF rRpkkoiBCRplhwBANaIGo64VeXvlgUNXvAVtelFJKnWwuQBXOqNpN6tnE6O3U+mdY6EI KhjTUuYCukQ3n5RW1ViuDMKzTrgp0O9s5zbe2xbOkeX+jiXzzsVJLmH/yub36nb/uTS9 HsXg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1775126852; x=1775731652; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=Tm0K7gfsRpBFVrENvrubslPphiODpeHwAJ+aZ5FZLuc=; b=FlAzh8urW6VtDmpUrQ4PIzbfsw8YxRyC0u6808JRJo0JZGDcST3X4FB7izpnd+OmP/ tiOZCgyqtnvKfHJOyqjhbNVn/lbYJ78gBTIfaA61mDLrowRxq6xdMwOFxrPoRLFX2Dyg elqeBC2BIIut92kS1thTWZHx1ZuLUqf4byz6zMdm0ga5hCicFvbeg4jS21GJG3wMLit4 ymhdeg3ihvs1Q7n3HxSxkMpxJTSatorShWhTdLHZ+xtEt/IRkXWIxcWtQXebiTtvyGF4 0h1NIDhLNxMvAle+UsvRJyiR5gHC3B9D8zRnsqYJEWz9kWNpS3bwiPABDWAjoQYBgpUL B4LQ== X-Gm-Message-State: AOJu0YzDWOsv2om6poBHpliKwqJTujoEwPNq4xMV2CKCqe+YSVi8QIqC LFb7k1TuJj1v1+V4uuTldCLy/tE1Vgp04NiIXl10ear8U+Tscf5G2oBxUiG9sQ7N X-Gm-Gg: ATEYQzzCqvHHryOIyPZlc0Jc1lLNhDkoNbLXIVl4tt/uNWAe/oKqOHEln4HuWOI1wSn pt1bxl6S5bJqKjj34NMcPejC8+HHwi6Z9gcgiMR65aYnEsm5x3M66jN2Ala/+m1vyrVve7QMvEa 7047Hj6qnT/Gu3oAKcItDtMkRmDLrYfAn+SU+E7FDSumGInnIeeXiQgfmYyBT72zSbGdVj2pqQ+ rhFUPRsc0lhZIIcoX3SzPteQp6C09i2HwBe7k5kcDL9H9+VMcZrNsDHtEsItdyVWFGC9vIuPItn YPBXQAxKr6B9w65gMSVMeNMGIq1ZWwFMQ/92fVCG68nTnYuEKzpq9LBkmG+myCfgUZ8mMjjav6A kgM/UwEtUrcz3peVq7q2w44UkeDSC2Jwf8y+b+YkcLhzAZVFZm4BduBKcLE/1PHYYgKs0/As3+V neZpebnNnjMkaT+Tx/A9yMlPX1RQ== X-Received: by 2002:a17:906:6207:b0:b98:3d79:40c1 with SMTP id a640c23a62f3a-b9c13b0a2f2mr479932666b.33.1775126851612; Thu, 02 Apr 2026 03:47:31 -0700 (PDT) From: Mykola Kvach To: xen-devel@lists.xenproject.org Cc: Mykola Kvach , Volodymyr Babchuk , Bertrand Marquis , Jens Wiklander , Stefano Stabellini , Julien Grall , Michal Orzel Subject: [PATCH v8 06/13] xen/arm: tee: keep init_tee_secondary() for hotplug and resume Date: Thu, 2 Apr 2026 13:45:07 +0300 Message-ID: <666917e8e8e435640c272c19908f412c315b2dc6.1775125380.git.mykola_kvach@epam.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: References: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-purgate-ID: tlsNG-42698a/1775126852-1714D2A1-F11E7718/0/0 X-purgate-type: clean X-purgate-size: 892 X-ZohoMail-DKIM: pass (identity @gmail.com) X-ZM-MESSAGEID: 1775126882641154100 Content-Type: text/plain; charset="utf-8" From: Mykola Kvach init_tee_secondary() was marked __init and freed after boot. Calling it from the CPU hotplug/resume path then executed discarded code, which could crash Xen. Drop __init so the TEE mediator secondary init can run safely on hotplugged and resumed CPUs. Signed-off-by: Mykola Kvach --- xen/arch/arm/tee/tee.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/xen/arch/arm/tee/tee.c b/xen/arch/arm/tee/tee.c index 8501443c8e..00e561fc78 100644 --- a/xen/arch/arm/tee/tee.c +++ b/xen/arch/arm/tee/tee.c @@ -128,7 +128,7 @@ static int __init tee_init(void) =20 presmp_initcall(tee_init); =20 -void __init init_tee_secondary(void) +void init_tee_secondary(void) { if ( cur_mediator && cur_mediator->ops->init_secondary ) cur_mediator->ops->init_secondary(); --=20 2.43.0 From nobody Wed Apr 8 05:51:36 2026 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass(p=none dis=none) header.from=gmail.com ARC-Seal: i=1; a=rsa-sha256; t=1775126884; cv=none; d=zohomail.com; s=zohoarc; b=kWFDYrtGx1GoBhz/Q0rbBOPMhhrB733XUDF6xnmsPUVNcBh09AXldMsB1JJo+WA+/camS76a8B8ypungQ5gvL3THvBEx74v7Egw51LGNvrSJeBMocLSCvmyUDw3IQT9gAkY42Ymr4L2CSdGp4zA0DFZXtsPq2u+u6iAQ4AxpPHY= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1775126884; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=E2TvwCSkEBb4iQcEjFfj1UiUN2JVsCuECcxIJOnRftY=; b=hhIOKyrA4VkYvS1xhxs3+qPGg9P1n+QqNceiJf0NYBiKQYvNAdp+EBwiIkLlv1f8zQGrVL/qZAGLr3GS08397BQTHzCDOprpLFlnKPoFmJuWcU1QcwSLPO67n0klmNG6ploYUs4q1JexlwaKDRWuOZQTsLkyvugJEa8UbDgT2U0= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1775126884177503.56570612268035; Thu, 2 Apr 2026 03:48:04 -0700 (PDT) Received: from list by lists.xenproject.org with outflank-mailman.1271527.1559689 (Exim 4.92) (envelope-from ) id 1w8FaJ-0005RK-PK; Thu, 02 Apr 2026 10:47:35 +0000 Received: by outflank-mailman (output) from mailman id 1271527.1559689; Thu, 02 Apr 2026 10:47:35 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1w8FaJ-0005R1-GB; Thu, 02 Apr 2026 10:47:35 +0000 Received: by outflank-mailman (input) for mailman id 1271527; Thu, 02 Apr 2026 10:47:34 +0000 Received: from mx.expurgate.net ([195.190.135.10]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1w8FaI-0005CF-8o for xen-devel@lists.xenproject.org; Thu, 02 Apr 2026 10:47:34 +0000 Received: from mx.expurgate.net (helo=localhost) by mx.expurgate.net with esmtp id 1w8FaH-009qA7-Lh for xen-devel@lists.xenproject.org; Thu, 02 Apr 2026 12:47:33 +0200 Received: from [10.42.69.12] (helo=localhost) by localhost with ESMTP (eXpurgate MTA 0.9.1) (envelope-from ) id 69ce4932-e002-0a2a0a5209dd-0a2a450cd128-40 for ; Thu, 02 Apr 2026 12:47:33 +0200 Received: from [209.85.218.49] (helo=mail-ej1-f49.google.com) by tlsNG-d25034.mxtls.expurgate.net with ESMTPS (eXpurgate 4.56.0) (envelope-from ) id 69ce4945-f40c-0a2a450c0019-d155da31ad3b-3 for ; Thu, 02 Apr 2026 12:47:33 +0200 Received: by mail-ej1-f49.google.com with SMTP id a640c23a62f3a-b932fe2e1a7so84945766b.1 for ; Thu, 02 Apr 2026 03:47:33 -0700 (PDT) Received: from EPUAKYIW02F7.. ([45.12.26.38]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-b9c3d028955sm76392366b.61.2026.04.02.03.47.31 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 02 Apr 2026 03:47:32 -0700 (PDT) X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" Authentication-Results: eu.smtp.expurgate.cloud; dkim=pass header.s=20251104 header.d=gmail.com header.i="@gmail.com" header.h="Content-Transfer-Encoding:MIME-Version:References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1775126853; x=1775731653; darn=lists.xenproject.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=E2TvwCSkEBb4iQcEjFfj1UiUN2JVsCuECcxIJOnRftY=; b=jRdeBmAlBkJ9d7HKYA1KQjZt6GAJcV3zxI57FF1bfypMIDOk29Q5LUJt+GfHevhl/6 nmYuLiLgDgR8X8ULKWPvEfzvmF6DKwe2ZT4nIdBDkFZIoieRkoR9ucEPset5ELOvn1J8 PG7OHn8JPTd/jpwwnTLek4JuFDGWcSDEtI8/6DbM/5vFckkImvfT8snl07grHHz0IdgQ EyYUCz8b4MAJXIAH34lcwhi9mgzouWpabG9gf504rCzJAKAmVaQODJqZztl1XTQsw5eY Wvu9WotApk4fkEXZQkrdtjQb+4kuDf7zj6LKQti6G/+Oe2mWQKdP1WDOEx/u2kwCK+X7 28uA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1775126853; x=1775731653; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=E2TvwCSkEBb4iQcEjFfj1UiUN2JVsCuECcxIJOnRftY=; b=pk/Sn5S8f6f5Jd+huJEvjmV3RQOfPwM8QPuWz/TjmcxY7vudxAIg3omq8jqiPQTzd4 VgMn8VnTBQVOAmspVxwgvZVcfpkaGFVKwp88tAn0rj8DnnbEZxdLOSaWvfaVjduk0yHn KEDJ6HCHbZpB34hNVQYM910V65Y3tP9Yl0CSyoucGlQlWfI00EmiVZlDFvRDIMSa1pi6 zM232UvmnSBCqQEcDiTy6urGUqug2GvSowLDruMXXRaBTFDie+a4LIcZ1SbEtjGK2vDk p6k2DRAOBDvbrIx1oVM/OfiAE5Klo8BpX22tm/SZJlhjZBF+ImjyNZzl5AiPnVeioR4L cFkg== X-Gm-Message-State: AOJu0YympHjsWZAXTF0MdK44dh71T/9up9yYNAMPEDQmQxYoJFu890aP qXcAI3r0Vij825KpUHSILdSvesOCwfAOaIe3LvKrFueQ9UItrzcADuVNX+crkz6G X-Gm-Gg: ATEYQzzH20BgsMnl/jrvpnZDX9bb31UjHPdYvZk4bTq2i7uhIDxQlRyotzUwPlhDfXY Hpur2q8uikY71dLmP1ENLv1YKoqxmlVUISHpNYq1yTRQ9yTkqUp4NCEmLMOLrStGeZFbeWRR3GI hblxwoVGL4N0EA0bVpQtfDI5I0j4PE3xsTEUM6YQc/tS33Zpbq+4BMh2/Saww2CvBRIBztmzyFs fJKl7JZ1p08CUm6AeTJlRMKNKAXG+c9Kj3TUVbfF0cGl9LZqkG49t53KNd//rwAcupUn+cp28un mDH3CNZEBZS4UP123Ste+aX8DZsv58abKiJkTEd9kr6KC0xmfhlcKYXFgXfKOjZCoAO/b0MK9hk dU6Q26bjiQrbPAzb0Nlx9XM7BZHhVlAcIizkTHYNTK13qX7dvL2kmQQi4ZMAM+okwdcB/TYxHYz BfGe0Dp9K1M0w5VTEg2c9/MxSWIQ== X-Received: by 2002:a17:906:ba84:b0:b94:2648:4672 with SMTP id a640c23a62f3a-b9c13b7bdbemr484677266b.41.1775126852622; Thu, 02 Apr 2026 03:47:32 -0700 (PDT) From: Mykola Kvach To: xen-devel@lists.xenproject.org Cc: Mykola Kvach , Volodymyr Babchuk , Bertrand Marquis , Jens Wiklander , Stefano Stabellini , Julien Grall , Michal Orzel Subject: [PATCH v8 07/13] xen/arm: ffa: fix notification SRI across CPU hotplug/suspend Date: Thu, 2 Apr 2026 13:45:08 +0300 Message-ID: X-Mailer: git-send-email 2.43.0 In-Reply-To: References: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-purgate-ID: tlsNG-d25034/1775126853-81557A3D-86A1C0B4/0/0 X-purgate-type: clean X-purgate-size: 3590 X-ZohoMail-DKIM: pass (identity @gmail.com) X-ZM-MESSAGEID: 1775126886635154100 Content-Type: text/plain; charset="utf-8" From: Mykola Kvach The FF-A notification SRI interrupt handler was not correctly tied to CPU hotplug and suspend/resume. As a result, CPUs going offline and back online could end up with stale or missing handlers, breaking delivery of FF-A notifications. Signed-off-by: Mykola Kvach --- xen/arch/arm/tee/ffa_notif.c | 63 ++++++++++++++++++++++++++++-------- 1 file changed, 50 insertions(+), 13 deletions(-) diff --git a/xen/arch/arm/tee/ffa_notif.c b/xen/arch/arm/tee/ffa_notif.c index 186e726412..513c399594 100644 --- a/xen/arch/arm/tee/ffa_notif.c +++ b/xen/arch/arm/tee/ffa_notif.c @@ -360,10 +360,28 @@ static int32_t ffa_notification_bitmap_destroy(uint16= _t vm_id) return ffa_simple_call(FFA_NOTIFICATION_BITMAP_DESTROY, vm_id, 0, 0, 0= ); } =20 -void ffa_notif_init_interrupt(void) +static DEFINE_PER_CPU_READ_MOSTLY(struct irqaction, sri_irq); + +static int request_sri_irq(void) { int ret; + struct irqaction *sri_action =3D &this_cpu(sri_irq); + + sri_action->name =3D "FF-A notif"; + sri_action->handler =3D notif_irq_handler; + sri_action->dev_id =3D NULL; + sri_action->free_on_release =3D 0; + + ret =3D setup_irq(notif_sri_irq, 0, sri_action); + if ( ret ) + printk(XENLOG_ERR "ffa: setup_irq irq %u failed: error %d\n", + notif_sri_irq, ret); =20 + return ret; +} + +void ffa_notif_init_interrupt(void) +{ if ( fw_notif_enabled && notif_sri_irq < NR_GIC_SGI ) { /* @@ -376,14 +394,36 @@ void ffa_notif_init_interrupt(void) * pending, while the SPMC in the secure world will not notice that * the interrupt was lost. */ - ret =3D request_irq(notif_sri_irq, 0, notif_irq_handler, "FF-A not= if", - NULL); - if ( ret ) - printk(XENLOG_ERR "ffa: request_irq irq %u failed: error %d\n", - notif_sri_irq, ret); + request_sri_irq(); } } =20 +static void deinit_ffa_notif_interrupt(void) +{ + if ( fw_notif_enabled && notif_sri_irq < NR_GIC_SGI ) + release_irq(notif_sri_irq, NULL); +} + +static int cpu_ffa_notif_callback(struct notifier_block *nfb, + unsigned long action, + void *hcpu) +{ + switch ( action ) + { + case CPU_DYING: + deinit_ffa_notif_interrupt(); + break; + default: + break; + } + + return NOTIFY_DONE; +} + +static struct notifier_block cpu_ffa_notif_nfb =3D { + .notifier_call =3D cpu_ffa_notif_callback, +}; + void ffa_notif_init(void) { const struct arm_smccc_1_2_regs arg =3D { @@ -392,7 +432,6 @@ void ffa_notif_init(void) }; struct arm_smccc_1_2_regs resp; unsigned int irq; - int ret; =20 /* Only enable fw notification if all ABIs we need are supported */ if ( ffa_fw_supports_fid(FFA_NOTIFICATION_BITMAP_CREATE) && @@ -408,13 +447,11 @@ void ffa_notif_init(void) notif_sri_irq =3D irq; if ( irq >=3D NR_GIC_SGI ) irq_set_type(irq, IRQ_TYPE_EDGE_RISING); - ret =3D request_irq(irq, 0, notif_irq_handler, "FF-A notif", NULL); - if ( ret ) - { - printk(XENLOG_ERR "ffa: request_irq irq %u failed: error %d\n", - irq, ret); + + if ( request_sri_irq() ) return; - } + + register_cpu_notifier(&cpu_ffa_notif_nfb); fw_notif_enabled =3D true; } } --=20 2.43.0 From nobody Wed Apr 8 05:51:36 2026 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass(p=none dis=none) header.from=gmail.com ARC-Seal: i=1; a=rsa-sha256; t=1775126887; cv=none; d=zohomail.com; s=zohoarc; b=ENn0+zl0kVi0U9PFLk8KxyHg1LOfgF59RDYx/LKbcDTCjvXJvknv+V6v4nuQ275zzpEjyxUf9xiP4EPGt2z50wuSkDNuGvqqwNxxE/4dFelunpGnWcuoZlc4CaGVaU0dhzzJsVSTY4UXl2CpAvaI1H4BoC9Vk7LzgVFv+TGdf1I= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1775126887; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=k/YQHTjWfFNRfU47XoZ3xNzm7CYArEgw/v+Ue+GlOW0=; b=lz6JgZKbWXadez0NxaZSuVYZHZI4CC4vzGfy+Q1c3kUuTpxmApBSjOiFDT9p/XabfZWth06HD2vYqDZE12aW9Zeo/afcf7d49KgForrAvhs81kfcYIH8J5Y8pw4zhYn7/Vnm+PXEDCrOvfei575r5rmrqDsNRqm7RWdYpSXTuU8= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1775126887037381.83064070970454; Thu, 2 Apr 2026 03:48:07 -0700 (PDT) Received: from list by lists.xenproject.org with outflank-mailman.1271528.1559700 (Exim 4.92) (envelope-from ) id 1w8FaL-0005pC-5R; Thu, 02 Apr 2026 10:47:37 +0000 Received: by outflank-mailman (output) from mailman id 1271528.1559700; Thu, 02 Apr 2026 10:47:37 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1w8FaK-0005nm-Qw; Thu, 02 Apr 2026 10:47:36 +0000 Received: by outflank-mailman (input) for mailman id 1271528; Thu, 02 Apr 2026 10:47:35 +0000 Received: from mx.expurgate.net ([195.190.135.10]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1w8FaJ-0005RM-Pj for xen-devel@lists.xenproject.org; Thu, 02 Apr 2026 10:47:35 +0000 Received: from mx.expurgate.net (helo=localhost) by mx.expurgate.net with esmtp id 1w8FaJ-00Ej0u-5A for xen-devel@lists.xenproject.org; Thu, 02 Apr 2026 12:47:35 +0200 Received: from [10.42.69.5] (helo=localhost) by localhost with ESMTP (eXpurgate MTA 0.9.1) (envelope-from ) id 69ce4947-2eae-0a2a0a5409dd-0a2a4505b638-0 for ; Thu, 02 Apr 2026 12:47:35 +0200 Received: from [209.85.218.41] (helo=mail-ej1-f41.google.com) by tlsNG-c201ff.mxtls.expurgate.net with ESMTPS (eXpurgate 4.56.0) (envelope-from ) id 69ce4946-3760-0a2a45050019-d155da29e11d-3 for ; Thu, 02 Apr 2026 12:47:35 +0200 Received: by mail-ej1-f41.google.com with SMTP id a640c23a62f3a-b980785a0bfso91460766b.3 for ; Thu, 02 Apr 2026 03:47:35 -0700 (PDT) Received: from EPUAKYIW02F7.. ([45.12.26.38]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-b9c3d028955sm76392366b.61.2026.04.02.03.47.32 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 02 Apr 2026 03:47:33 -0700 (PDT) X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" Authentication-Results: eu.smtp.expurgate.cloud; dkim=pass header.s=20251104 header.d=gmail.com header.i="@gmail.com" header.h="Content-Transfer-Encoding:MIME-Version:References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1775126854; x=1775731654; darn=lists.xenproject.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=k/YQHTjWfFNRfU47XoZ3xNzm7CYArEgw/v+Ue+GlOW0=; b=lnfUgyTTEAX+efdmLlJO5cw9uROFTg3MldGocgRl1rxBowGzT8U5HjVsjWu9ZrQ1+1 36rt6CTizZU0Q1lK5Y7Lned7VpzbnxKD+1yc/sHuj3V3CA3c+jv4nqM4NJQNpVMFZcrp aiArnhpMRtkFJ2wRqjcPiniE4lXpJgSoL5LuOX9Y/Uayw6c9h305eCLESJWnzHJnKw+5 wQTBd1ruiTSm3/kty2w3oM3sszeENP3VCoj1oDXE0fHTmruNfqR/fac5LrmBEDj/XrFt QAgRA4QNKUsVCoRRbIqP7KmVpJQLr2bv8OAsJqt/iDGl3yk2HSMlKgwiEpjxw/NIi7Qv E+/g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1775126854; x=1775731654; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=k/YQHTjWfFNRfU47XoZ3xNzm7CYArEgw/v+Ue+GlOW0=; b=He7r4SakySDTerwigVxefZm7DpVJWaoPD0kZuS5U0Amt9ldQ+M4LL0TlCBzJ/9zBw3 uJ3uPdhQa1Wni3cq3uUCfcBAEPI58wkXxU9T+uiAtyiiU1YQrMGeH/SNy2GJVEQ748B1 BnvVxMaCztXaRNgxxmOaYcBSUHU2VUpmCuQMk8tVQ0Mvf3fibaEtHWVt25M8H9xJSvvd Noi0CeJq4PByR7l2VnGNO42oKPeq/DXa///dReuhjWd/mceEtL4PW8YAv4f6wdo/YH1U ZiBXqHSpk0bNKo9ptU6cQexROD4EKNWbaXRMZ56ojICZxMsHtZ0827kKce8QG+oavOT5 Zyfw== X-Gm-Message-State: AOJu0Yybx2pmpgQ4bs9hbrfn1oQ7+1y8zFGslDxcCbfy9aGHextfdmc7 jG74afandmpq3Nk7Fl+uufpYA+DgF19N2zBY1QZuhCYz3VdBn2L1/lZ6S0V0Sgu8 X-Gm-Gg: ATEYQzxhTrVQ7FD5jpQqwqgmf8AdLMIt5CxKt54wbTVwRFUFFZS2pp4a13P91Z0WUXv qTmZga0Je0vmowDML5RLYbajr9ls1Rzz+/V4vjdhe0wL8PAxa1JSD7pU+3Ps2VJCXaz6Y4i7btZ qmO06PwdcdrlX6jfPyGDpKEz7vy1/HPIh65zknyGr7ZYd1LlQkd56F6JVpgtvlneQgEvCD/PJ9g /iwzHFi5V+L1U7OCllYrkSK3/ENrfOq3quEer0994kY6wJiYNK6qmghcJOGErWOeA/rZFf06h1s 1Jr9GLnp6F1JgVCzniUqqBLAFm+dsruCS+w0YL5KC4lmxRW3QqMneaFHqO45pNSCidgzAEIwjiW V9COlAYvzDz6864inm8+cOVk89VPQ9nYotDB2zQzuVinR63KIukmH+fep/qAYUGpnO6kKQftvMW LAnmsCr5CIVweyrQJ9U5+0QA05dw== X-Received: by 2002:a17:907:e8a:b0:b97:eae9:d45e with SMTP id a640c23a62f3a-b9c13cc4dd3mr438756766b.50.1775126853726; Thu, 02 Apr 2026 03:47:33 -0700 (PDT) From: Mykola Kvach To: xen-devel@lists.xenproject.org Cc: Mykola Kvach , Stefano Stabellini , Julien Grall , Bertrand Marquis , Michal Orzel , Volodymyr Babchuk Subject: [PATCH v8 08/13] iommu/ipmmu-vmsa: Implement suspend/resume callbacks Date: Thu, 2 Apr 2026 13:45:09 +0300 Message-ID: <63b219c3cae5201c5db804f69c3b88ac41c9bdf6.1775125380.git.mykola_kvach@epam.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: References: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-purgate-ID: tlsNG-c201ff/1775126855-32B2096F-9C3AE72C/0/0 X-purgate-type: clean X-purgate-size: 13035 X-ZohoMail-DKIM: pass (identity @gmail.com) X-ZM-MESSAGEID: 1775126887871158500 Content-Type: text/plain; charset="utf-8" From: Oleksandr Tyshchenko Store and restore active context and micro-TLB registers. Tested on R-Car H3 Starter Kit. Signed-off-by: Oleksandr Tyshchenko Signed-off-by: Mykola Kvach --- Changes in V7: - moved suspend context allocation before pci stuff --- xen/drivers/passthrough/arm/ipmmu-vmsa.c | 305 ++++++++++++++++++++++- 1 file changed, 298 insertions(+), 7 deletions(-) diff --git a/xen/drivers/passthrough/arm/ipmmu-vmsa.c b/xen/drivers/passthr= ough/arm/ipmmu-vmsa.c index ea9fa9ddf3..6765bd3083 100644 --- a/xen/drivers/passthrough/arm/ipmmu-vmsa.c +++ b/xen/drivers/passthrough/arm/ipmmu-vmsa.c @@ -71,6 +71,8 @@ }) #endif =20 +#define dev_dbg(dev, fmt, ...) \ + dev_print(dev, XENLOG_DEBUG, fmt, ## __VA_ARGS__) #define dev_info(dev, fmt, ...) \ dev_print(dev, XENLOG_INFO, fmt, ## __VA_ARGS__) #define dev_warn(dev, fmt, ...) \ @@ -130,6 +132,24 @@ struct ipmmu_features { unsigned int imuctr_ttsel_mask; }; =20 +#ifdef CONFIG_SYSTEM_SUSPEND + +struct ipmmu_reg_ctx { + unsigned int imttlbr0; + unsigned int imttubr0; + unsigned int imttbcr; + unsigned int imctr; +}; + +struct ipmmu_vmsa_backup { + struct device *dev; + unsigned int *utlbs_val; + unsigned int *asids_val; + struct list_head list; +}; + +#endif + /* Root/Cache IPMMU device's information */ struct ipmmu_vmsa_device { struct device *dev; @@ -142,6 +162,9 @@ struct ipmmu_vmsa_device { struct ipmmu_vmsa_domain *domains[IPMMU_CTX_MAX]; unsigned int utlb_refcount[IPMMU_UTLB_MAX]; const struct ipmmu_features *features; +#ifdef CONFIG_SYSTEM_SUSPEND + struct ipmmu_reg_ctx *reg_backup[IPMMU_CTX_MAX]; +#endif }; =20 /* @@ -547,6 +570,245 @@ static void ipmmu_domain_free_context(struct ipmmu_vm= sa_device *mmu, spin_unlock_irqrestore(&mmu->lock, flags); } =20 +#ifdef CONFIG_SYSTEM_SUSPEND + +static DEFINE_SPINLOCK(ipmmu_devices_backup_lock); +static LIST_HEAD(ipmmu_devices_backup); + +static struct ipmmu_reg_ctx root_pgtable[IPMMU_CTX_MAX]; + +static uint32_t ipmmu_imuasid_read(struct ipmmu_vmsa_device *mmu, + unsigned int utlb) +{ + return ipmmu_read(mmu, ipmmu_utlb_reg(mmu, IMUASID(utlb))); +} + +static void ipmmu_utlbs_backup(struct ipmmu_vmsa_device *mmu) +{ + struct ipmmu_vmsa_backup *backup_data; + + dev_dbg(mmu->dev, "Handle micro-TLBs backup\n"); + + spin_lock(&ipmmu_devices_backup_lock); + + list_for_each_entry( backup_data, &ipmmu_devices_backup, list ) + { + struct iommu_fwspec *fwspec =3D dev_iommu_fwspec_get(backup_data->= dev); + unsigned int i; + + if ( to_ipmmu(backup_data->dev) !=3D mmu ) + continue; + + for ( i =3D 0; i < fwspec->num_ids; i++ ) + { + unsigned int utlb =3D fwspec->ids[i]; + + backup_data->asids_val[i] =3D ipmmu_imuasid_read(mmu, utlb); + backup_data->utlbs_val[i] =3D ipmmu_imuctr_read(mmu, utlb); + } + } + + spin_unlock(&ipmmu_devices_backup_lock); +} + +static void ipmmu_utlbs_restore(struct ipmmu_vmsa_device *mmu) +{ + struct ipmmu_vmsa_backup *backup_data; + + dev_dbg(mmu->dev, "Handle micro-TLBs restore\n"); + + spin_lock(&ipmmu_devices_backup_lock); + + list_for_each_entry( backup_data, &ipmmu_devices_backup, list ) + { + struct iommu_fwspec *fwspec =3D dev_iommu_fwspec_get(backup_data->= dev); + unsigned int i; + + if ( to_ipmmu(backup_data->dev) !=3D mmu ) + continue; + + for ( i =3D 0; i < fwspec->num_ids; i++ ) + { + unsigned int utlb =3D fwspec->ids[i]; + + ipmmu_imuasid_write(mmu, utlb, backup_data->asids_val[i]); + ipmmu_imuctr_write(mmu, utlb, backup_data->utlbs_val[i]); + } + } + + spin_unlock(&ipmmu_devices_backup_lock); +} + +static void ipmmu_domain_backup_context(struct ipmmu_vmsa_domain *domain) +{ + struct ipmmu_vmsa_device *mmu =3D domain->mmu->root; + struct ipmmu_reg_ctx *regs =3D mmu->reg_backup[domain->context_id]; + + dev_dbg(mmu->dev, "Handle domain context %u backup\n", domain->context= _id); + + regs->imttlbr0 =3D ipmmu_ctx_read_root(domain, IMTTLBR0); + regs->imttubr0 =3D ipmmu_ctx_read_root(domain, IMTTUBR0); + regs->imttbcr =3D ipmmu_ctx_read_root(domain, IMTTBCR); + regs->imctr =3D ipmmu_ctx_read_root(domain, IMCTR); +} + +static void ipmmu_domain_restore_context(struct ipmmu_vmsa_domain *domain) +{ + struct ipmmu_vmsa_device *mmu =3D domain->mmu->root; + struct ipmmu_reg_ctx *regs =3D mmu->reg_backup[domain->context_id]; + + dev_dbg(mmu->dev, "Handle domain context %u restore\n", domain->contex= t_id); + + ipmmu_ctx_write_root(domain, IMTTLBR0, regs->imttlbr0); + ipmmu_ctx_write_root(domain, IMTTUBR0, regs->imttubr0); + ipmmu_ctx_write_root(domain, IMTTBCR, regs->imttbcr); + ipmmu_ctx_write_all(domain, IMCTR, regs->imctr | IMCTR_FLUSH); +} + +/* + * Xen: Unlike Linux implementation, Xen uses a single driver instance + * for handling all IPMMUs. There is no framework for ipmmu_suspend/resume + * callbacks to be invoked for each IPMMU device. So, we need to iterate + * through all registered IPMMUs performing required actions. + * + * Also take care of restoring special settings, such as translation + * table format, etc. + */ +static int __must_check ipmmu_suspend(void) +{ + struct ipmmu_vmsa_device *mmu; + + if ( !iommu_enabled ) + return 0; + + printk(XENLOG_DEBUG "ipmmu: Suspending...\n"); + + spin_lock(&ipmmu_devices_lock); + + list_for_each_entry( mmu, &ipmmu_devices, list ) + { + if ( ipmmu_is_root(mmu) ) + { + unsigned int i; + + for ( i =3D 0; i < mmu->num_ctx; i++ ) + { + if ( !mmu->domains[i] ) + continue; + ipmmu_domain_backup_context(mmu->domains[i]); + } + } + else + ipmmu_utlbs_backup(mmu); + } + + spin_unlock(&ipmmu_devices_lock); + + return 0; +} + +static void ipmmu_resume(void) +{ + struct ipmmu_vmsa_device *mmu; + + if ( !iommu_enabled ) + return; + + printk(XENLOG_DEBUG "ipmmu: Resuming...\n"); + + spin_lock(&ipmmu_devices_lock); + + list_for_each_entry( mmu, &ipmmu_devices, list ) + { + uint32_t reg; + + /* Do not use security group function */ + reg =3D IMSCTLR + mmu->features->control_offset_base; + ipmmu_write(mmu, reg, ipmmu_read(mmu, reg) & ~IMSCTLR_USE_SECGRP); + + if ( ipmmu_is_root(mmu) ) + { + unsigned int i; + + /* Use stage 2 translation table format */ + reg =3D IMSAUXCTLR + mmu->features->control_offset_base; + ipmmu_write(mmu, reg, ipmmu_read(mmu, reg) | IMSAUXCTLR_S2PTE); + + for ( i =3D 0; i < mmu->num_ctx; i++ ) + { + if ( !mmu->domains[i] ) + continue; + ipmmu_domain_restore_context(mmu->domains[i]); + } + } + else + ipmmu_utlbs_restore(mmu); + } + + spin_unlock(&ipmmu_devices_lock); +} + +static int ipmmu_alloc_ctx_suspend(struct device *dev) +{ + struct ipmmu_vmsa_backup *backup_data; + unsigned int *utlbs_val, *asids_val; + struct iommu_fwspec *fwspec =3D dev_iommu_fwspec_get(dev); + + utlbs_val =3D xzalloc_array(unsigned int, fwspec->num_ids); + if ( !utlbs_val ) + return -ENOMEM; + + asids_val =3D xzalloc_array(unsigned int, fwspec->num_ids); + if ( !asids_val ) + { + xfree(utlbs_val); + return -ENOMEM; + } + + backup_data =3D xzalloc(struct ipmmu_vmsa_backup); + if ( !backup_data ) + { + xfree(utlbs_val); + xfree(asids_val); + return -ENOMEM; + } + + backup_data->dev =3D dev; + backup_data->utlbs_val =3D utlbs_val; + backup_data->asids_val =3D asids_val; + + spin_lock(&ipmmu_devices_backup_lock); + list_add(&backup_data->list, &ipmmu_devices_backup); + spin_unlock(&ipmmu_devices_backup_lock); + + return 0; +} + +#ifdef CONFIG_HAS_PCI +static void ipmmu_free_ctx_suspend(struct device *dev) +{ + struct ipmmu_vmsa_backup *backup_data, *tmp; + + spin_lock(&ipmmu_devices_backup_lock); + + list_for_each_entry_safe( backup_data, tmp, &ipmmu_devices_backup, lis= t ) + { + if ( backup_data->dev =3D=3D dev ) + { + list_del(&backup_data->list); + xfree(backup_data->utlbs_val); + xfree(backup_data->asids_val); + xfree(backup_data); + break; + } + } + + spin_unlock(&ipmmu_devices_backup_lock); +} +#endif /* CONFIG_HAS_PCI */ + +#endif /* CONFIG_SYSTEM_SUSPEND */ + static int ipmmu_domain_init_context(struct ipmmu_vmsa_domain *domain) { uint64_t ttbr; @@ -559,6 +821,9 @@ static int ipmmu_domain_init_context(struct ipmmu_vmsa_= domain *domain) return ret; =20 domain->context_id =3D ret; +#ifdef CONFIG_SYSTEM_SUSPEND + domain->mmu->root->reg_backup[ret] =3D &root_pgtable[ret]; +#endif =20 /* * TTBR0 @@ -615,6 +880,9 @@ static void ipmmu_domain_destroy_context(struct ipmmu_v= msa_domain *domain) ipmmu_ctx_write_root(domain, IMCTR, IMCTR_FLUSH); ipmmu_tlb_sync(domain); =20 +#ifdef CONFIG_SYSTEM_SUSPEND + domain->mmu->root->reg_backup[domain->context_id] =3D NULL; +#endif ipmmu_domain_free_context(domain->mmu->root, domain->context_id); } =20 @@ -1340,10 +1608,11 @@ static int ipmmu_add_device(u8 devfn, struct device= *dev) struct iommu_fwspec *fwspec; =20 #ifdef CONFIG_HAS_PCI + int ret; + if ( dev_is_pci(dev) ) { struct pci_dev *pdev =3D dev_to_pci(dev); - int ret; =20 if ( devfn !=3D pdev->devfn ) return 0; @@ -1371,6 +1640,15 @@ static int ipmmu_add_device(u8 devfn, struct device = *dev) /* Let Xen know that the master device is protected by an IOMMU. */ dt_device_set_protected(dev_to_dt(dev)); } + +#ifdef CONFIG_SYSTEM_SUSPEND + if ( ipmmu_alloc_ctx_suspend(dev) ) + { + dev_err(dev, "Failed to allocate context for suspend\n"); + return -ENOMEM; + } +#endif + #ifdef CONFIG_HAS_PCI if ( dev_is_pci(dev) ) { @@ -1379,26 +1657,28 @@ static int ipmmu_add_device(u8 devfn, struct device= *dev) struct pci_host_bridge *bridge; struct iommu_fwspec *fwspec_bridge; unsigned int utlb_osid0 =3D 0; - int ret; =20 bridge =3D pci_find_host_bridge(pdev->seg, pdev->bus); if ( !bridge ) { dev_err(dev, "Failed to find host bridge\n"); - return -ENODEV; + ret =3D -ENODEV; + goto free_suspend_ctx; } =20 fwspec_bridge =3D dev_iommu_fwspec_get(dt_to_dev(bridge->dt_node)); if ( fwspec_bridge->num_ids < 1 ) { dev_err(dev, "Failed to find host bridge uTLB\n"); - return -ENXIO; + ret =3D -ENXIO; + goto free_suspend_ctx; } =20 if ( fwspec->num_ids < 1 ) { dev_err(dev, "Failed to find uTLB"); - return -ENXIO; + ret =3D -ENXIO; + goto free_suspend_ctx; } =20 rcar4_pcie_osid_regs_init(bridge); @@ -1407,7 +1687,7 @@ static int ipmmu_add_device(u8 devfn, struct device *= dev) if ( ret < 0 ) { dev_err(dev, "No unused OSID regs\n"); - return ret; + goto free_suspend_ctx; } reg_id =3D ret; =20 @@ -1422,7 +1702,7 @@ static int ipmmu_add_device(u8 devfn, struct device *= dev) { rcar4_pcie_osid_bdf_clear(bridge, reg_id); rcar4_pcie_osid_reg_free(bridge, reg_id); - return ret; + goto free_suspend_ctx; } } #endif @@ -1431,6 +1711,13 @@ static int ipmmu_add_device(u8 devfn, struct device = *dev) dev_name(fwspec->iommu_dev), fwspec->num_ids); =20 return 0; +#ifdef CONFIG_HAS_PCI + free_suspend_ctx: +#ifdef CONFIG_SYSTEM_SUSPEND + ipmmu_free_ctx_suspend(dev); +#endif + return ret; +#endif } =20 static int ipmmu_iommu_domain_init(struct domain *d) @@ -1492,6 +1779,10 @@ static const struct iommu_ops ipmmu_iommu_ops =3D .unmap_page =3D arm_iommu_unmap_page, .dt_xlate =3D ipmmu_dt_xlate, .add_device =3D ipmmu_add_device, +#ifdef CONFIG_SYSTEM_SUSPEND + .suspend =3D ipmmu_suspend, + .resume =3D ipmmu_resume, +#endif }; =20 static __init int ipmmu_init(struct dt_device_node *node, const void *data) --=20 2.43.0 From nobody Wed Apr 8 05:51:36 2026 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass(p=none dis=none) header.from=gmail.com ARC-Seal: i=1; a=rsa-sha256; t=1775126890; cv=none; d=zohomail.com; s=zohoarc; b=MI01pHYzt+gMZ6lrlymMCYAvbr3jtC//vXvpH626bAuqiF74B/tjC39nbmRCnbJzBlm0xlayPBr39YShjaO32V0Q/wVcJJ1FpNPNsWMFKjXMihUie/nC0UMN7rqbJ9c2D8W2ZhWHSNMEW4AK9Uac7JOsZgZ5C/aLTTKvY5JoaYI= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1775126890; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=DKnYT2Yc1d3B8QvA2ZhD3FJdUHpvRkUQIwPnTMgKX5Y=; b=MTuEfSIlRE4gHCAuokNDTvT4IDNFV4YfT+hPPmGuhIjcby5QzquAJbuekDmif9bqy2SPZr6OUN16jmwn4j9qiJIzP8geRgbo5VW1969d7ZZr8CnK1jj/kL6zZ4LUF0tFhu300Iak8X0wKuGtIZXr3fltu5ZbAVYB0TCFJJ/p1rY= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1775126890742893.5204726904508; Thu, 2 Apr 2026 03:48:10 -0700 (PDT) Received: from list by lists.xenproject.org with outflank-mailman.1271531.1559708 (Exim 4.92) (envelope-from ) id 1w8FaM-00068u-VC; Thu, 02 Apr 2026 10:47:38 +0000 Received: by outflank-mailman (output) from mailman id 1271531.1559708; Thu, 02 Apr 2026 10:47:38 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1w8FaM-00068W-LF; Thu, 02 Apr 2026 10:47:38 +0000 Received: by outflank-mailman (input) for mailman id 1271531; Thu, 02 Apr 2026 10:47:37 +0000 Received: from mx.expurgate.net ([195.190.135.10]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1w8FaK-0005kv-QR for xen-devel@lists.xenproject.org; Thu, 02 Apr 2026 10:47:36 +0000 Received: from mx.expurgate.net (helo=localhost) by mx.expurgate.net with esmtp id 1w8FaK-004haJ-6K for xen-devel@lists.xenproject.org; Thu, 02 Apr 2026 12:47:36 +0200 Received: from [10.42.69.2] (helo=localhost) by localhost with ESMTP (eXpurgate MTA 0.9.1) (envelope-from ) id 69ce4943-5cb7-0a2a0a5109dd-0a2a4502ebd6-10 for ; Thu, 02 Apr 2026 12:47:36 +0200 Received: from [209.85.218.44] (helo=mail-ej1-f44.google.com) by tlsNG-720697.mxtls.expurgate.net with ESMTPS (eXpurgate 4.56.0) (envelope-from ) id 69ce4948-42fa-0a2a45020019-d155da2cf1bf-3 for ; Thu, 02 Apr 2026 12:47:36 +0200 Received: by mail-ej1-f44.google.com with SMTP id a640c23a62f3a-b9825ba7e8dso100467666b.3 for ; Thu, 02 Apr 2026 03:47:36 -0700 (PDT) Received: from EPUAKYIW02F7.. ([45.12.26.38]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-b9c3d028955sm76392366b.61.2026.04.02.03.47.33 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 02 Apr 2026 03:47:34 -0700 (PDT) X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" Authentication-Results: eu.smtp.expurgate.cloud; dkim=pass header.s=20251104 header.d=gmail.com header.i="@gmail.com" header.h="Content-Transfer-Encoding:MIME-Version:References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1775126855; x=1775731655; darn=lists.xenproject.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=DKnYT2Yc1d3B8QvA2ZhD3FJdUHpvRkUQIwPnTMgKX5Y=; b=bCrEUNUJ0ujzdDmlWh/za3go8C0Ea+LBzhUnSFUJ38jS8kUWk0cLniOqlZpbUSLSq+ FBjtALoMRQVzsuUdV1Lh60j+u7vS7cKDlZi2Xa4MRPe4ggIa0xjMC7IsGWwV3oLvB64w FPt0qtj6kP1VT3lKi3qxap/eD72vYNys+ETk75Rvf4tNMvyUILf468bjdb/ZmyayfyyT o3xjXlTUpkfDutf5Y5yDQ4qJ7AVuPtqptoqys+vC8nUzAgBDTslTv1JIumjOn7pCU89B gTuDwug4LpegVWWoxigP+w9/xgWVSQivVYJMpanebh8OzPDPZU9o+MBJobfqiev+Yzsr YnpQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1775126855; x=1775731655; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=DKnYT2Yc1d3B8QvA2ZhD3FJdUHpvRkUQIwPnTMgKX5Y=; b=GHkDA51O2lmTEW6QDZXWNZeSpNh0mGWB7O4wPFwK+WwK/Hx7W9apYYTmVGF0Q9VP9r DRr4XBkadXfNk1p3iO5v3nggkVl6sB8tOm2VbO0yDdtue28kifGKHJKbjDaKFYPC3o7c R7pFXWFn7QDmberRqY8VuqgFkHLBsRVlqVbyCaE1Om1VVhz8Xd4Zkqv7XrYEnULflPgC g9mmnr8/muXOKP0yKv5H9FJjMb0S6ddGC0A/lTEWnk2f/LENmr0s8cU6VyKz/d8PA0Ug FA4d8d4TM6ZmQISacB1UbwNGxzhAOnYMdFWa+UbeLZHWIJ7kuC52EAuOhjAHu2SLUbH4 RJFw== X-Gm-Message-State: AOJu0YxXbb82FalwVJV9nZ3hLvbK8K6IUzuAebn0zgE0zDQG/xP1Aey1 J6PogFycwjQdQ27Kqiiu7qVQhi6q5e4tcx7jaQ6RfD3EFQ9IL9xNH2n68gXm/9JV X-Gm-Gg: ATEYQzxdXI4uWAb226DU5VzAbYXHxZQKFV0p6O0gwnUL6dJweqNXMjlZ+RWdyZuAAIo ArMVgIwlRkb/491TiyNjyq79lOJ45GBESSX3ObFpLAg9m/IWv4eY1ezcrt0U84akApJOA5BWDQE Yi7CsKcg72kkd8xAUpFDFhbG8pj64siLGBAVuSu3HVTILj7OLmwmWFyxVs6qy5yzKAXrKEXVCcK Ao1tanTuXJK0puLg+hSFaWUck8h7Fue06dc7xV4wouL6MQBpMR2V00g6locnFdaMeWFsO3Ynh8A yjlv3e0OnZiY0eXSHyXoN2xSCoY6/VtAzl8BA6qfI0RDw155iUYqvolp0o3Izc1CeYwONxc3h+n F/pMAsYXS/Dt1pjMZfVKKmqq71esN/vb5hmpVQJvghick1WiaqZLkiv4LN9IjFlw02Ke2viNLH+ gZILTm6iaSn4sTBSyFX+p9ezIS3Q== X-Received: by 2002:a17:906:5789:b0:b93:ff4d:8e38 with SMTP id a640c23a62f3a-b9c138f657dmr370628066b.20.1775126855136; Thu, 02 Apr 2026 03:47:35 -0700 (PDT) From: Mykola Kvach To: xen-devel@lists.xenproject.org Cc: Mykola Kvach , Bertrand Marquis , Rahul Singh , Stefano Stabellini , Julien Grall , Michal Orzel , Volodymyr Babchuk Subject: [PATCH v8 09/13] arm/smmu-v3: add suspend/resume handlers Date: Thu, 2 Apr 2026 13:45:10 +0300 Message-ID: <038337ec0e3f1474c6cc6a92090e28c6972e4dc6.1775125380.git.mykola_kvach@epam.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: References: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-purgate-ID: tlsNG-720697/1775126856-AEB27CD1-92ABB1C3/0/0 X-purgate-type: clean X-purgate-size: 8574 X-ZohoMail-DKIM: pass (identity @gmail.com) X-ZM-MESSAGEID: 1775126891888158501 Content-Type: text/plain; charset="utf-8" Before we suspend SMMU, we want to ensure that all commands (especially ATC_INV) have been flushed by the CMDQ, i.e. the CMDQs are empty. The suspend callback configures the SMMU to abort new transactions, disables the main translation unit and then drains the command queue to ensure completion of any in-flight commands. The resume callback performs a full device reset via 'arm_smmu_device_reset' to bring the SMMU back to an operational state. Signed-off-by: Mykola Kvach --- Changes in V8: - Honor ARM_SMMU_FEAT_SEV when draining the CMDQ during suspend, matching the existing runtime CMD_SYNC path. - Fold the suspend rollback reset path into a helper and rename the error reporting to describe suspend rollback rather than resume. - Treat SMMU reset failure during resume as fatal instead of logging and continuing with a potentially unusable IOMMU. - cosmetic changes --- xen/drivers/passthrough/arm/smmu-v3.c | 172 ++++++++++++++++++++------ 1 file changed, 136 insertions(+), 36 deletions(-) diff --git a/xen/drivers/passthrough/arm/smmu-v3.c b/xen/drivers/passthroug= h/arm/smmu-v3.c index bf153227db..7607ffc9ca 100644 --- a/xen/drivers/passthrough/arm/smmu-v3.c +++ b/xen/drivers/passthrough/arm/smmu-v3.c @@ -1814,8 +1814,7 @@ static int arm_smmu_write_reg_sync(struct arm_smmu_de= vice *smmu, u32 val, } =20 /* GBPA is "special" */ -static int __init arm_smmu_update_gbpa(struct arm_smmu_device *smmu, - u32 set, u32 clr) +static int arm_smmu_update_gbpa(struct arm_smmu_device *smmu, u32 set, u32= clr) { int ret; u32 reg, __iomem *gbpa =3D smmu->base + ARM_SMMU_GBPA; @@ -1995,10 +1994,29 @@ err_free_evtq_irq: return ret; } =20 +static int arm_smmu_enable_irqs(struct arm_smmu_device *smmu) +{ + int ret; + u32 irqen_flags =3D IRQ_CTRL_EVTQ_IRQEN | IRQ_CTRL_GERROR_IRQEN; + + if ( smmu->features & ARM_SMMU_FEAT_PRI ) + irqen_flags |=3D IRQ_CTRL_PRIQ_IRQEN; + + /* Enable interrupt generation on the SMMU */ + ret =3D arm_smmu_write_reg_sync(smmu, irqen_flags, + ARM_SMMU_IRQ_CTRL, ARM_SMMU_IRQ_CTRLACK); + if ( ret ) + { + dev_warn(smmu->dev, "failed to enable irqs\n"); + return ret; + } + + return 0; +} + static int __init arm_smmu_setup_irqs(struct arm_smmu_device *smmu) { int ret, irq; - u32 irqen_flags =3D IRQ_CTRL_EVTQ_IRQEN | IRQ_CTRL_GERROR_IRQEN; =20 /* Disable IRQs first */ ret =3D arm_smmu_write_reg_sync(smmu, 0, ARM_SMMU_IRQ_CTRL, @@ -2028,22 +2046,7 @@ static int __init arm_smmu_setup_irqs(struct arm_smm= u_device *smmu) } } =20 - if (smmu->features & ARM_SMMU_FEAT_PRI) - irqen_flags |=3D IRQ_CTRL_PRIQ_IRQEN; - - /* Enable interrupt generation on the SMMU */ - ret =3D arm_smmu_write_reg_sync(smmu, irqen_flags, - ARM_SMMU_IRQ_CTRL, ARM_SMMU_IRQ_CTRLACK); - if (ret) { - dev_warn(smmu->dev, "failed to enable irqs\n"); - goto err_free_irqs; - } - return 0; - -err_free_irqs: - arm_smmu_free_irqs(smmu); - return ret; } =20 static int arm_smmu_device_disable(struct arm_smmu_device *smmu) @@ -2057,7 +2060,7 @@ static int arm_smmu_device_disable(struct arm_smmu_de= vice *smmu) return ret; } =20 -static int __init arm_smmu_device_reset(struct arm_smmu_device *smmu) +static int arm_smmu_device_reset(struct arm_smmu_device *smmu) { int ret; u32 reg, enables; @@ -2163,17 +2166,9 @@ static int __init arm_smmu_device_reset(struct arm_s= mmu_device *smmu) } } =20 - ret =3D arm_smmu_setup_irqs(smmu); - if (ret) { - dev_err(smmu->dev, "failed to setup irqs\n"); + ret =3D arm_smmu_enable_irqs(smmu); + if ( ret ) return ret; - } - - /* Initialize tasklets for threaded IRQs*/ - tasklet_init(&smmu->evtq_irq_tasklet, arm_smmu_evtq_tasklet, smmu); - tasklet_init(&smmu->priq_irq_tasklet, arm_smmu_priq_tasklet, smmu); - tasklet_init(&smmu->combined_irq_tasklet, arm_smmu_combined_irq_tasklet, - smmu); =20 /* Enable the SMMU interface, or ensure bypass */ if (disable_bypass) { @@ -2181,20 +2176,16 @@ static int __init arm_smmu_device_reset(struct arm_= smmu_device *smmu) } else { ret =3D arm_smmu_update_gbpa(smmu, 0, GBPA_ABORT); if (ret) - goto err_free_irqs; + return ret; } ret =3D arm_smmu_write_reg_sync(smmu, enables, ARM_SMMU_CR0, ARM_SMMU_CR0ACK); if (ret) { dev_err(smmu->dev, "failed to enable SMMU interface\n"); - goto err_free_irqs; + return ret; } =20 return 0; - -err_free_irqs: - arm_smmu_free_irqs(smmu); - return ret; } =20 static int arm_smmu_device_hw_probe(struct arm_smmu_device *smmu) @@ -2558,10 +2549,23 @@ static int __init arm_smmu_device_probe(struct plat= form_device *pdev) if (ret) goto out_free; =20 + ret =3D arm_smmu_setup_irqs(smmu); + if ( ret ) + { + dev_err(smmu->dev, "failed to setup irqs\n"); + goto out_free; + } + + /* Initialize tasklets for threaded IRQs*/ + tasklet_init(&smmu->evtq_irq_tasklet, arm_smmu_evtq_tasklet, smmu); + tasklet_init(&smmu->priq_irq_tasklet, arm_smmu_priq_tasklet, smmu); + tasklet_init(&smmu->combined_irq_tasklet, arm_smmu_combined_irq_tasklet, + smmu); + /* Reset the device */ ret =3D arm_smmu_device_reset(smmu); if (ret) - goto out_free; + goto out_free_irqs; =20 /* * Keep a list of all probed devices. This will be used to query @@ -2575,6 +2579,8 @@ static int __init arm_smmu_device_probe(struct platfo= rm_device *pdev) =20 return 0; =20 +out_free_irqs: + arm_smmu_free_irqs(smmu); =20 out_free: arm_smmu_free_structures(smmu); @@ -2855,6 +2861,96 @@ static void arm_smmu_iommu_xen_domain_teardown(struc= t domain *d) xfree(xen_domain); } =20 +#ifdef CONFIG_SYSTEM_SUSPEND + +static void arm_smmu_reset_for_suspend_rollback(struct arm_smmu_device *sm= mu) +{ + int ret =3D arm_smmu_device_reset(smmu); + + if ( ret ) + dev_err(smmu->dev, "Failed to reset during suspend rollback: %d\n", + ret); +} + +static int arm_smmu_suspend(void) +{ + struct arm_smmu_device *smmu; + int ret =3D 0; + + list_for_each_entry(smmu, &arm_smmu_devices, devices) + { + bool wfe =3D !!(smmu->features & ARM_SMMU_FEAT_SEV); + + /* Abort all transactions before disable to avoid spurious bypass */ + ret =3D arm_smmu_update_gbpa(smmu, GBPA_ABORT, 0); + if ( ret ) + goto fail; + + /* Disable the SMMU via CR0.EN and all queues except CMDQ */ + ret =3D arm_smmu_write_reg_sync(smmu, CR0_CMDQEN, ARM_SMMU_CR0, + ARM_SMMU_CR0ACK); + if ( ret ) + { + dev_err(smmu->dev, "Timed-out while disabling smmu\n"); + goto fail; + } + + /* + * At this point the SMMU is completely disabled and won't access + * any translation/config structures, even speculative accesses + * aren't performed as per the IHI0070 spec (section 6.3.9.6). + */ + + /* Wait for the CMDQs to be drained to flush any pending commands */ + ret =3D queue_poll_cons(&smmu->cmdq.q, true, wfe); + if ( ret ) + { + dev_err(smmu->dev, "Draining queues timed-out\n"); + goto fail; + } + + /* Disable everything */ + ret =3D arm_smmu_device_disable(smmu); + if ( ret ) + goto fail; + + dev_dbg(smmu->dev, "Suspended smmu\n"); + } + + return 0; + + fail: + /* Reset the device that failed as well as any already-suspended ones. */ + arm_smmu_reset_for_suspend_rollback(smmu); + + list_for_each_entry_continue_reverse(smmu, &arm_smmu_devices, devices) + arm_smmu_reset_for_suspend_rollback(smmu); + + return ret; +} + +static void arm_smmu_resume(void) +{ + int ret; + struct arm_smmu_device *smmu; + + list_for_each_entry(smmu, &arm_smmu_devices, devices) + { + dev_dbg(smmu->dev, "Resuming device\n"); + + /* + * The reset will re-initialize all the base addresses, queues, + * prod and cons maintained within struct arm_smmu_device as well as + * re-enable the interrupts. + */ + ret =3D arm_smmu_device_reset(smmu); + if ( ret ) + panic("SMMUv3: %s: Failed to reset during resume: %d\n", + dev_name(smmu->dev), ret); + } +} +#endif + static const struct iommu_ops arm_smmu_iommu_ops =3D { .page_sizes =3D PAGE_SIZE_4K, .init =3D arm_smmu_iommu_xen_domain_init, @@ -2867,6 +2963,10 @@ static const struct iommu_ops arm_smmu_iommu_ops =3D= { .unmap_page =3D arm_iommu_unmap_page, .dt_xlate =3D arm_smmu_dt_xlate, .add_device =3D arm_smmu_add_device, +#ifdef CONFIG_SYSTEM_SUSPEND + .suspend =3D arm_smmu_suspend, + .resume =3D arm_smmu_resume, +#endif }; =20 static __init int arm_smmu_dt_init(struct dt_device_node *dev, --=20 2.43.0 From nobody Wed Apr 8 05:51:36 2026 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass(p=none dis=none) header.from=gmail.com ARC-Seal: i=1; a=rsa-sha256; t=1775126879; cv=none; d=zohomail.com; s=zohoarc; b=YQ0t/uHuJVWAklSgAPBlBzM3SchT5ka7OPJuOV2KecGLMNceWzX7zPF1hZ0iFowgfIOPQmHjD3zx4szeipPyew2B+WBXHAxYn72IOKgPlnLmNCEghWPmQ7m8hwlDo556mYxowUhaAsE5m0d7vJ8eXKtublCCWpYARd4ircGoV90= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1775126879; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=Wn2YE/N1073l2aBvkRQ/5nSVLZsSRqqVHeilL+wTO7s=; b=YQKa1K20ElWOTT2kS5rnglIpzb/VeXxmXoBspgKGtI+nO3glxfP/vBnXH01W511mgTNnURHDeBHoP+w+Ym1fIksNmsvPaS5VTz6V03IKaNNmhm1PhEbSnrPzckoK4ZgHAidmd7faK6bC9dAaai0YMBdDg9Vxh21V6uJKwKBk0yo= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1775126879704835.8811696070123; Thu, 2 Apr 2026 03:47:59 -0700 (PDT) Received: from list by lists.xenproject.org with outflank-mailman.1271535.1559717 (Exim 4.92) (envelope-from ) id 1w8FaO-0006NJ-II; Thu, 02 Apr 2026 10:47:40 +0000 Received: by outflank-mailman (output) from mailman id 1271535.1559717; Thu, 02 Apr 2026 10:47:40 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1w8FaO-0006M9-19; Thu, 02 Apr 2026 10:47:40 +0000 Received: by outflank-mailman (input) for mailman id 1271535; Thu, 02 Apr 2026 10:47:38 +0000 Received: from mx.expurgate.net ([195.190.135.10]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1w8FaM-00064W-Hd for xen-devel@lists.xenproject.org; Thu, 02 Apr 2026 10:47:38 +0000 Received: from mx.expurgate.net (helo=localhost) by mx.expurgate.net with esmtp id 1w8FaL-00Ej0u-T9 for xen-devel@lists.xenproject.org; Thu, 02 Apr 2026 12:47:37 +0200 Received: from [10.42.69.5] (helo=localhost) by localhost with ESMTP (eXpurgate MTA 0.9.1) (envelope-from ) id 69ce4947-2eae-0a2a0a5409dd-0a2a4505b638-10 for ; Thu, 02 Apr 2026 12:47:37 +0200 Received: from [209.85.218.51] (helo=mail-ej1-f51.google.com) by tlsNG-c201ff.mxtls.expurgate.net with ESMTPS (eXpurgate 4.56.0) (envelope-from ) id 69ce4949-3760-0a2a45050019-d155da33c8ee-3 for ; Thu, 02 Apr 2026 12:47:37 +0200 Received: by mail-ej1-f51.google.com with SMTP id a640c23a62f3a-b97e6e48b24so109055566b.2 for ; Thu, 02 Apr 2026 03:47:37 -0700 (PDT) Received: from EPUAKYIW02F7.. ([45.12.26.38]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-b9c3d028955sm76392366b.61.2026.04.02.03.47.35 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 02 Apr 2026 03:47:35 -0700 (PDT) X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" Authentication-Results: eu.smtp.expurgate.cloud; dkim=pass header.s=20251104 header.d=gmail.com header.i="@gmail.com" header.h="Content-Transfer-Encoding:MIME-Version:References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1775126857; x=1775731657; darn=lists.xenproject.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Wn2YE/N1073l2aBvkRQ/5nSVLZsSRqqVHeilL+wTO7s=; b=i476hIDYFm4HFhwljw3XbCq42pjM+Ftx25bW+E5ROSF0ZOBHy5bckqYeb9vMLbe5Mq E2xZoYvjukoeowBDIWrlL3enDj5XHGcUwxnWH3Psvg3B4MWZUHybgUrNWVC//xWUg/u4 O8waZZ1Jl6JzxQMAJiikWW7I7Pjfz2iHJOwi43c2Ful9L2DJhZLhEQl1PSqWEgtoC9KI REEHmSd7O0b5Xi1VVvH1pnn3JkNP1fSB4bmAb8bche0tCosJIy1DgLKrBQz9kDLLzHXA A6cSWnArndeqoUAfh3CKVpMiPewpufAqiKfNZ+tYC9S4PqI//d42Hela85Jols7f7rR7 +mBg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1775126857; x=1775731657; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=Wn2YE/N1073l2aBvkRQ/5nSVLZsSRqqVHeilL+wTO7s=; b=kbrlyWWDW3NVU5PQTeutWPAwx9pM+QX1ZflYsPnI0CT2s8DU2go050bUK8sckq2TDB OuGQKkzRKX35dBFwY9I4l1lKb5OksSaVmSB6jGsmhEi63N8/ddozVb5+Co14O4IEmTF+ p9g/coRPWqBGHFy53FM0JILVDsr+LoDAcLiAgQS2TaiBK6zTPlVvah388Gj9SEq1CQ+C cixCw0Ejbrr0SzZPeM3emJ3f+d4+HrwhWifo0IqpE9gCZWT1bLQ096AqCzzHTCW2wTV5 WBn9F4h2TE0RTDnVcftt8dVEIgsx4pw0x9AiaErKG8hwDWzPxvAWnSedpMLZhb8WScFH fnew== X-Gm-Message-State: AOJu0YwAmA+uoZy8ssgWoPlOSrewkOb9JGcd/JMVp9bd+F7hRe8Ue9bE WpROyS3tgTtU2fUot4egjnZixRv65rNwOcCr+K+4IjvxXc3cLmsuok/wWiiWR4Si X-Gm-Gg: ATEYQzzHkIkMKvnF/cyPy/hY/HlxnjnmqFi1tSjOQqHSyKbsBqGRla9QAOCmtbzYaKC 8bd82D26v0zR8Q5dhEbfLU+GP6JoBcVdjsNEvL33P1ABfX+U2tUW5xbS4kq5guXZaWLOtKgHccU L1BRzPuwSSd+qiJrPyHY2deRfsWc+1eb0dmkwDtyCL2p4TOXn+yNSBKkGxffCSq9URxAJhcsoMK o625QSAkrh3tZ0YQmiIpjEc2rp9NyG5Ua9WB3695v9E5m0eQu5gG38g2YWSTBEhxUajgB2wpa8E GmeIID5aa0U+pas56tVqgBeiFcceAAsPdP5zpjCYz9AxywKR/6X50UBocfsNM+uSvEw7r7AVd5m vTiqH2wGmVLEaTSFZ5LOKYPTNdADZiYfppM8Rd99dEpRLAMQmRZp+KWJA3Z5r18ueglkd0D07yP z/6O6hRDHWbqRhS8sGdJPOSyPrLakFzLY07Vjb X-Received: by 2002:a17:907:86a5:b0:b98:6984:6630 with SMTP id a640c23a62f3a-b9c137b3ccdmr462677366b.8.1775126856213; Thu, 02 Apr 2026 03:47:36 -0700 (PDT) From: Mykola Kvach To: xen-devel@lists.xenproject.org Cc: Mykola Kvach , Stefano Stabellini , Julien Grall , Bertrand Marquis , Michal Orzel , Volodymyr Babchuk Subject: [PATCH v8 10/13] xen/arm: Resume memory management on Xen resume Date: Thu, 2 Apr 2026 13:45:11 +0300 Message-ID: <02b55a02f329d09c9f8b15d97ca9176b3c33017a.1775125380.git.mykola_kvach@epam.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: References: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-purgate-ID: tlsNG-c201ff/1775126857-3959396F-ED9056AE/0/0 X-purgate-type: clean X-purgate-size: 1771 X-ZohoMail-DKIM: pass (identity @gmail.com) X-ZM-MESSAGEID: 1775126882649154100 Content-Type: text/plain; charset="utf-8" From: Mirela Simonovic The MMU must be enabled during the resume path before restoring context, as virtual addresses are used to access the saved context data. This patch adds MMU setup during resume by reusing the existing enable_secondary_cpu_mm function, which enables data cache and the MMU. Before the MMU is enabled, the content of TTBR0_EL2 is changed to point to init_ttbr (page tables used at runtime). Signed-off-by: Mirela Simonovic Signed-off-by: Saeed Nowshadi Signed-off-by: Mykyta Poturai Signed-off-by: Mykola Kvach --- Changes in v7: - no functional changes, just moved commit --- xen/arch/arm/arm64/head.S | 24 ++++++++++++++++++++++++ 1 file changed, 24 insertions(+) diff --git a/xen/arch/arm/arm64/head.S b/xen/arch/arm/arm64/head.S index 72c7b24498..596e960152 100644 --- a/xen/arch/arm/arm64/head.S +++ b/xen/arch/arm/arm64/head.S @@ -561,6 +561,30 @@ END(efi_xen_start) =20 #endif /* CONFIG_ARM_EFI */ =20 +#ifdef CONFIG_SYSTEM_SUSPEND + +FUNC(hyp_resume) + /* Initialize the UART if earlyprintk has been enabled. */ +#ifdef CONFIG_EARLY_PRINTK + bl init_uart +#endif + PRINT_ID("- Xen resuming -\r\n") + + bl check_cpu_mode + bl cpu_init + + ldr x0, =3Dstart + adr x20, start /* x20 :=3D paddr (start) */ + sub x20, x20, x0 /* x20 :=3D phys-offset */ + ldr lr, =3Dmmu_resumed + b enable_secondary_cpu_mm + +mmu_resumed: + b . +END(hyp_resume) + +#endif /* CONFIG_SYSTEM_SUSPEND */ + /* * Local variables: * mode: ASM --=20 2.43.0 From nobody Wed Apr 8 05:51:36 2026 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass(p=none dis=none) header.from=gmail.com ARC-Seal: i=1; a=rsa-sha256; t=1775126887; cv=none; d=zohomail.com; s=zohoarc; b=Z2CrsKojLRT9GSpIJ7muNihDtjQkMxeH5pPYrESAbgZJ5izFxazy6uOVmt8tQY1O3mFvzzoBX/HENM2prFjgJA2cmnPd8ndq2OL/VhyBLwig1izDYx22c4AHdOENjmvvvKLSD6c7HGHl6hxZ6cmqfT92i1FKlKgvsnpYoQAXhKY= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1775126887; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=zPqg0vXhWWH1w1hlbbZTkVgax5cKCfDJE6WmuksoUYo=; b=nxnvTtWyYOQ25unGOoM9LgB2TdzPImBqEuc/OHVGgAJNagXRJoiX+X9mxfhafB2VkLlGQlH4T7vhf5qAvG+7QxY5iElrURu8cMOKBYNw1ugq33fDhXvXNefcmDW7uilAWSBNRpq63YMUph2Muiz/8guEDVbDyAZ6fA2Znmtw6Y4= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1775126887254176.88999223322264; Thu, 2 Apr 2026 03:48:07 -0700 (PDT) Received: from list by lists.xenproject.org with outflank-mailman.1271537.1559723 (Exim 4.92) (envelope-from ) id 1w8FaP-0006U2-Gm; Thu, 02 Apr 2026 10:47:41 +0000 Received: by outflank-mailman (output) from mailman id 1271537.1559723; Thu, 02 Apr 2026 10:47:41 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1w8FaO-0006RL-IG; Thu, 02 Apr 2026 10:47:40 +0000 Received: by outflank-mailman (input) for mailman id 1271537; Thu, 02 Apr 2026 10:47:39 +0000 Received: from mx.expurgate.net ([195.190.135.10]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1w8FaM-00067Z-PE for xen-devel@lists.xenproject.org; Thu, 02 Apr 2026 10:47:38 +0000 Received: from mx.expurgate.net (helo=localhost) by mx.expurgate.net with esmtp id 1w8FaM-00Ej0u-4t for xen-devel@lists.xenproject.org; Thu, 02 Apr 2026 12:47:38 +0200 Received: from [10.42.69.5] (helo=localhost) by localhost with ESMTP (eXpurgate MTA 0.9.1) (envelope-from ) id 69ce4947-2eae-0a2a0a5409dd-0a2a4505b638-12 for ; Thu, 02 Apr 2026 12:47:38 +0200 Received: from [209.85.218.45] (helo=mail-ej1-f45.google.com) by tlsNG-c201ff.mxtls.expurgate.net with ESMTPS (eXpurgate 4.56.0) (envelope-from ) id 69ce494a-3760-0a2a45050019-d155da2da5b5-3 for ; Thu, 02 Apr 2026 12:47:38 +0200 Received: by mail-ej1-f45.google.com with SMTP id a640c23a62f3a-b9a0762ed5fso112580866b.1 for ; Thu, 02 Apr 2026 03:47:38 -0700 (PDT) Received: from EPUAKYIW02F7.. ([45.12.26.38]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-b9c3d028955sm76392366b.61.2026.04.02.03.47.36 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 02 Apr 2026 03:47:36 -0700 (PDT) X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" Authentication-Results: eu.smtp.expurgate.cloud; dkim=pass header.s=20251104 header.d=gmail.com header.i="@gmail.com" header.h="Content-Transfer-Encoding:MIME-Version:References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1775126857; x=1775731657; darn=lists.xenproject.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=zPqg0vXhWWH1w1hlbbZTkVgax5cKCfDJE6WmuksoUYo=; b=XzNNXBZ0XzWFnGtiBZ1yT3Mphevk23hj5Lzrx0nbE/kKI0Kd6lkbAs4bNnRgse3cOl BVX2smmpF5UqejCDojw+b3v3bpeLIeaY5poKd2LGN+m6EEg4Z6pa0q8RBmRePnMFvEjk xPqnhAhSeDJ5+FqKn3Cz6MOVJjOvuF5m5BTqNK0hJ/XJXgAcN0SQT5Y3odwjhO/h/HcM fiu6IeEWNTh4N2hh7AdxvXtwFkcw1XNH2ehMxtuE0lydLq3A0QtCRga98qdC4CwgEVu0 044j0x/l9N4kMyS9G4o18D4/ywWIYf/yQL9xB1xc743Mrj5EwRfwYiIdvZJGmIeqrjmW 00fg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1775126857; x=1775731657; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=zPqg0vXhWWH1w1hlbbZTkVgax5cKCfDJE6WmuksoUYo=; b=MFDW1hOZmdwkhYwFXb/iV4a3OsW+icPDVCZ71xYZxBYE87ZKFahTNlKruEVvRdmDD3 rNNvr2tYD9ZSDQSKKV2CmvBgzOGeuhas54bBXNOyf0szn84Suzq3OKAGfhfYnf7qHAuY L727wWKLIE5cYZCQuEYvuI+Xr4SfEb+TooOxQhpzpK+RiXylpkmLdI39MGYCd0Sec3nh 3BxPuNFORmr5Tss6Am0TUCxF6aOkD7ykJfRxleM1CVyfllUDOMKdt0Y60rsMmGZAqlf1 w9n59HD5nc2Z48tWqTB9PlUWhwO84RVMcr45llWE30eJOYKZmPPux3QBH3Lpkgdhe+++ IYvw== X-Gm-Message-State: AOJu0Yws0fxml572L6y7c59gdQbXlu4MXPTNWXHbpL0//AbjS+QDLrYa XxEOVG0Y7WeLFX8lITdx2Tu8nr4a+Hl+K9puPB9oiuPDZHkI6xCcKbvgUxEhEQrz X-Gm-Gg: ATEYQzwHtZUkKsviNXdX3X+dd2X1K67kgj3P/lbhGWi2Ft/T6uYs1m9yObm0zQ8NJdB ROY/tmdEMuNCjt0gyoIgvk7Eq6wiFws6ze24LrojsYWFOkd6teo+7Fz7qsCTsrYPR3GSAqU0f9v VOAuAXrzLm79qSLbdQnvqgNLe8+k33PX9Ndb8K4ekJBVVQUaJQITpcA/GiAbsqnfd4U7GglFJdF mY9ughY6Oo4wmZjh2ZoPDfNxftA6j1UV0KfODNK7V602yqFaJ0rywDd/2z9lptQSsDy/gY3DDOc JTvHiKiSqp9CvtHZrsOEGHQ9cPaqWPMLnTCGJL9KuEh42iS9TCYLOSy/CCoNl43K1qBjJ5W4DkD p5qO18Y0IpmjDetYwh5dYZz7NoEBueZdOnjDoaDIeiN5JAtqH8FU5Ln/lc18KjU6HhX+9uuvitm h8Fi1F+vMyYHoAohm0elXluPSNSg== X-Received: by 2002:a17:907:c22:b0:b98:f1d:6a63 with SMTP id a640c23a62f3a-b9c46e25d10mr98516666b.9.1775126857055; Thu, 02 Apr 2026 03:47:37 -0700 (PDT) From: Mykola Kvach To: xen-devel@lists.xenproject.org Cc: Mykola Kvach , Stefano Stabellini , Julien Grall , Bertrand Marquis , Michal Orzel , Volodymyr Babchuk Subject: [PATCH v8 11/13] xen/arm: Save/restore context on suspend/resume Date: Thu, 2 Apr 2026 13:45:12 +0300 Message-ID: X-Mailer: git-send-email 2.43.0 In-Reply-To: References: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-purgate-ID: tlsNG-c201ff/1775126858-3979296F-423BE752/0/0 X-purgate-type: clean X-purgate-size: 7252 X-ZohoMail-DKIM: pass (identity @gmail.com) X-ZM-MESSAGEID: 1775126888684154100 Content-Type: text/plain; charset="utf-8" From: Mirela Simonovic The context of CPU general purpose and system control registers must be saved on suspend and restored on resume. This is implemented in prepare_resume_ctx and before the return from the hyp_resume function. The prepare_resume_ctx must be invoked just before the PSCI system suspend call is issued to the ATF. The prepare_resume_ctx must return a non-zero value so that the calling 'if' statement evaluates to true, causing the system suspend to be invoked. Upon resume, the context saved on suspend will be restored, including the link register. Therefore, after restoring the context, the control flow will return to the address pointed to by the saved link register, which is the place from which prepare_resume_ctx was called. To ensure that the calling 'if' statement does not again evaluate to true and initiate system suspend, hyp_resume must return a zero value after restoring the context. Note that the order of saving register context into cpu_context structure must match the order of restoring. Support for ARM32 is not implemented. Instead, compilation fails with a build-time error if suspend is enabled for ARM32. Signed-off-by: Mirela Simonovic Signed-off-by: Saeed Nowshadi Signed-off-by: Mykyta Poturai Signed-off-by: Mykola Kvach --- Changes in v8: - fix alignments in code Changes in v7: - no changes --- xen/arch/arm/Makefile | 1 + xen/arch/arm/arm64/head.S | 90 +++++++++++++++++++++++++++++- xen/arch/arm/include/asm/suspend.h | 26 +++++++++ xen/arch/arm/suspend.c | 14 +++++ 4 files changed, 130 insertions(+), 1 deletion(-) create mode 100644 xen/arch/arm/suspend.c diff --git a/xen/arch/arm/Makefile b/xen/arch/arm/Makefile index 69200b2728..c36158271a 100644 --- a/xen/arch/arm/Makefile +++ b/xen/arch/arm/Makefile @@ -51,6 +51,7 @@ obj-y +=3D setup.o obj-y +=3D shutdown.o obj-y +=3D smp.o obj-y +=3D smpboot.o +obj-$(CONFIG_SYSTEM_SUSPEND) +=3D suspend.o obj-$(CONFIG_SYSCTL) +=3D sysctl.o obj-y +=3D time.o obj-y +=3D traps.o diff --git a/xen/arch/arm/arm64/head.S b/xen/arch/arm/arm64/head.S index 596e960152..2cb02ee314 100644 --- a/xen/arch/arm/arm64/head.S +++ b/xen/arch/arm/arm64/head.S @@ -562,6 +562,52 @@ END(efi_xen_start) #endif /* CONFIG_ARM_EFI */ =20 #ifdef CONFIG_SYSTEM_SUSPEND +/* + * int prepare_resume_ctx(struct cpu_context *ptr) + * + * x0 - pointer to the storage where callee's context will be saved + * + * CPU context saved here will be restored on resume in hyp_resume functio= n. + * prepare_resume_ctx shall return a non-zero value. Upon restoring context + * hyp_resume shall return value zero instead. From C code that invokes + * prepare_resume_ctx, the return value is interpreted to determine whether + * the context is saved (prepare_resume_ctx) or restored (hyp_resume). + */ +FUNC(prepare_resume_ctx) + /* Store callee-saved registers */ + stp x19, x20, [x0], #16 + stp x21, x22, [x0], #16 + stp x23, x24, [x0], #16 + stp x25, x26, [x0], #16 + stp x27, x28, [x0], #16 + stp x29, lr, [x0], #16 + + /* Store stack-pointer */ + mov x2, sp + str x2, [x0], #8 + + /* Store system control registers */ + mrs x2, VBAR_EL2 + str x2, [x0], #8 + mrs x2, VTCR_EL2 + str x2, [x0], #8 + mrs x2, VTTBR_EL2 + str x2, [x0], #8 + mrs x2, TPIDR_EL2 + str x2, [x0], #8 + mrs x2, MDCR_EL2 + str x2, [x0], #8 + mrs x2, HSTR_EL2 + str x2, [x0], #8 + mrs x2, CPTR_EL2 + str x2, [x0], #8 + mrs x2, HCR_EL2 + str x2, [x0], #8 + + /* prepare_resume_ctx must return a non-zero value */ + mov x0, #1 + ret +END(prepare_resume_ctx) =20 FUNC(hyp_resume) /* Initialize the UART if earlyprintk has been enabled. */ @@ -580,7 +626,49 @@ FUNC(hyp_resume) b enable_secondary_cpu_mm =20 mmu_resumed: - b . + /* Now we can access the cpu_context, so restore the context here = */ + ldr x0, =3Dcpu_context + + /* Restore callee-saved registers */ + ldp x19, x20, [x0], #16 + ldp x21, x22, [x0], #16 + ldp x23, x24, [x0], #16 + ldp x25, x26, [x0], #16 + ldp x27, x28, [x0], #16 + ldp x29, lr, [x0], #16 + + /* Restore stack pointer */ + ldr x2, [x0], #8 + mov sp, x2 + + /* Restore system control registers */ + ldr x2, [x0], #8 + msr VBAR_EL2, x2 + ldr x2, [x0], #8 + msr VTCR_EL2, x2 + ldr x2, [x0], #8 + msr VTTBR_EL2, x2 + ldr x2, [x0], #8 + msr TPIDR_EL2, x2 + ldr x2, [x0], #8 + msr MDCR_EL2, x2 + ldr x2, [x0], #8 + msr HSTR_EL2, x2 + ldr x2, [x0], #8 + msr CPTR_EL2, x2 + ldr x2, [x0], #8 + msr HCR_EL2, x2 + isb + + /* + * Since context is restored return from this function will appear + * as return from prepare_resume_ctx. To distinguish a return from + * prepare_resume_ctx which is called upon finalizing the suspend, + * as opposed to return from this function which executes on resum= e, + * we need to return zero value here. + */ + mov x0, #0 + ret END(hyp_resume) =20 #endif /* CONFIG_SYSTEM_SUSPEND */ diff --git a/xen/arch/arm/include/asm/suspend.h b/xen/arch/arm/include/asm/= suspend.h index 31a98a1f1b..c127fa3d78 100644 --- a/xen/arch/arm/include/asm/suspend.h +++ b/xen/arch/arm/include/asm/suspend.h @@ -3,6 +3,8 @@ #ifndef ARM_SUSPEND_H #define ARM_SUSPEND_H =20 +#include + struct domain; struct vcpu; struct vcpu_guest_context; @@ -14,6 +16,30 @@ struct resume_info { =20 void arch_domain_resume(struct domain *d); =20 +#ifdef CONFIG_SYSTEM_SUSPEND +#ifdef CONFIG_ARM_64 +struct cpu_context { + register_t callee_regs[12]; + register_t sp; + register_t vbar_el2; + register_t vtcr_el2; + register_t vttbr_el2; + register_t tpidr_el2; + register_t mdcr_el2; + register_t hstr_el2; + register_t cptr_el2; + register_t hcr_el2; +} __aligned(16); +#else +#error "Define cpu_context structure for arm32" +#endif + +extern struct cpu_context cpu_context; + +int prepare_resume_ctx(struct cpu_context *ptr); +void hyp_resume(void); +#endif /* CONFIG_SYSTEM_SUSPEND */ + #endif /* ARM_SUSPEND_H */ =20 /* diff --git a/xen/arch/arm/suspend.c b/xen/arch/arm/suspend.c new file mode 100644 index 0000000000..e38566b0b7 --- /dev/null +++ b/xen/arch/arm/suspend.c @@ -0,0 +1,14 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ + +#include + +struct cpu_context cpu_context =3D {}; + +/* + * Local variables: + * mode: C + * c-file-style: "BSD" + * c-basic-offset: 4 + * indent-tabs-mode: nil + * End: + */ --=20 2.43.0 From nobody Wed Apr 8 05:51:36 2026 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass(p=none dis=none) header.from=gmail.com ARC-Seal: i=1; a=rsa-sha256; t=1775126886; cv=none; d=zohomail.com; s=zohoarc; b=i754AjvisQGEKu0ScSUIGHxNv05d3BN3KLC2htw9coOMWKwSUwSJVq+duxE6CZi2BXvoPUl4WSbUm5BNxfDLSs3gcGVU7I8xpm/Q7HTeZYvqPmyJ9Nj9cuDKrTXa/xTPIJlQjXw7OD61Sxt9Ufgj76sGswCmKZ0azbX+palh+T4= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1775126886; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=81pZ/JS38wYFIEh/dUJCFhVaISUWS/A04krI0+LMlhc=; b=Nq6fEJoyhfrTqhDyz92wsyI6GTXy0D6lG4tCLRyqQr9MElXY2hMGv8KbvmA67ldR1K6Nq721QnF/zxuEpkepPeBaow5i93NarEh/bFPupA2USj7MVhPBLsb9vLiARLeKGZ5XqTop5FUaILPEXv5hgqyvsUP9d8S+WGGXGyp2jlE= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 177512688678789.20957540294478; Thu, 2 Apr 2026 03:48:06 -0700 (PDT) Received: from list by lists.xenproject.org with outflank-mailman.1271541.1559727 (Exim 4.92) (envelope-from ) id 1w8FaQ-0006iW-I0; Thu, 02 Apr 2026 10:47:42 +0000 Received: by outflank-mailman (output) from mailman id 1271541.1559727; Thu, 02 Apr 2026 10:47:42 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1w8FaQ-0006fr-2F; Thu, 02 Apr 2026 10:47:42 +0000 Received: by outflank-mailman (input) for mailman id 1271541; Thu, 02 Apr 2026 10:47:40 +0000 Received: from mx.expurgate.net ([195.190.135.10]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1w8FaN-0006KE-Vk for xen-devel@lists.xenproject.org; Thu, 02 Apr 2026 10:47:39 +0000 Received: from mx.expurgate.net (helo=localhost) by mx.expurgate.net with esmtp id 1w8FaN-009qA7-Bp for xen-devel@lists.xenproject.org; Thu, 02 Apr 2026 12:47:39 +0200 Received: from [10.42.69.10] (helo=localhost) by localhost with ESMTP (eXpurgate MTA 0.9.1) (envelope-from ) id 69ce493f-e002-0a2a0a5209dd-0a2a450a8a2e-32 for ; Thu, 02 Apr 2026 12:47:39 +0200 Received: from [209.85.218.47] (helo=mail-ej1-f47.google.com) by tlsNG-4011c0.mxtls.expurgate.net with ESMTPS (eXpurgate 4.56.0) (envelope-from ) id 69ce494b-ee98-0a2a450a0019-d155da2fc914-3 for ; Thu, 02 Apr 2026 12:47:39 +0200 Received: by mail-ej1-f47.google.com with SMTP id a640c23a62f3a-b9c3a9fe80fso90523266b.3 for ; Thu, 02 Apr 2026 03:47:39 -0700 (PDT) Received: from EPUAKYIW02F7.. ([45.12.26.38]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-b9c3d028955sm76392366b.61.2026.04.02.03.47.37 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 02 Apr 2026 03:47:37 -0700 (PDT) X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" Authentication-Results: eu.smtp.expurgate.cloud; dkim=pass header.s=20251104 header.d=gmail.com header.i="@gmail.com" header.h="Content-Transfer-Encoding:MIME-Version:References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1775126858; x=1775731658; darn=lists.xenproject.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=81pZ/JS38wYFIEh/dUJCFhVaISUWS/A04krI0+LMlhc=; b=SDOkkIyDwrY3ZrWFPgJIw6PL8GyiWY1v5MhVp/l8F+tUoVVT/nTNpetUO+QpqS1r7K zVp0FZBbGrtCJBMqBNCcvsowgw4zpY8/YUEW9LeDtnsBMtWUXeO41YE3oLdFMqGqHvt1 CP1Wya6R0lmaSa4W8qlwbgV7YGRiT5Uoug0MX+p/T/V5BeDhjvPrUP8hUHsriXHQWI89 zQXbF1otUr7OzWXseQR0+sbz18giN8vhqxhocfzc9F7XS8gynS+P19Wkv7non3zH2veW O4IpKpNz5B2Dp+I9e3i8/FpyhTxtC3mH2r/U+FuHjhbiKZGp5e71qaN+2byVyCSmvRDi NyZA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1775126858; x=1775731658; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=81pZ/JS38wYFIEh/dUJCFhVaISUWS/A04krI0+LMlhc=; b=jPQX7Jh21qUuiZbps5Ze29GE8Z71NPr2wQJ7qKKNDo2RFaY3Zh6l3MrMjDA5lhCIQ2 IuY9n3ZamtEm5zF87bN0HYqFS0zdj1rDk+zbBZeCDFtqlz9hqJlKdncZj5xM0OnUxRkH yqLQmxfB6NIiOik8wXuyI+U4i2ey6fzd/e/M7kh4qTYJFePPLd+AZE3DiDVEU6QhXYqI XvxGtU3nnmyK7BjsHd1oZ/HmWk9xMgGhVFJePsb2kbRtFr0e8fTKhQ5PobBALrWtkJWe qIoPbulXdNr5h1fJDW6dVJthPAsra0pX5wLBiLzQ2IvBJEQRAjk/nQGSorUSFf5VIPw3 4lEw== X-Gm-Message-State: AOJu0YyGkOx6Tne3XBgqnG7RjihFwrKo0JtI+eoHbceW/KVhBkeQHgkM si+F6AC9gVRA+PSPQAhbud76IBCKxt+hbWl0Y7Ho65J4BAx+NSMscCS7Uq4sy+It X-Gm-Gg: ATEYQzx9FrkaJ3Hf5CgmnbwtWadW6wmisEoY0mpleq6oJPr11SMkZvFMHzeNgxcuh7/ NBmatmBV9ALkWiZ9THJNtVIAjuK5KVMT+Cs89XM0E3XjqWk7RxXl73BVwNHBpRAUg22ZPCWSDbL VFkNcolwKBKfFcGTzSlfnsxAou71R8POyudW5RSwiY/OEpSexywPhAg81pesMKgjXpmDkX4G+JO eEwLy+8z59PQFwbJTcFF9X0r0utCFsL9SuZBIbTZuehvf5+WxDi6XGvhx5ATFIaP2WGstdonS2V 1Tayzk2mflimsLaIvEszKY42uNYwD+UhUa8migUBHB+ovhQJsnDWRiPkGpFNvFhEY3N4qiyu87W u25+z7xbSUIySgCzZzuWR0QQwSp5Mgs0gk2w+NNcvrtwGbd96E60otacDvOZEe8q5mA8JYAOIRl fGxAM+jhf2gQUFC23q0QIW5KoQGQ== X-Received: by 2002:a17:906:a50:b0:b98:8d3a:7d26 with SMTP id a640c23a62f3a-b9c138fa8bemr342137866b.15.1775126858181; Thu, 02 Apr 2026 03:47:38 -0700 (PDT) From: Mykola Kvach To: xen-devel@lists.xenproject.org Cc: Mykola Kvach , Stefano Stabellini , Julien Grall , Bertrand Marquis , Michal Orzel , Volodymyr Babchuk Subject: [PATCH v8 12/13] xen/arm: Implement PSCI SYSTEM_SUSPEND call (host interface) Date: Thu, 2 Apr 2026 13:45:13 +0300 Message-ID: <8767d4129cd5bc56086172e58320539be7da4059.1775125380.git.mykola_kvach@epam.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: References: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-purgate-ID: tlsNG-4011c0/1775126859-BE3510B1-A701E6B6/0/0 X-purgate-type: clean X-purgate-size: 2949 X-ZohoMail-DKIM: pass (identity @gmail.com) X-ZM-MESSAGEID: 1775126888638154100 Content-Type: text/plain; charset="utf-8" From: Mirela Simonovic Invoke PSCI SYSTEM_SUSPEND to finalize Xen's suspend sequence on ARM64 plat= forms. Pass the resume entry point (hyp_resume) as the first argument to EL3. The = resume handler is currently a stub and will be implemented later in assembly. Igno= re the context ID argument, as is done in Linux. Only enable this path when CONFIG_SYSTEM_SUSPEND is set and PSCI version is >=3D 1.0. Signed-off-by: Mirela Simonovic Signed-off-by: Saeed Nowshadi Signed-off-by: Mykyta Poturai Signed-off-by: Mykola Kvach --- Changes in v7: - no changes --- xen/arch/arm/include/asm/psci.h | 1 + xen/arch/arm/psci.c | 23 ++++++++++++++++++++++- 2 files changed, 23 insertions(+), 1 deletion(-) diff --git a/xen/arch/arm/include/asm/psci.h b/xen/arch/arm/include/asm/psc= i.h index 48a93e6b79..bb3c73496e 100644 --- a/xen/arch/arm/include/asm/psci.h +++ b/xen/arch/arm/include/asm/psci.h @@ -23,6 +23,7 @@ int call_psci_cpu_on(int cpu); void call_psci_cpu_off(void); void call_psci_system_off(void); void call_psci_system_reset(void); +int call_psci_system_suspend(void); =20 /* Range of allocated PSCI function numbers */ #define PSCI_FNUM_MIN_VALUE _AC(0,U) diff --git a/xen/arch/arm/psci.c b/xen/arch/arm/psci.c index b6860a7760..c9d126b195 100644 --- a/xen/arch/arm/psci.c +++ b/xen/arch/arm/psci.c @@ -17,17 +17,20 @@ #include #include #include +#include =20 /* * While a 64-bit OS can make calls with SMC32 calling conventions, for * some calls it is necessary to use SMC64 to pass or return 64-bit values. - * For such calls PSCI_0_2_FN_NATIVE(x) will choose the appropriate + * For such calls PSCI_*_FN_NATIVE(x) will choose the appropriate * (native-width) function ID. */ #ifdef CONFIG_ARM_64 #define PSCI_0_2_FN_NATIVE(name) PSCI_0_2_FN64_##name +#define PSCI_1_0_FN_NATIVE(name) PSCI_1_0_FN64_##name #else #define PSCI_0_2_FN_NATIVE(name) PSCI_0_2_FN32_##name +#define PSCI_1_0_FN_NATIVE(name) PSCI_1_0_FN32_##name #endif =20 uint32_t psci_ver; @@ -60,6 +63,24 @@ void call_psci_cpu_off(void) } } =20 +int call_psci_system_suspend(void) +{ +#ifdef CONFIG_SYSTEM_SUSPEND + struct arm_smccc_res res; + + if ( psci_ver < PSCI_VERSION(1, 0) ) + return PSCI_NOT_SUPPORTED; + + /* 2nd argument (context ID) is not used */ + arm_smccc_smc(PSCI_1_0_FN_NATIVE(SYSTEM_SUSPEND), __pa(hyp_resume), &r= es); + return PSCI_RET(res); +#else + dprintk(XENLOG_WARNING, + "SYSTEM_SUSPEND not supported (CONFIG_SYSTEM_SUSPEND disabled)= \n"); + return PSCI_NOT_SUPPORTED; +#endif +} + void call_psci_system_off(void) { if ( psci_ver > PSCI_VERSION(0, 1) ) --=20 2.43.0 From nobody Wed Apr 8 05:51:36 2026 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass(p=none dis=none) header.from=gmail.com ARC-Seal: i=1; a=rsa-sha256; t=1775126889; cv=none; d=zohomail.com; s=zohoarc; b=lybQKUXMjD2KlZt3+0UR2xqv70jAJNYxSDZnODFR3nFjbVlH0ckN2WFW7g7+IZ+9zpT/hS1nVMFDih9lCChDCRdc0hwX/Ey8LTBdoi8ayYhg37GfxHXLcrP2Tthzm1zfjrQ3oe/8S8cwENYdli+C0KgrFFwunIao+kA8cb771tQ= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1775126889; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=PNu7eJ7Z0b3llwWlHGEbTagm+9rNR0YOT8JLveV1JGo=; b=hISyu3KANEsVJEuDDTFLCkDaaUKbUYnDoKTuVXfCFbo3pfF2mQwiBokObX/4nzy6vDAdtWtL7/7/GcWxo+FnWCJftX28zmC6lkfP5ESQLtC550MJG9kQMj8kPC1jLBNWLAagzN5aorxDBWbnW+4oQiuS/e421M9vE9WLNG6ELE0= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1775126889757270.3211740919754; Thu, 2 Apr 2026 03:48:09 -0700 (PDT) Received: from list by lists.xenproject.org with outflank-mailman.1271546.1559732 (Exim 4.92) (envelope-from ) id 1w8FaR-0006uD-Ld; Thu, 02 Apr 2026 10:47:43 +0000 Received: by outflank-mailman (output) from mailman id 1271546.1559732; Thu, 02 Apr 2026 10:47:43 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1w8FaR-0006sC-4i; Thu, 02 Apr 2026 10:47:43 +0000 Received: by outflank-mailman (input) for mailman id 1271546; Thu, 02 Apr 2026 10:47:41 +0000 Received: from mx.expurgate.net ([195.190.135.10]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1w8FaP-0006V5-2R for xen-devel@lists.xenproject.org; Thu, 02 Apr 2026 10:47:41 +0000 Received: from mx.expurgate.net (helo=localhost) by mx.expurgate.net with esmtp id 1w8FaO-009qA7-Ef for xen-devel@lists.xenproject.org; Thu, 02 Apr 2026 12:47:40 +0200 Received: from [10.42.69.12] (helo=localhost) by localhost with ESMTP (eXpurgate MTA 0.9.1) (envelope-from ) id 69ce4948-e002-0a2a0a5209dd-0a2a450c8ce0-20 for ; Thu, 02 Apr 2026 12:47:40 +0200 Received: from [209.85.218.45] (helo=mail-ej1-f45.google.com) by tlsNG-d25034.mxtls.expurgate.net with ESMTPS (eXpurgate 4.56.0) (envelope-from ) id 69ce494c-f40c-0a2a450c0019-d155da2db009-3 for ; Thu, 02 Apr 2026 12:47:40 +0200 Received: by mail-ej1-f45.google.com with SMTP id a640c23a62f3a-b8f97c626aaso122290666b.2 for ; Thu, 02 Apr 2026 03:47:40 -0700 (PDT) Received: from EPUAKYIW02F7.. ([45.12.26.38]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-b9c3d028955sm76392366b.61.2026.04.02.03.47.38 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 02 Apr 2026 03:47:38 -0700 (PDT) X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" Authentication-Results: eu.smtp.expurgate.cloud; dkim=pass header.s=20251104 header.d=gmail.com header.i="@gmail.com" header.h="Content-Transfer-Encoding:MIME-Version:References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1775126860; x=1775731660; darn=lists.xenproject.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=PNu7eJ7Z0b3llwWlHGEbTagm+9rNR0YOT8JLveV1JGo=; b=KbGPLCldm1ssHRBbB1PjHxZV4J4qieBqYD1xDaTjCJl7e/yJ//qd0XEpJR2zRdSocS ZQAWJrK5m0bDoHP/uXSNJcZ8y/o6gJmTEVvTu1iTsOPqTkLYcZsBE1qgj9jxspDQz1L4 5krMpNBx/Gd8K4HFKZ8ZPKTMWHQvWIWY2lUQ0mz94SVgV5Y9KtvggPCt1vduNCVSArA7 D2CcmNA5n9YAR/z1kWbyR/THqBftvr5xnIb5ONiyndhVwgwBwfJ/HVqX1J2/HYAQDT4o rdP2ap/v7LySDZn7g7IDeOati7/MKyRR7OQtmNjg/L7A5ThJl5FWnsxAgaLZWhQLoHyE emCw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1775126860; x=1775731660; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=PNu7eJ7Z0b3llwWlHGEbTagm+9rNR0YOT8JLveV1JGo=; b=YNat4XG4aWdaWH4ArFu/kAToUf/pXRrJZBWnvxoTo3yNXtNgTF6COzGO6hMxrudAjo 9XnmXlU8NzIWR+1d7UDFUGXwY+ZTWoLIwKzZQ7HdWeH5Df+NtgQmw1JAGr4xHEnCg8H0 NsCOBggaawCAdWTIW9V01dPrfn0pUg+gqJgmJMYG+jABVuCohhK5miN684YQEBxa+FQ+ qKr43TcwMVR+h/r0QqZsyi7QcUUZ3JKcett6rysAREs70Gg8dcl6MSDjfoQH6Z2YIVG1 Zq1bcHr3nospWcJJdwLgCQ0s79hFaj5K0zypZWvHH6VcnPOWVp1N7pN3yaqtAb+TFuyQ TqDQ== X-Gm-Message-State: AOJu0YxOKPtSQmRzA9cWSgOCOVuPGrqOgFX3tsP4L/gnHgMf3BKVWXkG nV248zl47P3pUHLnGI0C3csMi6tRqugP+OUEHlZsmuzkowiBRRPBQzNQtnr26ejn X-Gm-Gg: ATEYQzxnLd9O2cxo0RJ9n88S8VczcX1T+qFGxTSocNYiZzM57/bHoO04XGNf9irtKk5 iq1T3rbl2HEeTyqfbdcQevZKcdSnYljRZLX+C/3LUoESMmYTdPMP65rEB+a49vV5ryxQza5syRA NV7xQbssNwLG5HWjwW5T8AuKQejS/m8z5cuFVOt34jjPq9KL++/SfGX44t6AAhtcPRFlLScTZEd sTQ1AhHPKxQFx9LmMJzpQZ0/Doh2QW4Be4zsdO4CK5VFbyzYNClDVDrJhrdXkiW7a58E036GS8T lQkez+/rnFNLoFu+raeKr5OI7UauNIvevYnFAaDLp6DpGz1gokEa7hQibHSrp81nHB8oovsAm15 jei8vtho8CTtJL67kCdxQmoNq3C2i+BMYmWJqGGlao7Vixow9bYdeV9asyVBnJKln7YjnCoVff9 Z/XQPh3KrD7aoGhsflb2EHg8z5d1CkHSynLZM1 X-Received: by 2002:a17:907:6d16:b0:b9c:4248:7773 with SMTP id a640c23a62f3a-b9c42487e4amr138809366b.26.1775126859321; Thu, 02 Apr 2026 03:47:39 -0700 (PDT) From: Mykola Kvach To: xen-devel@lists.xenproject.org Cc: Mykola Kvach , Stefano Stabellini , Julien Grall , Bertrand Marquis , Michal Orzel , Volodymyr Babchuk , Andrew Cooper , Anthony PERARD , Jan Beulich , =?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= , Rahul Singh Subject: [PATCH v8 13/13] xen/arm: Add support for system suspend triggered by hardware domain Date: Thu, 2 Apr 2026 13:45:14 +0300 Message-ID: <9b0f8922cd33df2b8e8038d639c1b8d8d73ce401.1775125380.git.mykola_kvach@epam.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: References: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-purgate-ID: tlsNG-d25034/1775126860-FFF42A3D-F1949B49/0/0 X-purgate-type: clean X-purgate-size: 14142 X-ZohoMail-DKIM: pass (identity @gmail.com) X-ZM-MESSAGEID: 1775126892780154100 Content-Type: text/plain; charset="utf-8" From: Mirela Simonovic Trigger Xen suspend when the hardware domain initiates suspend via SHUTDOWN_suspend. Redirect system suspend to CPU#0 to ensure the suspend logic runs on the boot CPU, as required. Introduce full suspend/resume infrastructure gated by CONFIG_SYSTEM_SUSPEND, including logic to: - disable and enable non-boot physical CPUs - freeze and thaw domains - suspend and resume the GIC, timer, iommu and console - maintain system state before and after suspend On boot, init_ttbr is normally initialized during secondary CPU hotplug. On uniprocessor systems, this would leave init_ttbr uninitialized, causing resume to fail. To address this, the boot CPU now sets init_ttbr during suspend. Remove the restriction in the vPSCI interface preventing suspend from the hardware domain. Select HAS_SYSTEM_SUSPEND for ARM_64. Introduce CONFIG_HAS_HWDOM_SYSTEM_SUSPEND as an architecture-selected capability for platforms where the hardware domain survives SHUTDOWN_suspend without hwdom_shutdown(). ARM_64 selects it with SYSTEM_SUSPEND enabled; other architectures keep the existing behaviour. Note: the code is behind CONFIG_HAS_SYSTEM_SUSPEND, which is currently only selected when UNSUPPORTED is set and when MPU isn't set, so the functionali= ty is built but disabled by default. Signed-off-by: Mirela Simonovic Signed-off-by: Saeed Nowshadi Signed-off-by: Mykyta Poturai Signed-off-by: Mykola Kvach --- Changes in V8: - Add a pre-suspend check in system_suspend() after scheduler_disable() to require all domains to be in the shut down state with SHUTDOWN_suspend before proceeding with the global suspend flow. - Drop the common-level depends on !ARM_64 || !SYSTEM_SUSPEND from CONFIG_HAS_HWDOM_SHUTDOWN_ON_SUSPEND and model the ARM64 suspend case with an arch-selected capability instead. - Rename CONFIG_HAS_HWDOM_SHUTDOWN_ON_SUSPEND to CONFIG_HAS_HWDOM_SYSTEM_SU= SPEND. - Rename need_hwdom_shutdown() to want_hwdom_shutdown(). Changes in V7: - Control domain is responsible for host suspend - Move the is_hardware_domain check into host_system_suspend() - Add an empty inline host_system_suspend() function when SYSTEM_SUSPEND config is disabled - Use IS_ENABLED() for config checking instead of #ifdef - Replace #ifdef checks in domain_shutdown() with IS_ENABLED() to simplify control flow. - Factor hardware domain shutdown condition into a helper (need_hwdom_shutdown()) to avoid preprocessor directives inside the funct= ion. - Squash with iommu suspend/resume commit --- xen/arch/arm/Kconfig | 2 + xen/arch/arm/include/asm/mm.h | 2 + xen/arch/arm/include/asm/suspend.h | 7 +- xen/arch/arm/mmu/smpboot.c | 2 +- xen/arch/arm/suspend.c | 181 +++++++++++++++++++++++++++++ xen/arch/arm/vpsci.c | 12 +- xen/common/Kconfig | 3 + xen/common/domain.c | 7 +- xen/drivers/passthrough/arm/smmu.c | 10 ++ 9 files changed, 220 insertions(+), 6 deletions(-) diff --git a/xen/arch/arm/Kconfig b/xen/arch/arm/Kconfig index 2f2b501fda..c2e63ce8ff 100644 --- a/xen/arch/arm/Kconfig +++ b/xen/arch/arm/Kconfig @@ -8,6 +8,8 @@ config ARM_64 depends on !ARM_32 select 64BIT select HAS_FAST_MULTIPLY + select HAS_HWDOM_SYSTEM_SUSPEND if SYSTEM_SUSPEND + select HAS_SYSTEM_SUSPEND if !MPU && UNSUPPORTED select HAS_VPCI_GUEST_SUPPORT if PCI_PASSTHROUGH =20 config ARM diff --git a/xen/arch/arm/include/asm/mm.h b/xen/arch/arm/include/asm/mm.h index 72a6928624..87b54a55dc 100644 --- a/xen/arch/arm/include/asm/mm.h +++ b/xen/arch/arm/include/asm/mm.h @@ -360,6 +360,8 @@ static inline void page_set_xenheap_gfn(struct page_inf= o *p, gfn_t gfn) } while ( (y =3D cmpxchg(&p->u.inuse.type_info, x, nx)) !=3D x ); } =20 +void set_init_ttbr(lpae_t *root); + #endif /* __ARCH_ARM_MM__ */ /* * Local variables: diff --git a/xen/arch/arm/include/asm/suspend.h b/xen/arch/arm/include/asm/= suspend.h index c127fa3d78..c36ba23b10 100644 --- a/xen/arch/arm/include/asm/suspend.h +++ b/xen/arch/arm/include/asm/suspend.h @@ -38,7 +38,12 @@ extern struct cpu_context cpu_context; =20 int prepare_resume_ctx(struct cpu_context *ptr); void hyp_resume(void); -#endif /* CONFIG_SYSTEM_SUSPEND */ +void host_system_suspend(struct domain *d); + +#else /* !CONFIG_SYSTEM_SUSPEND */ + +static inline void host_system_suspend(struct domain *d) { (void)d; } +#endif =20 #endif /* ARM_SUSPEND_H */ =20 diff --git a/xen/arch/arm/mmu/smpboot.c b/xen/arch/arm/mmu/smpboot.c index 37e91d72b7..ff508ecf40 100644 --- a/xen/arch/arm/mmu/smpboot.c +++ b/xen/arch/arm/mmu/smpboot.c @@ -72,7 +72,7 @@ static void clear_boot_pagetables(void) clear_table(boot_third); } =20 -static void set_init_ttbr(lpae_t *root) +void set_init_ttbr(lpae_t *root) { /* * init_ttbr is part of the identity mapping which is read-only. So diff --git a/xen/arch/arm/suspend.c b/xen/arch/arm/suspend.c index e38566b0b7..4d1289776b 100644 --- a/xen/arch/arm/suspend.c +++ b/xen/arch/arm/suspend.c @@ -1,9 +1,190 @@ /* SPDX-License-Identifier: GPL-2.0-only */ =20 +#include #include =20 +#include +#include +#include +#include +#include +#include +#include + struct cpu_context cpu_context =3D {}; =20 +static int can_system_suspend(void) +{ + int ret =3D 0; + struct domain *d; + + rcu_read_lock(&domlist_read_lock); + + for_each_domain ( d ) + { + bool domain_suspended; + + spin_lock(&d->shutdown_lock); + domain_suspended =3D d->is_shut_down && + d->shutdown_code =3D=3D SHUTDOWN_suspend; + spin_unlock(&d->shutdown_lock); + + if ( domain_suspended ) + continue; + + printk(XENLOG_ERR + "System suspend requires all domains to be shut down for su= spend (dom%d: isn't in suspend state)\n", + d->domain_id); + + ret =3D -EBUSY; + break; + } + + rcu_read_unlock(&domlist_read_lock); + + return ret; +} + +/* Xen suspend. data identifies the domain that initiated suspend. */ +static void system_suspend(void *data) +{ + int status; + unsigned long flags; + struct domain *d =3D (struct domain *)data; + + BUG_ON(system_state !=3D SYS_STATE_active); + + system_state =3D SYS_STATE_suspend; + + printk("Xen suspending...\n"); + + freeze_domains(); + scheduler_disable(); + + status =3D can_system_suspend(); + if ( status ) + { + system_state =3D SYS_STATE_resume; + goto resume_scheduler; + } + + /* + * Non-boot CPUs have to be disabled on suspend and enabled on resume + * (hotplug-based mechanism). Disabling non-boot CPUs will lead to PSCI + * CPU_OFF to be called by each non-boot CPU. Depending on the underly= ing + * platform capabilities, this may lead to the physical powering down = of + * CPUs. + */ + status =3D disable_nonboot_cpus(); + if ( status ) + { + system_state =3D SYS_STATE_resume; + goto resume_nonboot_cpus; + } + + time_suspend(); + + status =3D iommu_suspend(); + if ( status ) + { + system_state =3D SYS_STATE_resume; + goto resume_time; + } + + console_start_sync(); + status =3D console_suspend(); + if ( status ) + { + dprintk(XENLOG_ERR, "Failed to suspend the console, err=3D%d\n", s= tatus); + system_state =3D SYS_STATE_resume; + goto resume_end_sync; + } + + local_irq_save(flags); + status =3D gic_suspend(); + if ( status ) + { + system_state =3D SYS_STATE_resume; + goto resume_irqs; + } + + set_init_ttbr(xen_pgtable); + + /* + * Enable identity mapping before entering suspend to simplify + * the resume path + */ + update_boot_mapping(true); + + if ( prepare_resume_ctx(&cpu_context) ) + { + status =3D call_psci_system_suspend(); + /* + * If suspend is finalized properly by above system suspend PSCI c= all, + * the code below in this 'if' branch will never execute. Execution + * will continue from hyp_resume which is the hypervisor's resume = point. + * In hyp_resume CPU context will be restored and since link-regis= ter is + * restored as well, it will appear to return from prepare_resume_= ctx. + * The difference in returning from prepare_resume_ctx on system s= uspend + * versus resume is in function's return value: on suspend, the re= turn + * value is a non-zero value, on resume it is zero. That is why the + * control flow will not re-enter this 'if' branch on resume. + */ + if ( status ) + dprintk(XENLOG_WARNING, "PSCI system suspend failed, err=3D%d\= n", + status); + } + + system_state =3D SYS_STATE_resume; + update_boot_mapping(false); + + gic_resume(); + + resume_irqs: + local_irq_restore(flags); + + console_resume(); + resume_end_sync: + console_end_sync(); + + iommu_resume(); + + resume_time: + time_resume(); + + resume_nonboot_cpus: + /* + * The rcu_barrier() has to be added to ensure that the per cpu area is + * freed before a non-boot CPU tries to initialize it (_free_percpu_ar= ea() + * has to be called before the init_percpu_area()). This scenario occu= rs + * when non-boot CPUs are hot-unplugged on suspend and hotplugged on r= esume. + */ + rcu_barrier(); + enable_nonboot_cpus(); + + resume_scheduler: + scheduler_enable(); + thaw_domains(); + + system_state =3D SYS_STATE_active; + + printk("Resume (status %d)\n", status); + + domain_resume(d); +} + +static DECLARE_TASKLET(system_suspend_tasklet, system_suspend, NULL); + +void host_system_suspend(struct domain *d) +{ + system_suspend_tasklet.data =3D (void *)d; + /* + * The suspend procedure has to be finalized by the pCPU#0 (non-boot p= CPUs + * will be disabled during the suspend). + */ + tasklet_schedule_on_cpu(&system_suspend_tasklet, 0); +} + /* * Local variables: * mode: C diff --git a/xen/arch/arm/vpsci.c b/xen/arch/arm/vpsci.c index bd87ec430d..8fb9172186 100644 --- a/xen/arch/arm/vpsci.c +++ b/xen/arch/arm/vpsci.c @@ -5,6 +5,7 @@ =20 #include #include +#include #include #include #include @@ -232,8 +233,7 @@ static int32_t do_psci_1_0_system_suspend(register_t ep= oint, register_t cid) if ( is_64bit_domain(d) && is_thumb ) return PSCI_INVALID_ADDRESS; =20 - /* SYSTEM_SUSPEND is not supported for the hardware domain yet */ - if ( is_hardware_domain(d) ) + if ( !IS_ENABLED(CONFIG_SYSTEM_SUSPEND) && is_hardware_domain(d) ) return PSCI_NOT_SUPPORTED; =20 /* Ensure that all CPUs other than the calling one are offline */ @@ -266,6 +266,9 @@ static int32_t do_psci_1_0_system_suspend(register_t ep= oint, register_t cid) "SYSTEM_SUSPEND requested, epoint=3D%#"PRIregister", cid=3D%#"= PRIregister"\n", epoint, cid); =20 + if ( is_control_domain(d) ) + host_system_suspend(d); + return rc; } =20 @@ -290,7 +293,10 @@ static int32_t do_psci_1_0_features(uint32_t psci_func= _id) return 0; case PSCI_1_0_FN32_SYSTEM_SUSPEND: case PSCI_1_0_FN64_SYSTEM_SUSPEND: - return is_hardware_domain(current->domain) ? PSCI_NOT_SUPPORTED : = 0; + if ( IS_ENABLED(CONFIG_SYSTEM_SUSPEND) || + !is_hardware_domain(current->domain) ) + return 0; + fallthrough; default: return PSCI_NOT_SUPPORTED; } diff --git a/xen/common/Kconfig b/xen/common/Kconfig index 0a20aa0a12..feb1336f46 100644 --- a/xen/common/Kconfig +++ b/xen/common/Kconfig @@ -137,6 +137,9 @@ config HAS_EX_TABLE config HAS_FAST_MULTIPLY bool =20 +config HAS_HWDOM_SYSTEM_SUSPEND + bool + config HAS_IOPORTS bool =20 diff --git a/xen/common/domain.c b/xen/common/domain.c index bb9e210c28..d3edfb2a13 100644 --- a/xen/common/domain.c +++ b/xen/common/domain.c @@ -1375,6 +1375,11 @@ void __domain_crash(struct domain *d) domain_shutdown(d, SHUTDOWN_crash); } =20 +static inline bool want_hwdom_shutdown(uint8_t reason) +{ + return !IS_ENABLED(CONFIG_HAS_HWDOM_SYSTEM_SUSPEND) || + reason !=3D SHUTDOWN_suspend; +} =20 int domain_shutdown(struct domain *d, u8 reason) { @@ -1391,7 +1396,7 @@ int domain_shutdown(struct domain *d, u8 reason) d->shutdown_code =3D reason; reason =3D d->shutdown_code; =20 - if ( is_hardware_domain(d) ) + if ( is_hardware_domain(d) && want_hwdom_shutdown(reason) ) hwdom_shutdown(reason); =20 if ( d->is_shutting_down ) diff --git a/xen/drivers/passthrough/arm/smmu.c b/xen/drivers/passthrough/a= rm/smmu.c index 22d306d0cb..45f29ef8ec 100644 --- a/xen/drivers/passthrough/arm/smmu.c +++ b/xen/drivers/passthrough/arm/smmu.c @@ -2947,6 +2947,13 @@ static void arm_smmu_iommu_domain_teardown(struct do= main *d) xfree(xen_domain); } =20 +#ifdef CONFIG_SYSTEM_SUSPEND +static int arm_smmu_suspend(void) +{ + return -ENOSYS; +} +#endif + static const struct iommu_ops arm_smmu_iommu_ops =3D { .page_sizes =3D PAGE_SIZE_4K, .init =3D arm_smmu_iommu_domain_init, @@ -2960,6 +2967,9 @@ static const struct iommu_ops arm_smmu_iommu_ops =3D { .map_page =3D arm_iommu_map_page, .unmap_page =3D arm_iommu_unmap_page, .dt_xlate =3D arm_smmu_dt_xlate_generic, +#ifdef CONFIG_SYSTEM_SUSPEND + .suspend =3D arm_smmu_suspend, +#endif }; =20 static struct arm_smmu_device *find_smmu(const struct device *dev) --=20 2.43.0