From nobody Sun Dec 14 05:53:41 2025 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass(p=none dis=none) header.from=gmail.com ARC-Seal: i=1; a=rsa-sha256; t=1765478771; cv=none; d=zohomail.com; s=zohoarc; b=WX8QJglU1z+InENzqM43Nzb1H2/uqmhAXEK5UKVDv9O9E058t57gI+/ZLnZRiVdegulY+Mxl8xcx5CXBDmXd7swceMz4nAW9NcS3xK92LbJ6NNQ2i3xYy33oR5HPRM0SmAg4VflWedTOADCEcUXMk6Jmgobq8vbbUHa3L/0xkOU= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1765478771; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=tX1Fz0zPBQzk2eDKaQqJOs/xihDeP0j+jDiBR8XzM1w=; b=h1nk2TiCvXYQOSjiqJ+tYKEdAfYtaM01cUsa71h2/Y20VsdCtZyM1vlcjhHBqvD7TV5vl3mFnu1/h4ACE/4Ye13qJC4pNTBLQ3VDN31iJ3I7omLRgFcoamzTZI6K05ZAKWj0uWiqSO6YOC3f1cf5V+QqhyVybZKeTQNqk8F9ioY= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1765478771106825.1745494107464; Thu, 11 Dec 2025 10:46:11 -0800 (PST) Received: from list by lists.xenproject.org with outflank-mailman.1184545.1506954 (Exim 4.92) (envelope-from ) id 1vTlfj-0003IW-R6; Thu, 11 Dec 2025 18:45:51 +0000 Received: by outflank-mailman (output) from mailman id 1184545.1506954; Thu, 11 Dec 2025 18:45:51 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1vTlfj-0003IP-Nz; Thu, 11 Dec 2025 18:45:51 +0000 Received: by outflank-mailman (input) for mailman id 1184545; Thu, 11 Dec 2025 18:45:50 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1vTlfi-0003I8-Fb for xen-devel@lists.xenproject.org; Thu, 11 Dec 2025 18:45:50 +0000 Received: from mail-ej1-x62c.google.com (mail-ej1-x62c.google.com [2a00:1450:4864:20::62c]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id 9c700df4-d6c1-11f0-b15b-2bf370ae4941; Thu, 11 Dec 2025 19:45:49 +0100 (CET) Received: by mail-ej1-x62c.google.com with SMTP id a640c23a62f3a-b770f4accc0so77284166b.1 for ; Thu, 11 Dec 2025 10:45:49 -0800 (PST) Received: from EPUAKYIW02F7.. (pool185-5-253-4.as6723.net. [185.5.253.4]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-b7cfa5d0b0dsm345870466b.67.2025.12.11.10.45.46 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 11 Dec 2025 10:45:47 -0800 (PST) X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 9c700df4-d6c1-11f0-b15b-2bf370ae4941 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1765478748; x=1766083548; darn=lists.xenproject.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=tX1Fz0zPBQzk2eDKaQqJOs/xihDeP0j+jDiBR8XzM1w=; b=PEYCGL9+Lc2cDiL7DCeZT2XJ+aXRajwTTz+5n5r1AELWboYLg5NIxjTX+FQJpJ0wF7 JUIjuav4iZ4Dw6gjnvwRVm7Egxjo96FnFXvXHrCpXMB37X7/kW7myUTsctO+7fDXagj3 bhlxmVIZOGlxmzplc3KGkY7s1nR2FZBG0H+0f8PJBdYkzTlGRPclWETNaOs4ngR/s2TD mqNEcqQrW99fEm0l04Lq+o07aR8Wp3HAyMJJA5pm/YOgOr3hQY+4j46cM8Q5iPR9G3Ra BM92/TQsVnubz4bKNw3ePnZGdHbX39NzPDnVEkzBHUOItFt2figjR7qwYuO0x9iQbCIb O4Bg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1765478748; x=1766083548; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=tX1Fz0zPBQzk2eDKaQqJOs/xihDeP0j+jDiBR8XzM1w=; b=c/mjExmCXfk54moZ8BHEJYAFlCpGyE3/YfwnrvepRdW7fze1PPJUABwi31o+tOVFxP 9IrV3e6cV6e4zRb+ePsupfOPVndaRp/z2HO+Z2f5ESXZHQCmrlsDeQUtbE3/Gm78jUre NiM6IEKlSPiLwe9cIjYuCivyg+e8qiIJlKVza3seE35HM8WUfJS/Xiy9uXaLz1mkuTgg 3ZUjuMWJryfgWsV6ishnDYNcz8XiofwBNcVpThuR/MgFhp60Dga/M7FzqtKosGaLTaOs jHUv6n9IDvxtn1SUGhwvUMhC+ZmrP3iM2x5MO4P76avZOqOeWJ9buUJ3ciBymE1tqS07 qdPA== X-Gm-Message-State: AOJu0YwvYVAlvW9Vm0CvUsVELFtDj34Rf2ZZNpFStDx+511cflSL1v10 y7wNxuAhSxMt3sbrVgb2JP1ew/W/EiR1xd6ahds5JHMc0KMY7Gg4jSOPzYiPYLaQ X-Gm-Gg: AY/fxX4NGDPWvtpEUBz0nMrglyUqx91wMJJJiOp6KRy3EuF38BjQlg1urnEVscJl5tB 6z99nBim4rk5zgB/ejBs/D3wic2eZ/hJUgTSNF9dFgk8gngiHnba/adW6iUMmYZmHwdO8xkSTny MGIFgf0RNiMwv9SV0+tvYeEXhGJms0YYhyCtzRgQEQYhmRXdABFhALlQZDG06tW7ujPXgXIqEyU OyxltFiLcK2FZs5LI8vdtlY2eqznjaUo68llfcHcjrtFBF7LmgHabAU7q/keg4Qp1dd7rMt8ge4 wRnfvUkVTSD/cLhfL+a59pzAEtaK2zMUKUm9TdPu8D4t9Ekz/uwNrvWjazY3AqxbyV9ljckxxo1 bfqtQ07aiTPf2TEFC0Sel77F2Z1TisXDXQUNxoozVlpDWJEyD/zCJtltXrlFPgREROIdzxmS5z/ 8CKs3WHZBKp5dKXdPHknUj8/uJ1y9X8KOg70/F4f/6 X-Google-Smtp-Source: AGHT+IFtIgtxvZQd2O4KXziacvx35KTlgnY3AYtRsNoNXddsn3jOrLbe7ti6bJQfTJv3enbKUwDNPg== X-Received: by 2002:a17:907:9718:b0:b5c:66ce:bfe6 with SMTP id a640c23a62f3a-b7ce84d963amr692216066b.55.1765478748133; Thu, 11 Dec 2025 10:45:48 -0800 (PST) From: Mykola Kvach To: xen-devel@lists.xenproject.org Cc: Mykola Kvach , Mykola Kvach , Stefano Stabellini , Julien Grall , Bertrand Marquis , Michal Orzel , Volodymyr Babchuk , Saeed Nowshadi Subject: [PATCH v7 01/12] xen/arm: Add suspend and resume timer helpers Date: Thu, 11 Dec 2025 20:43:25 +0200 Message-ID: <6eb8f80c070f6b1198814df8d80dcc2e60d44ea8.1765472890.git.mykola_kvach@epam.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: References: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @gmail.com) X-ZM-MESSAGEID: 1765478772905158500 Content-Type: text/plain; charset="utf-8" From: Mirela Simonovic Timer interrupts must be disabled while the system is suspended to prevent spurious wake-ups. Suspending timers in Xen consists of disabling the physical timer and the hypervisor timer on the current CPU. The virtual timer does not need explicit handling here, as it is already disabled on vCPU context switch and its state is restored per-vCPU on the next context restore. Resuming consists of raising TIMER_SOFTIRQ, which prompts the generic timer code to reprogram the hypervisor timer with the correct timeout. Xen does not use or expose the physical timer, so it remains disabled across suspend/resume. Introduce a new helper, disable_phys_hyp_timers(), to encapsulate disabling of the physical and hypervisor timers. Signed-off-by: Mirela Simonovic Signed-off-by: Saeed Nowshadi Signed-off-by: Mykola Kvach --- Changes in V7: - Dropped EL1/EL2 wording; use "physical timer" and "hypervisor timer" - Renamed helper to disable_phys_hyp_timers() to reflect its actual scope - Clarified virtual timer handling (disabled on vCPU switch-out, restored= on context restore) and added comments in suspend/resume paths - Added resume comment explaining which timers are restored by TIMER_SOFT= IRQ --- xen/arch/arm/include/asm/time.h | 5 ++++ xen/arch/arm/time.c | 44 ++++++++++++++++++++++++++++----- 2 files changed, 43 insertions(+), 6 deletions(-) diff --git a/xen/arch/arm/include/asm/time.h b/xen/arch/arm/include/asm/tim= e.h index 49ad8c1a6d..f4fd0c6af5 100644 --- a/xen/arch/arm/include/asm/time.h +++ b/xen/arch/arm/include/asm/time.h @@ -108,6 +108,11 @@ void preinit_xen_time(void); =20 void force_update_vcpu_system_time(struct vcpu *v); =20 +#ifdef CONFIG_SYSTEM_SUSPEND +void time_suspend(void); +void time_resume(void); +#endif /* CONFIG_SYSTEM_SUSPEND */ + #endif /* __ARM_TIME_H__ */ /* * Local variables: diff --git a/xen/arch/arm/time.c b/xen/arch/arm/time.c index 3710eab109..b90dc6c68c 100644 --- a/xen/arch/arm/time.c +++ b/xen/arch/arm/time.c @@ -306,6 +306,14 @@ static void check_timer_irq_cfg(unsigned int irq, cons= t char *which) static DEFINE_PER_CPU_READ_MOSTLY(struct irqaction, irq_hyp); static DEFINE_PER_CPU_READ_MOSTLY(struct irqaction, irq_virt); =20 +/* Disable physical and hypervisor timers on the current CPU */ +static inline void disable_phys_hyp_timers(void) +{ + WRITE_SYSREG(0, CNTP_CTL_EL0); /* Physical timer disabled */ + WRITE_SYSREG(0, CNTHP_CTL_EL2); /* Hypervisor's timer disabled */ + isb(); +} + /* Set up the timer interrupt on this CPU */ void init_timer_interrupt(void) { @@ -316,9 +324,7 @@ void init_timer_interrupt(void) WRITE_SYSREG64(0, CNTVOFF_EL2); /* No VM-specific offset */ /* Do not let the VMs program the physical timer, only read the physic= al counter */ WRITE_SYSREG(CNTHCTL_EL2_EL1PCTEN, CNTHCTL_EL2); - WRITE_SYSREG(0, CNTP_CTL_EL0); /* Physical timer disabled */ - WRITE_SYSREG(0, CNTHP_CTL_EL2); /* Hypervisor's timer disabled */ - isb(); + disable_phys_hyp_timers(); =20 hyp_action->name =3D "hyptimer"; hyp_action->handler =3D htimer_interrupt; @@ -343,9 +349,7 @@ void init_timer_interrupt(void) */ static void deinit_timer_interrupt(void) { - WRITE_SYSREG(0, CNTP_CTL_EL0); /* Disable physical timer */ - WRITE_SYSREG(0, CNTHP_CTL_EL2); /* Disable hypervisor's timer */ - isb(); + disable_phys_hyp_timers(); =20 release_irq(timer_irq[TIMER_HYP_PPI], NULL); release_irq(timer_irq[TIMER_VIRT_PPI], NULL); @@ -385,6 +389,34 @@ void domain_set_time_offset(struct domain *d, int64_t = time_offset_seconds) /* XXX update guest visible wallclock time */ } =20 +#ifdef CONFIG_SYSTEM_SUSPEND + +void time_suspend(void) +{ + /* CNTV already disabled by virt_timer_save() during vcpu context swit= ch. */ + disable_phys_hyp_timers(); +} + +void time_resume(void) +{ + /* + * Raising TIMER_SOFTIRQ triggers generic timer code to reprogram the + * hypervisor timer with the correct timeout (not known here). + * + * Xen doesn't use or expose the physical timer, so it remains disabled + * across suspend/resume. + * + * The virtual timer state is restored per-vCPU on the next context sw= itch. + * + * No further action is needed to restore timekeeping after power down, + * since the system counter is unaffected. See ARM DDI 0487 L.a, D12.1= .2 + * "The system counter must be implemented in an always-on power domai= n." + */ + raise_softirq(TIMER_SOFTIRQ); +} + +#endif /* CONFIG_SYSTEM_SUSPEND */ + static int cpu_time_callback(struct notifier_block *nfb, unsigned long action, void *hcpu) --=20 2.43.0 From nobody Sun Dec 14 05:53:41 2025 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass(p=none dis=none) header.from=gmail.com ARC-Seal: i=1; a=rsa-sha256; t=1765478770; cv=none; d=zohomail.com; s=zohoarc; b=YVDKX+7PIYplRgMdKdRNUdaKa1i1UnXKxR4wjZc7Tcj9SfZ+HuLVlWGE6jRaDgaA6XlMQhBc0ZQ2+DHOOyxCya1bzmoFUHdVZzfHqSDVsuK7KS/bARknwG9d+tXeHc+6WVr11tO/7XehNUtp3LcnIGXrDZSojwDmYRzdGNWCsuk= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1765478770; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=MRsGxV9gXG/ZC0I4AbwNbd/2+lrn7b0mKg3wOgq8N30=; b=RbXg26MGtAB36o4xmYn8ccEjp5NXurrjR9pDHw1OHcFEH0N4OoWleehpnoy1bzubKAg9d6jp4YBj5Im6H666q7sixd9nHPW6gerZJX5wwD0NsEzbGeyNly4aP/l4PyaMmySi7V1HP04LB+Ozf6hYvNfCJbd2mt4k8WV9RNM7m+o= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1765478770807214.01495137510324; Thu, 11 Dec 2025 10:46:10 -0800 (PST) Received: from list by lists.xenproject.org with outflank-mailman.1184547.1506967 (Exim 4.92) (envelope-from ) id 1vTlfk-0003R0-HL; Thu, 11 Dec 2025 18:45:52 +0000 Received: by outflank-mailman (output) from mailman id 1184547.1506967; Thu, 11 Dec 2025 18:45:52 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1vTlfk-0003QY-AU; Thu, 11 Dec 2025 18:45:52 +0000 Received: by outflank-mailman (input) for mailman id 1184547; Thu, 11 Dec 2025 18:45:51 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1vTlfj-0003I8-9j for xen-devel@lists.xenproject.org; Thu, 11 Dec 2025 18:45:51 +0000 Received: from mail-ej1-x62c.google.com (mail-ej1-x62c.google.com [2a00:1450:4864:20::62c]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id 9d4f3a67-d6c1-11f0-b15b-2bf370ae4941; Thu, 11 Dec 2025 19:45:50 +0100 (CET) Received: by mail-ej1-x62c.google.com with SMTP id a640c23a62f3a-b727f452fffso71489766b.1 for ; Thu, 11 Dec 2025 10:45:50 -0800 (PST) Received: from EPUAKYIW02F7.. (pool185-5-253-4.as6723.net. [185.5.253.4]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-b7cfa5d0b0dsm345870466b.67.2025.12.11.10.45.48 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 11 Dec 2025 10:45:48 -0800 (PST) X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 9d4f3a67-d6c1-11f0-b15b-2bf370ae4941 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1765478750; x=1766083550; darn=lists.xenproject.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=MRsGxV9gXG/ZC0I4AbwNbd/2+lrn7b0mKg3wOgq8N30=; b=dixB5wl8ej22SEtNyazYU6wLrQaeyrDU9pmwBv8+s2mMJn3zCFWSkBL955w7JWR8R+ rlUyN5RuxzEqoeyxL2PztYgl03jTFX/RG0ldrBQJ3xoWKAlNT8zg1C4cxfckLBF123CQ wldsMInItTSgpv9F9ocDZOl5rBhSz1poZaS6XieZ7OGOVQUWOAFv2owEEmQvOnPL6r64 XsrMP0o0LrXXqUFNsrDB+2O1854iwSM/dtUmvvoAoNHsV5VLyypoUQdpdHQe6/FJal3m V3MhktAPWSAX/bFSK/+zZI3O+MCyo+P5bYLZYmy6IQPjtjCsePT0YEXOgz3C/pSGH09u cwDg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1765478750; x=1766083550; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=MRsGxV9gXG/ZC0I4AbwNbd/2+lrn7b0mKg3wOgq8N30=; b=LIkIyKn5D/1TLxcP80nmN0lzQ24DHvdtKaizuxofkYEJGoQD1zsthTZym1olzMBVMD sEscO4+p3AsgqoRwM/E1ED1WjI2zV1ZMKfj6pfVqUmEhgFd1wJD67ucpG6ogMOOTGvYt Q0AacwTL4+AkDILQcA1O9fDB2EvrjDgNPYmKn+yeEhBxBfLHQuyn7JbwqCT0z3hMUEdX npvxJCX0tWFZw0LF42S/t8ohDE6GaETbaRBLxkr5mGhT59N4XSPAuoBDLcLKWRtILTyX 2TQMcm++VyaAGWzP4Wnpb3pDNGGGBPABoxRaYUuTIBhgLEJ5WdX47BVuW2s1isRjNm9S FC+g== X-Gm-Message-State: AOJu0YwiUoMxaB2MtLzabWCFgcU1BnlFCeQObcpbFHQOXkDYKXJZmq9p TF+rVER4Yt7WfWb/ijtiSaYP+abqXq5HzDhkiYJyAQa3+yNXyqasu/HIH5n/8IfP X-Gm-Gg: AY/fxX5GFBNYeoSEZgHcIzkVwTlQ+Xb6JHOpoTeofgnUSrPdsTF9NcgLCsTxe0WhAXH VH26eO46JJbQtRw6yR1TERGeI6VVK69pTj4auY6+Y23cIPdvbrSLFapHhyjjgO6qzdLbcgB5fSs kCDP0ZcDKXzDa1fZq9KTZXezwevnpNIephvImt7EtKGI604Suh1u1hJoefia9xJGQID7coB7C8Q k0G38eYwUeOdZD/1ggc1/HGRHz4MO2jmQyju0qjDEpebaOWJhnAyX5g7mfJ7KPZIoT1hlQtLc/1 6vWuudZrj6iaUAcgZ+fIZ30b2x9CKKhpc0OdQlKC4hLhNiAy0N3Y7V5QFXWFOi4tRedWXxDV5of BTF9mFniKEp8igGqzYegYVoNG2nXl+CLqaMTEiBH+BWBSUCbCnJOxAypjBkLCVdhyMq3sMQJj5e fuwHhc5i5DBLClOoI0kKg19/hxIviwZt8hBEwdl6OZ X-Google-Smtp-Source: AGHT+IEAHYkNBQOlPEi8KqOjFKwI0oh4hz5zxz+lP2CuRzm2gBo3f1ff4xjR6uEUqdSIRULlHee29Q== X-Received: by 2002:a17:907:1c91:b0:b7a:71b7:2409 with SMTP id a640c23a62f3a-b7d02ae0bb0mr393064966b.15.1765478749417; Thu, 11 Dec 2025 10:45:49 -0800 (PST) From: Mykola Kvach To: xen-devel@lists.xenproject.org Cc: Mykola Kvach , Mykola Kvach , Stefano Stabellini , Julien Grall , Bertrand Marquis , Michal Orzel , Volodymyr Babchuk , Saeed Nowshadi , Mykyta Poturai Subject: [PATCH v7 02/12] xen/arm: gic-v2: Implement GIC suspend/resume functions Date: Thu, 11 Dec 2025 20:43:26 +0200 Message-ID: X-Mailer: git-send-email 2.43.0 In-Reply-To: References: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @gmail.com) X-ZM-MESSAGEID: 1765478772934158500 Content-Type: text/plain; charset="utf-8" From: Mirela Simonovic System suspend may lead to a state where GIC would be powered down. Therefore, Xen should save/restore the context of GIC on suspend/resume. Note that the context consists of states of registers which are controlled by the hypervisor. Other GIC registers which are accessible by guests are saved/restored on context switch. Signed-off-by: Mirela Simonovic Signed-off-by: Saeed Nowshadi Signed-off-by: Mykyta Poturai Signed-off-by: Mykola Kvach --- Changes in V7: - Allocate one contiguous memory block for the GICv2 dist suspend context. - gicv2_resume() no longer unconditionally re-enables the distributor/CPU i= nterface; it now writes back the saved CTLR values as-is. - gicv2_alloc_context() now returns 0 on success and panics on failure, sin= ce suspend context allocation is not recoverable. --- xen/arch/arm/gic-v2.c | 126 +++++++++++++++++++++++++++++++++ xen/arch/arm/gic.c | 29 ++++++++ xen/arch/arm/include/asm/gic.h | 12 ++++ 3 files changed, 167 insertions(+) diff --git a/xen/arch/arm/gic-v2.c b/xen/arch/arm/gic-v2.c index b23e72a3d0..0b2f7b3862 100644 --- a/xen/arch/arm/gic-v2.c +++ b/xen/arch/arm/gic-v2.c @@ -1098,6 +1098,123 @@ static int gicv2_iomem_deny_access(struct domain *d) return iomem_deny_access(d, mfn, mfn + nr); } =20 +#ifdef CONFIG_SYSTEM_SUSPEND + +/* This struct represent block of 32 IRQs */ +struct irq_block { + uint32_t icfgr[2]; /* 2 registers of 16 IRQs each */ + uint32_t ipriorityr[8]; + uint32_t isenabler; + uint32_t isactiver; + uint32_t itargetsr[8]; +}; + +/* GICv2 registers to be saved/restored on system suspend/resume */ +struct gicv2_context { + /* GICC context */ + struct cpu_ctx { + uint32_t ctlr; + uint32_t pmr; + uint32_t bpr; + } cpu; + + /* GICD context */ + struct dist_ctx { + uint32_t ctlr; + struct irq_block *irqs; + } dist; +}; + +static struct gicv2_context gic_ctx; + +static int gicv2_suspend(void) +{ + unsigned int i, blocks =3D DIV_ROUND_UP(gicv2_info.nr_lines, 32); + + /* Save GICC configuration */ + gic_ctx.cpu.ctlr =3D readl_gicc(GICC_CTLR); + gic_ctx.cpu.pmr =3D readl_gicc(GICC_PMR); + gic_ctx.cpu.bpr =3D readl_gicc(GICC_BPR); + + /* Save GICD configuration */ + gic_ctx.dist.ctlr =3D readl_gicd(GICD_CTLR); + + for ( i =3D 0; i < blocks; i++ ) + { + struct irq_block *irqs =3D gic_ctx.dist.irqs + i; + size_t j, off =3D i * sizeof(irqs->isenabler); + + irqs->isenabler =3D readl_gicd(GICD_ISENABLER + off); + irqs->isactiver =3D readl_gicd(GICD_ISACTIVER + off); + + off =3D i * sizeof(irqs->ipriorityr); + for ( j =3D 0; j < ARRAY_SIZE(irqs->ipriorityr); j++ ) + { + irqs->ipriorityr[j] =3D readl_gicd(GICD_IPRIORITYR + off + j *= 4); + irqs->itargetsr[j] =3D readl_gicd(GICD_ITARGETSR + off + j * 4= ); + } + + off =3D i * sizeof(irqs->icfgr); + for ( j =3D 0; j < ARRAY_SIZE(irqs->icfgr); j++ ) + irqs->icfgr[j] =3D readl_gicd(GICD_ICFGR + off + j * 4); + } + + return 0; +} + +static void gicv2_resume(void) +{ + unsigned int i, blocks =3D DIV_ROUND_UP(gicv2_info.nr_lines, 32); + + gicv2_cpu_disable(); + /* Disable distributor */ + writel_gicd(0, GICD_CTLR); + + for ( i =3D 0; i < blocks; i++ ) + { + struct irq_block *irqs =3D gic_ctx.dist.irqs + i; + size_t j, off =3D i * sizeof(irqs->isenabler); + + writel_gicd(0xffffffffU, GICD_ICENABLER + off); + writel_gicd(irqs->isenabler, GICD_ISENABLER + off); + + writel_gicd(0xffffffffU, GICD_ICACTIVER + off); + writel_gicd(irqs->isactiver, GICD_ISACTIVER + off); + + off =3D i * sizeof(irqs->ipriorityr); + for ( j =3D 0; j < ARRAY_SIZE(irqs->ipriorityr); j++ ) + { + writel_gicd(irqs->ipriorityr[j], GICD_IPRIORITYR + off + j * 4= ); + writel_gicd(irqs->itargetsr[j], GICD_ITARGETSR + off + j * 4); + } + + off =3D i * sizeof(irqs->icfgr); + for ( j =3D 0; j < ARRAY_SIZE(irqs->icfgr); j++ ) + writel_gicd(irqs->icfgr[j], GICD_ICFGR + off + j * 4); + } + + /* Make sure all registers are restored and enable distributor */ + writel_gicd(gic_ctx.dist.ctlr, GICD_CTLR); + + /* Restore GIC CPU interface configuration */ + writel_gicc(gic_ctx.cpu.pmr, GICC_PMR); + writel_gicc(gic_ctx.cpu.bpr, GICC_BPR); + + /* Enable GIC CPU interface */ + writel_gicc(gic_ctx.cpu.ctlr, GICC_CTLR); +} + +static void __init gicv2_alloc_context(void) +{ + uint32_t blocks =3D DIV_ROUND_UP(gicv2_info.nr_lines, 32); + + gic_ctx.dist.irqs =3D xzalloc_array(struct irq_block, blocks); + if ( !gic_ctx.dist.irqs ) + panic("Failed to allocate memory for GICv2 suspend context\n"); +} + +#endif /* CONFIG_SYSTEM_SUSPEND */ + #ifdef CONFIG_ACPI static unsigned long gicv2_get_hwdom_extra_madt_size(const struct domain *= d) { @@ -1302,6 +1419,11 @@ static int __init gicv2_init(void) =20 spin_unlock(&gicv2.lock); =20 +#ifdef CONFIG_SYSTEM_SUSPEND + /* Allocate memory to be used for saving GIC context during the suspen= d */ + gicv2_alloc_context(); +#endif /* CONFIG_SYSTEM_SUSPEND */ + return 0; } =20 @@ -1345,6 +1467,10 @@ static const struct gic_hw_operations gicv2_ops =3D { .map_hwdom_extra_mappings =3D gicv2_map_hwdom_extra_mappings, .iomem_deny_access =3D gicv2_iomem_deny_access, .do_LPI =3D gicv2_do_LPI, +#ifdef CONFIG_SYSTEM_SUSPEND + .suspend =3D gicv2_suspend, + .resume =3D gicv2_resume, +#endif /* CONFIG_SYSTEM_SUSPEND */ }; =20 /* Set up the GIC */ diff --git a/xen/arch/arm/gic.c b/xen/arch/arm/gic.c index ee75258fc3..7727ffed5a 100644 --- a/xen/arch/arm/gic.c +++ b/xen/arch/arm/gic.c @@ -432,6 +432,35 @@ int gic_iomem_deny_access(struct domain *d) return gic_hw_ops->iomem_deny_access(d); } =20 +#ifdef CONFIG_SYSTEM_SUSPEND + +int gic_suspend(void) +{ + /* Must be called by boot CPU#0 with interrupts disabled */ + ASSERT(!local_irq_is_enabled()); + ASSERT(!smp_processor_id()); + + if ( !gic_hw_ops->suspend || !gic_hw_ops->resume ) + return -ENOSYS; + + return gic_hw_ops->suspend(); +} + +void gic_resume(void) +{ + /* + * Must be called by boot CPU#0 with interrupts disabled after gic_sus= pend + * has returned successfully. + */ + ASSERT(!local_irq_is_enabled()); + ASSERT(!smp_processor_id()); + ASSERT(gic_hw_ops->resume); + + gic_hw_ops->resume(); +} + +#endif /* CONFIG_SYSTEM_SUSPEND */ + static int cpu_gic_callback(struct notifier_block *nfb, unsigned long action, void *hcpu) diff --git a/xen/arch/arm/include/asm/gic.h b/xen/arch/arm/include/asm/gic.h index 8e713aa477..8e8f4ac4c5 100644 --- a/xen/arch/arm/include/asm/gic.h +++ b/xen/arch/arm/include/asm/gic.h @@ -280,6 +280,12 @@ extern int gicv_setup(struct domain *d); extern void gic_save_state(struct vcpu *v); extern void gic_restore_state(struct vcpu *v); =20 +#ifdef CONFIG_SYSTEM_SUSPEND +/* Suspend/resume */ +extern int gic_suspend(void); +extern void gic_resume(void); +#endif /* CONFIG_SYSTEM_SUSPEND */ + /* SGI (AKA IPIs) */ enum gic_sgi { GIC_SGI_EVENT_CHECK, @@ -423,6 +429,12 @@ struct gic_hw_operations { int (*iomem_deny_access)(struct domain *d); /* Handle LPIs, which require special handling */ void (*do_LPI)(unsigned int lpi); +#ifdef CONFIG_SYSTEM_SUSPEND + /* Save GIC configuration due to the system suspend */ + int (*suspend)(void); + /* Restore GIC configuration due to the system resume */ + void (*resume)(void); +#endif /* CONFIG_SYSTEM_SUSPEND */ }; =20 extern const struct gic_hw_operations *gic_hw_ops; --=20 2.43.0 From nobody Sun Dec 14 05:53:41 2025 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass(p=none dis=none) header.from=gmail.com ARC-Seal: i=1; a=rsa-sha256; t=1765478777; cv=none; d=zohomail.com; s=zohoarc; b=LSjhH8/IpmKvXvO4pqxUdMtIciDoAx4OsDUmUy115yTlXi8EpuN/aPM6PaL0LFxhTPykBgDUoFUVBQF+gqIrjIyLIgpj8ge30n7wMRrP/6UG8aSsq8IjZO5fG7+AXX/4Losv4don9veTlZarlyVvchGH4nb5FrNwopyKNxP1/wQ= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1765478777; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=EhE7q8NSEMoDW1qyGPa35+oWlUcqnOapH/rPEjyoD/A=; b=N+X/upByW7b6Koi4jwh5ER0K/fMaSZ0aRC2osLt6e1VETmVaYosMvmOCzJ7RajNx9DguQ5/4DLmcasfQFFledxt7xOgiVFUUGhIvZdTbAD3f9IvnZ2OMiW7YBLZ734wkeZiCv1NE95ILXFJzivRAWUqB0MlFuYkD3bUPgOFF6L8= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1765478777625783.8074479217521; Thu, 11 Dec 2025 10:46:17 -0800 (PST) Received: from list by lists.xenproject.org with outflank-mailman.1184548.1506984 (Exim 4.92) (envelope-from ) id 1vTlfn-0003zT-Sg; Thu, 11 Dec 2025 18:45:55 +0000 Received: by outflank-mailman (output) from mailman id 1184548.1506984; Thu, 11 Dec 2025 18:45:55 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1vTlfn-0003zK-Ok; Thu, 11 Dec 2025 18:45:55 +0000 Received: by outflank-mailman (input) for mailman id 1184548; Thu, 11 Dec 2025 18:45:54 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1vTlfm-0003I9-3B for xen-devel@lists.xenproject.org; Thu, 11 Dec 2025 18:45:54 +0000 Received: from mail-ed1-x52e.google.com (mail-ed1-x52e.google.com [2a00:1450:4864:20::52e]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id 9e00d190-d6c1-11f0-9cce-f158ae23cfc8; Thu, 11 Dec 2025 19:45:51 +0100 (CET) Received: by mail-ed1-x52e.google.com with SMTP id 4fb4d7f45d1cf-6495c4577adso611014a12.3 for ; Thu, 11 Dec 2025 10:45:51 -0800 (PST) Received: from EPUAKYIW02F7.. (pool185-5-253-4.as6723.net. [185.5.253.4]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-b7cfa5d0b0dsm345870466b.67.2025.12.11.10.45.49 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 11 Dec 2025 10:45:50 -0800 (PST) X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 9e00d190-d6c1-11f0-9cce-f158ae23cfc8 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1765478751; x=1766083551; darn=lists.xenproject.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=EhE7q8NSEMoDW1qyGPa35+oWlUcqnOapH/rPEjyoD/A=; b=nf0yLhwri1OPMOZn9ELvUFlnbFmIMgiunksCSKPETK4r4I94B58PvYoCYozM5mDOw2 EgWVbsP50653DI03suGDI/RmMtZQrJLA8U8WdrFMpzthnYdNQkAp87Cv69IUYaSx2c1I wgGibAp2AS0403dgqdLg/Us+ATzj4ERiJDDncUNn9jyFfN8EHVOre6P9c8kiCgEYHhlZ B4mhbKeov254MWKfkGdONwHGwQPQNipr6Pk2PsWcaxK48wQSK7WsaFtLm5CpXTvheWXf rql+/4eKOTAEDWpiyrk50PPDMvf3eziWL0QhatF87EvmHaUh1Fq6q5HJ08iizDku3KFH fr2g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1765478751; x=1766083551; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=EhE7q8NSEMoDW1qyGPa35+oWlUcqnOapH/rPEjyoD/A=; b=F7EcjmRgdTvxdCfqUxJnvV2r27Pv2wtsgIXrPO96r61UYzS+Hsyp9DUduxIzpI1p1m aiOhm85M1a8weCA7uZZZNSSfOWhRPwW+3G+MVw2qkz6f+G/1sFQX1tF9qgB+YOUNC3RR uM/MPCkYqQ/ARnMGHqz4rIX8tTFkKajUTknyk46HLbCdDr3Wg+Q77Q5AetnEzgntZ1q8 rLLjQ5bwbgZpbjkENm7X8sPgc9zoFJ8hQPlKkF/aGQneGAv76spcxWvScNpXfA4A6mQQ iPCcjOMtqc3mmHaP6V7l6HvjYyCOMmF11R2C3GgC3dDp9R159/O9vm5jlokN/NQU5uZ4 6Q1Q== X-Gm-Message-State: AOJu0YyYwpwEbQqYRpYzcg54PWU7DwbWlDlawvk+u3IAD6Q6cUA++2Q2 AEb+wBi7OMgn38f9hDZCu/+J2+LCKGJJ1qDgSWBRI6wp7zZ+EeqESIMrGkYmKq9I X-Gm-Gg: AY/fxX50tEYlxEvAJXt4oHTwLN88F0e+AG5QCzlgIKB+KpHCVGQdhO3x8erZWsg8v1T Sn8CQcd1h1ACKNGPJmPM8rzqFOn3qhv1HFc5+B2M3WqOTZ4c4YlHEl5aM2IslJanVJWU1hQwfN0 lsU3ougqua5bZoK6cyOdP170TuOOH6Ndzj63/Ic5jhRxrJg+piw+w8Fh3djtj4Ds935te0T1YrL /1HUmLKWnC4auH3RvfVVCRtihYkhhXzvTe4AEeSYJch1yUJo/sKpZf9HbvNW4F+f2A6gOAVpqvF Cy2fSeR3e+3bW9M6JPLZscSp08DQoj5l0GrV2mRHcr1scOxuDnWJASUr7iDpsLKb0D0MsSKDZ7p HlixeboIMMVSgwF3l2CmOdCbhUvVsD0TOdxcObyshusZvDVaDtIcPdmz22jsFbZo96ji2D17BNg PzDnW3pQbAvyQ6cF9YDQDB0PL9SXZzCuO37puP5W/C X-Google-Smtp-Source: AGHT+IGcxLBqr3Vz1ok6ptsy7e2VcUY46QzsnsOsCXMZoVS/cKgBMR9zbKpr6lkMP6pdgI2BkNLDvA== X-Received: by 2002:a17:907:97d6:b0:b73:74d6:d360 with SMTP id a640c23a62f3a-b7ce8477b36mr771357466b.40.1765478750723; Thu, 11 Dec 2025 10:45:50 -0800 (PST) From: Mykola Kvach To: xen-devel@lists.xenproject.org Cc: Mykola Kvach , Mykola Kvach , Stefano Stabellini , Julien Grall , Bertrand Marquis , Michal Orzel , Volodymyr Babchuk Subject: [PATCH v7 03/12] xen/arm: gic-v3: Implement GICv3 suspend/resume functions Date: Thu, 11 Dec 2025 20:43:27 +0200 Message-ID: <9f084beff76e40fed2138ba2d59145a96b930c63.1765472890.git.mykola_kvach@epam.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: References: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @gmail.com) X-ZM-MESSAGEID: 1765478778654158500 Content-Type: text/plain; charset="utf-8" From: Mykola Kvach System suspend may lead to a state where GIC would be powered down. Therefore, Xen should save/restore the context of GIC on suspend/resume. Note that the context consists of states of registers which are controlled by the hypervisor. Other GIC registers which are accessible by guests are saved/restored on context switch. Signed-off-by: Mykola Kvach --- Changes in V7: - restore LPI regs on resume - add timeout during redist disabling - squash with suspend/resume handling for GICv3 eSPI registers - drop ITS guard paths so suspend/resume always runs; switch missing ctx allocation to panic - trim TODO comments; narrow redistributor storage to PPI icfgr - keep distributor context allocation even without ITS; adjust resume to use GENMASK(31, 0) for clearing enables - drop storage of the SGI configuration register, as SGIs are always edge-triggered --- xen/arch/arm/gic-v3-lpi.c | 3 + xen/arch/arm/gic-v3.c | 319 ++++++++++++++++++++++++- xen/arch/arm/include/asm/gic_v3_defs.h | 1 + 3 files changed, 320 insertions(+), 3 deletions(-) diff --git a/xen/arch/arm/gic-v3-lpi.c b/xen/arch/arm/gic-v3-lpi.c index de5052e5cf..61a6e18303 100644 --- a/xen/arch/arm/gic-v3-lpi.c +++ b/xen/arch/arm/gic-v3-lpi.c @@ -391,6 +391,9 @@ static int cpu_callback(struct notifier_block *nfb, uns= igned long action, switch ( action ) { case CPU_UP_PREPARE: + if ( system_state =3D=3D SYS_STATE_resume ) + break; + rc =3D gicv3_lpi_allocate_pendtable(cpu); if ( rc ) printk(XENLOG_ERR "Unable to allocate the pendtable for CPU%lu= \n", diff --git a/xen/arch/arm/gic-v3.c b/xen/arch/arm/gic-v3.c index bc07f97c16..dc5e58066d 100644 --- a/xen/arch/arm/gic-v3.c +++ b/xen/arch/arm/gic-v3.c @@ -1067,12 +1067,12 @@ out: return res; } =20 -static void gicv3_hyp_disable(void) +static void gicv3_hyp_enable(bool enable) { register_t hcr; =20 hcr =3D READ_SYSREG(ICH_HCR_EL2); - hcr &=3D ~GICH_HCR_EN; + hcr =3D enable ? (hcr | GICH_HCR_EN) : (hcr & ~GICH_HCR_EN); WRITE_SYSREG(hcr, ICH_HCR_EL2); isb(); } @@ -1179,7 +1179,7 @@ static void gicv3_disable_interface(void) spin_lock(&gicv3.lock); =20 gicv3_cpu_disable(); - gicv3_hyp_disable(); + gicv3_hyp_enable(false); =20 spin_unlock(&gicv3.lock); } @@ -1915,6 +1915,311 @@ static bool gic_dist_supports_lpis(void) return (readl_relaxed(GICD + GICD_TYPER) & GICD_TYPE_LPIS); } =20 +#ifdef CONFIG_SYSTEM_SUSPEND + +/* This struct represent block of 32 IRQs */ +struct dist_irq_block { + uint32_t icfgr[2]; + uint32_t ipriorityr[8]; + uint64_t irouter[32]; + uint32_t isactiver; + uint32_t isenabler; +}; + +struct redist_ctx { + uint32_t ctlr; + uint32_t icfgr; /* only PPIs stored */ + uint32_t igroupr; + uint32_t ipriorityr[8]; + uint32_t isactiver; + uint32_t isenabler; + + uint64_t pendbase; + uint64_t propbase; +}; + +/* GICv3 registers to be saved/restored on system suspend/resume */ +struct gicv3_ctx { + struct dist_ctx { + uint32_t ctlr; + struct dist_irq_block *irqs, *espi_irqs; + } dist; + + /* have only one rdist structure for last running CPU during suspend */ + struct redist_ctx rdist; + + struct cpu_ctx { + uint32_t ctlr; + uint32_t pmr; + uint32_t bpr; + uint32_t sre_el2; + uint32_t grpen; + } cpu; +}; + +static struct gicv3_ctx gicv3_ctx; + +static void __init gicv3_alloc_context(void) +{ + uint32_t blocks =3D DIV_ROUND_UP(gicv3_info.nr_lines, 32); + + /* The spec allows for systems without any SPIs */ + if ( blocks > 1 ) + { + gicv3_ctx.dist.irqs =3D xzalloc_array(struct dist_irq_block, block= s - 1); + if ( !gicv3_ctx.dist.irqs ) + panic("Failed to allocate memory for GICv3 suspend context\n"); + } + +#ifdef CONFIG_GICV3_ESPI + if ( !gic_number_espis() ) + return; + + blocks =3D gic_number_espis() / 32; + gicv3_ctx.dist.espi_irqs =3D xzalloc_array(struct dist_irq_block, bloc= ks); + if ( !gicv3_ctx.dist.espi_irqs ) + panic("Failed to allocate memory for GICv3 eSPI suspend context\n"= ); +#endif +} + +static int gicv3_disable_redist(void) +{ + void __iomem* waker =3D GICD_RDIST_BASE + GICR_WAKER; + s_time_t deadline; + + /* + * Avoid infinite loop if Non-secure does not have access to GICR_WAKE= R. + * See Arm IHI 0069H.b, 12.11.42 GICR_WAKER: + * When GICD_CTLR.DS =3D=3D 0 and an access is Non-secure accesses= to this + * register are RAZ/WI. + */ + if ( !(readl_relaxed(GICD + GICD_CTLR) & GICD_CTLR_DS) ) + return 0; + + deadline =3D NOW() + MILLISECS(1000); + + writel_relaxed(readl_relaxed(waker) | GICR_WAKER_ProcessorSleep, waker= ); + while ( (readl_relaxed(waker) & GICR_WAKER_ChildrenAsleep) =3D=3D 0 ) + { + if ( NOW() > deadline ) + { + printk("GICv3: Timeout waiting for redistributor to sleep\n"); + return -ETIMEDOUT; + } + cpu_relax(); + udelay(10); + } + + return 0; +} + +#define GET_SPI_REG_OFFSET(name, is_espi) \ + ((is_espi) ? GICD_##name##nE : GICD_##name) + +static void gicv3_store_spi_irq_block(struct dist_irq_block *irqs, + unsigned int i, bool is_espi) +{ + void __iomem *base; + unsigned int irq; + + base =3D GICD + GET_SPI_REG_OFFSET(ICFGR, is_espi) + i * sizeof(irqs->= icfgr); + irqs->icfgr[0] =3D readl_relaxed(base); + irqs->icfgr[1] =3D readl_relaxed(base + 4); + + base =3D GICD + GET_SPI_REG_OFFSET(IPRIORITYR, is_espi); + base +=3D i * sizeof(irqs->ipriorityr); + for ( irq =3D 0; irq < ARRAY_SIZE(irqs->ipriorityr); irq++ ) + irqs->ipriorityr[irq] =3D readl_relaxed(base + 4 * irq); + + base =3D GICD + GET_SPI_REG_OFFSET(IROUTER, is_espi); + base +=3D i * sizeof(irqs->irouter); + for ( irq =3D 0; irq < ARRAY_SIZE(irqs->irouter); irq++ ) + irqs->irouter[irq] =3D readq_relaxed_non_atomic(base + 8 * irq); + + base =3D GICD + GET_SPI_REG_OFFSET(ISACTIVER, is_espi); + base +=3D i * sizeof(irqs->isactiver); + irqs->isactiver =3D readl_relaxed(base); + + base =3D GICD + GET_SPI_REG_OFFSET(ISENABLER, is_espi); + base +=3D i * sizeof(irqs->isenabler); + irqs->isenabler =3D readl_relaxed(base); +} + +static void gicv3_restore_spi_irq_block(struct dist_irq_block *irqs, + unsigned int i, bool is_espi) +{ + void __iomem *base; + unsigned int irq; + + base =3D GICD + GET_SPI_REG_OFFSET(ICFGR, is_espi) + i * sizeof(irqs->= icfgr); + writel_relaxed(irqs->icfgr[0], base); + writel_relaxed(irqs->icfgr[1], base + 4); + + base =3D GICD + GET_SPI_REG_OFFSET(IPRIORITYR, is_espi); + base +=3D i * sizeof(irqs->ipriorityr); + for ( irq =3D 0; irq < ARRAY_SIZE(irqs->ipriorityr); irq++ ) + writel_relaxed(irqs->ipriorityr[irq], base + 4 * irq); + + base =3D GICD + GET_SPI_REG_OFFSET(IROUTER, is_espi); + base +=3D i * sizeof(irqs->irouter); + for ( irq =3D 0; irq < ARRAY_SIZE(irqs->irouter); irq++ ) + writeq_relaxed_non_atomic(irqs->irouter[irq], base + 8 * irq); + + base =3D GICD + GET_SPI_REG_OFFSET(ICENABLER, is_espi) + i * 4; + writel_relaxed(GENMASK(31, 0), base); + + base =3D GICD + GET_SPI_REG_OFFSET(ISENABLER, is_espi); + base +=3D i * sizeof(irqs->isenabler); + writel_relaxed(irqs->isenabler, base); + + base =3D GICD + GET_SPI_REG_OFFSET(ICACTIVER, is_espi) + i * 4; + writel_relaxed(GENMASK(31, 0), base); + + base =3D GICD + GET_SPI_REG_OFFSET(ISACTIVER, is_espi); + base +=3D i * sizeof(irqs->isactiver); + writel_relaxed(irqs->isactiver, base); +} + +static int gicv3_suspend(void) +{ + unsigned int i; + void __iomem *base; + int ret; + struct redist_ctx *rdist =3D &gicv3_ctx.rdist; + + /* Save GICC configuration */ + gicv3_ctx.cpu.ctlr =3D READ_SYSREG(ICC_CTLR_EL1); + gicv3_ctx.cpu.pmr =3D READ_SYSREG(ICC_PMR_EL1); + gicv3_ctx.cpu.bpr =3D READ_SYSREG(ICC_BPR1_EL1); + gicv3_ctx.cpu.sre_el2 =3D READ_SYSREG(ICC_SRE_EL2); + gicv3_ctx.cpu.grpen =3D READ_SYSREG(ICC_IGRPEN1_EL1); + + gicv3_disable_interface(); + + ret =3D gicv3_disable_redist(); + if ( ret ) + return out_enable_iface; + + /* Save GICR configuration */ + gicv3_redist_wait_for_rwp(); + + base =3D GICD_RDIST_SGI_BASE; + + rdist->ctlr =3D readl_relaxed(base + GICR_CTLR); + + /* Save priority on PPI and SGI interrupts */ + for ( i =3D 0; i < NR_GIC_LOCAL_IRQS / 4; i++ ) + rdist->ipriorityr[i] =3D readl_relaxed(base + GICR_IPRIORITYR0 + 4= * i); + + rdist->isactiver =3D readl_relaxed(base + GICR_ISACTIVER0); + rdist->isenabler =3D readl_relaxed(base + GICR_ISENABLER0); + rdist->igroupr =3D readl_relaxed(base + GICR_IGROUPR0); + rdist->icfgr =3D readl_relaxed(base + GICR_ICFGR1); + + rdist->propbase =3D readq_relaxed(base + GICR_PROPBASER); + rdist->pendbase =3D readq_relaxed(base + GICR_PENDBASER); + + /* Save GICD configuration */ + gicv3_dist_wait_for_rwp(); + gicv3_ctx.dist.ctlr =3D readl_relaxed(GICD + GICD_CTLR); + + for ( i =3D 1; i < DIV_ROUND_UP(gicv3_info.nr_lines, 32); i++ ) + gicv3_store_spi_irq_block(gicv3_ctx.dist.irqs + i - 1, i, false); + +#ifdef CONFIG_GICV3_ESPI + for ( i =3D 0; i < gic_number_espis() / 32; i++ ) + gicv3_store_spi_irq_block(gicv3_ctx.dist.espi_irqs + i, i, true); +#endif + + return 0; + + out_enable_iface: + gicv3_hyp_enable(true); + WRITE_SYSREG(gicv3_ctx.cpu.ctlr, ICC_CTLR_EL1); + isb(); + + return ret; +} + +static void gicv3_resume(void) +{ + int ret; + unsigned int i; + void __iomem *base; + struct redist_ctx *rdist =3D &gicv3_ctx.rdist; + + writel_relaxed(0, GICD + GICD_CTLR); + + for ( i =3D NR_GIC_LOCAL_IRQS; i < gicv3_info.nr_lines; i +=3D 32 ) + writel_relaxed(GENMASK(31, 0), GICD + GICD_IGROUPR + (i / 32) * 4); + + for ( i =3D 1; i < DIV_ROUND_UP(gicv3_info.nr_lines, 32); i++ ) + gicv3_restore_spi_irq_block(gicv3_ctx.dist.irqs + i - 1, i, false); + +#ifdef CONFIG_GICV3_ESPI + for ( i =3D 0; i < gic_number_espis() / 32; i++ ) + gicv3_restore_spi_irq_block(gicv3_ctx.dist.espi_irqs + i, i, true); +#endif + + writel_relaxed(gicv3_ctx.dist.ctlr, GICD + GICD_CTLR); + gicv3_dist_wait_for_rwp(); + + ret =3D gicv3_lpi_init_rdist(GICD_RDIST_BASE); + /* + * If LPIs are already enabled, assume firmware or the still-powered + * redistributor has valid PROPBASER/PENDBASER and skip reprogramming. + * Return -EBUSY so callers can ignore this case. + */ + if ( ret && ret !=3D -ENODEV && ret !=3D -EBUSY ) + panic("GICv3: Failed to re-initialize LPIs during resume\n"); + else if ( ret =3D=3D -EBUSY ) /* extra checks, just to be sure */ + { + base =3D GICD_RDIST_BASE; + if ( readq_relaxed(base + GICR_PROPBASER) !=3D rdist->propbase || + readq_relaxed(base + GICR_PENDBASER) !=3D rdist->pendbase ) + { + panic("GICv3: LPIs already enabled with unexpected PROPBASER/P= ENDBASER during resume\n"); + } + } + + /* Restore GICR (Redistributor) configuration */ + if ( gicv3_enable_redist() ) + panic("GICv3: Failed to re-enable redistributor during resume\n"); + + base =3D GICD_RDIST_SGI_BASE; + + writel_relaxed(GENMASK(31, 0), base + GICR_ICENABLER0); + gicv3_redist_wait_for_rwp(); + + for ( i =3D 0; i < NR_GIC_LOCAL_IRQS / 4; i++ ) + writel_relaxed(rdist->ipriorityr[i], base + GICR_IPRIORITYR0 + i *= 4); + + writel_relaxed(rdist->isactiver, base + GICR_ISACTIVER0); + writel_relaxed(rdist->igroupr, base + GICR_IGROUPR0); + writel_relaxed(rdist->icfgr, base + GICR_ICFGR1); + + gicv3_redist_wait_for_rwp(); + + writel_relaxed(rdist->isenabler, base + GICR_ISENABLER0); + writel_relaxed(rdist->ctlr, GICD_RDIST_BASE + GICR_CTLR); + + gicv3_redist_wait_for_rwp(); + + WRITE_SYSREG(gicv3_ctx.cpu.sre_el2, ICC_SRE_EL2); + isb(); + + /* Restore CPU interface (System registers) */ + WRITE_SYSREG(gicv3_ctx.cpu.pmr, ICC_PMR_EL1); + WRITE_SYSREG(gicv3_ctx.cpu.bpr, ICC_BPR1_EL1); + WRITE_SYSREG(gicv3_ctx.cpu.ctlr, ICC_CTLR_EL1); + WRITE_SYSREG(gicv3_ctx.cpu.grpen, ICC_IGRPEN1_EL1); + isb(); + + gicv3_hyp_init(); +} + +#endif /* CONFIG_SYSTEM_SUSPEND */ + /* Set up the GIC */ static int __init gicv3_init(void) { @@ -1989,6 +2294,10 @@ static int __init gicv3_init(void) =20 gicv3_hyp_init(); =20 +#ifdef CONFIG_SYSTEM_SUSPEND + gicv3_alloc_context(); +#endif + out: spin_unlock(&gicv3.lock); =20 @@ -2028,6 +2337,10 @@ static const struct gic_hw_operations gicv3_ops =3D { #endif .iomem_deny_access =3D gicv3_iomem_deny_access, .do_LPI =3D gicv3_do_LPI, +#ifdef CONFIG_SYSTEM_SUSPEND + .suspend =3D gicv3_suspend, + .resume =3D gicv3_resume, +#endif }; =20 static int __init gicv3_dt_preinit(struct dt_device_node *node, const void= *data) diff --git a/xen/arch/arm/include/asm/gic_v3_defs.h b/xen/arch/arm/include/= asm/gic_v3_defs.h index c373b94d19..992c8f9c2f 100644 --- a/xen/arch/arm/include/asm/gic_v3_defs.h +++ b/xen/arch/arm/include/asm/gic_v3_defs.h @@ -94,6 +94,7 @@ #define GICD_TYPE_LPIS (1U << 17) =20 #define GICD_CTLR_RWP (1UL << 31) +#define GICD_CTLR_DS (1U << 6) #define GICD_CTLR_ARE_NS (1U << 4) #define GICD_CTLR_ENABLE_G1A (1U << 1) #define GICD_CTLR_ENABLE_G1 (1U << 0) --=20 2.43.0 From nobody Sun Dec 14 05:53:41 2025 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass(p=none dis=none) header.from=gmail.com ARC-Seal: i=1; a=rsa-sha256; t=1765478783; cv=none; d=zohomail.com; s=zohoarc; b=R8bUaCE8Q7S85RDeTr+3A3v/VDozBBeAQMuPnbvywnWKVt51vsRwh0b1wdIch4D9ozapBAKUxsksGceVxgKq2eIYT050cree+6I67MtCj5NiwejYUTJ/J4VOPZ+OcPKvMGO5kZw/rAG7S/tcAz5jjHgI9bhj8rdYB4IIXgT94PI= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1765478783; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=WjTxMfuV2ry3Bt7T4zhRYcS+sdtazS8raHtB9fvxlRg=; b=BiG6uLP+yFXNfmgt4jBqrspzLcVUj6LwWvf+hOfiawitVWmw2yo3dVL4rECTXDyBmdCs6za/1+oVxwex5C34a7fsIhjDlrmM4QBZeoEDHBH43MMArqLe9xTdchKGZDCqZxM5PTwjH1WpjIM2+Qx0PzS6viWASLBRwXabxRk2J6A= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1765478783795378.7528173028197; Thu, 11 Dec 2025 10:46:23 -0800 (PST) Received: from list by lists.xenproject.org with outflank-mailman.1184550.1507004 (Exim 4.92) (envelope-from ) id 1vTlfp-0004QG-Jn; Thu, 11 Dec 2025 18:45:57 +0000 Received: by outflank-mailman (output) from mailman id 1184550.1507004; Thu, 11 Dec 2025 18:45:57 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1vTlfp-0004Pq-D3; Thu, 11 Dec 2025 18:45:57 +0000 Received: by outflank-mailman (input) for mailman id 1184550; Thu, 11 Dec 2025 18:45:55 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1vTlfn-0003I9-3E for xen-devel@lists.xenproject.org; Thu, 11 Dec 2025 18:45:55 +0000 Received: from mail-ej1-x62f.google.com (mail-ej1-x62f.google.com [2a00:1450:4864:20::62f]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id 9ec41371-d6c1-11f0-9cce-f158ae23cfc8; Thu, 11 Dec 2025 19:45:53 +0100 (CET) Received: by mail-ej1-x62f.google.com with SMTP id a640c23a62f3a-b73161849e1so115603166b.2 for ; Thu, 11 Dec 2025 10:45:53 -0800 (PST) Received: from EPUAKYIW02F7.. (pool185-5-253-4.as6723.net. [185.5.253.4]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-b7cfa5d0b0dsm345870466b.67.2025.12.11.10.45.50 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 11 Dec 2025 10:45:51 -0800 (PST) X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 9ec41371-d6c1-11f0-9cce-f158ae23cfc8 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1765478752; x=1766083552; darn=lists.xenproject.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=WjTxMfuV2ry3Bt7T4zhRYcS+sdtazS8raHtB9fvxlRg=; b=bSK4HOzoiDQFYjIoVqXY8kgjJzSpETDmYnfBEXRZvNANGQKTm9p0ywKqG6wUyFDTqS wBDtJSeHow1HjXEcDSyoeZvBNAIXwqPq61HriEpCSPxmsrLCQZVSjoo5KqLJt1GbgeXw TsLRJmHYbJlSR8R5GjwsqTzDiTK3Rahk2+LIZhVbwaW7lxCv4TGecUN6CSgkxIAfEx/7 KaoTQ02/8T28EvCL+XR/kn4+18vqWAbOjbgU/k97Db5bDfYnRE2XnkJZYDI56HSvKo76 UszKHjSy4dkLt3v6ubejN6jnP2ps4z/h5zJeg0L8gM6DYG76Oh1fedP0xAlKbRbRso8Z SC3w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1765478752; x=1766083552; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=WjTxMfuV2ry3Bt7T4zhRYcS+sdtazS8raHtB9fvxlRg=; b=KVOP8djSmzo9sUmMBYDQlvz0PxSDlCmVenS4PBmhEw1mGWrgrDtmj7cgekm71akmQ/ ZjGzjQ4UynoIO5kHOvjkmwvaduXdBIhSg3NB43q2U0U6hTwkbDv6/flrqSoHCyTkWXBZ ULcBSoyddL73223FegwpQV51cgi5OYbVXQcbfnRYq5woCM0iWBbBGJrcsr3p+FdHv3Kz YR7B/ShCP0Ij745lBcDoQfQotCLHrOU4IK/pX/GijIVX/h3Sqf2Qi+ZBNh2BcZOtVtOu 1Zghxh1uturvqe02uordyiqX4k3w8TcH3AIBANseOAYkdfWWdFbAX/pb4Ow+PIfMmubc xmPg== X-Gm-Message-State: AOJu0Yy2ktTjTdAGe2B51dqscPixiUu3KJ2OOH+remT9PaGJML79Emyp v3F39tg9vgCqr17Gz80Kv8xrxsqf0Nr2/toVYRnhKPQALs3TFk+G6bX1jMVj6v6b X-Gm-Gg: AY/fxX7DhzUGnnkieP3XiI24XtyGUa5CWywMOmm50EWvBX58bND16ZBqFnK9N9eFw0V cW5zmbpWUTmdVLZXd+JPewGNguKGH0UCMLgejP0B0f2BaEXWlDXwba2G7d5anyIFT24hMrAgtuH O2xSxWVRv/PC7nxI5BGb499NL9mL2U/n4GJ0egaHhaTdCW2y6eK0oNx6w/yczoJmnRCHw/zHl8O F77k1R3P0q3Lk7nBXhAGBjkRgGhiT75wDB7qcSgEkUU+HlrA2ia1Z4bS8jXyjRouJXVjo4QD+uk zUSZXYjGrhuuhsw9FvJkfxXXn7bN6kCM5iwoqD+nGdUSnM/F6bKVcx1+q5DmTCk+qRzpbqj16z6 u6oBiNts5SB63xcq/JVOD/YylnSCBsH7LD5ugT7roF+1D7lGp4n9LDvRbIuG/ZLS2oTDLlSLvBL VW+et9th2N0mIcbAEzpVTP+lEnmfThvQQhgGLJLVN8 X-Google-Smtp-Source: AGHT+IFjRICZYFD+6bCjcyCTiWc0ssNriVYuF6uCkR9q+ZlpE+Pm25lA1i4Aqp+4yHS6qOgburziRw== X-Received: by 2002:a17:907:9282:b0:b79:cef7:6574 with SMTP id a640c23a62f3a-b7ce822f461mr839551466b.4.1765478751978; Thu, 11 Dec 2025 10:45:51 -0800 (PST) From: Mykola Kvach To: xen-devel@lists.xenproject.org Cc: Mykola Kvach , Mykola Kvach , Stefano Stabellini , Julien Grall , Bertrand Marquis , Michal Orzel , Volodymyr Babchuk , Andrew Cooper , Anthony PERARD , Jan Beulich , =?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= Subject: [PATCH v7 04/12] xen/arm: gic-v3: add ITS suspend/resume support Date: Thu, 11 Dec 2025 20:43:28 +0200 Message-ID: <2fade2b96128053fbe3ed59f1d5e3444b32b96c3.1765472890.git.mykola_kvach@epam.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: References: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @gmail.com) X-ZM-MESSAGEID: 1765478784633158500 Content-Type: text/plain; charset="utf-8" From: Mykola Kvach Handle system suspend/resume for GICv3 with an ITS present so LPIs keep working after firmware powers the GIC down. Snapshot the CPU interface, distributor and last-CPU redistributor state, disable the ITS to cache its CTLR/CBASER/BASER registers, then restore everything and re-arm the collection on resume. Add list_for_each_entry_continue_reverse() in list.h for the ITS suspend error path that needs to roll back partially saved state. Based on Linux commit: dba0bc7b76dc ("irqchip/gic-v3-its: Add ability to sa= ve/restore ITS state") Signed-off-by: Mykola Kvach --- xen/arch/arm/gic-v3-its.c | 91 +++++++++++++++++++++++++++ xen/arch/arm/gic-v3.c | 15 ++++- xen/arch/arm/include/asm/gic_v3_its.h | 23 +++++++ xen/include/xen/list.h | 14 +++++ 4 files changed, 140 insertions(+), 3 deletions(-) diff --git a/xen/arch/arm/gic-v3-its.c b/xen/arch/arm/gic-v3-its.c index 34833166ad..08a3d8d1ef 100644 --- a/xen/arch/arm/gic-v3-its.c +++ b/xen/arch/arm/gic-v3-its.c @@ -1209,6 +1209,97 @@ int gicv3_its_init(void) return 0; } =20 +#ifdef CONFIG_SYSTEM_SUSPEND +int gicv3_its_suspend(void) +{ + struct host_its *its; + int ret; + + list_for_each_entry(its, &host_its_list, entry) + { + unsigned int i; + void __iomem *base =3D its->its_base; + + its->suspend_ctx.ctlr =3D readl_relaxed(base + GITS_CTLR); + ret =3D gicv3_disable_its(its); + if ( ret ) + { + writel_relaxed(its->suspend_ctx.ctlr, base + GITS_CTLR); + goto err; + } + + its->suspend_ctx.cbaser =3D readq_relaxed(base + GITS_CBASER); + + for (i =3D 0; i < GITS_BASER_NR_REGS; i++) { + uint64_t baser =3D readq_relaxed(base + GITS_BASER0 + i * 8); + + if ( !(baser & GITS_VALID_BIT) ) + continue; + + its->suspend_ctx.baser[i] =3D baser; + } + } + + return 0; + + err: + list_for_each_entry_continue_reverse(its, &host_its_list, entry) + writel_relaxed(its->suspend_ctx.ctlr, its->its_base + GITS_CTLR); + + return ret; +} + +void gicv3_its_resume(void) +{ + struct host_its *its; + int ret; + + list_for_each_entry(its, &host_its_list, entry) + { + void __iomem *base; + unsigned int i; + + base =3D its->its_base; + + /* + * Make sure that the ITS is disabled. If it fails to quiesce, + * don't restore it since writing to CBASER or BASER + * registers is undefined according to the GIC v3 ITS + * Specification. + * + * Firmware resuming with the ITS enabled is terminally broken. + */ + WARN_ON(readl_relaxed(base + GITS_CTLR) & GITS_CTLR_ENABLE); + ret =3D gicv3_disable_its(its); + if ( ret ) + continue; + + writeq_relaxed(its->suspend_ctx.cbaser, base + GITS_CBASER); + + /* + * Writing CBASER resets CREADR to 0, so make CWRITER and + * cmd_write line up with it. + */ + writeq_relaxed(0, base + GITS_CWRITER); + + /* Restore GITS_BASER from the value cache. */ + for (i =3D 0; i < GITS_BASER_NR_REGS; i++) { + uint64_t baser =3D its->suspend_ctx.baser[i]; + + if ( !(baser & GITS_VALID_BIT) ) + continue; + + writeq_relaxed(baser, base + GITS_BASER0 + i * 8); + } + writel_relaxed(its->suspend_ctx.ctlr, base + GITS_CTLR); + } + + ret =3D gicv3_its_setup_collection(smp_processor_id()); + if ( ret ) + panic("GICv3: ITS: resume setup collection failed: %d\n", ret); +} + +#endif /* CONFIG_SYSTEM_SUSPEND */ =20 /* * Local variables: diff --git a/xen/arch/arm/gic-v3.c b/xen/arch/arm/gic-v3.c index dc5e58066d..cde76b182f 100644 --- a/xen/arch/arm/gic-v3.c +++ b/xen/arch/arm/gic-v3.c @@ -857,7 +857,7 @@ static bool gicv3_enable_lpis(void) return true; } =20 -static int __init gicv3_populate_rdist(void) +static int gicv3_populate_rdist(void) { int i; uint32_t aff; @@ -927,7 +927,7 @@ static int __init gicv3_populate_rdist(void) ret =3D gicv3_lpi_init_rdist(ptr); if ( ret && ret !=3D -ENODEV ) { - printk("GICv3: CPU%d: Cannot initialize LPIs: %u\n= ", + printk("GICv3: CPU%d: Cannot initialize LPIs: %d\n= ", smp_processor_id(), ret); break; } @@ -2096,9 +2096,13 @@ static int gicv3_suspend(void) =20 gicv3_disable_interface(); =20 + ret =3D gicv3_its_suspend(); + if ( ret ) + goto out_enable_iface; + ret =3D gicv3_disable_redist(); if ( ret ) - return out_enable_iface; + goto out_its_resume; =20 /* Save GICR configuration */ gicv3_redist_wait_for_rwp(); @@ -2133,6 +2137,9 @@ static int gicv3_suspend(void) =20 return 0; =20 + out_its_resume: + gicv3_its_resume(); + out_enable_iface: gicv3_hyp_enable(true); WRITE_SYSREG(gicv3_ctx.cpu.ctlr, ICC_CTLR_EL1); @@ -2205,6 +2212,8 @@ static void gicv3_resume(void) =20 gicv3_redist_wait_for_rwp(); =20 + gicv3_its_resume(); + WRITE_SYSREG(gicv3_ctx.cpu.sre_el2, ICC_SRE_EL2); isb(); =20 diff --git a/xen/arch/arm/include/asm/gic_v3_its.h b/xen/arch/arm/include/a= sm/gic_v3_its.h index fc5a84892c..492c468ae0 100644 --- a/xen/arch/arm/include/asm/gic_v3_its.h +++ b/xen/arch/arm/include/asm/gic_v3_its.h @@ -129,6 +129,13 @@ struct host_its { spinlock_t cmd_lock; void *cmd_buf; unsigned int flags; +#ifdef CONFIG_SYSTEM_SUSPEND + struct suspend_ctx { + uint32_t ctlr; + uint64_t cbaser; + uint64_t baser[GITS_BASER_NR_REGS]; + } suspend_ctx; +#endif }; =20 /* Map a collection for this host CPU to each host ITS. */ @@ -204,6 +211,11 @@ uint64_t gicv3_its_get_cacheability(void); uint64_t gicv3_its_get_shareability(void); unsigned int gicv3_its_get_memflags(void); =20 +#ifdef CONFIG_SYSTEM_SUSPEND +int gicv3_its_suspend(void); +void gicv3_its_resume(void); +#endif + #else =20 #ifdef CONFIG_ACPI @@ -271,6 +283,17 @@ static inline int gicv3_its_make_hwdom_dt_nodes(const = struct domain *d, return 0; } =20 +#ifdef CONFIG_SYSTEM_SUSPEND +static inline int gicv3_its_suspend(void) +{ + return 0; +} + +static inline void gicv3_its_resume(void) +{ +} +#endif + #endif /* CONFIG_HAS_ITS */ =20 #endif diff --git a/xen/include/xen/list.h b/xen/include/xen/list.h index 98d8482dab..87c940ac3b 100644 --- a/xen/include/xen/list.h +++ b/xen/include/xen/list.h @@ -535,6 +535,20 @@ static inline void list_splice_init(struct list_head *= list, &(pos)->member !=3D (head); = \ (pos) =3D list_entry((pos)->member.next, typeof(*(pos)), member)) =20 +/** + * list_for_each_entry_continue_reverse - iterate backwards from the given= point + * @pos: the type * to use as a loop cursor. + * @head: the head for your list. + * @member: the name of the list_head within the struct. + * + * Start to iterate over list of given type backwards, continuing after + * the current position. + */ +#define list_for_each_entry_continue_reverse(pos, head, member) \ + for ((pos) =3D list_entry((pos)->member.prev, typeof(*(pos)), member);= \ + &(pos)->member !=3D (head); = \ + (pos) =3D list_entry((pos)->member.prev, typeof(*(pos)), member)) + /** * list_for_each_entry_from - iterate over list of given type from the * current point --=20 2.43.0 From nobody Sun Dec 14 05:53:41 2025 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass(p=none dis=none) header.from=gmail.com ARC-Seal: i=1; a=rsa-sha256; t=1765478776; cv=none; d=zohomail.com; s=zohoarc; b=KEzToNzIS5y5i24qkSTft2odWx44ATS77KjTuIRU/CP6owi1MxQ/Vd6LFFvWGUojcS/KdlMYcLouSOPid1S4l+mtWoSrHgbV4CuWg2zVN6nelAATVVoTd3PPbnwnrkXhuWTUDlRB5RfNJk6pqSPfdpcJBdbXBkVtfxBwk8/y/e4= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1765478776; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=Tm0K7gfsRpBFVrENvrubslPphiODpeHwAJ+aZ5FZLuc=; b=L3tUFD7W8njNIvEm1nlMkHCk3yu3GKe5KSl6dFVaqdtzGQRnZZ3kb7rDkAzYeC9VzxBfqP1xL7iYfysR635cbPOLXTxGpAX+M57RpHEMr3ELtuxLf6rEZ+tHEWb+IqSTidQjRzMtMA8OUQvsd4n9BHVnU+0J+2zr2VRm2osST/4= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1765478776826310.7018275333554; Thu, 11 Dec 2025 10:46:16 -0800 (PST) Received: from list by lists.xenproject.org with outflank-mailman.1184549.1506990 (Exim 4.92) (envelope-from ) id 1vTlfo-000436-8z; Thu, 11 Dec 2025 18:45:56 +0000 Received: by outflank-mailman (output) from mailman id 1184549.1506990; Thu, 11 Dec 2025 18:45:56 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1vTlfo-00042t-1i; Thu, 11 Dec 2025 18:45:56 +0000 Received: by outflank-mailman (input) for mailman id 1184549; Thu, 11 Dec 2025 18:45:54 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1vTlfm-0003I8-QQ for xen-devel@lists.xenproject.org; Thu, 11 Dec 2025 18:45:54 +0000 Received: from mail-ej1-x62d.google.com (mail-ej1-x62d.google.com [2a00:1450:4864:20::62d]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id 9f5d2dff-d6c1-11f0-b15b-2bf370ae4941; Thu, 11 Dec 2025 19:45:54 +0100 (CET) Received: by mail-ej1-x62d.google.com with SMTP id a640c23a62f3a-b736d883ac4so80879166b.2 for ; Thu, 11 Dec 2025 10:45:54 -0800 (PST) Received: from EPUAKYIW02F7.. (pool185-5-253-4.as6723.net. [185.5.253.4]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-b7cfa5d0b0dsm345870466b.67.2025.12.11.10.45.52 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 11 Dec 2025 10:45:52 -0800 (PST) X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 9f5d2dff-d6c1-11f0-b15b-2bf370ae4941 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1765478753; x=1766083553; darn=lists.xenproject.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Tm0K7gfsRpBFVrENvrubslPphiODpeHwAJ+aZ5FZLuc=; b=VAtN9ae8nNmfC4oN1F4H4AKYzL+oZDGTL4OYCeAN6OK5L+kxX9rFYbErtaowWWOSSB 8Kz7BUIME292K5+MkMRQZXV8jn83caFqqXOHp3pnCP8UYUq76CHlkS/8TTz7+3IA/yyn Z1sN8+r/JuQqBAPTFBkU6ixyTEbH2amdkcP4KJLSDtNdqen968IkVBoyi/TUvk9OPtfV 3AsiCRf7cIkswQMuSFsm8TqpH4IH/UEtDaLJpwBkvukyJA14wnUbzoz+iM5137PJufQ+ POGOlFsg8gW/c3lgAXcqYWOsPnA5+JWHtmwDrwAxK209H2cWfQrXw7eBErMAYysJFNBA 8aDQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1765478753; x=1766083553; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=Tm0K7gfsRpBFVrENvrubslPphiODpeHwAJ+aZ5FZLuc=; b=SkrsbzEl6ojawM3qN/is+iRHV9N+bEGRqXX9bNyUpv3f/8Vw9J6mUH0uvogTqQMM9k JjR86jA+OJn4F91qqFW6IwKLMdJbiELrey45c+K1+6Q4wLxGeor1DOjrpRCDQeDs3GvY GRJT8w0BNHk5AQgsfGgCutOHUlaNYP77pb/jS7lRqTjNhvyk3cVVvzfrE3f8Wc6qj8fa Z5V9dc5+HI3paMUdBgzg1WLQXd0gNEFQ7RCgxWe2lGIO2UHoHZMII2cMGz4i5vW0rtuJ DUsoeENc7XSNiozfjOOx2WjpD+AwEneNvWegUa13L2GSyFYb5KfZX4ZmNrADwWqunzF0 upqA== X-Gm-Message-State: AOJu0YzNotDLwY9yGqJhbjWVfUODzAhb4RCIGEqX6SDJjXJCnzrMPiTb i4QRSmQZ8/FtEHXsUzLTM41PrlJMo8k7TQCum7mwf8VpS1cJ/F5TkoU69+E1WVxJ X-Gm-Gg: AY/fxX5pgfkAG4kAqGWrN9szAQ5Vtjaky5+jjLUYrVQFtSexOkZZhTp1npiuXYPMef7 aTDivUBNKSx3H5aNfRyhE+edDGxhg0pEPQrP84Ucoxbd42LbK56lDX3aiRLDG6LnAAyYfYI6ANZ p6/jS4FmjQYRInjmbuYd8XS90RpOy8ysq9oD3D8YbKTStEiOG0wCaPc6Kn0JAgGMneOSsM8EhhV +u5mUoOIa9SNYgHZ/ZgT7tV6HvKBSBsGC99+R9PkDb5vbXwZLnDUo+6ZAilspjnk76Wzz+I7G7/ KV6HvURiJQ474crDPSlRlU6fPOYu+8sRYRExqVMpPCK8cQbgQIGX1++xc42b7xbsveC2/j/0bxI Nhnj/gZXgkh8Et087grbsyQkQq++/d9QfMq//y7KfItiEZz8OqP13cFtPCm/O+W6uZK93lABLwj YMp1LryfGEuiWqV6E/cjbnNcHXggj0fBISSy8rIQQw7hP0leFijlU= X-Google-Smtp-Source: AGHT+IE1B4v2NPTDPe7SduJz6oPKzgzBUZogvMigM1+f1OFVFif+aNHzSsolQ3PFCykuMUtOq1+ttg== X-Received: by 2002:a17:907:9602:b0:b73:5acd:465e with SMTP id a640c23a62f3a-b7ce82322a4mr699424166b.11.1765478753252; Thu, 11 Dec 2025 10:45:53 -0800 (PST) From: Mykola Kvach To: xen-devel@lists.xenproject.org Cc: Mykola Kvach , Mykola Kvach , Volodymyr Babchuk , Bertrand Marquis , Stefano Stabellini , Julien Grall , Michal Orzel Subject: [PATCH v7 05/12] xen/arm: tee: keep init_tee_secondary() for hotplug and resume Date: Thu, 11 Dec 2025 20:43:29 +0200 Message-ID: X-Mailer: git-send-email 2.43.0 In-Reply-To: References: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @gmail.com) X-ZM-MESSAGEID: 1765478778524158500 Content-Type: text/plain; charset="utf-8" From: Mykola Kvach init_tee_secondary() was marked __init and freed after boot. Calling it from the CPU hotplug/resume path then executed discarded code, which could crash Xen. Drop __init so the TEE mediator secondary init can run safely on hotplugged and resumed CPUs. Signed-off-by: Mykola Kvach --- xen/arch/arm/tee/tee.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/xen/arch/arm/tee/tee.c b/xen/arch/arm/tee/tee.c index 8501443c8e..00e561fc78 100644 --- a/xen/arch/arm/tee/tee.c +++ b/xen/arch/arm/tee/tee.c @@ -128,7 +128,7 @@ static int __init tee_init(void) =20 presmp_initcall(tee_init); =20 -void __init init_tee_secondary(void) +void init_tee_secondary(void) { if ( cur_mediator && cur_mediator->ops->init_secondary ) cur_mediator->ops->init_secondary(); --=20 2.43.0 From nobody Sun Dec 14 05:53:41 2025 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass(p=none dis=none) header.from=gmail.com ARC-Seal: i=1; a=rsa-sha256; t=1765478777; cv=none; d=zohomail.com; s=zohoarc; b=OveKeJOLk+Y00++zyhKAVLW2MrVd0vwXfRXkwx0Y7WneEcCA9S5G29Bm14NXle/lSBpIU0IBG3Ururaf17U3obSuQ5JEypl7CpYvVoIiuj6SQMn0LkcsL8igGydQ+0Rw6iBatG1Yn+uHGlzWQJsNLsohnNmFw2KXC/tKO5avsnk= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1765478777; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=rz0HDghJ0IsRbhxCGz59UWa71x4c1jNbwB3bP+NDcUo=; b=U9VK/3apy21Px+FPLSKSCvoOwfgXHy9Y9QMfYH6rI2n691korDHwh4hmRWyYKU5THEnYreoqZqZKh5Bwj7dhXVj8O95nhnCWNdlof2QfowimzShvTTHFCjrnNkS+0VXANnvvUTI8ARVuwU9hoRBWXqvL57MrQBEmH+wph1o3AF8= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1765478777410735.0840076736492; Thu, 11 Dec 2025 10:46:17 -0800 (PST) Received: from list by lists.xenproject.org with outflank-mailman.1184551.1507010 (Exim 4.92) (envelope-from ) id 1vTlfq-0004aO-7M; Thu, 11 Dec 2025 18:45:58 +0000 Received: by outflank-mailman (output) from mailman id 1184551.1507010; Thu, 11 Dec 2025 18:45:58 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1vTlfq-0004Yl-0K; Thu, 11 Dec 2025 18:45:58 +0000 Received: by outflank-mailman (input) for mailman id 1184551; Thu, 11 Dec 2025 18:45:56 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1vTlfo-0003I8-4x for xen-devel@lists.xenproject.org; Thu, 11 Dec 2025 18:45:56 +0000 Received: from mail-ed1-x52d.google.com (mail-ed1-x52d.google.com [2a00:1450:4864:20::52d]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id a01b52bc-d6c1-11f0-b15b-2bf370ae4941; Thu, 11 Dec 2025 19:45:55 +0100 (CET) Received: by mail-ed1-x52d.google.com with SMTP id 4fb4d7f45d1cf-64198771a9bso828655a12.2 for ; Thu, 11 Dec 2025 10:45:55 -0800 (PST) Received: from EPUAKYIW02F7.. (pool185-5-253-4.as6723.net. [185.5.253.4]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-b7cfa5d0b0dsm345870466b.67.2025.12.11.10.45.53 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 11 Dec 2025 10:45:53 -0800 (PST) X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: a01b52bc-d6c1-11f0-b15b-2bf370ae4941 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1765478754; x=1766083554; darn=lists.xenproject.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=rz0HDghJ0IsRbhxCGz59UWa71x4c1jNbwB3bP+NDcUo=; b=KWAP0RRJySseb9OVoG9E0sONatv2MyeoRDoINiWn5kEZFyOflwwtu+K24cGS/JGwvf cB8xYP4HD2EOYzynuOHeaH8kiaGE2IdFCM2e5wLBiK6aiGdL1c+DBCkmJWJPykI9c3+G UcMZBC+hp6K/wQKLiBiC9vz2ao0fc3zGUUK3BChJAAtS3Iclc9Ev10yHV8wZijQrWEgM O8D6KTkrvLytYSXbpFNYOZGQwOnj/IkfWcMhGf5Xck9SJYfZdxwvXF4uq38daF0dGiWn 5VwB8ty8FJGzSqxuxPCnVYxQYP2r+U3SSMekclraC7LahhbOU6080hh9sW1nhKnSQt+0 Lm+w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1765478754; x=1766083554; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=rz0HDghJ0IsRbhxCGz59UWa71x4c1jNbwB3bP+NDcUo=; b=mv/CYCTkYF9jRrfWz+mg5DrFKpbUa4utNb4yYurniQy6KS6xe3p3fu1Tn/BhSjeoO8 hPuekvHJBta5NOwEpym5VtjUvwXZaO6Pya8ZA9uBUuVv2HRjBMT7YFCeRshAk8nrBm8f ILYjpjYMmttNYJ9NLZo8AdOy/5d0OimgDu9O7+ZGdEX4g0LtTCQRPrIilgAlVj7iyuaM d6kSBChLRk5WFFktDJVNSNcT9iaZa4KvoYq7OYjBknsHX5qhMBAHCgBJ6JQmwjdCcX36 VeTl/YRNGIh82gAcZSQaz1ZfeBoMEMuoTHZAgaiBWEtQm8S5EuxwB27BE7C3VLGqkGyq PSMQ== X-Gm-Message-State: AOJu0YzlWglAbmOLPGN6BCgVRHFkvRcJKSI75FU/eet+SPubCddsXWsq kYtkSrULFN+JEs/IBbY2iCv9hrmzAlqYkhRVdxkY+1IJauf5q8f4dUCEdj2BNOnG X-Gm-Gg: AY/fxX4NhdNTr/e3Zgc7eNMFQOFtW8aNvrISRLSMFpBOgYNiBOy8Mtx+yJDGfOaPDk5 gowerT8pXLspaPhdfAp3eVqx3c845Fgbhd7LpaCrnxlXolNjT9iSCRq6371Q7nAl5h+tKXDFulI 9rh2CXQHeZ8cYwfGllARLcopXcKH9pRS23xWtYegrLC+AtSipkrAyLCW3ZPCyTYJY8VTOztnKLX AuHnO2365P1dlHVoyZ+/A9CfVQNkyE2z6ikLABRejusviHeSCG51ANFU9eY7H6KSzsqF0zcpZnb klkcZ9oKUmH6saCMa/H9kRv9rjGh60KUHCNED9mP7dYA/Qy037mL3rRvpodJiHjjd+/3T6zuR3t mYLcZkz5yuVxZOe9vslkBN3q/pgt9LTIv7VUWdRI6cK6Z6Tf8mr9CErSa8u44mYZlPv62RMpd2R +qmoSAQzpFaP7n7u3wBgIbVUVFl7TyGJGOj7gb+Yst X-Google-Smtp-Source: AGHT+IFl62FZ0FgJSyr4XEZouQeIWhJVM6kfXy3OW7rtGR1k5t90f62MNqwbncILJbOinFyLpJdUeA== X-Received: by 2002:a17:907:d10:b0:b73:210a:44e with SMTP id a640c23a62f3a-b7ce8414319mr770611666b.30.1765478754270; Thu, 11 Dec 2025 10:45:54 -0800 (PST) From: Mykola Kvach To: xen-devel@lists.xenproject.org Cc: Mykola Kvach , Mykola Kvach , Volodymyr Babchuk , Bertrand Marquis , Stefano Stabellini , Julien Grall , Michal Orzel Subject: [PATCH v7 06/12] xen/arm: ffa: fix notification SRI across CPU hotplug/suspend Date: Thu, 11 Dec 2025 20:43:30 +0200 Message-ID: X-Mailer: git-send-email 2.43.0 In-Reply-To: References: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @gmail.com) X-ZM-MESSAGEID: 1765478778515158500 Content-Type: text/plain; charset="utf-8" From: Mykola Kvach The FF-A notification SRI interrupt handler was not correctly tied to CPU hotplug and suspend/resume. As a result, CPUs going offline and back online could end up with stale or missing handlers, breaking delivery of FF-A notifications. Signed-off-by: Mykola Kvach --- xen/arch/arm/tee/ffa_notif.c | 63 ++++++++++++++++++++++++++++-------- 1 file changed, 50 insertions(+), 13 deletions(-) diff --git a/xen/arch/arm/tee/ffa_notif.c b/xen/arch/arm/tee/ffa_notif.c index 86bef6b3b2..84f5c6b43b 100644 --- a/xen/arch/arm/tee/ffa_notif.c +++ b/xen/arch/arm/tee/ffa_notif.c @@ -359,10 +359,28 @@ static int32_t ffa_notification_bitmap_destroy(uint16= _t vm_id) return ffa_simple_call(FFA_NOTIFICATION_BITMAP_DESTROY, vm_id, 0, 0, 0= ); } =20 -void ffa_notif_init_interrupt(void) +static DEFINE_PER_CPU_READ_MOSTLY(struct irqaction, sri_irq); + +static int request_sri_irq(void) { int ret; + struct irqaction *sri_action =3D &this_cpu(sri_irq); + + sri_action->name =3D "FF-A notif"; + sri_action->handler =3D notif_irq_handler; + sri_action->dev_id =3D NULL; + sri_action->free_on_release =3D 0; + + ret =3D setup_irq(notif_sri_irq, 0, sri_action); + if ( ret ) + printk(XENLOG_ERR "ffa: setup_irq irq %u failed: error %d\n", + notif_sri_irq, ret); =20 + return ret; +} + +void ffa_notif_init_interrupt(void) +{ if ( fw_notif_enabled && notif_sri_irq < NR_GIC_SGI ) { /* @@ -375,14 +393,36 @@ void ffa_notif_init_interrupt(void) * pending, while the SPMC in the secure world will not notice that * the interrupt was lost. */ - ret =3D request_irq(notif_sri_irq, 0, notif_irq_handler, "FF-A not= if", - NULL); - if ( ret ) - printk(XENLOG_ERR "ffa: request_irq irq %u failed: error %d\n", - notif_sri_irq, ret); + request_sri_irq(); } } =20 +static void deinit_ffa_notif_interrupt(void) +{ + if ( fw_notif_enabled && notif_sri_irq < NR_GIC_SGI ) + release_irq(notif_sri_irq, NULL); +} + +static int cpu_ffa_notif_callback(struct notifier_block *nfb, + unsigned long action, + void *hcpu) +{ + switch ( action ) + { + case CPU_DYING: + deinit_ffa_notif_interrupt(); + break; + default: + break; + } + + return NOTIFY_DONE; +} + +static struct notifier_block cpu_ffa_notif_nfb =3D { + .notifier_call =3D cpu_ffa_notif_callback, +}; + void ffa_notif_init(void) { const struct arm_smccc_1_2_regs arg =3D { @@ -391,7 +431,6 @@ void ffa_notif_init(void) }; struct arm_smccc_1_2_regs resp; unsigned int irq; - int ret; =20 /* Only enable fw notification if all ABIs we need are supported */ if ( ffa_fw_supports_fid(FFA_NOTIFICATION_BITMAP_CREATE) && @@ -407,13 +446,11 @@ void ffa_notif_init(void) notif_sri_irq =3D irq; if ( irq >=3D NR_GIC_SGI ) irq_set_type(irq, IRQ_TYPE_EDGE_RISING); - ret =3D request_irq(irq, 0, notif_irq_handler, "FF-A notif", NULL); - if ( ret ) - { - printk(XENLOG_ERR "ffa: request_irq irq %u failed: error %d\n", - irq, ret); + + if ( request_sri_irq() ) return; - } + + register_cpu_notifier(&cpu_ffa_notif_nfb); fw_notif_enabled =3D true; } } --=20 2.43.0 From nobody Sun Dec 14 05:53:41 2025 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass(p=none dis=none) header.from=gmail.com ARC-Seal: i=1; a=rsa-sha256; t=1765478781; cv=none; d=zohomail.com; s=zohoarc; b=LUxT3NXCe1vk9Mm62mxRfKTAv74avxoLxs5p1xz0BUKnaCXNThuwN5ql6eBk6GSUv1cdm2qtRATFQx9g9DFOT8J4tSNKBNvlrWJO3Wn8+oGDCTvYycdYi6UmH6nap8pAmdSvSTKJoWfiJBexny+3mOiqCC1KmBCWCRqQsAnhsg8= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1765478781; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=k/YQHTjWfFNRfU47XoZ3xNzm7CYArEgw/v+Ue+GlOW0=; b=g8Y5iH/nj6jJ7/oucDdiUe2heKZBP5EZoKMiFgYdphvBR6mMIkBjH3Obk0HfwFltcxRp+eqGjQb/bhaabxnHDLNQrk36rL0pH2Qse7YUWuF82+kI7jKst8ziJgx4xCqmjHit2huz0DlcfIeVrybw4iWQY4xaFPnydUOqX/B+75k= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1765478781157777.2686042753561; Thu, 11 Dec 2025 10:46:21 -0800 (PST) Received: from list by lists.xenproject.org with outflank-mailman.1184553.1507028 (Exim 4.92) (envelope-from ) id 1vTlfs-00056l-Rl; Thu, 11 Dec 2025 18:46:00 +0000 Received: by outflank-mailman (output) from mailman id 1184553.1507028; Thu, 11 Dec 2025 18:46:00 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1vTlfs-00054Y-JI; Thu, 11 Dec 2025 18:46:00 +0000 Received: by outflank-mailman (input) for mailman id 1184553; Thu, 11 Dec 2025 18:45:59 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1vTlfr-0003I9-0K for xen-devel@lists.xenproject.org; Thu, 11 Dec 2025 18:45:59 +0000 Received: from mail-ej1-x634.google.com (mail-ej1-x634.google.com [2a00:1450:4864:20::634]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id a104575a-d6c1-11f0-9cce-f158ae23cfc8; Thu, 11 Dec 2025 19:45:56 +0100 (CET) Received: by mail-ej1-x634.google.com with SMTP id a640c23a62f3a-b7a6e56193cso84167366b.3 for ; Thu, 11 Dec 2025 10:45:56 -0800 (PST) Received: from EPUAKYIW02F7.. (pool185-5-253-4.as6723.net. [185.5.253.4]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-b7cfa5d0b0dsm345870466b.67.2025.12.11.10.45.54 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 11 Dec 2025 10:45:54 -0800 (PST) X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: a104575a-d6c1-11f0-9cce-f158ae23cfc8 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1765478756; x=1766083556; darn=lists.xenproject.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=k/YQHTjWfFNRfU47XoZ3xNzm7CYArEgw/v+Ue+GlOW0=; b=HbQimVh+9Ftqd8z0fD/kPvnVs7wrtno7QCfLdeRYtXvk6wptP9SaGD/jH+mQyHBNT6 aVEzOKTWueF8Qkg/DrhmRm3s80abL1MViRV1fYVVUb+P1OFvSVpGbVLTiBmGbjs8WmEi hOU6+iVDKtowkvgxpIXvu/bqiqpKAbjh2MH8CIpwo0kIVC6+enM78LDuaD12v1+BkhM+ sxqE+zX2ODk+PYagHiHNEaS26NY4P3lVaqEbqxPCZzd9kB8oCvRsiNnMQ2UMUkjz4+wW BMlYTL5SJj5D2iooiz6B/mji8VY7U0vW3aH5vUNs9NqIrnFvTVL5QsdpI4WNHo2v/5R4 eQdw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1765478756; x=1766083556; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=k/YQHTjWfFNRfU47XoZ3xNzm7CYArEgw/v+Ue+GlOW0=; b=ilRJX4An0otrYhMMgeNaAyRoB9s3OSUHDCpQ/h7JRKGc0z+J2ViejWrmtWN1UFLfyP 4gps0uOn8sJDvtMlH1MjrjV2rjQFLvCP/7Z00xD9Jo9rKPfNvqTUDGtuR4x3vBVx/VN8 vybtF/sqD1In7BUY5CQpm7sfxZuf7PBVPf5DiM1p/QQ/tr7cweVk0SmKebIgPbpjl3zp D6khscJat3px8F88VNRcyIEGYvLMuYd5xXrogIKSVrZzL7ZTe7ShI3mtPsHTQToTXxSX aXTR4ZvWeTW2nWzzTOnoCXshiqtDyMXdViEmFEKJ/hefSJiXJiESZwfpbxdgHS6jnCTu nxsw== X-Gm-Message-State: AOJu0Ywy3gq5rKmjWEyPZRhWH4DdPZCdkPlNoY3IHqWeTxerYAHQQqvO igtP0xbW4L6nVx8JnzWCvZ6Jbt8LPZ2UjiSaJEjzoiP1fNT3AjJZL4/+WqqFz0ZX X-Gm-Gg: AY/fxX5rr3sQ+gwmAG2faWeE7/qJimfg1/MWRB8D+kHyQggXE1D+ZNTdntADlyTSbY9 j23CaXO9lMlR4wTUbLvTTAPu43/IGm6JQfXRk2d5nWBVTrukv1WWXnsXKQ2Zh5k8tH2Diqf9J71 VaN04CeGfEWA3+evrLIPK23szIuKtrQmdyadHfNtwN4gNH1nAZGwPRjyrGWEe/L2WLlGKBk6Hm7 pLwPx2tHf9dode0T4FY/hPyiLvRowwqMTdMWqw0hEt8Mldi5WvR/S6Q00iGhH6olVzhIr+P7vpI hWMnTIR0NCP4UvZUwRIAtD8duHwoTp1BBsCEHx4F2b0V9P1RR6QAM7BcGnEuuw3hdYB09DazAAx 1iq0eP7UeIKjkCCq+NgEVGJgwtN6gmQvVoawi4IFgPNc0MwtfWIMHtBf4co0N9OJ5JtdAy/7WIv kVmLzuULxp9Gvs60ULLrHVmL6Y3XQDRd305975o6xL X-Google-Smtp-Source: AGHT+IH85KHr2aOHxwE9RCp0pBn4R4LzUOG7kFSsLpxoAJMEMAJUDSPELOGwpr48HzXuALSeeczMXw== X-Received: by 2002:a17:906:7316:b0:b76:3476:a83c with SMTP id a640c23a62f3a-b7ce84d9466mr815044466b.40.1765478755558; Thu, 11 Dec 2025 10:45:55 -0800 (PST) From: Mykola Kvach To: xen-devel@lists.xenproject.org Cc: Mykola Kvach , Mykola Kvach , Stefano Stabellini , Julien Grall , Bertrand Marquis , Michal Orzel , Volodymyr Babchuk Subject: [PATCH v7 07/12] iommu/ipmmu-vmsa: Implement suspend/resume callbacks Date: Thu, 11 Dec 2025 20:43:31 +0200 Message-ID: <220c777ee30fd35afeedefff5a73d62d6ea1e0ac.1765472890.git.mykola_kvach@epam.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: References: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @gmail.com) X-ZM-MESSAGEID: 1765478782530158500 Content-Type: text/plain; charset="utf-8" From: Oleksandr Tyshchenko Store and restore active context and micro-TLB registers. Tested on R-Car H3 Starter Kit. Signed-off-by: Oleksandr Tyshchenko Signed-off-by: Mykola Kvach --- Changes in V7: - moved suspend context allocation before pci stuff --- xen/drivers/passthrough/arm/ipmmu-vmsa.c | 305 ++++++++++++++++++++++- 1 file changed, 298 insertions(+), 7 deletions(-) diff --git a/xen/drivers/passthrough/arm/ipmmu-vmsa.c b/xen/drivers/passthr= ough/arm/ipmmu-vmsa.c index ea9fa9ddf3..6765bd3083 100644 --- a/xen/drivers/passthrough/arm/ipmmu-vmsa.c +++ b/xen/drivers/passthrough/arm/ipmmu-vmsa.c @@ -71,6 +71,8 @@ }) #endif =20 +#define dev_dbg(dev, fmt, ...) \ + dev_print(dev, XENLOG_DEBUG, fmt, ## __VA_ARGS__) #define dev_info(dev, fmt, ...) \ dev_print(dev, XENLOG_INFO, fmt, ## __VA_ARGS__) #define dev_warn(dev, fmt, ...) \ @@ -130,6 +132,24 @@ struct ipmmu_features { unsigned int imuctr_ttsel_mask; }; =20 +#ifdef CONFIG_SYSTEM_SUSPEND + +struct ipmmu_reg_ctx { + unsigned int imttlbr0; + unsigned int imttubr0; + unsigned int imttbcr; + unsigned int imctr; +}; + +struct ipmmu_vmsa_backup { + struct device *dev; + unsigned int *utlbs_val; + unsigned int *asids_val; + struct list_head list; +}; + +#endif + /* Root/Cache IPMMU device's information */ struct ipmmu_vmsa_device { struct device *dev; @@ -142,6 +162,9 @@ struct ipmmu_vmsa_device { struct ipmmu_vmsa_domain *domains[IPMMU_CTX_MAX]; unsigned int utlb_refcount[IPMMU_UTLB_MAX]; const struct ipmmu_features *features; +#ifdef CONFIG_SYSTEM_SUSPEND + struct ipmmu_reg_ctx *reg_backup[IPMMU_CTX_MAX]; +#endif }; =20 /* @@ -547,6 +570,245 @@ static void ipmmu_domain_free_context(struct ipmmu_vm= sa_device *mmu, spin_unlock_irqrestore(&mmu->lock, flags); } =20 +#ifdef CONFIG_SYSTEM_SUSPEND + +static DEFINE_SPINLOCK(ipmmu_devices_backup_lock); +static LIST_HEAD(ipmmu_devices_backup); + +static struct ipmmu_reg_ctx root_pgtable[IPMMU_CTX_MAX]; + +static uint32_t ipmmu_imuasid_read(struct ipmmu_vmsa_device *mmu, + unsigned int utlb) +{ + return ipmmu_read(mmu, ipmmu_utlb_reg(mmu, IMUASID(utlb))); +} + +static void ipmmu_utlbs_backup(struct ipmmu_vmsa_device *mmu) +{ + struct ipmmu_vmsa_backup *backup_data; + + dev_dbg(mmu->dev, "Handle micro-TLBs backup\n"); + + spin_lock(&ipmmu_devices_backup_lock); + + list_for_each_entry( backup_data, &ipmmu_devices_backup, list ) + { + struct iommu_fwspec *fwspec =3D dev_iommu_fwspec_get(backup_data->= dev); + unsigned int i; + + if ( to_ipmmu(backup_data->dev) !=3D mmu ) + continue; + + for ( i =3D 0; i < fwspec->num_ids; i++ ) + { + unsigned int utlb =3D fwspec->ids[i]; + + backup_data->asids_val[i] =3D ipmmu_imuasid_read(mmu, utlb); + backup_data->utlbs_val[i] =3D ipmmu_imuctr_read(mmu, utlb); + } + } + + spin_unlock(&ipmmu_devices_backup_lock); +} + +static void ipmmu_utlbs_restore(struct ipmmu_vmsa_device *mmu) +{ + struct ipmmu_vmsa_backup *backup_data; + + dev_dbg(mmu->dev, "Handle micro-TLBs restore\n"); + + spin_lock(&ipmmu_devices_backup_lock); + + list_for_each_entry( backup_data, &ipmmu_devices_backup, list ) + { + struct iommu_fwspec *fwspec =3D dev_iommu_fwspec_get(backup_data->= dev); + unsigned int i; + + if ( to_ipmmu(backup_data->dev) !=3D mmu ) + continue; + + for ( i =3D 0; i < fwspec->num_ids; i++ ) + { + unsigned int utlb =3D fwspec->ids[i]; + + ipmmu_imuasid_write(mmu, utlb, backup_data->asids_val[i]); + ipmmu_imuctr_write(mmu, utlb, backup_data->utlbs_val[i]); + } + } + + spin_unlock(&ipmmu_devices_backup_lock); +} + +static void ipmmu_domain_backup_context(struct ipmmu_vmsa_domain *domain) +{ + struct ipmmu_vmsa_device *mmu =3D domain->mmu->root; + struct ipmmu_reg_ctx *regs =3D mmu->reg_backup[domain->context_id]; + + dev_dbg(mmu->dev, "Handle domain context %u backup\n", domain->context= _id); + + regs->imttlbr0 =3D ipmmu_ctx_read_root(domain, IMTTLBR0); + regs->imttubr0 =3D ipmmu_ctx_read_root(domain, IMTTUBR0); + regs->imttbcr =3D ipmmu_ctx_read_root(domain, IMTTBCR); + regs->imctr =3D ipmmu_ctx_read_root(domain, IMCTR); +} + +static void ipmmu_domain_restore_context(struct ipmmu_vmsa_domain *domain) +{ + struct ipmmu_vmsa_device *mmu =3D domain->mmu->root; + struct ipmmu_reg_ctx *regs =3D mmu->reg_backup[domain->context_id]; + + dev_dbg(mmu->dev, "Handle domain context %u restore\n", domain->contex= t_id); + + ipmmu_ctx_write_root(domain, IMTTLBR0, regs->imttlbr0); + ipmmu_ctx_write_root(domain, IMTTUBR0, regs->imttubr0); + ipmmu_ctx_write_root(domain, IMTTBCR, regs->imttbcr); + ipmmu_ctx_write_all(domain, IMCTR, regs->imctr | IMCTR_FLUSH); +} + +/* + * Xen: Unlike Linux implementation, Xen uses a single driver instance + * for handling all IPMMUs. There is no framework for ipmmu_suspend/resume + * callbacks to be invoked for each IPMMU device. So, we need to iterate + * through all registered IPMMUs performing required actions. + * + * Also take care of restoring special settings, such as translation + * table format, etc. + */ +static int __must_check ipmmu_suspend(void) +{ + struct ipmmu_vmsa_device *mmu; + + if ( !iommu_enabled ) + return 0; + + printk(XENLOG_DEBUG "ipmmu: Suspending...\n"); + + spin_lock(&ipmmu_devices_lock); + + list_for_each_entry( mmu, &ipmmu_devices, list ) + { + if ( ipmmu_is_root(mmu) ) + { + unsigned int i; + + for ( i =3D 0; i < mmu->num_ctx; i++ ) + { + if ( !mmu->domains[i] ) + continue; + ipmmu_domain_backup_context(mmu->domains[i]); + } + } + else + ipmmu_utlbs_backup(mmu); + } + + spin_unlock(&ipmmu_devices_lock); + + return 0; +} + +static void ipmmu_resume(void) +{ + struct ipmmu_vmsa_device *mmu; + + if ( !iommu_enabled ) + return; + + printk(XENLOG_DEBUG "ipmmu: Resuming...\n"); + + spin_lock(&ipmmu_devices_lock); + + list_for_each_entry( mmu, &ipmmu_devices, list ) + { + uint32_t reg; + + /* Do not use security group function */ + reg =3D IMSCTLR + mmu->features->control_offset_base; + ipmmu_write(mmu, reg, ipmmu_read(mmu, reg) & ~IMSCTLR_USE_SECGRP); + + if ( ipmmu_is_root(mmu) ) + { + unsigned int i; + + /* Use stage 2 translation table format */ + reg =3D IMSAUXCTLR + mmu->features->control_offset_base; + ipmmu_write(mmu, reg, ipmmu_read(mmu, reg) | IMSAUXCTLR_S2PTE); + + for ( i =3D 0; i < mmu->num_ctx; i++ ) + { + if ( !mmu->domains[i] ) + continue; + ipmmu_domain_restore_context(mmu->domains[i]); + } + } + else + ipmmu_utlbs_restore(mmu); + } + + spin_unlock(&ipmmu_devices_lock); +} + +static int ipmmu_alloc_ctx_suspend(struct device *dev) +{ + struct ipmmu_vmsa_backup *backup_data; + unsigned int *utlbs_val, *asids_val; + struct iommu_fwspec *fwspec =3D dev_iommu_fwspec_get(dev); + + utlbs_val =3D xzalloc_array(unsigned int, fwspec->num_ids); + if ( !utlbs_val ) + return -ENOMEM; + + asids_val =3D xzalloc_array(unsigned int, fwspec->num_ids); + if ( !asids_val ) + { + xfree(utlbs_val); + return -ENOMEM; + } + + backup_data =3D xzalloc(struct ipmmu_vmsa_backup); + if ( !backup_data ) + { + xfree(utlbs_val); + xfree(asids_val); + return -ENOMEM; + } + + backup_data->dev =3D dev; + backup_data->utlbs_val =3D utlbs_val; + backup_data->asids_val =3D asids_val; + + spin_lock(&ipmmu_devices_backup_lock); + list_add(&backup_data->list, &ipmmu_devices_backup); + spin_unlock(&ipmmu_devices_backup_lock); + + return 0; +} + +#ifdef CONFIG_HAS_PCI +static void ipmmu_free_ctx_suspend(struct device *dev) +{ + struct ipmmu_vmsa_backup *backup_data, *tmp; + + spin_lock(&ipmmu_devices_backup_lock); + + list_for_each_entry_safe( backup_data, tmp, &ipmmu_devices_backup, lis= t ) + { + if ( backup_data->dev =3D=3D dev ) + { + list_del(&backup_data->list); + xfree(backup_data->utlbs_val); + xfree(backup_data->asids_val); + xfree(backup_data); + break; + } + } + + spin_unlock(&ipmmu_devices_backup_lock); +} +#endif /* CONFIG_HAS_PCI */ + +#endif /* CONFIG_SYSTEM_SUSPEND */ + static int ipmmu_domain_init_context(struct ipmmu_vmsa_domain *domain) { uint64_t ttbr; @@ -559,6 +821,9 @@ static int ipmmu_domain_init_context(struct ipmmu_vmsa_= domain *domain) return ret; =20 domain->context_id =3D ret; +#ifdef CONFIG_SYSTEM_SUSPEND + domain->mmu->root->reg_backup[ret] =3D &root_pgtable[ret]; +#endif =20 /* * TTBR0 @@ -615,6 +880,9 @@ static void ipmmu_domain_destroy_context(struct ipmmu_v= msa_domain *domain) ipmmu_ctx_write_root(domain, IMCTR, IMCTR_FLUSH); ipmmu_tlb_sync(domain); =20 +#ifdef CONFIG_SYSTEM_SUSPEND + domain->mmu->root->reg_backup[domain->context_id] =3D NULL; +#endif ipmmu_domain_free_context(domain->mmu->root, domain->context_id); } =20 @@ -1340,10 +1608,11 @@ static int ipmmu_add_device(u8 devfn, struct device= *dev) struct iommu_fwspec *fwspec; =20 #ifdef CONFIG_HAS_PCI + int ret; + if ( dev_is_pci(dev) ) { struct pci_dev *pdev =3D dev_to_pci(dev); - int ret; =20 if ( devfn !=3D pdev->devfn ) return 0; @@ -1371,6 +1640,15 @@ static int ipmmu_add_device(u8 devfn, struct device = *dev) /* Let Xen know that the master device is protected by an IOMMU. */ dt_device_set_protected(dev_to_dt(dev)); } + +#ifdef CONFIG_SYSTEM_SUSPEND + if ( ipmmu_alloc_ctx_suspend(dev) ) + { + dev_err(dev, "Failed to allocate context for suspend\n"); + return -ENOMEM; + } +#endif + #ifdef CONFIG_HAS_PCI if ( dev_is_pci(dev) ) { @@ -1379,26 +1657,28 @@ static int ipmmu_add_device(u8 devfn, struct device= *dev) struct pci_host_bridge *bridge; struct iommu_fwspec *fwspec_bridge; unsigned int utlb_osid0 =3D 0; - int ret; =20 bridge =3D pci_find_host_bridge(pdev->seg, pdev->bus); if ( !bridge ) { dev_err(dev, "Failed to find host bridge\n"); - return -ENODEV; + ret =3D -ENODEV; + goto free_suspend_ctx; } =20 fwspec_bridge =3D dev_iommu_fwspec_get(dt_to_dev(bridge->dt_node)); if ( fwspec_bridge->num_ids < 1 ) { dev_err(dev, "Failed to find host bridge uTLB\n"); - return -ENXIO; + ret =3D -ENXIO; + goto free_suspend_ctx; } =20 if ( fwspec->num_ids < 1 ) { dev_err(dev, "Failed to find uTLB"); - return -ENXIO; + ret =3D -ENXIO; + goto free_suspend_ctx; } =20 rcar4_pcie_osid_regs_init(bridge); @@ -1407,7 +1687,7 @@ static int ipmmu_add_device(u8 devfn, struct device *= dev) if ( ret < 0 ) { dev_err(dev, "No unused OSID regs\n"); - return ret; + goto free_suspend_ctx; } reg_id =3D ret; =20 @@ -1422,7 +1702,7 @@ static int ipmmu_add_device(u8 devfn, struct device *= dev) { rcar4_pcie_osid_bdf_clear(bridge, reg_id); rcar4_pcie_osid_reg_free(bridge, reg_id); - return ret; + goto free_suspend_ctx; } } #endif @@ -1431,6 +1711,13 @@ static int ipmmu_add_device(u8 devfn, struct device = *dev) dev_name(fwspec->iommu_dev), fwspec->num_ids); =20 return 0; +#ifdef CONFIG_HAS_PCI + free_suspend_ctx: +#ifdef CONFIG_SYSTEM_SUSPEND + ipmmu_free_ctx_suspend(dev); +#endif + return ret; +#endif } =20 static int ipmmu_iommu_domain_init(struct domain *d) @@ -1492,6 +1779,10 @@ static const struct iommu_ops ipmmu_iommu_ops =3D .unmap_page =3D arm_iommu_unmap_page, .dt_xlate =3D ipmmu_dt_xlate, .add_device =3D ipmmu_add_device, +#ifdef CONFIG_SYSTEM_SUSPEND + .suspend =3D ipmmu_suspend, + .resume =3D ipmmu_resume, +#endif }; =20 static __init int ipmmu_init(struct dt_device_node *node, const void *data) --=20 2.43.0 From nobody Sun Dec 14 05:53:41 2025 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass(p=none dis=none) header.from=gmail.com ARC-Seal: i=1; a=rsa-sha256; t=1765478786; cv=none; d=zohomail.com; s=zohoarc; b=CvJXbXUaGVBXgKikk+fPaG61MGXNk8ERWnha2fjCr+WhXvvqDsf8n+b0Yr580OLc6DY6XHGtti4X5spUU8jpIVdGKENB9fV8LB9anY20Nz4SVg787Ul1ot7dwaBYh6zzMGFBGMJxHq8Xmw5sea8tC+QNlJLk0G6O1NXNMc66e/Q= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1765478786; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=YyX+DKhss3Tir9n7VoBzOtKgYwwNvTr5WMzP0cAUYTo=; b=R6Z1UfhenCpOeMoLC3mWKRjahNNpUKks3l9zimOpFCz0UeLJ1+B777NcQ9nW303LVBP/OE9zYdJuLBJWTfcCCHk8i3DZvQpm9jLMK74USOitAoTBFnYY4KxDDb74xHRWg2hD8b5V7q6v9lGMoXQG5jB0iz3nEF9wTzS/OiAkHsQ= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1765478786576190.6617171409771; Thu, 11 Dec 2025 10:46:26 -0800 (PST) Received: from list by lists.xenproject.org with outflank-mailman.1184552.1507024 (Exim 4.92) (envelope-from ) id 1vTlfs-00052j-EQ; Thu, 11 Dec 2025 18:46:00 +0000 Received: by outflank-mailman (output) from mailman id 1184552.1507024; Thu, 11 Dec 2025 18:46:00 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1vTlfs-00052Y-9j; Thu, 11 Dec 2025 18:46:00 +0000 Received: by outflank-mailman (input) for mailman id 1184552; Thu, 11 Dec 2025 18:45:58 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1vTlfq-0003I8-OZ for xen-devel@lists.xenproject.org; Thu, 11 Dec 2025 18:45:58 +0000 Received: from mail-ej1-x632.google.com (mail-ej1-x632.google.com [2a00:1450:4864:20::632]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id a1a96d36-d6c1-11f0-b15b-2bf370ae4941; Thu, 11 Dec 2025 19:45:57 +0100 (CET) Received: by mail-ej1-x632.google.com with SMTP id a640c23a62f3a-b734fcbf1e3so90814766b.3 for ; Thu, 11 Dec 2025 10:45:58 -0800 (PST) Received: from EPUAKYIW02F7.. (pool185-5-253-4.as6723.net. [185.5.253.4]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-b7cfa5d0b0dsm345870466b.67.2025.12.11.10.45.55 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 11 Dec 2025 10:45:56 -0800 (PST) X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: a1a96d36-d6c1-11f0-b15b-2bf370ae4941 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1765478757; x=1766083557; darn=lists.xenproject.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=YyX+DKhss3Tir9n7VoBzOtKgYwwNvTr5WMzP0cAUYTo=; b=SeqabM84oMlxwqvC9qGw216lkj/k6KV2LnAvS0TPSvHsyRLEN/t2xMUrxPRl8qqYnK o1hzMP15Q2yDfptcZKqlFrqVPiSNnyShUcryIaKEmqEsQZTImE3HgfQWMh4FbIuADwlK VynWznj/G9dqfOWi9DTiAVM76ItI21u1+dDa64zmEd/mGIuMg+X/HPppdNJWs8yaQctx OAoy69Hq+kEp87Ub/0kqNQQKm3TBk5sm3yNm0QFaisBE2sleRBSbNhmEySnP6fgKQaxB hq4CtStARO8ACsbW27XHsbkku3zm0O/71jywK6WT0hWkUBuWoEpLusFnt7yRlktkVIZ2 1lGw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1765478757; x=1766083557; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=YyX+DKhss3Tir9n7VoBzOtKgYwwNvTr5WMzP0cAUYTo=; b=bb3ayGURspc+VaQNkBWEp5NQbkBdElX37swNx56XOe41UMlVoBWwl3KmagthXQtqRc zxvVewya7kspU/ly2blIoHACz61CHQqkxv/SmZH/bk/Yg5PNEc/uHMyDB5g5bixUCN3+ 4ODA0KqkQ8sPJdlSAryzNNVGvjH59D/jAYTvbGGUbryrBY3oOR4PhOJexpY3I6XeKQkR VFHT5qKCZOPwcgQzlHz0LrRuLBIMF9G44kojDrV3eIIqJAj+I9pK1GoWl27601E3OdN6 T15FepqEPxp0pV0KBTYFnUS5YC9yg2CyXNJYQVPufM2FBvr9rhFT30wITEpFyrqDo3iZ 9wDQ== X-Gm-Message-State: AOJu0Yw9FNZwarjCXykUs1z4rZla+CjgaSezchPNPpP5xxM1Ocw70/uj pWtNBMjZnSjtTaSbGAPKTkqsSij+Y7FJriaX8vhIqfZMqJeaoPkGRxMCp0cmF5P4 X-Gm-Gg: AY/fxX5S3XQ9XxJ6wT/ztpgsEdWea2B38IJ/7aUdQkpYicv3disAl2aZLuHOna4e4by Fl5sWNHDLPVJnSlfp7WeMH1unMrE2NX1DmNnwac34criY3HeAQ/l6nU7yikr6qIXKen5rXeiW+b 4pY2thGg7BUTZufzav042/m+OUn+7H1JgwafvbCaJXjDhEMnsKZUnvwJbdvGgCcSZyH+7lEA8Kj xLAQBvPVaSMUFBFJc8GYnAG5slwyZm7ULz/UOZfqVyPYf95rCpJMJgOJcOroM0KGjkzqZZeJpsO T1B1+82rgCpsTyIt5f5QzIfFGNcnIKu0pV+hrFsUSn2HFXVucCbqgjxax9psyTH4ES4uSe1Kj6o RR03HnDoTnUGeN/1xd5hUZ24OE7yhGE1q9+EbEwQIBkrJNCJKwco+XmbWHZvpoSifXfOgcM2H55 16QuQVox02Z1uAQ31kBDKEVy0jXBqgk5679Prc1DVE X-Google-Smtp-Source: AGHT+IHqZwnZeJZ9SiPy9sMGvjH5yQTECZVUy2zQSTwn/fCCIricEV1JkHIoYJqUXa8UozBVjbaDgw== X-Received: by 2002:a17:907:6ea7:b0:b73:5c12:3f8a with SMTP id a640c23a62f3a-b7ce826e8b7mr710504466b.18.1765478756907; Thu, 11 Dec 2025 10:45:56 -0800 (PST) From: Mykola Kvach To: xen-devel@lists.xenproject.org Cc: Mykola Kvach , Mykola Kvach , Bertrand Marquis , Rahul Singh , Stefano Stabellini , Julien Grall , Michal Orzel , Volodymyr Babchuk , Pranjal Shrivastava Subject: [PATCH v7 08/12] arm/smmu-v3: add suspend/resume handlers Date: Thu, 11 Dec 2025 20:43:32 +0200 Message-ID: <58c1873d355f5ea9b5182349895905d25cb57256.1765472890.git.mykola_kvach@epam.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: References: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @gmail.com) X-ZM-MESSAGEID: 1765478788667158500 Content-Type: text/plain; charset="utf-8" From: Mykola Kvach Before we suspend SMMU, we want to ensure that all commands (especially ATC_INV) have been flushed by the CMDQ, i.e. the CMDQs are empty. The suspend callback configures the SMMU to abort new transactions, disables the main translation unit and then drains the command queue to ensure completion of any in-flight commands. The resume callback performs a full device reset via 'arm_smmu_device_reset' to bring the SMMU back to an operational state. Link: https://lore.kernel.org/linux-iommu/20251117191433.3360130-1-praan@go= ogle.com / Based-on-patch-by: Pranjal Shrivastava Signed-off-by: Mykola Kvach --- xen/drivers/passthrough/arm/smmu-v3.c | 170 ++++++++++++++++++++------ 1 file changed, 134 insertions(+), 36 deletions(-) diff --git a/xen/drivers/passthrough/arm/smmu-v3.c b/xen/drivers/passthroug= h/arm/smmu-v3.c index bf153227db..10c4c5dee0 100644 --- a/xen/drivers/passthrough/arm/smmu-v3.c +++ b/xen/drivers/passthrough/arm/smmu-v3.c @@ -1814,8 +1814,7 @@ static int arm_smmu_write_reg_sync(struct arm_smmu_de= vice *smmu, u32 val, } =20 /* GBPA is "special" */ -static int __init arm_smmu_update_gbpa(struct arm_smmu_device *smmu, - u32 set, u32 clr) +static int arm_smmu_update_gbpa(struct arm_smmu_device *smmu, u32 set, u32= clr) { int ret; u32 reg, __iomem *gbpa =3D smmu->base + ARM_SMMU_GBPA; @@ -1995,10 +1994,29 @@ err_free_evtq_irq: return ret; } =20 +static int arm_smmu_enable_irqs(struct arm_smmu_device *smmu) +{ + int ret; + u32 irqen_flags =3D IRQ_CTRL_EVTQ_IRQEN | IRQ_CTRL_GERROR_IRQEN; + + if ( smmu->features & ARM_SMMU_FEAT_PRI ) + irqen_flags |=3D IRQ_CTRL_PRIQ_IRQEN; + + /* Enable interrupt generation on the SMMU */ + ret =3D arm_smmu_write_reg_sync(smmu, irqen_flags, + ARM_SMMU_IRQ_CTRL, ARM_SMMU_IRQ_CTRLACK); + if ( ret ) + { + dev_warn(smmu->dev, "failed to enable irqs\n"); + return ret; + } + + return 0; +} + static int __init arm_smmu_setup_irqs(struct arm_smmu_device *smmu) { int ret, irq; - u32 irqen_flags =3D IRQ_CTRL_EVTQ_IRQEN | IRQ_CTRL_GERROR_IRQEN; =20 /* Disable IRQs first */ ret =3D arm_smmu_write_reg_sync(smmu, 0, ARM_SMMU_IRQ_CTRL, @@ -2028,22 +2046,7 @@ static int __init arm_smmu_setup_irqs(struct arm_smm= u_device *smmu) } } =20 - if (smmu->features & ARM_SMMU_FEAT_PRI) - irqen_flags |=3D IRQ_CTRL_PRIQ_IRQEN; - - /* Enable interrupt generation on the SMMU */ - ret =3D arm_smmu_write_reg_sync(smmu, irqen_flags, - ARM_SMMU_IRQ_CTRL, ARM_SMMU_IRQ_CTRLACK); - if (ret) { - dev_warn(smmu->dev, "failed to enable irqs\n"); - goto err_free_irqs; - } - return 0; - -err_free_irqs: - arm_smmu_free_irqs(smmu); - return ret; } =20 static int arm_smmu_device_disable(struct arm_smmu_device *smmu) @@ -2057,7 +2060,7 @@ static int arm_smmu_device_disable(struct arm_smmu_de= vice *smmu) return ret; } =20 -static int __init arm_smmu_device_reset(struct arm_smmu_device *smmu) +static int arm_smmu_device_reset(struct arm_smmu_device *smmu) { int ret; u32 reg, enables; @@ -2163,17 +2166,9 @@ static int __init arm_smmu_device_reset(struct arm_s= mmu_device *smmu) } } =20 - ret =3D arm_smmu_setup_irqs(smmu); - if (ret) { - dev_err(smmu->dev, "failed to setup irqs\n"); + ret =3D arm_smmu_enable_irqs(smmu); + if ( ret ) return ret; - } - - /* Initialize tasklets for threaded IRQs*/ - tasklet_init(&smmu->evtq_irq_tasklet, arm_smmu_evtq_tasklet, smmu); - tasklet_init(&smmu->priq_irq_tasklet, arm_smmu_priq_tasklet, smmu); - tasklet_init(&smmu->combined_irq_tasklet, arm_smmu_combined_irq_tasklet, - smmu); =20 /* Enable the SMMU interface, or ensure bypass */ if (disable_bypass) { @@ -2181,20 +2176,16 @@ static int __init arm_smmu_device_reset(struct arm_= smmu_device *smmu) } else { ret =3D arm_smmu_update_gbpa(smmu, 0, GBPA_ABORT); if (ret) - goto err_free_irqs; + return ret; } ret =3D arm_smmu_write_reg_sync(smmu, enables, ARM_SMMU_CR0, ARM_SMMU_CR0ACK); if (ret) { dev_err(smmu->dev, "failed to enable SMMU interface\n"); - goto err_free_irqs; + return ret; } =20 return 0; - -err_free_irqs: - arm_smmu_free_irqs(smmu); - return ret; } =20 static int arm_smmu_device_hw_probe(struct arm_smmu_device *smmu) @@ -2558,10 +2549,23 @@ static int __init arm_smmu_device_probe(struct plat= form_device *pdev) if (ret) goto out_free; =20 + ret =3D arm_smmu_setup_irqs(smmu); + if ( ret ) + { + dev_err(smmu->dev, "failed to setup irqs\n"); + goto out_free; + } + + /* Initialize tasklets for threaded IRQs*/ + tasklet_init(&smmu->evtq_irq_tasklet, arm_smmu_evtq_tasklet, smmu); + tasklet_init(&smmu->priq_irq_tasklet, arm_smmu_priq_tasklet, smmu); + tasklet_init(&smmu->combined_irq_tasklet, arm_smmu_combined_irq_tasklet, + smmu); + /* Reset the device */ ret =3D arm_smmu_device_reset(smmu); if (ret) - goto out_free; + goto out_free_irqs; =20 /* * Keep a list of all probed devices. This will be used to query @@ -2575,6 +2579,8 @@ static int __init arm_smmu_device_probe(struct platfo= rm_device *pdev) =20 return 0; =20 +out_free_irqs: + arm_smmu_free_irqs(smmu); =20 out_free: arm_smmu_free_structures(smmu); @@ -2855,6 +2861,94 @@ static void arm_smmu_iommu_xen_domain_teardown(struc= t domain *d) xfree(xen_domain); } =20 +#ifdef CONFIG_SYSTEM_SUSPEND + +static int arm_smmu_suspend(void) +{ + struct arm_smmu_device *smmu; + int ret =3D 0; + + list_for_each_entry(smmu, &arm_smmu_devices, devices) + { + /* Abort all transactions before disable to avoid spurious bypass */ + ret =3D arm_smmu_update_gbpa(smmu, GBPA_ABORT, 0); + if ( ret ) + goto fail; + + /* Disable the SMMU via CR0.EN and all queues except CMDQ */ + ret =3D arm_smmu_write_reg_sync(smmu, CR0_CMDQEN, ARM_SMMU_CR0, + ARM_SMMU_CR0ACK); + if ( ret ) + { + dev_err(smmu->dev, "Timed-out while disabling smmu\n"); + goto fail; + } + + /* + * At this point the SMMU is completely disabled and won't access + * any translation/config structures, even speculative accesses + * aren't performed as per the IHI0070 spec (section 6.3.9.6). + */ + + /* Wait for the CMDQs to be drained to flush any pending commands */ + ret =3D queue_poll_cons(&smmu->cmdq.q, true, 0); + if ( ret ) + { + dev_err(smmu->dev, "Draining queues timed-out\n"); + goto fail; + } + + /* Disable everything */ + ret =3D arm_smmu_device_disable(smmu); + if ( ret ) + goto fail; + + dev_dbg(smmu->dev, "Suspended smmu\n"); + } + + return 0; + + fail: + { + int rc; + + /* Reset the device that failed as well as any already-suspended ones. */ + rc =3D arm_smmu_device_reset(smmu); + if ( rc ) + dev_err(smmu->dev, "Failed to reset during resume operation: %d\n", rc); + + list_for_each_entry_continue_reverse(smmu, &arm_smmu_devices, devices) + { + rc =3D arm_smmu_device_reset(smmu); + if ( rc ) + dev_err(smmu->dev, "Failed to reset during resume operation: %d\n", rc= ); + } + } + + return ret; +} + +static void arm_smmu_resume(void) +{ + int ret; + struct arm_smmu_device *smmu; + + list_for_each_entry(smmu, &arm_smmu_devices, devices) + { + dev_dbg(smmu->dev, "Resuming device\n"); + + /* + * The reset will re-initialize all the base addresses, queues, + * prod and cons maintained within struct arm_smmu_device as well as + * re-enable the interrupts. + */ + ret =3D arm_smmu_device_reset(smmu); + if ( ret ) + dev_err(smmu->dev, "Failed to reset during resume operation: %d\n", ret= ); + } +} +#endif + static const struct iommu_ops arm_smmu_iommu_ops =3D { .page_sizes =3D PAGE_SIZE_4K, .init =3D arm_smmu_iommu_xen_domain_init, @@ -2867,6 +2961,10 @@ static const struct iommu_ops arm_smmu_iommu_ops =3D= { .unmap_page =3D arm_iommu_unmap_page, .dt_xlate =3D arm_smmu_dt_xlate, .add_device =3D arm_smmu_add_device, +#ifdef CONFIG_SYSTEM_SUSPEND + .suspend =3D arm_smmu_suspend, + .resume =3D arm_smmu_resume, +#endif }; =20 static __init int arm_smmu_dt_init(struct dt_device_node *dev, --=20 2.43.0 From nobody Sun Dec 14 05:53:41 2025 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass(p=none dis=none) header.from=gmail.com ARC-Seal: i=1; a=rsa-sha256; t=1765478788; cv=none; d=zohomail.com; s=zohoarc; b=FNu3p1BaE/rhnmZYyyjK8/GUj/ds4vFub1kW6L/yiXkZtVIqDGTBa6ZsJicSpnkPv7VX46pK+dUcfuA8zACyYbraYnQs1aWmQTZDr6acnTQKZXhAvskCMkskgqqrP/WU/Zmqely1Y3zTBeOUlHgguyULjq7eF+QoJX8TFwgdHJw= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1765478788; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=Wn2YE/N1073l2aBvkRQ/5nSVLZsSRqqVHeilL+wTO7s=; b=elzFrfWTH1UFYpPkc9q761/QQZ2wyGsYFMQqljnhxb5ChEHj+f806Vii92ApOwEIXLIP/jbHbG26z0APwkHLH/lTUwbE2a6RlR0N0tr++wylMW1H5dA+ndQtb63eANoBewSUuPZBjZWZTxbyn1KJyB8C5KZj9+Pm6EE9PMuERns= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1765478788398982.8976535837189; Thu, 11 Dec 2025 10:46:28 -0800 (PST) Received: from list by lists.xenproject.org with outflank-mailman.1184555.1507037 (Exim 4.92) (envelope-from ) id 1vTlft-0005In-OC; Thu, 11 Dec 2025 18:46:01 +0000 Received: by outflank-mailman (output) from mailman id 1184555.1507037; Thu, 11 Dec 2025 18:46:01 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1vTlft-0005GH-Ey; Thu, 11 Dec 2025 18:46:01 +0000 Received: by outflank-mailman (input) for mailman id 1184555; Thu, 11 Dec 2025 18:45:59 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1vTlfr-0003I8-Of for xen-devel@lists.xenproject.org; Thu, 11 Dec 2025 18:45:59 +0000 Received: from mail-ed1-x52b.google.com (mail-ed1-x52b.google.com [2a00:1450:4864:20::52b]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id a24f9c8a-d6c1-11f0-b15b-2bf370ae4941; Thu, 11 Dec 2025 19:45:59 +0100 (CET) Received: by mail-ed1-x52b.google.com with SMTP id 4fb4d7f45d1cf-640c6577120so745990a12.1 for ; Thu, 11 Dec 2025 10:45:59 -0800 (PST) Received: from EPUAKYIW02F7.. (pool185-5-253-4.as6723.net. [185.5.253.4]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-b7cfa5d0b0dsm345870466b.67.2025.12.11.10.45.57 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 11 Dec 2025 10:45:57 -0800 (PST) X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: a24f9c8a-d6c1-11f0-b15b-2bf370ae4941 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1765478758; x=1766083558; darn=lists.xenproject.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Wn2YE/N1073l2aBvkRQ/5nSVLZsSRqqVHeilL+wTO7s=; b=U2K1rrkP8HrRAsKbYGydTyb4MHfxJJMX2vqC62fUA0rEjBSDoPe3hP/Lw2ysqUn+Pb vlP/MgWQTsYAivoBfIP+3Ly25bW4QKD0aCoA1hkzFcoWLBcDkFZUaPRrk6Ufajo04is8 C2N0MyubzOkCVMIaTe1d/PDGIRm9CGJOu6/vlRRGKRIoJeBwZkh3sz6hLc4q89q825SO h3IOjKDisWq/jutw+glsa8IZ14MB89eEsHOWdeh+BY1K4EKMUP5he1NgUCe0VcCQ80fk apgqG7RtiMb4b8wtKMp8R9SjiG8oHr6lA3dC5LmhPi7xbuHRugRbWdJ17E9BMc5F9nLe RggQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1765478758; x=1766083558; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=Wn2YE/N1073l2aBvkRQ/5nSVLZsSRqqVHeilL+wTO7s=; b=UxIW7P/YKIBLnJhn8cSehbj0mdQPpNy3B0O87EDoLMq5ngMYFloL65ZHcefuDP+Nri VFOBAqpjj6IdREzODk8FA4gxG9yVRlmbttv2AZB1ToH47tU97Z2cpiCBcokUP7q+jEWG qgm4oBAczrweZ0W7KkSOX2XJmXcZvcIeOaVcsw9QMDWJOAm6PYS3yDWkSYINsSYZuV2G dseqCcHbzwI8Bm64TuOxp1VHmm84j172X9CT+2KwWXSy8ZO2r8wJCuuyBttEmTNZVUfa OTeVo/Z6kwgpdHsUbvn9MnDdSj4clA+gi2+RMH3a61D8zIPauSlt4dZmf6z4gJUIw81x Juvw== X-Gm-Message-State: AOJu0YxJ/86zoUuTO2PEe7N5iI8tqDwFSfJAuOrOJCsZjDuWZXK3GUVc yYFMIsCo4SW9k8HsG75E4nHZrMG6G7ANv9/ahRtS23fAjSXmSFPh4DymvQoS5BW0 X-Gm-Gg: AY/fxX5n77E9gqy2P20dmhlv55tB6jVKiOW3Qd6ewAkanOvgTy8frZbRaozvctTySaq zNHsEviaQyUvhngx+WhmGKm0ZZAVmjl40xyikW0KQsnOCMvxDSzwH+xploUKo8YgeyUgKKLwvxV R+Din+y/82CVd/75h50eYSjOP3dW8Hp3S3dN6jk1POO8DkjC7+QxCAOcAm+x9yUdLXskYu+952M 9TwFlkgWIAkniooDbM1sUxKiQwHYfLQNzBGGPjmhFKEGReU7WUJCn7XWTKcPl9KtIIHCJjXf/He JTTYjPpvTpPGR2wV8brVRC6SMS4+ZbhjOYIK7oVvlref0b+guzIdEx5GaGNpFIMiCCjZnDLkKnR GM2svAZNONoEi80W8f7kycSO2uG2cDqfinKxprSgu1AhlL+Y5+8+dufx7ggr/JryoL+Pjvx2WrQ obrZjFQWQ+pyeJK9nY5DoGCEFnp4iXynvoXSC6zJ1l X-Google-Smtp-Source: AGHT+IG/6t+gwx01s3fl4EBu4KpU0cbUis+6SFVAgfqdG0pLbcu3Q3IeGyxpzSPn7jghoibq6b4loQ== X-Received: by 2002:a17:907:7f04:b0:b7a:2ba7:18be with SMTP id a640c23a62f3a-b7ce84cc764mr748318866b.62.1765478758172; Thu, 11 Dec 2025 10:45:58 -0800 (PST) From: Mykola Kvach To: xen-devel@lists.xenproject.org Cc: Mykola Kvach , Mykola Kvach , Stefano Stabellini , Julien Grall , Bertrand Marquis , Michal Orzel , Volodymyr Babchuk , Saeed Nowshadi , Mykyta Poturai Subject: [PATCH v7 09/12] xen/arm: Resume memory management on Xen resume Date: Thu, 11 Dec 2025 20:43:33 +0200 Message-ID: X-Mailer: git-send-email 2.43.0 In-Reply-To: References: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @gmail.com) X-ZM-MESSAGEID: 1765478788580158500 Content-Type: text/plain; charset="utf-8" From: Mirela Simonovic The MMU must be enabled during the resume path before restoring context, as virtual addresses are used to access the saved context data. This patch adds MMU setup during resume by reusing the existing enable_secondary_cpu_mm function, which enables data cache and the MMU. Before the MMU is enabled, the content of TTBR0_EL2 is changed to point to init_ttbr (page tables used at runtime). Signed-off-by: Mirela Simonovic Signed-off-by: Saeed Nowshadi Signed-off-by: Mykyta Poturai Signed-off-by: Mykola Kvach --- Changes in v7: - no functional changes, just moved commit --- xen/arch/arm/arm64/head.S | 24 ++++++++++++++++++++++++ 1 file changed, 24 insertions(+) diff --git a/xen/arch/arm/arm64/head.S b/xen/arch/arm/arm64/head.S index 72c7b24498..596e960152 100644 --- a/xen/arch/arm/arm64/head.S +++ b/xen/arch/arm/arm64/head.S @@ -561,6 +561,30 @@ END(efi_xen_start) =20 #endif /* CONFIG_ARM_EFI */ =20 +#ifdef CONFIG_SYSTEM_SUSPEND + +FUNC(hyp_resume) + /* Initialize the UART if earlyprintk has been enabled. */ +#ifdef CONFIG_EARLY_PRINTK + bl init_uart +#endif + PRINT_ID("- Xen resuming -\r\n") + + bl check_cpu_mode + bl cpu_init + + ldr x0, =3Dstart + adr x20, start /* x20 :=3D paddr (start) */ + sub x20, x20, x0 /* x20 :=3D phys-offset */ + ldr lr, =3Dmmu_resumed + b enable_secondary_cpu_mm + +mmu_resumed: + b . +END(hyp_resume) + +#endif /* CONFIG_SYSTEM_SUSPEND */ + /* * Local variables: * mode: ASM --=20 2.43.0 From nobody Sun Dec 14 05:53:41 2025 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass(p=none dis=none) header.from=gmail.com ARC-Seal: i=1; a=rsa-sha256; t=1765478781; cv=none; d=zohomail.com; s=zohoarc; b=KqQJWSFmR2peGE1RgzhL42Ot3uDqoSm6dXp34TZoo9TZtsyZ+omFXDmkEiRYUGq/Xvu0FC+fn3yo8nq+uwunptvhCeqcoJs8CCVqJWy6LXHodHOsh4mRFD9/blyuc1nY95FT5nrgi5pgHOxdrLWoDvaboPdkGg6eMQ7bCeOhraU= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1765478781; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=OaZfuWRbnwPcnW+s48rbWKO1grkPvQJtBoI9rAf5Hcg=; b=I539ZjLuNzxsJOn0mAxhNm8+0B4rUAL8M8zOVY6PJ3NGE4VhGS6FIikERUz0SeixGfZl8ELFIkyrQExacI/eluDBgLeijQwe2t4xOH8B4Kl86E9F1Hx5DuAJLbdXhS3uiq6XWeeBecTDK87ZtByawVq9m9g5UkRZZsl2GFkopU8= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1765478781212806.6422853552928; Thu, 11 Dec 2025 10:46:21 -0800 (PST) Received: from list by lists.xenproject.org with outflank-mailman.1184556.1507051 (Exim 4.92) (envelope-from ) id 1vTlfv-0005ic-At; Thu, 11 Dec 2025 18:46:03 +0000 Received: by outflank-mailman (output) from mailman id 1184556.1507051; Thu, 11 Dec 2025 18:46:03 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1vTlfv-0005gO-2l; Thu, 11 Dec 2025 18:46:03 +0000 Received: by outflank-mailman (input) for mailman id 1184556; Thu, 11 Dec 2025 18:46:02 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1vTlfu-0003I9-Eg for xen-devel@lists.xenproject.org; Thu, 11 Dec 2025 18:46:02 +0000 Received: from mail-ed1-x533.google.com (mail-ed1-x533.google.com [2a00:1450:4864:20::533]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id a334c430-d6c1-11f0-9cce-f158ae23cfc8; Thu, 11 Dec 2025 19:46:00 +0100 (CET) Received: by mail-ed1-x533.google.com with SMTP id 4fb4d7f45d1cf-64198771a9bso828802a12.2 for ; Thu, 11 Dec 2025 10:46:00 -0800 (PST) Received: from EPUAKYIW02F7.. (pool185-5-253-4.as6723.net. [185.5.253.4]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-b7cfa5d0b0dsm345870466b.67.2025.12.11.10.45.58 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 11 Dec 2025 10:45:58 -0800 (PST) X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: a334c430-d6c1-11f0-9cce-f158ae23cfc8 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1765478760; x=1766083560; darn=lists.xenproject.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=OaZfuWRbnwPcnW+s48rbWKO1grkPvQJtBoI9rAf5Hcg=; b=C52yXMweSmOi7jHTFVsaG73CghSzs8FSjnivaGCnP3l+7OSj5weXg0khpS8ukU2zfZ qW632MP+HhJ2r6lqcq4E57g3TwtjNYVdzcg1lj8PK2o/wFH0hgVU7mh/WXpaew93YKiY Ie2ZbYcWO43g62MlkQuyHotyzvs0dBk0kcgG87d8iz9WIZFS3PCnnsohOtFk0ffjPTHe F5gh/7xTPntknn5A/9J3u4fQ2sL5GxZKPW53Nbfog7IlH24QyJcrf+rcF3nwo3aXAd3g vO3WSzMJe3QduB51qR/ceq6SWY/bSmUO790eqH/+g3TaeVtQyrSnn6j7YJvpGbSgpY4f Xg9g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1765478760; x=1766083560; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=OaZfuWRbnwPcnW+s48rbWKO1grkPvQJtBoI9rAf5Hcg=; b=d4XnlUPbEyPvZY9JUxUt+hJXd6pa+MR5zEOaoaRx9ugYn+M3vbuD3pO8wRUMMm4E59 EXAEXaT/sG9EyzdAdD6MNMaKp1+HDcqrTAP2WMpXkyqpIP8q3kMFKnVYxBb9zQhK4FVA z5oI+6nioDiYXW+0JbTelbNpMgmadh3ok57JcFHB9Huj1MnfdKShqKNTZ3suNXggP4eG qjJb3d0ISK+lQbT1MJsIfAfBzXQ4dZ77L4IYfN7LPu00fgWRaadA9upFyeZ4VVFCrghy bOQl07/5/rhEbDWo70J57YI057vcT3SFSRCZlbXqXg7tTS3vX2dLo1CC3eQPqHLpgDE3 X6pg== X-Gm-Message-State: AOJu0YyoMWHl512sC4tv3c54pi+rSm7W6LtpdFA+/0f6lLXsHUGW+i5c +fjiYw+hSq2q8TvCOqrXN01jwYBl7/dV+sOHGOyBg9nDOKMBv1LrAHRJpCTaQZJu X-Gm-Gg: AY/fxX5VyGaLQ3ufu/gDihPZIqZJaYr4pB2sA7XR1EPKeYu0GYKMe7j+R665L4JDaVo IZUCjo9d4T/ezqJjwCegPgfXvxPpqk8v7YN/+5aHqPpN7IL3tv4VLwWcOUwGkHa14lQ4np9nJ0s qsmx/r9L/AziSAIp7KagoUW+/havVOUG5uZSkhQ/bGWe3Ac9wif0fvJEGGKjP74qWpvMJaZzszR K5SCC5R0iglHT9+QQ7zWK8xq9qDjyaunWr4N1ekUtJy8rkyC3FQeyh3il0xDucngCkzVRb8skhd vqkhK2XWT+USQCToG54R+yK41qD5a1PKBWYUplkY6LawoiGpdo+XNzpsCXVjs9T+6ciecYVYqkg FMNMTiLIxOgwiMzRBDEfh9l5vnIxGykyq80Yip0WazmpQA8M2vG4SpNOGTEHAfWGB95K9nHO0Dw qGc+m3KveXc0zRqPN6/lzwa8OPK2dCr094lqRsg+3h X-Google-Smtp-Source: AGHT+IFhllThVrkjPTfuO84mpaywMKGq1Skw0Lhon1pOtwVx4+wXl4IJijzRncp8fHZyY+VLi8RDGQ== X-Received: by 2002:a17:907:940e:b0:b71:29f7:47dd with SMTP id a640c23a62f3a-b7ce8476094mr829779766b.46.1765478759399; Thu, 11 Dec 2025 10:45:59 -0800 (PST) From: Mykola Kvach To: xen-devel@lists.xenproject.org Cc: Mykola Kvach , Mykola Kvach , Stefano Stabellini , Julien Grall , Bertrand Marquis , Michal Orzel , Volodymyr Babchuk , Saeed Nowshadi , Mykyta Poturai Subject: [PATCH v7 10/12] xen/arm: Save/restore context on suspend/resume Date: Thu, 11 Dec 2025 20:43:34 +0200 Message-ID: <0dd3c42e1801674e19d2b3b3ca834b7bab62ca3f.1765472890.git.mykola_kvach@epam.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: References: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @gmail.com) X-ZM-MESSAGEID: 1765478782595158500 Content-Type: text/plain; charset="utf-8" From: Mirela Simonovic The context of CPU general purpose and system control registers must be saved on suspend and restored on resume. This is implemented in prepare_resume_ctx and before the return from the hyp_resume function. The prepare_resume_ctx must be invoked just before the PSCI system suspend call is issued to the ATF. The prepare_resume_ctx must return a non-zero value so that the calling 'if' statement evaluates to true, causing the system suspend to be invoked. Upon resume, the context saved on suspend will be restored, including the link register. Therefore, after restoring the context, the control flow will return to the address pointed to by the saved link register, which is the place from which prepare_resume_ctx was called. To ensure that the calling 'if' statement does not again evaluate to true and initiate system suspend, hyp_resume must return a zero value after restoring the context. Note that the order of saving register context into cpu_context structure must match the order of restoring. Support for ARM32 is not implemented. Instead, compilation fails with a build-time error if suspend is enabled for ARM32. Signed-off-by: Mirela Simonovic Signed-off-by: Saeed Nowshadi Signed-off-by: Mykyta Poturai Signed-off-by: Mykola Kvach --- Changes in v7: - no changes --- xen/arch/arm/Makefile | 1 + xen/arch/arm/arm64/head.S | 90 +++++++++++++++++++++++++++++- xen/arch/arm/include/asm/suspend.h | 26 +++++++++ xen/arch/arm/suspend.c | 14 +++++ 4 files changed, 130 insertions(+), 1 deletion(-) create mode 100644 xen/arch/arm/suspend.c diff --git a/xen/arch/arm/Makefile b/xen/arch/arm/Makefile index 7494a0f926..4a549a1ac0 100644 --- a/xen/arch/arm/Makefile +++ b/xen/arch/arm/Makefile @@ -51,6 +51,7 @@ obj-y +=3D setup.o obj-y +=3D shutdown.o obj-y +=3D smp.o obj-y +=3D smpboot.o +obj-$(CONFIG_SYSTEM_SUSPEND) +=3D suspend.o obj-$(CONFIG_SYSCTL) +=3D sysctl.o obj-y +=3D time.o obj-y +=3D traps.o diff --git a/xen/arch/arm/arm64/head.S b/xen/arch/arm/arm64/head.S index 596e960152..c6594c0bdd 100644 --- a/xen/arch/arm/arm64/head.S +++ b/xen/arch/arm/arm64/head.S @@ -562,6 +562,52 @@ END(efi_xen_start) #endif /* CONFIG_ARM_EFI */ =20 #ifdef CONFIG_SYSTEM_SUSPEND +/* + * int prepare_resume_ctx(struct cpu_context *ptr) + * + * x0 - pointer to the storage where callee's context will be saved + * + * CPU context saved here will be restored on resume in hyp_resume functio= n. + * prepare_resume_ctx shall return a non-zero value. Upon restoring context + * hyp_resume shall return value zero instead. From C code that invokes + * prepare_resume_ctx, the return value is interpreted to determine whether + * the context is saved (prepare_resume_ctx) or restored (hyp_resume). + */ +FUNC(prepare_resume_ctx) + /* Store callee-saved registers */ + stp x19, x20, [x0], #16 + stp x21, x22, [x0], #16 + stp x23, x24, [x0], #16 + stp x25, x26, [x0], #16 + stp x27, x28, [x0], #16 + stp x29, lr, [x0], #16 + + /* Store stack-pointer */ + mov x2, sp + str x2, [x0], #8 + + /* Store system control registers */ + mrs x2, VBAR_EL2 + str x2, [x0], #8 + mrs x2, VTCR_EL2 + str x2, [x0], #8 + mrs x2, VTTBR_EL2 + str x2, [x0], #8 + mrs x2, TPIDR_EL2 + str x2, [x0], #8 + mrs x2, MDCR_EL2 + str x2, [x0], #8 + mrs x2, HSTR_EL2 + str x2, [x0], #8 + mrs x2, CPTR_EL2 + str x2, [x0], #8 + mrs x2, HCR_EL2 + str x2, [x0], #8 + + /* prepare_resume_ctx must return a non-zero value */ + mov x0, #1 + ret +END(prepare_resume_ctx) =20 FUNC(hyp_resume) /* Initialize the UART if earlyprintk has been enabled. */ @@ -580,7 +626,49 @@ FUNC(hyp_resume) b enable_secondary_cpu_mm =20 mmu_resumed: - b . + /* Now we can access the cpu_context, so restore the context here = */ + ldr x0, =3Dcpu_context + + /* Restore callee-saved registers */ + ldp x19, x20, [x0], #16 + ldp x21, x22, [x0], #16 + ldp x23, x24, [x0], #16 + ldp x25, x26, [x0], #16 + ldp x27, x28, [x0], #16 + ldp x29, lr, [x0], #16 + + /* Restore stack pointer */ + ldr x2, [x0], #8 + mov sp, x2 + + /* Restore system control registers */ + ldr x2, [x0], #8 + msr VBAR_EL2, x2 + ldr x2, [x0], #8 + msr VTCR_EL2, x2 + ldr x2, [x0], #8 + msr VTTBR_EL2, x2 + ldr x2, [x0], #8 + msr TPIDR_EL2, x2 + ldr x2, [x0], #8 + msr MDCR_EL2, x2 + ldr x2, [x0], #8 + msr HSTR_EL2, x2 + ldr x2, [x0], #8 + msr CPTR_EL2, x2 + ldr x2, [x0], #8 + msr HCR_EL2, x2 + isb + + /* + * Since context is restored return from this function will appear + * as return from prepare_resume_ctx. To distinguish a return from + * prepare_resume_ctx which is called upon finalizing the suspend, + * as opposed to return from this function which executes on resum= e, + * we need to return zero value here. + */ + mov x0, #0 + ret END(hyp_resume) =20 #endif /* CONFIG_SYSTEM_SUSPEND */ diff --git a/xen/arch/arm/include/asm/suspend.h b/xen/arch/arm/include/asm/= suspend.h index 313d03ea59..d3a88ea979 100644 --- a/xen/arch/arm/include/asm/suspend.h +++ b/xen/arch/arm/include/asm/suspend.h @@ -3,6 +3,8 @@ #ifndef ARM_SUSPEND_H #define ARM_SUSPEND_H =20 +#include + struct domain; struct vcpu; =20 @@ -14,6 +16,30 @@ struct resume_info { =20 int arch_domain_resume(struct domain *d); =20 +#ifdef CONFIG_SYSTEM_SUSPEND +#ifdef CONFIG_ARM_64 +struct cpu_context { + register_t callee_regs[12]; + register_t sp; + register_t vbar_el2; + register_t vtcr_el2; + register_t vttbr_el2; + register_t tpidr_el2; + register_t mdcr_el2; + register_t hstr_el2; + register_t cptr_el2; + register_t hcr_el2; +} __aligned(16); +#else +#error "Define cpu_context structure for arm32" +#endif + +extern struct cpu_context cpu_context; + +int prepare_resume_ctx(struct cpu_context *ptr); +void hyp_resume(void); +#endif /* CONFIG_SYSTEM_SUSPEND */ + #endif /* ARM_SUSPEND_H */ =20 /* diff --git a/xen/arch/arm/suspend.c b/xen/arch/arm/suspend.c new file mode 100644 index 0000000000..e38566b0b7 --- /dev/null +++ b/xen/arch/arm/suspend.c @@ -0,0 +1,14 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ + +#include + +struct cpu_context cpu_context =3D {}; + +/* + * Local variables: + * mode: C + * c-file-style: "BSD" + * c-basic-offset: 4 + * indent-tabs-mode: nil + * End: + */ --=20 2.43.0 From nobody Sun Dec 14 05:53:41 2025 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass(p=none dis=none) header.from=gmail.com ARC-Seal: i=1; a=rsa-sha256; t=1765478785; cv=none; d=zohomail.com; s=zohoarc; b=JqUxPHTRWC0zdU2vk2gDyNwycrcBDSrqvAk4KAh3UeVeMciYJ23T9doNd7uVodyftsH2vMMCS21quisrncSm587s0Ivg5N0mxWnRetkxZo4ipViI2+ejQWnZuDjQ7tBMBYbl38Wj5jmbbqoy4rgkab+xEzxDK4F80aEMck4bo8k= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1765478785; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=81pZ/JS38wYFIEh/dUJCFhVaISUWS/A04krI0+LMlhc=; b=g3j8nBxFlPUKNMNA+hLH6slWyQYe9GDD0YVfnT3EXAf3wvzwTI/5nNExEp4koEZVPblw5aPYJzTD/d5EQ2PEJGYEm0j1ossoqWZZgyWOjSMYS6rUc3/cXoLaAovaHiOfj6kzUsO8I07hrLcBK878cbqmMTaooGfxmIdt/dJpuro= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1765478785195904.8089756551002; Thu, 11 Dec 2025 10:46:25 -0800 (PST) Received: from list by lists.xenproject.org with outflank-mailman.1184558.1507064 (Exim 4.92) (envelope-from ) id 1vTlfx-0006E5-SZ; Thu, 11 Dec 2025 18:46:05 +0000 Received: by outflank-mailman (output) from mailman id 1184558.1507064; Thu, 11 Dec 2025 18:46:05 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1vTlfx-0006DR-IT; Thu, 11 Dec 2025 18:46:05 +0000 Received: by outflank-mailman (input) for mailman id 1184558; Thu, 11 Dec 2025 18:46:03 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1vTlfv-0003I9-KF for xen-devel@lists.xenproject.org; Thu, 11 Dec 2025 18:46:03 +0000 Received: from mail-ej1-x636.google.com (mail-ej1-x636.google.com [2a00:1450:4864:20::636]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id a3f6eb7e-d6c1-11f0-9cce-f158ae23cfc8; Thu, 11 Dec 2025 19:46:01 +0100 (CET) Received: by mail-ej1-x636.google.com with SMTP id a640c23a62f3a-b72b495aa81so81991966b.2 for ; Thu, 11 Dec 2025 10:46:01 -0800 (PST) Received: from EPUAKYIW02F7.. (pool185-5-253-4.as6723.net. [185.5.253.4]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-b7cfa5d0b0dsm345870466b.67.2025.12.11.10.45.59 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 11 Dec 2025 10:46:00 -0800 (PST) X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: a3f6eb7e-d6c1-11f0-9cce-f158ae23cfc8 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1765478761; x=1766083561; darn=lists.xenproject.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=81pZ/JS38wYFIEh/dUJCFhVaISUWS/A04krI0+LMlhc=; b=kxgmo0qK7QjMeAxJbEdE7y0SawvyasAl4w5kAwni+EMdKbGLX/XuePL4LIl8mw7mi2 7diYC3JXXTIyxf1RM2SfOLSmam6001DDmYeKTeu/ZV6nX7Luna6k/itBfpAXEUh09u4s I/AaMjFiHYvOegu1w5SaVRRU/KCAHAI0BIAaX97QEzLgw7IawEdib4FU6WbEw0zPsqTQ ebCG5wnWUCITEKWzm6y3sSWaZ3D+qfkX1K9KXDZVC8TmqV24P65o4cD0zTJXNBLxPwbQ phMCoaanZW4Z0K9VbOeUwTOnMjgcvfRsDEQt4n5UGHIpdnL3ONEgokE6YaUxBO6GG4Ef PfZA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1765478761; x=1766083561; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=81pZ/JS38wYFIEh/dUJCFhVaISUWS/A04krI0+LMlhc=; b=pivqI+ChjySGcTXJ5bbWuJJjJyJpYyYzirDcARCT4XS5ruU6eysQDLKk/PPk7S9M5f PQMK65kdOk7BGcIUysl9A9Nwk0Zwg8LkargOegmYmGvNhrkodiOigPvDWWlOuRI4HPvp eBLCuYvWQv6iDCB3DiE9ToA1Of/Jsmhasq6NkwV0XZCnlR/1z8R6k82etX0HFr7SBI2W 09MVV7gDFA0LZNNYG73d43VZHeb8Ly1PN7ux14AOAcPnYRR1+ZVT8tsqdd4nsRzLroQk FFV8tUnxqM22c3k9LeF+FWI1WgfPmrNRigY3D7pnLK7eYMBaY7j5DZqzMi35rgl3o6Vv wDHw== X-Gm-Message-State: AOJu0YyFc3yESkfyzXRRiGw9amjczz2a4UtSBSCMOP3myz5Y26FGt6fd 49pz0rzlLymHCYNIs0g4fw31fJ1KGn7X9HHEKSJ+yL5tihJhqTpi/uty6oIv0dHn X-Gm-Gg: AY/fxX6ghitdgONyjfccQfBDqOdfN9RFRuxGDslzp7ZNFgKgekqz+VIbLhL21rb6uJ2 RYmfeXwidTfSOlUtOEA255ce7GL+SPNUvoHj/73Uj0DPsOHCFBsHZ7fUxFHuEPW8Xr7E64nBOht pLZtOLaBEsmlqjeZMahRfvG0p8W5boH7KkMZPyf2iO58kSRxfSH9vI76GMpyAx/E/nSOZzQS7OU 0riPjCpMuzS5WAwNrpR/s8hAirr660vur+jvlx3ZcumRmYvJgO6xmNp3NtBZ6SXXA+lBmwiVRiO 8rCj9BNAbuWL3ywwa6OHOFIqE1Pu9/jwYOf/BnjFUqulmKlFwrMhtokv+5JA9OQxI26KZkvO1a/ U9KwX3LBp+BT7lRFn/SKNBUvz+N/wglRa09iuaC7xVqcNenlaM+LJ7uYWhZDQAvcBOvllnv55ui N93DcxJd8AX+OFALU8A3VPzP1g+giz/LAf3eO5mRht X-Google-Smtp-Source: AGHT+IGCbNd/+PkXSwAoSrqHekaDi2vIULmpYvyFroTTVzA8/HLcn+jzgYkEFCZ7+nnOS/XPw90xzw== X-Received: by 2002:a17:907:6e9f:b0:b72:c103:88db with SMTP id a640c23a62f3a-b7ce83464c0mr650886566b.9.1765478760742; Thu, 11 Dec 2025 10:46:00 -0800 (PST) From: Mykola Kvach To: xen-devel@lists.xenproject.org Cc: Mykola Kvach , Mykola Kvach , Stefano Stabellini , Julien Grall , Bertrand Marquis , Michal Orzel , Volodymyr Babchuk , Saeed Nowshadi , Mykyta Poturai Subject: [PATCH v7 11/12] xen/arm: Implement PSCI SYSTEM_SUSPEND call (host interface) Date: Thu, 11 Dec 2025 20:43:35 +0200 Message-ID: <0808318bbede4bee6f36c08b06e2c0bb22c07fe9.1765472890.git.mykola_kvach@epam.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: References: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @gmail.com) X-ZM-MESSAGEID: 1765478786592158500 Content-Type: text/plain; charset="utf-8" From: Mirela Simonovic Invoke PSCI SYSTEM_SUSPEND to finalize Xen's suspend sequence on ARM64 plat= forms. Pass the resume entry point (hyp_resume) as the first argument to EL3. The = resume handler is currently a stub and will be implemented later in assembly. Igno= re the context ID argument, as is done in Linux. Only enable this path when CONFIG_SYSTEM_SUSPEND is set and PSCI version is >=3D 1.0. Signed-off-by: Mirela Simonovic Signed-off-by: Saeed Nowshadi Signed-off-by: Mykyta Poturai Signed-off-by: Mykola Kvach --- Changes in v7: - no changes --- xen/arch/arm/include/asm/psci.h | 1 + xen/arch/arm/psci.c | 23 ++++++++++++++++++++++- 2 files changed, 23 insertions(+), 1 deletion(-) diff --git a/xen/arch/arm/include/asm/psci.h b/xen/arch/arm/include/asm/psc= i.h index 48a93e6b79..bb3c73496e 100644 --- a/xen/arch/arm/include/asm/psci.h +++ b/xen/arch/arm/include/asm/psci.h @@ -23,6 +23,7 @@ int call_psci_cpu_on(int cpu); void call_psci_cpu_off(void); void call_psci_system_off(void); void call_psci_system_reset(void); +int call_psci_system_suspend(void); =20 /* Range of allocated PSCI function numbers */ #define PSCI_FNUM_MIN_VALUE _AC(0,U) diff --git a/xen/arch/arm/psci.c b/xen/arch/arm/psci.c index b6860a7760..c9d126b195 100644 --- a/xen/arch/arm/psci.c +++ b/xen/arch/arm/psci.c @@ -17,17 +17,20 @@ #include #include #include +#include =20 /* * While a 64-bit OS can make calls with SMC32 calling conventions, for * some calls it is necessary to use SMC64 to pass or return 64-bit values. - * For such calls PSCI_0_2_FN_NATIVE(x) will choose the appropriate + * For such calls PSCI_*_FN_NATIVE(x) will choose the appropriate * (native-width) function ID. */ #ifdef CONFIG_ARM_64 #define PSCI_0_2_FN_NATIVE(name) PSCI_0_2_FN64_##name +#define PSCI_1_0_FN_NATIVE(name) PSCI_1_0_FN64_##name #else #define PSCI_0_2_FN_NATIVE(name) PSCI_0_2_FN32_##name +#define PSCI_1_0_FN_NATIVE(name) PSCI_1_0_FN32_##name #endif =20 uint32_t psci_ver; @@ -60,6 +63,24 @@ void call_psci_cpu_off(void) } } =20 +int call_psci_system_suspend(void) +{ +#ifdef CONFIG_SYSTEM_SUSPEND + struct arm_smccc_res res; + + if ( psci_ver < PSCI_VERSION(1, 0) ) + return PSCI_NOT_SUPPORTED; + + /* 2nd argument (context ID) is not used */ + arm_smccc_smc(PSCI_1_0_FN_NATIVE(SYSTEM_SUSPEND), __pa(hyp_resume), &r= es); + return PSCI_RET(res); +#else + dprintk(XENLOG_WARNING, + "SYSTEM_SUSPEND not supported (CONFIG_SYSTEM_SUSPEND disabled)= \n"); + return PSCI_NOT_SUPPORTED; +#endif +} + void call_psci_system_off(void) { if ( psci_ver > PSCI_VERSION(0, 1) ) --=20 2.43.0 From nobody Sun Dec 14 05:53:41 2025 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass(p=none dis=none) header.from=gmail.com ARC-Seal: i=1; a=rsa-sha256; t=1765478872; cv=none; d=zohomail.com; s=zohoarc; b=ctWsjxFkq3ZZHipelA0yLxO0+rN+kYxMCdDgSB5IdxnS75jj7WPB8SGqVQLIwKK2+GLFhCFWuVGKckJ3A/cwjjTN2s5yfXRdtfc+pkky3u1JGJMjep0N/5GpIMgi0OzK/s8EfoCuIsYt5uj/54dN0wD0zUKKqFnvRxqki420jbE= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1765478872; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=RHRMDHeaPCNuYKtmY3W4kszvrqWpbqarndet93/8qB0=; b=JUi87/NQ+7R2KZ39FBJG5Igb78q/lH2B+brr5f86ti5uf3FF8zkRWIZJa5vIBMwBlDVBl6KuV7TfpwYNWEJw85dPuvUFoQktiiHJzXAnByOjp1fS8olKAjhqq8OPoxI9zEt2QUwgKaGhoAjrOH/gQw2yXlxI+joJTTsAzErFLeA= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1765478872836757.1777089263599; Thu, 11 Dec 2025 10:47:52 -0800 (PST) Received: from list by lists.xenproject.org with outflank-mailman.1184630.1507073 (Exim 4.92) (envelope-from ) id 1vTlhR-0001Ne-FT; Thu, 11 Dec 2025 18:47:37 +0000 Received: by outflank-mailman (output) from mailman id 1184630.1507073; Thu, 11 Dec 2025 18:47:37 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1vTlhR-0001NV-BJ; Thu, 11 Dec 2025 18:47:37 +0000 Received: by outflank-mailman (input) for mailman id 1184630; Thu, 11 Dec 2025 18:47:36 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1vTlfy-0003I8-9B for xen-devel@lists.xenproject.org; Thu, 11 Dec 2025 18:46:06 +0000 Received: from mail-ed1-x536.google.com (mail-ed1-x536.google.com [2a00:1450:4864:20::536]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id a4d50f04-d6c1-11f0-b15b-2bf370ae4941; Thu, 11 Dec 2025 19:46:03 +0100 (CET) Received: by mail-ed1-x536.google.com with SMTP id 4fb4d7f45d1cf-64979bee42aso645479a12.0 for ; Thu, 11 Dec 2025 10:46:03 -0800 (PST) Received: from EPUAKYIW02F7.. (pool185-5-253-4.as6723.net. [185.5.253.4]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-b7cfa5d0b0dsm345870466b.67.2025.12.11.10.46.00 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 11 Dec 2025 10:46:01 -0800 (PST) X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: a4d50f04-d6c1-11f0-b15b-2bf370ae4941 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1765478762; x=1766083562; darn=lists.xenproject.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=RHRMDHeaPCNuYKtmY3W4kszvrqWpbqarndet93/8qB0=; b=EwzoleVNbhAM9jSH607I+GHKlq3z7YtVxbQJVYnqYZTENvDTPpWGxtLYlfDSu73/6h zJ8POyKGDzgT1BOLPb5/d64gMEIcQ3/n2z/QCcFjak5vlp7Qu6ozMMBdf/3NuvqG7BwV dcNjzMzVE2orQAKq3UTLwRXApDYPD+8Mf64SNpxGRTsGLg16iibZ/Uy5VnGEYBHasUBs 0p8K/U2Ig4nhrKT5Xv2Fvo0l+YExKNz1/jR324hfMIvjem6i6v84ItLCNubgyFpRvZiu bl9ZPy8C+DZYDkx8dZL/w2ISKjRgW68yRKNPnPbRC5aOc4Gj0TEPAdHgQBAz4KXdqNGv KLPA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1765478762; x=1766083562; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=RHRMDHeaPCNuYKtmY3W4kszvrqWpbqarndet93/8qB0=; b=oZtLcDvK9jsXHSB6qOTNw2/GiGrRhpSXNvjkhUhn0SB4akdSyrIoEF1FSzDW8y3JHg vhvIQMwTjVuNvQXrEBVk3n0AmHEkm0RIU2sAbnSGpd290QeycxQ6EX0pTY6uFggws96M uK/Ql+DEqdBcmdVi87Yry5cbyF1gIK9/kpAEl4r4hzj/15ebBIbUyd/NHmOcyQkvRPqD V4oHWILfNdJ6ESCntBlPx765SRiyB39JI6cwFgRvoMo1ckLUTnahX+oy3D3pNrH1RcTk Z4TPvKNb4GHSPq310pbv/jvt+23Go+8AQd6K/+NxXNnhiX/soA580IJZbCiNSML2TS2o xZlA== X-Gm-Message-State: AOJu0YwhwKaAp5/QMHBLoNINiBt73U0IuvrZlflftbc8dku6iS2A8Iuf Qu4d8qLwXztzQgAkl6Bf5ErLkPGvbvhL+AFb+0MRAUyoplc8JyIXW68aj9mDLuMe X-Gm-Gg: AY/fxX6an+i/JMPTbjGGfnvsiqkkDrk4suiXUXY2bmaNnjdn6VV6bUwyMKLb8G+dFXE w4W02rpjOZ+3/LRqPR+qgIJRXGljybb9MFuo56QIHO+fvy1wwdFnOqqKTDS3KAEcfD3Wjp66crY XHep2TDS5lWVeUDgqWMf2gJ+ARq2PJY1qOtd5apxPRb1kW3P4tQyZS1qZhemnZcEKzkQItqSKbV UEmAC54BxMXPGq5/oha1Hhrz0Iet2lrywRm8zHLYvR+uqW77s+hINgeYGiziRcnvemHEi4MgKql VNh3Cg6bOVF3iq+d7tPcK9eGF7bGfcrnV0YOR2OTEeJyR9mBj9R2Lp/GIKT0MUnq3VrICbCyS34 03Xfhj+fTF1i6F2LZ+mRWH1DHTfA785HzXBhK+4JlkYCPdLBW18+YZwe+ko+nK3gmIyNgnGa8Rt m6fhqJPo4upN/LQoAQG8YIFY/mvBO5igAFP3tITmR3 X-Google-Smtp-Source: AGHT+IGVitvuV8prBdOMpYKRyd/3qrDnNbonRVxqr9ZG0MV5VKtiI2D+Iln3xxDRWYgevVysh/gGSg== X-Received: by 2002:a17:907:608c:b0:b76:f090:777b with SMTP id a640c23a62f3a-b7ce82dd99dmr760281066b.22.1765478762150; Thu, 11 Dec 2025 10:46:02 -0800 (PST) From: Mykola Kvach To: xen-devel@lists.xenproject.org Cc: Mykola Kvach , Mykola Kvach , Stefano Stabellini , Julien Grall , Bertrand Marquis , Michal Orzel , Volodymyr Babchuk , Andrew Cooper , Anthony PERARD , Jan Beulich , =?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= , Rahul Singh , Saeed Nowshadi , Mykyta Poturai Subject: [PATCH v7 12/12] xen/arm: Add support for system suspend triggered by control domain Date: Thu, 11 Dec 2025 20:43:36 +0200 Message-ID: <66fffded45f39599c62a1e4dad83c34f9de51d7d.1765472890.git.mykola_kvach@epam.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: References: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @gmail.com) X-ZM-MESSAGEID: 1765478873052158500 Content-Type: text/plain; charset="utf-8" From: Mirela Simonovic Trigger Xen suspend when the control domain initiates suspend via SHUTDOWN_suspend. Redirect system suspend to CPU#0 to ensure the suspend logic runs on the boot CPU, as required. Introduce full suspend/resume infrastructure gated by CONFIG_SYSTEM_SUSPEND, including logic to: - disable and enable non-boot physical CPUs - freeze and thaw domains - suspend and resume the GIC, timer, iommu and console - maintain system state before and after suspend On boot, init_ttbr is normally initialized during secondary CPU hotplug. On uniprocessor systems, this would leave init_ttbr uninitialized, causing resume to fail. To address this, the boot CPU now sets init_ttbr during suspend. Select HAS_SYSTEM_SUSPEND for ARM_64. Introduce CONFIG_HWDOM_SHUTDOWN_ON_SUSPEND (default y, disabled only on ARM= _64 with SYSTEM_SUSPEND) so we skip hwdom_shutdown() for SHUTDOWN_suspend when = the hardware domain survives suspend. Non-ARM behaviour is unchanged, and the ARM/system-suspend case no longer needs arch-specific checks in the code. Note: the code is guarded by CONFIG_SYSTEM_SUSPEND, which is currently only selected when UNSUPPORTED is set, and thus the functionality is neither ena= bled by default nor even built. Signed-off-by: Mirela Simonovic Signed-off-by: Saeed Nowshadi Signed-off-by: Mykyta Poturai Signed-off-by: Mykola Kvach --- Changes in V7: - Control domain is responsible for host suspend - Add an empty inline host_system_suspend() function when SYSTEM_SUSPEND config is disabled - Use IS_ENABLED() for config checking instead of #ifdef - Replace #ifdef checks in domain_shutdown() with IS_ENABLED() to simplify control flow. - Factor hardware domain shutdown condition into a helper (need_hwdom_shutdown()) to avoid preprocessor directives inside the funct= ion. - Squash with iommu suspend/resume commit --- xen/arch/arm/Kconfig | 1 + xen/arch/arm/include/asm/mm.h | 2 + xen/arch/arm/include/asm/suspend.h | 7 +- xen/arch/arm/mmu/smpboot.c | 2 +- xen/arch/arm/suspend.c | 139 +++++++++++++++++++++++++++++ xen/arch/arm/vpsci.c | 12 ++- xen/common/Kconfig | 5 ++ xen/common/domain.c | 7 +- xen/drivers/passthrough/arm/smmu.c | 10 +++ 9 files changed, 180 insertions(+), 5 deletions(-) diff --git a/xen/arch/arm/Kconfig b/xen/arch/arm/Kconfig index cf6af68299..86fa0b762a 100644 --- a/xen/arch/arm/Kconfig +++ b/xen/arch/arm/Kconfig @@ -8,6 +8,7 @@ config ARM_64 depends on !ARM_32 select 64BIT select HAS_FAST_MULTIPLY + select HAS_SYSTEM_SUSPEND if !MPU && UNSUPPORTED select HAS_VPCI_GUEST_SUPPORT if PCI_PASSTHROUGH =20 config ARM diff --git a/xen/arch/arm/include/asm/mm.h b/xen/arch/arm/include/asm/mm.h index ec2d2dc537..e4e795fa1f 100644 --- a/xen/arch/arm/include/asm/mm.h +++ b/xen/arch/arm/include/asm/mm.h @@ -364,6 +364,8 @@ static inline void page_set_xenheap_gfn(struct page_inf= o *p, gfn_t gfn) } while ( (y =3D cmpxchg(&p->u.inuse.type_info, x, nx)) !=3D x ); } =20 +void set_init_ttbr(lpae_t *root); + #endif /* __ARCH_ARM_MM__ */ /* * Local variables: diff --git a/xen/arch/arm/include/asm/suspend.h b/xen/arch/arm/include/asm/= suspend.h index d3a88ea979..2fb7cd8d56 100644 --- a/xen/arch/arm/include/asm/suspend.h +++ b/xen/arch/arm/include/asm/suspend.h @@ -38,7 +38,12 @@ extern struct cpu_context cpu_context; =20 int prepare_resume_ctx(struct cpu_context *ptr); void hyp_resume(void); -#endif /* CONFIG_SYSTEM_SUSPEND */ +void host_system_suspend(struct domain *d); + +#else /* !CONFIG_SYSTEM_SUSPEND */ + +static inline void host_system_suspend(struct domain *d) { (void)d; } +#endif =20 #endif /* ARM_SUSPEND_H */ =20 diff --git a/xen/arch/arm/mmu/smpboot.c b/xen/arch/arm/mmu/smpboot.c index 37e91d72b7..ff508ecf40 100644 --- a/xen/arch/arm/mmu/smpboot.c +++ b/xen/arch/arm/mmu/smpboot.c @@ -72,7 +72,7 @@ static void clear_boot_pagetables(void) clear_table(boot_third); } =20 -static void set_init_ttbr(lpae_t *root) +void set_init_ttbr(lpae_t *root) { /* * init_ttbr is part of the identity mapping which is read-only. So diff --git a/xen/arch/arm/suspend.c b/xen/arch/arm/suspend.c index e38566b0b7..c34b578e18 100644 --- a/xen/arch/arm/suspend.c +++ b/xen/arch/arm/suspend.c @@ -1,9 +1,148 @@ /* SPDX-License-Identifier: GPL-2.0-only */ =20 +#include #include =20 +#include +#include +#include +#include +#include +#include + struct cpu_context cpu_context =3D {}; =20 +/* Xen suspend. data identifies the domain that initiated suspend. */ +static void system_suspend(void *data) +{ + int status; + unsigned long flags; + struct domain *d =3D (struct domain *)data; + + BUG_ON(system_state !=3D SYS_STATE_active); + + system_state =3D SYS_STATE_suspend; + + printk("Xen suspending...\n"); + + freeze_domains(); + scheduler_disable(); + + /* + * Non-boot CPUs have to be disabled on suspend and enabled on resume + * (hotplug-based mechanism). Disabling non-boot CPUs will lead to PSCI + * CPU_OFF to be called by each non-boot CPU. Depending on the underly= ing + * platform capabilities, this may lead to the physical powering down = of + * CPUs. + */ + status =3D disable_nonboot_cpus(); + if ( status ) + { + system_state =3D SYS_STATE_resume; + goto resume_nonboot_cpus; + } + + time_suspend(); + + status =3D iommu_suspend(); + if ( status ) + { + system_state =3D SYS_STATE_resume; + goto resume_time; + } + + console_start_sync(); + status =3D console_suspend(); + if ( status ) + { + dprintk(XENLOG_ERR, "Failed to suspend the console, err=3D%d\n", s= tatus); + system_state =3D SYS_STATE_resume; + goto resume_end_sync; + } + + local_irq_save(flags); + status =3D gic_suspend(); + if ( status ) + { + system_state =3D SYS_STATE_resume; + goto resume_irqs; + } + + set_init_ttbr(xen_pgtable); + + /* + * Enable identity mapping before entering suspend to simplify + * the resume path + */ + update_boot_mapping(true); + + if ( prepare_resume_ctx(&cpu_context) ) + { + status =3D call_psci_system_suspend(); + /* + * If suspend is finalized properly by above system suspend PSCI c= all, + * the code below in this 'if' branch will never execute. Execution + * will continue from hyp_resume which is the hypervisor's resume = point. + * In hyp_resume CPU context will be restored and since link-regis= ter is + * restored as well, it will appear to return from prepare_resume_= ctx. + * The difference in returning from prepare_resume_ctx on system s= uspend + * versus resume is in function's return value: on suspend, the re= turn + * value is a non-zero value, on resume it is zero. That is why the + * control flow will not re-enter this 'if' branch on resume. + */ + if ( status ) + dprintk(XENLOG_WARNING, "PSCI system suspend failed, err=3D%d\= n", + status); + } + + system_state =3D SYS_STATE_resume; + update_boot_mapping(false); + + gic_resume(); + + resume_irqs: + local_irq_restore(flags); + + console_resume(); + resume_end_sync: + console_end_sync(); + + iommu_resume(); + + resume_time: + time_resume(); + + resume_nonboot_cpus: + /* + * The rcu_barrier() has to be added to ensure that the per cpu area is + * freed before a non-boot CPU tries to initialize it (_free_percpu_ar= ea() + * has to be called before the init_percpu_area()). This scenario occu= rs + * when non-boot CPUs are hot-unplugged on suspend and hotplugged on r= esume. + */ + rcu_barrier(); + enable_nonboot_cpus(); + scheduler_enable(); + thaw_domains(); + + system_state =3D SYS_STATE_active; + + printk("Resume (status %d)\n", status); + + domain_resume(d); +} + +static DECLARE_TASKLET(system_suspend_tasklet, system_suspend, NULL); + +void host_system_suspend(struct domain *d) +{ + system_suspend_tasklet.data =3D (void *)d; + /* + * The suspend procedure has to be finalized by the pCPU#0 (non-boot p= CPUs + * will be disabled during the suspend). + */ + tasklet_schedule_on_cpu(&system_suspend_tasklet, 0); +} + /* * Local variables: * mode: C diff --git a/xen/arch/arm/vpsci.c b/xen/arch/arm/vpsci.c index c4d616ec68..891d9f2cb9 100644 --- a/xen/arch/arm/vpsci.c +++ b/xen/arch/arm/vpsci.c @@ -4,6 +4,7 @@ #include =20 #include +#include #include #include #include @@ -210,6 +211,11 @@ static void do_psci_0_2_system_reset(void) domain_shutdown(d,SHUTDOWN_reboot); } =20 +static bool can_trigger_host_suspend(struct domain *d) +{ + return is_control_domain(d); +} + static int32_t do_psci_1_0_system_suspend(register_t epoint, register_t ci= d) { int32_t rc; @@ -221,8 +227,7 @@ static int32_t do_psci_1_0_system_suspend(register_t ep= oint, register_t cid) if ( is_64bit_domain(d) && is_thumb ) return PSCI_INVALID_ADDRESS; =20 - /* SYSTEM_SUSPEND is not supported for the hardware domain yet */ - if ( is_hardware_domain(d) ) + if ( !IS_ENABLED(CONFIG_SYSTEM_SUSPEND) && can_trigger_host_suspend(d)= ) return PSCI_NOT_SUPPORTED; =20 /* Ensure that all CPUs other than the calling one are offline */ @@ -249,6 +254,9 @@ static int32_t do_psci_1_0_system_suspend(register_t ep= oint, register_t cid) "SYSTEM_SUSPEND requested, epoint=3D%#"PRIregister", cid=3D%#"= PRIregister"\n", epoint, cid); =20 + if ( can_trigger_host_suspend(d) ) + host_system_suspend(d); + return rc; } =20 diff --git a/xen/common/Kconfig b/xen/common/Kconfig index 401d5046f6..31f54def0b 100644 --- a/xen/common/Kconfig +++ b/xen/common/Kconfig @@ -137,6 +137,11 @@ config HAS_EX_TABLE config HAS_FAST_MULTIPLY bool =20 +config HAS_HWDOM_SHUTDOWN_ON_SUSPEND + bool + default y + depends on !ARM_64 || !SYSTEM_SUSPEND + config HAS_IOPORTS bool =20 diff --git a/xen/common/domain.c b/xen/common/domain.c index 8f895108fd..b8a8c68428 100644 --- a/xen/common/domain.c +++ b/xen/common/domain.c @@ -1324,6 +1324,11 @@ void __domain_crash(struct domain *d) domain_shutdown(d, SHUTDOWN_crash); } =20 +static inline bool need_hwdom_shutdown(uint8_t reason) +{ + return IS_ENABLED(CONFIG_HAS_HWDOM_SHUTDOWN_ON_SUSPEND) || + reason !=3D SHUTDOWN_suspend; +} =20 int domain_shutdown(struct domain *d, u8 reason) { @@ -1340,7 +1345,7 @@ int domain_shutdown(struct domain *d, u8 reason) d->shutdown_code =3D reason; reason =3D d->shutdown_code; =20 - if ( is_hardware_domain(d) ) + if ( is_hardware_domain(d) && need_hwdom_shutdown(reason) ) hwdom_shutdown(reason); =20 if ( d->is_shutting_down ) diff --git a/xen/drivers/passthrough/arm/smmu.c b/xen/drivers/passthrough/a= rm/smmu.c index 22d306d0cb..45f29ef8ec 100644 --- a/xen/drivers/passthrough/arm/smmu.c +++ b/xen/drivers/passthrough/arm/smmu.c @@ -2947,6 +2947,13 @@ static void arm_smmu_iommu_domain_teardown(struct do= main *d) xfree(xen_domain); } =20 +#ifdef CONFIG_SYSTEM_SUSPEND +static int arm_smmu_suspend(void) +{ + return -ENOSYS; +} +#endif + static const struct iommu_ops arm_smmu_iommu_ops =3D { .page_sizes =3D PAGE_SIZE_4K, .init =3D arm_smmu_iommu_domain_init, @@ -2960,6 +2967,9 @@ static const struct iommu_ops arm_smmu_iommu_ops =3D { .map_page =3D arm_iommu_map_page, .unmap_page =3D arm_iommu_unmap_page, .dt_xlate =3D arm_smmu_dt_xlate_generic, +#ifdef CONFIG_SYSTEM_SUSPEND + .suspend =3D arm_smmu_suspend, +#endif }; =20 static struct arm_smmu_device *find_smmu(const struct device *dev) --=20 2.43.0