From nobody Sat May 30 11:15:55 2026 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass(p=none dis=none) header.from=gmail.com ARC-Seal: i=1; a=rsa-sha256; t=1778605812; cv=none; d=zohomail.com; s=zohoarc; b=MdR1wL9Iy4rE63zlTuET6Fu5yR/zy/8ytruIGbD3h4fWSOH2wNWtIBVTBkHOxnCXHA8afoG0412ZJxXXkjMw0qAHN56HHxPQz55/Fo21MPYHFl5NSr2KNYnH7G3lPesp/fRu0d7a1on6lchFAUT/VfzzN3i7gd+fIVtBLfcYmU0= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1778605812; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=j/x0rT1oB7OVwr8zqPawvId6h6lac1Z+SScMfAw/LeE=; b=GXTKoQM42jg+eR2Ft4THKIdG+mg5rulfqGy2jEfhtf3aQ8+Q98oEPeR090SFRIWUcxu2T5O7jH/zXa1OcCXG8qy61RNSfxGtIB9sk9Fa+xDkZJsAoFR503B6IJVhVMxaM0CMOR7H1VAqXm8PLXLFohuS8Liio9CilOVNUFfYYVE= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1778605812249162.0159354311802; Tue, 12 May 2026 10:10:12 -0700 (PDT) Received: from list by lists.xenproject.org with outflank-mailman.1307274.1578983 (Exim 4.92) (envelope-from ) id 1wMqc5-0005fJ-1v; Tue, 12 May 2026 17:09:45 +0000 Received: by outflank-mailman (output) from mailman id 1307274.1578983; Tue, 12 May 2026 17:09:45 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1wMqc4-0005ef-VH; Tue, 12 May 2026 17:09:44 +0000 Received: by outflank-mailman (input) for mailman id 1307274; Tue, 12 May 2026 17:09:43 +0000 Received: from mx.expurgate.net ([195.190.135.10]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1wMqc3-0005XT-D3 for xen-devel@lists.xenproject.org; Tue, 12 May 2026 17:09:43 +0000 Received: from mx.expurgate.net (helo=localhost) by mx.expurgate.net with esmtp id 1wMqc2-00HVXc-Q6 for xen-devel@lists.xenproject.org; Tue, 12 May 2026 19:09:42 +0200 Received: from [10.42.69.10] (helo=localhost) by localhost with ESMTP (eXpurgate MTA 0.9.1) (envelope-from ) id 6a035eaa-5cb7-0a2a0a5109dd-0a2a450a84bc-40 for ; Tue, 12 May 2026 19:09:42 +0200 Received: from [209.85.128.50] (helo=mail-wm1-f50.google.com) by tlsNG-4011c0.mxtls.expurgate.net with ESMTPS (eXpurgate 4.56.1) (envelope-from ) id 6a035ed6-56b3-0a2a450a0019-d1558032c817-3 for ; Tue, 12 May 2026 19:09:42 +0200 Received: by mail-wm1-f50.google.com with SMTP id 5b1f17b1804b1-4891e5b9c1fso53835975e9.2 for ; Tue, 12 May 2026 10:09:42 -0700 (PDT) Received: from EPUAKYIW02F7.. ([45.12.26.18]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-48fc8cccf90sm10838845e9.0.2026.05.12.10.09.40 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 12 May 2026 10:09:41 -0700 (PDT) X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" Authentication-Results: eu.smtp.expurgate.cloud; dkim=pass header.s=20251104 header.d=gmail.com header.i="@gmail.com" header.h="Content-Transfer-Encoding:MIME-Version:References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1778605782; x=1779210582; darn=lists.xenproject.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=j/x0rT1oB7OVwr8zqPawvId6h6lac1Z+SScMfAw/LeE=; b=aX1Lyq+Px+KI01SjPa6ASdPvaDCV/EOHsqRaNMl06VQxpdbqvQWd02AgwMkqMUFcaE QjGWmpMj0NPkzYPpRpCQ+DrHGByu8Sk/yKTpuY5mXYONiC+44b7JmILgKehRzmo3IaSn 7/D2dx8IsHA0mESVHeqyVTxfYdq5O9Pjhn3xoydCPAnXah49o7UEbHeV7mIa9ojyUvyL LxzeBVqwJhMXENYiKWfHeRfSUKR0PhtoBv4903Yqm+xlMpFSUo5FfpPjcATIphlqRG1R DbnNLqoOJiAzzieB859na/lp7dO03blR8fE7mGXzMcFMTST3Dvgo8t0r26Iwx1wBVacS af3Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1778605782; x=1779210582; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=j/x0rT1oB7OVwr8zqPawvId6h6lac1Z+SScMfAw/LeE=; b=aeQ4Rj46u2cWh1zbJ1jzHjetGELiYlHLwHcpmqcAKstVWSdLj7xV7N/vtooYvQYnEP nvIY/TEQSC5dS+/eJU0qu1Uzhv2DHKagQPa07DtVlil375VTTuUokGQDxjaVL9tg7Ru2 tlEDD2xj6aaJFdIYHUuOxZFSLjHbpY2F4AjzvSA8jvul/Dzyz4Cjvgc4w9uf3txlalqG BeqiojUtHduk4L7oszvWtUc1+hlwq8WiVqQ1nmFTwmDuBT+14yS8ZtjuUruvZk1sQUov wvQ9BduhU4vkYclLZIpz3hFWBB8nZxzGip7SUM803AJddvHKnzzjORoI/3rr5lkJnvi6 /G1Q== X-Gm-Message-State: AOJu0YxUHd8GDsKxtPMdf8k9kKkY/jYllYsKeolYdqkK9o1sykOfvP27 pbmjw+FJ3J+WqcKl1xMPMipa4pCXfHuMY7ScH+YfLXTM0uBmKtEVVDTYrDhd1g== X-Gm-Gg: Acq92OFAOmArw3ZTNgdlkcGi+Ur/uQ6qPV5dTLUpGSUGVBmHceW8xBkqcEFaBcAFxeD xul8tT2nAz1uSM4DNaXqKQIWbDlarM4KzBKjPE7MnV14pGvoRFGrtkUbsQEW5zTGs5fc0H9bntI lUqpyFO7dIm9AKK9zN+ZljStyvHMz4DIh9G8rjF8mIFlRzHE9yeORKJkeSeN83t7lZLDpmKIehv PzGLFiln+/phqyD6/pfK52EONEy7JXn3onCFMbnDx2OpkBOHS/+uhmK5EDXlP2yTdECS61KIwo3 IZRjFSTUpboLmuMxvzYfIhLpN//a27fh92XRtwqyQaYMh+f/M6w68jEDdJ/NTtdN5BAqsiqMp1O d/Rgcpe70CzEauyg70SAIymnVUzBEmZpUF5L6lQAig4lqi47CJ1NZg3fVa8Yj2HTIYozkkHi+pM krQ6aqSy9QiMu8RK1L2VXsRZiM8g== X-Received: by 2002:a05:600c:a11a:b0:48e:635a:18d2 with SMTP id 5b1f17b1804b1-48e8fe4dc0bmr45550805e9.2.1778605781824; Tue, 12 May 2026 10:09:41 -0700 (PDT) From: Mykola Kvach To: xen-devel@lists.xenproject.org Cc: Mykola Kvach , Stefano Stabellini , Julien Grall , Bertrand Marquis , Michal Orzel , Volodymyr Babchuk , Julien Grall , Luca Fancellu Subject: [PATCH v9 01/13] xen/arm: Add suspend and resume timer helpers Date: Tue, 12 May 2026 20:07:08 +0300 Message-ID: <178c6b2d729d6d341e3647cb63f36af9df6c3301.1778605274.git.mykola_kvach@epam.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: References: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-purgate-ID: tlsNG-4011c0/1778605782-6FF598B7-1695E299/0/0 X-purgate-type: clean X-purgate-size: 5064 X-ZohoMail-DKIM: pass (identity @gmail.com) X-ZM-MESSAGEID: 1778605813942154100 Content-Type: text/plain; charset="utf-8" From: Mirela Simonovic Timer interrupts must be disabled while the system is suspended to prevent spurious wake-ups. Suspending timers in Xen consists of disabling the physical timer and the hypervisor timer on the current CPU. The virtual timer does not need explicit handling here, as it is already disabled on vCPU context switch and its state is restored per-vCPU on the next context restore. Resuming consists of raising TIMER_SOFTIRQ, which prompts the generic timer code to reprogram the hypervisor timer with the correct timeout. Xen does not use or expose the physical timer, so it remains disabled across suspend/resume. Introduce a new helper, disable_phys_hyp_timers(), to encapsulate disabling of the physical and hypervisor timers. Signed-off-by: Mirela Simonovic Signed-off-by: Saeed Nowshadi Signed-off-by: Mykola Kvach Acked-by: Julien Grall Reviewed-by: Luca Fancellu --- Changes in V7: - Dropped EL1/EL2 wording; use "physical timer" and "hypervisor timer" - Renamed helper to disable_phys_hyp_timers() to reflect its actual scope - Clarified virtual timer handling (disabled on vCPU switch-out, restored on context restore) and added comments in suspend/resume paths - Added resume comment explaining which timers are restored by TIMER_SOFTIRQ --- xen/arch/arm/include/asm/time.h | 5 ++++ xen/arch/arm/time.c | 44 ++++++++++++++++++++++++++++----- 2 files changed, 43 insertions(+), 6 deletions(-) diff --git a/xen/arch/arm/include/asm/time.h b/xen/arch/arm/include/asm/tim= e.h index c194dbb9f5..9313b157ea 100644 --- a/xen/arch/arm/include/asm/time.h +++ b/xen/arch/arm/include/asm/time.h @@ -105,6 +105,11 @@ void preinit_xen_time(void); =20 void force_update_vcpu_system_time(struct vcpu *v); =20 +#ifdef CONFIG_SYSTEM_SUSPEND +void time_suspend(void); +void time_resume(void); +#endif /* CONFIG_SYSTEM_SUSPEND */ + #endif /* __ARM_TIME_H__ */ /* * Local variables: diff --git a/xen/arch/arm/time.c b/xen/arch/arm/time.c index 6955b2788f..fff8e4aca6 100644 --- a/xen/arch/arm/time.c +++ b/xen/arch/arm/time.c @@ -296,6 +296,14 @@ static void check_timer_irq_cfg(unsigned int irq, cons= t char *which) static DEFINE_PER_CPU_READ_MOSTLY(struct irqaction, irq_hyp); static DEFINE_PER_CPU_READ_MOSTLY(struct irqaction, irq_virt); =20 +/* Disable physical and hypervisor timers on the current CPU */ +static inline void disable_phys_hyp_timers(void) +{ + WRITE_SYSREG(0, CNTP_CTL_EL0); /* Physical timer disabled */ + WRITE_SYSREG(0, CNTHP_CTL_EL2); /* Hypervisor's timer disabled */ + isb(); +} + /* Set up the timer interrupt on this CPU */ void init_timer_interrupt(void) { @@ -306,9 +314,7 @@ void init_timer_interrupt(void) WRITE_SYSREG64(0, CNTVOFF_EL2); /* No VM-specific offset */ /* Do not let the VMs program the physical timer, only read the physic= al counter */ WRITE_SYSREG(CNTHCTL_EL2_EL1PCTEN, CNTHCTL_EL2); - WRITE_SYSREG(0, CNTP_CTL_EL0); /* Physical timer disabled */ - WRITE_SYSREG(0, CNTHP_CTL_EL2); /* Hypervisor's timer disabled */ - isb(); + disable_phys_hyp_timers(); =20 hyp_action->name =3D "hyptimer"; hyp_action->handler =3D htimer_interrupt; @@ -333,9 +339,7 @@ void init_timer_interrupt(void) */ static void deinit_timer_interrupt(void) { - WRITE_SYSREG(0, CNTP_CTL_EL0); /* Disable physical timer */ - WRITE_SYSREG(0, CNTHP_CTL_EL2); /* Disable hypervisor's timer */ - isb(); + disable_phys_hyp_timers(); =20 release_irq(timer_irq[TIMER_HYP_PPI], NULL); release_irq(timer_irq[TIMER_VIRT_PPI], NULL); @@ -375,6 +379,34 @@ void domain_set_time_offset(struct domain *d, int64_t = time_offset_seconds) /* XXX update guest visible wallclock time */ } =20 +#ifdef CONFIG_SYSTEM_SUSPEND + +void time_suspend(void) +{ + /* CNTV already disabled by virt_timer_save() during vcpu context swit= ch. */ + disable_phys_hyp_timers(); +} + +void time_resume(void) +{ + /* + * Raising TIMER_SOFTIRQ triggers generic timer code to reprogram the + * hypervisor timer with the correct timeout (not known here). + * + * Xen doesn't use or expose the physical timer, so it remains disabled + * across suspend/resume. + * + * The virtual timer state is restored per-vCPU on the next context sw= itch. + * + * No further action is needed to restore timekeeping after power down, + * since the system counter is unaffected. See ARM DDI 0487 L.a, D12.1= .2 + * "The system counter must be implemented in an always-on power domai= n." + */ + raise_softirq(TIMER_SOFTIRQ); +} + +#endif /* CONFIG_SYSTEM_SUSPEND */ + static int cpu_time_callback(struct notifier_block *nfb, unsigned long action, void *hcpu) --=20 2.43.0 From nobody Sat May 30 11:15:55 2026 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass(p=none dis=none) header.from=gmail.com ARC-Seal: i=1; a=rsa-sha256; t=1778605824; cv=none; d=zohomail.com; s=zohoarc; b=lZHp8SwEESwvNFybF8nTT2sQlsVKBkG9I98LEGLHKnZb5YnVsu8PoDxmWP8z5EhbpM/B0Do6e/cK9q8kaqkZvoNf555twfNjAW0vCPbpnTms47vFEBlYg83HjUWMxAT3UeCfARxG5qH3x61007dSeNnVnj1g3fVSdiHVvalVHzQ= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1778605824; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=b8mWSeU/iVgrcW8itSSelUy+85kdfMSQifdUDUET2DY=; b=FPHlLMJtFwJ6r/z5d9D3133BPhvZobo/BKRtiD1eUji+eCPMZpsX9teu6XLOJx2fMMNKGOeplKpiQAyi04Y/2g9QsP0j0udiXxY8/05cAPv5fxjXTsyy3aNuqA85/1IZ5OBRbZ5VD6S5Zto4i3sNVHhvC2lyJUrIvwne3X69wKw= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1778605824669160.8716215288706; Tue, 12 May 2026 10:10:24 -0700 (PDT) Received: from list by lists.xenproject.org with outflank-mailman.1307275.1578998 (Exim 4.92) (envelope-from ) id 1wMqc6-00062Q-DI; Tue, 12 May 2026 17:09:46 +0000 Received: by outflank-mailman (output) from mailman id 1307275.1578998; Tue, 12 May 2026 17:09:46 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1wMqc6-00062I-AP; Tue, 12 May 2026 17:09:46 +0000 Received: by outflank-mailman (input) for mailman id 1307275; Tue, 12 May 2026 17:09:44 +0000 Received: from mx.expurgate.net ([195.190.135.10]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1wMqc4-0005cE-PB for xen-devel@lists.xenproject.org; Tue, 12 May 2026 17:09:44 +0000 Received: from mx.expurgate.net (helo=localhost) by mx.expurgate.net with esmtp id 1wMqc4-00AZDT-64 for xen-devel@lists.xenproject.org; Tue, 12 May 2026 19:09:44 +0200 Received: from [10.42.69.3] (helo=localhost) by localhost with ESMTP (eXpurgate MTA 0.9.1) (envelope-from ) id 6a035ece-e002-0a2a0a5209dd-0a2a4503d7ac-8 for ; Tue, 12 May 2026 19:09:44 +0200 Received: from [209.85.128.42] (helo=mail-wm1-f42.google.com) by tlsNG-33051d.mxtls.expurgate.net with ESMTPS (eXpurgate 4.56.1) (envelope-from ) id 6a035ed8-672d-0a2a45030019-d155802ac93e-3 for ; Tue, 12 May 2026 19:09:44 +0200 Received: by mail-wm1-f42.google.com with SMTP id 5b1f17b1804b1-488b8bc6bc9so36427515e9.3 for ; Tue, 12 May 2026 10:09:44 -0700 (PDT) Received: from EPUAKYIW02F7.. ([45.12.26.18]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-48fc8cccf90sm10838845e9.0.2026.05.12.10.09.41 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 12 May 2026 10:09:42 -0700 (PDT) X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" Authentication-Results: eu.smtp.expurgate.cloud; dkim=pass header.s=20251104 header.d=gmail.com header.i="@gmail.com" header.h="Content-Transfer-Encoding:MIME-Version:References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1778605783; x=1779210583; darn=lists.xenproject.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=b8mWSeU/iVgrcW8itSSelUy+85kdfMSQifdUDUET2DY=; b=LTPP27S+Io1RZQjOui0W3qz/JnNkHBqUjIVY3wmDAlNzQfCa8vrYp7qL/fbtTIc7OQ hJpFIsh2h5sMTl/qwY8UAzcwT2Kpg3yrNxFtWFNIV+nCb+LwmGImTpkGGCOCTQ0XHyoQ O/A3/ro7kMd3J6WHhLKBSD8jpv6TQD0pudvISbCvlhl2A+ae3Bh5uTEJ3NYavcU3FcT8 wNgSXYroYryWf49b1i8T4mQmmmz9/dxA4YOIyiRvmpqEJLHv5gSYyGwz3qUlnT+89gDh Re+0wJdz+1zFqw8TVBKk48nyyWHYsXVxF5sesObe1xJ9XkdpilXZjx1QSP22P5D11F/T y4cA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1778605783; x=1779210583; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=b8mWSeU/iVgrcW8itSSelUy+85kdfMSQifdUDUET2DY=; b=E10hxMnlujv+pfw7jv8DZjTaWMhSMJXWj7/1I2lo/dOYC6A49CUEBpVDF57K1vnaKv cqm3HOZ4lASRg5sjc4OXe4ZenQzLvOMOH/g91MnmZ4aQHQ/HWU+Ipvm9BFrUW0ofq/Kt i/fdtBEgO+rnS4B/+m5s5oGll/mFlLHZrRUxm36VXLnrhpmvqaIzA7aqTnxzis57nqPb tQbdVEeKVAJsjteJaUrACO6GvF7wyl4vKTttgKA6J7gsgr3z/AD2DAaDBXLf/YdPfidv dBXZH6VDtFbgcN4iH6HJYnlxdskEg66ELZCRlz4WGalodUT7Oi5RKkwkCczcZt1Mdbq1 jqew== X-Gm-Message-State: AOJu0YyUG4zpY0blEwCqNL+qBeZ4/U5LDaelOzGlfD5Y5uCwgcdBOgaC H15l6fdFYN7bIKeorzrwNFeugdKVSeI8pihQzBUdJtIvxYhcLIiboKdWl5rsgg== X-Gm-Gg: Acq92OG/R6lU0U+iucan/6yYLwJY678mJXxXDHRzumHdDiSlaNNoQJSacgSrjQZ0ApS /CRArLJOBn7d6YEt55beP2zSebZQmHFk372dbTtmFldHbEfAyeSeKLLlXwWeTg5Vi5XDFmM5tqY NbW5wcQPHy1rvrtC3rd92r3E86NbZVqD/YWv2cTSFtvCxaCNX8xRMXHrF0kqpW5IfNamLZpojdb Jhoq4UTrQZVhEgovWCb071++vmZ9UEc3/YSB0tFwq5EVmK/cZnfDUloDSkb+Ixg0FwvBMuEq7bL sjpyfDoNwufLkNuSDfs5rEofr5eqIsysTS6dSd/1q6gfi7K8SA6wOjIJeJS+Rf423uOfO1E9jIN C8oo9WhbmXHbk6XwFOaihbcnV3oa5lxPCFv8LnjU8PNzzAMh0IppvtTCw3eWzmwsE2KIJCpQ+b2 s8z+thUo5EnuwcxZrj32xmvke9Og== X-Received: by 2002:a05:600c:c166:b0:48a:93f8:dd02 with SMTP id 5b1f17b1804b1-48e8fe7227cmr61787625e9.14.1778605783038; Tue, 12 May 2026 10:09:43 -0700 (PDT) From: Mykola Kvach To: xen-devel@lists.xenproject.org Cc: Mykola Kvach , Stefano Stabellini , Julien Grall , Bertrand Marquis , Michal Orzel , Volodymyr Babchuk Subject: [PATCH v9 02/13] xen/arm: gic-v2: Implement GIC suspend/resume functions Date: Tue, 12 May 2026 20:07:09 +0300 Message-ID: <613c8d67a66924ea62c2436dac14708d939f6784.1778605274.git.mykola_kvach@epam.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: References: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-purgate-ID: tlsNG-33051d/1778605784-37141938-096D7344/0/0 X-purgate-type: clean X-purgate-size: 11669 X-ZohoMail-DKIM: pass (identity @gmail.com) X-ZM-MESSAGEID: 1778605825757158500 Content-Type: text/plain; charset="utf-8" From: Mirela Simonovic System suspend may lead to a state where GIC would be powered down. Therefore, Xen should save/restore the context of GIC on suspend/resume. Note that the context consists of states of registers which are controlled by the hypervisor. Other GIC registers which are accessible by guests are saved/restored on context switch. Transient physical SGI pending state (GICD_CPENDSGIRn/GICD_SPENDSGIRn) is intentionally excluded. CPU-interface active-priority state is also not restored across suspend/resume. Xen reaches the final suspend path at a quiescent point, so there is no active-priority execution context to replay after resume. Enforce this with a runtime check after disabling the CPU interface: if any implemented GICC_APRn word is still non-zero, restore GICC_CTLR and abort suspend with -EBUSY. This does not apply to distributor active state. With GICv2 EOImode=3D=3D1, EOIR only drops the interrupt priority; final deactivation is a separate step. For guest-routed interrupts, Xen can have already EOIed the physical IRQ while deactivation is still pending on the vGIC/GICV path. Therefore GICD_ISACTIVER is preserved as architectural in-flight interrupt state. Signed-off-by: Mirela Simonovic Signed-off-by: Saeed Nowshadi Signed-off-by: Mykyta Poturai Signed-off-by: Mykola Kvach --- Changes in V9: - Skip saving/restoring GICD_ITARGETSR0..7 because SGI/PPI target registers hold no state (read-only on MP, RAZ/WI on UP). - Add a runtime GICC_APRn quiescence check after disabling the CPU interface, and restore GICC_CTLR before returning -EBUSY. Changes in V8: - disable cpu interface + distributor before suspend - change 0xffffffff to GENMASK; - cosmetic changes; Changes in V7: - Allocate one contiguous memory block for the GICv2 dist suspend context. - gicv2_resume() no longer unconditionally re-enables the distributor/CPU interface; it now writes back the saved CTLR values as-is. - gicv2_alloc_context() now returns 0 on success and panics on failure, since suspend context allocation is not recoverable. --- xen/arch/arm/gic-v2.c | 181 +++++++++++++++++++++++++++++++++ xen/arch/arm/gic.c | 29 ++++++ xen/arch/arm/include/asm/gic.h | 12 +++ 3 files changed, 222 insertions(+) diff --git a/xen/arch/arm/gic-v2.c b/xen/arch/arm/gic-v2.c index 43a379fdda..1970c4feb9 100644 --- a/xen/arch/arm/gic-v2.c +++ b/xen/arch/arm/gic-v2.c @@ -1108,6 +1108,178 @@ static int gicv2_iomem_deny_access(struct domain *d) return iomem_deny_access(d, mfn, mfn + nr - 1); } =20 +#ifdef CONFIG_SYSTEM_SUSPEND + +/* This struct represents block of 32 IRQs */ +struct irq_block { + uint32_t icfgr[2]; /* 2 registers of 16 IRQs each */ + uint32_t ipriorityr[8]; + uint32_t isenabler; + uint32_t isactiver; + uint32_t itargetsr[8]; +}; + +/* GICv2 registers to be saved/restored on system suspend/resume */ +struct gicv2_context { + /* GICC context */ + struct cpu_ctx { + uint32_t ctlr; + uint32_t pmr; + uint32_t bpr; + } cpu; + + /* GICD context */ + struct dist_ctx { + uint32_t ctlr; + /* Includes banked SGI/PPI state for the boot CPU. */ + struct irq_block *irqs; + } dist; +}; + +static struct gicv2_context gic_ctx; + +#define GICV2_NR_APRS 4 + +static int gicv2_suspend(void) +{ + unsigned int i, blocks =3D DIV_ROUND_UP(gicv2_info.nr_lines, 32); + + /* Save GICC_CTLR configuration. */ + gic_ctx.cpu.ctlr =3D readl_gicc(GICC_CTLR); + + /* Quiesce the GIC CPU interface before suspend. */ + gicv2_cpu_disable(); + + /* + * Check the active-priority state for the group Xen drives through the + * CPU interface. GICC_CTL_ENABLE enables Group 0 without SecurityExtn= and + * Group 1 in Xen's Non-secure view with SecurityExtn, and in both cas= es + * the relevant state is visible through GICC_APRn. Only zero/non-zero + * matters here, not the implementation-defined bit layout. + */ + for ( i =3D 0; i < GICV2_NR_APRS; i++ ) + { + uint32_t apr =3D readl_gicc(GICC_APR + i * 4); + + if ( !apr ) + continue; + + printk(XENLOG_ERR "GICv2: suspend aborted: GICC_APR%u=3D%#08x\n", = i, apr); + writel_gicc(gic_ctx.cpu.ctlr, GICC_CTLR); + return -EBUSY; + } + + /* Save GICD configuration */ + gic_ctx.dist.ctlr =3D readl_gicd(GICD_CTLR); + writel_gicd(0, GICD_CTLR); + + gic_ctx.cpu.pmr =3D readl_gicc(GICC_PMR); + gic_ctx.cpu.bpr =3D readl_gicc(GICC_BPR); + + for ( i =3D 0; i < blocks; i++ ) + { + struct irq_block *irqs =3D gic_ctx.dist.irqs + i; + size_t j, off =3D i * sizeof(irqs->isenabler); + + irqs->isenabler =3D readl_gicd(GICD_ISENABLER + off); + + /* + * Save distributor active state as part of the hypervisor-owned + * physical interrupt state. In GICv2 EOImode=3D=3D1, EOIR only dr= ops the + * priority; final deactivation is separate. For guest-routed + * interrupts, Xen may have EOIed the physical IRQ while the guest= /vGIC + * side still owns the deactivate step. Therefore GICD_ISACTIVER c= an + * legitimately remain set even though transient SGI pending state= and + * CPU-interface active-priority state are expected to be quiesced= here. + */ + irqs->isactiver =3D readl_gicd(GICD_ISACTIVER + off); + + off =3D i * sizeof(irqs->ipriorityr); + for ( j =3D 0; j < ARRAY_SIZE(irqs->ipriorityr); j++ ) + irqs->ipriorityr[j] =3D readl_gicd(GICD_IPRIORITYR + off + j *= 4); + + /* + * GICD_ITARGETSR0..7 cover SGIs/PPIs and hold no state to save: + * they are read-only on multiprocessor implementations and RAZ/WI + * on uniprocessor implementations. + */ + if ( i ) + { + off =3D i * sizeof(irqs->itargetsr); + for ( j =3D 0; j < ARRAY_SIZE(irqs->itargetsr); j++ ) + irqs->itargetsr[j] =3D readl_gicd(GICD_ITARGETSR + off + j= * 4); + } + + off =3D i * sizeof(irqs->icfgr); + for ( j =3D 0; j < ARRAY_SIZE(irqs->icfgr); j++ ) + irqs->icfgr[j] =3D readl_gicd(GICD_ICFGR + off + j * 4); + } + + return 0; +} + +static void gicv2_resume(void) +{ + unsigned int i, blocks =3D DIV_ROUND_UP(gicv2_info.nr_lines, 32); + + gicv2_cpu_disable(); + /* Disable distributor */ + writel_gicd(0, GICD_CTLR); + + for ( i =3D 0; i < blocks; i++ ) + { + struct irq_block *irqs =3D gic_ctx.dist.irqs + i; + size_t j, off =3D i * sizeof(irqs->isenabler); + + writel_gicd(GENMASK(31, 0), GICD_ICENABLER + off); + writel_gicd(irqs->isenabler, GICD_ISENABLER + off); + + writel_gicd(GENMASK(31, 0), GICD_ICACTIVER + off); + writel_gicd(irqs->isactiver, GICD_ISACTIVER + off); + + off =3D i * sizeof(irqs->ipriorityr); + for ( j =3D 0; j < ARRAY_SIZE(irqs->ipriorityr); j++ ) + writel_gicd(irqs->ipriorityr[j], GICD_IPRIORITYR + off + j * 4= ); + + /* + * GICD_ITARGETSR0..7 cover SGIs/PPIs and hold no state to save: + * they are read-only on multiprocessor implementations and RAZ/WI + * on uniprocessor implementations. + */ + if ( i ) + { + off =3D i * sizeof(irqs->itargetsr); + for ( j =3D 0; j < ARRAY_SIZE(irqs->itargetsr); j++ ) + writel_gicd(irqs->itargetsr[j], GICD_ITARGETSR + off + j *= 4); + } + + off =3D i * sizeof(irqs->icfgr); + for ( j =3D 0; j < ARRAY_SIZE(irqs->icfgr); j++ ) + writel_gicd(irqs->icfgr[j], GICD_ICFGR + off + j * 4); + } + + /* Make sure all registers are restored and enable distributor */ + writel_gicd(gic_ctx.dist.ctlr, GICD_CTLR); + + /* Restore GIC CPU interface configuration */ + writel_gicc(gic_ctx.cpu.pmr, GICC_PMR); + writel_gicc(gic_ctx.cpu.bpr, GICC_BPR); + + /* Enable GIC CPU interface */ + writel_gicc(gic_ctx.cpu.ctlr, GICC_CTLR); +} + +static void __init gicv2_alloc_context(void) +{ + uint32_t blocks =3D DIV_ROUND_UP(gicv2_info.nr_lines, 32); + + gic_ctx.dist.irqs =3D xzalloc_array(struct irq_block, blocks); + if ( !gic_ctx.dist.irqs ) + panic("Failed to allocate memory for GICv2 suspend context\n"); +} + +#endif /* CONFIG_SYSTEM_SUSPEND */ + #ifdef CONFIG_ACPI static unsigned long gicv2_get_hwdom_extra_madt_size(const struct domain *= d) { @@ -1312,6 +1484,11 @@ static int __init gicv2_init(void) =20 spin_unlock(&gicv2.lock); =20 +#ifdef CONFIG_SYSTEM_SUSPEND + /* Allocate memory to be used for saving GIC context during the suspen= d */ + gicv2_alloc_context(); +#endif /* CONFIG_SYSTEM_SUSPEND */ + return 0; } =20 @@ -1355,6 +1532,10 @@ static const struct gic_hw_operations gicv2_ops =3D { .map_hwdom_extra_mappings =3D gicv2_map_hwdom_extra_mappings, .iomem_deny_access =3D gicv2_iomem_deny_access, .do_LPI =3D gicv2_do_LPI, +#ifdef CONFIG_SYSTEM_SUSPEND + .suspend =3D gicv2_suspend, + .resume =3D gicv2_resume, +#endif /* CONFIG_SYSTEM_SUSPEND */ }; =20 /* Set up the GIC */ diff --git a/xen/arch/arm/gic.c b/xen/arch/arm/gic.c index ee75258fc3..7727ffed5a 100644 --- a/xen/arch/arm/gic.c +++ b/xen/arch/arm/gic.c @@ -432,6 +432,35 @@ int gic_iomem_deny_access(struct domain *d) return gic_hw_ops->iomem_deny_access(d); } =20 +#ifdef CONFIG_SYSTEM_SUSPEND + +int gic_suspend(void) +{ + /* Must be called by boot CPU#0 with interrupts disabled */ + ASSERT(!local_irq_is_enabled()); + ASSERT(!smp_processor_id()); + + if ( !gic_hw_ops->suspend || !gic_hw_ops->resume ) + return -ENOSYS; + + return gic_hw_ops->suspend(); +} + +void gic_resume(void) +{ + /* + * Must be called by boot CPU#0 with interrupts disabled after gic_sus= pend + * has returned successfully. + */ + ASSERT(!local_irq_is_enabled()); + ASSERT(!smp_processor_id()); + ASSERT(gic_hw_ops->resume); + + gic_hw_ops->resume(); +} + +#endif /* CONFIG_SYSTEM_SUSPEND */ + static int cpu_gic_callback(struct notifier_block *nfb, unsigned long action, void *hcpu) diff --git a/xen/arch/arm/include/asm/gic.h b/xen/arch/arm/include/asm/gic.h index ff22dea40d..fbf0d69edd 100644 --- a/xen/arch/arm/include/asm/gic.h +++ b/xen/arch/arm/include/asm/gic.h @@ -301,6 +301,12 @@ extern int gicv_setup(struct domain *d); extern void gic_save_state(struct vcpu *v); extern void gic_restore_state(struct vcpu *v); =20 +#ifdef CONFIG_SYSTEM_SUSPEND +/* Suspend/resume */ +extern int gic_suspend(void); +extern void gic_resume(void); +#endif /* CONFIG_SYSTEM_SUSPEND */ + /* SGI (AKA IPIs) */ enum gic_sgi { GIC_SGI_EVENT_CHECK, @@ -444,6 +450,12 @@ struct gic_hw_operations { int (*iomem_deny_access)(struct domain *d); /* Handle LPIs, which require special handling */ void (*do_LPI)(unsigned int lpi); +#ifdef CONFIG_SYSTEM_SUSPEND + /* Save GIC configuration due to the system suspend */ + int (*suspend)(void); + /* Restore GIC configuration due to the system resume */ + void (*resume)(void); +#endif /* CONFIG_SYSTEM_SUSPEND */ }; =20 extern const struct gic_hw_operations *gic_hw_ops; --=20 2.43.0 From nobody Sat May 30 11:15:55 2026 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass(p=none dis=none) header.from=gmail.com ARC-Seal: i=1; a=rsa-sha256; t=1778605819; cv=none; d=zohomail.com; s=zohoarc; b=Hh6OU7jGodA/gy3wA+MpmyR8NIchsvhXvCZCYkODwwdFESGUXL6xdssF5uI/pvBS8qRS3pOQNiVFdPg2ScWRKlkP2nFl60n6NxDxzWR3ng3OSbrYWFiNXPD1e4xn4UfvNGe76LuSVWLWAHH4AE4r5dxVVfieyzmLe68gxPggjAY= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1778605819; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=XXX7LiIDl3A+pjPnWv0DkG9B7a9KxTjMmrGlAN2uPmo=; b=SUb/QgXAj4AkXZcFfPY2aoaElAwyzC7UhnehHIJ7DQ9agSiW4zty9oyx9DdmSSvnVTpWGIAqD4POzSdsmn5sWpzLQhmq8v6qwciYQCDQyuFeZf2U3SM02iARUHLkxmqQ+TBPUDT9yi7UncWcdVlzo6yvXJ7ylsiUr364+iKTdtw= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 177860581948374.17240778376595; Tue, 12 May 2026 10:10:19 -0700 (PDT) Received: from list by lists.xenproject.org with outflank-mailman.1307276.1579007 (Exim 4.92) (envelope-from ) id 1wMqc7-0006Fs-Ll; Tue, 12 May 2026 17:09:47 +0000 Received: by outflank-mailman (output) from mailman id 1307276.1579007; Tue, 12 May 2026 17:09:47 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1wMqc7-0006Fj-Ih; Tue, 12 May 2026 17:09:47 +0000 Received: by outflank-mailman (input) for mailman id 1307276; Tue, 12 May 2026 17:09:46 +0000 Received: from mx.expurgate.net ([195.190.135.10]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1wMqc6-0005xD-1w for xen-devel@lists.xenproject.org; Tue, 12 May 2026 17:09:46 +0000 Received: from mx.expurgate.net (helo=localhost) by mx.expurgate.net with esmtp id 1wMqc5-00AZDT-F1 for xen-devel@lists.xenproject.org; Tue, 12 May 2026 19:09:45 +0200 Received: from [10.42.69.7] (helo=localhost) by localhost with ESMTP (eXpurgate MTA 0.9.1) (envelope-from ) id 6a035eca-e002-0a2a0a5209dd-0a2a45078240-26 for ; Tue, 12 May 2026 19:09:45 +0200 Received: from [209.85.128.54] (helo=mail-wm1-f54.google.com) by tlsNG-ef75cf.mxtls.expurgate.net with ESMTPS (eXpurgate 4.56.1) (envelope-from ) id 6a035ed9-229c-0a2a45070019-d1558036e1eb-3 for ; Tue, 12 May 2026 19:09:45 +0200 Received: by mail-wm1-f54.google.com with SMTP id 5b1f17b1804b1-488a8ca4aadso53136715e9.3 for ; Tue, 12 May 2026 10:09:45 -0700 (PDT) Received: from EPUAKYIW02F7.. ([45.12.26.18]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-48fc8cccf90sm10838845e9.0.2026.05.12.10.09.43 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 12 May 2026 10:09:43 -0700 (PDT) X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" Authentication-Results: eu.smtp.expurgate.cloud; dkim=pass header.s=20251104 header.d=gmail.com header.i="@gmail.com" header.h="Content-Transfer-Encoding:MIME-Version:References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1778605784; x=1779210584; darn=lists.xenproject.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=XXX7LiIDl3A+pjPnWv0DkG9B7a9KxTjMmrGlAN2uPmo=; b=VWCqtQu9XJmjbzceO96JPfwoRYiZGQOusmT5/rBDrzxvI6P8YORsDVhS5MHneEwMbs Xata0uBfRfHfsRSRY84BLfZuwb9fL9kr0JMNAepgjUzHr59qk28T0n/9Id/hIObNaY8J XQeeJ9ucIsPJDfl11tO+Lv3tNx3gEeWUwXFr7z8QwrWc9rEWEssYExHscQEJg40TRfK7 0MSlxY4U30NZgtSYyHCiiqsDbG9gkeMAEKgiHfKe4kPq9UaEheBLIXpRLREKZo46NLTJ jHRd2D1wGY5yc7mTAC7L4BXLGV44lIPu+SpmHXCGFMenCzbKxFwIDfXTWEApKkXm3JQd 8iKg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1778605784; x=1779210584; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=XXX7LiIDl3A+pjPnWv0DkG9B7a9KxTjMmrGlAN2uPmo=; b=N9f/rUhvTqgC8V0/YhL6XAUhXEVK3531rbu4f7HRy5EEYPwveTp7I4AeUiMD++5aZ5 HG0AgOgbRpwwH/hnQbfYcCw9IFChv6A6Qo0donoLehRl4TvNKJRcU6X+LyRXkFSB8JW2 5zayS+3h6gMX52j+FNGX5ZpZs2MpkZZGYC+X47kYJR9YNvzRNIhOuxJCsaijzRFFhlUy uNtijB0pfoybVIFNZc6Vly4EPHPZfzRUR6wAtSseR2tFJ3tCECnYgkAdkKKCKml2+3QU 7PwYZFYJ/oUt9xBZoIpAnmMCrWjjw8x9pcuordZwglm3LdUttMzblAwj8OHweqdKvl0S 4zdQ== X-Gm-Message-State: AOJu0Ywsf9hUm62rrHN3faP+nziBm+jqoP+Cqs6Hcn/xPQY3v02IQjyA 3b4Hvfi8Ayy9IQKcB1K0WykeYcFU1wLdMrHIfOurVHirjMjrSLmA2/xBQluHCA== X-Gm-Gg: Acq92OEQcSPRBqQNCeTbx4q8ELVYrkiA/e49p9C8JjA5jsmcd7J6x57iCoSM8uv6Goe BGkEYNyDVgGvb13ueBzdB3Wg9L/6oqEhu3xgpA5hhOEJFYl1VICDlexQt/ajkaHFi7MuttIsSlw /OzJY2z96ORXeDRSzQT6GQoqUXCvJh36x2kC+DENqVo+d+3M4hCqGToHMB+NjpuMSrDZLmlzomx i6eMNQIKq46twG6tPQvmyavP5wzVvrp42I2K0j3DyeVuEp/C7gzRaESFC6zBVZlxD6jWStmecm8 AuQ8Aieax6UxYsDKhw34zqUTD8MemT5zfHOTbkpf9ZHeMQ+/e/wt6Rc1FwHFUmozqgF5LnG4BK8 hSBqcm5gCeusJkhw3H6bpx2tYh2JV4rtAjaV50XtiA5o+2q2nBGdmIkPEB+iRihUYp+6EQhadbA JxXC9OCozH65aOrSt/BaxYfNAs3A== X-Received: by 2002:a05:600c:37c6:b0:48e:7f22:d90 with SMTP id 5b1f17b1804b1-48e7f7de432mr182800855e9.4.1778605784332; Tue, 12 May 2026 10:09:44 -0700 (PDT) From: Mykola Kvach To: xen-devel@lists.xenproject.org Cc: Mykola Kvach , Stefano Stabellini , Julien Grall , Bertrand Marquis , Michal Orzel , Volodymyr Babchuk Subject: [PATCH v9 03/13] xen/arm: gic-v3: tolerate retained redistributor LPI state across CPU_OFF Date: Tue, 12 May 2026 20:07:10 +0300 Message-ID: X-Mailer: git-send-email 2.43.0 In-Reply-To: References: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-purgate-ID: tlsNG-ef75cf/1778605785-2887FC48-52A15AD8/0/0 X-purgate-type: clean X-purgate-size: 10164 X-ZohoMail-DKIM: pass (identity @gmail.com) X-ZM-MESSAGEID: 1778605821717158501 Content-Type: text/plain; charset="utf-8" From: Mykola Kvach PSCI does not guarantee that a GICv3 redistributor is powered down across CPU_OFF -> CPU_ON. DEN0022F.b says CPU_OFF powers down the calling core (5.5) and CPU_ON brings the core back with a defined initial CPU state (5.6, 6.4). However, PSCI leaves interrupt migration and GIC re-initialization to the supervisory software/firmware stack: the caller must migrate interrupts away before CPU_OFF (5.5.2), and the execution context that is lost in a powerdown state must be saved and restored by software (6.8). PSCI also calls out GIC management explicitly in 6.8, including retargeting SPIs, preventing PPIs/SGIs from targeting a powered down CPU, and reinitializing the CPU interface after CPU_ON. This matches the GIC architecture. IHI0069H.b Chapter 11.1 requires the PE and CPU interface to share a power domain, but explicitly allows the associated redistributor, distributor, and ITS to remain powered while the PE and CPU interface are off. All other GIC power-management behavior is IMPLEMENTATION DEFINED. DEN0050D Chapter 4.2, "Generic Interrupt Controller (GIC)", says the GICv3 redistributor may live either in the AP core power domain or in a relatively always-on parent domain. So after CPU_OFF -> CPU_ON a secondary CPU can legitimately come back to a live redistributor with GICR_CTLR.EnableLPIs still set. Handle that case in the LPI setup path instead of assuming a fully reset redistributor. The LPI path needs special care because the GIC spec makes redistributor LPI state sticky and partially implementation defined. IHI0069H.b 5.1.1 and 5.1.2 say that changing GICR_PROPBASER or GICR_PENDBASER while GICR_CTLR.EnableLPIs =3D=3D 1 is UNPREDICTABLE. After clearing EnableLPIs, software must wait for GICR_CTLR.RWP =3D=3D 0 before touching the pending table. The architecture also permits implementations where, once EnableLPIs has been set, clearing it again is not guaranteed to work. Where an ITS is present, the spec strongly recommends moving LPIs to another redistributor before clearing EnableLPIs. Because of that, treat a retained EnableLPIs state as valid when the redistributor still points at Xen's expected PROPBASER/PENDBASER tables. Only try to clear EnableLPIs when the retained configuration does not match Xen's state, and wait for RWP before reprogramming the tables. This is also consistent with platform firmware reality: PSCI and the GIC architecture allow platform-specific redistributor power handling, and not all platform firmware implementations force a full redistributor power-off through implementation-defined controls during CPU_OFF. Xen therefore needs to tolerate retained redistributor state on secondary CPU bring-up. Keep gicv3_populate_rdist() resident as well, because gicv3_cpu_init() reuses it on secondary CPU bring-up after init. Tested using Xen's non-boot CPU disable/enable path on Arm FVP_Base_RevC-2xAEMvA, both with and without: -C gic_distributor.allow-LPIEN-clear=3D1 -C gic_distributor.GICR-clear-enable-supported=3D1 and on Orange Pi 5. Signed-off-by: Mykola Kvach Reviewed-by: Luca Fancellu --- Changes in v9: - move gicv3_do_wait_for_rwp prototype from its related header to gic.h - drop __init from gicv3_populate_rdist(), which is reused on secondary CPU bring-up after boot - changed print format for smp_processor_id in gicv3_populate_rdist func - cosmetic changes --- xen/arch/arm/gic-v3-lpi.c | 77 +++++++++++++++++++++++++++++++++- xen/arch/arm/gic-v3.c | 23 ++++++---- xen/arch/arm/include/asm/gic.h | 4 ++ 3 files changed, 94 insertions(+), 10 deletions(-) diff --git a/xen/arch/arm/gic-v3-lpi.c b/xen/arch/arm/gic-v3-lpi.c index 9ee338edc2..36bb45738e 100644 --- a/xen/arch/arm/gic-v3-lpi.c +++ b/xen/arch/arm/gic-v3-lpi.c @@ -81,6 +81,13 @@ static DEFINE_PER_CPU(struct lpi_redist_data, lpi_redist= ); #define MAX_NR_HOST_LPIS (lpi_data.max_host_lpi_ids - LPI_OFFSET) #define HOST_LPIS_PER_PAGE (PAGE_SIZE / sizeof(union host_lpi)) =20 +#define GICR_PROPBASER_XEN_MASK GENMASK_ULL(51, 12) +/* + * For retained redistributor state, match the pending table by address on= ly. + * Attribute bits such as PTZ may not read back with the programmed value. + */ +#define GICR_PENDBASER_XEN_MASK GENMASK_ULL(51, 16) + static union host_lpi *gic_get_host_lpi(uint32_t plpi) { union host_lpi *block; @@ -296,6 +303,60 @@ static int gicv3_lpi_set_pendtable(void __iomem *rdist= _base) return 0; } =20 +static uint64_t gicv3_lpi_expected_proptable(void) +{ + return virt_to_maddr(lpi_data.lpi_property); +} + +static uint64_t gicv3_lpi_expected_pendtable(void) +{ + return virt_to_maddr(this_cpu(lpi_redist).pending_table); +} + +static bool gicv3_lpi_tables_match(void __iomem *rdist_base) +{ + uint64_t propbase, pendbase; + + if ( !lpi_data.lpi_property || !this_cpu(lpi_redist).pending_table ) + return false; + + propbase =3D readq_relaxed(rdist_base + GICR_PROPBASER); + pendbase =3D readq_relaxed(rdist_base + GICR_PENDBASER); + + return ((propbase & GICR_PROPBASER_XEN_MASK) =3D=3D + (gicv3_lpi_expected_proptable() & GICR_PROPBASER_XEN_MASK)) && + ((pendbase & GICR_PENDBASER_XEN_MASK) =3D=3D + (gicv3_lpi_expected_pendtable() & GICR_PENDBASER_XEN_MASK)); +} + +static int gicv3_lpi_disable_lpis(void __iomem *rdist_base) +{ + uint32_t reg =3D readl_relaxed(rdist_base + GICR_CTLR); + int ret; + + if ( !(reg & GICR_CTLR_ENABLE_LPIS) ) + return 0; + + writel_relaxed(reg & ~GICR_CTLR_ENABLE_LPIS, rdist_base + GICR_CTLR); + + /* + * The spec only guarantees programmability when we have observed the = bit + * cleared. Where clearing is supported, RWP must reach 0 before touch= ing + * PROPBASER/PENDBASER again. + */ + wmb(); + + ret =3D gicv3_do_wait_for_rwp(rdist_base); + if ( ret ) + return ret; + + reg =3D readl_relaxed(rdist_base + GICR_CTLR); + if ( reg & GICR_CTLR_ENABLE_LPIS ) + return -EBUSY; + + return 0; +} + /* * Tell a redistributor about the (shared) property table, allocating one * if not already done. @@ -374,7 +435,21 @@ int gicv3_lpi_init_rdist(void __iomem * rdist_base) /* Make sure LPIs are disabled before setting up the tables. */ reg =3D readl_relaxed(rdist_base + GICR_CTLR); if ( reg & GICR_CTLR_ENABLE_LPIS ) - return -EBUSY; + { + if ( gicv3_lpi_tables_match(rdist_base) ) + return -EBUSY; + + ret =3D gicv3_lpi_disable_lpis(rdist_base); + if ( ret =3D=3D -EBUSY ) + { + printk(XENLOG_ERR + "GICv3: CPU%u: LPIs still enabled with unexpected redis= tributor tables\n", + smp_processor_id()); + return -EINVAL; + } + if ( ret ) + return ret; + } =20 ret =3D gicv3_lpi_set_pendtable(rdist_base); if ( ret ) diff --git a/xen/arch/arm/gic-v3.c b/xen/arch/arm/gic-v3.c index 7f365cdbe9..cc0569a157 100644 --- a/xen/arch/arm/gic-v3.c +++ b/xen/arch/arm/gic-v3.c @@ -274,8 +274,8 @@ static void gicv3_enable_sre(void) isb(); } =20 -/* Wait for completion of a distributor change */ -static void gicv3_do_wait_for_rwp(void __iomem *base) +/* Wait for completion of a distributor/redistributor write-pending change= . */ +int gicv3_do_wait_for_rwp(void __iomem *base) { uint32_t val; bool timeout =3D false; @@ -295,17 +295,22 @@ static void gicv3_do_wait_for_rwp(void __iomem *base) } while ( 1 ); =20 if ( timeout ) + { dprintk(XENLOG_ERR, "RWP timeout\n"); + return -ETIMEDOUT; + } + + return 0; } =20 static void gicv3_dist_wait_for_rwp(void) { - gicv3_do_wait_for_rwp(GICD); + (void)gicv3_do_wait_for_rwp(GICD); } =20 static void gicv3_redist_wait_for_rwp(void) { - gicv3_do_wait_for_rwp(GICD_RDIST_BASE); + (void)gicv3_do_wait_for_rwp(GICD_RDIST_BASE); } =20 static void gicv3_wait_for_rwp(int irq) @@ -857,7 +862,7 @@ static bool gicv3_enable_lpis(void) return true; } =20 -static int __init gicv3_populate_rdist(void) +static int gicv3_populate_rdist(void) { int i; uint32_t aff; @@ -925,15 +930,15 @@ static int __init gicv3_populate_rdist(void) gicv3_set_redist_address(rdist_addr, procnum); =20 ret =3D gicv3_lpi_init_rdist(ptr); - if ( ret && ret !=3D -ENODEV ) + if ( ret && ret !=3D -ENODEV && ret !=3D -EBUSY ) { - printk("GICv3: CPU%d: Cannot initialize LPIs: %u\n= ", + printk("GICv3: CPU%u: Cannot initialize LPIs: %d\n= ", smp_processor_id(), ret); break; } } =20 - printk("GICv3: CPU%d: Found redistributor in region %d @%p= \n", + printk("GICv3: CPU%u: Found redistributor in region %d @%p= \n", smp_processor_id(), i, ptr); return 0; } @@ -953,7 +958,7 @@ static int __init gicv3_populate_rdist(void) } while ( !(typer & GICR_TYPER_LAST) ); } =20 - dprintk(XENLOG_ERR, "GICv3: CPU%d: mpidr 0x%"PRIregister" has no re-di= stributor!\n", + dprintk(XENLOG_ERR, "GICv3: CPU%u: mpidr 0x%"PRIregister" has no re-di= stributor!\n", smp_processor_id(), cpu_logical_map(smp_processor_id())); =20 return -ENODEV; diff --git a/xen/arch/arm/include/asm/gic.h b/xen/arch/arm/include/asm/gic.h index fbf0d69edd..9293aa56e2 100644 --- a/xen/arch/arm/include/asm/gic.h +++ b/xen/arch/arm/include/asm/gic.h @@ -301,6 +301,10 @@ extern int gicv_setup(struct domain *d); extern void gic_save_state(struct vcpu *v); extern void gic_restore_state(struct vcpu *v); =20 +#ifdef CONFIG_GICV3 +int gicv3_do_wait_for_rwp(void __iomem *base); +#endif + #ifdef CONFIG_SYSTEM_SUSPEND /* Suspend/resume */ extern int gic_suspend(void); --=20 2.43.0 From nobody Sat May 30 11:15:55 2026 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass(p=none dis=none) header.from=gmail.com ARC-Seal: i=1; a=rsa-sha256; t=1778605812; cv=none; d=zohomail.com; s=zohoarc; b=CF5oF/69KSfaChmGUmboUOiicsjmMzQsCSVVvSaOrlhT+4EtUm6FVvC67PzdPK7yLeJLGTXsDH7c5uqzDKEhTH66lPHqkX7abJ4iy3ednh2S+/HvLnMRYwNhg3OEoXpCI8vMROX4WhdDnnhyUkPpcrLvo/f6T5yRCQbHyCyYb6g= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1778605812; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=l8t5jxqUnUpYd4m0eBF1jttpmfOSXj9O/o10a7IBFAI=; b=n1Astgfay/fJQMjLh8E3b9VIQcfI+sNFD1aIgXNtsNqgSGKOuYfcW0EpO+BkOPmwL9Q3u7F600gx6NTFi/+zmZA0YiEGHZeRJj50uUMMJ90iri0CvbqwSq3NIbiPhBD6kGRlAgBD0+tc3fCFP8AK7bTOef9TGcGcOaok9FleR6c= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1778605812774843.3519645802445; Tue, 12 May 2026 10:10:12 -0700 (PDT) Received: from list by lists.xenproject.org with outflank-mailman.1307277.1579016 (Exim 4.92) (envelope-from ) id 1wMqc9-0006UH-2Y; Tue, 12 May 2026 17:09:49 +0000 Received: by outflank-mailman (output) from mailman id 1307277.1579016; Tue, 12 May 2026 17:09:49 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1wMqc8-0006U6-Uh; Tue, 12 May 2026 17:09:48 +0000 Received: by outflank-mailman (input) for mailman id 1307277; Tue, 12 May 2026 17:09:47 +0000 Received: from mx.expurgate.net ([195.190.135.10]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1wMqc7-0006Dp-FW for xen-devel@lists.xenproject.org; Tue, 12 May 2026 17:09:47 +0000 Received: from mx.expurgate.net (helo=localhost) by mx.expurgate.net with esmtp id 1wMqc6-007RzZ-Sm for xen-devel@lists.xenproject.org; Tue, 12 May 2026 19:09:46 +0200 Received: from [10.42.69.1] (helo=localhost) by localhost with ESMTP (eXpurgate MTA 0.9.1) (envelope-from ) id 6a035ec9-2eae-0a2a0a5409dd-0a2a4501afae-14 for ; Tue, 12 May 2026 19:09:46 +0200 Received: from [209.85.128.46] (helo=mail-wm1-f46.google.com) by tlsNG-d62444.mxtls.expurgate.net with ESMTPS (eXpurgate 4.56.1) (envelope-from ) id 6a035eda-c1f2-0a2a45010019-d155802edcc3-3 for ; Tue, 12 May 2026 19:09:46 +0200 Received: by mail-wm1-f46.google.com with SMTP id 5b1f17b1804b1-48909558b3aso59331895e9.0 for ; Tue, 12 May 2026 10:09:46 -0700 (PDT) Received: from EPUAKYIW02F7.. ([45.12.26.18]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-48fc8cccf90sm10838845e9.0.2026.05.12.10.09.44 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 12 May 2026 10:09:45 -0700 (PDT) X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" Authentication-Results: eu.smtp.expurgate.cloud; dkim=pass header.s=20251104 header.d=gmail.com header.i="@gmail.com" header.h="Content-Transfer-Encoding:MIME-Version:References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1778605786; x=1779210586; darn=lists.xenproject.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=l8t5jxqUnUpYd4m0eBF1jttpmfOSXj9O/o10a7IBFAI=; b=hrwrbB0Zb98YYjYphgzOt8hsQLzcmzoxySgsE4lAMA6OBIrpfSeFwA+rkYQLrrK9Wk KgLVO9HjR1cZSIsu0kVLj9Q++rkHuOpj5/qmGMLxo1footvzhzhn+94NxcAcHa87ykxL afs+zPwWZ/925dq0oTIlh+BMmDPXibxmH0iTwPI1U+FKiA/DJXPrpHMdywc9l0Bfc6B2 NW/BQssgSq8gmvJNXitGka+RAQaFFemc2xoQn/qF2VTuc1byCzJn2Vl+PggoAPCl6OXu YCGMt6b37SOtTddw8ofM/xwFbBOxJzZUlfB3Z4ZcmJihbqPgjTzIfBqCKZUjtbfPp9d1 b/Gw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1778605786; x=1779210586; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=l8t5jxqUnUpYd4m0eBF1jttpmfOSXj9O/o10a7IBFAI=; b=TCz4dqP4GDKlVkA3IWatnUg+qPku1Z7dkGsHvGIXp+o0S6PLAXdgtKKvhZ8IEb/0pe 6hQlBq5ztxhgEvyYdcNTeUKwA2SHlMmUbaeUZv34Qc1oOPskBwZ1vWYHnCTgTibeRGDQ YBnt0GVGRglUfbgITK6tiBZf03k/QTmgrDNhNxAnn9q/BU2knfherAYbu2gecu5hpDhl 3LxSSzVKPS1+oJnBstYXETOrGtmCQW97c6gdbtXt9uHbYmCV5CBxwIGF4iTI/f3nto47 SzAQF39/CNIMxYaY/CarjeSwWuf5ZhJx2IzSTTupyWUarsWNVg/+4970dkrYZdpx9mdN AXDQ== X-Gm-Message-State: AOJu0Yy0woTdzCjbXnk+FzfbBuBKQ/ZIPO6d6qDNFgEhHrCghieOFOct EA9T1VQTo7s558/JncI1ky4eXbPjOGta9KgRi6OIq3sG4yfRLW/O4i7TyFquSQ== X-Gm-Gg: Acq92OG8w732IG70gerqNlLBNZoXzbE2MTWv8BXjorW0s9T9Ebu0cpebM00GhYoJDCA ltlYBFFe02/KXNy2WMgNwBWUDve/pNGRhojfygaiYMB9YL8usJRCmdAG9B4mEs0gE2pDS8toZXR Y365rWk1pBriGhmoFl20UpOK1aJmQsJorFQMai6iHr4whtoacT1DgfYMc07Lo/OjE2r/vOKUSSM 3ty3BC+FRstMxZknyjQxRm23KxNqev84f3FB6krXrXmSLYgRyjdZAzn2rX7Hy9+wiPPug8nONKG QG9VFXaQiElL/pEs+b2S0RL9TYSsiS/Ec5Vj3iCgD9B0b/s62GnZBiRmbzMApgcg5nIe1VbQWBP 38JG6QswLjbMco9t2hSVGf5HypmnvUEJDiCSpf/g7/qjECX9Vt8+RDx1AN09KXbGstYWBENHKvN ipVW67iWJbNJvIqbu/tpZWvptnZg== X-Received: by 2002:a05:600c:46d2:b0:488:b187:3c with SMTP id 5b1f17b1804b1-48e51f32c65mr459278545e9.14.1778605785937; Tue, 12 May 2026 10:09:45 -0700 (PDT) From: Mykola Kvach To: xen-devel@lists.xenproject.org Cc: Mykola Kvach , Stefano Stabellini , Julien Grall , Bertrand Marquis , Michal Orzel , Volodymyr Babchuk Subject: [PATCH v9 04/13] xen/arm: gic-v3: Implement GICv3 suspend/resume functions Date: Tue, 12 May 2026 20:07:11 +0300 Message-ID: <8973fc6bf69d8b20cebb61289c1b8596b1a09900.1778605274.git.mykola_kvach@epam.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: References: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-purgate-ID: tlsNG-d62444/1778605786-B675CFF4-4A0A0646/0/0 X-purgate-type: clean X-purgate-size: 15807 X-ZohoMail-DKIM: pass (identity @gmail.com) X-ZM-MESSAGEID: 1778605813862158500 Content-Type: text/plain; charset="utf-8" From: Mykola Kvach System suspend may lead to a state where GIC would be powered down. Therefore, Xen should save/restore the context of GIC on suspend/resume. Note that the context consists of states of registers which are controlled by the hypervisor. Other GIC registers which are accessible by guests are saved/restored on context switch. Signed-off-by: Mykola Kvach --- Changes in V9: - fix the suspend-context comment typo and split dist_ctx declarations; - restore ICC_IGRPEN1_EL1 on the suspend error path; - re-initialize GICD_IGROUPRnE during resume; - restore GICD_IROUTER only after re-enabling ARE_NS during resume. Changes in V8: - use right rdist base for prop/pend baser and ctrl Changes in V7: - restore LPI regs on resume - add timeout during redist disabling - squash with suspend/resume handling for GICv3 eSPI registers - drop ITS guard paths so suspend/resume always runs; switch missing ctx allocation to panic - trim TODO comments; narrow redistributor storage to PPI icfgr - keep distributor context allocation even without ITS; adjust resume to use GENMASK(31, 0) for clearing enables - drop storage of the SGI configuration register, as SGIs are always edge-triggered --- xen/arch/arm/gic-v3-lpi.c | 3 + xen/arch/arm/gic-v3.c | 362 ++++++++++++++++++++++++- xen/arch/arm/include/asm/gic_v3_defs.h | 1 + 3 files changed, 363 insertions(+), 3 deletions(-) diff --git a/xen/arch/arm/gic-v3-lpi.c b/xen/arch/arm/gic-v3-lpi.c index 36bb45738e..b0289ee4dc 100644 --- a/xen/arch/arm/gic-v3-lpi.c +++ b/xen/arch/arm/gic-v3-lpi.c @@ -467,6 +467,9 @@ static int cpu_callback(struct notifier_block *nfb, uns= igned long action, switch ( action ) { case CPU_UP_PREPARE: + if ( system_state =3D=3D SYS_STATE_resume ) + break; + rc =3D gicv3_lpi_allocate_pendtable(cpu); if ( rc ) printk(XENLOG_ERR "Unable to allocate the pendtable for CPU%lu= \n", diff --git a/xen/arch/arm/gic-v3.c b/xen/arch/arm/gic-v3.c index cc0569a157..6fc4354e2e 100644 --- a/xen/arch/arm/gic-v3.c +++ b/xen/arch/arm/gic-v3.c @@ -1072,12 +1072,12 @@ out: return res; } =20 -static void gicv3_hyp_disable(void) +static void gicv3_hyp_enable(bool enable) { register_t hcr; =20 hcr =3D READ_SYSREG(ICH_HCR_EL2); - hcr &=3D ~GICH_HCR_EN; + hcr =3D enable ? (hcr | GICH_HCR_EN) : (hcr & ~GICH_HCR_EN); WRITE_SYSREG(hcr, ICH_HCR_EL2); isb(); } @@ -1184,7 +1184,7 @@ static void gicv3_disable_interface(void) spin_lock(&gicv3.lock); =20 gicv3_cpu_disable(); - gicv3_hyp_disable(); + gicv3_hyp_enable(false); =20 spin_unlock(&gicv3.lock); } @@ -1920,6 +1920,354 @@ static bool gic_dist_supports_lpis(void) return (readl_relaxed(GICD + GICD_TYPER) & GICD_TYPE_LPIS); } =20 +#ifdef CONFIG_SYSTEM_SUSPEND + +/* This struct represents a block of 32 IRQs */ +struct dist_irq_block { + uint32_t icfgr[2]; + uint32_t ipriorityr[8]; + uint64_t irouter[32]; + uint32_t isactiver; + uint32_t isenabler; +}; + +struct redist_ctx { + uint32_t ctlr; + uint32_t icfgr; /* only PPIs stored */ + uint32_t igroupr; + uint32_t ipriorityr[8]; + uint32_t isactiver; + uint32_t isenabler; + + uint64_t pendbase; + uint64_t propbase; +}; + +/* GICv3 registers to be saved/restored on system suspend/resume */ +struct gicv3_ctx { + struct dist_ctx { + uint32_t ctlr; + struct dist_irq_block *irqs; + struct dist_irq_block *espi_irqs; + } dist; + + /* have only one rdist structure for last running CPU during suspend */ + struct redist_ctx rdist; + + struct cpu_ctx { + uint32_t ctlr; + uint32_t pmr; + uint32_t bpr; + uint32_t sre_el2; + uint32_t grpen; + } cpu; +}; + +static struct gicv3_ctx gicv3_ctx; + +static void __init gicv3_alloc_context(void) +{ + uint32_t blocks =3D DIV_ROUND_UP(gicv3_info.nr_lines, 32); + + /* The spec allows for systems without any SPIs */ + if ( blocks > 1 ) + { + gicv3_ctx.dist.irqs =3D xzalloc_array(struct dist_irq_block, block= s - 1); + if ( !gicv3_ctx.dist.irqs ) + panic("Failed to allocate memory for GICv3 suspend context\n"); + } + +#ifdef CONFIG_GICV3_ESPI + if ( !gic_number_espis() ) + return; + + blocks =3D gic_number_espis() / 32; + gicv3_ctx.dist.espi_irqs =3D xzalloc_array(struct dist_irq_block, bloc= ks); + if ( !gicv3_ctx.dist.espi_irqs ) + panic("Failed to allocate memory for GICv3 eSPI suspend context\n"= ); +#endif +} + +static int gicv3_disable_redist(void) +{ + void __iomem *waker =3D GICD_RDIST_BASE + GICR_WAKER; + s_time_t deadline; + + /* + * Avoid infinite loop if Non-secure does not have access to GICR_WAKE= R. + * See Arm IHI 0069H.b, 12.11.42 GICR_WAKER: + * When GICD_CTLR.DS =3D=3D 0 and an access is Non-secure accesses= to this + * register are RAZ/WI. + */ + if ( !(readl_relaxed(GICD + GICD_CTLR) & GICD_CTLR_DS) ) + return 0; + + deadline =3D NOW() + MILLISECS(1000); + + writel_relaxed(readl_relaxed(waker) | GICR_WAKER_ProcessorSleep, waker= ); + while ( (readl_relaxed(waker) & GICR_WAKER_ChildrenAsleep) =3D=3D 0 ) + { + if ( NOW() > deadline ) + { + printk("GICv3: Timeout waiting for redistributor to sleep\n"); + return -ETIMEDOUT; + } + cpu_relax(); + udelay(10); + } + + return 0; +} + +#define GET_SPI_REG_OFFSET(name, is_espi) \ + ((is_espi) ? GICD_##name##nE : GICD_##name) + +static void gicv3_store_spi_irq_block(struct dist_irq_block *irqs, + unsigned int i, bool is_espi) +{ + void __iomem *base; + unsigned int irq; + + base =3D GICD + GET_SPI_REG_OFFSET(ICFGR, is_espi) + i * sizeof(irqs->= icfgr); + irqs->icfgr[0] =3D readl_relaxed(base); + irqs->icfgr[1] =3D readl_relaxed(base + 4); + + base =3D GICD + GET_SPI_REG_OFFSET(IPRIORITYR, is_espi); + base +=3D i * sizeof(irqs->ipriorityr); + for ( irq =3D 0; irq < ARRAY_SIZE(irqs->ipriorityr); irq++ ) + irqs->ipriorityr[irq] =3D readl_relaxed(base + 4 * irq); + + base =3D GICD + GET_SPI_REG_OFFSET(IROUTER, is_espi); + base +=3D i * sizeof(irqs->irouter); + for ( irq =3D 0; irq < ARRAY_SIZE(irqs->irouter); irq++ ) + irqs->irouter[irq] =3D readq_relaxed_non_atomic(base + 8 * irq); + + base =3D GICD + GET_SPI_REG_OFFSET(ISACTIVER, is_espi); + base +=3D i * sizeof(irqs->isactiver); + irqs->isactiver =3D readl_relaxed(base); + + base =3D GICD + GET_SPI_REG_OFFSET(ISENABLER, is_espi); + base +=3D i * sizeof(irqs->isenabler); + irqs->isenabler =3D readl_relaxed(base); +} + +static void gicv3_restore_spi_irq_config(struct dist_irq_block *irqs, + unsigned int i, bool is_espi) +{ + void __iomem *base; + unsigned int irq; + + base =3D GICD + GET_SPI_REG_OFFSET(ICFGR, is_espi) + i * sizeof(irqs->= icfgr); + writel_relaxed(irqs->icfgr[0], base); + writel_relaxed(irqs->icfgr[1], base + 4); + + base =3D GICD + GET_SPI_REG_OFFSET(IPRIORITYR, is_espi); + base +=3D i * sizeof(irqs->ipriorityr); + for ( irq =3D 0; irq < ARRAY_SIZE(irqs->ipriorityr); irq++ ) + writel_relaxed(irqs->ipriorityr[irq], base + 4 * irq); +} + +static void gicv3_restore_spi_irq_routing(struct dist_irq_block *irqs, + unsigned int i, bool is_espi) +{ + void __iomem *base; + unsigned int irq; + + base =3D GICD + GET_SPI_REG_OFFSET(IROUTER, is_espi); + base +=3D i * sizeof(irqs->irouter); + for ( irq =3D 0; irq < ARRAY_SIZE(irqs->irouter); irq++ ) + writeq_relaxed_non_atomic(irqs->irouter[irq], base + 8 * irq); +} + +static void gicv3_restore_spi_irq_state(struct dist_irq_block *irqs, + unsigned int i, bool is_espi) +{ + void __iomem *base; + + base =3D GICD + GET_SPI_REG_OFFSET(ICENABLER, is_espi) + i * 4; + writel_relaxed(GENMASK(31, 0), base); + + base =3D GICD + GET_SPI_REG_OFFSET(ISENABLER, is_espi); + base +=3D i * sizeof(irqs->isenabler); + writel_relaxed(irqs->isenabler, base); + + base =3D GICD + GET_SPI_REG_OFFSET(ICACTIVER, is_espi) + i * 4; + writel_relaxed(GENMASK(31, 0), base); + + base =3D GICD + GET_SPI_REG_OFFSET(ISACTIVER, is_espi); + base +=3D i * sizeof(irqs->isactiver); + writel_relaxed(irqs->isactiver, base); +} + +static int gicv3_suspend(void) +{ + unsigned int i; + void __iomem *base; + int ret; + struct redist_ctx *rdist =3D &gicv3_ctx.rdist; + + /* Save GICC configuration */ + gicv3_ctx.cpu.ctlr =3D READ_SYSREG(ICC_CTLR_EL1); + gicv3_ctx.cpu.pmr =3D READ_SYSREG(ICC_PMR_EL1); + gicv3_ctx.cpu.bpr =3D READ_SYSREG(ICC_BPR1_EL1); + gicv3_ctx.cpu.sre_el2 =3D READ_SYSREG(ICC_SRE_EL2); + gicv3_ctx.cpu.grpen =3D READ_SYSREG(ICC_IGRPEN1_EL1); + + gicv3_disable_interface(); + + ret =3D gicv3_disable_redist(); + if ( ret ) + goto out_enable_iface; + + /* Save GICR configuration */ + gicv3_redist_wait_for_rwp(); + + base =3D GICD_RDIST_BASE; + + rdist->ctlr =3D readl_relaxed(base + GICR_CTLR); + + rdist->propbase =3D readq_relaxed(base + GICR_PROPBASER); + rdist->pendbase =3D readq_relaxed(base + GICR_PENDBASER); + + base =3D GICD_RDIST_SGI_BASE; + + /* Save priority on PPI and SGI interrupts */ + for ( i =3D 0; i < NR_GIC_LOCAL_IRQS / 4; i++ ) + rdist->ipriorityr[i] =3D readl_relaxed(base + GICR_IPRIORITYR0 + 4= * i); + + rdist->isactiver =3D readl_relaxed(base + GICR_ISACTIVER0); + rdist->isenabler =3D readl_relaxed(base + GICR_ISENABLER0); + rdist->igroupr =3D readl_relaxed(base + GICR_IGROUPR0); + rdist->icfgr =3D readl_relaxed(base + GICR_ICFGR1); + + /* Save GICD configuration */ + gicv3_dist_wait_for_rwp(); + gicv3_ctx.dist.ctlr =3D readl_relaxed(GICD + GICD_CTLR); + + for ( i =3D 1; i < DIV_ROUND_UP(gicv3_info.nr_lines, 32); i++ ) + gicv3_store_spi_irq_block(gicv3_ctx.dist.irqs + i - 1, i, false); + +#ifdef CONFIG_GICV3_ESPI + for ( i =3D 0; i < gic_number_espis() / 32; i++ ) + gicv3_store_spi_irq_block(gicv3_ctx.dist.espi_irqs + i, i, true); +#endif + + return 0; + + out_enable_iface: + gicv3_hyp_enable(true); + WRITE_SYSREG(gicv3_ctx.cpu.grpen, ICC_IGRPEN1_EL1); + isb(); + + return ret; +} + +static void gicv3_resume(void) +{ + int ret; + unsigned int i; + uint32_t dist_ctlr; + void __iomem *base; + struct redist_ctx *rdist =3D &gicv3_ctx.rdist; + + writel_relaxed(0, GICD + GICD_CTLR); + + for ( i =3D NR_GIC_LOCAL_IRQS; i < gicv3_info.nr_lines; i +=3D 32 ) + writel_relaxed(GENMASK(31, 0), GICD + GICD_IGROUPR + (i / 32) * 4); + + for ( i =3D 1; i < DIV_ROUND_UP(gicv3_info.nr_lines, 32); i++ ) + gicv3_restore_spi_irq_config(gicv3_ctx.dist.irqs + i - 1, i, false= ); + +#ifdef CONFIG_GICV3_ESPI + for ( i =3D 0; i < gic_number_espis() / 32; i++ ) + { + writel_relaxed(GENMASK(31, 0), GICD + GICD_IGROUPRnE + i * 4); + gicv3_restore_spi_irq_config(gicv3_ctx.dist.espi_irqs + i, i, true= ); + } +#endif + + dist_ctlr =3D gicv3_ctx.dist.ctlr & GICD_CTLR_ARE_NS; + if ( dist_ctlr ) + { + writel_relaxed(dist_ctlr, GICD + GICD_CTLR); + gicv3_dist_wait_for_rwp(); + + for ( i =3D 1; i < DIV_ROUND_UP(gicv3_info.nr_lines, 32); i++ ) + gicv3_restore_spi_irq_routing(gicv3_ctx.dist.irqs + i - 1, i, + false); + +#ifdef CONFIG_GICV3_ESPI + for ( i =3D 0; i < gic_number_espis() / 32; i++ ) + gicv3_restore_spi_irq_routing(gicv3_ctx.dist.espi_irqs + i, i, + true); +#endif + } + + for ( i =3D 1; i < DIV_ROUND_UP(gicv3_info.nr_lines, 32); i++ ) + gicv3_restore_spi_irq_state(gicv3_ctx.dist.irqs + i - 1, i, false); + +#ifdef CONFIG_GICV3_ESPI + for ( i =3D 0; i < gic_number_espis() / 32; i++ ) + gicv3_restore_spi_irq_state(gicv3_ctx.dist.espi_irqs + i, i, true); +#endif + + writel_relaxed(gicv3_ctx.dist.ctlr, GICD + GICD_CTLR); + gicv3_dist_wait_for_rwp(); + + ret =3D gicv3_lpi_init_rdist(GICD_RDIST_BASE); + /* + * If LPIs are already enabled, assume firmware or the still-powered + * redistributor has valid PROPBASER/PENDBASER and skip reprogramming. + * Return -EBUSY so callers can ignore this case. + */ + if ( ret && ret !=3D -ENODEV && ret !=3D -EBUSY ) + panic("GICv3: Failed to re-initialize LPIs during resume\n"); + else if ( ret =3D=3D -EBUSY ) /* extra checks, just to be sure */ + { + base =3D GICD_RDIST_BASE; + if ( readq_relaxed(base + GICR_PROPBASER) !=3D rdist->propbase || + readq_relaxed(base + GICR_PENDBASER) !=3D rdist->pendbase ) + panic("GICv3: LPIs already enabled with unexpected PROPBASER/P= ENDBASER during resume\n"); + } + + /* Restore GICR (Redistributor) configuration */ + if ( gicv3_enable_redist() ) + panic("GICv3: Failed to re-enable redistributor during resume\n"); + + base =3D GICD_RDIST_SGI_BASE; + + writel_relaxed(GENMASK(31, 0), base + GICR_ICENABLER0); + gicv3_redist_wait_for_rwp(); + + for ( i =3D 0; i < NR_GIC_LOCAL_IRQS / 4; i++ ) + writel_relaxed(rdist->ipriorityr[i], base + GICR_IPRIORITYR0 + i *= 4); + + writel_relaxed(rdist->isactiver, base + GICR_ISACTIVER0); + writel_relaxed(rdist->igroupr, base + GICR_IGROUPR0); + writel_relaxed(rdist->icfgr, base + GICR_ICFGR1); + + gicv3_redist_wait_for_rwp(); + + writel_relaxed(rdist->isenabler, base + GICR_ISENABLER0); + writel_relaxed(rdist->ctlr, GICD_RDIST_BASE + GICR_CTLR); + + gicv3_redist_wait_for_rwp(); + + WRITE_SYSREG(gicv3_ctx.cpu.sre_el2, ICC_SRE_EL2); + isb(); + + /* Restore CPU interface (System registers) */ + WRITE_SYSREG(gicv3_ctx.cpu.pmr, ICC_PMR_EL1); + WRITE_SYSREG(gicv3_ctx.cpu.bpr, ICC_BPR1_EL1); + WRITE_SYSREG(gicv3_ctx.cpu.ctlr, ICC_CTLR_EL1); + WRITE_SYSREG(gicv3_ctx.cpu.grpen, ICC_IGRPEN1_EL1); + isb(); + + gicv3_hyp_init(); +} + +#endif /* CONFIG_SYSTEM_SUSPEND */ + /* Set up the GIC */ static int __init gicv3_init(void) { @@ -1994,6 +2342,10 @@ static int __init gicv3_init(void) =20 gicv3_hyp_init(); =20 +#ifdef CONFIG_SYSTEM_SUSPEND + gicv3_alloc_context(); +#endif + out: spin_unlock(&gicv3.lock); =20 @@ -2033,6 +2385,10 @@ static const struct gic_hw_operations gicv3_ops =3D { #endif .iomem_deny_access =3D gicv3_iomem_deny_access, .do_LPI =3D gicv3_do_LPI, +#ifdef CONFIG_SYSTEM_SUSPEND + .suspend =3D gicv3_suspend, + .resume =3D gicv3_resume, +#endif }; =20 static int __init gicv3_dt_preinit(struct dt_device_node *node, const void= *data) diff --git a/xen/arch/arm/include/asm/gic_v3_defs.h b/xen/arch/arm/include/= asm/gic_v3_defs.h index c373b94d19..992c8f9c2f 100644 --- a/xen/arch/arm/include/asm/gic_v3_defs.h +++ b/xen/arch/arm/include/asm/gic_v3_defs.h @@ -94,6 +94,7 @@ #define GICD_TYPE_LPIS (1U << 17) =20 #define GICD_CTLR_RWP (1UL << 31) +#define GICD_CTLR_DS (1U << 6) #define GICD_CTLR_ARE_NS (1U << 4) #define GICD_CTLR_ENABLE_G1A (1U << 1) #define GICD_CTLR_ENABLE_G1 (1U << 0) --=20 2.43.0 From nobody Sat May 30 11:15:55 2026 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass(p=none dis=none) header.from=gmail.com ARC-Seal: i=1; a=rsa-sha256; t=1778605817; cv=none; d=zohomail.com; s=zohoarc; b=cje7xNB6REtEF96K5KRXdSwM5PnFcelzmwiakG7mo2OooLZzgO76xXmx3IxnFAmQP+mtoEaSKqGUt+hGnOqZJpkylxMjASsBx/O5RWxK2niyjzVaNPCze9wEdRA5PrByNjNiQ7Qo2ANrYSPlvUizT/AXeRUgK4ZWHgTB94H2fLo= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1778605817; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=0IZE3vvZX8wTCOIvxrr5h+/UmQXvZVQaktiFrkuAMu4=; b=k6JXGN5/0ty0Y6WHFB4GVPdJZG+Cmi6XHOTnyBS5jjP+XCHin10xPUbbdxTRhry9X/7CG0kehufTzuLhhppAvm9dbduyA5SH+5+mOf44uwa4AHZrsvzsUzdylpXXArogRA6iDkBPkX6CtwvSALJQxwOCEyC5eDSZ+dKHN9O/y8I= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1778605817076860.2939297912492; Tue, 12 May 2026 10:10:17 -0700 (PDT) Received: from list by lists.xenproject.org with outflank-mailman.1307278.1579025 (Exim 4.92) (envelope-from ) id 1wMqcB-0006jx-CT; Tue, 12 May 2026 17:09:51 +0000 Received: by outflank-mailman (output) from mailman id 1307278.1579025; Tue, 12 May 2026 17:09:51 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1wMqcB-0006jh-7h; Tue, 12 May 2026 17:09:51 +0000 Received: by outflank-mailman (input) for mailman id 1307278; Tue, 12 May 2026 17:09:50 +0000 Received: from mx.expurgate.net ([195.190.135.10]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1wMqc9-0006ej-Vt for xen-devel@lists.xenproject.org; Tue, 12 May 2026 17:09:50 +0000 Received: from mx.expurgate.net (helo=localhost) by mx.expurgate.net with esmtp id 1wMqc9-007RzZ-9b for xen-devel@lists.xenproject.org; Tue, 12 May 2026 19:09:49 +0200 Received: from [10.42.69.12] (helo=localhost) by localhost with ESMTP (eXpurgate MTA 0.9.1) (envelope-from ) id 6a035ecd-2eae-0a2a0a5409dd-0a2a450cc83c-40 for ; Tue, 12 May 2026 19:09:49 +0200 Received: from [209.85.128.49] (helo=mail-wm1-f49.google.com) by tlsNG-d25034.mxtls.expurgate.net with ESMTPS (eXpurgate 4.56.1) (envelope-from ) id 6a035edd-62f1-0a2a450c0019-d1558031dcc7-3 for ; Tue, 12 May 2026 19:09:49 +0200 Received: by mail-wm1-f49.google.com with SMTP id 5b1f17b1804b1-48909558b3aso59332165e9.0 for ; Tue, 12 May 2026 10:09:49 -0700 (PDT) Received: from EPUAKYIW02F7.. ([45.12.26.18]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-48fc8cccf90sm10838845e9.0.2026.05.12.10.09.46 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 12 May 2026 10:09:47 -0700 (PDT) X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" Authentication-Results: eu.smtp.expurgate.cloud; dkim=pass header.s=20251104 header.d=gmail.com header.i="@gmail.com" header.h="Content-Transfer-Encoding:MIME-Version:References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1778605788; x=1779210588; darn=lists.xenproject.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=0IZE3vvZX8wTCOIvxrr5h+/UmQXvZVQaktiFrkuAMu4=; b=IcD5Ikeb+fSV5jb87eN5sdQt4h2F2BytaEeYehJUmw40qv/cp+XqzuB4FGVlAd+9SP GfpwGoHKfP2AF4D9VUK3Aj0Cz/KZz2sLaIFezm4Wd2nnGIpbvZrSIVUaUQSrnngX8hM4 k4wFFEh2UBX94XBGL1wy/S8f+1ki0X6Z6KCtAdEi4Z7t7zJOCsM//hUg2e+/EIX8VXFI TBmkki1ew5FG3Y2Cr9YQIt7pNS7SI/WLf6m7ioxXFUttOH4uf1rPCE3RLzuLxeiHzK+u S9ovfQFWIi67h3j5jTSXlXQFOU/ngOdYSmalLxkC6H4fxeetUDhbssLkgj5EJW7NaPLm fuSw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1778605788; x=1779210588; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=0IZE3vvZX8wTCOIvxrr5h+/UmQXvZVQaktiFrkuAMu4=; b=Dv8n+k4f4HLDPxVvDwiaJk5+VmA7K2Mp15xHPo97UC4Ps4V9yT5Swa07uWPv5DKHoh YhWVZ4L10HFxYVFpY/GKCfQXqA9//IQ9CZ0cO4nw9E9bF/FxLcIh6pGlRpi0C/j2aDL4 VkoAytNDjPouUiOcQvEFP9ptJAxe2VgfHCl1slepQwv04OOtbm+C0W4qgHPH3NBw03Be CGkm3cmQpKkzN2yWWs9xolP4a+cQMv4/QJCksPgO0MPLnONNBEOChPNjUpBgGh+Wts0U EwrNX30iZJfFbSK+5ifn3tUXzfdrACLOcGmR5tnE6868fOGllp3PbmMnHobkdxDJFDYa cJMw== X-Gm-Message-State: AOJu0YyTaEjCgIvA+gmHhO4EVHCcCY+1SnqMScpScK9BnLo7lkIuG64E 1Sj8piNmPPfoq0M1X7GYPS6pmpbMMq20NkKWXnV65eCPh/CC/brwcIwfIa8TOw== X-Gm-Gg: Acq92OHO1CIs/KGzFDNXleamW1+7YGauD0qAByiutdwvykQ9OF9+5f48aXhABcCy4+o 5pS6oHMCLb3/MHm2OoYLaXlWoS/pa3+RpBodYWEgfYYZiqPYbtY3v9S01rgZGgFiQ5/ns4qDiUb DhCMP78+I8B+4vthgPICauD1sOZiu/TbhzAt4RnDKxd3pSOH0GQGLW4tOuANx31RVf3vDKyPsiR wGBm0PocQOogWmBEIAfTcCRbcyVIoeI/YjSYvIAkVERk86ZskR+Rpa9Xu3xIFDc6+mAuew07iKp ibzuh2nAP6m8JlZjgGFHdgayaFpCtdmPSor8jkV9dx9LrwE0u1QsopUfxVp1VwtRDJygQRfnoJ3 F87d6gQWLDHrUTSKId17jLEtuYhZDTE56eA2s7EhKHlg/XMoMoREWSXCRulliWhXn7XNrisAUxK WUKeIuQrEququ3vsm7m1aoneA4GQ== X-Received: by 2002:a05:600c:3b0f:b0:48a:7965:b943 with SMTP id 5b1f17b1804b1-48e51f4e9admr478114025e9.29.1778605788240; Tue, 12 May 2026 10:09:48 -0700 (PDT) From: Mykola Kvach To: xen-devel@lists.xenproject.org Cc: Mykola Kvach , Stefano Stabellini , Julien Grall , Bertrand Marquis , Michal Orzel , Volodymyr Babchuk , Andrew Cooper , Anthony PERARD , Jan Beulich , =?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= Subject: [PATCH v9 05/13] xen/arm: gic-v3: add ITS suspend/resume support Date: Tue, 12 May 2026 20:07:12 +0300 Message-ID: X-Mailer: git-send-email 2.43.0 In-Reply-To: References: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-purgate-ID: tlsNG-d25034/1778605789-DB57FCF5-620AAA48/0/0 X-purgate-type: clean X-purgate-size: 9457 X-ZohoMail-DKIM: pass (identity @gmail.com) X-ZM-MESSAGEID: 1778605817651158500 Content-Type: text/plain; charset="utf-8" From: Mykola Kvach Handle system suspend/resume for GICv3 with an ITS present so LPIs keep working after firmware powers the GIC down. Save and restore the ITS CTLR, CBASER and BASER registers, and re-establish the collection mapping on resume. Add list_for_each_entry_continue_reverse() in list.h for the ITS suspend error path that needs to roll back partially saved state. Based on Linux commit dba0bc7b76dc: "irqchip/gic-v3-its: Add ability to save/restore ITS state". Signed-off-by: Mykola Kvach --- Changes in V9: - fix the ITS suspend/resume coding-style nits; - preserve the saved GITS_CTLR state while masking the read-only QUIESCENT bit. Changes in V8: - Reword the CBASER/CWRITER comment to match Xen and drop the stale Linux cmd_write reference. - Clarify the list_for_each_entry_continue_reverse() comment. - Factor out per-ITS helpers for collection setup and resume. - Restore each ITS and re-establish its collection mapping in the same loop, so a failed ITS resume is not followed by MAPC/SYNC on that un-restored instance. - panic in case when resume of an ITS failed - cleanup baser cache during suspend --- xen/arch/arm/gic-v3-its.c | 133 ++++++++++++++++++++++++-- xen/arch/arm/gic-v3.c | 11 ++- xen/arch/arm/include/asm/gic_v3_its.h | 23 +++++ xen/include/xen/list.h | 14 +++ 4 files changed, 171 insertions(+), 10 deletions(-) diff --git a/xen/arch/arm/gic-v3-its.c b/xen/arch/arm/gic-v3-its.c index 9005ce8ce5..582c26d964 100644 --- a/xen/arch/arm/gic-v3-its.c +++ b/xen/arch/arm/gic-v3-its.c @@ -335,6 +335,22 @@ static int its_send_cmd_inv(struct host_its *its, return its_send_command(its, cmd); } =20 +static int gicv3_its_setup_collection_single(struct host_its *its, + unsigned int cpu) +{ + int ret; + + ret =3D its_send_cmd_mapc(its, cpu, cpu); + if ( ret ) + return ret; + + ret =3D its_send_cmd_sync(its, cpu); + if ( ret ) + return ret; + + return gicv3_its_wait_commands(its); +} + /* Set up the (1:1) collection mapping for the given host CPU. */ int gicv3_its_setup_collection(unsigned int cpu) { @@ -343,15 +359,7 @@ int gicv3_its_setup_collection(unsigned int cpu) =20 list_for_each_entry(its, &host_its_list, entry) { - ret =3D its_send_cmd_mapc(its, cpu, cpu); - if ( ret ) - return ret; - - ret =3D its_send_cmd_sync(its, cpu); - if ( ret ) - return ret; - - ret =3D gicv3_its_wait_commands(its); + ret =3D gicv3_its_setup_collection_single(its, cpu); if ( ret ) return ret; } @@ -1210,6 +1218,113 @@ int gicv3_its_init(void) return 0; } =20 +#ifdef CONFIG_SYSTEM_SUSPEND +int gicv3_its_suspend(void) +{ + struct host_its *its; + int ret; + + list_for_each_entry( its, &host_its_list, entry ) + { + unsigned int i; + void __iomem *base =3D its->its_base; + + /* + * By the time Xen reaches gic_suspend(), every domain is already = in + * SHUTDOWN_suspend, so ITS-targeting interrupt sources are expect= ed + * to have been quiesced by the owning OS before SYSTEM_SUSPEND. + */ + /* Preserve saved GITS_CTLR state, excluding read-only QUIESCENT. = */ + its->suspend_ctx.ctlr =3D readl_relaxed(base + GITS_CTLR) & + ~GITS_CTLR_QUIESCENT; + ret =3D gicv3_disable_its(its); + if ( ret ) + { + writel_relaxed(its->suspend_ctx.ctlr, base + GITS_CTLR); + goto err; + } + + its->suspend_ctx.cbaser =3D readq_relaxed(base + GITS_CBASER); + + for ( i =3D 0; i < GITS_BASER_NR_REGS; i++ ) + { + uint64_t baser =3D readq_relaxed(base + GITS_BASER0 + i * 8); + + its->suspend_ctx.baser[i] =3D 0; + + if ( !(baser & GITS_VALID_BIT) ) + continue; + + its->suspend_ctx.baser[i] =3D baser; + } + } + + return 0; + + err: + list_for_each_entry_continue_reverse( its, &host_its_list, entry ) + writel_relaxed(its->suspend_ctx.ctlr, its->its_base + GITS_CTLR); + + return ret; +} + +static int gicv3_its_resume_single(struct host_its *its, unsigned int cpu) +{ + void __iomem *base =3D its->its_base; + unsigned int i; + int ret; + + /* + * Make sure that the ITS is disabled. If it fails to quiesce, + * don't restore it since writing to CBASER or BASER + * registers is undefined according to the GIC v3 ITS + * Specification. + */ + WARN_ON(readl_relaxed(base + GITS_CTLR) & GITS_CTLR_ENABLE); + ret =3D gicv3_disable_its(its); + if ( ret ) + return ret; + + writeq_relaxed(its->suspend_ctx.cbaser, base + GITS_CBASER); + + /* + * Writing CBASER resets CREADR to 0, so reset CWRITER to + * keep the command queue pointers aligned. + */ + writeq_relaxed(0, base + GITS_CWRITER); + + /* Restore GITS_BASER from the value cache. */ + for ( i =3D 0; i < GITS_BASER_NR_REGS; i++ ) + { + uint64_t baser =3D its->suspend_ctx.baser[i]; + + if ( !(baser & GITS_VALID_BIT) ) + continue; + + writeq_relaxed(baser, base + GITS_BASER0 + i * 8); + } + + writel_relaxed(its->suspend_ctx.ctlr, base + GITS_CTLR); + + return gicv3_its_setup_collection_single(its, cpu); +} + +void gicv3_its_resume(void) +{ + struct host_its *its; + unsigned int cpu =3D smp_processor_id(); + int ret; + + list_for_each_entry( its, &host_its_list, entry ) + { + ret =3D gicv3_its_resume_single(its, cpu); + if ( ret ) + panic("GICv3: ITS@%"PRIpaddr": failed to restore during resume= : %d\n", + its->addr, ret); + } +} + +#endif /* CONFIG_SYSTEM_SUSPEND */ =20 /* * Local variables: diff --git a/xen/arch/arm/gic-v3.c b/xen/arch/arm/gic-v3.c index 6fc4354e2e..8ef9f416d0 100644 --- a/xen/arch/arm/gic-v3.c +++ b/xen/arch/arm/gic-v3.c @@ -2115,10 +2115,14 @@ static int gicv3_suspend(void) =20 gicv3_disable_interface(); =20 - ret =3D gicv3_disable_redist(); + ret =3D gicv3_its_suspend(); if ( ret ) goto out_enable_iface; =20 + ret =3D gicv3_disable_redist(); + if ( ret ) + goto out_its_resume; + /* Save GICR configuration */ gicv3_redist_wait_for_rwp(); =20 @@ -2154,6 +2158,9 @@ static int gicv3_suspend(void) =20 return 0; =20 + out_its_resume: + gicv3_its_resume(); + out_enable_iface: gicv3_hyp_enable(true); WRITE_SYSREG(gicv3_ctx.cpu.grpen, ICC_IGRPEN1_EL1); @@ -2253,6 +2260,8 @@ static void gicv3_resume(void) =20 gicv3_redist_wait_for_rwp(); =20 + gicv3_its_resume(); + WRITE_SYSREG(gicv3_ctx.cpu.sre_el2, ICC_SRE_EL2); isb(); =20 diff --git a/xen/arch/arm/include/asm/gic_v3_its.h b/xen/arch/arm/include/a= sm/gic_v3_its.h index fc5a84892c..492c468ae0 100644 --- a/xen/arch/arm/include/asm/gic_v3_its.h +++ b/xen/arch/arm/include/asm/gic_v3_its.h @@ -129,6 +129,13 @@ struct host_its { spinlock_t cmd_lock; void *cmd_buf; unsigned int flags; +#ifdef CONFIG_SYSTEM_SUSPEND + struct suspend_ctx { + uint32_t ctlr; + uint64_t cbaser; + uint64_t baser[GITS_BASER_NR_REGS]; + } suspend_ctx; +#endif }; =20 /* Map a collection for this host CPU to each host ITS. */ @@ -204,6 +211,11 @@ uint64_t gicv3_its_get_cacheability(void); uint64_t gicv3_its_get_shareability(void); unsigned int gicv3_its_get_memflags(void); =20 +#ifdef CONFIG_SYSTEM_SUSPEND +int gicv3_its_suspend(void); +void gicv3_its_resume(void); +#endif + #else =20 #ifdef CONFIG_ACPI @@ -271,6 +283,17 @@ static inline int gicv3_its_make_hwdom_dt_nodes(const = struct domain *d, return 0; } =20 +#ifdef CONFIG_SYSTEM_SUSPEND +static inline int gicv3_its_suspend(void) +{ + return 0; +} + +static inline void gicv3_its_resume(void) +{ +} +#endif + #endif /* CONFIG_HAS_ITS */ =20 #endif diff --git a/xen/include/xen/list.h b/xen/include/xen/list.h index 98d8482dab..2aab274157 100644 --- a/xen/include/xen/list.h +++ b/xen/include/xen/list.h @@ -535,6 +535,20 @@ static inline void list_splice_init(struct list_head *= list, &(pos)->member !=3D (head); = \ (pos) =3D list_entry((pos)->member.next, typeof(*(pos)), member)) =20 +/** + * list_for_each_entry_continue_reverse - iterate backwards from the given= point + * @pos: the type * to use as a loop cursor. + * @head: the head for your list. + * @member: the name of the list_head within the struct. + * + * Iterate over list of given type backwards, starting from the element pr= evious + * to the current one in list order. + */ +#define list_for_each_entry_continue_reverse(pos, head, member) \ + for ((pos) =3D list_entry((pos)->member.prev, typeof(*(pos)), member);= \ + &(pos)->member !=3D (head); = \ + (pos) =3D list_entry((pos)->member.prev, typeof(*(pos)), member)) + /** * list_for_each_entry_from - iterate over list of given type from the * current point --=20 2.43.0 From nobody Sat May 30 11:15:55 2026 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass(p=none dis=none) header.from=gmail.com ARC-Seal: i=1; a=rsa-sha256; t=1778605818; cv=none; d=zohomail.com; s=zohoarc; b=j4ER+dXcwvtBd3K8YN2tkiQYucWK+AyqhAD1G94yE+5yqaxDi8oKMyjyYwXMNUMyUnKr3KqChJWR8dfMXgJD1LLZoksThjGRd/2mU7n9JKJXm/ViDxH2LRLMhdyCXYA4lHr184uh1yBbC/99euEWQuDdw0Muub/W939yllQvIDc= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1778605818; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=DakayXl1HcfjZmxzjuOvWT6rEYilW0CjJi4UVOgEk64=; b=Dj4GWlPyfF0oaA8829v61s48ip+wLkZqmWlbHpnWHsNKGoFH6j4dn/37ieWVRe2feGXqUaZL1go0VPNDcQLeZIZuQYUlDigbwGi6vsmSbZZnjZjQiY87dpb0RNDwXi+vX+hzoDUTt/hrHNuORHk/VofQ14BIVqd5nKPA3lBWoEQ= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1778605818726665.6666516439492; Tue, 12 May 2026 10:10:18 -0700 (PDT) Received: from list by lists.xenproject.org with outflank-mailman.1307279.1579034 (Exim 4.92) (envelope-from ) id 1wMqcC-000702-Qo; Tue, 12 May 2026 17:09:52 +0000 Received: by outflank-mailman (output) from mailman id 1307279.1579034; Tue, 12 May 2026 17:09:52 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1wMqcC-0006zo-Ll; Tue, 12 May 2026 17:09:52 +0000 Received: by outflank-mailman (input) for mailman id 1307279; Tue, 12 May 2026 17:09:51 +0000 Received: from mx.expurgate.net ([195.190.135.10]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1wMqcB-0006je-AA for xen-devel@lists.xenproject.org; Tue, 12 May 2026 17:09:51 +0000 Received: from mx.expurgate.net (helo=localhost) by mx.expurgate.net with esmtp id 1wMqcA-007RzZ-NY for xen-devel@lists.xenproject.org; Tue, 12 May 2026 19:09:50 +0200 Received: from [10.42.69.12] (helo=localhost) by localhost with ESMTP (eXpurgate MTA 0.9.1) (envelope-from ) id 6a035ecd-2eae-0a2a0a5409dd-0a2a450cc83c-42 for ; Tue, 12 May 2026 19:09:50 +0200 Received: from [209.85.128.44] (helo=mail-wm1-f44.google.com) by tlsNG-d25034.mxtls.expurgate.net with ESMTPS (eXpurgate 4.56.1) (envelope-from ) id 6a035ede-62f1-0a2a450c0019-d155802cd561-3 for ; Tue, 12 May 2026 19:09:50 +0200 Received: by mail-wm1-f44.google.com with SMTP id 5b1f17b1804b1-488ba840146so52227225e9.1 for ; Tue, 12 May 2026 10:09:50 -0700 (PDT) Received: from EPUAKYIW02F7.. ([45.12.26.18]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-48fc8cccf90sm10838845e9.0.2026.05.12.10.09.48 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 12 May 2026 10:09:48 -0700 (PDT) X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" Authentication-Results: eu.smtp.expurgate.cloud; dkim=pass header.s=20251104 header.d=gmail.com header.i="@gmail.com" header.h="Content-Transfer-Encoding:MIME-Version:References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1778605789; x=1779210589; darn=lists.xenproject.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=DakayXl1HcfjZmxzjuOvWT6rEYilW0CjJi4UVOgEk64=; b=cAQIIWgAWAGIPmnaZZAS4XIMFuJXfTRMsGWqcwEacYG+636c4ClEYaCqJ8enY7erIQ wki+qF17lLkph6PvoVud8mKLdwl2kPCIST2rJLFTKoQqwlbhymFWvjq4Y6R17ZCdh78W rj+0Egmbhcs5c+OGDRwKwrn+enmZcFAKPVVnehqiNkq7VB+ICgJLWkwRDecMJfEsL40J lAJF0jOIl3FRJ3XLp9m0vk3tnqWuG3pJzjsr93hDEfA5rxUuyY2cnbalTsIvtj8zoM7P u8Bc2LECdGECYhgDdZg3MzMhSPy7hMBGMsuK1VMY1Z4vWSUDU8IQNroX0Hl8mdn2FZY/ fShA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1778605789; x=1779210589; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=DakayXl1HcfjZmxzjuOvWT6rEYilW0CjJi4UVOgEk64=; b=hcNGrQ/ozKXRyXe/rcFSMf30VJQRDfslBkr7jv3MD1gmCFQoDMfZzwrsyGsZKymkko Uafu/ckrRbIELzPagxRkWQhTAWiax4BFCoi84ZM92uU11PF9rxUfGx6t9I2NaSIS91GU zzF7bnObajHLoLljfoMiH/6aZwWjmLKXQQQm0zL/iwAx9A2FA1EI8wGeLNa6YbQie0NM qTbybOojidbxf3Op1jggmmmcEZKyWxmrebV46Usb2og1ndICx+GmNTMn7shtNvk7XxWG lB/jOiH8JfI9sIDWp3Q4yd7NRHMuVAKD/o81KkOTbRq3SMDIttEXkvhBCX3YZwRCv6Lb trlg== X-Gm-Message-State: AOJu0YwtMLiGkk/Om7WiUdcOTOMbvWq0YU/XmRFqQorH+evb0xVze5rJ 1tFetFr19UUqaaE/LmIyLdKZJDzWVi59WQT4GB9qpZrKsk04R3jBdi0rKenuFA== X-Gm-Gg: Acq92OGejGBwkRi7a7ChsvyNgw9/a3NQYsDiKsv2rEgHqbx4mN0KTZM7AIce+UpjvOF xDVR4rmq4KgUMG9QChBkl/FdG23JrWQEwop57DrZuO1dnayzkx931U2+OEYdbMEjwIdLRnCa8kA O8L87oGAqtj1xS8Ogiy8LN3nYUtMIqK5rabaVZpDn3gCozC8Ad3VcL3CTVBZChNf4Y/44I2KXrJ OnvGReqKOz3ldnfzZCrhBGP+o8gZ7Ix9YMIOO8OrhQUBNQSYboGaHm5dhbhQl3ywfSSnOOHQ92K roidLMK/kp/ROmT/re2ADMVmm341SgWLO7P1R+MGRle+wkro5ZPnKB7T/Lv78O6YBgbJr+qhsIO ISW1+WzI676sbdZiUjeuGD6a7vKiE8Q3gLD8MpwvwNUD5A/t+Nn2qLdu+KjhM3LxJDhZbdRgM5m a5FyvZMDMDol7W4R3Q6+ZtGbHiAQ== X-Received: by 2002:a05:600c:a11a:b0:48e:60a3:220a with SMTP id 5b1f17b1804b1-48e8fe018bdmr46019965e9.0.1778605789514; Tue, 12 May 2026 10:09:49 -0700 (PDT) From: Mykola Kvach To: xen-devel@lists.xenproject.org Cc: Mykola Kvach , Volodymyr Babchuk , Bertrand Marquis , Jens Wiklander , Stefano Stabellini , Julien Grall , Michal Orzel , Luca Fancellu Subject: [PATCH v9 06/13] xen/arm: tee: keep init_tee_secondary() for hotplug and resume Date: Tue, 12 May 2026 20:07:13 +0300 Message-ID: X-Mailer: git-send-email 2.43.0 In-Reply-To: References: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-purgate-ID: tlsNG-d25034/1778605790-DB370CF5-72B238B9/0/0 X-purgate-type: clean X-purgate-size: 1063 X-ZohoMail-DKIM: pass (identity @gmail.com) X-ZM-MESSAGEID: 1778605820022154100 Content-Type: text/plain; charset="utf-8" From: Mykola Kvach init_tee_secondary() was marked __init and freed after boot. Calling it from the CPU hotplug/resume path then executed discarded code, which could crash Xen. Drop __init so the TEE mediator secondary init can run safely on hotplugged and resumed CPUs. Signed-off-by: Mykola Kvach Reviewed-by: Luca Fancellu Reviewed-by: Bertrand Marquis Reviewed-by: Volodymyr Babchuk --- xen/arch/arm/tee/tee.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/xen/arch/arm/tee/tee.c b/xen/arch/arm/tee/tee.c index 8501443c8e..00e561fc78 100644 --- a/xen/arch/arm/tee/tee.c +++ b/xen/arch/arm/tee/tee.c @@ -128,7 +128,7 @@ static int __init tee_init(void) =20 presmp_initcall(tee_init); =20 -void __init init_tee_secondary(void) +void init_tee_secondary(void) { if ( cur_mediator && cur_mediator->ops->init_secondary ) cur_mediator->ops->init_secondary(); --=20 2.43.0 From nobody Sat May 30 11:15:55 2026 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass(p=none dis=none) header.from=gmail.com ARC-Seal: i=1; a=rsa-sha256; t=1778605814; cv=none; d=zohomail.com; s=zohoarc; b=mPnHBr2LMWeewv3MaupwhN4MYCMdz8LwHa42ST4mqBs8F7CS/djZXHZygQQoE3fPX2LzxQ8lGKpyqoyDF3O3wej9I3cmaxrADb688ml08IY4euJrTZUXIKn7KIihHCoM7Jgf9IL8WhFNCtCUI07zY04enIInprgNrh8DeUVJE7w= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1778605814; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=F0jUJYHm37yuKRA04SvTDMOB3esx5h23DnnVhZYfk0I=; b=Mae4J1cfFVNLbA1QlCnuGSdzlv+4HuwkjSbUINFrAegXCbO0pOxo/9WgfkeEGluvCsjwX9I7+p6OilZBmdFMuTVYVET0dah4sWq9jX+GrfxmYHfS5XDmyqaeReB2gIASDdwgEHhMAtiM0Q82Ozi7uJPhstLf+c1rCwWSa6SDK04= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1778605814343844.0542183690868; Tue, 12 May 2026 10:10:14 -0700 (PDT) Received: from list by lists.xenproject.org with outflank-mailman.1307281.1579042 (Exim 4.92) (envelope-from ) id 1wMqcE-0007Fn-4E; Tue, 12 May 2026 17:09:54 +0000 Received: by outflank-mailman (output) from mailman id 1307281.1579042; Tue, 12 May 2026 17:09:54 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1wMqcE-0007FC-0C; Tue, 12 May 2026 17:09:54 +0000 Received: by outflank-mailman (input) for mailman id 1307281; Tue, 12 May 2026 17:09:52 +0000 Received: from mx.expurgate.net ([195.190.135.10]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1wMqcC-0006to-7x for xen-devel@lists.xenproject.org; Tue, 12 May 2026 17:09:52 +0000 Received: from mx.expurgate.net (helo=localhost) by mx.expurgate.net with esmtp id 1wMqcB-007RzZ-Km for xen-devel@lists.xenproject.org; Tue, 12 May 2026 19:09:51 +0200 Received: from [10.42.69.1] (helo=localhost) by localhost with ESMTP (eXpurgate MTA 0.9.1) (envelope-from ) id 6a035ec9-2eae-0a2a0a5409dd-0a2a4501afae-30 for ; Tue, 12 May 2026 19:09:51 +0200 Received: from [209.85.128.44] (helo=mail-wm1-f44.google.com) by tlsNG-d62444.mxtls.expurgate.net with ESMTPS (eXpurgate 4.56.1) (envelope-from ) id 6a035edf-c1f2-0a2a45010019-d155802cb4e9-3 for ; Tue, 12 May 2026 19:09:51 +0200 Received: by mail-wm1-f44.google.com with SMTP id 5b1f17b1804b1-488af96f6b2so69609145e9.0 for ; Tue, 12 May 2026 10:09:51 -0700 (PDT) Received: from EPUAKYIW02F7.. ([45.12.26.18]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-48fc8cccf90sm10838845e9.0.2026.05.12.10.09.49 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 12 May 2026 10:09:50 -0700 (PDT) X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" Authentication-Results: eu.smtp.expurgate.cloud; dkim=pass header.s=20251104 header.d=gmail.com header.i="@gmail.com" header.h="Content-Transfer-Encoding:MIME-Version:References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1778605791; x=1779210591; darn=lists.xenproject.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=F0jUJYHm37yuKRA04SvTDMOB3esx5h23DnnVhZYfk0I=; b=ggpm0ec0fA0bp1goMWuheTGx9TqOYlr+wHChAfDZ9lBlX256N+/Hq8h6qJfJGRvyVZ heyf7si4eQ6z5hURq5s9p1eGFWm87/QiSl1YrWs0vUkI6uyoY71iKnS3fKv0rb81/eX8 +E/MUPtsH162f+nXLOFVwNZUV7cSvRtn0jPh5x/GUxrYyh0M8CEMiRj7z7d3AtHqwv8U Lu1Tft/KXZYkWk3+aEw1cztnVddb8LrJMq2t38ESERAvp1wNj20bd+wkIwiGwSDnMRld rFDYaYEpVMQK30OFK+4XJGMyeN4ej7ytBlQvk4vgvBQX2o2aX8Z7yHik7zu4HnscL24S 4dbw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1778605791; x=1779210591; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=F0jUJYHm37yuKRA04SvTDMOB3esx5h23DnnVhZYfk0I=; b=AM5rBrh9ZHbuFbYao0P6tFdEQyaVYWFUsY5EUN/+3gkPZDU7epZ9XPnHhIMNRa8lZq v+meheFYvz1hmkFywnwoeN1IZCBorhec9+S+y1FvpdrT8CLFNOG/rPyr0tIq8AXahT3B w95NfvtXFRmCKaquLwCzcjYrRHqPX55ClJVBXsCOwBZORYlB3F3C75EwntJEyqJtVzFs fq88OuPtrm5IIH+3E9c1h0YkA2jQUeTyc6DtEao6+0ELHicQSaNWzJRiVelzZP/Ua0jy H/LuLllPOjmvbM9k2C1fx3V0zwbFGLznu2+aIoHtbMWTmmgw7Dww+fKr0uQS5MC6PADo p5JQ== X-Gm-Message-State: AOJu0YwPag7Q8H60E+YjjzSyAHlw0EP4S3Oqr9gZzs8S4ZXdNGc28elG hC2RU40RKNhcOB1wGp0Cs7nhMEFFUnh0C2oSUQfV5xxUkeu3jIFluZPUPzv7fQ== X-Gm-Gg: Acq92OFxx30AB80x+yygAaGwwrAbI+t9Q7jUNpYxRkMrerpoAABQnlu6/b3xN0m9ZHa 1twBTQxgQzsmGGXmqEbq91X1Eq7slS/+EyVYry73pKed/6B4ivfaKUWdxEA19dU0UqH5y/XzLGQ n8FFTFvC1rRyzp+2ZiwgywNwkWMYT1BwyRqtYYkNNUzkGMLkDMDALiwjYgMzySDp8i/Kv+Jg/Gd WFZR0XkCdz4ZyO2e/XTuFGA9gHYGqjNb1+TwbLk999jY8IBbtlvlePnxvQfv5EW4NooeigX+Wud 6lapYgM68xBfe6LG5prtpbYSieyIg9i/72VWKWX0fPoUF0AMeeuBQUS+y4cC+qOeETMSaVqb+Od JqgSdI3SEP759YlNL9/Grk9cwsOM5+SQMUlz8RkbZLfhYZQjrEcQVc83jrTPZGAQAEJaMfr5CBU xsMdnQlQrqYuSlBStaYBGLvJtiBA== X-Received: by 2002:a05:600c:8b68:b0:48a:5574:3a5b with SMTP id 5b1f17b1804b1-48e70809f8fmr252913185e9.27.1778605790729; Tue, 12 May 2026 10:09:50 -0700 (PDT) From: Mykola Kvach To: xen-devel@lists.xenproject.org Cc: Mykola Kvach , Volodymyr Babchuk , Bertrand Marquis , Jens Wiklander , Stefano Stabellini , Julien Grall , Michal Orzel , Luca Fancellu Subject: [PATCH v9 07/13] xen/arm: ffa: fix notification SRI across CPU hotplug/suspend Date: Tue, 12 May 2026 20:07:14 +0300 Message-ID: X-Mailer: git-send-email 2.43.0 In-Reply-To: References: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-purgate-ID: tlsNG-d62444/1778605791-AFF50FF4-A4C47A8C/0/0 X-purgate-type: clean X-purgate-size: 3700 X-ZohoMail-DKIM: pass (identity @gmail.com) X-ZM-MESSAGEID: 1778605815603158501 Content-Type: text/plain; charset="utf-8" From: Mykola Kvach The FF-A notification SRI interrupt handler was not correctly tied to CPU hotplug and suspend/resume. As a result, CPUs going offline and back online could end up with stale or missing handlers, breaking delivery of FF-A notifications. Signed-off-by: Mykola Kvach Reviewed-by: Luca Fancellu Reviewed-by: Bertrand Marquis --- xen/arch/arm/tee/ffa_notif.c | 63 ++++++++++++++++++++++++++++-------- 1 file changed, 50 insertions(+), 13 deletions(-) diff --git a/xen/arch/arm/tee/ffa_notif.c b/xen/arch/arm/tee/ffa_notif.c index 186e726412..513c399594 100644 --- a/xen/arch/arm/tee/ffa_notif.c +++ b/xen/arch/arm/tee/ffa_notif.c @@ -360,10 +360,28 @@ static int32_t ffa_notification_bitmap_destroy(uint16= _t vm_id) return ffa_simple_call(FFA_NOTIFICATION_BITMAP_DESTROY, vm_id, 0, 0, 0= ); } =20 -void ffa_notif_init_interrupt(void) +static DEFINE_PER_CPU_READ_MOSTLY(struct irqaction, sri_irq); + +static int request_sri_irq(void) { int ret; + struct irqaction *sri_action =3D &this_cpu(sri_irq); + + sri_action->name =3D "FF-A notif"; + sri_action->handler =3D notif_irq_handler; + sri_action->dev_id =3D NULL; + sri_action->free_on_release =3D 0; + + ret =3D setup_irq(notif_sri_irq, 0, sri_action); + if ( ret ) + printk(XENLOG_ERR "ffa: setup_irq irq %u failed: error %d\n", + notif_sri_irq, ret); =20 + return ret; +} + +void ffa_notif_init_interrupt(void) +{ if ( fw_notif_enabled && notif_sri_irq < NR_GIC_SGI ) { /* @@ -376,14 +394,36 @@ void ffa_notif_init_interrupt(void) * pending, while the SPMC in the secure world will not notice that * the interrupt was lost. */ - ret =3D request_irq(notif_sri_irq, 0, notif_irq_handler, "FF-A not= if", - NULL); - if ( ret ) - printk(XENLOG_ERR "ffa: request_irq irq %u failed: error %d\n", - notif_sri_irq, ret); + request_sri_irq(); } } =20 +static void deinit_ffa_notif_interrupt(void) +{ + if ( fw_notif_enabled && notif_sri_irq < NR_GIC_SGI ) + release_irq(notif_sri_irq, NULL); +} + +static int cpu_ffa_notif_callback(struct notifier_block *nfb, + unsigned long action, + void *hcpu) +{ + switch ( action ) + { + case CPU_DYING: + deinit_ffa_notif_interrupt(); + break; + default: + break; + } + + return NOTIFY_DONE; +} + +static struct notifier_block cpu_ffa_notif_nfb =3D { + .notifier_call =3D cpu_ffa_notif_callback, +}; + void ffa_notif_init(void) { const struct arm_smccc_1_2_regs arg =3D { @@ -392,7 +432,6 @@ void ffa_notif_init(void) }; struct arm_smccc_1_2_regs resp; unsigned int irq; - int ret; =20 /* Only enable fw notification if all ABIs we need are supported */ if ( ffa_fw_supports_fid(FFA_NOTIFICATION_BITMAP_CREATE) && @@ -408,13 +447,11 @@ void ffa_notif_init(void) notif_sri_irq =3D irq; if ( irq >=3D NR_GIC_SGI ) irq_set_type(irq, IRQ_TYPE_EDGE_RISING); - ret =3D request_irq(irq, 0, notif_irq_handler, "FF-A notif", NULL); - if ( ret ) - { - printk(XENLOG_ERR "ffa: request_irq irq %u failed: error %d\n", - irq, ret); + + if ( request_sri_irq() ) return; - } + + register_cpu_notifier(&cpu_ffa_notif_nfb); fw_notif_enabled =3D true; } } --=20 2.43.0 From nobody Sat May 30 11:15:55 2026 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass(p=none dis=none) header.from=gmail.com ARC-Seal: i=1; a=rsa-sha256; t=1778605827; cv=none; d=zohomail.com; s=zohoarc; b=mMxmzHsAbgU0KU1YZ6PS7n1H1yQWeh5r8Elfka+xYhCH+hwWizygLftgGNQCwGnsk2qqrreq4XInguvm1C5EZt9c73HSTRKceCjxCpjDl1AERwhoDSGf3iNTa+69mFM1PmC6zSRtNZ0XH01F5TRl6eiBXfUeBOhR+hckTEYX8rQ= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1778605827; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=pT1AJ8vJVr+Es3f8Ijw7DVXZgIihu6GcB10i8wyJqXY=; b=OufkGwai56slY9J4o/PyATxA3xrHg0qBB7s5H6hClGXyC94dVSRbVzNva4MiG09GQimHymmfEbL3H02WTAh8uLdHoCLzNlR9ll1zjFNCrVz6aerZ+Y3O9RMZG4lz6Y54TQGd+xhHVCXqhSZDXNpjpfT5CVTVGY0ZnNPgPeJNM3g= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1778605827449120.26001967584898; Tue, 12 May 2026 10:10:27 -0700 (PDT) Received: from list by lists.xenproject.org with outflank-mailman.1307283.1579047 (Exim 4.92) (envelope-from ) id 1wMqcE-0007IU-Fh; Tue, 12 May 2026 17:09:54 +0000 Received: by outflank-mailman (output) from mailman id 1307283.1579047; Tue, 12 May 2026 17:09:54 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1wMqcE-0007I9-9w; Tue, 12 May 2026 17:09:54 +0000 Received: by outflank-mailman (input) for mailman id 1307283; Tue, 12 May 2026 17:09:53 +0000 Received: from mx.expurgate.net ([195.190.135.10]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1wMqcD-0007Av-Q1 for xen-devel@lists.xenproject.org; Tue, 12 May 2026 17:09:53 +0000 Received: from mx.expurgate.net (helo=localhost) by mx.expurgate.net with esmtp id 1wMqcD-007RzZ-6w for xen-devel@lists.xenproject.org; Tue, 12 May 2026 19:09:53 +0200 Received: from [10.42.69.1] (helo=localhost) by localhost with ESMTP (eXpurgate MTA 0.9.1) (envelope-from ) id 6a035ec9-2eae-0a2a0a5409dd-0a2a4501afae-36 for ; Tue, 12 May 2026 19:09:53 +0200 Received: from [209.85.128.44] (helo=mail-wm1-f44.google.com) by tlsNG-d62444.mxtls.expurgate.net with ESMTPS (eXpurgate 4.56.1) (envelope-from ) id 6a035ee1-c1f2-0a2a45010019-d155802cc06e-3 for ; Tue, 12 May 2026 19:09:53 +0200 Received: by mail-wm1-f44.google.com with SMTP id 5b1f17b1804b1-488a9033b2cso53223425e9.2 for ; Tue, 12 May 2026 10:09:53 -0700 (PDT) Received: from EPUAKYIW02F7.. ([45.12.26.18]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-48fc8cccf90sm10838845e9.0.2026.05.12.10.09.51 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 12 May 2026 10:09:51 -0700 (PDT) X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" Authentication-Results: eu.smtp.expurgate.cloud; dkim=pass header.s=20251104 header.d=gmail.com header.i="@gmail.com" header.h="Content-Transfer-Encoding:MIME-Version:References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1778605792; x=1779210592; darn=lists.xenproject.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=pT1AJ8vJVr+Es3f8Ijw7DVXZgIihu6GcB10i8wyJqXY=; b=XQaNkCv8e/xa6SJp6IaorzUiICTIe73+1JmcUsd7VJNI0RZcR/jVPptk9XT+g4F8XT dpgdWPnMiKlTQPjnBTk7R7O/LBC8bki+4WzgRIelT0JFTndVNaBdBlT9UkgC24RfqeHi bCuebZD7KY52QT8zBbzic4P7578bpyk174LxK9ItpjjjFapdw/SB0I85P8m2LAZqa7n/ V1eop4xuJVZGvu5Iij9vfSL0VVVJa2NGkzOp2b8k+TOsrb8V3tEb41kZHw225i7HAiiw cSSgBR8CPd4SUxa5tVMRGafOHG5YNJZ5aDx90aOq9oWTMuLS5x21NwQ7xu7W6moho+uW xTCg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1778605792; x=1779210592; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=pT1AJ8vJVr+Es3f8Ijw7DVXZgIihu6GcB10i8wyJqXY=; b=TenhDgYb3Qkuxlbo6e+xIykiN77Qb26G4xBrgHnmruAXNWm+zKDD811w1t6nvZLnnM lIPvSfAxsvp9j+UFFm5srk3KuD0Bfq1pGjSeL0IjsIlVyA3R8oacExbAPXAUtYq1yWuZ tW7A3dEJ4J1uGvyxlSNsDRqXbqB/WFjEQ492mLKo5coS4piZwX2BYT+hR2QCNPtaTnTc ImYh2sNqDK01ZtRnSmmC8CKQG8cuxnBcConKb+gmvRT0JMvYETH0NuXpFiIwXJeR68lo 6Kaj9nYTzBcKjhh6mnaNmsZHodj4H+ec6RcGLcuuDkdPPzk4o1sTwwMVx3Nxog2gT0Of y4Mg== X-Gm-Message-State: AOJu0YzpJJL9q5yNmz9vUVyTuSIeXrRBJxIyBjUtiGkvB6O3yjh7kAzM mvkY3wvhIPmkkUJJ7KeboRfU+oM8QvVKB21AYkC7Ubv3IVfkUVo7mDoGxVMmCg== X-Gm-Gg: Acq92OHfz1CjpF/tiBPvlJR8Ais0/YSYfZOnUJe12xXnbNF/XnV7xbg8CF450W+kCAR 8DZpVYMQ2uMZCMm3Ov2xHbnztZKQ1c8lrGlF7RMx8hZUYW19PAisKeOeebwJ8aDW/az2smWpZSx kpzFAvdYPwGaxXyZne4KO46JYRLwsakaT1ItmhVYSqzHB2xi6qoLuRF4jo/NQIUqQF9bygu4Jrb IBmbyPVdkKE4v7QHpHd4TnT72kT+ESyexAuwpNqRI0sx5IdvUlKqV1fhWaiTJGiZvha9zPznRDN dem1VqLHSc3omvQp5aYdvdpX5AnQ6+/7Mt51AN8MjXLn0Gle8aa6UKeBp/K+NS0SDfXyeaJJHHj 3oazqu/gZOMW8sp86uTt/IzT16FqanYobOfW6VT63a7XHvxRzxq9B2TVoO8gkGSpnwSyUymNL7t w8Dt1a28it85GM4kC3kZsvmYYe+A== X-Received: by 2002:a05:600c:4ec7:b0:48e:51f8:eb37 with SMTP id 5b1f17b1804b1-48e706d8ad9mr242690535e9.25.1778605791897; Tue, 12 May 2026 10:09:51 -0700 (PDT) From: Mykola Kvach To: xen-devel@lists.xenproject.org Cc: Mykola Kvach , Stefano Stabellini , Julien Grall , Bertrand Marquis , Michal Orzel , Volodymyr Babchuk Subject: [PATCH v9 08/13] iommu/ipmmu-vmsa: Implement suspend/resume callbacks Date: Tue, 12 May 2026 20:07:15 +0300 Message-ID: <4162aa58c351677a4a267fe85989c6d4e07487d8.1778605274.git.mykola_kvach@epam.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: References: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-purgate-ID: tlsNG-d62444/1778605793-B715BFF4-1027C592/0/0 X-purgate-type: clean X-purgate-size: 13768 X-ZohoMail-DKIM: pass (identity @gmail.com) X-ZM-MESSAGEID: 1778605828348154101 Content-Type: text/plain; charset="utf-8" From: Oleksandr Tyshchenko Store and restore active context and micro-TLB registers. Tested on R-Car H3 Starter Kit. Signed-off-by: Oleksandr Tyshchenko Signed-off-by: Mykola Kvach --- Changes in V9: - set dt_device_set_protected() only after ipmmu_alloc_ctx_suspend() succeeds, so DT devices do not remain protected on allocation failure. Changes in V7: - moved suspend context allocation before pci stuff --- xen/drivers/passthrough/arm/ipmmu-vmsa.c | 319 +++++++++++++++++++++-- 1 file changed, 304 insertions(+), 15 deletions(-) diff --git a/xen/drivers/passthrough/arm/ipmmu-vmsa.c b/xen/drivers/passthr= ough/arm/ipmmu-vmsa.c index fa9ab9cb13..e1b47a5824 100644 --- a/xen/drivers/passthrough/arm/ipmmu-vmsa.c +++ b/xen/drivers/passthrough/arm/ipmmu-vmsa.c @@ -71,6 +71,8 @@ }) #endif =20 +#define dev_dbg(dev, fmt, ...) \ + dev_print(dev, XENLOG_DEBUG, fmt, ## __VA_ARGS__) #define dev_info(dev, fmt, ...) \ dev_print(dev, XENLOG_INFO, fmt, ## __VA_ARGS__) #define dev_warn(dev, fmt, ...) \ @@ -130,6 +132,24 @@ struct ipmmu_features { unsigned int imuctr_ttsel_mask; }; =20 +#ifdef CONFIG_SYSTEM_SUSPEND + +struct ipmmu_reg_ctx { + unsigned int imttlbr0; + unsigned int imttubr0; + unsigned int imttbcr; + unsigned int imctr; +}; + +struct ipmmu_vmsa_backup { + struct device *dev; + unsigned int *utlbs_val; + unsigned int *asids_val; + struct list_head list; +}; + +#endif + /* Root/Cache IPMMU device's information */ struct ipmmu_vmsa_device { struct device *dev; @@ -142,6 +162,9 @@ struct ipmmu_vmsa_device { struct ipmmu_vmsa_domain *domains[IPMMU_CTX_MAX]; unsigned int utlb_refcount[IPMMU_UTLB_MAX]; const struct ipmmu_features *features; +#ifdef CONFIG_SYSTEM_SUSPEND + struct ipmmu_reg_ctx *reg_backup[IPMMU_CTX_MAX]; +#endif }; =20 /* @@ -547,6 +570,245 @@ static void ipmmu_domain_free_context(struct ipmmu_vm= sa_device *mmu, spin_unlock_irqrestore(&mmu->lock, flags); } =20 +#ifdef CONFIG_SYSTEM_SUSPEND + +static DEFINE_SPINLOCK(ipmmu_devices_backup_lock); +static LIST_HEAD(ipmmu_devices_backup); + +static struct ipmmu_reg_ctx root_pgtable[IPMMU_CTX_MAX]; + +static uint32_t ipmmu_imuasid_read(struct ipmmu_vmsa_device *mmu, + unsigned int utlb) +{ + return ipmmu_read(mmu, ipmmu_utlb_reg(mmu, IMUASID(utlb))); +} + +static void ipmmu_utlbs_backup(struct ipmmu_vmsa_device *mmu) +{ + struct ipmmu_vmsa_backup *backup_data; + + dev_dbg(mmu->dev, "Handle micro-TLBs backup\n"); + + spin_lock(&ipmmu_devices_backup_lock); + + list_for_each_entry( backup_data, &ipmmu_devices_backup, list ) + { + struct iommu_fwspec *fwspec =3D dev_iommu_fwspec_get(backup_data->= dev); + unsigned int i; + + if ( to_ipmmu(backup_data->dev) !=3D mmu ) + continue; + + for ( i =3D 0; i < fwspec->num_ids; i++ ) + { + unsigned int utlb =3D fwspec->ids[i]; + + backup_data->asids_val[i] =3D ipmmu_imuasid_read(mmu, utlb); + backup_data->utlbs_val[i] =3D ipmmu_imuctr_read(mmu, utlb); + } + } + + spin_unlock(&ipmmu_devices_backup_lock); +} + +static void ipmmu_utlbs_restore(struct ipmmu_vmsa_device *mmu) +{ + struct ipmmu_vmsa_backup *backup_data; + + dev_dbg(mmu->dev, "Handle micro-TLBs restore\n"); + + spin_lock(&ipmmu_devices_backup_lock); + + list_for_each_entry( backup_data, &ipmmu_devices_backup, list ) + { + struct iommu_fwspec *fwspec =3D dev_iommu_fwspec_get(backup_data->= dev); + unsigned int i; + + if ( to_ipmmu(backup_data->dev) !=3D mmu ) + continue; + + for ( i =3D 0; i < fwspec->num_ids; i++ ) + { + unsigned int utlb =3D fwspec->ids[i]; + + ipmmu_imuasid_write(mmu, utlb, backup_data->asids_val[i]); + ipmmu_imuctr_write(mmu, utlb, backup_data->utlbs_val[i]); + } + } + + spin_unlock(&ipmmu_devices_backup_lock); +} + +static void ipmmu_domain_backup_context(struct ipmmu_vmsa_domain *domain) +{ + struct ipmmu_vmsa_device *mmu =3D domain->mmu->root; + struct ipmmu_reg_ctx *regs =3D mmu->reg_backup[domain->context_id]; + + dev_dbg(mmu->dev, "Handle domain context %u backup\n", domain->context= _id); + + regs->imttlbr0 =3D ipmmu_ctx_read_root(domain, IMTTLBR0); + regs->imttubr0 =3D ipmmu_ctx_read_root(domain, IMTTUBR0); + regs->imttbcr =3D ipmmu_ctx_read_root(domain, IMTTBCR); + regs->imctr =3D ipmmu_ctx_read_root(domain, IMCTR); +} + +static void ipmmu_domain_restore_context(struct ipmmu_vmsa_domain *domain) +{ + struct ipmmu_vmsa_device *mmu =3D domain->mmu->root; + struct ipmmu_reg_ctx *regs =3D mmu->reg_backup[domain->context_id]; + + dev_dbg(mmu->dev, "Handle domain context %u restore\n", domain->contex= t_id); + + ipmmu_ctx_write_root(domain, IMTTLBR0, regs->imttlbr0); + ipmmu_ctx_write_root(domain, IMTTUBR0, regs->imttubr0); + ipmmu_ctx_write_root(domain, IMTTBCR, regs->imttbcr); + ipmmu_ctx_write_all(domain, IMCTR, regs->imctr | IMCTR_FLUSH); +} + +/* + * Xen: Unlike Linux implementation, Xen uses a single driver instance + * for handling all IPMMUs. There is no framework for ipmmu_suspend/resume + * callbacks to be invoked for each IPMMU device. So, we need to iterate + * through all registered IPMMUs performing required actions. + * + * Also take care of restoring special settings, such as translation + * table format, etc. + */ +static int __must_check ipmmu_suspend(void) +{ + struct ipmmu_vmsa_device *mmu; + + if ( !iommu_enabled ) + return 0; + + printk(XENLOG_DEBUG "ipmmu: Suspending...\n"); + + spin_lock(&ipmmu_devices_lock); + + list_for_each_entry( mmu, &ipmmu_devices, list ) + { + if ( ipmmu_is_root(mmu) ) + { + unsigned int i; + + for ( i =3D 0; i < mmu->num_ctx; i++ ) + { + if ( !mmu->domains[i] ) + continue; + ipmmu_domain_backup_context(mmu->domains[i]); + } + } + else + ipmmu_utlbs_backup(mmu); + } + + spin_unlock(&ipmmu_devices_lock); + + return 0; +} + +static void ipmmu_resume(void) +{ + struct ipmmu_vmsa_device *mmu; + + if ( !iommu_enabled ) + return; + + printk(XENLOG_DEBUG "ipmmu: Resuming...\n"); + + spin_lock(&ipmmu_devices_lock); + + list_for_each_entry( mmu, &ipmmu_devices, list ) + { + uint32_t reg; + + /* Do not use security group function */ + reg =3D IMSCTLR + mmu->features->control_offset_base; + ipmmu_write(mmu, reg, ipmmu_read(mmu, reg) & ~IMSCTLR_USE_SECGRP); + + if ( ipmmu_is_root(mmu) ) + { + unsigned int i; + + /* Use stage 2 translation table format */ + reg =3D IMSAUXCTLR + mmu->features->control_offset_base; + ipmmu_write(mmu, reg, ipmmu_read(mmu, reg) | IMSAUXCTLR_S2PTE); + + for ( i =3D 0; i < mmu->num_ctx; i++ ) + { + if ( !mmu->domains[i] ) + continue; + ipmmu_domain_restore_context(mmu->domains[i]); + } + } + else + ipmmu_utlbs_restore(mmu); + } + + spin_unlock(&ipmmu_devices_lock); +} + +static int ipmmu_alloc_ctx_suspend(struct device *dev) +{ + struct ipmmu_vmsa_backup *backup_data; + unsigned int *utlbs_val, *asids_val; + struct iommu_fwspec *fwspec =3D dev_iommu_fwspec_get(dev); + + utlbs_val =3D xzalloc_array(unsigned int, fwspec->num_ids); + if ( !utlbs_val ) + return -ENOMEM; + + asids_val =3D xzalloc_array(unsigned int, fwspec->num_ids); + if ( !asids_val ) + { + xfree(utlbs_val); + return -ENOMEM; + } + + backup_data =3D xzalloc(struct ipmmu_vmsa_backup); + if ( !backup_data ) + { + xfree(utlbs_val); + xfree(asids_val); + return -ENOMEM; + } + + backup_data->dev =3D dev; + backup_data->utlbs_val =3D utlbs_val; + backup_data->asids_val =3D asids_val; + + spin_lock(&ipmmu_devices_backup_lock); + list_add(&backup_data->list, &ipmmu_devices_backup); + spin_unlock(&ipmmu_devices_backup_lock); + + return 0; +} + +#ifdef CONFIG_HAS_PCI +static void ipmmu_free_ctx_suspend(struct device *dev) +{ + struct ipmmu_vmsa_backup *backup_data, *tmp; + + spin_lock(&ipmmu_devices_backup_lock); + + list_for_each_entry_safe( backup_data, tmp, &ipmmu_devices_backup, lis= t ) + { + if ( backup_data->dev =3D=3D dev ) + { + list_del(&backup_data->list); + xfree(backup_data->utlbs_val); + xfree(backup_data->asids_val); + xfree(backup_data); + break; + } + } + + spin_unlock(&ipmmu_devices_backup_lock); +} +#endif /* CONFIG_HAS_PCI */ + +#endif /* CONFIG_SYSTEM_SUSPEND */ + static int ipmmu_domain_init_context(struct ipmmu_vmsa_domain *domain) { uint64_t ttbr; @@ -559,6 +821,9 @@ static int ipmmu_domain_init_context(struct ipmmu_vmsa_= domain *domain) return ret; =20 domain->context_id =3D ret; +#ifdef CONFIG_SYSTEM_SUSPEND + domain->mmu->root->reg_backup[ret] =3D &root_pgtable[ret]; +#endif =20 /* * TTBR0 @@ -615,6 +880,9 @@ static void ipmmu_domain_destroy_context(struct ipmmu_v= msa_domain *domain) ipmmu_ctx_write_root(domain, IMCTR, IMCTR_FLUSH); ipmmu_tlb_sync(domain); =20 +#ifdef CONFIG_SYSTEM_SUSPEND + domain->mmu->root->reg_backup[domain->context_id] =3D NULL; +#endif ipmmu_domain_free_context(domain->mmu->root, domain->context_id); } =20 @@ -1338,10 +1606,11 @@ static int ipmmu_add_device(u8 devfn, struct device= *dev) struct iommu_fwspec *fwspec; =20 #ifdef CONFIG_HAS_PCI + int ret; + if ( dev_is_pci(dev) ) { struct pci_dev *pdev =3D dev_to_pci(dev); - int ret; =20 if ( devfn !=3D pdev->devfn ) return 0; @@ -1358,17 +1627,24 @@ static int ipmmu_add_device(u8 devfn, struct device= *dev) if ( !to_ipmmu(dev) ) return -ENODEV; =20 - if ( !dev_is_pci(dev) ) + if ( !dev_is_pci(dev) && dt_device_is_protected(dev_to_dt(dev)) ) { - if ( dt_device_is_protected(dev_to_dt(dev)) ) - { - dev_err(dev, "Already added to IPMMU\n"); - return -EEXIST; - } + dev_err(dev, "Already added to IPMMU\n"); + return -EEXIST; + } =20 - /* Let Xen know that the master device is protected by an IOMMU. */ - dt_device_set_protected(dev_to_dt(dev)); +#ifdef CONFIG_SYSTEM_SUSPEND + if ( ipmmu_alloc_ctx_suspend(dev) ) + { + dev_err(dev, "Failed to allocate context for suspend\n"); + return -ENOMEM; } +#endif + + /* Let Xen know that the master device is protected by an IOMMU. */ + if ( !dev_is_pci(dev) ) + dt_device_set_protected(dev_to_dt(dev)); + #ifdef CONFIG_HAS_PCI if ( dev_is_pci(dev) ) { @@ -1377,26 +1653,28 @@ static int ipmmu_add_device(u8 devfn, struct device= *dev) struct pci_host_bridge *bridge; struct iommu_fwspec *fwspec_bridge; unsigned int utlb_osid0 =3D 0; - int ret; =20 bridge =3D pci_find_host_bridge(pdev->seg, pdev->bus); if ( !bridge ) { dev_err(dev, "Failed to find host bridge\n"); - return -ENODEV; + ret =3D -ENODEV; + goto free_suspend_ctx; } =20 fwspec_bridge =3D dev_iommu_fwspec_get(dt_to_dev(bridge->dt_node)); if ( fwspec_bridge->num_ids < 1 ) { dev_err(dev, "Failed to find host bridge uTLB\n"); - return -ENXIO; + ret =3D -ENXIO; + goto free_suspend_ctx; } =20 if ( fwspec->num_ids < 1 ) { dev_err(dev, "Failed to find uTLB"); - return -ENXIO; + ret =3D -ENXIO; + goto free_suspend_ctx; } =20 rcar4_pcie_osid_regs_init(bridge); @@ -1405,7 +1683,7 @@ static int ipmmu_add_device(u8 devfn, struct device *= dev) if ( ret < 0 ) { dev_err(dev, "No unused OSID regs\n"); - return ret; + goto free_suspend_ctx; } reg_id =3D ret; =20 @@ -1420,7 +1698,7 @@ static int ipmmu_add_device(u8 devfn, struct device *= dev) { rcar4_pcie_osid_bdf_clear(bridge, reg_id); rcar4_pcie_osid_reg_free(bridge, reg_id); - return ret; + goto free_suspend_ctx; } } #endif @@ -1429,6 +1707,13 @@ static int ipmmu_add_device(u8 devfn, struct device = *dev) dev_name(fwspec->iommu_dev), fwspec->num_ids); =20 return 0; +#ifdef CONFIG_HAS_PCI + free_suspend_ctx: +#ifdef CONFIG_SYSTEM_SUSPEND + ipmmu_free_ctx_suspend(dev); +#endif + return ret; +#endif } =20 static int ipmmu_iommu_domain_init(struct domain *d) @@ -1490,6 +1775,10 @@ static const struct iommu_ops ipmmu_iommu_ops =3D .unmap_page =3D arm_iommu_unmap_page, .dt_xlate =3D ipmmu_dt_xlate, .add_device =3D ipmmu_add_device, +#ifdef CONFIG_SYSTEM_SUSPEND + .suspend =3D ipmmu_suspend, + .resume =3D ipmmu_resume, +#endif }; =20 static __init int ipmmu_init(struct dt_device_node *node, const void *data) --=20 2.43.0 From nobody Sat May 30 11:15:55 2026 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass(p=none dis=none) header.from=gmail.com ARC-Seal: i=1; a=rsa-sha256; t=1778605821; cv=none; d=zohomail.com; s=zohoarc; b=AfjYEt4vCx0hSfcLXFEeIiDeib7PI/z92WVjI0naUGvk0b3JbYf9a6qTgPHbnSJE8d7ZHSnyjcq5uxaJPJJPju411basjvVv2DbGBONkWOyTf1WSIs/su3Y+1fOOmA+vdqVpp5fsFX3c9fQVfHCH2/XQz5c6qocCuH4J6BZphYk= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1778605821; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=mHxx4CKtYsSgQQ7LLyGnfT6MKbRYsbWPPOV7hCJys74=; b=CR9d/lf83zQGIgajZ3uzkSYyfnhRXxEKjabpdlPyZ3HKK5A4MKn8C9xvxhNFahOhfu67RAsbEgMQCCgkdrLZ3oCREvNZCv6KunZxefECzOEjtMp9cN6ApQ18QMVOzY5hpaG1485uiEW2hZCrUHWhpysY1s+3477pCnlBjUaUlTs= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1778605821306633.7984873947411; Tue, 12 May 2026 10:10:21 -0700 (PDT) Received: from list by lists.xenproject.org with outflank-mailman.1307285.1579062 (Exim 4.92) (envelope-from ) id 1wMqcH-0007rp-HP; Tue, 12 May 2026 17:09:57 +0000 Received: by outflank-mailman (output) from mailman id 1307285.1579062; Tue, 12 May 2026 17:09:57 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1wMqcH-0007rR-8A; Tue, 12 May 2026 17:09:57 +0000 Received: by outflank-mailman (input) for mailman id 1307285; Tue, 12 May 2026 17:09:55 +0000 Received: from mx.expurgate.net ([195.190.135.10]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1wMqcF-0007Om-10 for xen-devel@lists.xenproject.org; Tue, 12 May 2026 17:09:55 +0000 Received: from mx.expurgate.net (helo=localhost) by mx.expurgate.net with esmtp id 1wMqcE-007RzZ-D4 for xen-devel@lists.xenproject.org; Tue, 12 May 2026 19:09:54 +0200 Received: from [10.42.69.5] (helo=localhost) by localhost with ESMTP (eXpurgate MTA 0.9.1) (envelope-from ) id 6a035eb4-2eae-0a2a0a5409dd-0a2a45059d4c-28 for ; Tue, 12 May 2026 19:09:54 +0200 Received: from [209.85.128.48] (helo=mail-wm1-f48.google.com) by tlsNG-c201ff.mxtls.expurgate.net with ESMTPS (eXpurgate 4.56.1) (envelope-from ) id 6a035ee2-aaa8-0a2a45050019-d1558030b83b-3 for ; Tue, 12 May 2026 19:09:54 +0200 Received: by mail-wm1-f48.google.com with SMTP id 5b1f17b1804b1-488a88aeec9so64771745e9.2 for ; Tue, 12 May 2026 10:09:54 -0700 (PDT) Received: from EPUAKYIW02F7.. ([45.12.26.18]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-48fc8cccf90sm10838845e9.0.2026.05.12.10.09.51 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 12 May 2026 10:09:52 -0700 (PDT) X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" Authentication-Results: eu.smtp.expurgate.cloud; dkim=pass header.s=20251104 header.d=gmail.com header.i="@gmail.com" header.h="Content-Transfer-Encoding:MIME-Version:References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1778605793; x=1779210593; darn=lists.xenproject.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=mHxx4CKtYsSgQQ7LLyGnfT6MKbRYsbWPPOV7hCJys74=; b=Uv3lsVoKxgfWPyp0kXZNKz1M3ujv/+qaJDZWW2yowkB9UbYEkQwRZntmgDC77CxcwR 4RFP4FfNnQuzCETgvG1lUj1KqUGS5uYds3EuLAfE4fRL6C5uQCaYLiQ6vqEgOzrUW/1i v9T5dbrZ5NSp6Pto8T7iW+0bswxZWFqcdw64dsLiawg9GPgabsaZuXqd3+yuzKk894Up YlDeqV5HyC/0tbH06kPCS2KoJDfk9/sndr91rdzyISaIh7wzYHre5e6Np46W1aMxcN3+ lm2bWO6uO7mg3PaMEMojn5kqz6ESB8fdj/IaBATT/JNhNR4bdlp4YB6+naUEqU6GJvs1 yMkA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1778605793; x=1779210593; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=mHxx4CKtYsSgQQ7LLyGnfT6MKbRYsbWPPOV7hCJys74=; b=kQTGLUVNmiPZ6yu9f8tM9G2sd6bU79K5gZD4zNTXbYHPtiin9cCxmAI/9yF/SHMtYa LLVMBQCMlSc9Lo4hnCGbsubXeZFOY+W0csB9teo67BxEVJLh5f1RkqaZOwXzpuUK3Jb6 GezoBDq/odmyNYMOS3meBA9VfvrVyCKvHsCTstCjLC3p6Ui5JGasPdPmW1WR+gaZ0mnN xEGIvysNijtomEEwZhq/ZB2PR3Fv6bhPajsSdnXEZ3KQ8CDLr+GcXDSFEquMjKtXXSYK LodJ5m5yXxLp8ScM9NSZXHXFT0alHBC4OTsDmz85tzd+UhQcSuDel5NhfPo8axxvDV/N 092A== X-Gm-Message-State: AOJu0Yy2uoiPDFlYZ+M37GM+YntUj/SGjyB++kyfk4Syk67ptBQTAJ3n Qpy625WYZHIX4qwELMoJH67byoAoG34rmNcZV6KSDjhcqiJ00EDAt88UdIqSOg== X-Gm-Gg: Acq92OFB5DgZtFQJh0yFi+OIOprpa5Dljjdgt0jRd/ospwpMWvJs2nsZxmwnRmrI3Eu FW95G1dIC6lw44wAPYZ5fGyZ/BfMX4jw9GwU1sw7ZZ7OyWW5wHdm2GaFCUKSplp6c4S98RNH80e srKIA1U6Rvg52D+vCecZtORulLti2Vjo3QZ0NM9fsbRumYDqpETw167NP3ie/s8t053pp3n143Z 9P53eUgW+jCWFEkJSIgZ8uUQYYM2cw1xuil/osf2fqE8FxjSMtas2gG7Mshj2+0aCgsQw/SFRS6 EZNfq2YXN4FdPeG/eMCFkRM2Nu07KzgTNZcfvdnTsZ+HIRrszC0Nn6L2Rjwi9/60ZHTzg+Zcuhw GQv9ROpBQA0zFAOAh8q1OL5DSJ9/zYtCkkSVtR0K7Cr7bWOxWgWmDo+wcg84QAJiclkCWZkpZEk 4eYmisZ/Tq9rH/yze/PCWcB8hMIw== X-Received: by 2002:a05:600c:5306:b0:48e:526e:101a with SMTP id 5b1f17b1804b1-48e706c781emr247211505e9.12.1778605793481; Tue, 12 May 2026 10:09:53 -0700 (PDT) From: Mykola Kvach To: xen-devel@lists.xenproject.org Cc: Mykola Kvach , Bertrand Marquis , Rahul Singh , Stefano Stabellini , Julien Grall , Michal Orzel , Volodymyr Babchuk , Pranjal Shrivastava Subject: [PATCH v9 09/13] xen/arm: smmu-v3: add suspend/resume handlers Date: Tue, 12 May 2026 20:07:16 +0300 Message-ID: X-Mailer: git-send-email 2.43.0 In-Reply-To: References: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-purgate-ID: tlsNG-c201ff/1778605794-E1B9E443-16BCACED/0/0 X-purgate-type: clean X-purgate-size: 10168 X-ZohoMail-DKIM: pass (identity @gmail.com) X-ZM-MESSAGEID: 1778605821606158500 Content-Type: text/plain; charset="utf-8" Add system suspend/resume callbacks for the Arm SMMUv3 driver. During suspend, configure GBPA to abort incoming transactions, disable the translation interface while keeping CMDQ enabled, issue CMD_SYNC to ensure all previously issued commands have completed, then disable the SMMU. Resume uses arm_smmu_device_reset() to reprogram the SMMU and re-enable translation and interrupt generation. The IRQ setup split follows the approach from Pranjal Shrivastava's Linux arm-smmu-v3 runtime/system sleep series: IRQ handlers are requested once during probe, while reset/resume only restores SMMU hardware state and re-enables IRQ_CTRL. Only the pieces relevant to Xen's currently supported SMMUv3 path are ported here. Xen documents SMMUv3 MSI and PCI ATS as unsupported and not compiled/tested, so this patch does not restore SMMU MSI IRQ_CFGn registers nor reinitialize ATS/PRI endpoints. If those paths become usable, suspend/resume will need corresponding MSI restore and ATS/PRI quiesce/reinit steps. Link: https://lore.kernel.org/r/20260414194702.1229094-1-praan@google.com/ Based-on-patch-by: Pranjal Shrivastava Signed-off-by: Mykola Kvach --- Changes in V9: - Use CMD_SYNC in suspend instead of polling CMDQ_CONS, so the suspend path waits for command completion rather than only command consumption. - Document that arm_smmu_setup_irqs() is probe-only and that future Xen SMMUv3 MSI support will need to restore SMMU IRQ_CFGn registers on resume. - Restore the reference to Pranjal's Linux runtime/system sleep series and clarify that MSI/ATS/PRI resume handling is outside the supported Xen path. - Prefix the subject with xen/arm for consistency with the rest of the Arm suspend/resume series. Changes in V8: - Honor ARM_SMMU_FEAT_SEV when draining the CMDQ during suspend, matching the existing runtime CMD_SYNC path. - Fold the suspend rollback reset path into a helper and rename the error reporting to describe suspend rollback rather than resume. - Treat SMMU reset failure during resume as fatal instead of logging and continuing with a potentially unusable IOMMU. - cosmetic changes --- xen/drivers/passthrough/arm/smmu-v3.c | 178 ++++++++++++++++++++------ 1 file changed, 142 insertions(+), 36 deletions(-) diff --git a/xen/drivers/passthrough/arm/smmu-v3.c b/xen/drivers/passthroug= h/arm/smmu-v3.c index bf153227db..82c8ead979 100644 --- a/xen/drivers/passthrough/arm/smmu-v3.c +++ b/xen/drivers/passthrough/arm/smmu-v3.c @@ -1814,8 +1814,7 @@ static int arm_smmu_write_reg_sync(struct arm_smmu_de= vice *smmu, u32 val, } =20 /* GBPA is "special" */ -static int __init arm_smmu_update_gbpa(struct arm_smmu_device *smmu, - u32 set, u32 clr) +static int arm_smmu_update_gbpa(struct arm_smmu_device *smmu, u32 set, u32= clr) { int ret; u32 reg, __iomem *gbpa =3D smmu->base + ARM_SMMU_GBPA; @@ -1995,10 +1994,35 @@ err_free_evtq_irq: return ret; } =20 +static int arm_smmu_enable_irqs(struct arm_smmu_device *smmu) +{ + int ret; + u32 irqen_flags =3D IRQ_CTRL_EVTQ_IRQEN | IRQ_CTRL_GERROR_IRQEN; + + if ( smmu->features & ARM_SMMU_FEAT_PRI ) + irqen_flags |=3D IRQ_CTRL_PRIQ_IRQEN; + + /* Enable interrupt generation on the SMMU */ + ret =3D arm_smmu_write_reg_sync(smmu, irqen_flags, + ARM_SMMU_IRQ_CTRL, ARM_SMMU_IRQ_CTRLACK); + if ( ret ) + { + dev_warn(smmu->dev, "failed to enable irqs\n"); + return ret; + } + + return 0; +} + +/* + * Probe-time only: request host IRQs and, when available, program the SMM= U's + * MSI doorbells. Resume does not restore the SMMU *_IRQ_CFGn MSI register= s, + * so any host suspend support must treat the active MSI IRQ path as + * unsupported until that restore path exists. + */ static int __init arm_smmu_setup_irqs(struct arm_smmu_device *smmu) { int ret, irq; - u32 irqen_flags =3D IRQ_CTRL_EVTQ_IRQEN | IRQ_CTRL_GERROR_IRQEN; =20 /* Disable IRQs first */ ret =3D arm_smmu_write_reg_sync(smmu, 0, ARM_SMMU_IRQ_CTRL, @@ -2028,22 +2052,7 @@ static int __init arm_smmu_setup_irqs(struct arm_smm= u_device *smmu) } } =20 - if (smmu->features & ARM_SMMU_FEAT_PRI) - irqen_flags |=3D IRQ_CTRL_PRIQ_IRQEN; - - /* Enable interrupt generation on the SMMU */ - ret =3D arm_smmu_write_reg_sync(smmu, irqen_flags, - ARM_SMMU_IRQ_CTRL, ARM_SMMU_IRQ_CTRLACK); - if (ret) { - dev_warn(smmu->dev, "failed to enable irqs\n"); - goto err_free_irqs; - } - return 0; - -err_free_irqs: - arm_smmu_free_irqs(smmu); - return ret; } =20 static int arm_smmu_device_disable(struct arm_smmu_device *smmu) @@ -2057,7 +2066,7 @@ static int arm_smmu_device_disable(struct arm_smmu_de= vice *smmu) return ret; } =20 -static int __init arm_smmu_device_reset(struct arm_smmu_device *smmu) +static int arm_smmu_device_reset(struct arm_smmu_device *smmu) { int ret; u32 reg, enables; @@ -2163,17 +2172,9 @@ static int __init arm_smmu_device_reset(struct arm_s= mmu_device *smmu) } } =20 - ret =3D arm_smmu_setup_irqs(smmu); - if (ret) { - dev_err(smmu->dev, "failed to setup irqs\n"); + ret =3D arm_smmu_enable_irqs(smmu); + if ( ret ) return ret; - } - - /* Initialize tasklets for threaded IRQs*/ - tasklet_init(&smmu->evtq_irq_tasklet, arm_smmu_evtq_tasklet, smmu); - tasklet_init(&smmu->priq_irq_tasklet, arm_smmu_priq_tasklet, smmu); - tasklet_init(&smmu->combined_irq_tasklet, arm_smmu_combined_irq_tasklet, - smmu); =20 /* Enable the SMMU interface, or ensure bypass */ if (disable_bypass) { @@ -2181,20 +2182,16 @@ static int __init arm_smmu_device_reset(struct arm_= smmu_device *smmu) } else { ret =3D arm_smmu_update_gbpa(smmu, 0, GBPA_ABORT); if (ret) - goto err_free_irqs; + return ret; } ret =3D arm_smmu_write_reg_sync(smmu, enables, ARM_SMMU_CR0, ARM_SMMU_CR0ACK); if (ret) { dev_err(smmu->dev, "failed to enable SMMU interface\n"); - goto err_free_irqs; + return ret; } =20 return 0; - -err_free_irqs: - arm_smmu_free_irqs(smmu); - return ret; } =20 static int arm_smmu_device_hw_probe(struct arm_smmu_device *smmu) @@ -2558,10 +2555,23 @@ static int __init arm_smmu_device_probe(struct plat= form_device *pdev) if (ret) goto out_free; =20 + ret =3D arm_smmu_setup_irqs(smmu); + if ( ret ) + { + dev_err(smmu->dev, "failed to setup irqs\n"); + goto out_free; + } + + /* Initialize tasklets for threaded IRQs*/ + tasklet_init(&smmu->evtq_irq_tasklet, arm_smmu_evtq_tasklet, smmu); + tasklet_init(&smmu->priq_irq_tasklet, arm_smmu_priq_tasklet, smmu); + tasklet_init(&smmu->combined_irq_tasklet, arm_smmu_combined_irq_tasklet, + smmu); + /* Reset the device */ ret =3D arm_smmu_device_reset(smmu); if (ret) - goto out_free; + goto out_free_irqs; =20 /* * Keep a list of all probed devices. This will be used to query @@ -2575,6 +2585,8 @@ static int __init arm_smmu_device_probe(struct platfo= rm_device *pdev) =20 return 0; =20 +out_free_irqs: + arm_smmu_free_irqs(smmu); =20 out_free: arm_smmu_free_structures(smmu); @@ -2855,6 +2867,96 @@ static void arm_smmu_iommu_xen_domain_teardown(struc= t domain *d) xfree(xen_domain); } =20 +#ifdef CONFIG_SYSTEM_SUSPEND + +static void arm_smmu_reset_for_suspend_rollback(struct arm_smmu_device *sm= mu) +{ + int ret =3D arm_smmu_device_reset(smmu); + + if ( ret ) + dev_err(smmu->dev, "Failed to reset during suspend rollback: %d\n", + ret); +} + +static int arm_smmu_suspend(void) +{ + struct arm_smmu_device *smmu; + int ret =3D 0; + + list_for_each_entry(smmu, &arm_smmu_devices, devices) + { + /* Abort all transactions before disable to avoid spurious bypass */ + ret =3D arm_smmu_update_gbpa(smmu, GBPA_ABORT, 0); + if ( ret ) + goto fail; + + /* Disable the SMMU via CR0.EN and all queues except CMDQ */ + ret =3D arm_smmu_write_reg_sync(smmu, CR0_CMDQEN, ARM_SMMU_CR0, + ARM_SMMU_CR0ACK); + if ( ret ) + { + dev_err(smmu->dev, "Timed-out while disabling smmu\n"); + goto fail; + } + + /* + * At this point the translation interface is disabled and the + * SMMU won't access translation/config structures, even + * speculatively, as per the IHI0070 spec (section 6.3.9.6). + * CMDQ is still enabled so that a CMD_SYNC can complete any + * previously issued commands. + */ + + /* Ensure all previously issued commands have completed. */ + ret =3D arm_smmu_cmdq_issue_sync(smmu); + if ( ret ) + { + dev_err(smmu->dev, "Timed-out waiting for pending commands\n"); + goto fail; + } + + /* Disable everything */ + ret =3D arm_smmu_device_disable(smmu); + if ( ret ) + goto fail; + + dev_dbg(smmu->dev, "Suspended smmu\n"); + } + + return 0; + + fail: + /* Reset the device that failed as well as any already-suspended ones. */ + arm_smmu_reset_for_suspend_rollback(smmu); + + list_for_each_entry_continue_reverse(smmu, &arm_smmu_devices, devices) + arm_smmu_reset_for_suspend_rollback(smmu); + + return ret; +} + +static void arm_smmu_resume(void) +{ + int ret; + struct arm_smmu_device *smmu; + + list_for_each_entry(smmu, &arm_smmu_devices, devices) + { + dev_dbg(smmu->dev, "Resuming device\n"); + + /* + * The reset will re-initialize all the base addresses, queues, + * prod and cons maintained within struct arm_smmu_device as well as + * re-enable the interrupts. + */ + ret =3D arm_smmu_device_reset(smmu); + if ( ret ) + panic("SMMUv3: %s: Failed to reset during resume: %d\n", + dev_name(smmu->dev), ret); + } +} +#endif + static const struct iommu_ops arm_smmu_iommu_ops =3D { .page_sizes =3D PAGE_SIZE_4K, .init =3D arm_smmu_iommu_xen_domain_init, @@ -2867,6 +2969,10 @@ static const struct iommu_ops arm_smmu_iommu_ops =3D= { .unmap_page =3D arm_iommu_unmap_page, .dt_xlate =3D arm_smmu_dt_xlate, .add_device =3D arm_smmu_add_device, +#ifdef CONFIG_SYSTEM_SUSPEND + .suspend =3D arm_smmu_suspend, + .resume =3D arm_smmu_resume, +#endif }; =20 static __init int arm_smmu_dt_init(struct dt_device_node *dev, --=20 2.43.0 From nobody Sat May 30 11:15:55 2026 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass(p=none dis=none) header.from=gmail.com ARC-Seal: i=1; a=rsa-sha256; t=1778605819; cv=none; d=zohomail.com; s=zohoarc; b=GVIhcy0NuZq8lbfUrGA12AKZtQYNQHauavqgUUzswVj6XxL9JEvqHawFjwX+H2ky96ZO0a/Z7BW0CNfHvAVmT4WFIfNsTcw6EuB9J/Nod5HQosEB2u5MchxxJ95UXlhGzlvTuuHNriWwdUrTzma+Ff7BTa49ILquhsuqhwj6nuw= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1778605819; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=lkhNyEzobDpYCgF/TpDVG4OQjxYfp1uqUAAZRreqhtU=; b=HJbYFnoKK3WYo8/g9oimiY4/cZrcMWb4//YnOC7nWORYZfLLC2Pi5NiYAFt7qZIVFbojq9AKhEcuGJsGGoRlH78QkTawRXX7L2Oa39Hk4lz6nrYt62q/TBx9nGh8600SrOQAX9l+1t+wUOWGwQv0TBcXgVFF0E2q+vkYS/xwrrI= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1778605819370762.5014047682876; Tue, 12 May 2026 10:10:19 -0700 (PDT) Received: from list by lists.xenproject.org with outflank-mailman.1307286.1579066 (Exim 4.92) (envelope-from ) id 1wMqcI-0007xC-76; Tue, 12 May 2026 17:09:58 +0000 Received: by outflank-mailman (output) from mailman id 1307286.1579066; Tue, 12 May 2026 17:09:58 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1wMqcH-0007vj-Qu; Tue, 12 May 2026 17:09:57 +0000 Received: by outflank-mailman (input) for mailman id 1307286; Tue, 12 May 2026 17:09:56 +0000 Received: from mx.expurgate.net ([195.190.135.10]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1wMqcG-0007mm-Ix for xen-devel@lists.xenproject.org; Tue, 12 May 2026 17:09:56 +0000 Received: from mx.expurgate.net (helo=localhost) by mx.expurgate.net with esmtp id 1wMqcF-00AdLt-Vm for xen-devel@lists.xenproject.org; Tue, 12 May 2026 19:09:55 +0200 Received: from [10.42.69.6] (helo=localhost) by localhost with ESMTP (eXpurgate MTA 0.9.1) (envelope-from ) id 6a035ecd-bab6-0a2a0a5309dd-0a2a4506e5aa-36 for ; Tue, 12 May 2026 19:09:55 +0200 Received: from [209.85.128.47] (helo=mail-wm1-f47.google.com) by tlsNG-16d1c6.mxtls.expurgate.net with ESMTPS (eXpurgate 4.56.1) (envelope-from ) id 6a035ee3-7371-0a2a45060019-d155802fbd5a-3 for ; Tue, 12 May 2026 19:09:55 +0200 Received: by mail-wm1-f47.google.com with SMTP id 5b1f17b1804b1-488b0046078so48589675e9.1 for ; Tue, 12 May 2026 10:09:55 -0700 (PDT) Received: from EPUAKYIW02F7.. ([45.12.26.18]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-48fc8cccf90sm10838845e9.0.2026.05.12.10.09.53 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 12 May 2026 10:09:54 -0700 (PDT) X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" Authentication-Results: eu.smtp.expurgate.cloud; dkim=pass header.s=20251104 header.d=gmail.com header.i="@gmail.com" header.h="Content-Transfer-Encoding:MIME-Version:References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1778605795; x=1779210595; darn=lists.xenproject.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=lkhNyEzobDpYCgF/TpDVG4OQjxYfp1uqUAAZRreqhtU=; b=HIjHo2uRftMJxoWplko2MV09aZb+TgQQDwXc/9X8jOIBHNH+emuV09INzgmvNX0kWf gwVplwoTi0hvlTUKN+PwB7wJHMQqkESi5ftc48LyKXfqIgrDeaQsewfSXe+20cztL2WM gDJ91vUEYhbRGTIScsLG1AWpokGnkf/mQbAPzBqnlI+pS+Bo/dirE1XyPmUrLcJtKUyo 9G1lktAWbBU0CUdxKQKHNFIByR7qLVYdF1NtumGRL6M9ZsAqXFpV89nOUHX1fO6jOStq BUUOATjAc8I7TBYc/JJpT/43CPgz4uDj4ynsdRy1u08NIFQIzgaLd/xEWuha8MQJwU+j qbmw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1778605795; x=1779210595; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=lkhNyEzobDpYCgF/TpDVG4OQjxYfp1uqUAAZRreqhtU=; b=BPsQ6zxy/aLh+M8bOGOekgKfLUynOC/ToEgfF+0/y+1r23iQz+JHJgBbuuji2Xt7ff fz0gG4bhzgMS/IXdB6O6MbQSggOuBlrMjuDEPzeGHxpl3RFAMWmtK/H2nq6ONWccg5Mn 8KXQ8rbdgk4t2/C3T6Lg6gDRtCXEgpZMfAfyjqjoedfw2Dmkq77fXSu00vQMImU0A01r 6x0udCAKn4xzNrrxiRbTUJf8+fxpE5xiAzHzXSijtr08y4xFouSKyIy8SCxf426qc9/C 7uHiyYGxI+MvmuuzVehAXapAEozxOsc6QR6xcxxXsjzANjUNGNuklRKVGcZy8u50NfiY GnrA== X-Gm-Message-State: AOJu0YwnzLC1ypjX8naGysuUwsI4la7ULlOXMkjpF7XImAMGuOp1rZDY InL0mEcdMtgrcWcnTw4zUjmodabLWs9CCx6U1xwpt4eCZbBo0R+bxlowmWDBAQ== X-Gm-Gg: Acq92OGObMgOjuE+Wct8qwdrJBgPcddU6k+/Tu4wkCdEg56HGdtSK5bTsDmQtO3RJjl +pzQqqAl8xVpriARLEmxka02PNvMsLDRg/Lu9hIiPL0eLcnzEK1AREzJA3NvNKNaMb6KxL1ddkm uiSKbfJS3SHt2H/OuYWqO6Hnq2d8onA12cR7UE9KyftnKp0xC5lEAP37B2JJEghrkg6LmPY+LpF QA6MMNKQNJH/uW2lRomzX/iBDTxPa3XToiVcv1uiJlip5tnvB28M1URf3aV7xOFr0/Yvyh117Qh iZDYJnz8pGywViSWOq1f/sHQe6JtVSuCBfVApj8jK83QGG6LsHqS2kyTJfSH1U4Cr6jiMIUiL7u V970glNf4TaLHiD+e4dw1tw9M/N59j9Wm9P3IR6sCdJeHGBjELKjsVTSVnbGLowpMxhwY/pCpnS 3AHsLJ+s7E8twPiu0afXkhz1mVgw== X-Received: by 2002:a05:600c:5248:b0:48a:906a:9050 with SMTP id 5b1f17b1804b1-48e7069175fmr245499865e9.10.1778605794944; Tue, 12 May 2026 10:09:54 -0700 (PDT) From: Mykola Kvach To: xen-devel@lists.xenproject.org Cc: Mykola Kvach , Stefano Stabellini , Julien Grall , Bertrand Marquis , Michal Orzel , Volodymyr Babchuk Subject: [PATCH v9 10/13] xen/arm64: Save/restore CPU context across SYSTEM_SUSPEND Date: Tue, 12 May 2026 20:07:17 +0300 Message-ID: <46bbc9e008a65dd86fb2b368e4751eb7d2014afa.1778605274.git.mykola_kvach@epam.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: References: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-purgate-ID: tlsNG-16d1c6/1778605795-85361D75-333E9410/0/0 X-purgate-type: clean X-purgate-size: 11024 X-ZohoMail-DKIM: pass (identity @gmail.com) X-ZM-MESSAGEID: 1778605820194154100 Content-Type: text/plain; charset="utf-8" From: Mirela Simonovic On wakeup from PSCI SYSTEM_SUSPEND, Xen re-enters EL2 with the MMU and data cache disabled. The resume path must first switch back to Xen's runtime page tables before it can access the saved CPU context using virtual addresses. Add an arm64 hyp_resume trampoline that reuses enable_secondary_cpu_mm() to enable the data cache and MMU, switch to init_ttbr, and resume in the runtime virtual mapping. The trampoline then restores the saved CPU general-purpose and system-control register context. prepare_resume_ctx() must be invoked just before the PSCI system suspend call is issued to the platform firmware. It saves the current CPU context and returns a non-zero value so that the caller enters the physical SYSTEM_SUSPEND call. On resume, hyp_resume restores the saved context, including the saved link register. Control therefore returns to the place where prepare_resume_ctx() was called. To avoid re-entering the suspend path, the restored path sees prepare_resume_ctx() return zero. The assembly save/restore code uses offsets generated by asm-offsets.c from struct resume_cpu_context, keeping the assembly memory accesses in sync with the C structure layout. Support for ARM32 is not implemented. Instead, compilation fails with a build-time error if suspend is enabled for ARM32. Signed-off-by: Mirela Simonovic Signed-off-by: Saeed Nowshadi Signed-off-by: Mykyta Poturai Signed-off-by: Mykola Kvach Reviewed-by: Oleksandr Tyshchenko --- Changes in v9: - Drop the misleading prepare_resume_ctx() pointer argument and make both save/restore paths use the global resume_cpu_context. - Squash the arm64 resume trampoline into the context save/restore patch. - Document in code that hyp_resume relies on PSCI initial-state rules. - Use generic platform firmware wording instead of ATF-specific wording. - Rename the saved context type/storage to resume_cpu_context and rely on implicit zero-initialization for the file-scope object. - Use asm-offsets.c-generated RESUME_CTX_* offsets to keep the assembly save/restore code in sync with struct resume_cpu_context. Changes in v8: - Fix alignments in code. Changes in v7: - No functional changes, just moved commit. --- xen/arch/arm/Makefile | 1 + xen/arch/arm/arm64/asm-offsets.c | 20 +++++ xen/arch/arm/arm64/head.S | 118 +++++++++++++++++++++++++++++ xen/arch/arm/include/asm/suspend.h | 26 +++++++ xen/arch/arm/suspend.c | 14 ++++ 5 files changed, 179 insertions(+) create mode 100644 xen/arch/arm/suspend.c diff --git a/xen/arch/arm/Makefile b/xen/arch/arm/Makefile index 982c6c396a..c97df7f3a0 100644 --- a/xen/arch/arm/Makefile +++ b/xen/arch/arm/Makefile @@ -51,6 +51,7 @@ obj-y +=3D setup.o obj-y +=3D shutdown.o obj-y +=3D smp.o obj-y +=3D smpboot.o +obj-$(CONFIG_SYSTEM_SUSPEND) +=3D suspend.o obj-$(CONFIG_SYSCTL) +=3D sysctl.o obj-y +=3D time.o obj-y +=3D traps.o diff --git a/xen/arch/arm/arm64/asm-offsets.c b/xen/arch/arm/arm64/asm-offs= ets.c index 38a3894a3b..4da5fff584 100644 --- a/xen/arch/arm/arm64/asm-offsets.c +++ b/xen/arch/arm/arm64/asm-offsets.c @@ -13,6 +13,7 @@ #include #include #include +#include =20 #define DEFINE(_sym, _val) = \ asm volatile ( "\n.ascii\"=3D=3D>#define " #_sym " %0 /* " #_val " */<= =3D=3D\""\ @@ -57,6 +58,25 @@ void __dummy__(void) OFFSET(INITINFO_stack, struct init_info, stack); BLANK(); =20 +#ifdef CONFIG_SYSTEM_SUSPEND + OFFSET(RESUME_CTX_X19, struct resume_cpu_context, callee_regs[0]); + OFFSET(RESUME_CTX_X21, struct resume_cpu_context, callee_regs[2]); + OFFSET(RESUME_CTX_X23, struct resume_cpu_context, callee_regs[4]); + OFFSET(RESUME_CTX_X25, struct resume_cpu_context, callee_regs[6]); + OFFSET(RESUME_CTX_X27, struct resume_cpu_context, callee_regs[8]); + OFFSET(RESUME_CTX_X29, struct resume_cpu_context, callee_regs[10]); + OFFSET(RESUME_CTX_SP, struct resume_cpu_context, sp); + OFFSET(RESUME_CTX_VBAR_EL2, struct resume_cpu_context, vbar_el2); + OFFSET(RESUME_CTX_VTCR_EL2, struct resume_cpu_context, vtcr_el2); + OFFSET(RESUME_CTX_VTTBR_EL2, struct resume_cpu_context, vttbr_el2); + OFFSET(RESUME_CTX_TPIDR_EL2, struct resume_cpu_context, tpidr_el2); + OFFSET(RESUME_CTX_MDCR_EL2, struct resume_cpu_context, mdcr_el2); + OFFSET(RESUME_CTX_HSTR_EL2, struct resume_cpu_context, hstr_el2); + OFFSET(RESUME_CTX_CPTR_EL2, struct resume_cpu_context, cptr_el2); + OFFSET(RESUME_CTX_HCR_EL2, struct resume_cpu_context, hcr_el2); + BLANK(); +#endif + OFFSET(SMCCC_RES_a0, struct arm_smccc_res, a0); OFFSET(SMCCC_RES_a2, struct arm_smccc_res, a2); OFFSET(ARM_SMCCC_1_2_REGS_X0_OFFS, struct arm_smccc_1_2_regs, a0); diff --git a/xen/arch/arm/arm64/head.S b/xen/arch/arm/arm64/head.S index 72c7b24498..512a3c35b2 100644 --- a/xen/arch/arm/arm64/head.S +++ b/xen/arch/arm/arm64/head.S @@ -561,6 +561,124 @@ END(efi_xen_start) =20 #endif /* CONFIG_ARM_EFI */ =20 +#ifdef CONFIG_SYSTEM_SUSPEND +/* + * int prepare_resume_ctx(void) + * + * CPU context saved here will be restored on resume in hyp_resume functio= n. + * prepare_resume_ctx shall return a non-zero value. Upon restoring context + * hyp_resume shall return value zero instead. From C code that invokes + * prepare_resume_ctx, the return value is interpreted to determine whether + * the context is saved (prepare_resume_ctx) or restored (hyp_resume). + */ +FUNC(prepare_resume_ctx) + ldr x0, =3Dresume_cpu_context + + /* Store callee-saved registers */ + stp x19, x20, [x0, #RESUME_CTX_X19] + stp x21, x22, [x0, #RESUME_CTX_X21] + stp x23, x24, [x0, #RESUME_CTX_X23] + stp x25, x26, [x0, #RESUME_CTX_X25] + stp x27, x28, [x0, #RESUME_CTX_X27] + stp x29, lr, [x0, #RESUME_CTX_X29] + + /* Store stack-pointer */ + mov x2, sp + str x2, [x0, #RESUME_CTX_SP] + + /* Store system control registers */ + mrs x2, VBAR_EL2 + str x2, [x0, #RESUME_CTX_VBAR_EL2] + mrs x2, VTCR_EL2 + str x2, [x0, #RESUME_CTX_VTCR_EL2] + mrs x2, VTTBR_EL2 + str x2, [x0, #RESUME_CTX_VTTBR_EL2] + mrs x2, TPIDR_EL2 + str x2, [x0, #RESUME_CTX_TPIDR_EL2] + mrs x2, MDCR_EL2 + str x2, [x0, #RESUME_CTX_MDCR_EL2] + mrs x2, HSTR_EL2 + str x2, [x0, #RESUME_CTX_HSTR_EL2] + mrs x2, CPTR_EL2 + str x2, [x0, #RESUME_CTX_CPTR_EL2] + mrs x2, HCR_EL2 + str x2, [x0, #RESUME_CTX_HCR_EL2] + + /* prepare_resume_ctx must return a non-zero value */ + mov x0, #1 + ret +END(prepare_resume_ctx) + +FUNC(hyp_resume) + /* + * PSCI states that SYSTEM_SUSPEND follows the CPU_SUSPEND initial + * state rules, so PSCI-compliant firmware must enter the return + * exception level with DAIF masked. + */ + + /* Initialize the UART if earlyprintk has been enabled. */ +#ifdef CONFIG_EARLY_PRINTK + bl init_uart +#endif + PRINT_ID("- Xen resuming -\r\n") + + bl check_cpu_mode + bl cpu_init + + ldr x0, =3Dstart + adr x20, start /* x20 :=3D paddr (start) */ + sub x20, x20, x0 /* x20 :=3D phys-offset */ + ldr lr, =3Dmmu_resumed + b enable_secondary_cpu_mm + +mmu_resumed: + /* Now we can access the saved context, so restore it here. */ + ldr x0, =3Dresume_cpu_context + + /* Restore callee-saved registers */ + ldp x19, x20, [x0, #RESUME_CTX_X19] + ldp x21, x22, [x0, #RESUME_CTX_X21] + ldp x23, x24, [x0, #RESUME_CTX_X23] + ldp x25, x26, [x0, #RESUME_CTX_X25] + ldp x27, x28, [x0, #RESUME_CTX_X27] + ldp x29, lr, [x0, #RESUME_CTX_X29] + + /* Restore stack pointer */ + ldr x2, [x0, #RESUME_CTX_SP] + mov sp, x2 + + /* Restore system control registers */ + ldr x2, [x0, #RESUME_CTX_VBAR_EL2] + msr VBAR_EL2, x2 + ldr x2, [x0, #RESUME_CTX_VTCR_EL2] + msr VTCR_EL2, x2 + ldr x2, [x0, #RESUME_CTX_VTTBR_EL2] + msr VTTBR_EL2, x2 + ldr x2, [x0, #RESUME_CTX_TPIDR_EL2] + msr TPIDR_EL2, x2 + ldr x2, [x0, #RESUME_CTX_MDCR_EL2] + msr MDCR_EL2, x2 + ldr x2, [x0, #RESUME_CTX_HSTR_EL2] + msr HSTR_EL2, x2 + ldr x2, [x0, #RESUME_CTX_CPTR_EL2] + msr CPTR_EL2, x2 + ldr x2, [x0, #RESUME_CTX_HCR_EL2] + msr HCR_EL2, x2 + isb + + /* + * Since context is restored return from this function will appear + * as return from prepare_resume_ctx. To distinguish a return from + * prepare_resume_ctx which is called upon finalizing the suspend, + * as opposed to return from this function which executes on resum= e, + * we need to return zero value here. + */ + mov x0, #0 + ret +END(hyp_resume) + +#endif /* CONFIG_SYSTEM_SUSPEND */ + /* * Local variables: * mode: ASM diff --git a/xen/arch/arm/include/asm/suspend.h b/xen/arch/arm/include/asm/= suspend.h index 31a98a1f1b..2d9fc331fc 100644 --- a/xen/arch/arm/include/asm/suspend.h +++ b/xen/arch/arm/include/asm/suspend.h @@ -3,6 +3,8 @@ #ifndef ARM_SUSPEND_H #define ARM_SUSPEND_H =20 +#include + struct domain; struct vcpu; struct vcpu_guest_context; @@ -14,6 +16,30 @@ struct resume_info { =20 void arch_domain_resume(struct domain *d); =20 +#ifdef CONFIG_SYSTEM_SUSPEND +#ifdef CONFIG_ARM_64 +struct resume_cpu_context { + register_t callee_regs[12]; + register_t sp; + register_t vbar_el2; + register_t vtcr_el2; + register_t vttbr_el2; + register_t tpidr_el2; + register_t mdcr_el2; + register_t hstr_el2; + register_t cptr_el2; + register_t hcr_el2; +} __aligned(16); +#else +#error "Define resume_cpu_context structure for arm32" +#endif + +extern struct resume_cpu_context resume_cpu_context; + +int prepare_resume_ctx(void); +void hyp_resume(void); +#endif /* CONFIG_SYSTEM_SUSPEND */ + #endif /* ARM_SUSPEND_H */ =20 /* diff --git a/xen/arch/arm/suspend.c b/xen/arch/arm/suspend.c new file mode 100644 index 0000000000..6ea4a0f9cc --- /dev/null +++ b/xen/arch/arm/suspend.c @@ -0,0 +1,14 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ + +#include + +struct resume_cpu_context resume_cpu_context; + +/* + * Local variables: + * mode: C + * c-file-style: "BSD" + * c-basic-offset: 4 + * indent-tabs-mode: nil + * End: + */ --=20 2.43.0 From nobody Sat May 30 11:15:55 2026 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass(p=none dis=none) header.from=gmail.com ARC-Seal: i=1; a=rsa-sha256; t=1778605828; cv=none; d=zohomail.com; s=zohoarc; b=jE8/72Kpk2ufFbvF/6q98kB9px9jZYBmXazURYUyK6FS67KVMFaJfxo6pvkq9RL+yf/BfEwEMBrZ6FjX5Kxlfq2aK8QvKb/m5Aby6fiMglAEnJsxair/QqS2elSzQ04Y2Rv6IZdarGlWxiIdvWrCQ9eOtbyuSihiuWvKM0DAfBw= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1778605828; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=d9w5Gr7BSqTZeQ8Yv/eCImtGJXonwn8c1SkX4Mjo9nY=; b=M2dBjmfHtej014yDl5V6cfqMCzIuqnfkvEgVLaiB7xj3c3lMUqK01wSfoYXHPvSac3RofQw/YXvCCeUqpCia6nS4iixpZ0YDtt7c8pSuvWhYohyKg+Kt7DYXbV+zW2HUflkJRd3ax+tU4gj6b6b0Va+rtkxi2fqpXfHfC2F0fxA= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1778605828590979.8658181362063; Tue, 12 May 2026 10:10:28 -0700 (PDT) Received: from list by lists.xenproject.org with outflank-mailman.1307288.1579071 (Exim 4.92) (envelope-from ) id 1wMqcJ-00088e-1d; Tue, 12 May 2026 17:09:59 +0000 Received: by outflank-mailman (output) from mailman id 1307288.1579071; Tue, 12 May 2026 17:09:59 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1wMqcI-00085R-P0; Tue, 12 May 2026 17:09:58 +0000 Received: by outflank-mailman (input) for mailman id 1307288; Tue, 12 May 2026 17:09:57 +0000 Received: from mx.expurgate.net ([195.190.135.10]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1wMqcH-0007rj-GK for xen-devel@lists.xenproject.org; Tue, 12 May 2026 17:09:57 +0000 Received: from mx.expurgate.net (helo=localhost) by mx.expurgate.net with esmtp id 1wMqcG-00AZHt-TV for xen-devel@lists.xenproject.org; Tue, 12 May 2026 19:09:56 +0200 Received: from [10.42.69.7] (helo=localhost) by localhost with ESMTP (eXpurgate MTA 0.9.1) (envelope-from ) id 6a035eca-e002-0a2a0a5209dd-0a2a45078240-46 for ; Tue, 12 May 2026 19:09:56 +0200 Received: from [209.85.128.48] (helo=mail-wm1-f48.google.com) by tlsNG-ef75cf.mxtls.expurgate.net with ESMTPS (eXpurgate 4.56.1) (envelope-from ) id 6a035ee4-229c-0a2a45070019-d1558030e133-3 for ; Tue, 12 May 2026 19:09:56 +0200 Received: by mail-wm1-f48.google.com with SMTP id 5b1f17b1804b1-488a8ca4aadso53138125e9.3 for ; Tue, 12 May 2026 10:09:56 -0700 (PDT) Received: from EPUAKYIW02F7.. ([45.12.26.18]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-48fc8cccf90sm10838845e9.0.2026.05.12.10.09.55 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 12 May 2026 10:09:55 -0700 (PDT) X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" Authentication-Results: eu.smtp.expurgate.cloud; dkim=pass header.s=20251104 header.d=gmail.com header.i="@gmail.com" header.h="Content-Transfer-Encoding:MIME-Version:References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1778605796; x=1779210596; darn=lists.xenproject.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=d9w5Gr7BSqTZeQ8Yv/eCImtGJXonwn8c1SkX4Mjo9nY=; b=szPHLkmU0w+MO57eAG6IE6mDg0vJISqQMTT5hNI90Ct/2KHjOGUMQ6GPRIYml4NkHr RQm0evuGWfckv8u/uCFk2ENb7FUfyhOAh/XrDjAHBY31DI7o5ZL2JrAw2wJLKuSxPuGJ XT56pt2NYPIR+BIjx+/JPPLjfKQ2SGvBy/3UiUfsSn8CSs+McwAAYB+rGj2aEcwY+Isn 6OXvo/2aBwHY6TRTx/hGEbiTrkw95bxue79UOFZ0pYPo5lI1zUk/iPkmGPmUe2M1/mo5 1HO/T9HdaEVKU/yezcKFK8GqVLigLBWLR7WOkIpBWCTzX2iVSjVZn3+BSUxk7gHnHNRO AMDQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1778605796; x=1779210596; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=d9w5Gr7BSqTZeQ8Yv/eCImtGJXonwn8c1SkX4Mjo9nY=; b=DLRtppArV9t1MkYKhkOnw2yeGyLVrV366X/t9e+lpHZ4CYmzRj2y+lEpt+NwNaAE/z Xcr/BbQx8QEe3Y2mn81nwkEGrSdIceBimqgjf/FPVsr+YBgvAimjdjwXLhdP7e4XLfVF afPCo9qkW6eDowCnda4MZShea6n8+yyUSPiE84ln1ddZNYVDrr548T4A4WlP53xsBL5W MwjSPxJ/uH3M0kQNdivkOG7nFtPXU1XxfUakBBkOpnXyHJ4yx5PEk8G+tpy+kyRcMuPa YjP6l6fezQbrdvxGVs2KCwNGqgje8H+5r9CshVUue0o0O4HfvRELS9ojbmVKSmxr4q6k OqWg== X-Gm-Message-State: AOJu0YyGYhHP5N7D4S3U5WBDX1DDiD+7XjMOmGiAf3rUcxJIqE49AMzA hTBYLuFWY+ezv/yrz88hvQHBQ1hZ8S6QWwlIZvJyv6e7X+IwghsrVP0zsPlGlw== X-Gm-Gg: Acq92OEuj1mX4bYGJWPg2cpqXK9DSLcfh34G5gIoAV2atYRBoYkiRbtPUYbViC0ckHF J1t3ZaDgdLZJMEcVUvsziGSFIYb74kp91INljZarIjyuojyoNC5ccjtgZy1e3KD/X5z+2KFZI6S rz7ZHqsjzPrJf3v4OO5qQK04pDT1jhf1COU3tCTGXVckk0ADHLR5BhOzdA4QQclvVtuphCmF7oG xEzfUZqU33oAElLIy3BbHZzDMml/8h2W0VeNKPUGnCCfXiGFVM88sSR+U3QstXE5iFk2LCiaij/ K4A46UGHtV9e6+UgLAep8ZfjM/32RSG0ZDuxZaVbfueb+Gz3bcfvuCzQxokwB2HnO7di2YYbGpM SAHN1pzXwVnWJXa+aF4Jwu7nqa21lfrX0OVMkm/w8oT5DXcCt3GHfPJA8GBYL8jJWNaoxsEqn/P IQenMAzh4LFGI8lQpstyaogCE8JA== X-Received: by 2002:a05:600d:849c:20b0:488:a2ac:a34c with SMTP id 5b1f17b1804b1-48e51f220e7mr354003745e9.12.1778605796005; Tue, 12 May 2026 10:09:56 -0700 (PDT) From: Mykola Kvach To: xen-devel@lists.xenproject.org Cc: Mykola Kvach , Stefano Stabellini , Julien Grall , Bertrand Marquis , Michal Orzel , Volodymyr Babchuk Subject: [PATCH v9 11/13] xen/arm: Implement PSCI SYSTEM_SUSPEND call (host interface) Date: Tue, 12 May 2026 20:07:18 +0300 Message-ID: <9d76348fdd25f0ac7c449ed302ce24ac93b70e95.1778605274.git.mykola_kvach@epam.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: References: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-purgate-ID: tlsNG-ef75cf/1778605796-22D77C48-F1299423/0/0 X-purgate-type: clean X-purgate-size: 4029 X-ZohoMail-DKIM: pass (identity @gmail.com) X-ZM-MESSAGEID: 1778605829712158500 Content-Type: text/plain; charset="utf-8" From: Mirela Simonovic Invoke PSCI SYSTEM_SUSPEND to finalize Xen's suspend sequence on ARM64 platforms. Pass the Xen resume entry point (hyp_resume) to EL3 together with a zero context ID, matching Linux. This patch wires up only the host-side PSCI SYSTEM_SUSPEND invocation. The resume trampoline and context restore are provided by earlier patches in the series. Only enable this path when CONFIG_SYSTEM_SUSPEND is set and PSCI advertises SYSTEM_SUSPEND via PSCI_FEATURES. Signed-off-by: Mirela Simonovic Signed-off-by: Saeed Nowshadi Signed-off-by: Mykyta Poturai Signed-off-by: Mykola Kvach Reviewed-by: Luca Fancellu --- Changes in v9: - cache SYSTEM_SUSPEND support using PSCI_FEATURES and gate the host call on the cached capability - keep the cached SYSTEM_SUSPEND capability read-only after init - log whether firmware reports SYSTEM_SUSPEND support - pass an explicit zero context ID in the SYSTEM_SUSPEND call - drop the stale note claiming hyp_resume is still a stub --- xen/arch/arm/include/asm/psci.h | 1 + xen/arch/arm/psci.c | 31 ++++++++++++++++++++++++++++++- 2 files changed, 31 insertions(+), 1 deletion(-) diff --git a/xen/arch/arm/include/asm/psci.h b/xen/arch/arm/include/asm/psc= i.h index 48a93e6b79..bb3c73496e 100644 --- a/xen/arch/arm/include/asm/psci.h +++ b/xen/arch/arm/include/asm/psci.h @@ -23,6 +23,7 @@ int call_psci_cpu_on(int cpu); void call_psci_cpu_off(void); void call_psci_system_off(void); void call_psci_system_reset(void); +int call_psci_system_suspend(void); =20 /* Range of allocated PSCI function numbers */ #define PSCI_FNUM_MIN_VALUE _AC(0,U) diff --git a/xen/arch/arm/psci.c b/xen/arch/arm/psci.c index b6860a7760..e05dae1133 100644 --- a/xen/arch/arm/psci.c +++ b/xen/arch/arm/psci.c @@ -17,23 +17,27 @@ #include #include #include +#include =20 /* * While a 64-bit OS can make calls with SMC32 calling conventions, for * some calls it is necessary to use SMC64 to pass or return 64-bit values. - * For such calls PSCI_0_2_FN_NATIVE(x) will choose the appropriate + * For such calls PSCI_*_FN_NATIVE(x) will choose the appropriate * (native-width) function ID. */ #ifdef CONFIG_ARM_64 #define PSCI_0_2_FN_NATIVE(name) PSCI_0_2_FN64_##name +#define PSCI_1_0_FN_NATIVE(name) PSCI_1_0_FN64_##name #else #define PSCI_0_2_FN_NATIVE(name) PSCI_0_2_FN32_##name +#define PSCI_1_0_FN_NATIVE(name) PSCI_1_0_FN32_##name #endif =20 uint32_t psci_ver; uint32_t smccc_ver; =20 static uint32_t psci_cpu_on_nr; +static bool __ro_after_init has_psci_system_suspend; =20 #define PSCI_RET(res) ((int32_t)(res).a0) =20 @@ -60,6 +64,25 @@ void call_psci_cpu_off(void) } } =20 +int call_psci_system_suspend(void) +{ +#ifdef CONFIG_SYSTEM_SUSPEND + struct arm_smccc_res res; + + if ( !has_psci_system_suspend ) + return PSCI_NOT_SUPPORTED; + + /* Context ID is unused for the Xen resume path. */ + arm_smccc_smc(PSCI_1_0_FN_NATIVE(SYSTEM_SUSPEND), __pa(hyp_resume), 0, + &res); + return PSCI_RET(res); +#else + dprintk(XENLOG_WARNING, + "SYSTEM_SUSPEND not supported (CONFIG_SYSTEM_SUSPEND disabled)= \n"); + return PSCI_NOT_SUPPORTED; +#endif +} + void call_psci_system_off(void) { if ( psci_ver > PSCI_VERSION(0, 1) ) @@ -223,9 +246,15 @@ int __init psci_init(void) =20 psci_init_smccc(); =20 + has_psci_system_suspend =3D + psci_features(PSCI_1_0_FN_NATIVE(SYSTEM_SUSPEND)) =3D=3D 0; + printk(XENLOG_INFO "Using PSCI v%u.%u\n", PSCI_VERSION_MAJOR(psci_ver), PSCI_VERSION_MINOR(psci_ver)); =20 + printk(XENLOG_DEBUG "PSCI SYSTEM_SUSPEND is %ssupported by firmware\n", + has_psci_system_suspend ? "" : "not "); + return 0; } =20 --=20 2.43.0 From nobody Sat May 30 11:15:55 2026 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass(p=none dis=none) header.from=gmail.com ARC-Seal: i=1; a=rsa-sha256; t=1778605822; cv=none; d=zohomail.com; s=zohoarc; b=CrwRW2NXWMHzwRBO4/CJDFuGR/Xv0AEMiFeAYDbAmT/uxEIF6Lt3WbBeYNRDFx62QxvtysTIdZCpywnXJEIC4z3W08a/Cj5/Zd1N1UZ/cLbmOW5skoqfo+77EEbedfjo74r3nuSzcZmy3X3RdLlIN8C8pS3VCHw9ZKi/MFUZ+Ks= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1778605822; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=Qsze4I9Yt/cMI8M+tY/BjiP5Otbdm5FY4KxVXe9YCJU=; b=L5/XWCVjDbz1dOsWhTdajCFX1wlHIghABstGKll6vEwVSM4DMa5389lOJtTlBAlPZ6EFg3X/aTjwy3V66f0IhwFKkZfmZv+KE4EzBRtmwn37oX0DeSdcUK8tVjZk9OAqciT3Y/FO6Sq/0mDacFJxKeFNNWiKiQj15XdL+SN+Uh0= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1778605822062375.5308546335789; Tue, 12 May 2026 10:10:22 -0700 (PDT) Received: from list by lists.xenproject.org with outflank-mailman.1307291.1579087 (Exim 4.92) (envelope-from ) id 1wMqcL-0000HX-FO; Tue, 12 May 2026 17:10:01 +0000 Received: by outflank-mailman (output) from mailman id 1307291.1579087; Tue, 12 May 2026 17:10:01 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1wMqcL-0000FZ-7b; Tue, 12 May 2026 17:10:01 +0000 Received: by outflank-mailman (input) for mailman id 1307291; Tue, 12 May 2026 17:09:59 +0000 Received: from mx.expurgate.net ([195.190.135.10]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1wMqcI-00084X-T5 for xen-devel@lists.xenproject.org; Tue, 12 May 2026 17:09:59 +0000 Received: from mx.expurgate.net (helo=localhost) by mx.expurgate.net with esmtp id 1wMqcI-007RzZ-9l for xen-devel@lists.xenproject.org; Tue, 12 May 2026 19:09:58 +0200 Received: from [10.42.69.1] (helo=localhost) by localhost with ESMTP (eXpurgate MTA 0.9.1) (envelope-from ) id 6a035ec9-2eae-0a2a0a5409dd-0a2a4501afae-42 for ; Tue, 12 May 2026 19:09:58 +0200 Received: from [209.85.128.45] (helo=mail-wm1-f45.google.com) by tlsNG-d62444.mxtls.expurgate.net with ESMTPS (eXpurgate 4.56.1) (envelope-from ) id 6a035ee6-c1f2-0a2a45010019-d155802dbd81-3 for ; Tue, 12 May 2026 19:09:58 +0200 Received: by mail-wm1-f45.google.com with SMTP id 5b1f17b1804b1-488b0046078so48589885e9.1 for ; Tue, 12 May 2026 10:09:58 -0700 (PDT) Received: from EPUAKYIW02F7.. ([45.12.26.18]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-48fc8cccf90sm10838845e9.0.2026.05.12.10.09.56 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 12 May 2026 10:09:56 -0700 (PDT) X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" Authentication-Results: eu.smtp.expurgate.cloud; dkim=pass header.s=20251104 header.d=gmail.com header.i="@gmail.com" header.h="Content-Transfer-Encoding:MIME-Version:References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1778605797; x=1779210597; darn=lists.xenproject.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Qsze4I9Yt/cMI8M+tY/BjiP5Otbdm5FY4KxVXe9YCJU=; b=AAplRYV8rMejoKhWP2iI5KYR3jhS94dbaV8i23b/5m4XGMGJDoP7OlpkVvuwS2Q7xN kQ/aodTJpgRz5p1b6p7S/tEF1sr2E2B39Lo/Es1royG84ySfOnZxOY1iQu6kTPvdBji5 QwdI8p/9DwyZnDRHMsPXvARSI0RDjV4RDj3P397X5Elo1wD94FKYr3J2bncufuMCUvRB zGx9qBgL7PfHvUio05D1QVfTztFIXVOjs5iYimejj1toxoE+OoB+0T/8epQH1b0tmxeL H/WSQ4a4Y/EjHuzVPCXBJ1tWzoXImwN4wWs52yMcQlYTHrq9Nv9DBJbMv54NZPbIi/4i YsEA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1778605797; x=1779210597; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=Qsze4I9Yt/cMI8M+tY/BjiP5Otbdm5FY4KxVXe9YCJU=; b=i2/z9x+EUVG9TdespivNlyXYkONUdhX+BOTWfPDpI2xr/nGcalBxOTqflQgCbr7yil qnjAInXsk7WoO3GHzc4dULNpRdSRL9bSGwMvGDizh6AMJFpORAxog5nuy83ar4iJdvVy gNHsj+Oomt/c4Ds57hs7NlStGxd9q/E/FkQUGAGYccbnoXgiKV/fulC8oTWHWHx6joaK ItmviNcpNXRFi5LALlY/JBVTIBH54tAnRSZdInZ2OnzeQPTU55zAcDLwIJkCUpOU5vAj jWhWutdQcUIyT3ySXrv7J2rHO2P7rmdCiBLzDijCdCoUQYxFkZIjowzat0q+STBLGLdv xNDA== X-Gm-Message-State: AOJu0Yy3tfi+P5PmyVZl2y/jJENV5vOr/ZetMvdms7PESqVNykH4mLyu u7NEL/LG0kgwcn8NGm9pDJv+TK3DeetUCmsR6ySr43w9liinUty/ROriuy+1dA== X-Gm-Gg: Acq92OFehrvUxPKUSJlVVi4A/MuAM5GSY3DASD6M8CbUEnShfugeV0T8XDzXQWY7NXt mSJBGFV2oGMO45/xH8FKRSVt6Ut6TfuBDwr7+I6JIc7ew62qc1qv7UmSGn8B+YjammrAVmpD2Kj hYvnhBGwNsNRl/AekKOXcIx1j5rSGfbWoSvu0rJ+RkApfgqS9fb30fthx8JtbpeXR/aqosV0MJG IKVaCKloYPkjF68D2cPGJYSP0+ofKa1Y3TYakKVy+lMrji04J6gADSlg4I5oJfvMJLZ6OwO/djs Sc/EnYbeVGZsnnPRE8Ir+HNVYsUXvHovxSdYsfBw2M8Rmnf1XPCzL4e3xqA8nQ9hRAflvbyCMa7 zr9bq2gf7zc7m1gPAz7ZwrhwJY0s+0HnUVop/aTEEfWnmOOxEfhCdJXj0nw+JNaHhtapIAq3W2t Uh2vXBx+fVYeAqt8Dqe2L83zXp2w== X-Received: by 2002:a05:600c:8b28:b0:489:149a:f9e7 with SMTP id 5b1f17b1804b1-48e706e1ce9mr265592235e9.27.1778605797177; Tue, 12 May 2026 10:09:57 -0700 (PDT) From: Mykola Kvach To: xen-devel@lists.xenproject.org Cc: Mykola Kvach , Stefano Stabellini , Julien Grall , Bertrand Marquis , Michal Orzel , Volodymyr Babchuk , Andrew Cooper , Anthony PERARD , Jan Beulich , =?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= , Rahul Singh Subject: [PATCH v9 12/13] xen/arm: Add vPSCI SYSTEM_SUSPEND policy Date: Tue, 12 May 2026 20:07:19 +0300 Message-ID: <06448a30ef6694a55ac85d4c8c78ca2d71342430.1778605274.git.mykola_kvach@epam.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: References: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-purgate-ID: tlsNG-d62444/1778605798-B5F40FF4-B662D68F/0/0 X-purgate-type: clean X-purgate-size: 16491 X-ZohoMail-DKIM: pass (identity @gmail.com) X-ZM-MESSAGEID: 1778605823802158501 Content-Type: text/plain; charset="utf-8" From: Mykola Kvach Introduce CONFIG_HAS_HWDOM_SYSTEM_SUSPEND as an architecture-selected capability for platforms where the hardware domain can be parked with SHUTDOWN_suspend without calling hwdom_shutdown(). Expose PSCI SYSTEM_SUSPEND as a vPSCI operation for all domains. For non-control domains, including the hardware domain when it is not acting as= a control domain, the call is handled as a guest/domain suspend request and parks the domain in SHUTDOWN_suspend. Control domains need additional sequencing because their SYSTEM_SUSPEND request is used to coordinate host-wide suspend. A non-last awake control domain may be parked in SHUTDOWN_suspend without requiring the host suspend path to be available. The last awake control domain is treated as the point where the request becomes a host-suspend request, and it may only proceed when all non-control domains are already in SHUTDOWN_suspend and the host suspend path is available. Keep the control-domain sequencing and domain-readiness checks out of PSCI_FEATURES. They are per-attempt runtime conditions rather than stable P= SCI function availability. Advertise SYSTEM_SUSPEND as implemented by vPSCI and enforce the sequencing policy in the call handler. Select HAS_HWDOM_SYSTEM_SUSPEND independently from CONFIG_SYSTEM_SUSPEND so that SHUTDOWN_suspend from the hardware domain can be treated as a domain suspend state rather than as a hardware-domain initiated host shutdown. This does not by itself imply that host-wide suspend is available. Add host_system_suspend_allowed() to combine the host PSCI SYSTEM_SUSPEND capability with runtime blockers reported by Xen-owned subsystems. Add runtime blockers for registered serial, IOMMU, GIC and SMMUv3 MSI IRQ paths lacking suspend/resume support. These blockers are runtime based, so they only apply to drivers or paths that Xen actually uses on the platform. For SMMUv3, the blocker applies only when Xen actually uses the MSI IRQ path, since resume does not restore the SMMU *_IRQ_CFGn MSI registers yet. Signed-off-by: Mykola Kvach Reviewed-by: Luca Fancellu --- xen/arch/arm/Kconfig | 1 + xen/arch/arm/gic.c | 6 ++ xen/arch/arm/include/asm/psci.h | 3 + xen/arch/arm/include/asm/suspend.h | 10 ++- xen/arch/arm/psci.c | 7 ++ xen/arch/arm/suspend.c | 40 +++++++++ xen/arch/arm/vpsci.c | 114 +++++++++++++++++++++++--- xen/common/Kconfig | 3 + xen/common/domain.c | 7 +- xen/drivers/char/serial.c | 12 +++ xen/drivers/passthrough/arm/iommu.c | 4 + xen/drivers/passthrough/arm/smmu-v3.c | 4 + xen/include/xen/serial.h | 1 + xen/include/xen/suspend.h | 2 + 14 files changed, 201 insertions(+), 13 deletions(-) diff --git a/xen/arch/arm/Kconfig b/xen/arch/arm/Kconfig index 79622b46a1..54a5bfb9ae 100644 --- a/xen/arch/arm/Kconfig +++ b/xen/arch/arm/Kconfig @@ -19,6 +19,7 @@ config ARM select HAS_ALTERNATIVE if HAS_VMAP select HAS_DEVICE_TREE_DISCOVERY select HAS_DOM0LESS + select HAS_HWDOM_SYSTEM_SUSPEND if !MPU select HAS_GRANT_CACHE_FLUSH if GRANT_TABLE select HAS_STACK_PROTECTOR select HAS_UBSAN diff --git a/xen/arch/arm/gic.c b/xen/arch/arm/gic.c index 7727ffed5a..a5efcaeb4c 100644 --- a/xen/arch/arm/gic.c +++ b/xen/arch/arm/gic.c @@ -26,6 +26,7 @@ #include #include #include +#include #include #include =20 @@ -44,6 +45,11 @@ static void __init __maybe_unused build_assertions(void) void register_gic_ops(const struct gic_hw_operations *ops) { gic_hw_ops =3D ops; + +#ifdef CONFIG_SYSTEM_SUSPEND + if ( !ops->suspend || !ops->resume ) + host_system_suspend_disable("GIC driver lacks suspend/resume suppo= rt"); +#endif } =20 static void clear_cpu_lr_mask(void) diff --git a/xen/arch/arm/include/asm/psci.h b/xen/arch/arm/include/asm/psc= i.h index bb3c73496e..142fa1bfe5 100644 --- a/xen/arch/arm/include/asm/psci.h +++ b/xen/arch/arm/include/asm/psci.h @@ -24,6 +24,9 @@ void call_psci_cpu_off(void); void call_psci_system_off(void); void call_psci_system_reset(void); int call_psci_system_suspend(void); +#ifdef CONFIG_SYSTEM_SUSPEND +bool psci_system_suspend_allowed(void); +#endif =20 /* Range of allocated PSCI function numbers */ #define PSCI_FNUM_MIN_VALUE _AC(0,U) diff --git a/xen/arch/arm/include/asm/suspend.h b/xen/arch/arm/include/asm/= suspend.h index 2d9fc331fc..87db12eac3 100644 --- a/xen/arch/arm/include/asm/suspend.h +++ b/xen/arch/arm/include/asm/suspend.h @@ -38,7 +38,15 @@ extern struct resume_cpu_context resume_cpu_context; =20 int prepare_resume_ctx(void); void hyp_resume(void); -#endif /* CONFIG_SYSTEM_SUSPEND */ +bool host_system_suspend_allowed(void); +void host_system_suspend_disable(const char *reason); + +#else /* !CONFIG_SYSTEM_SUSPEND */ + +static inline bool host_system_suspend_allowed(void) { return false; } +static inline void host_system_suspend_disable(const char *reason) {} + +#endif =20 #endif /* ARM_SUSPEND_H */ =20 diff --git a/xen/arch/arm/psci.c b/xen/arch/arm/psci.c index e05dae1133..e9d78668fd 100644 --- a/xen/arch/arm/psci.c +++ b/xen/arch/arm/psci.c @@ -41,6 +41,13 @@ static bool __ro_after_init has_psci_system_suspend; =20 #define PSCI_RET(res) ((int32_t)(res).a0) =20 +#ifdef CONFIG_SYSTEM_SUSPEND +bool psci_system_suspend_allowed(void) +{ + return has_psci_system_suspend; +} +#endif + int call_psci_cpu_on(int cpu) { struct arm_smccc_res res; diff --git a/xen/arch/arm/suspend.c b/xen/arch/arm/suspend.c index 6ea4a0f9cc..a571035d2c 100644 --- a/xen/arch/arm/suspend.c +++ b/xen/arch/arm/suspend.c @@ -1,9 +1,49 @@ /* SPDX-License-Identifier: GPL-2.0-only */ =20 +#include #include =20 +#include +#include + struct resume_cpu_context resume_cpu_context; =20 +/* + * Non-PSCI infrastructure can make host suspend impossible even when the = PSCI + * SYSTEM_SUSPEND conduit is present, e.g. when a Xen-owned driver has no = valid + * suspend/resume path. + * + * This gate is checked only when the last awake control domain attempts to + * turn a guest SYSTEM_SUSPEND request into a host-suspend request. + */ +static bool host_system_suspend_runtime_allowed =3D true; + +static bool host_serial_suspend_allowed(void) +{ + if ( serial_suspend_supported() ) + return true; + + printk_once(XENLOG_INFO + "Host SYSTEM_SUSPEND blocked: serial driver lacks suspend/= resume support\n"); + + return false; +} + +bool host_system_suspend_allowed(void) +{ + return psci_system_suspend_allowed() && + host_serial_suspend_allowed() && + host_system_suspend_runtime_allowed; +} + +void host_system_suspend_disable(const char *reason) +{ + host_system_suspend_runtime_allowed =3D false; + + printk(XENLOG_INFO "Host SYSTEM_SUSPEND blocked: %s\n", + reason ? reason : "unsupported suspend/resume path"); +} + /* * Local variables: * mode: C diff --git a/xen/arch/arm/vpsci.c b/xen/arch/arm/vpsci.c index ac6af6118f..48a963ae3e 100644 --- a/xen/arch/arm/vpsci.c +++ b/xen/arch/arm/vpsci.c @@ -5,6 +5,7 @@ =20 #include #include +#include #include #include #include @@ -219,6 +220,89 @@ static void do_psci_0_2_system_reset(void) domain_shutdown(d,SHUTDOWN_reboot); } =20 +/* + * Serialise SYSTEM_SUSPEND policy decisions with the domain suspend trans= ition, + * so multiple control domains cannot all observe each other as still awak= e. + */ +static DEFINE_SPINLOCK(vpsci_system_suspend_lock); + +static bool domain_in_suspend_state(struct domain *d) +{ + bool suspended; + + spin_lock(&d->shutdown_lock); + suspended =3D d->is_shut_down && d->shutdown_code =3D=3D SHUTDOWN_susp= end; + spin_unlock(&d->shutdown_lock); + + return suspended; +} + +static int32_t domain_psci_system_suspend_policy(struct domain *d) +{ + struct domain *other; + bool last_awake_control_domain =3D true; + bool awake_non_control_domain =3D false; + + /* Only control domains participate in sequencing policy. */ + if ( !is_control_domain(d) ) + return 0; + + rcu_read_lock(&domlist_read_lock); + + for_each_domain ( other ) + { + bool suspended; + + if ( other =3D=3D d ) + continue; + + suspended =3D domain_in_suspend_state(other); + if ( suspended ) + continue; + + if ( is_control_domain(other) ) + { + last_awake_control_domain =3D false; + break; + } + + awake_non_control_domain =3D true; + } + + rcu_read_unlock(&domlist_read_lock); + + /* + * Another control domain is still awake. This request is only the fir= st + * phase of the sequencing: park this control domain and leave the host + * running. Host-wide suspend gates must not block this intermediate s= tate. + */ + if ( !last_awake_control_domain ) + return 0; + + /* + * This is the last awake control domain. It must not be parked unless= the + * request can proceed as a host-suspend request; otherwise Xen would = lose + * the last domain that can coordinate the system suspend. + */ + if ( awake_non_control_domain ) + { + printk(XENLOG_DEBUG + "SYSTEM_SUSPEND denied: last awake control domain dom%u req= uested host suspend while non-control domains are still awake\n", + d->domain_id); + return PSCI_DENIED; + } + + /* + * Host-wide gates are relevant only for the last-control-domain case.= They + * must not block parking of a non-last control domain, but they must = reject + * the last control domain when host suspend is not available. + */ + if ( !host_system_suspend_allowed() ) + return PSCI_NOT_SUPPORTED; + + return 0; +} + static int32_t do_psci_1_0_system_suspend(register_t epoint, register_t ci= d) { int32_t rc; @@ -232,10 +316,6 @@ static int32_t do_psci_1_0_system_suspend(register_t e= point, register_t cid) if ( is_64bit_domain(d) && is_thumb ) return PSCI_INVALID_ADDRESS; =20 - /* SYSTEM_SUSPEND is not supported for the hardware domain yet */ - if ( is_hardware_domain(d) ) - return PSCI_NOT_SUPPORTED; - /* Ensure that all CPUs other than the calling one are offline */ domain_lock(d); for_each_vcpu ( d, v ) @@ -252,16 +332,29 @@ static int32_t do_psci_1_0_system_suspend(register_t = epoint, register_t cid) if ( rc ) return PSCI_DENIED; =20 - rc =3D domain_shutdown(d, SHUTDOWN_suspend); + spin_lock(&vpsci_system_suspend_lock); + + rc =3D domain_psci_system_suspend_policy(d); + if ( !rc ) + { + rc =3D domain_shutdown(d, SHUTDOWN_suspend); + if ( rc ) + rc =3D PSCI_DENIED; + else + { + rctx->ctxt =3D ctxt; + rctx->wake_cpu =3D current; + } + } + + spin_unlock(&vpsci_system_suspend_lock); + if ( rc ) { free_vcpu_guest_context(ctxt); - return PSCI_DENIED; + return rc; } =20 - rctx->ctxt =3D ctxt; - rctx->wake_cpu =3D current; - gprintk(XENLOG_DEBUG, "SYSTEM_SUSPEND requested, epoint=3D%#"PRIregister", cid=3D%#"= PRIregister"\n", epoint, cid); @@ -287,10 +380,9 @@ static int32_t do_psci_1_0_features(uint32_t psci_func= _id) case PSCI_0_2_FN32_SYSTEM_RESET: case PSCI_1_0_FN32_PSCI_FEATURES: case ARM_SMCCC_VERSION_FID: - return 0; case PSCI_1_0_FN32_SYSTEM_SUSPEND: case PSCI_1_0_FN64_SYSTEM_SUSPEND: - return is_hardware_domain(current->domain) ? PSCI_NOT_SUPPORTED : = 0; + return 0; default: return PSCI_NOT_SUPPORTED; } diff --git a/xen/common/Kconfig b/xen/common/Kconfig index 5ff71480ee..816a1a4ecb 100644 --- a/xen/common/Kconfig +++ b/xen/common/Kconfig @@ -140,6 +140,9 @@ config HAS_EX_TABLE config HAS_FAST_MULTIPLY bool =20 +config HAS_HWDOM_SYSTEM_SUSPEND + bool + config HAS_IOPORTS bool =20 diff --git a/xen/common/domain.c b/xen/common/domain.c index bb9e210c28..d3edfb2a13 100644 --- a/xen/common/domain.c +++ b/xen/common/domain.c @@ -1375,6 +1375,11 @@ void __domain_crash(struct domain *d) domain_shutdown(d, SHUTDOWN_crash); } =20 +static inline bool want_hwdom_shutdown(uint8_t reason) +{ + return !IS_ENABLED(CONFIG_HAS_HWDOM_SYSTEM_SUSPEND) || + reason !=3D SHUTDOWN_suspend; +} =20 int domain_shutdown(struct domain *d, u8 reason) { @@ -1391,7 +1396,7 @@ int domain_shutdown(struct domain *d, u8 reason) d->shutdown_code =3D reason; reason =3D d->shutdown_code; =20 - if ( is_hardware_domain(d) ) + if ( is_hardware_domain(d) && want_hwdom_shutdown(reason) ) hwdom_shutdown(reason); =20 if ( d->is_shutting_down ) diff --git a/xen/drivers/char/serial.c b/xen/drivers/char/serial.c index adb312d796..cc2b5b5dee 100644 --- a/xen/drivers/char/serial.c +++ b/xen/drivers/char/serial.c @@ -497,6 +497,8 @@ const struct vuart_info *serial_vuart_info(int idx) =20 #ifdef CONFIG_SYSTEM_SUSPEND =20 +static bool __read_mostly serial_suspend_available =3D true; + void serial_suspend(void) { int i; @@ -513,6 +515,11 @@ void serial_resume(void) com[i].driver->resume(&com[i]); } =20 +bool serial_suspend_supported(void) +{ + return serial_suspend_available; +} + #endif /* CONFIG_SYSTEM_SUSPEND */ =20 void __init serial_register_uart(int idx, struct uart_driver *driver, @@ -521,6 +528,11 @@ void __init serial_register_uart(int idx, struct uart_= driver *driver, /* Store UART-specific info. */ com[idx].driver =3D driver; com[idx].uart =3D uart; + +#ifdef CONFIG_SYSTEM_SUSPEND + if ( !driver->suspend || !driver->resume ) + serial_suspend_available =3D false; +#endif } =20 void __init serial_async_transmit(struct serial_port *port) diff --git a/xen/drivers/passthrough/arm/iommu.c b/xen/drivers/passthrough/= arm/iommu.c index 100545e23f..420b9fbdf0 100644 --- a/xen/drivers/passthrough/arm/iommu.c +++ b/xen/drivers/passthrough/arm/iommu.c @@ -19,6 +19,7 @@ #include #include #include +#include =20 #include =20 @@ -46,6 +47,9 @@ void __init iommu_set_ops(const struct iommu_ops *ops) } =20 iommu_ops =3D ops; + + if ( !ops->suspend || !ops->resume ) + host_system_suspend_disable("IOMMU driver lacks suspend/resume sup= port"); } =20 int __init iommu_hardware_setup(void) diff --git a/xen/drivers/passthrough/arm/smmu-v3.c b/xen/drivers/passthroug= h/arm/smmu-v3.c index 82c8ead979..443dd0e893 100644 --- a/xen/drivers/passthrough/arm/smmu-v3.c +++ b/xen/drivers/passthrough/arm/smmu-v3.c @@ -91,6 +91,7 @@ #include #include #include +#include =20 #include "smmu-v3.h" =20 @@ -1903,6 +1904,9 @@ static void arm_smmu_setup_msis(struct arm_smmu_devic= e *smmu) } } =20 + host_system_suspend_disable( + "SMMUv3 MSI IRQ path is unsupported for host suspend"); + /* Add callback to free MSIs on teardown */ devm_add_action(dev, arm_smmu_free_msis, dev); } diff --git a/xen/include/xen/serial.h b/xen/include/xen/serial.h index 8e18445552..418b00ead0 100644 --- a/xen/include/xen/serial.h +++ b/xen/include/xen/serial.h @@ -137,6 +137,7 @@ const struct vuart_info* serial_vuart_info(int idx); /* Serial suspend/resume. */ void serial_suspend(void); void serial_resume(void); +bool serial_suspend_supported(void); #endif =20 /* diff --git a/xen/include/xen/suspend.h b/xen/include/xen/suspend.h index 6f94fd53b0..a941331035 100644 --- a/xen/include/xen/suspend.h +++ b/xen/include/xen/suspend.h @@ -6,6 +6,8 @@ #if __has_include() #include #else +struct domain; + static inline void arch_domain_resume(struct domain *d) {} #endif =20 --=20 2.43.0 From nobody Sat May 30 11:15:55 2026 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass(p=none dis=none) header.from=gmail.com ARC-Seal: i=1; a=rsa-sha256; t=1778605825; cv=none; d=zohomail.com; s=zohoarc; b=D5+BpwrGJ2jZj3g4djPEIwvk5NjOdXQxUW7drwOo9M86b8viEOH2L1yNOjJX1HhRD6owPDINn76C5KO1LB0ZIaWuUAYFZWtaWE1ve70namPDuDTwlsiNLkrdnRu7DV6BegJpQR+Jz6jppY7nlm55xOQiuBzzYZMVpy7tvgb6eqc= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1778605825; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=ibGyBzTWotDComsaOUE0ixj57R/0uTs4BWKAvB2xeVg=; b=Cnxr2iGEdeNaHvCLzQOKwEk2b/pxm7nBUhnYnqMvCxFWHSUEKJPN9DusJehTc/ECzCHv+ihITGGXmPwEJroK7gLtC5gCtfMjJFQexQ4LBLEx1wcNlxkihHv3+BCh+aQ+FUP6cJb84dpMMHmdDgHYZoMLcfueKY441qW6QMFlR6U= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1778605825658135.31516714815746; Tue, 12 May 2026 10:10:25 -0700 (PDT) Received: from list by lists.xenproject.org with outflank-mailman.1307294.1579093 (Exim 4.92) (envelope-from ) id 1wMqcM-0000Qh-8W; Tue, 12 May 2026 17:10:02 +0000 Received: by outflank-mailman (output) from mailman id 1307294.1579093; Tue, 12 May 2026 17:10:02 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1wMqcL-0000Oq-U9; Tue, 12 May 2026 17:10:01 +0000 Received: by outflank-mailman (input) for mailman id 1307294; Tue, 12 May 2026 17:10:00 +0000 Received: from mx.expurgate.net ([195.190.135.10]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1wMqcJ-0008JG-V4 for xen-devel@lists.xenproject.org; Tue, 12 May 2026 17:10:00 +0000 Received: from mx.expurgate.net (helo=localhost) by mx.expurgate.net with esmtp id 1wMqcJ-00HVbk-Bv for xen-devel@lists.xenproject.org; Tue, 12 May 2026 19:09:59 +0200 Received: from [10.42.69.7] (helo=localhost) by localhost with ESMTP (eXpurgate MTA 0.9.1) (envelope-from ) id 6a035ee6-5cb7-0a2a0a5109dd-0a2a4507b76a-2 for ; Tue, 12 May 2026 19:09:59 +0200 Received: from [209.85.128.41] (helo=mail-wm1-f41.google.com) by tlsNG-ef75cf.mxtls.expurgate.net with ESMTPS (eXpurgate 4.56.1) (envelope-from ) id 6a035ee7-229c-0a2a45070019-d1558029c526-3 for ; Tue, 12 May 2026 19:09:59 +0200 Received: by mail-wm1-f41.google.com with SMTP id 5b1f17b1804b1-488af9fdaa7so33351435e9.1 for ; Tue, 12 May 2026 10:09:59 -0700 (PDT) Received: from EPUAKYIW02F7.. ([45.12.26.18]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-48fc8cccf90sm10838845e9.0.2026.05.12.10.09.57 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 12 May 2026 10:09:57 -0700 (PDT) X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" Authentication-Results: eu.smtp.expurgate.cloud; dkim=pass header.s=20251104 header.d=gmail.com header.i="@gmail.com" header.h="Content-Transfer-Encoding:MIME-Version:References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1778605798; x=1779210598; darn=lists.xenproject.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=ibGyBzTWotDComsaOUE0ixj57R/0uTs4BWKAvB2xeVg=; b=KC25qTOVAhEmuxwwYEr6lHcuM//ugu9+kW5sqVHg9zKVn6uxFZB+dvkI+csnHr1l+A QN8kvMClmBDEWqb9sp4Z2R2bvB+ZOZBzCnJtHScxH4jj7xKj/+i8tNW+I+VwXIPfqFrE /7pBR0hJuHuP7krjkiglL1bcdBcFagejA3+bRTZLCJhVrIf2hrk/y3n/aBEcFamEOtjI DhyY7Rqy0O0jdYUDsuB724FE7u9W8PASsPr9LtXdPok3p5AYrkHpE98T42XdnyOc0grC ROY0O+7n62GGw8rq5rUELY0SQpmD8F8k3yc15kXuFo3f1L739HLyIcamQHjsbvLSv4Jw wcvw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1778605798; x=1779210598; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=ibGyBzTWotDComsaOUE0ixj57R/0uTs4BWKAvB2xeVg=; b=aQS90JYfzd8F0UdOujDLBmZxSjCiu3GN7wyIzonZTVnIFyENrZUpOieckn8aI/jlYH q7WjdEklzgcqJNPkl4VluErLaVFQ6Bl28BxR6K8Cn0/ZT9+sdaVnuS44A1z3TDwAPo9T PYclqaJbsPYDgny6y3egawo51Ft2CGZBAHkttCmDYdBnxUF+mGSQzdhdjn3qpma1G3av tDfPZYxJsiX9l5nr3YSsqRmnzIMU+Xa0x23g9LM43iyVkkih5Za0lF7WFGjCwhB+kTS/ OEwFvaACPaR/9FjEWkBgQmlw6h6VjtSDhU6E9VqPzxkyfYiG65Za/Ecu0YEBQpoLcD5H GhKg== X-Gm-Message-State: AOJu0YwRh9y9Iwh0yDonJqnUNN4Srfc7osturxquVeFEtGVx+67YpwPh xsDlB2JjYNs7FrS/JSk73nch0n/qhn4vtbvXah4oVvJLEi7N4ig+WYh3unIhKg== X-Gm-Gg: Acq92OHTFyxpq7iJD8zwzFUXxLn1RtuS49dAr9+esHsPBmhBOkplWEf8I3hc2a32TLX 8DAMLehwvKmbA4lEjfJiRp9gvtmkcLJfDI6W1yWmvCb9FEIxxDyA+dSx3q5espbrM7b288R+XKd DS3oxGrsJ7E5lA1xFhEsmN0fM+Dz9zDHib1m9LnkD0wJl7L5rhGbBE1TTzY0I23lK8ReXk7VZwQ V0fRa00uj7hCljpr6MmIaxhHgP2uw7Z5WNPWbzj+Nj2Em0KwdcvQAXwy9uwer0PpY1VU/wF8pzV a9fEnLdHtgrYMCPmeoAxsQmls2A3eCWUiD5nT1T2p6XNjIjorqis0ocb+8J9SOzGWP2CpLHT571 6rX2EN+G+CaN36njrNNkshFOLFkhj8NyBY5oSAnMdbhwTBNLkYqf9SO9N5yvIofQOb9zBYe5Fwx 3XWZbwtQ1QIxEM40W6TR3eN2SD92gslHLnJdhf X-Received: by 2002:a05:600c:3f0f:b0:488:a797:f0ac with SMTP id 5b1f17b1804b1-48e9007290cmr63915715e9.28.1778605798406; Tue, 12 May 2026 10:09:58 -0700 (PDT) From: Mykola Kvach To: xen-devel@lists.xenproject.org Cc: Mykola Kvach , Stefano Stabellini , Julien Grall , Bertrand Marquis , Michal Orzel , Volodymyr Babchuk Subject: [PATCH v9 13/13] xen/arm: Add host system suspend backend Date: Tue, 12 May 2026 20:07:20 +0300 Message-ID: <2268857aa7eef95444169564c17f7e9a6f59b1c5.1778605274.git.mykola_kvach@epam.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: References: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-purgate-ID: tlsNG-ef75cf/1778605799-22772C48-87001D5D/0/0 X-purgate-type: clean X-purgate-size: 11802 X-ZohoMail-DKIM: pass (identity @gmail.com) X-ZM-MESSAGEID: 1778605828336154100 Content-Type: text/plain; charset="utf-8" From: Mirela Simonovic Add the Xen-wide suspend/resume backend used after a control-domain vPSCI SYSTEM_SUSPEND request has been accepted. The vPSCI policy, runtime driver blockers and control-domain sequencing checks are handled by the preceding commit; this change adds the code that actually drives the host suspend attempt. The backend runs from a tasklet scheduled on pCPU0, because non-boot CPUs are disabled during suspend. It freezes domains, disables the scheduler and then disables non-boot CPUs. Host-side suspend participants are handled in phases. IOMMU and console state are suspended first. Local IRQs are then disabled before suspending timer and GIC state. On resume or failure, the completed suspend phases are unwound in reverse: GIC and timer state are restored while IRQs are still disabled, local IRQs are restored, and then console and IOMMU state are restored. On boot, init_ttbr is normally initialized during secondary CPU hotplug. On uniprocessor systems this can leave init_ttbr uninitialized, so set it from the boot CPU before entering suspend. Note: the code is behind CONFIG_HAS_SYSTEM_SUSPEND, which is currently only selected when UNSUPPORTED is set and MPU is not set. Signed-off-by: Mirela Simonovic Signed-off-by: Saeed Nowshadi Signed-off-by: Mykyta Poturai Signed-off-by: Mykola Kvach Reviewed-by: Luca Fancellu --- Changes in V9: - Split vPSCI availability policy, runtime host-suspend blockers and the domain-readiness precheck into the preceding commit. - Trigger the host suspend backend from the control-domain SYSTEM_SUSPEND path. - Reorder the host suspend/resume phases so the timer is suspended with local IRQs disabled and local IRQs are restored after the GIC and timer resume paths, before the console and IOMMU resume paths. - Move HAS_HWDOM_SYSTEM_SUSPEND and related logic to policy patch. Changes in V8: - Add a pre-suspend check in system_suspend() after scheduler_disable() to require all domains to be in the shut down state with SHUTDOWN_suspend before proceeding with the global suspend flow. - Drop the common-level depends on !ARM_64 || !SYSTEM_SUSPEND from CONFIG_HAS_HWDOM_SHUTDOWN_ON_SUSPEND and model the ARM64 suspend case with an arch-selected capability instead. - Rename CONFIG_HAS_HWDOM_SHUTDOWN_ON_SUSPEND to CONFIG_HAS_HWDOM_SYSTEM_SUSPEND. - Rename need_hwdom_shutdown() to want_hwdom_shutdown(). Changes in V7: - Control domain is responsible for host suspend. - Add an empty inline host_system_suspend() function when SYSTEM_SUSPEND config is disabled. - Use IS_ENABLED() for config checking instead of #ifdef. - Replace #ifdef checks in domain_shutdown() with IS_ENABLED() to simplify control flow. - Factor hardware domain shutdown condition into a helper (need_hwdom_shutdown()) to avoid preprocessor directives inside the function. - Squash with iommu suspend/resume commit. --- xen/arch/arm/Kconfig | 1 + xen/arch/arm/include/asm/mm.h | 2 + xen/arch/arm/include/asm/suspend.h | 2 + xen/arch/arm/mmu/smpboot.c | 2 +- xen/arch/arm/suspend.c | 140 +++++++++++++++++++++++++++++ xen/arch/arm/vpsci.c | 10 ++- 6 files changed, 154 insertions(+), 3 deletions(-) diff --git a/xen/arch/arm/Kconfig b/xen/arch/arm/Kconfig index 54a5bfb9ae..119bc00674 100644 --- a/xen/arch/arm/Kconfig +++ b/xen/arch/arm/Kconfig @@ -9,6 +9,7 @@ config ARM_64 select 64BIT select HAS_DOMAIN_TYPE select HAS_FAST_MULTIPLY + select HAS_SYSTEM_SUSPEND if !MPU && UNSUPPORTED select HAS_VPCI_GUEST_SUPPORT if PCI_PASSTHROUGH =20 config ARM diff --git a/xen/arch/arm/include/asm/mm.h b/xen/arch/arm/include/asm/mm.h index 2eb8465aa9..de119cad3a 100644 --- a/xen/arch/arm/include/asm/mm.h +++ b/xen/arch/arm/include/asm/mm.h @@ -360,6 +360,8 @@ static inline void page_set_xenheap_gfn(struct page_inf= o *p, gfn_t gfn) } while ( (y =3D cmpxchg(&p->u.inuse.type_info, x, nx)) !=3D x ); } =20 +void set_init_ttbr(lpae_t *root); + #endif /* __ARCH_ARM_MM__ */ /* * Local variables: diff --git a/xen/arch/arm/include/asm/suspend.h b/xen/arch/arm/include/asm/= suspend.h index 87db12eac3..a194dbb21a 100644 --- a/xen/arch/arm/include/asm/suspend.h +++ b/xen/arch/arm/include/asm/suspend.h @@ -40,11 +40,13 @@ int prepare_resume_ctx(void); void hyp_resume(void); bool host_system_suspend_allowed(void); void host_system_suspend_disable(const char *reason); +void host_system_suspend(struct domain *d); =20 #else /* !CONFIG_SYSTEM_SUSPEND */ =20 static inline bool host_system_suspend_allowed(void) { return false; } static inline void host_system_suspend_disable(const char *reason) {} +static inline void host_system_suspend(struct domain *d) {} =20 #endif =20 diff --git a/xen/arch/arm/mmu/smpboot.c b/xen/arch/arm/mmu/smpboot.c index 37e91d72b7..ff508ecf40 100644 --- a/xen/arch/arm/mmu/smpboot.c +++ b/xen/arch/arm/mmu/smpboot.c @@ -72,7 +72,7 @@ static void clear_boot_pagetables(void) clear_table(boot_third); } =20 -static void set_init_ttbr(lpae_t *root) +void set_init_ttbr(lpae_t *root) { /* * init_ttbr is part of the identity mapping which is read-only. So diff --git a/xen/arch/arm/suspend.c b/xen/arch/arm/suspend.c index a571035d2c..b1cc67fbdb 100644 --- a/xen/arch/arm/suspend.c +++ b/xen/arch/arm/suspend.c @@ -1,10 +1,16 @@ /* SPDX-License-Identifier: GPL-2.0-only */ =20 +#include #include #include =20 +#include +#include +#include #include +#include #include +#include =20 struct resume_cpu_context resume_cpu_context; =20 @@ -44,6 +50,140 @@ void host_system_suspend_disable(const char *reason) reason ? reason : "unsupported suspend/resume path"); } =20 +/* Xen suspend. data identifies the domain that initiated suspend. */ +static void system_suspend(void *data) +{ + int status; + unsigned long flags; + struct domain *d =3D (struct domain *)data; + + BUG_ON(system_state !=3D SYS_STATE_active); + + system_state =3D SYS_STATE_suspend; + + printk("Xen suspending...\n"); + + freeze_domains(); + scheduler_disable(); + + /* + * Non-boot CPUs have to be disabled on suspend and enabled on resume + * (hotplug-based mechanism). Disabling non-boot CPUs will lead to PSCI + * CPU_OFF to be called by each non-boot CPU. Depending on the underly= ing + * platform capabilities, this may lead to the physical powering down = of + * CPUs. + */ + status =3D disable_nonboot_cpus(); + if ( status ) + { + system_state =3D SYS_STATE_resume; + goto resume_nonboot_cpus; + } + + console_start_sync(); + status =3D iommu_suspend(); + if ( status ) + { + system_state =3D SYS_STATE_resume; + goto resume_end_sync; + } + + status =3D console_suspend(); + if ( status ) + { + dprintk(XENLOG_ERR, "Failed to suspend the console, err=3D%d\n", s= tatus); + system_state =3D SYS_STATE_resume; + goto resume_iommu; + } + + local_irq_save(flags); + + time_suspend(); + + status =3D gic_suspend(); + if ( status ) + { + system_state =3D SYS_STATE_resume; + goto resume_time; + } + + set_init_ttbr(xen_pgtable); + + /* + * Enable identity mapping before entering suspend to simplify + * the resume path + */ + update_boot_mapping(true); + + if ( prepare_resume_ctx() ) + { + status =3D call_psci_system_suspend(); + /* + * If suspend is finalized properly by above system suspend PSCI c= all, + * the code below in this 'if' branch will never execute. Execution + * will continue from hyp_resume which is the hypervisor's resume = point. + * In hyp_resume CPU context will be restored and since link-regis= ter is + * restored as well, it will appear to return from prepare_resume_= ctx. + * The difference in returning from prepare_resume_ctx on system s= uspend + * versus resume is in function's return value: on suspend, the re= turn + * value is a non-zero value, on resume it is zero. That is why the + * control flow will not re-enter this 'if' branch on resume. + */ + if ( status ) + dprintk(XENLOG_WARNING, "PSCI system suspend failed, err=3D%d\= n", + status); + } + + system_state =3D SYS_STATE_resume; + update_boot_mapping(false); + + gic_resume(); + + resume_time: + time_resume(); + + local_irq_restore(flags); + + console_resume(); + + resume_iommu: + iommu_resume(); + + resume_end_sync: + console_end_sync(); + + resume_nonboot_cpus: + /* + * The rcu_barrier() has to be added to ensure that the per cpu area is + * freed before a non-boot CPU tries to initialize it (_free_percpu_ar= ea() + * has to be called before the init_percpu_area()). This scenario occu= rs + * when non-boot CPUs are hot-unplugged on suspend and hotplugged on r= esume. + */ + rcu_barrier(); + enable_nonboot_cpus(); + + scheduler_enable(); + thaw_domains(); + + system_state =3D SYS_STATE_active; + + printk("Resume (status %d)\n", status); + + domain_resume(d); +} + +static DECLARE_TASKLET(system_suspend_tasklet, system_suspend, NULL); + +void host_system_suspend(struct domain *d) +{ + system_suspend_tasklet.data =3D (void *)d; + /* + * The suspend procedure has to be finalized by the pCPU#0 (non-boot p= CPUs + * will be disabled during the suspend). + */ + tasklet_schedule_on_cpu(&system_suspend_tasklet, 0); +} + /* * Local variables: * mode: C diff --git a/xen/arch/arm/vpsci.c b/xen/arch/arm/vpsci.c index 48a963ae3e..79100cbb1e 100644 --- a/xen/arch/arm/vpsci.c +++ b/xen/arch/arm/vpsci.c @@ -237,7 +237,8 @@ static bool domain_in_suspend_state(struct domain *d) return suspended; } =20 -static int32_t domain_psci_system_suspend_policy(struct domain *d) +static int32_t domain_psci_system_suspend_policy(struct domain *d, + bool *host_suspend) { struct domain *other; bool last_awake_control_domain =3D true; @@ -300,6 +301,7 @@ static int32_t domain_psci_system_suspend_policy(struct= domain *d) if ( !host_system_suspend_allowed() ) return PSCI_NOT_SUPPORTED; =20 + *host_suspend =3D true; return 0; } =20 @@ -310,6 +312,7 @@ static int32_t do_psci_1_0_system_suspend(register_t ep= oint, register_t cid) struct vcpu *v; struct domain *d =3D current->domain; bool is_thumb =3D epoint & 1; + bool host_suspend =3D false; struct resume_info *rctx =3D &d->arch.resume_ctx; =20 /* THUMB set is not allowed with 64-bit domain */ @@ -334,7 +337,7 @@ static int32_t do_psci_1_0_system_suspend(register_t ep= oint, register_t cid) =20 spin_lock(&vpsci_system_suspend_lock); =20 - rc =3D domain_psci_system_suspend_policy(d); + rc =3D domain_psci_system_suspend_policy(d, &host_suspend); if ( !rc ) { rc =3D domain_shutdown(d, SHUTDOWN_suspend); @@ -359,6 +362,9 @@ static int32_t do_psci_1_0_system_suspend(register_t ep= oint, register_t cid) "SYSTEM_SUSPEND requested, epoint=3D%#"PRIregister", cid=3D%#"= PRIregister"\n", epoint, cid); =20 + if ( host_suspend ) + host_system_suspend(d); + return rc; } =20 --=20 2.43.0