From nobody Thu Oct 30 22:53:02 2025 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass(p=none dis=none) header.from=gmail.com ARC-Seal: i=1; a=rsa-sha256; t=1754945355; cv=none; d=zohomail.com; s=zohoarc; b=S8udbLkSKyWzI1Fd4nAWwdiPS88EuMlMd0z2BmP++6t7SXLnpM8ZjKRSpZ1aQaEFkN6HH022qo9gCN3DSivmFfiScA27MHo2V3NBLIIGyPlDPRup1NsWejlr70/DHEsfTQDmnBj8CNtiqfgZF+itBnebclRvwQ96xBzxrL+0cvs= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1754945355; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=uqPcozH5cY7WTdc/VNhtz0ZQbmCe6X/v3JhqZHPKizE=; b=OZvInrIqklVH6WYZixA8P6LZ/xHdfaAOFq7zOnArNVHRaEF6Y+07KlNA8MYjmb4mmyP2bkoT6B4Aa8vSRG2hkP6CwV23/oXkAe2DLwqmUa1Sr4YUUQ0h+0rOarRQyuavHwANdADED5BwZLzZsZUZj+Uu6gKWALXS0+a0uzZ3RBo= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1754945355954831.7430829548933; Mon, 11 Aug 2025 13:49:15 -0700 (PDT) Received: from list by lists.xenproject.org with outflank-mailman.1077944.1438956 (Exim 4.92) (envelope-from ) id 1ulZRr-0005qb-WE; Mon, 11 Aug 2025 20:48:52 +0000 Received: by outflank-mailman (output) from mailman id 1077944.1438956; Mon, 11 Aug 2025 20:48:51 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1ulZRr-0005qU-TZ; Mon, 11 Aug 2025 20:48:51 +0000 Received: by outflank-mailman (input) for mailman id 1077944; Mon, 11 Aug 2025 20:48:50 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1ulZRq-0005co-7n for xen-devel@lists.xenproject.org; Mon, 11 Aug 2025 20:48:50 +0000 Received: from mail-ej1-x632.google.com (mail-ej1-x632.google.com [2a00:1450:4864:20::632]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id 951bccd6-76f4-11f0-a327-13f23c93f187; Mon, 11 Aug 2025 22:48:49 +0200 (CEST) Received: by mail-ej1-x632.google.com with SMTP id a640c23a62f3a-adfb562266cso656737066b.0 for ; Mon, 11 Aug 2025 13:48:49 -0700 (PDT) Received: from yp-VivoBook-ASUSLaptop-M1503QA-M1503QA.Dlink ([77.52.179.38]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-af91a0a3bd2sm2065710466b.54.2025.08.11.13.48.46 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 11 Aug 2025 13:48:47 -0700 (PDT) X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 951bccd6-76f4-11f0-a327-13f23c93f187 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1754945329; x=1755550129; darn=lists.xenproject.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=uqPcozH5cY7WTdc/VNhtz0ZQbmCe6X/v3JhqZHPKizE=; b=PKounsQ3qXQ1uXh7Cbt4dJu7ffgtpXdtjQ+7V4KK6ZGNyB3NhFpoCg41AjVYBVEzpc +ywX2UVGek4uR0DL0LkJLbAKb+UOZ8IUFVi4tdaj2t4v4f1q5FPQvwG2m0HqLt63nq8D yw17SwtqVb0RqgGecgN8w+foOmsTzeAe4r/6c221uGQp5tlcHmHoFoq8vHl7IYC9FTCG 9K0dKSfgf56S6morcmuYKl5twUwZhlQ5f1M5kbvwHNN3BMQ94UVwZ5gQNI5lNPkiWJD8 K9UIg38vt4RzWlsUzLy0lutUutZSeIFu/QnPbWbdz/I2aLLiwJ+1Y0M8jjJsQuQAsn6g ziqw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1754945329; x=1755550129; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=uqPcozH5cY7WTdc/VNhtz0ZQbmCe6X/v3JhqZHPKizE=; b=X5AGP83O8PdhaiJJsCvu9YA141H3cwIKs+Vh0zd3NtUK6V+CHRG4bMuGPjj5kof4CP OCD06j9fdcaBPkoU+HgZoBnYnbPRyD7hkTnGNT03v9ptzoUvwforuvatwSak+7kflFoX AkXhe1cGP8PG53C3v3BC7nhbBcbAQ52LXYkLYZ3J8H/YEZg0fCl11uK7HyHgT0qbpGAb wUTc+8Kc+03KWgVMUBOpM4BZGRv/Q3U+NW69jt1sf9V97GqczdZE/mG0vxv5sxG1iqYh nqhdDhVvC5lN/+9DW+amVNxiWeTlcgKkBBr5cegzDpomdbMlKUSALHPNKpkmlBVGCP4p Uu/w== X-Gm-Message-State: AOJu0YznO+I0Wr02zk2LZc4hJuNd9MCF6Ks/l3dRUvlTAeWahpGbZN1c IXCdrHdNaiqPG1nv3Qkzio9oP8/G7VhAt6QwnzP8VAWoB+FzWHrpSKQqPjA2rQ== X-Gm-Gg: ASbGncuGNSj/sUYmymIn8gtikiOL4o7wgXwC/1U1MZ6cSf/QhcYRRH/00vCyd7j1LGO 6o34Yj91rWuGdkMew1Jv+9r8R4rW8ge3KRR34e6hEJqzMYhY9o1z7yfoe7Hk8Sy4cMbqweChySo MIgKWBdRe8/yFCL82UEq6QlWOaVcsyL/8a9ChT1rBNBJkbvv+Kcm5dMRS2ugxEHMTF09HntBf3X y8p5ffYBr97mW899gHee5rN4AoaqRZZCjZXCWHP8CmlSIRfeE6Uyk4P57IlhxFe7tOtur5biF3O o6zaDHai6a5kSzcwhCvCgKAh1TvMX6PXcvKa5yEgdffovWlG1TE4JMu0LPhYBBln+UM2QFdbW9X 6K+WkSpTZiYYD7QbZZaPJoWoYuC94Nbnexzudm00QRFiEqGFMzrbG96QKnnGrrSc7Qdo= X-Google-Smtp-Source: AGHT+IEu3D0gEmW80fyDtMme/VtHa6WjaROJtkDiDayImRqcZBL5eh09kXVI5SF+mCQWOVhf9h9k5Q== X-Received: by 2002:a17:906:794a:b0:af6:3194:f024 with SMTP id a640c23a62f3a-afa1dff3b08mr83862166b.13.1754945328309; Mon, 11 Aug 2025 13:48:48 -0700 (PDT) From: Mykola Kvach To: xen-devel@lists.xenproject.org Cc: Mirela Simonovic , Stefano Stabellini , Julien Grall , Bertrand Marquis , Michal Orzel , Volodymyr Babchuk , Saeed Nowshadi , Mykola Kvach Subject: [PATCH v5 01/12] xen/arm: Add suspend and resume timer helpers Date: Mon, 11 Aug 2025 23:47:57 +0300 Message-ID: <412e0f48636feda1825885a8aed92386f694c8ca.1754943874.git.mykola_kvach@epam.com> X-Mailer: git-send-email 2.48.1 In-Reply-To: References: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @gmail.com) X-ZM-MESSAGEID: 1754945357724124100 Content-Type: text/plain; charset="utf-8" From: Mirela Simonovic Timer interrupts must be disabled while the system is suspended to prevent spurious wake-ups. Suspending the timers involves disabling both the EL1 physical timer and the EL2 hypervisor timer. Resuming consists of raising the TIMER_SOFTIRQ, which prompts the generic timer code to reprogram the EL2 timer as needed. Re-enabling of the EL1 timer is left to the entity that uses it. Introduce a new helper, disable_physical_timers, to encapsulate disabling of the physical timers. Signed-off-by: Mirela Simonovic Signed-off-by: Saeed Nowshadi Signed-off-by: Mykola Kvach --- Changes in V4: - Rephrased comment and commit message for better clarity - Created separate function for disabling physical timers Changes in V3: - time_suspend and time_resume are now conditionally compiled under CONFIG_SYSTEM_SUSPEND --- xen/arch/arm/include/asm/time.h | 5 +++++ xen/arch/arm/time.c | 38 +++++++++++++++++++++++++++------ 2 files changed, 37 insertions(+), 6 deletions(-) diff --git a/xen/arch/arm/include/asm/time.h b/xen/arch/arm/include/asm/tim= e.h index 49ad8c1a6d..f4fd0c6af5 100644 --- a/xen/arch/arm/include/asm/time.h +++ b/xen/arch/arm/include/asm/time.h @@ -108,6 +108,11 @@ void preinit_xen_time(void); =20 void force_update_vcpu_system_time(struct vcpu *v); =20 +#ifdef CONFIG_SYSTEM_SUSPEND +void time_suspend(void); +void time_resume(void); +#endif /* CONFIG_SYSTEM_SUSPEND */ + #endif /* __ARM_TIME_H__ */ /* * Local variables: diff --git a/xen/arch/arm/time.c b/xen/arch/arm/time.c index e74d30d258..ad984fdfdd 100644 --- a/xen/arch/arm/time.c +++ b/xen/arch/arm/time.c @@ -303,6 +303,14 @@ static void check_timer_irq_cfg(unsigned int irq, cons= t char *which) "WARNING: %s-timer IRQ%u is not level triggered.\n", which, irq= ); } =20 +/* Disable physical timers for EL1 and EL2 on the current CPU */ +static inline void disable_physical_timers(void) +{ + WRITE_SYSREG(0, CNTP_CTL_EL0); /* Physical timer disabled */ + WRITE_SYSREG(0, CNTHP_CTL_EL2); /* Hypervisor's timer disabled */ + isb(); +} + /* Set up the timer interrupt on this CPU */ void init_timer_interrupt(void) { @@ -310,9 +318,7 @@ void init_timer_interrupt(void) WRITE_SYSREG64(0, CNTVOFF_EL2); /* No VM-specific offset */ /* Do not let the VMs program the physical timer, only read the physic= al counter */ WRITE_SYSREG(CNTHCTL_EL2_EL1PCTEN, CNTHCTL_EL2); - WRITE_SYSREG(0, CNTP_CTL_EL0); /* Physical timer disabled */ - WRITE_SYSREG(0, CNTHP_CTL_EL2); /* Hypervisor's timer disabled */ - isb(); + disable_physical_timers(); =20 request_irq(timer_irq[TIMER_HYP_PPI], 0, htimer_interrupt, "hyptimer", NULL); @@ -330,9 +336,7 @@ void init_timer_interrupt(void) */ static void deinit_timer_interrupt(void) { - WRITE_SYSREG(0, CNTP_CTL_EL0); /* Disable physical timer */ - WRITE_SYSREG(0, CNTHP_CTL_EL2); /* Disable hypervisor's timer */ - isb(); + disable_physical_timers(); =20 release_irq(timer_irq[TIMER_HYP_PPI], NULL); release_irq(timer_irq[TIMER_VIRT_PPI], NULL); @@ -372,6 +376,28 @@ void domain_set_time_offset(struct domain *d, int64_t = time_offset_seconds) /* XXX update guest visible wallclock time */ } =20 +#ifdef CONFIG_SYSTEM_SUSPEND + +void time_suspend(void) +{ + disable_physical_timers(); +} + +void time_resume(void) +{ + /* + * Raising the timer softirq triggers generic code to call reprogram_t= imer + * with the correct timeout (not known here). + * + * No further action is needed to restore timekeeping after power down, + * since the system counter is unaffected. See ARM DDI 0487 L.a, D12.1= .2 + * "The system counter must be implemented in an always-on power domai= n." + */ + raise_softirq(TIMER_SOFTIRQ); +} + +#endif /* CONFIG_SYSTEM_SUSPEND */ + static int cpu_time_callback(struct notifier_block *nfb, unsigned long action, void *hcpu) --=20 2.48.1 From nobody Thu Oct 30 22:53:02 2025 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass(p=none dis=none) header.from=gmail.com ARC-Seal: i=1; a=rsa-sha256; t=1754945351; cv=none; d=zohomail.com; s=zohoarc; b=eAw62I7cGrBSJRWQLxTXMLEBJT1Wp/6TET4YpXdZ//BvUfvy5B/mXNc7oL83Ip1IU03ggMBhZ1W5++gvlH1VNiCjUzm9S3TiT4Be1g8QXCRpKU0BVRYo1e0w2mIQaeQWMh8t5MZ63gwKH89JxRa7+Jr1FopCTZ7jrG2uyTbcQic= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1754945351; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=Jf3vC/bMLxsK+qQIEq63MqPOkecni+X3pTICciqQJuw=; b=Knenk4bsXJUYtMIudQiMXDz1fJyD1ZvblbjFPaxpkCBkxPsTxZNeV0vke78vX47AL84Kqx6yzYopui75RTyYNyxBu8nqtUMOeBL8WaCfhFT/qL2ac4i+vVkCGYPLceLNsEOuMCLZLu3/GPUhV5D43qsSd2OdlaY34cMbgQZmBV4= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1754945351115399.76058786236854; Mon, 11 Aug 2025 13:49:11 -0700 (PDT) Received: from list by lists.xenproject.org with outflank-mailman.1077945.1438966 (Exim 4.92) (envelope-from ) id 1ulZRu-00065X-Be; Mon, 11 Aug 2025 20:48:54 +0000 Received: by outflank-mailman (output) from mailman id 1077945.1438966; Mon, 11 Aug 2025 20:48:54 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1ulZRu-00065P-8R; Mon, 11 Aug 2025 20:48:54 +0000 Received: by outflank-mailman (input) for mailman id 1077945; Mon, 11 Aug 2025 20:48:53 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1ulZRt-00063z-8g for xen-devel@lists.xenproject.org; Mon, 11 Aug 2025 20:48:53 +0000 Received: from mail-ej1-x629.google.com (mail-ej1-x629.google.com [2a00:1450:4864:20::629]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id 95fa957c-76f4-11f0-b898-0df219b8e170; Mon, 11 Aug 2025 22:48:51 +0200 (CEST) Received: by mail-ej1-x629.google.com with SMTP id a640c23a62f3a-af66f444488so681718366b.0 for ; Mon, 11 Aug 2025 13:48:51 -0700 (PDT) Received: from yp-VivoBook-ASUSLaptop-M1503QA-M1503QA.Dlink ([77.52.179.38]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-af91a0a3bd2sm2065710466b.54.2025.08.11.13.48.48 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 11 Aug 2025 13:48:49 -0700 (PDT) X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 95fa957c-76f4-11f0-b898-0df219b8e170 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1754945330; x=1755550130; darn=lists.xenproject.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Jf3vC/bMLxsK+qQIEq63MqPOkecni+X3pTICciqQJuw=; b=OlN1c0VwLF7hxXz4fD6XS4+JI+erNwONBy1g1LW22l7j4/87QIS+XlCDbjla1rVvKh Y7iqkxgXrTkvwKNGjFeVOTjmtyBPn56+d18/yYYEDA/7R/r2LE6J+LsiYpNvOt27t1A1 +0mjuVYO22aOju+KscwFgAzcYmSnc99EljxyOuK7xSCq5mHyiVQg0bKFTBMbX9pkM1QU ULzQrWqqQsDpO3fg+VhSm6TRL19lCX7ECDTMZ8TuTinzjX6DSM6zpzXtHhVDuqznvIUf o0DurDNnCtkRdC+2vcpl1kx23EVeXqef5kyQQBYnq2hTkVrwWSpWtrVO32wY9PK5V1ie J3tw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1754945330; x=1755550130; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Jf3vC/bMLxsK+qQIEq63MqPOkecni+X3pTICciqQJuw=; b=kDn+byP24nuv5IQHaGffR+TWubMbuudds9ZhM+p5nSCLHxMuZYssUp2W5yBKiyFWLv FcX2dD9ady9TRq4a3zpZeu/d/D0gH94SLoCXaa+VxGXEjXTnaZWYMJhv3XR0i9Ft1lqJ LJstCaamroOgwOC4jODlWdXbkHvNvP4hmeosNrKA+w9rdmtEXU2Qwz2cDPD7Tv56o2Gc 9Hqm3jJZQmkpKowzVvfnzzOXT9wxpf97zRTpoBzCBFy3+iS1sQF9nR8gs5h9KegnC5Gb WRF5ypK788C2ph9nmkFwZX5OIaXITjfOxhMjrpLubBVuLWrw1ul3GEcl+dOzAZn1p2oO 0J6Q== X-Gm-Message-State: AOJu0YxhQpNPk6pFEca3yFRtC+SLR0THyHIPFybdl23iTyLRLcWWrW0K LguHCJ5uFkFVm2+QBjKyFYC62xQu547+S4xpfPqhKFYOvy5n/IxyW9L36ikkHQ== X-Gm-Gg: ASbGncsUOZdk5s++PDxftvmCbgRgGov9T2w3IFZPYuDoFS3l1FlJnh8nsJ8d+W+JsvK kIRgdRJmj+2UQ5IKcNaJF3BRt3H43wa4StTU5bWPmNHtbhEy9zDWyXqqhW8TFHiFSGOIWFEFKcm SF93j15TCfHpC0dlPEumN0ADy0GKcS5QrgEVT0rKlj/3wt2A15/Qovv8+UXp8mNoQT3xB53YIVJ M8PcO4qeyq/Ky4y3YA9pJk1kMnXn04HwN2YKPZy5M+i59fFNp5/OAc5zi5CavPWtp+azW1sjBs6 nvRBzt94ZQOJInV0DEf3q2vTIs9RBZMZYJtCTyRLwyvoES8uPOBCrvRg94sctcn8UqsJR5YEkB8 35qMfSbGvWCF5U/qMK5flN8OijCBKx6pFY6getDKJRDhxqqTfVY0aXSF91NrCz7Vr1aBsgyJY+v xm0g== X-Google-Smtp-Source: AGHT+IFBO8GBEBI4lp8QB14504qUcSt99Pg0isBR7NzLTnjZPSQSGyh0nQRdh+b79bbwVdR8Na+iXw== X-Received: by 2002:a17:906:fb05:b0:af9:e505:f620 with SMTP id a640c23a62f3a-af9e505fb9bmr705198266b.57.1754945330021; Mon, 11 Aug 2025 13:48:50 -0700 (PDT) From: Mykola Kvach To: xen-devel@lists.xenproject.org Cc: Mirela Simonovic , Stefano Stabellini , Julien Grall , Bertrand Marquis , Michal Orzel , Volodymyr Babchuk , Saeed Nowshadi , Mykyta Poturai , Mykola Kvach Subject: [PATCH v5 02/12] xen/arm: gic-v2: Implement GIC suspend/resume functions Date: Mon, 11 Aug 2025 23:47:58 +0300 Message-ID: X-Mailer: git-send-email 2.48.1 In-Reply-To: References: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @gmail.com) X-ZM-MESSAGEID: 1754945352921116600 Content-Type: text/plain; charset="utf-8" From: Mirela Simonovic System suspend may lead to a state where GIC would be powered down. Therefore, Xen should save/restore the context of GIC on suspend/resume. Note that the context consists of states of registers which are controlled by the hypervisor. Other GIC registers which are accessible by guests are saved/restored on context switch. Tested on Xilinx Ultrascale+ MPSoC with (and without) powering down the GIC. Signed-off-by: Mirela Simonovic Signed-off-by: Saeed Nowshadi Signed-off-by: Mykyta Poturai Signed-off-by: Mykola Kvach --- Changes in v4: - Add error logging for allocation failures Changes in v3: - Drop asserts and return error codes instead. - Wrap code with CONFIG_SYSTEM_SUSPEND. Changes in v2: - Minor fixes after review. --- xen/arch/arm/gic-v2.c | 154 +++++++++++++++++++++++++++++++++ xen/arch/arm/gic.c | 29 +++++++ xen/arch/arm/include/asm/gic.h | 12 +++ 3 files changed, 195 insertions(+) diff --git a/xen/arch/arm/gic-v2.c b/xen/arch/arm/gic-v2.c index b23e72a3d0..dce8f5e924 100644 --- a/xen/arch/arm/gic-v2.c +++ b/xen/arch/arm/gic-v2.c @@ -1098,6 +1098,151 @@ static int gicv2_iomem_deny_access(struct domain *d) return iomem_deny_access(d, mfn, mfn + nr); } =20 +#ifdef CONFIG_SYSTEM_SUSPEND + +/* GICv2 registers to be saved/restored on system suspend/resume */ +struct gicv2_context { + /* GICC context */ + uint32_t gicc_ctlr; + uint32_t gicc_pmr; + uint32_t gicc_bpr; + /* GICD context */ + uint32_t gicd_ctlr; + uint32_t *gicd_isenabler; + uint32_t *gicd_isactiver; + uint32_t *gicd_ipriorityr; + uint32_t *gicd_itargetsr; + uint32_t *gicd_icfgr; +}; + +static struct gicv2_context gicv2_context; + +static int gicv2_suspend(void) +{ + unsigned int i; + + if ( !gicv2_context.gicd_isenabler ) + { + dprintk(XENLOG_WARNING, "%s:%d: GICv2 suspend context not allocate= d!\n", + __func__, __LINE__); + return -ENOMEM; + } + + /* Save GICC configuration */ + gicv2_context.gicc_ctlr =3D readl_gicc(GICC_CTLR); + gicv2_context.gicc_pmr =3D readl_gicc(GICC_PMR); + gicv2_context.gicc_bpr =3D readl_gicc(GICC_BPR); + + /* Save GICD configuration */ + gicv2_context.gicd_ctlr =3D readl_gicd(GICD_CTLR); + + for ( i =3D 0; i < DIV_ROUND_UP(gicv2_info.nr_lines, 32); i++ ) + gicv2_context.gicd_isenabler[i] =3D readl_gicd(GICD_ISENABLER + i = * 4); + + for ( i =3D 0; i < DIV_ROUND_UP(gicv2_info.nr_lines, 32); i++ ) + gicv2_context.gicd_isactiver[i] =3D readl_gicd(GICD_ISACTIVER + i = * 4); + + for ( i =3D 0; i < DIV_ROUND_UP(gicv2_info.nr_lines, 4); i++ ) + gicv2_context.gicd_ipriorityr[i] =3D readl_gicd(GICD_IPRIORITYR + = i * 4); + + for ( i =3D 0; i < DIV_ROUND_UP(gicv2_info.nr_lines, 4); i++ ) + gicv2_context.gicd_itargetsr[i] =3D readl_gicd(GICD_ITARGETSR + i = * 4); + + for ( i =3D 0; i < DIV_ROUND_UP(gicv2_info.nr_lines, 16); i++ ) + gicv2_context.gicd_icfgr[i] =3D readl_gicd(GICD_ICFGR + i * 4); + + return 0; +} + +static void gicv2_resume(void) +{ + unsigned int i; + + if ( !gicv2_context.gicd_isenabler ) + { + dprintk(XENLOG_WARNING, "%s:%d: GICv2 suspend context not allocate= d!\n", + __func__, __LINE__); + return; + } + + gicv2_cpu_disable(); + /* Disable distributor */ + writel_gicd(0, GICD_CTLR); + + /* Restore GICD configuration */ + for ( i =3D 0; i < DIV_ROUND_UP(gicv2_info.nr_lines, 32); i++ ) { + writel_gicd(0xffffffff, GICD_ICENABLER + i * 4); + writel_gicd(gicv2_context.gicd_isenabler[i], GICD_ISENABLER + i * = 4); + } + + for ( i =3D 0; i < DIV_ROUND_UP(gicv2_info.nr_lines, 32); i++ ) { + writel_gicd(0xffffffff, GICD_ICACTIVER + i * 4); + writel_gicd(gicv2_context.gicd_isactiver[i], GICD_ISACTIVER + i * = 4); + } + + for ( i =3D 0; i < DIV_ROUND_UP(gicv2_info.nr_lines, 4); i++ ) + writel_gicd(gicv2_context.gicd_ipriorityr[i], GICD_IPRIORITYR + i = * 4); + + for ( i =3D 0; i < DIV_ROUND_UP(gicv2_info.nr_lines, 4); i++ ) + writel_gicd(gicv2_context.gicd_itargetsr[i], GICD_ITARGETSR + i * = 4); + + for ( i =3D 0; i < DIV_ROUND_UP(gicv2_info.nr_lines, 16); i++ ) + writel_gicd(gicv2_context.gicd_icfgr[i], GICD_ICFGR + i * 4); + + /* Make sure all registers are restored and enable distributor */ + writel_gicd(gicv2_context.gicd_ctlr | GICD_CTL_ENABLE, GICD_CTLR); + + /* Restore GIC CPU interface configuration */ + writel_gicc(gicv2_context.gicc_pmr, GICC_PMR); + writel_gicc(gicv2_context.gicc_bpr, GICC_BPR); + + /* Enable GIC CPU interface */ + writel_gicc(gicv2_context.gicc_ctlr | GICC_CTL_ENABLE | GICC_CTL_EOI, + GICC_CTLR); +} + +static void gicv2_alloc_context(struct gicv2_context *gc) +{ + uint32_t n =3D gicv2_info.nr_lines; + + gc->gicd_isenabler =3D xzalloc_array(uint32_t, DIV_ROUND_UP(n, 32)); + if ( !gc->gicd_isenabler ) + goto err_free; + + gc->gicd_isactiver =3D xzalloc_array(uint32_t, DIV_ROUND_UP(n, 32)); + if ( !gc->gicd_isactiver ) + goto err_free; + + gc->gicd_itargetsr =3D xzalloc_array(uint32_t, DIV_ROUND_UP(n, 4)); + if ( !gc->gicd_itargetsr ) + goto err_free; + + gc->gicd_ipriorityr =3D xzalloc_array(uint32_t, DIV_ROUND_UP(n, 4)); + if ( !gc->gicd_ipriorityr ) + goto err_free; + + gc->gicd_icfgr =3D xzalloc_array(uint32_t, DIV_ROUND_UP(n, 16)); + if ( !gc->gicd_icfgr ) + goto err_free; + + return; + + err_free: + dprintk(XENLOG_ERR, + "%s:%d: failed to allocate memory for GICv2 suspend context\n", + __func__, __LINE__); + + xfree(gc->gicd_icfgr); + xfree(gc->gicd_ipriorityr); + xfree(gc->gicd_itargetsr); + xfree(gc->gicd_isactiver); + xfree(gc->gicd_isenabler); + + memset(gc, 0, sizeof(*gc)); +} + +#endif /* CONFIG_SYSTEM_SUSPEND */ + #ifdef CONFIG_ACPI static unsigned long gicv2_get_hwdom_extra_madt_size(const struct domain *= d) { @@ -1302,6 +1447,11 @@ static int __init gicv2_init(void) =20 spin_unlock(&gicv2.lock); =20 +#ifdef CONFIG_SYSTEM_SUSPEND + /* Allocate memory to be used for saving GIC context during the suspen= d */ + gicv2_alloc_context(&gicv2_context); +#endif /* CONFIG_SYSTEM_SUSPEND */ + return 0; } =20 @@ -1345,6 +1495,10 @@ static const struct gic_hw_operations gicv2_ops =3D { .map_hwdom_extra_mappings =3D gicv2_map_hwdom_extra_mappings, .iomem_deny_access =3D gicv2_iomem_deny_access, .do_LPI =3D gicv2_do_LPI, +#ifdef CONFIG_SYSTEM_SUSPEND + .suspend =3D gicv2_suspend, + .resume =3D gicv2_resume, +#endif /* CONFIG_SYSTEM_SUSPEND */ }; =20 /* Set up the GIC */ diff --git a/xen/arch/arm/gic.c b/xen/arch/arm/gic.c index e80fe0ca24..a018bd7715 100644 --- a/xen/arch/arm/gic.c +++ b/xen/arch/arm/gic.c @@ -425,6 +425,35 @@ int gic_iomem_deny_access(struct domain *d) return gic_hw_ops->iomem_deny_access(d); } =20 +#ifdef CONFIG_SYSTEM_SUSPEND + +int gic_suspend(void) +{ + /* Must be called by boot CPU#0 with interrupts disabled */ + ASSERT(!local_irq_is_enabled()); + ASSERT(!smp_processor_id()); + + if ( !gic_hw_ops->suspend || !gic_hw_ops->resume ) + return -ENOSYS; + + return gic_hw_ops->suspend(); +} + +void gic_resume(void) +{ + /* + * Must be called by boot CPU#0 with interrupts disabled after gic_sus= pend + * has returned successfully. + */ + ASSERT(!local_irq_is_enabled()); + ASSERT(!smp_processor_id()); + ASSERT(gic_hw_ops->resume); + + gic_hw_ops->resume(); +} + +#endif /* CONFIG_SYSTEM_SUSPEND */ + static int cpu_gic_callback(struct notifier_block *nfb, unsigned long action, void *hcpu) diff --git a/xen/arch/arm/include/asm/gic.h b/xen/arch/arm/include/asm/gic.h index 541f0eeb80..a706303008 100644 --- a/xen/arch/arm/include/asm/gic.h +++ b/xen/arch/arm/include/asm/gic.h @@ -280,6 +280,12 @@ extern int gicv_setup(struct domain *d); extern void gic_save_state(struct vcpu *v); extern void gic_restore_state(struct vcpu *v); =20 +#ifdef CONFIG_SYSTEM_SUSPEND +/* Suspend/resume */ +extern int gic_suspend(void); +extern void gic_resume(void); +#endif /* CONFIG_SYSTEM_SUSPEND */ + /* SGI (AKA IPIs) */ enum gic_sgi { GIC_SGI_EVENT_CHECK, @@ -395,6 +401,12 @@ struct gic_hw_operations { int (*iomem_deny_access)(struct domain *d); /* Handle LPIs, which require special handling */ void (*do_LPI)(unsigned int lpi); +#ifdef CONFIG_SYSTEM_SUSPEND + /* Save GIC configuration due to the system suspend */ + int (*suspend)(void); + /* Restore GIC configuration due to the system resume */ + void (*resume)(void); +#endif /* CONFIG_SYSTEM_SUSPEND */ }; =20 extern const struct gic_hw_operations *gic_hw_ops; --=20 2.48.1 From nobody Thu Oct 30 22:53:02 2025 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass(p=none dis=none) header.from=gmail.com ARC-Seal: i=1; a=rsa-sha256; t=1754945350; cv=none; d=zohomail.com; s=zohoarc; b=KGk00UzjXukRkTgd3xop+Fs0ML/KQJwykqo8N65PX9eIZ3XHu3Zwaix7Th3pVM9ZEFoSaCk51bRXfdykZkLHtD128tLfrSq8PugB2mFjXOAi6N/4NW/iVahkSvngeVQJpMmiXbRLfjOBA+wysg4mxLjEkii0qsgbAEUx4znXMYo= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1754945350; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=D2plmqjz1DgU+9gEpcztVWyZmRTrqXcD635KTVAFbqc=; b=hJpqjsb33StwHjBdR1aMYah+pJLwseQNQmPoec7vH7FRTe+U2u1j0sKLOsHSWg8jmrvCMfdyVj6y+f07Cg7yP6grGESu5fub0nWvczv2Hx/1s9jwwIbYGWfP5ewQ8HaBu5Pc4717WTexEE00sdtIuqQEvRHE1JGbCXf37jphcbs= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1754945350391160.9340224312108; Mon, 11 Aug 2025 13:49:10 -0700 (PDT) Received: from list by lists.xenproject.org with outflank-mailman.1077946.1438976 (Exim 4.92) (envelope-from ) id 1ulZRv-0006K9-IA; Mon, 11 Aug 2025 20:48:55 +0000 Received: by outflank-mailman (output) from mailman id 1077946.1438976; Mon, 11 Aug 2025 20:48:55 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1ulZRv-0006K0-F1; Mon, 11 Aug 2025 20:48:55 +0000 Received: by outflank-mailman (input) for mailman id 1077946; Mon, 11 Aug 2025 20:48:54 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1ulZRu-00063z-Mh for xen-devel@lists.xenproject.org; Mon, 11 Aug 2025 20:48:54 +0000 Received: from mail-ej1-x636.google.com (mail-ej1-x636.google.com [2a00:1450:4864:20::636]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id 9725a984-76f4-11f0-b898-0df219b8e170; Mon, 11 Aug 2025 22:48:53 +0200 (CEST) Received: by mail-ej1-x636.google.com with SMTP id a640c23a62f3a-af9611d8ff7so848816366b.1 for ; Mon, 11 Aug 2025 13:48:53 -0700 (PDT) Received: from yp-VivoBook-ASUSLaptop-M1503QA-M1503QA.Dlink ([77.52.179.38]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-af91a0a3bd2sm2065710466b.54.2025.08.11.13.48.50 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 11 Aug 2025 13:48:50 -0700 (PDT) X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 9725a984-76f4-11f0-b898-0df219b8e170 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1754945332; x=1755550132; darn=lists.xenproject.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=D2plmqjz1DgU+9gEpcztVWyZmRTrqXcD635KTVAFbqc=; b=b4e03BYvNz0GSvEHkNtxhAhwhKHTgZNlnGygFoDqwrNdcRUME8jyDl5OXg+W108l5Z S21DgqtZC/6iZQbQ3Fzph3N4c3kSNkSVF8/WQANxGxfHl11SXs4isiHLDNTvJReBKKN0 VCXWOfo0alXm1yeU/tiIiIRUweSg7p7ubEGC2mETDemZKh6q0cXWbj+I9tTlBgWXT1Dx VgL9X31AHHFn+9P8Iqv55h225yZ4omvcWZRkDHXZlMavSfbdbskX+P8+1ARumwquyOQx tBrWtgnu90N4OP5Oxa6zP3PTutMzIqDoGkLoCj+ewuNZszcEEQDRJoS/JcwPYsq7/4EJ IDhw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1754945332; x=1755550132; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=D2plmqjz1DgU+9gEpcztVWyZmRTrqXcD635KTVAFbqc=; b=Cqc/y2Cq6qs+9FE37W6xkyTC0PCaEZtnLHRu9yhZowvfpz/VhBihTQeuROS+2dQd4T FwakowQ+nbcZldVPcaXeDr79xyTTe+z3Vjis+ZMn+D+LuUayUszhUOe6/sRvv2Fciq0y /sYk/Ctpb6ptb5yx4Hc2h4l3JhzKFWnqGZ2rEyVzXR1xMSoq0ghe9BwPIsPgWaYyIKxr 3wMeByS1Plzd23FqPNJiFszKIBk9EOSb/n8IQExuUV9aZzfdjFAT6yBntmXn4zh4Ff9T CL7hFsKLiq7Wz5drt4KulD1ewuxc+/AUQs0p2d2/itaR78RMuaebpi6cRwpZnsCvVnec /8Dw== X-Gm-Message-State: AOJu0YwMZsaYet7gS1rlqP3/NnFJGM5RUxabWL8D7hOcQHbF7BwMV7Jo g1jJ1KgoEZ8YPM8P8DjBOuTqhVE9H81vZMDPSlmjJ58IUwUM6SwRu4Lb0ddf7A== X-Gm-Gg: ASbGncuUtIVWyfDaO0/cmgfjgUtj81J9GN9nd1cpoSSa6d3WeR7JDxKDUWNRvPKB5Ir mce9uMRcuO4Izqac7EOsEvH7zihJ4PNNUHH3p8D1HC2N5y9Y0lfRv3eoXw9Y0DrkwZdt/kCgYao 1QK3Aq0TqeVpg4tAb7cC/YY2oA+8ZwLJ3MoPXrMf/b1ziMoBZVvUiycyYFrESDzLKAqG1N0k3Sc Ma+TdAPoaEy3RCpl2C29NL07u2FD107ls4vI/MOc45ysTg6cIkZt7/+Pz7E1xrIsqUrvhqaP1cD H+SneS3fAxV07KB4EypGsxIaXQHX0Ylftu3XMESnClqdcK4frg0+1kQztiu2lPFwjGz7o/Nc8ho yiW+7kEDqxJiG2AhZ/S2fvFqLWnf2RjRc61yuFHUB0Fe1j6+bZHgZ1bZzFW7ZZqODw+4= X-Google-Smtp-Source: AGHT+IEfEKJYD9M3gJWf+20en40xDaPHCADL22orakBaGISswdWEmQXSl3jT0ikHDz+HYSPYdX9Vgg== X-Received: by 2002:a17:906:6b93:b0:acb:37ae:619c with SMTP id a640c23a62f3a-afa1d6d2c48mr79546266b.15.1754945331921; Mon, 11 Aug 2025 13:48:51 -0700 (PDT) From: Mykola Kvach To: xen-devel@lists.xenproject.org Cc: Mykola Kvach , Stefano Stabellini , Julien Grall , Bertrand Marquis , Michal Orzel , Volodymyr Babchuk Subject: [PATCH v5 03/12] xen/arm: gic-v3: Implement GICv3 suspend/resume functions Date: Mon, 11 Aug 2025 23:47:59 +0300 Message-ID: <451b8a0527a6193b6687e1c85bd254b4dfda142d.1754943874.git.mykola_kvach@epam.com> X-Mailer: git-send-email 2.48.1 In-Reply-To: References: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @gmail.com) X-ZM-MESSAGEID: 1754945352837116600 Content-Type: text/plain; charset="utf-8" From: Mykola Kvach System suspend may lead to a state where GIC would be powered down. Therefore, Xen should save/restore the context of GIC on suspend/resume. Note that the context consists of states of registers which are controlled by the hypervisor. Other GIC registers which are accessible by guests are saved/restored on context switch. Signed-off-by: Mykola Kvach --- xen/arch/arm/gic-v3.c | 233 ++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 233 insertions(+) diff --git a/xen/arch/arm/gic-v3.c b/xen/arch/arm/gic-v3.c index cd3e1acf79..a9b65ff5d4 100644 --- a/xen/arch/arm/gic-v3.c +++ b/xen/arch/arm/gic-v3.c @@ -1776,6 +1776,231 @@ static bool gic_dist_supports_lpis(void) return (readl_relaxed(GICD + GICD_TYPER) & GICD_TYPE_LPIS); } =20 +#ifdef CONFIG_SYSTEM_SUSPEND + +/* GICv3 registers to be saved/restored on system suspend/resume */ +struct gicv3_ctx { + struct dist_ctx { + uint32_t ctlr; + /* + * This struct represent block of 32 IRQs + * TODO: store extended SPI configuration (GICv3.1+) + */ + struct irq_regs { + uint32_t icfgr[2]; + uint32_t ipriorityr[8]; + uint64_t irouter[32]; + uint32_t isactiver; + uint32_t isenabler; + } *irqs; + } dist; + + /* have only one rdist structure for last running CPU during suspend */ + struct redist_ctx { + uint32_t ctlr; + /* TODO: handle case when we have more than 16 PPIs (GICv3.1+) */ + uint32_t icfgr[2]; + uint32_t igroupr; + uint32_t ipriorityr[8]; + uint32_t isactiver; + uint32_t isenabler; + } rdist; + + struct cpu_ctx { + uint32_t ctlr; + uint32_t pmr; + uint32_t bpr; + uint32_t sre_el2; + uint32_t grpen; + } cpu; +}; + +static struct gicv3_ctx gicv3_ctx; + +static void __init gicv3_alloc_context(void) +{ + uint32_t blocks =3D DIV_ROUND_UP(gicv3_info.nr_lines, 32); + + if ( gicv3_its_host_has_its() ) + return; + + /* according to spec it is possible don't have SPIs */ + if ( blocks =3D=3D 1 ) + return; + + gicv3_ctx.dist.irqs =3D xzalloc_array(typeof(*gicv3_ctx.dist.irqs), bl= ocks - 1); + if ( !gicv3_ctx.dist.irqs ) + dprintk(XENLOG_ERR, + "%s:%d: failed to allocate memory for GICv3 suspend contex= t\n", + __func__, __LINE__); +} + +static void gicv3_disable_redist(void) +{ + void __iomem* waker =3D GICD_RDIST_BASE + GICR_WAKER; + + writel_relaxed(readl_relaxed(waker) | GICR_WAKER_ProcessorSleep, waker= ); + while ( (readl_relaxed(waker) & GICR_WAKER_ChildrenAsleep) =3D=3D 0 ); +} + +static int gicv3_suspend(void) +{ + unsigned int i; + void __iomem *base; + typeof(gicv3_ctx.rdist)* rdist =3D &gicv3_ctx.rdist; + + /* TODO: implement support for ITS */ + if ( gicv3_its_host_has_its() ) + return -EOPNOTSUPP; + + if ( !gicv3_ctx.dist.irqs && gicv3_info.nr_lines > NR_GIC_LOCAL_IRQS ) + { + dprintk(XENLOG_WARNING, + "%s:%d: GICv3 suspend context is not allocated!\n", + __func__, __LINE__); + return -ENOMEM; + } + + gicv3_save_state(current); + + /* Save GICC configuration */ + gicv3_ctx.cpu.ctlr =3D READ_SYSREG(ICC_CTLR_EL1); + gicv3_ctx.cpu.pmr =3D READ_SYSREG(ICC_PMR_EL1); + gicv3_ctx.cpu.bpr =3D READ_SYSREG(ICC_BPR1_EL1); + gicv3_ctx.cpu.sre_el2 =3D READ_SYSREG(ICC_SRE_EL2); + gicv3_ctx.cpu.grpen =3D READ_SYSREG(ICC_IGRPEN1_EL1); + + gicv3_disable_interface(); + gicv3_disable_redist(); + + /* Save GICR configuration */ + gicv3_redist_wait_for_rwp(); + + base =3D GICD_RDIST_SGI_BASE; + + rdist->ctlr =3D readl_relaxed(base + GICR_CTLR); + + /* Set priority on PPI and SGI interrupts */ + for (i =3D 0; i < NR_GIC_LOCAL_IRQS / 4; i +=3D 4) + rdist->ipriorityr[i] =3D readl_relaxed(base + GICR_IPRIORITYR0 + 4= * i); + + rdist->isactiver =3D readl_relaxed(base + GICR_ISACTIVER0); + rdist->isenabler =3D readl_relaxed(base + GICR_ISENABLER0); + rdist->igroupr =3D readl_relaxed(base + GICR_IGROUPR0); + rdist->icfgr[0] =3D readl_relaxed(base + GICR_ICFGR0); + rdist->icfgr[1] =3D readl_relaxed(base + GICR_ICFGR1); + + /* Save GICD configuration */ + gicv3_dist_wait_for_rwp(); + gicv3_ctx.dist.ctlr =3D readl_relaxed(GICD + GICD_CTLR); + + for ( i =3D 1; i < DIV_ROUND_UP(gicv3_info.nr_lines, 32); i++ ) + { + typeof(gicv3_ctx.dist.irqs) irqs =3D gicv3_ctx.dist.irqs + i - 1; + unsigned int irq; + + base =3D GICD + GICD_ICFGR + 8 * i; + irqs->icfgr[0] =3D readl_relaxed(base); + irqs->icfgr[1] =3D readl_relaxed(base + 4); + + base =3D GICD + GICD_IPRIORITYR + 32 * i; + for ( irq =3D 0; irq < 8; irq++ ) + irqs->ipriorityr[irq] =3D readl_relaxed(base + 4 * irq); + + base =3D GICD + GICD_IROUTER + 32 * i; + for ( irq =3D 0; irq < 32; irq++ ) + irqs->irouter[irq] =3D readq_relaxed_non_atomic(base + 8 * irq= ); + + irqs->isactiver =3D readl_relaxed(GICD + GICD_ISACTIVER + 4 * i); + irqs->isenabler =3D readl_relaxed(GICD + GICD_ISENABLER + 4 * i); + } + + return 0; +} + +static void gicv3_resume(void) +{ + unsigned int i; + void __iomem *base; + typeof(gicv3_ctx.rdist)* rdist =3D &gicv3_ctx.rdist; + + if ( !gicv3_ctx.dist.irqs && gicv3_info.nr_lines > NR_GIC_LOCAL_IRQS ) + { + dprintk(XENLOG_WARNING, "%s:%d: GICv3 suspend context not allocate= d!\n", + __func__, __LINE__); + return; + } + + writel_relaxed(0, GICD + GICD_CTLR); + + for ( i =3D NR_GIC_LOCAL_IRQS; i < gicv3_info.nr_lines; i +=3D 32 ) + writel_relaxed(GENMASK(31, 0), GICD + GICD_IGROUPR + (i / 32) * 4); + + for ( i =3D 1; i < DIV_ROUND_UP(gicv3_info.nr_lines, 32); i++ ) + { + typeof(gicv3_ctx.dist.irqs) irqs =3D gicv3_ctx.dist.irqs + i - 1; + unsigned int irq; + + base =3D GICD + GICD_ICFGR + 8 * i; + writel_relaxed(irqs->icfgr[0], base); + writel_relaxed(irqs->icfgr[1], base + 4); + + base =3D GICD + GICD_IPRIORITYR + 32 * i; + for ( irq =3D 0; irq < 8; irq++ ) + writel_relaxed(irqs->ipriorityr[irq], base + 4 * irq ); + + base =3D GICD + GICD_IROUTER + 32 * i; + for ( irq =3D 0; irq < 32; irq++ ) + writeq_relaxed_non_atomic(irqs->irouter[irq], base + 8 * irq); + + writel_relaxed(irqs->isenabler, GICD + GICD_ISENABLER + i * 4); + writel_relaxed(irqs->isactiver, GICD + GICD_ISACTIVER + i * 4); + } + + writel_relaxed(gicv3_ctx.dist.ctlr, GICD + GICD_CTLR); + gicv3_dist_wait_for_rwp(); + + /* Restore GICR (Redistributor) configuration */ + gicv3_enable_redist(); + + base =3D GICD_RDIST_SGI_BASE; + + writel_relaxed(0xffffffff, base + GICR_ICENABLER0); + gicv3_redist_wait_for_rwp(); + + for (i =3D 0; i < NR_GIC_LOCAL_IRQS / 4; i +=3D 4) + writel_relaxed(rdist->ipriorityr[i], base + GICR_IPRIORITYR0 + i *= 4); + + writel_relaxed(rdist->isactiver, base + GICR_ISACTIVER0); + + writel_relaxed(rdist->igroupr, base + GICR_IGROUPR0); + writel_relaxed(rdist->icfgr[0], base + GICR_ICFGR0); + writel_relaxed(rdist->icfgr[1], base + GICR_ICFGR1); + + gicv3_redist_wait_for_rwp(); + + writel_relaxed(rdist->isenabler, base + GICR_ISENABLER0); + writel_relaxed(rdist->ctlr, GICD_RDIST_BASE + GICR_CTLR); + + gicv3_redist_wait_for_rwp(); + + WRITE_SYSREG(gicv3_ctx.cpu.sre_el2, ICC_SRE_EL2); + isb(); + + /* Restore CPU interface (System registers) */ + WRITE_SYSREG(gicv3_ctx.cpu.pmr, ICC_PMR_EL1); + WRITE_SYSREG(gicv3_ctx.cpu.bpr, ICC_BPR1_EL1); + WRITE_SYSREG(gicv3_ctx.cpu.ctlr, ICC_CTLR_EL1); + WRITE_SYSREG(gicv3_ctx.cpu.grpen, ICC_IGRPEN1_EL1); + isb(); + + gicv3_hyp_init(); + + gicv3_restore_state(current); +} + +#endif /* CONFIG_SYSTEM_SUSPEND */ + /* Set up the GIC */ static int __init gicv3_init(void) { @@ -1850,6 +2075,10 @@ static int __init gicv3_init(void) =20 gicv3_hyp_init(); =20 +#ifdef CONFIG_SYSTEM_SUSPEND + gicv3_alloc_context(); +#endif + out: spin_unlock(&gicv3.lock); =20 @@ -1889,6 +2118,10 @@ static const struct gic_hw_operations gicv3_ops =3D { #endif .iomem_deny_access =3D gicv3_iomem_deny_access, .do_LPI =3D gicv3_do_LPI, +#ifdef CONFIG_SYSTEM_SUSPEND + .suspend =3D gicv3_suspend, + .resume =3D gicv3_resume, +#endif }; =20 static int __init gicv3_dt_preinit(struct dt_device_node *node, const void= *data) --=20 2.48.1 From nobody Thu Oct 30 22:53:02 2025 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass(p=none dis=none) header.from=gmail.com ARC-Seal: i=1; a=rsa-sha256; t=1754945352; cv=none; d=zohomail.com; s=zohoarc; b=L5RjV0Cd2IlK7dEQP5MZhBPIqSf7GmTOzIRFR9v8JEj/FNBKyti5hMujsnE3agF7Z3R1TeU7tfTMA11QjceX2dZNw4eYl+TtDa2yWgwdSPiBs3eFD6HKbXivp4/KKvPXebfNic2UOoojTX1cdmbKcwd2UaxsYWoTeoA1zeOlUb0= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1754945352; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=t3qd7wn/+BTaUGKnbQ/zLRIfyUfmlIN1J0Bwt1qcE10=; b=SGLkhibzw+3t0cuES3EEa5NfKe0c+rNgh0fV5UPx/Uzog1PMME8c+THJMKjw8xbK/Qw35rMyYAv3e0iBMDHkRg51wVeH2lHOyplSRNEqZPTHs5yQBdieraKcrqI+3WSvxTPdrz7YwqwDR7B0tn1xyDehjpqqhuIspbtTyHnsJxY= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1754945352329202.6137577482915; Mon, 11 Aug 2025 13:49:12 -0700 (PDT) Received: from list by lists.xenproject.org with outflank-mailman.1077947.1438986 (Exim 4.92) (envelope-from ) id 1ulZRw-0006Zt-Qr; Mon, 11 Aug 2025 20:48:56 +0000 Received: by outflank-mailman (output) from mailman id 1077947.1438986; Mon, 11 Aug 2025 20:48:56 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1ulZRw-0006Zg-ND; Mon, 11 Aug 2025 20:48:56 +0000 Received: by outflank-mailman (input) for mailman id 1077947; Mon, 11 Aug 2025 20:48:55 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1ulZRv-0005co-NW for xen-devel@lists.xenproject.org; Mon, 11 Aug 2025 20:48:55 +0000 Received: from mail-ed1-x52c.google.com (mail-ed1-x52c.google.com [2a00:1450:4864:20::52c]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id 9863a886-76f4-11f0-a327-13f23c93f187; Mon, 11 Aug 2025 22:48:55 +0200 (CEST) Received: by mail-ed1-x52c.google.com with SMTP id 4fb4d7f45d1cf-61592ff5ebbso7470419a12.3 for ; Mon, 11 Aug 2025 13:48:55 -0700 (PDT) Received: from yp-VivoBook-ASUSLaptop-M1503QA-M1503QA.Dlink ([77.52.179.38]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-af91a0a3bd2sm2065710466b.54.2025.08.11.13.48.52 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 11 Aug 2025 13:48:53 -0700 (PDT) X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 9863a886-76f4-11f0-a327-13f23c93f187 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1754945334; x=1755550134; darn=lists.xenproject.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=t3qd7wn/+BTaUGKnbQ/zLRIfyUfmlIN1J0Bwt1qcE10=; b=kB1jbKl/DEHDketgCQPVFERAjE0SPcL4FMHCQy5fbVRw5LmrtsnyRO09jBUxa8Lw2j 6cedYJTwrVdfJ3EGFyu9+bfNbOO77qvaizD9BkkZZpzyR6Iz8Xv3a1rJOG0DXlWNo6Py ZR6V+iGxoXFv36zzKVIZxAKjEc1XurqSn2CBou86iDCbDsbLsdW9mFRiSQ3w0T6WWrK/ pHgqox5ueR8i1JAjTnaYuKY/hNNDRivpurbhGTlha4A2DksykKx3YYOKg1mWz82ThmrF 4ZXcGFXPXbESUal9r8RDMqba7SUywDq02PVOwz513H6tFQr3Mu4xvS5wbdQSrbOZyjm7 1KrQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1754945334; x=1755550134; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=t3qd7wn/+BTaUGKnbQ/zLRIfyUfmlIN1J0Bwt1qcE10=; b=EPL/o3iYrKqiEthiryZKv0t/cBsGwjPMgaMx6O8uC9mFSdgjMFNjMF18L2hyiMnTOz hTl+pBNuazZyEm2MxVLKhL2v6nwtgOTaweunBFfrzSQWrIzeJoDa4DsnmsR829ABj2fR 4ybWVi/GwPuEGtDblwUveu/Xhv03Nd/Q8t/o8ui7MWmJxtqAclhCIHNHlHgOw9xs8nCN ai//jb0/lmweQ96LexpMjDk3FNXf1HJzn9AuAJoBfJEeL4MXzEnfWAsZocPSU6SawJ6g 8kjWMfzoLZmWkBwUw9s3Ego3KYL3ljBAB4J4yX9xHFVFVaIA5sEetAscWqUauK21VC51 bLVQ== X-Gm-Message-State: AOJu0Yw3dkpAqXNIjKBBgTZ6VhqzI5YAVd0S24i1NjA9YP9MAyekrw4i Kx6UyenSfyvNyNR+lqYUgF+e/f0xlsMX8AmRuxIVLVyJpJlXpnjnIh9yJJQQXA== X-Gm-Gg: ASbGncvJbAAQ2sh/Tqv70UkTLjCU3M/KVB1L/gX6MyuMvBo5nPXCBueSXE79U/Xyuak CHUeiNELwvjXojT6Y5AOG7i41iHW1kNliPNhOw1pE8pzB9VzdMFx3wOlDb0wOydFyLlnkR+y5Br 0HKH9pZuF7Bd9DOB0BskOFru8suJNzfsILKIotGeMGiLuI1X9pUaOi1j2/M1a8TrDn4tXCWR1Xi nWudb40ViLxL4CKmSlbj3HCEJT6qt5g44osOL32XbNkvGPPuLBz78z0C/c+mn6SWBmTvkKWiLVb 5VZgpsY1Xgbux4hGYrlHgqRaYzhfBuZujodQrx1W3wx41GvkaMV7HeGdRxRcdJtbr0zQtA+gBNW dj6oPJdJ2hkrJydp3b6dOjxXRp/aYCyGhzGM9JeVEpyl2bDVUT94qhxUKIbARiGBkdLg= X-Google-Smtp-Source: AGHT+IHyc4RdgDDPGpJFPoO/+5S4rhYm2JlYjM24gD7Vc95QPmXWGO9du6N0t/9Ny5v0LhPnvvAfWA== X-Received: by 2002:a17:907:60cc:b0:ae3:a812:a780 with SMTP id a640c23a62f3a-afa1e1ed74fmr73582466b.61.1754945333973; Mon, 11 Aug 2025 13:48:53 -0700 (PDT) From: Mykola Kvach To: xen-devel@lists.xenproject.org Cc: Mykola Kvach , Stefano Stabellini , Julien Grall , Bertrand Marquis , Michal Orzel , Volodymyr Babchuk Subject: [PATCH v5 04/12] xen/arm: Prevent crash during disable_nonboot_cpus on suspend Date: Mon, 11 Aug 2025 23:48:00 +0300 Message-ID: <98957da5c5068ae8340a21a9aa15a962905a8a22.1754943874.git.mykola_kvach@epam.com> X-Mailer: git-send-email 2.48.1 In-Reply-To: References: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @gmail.com) X-ZM-MESSAGEID: 1754945353786124100 Content-Type: text/plain; charset="utf-8" From: Mykola Kvach If we call disable_nonboot_cpus on ARM64 with system_state set to SYS_STATE_suspend, the following assertion will be triggered: ``` (XEN) [ 25.582712] Disabling non-boot CPUs ... (XEN) [ 25.587032] Assertion '!in_irq() && (local_irq_is_enabled() || num= _online_cpus() <=3D 1)' failed at common/xmalloc_tlsf.c:714 [...] (XEN) [ 25.975069] Xen call trace: (XEN) [ 25.978353] [<00000a000022e098>] xfree+0x130/0x1a4 (PC) (XEN) [ 25.984314] [<00000a000022e08c>] xfree+0x124/0x1a4 (LR) (XEN) [ 25.990276] [<00000a00002747d4>] release_irq+0xe4/0xe8 (XEN) [ 25.996152] [<00000a0000278588>] time.c#cpu_time_callback+0x44/= 0x60 (XEN) [ 26.003150] [<00000a000021d678>] notifier_call_chain+0x7c/0xa0 (XEN) [ 26.009717] [<00000a00002018e0>] cpu.c#cpu_notifier_call_chain+= 0x24/0x48 (XEN) [ 26.017148] [<00000a000020192c>] cpu.c#_take_cpu_down+0x28/0x34 (XEN) [ 26.023801] [<00000a0000201944>] cpu.c#take_cpu_down+0xc/0x18 (XEN) [ 26.030281] [<00000a0000225c5c>] stop_machine.c#stopmachine_act= ion+0xbc/0xe4 (XEN) [ 26.038057] [<00000a00002264bc>] tasklet.c#do_tasklet_work+0xb8= /0x100 (XEN) [ 26.045229] [<00000a00002268a4>] do_tasklet+0x68/0xb0 (XEN) [ 26.051018] [<00000a000026e120>] domain.c#idle_loop+0x7c/0x194 (XEN) [ 26.057585] [<00000a0000277e30>] start_secondary+0x21c/0x220 (XEN) [ 26.063978] [<00000a0000361258>] 00000a0000361258 ``` This happens because before invoking take_cpu_down via the stop_machine_run function on the target CPU, stop_machine_run requests the STOPMACHINE_DISABLE_IRQ state on that CPU. Releasing memory in the release_irq function then triggers the assertion: /* * Heap allocations may need TLB flushes which may require IRQs to be * enabled (except when only 1 PCPU is online). */ This patch adds system state checks to guard calls to request_irq and release_irq. These calls are now skipped when system_state is SYS_STATE_{resume,suspend}, preventing unsafe operations during suspend/resume handling. Signed-off-by: Mykola Kvach --- Changes in V4: - removed the prior tasklet-based workaround in favor of a more straightforward and safer solution - reworked the approach by adding explicit system state checks around request_irq and release_irq calls, skips these calls during suspend and resume states to avoid unsafe memory operations when IRQs are disabled --- xen/arch/arm/gic.c | 6 ++++++ xen/arch/arm/tee/ffa_notif.c | 2 +- xen/arch/arm/time.c | 18 ++++++++++++------ 3 files changed, 19 insertions(+), 7 deletions(-) diff --git a/xen/arch/arm/gic.c b/xen/arch/arm/gic.c index a018bd7715..9856cb1592 100644 --- a/xen/arch/arm/gic.c +++ b/xen/arch/arm/gic.c @@ -388,6 +388,9 @@ void gic_dump_info(struct vcpu *v) =20 void init_maintenance_interrupt(void) { + if ( system_state =3D=3D SYS_STATE_resume ) + return; + request_irq(gic_hw_ops->info->maintenance_irq, 0, maintenance_interrup= t, "irq-maintenance", NULL); } @@ -461,6 +464,9 @@ static int cpu_gic_callback(struct notifier_block *nfb, switch ( action ) { case CPU_DYING: + if ( system_state =3D=3D SYS_STATE_suspend ) + break; + /* This is reverting the work done in init_maintenance_interrupt */ release_irq(gic_hw_ops->info->maintenance_irq, NULL); break; diff --git a/xen/arch/arm/tee/ffa_notif.c b/xen/arch/arm/tee/ffa_notif.c index 00efaf8f73..06f715a82b 100644 --- a/xen/arch/arm/tee/ffa_notif.c +++ b/xen/arch/arm/tee/ffa_notif.c @@ -347,7 +347,7 @@ void ffa_notif_init_interrupt(void) { int ret; =20 - if ( notif_enabled && notif_sri_irq < NR_GIC_SGI ) + if ( notif_enabled && notif_sri_irq < NR_GIC_SGI && system_state !=3D = SYS_STATE_resume ) { /* * An error here is unlikely since the primary CPU has already diff --git a/xen/arch/arm/time.c b/xen/arch/arm/time.c index ad984fdfdd..b2e07ade43 100644 --- a/xen/arch/arm/time.c +++ b/xen/arch/arm/time.c @@ -320,10 +320,13 @@ void init_timer_interrupt(void) WRITE_SYSREG(CNTHCTL_EL2_EL1PCTEN, CNTHCTL_EL2); disable_physical_timers(); =20 - request_irq(timer_irq[TIMER_HYP_PPI], 0, htimer_interrupt, - "hyptimer", NULL); - request_irq(timer_irq[TIMER_VIRT_PPI], 0, vtimer_interrupt, - "virtimer", NULL); + if ( system_state !=3D SYS_STATE_resume ) + { + request_irq(timer_irq[TIMER_HYP_PPI], 0, htimer_interrupt, + "hyptimer", NULL); + request_irq(timer_irq[TIMER_VIRT_PPI], 0, vtimer_interrupt, + "virtimer", NULL); + } =20 check_timer_irq_cfg(timer_irq[TIMER_HYP_PPI], "hypervisor"); check_timer_irq_cfg(timer_irq[TIMER_VIRT_PPI], "virtual"); @@ -338,8 +341,11 @@ static void deinit_timer_interrupt(void) { disable_physical_timers(); =20 - release_irq(timer_irq[TIMER_HYP_PPI], NULL); - release_irq(timer_irq[TIMER_VIRT_PPI], NULL); + if ( system_state !=3D SYS_STATE_suspend ) + { + release_irq(timer_irq[TIMER_HYP_PPI], NULL); + release_irq(timer_irq[TIMER_VIRT_PPI], NULL); + } } =20 /* Wait a set number of microseconds */ --=20 2.48.1 From nobody Thu Oct 30 22:53:02 2025 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass(p=none dis=none) header.from=gmail.com ARC-Seal: i=1; a=rsa-sha256; t=1754945355; cv=none; d=zohomail.com; s=zohoarc; b=Ybzzwvor1lXbf1wMslpFJLq7fVowhnoayonc5Wegi83qU8DuU7YVg3N7g5+LJjHrwVRLsMuHw9hIthItyeTlYQGGyOds8O+RHO6PWCabeqXXo5tjLnvXxwyTY03uLI4dg2OCQNrks+Tzk+2eYoTs0K2j1O74xplJA8wbMu9OdZs= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1754945355; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=qwUl1czZ2m+Uelj6p4EfRluuF8y733PPdTpEjfbyiwk=; b=Yh9jRff0aCoIiOUaLWPcIqYgq0N6f15htdg5JZ8lE1BTYZNjzf06VAqq99ZfYmuvD1k/hzzEUnxOwJE8tTPTPSbbttJYBwEDacoc/WzVCKMkY9z88zEdj2ckHUImcMJUEIrN7ZK2mZ6ZGg193GuOW/UppAyZIhIqnSff/uBx1oc= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1754945355211817.5745334748026; Mon, 11 Aug 2025 13:49:15 -0700 (PDT) Received: from list by lists.xenproject.org with outflank-mailman.1077949.1438995 (Exim 4.92) (envelope-from ) id 1ulZS1-0006xq-6A; Mon, 11 Aug 2025 20:49:01 +0000 Received: by outflank-mailman (output) from mailman id 1077949.1438995; Mon, 11 Aug 2025 20:49:01 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1ulZS1-0006xe-2Q; Mon, 11 Aug 2025 20:49:01 +0000 Received: by outflank-mailman (input) for mailman id 1077949; Mon, 11 Aug 2025 20:48:59 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1ulZRz-00063z-Gn for xen-devel@lists.xenproject.org; Mon, 11 Aug 2025 20:48:59 +0000 Received: from mail-ed1-x52d.google.com (mail-ed1-x52d.google.com [2a00:1450:4864:20::52d]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id 9976de08-76f4-11f0-b898-0df219b8e170; Mon, 11 Aug 2025 22:48:56 +0200 (CEST) Received: by mail-ed1-x52d.google.com with SMTP id 4fb4d7f45d1cf-615756b1e99so6565677a12.0 for ; Mon, 11 Aug 2025 13:48:56 -0700 (PDT) Received: from yp-VivoBook-ASUSLaptop-M1503QA-M1503QA.Dlink ([77.52.179.38]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-af91a0a3bd2sm2065710466b.54.2025.08.11.13.48.54 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 11 Aug 2025 13:48:55 -0700 (PDT) X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 9976de08-76f4-11f0-b898-0df219b8e170 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1754945336; x=1755550136; darn=lists.xenproject.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=qwUl1czZ2m+Uelj6p4EfRluuF8y733PPdTpEjfbyiwk=; b=TchvzPdgOoqkdhKVvUH3VzjSmn/APp2a3PgqK8vBlUV8WPaGAAt2/CNXX3eqC5yVQT TWHezV1jD2Au4y+rtw8Ntb0fXIbCWMYGCnzXGVCxjL4bHxEa0X8pohQzhKzMyeZtSiUs 7RL18jQOeJpfLZghAKHdi53hrQHEDMjg5fEhWfhy+MK5uvsHRcE1lgaVIzuaqgveLpre j7xeIF5s4S1dHd32rsefj9bzAR/VmldiMOzk7KZtTajoTYjXg2FDxDIO8ftvH1CKUtzO ZwVNxzJ7QVVt9IAf3w7OpCp4LMkmJDOU0DYg2oipg7BwZcSBkEaXVJ0RRORmrF0RD5dZ Tcww== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1754945336; x=1755550136; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=qwUl1czZ2m+Uelj6p4EfRluuF8y733PPdTpEjfbyiwk=; b=smx45UGZjmmQGHegnYBN7CDPjL4D67Tm1e4hOQxxTbXTwOcv6ecPZdy6yknIikFv/v w5AM/fyOB2PMbx++Lc0sX+VpfxPzuIy+ig5LVdm74oagl60i7CleSGlppPxgIwGBz/Rr PYYGcjTHgYqyF8GRmlOWp0dgNJKIna2U8mF2/4H9ehi6dp+qZdxGch2UR3LUCHlLrzBZ +q7tEjdjJu3gg3KL4yi365jKubAYsLr0KNrdH4u0zrF2m6rfoFzGPDgOMhyFUNfqCWFD 499QsNGV8QEZxeRoi/VLH67qPxkbHHNp7byNplfwOItLcWkc1/XPETkJRoayn3ELesuw 8QMQ== X-Gm-Message-State: AOJu0YzLPkbmRnLrJQ18h0FkZhIAwnliAZh//VNFsRhoGR1h7OpHT07w BNKb0niY3TavQsE/hhS+5nZJc7Muoy5wM/zgmEBpxrYqFxwiolTPWfsI9LW0/g== X-Gm-Gg: ASbGncvjcTowBG92vD3LlO9u/w7K4XK2e7RnjmkbDkc8UhDtux9YjYIFLLW3keFFVwo td/Lz2om7ESb5mx20Y6FfDdRsxD5kWADCXo3sJMC1f1LQf1Xumuj0fU2uOJADtVQtjKHYTSgjpH iHiorIH/RBYw/CjLLBlE1HDYfQjiGRcRRetTXWcNW1Vy2g6HmD/L4n6JNgRr0ONNbhVwFJHSLha ATJNyMl6Lx++MlvEpGtLUHwkc8vtBA5pC20alE3KqPOxtKah6lHy8srm96tZfZWc7Q3MnAct4OF yVfWn0+Yc8/S1VX5fR6QrNCeCdhL88NssKfRAFxr6uPVLPjHanlPTH/eWVptF8QkMN1SRY/Oh9t YMMLhR6SRllUQe3/G72HhKQMd9ZMXTFNS3m/XFBuw9Kmj+cKnW5880L7bSLClYToZEDS6tv65jB /n8w== X-Google-Smtp-Source: AGHT+IFO5hq4ySBJNwPMmfUKsL09AJOr0uCUIAwDZZTQIhw7zIqyOTdwwRDDjdCGC7eAi/wFj/9Hsg== X-Received: by 2002:a17:907:c25:b0:ade:316e:bfc with SMTP id a640c23a62f3a-af9c6457eb7mr1272700166b.21.1754945335933; Mon, 11 Aug 2025 13:48:55 -0700 (PDT) From: Mykola Kvach To: xen-devel@lists.xenproject.org Cc: Mykola Kvach , Stefano Stabellini , Julien Grall , Bertrand Marquis , Michal Orzel , Volodymyr Babchuk Subject: [PATCH v5 05/12] xen/arm: irq: avoid local IRQ descriptors reinit on system resume Date: Mon, 11 Aug 2025 23:48:01 +0300 Message-ID: <7686bc019f0c0eff94720e4ca16e00f895819087.1754943874.git.mykola_kvach@epam.com> X-Mailer: git-send-email 2.48.1 In-Reply-To: References: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @gmail.com) X-ZM-MESSAGEID: 1754945355706124100 Content-Type: text/plain; charset="utf-8" From: Mykola Kvach On ARM, during system resume, CPUs are brought online again. This normally triggers init_local_irq_data, which reinitializes IRQ descriptors for banked interrupts (SGIs and PPIs). These descriptors are statically allocated per CPU and retain valid state across suspend/resume cycles. Re-initializing them on resume is unnecessary and may result in loss of interrupt configuration or restored state. This patch skips init_local_irq_data when system_state is set to SYS_STATE_resume to preserve banked IRQ descs state during resume. Signed-off-by: Mykola Kvach Reviewed-by: Volodymyr Babchuk --- xen/arch/arm/irq.c | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/xen/arch/arm/irq.c b/xen/arch/arm/irq.c index 4bbf0b0664..148f184f8b 100644 --- a/xen/arch/arm/irq.c +++ b/xen/arch/arm/irq.c @@ -122,6 +122,10 @@ static int cpu_callback(struct notifier_block *nfb, un= signed long action, switch ( action ) { case CPU_UP_PREPARE: + /* Skip local IRQ cleanup on resume */ + if ( system_state =3D=3D SYS_STATE_resume ) + break; + rc =3D init_local_irq_data(cpu); if ( rc ) printk(XENLOG_ERR "Unable to allocate local IRQ for CPU%u\n", --=20 2.48.1 From nobody Thu Oct 30 22:53:02 2025 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass(p=none dis=none) header.from=gmail.com ARC-Seal: i=1; a=rsa-sha256; t=1754945361; cv=none; d=zohomail.com; s=zohoarc; b=NQutwhb3CYmvE1D5sKEK7A+K1MF3Bx/BuAixBWMZGUhFKx7K8rilPDZn52m4mGuCXaoXR1dXW5JpLhQWMFASYbcQVUrkTMgI79NbOdI9VY2vr+zUruFEKoVHd2HQUqrPdB/6D96zSQ0i7DXR5d56OBrrglWJbBCCGUZtIRqZKTg= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1754945361; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=Q2qe+4To2d8/snjQhUCMC9AqGLOoTxivgLe2IekgTVE=; b=ePlnsqEbqel1DGwmciHAuloTyACKDLBRfiRI7iZI5u2yq606Uv9XU8dUvkCfIv3lsMmj9linFlEiGlRE/6Vkp5lg0EX8UHLXSPDz1uknp0oZkMoIwlJW8STyk43XGXp+hnQ4L9KjM8fDD3W3Vg9ZRsVAngEONIpuHXI5+z0XUCo= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1754945361001118.69890312664336; Mon, 11 Aug 2025 13:49:21 -0700 (PDT) Received: from list by lists.xenproject.org with outflank-mailman.1077950.1439000 (Exim 4.92) (envelope-from ) id 1ulZS1-00071r-HY; Mon, 11 Aug 2025 20:49:01 +0000 Received: by outflank-mailman (output) from mailman id 1077950.1439000; Mon, 11 Aug 2025 20:49:01 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1ulZS1-00070H-C1; Mon, 11 Aug 2025 20:49:01 +0000 Received: by outflank-mailman (input) for mailman id 1077950; Mon, 11 Aug 2025 20:49:00 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1ulZS0-00063z-H7 for xen-devel@lists.xenproject.org; Mon, 11 Aug 2025 20:49:00 +0000 Received: from mail-ej1-x635.google.com (mail-ej1-x635.google.com [2a00:1450:4864:20::635]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id 9a5429d9-76f4-11f0-b898-0df219b8e170; Mon, 11 Aug 2025 22:48:58 +0200 (CEST) Received: by mail-ej1-x635.google.com with SMTP id a640c23a62f3a-ae0bde4d5c9so907144566b.3 for ; Mon, 11 Aug 2025 13:48:58 -0700 (PDT) Received: from yp-VivoBook-ASUSLaptop-M1503QA-M1503QA.Dlink ([77.52.179.38]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-af91a0a3bd2sm2065710466b.54.2025.08.11.13.48.56 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 11 Aug 2025 13:48:56 -0700 (PDT) X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 9a5429d9-76f4-11f0-b898-0df219b8e170 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1754945337; x=1755550137; darn=lists.xenproject.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Q2qe+4To2d8/snjQhUCMC9AqGLOoTxivgLe2IekgTVE=; b=jQejmo/yC4pihO7Ag8Fsjgj6EbaWTt8Sr6ami7kPJBsBBwB+ebKFLSHtW5hUIVuROh aqNIc1lYkQntdVqwB10pIVjsxwafjbadTpdqjhid5Xxwv1KDaTj6q/L5qXByRD6btf8R batDYLAReobfrAPW7QWHQh+9myXFYwsDl0tB44FDeHhIf6dzXNCDkGbbSz+IjnGV8Xcg tOQ75CNO9LcaO4EN3GeRbSuv5//+VdSgcOejGEHXUV4lPgDSM4sg/X3JCKQKy3rbmHgO tbadLq2pBSfm9NOijkk2DB6g+a+Y1hR+bTu3h6CVc7T2B1DF93Z3+i4XCpEi/PmzDUCf 9ibA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1754945337; x=1755550137; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Q2qe+4To2d8/snjQhUCMC9AqGLOoTxivgLe2IekgTVE=; b=rGUWrlru8UPN0Mct1LSyg1D1+JNJT+j4WJXRpYockoxaVwViMg5QLu9BaFJZEsE5y8 0Jo3FQCYFjX7svfhIVtxt9IJLuxjel7iSMYuaGL5P2b3mIGbnNe+XwTx5D5cXR2skX38 DQLVQoJ1jsqWDAYmF9zm6cVvbMXjBsI6GQnRbpRawBlONEP8UKHkxZ3zE/SqgU7H/UO3 3AjAi6PhK7MvnGo1yhLIGr3xWfCglAwi34GyHzHFV8wY9E0fJas+xXTCTSeDcYakorf3 6SQwVQPrBeKqLIvIMtacBfJbTniYPYrYZQ+zmE0H8ADfeMoFmUqk9dfROx7GL1VMwMD/ S6YQ== X-Gm-Message-State: AOJu0YygocUMG7P2N44Bpi6A/6laG1PYY7/3GFuVyIm+EtKQ9ulorEz2 D/tb2o8TooSxSGbt8lHorRj4YW2zs6ZQGyTrk4AbhWn0S1W8es7ncTLw38Xwkg== X-Gm-Gg: ASbGncsQecLd4ZPkEGmy1z0dU1m28+xUBPgKH/dId39BEcQIoxigwhOPTqaAD6eaQBw 9XN+bX3qBQd68XJLzF/YyGLT72dutEhf6WV6MjzjM2pRaTLkf+dmgXTuQcwOP9BI+Eq3fK0bbOg syBDEDH9nq4I+qC2B3AevWNsvaiOpcJWFDRXbFSZJE9K5KYzRgHjcTLSSyxEjZwJYxOg7UAe5PI QjktrgSsYexGn7WEpbB7VBnyxdGEpWGuAZNtAGDIeKldGxndAW8wHJ3QMWrgaFLi4LNNqC2+sT+ ICF3V77WrUxw/XkFc4q/dkKGDoEz15CjXwXn7upM2xn6U2yDNbK1S7baIAr8oKiY78AwkP+bj8U YWA2zw5KgWt+iQHnX5amSoCQPDu6NuhMfSPdzWoY6nlb134jA2QOQ3T+ZosYOxtWv3C4= X-Google-Smtp-Source: AGHT+IHdCavzk8aGLAAEau1dbT80tWOU/vCYdkrblwV5rQHlaJaQKN+3lli5Az81ynaQ9zYS83kj3g== X-Received: by 2002:a17:906:f59f:b0:ad5:2e5b:d16b with SMTP id a640c23a62f3a-afa1e0290bcmr80256266b.27.1754945337329; Mon, 11 Aug 2025 13:48:57 -0700 (PDT) From: Mykola Kvach To: xen-devel@lists.xenproject.org Cc: Mykola Kvach , Stefano Stabellini , Julien Grall , Bertrand Marquis , Michal Orzel , Volodymyr Babchuk Subject: [PATCH v5 06/12] xen/arm: irq: Restore state of local IRQs during system resume Date: Mon, 11 Aug 2025 23:48:02 +0300 Message-ID: <779a90853f249f8d8c4973874baeee1f787e92e0.1754943875.git.mykola_kvach@epam.com> X-Mailer: git-send-email 2.48.1 In-Reply-To: References: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @gmail.com) X-ZM-MESSAGEID: 1754945362606116600 Content-Type: text/plain; charset="utf-8" From: Mykola Kvach On ARM, the first 32 interrupts (SGIs and PPIs) are banked per-CPU and not restored by gic_resume (for secondary cpus). This patch introduces restore_local_irqs_on_resume, a function that restores the state of local interrupts on the target CPU during system resume. It iterates over all local IRQs and re-enables those that were not disabled, reprogramming their routing and affinity accordingly. The function is invoked from start_secondary, ensuring that local IRQ state is restored early during CPU bring-up after suspend. Signed-off-by: Mykola Kvach --- xen/arch/arm/irq.c | 44 ++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 44 insertions(+) diff --git a/xen/arch/arm/irq.c b/xen/arch/arm/irq.c index 148f184f8b..b3ff38dc27 100644 --- a/xen/arch/arm/irq.c +++ b/xen/arch/arm/irq.c @@ -113,6 +113,47 @@ static int init_local_irq_data(unsigned int cpu) return 0; } =20 +/* + * The first 32 interrupts (PPIs and SGIs) are per-CPU, + * so call this function on the target CPU to restore them. + * + * SPIs are restored via gic_resume. + */ +static void restore_local_irqs_on_resume(void) +{ + int irq; + + if ( system_state !=3D SYS_STATE_resume ) + return; + + spin_lock(&local_irqs_type_lock); + + for ( irq =3D 0; irq < NR_LOCAL_IRQS; irq++ ) + { + struct irq_desc *desc =3D irq_to_desc(irq); + + spin_lock(&desc->lock); + + if ( test_bit(_IRQ_DISABLED, &desc->status) ) + { + spin_unlock(&desc->lock); + continue; + } + + /* it is needed to avoid asserts in below calls */ + set_bit(_IRQ_DISABLED, &desc->status); + + gic_route_irq_to_xen(desc, GIC_PRI_IRQ); + + /* _IRQ_DISABLED is cleared by below call */ + desc->handler->startup(desc); + + spin_unlock(&desc->lock); + } + + spin_unlock(&local_irqs_type_lock); +} + static int cpu_callback(struct notifier_block *nfb, unsigned long action, void *hcpu) { @@ -131,6 +172,9 @@ static int cpu_callback(struct notifier_block *nfb, uns= igned long action, printk(XENLOG_ERR "Unable to allocate local IRQ for CPU%u\n", cpu); break; + case CPU_STARTING: + restore_local_irqs_on_resume(); + break; } =20 return notifier_from_errno(rc); --=20 2.48.1 From nobody Thu Oct 30 22:53:02 2025 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass(p=none dis=none) header.from=gmail.com ARC-Seal: i=1; a=rsa-sha256; t=1754945358; cv=none; d=zohomail.com; s=zohoarc; b=MQJFF/FKP64P3kVv74iFk8apVaEOUdz/NKuLaIBX2Fslc4y4U0fIObKof5onSFYEGoxZyxngLvsb0ZgaQLfi+u6C23v9V+GikT/B76sHdl6JCW1dgQF4n2rBXKes/BhPYRnml50iHA5HIC5bnF7Wsg0M/1sQHjpJj3kKHl0xMjQ= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1754945358; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=TGMaJnHIHucvLaFfdzAI/fXgS3JEgG5T2t9Vqll28+g=; b=jTORJG8koyXjBxrr2H1YrveN8mSwdcPUaCbfKwdTdBVcSyqMGT2UC4FQl1Sf/Hwaz4L5s3H3bkge5Ly0+s3rokRnEpP9Uk4uiVX7BGG7SEWIzqB/sPhQ0UGwdffdwqpTDIiW1MWAtvEMZbM3YazG0L1dCN0O9jJdTh47vI+P0zw= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 175494535865229.635798367008306; Mon, 11 Aug 2025 13:49:18 -0700 (PDT) Received: from list by lists.xenproject.org with outflank-mailman.1077952.1439015 (Exim 4.92) (envelope-from ) id 1ulZS2-0007OO-Uy; Mon, 11 Aug 2025 20:49:02 +0000 Received: by outflank-mailman (output) from mailman id 1077952.1439015; Mon, 11 Aug 2025 20:49:02 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1ulZS2-0007Mq-LZ; Mon, 11 Aug 2025 20:49:02 +0000 Received: by outflank-mailman (input) for mailman id 1077952; Mon, 11 Aug 2025 20:49:01 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1ulZS1-0005co-IM for xen-devel@lists.xenproject.org; Mon, 11 Aug 2025 20:49:01 +0000 Received: from mail-ej1-x633.google.com (mail-ej1-x633.google.com [2a00:1450:4864:20::633]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id 9b8932a9-76f4-11f0-a327-13f23c93f187; Mon, 11 Aug 2025 22:49:00 +0200 (CEST) Received: by mail-ej1-x633.google.com with SMTP id a640c23a62f3a-af96524c5a9so630348466b.1 for ; Mon, 11 Aug 2025 13:49:00 -0700 (PDT) Received: from yp-VivoBook-ASUSLaptop-M1503QA-M1503QA.Dlink ([77.52.179.38]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-af91a0a3bd2sm2065710466b.54.2025.08.11.13.48.57 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 11 Aug 2025 13:48:58 -0700 (PDT) X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 9b8932a9-76f4-11f0-a327-13f23c93f187 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1754945339; x=1755550139; darn=lists.xenproject.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=TGMaJnHIHucvLaFfdzAI/fXgS3JEgG5T2t9Vqll28+g=; b=Fk1uKYmhbp6Dt1kvzLlTWihbBXBs50l41FisAEc9+Cm1Q5om4TbfD0BSrea67Rqa5G urRWtINuccn7gEDz9Nu4X/y7Mtxhw2ougyGD5q5NFSOoW4UwGABhNPjm1cnTerGdoA6s uD0Ljdv0cG9kkm/GOi43WjrLizJEtfIhM7GP+S5QFqQSRvGjoSQWQOGGp9q9EgoCu9gt OEdwYLE3mp7BB8K+ZoyJCLsQoao/FaCpdkBvhfoeEvLbPfn1sjxqpJKjtDh8Nu14Xq+K lVevb2FjgVty0nqOcyiZSU2tF2vgJHGE3YrXMrVuImqhRViViOYRRR4QfkhtIj80hTF9 h8Yg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1754945339; x=1755550139; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=TGMaJnHIHucvLaFfdzAI/fXgS3JEgG5T2t9Vqll28+g=; b=Sw34vgm4HVd1t5sxqPVcefWXICzECjgGcfKYtRa2stE8ztCBDqd56639flBXxCvF7R Dmf81AILBLMsM2m3byMWAMGtWOl/JuHxc23m6/IsiiGE6FOAi4+KDtPGZWM30ShCnWCC alYwz0o3lpL45gca7w8JSv6oG/2YX3NFxpcTUu+B8BKGyRg+eWebSe71rTBIxdI3ojs6 t6a3hWnnWeCxoVG8mrbpXAWo2vAOIPNaPJVIjSDFOPWRO+SosbZ/7WhNEiR6ndExflWq tFWyyPAgsh6Z7y32Zdl8/m9zruEKA2x6jLd72VuzjHiOqjl//3vvD4ELLfMWw+ovEmot GAdw== X-Gm-Message-State: AOJu0Yw0tB9FqFVuNXlkK0kYEwsCBE0pLkn7T6ZfdZRXQ0AXUl5mtprk 2ZCS2nlcitHO870TyjR9Ja93JIS0PrmP6YWO796bwCaqkSLuHEoFLbD9bD2hYA== X-Gm-Gg: ASbGncsmwXEOHFC6swQ9Np27wmPEHcRuQiFdDLeyYATDGLfdJ4QYBpjfYSjjIXEZqgf MOOjWrbhfk4elerU9Zv/jy1cNP9zrPXcnCsWa8/b1qRaxaUesayPW5GsYwELV/+zIGAWdwdnkoL 8pzQ3lTOT7tgxve4eNz3eQ/5dgB/QaBqrC+ckfjw+kTWfvBtOceqIFM9D2EoknPN6lY3Ngn4GoF +JEzJVqDq22HZbTpYto+oiduwV+MtEtpvff8orQepieibCW8KiCh/gRJuEX+YEUDEURvGyalPFi djcXVVhMB1jSZs7mLsVa/6kzC+M5zjut+/aQHA5LS2VPCrZ35htybrKsAGazG4Xq1/0s2Bw9t2F 1uJKMqx0SyGrEpjijXLbMJSAEl2Py7dsIF7kScD9xJ1OT0WvfxsAYA3/wLORm4nMcc2c= X-Google-Smtp-Source: AGHT+IEot2xPeTED6Qm4nrzq5Fo3yY//aC2fJh6j7EGTtl8PuKddoJ3YUpgLNtMQdKNmOPlPQZkJsQ== X-Received: by 2002:a17:907:7ba2:b0:af9:3e23:831 with SMTP id a640c23a62f3a-af9c64d40a3mr1241421266b.39.1754945339166; Mon, 11 Aug 2025 13:48:59 -0700 (PDT) From: Mykola Kvach To: xen-devel@lists.xenproject.org Cc: Mirela Simonovic , Stefano Stabellini , Julien Grall , Bertrand Marquis , Michal Orzel , Volodymyr Babchuk , Andrew Cooper , Anthony PERARD , Jan Beulich , =?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= , Saeed Nowshadi , Mykyta Poturai , Mykola Kvach Subject: [PATCH v5 07/12] xen/arm: Add support for system suspend triggered by hardware domain Date: Mon, 11 Aug 2025 23:48:03 +0300 Message-ID: X-Mailer: git-send-email 2.48.1 In-Reply-To: References: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @gmail.com) X-ZM-MESSAGEID: 1754945359908124100 Content-Type: text/plain; charset="utf-8" From: Mirela Simonovic Trigger Xen suspend when the hardware domain initiates suspend via SHUTDOWN_suspend. Redirect system suspend to CPU#0 to ensure the suspend logic runs on the boot CPU, as required. Introduce full suspend/resume infrastructure gated by CONFIG_SYSTEM_SUSPEND, including logic to: - disable and enable non-boot physical CPUs - freeze and thaw domains - suspend and resume the GIC, timer and console - maintain system state before and after suspend Remove the restriction in the PSCI interface preventing suspend from the hardware domain. Select HAS_SYSTEM_SUSPEND for ARM_64. Signed-off-by: Mirela Simonovic Signed-off-by: Saeed Nowshadi Signed-off-by: Mykyta Poturai Signed-off-by: Mykola Kvach --- Changes in v5: - select HAS_SYSTEM_SUSPEND in ARM_64 instead of ARM - check llc_coloring_enabled instead of LLC_COLORING during the selection of HAS_SYSTEM_SUSPEND config - call host_system_suspend from guest PSCI system suspend instead of arch_domain_shutdown, reducing the complexity of the new code - update some comments Changes introduced in V4: - drop code for saving and restoring VCPU context (for more information refer part 1 of patch series) - remove IOMMU suspend and resume calls until they will be implemented - move system suspend logic to arch_domain_shutdown, invoked from domain_shutdown - apply minor fixes such as renaming functions, updating comments, and modifying the commit message to reflect these changes - add console_end_sync to resume path after system suspend Changes introduced in V3: - merge changes from other commits into this patch (previously stashed): 1) disable/enable non-boot physical CPUs and freeze/thaw domains during suspend/resume 2) suspend/resume the timer, GIC, console, IOMMU, and hardware domain --- xen/arch/arm/Kconfig | 1 + xen/arch/arm/Makefile | 1 + xen/arch/arm/include/asm/suspend.h | 22 +++++ xen/arch/arm/suspend.c | 151 +++++++++++++++++++++++++++++ xen/arch/arm/vpsci.c | 12 ++- xen/common/domain.c | 4 + 6 files changed, 190 insertions(+), 1 deletion(-) create mode 100644 xen/arch/arm/include/asm/suspend.h create mode 100644 xen/arch/arm/suspend.c diff --git a/xen/arch/arm/Kconfig b/xen/arch/arm/Kconfig index a0c8160474..ccdbaa5bc3 100644 --- a/xen/arch/arm/Kconfig +++ b/xen/arch/arm/Kconfig @@ -8,6 +8,7 @@ config ARM_64 depends on !ARM_32 select 64BIT select HAS_FAST_MULTIPLY + select HAS_SYSTEM_SUSPEND if UNSUPPORTED select HAS_VPCI_GUEST_SUPPORT if PCI_PASSTHROUGH =20 config ARM diff --git a/xen/arch/arm/Makefile b/xen/arch/arm/Makefile index f833cdf207..3f6247adee 100644 --- a/xen/arch/arm/Makefile +++ b/xen/arch/arm/Makefile @@ -51,6 +51,7 @@ obj-y +=3D setup.o obj-y +=3D shutdown.o obj-y +=3D smp.o obj-y +=3D smpboot.o +obj-$(CONFIG_SYSTEM_SUSPEND) +=3D suspend.o obj-$(CONFIG_SYSCTL) +=3D sysctl.o obj-y +=3D time.o obj-y +=3D traps.o diff --git a/xen/arch/arm/include/asm/suspend.h b/xen/arch/arm/include/asm/= suspend.h new file mode 100644 index 0000000000..78d0e2383b --- /dev/null +++ b/xen/arch/arm/include/asm/suspend.h @@ -0,0 +1,22 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ + +#ifndef __ASM_ARM_SUSPEND_H__ +#define __ASM_ARM_SUSPEND_H__ + +#ifdef CONFIG_SYSTEM_SUSPEND + +int host_system_suspend(void); + +#endif /* CONFIG_SYSTEM_SUSPEND */ + +#endif /* __ASM_ARM_SUSPEND_H__ */ + + /* + * Local variables: + * mode: C + * c-file-style: "BSD" + * c-basic-offset: 4 + * tab-width: 4 + * indent-tabs-mode: nil + * End: + */ diff --git a/xen/arch/arm/suspend.c b/xen/arch/arm/suspend.c new file mode 100644 index 0000000000..abf4b928ce --- /dev/null +++ b/xen/arch/arm/suspend.c @@ -0,0 +1,151 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ + +#include +#include +#include +#include + +/* + * TODO list: + * - Test system suspend with LLC_COLORING enabled and verify functionali= ty + * - Implement IOMMU suspend/resume handlers and integrate them + * into the suspend/resume path (IPMMU and SMMU) + * - Enable "xl suspend" support on ARM architecture + * - Properly disable Xen timer watchdog from relevant services + * - Add suspend/resume CI test for ARM (QEMU if feasible) + * - Investigate feasibility and need for implementing system suspend on = ARM32 + */ + +/* Xen suspend. Note: data is not used (suspend is the suspend to RAM) */ +static long system_suspend(void *data) +{ + int status; + unsigned long flags; + + BUG_ON(system_state !=3D SYS_STATE_active); + + system_state =3D SYS_STATE_suspend; + freeze_domains(); + scheduler_disable(); + + /* + * Non-boot CPUs have to be disabled on suspend and enabled on resume + * (hotplug-based mechanism). Disabling non-boot CPUs will lead to PSCI + * CPU_OFF to be called by each non-boot CPU. Depending on the underly= ing + * platform capabilities, this may lead to the physical powering down = of + * CPUs. Tested on Xilinx Zynq Ultrascale+ MPSoC (including power down= of + * each non-boot CPU). + */ + status =3D disable_nonboot_cpus(); + if ( status ) + { + system_state =3D SYS_STATE_resume; + goto resume_nonboot_cpus; + } + + time_suspend(); + + local_irq_save(flags); + status =3D gic_suspend(); + if ( status ) + { + system_state =3D SYS_STATE_resume; + goto resume_irqs; + } + + printk("Xen suspending...\n"); + + console_start_sync(); + status =3D console_suspend(); + if ( status ) + { + dprintk(XENLOG_ERR, "Failed to suspend the console, err=3D%d\n", s= tatus); + system_state =3D SYS_STATE_resume; + goto resume_console; + } + + /* + * Enable identity mapping before entering suspend to simplify + * the resume path + */ + update_boot_mapping(true); + + system_state =3D SYS_STATE_resume; + update_boot_mapping(false); + + resume_console: + console_resume(); + console_end_sync(); + + gic_resume(); + + resume_irqs: + local_irq_restore(flags); + time_resume(); + + resume_nonboot_cpus: + /* + * The rcu_barrier() has to be added to ensure that the per cpu area is + * freed before a non-boot CPU tries to initialize it (_free_percpu_ar= ea() + * has to be called before the init_percpu_area()). This scenario occu= rs + * when non-boot CPUs are hot-unplugged on suspend and hotplugged on r= esume. + */ + rcu_barrier(); + enable_nonboot_cpus(); + scheduler_enable(); + thaw_domains(); + + system_state =3D SYS_STATE_active; + + /* The resume of hardware domain should always follow Xen's resume. */ + domain_resume(hardware_domain); + + printk("Resume (status %d)\n", status); + return status; +} + +int host_system_suspend(void) +{ + int status; + + /* TODO: drop check after verification that features can work together= */ + if ( llc_coloring_enabled ) + { + dprintk(XENLOG_ERR, + "System suspend is not supported with LLC_COLORING enabled= \n"); + return -ENOSYS; + } + + /* + * system_suspend should be called when Dom0 finalizes the suspend + * procedure from its boot core (VCPU#0). However, Dom0's VCPU#0 could + * be mapped to any PCPU (this function could be executed by any PCPU). + * The suspend procedure has to be finalized by the PCPU#0 (non-boot + * PCPUs will be disabled during the suspend). + */ + status =3D continue_hypercall_on_cpu(0, system_suspend, NULL); + + /* + * If an error happened, there is nothing that needs to be done here + * because the system_suspend always returns in fully functional state + * no matter what the outcome of suspend procedure is. If the system + * suspended successfully the function will return 0 after the resume. + * Otherwise, if an error is returned it means Xen did not suspended, + * but it is still in the same state as if the system_suspend was never + * called. We dump a debug message in case of an error for debugging/ + * logging purpose. + */ + if ( status ) + dprintk(XENLOG_ERR, "Failed to suspend, errno=3D%d\n", status); + + return status; +} + +/* + * Local variables: + * mode: C + * c-file-style: "BSD" + * c-basic-offset: 4 + * indent-tabs-mode: nil + * End: + */ diff --git a/xen/arch/arm/vpsci.c b/xen/arch/arm/vpsci.c index 67d369a8a2..757e719ea7 100644 --- a/xen/arch/arm/vpsci.c +++ b/xen/arch/arm/vpsci.c @@ -4,6 +4,7 @@ #include =20 #include +#include #include #include #include @@ -214,9 +215,10 @@ static int32_t do_psci_1_0_system_suspend(register_t e= point, register_t cid) struct vcpu *v; struct domain *d =3D current->domain; =20 - /* SYSTEM_SUSPEND is not supported for the hardware domain yet */ +#ifndef CONFIG_SYSTEM_SUSPEND if ( is_hardware_domain(d) ) return PSCI_NOT_SUPPORTED; +#endif =20 /* Ensure that all CPUs other than the calling one are offline */ domain_lock(d); @@ -234,6 +236,14 @@ static int32_t do_psci_1_0_system_suspend(register_t e= point, register_t cid) if ( rc ) return PSCI_DENIED; =20 +#ifdef CONFIG_SYSTEM_SUSPEND + if ( is_hardware_domain(d) && host_system_suspend() ) + { + domain_resume_nopause(d); + return PSCI_DENIED; + } +#endif + rc =3D do_setup_vcpu_ctx(current, epoint, cid); if ( rc !=3D PSCI_SUCCESS ) { diff --git a/xen/common/domain.c b/xen/common/domain.c index c3609b0cb0..414a691242 100644 --- a/xen/common/domain.c +++ b/xen/common/domain.c @@ -1311,7 +1311,11 @@ int domain_shutdown(struct domain *d, u8 reason) d->shutdown_code =3D reason; reason =3D d->shutdown_code; =20 +#if defined(CONFIG_SYSTEM_SUSPEND) && defined(CONFIG_ARM) + if ( reason !=3D SHUTDOWN_suspend && is_hardware_domain(d) ) +#else if ( is_hardware_domain(d) ) +#endif hwdom_shutdown(reason); =20 if ( d->is_shutting_down ) --=20 2.48.1 From nobody Thu Oct 30 22:53:02 2025 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass(p=none dis=none) header.from=gmail.com ARC-Seal: i=1; a=rsa-sha256; t=1754945364; cv=none; d=zohomail.com; s=zohoarc; b=CqW8IJm/WZ5ZMYVCt9WbbRMbdaDfCBM5CXZSmqpHR0NEfPYxBfMajpBK2barI/kEfmztsikvtfCGTUcu0/GV9OqA5qreqCWJuyZ2+95kB2SUUWGv841Ay8ba0FHUEvUrkUurOCluB+rsBNP4cTlYDrKm3z3QjGj5BWn8JPsdwww= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1754945364; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=WV3I3iVcTMMqNyOJNyQt08YG5fWdAf6qVI+oDW1WgEk=; b=flhKzc7rXUYeOvFJoVXG9iY9mxf/ytttG+KqxW/3NCJjw630Ia/rxRT6H1akwLB1api6VY7/jKe802C9nvBgKvnFa7qdGdLksr1crgex4O6agcRpIwXhwFO+UaSz0S001Vo246akvzwM/ecJdwv10TkcYvmWc6dOpr5sxreKmdY= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1754945364765838.0920469655823; Mon, 11 Aug 2025 13:49:24 -0700 (PDT) Received: from list by lists.xenproject.org with outflank-mailman.1077954.1439025 (Exim 4.92) (envelope-from ) id 1ulZS5-0007qA-7B; Mon, 11 Aug 2025 20:49:05 +0000 Received: by outflank-mailman (output) from mailman id 1077954.1439025; Mon, 11 Aug 2025 20:49:05 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1ulZS5-0007po-1j; Mon, 11 Aug 2025 20:49:05 +0000 Received: by outflank-mailman (input) for mailman id 1077954; Mon, 11 Aug 2025 20:49:03 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1ulZS3-0005co-Io for xen-devel@lists.xenproject.org; Mon, 11 Aug 2025 20:49:03 +0000 Received: from mail-ej1-x631.google.com (mail-ej1-x631.google.com [2a00:1450:4864:20::631]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id 9c916888-76f4-11f0-a327-13f23c93f187; Mon, 11 Aug 2025 22:49:02 +0200 (CEST) Received: by mail-ej1-x631.google.com with SMTP id a640c23a62f3a-af9a25f091fso642644066b.0 for ; Mon, 11 Aug 2025 13:49:02 -0700 (PDT) Received: from yp-VivoBook-ASUSLaptop-M1503QA-M1503QA.Dlink ([77.52.179.38]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-af91a0a3bd2sm2065710466b.54.2025.08.11.13.48.59 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 11 Aug 2025 13:49:00 -0700 (PDT) X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 9c916888-76f4-11f0-a327-13f23c93f187 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1754945341; x=1755550141; darn=lists.xenproject.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=WV3I3iVcTMMqNyOJNyQt08YG5fWdAf6qVI+oDW1WgEk=; b=cKJuw1XecDJZ8r3iZ/XGO0iRRYWQOExemJyAASxe95qw/V3a39O5JPFAzCaaSWCbVV p6gMtcI6aqhv97XcLr5Astq1zgLzyvO+b7K/TZzKVEo5VKecuDtGVfJ31NAgZ694XdwI oS5+rR8Muz1czrdCLOGrIb/67NJaymkPpuSVETaQnU+Fre9GCkX/7cmrp36s0BIzrZTk YwexBTOk2DYu22R4wkqFwHsUqXgtRAZusYYdxK5yKuvIMpy8kX/paXGK+/gKPEhb1hAU oydJJqy6mvNOVoaxzKeNLzmk8K8uwkJ16ugTc2mhGEW1Qdj/EGSOUlmLXDIQXUG4vIW4 XveA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1754945341; x=1755550141; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=WV3I3iVcTMMqNyOJNyQt08YG5fWdAf6qVI+oDW1WgEk=; b=L4B6LUv6b5/9x7XKwuzt+qaMjHl4TKK7dbgxjkmZb4bi/NHa1FHZGsJ8ocdiIU20WY 8hLQXxZizGan030sTfWgIYnAe7X7+JKn5SO2TMclofmuxdXNVCtziCZYX211cBty8EDH GoePd2d6xf1kXXivSXA3/IF7e7xpXLWq1jgTT7MS3mjjYcYy4INUtwXF1Ff3M/nxYLgh zL8BXCYtGUHhXXmbmWIPGKUhzi1BainhSOIX46cMr7p4qlfeoS/u0hdldlOdlaUm5KKl 4n+RW/g6nAyvByALCpys7f6xW9l3K9W+PdY5iVY0JB4/rSaV3aXEY1TkPYISbswG/+yE qQKg== X-Gm-Message-State: AOJu0Yxewy7hGe3ExZSzk7GlO7UPQao++YTypEbDvnvGOyTMTfR7z7XE JCHOMADFwc9YrKlFhWN/T8nVLW7ZkEBp6CqFwMH2QRMiiLhAZMFkZNbKEsrpNg== X-Gm-Gg: ASbGncuao7HGlF4Qa3XsxbGPsyBvDRKawS8L5z7iCaTURDnGWVNf79YQxkDGS9d07xS XeUxJv2uAsZ+r/d9jqQMCziRc86EwIOPzf26E6NAxnwGupjYGz2U2FcKTjCaVvDw4x3mvfOFXh0 SQLx5vDTydr3+H3Rwoqe/qCxLLtO8N7eAZ3NFfSMwJHc0+i4VGOyR1F3RtARB9Y7Y0VcUjRHn+5 cNWcWvjJJv+lFikjobhYd8/oEUob5t/CPa/ce3LMrA2EkCyuqtx9qBWhvAIhpA/WPIIudsxjhYy hCM6ZC+OzYvSl1Je2XSfHd1fVgc0BNXeHyRncNtxgUKKcPjWK2eZ6EIoY26lvpMNWqaY3MLNApA eQshwr3ypTCvk7jq02ajNPOEkF5sZsRTB2QJfXQMTdo8rdPlCJqln2gYI/DENW1MhfCk= X-Google-Smtp-Source: AGHT+IH8tgHqjKRy9ou+a/j6Sb5A7FZAPXUwRxtrBIzlXSCp55Ks+Gv58Suvi0Zmu2sioQAf9PPhBQ== X-Received: by 2002:a17:907:3dab:b0:adf:7740:9284 with SMTP id a640c23a62f3a-afa1e13bccemr83193966b.57.1754945340811; Mon, 11 Aug 2025 13:49:00 -0700 (PDT) From: Mykola Kvach To: xen-devel@lists.xenproject.org Cc: Mirela Simonovic , Stefano Stabellini , Julien Grall , Bertrand Marquis , Michal Orzel , Volodymyr Babchuk , Saeed Nowshadi , Mykyta Poturai , Mykola Kvach Subject: [PATCH v5 08/12] xen/arm: Implement PSCI SYSTEM_SUSPEND call (host interface) Date: Mon, 11 Aug 2025 23:48:04 +0300 Message-ID: X-Mailer: git-send-email 2.48.1 In-Reply-To: References: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @gmail.com) X-ZM-MESSAGEID: 1754945366757116600 Content-Type: text/plain; charset="utf-8" From: Mirela Simonovic Invoke PSCI SYSTEM_SUSPEND to finalize Xen's suspend sequence on ARM64 plat= forms. Pass the resume entry point (hyp_resume) as the first argument to EL3. The = resume handler is currently a stub and will be implemented later in assembly. Igno= re the context ID argument, as is done in Linux. Only enable this path when CONFIG_SYSTEM_SUSPEND is set and PSCI version is >=3D 1.0. Signed-off-by: Mirela Simonovic Signed-off-by: Saeed Nowshadi Signed-off-by: Mykyta Poturai Signed-off-by: Mykola Kvach --- Changes in v4: - select the appropriate PSCI SYSTEM_SUSPEND function ID based on platform - update comments and commit message to reflect recent changes Changes in v3: - return PSCI_NOT_SUPPORTED instead of a hardcoded 1 on ARM32 - check PSCI version before invoking SYSTEM_SUSPEND in call_psci_system_s= uspend --- xen/arch/arm/arm64/head.S | 8 ++++++++ xen/arch/arm/include/asm/psci.h | 1 + xen/arch/arm/include/asm/suspend.h | 2 ++ xen/arch/arm/psci.c | 23 ++++++++++++++++++++++- xen/arch/arm/suspend.c | 5 +++++ 5 files changed, 38 insertions(+), 1 deletion(-) diff --git a/xen/arch/arm/arm64/head.S b/xen/arch/arm/arm64/head.S index 72c7b24498..3522c497c5 100644 --- a/xen/arch/arm/arm64/head.S +++ b/xen/arch/arm/arm64/head.S @@ -561,6 +561,14 @@ END(efi_xen_start) =20 #endif /* CONFIG_ARM_EFI */ =20 +#ifdef CONFIG_SYSTEM_SUSPEND + +FUNC(hyp_resume) + b . +END(hyp_resume) + +#endif /* CONFIG_SYSTEM_SUSPEND */ + /* * Local variables: * mode: ASM diff --git a/xen/arch/arm/include/asm/psci.h b/xen/arch/arm/include/asm/psc= i.h index 48a93e6b79..bb3c73496e 100644 --- a/xen/arch/arm/include/asm/psci.h +++ b/xen/arch/arm/include/asm/psci.h @@ -23,6 +23,7 @@ int call_psci_cpu_on(int cpu); void call_psci_cpu_off(void); void call_psci_system_off(void); void call_psci_system_reset(void); +int call_psci_system_suspend(void); =20 /* Range of allocated PSCI function numbers */ #define PSCI_FNUM_MIN_VALUE _AC(0,U) diff --git a/xen/arch/arm/include/asm/suspend.h b/xen/arch/arm/include/asm/= suspend.h index 78d0e2383b..55041a5d06 100644 --- a/xen/arch/arm/include/asm/suspend.h +++ b/xen/arch/arm/include/asm/suspend.h @@ -7,6 +7,8 @@ =20 int host_system_suspend(void); =20 +void hyp_resume(void); + #endif /* CONFIG_SYSTEM_SUSPEND */ =20 #endif /* __ASM_ARM_SUSPEND_H__ */ diff --git a/xen/arch/arm/psci.c b/xen/arch/arm/psci.c index b6860a7760..c9d126b195 100644 --- a/xen/arch/arm/psci.c +++ b/xen/arch/arm/psci.c @@ -17,17 +17,20 @@ #include #include #include +#include =20 /* * While a 64-bit OS can make calls with SMC32 calling conventions, for * some calls it is necessary to use SMC64 to pass or return 64-bit values. - * For such calls PSCI_0_2_FN_NATIVE(x) will choose the appropriate + * For such calls PSCI_*_FN_NATIVE(x) will choose the appropriate * (native-width) function ID. */ #ifdef CONFIG_ARM_64 #define PSCI_0_2_FN_NATIVE(name) PSCI_0_2_FN64_##name +#define PSCI_1_0_FN_NATIVE(name) PSCI_1_0_FN64_##name #else #define PSCI_0_2_FN_NATIVE(name) PSCI_0_2_FN32_##name +#define PSCI_1_0_FN_NATIVE(name) PSCI_1_0_FN32_##name #endif =20 uint32_t psci_ver; @@ -60,6 +63,24 @@ void call_psci_cpu_off(void) } } =20 +int call_psci_system_suspend(void) +{ +#ifdef CONFIG_SYSTEM_SUSPEND + struct arm_smccc_res res; + + if ( psci_ver < PSCI_VERSION(1, 0) ) + return PSCI_NOT_SUPPORTED; + + /* 2nd argument (context ID) is not used */ + arm_smccc_smc(PSCI_1_0_FN_NATIVE(SYSTEM_SUSPEND), __pa(hyp_resume), &r= es); + return PSCI_RET(res); +#else + dprintk(XENLOG_WARNING, + "SYSTEM_SUSPEND not supported (CONFIG_SYSTEM_SUSPEND disabled)= \n"); + return PSCI_NOT_SUPPORTED; +#endif +} + void call_psci_system_off(void) { if ( psci_ver > PSCI_VERSION(0, 1) ) diff --git a/xen/arch/arm/suspend.c b/xen/arch/arm/suspend.c index abf4b928ce..11e86b7f51 100644 --- a/xen/arch/arm/suspend.c +++ b/xen/arch/arm/suspend.c @@ -1,5 +1,6 @@ /* SPDX-License-Identifier: GPL-2.0-only */ =20 +#include #include #include #include @@ -70,6 +71,10 @@ static long system_suspend(void *data) */ update_boot_mapping(true); =20 + status =3D call_psci_system_suspend(); + if ( status ) + dprintk(XENLOG_WARNING, "PSCI system suspend failed, err=3D%d\n", = status); + system_state =3D SYS_STATE_resume; update_boot_mapping(false); =20 --=20 2.48.1 From nobody Thu Oct 30 22:53:02 2025 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass(p=none dis=none) header.from=gmail.com ARC-Seal: i=1; a=rsa-sha256; t=1754945361; cv=none; d=zohomail.com; s=zohoarc; b=J4b4oR+qx++crKPcirT5d+R2u22xrYHjMtkGKw4zcQ7vEm3Vw3hVOIiquOv3pLA/BY8p9GpDgSsvJxw+FprbEFyJSni+pxufbN59XUXvm6eHCC/X9p9K3mnwU9sOoAmTGt5JfMqt/OyVRJ0E34byF+LBmEITNYktXABLMZJrUns= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1754945361; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=8vCXXOUavoZEz6D2AXJtCOcWlfxhKLbJDHEKs+IY4pA=; b=G0KPEg7+QfdA9NozrEiaOP6tK2dO7dVpf18pbe44LJMjc6VizscVfkb115e8nlnUZHDZD45ZT6ykXhQVnlc7X1//2o7qkZVhcwKCqg77cZrXgaiwvgJFMfcFfs6wZpMTVkyhonoQzCrRPWDeED/YyoRKC2jVUmarISW3NuqpP1s= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1754945360999488.79877588021384; Mon, 11 Aug 2025 13:49:20 -0700 (PDT) Received: from list by lists.xenproject.org with outflank-mailman.1077956.1439036 (Exim 4.92) (envelope-from ) id 1ulZS7-0008Di-Tq; Mon, 11 Aug 2025 20:49:07 +0000 Received: by outflank-mailman (output) from mailman id 1077956.1439036; Mon, 11 Aug 2025 20:49:07 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1ulZS7-0008DN-Nd; Mon, 11 Aug 2025 20:49:07 +0000 Received: by outflank-mailman (input) for mailman id 1077956; Mon, 11 Aug 2025 20:49:06 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1ulZS6-00063z-1e for xen-devel@lists.xenproject.org; Mon, 11 Aug 2025 20:49:06 +0000 Received: from mail-ed1-x52c.google.com (mail-ed1-x52c.google.com [2a00:1450:4864:20::52c]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id 9d80cc54-76f4-11f0-b898-0df219b8e170; Mon, 11 Aug 2025 22:49:03 +0200 (CEST) Received: by mail-ed1-x52c.google.com with SMTP id 4fb4d7f45d1cf-6156463fae9so9170657a12.0 for ; Mon, 11 Aug 2025 13:49:03 -0700 (PDT) Received: from yp-VivoBook-ASUSLaptop-M1503QA-M1503QA.Dlink ([77.52.179.38]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-af91a0a3bd2sm2065710466b.54.2025.08.11.13.49.00 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 11 Aug 2025 13:49:01 -0700 (PDT) X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 9d80cc54-76f4-11f0-b898-0df219b8e170 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1754945343; x=1755550143; darn=lists.xenproject.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=8vCXXOUavoZEz6D2AXJtCOcWlfxhKLbJDHEKs+IY4pA=; b=PhCLYhne379ksm8mcXqWW6nb/iLzlDTY6PJ5GbnlMgj3/FF1d8Bd84tL+49N+uCEnx /usrWKeDFbUKA/8cgY3g/MqZHgkbZ3w11Y2W1n4TSgcZI6cGz1e0uOmB2VzLm+MWzuX4 KVN0v7P/U44M5moXnGyVTu47u7WvBLfL/rHGR7MSt3eYXNHxNK+pTMfK9Xr8Wbnr5gfA ey8nmIY7quiCp7OjBJbSerV0jjKBXNBcJcJvsJ4MiD+IEpHpNn/Iw2JG7DUhjfHmZTWP K5+DRLiScp8zW41VvwVXeZArqTVWg2iQEIHyFYi+ZrJS9i4iKzunVEDwX0snakW8USuq iExg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1754945343; x=1755550143; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=8vCXXOUavoZEz6D2AXJtCOcWlfxhKLbJDHEKs+IY4pA=; b=MTJJj+PjKq7yBbpSUCbmMye1DXj17kda72RFV9Kwoqkhlqj6R5kHYZt0v2h/LJSE+0 0ycRtblroQ8+ALYQoZWleGneSgyyrd5b6R7Hqc874uRWtQUIwxRqlxkTmWBSEfwUHvjU T8aAbRPdKTB/5KR2OpWLzYckgaz3t6tbY+1ki9DcSqDFzP88Qx62aPbNN5W6mA53ago2 Li7uoprSGn9K+9x869kUYwGkHhpq0PNgX3ouAHii0J9GWW2VNdsLtc2B+amwZJdJydAh uqmygQnszEYNl86ySVdePxYM44fkSrW/TY2CslgwxLHlBT50CCrkXbh03kL26vgz94F9 oG3g== X-Gm-Message-State: AOJu0YwKB6YPA5JU1CpuS56yder6XzAa7lgawRKRoGTjZEXRC2dOW9Wo Fyqacw7vnLlJS+F/dqUXL3af70fLOY+pXjxCov1rJeWoiacc4HFGReuz3rGBDg== X-Gm-Gg: ASbGncuhQT2UOZ4J5Yp2QHVjY2i3T4Y/38LT1g2kQ1RWa+2sziUcZ2bPZ38DsG0tRwX PLNdcECDTPu+b7b1BeumjxvikuRGs2nFUBBeRtuipzPXkD7KYU5JIEFJ2q5POWR6SDdvbewD55V zKtSib3yAxgbpW82zYIuVdt7jkQDHP6lcVcII6dQAVmpHE4A9RiEBgesLD3rP1FbA4kpXOYNvET zwxlIABHmG1ycChKW9uaniZ40X/tsG7MJhaEywvm64uh/7EADXMYjTjIc9PYcarstOGWS1vwRFV Q6zaJQf0BjA+QTi2QYWRojIH+j1djCJSVFbas/wwGGQ/n4pv8lD8h/d7yVmkIRxTAFv3X1+VHTf 7/BLpyqjZky69zv7PUyAonRtaG5nbsBUULRN8dyoZBnW+Rfz/8mb4Nn0bh9jo4MjZPes= X-Google-Smtp-Source: AGHT+IFvx2QD9CEtIMTGc3JxuJiqe7FLmH+70iO+zvpaaiT8vngEUoYYhmLIc6hkC66nAn22bDo2qg== X-Received: by 2002:a17:907:1b0e:b0:ae6:c334:af3a with SMTP id a640c23a62f3a-afa1d617a33mr113078066b.6.1754945342363; Mon, 11 Aug 2025 13:49:02 -0700 (PDT) From: Mykola Kvach To: xen-devel@lists.xenproject.org Cc: Mirela Simonovic , Stefano Stabellini , Julien Grall , Bertrand Marquis , Michal Orzel , Volodymyr Babchuk , Saeed Nowshadi , Mykyta Poturai , Mykola Kvach Subject: [PATCH v5 09/12] xen/arm: Resume memory management on Xen resume Date: Mon, 11 Aug 2025 23:48:05 +0300 Message-ID: X-Mailer: git-send-email 2.48.1 In-Reply-To: References: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @gmail.com) X-ZM-MESSAGEID: 1754945361778124100 Content-Type: text/plain; charset="utf-8" From: Mirela Simonovic The MMU must be enabled during the resume path before restoring context, as virtual addresses are used to access the saved context data. This patch adds MMU setup during resume by reusing the existing enable_secondary_cpu_mm function, which enables data cache and the MMU. Before the MMU is enabled, the content of TTBR0_EL2 is changed to point to init_ttbr (page tables used at runtime). On boot, init_ttbr is normally initialized during secondary CPU hotplug. On uniprocessor systems, this would leave init_ttbr uninitialized, causing resume to fail. To address this, the boot CPU now sets init_ttbr during suspend. Signed-off-by: Mirela Simonovic Signed-off-by: Saeed Nowshadi Signed-off-by: Mykyta Poturai Signed-off-by: Mykola Kvach --- Changes in v4: - Drop unnecessary DAIF masking; interrupts are already masked on resume - Remove leftover TLB flush instructions; flushing is done in enable_mmu - Avoid setting x19 in hyp_resume; not needed - Replace prepare_secondary_mm with set_init_ttbr; call it from system_susp= end Changes in v3: - Update commit message for clarity - Replace create_page_tables, enable_mmu, and mmu_init_secondary_cpu with enable_secondary_cpu_mm - Move prepare_secondary_mm to start_xen to avoid crash - Add early UART init during resume Changes in v2: - Move hyp_resume to head.S to keep resume logic together - Simplify hyp_resume using existing helpers: check_cpu_mode, cpu_init, create_page_tables, enable_mmu --- xen/arch/arm/arm64/head.S | 16 ++++++++++++++++ xen/arch/arm/include/asm/mm.h | 2 ++ xen/arch/arm/mmu/smpboot.c | 2 +- xen/arch/arm/suspend.c | 2 ++ 4 files changed, 21 insertions(+), 1 deletion(-) diff --git a/xen/arch/arm/arm64/head.S b/xen/arch/arm/arm64/head.S index 3522c497c5..596e960152 100644 --- a/xen/arch/arm/arm64/head.S +++ b/xen/arch/arm/arm64/head.S @@ -564,6 +564,22 @@ END(efi_xen_start) #ifdef CONFIG_SYSTEM_SUSPEND =20 FUNC(hyp_resume) + /* Initialize the UART if earlyprintk has been enabled. */ +#ifdef CONFIG_EARLY_PRINTK + bl init_uart +#endif + PRINT_ID("- Xen resuming -\r\n") + + bl check_cpu_mode + bl cpu_init + + ldr x0, =3Dstart + adr x20, start /* x20 :=3D paddr (start) */ + sub x20, x20, x0 /* x20 :=3D phys-offset */ + ldr lr, =3Dmmu_resumed + b enable_secondary_cpu_mm + +mmu_resumed: b . END(hyp_resume) =20 diff --git a/xen/arch/arm/include/asm/mm.h b/xen/arch/arm/include/asm/mm.h index fb79aeb088..3400cb2bff 100644 --- a/xen/arch/arm/include/asm/mm.h +++ b/xen/arch/arm/include/asm/mm.h @@ -365,6 +365,8 @@ static inline void page_set_xenheap_gfn(struct page_inf= o *p, gfn_t gfn) } while ( (y =3D cmpxchg(&p->u.inuse.type_info, x, nx)) !=3D x ); } =20 +void set_init_ttbr(lpae_t *root); + #endif /* __ARCH_ARM_MM__ */ /* * Local variables: diff --git a/xen/arch/arm/mmu/smpboot.c b/xen/arch/arm/mmu/smpboot.c index 37e91d72b7..ff508ecf40 100644 --- a/xen/arch/arm/mmu/smpboot.c +++ b/xen/arch/arm/mmu/smpboot.c @@ -72,7 +72,7 @@ static void clear_boot_pagetables(void) clear_table(boot_third); } =20 -static void set_init_ttbr(lpae_t *root) +void set_init_ttbr(lpae_t *root) { /* * init_ttbr is part of the identity mapping which is read-only. So diff --git a/xen/arch/arm/suspend.c b/xen/arch/arm/suspend.c index 11e86b7f51..08b6acaede 100644 --- a/xen/arch/arm/suspend.c +++ b/xen/arch/arm/suspend.c @@ -65,6 +65,8 @@ static long system_suspend(void *data) goto resume_console; } =20 + set_init_ttbr(xen_pgtable); + /* * Enable identity mapping before entering suspend to simplify * the resume path --=20 2.48.1 From nobody Thu Oct 30 22:53:02 2025 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass(p=none dis=none) header.from=gmail.com ARC-Seal: i=1; a=rsa-sha256; t=1754945367; cv=none; d=zohomail.com; s=zohoarc; b=m4xeYhuQbQD8cNAMPnOjvmiPu9WlIATFNWdfUDC5LrjvUmWUcEsBzXyNfq9Xm1WxWzQ4WsQ4D8yZjxWP9NdZ9ptvle1xXVoLgIxWOHld9nta8wHZalazP9EWPyIbWz4Jjw4RJ02c/FXnIKujO61LTlPmWvqh7uujnCTfvvmL1c4= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1754945367; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=P9RalIq5/KBIQXS8hw4c9tUFcC4ZusIbNEKpuKw1Hyg=; b=VkojbkipA4MN7bMo/A1z74i9qcnAHlgb6RYhWpA68VtmqU4+TSpi67Jgtwj9thYBSLzjT4s1Id95sbfayfoxSlNL6ys6/WWyoqaBx3VzWAvo5grT89uVxXVlf69Fgm49IxP8kN85Bk3T3bPR4JEe0b+fEasZl50VwWzr5z+mG+k= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1754945367422260.7816089877123; Mon, 11 Aug 2025 13:49:27 -0700 (PDT) Received: from list by lists.xenproject.org with outflank-mailman.1077957.1439042 (Exim 4.92) (envelope-from ) id 1ulZS8-0008Jq-KV; Mon, 11 Aug 2025 20:49:08 +0000 Received: by outflank-mailman (output) from mailman id 1077957.1439042; Mon, 11 Aug 2025 20:49:08 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1ulZS8-0008IZ-5k; Mon, 11 Aug 2025 20:49:08 +0000 Received: by outflank-mailman (input) for mailman id 1077957; Mon, 11 Aug 2025 20:49:07 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1ulZS7-00063z-26 for xen-devel@lists.xenproject.org; Mon, 11 Aug 2025 20:49:07 +0000 Received: from mail-ed1-x535.google.com (mail-ed1-x535.google.com [2a00:1450:4864:20::535]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id 9e843ad0-76f4-11f0-b898-0df219b8e170; Mon, 11 Aug 2025 22:49:05 +0200 (CEST) Received: by mail-ed1-x535.google.com with SMTP id 4fb4d7f45d1cf-61571192ba5so7526980a12.2 for ; Mon, 11 Aug 2025 13:49:05 -0700 (PDT) Received: from yp-VivoBook-ASUSLaptop-M1503QA-M1503QA.Dlink ([77.52.179.38]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-af91a0a3bd2sm2065710466b.54.2025.08.11.13.49.02 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 11 Aug 2025 13:49:03 -0700 (PDT) X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 9e843ad0-76f4-11f0-b898-0df219b8e170 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1754945344; x=1755550144; darn=lists.xenproject.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=P9RalIq5/KBIQXS8hw4c9tUFcC4ZusIbNEKpuKw1Hyg=; b=BK+qC/4Ot6ECLlOVoJgrNb72cJbBdGLAtIxAvvQp6K0o//L0njqhG4Riabldy72E5K A11iM8vhpPCm9+j1znQEe6XFm31uEir0mbjO2NUtqdluyDrbrgqKF1VkcemmfS/JyQm5 ZeDvpkE5KtQI51uSbdvIG6Bfmx9qJcQBmQhl5nJNezYqDGzXcuDgKckh3h4qQk5Zd2Ye YpaE6JdLUjro5VqHsmjr0rZpJr7J2WDJUfWr2LeTCTZeehpaP8pwI9Ke04SPyxMCH1JU FTu2VdfIkKxQm2i6InH+QxlMvWvbIJFFU8Gwme19t+nTk1/Oro4IhCJZpQoU7FyJew4W 1Lqg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1754945344; x=1755550144; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=P9RalIq5/KBIQXS8hw4c9tUFcC4ZusIbNEKpuKw1Hyg=; b=r+o+BSnqk6KK5pXeALDLt9w5JEhIrEWecI7cG7lRyPyYqA7vxRhFCXQ+viqRdQW9wC +ncgJqqWPFn7tdALLAXLCuNd8/dIHHxhlOXGLlnYSFpWNuNPLN1prF9g8HMX6v0LRr/c 5UxUs33Q6PfVP7CQKae8/iudZDGf+/fQT/E2vt98CltHBPVYTCcPR9khwLwAEEut52cP xUgPdC3qDHF0gVNS9lPCwklsPdzYldTdNQ9UvaEpieLrQw2fBgzkVn6aC88vAm3Eg3Ml Nrmjlltgn2/UkKaVj3BFpzdkB6g+FFO6nZrLoxIMJTE8n1DLi9dnH4vxKmYI47EM1v3H LIfg== X-Gm-Message-State: AOJu0YzFZe5QY8iipqbYFRP+dmnU13RFn1ud9xzZYMYrIk1AleRxQuJd 7m/+mTGKjq/NtW15oN2Y32a0dA5c5L172sq7IP6fWLKkHRJ/rGevXyFhvXndzg== X-Gm-Gg: ASbGnctD3BLm8jY3XiSae7jLv4S9mQc7SSELYNUlyFDpwO/gJzHBRjWUBYW/B7PYcWP kbdVKY/HZ16J3St0nZugPAdBif9NjrYa3o2pMbwyP7um8ng8Z1z5zHH2jfuLd5XfAKIaDSU1n1O IhlSBcLFyBg31rcTpdBJwaaHWLqX800RcI0kjo1dx9frbJeYlMtxKb/t2bKq45F+YNYnPlRn75X M2wZ8J7uwAUzIXtvAQxyhuQ1IEyez2ypkeCpe3R1OWVou7dUv3cI8FUxn8bI93aVuv2CyoQpbGy RQEb3/u8I+K4PXOWOFH9IMBajBIfBO0bVf1nhBAxrdnk59/7XhDXRohWy0Qod1t1Y8uuhme8XkN fOJOI5CecSIa7ziWLh+lb1cO1J/ETQgLswUADIm0ByfdZjn4FhXfAVM8ZvX5IAQCQ64g= X-Google-Smtp-Source: AGHT+IEtFQqkM1ql8OqSxn4kTFyvYW+XZoshQH+piJx543uLlQd9H+DlAyUW1k/3V4dpjWvOZpVLzQ== X-Received: by 2002:a17:907:2d2c:b0:ae3:8c9b:bd61 with SMTP id a640c23a62f3a-afa1dff772amr72501366b.12.1754945344280; Mon, 11 Aug 2025 13:49:04 -0700 (PDT) From: Mykola Kvach To: xen-devel@lists.xenproject.org Cc: Mirela Simonovic , Stefano Stabellini , Julien Grall , Bertrand Marquis , Michal Orzel , Volodymyr Babchuk , Saeed Nowshadi , Mykyta Poturai , Mykola Kvach Subject: [PATCH v5 10/12] xen/arm: Save/restore context on suspend/resume Date: Mon, 11 Aug 2025 23:48:06 +0300 Message-ID: <5441d6712a4ca0e61fd066606b9a96414123aaad.1754943875.git.mykola_kvach@epam.com> X-Mailer: git-send-email 2.48.1 In-Reply-To: References: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @gmail.com) X-ZM-MESSAGEID: 1754945369918124100 Content-Type: text/plain; charset="utf-8" From: Mirela Simonovic The context of CPU general purpose and system control registers has to be saved on suspend and restored on resume. This is implemented in hyp_suspend and before the return from hyp_resume function. The hyp_suspend is invoked just before the PSCI system suspend call is issued to the ATF. The hyp_suspend has to return a non-zero value so that the calling 'if' statement evaluates to true, causing the system suspend to be invoked. Upon the resume, context saved on suspend will be restored, including the link register. Therefore, after restoring the context the control flow will return to the address pointed by the saved link register, which is the place from which the hyp_suspend was called. To ensure that the calling 'if' statement doesn't again evaluate to true and initiate system suspend, hyp_resume has to return a zero value after restoring the context. Note that the order of saving register context into cpu_context structure has to match the order of restoring. Support for ARM32 is not implemented. Instead, compilation fails with a build-time error if suspend is enabled for ARM32. Signed-off-by: Mirela Simonovic Signed-off-by: Saeed Nowshadi Signed-off-by: Mykyta Poturai Signed-off-by: Mykola Kvach --- Changes in v4: - produce build-time error for ARM32 when CONFIG_SYSTEM_SUSPEND is enabled - use register_t instead of uint64_t in cpu_context structure --- xen/arch/arm/arm64/head.S | 91 +++++++++++++++++++++++++++++- xen/arch/arm/include/asm/suspend.h | 20 +++++++ xen/arch/arm/suspend.c | 23 +++++++- 3 files changed, 130 insertions(+), 4 deletions(-) diff --git a/xen/arch/arm/arm64/head.S b/xen/arch/arm/arm64/head.S index 596e960152..ad8b48de3a 100644 --- a/xen/arch/arm/arm64/head.S +++ b/xen/arch/arm/arm64/head.S @@ -562,6 +562,52 @@ END(efi_xen_start) #endif /* CONFIG_ARM_EFI */ =20 #ifdef CONFIG_SYSTEM_SUSPEND +/* + * int hyp_suspend(struct cpu_context *ptr) + * + * x0 - pointer to the storage where callee's context will be saved + * + * CPU context saved here will be restored on resume in hyp_resume functio= n. + * hyp_suspend shall return a non-zero value. Upon restoring context + * hyp_resume shall return value zero instead. From C code that invokes + * hyp_suspend, the return value is interpreted to determine whether the c= ontext + * is saved (hyp_suspend) or restored (hyp_resume). + */ +FUNC(hyp_suspend) + /* Store callee-saved registers */ + stp x19, x20, [x0], #16 + stp x21, x22, [x0], #16 + stp x23, x24, [x0], #16 + stp x25, x26, [x0], #16 + stp x27, x28, [x0], #16 + stp x29, lr, [x0], #16 + + /* Store stack-pointer */ + mov x2, sp + str x2, [x0], #8 + + /* Store system control registers */ + mrs x2, VBAR_EL2 + str x2, [x0], #8 + mrs x2, VTCR_EL2 + str x2, [x0], #8 + mrs x2, VTTBR_EL2 + str x2, [x0], #8 + mrs x2, TPIDR_EL2 + str x2, [x0], #8 + mrs x2, MDCR_EL2 + str x2, [x0], #8 + mrs x2, HSTR_EL2 + str x2, [x0], #8 + mrs x2, CPTR_EL2 + str x2, [x0], #8 + mrs x2, HCR_EL2 + str x2, [x0], #8 + + /* hyp_suspend must return a non-zero value */ + mov x0, #1 + ret +END(hyp_suspend) =20 FUNC(hyp_resume) /* Initialize the UART if earlyprintk has been enabled. */ @@ -580,7 +626,50 @@ FUNC(hyp_resume) b enable_secondary_cpu_mm =20 mmu_resumed: - b . + /* + * Now we can access the cpu_context, so restore the context here + * TODO: can we reuse __context_switch and saved_context struct he= re ? + */ + ldr x0, =3Dcpu_context + + /* Restore callee-saved registers */ + ldp x19, x20, [x0], #16 + ldp x21, x22, [x0], #16 + ldp x23, x24, [x0], #16 + ldp x25, x26, [x0], #16 + ldp x27, x28, [x0], #16 + ldp x29, lr, [x0], #16 + + /* Restore stack pointer */ + ldr x2, [x0], #8 + mov sp, x2 + + /* Restore system control registers */ + ldr x2, [x0], #8 + msr VBAR_EL2, x2 + ldr x2, [x0], #8 + msr VTCR_EL2, x2 + ldr x2, [x0], #8 + msr VTTBR_EL2, x2 + ldr x2, [x0], #8 + msr TPIDR_EL2, x2 + ldr x2, [x0], #8 + msr MDCR_EL2, x2 + ldr x2, [x0], #8 + msr HSTR_EL2, x2 + ldr x2, [x0], #8 + msr CPTR_EL2, x2 + ldr x2, [x0], #8 + msr HCR_EL2, x2 + isb + + /* Since context is restored return from this function will appear= as + * return from hyp_suspend. To distinguish a return from hyp_suspe= nd + * which is called upon finalizing the suspend, as opposed to retu= rn + * from this function which executes on resume, we need to return = zero + * value here. */ + mov x0, #0 + ret END(hyp_resume) =20 #endif /* CONFIG_SYSTEM_SUSPEND */ diff --git a/xen/arch/arm/include/asm/suspend.h b/xen/arch/arm/include/asm/= suspend.h index 55041a5d06..ae71ccb87b 100644 --- a/xen/arch/arm/include/asm/suspend.h +++ b/xen/arch/arm/include/asm/suspend.h @@ -5,9 +5,29 @@ =20 #ifdef CONFIG_SYSTEM_SUSPEND =20 +#ifdef CONFIG_ARM_64 +struct cpu_context { + register_t callee_regs[12]; + register_t sp; + register_t vbar_el2; + register_t vtcr_el2; + register_t vttbr_el2; + register_t tpidr_el2; + register_t mdcr_el2; + register_t hstr_el2; + register_t cptr_el2; + register_t hcr_el2; +} __aligned(16); +#else +#error "Define cpu_context structure for arm32" +#endif + +extern struct cpu_context cpu_context; + int host_system_suspend(void); =20 void hyp_resume(void); +int hyp_suspend(struct cpu_context *ptr); =20 #endif /* CONFIG_SYSTEM_SUSPEND */ =20 diff --git a/xen/arch/arm/suspend.c b/xen/arch/arm/suspend.c index 08b6acaede..b5398e5ca6 100644 --- a/xen/arch/arm/suspend.c +++ b/xen/arch/arm/suspend.c @@ -1,6 +1,7 @@ /* SPDX-License-Identifier: GPL-2.0-only */ =20 #include +#include #include #include #include @@ -17,6 +18,8 @@ * - Investigate feasibility and need for implementing system suspend on = ARM32 */ =20 +struct cpu_context cpu_context; + /* Xen suspend. Note: data is not used (suspend is the suspend to RAM) */ static long system_suspend(void *data) { @@ -73,9 +76,23 @@ static long system_suspend(void *data) */ update_boot_mapping(true); =20 - status =3D call_psci_system_suspend(); - if ( status ) - dprintk(XENLOG_WARNING, "PSCI system suspend failed, err=3D%d\n", = status); + if ( hyp_suspend(&cpu_context) ) + { + status =3D call_psci_system_suspend(); + /* + * If suspend is finalized properly by above system suspend PSCI c= all, + * the code below in this 'if' branch will never execute. Execution + * will continue from hyp_resume which is the hypervisor's resume = point. + * In hyp_resume CPU context will be restored and since link-regis= ter is + * restored as well, it will appear to return from hyp_suspend. The + * difference in returning from hyp_suspend on system suspend vers= us + * resume is in function's return value: on suspend, the return va= lue is + * a non-zero value, on resume it is zero. That is why the control= flow + * will not re-enter this 'if' branch on resume. + */ + if ( status ) + dprintk(XENLOG_WARNING, "PSCI system suspend failed, err=3D%d\= n", status); + } =20 system_state =3D SYS_STATE_resume; update_boot_mapping(false); --=20 2.48.1 From nobody Thu Oct 30 22:53:02 2025 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass(p=none dis=none) header.from=gmail.com ARC-Seal: i=1; a=rsa-sha256; t=1754945479; cv=none; d=zohomail.com; s=zohoarc; b=aWWmOZZcrva4YkUAT8L/q5nnbwUDSFr/Y9TN5ldmNNBzkkbLPMxQHiUtUu6kGr8195TOt9ors+X9bBn4FiWGvG5V8xohctDNXuWcDUGYczX9dy0GRERIB6I5kI86RjaZZsbgiyaP5/7JCIwvT16FZOPPGMov/DvgDKAzuMO6RNo= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1754945479; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=DKWyc9yALBseWqY+orGA6xUBZdWqywovXSPAegimSF0=; b=DhhOsLL1N9AfvFcSA3OPCcx1S125WrT8shrJ2QOOcHXACPFheSnFyKvamoHfJ8WCHAvMd3mWJbpeyChX6dxh0RgDNkLOCHBShzxeQBXfOmSGOg6cO4KLV0s/2O+GnvW398RS8prYBa4yhylU8TcYZOn913MabfHi5hqXY5qjJRc= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1754945479104823.6920053621635; Mon, 11 Aug 2025 13:51:19 -0700 (PDT) Received: from list by lists.xenproject.org with outflank-mailman.1078031.1439056 (Exim 4.92) (envelope-from ) id 1ulZU2-00048P-PI; Mon, 11 Aug 2025 20:51:06 +0000 Received: by outflank-mailman (output) from mailman id 1078031.1439056; Mon, 11 Aug 2025 20:51:06 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1ulZU2-00048I-K4; Mon, 11 Aug 2025 20:51:06 +0000 Received: by outflank-mailman (input) for mailman id 1078031; Mon, 11 Aug 2025 20:51:05 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1ulZSA-00063z-2n for xen-devel@lists.xenproject.org; Mon, 11 Aug 2025 20:49:10 +0000 Received: from mail-ej1-x634.google.com (mail-ej1-x634.google.com [2a00:1450:4864:20::634]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id 9fa45a96-76f4-11f0-b898-0df219b8e170; Mon, 11 Aug 2025 22:49:07 +0200 (CEST) Received: by mail-ej1-x634.google.com with SMTP id a640c23a62f3a-af94e75445dso864827266b.0 for ; Mon, 11 Aug 2025 13:49:07 -0700 (PDT) Received: from yp-VivoBook-ASUSLaptop-M1503QA-M1503QA.Dlink ([77.52.179.38]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-af91a0a3bd2sm2065710466b.54.2025.08.11.13.49.04 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 11 Aug 2025 13:49:05 -0700 (PDT) X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 9fa45a96-76f4-11f0-b898-0df219b8e170 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1754945346; x=1755550146; darn=lists.xenproject.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=DKWyc9yALBseWqY+orGA6xUBZdWqywovXSPAegimSF0=; b=JY7DhqLC4jP6D+BTpGs5YLEur644BkZv1h+Bk53KRz3ABuQioO1y9u7pa1aRh8kMh/ jZQygWQBY4jCk0K2uURsJAMZqwPR7ryX08y6a43UpHM66W7z2WIx33SdWEmULYM3uaae 5BeE35Tq89hH7KxNSWQiuyDcgsR0VdpLn49bC+gSQgUyNOYzXz7iKlfoTJhjhciEjIp9 GzJwOZQk6UuE/VfleSPTEkCnci/r085kxpwlnZZu9H5J3vpbbSJewhwmsVJMKGUxyl7a YOxVIwIBdKQ69MGMpsbW0DGMqwn7AXus4RP9ozYuZbzOFVoCwhqpssxxJrabUpBRCeIs fWoA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1754945346; x=1755550146; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=DKWyc9yALBseWqY+orGA6xUBZdWqywovXSPAegimSF0=; b=mS2wys4dx9zXodZmWxgk+zbkFzuIvyNeZC2tb15BR/oeysnqnkGjqc05u1DVrHJJsY WCzzwS6Wtq8wucsGRqZD9pW2E35WCF4bXMH2THA8u9X0h1DAkt3ipyKJwjk4jz2AJSMr WfzaDoPfP4mhUXNslTFJKMn8YfNxSmN6L+7nprxZMo3/HmFU13ut4AW8MvArqG/DocNg LGi13hythkxnTULxveh9UWEDSqGu5tGGrFFAOymQz6s9dslHO48Qgxi7RrxSC8z8KSFf cduveHVhOOl1PLrEiZ3PDnqJ/kdMzhKyrjLcvME1sTMlesnenzyZg72a2JhuJaBofklK p7Cg== X-Gm-Message-State: AOJu0Yzm1A8/JuVc9nqWbQViG03vRheCVt/h4unM82DvsZF6tgdn3Mte Kp3ggu6+92g3LkZEcuNY05PCIP8+EN2RPZruypIJRG0JaZY2xqVA7+WejuufpQ== X-Gm-Gg: ASbGncsL9GD1VP5PT77/pDqHJF9f5RPfO2y5hAwZxDvL3yAf8eS2ZSENdwnSfYzKU/h bp+aO/gQe31uMIuReoNnoEOOLcuyo4wNbF2gTU6tpY9s6cjTVOcyvCLOgf/h8YKftEr3JDEPrHX bmhVyRtUscOtg4GJtACBFbZLGbTwPWIG5CnH2TxVSBQ4s7DYL5H4Cv1RldTFWxsVuShmIfL8Tv4 PXvEOmJKjcATpaRBOcbHPCdEdnGowJod7YeHPduHzNY7mfQ7txj6vUZ0+4Di9VqmrVSVT1YhEiG QCdojfX1a2WTkNY0rH7g/T2u2zAssraAXdQDHxLiPYnb5Lgh0kD4RvlNBe2e56ITx+eysZSfZRo mx4oVBT0X36WSN4aIeSDHAXr4S1tfxY4sk5RCbjc+BelDd4ArzIE+sRtS7lI55H9mtmpbdTXoQv ioFA== X-Google-Smtp-Source: AGHT+IEvypqo7NpLmFUTRzjOWjD+M/zWBTlCHQtbMInufWtjBmIYbsnsHqopRx+UCqdZKj8BBLeoGw== X-Received: by 2002:a17:906:ef0b:b0:ade:4121:8d52 with SMTP id a640c23a62f3a-afa1e040d30mr79910766b.16.1754945346249; Mon, 11 Aug 2025 13:49:06 -0700 (PDT) From: Mykola Kvach To: xen-devel@lists.xenproject.org Cc: Oleksandr Tyshchenko , Stefano Stabellini , Julien Grall , Bertrand Marquis , Michal Orzel , Volodymyr Babchuk , Mykola Kvach Subject: [PATCH v5 11/12] iommu/ipmmu-vmsa: Implement suspend/resume callbacks Date: Mon, 11 Aug 2025 23:48:07 +0300 Message-ID: <1689d707b930b1ea4f63cc150810e548962cda81.1754943875.git.mykola_kvach@epam.com> X-Mailer: git-send-email 2.48.1 In-Reply-To: References: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @gmail.com) X-ZM-MESSAGEID: 1754945503869124100 Content-Type: text/plain; charset="utf-8" From: Oleksandr Tyshchenko Store and restore active context and micro-TLB registers. Tested on R-Car H3 Starter Kit. Signed-off-by: Oleksandr Tyshchenko Signed-off-by: Mykola Kvach --- xen/drivers/passthrough/arm/ipmmu-vmsa.c | 269 +++++++++++++++++++++++ 1 file changed, 269 insertions(+) diff --git a/xen/drivers/passthrough/arm/ipmmu-vmsa.c b/xen/drivers/passthr= ough/arm/ipmmu-vmsa.c index dac0dd6d46..ced762657a 100644 --- a/xen/drivers/passthrough/arm/ipmmu-vmsa.c +++ b/xen/drivers/passthrough/arm/ipmmu-vmsa.c @@ -58,6 +58,8 @@ #define dev_print(dev, lvl, fmt, ...) \ printk(lvl "ipmmu: %s: " fmt, dev_name(dev), ## __VA_ARGS__) =20 +#define dev_dbg(dev, fmt, ...) \ + dev_print(dev, XENLOG_DEBUG, fmt, ## __VA_ARGS__) #define dev_info(dev, fmt, ...) \ dev_print(dev, XENLOG_INFO, fmt, ## __VA_ARGS__) #define dev_warn(dev, fmt, ...) \ @@ -117,6 +119,23 @@ struct ipmmu_features { unsigned int imuctr_ttsel_mask; }; =20 +#ifdef CONFIG_SYSTEM_SUSPEND + +struct hw_register { + const char *reg_name; + unsigned int reg_offset; + unsigned int reg_data; +}; + +struct ipmmu_vmsa_backup { + struct device *dev; + unsigned int *utlbs_val; + unsigned int *asids_val; + struct list_head list; +}; + +#endif + /* Root/Cache IPMMU device's information */ struct ipmmu_vmsa_device { struct device *dev; @@ -129,6 +148,9 @@ struct ipmmu_vmsa_device { struct ipmmu_vmsa_domain *domains[IPMMU_CTX_MAX]; unsigned int utlb_refcount[IPMMU_UTLB_MAX]; const struct ipmmu_features *features; +#ifdef CONFIG_SYSTEM_SUSPEND + struct hw_register *reg_backup[IPMMU_CTX_MAX]; +#endif }; =20 /* @@ -534,6 +556,235 @@ static void ipmmu_domain_free_context(struct ipmmu_vm= sa_device *mmu, spin_unlock_irqrestore(&mmu->lock, flags); } =20 +#ifdef CONFIG_SYSTEM_SUSPEND + +static DEFINE_SPINLOCK(ipmmu_devices_backup_lock); +static LIST_HEAD(ipmmu_devices_backup); + +#define HW_REGISTER_BACKUP_SIZE 4 + +static struct hw_register root_pgtable[IPMMU_CTX_MAX][HW_REGISTER_BACKUP_S= IZE] =3D { + [0 ... (IPMMU_CTX_MAX - 1)] =3D { + {"IMTTLBR0", IMTTLBR0, 0}, + {"IMTTUBR0", IMTTUBR0, 0}, + {"IMTTBCR", IMTTBCR, 0}, + {"IMCTR", IMCTR, 0}, + } +}; + +static uint32_t ipmmu_imuasid_read(struct ipmmu_vmsa_device *mmu, + unsigned int utlb) +{ + return ipmmu_read(mmu, ipmmu_utlb_reg(mmu, IMUASID(utlb))); +} + +static void ipmmu_utlbs_backup(struct ipmmu_vmsa_device *mmu) +{ + struct ipmmu_vmsa_backup *backup_data; + + dev_dbg(mmu->dev, "Handle micro-TLBs backup\n"); + + spin_lock(&ipmmu_devices_backup_lock); + + list_for_each_entry( backup_data, &ipmmu_devices_backup, list ) + { + struct iommu_fwspec *fwspec =3D dev_iommu_fwspec_get(backup_data->= dev); + unsigned int i; + + if ( to_ipmmu(backup_data->dev) !=3D mmu ) + continue; + + for ( i =3D 0; i < fwspec->num_ids; i++ ) + { + unsigned int utlb =3D fwspec->ids[i]; + + backup_data->asids_val[i] =3D ipmmu_imuasid_read(mmu, utlb); + backup_data->utlbs_val[i] =3D ipmmu_imuctr_read(mmu, utlb); + } + } + + spin_unlock(&ipmmu_devices_backup_lock); +} + +static void ipmmu_utlbs_restore(struct ipmmu_vmsa_device *mmu) +{ + struct ipmmu_vmsa_backup *backup_data; + + dev_dbg(mmu->dev, "Handle micro-TLBs restore\n"); + + spin_lock(&ipmmu_devices_backup_lock); + + list_for_each_entry( backup_data, &ipmmu_devices_backup, list ) + { + struct iommu_fwspec *fwspec =3D dev_iommu_fwspec_get(backup_data->= dev); + unsigned int i; + + if ( to_ipmmu(backup_data->dev) !=3D mmu ) + continue; + + for ( i =3D 0; i < fwspec->num_ids; i++ ) + { + unsigned int utlb =3D fwspec->ids[i]; + + ipmmu_imuasid_write(mmu, utlb, backup_data->asids_val[i]); + ipmmu_imuctr_write(mmu, utlb, backup_data->utlbs_val[i]); + } + } + + spin_unlock(&ipmmu_devices_backup_lock); +} + +static void ipmmu_domain_backup_context(struct ipmmu_vmsa_domain *domain) +{ + struct ipmmu_vmsa_device *mmu =3D domain->mmu->root; + struct hw_register *reg =3D mmu->reg_backup[domain->context_id]; + unsigned int i; + + dev_dbg(mmu->dev, "Handle domain context %u backup\n", domain->context= _id); + + for ( i =3D 0; i < HW_REGISTER_BACKUP_SIZE; i++ ) + reg[i].reg_data =3D ipmmu_ctx_read_root(domain, reg[i].reg_offset); +} + +static void ipmmu_domain_restore_context(struct ipmmu_vmsa_domain *domain) +{ + struct ipmmu_vmsa_device *mmu =3D domain->mmu->root; + struct hw_register *reg =3D mmu->reg_backup[domain->context_id]; + unsigned int i; + + dev_dbg(mmu->dev, "Handle domain context %u restore\n", domain->contex= t_id); + + for ( i =3D 0; i < HW_REGISTER_BACKUP_SIZE; i++ ) + { + if ( reg[i].reg_offset !=3D IMCTR ) + ipmmu_ctx_write_root(domain, reg[i].reg_offset, reg[i].reg_dat= a); + else + ipmmu_ctx_write_all(domain, reg[i].reg_offset, + reg[i].reg_data | IMCTR_FLUSH); + } +} + +/* + * Xen: Unlike Linux implementation, Xen uses a single driver instance + * for handling all IPMMUs. There is no framework for ipmmu_suspend/resume + * callbacks to be invoked for each IPMMU device. So, we need to iterate + * through all registered IPMMUs performing required actions. + * + * Also take care of restoring special settings, such as translation + * table format, etc. + */ +static int __must_check ipmmu_suspend(void) +{ + struct ipmmu_vmsa_device *mmu; + + if ( !iommu_enabled ) + return 0; + + printk(XENLOG_DEBUG "ipmmu: Suspending ...\n"); + + spin_lock(&ipmmu_devices_lock); + + list_for_each_entry( mmu, &ipmmu_devices, list ) + { + if ( ipmmu_is_root(mmu) ) + { + unsigned int i; + + for ( i =3D 0; i < mmu->num_ctx; i++ ) + { + if ( !mmu->domains[i] ) + continue; + ipmmu_domain_backup_context(mmu->domains[i]); + } + } + else + ipmmu_utlbs_backup(mmu); + } + + spin_unlock(&ipmmu_devices_lock); + + return 0; +} + +static void ipmmu_resume(void) +{ + struct ipmmu_vmsa_device *mmu; + + if ( !iommu_enabled ) + return; + + printk(XENLOG_DEBUG "ipmmu: Resuming ...\n"); + + spin_lock(&ipmmu_devices_lock); + + list_for_each_entry( mmu, &ipmmu_devices, list ) + { + uint32_t reg; + + /* Do not use security group function */ + reg =3D IMSCTLR + mmu->features->control_offset_base; + ipmmu_write(mmu, reg, ipmmu_read(mmu, reg) & ~IMSCTLR_USE_SECGRP); + + if ( ipmmu_is_root(mmu) ) + { + unsigned int i; + + /* Use stage 2 translation table format */ + reg =3D IMSAUXCTLR + mmu->features->control_offset_base; + ipmmu_write(mmu, reg, ipmmu_read(mmu, reg) | IMSAUXCTLR_S2PTE); + + for ( i =3D 0; i < mmu->num_ctx; i++ ) + { + if ( !mmu->domains[i] ) + continue; + ipmmu_domain_restore_context(mmu->domains[i]); + } + } + else + ipmmu_utlbs_restore(mmu); + } + + spin_unlock(&ipmmu_devices_lock); +} + +static int ipmmu_alloc_ctx_suspend(struct device *dev) +{ + struct ipmmu_vmsa_backup *backup_data; + unsigned int *utlbs_val, *asids_val; + struct iommu_fwspec *fwspec =3D dev_iommu_fwspec_get(dev); + + utlbs_val =3D xzalloc_array(unsigned int, fwspec->num_ids); + if ( !utlbs_val ) + return -ENOMEM; + + asids_val =3D xzalloc_array(unsigned int, fwspec->num_ids); + if ( !asids_val ) + { + xfree(utlbs_val); + return -ENOMEM; + } + + backup_data =3D xzalloc(struct ipmmu_vmsa_backup); + if ( !backup_data ) + { + xfree(utlbs_val); + xfree(asids_val); + return -ENOMEM; + } + + backup_data->dev =3D dev; + backup_data->utlbs_val =3D utlbs_val; + backup_data->asids_val =3D asids_val; + + spin_lock(&ipmmu_devices_backup_lock); + list_add(&backup_data->list, &ipmmu_devices_backup); + spin_unlock(&ipmmu_devices_backup_lock); + + return 0; +} + +#endif /* CONFIG_SYSTEM_SUSPEND */ + static int ipmmu_domain_init_context(struct ipmmu_vmsa_domain *domain) { uint64_t ttbr; @@ -546,6 +797,9 @@ static int ipmmu_domain_init_context(struct ipmmu_vmsa_= domain *domain) return ret; =20 domain->context_id =3D ret; +#ifdef CONFIG_SYSTEM_SUSPEND + domain->mmu->root->reg_backup[ret] =3D root_pgtable[ret]; +#endif =20 /* * TTBR0 @@ -602,6 +856,9 @@ static void ipmmu_domain_destroy_context(struct ipmmu_v= msa_domain *domain) ipmmu_ctx_write_root(domain, IMCTR, IMCTR_FLUSH); ipmmu_tlb_sync(domain); =20 +#ifdef CONFIG_SYSTEM_SUSPEND + domain->mmu->root->reg_backup[domain->context_id] =3D NULL; +#endif ipmmu_domain_free_context(domain->mmu->root, domain->context_id); } =20 @@ -1307,6 +1564,14 @@ static int ipmmu_add_device(u8 devfn, struct device = *dev) /* Let Xen know that the master device is protected by an IOMMU. */ dt_device_set_protected(dev_to_dt(dev)); =20 +#ifdef CONFIG_SYSTEM_SUSPEND + if ( ipmmu_alloc_ctx_suspend(dev) ) + { + dev_err(dev, "Failed to allocate context for suspend\n"); + return -ENOMEM; + } +#endif + dev_info(dev, "Added master device (IPMMU %s micro-TLBs %u)\n", dev_name(fwspec->iommu_dev), fwspec->num_ids); =20 @@ -1372,6 +1637,10 @@ static const struct iommu_ops ipmmu_iommu_ops =3D .unmap_page =3D arm_iommu_unmap_page, .dt_xlate =3D ipmmu_dt_xlate, .add_device =3D ipmmu_add_device, +#ifdef CONFIG_SYSTEM_SUSPEND + .suspend =3D ipmmu_suspend, + .resume =3D ipmmu_resume, +#endif }; =20 static __init int ipmmu_init(struct dt_device_node *node, const void *data) --=20 2.48.1 From nobody Thu Oct 30 22:53:02 2025 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass(p=none dis=none) header.from=gmail.com ARC-Seal: i=1; a=rsa-sha256; t=1754945479; cv=none; d=zohomail.com; s=zohoarc; b=m2rS3kxd/i/rsY1sWTD2xubRWqDj5lVfeTnf7GKKtAwaOA+9R2jbpe4oSmmbpN9PsPr+e7+WKKi0IbRxYR/SQ9V9Kuy6CmP/qFL1/dDEOCPEvF0xicGBdVNsxmjh/Z2WZ69c2Ts7ttuMIMsetHoD0rdnIqARV2a9VpJoT8pPj20= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1754945479; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=dIP/M+K6/8Gg84x9uEVu6zbj0BNJ3eSed4CBkvQHa9c=; b=PX/HQsG9OQVJn/FGrcK1jEfdxewM1YVfSQ333MgemFGkB/iSnuCGk14rnAko8pzrAwUO3E1Y05Lo470VvMgQXRFMBfsEzXuXCpMlfCtYjnpQL1nA9O9JlJbdexuuR8FU9j3UWlBet9OWIt62ORO/jkBkmnx82/zxZykggUxiLVw= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1754945479679763.8442199599098; Mon, 11 Aug 2025 13:51:19 -0700 (PDT) Received: from list by lists.xenproject.org with outflank-mailman.1078034.1439060 (Exim 4.92) (envelope-from ) id 1ulZU2-00049z-VX; Mon, 11 Aug 2025 20:51:06 +0000 Received: by outflank-mailman (output) from mailman id 1078034.1439060; Mon, 11 Aug 2025 20:51:06 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1ulZU2-000495-R5; Mon, 11 Aug 2025 20:51:06 +0000 Received: by outflank-mailman (input) for mailman id 1078034; Mon, 11 Aug 2025 20:51:06 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1ulZSC-00063z-2W for xen-devel@lists.xenproject.org; Mon, 11 Aug 2025 20:49:12 +0000 Received: from mail-ej1-x635.google.com (mail-ej1-x635.google.com [2a00:1450:4864:20::635]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id a0ebe8ef-76f4-11f0-b898-0df219b8e170; Mon, 11 Aug 2025 22:49:09 +0200 (CEST) Received: by mail-ej1-x635.google.com with SMTP id a640c23a62f3a-af96fba3b37so901496666b.3 for ; Mon, 11 Aug 2025 13:49:09 -0700 (PDT) Received: from yp-VivoBook-ASUSLaptop-M1503QA-M1503QA.Dlink ([77.52.179.38]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-af91a0a3bd2sm2065710466b.54.2025.08.11.13.49.06 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 11 Aug 2025 13:49:07 -0700 (PDT) X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: a0ebe8ef-76f4-11f0-b898-0df219b8e170 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1754945348; x=1755550148; darn=lists.xenproject.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=dIP/M+K6/8Gg84x9uEVu6zbj0BNJ3eSed4CBkvQHa9c=; b=VWcGAPZBpERMBeTvivvo/Idm/+fmaWcFlyjDHpZlWGzmURa/1uYPnEXB4uOztAdOKu xgt2A3dxWzuQu9sPdim1ptnWfjwwzE1YdJQRHSpJQ99L5wZrtayhbW3pfZfr9r3TsS9H Nnf/hFXUxdVTU/3FrCd98Itolan6xi4pf8wCceQ5aPmSJQkbaBfBDcHBEkNdW9ZdwjvK IwNUs3la0eOxKwhcaDalIwZl8gsuavq3wBvd4DLLS4OHSxvQz3GA4Oow3udUAmgcpqiQ 4ZSzMYjtKRc1ogxveEvfiMkXQ6JLkKad6ZRmh/8XacsEDwgiMkXeKTAFOeteNq3kI260 MRXA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1754945348; x=1755550148; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=dIP/M+K6/8Gg84x9uEVu6zbj0BNJ3eSed4CBkvQHa9c=; b=WJHhhjUX8Q9BKXdr9BAKU0fo6PM0CenhJgRNycezTXMVMWhDCELPsia3TMlX22TUW1 am0dGXhblYypHIwHYC6PoKQmR137onITbkM6wa0WR79onsei3muWAYMM0Gk7JD9qPXe3 tm11xqohfLBsJCuN3EjCQuZoMJ7ANu0yD0XPkpGl+xORRxCJmWPAeRa7I/YqcrULJMEa DNiyAPQO5Rtzlyv9MOmiUlH2WMyvQmCiVNev43BtK5KzlTOYZhmkzp5VZajicZCCzMX7 JWYlLdB03+zOxiohI5xKLGuX1wBI9gD7Z2Af4VwiJha7xAQG4k8HKWqdmCZwR51WVld3 FuKA== X-Gm-Message-State: AOJu0YxClLPgAUf0qe6eqeWdmKZ8f/VbRW2I1PAKnMumpvlt46Iln6nz M/H9MrwC+dmtctK0yPK5+rq50HkzrgKTNcGdMKnpeUO1q2P2vAoERb6Ky4W6Dw== X-Gm-Gg: ASbGncsGho4WsvChTsQRg6FwKnQl8QSbo2UhhrlzeltcmZVpxIx2lpTTg+raTHndCmJ 7+OJv4hK9hO4HN1GggYUwvOy/aKtkVeG2MhOg040mpFWCXr1/JVT7RW2vtDMq/MZSmykybHhAuM IhB65pXKXAitIKJZFc57bT+k6SL1cPRR+c8GxXP/Xq+91t++OR6t42p8vppYlPlKakO3wkK8cj9 f5V98NDTFXqHSOD1Tu3g6cWW3vmAygGPn0PzV3xmohyg35LmyHH/HEktn5gh4+RVu+e2RiOFXtK dur5MALwICrehLr9yHTbqd0tcWSzz6jFxaLet/N8Y1Tq92YhGSRIWEY4M31/0/HtIRLOsuqFVEe oChyBTTUX20oz7HF20jYAT9Zn7+HjP3tamoE9qPRNAvB1VSDyAGz3b9XudaKJam505yM= X-Google-Smtp-Source: AGHT+IEJtWYk61Cx5sHSdxBIm2ViHlPQLCpjRbplQ6lagreygGR+Aj8C787BawE2mZ9+xtZbssyL4A== X-Received: by 2002:a17:907:9811:b0:ad8:a04e:dbd9 with SMTP id a640c23a62f3a-afa1e12c984mr75823566b.31.1754945348421; Mon, 11 Aug 2025 13:49:08 -0700 (PDT) From: Mykola Kvach To: xen-devel@lists.xenproject.org Cc: Oleksandr Tyshchenko , Stefano Stabellini , Julien Grall , Bertrand Marquis , Michal Orzel , Volodymyr Babchuk , Mykola Kvach , Oleksandr Andrushchenko Subject: [PATCH v5 12/12] xen/arm: Suspend/resume IOMMU on Xen suspend/resume Date: Mon, 11 Aug 2025 23:48:08 +0300 Message-ID: <821e0762f64c8b33260a59c672eabf1da5fb4dd4.1754943875.git.mykola_kvach@epam.com> X-Mailer: git-send-email 2.48.1 In-Reply-To: References: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @gmail.com) X-ZM-MESSAGEID: 1754945481331124100 Content-Type: text/plain; charset="utf-8" From: Oleksandr Tyshchenko This is done using generic iommu_suspend/resume functions that cause IOMMU driver specific suspend/resume handlers to be called for enabled IOMMU (if one has suspend/resume driver handlers implemented). These handlers work only when IPMMU-VMSA is used; otherwise, we return an error during system suspend requests. Signed-off-by: Oleksandr Tyshchenko Signed-off-by: Mykola Kvach Tested-by: Oleksandr Andrushchenko --- xen/arch/arm/suspend.c | 21 +++++++++++++++++++++ 1 file changed, 21 insertions(+) diff --git a/xen/arch/arm/suspend.c b/xen/arch/arm/suspend.c index b5398e5ca6..cb86426ebd 100644 --- a/xen/arch/arm/suspend.c +++ b/xen/arch/arm/suspend.c @@ -4,6 +4,7 @@ #include #include #include +#include #include #include =20 @@ -49,6 +50,13 @@ static long system_suspend(void *data) =20 time_suspend(); =20 + status =3D iommu_suspend(); + if ( status ) + { + system_state =3D SYS_STATE_resume; + goto resume_time; + } + local_irq_save(flags); status =3D gic_suspend(); if ( status ) @@ -105,6 +113,10 @@ static long system_suspend(void *data) =20 resume_irqs: local_irq_restore(flags); + + iommu_resume(); + + resume_time: time_resume(); =20 resume_nonboot_cpus: @@ -140,6 +152,15 @@ int host_system_suspend(void) return -ENOSYS; } =20 + /* TODO: drop check once suspend/resume support for SMMU is implemente= d */ +#ifndef CONFIG_IPMMU_VMSA + if ( iommu_enabled ) + { + dprintk(XENLOG_ERR, "IOMMU is enabled, suspend not supported yet\n= "); + return -ENOSYS; + } +#endif + /* * system_suspend should be called when Dom0 finalizes the suspend * procedure from its boot core (VCPU#0). However, Dom0's VCPU#0 could --=20 2.48.1