From nobody Fri Dec 19 19:19:25 2025 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass(p=none dis=none) header.from=gmail.com ARC-Seal: i=1; a=rsa-sha256; t=1748855121; cv=none; d=zohomail.com; s=zohoarc; b=HEgejmYcCZ3nEB0n6UV6VSwPQu9o+QvME97BDyLDcfPrOToDwc4ti5+SLoLD6D3nakj7Cr1LifWZM8p+7XTySoqlSmDJvbpTnP/wVBnc5zHcP5UTKMey5IqkEuvKYsfQ2LLu/1UIuUtyayV9bMVcAj5MDKFQ4o/wPWecti47wP0= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1748855121; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=Da/Gwj5SRn8BTf5+gC6RkaoWtjYeeb+pM91gG3eXP48=; b=YBYe3y5v+RIv1guOa4F6bqiiATkgQ/os2WHj0jHJFIsWm1EaYig0epL2uPP05CmGB1yHzXyfH5OsL81YQe2gk562tz0AoA4V+Zzejj4DI+AQhCDPk221G4q/fFElGAnM3uEtpfJW15venyxOWJo6Y36qvEGe1ND2NF1dgs8flwY= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1748855121200743.0105468009567; Mon, 2 Jun 2025 02:05:21 -0700 (PDT) Received: from list by lists.xenproject.org with outflank-mailman.1002941.1382354 (Exim 4.92) (envelope-from ) id 1uM16E-0007s0-Dw; Mon, 02 Jun 2025 09:04:54 +0000 Received: by outflank-mailman (output) from mailman id 1002941.1382354; Mon, 02 Jun 2025 09:04:54 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1uM16E-0007qN-A1; Mon, 02 Jun 2025 09:04:54 +0000 Received: by outflank-mailman (input) for mailman id 1002941; Mon, 02 Jun 2025 09:04:53 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1uM16D-0007kh-Ch for xen-devel@lists.xenproject.org; Mon, 02 Jun 2025 09:04:53 +0000 Received: from mail-lf1-x12f.google.com (mail-lf1-x12f.google.com [2a00:1450:4864:20::12f]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id a49eed8e-3f90-11f0-b894-0df219b8e170; Mon, 02 Jun 2025 11:04:51 +0200 (CEST) Received: by mail-lf1-x12f.google.com with SMTP id 2adb3069b0e04-5534f3722caso252521e87.0 for ; Mon, 02 Jun 2025 02:04:51 -0700 (PDT) Received: from yp-VivoBook-ASUSLaptop-M1503QA-M1503QA.. ([95.67.15.120]) by smtp.gmail.com with ESMTPSA id 2adb3069b0e04-5533791cdc3sm1507541e87.176.2025.06.02.02.04.50 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 02 Jun 2025 02:04:50 -0700 (PDT) X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: a49eed8e-3f90-11f0-b894-0df219b8e170 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1748855091; x=1749459891; darn=lists.xenproject.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Da/Gwj5SRn8BTf5+gC6RkaoWtjYeeb+pM91gG3eXP48=; b=WR3l8b//HTc/vvydLQNzqKH2jZoqJMoFtrFpzqgrYQPUH6MXU8THOumo8n404ixRI2 RGxFsmuwH1drmlFovrnFeLlMVD9QmHZrTahXTcBz2GCkNowBvyoMj7jbpqM2OQE7GSrR 5e7Vq6k1+T61WtN4QMr2a7V03bO/fOiNge4jWH5bdrAHhTrDpVTCbfJEl6/O8qXdfYM7 kOned9m/DghJ4+GgUfYPUb8So0lS7BphcfRQ6LHJWko98qvx6PGSrzH1PCNpx9ANogjp 1kJUkfEhsw89dQvlrVrG/HkVBQJ+ansNuX4imaCi4dIGXve6mDdUV6D4VT1Dg8p92OMu Sp4g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1748855091; x=1749459891; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Da/Gwj5SRn8BTf5+gC6RkaoWtjYeeb+pM91gG3eXP48=; b=SnFdF6Hxf+muuaNfSSo1bKD6t67vMHAUmp5DMs4wFVQD1Fnn4ZmzjCwk2zdUzIObIC 90jMPSScRlo7n7Mx51j68N6SPkvt2FDzIJdX9xTmdEFoEaITUsC3kNYQIrfnHlYLlcpE UnHfmM12DWCy0O31QgR8gMpJ4HjFKBvKXeA69oOnPKsn5d2f4EoTIPc1UJmmCv8ClY2X LuwD9mO2MBSc+oEeUBsMdAYnJTyaZpEF14ilN8quxfShQFV/vNjLaR2Rg310lvuhGH8h bhWvsvc1XPwr6spj1HLw8hMhmyFRLTX1rrMaeznTNWm8kywJM6gCuTAOZq+yPKSjt16p BqMg== X-Gm-Message-State: AOJu0Yx3wM+sD2QSOmtPfBtvDY2+al5/3jnop+t6tv16o2McSRQ6Yfhm AwvXg2GUVHeGO6xABcqsWhkTEyyjuKHoCeZB2/4PE0TNn5mgAzK7H1jIH34yqdQg X-Gm-Gg: ASbGnctaZ/53f0YRPEPCkBFUhuGY/LMbB9CNOVsWvIyEDN7gdil1RQ7cos1CnxvVATp 8Hxzze1/us7avs4oq0NI+9TmgIx81at9rfNEkIo0R0/q+0Tu1eFSG7IeTT2K/vkV7aGHSpsFAjH D0e8ELsLPFuGqWEHLB6LcwFeB821jratEyDYQXPP3GzbzH6LUgXCto87iuWlgAeDrxtfJWQlg0A eKMfxWc5xzBS8t+0t8NVQzbT16g2H8w/JiEs4ByK15ZrtOYoiB4PkwMD13ru+TGpMi7qQT0k3fV N8H0uQUnUt9AFGHcDWcBx+2DDlxTdvHrnvLOMROT9vN741/xZuvlyqIgCEOXtyhRfzX0bgRKWXH A95bEO59MTam1KZ8= X-Google-Smtp-Source: AGHT+IGSGQlhdjKcLLakl9LRgHis7Wp+dmnc2fkNECPVD/CuspGse8lLtiZIhN0uW5p+NxZ0WdrUqQ== X-Received: by 2002:a05:6512:3b2a:b0:553:3892:5ecc with SMTP id 2adb3069b0e04-55342f928d0mr1769232e87.36.1748855090890; Mon, 02 Jun 2025 02:04:50 -0700 (PDT) From: Mykola Kvach To: xen-devel@lists.xenproject.org Cc: xakep.amatop@gmail.com, Jan Beulich , Andrew Cooper , =?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= , Anthony PERARD , Michal Orzel , Julien Grall , Stefano Stabellini Subject: [PATCH v4][PART 2 01/10] xen/x86: Move freeze/thaw_domains to common code Date: Mon, 2 Jun 2025 12:04:12 +0300 Message-ID: X-Mailer: git-send-email 2.48.1 In-Reply-To: References: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @gmail.com) X-ZM-MESSAGEID: 1748855122083116600 Content-Type: text/plain; charset="utf-8" From: Mirela Simonovic The freeze_domains and thaw_domains functions are currently defined in x86-specific suspend code. These functions are also useful on other architectures, such as ARM, for implementing system-wide suspend and resume functionality. This patch moves these functions to common code so they can be reused across architectures. Signed-off-by: Mirela Simonovic Signed-off-by: Saeed Nowshadi Signed-off-by: Mykyta Poturai Signed-off-by: Mykola Kvach Acked-by: Jan Beulich --- Changes in v4: - Enhanced commit message with a clearer explanation of the rationale for moving freeze/thaw_domains to common code. --- xen/arch/x86/acpi/power.c | 25 ------------------------- xen/common/domain.c | 25 +++++++++++++++++++++++++ xen/include/xen/sched.h | 3 +++ 3 files changed, 28 insertions(+), 25 deletions(-) diff --git a/xen/arch/x86/acpi/power.c b/xen/arch/x86/acpi/power.c index 448aa9f3a7..2ac162c997 100644 --- a/xen/arch/x86/acpi/power.c +++ b/xen/arch/x86/acpi/power.c @@ -138,31 +138,6 @@ static void device_power_up(enum dev_power_saved saved) } } =20 -static void freeze_domains(void) -{ - struct domain *d; - - rcu_read_lock(&domlist_read_lock); - /* - * Note that we iterate in order of domain-id. Hence we will pause dom0 - * first which is required for correctness (as only dom0 can add domai= ns to - * the domain list). Otherwise we could miss concurrently-created doma= ins. - */ - for_each_domain ( d ) - domain_pause(d); - rcu_read_unlock(&domlist_read_lock); -} - -static void thaw_domains(void) -{ - struct domain *d; - - rcu_read_lock(&domlist_read_lock); - for_each_domain ( d ) - domain_unpause(d); - rcu_read_unlock(&domlist_read_lock); -} - static void acpi_sleep_prepare(u32 state) { void *wakeup_vector_va; diff --git a/xen/common/domain.c b/xen/common/domain.c index abf1969e60..58c5ffc466 100644 --- a/xen/common/domain.c +++ b/xen/common/domain.c @@ -2405,6 +2405,31 @@ domid_t get_initial_domain_id(void) return hardware_domid; } =20 +void freeze_domains(void) +{ + struct domain *d; + + rcu_read_lock(&domlist_read_lock); + /* + * Note that we iterate in order of domain-id. Hence we will pause dom0 + * first which is required for correctness (as only dom0 can add domai= ns to + * the domain list). Otherwise we could miss concurrently-created doma= ins. + */ + for_each_domain ( d ) + domain_pause(d); + rcu_read_unlock(&domlist_read_lock); +} + +void thaw_domains(void) +{ + struct domain *d; + + rcu_read_lock(&domlist_read_lock); + for_each_domain ( d ) + domain_unpause(d); + rcu_read_unlock(&domlist_read_lock); +} + /* * Local variables: * mode: C diff --git a/xen/include/xen/sched.h b/xen/include/xen/sched.h index 559d201e0c..071ee19062 100644 --- a/xen/include/xen/sched.h +++ b/xen/include/xen/sched.h @@ -1070,6 +1070,9 @@ static inline struct vcpu *domain_vcpu(const struct d= omain *d, return vcpu_id >=3D d->max_vcpus ? NULL : d->vcpu[idx]; } =20 +void freeze_domains(void); +void thaw_domains(void); + void cpu_init(void); =20 /* --=20 2.48.1 From nobody Fri Dec 19 19:19:25 2025 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass(p=none dis=none) header.from=gmail.com ARC-Seal: i=1; a=rsa-sha256; t=1748855129; cv=none; d=zohomail.com; s=zohoarc; b=UpRtIzcIAb0uLzHam/ktKgpYNA2IjQRDRfNJv0Da5q6onVzVG2Wme/HEpQyzbOWvXdfJZ3pDvN0XJ34wCs14pIYNAyUMir6ERSKZbNIUcuOVtUzuZw8KK9BJjkDy0t77pSzRGcKVDSQ1/aDuk7g5iFCIjpPjX6cQm+3BifUVBx0= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1748855129; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=uqPcozH5cY7WTdc/VNhtz0ZQbmCe6X/v3JhqZHPKizE=; b=ikczmvswDlU+bvDZEgdMiDtgKChXdDTcGw8IAhg+mSfwXHDBZF0T08qq5zBLV0sK4cALzdlACBgX2gpt5ktcI/QxNM+s45jWcQnocKWbVFx3KMibz1uZju99Rg4uwpvkO/J7mVLjI4Vlm2Muqc8kudCIkFYc2zcv7tk/EEJIUis= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1748855129180925.4529406741246; Mon, 2 Jun 2025 02:05:29 -0700 (PDT) Received: from list by lists.xenproject.org with outflank-mailman.1002943.1382378 (Exim 4.92) (envelope-from ) id 1uM16H-0008Rj-Ru; Mon, 02 Jun 2025 09:04:57 +0000 Received: by outflank-mailman (output) from mailman id 1002943.1382378; Mon, 02 Jun 2025 09:04:57 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1uM16H-0008RZ-Oi; Mon, 02 Jun 2025 09:04:57 +0000 Received: by outflank-mailman (input) for mailman id 1002943; Mon, 02 Jun 2025 09:04:56 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1uM16G-0007kh-Gw for xen-devel@lists.xenproject.org; Mon, 02 Jun 2025 09:04:56 +0000 Received: from mail-lf1-x133.google.com (mail-lf1-x133.google.com [2a00:1450:4864:20::133]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id a6756e62-3f90-11f0-b894-0df219b8e170; Mon, 02 Jun 2025 11:04:55 +0200 (CEST) Received: by mail-lf1-x133.google.com with SMTP id 2adb3069b0e04-551fe46934eso4789596e87.1 for ; Mon, 02 Jun 2025 02:04:54 -0700 (PDT) Received: from yp-VivoBook-ASUSLaptop-M1503QA-M1503QA.. ([95.67.15.120]) by smtp.gmail.com with ESMTPSA id 2adb3069b0e04-5533791cdc3sm1507541e87.176.2025.06.02.02.04.51 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 02 Jun 2025 02:04:53 -0700 (PDT) X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: a6756e62-3f90-11f0-b894-0df219b8e170 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1748855094; x=1749459894; darn=lists.xenproject.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=uqPcozH5cY7WTdc/VNhtz0ZQbmCe6X/v3JhqZHPKizE=; b=ZVxmGu10AD80T/aX8tjvpkfnwdwwNWZ1Q5QtVFlL6jgZf9h6HEaAP7nSLceSVp8sTw e103hUe2USC7pmv4VDkpRZpFFXktKXXa9/tEt7Ba5NuqBCw2I6ZZwONAQ8CAulhsusjf 8KEqhmp+azMXiYIiDtmjI6UiYBZc1Mp3TIH8+yVGc+BbN5gA8yw0kGlqxc8t3QSyIrl0 yoikQn8JbkdOg1smVhMpdz1M1JxDzgtV5Fg2JfGGlc/BkmmRq7aZPtQwlSZfWQRkJJpl d4MXRyGm/lysnhMC90IfhbaAyUQEC78wcP2k6wsPtnmZPTydHPM22TI3jpUaLguvGk8F 87Jw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1748855094; x=1749459894; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=uqPcozH5cY7WTdc/VNhtz0ZQbmCe6X/v3JhqZHPKizE=; b=RBIdOQIuGB0gx4Ma8CcNhGaI/6c24zsFlH8+yWWzKMM/KegRY3CZ5Nb9VPkZlxM3fF 5u2QuDOqDI8Zm4LFDUcOxrP6EiZYfFjlujYyoEPp/rUFQndtYA7p1ogUwvMHkuD8pWwv CprKs7P2XAjXorkFehmyQAw4ouMdyIGrnUkPLWU7GFF38GeN2Ou+TSv+FSr0MN0r2RJK 66mdyFzCQtRe8nz3CQWJh4SD0zFqOFtKVsyyErnJserqSm654Q6J8Glj973hbp2FFIPd kM7irSGe6an848BML4u9MMKaeLaJiYP46A6SxO4B3vCykB9gtrFqVFUOG0yncWXeJtQs 4WvA== X-Gm-Message-State: AOJu0YyXrwYTaMcazVjQwp4YFb0yfdt8vHOp+4sb5UxHwrhm6a6AXeRL 1KPegG17nCoHzSYCtGVM4a8I9YkDg0HYab2ZzS6QaTOmlYBqgBcG4Tw+NM7MmvCr X-Gm-Gg: ASbGncut6InjXlKY7uekZRyCE5Hd+zwA7HiAV7BQBdKf3lyKnCrz2vfwJqGXSpdF0il 0xkR6lCYWLH/gedE2XKDpsiOleia8zu0UL966hEZspyCgnpsn4Kq1k12XoaBpcvf/1WgAGgcyvt p47y1fDqvRuHQczh40DOyBd3R/KJh2dybNSu51DN4pi+zqCEAf8/gB64hIk6z/3I3PY7Z0HhY+T 7pbOtw7dSyrrj2VW3SsNDg/yI0909egj26v78oB4sOCT9aKW3iyFwMj5w+8w+JSgaIVGXqoUrQS K25eDyLLmHFMuIQaDFtcHp7p5QTWofJYFmpKUjKW1sf+4Imq3ri2cyWwvWkdsYIzJjpK8qMiaPn Z3CyleiTAZ7Wt9gI= X-Google-Smtp-Source: AGHT+IFEWk5B09gxMmpBvB5cg/vxGWxO+XS2KdbPTd/vJrhO5L40I2jeOSWpJi89b7J3UM5ZWg5JqQ== X-Received: by 2002:a05:6512:3b86:b0:54a:c4af:15 with SMTP id 2adb3069b0e04-5533ba8be3emr3554424e87.19.1748855093947; Mon, 02 Jun 2025 02:04:53 -0700 (PDT) From: Mykola Kvach To: xen-devel@lists.xenproject.org Cc: xakep.amatop@gmail.com, Stefano Stabellini , Julien Grall , Bertrand Marquis , Michal Orzel , Volodymyr Babchuk Subject: [PATCH v4][PART 2 02/10] xen/arm: Add suspend and resume timer helpers Date: Mon, 2 Jun 2025 12:04:13 +0300 Message-ID: X-Mailer: git-send-email 2.48.1 In-Reply-To: References: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @gmail.com) X-ZM-MESSAGEID: 1748855130078116600 Content-Type: text/plain; charset="utf-8" From: Mirela Simonovic Timer interrupts must be disabled while the system is suspended to prevent spurious wake-ups. Suspending the timers involves disabling both the EL1 physical timer and the EL2 hypervisor timer. Resuming consists of raising the TIMER_SOFTIRQ, which prompts the generic timer code to reprogram the EL2 timer as needed. Re-enabling of the EL1 timer is left to the entity that uses it. Introduce a new helper, disable_physical_timers, to encapsulate disabling of the physical timers. Signed-off-by: Mirela Simonovic Signed-off-by: Saeed Nowshadi Signed-off-by: Mykola Kvach --- Changes in V4: - Rephrased comment and commit message for better clarity - Created separate function for disabling physical timers Changes in V3: - time_suspend and time_resume are now conditionally compiled under CONFIG_SYSTEM_SUSPEND --- xen/arch/arm/include/asm/time.h | 5 +++++ xen/arch/arm/time.c | 38 +++++++++++++++++++++++++++------ 2 files changed, 37 insertions(+), 6 deletions(-) diff --git a/xen/arch/arm/include/asm/time.h b/xen/arch/arm/include/asm/tim= e.h index 49ad8c1a6d..f4fd0c6af5 100644 --- a/xen/arch/arm/include/asm/time.h +++ b/xen/arch/arm/include/asm/time.h @@ -108,6 +108,11 @@ void preinit_xen_time(void); =20 void force_update_vcpu_system_time(struct vcpu *v); =20 +#ifdef CONFIG_SYSTEM_SUSPEND +void time_suspend(void); +void time_resume(void); +#endif /* CONFIG_SYSTEM_SUSPEND */ + #endif /* __ARM_TIME_H__ */ /* * Local variables: diff --git a/xen/arch/arm/time.c b/xen/arch/arm/time.c index e74d30d258..ad984fdfdd 100644 --- a/xen/arch/arm/time.c +++ b/xen/arch/arm/time.c @@ -303,6 +303,14 @@ static void check_timer_irq_cfg(unsigned int irq, cons= t char *which) "WARNING: %s-timer IRQ%u is not level triggered.\n", which, irq= ); } =20 +/* Disable physical timers for EL1 and EL2 on the current CPU */ +static inline void disable_physical_timers(void) +{ + WRITE_SYSREG(0, CNTP_CTL_EL0); /* Physical timer disabled */ + WRITE_SYSREG(0, CNTHP_CTL_EL2); /* Hypervisor's timer disabled */ + isb(); +} + /* Set up the timer interrupt on this CPU */ void init_timer_interrupt(void) { @@ -310,9 +318,7 @@ void init_timer_interrupt(void) WRITE_SYSREG64(0, CNTVOFF_EL2); /* No VM-specific offset */ /* Do not let the VMs program the physical timer, only read the physic= al counter */ WRITE_SYSREG(CNTHCTL_EL2_EL1PCTEN, CNTHCTL_EL2); - WRITE_SYSREG(0, CNTP_CTL_EL0); /* Physical timer disabled */ - WRITE_SYSREG(0, CNTHP_CTL_EL2); /* Hypervisor's timer disabled */ - isb(); + disable_physical_timers(); =20 request_irq(timer_irq[TIMER_HYP_PPI], 0, htimer_interrupt, "hyptimer", NULL); @@ -330,9 +336,7 @@ void init_timer_interrupt(void) */ static void deinit_timer_interrupt(void) { - WRITE_SYSREG(0, CNTP_CTL_EL0); /* Disable physical timer */ - WRITE_SYSREG(0, CNTHP_CTL_EL2); /* Disable hypervisor's timer */ - isb(); + disable_physical_timers(); =20 release_irq(timer_irq[TIMER_HYP_PPI], NULL); release_irq(timer_irq[TIMER_VIRT_PPI], NULL); @@ -372,6 +376,28 @@ void domain_set_time_offset(struct domain *d, int64_t = time_offset_seconds) /* XXX update guest visible wallclock time */ } =20 +#ifdef CONFIG_SYSTEM_SUSPEND + +void time_suspend(void) +{ + disable_physical_timers(); +} + +void time_resume(void) +{ + /* + * Raising the timer softirq triggers generic code to call reprogram_t= imer + * with the correct timeout (not known here). + * + * No further action is needed to restore timekeeping after power down, + * since the system counter is unaffected. See ARM DDI 0487 L.a, D12.1= .2 + * "The system counter must be implemented in an always-on power domai= n." + */ + raise_softirq(TIMER_SOFTIRQ); +} + +#endif /* CONFIG_SYSTEM_SUSPEND */ + static int cpu_time_callback(struct notifier_block *nfb, unsigned long action, void *hcpu) --=20 2.48.1 From nobody Fri Dec 19 19:19:25 2025 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass(p=none dis=none) header.from=gmail.com ARC-Seal: i=1; a=rsa-sha256; t=1748855119; cv=none; d=zohomail.com; s=zohoarc; b=HgfOiA9tfPT1kyIRDFm0/86jo1e9ymv2WB53HoIAzChYq9KKTAZGkkoA0nTY74JPMNxGj/blf11JpZ1I/QSBhYNVCOxE1xxd3yyacZthvmxOKq6nH7sJPe4kO6UZzyajgzHfwatBxqsj0iSQ5ou+uIfU4BxH2tGovi0kDUvDtOs= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1748855119; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=Jf3vC/bMLxsK+qQIEq63MqPOkecni+X3pTICciqQJuw=; b=W7h3R8HieKwSNm6Z+yZLwt0VsNj2DJFS5DirV9vmy4yXjVyS8G/6173s9cjfWG7JT+rNhT5dd7FDy1q+lRuaccmBNfA9Jm74+ZEsIH3nxrCFbp0HbjdkpftYTIPfxEQ2SmonYnf/qZ81cigBjEyEle8vLpYi4Wzouk8c26k3Tyo= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1748855119709929.7717304188268; Mon, 2 Jun 2025 02:05:19 -0700 (PDT) Received: from list by lists.xenproject.org with outflank-mailman.1002944.1382383 (Exim 4.92) (envelope-from ) id 1uM16I-0008V5-6Y; Mon, 02 Jun 2025 09:04:58 +0000 Received: by outflank-mailman (output) from mailman id 1002944.1382383; Mon, 02 Jun 2025 09:04:58 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1uM16I-0008Ud-0l; Mon, 02 Jun 2025 09:04:58 +0000 Received: by outflank-mailman (input) for mailman id 1002944; Mon, 02 Jun 2025 09:04:56 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1uM16G-00081k-OJ for xen-devel@lists.xenproject.org; Mon, 02 Jun 2025 09:04:56 +0000 Received: from mail-lf1-x129.google.com (mail-lf1-x129.google.com [2a00:1450:4864:20::129]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id a73e7c91-3f90-11f0-a300-13f23c93f187; Mon, 02 Jun 2025 11:04:56 +0200 (CEST) Received: by mail-lf1-x129.google.com with SMTP id 2adb3069b0e04-55324e35f49so4990016e87.3 for ; Mon, 02 Jun 2025 02:04:56 -0700 (PDT) Received: from yp-VivoBook-ASUSLaptop-M1503QA-M1503QA.. ([95.67.15.120]) by smtp.gmail.com with ESMTPSA id 2adb3069b0e04-5533791cdc3sm1507541e87.176.2025.06.02.02.04.54 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 02 Jun 2025 02:04:54 -0700 (PDT) X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: a73e7c91-3f90-11f0-a300-13f23c93f187 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1748855095; x=1749459895; darn=lists.xenproject.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Jf3vC/bMLxsK+qQIEq63MqPOkecni+X3pTICciqQJuw=; b=JsMjDz/WcVkQycjqVs9dX9i37frKDZThoti3L77TXgIhdUhhOv4iFzfeYhO4mghaSi XPdRFh7kEoExwUV8VC3iKOXE1nL6mKDiCOTGaFQJsQmtq/OzxbMVBAyBIncH5vmPttT+ mQsuok/isvvTJohfqoqkZpvliBg/JNKtuMqnFKxPKGEyXRoVCS6y2MhjfMhL2UrfeeeQ aKafARywnwZMch5KPJ1kiI4YAOfWnB0JmDYNx/WooyFhxQ/ZFpEeX4g9JhqPfjCnvJDe umWujBteDkEGTNMt0e4P8pqroBTQ3E6/KW/L8V5sMHgmRX8egHT1mzAmTjHhsDaBol88 gZ3w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1748855095; x=1749459895; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Jf3vC/bMLxsK+qQIEq63MqPOkecni+X3pTICciqQJuw=; b=pJ25wImjuYFzDH97JQDArTAgV/ae83/oEsZYFaLWJaEsjpEwJoky88lFrM4Xnmgv4B x1h3IOEp8TOonvjkcNIxil/TvKmI4u/uTo1UUaPqanKXbtUl20QsmRs9+1+INSzkoaTE Qtco3qzrCtj3vs7j7IkxCRAiojonUA50OJbWlIPmGF6JpAwer95YDguU1m39PTyuLTVP sZJ7225qFUkXjFyjnj1tF9+1m6p0MHxcATlCcqlRoMEo9psMCNa1jAYG05wQsnfpgBki 7cpZTxXWTmgPrvsCXaSm8PLGZRJRt+WWuf8V+4M9gS2x7M4o2GQz0VV29qFLNwHnt7L0 U4Zw== X-Gm-Message-State: AOJu0YzunoFa8HOLmdwiJrHIFQuBitnJzyYnn55cbim8SaZw24dFswRB 499g7T94BT0bth146cAr3JM3nABaUbQd+ByMNmySipJorKsTAD/8L+qqy4zy0PsO X-Gm-Gg: ASbGncsUbqN2oZ0mQfcSzszc2orshWIrxF1sg+rayu4pbmKT5eIvS6Xqdv8VNyS76UX HGl/bgtbC2NWGqDiEiH5mDsbdEUjf0nn8ZMo+Fcr/JUsbF1kE+GIT+3InL2o3viQik05HI9RFdK T5Fspd/GHQFHlF5QlY0wRYFBKPB0fkzWylu2uS1ads4sifZUaclMBB/k7PAar9WojooWH6kgqr4 TbZ2fCrjFacJJ7LTCBLV9AcoKGCZ6xEdAP8FRkOBsKxmaAFkWAAFOEi5bX+sTQSPo09ia0tZbg4 bZ5sPYJF5ULHUFUIg0FF6JFamAdKveu3+TlcxVEYq2gUrYS50A/aojBh8ie0NT8DILWMeeE90HJ rhKIFyy4fqj3ae/025Y2udGPsVw== X-Google-Smtp-Source: AGHT+IHzhIcJPLhYeyY8gRjIbAaAby/DEUcUcJeunz8LbzATZzYj/HE4uNg6xTTusSuHPyQMGBDzRw== X-Received: by 2002:a05:6512:2313:b0:553:202e:a41b with SMTP id 2adb3069b0e04-5533b8f3e6emr3760415e87.15.1748855095180; Mon, 02 Jun 2025 02:04:55 -0700 (PDT) From: Mykola Kvach To: xen-devel@lists.xenproject.org Cc: xakep.amatop@gmail.com, Stefano Stabellini , Julien Grall , Bertrand Marquis , Michal Orzel , Volodymyr Babchuk Subject: [PATCH v4][PART 2 03/10] xen/arm: Implement GIC suspend/resume functions (gicv2 only) Date: Mon, 2 Jun 2025 12:04:14 +0300 Message-ID: X-Mailer: git-send-email 2.48.1 In-Reply-To: References: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @gmail.com) X-ZM-MESSAGEID: 1748855122276116600 Content-Type: text/plain; charset="utf-8" From: Mirela Simonovic System suspend may lead to a state where GIC would be powered down. Therefore, Xen should save/restore the context of GIC on suspend/resume. Note that the context consists of states of registers which are controlled by the hypervisor. Other GIC registers which are accessible by guests are saved/restored on context switch. Tested on Xilinx Ultrascale+ MPSoC with (and without) powering down the GIC. Signed-off-by: Mirela Simonovic Signed-off-by: Saeed Nowshadi Signed-off-by: Mykyta Poturai Signed-off-by: Mykola Kvach --- Changes in v4: - Add error logging for allocation failures Changes in v3: - Drop asserts and return error codes instead. - Wrap code with CONFIG_SYSTEM_SUSPEND. Changes in v2: - Minor fixes after review. --- xen/arch/arm/gic-v2.c | 154 +++++++++++++++++++++++++++++++++ xen/arch/arm/gic.c | 29 +++++++ xen/arch/arm/include/asm/gic.h | 12 +++ 3 files changed, 195 insertions(+) diff --git a/xen/arch/arm/gic-v2.c b/xen/arch/arm/gic-v2.c index b23e72a3d0..dce8f5e924 100644 --- a/xen/arch/arm/gic-v2.c +++ b/xen/arch/arm/gic-v2.c @@ -1098,6 +1098,151 @@ static int gicv2_iomem_deny_access(struct domain *d) return iomem_deny_access(d, mfn, mfn + nr); } =20 +#ifdef CONFIG_SYSTEM_SUSPEND + +/* GICv2 registers to be saved/restored on system suspend/resume */ +struct gicv2_context { + /* GICC context */ + uint32_t gicc_ctlr; + uint32_t gicc_pmr; + uint32_t gicc_bpr; + /* GICD context */ + uint32_t gicd_ctlr; + uint32_t *gicd_isenabler; + uint32_t *gicd_isactiver; + uint32_t *gicd_ipriorityr; + uint32_t *gicd_itargetsr; + uint32_t *gicd_icfgr; +}; + +static struct gicv2_context gicv2_context; + +static int gicv2_suspend(void) +{ + unsigned int i; + + if ( !gicv2_context.gicd_isenabler ) + { + dprintk(XENLOG_WARNING, "%s:%d: GICv2 suspend context not allocate= d!\n", + __func__, __LINE__); + return -ENOMEM; + } + + /* Save GICC configuration */ + gicv2_context.gicc_ctlr =3D readl_gicc(GICC_CTLR); + gicv2_context.gicc_pmr =3D readl_gicc(GICC_PMR); + gicv2_context.gicc_bpr =3D readl_gicc(GICC_BPR); + + /* Save GICD configuration */ + gicv2_context.gicd_ctlr =3D readl_gicd(GICD_CTLR); + + for ( i =3D 0; i < DIV_ROUND_UP(gicv2_info.nr_lines, 32); i++ ) + gicv2_context.gicd_isenabler[i] =3D readl_gicd(GICD_ISENABLER + i = * 4); + + for ( i =3D 0; i < DIV_ROUND_UP(gicv2_info.nr_lines, 32); i++ ) + gicv2_context.gicd_isactiver[i] =3D readl_gicd(GICD_ISACTIVER + i = * 4); + + for ( i =3D 0; i < DIV_ROUND_UP(gicv2_info.nr_lines, 4); i++ ) + gicv2_context.gicd_ipriorityr[i] =3D readl_gicd(GICD_IPRIORITYR + = i * 4); + + for ( i =3D 0; i < DIV_ROUND_UP(gicv2_info.nr_lines, 4); i++ ) + gicv2_context.gicd_itargetsr[i] =3D readl_gicd(GICD_ITARGETSR + i = * 4); + + for ( i =3D 0; i < DIV_ROUND_UP(gicv2_info.nr_lines, 16); i++ ) + gicv2_context.gicd_icfgr[i] =3D readl_gicd(GICD_ICFGR + i * 4); + + return 0; +} + +static void gicv2_resume(void) +{ + unsigned int i; + + if ( !gicv2_context.gicd_isenabler ) + { + dprintk(XENLOG_WARNING, "%s:%d: GICv2 suspend context not allocate= d!\n", + __func__, __LINE__); + return; + } + + gicv2_cpu_disable(); + /* Disable distributor */ + writel_gicd(0, GICD_CTLR); + + /* Restore GICD configuration */ + for ( i =3D 0; i < DIV_ROUND_UP(gicv2_info.nr_lines, 32); i++ ) { + writel_gicd(0xffffffff, GICD_ICENABLER + i * 4); + writel_gicd(gicv2_context.gicd_isenabler[i], GICD_ISENABLER + i * = 4); + } + + for ( i =3D 0; i < DIV_ROUND_UP(gicv2_info.nr_lines, 32); i++ ) { + writel_gicd(0xffffffff, GICD_ICACTIVER + i * 4); + writel_gicd(gicv2_context.gicd_isactiver[i], GICD_ISACTIVER + i * = 4); + } + + for ( i =3D 0; i < DIV_ROUND_UP(gicv2_info.nr_lines, 4); i++ ) + writel_gicd(gicv2_context.gicd_ipriorityr[i], GICD_IPRIORITYR + i = * 4); + + for ( i =3D 0; i < DIV_ROUND_UP(gicv2_info.nr_lines, 4); i++ ) + writel_gicd(gicv2_context.gicd_itargetsr[i], GICD_ITARGETSR + i * = 4); + + for ( i =3D 0; i < DIV_ROUND_UP(gicv2_info.nr_lines, 16); i++ ) + writel_gicd(gicv2_context.gicd_icfgr[i], GICD_ICFGR + i * 4); + + /* Make sure all registers are restored and enable distributor */ + writel_gicd(gicv2_context.gicd_ctlr | GICD_CTL_ENABLE, GICD_CTLR); + + /* Restore GIC CPU interface configuration */ + writel_gicc(gicv2_context.gicc_pmr, GICC_PMR); + writel_gicc(gicv2_context.gicc_bpr, GICC_BPR); + + /* Enable GIC CPU interface */ + writel_gicc(gicv2_context.gicc_ctlr | GICC_CTL_ENABLE | GICC_CTL_EOI, + GICC_CTLR); +} + +static void gicv2_alloc_context(struct gicv2_context *gc) +{ + uint32_t n =3D gicv2_info.nr_lines; + + gc->gicd_isenabler =3D xzalloc_array(uint32_t, DIV_ROUND_UP(n, 32)); + if ( !gc->gicd_isenabler ) + goto err_free; + + gc->gicd_isactiver =3D xzalloc_array(uint32_t, DIV_ROUND_UP(n, 32)); + if ( !gc->gicd_isactiver ) + goto err_free; + + gc->gicd_itargetsr =3D xzalloc_array(uint32_t, DIV_ROUND_UP(n, 4)); + if ( !gc->gicd_itargetsr ) + goto err_free; + + gc->gicd_ipriorityr =3D xzalloc_array(uint32_t, DIV_ROUND_UP(n, 4)); + if ( !gc->gicd_ipriorityr ) + goto err_free; + + gc->gicd_icfgr =3D xzalloc_array(uint32_t, DIV_ROUND_UP(n, 16)); + if ( !gc->gicd_icfgr ) + goto err_free; + + return; + + err_free: + dprintk(XENLOG_ERR, + "%s:%d: failed to allocate memory for GICv2 suspend context\n", + __func__, __LINE__); + + xfree(gc->gicd_icfgr); + xfree(gc->gicd_ipriorityr); + xfree(gc->gicd_itargetsr); + xfree(gc->gicd_isactiver); + xfree(gc->gicd_isenabler); + + memset(gc, 0, sizeof(*gc)); +} + +#endif /* CONFIG_SYSTEM_SUSPEND */ + #ifdef CONFIG_ACPI static unsigned long gicv2_get_hwdom_extra_madt_size(const struct domain *= d) { @@ -1302,6 +1447,11 @@ static int __init gicv2_init(void) =20 spin_unlock(&gicv2.lock); =20 +#ifdef CONFIG_SYSTEM_SUSPEND + /* Allocate memory to be used for saving GIC context during the suspen= d */ + gicv2_alloc_context(&gicv2_context); +#endif /* CONFIG_SYSTEM_SUSPEND */ + return 0; } =20 @@ -1345,6 +1495,10 @@ static const struct gic_hw_operations gicv2_ops =3D { .map_hwdom_extra_mappings =3D gicv2_map_hwdom_extra_mappings, .iomem_deny_access =3D gicv2_iomem_deny_access, .do_LPI =3D gicv2_do_LPI, +#ifdef CONFIG_SYSTEM_SUSPEND + .suspend =3D gicv2_suspend, + .resume =3D gicv2_resume, +#endif /* CONFIG_SYSTEM_SUSPEND */ }; =20 /* Set up the GIC */ diff --git a/xen/arch/arm/gic.c b/xen/arch/arm/gic.c index e80fe0ca24..a018bd7715 100644 --- a/xen/arch/arm/gic.c +++ b/xen/arch/arm/gic.c @@ -425,6 +425,35 @@ int gic_iomem_deny_access(struct domain *d) return gic_hw_ops->iomem_deny_access(d); } =20 +#ifdef CONFIG_SYSTEM_SUSPEND + +int gic_suspend(void) +{ + /* Must be called by boot CPU#0 with interrupts disabled */ + ASSERT(!local_irq_is_enabled()); + ASSERT(!smp_processor_id()); + + if ( !gic_hw_ops->suspend || !gic_hw_ops->resume ) + return -ENOSYS; + + return gic_hw_ops->suspend(); +} + +void gic_resume(void) +{ + /* + * Must be called by boot CPU#0 with interrupts disabled after gic_sus= pend + * has returned successfully. + */ + ASSERT(!local_irq_is_enabled()); + ASSERT(!smp_processor_id()); + ASSERT(gic_hw_ops->resume); + + gic_hw_ops->resume(); +} + +#endif /* CONFIG_SYSTEM_SUSPEND */ + static int cpu_gic_callback(struct notifier_block *nfb, unsigned long action, void *hcpu) diff --git a/xen/arch/arm/include/asm/gic.h b/xen/arch/arm/include/asm/gic.h index 541f0eeb80..a706303008 100644 --- a/xen/arch/arm/include/asm/gic.h +++ b/xen/arch/arm/include/asm/gic.h @@ -280,6 +280,12 @@ extern int gicv_setup(struct domain *d); extern void gic_save_state(struct vcpu *v); extern void gic_restore_state(struct vcpu *v); =20 +#ifdef CONFIG_SYSTEM_SUSPEND +/* Suspend/resume */ +extern int gic_suspend(void); +extern void gic_resume(void); +#endif /* CONFIG_SYSTEM_SUSPEND */ + /* SGI (AKA IPIs) */ enum gic_sgi { GIC_SGI_EVENT_CHECK, @@ -395,6 +401,12 @@ struct gic_hw_operations { int (*iomem_deny_access)(struct domain *d); /* Handle LPIs, which require special handling */ void (*do_LPI)(unsigned int lpi); +#ifdef CONFIG_SYSTEM_SUSPEND + /* Save GIC configuration due to the system suspend */ + int (*suspend)(void); + /* Restore GIC configuration due to the system resume */ + void (*resume)(void); +#endif /* CONFIG_SYSTEM_SUSPEND */ }; =20 extern const struct gic_hw_operations *gic_hw_ops; --=20 2.48.1 From nobody Fri Dec 19 19:19:25 2025 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass(p=none dis=none) header.from=gmail.com ARC-Seal: i=1; a=rsa-sha256; t=1748855116; cv=none; d=zohomail.com; s=zohoarc; b=V6+5rIhoGAFGcHh9n+1P5JNDZRA2vIMJcoBGU03QSND15ErpLj1e5OvQS65hlgJIbGrJieyWHnstQXPu2qm3WsTzaSQZAp8EvX341/GwOUMza9dGmdUK5TqKpsyLG1uT0K0AAffxTFolTpGvjqiNeFQ/rEwcBw+EChWf/y2Z3vk= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1748855116; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=t3qd7wn/+BTaUGKnbQ/zLRIfyUfmlIN1J0Bwt1qcE10=; b=klOlamIc5aEIp92iqLn2LMTNwGAia8VCmqYie9jd7pR3wgYI2joPgtTl6vLxaBsUgORV4q0aRJXO/K4/TbKVY34Bb77HsdBiXyQruHy14RH0oC9fTWTHio+ob50YxD1X5srBBHRWXm3Yo6V5J9jnKNZ1U3Z4DBEUw8SBqtBOVNA= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1748855116209117.01476672435456; Mon, 2 Jun 2025 02:05:16 -0700 (PDT) Received: from list by lists.xenproject.org with outflank-mailman.1002945.1382398 (Exim 4.92) (envelope-from ) id 1uM16K-0000VU-HH; Mon, 02 Jun 2025 09:05:00 +0000 Received: by outflank-mailman (output) from mailman id 1002945.1382398; Mon, 02 Jun 2025 09:05:00 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1uM16K-0000VJ-DR; Mon, 02 Jun 2025 09:05:00 +0000 Received: by outflank-mailman (input) for mailman id 1002945; Mon, 02 Jun 2025 09:04:58 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1uM16I-0007kh-RD for xen-devel@lists.xenproject.org; Mon, 02 Jun 2025 09:04:58 +0000 Received: from mail-lf1-x133.google.com (mail-lf1-x133.google.com [2a00:1450:4864:20::133]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id a7e47e97-3f90-11f0-b894-0df219b8e170; Mon, 02 Jun 2025 11:04:57 +0200 (CEST) Received: by mail-lf1-x133.google.com with SMTP id 2adb3069b0e04-551ed563740so5335407e87.2 for ; Mon, 02 Jun 2025 02:04:57 -0700 (PDT) Received: from yp-VivoBook-ASUSLaptop-M1503QA-M1503QA.. ([95.67.15.120]) by smtp.gmail.com with ESMTPSA id 2adb3069b0e04-5533791cdc3sm1507541e87.176.2025.06.02.02.04.55 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 02 Jun 2025 02:04:55 -0700 (PDT) X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: a7e47e97-3f90-11f0-b894-0df219b8e170 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1748855096; x=1749459896; darn=lists.xenproject.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=t3qd7wn/+BTaUGKnbQ/zLRIfyUfmlIN1J0Bwt1qcE10=; b=WbGXqU9a88+HORAGiOjUP8+B+eIE86O8z3KV/Bcoxoh19SaPQMYog2NRJ07SdHAyl6 4VgxgCjJdZcbaX2pR3LeVCZkfmYNxR+i2w9AoFRFM+Xe6GWSVhycM0y12dbRvCIGhqlz mZ+lTmJE/1t/QQGjmqGL/tnJ6B35Xp6lPtJ1hsXos/prFL5EtYCU1Yc82R3W0rBfqZKB KY0kjDt1EKgXxKuH+G/wQnuuECU6ZXfaLS0ERcDKJVIto2+P0/0zI7cWJmtJQNQHTBYN HWMg11afFY4b7ti7V/1bhZZQSEZTU/7MlOkA2netRfJkMNWA2rKZij1EWkoKgDYhxtgQ +HbQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1748855096; x=1749459896; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=t3qd7wn/+BTaUGKnbQ/zLRIfyUfmlIN1J0Bwt1qcE10=; b=j635ycgOAaqvuXveHQ1JJlvnDXaaCoq2njBffEgQO4EezFrbvqW6dXrmP/e/LwpHXU bLIJ10Yv2GS9eZg7sV1qcBECLyu+Bul09DZdS3fOu+qn1GRU9LZn+l1fzhd2mhA8Oihr T3mESP5aFhgdyTyjPrvIML21l29aTxixd7v5u5SK9gbHYGqkDHxA7BscwYvSTjffMb+Y cBhlHuvbh6u14Cv5M09f5tTy2nH2yfwqBBnxAIMsYaSop4jWl8onLommCbZgZVjixzGb E5rkavQfjwiKMKVkLmYgi/WE4yIOuSsVc8KxK50MYSy5N0OXSEmojeXBkcu7/KvCBKcC qHog== X-Gm-Message-State: AOJu0YyWzGAGFHfgJqdn5mDliYi5t/W0+Z9BhDeorBaN7fJXYFT1DyLy DGafaXQClFJO2d/O1TRv1JZ2L3NBTSLXJPPluviPEYb73C0J5wY6He4ZP4uvVa0x X-Gm-Gg: ASbGncs0A7F9NGYYhCx6ntSHZ1pqGvNfHx2llE5WcIUZgNfnaXgAYxV5yaUkQcoPTQE BSeZe9gE0I5pbMT+47uE+XXsghVwAFohyKbO0RYzU+E1fdrwtydTyW2Tp8MLpk14yh1DgrkqUhf I6ph+7MYE0SDN1Gb/lwYJD8RVM2gOyual1SahyZFwiDsN+940L1pGAUq1jJ1KWDdMyUfGUxClwe N+1ezCJG4Yma11TBDR1r5SDt/5AZ47Ij+RtwFi/5tCNqRjt6OS+zDCv25GludHQ/xAiBfrA86po IcrrHBgwE44Dee5+Q0CRgkCmgyFMUUnyzIR00De91Y1Jr7nmwbUzdTT/hT8G69CFk2UTrdU7ET3 9GT9Q4r3OKiw1uk0= X-Google-Smtp-Source: AGHT+IGi3/KfBHShC4Zm7tMGUh4qiGmrbniKVBV5ymlS3Q6WgyqayZLY6h/yrit9MCILP5HY1wYUag== X-Received: by 2002:a05:6512:2215:b0:553:3127:b00 with SMTP id 2adb3069b0e04-5533b90b7d6mr4336691e87.32.1748855096396; Mon, 02 Jun 2025 02:04:56 -0700 (PDT) From: Mykola Kvach To: xen-devel@lists.xenproject.org Cc: xakep.amatop@gmail.com, Stefano Stabellini , Julien Grall , Bertrand Marquis , Michal Orzel , Volodymyr Babchuk Subject: [PATCH v4][PART 2 04/10] xen/arm: Prevent crash during disable_nonboot_cpus on suspend Date: Mon, 2 Jun 2025 12:04:15 +0300 Message-ID: X-Mailer: git-send-email 2.48.1 In-Reply-To: References: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @gmail.com) X-ZM-MESSAGEID: 1748855120373116600 Content-Type: text/plain; charset="utf-8" From: Mykola Kvach If we call disable_nonboot_cpus on ARM64 with system_state set to SYS_STATE_suspend, the following assertion will be triggered: ``` (XEN) [ 25.582712] Disabling non-boot CPUs ... (XEN) [ 25.587032] Assertion '!in_irq() && (local_irq_is_enabled() || num= _online_cpus() <=3D 1)' failed at common/xmalloc_tlsf.c:714 [...] (XEN) [ 25.975069] Xen call trace: (XEN) [ 25.978353] [<00000a000022e098>] xfree+0x130/0x1a4 (PC) (XEN) [ 25.984314] [<00000a000022e08c>] xfree+0x124/0x1a4 (LR) (XEN) [ 25.990276] [<00000a00002747d4>] release_irq+0xe4/0xe8 (XEN) [ 25.996152] [<00000a0000278588>] time.c#cpu_time_callback+0x44/= 0x60 (XEN) [ 26.003150] [<00000a000021d678>] notifier_call_chain+0x7c/0xa0 (XEN) [ 26.009717] [<00000a00002018e0>] cpu.c#cpu_notifier_call_chain+= 0x24/0x48 (XEN) [ 26.017148] [<00000a000020192c>] cpu.c#_take_cpu_down+0x28/0x34 (XEN) [ 26.023801] [<00000a0000201944>] cpu.c#take_cpu_down+0xc/0x18 (XEN) [ 26.030281] [<00000a0000225c5c>] stop_machine.c#stopmachine_act= ion+0xbc/0xe4 (XEN) [ 26.038057] [<00000a00002264bc>] tasklet.c#do_tasklet_work+0xb8= /0x100 (XEN) [ 26.045229] [<00000a00002268a4>] do_tasklet+0x68/0xb0 (XEN) [ 26.051018] [<00000a000026e120>] domain.c#idle_loop+0x7c/0x194 (XEN) [ 26.057585] [<00000a0000277e30>] start_secondary+0x21c/0x220 (XEN) [ 26.063978] [<00000a0000361258>] 00000a0000361258 ``` This happens because before invoking take_cpu_down via the stop_machine_run function on the target CPU, stop_machine_run requests the STOPMACHINE_DISABLE_IRQ state on that CPU. Releasing memory in the release_irq function then triggers the assertion: /* * Heap allocations may need TLB flushes which may require IRQs to be * enabled (except when only 1 PCPU is online). */ This patch adds system state checks to guard calls to request_irq and release_irq. These calls are now skipped when system_state is SYS_STATE_{resume,suspend}, preventing unsafe operations during suspend/resume handling. Signed-off-by: Mykola Kvach --- Changes in V4: - removed the prior tasklet-based workaround in favor of a more straightforward and safer solution - reworked the approach by adding explicit system state checks around request_irq and release_irq calls, skips these calls during suspend and resume states to avoid unsafe memory operations when IRQs are disabled --- xen/arch/arm/gic.c | 6 ++++++ xen/arch/arm/tee/ffa_notif.c | 2 +- xen/arch/arm/time.c | 18 ++++++++++++------ 3 files changed, 19 insertions(+), 7 deletions(-) diff --git a/xen/arch/arm/gic.c b/xen/arch/arm/gic.c index a018bd7715..9856cb1592 100644 --- a/xen/arch/arm/gic.c +++ b/xen/arch/arm/gic.c @@ -388,6 +388,9 @@ void gic_dump_info(struct vcpu *v) =20 void init_maintenance_interrupt(void) { + if ( system_state =3D=3D SYS_STATE_resume ) + return; + request_irq(gic_hw_ops->info->maintenance_irq, 0, maintenance_interrup= t, "irq-maintenance", NULL); } @@ -461,6 +464,9 @@ static int cpu_gic_callback(struct notifier_block *nfb, switch ( action ) { case CPU_DYING: + if ( system_state =3D=3D SYS_STATE_suspend ) + break; + /* This is reverting the work done in init_maintenance_interrupt */ release_irq(gic_hw_ops->info->maintenance_irq, NULL); break; diff --git a/xen/arch/arm/tee/ffa_notif.c b/xen/arch/arm/tee/ffa_notif.c index 00efaf8f73..06f715a82b 100644 --- a/xen/arch/arm/tee/ffa_notif.c +++ b/xen/arch/arm/tee/ffa_notif.c @@ -347,7 +347,7 @@ void ffa_notif_init_interrupt(void) { int ret; =20 - if ( notif_enabled && notif_sri_irq < NR_GIC_SGI ) + if ( notif_enabled && notif_sri_irq < NR_GIC_SGI && system_state !=3D = SYS_STATE_resume ) { /* * An error here is unlikely since the primary CPU has already diff --git a/xen/arch/arm/time.c b/xen/arch/arm/time.c index ad984fdfdd..b2e07ade43 100644 --- a/xen/arch/arm/time.c +++ b/xen/arch/arm/time.c @@ -320,10 +320,13 @@ void init_timer_interrupt(void) WRITE_SYSREG(CNTHCTL_EL2_EL1PCTEN, CNTHCTL_EL2); disable_physical_timers(); =20 - request_irq(timer_irq[TIMER_HYP_PPI], 0, htimer_interrupt, - "hyptimer", NULL); - request_irq(timer_irq[TIMER_VIRT_PPI], 0, vtimer_interrupt, - "virtimer", NULL); + if ( system_state !=3D SYS_STATE_resume ) + { + request_irq(timer_irq[TIMER_HYP_PPI], 0, htimer_interrupt, + "hyptimer", NULL); + request_irq(timer_irq[TIMER_VIRT_PPI], 0, vtimer_interrupt, + "virtimer", NULL); + } =20 check_timer_irq_cfg(timer_irq[TIMER_HYP_PPI], "hypervisor"); check_timer_irq_cfg(timer_irq[TIMER_VIRT_PPI], "virtual"); @@ -338,8 +341,11 @@ static void deinit_timer_interrupt(void) { disable_physical_timers(); =20 - release_irq(timer_irq[TIMER_HYP_PPI], NULL); - release_irq(timer_irq[TIMER_VIRT_PPI], NULL); + if ( system_state !=3D SYS_STATE_suspend ) + { + release_irq(timer_irq[TIMER_HYP_PPI], NULL); + release_irq(timer_irq[TIMER_VIRT_PPI], NULL); + } } =20 /* Wait a set number of microseconds */ --=20 2.48.1 From nobody Fri Dec 19 19:19:25 2025 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass(p=none dis=none) header.from=gmail.com ARC-Seal: i=1; a=rsa-sha256; t=1748855116; cv=none; d=zohomail.com; s=zohoarc; b=TyowngU4VqWsHEs120+IA1bt/g/tAuwn+idb6XWDhEVG8Edffz/DLGgVg0wWj5pCit2VY9DEm1OY0oS7ucCDxZdhLRreuJXZW6Sz1VzZGRxCFyfErY93FzFT2Lz0uiKbx2Sb0W8lSJ3z8e3xcOhqZqdg65gCq3/CRyEFkfAh23A= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1748855116; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=RiaE/YXxsWcT0rjjz9SjADvJ7JVMLOm41080TZwDgkw=; b=G9WktAJ5nUbbBwWQs+LuQXwdl9YNhHnWEQbxAg9O2Hf5j/EugxO2U/BW+EU6UMB05UlHBXAmo02pd8bgMUpmDxXMIfGCJSBFRgSlcwWbeA84wrhbO8NnKJ1vBzRmZuiZlTnPTHKIm/qvugQQRDogDG19SS7nlxFs+WFnlk5WBZk= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1748855116321276.4358035851359; Mon, 2 Jun 2025 02:05:16 -0700 (PDT) Received: from list by lists.xenproject.org with outflank-mailman.1002946.1382404 (Exim 4.92) (envelope-from ) id 1uM16K-0000Yr-Tm; Mon, 02 Jun 2025 09:05:00 +0000 Received: by outflank-mailman (output) from mailman id 1002946.1382404; Mon, 02 Jun 2025 09:05:00 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1uM16K-0000Y6-Li; Mon, 02 Jun 2025 09:05:00 +0000 Received: by outflank-mailman (input) for mailman id 1002946; Mon, 02 Jun 2025 09:05:00 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1uM16J-0007kh-Vr for xen-devel@lists.xenproject.org; Mon, 02 Jun 2025 09:04:59 +0000 Received: from mail-lf1-x131.google.com (mail-lf1-x131.google.com [2a00:1450:4864:20::131]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id a89ace8b-3f90-11f0-b894-0df219b8e170; Mon, 02 Jun 2025 11:04:58 +0200 (CEST) Received: by mail-lf1-x131.google.com with SMTP id 2adb3069b0e04-54e816aeca6so5395978e87.2 for ; Mon, 02 Jun 2025 02:04:58 -0700 (PDT) Received: from yp-VivoBook-ASUSLaptop-M1503QA-M1503QA.. ([95.67.15.120]) by smtp.gmail.com with ESMTPSA id 2adb3069b0e04-5533791cdc3sm1507541e87.176.2025.06.02.02.04.56 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 02 Jun 2025 02:04:57 -0700 (PDT) X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: a89ace8b-3f90-11f0-b894-0df219b8e170 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1748855098; x=1749459898; darn=lists.xenproject.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=RiaE/YXxsWcT0rjjz9SjADvJ7JVMLOm41080TZwDgkw=; b=ijoWbsD1V7y9lQ+/gVXTVzayAMW0UlokSVUX5g0WZWbRyn7/IBHqyW24dqdqt1RJ1m 18mt4BEbYEwbSV/EoqG8mls6BinlCTVVsJdaz0dKsRg0ZDfwxxzGe3XgtG7U2lQO1GTH O9kGaIkZ9kpTPw+RbAMDPFzMFQm4hUYQgx2QlMsXwPpkD5RDYmjoIzww8lqFWdoaOUpg kX2TJsP1Tq2sBmU59TqSDmMweb9ZHGub7Ed9irWQKidYHIGS0yFx45FyO45YvHw4V6pR /uTuyuIEPV0G4FFMhbO3gQADVgjCR+f/S+dfYlNHSQs6/8zWknrpoVisqiV34hvoDF/4 PVDw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1748855098; x=1749459898; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=RiaE/YXxsWcT0rjjz9SjADvJ7JVMLOm41080TZwDgkw=; b=fTnz3cz+ptbfj0hGuf1sTJdkADw1986Trd3CMPRlNleCXvz+rzG++EbE8eCYwnm0Qj /4/MRN2ITTgRqJuAMjHg704mGzsIJP3SV5A03fZ6Pcfay2gPCPmkrA0bJomF8Bx/BB1d NhgEcE4qglpodO9Eq5TXBGVTzLxALMMg3iXf7uNS4/05+nIvYG/20n4/RiMt2CtSBSRS YIGN6LCW9GwfZ99gMVEpBIVj8D/qj87jSQUYE7uAs+WDbaR4MjKSnbV40mzqw+Js1cho 10QeKG0+VpdzKiTHhvUwI3NEYDsaYuW8U3M9HBBrTf9eFG+WFkA7AWKckTijvRTQ0d85 zGpw== X-Gm-Message-State: AOJu0Yzr8AgXJ6UH9oVRvBhgkaz000wNhD7nWzj2rG6EAhHWIC05fEbb RqCe9duk8Dt/F5lRJBHNv3v+7ntIghi9Zp0PedTAOlqh3g2icCKE6B/V56QNwsui X-Gm-Gg: ASbGncuswR0OiqA+UP2YrCI2ci5hTSzk2gA5/iivrvYZjbK9KIb61o0uEjKJ1cv6SU8 6ixFT4wT0140n+m8M26wcymuSrUFmABA7k3Ni8AkBxg/5rdY0ULXUKg/jJYO/oZXM5YRZTMrgyT /XfWtdto4aE//qHz3/x3mrIdgT2/p1aec4Q/BhDxjlFDq/r2E/yW0k+MnC9YqxOYctQCBSn1jmx 4pdIakiKLDJAUYb/G8/M58da7fccgTCL9cXfXPIkqxYV9BRJcrf3pyNTcGcNUKNchuyzdEqRbdD /et4p2Y9FLaMejqqDx4QsbZV4Gi8bZllBezQDfveJobCeYhBxkiGlvIXornVWok82WFJq94qF7Q zte7FpuO9sUtP8eQ= X-Google-Smtp-Source: AGHT+IHT93ClrOVTknOM+Ss/N0Bog9/86dU4UzyGwalCHDLaSj4MeQjn7jNqW8uBN0iRuRnBK45f8w== X-Received: by 2002:a05:6512:a91:b0:553:3654:3319 with SMTP id 2adb3069b0e04-5533b930abamr4209993e87.43.1748855097582; Mon, 02 Jun 2025 02:04:57 -0700 (PDT) From: Mykola Kvach To: xen-devel@lists.xenproject.org Cc: xakep.amatop@gmail.com, Stefano Stabellini , Julien Grall , Bertrand Marquis , Michal Orzel , Volodymyr Babchuk Subject: [PATCH v4][PART 2 05/10] xen/arm: irq: avoid local IRQ descriptors reinit on system resume Date: Mon, 2 Jun 2025 12:04:16 +0300 Message-ID: <36ef314ff29957fb9fa727319535a0479b80ca28.1748848482.git.mykola_kvach@epam.com> X-Mailer: git-send-email 2.48.1 In-Reply-To: References: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @gmail.com) X-ZM-MESSAGEID: 1748855118180116600 Content-Type: text/plain; charset="utf-8" From: Mykola Kvach On ARM, during system resume, CPUs are brought online again. This normally triggers init_local_irq_data, which reinitializes IRQ descriptors for banked interrupts (SGIs and PPIs). These descriptors are statically allocated per CPU and retain valid state across suspend/resume cycles. Re-initializing them on resume is unnecessary and may result in loss of interrupt configuration or restored state. This patch skips init_local_irq_data when system_state is set to SYS_STATE_resume to preserve banked IRQ descs state during resume. Signed-off-by: Mykola Kvach --- xen/arch/arm/irq.c | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/xen/arch/arm/irq.c b/xen/arch/arm/irq.c index 03fbb90c6c..372716a73b 100644 --- a/xen/arch/arm/irq.c +++ b/xen/arch/arm/irq.c @@ -122,6 +122,10 @@ static int cpu_callback(struct notifier_block *nfb, un= signed long action, switch ( action ) { case CPU_UP_PREPARE: + /* Skip local IRQ cleanup on resume */ + if ( system_state =3D=3D SYS_STATE_resume ) + break; + rc =3D init_local_irq_data(cpu); if ( rc ) printk(XENLOG_ERR "Unable to allocate local IRQ for CPU%u\n", --=20 2.48.1 From nobody Fri Dec 19 19:19:25 2025 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass(p=none dis=none) header.from=gmail.com ARC-Seal: i=1; a=rsa-sha256; t=1748855121; cv=none; d=zohomail.com; s=zohoarc; b=O3JT/qTclq5svhra8EoJXWv8qoi2IhQl+Wv2wntaIEG3tFGbITyjgnayQOOMnjyKMltXmiN5Vsh6vZoqK0C7odN7LlIGLD0O9hRrY0MIs9AKMeDcHuhtbeOqgJ+VxOH9P8J9d8eNr7QhOkeBBayjea6AOCDvnRpJVO/PK1TwMCY= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1748855121; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=gLtZvLmNS5IQ5nQywV+KESH/yusLqzvm5c+C1sq8VQE=; b=gk6Am77yMUkCggsOIFHT6DTBVA1/8NmdOhCCIJUNznBoF3IPTu7R6cjRUa/0tW1CwM19bQJKM5SlxcUIpg2Eq9l+Sllg+/OYs5htv3f6j3hGLQHITwtHPQZn6/KSBMRci9tw5e+UNmjjOOKnc0IlVRX2TCo4LKlfzGJKsDupOC0= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1748855121080536.468507501727; Mon, 2 Jun 2025 02:05:21 -0700 (PDT) Received: from list by lists.xenproject.org with outflank-mailman.1002947.1382417 (Exim 4.92) (envelope-from ) id 1uM16M-0000yj-4j; Mon, 02 Jun 2025 09:05:02 +0000 Received: by outflank-mailman (output) from mailman id 1002947.1382417; Mon, 02 Jun 2025 09:05:02 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1uM16L-0000xn-Vx; Mon, 02 Jun 2025 09:05:01 +0000 Received: by outflank-mailman (input) for mailman id 1002947; Mon, 02 Jun 2025 09:05:01 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1uM16L-0007kh-6a for xen-devel@lists.xenproject.org; Mon, 02 Jun 2025 09:05:01 +0000 Received: from mail-lf1-x135.google.com (mail-lf1-x135.google.com [2a00:1450:4864:20::135]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id a94a2ed8-3f90-11f0-b894-0df219b8e170; Mon, 02 Jun 2025 11:04:59 +0200 (CEST) Received: by mail-lf1-x135.google.com with SMTP id 2adb3069b0e04-55220699ba8so5134319e87.2 for ; Mon, 02 Jun 2025 02:04:59 -0700 (PDT) Received: from yp-VivoBook-ASUSLaptop-M1503QA-M1503QA.. ([95.67.15.120]) by smtp.gmail.com with ESMTPSA id 2adb3069b0e04-5533791cdc3sm1507541e87.176.2025.06.02.02.04.57 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 02 Jun 2025 02:04:58 -0700 (PDT) X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: a94a2ed8-3f90-11f0-b894-0df219b8e170 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1748855099; x=1749459899; darn=lists.xenproject.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=gLtZvLmNS5IQ5nQywV+KESH/yusLqzvm5c+C1sq8VQE=; b=ZI6qqWrhzS7h7+3frDr+CYM7CAaAxcp4CfwrFfUakAtDva5BHNjlh50FLoTRLHQzMG wkyTjMl6cGeMxW1njm/PKcDpProuJEa1FcVX4CKQm77WOOQ0zOiKovwzV/Bn3NfSWYuM 6VGwUp54NsmO5F00OPe70O6jaN5A5dc3Kui6DKfPhfEHaUe+/8ZEEJDRXcdfAq69GTDs am2NkT2GHzQ20nKI/Po+4nwKkjct1dsflb6o1WYG4b5haWuNxkGC2gvArCYTIRYQ11Ny ncVhpUFEOabhO5kabtZHggfPR7+CVK/nCk7pI5eIrWe/X73gz8+9Qc0Wnm4DGF2Jb3C4 eI/A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1748855099; x=1749459899; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=gLtZvLmNS5IQ5nQywV+KESH/yusLqzvm5c+C1sq8VQE=; b=r8RfCvmE/GB7aO1FXW4s5ErzP5KW47yHxb2kofW8Gy3IHwSv5adWvZRhpXsYT9mRcn 2uPEeyQhki/gb52g8UMTOtY0fq17TGEmI1iN62C0TTc0hEyEs2ZlSELBwQlir8FwPAv2 WservR/cXepMdCYw/6shY6Z3+X+r8mGUY3yyg/XievJAoJAfWRDWEOZv0QfrQ4B2sbzE RrdMxjIcKbLiZ7ysRK6WCIQ3PNUewSio4XFpLL68wr6XCQ2Akm+mg8oBBoPo0t9oKvf3 dcG1+AFDtiNAXBPiYPshbg0Fa2vmbihCccW40RI8i3SfoOtAYR6fqmjrb1EjmDLLAcSV 629Q== X-Gm-Message-State: AOJu0YwUx+DzBVvTQ6SPf1iAMQ9CrKprQZFgbxXut4IYnIxkIeP4q/9i zhuzoY3fzxvn0SS+oKtTW1uYwCzpiHier03NooYrWjj0fhYMMxE6Zye9IZg0ReRw X-Gm-Gg: ASbGnctnsjJp8ZuBCj3jQ7PgBsT2U6izBiVKLl1BfpXomkDGTHYo8LaYDfKtqAwm3vW JY/68jokWsaB2vyiLVVRy0gJEvOAaTHrGk3J3IGyki85N+ftLn5XLrPtnTUF+90jYUotMi5ZWjE /jTCe03JwsriFe+f+dIm3A2b+7pFRWh9Knnue1SP3Uovj7Io9UXwRWCrNrLT+o2Tdz7z3/tsOdX YP9aSJX0PW4z1FE8KppwxX/EO5bXBvipEL6HvSfDsi23XTUaFNsu2tJSAG0NzkVz48iAjrIZhZI uRKjFgffRuqInV7K7FqUJYccdK0829eR2w04PQMI580qyS5nG5rW/6WZOJw/V2I9+DPODgjFFjZ sEJThol4GB/s98KPz4HFK+NvBIw== X-Google-Smtp-Source: AGHT+IEvX585OOmt7ZiEiMyg7i6PA8bef54WnayP0mdDtUFr/dnbjwdOuWJmj0LjXKmXClWMsr03gg== X-Received: by 2002:a05:6512:3d13:b0:550:e5c4:b0a1 with SMTP id 2adb3069b0e04-5533d13e73dmr2804533e87.11.1748855098750; Mon, 02 Jun 2025 02:04:58 -0700 (PDT) From: Mykola Kvach To: xen-devel@lists.xenproject.org Cc: xakep.amatop@gmail.com, Stefano Stabellini , Julien Grall , Bertrand Marquis , Michal Orzel , Volodymyr Babchuk Subject: [PATCH v4][PART 2 06/10] xen/arm: irq: Restore state of local IRQs during system resume Date: Mon, 2 Jun 2025 12:04:17 +0300 Message-ID: <8cbea1b2335ba0f2ea189c653c62bd591882b295.1748848482.git.mykola_kvach@epam.com> X-Mailer: git-send-email 2.48.1 In-Reply-To: References: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @gmail.com) X-ZM-MESSAGEID: 1748855122012116600 Content-Type: text/plain; charset="utf-8" From: Mykola Kvach On ARM, the first 32 interrupts (SGIs and PPIs) are banked per-CPU and not restored by gic_resume (for secondary cpus). This patch introduces restore_local_irqs_on_resume, a function that restores the state of local interrupts on the target CPU during system resume. It iterates over all local IRQs and re-enables those that were not disabled, reprogramming their routing and affinity accordingly. The function is invoked from start_secondary, ensuring that local IRQ state is restored early during CPU bring-up after suspend. Signed-off-by: Mykola Kvach --- xen/arch/arm/irq.c | 44 ++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 44 insertions(+) diff --git a/xen/arch/arm/irq.c b/xen/arch/arm/irq.c index 372716a73b..f32f8b56ca 100644 --- a/xen/arch/arm/irq.c +++ b/xen/arch/arm/irq.c @@ -113,6 +113,47 @@ static int init_local_irq_data(unsigned int cpu) return 0; } =20 +/* + * The first 32 interrupts (PPIs and SGIs) are per-CPU, + * so call this function on the target CPU to restore them. + * + * SPIs are restored via gic_resume. + */ +static void restore_local_irqs_on_resume(void) +{ + int irq; + + if ( system_state !=3D SYS_STATE_resume ) + return; + + spin_lock(&local_irqs_type_lock); + + for ( irq =3D 0; irq < NR_LOCAL_IRQS; irq++ ) + { + struct irq_desc *desc =3D irq_to_desc(irq); + + spin_lock(&desc->lock); + + if ( test_bit(_IRQ_DISABLED, &desc->status) ) + { + spin_unlock(&desc->lock); + continue; + } + + /* it is needed to avoid asserts in below calls */ + set_bit(_IRQ_DISABLED, &desc->status); + + gic_route_irq_to_xen(desc, GIC_PRI_IRQ); + + /* _IRQ_DISABLED is cleared by below call */ + desc->handler->startup(desc); + + spin_unlock(&desc->lock); + } + + spin_unlock(&local_irqs_type_lock); +} + static int cpu_callback(struct notifier_block *nfb, unsigned long action, void *hcpu) { @@ -131,6 +172,9 @@ static int cpu_callback(struct notifier_block *nfb, uns= igned long action, printk(XENLOG_ERR "Unable to allocate local IRQ for CPU%u\n", cpu); break; + case CPU_STARTING: + restore_local_irqs_on_resume(); + break; } =20 return notifier_from_errno(rc); --=20 2.48.1 From nobody Fri Dec 19 19:19:25 2025 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass(p=none dis=none) header.from=gmail.com ARC-Seal: i=1; a=rsa-sha256; t=1748855119; cv=none; d=zohomail.com; s=zohoarc; b=XypZco6HAW4TlJZePD4go4PPlaDKJELErUKGcVBwJon8XNKfhWLno18hK2QcNP6TOrTsCURTPGQFH2566/34/DmbK9nfrJ6aAogheE6DgZq7DNR0N19Ga9Z1ZO2ZgQzMtE2nJgm0dREp9KiOJMmhPKKwaaCg8pbSyMsspjygZos= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1748855119; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=NXZs6M8P80Im90jJSBLKCDHJny8JfIVqdVI3nVOxBmE=; b=nGRMa8Zw1ZFRA589PHjwYxeyPpDP+zFYCougV9sHsFxX9zJIBvyAav14a687ctsFFFCKX7Rs2sjaMZSrcdZ/n5yVUx96cpnC53Gyy5TPi5LxtT/d9JtiIXQ479CvLNH9MOSNPJoFZxqtX9O3vM80e5KZtZrmJwkaYa/WIv/LQgQ= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 174885511938355.68237649920832; Mon, 2 Jun 2025 02:05:19 -0700 (PDT) Received: from list by lists.xenproject.org with outflank-mailman.1002948.1382428 (Exim 4.92) (envelope-from ) id 1uM16P-0001Q8-Gd; Mon, 02 Jun 2025 09:05:05 +0000 Received: by outflank-mailman (output) from mailman id 1002948.1382428; Mon, 02 Jun 2025 09:05:05 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1uM16P-0001Pu-BS; Mon, 02 Jun 2025 09:05:05 +0000 Received: by outflank-mailman (input) for mailman id 1002948; Mon, 02 Jun 2025 09:05:03 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1uM16N-0007kh-BE for xen-devel@lists.xenproject.org; Mon, 02 Jun 2025 09:05:03 +0000 Received: from mail-lf1-x12c.google.com (mail-lf1-x12c.google.com [2a00:1450:4864:20::12c]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id aa775ed5-3f90-11f0-b894-0df219b8e170; Mon, 02 Jun 2025 11:05:01 +0200 (CEST) Received: by mail-lf1-x12c.google.com with SMTP id 2adb3069b0e04-553280c345cso5290525e87.0 for ; Mon, 02 Jun 2025 02:05:01 -0700 (PDT) Received: from yp-VivoBook-ASUSLaptop-M1503QA-M1503QA.. ([95.67.15.120]) by smtp.gmail.com with ESMTPSA id 2adb3069b0e04-5533791cdc3sm1507541e87.176.2025.06.02.02.04.59 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 02 Jun 2025 02:04:59 -0700 (PDT) X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: aa775ed5-3f90-11f0-b894-0df219b8e170 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1748855101; x=1749459901; darn=lists.xenproject.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=NXZs6M8P80Im90jJSBLKCDHJny8JfIVqdVI3nVOxBmE=; b=kUKBdBHmu47ZOW8rZpCaqSNoS2j1Ly1SKnz5MyfJBUgaE+F9ZxGtfjfEoF7nIgcljN Dx+iu2Kk6SRD+WY+uVbdm9cDN6ZwfNSL6Cc4GcmbGcpZDHrvmr2Ffl17fOrUpu1UEQL0 eKiPEN3IlFrnxy5ZtOisjvW/op4oM7T1hPMfr1PMN4Q3V8fawXc7UOBx/VzDWol6MlFH DlC+mwSQOqGdeXwNwpIPAsiBJRV9I9dykcyzCY7vmBp7+CoqEP3WfRzbqDmwr/egXxzn pr8cgup0VWKC3VRWL8Z/Kfu5IhCsImRwEZby0qUU1p4hYtLwB0JLbA3ebMtKhc9d452K mYDQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1748855101; x=1749459901; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=NXZs6M8P80Im90jJSBLKCDHJny8JfIVqdVI3nVOxBmE=; b=ANEP71lLSlm8PaObWIVoqytAf9EU0Je2aewD6PixAPQSpiEDysM94S8Ka+o+6nZxCp Y4sW28icVwf82R71h0578bL7B26+5WlMAMBStpc9aurLxAS1eW+SWIbN22EKUpECL1wN +5PA2+dB7DA394OwsqS+rC6wQbsLPJXySu9ebG+PQHZWT7ZFL5XKNfdGkbDY7p7YVwkr b2Mu8FyXslf2vbeIbidcJQPsiZxfEM3USL9f2jEVatXItS9shrtO+0Dm/iIGZbkS8FOA maSavfUYlrxb4RYnJTYIvqR82UUc0IGDGNMxf80dbVV5zV4d9tt49uDYdbFloVX3a8fv ByTQ== X-Gm-Message-State: AOJu0YwvfMErhR1vUTjs1Ei/OoiFv33bSuLQzsnMvJWGI9fj+dDTcogT idn3uHJik4vMeFL4Bkftnvw+cRcS0U1Sl6f21ipQwMCRb3lFfgZZBG6TovHmJagAK9o= X-Gm-Gg: ASbGncsER2rWTvBEY+isQnsuWJ5Yk60pQMkPEmt7WmGKagIZNgR5HIfR3bWoa2fbz7R 2+afRq5K7FadrWfsTA4UAK/tj+WSCbAgxisyyyUUK9BpDYAcxCizEzrYgFN6Y38v7CMsY9d9LyB 8q+iYzcustv/YTIyRO0LRYI8mKNsztPikbS+ixo02SooMJbVwYey2X2onUHF1/tbWCWOL/2y3OZ O/3kmYmN88ukfiPrRfdE1DV6wjb3m7E+FAfoWy6f+8pKVONqYCe6Db/Gi87ehF42kpEQMeZC8ZS QFquvUMItrfxmYXUL1FGQXyaL0wLhuPsSvoYvhx/qFgQl9bBGZNdWVcYRQSPAru9hVJErFZs99s hQJ7Rh2m3XFTzPm9JGpsfhXd9fA== X-Google-Smtp-Source: AGHT+IEuTNPnBUZrCCHATRKgs5I7i+11AlP+JJAf9fOMKvbdRpaN5dntYb1dXqpKLf04epIJMtzGHw== X-Received: by 2002:a05:6512:3352:b0:553:50c6:b86c with SMTP id 2adb3069b0e04-55350c6b915mr46633e87.57.1748855100526; Mon, 02 Jun 2025 02:05:00 -0700 (PDT) From: Mykola Kvach To: xen-devel@lists.xenproject.org Cc: xakep.amatop@gmail.com, Stefano Stabellini , Julien Grall , Bertrand Marquis , Michal Orzel , Volodymyr Babchuk , Andrew Cooper , Anthony PERARD , Jan Beulich , =?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= , Shawn Anastasio , Alistair Francis , Bob Eshleman , Connor Davis , Oleksii Kurochko Subject: [PATCH v4][PART 2 07/10] xen/arm: Add support for system suspend triggered by hardware domain Date: Mon, 2 Jun 2025 12:04:18 +0300 Message-ID: <7bd75ecfff5b0a75ea5abd7cc4934582d7e1250c.1748848482.git.mykola_kvach@epam.com> X-Mailer: git-send-email 2.48.1 In-Reply-To: References: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @gmail.com) X-ZM-MESSAGEID: 1748855120380116600 Content-Type: text/plain; charset="utf-8" From: Mirela Simonovic Trigger Xen suspend when the hardware domain initiates suspend via SHUTDOWN_suspend. Redirect system suspend to CPU#0 to ensure the suspend logic runs on the boot CPU, as required. Introduce full suspend/resume infrastructure gated by CONFIG_SYSTEM_SUSPEND, including logic to: - disable and enable non-boot physical CPUs - freeze and thaw domains - suspend and resume the GIC, timer and console - maintain system state before and after suspend Extend arch_domain_shutdown to return a status code and integrate system_suspend into the ARM-specific path. Update stub implementations and function signatures across other architectures (x86, riscv, ppc) for consistency. Remove the restriction in the PSCI interface preventing suspend from the hardware domain. Update Kconfig and Makefile to support conditional inclusion of the new suspend.c implementation.Limit SYSTEM_SUSPEND support to ARM64 with UNSUPPORTED enabled and LLC_COLORING disabled. Signed-off-by: Mirela Simonovic Signed-off-by: Saeed Nowshadi Signed-off-by: Mykyta Poturai Signed-off-by: Mykola Kvach --- Changes introduced in V4: - drop code for saving and restoring VCPU context (for more information refer part 1 of patch series) - remove IOMMU suspend and resume calls until they will be implemented - move system suspend logic to arch_domain_shutdown, invoked from domain_shutdown - apply minor fixes such as renaming functions, updating comments, and modifying the commit message to reflect these changes - add console_end_sync to resume path after system suspend Changes introduced in V3: - merge changes from other commits into this patch (previously stashed): 1) disable/enable non-boot physical CPUs and freeze/thaw domains during suspend/resume 2) suspend/resume the timer, GIC, console, IOMMU, and hardware domain --- xen/arch/arm/Kconfig | 1 + xen/arch/arm/Makefile | 1 + xen/arch/arm/domain.c | 21 ++++- xen/arch/arm/include/asm/suspend.h | 22 +++++ xen/arch/arm/suspend.c | 143 +++++++++++++++++++++++++++++ xen/arch/arm/vpsci.c | 4 - xen/arch/ppc/stubs.c | 2 +- xen/arch/riscv/stubs.c | 2 +- xen/arch/x86/domain.c | 4 +- xen/common/domain.c | 9 +- xen/include/xen/domain.h | 2 +- 11 files changed, 200 insertions(+), 11 deletions(-) create mode 100644 xen/arch/arm/include/asm/suspend.h create mode 100644 xen/arch/arm/suspend.c diff --git a/xen/arch/arm/Kconfig b/xen/arch/arm/Kconfig index 57919d8b3a..80d0a6bf54 100644 --- a/xen/arch/arm/Kconfig +++ b/xen/arch/arm/Kconfig @@ -19,6 +19,7 @@ config ARM select HAS_DOM0LESS select HAS_GRANT_CACHE_FLUSH if GRANT_TABLE select HAS_STACK_PROTECTOR + select HAS_SYSTEM_SUSPEND if ARM_64 && !LLC_COLORING && UNSUPPORTED select HAS_UBSAN =20 config ARCH_DEFCONFIG diff --git a/xen/arch/arm/Makefile b/xen/arch/arm/Makefile index eeeac4e653..381969dd90 100644 --- a/xen/arch/arm/Makefile +++ b/xen/arch/arm/Makefile @@ -53,6 +53,7 @@ obj-y +=3D smp.o obj-y +=3D smpboot.o obj-$(CONFIG_STATIC_MEMORY) +=3D static-memory.init.o obj-$(CONFIG_STATIC_SHM) +=3D static-shmem.init.o +obj-$(CONFIG_SYSTEM_SUSPEND) +=3D suspend.o obj-y +=3D sysctl.o obj-y +=3D time.o obj-y +=3D traps.o diff --git a/xen/arch/arm/domain.c b/xen/arch/arm/domain.c index 45aeb8bddc..75092a997f 100644 --- a/xen/arch/arm/domain.c +++ b/xen/arch/arm/domain.c @@ -24,11 +24,14 @@ #include #include #include +#include #include #include #include #include =20 +#include + #include "vpci.h" #include "vuart.h" =20 @@ -857,8 +860,24 @@ void arch_domain_destroy(struct domain *d) domain_io_free(d); } =20 -void arch_domain_shutdown(struct domain *d) +int arch_domain_shutdown(struct domain *d) { + switch ( d->shutdown_code ) + { + case SHUTDOWN_suspend: +#ifdef CONFIG_SYSTEM_SUSPEND + if ( !is_hardware_domain(d) ) + break; + + return host_system_suspend(); +#else + break; +#endif + default: + break; + } + + return 0; } =20 void arch_domain_pause(struct domain *d) diff --git a/xen/arch/arm/include/asm/suspend.h b/xen/arch/arm/include/asm/= suspend.h new file mode 100644 index 0000000000..78d0e2383b --- /dev/null +++ b/xen/arch/arm/include/asm/suspend.h @@ -0,0 +1,22 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ + +#ifndef __ASM_ARM_SUSPEND_H__ +#define __ASM_ARM_SUSPEND_H__ + +#ifdef CONFIG_SYSTEM_SUSPEND + +int host_system_suspend(void); + +#endif /* CONFIG_SYSTEM_SUSPEND */ + +#endif /* __ASM_ARM_SUSPEND_H__ */ + + /* + * Local variables: + * mode: C + * c-file-style: "BSD" + * c-basic-offset: 4 + * tab-width: 4 + * indent-tabs-mode: nil + * End: + */ diff --git a/xen/arch/arm/suspend.c b/xen/arch/arm/suspend.c new file mode 100644 index 0000000000..13e59015c0 --- /dev/null +++ b/xen/arch/arm/suspend.c @@ -0,0 +1,143 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ + +#include +#include +#include + +/* + * TODO list: + * - Test system suspend with LLC_COLORING enabled and verify functionali= ty + * - Implement IOMMU suspend/resume handlers and integrate them + * into the suspend/resume path + * - Add GICv3 suspend/resume support if required + * - Enable "xl suspend" support on ARM architecture + * - Properly disable Xen timer watchdog from relevant services + * - Add suspend/resume CI test for ARM (QEMU if feasible) + * - Investigate feasibility and need for implementing system suspend on = ARM32 + */ + +/* Xen suspend. Note: data is not used (suspend is the suspend to RAM) */ +static long system_suspend(void *data) +{ + int status; + unsigned long flags; + + BUG_ON(system_state !=3D SYS_STATE_active); + + system_state =3D SYS_STATE_suspend; + freeze_domains(); + scheduler_disable(); + + /* + * Non-boot CPUs have to be disabled on suspend and enabled on resume + * (hotplug-based mechanism). Disabling non-boot CPUs will lead to PSCI + * CPU_OFF to be called by each non-boot CPU. Depending on the underly= ing + * platform capabilities, this may lead to the physical powering down = of + * CPUs. Tested on Xilinx Zynq Ultrascale+ MPSoC (including power down= of + * each non-boot CPU). + */ + status =3D disable_nonboot_cpus(); + if ( status ) + { + system_state =3D SYS_STATE_resume; + goto resume_nonboot_cpus; + } + + time_suspend(); + + local_irq_save(flags); + status =3D gic_suspend(); + if ( status ) + { + system_state =3D SYS_STATE_resume; + goto resume_irqs; + } + + printk("Xen suspending...\n"); + + console_start_sync(); + status =3D console_suspend(); + if ( status ) + { + dprintk(XENLOG_ERR, "Failed to suspend the console, err=3D%d\n", s= tatus); + system_state =3D SYS_STATE_resume; + goto resume_console; + } + + /* + * Enable identity mapping before entering suspend to simplify + * the resume path + */ + update_boot_mapping(true); + + system_state =3D SYS_STATE_resume; + update_boot_mapping(false); + + resume_console: + console_resume(); + console_end_sync(); + + gic_resume(); + + resume_irqs: + local_irq_restore(flags); + time_resume(); + + resume_nonboot_cpus: + /* + * The rcu_barrier() has to be added to ensure that the per cpu area is + * freed before a non-boot CPU tries to initialize it (_free_percpu_ar= ea() + * has to be called before the init_percpu_area()). This scenario occu= rs + * when non-boot CPUs are hot-unplugged on suspend and hotplugged on r= esume. + */ + rcu_barrier(); + enable_nonboot_cpus(); + scheduler_enable(); + thaw_domains(); + + system_state =3D SYS_STATE_active; + + /* The resume of hardware domain should always follow Xen's resume. */ + domain_resume(hardware_domain); + + printk("Resume (status %d)\n", status); + return status; +} + +int host_system_suspend(void) +{ + int status; + + /* + * system_suspend should be called when Dom0 finalizes the suspend + * procedure from its boot core (VCPU#0). However, Dom0's VCPU#0 could + * be mapped to any PCPU (this function could be executed by any PCPU). + * The suspend procedure has to be finalized by the PCPU#0 (non-boot + * PCPUs will be disabled during the suspend). + */ + status =3D continue_hypercall_on_cpu(0, system_suspend, NULL); + + /* + * If an error happened, there is nothing that needs to be done here + * because the system_suspend always returns in fully functional state + * no matter what the outcome of suspend procedure is. If the system + * suspended successfully the function will return 0 after the resume. + * Otherwise, if an error is returned it means Xen did not suspended, + * but it is still in the same state as if the system_suspend was never + * called. We dump a debug message in case of an error for debugging/ + * logging purpose. + */ + if ( status ) + dprintk(XENLOG_ERR, "Failed to suspend, errno=3D%d\n", status); + + return status; +} + +/* + * Local variables: + * mode: C + * c-file-style: "BSD" + * c-basic-offset: 4 + * indent-tabs-mode: nil + * End: + */ diff --git a/xen/arch/arm/vpsci.c b/xen/arch/arm/vpsci.c index 87a63b0a94..27e413b1fb 100644 --- a/xen/arch/arm/vpsci.c +++ b/xen/arch/arm/vpsci.c @@ -202,10 +202,6 @@ static int32_t do_psci_1_0_system_suspend(register_t e= point, register_t cid) struct vcpu *v; struct domain *d =3D current->domain; =20 - /* Drop this check once SYSTEM_SUSPEND is supported in hardware domain= */ - if ( is_hardware_domain(d) ) - return PSCI_NOT_SUPPORTED; - /* Ensure that all CPUs other than the calling one are offline */ for_each_vcpu ( d, v ) { diff --git a/xen/arch/ppc/stubs.c b/xen/arch/ppc/stubs.c index 671e71aa0a..5bb08dcd95 100644 --- a/xen/arch/ppc/stubs.c +++ b/xen/arch/ppc/stubs.c @@ -219,7 +219,7 @@ void arch_domain_destroy(struct domain *d) BUG_ON("unimplemented"); } =20 -void arch_domain_shutdown(struct domain *d) +int arch_domain_shutdown(struct domain *d) { BUG_ON("unimplemented"); } diff --git a/xen/arch/riscv/stubs.c b/xen/arch/riscv/stubs.c index 83416d3350..e2cb6c2579 100644 --- a/xen/arch/riscv/stubs.c +++ b/xen/arch/riscv/stubs.c @@ -204,7 +204,7 @@ void arch_domain_destroy(struct domain *d) BUG_ON("unimplemented"); } =20 -void arch_domain_shutdown(struct domain *d) +int arch_domain_shutdown(struct domain *d) { BUG_ON("unimplemented"); } diff --git a/xen/arch/x86/domain.c b/xen/arch/x86/domain.c index 7536b6c871..ab4b54fc3c 100644 --- a/xen/arch/x86/domain.c +++ b/xen/arch/x86/domain.c @@ -951,10 +951,12 @@ void arch_domain_destroy(struct domain *d) psr_domain_free(d); } =20 -void arch_domain_shutdown(struct domain *d) +int arch_domain_shutdown(struct domain *d) { if ( is_viridian_domain(d) ) viridian_time_domain_freeze(d); + + return 0; } =20 void arch_domain_pause(struct domain *d) diff --git a/xen/common/domain.c b/xen/common/domain.c index 58c5ffc466..6a3140561e 100644 --- a/xen/common/domain.c +++ b/xen/common/domain.c @@ -1276,6 +1276,7 @@ void __domain_crash(struct domain *d) int domain_shutdown(struct domain *d, u8 reason) { struct vcpu *v; + int ret; =20 #ifdef CONFIG_X86 if ( pv_shim ) @@ -1288,7 +1289,11 @@ int domain_shutdown(struct domain *d, u8 reason) d->shutdown_code =3D reason; reason =3D d->shutdown_code; =20 +#if defined(CONFIG_SYSTEM_SUSPEND) && defined(CONFIG_ARM) + if ( reason !=3D SHUTDOWN_suspend && is_hardware_domain(d) ) +#else if ( is_hardware_domain(d) ) +#endif hwdom_shutdown(reason); =20 if ( d->is_shutting_down ) @@ -1311,13 +1316,13 @@ int domain_shutdown(struct domain *d, u8 reason) v->paused_for_shutdown =3D 1; } =20 - arch_domain_shutdown(d); + ret =3D arch_domain_shutdown(d); =20 __domain_finalise_shutdown(d); =20 spin_unlock(&d->shutdown_lock); =20 - return 0; + return ret; } =20 void domain_resume(struct domain *d) diff --git a/xen/include/xen/domain.h b/xen/include/xen/domain.h index e10baf2615..c0cbbd808f 100644 --- a/xen/include/xen/domain.h +++ b/xen/include/xen/domain.h @@ -101,7 +101,7 @@ int arch_domain_create(struct domain *d, int arch_domain_teardown(struct domain *d); void arch_domain_destroy(struct domain *d); =20 -void arch_domain_shutdown(struct domain *d); +int arch_domain_shutdown(struct domain *d); void arch_domain_pause(struct domain *d); void arch_domain_unpause(struct domain *d); =20 --=20 2.48.1 From nobody Fri Dec 19 19:19:25 2025 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass(p=none dis=none) header.from=gmail.com ARC-Seal: i=1; a=rsa-sha256; t=1748855124; cv=none; d=zohomail.com; s=zohoarc; b=nwKAbVU93w8Jk+fGZbk3oSAC/HM4xlvTRkNHc3iZrH8RDtbvKxTaiJISqdSP9Izfwk7JbqMRYmsEN/+vuCwPsWW+n7gIhPo+0FktytEgLPWLCsJAmZAnPEpNKqmf2mlY2QqQq297Y4rnFBT0+p7tQObdCm10Z4ocYGFSSy5g8cE= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1748855124; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=xVrfy+vq7RG5yhPUKzBrjh3pK5+yQTsG+Zt3de4iBS0=; b=NWb0x5kLv5olvY2xC8HQL5Wdxn3FUyVAhNZuy4y9TfHlG6zQFHRB6JTpKToCURu6tKLUe8vRpPosyoj+rA6lluRm0lwQc2Xi7Dq1fJlrq7wXww2pMWJJ5CLjG8jSi5vHUUTRT5f1QfLcQjD5zG6J/6y8cSWJpi94DDc1mjFpuz0= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1748855124071459.9494097232863; Mon, 2 Jun 2025 02:05:24 -0700 (PDT) Received: from list by lists.xenproject.org with outflank-mailman.1002950.1382434 (Exim 4.92) (envelope-from ) id 1uM16Q-0001XK-3V; Mon, 02 Jun 2025 09:05:06 +0000 Received: by outflank-mailman (output) from mailman id 1002950.1382434; Mon, 02 Jun 2025 09:05:06 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1uM16P-0001WP-VB; Mon, 02 Jun 2025 09:05:05 +0000 Received: by outflank-mailman (input) for mailman id 1002950; Mon, 02 Jun 2025 09:05:04 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1uM16O-0007kh-GL for xen-devel@lists.xenproject.org; Mon, 02 Jun 2025 09:05:04 +0000 Received: from mail-lf1-x136.google.com (mail-lf1-x136.google.com [2a00:1450:4864:20::136]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id ab32a00e-3f90-11f0-b894-0df219b8e170; Mon, 02 Jun 2025 11:05:02 +0200 (CEST) Received: by mail-lf1-x136.google.com with SMTP id 2adb3069b0e04-55320ddb9edso4626327e87.1 for ; Mon, 02 Jun 2025 02:05:02 -0700 (PDT) Received: from yp-VivoBook-ASUSLaptop-M1503QA-M1503QA.. ([95.67.15.120]) by smtp.gmail.com with ESMTPSA id 2adb3069b0e04-5533791cdc3sm1507541e87.176.2025.06.02.02.05.00 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 02 Jun 2025 02:05:01 -0700 (PDT) X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: ab32a00e-3f90-11f0-b894-0df219b8e170 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1748855102; x=1749459902; darn=lists.xenproject.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=xVrfy+vq7RG5yhPUKzBrjh3pK5+yQTsG+Zt3de4iBS0=; b=YPAX9AgwzlBcLOXw9GYbEM9R9RjEDqkbAGnvs2wEJsBONx8PgDU3ViCPGR8HbkOa2U 3y2FndoZZ1sGJFzVnjSZStGamjjqz27jdjU9KIdPhWHdLRsllnRk0+GPjH3B/JGSDhWR HdIwT4MBiYYQ4/Sz0o4jKZ/fNCxK/+W1dz/p8p8WhI1z5hsKNDo5mwQDRPbjLGu8QMoz tr48Lv+/nqBiEM8MZnmSeYcPYH8qXuk6BUvQCXWM2J0AbRuaYGoTEEOoNR1iqjH8JNGC m4jYqBwObqZskDiLxdpWrMHTF3GxY419s+HMe00H/ab+n7Ut/tL7UQDy7sugt2x+czaS whbA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1748855102; x=1749459902; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=xVrfy+vq7RG5yhPUKzBrjh3pK5+yQTsG+Zt3de4iBS0=; b=w3VycqwDzvAvf+iUF0x2yZ6/aqvjen8/5bYpeZyAz3s3eD1CXKvp/YLlXdcc1loAVw qmcUImH67JEZmvpUpeG+SzH8Pt+CZAc/LTxqCl64tCKKnKKRPPs7ehnaMMssnOmdjY42 eCaKIiM34IP3khWjlwUU7NudxD4Vj4xrohpsRr+vBN1Hgte6iAAkneJxWQo55HOqKADG Z0+R00v0ng/mGePlH7DQINB2ZjXKzkVV+FQI13IPzf6IePsWIy3OnwBGR6IQ5NoTWDgl Q1lzJgee25gkNS6XJIv8YANr8aeYhda4Wwe6aKoS/05H4mzgPgXzgx6HvcjszJ29l3lR vjZg== X-Gm-Message-State: AOJu0YxAmmo1mwrk/2D1zt+us3yBiuFzCq5L35kX/NPonMmv8nKAsxqg Qn1jcAJ+mG2qDWAe5fmWqmfEofIKwhc+A5p5BabC0wszyo1hMXBRYy6CNDuUKVgK X-Gm-Gg: ASbGncu1cAs7iHtMGfcCKag1hxdsnwuvvEulGDoqRdrlQTHJug8iV6dnSwo1XowWEuM OKz4i11eLTmTCA9JlaTFouXhSFUIEL2S1gBz7lphWVzu0/BafGXoxcrJ3GuycpeV5YeP3SoeLE3 hyTeV28JYhXRt4deXu1IEyHzcHNSanyUFQy/qxfjYD9pBQ7HI9Tvn53A+kSca3jPhvqrBjIY6nk jVjfD7KgGyt+KEYSeNQpNZ0vJk4ryZYnrppWfmoiqtdzBVr0aiLvMftNSjOTGuLQKThN5/taWVv eLa7s3NaqgpvAGWYL7AyL1KNkU64viXOSJMecWE/41xgIhAGlzWg6HBxfFtFG3zN97fPYfp7ckt fQvDK43vpz7u4Ntg= X-Google-Smtp-Source: AGHT+IEzvXRcRJQ8PQGVzDpd6OcQsKQvl49VrWAqrNtL/IwwXOCr/Woxdx0f+/LYoggsghv0r2tfcg== X-Received: by 2002:a05:6512:1284:b0:553:2f49:ff2a with SMTP id 2adb3069b0e04-5533b92e3c3mr3620581e87.38.1748855101772; Mon, 02 Jun 2025 02:05:01 -0700 (PDT) From: Mykola Kvach To: xen-devel@lists.xenproject.org Cc: xakep.amatop@gmail.com, Stefano Stabellini , Julien Grall , Bertrand Marquis , Michal Orzel , Volodymyr Babchuk Subject: [PATCH v4][PART 2 08/10] xen/arm: Implement PSCI SYSTEM_SUSPEND call (host interface) Date: Mon, 2 Jun 2025 12:04:19 +0300 Message-ID: <482f5988adbfe209ef01ebffd48f37e2b210556e.1748848482.git.mykola_kvach@epam.com> X-Mailer: git-send-email 2.48.1 In-Reply-To: References: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @gmail.com) X-ZM-MESSAGEID: 1748855126275116600 Content-Type: text/plain; charset="utf-8" From: Mirela Simonovic Invoke PSCI SYSTEM_SUSPEND to finalize Xen's suspend sequence on ARM64 plat= forms. Pass the resume entry point (hyp_resume) as the first argument to EL3. The = resume handler is currently a stub and will be implemented later in assembly. Igno= re the context ID argument, as is done in Linux. Only enable this path when CONFIG_SYSTEM_SUSPEND is set and PSCI version is >=3D 1.0. Signed-off-by: Mirela Simonovic Signed-off-by: Saeed Nowshadi Signed-off-by: Mykyta Poturai Signed-off-by: Mykola Kvach --- Changes in v4: - select the appropriate PSCI SYSTEM_SUSPEND function ID based on arch - update comments and commit message to reflect recent changes Changes in v3: - return PSCI_NOT_SUPPORTED instead of a hardcoded 1 on ARM32 - check PSCI version before invoking SYSTEM_SUSPEND in call_psci_system_s= uspend --- xen/arch/arm/arm64/head.S | 8 ++++++++ xen/arch/arm/include/asm/psci.h | 1 + xen/arch/arm/include/asm/suspend.h | 2 ++ xen/arch/arm/psci.c | 23 ++++++++++++++++++++++- xen/arch/arm/suspend.c | 5 +++++ 5 files changed, 38 insertions(+), 1 deletion(-) diff --git a/xen/arch/arm/arm64/head.S b/xen/arch/arm/arm64/head.S index 72c7b24498..3522c497c5 100644 --- a/xen/arch/arm/arm64/head.S +++ b/xen/arch/arm/arm64/head.S @@ -561,6 +561,14 @@ END(efi_xen_start) =20 #endif /* CONFIG_ARM_EFI */ =20 +#ifdef CONFIG_SYSTEM_SUSPEND + +FUNC(hyp_resume) + b . +END(hyp_resume) + +#endif /* CONFIG_SYSTEM_SUSPEND */ + /* * Local variables: * mode: ASM diff --git a/xen/arch/arm/include/asm/psci.h b/xen/arch/arm/include/asm/psc= i.h index 48a93e6b79..bb3c73496e 100644 --- a/xen/arch/arm/include/asm/psci.h +++ b/xen/arch/arm/include/asm/psci.h @@ -23,6 +23,7 @@ int call_psci_cpu_on(int cpu); void call_psci_cpu_off(void); void call_psci_system_off(void); void call_psci_system_reset(void); +int call_psci_system_suspend(void); =20 /* Range of allocated PSCI function numbers */ #define PSCI_FNUM_MIN_VALUE _AC(0,U) diff --git a/xen/arch/arm/include/asm/suspend.h b/xen/arch/arm/include/asm/= suspend.h index 78d0e2383b..55041a5d06 100644 --- a/xen/arch/arm/include/asm/suspend.h +++ b/xen/arch/arm/include/asm/suspend.h @@ -7,6 +7,8 @@ =20 int host_system_suspend(void); =20 +void hyp_resume(void); + #endif /* CONFIG_SYSTEM_SUSPEND */ =20 #endif /* __ASM_ARM_SUSPEND_H__ */ diff --git a/xen/arch/arm/psci.c b/xen/arch/arm/psci.c index b6860a7760..c9d126b195 100644 --- a/xen/arch/arm/psci.c +++ b/xen/arch/arm/psci.c @@ -17,17 +17,20 @@ #include #include #include +#include =20 /* * While a 64-bit OS can make calls with SMC32 calling conventions, for * some calls it is necessary to use SMC64 to pass or return 64-bit values. - * For such calls PSCI_0_2_FN_NATIVE(x) will choose the appropriate + * For such calls PSCI_*_FN_NATIVE(x) will choose the appropriate * (native-width) function ID. */ #ifdef CONFIG_ARM_64 #define PSCI_0_2_FN_NATIVE(name) PSCI_0_2_FN64_##name +#define PSCI_1_0_FN_NATIVE(name) PSCI_1_0_FN64_##name #else #define PSCI_0_2_FN_NATIVE(name) PSCI_0_2_FN32_##name +#define PSCI_1_0_FN_NATIVE(name) PSCI_1_0_FN32_##name #endif =20 uint32_t psci_ver; @@ -60,6 +63,24 @@ void call_psci_cpu_off(void) } } =20 +int call_psci_system_suspend(void) +{ +#ifdef CONFIG_SYSTEM_SUSPEND + struct arm_smccc_res res; + + if ( psci_ver < PSCI_VERSION(1, 0) ) + return PSCI_NOT_SUPPORTED; + + /* 2nd argument (context ID) is not used */ + arm_smccc_smc(PSCI_1_0_FN_NATIVE(SYSTEM_SUSPEND), __pa(hyp_resume), &r= es); + return PSCI_RET(res); +#else + dprintk(XENLOG_WARNING, + "SYSTEM_SUSPEND not supported (CONFIG_SYSTEM_SUSPEND disabled)= \n"); + return PSCI_NOT_SUPPORTED; +#endif +} + void call_psci_system_off(void) { if ( psci_ver > PSCI_VERSION(0, 1) ) diff --git a/xen/arch/arm/suspend.c b/xen/arch/arm/suspend.c index 13e59015c0..e3a9b68622 100644 --- a/xen/arch/arm/suspend.c +++ b/xen/arch/arm/suspend.c @@ -1,5 +1,6 @@ /* SPDX-License-Identifier: GPL-2.0-only */ =20 +#include #include #include #include @@ -70,6 +71,10 @@ static long system_suspend(void *data) */ update_boot_mapping(true); =20 + status =3D call_psci_system_suspend(); + if ( status ) + dprintk(XENLOG_WARNING, "PSCI system suspend failed, err=3D%d\n", = status); + system_state =3D SYS_STATE_resume; update_boot_mapping(false); =20 --=20 2.48.1 From nobody Fri Dec 19 19:19:25 2025 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass(p=none dis=none) header.from=gmail.com ARC-Seal: i=1; a=rsa-sha256; t=1748855118; cv=none; d=zohomail.com; s=zohoarc; b=Lufau691YwiYTCqERpJJK4sDMZCGsaBVMricnxz0Tlanxg0Mm3nvuVCuiUuWhdBagKkeXTWgjyh0d6emXSpDPfQlNhYFszCEDrHmBQfTdezPZ+w6CTK5jeiCGU+KZanBdF1NxT3Sno3be9UXp6Yg68baNnropfVgjUWmVPSsJJY= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1748855118; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=teuh5RSMor4Ejk4U6N2jEUsE+RWNO2fK7HvyuJpLGDk=; b=kBh8LggQEHDCfOplTkdztdqpaclLhWuY/555ucL8KbpLDe/yJnDF/MwTP4h3NynNoKmjbBVn+7vll2DVy1RFs63kSFGeccX8CRt1Y+2oQiqpqsj+G9x9VopIBY768FpNRzxMaGEADezHh92MGv1dsUE2tNgHTEgpaZQi25MZEkU= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1748855118975112.10813137831212; Mon, 2 Jun 2025 02:05:18 -0700 (PDT) Received: from list by lists.xenproject.org with outflank-mailman.1002951.1382437 (Exim 4.92) (envelope-from ) id 1uM16Q-0001by-Gq; Mon, 02 Jun 2025 09:05:06 +0000 Received: by outflank-mailman (output) from mailman id 1002951.1382437; Mon, 02 Jun 2025 09:05:06 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1uM16Q-0001aw-87; Mon, 02 Jun 2025 09:05:06 +0000 Received: by outflank-mailman (input) for mailman id 1002951; Mon, 02 Jun 2025 09:05:05 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1uM16P-00081k-4S for xen-devel@lists.xenproject.org; Mon, 02 Jun 2025 09:05:05 +0000 Received: from mail-lf1-x12a.google.com (mail-lf1-x12a.google.com [2a00:1450:4864:20::12a]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id abc4a5a5-3f90-11f0-a300-13f23c93f187; Mon, 02 Jun 2025 11:05:03 +0200 (CEST) Received: by mail-lf1-x12a.google.com with SMTP id 2adb3069b0e04-55350d0eedeso56746e87.2 for ; Mon, 02 Jun 2025 02:05:03 -0700 (PDT) Received: from yp-VivoBook-ASUSLaptop-M1503QA-M1503QA.. ([95.67.15.120]) by smtp.gmail.com with ESMTPSA id 2adb3069b0e04-5533791cdc3sm1507541e87.176.2025.06.02.02.05.01 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 02 Jun 2025 02:05:02 -0700 (PDT) X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: abc4a5a5-3f90-11f0-a300-13f23c93f187 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1748855103; x=1749459903; darn=lists.xenproject.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=teuh5RSMor4Ejk4U6N2jEUsE+RWNO2fK7HvyuJpLGDk=; b=SOvwPxt5zh9+aUvv1EWJwG8xBGuXyAU+k218MbkKmTvz6VwOLoQ1ujp5qKnUOAhwr5 VKDP8vI7gtY1DvGOKt2vLUQE9luC8oCP3itSAk94oF2bVYVK6AbKUT3/J7e+tZ2fjoN6 HcHQPVZyGViXW10NY4sVkxxV8uGhQb6Fny+uWd+Cr4P1TG+BYAWjDrZc4pb7f4dAcWMP 5OhdE0IChlHmsA1tB7W0KB5FWOGquADN3ISFRPgyapdd+OcZ1R7X7YYfupygDfe/a8dx H3AQfuSYjACxJsYsxn0Tp/UAlToE2saSE0Agp4Nk+UM3QX2NpL3Bv0H1698mZe8JAA0v V46w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1748855103; x=1749459903; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=teuh5RSMor4Ejk4U6N2jEUsE+RWNO2fK7HvyuJpLGDk=; b=qXex9VcZkLLH6fBJQNSEBdKcpPwg8xjjrR2jg7JzElH8WEd+AT9Yw7rwd/9pMUkUgR VCW1xbL5vo3GrAk77V/DMdTLbFOQbUlEzUkNvHcIRR9BIBKYIRibePRz1PiRzM5QyW74 1Oyg6h55UV8PaTJh9PJIMkd7xs4rZ1E6XVI0h4O+Y+kWY4cElUcokJAiI/NID8qgQBXj OPdGlFZ8vNLTicAlPTtEjAbguWlr+0Uf66LR9JvMdJAsgM8w7HdGBvo1YdmIEVpVS1F2 QfU8g6NUx8bYrXbUN4nwxPS96JYhcYdeQtfoDK5QYR6qIiTYB6xCtdMPzy8YvPDO4D0o 3nPA== X-Gm-Message-State: AOJu0Yx8btZ1eIblxl9MiM2Zxg3UdjQ+almSeRi91jbLUsTcBggbF2OJ 20oc/1tiy8FUNiusHAHOz8dowS4h8BOzknEOxCsSaJIoC6JvnfF8kOtqjSNMymFe X-Gm-Gg: ASbGnct0/P4nv4VBR1sw31Fdq3XANiW802W0j6UX9WRAAmhFys3IS0Sow71AUq3eMlp zZm55lwXeLjqwHsaqB87MJg1ODfEYDMmPbEJnk1HjLGQT2G836YbYBxqknqycKPmLqKq9NBl8C1 CrgeZVWo2OLa4VPEH+1+lYTuTXT5oo1Y3FMFduZYtUGoLueIeXUKloOqYMdVZ3LJZa4idgHL4ov /do//50j9JuVZLEal3JtqsF61WpZ8cEVjCZiAMKY3jzRBCycznUmvJukeQyGTrkwh9lGrjfSPmv H1RkJ2KQeH7WK0OkfdU2H0vSkHMwFNCUXlSaDhqUad+k7Nzkz7PkV2tYSpAR1rlEywyjafndhUt oP9jFT+4V9X2CHVEtfkduWG80Hw== X-Google-Smtp-Source: AGHT+IEJbNUnDx2JeNnvdW9LV+EzHVXhdrAnE0wIQugHmKVHXrDvwNO2KkiktLN3zfaenjrUIhcqXw== X-Received: by 2002:a05:6512:3b23:b0:553:27cb:b84e with SMTP id 2adb3069b0e04-5533d1bb77amr3481253e87.41.1748855102981; Mon, 02 Jun 2025 02:05:02 -0700 (PDT) From: Mykola Kvach To: xen-devel@lists.xenproject.org Cc: xakep.amatop@gmail.com, Stefano Stabellini , Julien Grall , Bertrand Marquis , Michal Orzel , Volodymyr Babchuk Subject: [PATCH v4][PART 2 09/10] xen/arm: Resume memory management on Xen resume Date: Mon, 2 Jun 2025 12:04:20 +0300 Message-ID: X-Mailer: git-send-email 2.48.1 In-Reply-To: References: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @gmail.com) X-ZM-MESSAGEID: 1748855120363116600 Content-Type: text/plain; charset="utf-8" From: Mirela Simonovic The MMU must be enabled during the resume path before restoring context, as virtual addresses are used to access the saved context data. This patch adds MMU setup during resume by reusing the existing enable_secondary_cpu_mm function, which enables data cache and the MMU. Before the MMU is enabled, the content of TTBR0_EL2 is changed to point to init_ttbr (page tables used at runtime). On boot, init_ttbr is normally initialized during secondary CPU hotplug. On uniprocessor systems, this would leave init_ttbr uninitialized, causing resume to fail. To address this, the boot CPU now sets init_ttbr during suspend. Signed-off-by: Mirela Simonovic Signed-off-by: Saeed Nowshadi Signed-off-by: Mykyta Poturai Signed-off-by: Mykola Kvach --- Changes in v4: - Drop unnecessary DAIF masking; interrupts are already masked on resume - Remove leftover TLB flush instructions; flushing is done in enable_mmu - Avoid setting x19 in hyp_resume; not needed - Replace prepare_secondary_mm with set_init_ttbr; call it from system_susp= end Changes in v3: - Update commit message for clarity - Replace create_page_tables, enable_mmu, and mmu_init_secondary_cpu with enable_secondary_cpu_mm - Move prepare_secondary_mm to start_xen to avoid crash - Add early UART init during resume Changes in v2: - Move hyp_resume to head.S to keep resume logic together - Simplify hyp_resume using existing helpers: check_cpu_mode, cpu_init, create_page_tables, enable_mmu --- xen/arch/arm/arm64/head.S | 16 ++++++++++++++++ xen/arch/arm/include/asm/mm.h | 2 ++ xen/arch/arm/mmu/smpboot.c | 2 +- xen/arch/arm/suspend.c | 2 ++ 4 files changed, 21 insertions(+), 1 deletion(-) diff --git a/xen/arch/arm/arm64/head.S b/xen/arch/arm/arm64/head.S index 3522c497c5..596e960152 100644 --- a/xen/arch/arm/arm64/head.S +++ b/xen/arch/arm/arm64/head.S @@ -564,6 +564,22 @@ END(efi_xen_start) #ifdef CONFIG_SYSTEM_SUSPEND =20 FUNC(hyp_resume) + /* Initialize the UART if earlyprintk has been enabled. */ +#ifdef CONFIG_EARLY_PRINTK + bl init_uart +#endif + PRINT_ID("- Xen resuming -\r\n") + + bl check_cpu_mode + bl cpu_init + + ldr x0, =3Dstart + adr x20, start /* x20 :=3D paddr (start) */ + sub x20, x20, x0 /* x20 :=3D phys-offset */ + ldr lr, =3Dmmu_resumed + b enable_secondary_cpu_mm + +mmu_resumed: b . END(hyp_resume) =20 diff --git a/xen/arch/arm/include/asm/mm.h b/xen/arch/arm/include/asm/mm.h index 011bc1fd30..868282ec06 100644 --- a/xen/arch/arm/include/asm/mm.h +++ b/xen/arch/arm/include/asm/mm.h @@ -363,6 +363,8 @@ static inline void page_set_xenheap_gfn(struct page_inf= o *p, gfn_t gfn) } while ( (y =3D cmpxchg(&p->u.inuse.type_info, x, nx)) !=3D x ); } =20 +void set_init_ttbr(lpae_t *root); + #endif /* __ARCH_ARM_MM__ */ /* * Local variables: diff --git a/xen/arch/arm/mmu/smpboot.c b/xen/arch/arm/mmu/smpboot.c index 37e91d72b7..ff508ecf40 100644 --- a/xen/arch/arm/mmu/smpboot.c +++ b/xen/arch/arm/mmu/smpboot.c @@ -72,7 +72,7 @@ static void clear_boot_pagetables(void) clear_table(boot_third); } =20 -static void set_init_ttbr(lpae_t *root) +void set_init_ttbr(lpae_t *root) { /* * init_ttbr is part of the identity mapping which is read-only. So diff --git a/xen/arch/arm/suspend.c b/xen/arch/arm/suspend.c index e3a9b68622..f3ad8517ac 100644 --- a/xen/arch/arm/suspend.c +++ b/xen/arch/arm/suspend.c @@ -65,6 +65,8 @@ static long system_suspend(void *data) goto resume_console; } =20 + set_init_ttbr(xen_pgtable); + /* * Enable identity mapping before entering suspend to simplify * the resume path --=20 2.48.1 From nobody Fri Dec 19 19:19:25 2025 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass(p=none dis=none) header.from=gmail.com ARC-Seal: i=1; a=rsa-sha256; t=1748855124; cv=none; d=zohomail.com; s=zohoarc; b=HJQMooxRCVUc1dd0SdCSoL6gpATJcBWa8X4fAhcsCgpVqzpfkm+8f1bq6LlUyP7q+a2Ggscb6zDVkqIZjsgsluw7/Au84PL4GxoYEt8L9N7tzrzXX8MDLe5Goct44LsO6exVBcXdqUVBIF5nEKn/Z2dO3LW9E9W5G5vwHrqNZWY= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1748855124; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=bD1e3cReyKifgfKVVK1ivsJtIILbWIwhcDmXv0PowdU=; b=I42J23QiSeg1M0FoD0PVLYhM8Li9KOrQlSERvhcJ/hDOLYe2heYB2Ke2B7Z9Yuk1xWyo0ynnoOJntzmzT1zAJG445149F8SVwinRhAgn0SrR0AtZLnK64kpsdjGACiQy/KCpjjNM8sl7+xR77VkBdvYFIgnX+p9IN3x27/GGW9k= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 174885512446661.031643564309434; Mon, 2 Jun 2025 02:05:24 -0700 (PDT) Received: from list by lists.xenproject.org with outflank-mailman.1002952.1382451 (Exim 4.92) (envelope-from ) id 1uM16S-00022g-4b; Mon, 02 Jun 2025 09:05:08 +0000 Received: by outflank-mailman (output) from mailman id 1002952.1382451; Mon, 02 Jun 2025 09:05:08 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1uM16R-00021S-Ve; Mon, 02 Jun 2025 09:05:07 +0000 Received: by outflank-mailman (input) for mailman id 1002952; Mon, 02 Jun 2025 09:05:06 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1uM16Q-00081k-4b for xen-devel@lists.xenproject.org; Mon, 02 Jun 2025 09:05:06 +0000 Received: from mail-lf1-x12a.google.com (mail-lf1-x12a.google.com [2a00:1450:4864:20::12a]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id aca78b68-3f90-11f0-a300-13f23c93f187; Mon, 02 Jun 2025 11:05:05 +0200 (CEST) Received: by mail-lf1-x12a.google.com with SMTP id 2adb3069b0e04-55329bd977aso5224207e87.1 for ; Mon, 02 Jun 2025 02:05:05 -0700 (PDT) Received: from yp-VivoBook-ASUSLaptop-M1503QA-M1503QA.. ([95.67.15.120]) by smtp.gmail.com with ESMTPSA id 2adb3069b0e04-5533791cdc3sm1507541e87.176.2025.06.02.02.05.03 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 02 Jun 2025 02:05:03 -0700 (PDT) X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: aca78b68-3f90-11f0-a300-13f23c93f187 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1748855105; x=1749459905; darn=lists.xenproject.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=bD1e3cReyKifgfKVVK1ivsJtIILbWIwhcDmXv0PowdU=; b=aqU9jxcieiQz9hYAEEwZt2MFz4xNK1EvE5cUp5vCASp1MUDMaJ1AvfUTBjaEb80wyd +r9t4lIld9HOv+3l8UzM/pf6uXIRj62T2pUKyHr+QGl+6snbVs72vTT2JRX662x954kp PwmmzhbCJH7Ks0sRBWuLzicerVKm4oMKi0HzxaQzQBHBtZ+0MbIcEZxHeWc4cQe5xAg2 p6BDfgrvJYa9yBzfynVqJY5lF4Lk2oavIJwFIt/RmZ2JKWfxfJEsGelsq7ALq9BtxAyY cwJRT5bWZnHOaK9LYBpsJyXYFbpY9RnzYESwlfwsUCSZ5RLSn5Wdt69nMEb7ORXIpqt2 LiVw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1748855105; x=1749459905; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=bD1e3cReyKifgfKVVK1ivsJtIILbWIwhcDmXv0PowdU=; b=NSL8C/5yCaladbBY6T4zXXPOsIeDrEpw3k/USDYV2b0TAUFndc83F2ql7T9YPV2EBP VoFlwGPujmE+ennOY5yysIoiwcDw0ye86uGwZuILkjg6Y8zvpBVOhGGNnDTpBjLEgLlm 8lbkCQBykkMzhe7FxOHXXI7YGuyzo3ckje9xKsKlNGQLw1C5r+oeLMTKv3/J/eQO0n2m KcCRBO3Z1ZQ7VCSRLnydDlN+3S5DeV1tSIG2AG9hAC1BxGgdyg4SrWvXstMcgJzVhxJT 4ia4tD+3LEbc84twuYmvbNWYbdtL+QKEjd1oA9K5Epa+3ojyduprcwZf9VcgSM1AObeQ Zzlw== X-Gm-Message-State: AOJu0YynydaVDej2eOUO5FL78/RJBnvW8CPL4SKNF2Q7xxoYiMqgKi8/ Kcu+nHTFGBEd7MK9ta/l0bHfUENMprTYKIXKOe5h7Bzwnu3YcE0eQobFN7O++WZ1 X-Gm-Gg: ASbGncuoLwDiNz8VRqy0zgRb1TwMy9TsponFm4voNadTq9RiKnZpMwjieuM5FIhukR4 zalGoBILXi/b3ec6jun66U/fQUSBqvm6h/IpeJ0jpKEnTq4W356mplmfFFXXanB8G/P2yuUbu5h 0X2nuA5/hM/IS9JtkpFnhJ+2dQ2z4r3Fiks84oJ5469EZ8eTxeYXJpmNbkiHM4MHEO99m5cR5Vk g0triTtWOiyjo37cX2Gjzzoiq1RjP3AdsohAV9zd5GydWnVkRvdOu9rMgQX8T4J//CxBYwoNeXD Kp086njUD33n2p6Ugf1NH67QlIVVO3UNra3kFqpgFd/J4TNKc2WZjgNBom3ARHUGDTpw9kMYbjn D71IApzUVBaAOhIzOmLI194Ljmg== X-Google-Smtp-Source: AGHT+IFaBoA7Yo/WUepczn6Qruy6S1UWK4p6ZJeemhxCrjIh1ez95m44qZ2wj4raboedU+yZ8Qfwqw== X-Received: by 2002:a05:6512:1592:b0:553:2159:8716 with SMTP id 2adb3069b0e04-5533ba99850mr3614033e87.26.1748855104208; Mon, 02 Jun 2025 02:05:04 -0700 (PDT) From: Mykola Kvach To: xen-devel@lists.xenproject.org Cc: xakep.amatop@gmail.com, Stefano Stabellini , Julien Grall , Bertrand Marquis , Michal Orzel , Volodymyr Babchuk Subject: [PATCH v4][PART 2 10/10] xen/arm: Save/restore context on suspend/resume Date: Mon, 2 Jun 2025 12:04:21 +0300 Message-ID: <8f5904bc8f4d8ee39e248b3a2213fb3ed6815aff.1748848482.git.mykola_kvach@epam.com> X-Mailer: git-send-email 2.48.1 In-Reply-To: References: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @gmail.com) X-ZM-MESSAGEID: 1748855126286116600 Content-Type: text/plain; charset="utf-8" From: Mirela Simonovic The context of CPU general purpose and system control registers has to be saved on suspend and restored on resume. This is implemented in hyp_suspend and before the return from hyp_resume function. The hyp_suspend is invoked just before the PSCI system suspend call is issued to the ATF. The hyp_suspend has to return a non-zero value so that the calling 'if' statement evaluates to true, causing the system suspend to be invoked. Upon the resume, context saved on suspend will be restored, including the link register. Therefore, after restoring the context the control flow will return to the address pointed by the saved link register, which is the place from which the hyp_suspend was called. To ensure that the calling 'if' statement doesn't again evaluate to true and initiate system suspend, hyp_resume has to return a zero value after restoring the context. Note that the order of saving register context into cpu_context structure has to match the order of restoring. Support for ARM32 is not implemented. Instead, compilation fails with a build-time error if suspend is enabled for ARM32. Signed-off-by: Mirela Simonovic Signed-off-by: Saeed Nowshadi Signed-off-by: Mykyta Poturai Signed-off-by: Mykola Kvach --- Changes in v4: - produce build-time error for ARM32 when CONFIG_SYSTEM_SUSPEND is enabled - use register_t instead of uint64_t in cpu_context structure --- xen/arch/arm/arm64/head.S | 88 +++++++++++++++++++++++++++++- xen/arch/arm/include/asm/suspend.h | 20 +++++++ xen/arch/arm/suspend.c | 23 +++++++- 3 files changed, 127 insertions(+), 4 deletions(-) diff --git a/xen/arch/arm/arm64/head.S b/xen/arch/arm/arm64/head.S index 596e960152..70bb305ce3 100644 --- a/xen/arch/arm/arm64/head.S +++ b/xen/arch/arm/arm64/head.S @@ -562,6 +562,52 @@ END(efi_xen_start) #endif /* CONFIG_ARM_EFI */ =20 #ifdef CONFIG_SYSTEM_SUSPEND +/* + * int hyp_suspend(struct cpu_context *ptr) + * + * x0 - pointer to the storage where callee's context will be saved + * + * CPU context saved here will be restored on resume in hyp_resume functio= n. + * hyp_suspend shall return a non-zero value. Upon restoring context + * hyp_resume shall return value zero instead. From C code that invokes + * hyp_suspend, the return value is interpreted to determine whether the c= ontext + * is saved (hyp_suspend) or restored (hyp_resume). + */ +FUNC(hyp_suspend) + /* Store callee-saved registers */ + stp x19, x20, [x0], #16 + stp x21, x22, [x0], #16 + stp x23, x24, [x0], #16 + stp x25, x26, [x0], #16 + stp x27, x28, [x0], #16 + stp x29, lr, [x0], #16 + + /* Store stack-pointer */ + mov x2, sp + str x2, [x0], #8 + + /* Store system control registers */ + mrs x2, VBAR_EL2 + str x2, [x0], #8 + mrs x2, VTCR_EL2 + str x2, [x0], #8 + mrs x2, VTTBR_EL2 + str x2, [x0], #8 + mrs x2, TPIDR_EL2 + str x2, [x0], #8 + mrs x2, MDCR_EL2 + str x2, [x0], #8 + mrs x2, HSTR_EL2 + str x2, [x0], #8 + mrs x2, CPTR_EL2 + str x2, [x0], #8 + mrs x2, HCR_EL2 + str x2, [x0], #8 + + /* hyp_suspend must return a non-zero value */ + mov x0, #1 + ret +END(hyp_suspend) =20 FUNC(hyp_resume) /* Initialize the UART if earlyprintk has been enabled. */ @@ -580,7 +626,47 @@ FUNC(hyp_resume) b enable_secondary_cpu_mm =20 mmu_resumed: - b . + /* Now we can access the cpu_context, so restore the context here = */ + ldr x0, =3Dcpu_context + + /* Restore callee-saved registers */ + ldp x19, x20, [x0], #16 + ldp x21, x22, [x0], #16 + ldp x23, x24, [x0], #16 + ldp x25, x26, [x0], #16 + ldp x27, x28, [x0], #16 + ldp x29, lr, [x0], #16 + + /* Restore stack pointer */ + ldr x2, [x0], #8 + mov sp, x2 + + /* Restore system control registers */ + ldr x2, [x0], #8 + msr VBAR_EL2, x2 + ldr x2, [x0], #8 + msr VTCR_EL2, x2 + ldr x2, [x0], #8 + msr VTTBR_EL2, x2 + ldr x2, [x0], #8 + msr TPIDR_EL2, x2 + ldr x2, [x0], #8 + msr MDCR_EL2, x2 + ldr x2, [x0], #8 + msr HSTR_EL2, x2 + ldr x2, [x0], #8 + msr CPTR_EL2, x2 + ldr x2, [x0], #8 + msr HCR_EL2, x2 + isb + + /* Since context is restored return from this function will appear= as + * return from hyp_suspend. To distinguish a return from hyp_suspe= nd + * which is called upon finalizing the suspend, as opposed to retu= rn + * from this function which executes on resume, we need to return = zero + * value here. */ + mov x0, #0 + ret END(hyp_resume) =20 #endif /* CONFIG_SYSTEM_SUSPEND */ diff --git a/xen/arch/arm/include/asm/suspend.h b/xen/arch/arm/include/asm/= suspend.h index 55041a5d06..ae71ccb87b 100644 --- a/xen/arch/arm/include/asm/suspend.h +++ b/xen/arch/arm/include/asm/suspend.h @@ -5,9 +5,29 @@ =20 #ifdef CONFIG_SYSTEM_SUSPEND =20 +#ifdef CONFIG_ARM_64 +struct cpu_context { + register_t callee_regs[12]; + register_t sp; + register_t vbar_el2; + register_t vtcr_el2; + register_t vttbr_el2; + register_t tpidr_el2; + register_t mdcr_el2; + register_t hstr_el2; + register_t cptr_el2; + register_t hcr_el2; +} __aligned(16); +#else +#error "Define cpu_context structure for arm32" +#endif + +extern struct cpu_context cpu_context; + int host_system_suspend(void); =20 void hyp_resume(void); +int hyp_suspend(struct cpu_context *ptr); =20 #endif /* CONFIG_SYSTEM_SUSPEND */ =20 diff --git a/xen/arch/arm/suspend.c b/xen/arch/arm/suspend.c index f3ad8517ac..b1b8644a75 100644 --- a/xen/arch/arm/suspend.c +++ b/xen/arch/arm/suspend.c @@ -1,6 +1,7 @@ /* SPDX-License-Identifier: GPL-2.0-only */ =20 #include +#include #include #include #include @@ -17,6 +18,8 @@ * - Investigate feasibility and need for implementing system suspend on = ARM32 */ =20 +struct cpu_context cpu_context; + /* Xen suspend. Note: data is not used (suspend is the suspend to RAM) */ static long system_suspend(void *data) { @@ -73,9 +76,23 @@ static long system_suspend(void *data) */ update_boot_mapping(true); =20 - status =3D call_psci_system_suspend(); - if ( status ) - dprintk(XENLOG_WARNING, "PSCI system suspend failed, err=3D%d\n", = status); + if ( hyp_suspend(&cpu_context) ) + { + status =3D call_psci_system_suspend(); + /* + * If suspend is finalized properly by above system suspend PSCI c= all, + * the code below in this 'if' branch will never execute. Execution + * will continue from hyp_resume which is the hypervisor's resume = point. + * In hyp_resume CPU context will be restored and since link-regis= ter is + * restored as well, it will appear to return from hyp_suspend. The + * difference in returning from hyp_suspend on system suspend vers= us + * resume is in function's return value: on suspend, the return va= lue is + * a non-zero value, on resume it is zero. That is why the control= flow + * will not re-enter this 'if' branch on resume. + */ + if ( status ) + dprintk(XENLOG_WARNING, "PSCI system suspend failed, err=3D%d\= n", status); + } =20 system_state =3D SYS_STATE_resume; update_boot_mapping(false); --=20 2.48.1