During a kexec restart, only the kernel is reloaded while the state held
in TF-A and the platform firmware persists. This leaves the freshly
booted kernel out of sync with the firmware for things like the SGI
number, shutdown scope, allocated peripheral devices and registered
notifier callbacks, which can lead to unexpected behaviour after a
kexec.

Introduce a mechanism to clear the stale firmware/TF-A state so the
reloaded kernel starts from a clean baseline:

  - Add TF_A_CLEAR_PM_STATE to clear TF-A specific PM state.
  - Add PM_DEV_ALL_PERIPH to release all peripheral devices in one call.
  - Add PM_ALL_NOTIFIERS to unregister all notifier callbacks in one call.

On a graceful kexec reboot the cleanup is performed from
zynqmp_firmware_shutdown(); on a crash (kdump) restart it is performed
from zynqmp_firmware_probe() of the reloaded kernel. Each cleanup step
is guarded by a feature check so it degrades gracefully on firmware/TF-A
versions that do not implement the new APIs.

Changes in v2:
  - Drop patch 4/4 (feature check improvements) from this series per
    review feedback. Those changes are sent separately as:
    "[PATCH 1/2] firmware: xilinx: Use TF-A feature check for
    TF-A-specific APIs" and "[PATCH 2/2] firmware: xilinx: Propagate
    actual error from feature check"

Jay Buddhabhatti (3):
  firmware: xilinx: Add support to clear TF-A PM state
  firmware: xilinx: Release all peripheral devices from firmware
  firmware: xilinx: Clear firmware notifiers across kexec transitions

 drivers/firmware/xilinx/zynqmp.c     | 77 ++++++++++++++++++++++++++-
 include/linux/firmware/xlnx-zynqmp.h | 10 +++-
 2 files changed, 83 insertions(+), 4 deletions(-)

-- 
2.34.1

During a kexec restart, only the kernel is reloaded while the state held
in TF-A and the platform firmware persists. This leaves the freshly
booted kernel out of sync with the firmware for things like the SGI
number, shutdown scope, allocated peripheral devices and registered
notifier callbacks, which can lead to unexpected behaviour after a
kexec.

Introduce a mechanism to clear the stale firmware/TF-A state so the
reloaded kernel starts from a clean baseline:

  - Add TF_A_CLEAR_PM_STATE to clear TF-A specific PM state.
  - Add PM_DEV_ALL_PERIPH to release all peripheral devices in one call.
  - Add PM_ALL_NOTIFIERS to unregister all notifier callbacks in one call.

On a graceful kexec reboot the cleanup is performed from
zynqmp_firmware_shutdown(); on a crash (kdump) restart it is performed
from zynqmp_firmware_probe() of the reloaded kernel. Each cleanup step
is guarded by a feature check so it degrades gracefully on firmware/TF-A
versions that do not implement the new APIs.

Changes in v3:
  - Corrected subject prefix

Changes in v2:
  - Drop patch 4/4 (feature check improvements) from this series per
    review feedback. Those changes are sent separately as:
    "[PATCH 1/2] firmware: xilinx: Use TF-A feature check for
    TF-A-specific APIs" and "[PATCH 2/2] firmware: xilinx: Propagate
    actual error from feature check"

Jay Buddhabhatti (3):
  firmware: xilinx: Add support to clear TF-A PM state
  firmware: xilinx: Release all peripheral devices from firmware
  firmware: xilinx: Clear firmware notifiers across kexec transitions

 drivers/firmware/xilinx/zynqmp.c     | 75 +++++++++++++++++++++++++++-
 include/linux/firmware/xlnx-zynqmp.h | 10 +++-
 2 files changed, 83 insertions(+), 2 deletions(-)

-- 
2.34.1

Currently, during a kexec restart, only the kernel is reloaded, while the
TF-A specific data remain unchanged. This leads to a mismatch between the
kernel state and the TF-A firmware state like SGI number and shutdown
scope variable.

For example, the kernel registers an SGI number with TF-A so that TF-A can
notify the kernel of events via that SGI. TF-A stores this SGI number in
its internal state. After a kexec, the newly loaded kernel re-registers
and may request a different SGI number, but the stale value programmed in
TF-A remains, so event notifications are delivered on the old SGI and are
missed by the new kernel. The shutdown scope variable has a similar stale
state problem.

To resolve this, the TF_A_CLEAR_PM_STATE PM API is introduced to clear
TF-A-specific state during kexec. On a graceful reboot, this API is
triggered by zynqmp_firmware_shutdown(), while in a crash kernel scenario,
it is invoked by zynqmp_firmware_probe() in the reloaded kernel.

Signed-off-by: Jay Buddhabhatti <jay.buddhabhatti@amd.com>
Reviewed-by: Radhey Shyam Pandey <radhey.shyam.pandey@amd.com>
Reviewed-by: Prasanna Kumar T S M <ptsm@linux.microsoft.com>
---
 drivers/firmware/xilinx/zynqmp.c     | 48 +++++++++++++++++++++++++++-
 include/linux/firmware/xlnx-zynqmp.h |  3 +-
 2 files changed, 49 insertions(+), 2 deletions(-)

diff --git a/drivers/firmware/xilinx/zynqmp.c b/drivers/firmware/xilinx/zynqmp.c
index af838b2dc327..95ffd8f33ce9 100644
--- a/drivers/firmware/xilinx/zynqmp.c
+++ b/drivers/firmware/xilinx/zynqmp.c
@@ -3,7 +3,7 @@
  * Xilinx Zynq MPSoC Firmware layer
  *
  *  Copyright (C) 2014-2022 Xilinx, Inc.
- *  Copyright (C) 2022 - 2025 Advanced Micro Devices, Inc.
+ *  Copyright (C) 2022 - 2026 Advanced Micro Devices, Inc.
  *
  *  Michal Simek <michal.simek@amd.com>
  *  Davorin Mista <davorin.mista@aggios.com>
@@ -13,6 +13,7 @@
 
 #include <linux/arm-smccc.h>
 #include <linux/compiler.h>
+#include <linux/crash_dump.h>
 #include <linux/device.h>
 #include <linux/init.h>
 #include <linux/mfd/core.h>
@@ -2065,6 +2066,42 @@ static struct attribute *zynqmp_firmware_attrs[] = {
 
 ATTRIBUTE_GROUPS(zynqmp_firmware);
 
+/**
+ * zynqmp_clear_pm_state() - Clear subsystem state
+ * @dev: Device pointer used for logging
+ *
+ * Clears PM specific data in TF-A.
+ *
+ * Return: Returns status, either success or error
+ */
+static int zynqmp_clear_pm_state(struct device *dev)
+{
+	u32 pm_family_code;
+	int ret;
+
+	/* Get the Family code of platform */
+	ret = zynqmp_pm_get_family_info(&pm_family_code);
+	if (ret < 0)
+		return ret;
+
+	if (pm_family_code != PM_ZYNQMP_FAMILY_CODE) {
+		/* Check if the TF-A supports the TF_A_CLEAR_PM_STATE */
+		ret = do_feature_check_call(TF_A_CLEAR_PM_STATE);
+		if (ret >= 0 && ((ret & FIRMWARE_VERSION_MASK) >= PM_API_VERSION_1)) {
+			/* Clear PM specific data in TF-A */
+			ret = zynqmp_pm_invoke_fn(TF_A_CLEAR_PM_STATE, NULL, 0);
+			if (ret)
+				dev_err(dev,
+					"Failed to clear TF-A specific subsystem state: %d\n", ret);
+		} else {
+			dev_warn(dev, "TF_A_CLEAR_PM_STATE is not supported in TF-A: %d\n", ret);
+			ret = 0;
+		}
+	}
+
+	return ret;
+}
+
 static int zynqmp_firmware_probe(struct platform_device *pdev)
 {
 	struct device *dev = &pdev->dev;
@@ -2118,6 +2155,9 @@ static int zynqmp_firmware_probe(struct platform_device *pdev)
 	if (ret < 0)
 		return ret;
 
+	if (is_kdump_kernel())
+		zynqmp_clear_pm_state(dev);
+
 	/* Check trustzone version number */
 	ret = zynqmp_pm_get_trustzone_version(&pm_tz_version);
 	if (ret)
@@ -2151,6 +2191,11 @@ static int zynqmp_firmware_probe(struct platform_device *pdev)
 	return of_platform_populate(dev->of_node, NULL, NULL, dev);
 }
 
+static void zynqmp_firmware_shutdown(struct platform_device *pdev)
+{
+	zynqmp_clear_pm_state(&pdev->dev);
+}
+
 static void zynqmp_firmware_remove(struct platform_device *pdev)
 {
 	struct pm_api_feature_data *feature_data;
@@ -2210,5 +2255,6 @@ static struct platform_driver zynqmp_firmware_driver = {
 	},
 	.probe = zynqmp_firmware_probe,
 	.remove = zynqmp_firmware_remove,
+	.shutdown = zynqmp_firmware_shutdown,
 };
 module_platform_driver(zynqmp_firmware_driver);
diff --git a/include/linux/firmware/xlnx-zynqmp.h b/include/linux/firmware/xlnx-zynqmp.h
index 7e27b0f7bf7e..baaa88b0b197 100644
--- a/include/linux/firmware/xlnx-zynqmp.h
+++ b/include/linux/firmware/xlnx-zynqmp.h
@@ -3,7 +3,7 @@
  * Xilinx Zynq MPSoC Firmware layer
  *
  *  Copyright (C) 2014-2021 Xilinx
- *  Copyright (C) 2022 - 2025 Advanced Micro Devices, Inc.
+ *  Copyright (C) 2022 - 2026 Advanced Micro Devices, Inc.
  *
  *  Michal Simek <michal.simek@amd.com>
  *  Davorin Mista <davorin.mista@aggios.com>
@@ -66,6 +66,7 @@
 #define FIRMWARE_VERSION_MASK		0xFFFFU
 
 /* ATF only commands */
+#define TF_A_CLEAR_PM_STATE		0xa05
 #define TF_A_PM_REGISTER_SGI		0xa04
 #define PM_GET_TRUSTZONE_VERSION	0xa03
 #define PM_SET_SUSPEND_MODE		0xa02
-- 
2.34.1

On Fri, Jul 24, 2026 at 03:19:46AM -0700, Jay Buddhabhatti wrote:
> Currently, during a kexec restart, only the kernel is reloaded, while the
> TF-A specific data remain unchanged. This leads to a mismatch between the
> kernel state and the TF-A firmware state like SGI number and shutdown
> scope variable.
> 

Avoid using TF-A in general as it can be any other firmware as well.
You can always generalise as PSCI or EL3 firmware.

> For example, the kernel registers an SGI number with TF-A so that TF-A can
> notify the kernel of events via that SGI. TF-A stores this SGI number in
> its internal state. After a kexec, the newly loaded kernel re-registers
> and may request a different SGI number, but the stale value programmed in
> TF-A remains, so event notifications are delivered on the old SGI and are
> missed by the new kernel. The shutdown scope variable has a similar stale
> state problem.
>

Doesn't it reset it every time CPU_OFF or CPU_ON is called ?

> To resolve this, the TF_A_CLEAR_PM_STATE PM API is introduced to clear
> TF-A-specific state during kexec. On a graceful reboot, this API is
> triggered by zynqmp_firmware_shutdown(), while in a crash kernel scenario,
> it is invoked by zynqmp_firmware_probe() in the reloaded kernel.
> 

I don't see the need for such an API just for kexec needs. You must deal
with it as part of CPU_ON as it should start with clean slate as SGIs are
per CPU, no ?

-- 
Regards,
Sudeep

During a kexec restart, only the kernel is reloaded while devices
allocated in firmware persist, causing state mismatches between the
kernel and firmware.

Introduce PM_DEV_ALL_PERIPH node ID (0x18224FFFU) to release all
peripheral devices during kexec. On graceful restarts, this happens in
zynqmp_firmware_shutdown(). On crash kernel restarts, it happens in
zynqmp_firmware_probe() of the reloaded kernel.

Releasing all peripherals depends on firmware support for the
PM_DEV_ALL_PERIPH node ID. On firmware that does not implement it (the
feature check reports a version below PM_API_VERSION_3) the release is
skipped and a warning such as "Bulk device release is not supported by
firmware" is logged, e.g. on Versal NET firmware that predates this API.

Signed-off-by: Jay Buddhabhatti <jay.buddhabhatti@amd.com>
Reviewed-by: Radhey Shyam Pandey <radhey.shyam.pandey@amd.com>
Reviewed-by: Prasanna Kumar T S M <ptsm@linux.microsoft.com>
---
 drivers/firmware/xilinx/zynqmp.c     | 15 ++++++++++++++-
 include/linux/firmware/xlnx-zynqmp.h |  4 ++++
 2 files changed, 18 insertions(+), 1 deletion(-)

diff --git a/drivers/firmware/xilinx/zynqmp.c b/drivers/firmware/xilinx/zynqmp.c
index 95ffd8f33ce9..64d2109eefd2 100644
--- a/drivers/firmware/xilinx/zynqmp.c
+++ b/drivers/firmware/xilinx/zynqmp.c
@@ -2070,7 +2070,7 @@ ATTRIBUTE_GROUPS(zynqmp_firmware);
  * zynqmp_clear_pm_state() - Clear subsystem state
  * @dev: Device pointer used for logging
  *
- * Clears PM specific data in TF-A.
+ * Clears PM specific data in TF-A and firmware.
  *
  * Return: Returns status, either success or error
  */
@@ -2097,6 +2097,19 @@ static int zynqmp_clear_pm_state(struct device *dev)
 			dev_warn(dev, "TF_A_CLEAR_PM_STATE is not supported in TF-A: %d\n", ret);
 			ret = 0;
 		}
+
+		/* Check if the firmware supports the PM_DEV_ALL_PERIPH node ID */
+		ret = do_feature_check_call(PM_RELEASE_NODE);
+		if (ret >= 0 && ((ret & FIRMWARE_VERSION_MASK) >= PM_API_VERSION_3)) {
+			/* Attempt to release all peripheral devices via firmware */
+			ret = zynqmp_pm_release_node(PM_DEV_ALL_PERIPH);
+			if (ret)
+				dev_err(dev, "Failed to release all peripheral devices: %d\n", ret);
+		} else {
+			dev_warn(dev,
+				 "Bulk device release is not supported by firmware: %d\n", ret);
+			ret = 0;
+		}
 	}
 
 	return ret;
diff --git a/include/linux/firmware/xlnx-zynqmp.h b/include/linux/firmware/xlnx-zynqmp.h
index baaa88b0b197..ac39e5492961 100644
--- a/include/linux/firmware/xlnx-zynqmp.h
+++ b/include/linux/firmware/xlnx-zynqmp.h
@@ -50,6 +50,7 @@
 /* PM API versions */
 #define PM_API_VERSION_1	1
 #define PM_API_VERSION_2	2
+#define PM_API_VERSION_3	3
 
 #define PM_PINCTRL_PARAM_SET_VERSION	2
 
@@ -145,6 +146,9 @@
 #define XPM_EVENT_ERROR_MASK_NOC_NCR		BIT(13)
 #define XPM_EVENT_ERROR_MASK_NOC_CR		BIT(12)
 
+/* Node ID for all peripheral devices */
+#define PM_DEV_ALL_PERIPH	0x18224FFFU
+
 enum pm_module_id {
 	PM_MODULE_ID = 0x0,
 	XPM_MODULE_ID = 0x2,
-- 
2.34.1

During a kexec restart, only the kernel is reloaded but notifier callbacks
in firmware persist, causing state mismatches between kernel and firmware.

To address this, introduce PM_ALL_NOTIFIERS node ID to unregister all
notifier callbacks during kexec. On a graceful kexec restart, this occurs
in zynqmp_firmware_shutdown(). On a crash kernel restart, it happens in
zynqmp_firmware_probe() in the reloaded kernel.

Unregistering all notifiers depends on firmware support for the
PM_ALL_NOTIFIERS node ID. On firmware that does not implement it (the
feature check reports a version below PM_API_VERSION_3) the step is
skipped and a warning such as "Firmware doesn't support unregister all
notifiers at once" is logged, e.g. on Versal NET firmware that predates
this API.

Signed-off-by: Jay Buddhabhatti <jay.buddhabhatti@amd.com>
Reviewed-by: Radhey Shyam Pandey <radhey.shyam.pandey@amd.com>
Reviewed-by: Prasanna Kumar T S M <ptsm@linux.microsoft.com>
---
 drivers/firmware/xilinx/zynqmp.c     | 14 ++++++++++++++
 include/linux/firmware/xlnx-zynqmp.h |  3 +++
 2 files changed, 17 insertions(+)

diff --git a/drivers/firmware/xilinx/zynqmp.c b/drivers/firmware/xilinx/zynqmp.c
index 64d2109eefd2..fc7212f554ee 100644
--- a/drivers/firmware/xilinx/zynqmp.c
+++ b/drivers/firmware/xilinx/zynqmp.c
@@ -2110,6 +2110,20 @@ static int zynqmp_clear_pm_state(struct device *dev)
 				 "Bulk device release is not supported by firmware: %d\n", ret);
 			ret = 0;
 		}
+
+		/* Check if the firmware supports the PM_ALL_NOTIFIERS node ID */
+		ret = do_feature_check_call(PM_REGISTER_NOTIFIER);
+		if (ret >= 0 && ((ret & FIRMWARE_VERSION_MASK) >= PM_API_VERSION_3)) {
+			/* Attempt to unregister all notifier callbacks via firmware */
+			ret = zynqmp_pm_register_notifier(PM_ALL_NOTIFIERS, 0, 0, 0);
+			if (ret)
+				dev_err(dev, "Failed to unregister all notifiers: %d\n", ret);
+		} else {
+			dev_warn(dev,
+				 "Firmware doesn't support unregister all notifiers at once: %d\n",
+				 ret);
+			ret = 0;
+		}
 	}
 
 	return ret;
diff --git a/include/linux/firmware/xlnx-zynqmp.h b/include/linux/firmware/xlnx-zynqmp.h
index ac39e5492961..69a2f74269f3 100644
--- a/include/linux/firmware/xlnx-zynqmp.h
+++ b/include/linux/firmware/xlnx-zynqmp.h
@@ -149,6 +149,9 @@
 /* Node ID for all peripheral devices */
 #define PM_DEV_ALL_PERIPH	0x18224FFFU
 
+/* Node ID for all notifier callbacks */
+#define PM_ALL_NOTIFIERS	0xFFFFFFFFU
+
 enum pm_module_id {
 	PM_MODULE_ID = 0x0,
 	XPM_MODULE_ID = 0x2,
-- 
2.34.1