From: Jean-Philippe Brucker <jean-philippe@linaro.org>

Previously, all `PSCI_CPU_{ON, OFF}` calls were handled directly by KVM.
However, with the introduction of this new vCPU hotplug-like feature, these
hypervisor calls are now trapped to QEMU for policy checks. This shift can lead
to inconsistent vCPU states between KVM and QEMU, particularly when the vCPU has
been recently administratively enabled and is transitioning from either unparked
state in QOM due to 'lazy realization' or even from 'powered-off' state.
Therefore, it is crucial to synchronize the vCPU state with KVM, especially in
the context of a cold reset of the QOM vCPU. The same applies when PSCI CPU_OFF
is being handled by Qemu, it must ensure that kVM vCPUs are powered-off as well.

To ensure this synchronization, mark the QOM vCPU as "dirty" to trigger a call
to `kvm_arch_put_registers()`. This guarantees that KVM’s `MP_STATE` is updated
accordingly, forcing synchronization of the `mp_state` between QEMU and KVM.

Signed-off-by: Jean-Philippe Brucker <jean-philippe@linaro.org>
Signed-off-by: Salil Mehta <salil.mehta@huawei.com>
---
 target/arm/arm-powerctl.c | 1 +
 target/arm/kvm.c          | 7 +++++++
 2 files changed, 8 insertions(+)

diff --git a/target/arm/arm-powerctl.c b/target/arm/arm-powerctl.c
index ab4422b261..89074918a9 100644
--- a/target/arm/arm-powerctl.c
+++ b/target/arm/arm-powerctl.c
@@ -263,6 +263,7 @@ static void arm_set_cpu_off_async_work(CPUState *target_cpu_state,
 
     assert(bql_locked());
     target_cpu->power_state = PSCI_OFF;
+    target_cpu_state->vcpu_dirty = true;
     target_cpu_state->halted = 1;
     target_cpu_state->exception_index = EXCP_HLT;
 }
diff --git a/target/arm/kvm.c b/target/arm/kvm.c
index 98eb6db9ed..c4b68a0b17 100644
--- a/target/arm/kvm.c
+++ b/target/arm/kvm.c
@@ -1026,6 +1026,7 @@ bool kvm_arm_cpu_post_load(ARMCPU *cpu)
 void kvm_arm_reset_vcpu(ARMCPU *cpu)
 {
     int ret;
+    CPUState *cs = CPU(cpu);
 
     /* Re-init VCPU so that all registers are set to
      * their respective reset values.
@@ -1047,6 +1048,12 @@ void kvm_arm_reset_vcpu(ARMCPU *cpu)
      * for the same reason we do so in kvm_arch_get_registers().
      */
     write_list_to_cpustate(cpu);
+
+    /*
+     * Ensure we call kvm_arch_put_registers(). The vCPU isn't marked dirty if
+     * it was parked in KVM and is now booting from a PSCI CPU_ON call.
+     */
+    cs->vcpu_dirty = true;
 }
 
 void kvm_arm_create_host_vcpu(ARMCPU *cpu)
-- 
2.34.1