[PATCH v2 10/19] target/ppc: Migrate DECR SPR

Nicholas Piggin posted 19 patches 1 year, 3 months ago
[PATCH v2 10/19] target/ppc: Migrate DECR SPR
Posted by Nicholas Piggin 1 year, 3 months ago
TCG does not maintain the DEC reigster in the SPR array, so it does get
migrated. TCG also needs to re-start the decrementer timer on the
destination machine.

Load and store the decrementer into the SPR when migrating. This works
for the level-triggered (book3s) decrementer, and should be compatible
with existing KVM machines that do keep the DEC value there.

This fixes lost decrementer interrupt on migration that can cause
hangs, as well as other problems including record-replay bugs.

Signed-off-by: Nicholas Piggin <npiggin@gmail.com>
---
 target/ppc/machine.c | 14 ++++++++++++++
 1 file changed, 14 insertions(+)

diff --git a/target/ppc/machine.c b/target/ppc/machine.c
index 8234e35d69..8a190c4853 100644
--- a/target/ppc/machine.c
+++ b/target/ppc/machine.c
@@ -209,6 +209,14 @@ static int cpu_pre_save(void *opaque)
     /* Used to retain migration compatibility for pre 6.0 for 601 machines. */
     env->hflags_compat_nmsr = 0;
 
+    if (tcg_enabled()) {
+        /*
+         * TCG does not maintain the DECR spr (unlike KVM) so have to save
+         * it here.
+         */
+        env->spr[SPR_DECR] = cpu_ppc_load_decr(env);
+    }
+
     return 0;
 }
 
@@ -319,6 +327,12 @@ static int cpu_post_load(void *opaque, int version_id)
         ppc_update_ciabr(env);
         ppc_update_daw0(env);
 #endif
+        /*
+         * TCG needs to re-start the decrementer timer and/or raise the
+         * interrupt. This works for level-triggered decrementer. Edge
+         * triggered types (including HDEC) would need to carry more state.
+         */
+        cpu_ppc_store_decr(env, env->spr[SPR_DECR]);
         pmu_mmcr01_updated(env);
     }
 
-- 
2.40.1
Re: [PATCH v2 10/19] target/ppc: Migrate DECR SPR
Posted by Cédric Le Goater 1 year, 3 months ago
Hello Nick,

On 8/8/23 06:19, Nicholas Piggin wrote:
> TCG does not maintain the DEC reigster in the SPR array, so it does get
> migrated. TCG also needs to re-start the decrementer timer on the
> destination machine.
> 
> Load and store the decrementer into the SPR when migrating. This works
> for the level-triggered (book3s) decrementer, and should be compatible
> with existing KVM machines that do keep the DEC value there.
> 
> This fixes lost decrementer interrupt on migration that can cause
> hangs, as well as other problems including record-replay bugs.
> 
> Signed-off-by: Nicholas Piggin <npiggin@gmail.com>
> ---
>   target/ppc/machine.c | 14 ++++++++++++++
>   1 file changed, 14 insertions(+)
> 
> diff --git a/target/ppc/machine.c b/target/ppc/machine.c
> index 8234e35d69..8a190c4853 100644
> --- a/target/ppc/machine.c
> +++ b/target/ppc/machine.c
> @@ -209,6 +209,14 @@ static int cpu_pre_save(void *opaque)
>       /* Used to retain migration compatibility for pre 6.0 for 601 machines. */
>       env->hflags_compat_nmsr = 0;
>   
> +    if (tcg_enabled()) {
> +        /*
> +         * TCG does not maintain the DECR spr (unlike KVM) so have to save
> +         * it here.
> +         */
> +        env->spr[SPR_DECR] = cpu_ppc_load_decr(env);
> +    }
> +
>       return 0;
>   }
>   
> @@ -319,6 +327,12 @@ static int cpu_post_load(void *opaque, int version_id)
>           ppc_update_ciabr(env);
>           ppc_update_daw0(env);
>   #endif
> +        /*
> +         * TCG needs to re-start the decrementer timer and/or raise the
> +         * interrupt. This works for level-triggered decrementer. Edge
> +         * triggered types (including HDEC) would need to carry more state.
> +         */
> +        cpu_ppc_store_decr(env, env->spr[SPR_DECR]);
>           pmu_mmcr01_updated(env);
>       }


This doesn't apply. I am missing some patch ?

Thanks,

C.
Re: [PATCH v2 10/19] target/ppc: Migrate DECR SPR
Posted by Nicholas Piggin 1 year, 3 months ago
On Wed Aug 9, 2023 at 10:56 PM AEST, Cédric Le Goater wrote:
> Hello Nick,
>
> On 8/8/23 06:19, Nicholas Piggin wrote:
> > TCG does not maintain the DEC reigster in the SPR array, so it does get
> > migrated. TCG also needs to re-start the decrementer timer on the
> > destination machine.
> > 
> > Load and store the decrementer into the SPR when migrating. This works
> > for the level-triggered (book3s) decrementer, and should be compatible
> > with existing KVM machines that do keep the DEC value there.
> > 
> > This fixes lost decrementer interrupt on migration that can cause
> > hangs, as well as other problems including record-replay bugs.
> > 
> > Signed-off-by: Nicholas Piggin <npiggin@gmail.com>
> > ---
> >   target/ppc/machine.c | 14 ++++++++++++++
> >   1 file changed, 14 insertions(+)
> > 
> > diff --git a/target/ppc/machine.c b/target/ppc/machine.c
> > index 8234e35d69..8a190c4853 100644
> > --- a/target/ppc/machine.c
> > +++ b/target/ppc/machine.c
> > @@ -209,6 +209,14 @@ static int cpu_pre_save(void *opaque)
> >       /* Used to retain migration compatibility for pre 6.0 for 601 machines. */
> >       env->hflags_compat_nmsr = 0;
> >   
> > +    if (tcg_enabled()) {
> > +        /*
> > +         * TCG does not maintain the DECR spr (unlike KVM) so have to save
> > +         * it here.
> > +         */
> > +        env->spr[SPR_DECR] = cpu_ppc_load_decr(env);
> > +    }
> > +
> >       return 0;
> >   }
> >   
> > @@ -319,6 +327,12 @@ static int cpu_post_load(void *opaque, int version_id)
> >           ppc_update_ciabr(env);
> >           ppc_update_daw0(env);
> >   #endif
> > +        /*
> > +         * TCG needs to re-start the decrementer timer and/or raise the
> > +         * interrupt. This works for level-triggered decrementer. Edge
> > +         * triggered types (including HDEC) would need to carry more state.
> > +         */
> > +        cpu_ppc_store_decr(env, env->spr[SPR_DECR]);
> >           pmu_mmcr01_updated(env);
> >       }
>
>
> This doesn't apply. I am missing some patch ?

Oh, it's a small dependency on the debug patches, sorry forgot to
mention (see the ciabr/daw0 in the top of the context).

Thanks,
Nick