docs/devel/migration: start a debugging section

[PATCH] docs/devel/migration: start a debugging section

Posted by Marc-André Lureau 4 years ago

Explain how to use analyze-migration.py, this may help.

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
---
 docs/devel/migration.rst | 20 ++++++++++++++++++++
 1 file changed, 20 insertions(+)

diff --git a/docs/devel/migration.rst b/docs/devel/migration.rst
index e88918f7639..2eb08624fc3 100644
--- a/docs/devel/migration.rst
+++ b/docs/devel/migration.rst
@@ -50,6 +50,26 @@ All these migration protocols use the same infrastructure to
 save/restore state devices.  This infrastructure is shared with the
 savevm/loadvm functionality.
 
+Debugging
+=========
+
+The migration stream can be analyzed thanks to `scripts/analyze_migration.py`.
+
+Example usage:
+
+.. code-block:: shell
+
+  $ qemu-system-x86_64
+   (qemu) migrate "exec:cat > mig"
+  $ ./scripts/analyze_migration.py -f mig
+  {
+    "ram (3)": {
+        "section sizes": {
+            "pc.ram": "0x0000000008000000",
+  ...
+
+See also ``analyze_migration.py -h`` help for more options.
+
 Common infrastructure
 =====================
 
-- 
2.26.0.rc2.42.g98cedd0233

Re: [PATCH] docs/devel/migration: start a debugging section

Posted by Dr. David Alan Gilbert 4 years ago

* Marc-André Lureau (marcandre.lureau@redhat.com) wrote:
> Explain how to use analyze-migration.py, this may help.
> 
> Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>

Yes, it's quite useful for things like this; although things where
multiple instances/instance_id's are the problem can be tricikier and
some tracing can help.


Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>

> ---
>  docs/devel/migration.rst | 20 ++++++++++++++++++++
>  1 file changed, 20 insertions(+)
> 
> diff --git a/docs/devel/migration.rst b/docs/devel/migration.rst
> index e88918f7639..2eb08624fc3 100644
> --- a/docs/devel/migration.rst
> +++ b/docs/devel/migration.rst
> @@ -50,6 +50,26 @@ All these migration protocols use the same infrastructure to
>  save/restore state devices.  This infrastructure is shared with the
>  savevm/loadvm functionality.
>  
> +Debugging
> +=========
> +
> +The migration stream can be analyzed thanks to `scripts/analyze_migration.py`.
> +
> +Example usage:
> +
> +.. code-block:: shell
> +
> +  $ qemu-system-x86_64
> +   (qemu) migrate "exec:cat > mig"
> +  $ ./scripts/analyze_migration.py -f mig
> +  {
> +    "ram (3)": {
> +        "section sizes": {
> +            "pc.ram": "0x0000000008000000",
> +  ...
> +
> +See also ``analyze_migration.py -h`` help for more options.
> +
>  Common infrastructure
>  =====================
>  
> -- 
> 2.26.0.rc2.42.g98cedd0233
> 
--
Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK

Re: [PATCH] docs/devel/migration: start a debugging section

Posted by Daniel P. Berrangé 4 years ago

On Mon, Mar 30, 2020 at 07:48:52PM +0200, Marc-André Lureau wrote:
> Explain how to use analyze-migration.py, this may help.
> 
> Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
> ---
>  docs/devel/migration.rst | 20 ++++++++++++++++++++
>  1 file changed, 20 insertions(+)
> 
> diff --git a/docs/devel/migration.rst b/docs/devel/migration.rst
> index e88918f7639..2eb08624fc3 100644
> --- a/docs/devel/migration.rst
> +++ b/docs/devel/migration.rst
> @@ -50,6 +50,26 @@ All these migration protocols use the same infrastructure to
>  save/restore state devices.  This infrastructure is shared with the
>  savevm/loadvm functionality.
>  
> +Debugging
> +=========
> +
> +The migration stream can be analyzed thanks to `scripts/analyze_migration.py`.
> +
> +Example usage:
> +
> +.. code-block:: shell
> +
> +  $ qemu-system-x86_64
> +   (qemu) migrate "exec:cat > mig"
> +  $ ./scripts/analyze_migration.py -f mig
> +  {
> +    "ram (3)": {
> +        "section sizes": {
> +            "pc.ram": "0x0000000008000000",
> +  ...
> +
> +See also ``analyze_migration.py -h`` help for more options.

Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>

Side note: who else loves the fact that we have both spellings
of analyse - 'z' and 's' in the scripts directory. We ought to
pick one :-)

Another side note - could analyze_migration be used as the basis
for doing compatibility testing of migration support between QEMU
versions ?

eg, we can have a pair of files  "foo.argv" and "foo.migration"
containing the QEMU argv to run, and the corresponding output
expected from "analyze_migration.py" (perhaps we certain bits like
precise register values scrubbed).  On each release commit a new
pairs to git for new machine types, with various interesting argv,
and then we can validate that all future versions of QEMU produce
the same output & thus remain migration compatible ?  This feels
like this kind of approach could have caught the regression today
with the duplicated  serial device migration output sections. 

Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

Re: [PATCH] docs/devel/migration: start a debugging section

Posted by Dr. David Alan Gilbert 4 years ago

* Daniel P. Berrangé (berrange@redhat.com) wrote:
> On Mon, Mar 30, 2020 at 07:48:52PM +0200, Marc-André Lureau wrote:
> > Explain how to use analyze-migration.py, this may help.
> > 
> > Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
> > ---
> >  docs/devel/migration.rst | 20 ++++++++++++++++++++
> >  1 file changed, 20 insertions(+)
> > 
> > diff --git a/docs/devel/migration.rst b/docs/devel/migration.rst
> > index e88918f7639..2eb08624fc3 100644
> > --- a/docs/devel/migration.rst
> > +++ b/docs/devel/migration.rst
> > @@ -50,6 +50,26 @@ All these migration protocols use the same infrastructure to
> >  save/restore state devices.  This infrastructure is shared with the
> >  savevm/loadvm functionality.
> >  
> > +Debugging
> > +=========
> > +
> > +The migration stream can be analyzed thanks to `scripts/analyze_migration.py`.
> > +
> > +Example usage:
> > +
> > +.. code-block:: shell
> > +
> > +  $ qemu-system-x86_64
> > +   (qemu) migrate "exec:cat > mig"
> > +  $ ./scripts/analyze_migration.py -f mig
> > +  {
> > +    "ram (3)": {
> > +        "section sizes": {
> > +            "pc.ram": "0x0000000008000000",
> > +  ...
> > +
> > +See also ``analyze_migration.py -h`` help for more options.
> 
> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
> 
> Side note: who else loves the fact that we have both spellings
> of analyse - 'z' and 's' in the scripts directory. We ought to
> pick one :-)

Yes I hit that.

> Another side note - could analyze_migration be used as the basis
> for doing compatibility testing of migration support between QEMU
> versions ?

I use scripts/vmstate-static-checker.py for this.

> eg, we can have a pair of files  "foo.argv" and "foo.migration"
> containing the QEMU argv to run, and the corresponding output
> expected from "analyze_migration.py" (perhaps we certain bits like
> precise register values scrubbed).  On each release commit a new
> pairs to git for new machine types, with various interesting argv,
> and then we can validate that all future versions of QEMU produce
> the same output & thus remain migration compatible ?  This feels
> like this kind of approach could have caught the regression today
> with the duplicated  serial device migration output sections. 

It could; although it can be a bit more subtle - the analysis
is done on a separate XML structure that only mostly reflects the
data structure in the file.  For example, in the case we were
fighting today we inserted an extra block level as far as the XML sees
it, but that doesn't change the onwire format.

Dave


> Regards,
> Daniel
> -- 
> |: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
> |: https://libvirt.org         -o-            https://fstop138.berrange.com :|
> |: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|
--
Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK

Re: [PATCH] docs/devel/migration: start a debugging section

Posted by Dr. David Alan Gilbert 3 years, 11 months ago

* Marc-André Lureau (marcandre.lureau@redhat.com) wrote:
> Explain how to use analyze-migration.py, this may help.
> 
> Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>

Queued

> ---
>  docs/devel/migration.rst | 20 ++++++++++++++++++++
>  1 file changed, 20 insertions(+)
> 
> diff --git a/docs/devel/migration.rst b/docs/devel/migration.rst
> index e88918f7639..2eb08624fc3 100644
> --- a/docs/devel/migration.rst
> +++ b/docs/devel/migration.rst
> @@ -50,6 +50,26 @@ All these migration protocols use the same infrastructure to
>  save/restore state devices.  This infrastructure is shared with the
>  savevm/loadvm functionality.
>  
> +Debugging
> +=========
> +
> +The migration stream can be analyzed thanks to `scripts/analyze_migration.py`.
> +
> +Example usage:
> +
> +.. code-block:: shell
> +
> +  $ qemu-system-x86_64
> +   (qemu) migrate "exec:cat > mig"
> +  $ ./scripts/analyze_migration.py -f mig
> +  {
> +    "ram (3)": {
> +        "section sizes": {
> +            "pc.ram": "0x0000000008000000",
> +  ...
> +
> +See also ``analyze_migration.py -h`` help for more options.
> +
>  Common infrastructure
>  =====================
>  
> -- 
> 2.26.0.rc2.42.g98cedd0233
> 
> 
--
Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK