[PULL 00/29] Migration pull patches (second try)

Juan Quintela posted 29 patches 29 weeks ago
Test docker-mingw@fedora passed
Test checkpatch failed
Test docker-quick@centos7 passed
Test FreeBSD passed
Patches applied successfully (tree, apply log)
git fetch https://github.com/patchew-project/qemu tags/patchew/20200114092606.1761-1-quintela@redhat.com
Maintainers: Paolo Bonzini <pbonzini@redhat.com>, "Daniel P. Berrangé" <berrange@redhat.com>, Jason Wang <jasowang@redhat.com>, Eduardo Habkost <ehabkost@redhat.com>, Stefan Weil <sw@weilnetz.de>, "Marc-André Lureau" <marcandre.lureau@redhat.com>, Corey Minyard <cminyard@mvista.com>, Peter Maydell <peter.maydell@linaro.org>, Juan Quintela <quintela@redhat.com>, "Michael S. Tsirkin" <mst@redhat.com>, Stefan Berger <stefanb@linux.ibm.com>, Andrzej Zaborowski <balrogg@gmail.com>, David Gibson <david@gibson.dropbear.id.au>, Laurent Vivier <lvivier@redhat.com>, Thomas Huth <thuth@redhat.com>, Richard Henderson <rth@twiddle.net>, "Dr. David Alan Gilbert" <dgilbert@redhat.com>, Marcel Apfelbaum <marcel.apfelbaum@gmail.com>
backends/dbus-vmstate.c      |   3 +-
exec.c                       |   4 +-
hw/arm/stellaris.c           |   2 +-
hw/core/qdev.c               |   3 +-
hw/display/ads7846.c         |   2 +-
hw/i2c/core.c                |   2 +-
hw/input/stellaris_input.c   |   3 +-
hw/intc/apic_common.c        |   7 +-
hw/misc/max111x.c            |   3 +-
hw/net/eepro100.c            |   3 +-
hw/pci/pci.c                 |   2 +-
hw/ppc/spapr.c               |   2 +-
hw/timer/arm_timer.c         |   2 +-
hw/tpm/tpm_emulator.c        |   3 +-
include/migration/register.h |   2 +-
include/migration/vmstate.h  |  25 ++++-
include/qemu/queue.h         |  39 ++++++++
migration/migration.c        |  72 +++++++-------
migration/migration.h        |   1 +
migration/ram.c              | 183 ++++++++++++++++++++++++++---------
migration/savevm.c           |  61 ++++++++----
migration/trace-events       |   9 +-
migration/vmstate-types.c    |  70 ++++++++++++++
stubs/vmstate.c              |   2 +-
tests/qtest/migration-test.c |  97 ++++++++++++++++++-
tests/test-vmstate.c         | 170 ++++++++++++++++++++++++++++++++
vl.c                         |  10 +-
27 files changed, 653 insertions(+), 129 deletions(-)

[PULL 00/29] Migration pull patches (second try)

Posted by Juan Quintela 29 weeks ago
The following changes since commit 3c8a6575985b1652b45bfa670b5e1907d642cfa0:

  Merge remote-tracking branch 'remotes/kraxel/tags/usb-20200113-pull-request' into staging (2020-01-13 14:19:57 +0000)

are available in the Git repository at:

  https://github.com/juanquintela/qemu.git tags/migration-pull-pull-request

for you to fetch changes up to 4eafab585c091050b5ae63130f46fe46ac919c3a:

  migration: Support QLIST migration (2020-01-14 10:17:12 +0100)

----------------------------------------------------------------
Migration pull request
- updated QList patch
- initialize local msg variable

----------------------------------------------------------------

Alexey Romko (1):
  Bug #1829242 correction.

Daniel Henrique Barboza (1):
  ram.c: remove unneeded labels

Dr. David Alan Gilbert (1):
  migration: Rate limit inside host pages

Eric Auger (1):
  migration: Support QLIST migration

Fangrui Song (1):
  migration: Fix incorrect integer->float conversion caught by clang

Jiahui Cen (2):
  migration/multifd: fix nullptr access in terminating multifd threads
  migration/multifd: fix destroyed mutex access in terminating multifd
    threads

Juan Quintela (4):
  multifd: Initialize local variable
  migration-test: Add migration multifd test
  migration: Make sure that we don't call write() in case of error
  migration-test: introduce functions to handle string parameters

Laurent Vivier (2):
  migration-test: ppc64: fix FORTH test program
  runstate: ignore finishmigrate -> prelaunch transition

Marc-André Lureau (1):
  misc: use QEMU_IS_ALIGNED

Peter Xu (3):
  migration: Define VMSTATE_INSTANCE_ID_ANY
  migration: Change SaveStateEntry.instance_id into uint32_t
  apic: Use 32bit APIC ID for migration instance ID

Scott Cheloha (2):
  migration: add savevm_state_handler_remove()
  migration: savevm_state_handler_insert: constant-time element
    insertion

Wei Yang (8):
  migration/postcopy: reduce memset when it is zero page and
    matches_target_page_size
  migration/postcopy: wait for decompress thread in precopy
  migration/postcopy: count target page number to decide the
    place_needed
  migration/postcopy: set all_zero to true on the first target page
  migration/postcopy: enable random order target page arrival
  migration/postcopy: enable compress during postcopy
  migration/multifd: clean pages after filling packet
  migration/multifd: not use multifd during postcopy

Yury Kotov (2):
  migration: Fix the re-run check of the migrate-incoming command
  migration/ram: Yield periodically to the main loop

 backends/dbus-vmstate.c      |   3 +-
 exec.c                       |   4 +-
 hw/arm/stellaris.c           |   2 +-
 hw/core/qdev.c               |   3 +-
 hw/display/ads7846.c         |   2 +-
 hw/i2c/core.c                |   2 +-
 hw/input/stellaris_input.c   |   3 +-
 hw/intc/apic_common.c        |   7 +-
 hw/misc/max111x.c            |   3 +-
 hw/net/eepro100.c            |   3 +-
 hw/pci/pci.c                 |   2 +-
 hw/ppc/spapr.c               |   2 +-
 hw/timer/arm_timer.c         |   2 +-
 hw/tpm/tpm_emulator.c        |   3 +-
 include/migration/register.h |   2 +-
 include/migration/vmstate.h  |  25 ++++-
 include/qemu/queue.h         |  39 ++++++++
 migration/migration.c        |  72 +++++++-------
 migration/migration.h        |   1 +
 migration/ram.c              | 183 ++++++++++++++++++++++++++---------
 migration/savevm.c           |  61 ++++++++----
 migration/trace-events       |   9 +-
 migration/vmstate-types.c    |  70 ++++++++++++++
 stubs/vmstate.c              |   2 +-
 tests/qtest/migration-test.c |  97 ++++++++++++++++++-
 tests/test-vmstate.c         | 170 ++++++++++++++++++++++++++++++++
 vl.c                         |  10 +-
 27 files changed, 653 insertions(+), 129 deletions(-)

-- 
2.24.1


Re: [PULL 00/29] Migration pull patches (second try)

Posted by no-reply@patchew.org 29 weeks ago
Patchew URL: https://patchew.org/QEMU/20200114092606.1761-1-quintela@redhat.com/



Hi,

This series seems to have some coding style problems. See output below for
more information:

Subject: [PULL 00/29] Migration pull patches (second try)
Type: series
Message-id: 20200114092606.1761-1-quintela@redhat.com

=== TEST SCRIPT BEGIN ===
#!/bin/bash
git rev-parse base > /dev/null || exit 0
git config --local diff.renamelimit 0
git config --local diff.renames True
git config --local diff.algorithm histogram
./scripts/checkpatch.pl --mailback base..
=== TEST SCRIPT END ===

Switched to a new branch 'test'
0c3394b migration: Support QLIST migration
9f8433e apic: Use 32bit APIC ID for migration instance ID
b29c843 migration: Change SaveStateEntry.instance_id into uint32_t
efd00fa migration: Define VMSTATE_INSTANCE_ID_ANY
c9e615f Bug #1829242 correction.
90e17af migration/multifd: fix destroyed mutex access in terminating multifd threads
40d8962 migration/multifd: fix nullptr access in terminating multifd threads
3291d93 migration/multifd: not use multifd during postcopy
ac5f0e5 migration/multifd: clean pages after filling packet
3f9f3e0 migration/postcopy: enable compress during postcopy
b533fd6 migration/postcopy: enable random order target page arrival
51a912d migration/postcopy: set all_zero to true on the first target page
1d79640 migration/postcopy: count target page number to decide the place_needed
1bb0660 migration/postcopy: wait for decompress thread in precopy
2d9ee73 migration/postcopy: reduce memset when it is zero page and matches_target_page_size
1656129 migration/ram: Yield periodically to the main loop
34e9a27 migration: savevm_state_handler_insert: constant-time element insertion
eecc4fb migration: add savevm_state_handler_remove()
5eaf803 misc: use QEMU_IS_ALIGNED
1262d9d migration: Fix the re-run check of the migrate-incoming command
bda9064 migration: Fix incorrect integer->float conversion caught by clang
af366d8 migration: Rate limit inside host pages
b70f326 ram.c: remove unneeded labels
076fa7c runstate: ignore finishmigrate -> prelaunch transition
139b217 migration-test: ppc64: fix FORTH test program
f821049 migration-test: introduce functions to handle string parameters
70de133 migration: Make sure that we don't call write() in case of error
397caec migration-test: Add migration multifd test
9a898ef multifd: Initialize local variable

=== OUTPUT BEGIN ===
1/29 Checking commit 9a898efa8cdd (multifd: Initialize local variable)
ERROR: Missing Signed-off-by: line(s)

total: 1 errors, 0 warnings, 8 lines checked

Patch 1/29 has style problems, please review.  If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.

2/29 Checking commit 397caec25c6d (migration-test: Add migration multifd test)
3/29 Checking commit 70de13354cf7 (migration: Make sure that we don't call write() in case of error)
4/29 Checking commit f8210496ba76 (migration-test: introduce functions to handle string parameters)
5/29 Checking commit 139b217960ef (migration-test: ppc64: fix FORTH test program)
6/29 Checking commit 076fa7cc767e (runstate: ignore finishmigrate -> prelaunch transition)
7/29 Checking commit b70f326cad3e (ram.c: remove unneeded labels)
8/29 Checking commit af366d8d4aa3 (migration: Rate limit inside host pages)
9/29 Checking commit bda906418d94 (migration: Fix incorrect integer->float conversion caught by clang)
10/29 Checking commit 1262d9df6af7 (migration: Fix the re-run check of the migrate-incoming command)
11/29 Checking commit 5eaf8034e18e (misc: use QEMU_IS_ALIGNED)
12/29 Checking commit eecc4fb12fe4 (migration: add savevm_state_handler_remove())
13/29 Checking commit 34e9a27c5352 (migration: savevm_state_handler_insert: constant-time element insertion)
14/29 Checking commit 165612933a8a (migration/ram: Yield periodically to the main loop)
15/29 Checking commit 2d9ee73dc120 (migration/postcopy: reduce memset when it is zero page and matches_target_page_size)
16/29 Checking commit 1bb06608be50 (migration/postcopy: wait for decompress thread in precopy)
17/29 Checking commit 1d7964001e43 (migration/postcopy: count target page number to decide the place_needed)
18/29 Checking commit 51a912d14296 (migration/postcopy: set all_zero to true on the first target page)
19/29 Checking commit b533fd62c4f2 (migration/postcopy: enable random order target page arrival)
20/29 Checking commit 3f9f3e010fc0 (migration/postcopy: enable compress during postcopy)
21/29 Checking commit ac5f0e5b4216 (migration/multifd: clean pages after filling packet)
22/29 Checking commit 3291d939accf (migration/multifd: not use multifd during postcopy)
23/29 Checking commit 40d8962c5339 (migration/multifd: fix nullptr access in terminating multifd threads)
24/29 Checking commit 90e17aff65e2 (migration/multifd: fix destroyed mutex access in terminating multifd threads)
25/29 Checking commit c9e615f603c9 (Bug #1829242 correction.)
26/29 Checking commit efd00face1e8 (migration: Define VMSTATE_INSTANCE_ID_ANY)
27/29 Checking commit b29c84322acc (migration: Change SaveStateEntry.instance_id into uint32_t)
28/29 Checking commit 9f8433e74628 (apic: Use 32bit APIC ID for migration instance ID)
29/29 Checking commit 0c3394b08365 (migration: Support QLIST migration)
=== OUTPUT END ===

Test command exited with code: 1


The full log is available at
http://patchew.org/logs/20200114092606.1761-1-quintela@redhat.com/testing.checkpatch/?type=message.
---
Email generated automatically by Patchew [https://patchew.org/].
Please send your feedback to patchew-devel@redhat.com

Re: [PULL 00/29] Migration pull patches (second try)

Posted by Peter Maydell 29 weeks ago
On Tue, 14 Jan 2020 at 09:26, Juan Quintela <quintela@redhat.com> wrote:
>
> The following changes since commit 3c8a6575985b1652b45bfa670b5e1907d642cfa0:
>
>   Merge remote-tracking branch 'remotes/kraxel/tags/usb-20200113-pull-request' into staging (2020-01-13 14:19:57 +0000)
>
> are available in the Git repository at:
>
>   https://github.com/juanquintela/qemu.git tags/migration-pull-pull-request
>
> for you to fetch changes up to 4eafab585c091050b5ae63130f46fe46ac919c3a:
>
>   migration: Support QLIST migration (2020-01-14 10:17:12 +0100)
>
> ----------------------------------------------------------------
> Migration pull request
> - updated QList patch
> - initialize local msg variable
>
> ----------------------------------------------------------------

Still fails on hosts where ram_addr_t is 32 bits.
Looks like you still haven't got the fix for the problem
Dan pointed out where multifd_save_setup() is using the
wrong type when it calculates p->packetlen to allocate the
data structure.

thanks
-- PMM

Re: [PULL 00/29] Migration pull patches (second try)

Posted by Juan Quintela 29 weeks ago
Peter Maydell <peter.maydell@linaro.org> wrote:
> On Tue, 14 Jan 2020 at 09:26, Juan Quintela <quintela@redhat.com> wrote:
>>
>> The following changes since commit 3c8a6575985b1652b45bfa670b5e1907d642cfa0:
>>
>>   Merge remote-tracking branch
>> 'remotes/kraxel/tags/usb-20200113-pull-request' into staging
>> (2020-01-13 14:19:57 +0000)
>>
>> are available in the Git repository at:
>>
>>   https://github.com/juanquintela/qemu.git tags/migration-pull-pull-request
>>
>> for you to fetch changes up to 4eafab585c091050b5ae63130f46fe46ac919c3a:
>>
>>   migration: Support QLIST migration (2020-01-14 10:17:12 +0100)
>>
>> ----------------------------------------------------------------
>> Migration pull request
>> - updated QList patch
>> - initialize local msg variable
>>
>> ----------------------------------------------------------------
>
> Still fails on hosts where ram_addr_t is 32 bits.
> Looks like you still haven't got the fix for the problem
> Dan pointed out where multifd_save_setup() is using the
> wrong type when it calculates p->packetlen to allocate the
> data structure.

Ouch.  Then there is "yet" another different problem. I fixed the one
that daniel pointed.

Working there.


Re: [PULL 00/29] Migration pull patches (second try)

Posted by Daniel P. Berrangé 29 weeks ago
On Tue, Jan 14, 2020 at 12:22:10PM +0100, Juan Quintela wrote:
> Peter Maydell <peter.maydell@linaro.org> wrote:
> > On Tue, 14 Jan 2020 at 09:26, Juan Quintela <quintela@redhat.com> wrote:
> >>
> >> The following changes since commit 3c8a6575985b1652b45bfa670b5e1907d642cfa0:
> >>
> >>   Merge remote-tracking branch
> >> 'remotes/kraxel/tags/usb-20200113-pull-request' into staging
> >> (2020-01-13 14:19:57 +0000)
> >>
> >> are available in the Git repository at:
> >>
> >>   https://github.com/juanquintela/qemu.git tags/migration-pull-pull-request
> >>
> >> for you to fetch changes up to 4eafab585c091050b5ae63130f46fe46ac919c3a:
> >>
> >>   migration: Support QLIST migration (2020-01-14 10:17:12 +0100)
> >>
> >> ----------------------------------------------------------------
> >> Migration pull request
> >> - updated QList patch
> >> - initialize local msg variable
> >>
> >> ----------------------------------------------------------------
> >
> > Still fails on hosts where ram_addr_t is 32 bits.
> > Looks like you still haven't got the fix for the problem
> > Dan pointed out where multifd_save_setup() is using the
> > wrong type when it calculates p->packetlen to allocate the
> > data structure.
> 
> Ouch.  Then there is "yet" another different problem. I fixed the one
> that daniel pointed.

Err, no you only fixed one of the two bugs I pointed out (for a second
time)

Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|


Re: [PULL 00/29] Migration pull patches (second try)

Posted by Peter Maydell 29 weeks ago
On Tue, 14 Jan 2020 at 11:22, Juan Quintela <quintela@redhat.com> wrote:
>
> Peter Maydell <peter.maydell@linaro.org> wrote:
> > Still fails on hosts where ram_addr_t is 32 bits.
> > Looks like you still haven't got the fix for the problem
> > Dan pointed out where multifd_save_setup() is using the
> > wrong type when it calculates p->packetlen to allocate the
> > data structure.
>
> Ouch.  Then there is "yet" another different problem. I fixed the one
> that daniel pointed.

Daniel pointed out both bugs back in July, in the same email,
and again this time around...

thanks
-- PMM

Re: [PULL 00/29] Migration pull patches (second try)

Posted by Peter Maydell 29 weeks ago
On Tue, 14 Jan 2020 at 09:26, Juan Quintela <quintela@redhat.com> wrote:
>
> The following changes since commit 3c8a6575985b1652b45bfa670b5e1907d642cfa0:
>
>   Merge remote-tracking branch 'remotes/kraxel/tags/usb-20200113-pull-request' into staging (2020-01-13 14:19:57 +0000)
>
> are available in the Git repository at:
>
>   https://github.com/juanquintela/qemu.git tags/migration-pull-pull-request
>
> for you to fetch changes up to 4eafab585c091050b5ae63130f46fe46ac919c3a:
>
>   migration: Support QLIST migration (2020-01-14 10:17:12 +0100)
>
> ----------------------------------------------------------------
> Migration pull request
> - updated QList patch
> - initialize local msg variable

Also, your patch 1 has no Signed-off-by: line. Ideally you
should post it on the list before putting it in a pullreq...

thanks
-- PMM