[v1] migration queue

[Qemu-devel] [PULL 00/21] migration queue

Posted by Dr. David Alan Gilbert (git) 5 years, 1 month ago

From: "Dr. David Alan Gilbert" <dgilbert@redhat.com>

The following changes since commit 0984a157c1c053394adbf64ed7de97f1aebe6a2d:

  Merge remote-tracking branch 'remotes/jasowang/tags/net-pull-request' into staging (2019-03-05 09:33:20 +0000)

are available in the Git repository at:

  git://github.com/dagrh/qemu.git tags/pull-migration-20190305a

for you to fetch changes up to 3feabbb5fc5e3d5504c735a77ebd1d7fc1eabb9b:

  qapi/migration.json: Remove a variable that doesn't exist in example (2019-03-05 18:04:05 +0000)

----------------------------------------------------------------
Migration pull 2019-03-05

   a) 3 small fixes including the cancel problem
     that caused the ahci migration test to fail
     intermittently
   b) Yury's ignore-shared feature
   c) Juan's extra tests
   d) Wei Wang's free page hinting
   e) Some Colo fixes from Zhang Chen

----------------------------------------------------------------
Dr. David Alan Gilbert (2):
      migration: Fix cancel state
      migration/rdma: Fix qemu_rdma_cleanup null check

Juan Quintela (3):
      tests: Add migration xbzrle test
      migration: Create socket-address parameter
      tests: Add basic migration precopy tcp test

Marcel Apfelbaum (1):
      migration/rdma: clang compilation fix

Wei Wang (7):
      bitmap: fix bitmap_count_one
      bitmap: bitmap_count_one_with_offset
      migration: use bitmap_mutex in migration_bitmap_clear_dirty
      migration: API to clear bits of guest free pages from the dirty bitmap
      migration/ram.c: add a notifier chain for precopy
      migration/ram.c: add the free page optimization enable flag
      virtio-balloon: VIRTIO_BALLOON_F_FREE_PAGE_HINT

Yury Kotov (5):
      exec: Change RAMBlockIterFunc definition
      migration: Introduce ignore-shared capability
      migration: Add an ability to ignore shared RAM blocks
      tests/migration-test: Add a test for ignore-shared capability
      migration: Add capabilities validation

Zhang Chen (3):
      Migration/colo.c: Fix double close bug when occur COLO failover
      Migration/colo.c: Make COLO node running after failover
      qapi/migration.json: Remove a variable that doesn't exist in example

 exec.c                             |  38 ++---
 hmp.c                              |  33 ++++
 hw/virtio/virtio-balloon.c         | 263 +++++++++++++++++++++++++++++++
 include/exec/cpu-common.h          |   7 +-
 include/hw/virtio/virtio-balloon.h |  28 +++-
 include/migration/misc.h           |  22 +++
 include/qemu/bitmap.h              |  17 ++
 migration/colo.c                   |   2 +-
 migration/migration.c              |  52 ++++++-
 migration/migration.h              |   9 +-
 migration/postcopy-ram.c           |  48 +++---
 migration/ram.c                    | 231 ++++++++++++++++++++++++----
 migration/rdma.c                   |  18 ++-
 migration/savevm.c                 | 152 ++++++++++++++++++
 migration/socket.c                 |  11 ++
 qapi/migration.json                |  13 +-
 qapi/sockets.json                  |  18 +++
 stubs/ram-block.c                  |  15 ++
 tests/migration-test.c             | 308 +++++++++++++++++++++++++++++++++----
 util/vfio-helpers.c                |   6 +-
 vl.c                               |   1 +
 21 files changed, 1169 insertions(+), 123 deletions(-)

Re: [Qemu-devel] [PULL 00/21] migration queue

Posted by Eric Blake 5 years, 1 month ago

On 3/5/19 12:15 PM, Dr. David Alan Gilbert (git) wrote:
> From: "Dr. David Alan Gilbert" <dgilbert@redhat.com>
> 
> The following changes since commit 0984a157c1c053394adbf64ed7de97f1aebe6a2d:
> 
>    Merge remote-tracking branch 'remotes/jasowang/tags/net-pull-request' into staging (2019-03-05 09:33:20 +0000)
> 
> are available in the Git repository at:
> 
>    git://github.com/dagrh/qemu.git tags/pull-migration-20190305a
> 
> for you to fetch changes up to 3feabbb5fc5e3d5504c735a77ebd1d7fc1eabb9b:
> 
>    qapi/migration.json: Remove a variable that doesn't exist in example (2019-03-05 18:04:05 +0000)
> 

> Juan Quintela (3):
>        tests: Add migration xbzrle test
>        migration: Create socket-address parameter

I raised some comments on this one (10/21). I'll leave it up to others 
to decide if those warrant a v2 pull request or merely a followup patch.

-- 
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3226
Virtualization:  qemu.org | libvirt.org

Re: [Qemu-devel] [PULL 00/21] migration queue

Posted by no-reply@patchew.org 5 years, 1 month ago

Patchew URL: https://patchew.org/QEMU/20190305181602.9051-1-dgilbert@redhat.com/



Hi,

This series seems to have some coding style problems. See output below for
more information:

Type: series
Message-id: 20190305181602.9051-1-dgilbert@redhat.com
Subject: [Qemu-devel] [PULL 00/21] migration queue

=== TEST SCRIPT BEGIN ===
#!/bin/bash
git rev-parse base > /dev/null || exit 0
git config --local diff.renamelimit 0
git config --local diff.renames True
git config --local diff.algorithm histogram
./scripts/checkpatch.pl --mailback base..
=== TEST SCRIPT END ===

Updating 3c8cf5a9c21ff8782164d1def7f44bd888713384
From https://github.com/patchew-project/qemu
 * [new tag]               patchew/20190305181602.9051-1-dgilbert@redhat.com -> patchew/20190305181602.9051-1-dgilbert@redhat.com
Switched to a new branch 'test'
ae493dd345 qapi/migration.json: Remove a variable that doesn't exist in example
dae452acf7 Migration/colo.c: Make COLO node running after failover
8d7745295b Migration/colo.c: Fix double close bug when occur COLO failover
008b59edc7 virtio-balloon: VIRTIO_BALLOON_F_FREE_PAGE_HINT
14f87199a1 migration/ram.c: add the free page optimization enable flag
ca2699e456 migration/ram.c: add a notifier chain for precopy
606c6ef429 migration: API to clear bits of guest free pages from the dirty bitmap
804e796327 migration: use bitmap_mutex in migration_bitmap_clear_dirty
060dfa78ac bitmap: bitmap_count_one_with_offset
0cff623ce1 bitmap: fix bitmap_count_one
ca9b2b6cdd tests: Add basic migration precopy tcp test
0c3dd1d54e migration: Create socket-address parameter
719012c3c7 tests: Add migration xbzrle test
74c0f9fd44 migration: Add capabilities validation
8c6537da13 tests/migration-test: Add a test for ignore-shared capability
2ee34efd18 migration: Add an ability to ignore shared RAM blocks
190f582e4e migration: Introduce ignore-shared capability
02a8200adf exec: Change RAMBlockIterFunc definition
1990ba02eb migration/rdma: clang compilation fix
7a26e3977e migration/rdma: Fix qemu_rdma_cleanup null check
106518cdad migration: Fix cancel state

=== OUTPUT BEGIN ===
1/21 Checking commit 106518cdad41 (migration: Fix cancel state)
2/21 Checking commit 7a26e3977ebc (migration/rdma: Fix qemu_rdma_cleanup null check)
3/21 Checking commit 1990ba02eb5d (migration/rdma: clang compilation fix)
4/21 Checking commit 02a8200adf94 (exec: Change RAMBlockIterFunc definition)
5/21 Checking commit 190f582e4e8c (migration: Introduce ignore-shared capability)
6/21 Checking commit 2ee34efd18ee (migration: Add an ability to ignore shared RAM blocks)
ERROR: Macros with multiple statements should be enclosed in a do - while loop
#149: FILE: migration/ram.c:169:
+#define RAMBLOCK_FOREACH_NOT_IGNORED(block)            \
+    INTERNAL_RAMBLOCK_FOREACH(block)                   \
+        if (ramblock_is_ignored(block)) {} else

ERROR: trailing statements should be on next line
#151: FILE: migration/ram.c:171:
+        if (ramblock_is_ignored(block)) {} else

total: 2 errors, 0 warnings, 386 lines checked

Patch 6/21 has style problems, please review.  If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.

7/21 Checking commit 8c6537da13a8 (tests/migration-test: Add a test for ignore-shared capability)
ERROR: if this code is redundant consider removing it
#235: FILE: tests/migration-test.c:829:
+#if 0

total: 1 errors, 0 warnings, 255 lines checked

Patch 7/21 has style problems, please review.  If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.

8/21 Checking commit 74c0f9fd446e (migration: Add capabilities validation)
9/21 Checking commit 719012c3c7e2 (tests: Add migration xbzrle test)
10/21 Checking commit 0c3dd1d54ea5 (migration: Create socket-address parameter)
11/21 Checking commit ca9b2b6cdd76 (tests: Add basic migration precopy tcp test)
12/21 Checking commit 0cff623ce1ca (bitmap: fix bitmap_count_one)
13/21 Checking commit 060dfa78acbd (bitmap: bitmap_count_one_with_offset)
14/21 Checking commit 804e79632706 (migration: use bitmap_mutex in migration_bitmap_clear_dirty)
15/21 Checking commit 606c6ef42982 (migration: API to clear bits of guest free pages from the dirty bitmap)
16/21 Checking commit ca2699e45688 (migration/ram.c: add a notifier chain for precopy)
17/21 Checking commit 14f87199a142 (migration/ram.c: add the free page optimization enable flag)
18/21 Checking commit 008b59edc7e4 (virtio-balloon: VIRTIO_BALLOON_F_FREE_PAGE_HINT)
19/21 Checking commit 8d7745295b47 (Migration/colo.c: Fix double close bug when occur COLO failover)
20/21 Checking commit dae452acf7ef (Migration/colo.c: Make COLO node running after failover)
21/21 Checking commit ae493dd3459d (qapi/migration.json: Remove a variable that doesn't exist in example)
=== OUTPUT END ===

Test command exited with code: 1


The full log is available at
http://patchew.org/logs/20190305181602.9051-1-dgilbert@redhat.com/testing.checkpatch/?type=message.
---
Email generated automatically by Patchew [http://patchew.org/].
Please send your feedback to patchew-devel@redhat.com

Re: [Qemu-devel] [PULL 00/21] migration queue

Posted by Philippe Mathieu-Daudé 5 years, 1 month ago

On 3/5/19 7:15 PM, Dr. David Alan Gilbert (git) wrote:
> From: "Dr. David Alan Gilbert" <dgilbert@redhat.com>
> 
> The following changes since commit 0984a157c1c053394adbf64ed7de97f1aebe6a2d:
> 
>   Merge remote-tracking branch 'remotes/jasowang/tags/net-pull-request' into staging (2019-03-05 09:33:20 +0000)
> 
> are available in the Git repository at:
> 
>   git://github.com/dagrh/qemu.git tags/pull-migration-20190305a
> 
> for you to fetch changes up to 3feabbb5fc5e3d5504c735a77ebd1d7fc1eabb9b:
> 
>   qapi/migration.json: Remove a variable that doesn't exist in example (2019-03-05 18:04:05 +0000)
> 
> ----------------------------------------------------------------
> Migration pull 2019-03-05
> 
>    a) 3 small fixes including the cancel problem
>      that caused the ahci migration test to fail
>      intermittently
>    b) Yury's ignore-shared feature
>    c) Juan's extra tests
>    d) Wei Wang's free page hinting
>    e) Some Colo fixes from Zhang Chen
> 
> ----------------------------------------------------------------
> Dr. David Alan Gilbert (2):
>       migration: Fix cancel state
>       migration/rdma: Fix qemu_rdma_cleanup null check
> 
> Juan Quintela (3):
>       tests: Add migration xbzrle test
>       migration: Create socket-address parameter

Replying to cover for Peter scripts:

In patch 10/21 Daniel said he'll respin.

>       tests: Add basic migration precopy tcp test
> 
> Marcel Apfelbaum (1):
>       migration/rdma: clang compilation fix
> 
> Wei Wang (7):
>       bitmap: fix bitmap_count_one
>       bitmap: bitmap_count_one_with_offset
>       migration: use bitmap_mutex in migration_bitmap_clear_dirty
>       migration: API to clear bits of guest free pages from the dirty bitmap
>       migration/ram.c: add a notifier chain for precopy
>       migration/ram.c: add the free page optimization enable flag
>       virtio-balloon: VIRTIO_BALLOON_F_FREE_PAGE_HINT
> 
> Yury Kotov (5):
>       exec: Change RAMBlockIterFunc definition
>       migration: Introduce ignore-shared capability
>       migration: Add an ability to ignore shared RAM blocks
>       tests/migration-test: Add a test for ignore-shared capability
>       migration: Add capabilities validation
> 
> Zhang Chen (3):
>       Migration/colo.c: Fix double close bug when occur COLO failover
>       Migration/colo.c: Make COLO node running after failover
>       qapi/migration.json: Remove a variable that doesn't exist in example
> 
>  exec.c                             |  38 ++---
>  hmp.c                              |  33 ++++
>  hw/virtio/virtio-balloon.c         | 263 +++++++++++++++++++++++++++++++
>  include/exec/cpu-common.h          |   7 +-
>  include/hw/virtio/virtio-balloon.h |  28 +++-
>  include/migration/misc.h           |  22 +++
>  include/qemu/bitmap.h              |  17 ++
>  migration/colo.c                   |   2 +-
>  migration/migration.c              |  52 ++++++-
>  migration/migration.h              |   9 +-
>  migration/postcopy-ram.c           |  48 +++---
>  migration/ram.c                    | 231 ++++++++++++++++++++++++----
>  migration/rdma.c                   |  18 ++-
>  migration/savevm.c                 | 152 ++++++++++++++++++
>  migration/socket.c                 |  11 ++
>  qapi/migration.json                |  13 +-
>  qapi/sockets.json                  |  18 +++
>  stubs/ram-block.c                  |  15 ++
>  tests/migration-test.c             | 308 +++++++++++++++++++++++++++++++++----
>  util/vfio-helpers.c                |   6 +-
>  vl.c                               |   1 +
>  21 files changed, 1169 insertions(+), 123 deletions(-)
>

Re: [Qemu-devel] [PULL 00/21] migration queue

Posted by Markus Armbruster 5 years, 1 month ago

Philippe Mathieu-Daudé <philmd@redhat.com> writes:

> On 3/5/19 7:15 PM, Dr. David Alan Gilbert (git) wrote:
>> From: "Dr. David Alan Gilbert" <dgilbert@redhat.com>
>> 
>> The following changes since commit 0984a157c1c053394adbf64ed7de97f1aebe6a2d:
>> 
>>   Merge remote-tracking branch 'remotes/jasowang/tags/net-pull-request' into staging (2019-03-05 09:33:20 +0000)
>> 
>> are available in the Git repository at:
>> 
>>   git://github.com/dagrh/qemu.git tags/pull-migration-20190305a
>> 
>> for you to fetch changes up to 3feabbb5fc5e3d5504c735a77ebd1d7fc1eabb9b:
>> 
>>   qapi/migration.json: Remove a variable that doesn't exist in example (2019-03-05 18:04:05 +0000)
>> 
>> ----------------------------------------------------------------
>> Migration pull 2019-03-05
>> 
>>    a) 3 small fixes including the cancel problem
>>      that caused the ahci migration test to fail
>>      intermittently
>>    b) Yury's ignore-shared feature
>>    c) Juan's extra tests
>>    d) Wei Wang's free page hinting
>>    e) Some Colo fixes from Zhang Chen
>> 
>> ----------------------------------------------------------------
>> Dr. David Alan Gilbert (2):
>>       migration: Fix cancel state
>>       migration/rdma: Fix qemu_rdma_cleanup null check
>> 
>> Juan Quintela (3):
>>       tests: Add migration xbzrle test
>>       migration: Create socket-address parameter
>
> Replying to cover for Peter scripts:
>
> In patch 10/21 Daniel said he'll respin.

Just to make sure this sticks:

NAK
NACK
NAcked-by: Markus Armbruster <armbru@redhat.com>

;)

Re: [Qemu-devel] [PULL 00/21] migration queue

Posted by no-reply@patchew.org 5 years, 1 month ago

Patchew URL: https://patchew.org/QEMU/20190305181602.9051-1-dgilbert@redhat.com/



Hi,

This series seems to have some coding style problems. See output below for
more information:

Type: series
Message-id: 20190305181602.9051-1-dgilbert@redhat.com
Subject: [Qemu-devel] [PULL 00/21] migration queue

=== TEST SCRIPT BEGIN ===
#!/bin/bash
git rev-parse base > /dev/null || exit 0
git config --local diff.renamelimit 0
git config --local diff.renames True
git config --local diff.algorithm histogram
./scripts/checkpatch.pl --mailback base..
=== TEST SCRIPT END ===

Updating 3c8cf5a9c21ff8782164d1def7f44bd888713384
From https://github.com/patchew-project/qemu
 t [tag update]            patchew/20190305181602.9051-1-dgilbert@redhat.com -> patchew/20190305181602.9051-1-dgilbert@redhat.com
Switched to a new branch 'test'
ed12bcbe36 qapi/migration.json: Remove a variable that doesn't exist in example
f7ebfda83a Migration/colo.c: Make COLO node running after failover
32db19110d Migration/colo.c: Fix double close bug when occur COLO failover
4ede2cb898 virtio-balloon: VIRTIO_BALLOON_F_FREE_PAGE_HINT
e2179bfdaf migration/ram.c: add the free page optimization enable flag
0b2dcc781d migration/ram.c: add a notifier chain for precopy
a4872ec0c9 migration: API to clear bits of guest free pages from the dirty bitmap
4d16332d3d migration: use bitmap_mutex in migration_bitmap_clear_dirty
3d43e07311 bitmap: bitmap_count_one_with_offset
81ebde4066 bitmap: fix bitmap_count_one
4805e66bc6 tests: Add basic migration precopy tcp test
c2c09c0a2e migration: Create socket-address parameter
791156d5b0 tests: Add migration xbzrle test
1a91c1dc80 migration: Add capabilities validation
19953886de tests/migration-test: Add a test for ignore-shared capability
8d13ec1c7e migration: Add an ability to ignore shared RAM blocks
5a6f22f321 migration: Introduce ignore-shared capability
6e514e3671 exec: Change RAMBlockIterFunc definition
0b5d5ed075 migration/rdma: clang compilation fix
2f25212829 migration/rdma: Fix qemu_rdma_cleanup null check
8c62cfd9be migration: Fix cancel state

=== OUTPUT BEGIN ===
1/21 Checking commit 8c62cfd9beaf (migration: Fix cancel state)
2/21 Checking commit 2f252128292b (migration/rdma: Fix qemu_rdma_cleanup null check)
3/21 Checking commit 0b5d5ed075d6 (migration/rdma: clang compilation fix)
4/21 Checking commit 6e514e3671d0 (exec: Change RAMBlockIterFunc definition)
5/21 Checking commit 5a6f22f32115 (migration: Introduce ignore-shared capability)
6/21 Checking commit 8d13ec1c7e0c (migration: Add an ability to ignore shared RAM blocks)
ERROR: Macros with multiple statements should be enclosed in a do - while loop
#149: FILE: migration/ram.c:169:
+#define RAMBLOCK_FOREACH_NOT_IGNORED(block)            \
+    INTERNAL_RAMBLOCK_FOREACH(block)                   \
+        if (ramblock_is_ignored(block)) {} else

ERROR: trailing statements should be on next line
#151: FILE: migration/ram.c:171:
+        if (ramblock_is_ignored(block)) {} else

total: 2 errors, 0 warnings, 386 lines checked

Patch 6/21 has style problems, please review.  If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.

7/21 Checking commit 19953886de14 (tests/migration-test: Add a test for ignore-shared capability)
ERROR: if this code is redundant consider removing it
#235: FILE: tests/migration-test.c:829:
+#if 0

total: 1 errors, 0 warnings, 255 lines checked

Patch 7/21 has style problems, please review.  If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.

8/21 Checking commit 1a91c1dc805c (migration: Add capabilities validation)
9/21 Checking commit 791156d5b020 (tests: Add migration xbzrle test)
10/21 Checking commit c2c09c0a2e2c (migration: Create socket-address parameter)
11/21 Checking commit 4805e66bc6b4 (tests: Add basic migration precopy tcp test)
12/21 Checking commit 81ebde4066f7 (bitmap: fix bitmap_count_one)
13/21 Checking commit 3d43e07311cc (bitmap: bitmap_count_one_with_offset)
14/21 Checking commit 4d16332d3d3c (migration: use bitmap_mutex in migration_bitmap_clear_dirty)
15/21 Checking commit a4872ec0c961 (migration: API to clear bits of guest free pages from the dirty bitmap)
16/21 Checking commit 0b2dcc781dd5 (migration/ram.c: add a notifier chain for precopy)
17/21 Checking commit e2179bfdafa8 (migration/ram.c: add the free page optimization enable flag)
18/21 Checking commit 4ede2cb898e0 (virtio-balloon: VIRTIO_BALLOON_F_FREE_PAGE_HINT)
19/21 Checking commit 32db19110de5 (Migration/colo.c: Fix double close bug when occur COLO failover)
20/21 Checking commit f7ebfda83a46 (Migration/colo.c: Make COLO node running after failover)
21/21 Checking commit ed12bcbe36cf (qapi/migration.json: Remove a variable that doesn't exist in example)
=== OUTPUT END ===

Test command exited with code: 1


The full log is available at
http://patchew.org/logs/20190305181602.9051-1-dgilbert@redhat.com/testing.checkpatch/?type=message.
---
Email generated automatically by Patchew [http://patchew.org/].
Please send your feedback to patchew-devel@redhat.com